2024-09-12T01:42:12.030 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-12T01:42:12.056 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-12T01:42:12.165 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901574 branch: squid description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-quiesce/with-quiesce 6-workunit/fs/test_o_trunc}} email: ceph-qa@ceph.com first_in_suite: false job_id: '7901574' kernel: client: sha1: distro kdb: 1 sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: cephfs: max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 mon warn on pool no app: false osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - slow metadata IO - SLOW_OPS - slow request sha1: b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc subvols: create: 2 subvol_options: --namespace-isolated ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt dry_run: true install: ceph: flavor: default sha1: b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=secure - wsync syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: 086c737fa75c22d521a3932c59b31c5e1d1c9ef4 owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 481 sha1: b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc sleep_before_teardown: 0 subset: 57/512 suite: fs suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_086c737fa75c22d521a3932c59b31c5e1d1c9ef4/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: 086c737fa75c22d521a3932c59b31c5e1d1c9ef4 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi016.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIMVFIA+D3y15bDms2nhg4+uQa5zxug9vly+aViZbDMN08Tth36FLpYtiFleUGyJ9CLOPfCq8obk9qQ2PfKzdgA= smithi033.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAvLyFALqUB5PWCAkJ+nZagoUF0TzD8Y3KhWTNK1J760piF1X8RyaaOt8t7pHQgE3J8MxoUhznsCJctmds6nkj8= smithi184.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBITqd+BFNYQvPcT91aX8dnGy0PCnWxt5MTpdjmrsNNIu2i3VMnE8XZDXgr/137DzYvgo2I/tfuNQJPlsvtr5XKU= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate true - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - quiescer: null - workunit: clients: all: - fs/test_o_trunc.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-11_21:24:17 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-12T01:42:12.166 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_086c737fa75c22d521a3932c59b31c5e1d1c9ef4/qa; will attempt to use it 2024-09-12T01:42:12.167 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_086c737fa75c22d521a3932c59b31c5e1d1c9ef4/qa/tasks 2024-09-12T01:42:12.167 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-12T01:42:12.168 INFO:teuthology.task.internal:Checking packages... 2024-09-12T01:42:12.189 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc' 2024-09-12T01:42:12.189 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-12T01:42:12.190 INFO:teuthology.packaging:ref: None 2024-09-12T01:42:12.190 INFO:teuthology.packaging:tag: None 2024-09-12T01:42:12.190 INFO:teuthology.packaging:branch: squid 2024-09-12T01:42:12.190 INFO:teuthology.packaging:sha1: b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T01:42:12.190 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-09-12T01:42:12.417 INFO:teuthology.task.internal:Found packages for ceph version 19.1.1-314.g086c737f 2024-09-12T01:42:12.419 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-12T01:42:12.426 INFO:teuthology.task.internal:no buildpackages task found 2024-09-12T01:42:12.426 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-12T01:42:12.591 INFO:teuthology.task.internal:Saving configuration 2024-09-12T01:42:12.612 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-12T01:42:12.653 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-12T01:42:12.684 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi016.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901574', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-12 01:37:21.781067', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIMVFIA+D3y15bDms2nhg4+uQa5zxug9vly+aViZbDMN08Tth36FLpYtiFleUGyJ9CLOPfCq8obk9qQ2PfKzdgA='} 2024-09-12T01:42:12.715 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi033.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901574', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-12 01:37:21.783059', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAvLyFALqUB5PWCAkJ+nZagoUF0TzD8Y3KhWTNK1J760piF1X8RyaaOt8t7pHQgE3J8MxoUhznsCJctmds6nkj8='} 2024-09-12T01:42:12.748 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi184.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901574', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-12 01:37:21.784747', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBITqd+BFNYQvPcT91aX8dnGy0PCnWxt5MTpdjmrsNNIu2i3VMnE8XZDXgr/137DzYvgo2I/tfuNQJPlsvtr5XKU='} 2024-09-12T01:42:12.748 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-12T01:42:12.755 INFO:teuthology.task.internal:roles: ubuntu@smithi016.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-09-12T01:42:12.755 INFO:teuthology.task.internal:roles: ubuntu@smithi033.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-09-12T01:42:12.755 INFO:teuthology.task.internal:roles: ubuntu@smithi184.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-09-12T01:42:12.755 INFO:teuthology.run_tasks:Running task console_log... 2024-09-12T01:42:13.017 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7feb0e88f490>, signals=[15]) 2024-09-12T01:42:13.017 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-12T01:42:13.023 INFO:teuthology.task.internal:Opening connections... 2024-09-12T01:42:13.023 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi016.front.sepia.ceph.com 2024-09-12T01:42:13.024 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T01:42:13.109 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi033.front.sepia.ceph.com 2024-09-12T01:42:13.110 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi033.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T01:42:13.192 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi184.front.sepia.ceph.com 2024-09-12T01:42:13.193 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi184.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T01:42:13.281 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-12T01:42:13.288 DEBUG:teuthology.orchestra.run.smithi016:> uname -m 2024-09-12T01:42:13.306 INFO:teuthology.orchestra.run.smithi016.stdout:x86_64 2024-09-12T01:42:13.306 DEBUG:teuthology.orchestra.run.smithi016:> cat /etc/os-release 2024-09-12T01:42:13.364 INFO:teuthology.orchestra.run.smithi016.stdout:NAME="CentOS Stream" 2024-09-12T01:42:13.364 INFO:teuthology.orchestra.run.smithi016.stdout:VERSION="9" 2024-09-12T01:42:13.364 INFO:teuthology.orchestra.run.smithi016.stdout:ID="centos" 2024-09-12T01:42:13.364 INFO:teuthology.orchestra.run.smithi016.stdout:ID_LIKE="rhel fedora" 2024-09-12T01:42:13.365 INFO:teuthology.orchestra.run.smithi016.stdout:VERSION_ID="9" 2024-09-12T01:42:13.365 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM_ID="platform:el9" 2024-09-12T01:42:13.365 INFO:teuthology.orchestra.run.smithi016.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-12T01:42:13.365 INFO:teuthology.orchestra.run.smithi016.stdout:ANSI_COLOR="0;31" 2024-09-12T01:42:13.365 INFO:teuthology.orchestra.run.smithi016.stdout:LOGO="fedora-logo-icon" 2024-09-12T01:42:13.365 INFO:teuthology.orchestra.run.smithi016.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-12T01:42:13.365 INFO:teuthology.orchestra.run.smithi016.stdout:HOME_URL="https://centos.org/" 2024-09-12T01:42:13.365 INFO:teuthology.orchestra.run.smithi016.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-12T01:42:13.365 INFO:teuthology.orchestra.run.smithi016.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-12T01:42:13.365 INFO:teuthology.orchestra.run.smithi016.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-12T01:42:13.366 INFO:teuthology.lock.ops:Updating smithi016.front.sepia.ceph.com on lock server 2024-09-12T01:42:13.398 DEBUG:teuthology.orchestra.run.smithi033:> uname -m 2024-09-12T01:42:13.417 INFO:teuthology.orchestra.run.smithi033.stdout:x86_64 2024-09-12T01:42:13.417 DEBUG:teuthology.orchestra.run.smithi033:> cat /etc/os-release 2024-09-12T01:42:13.475 INFO:teuthology.orchestra.run.smithi033.stdout:NAME="CentOS Stream" 2024-09-12T01:42:13.475 INFO:teuthology.orchestra.run.smithi033.stdout:VERSION="9" 2024-09-12T01:42:13.475 INFO:teuthology.orchestra.run.smithi033.stdout:ID="centos" 2024-09-12T01:42:13.476 INFO:teuthology.orchestra.run.smithi033.stdout:ID_LIKE="rhel fedora" 2024-09-12T01:42:13.476 INFO:teuthology.orchestra.run.smithi033.stdout:VERSION_ID="9" 2024-09-12T01:42:13.476 INFO:teuthology.orchestra.run.smithi033.stdout:PLATFORM_ID="platform:el9" 2024-09-12T01:42:13.476 INFO:teuthology.orchestra.run.smithi033.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-12T01:42:13.476 INFO:teuthology.orchestra.run.smithi033.stdout:ANSI_COLOR="0;31" 2024-09-12T01:42:13.476 INFO:teuthology.orchestra.run.smithi033.stdout:LOGO="fedora-logo-icon" 2024-09-12T01:42:13.476 INFO:teuthology.orchestra.run.smithi033.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-12T01:42:13.477 INFO:teuthology.orchestra.run.smithi033.stdout:HOME_URL="https://centos.org/" 2024-09-12T01:42:13.477 INFO:teuthology.orchestra.run.smithi033.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-12T01:42:13.477 INFO:teuthology.orchestra.run.smithi033.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-12T01:42:13.477 INFO:teuthology.orchestra.run.smithi033.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-12T01:42:13.477 INFO:teuthology.lock.ops:Updating smithi033.front.sepia.ceph.com on lock server 2024-09-12T01:42:13.507 DEBUG:teuthology.orchestra.run.smithi184:> uname -m 2024-09-12T01:42:13.526 INFO:teuthology.orchestra.run.smithi184.stdout:x86_64 2024-09-12T01:42:13.526 DEBUG:teuthology.orchestra.run.smithi184:> cat /etc/os-release 2024-09-12T01:42:13.585 INFO:teuthology.orchestra.run.smithi184.stdout:NAME="CentOS Stream" 2024-09-12T01:42:13.585 INFO:teuthology.orchestra.run.smithi184.stdout:VERSION="9" 2024-09-12T01:42:13.585 INFO:teuthology.orchestra.run.smithi184.stdout:ID="centos" 2024-09-12T01:42:13.586 INFO:teuthology.orchestra.run.smithi184.stdout:ID_LIKE="rhel fedora" 2024-09-12T01:42:13.586 INFO:teuthology.orchestra.run.smithi184.stdout:VERSION_ID="9" 2024-09-12T01:42:13.586 INFO:teuthology.orchestra.run.smithi184.stdout:PLATFORM_ID="platform:el9" 2024-09-12T01:42:13.586 INFO:teuthology.orchestra.run.smithi184.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-12T01:42:13.586 INFO:teuthology.orchestra.run.smithi184.stdout:ANSI_COLOR="0;31" 2024-09-12T01:42:13.586 INFO:teuthology.orchestra.run.smithi184.stdout:LOGO="fedora-logo-icon" 2024-09-12T01:42:13.586 INFO:teuthology.orchestra.run.smithi184.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-12T01:42:13.586 INFO:teuthology.orchestra.run.smithi184.stdout:HOME_URL="https://centos.org/" 2024-09-12T01:42:13.586 INFO:teuthology.orchestra.run.smithi184.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-12T01:42:13.586 INFO:teuthology.orchestra.run.smithi184.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-12T01:42:13.586 INFO:teuthology.orchestra.run.smithi184.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-12T01:42:13.587 INFO:teuthology.lock.ops:Updating smithi184.front.sepia.ceph.com on lock server 2024-09-12T01:42:13.615 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-12T01:42:13.626 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-12T01:42:13.845 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-12T01:42:13.845 DEBUG:teuthology.orchestra.run.smithi016:> test '!' -e /home/ubuntu/cephtest 2024-09-12T01:42:13.848 DEBUG:teuthology.orchestra.run.smithi033:> test '!' -e /home/ubuntu/cephtest 2024-09-12T01:42:13.850 DEBUG:teuthology.orchestra.run.smithi184:> test '!' -e /home/ubuntu/cephtest 2024-09-12T01:42:13.868 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-12T01:42:13.874 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-12T01:42:13.875 DEBUG:teuthology.orchestra.run.smithi016:> test -z $(ls -A /var/lib/ceph) 2024-09-12T01:42:13.905 DEBUG:teuthology.orchestra.run.smithi033:> test -z $(ls -A /var/lib/ceph) 2024-09-12T01:42:13.907 DEBUG:teuthology.orchestra.run.smithi184:> test -z $(ls -A /var/lib/ceph) 2024-09-12T01:42:13.930 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-12T01:42:14.181 INFO:teuthology.run_tasks:Running task kernel... 2024-09-12T01:42:14.197 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': 1, 'sha1': 'distro'} 2024-09-12T01:42:14.197 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-09-12T01:42:14.197 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-09-12T01:42:14.198 DEBUG:teuthology.orchestra.run.smithi016:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-12T01:42:14.214 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T01:42:14.215 DEBUG:teuthology.orchestra.run.smithi016:> uname -r 2024-09-12T01:42:14.272 INFO:teuthology.orchestra.run.smithi016.stdout:5.14.0-503.el9.x86_64 2024-09-12T01:42:14.273 INFO:teuthology.task.kernel:Running kernel on smithi016: 5.14.0-503.el9.x86_64 2024-09-12T01:42:14.273 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum install -y kernel 2024-09-12T01:42:16.911 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.2 MB 00:00 2024-09-12T01:42:18.862 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - AppStream 27 MB/s | 20 MB 00:00 2024-09-12T01:42:26.662 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - CRB 1.7 MB/s | 6.5 MB 00:03 2024-09-12T01:42:28.171 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - Extras packages 72 kB/s | 19 kB 00:00 2024-09-12T01:42:29.342 INFO:teuthology.orchestra.run.smithi016.stdout:Extra Packages for Enterprise Linux 22 MB/s | 23 MB 00:01 2024-09-12T01:42:35.015 INFO:teuthology.orchestra.run.smithi016.stdout:lab-extras 49 kB/s | 1.7 kB 00:00 2024-09-12T01:42:37.088 INFO:teuthology.orchestra.run.smithi016.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-12T01:42:37.088 INFO:teuthology.orchestra.run.smithi016.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-12T01:42:37.184 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2024-09-12T01:42:37.193 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2024-09-12T01:42:37.193 INFO:teuthology.orchestra.run.smithi016.stdout: Package Architecture Version Repository Size 2024-09-12T01:42:37.193 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2024-09-12T01:42:37.194 INFO:teuthology.orchestra.run.smithi016.stdout:Installing: 2024-09-12T01:42:37.194 INFO:teuthology.orchestra.run.smithi016.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-12T01:42:37.194 INFO:teuthology.orchestra.run.smithi016.stdout:Installing dependencies: 2024-09-12T01:42:37.194 INFO:teuthology.orchestra.run.smithi016.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-12T01:42:37.194 INFO:teuthology.orchestra.run.smithi016.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-12T01:42:37.194 INFO:teuthology.orchestra.run.smithi016.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-12T01:42:37.194 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:42:37.194 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction Summary 2024-09-12T01:42:37.194 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2024-09-12T01:42:37.194 INFO:teuthology.orchestra.run.smithi016.stdout:Install 4 Packages 2024-09-12T01:42:37.194 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:42:37.195 INFO:teuthology.orchestra.run.smithi016.stdout:Total download size: 79 M 2024-09-12T01:42:37.195 INFO:teuthology.orchestra.run.smithi016.stdout:Installed size: 126 M 2024-09-12T01:42:37.195 INFO:teuthology.orchestra.run.smithi016.stdout:Downloading Packages: 2024-09-12T01:42:37.333 INFO:teuthology.orchestra.run.smithi016.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 192 kB/s | 23 kB 00:00 2024-09-12T01:42:38.750 INFO:teuthology.orchestra.run.smithi016.stdout:(2/4): kernel-modules-core-5.14.0-505.el9.x86_6 20 MB/s | 29 MB 00:01 2024-09-12T01:42:39.150 INFO:teuthology.orchestra.run.smithi016.stdout:(3/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 18 MB/s | 35 MB 00:01 2024-09-12T01:42:39.309 INFO:teuthology.orchestra.run.smithi016.stdout:(4/4): kernel-core-5.14.0-505.el9.x86_64.rpm 7.5 MB/s | 16 MB 00:02 2024-09-12T01:42:39.309 INFO:teuthology.orchestra.run.smithi016.stdout:-------------------------------------------------------------------------------- 2024-09-12T01:42:39.309 INFO:teuthology.orchestra.run.smithi016.stdout:Total 37 MB/s | 79 MB 00:02 2024-09-12T01:42:40.021 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction check 2024-09-12T01:42:40.419 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction check succeeded. 2024-09-12T01:42:40.419 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction test 2024-09-12T01:42:41.382 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction test succeeded. 2024-09-12T01:42:41.382 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction 2024-09-12T01:42:43.964 INFO:teuthology.orchestra.run.smithi016.stdout: Preparing : 1/1 2024-09-12T01:42:44.661 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-12T01:42:44.742 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-12T01:42:47.537 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-12T01:42:47.663 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-12T01:42:52.790 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-12T01:42:52.839 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-12T01:42:57.974 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-12T01:43:36.933 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-12T01:43:36.974 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-12T01:43:37.325 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-12T01:43:37.325 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-12T01:43:37.325 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-12T01:43:37.326 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-12T01:43:38.170 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-12T01:43:38.170 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:43:38.171 INFO:teuthology.orchestra.run.smithi016.stdout:Installed: 2024-09-12T01:43:38.171 INFO:teuthology.orchestra.run.smithi016.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-12T01:43:38.171 INFO:teuthology.orchestra.run.smithi016.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-12T01:43:38.171 INFO:teuthology.orchestra.run.smithi016.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-12T01:43:38.171 INFO:teuthology.orchestra.run.smithi016.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-12T01:43:38.171 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:43:38.171 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2024-09-12T01:43:38.505 DEBUG:teuthology.orchestra.run.smithi016:> rpm -q kernel | sort -rV | head -n 1 2024-09-12T01:43:38.604 INFO:teuthology.orchestra.run.smithi016.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-12T01:43:38.605 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-12T01:43:38.605 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-12T01:43:38.605 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-12T01:43:38.605 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-12T01:43:38.605 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-09-12T01:43:38.605 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi016.front.sepia.ceph.com, path=None, version=distro) 2024-09-12T01:43:38.606 DEBUG:teuthology.orchestra.run.smithi016:> rpm -q kernel | sort -rV | head -n 1 2024-09-12T01:43:38.663 INFO:teuthology.orchestra.run.smithi016.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-12T01:43:38.664 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-12T01:43:38.664 DEBUG:teuthology.orchestra.run.smithi016:> sudo rpm -qi grub2-tools 2024-09-12T01:43:38.707 INFO:teuthology.orchestra.run.smithi016.stdout:Name : grub2-tools 2024-09-12T01:43:38.707 INFO:teuthology.orchestra.run.smithi016.stdout:Epoch : 1 2024-09-12T01:43:38.707 INFO:teuthology.orchestra.run.smithi016.stdout:Version : 2.06 2024-09-12T01:43:38.708 INFO:teuthology.orchestra.run.smithi016.stdout:Release : 82.el9 2024-09-12T01:43:38.708 INFO:teuthology.orchestra.run.smithi016.stdout:Architecture: x86_64 2024-09-12T01:43:38.708 INFO:teuthology.orchestra.run.smithi016.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-12T01:43:38.708 INFO:teuthology.orchestra.run.smithi016.stdout:Group : Unspecified 2024-09-12T01:43:38.708 INFO:teuthology.orchestra.run.smithi016.stdout:Size : 8274670 2024-09-12T01:43:38.708 INFO:teuthology.orchestra.run.smithi016.stdout:License : GPLv3+ 2024-09-12T01:43:38.708 INFO:teuthology.orchestra.run.smithi016.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-12T01:43:38.708 INFO:teuthology.orchestra.run.smithi016.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-12T01:43:38.708 INFO:teuthology.orchestra.run.smithi016.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-12T01:43:38.708 INFO:teuthology.orchestra.run.smithi016.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-12T01:43:38.709 INFO:teuthology.orchestra.run.smithi016.stdout:Packager : builder@centos.org 2024-09-12T01:43:38.709 INFO:teuthology.orchestra.run.smithi016.stdout:Vendor : CentOS 2024-09-12T01:43:38.709 INFO:teuthology.orchestra.run.smithi016.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-12T01:43:38.709 INFO:teuthology.orchestra.run.smithi016.stdout:Summary : Support tools for GRUB. 2024-09-12T01:43:38.709 INFO:teuthology.orchestra.run.smithi016.stdout:Description : 2024-09-12T01:43:38.709 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:43:38.709 INFO:teuthology.orchestra.run.smithi016.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-12T01:43:38.709 INFO:teuthology.orchestra.run.smithi016.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-12T01:43:38.709 INFO:teuthology.orchestra.run.smithi016.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-12T01:43:38.709 INFO:teuthology.orchestra.run.smithi016.stdout:hardware devices. 2024-09-12T01:43:38.709 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:43:38.709 INFO:teuthology.orchestra.run.smithi016.stdout:This subpackage provides tools for support of all platforms. 2024-09-12T01:43:38.710 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-12T01:43:38.710 INFO:teuthology.task.kernel:Updating grub on smithi016 to boot 5.14.0-505.el9.x86_64 2024-09-12T01:43:38.710 DEBUG:teuthology.orchestra.run.smithi016:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-12T01:43:39.380 INFO:teuthology.orchestra.run.smithi016.stderr:Generating grub configuration file ... 2024-09-12T01:43:40.727 INFO:teuthology.orchestra.run.smithi016.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-12T01:43:40.759 INFO:teuthology.orchestra.run.smithi016.stderr:done 2024-09-12T01:43:40.760 DEBUG:teuthology.orchestra.run.smithi016:> mktemp 2024-09-12T01:43:40.775 INFO:teuthology.orchestra.run.smithi016.stdout:/tmp/tmp.2lQMgxTDZI 2024-09-12T01:43:40.776 DEBUG:teuthology.orchestra.run.smithi016:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.2lQMgxTDZI 2024-09-12T01:43:40.841 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 0666 /tmp/tmp.2lQMgxTDZI 2024-09-12T01:43:40.982 DEBUG:teuthology.orchestra.remote:smithi016:/tmp/tmp.2lQMgxTDZI is 6KB 2024-09-12T01:43:41.031 DEBUG:teuthology.orchestra.run.smithi016:> rm -fr /tmp/tmp.2lQMgxTDZI 2024-09-12T01:43:41.046 DEBUG:teuthology.orchestra.run.smithi016:> sudo /bin/ls /boot/loader/entries || true 2024-09-12T01:43:41.108 INFO:teuthology.orchestra.run.smithi016.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-12T01:43:41.108 INFO:teuthology.orchestra.run.smithi016.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-12T01:43:41.108 INFO:teuthology.orchestra.run.smithi016.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-12T01:43:41.109 INFO:teuthology.orchestra.run.smithi016.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-12T01:43:41.110 DEBUG:teuthology.orchestra.run.smithi016:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-12T01:43:41.274 DEBUG:teuthology.orchestra.run.smithi016:> sudo shutdown -r now 2024-09-12T01:44:11.276 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-09-12T01:44:11.277 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi016.front.sepia.ceph.com' 2024-09-12T01:44:11.278 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T01:45:11.288 DEBUG:teuthology.orchestra.remote:timed out 2024-09-12T01:45:20.289 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi016.front.sepia.ceph.com' 2024-09-12T01:45:20.291 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T01:45:20.646 DEBUG:teuthology.orchestra.run.smithi016:> true 2024-09-12T01:45:21.025 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi016.front.sepia.ceph.com' 2024-09-12T01:45:21.025 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-505.el9.x86_64"... 2024-09-12T01:45:21.026 DEBUG:teuthology.orchestra.run.smithi016:> uname -r 2024-09-12T01:45:21.041 INFO:teuthology.orchestra.run.smithi016.stdout:5.14.0-505.el9.x86_64 2024-09-12T01:45:21.041 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-12T01:45:21.041 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-12T01:45:21.041 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-12T01:45:22.042 DEBUG:teuthology.parallel:result is None 2024-09-12T01:45:22.042 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-12T01:45:22.050 INFO:teuthology.task.internal:Creating test directory... 2024-09-12T01:45:22.050 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-12T01:45:22.052 DEBUG:teuthology.orchestra.run.smithi033:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-12T01:45:22.055 DEBUG:teuthology.orchestra.run.smithi184:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-12T01:45:22.073 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-12T01:45:22.211 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-12T01:45:22.259 INFO:teuthology.task.internal:Creating archive directory... 2024-09-12T01:45:22.259 DEBUG:teuthology.orchestra.run.smithi016:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-12T01:45:22.262 DEBUG:teuthology.orchestra.run.smithi033:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-12T01:45:22.264 DEBUG:teuthology.orchestra.run.smithi184:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-12T01:45:22.356 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-12T01:45:22.557 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-12T01:45:22.558 DEBUG:teuthology.orchestra.run.smithi033:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-12T01:45:22.575 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T01:45:22.575 DEBUG:teuthology.orchestra.run.smithi184:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-12T01:45:22.592 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T01:45:22.593 DEBUG:teuthology.orchestra.run.smithi016:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-12T01:45:22.596 DEBUG:teuthology.orchestra.run.smithi033:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-12T01:45:22.618 DEBUG:teuthology.orchestra.run.smithi184:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-12T01:45:22.657 INFO:teuthology.orchestra.run.smithi033.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-12T01:45:22.678 INFO:teuthology.orchestra.run.smithi033.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-12T01:45:22.681 INFO:teuthology.orchestra.run.smithi184.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-12T01:45:22.700 INFO:teuthology.orchestra.run.smithi184.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-12T01:45:22.749 INFO:teuthology.orchestra.run.smithi016.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-12T01:45:22.885 INFO:teuthology.orchestra.run.smithi016.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-12T01:45:22.887 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-12T01:45:22.895 INFO:teuthology.task.internal:Configuring sudo... 2024-09-12T01:45:22.895 DEBUG:teuthology.orchestra.run.smithi016:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-12T01:45:22.897 DEBUG:teuthology.orchestra.run.smithi033:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-12T01:45:22.900 DEBUG:teuthology.orchestra.run.smithi184:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-12T01:45:22.930 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-12T01:45:23.136 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-12T01:45:23.136 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-12T01:45:23.138 DEBUG:teuthology.orchestra.run.smithi033:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-12T01:45:23.141 DEBUG:teuthology.orchestra.run.smithi184:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-12T01:45:23.159 DEBUG:teuthology.orchestra.run.smithi016:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-12T01:45:23.242 DEBUG:teuthology.orchestra.run.smithi016:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-12T01:45:23.286 DEBUG:teuthology.orchestra.run.smithi016:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-12T01:45:23.376 DEBUG:teuthology.orchestra.run.smithi016:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-12T01:45:23.409 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-12T01:45:23.410 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-12T01:45:23.475 DEBUG:teuthology.orchestra.run.smithi033:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-12T01:45:23.528 DEBUG:teuthology.orchestra.run.smithi033:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-12T01:45:23.569 DEBUG:teuthology.orchestra.run.smithi033:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-12T01:45:23.660 DEBUG:teuthology.orchestra.run.smithi033:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-12T01:45:23.698 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-09-12T01:45:23.698 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-12T01:45:23.764 DEBUG:teuthology.orchestra.run.smithi184:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-12T01:45:23.816 DEBUG:teuthology.orchestra.run.smithi184:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-12T01:45:23.868 DEBUG:teuthology.orchestra.run.smithi184:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-12T01:45:23.963 DEBUG:teuthology.orchestra.run.smithi184:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-12T01:45:23.996 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-12T01:45:23.996 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-12T01:45:24.065 DEBUG:teuthology.orchestra.run.smithi016:> sudo service rsyslog restart 2024-09-12T01:45:24.068 DEBUG:teuthology.orchestra.run.smithi033:> sudo service rsyslog restart 2024-09-12T01:45:24.070 DEBUG:teuthology.orchestra.run.smithi184:> sudo service rsyslog restart 2024-09-12T01:45:24.108 INFO:teuthology.orchestra.run.smithi033.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-12T01:45:24.121 INFO:teuthology.orchestra.run.smithi016.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-12T01:45:24.145 INFO:teuthology.orchestra.run.smithi184.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-12T01:45:24.665 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-12T01:45:24.672 INFO:teuthology.task.internal:Starting timer... 2024-09-12T01:45:24.672 INFO:teuthology.run_tasks:Running task pcp... 2024-09-12T01:45:24.825 INFO:teuthology.run_tasks:Running task selinux... 2024-09-12T01:45:24.855 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-09-12T01:45:24.856 DEBUG:teuthology.orchestra.run.smithi016:> sudo service auditd rotate 2024-09-12T01:45:24.945 INFO:teuthology.orchestra.run.smithi016.stdout:Rotating logs: 2024-09-12T01:45:24.951 DEBUG:teuthology.orchestra.run.smithi033:> sudo service auditd rotate 2024-09-12T01:45:25.003 INFO:teuthology.orchestra.run.smithi033.stdout:Rotating logs: 2024-09-12T01:45:25.005 DEBUG:teuthology.orchestra.run.smithi184:> sudo service auditd rotate 2024-09-12T01:45:25.068 INFO:teuthology.orchestra.run.smithi184.stdout:Rotating logs: 2024-09-12T01:45:25.070 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-12T01:45:25.070 DEBUG:teuthology.orchestra.run.smithi016:> /usr/sbin/getenforce 2024-09-12T01:45:25.116 INFO:teuthology.orchestra.run.smithi016.stdout:Permissive 2024-09-12T01:45:25.116 DEBUG:teuthology.orchestra.run.smithi033:> /usr/sbin/getenforce 2024-09-12T01:45:25.140 INFO:teuthology.orchestra.run.smithi033.stdout:Permissive 2024-09-12T01:45:25.140 DEBUG:teuthology.orchestra.run.smithi184:> /usr/sbin/getenforce 2024-09-12T01:45:25.167 INFO:teuthology.orchestra.run.smithi184.stdout:Permissive 2024-09-12T01:45:25.167 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi016.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi033.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi184.front.sepia.ceph.com': 'permissive'} 2024-09-12T01:45:25.167 DEBUG:teuthology.orchestra.run.smithi016:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-12T01:45:25.262 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T01:45:25.262 DEBUG:teuthology.orchestra.run.smithi033:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-12T01:45:25.289 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T01:45:25.290 DEBUG:teuthology.orchestra.run.smithi184:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-12T01:45:25.319 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T01:45:25.319 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-12T01:45:25.319 DEBUG:teuthology.orchestra.run.smithi016:> sudo /usr/sbin/setenforce permissive 2024-09-12T01:45:25.594 DEBUG:teuthology.orchestra.run.smithi033:> sudo /usr/sbin/setenforce permissive 2024-09-12T01:45:25.636 DEBUG:teuthology.orchestra.run.smithi184:> sudo /usr/sbin/setenforce permissive 2024-09-12T01:45:25.671 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-12T01:45:25.682 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-12T01:45:25.694 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-12T01:45:25.695 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi016.front.sepia.ceph.com,smithi033.front.sepia.ceph.com,smithi184.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-12T01:49:46.449 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi016.front.sepia.ceph.com'), Remote(name='ubuntu@smithi033.front.sepia.ceph.com'), Remote(name='ubuntu@smithi184.front.sepia.ceph.com')] 2024-09-12T01:49:46.451 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi016.front.sepia.ceph.com' 2024-09-12T01:49:46.452 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T01:49:46.538 DEBUG:teuthology.orchestra.run.smithi016:> true 2024-09-12T01:49:46.621 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi016.front.sepia.ceph.com' 2024-09-12T01:49:46.622 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi033.front.sepia.ceph.com' 2024-09-12T01:49:46.622 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi033.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T01:49:46.703 DEBUG:teuthology.orchestra.run.smithi033:> true 2024-09-12T01:49:46.784 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi033.front.sepia.ceph.com' 2024-09-12T01:49:46.784 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi184.front.sepia.ceph.com' 2024-09-12T01:49:46.785 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi184.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T01:49:46.871 DEBUG:teuthology.orchestra.run.smithi184:> true 2024-09-12T01:49:46.953 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi184.front.sepia.ceph.com' 2024-09-12T01:49:46.953 INFO:teuthology.run_tasks:Running task clock... 2024-09-12T01:49:46.965 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-12T01:49:46.966 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-12T01:49:46.966 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-12T01:49:46.969 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-12T01:49:46.969 DEBUG:teuthology.orchestra.run.smithi033:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-12T01:49:46.971 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-12T01:49:46.971 DEBUG:teuthology.orchestra.run.smithi184:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-12T01:49:46.998 INFO:teuthology.orchestra.run.smithi033.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-12T01:49:47.002 INFO:teuthology.orchestra.run.smithi016.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-12T01:49:47.013 INFO:teuthology.orchestra.run.smithi033.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-12T01:49:47.019 INFO:teuthology.orchestra.run.smithi016.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-12T01:49:47.025 INFO:teuthology.orchestra.run.smithi184.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-12T01:49:47.037 INFO:teuthology.orchestra.run.smithi033.stderr:sudo: ntpd: command not found 2024-09-12T01:49:47.040 INFO:teuthology.orchestra.run.smithi184.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-12T01:49:47.047 INFO:teuthology.orchestra.run.smithi016.stderr:sudo: ntpd: command not found 2024-09-12T01:49:47.048 INFO:teuthology.orchestra.run.smithi033.stdout:506 Cannot talk to daemon 2024-09-12T01:49:47.059 INFO:teuthology.orchestra.run.smithi016.stdout:506 Cannot talk to daemon 2024-09-12T01:49:47.064 INFO:teuthology.orchestra.run.smithi033.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-12T01:49:47.065 INFO:teuthology.orchestra.run.smithi184.stderr:sudo: ntpd: command not found 2024-09-12T01:49:47.075 INFO:teuthology.orchestra.run.smithi016.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-12T01:49:47.077 INFO:teuthology.orchestra.run.smithi184.stdout:506 Cannot talk to daemon 2024-09-12T01:49:47.077 INFO:teuthology.orchestra.run.smithi033.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-12T01:49:47.090 INFO:teuthology.orchestra.run.smithi016.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-12T01:49:47.091 INFO:teuthology.orchestra.run.smithi184.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-12T01:49:47.105 INFO:teuthology.orchestra.run.smithi184.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-12T01:49:47.131 INFO:teuthology.orchestra.run.smithi033.stderr:bash: line 1: ntpq: command not found 2024-09-12T01:49:47.134 INFO:teuthology.orchestra.run.smithi033.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-12T01:49:47.134 INFO:teuthology.orchestra.run.smithi033.stdout:=============================================================================== 2024-09-12T01:49:47.134 INFO:teuthology.orchestra.run.smithi033.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T01:49:47.134 INFO:teuthology.orchestra.run.smithi033.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T01:49:47.134 INFO:teuthology.orchestra.run.smithi033.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T01:49:47.134 INFO:teuthology.orchestra.run.smithi033.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T01:49:47.151 INFO:teuthology.orchestra.run.smithi016.stderr:bash: line 1: ntpq: command not found 2024-09-12T01:49:47.155 INFO:teuthology.orchestra.run.smithi016.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-12T01:49:47.155 INFO:teuthology.orchestra.run.smithi016.stdout:=============================================================================== 2024-09-12T01:49:47.155 INFO:teuthology.orchestra.run.smithi016.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T01:49:47.155 INFO:teuthology.orchestra.run.smithi016.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T01:49:47.155 INFO:teuthology.orchestra.run.smithi016.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T01:49:47.156 INFO:teuthology.orchestra.run.smithi016.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T01:49:47.168 INFO:teuthology.orchestra.run.smithi184.stderr:bash: line 1: ntpq: command not found 2024-09-12T01:49:47.170 INFO:teuthology.orchestra.run.smithi184.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-12T01:49:47.170 INFO:teuthology.orchestra.run.smithi184.stdout:=============================================================================== 2024-09-12T01:49:47.170 INFO:teuthology.orchestra.run.smithi184.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T01:49:47.171 INFO:teuthology.orchestra.run.smithi184.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T01:49:47.171 INFO:teuthology.orchestra.run.smithi184.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T01:49:47.171 INFO:teuthology.orchestra.run.smithi184.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T01:49:47.172 INFO:teuthology.run_tasks:Running task install... 2024-09-12T01:49:47.180 DEBUG:teuthology.task.install:project ceph 2024-09-12T01:49:47.180 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc'}} 2024-09-12T01:49:47.180 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc'} 2024-09-12T01:49:47.181 INFO:teuthology.task.install:Using flavor: default 2024-09-12T01:49:47.187 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-12T01:49:47.187 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-09-12T01:49:47.188 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc', 'tag': None, 'wait_for_package': False} 2024-09-12T01:49:47.188 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T01:49:47.190 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc', 'tag': None, 'wait_for_package': False} 2024-09-12T01:49:47.191 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T01:49:47.192 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc', 'tag': None, 'wait_for_package': False} 2024-09-12T01:49:47.192 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T01:49:47.354 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/ 2024-09-12T01:49:47.354 INFO:teuthology.task.install.rpm:Package version is 19.1.1-298.gb53b6d3e 2024-09-12T01:49:47.367 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/ 2024-09-12T01:49:47.367 INFO:teuthology.task.install.rpm:Package version is 19.1.1-298.gb53b6d3e 2024-09-12T01:49:47.368 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/ 2024-09-12T01:49:47.368 INFO:teuthology.task.install.rpm:Package version is 19.1.1-298.gb53b6d3e 2024-09-12T01:49:47.476 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-12T01:49:47.476 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-09-12T01:49:47.476 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-12T01:49:47.488 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-12T01:49:47.488 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-12T01:49:47.489 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-12T01:49:47.498 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-12T01:49:47.498 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-12T01:49:47.498 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-12T01:49:47.502 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-12T01:49:47.502 DEBUG:teuthology.orchestra.run.smithi033:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-12T01:49:47.516 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-12T01:49:47.516 DEBUG:teuthology.orchestra.run.smithi184:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-12T01:49:47.527 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-12T01:49:47.527 DEBUG:teuthology.orchestra.run.smithi016:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-12T01:49:47.568 DEBUG:teuthology.orchestra.run.smithi033:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-12T01:49:47.583 DEBUG:teuthology.orchestra.run.smithi184:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-12T01:49:47.596 DEBUG:teuthology.orchestra.run.smithi016:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-12T01:49:47.643 DEBUG:teuthology.orchestra.run.smithi033:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-12T01:49:47.659 DEBUG:teuthology.orchestra.run.smithi184:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-12T01:49:47.674 DEBUG:teuthology.orchestra.run.smithi016:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-12T01:49:47.707 INFO:teuthology.orchestra.run.smithi033.stdout:check_obsoletes = 1 2024-09-12T01:49:47.708 DEBUG:teuthology.orchestra.run.smithi033:> sudo yum clean all 2024-09-12T01:49:47.724 INFO:teuthology.orchestra.run.smithi184.stdout:check_obsoletes = 1 2024-09-12T01:49:47.725 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum clean all 2024-09-12T01:49:47.739 INFO:teuthology.orchestra.run.smithi016.stdout:check_obsoletes = 1 2024-09-12T01:49:47.741 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum clean all 2024-09-12T01:49:47.939 INFO:teuthology.orchestra.run.smithi033.stdout:45 files removed 2024-09-12T01:49:47.949 INFO:teuthology.orchestra.run.smithi184.stdout:45 files removed 2024-09-12T01:49:47.967 DEBUG:teuthology.orchestra.run.smithi033:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-12T01:49:47.976 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-12T01:49:47.983 INFO:teuthology.orchestra.run.smithi016.stdout:45 files removed 2024-09-12T01:49:48.011 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-12T01:49:48.511 INFO:teuthology.orchestra.run.smithi033.stdout:ceph packages for x86_64 318 kB/s | 84 kB 00:00 2024-09-12T01:49:48.526 INFO:teuthology.orchestra.run.smithi184.stdout:ceph packages for x86_64 307 kB/s | 84 kB 00:00 2024-09-12T01:49:48.580 INFO:teuthology.orchestra.run.smithi016.stdout:ceph packages for x86_64 310 kB/s | 84 kB 00:00 2024-09-12T01:49:48.796 INFO:teuthology.orchestra.run.smithi033.stdout:ceph noarch packages 49 kB/s | 12 kB 00:00 2024-09-12T01:49:48.819 INFO:teuthology.orchestra.run.smithi184.stdout:ceph noarch packages 47 kB/s | 12 kB 00:00 2024-09-12T01:49:48.872 INFO:teuthology.orchestra.run.smithi016.stdout:ceph noarch packages 48 kB/s | 12 kB 00:00 2024-09-12T01:49:49.063 INFO:teuthology.orchestra.run.smithi033.stdout:ceph source packages 8.3 kB/s | 1.9 kB 00:00 2024-09-12T01:49:49.094 INFO:teuthology.orchestra.run.smithi184.stdout:ceph source packages 7.9 kB/s | 1.9 kB 00:00 2024-09-12T01:49:49.148 INFO:teuthology.orchestra.run.smithi016.stdout:ceph source packages 8.0 kB/s | 1.9 kB 00:00 2024-09-12T01:49:50.223 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - BaseOS 7.9 MB/s | 8.2 MB 00:01 2024-09-12T01:49:50.377 INFO:teuthology.orchestra.run.smithi184.stdout:CentOS Stream 9 - BaseOS 6.6 MB/s | 8.2 MB 00:01 2024-09-12T01:49:50.396 INFO:teuthology.orchestra.run.smithi033.stdout:CentOS Stream 9 - BaseOS 6.4 MB/s | 8.2 MB 00:01 2024-09-12T01:49:52.312 INFO:teuthology.orchestra.run.smithi184.stdout:CentOS Stream 9 - AppStream 25 MB/s | 20 MB 00:00 2024-09-12T01:49:52.372 INFO:teuthology.orchestra.run.smithi033.stdout:CentOS Stream 9 - AppStream 25 MB/s | 20 MB 00:00 2024-09-12T01:49:52.491 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - AppStream 18 MB/s | 20 MB 00:01 2024-09-12T01:49:56.824 INFO:teuthology.orchestra.run.smithi033.stdout:CentOS Stream 9 - CRB 12 MB/s | 6.5 MB 00:00 2024-09-12T01:49:56.842 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - CRB 18 MB/s | 6.5 MB 00:00 2024-09-12T01:49:57.479 INFO:teuthology.orchestra.run.smithi184.stdout:CentOS Stream 9 - CRB 5.1 MB/s | 6.5 MB 00:01 2024-09-12T01:49:58.310 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - Extras packages 102 kB/s | 19 kB 00:00 2024-09-12T01:49:58.482 INFO:teuthology.orchestra.run.smithi033.stdout:CentOS Stream 9 - Extras packages 47 kB/s | 19 kB 00:00 2024-09-12T01:49:59.289 INFO:teuthology.orchestra.run.smithi016.stdout:Extra Packages for Enterprise Linux 27 MB/s | 23 MB 00:00 2024-09-12T01:49:59.405 INFO:teuthology.orchestra.run.smithi184.stdout:CentOS Stream 9 - Extras packages 26 kB/s | 19 kB 00:00 2024-09-12T01:49:59.444 INFO:teuthology.orchestra.run.smithi033.stdout:Extra Packages for Enterprise Linux 27 MB/s | 23 MB 00:00 2024-09-12T01:50:00.326 INFO:teuthology.orchestra.run.smithi184.stdout:Extra Packages for Enterprise Linux 29 MB/s | 23 MB 00:00 2024-09-12T01:50:04.928 INFO:teuthology.orchestra.run.smithi016.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-09-12T01:50:05.125 INFO:teuthology.orchestra.run.smithi033.stdout:lab-extras 27 kB/s | 1.7 kB 00:00 2024-09-12T01:50:05.806 INFO:teuthology.orchestra.run.smithi184.stdout:lab-extras 34 kB/s | 1.7 kB 00:00 2024-09-12T01:50:07.367 INFO:teuthology.orchestra.run.smithi016.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-12T01:50:07.368 INFO:teuthology.orchestra.run.smithi016.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-12T01:50:07.381 INFO:teuthology.orchestra.run.smithi016.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-12T01:50:07.382 INFO:teuthology.orchestra.run.smithi016.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-12T01:50:07.383 INFO:teuthology.orchestra.run.smithi016.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-12T01:50:07.385 INFO:teuthology.orchestra.run.smithi016.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-12T01:50:07.386 INFO:teuthology.orchestra.run.smithi016.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-12T01:50:07.389 INFO:teuthology.orchestra.run.smithi016.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-12T01:50:07.390 INFO:teuthology.orchestra.run.smithi016.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-12T01:50:07.459 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2024-09-12T01:50:07.469 INFO:teuthology.orchestra.run.smithi016.stdout:====================================================================================== 2024-09-12T01:50:07.469 INFO:teuthology.orchestra.run.smithi016.stdout: Package Arch Version Repository Size 2024-09-12T01:50:07.469 INFO:teuthology.orchestra.run.smithi016.stdout:====================================================================================== 2024-09-12T01:50:07.469 INFO:teuthology.orchestra.run.smithi016.stdout:Installing: 2024-09-12T01:50:07.469 INFO:teuthology.orchestra.run.smithi016.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-12T01:50:07.470 INFO:teuthology.orchestra.run.smithi016.stdout: ceph x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 6.5 k 2024-09-12T01:50:07.470 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-base x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 5.4 M 2024-09-12T01:50:07.470 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-fuse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 897 k 2024-09-12T01:50:07.470 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 145 k 2024-09-12T01:50:07.470 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 1.5 M 2024-09-12T01:50:07.470 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-cephadm noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 146 k 2024-09-12T01:50:07.470 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-dashboard noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 3.6 M 2024-09-12T01:50:07.470 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 7.4 M 2024-09-12T01:50:07.470 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-rook noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 49 k 2024-09-12T01:50:07.470 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-radosgw x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 11 M 2024-09-12T01:50:07.470 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-test x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 49 M 2024-09-12T01:50:07.470 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-volume noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 285 k 2024-09-12T01:50:07.470 INFO:teuthology.orchestra.run.smithi016.stdout: cephadm noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 765 k 2024-09-12T01:50:07.470 INFO:teuthology.orchestra.run.smithi016.stdout: cephfs-mirror x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 228 k 2024-09-12T01:50:07.471 INFO:teuthology.orchestra.run.smithi016.stdout: cephfs-shell x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 27 k 2024-09-12T01:50:07.471 INFO:teuthology.orchestra.run.smithi016.stdout: cephfs-top noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 18 k 2024-09-12T01:50:07.471 INFO:teuthology.orchestra.run.smithi016.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-12T01:50:07.471 INFO:teuthology.orchestra.run.smithi016.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-12T01:50:07.471 INFO:teuthology.orchestra.run.smithi016.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-12T01:50:07.471 INFO:teuthology.orchestra.run.smithi016.stdout: libcephfs-devel x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 32 k 2024-09-12T01:50:07.471 INFO:teuthology.orchestra.run.smithi016.stdout: libcephfs2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 752 k 2024-09-12T01:50:07.471 INFO:teuthology.orchestra.run.smithi016.stdout: librados-devel x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 127 k 2024-09-12T01:50:07.471 INFO:teuthology.orchestra.run.smithi016.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-12T01:50:07.471 INFO:teuthology.orchestra.run.smithi016.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-12T01:50:07.471 INFO:teuthology.orchestra.run.smithi016.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-12T01:50:07.471 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-12T01:50:07.471 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-12T01:50:07.472 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-12T01:50:07.472 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cephfs x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 162 k 2024-09-12T01:50:07.472 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rados x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 321 k 2024-09-12T01:50:07.472 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rbd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 301 k 2024-09-12T01:50:07.472 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rgw x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 100 k 2024-09-12T01:50:07.472 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-fuse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 85 k 2024-09-12T01:50:07.472 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-mirror x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.1 M 2024-09-12T01:50:07.472 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-nbd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 171 k 2024-09-12T01:50:07.472 INFO:teuthology.orchestra.run.smithi016.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-12T01:50:07.472 INFO:teuthology.orchestra.run.smithi016.stdout:Upgrading: 2024-09-12T01:50:07.472 INFO:teuthology.orchestra.run.smithi016.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-12T01:50:07.472 INFO:teuthology.orchestra.run.smithi016.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-12T01:50:07.473 INFO:teuthology.orchestra.run.smithi016.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-12T01:50:07.473 INFO:teuthology.orchestra.run.smithi016.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-12T01:50:07.473 INFO:teuthology.orchestra.run.smithi016.stdout: librados2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.4 M 2024-09-12T01:50:07.473 INFO:teuthology.orchestra.run.smithi016.stdout: librbd1 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.2 M 2024-09-12T01:50:07.473 INFO:teuthology.orchestra.run.smithi016.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-12T01:50:07.473 INFO:teuthology.orchestra.run.smithi016.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-12T01:50:07.473 INFO:teuthology.orchestra.run.smithi016.stdout:Installing dependencies: 2024-09-12T01:50:07.473 INFO:teuthology.orchestra.run.smithi016.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-12T01:50:07.473 INFO:teuthology.orchestra.run.smithi016.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-12T01:50:07.473 INFO:teuthology.orchestra.run.smithi016.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-12T01:50:07.473 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-common x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 21 M 2024-09-12T01:50:07.473 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-grafana-dashboards noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 32 k 2024-09-12T01:50:07.474 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mds x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 2.4 M 2024-09-12T01:50:07.474 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-modules-core noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 249 k 2024-09-12T01:50:07.474 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mon x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 4.7 M 2024-09-12T01:50:07.474 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-osd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 17 M 2024-09-12T01:50:07.474 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-prometheus-alerts noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 16 k 2024-09-12T01:50:07.474 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-selinux x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 25 k 2024-09-12T01:50:07.474 INFO:teuthology.orchestra.run.smithi016.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-12T01:50:07.474 INFO:teuthology.orchestra.run.smithi016.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-12T01:50:07.474 INFO:teuthology.orchestra.run.smithi016.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-12T01:50:07.474 INFO:teuthology.orchestra.run.smithi016.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-12T01:50:07.474 INFO:teuthology.orchestra.run.smithi016.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-12T01:50:07.474 INFO:teuthology.orchestra.run.smithi016.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-12T01:50:07.474 INFO:teuthology.orchestra.run.smithi016.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-12T01:50:07.475 INFO:teuthology.orchestra.run.smithi016.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-12T01:50:07.475 INFO:teuthology.orchestra.run.smithi016.stdout: libcephsqlite x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 163 k 2024-09-12T01:50:07.475 INFO:teuthology.orchestra.run.smithi016.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-12T01:50:07.475 INFO:teuthology.orchestra.run.smithi016.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-12T01:50:07.475 INFO:teuthology.orchestra.run.smithi016.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-12T01:50:07.475 INFO:teuthology.orchestra.run.smithi016.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-12T01:50:07.475 INFO:teuthology.orchestra.run.smithi016.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-12T01:50:07.475 INFO:teuthology.orchestra.run.smithi016.stdout: libradosstriper1 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 506 k 2024-09-12T01:50:07.475 INFO:teuthology.orchestra.run.smithi016.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-12T01:50:07.475 INFO:teuthology.orchestra.run.smithi016.stdout: librgw2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 5.3 M 2024-09-12T01:50:07.475 INFO:teuthology.orchestra.run.smithi016.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-12T01:50:07.475 INFO:teuthology.orchestra.run.smithi016.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-12T01:50:07.475 INFO:teuthology.orchestra.run.smithi016.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-12T01:50:07.476 INFO:teuthology.orchestra.run.smithi016.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-12T01:50:07.476 INFO:teuthology.orchestra.run.smithi016.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-12T01:50:07.476 INFO:teuthology.orchestra.run.smithi016.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-12T01:50:07.476 INFO:teuthology.orchestra.run.smithi016.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-12T01:50:07.476 INFO:teuthology.orchestra.run.smithi016.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-12T01:50:07.476 INFO:teuthology.orchestra.run.smithi016.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-12T01:50:07.476 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-12T01:50:07.476 INFO:teuthology.orchestra.run.smithi016.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-12T01:50:07.476 INFO:teuthology.orchestra.run.smithi016.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-12T01:50:07.476 INFO:teuthology.orchestra.run.smithi016.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-12T01:50:07.476 INFO:teuthology.orchestra.run.smithi016.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-12T01:50:07.476 INFO:teuthology.orchestra.run.smithi016.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-12T01:50:07.477 INFO:teuthology.orchestra.run.smithi016.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-12T01:50:07.477 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-12T01:50:07.477 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ceph-argparse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 45 k 2024-09-12T01:50:07.477 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ceph-common x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 136 k 2024-09-12T01:50:07.477 INFO:teuthology.orchestra.run.smithi016.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-12T01:50:07.477 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-12T01:50:07.477 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-12T01:50:07.477 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-12T01:50:07.477 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-12T01:50:07.477 INFO:teuthology.orchestra.run.smithi016.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-12T01:50:07.477 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-12T01:50:07.477 INFO:teuthology.orchestra.run.smithi016.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-12T01:50:07.477 INFO:teuthology.orchestra.run.smithi016.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-12T01:50:07.477 INFO:teuthology.orchestra.run.smithi016.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-12T01:50:07.478 INFO:teuthology.orchestra.run.smithi016.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-12T01:50:07.478 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-12T01:50:07.478 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-12T01:50:07.478 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-12T01:50:07.478 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-12T01:50:07.478 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-12T01:50:07.478 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-12T01:50:07.478 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-12T01:50:07.478 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-12T01:50:07.478 INFO:teuthology.orchestra.run.smithi016.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-12T01:50:07.478 INFO:teuthology.orchestra.run.smithi016.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-12T01:50:07.478 INFO:teuthology.orchestra.run.smithi016.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-12T01:50:07.478 INFO:teuthology.orchestra.run.smithi016.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-12T01:50:07.479 INFO:teuthology.orchestra.run.smithi016.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-12T01:50:07.479 INFO:teuthology.orchestra.run.smithi016.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-12T01:50:07.479 INFO:teuthology.orchestra.run.smithi016.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-12T01:50:07.479 INFO:teuthology.orchestra.run.smithi016.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-12T01:50:07.479 INFO:teuthology.orchestra.run.smithi016.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-12T01:50:07.479 INFO:teuthology.orchestra.run.smithi016.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-12T01:50:07.479 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-12T01:50:07.479 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-12T01:50:07.479 INFO:teuthology.orchestra.run.smithi016.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-12T01:50:07.479 INFO:teuthology.orchestra.run.smithi016.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-12T01:50:07.479 INFO:teuthology.orchestra.run.smithi016.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-12T01:50:07.479 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-12T01:50:07.479 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-12T01:50:07.480 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-12T01:50:07.480 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-12T01:50:07.480 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-12T01:50:07.480 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-12T01:50:07.480 INFO:teuthology.orchestra.run.smithi016.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-12T01:50:07.480 INFO:teuthology.orchestra.run.smithi016.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-12T01:50:07.480 INFO:teuthology.orchestra.run.smithi016.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-12T01:50:07.480 INFO:teuthology.orchestra.run.smithi016.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-12T01:50:07.480 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-12T01:50:07.480 INFO:teuthology.orchestra.run.smithi016.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-12T01:50:07.480 INFO:teuthology.orchestra.run.smithi016.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-12T01:50:07.480 INFO:teuthology.orchestra.run.smithi016.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-12T01:50:07.480 INFO:teuthology.orchestra.run.smithi016.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-12T01:50:07.480 INFO:teuthology.orchestra.run.smithi016.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-12T01:50:07.481 INFO:teuthology.orchestra.run.smithi016.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-12T01:50:07.481 INFO:teuthology.orchestra.run.smithi016.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-12T01:50:07.481 INFO:teuthology.orchestra.run.smithi016.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-12T01:50:07.481 INFO:teuthology.orchestra.run.smithi016.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-12T01:50:07.481 INFO:teuthology.orchestra.run.smithi016.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-12T01:50:07.481 INFO:teuthology.orchestra.run.smithi016.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-12T01:50:07.481 INFO:teuthology.orchestra.run.smithi016.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-12T01:50:07.481 INFO:teuthology.orchestra.run.smithi016.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-12T01:50:07.481 INFO:teuthology.orchestra.run.smithi016.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-12T01:50:07.481 INFO:teuthology.orchestra.run.smithi016.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-12T01:50:07.481 INFO:teuthology.orchestra.run.smithi016.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-12T01:50:07.481 INFO:teuthology.orchestra.run.smithi016.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-12T01:50:07.481 INFO:teuthology.orchestra.run.smithi016.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-12T01:50:07.481 INFO:teuthology.orchestra.run.smithi016.stdout:Installing weak dependencies: 2024-09-12T01:50:07.481 INFO:teuthology.orchestra.run.smithi016.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-12T01:50:07.482 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:50:07.482 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction Summary 2024-09-12T01:50:07.482 INFO:teuthology.orchestra.run.smithi016.stdout:====================================================================================== 2024-09-12T01:50:07.482 INFO:teuthology.orchestra.run.smithi016.stdout:Install 148 Packages 2024-09-12T01:50:07.482 INFO:teuthology.orchestra.run.smithi016.stdout:Upgrade 8 Packages 2024-09-12T01:50:07.482 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:50:07.482 INFO:teuthology.orchestra.run.smithi016.stdout:Total download size: 226 M 2024-09-12T01:50:07.482 INFO:teuthology.orchestra.run.smithi016.stdout:Downloading Packages: 2024-09-12T01:50:07.503 INFO:teuthology.orchestra.run.smithi033.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-12T01:50:07.504 INFO:teuthology.orchestra.run.smithi033.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-12T01:50:07.516 INFO:teuthology.orchestra.run.smithi033.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-12T01:50:07.517 INFO:teuthology.orchestra.run.smithi033.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-12T01:50:07.517 INFO:teuthology.orchestra.run.smithi033.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-12T01:50:07.518 INFO:teuthology.orchestra.run.smithi033.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-12T01:50:07.520 INFO:teuthology.orchestra.run.smithi033.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-12T01:50:07.522 INFO:teuthology.orchestra.run.smithi033.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-12T01:50:07.523 INFO:teuthology.orchestra.run.smithi033.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-12T01:50:07.589 INFO:teuthology.orchestra.run.smithi033.stdout:Dependencies resolved. 2024-09-12T01:50:07.609 INFO:teuthology.orchestra.run.smithi033.stdout:====================================================================================== 2024-09-12T01:50:07.609 INFO:teuthology.orchestra.run.smithi033.stdout: Package Arch Version Repository Size 2024-09-12T01:50:07.610 INFO:teuthology.orchestra.run.smithi033.stdout:====================================================================================== 2024-09-12T01:50:07.610 INFO:teuthology.orchestra.run.smithi033.stdout:Installing: 2024-09-12T01:50:07.610 INFO:teuthology.orchestra.run.smithi033.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-12T01:50:07.610 INFO:teuthology.orchestra.run.smithi033.stdout: ceph x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 6.5 k 2024-09-12T01:50:07.610 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-base x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 5.4 M 2024-09-12T01:50:07.610 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-fuse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 897 k 2024-09-12T01:50:07.610 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 145 k 2024-09-12T01:50:07.610 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 1.5 M 2024-09-12T01:50:07.610 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-cephadm noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 146 k 2024-09-12T01:50:07.610 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-dashboard noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 3.6 M 2024-09-12T01:50:07.610 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 7.4 M 2024-09-12T01:50:07.610 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-rook noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 49 k 2024-09-12T01:50:07.610 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-radosgw x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 11 M 2024-09-12T01:50:07.610 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-test x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 49 M 2024-09-12T01:50:07.611 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-volume noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 285 k 2024-09-12T01:50:07.611 INFO:teuthology.orchestra.run.smithi033.stdout: cephadm noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 765 k 2024-09-12T01:50:07.611 INFO:teuthology.orchestra.run.smithi033.stdout: cephfs-mirror x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 228 k 2024-09-12T01:50:07.611 INFO:teuthology.orchestra.run.smithi033.stdout: cephfs-shell x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 27 k 2024-09-12T01:50:07.611 INFO:teuthology.orchestra.run.smithi033.stdout: cephfs-top noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 18 k 2024-09-12T01:50:07.611 INFO:teuthology.orchestra.run.smithi033.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-12T01:50:07.611 INFO:teuthology.orchestra.run.smithi033.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-12T01:50:07.611 INFO:teuthology.orchestra.run.smithi033.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-12T01:50:07.611 INFO:teuthology.orchestra.run.smithi033.stdout: libcephfs-devel x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 32 k 2024-09-12T01:50:07.611 INFO:teuthology.orchestra.run.smithi033.stdout: libcephfs2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 752 k 2024-09-12T01:50:07.611 INFO:teuthology.orchestra.run.smithi033.stdout: librados-devel x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 127 k 2024-09-12T01:50:07.611 INFO:teuthology.orchestra.run.smithi033.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-12T01:50:07.611 INFO:teuthology.orchestra.run.smithi033.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-12T01:50:07.612 INFO:teuthology.orchestra.run.smithi033.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-12T01:50:07.612 INFO:teuthology.orchestra.run.smithi033.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-12T01:50:07.612 INFO:teuthology.orchestra.run.smithi033.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-12T01:50:07.612 INFO:teuthology.orchestra.run.smithi033.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-12T01:50:07.612 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cephfs x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 162 k 2024-09-12T01:50:07.612 INFO:teuthology.orchestra.run.smithi033.stdout: python3-rados x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 321 k 2024-09-12T01:50:07.612 INFO:teuthology.orchestra.run.smithi033.stdout: python3-rbd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 301 k 2024-09-12T01:50:07.612 INFO:teuthology.orchestra.run.smithi033.stdout: python3-rgw x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 100 k 2024-09-12T01:50:07.612 INFO:teuthology.orchestra.run.smithi033.stdout: rbd-fuse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 85 k 2024-09-12T01:50:07.612 INFO:teuthology.orchestra.run.smithi033.stdout: rbd-mirror x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.1 M 2024-09-12T01:50:07.612 INFO:teuthology.orchestra.run.smithi033.stdout: rbd-nbd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 171 k 2024-09-12T01:50:07.612 INFO:teuthology.orchestra.run.smithi033.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-12T01:50:07.612 INFO:teuthology.orchestra.run.smithi033.stdout:Upgrading: 2024-09-12T01:50:07.613 INFO:teuthology.orchestra.run.smithi033.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-12T01:50:07.613 INFO:teuthology.orchestra.run.smithi033.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-12T01:50:07.613 INFO:teuthology.orchestra.run.smithi033.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-12T01:50:07.613 INFO:teuthology.orchestra.run.smithi033.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-12T01:50:07.613 INFO:teuthology.orchestra.run.smithi033.stdout: librados2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.4 M 2024-09-12T01:50:07.613 INFO:teuthology.orchestra.run.smithi033.stdout: librbd1 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.2 M 2024-09-12T01:50:07.613 INFO:teuthology.orchestra.run.smithi033.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-12T01:50:07.613 INFO:teuthology.orchestra.run.smithi033.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-12T01:50:07.613 INFO:teuthology.orchestra.run.smithi033.stdout:Installing dependencies: 2024-09-12T01:50:07.613 INFO:teuthology.orchestra.run.smithi033.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-12T01:50:07.613 INFO:teuthology.orchestra.run.smithi033.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-12T01:50:07.613 INFO:teuthology.orchestra.run.smithi033.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-12T01:50:07.613 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-common x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 21 M 2024-09-12T01:50:07.613 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-grafana-dashboards noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 32 k 2024-09-12T01:50:07.614 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mds x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 2.4 M 2024-09-12T01:50:07.614 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-modules-core noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 249 k 2024-09-12T01:50:07.614 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mon x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 4.7 M 2024-09-12T01:50:07.614 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-osd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 17 M 2024-09-12T01:50:07.614 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-prometheus-alerts noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 16 k 2024-09-12T01:50:07.614 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-selinux x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 25 k 2024-09-12T01:50:07.614 INFO:teuthology.orchestra.run.smithi033.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-12T01:50:07.614 INFO:teuthology.orchestra.run.smithi033.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-12T01:50:07.614 INFO:teuthology.orchestra.run.smithi033.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-12T01:50:07.614 INFO:teuthology.orchestra.run.smithi033.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-12T01:50:07.614 INFO:teuthology.orchestra.run.smithi033.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-12T01:50:07.614 INFO:teuthology.orchestra.run.smithi033.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-12T01:50:07.614 INFO:teuthology.orchestra.run.smithi033.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-12T01:50:07.614 INFO:teuthology.orchestra.run.smithi033.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-12T01:50:07.615 INFO:teuthology.orchestra.run.smithi033.stdout: libcephsqlite x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 163 k 2024-09-12T01:50:07.615 INFO:teuthology.orchestra.run.smithi033.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-12T01:50:07.615 INFO:teuthology.orchestra.run.smithi033.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-12T01:50:07.615 INFO:teuthology.orchestra.run.smithi033.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-12T01:50:07.615 INFO:teuthology.orchestra.run.smithi033.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-12T01:50:07.615 INFO:teuthology.orchestra.run.smithi033.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-12T01:50:07.615 INFO:teuthology.orchestra.run.smithi033.stdout: libradosstriper1 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 506 k 2024-09-12T01:50:07.615 INFO:teuthology.orchestra.run.smithi033.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-12T01:50:07.615 INFO:teuthology.orchestra.run.smithi033.stdout: librgw2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 5.3 M 2024-09-12T01:50:07.615 INFO:teuthology.orchestra.run.smithi033.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-12T01:50:07.615 INFO:teuthology.orchestra.run.smithi033.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-12T01:50:07.615 INFO:teuthology.orchestra.run.smithi033.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-12T01:50:07.615 INFO:teuthology.orchestra.run.smithi033.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-12T01:50:07.615 INFO:teuthology.orchestra.run.smithi033.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-12T01:50:07.615 INFO:teuthology.orchestra.run.smithi033.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-12T01:50:07.616 INFO:teuthology.orchestra.run.smithi033.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-12T01:50:07.616 INFO:teuthology.orchestra.run.smithi033.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-12T01:50:07.616 INFO:teuthology.orchestra.run.smithi033.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-12T01:50:07.616 INFO:teuthology.orchestra.run.smithi033.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-12T01:50:07.616 INFO:teuthology.orchestra.run.smithi033.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-12T01:50:07.616 INFO:teuthology.orchestra.run.smithi033.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-12T01:50:07.616 INFO:teuthology.orchestra.run.smithi033.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-12T01:50:07.616 INFO:teuthology.orchestra.run.smithi033.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-12T01:50:07.616 INFO:teuthology.orchestra.run.smithi033.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-12T01:50:07.616 INFO:teuthology.orchestra.run.smithi033.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-12T01:50:07.616 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-12T01:50:07.616 INFO:teuthology.orchestra.run.smithi033.stdout: python3-ceph-argparse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 45 k 2024-09-12T01:50:07.616 INFO:teuthology.orchestra.run.smithi033.stdout: python3-ceph-common x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 136 k 2024-09-12T01:50:07.616 INFO:teuthology.orchestra.run.smithi033.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-12T01:50:07.616 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-12T01:50:07.617 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-12T01:50:07.617 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-12T01:50:07.617 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-12T01:50:07.617 INFO:teuthology.orchestra.run.smithi033.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-12T01:50:07.617 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-12T01:50:07.617 INFO:teuthology.orchestra.run.smithi033.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-12T01:50:07.617 INFO:teuthology.orchestra.run.smithi033.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-12T01:50:07.617 INFO:teuthology.orchestra.run.smithi033.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-12T01:50:07.617 INFO:teuthology.orchestra.run.smithi033.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-12T01:50:07.617 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-12T01:50:07.617 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-12T01:50:07.617 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-12T01:50:07.617 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-12T01:50:07.618 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-12T01:50:07.618 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-12T01:50:07.618 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-12T01:50:07.618 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-12T01:50:07.618 INFO:teuthology.orchestra.run.smithi033.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-12T01:50:07.618 INFO:teuthology.orchestra.run.smithi033.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-12T01:50:07.618 INFO:teuthology.orchestra.run.smithi033.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-12T01:50:07.618 INFO:teuthology.orchestra.run.smithi033.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-12T01:50:07.618 INFO:teuthology.orchestra.run.smithi033.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-12T01:50:07.618 INFO:teuthology.orchestra.run.smithi033.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-12T01:50:07.618 INFO:teuthology.orchestra.run.smithi033.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-12T01:50:07.618 INFO:teuthology.orchestra.run.smithi033.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-12T01:50:07.618 INFO:teuthology.orchestra.run.smithi033.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-12T01:50:07.618 INFO:teuthology.orchestra.run.smithi033.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-12T01:50:07.619 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-12T01:50:07.619 INFO:teuthology.orchestra.run.smithi033.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-12T01:50:07.619 INFO:teuthology.orchestra.run.smithi033.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-12T01:50:07.619 INFO:teuthology.orchestra.run.smithi033.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-12T01:50:07.619 INFO:teuthology.orchestra.run.smithi033.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-12T01:50:07.619 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-12T01:50:07.619 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-12T01:50:07.619 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-12T01:50:07.619 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-12T01:50:07.619 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-12T01:50:07.619 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-12T01:50:07.619 INFO:teuthology.orchestra.run.smithi033.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-12T01:50:07.619 INFO:teuthology.orchestra.run.smithi033.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-12T01:50:07.620 INFO:teuthology.orchestra.run.smithi033.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-12T01:50:07.620 INFO:teuthology.orchestra.run.smithi033.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-12T01:50:07.620 INFO:teuthology.orchestra.run.smithi033.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-12T01:50:07.620 INFO:teuthology.orchestra.run.smithi033.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-12T01:50:07.620 INFO:teuthology.orchestra.run.smithi033.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-12T01:50:07.620 INFO:teuthology.orchestra.run.smithi033.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-12T01:50:07.620 INFO:teuthology.orchestra.run.smithi033.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-12T01:50:07.620 INFO:teuthology.orchestra.run.smithi033.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-12T01:50:07.620 INFO:teuthology.orchestra.run.smithi033.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-12T01:50:07.620 INFO:teuthology.orchestra.run.smithi033.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-12T01:50:07.620 INFO:teuthology.orchestra.run.smithi033.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-12T01:50:07.620 INFO:teuthology.orchestra.run.smithi033.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-12T01:50:07.620 INFO:teuthology.orchestra.run.smithi033.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-12T01:50:07.620 INFO:teuthology.orchestra.run.smithi033.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-12T01:50:07.621 INFO:teuthology.orchestra.run.smithi033.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-12T01:50:07.621 INFO:teuthology.orchestra.run.smithi033.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-12T01:50:07.621 INFO:teuthology.orchestra.run.smithi033.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-12T01:50:07.621 INFO:teuthology.orchestra.run.smithi033.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-12T01:50:07.621 INFO:teuthology.orchestra.run.smithi033.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-12T01:50:07.621 INFO:teuthology.orchestra.run.smithi033.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-12T01:50:07.621 INFO:teuthology.orchestra.run.smithi033.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-12T01:50:07.621 INFO:teuthology.orchestra.run.smithi033.stdout:Installing weak dependencies: 2024-09-12T01:50:07.621 INFO:teuthology.orchestra.run.smithi033.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-12T01:50:07.621 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-12T01:50:07.621 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction Summary 2024-09-12T01:50:07.621 INFO:teuthology.orchestra.run.smithi033.stdout:====================================================================================== 2024-09-12T01:50:07.621 INFO:teuthology.orchestra.run.smithi033.stdout:Install 148 Packages 2024-09-12T01:50:07.621 INFO:teuthology.orchestra.run.smithi033.stdout:Upgrade 8 Packages 2024-09-12T01:50:07.622 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-12T01:50:07.622 INFO:teuthology.orchestra.run.smithi033.stdout:Total download size: 226 M 2024-09-12T01:50:07.622 INFO:teuthology.orchestra.run.smithi033.stdout:Downloading Packages: 2024-09-12T01:50:08.141 INFO:teuthology.orchestra.run.smithi184.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-12T01:50:08.142 INFO:teuthology.orchestra.run.smithi184.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-12T01:50:08.153 INFO:teuthology.orchestra.run.smithi184.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-12T01:50:08.153 INFO:teuthology.orchestra.run.smithi184.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-12T01:50:08.154 INFO:teuthology.orchestra.run.smithi184.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-12T01:50:08.155 INFO:teuthology.orchestra.run.smithi184.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-12T01:50:08.157 INFO:teuthology.orchestra.run.smithi184.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-12T01:50:08.159 INFO:teuthology.orchestra.run.smithi184.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-12T01:50:08.160 INFO:teuthology.orchestra.run.smithi184.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-12T01:50:08.222 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2024-09-12T01:50:08.237 INFO:teuthology.orchestra.run.smithi184.stdout:====================================================================================== 2024-09-12T01:50:08.237 INFO:teuthology.orchestra.run.smithi184.stdout: Package Arch Version Repository Size 2024-09-12T01:50:08.237 INFO:teuthology.orchestra.run.smithi184.stdout:====================================================================================== 2024-09-12T01:50:08.237 INFO:teuthology.orchestra.run.smithi184.stdout:Installing: 2024-09-12T01:50:08.238 INFO:teuthology.orchestra.run.smithi184.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-12T01:50:08.238 INFO:teuthology.orchestra.run.smithi184.stdout: ceph x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 6.5 k 2024-09-12T01:50:08.238 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-base x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 5.4 M 2024-09-12T01:50:08.238 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-fuse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 897 k 2024-09-12T01:50:08.238 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 145 k 2024-09-12T01:50:08.238 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 1.5 M 2024-09-12T01:50:08.238 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-cephadm noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 146 k 2024-09-12T01:50:08.238 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-dashboard noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 3.6 M 2024-09-12T01:50:08.238 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 7.4 M 2024-09-12T01:50:08.238 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-rook noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 49 k 2024-09-12T01:50:08.238 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-radosgw x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 11 M 2024-09-12T01:50:08.238 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-test x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 49 M 2024-09-12T01:50:08.238 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-volume noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 285 k 2024-09-12T01:50:08.239 INFO:teuthology.orchestra.run.smithi184.stdout: cephadm noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 765 k 2024-09-12T01:50:08.239 INFO:teuthology.orchestra.run.smithi184.stdout: cephfs-mirror x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 228 k 2024-09-12T01:50:08.239 INFO:teuthology.orchestra.run.smithi184.stdout: cephfs-shell x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 27 k 2024-09-12T01:50:08.239 INFO:teuthology.orchestra.run.smithi184.stdout: cephfs-top noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 18 k 2024-09-12T01:50:08.239 INFO:teuthology.orchestra.run.smithi184.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-12T01:50:08.239 INFO:teuthology.orchestra.run.smithi184.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-12T01:50:08.239 INFO:teuthology.orchestra.run.smithi184.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-12T01:50:08.239 INFO:teuthology.orchestra.run.smithi184.stdout: libcephfs-devel x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 32 k 2024-09-12T01:50:08.239 INFO:teuthology.orchestra.run.smithi184.stdout: libcephfs2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 752 k 2024-09-12T01:50:08.239 INFO:teuthology.orchestra.run.smithi184.stdout: librados-devel x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 127 k 2024-09-12T01:50:08.239 INFO:teuthology.orchestra.run.smithi184.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-12T01:50:08.239 INFO:teuthology.orchestra.run.smithi184.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-12T01:50:08.240 INFO:teuthology.orchestra.run.smithi184.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-12T01:50:08.240 INFO:teuthology.orchestra.run.smithi184.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-12T01:50:08.240 INFO:teuthology.orchestra.run.smithi184.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-12T01:50:08.240 INFO:teuthology.orchestra.run.smithi184.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-12T01:50:08.240 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cephfs x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 162 k 2024-09-12T01:50:08.240 INFO:teuthology.orchestra.run.smithi184.stdout: python3-rados x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 321 k 2024-09-12T01:50:08.240 INFO:teuthology.orchestra.run.smithi184.stdout: python3-rbd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 301 k 2024-09-12T01:50:08.240 INFO:teuthology.orchestra.run.smithi184.stdout: python3-rgw x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 100 k 2024-09-12T01:50:08.240 INFO:teuthology.orchestra.run.smithi184.stdout: rbd-fuse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 85 k 2024-09-12T01:50:08.240 INFO:teuthology.orchestra.run.smithi184.stdout: rbd-mirror x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.1 M 2024-09-12T01:50:08.240 INFO:teuthology.orchestra.run.smithi184.stdout: rbd-nbd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 171 k 2024-09-12T01:50:08.240 INFO:teuthology.orchestra.run.smithi184.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-12T01:50:08.241 INFO:teuthology.orchestra.run.smithi184.stdout:Upgrading: 2024-09-12T01:50:08.241 INFO:teuthology.orchestra.run.smithi184.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-12T01:50:08.241 INFO:teuthology.orchestra.run.smithi184.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-12T01:50:08.241 INFO:teuthology.orchestra.run.smithi184.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-12T01:50:08.241 INFO:teuthology.orchestra.run.smithi184.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-12T01:50:08.241 INFO:teuthology.orchestra.run.smithi184.stdout: librados2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.4 M 2024-09-12T01:50:08.241 INFO:teuthology.orchestra.run.smithi184.stdout: librbd1 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.2 M 2024-09-12T01:50:08.241 INFO:teuthology.orchestra.run.smithi184.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-12T01:50:08.241 INFO:teuthology.orchestra.run.smithi184.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-12T01:50:08.241 INFO:teuthology.orchestra.run.smithi184.stdout:Installing dependencies: 2024-09-12T01:50:08.241 INFO:teuthology.orchestra.run.smithi184.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-12T01:50:08.241 INFO:teuthology.orchestra.run.smithi184.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-12T01:50:08.241 INFO:teuthology.orchestra.run.smithi184.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-12T01:50:08.242 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-common x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 21 M 2024-09-12T01:50:08.242 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-grafana-dashboards noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 32 k 2024-09-12T01:50:08.242 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mds x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 2.4 M 2024-09-12T01:50:08.242 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-modules-core noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 249 k 2024-09-12T01:50:08.242 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mon x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 4.7 M 2024-09-12T01:50:08.242 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-osd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 17 M 2024-09-12T01:50:08.242 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-prometheus-alerts noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 16 k 2024-09-12T01:50:08.242 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-selinux x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 25 k 2024-09-12T01:50:08.242 INFO:teuthology.orchestra.run.smithi184.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-12T01:50:08.242 INFO:teuthology.orchestra.run.smithi184.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-12T01:50:08.242 INFO:teuthology.orchestra.run.smithi184.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-12T01:50:08.242 INFO:teuthology.orchestra.run.smithi184.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-12T01:50:08.243 INFO:teuthology.orchestra.run.smithi184.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-12T01:50:08.243 INFO:teuthology.orchestra.run.smithi184.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-12T01:50:08.243 INFO:teuthology.orchestra.run.smithi184.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-12T01:50:08.243 INFO:teuthology.orchestra.run.smithi184.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-12T01:50:08.243 INFO:teuthology.orchestra.run.smithi184.stdout: libcephsqlite x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 163 k 2024-09-12T01:50:08.243 INFO:teuthology.orchestra.run.smithi184.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-12T01:50:08.243 INFO:teuthology.orchestra.run.smithi184.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-12T01:50:08.243 INFO:teuthology.orchestra.run.smithi184.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-12T01:50:08.243 INFO:teuthology.orchestra.run.smithi184.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-12T01:50:08.243 INFO:teuthology.orchestra.run.smithi184.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-12T01:50:08.243 INFO:teuthology.orchestra.run.smithi184.stdout: libradosstriper1 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 506 k 2024-09-12T01:50:08.243 INFO:teuthology.orchestra.run.smithi184.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-12T01:50:08.243 INFO:teuthology.orchestra.run.smithi184.stdout: librgw2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 5.3 M 2024-09-12T01:50:08.244 INFO:teuthology.orchestra.run.smithi184.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-12T01:50:08.244 INFO:teuthology.orchestra.run.smithi184.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-12T01:50:08.244 INFO:teuthology.orchestra.run.smithi184.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-12T01:50:08.244 INFO:teuthology.orchestra.run.smithi184.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-12T01:50:08.244 INFO:teuthology.orchestra.run.smithi184.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-12T01:50:08.244 INFO:teuthology.orchestra.run.smithi184.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-12T01:50:08.244 INFO:teuthology.orchestra.run.smithi184.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-12T01:50:08.244 INFO:teuthology.orchestra.run.smithi184.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-12T01:50:08.244 INFO:teuthology.orchestra.run.smithi184.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-12T01:50:08.244 INFO:teuthology.orchestra.run.smithi184.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-12T01:50:08.244 INFO:teuthology.orchestra.run.smithi184.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-12T01:50:08.244 INFO:teuthology.orchestra.run.smithi184.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-12T01:50:08.244 INFO:teuthology.orchestra.run.smithi184.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-12T01:50:08.244 INFO:teuthology.orchestra.run.smithi184.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-12T01:50:08.245 INFO:teuthology.orchestra.run.smithi184.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-12T01:50:08.245 INFO:teuthology.orchestra.run.smithi184.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-12T01:50:08.245 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-12T01:50:08.245 INFO:teuthology.orchestra.run.smithi184.stdout: python3-ceph-argparse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 45 k 2024-09-12T01:50:08.245 INFO:teuthology.orchestra.run.smithi184.stdout: python3-ceph-common x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 136 k 2024-09-12T01:50:08.245 INFO:teuthology.orchestra.run.smithi184.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-12T01:50:08.245 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-12T01:50:08.245 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-12T01:50:08.245 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-12T01:50:08.245 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-12T01:50:08.245 INFO:teuthology.orchestra.run.smithi184.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-12T01:50:08.245 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-12T01:50:08.246 INFO:teuthology.orchestra.run.smithi184.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-12T01:50:08.246 INFO:teuthology.orchestra.run.smithi184.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-12T01:50:08.246 INFO:teuthology.orchestra.run.smithi184.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-12T01:50:08.246 INFO:teuthology.orchestra.run.smithi184.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-12T01:50:08.246 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-12T01:50:08.246 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-12T01:50:08.246 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-12T01:50:08.246 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-12T01:50:08.246 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-12T01:50:08.246 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-12T01:50:08.246 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-12T01:50:08.246 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-12T01:50:08.246 INFO:teuthology.orchestra.run.smithi184.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-12T01:50:08.247 INFO:teuthology.orchestra.run.smithi184.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-12T01:50:08.247 INFO:teuthology.orchestra.run.smithi184.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-12T01:50:08.247 INFO:teuthology.orchestra.run.smithi184.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-12T01:50:08.247 INFO:teuthology.orchestra.run.smithi184.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-12T01:50:08.247 INFO:teuthology.orchestra.run.smithi184.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-12T01:50:08.247 INFO:teuthology.orchestra.run.smithi184.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-12T01:50:08.247 INFO:teuthology.orchestra.run.smithi184.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-12T01:50:08.247 INFO:teuthology.orchestra.run.smithi184.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-12T01:50:08.247 INFO:teuthology.orchestra.run.smithi184.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-12T01:50:08.247 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-12T01:50:08.247 INFO:teuthology.orchestra.run.smithi184.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-12T01:50:08.247 INFO:teuthology.orchestra.run.smithi184.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-12T01:50:08.247 INFO:teuthology.orchestra.run.smithi184.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-12T01:50:08.248 INFO:teuthology.orchestra.run.smithi184.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-12T01:50:08.248 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-12T01:50:08.248 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-12T01:50:08.248 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-12T01:50:08.248 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-12T01:50:08.248 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-12T01:50:08.248 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-12T01:50:08.248 INFO:teuthology.orchestra.run.smithi184.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-12T01:50:08.248 INFO:teuthology.orchestra.run.smithi184.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-12T01:50:08.248 INFO:teuthology.orchestra.run.smithi184.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-12T01:50:08.248 INFO:teuthology.orchestra.run.smithi184.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-12T01:50:08.248 INFO:teuthology.orchestra.run.smithi184.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-12T01:50:08.248 INFO:teuthology.orchestra.run.smithi184.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-12T01:50:08.249 INFO:teuthology.orchestra.run.smithi184.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-12T01:50:08.249 INFO:teuthology.orchestra.run.smithi184.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-12T01:50:08.249 INFO:teuthology.orchestra.run.smithi184.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-12T01:50:08.249 INFO:teuthology.orchestra.run.smithi184.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-12T01:50:08.249 INFO:teuthology.orchestra.run.smithi184.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-12T01:50:08.249 INFO:teuthology.orchestra.run.smithi184.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-12T01:50:08.249 INFO:teuthology.orchestra.run.smithi184.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-12T01:50:08.249 INFO:teuthology.orchestra.run.smithi184.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-12T01:50:08.249 INFO:teuthology.orchestra.run.smithi184.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-12T01:50:08.249 INFO:teuthology.orchestra.run.smithi184.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-12T01:50:08.249 INFO:teuthology.orchestra.run.smithi184.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-12T01:50:08.249 INFO:teuthology.orchestra.run.smithi184.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-12T01:50:08.249 INFO:teuthology.orchestra.run.smithi184.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-12T01:50:08.250 INFO:teuthology.orchestra.run.smithi184.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-12T01:50:08.250 INFO:teuthology.orchestra.run.smithi184.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-12T01:50:08.250 INFO:teuthology.orchestra.run.smithi184.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-12T01:50:08.250 INFO:teuthology.orchestra.run.smithi184.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-12T01:50:08.250 INFO:teuthology.orchestra.run.smithi184.stdout:Installing weak dependencies: 2024-09-12T01:50:08.250 INFO:teuthology.orchestra.run.smithi184.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-12T01:50:08.250 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-12T01:50:08.250 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction Summary 2024-09-12T01:50:08.250 INFO:teuthology.orchestra.run.smithi184.stdout:====================================================================================== 2024-09-12T01:50:08.250 INFO:teuthology.orchestra.run.smithi184.stdout:Install 148 Packages 2024-09-12T01:50:08.250 INFO:teuthology.orchestra.run.smithi184.stdout:Upgrade 8 Packages 2024-09-12T01:50:08.250 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-12T01:50:08.251 INFO:teuthology.orchestra.run.smithi184.stdout:Total download size: 226 M 2024-09-12T01:50:08.251 INFO:teuthology.orchestra.run.smithi184.stdout:Downloading Packages: 2024-09-12T01:50:08.422 INFO:teuthology.orchestra.run.smithi016.stdout:(1/156): ceph-19.1.1-298.gb53b6d3e.el9.x86_64.r 52 kB/s | 6.5 kB 00:00 2024-09-12T01:50:08.451 INFO:teuthology.orchestra.run.smithi033.stdout:(1/156): ceph-19.1.1-298.gb53b6d3e.el9.x86_64.r 55 kB/s | 6.5 kB 00:00 2024-09-12T01:50:08.689 INFO:teuthology.orchestra.run.smithi016.stdout:(2/156): ceph-fuse-19.1.1-298.gb53b6d3e.el9.x86 3.3 MB/s | 897 kB 00:00 2024-09-12T01:50:08.835 INFO:teuthology.orchestra.run.smithi033.stdout:(2/156): ceph-fuse-19.1.1-298.gb53b6d3e.el9.x86 2.3 MB/s | 897 kB 00:00 2024-09-12T01:50:08.935 INFO:teuthology.orchestra.run.smithi033.stdout:(3/156): ceph-immutable-object-cache-19.1.1-298 1.4 MB/s | 145 kB 00:00 2024-09-12T01:50:08.939 INFO:teuthology.orchestra.run.smithi016.stdout:(3/156): ceph-base-19.1.1-298.gb53b6d3e.el9.x86 8.4 MB/s | 5.4 MB 00:00 2024-09-12T01:50:09.019 INFO:teuthology.orchestra.run.smithi033.stdout:(4/156): ceph-base-19.1.1-298.gb53b6d3e.el9.x86 7.9 MB/s | 5.4 MB 00:00 2024-09-12T01:50:09.081 INFO:teuthology.orchestra.run.smithi016.stdout:(4/156): ceph-immutable-object-cache-19.1.1-298 370 kB/s | 145 kB 00:00 2024-09-12T01:50:09.148 INFO:teuthology.orchestra.run.smithi016.stdout:(5/156): ceph-mds-19.1.1-298.gb53b6d3e.el9.x86_ 11 MB/s | 2.4 MB 00:00 2024-09-12T01:50:09.157 INFO:teuthology.orchestra.run.smithi184.stdout:(1/156): ceph-19.1.1-298.gb53b6d3e.el9.x86_64.r 27 kB/s | 6.5 kB 00:00 2024-09-12T01:50:09.227 INFO:teuthology.orchestra.run.smithi033.stdout:(5/156): ceph-mgr-19.1.1-298.gb53b6d3e.el9.x86_ 7.3 MB/s | 1.5 MB 00:00 2024-09-12T01:50:09.469 INFO:teuthology.orchestra.run.smithi033.stdout:(6/156): ceph-mds-19.1.1-298.gb53b6d3e.el9.x86_ 4.5 MB/s | 2.4 MB 00:00 2024-09-12T01:50:09.498 INFO:teuthology.orchestra.run.smithi016.stdout:(6/156): ceph-common-19.1.1-298.gb53b6d3e.el9.x 18 MB/s | 21 MB 00:01 2024-09-12T01:50:09.516 INFO:teuthology.orchestra.run.smithi184.stdout:(2/156): ceph-fuse-19.1.1-298.gb53b6d3e.el9.x86 2.4 MB/s | 897 kB 00:00 2024-09-12T01:50:09.548 INFO:teuthology.orchestra.run.smithi016.stdout:(7/156): ceph-mgr-19.1.1-298.gb53b6d3e.el9.x86_ 3.3 MB/s | 1.5 MB 00:00 2024-09-12T01:50:09.608 INFO:teuthology.orchestra.run.smithi184.stdout:(3/156): ceph-immutable-object-cache-19.1.1-298 1.5 MB/s | 145 kB 00:00 2024-09-12T01:50:09.636 INFO:teuthology.orchestra.run.smithi033.stdout:(7/156): ceph-mon-19.1.1-298.gb53b6d3e.el9.x86_ 12 MB/s | 4.7 MB 00:00 2024-09-12T01:50:09.640 INFO:teuthology.orchestra.run.smithi016.stdout:(8/156): ceph-mon-19.1.1-298.gb53b6d3e.el9.x86_ 9.6 MB/s | 4.7 MB 00:00 2024-09-12T01:50:09.732 INFO:teuthology.orchestra.run.smithi016.stdout:(9/156): ceph-selinux-19.1.1-298.gb53b6d3e.el9. 275 kB/s | 25 kB 00:00 2024-09-12T01:50:10.383 INFO:teuthology.orchestra.run.smithi184.stdout:(4/156): ceph-mds-19.1.1-298.gb53b6d3e.el9.x86_ 3.1 MB/s | 2.4 MB 00:00 2024-09-12T01:50:10.524 INFO:teuthology.orchestra.run.smithi016.stdout:(10/156): ceph-osd-19.1.1-298.gb53b6d3e.el9.x86 16 MB/s | 17 MB 00:01 2024-09-12T01:50:10.608 INFO:teuthology.orchestra.run.smithi016.stdout:(11/156): cephfs-mirror-19.1.1-298.gb53b6d3e.el 2.7 MB/s | 228 kB 00:00 2024-09-12T01:50:10.667 INFO:teuthology.orchestra.run.smithi184.stdout:(5/156): ceph-mgr-19.1.1-298.gb53b6d3e.el9.x86_ 5.4 MB/s | 1.5 MB 00:00 2024-09-12T01:50:10.674 INFO:teuthology.orchestra.run.smithi016.stdout:(12/156): cephfs-shell-19.1.1-298.gb53b6d3e.el9 407 kB/s | 27 kB 00:00 2024-09-12T01:50:10.750 INFO:teuthology.orchestra.run.smithi016.stdout:(13/156): libcephfs-devel-19.1.1-298.gb53b6d3e. 423 kB/s | 32 kB 00:00 2024-09-12T01:50:10.850 INFO:teuthology.orchestra.run.smithi016.stdout:(14/156): libcephfs2-19.1.1-298.gb53b6d3e.el9.x 7.4 MB/s | 752 kB 00:00 2024-09-12T01:50:10.903 INFO:teuthology.orchestra.run.smithi033.stdout:(8/156): ceph-osd-19.1.1-298.gb53b6d3e.el9.x86_ 12 MB/s | 17 MB 00:01 2024-09-12T01:50:10.917 INFO:teuthology.orchestra.run.smithi016.stdout:(15/156): libcephsqlite-19.1.1-298.gb53b6d3e.el 2.4 MB/s | 163 kB 00:00 2024-09-12T01:50:10.984 INFO:teuthology.orchestra.run.smithi016.stdout:(16/156): librados-devel-19.1.1-298.gb53b6d3e.e 1.8 MB/s | 127 kB 00:00 2024-09-12T01:50:11.028 INFO:teuthology.orchestra.run.smithi033.stdout:(9/156): ceph-radosgw-19.1.1-298.gb53b6d3e.el9. 7.6 MB/s | 11 MB 00:01 2024-09-12T01:50:11.051 INFO:teuthology.orchestra.run.smithi016.stdout:(17/156): libradosstriper1-19.1.1-298.gb53b6d3e 7.4 MB/s | 506 kB 00:00 2024-09-12T01:50:11.053 INFO:teuthology.orchestra.run.smithi033.stdout:(10/156): ceph-selinux-19.1.1-298.gb53b6d3e.el9 168 kB/s | 25 kB 00:00 2024-09-12T01:50:11.125 INFO:teuthology.orchestra.run.smithi184.stdout:(6/156): ceph-mon-19.1.1-298.gb53b6d3e.el9.x86_ 10 MB/s | 4.7 MB 00:00 2024-09-12T01:50:11.145 INFO:teuthology.orchestra.run.smithi033.stdout:(11/156): cephfs-mirror-19.1.1-298.gb53b6d3e.el 2.4 MB/s | 228 kB 00:00 2024-09-12T01:50:11.229 INFO:teuthology.orchestra.run.smithi033.stdout:(12/156): cephfs-shell-19.1.1-298.gb53b6d3e.el9 325 kB/s | 27 kB 00:00 2024-09-12T01:50:11.312 INFO:teuthology.orchestra.run.smithi033.stdout:(13/156): libcephfs-devel-19.1.1-298.gb53b6d3e. 381 kB/s | 32 kB 00:00 2024-09-12T01:50:11.352 INFO:teuthology.orchestra.run.smithi016.stdout:(18/156): librgw2-19.1.1-298.gb53b6d3e.el9.x86_ 18 MB/s | 5.3 MB 00:00 2024-09-12T01:50:11.388 INFO:teuthology.orchestra.run.smithi033.stdout:(14/156): libcephfs2-19.1.1-298.gb53b6d3e.el9.x 9.8 MB/s | 752 kB 00:00 2024-09-12T01:50:11.418 INFO:teuthology.orchestra.run.smithi016.stdout:(19/156): python3-ceph-argparse-19.1.1-298.gb53 680 kB/s | 45 kB 00:00 2024-09-12T01:50:11.454 INFO:teuthology.orchestra.run.smithi033.stdout:(15/156): libcephsqlite-19.1.1-298.gb53b6d3e.el 2.4 MB/s | 163 kB 00:00 2024-09-12T01:50:11.494 INFO:teuthology.orchestra.run.smithi016.stdout:(20/156): python3-ceph-common-19.1.1-298.gb53b6 1.8 MB/s | 136 kB 00:00 2024-09-12T01:50:11.521 INFO:teuthology.orchestra.run.smithi033.stdout:(16/156): librados-devel-19.1.1-298.gb53b6d3e.e 1.9 MB/s | 127 kB 00:00 2024-09-12T01:50:11.561 INFO:teuthology.orchestra.run.smithi016.stdout:(21/156): python3-cephfs-19.1.1-298.gb53b6d3e.e 2.4 MB/s | 162 kB 00:00 2024-09-12T01:50:11.597 INFO:teuthology.orchestra.run.smithi033.stdout:(17/156): libradosstriper1-19.1.1-298.gb53b6d3e 6.6 MB/s | 506 kB 00:00 2024-09-12T01:50:11.644 INFO:teuthology.orchestra.run.smithi016.stdout:(22/156): python3-rados-19.1.1-298.gb53b6d3e.el 3.8 MB/s | 321 kB 00:00 2024-09-12T01:50:11.728 INFO:teuthology.orchestra.run.smithi016.stdout:(23/156): python3-rbd-19.1.1-298.gb53b6d3e.el9. 3.5 MB/s | 301 kB 00:00 2024-09-12T01:50:11.845 INFO:teuthology.orchestra.run.smithi016.stdout:(24/156): python3-rgw-19.1.1-298.gb53b6d3e.el9. 852 kB/s | 100 kB 00:00 2024-09-12T01:50:11.947 INFO:teuthology.orchestra.run.smithi033.stdout:(18/156): librgw2-19.1.1-298.gb53b6d3e.el9.x86_ 15 MB/s | 5.3 MB 00:00 2024-09-12T01:50:11.989 INFO:teuthology.orchestra.run.smithi033.stdout:(19/156): python3-ceph-argparse-19.1.1-298.gb53 1.1 MB/s | 45 kB 00:00 2024-09-12T01:50:11.995 INFO:teuthology.orchestra.run.smithi016.stdout:(25/156): ceph-radosgw-19.1.1-298.gb53b6d3e.el9 4.3 MB/s | 11 MB 00:02 2024-09-12T01:50:12.021 INFO:teuthology.orchestra.run.smithi016.stdout:(26/156): rbd-fuse-19.1.1-298.gb53b6d3e.el9.x86 486 kB/s | 85 kB 00:00 2024-09-12T01:50:12.039 INFO:teuthology.orchestra.run.smithi033.stdout:(20/156): python3-ceph-common-19.1.1-298.gb53b6 2.7 MB/s | 136 kB 00:00 2024-09-12T01:50:12.088 INFO:teuthology.orchestra.run.smithi016.stdout:(27/156): rbd-nbd-19.1.1-298.gb53b6d3e.el9.x86_ 2.5 MB/s | 171 kB 00:00 2024-09-12T01:50:12.106 INFO:teuthology.orchestra.run.smithi033.stdout:(21/156): python3-cephfs-19.1.1-298.gb53b6d3e.e 2.4 MB/s | 162 kB 00:00 2024-09-12T01:50:12.154 INFO:teuthology.orchestra.run.smithi016.stdout:(28/156): ceph-grafana-dashboards-19.1.1-298.gb 479 kB/s | 32 kB 00:00 2024-09-12T01:50:12.173 INFO:teuthology.orchestra.run.smithi033.stdout:(22/156): python3-rados-19.1.1-298.gb53b6d3e.el 4.7 MB/s | 321 kB 00:00 2024-09-12T01:50:12.221 INFO:teuthology.orchestra.run.smithi016.stdout:(29/156): ceph-mgr-cephadm-19.1.1-298.gb53b6d3e 2.1 MB/s | 146 kB 00:00 2024-09-12T01:50:12.249 INFO:teuthology.orchestra.run.smithi033.stdout:(23/156): python3-rbd-19.1.1-298.gb53b6d3e.el9. 3.9 MB/s | 301 kB 00:00 2024-09-12T01:50:12.301 INFO:teuthology.orchestra.run.smithi184.stdout:(7/156): ceph-base-19.1.1-298.gb53b6d3e.el9.x86 1.6 MB/s | 5.4 MB 00:03 2024-09-12T01:50:12.316 INFO:teuthology.orchestra.run.smithi033.stdout:(24/156): python3-rgw-19.1.1-298.gb53b6d3e.el9. 1.5 MB/s | 100 kB 00:00 2024-09-12T01:50:12.407 INFO:teuthology.orchestra.run.smithi033.stdout:(25/156): rbd-fuse-19.1.1-298.gb53b6d3e.el9.x86 927 kB/s | 85 kB 00:00 2024-09-12T01:50:12.572 INFO:teuthology.orchestra.run.smithi016.stdout:(30/156): ceph-mgr-dashboard-19.1.1-298.gb53b6d 10 MB/s | 3.6 MB 00:00 2024-09-12T01:50:12.693 INFO:teuthology.orchestra.run.smithi033.stdout:(26/156): ceph-common-19.1.1-298.gb53b6d3e.el9. 4.9 MB/s | 21 MB 00:04 2024-09-12T01:50:12.714 INFO:teuthology.orchestra.run.smithi016.stdout:(31/156): rbd-mirror-19.1.1-298.gb53b6d3e.el9.x 4.3 MB/s | 3.1 MB 00:00 2024-09-12T01:50:12.758 INFO:teuthology.orchestra.run.smithi033.stdout:(27/156): rbd-mirror-19.1.1-298.gb53b6d3e.el9.x 8.8 MB/s | 3.1 MB 00:00 2024-09-12T01:50:12.798 INFO:teuthology.orchestra.run.smithi016.stdout:(32/156): ceph-mgr-modules-core-19.1.1-298.gb53 2.9 MB/s | 249 kB 00:00 2024-09-12T01:50:12.825 INFO:teuthology.orchestra.run.smithi033.stdout:(28/156): rbd-nbd-19.1.1-298.gb53b6d3e.el9.x86_ 1.3 MB/s | 171 kB 00:00 2024-09-12T01:50:12.850 INFO:teuthology.orchestra.run.smithi033.stdout:(29/156): ceph-grafana-dashboards-19.1.1-298.gb 347 kB/s | 32 kB 00:00 2024-09-12T01:50:12.918 INFO:teuthology.orchestra.run.smithi033.stdout:(30/156): ceph-mgr-cephadm-19.1.1-298.gb53b6d3e 1.5 MB/s | 146 kB 00:00 2024-09-12T01:50:13.006 INFO:teuthology.orchestra.run.smithi016.stdout:(33/156): ceph-mgr-rook-19.1.1-298.gb53b6d3e.el 237 kB/s | 49 kB 00:00 2024-09-12T01:50:13.051 INFO:teuthology.orchestra.run.smithi033.stdout:(31/156): ceph-mgr-dashboard-19.1.1-298.gb53b6d 18 MB/s | 3.6 MB 00:00 2024-09-12T01:50:13.093 INFO:teuthology.orchestra.run.smithi184.stdout:(8/156): ceph-osd-19.1.1-298.gb53b6d3e.el9.x86_ 8.6 MB/s | 17 MB 00:01 2024-09-12T01:50:13.101 INFO:teuthology.orchestra.run.smithi033.stdout:(32/156): ceph-mgr-modules-core-19.1.1-298.gb53 4.9 MB/s | 249 kB 00:00 2024-09-12T01:50:13.106 INFO:teuthology.orchestra.run.smithi016.stdout:(34/156): ceph-mgr-diskprediction-local-19.1.1- 14 MB/s | 7.4 MB 00:00 2024-09-12T01:50:13.131 INFO:teuthology.orchestra.run.smithi016.stdout:(35/156): ceph-prometheus-alerts-19.1.1-298.gb5 131 kB/s | 16 kB 00:00 2024-09-12T01:50:13.134 INFO:teuthology.orchestra.run.smithi184.stdout:(9/156): ceph-selinux-19.1.1-298.gb53b6d3e.el9. 608 kB/s | 25 kB 00:00 2024-09-12T01:50:13.152 INFO:teuthology.orchestra.run.smithi033.stdout:(33/156): ceph-mgr-rook-19.1.1-298.gb53b6d3e.el 986 kB/s | 49 kB 00:00 2024-09-12T01:50:13.165 INFO:teuthology.orchestra.run.smithi016.stdout:(36/156): ceph-volume-19.1.1-298.gb53b6d3e.el9. 4.8 MB/s | 285 kB 00:00 2024-09-12T01:50:13.210 INFO:teuthology.orchestra.run.smithi033.stdout:(34/156): ceph-prometheus-alerts-19.1.1-298.gb5 282 kB/s | 16 kB 00:00 2024-09-12T01:50:13.232 INFO:teuthology.orchestra.run.smithi016.stdout:(37/156): cephfs-top-19.1.1-298.gb53b6d3e.el9.n 274 kB/s | 18 kB 00:00 2024-09-12T01:50:13.277 INFO:teuthology.orchestra.run.smithi033.stdout:(35/156): ceph-volume-19.1.1-298.gb53b6d3e.el9. 4.2 MB/s | 285 kB 00:00 2024-09-12T01:50:13.308 INFO:teuthology.orchestra.run.smithi016.stdout:(38/156): cephadm-19.1.1-298.gb53b6d3e.el9.noar 4.3 MB/s | 765 kB 00:00 2024-09-12T01:50:13.353 INFO:teuthology.orchestra.run.smithi033.stdout:(36/156): cephadm-19.1.1-298.gb53b6d3e.el9.noar 10 MB/s | 765 kB 00:00 2024-09-12T01:50:13.403 INFO:teuthology.orchestra.run.smithi033.stdout:(37/156): cephfs-top-19.1.1-298.gb53b6d3e.el9.n 366 kB/s | 18 kB 00:00 2024-09-12T01:50:13.537 INFO:teuthology.orchestra.run.smithi033.stdout:(38/156): fuse-2.9.9-16.el9.x86_64.rpm 599 kB/s | 80 kB 00:00 2024-09-12T01:50:13.762 INFO:teuthology.orchestra.run.smithi033.stdout:(39/156): libgfortran-11.5.0-2.el9.x86_64.rpm 3.5 MB/s | 801 kB 00:00 2024-09-12T01:50:13.845 INFO:teuthology.orchestra.run.smithi033.stdout:(40/156): libquadmath-11.5.0-2.el9.x86_64.rpm 2.2 MB/s | 191 kB 00:00 2024-09-12T01:50:14.004 INFO:teuthology.orchestra.run.smithi033.stdout:(41/156): ceph-mgr-diskprediction-local-19.1.1- 6.8 MB/s | 7.4 MB 00:01 2024-09-12T01:50:14.029 INFO:teuthology.orchestra.run.smithi033.stdout:(42/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.3 MB/s | 253 kB 00:00 2024-09-12T01:50:14.080 INFO:teuthology.orchestra.run.smithi033.stdout:(43/156): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-09-12T01:50:14.155 INFO:teuthology.orchestra.run.smithi033.stdout:(44/156): python3-pycparser-2.20-6.el9.noarch.r 1.8 MB/s | 135 kB 00:00 2024-09-12T01:50:14.200 INFO:teuthology.orchestra.run.smithi016.stdout:(39/156): ceph-test-19.1.1-298.gb53b6d3e.el9.x8 11 MB/s | 49 MB 00:04 2024-09-12T01:50:14.225 INFO:teuthology.orchestra.run.smithi016.stdout:(40/156): fuse-2.9.9-16.el9.x86_64.rpm 80 kB/s | 80 kB 00:00 2024-09-12T01:50:14.239 INFO:teuthology.orchestra.run.smithi033.stdout:(45/156): python3-requests-2.25.1-8.el9.noarch. 1.5 MB/s | 125 kB 00:00 2024-09-12T01:50:14.322 INFO:teuthology.orchestra.run.smithi033.stdout:(46/156): python3-urllib3-1.26.5-6.el9.noarch.r 2.5 MB/s | 215 kB 00:00 2024-09-12T01:50:14.364 INFO:teuthology.orchestra.run.smithi033.stdout:(47/156): python3-cryptography-36.0.1-4.el9.x86 3.4 MB/s | 1.2 MB 00:00 2024-09-12T01:50:14.606 INFO:teuthology.orchestra.run.smithi033.stdout:(48/156): automake-1.16.2-8.el9.noarch.rpm 2.4 MB/s | 689 kB 00:00 2024-09-12T01:50:14.665 INFO:teuthology.orchestra.run.smithi033.stdout:(49/156): bison-3.7.4-5.el9.x86_64.rpm 3.1 MB/s | 943 kB 00:00 2024-09-12T01:50:14.690 INFO:teuthology.orchestra.run.smithi033.stdout:(50/156): boost-program-options-1.75.0-8.el9.x8 1.2 MB/s | 107 kB 00:00 2024-09-12T01:50:14.715 INFO:teuthology.orchestra.run.smithi033.stdout:(51/156): elfutils-libelf-devel-0.191-4.el9.x86 432 kB/s | 22 kB 00:00 2024-09-12T01:50:14.765 INFO:teuthology.orchestra.run.smithi033.stdout:(52/156): flexiblas-3.0.4-8.el9.x86_64.rpm 656 kB/s | 33 kB 00:00 2024-09-12T01:50:14.791 INFO:teuthology.orchestra.run.smithi033.stdout:(53/156): flex-2.6.4-9.el9.x86_64.rpm 3.0 MB/s | 313 kB 00:00 2024-09-12T01:50:14.841 INFO:teuthology.orchestra.run.smithi033.stdout:(54/156): flexiblas-openblas-openmp-3.0.4-8.el9 364 kB/s | 18 kB 00:00 2024-09-12T01:50:14.891 INFO:teuthology.orchestra.run.smithi033.stdout:(55/156): libaio-devel-0.3.111-13.el9.x86_64.rp 223 kB/s | 11 kB 00:00 2024-09-12T01:50:14.959 INFO:teuthology.orchestra.run.smithi033.stdout:(56/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.3 MB/s | 160 kB 00:00 2024-09-12T01:50:14.984 INFO:teuthology.orchestra.run.smithi016.stdout:(41/156): libquadmath-11.5.0-2.el9.x86_64.rpm 244 kB/s | 191 kB 00:00 2024-09-12T01:50:15.017 INFO:teuthology.orchestra.run.smithi033.stdout:(57/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 777 kB/s | 45 kB 00:00 2024-09-12T01:50:15.076 INFO:teuthology.orchestra.run.smithi184.stdout:(10/156): ceph-radosgw-19.1.1-298.gb53b6d3e.el9 3.8 MB/s | 11 MB 00:02 2024-09-12T01:50:15.101 INFO:teuthology.orchestra.run.smithi016.stdout:(42/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 289 kB/s | 253 kB 00:00 2024-09-12T01:50:15.293 INFO:teuthology.orchestra.run.smithi184.stdout:(11/156): ceph-common-19.1.1-298.gb53b6d3e.el9. 3.3 MB/s | 21 MB 00:06 2024-09-12T01:50:15.318 INFO:teuthology.orchestra.run.smithi184.stdout:(12/156): cephfs-mirror-19.1.1-298.gb53b6d3e.el 945 kB/s | 228 kB 00:00 2024-09-12T01:50:15.344 INFO:teuthology.orchestra.run.smithi184.stdout:(13/156): cephfs-shell-19.1.1-298.gb53b6d3e.el9 540 kB/s | 27 kB 00:00 2024-09-12T01:50:15.369 INFO:teuthology.orchestra.run.smithi184.stdout:(14/156): libcephfs-devel-19.1.1-298.gb53b6d3e. 630 kB/s | 32 kB 00:00 2024-09-12T01:50:15.436 INFO:teuthology.orchestra.run.smithi184.stdout:(15/156): libcephsqlite-19.1.1-298.gb53b6d3e.el 2.4 MB/s | 163 kB 00:00 2024-09-12T01:50:15.459 INFO:teuthology.orchestra.run.smithi016.stdout:(43/156): libgfortran-11.5.0-2.el9.x86_64.rpm 372 kB/s | 801 kB 00:02 2024-09-12T01:50:15.476 INFO:teuthology.orchestra.run.smithi033.stdout:(58/156): ceph-test-19.1.1-298.gb53b6d3e.el9.x8 11 MB/s | 49 MB 00:04 2024-09-12T01:50:15.485 INFO:teuthology.orchestra.run.smithi184.stdout:(16/156): libcephfs2-19.1.1-298.gb53b6d3e.el9.x 5.2 MB/s | 752 kB 00:00 2024-09-12T01:50:15.503 INFO:teuthology.orchestra.run.smithi184.stdout:(17/156): librados-devel-19.1.1-298.gb53b6d3e.e 1.9 MB/s | 127 kB 00:00 2024-09-12T01:50:15.509 INFO:teuthology.orchestra.run.smithi033.stdout:(59/156): librdkafka-1.6.1-102.el9.x86_64.rpm 1.3 MB/s | 662 kB 00:00 2024-09-12T01:50:15.551 INFO:teuthology.orchestra.run.smithi033.stdout:(60/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 3.8 MB/s | 3.0 MB 00:00 2024-09-12T01:50:15.577 INFO:teuthology.orchestra.run.smithi033.stdout:(61/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 367 kB/s | 24 kB 00:00 2024-09-12T01:50:15.586 INFO:teuthology.orchestra.run.smithi184.stdout:(18/156): libradosstriper1-19.1.1-298.gb53b6d3e 4.9 MB/s | 506 kB 00:00 2024-09-12T01:50:15.602 INFO:teuthology.orchestra.run.smithi033.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 948 kB/s | 47 kB 00:00 2024-09-12T01:50:15.659 INFO:teuthology.orchestra.run.smithi184.stdout:(19/156): python3-ceph-argparse-19.1.1-298.gb53 626 kB/s | 45 kB 00:00 2024-09-12T01:50:15.660 INFO:teuthology.orchestra.run.smithi033.stdout:(63/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.4 MB/s | 292 kB 00:00 2024-09-12T01:50:15.686 INFO:teuthology.orchestra.run.smithi033.stdout:(64/156): lua-5.4.4-4.el9.x86_64.rpm 2.2 MB/s | 188 kB 00:00 2024-09-12T01:50:15.719 INFO:teuthology.orchestra.run.smithi033.stdout:(65/156): openblas-0.3.26-2.el9.x86_64.rpm 660 kB/s | 38 kB 00:00 2024-09-12T01:50:15.741 INFO:teuthology.orchestra.run.smithi184.stdout:(20/156): python3-ceph-common-19.1.1-298.gb53b6 1.6 MB/s | 136 kB 00:00 2024-09-12T01:50:15.776 INFO:teuthology.orchestra.run.smithi016.stdout:(44/156): python3-ply-3.11-14.el9.noarch.rpm 158 kB/s | 106 kB 00:00 2024-09-12T01:50:15.778 INFO:teuthology.orchestra.run.smithi033.stdout:(66/156): libtool-2.4.6-46.el9.x86_64.rpm 1.9 MB/s | 581 kB 00:00 2024-09-12T01:50:15.795 INFO:teuthology.orchestra.run.smithi184.stdout:(21/156): python3-cephfs-19.1.1-298.gb53b6d3e.e 3.0 MB/s | 162 kB 00:00 2024-09-12T01:50:15.862 INFO:teuthology.orchestra.run.smithi184.stdout:(22/156): python3-rados-19.1.1-298.gb53b6d3e.el 4.7 MB/s | 321 kB 00:00 2024-09-12T01:50:15.929 INFO:teuthology.orchestra.run.smithi184.stdout:(23/156): python3-rbd-19.1.1-298.gb53b6d3e.el9. 4.4 MB/s | 301 kB 00:00 2024-09-12T01:50:15.980 INFO:teuthology.orchestra.run.smithi184.stdout:(24/156): python3-rgw-19.1.1-298.gb53b6d3e.el9. 1.9 MB/s | 100 kB 00:00 2024-09-12T01:50:16.020 INFO:teuthology.orchestra.run.smithi033.stdout:(67/156): postgresql-13.16-1.el9.x86_64.rpm 6.6 MB/s | 1.6 MB 00:00 2024-09-12T01:50:16.055 INFO:teuthology.orchestra.run.smithi184.stdout:(25/156): rbd-fuse-19.1.1-298.gb53b6d3e.el9.x86 1.1 MB/s | 85 kB 00:00 2024-09-12T01:50:16.162 INFO:teuthology.orchestra.run.smithi033.stdout:(68/156): postgresql-contrib-13.16-1.el9.x86_64 6.3 MB/s | 910 kB 00:00 2024-09-12T01:50:16.202 INFO:teuthology.orchestra.run.smithi016.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 182 kB/s | 135 kB 00:00 2024-09-12T01:50:16.254 INFO:teuthology.orchestra.run.smithi033.stdout:(69/156): openblas-openmp-0.3.26-2.el9.x86_64.r 8.7 MB/s | 4.9 MB 00:00 2024-09-12T01:50:16.279 INFO:teuthology.orchestra.run.smithi033.stdout:(70/156): postgresql-private-libs-13.16-1.el9.x 1.1 MB/s | 137 kB 00:00 2024-09-12T01:50:16.338 INFO:teuthology.orchestra.run.smithi033.stdout:(71/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 7.2 MB/s | 4.4 MB 00:00 2024-09-12T01:50:16.396 INFO:teuthology.orchestra.run.smithi033.stdout:(72/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-09-12T01:50:16.463 INFO:teuthology.orchestra.run.smithi033.stdout:(73/156): protobuf-3.14.0-13.el9.x86_64.rpm 5.5 MB/s | 1.0 MB 00:00 2024-09-12T01:50:16.468 INFO:teuthology.orchestra.run.smithi016.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 181 kB/s | 125 kB 00:00 2024-09-12T01:50:16.480 INFO:teuthology.orchestra.run.smithi184.stdout:(26/156): ceph-test-19.1.1-298.gb53b6d3e.el9.x8 15 MB/s | 49 MB 00:03 2024-09-12T01:50:16.522 INFO:teuthology.orchestra.run.smithi033.stdout:(74/156): python3-colorama-0.4.6-3.el9.noarch.r 1.0 MB/s | 61 kB 00:00 2024-09-12T01:50:16.547 INFO:teuthology.orchestra.run.smithi184.stdout:(27/156): rbd-nbd-19.1.1-298.gb53b6d3e.el9.x86_ 2.5 MB/s | 171 kB 00:00 2024-09-12T01:50:16.577 INFO:teuthology.orchestra.run.smithi016.stdout:(47/156): python3-cryptography-36.0.1-4.el9.x86 788 kB/s | 1.2 MB 00:01 2024-09-12T01:50:16.585 INFO:teuthology.orchestra.run.smithi033.stdout:(75/156): python3-devel-3.9.19-8.el9.x86_64.rpm 3.8 MB/s | 245 kB 00:00 2024-09-12T01:50:16.597 INFO:teuthology.orchestra.run.smithi184.stdout:(28/156): ceph-grafana-dashboards-19.1.1-298.gb 640 kB/s | 32 kB 00:00 2024-09-12T01:50:16.656 INFO:teuthology.orchestra.run.smithi033.stdout:(76/156): python3-jinja2-2.11.3-6.el9.noarch.rp 3.4 MB/s | 248 kB 00:00 2024-09-12T01:50:16.681 INFO:teuthology.orchestra.run.smithi184.stdout:(29/156): librgw2-19.1.1-298.gb53b6d3e.el9.x86_ 4.5 MB/s | 5.3 MB 00:01 2024-09-12T01:50:16.706 INFO:teuthology.orchestra.run.smithi184.stdout:(30/156): ceph-mgr-cephadm-19.1.1-298.gb53b6d3e 1.3 MB/s | 146 kB 00:00 2024-09-12T01:50:16.731 INFO:teuthology.orchestra.run.smithi033.stdout:(77/156): postgresql-server-13.16-1.el9.x86_64. 12 MB/s | 5.8 MB 00:00 2024-09-12T01:50:16.756 INFO:teuthology.orchestra.run.smithi033.stdout:(78/156): python3-jsonpatch-1.21-16.el9.noarch. 256 kB/s | 26 kB 00:00 2024-09-12T01:50:16.777 INFO:teuthology.orchestra.run.smithi016.stdout:(48/156): automake-1.16.2-8.el9.noarch.rpm 2.2 MB/s | 689 kB 00:00 2024-09-12T01:50:16.781 INFO:teuthology.orchestra.run.smithi184.stdout:(31/156): rbd-mirror-19.1.1-298.gb53b6d3e.el9.x 4.3 MB/s | 3.1 MB 00:00 2024-09-12T01:50:16.789 INFO:teuthology.orchestra.run.smithi033.stdout:(79/156): python3-jsonpointer-2.0-4.el9.noarch. 328 kB/s | 19 kB 00:00 2024-09-12T01:50:16.815 INFO:teuthology.orchestra.run.smithi033.stdout:(80/156): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 172 kB 00:00 2024-09-12T01:50:16.836 INFO:teuthology.orchestra.run.smithi016.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 1.8 MB/s | 107 kB 00:00 2024-09-12T01:50:16.848 INFO:teuthology.orchestra.run.smithi033.stdout:(81/156): python3-markupsafe-1.1.1-12.el9.x86_6 596 kB/s | 35 kB 00:00 2024-09-12T01:50:16.848 INFO:teuthology.orchestra.run.smithi184.stdout:(32/156): ceph-mgr-modules-core-19.1.1-298.gb53 3.7 MB/s | 249 kB 00:00 2024-09-12T01:50:16.895 INFO:teuthology.orchestra.run.smithi016.stdout:(50/156): bison-3.7.4-5.el9.x86_64.rpm 2.9 MB/s | 943 kB 00:00 2024-09-12T01:50:16.898 INFO:teuthology.orchestra.run.smithi184.stdout:(33/156): ceph-mgr-rook-19.1.1-298.gb53b6d3e.el 989 kB/s | 49 kB 00:00 2024-09-12T01:50:16.920 INFO:teuthology.orchestra.run.smithi016.stdout:(51/156): elfutils-libelf-devel-0.191-4.el9.x86 259 kB/s | 22 kB 00:00 2024-09-12T01:50:16.923 INFO:teuthology.orchestra.run.smithi033.stdout:(82/156): python3-babel-2.9.1-2.el9.noarch.rpm 11 MB/s | 6.0 MB 00:00 2024-09-12T01:50:16.949 INFO:teuthology.orchestra.run.smithi033.stdout:(83/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.4 MB/s | 250 kB 00:00 2024-09-12T01:50:16.949 INFO:teuthology.orchestra.run.smithi184.stdout:(34/156): ceph-prometheus-alerts-19.1.1-298.gb5 329 kB/s | 16 kB 00:00 2024-09-12T01:50:16.953 INFO:teuthology.orchestra.run.smithi016.stdout:(52/156): python3-urllib3-1.26.5-6.el9.noarch.r 286 kB/s | 215 kB 00:00 2024-09-12T01:50:16.970 INFO:teuthology.orchestra.run.smithi016.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 651 kB/s | 33 kB 00:00 2024-09-12T01:50:16.996 INFO:teuthology.orchestra.run.smithi016.stdout:(54/156): flex-2.6.4-9.el9.x86_64.rpm 3.0 MB/s | 313 kB 00:00 2024-09-12T01:50:16.999 INFO:teuthology.orchestra.run.smithi033.stdout:(84/156): python3-oauthlib-3.1.1-5.el9.noarch.r 2.9 MB/s | 222 kB 00:00 2024-09-12T01:50:17.013 INFO:teuthology.orchestra.run.smithi016.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 435 kB/s | 18 kB 00:00 2024-09-12T01:50:17.024 INFO:teuthology.orchestra.run.smithi033.stdout:(85/156): python3-packaging-20.9-5.el9.noarch.r 1.0 MB/s | 77 kB 00:00 2024-09-12T01:50:17.038 INFO:teuthology.orchestra.run.smithi016.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 265 kB/s | 11 kB 00:00 2024-09-12T01:50:17.050 INFO:teuthology.orchestra.run.smithi033.stdout:(86/156): python3-prettytable-0.7.2-27.el9.noar 844 kB/s | 42 kB 00:00 2024-09-12T01:50:17.063 INFO:teuthology.orchestra.run.smithi016.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-09-12T01:50:17.074 INFO:teuthology.orchestra.run.smithi184.stdout:(35/156): ceph-volume-19.1.1-298.gb53b6d3e.el9. 2.2 MB/s | 285 kB 00:00 2024-09-12T01:50:17.083 INFO:teuthology.orchestra.run.smithi033.stdout:(87/156): python3-protobuf-3.14.0-13.el9.noarch 4.5 MB/s | 269 kB 00:00 2024-09-12T01:50:17.088 INFO:teuthology.orchestra.run.smithi016.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 901 kB/s | 45 kB 00:00 2024-09-12T01:50:17.109 INFO:teuthology.orchestra.run.smithi033.stdout:(88/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.6 MB/s | 159 kB 00:00 2024-09-12T01:50:17.142 INFO:teuthology.orchestra.run.smithi033.stdout:(89/156): python3-pyasn1-modules-0.4.8-6.el9.no 4.7 MB/s | 279 kB 00:00 2024-09-12T01:50:17.168 INFO:teuthology.orchestra.run.smithi033.stdout:(90/156): python3-pytz-2021.1-5.el9.noarch.rpm 869 kB/s | 51 kB 00:00 2024-09-12T01:50:17.191 INFO:teuthology.orchestra.run.smithi184.stdout:(36/156): cephadm-19.1.1-298.gb53b6d3e.el9.noar 6.4 MB/s | 765 kB 00:00 2024-09-12T01:50:17.201 INFO:teuthology.orchestra.run.smithi033.stdout:(91/156): python3-requests-oauthlib-1.3.0-12.el 918 kB/s | 54 kB 00:00 2024-09-12T01:50:17.213 INFO:teuthology.orchestra.run.smithi016.stdout:(59/156): librdkafka-1.6.1-102.el9.x86_64.rpm 4.3 MB/s | 662 kB 00:00 2024-09-12T01:50:17.247 INFO:teuthology.orchestra.run.smithi016.stdout:(60/156): libtool-2.4.6-46.el9.x86_64.rpm 3.6 MB/s | 581 kB 00:00 2024-09-12T01:50:17.273 INFO:teuthology.orchestra.run.smithi016.stdout:(61/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 417 kB/s | 24 kB 00:00 2024-09-12T01:50:17.276 INFO:teuthology.orchestra.run.smithi033.stdout:(92/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 11 MB/s | 5.1 MB 00:00 2024-09-12T01:50:17.292 INFO:teuthology.orchestra.run.smithi184.stdout:(37/156): ceph-mgr-diskprediction-local-19.1.1- 13 MB/s | 7.4 MB 00:00 2024-09-12T01:50:17.298 INFO:teuthology.orchestra.run.smithi016.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 949 kB/s | 47 kB 00:00 2024-09-12T01:50:17.301 INFO:teuthology.orchestra.run.smithi033.stdout:(93/156): python3-toml-0.10.2-6.el9.noarch.rpm 415 kB/s | 42 kB 00:00 2024-09-12T01:50:17.317 INFO:teuthology.orchestra.run.smithi184.stdout:(38/156): cephfs-top-19.1.1-298.gb53b6d3e.el9.n 145 kB/s | 18 kB 00:00 2024-09-12T01:50:17.335 INFO:teuthology.orchestra.run.smithi033.stdout:(94/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 756 kB/s | 44 kB 00:00 2024-09-12T01:50:17.348 INFO:teuthology.orchestra.run.smithi016.stdout:(63/156): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 188 kB 00:00 2024-09-12T01:50:17.369 INFO:teuthology.orchestra.run.smithi033.stdout:(95/156): qatlib-24.02.0-1.el9.x86_64.rpm 3.2 MB/s | 221 kB 00:00 2024-09-12T01:50:17.373 INFO:teuthology.orchestra.run.smithi016.stdout:(64/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.8 MB/s | 292 kB 00:00 2024-09-12T01:50:17.375 INFO:teuthology.orchestra.run.smithi184.stdout:(39/156): ceph-mgr-dashboard-19.1.1-298.gb53b6d 5.1 MB/s | 3.6 MB 00:00 2024-09-12T01:50:17.394 INFO:teuthology.orchestra.run.smithi033.stdout:(96/156): qatlib-service-24.02.0-1.el9.x86_64.r 614 kB/s | 36 kB 00:00 2024-09-12T01:50:17.399 INFO:teuthology.orchestra.run.smithi016.stdout:(65/156): openblas-0.3.26-2.el9.x86_64.rpm 767 kB/s | 38 kB 00:00 2024-09-12T01:50:17.428 INFO:teuthology.orchestra.run.smithi033.stdout:(97/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 811 kB/s | 47 kB 00:00 2024-09-12T01:50:17.443 INFO:teuthology.orchestra.run.smithi184.stdout:(40/156): fuse-2.9.9-16.el9.x86_64.rpm 531 kB/s | 80 kB 00:00 2024-09-12T01:50:17.453 INFO:teuthology.orchestra.run.smithi033.stdout:(98/156): socat-1.7.4.1-6.el9.x86_64.rpm 5.1 MB/s | 304 kB 00:00 2024-09-12T01:50:17.487 INFO:teuthology.orchestra.run.smithi033.stdout:(99/156): uuid-1.6.2-55.el9.x86_64.rpm 988 kB/s | 58 kB 00:00 2024-09-12T01:50:17.512 INFO:teuthology.orchestra.run.smithi033.stdout:(100/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.6 MB/s | 96 kB 00:00 2024-09-12T01:50:17.518 INFO:teuthology.orchestra.run.smithi184.stdout:(41/156): libquadmath-11.5.0-2.el9.x86_64.rpm 1.3 MB/s | 191 kB 00:00 2024-09-12T01:50:17.543 INFO:teuthology.orchestra.run.smithi184.stdout:(42/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.5 MB/s | 253 kB 00:00 2024-09-12T01:50:17.545 INFO:teuthology.orchestra.run.smithi033.stdout:(101/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-09-12T01:50:17.571 INFO:teuthology.orchestra.run.smithi033.stdout:(102/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 764 kB/s | 45 kB 00:00 2024-09-12T01:50:17.585 INFO:teuthology.orchestra.run.smithi184.stdout:(43/156): libgfortran-11.5.0-2.el9.x86_64.rpm 2.9 MB/s | 801 kB 00:00 2024-09-12T01:50:17.610 INFO:teuthology.orchestra.run.smithi184.stdout:(44/156): python3-ply-3.11-14.el9.noarch.rpm 1.6 MB/s | 106 kB 00:00 2024-09-12T01:50:17.614 INFO:teuthology.orchestra.run.smithi033.stdout:(103/156): lua-devel-5.4.4-4.el9.x86_64.rpm 327 kB/s | 22 kB 00:00 2024-09-12T01:50:17.649 INFO:teuthology.orchestra.run.smithi016.stdout:(66/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 4.3 MB/s | 3.0 MB 00:00 2024-09-12T01:50:17.661 INFO:teuthology.orchestra.run.smithi184.stdout:(45/156): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-09-12T01:50:17.679 INFO:teuthology.orchestra.run.smithi033.stdout:(104/156): protobuf-compiler-3.14.0-13.el9.x86_ 7.8 MB/s | 863 kB 00:00 2024-09-12T01:50:17.686 INFO:teuthology.orchestra.run.smithi184.stdout:(46/156): python3-pycparser-2.20-6.el9.noarch.r 1.3 MB/s | 135 kB 00:00 2024-09-12T01:50:17.713 INFO:teuthology.orchestra.run.smithi033.stdout:(105/156): abseil-cpp-20211102.0-3.el9.x86_64.r 5.4 MB/s | 548 kB 00:00 2024-09-12T01:50:17.719 INFO:teuthology.orchestra.run.smithi184.stdout:(47/156): python3-cryptography-36.0.1-4.el9.x86 6.1 MB/s | 1.2 MB 00:00 2024-09-12T01:50:17.738 INFO:teuthology.orchestra.run.smithi033.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 850 kB/s | 19 kB 00:00 2024-09-12T01:50:17.745 INFO:teuthology.orchestra.run.smithi184.stdout:(48/156): python3-urllib3-1.26.5-6.el9.noarch.r 2.5 MB/s | 215 kB 00:00 2024-09-12T01:50:17.772 INFO:teuthology.orchestra.run.smithi033.stdout:(107/156): gperftools-libs-2.9.1-2.el9.x86_64.r 3.3 MB/s | 309 kB 00:00 2024-09-12T01:50:17.797 INFO:teuthology.orchestra.run.smithi033.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-09-12T01:50:17.823 INFO:teuthology.orchestra.run.smithi033.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-12T01:50:17.848 INFO:teuthology.orchestra.run.smithi033.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2024-09-12T01:50:17.873 INFO:teuthology.orchestra.run.smithi033.stdout:(111/156): luarocks-3.9.2-5.el9.noarch.rpm 6.0 MB/s | 151 kB 00:00 2024-09-12T01:50:17.878 INFO:teuthology.orchestra.run.smithi184.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 802 kB/s | 107 kB 00:00 2024-09-12T01:50:17.920 INFO:teuthology.orchestra.run.smithi184.stdout:(50/156): automake-1.16.2-8.el9.noarch.rpm 2.9 MB/s | 689 kB 00:00 2024-09-12T01:50:17.923 INFO:teuthology.orchestra.run.smithi033.stdout:(112/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 16 MB/s | 839 kB 00:00 2024-09-12T01:50:17.937 INFO:teuthology.orchestra.run.smithi184.stdout:(51/156): elfutils-libelf-devel-0.191-4.el9.x86 370 kB/s | 22 kB 00:00 2024-09-12T01:50:17.957 INFO:teuthology.orchestra.run.smithi033.stdout:(113/156): python3-asyncssh-2.13.2-5.el9.noarch 16 MB/s | 548 kB 00:00 2024-09-12T01:50:17.966 INFO:teuthology.orchestra.run.smithi016.stdout:(67/156): postgresql-13.16-1.el9.x86_64.rpm 5.0 MB/s | 1.6 MB 00:00 2024-09-12T01:50:17.979 INFO:teuthology.orchestra.run.smithi184.stdout:(52/156): bison-3.7.4-5.el9.x86_64.rpm 3.6 MB/s | 943 kB 00:00 2024-09-12T01:50:17.982 INFO:teuthology.orchestra.run.smithi033.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-12T01:50:18.004 INFO:teuthology.orchestra.run.smithi184.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 489 kB/s | 33 kB 00:00 2024-09-12T01:50:18.016 INFO:teuthology.orchestra.run.smithi033.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 974 kB/s | 32 kB 00:00 2024-09-12T01:50:18.030 INFO:teuthology.orchestra.run.smithi184.stdout:(54/156): flex-2.6.4-9.el9.x86_64.rpm 2.8 MB/s | 313 kB 00:00 2024-09-12T01:50:18.041 INFO:teuthology.orchestra.run.smithi033.stdout:(116/156): python3-certifi-2023.05.07-4.el9.noa 562 kB/s | 14 kB 00:00 2024-09-12T01:50:18.055 INFO:teuthology.orchestra.run.smithi184.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 363 kB/s | 18 kB 00:00 2024-09-12T01:50:18.080 INFO:teuthology.orchestra.run.smithi184.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 222 kB/s | 11 kB 00:00 2024-09-12T01:50:18.105 INFO:teuthology.orchestra.run.smithi184.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-09-12T01:50:18.131 INFO:teuthology.orchestra.run.smithi184.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 901 kB/s | 45 kB 00:00 2024-09-12T01:50:18.133 INFO:teuthology.orchestra.run.smithi016.stdout:(68/156): postgresql-contrib-13.16-1.el9.x86_64 5.3 MB/s | 910 kB 00:00 2024-09-12T01:50:18.189 INFO:teuthology.orchestra.run.smithi184.stdout:(59/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 14 MB/s | 3.0 MB 00:00 2024-09-12T01:50:18.192 INFO:teuthology.orchestra.run.smithi016.stdout:(69/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 5.6 MB/s | 4.4 MB 00:00 2024-09-12T01:50:18.214 INFO:teuthology.orchestra.run.smithi184.stdout:(60/156): librdkafka-1.6.1-102.el9.x86_64.rpm 6.0 MB/s | 662 kB 00:00 2024-09-12T01:50:18.217 INFO:teuthology.orchestra.run.smithi033.stdout:(117/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 15 MB/s | 16 MB 00:01 2024-09-12T01:50:18.217 INFO:teuthology.orchestra.run.smithi016.stdout:(70/156): postgresql-private-libs-13.16-1.el9.x 1.6 MB/s | 137 kB 00:00 2024-09-12T01:50:18.240 INFO:teuthology.orchestra.run.smithi184.stdout:(61/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 488 kB/s | 24 kB 00:00 2024-09-12T01:50:18.251 INFO:teuthology.orchestra.run.smithi033.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 822 kB/s | 172 kB 00:00 2024-09-12T01:50:18.265 INFO:teuthology.orchestra.run.smithi184.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 945 kB/s | 47 kB 00:00 2024-09-12T01:50:18.284 INFO:teuthology.orchestra.run.smithi033.stdout:(119/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 9.5 MB/s | 303 kB 00:00 2024-09-12T01:50:18.290 INFO:teuthology.orchestra.run.smithi184.stdout:(63/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-09-12T01:50:18.301 INFO:teuthology.orchestra.run.smithi016.stdout:(71/156): openblas-openmp-0.3.26-2.el9.x86_64.r 5.3 MB/s | 4.9 MB 00:00 2024-09-12T01:50:18.315 INFO:teuthology.orchestra.run.smithi184.stdout:(64/156): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 188 kB 00:00 2024-09-12T01:50:18.318 INFO:teuthology.orchestra.run.smithi033.stdout:(120/156): python3-google-auth-2.34.0-1.el9.noa 6.5 MB/s | 220 kB 00:00 2024-09-12T01:50:18.341 INFO:teuthology.orchestra.run.smithi184.stdout:(65/156): openblas-0.3.26-2.el9.x86_64.rpm 768 kB/s | 38 kB 00:00 2024-09-12T01:50:18.351 INFO:teuthology.orchestra.run.smithi016.stdout:(72/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.7 MB/s | 87 kB 00:00 2024-09-12T01:50:18.351 INFO:teuthology.orchestra.run.smithi033.stdout:(121/156): python3-cherrypy-18.6.1-2.el9.noarch 2.6 MB/s | 358 kB 00:00 2024-09-12T01:50:18.366 INFO:teuthology.orchestra.run.smithi184.stdout:(66/156): libtool-2.4.6-46.el9.x86_64.rpm 2.4 MB/s | 581 kB 00:00 2024-09-12T01:50:18.385 INFO:teuthology.orchestra.run.smithi033.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 4.2 MB/s | 144 kB 00:00 2024-09-12T01:50:18.410 INFO:teuthology.orchestra.run.smithi033.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 430 kB/s | 11 kB 00:00 2024-09-12T01:50:18.426 INFO:teuthology.orchestra.run.smithi016.stdout:(73/156): protobuf-3.14.0-13.el9.x86_64.rpm 4.8 MB/s | 1.0 MB 00:00 2024-09-12T01:50:18.435 INFO:teuthology.orchestra.run.smithi033.stdout:(124/156): python3-jaraco-classes-3.2.1-5.el9.n 712 kB/s | 18 kB 00:00 2024-09-12T01:50:18.450 INFO:teuthology.orchestra.run.smithi184.stdout:(67/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 41 MB/s | 4.4 MB 00:00 2024-09-12T01:50:18.485 INFO:teuthology.orchestra.run.smithi016.stdout:(74/156): python3-colorama-0.4.6-3.el9.noarch.r 1.0 MB/s | 61 kB 00:00 2024-09-12T01:50:18.500 INFO:teuthology.orchestra.run.smithi184.stdout:(68/156): postgresql-13.16-1.el9.x86_64.rpm 12 MB/s | 1.6 MB 00:00 2024-09-12T01:50:18.502 INFO:teuthology.orchestra.run.smithi033.stdout:(125/156): libarrow-9.0.0-11.el9.x86_64.rpm 5.8 MB/s | 4.4 MB 00:00 2024-09-12T01:50:18.528 INFO:teuthology.orchestra.run.smithi033.stdout:(126/156): python3-jaraco-collections-3.0.0-8.e 252 kB/s | 23 kB 00:00 2024-09-12T01:50:18.533 INFO:teuthology.orchestra.run.smithi184.stdout:(69/156): postgresql-contrib-13.16-1.el9.x86_64 11 MB/s | 910 kB 00:00 2024-09-12T01:50:18.553 INFO:teuthology.orchestra.run.smithi033.stdout:(127/156): python3-jaraco-functools-3.5.0-2.el9 388 kB/s | 19 kB 00:00 2024-09-12T01:50:18.559 INFO:teuthology.orchestra.run.smithi184.stdout:(70/156): postgresql-private-libs-13.16-1.el9.x 2.3 MB/s | 137 kB 00:00 2024-09-12T01:50:18.569 INFO:teuthology.orchestra.run.smithi016.stdout:(75/156): python3-devel-3.9.19-8.el9.x86_64.rpm 2.9 MB/s | 245 kB 00:00 2024-09-12T01:50:18.578 INFO:teuthology.orchestra.run.smithi033.stdout:(128/156): python3-jaraco-text-3.2.0-6.el9.noar 392 kB/s | 20 kB 00:00 2024-09-12T01:50:18.604 INFO:teuthology.orchestra.run.smithi033.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-12T01:50:18.626 INFO:teuthology.orchestra.run.smithi184.stdout:(71/156): openblas-openmp-0.3.26-2.el9.x86_64.r 16 MB/s | 4.9 MB 00:00 2024-09-12T01:50:18.646 INFO:teuthology.orchestra.run.smithi033.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 1.8 MB/s | 79 kB 00:00 2024-09-12T01:50:18.652 INFO:teuthology.orchestra.run.smithi016.stdout:(76/156): python3-jinja2-2.11.3-6.el9.noarch.rp 2.9 MB/s | 248 kB 00:00 2024-09-12T01:50:18.662 INFO:teuthology.orchestra.run.smithi033.stdout:(131/156): python3-natsort-7.1.1-5.el9.noarch.r 3.6 MB/s | 58 kB 00:00 2024-09-12T01:50:18.668 INFO:teuthology.orchestra.run.smithi184.stdout:(72/156): protobuf-3.14.0-13.el9.x86_64.rpm 9.3 MB/s | 1.0 MB 00:00 2024-09-12T01:50:18.693 INFO:teuthology.orchestra.run.smithi184.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 87 kB 00:00 2024-09-12T01:50:18.703 INFO:teuthology.orchestra.run.smithi016.stdout:(77/156): python3-jsonpatch-1.21-16.el9.noarch. 513 kB/s | 26 kB 00:00 2024-09-12T01:50:18.704 INFO:teuthology.orchestra.run.smithi033.stdout:(132/156): python3-pecan-1.4.2-3.el9.noarch.rpm 6.4 MB/s | 272 kB 00:00 2024-09-12T01:50:18.721 INFO:teuthology.orchestra.run.smithi033.stdout:(133/156): python3-portend-3.1.0-2.el9.noarch.r 992 kB/s | 16 kB 00:00 2024-09-12T01:50:18.735 INFO:teuthology.orchestra.run.smithi184.stdout:(74/156): python3-colorama-0.4.6-3.el9.noarch.r 1.4 MB/s | 61 kB 00:00 2024-09-12T01:50:18.744 INFO:teuthology.orchestra.run.smithi016.stdout:(78/156): python3-jsonpointer-2.0-4.el9.noarch. 460 kB/s | 19 kB 00:00 2024-09-12T01:50:18.746 INFO:teuthology.orchestra.run.smithi033.stdout:(134/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 3.6 MB/s | 90 kB 00:00 2024-09-12T01:50:18.764 INFO:teuthology.orchestra.run.smithi033.stdout:(135/156): python3-pyperclip-1.8.2-2.el9.noarch 1.7 MB/s | 28 kB 00:00 2024-09-12T01:50:18.797 INFO:teuthology.orchestra.run.smithi033.stdout:(136/156): python3-kubernetes-26.1.0-2.el9.noar 4.2 MB/s | 1.0 MB 00:00 2024-09-12T01:50:18.803 INFO:teuthology.orchestra.run.smithi016.stdout:(79/156): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 172 kB 00:00 2024-09-12T01:50:18.810 INFO:teuthology.orchestra.run.smithi184.stdout:(75/156): postgresql-server-13.16-1.el9.x86_64. 21 MB/s | 5.8 MB 00:00 2024-09-12T01:50:18.814 INFO:teuthology.orchestra.run.smithi033.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 613 kB/s | 31 kB 00:00 2024-09-12T01:50:18.835 INFO:teuthology.orchestra.run.smithi184.stdout:(76/156): python3-devel-3.9.19-8.el9.x86_64.rpm 2.4 MB/s | 245 kB 00:00 2024-09-12T01:50:18.853 INFO:teuthology.orchestra.run.smithi016.stdout:(80/156): python3-markupsafe-1.1.1-12.el9.x86_6 694 kB/s | 35 kB 00:00 2024-09-12T01:50:18.861 INFO:teuthology.orchestra.run.smithi184.stdout:(77/156): python3-jinja2-2.11.3-6.el9.noarch.rp 4.8 MB/s | 248 kB 00:00 2024-09-12T01:50:18.866 INFO:teuthology.orchestra.run.smithi033.stdout:(138/156): python3-grpcio-1.46.7-10.el9.x86_64. 3.7 MB/s | 2.0 MB 00:00 2024-09-12T01:50:18.878 INFO:teuthology.orchestra.run.smithi184.stdout:(78/156): python3-jsonpatch-1.21-16.el9.noarch. 612 kB/s | 26 kB 00:00 2024-09-12T01:50:18.889 INFO:teuthology.orchestra.run.smithi033.stdout:(139/156): python3-rsa-4.9-2.el9.noarch.rpm 790 kB/s | 59 kB 00:00 2024-09-12T01:50:18.903 INFO:teuthology.orchestra.run.smithi184.stdout:(79/156): python3-jsonpointer-2.0-4.el9.noarch. 458 kB/s | 19 kB 00:00 2024-09-12T01:50:18.924 INFO:teuthology.orchestra.run.smithi033.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 598 kB/s | 36 kB 00:00 2024-09-12T01:50:18.928 INFO:teuthology.orchestra.run.smithi184.stdout:(80/156): python3-mako-1.1.4-6.el9.noarch.rpm 3.3 MB/s | 172 kB 00:00 2024-09-12T01:50:18.948 INFO:teuthology.orchestra.run.smithi033.stdout:(141/156): python3-routes-2.5.1-5.el9.noarch.rp 1.4 MB/s | 188 kB 00:00 2024-09-12T01:50:18.953 INFO:teuthology.orchestra.run.smithi184.stdout:(81/156): python3-markupsafe-1.1.1-12.el9.x86_6 694 kB/s | 35 kB 00:00 2024-09-12T01:50:18.974 INFO:teuthology.orchestra.run.smithi033.stdout:(142/156): python3-typing-extensions-4.12.2-2.e 887 kB/s | 74 kB 00:00 2024-09-12T01:50:18.999 INFO:teuthology.orchestra.run.smithi033.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-09-12T01:50:19.024 INFO:teuthology.orchestra.run.smithi033.stdout:(144/156): python3-zc-lockfile-2.0-10.el9.noarc 802 kB/s | 20 kB 00:00 2024-09-12T01:50:19.037 INFO:teuthology.orchestra.run.smithi184.stdout:(82/156): python3-babel-2.9.1-2.el9.noarch.rpm 16 MB/s | 6.0 MB 00:00 2024-09-12T01:50:19.045 INFO:teuthology.orchestra.run.smithi016.stdout:(81/156): postgresql-server-13.16-1.el9.x86_64. 6.8 MB/s | 5.8 MB 00:00 2024-09-12T01:50:19.049 INFO:teuthology.orchestra.run.smithi033.stdout:(145/156): python3-webob-1.8.8-2.el9.noarch.rpm 1.8 MB/s | 230 kB 00:00 2024-09-12T01:50:19.062 INFO:teuthology.orchestra.run.smithi184.stdout:(83/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.3 MB/s | 250 kB 00:00 2024-09-12T01:50:19.075 INFO:teuthology.orchestra.run.smithi033.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-12T01:50:19.088 INFO:teuthology.orchestra.run.smithi184.stdout:(84/156): python3-oauthlib-3.1.1-5.el9.noarch.r 4.3 MB/s | 222 kB 00:00 2024-09-12T01:50:19.100 INFO:teuthology.orchestra.run.smithi033.stdout:(147/156): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-09-12T01:50:19.104 INFO:teuthology.orchestra.run.smithi016.stdout:(82/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.2 MB/s | 250 kB 00:00 2024-09-12T01:50:19.138 INFO:teuthology.orchestra.run.smithi184.stdout:(85/156): python3-packaging-20.9-5.el9.noarch.r 1.0 MB/s | 77 kB 00:00 2024-09-12T01:50:19.159 INFO:teuthology.orchestra.run.smithi033.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-09-12T01:50:19.163 INFO:teuthology.orchestra.run.smithi016.stdout:(83/156): python3-oauthlib-3.1.1-5.el9.noarch.r 3.7 MB/s | 222 kB 00:00 2024-09-12T01:50:19.163 INFO:teuthology.orchestra.run.smithi184.stdout:(86/156): python3-prettytable-0.7.2-27.el9.noar 561 kB/s | 42 kB 00:00 2024-09-12T01:50:19.189 INFO:teuthology.orchestra.run.smithi184.stdout:(87/156): python3-protobuf-3.14.0-13.el9.noarch 5.2 MB/s | 269 kB 00:00 2024-09-12T01:50:19.213 INFO:teuthology.orchestra.run.smithi016.stdout:(84/156): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-09-12T01:50:19.214 INFO:teuthology.orchestra.run.smithi184.stdout:(88/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-09-12T01:50:19.239 INFO:teuthology.orchestra.run.smithi184.stdout:(89/156): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-09-12T01:50:19.255 INFO:teuthology.orchestra.run.smithi016.stdout:(85/156): python3-prettytable-0.7.2-27.el9.noar 1.0 MB/s | 42 kB 00:00 2024-09-12T01:50:19.256 INFO:teuthology.orchestra.run.smithi184.stdout:(90/156): python3-pytz-2021.1-5.el9.noarch.rpm 1.2 MB/s | 51 kB 00:00 2024-09-12T01:50:19.273 INFO:teuthology.orchestra.run.smithi184.stdout:(91/156): python3-requests-oauthlib-1.3.0-12.el 1.6 MB/s | 54 kB 00:00 2024-09-12T01:50:19.313 INFO:teuthology.orchestra.run.smithi016.stdout:(86/156): python3-protobuf-3.14.0-13.el9.noarch 4.5 MB/s | 269 kB 00:00 2024-09-12T01:50:19.340 INFO:teuthology.orchestra.run.smithi184.stdout:(92/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 12 MB/s | 5.1 MB 00:00 2024-09-12T01:50:19.356 INFO:teuthology.orchestra.run.smithi184.stdout:(93/156): python3-toml-0.10.2-6.el9.noarch.rpm 499 kB/s | 42 kB 00:00 2024-09-12T01:50:19.359 INFO:teuthology.orchestra.run.smithi033.stdout:(149/156): librados2-19.1.1-298.gb53b6d3e.el9.x 12 MB/s | 3.4 MB 00:00 2024-09-12T01:50:19.374 INFO:teuthology.orchestra.run.smithi184.stdout:(94/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 1.3 MB/s | 44 kB 00:00 2024-09-12T01:50:19.407 INFO:teuthology.orchestra.run.smithi184.stdout:(95/156): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-09-12T01:50:19.422 INFO:teuthology.orchestra.run.smithi016.stdout:(87/156): python3-babel-2.9.1-2.el9.noarch.rpm 5.6 MB/s | 6.0 MB 00:01 2024-09-12T01:50:19.424 INFO:teuthology.orchestra.run.smithi184.stdout:(96/156): qatlib-service-24.02.0-1.el9.x86_64.r 716 kB/s | 36 kB 00:00 2024-09-12T01:50:19.441 INFO:teuthology.orchestra.run.smithi184.stdout:(97/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 1.4 MB/s | 47 kB 00:00 2024-09-12T01:50:19.448 INFO:teuthology.orchestra.run.smithi016.stdout:(88/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.2 MB/s | 159 kB 00:00 2024-09-12T01:50:19.451 INFO:teuthology.orchestra.run.smithi033.stdout:(150/156): librbd1-19.1.1-298.gb53b6d3e.el9.x86 9.0 MB/s | 3.2 MB 00:00 2024-09-12T01:50:19.475 INFO:teuthology.orchestra.run.smithi184.stdout:(98/156): socat-1.7.4.1-6.el9.x86_64.rpm 5.9 MB/s | 304 kB 00:00 2024-09-12T01:50:19.491 INFO:teuthology.orchestra.run.smithi184.stdout:(99/156): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-09-12T01:50:19.498 INFO:teuthology.orchestra.run.smithi016.stdout:(89/156): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-09-12T01:50:19.517 INFO:teuthology.orchestra.run.smithi184.stdout:(100/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 2.3 MB/s | 96 kB 00:00 2024-09-12T01:50:19.523 INFO:teuthology.orchestra.run.smithi016.stdout:(90/156): python3-pyasn1-modules-0.4.8-6.el9.no 2.7 MB/s | 279 kB 00:00 2024-09-12T01:50:19.542 INFO:teuthology.orchestra.run.smithi184.stdout:(101/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-09-12T01:50:19.549 INFO:teuthology.orchestra.run.smithi016.stdout:(91/156): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-09-12T01:50:19.559 INFO:teuthology.orchestra.run.smithi184.stdout:(102/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 1.0 MB/s | 45 kB 00:00 2024-09-12T01:50:19.559 INFO:teuthology.orchestra.run.smithi033.stdout:(151/156): NetworkManager-team-1.49.90-1.el9.x8 377 kB/s | 41 kB 00:00 2024-09-12T01:50:19.599 INFO:teuthology.orchestra.run.smithi016.stdout:(92/156): python3-toml-0.10.2-6.el9.noarch.rpm 835 kB/s | 42 kB 00:00 2024-09-12T01:50:19.657 INFO:teuthology.orchestra.run.smithi016.stdout:(93/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 761 kB/s | 44 kB 00:00 2024-09-12T01:50:19.668 INFO:teuthology.orchestra.run.smithi033.stdout:(152/156): NetworkManager-tui-1.49.90-1.el9.x86 2.2 MB/s | 249 kB 00:00 2024-09-12T01:50:19.716 INFO:teuthology.orchestra.run.smithi016.stdout:(94/156): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 221 kB 00:00 2024-09-12T01:50:19.801 INFO:teuthology.orchestra.run.smithi184.stdout:(103/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 30 MB/s | 16 MB 00:00 2024-09-12T01:50:19.816 INFO:teuthology.orchestra.run.smithi016.stdout:(95/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 5.3 MB/s | 5.1 MB 00:00 2024-09-12T01:50:19.835 INFO:teuthology.orchestra.run.smithi184.stdout:(104/156): lua-devel-5.4.4-4.el9.x86_64.rpm 76 kB/s | 22 kB 00:00 2024-09-12T01:50:19.842 INFO:teuthology.orchestra.run.smithi016.stdout:(96/156): qatlib-service-24.02.0-1.el9.x86_64.r 287 kB/s | 36 kB 00:00 2024-09-12T01:50:19.843 INFO:teuthology.orchestra.run.smithi033.stdout:(153/156): NetworkManager-1.49.90-1.el9.x86_64. 3.4 MB/s | 2.3 MB 00:00 2024-09-12T01:50:19.876 INFO:teuthology.orchestra.run.smithi016.stdout:(97/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 807 kB/s | 47 kB 00:00 2024-09-12T01:50:19.894 INFO:teuthology.orchestra.run.smithi184.stdout:(105/156): abseil-cpp-20211102.0-3.el9.x86_64.r 5.8 MB/s | 548 kB 00:00 2024-09-12T01:50:19.919 INFO:teuthology.orchestra.run.smithi184.stdout:(106/156): gperftools-libs-2.9.1-2.el9.x86_64.r 3.6 MB/s | 309 kB 00:00 2024-09-12T01:50:19.926 INFO:teuthology.orchestra.run.smithi016.stdout:(98/156): socat-1.7.4.1-6.el9.x86_64.rpm 3.5 MB/s | 304 kB 00:00 2024-09-12T01:50:19.944 INFO:teuthology.orchestra.run.smithi184.stdout:(107/156): grpc-data-1.46.7-10.el9.noarch.rpm 389 kB/s | 19 kB 00:00 2024-09-12T01:50:19.951 INFO:teuthology.orchestra.run.smithi016.stdout:(99/156): uuid-1.6.2-55.el9.x86_64.rpm 769 kB/s | 58 kB 00:00 2024-09-12T01:50:19.952 INFO:teuthology.orchestra.run.smithi033.stdout:(154/156): NetworkManager-libnm-1.49.90-1.el9.x 3.0 MB/s | 1.8 MB 00:00 2024-09-12T01:50:19.969 INFO:teuthology.orchestra.run.smithi184.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-09-12T01:50:19.976 INFO:teuthology.orchestra.run.smithi016.stdout:(100/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.9 MB/s | 96 kB 00:00 2024-09-12T01:50:20.002 INFO:teuthology.orchestra.run.smithi016.stdout:(101/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-09-12T01:50:20.003 INFO:teuthology.orchestra.run.smithi184.stdout:(109/156): protobuf-compiler-3.14.0-13.el9.x86_ 1.9 MB/s | 863 kB 00:00 2024-09-12T01:50:20.027 INFO:teuthology.orchestra.run.smithi016.stdout:(102/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 894 kB/s | 45 kB 00:00 2024-09-12T01:50:20.028 INFO:teuthology.orchestra.run.smithi184.stdout:(110/156): liboath-2.6.7-2.el9.x86_64.rpm 827 kB/s | 49 kB 00:00 2024-09-12T01:50:20.070 INFO:teuthology.orchestra.run.smithi184.stdout:(111/156): luarocks-3.9.2-5.el9.noarch.rpm 4.4 MB/s | 151 kB 00:00 2024-09-12T01:50:20.086 INFO:teuthology.orchestra.run.smithi016.stdout:(103/156): lua-devel-5.4.4-4.el9.x86_64.rpm 266 kB/s | 22 kB 00:00 2024-09-12T01:50:20.095 INFO:teuthology.orchestra.run.smithi184.stdout:(112/156): libunwind-1.6.2-1.el9.x86_64.rpm 731 kB/s | 67 kB 00:00 2024-09-12T01:50:20.102 INFO:teuthology.orchestra.run.smithi033.stdout:(155/156): openssl-3.2.2-6.el9.x86_64.rpm 3.1 MB/s | 1.4 MB 00:00 2024-09-12T01:50:20.154 INFO:teuthology.orchestra.run.smithi184.stdout:(113/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 10 MB/s | 839 kB 00:00 2024-09-12T01:50:20.161 INFO:teuthology.orchestra.run.smithi016.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 7.2 MB/s | 548 kB 00:00 2024-09-12T01:50:20.179 INFO:teuthology.orchestra.run.smithi184.stdout:(114/156): python3-asyncssh-2.13.2-5.el9.noarch 6.4 MB/s | 548 kB 00:00 2024-09-12T01:50:20.204 INFO:teuthology.orchestra.run.smithi184.stdout:(115/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 865 kB/s | 43 kB 00:00 2024-09-12T01:50:20.211 INFO:teuthology.orchestra.run.smithi016.stdout:(105/156): gperftools-libs-2.9.1-2.el9.x86_64.r 6.0 MB/s | 309 kB 00:00 2024-09-12T01:50:20.229 INFO:teuthology.orchestra.run.smithi184.stdout:(116/156): python3-cachetools-4.2.4-1.el9.noarc 640 kB/s | 32 kB 00:00 2024-09-12T01:50:20.245 INFO:teuthology.orchestra.run.smithi016.stdout:(106/156): protobuf-compiler-3.14.0-13.el9.x86_ 3.9 MB/s | 863 kB 00:00 2024-09-12T01:50:20.255 INFO:teuthology.orchestra.run.smithi184.stdout:(117/156): python3-certifi-2023.05.07-4.el9.noa 280 kB/s | 14 kB 00:00 2024-09-12T01:50:20.270 INFO:teuthology.orchestra.run.smithi016.stdout:(107/156): grpc-data-1.46.7-10.el9.noarch.rpm 332 kB/s | 19 kB 00:00 2024-09-12T01:50:20.288 INFO:teuthology.orchestra.run.smithi184.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 2.9 MB/s | 172 kB 00:00 2024-09-12T01:50:20.312 INFO:teuthology.orchestra.run.smithi016.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 784 kB/s | 26 kB 00:00 2024-09-12T01:50:20.346 INFO:teuthology.orchestra.run.smithi016.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 1.4 MB/s | 49 kB 00:00 2024-09-12T01:50:20.355 INFO:teuthology.orchestra.run.smithi184.stdout:(119/156): libarrow-9.0.0-11.el9.x86_64.rpm 10 MB/s | 4.4 MB 00:00 2024-09-12T01:50:20.369 INFO:teuthology.orchestra.run.smithi033.stdout:(156/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 4.6 MB/s | 2.4 MB 00:00 2024-09-12T01:50:20.371 INFO:teuthology.orchestra.run.smithi016.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2024-09-12T01:50:20.375 INFO:teuthology.orchestra.run.smithi033.stdout:-------------------------------------------------------------------------------- 2024-09-12T01:50:20.375 INFO:teuthology.orchestra.run.smithi033.stdout:Total 18 MB/s | 226 MB 00:12 2024-09-12T01:50:20.381 INFO:teuthology.orchestra.run.smithi184.stdout:(120/156): python3-cherrypy-18.6.1-2.el9.noarch 2.8 MB/s | 358 kB 00:00 2024-09-12T01:50:20.405 INFO:teuthology.orchestra.run.smithi016.stdout:(111/156): luarocks-3.9.2-5.el9.noarch.rpm 4.4 MB/s | 151 kB 00:00 2024-09-12T01:50:20.406 INFO:teuthology.orchestra.run.smithi184.stdout:(121/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.5 MB/s | 303 kB 00:00 2024-09-12T01:50:20.431 INFO:teuthology.orchestra.run.smithi184.stdout:(122/156): python3-google-auth-2.34.0-1.el9.noa 2.8 MB/s | 220 kB 00:00 2024-09-12T01:50:20.456 INFO:teuthology.orchestra.run.smithi184.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 436 kB/s | 11 kB 00:00 2024-09-12T01:50:20.463 INFO:teuthology.orchestra.run.smithi016.stdout:(112/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 14 MB/s | 839 kB 00:00 2024-09-12T01:50:20.482 INFO:teuthology.orchestra.run.smithi184.stdout:(124/156): python3-grpcio-tools-1.46.7-10.el9.x 2.8 MB/s | 144 kB 00:00 2024-09-12T01:50:20.506 INFO:teuthology.orchestra.run.smithi016.stdout:(113/156): python3-asyncssh-2.13.2-5.el9.noarch 13 MB/s | 548 kB 00:00 2024-09-12T01:50:20.507 INFO:teuthology.orchestra.run.smithi184.stdout:(125/156): python3-jaraco-classes-3.2.1-5.el9.n 351 kB/s | 18 kB 00:00 2024-09-12T01:50:20.530 INFO:teuthology.orchestra.run.smithi016.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-12T01:50:20.532 INFO:teuthology.orchestra.run.smithi184.stdout:(126/156): python3-jaraco-collections-3.0.0-8.e 462 kB/s | 23 kB 00:00 2024-09-12T01:50:20.556 INFO:teuthology.orchestra.run.smithi016.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-09-12T01:50:20.557 INFO:teuthology.orchestra.run.smithi184.stdout:(127/156): python3-jaraco-functools-3.5.0-2.el9 387 kB/s | 19 kB 00:00 2024-09-12T01:50:20.581 INFO:teuthology.orchestra.run.smithi016.stdout:(116/156): python3-certifi-2023.05.07-4.el9.noa 565 kB/s | 14 kB 00:00 2024-09-12T01:50:20.582 INFO:teuthology.orchestra.run.smithi184.stdout:(128/156): python3-jaraco-text-3.2.0-6.el9.noar 392 kB/s | 20 kB 00:00 2024-09-12T01:50:20.608 INFO:teuthology.orchestra.run.smithi016.stdout:(117/156): python3-cheroot-8.6.0-4.el9.noarch.r 6.7 MB/s | 172 kB 00:00 2024-09-12T01:50:20.608 INFO:teuthology.orchestra.run.smithi184.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-12T01:50:20.633 INFO:teuthology.orchestra.run.smithi184.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-12T01:50:20.667 INFO:teuthology.orchestra.run.smithi184.stdout:(131/156): python3-natsort-7.1.1-5.el9.noarch.r 1.7 MB/s | 58 kB 00:00 2024-09-12T01:50:20.674 INFO:teuthology.orchestra.run.smithi016.stdout:(118/156): libarrow-9.0.0-11.el9.x86_64.rpm 10 MB/s | 4.4 MB 00:00 2024-09-12T01:50:20.699 INFO:teuthology.orchestra.run.smithi016.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 3.8 MB/s | 358 kB 00:00 2024-09-12T01:50:20.700 INFO:teuthology.orchestra.run.smithi184.stdout:(132/156): python3-pecan-1.4.2-3.el9.noarch.rpm 8.0 MB/s | 272 kB 00:00 2024-09-12T01:50:20.725 INFO:teuthology.orchestra.run.smithi016.stdout:(120/156): python3-google-auth-2.34.0-1.el9.noa 8.6 MB/s | 220 kB 00:00 2024-09-12T01:50:20.734 INFO:teuthology.orchestra.run.smithi184.stdout:(133/156): python3-kubernetes-26.1.0-2.el9.noar 5.8 MB/s | 1.0 MB 00:00 2024-09-12T01:50:20.750 INFO:teuthology.orchestra.run.smithi016.stdout:(121/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.9 MB/s | 303 kB 00:00 2024-09-12T01:50:20.759 INFO:teuthology.orchestra.run.smithi184.stdout:(134/156): python3-portend-3.1.0-2.el9.noarch.r 281 kB/s | 16 kB 00:00 2024-09-12T01:50:20.783 INFO:teuthology.orchestra.run.smithi016.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 4.3 MB/s | 144 kB 00:00 2024-09-12T01:50:20.785 INFO:teuthology.orchestra.run.smithi184.stdout:(135/156): python3-pyperclip-1.8.2-2.el9.noarch 1.1 MB/s | 28 kB 00:00 2024-09-12T01:50:20.801 INFO:teuthology.orchestra.run.smithi016.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 628 kB/s | 11 kB 00:00 2024-09-12T01:50:20.810 INFO:teuthology.orchestra.run.smithi184.stdout:(136/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-09-12T01:50:20.817 INFO:teuthology.orchestra.run.smithi016.stdout:(124/156): python3-jaraco-classes-3.2.1-5.el9.n 1.0 MB/s | 18 kB 00:00 2024-09-12T01:50:20.835 INFO:teuthology.orchestra.run.smithi016.stdout:(125/156): python3-jaraco-collections-3.0.0-8.e 1.4 MB/s | 23 kB 00:00 2024-09-12T01:50:20.863 INFO:teuthology.orchestra.run.smithi016.stdout:(126/156): python3-jaraco-functools-3.5.0-2.el9 688 kB/s | 19 kB 00:00 2024-09-12T01:50:20.889 INFO:teuthology.orchestra.run.smithi016.stdout:(127/156): python3-jaraco-text-3.2.0-6.el9.noar 788 kB/s | 20 kB 00:00 2024-09-12T01:50:20.896 INFO:teuthology.orchestra.run.smithi184.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 276 kB/s | 31 kB 00:00 2024-09-12T01:50:20.944 INFO:teuthology.orchestra.run.smithi184.stdout:(138/156): python3-grpcio-1.46.7-10.el9.x86_64. 3.6 MB/s | 2.0 MB 00:00 2024-09-12T01:50:20.947 INFO:teuthology.orchestra.run.smithi016.stdout:(128/156): python3-grpcio-1.46.7-10.el9.x86_64. 9.2 MB/s | 2.0 MB 00:00 2024-09-12T01:50:20.969 INFO:teuthology.orchestra.run.smithi184.stdout:(139/156): python3-rsa-4.9-2.el9.noarch.rpm 815 kB/s | 59 kB 00:00 2024-09-12T01:50:20.972 INFO:teuthology.orchestra.run.smithi016.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-12T01:50:20.995 INFO:teuthology.orchestra.run.smithi184.stdout:(140/156): python3-routes-2.5.1-5.el9.noarch.rp 1.0 MB/s | 188 kB 00:00 2024-09-12T01:50:21.006 INFO:teuthology.orchestra.run.smithi016.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 2.3 MB/s | 79 kB 00:00 2024-09-12T01:50:21.020 INFO:teuthology.orchestra.run.smithi184.stdout:(141/156): python3-tempora-5.0.0-2.el9.noarch.r 475 kB/s | 36 kB 00:00 2024-09-12T01:50:21.045 INFO:teuthology.orchestra.run.smithi184.stdout:(142/156): python3-typing-extensions-4.12.2-2.e 989 kB/s | 74 kB 00:00 2024-09-12T01:50:21.056 INFO:teuthology.orchestra.run.smithi016.stdout:(131/156): python3-kubernetes-26.1.0-2.el9.noar 6.1 MB/s | 1.0 MB 00:00 2024-09-12T01:50:21.070 INFO:teuthology.orchestra.run.smithi184.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 1.8 MB/s | 90 kB 00:00 2024-09-12T01:50:21.082 INFO:teuthology.orchestra.run.smithi016.stdout:(132/156): python3-natsort-7.1.1-5.el9.noarch.r 766 kB/s | 58 kB 00:00 2024-09-12T01:50:21.096 INFO:teuthology.orchestra.run.smithi184.stdout:(144/156): python3-zc-lockfile-2.0-10.el9.noarc 804 kB/s | 20 kB 00:00 2024-09-12T01:50:21.107 INFO:teuthology.orchestra.run.smithi016.stdout:(133/156): python3-portend-3.1.0-2.el9.noarch.r 663 kB/s | 16 kB 00:00 2024-09-12T01:50:21.121 INFO:teuthology.orchestra.run.smithi184.stdout:(145/156): python3-webob-1.8.8-2.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-09-12T01:50:21.132 INFO:teuthology.orchestra.run.smithi016.stdout:(134/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 3.5 MB/s | 90 kB 00:00 2024-09-12T01:50:21.146 INFO:teuthology.orchestra.run.smithi184.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-12T01:50:21.157 INFO:teuthology.orchestra.run.smithi016.stdout:(135/156): python3-pecan-1.4.2-3.el9.noarch.rpm 2.6 MB/s | 272 kB 00:00 2024-09-12T01:50:21.171 INFO:teuthology.orchestra.run.smithi184.stdout:(147/156): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-09-12T01:50:21.183 INFO:teuthology.orchestra.run.smithi016.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 563 kB/s | 28 kB 00:00 2024-09-12T01:50:21.208 INFO:teuthology.orchestra.run.smithi016.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 612 kB/s | 31 kB 00:00 2024-09-12T01:50:21.213 INFO:teuthology.orchestra.run.smithi184.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 17 MB/s | 1.6 MB 00:00 2024-09-12T01:50:21.234 INFO:teuthology.orchestra.run.smithi016.stdout:(138/156): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-12T01:50:21.259 INFO:teuthology.orchestra.run.smithi016.stdout:(139/156): python3-tempora-5.0.0-2.el9.noarch.r 1.4 MB/s | 36 kB 00:00 2024-09-12T01:50:21.284 INFO:teuthology.orchestra.run.smithi016.stdout:(140/156): python3-routes-2.5.1-5.el9.noarch.rp 1.8 MB/s | 188 kB 00:00 2024-09-12T01:50:21.310 INFO:teuthology.orchestra.run.smithi016.stdout:(141/156): python3-typing-extensions-4.12.2-2.e 1.4 MB/s | 74 kB 00:00 2024-09-12T01:50:21.343 INFO:teuthology.orchestra.run.smithi016.stdout:(142/156): python3-websocket-client-1.2.3-2.el9 2.6 MB/s | 90 kB 00:00 2024-09-12T01:50:21.369 INFO:teuthology.orchestra.run.smithi016.stdout:(143/156): python3-webob-1.8.8-2.el9.noarch.rpm 2.7 MB/s | 230 kB 00:00 2024-09-12T01:50:21.394 INFO:teuthology.orchestra.run.smithi016.stdout:(144/156): python3-zc-lockfile-2.0-10.el9.noarc 800 kB/s | 20 kB 00:00 2024-09-12T01:50:21.419 INFO:teuthology.orchestra.run.smithi016.stdout:(145/156): re2-20211101-20.el9.x86_64.rpm 7.5 MB/s | 191 kB 00:00 2024-09-12T01:50:21.445 INFO:teuthology.orchestra.run.smithi016.stdout:(146/156): python3-werkzeug-2.0.3-3.el9.1.noarc 4.1 MB/s | 427 kB 00:00 2024-09-12T01:50:21.495 INFO:teuthology.orchestra.run.smithi016.stdout:(147/156): thrift-0.15.0-2.el9.x86_64.rpm 21 MB/s | 1.6 MB 00:00 2024-09-12T01:50:21.639 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction check 2024-09-12T01:50:21.739 INFO:teuthology.orchestra.run.smithi184.stdout:(149/156): NetworkManager-1.49.90-1.el9.x86_64. 4.4 MB/s | 2.3 MB 00:00 2024-09-12T01:50:21.806 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction check succeeded. 2024-09-12T01:50:21.806 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction test 2024-09-12T01:50:21.897 INFO:teuthology.orchestra.run.smithi184.stdout:(150/156): librbd1-19.1.1-298.gb53b6d3e.el9.x86 4.4 MB/s | 3.2 MB 00:00 2024-09-12T01:50:21.956 INFO:teuthology.orchestra.run.smithi184.stdout:(151/156): NetworkManager-team-1.49.90-1.el9.x8 700 kB/s | 41 kB 00:00 2024-09-12T01:50:22.031 INFO:teuthology.orchestra.run.smithi184.stdout:(152/156): NetworkManager-tui-1.49.90-1.el9.x86 3.3 MB/s | 249 kB 00:00 2024-09-12T01:50:22.037 INFO:teuthology.orchestra.run.smithi016.stdout:(148/156): librbd1-19.1.1-298.gb53b6d3e.el9.x86 5.8 MB/s | 3.2 MB 00:00 2024-09-12T01:50:22.256 INFO:teuthology.orchestra.run.smithi184.stdout:(153/156): librados2-19.1.1-298.gb53b6d3e.el9.x 3.1 MB/s | 3.4 MB 00:01 2024-09-12T01:50:22.337 INFO:teuthology.orchestra.run.smithi016.stdout:(149/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 5.8 MB/s | 16 MB 00:02 2024-09-12T01:50:22.356 INFO:teuthology.orchestra.run.smithi184.stdout:(154/156): NetworkManager-libnm-1.49.90-1.el9.x 2.9 MB/s | 1.8 MB 00:00 2024-09-12T01:50:22.390 INFO:teuthology.orchestra.run.smithi184.stdout:(155/156): openssl-3.2.2-6.el9.x86_64.rpm 3.8 MB/s | 1.4 MB 00:00 2024-09-12T01:50:22.413 INFO:teuthology.orchestra.run.smithi016.stdout:(150/156): librados2-19.1.1-298.gb53b6d3e.el9.x 3.5 MB/s | 3.4 MB 00:00 2024-09-12T01:50:22.890 INFO:teuthology.orchestra.run.smithi184.stdout:(156/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 3.8 MB/s | 2.4 MB 00:00 2024-09-12T01:50:22.896 INFO:teuthology.orchestra.run.smithi184.stdout:-------------------------------------------------------------------------------- 2024-09-12T01:50:22.897 INFO:teuthology.orchestra.run.smithi184.stdout:Total 15 MB/s | 226 MB 00:14 2024-09-12T01:50:22.996 INFO:teuthology.orchestra.run.smithi016.stdout:(151/156): NetworkManager-team-1.49.90-1.el9.x8 71 kB/s | 41 kB 00:00 2024-09-12T01:50:23.213 INFO:teuthology.orchestra.run.smithi016.stdout:(152/156): NetworkManager-1.49.90-1.el9.x86_64. 2.0 MB/s | 2.3 MB 00:01 2024-09-12T01:50:23.338 INFO:teuthology.orchestra.run.smithi016.stdout:(153/156): NetworkManager-libnm-1.49.90-1.el9.x 1.8 MB/s | 1.8 MB 00:01 2024-09-12T01:50:23.717 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction test succeeded. 2024-09-12T01:50:23.717 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction 2024-09-12T01:50:23.805 INFO:teuthology.orchestra.run.smithi016.stdout:(154/156): NetworkManager-tui-1.49.90-1.el9.x86 308 kB/s | 249 kB 00:00 2024-09-12T01:50:24.131 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction check 2024-09-12T01:50:24.155 INFO:teuthology.orchestra.run.smithi016.stdout:(155/156): openssl-3.2.2-6.el9.x86_64.rpm 1.4 MB/s | 1.4 MB 00:00 2024-09-12T01:50:24.282 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction check succeeded. 2024-09-12T01:50:24.283 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction test 2024-09-12T01:50:24.447 INFO:teuthology.orchestra.run.smithi016.stdout:(156/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 2.2 MB/s | 2.4 MB 00:01 2024-09-12T01:50:24.454 INFO:teuthology.orchestra.run.smithi016.stdout:-------------------------------------------------------------------------------- 2024-09-12T01:50:24.454 INFO:teuthology.orchestra.run.smithi016.stdout:Total 13 MB/s | 226 MB 00:16 2024-09-12T01:50:25.729 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction check 2024-09-12T01:50:25.897 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction check succeeded. 2024-09-12T01:50:25.898 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction test 2024-09-12T01:50:26.271 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction test succeeded. 2024-09-12T01:50:26.273 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction 2024-09-12T01:50:28.216 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction test succeeded. 2024-09-12T01:50:28.216 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction 2024-09-12T01:50:30.385 INFO:teuthology.orchestra.run.smithi033.stdout: Preparing : 1/1 2024-09-12T01:50:30.506 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-12T01:50:30.545 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-12T01:50:30.800 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-12T01:50:30.850 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-12T01:50:30.874 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-12T01:50:30.919 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-12T01:50:31.267 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-12T01:50:31.383 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-12T01:50:31.508 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-12T01:50:31.534 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-12T01:50:31.611 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-12T01:50:31.639 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 10/164 2024-09-12T01:50:31.810 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-12T01:50:31.869 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-12T01:50:31.897 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-12T01:50:32.003 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-12T01:50:32.031 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 14/164 2024-09-12T01:50:32.094 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 14/164 2024-09-12T01:50:32.116 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-12T01:50:32.138 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-12T01:50:32.350 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-12T01:50:32.485 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-12T01:50:32.622 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 17/164 2024-09-12T01:50:32.678 INFO:teuthology.orchestra.run.smithi184.stdout: Preparing : 1/1 2024-09-12T01:50:32.737 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-12T01:50:32.785 INFO:teuthology.orchestra.run.smithi184.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-12T01:50:32.807 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-12T01:50:32.877 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-12T01:50:33.002 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-12T01:50:33.034 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-12T01:50:33.036 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-12T01:50:33.062 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-12T01:50:33.104 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-12T01:50:33.297 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-12T01:50:33.398 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-12T01:50:33.426 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-12T01:50:33.432 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-12T01:50:33.476 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-12T01:50:33.537 INFO:teuthology.orchestra.run.smithi184.stdout: Upgrading : librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-12T01:50:33.552 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-12T01:50:33.603 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-12T01:50:33.642 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-12T01:50:33.659 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-12T01:50:33.688 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-12T01:50:33.694 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-12T01:50:33.735 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-12T01:50:33.762 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-12T01:50:33.781 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-12T01:50:33.809 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 10/164 2024-09-12T01:50:33.835 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-12T01:50:33.887 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-12T01:50:33.915 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-12T01:50:33.951 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-12T01:50:33.957 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-12T01:50:34.023 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-12T01:50:34.030 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-12T01:50:34.051 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-12T01:50:34.061 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x 34/164 2024-09-12T01:50:34.105 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9 35/164 2024-09-12T01:50:34.142 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 36/164 2024-09-12T01:50:34.163 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-12T01:50:34.185 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 14/164 2024-09-12T01:50:34.217 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-12T01:50:34.248 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 14/164 2024-09-12T01:50:34.265 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-12T01:50:34.270 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-12T01:50:34.292 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-12T01:50:34.343 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-12T01:50:34.488 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-12T01:50:34.494 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-12T01:50:34.552 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-12T01:50:34.626 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-12T01:50:34.734 INFO:teuthology.orchestra.run.smithi184.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 17/164 2024-09-12T01:50:34.833 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-12T01:50:34.929 INFO:teuthology.orchestra.run.smithi184.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-12T01:50:35.001 INFO:teuthology.orchestra.run.smithi016.stdout: Preparing : 1/1 2024-09-12T01:50:35.118 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-12T01:50:35.124 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-12T01:50:35.138 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-12T01:50:35.158 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-12T01:50:35.208 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-12T01:50:35.229 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-12T01:50:35.242 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-12T01:50:35.249 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-12T01:50:35.270 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-12T01:50:35.285 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-12T01:50:35.294 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-12T01:50:35.323 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 46/164 2024-09-12T01:50:35.346 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-12T01:50:35.355 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-12T01:50:35.365 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-12T01:50:35.391 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-12T01:50:35.435 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-12T01:50:35.519 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-12T01:50:35.528 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-12T01:50:35.577 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-12T01:50:35.584 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-12T01:50:35.617 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-12T01:50:35.642 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-12T01:50:35.661 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-12T01:50:35.697 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-12T01:50:35.715 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-12T01:50:35.715 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-12T01:50:35.724 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-12T01:50:35.752 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-12T01:50:35.788 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-12T01:50:35.793 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-12T01:50:35.837 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-12T01:50:35.837 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-12T01:50:35.846 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-12T01:50:35.871 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-12T01:50:35.877 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-12T01:50:35.912 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-12T01:50:35.915 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-12T01:50:35.964 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-12T01:50:35.965 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-12T01:50:35.989 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-12T01:50:35.997 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-12T01:50:36.023 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x 34/164 2024-09-12T01:50:36.057 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9 35/164 2024-09-12T01:50:36.073 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-12T01:50:36.087 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 36/164 2024-09-12T01:50:36.102 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 10/164 2024-09-12T01:50:36.118 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-12T01:50:36.155 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-12T01:50:36.201 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-12T01:50:36.275 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-12T01:50:36.278 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-12T01:50:36.300 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-12T01:50:36.332 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-12T01:50:36.342 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-12T01:50:36.368 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-12T01:50:36.411 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-12T01:50:36.451 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-12T01:50:36.462 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-12T01:50:36.475 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-12T01:50:36.491 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-12T01:50:36.503 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 14/164 2024-09-12T01:50:36.531 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-12T01:50:36.556 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-09-12T01:50:36.573 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 14/164 2024-09-12T01:50:36.581 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-12T01:50:36.595 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-12T01:50:36.619 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-12T01:50:36.660 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-12T01:50:36.789 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-12T01:50:36.823 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-12T01:50:36.840 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-12T01:50:36.868 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-12T01:50:36.903 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-12T01:50:36.942 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-12T01:50:36.982 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-12T01:50:36.984 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-12T01:50:37.011 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-12T01:50:37.020 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-12T01:50:37.065 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-12T01:50:37.071 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-12T01:50:37.095 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-12T01:50:37.104 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-12T01:50:37.119 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 17/164 2024-09-12T01:50:37.129 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-12T01:50:37.147 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-12T01:50:37.167 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 46/164 2024-09-12T01:50:37.180 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-12T01:50:37.209 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-12T01:50:37.236 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-12T01:50:37.289 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-12T01:50:37.321 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-12T01:50:37.337 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-12T01:50:37.367 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-12T01:50:37.368 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-12T01:50:37.429 INFO:teuthology.orchestra.run.smithi184.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-12T01:50:37.483 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-12T01:50:37.485 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-12T01:50:37.545 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-12T01:50:37.556 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-12T01:50:37.569 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-12T01:50:37.596 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-12T01:50:37.633 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-12T01:50:37.679 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-12T01:50:37.681 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-12T01:50:37.715 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-12T01:50:37.758 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-12T01:50:37.785 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-12T01:50:37.800 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-12T01:50:37.826 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-12T01:50:37.861 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-12T01:50:37.929 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-12T01:50:37.937 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-12T01:50:37.977 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-12T01:50:37.979 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-12T01:50:38.018 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-12T01:50:38.019 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-12T01:50:38.072 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-12T01:50:38.104 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-12T01:50:38.121 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-12T01:50:38.131 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-12T01:50:38.160 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-12T01:50:38.200 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-12T01:50:38.204 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-12T01:50:38.225 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-09-12T01:50:38.255 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-12T01:50:38.285 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-12T01:50:38.328 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-12T01:50:38.357 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-12T01:50:38.407 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-12T01:50:38.407 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-12T01:50:38.439 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x 34/164 2024-09-12T01:50:38.440 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-12T01:50:38.491 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9 35/164 2024-09-12T01:50:38.501 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-12T01:50:38.528 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 36/164 2024-09-12T01:50:38.546 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-12T01:50:38.581 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-12T01:50:38.601 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-12T01:50:38.605 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-12T01:50:38.643 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-12T01:50:38.658 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-12T01:50:38.715 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-12T01:50:38.730 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-12T01:50:38.750 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-12T01:50:38.759 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-12T01:50:38.785 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-12T01:50:38.787 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-12T01:50:38.822 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-12T01:50:38.861 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-12T01:50:38.875 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-12T01:50:38.892 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-12T01:50:38.930 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-12T01:50:38.930 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-12T01:50:38.965 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-12T01:50:38.975 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-12T01:50:38.998 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-12T01:50:39.007 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-12T01:50:39.040 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-12T01:50:39.074 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-12T01:50:39.086 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-12T01:50:39.161 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-12T01:50:39.205 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-12T01:50:39.236 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-12T01:50:39.338 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-12T01:50:39.396 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-12T01:50:39.487 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-12T01:50:39.577 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-12T01:50:39.612 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-12T01:50:39.663 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-12T01:50:39.701 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 46/164 2024-09-12T01:50:39.743 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-12T01:50:39.792 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-12T01:50:39.903 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-12T01:50:39.914 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-12T01:50:39.971 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-12T01:50:40.027 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-12T01:50:40.102 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-12T01:50:40.112 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-12T01:50:40.139 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-12T01:50:40.175 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-12T01:50:40.176 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-12T01:50:40.209 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-12T01:50:40.223 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-12T01:50:40.258 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-12T01:50:40.307 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-12T01:50:40.309 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-12T01:50:40.360 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-12T01:50:40.365 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-12T01:50:40.410 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-12T01:50:40.497 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-12T01:50:40.510 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-12T01:50:40.545 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-12T01:50:40.737 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-12T01:50:40.741 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-12T01:50:40.768 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-12T01:50:40.792 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-12T01:50:40.845 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-12T01:50:40.902 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-12T01:50:40.941 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-12T01:50:40.981 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-12T01:50:41.014 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-09-12T01:50:41.039 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-12T01:50:41.135 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-12T01:50:41.238 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-12T01:50:41.270 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-12T01:50:41.305 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-12T01:50:41.327 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-12T01:50:41.333 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-12T01:50:41.356 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-12T01:50:41.376 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-12T01:50:41.388 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-12T01:50:41.416 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-12T01:50:41.455 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-12T01:50:41.484 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-12T01:50:41.491 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-12T01:50:41.519 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-12T01:50:41.552 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-12T01:50:41.595 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-12T01:50:41.607 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-12T01:50:41.637 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-12T01:50:41.724 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-12T01:50:41.727 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-12T01:50:41.776 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 101/164 2024-09-12T01:50:41.785 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-12T01:50:41.827 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-12T01:50:41.851 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 101/164 2024-09-12T01:50:41.881 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 102/164 2024-09-12T01:50:41.917 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-12T01:50:41.934 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-12T01:50:41.988 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-12T01:50:42.423 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-12T01:50:42.454 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 105/164 2024-09-12T01:50:42.525 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 105/164 2024-09-12T01:50:42.555 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 106/164 2024-09-12T01:50:42.755 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-12T01:50:42.809 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-12T01:50:42.839 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-12T01:50:42.842 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-12T01:50:42.913 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-12T01:50:42.940 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-12T01:50:42.946 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-12T01:50:42.974 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-12T01:50:42.999 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-12T01:50:43.043 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-12T01:50:43.134 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-12T01:50:43.170 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-12T01:50:43.286 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-12T01:50:43.328 INFO:teuthology.orchestra.run.smithi184.stdout: Upgrading : librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 101/164 2024-09-12T01:50:43.363 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-12T01:50:43.401 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-12T01:50:43.436 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 101/164 2024-09-12T01:50:43.467 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 102/164 2024-09-12T01:50:43.491 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-12T01:50:43.502 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-12T01:50:43.984 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-12T01:50:44.015 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 105/164 2024-09-12T01:50:44.069 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 105/164 2024-09-12T01:50:44.099 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 106/164 2024-09-12T01:50:44.185 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-12T01:50:44.312 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-12T01:50:44.978 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-12T01:50:45.019 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-12T01:50:45.598 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-12T01:50:45.630 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-12T01:50:45.648 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 108/164 2024-09-12T01:50:45.698 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-12T01:50:45.726 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-12T01:50:45.730 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 108/164 2024-09-12T01:50:45.730 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-12T01:50:45.730 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-12T01:50:45.731 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-12T01:50:45.757 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 109/164 2024-09-12T01:50:45.759 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-12T01:50:46.094 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-12T01:50:46.139 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 101/164 2024-09-12T01:50:46.223 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 101/164 2024-09-12T01:50:46.252 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 102/164 2024-09-12T01:50:46.289 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-12T01:50:46.451 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-12T01:50:46.674 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-12T01:50:46.795 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-12T01:50:46.825 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 105/164 2024-09-12T01:50:46.897 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 105/164 2024-09-12T01:50:46.935 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 106/164 2024-09-12T01:50:47.220 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-12T01:50:47.266 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 108/164 2024-09-12T01:50:47.357 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 108/164 2024-09-12T01:50:47.358 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-12T01:50:47.358 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-12T01:50:47.358 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-12T01:50:47.392 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 109/164 2024-09-12T01:50:49.345 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-12T01:50:49.376 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-12T01:50:49.956 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-12T01:50:50.513 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 108/164 2024-09-12T01:50:50.513 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 108/164 2024-09-12T01:50:50.513 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-12T01:50:50.513 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-12T01:50:50.513 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:50:50.513 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 109/164 2024-09-12T01:50:57.947 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 109/164 2024-09-12T01:50:57.947 INFO:teuthology.orchestra.run.smithi033.stdout:skipping the directory /sys 2024-09-12T01:50:57.947 INFO:teuthology.orchestra.run.smithi033.stdout:skipping the directory /proc 2024-09-12T01:50:57.947 INFO:teuthology.orchestra.run.smithi033.stdout:skipping the directory /mnt 2024-09-12T01:50:57.948 INFO:teuthology.orchestra.run.smithi033.stdout:skipping the directory /var/tmp 2024-09-12T01:50:57.948 INFO:teuthology.orchestra.run.smithi033.stdout:skipping the directory /home 2024-09-12T01:50:57.948 INFO:teuthology.orchestra.run.smithi033.stdout:skipping the directory /root 2024-09-12T01:50:57.948 INFO:teuthology.orchestra.run.smithi033.stdout:skipping the directory /tmp 2024-09-12T01:50:57.948 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-12T01:50:58.134 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 110/164 2024-09-12T01:50:58.150 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 110/164 2024-09-12T01:50:58.150 INFO:teuthology.orchestra.run.smithi033.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T01:50:58.150 INFO:teuthology.orchestra.run.smithi033.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-12T01:50:58.150 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-12T01:50:59.204 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 109/164 2024-09-12T01:50:59.204 INFO:teuthology.orchestra.run.smithi184.stdout:skipping the directory /sys 2024-09-12T01:50:59.205 INFO:teuthology.orchestra.run.smithi184.stdout:skipping the directory /proc 2024-09-12T01:50:59.205 INFO:teuthology.orchestra.run.smithi184.stdout:skipping the directory /mnt 2024-09-12T01:50:59.205 INFO:teuthology.orchestra.run.smithi184.stdout:skipping the directory /var/tmp 2024-09-12T01:50:59.205 INFO:teuthology.orchestra.run.smithi184.stdout:skipping the directory /home 2024-09-12T01:50:59.205 INFO:teuthology.orchestra.run.smithi184.stdout:skipping the directory /root 2024-09-12T01:50:59.205 INFO:teuthology.orchestra.run.smithi184.stdout:skipping the directory /tmp 2024-09-12T01:50:59.205 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-12T01:50:59.387 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 110/164 2024-09-12T01:50:59.404 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 110/164 2024-09-12T01:50:59.404 INFO:teuthology.orchestra.run.smithi184.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T01:50:59.404 INFO:teuthology.orchestra.run.smithi184.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-12T01:50:59.404 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-12T01:50:59.676 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 111/164 2024-09-12T01:50:59.703 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 111/164 2024-09-12T01:50:59.703 INFO:teuthology.orchestra.run.smithi033.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T01:50:59.703 INFO:teuthology.orchestra.run.smithi033.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-12T01:50:59.703 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-12T01:50:59.704 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-12T01:50:59.704 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-12T01:50:59.937 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 112/164 2024-09-12T01:50:59.961 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 112/164 2024-09-12T01:50:59.961 INFO:teuthology.orchestra.run.smithi033.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T01:50:59.961 INFO:teuthology.orchestra.run.smithi033.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-12T01:50:59.961 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-12T01:50:59.961 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-12T01:50:59.961 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-12T01:51:00.430 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 113/164 2024-09-12T01:51:00.454 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 113/164 2024-09-12T01:51:00.455 INFO:teuthology.orchestra.run.smithi033.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T01:51:00.455 INFO:teuthology.orchestra.run.smithi033.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-12T01:51:00.455 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-12T01:51:00.455 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-12T01:51:00.455 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-12T01:51:00.570 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-12T01:51:00.628 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-12T01:51:00.692 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-12T01:51:00.892 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-12T01:51:00.897 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 111/164 2024-09-12T01:51:00.925 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 111/164 2024-09-12T01:51:00.926 INFO:teuthology.orchestra.run.smithi184.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T01:51:00.926 INFO:teuthology.orchestra.run.smithi184.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-12T01:51:00.926 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-12T01:51:00.926 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-12T01:51:00.926 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-12T01:51:00.949 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-12T01:51:01.031 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-12T01:51:01.129 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-12T01:51:01.173 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 112/164 2024-09-12T01:51:01.197 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 112/164 2024-09-12T01:51:01.197 INFO:teuthology.orchestra.run.smithi184.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T01:51:01.197 INFO:teuthology.orchestra.run.smithi184.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-12T01:51:01.197 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-12T01:51:01.197 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-12T01:51:01.197 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-12T01:51:01.273 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-12T01:51:01.323 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-12T01:51:01.354 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-12T01:51:01.403 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-12T01:51:01.640 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 113/164 2024-09-12T01:51:01.664 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 113/164 2024-09-12T01:51:01.664 INFO:teuthology.orchestra.run.smithi184.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T01:51:01.665 INFO:teuthology.orchestra.run.smithi184.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-12T01:51:01.665 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-12T01:51:01.665 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-12T01:51:01.665 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-12T01:51:01.741 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 125/164 2024-09-12T01:51:01.779 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-12T01:51:01.781 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 125/164 2024-09-12T01:51:01.813 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el 126/164 2024-09-12T01:51:01.835 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-12T01:51:01.887 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.e 127/164 2024-09-12T01:51:01.891 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-12T01:51:01.912 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 128/164 2024-09-12T01:51:02.075 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-12T01:51:02.124 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-12T01:51:02.194 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-12T01:51:02.282 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-12T01:51:02.413 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-12T01:51:02.458 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-12T01:51:02.489 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-12T01:51:02.517 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-12T01:51:02.679 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 109/164 2024-09-12T01:51:02.680 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /sys 2024-09-12T01:51:02.680 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /proc 2024-09-12T01:51:02.680 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /mnt 2024-09-12T01:51:02.680 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /var/tmp 2024-09-12T01:51:02.680 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /home 2024-09-12T01:51:02.680 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /root 2024-09-12T01:51:02.680 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /tmp 2024-09-12T01:51:02.680 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:51:02.869 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 110/164 2024-09-12T01:51:02.886 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 110/164 2024-09-12T01:51:02.886 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T01:51:02.886 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-12T01:51:02.887 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:51:02.926 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 125/164 2024-09-12T01:51:02.958 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 125/164 2024-09-12T01:51:02.989 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el 126/164 2024-09-12T01:51:03.025 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 128/164 2024-09-12T01:51:03.051 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.e 127/164 2024-09-12T01:51:03.055 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 129/164 2024-09-12T01:51:03.080 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 128/164 2024-09-12T01:51:04.074 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 129/164 2024-09-12T01:51:04.105 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 130/164 2024-09-12T01:51:04.118 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 128/164 2024-09-12T01:51:04.149 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 129/164 2024-09-12T01:51:04.216 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 130/164 2024-09-12T01:51:04.383 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9 131/164 2024-09-12T01:51:04.416 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 132/164 2024-09-12T01:51:04.439 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 111/164 2024-09-12T01:51:04.440 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 132/164 2024-09-12T01:51:04.440 INFO:teuthology.orchestra.run.smithi033.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T01:51:04.440 INFO:teuthology.orchestra.run.smithi033.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-12T01:51:04.440 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-12T01:51:04.440 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-12T01:51:04.440 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-12T01:51:04.468 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 111/164 2024-09-12T01:51:04.468 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T01:51:04.468 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-12T01:51:04.468 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-12T01:51:04.468 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-12T01:51:04.468 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:51:04.491 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 133/164 2024-09-12T01:51:04.504 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 133/164 2024-09-12T01:51:04.605 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 134/164 2024-09-12T01:51:04.658 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 135/164 2024-09-12T01:51:04.707 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 112/164 2024-09-12T01:51:04.732 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 112/164 2024-09-12T01:51:04.732 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T01:51:04.732 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-12T01:51:04.732 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-12T01:51:04.732 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-12T01:51:04.732 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:51:05.113 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 129/164 2024-09-12T01:51:05.141 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 130/164 2024-09-12T01:51:05.191 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 113/164 2024-09-12T01:51:05.216 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 113/164 2024-09-12T01:51:05.216 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T01:51:05.216 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-12T01:51:05.217 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-12T01:51:05.217 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-12T01:51:05.217 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:51:05.241 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 130/164 2024-09-12T01:51:05.330 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-12T01:51:05.396 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-12T01:51:05.412 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9 131/164 2024-09-12T01:51:05.443 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 132/164 2024-09-12T01:51:05.460 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-12T01:51:05.468 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 132/164 2024-09-12T01:51:05.468 INFO:teuthology.orchestra.run.smithi184.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T01:51:05.468 INFO:teuthology.orchestra.run.smithi184.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-12T01:51:05.468 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-12T01:51:05.468 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-12T01:51:05.468 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-12T01:51:05.518 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 133/164 2024-09-12T01:51:05.530 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 133/164 2024-09-12T01:51:05.632 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 134/164 2024-09-12T01:51:05.633 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-12T01:51:05.653 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-12T01:51:05.661 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 137/164 2024-09-12T01:51:05.683 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 135/164 2024-09-12T01:51:05.685 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 137/164 2024-09-12T01:51:05.685 INFO:teuthology.orchestra.run.smithi033.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T01:51:05.685 INFO:teuthology.orchestra.run.smithi033.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-12T01:51:05.685 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-12T01:51:05.685 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-12T01:51:05.685 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-12T01:51:05.708 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-12T01:51:05.728 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 138/164 2024-09-12T01:51:05.751 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 138/164 2024-09-12T01:51:05.751 INFO:teuthology.orchestra.run.smithi033.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T01:51:05.751 INFO:teuthology.orchestra.run.smithi033.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-12T01:51:05.752 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-12T01:51:05.793 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-12T01:51:05.804 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 139/164 2024-09-12T01:51:05.828 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 139/164 2024-09-12T01:51:05.829 INFO:teuthology.orchestra.run.smithi033.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T01:51:05.829 INFO:teuthology.orchestra.run.smithi033.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-12T01:51:05.829 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-12T01:51:05.829 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-12T01:51:05.830 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-12T01:51:05.887 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-12T01:51:06.029 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-12T01:51:06.077 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-12T01:51:06.108 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-12T01:51:06.135 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-12T01:51:06.154 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 140/164 2024-09-12T01:51:06.178 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 140/164 2024-09-12T01:51:06.178 INFO:teuthology.orchestra.run.smithi033.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T01:51:06.179 INFO:teuthology.orchestra.run.smithi033.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-12T01:51:06.179 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-12T01:51:06.179 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-12T01:51:06.179 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-12T01:51:06.598 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 125/164 2024-09-12T01:51:06.605 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-12T01:51:06.626 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 125/164 2024-09-12T01:51:06.638 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 137/164 2024-09-12T01:51:06.658 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el 126/164 2024-09-12T01:51:06.662 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 137/164 2024-09-12T01:51:06.662 INFO:teuthology.orchestra.run.smithi184.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T01:51:06.662 INFO:teuthology.orchestra.run.smithi184.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-12T01:51:06.662 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-12T01:51:06.662 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-12T01:51:06.662 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-12T01:51:06.705 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 138/164 2024-09-12T01:51:06.723 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.e 127/164 2024-09-12T01:51:06.729 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 138/164 2024-09-12T01:51:06.729 INFO:teuthology.orchestra.run.smithi184.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T01:51:06.729 INFO:teuthology.orchestra.run.smithi184.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-12T01:51:06.729 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-12T01:51:06.749 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 128/164 2024-09-12T01:51:06.789 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 139/164 2024-09-12T01:51:06.813 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 139/164 2024-09-12T01:51:06.813 INFO:teuthology.orchestra.run.smithi184.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T01:51:06.813 INFO:teuthology.orchestra.run.smithi184.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-12T01:51:06.813 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-12T01:51:06.813 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-12T01:51:06.814 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-12T01:51:07.123 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 140/164 2024-09-12T01:51:07.147 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 140/164 2024-09-12T01:51:07.147 INFO:teuthology.orchestra.run.smithi184.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T01:51:07.147 INFO:teuthology.orchestra.run.smithi184.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-12T01:51:07.147 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-12T01:51:07.147 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-12T01:51:07.148 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-12T01:51:07.860 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 128/164 2024-09-12T01:51:07.909 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 129/164 2024-09-12T01:51:08.932 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 129/164 2024-09-12T01:51:08.968 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 130/164 2024-09-12T01:51:09.076 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 130/164 2024-09-12T01:51:09.254 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9 131/164 2024-09-12T01:51:09.287 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 132/164 2024-09-12T01:51:09.313 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 132/164 2024-09-12T01:51:09.313 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T01:51:09.313 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-12T01:51:09.314 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-12T01:51:09.314 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-12T01:51:09.314 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:51:09.362 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 133/164 2024-09-12T01:51:09.375 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 133/164 2024-09-12T01:51:09.476 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 134/164 2024-09-12T01:51:09.532 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 135/164 2024-09-12T01:51:10.509 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-12T01:51:10.540 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 137/164 2024-09-12T01:51:10.566 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 137/164 2024-09-12T01:51:10.567 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T01:51:10.567 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-12T01:51:10.567 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-12T01:51:10.567 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-12T01:51:10.567 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:51:10.607 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 138/164 2024-09-12T01:51:10.632 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 138/164 2024-09-12T01:51:10.632 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T01:51:10.632 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-12T01:51:10.632 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:51:10.683 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 139/164 2024-09-12T01:51:10.709 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 139/164 2024-09-12T01:51:10.709 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T01:51:10.709 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-12T01:51:10.709 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-12T01:51:10.709 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-12T01:51:10.709 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:51:10.992 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 141/164 2024-09-12T01:51:11.034 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 142/164 2024-09-12T01:51:11.035 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 140/164 2024-09-12T01:51:11.061 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 140/164 2024-09-12T01:51:11.061 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T01:51:11.061 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-12T01:51:11.062 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-12T01:51:11.062 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-12T01:51:11.062 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:51:11.081 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 143/164 2024-09-12T01:51:11.106 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-12T01:51:11.136 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-12T01:51:11.335 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 146/164 2024-09-12T01:51:11.388 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-12T01:51:11.418 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_6 148/164 2024-09-12T01:51:11.449 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 149/164 2024-09-12T01:51:11.486 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 150/164 2024-09-12T01:51:11.685 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 141/164 2024-09-12T01:51:11.726 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 142/164 2024-09-12T01:51:11.767 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 143/164 2024-09-12T01:51:11.799 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-12T01:51:11.826 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-12T01:51:12.040 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 146/164 2024-09-12T01:51:12.089 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-12T01:51:12.096 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-12T01:51:12.111 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_6 148/164 2024-09-12T01:51:12.141 INFO:teuthology.orchestra.run.smithi184.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 149/164 2024-09-12T01:51:12.141 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-12T01:51:12.162 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-12T01:51:12.196 INFO:teuthology.orchestra.run.smithi184.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 150/164 2024-09-12T01:51:12.817 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-12T01:51:12.860 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-12T01:51:12.881 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-12T01:51:13.087 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 152/164 2024-09-12T01:51:13.188 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-12T01:51:13.230 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-12T01:51:13.323 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-12T01:51:13.366 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-12T01:51:13.391 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-12T01:51:13.417 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-12T01:51:13.417 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-12T01:51:13.518 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-12T01:51:13.518 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-12T01:51:13.585 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-12T01:51:13.610 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-12T01:51:13.616 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-12T01:51:13.616 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-12T01:51:13.707 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 152/164 2024-09-12T01:51:13.781 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-12T01:51:13.812 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-12T01:51:13.812 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-12T01:51:13.821 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-12T01:51:13.896 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-12T01:51:13.904 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-12T01:51:13.962 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-12T01:51:13.984 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-12T01:51:14.009 INFO:teuthology.orchestra.run.smithi184.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-12T01:51:14.044 INFO:teuthology.orchestra.run.smithi184.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-12T01:51:14.044 INFO:teuthology.orchestra.run.smithi184.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-12T01:51:14.128 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-12T01:51:14.128 INFO:teuthology.orchestra.run.smithi184.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-12T01:51:14.220 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-12T01:51:14.245 INFO:teuthology.orchestra.run.smithi184.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-12T01:51:14.251 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-12T01:51:14.251 INFO:teuthology.orchestra.run.smithi184.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-12T01:51:14.422 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-12T01:51:14.422 INFO:teuthology.orchestra.run.smithi184.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-12T01:51:14.506 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-12T01:51:14.569 INFO:teuthology.orchestra.run.smithi184.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-12T01:51:15.905 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 141/164 2024-09-12T01:51:15.947 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 142/164 2024-09-12T01:51:15.988 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 143/164 2024-09-12T01:51:16.019 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-12T01:51:16.038 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-12T01:51:16.252 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 146/164 2024-09-12T01:51:16.293 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-12T01:51:16.343 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_6 148/164 2024-09-12T01:51:16.378 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 149/164 2024-09-12T01:51:16.424 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 150/164 2024-09-12T01:51:17.143 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-12T01:51:17.181 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-12T01:51:17.203 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-12T01:51:18.119 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 152/164 2024-09-12T01:51:18.203 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-12T01:51:18.255 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-12T01:51:18.350 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-12T01:51:18.381 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-12T01:51:18.406 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-12T01:51:18.449 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-12T01:51:18.449 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-12T01:51:18.542 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-12T01:51:18.542 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-12T01:51:18.626 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-12T01:51:18.651 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-12T01:51:18.657 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-12T01:51:18.657 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-12T01:51:18.844 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-12T01:51:18.845 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-12T01:51:18.921 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-12T01:51:18.988 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-12T01:51:19.543 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-12T01:51:19.543 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/164 2024-09-12T01:51:19.543 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 2/164 2024-09-12T01:51:19.543 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 3/164 2024-09-12T01:51:19.543 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 4/164 2024-09-12T01:51:19.543 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 5/164 2024-09-12T01:51:19.543 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 6/164 2024-09-12T01:51:19.544 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 7/164 2024-09-12T01:51:19.544 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-12T01:51:19.544 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-12T01:51:19.544 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 10/164 2024-09-12T01:51:19.544 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 11/164 2024-09-12T01:51:19.544 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 12/164 2024-09-12T01:51:19.544 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-12T01:51:19.544 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 14/164 2024-09-12T01:51:19.544 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_6 15/164 2024-09-12T01:51:19.544 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 16/164 2024-09-12T01:51:19.545 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 17/164 2024-09-12T01:51:19.545 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 18/164 2024-09-12T01:51:19.545 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 19/164 2024-09-12T01:51:19.545 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 20/164 2024-09-12T01:51:19.545 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9 21/164 2024-09-12T01:51:19.545 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x 22/164 2024-09-12T01:51:19.545 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 23/164 2024-09-12T01:51:19.545 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 24/164 2024-09-12T01:51:19.545 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 25/164 2024-09-12T01:51:19.545 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 26/164 2024-09-12T01:51:19.545 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 27/164 2024-09-12T01:51:19.545 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 28/164 2024-09-12T01:51:19.545 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 29/164 2024-09-12T01:51:19.546 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.e 30/164 2024-09-12T01:51:19.546 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 31/164 2024-09-12T01:51:19.546 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 32/164 2024-09-12T01:51:19.546 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 33/164 2024-09-12T01:51:19.546 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9 34/164 2024-09-12T01:51:19.546 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 35/164 2024-09-12T01:51:19.546 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el 36/164 2024-09-12T01:51:19.546 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 37/164 2024-09-12T01:51:19.546 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 38/164 2024-09-12T01:51:19.546 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 39/164 2024-09-12T01:51:19.546 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-12T01:51:19.546 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-12T01:51:19.546 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-12T01:51:19.547 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-12T01:51:19.547 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-12T01:51:19.547 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-12T01:51:19.547 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-12T01:51:19.547 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-12T01:51:19.547 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-12T01:51:19.547 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-12T01:51:19.547 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-12T01:51:19.547 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-12T01:51:19.547 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-12T01:51:19.548 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-12T01:51:19.548 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-12T01:51:19.548 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-12T01:51:19.548 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-12T01:51:19.548 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-12T01:51:19.548 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-12T01:51:19.548 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-12T01:51:19.548 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-12T01:51:19.548 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-12T01:51:19.548 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-12T01:51:19.548 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-12T01:51:19.548 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-12T01:51:19.549 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-12T01:51:19.549 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-12T01:51:19.549 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-12T01:51:19.549 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-12T01:51:19.549 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-12T01:51:19.549 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-12T01:51:19.549 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-12T01:51:19.549 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-12T01:51:19.549 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-12T01:51:19.549 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-12T01:51:19.549 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-12T01:51:19.549 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-12T01:51:19.549 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-12T01:51:19.549 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-12T01:51:19.550 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-12T01:51:19.550 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-12T01:51:19.550 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-12T01:51:19.550 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-12T01:51:19.550 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-12T01:51:19.550 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-12T01:51:19.550 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-12T01:51:19.550 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-12T01:51:19.550 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-12T01:51:19.550 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-12T01:51:19.550 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-12T01:51:19.550 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-12T01:51:19.550 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-12T01:51:19.551 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-12T01:51:19.551 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-12T01:51:19.551 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-12T01:51:19.551 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-12T01:51:19.551 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-12T01:51:19.551 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-12T01:51:19.551 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-12T01:51:19.551 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-12T01:51:19.551 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-12T01:51:19.551 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-12T01:51:19.551 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-12T01:51:19.551 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-12T01:51:19.551 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-12T01:51:19.552 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-12T01:51:19.552 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-12T01:51:19.552 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/164 2024-09-12T01:51:19.552 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-12T01:51:19.552 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-12T01:51:19.552 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-12T01:51:19.552 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-12T01:51:19.552 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-12T01:51:19.552 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-12T01:51:19.552 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-12T01:51:19.552 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-12T01:51:19.552 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-12T01:51:19.552 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-12T01:51:19.553 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-12T01:51:19.553 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-12T01:51:19.553 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-12T01:51:19.553 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-12T01:51:19.553 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-12T01:51:19.553 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-12T01:51:19.553 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-12T01:51:19.553 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-12T01:51:19.553 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-12T01:51:19.553 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-12T01:51:19.553 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-12T01:51:19.553 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-12T01:51:19.553 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-12T01:51:19.553 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-12T01:51:19.554 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-12T01:51:19.554 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-12T01:51:19.554 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-12T01:51:19.554 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-12T01:51:19.554 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-12T01:51:19.554 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-12T01:51:19.554 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-12T01:51:19.554 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-12T01:51:19.554 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-12T01:51:19.554 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-12T01:51:19.554 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-12T01:51:19.554 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-12T01:51:19.554 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-12T01:51:19.555 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-12T01:51:19.555 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-12T01:51:19.555 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-12T01:51:19.555 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-12T01:51:19.555 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 149/164 2024-09-12T01:51:19.555 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-12T01:51:19.555 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 151/164 2024-09-12T01:51:19.555 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-12T01:51:19.555 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 153/164 2024-09-12T01:51:19.555 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-12T01:51:19.555 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 155/164 2024-09-12T01:51:19.555 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-12T01:51:19.555 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 157/164 2024-09-12T01:51:19.556 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-12T01:51:19.556 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 159/164 2024-09-12T01:51:19.556 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-12T01:51:19.556 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-12T01:51:19.556 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-12T01:51:19.556 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-12T01:51:20.104 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-12T01:51:20.104 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/164 2024-09-12T01:51:20.104 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 2/164 2024-09-12T01:51:20.105 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 3/164 2024-09-12T01:51:20.105 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 4/164 2024-09-12T01:51:20.105 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 5/164 2024-09-12T01:51:20.105 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 6/164 2024-09-12T01:51:20.105 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 7/164 2024-09-12T01:51:20.105 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-12T01:51:20.105 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-12T01:51:20.105 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 10/164 2024-09-12T01:51:20.105 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 11/164 2024-09-12T01:51:20.105 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 12/164 2024-09-12T01:51:20.105 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-12T01:51:20.105 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 14/164 2024-09-12T01:51:20.106 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_6 15/164 2024-09-12T01:51:20.106 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 16/164 2024-09-12T01:51:20.106 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 17/164 2024-09-12T01:51:20.106 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 18/164 2024-09-12T01:51:20.106 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 19/164 2024-09-12T01:51:20.106 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 20/164 2024-09-12T01:51:20.106 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9 21/164 2024-09-12T01:51:20.106 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x 22/164 2024-09-12T01:51:20.106 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 23/164 2024-09-12T01:51:20.106 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 24/164 2024-09-12T01:51:20.106 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 25/164 2024-09-12T01:51:20.106 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 26/164 2024-09-12T01:51:20.106 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 27/164 2024-09-12T01:51:20.106 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 28/164 2024-09-12T01:51:20.107 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 29/164 2024-09-12T01:51:20.107 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.e 30/164 2024-09-12T01:51:20.107 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 31/164 2024-09-12T01:51:20.107 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 32/164 2024-09-12T01:51:20.107 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 33/164 2024-09-12T01:51:20.107 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9 34/164 2024-09-12T01:51:20.107 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 35/164 2024-09-12T01:51:20.107 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el 36/164 2024-09-12T01:51:20.107 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 37/164 2024-09-12T01:51:20.107 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 38/164 2024-09-12T01:51:20.107 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 39/164 2024-09-12T01:51:20.107 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-12T01:51:20.108 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-12T01:51:20.108 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-12T01:51:20.108 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-12T01:51:20.108 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-12T01:51:20.108 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-12T01:51:20.108 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-12T01:51:20.108 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-12T01:51:20.108 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-12T01:51:20.108 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-12T01:51:20.108 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-12T01:51:20.108 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-12T01:51:20.108 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-12T01:51:20.109 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-12T01:51:20.109 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-12T01:51:20.109 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-12T01:51:20.109 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-12T01:51:20.109 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-12T01:51:20.109 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-12T01:51:20.109 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-12T01:51:20.109 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-12T01:51:20.109 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-12T01:51:20.109 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-12T01:51:20.109 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-12T01:51:20.109 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-12T01:51:20.109 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-12T01:51:20.109 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-12T01:51:20.110 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-12T01:51:20.110 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-12T01:51:20.110 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-12T01:51:20.110 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-12T01:51:20.110 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-12T01:51:20.110 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-12T01:51:20.110 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-12T01:51:20.110 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-12T01:51:20.110 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-12T01:51:20.110 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-12T01:51:20.110 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-12T01:51:20.110 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-12T01:51:20.110 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-12T01:51:20.111 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-12T01:51:20.111 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-12T01:51:20.111 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-12T01:51:20.111 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-12T01:51:20.111 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-12T01:51:20.111 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-12T01:51:20.111 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-12T01:51:20.111 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-12T01:51:20.111 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-12T01:51:20.111 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-12T01:51:20.111 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-12T01:51:20.111 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-12T01:51:20.111 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-12T01:51:20.112 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-12T01:51:20.112 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-12T01:51:20.112 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-12T01:51:20.112 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-12T01:51:20.112 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-12T01:51:20.112 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-12T01:51:20.112 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-12T01:51:20.112 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-12T01:51:20.112 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-12T01:51:20.112 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-12T01:51:20.112 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-12T01:51:20.112 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-12T01:51:20.113 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-12T01:51:20.113 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-12T01:51:20.113 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/164 2024-09-12T01:51:20.113 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-12T01:51:20.113 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-12T01:51:20.113 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-12T01:51:20.113 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-12T01:51:20.113 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-12T01:51:20.113 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-12T01:51:20.113 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-12T01:51:20.113 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-12T01:51:20.113 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-12T01:51:20.114 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-12T01:51:20.114 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-12T01:51:20.114 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-12T01:51:20.114 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-12T01:51:20.114 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-12T01:51:20.114 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-12T01:51:20.114 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-12T01:51:20.114 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-12T01:51:20.114 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-12T01:51:20.114 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-12T01:51:20.114 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-12T01:51:20.114 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-12T01:51:20.114 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-12T01:51:20.115 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-12T01:51:20.115 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-12T01:51:20.115 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-12T01:51:20.115 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-12T01:51:20.115 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-12T01:51:20.115 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-12T01:51:20.115 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-12T01:51:20.115 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-12T01:51:20.115 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-12T01:51:20.115 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-12T01:51:20.115 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-12T01:51:20.115 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-12T01:51:20.115 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-12T01:51:20.115 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-12T01:51:20.116 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-12T01:51:20.116 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-12T01:51:20.116 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-12T01:51:20.116 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-12T01:51:20.116 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-12T01:51:20.116 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 149/164 2024-09-12T01:51:20.116 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-12T01:51:20.116 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 151/164 2024-09-12T01:51:20.116 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-12T01:51:20.117 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 153/164 2024-09-12T01:51:20.117 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-12T01:51:20.117 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 155/164 2024-09-12T01:51:20.117 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-12T01:51:20.117 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 157/164 2024-09-12T01:51:20.117 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-12T01:51:20.117 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 159/164 2024-09-12T01:51:20.117 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-12T01:51:20.117 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-12T01:51:20.117 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-12T01:51:20.117 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-12T01:51:22.310 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-12T01:51:22.310 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-12T01:51:22.311 INFO:teuthology.orchestra.run.smithi033.stdout:Upgraded: 2024-09-12T01:51:22.311 INFO:teuthology.orchestra.run.smithi033.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-12T01:51:22.311 INFO:teuthology.orchestra.run.smithi033.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-12T01:51:22.311 INFO:teuthology.orchestra.run.smithi033.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-12T01:51:22.311 INFO:teuthology.orchestra.run.smithi033.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-12T01:51:22.311 INFO:teuthology.orchestra.run.smithi033.stdout: librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:51:22.311 INFO:teuthology.orchestra.run.smithi033.stdout: librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:51:22.311 INFO:teuthology.orchestra.run.smithi033.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-12T01:51:22.311 INFO:teuthology.orchestra.run.smithi033.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-12T01:51:22.311 INFO:teuthology.orchestra.run.smithi033.stdout:Installed: 2024-09-12T01:51:22.311 INFO:teuthology.orchestra.run.smithi033.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-12T01:51:22.311 INFO:teuthology.orchestra.run.smithi033.stdout: automake-1.16.2-8.el9.noarch 2024-09-12T01:51:22.311 INFO:teuthology.orchestra.run.smithi033.stdout: bison-3.7.4-5.el9.x86_64 2024-09-12T01:51:22.312 INFO:teuthology.orchestra.run.smithi033.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-12T01:51:22.312 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:51:22.312 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:51:22.312 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:51:22.312 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:51:22.312 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T01:51:22.312 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-immutable-object-cache-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:51:22.312 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:51:22.312 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:51:22.312 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T01:51:22.312 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T01:51:22.312 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T01:51:22.312 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T01:51:22.313 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T01:51:22.313 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:51:22.313 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:51:22.313 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T01:51:22.313 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:51:22.313 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:51:22.313 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:51:22.313 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T01:51:22.313 INFO:teuthology.orchestra.run.smithi033.stdout: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T01:51:22.313 INFO:teuthology.orchestra.run.smithi033.stdout: cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:51:22.313 INFO:teuthology.orchestra.run.smithi033.stdout: cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:51:22.313 INFO:teuthology.orchestra.run.smithi033.stdout: cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T01:51:22.313 INFO:teuthology.orchestra.run.smithi033.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-12T01:51:22.314 INFO:teuthology.orchestra.run.smithi033.stdout: flex-2.6.4-9.el9.x86_64 2024-09-12T01:51:22.314 INFO:teuthology.orchestra.run.smithi033.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-12T01:51:22.314 INFO:teuthology.orchestra.run.smithi033.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-12T01:51:22.314 INFO:teuthology.orchestra.run.smithi033.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-12T01:51:22.314 INFO:teuthology.orchestra.run.smithi033.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-12T01:51:22.314 INFO:teuthology.orchestra.run.smithi033.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-12T01:51:22.314 INFO:teuthology.orchestra.run.smithi033.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-12T01:51:22.314 INFO:teuthology.orchestra.run.smithi033.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-12T01:51:22.314 INFO:teuthology.orchestra.run.smithi033.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-12T01:51:22.314 INFO:teuthology.orchestra.run.smithi033.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-12T01:51:22.314 INFO:teuthology.orchestra.run.smithi033.stdout: libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:51:22.314 INFO:teuthology.orchestra.run.smithi033.stdout: libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:51:22.314 INFO:teuthology.orchestra.run.smithi033.stdout: libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:51:22.315 INFO:teuthology.orchestra.run.smithi033.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-12T01:51:22.315 INFO:teuthology.orchestra.run.smithi033.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-12T01:51:22.315 INFO:teuthology.orchestra.run.smithi033.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-12T01:51:22.315 INFO:teuthology.orchestra.run.smithi033.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-12T01:51:22.315 INFO:teuthology.orchestra.run.smithi033.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-12T01:51:22.315 INFO:teuthology.orchestra.run.smithi033.stdout: librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:51:22.315 INFO:teuthology.orchestra.run.smithi033.stdout: libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:51:22.315 INFO:teuthology.orchestra.run.smithi033.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-12T01:51:22.315 INFO:teuthology.orchestra.run.smithi033.stdout: librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:51:22.315 INFO:teuthology.orchestra.run.smithi033.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-12T01:51:22.315 INFO:teuthology.orchestra.run.smithi033.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-12T01:51:22.315 INFO:teuthology.orchestra.run.smithi033.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-12T01:51:22.315 INFO:teuthology.orchestra.run.smithi033.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-12T01:51:22.316 INFO:teuthology.orchestra.run.smithi033.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-12T01:51:22.316 INFO:teuthology.orchestra.run.smithi033.stdout: lua-5.4.4-4.el9.x86_64 2024-09-12T01:51:22.316 INFO:teuthology.orchestra.run.smithi033.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-12T01:51:22.316 INFO:teuthology.orchestra.run.smithi033.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-12T01:51:22.316 INFO:teuthology.orchestra.run.smithi033.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-12T01:51:22.316 INFO:teuthology.orchestra.run.smithi033.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-12T01:51:22.316 INFO:teuthology.orchestra.run.smithi033.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-12T01:51:22.316 INFO:teuthology.orchestra.run.smithi033.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-12T01:51:22.316 INFO:teuthology.orchestra.run.smithi033.stdout: postgresql-13.16-1.el9.x86_64 2024-09-12T01:51:22.316 INFO:teuthology.orchestra.run.smithi033.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-12T01:51:22.316 INFO:teuthology.orchestra.run.smithi033.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-12T01:51:22.316 INFO:teuthology.orchestra.run.smithi033.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-12T01:51:22.316 INFO:teuthology.orchestra.run.smithi033.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-12T01:51:22.317 INFO:teuthology.orchestra.run.smithi033.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-12T01:51:22.317 INFO:teuthology.orchestra.run.smithi033.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-12T01:51:22.317 INFO:teuthology.orchestra.run.smithi033.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-12T01:51:22.317 INFO:teuthology.orchestra.run.smithi033.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-12T01:51:22.317 INFO:teuthology.orchestra.run.smithi033.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-12T01:51:22.317 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-12T01:51:22.317 INFO:teuthology.orchestra.run.smithi033.stdout: python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:51:22.317 INFO:teuthology.orchestra.run.smithi033.stdout: python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:51:22.318 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:51:22.318 INFO:teuthology.orchestra.run.smithi033.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-12T01:51:22.318 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-12T01:51:22.318 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-12T01:51:22.318 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-12T01:51:22.318 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-12T01:51:22.318 INFO:teuthology.orchestra.run.smithi033.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-12T01:51:22.318 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-12T01:51:22.318 INFO:teuthology.orchestra.run.smithi033.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-12T01:51:22.318 INFO:teuthology.orchestra.run.smithi033.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-12T01:51:22.318 INFO:teuthology.orchestra.run.smithi033.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-12T01:51:22.318 INFO:teuthology.orchestra.run.smithi033.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-12T01:51:22.318 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-12T01:51:22.318 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-12T01:51:22.319 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-12T01:51:22.319 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-12T01:51:22.319 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-12T01:51:22.319 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-12T01:51:22.319 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-12T01:51:22.319 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-12T01:51:22.319 INFO:teuthology.orchestra.run.smithi033.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-12T01:51:22.319 INFO:teuthology.orchestra.run.smithi033.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-12T01:51:22.319 INFO:teuthology.orchestra.run.smithi033.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-12T01:51:22.319 INFO:teuthology.orchestra.run.smithi033.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-12T01:51:22.319 INFO:teuthology.orchestra.run.smithi033.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-12T01:51:22.319 INFO:teuthology.orchestra.run.smithi033.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-12T01:51:22.319 INFO:teuthology.orchestra.run.smithi033.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-12T01:51:22.320 INFO:teuthology.orchestra.run.smithi033.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-12T01:51:22.320 INFO:teuthology.orchestra.run.smithi033.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-12T01:51:22.320 INFO:teuthology.orchestra.run.smithi033.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-12T01:51:22.320 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-12T01:51:22.320 INFO:teuthology.orchestra.run.smithi033.stdout: python3-ply-3.11-14.el9.noarch 2024-09-12T01:51:22.320 INFO:teuthology.orchestra.run.smithi033.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-12T01:51:22.320 INFO:teuthology.orchestra.run.smithi033.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-12T01:51:22.320 INFO:teuthology.orchestra.run.smithi033.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-12T01:51:22.320 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-12T01:51:22.320 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-12T01:51:22.320 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-12T01:51:22.320 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-12T01:51:22.320 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-12T01:51:22.321 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-12T01:51:22.321 INFO:teuthology.orchestra.run.smithi033.stdout: python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:51:22.321 INFO:teuthology.orchestra.run.smithi033.stdout: python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:51:22.321 INFO:teuthology.orchestra.run.smithi033.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-12T01:51:22.321 INFO:teuthology.orchestra.run.smithi033.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-12T01:51:22.321 INFO:teuthology.orchestra.run.smithi033.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-12T01:51:22.321 INFO:teuthology.orchestra.run.smithi033.stdout: python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:51:22.321 INFO:teuthology.orchestra.run.smithi033.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-12T01:51:22.321 INFO:teuthology.orchestra.run.smithi033.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-12T01:51:22.321 INFO:teuthology.orchestra.run.smithi033.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-12T01:51:22.321 INFO:teuthology.orchestra.run.smithi033.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-12T01:51:22.321 INFO:teuthology.orchestra.run.smithi033.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-12T01:51:22.321 INFO:teuthology.orchestra.run.smithi033.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-12T01:51:22.321 INFO:teuthology.orchestra.run.smithi033.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-12T01:51:22.322 INFO:teuthology.orchestra.run.smithi033.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-12T01:51:22.322 INFO:teuthology.orchestra.run.smithi033.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-12T01:51:22.322 INFO:teuthology.orchestra.run.smithi033.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-12T01:51:22.322 INFO:teuthology.orchestra.run.smithi033.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-12T01:51:22.322 INFO:teuthology.orchestra.run.smithi033.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-12T01:51:22.322 INFO:teuthology.orchestra.run.smithi033.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-12T01:51:22.322 INFO:teuthology.orchestra.run.smithi033.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-12T01:51:22.322 INFO:teuthology.orchestra.run.smithi033.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-12T01:51:22.322 INFO:teuthology.orchestra.run.smithi033.stdout: rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:51:22.322 INFO:teuthology.orchestra.run.smithi033.stdout: rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:51:22.322 INFO:teuthology.orchestra.run.smithi033.stdout: rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:51:22.322 INFO:teuthology.orchestra.run.smithi033.stdout: re2-1:20211101-20.el9.x86_64 2024-09-12T01:51:22.322 INFO:teuthology.orchestra.run.smithi033.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-12T01:51:22.322 INFO:teuthology.orchestra.run.smithi033.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-12T01:51:22.323 INFO:teuthology.orchestra.run.smithi033.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-12T01:51:22.323 INFO:teuthology.orchestra.run.smithi033.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-12T01:51:22.323 INFO:teuthology.orchestra.run.smithi033.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-12T01:51:22.323 INFO:teuthology.orchestra.run.smithi033.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-12T01:51:22.323 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-12T01:51:22.324 INFO:teuthology.orchestra.run.smithi033.stdout:Complete! 2024-09-12T01:51:22.542 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-12T01:51:22.542 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-12T01:51:22.543 INFO:teuthology.orchestra.run.smithi184.stdout:Upgraded: 2024-09-12T01:51:22.543 INFO:teuthology.orchestra.run.smithi184.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-12T01:51:22.543 INFO:teuthology.orchestra.run.smithi184.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-12T01:51:22.543 INFO:teuthology.orchestra.run.smithi184.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-12T01:51:22.543 INFO:teuthology.orchestra.run.smithi184.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-12T01:51:22.543 INFO:teuthology.orchestra.run.smithi184.stdout: librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:51:22.543 INFO:teuthology.orchestra.run.smithi184.stdout: librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:51:22.543 INFO:teuthology.orchestra.run.smithi184.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-12T01:51:22.543 INFO:teuthology.orchestra.run.smithi184.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-12T01:51:22.543 INFO:teuthology.orchestra.run.smithi184.stdout:Installed: 2024-09-12T01:51:22.543 INFO:teuthology.orchestra.run.smithi184.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-12T01:51:22.543 INFO:teuthology.orchestra.run.smithi184.stdout: automake-1.16.2-8.el9.noarch 2024-09-12T01:51:22.543 INFO:teuthology.orchestra.run.smithi184.stdout: bison-3.7.4-5.el9.x86_64 2024-09-12T01:51:22.544 INFO:teuthology.orchestra.run.smithi184.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-12T01:51:22.544 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:51:22.544 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:51:22.544 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:51:22.544 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:51:22.544 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T01:51:22.544 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-immutable-object-cache-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:51:22.544 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:51:22.544 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:51:22.544 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T01:51:22.544 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T01:51:22.544 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T01:51:22.544 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T01:51:22.545 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T01:51:22.545 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:51:22.545 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:51:22.545 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T01:51:22.545 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:51:22.545 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:51:22.545 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:51:22.545 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T01:51:22.545 INFO:teuthology.orchestra.run.smithi184.stdout: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T01:51:22.545 INFO:teuthology.orchestra.run.smithi184.stdout: cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:51:22.545 INFO:teuthology.orchestra.run.smithi184.stdout: cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:51:22.545 INFO:teuthology.orchestra.run.smithi184.stdout: cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T01:51:22.545 INFO:teuthology.orchestra.run.smithi184.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-12T01:51:22.545 INFO:teuthology.orchestra.run.smithi184.stdout: flex-2.6.4-9.el9.x86_64 2024-09-12T01:51:22.546 INFO:teuthology.orchestra.run.smithi184.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-12T01:51:22.546 INFO:teuthology.orchestra.run.smithi184.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-12T01:51:22.546 INFO:teuthology.orchestra.run.smithi184.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-12T01:51:22.546 INFO:teuthology.orchestra.run.smithi184.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-12T01:51:22.546 INFO:teuthology.orchestra.run.smithi184.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-12T01:51:22.546 INFO:teuthology.orchestra.run.smithi184.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-12T01:51:22.546 INFO:teuthology.orchestra.run.smithi184.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-12T01:51:22.546 INFO:teuthology.orchestra.run.smithi184.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-12T01:51:22.546 INFO:teuthology.orchestra.run.smithi184.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-12T01:51:22.546 INFO:teuthology.orchestra.run.smithi184.stdout: libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:51:22.546 INFO:teuthology.orchestra.run.smithi184.stdout: libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:51:22.546 INFO:teuthology.orchestra.run.smithi184.stdout: libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:51:22.546 INFO:teuthology.orchestra.run.smithi184.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-12T01:51:22.547 INFO:teuthology.orchestra.run.smithi184.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-12T01:51:22.547 INFO:teuthology.orchestra.run.smithi184.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-12T01:51:22.547 INFO:teuthology.orchestra.run.smithi184.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-12T01:51:22.547 INFO:teuthology.orchestra.run.smithi184.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-12T01:51:22.547 INFO:teuthology.orchestra.run.smithi184.stdout: librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:51:22.547 INFO:teuthology.orchestra.run.smithi184.stdout: libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:51:22.547 INFO:teuthology.orchestra.run.smithi184.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-12T01:51:22.547 INFO:teuthology.orchestra.run.smithi184.stdout: librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:51:22.547 INFO:teuthology.orchestra.run.smithi184.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-12T01:51:22.547 INFO:teuthology.orchestra.run.smithi184.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-12T01:51:22.547 INFO:teuthology.orchestra.run.smithi184.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-12T01:51:22.547 INFO:teuthology.orchestra.run.smithi184.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-12T01:51:22.547 INFO:teuthology.orchestra.run.smithi184.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-12T01:51:22.548 INFO:teuthology.orchestra.run.smithi184.stdout: lua-5.4.4-4.el9.x86_64 2024-09-12T01:51:22.548 INFO:teuthology.orchestra.run.smithi184.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-12T01:51:22.548 INFO:teuthology.orchestra.run.smithi184.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-12T01:51:22.548 INFO:teuthology.orchestra.run.smithi184.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-12T01:51:22.548 INFO:teuthology.orchestra.run.smithi184.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-12T01:51:22.548 INFO:teuthology.orchestra.run.smithi184.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-12T01:51:22.548 INFO:teuthology.orchestra.run.smithi184.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-12T01:51:22.548 INFO:teuthology.orchestra.run.smithi184.stdout: postgresql-13.16-1.el9.x86_64 2024-09-12T01:51:22.548 INFO:teuthology.orchestra.run.smithi184.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-12T01:51:22.548 INFO:teuthology.orchestra.run.smithi184.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-12T01:51:22.548 INFO:teuthology.orchestra.run.smithi184.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-12T01:51:22.548 INFO:teuthology.orchestra.run.smithi184.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-12T01:51:22.548 INFO:teuthology.orchestra.run.smithi184.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-12T01:51:22.548 INFO:teuthology.orchestra.run.smithi184.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-12T01:51:22.549 INFO:teuthology.orchestra.run.smithi184.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-12T01:51:22.549 INFO:teuthology.orchestra.run.smithi184.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-12T01:51:22.549 INFO:teuthology.orchestra.run.smithi184.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-12T01:51:22.549 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-12T01:51:22.549 INFO:teuthology.orchestra.run.smithi184.stdout: python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:51:22.549 INFO:teuthology.orchestra.run.smithi184.stdout: python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:51:22.549 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:51:22.549 INFO:teuthology.orchestra.run.smithi184.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-12T01:51:22.549 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-12T01:51:22.549 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-12T01:51:22.549 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-12T01:51:22.549 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-12T01:51:22.549 INFO:teuthology.orchestra.run.smithi184.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-12T01:51:22.550 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-12T01:51:22.550 INFO:teuthology.orchestra.run.smithi184.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-12T01:51:22.550 INFO:teuthology.orchestra.run.smithi184.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-12T01:51:22.550 INFO:teuthology.orchestra.run.smithi184.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-12T01:51:22.550 INFO:teuthology.orchestra.run.smithi184.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-12T01:51:22.550 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-12T01:51:22.550 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-12T01:51:22.550 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-12T01:51:22.550 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-12T01:51:22.550 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-12T01:51:22.550 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-12T01:51:22.550 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-12T01:51:22.550 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-12T01:51:22.551 INFO:teuthology.orchestra.run.smithi184.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-12T01:51:22.551 INFO:teuthology.orchestra.run.smithi184.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-12T01:51:22.551 INFO:teuthology.orchestra.run.smithi184.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-12T01:51:22.551 INFO:teuthology.orchestra.run.smithi184.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-12T01:51:22.551 INFO:teuthology.orchestra.run.smithi184.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-12T01:51:22.551 INFO:teuthology.orchestra.run.smithi184.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-12T01:51:22.551 INFO:teuthology.orchestra.run.smithi184.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-12T01:51:22.551 INFO:teuthology.orchestra.run.smithi184.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-12T01:51:22.551 INFO:teuthology.orchestra.run.smithi184.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-12T01:51:22.551 INFO:teuthology.orchestra.run.smithi184.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-12T01:51:22.551 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-12T01:51:22.551 INFO:teuthology.orchestra.run.smithi184.stdout: python3-ply-3.11-14.el9.noarch 2024-09-12T01:51:22.551 INFO:teuthology.orchestra.run.smithi184.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-12T01:51:22.551 INFO:teuthology.orchestra.run.smithi184.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-12T01:51:22.552 INFO:teuthology.orchestra.run.smithi184.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-12T01:51:22.552 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-12T01:51:22.552 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-12T01:51:22.552 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-12T01:51:22.552 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-12T01:51:22.552 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-12T01:51:22.552 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-12T01:51:22.552 INFO:teuthology.orchestra.run.smithi184.stdout: python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:51:22.552 INFO:teuthology.orchestra.run.smithi184.stdout: python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:51:22.552 INFO:teuthology.orchestra.run.smithi184.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-12T01:51:22.552 INFO:teuthology.orchestra.run.smithi184.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-12T01:51:22.552 INFO:teuthology.orchestra.run.smithi184.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-12T01:51:22.552 INFO:teuthology.orchestra.run.smithi184.stdout: python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:51:22.553 INFO:teuthology.orchestra.run.smithi184.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-12T01:51:22.553 INFO:teuthology.orchestra.run.smithi184.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-12T01:51:22.553 INFO:teuthology.orchestra.run.smithi184.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-12T01:51:22.553 INFO:teuthology.orchestra.run.smithi184.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-12T01:51:22.553 INFO:teuthology.orchestra.run.smithi184.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-12T01:51:22.553 INFO:teuthology.orchestra.run.smithi184.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-12T01:51:22.553 INFO:teuthology.orchestra.run.smithi184.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-12T01:51:22.553 INFO:teuthology.orchestra.run.smithi184.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-12T01:51:22.553 INFO:teuthology.orchestra.run.smithi184.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-12T01:51:22.553 INFO:teuthology.orchestra.run.smithi184.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-12T01:51:22.553 INFO:teuthology.orchestra.run.smithi184.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-12T01:51:22.553 INFO:teuthology.orchestra.run.smithi184.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-12T01:51:22.553 INFO:teuthology.orchestra.run.smithi184.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-12T01:51:22.554 INFO:teuthology.orchestra.run.smithi184.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-12T01:51:22.554 INFO:teuthology.orchestra.run.smithi184.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-12T01:51:22.554 INFO:teuthology.orchestra.run.smithi184.stdout: rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:51:22.554 INFO:teuthology.orchestra.run.smithi184.stdout: rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:51:22.554 INFO:teuthology.orchestra.run.smithi184.stdout: rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:51:22.554 INFO:teuthology.orchestra.run.smithi184.stdout: re2-1:20211101-20.el9.x86_64 2024-09-12T01:51:22.554 INFO:teuthology.orchestra.run.smithi184.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-12T01:51:22.554 INFO:teuthology.orchestra.run.smithi184.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-12T01:51:22.554 INFO:teuthology.orchestra.run.smithi184.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-12T01:51:22.554 INFO:teuthology.orchestra.run.smithi184.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-12T01:51:22.554 INFO:teuthology.orchestra.run.smithi184.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-12T01:51:22.555 INFO:teuthology.orchestra.run.smithi184.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-12T01:51:22.555 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-12T01:51:22.555 INFO:teuthology.orchestra.run.smithi184.stdout:Complete! 2024-09-12T01:51:22.745 DEBUG:teuthology.parallel:result is None 2024-09-12T01:51:22.932 DEBUG:teuthology.parallel:result is None 2024-09-12T01:51:24.824 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-12T01:51:24.824 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/164 2024-09-12T01:51:24.824 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 2/164 2024-09-12T01:51:24.825 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 3/164 2024-09-12T01:51:24.825 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 4/164 2024-09-12T01:51:24.825 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 5/164 2024-09-12T01:51:24.825 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 6/164 2024-09-12T01:51:24.825 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 7/164 2024-09-12T01:51:24.825 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-12T01:51:24.825 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-12T01:51:24.825 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 10/164 2024-09-12T01:51:24.825 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 11/164 2024-09-12T01:51:24.825 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 12/164 2024-09-12T01:51:24.825 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-12T01:51:24.825 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 14/164 2024-09-12T01:51:24.826 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_6 15/164 2024-09-12T01:51:24.826 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 16/164 2024-09-12T01:51:24.826 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 17/164 2024-09-12T01:51:24.826 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 18/164 2024-09-12T01:51:24.826 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 19/164 2024-09-12T01:51:24.826 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 20/164 2024-09-12T01:51:24.826 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9 21/164 2024-09-12T01:51:24.826 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x 22/164 2024-09-12T01:51:24.826 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 23/164 2024-09-12T01:51:24.826 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 24/164 2024-09-12T01:51:24.826 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 25/164 2024-09-12T01:51:24.826 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 26/164 2024-09-12T01:51:24.826 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 27/164 2024-09-12T01:51:24.827 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 28/164 2024-09-12T01:51:24.827 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 29/164 2024-09-12T01:51:24.827 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.e 30/164 2024-09-12T01:51:24.838 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 31/164 2024-09-12T01:51:24.839 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 32/164 2024-09-12T01:51:24.839 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 33/164 2024-09-12T01:51:24.839 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9 34/164 2024-09-12T01:51:24.839 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 35/164 2024-09-12T01:51:24.839 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el 36/164 2024-09-12T01:51:24.839 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 37/164 2024-09-12T01:51:24.839 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 38/164 2024-09-12T01:51:24.839 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 39/164 2024-09-12T01:51:24.839 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-12T01:51:24.839 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-12T01:51:24.839 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-12T01:51:24.840 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-12T01:51:24.840 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-12T01:51:24.840 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-12T01:51:24.840 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-12T01:51:24.840 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-12T01:51:24.840 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-12T01:51:24.840 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-12T01:51:24.840 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-12T01:51:24.840 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-12T01:51:24.840 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-12T01:51:24.840 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-12T01:51:24.840 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-12T01:51:24.840 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-12T01:51:24.841 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-12T01:51:24.841 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-12T01:51:24.841 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-12T01:51:24.841 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-12T01:51:24.841 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-12T01:51:24.841 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-12T01:51:24.841 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-12T01:51:24.841 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-12T01:51:24.841 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-12T01:51:24.841 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-12T01:51:24.841 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-12T01:51:24.841 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-12T01:51:24.841 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-12T01:51:24.842 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-12T01:51:24.842 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-12T01:51:24.842 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-12T01:51:24.842 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-12T01:51:24.842 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-12T01:51:24.842 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-12T01:51:24.842 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-12T01:51:24.842 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-12T01:51:24.842 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-12T01:51:24.842 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-12T01:51:24.842 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-12T01:51:24.842 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-12T01:51:24.843 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-12T01:51:24.843 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-12T01:51:24.843 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-12T01:51:24.843 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-12T01:51:24.843 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-12T01:51:24.843 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-12T01:51:24.843 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-12T01:51:24.843 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-12T01:51:24.843 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-12T01:51:24.843 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-12T01:51:24.843 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-12T01:51:24.843 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-12T01:51:24.843 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-12T01:51:24.844 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-12T01:51:24.844 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-12T01:51:24.844 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-12T01:51:24.844 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-12T01:51:24.844 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-12T01:51:24.844 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-12T01:51:24.844 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-12T01:51:24.844 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-12T01:51:24.844 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-12T01:51:24.844 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-12T01:51:24.844 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-12T01:51:24.844 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-12T01:51:24.844 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-12T01:51:24.845 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/164 2024-09-12T01:51:24.845 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-12T01:51:24.845 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-12T01:51:24.845 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-12T01:51:24.845 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-12T01:51:24.845 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-12T01:51:24.845 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-12T01:51:24.845 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-12T01:51:24.845 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-12T01:51:24.845 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-12T01:51:24.845 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-12T01:51:24.845 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-12T01:51:24.846 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-12T01:51:24.846 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-12T01:51:24.846 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-12T01:51:24.846 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-12T01:51:24.846 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-12T01:51:24.846 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-12T01:51:24.846 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-12T01:51:24.846 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-12T01:51:24.846 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-12T01:51:24.846 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-12T01:51:24.846 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-12T01:51:24.846 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-12T01:51:24.846 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-12T01:51:24.846 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-12T01:51:24.847 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-12T01:51:24.847 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-12T01:51:24.847 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-12T01:51:24.847 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-12T01:51:24.847 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-12T01:51:24.847 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-12T01:51:24.847 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-12T01:51:24.847 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-12T01:51:24.847 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-12T01:51:24.847 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-12T01:51:24.847 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-12T01:51:24.847 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-12T01:51:24.847 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-12T01:51:24.848 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-12T01:51:24.848 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-12T01:51:24.848 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-12T01:51:24.848 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 149/164 2024-09-12T01:51:24.848 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-12T01:51:24.848 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 151/164 2024-09-12T01:51:24.848 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-12T01:51:24.848 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 153/164 2024-09-12T01:51:24.848 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-12T01:51:24.848 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 155/164 2024-09-12T01:51:24.848 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-12T01:51:24.848 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 157/164 2024-09-12T01:51:24.848 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-12T01:51:24.849 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 159/164 2024-09-12T01:51:24.849 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-12T01:51:24.849 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-12T01:51:24.849 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-12T01:51:24.849 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-12T01:51:27.876 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-12T01:51:27.876 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:51:27.876 INFO:teuthology.orchestra.run.smithi016.stdout:Upgraded: 2024-09-12T01:51:27.877 INFO:teuthology.orchestra.run.smithi016.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-12T01:51:27.877 INFO:teuthology.orchestra.run.smithi016.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-12T01:51:27.877 INFO:teuthology.orchestra.run.smithi016.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-12T01:51:27.877 INFO:teuthology.orchestra.run.smithi016.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-12T01:51:27.877 INFO:teuthology.orchestra.run.smithi016.stdout: librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:51:27.877 INFO:teuthology.orchestra.run.smithi016.stdout: librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:51:27.877 INFO:teuthology.orchestra.run.smithi016.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-12T01:51:27.877 INFO:teuthology.orchestra.run.smithi016.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-12T01:51:27.877 INFO:teuthology.orchestra.run.smithi016.stdout:Installed: 2024-09-12T01:51:27.877 INFO:teuthology.orchestra.run.smithi016.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-12T01:51:27.877 INFO:teuthology.orchestra.run.smithi016.stdout: automake-1.16.2-8.el9.noarch 2024-09-12T01:51:27.877 INFO:teuthology.orchestra.run.smithi016.stdout: bison-3.7.4-5.el9.x86_64 2024-09-12T01:51:27.878 INFO:teuthology.orchestra.run.smithi016.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-12T01:51:27.878 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:51:27.878 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:51:27.878 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:51:27.878 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:51:27.878 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T01:51:27.878 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-immutable-object-cache-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:51:27.878 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:51:27.878 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:51:27.878 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T01:51:27.878 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T01:51:27.878 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T01:51:27.879 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T01:51:27.879 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T01:51:27.879 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:51:27.879 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:51:27.879 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T01:51:27.879 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:51:27.879 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:51:27.879 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:51:27.879 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T01:51:27.879 INFO:teuthology.orchestra.run.smithi016.stdout: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T01:51:27.880 INFO:teuthology.orchestra.run.smithi016.stdout: cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:51:27.880 INFO:teuthology.orchestra.run.smithi016.stdout: cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:51:27.880 INFO:teuthology.orchestra.run.smithi016.stdout: cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T01:51:27.880 INFO:teuthology.orchestra.run.smithi016.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-12T01:51:27.880 INFO:teuthology.orchestra.run.smithi016.stdout: flex-2.6.4-9.el9.x86_64 2024-09-12T01:51:27.880 INFO:teuthology.orchestra.run.smithi016.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-12T01:51:27.880 INFO:teuthology.orchestra.run.smithi016.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-12T01:51:27.880 INFO:teuthology.orchestra.run.smithi016.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-12T01:51:27.880 INFO:teuthology.orchestra.run.smithi016.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-12T01:51:27.880 INFO:teuthology.orchestra.run.smithi016.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-12T01:51:27.880 INFO:teuthology.orchestra.run.smithi016.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-12T01:51:27.880 INFO:teuthology.orchestra.run.smithi016.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-12T01:51:27.881 INFO:teuthology.orchestra.run.smithi016.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-12T01:51:27.881 INFO:teuthology.orchestra.run.smithi016.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-12T01:51:27.881 INFO:teuthology.orchestra.run.smithi016.stdout: libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:51:27.881 INFO:teuthology.orchestra.run.smithi016.stdout: libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:51:27.881 INFO:teuthology.orchestra.run.smithi016.stdout: libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:51:27.881 INFO:teuthology.orchestra.run.smithi016.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-12T01:51:27.881 INFO:teuthology.orchestra.run.smithi016.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-12T01:51:27.881 INFO:teuthology.orchestra.run.smithi016.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-12T01:51:27.881 INFO:teuthology.orchestra.run.smithi016.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-12T01:51:27.881 INFO:teuthology.orchestra.run.smithi016.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-12T01:51:27.881 INFO:teuthology.orchestra.run.smithi016.stdout: librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:51:27.881 INFO:teuthology.orchestra.run.smithi016.stdout: libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:51:27.881 INFO:teuthology.orchestra.run.smithi016.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-12T01:51:27.882 INFO:teuthology.orchestra.run.smithi016.stdout: librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:51:27.882 INFO:teuthology.orchestra.run.smithi016.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-12T01:51:27.882 INFO:teuthology.orchestra.run.smithi016.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-12T01:51:27.882 INFO:teuthology.orchestra.run.smithi016.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-12T01:51:27.882 INFO:teuthology.orchestra.run.smithi016.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-12T01:51:27.882 INFO:teuthology.orchestra.run.smithi016.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-12T01:51:27.882 INFO:teuthology.orchestra.run.smithi016.stdout: lua-5.4.4-4.el9.x86_64 2024-09-12T01:51:27.882 INFO:teuthology.orchestra.run.smithi016.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-12T01:51:27.882 INFO:teuthology.orchestra.run.smithi016.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-12T01:51:27.883 INFO:teuthology.orchestra.run.smithi016.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-12T01:51:27.883 INFO:teuthology.orchestra.run.smithi016.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-12T01:51:27.883 INFO:teuthology.orchestra.run.smithi016.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-12T01:51:27.883 INFO:teuthology.orchestra.run.smithi016.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-12T01:51:27.883 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql-13.16-1.el9.x86_64 2024-09-12T01:51:27.883 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-12T01:51:27.883 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-12T01:51:27.883 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-12T01:51:27.883 INFO:teuthology.orchestra.run.smithi016.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-12T01:51:27.883 INFO:teuthology.orchestra.run.smithi016.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-12T01:51:27.883 INFO:teuthology.orchestra.run.smithi016.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-12T01:51:27.883 INFO:teuthology.orchestra.run.smithi016.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-12T01:51:27.883 INFO:teuthology.orchestra.run.smithi016.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-12T01:51:27.884 INFO:teuthology.orchestra.run.smithi016.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-12T01:51:27.884 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-12T01:51:27.884 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:51:27.884 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:51:27.884 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:51:27.884 INFO:teuthology.orchestra.run.smithi016.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-12T01:51:27.884 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-12T01:51:27.884 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-12T01:51:27.884 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-12T01:51:27.884 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-12T01:51:27.884 INFO:teuthology.orchestra.run.smithi016.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-12T01:51:27.884 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-12T01:51:27.884 INFO:teuthology.orchestra.run.smithi016.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-12T01:51:27.885 INFO:teuthology.orchestra.run.smithi016.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-12T01:51:27.885 INFO:teuthology.orchestra.run.smithi016.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-12T01:51:27.885 INFO:teuthology.orchestra.run.smithi016.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-12T01:51:27.885 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-12T01:51:27.885 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-12T01:51:27.885 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-12T01:51:27.885 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-12T01:51:27.885 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-12T01:51:27.885 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-12T01:51:27.885 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-12T01:51:27.885 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-12T01:51:27.885 INFO:teuthology.orchestra.run.smithi016.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-12T01:51:27.885 INFO:teuthology.orchestra.run.smithi016.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-12T01:51:27.886 INFO:teuthology.orchestra.run.smithi016.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-12T01:51:27.886 INFO:teuthology.orchestra.run.smithi016.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-12T01:51:27.886 INFO:teuthology.orchestra.run.smithi016.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-12T01:51:27.886 INFO:teuthology.orchestra.run.smithi016.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-12T01:51:27.886 INFO:teuthology.orchestra.run.smithi016.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-12T01:51:27.886 INFO:teuthology.orchestra.run.smithi016.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-12T01:51:27.886 INFO:teuthology.orchestra.run.smithi016.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-12T01:51:27.886 INFO:teuthology.orchestra.run.smithi016.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-12T01:51:27.886 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-12T01:51:27.887 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ply-3.11-14.el9.noarch 2024-09-12T01:51:27.887 INFO:teuthology.orchestra.run.smithi016.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-12T01:51:27.887 INFO:teuthology.orchestra.run.smithi016.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-12T01:51:27.887 INFO:teuthology.orchestra.run.smithi016.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-12T01:51:27.887 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-12T01:51:27.887 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-12T01:51:27.887 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-12T01:51:27.887 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-12T01:51:27.887 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-12T01:51:27.887 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-12T01:51:27.887 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:51:27.887 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:51:27.887 INFO:teuthology.orchestra.run.smithi016.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-12T01:51:27.888 INFO:teuthology.orchestra.run.smithi016.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-12T01:51:27.888 INFO:teuthology.orchestra.run.smithi016.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-12T01:51:27.888 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:51:27.888 INFO:teuthology.orchestra.run.smithi016.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-12T01:51:27.888 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-12T01:51:27.888 INFO:teuthology.orchestra.run.smithi016.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-12T01:51:27.888 INFO:teuthology.orchestra.run.smithi016.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-12T01:51:27.888 INFO:teuthology.orchestra.run.smithi016.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-12T01:51:27.888 INFO:teuthology.orchestra.run.smithi016.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-12T01:51:27.888 INFO:teuthology.orchestra.run.smithi016.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-12T01:51:27.888 INFO:teuthology.orchestra.run.smithi016.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-12T01:51:27.888 INFO:teuthology.orchestra.run.smithi016.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-12T01:51:27.888 INFO:teuthology.orchestra.run.smithi016.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-12T01:51:27.889 INFO:teuthology.orchestra.run.smithi016.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-12T01:51:27.889 INFO:teuthology.orchestra.run.smithi016.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-12T01:51:27.889 INFO:teuthology.orchestra.run.smithi016.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-12T01:51:27.889 INFO:teuthology.orchestra.run.smithi016.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-12T01:51:27.889 INFO:teuthology.orchestra.run.smithi016.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-12T01:51:27.889 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:51:27.889 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:51:27.889 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:51:27.889 INFO:teuthology.orchestra.run.smithi016.stdout: re2-1:20211101-20.el9.x86_64 2024-09-12T01:51:27.889 INFO:teuthology.orchestra.run.smithi016.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-12T01:51:27.889 INFO:teuthology.orchestra.run.smithi016.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-12T01:51:27.889 INFO:teuthology.orchestra.run.smithi016.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-12T01:51:27.889 INFO:teuthology.orchestra.run.smithi016.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-12T01:51:27.890 INFO:teuthology.orchestra.run.smithi016.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-12T01:51:27.890 INFO:teuthology.orchestra.run.smithi016.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-12T01:51:27.890 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:51:27.890 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2024-09-12T01:51:28.283 DEBUG:teuthology.parallel:result is None 2024-09-12T01:51:28.284 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T01:51:28.437 DEBUG:teuthology.orchestra.run.smithi016:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-12T01:51:28.460 INFO:teuthology.orchestra.run.smithi016.stdout:19.1.1-298.gb53b6d3e.el9 2024-09-12T01:51:28.461 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-298.gb53b6d3e.el9 2024-09-12T01:51:28.461 INFO:teuthology.task.install:The correct ceph version 19.1.1-298.gb53b6d3e is installed. 2024-09-12T01:51:28.463 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T01:51:28.609 DEBUG:teuthology.orchestra.run.smithi033:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-12T01:51:28.635 INFO:teuthology.orchestra.run.smithi033.stdout:19.1.1-298.gb53b6d3e.el9 2024-09-12T01:51:28.636 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-298.gb53b6d3e.el9 2024-09-12T01:51:28.636 INFO:teuthology.task.install:The correct ceph version 19.1.1-298.gb53b6d3e is installed. 2024-09-12T01:51:28.637 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T01:51:28.779 DEBUG:teuthology.orchestra.run.smithi184:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-12T01:51:28.806 INFO:teuthology.orchestra.run.smithi184.stdout:19.1.1-298.gb53b6d3e.el9 2024-09-12T01:51:28.806 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-298.gb53b6d3e.el9 2024-09-12T01:51:28.806 INFO:teuthology.task.install:The correct ceph version 19.1.1-298.gb53b6d3e is installed. 2024-09-12T01:51:28.808 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-12T01:51:28.808 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-12T01:51:28.808 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-12T01:51:28.840 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-09-12T01:51:28.840 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-12T01:51:28.872 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-12T01:51:28.872 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-12T01:51:28.905 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-12T01:51:28.905 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-12T01:51:28.905 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/usr/bin/daemon-helper 2024-09-12T01:51:28.934 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-12T01:51:29.002 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-09-12T01:51:29.002 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/usr/bin/daemon-helper 2024-09-12T01:51:29.031 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-12T01:51:29.100 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-12T01:51:29.101 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd of=/usr/bin/daemon-helper 2024-09-12T01:51:29.130 DEBUG:teuthology.orchestra.run.smithi184:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-12T01:51:29.200 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-12T01:51:29.200 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-12T01:51:29.200 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-12T01:51:29.228 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-12T01:51:29.295 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-09-12T01:51:29.296 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-12T01:51:29.325 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-12T01:51:29.394 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-12T01:51:29.394 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-12T01:51:29.422 DEBUG:teuthology.orchestra.run.smithi184:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-12T01:51:29.490 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-12T01:51:29.491 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-12T01:51:29.491 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/usr/bin/stdin-killer 2024-09-12T01:51:29.519 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-12T01:51:29.589 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-09-12T01:51:29.589 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/usr/bin/stdin-killer 2024-09-12T01:51:29.618 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-12T01:51:29.687 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-12T01:51:29.687 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd of=/usr/bin/stdin-killer 2024-09-12T01:51:29.715 DEBUG:teuthology.orchestra.run.smithi184:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-12T01:51:29.786 INFO:teuthology.run_tasks:Running task cephadm... 2024-09-12T01:51:29.891 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-09-12T01:51:29.891 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T01:51:29.891 INFO:tasks.cephadm:Cluster fsid is 879f3b56-70a9-11ef-bcea-c7b262605968 2024-09-12T01:51:29.892 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-09-12T01:51:29.892 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.16', 'mon.b': '172.21.15.33', 'mon.c': '172.21.15.184'} 2024-09-12T01:51:29.892 INFO:tasks.cephadm:First mon is mon.a on smithi016 2024-09-12T01:51:29.892 INFO:tasks.cephadm:First mgr is x 2024-09-12T01:51:29.892 INFO:tasks.cephadm:Normalizing hostnames... 2024-09-12T01:51:29.892 DEBUG:teuthology.orchestra.run.smithi016:> sudo hostname $(hostname -s) 2024-09-12T01:51:29.924 DEBUG:teuthology.orchestra.run.smithi033:> sudo hostname $(hostname -s) 2024-09-12T01:51:29.954 DEBUG:teuthology.orchestra.run.smithi184:> sudo hostname $(hostname -s) 2024-09-12T01:51:29.986 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-09-12T01:51:29.986 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T01:51:30.131 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/56038/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.1.1-298-gb53b6d3e', 'node_name': '172.21.5.36+adami06', 'job_name': 'ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.1.1-298.gb53b6d3e'}, 'url': 'https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-09-09 17:24:58.599585', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'squid', 'chacra_url': 'https://2.chacra.ceph.com/repos/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-09-12T01:51:30.277 INFO:tasks.util.chacra:got chacra host 2.chacra.ceph.com, ref squid, sha1 b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T01:51:30.278 INFO:tasks.cephadm:Discovered cachra url: https://2.chacra.ceph.com/binaries/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/x86_64/flavors/default/cephadm 2024-09-12T01:51:30.278 INFO:tasks.cephadm:Downloading cephadm from url: https://2.chacra.ceph.com/binaries/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/x86_64/flavors/default/cephadm 2024-09-12T01:51:30.278 DEBUG:teuthology.orchestra.run.smithi016:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-12T01:51:30.675 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r--. 1 ubuntu ubuntu 784498 Sep 12 01:51 /home/ubuntu/cephtest/cephadm 2024-09-12T01:51:30.676 DEBUG:teuthology.orchestra.run.smithi033:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-12T01:51:31.063 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-r--r--. 1 ubuntu ubuntu 784498 Sep 12 01:51 /home/ubuntu/cephtest/cephadm 2024-09-12T01:51:31.063 DEBUG:teuthology.orchestra.run.smithi184:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-12T01:51:31.446 INFO:teuthology.orchestra.run.smithi184.stdout:-rw-r--r--. 1 ubuntu ubuntu 784498 Sep 12 01:51 /home/ubuntu/cephtest/cephadm 2024-09-12T01:51:31.447 DEBUG:teuthology.orchestra.run.smithi016:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-12T01:51:31.467 DEBUG:teuthology.orchestra.run.smithi033:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-12T01:51:31.486 DEBUG:teuthology.orchestra.run.smithi184:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-12T01:51:31.522 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc on all hosts... 2024-09-12T01:51:31.523 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc pull 2024-09-12T01:51:31.525 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc pull 2024-09-12T01:51:31.529 DEBUG:teuthology.orchestra.run.smithi184:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc pull 2024-09-12T01:51:33.038 INFO:teuthology.orchestra.run.smithi033.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc... 2024-09-12T01:51:33.052 INFO:teuthology.orchestra.run.smithi184.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc... 2024-09-12T01:51:33.165 INFO:teuthology.orchestra.run.smithi016.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc... 2024-09-12T01:52:00.242 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T01:52:00.242 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_version": "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)", 2024-09-12T01:52:00.242 INFO:teuthology.orchestra.run.smithi016.stdout: "image_id": "54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594", 2024-09-12T01:52:00.242 INFO:teuthology.orchestra.run.smithi016.stdout: "repo_digests": [ 2024-09-12T01:52:00.242 INFO:teuthology.orchestra.run.smithi016.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66" 2024-09-12T01:52:00.242 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-09-12T01:52:00.242 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T01:52:02.045 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-12T01:52:02.046 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph_version": "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)", 2024-09-12T01:52:02.046 INFO:teuthology.orchestra.run.smithi033.stdout: "image_id": "54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594", 2024-09-12T01:52:02.046 INFO:teuthology.orchestra.run.smithi033.stdout: "repo_digests": [ 2024-09-12T01:52:02.046 INFO:teuthology.orchestra.run.smithi033.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66" 2024-09-12T01:52:02.046 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-12T01:52:02.046 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-12T01:52:03.579 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-12T01:52:03.579 INFO:teuthology.orchestra.run.smithi184.stdout: "ceph_version": "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)", 2024-09-12T01:52:03.579 INFO:teuthology.orchestra.run.smithi184.stdout: "image_id": "54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594", 2024-09-12T01:52:03.579 INFO:teuthology.orchestra.run.smithi184.stdout: "repo_digests": [ 2024-09-12T01:52:03.579 INFO:teuthology.orchestra.run.smithi184.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66" 2024-09-12T01:52:03.579 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-12T01:52:03.580 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-12T01:52:03.604 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /etc/ceph 2024-09-12T01:52:03.639 DEBUG:teuthology.orchestra.run.smithi033:> sudo mkdir -p /etc/ceph 2024-09-12T01:52:03.673 DEBUG:teuthology.orchestra.run.smithi184:> sudo mkdir -p /etc/ceph 2024-09-12T01:52:03.701 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 777 /etc/ceph 2024-09-12T01:52:03.732 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod 777 /etc/ceph 2024-09-12T01:52:03.761 DEBUG:teuthology.orchestra.run.smithi184:> sudo chmod 777 /etc/ceph 2024-09-12T01:52:03.786 INFO:tasks.cephadm:Writing seed config... 2024-09-12T01:52:03.787 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-09-12T01:52:03.787 INFO:tasks.cephadm: override: [client] debug client = 20 2024-09-12T01:52:03.788 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-09-12T01:52:03.788 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-09-12T01:52:03.788 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-09-12T01:52:03.788 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-09-12T01:52:03.788 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-09-12T01:52:03.788 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-09-12T01:52:03.788 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-09-12T01:52:03.788 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-09-12T01:52:03.788 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-09-12T01:52:03.788 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-09-12T01:52:03.788 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-09-12T01:52:03.788 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-09-12T01:52:03.788 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-09-12T01:52:03.789 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-09-12T01:52:03.789 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-09-12T01:52:03.789 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2024-09-12T01:52:03.789 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-09-12T01:52:03.789 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-09-12T01:52:03.789 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-09-12T01:52:03.789 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-09-12T01:52:03.789 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-09-12T01:52:03.789 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-09-12T01:52:03.789 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-09-12T01:52:03.789 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-09-12T01:52:03.789 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-09-12T01:52:03.789 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-09-12T01:52:03.790 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-09-12T01:52:03.790 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-09-12T01:52:03.790 INFO:tasks.cephadm: override: [mon] mon warn on pool no app = False 2024-09-12T01:52:03.790 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2024-09-12T01:52:03.790 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2024-09-12T01:52:03.790 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2024-09-12T01:52:03.790 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-09-12T01:52:03.790 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-09-12T01:52:03.790 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2024-09-12T01:52:03.790 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2024-09-12T01:52:03.790 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-09-12T01:52:03.790 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-09-12T01:52:03.790 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2024-09-12T01:52:03.790 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-09-12T01:52:03.791 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-09-12T01:52:03.791 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-09-12T01:52:03.791 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-09-12T01:52:03.791 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-09-12T01:52:03.791 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-09-12T01:52:03.791 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-09-12T01:52:03.791 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-09-12T01:52:03.792 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-12T01:52:03.792 DEBUG:teuthology.orchestra.run.smithi016:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-09-12T01:52:03.811 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 879f3b56-70a9-11ef-bcea-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 mon warn on pool no app = False [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_bal_replicate_threshold = 1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-09-12T01:52:03.811 DEBUG:teuthology.orchestra.run.smithi016:mon.a> sudo journalctl -f -n 0 -u ceph-879f3b56-70a9-11ef-bcea-c7b262605968@mon.a.service 2024-09-12T01:52:03.854 DEBUG:teuthology.orchestra.run.smithi016:mgr.x> sudo journalctl -f -n 0 -u ceph-879f3b56-70a9-11ef-bcea-c7b262605968@mgr.x.service 2024-09-12T01:52:03.897 INFO:tasks.cephadm:Bootstrapping... 2024-09-12T01:52:03.897 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc -v bootstrap --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.16 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-09-12T01:52:04.113 INFO:teuthology.orchestra.run.smithi016.stdout:-------------------------------------------------------------------------------- 2024-09-12T01:52:04.113 INFO:teuthology.orchestra.run.smithi016.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc', '-v', 'bootstrap', '--fsid', '879f3b56-70a9-11ef-bcea-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.16', '--skip-admin-label'] 2024-09-12T01:52:04.113 INFO:teuthology.orchestra.run.smithi016.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-09-12T01:52:04.113 INFO:teuthology.orchestra.run.smithi016.stdout:Verifying podman|docker is present... 2024-09-12T01:52:04.138 INFO:teuthology.orchestra.run.smithi016.stdout:/bin/podman: stdout 5.2.2 2024-09-12T01:52:04.139 INFO:teuthology.orchestra.run.smithi016.stdout:Verifying lvm2 is present... 2024-09-12T01:52:04.139 INFO:teuthology.orchestra.run.smithi016.stdout:Verifying time synchronization is in place... 2024-09-12T01:52:04.146 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-12T01:52:04.146 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-12T01:52:04.153 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-12T01:52:04.153 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout inactive 2024-09-12T01:52:04.160 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout enabled 2024-09-12T01:52:04.166 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout active 2024-09-12T01:52:04.167 INFO:teuthology.orchestra.run.smithi016.stdout:Unit chronyd.service is enabled and running 2024-09-12T01:52:04.167 INFO:teuthology.orchestra.run.smithi016.stdout:Repeating the final host check... 2024-09-12T01:52:04.190 INFO:teuthology.orchestra.run.smithi016.stdout:/bin/podman: stdout 5.2.2 2024-09-12T01:52:04.190 INFO:teuthology.orchestra.run.smithi016.stdout:podman (/bin/podman) version 5.2.2 is present 2024-09-12T01:52:04.191 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl is present 2024-09-12T01:52:04.191 INFO:teuthology.orchestra.run.smithi016.stdout:lvcreate is present 2024-09-12T01:52:04.197 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-12T01:52:04.197 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-12T01:52:04.203 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-12T01:52:04.203 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout inactive 2024-09-12T01:52:04.210 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout enabled 2024-09-12T01:52:04.216 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout active 2024-09-12T01:52:04.217 INFO:teuthology.orchestra.run.smithi016.stdout:Unit chronyd.service is enabled and running 2024-09-12T01:52:04.217 INFO:teuthology.orchestra.run.smithi016.stdout:Host looks OK 2024-09-12T01:52:04.217 INFO:teuthology.orchestra.run.smithi016.stdout:Cluster fsid: 879f3b56-70a9-11ef-bcea-c7b262605968 2024-09-12T01:52:04.217 INFO:teuthology.orchestra.run.smithi016.stdout:Acquiring lock 140275706557440 on /run/cephadm/879f3b56-70a9-11ef-bcea-c7b262605968.lock 2024-09-12T01:52:04.217 INFO:teuthology.orchestra.run.smithi016.stdout:Lock 140275706557440 acquired on /run/cephadm/879f3b56-70a9-11ef-bcea-c7b262605968.lock 2024-09-12T01:52:04.217 INFO:teuthology.orchestra.run.smithi016.stdout:Verifying IP 172.21.15.16 port 3300 ... 2024-09-12T01:52:04.218 INFO:teuthology.orchestra.run.smithi016.stdout:Verifying IP 172.21.15.16 port 6789 ... 2024-09-12T01:52:04.218 INFO:teuthology.orchestra.run.smithi016.stdout:Base mon IP(s) is [172.21.15.16:3300, 172.21.15.16:6789], mon addrv is [v2:172.21.15.16:3300,v1:172.21.15.16:6789] 2024-09-12T01:52:04.221 INFO:teuthology.orchestra.run.smithi016.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.16 metric 100 2024-09-12T01:52:04.221 INFO:teuthology.orchestra.run.smithi016.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.16 metric 100 2024-09-12T01:52:04.223 INFO:teuthology.orchestra.run.smithi016.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-09-12T01:52:04.223 INFO:teuthology.orchestra.run.smithi016.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 256 pref medium 2024-09-12T01:52:04.224 INFO:teuthology.orchestra.run.smithi016.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 1024 expires 1796sec hoplimit 64 pref medium 2024-09-12T01:52:04.226 INFO:teuthology.orchestra.run.smithi016.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-09-12T01:52:04.226 INFO:teuthology.orchestra.run.smithi016.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-09-12T01:52:04.226 INFO:teuthology.orchestra.run.smithi016.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-12T01:52:04.226 INFO:teuthology.orchestra.run.smithi016.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-09-12T01:52:04.226 INFO:teuthology.orchestra.run.smithi016.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:1564/64 scope link 2024-09-12T01:52:04.226 INFO:teuthology.orchestra.run.smithi016.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-12T01:52:04.227 INFO:teuthology.orchestra.run.smithi016.stdout:Mon IP `172.21.15.16` is in CIDR network `172.21.0.0/20` 2024-09-12T01:52:04.227 INFO:teuthology.orchestra.run.smithi016.stdout:Mon IP `172.21.15.16` is in CIDR network `172.21.0.0/20` 2024-09-12T01:52:04.227 INFO:teuthology.orchestra.run.smithi016.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-09-12T01:52:04.228 INFO:teuthology.orchestra.run.smithi016.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-09-12T01:52:04.228 INFO:teuthology.orchestra.run.smithi016.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc... 2024-09-12T01:52:04.735 INFO:teuthology.orchestra.run.smithi016.stdout:/bin/podman: stdout 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 2024-09-12T01:52:04.736 INFO:teuthology.orchestra.run.smithi016.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc... 2024-09-12T01:52:04.736 INFO:teuthology.orchestra.run.smithi016.stdout:/bin/podman: stderr Getting image source signatures 2024-09-12T01:52:04.736 INFO:teuthology.orchestra.run.smithi016.stdout:/bin/podman: stderr Copying blob sha256:e8b54c863393b7a8216a71737771b73b16a8e43ec7acf927c7faa175c255e551 2024-09-12T01:52:04.736 INFO:teuthology.orchestra.run.smithi016.stdout:/bin/podman: stderr Copying blob sha256:60180723200353b4e13b39fd162aecaf1bc097c859c7c6537324229e97c78567 2024-09-12T01:52:04.736 INFO:teuthology.orchestra.run.smithi016.stdout:/bin/podman: stderr Copying config sha256:54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 2024-09-12T01:52:04.736 INFO:teuthology.orchestra.run.smithi016.stdout:/bin/podman: stderr Writing manifest to image destination 2024-09-12T01:52:05.665 INFO:teuthology.orchestra.run.smithi016.stdout:ceph: stdout ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable) 2024-09-12T01:52:05.666 INFO:teuthology.orchestra.run.smithi016.stdout:Ceph version: ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable) 2024-09-12T01:52:05.666 INFO:teuthology.orchestra.run.smithi016.stdout:Extracting ceph user uid/gid from container image... 2024-09-12T01:52:06.688 INFO:teuthology.orchestra.run.smithi016.stdout:stat: stdout 167 167 2024-09-12T01:52:06.689 INFO:teuthology.orchestra.run.smithi016.stdout:Creating initial keys... 2024-09-12T01:52:07.625 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph-authtool: stdout AQBHSeJmk2bjABAABYYEM7i7bddn/HHW1GsRmw== 2024-09-12T01:52:08.581 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph-authtool: stdout AQBHSeJmKUUeOxAA9wlbzYJFOOcxngJFY8fuIQ== 2024-09-12T01:52:09.553 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph-authtool: stdout AQBISeJmsEB/NRAAiSg1b4CpFsFMjxgOZsnUjg== 2024-09-12T01:52:09.554 INFO:teuthology.orchestra.run.smithi016.stdout:Creating initial monmap... 2024-09-12T01:52:10.409 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-12T01:52:10.409 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-09-12T01:52:10.410 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 879f3b56-70a9-11ef-bcea-c7b262605968 2024-09-12T01:52:10.410 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-12T01:52:10.410 INFO:teuthology.orchestra.run.smithi016.stdout:monmaptool for a [v2:172.21.15.16:3300,v1:172.21.15.16:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-12T01:52:10.410 INFO:teuthology.orchestra.run.smithi016.stdout:setting min_mon_release = quincy 2024-09-12T01:52:10.410 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/monmaptool: set fsid to 879f3b56-70a9-11ef-bcea-c7b262605968 2024-09-12T01:52:10.410 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-12T01:52:10.410 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:52:10.410 INFO:teuthology.orchestra.run.smithi016.stdout:Creating mon... 2024-09-12T01:52:11.357 INFO:teuthology.orchestra.run.smithi016.stdout:create mon.a on 2024-09-12T01:52:11.576 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-09-12T01:52:11.735 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-09-12T01:52:11.927 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-879f3b56-70a9-11ef-bcea-c7b262605968.target → /etc/systemd/system/ceph-879f3b56-70a9-11ef-bcea-c7b262605968.target. 2024-09-12T01:52:11.927 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-879f3b56-70a9-11ef-bcea-c7b262605968.target → /etc/systemd/system/ceph-879f3b56-70a9-11ef-bcea-c7b262605968.target. 2024-09-12T01:52:12.190 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-879f3b56-70a9-11ef-bcea-c7b262605968@mon.a 2024-09-12T01:52:12.190 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stderr Failed to reset failed state of unit ceph-879f3b56-70a9-11ef-bcea-c7b262605968@mon.a.service: Unit ceph-879f3b56-70a9-11ef-bcea-c7b262605968@mon.a.service not loaded. 2024-09-12T01:52:12.360 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-879f3b56-70a9-11ef-bcea-c7b262605968.target.wants/ceph-879f3b56-70a9-11ef-bcea-c7b262605968@mon.a.service → /etc/systemd/system/ceph-879f3b56-70a9-11ef-bcea-c7b262605968@.service. 2024-09-12T01:52:12.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:12 smithi016 podman[30564]: 2024-09-12 01:52:12.506537458 +0000 UTC m=+0.016421427 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T01:52:12.899 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-12T01:52:12.899 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout disabled 2024-09-12T01:52:12.906 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-12T01:52:12.906 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout inactive 2024-09-12T01:52:12.906 INFO:teuthology.orchestra.run.smithi016.stdout:firewalld.service is not enabled 2024-09-12T01:52:12.906 INFO:teuthology.orchestra.run.smithi016.stdout:Not possible to enable service . firewalld.service is not available 2024-09-12T01:52:12.906 INFO:teuthology.orchestra.run.smithi016.stdout:Waiting for mon to start... 2024-09-12T01:52:12.906 INFO:teuthology.orchestra.run.smithi016.stdout:Waiting for mon... 2024-09-12T01:52:13.026 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:12 smithi016 podman[30564]: 2024-09-12 01:52:12.800706426 +0000 UTC m=+0.310590392 container init efac6124eaf2f9feeb62dd5e507b07625aa9f6ecd2dc5b4261fcf21d4765c379 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mon-a, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-12T01:52:13.027 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:12 smithi016 podman[30564]: 2024-09-12 01:52:12.803927333 +0000 UTC m=+0.313811292 container start efac6124eaf2f9feeb62dd5e507b07625aa9f6ecd2dc5b4261fcf21d4765c379 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_CLEAN=True) 2024-09-12T01:52:13.027 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:12 smithi016 ceph-mon[30583]: mkfs 879f3b56-70a9-11ef-bcea-c7b262605968 2024-09-12T01:52:13.027 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:12 smithi016 ceph-mon[30583]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-12T01:52:13.027 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:12 smithi016 bash[30564]: efac6124eaf2f9feeb62dd5e507b07625aa9f6ecd2dc5b4261fcf21d4765c379 2024-09-12T01:52:13.027 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:12 smithi016 systemd[1]: Started Ceph mon.a for 879f3b56-70a9-11ef-bcea-c7b262605968. 2024-09-12T01:52:13.965 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:13 smithi016 ceph-mon[30583]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-12T01:52:13.965 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:13 smithi016 ceph-mon[30583]: monmap epoch 1 2024-09-12T01:52:13.966 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:13 smithi016 ceph-mon[30583]: fsid 879f3b56-70a9-11ef-bcea-c7b262605968 2024-09-12T01:52:13.966 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:13 smithi016 ceph-mon[30583]: last_changed 2024-09-12T01:52:09.859051+0000 2024-09-12T01:52:13.966 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:13 smithi016 ceph-mon[30583]: created 2024-09-12T01:52:09.859051+0000 2024-09-12T01:52:13.966 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:13 smithi016 ceph-mon[30583]: min_mon_release 19 (squid) 2024-09-12T01:52:13.966 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:13 smithi016 ceph-mon[30583]: election_strategy: 1 2024-09-12T01:52:13.966 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:13 smithi016 ceph-mon[30583]: 0: [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] mon.a 2024-09-12T01:52:13.966 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:13 smithi016 ceph-mon[30583]: fsmap 2024-09-12T01:52:13.966 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:13 smithi016 ceph-mon[30583]: osdmap e1: 0 total, 0 up, 0 in 2024-09-12T01:52:13.967 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:13 smithi016 ceph-mon[30583]: mgrmap e1: no daemons active 2024-09-12T01:52:13.967 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:13 smithi016 ceph-mon[30583]: from='client.? 172.21.15.16:0/3154596957' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-09-12T01:52:13.974 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout cluster: 2024-09-12T01:52:13.974 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout id: 879f3b56-70a9-11ef-bcea-c7b262605968 2024-09-12T01:52:13.974 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-09-12T01:52:13.974 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 2024-09-12T01:52:13.975 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout services: 2024-09-12T01:52:13.975 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.547966s) 2024-09-12T01:52:13.975 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-09-12T01:52:13.975 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-09-12T01:52:13.975 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 2024-09-12T01:52:13.975 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout data: 2024-09-12T01:52:13.975 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-09-12T01:52:13.976 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-09-12T01:52:13.976 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-09-12T01:52:13.976 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout pgs: 2024-09-12T01:52:13.976 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 2024-09-12T01:52:13.976 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:13.315+0000 7f6613d7b640 1 Processor -- start 2024-09-12T01:52:13.976 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:13.315+0000 7f6613d7b640 1 -- start start 2024-09-12T01:52:13.976 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:13.315+0000 7f6613d7b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f660c107780 0x7f660c107b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:13.976 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:13.315+0000 7f6613d7b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f660c108150 con 0x7f660c107780 2024-09-12T01:52:13.976 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:13.316+0000 7f6611b17640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f660c107780 0x7f660c107b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:13.976 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:13.316+0000 7f6611b17640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f660c107780 0x7f660c107b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43376/0 (socket says 172.21.15.16:43376) 2024-09-12T01:52:13.977 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:13.316+0000 7f6611b17640 1 -- 172.21.15.16:0/3581072347 learned_addr learned my addr 172.21.15.16:0/3581072347 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:52:13.977 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:13.316+0000 7f6611b17640 1 -- 172.21.15.16:0/3581072347 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f660c108980 con 0x7f660c107780 2024-09-12T01:52:13.977 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:13.318+0000 7f6611b17640 1 --2- 172.21.15.16:0/3581072347 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f660c107780 0x7f660c107b80 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f6608009820 tx=0x7f660802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56d993d6f18d20d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:52:13.977 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:13.318+0000 7f6610b15640 1 -- 172.21.15.16:0/3581072347 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6608037070 con 0x7f660c107780 2024-09-12T01:52:13.977 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:13.318+0000 7f6610b15640 1 -- 172.21.15.16:0/3581072347 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6608032440 con 0x7f660c107780 2024-09-12T01:52:13.977 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:13.318+0000 7f6610b15640 1 -- 172.21.15.16:0/3581072347 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6608032740 con 0x7f660c107780 2024-09-12T01:52:13.977 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:13.319+0000 7f6613d7b640 1 -- 172.21.15.16:0/3581072347 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f660c107780 msgr2=0x7f660c107b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:52:13.977 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:13.319+0000 7f6613d7b640 1 --2- 172.21.15.16:0/3581072347 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f660c107780 0x7f660c107b80 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f6608009820 tx=0x7f660802a270 comp rx=0 tx=0).stop 2024-09-12T01:52:13.977 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:13.319+0000 7f6613d7b640 1 -- 172.21.15.16:0/3581072347 shutdown_connections 2024-09-12T01:52:13.977 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:13.319+0000 7f6613d7b640 1 --2- 172.21.15.16:0/3581072347 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f660c107780 0x7f660c107b80 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:13.978 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:13.319+0000 7f6613d7b640 1 -- 172.21.15.16:0/3581072347 >> 172.21.15.16:0/3581072347 conn(0x7f660c102f30 msgr2=0x7f660c105350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:52:13.978 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:13.319+0000 7f6613d7b640 1 -- 172.21.15.16:0/3581072347 shutdown_connections 2024-09-12T01:52:13.978 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:13.319+0000 7f6613d7b640 1 -- 172.21.15.16:0/3581072347 wait complete. 2024-09-12T01:52:13.978 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:13.320+0000 7f6613d7b640 1 Processor -- start 2024-09-12T01:52:13.978 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:13.320+0000 7f6613d7b640 1 -- start start 2024-09-12T01:52:13.978 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:13.320+0000 7f6613d7b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f660c107780 0x7f660c07daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:13.978 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:13.320+0000 7f6613d7b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f660c108eb0 con 0x7f660c107780 2024-09-12T01:52:13.978 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:13.320+0000 7f6611b17640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f660c107780 0x7f660c07daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:13.978 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:13.320+0000 7f6611b17640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f660c107780 0x7f660c07daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43384/0 (socket says 172.21.15.16:43384) 2024-09-12T01:52:13.978 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:13.320+0000 7f6611b17640 1 -- 172.21.15.16:0/3154596957 learned_addr learned my addr 172.21.15.16:0/3154596957 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:52:13.979 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:13.321+0000 7f6611b17640 1 -- 172.21.15.16:0/3154596957 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f660c07e030 con 0x7f660c107780 2024-09-12T01:52:13.979 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:13.321+0000 7f6611b17640 1 --2- 172.21.15.16:0/3154596957 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f660c107780 0x7f660c07daf0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f6608009820 tx=0x7f66080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:52:13.979 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:13.321+0000 7f6602ffd640 1 -- 172.21.15.16:0/3154596957 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6608041070 con 0x7f660c107780 2024-09-12T01:52:13.979 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:13.321+0000 7f6602ffd640 1 -- 172.21.15.16:0/3154596957 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f660803c7a0 con 0x7f660c107780 2024-09-12T01:52:13.979 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:13.321+0000 7f6613d7b640 1 -- 172.21.15.16:0/3154596957 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f660c07c220 con 0x7f660c107780 2024-09-12T01:52:13.979 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:13.321+0000 7f6602ffd640 1 -- 172.21.15.16:0/3154596957 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6608037040 con 0x7f660c107780 2024-09-12T01:52:13.979 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:13.321+0000 7f6613d7b640 1 -- 172.21.15.16:0/3154596957 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f660c07c760 con 0x7f660c107780 2024-09-12T01:52:13.979 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:13.321+0000 7f6602ffd640 1 -- 172.21.15.16:0/3154596957 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f6608044070 con 0x7f660c107780 2024-09-12T01:52:13.979 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:13.321+0000 7f6602ffd640 1 -- 172.21.15.16:0/3154596957 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f6608030780 con 0x7f660c107780 2024-09-12T01:52:13.979 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:13.322+0000 7f6613d7b640 1 -- 172.21.15.16:0/3154596957 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f660c109500 con 0x7f660c107780 2024-09-12T01:52:13.979 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:13.324+0000 7f6602ffd640 1 -- 172.21.15.16:0/3154596957 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f660803cbb0 con 0x7f660c107780 2024-09-12T01:52:13.980 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:13.387+0000 7f6613d7b640 1 -- 172.21.15.16:0/3154596957 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f660c193db0 con 0x7f660c107780 2024-09-12T01:52:13.980 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:13.387+0000 7f6602ffd640 1 -- 172.21.15.16:0/3154596957 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7f660803cd90 con 0x7f660c107780 2024-09-12T01:52:13.980 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:13.388+0000 7f6613d7b640 1 -- 172.21.15.16:0/3154596957 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f660c107780 msgr2=0x7f660c07daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:52:13.980 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:13.388+0000 7f6613d7b640 1 --2- 172.21.15.16:0/3154596957 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f660c107780 0x7f660c07daf0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f6608009820 tx=0x7f66080046c0 comp rx=0 tx=0).stop 2024-09-12T01:52:13.980 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:13.388+0000 7f6613d7b640 1 -- 172.21.15.16:0/3154596957 shutdown_connections 2024-09-12T01:52:13.980 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:13.388+0000 7f6613d7b640 1 --2- 172.21.15.16:0/3154596957 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f660c107780 0x7f660c07daf0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:13.980 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:13.388+0000 7f6613d7b640 1 -- 172.21.15.16:0/3154596957 >> 172.21.15.16:0/3154596957 conn(0x7f660c102f30 msgr2=0x7f660c18f180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:52:13.980 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:13.388+0000 7f6613d7b640 1 -- 172.21.15.16:0/3154596957 shutdown_connections 2024-09-12T01:52:13.980 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:13.388+0000 7f6613d7b640 1 -- 172.21.15.16:0/3154596957 wait complete. 2024-09-12T01:52:13.980 INFO:teuthology.orchestra.run.smithi016.stdout:mon is available 2024-09-12T01:52:13.980 INFO:teuthology.orchestra.run.smithi016.stdout:Assimilating anything we can from ceph.conf... 2024-09-12T01:52:15.047 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 2024-09-12T01:52:15.047 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout [global] 2024-09-12T01:52:15.047 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout fsid = 879f3b56-70a9-11ef-bcea-c7b262605968 2024-09-12T01:52:15.047 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.16:3300,v1:172.21.15.16:6789] 2024-09-12T01:52:15.047 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-12T01:52:15.047 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-12T01:52:15.047 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-12T01:52:15.047 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-12T01:52:15.047 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 2024-09-12T01:52:15.047 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-12T01:52:15.048 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-12T01:52:15.048 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 2024-09-12T01:52:15.048 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout [osd] 2024-09-12T01:52:15.048 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-12T01:52:15.048 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-12T01:52:15.048 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-12T01:52:15.048 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-12T01:52:15.048 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-12T01:52:15.048 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:14.391+0000 7f7e641cc640 1 Processor -- start 2024-09-12T01:52:15.048 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:14.391+0000 7f7e641cc640 1 -- start start 2024-09-12T01:52:15.048 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:14.391+0000 7f7e641cc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e5c107780 0x7f7e5c107b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:15.048 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:14.391+0000 7f7e641cc640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7e5c108150 con 0x7f7e5c107780 2024-09-12T01:52:15.048 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:14.391+0000 7f7e61f68640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e5c107780 0x7f7e5c107b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:15.049 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:14.391+0000 7f7e61f68640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e5c107780 0x7f7e5c107b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43392/0 (socket says 172.21.15.16:43392) 2024-09-12T01:52:15.049 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:14.391+0000 7f7e61f68640 1 -- 172.21.15.16:0/260765858 learned_addr learned my addr 172.21.15.16:0/260765858 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:52:15.049 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:14.392+0000 7f7e61f68640 1 -- 172.21.15.16:0/260765858 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e5c108980 con 0x7f7e5c107780 2024-09-12T01:52:15.049 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:14.392+0000 7f7e61f68640 1 --2- 172.21.15.16:0/260765858 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e5c107780 0x7f7e5c107b80 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f7e58009820 tx=0x7f7e5802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78f8501088c15a99 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:52:15.049 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:14.393+0000 7f7e60f66640 1 -- 172.21.15.16:0/260765858 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7e58037070 con 0x7f7e5c107780 2024-09-12T01:52:15.049 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:14.393+0000 7f7e60f66640 1 -- 172.21.15.16:0/260765858 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e58032440 con 0x7f7e5c107780 2024-09-12T01:52:15.049 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:14.393+0000 7f7e60f66640 1 -- 172.21.15.16:0/260765858 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7e58032740 con 0x7f7e5c107780 2024-09-12T01:52:15.049 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:14.393+0000 7f7e641cc640 1 -- 172.21.15.16:0/260765858 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e5c107780 msgr2=0x7f7e5c107b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:52:15.049 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:14.393+0000 7f7e641cc640 1 --2- 172.21.15.16:0/260765858 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e5c107780 0x7f7e5c107b80 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f7e58009820 tx=0x7f7e5802a270 comp rx=0 tx=0).stop 2024-09-12T01:52:15.049 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:14.393+0000 7f7e641cc640 1 -- 172.21.15.16:0/260765858 shutdown_connections 2024-09-12T01:52:15.049 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:14.393+0000 7f7e641cc640 1 --2- 172.21.15.16:0/260765858 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e5c107780 0x7f7e5c107b80 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:15.049 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:14.393+0000 7f7e641cc640 1 -- 172.21.15.16:0/260765858 >> 172.21.15.16:0/260765858 conn(0x7f7e5c102f30 msgr2=0x7f7e5c105350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:52:15.049 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:14.393+0000 7f7e641cc640 1 -- 172.21.15.16:0/260765858 shutdown_connections 2024-09-12T01:52:15.050 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:14.393+0000 7f7e641cc640 1 -- 172.21.15.16:0/260765858 wait complete. 2024-09-12T01:52:15.050 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:14.394+0000 7f7e641cc640 1 Processor -- start 2024-09-12T01:52:15.050 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:14.394+0000 7f7e641cc640 1 -- start start 2024-09-12T01:52:15.050 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:14.394+0000 7f7e641cc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e5c107780 0x7f7e5c07daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:15.050 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:14.394+0000 7f7e641cc640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7e5c108eb0 con 0x7f7e5c107780 2024-09-12T01:52:15.050 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:14.394+0000 7f7e61f68640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e5c107780 0x7f7e5c07daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:15.050 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:14.394+0000 7f7e61f68640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e5c107780 0x7f7e5c07daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43398/0 (socket says 172.21.15.16:43398) 2024-09-12T01:52:15.050 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:14.394+0000 7f7e61f68640 1 -- 172.21.15.16:0/3378163097 learned_addr learned my addr 172.21.15.16:0/3378163097 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:52:15.051 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:14.394+0000 7f7e61f68640 1 -- 172.21.15.16:0/3378163097 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e5c07e030 con 0x7f7e5c107780 2024-09-12T01:52:15.051 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:14.395+0000 7f7e61f68640 1 --2- 172.21.15.16:0/3378163097 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e5c107780 0x7f7e5c07daf0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f7e580090b0 tx=0x7f7e580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:52:15.051 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:14.395+0000 7f7e52ffd640 1 -- 172.21.15.16:0/3378163097 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7e58041070 con 0x7f7e5c107780 2024-09-12T01:52:15.051 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:14.395+0000 7f7e52ffd640 1 -- 172.21.15.16:0/3378163097 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e5803c7a0 con 0x7f7e5c107780 2024-09-12T01:52:15.051 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:14.395+0000 7f7e52ffd640 1 -- 172.21.15.16:0/3378163097 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7e58037040 con 0x7f7e5c107780 2024-09-12T01:52:15.051 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:14.395+0000 7f7e641cc640 1 -- 172.21.15.16:0/3378163097 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e5c07c220 con 0x7f7e5c107780 2024-09-12T01:52:15.051 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:14.395+0000 7f7e641cc640 1 -- 172.21.15.16:0/3378163097 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e5c07c760 con 0x7f7e5c107780 2024-09-12T01:52:15.051 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:14.395+0000 7f7e52ffd640 1 -- 172.21.15.16:0/3378163097 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f7e58044070 con 0x7f7e5c107780 2024-09-12T01:52:15.051 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:14.395+0000 7f7e52ffd640 1 -- 172.21.15.16:0/3378163097 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f7e5803d3f0 con 0x7f7e5c107780 2024-09-12T01:52:15.051 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:14.396+0000 7f7e641cc640 1 -- 172.21.15.16:0/3378163097 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e5c109500 con 0x7f7e5c107780 2024-09-12T01:52:15.051 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:14.398+0000 7f7e52ffd640 1 -- 172.21.15.16:0/3378163097 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f7e58046450 con 0x7f7e5c107780 2024-09-12T01:52:15.051 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:14.457+0000 7f7e641cc640 1 -- 172.21.15.16:0/3378163097 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f7e5c193e00 con 0x7f7e5c107780 2024-09-12T01:52:15.052 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:14.460+0000 7f7e52ffd640 1 -- 172.21.15.16:0/3378163097 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+429 (secure 0 0 0) 0x7f7e58046630 con 0x7f7e5c107780 2024-09-12T01:52:15.052 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:14.461+0000 7f7e52ffd640 1 -- 172.21.15.16:0/3378163097 <== mon.0 v2:172.21.15.16:3300/0 8 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7e5803cb20 con 0x7f7e5c107780 2024-09-12T01:52:15.052 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:14.461+0000 7f7e641cc640 1 -- 172.21.15.16:0/3378163097 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e5c107780 msgr2=0x7f7e5c07daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:52:15.052 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:14.461+0000 7f7e641cc640 1 --2- 172.21.15.16:0/3378163097 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e5c107780 0x7f7e5c07daf0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f7e580090b0 tx=0x7f7e580046c0 comp rx=0 tx=0).stop 2024-09-12T01:52:15.052 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:14.462+0000 7f7e641cc640 1 -- 172.21.15.16:0/3378163097 shutdown_connections 2024-09-12T01:52:15.052 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:14.462+0000 7f7e641cc640 1 --2- 172.21.15.16:0/3378163097 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e5c107780 0x7f7e5c07daf0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:15.052 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:14.462+0000 7f7e641cc640 1 -- 172.21.15.16:0/3378163097 >> 172.21.15.16:0/3378163097 conn(0x7f7e5c102f30 msgr2=0x7f7e5c18f1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:52:15.052 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:14.462+0000 7f7e641cc640 1 -- 172.21.15.16:0/3378163097 shutdown_connections 2024-09-12T01:52:15.052 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:14.462+0000 7f7e641cc640 1 -- 172.21.15.16:0/3378163097 wait complete. 2024-09-12T01:52:15.052 INFO:teuthology.orchestra.run.smithi016.stdout:Generating new minimal ceph.conf... 2024-09-12T01:52:15.164 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:14 smithi016 ceph-mon[30583]: from='client.? 172.21.15.16:0/3378163097' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-12T01:52:15.164 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:14 smithi016 ceph-mon[30583]: from='client.? 172.21.15.16:0/3378163097' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-09-12T01:52:16.183 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:15 smithi016 ceph-mon[30583]: from='client.? 172.21.15.16:0/2181651147' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:52:16.193 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:15.491+0000 7f7503c76640 1 Processor -- start 2024-09-12T01:52:16.194 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:15.491+0000 7f7503c76640 1 -- start start 2024-09-12T01:52:16.194 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:15.491+0000 7f7503c76640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74fc107780 0x7f74fc107b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:16.194 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:15.491+0000 7f7503c76640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f74fc108150 con 0x7f74fc107780 2024-09-12T01:52:16.194 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:15.491+0000 7f7501a12640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74fc107780 0x7f74fc107b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:16.194 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:15.491+0000 7f7501a12640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74fc107780 0x7f74fc107b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43402/0 (socket says 172.21.15.16:43402) 2024-09-12T01:52:16.194 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:15.491+0000 7f7501a12640 1 -- 172.21.15.16:0/4230783620 learned_addr learned my addr 172.21.15.16:0/4230783620 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:52:16.194 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:15.492+0000 7f7501a12640 1 -- 172.21.15.16:0/4230783620 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74fc108980 con 0x7f74fc107780 2024-09-12T01:52:16.194 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:15.492+0000 7f7501a12640 1 --2- 172.21.15.16:0/4230783620 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74fc107780 0x7f74fc107b80 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f74f8009820 tx=0x7f74f802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fdaff5f85fcb68d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:52:16.195 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:15.493+0000 7f7500a10640 1 -- 172.21.15.16:0/4230783620 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f74f8037070 con 0x7f74fc107780 2024-09-12T01:52:16.195 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:15.493+0000 7f7500a10640 1 -- 172.21.15.16:0/4230783620 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f74f8032900 con 0x7f74fc107780 2024-09-12T01:52:16.195 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:15.493+0000 7f7500a10640 1 -- 172.21.15.16:0/4230783620 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f74f8030850 con 0x7f74fc107780 2024-09-12T01:52:16.195 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:15.493+0000 7f7503c76640 1 -- 172.21.15.16:0/4230783620 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74fc107780 msgr2=0x7f74fc107b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:52:16.195 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:15.493+0000 7f7503c76640 1 --2- 172.21.15.16:0/4230783620 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74fc107780 0x7f74fc107b80 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f74f8009820 tx=0x7f74f802a270 comp rx=0 tx=0).stop 2024-09-12T01:52:16.195 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:15.493+0000 7f7503c76640 1 -- 172.21.15.16:0/4230783620 shutdown_connections 2024-09-12T01:52:16.195 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:15.493+0000 7f7503c76640 1 --2- 172.21.15.16:0/4230783620 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74fc107780 0x7f74fc107b80 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:16.195 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:15.493+0000 7f7503c76640 1 -- 172.21.15.16:0/4230783620 >> 172.21.15.16:0/4230783620 conn(0x7f74fc102f30 msgr2=0x7f74fc105350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:52:16.195 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:15.493+0000 7f7503c76640 1 -- 172.21.15.16:0/4230783620 shutdown_connections 2024-09-12T01:52:16.195 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:15.493+0000 7f7503c76640 1 -- 172.21.15.16:0/4230783620 wait complete. 2024-09-12T01:52:16.195 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:15.494+0000 7f7503c76640 1 Processor -- start 2024-09-12T01:52:16.195 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:15.494+0000 7f7503c76640 1 -- start start 2024-09-12T01:52:16.196 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:15.494+0000 7f7503c76640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74fc107780 0x7f74fc19b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:16.196 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:15.494+0000 7f7503c76640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f74fc10bd30 con 0x7f74fc107780 2024-09-12T01:52:16.196 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:15.494+0000 7f7501a12640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74fc107780 0x7f74fc19b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:16.196 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:15.494+0000 7f7501a12640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74fc107780 0x7f74fc19b710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43406/0 (socket says 172.21.15.16:43406) 2024-09-12T01:52:16.196 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:15.494+0000 7f7501a12640 1 -- 172.21.15.16:0/2181651147 learned_addr learned my addr 172.21.15.16:0/2181651147 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:52:16.196 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:15.495+0000 7f7501a12640 1 -- 172.21.15.16:0/2181651147 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74fc19bc50 con 0x7f74fc107780 2024-09-12T01:52:16.196 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:15.495+0000 7f7501a12640 1 --2- 172.21.15.16:0/2181651147 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74fc107780 0x7f74fc19b710 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f74f80097f0 tx=0x7f74f8030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:52:16.196 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:15.495+0000 7f74f2ffd640 1 -- 172.21.15.16:0/2181651147 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f74f8037040 con 0x7f74fc107780 2024-09-12T01:52:16.196 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:15.495+0000 7f74f2ffd640 1 -- 172.21.15.16:0/2181651147 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f74f8040070 con 0x7f74fc107780 2024-09-12T01:52:16.196 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:15.495+0000 7f7503c76640 1 -- 172.21.15.16:0/2181651147 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74fc19bee0 con 0x7f74fc107780 2024-09-12T01:52:16.196 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:15.495+0000 7f74f2ffd640 1 -- 172.21.15.16:0/2181651147 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f74f803a390 con 0x7f74fc107780 2024-09-12T01:52:16.196 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:15.495+0000 7f7503c76640 1 -- 172.21.15.16:0/2181651147 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74fc19ebd0 con 0x7f74fc107780 2024-09-12T01:52:16.196 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:15.495+0000 7f74f2ffd640 1 -- 172.21.15.16:0/2181651147 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f74f803a5e0 con 0x7f74fc107780 2024-09-12T01:52:16.197 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:15.495+0000 7f74f2ffd640 1 -- 172.21.15.16:0/2181651147 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f74f8045950 con 0x7f74fc107780 2024-09-12T01:52:16.197 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:15.496+0000 7f7503c76640 1 -- 172.21.15.16:0/2181651147 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74fc107c00 con 0x7f74fc107780 2024-09-12T01:52:16.197 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:15.498+0000 7f74f2ffd640 1 -- 172.21.15.16:0/2181651147 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f74f8045460 con 0x7f74fc107780 2024-09-12T01:52:16.197 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:15.559+0000 7f7503c76640 1 -- 172.21.15.16:0/2181651147 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f74fc19eda0 con 0x7f74fc107780 2024-09-12T01:52:16.197 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:15.559+0000 7f74f2ffd640 1 -- 172.21.15.16:0/2181651147 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7f74f803b430 con 0x7f74fc107780 2024-09-12T01:52:16.197 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:15.560+0000 7f7503c76640 1 -- 172.21.15.16:0/2181651147 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74fc107780 msgr2=0x7f74fc19b710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:52:16.197 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:15.560+0000 7f7503c76640 1 --2- 172.21.15.16:0/2181651147 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74fc107780 0x7f74fc19b710 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f74f80097f0 tx=0x7f74f8030d80 comp rx=0 tx=0).stop 2024-09-12T01:52:16.197 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:15.560+0000 7f7503c76640 1 -- 172.21.15.16:0/2181651147 shutdown_connections 2024-09-12T01:52:16.197 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:15.560+0000 7f7503c76640 1 --2- 172.21.15.16:0/2181651147 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74fc107780 0x7f74fc19b710 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:16.197 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:15.560+0000 7f7503c76640 1 -- 172.21.15.16:0/2181651147 >> 172.21.15.16:0/2181651147 conn(0x7f74fc102f30 msgr2=0x7f74fc103aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:52:16.197 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:15.560+0000 7f7503c76640 1 -- 172.21.15.16:0/2181651147 shutdown_connections 2024-09-12T01:52:16.197 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:15.560+0000 7f7503c76640 1 -- 172.21.15.16:0/2181651147 wait complete. 2024-09-12T01:52:16.198 INFO:teuthology.orchestra.run.smithi016.stdout:Restarting the monitor... 2024-09-12T01:52:16.474 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:16 smithi016 systemd[1]: Stopping Ceph mon.a for 879f3b56-70a9-11ef-bcea-c7b262605968... 2024-09-12T01:52:16.840 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:16 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mon-a[30579]: 2024-09-12T01:52:16.473+0000 7f904a0b3640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T01:52:16.840 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:16 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mon-a[30579]: 2024-09-12T01:52:16.473+0000 7f904a0b3640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-09-12T01:52:16.840 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:16 smithi016 podman[30826]: 2024-09-12 01:52:16.53547325 +0000 UTC m=+0.176317603 container died efac6124eaf2f9feeb62dd5e507b07625aa9f6ecd2dc5b4261fcf21d4765c379 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mon-a, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-09-12T01:52:17.140 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:16 smithi016 podman[30826]: 2024-09-12 01:52:16.940031206 +0000 UTC m=+0.580875558 container cleanup efac6124eaf2f9feeb62dd5e507b07625aa9f6ecd2dc5b4261fcf21d4765c379 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3) 2024-09-12T01:52:17.141 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:16 smithi016 bash[30826]: ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mon-a 2024-09-12T01:52:17.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 podman[30838]: 2024-09-12 01:52:17.140668134 +0000 UTC m=+0.600728288 container remove efac6124eaf2f9feeb62dd5e507b07625aa9f6ecd2dc5b4261fcf21d4765c379 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mon-a, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T01:52:17.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 systemd[1]: ceph-879f3b56-70a9-11ef-bcea-c7b262605968@mon.a.service: Deactivated successfully. 2024-09-12T01:52:17.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 systemd[1]: Stopped Ceph mon.a for 879f3b56-70a9-11ef-bcea-c7b262605968. 2024-09-12T01:52:17.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 systemd[1]: Starting Ceph mon.a for 879f3b56-70a9-11ef-bcea-c7b262605968... 2024-09-12T01:52:17.735 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 podman[30928]: 2024-09-12 01:52:17.466621463 +0000 UTC m=+0.103621381 container create ab3eea3e12feba118c6232cbc44962043a88bd3f49c4e975388298824a9c0110 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-09-12T01:52:17.735 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 podman[30928]: 2024-09-12 01:52:17.379206846 +0000 UTC m=+0.016206734 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T01:52:17.736 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 podman[30928]: 2024-09-12 01:52:17.660585029 +0000 UTC m=+0.297584905 container init ab3eea3e12feba118c6232cbc44962043a88bd3f49c4e975388298824a9c0110 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mon-a, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T01:52:17.736 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 podman[30928]: 2024-09-12 01:52:17.663794567 +0000 UTC m=+0.300794446 container start ab3eea3e12feba118c6232cbc44962043a88bd3f49c4e975388298824a9c0110 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mon-a, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-09-12T01:52:17.736 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: set uid:gid to 167:167 (ceph:ceph) 2024-09-12T01:52:17.736 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable), process ceph-mon, pid 2 2024-09-12T01:52:17.736 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: pidfile_write: ignore empty --pid-file 2024-09-12T01:52:17.736 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: load: jerasure load: lrc 2024-09-12T01:52:17.737 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: RocksDB version: 7.9.2 2024-09-12T01:52:17.737 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Git sha 0 2024-09-12T01:52:17.737 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Compile date 2024-09-09 16:23:28 2024-09-12T01:52:17.737 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: DB SUMMARY 2024-09-12T01:52:17.737 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: DB Session ID: A43HBCN0YL9ENU9XGOZQ 2024-09-12T01:52:17.737 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: CURRENT file: CURRENT 2024-09-12T01:52:17.737 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: IDENTITY file: IDENTITY 2024-09-12T01:52:17.738 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-09-12T01:52:17.738 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-09-12T01:52:17.738 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 106186 ; 2024-09-12T01:52:17.738 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.error_if_exists: 0 2024-09-12T01:52:17.741 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.create_if_missing: 0 2024-09-12T01:52:17.741 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.paranoid_checks: 1 2024-09-12T01:52:17.741 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-12T01:52:17.741 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-12T01:52:17.741 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-12T01:52:17.741 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.env: 0x556c1c0df940 2024-09-12T01:52:17.742 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.fs: PosixFileSystem 2024-09-12T01:52:17.742 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.info_log: 0x556c1cb136e0 2024-09-12T01:52:17.742 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.max_file_opening_threads: 16 2024-09-12T01:52:17.742 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.statistics: (nil) 2024-09-12T01:52:17.742 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.use_fsync: 0 2024-09-12T01:52:17.742 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.max_log_file_size: 0 2024-09-12T01:52:17.742 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-12T01:52:17.742 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-12T01:52:17.743 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.keep_log_file_num: 1000 2024-09-12T01:52:17.743 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.recycle_log_file_num: 0 2024-09-12T01:52:17.743 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.allow_fallocate: 1 2024-09-12T01:52:17.743 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.allow_mmap_reads: 0 2024-09-12T01:52:17.743 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.allow_mmap_writes: 0 2024-09-12T01:52:17.744 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.use_direct_reads: 0 2024-09-12T01:52:17.744 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-12T01:52:17.744 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.create_missing_column_families: 0 2024-09-12T01:52:17.744 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.db_log_dir: 2024-09-12T01:52:17.744 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.wal_dir: 2024-09-12T01:52:17.744 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-12T01:52:17.745 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-12T01:52:17.745 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-12T01:52:17.745 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-12T01:52:17.745 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-12T01:52:17.745 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-12T01:52:17.745 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.advise_random_on_open: 1 2024-09-12T01:52:17.745 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.db_write_buffer_size: 0 2024-09-12T01:52:17.746 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.write_buffer_manager: 0x556c1cb22dc0 2024-09-12T01:52:17.746 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-12T01:52:17.746 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-12T01:52:17.746 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-12T01:52:17.746 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.rate_limiter: (nil) 2024-09-12T01:52:17.746 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-12T01:52:17.747 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.wal_recovery_mode: 2 2024-09-12T01:52:17.747 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.enable_thread_tracking: 0 2024-09-12T01:52:17.747 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.enable_pipelined_write: 0 2024-09-12T01:52:17.747 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.unordered_write: 0 2024-09-12T01:52:17.747 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-12T01:52:17.747 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-12T01:52:17.748 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-12T01:52:17.748 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-12T01:52:17.748 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.row_cache: None 2024-09-12T01:52:17.748 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.wal_filter: None 2024-09-12T01:52:17.748 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-12T01:52:17.749 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.allow_ingest_behind: 0 2024-09-12T01:52:17.749 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.two_write_queues: 0 2024-09-12T01:52:17.749 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.manual_wal_flush: 0 2024-09-12T01:52:17.749 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.wal_compression: 0 2024-09-12T01:52:17.749 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.atomic_flush: 0 2024-09-12T01:52:17.749 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-12T01:52:17.750 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-12T01:52:17.750 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-12T01:52:17.750 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.log_readahead_size: 0 2024-09-12T01:52:17.750 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-12T01:52:17.750 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.best_efforts_recovery: 0 2024-09-12T01:52:17.750 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-12T01:52:17.750 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-12T01:52:17.751 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.allow_data_in_errors: 0 2024-09-12T01:52:17.751 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.db_host_id: __hostname__ 2024-09-12T01:52:17.751 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-12T01:52:17.751 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.max_background_jobs: 2 2024-09-12T01:52:17.751 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.max_background_compactions: -1 2024-09-12T01:52:17.751 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.max_subcompactions: 1 2024-09-12T01:52:17.752 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-12T01:52:17.752 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-12T01:52:17.752 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-12T01:52:17.752 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.max_total_wal_size: 0 2024-09-12T01:52:17.752 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-12T01:52:17.752 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-12T01:52:17.753 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-12T01:52:17.753 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-12T01:52:17.753 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.max_open_files: -1 2024-09-12T01:52:17.753 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.bytes_per_sync: 0 2024-09-12T01:52:17.753 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-12T01:52:17.754 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-12T01:52:17.754 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.compaction_readahead_size: 0 2024-09-12T01:52:17.754 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.max_background_flushes: -1 2024-09-12T01:52:17.754 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Compression algorithms supported: 2024-09-12T01:52:17.754 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: kZSTD supported: 0 2024-09-12T01:52:17.754 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: kXpressCompression supported: 0 2024-09-12T01:52:17.755 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: kBZip2Compression supported: 0 2024-09-12T01:52:17.755 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-12T01:52:17.755 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: kLZ4Compression supported: 1 2024-09-12T01:52:17.755 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: kZlibCompression supported: 1 2024-09-12T01:52:17.755 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: kLZ4HCCompression supported: 1 2024-09-12T01:52:17.755 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: kSnappyCompression supported: 1 2024-09-12T01:52:17.756 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-12T01:52:17.756 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-12T01:52:17.756 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-09-12T01:52:17.756 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-12T01:52:17.756 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-12T01:52:17.756 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.merge_operator: 2024-09-12T01:52:17.756 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.compaction_filter: None 2024-09-12T01:52:17.756 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.compaction_filter_factory: None 2024-09-12T01:52:17.756 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.sst_partitioner_factory: None 2024-09-12T01:52:17.757 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-12T01:52:17.757 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-12T01:52:17.757 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x556c1cb13320) 2024-09-12T01:52:17.757 INFO:journalctl@ceph.mon.a.smithi016.stdout: cache_index_and_filter_blocks: 1 2024-09-12T01:52:17.757 INFO:journalctl@ceph.mon.a.smithi016.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-12T01:52:17.757 INFO:journalctl@ceph.mon.a.smithi016.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-12T01:52:17.757 INFO:journalctl@ceph.mon.a.smithi016.stdout: pin_top_level_index_and_filter: 1 2024-09-12T01:52:17.757 INFO:journalctl@ceph.mon.a.smithi016.stdout: index_type: 0 2024-09-12T01:52:17.758 INFO:journalctl@ceph.mon.a.smithi016.stdout: data_block_index_type: 0 2024-09-12T01:52:17.758 INFO:journalctl@ceph.mon.a.smithi016.stdout: index_shortening: 1 2024-09-12T01:52:17.758 INFO:journalctl@ceph.mon.a.smithi016.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-12T01:52:17.758 INFO:journalctl@ceph.mon.a.smithi016.stdout: checksum: 4 2024-09-12T01:52:17.758 INFO:journalctl@ceph.mon.a.smithi016.stdout: no_block_cache: 0 2024-09-12T01:52:17.758 INFO:journalctl@ceph.mon.a.smithi016.stdout: block_cache: 0x556c1cb071f0 2024-09-12T01:52:17.759 INFO:journalctl@ceph.mon.a.smithi016.stdout: block_cache_name: BinnedLRUCache 2024-09-12T01:52:17.759 INFO:journalctl@ceph.mon.a.smithi016.stdout: block_cache_options: 2024-09-12T01:52:17.759 INFO:journalctl@ceph.mon.a.smithi016.stdout: capacity : 536870912 2024-09-12T01:52:17.759 INFO:journalctl@ceph.mon.a.smithi016.stdout: num_shard_bits : 4 2024-09-12T01:52:17.759 INFO:journalctl@ceph.mon.a.smithi016.stdout: strict_capacity_limit : 0 2024-09-12T01:52:17.759 INFO:journalctl@ceph.mon.a.smithi016.stdout: high_pri_pool_ratio: 0.000 2024-09-12T01:52:17.759 INFO:journalctl@ceph.mon.a.smithi016.stdout: block_cache_compressed: (nil) 2024-09-12T01:52:17.760 INFO:journalctl@ceph.mon.a.smithi016.stdout: persistent_cache: (nil) 2024-09-12T01:52:17.760 INFO:journalctl@ceph.mon.a.smithi016.stdout: block_size: 4096 2024-09-12T01:52:17.760 INFO:journalctl@ceph.mon.a.smithi016.stdout: block_size_deviation: 10 2024-09-12T01:52:17.760 INFO:journalctl@ceph.mon.a.smithi016.stdout: block_restart_interval: 16 2024-09-12T01:52:17.760 INFO:journalctl@ceph.mon.a.smithi016.stdout: index_block_restart_interval: 1 2024-09-12T01:52:17.761 INFO:journalctl@ceph.mon.a.smithi016.stdout: metadata_block_size: 4096 2024-09-12T01:52:17.761 INFO:journalctl@ceph.mon.a.smithi016.stdout: partition_filters: 0 2024-09-12T01:52:17.761 INFO:journalctl@ceph.mon.a.smithi016.stdout: use_delta_encoding: 1 2024-09-12T01:52:17.761 INFO:journalctl@ceph.mon.a.smithi016.stdout: filter_policy: bloomfilter 2024-09-12T01:52:17.761 INFO:journalctl@ceph.mon.a.smithi016.stdout: whole_key_filtering: 1 2024-09-12T01:52:17.762 INFO:journalctl@ceph.mon.a.smithi016.stdout: verify_compression: 0 2024-09-12T01:52:17.762 INFO:journalctl@ceph.mon.a.smithi016.stdout: read_amp_bytes_per_bit: 0 2024-09-12T01:52:17.762 INFO:journalctl@ceph.mon.a.smithi016.stdout: format_version: 5 2024-09-12T01:52:17.762 INFO:journalctl@ceph.mon.a.smithi016.stdout: enable_index_compression: 1 2024-09-12T01:52:17.762 INFO:journalctl@ceph.mon.a.smithi016.stdout: block_align: 0 2024-09-12T01:52:17.762 INFO:journalctl@ceph.mon.a.smithi016.stdout: max_auto_readahead_size: 262144 2024-09-12T01:52:17.762 INFO:journalctl@ceph.mon.a.smithi016.stdout: prepopulate_block_cache: 0 2024-09-12T01:52:17.762 INFO:journalctl@ceph.mon.a.smithi016.stdout: initial_auto_readahead_size: 8192 2024-09-12T01:52:17.762 INFO:journalctl@ceph.mon.a.smithi016.stdout: num_file_reads_for_auto_readahead: 2 2024-09-12T01:52:17.762 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.write_buffer_size: 33554432 2024-09-12T01:52:17.762 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.max_write_buffer_number: 2 2024-09-12T01:52:17.762 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.compression: NoCompression 2024-09-12T01:52:17.763 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.bottommost_compression: Disabled 2024-09-12T01:52:17.763 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.prefix_extractor: nullptr 2024-09-12T01:52:17.763 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-12T01:52:17.763 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.num_levels: 7 2024-09-12T01:52:17.763 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-12T01:52:17.763 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-12T01:52:17.763 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-12T01:52:17.763 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-12T01:52:17.763 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-12T01:52:17.763 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-12T01:52:17.763 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-12T01:52:17.763 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-12T01:52:17.763 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-12T01:52:17.763 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-12T01:52:17.764 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-12T01:52:17.764 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-12T01:52:17.764 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-12T01:52:17.764 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.compression_opts.level: 32767 2024-09-12T01:52:17.764 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.compression_opts.strategy: 0 2024-09-12T01:52:17.764 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-12T01:52:17.764 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-12T01:52:17.764 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-12T01:52:17.764 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-12T01:52:17.764 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.compression_opts.enabled: false 2024-09-12T01:52:17.764 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-12T01:52:17.764 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-12T01:52:17.764 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-12T01:52:17.765 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-12T01:52:17.765 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.target_file_size_base: 67108864 2024-09-12T01:52:17.765 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-12T01:52:17.765 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-12T01:52:17.765 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-12T01:52:17.765 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-12T01:52:17.765 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-12T01:52:17.765 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-12T01:52:17.765 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-12T01:52:17.765 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-12T01:52:17.765 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-12T01:52:17.765 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-12T01:52:17.765 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-12T01:52:17.765 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-12T01:52:17.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-12T01:52:17.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-12T01:52:17.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.arena_block_size: 1048576 2024-09-12T01:52:17.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-12T01:52:17.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-12T01:52:17.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.disable_auto_compactions: 0 2024-09-12T01:52:17.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-12T01:52:17.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-12T01:52:17.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-12T01:52:17.767 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-12T01:52:17.767 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-12T01:52:17.767 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-12T01:52:17.767 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-12T01:52:17.767 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-12T01:52:17.767 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-12T01:52:17.767 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-12T01:52:17.767 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-12T01:52:17.767 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.inplace_update_support: 0 2024-09-12T01:52:17.767 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-12T01:52:17.767 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-12T01:52:17.767 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-12T01:52:17.767 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-12T01:52:17.767 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.bloom_locality: 0 2024-09-12T01:52:17.768 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.max_successive_merges: 0 2024-09-12T01:52:17.768 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-12T01:52:17.768 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.paranoid_file_checks: 0 2024-09-12T01:52:17.768 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.force_consistency_checks: 1 2024-09-12T01:52:17.768 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.report_bg_io_stats: 0 2024-09-12T01:52:17.768 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.ttl: 2592000 2024-09-12T01:52:17.768 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-12T01:52:17.768 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-12T01:52:17.768 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-12T01:52:17.769 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.enable_blob_files: false 2024-09-12T01:52:17.769 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.min_blob_size: 0 2024-09-12T01:52:17.769 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.blob_file_size: 268435456 2024-09-12T01:52:17.769 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-12T01:52:17.769 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-12T01:52:17.769 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-12T01:52:17.769 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-12T01:52:17.769 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-12T01:52:17.769 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.blob_file_starting_level: 0 2024-09-12T01:52:17.769 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-12T01:52:17.769 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-09-12T01:52:17.770 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-09-12T01:52:17.770 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: e870dabc-2362-4453-9f69-7ea3aabb9a09 2024-09-12T01:52:17.770 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726105937691982, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-09-12T01:52:17.770 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-09-12T01:52:17.770 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726105937693106, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 101158, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 332, "table_properties": {"data_size": 99043, "index_size": 307, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14530, "raw_average_key_size": 49, "raw_value_size": 91191, "raw_average_value_size": 313, "num_data_blocks": 12, "num_entries": 291, "num_filter_entries": 291, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726105937, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "e870dabc-2362-4453-9f69-7ea3aabb9a09", "db_session_id": "A43HBCN0YL9ENU9XGOZQ", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-09-12T01:52:17.770 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726105937693206, "job": 1, "event": "recovery_finished"} 2024-09-12T01:52:17.770 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-09-12T01:52:17.770 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-12T01:52:17.770 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x556c1cb34e00 2024-09-12T01:52:17.770 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: DB pointer 0x556c1cc30000 2024-09-12T01:52:17.770 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-12T01:52:17.770 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-12T01:52:17.770 INFO:journalctl@ceph.mon.a.smithi016.stdout: ** DB Stats ** 2024-09-12T01:52:17.771 INFO:journalctl@ceph.mon.a.smithi016.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-12T01:52:17.771 INFO:journalctl@ceph.mon.a.smithi016.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-12T01:52:17.771 INFO:journalctl@ceph.mon.a.smithi016.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-12T01:52:17.771 INFO:journalctl@ceph.mon.a.smithi016.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-12T01:52:17.771 INFO:journalctl@ceph.mon.a.smithi016.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-12T01:52:17.771 INFO:journalctl@ceph.mon.a.smithi016.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-12T01:52:17.771 INFO:journalctl@ceph.mon.a.smithi016.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-12T01:52:17.771 INFO:journalctl@ceph.mon.a.smithi016.stdout: 2024-09-12T01:52:17.771 INFO:journalctl@ceph.mon.a.smithi016.stdout: ** Compaction Stats [default] ** 2024-09-12T01:52:17.771 INFO:journalctl@ceph.mon.a.smithi016.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-12T01:52:17.771 INFO:journalctl@ceph.mon.a.smithi016.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-12T01:52:17.771 INFO:journalctl@ceph.mon.a.smithi016.stdout: L0 2/0 100.65 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 136.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-12T01:52:17.771 INFO:journalctl@ceph.mon.a.smithi016.stdout: Sum 2/0 100.65 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 136.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-12T01:52:17.771 INFO:journalctl@ceph.mon.a.smithi016.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 136.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-12T01:52:17.772 INFO:journalctl@ceph.mon.a.smithi016.stdout: 2024-09-12T01:52:17.772 INFO:journalctl@ceph.mon.a.smithi016.stdout: ** Compaction Stats [default] ** 2024-09-12T01:52:17.772 INFO:journalctl@ceph.mon.a.smithi016.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-12T01:52:17.772 INFO:journalctl@ceph.mon.a.smithi016.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-12T01:52:17.772 INFO:journalctl@ceph.mon.a.smithi016.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 136.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-12T01:52:17.772 INFO:journalctl@ceph.mon.a.smithi016.stdout: 2024-09-12T01:52:17.772 INFO:journalctl@ceph.mon.a.smithi016.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-12T01:52:17.772 INFO:journalctl@ceph.mon.a.smithi016.stdout: 2024-09-12T01:52:17.773 INFO:journalctl@ceph.mon.a.smithi016.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-12T01:52:17.773 INFO:journalctl@ceph.mon.a.smithi016.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-12T01:52:17.773 INFO:journalctl@ceph.mon.a.smithi016.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-12T01:52:17.773 INFO:journalctl@ceph.mon.a.smithi016.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-12T01:52:17.773 INFO:journalctl@ceph.mon.a.smithi016.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-12T01:52:17.773 INFO:journalctl@ceph.mon.a.smithi016.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-12T01:52:17.773 INFO:journalctl@ceph.mon.a.smithi016.stdout: Cumulative compaction: 0.00 GB write, 28.82 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-12T01:52:17.773 INFO:journalctl@ceph.mon.a.smithi016.stdout: Interval compaction: 0.00 GB write, 28.82 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-12T01:52:17.773 INFO:journalctl@ceph.mon.a.smithi016.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-12T01:52:17.773 INFO:journalctl@ceph.mon.a.smithi016.stdout: Block cache BinnedLRUCache@0x556c1cb071f0#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-09-12T01:52:17.773 INFO:journalctl@ceph.mon.a.smithi016.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-09-12T01:52:17.773 INFO:journalctl@ceph.mon.a.smithi016.stdout: 2024-09-12T01:52:17.773 INFO:journalctl@ceph.mon.a.smithi016.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-12T01:52:17.774 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: starting mon.a rank 0 at public addrs [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] at bind addrs [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 879f3b56-70a9-11ef-bcea-c7b262605968 2024-09-12T01:52:17.774 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: mon.a@-1(???) e1 preinit fsid 879f3b56-70a9-11ef-bcea-c7b262605968 2024-09-12T01:52:17.774 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: mon.a@-1(???).mds e1 new map 2024-09-12T01:52:17.774 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: mon.a@-1(???).mds e1 print_map 2024-09-12T01:52:17.774 INFO:journalctl@ceph.mon.a.smithi016.stdout: e1 2024-09-12T01:52:17.774 INFO:journalctl@ceph.mon.a.smithi016.stdout: btime 2024-09-12T01:52:12:840866+0000 2024-09-12T01:52:17.774 INFO:journalctl@ceph.mon.a.smithi016.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-12T01:52:17.774 INFO:journalctl@ceph.mon.a.smithi016.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-12T01:52:17.774 INFO:journalctl@ceph.mon.a.smithi016.stdout: legacy client fscid: -1 2024-09-12T01:52:17.774 INFO:journalctl@ceph.mon.a.smithi016.stdout: 2024-09-12T01:52:17.774 INFO:journalctl@ceph.mon.a.smithi016.stdout: No filesystems configured 2024-09-12T01:52:17.774 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-12T01:52:17.774 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-12T01:52:17.774 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-12T01:52:17.775 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-12T01:52:17.775 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-09-12T01:52:17.775 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 bash[30928]: ab3eea3e12feba118c6232cbc44962043a88bd3f49c4e975388298824a9c0110 2024-09-12T01:52:17.775 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 systemd[1]: Started Ceph mon.a for 879f3b56-70a9-11ef-bcea-c7b262605968. 2024-09-12T01:52:17.775 INFO:teuthology.orchestra.run.smithi016.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-09-12T01:52:18.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-12T01:52:18.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: monmap epoch 1 2024-09-12T01:52:18.185 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: fsid 879f3b56-70a9-11ef-bcea-c7b262605968 2024-09-12T01:52:18.185 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: last_changed 2024-09-12T01:52:09.859051+0000 2024-09-12T01:52:18.185 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: created 2024-09-12T01:52:09.859051+0000 2024-09-12T01:52:18.185 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: min_mon_release 19 (squid) 2024-09-12T01:52:18.185 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: election_strategy: 1 2024-09-12T01:52:18.185 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: 0: [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] mon.a 2024-09-12T01:52:18.185 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: fsmap 2024-09-12T01:52:18.185 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: osdmap e1: 0 total, 0 up, 0 in 2024-09-12T01:52:18.185 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:17 smithi016 ceph-mon[30947]: mgrmap e1: no daemons active 2024-09-12T01:52:22.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:22 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/2674350203' entity='client.admin' 2024-09-12T01:52:27.518 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:21.072+0000 7fec29ccb640 1 Processor -- start 2024-09-12T01:52:27.518 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:21.072+0000 7fec29ccb640 1 -- start start 2024-09-12T01:52:27.518 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:21.072+0000 7fec29ccb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec2407de00 0x7fec2407c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:27.518 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:21.072+0000 7fec29ccb640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fec2407c830 con 0x7fec2407de00 2024-09-12T01:52:27.518 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:21.073+0000 7fec237fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec2407de00 0x7fec2407c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:27.518 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:21.073+0000 7fec237fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec2407de00 0x7fec2407c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43410/0 (socket says 172.21.15.16:43410) 2024-09-12T01:52:27.518 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:21.073+0000 7fec237fe640 1 -- 172.21.15.16:0/636917548 learned_addr learned my addr 172.21.15.16:0/636917548 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:52:27.519 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:21.073+0000 7fec237fe640 1 -- 172.21.15.16:0/636917548 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec2407c9b0 con 0x7fec2407de00 2024-09-12T01:52:27.519 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:21.075+0000 7fec237fe640 1 --2- 172.21.15.16:0/636917548 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec2407de00 0x7fec2407c260 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fec14009820 tx=0x7fec1402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcd1446a2a831528 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:52:27.519 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:21.075+0000 7fec227fc640 1 -- 172.21.15.16:0/636917548 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fec14037070 con 0x7fec2407de00 2024-09-12T01:52:27.519 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:21.075+0000 7fec227fc640 1 -- 172.21.15.16:0/636917548 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fec14032900 con 0x7fec2407de00 2024-09-12T01:52:27.519 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:21.075+0000 7fec227fc640 1 -- 172.21.15.16:0/636917548 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fec14030850 con 0x7fec2407de00 2024-09-12T01:52:27.519 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:21.076+0000 7fec29ccb640 1 -- 172.21.15.16:0/636917548 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec2407de00 msgr2=0x7fec2407c260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:52:27.519 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:21.076+0000 7fec29ccb640 1 --2- 172.21.15.16:0/636917548 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec2407de00 0x7fec2407c260 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fec14009820 tx=0x7fec1402a270 comp rx=0 tx=0).stop 2024-09-12T01:52:27.519 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:21.076+0000 7fec29ccb640 1 -- 172.21.15.16:0/636917548 shutdown_connections 2024-09-12T01:52:27.519 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:21.076+0000 7fec29ccb640 1 --2- 172.21.15.16:0/636917548 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec2407de00 0x7fec2407c260 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:27.519 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:21.076+0000 7fec29ccb640 1 -- 172.21.15.16:0/636917548 >> 172.21.15.16:0/636917548 conn(0x7fec24102f30 msgr2=0x7fec24105350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:52:27.519 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:21.076+0000 7fec29ccb640 1 -- 172.21.15.16:0/636917548 shutdown_connections 2024-09-12T01:52:27.519 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:21.076+0000 7fec29ccb640 1 -- 172.21.15.16:0/636917548 wait complete. 2024-09-12T01:52:27.519 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:21.077+0000 7fec29ccb640 1 Processor -- start 2024-09-12T01:52:27.519 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:21.077+0000 7fec29ccb640 1 -- start start 2024-09-12T01:52:27.520 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:21.077+0000 7fec29ccb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec2407de00 0x7fec2419d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:27.520 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:21.077+0000 7fec29ccb640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fec2410dd50 con 0x7fec2407de00 2024-09-12T01:52:27.520 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:21.077+0000 7fec237fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec2407de00 0x7fec2419d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:27.520 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:21.077+0000 7fec237fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec2407de00 0x7fec2419d8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43412/0 (socket says 172.21.15.16:43412) 2024-09-12T01:52:27.520 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:21.077+0000 7fec237fe640 1 -- 172.21.15.16:0/2674350203 learned_addr learned my addr 172.21.15.16:0/2674350203 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:52:27.520 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:21.078+0000 7fec237fe640 1 -- 172.21.15.16:0/2674350203 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec2419de20 con 0x7fec2407de00 2024-09-12T01:52:27.520 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:21.078+0000 7fec237fe640 1 --2- 172.21.15.16:0/2674350203 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec2407de00 0x7fec2419d8e0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fec140097f0 tx=0x7fec14030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:52:27.520 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:21.078+0000 7fec20ff9640 1 -- 172.21.15.16:0/2674350203 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fec14037040 con 0x7fec2407de00 2024-09-12T01:52:27.520 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:21.078+0000 7fec29ccb640 1 -- 172.21.15.16:0/2674350203 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec2419e0b0 con 0x7fec2407de00 2024-09-12T01:52:27.520 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:21.078+0000 7fec20ff9640 1 -- 172.21.15.16:0/2674350203 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fec14040070 con 0x7fec2407de00 2024-09-12T01:52:27.520 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:21.078+0000 7fec20ff9640 1 -- 172.21.15.16:0/2674350203 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fec1403a390 con 0x7fec2407de00 2024-09-12T01:52:27.520 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:21.078+0000 7fec29ccb640 1 -- 172.21.15.16:0/2674350203 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec241a0da0 con 0x7fec2407de00 2024-09-12T01:52:27.520 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:21.078+0000 7fec20ff9640 1 -- 172.21.15.16:0/2674350203 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fec1403a630 con 0x7fec2407de00 2024-09-12T01:52:27.521 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:21.078+0000 7fec20ff9640 1 -- 172.21.15.16:0/2674350203 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fec140459a0 con 0x7fec2407de00 2024-09-12T01:52:27.521 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:21.079+0000 7fec29ccb640 1 -- 172.21.15.16:0/2674350203 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec2407c2e0 con 0x7fec2407de00 2024-09-12T01:52:27.521 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:21.081+0000 7fec20ff9640 1 -- 172.21.15.16:0/2674350203 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fec14045460 con 0x7fec2407de00 2024-09-12T01:52:27.521 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:21.142+0000 7fec29ccb640 1 -- 172.21.15.16:0/2674350203 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7fec241a0f70 con 0x7fec2407de00 2024-09-12T01:52:27.521 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:21.144+0000 7fec20ff9640 1 -- 172.21.15.16:0/2674350203 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7fec1403b430 con 0x7fec2407de00 2024-09-12T01:52:27.521 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:21.144+0000 7fec29ccb640 1 -- 172.21.15.16:0/2674350203 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec2407de00 msgr2=0x7fec2419d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:52:27.521 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:21.144+0000 7fec29ccb640 1 --2- 172.21.15.16:0/2674350203 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec2407de00 0x7fec2419d8e0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fec140097f0 tx=0x7fec14030d80 comp rx=0 tx=0).stop 2024-09-12T01:52:27.521 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:21.145+0000 7fec29ccb640 1 -- 172.21.15.16:0/2674350203 shutdown_connections 2024-09-12T01:52:27.521 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:21.145+0000 7fec29ccb640 1 --2- 172.21.15.16:0/2674350203 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec2407de00 0x7fec2419d8e0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:27.521 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:21.145+0000 7fec29ccb640 1 -- 172.21.15.16:0/2674350203 >> 172.21.15.16:0/2674350203 conn(0x7fec24102f30 msgr2=0x7fec24103aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:52:27.521 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:21.145+0000 7fec29ccb640 1 -- 172.21.15.16:0/2674350203 shutdown_connections 2024-09-12T01:52:27.521 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:21.145+0000 7fec29ccb640 1 -- 172.21.15.16:0/2674350203 wait complete. 2024-09-12T01:52:27.521 INFO:teuthology.orchestra.run.smithi016.stdout:Wrote config to /etc/ceph/ceph.conf 2024-09-12T01:52:27.720 INFO:teuthology.orchestra.run.smithi016.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-09-12T01:52:27.721 INFO:teuthology.orchestra.run.smithi016.stdout:Creating mgr... 2024-09-12T01:52:27.721 INFO:teuthology.orchestra.run.smithi016.stdout:Verifying port 0.0.0.0:9283 ... 2024-09-12T01:52:27.722 INFO:teuthology.orchestra.run.smithi016.stdout:Verifying port 0.0.0.0:8765 ... 2024-09-12T01:52:28.720 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-879f3b56-70a9-11ef-bcea-c7b262605968@mgr.x 2024-09-12T01:52:28.720 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stderr Failed to reset failed state of unit ceph-879f3b56-70a9-11ef-bcea-c7b262605968@mgr.x.service: Unit ceph-879f3b56-70a9-11ef-bcea-c7b262605968@mgr.x.service not loaded. 2024-09-12T01:52:28.954 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-879f3b56-70a9-11ef-bcea-c7b262605968.target.wants/ceph-879f3b56-70a9-11ef-bcea-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-879f3b56-70a9-11ef-bcea-c7b262605968@.service. 2024-09-12T01:52:29.184 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 01:52:28 smithi016 systemd[1]: Starting Ceph mgr.x for 879f3b56-70a9-11ef-bcea-c7b262605968... 2024-09-12T01:52:29.515 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 01:52:29 smithi016 podman[31158]: 2024-09-12 01:52:29.090239814 +0000 UTC m=+0.016272054 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T01:52:29.515 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 01:52:29 smithi016 podman[31158]: 2024-09-12 01:52:29.515918059 +0000 UTC m=+0.441950301 container create ea29060771e897661e9e82aa42c5672ce70af5154490ae841c10e83a1eaa3d46 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-x, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2) 2024-09-12T01:52:29.801 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 01:52:29 smithi016 podman[31158]: 2024-09-12 01:52:29.726692194 +0000 UTC m=+0.652724433 container init ea29060771e897661e9e82aa42c5672ce70af5154490ae841c10e83a1eaa3d46 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-x, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-12T01:52:29.801 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 01:52:29 smithi016 podman[31158]: 2024-09-12 01:52:29.729790214 +0000 UTC m=+0.655822456 container start ea29060771e897661e9e82aa42c5672ce70af5154490ae841c10e83a1eaa3d46 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-x, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T01:52:29.816 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-12T01:52:29.816 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout disabled 2024-09-12T01:52:29.824 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-12T01:52:29.824 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout inactive 2024-09-12T01:52:29.824 INFO:teuthology.orchestra.run.smithi016.stdout:firewalld.service is not enabled 2024-09-12T01:52:29.825 INFO:teuthology.orchestra.run.smithi016.stdout:Not possible to enable service . firewalld.service is not available 2024-09-12T01:52:29.833 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-12T01:52:29.833 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout disabled 2024-09-12T01:52:29.841 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-12T01:52:29.841 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout inactive 2024-09-12T01:52:29.841 INFO:teuthology.orchestra.run.smithi016.stdout:firewalld.service is not enabled 2024-09-12T01:52:29.841 INFO:teuthology.orchestra.run.smithi016.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-09-12T01:52:29.841 INFO:teuthology.orchestra.run.smithi016.stdout:Waiting for mgr to start... 2024-09-12T01:52:29.841 INFO:teuthology.orchestra.run.smithi016.stdout:Waiting for mgr... 2024-09-12T01:52:30.083 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 01:52:29 smithi016 bash[31158]: ea29060771e897661e9e82aa42c5672ce70af5154490ae841c10e83a1eaa3d46 2024-09-12T01:52:30.083 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 01:52:29 smithi016 systemd[1]: Started Ceph mgr.x for 879f3b56-70a9-11ef-bcea-c7b262605968. 2024-09-12T01:52:30.083 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 01:52:29 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-x[31173]: 2024-09-12T01:52:29.860+0000 7f3211d03040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-12T01:52:30.083 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 01:52:29 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-x[31173]: 2024-09-12T01:52:29.921+0000 7f3211d03040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-12T01:52:30.083 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 01:52:30 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-x[31173]: 2024-09-12T01:52:30.082+0000 7f3211d03040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-12T01:52:30.357 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 01:52:30 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-x[31173]: 2024-09-12T01:52:30.154+0000 7f3211d03040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-12T01:52:30.357 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 01:52:30 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-x[31173]: 2024-09-12T01:52:30.313+0000 7f3211d03040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-12T01:52:30.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:30 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/2637131383' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-12T01:52:30.684 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 01:52:30 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-x[31173]: 2024-09-12T01:52:30.466+0000 7f3211d03040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-12T01:52:30.974 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 01:52:30 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-x[31173]: 2024-09-12T01:52:30.806+0000 7f3211d03040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-12T01:52:30.974 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 01:52:30 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-x[31173]: 2024-09-12T01:52:30.870+0000 7f3211d03040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-12T01:52:30.979 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 2024-09-12T01:52:30.979 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout { 2024-09-12T01:52:30.980 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "fsid": "879f3b56-70a9-11ef-bcea-c7b262605968", 2024-09-12T01:52:30.980 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "health": { 2024-09-12T01:52:30.980 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-12T01:52:30.980 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-12T01:52:30.980 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-12T01:52:30.980 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-09-12T01:52:30.980 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-12T01:52:30.980 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-12T01:52:30.980 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 0 2024-09-12T01:52:30.980 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout ], 2024-09-12T01:52:30.981 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-12T01:52:30.981 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "a" 2024-09-12T01:52:30.981 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout ], 2024-09-12T01:52:30.981 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "quorum_age": 12, 2024-09-12T01:52:30.981 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-12T01:52:30.981 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T01:52:30.981 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-12T01:52:30.981 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-12T01:52:30.981 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-09-12T01:52:30.981 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-12T01:52:30.981 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T01:52:30.981 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-12T01:52:30.981 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-12T01:52:30.981 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-12T01:52:30.990 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-12T01:52:30.990 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-12T01:52:30.990 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-12T01:52:30.990 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-09-12T01:52:30.990 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-12T01:52:30.990 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-12T01:52:30.990 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-12T01:52:30.990 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-12T01:52:30.990 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-12T01:52:30.990 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-12T01:52:30.990 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-12T01:52:30.990 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-12T01:52:30.991 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-12T01:52:30.991 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-09-12T01:52:30.991 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-12T01:52:30.991 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T01:52:30.991 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "btime": "2024-09-12T01:52:12:840866+0000", 2024-09-12T01:52:30.991 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-12T01:52:30.991 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-12T01:52:30.991 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-09-12T01:52:30.991 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-12T01:52:30.991 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-12T01:52:30.991 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-12T01:52:30.991 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-12T01:52:30.991 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-12T01:52:30.991 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-12T01:52:30.992 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "restful" 2024-09-12T01:52:30.992 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout ], 2024-09-12T01:52:30.992 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-12T01:52:30.992 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-09-12T01:52:30.992 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-12T01:52:30.992 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T01:52:30.992 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "modified": "2024-09-12T01:52:12.841650+0000", 2024-09-12T01:52:30.992 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-12T01:52:30.992 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-09-12T01:52:30.992 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-12T01:52:30.992 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout } 2024-09-12T01:52:30.992 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:30.284+0000 7f5b03ad1640 1 Processor -- start 2024-09-12T01:52:30.992 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:30.285+0000 7f5b03ad1640 1 -- start start 2024-09-12T01:52:30.993 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:30.285+0000 7f5b03ad1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5afc1065d0 0x7f5afc1069d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:30.993 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:30.285+0000 7f5b03ad1640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5afc106fa0 con 0x7f5afc1065d0 2024-09-12T01:52:30.993 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:30.285+0000 7f5b0186d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5afc1065d0 0x7f5afc1069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:30.993 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:30.285+0000 7f5b0186d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5afc1065d0 0x7f5afc1069d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39126/0 (socket says 172.21.15.16:39126) 2024-09-12T01:52:30.993 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:30.285+0000 7f5b0186d640 1 -- 172.21.15.16:0/1895756013 learned_addr learned my addr 172.21.15.16:0/1895756013 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:52:30.993 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:30.286+0000 7f5b0186d640 1 -- 172.21.15.16:0/1895756013 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5afc1077d0 con 0x7f5afc1065d0 2024-09-12T01:52:30.993 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:30.286+0000 7f5b0186d640 1 --2- 172.21.15.16:0/1895756013 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5afc1065d0 0x7f5afc1069d0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f5af8009820 tx=0x7f5af802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d97db7ec338d650 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:52:30.993 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:30.287+0000 7f5b0086b640 1 -- 172.21.15.16:0/1895756013 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5af8037070 con 0x7f5afc1065d0 2024-09-12T01:52:30.993 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:30.287+0000 7f5b0086b640 1 -- 172.21.15.16:0/1895756013 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5af8032900 con 0x7f5afc1065d0 2024-09-12T01:52:30.993 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:30.287+0000 7f5b0086b640 1 -- 172.21.15.16:0/1895756013 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5af8030850 con 0x7f5afc1065d0 2024-09-12T01:52:30.993 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:30.287+0000 7f5b03ad1640 1 -- 172.21.15.16:0/1895756013 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5afc1065d0 msgr2=0x7f5afc1069d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:52:30.993 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:30.287+0000 7f5b03ad1640 1 --2- 172.21.15.16:0/1895756013 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5afc1065d0 0x7f5afc1069d0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f5af8009820 tx=0x7f5af802a270 comp rx=0 tx=0).stop 2024-09-12T01:52:30.993 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:30.287+0000 7f5b03ad1640 1 -- 172.21.15.16:0/1895756013 shutdown_connections 2024-09-12T01:52:30.994 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:30.287+0000 7f5b03ad1640 1 --2- 172.21.15.16:0/1895756013 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5afc1065d0 0x7f5afc1069d0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:30.994 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:30.287+0000 7f5b03ad1640 1 -- 172.21.15.16:0/1895756013 >> 172.21.15.16:0/1895756013 conn(0x7f5afc101e20 msgr2=0x7f5afc104240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:52:30.994 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:30.288+0000 7f5b03ad1640 1 -- 172.21.15.16:0/1895756013 shutdown_connections 2024-09-12T01:52:30.994 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:30.288+0000 7f5b03ad1640 1 -- 172.21.15.16:0/1895756013 wait complete. 2024-09-12T01:52:30.994 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:30.288+0000 7f5b03ad1640 1 Processor -- start 2024-09-12T01:52:30.994 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:30.288+0000 7f5b03ad1640 1 -- start start 2024-09-12T01:52:30.994 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:30.288+0000 7f5b03ad1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5afc1065d0 0x7f5afc19b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:30.994 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:30.288+0000 7f5b03ad1640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5afc10ab80 con 0x7f5afc1065d0 2024-09-12T01:52:30.994 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:30.289+0000 7f5b0186d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5afc1065d0 0x7f5afc19b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:30.994 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:30.289+0000 7f5b0186d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5afc1065d0 0x7f5afc19b780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39132/0 (socket says 172.21.15.16:39132) 2024-09-12T01:52:30.994 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:30.289+0000 7f5b0186d640 1 -- 172.21.15.16:0/2637131383 learned_addr learned my addr 172.21.15.16:0/2637131383 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:52:30.994 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:30.289+0000 7f5b0186d640 1 -- 172.21.15.16:0/2637131383 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5afc19bcc0 con 0x7f5afc1065d0 2024-09-12T01:52:30.994 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:30.289+0000 7f5b0186d640 1 --2- 172.21.15.16:0/2637131383 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5afc1065d0 0x7f5afc19b780 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f5af80097f0 tx=0x7f5af8030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:52:30.994 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:30.289+0000 7f5aeeffd640 1 -- 172.21.15.16:0/2637131383 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5af8037040 con 0x7f5afc1065d0 2024-09-12T01:52:30.995 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:30.289+0000 7f5aeeffd640 1 -- 172.21.15.16:0/2637131383 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5af8040070 con 0x7f5afc1065d0 2024-09-12T01:52:30.995 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:30.289+0000 7f5b03ad1640 1 -- 172.21.15.16:0/2637131383 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5afc19bf50 con 0x7f5afc1065d0 2024-09-12T01:52:30.995 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:30.289+0000 7f5aeeffd640 1 -- 172.21.15.16:0/2637131383 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5af803a390 con 0x7f5afc1065d0 2024-09-12T01:52:30.995 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:30.289+0000 7f5b03ad1640 1 -- 172.21.15.16:0/2637131383 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5afc19ec40 con 0x7f5afc1065d0 2024-09-12T01:52:30.995 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:30.290+0000 7f5aeeffd640 1 -- 172.21.15.16:0/2637131383 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f5af803a650 con 0x7f5afc1065d0 2024-09-12T01:52:30.995 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:30.290+0000 7f5aeeffd640 1 -- 172.21.15.16:0/2637131383 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f5af80459f0 con 0x7f5afc1065d0 2024-09-12T01:52:30.995 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:30.290+0000 7f5b03ad1640 1 -- 172.21.15.16:0/2637131383 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5afc106a50 con 0x7f5afc1065d0 2024-09-12T01:52:30.995 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:30.293+0000 7f5aeeffd640 1 -- 172.21.15.16:0/2637131383 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f5af8045460 con 0x7f5afc1065d0 2024-09-12T01:52:30.995 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:30.356+0000 7f5b03ad1640 1 -- 172.21.15.16:0/2637131383 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f5afc1956a0 con 0x7f5afc1065d0 2024-09-12T01:52:30.995 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:30.357+0000 7f5aeeffd640 1 -- 172.21.15.16:0/2637131383 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1292 (secure 0 0 0) 0x7f5af803b430 con 0x7f5afc1065d0 2024-09-12T01:52:30.995 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:30.358+0000 7f5b03ad1640 1 -- 172.21.15.16:0/2637131383 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5afc1065d0 msgr2=0x7f5afc19b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:52:30.995 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:30.358+0000 7f5b03ad1640 1 --2- 172.21.15.16:0/2637131383 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5afc1065d0 0x7f5afc19b780 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f5af80097f0 tx=0x7f5af8030d80 comp rx=0 tx=0).stop 2024-09-12T01:52:30.995 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:30.358+0000 7f5b03ad1640 1 -- 172.21.15.16:0/2637131383 shutdown_connections 2024-09-12T01:52:30.995 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:30.358+0000 7f5b03ad1640 1 --2- 172.21.15.16:0/2637131383 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5afc1065d0 0x7f5afc19b780 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:30.996 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:30.358+0000 7f5b03ad1640 1 -- 172.21.15.16:0/2637131383 >> 172.21.15.16:0/2637131383 conn(0x7f5afc101e20 msgr2=0x7f5afc102d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:52:30.996 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:30.358+0000 7f5b03ad1640 1 -- 172.21.15.16:0/2637131383 shutdown_connections 2024-09-12T01:52:30.996 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:30.358+0000 7f5b03ad1640 1 -- 172.21.15.16:0/2637131383 wait complete. 2024-09-12T01:52:30.996 INFO:teuthology.orchestra.run.smithi016.stdout:mgr not available, waiting (1/15)... 2024-09-12T01:52:31.433 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 01:52:31 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-x[31173]: 2024-09-12T01:52:31.067+0000 7f3211d03040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-12T01:52:32.436 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 01:52:32 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-x[31173]: 2024-09-12T01:52:32.000+0000 7f3211d03040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-12T01:52:32.436 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 01:52:32 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-x[31173]: 2024-09-12T01:52:32.071+0000 7f3211d03040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-12T01:52:33.105 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 01:52:32 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-x[31173]: 2024-09-12T01:52:32.837+0000 7f3211d03040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-12T01:52:33.106 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 01:52:32 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-x[31173]: 2024-09-12T01:52:32.958+0000 7f3211d03040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-12T01:52:33.368 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 01:52:33 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-x[31173]: 2024-09-12T01:52:33.112+0000 7f3211d03040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-12T01:52:33.368 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 01:52:33 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-x[31173]: 2024-09-12T01:52:33.177+0000 7f3211d03040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-12T01:52:33.368 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 01:52:33 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-x[31173]: 2024-09-12T01:52:33.338+0000 7f3211d03040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-12T01:52:33.649 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:33 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/1735615726' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-12T01:52:33.650 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 01:52:33 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-x[31173]: 2024-09-12T01:52:33.414+0000 7f3211d03040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-12T01:52:33.650 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 01:52:33 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-x[31173]: 2024-09-12T01:52:33.481+0000 7f3211d03040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-12T01:52:33.650 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 01:52:33 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-x[31173]: 2024-09-12T01:52:33.649+0000 7f3211d03040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-12T01:52:33.933 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 01:52:33 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-x[31173]: 2024-09-12T01:52:33.721+0000 7f3211d03040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-12T01:52:33.933 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 01:52:33 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-x[31173]: 2024-09-12T01:52:33.792+0000 7f3211d03040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-12T01:52:33.934 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 01:52:33 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-x[31173]: 2024-09-12T01:52:33.880+0000 7f3211d03040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-12T01:52:34.002 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 2024-09-12T01:52:34.002 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout { 2024-09-12T01:52:34.002 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "fsid": "879f3b56-70a9-11ef-bcea-c7b262605968", 2024-09-12T01:52:34.002 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "health": { 2024-09-12T01:52:34.002 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-12T01:52:34.002 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-12T01:52:34.002 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-12T01:52:34.002 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-09-12T01:52:34.002 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-12T01:52:34.002 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-12T01:52:34.003 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 0 2024-09-12T01:52:34.003 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout ], 2024-09-12T01:52:34.003 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-12T01:52:34.003 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "a" 2024-09-12T01:52:34.003 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout ], 2024-09-12T01:52:34.003 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "quorum_age": 15, 2024-09-12T01:52:34.003 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-12T01:52:34.003 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T01:52:34.003 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-12T01:52:34.003 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-12T01:52:34.003 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-09-12T01:52:34.003 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-12T01:52:34.003 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T01:52:34.003 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-12T01:52:34.004 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-12T01:52:34.004 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-12T01:52:34.012 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-12T01:52:34.012 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-12T01:52:34.012 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-12T01:52:34.012 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-09-12T01:52:34.012 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-12T01:52:34.012 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-12T01:52:34.012 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-12T01:52:34.012 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-12T01:52:34.012 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-12T01:52:34.012 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-12T01:52:34.012 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-12T01:52:34.013 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-12T01:52:34.013 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-12T01:52:34.013 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-09-12T01:52:34.013 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-12T01:52:34.013 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T01:52:34.013 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "btime": "2024-09-12T01:52:12:840866+0000", 2024-09-12T01:52:34.013 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-12T01:52:34.013 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-12T01:52:34.013 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-09-12T01:52:34.013 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-12T01:52:34.013 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-12T01:52:34.013 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-12T01:52:34.013 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-12T01:52:34.014 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-12T01:52:34.014 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-12T01:52:34.014 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "restful" 2024-09-12T01:52:34.014 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout ], 2024-09-12T01:52:34.014 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-12T01:52:34.014 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-09-12T01:52:34.014 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-12T01:52:34.014 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T01:52:34.014 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "modified": "2024-09-12T01:52:12.841650+0000", 2024-09-12T01:52:34.014 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-12T01:52:34.014 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-09-12T01:52:34.014 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-12T01:52:34.014 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout } 2024-09-12T01:52:34.014 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:33.390+0000 7fb3f8a6d640 1 Processor -- start 2024-09-12T01:52:34.015 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:33.390+0000 7fb3f8a6d640 1 -- start start 2024-09-12T01:52:34.015 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:33.391+0000 7fb3f8a6d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3f41081c0 0x7fb3f410a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:34.015 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:33.391+0000 7fb3f8a6d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb3f407bdb0 con 0x7fb3f41081c0 2024-09-12T01:52:34.015 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:33.391+0000 7fb3f259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3f41081c0 0x7fb3f410a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:34.015 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:33.391+0000 7fb3f259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3f41081c0 0x7fb3f410a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34222/0 (socket says 172.21.15.16:34222) 2024-09-12T01:52:34.015 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:33.391+0000 7fb3f259c640 1 -- 172.21.15.16:0/3113512481 learned_addr learned my addr 172.21.15.16:0/3113512481 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:52:34.015 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:33.391+0000 7fb3f259c640 1 -- 172.21.15.16:0/3113512481 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3f410aaf0 con 0x7fb3f41081c0 2024-09-12T01:52:34.015 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:33.392+0000 7fb3f259c640 1 --2- 172.21.15.16:0/3113512481 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3f41081c0 0x7fb3f410a5b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb3e4009820 tx=0x7fb3e402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2e038e751f15c08 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:52:34.015 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:33.392+0000 7fb3f1d9b640 1 -- 172.21.15.16:0/3113512481 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb3e4037070 con 0x7fb3f41081c0 2024-09-12T01:52:34.015 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:33.392+0000 7fb3f1d9b640 1 -- 172.21.15.16:0/3113512481 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb3e4032900 con 0x7fb3f41081c0 2024-09-12T01:52:34.015 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:33.392+0000 7fb3f1d9b640 1 -- 172.21.15.16:0/3113512481 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb3e4030850 con 0x7fb3f41081c0 2024-09-12T01:52:34.015 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:33.393+0000 7fb3f8a6d640 1 -- 172.21.15.16:0/3113512481 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3f41081c0 msgr2=0x7fb3f410a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:52:34.015 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:33.393+0000 7fb3f8a6d640 1 --2- 172.21.15.16:0/3113512481 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3f41081c0 0x7fb3f410a5b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb3e4009820 tx=0x7fb3e402a270 comp rx=0 tx=0).stop 2024-09-12T01:52:34.016 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:33.393+0000 7fb3f8a6d640 1 -- 172.21.15.16:0/3113512481 shutdown_connections 2024-09-12T01:52:34.016 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:33.393+0000 7fb3f8a6d640 1 --2- 172.21.15.16:0/3113512481 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3f41081c0 0x7fb3f410a5b0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:34.016 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:33.393+0000 7fb3f8a6d640 1 -- 172.21.15.16:0/3113512481 >> 172.21.15.16:0/3113512481 conn(0x7fb3f4101e20 msgr2=0x7fb3f4104240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:52:34.016 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:33.393+0000 7fb3f8a6d640 1 -- 172.21.15.16:0/3113512481 shutdown_connections 2024-09-12T01:52:34.016 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:33.393+0000 7fb3f8a6d640 1 -- 172.21.15.16:0/3113512481 wait complete. 2024-09-12T01:52:34.016 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:33.394+0000 7fb3f8a6d640 1 Processor -- start 2024-09-12T01:52:34.016 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:33.394+0000 7fb3f8a6d640 1 -- start start 2024-09-12T01:52:34.016 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:33.394+0000 7fb3f8a6d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3f41081c0 0x7fb3f4107a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:34.016 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:33.394+0000 7fb3f8a6d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb3f410bad0 con 0x7fb3f41081c0 2024-09-12T01:52:34.016 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:33.395+0000 7fb3f259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3f41081c0 0x7fb3f4107a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:34.016 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:33.395+0000 7fb3f259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3f41081c0 0x7fb3f4107a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34234/0 (socket says 172.21.15.16:34234) 2024-09-12T01:52:34.016 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:33.395+0000 7fb3f259c640 1 -- 172.21.15.16:0/1735615726 learned_addr learned my addr 172.21.15.16:0/1735615726 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:52:34.016 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:33.395+0000 7fb3f259c640 1 -- 172.21.15.16:0/1735615726 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3f4107f80 con 0x7fb3f41081c0 2024-09-12T01:52:34.017 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:33.395+0000 7fb3f259c640 1 --2- 172.21.15.16:0/1735615726 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3f41081c0 0x7fb3f4107a40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb3e40097f0 tx=0x7fb3e4030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:52:34.017 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:33.395+0000 7fb3ebfff640 1 -- 172.21.15.16:0/1735615726 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb3e4037040 con 0x7fb3f41081c0 2024-09-12T01:52:34.017 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:33.395+0000 7fb3f8a6d640 1 -- 172.21.15.16:0/1735615726 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3f4106170 con 0x7fb3f41081c0 2024-09-12T01:52:34.017 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:33.395+0000 7fb3ebfff640 1 -- 172.21.15.16:0/1735615726 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb3e4040070 con 0x7fb3f41081c0 2024-09-12T01:52:34.017 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:33.395+0000 7fb3ebfff640 1 -- 172.21.15.16:0/1735615726 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb3e403a390 con 0x7fb3f41081c0 2024-09-12T01:52:34.017 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:33.395+0000 7fb3f8a6d640 1 -- 172.21.15.16:0/1735615726 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3f4106650 con 0x7fb3f41081c0 2024-09-12T01:52:34.017 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:33.396+0000 7fb3ebfff640 1 -- 172.21.15.16:0/1735615726 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fb3e403a630 con 0x7fb3f41081c0 2024-09-12T01:52:34.017 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:33.396+0000 7fb3ebfff640 1 -- 172.21.15.16:0/1735615726 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fb3e40459a0 con 0x7fb3f41081c0 2024-09-12T01:52:34.017 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:33.396+0000 7fb3f8a6d640 1 -- 172.21.15.16:0/1735615726 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3f4108620 con 0x7fb3f41081c0 2024-09-12T01:52:34.017 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:33.400+0000 7fb3ebfff640 1 -- 172.21.15.16:0/1735615726 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fb3e4045460 con 0x7fb3f41081c0 2024-09-12T01:52:34.017 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:33.463+0000 7fb3f8a6d640 1 -- 172.21.15.16:0/1735615726 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fb3f4106b80 con 0x7fb3f41081c0 2024-09-12T01:52:34.017 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:33.464+0000 7fb3ebfff640 1 -- 172.21.15.16:0/1735615726 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1292 (secure 0 0 0) 0x7fb3e403b430 con 0x7fb3f41081c0 2024-09-12T01:52:34.017 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:33.464+0000 7fb3f8a6d640 1 -- 172.21.15.16:0/1735615726 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3f41081c0 msgr2=0x7fb3f4107a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:52:34.018 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:33.464+0000 7fb3f8a6d640 1 --2- 172.21.15.16:0/1735615726 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3f41081c0 0x7fb3f4107a40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb3e40097f0 tx=0x7fb3e4030d80 comp rx=0 tx=0).stop 2024-09-12T01:52:34.018 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:33.465+0000 7fb3f8a6d640 1 -- 172.21.15.16:0/1735615726 shutdown_connections 2024-09-12T01:52:34.018 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:33.465+0000 7fb3f8a6d640 1 --2- 172.21.15.16:0/1735615726 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3f41081c0 0x7fb3f4107a40 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:34.018 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:33.465+0000 7fb3f8a6d640 1 -- 172.21.15.16:0/1735615726 >> 172.21.15.16:0/1735615726 conn(0x7fb3f4101e20 msgr2=0x7fb3f41029c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:52:34.018 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:33.465+0000 7fb3f8a6d640 1 -- 172.21.15.16:0/1735615726 shutdown_connections 2024-09-12T01:52:34.018 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:33.465+0000 7fb3f8a6d640 1 -- 172.21.15.16:0/1735615726 wait complete. 2024-09-12T01:52:34.018 INFO:teuthology.orchestra.run.smithi016.stdout:mgr not available, waiting (2/15)... 2024-09-12T01:52:34.433 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 01:52:33 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-x[31173]: 2024-09-12T01:52:33.944+0000 7f3211d03040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-12T01:52:34.433 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 01:52:34 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-x[31173]: 2024-09-12T01:52:34.008+0000 7f3211d03040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-12T01:52:35.183 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 01:52:34 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-x[31173]: 2024-09-12T01:52:34.842+0000 7f3211d03040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-12T01:52:35.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:34 smithi016 ceph-mon[30947]: Activating manager daemon x 2024-09-12T01:52:35.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:34 smithi016 ceph-mon[30947]: mgrmap e2: x(active, starting, since 0.00340211s) 2024-09-12T01:52:35.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:34 smithi016 ceph-mon[30947]: from='mgr.14100 172.21.15.16:0/3298182738' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-12T01:52:35.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:34 smithi016 ceph-mon[30947]: from='mgr.14100 172.21.15.16:0/3298182738' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-12T01:52:35.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:34 smithi016 ceph-mon[30947]: from='mgr.14100 172.21.15.16:0/3298182738' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-12T01:52:35.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:34 smithi016 ceph-mon[30947]: from='mgr.14100 172.21.15.16:0/3298182738' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T01:52:35.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:34 smithi016 ceph-mon[30947]: from='mgr.14100 172.21.15.16:0/3298182738' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-12T01:52:35.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:34 smithi016 ceph-mon[30947]: Manager daemon x is now available 2024-09-12T01:52:35.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:34 smithi016 ceph-mon[30947]: from='mgr.14100 172.21.15.16:0/3298182738' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-12T01:52:35.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:34 smithi016 ceph-mon[30947]: from='mgr.14100 172.21.15.16:0/3298182738' entity='mgr.x' 2024-09-12T01:52:35.185 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:34 smithi016 ceph-mon[30947]: from='mgr.14100 172.21.15.16:0/3298182738' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-12T01:52:35.185 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:34 smithi016 ceph-mon[30947]: from='mgr.14100 172.21.15.16:0/3298182738' entity='mgr.x' 2024-09-12T01:52:35.185 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:34 smithi016 ceph-mon[30947]: from='mgr.14100 172.21.15.16:0/3298182738' entity='mgr.x' 2024-09-12T01:52:36.933 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:36 smithi016 ceph-mon[30947]: mgrmap e3: x(active, since 1.00772s) 2024-09-12T01:52:36.934 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:36 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/2896701198' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-12T01:52:37.237 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 2024-09-12T01:52:37.238 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout { 2024-09-12T01:52:37.238 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "fsid": "879f3b56-70a9-11ef-bcea-c7b262605968", 2024-09-12T01:52:37.238 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "health": { 2024-09-12T01:52:37.238 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-12T01:52:37.238 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-12T01:52:37.238 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-12T01:52:37.238 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-09-12T01:52:37.238 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-12T01:52:37.238 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-12T01:52:37.238 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 0 2024-09-12T01:52:37.238 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout ], 2024-09-12T01:52:37.238 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-12T01:52:37.238 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "a" 2024-09-12T01:52:37.239 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout ], 2024-09-12T01:52:37.239 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "quorum_age": 18, 2024-09-12T01:52:37.239 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-12T01:52:37.239 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T01:52:37.239 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-12T01:52:37.239 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-12T01:52:37.239 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-09-12T01:52:37.239 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-12T01:52:37.239 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T01:52:37.239 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-12T01:52:37.239 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-12T01:52:37.239 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-12T01:52:37.248 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-12T01:52:37.248 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-12T01:52:37.248 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-12T01:52:37.248 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-09-12T01:52:37.248 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-12T01:52:37.248 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-12T01:52:37.248 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-12T01:52:37.249 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-12T01:52:37.249 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-12T01:52:37.249 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-12T01:52:37.249 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-12T01:52:37.249 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-12T01:52:37.249 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-12T01:52:37.249 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-09-12T01:52:37.249 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-12T01:52:37.249 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T01:52:37.249 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "btime": "2024-09-12T01:52:12:840866+0000", 2024-09-12T01:52:37.249 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-12T01:52:37.249 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-12T01:52:37.249 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-09-12T01:52:37.249 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-12T01:52:37.250 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-12T01:52:37.250 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-12T01:52:37.250 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-12T01:52:37.250 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-12T01:52:37.250 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-12T01:52:37.250 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "restful" 2024-09-12T01:52:37.250 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout ], 2024-09-12T01:52:37.250 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-12T01:52:37.250 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-09-12T01:52:37.250 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-12T01:52:37.250 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T01:52:37.250 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "modified": "2024-09-12T01:52:12.841650+0000", 2024-09-12T01:52:37.250 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-12T01:52:37.250 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-09-12T01:52:37.250 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-12T01:52:37.251 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout } 2024-09-12T01:52:37.251 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:36.400+0000 7f0e6cf55640 1 Processor -- start 2024-09-12T01:52:37.251 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:36.400+0000 7f0e6cf55640 1 -- start start 2024-09-12T01:52:37.251 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:36.401+0000 7f0e6cf55640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e68107ca0 0x7f0e681080a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:37.251 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:36.401+0000 7f0e6cf55640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0e68108670 con 0x7f0e68107ca0 2024-09-12T01:52:37.251 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:36.401+0000 7f0e6659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e68107ca0 0x7f0e681080a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:37.251 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:36.401+0000 7f0e6659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e68107ca0 0x7f0e681080a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34300/0 (socket says 172.21.15.16:34300) 2024-09-12T01:52:37.251 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:36.401+0000 7f0e6659c640 1 -- 172.21.15.16:0/295669156 learned_addr learned my addr 172.21.15.16:0/295669156 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:52:37.251 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:36.401+0000 7f0e6659c640 1 -- 172.21.15.16:0/295669156 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e68106950 con 0x7f0e68107ca0 2024-09-12T01:52:37.251 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:36.402+0000 7f0e6659c640 1 --2- 172.21.15.16:0/295669156 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e68107ca0 0x7f0e681080a0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f0e58009820 tx=0x7f0e5802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e92b8225aa3090a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:52:37.251 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:36.402+0000 7f0e6559a640 1 -- 172.21.15.16:0/295669156 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0e58037070 con 0x7f0e68107ca0 2024-09-12T01:52:37.251 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:36.402+0000 7f0e6559a640 1 -- 172.21.15.16:0/295669156 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0e58032900 con 0x7f0e68107ca0 2024-09-12T01:52:37.251 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:36.402+0000 7f0e6559a640 1 -- 172.21.15.16:0/295669156 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0e58030850 con 0x7f0e68107ca0 2024-09-12T01:52:37.252 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:36.403+0000 7f0e6cf55640 1 -- 172.21.15.16:0/295669156 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e68107ca0 msgr2=0x7f0e681080a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:52:37.252 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:36.403+0000 7f0e6cf55640 1 --2- 172.21.15.16:0/295669156 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e68107ca0 0x7f0e681080a0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f0e58009820 tx=0x7f0e5802a270 comp rx=0 tx=0).stop 2024-09-12T01:52:37.252 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:36.403+0000 7f0e6cf55640 1 -- 172.21.15.16:0/295669156 shutdown_connections 2024-09-12T01:52:37.252 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:36.403+0000 7f0e6cf55640 1 --2- 172.21.15.16:0/295669156 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e68107ca0 0x7f0e681080a0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:37.252 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:36.403+0000 7f0e6cf55640 1 -- 172.21.15.16:0/295669156 >> 172.21.15.16:0/295669156 conn(0x7f0e681015f0 msgr2=0x7f0e68103a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:52:37.252 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:36.403+0000 7f0e6cf55640 1 -- 172.21.15.16:0/295669156 shutdown_connections 2024-09-12T01:52:37.252 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:36.403+0000 7f0e6cf55640 1 -- 172.21.15.16:0/295669156 wait complete. 2024-09-12T01:52:37.252 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:36.404+0000 7f0e6cf55640 1 Processor -- start 2024-09-12T01:52:37.252 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:36.404+0000 7f0e6cf55640 1 -- start start 2024-09-12T01:52:37.252 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:36.404+0000 7f0e6cf55640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e68107ca0 0x7f0e6819b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:37.252 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:36.404+0000 7f0e6cf55640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0e6810ab90 con 0x7f0e68107ca0 2024-09-12T01:52:37.252 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:36.404+0000 7f0e6659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e68107ca0 0x7f0e6819b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:37.252 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:36.404+0000 7f0e6659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e68107ca0 0x7f0e6819b750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34312/0 (socket says 172.21.15.16:34312) 2024-09-12T01:52:37.253 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:36.404+0000 7f0e6659c640 1 -- 172.21.15.16:0/2896701198 learned_addr learned my addr 172.21.15.16:0/2896701198 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:52:37.253 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:36.405+0000 7f0e6659c640 1 -- 172.21.15.16:0/2896701198 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e6819bc90 con 0x7f0e68107ca0 2024-09-12T01:52:37.253 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:36.405+0000 7f0e6659c640 1 --2- 172.21.15.16:0/2896701198 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e68107ca0 0x7f0e6819b750 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f0e58009940 tx=0x7f0e58032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:52:37.253 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:36.405+0000 7f0e5f7fe640 1 -- 172.21.15.16:0/2896701198 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0e58042070 con 0x7f0e68107ca0 2024-09-12T01:52:37.253 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:36.405+0000 7f0e5f7fe640 1 -- 172.21.15.16:0/2896701198 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0e580308c0 con 0x7f0e68107ca0 2024-09-12T01:52:37.253 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:36.405+0000 7f0e6cf55640 1 -- 172.21.15.16:0/2896701198 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e6819bf20 con 0x7f0e68107ca0 2024-09-12T01:52:37.253 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:36.405+0000 7f0e5f7fe640 1 -- 172.21.15.16:0/2896701198 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0e58037040 con 0x7f0e68107ca0 2024-09-12T01:52:37.253 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:36.405+0000 7f0e6cf55640 1 -- 172.21.15.16:0/2896701198 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e6819ec10 con 0x7f0e68107ca0 2024-09-12T01:52:37.253 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:36.406+0000 7f0e5f7fe640 1 -- 172.21.15.16:0/2896701198 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 3) ==== 46170+0+0 (secure 0 0 0) 0x7f0e58045070 con 0x7f0e68107ca0 2024-09-12T01:52:37.253 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:36.406+0000 7f0e5f7fe640 1 --2- 172.21.15.16:0/2896701198 >> [v2:172.21.15.16:6800/2754331153,v1:172.21.15.16:6801/2754331153] conn(0x7f0e54038e60 0x7f0e5403b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:37.253 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:36.406+0000 7f0e6cf55640 1 -- 172.21.15.16:0/2896701198 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e6810ad10 con 0x7f0e68107ca0 2024-09-12T01:52:37.253 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:36.406+0000 7f0e5f7fe640 1 -- 172.21.15.16:0/2896701198 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f0e5806bec0 con 0x7f0e68107ca0 2024-09-12T01:52:37.253 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:36.407+0000 7f0e65d9b640 1 --2- 172.21.15.16:0/2896701198 >> [v2:172.21.15.16:6800/2754331153,v1:172.21.15.16:6801/2754331153] conn(0x7f0e54038e60 0x7f0e5403b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:37.254 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:36.407+0000 7f0e65d9b640 1 --2- 172.21.15.16:0/2896701198 >> [v2:172.21.15.16:6800/2754331153,v1:172.21.15.16:6801/2754331153] conn(0x7f0e54038e60 0x7f0e5403b320 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f0e500098c0 tx=0x7f0e50006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:52:37.254 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:36.412+0000 7f0e5f7fe640 1 -- 172.21.15.16:0/2896701198 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0e5803b330 con 0x7f0e68107ca0 2024-09-12T01:52:37.254 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:36.658+0000 7f0e6cf55640 1 -- 172.21.15.16:0/2896701198 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f0e6802da00 con 0x7f0e68107ca0 2024-09-12T01:52:37.254 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:36.658+0000 7f0e5f7fe640 1 -- 172.21.15.16:0/2896701198 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7f0e5803c760 con 0x7f0e68107ca0 2024-09-12T01:52:37.254 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:36.661+0000 7f0e6cf55640 1 -- 172.21.15.16:0/2896701198 >> [v2:172.21.15.16:6800/2754331153,v1:172.21.15.16:6801/2754331153] conn(0x7f0e54038e60 msgr2=0x7f0e5403b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:52:37.254 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:36.661+0000 7f0e6cf55640 1 --2- 172.21.15.16:0/2896701198 >> [v2:172.21.15.16:6800/2754331153,v1:172.21.15.16:6801/2754331153] conn(0x7f0e54038e60 0x7f0e5403b320 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f0e500098c0 tx=0x7f0e50006db0 comp rx=0 tx=0).stop 2024-09-12T01:52:37.254 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:36.661+0000 7f0e6cf55640 1 -- 172.21.15.16:0/2896701198 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e68107ca0 msgr2=0x7f0e6819b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:52:37.254 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:36.661+0000 7f0e6cf55640 1 --2- 172.21.15.16:0/2896701198 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e68107ca0 0x7f0e6819b750 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f0e58009940 tx=0x7f0e58032ce0 comp rx=0 tx=0).stop 2024-09-12T01:52:37.254 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:36.661+0000 7f0e6cf55640 1 -- 172.21.15.16:0/2896701198 shutdown_connections 2024-09-12T01:52:37.255 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:36.661+0000 7f0e6cf55640 1 --2- 172.21.15.16:0/2896701198 >> [v2:172.21.15.16:6800/2754331153,v1:172.21.15.16:6801/2754331153] conn(0x7f0e54038e60 0x7f0e5403b320 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:37.255 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:36.661+0000 7f0e6cf55640 1 --2- 172.21.15.16:0/2896701198 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e68107ca0 0x7f0e6819b750 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:37.255 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:36.661+0000 7f0e6cf55640 1 -- 172.21.15.16:0/2896701198 >> 172.21.15.16:0/2896701198 conn(0x7f0e681015f0 msgr2=0x7f0e68102510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:52:37.255 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:36.661+0000 7f0e6cf55640 1 -- 172.21.15.16:0/2896701198 shutdown_connections 2024-09-12T01:52:37.255 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:36.661+0000 7f0e6cf55640 1 -- 172.21.15.16:0/2896701198 wait complete. 2024-09-12T01:52:37.255 INFO:teuthology.orchestra.run.smithi016.stdout:mgr is available 2024-09-12T01:52:38.146 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:37 smithi016 ceph-mon[30947]: mgrmap e4: x(active, since 2s) 2024-09-12T01:52:38.146 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:37 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/3038619821' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-12T01:52:38.447 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 2024-09-12T01:52:38.447 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout [global] 2024-09-12T01:52:38.447 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout fsid = 879f3b56-70a9-11ef-bcea-c7b262605968 2024-09-12T01:52:38.448 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.16:3300,v1:172.21.15.16:6789] 2024-09-12T01:52:38.448 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-12T01:52:38.448 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-12T01:52:38.448 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-12T01:52:38.448 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-12T01:52:38.448 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 2024-09-12T01:52:38.448 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-12T01:52:38.448 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-12T01:52:38.448 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 2024-09-12T01:52:38.448 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout [osd] 2024-09-12T01:52:38.448 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-12T01:52:38.448 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-12T01:52:38.448 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-12T01:52:38.449 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-12T01:52:38.449 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-12T01:52:38.449 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:37.614+0000 7f493f05c640 1 Processor -- start 2024-09-12T01:52:38.449 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:37.614+0000 7f493f05c640 1 -- start start 2024-09-12T01:52:38.449 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:37.615+0000 7f493f05c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4938108180 0x7f493810a570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:38.449 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:37.615+0000 7f493f05c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f493807bdf0 con 0x7f4938108180 2024-09-12T01:52:38.449 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:37.615+0000 7f493cdf8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4938108180 0x7f493810a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:38.449 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:37.615+0000 7f493cdf8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4938108180 0x7f493810a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34318/0 (socket says 172.21.15.16:34318) 2024-09-12T01:52:38.449 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:37.615+0000 7f493cdf8640 1 -- 172.21.15.16:0/3495755734 learned_addr learned my addr 172.21.15.16:0/3495755734 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:52:38.449 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:37.615+0000 7f493cdf8640 1 -- 172.21.15.16:0/3495755734 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f493810aab0 con 0x7f4938108180 2024-09-12T01:52:38.449 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:37.616+0000 7f493cdf8640 1 --2- 172.21.15.16:0/3495755734 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4938108180 0x7f493810a570 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f492c009820 tx=0x7f492c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fae5aba08d3a9163 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:52:38.449 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:37.617+0000 7f49377fe640 1 -- 172.21.15.16:0/3495755734 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f492c037070 con 0x7f4938108180 2024-09-12T01:52:38.449 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:37.617+0000 7f49377fe640 1 -- 172.21.15.16:0/3495755734 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f492c032900 con 0x7f4938108180 2024-09-12T01:52:38.450 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:37.617+0000 7f49377fe640 1 -- 172.21.15.16:0/3495755734 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f492c030850 con 0x7f4938108180 2024-09-12T01:52:38.450 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:37.617+0000 7f493f05c640 1 -- 172.21.15.16:0/3495755734 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4938108180 msgr2=0x7f493810a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:52:38.450 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:37.617+0000 7f493f05c640 1 --2- 172.21.15.16:0/3495755734 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4938108180 0x7f493810a570 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f492c009820 tx=0x7f492c02a270 comp rx=0 tx=0).stop 2024-09-12T01:52:38.450 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:37.617+0000 7f493f05c640 1 -- 172.21.15.16:0/3495755734 shutdown_connections 2024-09-12T01:52:38.450 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:37.617+0000 7f493f05c640 1 --2- 172.21.15.16:0/3495755734 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4938108180 0x7f493810a570 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:38.450 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:37.617+0000 7f493f05c640 1 -- 172.21.15.16:0/3495755734 >> 172.21.15.16:0/3495755734 conn(0x7f4938101e20 msgr2=0x7f4938104240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:52:38.450 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:37.617+0000 7f493f05c640 1 -- 172.21.15.16:0/3495755734 shutdown_connections 2024-09-12T01:52:38.450 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:37.617+0000 7f493f05c640 1 -- 172.21.15.16:0/3495755734 wait complete. 2024-09-12T01:52:38.450 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:37.618+0000 7f493f05c640 1 Processor -- start 2024-09-12T01:52:38.450 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:37.618+0000 7f493f05c640 1 -- start start 2024-09-12T01:52:38.450 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:37.618+0000 7f493f05c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4938108180 0x7f4938107a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:38.450 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:37.618+0000 7f493f05c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f493810baf0 con 0x7f4938108180 2024-09-12T01:52:38.450 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:37.619+0000 7f493cdf8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4938108180 0x7f4938107a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:38.450 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:37.619+0000 7f493cdf8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4938108180 0x7f4938107a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34334/0 (socket says 172.21.15.16:34334) 2024-09-12T01:52:38.451 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:37.619+0000 7f493cdf8640 1 -- 172.21.15.16:0/3038619821 learned_addr learned my addr 172.21.15.16:0/3038619821 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:52:38.451 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:37.619+0000 7f493cdf8640 1 -- 172.21.15.16:0/3038619821 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4938107f40 con 0x7f4938108180 2024-09-12T01:52:38.451 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:37.619+0000 7f493cdf8640 1 --2- 172.21.15.16:0/3038619821 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4938108180 0x7f4938107a00 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f492c032ce0 tx=0x7f492c032d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:52:38.451 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:37.619+0000 7f4935ffb640 1 -- 172.21.15.16:0/3038619821 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f492c042070 con 0x7f4938108180 2024-09-12T01:52:38.451 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:37.619+0000 7f4935ffb640 1 -- 172.21.15.16:0/3038619821 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f492c0308c0 con 0x7f4938108180 2024-09-12T01:52:38.451 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:37.619+0000 7f493f05c640 1 -- 172.21.15.16:0/3038619821 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4938106130 con 0x7f4938108180 2024-09-12T01:52:38.451 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:37.619+0000 7f4935ffb640 1 -- 172.21.15.16:0/3038619821 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f492c037040 con 0x7f4938108180 2024-09-12T01:52:38.451 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:37.620+0000 7f493f05c640 1 -- 172.21.15.16:0/3038619821 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4938106670 con 0x7f4938108180 2024-09-12T01:52:38.451 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:37.620+0000 7f4935ffb640 1 -- 172.21.15.16:0/3038619821 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7f492c045070 con 0x7f4938108180 2024-09-12T01:52:38.451 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:37.621+0000 7f4935ffb640 1 --2- 172.21.15.16:0/3038619821 >> [v2:172.21.15.16:6800/2754331153,v1:172.21.15.16:6801/2754331153] conn(0x7f4928038f20 0x7f492803b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:38.451 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:37.621+0000 7f4935ffb640 1 -- 172.21.15.16:0/3038619821 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f492c06bfe0 con 0x7f4938108180 2024-09-12T01:52:38.451 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:37.621+0000 7f493f05c640 1 -- 172.21.15.16:0/3038619821 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49381085e0 con 0x7f4938108180 2024-09-12T01:52:38.451 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:37.621+0000 7f4937fff640 1 --2- 172.21.15.16:0/3038619821 >> [v2:172.21.15.16:6800/2754331153,v1:172.21.15.16:6801/2754331153] conn(0x7f4928038f20 0x7f492803b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:38.452 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:37.621+0000 7f4937fff640 1 --2- 172.21.15.16:0/3038619821 >> [v2:172.21.15.16:6800/2754331153,v1:172.21.15.16:6801/2754331153] conn(0x7f4928038f20 0x7f492803b3e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f49240098c0 tx=0x7f4924006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:52:38.452 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:37.626+0000 7f4935ffb640 1 -- 172.21.15.16:0/3038619821 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f492c045520 con 0x7f4938108180 2024-09-12T01:52:38.452 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:37.805+0000 7f493f05c640 1 -- 172.21.15.16:0/3038619821 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f49381068f0 con 0x7f4938108180 2024-09-12T01:52:38.452 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:37.806+0000 7f4935ffb640 1 -- 172.21.15.16:0/3038619821 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+429 (secure 0 0 0) 0x7f492c03cd50 con 0x7f4938108180 2024-09-12T01:52:38.452 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:37.808+0000 7f493f05c640 1 -- 172.21.15.16:0/3038619821 >> [v2:172.21.15.16:6800/2754331153,v1:172.21.15.16:6801/2754331153] conn(0x7f4928038f20 msgr2=0x7f492803b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:52:38.452 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:37.808+0000 7f493f05c640 1 --2- 172.21.15.16:0/3038619821 >> [v2:172.21.15.16:6800/2754331153,v1:172.21.15.16:6801/2754331153] conn(0x7f4928038f20 0x7f492803b3e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f49240098c0 tx=0x7f4924006db0 comp rx=0 tx=0).stop 2024-09-12T01:52:38.452 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:37.808+0000 7f493f05c640 1 -- 172.21.15.16:0/3038619821 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4938108180 msgr2=0x7f4938107a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:52:38.452 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:37.808+0000 7f493f05c640 1 --2- 172.21.15.16:0/3038619821 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4938108180 0x7f4938107a00 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f492c032ce0 tx=0x7f492c032d10 comp rx=0 tx=0).stop 2024-09-12T01:52:38.452 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:37.808+0000 7f493f05c640 1 -- 172.21.15.16:0/3038619821 shutdown_connections 2024-09-12T01:52:38.452 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:37.808+0000 7f493f05c640 1 --2- 172.21.15.16:0/3038619821 >> [v2:172.21.15.16:6800/2754331153,v1:172.21.15.16:6801/2754331153] conn(0x7f4928038f20 0x7f492803b3e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:38.452 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:37.808+0000 7f493f05c640 1 --2- 172.21.15.16:0/3038619821 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4938108180 0x7f4938107a00 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:38.452 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:37.808+0000 7f493f05c640 1 -- 172.21.15.16:0/3038619821 >> 172.21.15.16:0/3038619821 conn(0x7f4938101e20 msgr2=0x7f4938102a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:52:38.452 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:37.808+0000 7f493f05c640 1 -- 172.21.15.16:0/3038619821 shutdown_connections 2024-09-12T01:52:38.453 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:37.808+0000 7f493f05c640 1 -- 172.21.15.16:0/3038619821 wait complete. 2024-09-12T01:52:38.453 INFO:teuthology.orchestra.run.smithi016.stdout:Enabling cephadm module... 2024-09-12T01:52:39.320 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:39 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/2440357995' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-09-12T01:52:39.320 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 01:52:39 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-x[31173]: ignoring --setuser ceph since I am not root 2024-09-12T01:52:39.320 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 01:52:39 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-x[31173]: ignoring --setgroup ceph since I am not root 2024-09-12T01:52:39.320 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 01:52:39 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-x[31173]: 2024-09-12T01:52:39.260+0000 7f090a605040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-12T01:52:39.321 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 01:52:39 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-x[31173]: 2024-09-12T01:52:39.319+0000 7f090a605040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-12T01:52:39.684 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 01:52:39 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-x[31173]: 2024-09-12T01:52:39.477+0000 7f090a605040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-12T01:52:39.684 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 01:52:39 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-x[31173]: 2024-09-12T01:52:39.546+0000 7f090a605040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-12T01:52:39.770 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:38.833+0000 7f3050272640 1 Processor -- start 2024-09-12T01:52:39.770 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:38.833+0000 7f3050272640 1 -- start start 2024-09-12T01:52:39.770 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:38.834+0000 7f3050272640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3048074030 0x7f3048072490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:39.770 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:38.834+0000 7f3050272640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3048072a60 con 0x7f3048074030 2024-09-12T01:52:39.770 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:38.834+0000 7f304e00e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3048074030 0x7f3048072490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:39.770 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:38.834+0000 7f304e00e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3048074030 0x7f3048072490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34338/0 (socket says 172.21.15.16:34338) 2024-09-12T01:52:39.770 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:38.834+0000 7f304e00e640 1 -- 172.21.15.16:0/2867310410 learned_addr learned my addr 172.21.15.16:0/2867310410 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:52:39.770 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:38.834+0000 7f304e00e640 1 -- 172.21.15.16:0/2867310410 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3048072be0 con 0x7f3048074030 2024-09-12T01:52:39.771 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:38.835+0000 7f304e00e640 1 --2- 172.21.15.16:0/2867310410 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3048074030 0x7f3048072490 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f304000c310 tx=0x7f304002bbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d1375b37cdc9884 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:52:39.771 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:38.835+0000 7f304d00c640 1 -- 172.21.15.16:0/2867310410 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3040033070 con 0x7f3048074030 2024-09-12T01:52:39.771 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:38.835+0000 7f304d00c640 1 -- 172.21.15.16:0/2867310410 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f304002dc20 con 0x7f3048074030 2024-09-12T01:52:39.771 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:38.835+0000 7f304d00c640 1 -- 172.21.15.16:0/2867310410 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f304000aa90 con 0x7f3048074030 2024-09-12T01:52:39.771 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:38.836+0000 7f3050272640 1 -- 172.21.15.16:0/2867310410 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3048074030 msgr2=0x7f3048072490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:52:39.771 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:38.836+0000 7f3050272640 1 --2- 172.21.15.16:0/2867310410 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3048074030 0x7f3048072490 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f304000c310 tx=0x7f304002bbd0 comp rx=0 tx=0).stop 2024-09-12T01:52:39.771 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:38.836+0000 7f3050272640 1 -- 172.21.15.16:0/2867310410 shutdown_connections 2024-09-12T01:52:39.771 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:38.836+0000 7f3050272640 1 --2- 172.21.15.16:0/2867310410 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3048074030 0x7f3048072490 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:39.771 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:38.836+0000 7f3050272640 1 -- 172.21.15.16:0/2867310410 >> 172.21.15.16:0/2867310410 conn(0x7f304806dbb0 msgr2=0x7f304806fff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:52:39.771 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:38.836+0000 7f3050272640 1 -- 172.21.15.16:0/2867310410 shutdown_connections 2024-09-12T01:52:39.771 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:38.836+0000 7f3050272640 1 -- 172.21.15.16:0/2867310410 wait complete. 2024-09-12T01:52:39.771 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:38.837+0000 7f3050272640 1 Processor -- start 2024-09-12T01:52:39.771 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:38.837+0000 7f3050272640 1 -- start start 2024-09-12T01:52:39.772 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:38.837+0000 7f3050272640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30481a3ff0 0x7f30481a4410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:39.772 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:38.837+0000 7f3050272640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3048112160 con 0x7f30481a3ff0 2024-09-12T01:52:39.772 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:38.837+0000 7f304e00e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30481a3ff0 0x7f30481a4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:39.772 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:38.837+0000 7f304e00e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30481a3ff0 0x7f30481a4410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34348/0 (socket says 172.21.15.16:34348) 2024-09-12T01:52:39.772 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:38.837+0000 7f304e00e640 1 -- 172.21.15.16:0/2440357995 learned_addr learned my addr 172.21.15.16:0/2440357995 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:52:39.772 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:38.837+0000 7f304e00e640 1 -- 172.21.15.16:0/2440357995 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30481a4950 con 0x7f30481a3ff0 2024-09-12T01:52:39.772 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:38.837+0000 7f304e00e640 1 --2- 172.21.15.16:0/2440357995 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30481a3ff0 0x7f30481a4410 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f304000c2e0 tx=0x7f3040007dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:52:39.772 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:38.838+0000 7f303b7fe640 1 -- 172.21.15.16:0/2440357995 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3040033050 con 0x7f30481a3ff0 2024-09-12T01:52:39.772 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:38.838+0000 7f303b7fe640 1 -- 172.21.15.16:0/2440357995 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f304002d970 con 0x7f30481a3ff0 2024-09-12T01:52:39.772 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:38.838+0000 7f3050272640 1 -- 172.21.15.16:0/2440357995 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30481a4be0 con 0x7f30481a3ff0 2024-09-12T01:52:39.772 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:38.838+0000 7f303b7fe640 1 -- 172.21.15.16:0/2440357995 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3040043820 con 0x7f30481a3ff0 2024-09-12T01:52:39.772 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:38.838+0000 7f3050272640 1 -- 172.21.15.16:0/2440357995 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30481a7740 con 0x7f30481a3ff0 2024-09-12T01:52:39.772 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:38.839+0000 7f303b7fe640 1 -- 172.21.15.16:0/2440357995 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7f3040036070 con 0x7f30481a3ff0 2024-09-12T01:52:39.772 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:38.839+0000 7f3050272640 1 -- 172.21.15.16:0/2440357995 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3048072510 con 0x7f30481a3ff0 2024-09-12T01:52:39.773 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:38.839+0000 7f303b7fe640 1 --2- 172.21.15.16:0/2440357995 >> [v2:172.21.15.16:6800/2754331153,v1:172.21.15.16:6801/2754331153] conn(0x7f3034038f70 0x7f303403b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:39.773 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:38.839+0000 7f303b7fe640 1 -- 172.21.15.16:0/2440357995 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f304006d160 con 0x7f30481a3ff0 2024-09-12T01:52:39.773 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:38.839+0000 7f304d80d640 1 --2- 172.21.15.16:0/2440357995 >> [v2:172.21.15.16:6800/2754331153,v1:172.21.15.16:6801/2754331153] conn(0x7f3034038f70 0x7f303403b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:39.773 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:38.839+0000 7f304d80d640 1 --2- 172.21.15.16:0/2440357995 >> [v2:172.21.15.16:6800/2754331153,v1:172.21.15.16:6801/2754331153] conn(0x7f3034038f70 0x7f303403b430 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f30440098c0 tx=0x7f3044006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:52:39.773 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:38.844+0000 7f303b7fe640 1 -- 172.21.15.16:0/2440357995 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3040034260 con 0x7f30481a3ff0 2024-09-12T01:52:39.773 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:39.065+0000 7f3050272640 1 -- 172.21.15.16:0/2440357995 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f30481a7c80 con 0x7f30481a3ff0 2024-09-12T01:52:39.773 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:39.116+0000 7f303b7fe640 1 -- 172.21.15.16:0/2440357995 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7f3040008200 con 0x7f30481a3ff0 2024-09-12T01:52:39.773 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:39.118+0000 7f303b7fe640 1 -- 172.21.15.16:0/2440357995 <== mon.0 v2:172.21.15.16:3300/0 8 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7f3040038cf0 con 0x7f30481a3ff0 2024-09-12T01:52:39.773 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:39.120+0000 7f3050272640 1 -- 172.21.15.16:0/2440357995 >> [v2:172.21.15.16:6800/2754331153,v1:172.21.15.16:6801/2754331153] conn(0x7f3034038f70 msgr2=0x7f303403b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:52:39.773 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:39.120+0000 7f3050272640 1 --2- 172.21.15.16:0/2440357995 >> [v2:172.21.15.16:6800/2754331153,v1:172.21.15.16:6801/2754331153] conn(0x7f3034038f70 0x7f303403b430 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f30440098c0 tx=0x7f3044006db0 comp rx=0 tx=0).stop 2024-09-12T01:52:39.773 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:39.120+0000 7f3050272640 1 -- 172.21.15.16:0/2440357995 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30481a3ff0 msgr2=0x7f30481a4410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:52:39.773 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:39.120+0000 7f3050272640 1 --2- 172.21.15.16:0/2440357995 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30481a3ff0 0x7f30481a4410 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f304000c2e0 tx=0x7f3040007dd0 comp rx=0 tx=0).stop 2024-09-12T01:52:39.773 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:39.120+0000 7f3050272640 1 -- 172.21.15.16:0/2440357995 shutdown_connections 2024-09-12T01:52:39.774 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:39.120+0000 7f3050272640 1 --2- 172.21.15.16:0/2440357995 >> [v2:172.21.15.16:6800/2754331153,v1:172.21.15.16:6801/2754331153] conn(0x7f3034038f70 0x7f303403b430 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:39.774 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:39.120+0000 7f3050272640 1 --2- 172.21.15.16:0/2440357995 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30481a3ff0 0x7f30481a4410 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:39.774 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:39.120+0000 7f3050272640 1 -- 172.21.15.16:0/2440357995 >> 172.21.15.16:0/2440357995 conn(0x7f304806dbb0 msgr2=0x7f304806e7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:52:39.774 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:39.120+0000 7f3050272640 1 -- 172.21.15.16:0/2440357995 shutdown_connections 2024-09-12T01:52:39.774 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:39.120+0000 7f3050272640 1 -- 172.21.15.16:0/2440357995 wait complete. 2024-09-12T01:52:40.121 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 01:52:39 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-x[31173]: 2024-09-12T01:52:39.702+0000 7f090a605040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-12T01:52:40.121 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 01:52:39 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-x[31173]: 2024-09-12T01:52:39.853+0000 7f090a605040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-12T01:52:40.386 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:40 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/2440357995' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-09-12T01:52:40.386 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:40 smithi016 ceph-mon[30947]: mgrmap e5: x(active, since 4s) 2024-09-12T01:52:40.386 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 01:52:40 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-x[31173]: 2024-09-12T01:52:40.185+0000 7f090a605040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-12T01:52:40.386 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 01:52:40 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-x[31173]: 2024-09-12T01:52:40.248+0000 7f090a605040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-12T01:52:40.683 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 01:52:40 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-x[31173]: 2024-09-12T01:52:40.449+0000 7f090a605040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-12T01:52:41.388 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:41 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/503998778' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-12T01:52:41.402 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout { 2024-09-12T01:52:41.402 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-09-12T01:52:41.402 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-12T01:52:41.402 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-12T01:52:41.402 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-12T01:52:41.402 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout } 2024-09-12T01:52:41.403 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:40.578+0000 7fae3f258640 1 Processor -- start 2024-09-12T01:52:41.403 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:40.578+0000 7fae3f258640 1 -- start start 2024-09-12T01:52:41.403 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:40.579+0000 7fae3f258640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae38107780 0x7fae38107b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:41.403 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:40.579+0000 7fae3f258640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fae38108150 con 0x7fae38107780 2024-09-12T01:52:41.403 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:40.579+0000 7fae3cff4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae38107780 0x7fae38107b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:41.403 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:40.579+0000 7fae3cff4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae38107780 0x7fae38107b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34362/0 (socket says 172.21.15.16:34362) 2024-09-12T01:52:41.403 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:40.579+0000 7fae3cff4640 1 -- 172.21.15.16:0/224604531 learned_addr learned my addr 172.21.15.16:0/224604531 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:52:41.404 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:40.579+0000 7fae3cff4640 1 -- 172.21.15.16:0/224604531 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae38108980 con 0x7fae38107780 2024-09-12T01:52:41.404 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:40.580+0000 7fae3cff4640 1 --2- 172.21.15.16:0/224604531 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae38107780 0x7fae38107b80 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fae2c009820 tx=0x7fae2c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a3fcaef3b49944f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:52:41.404 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:40.580+0000 7fae377fe640 1 -- 172.21.15.16:0/224604531 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fae2c037070 con 0x7fae38107780 2024-09-12T01:52:41.404 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:40.580+0000 7fae377fe640 1 -- 172.21.15.16:0/224604531 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fae2c032900 con 0x7fae38107780 2024-09-12T01:52:41.404 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:40.580+0000 7fae377fe640 1 -- 172.21.15.16:0/224604531 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fae2c030850 con 0x7fae38107780 2024-09-12T01:52:41.404 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:40.581+0000 7fae3f258640 1 -- 172.21.15.16:0/224604531 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae38107780 msgr2=0x7fae38107b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:52:41.404 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:40.581+0000 7fae3f258640 1 --2- 172.21.15.16:0/224604531 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae38107780 0x7fae38107b80 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fae2c009820 tx=0x7fae2c02a270 comp rx=0 tx=0).stop 2024-09-12T01:52:41.404 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:40.581+0000 7fae3f258640 1 -- 172.21.15.16:0/224604531 shutdown_connections 2024-09-12T01:52:41.404 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:40.581+0000 7fae3f258640 1 --2- 172.21.15.16:0/224604531 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae38107780 0x7fae38107b80 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:41.404 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:40.581+0000 7fae3f258640 1 -- 172.21.15.16:0/224604531 >> 172.21.15.16:0/224604531 conn(0x7fae38102f30 msgr2=0x7fae38105350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:52:41.404 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:40.581+0000 7fae3f258640 1 -- 172.21.15.16:0/224604531 shutdown_connections 2024-09-12T01:52:41.404 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:40.581+0000 7fae3f258640 1 -- 172.21.15.16:0/224604531 wait complete. 2024-09-12T01:52:41.404 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:40.581+0000 7fae3f258640 1 Processor -- start 2024-09-12T01:52:41.405 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:40.582+0000 7fae3f258640 1 -- start start 2024-09-12T01:52:41.405 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:40.582+0000 7fae3f258640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae38107780 0x7fae3807daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:41.405 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:40.582+0000 7fae3f258640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fae3810bd30 con 0x7fae38107780 2024-09-12T01:52:41.405 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:40.582+0000 7fae3cff4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae38107780 0x7fae3807daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:41.405 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:40.582+0000 7fae3cff4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae38107780 0x7fae3807daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34368/0 (socket says 172.21.15.16:34368) 2024-09-12T01:52:41.405 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:40.582+0000 7fae3cff4640 1 -- 172.21.15.16:0/503998778 learned_addr learned my addr 172.21.15.16:0/503998778 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:52:41.405 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:40.582+0000 7fae3cff4640 1 -- 172.21.15.16:0/503998778 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae3807e030 con 0x7fae38107780 2024-09-12T01:52:41.405 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:40.582+0000 7fae3cff4640 1 --2- 172.21.15.16:0/503998778 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae38107780 0x7fae3807daf0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fae2c0097f0 tx=0x7fae2c030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:52:41.405 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:40.582+0000 7fae35ffb640 1 -- 172.21.15.16:0/503998778 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fae2c037040 con 0x7fae38107780 2024-09-12T01:52:41.405 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:40.582+0000 7fae35ffb640 1 -- 172.21.15.16:0/503998778 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fae2c040070 con 0x7fae38107780 2024-09-12T01:52:41.405 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:40.583+0000 7fae3f258640 1 -- 172.21.15.16:0/503998778 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae3807c220 con 0x7fae38107780 2024-09-12T01:52:41.405 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:40.583+0000 7fae35ffb640 1 -- 172.21.15.16:0/503998778 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fae2c03a390 con 0x7fae38107780 2024-09-12T01:52:41.405 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:40.583+0000 7fae3f258640 1 -- 172.21.15.16:0/503998778 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae3807c700 con 0x7fae38107780 2024-09-12T01:52:41.406 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:40.584+0000 7fae35ffb640 1 -- 172.21.15.16:0/503998778 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7fae2c03a5e0 con 0x7fae38107780 2024-09-12T01:52:41.406 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:40.584+0000 7fae35ffb640 1 --2- 172.21.15.16:0/503998778 >> [v2:172.21.15.16:6800/2754331153,v1:172.21.15.16:6801/2754331153] conn(0x7fae28038fc0 0x7fae2803b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:41.406 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:40.584+0000 7fae3f258640 1 -- 172.21.15.16:0/503998778 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae38107c00 con 0x7fae38107780 2024-09-12T01:52:41.406 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:40.584+0000 7fae37fff640 1 -- 172.21.15.16:0/503998778 >> [v2:172.21.15.16:6800/2754331153,v1:172.21.15.16:6801/2754331153] conn(0x7fae28038fc0 msgr2=0x7fae2803b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/2754331153 2024-09-12T01:52:41.406 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:40.584+0000 7fae35ffb640 1 -- 172.21.15.16:0/503998778 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fae2c06d1a0 con 0x7fae38107780 2024-09-12T01:52:41.406 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:40.584+0000 7fae37fff640 1 --2- 172.21.15.16:0/503998778 >> [v2:172.21.15.16:6800/2754331153,v1:172.21.15.16:6801/2754331153] conn(0x7fae28038fc0 0x7fae2803b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-12T01:52:41.406 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:40.589+0000 7fae35ffb640 1 -- 172.21.15.16:0/503998778 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fae2c06cf00 con 0x7fae38107780 2024-09-12T01:52:41.406 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:40.785+0000 7fae37fff640 1 -- 172.21.15.16:0/503998778 >> [v2:172.21.15.16:6800/2754331153,v1:172.21.15.16:6801/2754331153] conn(0x7fae28038fc0 msgr2=0x7fae2803b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/2754331153 2024-09-12T01:52:41.406 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:40.785+0000 7fae37fff640 1 --2- 172.21.15.16:0/503998778 >> [v2:172.21.15.16:6800/2754331153,v1:172.21.15.16:6801/2754331153] conn(0x7fae28038fc0 0x7fae2803b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-12T01:52:41.406 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:40.812+0000 7fae3f258640 1 -- 172.21.15.16:0/503998778 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fae3807cea0 con 0x7fae38107780 2024-09-12T01:52:41.406 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:40.813+0000 7fae35ffb640 1 -- 172.21.15.16:0/503998778 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7fae2c02fd60 con 0x7fae38107780 2024-09-12T01:52:41.406 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:40.815+0000 7fae3f258640 1 -- 172.21.15.16:0/503998778 >> [v2:172.21.15.16:6800/2754331153,v1:172.21.15.16:6801/2754331153] conn(0x7fae28038fc0 msgr2=0x7fae2803b480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T01:52:41.406 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:40.815+0000 7fae3f258640 1 --2- 172.21.15.16:0/503998778 >> [v2:172.21.15.16:6800/2754331153,v1:172.21.15.16:6801/2754331153] conn(0x7fae28038fc0 0x7fae2803b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:41.406 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:40.815+0000 7fae3f258640 1 -- 172.21.15.16:0/503998778 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae38107780 msgr2=0x7fae3807daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:52:41.407 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:40.815+0000 7fae3f258640 1 --2- 172.21.15.16:0/503998778 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae38107780 0x7fae3807daf0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fae2c0097f0 tx=0x7fae2c030d80 comp rx=0 tx=0).stop 2024-09-12T01:52:41.407 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:40.815+0000 7fae3f258640 1 -- 172.21.15.16:0/503998778 shutdown_connections 2024-09-12T01:52:41.407 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:40.815+0000 7fae3f258640 1 --2- 172.21.15.16:0/503998778 >> [v2:172.21.15.16:6800/2754331153,v1:172.21.15.16:6801/2754331153] conn(0x7fae28038fc0 0x7fae2803b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:41.407 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:40.815+0000 7fae3f258640 1 --2- 172.21.15.16:0/503998778 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae38107780 0x7fae3807daf0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:41.407 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:40.815+0000 7fae3f258640 1 -- 172.21.15.16:0/503998778 >> 172.21.15.16:0/503998778 conn(0x7fae38102f30 msgr2=0x7fae38103af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:52:41.407 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:40.815+0000 7fae3f258640 1 -- 172.21.15.16:0/503998778 shutdown_connections 2024-09-12T01:52:41.407 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:40.815+0000 7fae3f258640 1 -- 172.21.15.16:0/503998778 wait complete. 2024-09-12T01:52:41.407 INFO:teuthology.orchestra.run.smithi016.stdout:Waiting for the mgr to restart... 2024-09-12T01:52:41.407 INFO:teuthology.orchestra.run.smithi016.stdout:Waiting for mgr epoch 5... 2024-09-12T01:52:41.647 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 01:52:41 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-x[31173]: 2024-09-12T01:52:41.388+0000 7f090a605040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-12T01:52:41.647 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 01:52:41 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-x[31173]: 2024-09-12T01:52:41.467+0000 7f090a605040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-12T01:52:42.347 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 01:52:42 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-x[31173]: 2024-09-12T01:52:42.225+0000 7f090a605040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-12T01:52:42.347 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 01:52:42 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-x[31173]: 2024-09-12T01:52:42.346+0000 7f090a605040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-12T01:52:42.683 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 01:52:42 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-x[31173]: 2024-09-12T01:52:42.496+0000 7f090a605040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-12T01:52:42.683 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 01:52:42 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-x[31173]: 2024-09-12T01:52:42.560+0000 7f090a605040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-12T01:52:43.016 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 01:52:42 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-x[31173]: 2024-09-12T01:52:42.716+0000 7f090a605040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-12T01:52:43.016 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 01:52:42 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-x[31173]: 2024-09-12T01:52:42.787+0000 7f090a605040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-12T01:52:43.016 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 01:52:42 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-x[31173]: 2024-09-12T01:52:42.850+0000 7f090a605040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-12T01:52:43.016 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 01:52:43 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-x[31173]: 2024-09-12T01:52:43.015+0000 7f090a605040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-12T01:52:43.302 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 01:52:43 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-x[31173]: 2024-09-12T01:52:43.083+0000 7f090a605040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-12T01:52:43.302 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 01:52:43 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-x[31173]: 2024-09-12T01:52:43.153+0000 7f090a605040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-12T01:52:43.302 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 01:52:43 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-x[31173]: 2024-09-12T01:52:43.239+0000 7f090a605040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-12T01:52:43.302 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 01:52:43 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-x[31173]: 2024-09-12T01:52:43.301+0000 7f090a605040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-12T01:52:43.683 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 01:52:43 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-x[31173]: 2024-09-12T01:52:43.363+0000 7f090a605040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-12T01:52:44.683 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 01:52:44 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-x[31173]: 2024-09-12T01:52:44.192+0000 7f090a605040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-12T01:52:44.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:44 smithi016 ceph-mon[30947]: Active manager daemon x restarted 2024-09-12T01:52:44.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:44 smithi016 ceph-mon[30947]: Activating manager daemon x 2024-09-12T01:52:44.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:44 smithi016 ceph-mon[30947]: osdmap e2: 0 total, 0 up, 0 in 2024-09-12T01:52:44.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:44 smithi016 ceph-mon[30947]: mgrmap e6: x(active, starting, since 0.00380919s) 2024-09-12T01:52:44.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:44 smithi016 ceph-mon[30947]: from='mgr.14118 172.21.15.16:0/2569835181' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T01:52:44.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:44 smithi016 ceph-mon[30947]: from='mgr.14118 172.21.15.16:0/2569835181' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-12T01:52:44.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:44 smithi016 ceph-mon[30947]: from='mgr.14118 172.21.15.16:0/2569835181' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-12T01:52:44.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:44 smithi016 ceph-mon[30947]: from='mgr.14118 172.21.15.16:0/2569835181' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-12T01:52:44.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:44 smithi016 ceph-mon[30947]: from='mgr.14118 172.21.15.16:0/2569835181' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-12T01:52:44.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:44 smithi016 ceph-mon[30947]: Manager daemon x is now available 2024-09-12T01:52:44.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:44 smithi016 ceph-mon[30947]: from='mgr.14118 172.21.15.16:0/2569835181' entity='mgr.x' 2024-09-12T01:52:44.685 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:44 smithi016 ceph-mon[30947]: from='mgr.14118 172.21.15.16:0/2569835181' entity='mgr.x' 2024-09-12T01:52:44.685 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:44 smithi016 ceph-mon[30947]: from='mgr.14118 172.21.15.16:0/2569835181' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:52:44.685 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:44 smithi016 ceph-mon[30947]: from='mgr.14118 172.21.15.16:0/2569835181' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:52:44.685 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:44 smithi016 ceph-mon[30947]: from='mgr.14118 172.21.15.16:0/2569835181' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-12T01:52:44.685 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:44 smithi016 ceph-mon[30947]: from='mgr.14118 172.21.15.16:0/2569835181' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-12T01:52:45.499 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:45 smithi016 ceph-mon[30947]: Found migration_current of "None". Setting to last migration. 2024-09-12T01:52:45.499 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:45 smithi016 ceph-mon[30947]: mgrmap e7: x(active, since 1.00572s) 2024-09-12T01:52:45.499 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:45 smithi016 ceph-mon[30947]: from='mgr.14118 172.21.15.16:0/2569835181' entity='mgr.x' 2024-09-12T01:52:45.499 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:45 smithi016 ceph-mon[30947]: from='mgr.14118 172.21.15.16:0/2569835181' entity='mgr.x' 2024-09-12T01:52:45.776 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout { 2024-09-12T01:52:45.776 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-09-12T01:52:45.777 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-12T01:52:45.777 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout } 2024-09-12T01:52:45.777 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:41.827+0000 7fdbde60b640 1 Processor -- start 2024-09-12T01:52:45.777 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:41.827+0000 7fdbde60b640 1 -- start start 2024-09-12T01:52:45.777 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:41.827+0000 7fdbde60b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdbd81065e0 0x7fdbd81069e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:45.777 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:41.827+0000 7fdbde60b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fdbd8106fb0 con 0x7fdbd81065e0 2024-09-12T01:52:45.777 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:41.828+0000 7fdbd7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdbd81065e0 0x7fdbd81069e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:45.777 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:41.828+0000 7fdbd7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdbd81065e0 0x7fdbd81069e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52360/0 (socket says 172.21.15.16:52360) 2024-09-12T01:52:45.777 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:41.828+0000 7fdbd7fff640 1 -- 172.21.15.16:0/2680728618 learned_addr learned my addr 172.21.15.16:0/2680728618 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:52:45.778 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:41.828+0000 7fdbd7fff640 1 -- 172.21.15.16:0/2680728618 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbd81077e0 con 0x7fdbd81065e0 2024-09-12T01:52:45.778 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:41.829+0000 7fdbd7fff640 1 --2- 172.21.15.16:0/2680728618 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdbd81065e0 0x7fdbd81069e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fdbc8009820 tx=0x7fdbc802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52b733d2287acc25 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:52:45.778 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:41.829+0000 7fdbd77fe640 1 -- 172.21.15.16:0/2680728618 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdbc8037070 con 0x7fdbd81065e0 2024-09-12T01:52:45.778 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:41.829+0000 7fdbd77fe640 1 -- 172.21.15.16:0/2680728618 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fdbc8032900 con 0x7fdbd81065e0 2024-09-12T01:52:45.778 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:41.829+0000 7fdbd77fe640 1 -- 172.21.15.16:0/2680728618 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdbc8030850 con 0x7fdbd81065e0 2024-09-12T01:52:45.778 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:41.830+0000 7fdbde60b640 1 -- 172.21.15.16:0/2680728618 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdbd81065e0 msgr2=0x7fdbd81069e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:52:45.778 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:41.830+0000 7fdbde60b640 1 --2- 172.21.15.16:0/2680728618 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdbd81065e0 0x7fdbd81069e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fdbc8009820 tx=0x7fdbc802a270 comp rx=0 tx=0).stop 2024-09-12T01:52:45.778 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:41.830+0000 7fdbde60b640 1 -- 172.21.15.16:0/2680728618 shutdown_connections 2024-09-12T01:52:45.778 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:41.830+0000 7fdbde60b640 1 --2- 172.21.15.16:0/2680728618 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdbd81065e0 0x7fdbd81069e0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:45.779 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:41.830+0000 7fdbde60b640 1 -- 172.21.15.16:0/2680728618 >> 172.21.15.16:0/2680728618 conn(0x7fdbd8101d90 msgr2=0x7fdbd81041b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:52:45.779 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:41.830+0000 7fdbde60b640 1 -- 172.21.15.16:0/2680728618 shutdown_connections 2024-09-12T01:52:45.779 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:41.830+0000 7fdbde60b640 1 -- 172.21.15.16:0/2680728618 wait complete. 2024-09-12T01:52:45.779 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:41.830+0000 7fdbde60b640 1 Processor -- start 2024-09-12T01:52:45.779 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:41.830+0000 7fdbde60b640 1 -- start start 2024-09-12T01:52:45.779 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:41.831+0000 7fdbde60b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdbd81065e0 0x7fdbd8191810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:45.779 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:41.831+0000 7fdbde60b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fdbd810ab90 con 0x7fdbd81065e0 2024-09-12T01:52:45.779 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:41.831+0000 7fdbd7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdbd81065e0 0x7fdbd8191810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:45.779 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:41.831+0000 7fdbd7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdbd81065e0 0x7fdbd8191810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52372/0 (socket says 172.21.15.16:52372) 2024-09-12T01:52:45.779 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:41.831+0000 7fdbd7fff640 1 -- 172.21.15.16:0/1835068125 learned_addr learned my addr 172.21.15.16:0/1835068125 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:52:45.779 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:41.831+0000 7fdbd7fff640 1 -- 172.21.15.16:0/1835068125 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbd8191d50 con 0x7fdbd81065e0 2024-09-12T01:52:45.779 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:41.831+0000 7fdbd7fff640 1 --2- 172.21.15.16:0/1835068125 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdbd81065e0 0x7fdbd8191810 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fdbc80097f0 tx=0x7fdbc8030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:52:45.779 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:41.831+0000 7fdbd5ffb640 1 -- 172.21.15.16:0/1835068125 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdbc8037040 con 0x7fdbd81065e0 2024-09-12T01:52:45.780 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:41.831+0000 7fdbd5ffb640 1 -- 172.21.15.16:0/1835068125 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fdbc8040070 con 0x7fdbd81065e0 2024-09-12T01:52:45.780 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:41.831+0000 7fdbd5ffb640 1 -- 172.21.15.16:0/1835068125 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdbc803a390 con 0x7fdbd81065e0 2024-09-12T01:52:45.780 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:41.831+0000 7fdbde60b640 1 -- 172.21.15.16:0/1835068125 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbd818ff40 con 0x7fdbd81065e0 2024-09-12T01:52:45.780 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:41.831+0000 7fdbde60b640 1 -- 172.21.15.16:0/1835068125 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbd8190420 con 0x7fdbd81065e0 2024-09-12T01:52:45.780 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:41.832+0000 7fdbd5ffb640 1 -- 172.21.15.16:0/1835068125 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7fdbc803b430 con 0x7fdbd81065e0 2024-09-12T01:52:45.780 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:41.833+0000 7fdbd5ffb640 1 --2- 172.21.15.16:0/1835068125 >> [v2:172.21.15.16:6800/2754331153,v1:172.21.15.16:6801/2754331153] conn(0x7fdbc4038f70 0x7fdbc403b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:45.780 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:41.833+0000 7fdbd5ffb640 1 -- 172.21.15.16:0/1835068125 --> [v2:172.21.15.16:6800/2754331153,v1:172.21.15.16:6801/2754331153] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fdbc403bb40 con 0x7fdbc4038f70 2024-09-12T01:52:45.780 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:41.833+0000 7fdbd5ffb640 1 -- 172.21.15.16:0/1835068125 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fdbc806c160 con 0x7fdbd81065e0 2024-09-12T01:52:45.780 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:41.833+0000 7fdbcffff640 1 -- 172.21.15.16:0/1835068125 >> [v2:172.21.15.16:6800/2754331153,v1:172.21.15.16:6801/2754331153] conn(0x7fdbc4038f70 msgr2=0x7fdbc403b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/2754331153 2024-09-12T01:52:45.780 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:41.833+0000 7fdbcffff640 1 --2- 172.21.15.16:0/1835068125 >> [v2:172.21.15.16:6800/2754331153,v1:172.21.15.16:6801/2754331153] conn(0x7fdbc4038f70 0x7fdbc403b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-12T01:52:45.780 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:42.034+0000 7fdbcffff640 1 -- 172.21.15.16:0/1835068125 >> [v2:172.21.15.16:6800/2754331153,v1:172.21.15.16:6801/2754331153] conn(0x7fdbc4038f70 msgr2=0x7fdbc403b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/2754331153 2024-09-12T01:52:45.780 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:42.034+0000 7fdbcffff640 1 --2- 172.21.15.16:0/1835068125 >> [v2:172.21.15.16:6800/2754331153,v1:172.21.15.16:6801/2754331153] conn(0x7fdbc4038f70 0x7fdbc403b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-12T01:52:45.781 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:42.435+0000 7fdbcffff640 1 -- 172.21.15.16:0/1835068125 >> [v2:172.21.15.16:6800/2754331153,v1:172.21.15.16:6801/2754331153] conn(0x7fdbc4038f70 msgr2=0x7fdbc403b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/2754331153 2024-09-12T01:52:45.781 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:42.435+0000 7fdbcffff640 1 --2- 172.21.15.16:0/1835068125 >> [v2:172.21.15.16:6800/2754331153,v1:172.21.15.16:6801/2754331153] conn(0x7fdbc4038f70 0x7fdbc403b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-12T01:52:45.781 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:43.236+0000 7fdbcffff640 1 -- 172.21.15.16:0/1835068125 >> [v2:172.21.15.16:6800/2754331153,v1:172.21.15.16:6801/2754331153] conn(0x7fdbc4038f70 msgr2=0x7fdbc403b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/2754331153 2024-09-12T01:52:45.781 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:43.236+0000 7fdbcffff640 1 --2- 172.21.15.16:0/1835068125 >> [v2:172.21.15.16:6800/2754331153,v1:172.21.15.16:6801/2754331153] conn(0x7fdbc4038f70 0x7fdbc403b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-12T01:52:45.781 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:44.195+0000 7fdbd5ffb640 1 -- 172.21.15.16:0/1835068125 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mgrmap(e 6) ==== 46054+0+0 (secure 0 0 0) 0x7fdbc806af60 con 0x7fdbd81065e0 2024-09-12T01:52:45.781 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:44.196+0000 7fdbd5ffb640 1 -- 172.21.15.16:0/1835068125 >> [v2:172.21.15.16:6800/2754331153,v1:172.21.15.16:6801/2754331153] conn(0x7fdbc4038f70 msgr2=0x7fdbc403b430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T01:52:45.781 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:44.196+0000 7fdbd5ffb640 1 --2- 172.21.15.16:0/1835068125 >> [v2:172.21.15.16:6800/2754331153,v1:172.21.15.16:6801/2754331153] conn(0x7fdbc4038f70 0x7fdbc403b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:45.781 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:45.196+0000 7fdbd5ffb640 1 -- 172.21.15.16:0/1835068125 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mgrmap(e 7) ==== 46181+0+0 (secure 0 0 0) 0x7fdbc8032900 con 0x7fdbd81065e0 2024-09-12T01:52:45.781 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:45.197+0000 7fdbd5ffb640 1 --2- 172.21.15.16:0/1835068125 >> [v2:172.21.15.16:6800/2615861334,v1:172.21.15.16:6801/2615861334] conn(0x7fdbc403c8a0 0x7fdbc403ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:45.781 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:45.197+0000 7fdbd5ffb640 1 -- 172.21.15.16:0/1835068125 --> [v2:172.21.15.16:6800/2615861334,v1:172.21.15.16:6801/2615861334] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fdbc803de70 con 0x7fdbc403c8a0 2024-09-12T01:52:45.781 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:45.197+0000 7fdbcffff640 1 --2- 172.21.15.16:0/1835068125 >> [v2:172.21.15.16:6800/2615861334,v1:172.21.15.16:6801/2615861334] conn(0x7fdbc403c8a0 0x7fdbc403ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:45.781 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:45.198+0000 7fdbcffff640 1 --2- 172.21.15.16:0/1835068125 >> [v2:172.21.15.16:6800/2615861334,v1:172.21.15.16:6801/2615861334] conn(0x7fdbc403c8a0 0x7fdbc403ec90 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fdbc0003980 tx=0x7fdbc0006e10 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:52:45.781 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:45.199+0000 7fdbd5ffb640 1 -- 172.21.15.16:0/1835068125 <== mgr.14118 v2:172.21.15.16:6800/2615861334 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7fdbc803de70 con 0x7fdbc403c8a0 2024-09-12T01:52:45.781 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:45.206+0000 7fdbde60b640 1 -- 172.21.15.16:0/1835068125 --> [v2:172.21.15.16:6800/2615861334,v1:172.21.15.16:6801/2615861334] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7fdbd810ad10 con 0x7fdbc403c8a0 2024-09-12T01:52:45.782 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:45.206+0000 7fdbd5ffb640 1 -- 172.21.15.16:0/1835068125 <== mgr.14118 v2:172.21.15.16:6800/2615861334 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7fdbd810ad10 con 0x7fdbc403c8a0 2024-09-12T01:52:45.782 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:45.206+0000 7fdbde60b640 1 -- 172.21.15.16:0/1835068125 >> [v2:172.21.15.16:6800/2615861334,v1:172.21.15.16:6801/2615861334] conn(0x7fdbc403c8a0 msgr2=0x7fdbc403ec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:52:45.782 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:45.206+0000 7fdbde60b640 1 --2- 172.21.15.16:0/1835068125 >> [v2:172.21.15.16:6800/2615861334,v1:172.21.15.16:6801/2615861334] conn(0x7fdbc403c8a0 0x7fdbc403ec90 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fdbc0003980 tx=0x7fdbc0006e10 comp rx=0 tx=0).stop 2024-09-12T01:52:45.782 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:45.206+0000 7fdbde60b640 1 -- 172.21.15.16:0/1835068125 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdbd81065e0 msgr2=0x7fdbd8191810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:52:45.782 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:45.206+0000 7fdbde60b640 1 --2- 172.21.15.16:0/1835068125 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdbd81065e0 0x7fdbd8191810 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fdbc80097f0 tx=0x7fdbc8030d80 comp rx=0 tx=0).stop 2024-09-12T01:52:45.782 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:45.206+0000 7fdbde60b640 1 -- 172.21.15.16:0/1835068125 shutdown_connections 2024-09-12T01:52:45.782 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:45.206+0000 7fdbde60b640 1 --2- 172.21.15.16:0/1835068125 >> [v2:172.21.15.16:6800/2615861334,v1:172.21.15.16:6801/2615861334] conn(0x7fdbc403c8a0 0x7fdbc403ec90 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:45.782 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:45.206+0000 7fdbde60b640 1 --2- 172.21.15.16:0/1835068125 >> [v2:172.21.15.16:6800/2754331153,v1:172.21.15.16:6801/2754331153] conn(0x7fdbc4038f70 0x7fdbc403b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:45.782 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:45.206+0000 7fdbde60b640 1 --2- 172.21.15.16:0/1835068125 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdbd81065e0 0x7fdbd8191810 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:45.782 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:45.206+0000 7fdbde60b640 1 -- 172.21.15.16:0/1835068125 >> 172.21.15.16:0/1835068125 conn(0x7fdbd8101d90 msgr2=0x7fdbd8102cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:52:45.783 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:45.206+0000 7fdbde60b640 1 -- 172.21.15.16:0/1835068125 shutdown_connections 2024-09-12T01:52:45.783 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:45.206+0000 7fdbde60b640 1 -- 172.21.15.16:0/1835068125 wait complete. 2024-09-12T01:52:45.783 INFO:teuthology.orchestra.run.smithi016.stdout:mgr epoch 5 is available 2024-09-12T01:52:45.783 INFO:teuthology.orchestra.run.smithi016.stdout:Setting orchestrator backend to cephadm... 2024-09-12T01:52:46.373 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:46 smithi016 ceph-mon[30947]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-12T01:52:46.373 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:46 smithi016 ceph-mon[30947]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-12T01:52:46.934 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 01:52:46 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-x[31173]: 2024-09-12T01:52:46.817+0000 7f087191f640 -1 log_channel(cephadm) log [ERR] : [12/Sep/2024:01:52:46] ENGINE Error in HTTPServer.serve 2024-09-12T01:52:46.934 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 01:52:46 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-x[31173]: Traceback (most recent call last): 2024-09-12T01:52:46.934 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 01:52:46 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-x[31173]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-12T01:52:46.934 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 01:52:46 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-x[31173]: self._connections.run(self.expiration_interval) 2024-09-12T01:52:46.934 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 01:52:46 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-x[31173]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-12T01:52:46.935 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 01:52:46 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-x[31173]: self._run(expiration_interval) 2024-09-12T01:52:46.935 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 01:52:46 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-x[31173]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-12T01:52:46.935 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 01:52:46 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-x[31173]: new_conn = self._from_server_socket(self.server.socket) 2024-09-12T01:52:46.935 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 01:52:46 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-x[31173]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-12T01:52:46.935 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 01:52:46 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-x[31173]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-12T01:52:46.935 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 01:52:46 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-x[31173]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-12T01:52:46.935 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 01:52:46 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-x[31173]: s = self.context.wrap_socket( 2024-09-12T01:52:46.935 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 01:52:46 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-x[31173]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-12T01:52:46.935 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 01:52:46 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-x[31173]: return self.sslsocket_class._create( 2024-09-12T01:52:46.935 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 01:52:46 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-x[31173]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-12T01:52:46.935 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 01:52:46 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-x[31173]: self.do_handshake() 2024-09-12T01:52:46.935 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 01:52:46 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-x[31173]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-12T01:52:46.935 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 01:52:46 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-x[31173]: self._sslobj.do_handshake() 2024-09-12T01:52:46.936 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 01:52:46 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-x[31173]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-12T01:52:46.936 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 01:52:46 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-x[31173]: 2024-09-12T01:52:46.997 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:46.166+0000 7f61e6b76640 1 Processor -- start 2024-09-12T01:52:46.997 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:46.166+0000 7f61e6b76640 1 -- start start 2024-09-12T01:52:46.997 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:46.166+0000 7f61e6b76640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61e0108180 0x7f61e010a570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:46.997 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:46.167+0000 7f61e6b76640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f61e007bdf0 con 0x7f61e0108180 2024-09-12T01:52:46.998 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:46.167+0000 7f61e4912640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61e0108180 0x7f61e010a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:46.998 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:46.167+0000 7f61e4912640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61e0108180 0x7f61e010a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52434/0 (socket says 172.21.15.16:52434) 2024-09-12T01:52:46.998 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:46.167+0000 7f61e4912640 1 -- 172.21.15.16:0/21155323 learned_addr learned my addr 172.21.15.16:0/21155323 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:52:46.998 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:46.167+0000 7f61e4912640 1 -- 172.21.15.16:0/21155323 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61e010aab0 con 0x7f61e0108180 2024-09-12T01:52:46.998 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:46.168+0000 7f61e4912640 1 --2- 172.21.15.16:0/21155323 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61e0108180 0x7f61e010a570 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f61d4009820 tx=0x7f61d402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef1a9031b57aca34 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:52:46.998 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:46.168+0000 7f61df7fe640 1 -- 172.21.15.16:0/21155323 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f61d4037070 con 0x7f61e0108180 2024-09-12T01:52:46.998 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:46.168+0000 7f61df7fe640 1 -- 172.21.15.16:0/21155323 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f61d4032900 con 0x7f61e0108180 2024-09-12T01:52:46.998 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:46.168+0000 7f61df7fe640 1 -- 172.21.15.16:0/21155323 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f61d4030850 con 0x7f61e0108180 2024-09-12T01:52:46.998 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:46.168+0000 7f61e6b76640 1 -- 172.21.15.16:0/21155323 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61e0108180 msgr2=0x7f61e010a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:52:46.998 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:46.168+0000 7f61e6b76640 1 --2- 172.21.15.16:0/21155323 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61e0108180 0x7f61e010a570 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f61d4009820 tx=0x7f61d402a270 comp rx=0 tx=0).stop 2024-09-12T01:52:46.998 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:46.169+0000 7f61e6b76640 1 -- 172.21.15.16:0/21155323 shutdown_connections 2024-09-12T01:52:46.998 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:46.169+0000 7f61e6b76640 1 --2- 172.21.15.16:0/21155323 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61e0108180 0x7f61e010a570 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:46.999 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:46.169+0000 7f61e6b76640 1 -- 172.21.15.16:0/21155323 >> 172.21.15.16:0/21155323 conn(0x7f61e0101e20 msgr2=0x7f61e0104240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:52:46.999 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:46.169+0000 7f61e6b76640 1 -- 172.21.15.16:0/21155323 shutdown_connections 2024-09-12T01:52:46.999 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:46.169+0000 7f61e6b76640 1 -- 172.21.15.16:0/21155323 wait complete. 2024-09-12T01:52:46.999 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:46.169+0000 7f61e6b76640 1 Processor -- start 2024-09-12T01:52:46.999 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:46.169+0000 7f61e6b76640 1 -- start start 2024-09-12T01:52:46.999 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:46.169+0000 7f61e6b76640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61e0108180 0x7f61e019b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:46.999 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:46.169+0000 7f61e6b76640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f61e010baf0 con 0x7f61e0108180 2024-09-12T01:52:46.999 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:46.169+0000 7f61e4912640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61e0108180 0x7f61e019b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:46.999 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:46.169+0000 7f61e4912640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61e0108180 0x7f61e019b6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52438/0 (socket says 172.21.15.16:52438) 2024-09-12T01:52:46.999 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:46.169+0000 7f61e4912640 1 -- 172.21.15.16:0/205088509 learned_addr learned my addr 172.21.15.16:0/205088509 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:52:46.999 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:46.170+0000 7f61e4912640 1 -- 172.21.15.16:0/205088509 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61e019bc10 con 0x7f61e0108180 2024-09-12T01:52:46.999 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:46.170+0000 7f61e4912640 1 --2- 172.21.15.16:0/205088509 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61e0108180 0x7f61e019b6d0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f61d4032ce0 tx=0x7f61d4032d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:52:47.000 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:46.170+0000 7f61ddffb640 1 -- 172.21.15.16:0/205088509 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f61d4042070 con 0x7f61e0108180 2024-09-12T01:52:47.000 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:46.170+0000 7f61ddffb640 1 -- 172.21.15.16:0/205088509 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f61d40308c0 con 0x7f61e0108180 2024-09-12T01:52:47.000 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:46.170+0000 7f61e6b76640 1 -- 172.21.15.16:0/205088509 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61e019bea0 con 0x7f61e0108180 2024-09-12T01:52:47.000 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:46.170+0000 7f61ddffb640 1 -- 172.21.15.16:0/205088509 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f61d4037040 con 0x7f61e0108180 2024-09-12T01:52:47.000 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:46.170+0000 7f61e6b76640 1 -- 172.21.15.16:0/205088509 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61e019eb90 con 0x7f61e0108180 2024-09-12T01:52:47.000 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:46.171+0000 7f61ddffb640 1 -- 172.21.15.16:0/205088509 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 7) ==== 46181+0+0 (secure 0 0 0) 0x7f61d4045070 con 0x7f61e0108180 2024-09-12T01:52:47.000 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:46.171+0000 7f61e6b76640 1 -- 172.21.15.16:0/205088509 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61e01085e0 con 0x7f61e0108180 2024-09-12T01:52:47.000 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:46.172+0000 7f61ddffb640 1 --2- 172.21.15.16:0/205088509 >> [v2:172.21.15.16:6800/2615861334,v1:172.21.15.16:6801/2615861334] conn(0x7f61d0038e60 0x7f61d003b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:47.000 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:46.172+0000 7f61ddffb640 1 -- 172.21.15.16:0/205088509 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f61d406be90 con 0x7f61e0108180 2024-09-12T01:52:47.001 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:46.172+0000 7f61dffff640 1 --2- 172.21.15.16:0/205088509 >> [v2:172.21.15.16:6800/2615861334,v1:172.21.15.16:6801/2615861334] conn(0x7f61d0038e60 0x7f61d003b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:47.001 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:46.172+0000 7f61dffff640 1 --2- 172.21.15.16:0/205088509 >> [v2:172.21.15.16:6800/2615861334,v1:172.21.15.16:6801/2615861334] conn(0x7f61d0038e60 0x7f61d003b320 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f61cc0098c0 tx=0x7f61cc006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:52:47.001 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:46.176+0000 7f61ddffb640 1 -- 172.21.15.16:0/205088509 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f61d404b170 con 0x7f61e0108180 2024-09-12T01:52:47.001 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:46.251+0000 7f61ddffb640 1 -- 172.21.15.16:0/205088509 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f61d403c830 con 0x7f61e0108180 2024-09-12T01:52:47.001 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:46.372+0000 7f61e6b76640 1 -- 172.21.15.16:0/205088509 --> [v2:172.21.15.16:6800/2615861334,v1:172.21.15.16:6801/2615861334] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f61e0103120 con 0x7f61d0038e60 2024-09-12T01:52:47.001 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:46.377+0000 7f61ddffb640 1 -- 172.21.15.16:0/205088509 <== mgr.14118 v2:172.21.15.16:6800/2615861334 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f61e0103120 con 0x7f61d0038e60 2024-09-12T01:52:47.001 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:46.380+0000 7f61e6b76640 1 -- 172.21.15.16:0/205088509 >> [v2:172.21.15.16:6800/2615861334,v1:172.21.15.16:6801/2615861334] conn(0x7f61d0038e60 msgr2=0x7f61d003b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:52:47.001 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:46.380+0000 7f61e6b76640 1 --2- 172.21.15.16:0/205088509 >> [v2:172.21.15.16:6800/2615861334,v1:172.21.15.16:6801/2615861334] conn(0x7f61d0038e60 0x7f61d003b320 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f61cc0098c0 tx=0x7f61cc006db0 comp rx=0 tx=0).stop 2024-09-12T01:52:47.001 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:46.380+0000 7f61e6b76640 1 -- 172.21.15.16:0/205088509 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61e0108180 msgr2=0x7f61e019b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:52:47.001 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:46.380+0000 7f61e6b76640 1 --2- 172.21.15.16:0/205088509 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61e0108180 0x7f61e019b6d0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f61d4032ce0 tx=0x7f61d4032d10 comp rx=0 tx=0).stop 2024-09-12T01:52:47.001 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:46.380+0000 7f61e6b76640 1 -- 172.21.15.16:0/205088509 shutdown_connections 2024-09-12T01:52:47.001 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:46.380+0000 7f61e6b76640 1 --2- 172.21.15.16:0/205088509 >> [v2:172.21.15.16:6800/2615861334,v1:172.21.15.16:6801/2615861334] conn(0x7f61d0038e60 0x7f61d003b320 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:47.001 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:46.380+0000 7f61e6b76640 1 --2- 172.21.15.16:0/205088509 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61e0108180 0x7f61e019b6d0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:47.002 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:46.380+0000 7f61e6b76640 1 -- 172.21.15.16:0/205088509 >> 172.21.15.16:0/205088509 conn(0x7f61e0101e20 msgr2=0x7f61e0102a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:52:47.002 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:46.380+0000 7f61e6b76640 1 -- 172.21.15.16:0/205088509 shutdown_connections 2024-09-12T01:52:47.002 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:46.380+0000 7f61e6b76640 1 -- 172.21.15.16:0/205088509 wait complete. 2024-09-12T01:52:47.567 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:47 smithi016 ceph-mon[30947]: mgrmap e8: x(active, since 2s) 2024-09-12T01:52:47.567 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:47 smithi016 ceph-mon[30947]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:52:47.568 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:47 smithi016 ceph-mon[30947]: from='mgr.14118 172.21.15.16:0/2569835181' entity='mgr.x' 2024-09-12T01:52:47.568 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:47 smithi016 ceph-mon[30947]: from='mgr.14118 172.21.15.16:0/2569835181' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:52:47.568 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:47 smithi016 ceph-mon[30947]: [12/Sep/2024:01:52:46] ENGINE Bus STARTING 2024-09-12T01:52:47.568 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:47 smithi016 ceph-mon[30947]: [12/Sep/2024:01:52:46] ENGINE Serving on http://172.21.15.16:8765 2024-09-12T01:52:47.568 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:47 smithi016 ceph-mon[30947]: [12/Sep/2024:01:52:46] ENGINE Serving on https://172.21.15.16:7150 2024-09-12T01:52:47.568 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:47 smithi016 ceph-mon[30947]: [12/Sep/2024:01:52:46] ENGINE Bus STARTED 2024-09-12T01:52:47.568 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:47 smithi016 ceph-mon[30947]: from='mgr.14118 172.21.15.16:0/2569835181' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:52:47.568 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:47 smithi016 ceph-mon[30947]: [12/Sep/2024:01:52:46] ENGINE Error in HTTPServer.serve 2024-09-12T01:52:47.568 INFO:journalctl@ceph.mon.a.smithi016.stdout: Traceback (most recent call last): 2024-09-12T01:52:47.568 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-12T01:52:47.568 INFO:journalctl@ceph.mon.a.smithi016.stdout: self._connections.run(self.expiration_interval) 2024-09-12T01:52:47.568 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-12T01:52:47.568 INFO:journalctl@ceph.mon.a.smithi016.stdout: self._run(expiration_interval) 2024-09-12T01:52:47.569 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-12T01:52:47.569 INFO:journalctl@ceph.mon.a.smithi016.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-12T01:52:47.569 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-12T01:52:47.569 INFO:journalctl@ceph.mon.a.smithi016.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-12T01:52:47.569 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-12T01:52:47.569 INFO:journalctl@ceph.mon.a.smithi016.stdout: s = self.context.wrap_socket( 2024-09-12T01:52:47.569 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-12T01:52:47.569 INFO:journalctl@ceph.mon.a.smithi016.stdout: return self.sslsocket_class._create( 2024-09-12T01:52:47.570 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-12T01:52:47.570 INFO:journalctl@ceph.mon.a.smithi016.stdout: self.do_handshake() 2024-09-12T01:52:47.570 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-12T01:52:47.570 INFO:journalctl@ceph.mon.a.smithi016.stdout: self._sslobj.do_handshake() 2024-09-12T01:52:47.570 INFO:journalctl@ceph.mon.a.smithi016.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-12T01:52:48.121 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout value unchanged 2024-09-12T01:52:48.121 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:47.379+0000 7f7778fde640 1 Processor -- start 2024-09-12T01:52:48.121 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:47.379+0000 7f7778fde640 1 -- start start 2024-09-12T01:52:48.121 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:47.379+0000 7f7778fde640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77741077e0 0x7f7774107be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:48.121 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:47.379+0000 7f7778fde640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f77741081b0 con 0x7f77741077e0 2024-09-12T01:52:48.121 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:47.379+0000 7f777259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77741077e0 0x7f7774107be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:48.121 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:47.379+0000 7f777259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77741077e0 0x7f7774107be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52452/0 (socket says 172.21.15.16:52452) 2024-09-12T01:52:48.121 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:47.379+0000 7f777259c640 1 -- 172.21.15.16:0/2269580565 learned_addr learned my addr 172.21.15.16:0/2269580565 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:52:48.122 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:47.380+0000 7f777259c640 1 -- 172.21.15.16:0/2269580565 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7774108940 con 0x7f77741077e0 2024-09-12T01:52:48.122 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:47.380+0000 7f777259c640 1 --2- 172.21.15.16:0/2269580565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77741077e0 0x7f7774107be0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7764009820 tx=0x7f776402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61ce6adee65c6c62 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:52:48.122 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:47.381+0000 7f7771d9b640 1 -- 172.21.15.16:0/2269580565 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7764037070 con 0x7f77741077e0 2024-09-12T01:52:48.122 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:47.381+0000 7f7771d9b640 1 -- 172.21.15.16:0/2269580565 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7764032900 con 0x7f77741077e0 2024-09-12T01:52:48.122 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:47.381+0000 7f7771d9b640 1 -- 172.21.15.16:0/2269580565 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7764030850 con 0x7f77741077e0 2024-09-12T01:52:48.122 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:47.381+0000 7f7778fde640 1 -- 172.21.15.16:0/2269580565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77741077e0 msgr2=0x7f7774107be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:52:48.122 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:47.381+0000 7f7778fde640 1 --2- 172.21.15.16:0/2269580565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77741077e0 0x7f7774107be0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7764009820 tx=0x7f776402a270 comp rx=0 tx=0).stop 2024-09-12T01:52:48.122 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:47.381+0000 7f7778fde640 1 -- 172.21.15.16:0/2269580565 shutdown_connections 2024-09-12T01:52:48.122 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:47.381+0000 7f7778fde640 1 --2- 172.21.15.16:0/2269580565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77741077e0 0x7f7774107be0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:48.122 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:47.381+0000 7f7778fde640 1 -- 172.21.15.16:0/2269580565 >> 172.21.15.16:0/2269580565 conn(0x7f7774102fb0 msgr2=0x7f77741053d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:52:48.122 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:47.381+0000 7f7778fde640 1 -- 172.21.15.16:0/2269580565 shutdown_connections 2024-09-12T01:52:48.122 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:47.381+0000 7f7778fde640 1 -- 172.21.15.16:0/2269580565 wait complete. 2024-09-12T01:52:48.122 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:47.382+0000 7f7778fde640 1 Processor -- start 2024-09-12T01:52:48.123 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:47.382+0000 7f7778fde640 1 -- start start 2024-09-12T01:52:48.123 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:47.382+0000 7f7778fde640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77741077e0 0x7f777419b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:48.123 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:47.382+0000 7f7778fde640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f777410bcf0 con 0x7f77741077e0 2024-09-12T01:52:48.123 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:47.382+0000 7f777259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77741077e0 0x7f777419b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:48.123 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:47.382+0000 7f777259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77741077e0 0x7f777419b830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52456/0 (socket says 172.21.15.16:52456) 2024-09-12T01:52:48.123 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:47.382+0000 7f777259c640 1 -- 172.21.15.16:0/3335471417 learned_addr learned my addr 172.21.15.16:0/3335471417 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:52:48.123 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:47.383+0000 7f777259c640 1 -- 172.21.15.16:0/3335471417 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f777419bd70 con 0x7f77741077e0 2024-09-12T01:52:48.123 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:47.383+0000 7f777259c640 1 --2- 172.21.15.16:0/3335471417 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77741077e0 0x7f777419b830 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f7764009940 tx=0x7f77640326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:52:48.123 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:47.383+0000 7f776b7fe640 1 -- 172.21.15.16:0/3335471417 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7764042070 con 0x7f77741077e0 2024-09-12T01:52:48.123 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:47.383+0000 7f776b7fe640 1 -- 172.21.15.16:0/3335471417 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7764032be0 con 0x7f77741077e0 2024-09-12T01:52:48.123 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:47.383+0000 7f7778fde640 1 -- 172.21.15.16:0/3335471417 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f777419c000 con 0x7f77741077e0 2024-09-12T01:52:48.123 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:47.383+0000 7f776b7fe640 1 -- 172.21.15.16:0/3335471417 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7764037040 con 0x7f77741077e0 2024-09-12T01:52:48.123 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:47.383+0000 7f7778fde640 1 -- 172.21.15.16:0/3335471417 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f777419ecf0 con 0x7f77741077e0 2024-09-12T01:52:48.124 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:47.384+0000 7f776b7fe640 1 -- 172.21.15.16:0/3335471417 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f7764045070 con 0x7f77741077e0 2024-09-12T01:52:48.124 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:47.384+0000 7f7778fde640 1 -- 172.21.15.16:0/3335471417 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f777410be70 con 0x7f77741077e0 2024-09-12T01:52:48.124 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:47.384+0000 7f776b7fe640 1 --2- 172.21.15.16:0/3335471417 >> [v2:172.21.15.16:6800/2615861334,v1:172.21.15.16:6801/2615861334] conn(0x7f7760038f70 0x7f776003b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:48.124 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:47.384+0000 7f776b7fe640 1 -- 172.21.15.16:0/3335471417 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f776406bf20 con 0x7f77741077e0 2024-09-12T01:52:48.124 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:47.384+0000 7f776bfff640 1 --2- 172.21.15.16:0/3335471417 >> [v2:172.21.15.16:6800/2615861334,v1:172.21.15.16:6801/2615861334] conn(0x7f7760038f70 0x7f776003b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:48.124 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:47.385+0000 7f776bfff640 1 --2- 172.21.15.16:0/3335471417 >> [v2:172.21.15.16:6800/2615861334,v1:172.21.15.16:6801/2615861334] conn(0x7f7760038f70 0x7f776003b430 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f775c0098c0 tx=0x7f775c006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:52:48.124 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:47.389+0000 7f776b7fe640 1 -- 172.21.15.16:0/3335471417 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7764045e10 con 0x7f77741077e0 2024-09-12T01:52:48.124 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:47.566+0000 7f7778fde640 1 -- 172.21.15.16:0/3335471417 --> [v2:172.21.15.16:6800/2615861334,v1:172.21.15.16:6801/2615861334] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7f77741042c0 con 0x7f7760038f70 2024-09-12T01:52:48.124 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:47.567+0000 7f776b7fe640 1 -- 172.21.15.16:0/3335471417 <== mgr.14118 v2:172.21.15.16:6800/2615861334 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7f77741042c0 con 0x7f7760038f70 2024-09-12T01:52:48.124 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:47.569+0000 7f7778fde640 1 -- 172.21.15.16:0/3335471417 >> [v2:172.21.15.16:6800/2615861334,v1:172.21.15.16:6801/2615861334] conn(0x7f7760038f70 msgr2=0x7f776003b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:52:48.124 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:47.569+0000 7f7778fde640 1 --2- 172.21.15.16:0/3335471417 >> [v2:172.21.15.16:6800/2615861334,v1:172.21.15.16:6801/2615861334] conn(0x7f7760038f70 0x7f776003b430 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f775c0098c0 tx=0x7f775c006db0 comp rx=0 tx=0).stop 2024-09-12T01:52:48.124 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:47.569+0000 7f7778fde640 1 -- 172.21.15.16:0/3335471417 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77741077e0 msgr2=0x7f777419b830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:52:48.124 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:47.569+0000 7f7778fde640 1 --2- 172.21.15.16:0/3335471417 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77741077e0 0x7f777419b830 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f7764009940 tx=0x7f77640326e0 comp rx=0 tx=0).stop 2024-09-12T01:52:48.125 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:47.569+0000 7f7778fde640 1 -- 172.21.15.16:0/3335471417 shutdown_connections 2024-09-12T01:52:48.125 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:47.569+0000 7f7778fde640 1 --2- 172.21.15.16:0/3335471417 >> [v2:172.21.15.16:6800/2615861334,v1:172.21.15.16:6801/2615861334] conn(0x7f7760038f70 0x7f776003b430 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:48.125 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:47.569+0000 7f7778fde640 1 --2- 172.21.15.16:0/3335471417 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77741077e0 0x7f777419b830 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:48.125 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:47.569+0000 7f7778fde640 1 -- 172.21.15.16:0/3335471417 >> 172.21.15.16:0/3335471417 conn(0x7f7774102fb0 msgr2=0x7f7774103bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:52:48.125 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:47.569+0000 7f7778fde640 1 -- 172.21.15.16:0/3335471417 shutdown_connections 2024-09-12T01:52:48.125 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:47.569+0000 7f7778fde640 1 -- 172.21.15.16:0/3335471417 wait complete. 2024-09-12T01:52:48.125 INFO:teuthology.orchestra.run.smithi016.stdout:Generating ssh key... 2024-09-12T01:52:48.368 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:48 smithi016 ceph-mon[30947]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:52:49.169 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 01:52:48 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-x[31173]: Generating public/private rsa key pair. 2024-09-12T01:52:49.169 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 01:52:48 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-x[31173]: Your identification has been saved in /tmp/tmpkxvlan_x/key 2024-09-12T01:52:49.169 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 01:52:48 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-x[31173]: Your public key has been saved in /tmp/tmpkxvlan_x/key.pub 2024-09-12T01:52:49.169 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 01:52:48 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-x[31173]: The key fingerprint is: 2024-09-12T01:52:49.169 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 01:52:48 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-x[31173]: SHA256:j4JR+WqINFLO3K90QPrOaXI1/CHBAuuHWeoo0UyWXEc ceph-879f3b56-70a9-11ef-bcea-c7b262605968 2024-09-12T01:52:49.170 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 01:52:48 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-x[31173]: The key's randomart image is: 2024-09-12T01:52:49.170 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 01:52:48 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-x[31173]: +---[RSA 3072]----+ 2024-09-12T01:52:49.170 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 01:52:48 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-x[31173]: | .E | 2024-09-12T01:52:49.170 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 01:52:48 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-x[31173]: | . . .. | 2024-09-12T01:52:49.170 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 01:52:48 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-x[31173]: | ..=.oo | 2024-09-12T01:52:49.170 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 01:52:48 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-x[31173]: | =*+o.o. | 2024-09-12T01:52:49.170 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 01:52:48 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-x[31173]: |.*B=+o .S | 2024-09-12T01:52:49.170 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 01:52:48 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-x[31173]: |.oB+.*=..o | 2024-09-12T01:52:49.170 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 01:52:48 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-x[31173]: | +..=.*+... | 2024-09-12T01:52:49.170 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 01:52:48 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-x[31173]: |o o+o= .. | 2024-09-12T01:52:49.170 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 01:52:48 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-x[31173]: |. += | 2024-09-12T01:52:49.170 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 01:52:48 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-x[31173]: +----[SHA256]-----+ 2024-09-12T01:52:49.435 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:48.542+0000 7fa25c659640 1 Processor -- start 2024-09-12T01:52:49.436 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:48.542+0000 7fa25c659640 1 -- start start 2024-09-12T01:52:49.436 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:48.542+0000 7fa25c659640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa254106610 0x7fa254106a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:49.436 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:48.542+0000 7fa25c659640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa254106fe0 con 0x7fa254106610 2024-09-12T01:52:49.436 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:48.542+0000 7fa25a3f5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa254106610 0x7fa254106a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:49.436 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:48.542+0000 7fa25a3f5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa254106610 0x7fa254106a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52460/0 (socket says 172.21.15.16:52460) 2024-09-12T01:52:49.436 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:48.542+0000 7fa25a3f5640 1 -- 172.21.15.16:0/330972741 learned_addr learned my addr 172.21.15.16:0/330972741 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:52:49.436 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:48.543+0000 7fa25a3f5640 1 -- 172.21.15.16:0/330972741 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa254107810 con 0x7fa254106610 2024-09-12T01:52:49.436 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:48.543+0000 7fa25a3f5640 1 --2- 172.21.15.16:0/330972741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa254106610 0x7fa254106a10 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa250009820 tx=0x7fa25002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11c28ddd30d56cc0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:52:49.436 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:48.544+0000 7fa2593f3640 1 -- 172.21.15.16:0/330972741 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa250037070 con 0x7fa254106610 2024-09-12T01:52:49.436 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:48.544+0000 7fa2593f3640 1 -- 172.21.15.16:0/330972741 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa250032900 con 0x7fa254106610 2024-09-12T01:52:49.437 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:48.544+0000 7fa2593f3640 1 -- 172.21.15.16:0/330972741 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa250030850 con 0x7fa254106610 2024-09-12T01:52:49.437 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:48.544+0000 7fa25c659640 1 -- 172.21.15.16:0/330972741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa254106610 msgr2=0x7fa254106a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:52:49.437 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:48.544+0000 7fa25c659640 1 --2- 172.21.15.16:0/330972741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa254106610 0x7fa254106a10 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa250009820 tx=0x7fa25002a270 comp rx=0 tx=0).stop 2024-09-12T01:52:49.437 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:48.544+0000 7fa25c659640 1 -- 172.21.15.16:0/330972741 shutdown_connections 2024-09-12T01:52:49.437 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:48.544+0000 7fa25c659640 1 --2- 172.21.15.16:0/330972741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa254106610 0x7fa254106a10 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:49.437 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:48.544+0000 7fa25c659640 1 -- 172.21.15.16:0/330972741 >> 172.21.15.16:0/330972741 conn(0x7fa254101e20 msgr2=0x7fa254104240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:52:49.437 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:48.544+0000 7fa25c659640 1 -- 172.21.15.16:0/330972741 shutdown_connections 2024-09-12T01:52:49.437 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:48.544+0000 7fa25c659640 1 -- 172.21.15.16:0/330972741 wait complete. 2024-09-12T01:52:49.437 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:48.545+0000 7fa25c659640 1 Processor -- start 2024-09-12T01:52:49.437 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:48.545+0000 7fa25c659640 1 -- start start 2024-09-12T01:52:49.437 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:48.545+0000 7fa25c659640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa254106610 0x7fa254191840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:49.437 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:48.545+0000 7fa25c659640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa25410abc0 con 0x7fa254106610 2024-09-12T01:52:49.438 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:48.545+0000 7fa25a3f5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa254106610 0x7fa254191840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:49.438 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:48.545+0000 7fa25a3f5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa254106610 0x7fa254191840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52468/0 (socket says 172.21.15.16:52468) 2024-09-12T01:52:49.438 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:48.545+0000 7fa25a3f5640 1 -- 172.21.15.16:0/1719340667 learned_addr learned my addr 172.21.15.16:0/1719340667 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:52:49.438 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:48.546+0000 7fa25a3f5640 1 -- 172.21.15.16:0/1719340667 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa254191d80 con 0x7fa254106610 2024-09-12T01:52:49.438 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:48.546+0000 7fa25a3f5640 1 --2- 172.21.15.16:0/1719340667 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa254106610 0x7fa254191840 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa2500097f0 tx=0x7fa250030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:52:49.438 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:48.546+0000 7fa24b7fe640 1 -- 172.21.15.16:0/1719340667 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa250037040 con 0x7fa254106610 2024-09-12T01:52:49.438 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:48.546+0000 7fa24b7fe640 1 -- 172.21.15.16:0/1719340667 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa250040070 con 0x7fa254106610 2024-09-12T01:52:49.438 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:48.546+0000 7fa25c659640 1 -- 172.21.15.16:0/1719340667 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa25418ff70 con 0x7fa254106610 2024-09-12T01:52:49.438 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:48.546+0000 7fa24b7fe640 1 -- 172.21.15.16:0/1719340667 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa25003a390 con 0x7fa254106610 2024-09-12T01:52:49.438 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:48.546+0000 7fa25c659640 1 -- 172.21.15.16:0/1719340667 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa254190450 con 0x7fa254106610 2024-09-12T01:52:49.438 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:48.547+0000 7fa24b7fe640 1 -- 172.21.15.16:0/1719340667 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fa25003a5e0 con 0x7fa254106610 2024-09-12T01:52:49.438 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:48.547+0000 7fa24b7fe640 1 --2- 172.21.15.16:0/1719340667 >> [v2:172.21.15.16:6800/2615861334,v1:172.21.15.16:6801/2615861334] conn(0x7fa240038fc0 0x7fa24003b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:49.438 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:48.547+0000 7fa24b7fe640 1 -- 172.21.15.16:0/1719340667 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fa25006c5d0 con 0x7fa254106610 2024-09-12T01:52:49.438 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:48.547+0000 7fa25c659640 1 -- 172.21.15.16:0/1719340667 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa25410ad40 con 0x7fa254106610 2024-09-12T01:52:49.439 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:48.547+0000 7fa259bf4640 1 --2- 172.21.15.16:0/1719340667 >> [v2:172.21.15.16:6800/2615861334,v1:172.21.15.16:6801/2615861334] conn(0x7fa240038fc0 0x7fa24003b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:49.439 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:48.548+0000 7fa259bf4640 1 --2- 172.21.15.16:0/1719340667 >> [v2:172.21.15.16:6800/2615861334,v1:172.21.15.16:6801/2615861334] conn(0x7fa240038fc0 0x7fa24003b480 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fa2440098c0 tx=0x7fa244006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:52:49.439 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:48.552+0000 7fa24b7fe640 1 -- 172.21.15.16:0/1719340667 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa25003a920 con 0x7fa254106610 2024-09-12T01:52:49.439 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:48.730+0000 7fa25c659640 1 -- 172.21.15.16:0/1719340667 --> [v2:172.21.15.16:6800/2615861334,v1:172.21.15.16:6801/2615861334] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7fa2541956c0 con 0x7fa240038fc0 2024-09-12T01:52:49.439 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:48.869+0000 7fa24b7fe640 1 -- 172.21.15.16:0/1719340667 <== mgr.14118 v2:172.21.15.16:6800/2615861334 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fa2541956c0 con 0x7fa240038fc0 2024-09-12T01:52:49.439 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:48.871+0000 7fa25c659640 1 -- 172.21.15.16:0/1719340667 >> [v2:172.21.15.16:6800/2615861334,v1:172.21.15.16:6801/2615861334] conn(0x7fa240038fc0 msgr2=0x7fa24003b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:52:49.439 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:48.871+0000 7fa25c659640 1 --2- 172.21.15.16:0/1719340667 >> [v2:172.21.15.16:6800/2615861334,v1:172.21.15.16:6801/2615861334] conn(0x7fa240038fc0 0x7fa24003b480 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fa2440098c0 tx=0x7fa244006db0 comp rx=0 tx=0).stop 2024-09-12T01:52:49.439 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:48.871+0000 7fa25c659640 1 -- 172.21.15.16:0/1719340667 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa254106610 msgr2=0x7fa254191840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:52:49.439 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:48.871+0000 7fa25c659640 1 --2- 172.21.15.16:0/1719340667 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa254106610 0x7fa254191840 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa2500097f0 tx=0x7fa250030d80 comp rx=0 tx=0).stop 2024-09-12T01:52:49.439 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:48.871+0000 7fa25c659640 1 -- 172.21.15.16:0/1719340667 shutdown_connections 2024-09-12T01:52:49.439 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:48.871+0000 7fa25c659640 1 --2- 172.21.15.16:0/1719340667 >> [v2:172.21.15.16:6800/2615861334,v1:172.21.15.16:6801/2615861334] conn(0x7fa240038fc0 0x7fa24003b480 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:49.439 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:48.871+0000 7fa25c659640 1 --2- 172.21.15.16:0/1719340667 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa254106610 0x7fa254191840 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:49.439 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:48.871+0000 7fa25c659640 1 -- 172.21.15.16:0/1719340667 >> 172.21.15.16:0/1719340667 conn(0x7fa254101e20 msgr2=0x7fa254102d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:52:49.440 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:48.871+0000 7fa25c659640 1 -- 172.21.15.16:0/1719340667 shutdown_connections 2024-09-12T01:52:49.440 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:48.871+0000 7fa25c659640 1 -- 172.21.15.16:0/1719340667 wait complete. 2024-09-12T01:52:50.043 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:49 smithi016 ceph-mon[30947]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:52:50.043 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:49 smithi016 ceph-mon[30947]: Generating ssh key... 2024-09-12T01:52:50.043 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:49 smithi016 ceph-mon[30947]: from='mgr.14118 172.21.15.16:0/2569835181' entity='mgr.x' 2024-09-12T01:52:50.043 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:49 smithi016 ceph-mon[30947]: from='mgr.14118 172.21.15.16:0/2569835181' entity='mgr.x' 2024-09-12T01:52:50.626 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDXax7ciWzoEL/3Zpb3dWbBuvhwWB5WYE+lekT8TQbOyZE+KRRh5EnP/BNJyD0jlGM7ijCJvTlMiwBoCKZQIovzip7esb+aYqLZ/iOvYwFMvcCl1VrGKkC82DsdaNkpCGBMNQOw7ypYx6AGQ/zl/bcTVhJrBjFScjqKifBeJTZPp3Qy/2RNPpie2tyqEMh3VFMeLAtr4GoH82TjMwF+ZPHtRKLyUS0uolyduRySxkGix566xnX8lDGiYcLqHLp5KTKbGaLLJhP4XNbLvD2fWir4vxaYUvjJbFlTBepqNvmYt1mp5rLyXt07F1GnfCxOlXFGCs5DWzHyl1s6Uzfq1Z7TdIsBhfBFBlvLvXIzBDevsEgPV/X9BJvowGh/BA/FFeESKQnNjuKYYM2TPbGkM2brpHJMtZhYl9xgVdADzPjb4y++XAgyq6bosTuv76LzBCyiviOZe+2IHD1YTzWmndGIBpJXmTnL9DL/0hL5kbpTY1J4nl7B1PyK6cAEIlb7BM8= ceph-879f3b56-70a9-11ef-bcea-c7b262605968 2024-09-12T01:52:50.626 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:49.854+0000 7fdf67f28640 1 Processor -- start 2024-09-12T01:52:50.628 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:49.854+0000 7fdf67f28640 1 -- start start 2024-09-12T01:52:50.628 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:49.854+0000 7fdf67f28640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf60107780 0x7fdf60107b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:50.628 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:49.854+0000 7fdf67f28640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fdf60108150 con 0x7fdf60107780 2024-09-12T01:52:50.628 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:49.854+0000 7fdf65cc4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf60107780 0x7fdf60107b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:50.628 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:49.854+0000 7fdf65cc4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf60107780 0x7fdf60107b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52476/0 (socket says 172.21.15.16:52476) 2024-09-12T01:52:50.628 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:49.854+0000 7fdf65cc4640 1 -- 172.21.15.16:0/397176989 learned_addr learned my addr 172.21.15.16:0/397176989 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:52:50.628 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:49.855+0000 7fdf65cc4640 1 -- 172.21.15.16:0/397176989 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf60108980 con 0x7fdf60107780 2024-09-12T01:52:50.628 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:49.855+0000 7fdf65cc4640 1 --2- 172.21.15.16:0/397176989 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf60107780 0x7fdf60107b80 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fdf5c009820 tx=0x7fdf5c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7536b9afe2d87d5a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:52:50.628 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:49.855+0000 7fdf64cc2640 1 -- 172.21.15.16:0/397176989 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdf5c037070 con 0x7fdf60107780 2024-09-12T01:52:50.629 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:49.856+0000 7fdf64cc2640 1 -- 172.21.15.16:0/397176989 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fdf5c032900 con 0x7fdf60107780 2024-09-12T01:52:50.629 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:49.856+0000 7fdf64cc2640 1 -- 172.21.15.16:0/397176989 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdf5c030850 con 0x7fdf60107780 2024-09-12T01:52:50.629 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:49.856+0000 7fdf67f28640 1 -- 172.21.15.16:0/397176989 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf60107780 msgr2=0x7fdf60107b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:52:50.629 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:49.856+0000 7fdf67f28640 1 --2- 172.21.15.16:0/397176989 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf60107780 0x7fdf60107b80 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fdf5c009820 tx=0x7fdf5c02a270 comp rx=0 tx=0).stop 2024-09-12T01:52:50.629 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:49.856+0000 7fdf67f28640 1 -- 172.21.15.16:0/397176989 shutdown_connections 2024-09-12T01:52:50.629 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:49.856+0000 7fdf67f28640 1 --2- 172.21.15.16:0/397176989 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf60107780 0x7fdf60107b80 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:50.629 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:49.856+0000 7fdf67f28640 1 -- 172.21.15.16:0/397176989 >> 172.21.15.16:0/397176989 conn(0x7fdf60102f30 msgr2=0x7fdf60105350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:52:50.629 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:49.856+0000 7fdf67f28640 1 -- 172.21.15.16:0/397176989 shutdown_connections 2024-09-12T01:52:50.629 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:49.856+0000 7fdf67f28640 1 -- 172.21.15.16:0/397176989 wait complete. 2024-09-12T01:52:50.629 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:49.857+0000 7fdf67f28640 1 Processor -- start 2024-09-12T01:52:50.629 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:49.857+0000 7fdf67f28640 1 -- start start 2024-09-12T01:52:50.629 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:49.857+0000 7fdf67f28640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf60107780 0x7fdf6019d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:50.629 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:49.857+0000 7fdf67f28640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fdf6010bd30 con 0x7fdf60107780 2024-09-12T01:52:50.630 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:49.857+0000 7fdf65cc4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf60107780 0x7fdf6019d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:50.630 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:49.857+0000 7fdf65cc4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf60107780 0x7fdf6019d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52484/0 (socket says 172.21.15.16:52484) 2024-09-12T01:52:50.630 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:49.857+0000 7fdf65cc4640 1 -- 172.21.15.16:0/2594517239 learned_addr learned my addr 172.21.15.16:0/2594517239 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:52:50.630 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:49.858+0000 7fdf65cc4640 1 -- 172.21.15.16:0/2594517239 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf6019de80 con 0x7fdf60107780 2024-09-12T01:52:50.630 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:49.858+0000 7fdf65cc4640 1 --2- 172.21.15.16:0/2594517239 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf60107780 0x7fdf6019d940 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fdf5c0097f0 tx=0x7fdf5c030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:52:50.630 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:49.858+0000 7fdf56ffd640 1 -- 172.21.15.16:0/2594517239 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdf5c037040 con 0x7fdf60107780 2024-09-12T01:52:50.630 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:49.858+0000 7fdf67f28640 1 -- 172.21.15.16:0/2594517239 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf6019e110 con 0x7fdf60107780 2024-09-12T01:52:50.630 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:49.858+0000 7fdf56ffd640 1 -- 172.21.15.16:0/2594517239 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fdf5c040070 con 0x7fdf60107780 2024-09-12T01:52:50.630 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:49.858+0000 7fdf56ffd640 1 -- 172.21.15.16:0/2594517239 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdf5c03a390 con 0x7fdf60107780 2024-09-12T01:52:50.630 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:49.858+0000 7fdf67f28640 1 -- 172.21.15.16:0/2594517239 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf601a0e00 con 0x7fdf60107780 2024-09-12T01:52:50.630 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:49.859+0000 7fdf56ffd640 1 -- 172.21.15.16:0/2594517239 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fdf5c03a630 con 0x7fdf60107780 2024-09-12T01:52:50.630 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:49.859+0000 7fdf67f28640 1 -- 172.21.15.16:0/2594517239 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf60107c00 con 0x7fdf60107780 2024-09-12T01:52:50.630 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:49.859+0000 7fdf56ffd640 1 --2- 172.21.15.16:0/2594517239 >> [v2:172.21.15.16:6800/2615861334,v1:172.21.15.16:6801/2615861334] conn(0x7fdf4c038f70 0x7fdf4c03b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:50.630 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:49.859+0000 7fdf56ffd640 1 -- 172.21.15.16:0/2594517239 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fdf5c06c600 con 0x7fdf60107780 2024-09-12T01:52:50.631 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:49.859+0000 7fdf654c3640 1 --2- 172.21.15.16:0/2594517239 >> [v2:172.21.15.16:6800/2615861334,v1:172.21.15.16:6801/2615861334] conn(0x7fdf4c038f70 0x7fdf4c03b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:50.631 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:49.860+0000 7fdf654c3640 1 --2- 172.21.15.16:0/2594517239 >> [v2:172.21.15.16:6800/2615861334,v1:172.21.15.16:6801/2615861334] conn(0x7fdf4c038f70 0x7fdf4c03b430 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fdf500098c0 tx=0x7fdf50006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:52:50.631 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:49.864+0000 7fdf56ffd640 1 -- 172.21.15.16:0/2594517239 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fdf5c043bf0 con 0x7fdf60107780 2024-09-12T01:52:50.631 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:50.042+0000 7fdf67f28640 1 -- 172.21.15.16:0/2594517239 --> [v2:172.21.15.16:6800/2615861334,v1:172.21.15.16:6801/2615861334] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7fdf60104130 con 0x7fdf4c038f70 2024-09-12T01:52:50.631 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:50.043+0000 7fdf56ffd640 1 -- 172.21.15.16:0/2594517239 <== mgr.14118 v2:172.21.15.16:6800/2615861334 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7fdf60104130 con 0x7fdf4c038f70 2024-09-12T01:52:50.631 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:50.045+0000 7fdf67f28640 1 -- 172.21.15.16:0/2594517239 >> [v2:172.21.15.16:6800/2615861334,v1:172.21.15.16:6801/2615861334] conn(0x7fdf4c038f70 msgr2=0x7fdf4c03b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:52:50.631 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:50.045+0000 7fdf67f28640 1 --2- 172.21.15.16:0/2594517239 >> [v2:172.21.15.16:6800/2615861334,v1:172.21.15.16:6801/2615861334] conn(0x7fdf4c038f70 0x7fdf4c03b430 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fdf500098c0 tx=0x7fdf50006db0 comp rx=0 tx=0).stop 2024-09-12T01:52:50.631 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:50.045+0000 7fdf67f28640 1 -- 172.21.15.16:0/2594517239 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf60107780 msgr2=0x7fdf6019d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:52:50.631 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:50.045+0000 7fdf67f28640 1 --2- 172.21.15.16:0/2594517239 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf60107780 0x7fdf6019d940 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fdf5c0097f0 tx=0x7fdf5c030d80 comp rx=0 tx=0).stop 2024-09-12T01:52:50.631 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:50.045+0000 7fdf67f28640 1 -- 172.21.15.16:0/2594517239 shutdown_connections 2024-09-12T01:52:50.631 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:50.045+0000 7fdf67f28640 1 --2- 172.21.15.16:0/2594517239 >> [v2:172.21.15.16:6800/2615861334,v1:172.21.15.16:6801/2615861334] conn(0x7fdf4c038f70 0x7fdf4c03b430 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:50.631 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:50.045+0000 7fdf67f28640 1 --2- 172.21.15.16:0/2594517239 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf60107780 0x7fdf6019d940 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:50.631 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:50.045+0000 7fdf67f28640 1 -- 172.21.15.16:0/2594517239 >> 172.21.15.16:0/2594517239 conn(0x7fdf60102f30 msgr2=0x7fdf60103a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:52:50.631 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:50.045+0000 7fdf67f28640 1 -- 172.21.15.16:0/2594517239 shutdown_connections 2024-09-12T01:52:50.632 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:50.045+0000 7fdf67f28640 1 -- 172.21.15.16:0/2594517239 wait complete. 2024-09-12T01:52:50.632 INFO:teuthology.orchestra.run.smithi016.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-09-12T01:52:50.632 INFO:teuthology.orchestra.run.smithi016.stdout:Adding key to root@localhost authorized_keys... 2024-09-12T01:52:50.632 INFO:teuthology.orchestra.run.smithi016.stdout:Adding host smithi016... 2024-09-12T01:52:50.933 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:50 smithi016 ceph-mon[30947]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:52:52.577 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:52 smithi016 ceph-mon[30947]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi016", "addr": "172.21.15.16", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:52:53.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:53 smithi016 ceph-mon[30947]: Deploying cephadm binary to smithi016 2024-09-12T01:52:55.676 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout Added host 'smithi016' with addr '172.21.15.16' 2024-09-12T01:52:55.677 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:51.062+0000 7efc051f4640 1 Processor -- start 2024-09-12T01:52:55.677 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:51.062+0000 7efc051f4640 1 -- start start 2024-09-12T01:52:55.677 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:51.062+0000 7efc051f4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc00108140 0x7efc0010a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:55.677 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:51.062+0000 7efc051f4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7efc0007be30 con 0x7efc00108140 2024-09-12T01:52:55.677 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:51.063+0000 7efbfed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc00108140 0x7efc0010a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:55.677 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:51.063+0000 7efbfed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc00108140 0x7efc0010a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52492/0 (socket says 172.21.15.16:52492) 2024-09-12T01:52:55.677 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:51.063+0000 7efbfed9d640 1 -- 172.21.15.16:0/1325790696 learned_addr learned my addr 172.21.15.16:0/1325790696 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:52:55.677 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:51.063+0000 7efbfed9d640 1 -- 172.21.15.16:0/1325790696 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc0010aa70 con 0x7efc00108140 2024-09-12T01:52:55.677 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:51.064+0000 7efbfed9d640 1 --2- 172.21.15.16:0/1325790696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc00108140 0x7efc0010a530 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7efbf0009820 tx=0x7efbf002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9691211ac9e6c8cb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:52:55.677 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:51.064+0000 7efbfdd9b640 1 -- 172.21.15.16:0/1325790696 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efbf0037070 con 0x7efc00108140 2024-09-12T01:52:55.677 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:51.064+0000 7efbfdd9b640 1 -- 172.21.15.16:0/1325790696 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7efbf0032900 con 0x7efc00108140 2024-09-12T01:52:55.677 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:51.064+0000 7efbfdd9b640 1 -- 172.21.15.16:0/1325790696 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efbf0030850 con 0x7efc00108140 2024-09-12T01:52:55.678 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:51.065+0000 7efc051f4640 1 -- 172.21.15.16:0/1325790696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc00108140 msgr2=0x7efc0010a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:52:55.678 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:51.065+0000 7efc051f4640 1 --2- 172.21.15.16:0/1325790696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc00108140 0x7efc0010a530 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7efbf0009820 tx=0x7efbf002a270 comp rx=0 tx=0).stop 2024-09-12T01:52:55.678 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:51.065+0000 7efc051f4640 1 -- 172.21.15.16:0/1325790696 shutdown_connections 2024-09-12T01:52:55.678 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:51.065+0000 7efc051f4640 1 --2- 172.21.15.16:0/1325790696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc00108140 0x7efc0010a530 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:55.678 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:51.065+0000 7efc051f4640 1 -- 172.21.15.16:0/1325790696 >> 172.21.15.16:0/1325790696 conn(0x7efc00101e20 msgr2=0x7efc00104240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:52:55.678 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:51.065+0000 7efc051f4640 1 -- 172.21.15.16:0/1325790696 shutdown_connections 2024-09-12T01:52:55.678 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:51.065+0000 7efc051f4640 1 -- 172.21.15.16:0/1325790696 wait complete. 2024-09-12T01:52:55.678 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:51.065+0000 7efc051f4640 1 Processor -- start 2024-09-12T01:52:55.678 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:51.066+0000 7efc051f4640 1 -- start start 2024-09-12T01:52:55.678 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:51.066+0000 7efc051f4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc00108140 0x7efc0019b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:55.678 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:51.066+0000 7efc051f4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7efc0010bb10 con 0x7efc00108140 2024-09-12T01:52:55.678 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:51.066+0000 7efbfed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc00108140 0x7efc0019b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:55.678 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:51.066+0000 7efbfed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc00108140 0x7efc0019b6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52494/0 (socket says 172.21.15.16:52494) 2024-09-12T01:52:55.679 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:51.066+0000 7efbfed9d640 1 -- 172.21.15.16:0/3608153749 learned_addr learned my addr 172.21.15.16:0/3608153749 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:52:55.679 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:51.066+0000 7efbfed9d640 1 -- 172.21.15.16:0/3608153749 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc0019bc00 con 0x7efc00108140 2024-09-12T01:52:55.679 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:51.066+0000 7efbfed9d640 1 --2- 172.21.15.16:0/3608153749 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc00108140 0x7efc0019b6c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7efbf0032ce0 tx=0x7efbf0032d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:52:55.679 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:51.067+0000 7efbe7fff640 1 -- 172.21.15.16:0/3608153749 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efbf0042070 con 0x7efc00108140 2024-09-12T01:52:55.679 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:51.067+0000 7efbe7fff640 1 -- 172.21.15.16:0/3608153749 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7efbf00308c0 con 0x7efc00108140 2024-09-12T01:52:55.679 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:51.067+0000 7efc051f4640 1 -- 172.21.15.16:0/3608153749 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc0019be90 con 0x7efc00108140 2024-09-12T01:52:55.679 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:51.067+0000 7efbe7fff640 1 -- 172.21.15.16:0/3608153749 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efbf0037040 con 0x7efc00108140 2024-09-12T01:52:55.679 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:51.067+0000 7efc051f4640 1 -- 172.21.15.16:0/3608153749 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc0019eb80 con 0x7efc00108140 2024-09-12T01:52:55.679 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:51.067+0000 7efbe7fff640 1 -- 172.21.15.16:0/3608153749 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7efbf0045070 con 0x7efc00108140 2024-09-12T01:52:55.679 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:51.068+0000 7efbe7fff640 1 --2- 172.21.15.16:0/3608153749 >> [v2:172.21.15.16:6800/2615861334,v1:172.21.15.16:6801/2615861334] conn(0x7efbe8038f70 0x7efbe803b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:55.679 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:51.068+0000 7efbe7fff640 1 -- 172.21.15.16:0/3608153749 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7efbf006c080 con 0x7efc00108140 2024-09-12T01:52:55.679 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:51.068+0000 7efc051f4640 1 -- 172.21.15.16:0/3608153749 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc001085a0 con 0x7efc00108140 2024-09-12T01:52:55.679 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:51.068+0000 7efbfe59c640 1 --2- 172.21.15.16:0/3608153749 >> [v2:172.21.15.16:6800/2615861334,v1:172.21.15.16:6801/2615861334] conn(0x7efbe8038f70 0x7efbe803b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:55.680 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:51.068+0000 7efbfe59c640 1 --2- 172.21.15.16:0/3608153749 >> [v2:172.21.15.16:6800/2615861334,v1:172.21.15.16:6801/2615861334] conn(0x7efbe8038f70 0x7efbe803b430 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7efbf400ad30 tx=0x7efbf40093f0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:52:55.680 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:51.073+0000 7efbe7fff640 1 -- 172.21.15.16:0/3608153749 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7efbf003cd00 con 0x7efc00108140 2024-09-12T01:52:55.680 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:51.254+0000 7efc051f4640 1 -- 172.21.15.16:0/3608153749 --> [v2:172.21.15.16:6800/2615861334,v1:172.21.15.16:6801/2615861334] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi016", "addr": "172.21.15.16", "target": ["mon-mgr", ""]}) -- 0x7efc00103100 con 0x7efbe8038f70 2024-09-12T01:52:55.680 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:55.111+0000 7efbe7fff640 1 -- 172.21.15.16:0/3608153749 <== mgr.14118 v2:172.21.15.16:6800/2615861334 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7efc00103100 con 0x7efbe8038f70 2024-09-12T01:52:55.680 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:55.113+0000 7efc051f4640 1 -- 172.21.15.16:0/3608153749 >> [v2:172.21.15.16:6800/2615861334,v1:172.21.15.16:6801/2615861334] conn(0x7efbe8038f70 msgr2=0x7efbe803b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:52:55.680 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:55.113+0000 7efc051f4640 1 --2- 172.21.15.16:0/3608153749 >> [v2:172.21.15.16:6800/2615861334,v1:172.21.15.16:6801/2615861334] conn(0x7efbe8038f70 0x7efbe803b430 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7efbf400ad30 tx=0x7efbf40093f0 comp rx=0 tx=0).stop 2024-09-12T01:52:55.680 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:55.113+0000 7efc051f4640 1 -- 172.21.15.16:0/3608153749 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc00108140 msgr2=0x7efc0019b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:52:55.680 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:55.113+0000 7efc051f4640 1 --2- 172.21.15.16:0/3608153749 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc00108140 0x7efc0019b6c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7efbf0032ce0 tx=0x7efbf0032d10 comp rx=0 tx=0).stop 2024-09-12T01:52:55.680 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:55.113+0000 7efc051f4640 1 -- 172.21.15.16:0/3608153749 shutdown_connections 2024-09-12T01:52:55.680 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:55.113+0000 7efc051f4640 1 --2- 172.21.15.16:0/3608153749 >> [v2:172.21.15.16:6800/2615861334,v1:172.21.15.16:6801/2615861334] conn(0x7efbe8038f70 0x7efbe803b430 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:55.680 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:55.113+0000 7efc051f4640 1 --2- 172.21.15.16:0/3608153749 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc00108140 0x7efc0019b6c0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:55.680 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:55.113+0000 7efc051f4640 1 -- 172.21.15.16:0/3608153749 >> 172.21.15.16:0/3608153749 conn(0x7efc00101e20 msgr2=0x7efc001029f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:52:55.680 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:55.114+0000 7efc051f4640 1 -- 172.21.15.16:0/3608153749 shutdown_connections 2024-09-12T01:52:55.681 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:55.114+0000 7efc051f4640 1 -- 172.21.15.16:0/3608153749 wait complete. 2024-09-12T01:52:55.681 INFO:teuthology.orchestra.run.smithi016.stdout:Deploying unmanaged mon service... 2024-09-12T01:52:56.309 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:56 smithi016 ceph-mon[30947]: from='mgr.14118 172.21.15.16:0/2569835181' entity='mgr.x' 2024-09-12T01:52:56.310 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:56 smithi016 ceph-mon[30947]: Added host smithi016 2024-09-12T01:52:56.310 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:56 smithi016 ceph-mon[30947]: from='mgr.14118 172.21.15.16:0/2569835181' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:52:57.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:57 smithi016 ceph-mon[30947]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:52:57.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:57 smithi016 ceph-mon[30947]: Saving service mon spec with placement count:5 2024-09-12T01:52:57.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:57 smithi016 ceph-mon[30947]: from='mgr.14118 172.21.15.16:0/2569835181' entity='mgr.x' 2024-09-12T01:52:57.999 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-09-12T01:52:58.000 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:56.129+0000 7f759d541640 1 Processor -- start 2024-09-12T01:52:58.000 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:56.129+0000 7f759d541640 1 -- start start 2024-09-12T01:52:58.000 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:56.129+0000 7f759d541640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7598108200 0x7f759810a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:58.000 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:56.129+0000 7f759d541640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f759807bdb0 con 0x7f7598108200 2024-09-12T01:52:58.001 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:56.129+0000 7f7596ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7598108200 0x7f759810a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:58.001 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:56.129+0000 7f7596ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7598108200 0x7f759810a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40992/0 (socket says 172.21.15.16:40992) 2024-09-12T01:52:58.001 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:56.129+0000 7f7596ffd640 1 -- 172.21.15.16:0/3049657081 learned_addr learned my addr 172.21.15.16:0/3049657081 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:52:58.001 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:56.129+0000 7f7596ffd640 1 -- 172.21.15.16:0/3049657081 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f759810ab30 con 0x7f7598108200 2024-09-12T01:52:58.001 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:56.130+0000 7f7596ffd640 1 --2- 172.21.15.16:0/3049657081 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7598108200 0x7f759810a5f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f7588009820 tx=0x7f758802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f994d769cf4554c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:52:58.001 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:56.131+0000 7f7595ffb640 1 -- 172.21.15.16:0/3049657081 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7588037070 con 0x7f7598108200 2024-09-12T01:52:58.001 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:56.131+0000 7f7595ffb640 1 -- 172.21.15.16:0/3049657081 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7588032900 con 0x7f7598108200 2024-09-12T01:52:58.001 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:56.131+0000 7f7595ffb640 1 -- 172.21.15.16:0/3049657081 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7588030850 con 0x7f7598108200 2024-09-12T01:52:58.001 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:56.131+0000 7f759d541640 1 -- 172.21.15.16:0/3049657081 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7598108200 msgr2=0x7f759810a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:52:58.001 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:56.131+0000 7f759d541640 1 --2- 172.21.15.16:0/3049657081 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7598108200 0x7f759810a5f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f7588009820 tx=0x7f758802a270 comp rx=0 tx=0).stop 2024-09-12T01:52:58.001 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:56.131+0000 7f759d541640 1 -- 172.21.15.16:0/3049657081 shutdown_connections 2024-09-12T01:52:58.002 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:56.131+0000 7f759d541640 1 --2- 172.21.15.16:0/3049657081 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7598108200 0x7f759810a5f0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:58.002 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:56.131+0000 7f759d541640 1 -- 172.21.15.16:0/3049657081 >> 172.21.15.16:0/3049657081 conn(0x7f7598101e20 msgr2=0x7f7598104240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:52:58.002 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:56.131+0000 7f759d541640 1 -- 172.21.15.16:0/3049657081 shutdown_connections 2024-09-12T01:52:58.002 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:56.131+0000 7f759d541640 1 -- 172.21.15.16:0/3049657081 wait complete. 2024-09-12T01:52:58.002 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:56.132+0000 7f759d541640 1 Processor -- start 2024-09-12T01:52:58.002 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:56.132+0000 7f759d541640 1 -- start start 2024-09-12T01:52:58.002 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:56.132+0000 7f759d541640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7598108200 0x7f759819d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:58.002 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:56.132+0000 7f759d541640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f759810bb10 con 0x7f7598108200 2024-09-12T01:52:58.002 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:56.132+0000 7f7596ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7598108200 0x7f759819d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:58.002 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:56.132+0000 7f7596ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7598108200 0x7f759819d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40994/0 (socket says 172.21.15.16:40994) 2024-09-12T01:52:58.002 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:56.132+0000 7f7596ffd640 1 -- 172.21.15.16:0/3784207436 learned_addr learned my addr 172.21.15.16:0/3784207436 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:52:58.002 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:56.132+0000 7f7596ffd640 1 -- 172.21.15.16:0/3784207436 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f759819de60 con 0x7f7598108200 2024-09-12T01:52:58.003 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:56.133+0000 7f7596ffd640 1 --2- 172.21.15.16:0/3784207436 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7598108200 0x7f759819d920 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f75880097f0 tx=0x7f7588030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:52:58.003 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:56.133+0000 7f757ffff640 1 -- 172.21.15.16:0/3784207436 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7588037040 con 0x7f7598108200 2024-09-12T01:52:58.003 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:56.133+0000 7f757ffff640 1 -- 172.21.15.16:0/3784207436 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7588040070 con 0x7f7598108200 2024-09-12T01:52:58.003 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:56.133+0000 7f759d541640 1 -- 172.21.15.16:0/3784207436 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f759819e0f0 con 0x7f7598108200 2024-09-12T01:52:58.003 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:56.133+0000 7f757ffff640 1 -- 172.21.15.16:0/3784207436 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f758803a390 con 0x7f7598108200 2024-09-12T01:52:58.003 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:56.133+0000 7f759d541640 1 -- 172.21.15.16:0/3784207436 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75981a0de0 con 0x7f7598108200 2024-09-12T01:52:58.003 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:56.134+0000 7f757ffff640 1 -- 172.21.15.16:0/3784207436 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f7588032900 con 0x7f7598108200 2024-09-12T01:52:58.003 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:56.134+0000 7f759d541640 1 -- 172.21.15.16:0/3784207436 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7598108660 con 0x7f7598108200 2024-09-12T01:52:58.003 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:56.134+0000 7f757ffff640 1 --2- 172.21.15.16:0/3784207436 >> [v2:172.21.15.16:6800/2615861334,v1:172.21.15.16:6801/2615861334] conn(0x7f7580038fc0 0x7f758003b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:58.003 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:56.134+0000 7f757ffff640 1 -- 172.21.15.16:0/3784207436 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f758806c460 con 0x7f7598108200 2024-09-12T01:52:58.003 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:56.134+0000 7f75967fc640 1 --2- 172.21.15.16:0/3784207436 >> [v2:172.21.15.16:6800/2615861334,v1:172.21.15.16:6801/2615861334] conn(0x7f7580038fc0 0x7f758003b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:58.004 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:56.134+0000 7f75967fc640 1 --2- 172.21.15.16:0/3784207436 >> [v2:172.21.15.16:6800/2615861334,v1:172.21.15.16:6801/2615861334] conn(0x7f7580038fc0 0x7f758003b480 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f758c0098c0 tx=0x7f758c006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:52:58.004 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:56.139+0000 7f757ffff640 1 -- 172.21.15.16:0/3784207436 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7588040ce0 con 0x7f7598108200 2024-09-12T01:52:58.004 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:56.334+0000 7f759d541640 1 -- 172.21.15.16:0/3784207436 --> [v2:172.21.15.16:6800/2615861334,v1:172.21.15.16:6801/2615861334] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f7598103070 con 0x7f7580038fc0 2024-09-12T01:52:58.004 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:56.338+0000 7f757ffff640 1 -- 172.21.15.16:0/3784207436 <== mgr.14118 v2:172.21.15.16:6800/2615861334 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f7598103070 con 0x7f7580038fc0 2024-09-12T01:52:58.004 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:56.341+0000 7f759d541640 1 -- 172.21.15.16:0/3784207436 >> [v2:172.21.15.16:6800/2615861334,v1:172.21.15.16:6801/2615861334] conn(0x7f7580038fc0 msgr2=0x7f758003b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:52:58.004 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:56.341+0000 7f759d541640 1 --2- 172.21.15.16:0/3784207436 >> [v2:172.21.15.16:6800/2615861334,v1:172.21.15.16:6801/2615861334] conn(0x7f7580038fc0 0x7f758003b480 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f758c0098c0 tx=0x7f758c006db0 comp rx=0 tx=0).stop 2024-09-12T01:52:58.004 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:56.341+0000 7f759d541640 1 -- 172.21.15.16:0/3784207436 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7598108200 msgr2=0x7f759819d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:52:58.004 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:56.341+0000 7f759d541640 1 --2- 172.21.15.16:0/3784207436 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7598108200 0x7f759819d920 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f75880097f0 tx=0x7f7588030d80 comp rx=0 tx=0).stop 2024-09-12T01:52:58.004 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:56.341+0000 7f759d541640 1 -- 172.21.15.16:0/3784207436 shutdown_connections 2024-09-12T01:52:58.004 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:56.341+0000 7f759d541640 1 --2- 172.21.15.16:0/3784207436 >> [v2:172.21.15.16:6800/2615861334,v1:172.21.15.16:6801/2615861334] conn(0x7f7580038fc0 0x7f758003b480 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:58.004 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:56.341+0000 7f759d541640 1 --2- 172.21.15.16:0/3784207436 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7598108200 0x7f759819d920 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:58.004 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:56.341+0000 7f759d541640 1 -- 172.21.15.16:0/3784207436 >> 172.21.15.16:0/3784207436 conn(0x7f7598101e20 msgr2=0x7f7598102960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:52:58.004 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:56.341+0000 7f759d541640 1 -- 172.21.15.16:0/3784207436 shutdown_connections 2024-09-12T01:52:58.005 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:56.341+0000 7f759d541640 1 -- 172.21.15.16:0/3784207436 wait complete. 2024-09-12T01:52:58.005 INFO:teuthology.orchestra.run.smithi016.stdout:Deploying unmanaged mgr service... 2024-09-12T01:52:58.638 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:58 smithi016 ceph-mon[30947]: from='mgr.14118 172.21.15.16:0/2569835181' entity='mgr.x' 2024-09-12T01:52:58.638 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:58 smithi016 ceph-mon[30947]: from='mgr.14118 172.21.15.16:0/2569835181' entity='mgr.x' 2024-09-12T01:52:59.308 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-09-12T01:52:59.308 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:58.417+0000 7f382acaa640 1 Processor -- start 2024-09-12T01:52:59.308 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:58.417+0000 7f382acaa640 1 -- start start 2024-09-12T01:52:59.308 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:58.417+0000 7f382acaa640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3824105a80 0x7f3824105e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:59.308 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:58.417+0000 7f382acaa640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3824106450 con 0x7f3824105a80 2024-09-12T01:52:59.308 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:58.417+0000 7f3828a46640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3824105a80 0x7f3824105e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:59.308 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:58.418+0000 7f3828a46640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3824105a80 0x7f3824105e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41002/0 (socket says 172.21.15.16:41002) 2024-09-12T01:52:59.309 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:58.418+0000 7f3828a46640 1 -- 172.21.15.16:0/856580994 learned_addr learned my addr 172.21.15.16:0/856580994 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:52:59.309 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:58.418+0000 7f3828a46640 1 -- 172.21.15.16:0/856580994 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3824108a20 con 0x7f3824105a80 2024-09-12T01:52:59.309 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:58.418+0000 7f3828a46640 1 --2- 172.21.15.16:0/856580994 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3824105a80 0x7f3824105e80 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f3814009820 tx=0x7f381402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb5b840a7d095e80 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:52:59.309 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:58.419+0000 7f38237fe640 1 -- 172.21.15.16:0/856580994 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3814037070 con 0x7f3824105a80 2024-09-12T01:52:59.309 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:58.419+0000 7f38237fe640 1 -- 172.21.15.16:0/856580994 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3814032920 con 0x7f3824105a80 2024-09-12T01:52:59.309 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:58.419+0000 7f38237fe640 1 -- 172.21.15.16:0/856580994 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3814030810 con 0x7f3824105a80 2024-09-12T01:52:59.309 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:58.419+0000 7f382acaa640 1 -- 172.21.15.16:0/856580994 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3824105a80 msgr2=0x7f3824105e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:52:59.309 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:58.419+0000 7f382acaa640 1 --2- 172.21.15.16:0/856580994 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3824105a80 0x7f3824105e80 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f3814009820 tx=0x7f381402a270 comp rx=0 tx=0).stop 2024-09-12T01:52:59.309 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:58.419+0000 7f382acaa640 1 -- 172.21.15.16:0/856580994 shutdown_connections 2024-09-12T01:52:59.309 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:58.419+0000 7f382acaa640 1 --2- 172.21.15.16:0/856580994 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3824105a80 0x7f3824105e80 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:59.309 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:58.419+0000 7f382acaa640 1 -- 172.21.15.16:0/856580994 >> 172.21.15.16:0/856580994 conn(0x7f38241015f0 msgr2=0x7f3824103a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:52:59.309 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:58.419+0000 7f382acaa640 1 -- 172.21.15.16:0/856580994 shutdown_connections 2024-09-12T01:52:59.310 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:58.419+0000 7f382acaa640 1 -- 172.21.15.16:0/856580994 wait complete. 2024-09-12T01:52:59.310 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:58.420+0000 7f382acaa640 1 Processor -- start 2024-09-12T01:52:59.310 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:58.420+0000 7f382acaa640 1 -- start start 2024-09-12T01:52:59.310 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:58.420+0000 7f382acaa640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3824105a80 0x7f382419b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:59.310 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:58.420+0000 7f382acaa640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f382410bdd0 con 0x7f3824105a80 2024-09-12T01:52:59.310 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:58.420+0000 7f3828a46640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3824105a80 0x7f382419b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:59.310 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:58.420+0000 7f3828a46640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3824105a80 0x7f382419b750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41018/0 (socket says 172.21.15.16:41018) 2024-09-12T01:52:59.310 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:58.420+0000 7f3828a46640 1 -- 172.21.15.16:0/910275135 learned_addr learned my addr 172.21.15.16:0/910275135 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:52:59.310 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:58.421+0000 7f3828a46640 1 -- 172.21.15.16:0/910275135 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f382419bc90 con 0x7f3824105a80 2024-09-12T01:52:59.310 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:58.421+0000 7f3828a46640 1 --2- 172.21.15.16:0/910275135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3824105a80 0x7f382419b750 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f3814030380 tx=0x7f3814032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:52:59.310 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:58.421+0000 7f3821ffb640 1 -- 172.21.15.16:0/910275135 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3814037040 con 0x7f3824105a80 2024-09-12T01:52:59.310 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:58.421+0000 7f3821ffb640 1 -- 172.21.15.16:0/910275135 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3814040070 con 0x7f3824105a80 2024-09-12T01:52:59.310 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:58.421+0000 7f3821ffb640 1 -- 172.21.15.16:0/910275135 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f381403a6d0 con 0x7f3824105a80 2024-09-12T01:52:59.311 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:58.421+0000 7f382acaa640 1 -- 172.21.15.16:0/910275135 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f382419bf20 con 0x7f3824105a80 2024-09-12T01:52:59.311 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:58.421+0000 7f382acaa640 1 -- 172.21.15.16:0/910275135 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f382419ec10 con 0x7f3824105a80 2024-09-12T01:52:59.311 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:58.422+0000 7f3821ffb640 1 -- 172.21.15.16:0/910275135 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f381403bb20 con 0x7f3824105a80 2024-09-12T01:52:59.311 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:58.422+0000 7f3821ffb640 1 --2- 172.21.15.16:0/910275135 >> [v2:172.21.15.16:6800/2615861334,v1:172.21.15.16:6801/2615861334] conn(0x7f380c038fc0 0x7f380c03b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:59.311 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:58.422+0000 7f3821ffb640 1 -- 172.21.15.16:0/910275135 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f381406c5b0 con 0x7f3824105a80 2024-09-12T01:52:59.311 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:58.422+0000 7f382acaa640 1 -- 172.21.15.16:0/910275135 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3824105f00 con 0x7f3824105a80 2024-09-12T01:52:59.311 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:58.422+0000 7f3823fff640 1 --2- 172.21.15.16:0/910275135 >> [v2:172.21.15.16:6800/2615861334,v1:172.21.15.16:6801/2615861334] conn(0x7f380c038fc0 0x7f380c03b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:59.312 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:58.423+0000 7f3823fff640 1 --2- 172.21.15.16:0/910275135 >> [v2:172.21.15.16:6800/2615861334,v1:172.21.15.16:6801/2615861334] conn(0x7f380c038fc0 0x7f380c03b480 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f38180098c0 tx=0x7f3818006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:52:59.312 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:58.427+0000 7f3821ffb640 1 -- 172.21.15.16:0/910275135 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3814031e60 con 0x7f3824105a80 2024-09-12T01:52:59.312 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:58.637+0000 7f382acaa640 1 -- 172.21.15.16:0/910275135 --> [v2:172.21.15.16:6800/2615861334,v1:172.21.15.16:6801/2615861334] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f3824102870 con 0x7f380c038fc0 2024-09-12T01:52:59.312 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:58.641+0000 7f3821ffb640 1 -- 172.21.15.16:0/910275135 <== mgr.14118 v2:172.21.15.16:6800/2615861334 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f3824102870 con 0x7f380c038fc0 2024-09-12T01:52:59.312 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:58.644+0000 7f382acaa640 1 -- 172.21.15.16:0/910275135 >> [v2:172.21.15.16:6800/2615861334,v1:172.21.15.16:6801/2615861334] conn(0x7f380c038fc0 msgr2=0x7f380c03b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:52:59.312 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:58.644+0000 7f382acaa640 1 --2- 172.21.15.16:0/910275135 >> [v2:172.21.15.16:6800/2615861334,v1:172.21.15.16:6801/2615861334] conn(0x7f380c038fc0 0x7f380c03b480 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f38180098c0 tx=0x7f3818006db0 comp rx=0 tx=0).stop 2024-09-12T01:52:59.312 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:58.644+0000 7f382acaa640 1 -- 172.21.15.16:0/910275135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3824105a80 msgr2=0x7f382419b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:52:59.312 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:58.644+0000 7f382acaa640 1 --2- 172.21.15.16:0/910275135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3824105a80 0x7f382419b750 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f3814030380 tx=0x7f3814032670 comp rx=0 tx=0).stop 2024-09-12T01:52:59.312 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:58.644+0000 7f382acaa640 1 -- 172.21.15.16:0/910275135 shutdown_connections 2024-09-12T01:52:59.312 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:58.644+0000 7f382acaa640 1 --2- 172.21.15.16:0/910275135 >> [v2:172.21.15.16:6800/2615861334,v1:172.21.15.16:6801/2615861334] conn(0x7f380c038fc0 0x7f380c03b480 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:59.312 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:58.644+0000 7f382acaa640 1 --2- 172.21.15.16:0/910275135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3824105a80 0x7f382419b750 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:59.313 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:58.644+0000 7f382acaa640 1 -- 172.21.15.16:0/910275135 >> 172.21.15.16:0/910275135 conn(0x7f38241015f0 msgr2=0x7f3824102160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:52:59.313 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:58.644+0000 7f382acaa640 1 -- 172.21.15.16:0/910275135 shutdown_connections 2024-09-12T01:52:59.313 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:58.644+0000 7f382acaa640 1 -- 172.21.15.16:0/910275135 wait complete. 2024-09-12T01:52:59.930 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:59 smithi016 ceph-mon[30947]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:52:59.930 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:59 smithi016 ceph-mon[30947]: Saving service mgr spec with placement count:2 2024-09-12T01:52:59.930 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:52:59 smithi016 ceph-mon[30947]: from='mgr.14118 172.21.15.16:0/2569835181' entity='mgr.x' 2024-09-12T01:53:00.632 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:59.823+0000 7f08ff63d640 1 Processor -- start 2024-09-12T01:53:00.632 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:59.823+0000 7f08ff63d640 1 -- start start 2024-09-12T01:53:00.632 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:59.823+0000 7f08ff63d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08f8105ff0 0x7f08f81063f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:00.632 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:59.823+0000 7f08ff63d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f08f807bdb0 con 0x7f08f8105ff0 2024-09-12T01:53:00.633 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:59.823+0000 7f08fd3d9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08f8105ff0 0x7f08f81063f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:00.633 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:59.823+0000 7f08fd3d9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08f8105ff0 0x7f08f81063f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41024/0 (socket says 172.21.15.16:41024) 2024-09-12T01:53:00.633 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:59.823+0000 7f08fd3d9640 1 -- 172.21.15.16:0/1004001966 learned_addr learned my addr 172.21.15.16:0/1004001966 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:53:00.633 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:59.824+0000 7f08fd3d9640 1 -- 172.21.15.16:0/1004001966 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08f8106930 con 0x7f08f8105ff0 2024-09-12T01:53:00.633 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:59.824+0000 7f08fd3d9640 1 --2- 172.21.15.16:0/1004001966 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08f8105ff0 0x7f08f81063f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f08f4009820 tx=0x7f08f402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=750f1aa51101e303 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:53:00.633 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:59.825+0000 7f08effff640 1 -- 172.21.15.16:0/1004001966 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f08f4037070 con 0x7f08f8105ff0 2024-09-12T01:53:00.633 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:59.825+0000 7f08effff640 1 -- 172.21.15.16:0/1004001966 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08f4032920 con 0x7f08f8105ff0 2024-09-12T01:53:00.633 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:59.825+0000 7f08effff640 1 -- 172.21.15.16:0/1004001966 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f08f4030810 con 0x7f08f8105ff0 2024-09-12T01:53:00.634 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:59.825+0000 7f08ff63d640 1 -- 172.21.15.16:0/1004001966 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08f8105ff0 msgr2=0x7f08f81063f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:53:00.634 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:59.825+0000 7f08ff63d640 1 --2- 172.21.15.16:0/1004001966 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08f8105ff0 0x7f08f81063f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f08f4009820 tx=0x7f08f402a270 comp rx=0 tx=0).stop 2024-09-12T01:53:00.634 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:59.825+0000 7f08ff63d640 1 -- 172.21.15.16:0/1004001966 shutdown_connections 2024-09-12T01:53:00.634 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:59.825+0000 7f08ff63d640 1 --2- 172.21.15.16:0/1004001966 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08f8105ff0 0x7f08f81063f0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:00.634 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:59.825+0000 7f08ff63d640 1 -- 172.21.15.16:0/1004001966 >> 172.21.15.16:0/1004001966 conn(0x7f08f8101e20 msgr2=0x7f08f8104240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:53:00.634 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:59.825+0000 7f08ff63d640 1 -- 172.21.15.16:0/1004001966 shutdown_connections 2024-09-12T01:53:00.634 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:59.825+0000 7f08ff63d640 1 -- 172.21.15.16:0/1004001966 wait complete. 2024-09-12T01:53:00.635 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:59.826+0000 7f08ff63d640 1 Processor -- start 2024-09-12T01:53:00.635 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:59.826+0000 7f08ff63d640 1 -- start start 2024-09-12T01:53:00.635 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:59.826+0000 7f08ff63d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08f8105ff0 0x7f08f819d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:00.635 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:59.826+0000 7f08ff63d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f08f8107a70 con 0x7f08f8105ff0 2024-09-12T01:53:00.635 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:59.826+0000 7f08fd3d9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08f8105ff0 0x7f08f819d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:00.635 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:59.826+0000 7f08fd3d9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08f8105ff0 0x7f08f819d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41040/0 (socket says 172.21.15.16:41040) 2024-09-12T01:53:00.635 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:59.826+0000 7f08fd3d9640 1 -- 172.21.15.16:0/1904371000 learned_addr learned my addr 172.21.15.16:0/1904371000 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:53:00.636 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:59.827+0000 7f08fd3d9640 1 -- 172.21.15.16:0/1904371000 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08f819de40 con 0x7f08f8105ff0 2024-09-12T01:53:00.636 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:59.827+0000 7f08fd3d9640 1 --2- 172.21.15.16:0/1904371000 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08f8105ff0 0x7f08f819d900 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f08f40097f0 tx=0x7f08f40327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:53:00.636 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:59.827+0000 7f08ee7fc640 1 -- 172.21.15.16:0/1904371000 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f08f4037040 con 0x7f08f8105ff0 2024-09-12T01:53:00.636 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:59.827+0000 7f08ee7fc640 1 -- 172.21.15.16:0/1904371000 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08f4030cc0 con 0x7f08f8105ff0 2024-09-12T01:53:00.636 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:59.827+0000 7f08ff63d640 1 -- 172.21.15.16:0/1904371000 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08f819e0d0 con 0x7f08f8105ff0 2024-09-12T01:53:00.636 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:59.827+0000 7f08ee7fc640 1 -- 172.21.15.16:0/1904371000 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f08f403a6b0 con 0x7f08f8105ff0 2024-09-12T01:53:00.636 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:59.827+0000 7f08ff63d640 1 -- 172.21.15.16:0/1904371000 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08f81a0dc0 con 0x7f08f8105ff0 2024-09-12T01:53:00.636 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:59.828+0000 7f08ee7fc640 1 -- 172.21.15.16:0/1904371000 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f08f403a850 con 0x7f08f8105ff0 2024-09-12T01:53:00.636 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:59.828+0000 7f08ff63d640 1 -- 172.21.15.16:0/1904371000 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08f81063f0 con 0x7f08f8105ff0 2024-09-12T01:53:00.637 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:59.828+0000 7f08ee7fc640 1 --2- 172.21.15.16:0/1904371000 >> [v2:172.21.15.16:6800/2615861334,v1:172.21.15.16:6801/2615861334] conn(0x7f08e4038fc0 0x7f08e403b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:00.637 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:59.828+0000 7f08ee7fc640 1 -- 172.21.15.16:0/1904371000 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f08f406c5d0 con 0x7f08f8105ff0 2024-09-12T01:53:00.637 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:59.829+0000 7f08fcbd8640 1 --2- 172.21.15.16:0/1904371000 >> [v2:172.21.15.16:6800/2615861334,v1:172.21.15.16:6801/2615861334] conn(0x7f08e4038fc0 0x7f08e403b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:00.637 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:59.829+0000 7f08fcbd8640 1 --2- 172.21.15.16:0/1904371000 >> [v2:172.21.15.16:6800/2615861334,v1:172.21.15.16:6801/2615861334] conn(0x7f08e4038fc0 0x7f08e403b480 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f08e80098c0 tx=0x7f08e8006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:53:00.637 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:52:59.833+0000 7f08ee7fc640 1 -- 172.21.15.16:0/1904371000 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f08f402cae0 con 0x7f08f8105ff0 2024-09-12T01:53:00.637 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:00.012+0000 7f08ff63d640 1 -- 172.21.15.16:0/1904371000 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f08f8196d70 con 0x7f08f8105ff0 2024-09-12T01:53:00.637 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:00.014+0000 7f08ee7fc640 1 -- 172.21.15.16:0/1904371000 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7f08f4044920 con 0x7f08f8105ff0 2024-09-12T01:53:00.637 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:00.016+0000 7f08ff63d640 1 -- 172.21.15.16:0/1904371000 >> [v2:172.21.15.16:6800/2615861334,v1:172.21.15.16:6801/2615861334] conn(0x7f08e4038fc0 msgr2=0x7f08e403b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:53:00.637 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:00.017+0000 7f08ff63d640 1 --2- 172.21.15.16:0/1904371000 >> [v2:172.21.15.16:6800/2615861334,v1:172.21.15.16:6801/2615861334] conn(0x7f08e4038fc0 0x7f08e403b480 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f08e80098c0 tx=0x7f08e8006db0 comp rx=0 tx=0).stop 2024-09-12T01:53:00.637 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:00.017+0000 7f08ff63d640 1 -- 172.21.15.16:0/1904371000 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08f8105ff0 msgr2=0x7f08f819d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:53:00.637 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:00.017+0000 7f08ff63d640 1 --2- 172.21.15.16:0/1904371000 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08f8105ff0 0x7f08f819d900 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f08f40097f0 tx=0x7f08f40327b0 comp rx=0 tx=0).stop 2024-09-12T01:53:00.637 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:00.017+0000 7f08ff63d640 1 -- 172.21.15.16:0/1904371000 shutdown_connections 2024-09-12T01:53:00.638 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:00.017+0000 7f08ff63d640 1 --2- 172.21.15.16:0/1904371000 >> [v2:172.21.15.16:6800/2615861334,v1:172.21.15.16:6801/2615861334] conn(0x7f08e4038fc0 0x7f08e403b480 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:00.638 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:00.017+0000 7f08ff63d640 1 --2- 172.21.15.16:0/1904371000 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08f8105ff0 0x7f08f819d900 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:00.638 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:00.017+0000 7f08ff63d640 1 -- 172.21.15.16:0/1904371000 >> 172.21.15.16:0/1904371000 conn(0x7f08f8101e20 msgr2=0x7f08f8191c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:53:00.638 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:00.017+0000 7f08ff63d640 1 -- 172.21.15.16:0/1904371000 shutdown_connections 2024-09-12T01:53:00.638 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:00.017+0000 7f08ff63d640 1 -- 172.21.15.16:0/1904371000 wait complete. 2024-09-12T01:53:01.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:01 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/1904371000' entity='client.admin' 2024-09-12T01:53:01.890 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:01.054+0000 7fb507fa2640 1 Processor -- start 2024-09-12T01:53:01.891 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:01.054+0000 7fb507fa2640 1 -- start start 2024-09-12T01:53:01.891 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:01.054+0000 7fb507fa2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb50007de00 0x7fb50007c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:01.891 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:01.054+0000 7fb507fa2640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb50007c830 con 0x7fb50007de00 2024-09-12T01:53:01.891 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:01.055+0000 7fb505d3e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb50007de00 0x7fb50007c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:01.891 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:01.055+0000 7fb505d3e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb50007de00 0x7fb50007c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41050/0 (socket says 172.21.15.16:41050) 2024-09-12T01:53:01.891 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:01.055+0000 7fb505d3e640 1 -- 172.21.15.16:0/3859618121 learned_addr learned my addr 172.21.15.16:0/3859618121 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:53:01.891 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:01.055+0000 7fb505d3e640 1 -- 172.21.15.16:0/3859618121 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb50007c9b0 con 0x7fb50007de00 2024-09-12T01:53:01.891 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:01.056+0000 7fb505d3e640 1 --2- 172.21.15.16:0/3859618121 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb50007de00 0x7fb50007c260 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fb4f0009e80 tx=0x7fb4f002a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af2558f2bd53537 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:53:01.891 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:01.056+0000 7fb504d3c640 1 -- 172.21.15.16:0/3859618121 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb4f0036070 con 0x7fb50007de00 2024-09-12T01:53:01.891 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:01.056+0000 7fb504d3c640 1 -- 172.21.15.16:0/3859618121 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4f00318e0 con 0x7fb50007de00 2024-09-12T01:53:01.891 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:01.056+0000 7fb504d3c640 1 -- 172.21.15.16:0/3859618121 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb4f0007a00 con 0x7fb50007de00 2024-09-12T01:53:01.891 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:01.057+0000 7fb507fa2640 1 -- 172.21.15.16:0/3859618121 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb50007de00 msgr2=0x7fb50007c260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:53:01.892 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:01.057+0000 7fb507fa2640 1 --2- 172.21.15.16:0/3859618121 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb50007de00 0x7fb50007c260 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fb4f0009e80 tx=0x7fb4f002a7a0 comp rx=0 tx=0).stop 2024-09-12T01:53:01.892 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:01.057+0000 7fb507fa2640 1 -- 172.21.15.16:0/3859618121 shutdown_connections 2024-09-12T01:53:01.892 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:01.057+0000 7fb507fa2640 1 --2- 172.21.15.16:0/3859618121 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb50007de00 0x7fb50007c260 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:01.892 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:01.057+0000 7fb507fa2640 1 -- 172.21.15.16:0/3859618121 >> 172.21.15.16:0/3859618121 conn(0x7fb500102f30 msgr2=0x7fb500105350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:53:01.892 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:01.057+0000 7fb507fa2640 1 -- 172.21.15.16:0/3859618121 shutdown_connections 2024-09-12T01:53:01.892 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:01.057+0000 7fb507fa2640 1 -- 172.21.15.16:0/3859618121 wait complete. 2024-09-12T01:53:01.892 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:01.057+0000 7fb507fa2640 1 Processor -- start 2024-09-12T01:53:01.892 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:01.057+0000 7fb507fa2640 1 -- start start 2024-09-12T01:53:01.892 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:01.058+0000 7fb507fa2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb50007de00 0x7fb50019d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:01.892 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:01.058+0000 7fb507fa2640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb50010dda0 con 0x7fb50007de00 2024-09-12T01:53:01.892 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:01.058+0000 7fb505d3e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb50007de00 0x7fb50019d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:01.892 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:01.058+0000 7fb505d3e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb50007de00 0x7fb50019d910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41062/0 (socket says 172.21.15.16:41062) 2024-09-12T01:53:01.892 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:01.058+0000 7fb505d3e640 1 -- 172.21.15.16:0/2561552344 learned_addr learned my addr 172.21.15.16:0/2561552344 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:53:01.893 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:01.058+0000 7fb505d3e640 1 -- 172.21.15.16:0/2561552344 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb50019de50 con 0x7fb50007de00 2024-09-12T01:53:01.893 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:01.058+0000 7fb505d3e640 1 --2- 172.21.15.16:0/2561552344 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb50007de00 0x7fb50019d910 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb4f002acc0 tx=0x7fb4f0031cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:53:01.893 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:01.058+0000 7fb4feffd640 1 -- 172.21.15.16:0/2561552344 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb4f0036050 con 0x7fb50007de00 2024-09-12T01:53:01.893 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:01.058+0000 7fb4feffd640 1 -- 172.21.15.16:0/2561552344 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4f0037040 con 0x7fb50007de00 2024-09-12T01:53:01.893 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:01.058+0000 7fb507fa2640 1 -- 172.21.15.16:0/2561552344 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb50019e0e0 con 0x7fb50007de00 2024-09-12T01:53:01.893 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:01.058+0000 7fb4feffd640 1 -- 172.21.15.16:0/2561552344 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb4f003d8e0 con 0x7fb50007de00 2024-09-12T01:53:01.893 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:01.059+0000 7fb507fa2640 1 -- 172.21.15.16:0/2561552344 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5001a0dd0 con 0x7fb50007de00 2024-09-12T01:53:01.893 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:01.060+0000 7fb4feffd640 1 -- 172.21.15.16:0/2561552344 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fb4f0044020 con 0x7fb50007de00 2024-09-12T01:53:01.893 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:01.060+0000 7fb507fa2640 1 -- 172.21.15.16:0/2561552344 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb50007c2e0 con 0x7fb50007de00 2024-09-12T01:53:01.893 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:01.060+0000 7fb4feffd640 1 --2- 172.21.15.16:0/2561552344 >> [v2:172.21.15.16:6800/2615861334,v1:172.21.15.16:6801/2615861334] conn(0x7fb4e8038fc0 0x7fb4e803b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:01.893 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:01.060+0000 7fb4feffd640 1 -- 172.21.15.16:0/2561552344 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fb4f0042020 con 0x7fb50007de00 2024-09-12T01:53:01.893 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:01.060+0000 7fb50553d640 1 --2- 172.21.15.16:0/2561552344 >> [v2:172.21.15.16:6800/2615861334,v1:172.21.15.16:6801/2615861334] conn(0x7fb4e8038fc0 0x7fb4e803b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:01.894 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:01.061+0000 7fb50553d640 1 --2- 172.21.15.16:0/2561552344 >> [v2:172.21.15.16:6800/2615861334,v1:172.21.15.16:6801/2615861334] conn(0x7fb4e8038fc0 0x7fb4e803b480 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fb4f40098c0 tx=0x7fb4f4006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:53:01.894 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:01.065+0000 7fb4feffd640 1 -- 172.21.15.16:0/2561552344 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb4f004b020 con 0x7fb50007de00 2024-09-12T01:53:01.894 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:01.245+0000 7fb507fa2640 1 -- 172.21.15.16:0/2561552344 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7fb5001a0fa0 con 0x7fb50007de00 2024-09-12T01:53:01.894 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:01.248+0000 7fb4feffd640 1 -- 172.21.15.16:0/2561552344 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7fb4f0046410 con 0x7fb50007de00 2024-09-12T01:53:01.894 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:01.251+0000 7fb507fa2640 1 -- 172.21.15.16:0/2561552344 >> [v2:172.21.15.16:6800/2615861334,v1:172.21.15.16:6801/2615861334] conn(0x7fb4e8038fc0 msgr2=0x7fb4e803b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:53:01.894 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:01.251+0000 7fb507fa2640 1 --2- 172.21.15.16:0/2561552344 >> [v2:172.21.15.16:6800/2615861334,v1:172.21.15.16:6801/2615861334] conn(0x7fb4e8038fc0 0x7fb4e803b480 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fb4f40098c0 tx=0x7fb4f4006db0 comp rx=0 tx=0).stop 2024-09-12T01:53:01.894 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:01.251+0000 7fb507fa2640 1 -- 172.21.15.16:0/2561552344 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb50007de00 msgr2=0x7fb50019d910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:53:01.894 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:01.251+0000 7fb507fa2640 1 --2- 172.21.15.16:0/2561552344 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb50007de00 0x7fb50019d910 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb4f002acc0 tx=0x7fb4f0031cc0 comp rx=0 tx=0).stop 2024-09-12T01:53:01.894 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:01.251+0000 7fb507fa2640 1 -- 172.21.15.16:0/2561552344 shutdown_connections 2024-09-12T01:53:01.895 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:01.251+0000 7fb507fa2640 1 --2- 172.21.15.16:0/2561552344 >> [v2:172.21.15.16:6800/2615861334,v1:172.21.15.16:6801/2615861334] conn(0x7fb4e8038fc0 0x7fb4e803b480 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:01.895 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:01.251+0000 7fb507fa2640 1 --2- 172.21.15.16:0/2561552344 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb50007de00 0x7fb50019d910 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:01.895 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:01.251+0000 7fb507fa2640 1 -- 172.21.15.16:0/2561552344 >> 172.21.15.16:0/2561552344 conn(0x7fb500102f30 msgr2=0x7fb500103aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:53:01.895 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:01.251+0000 7fb507fa2640 1 -- 172.21.15.16:0/2561552344 shutdown_connections 2024-09-12T01:53:01.895 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:01.251+0000 7fb507fa2640 1 -- 172.21.15.16:0/2561552344 wait complete. 2024-09-12T01:53:01.895 INFO:teuthology.orchestra.run.smithi016.stdout:Enabling the dashboard module... 2024-09-12T01:53:02.137 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:02 smithi016 ceph-mon[30947]: from='mgr.14118 172.21.15.16:0/2569835181' entity='mgr.x' 2024-09-12T01:53:02.138 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:02 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/2561552344' entity='client.admin' 2024-09-12T01:53:02.138 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:02 smithi016 ceph-mon[30947]: from='mgr.14118 172.21.15.16:0/2569835181' entity='mgr.x' 2024-09-12T01:53:03.132 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:03 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/1119547521' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-09-12T01:53:03.433 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 01:53:03 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-x[31173]: ignoring --setuser ceph since I am not root 2024-09-12T01:53:03.434 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 01:53:03 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-x[31173]: ignoring --setgroup ceph since I am not root 2024-09-12T01:53:03.434 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 01:53:03 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-x[31173]: 2024-09-12T01:53:03.289+0000 7f197d386040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-12T01:53:03.434 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 01:53:03 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-x[31173]: 2024-09-12T01:53:03.348+0000 7f197d386040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-12T01:53:03.878 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 01:53:03 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-x[31173]: 2024-09-12T01:53:03.504+0000 7f197d386040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-12T01:53:03.878 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 01:53:03 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-x[31173]: 2024-09-12T01:53:03.575+0000 7f197d386040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-12T01:53:03.878 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 01:53:03 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-x[31173]: 2024-09-12T01:53:03.729+0000 7f197d386040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-12T01:53:03.878 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 01:53:03 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-x[31173]: 2024-09-12T01:53:03.877+0000 7f197d386040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-12T01:53:04.195 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:02.317+0000 7f7c0ff0d640 1 Processor -- start 2024-09-12T01:53:04.195 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:02.317+0000 7f7c0ff0d640 1 -- start start 2024-09-12T01:53:04.195 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:02.317+0000 7f7c0ff0d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c08107700 0x7f7c08107b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:04.195 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:02.317+0000 7f7c0ff0d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7c081080d0 con 0x7f7c08107700 2024-09-12T01:53:04.195 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:02.317+0000 7f7c0dca9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c08107700 0x7f7c08107b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:04.195 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:02.317+0000 7f7c0dca9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c08107700 0x7f7c08107b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60974/0 (socket says 172.21.15.16:60974) 2024-09-12T01:53:04.195 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:02.317+0000 7f7c0dca9640 1 -- 172.21.15.16:0/2464181952 learned_addr learned my addr 172.21.15.16:0/2464181952 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:53:04.195 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:02.318+0000 7f7c0dca9640 1 -- 172.21.15.16:0/2464181952 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c08108900 con 0x7f7c08107700 2024-09-12T01:53:04.195 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:02.318+0000 7f7c0dca9640 1 --2- 172.21.15.16:0/2464181952 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c08107700 0x7f7c08107b00 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f7c04009820 tx=0x7f7c0402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86971e1abc1898c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:53:04.195 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:02.318+0000 7f7c0cca7640 1 -- 172.21.15.16:0/2464181952 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7c04037070 con 0x7f7c08107700 2024-09-12T01:53:04.196 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:02.318+0000 7f7c0cca7640 1 -- 172.21.15.16:0/2464181952 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c04032920 con 0x7f7c08107700 2024-09-12T01:53:04.196 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:02.318+0000 7f7c0cca7640 1 -- 172.21.15.16:0/2464181952 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7c04030810 con 0x7f7c08107700 2024-09-12T01:53:04.196 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:02.319+0000 7f7c0ff0d640 1 -- 172.21.15.16:0/2464181952 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c08107700 msgr2=0x7f7c08107b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:53:04.196 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:02.319+0000 7f7c0ff0d640 1 --2- 172.21.15.16:0/2464181952 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c08107700 0x7f7c08107b00 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f7c04009820 tx=0x7f7c0402a270 comp rx=0 tx=0).stop 2024-09-12T01:53:04.196 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:02.319+0000 7f7c0ff0d640 1 -- 172.21.15.16:0/2464181952 shutdown_connections 2024-09-12T01:53:04.196 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:02.319+0000 7f7c0ff0d640 1 --2- 172.21.15.16:0/2464181952 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c08107700 0x7f7c08107b00 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:04.196 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:02.319+0000 7f7c0ff0d640 1 -- 172.21.15.16:0/2464181952 >> 172.21.15.16:0/2464181952 conn(0x7f7c08102f30 msgr2=0x7f7c08105350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:53:04.196 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:02.319+0000 7f7c0ff0d640 1 -- 172.21.15.16:0/2464181952 shutdown_connections 2024-09-12T01:53:04.196 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:02.319+0000 7f7c0ff0d640 1 -- 172.21.15.16:0/2464181952 wait complete. 2024-09-12T01:53:04.196 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:02.320+0000 7f7c0ff0d640 1 Processor -- start 2024-09-12T01:53:04.196 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:02.320+0000 7f7c0ff0d640 1 -- start start 2024-09-12T01:53:04.196 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:02.320+0000 7f7c0ff0d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c08107700 0x7f7c0819b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:04.196 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:02.320+0000 7f7c0ff0d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7c0810bcb0 con 0x7f7c08107700 2024-09-12T01:53:04.197 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:02.320+0000 7f7c0dca9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c08107700 0x7f7c0819b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:04.197 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:02.320+0000 7f7c0dca9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c08107700 0x7f7c0819b700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60976/0 (socket says 172.21.15.16:60976) 2024-09-12T01:53:04.197 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:02.320+0000 7f7c0dca9640 1 -- 172.21.15.16:0/1119547521 learned_addr learned my addr 172.21.15.16:0/1119547521 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:53:04.197 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:02.320+0000 7f7c0dca9640 1 -- 172.21.15.16:0/1119547521 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c0819bc40 con 0x7f7c08107700 2024-09-12T01:53:04.197 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:02.320+0000 7f7c0dca9640 1 --2- 172.21.15.16:0/1119547521 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c08107700 0x7f7c0819b700 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7c040097f0 tx=0x7f7c040327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:53:04.197 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:02.321+0000 7f7bfeffd640 1 -- 172.21.15.16:0/1119547521 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7c04037040 con 0x7f7c08107700 2024-09-12T01:53:04.197 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:02.321+0000 7f7bfeffd640 1 -- 172.21.15.16:0/1119547521 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c04030cc0 con 0x7f7c08107700 2024-09-12T01:53:04.197 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:02.321+0000 7f7bfeffd640 1 -- 172.21.15.16:0/1119547521 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7c0403a640 con 0x7f7c08107700 2024-09-12T01:53:04.197 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:02.321+0000 7f7c0ff0d640 1 -- 172.21.15.16:0/1119547521 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c0819bed0 con 0x7f7c08107700 2024-09-12T01:53:04.197 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:02.321+0000 7f7c0ff0d640 1 -- 172.21.15.16:0/1119547521 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c0819ebc0 con 0x7f7c08107700 2024-09-12T01:53:04.197 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:02.322+0000 7f7bfeffd640 1 -- 172.21.15.16:0/1119547521 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f7c0403b3f0 con 0x7f7c08107700 2024-09-12T01:53:04.197 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:02.322+0000 7f7c0ff0d640 1 -- 172.21.15.16:0/1119547521 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c0810be30 con 0x7f7c08107700 2024-09-12T01:53:04.197 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:02.322+0000 7f7bfeffd640 1 --2- 172.21.15.16:0/1119547521 >> [v2:172.21.15.16:6800/2615861334,v1:172.21.15.16:6801/2615861334] conn(0x7f7bf4038fc0 0x7f7bf403b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:04.197 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:02.322+0000 7f7bfeffd640 1 -- 172.21.15.16:0/1119547521 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f7c0406c4b0 con 0x7f7c08107700 2024-09-12T01:53:04.198 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:02.322+0000 7f7c0d4a8640 1 --2- 172.21.15.16:0/1119547521 >> [v2:172.21.15.16:6800/2615861334,v1:172.21.15.16:6801/2615861334] conn(0x7f7bf4038fc0 0x7f7bf403b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:04.198 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:02.322+0000 7f7c0d4a8640 1 --2- 172.21.15.16:0/1119547521 >> [v2:172.21.15.16:6800/2615861334,v1:172.21.15.16:6801/2615861334] conn(0x7f7bf4038fc0 0x7f7bf403b480 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f7bf80098c0 tx=0x7f7bf8006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:53:04.198 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:02.327+0000 7f7bfeffd640 1 -- 172.21.15.16:0/1119547521 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7c04042e10 con 0x7f7c08107700 2024-09-12T01:53:04.198 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:02.572+0000 7f7c0ff0d640 1 -- 172.21.15.16:0/1119547521 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7f7c0819ed90 con 0x7f7c08107700 2024-09-12T01:53:04.198 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:03.131+0000 7f7bfeffd640 1 -- 172.21.15.16:0/1119547521 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7f7c040328c0 con 0x7f7c08107700 2024-09-12T01:53:04.198 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:03.135+0000 7f7bfeffd640 1 -- 172.21.15.16:0/1119547521 <== mon.0 v2:172.21.15.16:3300/0 8 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7f7c04049ca0 con 0x7f7c08107700 2024-09-12T01:53:04.198 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:03.135+0000 7f7c0ff0d640 1 -- 172.21.15.16:0/1119547521 >> [v2:172.21.15.16:6800/2615861334,v1:172.21.15.16:6801/2615861334] conn(0x7f7bf4038fc0 msgr2=0x7f7bf403b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:53:04.198 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:03.135+0000 7f7c0ff0d640 1 --2- 172.21.15.16:0/1119547521 >> [v2:172.21.15.16:6800/2615861334,v1:172.21.15.16:6801/2615861334] conn(0x7f7bf4038fc0 0x7f7bf403b480 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f7bf80098c0 tx=0x7f7bf8006db0 comp rx=0 tx=0).stop 2024-09-12T01:53:04.198 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:03.135+0000 7f7c0ff0d640 1 -- 172.21.15.16:0/1119547521 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c08107700 msgr2=0x7f7c0819b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:53:04.198 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:03.135+0000 7f7c0ff0d640 1 --2- 172.21.15.16:0/1119547521 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c08107700 0x7f7c0819b700 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7c040097f0 tx=0x7f7c040327b0 comp rx=0 tx=0).stop 2024-09-12T01:53:04.198 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:03.136+0000 7f7c0ff0d640 1 -- 172.21.15.16:0/1119547521 shutdown_connections 2024-09-12T01:53:04.198 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:03.136+0000 7f7c0ff0d640 1 --2- 172.21.15.16:0/1119547521 >> [v2:172.21.15.16:6800/2615861334,v1:172.21.15.16:6801/2615861334] conn(0x7f7bf4038fc0 0x7f7bf403b480 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:04.198 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:03.136+0000 7f7c0ff0d640 1 --2- 172.21.15.16:0/1119547521 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c08107700 0x7f7c0819b700 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:04.198 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:03.136+0000 7f7c0ff0d640 1 -- 172.21.15.16:0/1119547521 >> 172.21.15.16:0/1119547521 conn(0x7f7c08102f30 msgr2=0x7f7c08103aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:53:04.199 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:03.136+0000 7f7c0ff0d640 1 -- 172.21.15.16:0/1119547521 shutdown_connections 2024-09-12T01:53:04.199 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:03.136+0000 7f7c0ff0d640 1 -- 172.21.15.16:0/1119547521 wait complete. 2024-09-12T01:53:04.403 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:04 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/1119547521' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-09-12T01:53:04.403 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:04 smithi016 ceph-mon[30947]: mgrmap e9: x(active, since 18s) 2024-09-12T01:53:04.403 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 01:53:04 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-x[31173]: 2024-09-12T01:53:04.207+0000 7f197d386040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-12T01:53:04.403 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 01:53:04 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-x[31173]: 2024-09-12T01:53:04.272+0000 7f197d386040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-12T01:53:04.683 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 01:53:04 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-x[31173]: 2024-09-12T01:53:04.469+0000 7f197d386040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-12T01:53:05.402 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:05 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/2382576445' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-12T01:53:05.410 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout { 2024-09-12T01:53:05.410 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-09-12T01:53:05.410 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-12T01:53:05.410 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-12T01:53:05.410 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-12T01:53:05.410 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout } 2024-09-12T01:53:05.410 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:04.587+0000 7fb0a9d51640 1 Processor -- start 2024-09-12T01:53:05.410 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:04.587+0000 7fb0a9d51640 1 -- start start 2024-09-12T01:53:05.410 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:04.587+0000 7fb0a9d51640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0a4108140 0x7fb0a410a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:05.411 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:04.587+0000 7fb0a9d51640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb0a407be30 con 0x7fb0a4108140 2024-09-12T01:53:05.411 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:04.588+0000 7fb0a37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0a4108140 0x7fb0a410a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:05.411 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:04.588+0000 7fb0a37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0a4108140 0x7fb0a410a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60990/0 (socket says 172.21.15.16:60990) 2024-09-12T01:53:05.411 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:04.588+0000 7fb0a37fe640 1 -- 172.21.15.16:0/76695706 learned_addr learned my addr 172.21.15.16:0/76695706 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:53:05.411 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:04.588+0000 7fb0a37fe640 1 -- 172.21.15.16:0/76695706 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0a410aa70 con 0x7fb0a4108140 2024-09-12T01:53:05.411 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:04.589+0000 7fb0a37fe640 1 --2- 172.21.15.16:0/76695706 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0a4108140 0x7fb0a410a530 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb094009e80 tx=0x7fb09402a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=754bcd8b32198d63 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:53:05.411 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:04.589+0000 7fb0a27fc640 1 -- 172.21.15.16:0/76695706 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb094036070 con 0x7fb0a4108140 2024-09-12T01:53:05.411 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:04.589+0000 7fb0a27fc640 1 -- 172.21.15.16:0/76695706 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0940318e0 con 0x7fb0a4108140 2024-09-12T01:53:05.411 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:04.589+0000 7fb0a27fc640 1 -- 172.21.15.16:0/76695706 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb094007a00 con 0x7fb0a4108140 2024-09-12T01:53:05.411 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:04.590+0000 7fb0a9d51640 1 -- 172.21.15.16:0/76695706 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0a4108140 msgr2=0x7fb0a410a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:53:05.411 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:04.590+0000 7fb0a9d51640 1 --2- 172.21.15.16:0/76695706 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0a4108140 0x7fb0a410a530 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb094009e80 tx=0x7fb09402a7a0 comp rx=0 tx=0).stop 2024-09-12T01:53:05.411 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:04.590+0000 7fb0a9d51640 1 -- 172.21.15.16:0/76695706 shutdown_connections 2024-09-12T01:53:05.411 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:04.590+0000 7fb0a9d51640 1 --2- 172.21.15.16:0/76695706 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0a4108140 0x7fb0a410a530 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:05.411 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:04.590+0000 7fb0a9d51640 1 -- 172.21.15.16:0/76695706 >> 172.21.15.16:0/76695706 conn(0x7fb0a4101e20 msgr2=0x7fb0a4104240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:53:05.412 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:04.590+0000 7fb0a9d51640 1 -- 172.21.15.16:0/76695706 shutdown_connections 2024-09-12T01:53:05.412 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:04.590+0000 7fb0a9d51640 1 -- 172.21.15.16:0/76695706 wait complete. 2024-09-12T01:53:05.412 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:04.590+0000 7fb0a9d51640 1 Processor -- start 2024-09-12T01:53:05.412 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:04.591+0000 7fb0a9d51640 1 -- start start 2024-09-12T01:53:05.412 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:04.591+0000 7fb0a9d51640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0a4108140 0x7fb0a41079c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:05.412 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:04.591+0000 7fb0a9d51640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb0a410bb50 con 0x7fb0a4108140 2024-09-12T01:53:05.412 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:04.591+0000 7fb0a37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0a4108140 0x7fb0a41079c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:05.412 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:04.591+0000 7fb0a37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0a4108140 0x7fb0a41079c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32768/0 (socket says 172.21.15.16:32768) 2024-09-12T01:53:05.412 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:04.591+0000 7fb0a37fe640 1 -- 172.21.15.16:0/2382576445 learned_addr learned my addr 172.21.15.16:0/2382576445 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:53:05.412 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:04.591+0000 7fb0a37fe640 1 -- 172.21.15.16:0/2382576445 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0a4107f00 con 0x7fb0a4108140 2024-09-12T01:53:05.412 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:04.591+0000 7fb0a37fe640 1 --2- 172.21.15.16:0/2382576445 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0a4108140 0x7fb0a41079c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb094031cf0 tx=0x7fb094031d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:53:05.412 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:04.592+0000 7fb0a0ff9640 1 -- 172.21.15.16:0/2382576445 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb094036050 con 0x7fb0a4108140 2024-09-12T01:53:05.412 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:04.592+0000 7fb0a0ff9640 1 -- 172.21.15.16:0/2382576445 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb094042070 con 0x7fb0a4108140 2024-09-12T01:53:05.413 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:04.592+0000 7fb0a9d51640 1 -- 172.21.15.16:0/2382576445 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0a41060f0 con 0x7fb0a4108140 2024-09-12T01:53:05.413 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:04.592+0000 7fb0a0ff9640 1 -- 172.21.15.16:0/2382576445 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb09403d9b0 con 0x7fb0a4108140 2024-09-12T01:53:05.413 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:04.592+0000 7fb0a9d51640 1 -- 172.21.15.16:0/2382576445 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0a4106630 con 0x7fb0a4108140 2024-09-12T01:53:05.413 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:04.593+0000 7fb0a0ff9640 1 -- 172.21.15.16:0/2382576445 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7fb09403db50 con 0x7fb0a4108140 2024-09-12T01:53:05.413 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:04.593+0000 7fb0a9d51640 1 -- 172.21.15.16:0/2382576445 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0a41085a0 con 0x7fb0a4108140 2024-09-12T01:53:05.413 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:04.593+0000 7fb0a0ff9640 1 --2- 172.21.15.16:0/2382576445 >> [v2:172.21.15.16:6800/2615861334,v1:172.21.15.16:6801/2615861334] conn(0x7fb08c039020 0x7fb08c03b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:05.413 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:04.593+0000 7fb0a0ff9640 1 -- 172.21.15.16:0/2382576445 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fb09406d370 con 0x7fb0a4108140 2024-09-12T01:53:05.413 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:04.593+0000 7fb0a2ffd640 1 -- 172.21.15.16:0/2382576445 >> [v2:172.21.15.16:6800/2615861334,v1:172.21.15.16:6801/2615861334] conn(0x7fb08c039020 msgr2=0x7fb08c03b4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/2615861334 2024-09-12T01:53:05.413 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:04.593+0000 7fb0a2ffd640 1 --2- 172.21.15.16:0/2382576445 >> [v2:172.21.15.16:6800/2615861334,v1:172.21.15.16:6801/2615861334] conn(0x7fb08c039020 0x7fb08c03b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-12T01:53:05.413 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:04.598+0000 7fb0a0ff9640 1 -- 172.21.15.16:0/2382576445 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb094044310 con 0x7fb0a4108140 2024-09-12T01:53:05.413 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:04.794+0000 7fb0a2ffd640 1 -- 172.21.15.16:0/2382576445 >> [v2:172.21.15.16:6800/2615861334,v1:172.21.15.16:6801/2615861334] conn(0x7fb08c039020 msgr2=0x7fb08c03b4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/2615861334 2024-09-12T01:53:05.413 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:04.794+0000 7fb0a2ffd640 1 --2- 172.21.15.16:0/2382576445 >> [v2:172.21.15.16:6800/2615861334,v1:172.21.15.16:6801/2615861334] conn(0x7fb08c039020 0x7fb08c03b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-12T01:53:05.413 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:04.820+0000 7fb0a9d51640 1 -- 172.21.15.16:0/2382576445 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fb0a41068b0 con 0x7fb0a4108140 2024-09-12T01:53:05.414 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:04.820+0000 7fb0a0ff9640 1 -- 172.21.15.16:0/2382576445 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7fb094045520 con 0x7fb0a4108140 2024-09-12T01:53:05.414 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:04.823+0000 7fb0a9d51640 1 -- 172.21.15.16:0/2382576445 >> [v2:172.21.15.16:6800/2615861334,v1:172.21.15.16:6801/2615861334] conn(0x7fb08c039020 msgr2=0x7fb08c03b4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T01:53:05.414 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:04.823+0000 7fb0a9d51640 1 --2- 172.21.15.16:0/2382576445 >> [v2:172.21.15.16:6800/2615861334,v1:172.21.15.16:6801/2615861334] conn(0x7fb08c039020 0x7fb08c03b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:05.414 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:04.823+0000 7fb0a9d51640 1 -- 172.21.15.16:0/2382576445 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0a4108140 msgr2=0x7fb0a41079c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:53:05.414 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:04.823+0000 7fb0a9d51640 1 --2- 172.21.15.16:0/2382576445 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0a4108140 0x7fb0a41079c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb094031cf0 tx=0x7fb094031d20 comp rx=0 tx=0).stop 2024-09-12T01:53:05.414 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:04.823+0000 7fb0a9d51640 1 -- 172.21.15.16:0/2382576445 shutdown_connections 2024-09-12T01:53:05.414 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:04.823+0000 7fb0a9d51640 1 --2- 172.21.15.16:0/2382576445 >> [v2:172.21.15.16:6800/2615861334,v1:172.21.15.16:6801/2615861334] conn(0x7fb08c039020 0x7fb08c03b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:05.414 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:04.823+0000 7fb0a9d51640 1 --2- 172.21.15.16:0/2382576445 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0a4108140 0x7fb0a41079c0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:05.414 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:04.823+0000 7fb0a9d51640 1 -- 172.21.15.16:0/2382576445 >> 172.21.15.16:0/2382576445 conn(0x7fb0a4101e20 msgr2=0x7fb0a41029f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:53:05.414 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:04.823+0000 7fb0a9d51640 1 -- 172.21.15.16:0/2382576445 shutdown_connections 2024-09-12T01:53:05.414 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:04.823+0000 7fb0a9d51640 1 -- 172.21.15.16:0/2382576445 wait complete. 2024-09-12T01:53:05.414 INFO:teuthology.orchestra.run.smithi016.stdout:Waiting for the mgr to restart... 2024-09-12T01:53:05.414 INFO:teuthology.orchestra.run.smithi016.stdout:Waiting for mgr epoch 9... 2024-09-12T01:53:05.683 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 01:53:05 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-x[31173]: 2024-09-12T01:53:05.407+0000 7f197d386040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-12T01:53:05.683 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 01:53:05 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-x[31173]: 2024-09-12T01:53:05.480+0000 7f197d386040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-12T01:53:06.301 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 01:53:06 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-x[31173]: 2024-09-12T01:53:06.242+0000 7f197d386040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-12T01:53:06.583 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 01:53:06 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-x[31173]: 2024-09-12T01:53:06.366+0000 7f197d386040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-12T01:53:06.584 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 01:53:06 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-x[31173]: 2024-09-12T01:53:06.518+0000 7f197d386040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-12T01:53:06.584 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 01:53:06 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-x[31173]: 2024-09-12T01:53:06.583+0000 7f197d386040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-12T01:53:06.876 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 01:53:06 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-x[31173]: 2024-09-12T01:53:06.741+0000 7f197d386040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-12T01:53:06.877 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 01:53:06 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-x[31173]: 2024-09-12T01:53:06.813+0000 7f197d386040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-12T01:53:06.877 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 01:53:06 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-x[31173]: 2024-09-12T01:53:06.876+0000 7f197d386040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-12T01:53:07.180 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 01:53:07 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-x[31173]: 2024-09-12T01:53:07.042+0000 7f197d386040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-12T01:53:07.181 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 01:53:07 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-x[31173]: 2024-09-12T01:53:07.111+0000 7f197d386040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-12T01:53:07.181 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 01:53:07 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-x[31173]: 2024-09-12T01:53:07.180+0000 7f197d386040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-12T01:53:07.433 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 01:53:07 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-x[31173]: 2024-09-12T01:53:07.267+0000 7f197d386040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-12T01:53:07.434 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 01:53:07 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-x[31173]: 2024-09-12T01:53:07.330+0000 7f197d386040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-12T01:53:07.434 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 01:53:07 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-x[31173]: 2024-09-12T01:53:07.392+0000 7f197d386040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-12T01:53:08.475 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 01:53:08 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-x[31173]: 2024-09-12T01:53:08.216+0000 7f197d386040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-12T01:53:08.475 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:08 smithi016 ceph-mon[30947]: Active manager daemon x restarted 2024-09-12T01:53:08.476 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:08 smithi016 ceph-mon[30947]: Activating manager daemon x 2024-09-12T01:53:08.476 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:08 smithi016 ceph-mon[30947]: osdmap e3: 0 total, 0 up, 0 in 2024-09-12T01:53:08.476 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:08 smithi016 ceph-mon[30947]: mgrmap e10: x(active, starting, since 0.00336036s) 2024-09-12T01:53:08.476 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:08 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T01:53:08.476 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:08 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-12T01:53:08.476 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:08 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-12T01:53:08.477 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:08 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-12T01:53:08.477 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:08 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-12T01:53:08.477 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:08 smithi016 ceph-mon[30947]: Manager daemon x is now available 2024-09-12T01:53:08.477 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:08 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:53:08.477 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:08 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-12T01:53:08.477 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:08 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-12T01:53:10.163 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout { 2024-09-12T01:53:10.163 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-09-12T01:53:10.163 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-12T01:53:10.164 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout } 2024-09-12T01:53:10.164 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:05.810+0000 7f800afd0640 1 Processor -- start 2024-09-12T01:53:10.164 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:05.810+0000 7f800afd0640 1 -- start start 2024-09-12T01:53:10.164 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:05.810+0000 7f800afd0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8004108150 0x7f800410a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:10.164 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:05.810+0000 7f800afd0640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f800407be30 con 0x7f8004108150 2024-09-12T01:53:10.164 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:05.811+0000 7f8008d6c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8004108150 0x7f800410a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:10.164 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:05.811+0000 7f8008d6c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8004108150 0x7f800410a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32784/0 (socket says 172.21.15.16:32784) 2024-09-12T01:53:10.164 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:05.811+0000 7f8008d6c640 1 -- 172.21.15.16:0/2326485376 learned_addr learned my addr 172.21.15.16:0/2326485376 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:53:10.164 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:05.811+0000 7f8008d6c640 1 -- 172.21.15.16:0/2326485376 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f800410aa80 con 0x7f8004108150 2024-09-12T01:53:10.164 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:05.812+0000 7f8008d6c640 1 --2- 172.21.15.16:0/2326485376 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8004108150 0x7f800410a540 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f7ff8009820 tx=0x7f7ff802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a57077585fa9b51 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:53:10.164 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:05.812+0000 7f80037fe640 1 -- 172.21.15.16:0/2326485376 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7ff8037070 con 0x7f8004108150 2024-09-12T01:53:10.165 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:05.812+0000 7f80037fe640 1 -- 172.21.15.16:0/2326485376 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ff8032920 con 0x7f8004108150 2024-09-12T01:53:10.165 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:05.812+0000 7f80037fe640 1 -- 172.21.15.16:0/2326485376 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7ff8030810 con 0x7f8004108150 2024-09-12T01:53:10.165 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:05.812+0000 7f800afd0640 1 -- 172.21.15.16:0/2326485376 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8004108150 msgr2=0x7f800410a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:53:10.165 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:05.812+0000 7f800afd0640 1 --2- 172.21.15.16:0/2326485376 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8004108150 0x7f800410a540 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f7ff8009820 tx=0x7f7ff802a270 comp rx=0 tx=0).stop 2024-09-12T01:53:10.165 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:05.813+0000 7f800afd0640 1 -- 172.21.15.16:0/2326485376 shutdown_connections 2024-09-12T01:53:10.165 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:05.813+0000 7f800afd0640 1 --2- 172.21.15.16:0/2326485376 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8004108150 0x7f800410a540 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:10.165 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:05.813+0000 7f800afd0640 1 -- 172.21.15.16:0/2326485376 >> 172.21.15.16:0/2326485376 conn(0x7f8004101d90 msgr2=0x7f80041041b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:53:10.165 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:05.813+0000 7f800afd0640 1 -- 172.21.15.16:0/2326485376 shutdown_connections 2024-09-12T01:53:10.165 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:05.813+0000 7f800afd0640 1 -- 172.21.15.16:0/2326485376 wait complete. 2024-09-12T01:53:10.165 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:05.813+0000 7f800afd0640 1 Processor -- start 2024-09-12T01:53:10.165 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:05.813+0000 7f800afd0640 1 -- start start 2024-09-12T01:53:10.165 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:05.814+0000 7f800afd0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8004108150 0x7f800419b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:10.165 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:05.814+0000 7f800afd0640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f800410bb60 con 0x7f8004108150 2024-09-12T01:53:10.166 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:05.814+0000 7f8008d6c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8004108150 0x7f800419b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:10.166 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:05.814+0000 7f8008d6c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8004108150 0x7f800419b700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32794/0 (socket says 172.21.15.16:32794) 2024-09-12T01:53:10.166 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:05.814+0000 7f8008d6c640 1 -- 172.21.15.16:0/554319074 learned_addr learned my addr 172.21.15.16:0/554319074 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:53:10.166 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:05.814+0000 7f8008d6c640 1 -- 172.21.15.16:0/554319074 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f800419bc40 con 0x7f8004108150 2024-09-12T01:53:10.166 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:05.814+0000 7f8008d6c640 1 --2- 172.21.15.16:0/554319074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8004108150 0x7f800419b700 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7ff80097f0 tx=0x7f7ff80327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:53:10.166 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:05.814+0000 7f8001ffb640 1 -- 172.21.15.16:0/554319074 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7ff8037040 con 0x7f8004108150 2024-09-12T01:53:10.166 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:05.814+0000 7f8001ffb640 1 -- 172.21.15.16:0/554319074 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ff8030cc0 con 0x7f8004108150 2024-09-12T01:53:10.166 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:05.814+0000 7f800afd0640 1 -- 172.21.15.16:0/554319074 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f800419bed0 con 0x7f8004108150 2024-09-12T01:53:10.166 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:05.814+0000 7f8001ffb640 1 -- 172.21.15.16:0/554319074 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7ff803a6b0 con 0x7f8004108150 2024-09-12T01:53:10.166 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:05.815+0000 7f800afd0640 1 -- 172.21.15.16:0/554319074 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f800419ebc0 con 0x7f8004108150 2024-09-12T01:53:10.166 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:05.815+0000 7f8001ffb640 1 -- 172.21.15.16:0/554319074 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7f7ff803a850 con 0x7f8004108150 2024-09-12T01:53:10.166 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:05.816+0000 7f8001ffb640 1 --2- 172.21.15.16:0/554319074 >> [v2:172.21.15.16:6800/2615861334,v1:172.21.15.16:6801/2615861334] conn(0x7f7ff4039020 0x7f7ff403b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:10.166 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:05.816+0000 7f800afd0640 1 -- 172.21.15.16:0/554319074 --> [v2:172.21.15.16:6800/2615861334,v1:172.21.15.16:6801/2615861334] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f80041085b0 con 0x7f7ff4039020 2024-09-12T01:53:10.167 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:05.816+0000 7f8001ffb640 1 -- 172.21.15.16:0/554319074 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f7ff806d560 con 0x7f8004108150 2024-09-12T01:53:10.167 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:05.816+0000 7f8003fff640 1 -- 172.21.15.16:0/554319074 >> [v2:172.21.15.16:6800/2615861334,v1:172.21.15.16:6801/2615861334] conn(0x7f7ff4039020 msgr2=0x7f7ff403b4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/2615861334 2024-09-12T01:53:10.167 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:05.816+0000 7f8003fff640 1 --2- 172.21.15.16:0/554319074 >> [v2:172.21.15.16:6800/2615861334,v1:172.21.15.16:6801/2615861334] conn(0x7f7ff4039020 0x7f7ff403b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-12T01:53:10.167 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:06.017+0000 7f8003fff640 1 -- 172.21.15.16:0/554319074 >> [v2:172.21.15.16:6800/2615861334,v1:172.21.15.16:6801/2615861334] conn(0x7f7ff4039020 msgr2=0x7f7ff403b4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/2615861334 2024-09-12T01:53:10.167 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:06.017+0000 7f8003fff640 1 --2- 172.21.15.16:0/554319074 >> [v2:172.21.15.16:6800/2615861334,v1:172.21.15.16:6801/2615861334] conn(0x7f7ff4039020 0x7f7ff403b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-12T01:53:10.167 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:06.418+0000 7f8003fff640 1 -- 172.21.15.16:0/554319074 >> [v2:172.21.15.16:6800/2615861334,v1:172.21.15.16:6801/2615861334] conn(0x7f7ff4039020 msgr2=0x7f7ff403b4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/2615861334 2024-09-12T01:53:10.167 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:06.418+0000 7f8003fff640 1 --2- 172.21.15.16:0/554319074 >> [v2:172.21.15.16:6800/2615861334,v1:172.21.15.16:6801/2615861334] conn(0x7f7ff4039020 0x7f7ff403b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-12T01:53:10.167 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:07.219+0000 7f8003fff640 1 -- 172.21.15.16:0/554319074 >> [v2:172.21.15.16:6800/2615861334,v1:172.21.15.16:6801/2615861334] conn(0x7f7ff4039020 msgr2=0x7f7ff403b4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/2615861334 2024-09-12T01:53:10.167 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:07.219+0000 7f8003fff640 1 --2- 172.21.15.16:0/554319074 >> [v2:172.21.15.16:6800/2615861334,v1:172.21.15.16:6801/2615861334] conn(0x7f7ff4039020 0x7f7ff403b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-12T01:53:10.167 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:08.219+0000 7f8001ffb640 1 -- 172.21.15.16:0/554319074 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mgrmap(e 10) ==== 46067+0+0 (secure 0 0 0) 0x7f7ff806c010 con 0x7f8004108150 2024-09-12T01:53:10.167 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:08.220+0000 7f8001ffb640 1 -- 172.21.15.16:0/554319074 >> [v2:172.21.15.16:6800/2615861334,v1:172.21.15.16:6801/2615861334] conn(0x7f7ff4039020 msgr2=0x7f7ff403b4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T01:53:10.167 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:08.220+0000 7f8001ffb640 1 --2- 172.21.15.16:0/554319074 >> [v2:172.21.15.16:6800/2615861334,v1:172.21.15.16:6801/2615861334] conn(0x7f7ff4039020 0x7f7ff403b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:10.168 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:09.221+0000 7f8001ffb640 1 -- 172.21.15.16:0/554319074 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mgrmap(e 11) ==== 46194+0+0 (secure 0 0 0) 0x7f7ff806c300 con 0x7f8004108150 2024-09-12T01:53:10.168 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:09.221+0000 7f8001ffb640 1 --2- 172.21.15.16:0/554319074 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f7ff403c8f0 0x7f7ff403ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:10.168 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:09.222+0000 7f8001ffb640 1 -- 172.21.15.16:0/554319074 --> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f7ff8031cb0 con 0x7f7ff403c8f0 2024-09-12T01:53:10.168 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:09.222+0000 7f8003fff640 1 --2- 172.21.15.16:0/554319074 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f7ff403c8f0 0x7f7ff403ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:10.168 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:09.223+0000 7f8003fff640 1 --2- 172.21.15.16:0/554319074 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f7ff403c8f0 0x7f7ff403ece0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f7ff0003980 tx=0x7f7ff00092b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:53:10.168 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:09.223+0000 7f8001ffb640 1 -- 172.21.15.16:0/554319074 <== mgr.14150 v2:172.21.15.16:6800/3887444820 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f7ff8031cb0 con 0x7f7ff403c8f0 2024-09-12T01:53:10.168 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:09.230+0000 7f800afd0640 1 -- 172.21.15.16:0/554319074 --> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f8004103080 con 0x7f7ff403c8f0 2024-09-12T01:53:10.168 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:09.230+0000 7f8001ffb640 1 -- 172.21.15.16:0/554319074 <== mgr.14150 v2:172.21.15.16:6800/3887444820 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7f8004103080 con 0x7f7ff403c8f0 2024-09-12T01:53:10.168 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:09.231+0000 7f800afd0640 1 -- 172.21.15.16:0/554319074 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f7ff403c8f0 msgr2=0x7f7ff403ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:53:10.168 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:09.231+0000 7f800afd0640 1 --2- 172.21.15.16:0/554319074 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f7ff403c8f0 0x7f7ff403ece0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f7ff0003980 tx=0x7f7ff00092b0 comp rx=0 tx=0).stop 2024-09-12T01:53:10.168 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:09.231+0000 7f800afd0640 1 -- 172.21.15.16:0/554319074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8004108150 msgr2=0x7f800419b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:53:10.168 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:09.231+0000 7f800afd0640 1 --2- 172.21.15.16:0/554319074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8004108150 0x7f800419b700 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7ff80097f0 tx=0x7f7ff80327b0 comp rx=0 tx=0).stop 2024-09-12T01:53:10.168 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:09.231+0000 7f800afd0640 1 -- 172.21.15.16:0/554319074 shutdown_connections 2024-09-12T01:53:10.169 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:09.231+0000 7f800afd0640 1 --2- 172.21.15.16:0/554319074 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f7ff403c8f0 0x7f7ff403ece0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:10.169 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:09.231+0000 7f800afd0640 1 --2- 172.21.15.16:0/554319074 >> [v2:172.21.15.16:6800/2615861334,v1:172.21.15.16:6801/2615861334] conn(0x7f7ff4039020 0x7f7ff403b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:10.169 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:09.231+0000 7f800afd0640 1 --2- 172.21.15.16:0/554319074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8004108150 0x7f800419b700 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:10.169 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:09.231+0000 7f800afd0640 1 -- 172.21.15.16:0/554319074 >> 172.21.15.16:0/554319074 conn(0x7f8004101d90 msgr2=0x7f8004102970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:53:10.169 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:09.231+0000 7f800afd0640 1 -- 172.21.15.16:0/554319074 shutdown_connections 2024-09-12T01:53:10.169 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:09.231+0000 7f800afd0640 1 -- 172.21.15.16:0/554319074 wait complete. 2024-09-12T01:53:10.169 INFO:teuthology.orchestra.run.smithi016.stdout:mgr epoch 9 is available 2024-09-12T01:53:10.169 INFO:teuthology.orchestra.run.smithi016.stdout:Generating a dashboard self-signed certificate... 2024-09-12T01:53:10.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:10 smithi016 ceph-mon[30947]: mgrmap e11: x(active, since 1.00606s) 2024-09-12T01:53:10.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:10 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:53:10.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:10 smithi016 ceph-mon[30947]: [12/Sep/2024:01:53:10] ENGINE Bus STARTING 2024-09-12T01:53:10.434 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 01:53:10 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-x[31173]: 2024-09-12T01:53:10.304+0000 7f18e869a640 -1 log_channel(cephadm) log [ERR] : [12/Sep/2024:01:53:10] ENGINE Error in HTTPServer.serve 2024-09-12T01:53:10.434 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 01:53:10 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-x[31173]: Traceback (most recent call last): 2024-09-12T01:53:10.434 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 01:53:10 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-x[31173]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-12T01:53:10.434 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 01:53:10 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-x[31173]: self._connections.run(self.expiration_interval) 2024-09-12T01:53:10.434 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 01:53:10 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-x[31173]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-12T01:53:10.434 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 01:53:10 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-x[31173]: self._run(expiration_interval) 2024-09-12T01:53:10.434 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 01:53:10 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-x[31173]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-12T01:53:10.434 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 01:53:10 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-x[31173]: new_conn = self._from_server_socket(self.server.socket) 2024-09-12T01:53:10.434 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 01:53:10 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-x[31173]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-12T01:53:10.435 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 01:53:10 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-x[31173]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-12T01:53:10.435 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 01:53:10 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-x[31173]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-12T01:53:10.435 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 01:53:10 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-x[31173]: s = self.context.wrap_socket( 2024-09-12T01:53:10.435 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 01:53:10 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-x[31173]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-12T01:53:10.435 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 01:53:10 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-x[31173]: return self.sslsocket_class._create( 2024-09-12T01:53:10.435 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 01:53:10 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-x[31173]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-12T01:53:10.435 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 01:53:10 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-x[31173]: self.do_handshake() 2024-09-12T01:53:10.435 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 01:53:10 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-x[31173]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-12T01:53:10.435 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 01:53:10 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-x[31173]: self._sslobj.do_handshake() 2024-09-12T01:53:10.435 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 01:53:10 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-x[31173]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-12T01:53:10.435 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 01:53:10 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-x[31173]: 2024-09-12T01:53:11.671 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:11 smithi016 ceph-mon[30947]: [12/Sep/2024:01:53:10] ENGINE Serving on https://172.21.15.16:7150 2024-09-12T01:53:11.671 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:11 smithi016 ceph-mon[30947]: [12/Sep/2024:01:53:10] ENGINE Error in HTTPServer.serve 2024-09-12T01:53:11.671 INFO:journalctl@ceph.mon.a.smithi016.stdout: Traceback (most recent call last): 2024-09-12T01:53:11.671 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-12T01:53:11.671 INFO:journalctl@ceph.mon.a.smithi016.stdout: self._connections.run(self.expiration_interval) 2024-09-12T01:53:11.671 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-12T01:53:11.671 INFO:journalctl@ceph.mon.a.smithi016.stdout: self._run(expiration_interval) 2024-09-12T01:53:11.672 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-12T01:53:11.672 INFO:journalctl@ceph.mon.a.smithi016.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-12T01:53:11.672 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-12T01:53:11.672 INFO:journalctl@ceph.mon.a.smithi016.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-12T01:53:11.672 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-12T01:53:11.672 INFO:journalctl@ceph.mon.a.smithi016.stdout: s = self.context.wrap_socket( 2024-09-12T01:53:11.672 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-12T01:53:11.672 INFO:journalctl@ceph.mon.a.smithi016.stdout: return self.sslsocket_class._create( 2024-09-12T01:53:11.672 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-12T01:53:11.672 INFO:journalctl@ceph.mon.a.smithi016.stdout: self.do_handshake() 2024-09-12T01:53:11.672 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-12T01:53:11.672 INFO:journalctl@ceph.mon.a.smithi016.stdout: self._sslobj.do_handshake() 2024-09-12T01:53:11.672 INFO:journalctl@ceph.mon.a.smithi016.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-12T01:53:11.673 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:11 smithi016 ceph-mon[30947]: [12/Sep/2024:01:53:10] ENGINE Serving on http://172.21.15.16:8765 2024-09-12T01:53:11.673 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:11 smithi016 ceph-mon[30947]: [12/Sep/2024:01:53:10] ENGINE Bus STARTED 2024-09-12T01:53:11.673 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:11 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:53:11.673 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:11 smithi016 ceph-mon[30947]: mgrmap e12: x(active, since 2s) 2024-09-12T01:53:11.673 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:11 smithi016 ceph-mon[30947]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:53:11.673 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:11 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:53:11.673 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:11 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:53:11.679 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-09-12T01:53:11.679 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:10.636+0000 7fa6b8c59640 1 Processor -- start 2024-09-12T01:53:11.680 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:10.636+0000 7fa6b8c59640 1 -- start start 2024-09-12T01:53:11.680 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:10.636+0000 7fa6b8c59640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6b4107730 0x7fa6b4107b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:11.680 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:10.637+0000 7fa6b8c59640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa6b4108100 con 0x7fa6b4107730 2024-09-12T01:53:11.680 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:10.637+0000 7fa6b259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6b4107730 0x7fa6b4107b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:11.680 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:10.637+0000 7fa6b259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6b4107730 0x7fa6b4107b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32850/0 (socket says 172.21.15.16:32850) 2024-09-12T01:53:11.680 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:10.637+0000 7fa6b259c640 1 -- 172.21.15.16:0/905924018 learned_addr learned my addr 172.21.15.16:0/905924018 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:53:11.680 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:10.637+0000 7fa6b259c640 1 -- 172.21.15.16:0/905924018 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6b4108930 con 0x7fa6b4107730 2024-09-12T01:53:11.680 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:10.638+0000 7fa6b259c640 1 --2- 172.21.15.16:0/905924018 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6b4107730 0x7fa6b4107b30 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa6a4009820 tx=0x7fa6a402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7703858a345c3bdc server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:53:11.680 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:10.638+0000 7fa6b159a640 1 -- 172.21.15.16:0/905924018 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa6a4037070 con 0x7fa6b4107730 2024-09-12T01:53:11.680 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:10.638+0000 7fa6b159a640 1 -- 172.21.15.16:0/905924018 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6a4032920 con 0x7fa6b4107730 2024-09-12T01:53:11.681 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:10.638+0000 7fa6b159a640 1 -- 172.21.15.16:0/905924018 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa6a4030810 con 0x7fa6b4107730 2024-09-12T01:53:11.681 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:10.639+0000 7fa6b8c59640 1 -- 172.21.15.16:0/905924018 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6b4107730 msgr2=0x7fa6b4107b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:53:11.681 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:10.639+0000 7fa6b8c59640 1 --2- 172.21.15.16:0/905924018 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6b4107730 0x7fa6b4107b30 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa6a4009820 tx=0x7fa6a402a270 comp rx=0 tx=0).stop 2024-09-12T01:53:11.681 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:10.639+0000 7fa6b8c59640 1 -- 172.21.15.16:0/905924018 shutdown_connections 2024-09-12T01:53:11.681 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:10.639+0000 7fa6b8c59640 1 --2- 172.21.15.16:0/905924018 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6b4107730 0x7fa6b4107b30 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:11.681 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:10.639+0000 7fa6b8c59640 1 -- 172.21.15.16:0/905924018 >> 172.21.15.16:0/905924018 conn(0x7fa6b4102ee0 msgr2=0x7fa6b4105300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:53:11.681 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:10.639+0000 7fa6b8c59640 1 -- 172.21.15.16:0/905924018 shutdown_connections 2024-09-12T01:53:11.681 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:10.639+0000 7fa6b8c59640 1 -- 172.21.15.16:0/905924018 wait complete. 2024-09-12T01:53:11.681 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:10.640+0000 7fa6b8c59640 1 Processor -- start 2024-09-12T01:53:11.681 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:10.640+0000 7fa6b8c59640 1 -- start start 2024-09-12T01:53:11.681 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:10.640+0000 7fa6b8c59640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6b4107730 0x7fa6b407daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:11.681 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:10.640+0000 7fa6b8c59640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa6b410bce0 con 0x7fa6b4107730 2024-09-12T01:53:11.682 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:10.640+0000 7fa6b259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6b4107730 0x7fa6b407daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:11.682 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:10.640+0000 7fa6b259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6b4107730 0x7fa6b407daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32864/0 (socket says 172.21.15.16:32864) 2024-09-12T01:53:11.682 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:10.640+0000 7fa6b259c640 1 -- 172.21.15.16:0/2611064868 learned_addr learned my addr 172.21.15.16:0/2611064868 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:53:11.682 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:10.641+0000 7fa6b259c640 1 -- 172.21.15.16:0/2611064868 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6b407e030 con 0x7fa6b4107730 2024-09-12T01:53:11.682 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:10.641+0000 7fa6b259c640 1 --2- 172.21.15.16:0/2611064868 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6b4107730 0x7fa6b407daf0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fa6a4030560 tx=0x7fa6a4032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:53:11.682 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:10.641+0000 7fa6a37fe640 1 -- 172.21.15.16:0/2611064868 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa6a4037040 con 0x7fa6b4107730 2024-09-12T01:53:11.682 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:10.641+0000 7fa6a37fe640 1 -- 172.21.15.16:0/2611064868 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6a4040070 con 0x7fa6b4107730 2024-09-12T01:53:11.682 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:10.641+0000 7fa6b8c59640 1 -- 172.21.15.16:0/2611064868 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6b407c220 con 0x7fa6b4107730 2024-09-12T01:53:11.682 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:10.641+0000 7fa6a37fe640 1 -- 172.21.15.16:0/2611064868 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa6a403a740 con 0x7fa6b4107730 2024-09-12T01:53:11.683 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:10.641+0000 7fa6b8c59640 1 -- 172.21.15.16:0/2611064868 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6b407c760 con 0x7fa6b4107730 2024-09-12T01:53:11.683 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:10.642+0000 7fa6a37fe640 1 -- 172.21.15.16:0/2611064868 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 11) ==== 46194+0+0 (secure 0 0 0) 0x7fa6a403a8e0 con 0x7fa6b4107730 2024-09-12T01:53:11.683 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:10.642+0000 7fa6a37fe640 1 --2- 172.21.15.16:0/2611064868 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fa69c038f00 0x7fa69c03b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:11.683 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:10.642+0000 7fa6a37fe640 1 -- 172.21.15.16:0/2611064868 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fa6a406c580 con 0x7fa6b4107730 2024-09-12T01:53:11.683 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:10.642+0000 7fa6b1d9b640 1 --2- 172.21.15.16:0/2611064868 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fa69c038f00 0x7fa69c03b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:11.683 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:10.642+0000 7fa6b8c59640 1 -- 172.21.15.16:0/2611064868 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6b410be60 con 0x7fa6b4107730 2024-09-12T01:53:11.683 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:10.643+0000 7fa6b1d9b640 1 --2- 172.21.15.16:0/2611064868 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fa69c038f00 0x7fa69c03b3c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa6a80098c0 tx=0x7fa6a8006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:53:11.683 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:10.647+0000 7fa6a37fe640 1 -- 172.21.15.16:0/2611064868 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa6a4044c70 con 0x7fa6b4107730 2024-09-12T01:53:11.683 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:10.803+0000 7fa6a37fe640 1 -- 172.21.15.16:0/2611064868 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7fa6a4045980 con 0x7fa6b4107730 2024-09-12T01:53:11.683 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:10.840+0000 7fa6b8c59640 1 -- 172.21.15.16:0/2611064868 --> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7fa6b407cde0 con 0x7fa69c038f00 2024-09-12T01:53:11.683 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:11.028+0000 7fa6a37fe640 1 -- 172.21.15.16:0/2611064868 <== mgr.14150 v2:172.21.15.16:6800/3887444820 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7fa6b407cde0 con 0x7fa69c038f00 2024-09-12T01:53:11.683 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:11.030+0000 7fa6b8c59640 1 -- 172.21.15.16:0/2611064868 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fa69c038f00 msgr2=0x7fa69c03b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:53:11.683 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:11.030+0000 7fa6b8c59640 1 --2- 172.21.15.16:0/2611064868 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fa69c038f00 0x7fa69c03b3c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa6a80098c0 tx=0x7fa6a8006db0 comp rx=0 tx=0).stop 2024-09-12T01:53:11.684 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:11.030+0000 7fa6b8c59640 1 -- 172.21.15.16:0/2611064868 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6b4107730 msgr2=0x7fa6b407daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:53:11.684 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:11.030+0000 7fa6b8c59640 1 --2- 172.21.15.16:0/2611064868 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6b4107730 0x7fa6b407daf0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fa6a4030560 tx=0x7fa6a4032670 comp rx=0 tx=0).stop 2024-09-12T01:53:11.684 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:11.030+0000 7fa6b8c59640 1 -- 172.21.15.16:0/2611064868 shutdown_connections 2024-09-12T01:53:11.684 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:11.030+0000 7fa6b8c59640 1 --2- 172.21.15.16:0/2611064868 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fa69c038f00 0x7fa69c03b3c0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:11.684 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:11.030+0000 7fa6b8c59640 1 --2- 172.21.15.16:0/2611064868 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6b4107730 0x7fa6b407daf0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:11.684 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:11.030+0000 7fa6b8c59640 1 -- 172.21.15.16:0/2611064868 >> 172.21.15.16:0/2611064868 conn(0x7fa6b4102ee0 msgr2=0x7fa6b4103a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:53:11.684 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:11.030+0000 7fa6b8c59640 1 -- 172.21.15.16:0/2611064868 shutdown_connections 2024-09-12T01:53:11.684 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:11.030+0000 7fa6b8c59640 1 -- 172.21.15.16:0/2611064868 wait complete. 2024-09-12T01:53:11.684 INFO:teuthology.orchestra.run.smithi016.stdout:Creating initial admin user... 2024-09-12T01:53:13.539 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:13 smithi016 ceph-mon[30947]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:53:13.540 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:13 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:53:13.828 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$AIRuP4pe0RHzCIUAl/39tuPwGshRHXTO2Wt1fODfEgTHb4OJuqvei", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1726105993, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-09-12T01:53:13.829 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:12.778+0000 7f838a99d640 1 Processor -- start 2024-09-12T01:53:13.829 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:12.778+0000 7f838a99d640 1 -- start start 2024-09-12T01:53:13.829 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:12.778+0000 7f838a99d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8384107700 0x7f8384107b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:13.829 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:12.778+0000 7f838a99d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f83841080d0 con 0x7f8384107700 2024-09-12T01:53:13.829 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:12.779+0000 7f8383fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8384107700 0x7f8384107b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:13.830 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:12.779+0000 7f8383fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8384107700 0x7f8384107b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41492/0 (socket says 172.21.15.16:41492) 2024-09-12T01:53:13.830 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:12.779+0000 7f8383fff640 1 -- 172.21.15.16:0/916837603 learned_addr learned my addr 172.21.15.16:0/916837603 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:53:13.830 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:12.779+0000 7f8383fff640 1 -- 172.21.15.16:0/916837603 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8384108900 con 0x7f8384107700 2024-09-12T01:53:13.830 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:12.780+0000 7f8383fff640 1 --2- 172.21.15.16:0/916837603 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8384107700 0x7f8384107b00 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f8374009820 tx=0x7f837402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d860a27bff17da48 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:53:13.830 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:12.780+0000 7f8382ffd640 1 -- 172.21.15.16:0/916837603 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8374037070 con 0x7f8384107700 2024-09-12T01:53:13.831 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:12.780+0000 7f8382ffd640 1 -- 172.21.15.16:0/916837603 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8374032920 con 0x7f8384107700 2024-09-12T01:53:13.831 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:12.780+0000 7f8382ffd640 1 -- 172.21.15.16:0/916837603 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8374030810 con 0x7f8384107700 2024-09-12T01:53:13.831 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:12.781+0000 7f838a99d640 1 -- 172.21.15.16:0/916837603 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8384107700 msgr2=0x7f8384107b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:53:13.831 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:12.781+0000 7f838a99d640 1 --2- 172.21.15.16:0/916837603 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8384107700 0x7f8384107b00 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f8374009820 tx=0x7f837402a270 comp rx=0 tx=0).stop 2024-09-12T01:53:13.831 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:12.781+0000 7f838a99d640 1 -- 172.21.15.16:0/916837603 shutdown_connections 2024-09-12T01:53:13.831 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:12.781+0000 7f838a99d640 1 --2- 172.21.15.16:0/916837603 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8384107700 0x7f8384107b00 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:13.832 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:12.781+0000 7f838a99d640 1 -- 172.21.15.16:0/916837603 >> 172.21.15.16:0/916837603 conn(0x7f8384102e90 msgr2=0x7f83841052d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:53:13.832 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:12.781+0000 7f838a99d640 1 -- 172.21.15.16:0/916837603 shutdown_connections 2024-09-12T01:53:13.832 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:12.781+0000 7f838a99d640 1 -- 172.21.15.16:0/916837603 wait complete. 2024-09-12T01:53:13.832 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:12.781+0000 7f838a99d640 1 Processor -- start 2024-09-12T01:53:13.832 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:12.781+0000 7f838a99d640 1 -- start start 2024-09-12T01:53:13.833 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:12.782+0000 7f838a99d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8384107700 0x7f838419b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:13.833 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:12.782+0000 7f838a99d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f838410bcb0 con 0x7f8384107700 2024-09-12T01:53:13.833 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:12.782+0000 7f8383fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8384107700 0x7f838419b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:13.833 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:12.782+0000 7f8383fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8384107700 0x7f838419b750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41500/0 (socket says 172.21.15.16:41500) 2024-09-12T01:53:13.833 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:12.782+0000 7f8383fff640 1 -- 172.21.15.16:0/97636508 learned_addr learned my addr 172.21.15.16:0/97636508 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:53:13.833 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:12.782+0000 7f8383fff640 1 -- 172.21.15.16:0/97636508 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f838419bc90 con 0x7f8384107700 2024-09-12T01:53:13.834 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:12.782+0000 7f8383fff640 1 --2- 172.21.15.16:0/97636508 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8384107700 0x7f838419b750 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f8374030560 tx=0x7f8374032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:53:13.834 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:12.783+0000 7f83817fa640 1 -- 172.21.15.16:0/97636508 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8374037040 con 0x7f8384107700 2024-09-12T01:53:13.834 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:12.783+0000 7f83817fa640 1 -- 172.21.15.16:0/97636508 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8374040070 con 0x7f8384107700 2024-09-12T01:53:13.834 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:12.783+0000 7f838a99d640 1 -- 172.21.15.16:0/97636508 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f838419bf20 con 0x7f8384107700 2024-09-12T01:53:13.834 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:12.783+0000 7f83817fa640 1 -- 172.21.15.16:0/97636508 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f837403a740 con 0x7f8384107700 2024-09-12T01:53:13.835 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:12.783+0000 7f838a99d640 1 -- 172.21.15.16:0/97636508 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f838419ec10 con 0x7f8384107700 2024-09-12T01:53:13.835 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:12.784+0000 7f83817fa640 1 -- 172.21.15.16:0/97636508 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7f837403a9b0 con 0x7f8384107700 2024-09-12T01:53:13.835 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:12.784+0000 7f83817fa640 1 --2- 172.21.15.16:0/97636508 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f836c038fc0 0x7f836c03b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:13.835 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:12.784+0000 7f838a99d640 1 -- 172.21.15.16:0/97636508 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f838410be30 con 0x7f8384107700 2024-09-12T01:53:13.835 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:12.784+0000 7f83817fa640 1 -- 172.21.15.16:0/97636508 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f837406c700 con 0x7f8384107700 2024-09-12T01:53:13.835 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:12.784+0000 7f83837fe640 1 --2- 172.21.15.16:0/97636508 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f836c038fc0 0x7f836c03b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:13.836 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:12.784+0000 7f83837fe640 1 --2- 172.21.15.16:0/97636508 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f836c038fc0 0x7f836c03b480 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f83780098c0 tx=0x7f8378006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:53:13.836 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:12.789+0000 7f83817fa640 1 -- 172.21.15.16:0/97636508 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8374042e30 con 0x7f8384107700 2024-09-12T01:53:13.836 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:12.979+0000 7f838a99d640 1 -- 172.21.15.16:0/97636508 --> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7f8384107b80 con 0x7f836c038fc0 2024-09-12T01:53:13.836 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:13.227+0000 7f83817fa640 1 -- 172.21.15.16:0/97636508 <== mgr.14150 v2:172.21.15.16:6800/3887444820 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7f8384107b80 con 0x7f836c038fc0 2024-09-12T01:53:13.836 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:13.229+0000 7f838a99d640 1 -- 172.21.15.16:0/97636508 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f836c038fc0 msgr2=0x7f836c03b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:53:13.837 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:13.229+0000 7f838a99d640 1 --2- 172.21.15.16:0/97636508 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f836c038fc0 0x7f836c03b480 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f83780098c0 tx=0x7f8378006db0 comp rx=0 tx=0).stop 2024-09-12T01:53:13.837 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:13.229+0000 7f838a99d640 1 -- 172.21.15.16:0/97636508 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8384107700 msgr2=0x7f838419b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:53:13.837 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:13.229+0000 7f838a99d640 1 --2- 172.21.15.16:0/97636508 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8384107700 0x7f838419b750 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f8374030560 tx=0x7f8374032670 comp rx=0 tx=0).stop 2024-09-12T01:53:13.837 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:13.230+0000 7f838a99d640 1 -- 172.21.15.16:0/97636508 shutdown_connections 2024-09-12T01:53:13.837 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:13.230+0000 7f838a99d640 1 --2- 172.21.15.16:0/97636508 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f836c038fc0 0x7f836c03b480 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:13.838 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:13.230+0000 7f838a99d640 1 --2- 172.21.15.16:0/97636508 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8384107700 0x7f838419b750 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:13.838 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:13.230+0000 7f838a99d640 1 -- 172.21.15.16:0/97636508 >> 172.21.15.16:0/97636508 conn(0x7f8384102e90 msgr2=0x7f8384103a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:53:13.838 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:13.230+0000 7f838a99d640 1 -- 172.21.15.16:0/97636508 shutdown_connections 2024-09-12T01:53:13.838 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:13.230+0000 7f838a99d640 1 -- 172.21.15.16:0/97636508 wait complete. 2024-09-12T01:53:13.838 INFO:teuthology.orchestra.run.smithi016.stdout:Fetching dashboard port number... 2024-09-12T01:53:14.930 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 8443 2024-09-12T01:53:14.930 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:14.192+0000 7fa91c038640 1 Processor -- start 2024-09-12T01:53:14.930 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:14.192+0000 7fa91c038640 1 -- start start 2024-09-12T01:53:14.931 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:14.193+0000 7fa91c038640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa914108140 0x7fa91410a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:14.931 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:14.193+0000 7fa91c038640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa91407be30 con 0x7fa914108140 2024-09-12T01:53:14.931 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:14.193+0000 7fa919dd4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa914108140 0x7fa91410a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:14.931 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:14.193+0000 7fa919dd4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa914108140 0x7fa91410a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41510/0 (socket says 172.21.15.16:41510) 2024-09-12T01:53:14.931 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:14.193+0000 7fa919dd4640 1 -- 172.21.15.16:0/3015428293 learned_addr learned my addr 172.21.15.16:0/3015428293 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:53:14.931 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:14.193+0000 7fa919dd4640 1 -- 172.21.15.16:0/3015428293 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa91410aa70 con 0x7fa914108140 2024-09-12T01:53:14.931 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:14.194+0000 7fa919dd4640 1 --2- 172.21.15.16:0/3015428293 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa914108140 0x7fa91410a530 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa910009820 tx=0x7fa91002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2176e292024ed9d4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:53:14.931 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:14.194+0000 7fa918dd2640 1 -- 172.21.15.16:0/3015428293 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa910037070 con 0x7fa914108140 2024-09-12T01:53:14.931 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:14.194+0000 7fa918dd2640 1 -- 172.21.15.16:0/3015428293 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa910032920 con 0x7fa914108140 2024-09-12T01:53:14.932 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:14.194+0000 7fa918dd2640 1 -- 172.21.15.16:0/3015428293 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa910030810 con 0x7fa914108140 2024-09-12T01:53:14.932 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:14.195+0000 7fa91c038640 1 -- 172.21.15.16:0/3015428293 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa914108140 msgr2=0x7fa91410a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:53:14.932 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:14.195+0000 7fa91c038640 1 --2- 172.21.15.16:0/3015428293 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa914108140 0x7fa91410a530 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa910009820 tx=0x7fa91002a270 comp rx=0 tx=0).stop 2024-09-12T01:53:14.932 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:14.195+0000 7fa91c038640 1 -- 172.21.15.16:0/3015428293 shutdown_connections 2024-09-12T01:53:14.932 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:14.195+0000 7fa91c038640 1 --2- 172.21.15.16:0/3015428293 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa914108140 0x7fa91410a530 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:14.932 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:14.195+0000 7fa91c038640 1 -- 172.21.15.16:0/3015428293 >> 172.21.15.16:0/3015428293 conn(0x7fa914101e20 msgr2=0x7fa914104240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:53:14.932 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:14.195+0000 7fa91c038640 1 -- 172.21.15.16:0/3015428293 shutdown_connections 2024-09-12T01:53:14.932 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:14.195+0000 7fa91c038640 1 -- 172.21.15.16:0/3015428293 wait complete. 2024-09-12T01:53:14.932 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:14.195+0000 7fa91c038640 1 Processor -- start 2024-09-12T01:53:14.933 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:14.195+0000 7fa91c038640 1 -- start start 2024-09-12T01:53:14.933 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:14.196+0000 7fa91c038640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa914108140 0x7fa91419d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:14.933 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:14.196+0000 7fa91c038640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa91410bb50 con 0x7fa914108140 2024-09-12T01:53:14.933 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:14.196+0000 7fa919dd4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa914108140 0x7fa91419d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:14.933 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:14.196+0000 7fa919dd4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa914108140 0x7fa91419d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41520/0 (socket says 172.21.15.16:41520) 2024-09-12T01:53:14.933 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:14.196+0000 7fa919dd4640 1 -- 172.21.15.16:0/1811276787 learned_addr learned my addr 172.21.15.16:0/1811276787 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:53:14.933 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:14.196+0000 7fa919dd4640 1 -- 172.21.15.16:0/1811276787 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa91419de30 con 0x7fa914108140 2024-09-12T01:53:14.933 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:14.196+0000 7fa919dd4640 1 --2- 172.21.15.16:0/1811276787 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa914108140 0x7fa91419d8f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa9100097f0 tx=0x7fa9100327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:53:14.933 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:14.196+0000 7fa90affd640 1 -- 172.21.15.16:0/1811276787 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa910037040 con 0x7fa914108140 2024-09-12T01:53:14.934 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:14.196+0000 7fa90affd640 1 -- 172.21.15.16:0/1811276787 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa910030cc0 con 0x7fa914108140 2024-09-12T01:53:14.934 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:14.196+0000 7fa91c038640 1 -- 172.21.15.16:0/1811276787 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa91419e0c0 con 0x7fa914108140 2024-09-12T01:53:14.934 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:14.196+0000 7fa90affd640 1 -- 172.21.15.16:0/1811276787 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa91003a6b0 con 0x7fa914108140 2024-09-12T01:53:14.934 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:14.196+0000 7fa91c038640 1 -- 172.21.15.16:0/1811276787 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9141a0db0 con 0x7fa914108140 2024-09-12T01:53:14.934 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:14.197+0000 7fa90affd640 1 -- 172.21.15.16:0/1811276787 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7fa91003a850 con 0x7fa914108140 2024-09-12T01:53:14.934 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:14.197+0000 7fa90affd640 1 --2- 172.21.15.16:0/1811276787 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fa900039070 0x7fa90003b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:14.934 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:14.197+0000 7fa90affd640 1 -- 172.21.15.16:0/1811276787 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fa91006c6b0 con 0x7fa914108140 2024-09-12T01:53:14.934 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:14.198+0000 7fa9195d3640 1 --2- 172.21.15.16:0/1811276787 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fa900039070 0x7fa90003b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:14.934 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:14.198+0000 7fa91c038640 1 -- 172.21.15.16:0/1811276787 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9141085a0 con 0x7fa914108140 2024-09-12T01:53:14.934 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:14.198+0000 7fa9195d3640 1 --2- 172.21.15.16:0/1811276787 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fa900039070 0x7fa90003b530 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fa9040098c0 tx=0x7fa904006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:53:14.935 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:14.202+0000 7fa90affd640 1 -- 172.21.15.16:0/1811276787 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa910044920 con 0x7fa914108140 2024-09-12T01:53:14.935 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:14.247+0000 7fa90affd640 1 -- 172.21.15.16:0/1811276787 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fa910044400 con 0x7fa914108140 2024-09-12T01:53:14.935 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:14.378+0000 7fa91c038640 1 -- 172.21.15.16:0/1811276787 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7fa9141a1390 con 0x7fa914108140 2024-09-12T01:53:14.935 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:14.378+0000 7fa90affd640 1 -- 172.21.15.16:0/1811276787 <== mon.0 v2:172.21.15.16:3300/0 8 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7fa910065cd0 con 0x7fa914108140 2024-09-12T01:53:14.935 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:14.380+0000 7fa91c038640 1 -- 172.21.15.16:0/1811276787 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fa900039070 msgr2=0x7fa90003b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:53:14.935 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:14.380+0000 7fa91c038640 1 --2- 172.21.15.16:0/1811276787 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fa900039070 0x7fa90003b530 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fa9040098c0 tx=0x7fa904006db0 comp rx=0 tx=0).stop 2024-09-12T01:53:14.935 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:14.380+0000 7fa91c038640 1 -- 172.21.15.16:0/1811276787 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa914108140 msgr2=0x7fa91419d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:53:14.935 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:14.380+0000 7fa91c038640 1 --2- 172.21.15.16:0/1811276787 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa914108140 0x7fa91419d8f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa9100097f0 tx=0x7fa9100327b0 comp rx=0 tx=0).stop 2024-09-12T01:53:14.935 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:14.380+0000 7fa91c038640 1 -- 172.21.15.16:0/1811276787 shutdown_connections 2024-09-12T01:53:14.935 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:14.381+0000 7fa91c038640 1 --2- 172.21.15.16:0/1811276787 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fa900039070 0x7fa90003b530 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:14.935 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:14.381+0000 7fa91c038640 1 --2- 172.21.15.16:0/1811276787 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa914108140 0x7fa91419d8f0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:14.936 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:14.381+0000 7fa91c038640 1 -- 172.21.15.16:0/1811276787 >> 172.21.15.16:0/1811276787 conn(0x7fa914101e20 msgr2=0x7fa914102960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:53:14.936 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:14.381+0000 7fa91c038640 1 -- 172.21.15.16:0/1811276787 shutdown_connections 2024-09-12T01:53:14.936 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:14.381+0000 7fa91c038640 1 -- 172.21.15.16:0/1811276787 wait complete. 2024-09-12T01:53:14.938 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-12T01:53:14.939 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout disabled 2024-09-12T01:53:14.947 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-12T01:53:14.947 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout inactive 2024-09-12T01:53:14.947 INFO:teuthology.orchestra.run.smithi016.stdout:firewalld.service is not enabled 2024-09-12T01:53:14.947 INFO:teuthology.orchestra.run.smithi016.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-09-12T01:53:14.949 INFO:teuthology.orchestra.run.smithi016.stdout:Ceph Dashboard is now available at: 2024-09-12T01:53:14.949 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:53:14.949 INFO:teuthology.orchestra.run.smithi016.stdout: URL: https://smithi016.front.sepia.ceph.com:8443/ 2024-09-12T01:53:14.949 INFO:teuthology.orchestra.run.smithi016.stdout: User: admin 2024-09-12T01:53:14.949 INFO:teuthology.orchestra.run.smithi016.stdout: Password: bjxjj77o3d 2024-09-12T01:53:14.949 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:53:14.950 INFO:teuthology.orchestra.run.smithi016.stdout:Saving cluster configuration to /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/config directory 2024-09-12T01:53:15.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:15 smithi016 ceph-mon[30947]: mgrmap e13: x(active, since 6s) 2024-09-12T01:53:15.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:15 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/1811276787' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-09-12T01:53:16.328 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:15.449+0000 7fc48419a640 1 Processor -- start 2024-09-12T01:53:16.328 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:15.449+0000 7fc48419a640 1 -- start start 2024-09-12T01:53:16.328 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:15.449+0000 7fc48419a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc47c105a60 0x7fc47c105e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:16.328 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:15.449+0000 7fc48419a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc47c106430 con 0x7fc47c105a60 2024-09-12T01:53:16.328 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:15.449+0000 7fc481f36640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc47c105a60 0x7fc47c105e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:16.329 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:15.449+0000 7fc481f36640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc47c105a60 0x7fc47c105e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41530/0 (socket says 172.21.15.16:41530) 2024-09-12T01:53:16.329 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:15.449+0000 7fc481f36640 1 -- 172.21.15.16:0/2509754674 learned_addr learned my addr 172.21.15.16:0/2509754674 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:53:16.329 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:15.450+0000 7fc481f36640 1 -- 172.21.15.16:0/2509754674 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc47c108a40 con 0x7fc47c105a60 2024-09-12T01:53:16.329 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:15.450+0000 7fc481f36640 1 --2- 172.21.15.16:0/2509754674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc47c105a60 0x7fc47c105e60 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc478009820 tx=0x7fc47802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46c1ce40a091ef52 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:53:16.329 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:15.451+0000 7fc480f34640 1 -- 172.21.15.16:0/2509754674 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc478037070 con 0x7fc47c105a60 2024-09-12T01:53:16.329 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:15.451+0000 7fc480f34640 1 -- 172.21.15.16:0/2509754674 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc478032920 con 0x7fc47c105a60 2024-09-12T01:53:16.329 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:15.451+0000 7fc480f34640 1 -- 172.21.15.16:0/2509754674 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc478030810 con 0x7fc47c105a60 2024-09-12T01:53:16.329 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:15.451+0000 7fc48419a640 1 -- 172.21.15.16:0/2509754674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc47c105a60 msgr2=0x7fc47c105e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:53:16.329 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:15.451+0000 7fc48419a640 1 --2- 172.21.15.16:0/2509754674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc47c105a60 0x7fc47c105e60 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc478009820 tx=0x7fc47802a270 comp rx=0 tx=0).stop 2024-09-12T01:53:16.329 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:15.451+0000 7fc48419a640 1 -- 172.21.15.16:0/2509754674 shutdown_connections 2024-09-12T01:53:16.329 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:15.451+0000 7fc48419a640 1 --2- 172.21.15.16:0/2509754674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc47c105a60 0x7fc47c105e60 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:16.329 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:15.451+0000 7fc48419a640 1 -- 172.21.15.16:0/2509754674 >> 172.21.15.16:0/2509754674 conn(0x7fc47c1015f0 msgr2=0x7fc47c103a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:53:16.329 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:15.451+0000 7fc48419a640 1 -- 172.21.15.16:0/2509754674 shutdown_connections 2024-09-12T01:53:16.330 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:15.451+0000 7fc48419a640 1 -- 172.21.15.16:0/2509754674 wait complete. 2024-09-12T01:53:16.330 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:15.452+0000 7fc48419a640 1 Processor -- start 2024-09-12T01:53:16.330 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:15.452+0000 7fc48419a640 1 -- start start 2024-09-12T01:53:16.330 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:15.452+0000 7fc48419a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc47c105a60 0x7fc47c19d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:16.330 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:15.452+0000 7fc48419a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc47c10bc70 con 0x7fc47c105a60 2024-09-12T01:53:16.330 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:15.452+0000 7fc481f36640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc47c105a60 0x7fc47c19d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:16.330 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:15.452+0000 7fc481f36640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc47c105a60 0x7fc47c19d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41532/0 (socket says 172.21.15.16:41532) 2024-09-12T01:53:16.330 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:15.452+0000 7fc481f36640 1 -- 172.21.15.16:0/3268915415 learned_addr learned my addr 172.21.15.16:0/3268915415 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:53:16.330 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:15.453+0000 7fc481f36640 1 -- 172.21.15.16:0/3268915415 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc47c19def0 con 0x7fc47c105a60 2024-09-12T01:53:16.330 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:15.453+0000 7fc481f36640 1 --2- 172.21.15.16:0/3268915415 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc47c105a60 0x7fc47c19d9b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc478030560 tx=0x7fc478032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:53:16.330 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:15.453+0000 7fc472ffd640 1 -- 172.21.15.16:0/3268915415 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc478037040 con 0x7fc47c105a60 2024-09-12T01:53:16.330 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:15.453+0000 7fc472ffd640 1 -- 172.21.15.16:0/3268915415 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc478040070 con 0x7fc47c105a60 2024-09-12T01:53:16.330 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:15.453+0000 7fc48419a640 1 -- 172.21.15.16:0/3268915415 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc47c19e180 con 0x7fc47c105a60 2024-09-12T01:53:16.330 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:15.453+0000 7fc472ffd640 1 -- 172.21.15.16:0/3268915415 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc47803a740 con 0x7fc47c105a60 2024-09-12T01:53:16.331 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:15.453+0000 7fc48419a640 1 -- 172.21.15.16:0/3268915415 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc47c1a0e70 con 0x7fc47c105a60 2024-09-12T01:53:16.331 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:15.454+0000 7fc472ffd640 1 -- 172.21.15.16:0/3268915415 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fc47803b420 con 0x7fc47c105a60 2024-09-12T01:53:16.331 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:15.454+0000 7fc48419a640 1 -- 172.21.15.16:0/3268915415 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc47c105ee0 con 0x7fc47c105a60 2024-09-12T01:53:16.331 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:15.454+0000 7fc472ffd640 1 --2- 172.21.15.16:0/3268915415 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc4680390c0 0x7fc46803b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:16.331 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:15.454+0000 7fc472ffd640 1 -- 172.21.15.16:0/3268915415 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fc47806c7d0 con 0x7fc47c105a60 2024-09-12T01:53:16.331 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:15.455+0000 7fc481735640 1 --2- 172.21.15.16:0/3268915415 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc4680390c0 0x7fc46803b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:16.331 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:15.455+0000 7fc481735640 1 --2- 172.21.15.16:0/3268915415 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc4680390c0 0x7fc46803b580 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fc46c0098c0 tx=0x7fc46c006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:53:16.331 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:15.459+0000 7fc472ffd640 1 -- 172.21.15.16:0/3268915415 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc478042b50 con 0x7fc47c105a60 2024-09-12T01:53:16.331 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:15.726+0000 7fc48419a640 1 -- 172.21.15.16:0/3268915415 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7fc47c1a1460 con 0x7fc47c105a60 2024-09-12T01:53:16.331 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:15.729+0000 7fc472ffd640 1 -- 172.21.15.16:0/3268915415 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7fc478043360 con 0x7fc47c105a60 2024-09-12T01:53:16.331 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-09-12T01:53:16.331 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:15.731+0000 7fc48419a640 1 -- 172.21.15.16:0/3268915415 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc4680390c0 msgr2=0x7fc46803b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:53:16.331 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:15.731+0000 7fc48419a640 1 --2- 172.21.15.16:0/3268915415 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc4680390c0 0x7fc46803b580 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fc46c0098c0 tx=0x7fc46c006db0 comp rx=0 tx=0).stop 2024-09-12T01:53:16.332 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:15.731+0000 7fc48419a640 1 -- 172.21.15.16:0/3268915415 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc47c105a60 msgr2=0x7fc47c19d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:53:16.332 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:15.731+0000 7fc48419a640 1 --2- 172.21.15.16:0/3268915415 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc47c105a60 0x7fc47c19d9b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc478030560 tx=0x7fc478032670 comp rx=0 tx=0).stop 2024-09-12T01:53:16.332 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:15.731+0000 7fc48419a640 1 -- 172.21.15.16:0/3268915415 shutdown_connections 2024-09-12T01:53:16.332 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:15.731+0000 7fc48419a640 1 --2- 172.21.15.16:0/3268915415 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc4680390c0 0x7fc46803b580 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:16.332 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:15.731+0000 7fc48419a640 1 --2- 172.21.15.16:0/3268915415 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc47c105a60 0x7fc47c19d9b0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:16.332 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:15.731+0000 7fc48419a640 1 -- 172.21.15.16:0/3268915415 >> 172.21.15.16:0/3268915415 conn(0x7fc47c1015f0 msgr2=0x7fc47c102130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:53:16.332 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:15.731+0000 7fc48419a640 1 -- 172.21.15.16:0/3268915415 shutdown_connections 2024-09-12T01:53:16.332 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T01:53:15.731+0000 7fc48419a640 1 -- 172.21.15.16:0/3268915415 wait complete. 2024-09-12T01:53:16.332 INFO:teuthology.orchestra.run.smithi016.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-09-12T01:53:16.332 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:53:16.332 INFO:teuthology.orchestra.run.smithi016.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-09-12T01:53:16.332 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:53:16.332 INFO:teuthology.orchestra.run.smithi016.stdout:Or, if you are only running a single cluster on this host: 2024-09-12T01:53:16.332 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:53:16.333 INFO:teuthology.orchestra.run.smithi016.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-09-12T01:53:16.333 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:53:16.333 INFO:teuthology.orchestra.run.smithi016.stdout:Please consider enabling telemetry to help improve Ceph: 2024-09-12T01:53:16.333 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:53:16.333 INFO:teuthology.orchestra.run.smithi016.stdout: ceph telemetry on 2024-09-12T01:53:16.333 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:53:16.333 INFO:teuthology.orchestra.run.smithi016.stdout:For more information see: 2024-09-12T01:53:16.333 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:53:16.333 INFO:teuthology.orchestra.run.smithi016.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-09-12T01:53:16.333 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:53:16.333 INFO:teuthology.orchestra.run.smithi016.stdout:Bootstrap complete. 2024-09-12T01:53:16.375 INFO:tasks.cephadm:Fetching config... 2024-09-12T01:53:16.375 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-12T01:53:16.375 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-09-12T01:53:16.394 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-09-12T01:53:16.394 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-12T01:53:16.394 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-09-12T01:53:16.451 INFO:tasks.cephadm:Fetching mon keyring... 2024-09-12T01:53:16.451 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-12T01:53:16.451 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/keyring of=/dev/stdout 2024-09-12T01:53:16.519 INFO:tasks.cephadm:Fetching pub ssh key... 2024-09-12T01:53:16.519 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-12T01:53:16.519 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-09-12T01:53:16.576 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-09-12T01:53:16.577 DEBUG:teuthology.orchestra.run.smithi016:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDXax7ciWzoEL/3Zpb3dWbBuvhwWB5WYE+lekT8TQbOyZE+KRRh5EnP/BNJyD0jlGM7ijCJvTlMiwBoCKZQIovzip7esb+aYqLZ/iOvYwFMvcCl1VrGKkC82DsdaNkpCGBMNQOw7ypYx6AGQ/zl/bcTVhJrBjFScjqKifBeJTZPp3Qy/2RNPpie2tyqEMh3VFMeLAtr4GoH82TjMwF+ZPHtRKLyUS0uolyduRySxkGix566xnX8lDGiYcLqHLp5KTKbGaLLJhP4XNbLvD2fWir4vxaYUvjJbFlTBepqNvmYt1mp5rLyXt07F1GnfCxOlXFGCs5DWzHyl1s6Uzfq1Z7TdIsBhfBFBlvLvXIzBDevsEgPV/X9BJvowGh/BA/FFeESKQnNjuKYYM2TPbGkM2brpHJMtZhYl9xgVdADzPjb4y++XAgyq6bosTuv76LzBCyiviOZe+2IHD1YTzWmndGIBpJXmTnL9DL/0hL5kbpTY1J4nl7B1PyK6cAEIlb7BM8= ceph-879f3b56-70a9-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-12T01:53:16.670 INFO:teuthology.orchestra.run.smithi016.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDXax7ciWzoEL/3Zpb3dWbBuvhwWB5WYE+lekT8TQbOyZE+KRRh5EnP/BNJyD0jlGM7ijCJvTlMiwBoCKZQIovzip7esb+aYqLZ/iOvYwFMvcCl1VrGKkC82DsdaNkpCGBMNQOw7ypYx6AGQ/zl/bcTVhJrBjFScjqKifBeJTZPp3Qy/2RNPpie2tyqEMh3VFMeLAtr4GoH82TjMwF+ZPHtRKLyUS0uolyduRySxkGix566xnX8lDGiYcLqHLp5KTKbGaLLJhP4XNbLvD2fWir4vxaYUvjJbFlTBepqNvmYt1mp5rLyXt07F1GnfCxOlXFGCs5DWzHyl1s6Uzfq1Z7TdIsBhfBFBlvLvXIzBDevsEgPV/X9BJvowGh/BA/FFeESKQnNjuKYYM2TPbGkM2brpHJMtZhYl9xgVdADzPjb4y++XAgyq6bosTuv76LzBCyiviOZe+2IHD1YTzWmndGIBpJXmTnL9DL/0hL5kbpTY1J4nl7B1PyK6cAEIlb7BM8= ceph-879f3b56-70a9-11ef-bcea-c7b262605968 2024-09-12T01:53:16.682 DEBUG:teuthology.orchestra.run.smithi033:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDXax7ciWzoEL/3Zpb3dWbBuvhwWB5WYE+lekT8TQbOyZE+KRRh5EnP/BNJyD0jlGM7ijCJvTlMiwBoCKZQIovzip7esb+aYqLZ/iOvYwFMvcCl1VrGKkC82DsdaNkpCGBMNQOw7ypYx6AGQ/zl/bcTVhJrBjFScjqKifBeJTZPp3Qy/2RNPpie2tyqEMh3VFMeLAtr4GoH82TjMwF+ZPHtRKLyUS0uolyduRySxkGix566xnX8lDGiYcLqHLp5KTKbGaLLJhP4XNbLvD2fWir4vxaYUvjJbFlTBepqNvmYt1mp5rLyXt07F1GnfCxOlXFGCs5DWzHyl1s6Uzfq1Z7TdIsBhfBFBlvLvXIzBDevsEgPV/X9BJvowGh/BA/FFeESKQnNjuKYYM2TPbGkM2brpHJMtZhYl9xgVdADzPjb4y++XAgyq6bosTuv76LzBCyiviOZe+2IHD1YTzWmndGIBpJXmTnL9DL/0hL5kbpTY1J4nl7B1PyK6cAEIlb7BM8= ceph-879f3b56-70a9-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-12T01:53:16.724 INFO:teuthology.orchestra.run.smithi033.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDXax7ciWzoEL/3Zpb3dWbBuvhwWB5WYE+lekT8TQbOyZE+KRRh5EnP/BNJyD0jlGM7ijCJvTlMiwBoCKZQIovzip7esb+aYqLZ/iOvYwFMvcCl1VrGKkC82DsdaNkpCGBMNQOw7ypYx6AGQ/zl/bcTVhJrBjFScjqKifBeJTZPp3Qy/2RNPpie2tyqEMh3VFMeLAtr4GoH82TjMwF+ZPHtRKLyUS0uolyduRySxkGix566xnX8lDGiYcLqHLp5KTKbGaLLJhP4XNbLvD2fWir4vxaYUvjJbFlTBepqNvmYt1mp5rLyXt07F1GnfCxOlXFGCs5DWzHyl1s6Uzfq1Z7TdIsBhfBFBlvLvXIzBDevsEgPV/X9BJvowGh/BA/FFeESKQnNjuKYYM2TPbGkM2brpHJMtZhYl9xgVdADzPjb4y++XAgyq6bosTuv76LzBCyiviOZe+2IHD1YTzWmndGIBpJXmTnL9DL/0hL5kbpTY1J4nl7B1PyK6cAEIlb7BM8= ceph-879f3b56-70a9-11ef-bcea-c7b262605968 2024-09-12T01:53:16.737 DEBUG:teuthology.orchestra.run.smithi184:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDXax7ciWzoEL/3Zpb3dWbBuvhwWB5WYE+lekT8TQbOyZE+KRRh5EnP/BNJyD0jlGM7ijCJvTlMiwBoCKZQIovzip7esb+aYqLZ/iOvYwFMvcCl1VrGKkC82DsdaNkpCGBMNQOw7ypYx6AGQ/zl/bcTVhJrBjFScjqKifBeJTZPp3Qy/2RNPpie2tyqEMh3VFMeLAtr4GoH82TjMwF+ZPHtRKLyUS0uolyduRySxkGix566xnX8lDGiYcLqHLp5KTKbGaLLJhP4XNbLvD2fWir4vxaYUvjJbFlTBepqNvmYt1mp5rLyXt07F1GnfCxOlXFGCs5DWzHyl1s6Uzfq1Z7TdIsBhfBFBlvLvXIzBDevsEgPV/X9BJvowGh/BA/FFeESKQnNjuKYYM2TPbGkM2brpHJMtZhYl9xgVdADzPjb4y++XAgyq6bosTuv76LzBCyiviOZe+2IHD1YTzWmndGIBpJXmTnL9DL/0hL5kbpTY1J4nl7B1PyK6cAEIlb7BM8= ceph-879f3b56-70a9-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-12T01:53:16.778 INFO:teuthology.orchestra.run.smithi184.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDXax7ciWzoEL/3Zpb3dWbBuvhwWB5WYE+lekT8TQbOyZE+KRRh5EnP/BNJyD0jlGM7ijCJvTlMiwBoCKZQIovzip7esb+aYqLZ/iOvYwFMvcCl1VrGKkC82DsdaNkpCGBMNQOw7ypYx6AGQ/zl/bcTVhJrBjFScjqKifBeJTZPp3Qy/2RNPpie2tyqEMh3VFMeLAtr4GoH82TjMwF+ZPHtRKLyUS0uolyduRySxkGix566xnX8lDGiYcLqHLp5KTKbGaLLJhP4XNbLvD2fWir4vxaYUvjJbFlTBepqNvmYt1mp5rLyXt07F1GnfCxOlXFGCs5DWzHyl1s6Uzfq1Z7TdIsBhfBFBlvLvXIzBDevsEgPV/X9BJvowGh/BA/FFeESKQnNjuKYYM2TPbGkM2brpHJMtZhYl9xgVdADzPjb4y++XAgyq6bosTuv76LzBCyiviOZe+2IHD1YTzWmndGIBpJXmTnL9DL/0hL5kbpTY1J4nl7B1PyK6cAEIlb7BM8= ceph-879f3b56-70a9-11ef-bcea-c7b262605968 2024-09-12T01:53:16.792 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-09-12T01:53:16.922 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:16 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/3268915415' entity='client.admin' 2024-09-12T01:53:17.045 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:53:17.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:17.546+0000 7f2615233640 1 -- 172.21.15.16:0/1969625305 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2610100540 msgr2=0x7f2610100940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:53:17.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:17.546+0000 7f2615233640 1 --2- 172.21.15.16:0/1969625305 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2610100540 0x7f2610100940 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f2600009880 tx=0x7f260002a5d0 comp rx=0 tx=0).stop 2024-09-12T01:53:17.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:17.546+0000 7f2615233640 1 -- 172.21.15.16:0/1969625305 shutdown_connections 2024-09-12T01:53:17.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:17.546+0000 7f2615233640 1 --2- 172.21.15.16:0/1969625305 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2610100540 0x7f2610100940 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:17.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:17.546+0000 7f2615233640 1 -- 172.21.15.16:0/1969625305 >> 172.21.15.16:0/1969625305 conn(0x7f26100fbcd0 msgr2=0x7f26100fe110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:53:17.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:17.546+0000 7f2615233640 1 -- 172.21.15.16:0/1969625305 shutdown_connections 2024-09-12T01:53:17.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:17.546+0000 7f2615233640 1 -- 172.21.15.16:0/1969625305 wait complete. 2024-09-12T01:53:17.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:17.547+0000 7f2615233640 1 Processor -- start 2024-09-12T01:53:17.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:17.547+0000 7f2615233640 1 -- start start 2024-09-12T01:53:17.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:17.547+0000 7f2615233640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2610100540 0x7f2610195c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:17.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:17.547+0000 7f2615233640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f26101065f0 con 0x7f2610100540 2024-09-12T01:53:17.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:17.547+0000 7f260ed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2610100540 0x7f2610195c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:17.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:17.547+0000 7f260ed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2610100540 0x7f2610195c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41548/0 (socket says 172.21.15.16:41548) 2024-09-12T01:53:17.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:17.547+0000 7f260ed9d640 1 -- 172.21.15.16:0/3600911206 learned_addr learned my addr 172.21.15.16:0/3600911206 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:53:17.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:17.547+0000 7f260ed9d640 1 -- 172.21.15.16:0/3600911206 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26101961a0 con 0x7f2610100540 2024-09-12T01:53:17.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:17.548+0000 7f260ed9d640 1 --2- 172.21.15.16:0/3600911206 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2610100540 0x7f2610195c60 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f2600004f60 tx=0x7f26000048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:53:17.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:17.548+0000 7f25f7fff640 1 -- 172.21.15.16:0/3600911206 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2600046070 con 0x7f2610100540 2024-09-12T01:53:17.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:17.548+0000 7f25f7fff640 1 -- 172.21.15.16:0/3600911206 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2600004cc0 con 0x7f2610100540 2024-09-12T01:53:17.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:17.548+0000 7f2615233640 1 -- 172.21.15.16:0/3600911206 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2610196430 con 0x7f2610100540 2024-09-12T01:53:17.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:17.548+0000 7f25f7fff640 1 -- 172.21.15.16:0/3600911206 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2600035070 con 0x7f2610100540 2024-09-12T01:53:17.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:17.548+0000 7f2615233640 1 -- 172.21.15.16:0/3600911206 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26100696b0 con 0x7f2610100540 2024-09-12T01:53:17.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:17.549+0000 7f25f7fff640 1 -- 172.21.15.16:0/3600911206 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f2600049070 con 0x7f2610100540 2024-09-12T01:53:17.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:17.549+0000 7f2615233640 1 -- 172.21.15.16:0/3600911206 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2610106d70 con 0x7f2610100540 2024-09-12T01:53:17.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:17.549+0000 7f25f7fff640 1 --2- 172.21.15.16:0/3600911206 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f25f8039060 0x7f25f803b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:17.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:17.549+0000 7f25f7fff640 1 -- 172.21.15.16:0/3600911206 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f260003e070 con 0x7f2610100540 2024-09-12T01:53:17.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:17.549+0000 7f260e59c640 1 --2- 172.21.15.16:0/3600911206 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f25f8039060 0x7f25f803b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:17.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:17.549+0000 7f260e59c640 1 --2- 172.21.15.16:0/3600911206 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f25f8039060 0x7f25f803b520 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f26040098c0 tx=0x7f2604006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:53:17.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:17.554+0000 7f25f7fff640 1 -- 172.21.15.16:0/3600911206 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f26000303a0 con 0x7f2610100540 2024-09-12T01:53:17.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:17.730+0000 7f2615233640 1 -- 172.21.15.16:0/3600911206 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f261006a2e0 con 0x7f2610100540 2024-09-12T01:53:17.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:17.732+0000 7f25f7fff640 1 -- 172.21.15.16:0/3600911206 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f2600050020 con 0x7f2610100540 2024-09-12T01:53:17.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:17.734+0000 7f2615233640 1 -- 172.21.15.16:0/3600911206 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f25f8039060 msgr2=0x7f25f803b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:53:17.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:17.734+0000 7f2615233640 1 --2- 172.21.15.16:0/3600911206 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f25f8039060 0x7f25f803b520 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f26040098c0 tx=0x7f2604006db0 comp rx=0 tx=0).stop 2024-09-12T01:53:17.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:17.734+0000 7f2615233640 1 -- 172.21.15.16:0/3600911206 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2610100540 msgr2=0x7f2610195c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:53:17.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:17.734+0000 7f2615233640 1 --2- 172.21.15.16:0/3600911206 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2610100540 0x7f2610195c60 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f2600004f60 tx=0x7f26000048f0 comp rx=0 tx=0).stop 2024-09-12T01:53:17.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:17.734+0000 7f2615233640 1 -- 172.21.15.16:0/3600911206 shutdown_connections 2024-09-12T01:53:17.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:17.734+0000 7f2615233640 1 --2- 172.21.15.16:0/3600911206 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f25f8039060 0x7f25f803b520 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:17.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:17.734+0000 7f2615233640 1 --2- 172.21.15.16:0/3600911206 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2610100540 0x7f2610195c60 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:17.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:17.734+0000 7f2615233640 1 -- 172.21.15.16:0/3600911206 >> 172.21.15.16:0/3600911206 conn(0x7f26100fbcd0 msgr2=0x7f26100fc6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:53:17.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:17.735+0000 7f2615233640 1 -- 172.21.15.16:0/3600911206 shutdown_connections 2024-09-12T01:53:17.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:17.735+0000 7f2615233640 1 -- 172.21.15.16:0/3600911206 wait complete. 2024-09-12T01:53:18.414 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-09-12T01:53:18.414 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-09-12T01:53:19.015 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:53:19.138 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:18 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/3600911206' entity='client.admin' 2024-09-12T01:53:19.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:19.508+0000 7f4ba8cc7640 1 -- 172.21.15.16:0/1325949387 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ba4100540 msgr2=0x7f4ba4100940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:53:19.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:19.508+0000 7f4ba8cc7640 1 --2- 172.21.15.16:0/1325949387 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ba4100540 0x7f4ba4100940 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f4b94009880 tx=0x7f4b9402a5d0 comp rx=0 tx=0).stop 2024-09-12T01:53:19.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:19.508+0000 7f4ba8cc7640 1 -- 172.21.15.16:0/1325949387 shutdown_connections 2024-09-12T01:53:19.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:19.508+0000 7f4ba8cc7640 1 --2- 172.21.15.16:0/1325949387 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ba4100540 0x7f4ba4100940 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:19.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:19.508+0000 7f4ba8cc7640 1 -- 172.21.15.16:0/1325949387 >> 172.21.15.16:0/1325949387 conn(0x7f4ba40fbcd0 msgr2=0x7f4ba40fe110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:53:19.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:19.509+0000 7f4ba8cc7640 1 -- 172.21.15.16:0/1325949387 shutdown_connections 2024-09-12T01:53:19.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:19.509+0000 7f4ba8cc7640 1 -- 172.21.15.16:0/1325949387 wait complete. 2024-09-12T01:53:19.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:19.509+0000 7f4ba8cc7640 1 Processor -- start 2024-09-12T01:53:19.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:19.510+0000 7f4ba8cc7640 1 -- start start 2024-09-12T01:53:19.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:19.510+0000 7f4ba8cc7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ba4100540 0x7f4ba418d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:19.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:19.510+0000 7f4ba8cc7640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4ba41065f0 con 0x7f4ba4100540 2024-09-12T01:53:19.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:19.510+0000 7f4ba259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ba4100540 0x7f4ba418d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:19.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:19.510+0000 7f4ba259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ba4100540 0x7f4ba418d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41576/0 (socket says 172.21.15.16:41576) 2024-09-12T01:53:19.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:19.510+0000 7f4ba259c640 1 -- 172.21.15.16:0/429426499 learned_addr learned my addr 172.21.15.16:0/429426499 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:53:19.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:19.510+0000 7f4ba259c640 1 -- 172.21.15.16:0/429426499 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ba418dc60 con 0x7f4ba4100540 2024-09-12T01:53:19.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:19.510+0000 7f4ba259c640 1 --2- 172.21.15.16:0/429426499 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ba4100540 0x7f4ba418d720 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f4b94004f60 tx=0x7f4b940048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:53:19.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:19.511+0000 7f4b9bfff640 1 -- 172.21.15.16:0/429426499 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4b9403f070 con 0x7f4ba4100540 2024-09-12T01:53:19.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:19.511+0000 7f4b9bfff640 1 -- 172.21.15.16:0/429426499 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b94004d40 con 0x7f4ba4100540 2024-09-12T01:53:19.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:19.511+0000 7f4ba8cc7640 1 -- 172.21.15.16:0/429426499 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ba418be50 con 0x7f4ba4100540 2024-09-12T01:53:19.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:19.511+0000 7f4b9bfff640 1 -- 172.21.15.16:0/429426499 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4b94035030 con 0x7f4ba4100540 2024-09-12T01:53:19.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:19.511+0000 7f4ba8cc7640 1 -- 172.21.15.16:0/429426499 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ba418c390 con 0x7f4ba4100540 2024-09-12T01:53:19.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:19.511+0000 7f4b9bfff640 1 -- 172.21.15.16:0/429426499 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f4b94042070 con 0x7f4ba4100540 2024-09-12T01:53:19.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:19.512+0000 7f4ba8cc7640 1 -- 172.21.15.16:0/429426499 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ba4106d70 con 0x7f4ba4100540 2024-09-12T01:53:19.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:19.512+0000 7f4b9bfff640 1 --2- 172.21.15.16:0/429426499 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f4b90039060 0x7f4b9003b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:19.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:19.512+0000 7f4b9bfff640 1 -- 172.21.15.16:0/429426499 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f4b9406cd40 con 0x7f4ba4100540 2024-09-12T01:53:19.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:19.512+0000 7f4b99d9b640 1 --2- 172.21.15.16:0/429426499 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f4b90039060 0x7f4b9003b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:19.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:19.512+0000 7f4b99d9b640 1 --2- 172.21.15.16:0/429426499 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f4b90039060 0x7f4b9003b520 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f4b8c0098c0 tx=0x7f4b8c006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:53:19.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:19.517+0000 7f4b9bfff640 1 -- 172.21.15.16:0/429426499 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4b94039350 con 0x7f4ba4100540 2024-09-12T01:53:19.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:19.697+0000 7f4ba8cc7640 1 -- 172.21.15.16:0/429426499 --> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7f4ba418ca50 con 0x7f4b90039060 2024-09-12T01:53:19.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:19.700+0000 7f4b9bfff640 1 -- 172.21.15.16:0/429426499 <== mgr.14150 v2:172.21.15.16:6800/3887444820 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f4ba418ca50 con 0x7f4b90039060 2024-09-12T01:53:19.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:19.703+0000 7f4ba8cc7640 1 -- 172.21.15.16:0/429426499 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f4b90039060 msgr2=0x7f4b9003b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:53:19.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:19.703+0000 7f4ba8cc7640 1 --2- 172.21.15.16:0/429426499 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f4b90039060 0x7f4b9003b520 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f4b8c0098c0 tx=0x7f4b8c006db0 comp rx=0 tx=0).stop 2024-09-12T01:53:19.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:19.703+0000 7f4ba8cc7640 1 -- 172.21.15.16:0/429426499 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ba4100540 msgr2=0x7f4ba418d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:53:19.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:19.703+0000 7f4ba8cc7640 1 --2- 172.21.15.16:0/429426499 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ba4100540 0x7f4ba418d720 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f4b94004f60 tx=0x7f4b940048f0 comp rx=0 tx=0).stop 2024-09-12T01:53:19.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:19.703+0000 7f4ba8cc7640 1 -- 172.21.15.16:0/429426499 shutdown_connections 2024-09-12T01:53:19.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:19.703+0000 7f4ba8cc7640 1 --2- 172.21.15.16:0/429426499 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f4b90039060 0x7f4b9003b520 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:19.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:19.703+0000 7f4ba8cc7640 1 --2- 172.21.15.16:0/429426499 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ba4100540 0x7f4ba418d720 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:19.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:19.703+0000 7f4ba8cc7640 1 -- 172.21.15.16:0/429426499 >> 172.21.15.16:0/429426499 conn(0x7f4ba40fbcd0 msgr2=0x7f4ba40fc8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:53:19.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:19.703+0000 7f4ba8cc7640 1 -- 172.21.15.16:0/429426499 shutdown_connections 2024-09-12T01:53:19.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:19.703+0000 7f4ba8cc7640 1 -- 172.21.15.16:0/429426499 wait complete. 2024-09-12T01:53:20.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:19 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:53:20.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:19 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:53:20.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:19 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi016", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:53:20.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:19 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:53:20.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:19 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:53:20.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:19 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:53:20.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:19 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:53:20.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:19 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:53:20.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:19 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:53:20.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:19 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:53:20.527 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi033 2024-09-12T01:53:20.528 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-09-12T01:53:20.528 DEBUG:teuthology.orchestra.run.smithi033:> dd of=/etc/ceph/ceph.conf 2024-09-12T01:53:20.547 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-09-12T01:53:20.547 DEBUG:teuthology.orchestra.run.smithi033:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-12T01:53:20.605 INFO:tasks.cephadm:Adding host smithi033 to orchestrator... 2024-09-12T01:53:20.605 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph orch host add smithi033 2024-09-12T01:53:20.876 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:53:21.153 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:20 smithi016 ceph-mon[30947]: from='client.14172 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:53:21.153 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:20 smithi016 ceph-mon[30947]: Updating smithi016:/etc/ceph/ceph.conf 2024-09-12T01:53:21.153 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:20 smithi016 ceph-mon[30947]: Updating smithi016:/var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T01:53:21.153 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:20 smithi016 ceph-mon[30947]: Updating smithi016:/etc/ceph/ceph.client.admin.keyring 2024-09-12T01:53:21.153 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:20 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:53:21.153 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:20 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:53:21.154 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:20 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:53:21.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:21.372+0000 7f5c6f44f640 1 -- 172.21.15.16:0/464642776 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c68102730 msgr2=0x7f5c68102b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:53:21.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:21.372+0000 7f5c6f44f640 1 --2- 172.21.15.16:0/464642776 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c68102730 0x7f5c68102b10 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f5c64009c10 tx=0x7f5c6402aa10 comp rx=0 tx=0).stop 2024-09-12T01:53:21.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:21.372+0000 7f5c6f44f640 1 -- 172.21.15.16:0/464642776 shutdown_connections 2024-09-12T01:53:21.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:21.372+0000 7f5c6f44f640 1 --2- 172.21.15.16:0/464642776 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c68102730 0x7f5c68102b10 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:21.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:21.372+0000 7f5c6f44f640 1 -- 172.21.15.16:0/464642776 >> 172.21.15.16:0/464642776 conn(0x7f5c68077310 msgr2=0x7f5c680fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:53:21.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:21.372+0000 7f5c6f44f640 1 -- 172.21.15.16:0/464642776 shutdown_connections 2024-09-12T01:53:21.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:21.372+0000 7f5c6f44f640 1 -- 172.21.15.16:0/464642776 wait complete. 2024-09-12T01:53:21.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:21.372+0000 7f5c6f44f640 1 Processor -- start 2024-09-12T01:53:21.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:21.373+0000 7f5c6f44f640 1 -- start start 2024-09-12T01:53:21.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:21.373+0000 7f5c6f44f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c68102730 0x7f5c681971b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:21.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:21.373+0000 7f5c6f44f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5c6810bc80 con 0x7f5c68102730 2024-09-12T01:53:21.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:21.373+0000 7f5c6d1eb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c68102730 0x7f5c681971b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:21.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:21.373+0000 7f5c6d1eb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c68102730 0x7f5c681971b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38596/0 (socket says 172.21.15.16:38596) 2024-09-12T01:53:21.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:21.373+0000 7f5c6d1eb640 1 -- 172.21.15.16:0/2335403268 learned_addr learned my addr 172.21.15.16:0/2335403268 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:53:21.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:21.373+0000 7f5c6d1eb640 1 -- 172.21.15.16:0/2335403268 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c681976f0 con 0x7f5c68102730 2024-09-12T01:53:21.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:21.373+0000 7f5c6d1eb640 1 --2- 172.21.15.16:0/2335403268 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c68102730 0x7f5c681971b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f5c64005dd0 tx=0x7f5c64030c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:53:21.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:21.373+0000 7f5c5e7fc640 1 -- 172.21.15.16:0/2335403268 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5c64034070 con 0x7f5c68102730 2024-09-12T01:53:21.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:21.373+0000 7f5c5e7fc640 1 -- 172.21.15.16:0/2335403268 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c6402f8c0 con 0x7f5c68102730 2024-09-12T01:53:21.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:21.374+0000 7f5c6f44f640 1 -- 172.21.15.16:0/2335403268 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c68197980 con 0x7f5c68102730 2024-09-12T01:53:21.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:21.374+0000 7f5c5e7fc640 1 -- 172.21.15.16:0/2335403268 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5c64004520 con 0x7f5c68102730 2024-09-12T01:53:21.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:21.374+0000 7f5c6f44f640 1 -- 172.21.15.16:0/2335403268 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c68197d20 con 0x7f5c68102730 2024-09-12T01:53:21.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:21.374+0000 7f5c5e7fc640 1 -- 172.21.15.16:0/2335403268 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f5c64037020 con 0x7f5c68102730 2024-09-12T01:53:21.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:21.375+0000 7f5c6f44f640 1 -- 172.21.15.16:0/2335403268 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c68069f90 con 0x7f5c68102730 2024-09-12T01:53:21.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:21.375+0000 7f5c5e7fc640 1 --2- 172.21.15.16:0/2335403268 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f5c54039060 0x7f5c5403b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:21.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:21.375+0000 7f5c5e7fc640 1 -- 172.21.15.16:0/2335403268 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f5c6406cef0 con 0x7f5c68102730 2024-09-12T01:53:21.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:21.375+0000 7f5c6c9ea640 1 --2- 172.21.15.16:0/2335403268 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f5c54039060 0x7f5c5403b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:21.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:21.375+0000 7f5c6c9ea640 1 --2- 172.21.15.16:0/2335403268 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f5c54039060 0x7f5c5403b520 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f5c580098c0 tx=0x7f5c58006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:53:21.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:21.379+0000 7f5c5e7fc640 1 -- 172.21.15.16:0/2335403268 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5c6403f070 con 0x7f5c68102730 2024-09-12T01:53:21.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:21.560+0000 7f5c6f44f640 1 -- 172.21.15.16:0/2335403268 --> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi033", "target": ["mon-mgr", ""]}) -- 0x7f5c68198580 con 0x7f5c54039060 2024-09-12T01:53:22.183 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:21 smithi016 ceph-mon[30947]: Updating smithi016:/var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-12T01:53:23.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:22 smithi016 ceph-mon[30947]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi033", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:53:24.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:23 smithi016 ceph-mon[30947]: Deploying cephadm binary to smithi033 2024-09-12T01:53:25.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:25.499+0000 7f5c5e7fc640 1 -- 172.21.15.16:0/2335403268 <== mgr.14150 v2:172.21.15.16:6800/3887444820 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f5c68198580 con 0x7f5c54039060 2024-09-12T01:53:25.500 INFO:teuthology.orchestra.run.smithi016.stdout:Added host 'smithi033' with addr '172.21.15.33' 2024-09-12T01:53:25.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:25.501+0000 7f5c6f44f640 1 -- 172.21.15.16:0/2335403268 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f5c54039060 msgr2=0x7f5c5403b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:53:25.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:25.501+0000 7f5c6f44f640 1 --2- 172.21.15.16:0/2335403268 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f5c54039060 0x7f5c5403b520 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f5c580098c0 tx=0x7f5c58006db0 comp rx=0 tx=0).stop 2024-09-12T01:53:25.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:25.501+0000 7f5c6f44f640 1 -- 172.21.15.16:0/2335403268 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c68102730 msgr2=0x7f5c681971b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:53:25.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:25.501+0000 7f5c6f44f640 1 --2- 172.21.15.16:0/2335403268 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c68102730 0x7f5c681971b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f5c64005dd0 tx=0x7f5c64030c30 comp rx=0 tx=0).stop 2024-09-12T01:53:25.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:25.501+0000 7f5c6f44f640 1 -- 172.21.15.16:0/2335403268 shutdown_connections 2024-09-12T01:53:25.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:25.501+0000 7f5c6f44f640 1 --2- 172.21.15.16:0/2335403268 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f5c54039060 0x7f5c5403b520 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:25.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:25.501+0000 7f5c6f44f640 1 --2- 172.21.15.16:0/2335403268 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c68102730 0x7f5c681971b0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:25.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:25.502+0000 7f5c6f44f640 1 -- 172.21.15.16:0/2335403268 >> 172.21.15.16:0/2335403268 conn(0x7f5c68077310 msgr2=0x7f5c681054c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:53:25.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:25.502+0000 7f5c6f44f640 1 -- 172.21.15.16:0/2335403268 shutdown_connections 2024-09-12T01:53:25.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:25.502+0000 7f5c6f44f640 1 -- 172.21.15.16:0/2335403268 wait complete. 2024-09-12T01:53:26.258 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph orch host ls --format=json 2024-09-12T01:53:26.514 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:53:26.787 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:26 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:53:26.787 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:26 smithi016 ceph-mon[30947]: Added host smithi033 2024-09-12T01:53:26.787 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:26 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:53:26.788 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:26 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:53:26.788 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:26 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:53:26.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:26.994+0000 7f27dc8cb640 1 -- 172.21.15.16:0/2506884800 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27d8102750 msgr2=0x7f27d8102b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:53:26.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:26.994+0000 7f27dc8cb640 1 --2- 172.21.15.16:0/2506884800 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27d8102750 0x7f27d8102b30 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f27c8009c20 tx=0x7f27c802ab20 comp rx=0 tx=0).stop 2024-09-12T01:53:26.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:26.994+0000 7f27dc8cb640 1 -- 172.21.15.16:0/2506884800 shutdown_connections 2024-09-12T01:53:26.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:26.994+0000 7f27dc8cb640 1 --2- 172.21.15.16:0/2506884800 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27d8102750 0x7f27d8102b30 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:26.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:26.994+0000 7f27dc8cb640 1 -- 172.21.15.16:0/2506884800 >> 172.21.15.16:0/2506884800 conn(0x7f27d80fc1b0 msgr2=0x7f27d80fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:53:26.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:26.994+0000 7f27dc8cb640 1 -- 172.21.15.16:0/2506884800 shutdown_connections 2024-09-12T01:53:26.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:26.994+0000 7f27dc8cb640 1 -- 172.21.15.16:0/2506884800 wait complete. 2024-09-12T01:53:26.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:26.995+0000 7f27dc8cb640 1 Processor -- start 2024-09-12T01:53:26.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:26.995+0000 7f27dc8cb640 1 -- start start 2024-09-12T01:53:26.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:26.995+0000 7f27dc8cb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27d8102750 0x7f27d81971d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:26.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:26.995+0000 7f27dc8cb640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f27d810bcc0 con 0x7f27d8102750 2024-09-12T01:53:26.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:26.995+0000 7f27d659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27d8102750 0x7f27d81971d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:26.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:26.996+0000 7f27d659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27d8102750 0x7f27d81971d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38614/0 (socket says 172.21.15.16:38614) 2024-09-12T01:53:26.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:26.996+0000 7f27d659c640 1 -- 172.21.15.16:0/926331945 learned_addr learned my addr 172.21.15.16:0/926331945 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:53:26.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:26.996+0000 7f27d659c640 1 -- 172.21.15.16:0/926331945 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27d8197710 con 0x7f27d8102750 2024-09-12T01:53:26.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:26.996+0000 7f27d659c640 1 --2- 172.21.15.16:0/926331945 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27d8102750 0x7f27d81971d0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f27c8030040 tx=0x7f27c802eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:53:26.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:26.996+0000 7f27cf7fe640 1 -- 172.21.15.16:0/926331945 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f27c8047070 con 0x7f27d8102750 2024-09-12T01:53:26.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:26.996+0000 7f27cf7fe640 1 -- 172.21.15.16:0/926331945 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27c80368e0 con 0x7f27d8102750 2024-09-12T01:53:26.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:26.996+0000 7f27dc8cb640 1 -- 172.21.15.16:0/926331945 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27d81979a0 con 0x7f27d8102750 2024-09-12T01:53:26.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:26.996+0000 7f27cf7fe640 1 -- 172.21.15.16:0/926331945 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f27c8042420 con 0x7f27d8102750 2024-09-12T01:53:26.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:26.996+0000 7f27dc8cb640 1 -- 172.21.15.16:0/926331945 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27d8197e00 con 0x7f27d8102750 2024-09-12T01:53:26.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:26.997+0000 7f27cf7fe640 1 -- 172.21.15.16:0/926331945 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f27c804a070 con 0x7f27d8102750 2024-09-12T01:53:26.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:26.997+0000 7f27dc8cb640 1 -- 172.21.15.16:0/926331945 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27d806a7b0 con 0x7f27d8102750 2024-09-12T01:53:26.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:26.997+0000 7f27cf7fe640 1 --2- 172.21.15.16:0/926331945 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f27c40390c0 0x7f27c403b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:26.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:26.997+0000 7f27cf7fe640 1 -- 172.21.15.16:0/926331945 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f27c803d030 con 0x7f27d8102750 2024-09-12T01:53:26.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:26.997+0000 7f27d5d9b640 1 --2- 172.21.15.16:0/926331945 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f27c40390c0 0x7f27c403b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:26.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:26.998+0000 7f27d5d9b640 1 --2- 172.21.15.16:0/926331945 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f27c40390c0 0x7f27c403b580 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f27c00098c0 tx=0x7f27c0006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:53:27.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:27.002+0000 7f27cf7fe640 1 -- 172.21.15.16:0/926331945 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f27c8032030 con 0x7f27d8102750 2024-09-12T01:53:27.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:27.182+0000 7f27dc8cb640 1 -- 172.21.15.16:0/926331945 --> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f27d8198590 con 0x7f27c40390c0 2024-09-12T01:53:27.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:27.183+0000 7f27cf7fe640 1 -- 172.21.15.16:0/926331945 <== mgr.14150 v2:172.21.15.16:6800/3887444820 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+159 (secure 0 0 0) 0x7f27d8198590 con 0x7f27c40390c0 2024-09-12T01:53:27.183 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:53:27.183 INFO:teuthology.orchestra.run.smithi016.stdout:[{"addr": "172.21.15.16", "hostname": "smithi016", "labels": [], "status": ""}, {"addr": "172.21.15.33", "hostname": "smithi033", "labels": [], "status": ""}] 2024-09-12T01:53:27.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:27.185+0000 7f27dc8cb640 1 -- 172.21.15.16:0/926331945 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f27c40390c0 msgr2=0x7f27c403b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:53:27.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:27.185+0000 7f27dc8cb640 1 --2- 172.21.15.16:0/926331945 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f27c40390c0 0x7f27c403b580 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f27c00098c0 tx=0x7f27c0006db0 comp rx=0 tx=0).stop 2024-09-12T01:53:27.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:27.185+0000 7f27dc8cb640 1 -- 172.21.15.16:0/926331945 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27d8102750 msgr2=0x7f27d81971d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:53:27.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:27.185+0000 7f27dc8cb640 1 --2- 172.21.15.16:0/926331945 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27d8102750 0x7f27d81971d0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f27c8030040 tx=0x7f27c802eb50 comp rx=0 tx=0).stop 2024-09-12T01:53:27.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:27.185+0000 7f27dc8cb640 1 -- 172.21.15.16:0/926331945 shutdown_connections 2024-09-12T01:53:27.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:27.185+0000 7f27dc8cb640 1 --2- 172.21.15.16:0/926331945 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f27c40390c0 0x7f27c403b580 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:27.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:27.185+0000 7f27dc8cb640 1 --2- 172.21.15.16:0/926331945 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27d8102750 0x7f27d81971d0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:27.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:27.185+0000 7f27dc8cb640 1 -- 172.21.15.16:0/926331945 >> 172.21.15.16:0/926331945 conn(0x7f27d80fc1b0 msgr2=0x7f27d8105510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:53:27.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:27.185+0000 7f27dc8cb640 1 -- 172.21.15.16:0/926331945 shutdown_connections 2024-09-12T01:53:27.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:27.185+0000 7f27dc8cb640 1 -- 172.21.15.16:0/926331945 wait complete. 2024-09-12T01:53:27.757 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi184 2024-09-12T01:53:27.757 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-12T01:53:27.757 DEBUG:teuthology.orchestra.run.smithi184:> dd of=/etc/ceph/ceph.conf 2024-09-12T01:53:27.777 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-12T01:53:27.778 DEBUG:teuthology.orchestra.run.smithi184:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-12T01:53:27.836 INFO:tasks.cephadm:Adding host smithi184 to orchestrator... 2024-09-12T01:53:27.837 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph orch host add smithi184 2024-09-12T01:53:28.094 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:53:28.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:27 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:53:28.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:27 smithi016 ceph-mon[30947]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T01:53:28.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:28.544+0000 7fbe9b0f8640 1 -- 172.21.15.16:0/1594178125 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe94106560 msgr2=0x7fbe94106940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:53:28.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:28.544+0000 7fbe9b0f8640 1 --2- 172.21.15.16:0/1594178125 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe94106560 0x7fbe94106940 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fbe88009880 tx=0x7fbe8802a5d0 comp rx=0 tx=0).stop 2024-09-12T01:53:28.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:28.544+0000 7fbe9b0f8640 1 -- 172.21.15.16:0/1594178125 shutdown_connections 2024-09-12T01:53:28.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:28.544+0000 7fbe9b0f8640 1 --2- 172.21.15.16:0/1594178125 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe94106560 0x7fbe94106940 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:28.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:28.544+0000 7fbe9b0f8640 1 -- 172.21.15.16:0/1594178125 >> 172.21.15.16:0/1594178125 conn(0x7fbe940fc1b0 msgr2=0x7fbe940fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:53:28.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:28.544+0000 7fbe9b0f8640 1 -- 172.21.15.16:0/1594178125 shutdown_connections 2024-09-12T01:53:28.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:28.544+0000 7fbe9b0f8640 1 -- 172.21.15.16:0/1594178125 wait complete. 2024-09-12T01:53:28.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:28.544+0000 7fbe9b0f8640 1 Processor -- start 2024-09-12T01:53:28.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:28.545+0000 7fbe9b0f8640 1 -- start start 2024-09-12T01:53:28.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:28.545+0000 7fbe9b0f8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe94106560 0x7fbe940fc660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:28.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:28.545+0000 7fbe9b0f8640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbe9410ab90 con 0x7fbe94106560 2024-09-12T01:53:28.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:28.545+0000 7fbe98e94640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe94106560 0x7fbe940fc660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:28.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:28.545+0000 7fbe98e94640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe94106560 0x7fbe940fc660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38638/0 (socket says 172.21.15.16:38638) 2024-09-12T01:53:28.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:28.545+0000 7fbe98e94640 1 -- 172.21.15.16:0/3194729187 learned_addr learned my addr 172.21.15.16:0/3194729187 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:53:28.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:28.545+0000 7fbe98e94640 1 -- 172.21.15.16:0/3194729187 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe940fe550 con 0x7fbe94106560 2024-09-12T01:53:28.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:28.545+0000 7fbe98e94640 1 --2- 172.21.15.16:0/3194729187 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe94106560 0x7fbe940fc660 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fbe88009230 tx=0x7fbe88003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:53:28.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:28.546+0000 7fbe91ffb640 1 -- 172.21.15.16:0/3194729187 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbe8803f070 con 0x7fbe94106560 2024-09-12T01:53:28.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:28.546+0000 7fbe91ffb640 1 -- 172.21.15.16:0/3194729187 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe8803bc80 con 0x7fbe94106560 2024-09-12T01:53:28.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:28.546+0000 7fbe9b0f8640 1 -- 172.21.15.16:0/3194729187 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe940fcba0 con 0x7fbe94106560 2024-09-12T01:53:28.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:28.546+0000 7fbe91ffb640 1 -- 172.21.15.16:0/3194729187 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbe88035030 con 0x7fbe94106560 2024-09-12T01:53:28.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:28.546+0000 7fbe9b0f8640 1 -- 172.21.15.16:0/3194729187 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe940fd060 con 0x7fbe94106560 2024-09-12T01:53:28.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:28.547+0000 7fbe91ffb640 1 -- 172.21.15.16:0/3194729187 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fbe88042070 con 0x7fbe94106560 2024-09-12T01:53:28.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:28.547+0000 7fbe9b0f8640 1 -- 172.21.15.16:0/3194729187 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe94101bd0 con 0x7fbe94106560 2024-09-12T01:53:28.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:28.547+0000 7fbe91ffb640 1 --2- 172.21.15.16:0/3194729187 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fbe840390c0 0x7fbe8403b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:28.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:28.547+0000 7fbe91ffb640 1 -- 172.21.15.16:0/3194729187 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fbe8806ce00 con 0x7fbe94106560 2024-09-12T01:53:28.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:28.547+0000 7fbe93fff640 1 --2- 172.21.15.16:0/3194729187 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fbe840390c0 0x7fbe8403b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:28.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:28.547+0000 7fbe93fff640 1 --2- 172.21.15.16:0/3194729187 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fbe840390c0 0x7fbe8403b580 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fbe800098c0 tx=0x7fbe80006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:53:28.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:28.552+0000 7fbe91ffb640 1 -- 172.21.15.16:0/3194729187 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbe880314c0 con 0x7fbe94106560 2024-09-12T01:53:28.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:28.729+0000 7fbe9b0f8640 1 -- 172.21.15.16:0/3194729187 --> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi184", "target": ["mon-mgr", ""]}) -- 0x7fbe940fdb90 con 0x7fbe840390c0 2024-09-12T01:53:29.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:29 smithi016 ceph-mon[30947]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:53:29.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:29 smithi016 ceph-mon[30947]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi184", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:53:31.183 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:30 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:53:31.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:30 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:53:31.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:30 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:53:31.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:30 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:53:31.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:30 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi033", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:53:31.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:30 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:53:31.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:30 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:53:31.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:30 smithi016 ceph-mon[30947]: Updating smithi033:/etc/ceph/ceph.conf 2024-09-12T01:53:31.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:30 smithi016 ceph-mon[30947]: Deploying cephadm binary to smithi184 2024-09-12T01:53:31.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:30 smithi016 ceph-mon[30947]: Updating smithi033:/var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T01:53:31.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:30 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:53:31.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:30 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:53:31.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:30 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:53:32.183 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:31 smithi016 ceph-mon[30947]: Updating smithi033:/etc/ceph/ceph.client.admin.keyring 2024-09-12T01:53:32.183 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:31 smithi016 ceph-mon[30947]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:53:32.183 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:31 smithi016 ceph-mon[30947]: Updating smithi033:/var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-12T01:53:32.634 INFO:teuthology.orchestra.run.smithi016.stdout:Added host 'smithi184' with addr '172.21.15.184' 2024-09-12T01:53:32.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:32.633+0000 7fbe91ffb640 1 -- 172.21.15.16:0/3194729187 <== mgr.14150 v2:172.21.15.16:6800/3887444820 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7fbe940fdb90 con 0x7fbe840390c0 2024-09-12T01:53:32.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:32.635+0000 7fbe9b0f8640 1 -- 172.21.15.16:0/3194729187 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fbe840390c0 msgr2=0x7fbe8403b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:53:32.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:32.635+0000 7fbe9b0f8640 1 --2- 172.21.15.16:0/3194729187 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fbe840390c0 0x7fbe8403b580 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fbe800098c0 tx=0x7fbe80006db0 comp rx=0 tx=0).stop 2024-09-12T01:53:32.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:32.635+0000 7fbe9b0f8640 1 -- 172.21.15.16:0/3194729187 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe94106560 msgr2=0x7fbe940fc660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:53:32.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:32.635+0000 7fbe9b0f8640 1 --2- 172.21.15.16:0/3194729187 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe94106560 0x7fbe940fc660 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fbe88009230 tx=0x7fbe88003d40 comp rx=0 tx=0).stop 2024-09-12T01:53:32.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:32.636+0000 7fbe9b0f8640 1 -- 172.21.15.16:0/3194729187 shutdown_connections 2024-09-12T01:53:32.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:32.636+0000 7fbe9b0f8640 1 --2- 172.21.15.16:0/3194729187 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fbe840390c0 0x7fbe8403b580 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:32.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:32.636+0000 7fbe9b0f8640 1 --2- 172.21.15.16:0/3194729187 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe94106560 0x7fbe940fc660 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:32.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:32.636+0000 7fbe9b0f8640 1 -- 172.21.15.16:0/3194729187 >> 172.21.15.16:0/3194729187 conn(0x7fbe940fc1b0 msgr2=0x7fbe94069ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:53:32.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:32.636+0000 7fbe9b0f8640 1 -- 172.21.15.16:0/3194729187 shutdown_connections 2024-09-12T01:53:32.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:32.636+0000 7fbe9b0f8640 1 -- 172.21.15.16:0/3194729187 wait complete. 2024-09-12T01:53:33.265 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph orch host ls --format=json 2024-09-12T01:53:33.513 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:53:33.933 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:33 smithi016 ceph-mon[30947]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:53:33.933 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:33 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:53:33.934 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:33 smithi016 ceph-mon[30947]: Added host smithi184 2024-09-12T01:53:33.934 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:33 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:53:33.934 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:33 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:53:33.934 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:33 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:53:34.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:33.955+0000 7f8b590ec640 1 -- 172.21.15.16:0/134709165 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b54106540 msgr2=0x7f8b54106920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:53:34.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:33.955+0000 7f8b590ec640 1 --2- 172.21.15.16:0/134709165 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b54106540 0x7f8b54106920 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f8b44009810 tx=0x7f8b4402a540 comp rx=0 tx=0).stop 2024-09-12T01:53:34.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:33.955+0000 7f8b590ec640 1 -- 172.21.15.16:0/134709165 shutdown_connections 2024-09-12T01:53:34.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:33.955+0000 7f8b590ec640 1 --2- 172.21.15.16:0/134709165 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b54106540 0x7f8b54106920 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:34.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:33.955+0000 7f8b590ec640 1 -- 172.21.15.16:0/134709165 >> 172.21.15.16:0/134709165 conn(0x7f8b54077310 msgr2=0x7f8b540fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:53:34.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:33.956+0000 7f8b590ec640 1 -- 172.21.15.16:0/134709165 shutdown_connections 2024-09-12T01:53:34.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:33.956+0000 7f8b590ec640 1 -- 172.21.15.16:0/134709165 wait complete. 2024-09-12T01:53:34.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:33.956+0000 7f8b590ec640 1 Processor -- start 2024-09-12T01:53:34.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:33.956+0000 7f8b590ec640 1 -- start start 2024-09-12T01:53:34.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:33.957+0000 7f8b590ec640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b54106540 0x7f8b54199300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:34.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:33.957+0000 7f8b590ec640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8b5410ab70 con 0x7f8b54106540 2024-09-12T01:53:34.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:33.957+0000 7f8b52d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b54106540 0x7f8b54199300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:34.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:33.957+0000 7f8b52d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b54106540 0x7f8b54199300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37174/0 (socket says 172.21.15.16:37174) 2024-09-12T01:53:34.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:33.957+0000 7f8b52d9d640 1 -- 172.21.15.16:0/573042219 learned_addr learned my addr 172.21.15.16:0/573042219 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:53:34.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:33.957+0000 7f8b52d9d640 1 -- 172.21.15.16:0/573042219 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b54199840 con 0x7f8b54106540 2024-09-12T01:53:34.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:33.957+0000 7f8b52d9d640 1 --2- 172.21.15.16:0/573042219 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b54106540 0x7f8b54199300 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f8b440045e0 tx=0x7f8b44004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:53:34.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:33.957+0000 7f8b3bfff640 1 -- 172.21.15.16:0/573042219 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8b44030e70 con 0x7f8b54106540 2024-09-12T01:53:34.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:33.957+0000 7f8b3bfff640 1 -- 172.21.15.16:0/573042219 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b440318e0 con 0x7f8b54106540 2024-09-12T01:53:34.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:33.957+0000 7f8b590ec640 1 -- 172.21.15.16:0/573042219 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b54199ad0 con 0x7f8b54106540 2024-09-12T01:53:34.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:33.957+0000 7f8b3bfff640 1 -- 172.21.15.16:0/573042219 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8b440049c0 con 0x7f8b54106540 2024-09-12T01:53:34.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:33.957+0000 7f8b590ec640 1 -- 172.21.15.16:0/573042219 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b5419dbd0 con 0x7f8b54106540 2024-09-12T01:53:34.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:33.958+0000 7f8b3bfff640 1 -- 172.21.15.16:0/573042219 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f8b44039040 con 0x7f8b54106540 2024-09-12T01:53:34.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:33.958+0000 7f8b590ec640 1 -- 172.21.15.16:0/573042219 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b54101bb0 con 0x7f8b54106540 2024-09-12T01:53:34.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:33.959+0000 7f8b3bfff640 1 --2- 172.21.15.16:0/573042219 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f8b3c039060 0x7f8b3c03b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:34.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:33.959+0000 7f8b3bfff640 1 -- 172.21.15.16:0/573042219 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f8b4406d290 con 0x7f8b54106540 2024-09-12T01:53:34.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:33.959+0000 7f8b5259c640 1 --2- 172.21.15.16:0/573042219 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f8b3c039060 0x7f8b3c03b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:34.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:33.959+0000 7f8b5259c640 1 --2- 172.21.15.16:0/573042219 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f8b3c039060 0x7f8b3c03b520 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f8b480098c0 tx=0x7f8b48006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:53:34.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:33.963+0000 7f8b3bfff640 1 -- 172.21.15.16:0/573042219 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8b44040030 con 0x7f8b54106540 2024-09-12T01:53:34.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:34.145+0000 7f8b590ec640 1 -- 172.21.15.16:0/573042219 --> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8b5419a660 con 0x7f8b3c039060 2024-09-12T01:53:34.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:34.146+0000 7f8b3bfff640 1 -- 172.21.15.16:0/573042219 <== mgr.14150 v2:172.21.15.16:6800/3887444820 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+239 (secure 0 0 0) 0x7f8b5419a660 con 0x7f8b3c039060 2024-09-12T01:53:34.146 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:53:34.147 INFO:teuthology.orchestra.run.smithi016.stdout:[{"addr": "172.21.15.16", "hostname": "smithi016", "labels": [], "status": ""}, {"addr": "172.21.15.33", "hostname": "smithi033", "labels": [], "status": ""}, {"addr": "172.21.15.184", "hostname": "smithi184", "labels": [], "status": ""}] 2024-09-12T01:53:34.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:34.148+0000 7f8b590ec640 1 -- 172.21.15.16:0/573042219 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f8b3c039060 msgr2=0x7f8b3c03b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:53:34.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:34.148+0000 7f8b590ec640 1 --2- 172.21.15.16:0/573042219 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f8b3c039060 0x7f8b3c03b520 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f8b480098c0 tx=0x7f8b48006db0 comp rx=0 tx=0).stop 2024-09-12T01:53:34.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:34.148+0000 7f8b590ec640 1 -- 172.21.15.16:0/573042219 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b54106540 msgr2=0x7f8b54199300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:53:34.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:34.148+0000 7f8b590ec640 1 --2- 172.21.15.16:0/573042219 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b54106540 0x7f8b54199300 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f8b440045e0 tx=0x7f8b44004610 comp rx=0 tx=0).stop 2024-09-12T01:53:34.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:34.148+0000 7f8b590ec640 1 -- 172.21.15.16:0/573042219 shutdown_connections 2024-09-12T01:53:34.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:34.148+0000 7f8b590ec640 1 --2- 172.21.15.16:0/573042219 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f8b3c039060 0x7f8b3c03b520 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:34.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:34.148+0000 7f8b590ec640 1 --2- 172.21.15.16:0/573042219 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b54106540 0x7f8b54199300 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:34.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:34.148+0000 7f8b590ec640 1 -- 172.21.15.16:0/573042219 >> 172.21.15.16:0/573042219 conn(0x7f8b54077310 msgr2=0x7f8b540fdcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:53:34.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:34.149+0000 7f8b590ec640 1 -- 172.21.15.16:0/573042219 shutdown_connections 2024-09-12T01:53:34.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:34.149+0000 7f8b590ec640 1 -- 172.21.15.16:0/573042219 wait complete. 2024-09-12T01:53:34.722 INFO:tasks.cephadm:Setting crush tunables to default 2024-09-12T01:53:34.723 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph osd crush tunables default 2024-09-12T01:53:34.971 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:53:35.423 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:34 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:53:35.423 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:34 smithi016 ceph-mon[30947]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T01:53:35.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:35.507+0000 7fcda9265640 1 -- 172.21.15.16:0/1270809310 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcda4106560 msgr2=0x7fcda4106940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:53:35.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:35.507+0000 7fcda9265640 1 --2- 172.21.15.16:0/1270809310 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcda4106560 0x7fcda4106940 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fcd94009880 tx=0x7fcd9402a5d0 comp rx=0 tx=0).stop 2024-09-12T01:53:35.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:35.507+0000 7fcda9265640 1 -- 172.21.15.16:0/1270809310 shutdown_connections 2024-09-12T01:53:35.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:35.507+0000 7fcda9265640 1 --2- 172.21.15.16:0/1270809310 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcda4106560 0x7fcda4106940 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:35.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:35.507+0000 7fcda9265640 1 -- 172.21.15.16:0/1270809310 >> 172.21.15.16:0/1270809310 conn(0x7fcda40fc1b0 msgr2=0x7fcda40fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:53:35.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:35.507+0000 7fcda9265640 1 -- 172.21.15.16:0/1270809310 shutdown_connections 2024-09-12T01:53:35.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:35.507+0000 7fcda9265640 1 -- 172.21.15.16:0/1270809310 wait complete. 2024-09-12T01:53:35.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:35.508+0000 7fcda9265640 1 Processor -- start 2024-09-12T01:53:35.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:35.508+0000 7fcda9265640 1 -- start start 2024-09-12T01:53:35.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:35.508+0000 7fcda9265640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcda4106560 0x7fcda4197100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:35.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:35.508+0000 7fcda9265640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fcda410ab90 con 0x7fcda4106560 2024-09-12T01:53:35.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:35.508+0000 7fcda2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcda4106560 0x7fcda4197100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:35.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:35.508+0000 7fcda2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcda4106560 0x7fcda4197100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37196/0 (socket says 172.21.15.16:37196) 2024-09-12T01:53:35.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:35.508+0000 7fcda2ffd640 1 -- 172.21.15.16:0/3382167872 learned_addr learned my addr 172.21.15.16:0/3382167872 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:53:35.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:35.508+0000 7fcda2ffd640 1 -- 172.21.15.16:0/3382167872 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcda4197640 con 0x7fcda4106560 2024-09-12T01:53:35.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:35.508+0000 7fcda2ffd640 1 --2- 172.21.15.16:0/3382167872 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcda4106560 0x7fcda4197100 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fcd94009230 tx=0x7fcd94003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:53:35.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:35.509+0000 7fcda0ff9640 1 -- 172.21.15.16:0/3382167872 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcd9403f070 con 0x7fcda4106560 2024-09-12T01:53:35.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:35.509+0000 7fcda0ff9640 1 -- 172.21.15.16:0/3382167872 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd9403bc80 con 0x7fcda4106560 2024-09-12T01:53:35.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:35.509+0000 7fcda9265640 1 -- 172.21.15.16:0/3382167872 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcda41978d0 con 0x7fcda4106560 2024-09-12T01:53:35.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:35.509+0000 7fcda0ff9640 1 -- 172.21.15.16:0/3382167872 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcd94035030 con 0x7fcda4106560 2024-09-12T01:53:35.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:35.509+0000 7fcda9265640 1 -- 172.21.15.16:0/3382167872 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcda419b9d0 con 0x7fcda4106560 2024-09-12T01:53:35.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:35.510+0000 7fcda0ff9640 1 -- 172.21.15.16:0/3382167872 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fcd94042070 con 0x7fcda4106560 2024-09-12T01:53:35.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:35.510+0000 7fcda9265640 1 -- 172.21.15.16:0/3382167872 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcda4101bd0 con 0x7fcda4106560 2024-09-12T01:53:35.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:35.510+0000 7fcda0ff9640 1 --2- 172.21.15.16:0/3382167872 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fcd900390c0 0x7fcd9003b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:35.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:35.510+0000 7fcda0ff9640 1 -- 172.21.15.16:0/3382167872 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fcd9406c950 con 0x7fcda4106560 2024-09-12T01:53:35.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:35.510+0000 7fcd9bfff640 1 --2- 172.21.15.16:0/3382167872 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fcd900390c0 0x7fcd9003b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:35.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:35.510+0000 7fcd9bfff640 1 --2- 172.21.15.16:0/3382167872 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fcd900390c0 0x7fcd9003b580 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fcd8c0098c0 tx=0x7fcd8c006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:53:35.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:35.515+0000 7fcda0ff9640 1 -- 172.21.15.16:0/3382167872 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcd940372d0 con 0x7fcda4106560 2024-09-12T01:53:35.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:35.695+0000 7fcda9265640 1 -- 172.21.15.16:0/3382167872 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7fcda41069c0 con 0x7fcda4106560 2024-09-12T01:53:35.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:35.973+0000 7fcda0ff9640 1 -- 172.21.15.16:0/3382167872 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7fcd94031570 con 0x7fcda4106560 2024-09-12T01:53:35.973 INFO:teuthology.orchestra.run.smithi016.stderr:adjusted tunables profile to default 2024-09-12T01:53:35.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:35.975+0000 7fcda9265640 1 -- 172.21.15.16:0/3382167872 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fcd900390c0 msgr2=0x7fcd9003b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:53:35.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:35.975+0000 7fcda9265640 1 --2- 172.21.15.16:0/3382167872 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fcd900390c0 0x7fcd9003b580 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fcd8c0098c0 tx=0x7fcd8c006db0 comp rx=0 tx=0).stop 2024-09-12T01:53:35.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:35.975+0000 7fcda9265640 1 -- 172.21.15.16:0/3382167872 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcda4106560 msgr2=0x7fcda4197100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:53:35.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:35.975+0000 7fcda9265640 1 --2- 172.21.15.16:0/3382167872 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcda4106560 0x7fcda4197100 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fcd94009230 tx=0x7fcd94003d40 comp rx=0 tx=0).stop 2024-09-12T01:53:35.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:35.975+0000 7fcda9265640 1 -- 172.21.15.16:0/3382167872 shutdown_connections 2024-09-12T01:53:35.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:35.975+0000 7fcda9265640 1 --2- 172.21.15.16:0/3382167872 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fcd900390c0 0x7fcd9003b580 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:35.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:35.975+0000 7fcda9265640 1 --2- 172.21.15.16:0/3382167872 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcda4106560 0x7fcda4197100 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:35.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:35.975+0000 7fcda9265640 1 -- 172.21.15.16:0/3382167872 >> 172.21.15.16:0/3382167872 conn(0x7fcda40fc1b0 msgr2=0x7fcda4069ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:53:35.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:35.975+0000 7fcda9265640 1 -- 172.21.15.16:0/3382167872 shutdown_connections 2024-09-12T01:53:35.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:35.975+0000 7fcda9265640 1 -- 172.21.15.16:0/3382167872 wait complete. 2024-09-12T01:53:36.267 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:35 smithi016 ceph-mon[30947]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:53:36.267 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:35 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/3382167872' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-12T01:53:36.564 INFO:tasks.cephadm:Adding mon.a on smithi016 2024-09-12T01:53:36.564 INFO:tasks.cephadm:Adding mon.b on smithi033 2024-09-12T01:53:36.564 INFO:tasks.cephadm:Adding mon.c on smithi184 2024-09-12T01:53:36.564 DEBUG:teuthology.orchestra.run.smithi184:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph orch apply mon '3;smithi016:172.21.15.16=a;smithi033:172.21.15.33=b;smithi184:172.21.15.184=c' 2024-09-12T01:53:36.808 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-12T01:53:36.859 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-12T01:53:37.339 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:37.288+0000 7f1864dc7640 1 -- 172.21.15.184:0/2569122208 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1860102c20 msgr2=0x7f1860103020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:53:37.339 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:37.288+0000 7f1864dc7640 1 --2- 172.21.15.184:0/2569122208 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1860102c20 0x7f1860103020 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1850009880 tx=0x7f185002a5d0 comp rx=0 tx=0).stop 2024-09-12T01:53:37.339 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:37.288+0000 7f1864dc7640 1 -- 172.21.15.184:0/2569122208 shutdown_connections 2024-09-12T01:53:37.339 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:37.288+0000 7f1864dc7640 1 --2- 172.21.15.184:0/2569122208 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1860102c20 0x7f1860103020 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:37.340 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:37.288+0000 7f1864dc7640 1 -- 172.21.15.184:0/2569122208 >> 172.21.15.184:0/2569122208 conn(0x7f18600fb450 msgr2=0x7f18600fd890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:53:37.340 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:37.288+0000 7f1864dc7640 1 -- 172.21.15.184:0/2569122208 shutdown_connections 2024-09-12T01:53:37.340 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:37.288+0000 7f1864dc7640 1 -- 172.21.15.184:0/2569122208 wait complete. 2024-09-12T01:53:37.340 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:37.289+0000 7f1864dc7640 1 Processor -- start 2024-09-12T01:53:37.340 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:37.289+0000 7f1864dc7640 1 -- start start 2024-09-12T01:53:37.340 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:37.289+0000 7f1864dc7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1860102c20 0x7f1860196db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:37.340 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:37.289+0000 7f1864dc7640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f18601076e0 con 0x7f1860102c20 2024-09-12T01:53:37.340 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:37.290+0000 7f185e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1860102c20 0x7f1860196db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:37.340 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:37.290+0000 7f185e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1860102c20 0x7f1860196db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.184:59010/0 (socket says 172.21.15.184:59010) 2024-09-12T01:53:37.340 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:37.290+0000 7f185e59c640 1 -- 172.21.15.184:0/3676135277 learned_addr learned my addr 172.21.15.184:0/3676135277 (peer_addr_for_me v2:172.21.15.184:0/0) 2024-09-12T01:53:37.341 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:37.290+0000 7f185e59c640 1 -- 172.21.15.184:0/3676135277 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18601972f0 con 0x7f1860102c20 2024-09-12T01:53:37.341 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:37.290+0000 7f185e59c640 1 --2- 172.21.15.184:0/3676135277 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1860102c20 0x7f1860196db0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1850004f60 tx=0x7f18500048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:53:37.341 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:37.290+0000 7f184f7fe640 1 -- 172.21.15.184:0/3676135277 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f185003f070 con 0x7f1860102c20 2024-09-12T01:53:37.341 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:37.290+0000 7f184f7fe640 1 -- 172.21.15.184:0/3676135277 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1850004d40 con 0x7f1860102c20 2024-09-12T01:53:37.341 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:37.290+0000 7f184f7fe640 1 -- 172.21.15.184:0/3676135277 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1850035030 con 0x7f1860102c20 2024-09-12T01:53:37.341 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:37.290+0000 7f1864dc7640 1 -- 172.21.15.184:0/3676135277 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1860197580 con 0x7f1860102c20 2024-09-12T01:53:37.341 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:37.291+0000 7f1864dc7640 1 -- 172.21.15.184:0/3676135277 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f186019a270 con 0x7f1860102c20 2024-09-12T01:53:37.341 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:37.291+0000 7f184f7fe640 1 -- 172.21.15.184:0/3676135277 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f1850042070 con 0x7f1860102c20 2024-09-12T01:53:37.341 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:37.291+0000 7f1864dc7640 1 -- 172.21.15.184:0/3676135277 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1860107e60 con 0x7f1860102c20 2024-09-12T01:53:37.341 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:37.292+0000 7f184f7fe640 1 --2- 172.21.15.184:0/3676135277 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f1848039060 0x7f184803b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:37.342 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:37.292+0000 7f184f7fe640 1 -- 172.21.15.184:0/3676135277 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f185006cde0 con 0x7f1860102c20 2024-09-12T01:53:37.342 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:37.292+0000 7f185dd9b640 1 --2- 172.21.15.184:0/3676135277 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f1848039060 0x7f184803b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:37.342 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:37.293+0000 7f185dd9b640 1 --2- 172.21.15.184:0/3676135277 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f1848039060 0x7f184803b520 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f18540098c0 tx=0x7f1854006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:53:37.342 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:37.297+0000 7f184f7fe640 1 -- 172.21.15.184:0/3676135277 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1850039350 con 0x7f1860102c20 2024-09-12T01:53:37.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:36 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/3382167872' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-12T01:53:37.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:36 smithi016 ceph-mon[30947]: osdmap e4: 0 total, 0 up, 0 in 2024-09-12T01:53:37.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:36 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:53:37.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:36 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:53:37.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:36 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:53:37.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:36 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:53:37.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:36 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi184", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:53:37.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:36 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:53:37.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:36 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:53:37.485 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:37.484+0000 7f1864dc7640 1 -- 172.21.15.184:0/3676135277 --> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi016:172.21.15.16=a;smithi033:172.21.15.33=b;smithi184:172.21.15.184=c", "target": ["mon-mgr", ""]}) -- 0x7f186002d8f0 con 0x7f1848039060 2024-09-12T01:53:37.490 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:37.489+0000 7f184f7fe640 1 -- 172.21.15.184:0/3676135277 <== mgr.14150 v2:172.21.15.16:6800/3887444820 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f186002d8f0 con 0x7f1848039060 2024-09-12T01:53:37.490 INFO:teuthology.orchestra.run.smithi184.stdout:Scheduled mon update... 2024-09-12T01:53:37.492 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:37.492+0000 7f1864dc7640 1 -- 172.21.15.184:0/3676135277 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f1848039060 msgr2=0x7f184803b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:53:37.492 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:37.492+0000 7f1864dc7640 1 --2- 172.21.15.184:0/3676135277 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f1848039060 0x7f184803b520 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f18540098c0 tx=0x7f1854006db0 comp rx=0 tx=0).stop 2024-09-12T01:53:37.492 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:37.492+0000 7f1864dc7640 1 -- 172.21.15.184:0/3676135277 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1860102c20 msgr2=0x7f1860196db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:53:37.492 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:37.492+0000 7f1864dc7640 1 --2- 172.21.15.184:0/3676135277 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1860102c20 0x7f1860196db0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1850004f60 tx=0x7f18500048f0 comp rx=0 tx=0).stop 2024-09-12T01:53:37.493 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:37.492+0000 7f1864dc7640 1 -- 172.21.15.184:0/3676135277 shutdown_connections 2024-09-12T01:53:37.493 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:37.492+0000 7f1864dc7640 1 --2- 172.21.15.184:0/3676135277 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f1848039060 0x7f184803b520 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:37.493 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:37.492+0000 7f1864dc7640 1 --2- 172.21.15.184:0/3676135277 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1860102c20 0x7f1860196db0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:37.493 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:37.492+0000 7f1864dc7640 1 -- 172.21.15.184:0/3676135277 >> 172.21.15.184:0/3676135277 conn(0x7f18600fb450 msgr2=0x7f18600fbe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:53:37.493 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:37.492+0000 7f1864dc7640 1 -- 172.21.15.184:0/3676135277 shutdown_connections 2024-09-12T01:53:37.493 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:37.492+0000 7f1864dc7640 1 -- 172.21.15.184:0/3676135277 wait complete. 2024-09-12T01:53:38.078 DEBUG:teuthology.orchestra.run.smithi033:mon.b> sudo journalctl -f -n 0 -u ceph-879f3b56-70a9-11ef-bcea-c7b262605968@mon.b.service 2024-09-12T01:53:38.081 DEBUG:teuthology.orchestra.run.smithi184:mon.c> sudo journalctl -f -n 0 -u ceph-879f3b56-70a9-11ef-bcea-c7b262605968@mon.c.service 2024-09-12T01:53:38.084 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-12T01:53:38.084 DEBUG:teuthology.orchestra.run.smithi184:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-12T01:53:38.183 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:37 smithi016 ceph-mon[30947]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:53:38.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:37 smithi016 ceph-mon[30947]: Updating smithi184:/etc/ceph/ceph.conf 2024-09-12T01:53:38.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:37 smithi016 ceph-mon[30947]: Updating smithi184:/var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T01:53:38.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:37 smithi016 ceph-mon[30947]: Updating smithi184:/etc/ceph/ceph.client.admin.keyring 2024-09-12T01:53:38.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:37 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:53:38.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:37 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:53:38.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:37 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:53:38.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:37 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:53:38.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:37 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T01:53:38.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:37 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:53:38.750 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T01:53:39.176 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:39.171+0000 7fc6c1f26640 1 -- 172.21.15.184:0/1197936690 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6bc102750 msgr2=0x7fc6bc102b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:53:39.176 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:39.171+0000 7fc6c1f26640 1 --2- 172.21.15.184:0/1197936690 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6bc102750 0x7fc6bc102b30 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc6ac009880 tx=0x7fc6ac02a670 comp rx=0 tx=0).stop 2024-09-12T01:53:39.176 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:39.171+0000 7fc6c1f26640 1 -- 172.21.15.184:0/1197936690 shutdown_connections 2024-09-12T01:53:39.176 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:39.171+0000 7fc6c1f26640 1 --2- 172.21.15.184:0/1197936690 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6bc102750 0x7fc6bc102b30 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:39.177 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:39.172+0000 7fc6c1f26640 1 -- 172.21.15.184:0/1197936690 >> 172.21.15.184:0/1197936690 conn(0x7fc6bc0fc1b0 msgr2=0x7fc6bc0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:53:39.177 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:39.172+0000 7fc6c1f26640 1 -- 172.21.15.184:0/1197936690 shutdown_connections 2024-09-12T01:53:39.177 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:39.172+0000 7fc6c1f26640 1 -- 172.21.15.184:0/1197936690 wait complete. 2024-09-12T01:53:39.177 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:39.172+0000 7fc6c1f26640 1 Processor -- start 2024-09-12T01:53:39.177 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:39.172+0000 7fc6c1f26640 1 -- start start 2024-09-12T01:53:39.177 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:39.172+0000 7fc6c1f26640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6bc102750 0x7fc6bc1971d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:39.177 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:39.173+0000 7fc6c1f26640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc6bc10bcc0 con 0x7fc6bc102750 2024-09-12T01:53:39.177 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:39.173+0000 7fc6bb7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6bc102750 0x7fc6bc1971d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:39.177 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:39.173+0000 7fc6bb7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6bc102750 0x7fc6bc1971d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.184:59032/0 (socket says 172.21.15.184:59032) 2024-09-12T01:53:39.177 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:39.173+0000 7fc6bb7fe640 1 -- 172.21.15.184:0/3713803976 learned_addr learned my addr 172.21.15.184:0/3713803976 (peer_addr_for_me v2:172.21.15.184:0/0) 2024-09-12T01:53:39.177 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:39.173+0000 7fc6bb7fe640 1 -- 172.21.15.184:0/3713803976 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6bc197710 con 0x7fc6bc102750 2024-09-12T01:53:39.177 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:39.173+0000 7fc6bb7fe640 1 --2- 172.21.15.184:0/3713803976 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6bc102750 0x7fc6bc1971d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc6ac009230 tx=0x7fc6ac003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:53:39.177 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:39.174+0000 7fc6b8ff9640 1 -- 172.21.15.184:0/3713803976 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc6ac03f070 con 0x7fc6bc102750 2024-09-12T01:53:39.178 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:39.174+0000 7fc6c1f26640 1 -- 172.21.15.184:0/3713803976 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6bc1979a0 con 0x7fc6bc102750 2024-09-12T01:53:39.178 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:39.174+0000 7fc6b8ff9640 1 -- 172.21.15.184:0/3713803976 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc6ac03bc80 con 0x7fc6bc102750 2024-09-12T01:53:39.178 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:39.174+0000 7fc6b8ff9640 1 -- 172.21.15.184:0/3713803976 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc6ac035030 con 0x7fc6bc102750 2024-09-12T01:53:39.178 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:39.174+0000 7fc6c1f26640 1 -- 172.21.15.184:0/3713803976 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6bc197e00 con 0x7fc6bc102750 2024-09-12T01:53:39.178 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:39.175+0000 7fc6b8ff9640 1 -- 172.21.15.184:0/3713803976 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fc6ac042070 con 0x7fc6bc102750 2024-09-12T01:53:39.178 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:39.175+0000 7fc6c1f26640 1 -- 172.21.15.184:0/3713803976 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6bc06a7b0 con 0x7fc6bc102750 2024-09-12T01:53:39.178 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:39.175+0000 7fc6b8ff9640 1 --2- 172.21.15.184:0/3713803976 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc6a4039060 0x7fc6a403b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:39.178 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:39.175+0000 7fc6b8ff9640 1 -- 172.21.15.184:0/3713803976 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fc6ac06cd00 con 0x7fc6bc102750 2024-09-12T01:53:39.178 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:39.175+0000 7fc6baffd640 1 --2- 172.21.15.184:0/3713803976 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc6a4039060 0x7fc6a403b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:39.178 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:39.176+0000 7fc6baffd640 1 --2- 172.21.15.184:0/3713803976 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc6a4039060 0x7fc6a403b520 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fc6b00098c0 tx=0x7fc6b0006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:53:39.180 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:39.180+0000 7fc6b8ff9640 1 -- 172.21.15.184:0/3713803976 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc6ac031e20 con 0x7fc6bc102750 2024-09-12T01:53:39.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:38 smithi016 ceph-mon[30947]: Updating smithi184:/var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-12T01:53:39.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:38 smithi016 ceph-mon[30947]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi016:172.21.15.16=a;smithi033:172.21.15.33=b;smithi184:172.21.15.184=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:53:39.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:38 smithi016 ceph-mon[30947]: Saving service mon spec with placement smithi016:172.21.15.16=a;smithi033:172.21.15.33=b;smithi184:172.21.15.184=c;count:3 2024-09-12T01:53:39.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:38 smithi016 ceph-mon[30947]: Deploying daemon mon.c on smithi184 2024-09-12T01:53:39.415 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:39.414+0000 7fc6c1f26640 1 -- 172.21.15.184:0/3713803976 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fc6bc198590 con 0x7fc6bc102750 2024-09-12T01:53:39.415 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:39.415+0000 7fc6b8ff9640 1 -- 172.21.15.184:0/3713803976 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7fc6ac031570 con 0x7fc6bc102750 2024-09-12T01:53:39.415 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-12T01:53:39.416 INFO:teuthology.orchestra.run.smithi184.stdout:{"epoch":1,"fsid":"879f3b56-70a9-11ef-bcea-c7b262605968","modified":"2024-09-12T01:52:09.859051Z","created":"2024-09-12T01:52:09.859051Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:3300","nonce":0},{"type":"v1","addr":"172.21.15.16:6789","nonce":0}]},"addr":"172.21.15.16:6789/0","public_addr":"172.21.15.16:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-12T01:53:39.416 INFO:teuthology.orchestra.run.smithi184.stderr:dumped monmap epoch 1 2024-09-12T01:53:39.418 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:39.417+0000 7fc6c1f26640 1 -- 172.21.15.184:0/3713803976 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc6a4039060 msgr2=0x7fc6a403b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:53:39.418 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:39.417+0000 7fc6c1f26640 1 --2- 172.21.15.184:0/3713803976 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc6a4039060 0x7fc6a403b520 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fc6b00098c0 tx=0x7fc6b0006db0 comp rx=0 tx=0).stop 2024-09-12T01:53:39.418 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:39.417+0000 7fc6c1f26640 1 -- 172.21.15.184:0/3713803976 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6bc102750 msgr2=0x7fc6bc1971d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:53:39.418 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:39.417+0000 7fc6c1f26640 1 --2- 172.21.15.184:0/3713803976 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6bc102750 0x7fc6bc1971d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc6ac009230 tx=0x7fc6ac003d40 comp rx=0 tx=0).stop 2024-09-12T01:53:39.418 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:39.417+0000 7fc6c1f26640 1 -- 172.21.15.184:0/3713803976 shutdown_connections 2024-09-12T01:53:39.418 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:39.417+0000 7fc6c1f26640 1 --2- 172.21.15.184:0/3713803976 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc6a4039060 0x7fc6a403b520 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:39.418 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:39.417+0000 7fc6c1f26640 1 --2- 172.21.15.184:0/3713803976 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6bc102750 0x7fc6bc1971d0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:39.418 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:39.417+0000 7fc6c1f26640 1 -- 172.21.15.184:0/3713803976 >> 172.21.15.184:0/3713803976 conn(0x7fc6bc0fc1b0 msgr2=0x7fc6bc105510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:53:39.418 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:39.417+0000 7fc6c1f26640 1 -- 172.21.15.184:0/3713803976 shutdown_connections 2024-09-12T01:53:39.418 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:39.417+0000 7fc6c1f26640 1 -- 172.21.15.184:0/3713803976 wait complete. 2024-09-12T01:53:40.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:39 smithi016 ceph-mon[30947]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:53:40.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:39 smithi016 ceph-mon[30947]: from='client.? 172.21.15.184:0/3713803976' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-12T01:53:40.974 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-12T01:53:40.975 DEBUG:teuthology.orchestra.run.smithi184:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-12T01:53:41.269 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T01:53:41.884 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:41.882+0000 7fafe0a5c640 1 -- 172.21.15.184:0/293513231 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafdc1028d0 msgr2=0x7fafdc102cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:53:41.885 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:41.882+0000 7fafe0a5c640 1 --2- 172.21.15.184:0/293513231 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafdc1028d0 0x7fafdc102cb0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fafcc009880 tx=0x7fafcc02a670 comp rx=0 tx=0).stop 2024-09-12T01:53:41.885 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:41.882+0000 7fafe0a5c640 1 -- 172.21.15.184:0/293513231 shutdown_connections 2024-09-12T01:53:41.885 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:41.882+0000 7fafe0a5c640 1 --2- 172.21.15.184:0/293513231 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafdc1028d0 0x7fafdc102cb0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:41.885 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:41.882+0000 7fafe0a5c640 1 -- 172.21.15.184:0/293513231 >> 172.21.15.184:0/293513231 conn(0x7fafdc0fc290 msgr2=0x7fafdc0fe6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:53:41.885 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:41.882+0000 7fafe0a5c640 1 -- 172.21.15.184:0/293513231 shutdown_connections 2024-09-12T01:53:41.885 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:41.882+0000 7fafe0a5c640 1 -- 172.21.15.184:0/293513231 wait complete. 2024-09-12T01:53:41.885 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:41.883+0000 7fafe0a5c640 1 Processor -- start 2024-09-12T01:53:41.885 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:41.883+0000 7fafe0a5c640 1 -- start start 2024-09-12T01:53:41.885 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:41.883+0000 7fafe0a5c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafdc1028d0 0x7fafdc1972e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:41.885 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:41.883+0000 7fafe0a5c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fafdc10bdd0 con 0x7fafdc1028d0 2024-09-12T01:53:41.885 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:41.883+0000 7fafda59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafdc1028d0 0x7fafdc1972e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:41.885 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:41.884+0000 7fafda59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafdc1028d0 0x7fafdc1972e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.184:55222/0 (socket says 172.21.15.184:55222) 2024-09-12T01:53:41.886 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:41.884+0000 7fafda59c640 1 -- 172.21.15.184:0/339850880 learned_addr learned my addr 172.21.15.184:0/339850880 (peer_addr_for_me v2:172.21.15.184:0/0) 2024-09-12T01:53:41.886 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:41.884+0000 7fafda59c640 1 -- 172.21.15.184:0/339850880 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafdc197820 con 0x7fafdc1028d0 2024-09-12T01:53:41.886 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:41.884+0000 7fafda59c640 1 --2- 172.21.15.184:0/339850880 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafdc1028d0 0x7fafdc1972e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fafcc009230 tx=0x7fafcc003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:53:41.886 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:41.884+0000 7fafcb7fe640 1 -- 172.21.15.184:0/339850880 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fafcc03f070 con 0x7fafdc1028d0 2024-09-12T01:53:41.886 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:41.884+0000 7fafcb7fe640 1 -- 172.21.15.184:0/339850880 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fafcc03bc80 con 0x7fafdc1028d0 2024-09-12T01:53:41.886 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:41.884+0000 7fafe0a5c640 1 -- 172.21.15.184:0/339850880 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafdc197ab0 con 0x7fafdc1028d0 2024-09-12T01:53:41.886 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:41.884+0000 7fafcb7fe640 1 -- 172.21.15.184:0/339850880 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fafcc035030 con 0x7fafdc1028d0 2024-09-12T01:53:41.886 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:41.884+0000 7fafe0a5c640 1 -- 172.21.15.184:0/339850880 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafdc197f10 con 0x7fafdc1028d0 2024-09-12T01:53:41.887 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:41.885+0000 7fafcb7fe640 1 -- 172.21.15.184:0/339850880 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fafcc042070 con 0x7fafdc1028d0 2024-09-12T01:53:41.887 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:41.885+0000 7fafe0a5c640 1 -- 172.21.15.184:0/339850880 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafdc06acb0 con 0x7fafdc1028d0 2024-09-12T01:53:41.887 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:41.885+0000 7fafcb7fe640 1 --2- 172.21.15.184:0/339850880 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fafc4039060 0x7fafc403b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:41.887 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:41.886+0000 7fafcb7fe640 1 -- 172.21.15.184:0/339850880 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fafcc06cdb0 con 0x7fafdc1028d0 2024-09-12T01:53:41.887 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:41.886+0000 7fafd9d9b640 1 --2- 172.21.15.184:0/339850880 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fafc4039060 0x7fafc403b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:41.887 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:41.886+0000 7fafd9d9b640 1 --2- 172.21.15.184:0/339850880 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fafc4039060 0x7fafc403b520 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fafd00098c0 tx=0x7fafd0006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:53:41.891 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:41.891+0000 7fafcb7fe640 1 -- 172.21.15.184:0/339850880 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fafcc031e20 con 0x7fafdc1028d0 2024-09-12T01:53:42.148 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:42.146+0000 7fafe0a5c640 1 -- 172.21.15.184:0/339850880 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fafdc1986a0 con 0x7fafdc1028d0 2024-09-12T01:53:42.148 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:42.147+0000 7fafcb7fe640 1 -- 172.21.15.184:0/339850880 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7fafcc03ac60 con 0x7fafdc1028d0 2024-09-12T01:53:42.148 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-12T01:53:42.148 INFO:teuthology.orchestra.run.smithi184.stdout:{"epoch":1,"fsid":"879f3b56-70a9-11ef-bcea-c7b262605968","modified":"2024-09-12T01:52:09.859051Z","created":"2024-09-12T01:52:09.859051Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:3300","nonce":0},{"type":"v1","addr":"172.21.15.16:6789","nonce":0}]},"addr":"172.21.15.16:6789/0","public_addr":"172.21.15.16:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-12T01:53:42.149 INFO:teuthology.orchestra.run.smithi184.stderr:dumped monmap epoch 1 2024-09-12T01:53:42.150 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:42.149+0000 7fafe0a5c640 1 -- 172.21.15.184:0/339850880 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fafc4039060 msgr2=0x7fafc403b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:53:42.150 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:42.149+0000 7fafe0a5c640 1 --2- 172.21.15.184:0/339850880 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fafc4039060 0x7fafc403b520 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fafd00098c0 tx=0x7fafd0006db0 comp rx=0 tx=0).stop 2024-09-12T01:53:42.150 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:42.149+0000 7fafe0a5c640 1 -- 172.21.15.184:0/339850880 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafdc1028d0 msgr2=0x7fafdc1972e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:53:42.150 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:42.149+0000 7fafe0a5c640 1 --2- 172.21.15.184:0/339850880 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafdc1028d0 0x7fafdc1972e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fafcc009230 tx=0x7fafcc003d40 comp rx=0 tx=0).stop 2024-09-12T01:53:42.150 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:42.149+0000 7fafe0a5c640 1 -- 172.21.15.184:0/339850880 shutdown_connections 2024-09-12T01:53:42.150 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:42.149+0000 7fafe0a5c640 1 --2- 172.21.15.184:0/339850880 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fafc4039060 0x7fafc403b520 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:42.150 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:42.149+0000 7fafe0a5c640 1 --2- 172.21.15.184:0/339850880 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafdc1028d0 0x7fafdc1972e0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:42.150 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:42.149+0000 7fafe0a5c640 1 -- 172.21.15.184:0/339850880 >> 172.21.15.184:0/339850880 conn(0x7fafdc0fc290 msgr2=0x7fafdc105690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:53:42.150 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:42.149+0000 7fafe0a5c640 1 -- 172.21.15.184:0/339850880 shutdown_connections 2024-09-12T01:53:42.150 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:42.150+0000 7fafe0a5c640 1 -- 172.21.15.184:0/339850880 wait complete. 2024-09-12T01:53:42.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:41 smithi016 ceph-mon[30947]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:53:43.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:53:43 smithi184 systemd[1]: Started Ceph mon.c for 879f3b56-70a9-11ef-bcea-c7b262605968. 2024-09-12T01:53:43.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:42 smithi016 ceph-mon[30947]: from='client.? 172.21.15.184:0/339850880' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-12T01:53:43.734 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-12T01:53:43.734 DEBUG:teuthology.orchestra.run.smithi184:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-12T01:53:43.991 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T01:53:48.482 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:53:48 smithi184 ceph-mon[35593]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:53:48.483 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:53:48 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T01:53:48.483 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:53:48 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T01:53:48.483 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:53:48 smithi184 ceph-mon[35593]: mon.a calling monitor election 2024-09-12T01:53:48.483 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:53:48 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T01:53:48.483 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:53:48 smithi184 ceph-mon[35593]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:53:48.483 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:53:48 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T01:53:48.483 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:53:48 smithi184 ceph-mon[35593]: mon.c calling monitor election 2024-09-12T01:53:48.483 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:53:48 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T01:53:48.483 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:53:48 smithi184 ceph-mon[35593]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:53:48.483 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:53:48 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T01:53:48.483 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:53:48 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T01:53:48.484 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:53:48 smithi184 ceph-mon[35593]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-12T01:53:48.484 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:53:48 smithi184 ceph-mon[35593]: monmap epoch 2 2024-09-12T01:53:48.484 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:53:48 smithi184 ceph-mon[35593]: fsid 879f3b56-70a9-11ef-bcea-c7b262605968 2024-09-12T01:53:48.484 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:53:48 smithi184 ceph-mon[35593]: last_changed 2024-09-12T01:53:43.139383+0000 2024-09-12T01:53:48.484 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:53:48 smithi184 ceph-mon[35593]: created 2024-09-12T01:52:09.859051+0000 2024-09-12T01:53:48.484 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:53:48 smithi184 ceph-mon[35593]: min_mon_release 19 (squid) 2024-09-12T01:53:48.484 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:53:48 smithi184 ceph-mon[35593]: election_strategy: 1 2024-09-12T01:53:48.484 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:53:48 smithi184 ceph-mon[35593]: 0: [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] mon.a 2024-09-12T01:53:48.484 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:53:48 smithi184 ceph-mon[35593]: 1: [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] mon.c 2024-09-12T01:53:48.484 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:53:48 smithi184 ceph-mon[35593]: fsmap 2024-09-12T01:53:48.484 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:53:48 smithi184 ceph-mon[35593]: osdmap e4: 0 total, 0 up, 0 in 2024-09-12T01:53:48.484 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:53:48 smithi184 ceph-mon[35593]: mgrmap e13: x(active, since 39s) 2024-09-12T01:53:48.485 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:53:48 smithi184 ceph-mon[35593]: overall HEALTH_OK 2024-09-12T01:53:48.485 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:53:48 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:53:48.485 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:53:48 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:53:48.485 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:53:48 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:53:48.485 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:53:48 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T01:53:48.485 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:53:48 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:53:48.485 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:53:48 smithi184 ceph-mon[35593]: Deploying daemon mon.b on smithi033 2024-09-12T01:53:48.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:48 smithi016 ceph-mon[30947]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:53:48.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:48 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T01:53:48.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:48 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T01:53:48.685 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:48 smithi016 ceph-mon[30947]: mon.a calling monitor election 2024-09-12T01:53:48.685 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:48 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T01:53:48.685 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:48 smithi016 ceph-mon[30947]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:53:48.685 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:48 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T01:53:48.685 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:48 smithi016 ceph-mon[30947]: mon.c calling monitor election 2024-09-12T01:53:48.685 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:48 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T01:53:48.685 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:48 smithi016 ceph-mon[30947]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:53:48.685 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:48 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T01:53:48.685 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:48 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T01:53:48.685 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:48 smithi016 ceph-mon[30947]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-12T01:53:48.685 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:48 smithi016 ceph-mon[30947]: monmap epoch 2 2024-09-12T01:53:48.686 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:48 smithi016 ceph-mon[30947]: fsid 879f3b56-70a9-11ef-bcea-c7b262605968 2024-09-12T01:53:48.686 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:48 smithi016 ceph-mon[30947]: last_changed 2024-09-12T01:53:43.139383+0000 2024-09-12T01:53:48.686 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:48 smithi016 ceph-mon[30947]: created 2024-09-12T01:52:09.859051+0000 2024-09-12T01:53:48.686 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:48 smithi016 ceph-mon[30947]: min_mon_release 19 (squid) 2024-09-12T01:53:48.686 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:48 smithi016 ceph-mon[30947]: election_strategy: 1 2024-09-12T01:53:48.686 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:48 smithi016 ceph-mon[30947]: 0: [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] mon.a 2024-09-12T01:53:48.686 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:48 smithi016 ceph-mon[30947]: 1: [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] mon.c 2024-09-12T01:53:48.686 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:48 smithi016 ceph-mon[30947]: fsmap 2024-09-12T01:53:48.686 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:48 smithi016 ceph-mon[30947]: osdmap e4: 0 total, 0 up, 0 in 2024-09-12T01:53:48.686 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:48 smithi016 ceph-mon[30947]: mgrmap e13: x(active, since 39s) 2024-09-12T01:53:48.686 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:48 smithi016 ceph-mon[30947]: overall HEALTH_OK 2024-09-12T01:53:48.686 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:48 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:53:48.687 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:48 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:53:48.687 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:48 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:53:48.687 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:48 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T01:53:48.687 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:48 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:53:48.687 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:48 smithi016 ceph-mon[30947]: Deploying daemon mon.b on smithi033 2024-09-12T01:53:48.813 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:48.812+0000 7fd63b8d2640 1 -- 172.21.15.184:0/1993542453 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd61c003530 msgr2=0x7fd61c0059c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:53:48.814 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:48.812+0000 7fd63b8d2640 1 --2- 172.21.15.184:0/1993542453 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd61c003530 0x7fd61c0059c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fd6341016a0 tx=0x7fd62c02c490 comp rx=0 tx=0).stop 2024-09-12T01:53:48.814 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:48.812+0000 7fd63b8d2640 1 -- 172.21.15.184:0/1993542453 shutdown_connections 2024-09-12T01:53:48.814 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:48.812+0000 7fd63b8d2640 1 --2- 172.21.15.184:0/1993542453 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd61c003530 0x7fd61c0059c0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:48.814 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:48.812+0000 7fd63b8d2640 1 -- 172.21.15.184:0/1993542453 >> 172.21.15.184:0/1993542453 conn(0x7fd6340fc220 msgr2=0x7fd6340fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:53:48.814 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:48.812+0000 7fd63b8d2640 1 -- 172.21.15.184:0/1993542453 shutdown_connections 2024-09-12T01:53:48.814 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:48.812+0000 7fd63b8d2640 1 -- 172.21.15.184:0/1993542453 wait complete. 2024-09-12T01:53:48.814 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:48.813+0000 7fd63b8d2640 1 Processor -- start 2024-09-12T01:53:48.814 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:48.813+0000 7fd63b8d2640 1 -- start start 2024-09-12T01:53:48.817 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:48.814+0000 7fd63b8d2640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd61c003530 0x7fd6341972a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:48.817 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:48.814+0000 7fd63b8d2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6341977e0 0x7fd63419bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:48.817 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:48.814+0000 7fd63b8d2640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd634108670 con 0x7fd6341977e0 2024-09-12T01:53:48.817 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:48.814+0000 7fd63b8d2640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd6341084f0 con 0x7fd61c003530 2024-09-12T01:53:48.817 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:48.814+0000 7fd63966e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd61c003530 0x7fd6341972a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:48.817 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:48.814+0000 7fd63966e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd61c003530 0x7fd6341972a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.184:49928/0 (socket says 172.21.15.184:49928) 2024-09-12T01:53:48.817 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:48.814+0000 7fd63966e640 1 -- 172.21.15.184:0/4144788133 learned_addr learned my addr 172.21.15.184:0/4144788133 (peer_addr_for_me v2:172.21.15.184:0/0) 2024-09-12T01:53:48.817 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:48.814+0000 7fd638e6d640 1 --2- 172.21.15.184:0/4144788133 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6341977e0 0x7fd63419bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:48.818 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:48.814+0000 7fd63966e640 1 -- 172.21.15.184:0/4144788133 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd61c003530 msgr2=0x7fd6341972a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-09-12T01:53:48.818 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:48.814+0000 7fd63966e640 1 -- 172.21.15.184:0/4144788133 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd61c003530 msgr2=0x7fd6341972a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-12T01:53:48.818 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:48.814+0000 7fd63966e640 1 --2- 172.21.15.184:0/4144788133 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd61c003530 0x7fd6341972a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-12T01:53:48.818 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:48.814+0000 7fd63966e640 1 --2- 172.21.15.184:0/4144788133 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd61c003530 0x7fd6341972a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-12T01:53:48.818 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:48.814+0000 7fd638e6d640 1 -- 172.21.15.184:0/4144788133 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd61c003530 msgr2=0x7fd6341972a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T01:53:48.818 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:48.814+0000 7fd638e6d640 1 --2- 172.21.15.184:0/4144788133 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd61c003530 0x7fd6341972a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:48.818 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:48.814+0000 7fd638e6d640 1 -- 172.21.15.184:0/4144788133 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd634197e00 con 0x7fd6341977e0 2024-09-12T01:53:48.818 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:48.815+0000 7fd638e6d640 1 --2- 172.21.15.184:0/4144788133 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6341977e0 0x7fd63419bb70 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fd62800b3d0 tx=0x7fd62800b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:53:48.818 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:48.815+0000 7fd6327fc640 1 -- 172.21.15.184:0/4144788133 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fd628011630 con 0x7fd6341977e0 2024-09-12T01:53:48.818 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:48.815+0000 7fd6327fc640 1 -- 172.21.15.184:0/4144788133 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd628011cb0 con 0x7fd6341977e0 2024-09-12T01:53:48.818 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:48.815+0000 7fd63b8d2640 1 -- 172.21.15.184:0/4144788133 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd63419c0b0 con 0x7fd6341977e0 2024-09-12T01:53:48.818 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:48.815+0000 7fd6327fc640 1 -- 172.21.15.184:0/4144788133 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fd628010df0 con 0x7fd6341977e0 2024-09-12T01:53:48.819 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:48.815+0000 7fd63b8d2640 1 -- 172.21.15.184:0/4144788133 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd63419c670 con 0x7fd6341977e0 2024-09-12T01:53:48.819 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:48.816+0000 7fd6327fc640 1 -- 172.21.15.184:0/4144788133 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fd628018020 con 0x7fd6341977e0 2024-09-12T01:53:48.819 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:48.816+0000 7fd6327fc640 1 --2- 172.21.15.184:0/4144788133 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fd620038e50 0x7fd62003b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:48.819 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:48.816+0000 7fd63b8d2640 1 -- 172.21.15.184:0/4144788133 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd63419c980 con 0x7fd6341977e0 2024-09-12T01:53:48.819 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:48.816+0000 7fd6327fc640 1 -- 172.21.15.184:0/4144788133 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fd62804bbe0 con 0x7fd6341977e0 2024-09-12T01:53:48.819 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:48.817+0000 7fd63966e640 1 --2- 172.21.15.184:0/4144788133 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fd620038e50 0x7fd62003b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:48.819 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:48.817+0000 7fd63966e640 1 --2- 172.21.15.184:0/4144788133 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fd620038e50 0x7fd62003b310 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd62c012c10 tx=0x7fd62c0323a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:53:48.822 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:48.822+0000 7fd6327fc640 1 -- 172.21.15.184:0/4144788133 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd62801f020 con 0x7fd6341977e0 2024-09-12T01:53:49.057 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:49.057+0000 7fd63b8d2640 1 -- 172.21.15.184:0/4144788133 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fd634198660 con 0x7fd6341977e0 2024-09-12T01:53:49.058 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:49.058+0000 7fd6327fc640 1 -- 172.21.15.184:0/4144788133 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7fd628018310 con 0x7fd6341977e0 2024-09-12T01:53:49.058 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-12T01:53:49.058 INFO:teuthology.orchestra.run.smithi184.stdout:{"epoch":2,"fsid":"879f3b56-70a9-11ef-bcea-c7b262605968","modified":"2024-09-12T01:53:43.139383Z","created":"2024-09-12T01:52:09.859051Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:3300","nonce":0},{"type":"v1","addr":"172.21.15.16:6789","nonce":0}]},"addr":"172.21.15.16:6789/0","public_addr":"172.21.15.16:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:3300","nonce":0},{"type":"v1","addr":"172.21.15.184:6789","nonce":0}]},"addr":"172.21.15.184:6789/0","public_addr":"172.21.15.184:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-12T01:53:49.058 INFO:teuthology.orchestra.run.smithi184.stderr:dumped monmap epoch 2 2024-09-12T01:53:49.060 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:49.060+0000 7fd63b8d2640 1 -- 172.21.15.184:0/4144788133 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fd620038e50 msgr2=0x7fd62003b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:53:49.060 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:49.060+0000 7fd63b8d2640 1 --2- 172.21.15.184:0/4144788133 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fd620038e50 0x7fd62003b310 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd62c012c10 tx=0x7fd62c0323a0 comp rx=0 tx=0).stop 2024-09-12T01:53:49.060 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:49.060+0000 7fd63b8d2640 1 -- 172.21.15.184:0/4144788133 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6341977e0 msgr2=0x7fd63419bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:53:49.060 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:49.060+0000 7fd63b8d2640 1 --2- 172.21.15.184:0/4144788133 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6341977e0 0x7fd63419bb70 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fd62800b3d0 tx=0x7fd62800b890 comp rx=0 tx=0).stop 2024-09-12T01:53:49.060 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:49.060+0000 7fd63b8d2640 1 -- 172.21.15.184:0/4144788133 shutdown_connections 2024-09-12T01:53:49.060 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:49.060+0000 7fd63b8d2640 1 --2- 172.21.15.184:0/4144788133 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fd620038e50 0x7fd62003b310 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:49.061 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:49.060+0000 7fd63b8d2640 1 --2- 172.21.15.184:0/4144788133 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6341977e0 0x7fd63419bb70 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:49.061 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:49.060+0000 7fd63b8d2640 1 --2- 172.21.15.184:0/4144788133 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd61c003530 0x7fd6341972a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:49.061 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:49.060+0000 7fd63b8d2640 1 -- 172.21.15.184:0/4144788133 >> 172.21.15.184:0/4144788133 conn(0x7fd6340fc220 msgr2=0x7fd6340fd250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:53:49.061 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:49.060+0000 7fd63b8d2640 1 -- 172.21.15.184:0/4144788133 shutdown_connections 2024-09-12T01:53:49.061 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:49.060+0000 7fd63b8d2640 1 -- 172.21.15.184:0/4144788133 wait complete. 2024-09-12T01:53:49.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:53:49 smithi184 ceph-mon[35593]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:53:49.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:53:49 smithi184 ceph-mon[35593]: from='client.? 172.21.15.184:0/4144788133' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-12T01:53:49.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:53:49 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T01:53:49.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:49 smithi016 ceph-mon[30947]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:53:49.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:49 smithi016 ceph-mon[30947]: from='client.? 172.21.15.184:0/4144788133' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-12T01:53:49.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:49 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T01:53:50.668 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-12T01:53:50.669 DEBUG:teuthology.orchestra.run.smithi184:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-12T01:53:50.922 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T01:53:51.355 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:53:51 smithi184 ceph-mon[35593]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:53:51.359 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:51.355+0000 7fbe70d1f640 1 -- 172.21.15.184:0/4223689503 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe6c106630 msgr2=0x7fbe6c106a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:53:51.359 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:51.355+0000 7fbe70d1f640 1 --2- 172.21.15.184:0/4223689503 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe6c106630 0x7fbe6c106a10 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fbe5c009880 tx=0x7fbe5c02a5d0 comp rx=0 tx=0).stop 2024-09-12T01:53:51.359 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:51.355+0000 7fbe70d1f640 1 -- 172.21.15.184:0/4223689503 shutdown_connections 2024-09-12T01:53:51.359 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:51.355+0000 7fbe70d1f640 1 --2- 172.21.15.184:0/4223689503 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe6c106630 0x7fbe6c106a10 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:51.359 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:51.355+0000 7fbe70d1f640 1 -- 172.21.15.184:0/4223689503 >> 172.21.15.184:0/4223689503 conn(0x7fbe6c0fc220 msgr2=0x7fbe6c0fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:53:51.359 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:51.355+0000 7fbe70d1f640 1 -- 172.21.15.184:0/4223689503 shutdown_connections 2024-09-12T01:53:51.359 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:51.355+0000 7fbe70d1f640 1 -- 172.21.15.184:0/4223689503 wait complete. 2024-09-12T01:53:51.359 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:51.356+0000 7fbe70d1f640 1 Processor -- start 2024-09-12T01:53:51.359 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:51.356+0000 7fbe70d1f640 1 -- start start 2024-09-12T01:53:51.359 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:51.356+0000 7fbe70d1f640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbe6c106630 0x7fbe6c197420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:51.359 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:51.356+0000 7fbe70d1f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe6c197960 0x7fbe6c19bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:51.360 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:51.356+0000 7fbe70d1f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbe6c10ac60 con 0x7fbe6c197960 2024-09-12T01:53:51.360 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:51.356+0000 7fbe70d1f640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fbe6c10aae0 con 0x7fbe6c106630 2024-09-12T01:53:51.360 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:51.356+0000 7fbe6a59c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbe6c106630 0x7fbe6c197420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:51.360 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:51.356+0000 7fbe6a59c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbe6c106630 0x7fbe6c197420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.184:58708/0 (socket says 172.21.15.184:58708) 2024-09-12T01:53:51.360 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:51.356+0000 7fbe6a59c640 1 -- 172.21.15.184:0/3171527356 learned_addr learned my addr 172.21.15.184:0/3171527356 (peer_addr_for_me v2:172.21.15.184:0/0) 2024-09-12T01:53:51.360 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:51.356+0000 7fbe6a59c640 1 -- 172.21.15.184:0/3171527356 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbe6c106630 msgr2=0x7fbe6c197420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-09-12T01:53:51.360 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:51.356+0000 7fbe6a59c640 1 -- 172.21.15.184:0/3171527356 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbe6c106630 msgr2=0x7fbe6c197420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-12T01:53:51.360 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:51.356+0000 7fbe6a59c640 1 --2- 172.21.15.184:0/3171527356 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbe6c106630 0x7fbe6c197420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-12T01:53:51.360 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:51.356+0000 7fbe6a59c640 1 --2- 172.21.15.184:0/3171527356 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbe6c106630 0x7fbe6c197420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-12T01:53:51.360 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:51.356+0000 7fbe69d9b640 1 --2- 172.21.15.184:0/3171527356 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe6c197960 0x7fbe6c19bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:51.360 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:51.357+0000 7fbe69d9b640 1 -- 172.21.15.184:0/3171527356 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbe6c106630 msgr2=0x7fbe6c197420 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T01:53:51.360 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:51.357+0000 7fbe69d9b640 1 --2- 172.21.15.184:0/3171527356 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbe6c106630 0x7fbe6c197420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:51.361 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:51.357+0000 7fbe69d9b640 1 -- 172.21.15.184:0/3171527356 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe6c197f80 con 0x7fbe6c197960 2024-09-12T01:53:51.361 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:51.357+0000 7fbe69d9b640 1 --2- 172.21.15.184:0/3171527356 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe6c197960 0x7fbe6c19bcf0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fbe6000b3d0 tx=0x7fbe6000b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:53:51.361 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:51.357+0000 7fbe5b7fe640 1 -- 172.21.15.184:0/3171527356 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fbe60011630 con 0x7fbe6c197960 2024-09-12T01:53:51.361 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:51.357+0000 7fbe5b7fe640 1 -- 172.21.15.184:0/3171527356 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe60011cb0 con 0x7fbe6c197960 2024-09-12T01:53:51.361 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:51.357+0000 7fbe70d1f640 1 -- 172.21.15.184:0/3171527356 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe6c19c230 con 0x7fbe6c197960 2024-09-12T01:53:51.361 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:51.357+0000 7fbe5b7fe640 1 -- 172.21.15.184:0/3171527356 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fbe60010df0 con 0x7fbe6c197960 2024-09-12T01:53:51.361 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:51.357+0000 7fbe70d1f640 1 -- 172.21.15.184:0/3171527356 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe6c19c770 con 0x7fbe6c197960 2024-09-12T01:53:51.361 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:51.358+0000 7fbe5b7fe640 1 -- 172.21.15.184:0/3171527356 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fbe60018020 con 0x7fbe6c197960 2024-09-12T01:53:51.361 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:51.358+0000 7fbe70d1f640 1 -- 172.21.15.184:0/3171527356 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe6c101ca0 con 0x7fbe6c197960 2024-09-12T01:53:51.361 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:51.358+0000 7fbe5b7fe640 1 --2- 172.21.15.184:0/3171527356 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fbe54038e00 0x7fbe5403b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:51.361 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:51.358+0000 7fbe5b7fe640 1 -- 172.21.15.184:0/3171527356 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fbe6004bc10 con 0x7fbe6c197960 2024-09-12T01:53:51.361 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:51.359+0000 7fbe6a59c640 1 --2- 172.21.15.184:0/3171527356 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fbe54038e00 0x7fbe5403b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:51.362 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:51.359+0000 7fbe6a59c640 1 --2- 172.21.15.184:0/3171527356 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fbe54038e00 0x7fbe5403b2c0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fbe5c0096c0 tx=0x7fbe5c005a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:53:51.364 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:51.364+0000 7fbe5b7fe640 1 -- 172.21.15.184:0/3171527356 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbe6001f020 con 0x7fbe6c197960 2024-09-12T01:53:51.592 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:51.591+0000 7fbe70d1f640 1 -- 172.21.15.184:0/3171527356 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fbe6c19ca80 con 0x7fbe6c197960 2024-09-12T01:53:51.592 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:51.592+0000 7fbe5b7fe640 1 -- 172.21.15.184:0/3171527356 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7fbe60018310 con 0x7fbe6c197960 2024-09-12T01:53:51.592 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-12T01:53:51.593 INFO:teuthology.orchestra.run.smithi184.stdout:{"epoch":2,"fsid":"879f3b56-70a9-11ef-bcea-c7b262605968","modified":"2024-09-12T01:53:43.139383Z","created":"2024-09-12T01:52:09.859051Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:3300","nonce":0},{"type":"v1","addr":"172.21.15.16:6789","nonce":0}]},"addr":"172.21.15.16:6789/0","public_addr":"172.21.15.16:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:3300","nonce":0},{"type":"v1","addr":"172.21.15.184:6789","nonce":0}]},"addr":"172.21.15.184:6789/0","public_addr":"172.21.15.184:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-12T01:53:51.593 INFO:teuthology.orchestra.run.smithi184.stderr:dumped monmap epoch 2 2024-09-12T01:53:51.594 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:51.594+0000 7fbe70d1f640 1 -- 172.21.15.184:0/3171527356 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fbe54038e00 msgr2=0x7fbe5403b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:53:51.594 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:51.594+0000 7fbe70d1f640 1 --2- 172.21.15.184:0/3171527356 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fbe54038e00 0x7fbe5403b2c0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fbe5c0096c0 tx=0x7fbe5c005a40 comp rx=0 tx=0).stop 2024-09-12T01:53:51.594 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:51.594+0000 7fbe70d1f640 1 -- 172.21.15.184:0/3171527356 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe6c197960 msgr2=0x7fbe6c19bcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:53:51.594 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:51.594+0000 7fbe70d1f640 1 --2- 172.21.15.184:0/3171527356 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe6c197960 0x7fbe6c19bcf0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fbe6000b3d0 tx=0x7fbe6000b890 comp rx=0 tx=0).stop 2024-09-12T01:53:51.594 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:51.594+0000 7fbe70d1f640 1 -- 172.21.15.184:0/3171527356 shutdown_connections 2024-09-12T01:53:51.595 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:51.594+0000 7fbe70d1f640 1 --2- 172.21.15.184:0/3171527356 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fbe54038e00 0x7fbe5403b2c0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:51.595 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:51.594+0000 7fbe70d1f640 1 --2- 172.21.15.184:0/3171527356 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe6c197960 0x7fbe6c19bcf0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:51.595 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:51.594+0000 7fbe70d1f640 1 --2- 172.21.15.184:0/3171527356 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbe6c106630 0x7fbe6c197420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:51.595 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:51.594+0000 7fbe70d1f640 1 -- 172.21.15.184:0/3171527356 >> 172.21.15.184:0/3171527356 conn(0x7fbe6c0fc220 msgr2=0x7fbe6c1900f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:53:51.595 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:51.594+0000 7fbe70d1f640 1 -- 172.21.15.184:0/3171527356 shutdown_connections 2024-09-12T01:53:51.595 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:51.594+0000 7fbe70d1f640 1 -- 172.21.15.184:0/3171527356 wait complete. 2024-09-12T01:53:51.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:51 smithi016 ceph-mon[30947]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:53:51.962 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:53:51 smithi033 ceph-mon[35332]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-12T01:53:51.963 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:53:51 smithi033 bash[35314]: 0883b7e46f0612c0ad18a20b2ec73fe09a8dd0fe22c4713ed6e8ae3c8906c2c9 2024-09-12T01:53:52.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:53:51 smithi033 systemd[1]: Started Ceph mon.b for 879f3b56-70a9-11ef-bcea-c7b262605968. 2024-09-12T01:53:53.148 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-12T01:53:53.149 DEBUG:teuthology.orchestra.run.smithi184:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-12T01:53:53.401 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T01:53:57.022 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:57.021+0000 7f0f7d48f640 1 -- 172.21.15.184:0/3181403015 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f60003530 msgr2=0x7f0f600059c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:53:57.022 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:57.021+0000 7f0f7d48f640 1 --2- 172.21.15.184:0/3181403015 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f60003530 0x7f0f600059c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f0f7806aa60 tx=0x7f0f6802c150 comp rx=0 tx=0).stop 2024-09-12T01:53:57.022 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:57.021+0000 7f0f7d48f640 1 -- 172.21.15.184:0/3181403015 shutdown_connections 2024-09-12T01:53:57.022 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:57.021+0000 7f0f7d48f640 1 --2- 172.21.15.184:0/3181403015 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f60003530 0x7f0f600059c0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:57.022 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:57.021+0000 7f0f7d48f640 1 -- 172.21.15.184:0/3181403015 >> 172.21.15.184:0/3181403015 conn(0x7f0f780fc220 msgr2=0x7f0f780fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:53:57.022 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:57.021+0000 7f0f7d48f640 1 -- 172.21.15.184:0/3181403015 shutdown_connections 2024-09-12T01:53:57.023 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:57.021+0000 7f0f7d48f640 1 -- 172.21.15.184:0/3181403015 wait complete. 2024-09-12T01:53:57.025 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:57.022+0000 7f0f7d48f640 1 Processor -- start 2024-09-12T01:53:57.025 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:57.023+0000 7f0f7d48f640 1 -- start start 2024-09-12T01:53:57.025 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:57.023+0000 7f0f7d48f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f60003530 0x7f0f78100480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:57.025 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:57.023+0000 7f0f7d48f640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0f781009c0 0x7f0f78100e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:57.025 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:57.023+0000 7f0f7d48f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f781a3570 0x7f0f781a5960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:57.025 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:57.023+0000 7f0f7d48f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0f78109780 con 0x7f0f781a3570 2024-09-12T01:53:57.026 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:57.023+0000 7f0f7d48f640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0f78109600 con 0x7f0f60003530 2024-09-12T01:53:57.026 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:57.023+0000 7f0f7d48f640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f0f78109900 con 0x7f0f781009c0 2024-09-12T01:53:57.026 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:57.023+0000 7f0f767fc640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0f781009c0 0x7f0f78100e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:57.026 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:57.023+0000 7f0f767fc640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0f781009c0 0x7f0f78100e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.184:58712/0 (socket says 172.21.15.184:58712) 2024-09-12T01:53:57.026 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:57.023+0000 7f0f767fc640 1 -- 172.21.15.184:0/341243400 learned_addr learned my addr 172.21.15.184:0/341243400 (peer_addr_for_me v2:172.21.15.184:0/0) 2024-09-12T01:53:57.026 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:57.023+0000 7f0f777fe640 1 --2- 172.21.15.184:0/341243400 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f781a3570 0x7f0f781a5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:57.026 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:57.023+0000 7f0f767fc640 1 -- 172.21.15.184:0/341243400 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0f781009c0 msgr2=0x7f0f78100e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-09-12T01:53:57.026 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:57.023+0000 7f0f767fc640 1 -- 172.21.15.184:0/341243400 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0f781009c0 msgr2=0x7f0f78100e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-12T01:53:57.026 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:57.023+0000 7f0f767fc640 1 --2- 172.21.15.184:0/341243400 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0f781009c0 0x7f0f78100e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-12T01:53:57.026 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:57.023+0000 7f0f767fc640 1 --2- 172.21.15.184:0/341243400 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0f781009c0 0x7f0f78100e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-12T01:53:57.026 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:57.023+0000 7f0f76ffd640 1 --2- 172.21.15.184:0/341243400 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f60003530 0x7f0f78100480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:57.026 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:57.024+0000 7f0f76ffd640 1 -- 172.21.15.184:0/341243400 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f60003530 msgr2=0x7f0f78100480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-09-12T01:53:57.026 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:57.024+0000 7f0f76ffd640 1 -- 172.21.15.184:0/341243400 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f60003530 msgr2=0x7f0f78100480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-12T01:53:57.027 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:57.024+0000 7f0f76ffd640 1 --2- 172.21.15.184:0/341243400 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f60003530 0x7f0f78100480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-12T01:53:57.027 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:57.024+0000 7f0f76ffd640 1 --2- 172.21.15.184:0/341243400 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f60003530 0x7f0f78100480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-12T01:53:57.027 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:57.024+0000 7f0f777fe640 1 -- 172.21.15.184:0/341243400 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0f781009c0 msgr2=0x7f0f78100e20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T01:53:57.027 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:57.024+0000 7f0f777fe640 1 --2- 172.21.15.184:0/341243400 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0f781009c0 0x7f0f78100e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:57.027 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:57.024+0000 7f0f777fe640 1 -- 172.21.15.184:0/341243400 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f60003530 msgr2=0x7f0f78100480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T01:53:57.027 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:57.024+0000 7f0f777fe640 1 --2- 172.21.15.184:0/341243400 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f60003530 0x7f0f78100480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:57.027 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:57.024+0000 7f0f777fe640 1 -- 172.21.15.184:0/341243400 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f781a6030 con 0x7f0f781a3570 2024-09-12T01:53:57.027 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:57.024+0000 7f0f777fe640 1 --2- 172.21.15.184:0/341243400 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f781a3570 0x7f0f781a5960 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f0f7000bd50 tx=0x7f0f7000be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:53:57.027 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:57.024+0000 7f0f5ffff640 1 -- 172.21.15.184:0/341243400 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f70019070 con 0x7f0f781a3570 2024-09-12T01:53:57.027 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:57.024+0000 7f0f5ffff640 1 -- 172.21.15.184:0/341243400 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f70014920 con 0x7f0f781a3570 2024-09-12T01:53:57.027 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:57.024+0000 7f0f5ffff640 1 -- 172.21.15.184:0/341243400 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f70012990 con 0x7f0f781a3570 2024-09-12T01:53:57.027 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:57.024+0000 7f0f7d48f640 1 -- 172.21.15.184:0/341243400 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f781a6320 con 0x7f0f781a3570 2024-09-12T01:53:57.027 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:57.024+0000 7f0f7d48f640 1 -- 172.21.15.184:0/341243400 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f781a6860 con 0x7f0f781a3570 2024-09-12T01:53:57.028 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:57.025+0000 7f0f5ffff640 1 -- 172.21.15.184:0/341243400 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f0f7001c070 con 0x7f0f781a3570 2024-09-12T01:53:57.029 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:57.025+0000 7f0f5ffff640 1 --2- 172.21.15.184:0/341243400 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f0f640394f0 0x7f0f6403b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:57.029 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:57.025+0000 7f0f7d48f640 1 -- 172.21.15.184:0/341243400 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f7810a890 con 0x7f0f781a3570 2024-09-12T01:53:57.029 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:57.026+0000 7f0f5ffff640 1 -- 172.21.15.184:0/341243400 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f0f7004bf10 con 0x7f0f781a3570 2024-09-12T01:53:57.029 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:57.026+0000 7f0f76ffd640 1 --2- 172.21.15.184:0/341243400 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f0f640394f0 0x7f0f6403b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:57.029 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:57.026+0000 7f0f76ffd640 1 --2- 172.21.15.184:0/341243400 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f0f640394f0 0x7f0f6403b9b0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f0f68005dc0 tx=0x7f0f68005d10 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:53:57.031 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:57.031+0000 7f0f5ffff640 1 -- 172.21.15.184:0/341243400 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0f700381f0 con 0x7f0f781a3570 2024-09-12T01:53:57.275 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:57.275+0000 7f0f7d48f640 1 -- 172.21.15.184:0/341243400 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f0f781a6b20 con 0x7f0f781a3570 2024-09-12T01:53:57.275 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:53:57 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T01:53:57.276 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:53:57 smithi184 ceph-mon[35593]: mon.c calling monitor election 2024-09-12T01:53:57.276 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:53:57 smithi184 ceph-mon[35593]: mon.a calling monitor election 2024-09-12T01:53:57.276 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:53:57 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T01:53:57.276 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:53:57 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T01:53:57.276 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:53:57 smithi184 ceph-mon[35593]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:53:57.276 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:53:57 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T01:53:57.276 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:53:57 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T01:53:57.276 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:53:57 smithi184 ceph-mon[35593]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:53:57.276 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:53:57 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T01:53:57.276 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:53:57 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T01:53:57.277 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:53:57 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T01:53:57.277 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:53:57 smithi184 ceph-mon[35593]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-12T01:53:57.277 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:53:57 smithi184 ceph-mon[35593]: monmap epoch 3 2024-09-12T01:53:57.277 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:53:57 smithi184 ceph-mon[35593]: fsid 879f3b56-70a9-11ef-bcea-c7b262605968 2024-09-12T01:53:57.277 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:53:57 smithi184 ceph-mon[35593]: last_changed 2024-09-12T01:53:51.950369+0000 2024-09-12T01:53:57.277 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:53:57 smithi184 ceph-mon[35593]: created 2024-09-12T01:52:09.859051+0000 2024-09-12T01:53:57.277 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:53:57 smithi184 ceph-mon[35593]: min_mon_release 19 (squid) 2024-09-12T01:53:57.277 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:53:57 smithi184 ceph-mon[35593]: election_strategy: 1 2024-09-12T01:53:57.277 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:53:57 smithi184 ceph-mon[35593]: 0: [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] mon.a 2024-09-12T01:53:57.277 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:53:57 smithi184 ceph-mon[35593]: 1: [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] mon.c 2024-09-12T01:53:57.277 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:53:57 smithi184 ceph-mon[35593]: 2: [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] mon.b 2024-09-12T01:53:57.277 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:53:57 smithi184 ceph-mon[35593]: fsmap 2024-09-12T01:53:57.278 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:53:57 smithi184 ceph-mon[35593]: osdmap e4: 0 total, 0 up, 0 in 2024-09-12T01:53:57.278 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:53:57 smithi184 ceph-mon[35593]: mgrmap e13: x(active, since 48s) 2024-09-12T01:53:57.278 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:53:57 smithi184 ceph-mon[35593]: overall HEALTH_OK 2024-09-12T01:53:57.278 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:53:57 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:53:57.278 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:53:57 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:53:57.278 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:53:57 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:53:57.278 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:53:57 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:53:57.278 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:57.275+0000 7f0f5ffff640 1 -- 172.21.15.184:0/341243400 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1279 (secure 0 0 0) 0x7f0f70011df0 con 0x7f0f781a3570 2024-09-12T01:53:57.279 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-12T01:53:57.279 INFO:teuthology.orchestra.run.smithi184.stdout:{"epoch":3,"fsid":"879f3b56-70a9-11ef-bcea-c7b262605968","modified":"2024-09-12T01:53:51.950369Z","created":"2024-09-12T01:52:09.859051Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:3300","nonce":0},{"type":"v1","addr":"172.21.15.16:6789","nonce":0}]},"addr":"172.21.15.16:6789/0","public_addr":"172.21.15.16:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:3300","nonce":0},{"type":"v1","addr":"172.21.15.184:6789","nonce":0}]},"addr":"172.21.15.184:6789/0","public_addr":"172.21.15.184:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:3300","nonce":0},{"type":"v1","addr":"172.21.15.33:6789","nonce":0}]},"addr":"172.21.15.33:6789/0","public_addr":"172.21.15.33:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-12T01:53:57.280 INFO:teuthology.orchestra.run.smithi184.stderr:dumped monmap epoch 3 2024-09-12T01:53:57.280 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:57.278+0000 7f0f7d48f640 1 -- 172.21.15.184:0/341243400 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f0f640394f0 msgr2=0x7f0f6403b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:53:57.280 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:57.278+0000 7f0f7d48f640 1 --2- 172.21.15.184:0/341243400 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f0f640394f0 0x7f0f6403b9b0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f0f68005dc0 tx=0x7f0f68005d10 comp rx=0 tx=0).stop 2024-09-12T01:53:57.280 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:57.278+0000 7f0f7d48f640 1 -- 172.21.15.184:0/341243400 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f781a3570 msgr2=0x7f0f781a5960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:53:57.280 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:57.278+0000 7f0f7d48f640 1 --2- 172.21.15.184:0/341243400 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f781a3570 0x7f0f781a5960 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f0f7000bd50 tx=0x7f0f7000be40 comp rx=0 tx=0).stop 2024-09-12T01:53:57.280 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:57.278+0000 7f0f7d48f640 1 -- 172.21.15.184:0/341243400 shutdown_connections 2024-09-12T01:53:57.280 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:57.278+0000 7f0f7d48f640 1 --2- 172.21.15.184:0/341243400 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f0f640394f0 0x7f0f6403b9b0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:57.280 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:57.278+0000 7f0f7d48f640 1 --2- 172.21.15.184:0/341243400 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f781a3570 0x7f0f781a5960 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:57.280 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:57.278+0000 7f0f7d48f640 1 --2- 172.21.15.184:0/341243400 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0f781009c0 0x7f0f78100e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:57.280 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:57.278+0000 7f0f7d48f640 1 --2- 172.21.15.184:0/341243400 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f60003530 0x7f0f78100480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:57.280 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:57.278+0000 7f0f7d48f640 1 -- 172.21.15.184:0/341243400 >> 172.21.15.184:0/341243400 conn(0x7f0f780fc220 msgr2=0x7f0f780fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:53:57.281 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:57.279+0000 7f0f7d48f640 1 -- 172.21.15.184:0/341243400 shutdown_connections 2024-09-12T01:53:57.281 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:53:57.279+0000 7f0f7d48f640 1 -- 172.21.15.184:0/341243400 wait complete. 2024-09-12T01:53:57.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:57 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T01:53:57.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:57 smithi016 ceph-mon[30947]: mon.c calling monitor election 2024-09-12T01:53:57.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:57 smithi016 ceph-mon[30947]: mon.a calling monitor election 2024-09-12T01:53:57.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:57 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T01:53:57.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:57 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T01:53:57.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:57 smithi016 ceph-mon[30947]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:53:57.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:57 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T01:53:57.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:57 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T01:53:57.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:57 smithi016 ceph-mon[30947]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:53:57.435 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:57 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T01:53:57.435 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:57 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T01:53:57.435 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:57 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T01:53:57.435 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:57 smithi016 ceph-mon[30947]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-12T01:53:57.435 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:57 smithi016 ceph-mon[30947]: monmap epoch 3 2024-09-12T01:53:57.435 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:57 smithi016 ceph-mon[30947]: fsid 879f3b56-70a9-11ef-bcea-c7b262605968 2024-09-12T01:53:57.435 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:57 smithi016 ceph-mon[30947]: last_changed 2024-09-12T01:53:51.950369+0000 2024-09-12T01:53:57.435 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:57 smithi016 ceph-mon[30947]: created 2024-09-12T01:52:09.859051+0000 2024-09-12T01:53:57.435 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:57 smithi016 ceph-mon[30947]: min_mon_release 19 (squid) 2024-09-12T01:53:57.435 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:57 smithi016 ceph-mon[30947]: election_strategy: 1 2024-09-12T01:53:57.435 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:57 smithi016 ceph-mon[30947]: 0: [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] mon.a 2024-09-12T01:53:57.435 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:57 smithi016 ceph-mon[30947]: 1: [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] mon.c 2024-09-12T01:53:57.435 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:57 smithi016 ceph-mon[30947]: 2: [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] mon.b 2024-09-12T01:53:57.435 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:57 smithi016 ceph-mon[30947]: fsmap 2024-09-12T01:53:57.436 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:57 smithi016 ceph-mon[30947]: osdmap e4: 0 total, 0 up, 0 in 2024-09-12T01:53:57.436 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:57 smithi016 ceph-mon[30947]: mgrmap e13: x(active, since 48s) 2024-09-12T01:53:57.436 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:57 smithi016 ceph-mon[30947]: overall HEALTH_OK 2024-09-12T01:53:57.436 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:57 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:53:57.436 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:57 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:53:57.436 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:57 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:53:57.436 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:57 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:53:57.828 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-09-12T01:53:57.828 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph config generate-minimal-conf 2024-09-12T01:53:58.094 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:53:58.121 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:53:58 smithi184 ceph-mon[35593]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:53:58.122 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:53:58 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:53:58.122 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:53:58 smithi184 ceph-mon[35593]: from='client.? 172.21.15.184:0/341243400' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-12T01:53:58.122 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:53:58 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T01:53:58.183 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:58 smithi016 ceph-mon[30947]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:53:58.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:58 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:53:58.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:58 smithi016 ceph-mon[30947]: from='client.? 172.21.15.184:0/341243400' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-12T01:53:58.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:58 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T01:53:58.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:58.561+0000 7f2fd2a0c640 1 -- 172.21.15.16:0/3379383132 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fcc106560 msgr2=0x7f2fcc106940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:53:58.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:58.561+0000 7f2fd2a0c640 1 --2- 172.21.15.16:0/3379383132 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fcc106560 0x7f2fcc106940 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f2fbc009880 tx=0x7f2fbc02a5d0 comp rx=0 tx=0).stop 2024-09-12T01:53:58.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:58.561+0000 7f2fd2a0c640 1 -- 172.21.15.16:0/3379383132 shutdown_connections 2024-09-12T01:53:58.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:58.561+0000 7f2fd2a0c640 1 --2- 172.21.15.16:0/3379383132 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fcc106560 0x7f2fcc106940 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:58.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:58.561+0000 7f2fd2a0c640 1 -- 172.21.15.16:0/3379383132 >> 172.21.15.16:0/3379383132 conn(0x7f2fcc0fc1b0 msgr2=0x7f2fcc0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:53:58.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:58.561+0000 7f2fd2a0c640 1 -- 172.21.15.16:0/3379383132 shutdown_connections 2024-09-12T01:53:58.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:58.561+0000 7f2fd2a0c640 1 -- 172.21.15.16:0/3379383132 wait complete. 2024-09-12T01:53:58.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:58.562+0000 7f2fd2a0c640 1 Processor -- start 2024-09-12T01:53:58.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:58.562+0000 7f2fd2a0c640 1 -- start start 2024-09-12T01:53:58.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:58.562+0000 7f2fd2a0c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2fcc106560 0x7f2fcc199590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:58.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:58.562+0000 7f2fd2a0c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fcc199ad0 0x7f2fcc19de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:58.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:58.562+0000 7f2fd2a0c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2fcc19e3a0 0x7f2fcc19e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:58.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:58.562+0000 7f2fd2a0c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2fcc109a40 con 0x7f2fcc199ad0 2024-09-12T01:53:58.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:58.562+0000 7f2fd2a0c640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2fcc1098c0 con 0x7f2fcc19e3a0 2024-09-12T01:53:58.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:58.562+0000 7f2fd2a0c640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f2fcc109bc0 con 0x7f2fcc106560 2024-09-12T01:53:58.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:58.563+0000 7f2fcb7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fcc199ad0 0x7f2fcc19de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:58.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:58.563+0000 7f2fcb7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fcc199ad0 0x7f2fcc19de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42862/0 (socket says 172.21.15.16:42862) 2024-09-12T01:53:58.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:58.563+0000 7f2fcb7fe640 1 -- 172.21.15.16:0/543096935 learned_addr learned my addr 172.21.15.16:0/543096935 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:53:58.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:58.563+0000 7f2fcbfff640 1 --2- 172.21.15.16:0/543096935 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2fcc106560 0x7f2fcc199590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:58.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:58.563+0000 7f2fcb7fe640 1 -- 172.21.15.16:0/543096935 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2fcc106560 msgr2=0x7f2fcc199590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:53:58.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:58.563+0000 7f2fcb7fe640 1 --2- 172.21.15.16:0/543096935 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2fcc106560 0x7f2fcc199590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:58.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:58.563+0000 7f2fcb7fe640 1 -- 172.21.15.16:0/543096935 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2fcc19e3a0 msgr2=0x7f2fcc19e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:53:58.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:58.563+0000 7f2fcb7fe640 1 --2- 172.21.15.16:0/543096935 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2fcc19e3a0 0x7f2fcc19e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:58.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:58.563+0000 7f2fcb7fe640 1 -- 172.21.15.16:0/543096935 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fcc1a6110 con 0x7f2fcc199ad0 2024-09-12T01:53:58.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:58.563+0000 7f2fcb7fe640 1 --2- 172.21.15.16:0/543096935 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fcc199ad0 0x7f2fcc19de60 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f2fc000b520 tx=0x7f2fc000b9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:53:58.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:58.563+0000 7f2fc97fa640 1 -- 172.21.15.16:0/543096935 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fc0004230 con 0x7f2fcc199ad0 2024-09-12T01:53:58.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:58.563+0000 7f2fd2a0c640 1 -- 172.21.15.16:0/543096935 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fcc1a6340 con 0x7f2fcc199ad0 2024-09-12T01:53:58.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:58.563+0000 7f2fc97fa640 1 -- 172.21.15.16:0/543096935 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2fc00043d0 con 0x7f2fcc199ad0 2024-09-12T01:53:58.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:58.563+0000 7f2fc97fa640 1 -- 172.21.15.16:0/543096935 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fc0003d60 con 0x7f2fcc199ad0 2024-09-12T01:53:58.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:58.563+0000 7f2fd2a0c640 1 -- 172.21.15.16:0/543096935 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fcc1a6920 con 0x7f2fcc199ad0 2024-09-12T01:53:58.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:58.564+0000 7f2fc97fa640 1 -- 172.21.15.16:0/543096935 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f2fc000e070 con 0x7f2fcc199ad0 2024-09-12T01:53:58.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:58.564+0000 7f2fd2a0c640 1 -- 172.21.15.16:0/543096935 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2fcc1069c0 con 0x7f2fcc199ad0 2024-09-12T01:53:58.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:58.565+0000 7f2fc97fa640 1 --2- 172.21.15.16:0/543096935 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f2fb40394e0 0x7f2fb403b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:58.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:58.565+0000 7f2fc97fa640 1 -- 172.21.15.16:0/543096935 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f2fc004ba50 con 0x7f2fcc199ad0 2024-09-12T01:53:58.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:58.565+0000 7f2fcbfff640 1 --2- 172.21.15.16:0/543096935 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f2fb40394e0 0x7f2fb403b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:58.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:58.565+0000 7f2fcbfff640 1 --2- 172.21.15.16:0/543096935 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f2fb40394e0 0x7f2fb403b9a0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f2fbc0096c0 tx=0x7f2fbc005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:53:58.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:58.569+0000 7f2fc97fa640 1 -- 172.21.15.16:0/543096935 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2fc0014020 con 0x7f2fcc199ad0 2024-09-12T01:53:58.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:58.811+0000 7f2fd2a0c640 1 -- 172.21.15.16:0/543096935 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f2fcc1a6d50 con 0x7f2fcc199ad0 2024-09-12T01:53:58.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:58.812+0000 7f2fc97fa640 1 -- 172.21.15.16:0/543096935 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+273 (secure 0 0 0) 0x7f2fc000ed70 con 0x7f2fcc199ad0 2024-09-12T01:53:58.814 INFO:teuthology.orchestra.run.smithi016.stdout:# minimal ceph.conf for 879f3b56-70a9-11ef-bcea-c7b262605968 2024-09-12T01:53:58.814 INFO:teuthology.orchestra.run.smithi016.stdout:[global] 2024-09-12T01:53:58.814 INFO:teuthology.orchestra.run.smithi016.stdout: fsid = 879f3b56-70a9-11ef-bcea-c7b262605968 2024-09-12T01:53:58.814 INFO:teuthology.orchestra.run.smithi016.stdout: mon_host = [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] 2024-09-12T01:53:58.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:58.814+0000 7f2fd2a0c640 1 -- 172.21.15.16:0/543096935 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f2fb40394e0 msgr2=0x7f2fb403b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:53:58.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:58.814+0000 7f2fd2a0c640 1 --2- 172.21.15.16:0/543096935 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f2fb40394e0 0x7f2fb403b9a0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f2fbc0096c0 tx=0x7f2fbc005a20 comp rx=0 tx=0).stop 2024-09-12T01:53:58.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:58.814+0000 7f2fd2a0c640 1 -- 172.21.15.16:0/543096935 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fcc199ad0 msgr2=0x7f2fcc19de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:53:58.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:58.814+0000 7f2fd2a0c640 1 --2- 172.21.15.16:0/543096935 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fcc199ad0 0x7f2fcc19de60 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f2fc000b520 tx=0x7f2fc000b9e0 comp rx=0 tx=0).stop 2024-09-12T01:53:58.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:58.814+0000 7f2fd2a0c640 1 -- 172.21.15.16:0/543096935 shutdown_connections 2024-09-12T01:53:58.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:58.814+0000 7f2fd2a0c640 1 --2- 172.21.15.16:0/543096935 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f2fb40394e0 0x7f2fb403b9a0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:58.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:58.814+0000 7f2fd2a0c640 1 --2- 172.21.15.16:0/543096935 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2fcc19e3a0 0x7f2fcc19e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:58.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:58.814+0000 7f2fd2a0c640 1 --2- 172.21.15.16:0/543096935 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fcc199ad0 0x7f2fcc19de60 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:58.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:58.814+0000 7f2fd2a0c640 1 --2- 172.21.15.16:0/543096935 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2fcc106560 0x7f2fcc199590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:58.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:58.814+0000 7f2fd2a0c640 1 -- 172.21.15.16:0/543096935 >> 172.21.15.16:0/543096935 conn(0x7f2fcc0fc1b0 msgr2=0x7f2fcc0fc590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:53:58.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:58.814+0000 7f2fd2a0c640 1 -- 172.21.15.16:0/543096935 shutdown_connections 2024-09-12T01:53:58.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:53:58.814+0000 7f2fd2a0c640 1 -- 172.21.15.16:0/543096935 wait complete. 2024-09-12T01:53:59.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:53:58 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T01:53:59.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:53:58 smithi033 ceph-mon[35332]: mon.c calling monitor election 2024-09-12T01:53:59.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:53:58 smithi033 ceph-mon[35332]: mon.a calling monitor election 2024-09-12T01:53:59.420 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:53:58 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T01:53:59.420 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:53:58 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T01:53:59.420 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:53:58 smithi033 ceph-mon[35332]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:53:59.420 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:53:58 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T01:53:59.420 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:53:58 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T01:53:59.420 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:53:58 smithi033 ceph-mon[35332]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:53:59.420 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:53:58 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T01:53:59.420 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:53:58 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T01:53:59.420 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:53:58 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T01:53:59.421 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:53:58 smithi033 ceph-mon[35332]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-12T01:53:59.421 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:53:58 smithi033 ceph-mon[35332]: monmap epoch 3 2024-09-12T01:53:59.421 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:53:58 smithi033 ceph-mon[35332]: fsid 879f3b56-70a9-11ef-bcea-c7b262605968 2024-09-12T01:53:59.421 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:53:58 smithi033 ceph-mon[35332]: last_changed 2024-09-12T01:53:51.950369+0000 2024-09-12T01:53:59.421 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:53:58 smithi033 ceph-mon[35332]: created 2024-09-12T01:52:09.859051+0000 2024-09-12T01:53:59.421 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:53:58 smithi033 ceph-mon[35332]: min_mon_release 19 (squid) 2024-09-12T01:53:59.421 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:53:58 smithi033 ceph-mon[35332]: election_strategy: 1 2024-09-12T01:53:59.421 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:53:58 smithi033 ceph-mon[35332]: 0: [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] mon.a 2024-09-12T01:53:59.421 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:53:58 smithi033 ceph-mon[35332]: 1: [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] mon.c 2024-09-12T01:53:59.421 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:53:58 smithi033 ceph-mon[35332]: 2: [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] mon.b 2024-09-12T01:53:59.422 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:53:58 smithi033 ceph-mon[35332]: fsmap 2024-09-12T01:53:59.422 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:53:58 smithi033 ceph-mon[35332]: osdmap e4: 0 total, 0 up, 0 in 2024-09-12T01:53:59.422 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:53:58 smithi033 ceph-mon[35332]: mgrmap e13: x(active, since 48s) 2024-09-12T01:53:59.422 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:53:58 smithi033 ceph-mon[35332]: overall HEALTH_OK 2024-09-12T01:53:59.422 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:53:58 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:53:59.422 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:53:58 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:53:59.422 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:53:58 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:53:59.422 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:53:58 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:53:59.422 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:53:58 smithi033 ceph-mon[35332]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:53:59.422 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:53:58 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:53:59.422 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:53:58 smithi033 ceph-mon[35332]: from='client.? 172.21.15.184:0/341243400' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-12T01:53:59.422 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:53:58 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T01:53:59.591 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-09-12T01:53:59.591 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-12T01:53:59.591 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/etc/ceph/ceph.conf 2024-09-12T01:53:59.622 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-12T01:53:59.622 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-12T01:53:59.688 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-09-12T01:53:59.688 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/etc/ceph/ceph.conf 2024-09-12T01:53:59.717 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-09-12T01:53:59.717 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-12T01:53:59.783 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-12T01:53:59.783 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd of=/etc/ceph/ceph.conf 2024-09-12T01:53:59.810 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-12T01:53:59.810 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-12T01:53:59.860 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:59 smithi016 ceph-mon[30947]: mon.b calling monitor election 2024-09-12T01:53:59.860 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:59 smithi016 ceph-mon[30947]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:53:59.860 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:59 smithi016 ceph-mon[30947]: Updating smithi016:/etc/ceph/ceph.conf 2024-09-12T01:53:59.860 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:59 smithi016 ceph-mon[30947]: Updating smithi033:/etc/ceph/ceph.conf 2024-09-12T01:53:59.860 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:59 smithi016 ceph-mon[30947]: Updating smithi184:/etc/ceph/ceph.conf 2024-09-12T01:53:59.860 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:59 smithi016 ceph-mon[30947]: Updating smithi184:/var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T01:53:59.860 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:59 smithi016 ceph-mon[30947]: Updating smithi033:/var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T01:53:59.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:59 smithi016 ceph-mon[30947]: Updating smithi016:/var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T01:53:59.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:59 smithi016 ceph-mon[30947]: mon.b calling monitor election 2024-09-12T01:53:59.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:59 smithi016 ceph-mon[30947]: mon.c calling monitor election 2024-09-12T01:53:59.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:59 smithi016 ceph-mon[30947]: mon.a calling monitor election 2024-09-12T01:53:59.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:59 smithi016 ceph-mon[30947]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-12T01:53:59.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:59 smithi016 ceph-mon[30947]: monmap epoch 3 2024-09-12T01:53:59.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:59 smithi016 ceph-mon[30947]: fsid 879f3b56-70a9-11ef-bcea-c7b262605968 2024-09-12T01:53:59.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:59 smithi016 ceph-mon[30947]: last_changed 2024-09-12T01:53:51.950369+0000 2024-09-12T01:53:59.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:59 smithi016 ceph-mon[30947]: created 2024-09-12T01:52:09.859051+0000 2024-09-12T01:53:59.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:59 smithi016 ceph-mon[30947]: min_mon_release 19 (squid) 2024-09-12T01:53:59.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:59 smithi016 ceph-mon[30947]: election_strategy: 1 2024-09-12T01:53:59.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:59 smithi016 ceph-mon[30947]: 0: [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] mon.a 2024-09-12T01:53:59.862 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:59 smithi016 ceph-mon[30947]: 1: [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] mon.c 2024-09-12T01:53:59.862 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:59 smithi016 ceph-mon[30947]: 2: [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] mon.b 2024-09-12T01:53:59.862 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:59 smithi016 ceph-mon[30947]: fsmap 2024-09-12T01:53:59.862 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:59 smithi016 ceph-mon[30947]: osdmap e4: 0 total, 0 up, 0 in 2024-09-12T01:53:59.862 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:59 smithi016 ceph-mon[30947]: mgrmap e13: x(active, since 50s) 2024-09-12T01:53:59.862 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:59 smithi016 ceph-mon[30947]: overall HEALTH_OK 2024-09-12T01:53:59.862 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:59 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:53:59.862 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:59 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:53:59.862 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:59 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:53:59.862 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:59 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:53:59.862 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:59 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:53:59.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:59 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:53:59.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:59 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:53:59.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:59 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:53:59.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:59 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:53:59.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:59 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:53:59.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:59 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:53:59.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:59 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T01:53:59.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:59 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-12T01:53:59.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:53:59 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:53:59.872 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:53:59 smithi184 ceph-mon[35593]: mon.b calling monitor election 2024-09-12T01:53:59.872 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:53:59 smithi184 ceph-mon[35593]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:53:59.873 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:53:59 smithi184 ceph-mon[35593]: Updating smithi016:/etc/ceph/ceph.conf 2024-09-12T01:53:59.873 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:53:59 smithi184 ceph-mon[35593]: Updating smithi033:/etc/ceph/ceph.conf 2024-09-12T01:53:59.873 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:53:59 smithi184 ceph-mon[35593]: Updating smithi184:/etc/ceph/ceph.conf 2024-09-12T01:53:59.873 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:53:59 smithi184 ceph-mon[35593]: Updating smithi184:/var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T01:53:59.873 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:53:59 smithi184 ceph-mon[35593]: Updating smithi033:/var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T01:53:59.873 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:53:59 smithi184 ceph-mon[35593]: Updating smithi016:/var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T01:53:59.873 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:53:59 smithi184 ceph-mon[35593]: mon.b calling monitor election 2024-09-12T01:53:59.873 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:53:59 smithi184 ceph-mon[35593]: mon.c calling monitor election 2024-09-12T01:53:59.874 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:53:59 smithi184 ceph-mon[35593]: mon.a calling monitor election 2024-09-12T01:53:59.874 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:53:59 smithi184 ceph-mon[35593]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-12T01:53:59.874 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:53:59 smithi184 ceph-mon[35593]: monmap epoch 3 2024-09-12T01:53:59.874 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:53:59 smithi184 ceph-mon[35593]: fsid 879f3b56-70a9-11ef-bcea-c7b262605968 2024-09-12T01:53:59.874 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:53:59 smithi184 ceph-mon[35593]: last_changed 2024-09-12T01:53:51.950369+0000 2024-09-12T01:53:59.874 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:53:59 smithi184 ceph-mon[35593]: created 2024-09-12T01:52:09.859051+0000 2024-09-12T01:53:59.875 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:53:59 smithi184 ceph-mon[35593]: min_mon_release 19 (squid) 2024-09-12T01:53:59.875 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:53:59 smithi184 ceph-mon[35593]: election_strategy: 1 2024-09-12T01:53:59.875 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:53:59 smithi184 ceph-mon[35593]: 0: [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] mon.a 2024-09-12T01:53:59.875 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:53:59 smithi184 ceph-mon[35593]: 1: [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] mon.c 2024-09-12T01:53:59.875 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:53:59 smithi184 ceph-mon[35593]: 2: [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] mon.b 2024-09-12T01:53:59.875 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:53:59 smithi184 ceph-mon[35593]: fsmap 2024-09-12T01:53:59.875 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:53:59 smithi184 ceph-mon[35593]: osdmap e4: 0 total, 0 up, 0 in 2024-09-12T01:53:59.875 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:53:59 smithi184 ceph-mon[35593]: mgrmap e13: x(active, since 50s) 2024-09-12T01:53:59.876 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:53:59 smithi184 ceph-mon[35593]: overall HEALTH_OK 2024-09-12T01:53:59.876 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:53:59 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:53:59.876 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:53:59 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:53:59.876 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:53:59 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:53:59.876 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:53:59 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:53:59.876 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:53:59 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:53:59.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:53:59 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:53:59.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:53:59 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:53:59.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:53:59 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:53:59.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:53:59 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:53:59.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:53:59 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:53:59.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:53:59 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:53:59.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:53:59 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T01:53:59.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:53:59 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-12T01:53:59.878 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:53:59 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:53:59.879 INFO:tasks.cephadm:Adding mgr.x on smithi016 2024-09-12T01:53:59.879 INFO:tasks.cephadm:Adding mgr.y on smithi033 2024-09-12T01:53:59.879 INFO:tasks.cephadm:Adding mgr.z on smithi184 2024-09-12T01:53:59.880 DEBUG:teuthology.orchestra.run.smithi184:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph orch apply mgr '3;smithi016=x;smithi033=y;smithi184=z' 2024-09-12T01:53:59.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:53:59 smithi033 ceph-mon[35332]: mon.b calling monitor election 2024-09-12T01:53:59.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:53:59 smithi033 ceph-mon[35332]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:53:59.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:53:59 smithi033 ceph-mon[35332]: Updating smithi016:/etc/ceph/ceph.conf 2024-09-12T01:53:59.920 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:53:59 smithi033 ceph-mon[35332]: Updating smithi033:/etc/ceph/ceph.conf 2024-09-12T01:53:59.920 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:53:59 smithi033 ceph-mon[35332]: Updating smithi184:/etc/ceph/ceph.conf 2024-09-12T01:53:59.920 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:53:59 smithi033 ceph-mon[35332]: Updating smithi184:/var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T01:53:59.920 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:53:59 smithi033 ceph-mon[35332]: Updating smithi033:/var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T01:53:59.920 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:53:59 smithi033 ceph-mon[35332]: Updating smithi016:/var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T01:53:59.920 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:53:59 smithi033 ceph-mon[35332]: mon.b calling monitor election 2024-09-12T01:53:59.920 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:53:59 smithi033 ceph-mon[35332]: mon.c calling monitor election 2024-09-12T01:53:59.920 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:53:59 smithi033 ceph-mon[35332]: mon.a calling monitor election 2024-09-12T01:53:59.920 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:53:59 smithi033 ceph-mon[35332]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-12T01:53:59.920 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:53:59 smithi033 ceph-mon[35332]: monmap epoch 3 2024-09-12T01:53:59.920 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:53:59 smithi033 ceph-mon[35332]: fsid 879f3b56-70a9-11ef-bcea-c7b262605968 2024-09-12T01:53:59.920 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:53:59 smithi033 ceph-mon[35332]: last_changed 2024-09-12T01:53:51.950369+0000 2024-09-12T01:53:59.921 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:53:59 smithi033 ceph-mon[35332]: created 2024-09-12T01:52:09.859051+0000 2024-09-12T01:53:59.921 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:53:59 smithi033 ceph-mon[35332]: min_mon_release 19 (squid) 2024-09-12T01:53:59.921 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:53:59 smithi033 ceph-mon[35332]: election_strategy: 1 2024-09-12T01:53:59.921 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:53:59 smithi033 ceph-mon[35332]: 0: [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] mon.a 2024-09-12T01:53:59.921 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:53:59 smithi033 ceph-mon[35332]: 1: [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] mon.c 2024-09-12T01:53:59.921 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:53:59 smithi033 ceph-mon[35332]: 2: [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] mon.b 2024-09-12T01:53:59.921 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:53:59 smithi033 ceph-mon[35332]: fsmap 2024-09-12T01:53:59.921 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:53:59 smithi033 ceph-mon[35332]: osdmap e4: 0 total, 0 up, 0 in 2024-09-12T01:53:59.921 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:53:59 smithi033 ceph-mon[35332]: mgrmap e13: x(active, since 50s) 2024-09-12T01:53:59.921 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:53:59 smithi033 ceph-mon[35332]: overall HEALTH_OK 2024-09-12T01:53:59.921 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:53:59 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:53:59.921 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:53:59 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:53:59.922 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:53:59 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:53:59.922 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:53:59 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:53:59.922 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:53:59 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:53:59.922 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:53:59 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:53:59.922 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:53:59 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:53:59.922 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:53:59 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:53:59.922 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:53:59 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:53:59.922 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:53:59 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:53:59.922 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:53:59 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:53:59.922 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:53:59 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T01:53:59.922 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:53:59 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-12T01:53:59.923 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:53:59 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:54:00.157 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T01:54:00.575 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:54:00.573+0000 7fbda1a8e640 1 -- 172.21.15.184:0/291953267 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd9c1005e0 msgr2=0x7fbd9c1009e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:54:00.575 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:54:00.573+0000 7fbda1a8e640 1 --2- 172.21.15.184:0/291953267 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd9c1005e0 0x7fbd9c1009e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fbd8c009880 tx=0x7fbd8c02a5d0 comp rx=0 tx=0).stop 2024-09-12T01:54:00.575 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:54:00.574+0000 7fbda1a8e640 1 -- 172.21.15.184:0/291953267 shutdown_connections 2024-09-12T01:54:00.575 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:54:00.574+0000 7fbda1a8e640 1 --2- 172.21.15.184:0/291953267 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd9c1005e0 0x7fbd9c1009e0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:00.575 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:54:00.574+0000 7fbda1a8e640 1 -- 172.21.15.184:0/291953267 >> 172.21.15.184:0/291953267 conn(0x7fbd9c0fbdb0 msgr2=0x7fbd9c0fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:54:00.575 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:54:00.574+0000 7fbda1a8e640 1 -- 172.21.15.184:0/291953267 shutdown_connections 2024-09-12T01:54:00.575 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:54:00.574+0000 7fbda1a8e640 1 -- 172.21.15.184:0/291953267 wait complete. 2024-09-12T01:54:00.575 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:54:00.574+0000 7fbda1a8e640 1 Processor -- start 2024-09-12T01:54:00.576 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:54:00.574+0000 7fbda1a8e640 1 -- start start 2024-09-12T01:54:00.576 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:54:00.574+0000 7fbda1a8e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbd9c1005e0 0x7fbd9c197040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:00.576 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:54:00.574+0000 7fbda1a8e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd9c197580 0x7fbd9c19c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:00.576 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:54:00.574+0000 7fbda1a8e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd9c197a00 0x7fbd9c19eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:00.576 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:54:00.574+0000 7fbda1a8e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbd9c1066d0 con 0x7fbd9c197a00 2024-09-12T01:54:00.576 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:54:00.574+0000 7fbda1a8e640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbd9c106550 con 0x7fbd9c197580 2024-09-12T01:54:00.576 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:54:00.574+0000 7fbda1a8e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fbd9c106850 con 0x7fbd9c1005e0 2024-09-12T01:54:00.576 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:54:00.574+0000 7fbd9b7fe640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbd9c1005e0 0x7fbd9c197040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:00.576 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:54:00.574+0000 7fbd9b7fe640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbd9c1005e0 0x7fbd9c197040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.184:37948/0 (socket says 172.21.15.184:37948) 2024-09-12T01:54:00.576 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:54:00.575+0000 7fbd9b7fe640 1 -- 172.21.15.184:0/1812037321 learned_addr learned my addr 172.21.15.184:0/1812037321 (peer_addr_for_me v2:172.21.15.184:0/0) 2024-09-12T01:54:00.576 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:54:00.575+0000 7fbd93fff640 1 --2- 172.21.15.184:0/1812037321 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd9c197580 0x7fbd9c19c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:00.576 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:54:00.575+0000 7fbd9bfff640 1 --2- 172.21.15.184:0/1812037321 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd9c197a00 0x7fbd9c19eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:00.577 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:54:00.575+0000 7fbd9b7fe640 1 -- 172.21.15.184:0/1812037321 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd9c197580 msgr2=0x7fbd9c19c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:54:00.577 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:54:00.575+0000 7fbd9b7fe640 1 --2- 172.21.15.184:0/1812037321 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd9c197580 0x7fbd9c19c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:00.577 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:54:00.575+0000 7fbd9b7fe640 1 -- 172.21.15.184:0/1812037321 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd9c197a00 msgr2=0x7fbd9c19eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:54:00.577 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:54:00.575+0000 7fbd9b7fe640 1 --2- 172.21.15.184:0/1812037321 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd9c197a00 0x7fbd9c19eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:00.577 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:54:00.575+0000 7fbd9b7fe640 1 -- 172.21.15.184:0/1812037321 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd9c19f440 con 0x7fbd9c1005e0 2024-09-12T01:54:00.578 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:54:00.575+0000 7fbd9b7fe640 1 --2- 172.21.15.184:0/1812037321 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbd9c1005e0 0x7fbd9c197040 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fbd8c031610 tx=0x7fbd8c0043c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:54:00.578 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:54:00.575+0000 7fbd997fa640 1 -- 172.21.15.184:0/1812037321 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd8c03f030 con 0x7fbd9c1005e0 2024-09-12T01:54:00.578 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:54:00.575+0000 7fbd997fa640 1 -- 172.21.15.184:0/1812037321 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd8c003ce0 con 0x7fbd9c1005e0 2024-09-12T01:54:00.578 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:54:00.575+0000 7fbda1a8e640 1 -- 172.21.15.184:0/1812037321 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd9c19f6d0 con 0x7fbd9c1005e0 2024-09-12T01:54:00.579 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:54:00.575+0000 7fbd997fa640 1 -- 172.21.15.184:0/1812037321 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd8c035070 con 0x7fbd9c1005e0 2024-09-12T01:54:00.579 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:54:00.575+0000 7fbda1a8e640 1 -- 172.21.15.184:0/1812037321 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd9c19fc80 con 0x7fbd9c1005e0 2024-09-12T01:54:00.579 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:54:00.576+0000 7fbd997fa640 1 -- 172.21.15.184:0/1812037321 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fbd8c042030 con 0x7fbd9c1005e0 2024-09-12T01:54:00.579 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:54:00.576+0000 7fbda1a8e640 1 -- 172.21.15.184:0/1812037321 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd9c190860 con 0x7fbd9c1005e0 2024-09-12T01:54:00.579 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:54:00.576+0000 7fbd997fa640 1 --2- 172.21.15.184:0/1812037321 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fbd880394f0 0x7fbd8803b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:00.579 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:54:00.576+0000 7fbd997fa640 1 -- 172.21.15.184:0/1812037321 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fbd8c06cbf0 con 0x7fbd9c1005e0 2024-09-12T01:54:00.579 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:54:00.577+0000 7fbd93fff640 1 --2- 172.21.15.184:0/1812037321 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fbd880394f0 0x7fbd8803b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:00.580 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:54:00.577+0000 7fbd93fff640 1 --2- 172.21.15.184:0/1812037321 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fbd880394f0 0x7fbd8803b9b0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c198560 tx=0x7fbd84009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:54:00.581 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:54:00.581+0000 7fbd997fa640 1 -- 172.21.15.184:0/1812037321 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbd8c03bdc0 con 0x7fbd9c1005e0 2024-09-12T01:54:00.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:00 smithi016 ceph-mon[30947]: Reconfiguring mon.a (unknown last config time)... 2024-09-12T01:54:00.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:00 smithi016 ceph-mon[30947]: Reconfiguring daemon mon.a on smithi016 2024-09-12T01:54:00.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:00 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T01:54:00.759 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:54:00.758+0000 7fbda1a8e640 1 -- 172.21.15.184:0/1812037321 --> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi016=x;smithi033=y;smithi184=z", "target": ["mon-mgr", ""]}) -- 0x7fbd9c0626c0 con 0x7fbd880394f0 2024-09-12T01:54:00.764 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:54:00.764+0000 7fbd997fa640 1 -- 172.21.15.184:0/1812037321 <== mgr.14150 v2:172.21.15.16:6800/3887444820 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fbd9c0626c0 con 0x7fbd880394f0 2024-09-12T01:54:00.764 INFO:teuthology.orchestra.run.smithi184.stdout:Scheduled mgr update... 2024-09-12T01:54:00.767 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:54:00.766+0000 7fbda1a8e640 1 -- 172.21.15.184:0/1812037321 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fbd880394f0 msgr2=0x7fbd8803b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:54:00.767 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:54:00.766+0000 7fbda1a8e640 1 --2- 172.21.15.184:0/1812037321 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fbd880394f0 0x7fbd8803b9b0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c198560 tx=0x7fbd84009190 comp rx=0 tx=0).stop 2024-09-12T01:54:00.767 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:54:00.766+0000 7fbda1a8e640 1 -- 172.21.15.184:0/1812037321 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbd9c1005e0 msgr2=0x7fbd9c197040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:54:00.767 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:54:00.766+0000 7fbda1a8e640 1 --2- 172.21.15.184:0/1812037321 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbd9c1005e0 0x7fbd9c197040 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fbd8c031610 tx=0x7fbd8c0043c0 comp rx=0 tx=0).stop 2024-09-12T01:54:00.767 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:54:00.766+0000 7fbda1a8e640 1 -- 172.21.15.184:0/1812037321 shutdown_connections 2024-09-12T01:54:00.767 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:54:00.766+0000 7fbda1a8e640 1 --2- 172.21.15.184:0/1812037321 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fbd880394f0 0x7fbd8803b9b0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:00.767 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:54:00.766+0000 7fbda1a8e640 1 --2- 172.21.15.184:0/1812037321 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd9c197a00 0x7fbd9c19eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:00.767 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:54:00.766+0000 7fbda1a8e640 1 --2- 172.21.15.184:0/1812037321 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd9c197580 0x7fbd9c19c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:00.767 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:54:00.766+0000 7fbda1a8e640 1 --2- 172.21.15.184:0/1812037321 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbd9c1005e0 0x7fbd9c197040 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:00.767 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:54:00.766+0000 7fbda1a8e640 1 -- 172.21.15.184:0/1812037321 >> 172.21.15.184:0/1812037321 conn(0x7fbd9c0fbdb0 msgr2=0x7fbd9c06a190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:54:00.767 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:54:00.766+0000 7fbda1a8e640 1 -- 172.21.15.184:0/1812037321 shutdown_connections 2024-09-12T01:54:00.768 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:54:00.766+0000 7fbda1a8e640 1 -- 172.21.15.184:0/1812037321 wait complete. 2024-09-12T01:54:00.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:00 smithi184 ceph-mon[35593]: Reconfiguring mon.a (unknown last config time)... 2024-09-12T01:54:00.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:00 smithi184 ceph-mon[35593]: Reconfiguring daemon mon.a on smithi016 2024-09-12T01:54:00.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:00 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T01:54:00.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:00 smithi033 ceph-mon[35332]: Reconfiguring mon.a (unknown last config time)... 2024-09-12T01:54:00.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:00 smithi033 ceph-mon[35332]: Reconfiguring daemon mon.a on smithi016 2024-09-12T01:54:00.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:00 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T01:54:01.349 DEBUG:teuthology.orchestra.run.smithi033:mgr.y> sudo journalctl -f -n 0 -u ceph-879f3b56-70a9-11ef-bcea-c7b262605968@mgr.y.service 2024-09-12T01:54:01.351 DEBUG:teuthology.orchestra.run.smithi184:mgr.z> sudo journalctl -f -n 0 -u ceph-879f3b56-70a9-11ef-bcea-c7b262605968@mgr.z.service 2024-09-12T01:54:01.353 INFO:tasks.cephadm:Deploying OSDs... 2024-09-12T01:54:01.354 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-12T01:54:01.354 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/scratch_devs of=/dev/stdout 2024-09-12T01:54:01.374 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-12T01:54:01.375 DEBUG:teuthology.orchestra.run.smithi016:> stat /dev/vg_nvme/lv_1 2024-09-12T01:54:01.433 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 01:54:00 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-x[31173]: 2024-09-12T01:54:00.949+0000 7f191c08f640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-12T01:54:01.434 INFO:teuthology.orchestra.run.smithi016.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-12T01:54:01.434 INFO:teuthology.orchestra.run.smithi016.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T01:54:01.434 INFO:teuthology.orchestra.run.smithi016.stdout:Device: 5h/5d Inode: 668 Links: 1 2024-09-12T01:54:01.434 INFO:teuthology.orchestra.run.smithi016.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T01:54:01.435 INFO:teuthology.orchestra.run.smithi016.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T01:54:01.435 INFO:teuthology.orchestra.run.smithi016.stdout:Access: 2024-09-12 01:53:21.101486700 +0000 2024-09-12T01:54:01.435 INFO:teuthology.orchestra.run.smithi016.stdout:Modify: 2024-09-12 01:48:54.190297399 +0000 2024-09-12T01:54:01.435 INFO:teuthology.orchestra.run.smithi016.stdout:Change: 2024-09-12 01:48:54.190297399 +0000 2024-09-12T01:54:01.435 INFO:teuthology.orchestra.run.smithi016.stdout: Birth: 2024-09-12 01:48:54.190297399 +0000 2024-09-12T01:54:01.435 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-12T01:54:01.505 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records in 2024-09-12T01:54:01.505 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records out 2024-09-12T01:54:01.505 INFO:teuthology.orchestra.run.smithi016.stderr:512 bytes copied, 0.00013243 s, 3.9 MB/s 2024-09-12T01:54:01.507 DEBUG:teuthology.orchestra.run.smithi016:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-12T01:54:01.566 DEBUG:teuthology.orchestra.run.smithi016:> stat /dev/vg_nvme/lv_2 2024-09-12T01:54:01.623 INFO:teuthology.orchestra.run.smithi016.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-12T01:54:01.624 INFO:teuthology.orchestra.run.smithi016.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T01:54:01.624 INFO:teuthology.orchestra.run.smithi016.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-09-12T01:54:01.624 INFO:teuthology.orchestra.run.smithi016.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T01:54:01.624 INFO:teuthology.orchestra.run.smithi016.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T01:54:01.624 INFO:teuthology.orchestra.run.smithi016.stdout:Access: 2024-09-12 01:53:21.129486270 +0000 2024-09-12T01:54:01.624 INFO:teuthology.orchestra.run.smithi016.stdout:Modify: 2024-09-12 01:48:54.524292385 +0000 2024-09-12T01:54:01.624 INFO:teuthology.orchestra.run.smithi016.stdout:Change: 2024-09-12 01:48:54.524292385 +0000 2024-09-12T01:54:01.624 INFO:teuthology.orchestra.run.smithi016.stdout: Birth: 2024-09-12 01:48:54.524292385 +0000 2024-09-12T01:54:01.624 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-12T01:54:01.692 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records in 2024-09-12T01:54:01.692 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records out 2024-09-12T01:54:01.692 INFO:teuthology.orchestra.run.smithi016.stderr:512 bytes copied, 0.000124665 s, 4.1 MB/s 2024-09-12T01:54:01.693 DEBUG:teuthology.orchestra.run.smithi016:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-12T01:54:01.751 DEBUG:teuthology.orchestra.run.smithi016:> stat /dev/vg_nvme/lv_3 2024-09-12T01:54:01.810 INFO:teuthology.orchestra.run.smithi016.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-12T01:54:01.810 INFO:teuthology.orchestra.run.smithi016.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T01:54:01.810 INFO:teuthology.orchestra.run.smithi016.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-09-12T01:54:01.810 INFO:teuthology.orchestra.run.smithi016.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T01:54:01.810 INFO:teuthology.orchestra.run.smithi016.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T01:54:01.811 INFO:teuthology.orchestra.run.smithi016.stdout:Access: 2024-09-12 01:53:21.161485780 +0000 2024-09-12T01:54:01.811 INFO:teuthology.orchestra.run.smithi016.stdout:Modify: 2024-09-12 01:48:54.872287161 +0000 2024-09-12T01:54:01.811 INFO:teuthology.orchestra.run.smithi016.stdout:Change: 2024-09-12 01:48:54.872287161 +0000 2024-09-12T01:54:01.811 INFO:teuthology.orchestra.run.smithi016.stdout: Birth: 2024-09-12 01:48:54.872287161 +0000 2024-09-12T01:54:01.811 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-12T01:54:01.875 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:01 smithi016 ceph-mon[30947]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:54:01.875 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:01 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:54:01.875 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:01 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:54:01.875 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:01 smithi016 ceph-mon[30947]: Reconfiguring mon.b (monmap changed)... 2024-09-12T01:54:01.875 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:01 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T01:54:01.875 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:01 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-12T01:54:01.875 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:01 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:54:01.875 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:01 smithi016 ceph-mon[30947]: Reconfiguring daemon mon.b on smithi033 2024-09-12T01:54:01.875 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:01 smithi016 ceph-mon[30947]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi016=x;smithi033=y;smithi184=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:54:01.875 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:01 smithi016 ceph-mon[30947]: Saving service mgr spec with placement smithi016=x;smithi033=y;smithi184=z;count:3 2024-09-12T01:54:01.875 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:01 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:54:01.877 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records in 2024-09-12T01:54:01.877 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records out 2024-09-12T01:54:01.877 INFO:teuthology.orchestra.run.smithi016.stderr:512 bytes copied, 9.8638e-05 s, 5.2 MB/s 2024-09-12T01:54:01.878 DEBUG:teuthology.orchestra.run.smithi016:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-12T01:54:01.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:01 smithi033 ceph-mon[35332]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:54:01.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:01 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:54:01.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:01 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:54:01.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:01 smithi033 ceph-mon[35332]: Reconfiguring mon.b (monmap changed)... 2024-09-12T01:54:01.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:01 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T01:54:01.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:01 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-12T01:54:01.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:01 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:54:01.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:01 smithi033 ceph-mon[35332]: Reconfiguring daemon mon.b on smithi033 2024-09-12T01:54:01.920 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:01 smithi033 ceph-mon[35332]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi016=x;smithi033=y;smithi184=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:54:01.920 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:01 smithi033 ceph-mon[35332]: Saving service mgr spec with placement smithi016=x;smithi033=y;smithi184=z;count:3 2024-09-12T01:54:01.920 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:01 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:54:01.937 DEBUG:teuthology.orchestra.run.smithi016:> stat /dev/vg_nvme/lv_4 2024-09-12T01:54:01.995 INFO:teuthology.orchestra.run.smithi016.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-12T01:54:01.995 INFO:teuthology.orchestra.run.smithi016.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T01:54:01.995 INFO:teuthology.orchestra.run.smithi016.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-09-12T01:54:01.995 INFO:teuthology.orchestra.run.smithi016.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T01:54:01.995 INFO:teuthology.orchestra.run.smithi016.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T01:54:01.995 INFO:teuthology.orchestra.run.smithi016.stdout:Access: 2024-09-12 01:53:21.189485351 +0000 2024-09-12T01:54:01.995 INFO:teuthology.orchestra.run.smithi016.stdout:Modify: 2024-09-12 01:48:55.215282012 +0000 2024-09-12T01:54:01.996 INFO:teuthology.orchestra.run.smithi016.stdout:Change: 2024-09-12 01:48:55.215282012 +0000 2024-09-12T01:54:01.996 INFO:teuthology.orchestra.run.smithi016.stdout: Birth: 2024-09-12 01:48:55.215282012 +0000 2024-09-12T01:54:01.996 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-12T01:54:02.060 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records in 2024-09-12T01:54:02.061 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records out 2024-09-12T01:54:02.061 INFO:teuthology.orchestra.run.smithi016.stderr:512 bytes copied, 0.000112639 s, 4.5 MB/s 2024-09-12T01:54:02.062 DEBUG:teuthology.orchestra.run.smithi016:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-12T01:54:02.119 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-09-12T01:54:02.119 DEBUG:teuthology.orchestra.run.smithi033:> dd if=/scratch_devs of=/dev/stdout 2024-09-12T01:54:02.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:01 smithi184 ceph-mon[35593]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:54:02.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:01 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:54:02.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:01 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:54:02.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:01 smithi184 ceph-mon[35593]: Reconfiguring mon.b (monmap changed)... 2024-09-12T01:54:02.128 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:01 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T01:54:02.128 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:01 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-12T01:54:02.128 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:01 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:54:02.128 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:01 smithi184 ceph-mon[35593]: Reconfiguring daemon mon.b on smithi033 2024-09-12T01:54:02.128 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:01 smithi184 ceph-mon[35593]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi016=x;smithi033=y;smithi184=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:54:02.128 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:01 smithi184 ceph-mon[35593]: Saving service mgr spec with placement smithi016=x;smithi033=y;smithi184=z;count:3 2024-09-12T01:54:02.128 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:01 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:54:02.135 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-12T01:54:02.135 DEBUG:teuthology.orchestra.run.smithi033:> stat /dev/vg_nvme/lv_1 2024-09-12T01:54:02.190 INFO:teuthology.orchestra.run.smithi033.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-12T01:54:02.190 INFO:teuthology.orchestra.run.smithi033.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T01:54:02.190 INFO:teuthology.orchestra.run.smithi033.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-09-12T01:54:02.190 INFO:teuthology.orchestra.run.smithi033.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T01:54:02.190 INFO:teuthology.orchestra.run.smithi033.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T01:54:02.190 INFO:teuthology.orchestra.run.smithi033.stdout:Access: 2024-09-12 01:53:51.400837733 +0000 2024-09-12T01:54:02.190 INFO:teuthology.orchestra.run.smithi033.stdout:Modify: 2024-09-12 01:49:06.834273212 +0000 2024-09-12T01:54:02.190 INFO:teuthology.orchestra.run.smithi033.stdout:Change: 2024-09-12 01:49:06.834273212 +0000 2024-09-12T01:54:02.190 INFO:teuthology.orchestra.run.smithi033.stdout: Birth: 2024-09-12 01:49:06.834273212 +0000 2024-09-12T01:54:02.191 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-12T01:54:02.253 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records in 2024-09-12T01:54:02.254 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records out 2024-09-12T01:54:02.254 INFO:teuthology.orchestra.run.smithi033.stderr:512 bytes copied, 0.000287865 s, 1.8 MB/s 2024-09-12T01:54:02.255 DEBUG:teuthology.orchestra.run.smithi033:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-12T01:54:02.311 DEBUG:teuthology.orchestra.run.smithi033:> stat /dev/vg_nvme/lv_2 2024-09-12T01:54:02.365 INFO:teuthology.orchestra.run.smithi033.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-12T01:54:02.366 INFO:teuthology.orchestra.run.smithi033.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T01:54:02.366 INFO:teuthology.orchestra.run.smithi033.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-09-12T01:54:02.366 INFO:teuthology.orchestra.run.smithi033.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T01:54:02.366 INFO:teuthology.orchestra.run.smithi033.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T01:54:02.366 INFO:teuthology.orchestra.run.smithi033.stdout:Access: 2024-09-12 01:53:51.400837733 +0000 2024-09-12T01:54:02.366 INFO:teuthology.orchestra.run.smithi033.stdout:Modify: 2024-09-12 01:49:07.164267613 +0000 2024-09-12T01:54:02.366 INFO:teuthology.orchestra.run.smithi033.stdout:Change: 2024-09-12 01:49:07.164267613 +0000 2024-09-12T01:54:02.366 INFO:teuthology.orchestra.run.smithi033.stdout: Birth: 2024-09-12 01:49:07.164267613 +0000 2024-09-12T01:54:02.366 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-12T01:54:02.430 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records in 2024-09-12T01:54:02.430 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records out 2024-09-12T01:54:02.430 INFO:teuthology.orchestra.run.smithi033.stderr:512 bytes copied, 0.00018589 s, 2.8 MB/s 2024-09-12T01:54:02.431 DEBUG:teuthology.orchestra.run.smithi033:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-12T01:54:02.487 DEBUG:teuthology.orchestra.run.smithi033:> stat /dev/vg_nvme/lv_3 2024-09-12T01:54:02.543 INFO:teuthology.orchestra.run.smithi033.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-12T01:54:02.543 INFO:teuthology.orchestra.run.smithi033.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T01:54:02.543 INFO:teuthology.orchestra.run.smithi033.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-09-12T01:54:02.543 INFO:teuthology.orchestra.run.smithi033.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T01:54:02.543 INFO:teuthology.orchestra.run.smithi033.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T01:54:02.544 INFO:teuthology.orchestra.run.smithi033.stdout:Access: 2024-09-12 01:53:51.400837733 +0000 2024-09-12T01:54:02.544 INFO:teuthology.orchestra.run.smithi033.stdout:Modify: 2024-09-12 01:49:07.506261809 +0000 2024-09-12T01:54:02.544 INFO:teuthology.orchestra.run.smithi033.stdout:Change: 2024-09-12 01:49:07.506261809 +0000 2024-09-12T01:54:02.544 INFO:teuthology.orchestra.run.smithi033.stdout: Birth: 2024-09-12 01:49:07.506261809 +0000 2024-09-12T01:54:02.544 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-12T01:54:02.606 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records in 2024-09-12T01:54:02.606 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records out 2024-09-12T01:54:02.606 INFO:teuthology.orchestra.run.smithi033.stderr:512 bytes copied, 0.00023153 s, 2.2 MB/s 2024-09-12T01:54:02.607 DEBUG:teuthology.orchestra.run.smithi033:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-12T01:54:02.663 DEBUG:teuthology.orchestra.run.smithi033:> stat /dev/vg_nvme/lv_4 2024-09-12T01:54:02.718 INFO:teuthology.orchestra.run.smithi033.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-12T01:54:02.719 INFO:teuthology.orchestra.run.smithi033.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T01:54:02.719 INFO:teuthology.orchestra.run.smithi033.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-09-12T01:54:02.719 INFO:teuthology.orchestra.run.smithi033.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T01:54:02.719 INFO:teuthology.orchestra.run.smithi033.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T01:54:02.719 INFO:teuthology.orchestra.run.smithi033.stdout:Access: 2024-09-12 01:53:51.401837717 +0000 2024-09-12T01:54:02.719 INFO:teuthology.orchestra.run.smithi033.stdout:Modify: 2024-09-12 01:49:07.850255972 +0000 2024-09-12T01:54:02.719 INFO:teuthology.orchestra.run.smithi033.stdout:Change: 2024-09-12 01:49:07.850255972 +0000 2024-09-12T01:54:02.719 INFO:teuthology.orchestra.run.smithi033.stdout: Birth: 2024-09-12 01:49:07.850255972 +0000 2024-09-12T01:54:02.719 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-12T01:54:02.782 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records in 2024-09-12T01:54:02.782 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records out 2024-09-12T01:54:02.782 INFO:teuthology.orchestra.run.smithi033.stderr:512 bytes copied, 0.000184218 s, 2.8 MB/s 2024-09-12T01:54:02.783 DEBUG:teuthology.orchestra.run.smithi033:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-12T01:54:02.840 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-12T01:54:02.840 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/scratch_devs of=/dev/stdout 2024-09-12T01:54:02.856 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-12T01:54:02.857 DEBUG:teuthology.orchestra.run.smithi184:> stat /dev/vg_nvme/lv_1 2024-09-12T01:54:02.913 INFO:teuthology.orchestra.run.smithi184.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-12T01:54:02.914 INFO:teuthology.orchestra.run.smithi184.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T01:54:02.914 INFO:teuthology.orchestra.run.smithi184.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-09-12T01:54:02.914 INFO:teuthology.orchestra.run.smithi184.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T01:54:02.914 INFO:teuthology.orchestra.run.smithi184.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T01:54:02.914 INFO:teuthology.orchestra.run.smithi184.stdout:Access: 2024-09-12 01:53:42.041071442 +0000 2024-09-12T01:54:02.914 INFO:teuthology.orchestra.run.smithi184.stdout:Modify: 2024-09-12 01:49:25.084863978 +0000 2024-09-12T01:54:02.914 INFO:teuthology.orchestra.run.smithi184.stdout:Change: 2024-09-12 01:49:25.084863978 +0000 2024-09-12T01:54:02.914 INFO:teuthology.orchestra.run.smithi184.stdout: Birth: 2024-09-12 01:49:25.084863978 +0000 2024-09-12T01:54:02.915 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-12T01:54:02.981 INFO:teuthology.orchestra.run.smithi184.stderr:1+0 records in 2024-09-12T01:54:02.981 INFO:teuthology.orchestra.run.smithi184.stderr:1+0 records out 2024-09-12T01:54:02.981 INFO:teuthology.orchestra.run.smithi184.stderr:512 bytes copied, 0.000246915 s, 2.1 MB/s 2024-09-12T01:54:02.982 DEBUG:teuthology.orchestra.run.smithi184:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-12T01:54:03.038 DEBUG:teuthology.orchestra.run.smithi184:> stat /dev/vg_nvme/lv_2 2024-09-12T01:54:03.095 INFO:teuthology.orchestra.run.smithi184.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-12T01:54:03.096 INFO:teuthology.orchestra.run.smithi184.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T01:54:03.096 INFO:teuthology.orchestra.run.smithi184.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-09-12T01:54:03.096 INFO:teuthology.orchestra.run.smithi184.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T01:54:03.096 INFO:teuthology.orchestra.run.smithi184.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T01:54:03.096 INFO:teuthology.orchestra.run.smithi184.stdout:Access: 2024-09-12 01:53:42.042071421 +0000 2024-09-12T01:54:03.096 INFO:teuthology.orchestra.run.smithi184.stdout:Modify: 2024-09-12 01:49:25.408857110 +0000 2024-09-12T01:54:03.096 INFO:teuthology.orchestra.run.smithi184.stdout:Change: 2024-09-12 01:49:25.408857110 +0000 2024-09-12T01:54:03.096 INFO:teuthology.orchestra.run.smithi184.stdout: Birth: 2024-09-12 01:49:25.408857110 +0000 2024-09-12T01:54:03.097 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-12T01:54:03.158 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:02 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:54:03.158 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:02 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:54:03.158 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:02 smithi184 ceph-mon[35593]: Reconfiguring mon.c (monmap changed)... 2024-09-12T01:54:03.158 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:02 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T01:54:03.158 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:02 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-12T01:54:03.158 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:02 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:54:03.158 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:02 smithi184 ceph-mon[35593]: Reconfiguring daemon mon.c on smithi184 2024-09-12T01:54:03.160 INFO:teuthology.orchestra.run.smithi184.stderr:1+0 records in 2024-09-12T01:54:03.160 INFO:teuthology.orchestra.run.smithi184.stderr:1+0 records out 2024-09-12T01:54:03.160 INFO:teuthology.orchestra.run.smithi184.stderr:512 bytes copied, 0.000190192 s, 2.7 MB/s 2024-09-12T01:54:03.161 DEBUG:teuthology.orchestra.run.smithi184:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-12T01:54:03.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:02 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:54:03.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:02 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:54:03.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:02 smithi033 ceph-mon[35332]: Reconfiguring mon.c (monmap changed)... 2024-09-12T01:54:03.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:02 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T01:54:03.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:02 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-12T01:54:03.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:02 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:54:03.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:02 smithi033 ceph-mon[35332]: Reconfiguring daemon mon.c on smithi184 2024-09-12T01:54:03.183 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:02 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:54:03.183 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:02 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:54:03.183 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:02 smithi016 ceph-mon[30947]: Reconfiguring mon.c (monmap changed)... 2024-09-12T01:54:03.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:02 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T01:54:03.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:02 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-12T01:54:03.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:02 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:54:03.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:02 smithi016 ceph-mon[30947]: Reconfiguring daemon mon.c on smithi184 2024-09-12T01:54:03.219 DEBUG:teuthology.orchestra.run.smithi184:> stat /dev/vg_nvme/lv_3 2024-09-12T01:54:03.275 INFO:teuthology.orchestra.run.smithi184.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-12T01:54:03.275 INFO:teuthology.orchestra.run.smithi184.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T01:54:03.275 INFO:teuthology.orchestra.run.smithi184.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-09-12T01:54:03.275 INFO:teuthology.orchestra.run.smithi184.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T01:54:03.275 INFO:teuthology.orchestra.run.smithi184.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T01:54:03.275 INFO:teuthology.orchestra.run.smithi184.stdout:Access: 2024-09-12 01:53:42.042071421 +0000 2024-09-12T01:54:03.275 INFO:teuthology.orchestra.run.smithi184.stdout:Modify: 2024-09-12 01:49:25.755849755 +0000 2024-09-12T01:54:03.275 INFO:teuthology.orchestra.run.smithi184.stdout:Change: 2024-09-12 01:49:25.755849755 +0000 2024-09-12T01:54:03.275 INFO:teuthology.orchestra.run.smithi184.stdout: Birth: 2024-09-12 01:49:25.755849755 +0000 2024-09-12T01:54:03.276 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-12T01:54:03.340 INFO:teuthology.orchestra.run.smithi184.stderr:1+0 records in 2024-09-12T01:54:03.340 INFO:teuthology.orchestra.run.smithi184.stderr:1+0 records out 2024-09-12T01:54:03.340 INFO:teuthology.orchestra.run.smithi184.stderr:512 bytes copied, 0.000177215 s, 2.9 MB/s 2024-09-12T01:54:03.341 DEBUG:teuthology.orchestra.run.smithi184:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-12T01:54:03.397 DEBUG:teuthology.orchestra.run.smithi184:> stat /dev/vg_nvme/lv_4 2024-09-12T01:54:03.453 INFO:teuthology.orchestra.run.smithi184.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-12T01:54:03.453 INFO:teuthology.orchestra.run.smithi184.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T01:54:03.453 INFO:teuthology.orchestra.run.smithi184.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-09-12T01:54:03.453 INFO:teuthology.orchestra.run.smithi184.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T01:54:03.453 INFO:teuthology.orchestra.run.smithi184.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T01:54:03.453 INFO:teuthology.orchestra.run.smithi184.stdout:Access: 2024-09-12 01:53:42.042071421 +0000 2024-09-12T01:54:03.453 INFO:teuthology.orchestra.run.smithi184.stdout:Modify: 2024-09-12 01:49:26.076842950 +0000 2024-09-12T01:54:03.453 INFO:teuthology.orchestra.run.smithi184.stdout:Change: 2024-09-12 01:49:26.076842950 +0000 2024-09-12T01:54:03.453 INFO:teuthology.orchestra.run.smithi184.stdout: Birth: 2024-09-12 01:49:26.076842950 +0000 2024-09-12T01:54:03.454 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-12T01:54:03.516 INFO:teuthology.orchestra.run.smithi184.stderr:1+0 records in 2024-09-12T01:54:03.516 INFO:teuthology.orchestra.run.smithi184.stderr:1+0 records out 2024-09-12T01:54:03.516 INFO:teuthology.orchestra.run.smithi184.stderr:512 bytes copied, 0.000212236 s, 2.4 MB/s 2024-09-12T01:54:03.517 DEBUG:teuthology.orchestra.run.smithi184:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-12T01:54:03.575 INFO:tasks.cephadm:Deploying osd.0 on smithi016 with /dev/vg_nvme/lv_4... 2024-09-12T01:54:03.575 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-12T01:54:03.839 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:54:04.285 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:04 smithi016 ceph-mon[30947]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:54:04.285 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:04 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:54:04.285 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:04 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:54:04.285 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:04 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:54:04.285 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:04 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:54:04.286 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:04 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:54:04.286 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:04 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:54:04.286 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:04 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-12T01:54:04.286 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:04 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-12T01:54:04.286 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:04 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-12T01:54:04.286 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:04 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:54:04.286 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:04 smithi016 ceph-mon[30947]: Deploying daemon mgr.z on smithi184 2024-09-12T01:54:04.290 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:04 smithi184 ceph-mon[35593]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:54:04.290 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:04 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:54:04.290 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:04 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:54:04.291 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:04 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:54:04.291 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:04 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:54:04.291 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:04 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:54:04.291 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:04 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:54:04.291 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:04 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-12T01:54:04.291 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:04 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-12T01:54:04.291 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:04 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-12T01:54:04.291 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:04 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:54:04.292 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:04 smithi184 ceph-mon[35593]: Deploying daemon mgr.z on smithi184 2024-09-12T01:54:04.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:04 smithi033 ceph-mon[35332]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:54:04.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:04 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:54:04.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:04 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:54:04.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:04 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:54:04.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:04 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:54:04.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:04 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:54:04.420 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:04 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:54:04.420 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:04 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-12T01:54:04.420 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:04 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-12T01:54:04.420 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:04 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-12T01:54:04.420 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:04 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:54:04.420 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:04 smithi033 ceph-mon[35332]: Deploying daemon mgr.z on smithi184 2024-09-12T01:54:05.079 INFO:journalctl@ceph.mgr.z.smithi184.stdout:Sep 12 01:54:04 smithi184 systemd[1]: Starting Ceph mgr.z for 879f3b56-70a9-11ef-bcea-c7b262605968... 2024-09-12T01:54:05.363 INFO:journalctl@ceph.mgr.z.smithi184.stdout:Sep 12 01:54:05 smithi184 podman[37380]: 2024-09-12 01:54:04.981854073 +0000 UTC m=+0.015834496 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T01:54:05.364 INFO:journalctl@ceph.mgr.z.smithi184.stdout:Sep 12 01:54:05 smithi184 podman[37380]: 2024-09-12 01:54:05.127763004 +0000 UTC m=+0.161743427 container create 13b9ff9064fad2b8fa478de9dbcaf876a88fe4f6848ca04ad8e190d1e71cbf2c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-z, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T01:54:05.364 INFO:journalctl@ceph.mgr.z.smithi184.stdout:Sep 12 01:54:05 smithi184 podman[37380]: 2024-09-12 01:54:05.363672083 +0000 UTC m=+0.397652511 container init 13b9ff9064fad2b8fa478de9dbcaf876a88fe4f6848ca04ad8e190d1e71cbf2c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-z, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T01:54:05.627 INFO:journalctl@ceph.mgr.z.smithi184.stdout:Sep 12 01:54:05 smithi184 podman[37380]: 2024-09-12 01:54:05.366457105 +0000 UTC m=+0.400437537 container start 13b9ff9064fad2b8fa478de9dbcaf876a88fe4f6848ca04ad8e190d1e71cbf2c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-z, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2) 2024-09-12T01:54:05.627 INFO:journalctl@ceph.mgr.z.smithi184.stdout:Sep 12 01:54:05 smithi184 ceph-mgr[37399]: -- 172.21.15.184:0/1770402814 <== mon.1 v2:172.21.15.184:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5611988490e0 con 0x561198826800 2024-09-12T01:54:05.627 INFO:journalctl@ceph.mgr.z.smithi184.stdout:Sep 12 01:54:05 smithi184 bash[37380]: 13b9ff9064fad2b8fa478de9dbcaf876a88fe4f6848ca04ad8e190d1e71cbf2c 2024-09-12T01:54:05.628 INFO:journalctl@ceph.mgr.z.smithi184.stdout:Sep 12 01:54:05 smithi184 systemd[1]: Started Ceph mgr.z for 879f3b56-70a9-11ef-bcea-c7b262605968. 2024-09-12T01:54:05.628 INFO:journalctl@ceph.mgr.z.smithi184.stdout:Sep 12 01:54:05 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-z[37395]: 2024-09-12T01:54:05.484+0000 7f9ac8067040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-12T01:54:05.628 INFO:journalctl@ceph.mgr.z.smithi184.stdout:Sep 12 01:54:05 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-z[37395]: 2024-09-12T01:54:05.547+0000 7f9ac8067040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-12T01:54:06.020 INFO:journalctl@ceph.mgr.z.smithi184.stdout:Sep 12 01:54:05 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-z[37395]: 2024-09-12T01:54:05.696+0000 7f9ac8067040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-12T01:54:06.020 INFO:journalctl@ceph.mgr.z.smithi184.stdout:Sep 12 01:54:05 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-z[37395]: 2024-09-12T01:54:05.761+0000 7f9ac8067040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-12T01:54:06.020 INFO:journalctl@ceph.mgr.z.smithi184.stdout:Sep 12 01:54:05 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-z[37395]: 2024-09-12T01:54:05.907+0000 7f9ac8067040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-12T01:54:06.268 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:06 smithi033 ceph-mon[35332]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:54:06.268 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:06 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:54:06.268 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:06 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:54:06.268 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:06 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:54:06.268 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:06 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-12T01:54:06.269 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:06 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-12T01:54:06.269 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:06 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-12T01:54:06.269 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:06 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:54:06.312 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:06 smithi016 ceph-mon[30947]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:54:06.312 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:06 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:54:06.312 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:06 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:54:06.312 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:06 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:54:06.312 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:06 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-12T01:54:06.312 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:06 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-12T01:54:06.313 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:06 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-12T01:54:06.313 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:06 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:54:06.316 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:54:06.343 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi016:vg_nvme/lv_4 2024-09-12T01:54:06.360 INFO:journalctl@ceph.mgr.z.smithi184.stdout:Sep 12 01:54:06 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-z[37395]: 2024-09-12T01:54:06.048+0000 7f9ac8067040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-12T01:54:06.361 INFO:journalctl@ceph.mgr.z.smithi184.stdout:Sep 12 01:54:06 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-z[37395]: 2024-09-12T01:54:06.360+0000 7f9ac8067040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-12T01:54:06.361 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:06 smithi184 ceph-mon[35593]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:54:06.361 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:06 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:54:06.362 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:06 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:54:06.362 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:06 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:54:06.362 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:06 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-12T01:54:06.362 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:06 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-12T01:54:06.362 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:06 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-12T01:54:06.362 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:06 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:54:06.595 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:54:06.627 INFO:journalctl@ceph.mgr.z.smithi184.stdout:Sep 12 01:54:06 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-z[37395]: 2024-09-12T01:54:06.420+0000 7f9ac8067040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-12T01:54:06.628 INFO:journalctl@ceph.mgr.z.smithi184.stdout:Sep 12 01:54:06 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-z[37395]: 2024-09-12T01:54:06.600+0000 7f9ac8067040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-12T01:54:07.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:54:07.008+0000 7fd410572640 1 -- 172.21.15.16:0/3876108670 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4081026d0 msgr2=0x7fd408108f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:54:07.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:54:07.008+0000 7fd410572640 1 --2- 172.21.15.16:0/3876108670 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4081026d0 0x7fd408108f60 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd400009ca0 tx=0x7fd40002a8c0 comp rx=0 tx=0).stop 2024-09-12T01:54:07.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:54:07.008+0000 7fd410572640 1 -- 172.21.15.16:0/3876108670 shutdown_connections 2024-09-12T01:54:07.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:54:07.008+0000 7fd410572640 1 --2- 172.21.15.16:0/3876108670 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4081026d0 0x7fd408108f60 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:07.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:54:07.008+0000 7fd410572640 1 --2- 172.21.15.16:0/3876108670 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd408101d10 0x7fd408102190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:07.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:54:07.008+0000 7fd410572640 1 --2- 172.21.15.16:0/3876108670 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd408100b10 0x7fd408100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:07.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:54:07.008+0000 7fd410572640 1 -- 172.21.15.16:0/3876108670 >> 172.21.15.16:0/3876108670 conn(0x7fd4080fc2a0 msgr2=0x7fd4080fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:54:07.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:54:07.008+0000 7fd410572640 1 -- 172.21.15.16:0/3876108670 shutdown_connections 2024-09-12T01:54:07.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:54:07.008+0000 7fd410572640 1 -- 172.21.15.16:0/3876108670 wait complete. 2024-09-12T01:54:07.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:54:07.009+0000 7fd410572640 1 Processor -- start 2024-09-12T01:54:07.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:54:07.009+0000 7fd410572640 1 -- start start 2024-09-12T01:54:07.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:54:07.009+0000 7fd410572640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd408100b10 0x7fd408197eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:07.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:54:07.009+0000 7fd410572640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd408101d10 0x7fd4081983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:07.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:54:07.009+0000 7fd410572640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4081026d0 0x7fd40819f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:07.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:54:07.009+0000 7fd410572640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd40810c790 con 0x7fd4081026d0 2024-09-12T01:54:07.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:54:07.009+0000 7fd410572640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd40810c610 con 0x7fd408100b10 2024-09-12T01:54:07.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:54:07.009+0000 7fd410572640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd40810c910 con 0x7fd408101d10 2024-09-12T01:54:07.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:54:07.009+0000 7fd40eb0f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4081026d0 0x7fd40819f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:07.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:54:07.009+0000 7fd40eb0f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4081026d0 0x7fd40819f470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57396/0 (socket says 172.21.15.16:57396) 2024-09-12T01:54:07.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:54:07.009+0000 7fd40db0d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd408101d10 0x7fd4081983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:07.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:54:07.009+0000 7fd40eb0f640 1 -- 172.21.15.16:0/363084655 learned_addr learned my addr 172.21.15.16:0/363084655 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:54:07.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:54:07.009+0000 7fd40e30e640 1 --2- 172.21.15.16:0/363084655 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd408100b10 0x7fd408197eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:07.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:54:07.010+0000 7fd40eb0f640 1 -- 172.21.15.16:0/363084655 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd408101d10 msgr2=0x7fd4081983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:54:07.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:54:07.010+0000 7fd40eb0f640 1 --2- 172.21.15.16:0/363084655 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd408101d10 0x7fd4081983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:07.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:54:07.010+0000 7fd40eb0f640 1 -- 172.21.15.16:0/363084655 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd408100b10 msgr2=0x7fd408197eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:54:07.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:54:07.010+0000 7fd40eb0f640 1 --2- 172.21.15.16:0/363084655 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd408100b10 0x7fd408197eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:07.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:54:07.010+0000 7fd40eb0f640 1 -- 172.21.15.16:0/363084655 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd40819fae0 con 0x7fd4081026d0 2024-09-12T01:54:07.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:54:07.010+0000 7fd40e30e640 1 --2- 172.21.15.16:0/363084655 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd408100b10 0x7fd408197eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:54:07.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:54:07.010+0000 7fd40eb0f640 1 --2- 172.21.15.16:0/363084655 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4081026d0 0x7fd40819f470 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd400005cc0 tx=0x7fd400009890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:54:07.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:54:07.010+0000 7fd3ff7fe640 1 -- 172.21.15.16:0/363084655 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd400003b70 con 0x7fd4081026d0 2024-09-12T01:54:07.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:54:07.010+0000 7fd410572640 1 -- 172.21.15.16:0/363084655 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd40819fd70 con 0x7fd4081026d0 2024-09-12T01:54:07.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:54:07.010+0000 7fd3ff7fe640 1 -- 172.21.15.16:0/363084655 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd400003d10 con 0x7fd4081026d0 2024-09-12T01:54:07.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:54:07.010+0000 7fd3ff7fe640 1 -- 172.21.15.16:0/363084655 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd40002fc20 con 0x7fd4081026d0 2024-09-12T01:54:07.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:54:07.010+0000 7fd410572640 1 -- 172.21.15.16:0/363084655 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4081a0270 con 0x7fd4081026d0 2024-09-12T01:54:07.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:54:07.011+0000 7fd3ff7fe640 1 -- 172.21.15.16:0/363084655 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fd400004050 con 0x7fd4081026d0 2024-09-12T01:54:07.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:54:07.011+0000 7fd410572640 1 -- 172.21.15.16:0/363084655 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4081a04d0 con 0x7fd4081026d0 2024-09-12T01:54:07.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:54:07.011+0000 7fd3ff7fe640 1 --2- 172.21.15.16:0/363084655 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fd3f40392c0 0x7fd3f403b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:07.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:54:07.011+0000 7fd3ff7fe640 1 -- 172.21.15.16:0/363084655 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fd40006d6f0 con 0x7fd4081026d0 2024-09-12T01:54:07.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:54:07.011+0000 7fd40e30e640 1 --2- 172.21.15.16:0/363084655 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fd3f40392c0 0x7fd3f403b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:07.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:54:07.012+0000 7fd40e30e640 1 --2- 172.21.15.16:0/363084655 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fd3f40392c0 0x7fd3f403b780 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fd4040058c0 tx=0x7fd404005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:54:07.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:54:07.016+0000 7fd3ff7fe640 1 -- 172.21.15.16:0/363084655 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd40004cc00 con 0x7fd4081026d0 2024-09-12T01:54:07.155 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:07 smithi033 ceph-mon[35332]: Deploying daemon mgr.y on smithi033 2024-09-12T01:54:07.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:54:07.202+0000 7fd410572640 1 -- 172.21.15.16:0/363084655 --> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fd4080fe450 con 0x7fd3f40392c0 2024-09-12T01:54:07.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:07 smithi184 ceph-mon[35593]: Deploying daemon mgr.y on smithi033 2024-09-12T01:54:07.419 INFO:journalctl@ceph.mgr.y.smithi033.stdout:Sep 12 01:54:07 smithi033 systemd[1]: Starting Ceph mgr.y for 879f3b56-70a9-11ef-bcea-c7b262605968... 2024-09-12T01:54:07.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:07 smithi016 ceph-mon[30947]: Deploying daemon mgr.y on smithi033 2024-09-12T01:54:07.817 INFO:journalctl@ceph.mgr.y.smithi033.stdout:Sep 12 01:54:07 smithi033 podman[36686]: 2024-09-12 01:54:07.522917138 +0000 UTC m=+0.100368575 container create c98e15cd9dd54a0647d7f1b039d3e31c7457aa76c78d6a31575d2457deec1f9a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-y, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-09-12T01:54:07.817 INFO:journalctl@ceph.mgr.y.smithi033.stdout:Sep 12 01:54:07 smithi033 podman[36686]: 2024-09-12 01:54:07.438197473 +0000 UTC m=+0.015648860 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T01:54:07.817 INFO:journalctl@ceph.mgr.y.smithi033.stdout:Sep 12 01:54:07 smithi033 podman[36686]: 2024-09-12 01:54:07.712378601 +0000 UTC m=+0.289829986 container init c98e15cd9dd54a0647d7f1b039d3e31c7457aa76c78d6a31575d2457deec1f9a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-y, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-09-12T01:54:07.817 INFO:journalctl@ceph.mgr.y.smithi033.stdout:Sep 12 01:54:07 smithi033 podman[36686]: 2024-09-12 01:54:07.71516178 +0000 UTC m=+0.292613165 container start c98e15cd9dd54a0647d7f1b039d3e31c7457aa76c78d6a31575d2457deec1f9a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-y, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, ceph=True, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-09-12T01:54:07.817 INFO:journalctl@ceph.mgr.y.smithi033.stdout:Sep 12 01:54:07 smithi033 ceph-mgr[36704]: -- 172.21.15.33:0/1565483172 <== mon.2 v2:172.21.15.33:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560d2d1e10e0 con 0x560d2d1bf400 2024-09-12T01:54:07.877 INFO:journalctl@ceph.mgr.z.smithi184.stdout:Sep 12 01:54:07 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-z[37395]: 2024-09-12T01:54:07.476+0000 7f9ac8067040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-12T01:54:07.877 INFO:journalctl@ceph.mgr.z.smithi184.stdout:Sep 12 01:54:07 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-z[37395]: 2024-09-12T01:54:07.543+0000 7f9ac8067040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-12T01:54:08.131 INFO:journalctl@ceph.mgr.y.smithi033.stdout:Sep 12 01:54:07 smithi033 bash[36686]: c98e15cd9dd54a0647d7f1b039d3e31c7457aa76c78d6a31575d2457deec1f9a 2024-09-12T01:54:08.131 INFO:journalctl@ceph.mgr.y.smithi033.stdout:Sep 12 01:54:07 smithi033 systemd[1]: Started Ceph mgr.y for 879f3b56-70a9-11ef-bcea-c7b262605968. 2024-09-12T01:54:08.132 INFO:journalctl@ceph.mgr.y.smithi033.stdout:Sep 12 01:54:07 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-y[36700]: 2024-09-12T01:54:07.838+0000 7f8cd2c2c040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-12T01:54:08.132 INFO:journalctl@ceph.mgr.y.smithi033.stdout:Sep 12 01:54:07 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-y[36700]: 2024-09-12T01:54:07.901+0000 7f8cd2c2c040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-12T01:54:08.132 INFO:journalctl@ceph.mgr.y.smithi033.stdout:Sep 12 01:54:08 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-y[36700]: 2024-09-12T01:54:08.060+0000 7f8cd2c2c040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-12T01:54:08.132 INFO:journalctl@ceph.mgr.y.smithi033.stdout:Sep 12 01:54:08 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-y[36700]: 2024-09-12T01:54:08.131+0000 7f8cd2c2c040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-12T01:54:08.132 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:08 smithi033 ceph-mon[35332]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:54:08.132 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:08 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T01:54:08.132 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:08 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T01:54:08.132 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:08 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:54:08.132 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:08 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:54:08.132 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:08 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:54:08.132 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:08 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:54:08.133 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:08 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:54:08.133 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:08 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:54:08.183 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:08 smithi016 ceph-mon[30947]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:54:08.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:08 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T01:54:08.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:08 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T01:54:08.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:08 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:54:08.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:08 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:54:08.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:08 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:54:08.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:08 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:54:08.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:08 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:54:08.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:08 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:54:08.287 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:08 smithi184 ceph-mon[35593]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:54:08.287 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:08 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T01:54:08.287 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:08 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T01:54:08.288 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:08 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:54:08.288 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:08 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:54:08.288 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:08 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:54:08.288 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:08 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:54:08.288 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:08 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:54:08.288 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:08 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:54:08.419 INFO:journalctl@ceph.mgr.y.smithi033.stdout:Sep 12 01:54:08 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-y[36700]: 2024-09-12T01:54:08.293+0000 7f8cd2c2c040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-12T01:54:08.561 INFO:journalctl@ceph.mgr.z.smithi184.stdout:Sep 12 01:54:08 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-z[37395]: 2024-09-12T01:54:08.286+0000 7f9ac8067040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-12T01:54:08.561 INFO:journalctl@ceph.mgr.z.smithi184.stdout:Sep 12 01:54:08 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-z[37395]: 2024-09-12T01:54:08.417+0000 7f9ac8067040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-12T01:54:08.561 INFO:journalctl@ceph.mgr.z.smithi184.stdout:Sep 12 01:54:08 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-z[37395]: 2024-09-12T01:54:08.560+0000 7f9ac8067040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-12T01:54:08.727 INFO:journalctl@ceph.mgr.y.smithi033.stdout:Sep 12 01:54:08 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-y[36700]: 2024-09-12T01:54:08.449+0000 7f8cd2c2c040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-12T01:54:08.849 INFO:journalctl@ceph.mgr.z.smithi184.stdout:Sep 12 01:54:08 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-z[37395]: 2024-09-12T01:54:08.625+0000 7f9ac8067040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-12T01:54:08.849 INFO:journalctl@ceph.mgr.z.smithi184.stdout:Sep 12 01:54:08 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-z[37395]: 2024-09-12T01:54:08.779+0000 7f9ac8067040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-12T01:54:08.849 INFO:journalctl@ceph.mgr.z.smithi184.stdout:Sep 12 01:54:08 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-z[37395]: 2024-09-12T01:54:08.848+0000 7f9ac8067040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-12T01:54:09.030 INFO:journalctl@ceph.mgr.y.smithi033.stdout:Sep 12 01:54:08 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-y[36700]: 2024-09-12T01:54:08.782+0000 7f8cd2c2c040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-12T01:54:09.030 INFO:journalctl@ceph.mgr.y.smithi033.stdout:Sep 12 01:54:08 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-y[36700]: 2024-09-12T01:54:08.847+0000 7f8cd2c2c040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-12T01:54:09.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:09 smithi184 ceph-mon[35593]: from='client.14226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:54:09.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:09 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:54:09.127 INFO:journalctl@ceph.mgr.z.smithi184.stdout:Sep 12 01:54:08 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-z[37395]: 2024-09-12T01:54:08.909+0000 7f9ac8067040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-12T01:54:09.128 INFO:journalctl@ceph.mgr.z.smithi184.stdout:Sep 12 01:54:09 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-z[37395]: 2024-09-12T01:54:09.071+0000 7f9ac8067040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-12T01:54:09.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:09 smithi033 ceph-mon[35332]: from='client.14226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:54:09.471 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:09 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:54:09.472 INFO:journalctl@ceph.mgr.y.smithi033.stdout:Sep 12 01:54:09 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-y[36700]: 2024-09-12T01:54:09.048+0000 7f8cd2c2c040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-12T01:54:09.472 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:09 smithi016 ceph-mon[30947]: from='client.14226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:54:09.473 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:09 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:54:09.473 INFO:journalctl@ceph.mgr.z.smithi184.stdout:Sep 12 01:54:09 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-z[37395]: 2024-09-12T01:54:09.139+0000 7f9ac8067040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-12T01:54:09.473 INFO:journalctl@ceph.mgr.z.smithi184.stdout:Sep 12 01:54:09 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-z[37395]: 2024-09-12T01:54:09.215+0000 7f9ac8067040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-12T01:54:09.473 INFO:journalctl@ceph.mgr.z.smithi184.stdout:Sep 12 01:54:09 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-z[37395]: 2024-09-12T01:54:09.306+0000 7f9ac8067040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-12T01:54:09.473 INFO:journalctl@ceph.mgr.z.smithi184.stdout:Sep 12 01:54:09 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-z[37395]: 2024-09-12T01:54:09.370+0000 7f9ac8067040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-12T01:54:09.473 INFO:journalctl@ceph.mgr.z.smithi184.stdout:Sep 12 01:54:09 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-z[37395]: 2024-09-12T01:54:09.429+0000 7f9ac8067040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-12T01:54:10.377 INFO:journalctl@ceph.mgr.z.smithi184.stdout:Sep 12 01:54:10 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-z[37395]: 2024-09-12T01:54:10.221+0000 7f9ac8067040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-12T01:54:10.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:10 smithi184 ceph-mon[35593]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:54:10.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:10 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:54:10.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:10 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:54:10.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:10 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:54:10.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:10 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:54:10.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:10 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:54:10.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:10 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:54:10.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:10 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:54:10.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:10 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-12T01:54:10.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:10 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-12T01:54:10.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:10 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:54:10.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:10 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/2110400201' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3ea41eb6-e384-4ddc-810f-0f103c127849"}]: dispatch 2024-09-12T01:54:10.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:10 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/2110400201' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3ea41eb6-e384-4ddc-810f-0f103c127849"}]': finished 2024-09-12T01:54:10.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:10 smithi184 ceph-mon[35593]: osdmap e5: 1 total, 0 up, 1 in 2024-09-12T01:54:10.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:10 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T01:54:10.379 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:10 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/962475374' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T01:54:10.419 INFO:journalctl@ceph.mgr.y.smithi033.stdout:Sep 12 01:54:09 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-y[36700]: 2024-09-12T01:54:09.977+0000 7f8cd2c2c040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-12T01:54:10.419 INFO:journalctl@ceph.mgr.y.smithi033.stdout:Sep 12 01:54:10 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-y[36700]: 2024-09-12T01:54:10.046+0000 7f8cd2c2c040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-12T01:54:10.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:10 smithi033 ceph-mon[35332]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:54:10.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:10 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:54:10.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:10 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:54:10.420 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:10 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:54:10.420 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:10 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:54:10.420 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:10 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:54:10.420 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:10 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:54:10.420 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:10 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:54:10.420 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:10 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-12T01:54:10.420 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:10 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-12T01:54:10.420 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:10 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:54:10.420 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:10 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/2110400201' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3ea41eb6-e384-4ddc-810f-0f103c127849"}]: dispatch 2024-09-12T01:54:10.420 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:10 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/2110400201' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3ea41eb6-e384-4ddc-810f-0f103c127849"}]': finished 2024-09-12T01:54:10.420 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:10 smithi033 ceph-mon[35332]: osdmap e5: 1 total, 0 up, 1 in 2024-09-12T01:54:10.420 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:10 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T01:54:10.420 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:10 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/962475374' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T01:54:10.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:10 smithi016 ceph-mon[30947]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:54:10.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:10 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:54:10.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:10 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:54:10.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:10 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:54:10.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:10 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:54:10.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:10 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:54:10.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:10 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:54:10.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:10 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:54:10.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:10 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-12T01:54:10.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:10 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-12T01:54:10.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:10 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:54:10.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:10 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/2110400201' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3ea41eb6-e384-4ddc-810f-0f103c127849"}]: dispatch 2024-09-12T01:54:10.435 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:10 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/2110400201' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3ea41eb6-e384-4ddc-810f-0f103c127849"}]': finished 2024-09-12T01:54:10.435 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:10 smithi016 ceph-mon[30947]: osdmap e5: 1 total, 0 up, 1 in 2024-09-12T01:54:10.435 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:10 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T01:54:10.435 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:10 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/962475374' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T01:54:11.066 INFO:journalctl@ceph.mgr.y.smithi033.stdout:Sep 12 01:54:10 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-y[36700]: 2024-09-12T01:54:10.797+0000 7f8cd2c2c040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-12T01:54:11.066 INFO:journalctl@ceph.mgr.y.smithi033.stdout:Sep 12 01:54:10 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-y[36700]: 2024-09-12T01:54:10.917+0000 7f8cd2c2c040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-12T01:54:11.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:54:11.086+0000 7fd3ff7fe640 1 -- 172.21.15.16:0/363084655 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mgrmap(e 14) ==== 92060+0+0 (secure 0 0 0) 0x7fd4000433d0 con 0x7fd4081026d0 2024-09-12T01:54:11.359 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:11 smithi033 ceph-mon[35332]: Reconfiguring mgr.x (unknown last config time)... 2024-09-12T01:54:11.359 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:11 smithi033 ceph-mon[35332]: Reconfiguring daemon mgr.x on smithi016 2024-09-12T01:54:11.360 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:11 smithi033 ceph-mon[35332]: Standby manager daemon z started 2024-09-12T01:54:11.360 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:11 smithi033 ceph-mon[35332]: from='mgr.? 172.21.15.184:0/3032645249' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-12T01:54:11.360 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:11 smithi033 ceph-mon[35332]: from='mgr.? 172.21.15.184:0/3032645249' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-12T01:54:11.360 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:11 smithi033 ceph-mon[35332]: from='mgr.? 172.21.15.184:0/3032645249' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-12T01:54:11.360 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:11 smithi033 ceph-mon[35332]: from='mgr.? 172.21.15.184:0/3032645249' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-12T01:54:11.360 INFO:journalctl@ceph.mgr.y.smithi033.stdout:Sep 12 01:54:11 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-y[36700]: 2024-09-12T01:54:11.066+0000 7f8cd2c2c040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-12T01:54:11.360 INFO:journalctl@ceph.mgr.y.smithi033.stdout:Sep 12 01:54:11 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-y[36700]: 2024-09-12T01:54:11.133+0000 7f8cd2c2c040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-12T01:54:11.361 INFO:journalctl@ceph.mgr.y.smithi033.stdout:Sep 12 01:54:11 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-y[36700]: 2024-09-12T01:54:11.288+0000 7f8cd2c2c040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-12T01:54:11.361 INFO:journalctl@ceph.mgr.y.smithi033.stdout:Sep 12 01:54:11 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-y[36700]: 2024-09-12T01:54:11.358+0000 7f8cd2c2c040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-12T01:54:11.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:11 smithi184 ceph-mon[35593]: Reconfiguring mgr.x (unknown last config time)... 2024-09-12T01:54:11.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:11 smithi184 ceph-mon[35593]: Reconfiguring daemon mgr.x on smithi016 2024-09-12T01:54:11.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:11 smithi184 ceph-mon[35593]: Standby manager daemon z started 2024-09-12T01:54:11.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:11 smithi184 ceph-mon[35593]: from='mgr.? 172.21.15.184:0/3032645249' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-12T01:54:11.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:11 smithi184 ceph-mon[35593]: from='mgr.? 172.21.15.184:0/3032645249' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-12T01:54:11.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:11 smithi184 ceph-mon[35593]: from='mgr.? 172.21.15.184:0/3032645249' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-12T01:54:11.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:11 smithi184 ceph-mon[35593]: from='mgr.? 172.21.15.184:0/3032645249' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-12T01:54:11.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:11 smithi016 ceph-mon[30947]: Reconfiguring mgr.x (unknown last config time)... 2024-09-12T01:54:11.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:11 smithi016 ceph-mon[30947]: Reconfiguring daemon mgr.x on smithi016 2024-09-12T01:54:11.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:11 smithi016 ceph-mon[30947]: Standby manager daemon z started 2024-09-12T01:54:11.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:11 smithi016 ceph-mon[30947]: from='mgr.? 172.21.15.184:0/3032645249' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-12T01:54:11.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:11 smithi016 ceph-mon[30947]: from='mgr.? 172.21.15.184:0/3032645249' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-12T01:54:11.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:11 smithi016 ceph-mon[30947]: from='mgr.? 172.21.15.184:0/3032645249' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-12T01:54:11.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:11 smithi016 ceph-mon[30947]: from='mgr.? 172.21.15.184:0/3032645249' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-12T01:54:11.653 INFO:journalctl@ceph.mgr.y.smithi033.stdout:Sep 12 01:54:11 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-y[36700]: 2024-09-12T01:54:11.421+0000 7f8cd2c2c040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-12T01:54:11.653 INFO:journalctl@ceph.mgr.y.smithi033.stdout:Sep 12 01:54:11 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-y[36700]: 2024-09-12T01:54:11.585+0000 7f8cd2c2c040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-12T01:54:11.919 INFO:journalctl@ceph.mgr.y.smithi033.stdout:Sep 12 01:54:11 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-y[36700]: 2024-09-12T01:54:11.653+0000 7f8cd2c2c040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-12T01:54:11.920 INFO:journalctl@ceph.mgr.y.smithi033.stdout:Sep 12 01:54:11 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-y[36700]: 2024-09-12T01:54:11.721+0000 7f8cd2c2c040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-12T01:54:11.920 INFO:journalctl@ceph.mgr.y.smithi033.stdout:Sep 12 01:54:11 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-y[36700]: 2024-09-12T01:54:11.807+0000 7f8cd2c2c040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-12T01:54:11.920 INFO:journalctl@ceph.mgr.y.smithi033.stdout:Sep 12 01:54:11 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-y[36700]: 2024-09-12T01:54:11.869+0000 7f8cd2c2c040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-12T01:54:12.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:12 smithi184 ceph-mon[35593]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:54:12.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:12 smithi184 ceph-mon[35593]: mgrmap e14: x(active, since 62s), standbys: z 2024-09-12T01:54:12.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:12 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-12T01:54:12.419 INFO:journalctl@ceph.mgr.y.smithi033.stdout:Sep 12 01:54:11 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-y[36700]: 2024-09-12T01:54:11.930+0000 7f8cd2c2c040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-12T01:54:12.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:12 smithi033 ceph-mon[35332]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:54:12.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:12 smithi033 ceph-mon[35332]: mgrmap e14: x(active, since 62s), standbys: z 2024-09-12T01:54:12.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:12 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-12T01:54:12.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:12 smithi016 ceph-mon[30947]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:54:12.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:12 smithi016 ceph-mon[30947]: mgrmap e14: x(active, since 62s), standbys: z 2024-09-12T01:54:12.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:12 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-12T01:54:13.085 INFO:journalctl@ceph.mgr.y.smithi033.stdout:Sep 12 01:54:12 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-y[36700]: 2024-09-12T01:54:12.755+0000 7f8cd2c2c040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-12T01:54:13.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:54:13.094+0000 7fd3ff7fe640 1 -- 172.21.15.16:0/363084655 <== mon.0 v2:172.21.15.16:3300/0 8 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7fd40004dc60 con 0x7fd4081026d0 2024-09-12T01:54:13.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:13 smithi184 ceph-mon[35593]: Standby manager daemon y started 2024-09-12T01:54:13.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:13 smithi184 ceph-mon[35593]: from='mgr.? 172.21.15.33:0/506554440' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-12T01:54:13.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:13 smithi184 ceph-mon[35593]: from='mgr.? 172.21.15.33:0/506554440' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-12T01:54:13.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:13 smithi184 ceph-mon[35593]: from='mgr.? 172.21.15.33:0/506554440' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-12T01:54:13.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:13 smithi184 ceph-mon[35593]: from='mgr.? 172.21.15.33:0/506554440' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-12T01:54:13.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:13 smithi033 ceph-mon[35332]: Standby manager daemon y started 2024-09-12T01:54:13.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:13 smithi033 ceph-mon[35332]: from='mgr.? 172.21.15.33:0/506554440' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-12T01:54:13.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:13 smithi033 ceph-mon[35332]: from='mgr.? 172.21.15.33:0/506554440' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-12T01:54:13.420 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:13 smithi033 ceph-mon[35332]: from='mgr.? 172.21.15.33:0/506554440' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-12T01:54:13.420 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:13 smithi033 ceph-mon[35332]: from='mgr.? 172.21.15.33:0/506554440' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-12T01:54:13.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:13 smithi016 ceph-mon[30947]: Standby manager daemon y started 2024-09-12T01:54:13.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:13 smithi016 ceph-mon[30947]: from='mgr.? 172.21.15.33:0/506554440' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-12T01:54:13.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:13 smithi016 ceph-mon[30947]: from='mgr.? 172.21.15.33:0/506554440' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-12T01:54:13.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:13 smithi016 ceph-mon[30947]: from='mgr.? 172.21.15.33:0/506554440' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-12T01:54:13.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:13 smithi016 ceph-mon[30947]: from='mgr.? 172.21.15.33:0/506554440' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-12T01:54:14.342 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:14 smithi016 ceph-mon[30947]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:54:14.342 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:14 smithi016 ceph-mon[30947]: mgrmap e15: x(active, since 64s), standbys: z, y 2024-09-12T01:54:14.342 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:14 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-12T01:54:14.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:14 smithi184 ceph-mon[35593]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:54:14.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:14 smithi184 ceph-mon[35593]: mgrmap e15: x(active, since 64s), standbys: z, y 2024-09-12T01:54:14.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:14 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-12T01:54:14.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:14 smithi033 ceph-mon[35332]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:54:14.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:14 smithi033 ceph-mon[35332]: mgrmap e15: x(active, since 64s), standbys: z, y 2024-09-12T01:54:14.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:14 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-12T01:54:16.261 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:16 smithi016 ceph-mon[30947]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:54:16.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:16 smithi184 ceph-mon[35593]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:54:16.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:16 smithi033 ceph-mon[35332]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:54:17.417 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:17 smithi016 ceph-mon[30947]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:54:17.417 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:17 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:54:17.417 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:17 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:54:17.417 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:17 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:54:17.417 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:17 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:54:17.418 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:17 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:54:17.418 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:17 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:54:17.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:17 smithi184 ceph-mon[35593]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:54:17.628 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:17 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:54:17.628 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:17 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:54:17.628 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:17 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:54:17.628 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:17 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:54:17.628 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:17 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:54:17.628 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:17 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:54:17.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:17 smithi033 ceph-mon[35332]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:54:17.670 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:17 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:54:17.670 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:17 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:54:17.670 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:17 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:54:17.670 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:17 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:54:17.670 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:17 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:54:17.670 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:17 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:54:19.402 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:19 smithi016 ceph-mon[30947]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:54:19.403 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:19 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-12T01:54:19.403 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:19 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:54:19.403 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:19 smithi016 ceph-mon[30947]: Deploying daemon osd.0 on smithi016 2024-09-12T01:54:19.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:19 smithi184 ceph-mon[35593]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:54:19.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:19 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-12T01:54:19.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:19 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:54:19.628 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:19 smithi184 ceph-mon[35593]: Deploying daemon osd.0 on smithi016 2024-09-12T01:54:19.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:19 smithi033 ceph-mon[35332]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:54:19.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:19 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-12T01:54:19.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:19 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:54:19.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:19 smithi033 ceph-mon[35332]: Deploying daemon osd.0 on smithi016 2024-09-12T01:54:21.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:21 smithi016 ceph-mon[30947]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:54:21.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:21 smithi184 ceph-mon[35593]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:54:21.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:21 smithi033 ceph-mon[35332]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:54:23.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:23 smithi184 ceph-mon[35593]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:54:23.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:23 smithi033 ceph-mon[35332]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:54:23.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:23 smithi016 ceph-mon[30947]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:54:25.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:25 smithi184 ceph-mon[35593]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:54:25.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:25 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:54:25.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:25 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:54:25.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:25 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:54:25.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:25 smithi033 ceph-mon[35332]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:54:25.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:25 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:54:25.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:25 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:54:25.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:25 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:54:25.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:25 smithi016 ceph-mon[30947]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:54:25.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:25 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:54:25.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:25 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:54:25.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:25 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:54:27.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:27 smithi016 ceph-mon[30947]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:54:27.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:27 smithi184 ceph-mon[35593]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:54:27.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:27 smithi033 ceph-mon[35332]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:54:28.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:28 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:54:28.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:28 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:54:28.878 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:28 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:54:28.878 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:28 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:54:28.878 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:28 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:54:28.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:28 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:54:28.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:28 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:54:28.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:28 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:54:28.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:28 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:54:28.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:28 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:54:28.933 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:28 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:54:28.934 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:28 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:54:28.934 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:28 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:54:28.934 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:28 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:54:28.934 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:28 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:54:29.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:54:29.262+0000 7fd3ff7fe640 1 -- 172.21.15.16:0/363084655 <== mgr.14150 v2:172.21.15.16:6800/3887444820 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fd4080fe450 con 0x7fd3f40392c0 2024-09-12T01:54:29.262 INFO:teuthology.orchestra.run.smithi016.stdout:Created osd(s) 0 on host 'smithi016' 2024-09-12T01:54:29.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:54:29.264+0000 7fd410572640 1 -- 172.21.15.16:0/363084655 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fd3f40392c0 msgr2=0x7fd3f403b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:54:29.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:54:29.264+0000 7fd410572640 1 --2- 172.21.15.16:0/363084655 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fd3f40392c0 0x7fd3f403b780 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fd4040058c0 tx=0x7fd404005850 comp rx=0 tx=0).stop 2024-09-12T01:54:29.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:54:29.264+0000 7fd410572640 1 -- 172.21.15.16:0/363084655 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4081026d0 msgr2=0x7fd40819f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:54:29.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:54:29.264+0000 7fd410572640 1 --2- 172.21.15.16:0/363084655 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4081026d0 0x7fd40819f470 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd400005cc0 tx=0x7fd400009890 comp rx=0 tx=0).stop 2024-09-12T01:54:29.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:54:29.264+0000 7fd410572640 1 -- 172.21.15.16:0/363084655 shutdown_connections 2024-09-12T01:54:29.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:54:29.264+0000 7fd410572640 1 --2- 172.21.15.16:0/363084655 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fd3f40392c0 0x7fd3f403b780 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:29.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:54:29.264+0000 7fd410572640 1 --2- 172.21.15.16:0/363084655 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4081026d0 0x7fd40819f470 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:29.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:54:29.264+0000 7fd410572640 1 --2- 172.21.15.16:0/363084655 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd408101d10 0x7fd4081983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:29.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:54:29.264+0000 7fd410572640 1 --2- 172.21.15.16:0/363084655 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd408100b10 0x7fd408197eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:29.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:54:29.264+0000 7fd410572640 1 -- 172.21.15.16:0/363084655 >> 172.21.15.16:0/363084655 conn(0x7fd4080fc2a0 msgr2=0x7fd4080fdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:54:29.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:54:29.264+0000 7fd410572640 1 -- 172.21.15.16:0/363084655 shutdown_connections 2024-09-12T01:54:29.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:54:29.264+0000 7fd410572640 1 -- 172.21.15.16:0/363084655 wait complete. 2024-09-12T01:54:29.820 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:29 smithi016 ceph-mon[30947]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:54:29.820 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:29 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:54:29.820 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:29 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:54:29.820 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:29 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:54:29.848 DEBUG:teuthology.orchestra.run.smithi016:osd.0> sudo journalctl -f -n 0 -u ceph-879f3b56-70a9-11ef-bcea-c7b262605968@osd.0.service 2024-09-12T01:54:29.852 INFO:tasks.cephadm:Deploying osd.1 on smithi033 with /dev/vg_nvme/lv_4... 2024-09-12T01:54:29.852 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-12T01:54:29.876 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:29 smithi033 ceph-mon[35332]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:54:29.876 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:29 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:54:29.876 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:29 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:54:29.876 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:29 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:54:29.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:29 smithi184 ceph-mon[35593]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:54:29.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:29 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:54:29.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:29 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:54:29.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:29 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:54:30.100 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T01:54:31.313 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 12 01:54:31 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-0[44855]: 2024-09-12T01:54:31.052+0000 7f3e77566640 -1 osd.0 0 log_to_monitors true 2024-09-12T01:54:31.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:31 smithi016 ceph-mon[30947]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:54:31.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:31 smithi016 ceph-mon[30947]: from='osd.0 [v2:172.21.15.16:6802/4000187867,v1:172.21.15.16:6803/4000187867]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-12T01:54:31.746 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:31 smithi033 ceph-mon[35332]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:54:31.746 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:31 smithi033 ceph-mon[35332]: from='osd.0 [v2:172.21.15.16:6802/4000187867,v1:172.21.15.16:6803/4000187867]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-12T01:54:31.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:31 smithi184 ceph-mon[35593]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:54:31.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:31 smithi184 ceph-mon[35593]: from='osd.0 [v2:172.21.15.16:6802/4000187867,v1:172.21.15.16:6803/4000187867]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-12T01:54:32.353 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-12T01:54:32.378 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi033:vg_nvme/lv_4 2024-09-12T01:54:32.671 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T01:54:32.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:32 smithi016 ceph-mon[30947]: from='osd.0 [v2:172.21.15.16:6802/4000187867,v1:172.21.15.16:6803/4000187867]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-12T01:54:32.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:32 smithi016 ceph-mon[30947]: osdmap e6: 1 total, 0 up, 1 in 2024-09-12T01:54:32.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:32 smithi016 ceph-mon[30947]: from='osd.0 [v2:172.21.15.16:6802/4000187867,v1:172.21.15.16:6803/4000187867]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-09-12T01:54:32.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:32 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T01:54:32.683 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 12 01:54:32 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-0[44855]: 2024-09-12T01:54:32.514+0000 7f3e72cc6640 -1 osd.0 0 waiting for initial osdmap 2024-09-12T01:54:32.684 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 12 01:54:32 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-0[44855]: 2024-09-12T01:54:32.516+0000 7f3e6f300640 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T01:54:32.793 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:32 smithi033 ceph-mon[35332]: from='osd.0 [v2:172.21.15.16:6802/4000187867,v1:172.21.15.16:6803/4000187867]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-12T01:54:32.793 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:32 smithi033 ceph-mon[35332]: osdmap e6: 1 total, 0 up, 1 in 2024-09-12T01:54:32.793 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:32 smithi033 ceph-mon[35332]: from='osd.0 [v2:172.21.15.16:6802/4000187867,v1:172.21.15.16:6803/4000187867]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-09-12T01:54:32.793 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:32 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T01:54:32.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:32 smithi184 ceph-mon[35593]: from='osd.0 [v2:172.21.15.16:6802/4000187867,v1:172.21.15.16:6803/4000187867]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-12T01:54:32.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:32 smithi184 ceph-mon[35593]: osdmap e6: 1 total, 0 up, 1 in 2024-09-12T01:54:32.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:32 smithi184 ceph-mon[35593]: from='osd.0 [v2:172.21.15.16:6802/4000187867,v1:172.21.15.16:6803/4000187867]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-09-12T01:54:32.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:32 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T01:54:33.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:54:33.139+0000 7f049f85f640 1 -- 172.21.15.33:0/2699008908 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04981037d0 msgr2=0x7f049810a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:54:33.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:54:33.139+0000 7f049f85f640 1 --2- 172.21.15.33:0/2699008908 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04981037d0 0x7f049810a060 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0490009ca0 tx=0x7f049002a8c0 comp rx=0 tx=0).stop 2024-09-12T01:54:33.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:54:33.139+0000 7f049f85f640 1 -- 172.21.15.33:0/2699008908 shutdown_connections 2024-09-12T01:54:33.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:54:33.139+0000 7f049f85f640 1 --2- 172.21.15.33:0/2699008908 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04981037d0 0x7f049810a060 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:33.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:54:33.139+0000 7f049f85f640 1 --2- 172.21.15.33:0/2699008908 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0498102e10 0x7f0498103290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:33.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:54:33.139+0000 7f049f85f640 1 --2- 172.21.15.33:0/2699008908 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0498101c10 0x7f0498102010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:33.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:54:33.139+0000 7f049f85f640 1 -- 172.21.15.33:0/2699008908 >> 172.21.15.33:0/2699008908 conn(0x7f04980fd3c0 msgr2=0x7f04980ff7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:54:33.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:54:33.140+0000 7f049f85f640 1 -- 172.21.15.33:0/2699008908 shutdown_connections 2024-09-12T01:54:33.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:54:33.140+0000 7f049f85f640 1 -- 172.21.15.33:0/2699008908 wait complete. 2024-09-12T01:54:33.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:54:33.140+0000 7f049f85f640 1 Processor -- start 2024-09-12T01:54:33.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:54:33.140+0000 7f049f85f640 1 -- start start 2024-09-12T01:54:33.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:54:33.141+0000 7f049f85f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0498101c10 0x7f049819e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:33.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:54:33.141+0000 7f049f85f640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0498102e10 0x7f049819e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:33.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:54:33.141+0000 7f049f85f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04981037d0 0x7f04981a5a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:33.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:54:33.141+0000 7f049f85f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f049810d8e0 con 0x7f0498101c10 2024-09-12T01:54:33.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:54:33.141+0000 7f049f85f640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f049810d760 con 0x7f04981037d0 2024-09-12T01:54:33.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:54:33.141+0000 7f049f85f640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f049810da60 con 0x7f0498102e10 2024-09-12T01:54:33.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:54:33.141+0000 7f049d5fb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0498101c10 0x7f049819e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:33.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:54:33.141+0000 7f049ddfc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04981037d0 0x7f04981a5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:33.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:54:33.141+0000 7f049d5fb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0498101c10 0x7f049819e4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.33:41860/0 (socket says 172.21.15.33:41860) 2024-09-12T01:54:33.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:54:33.141+0000 7f049d5fb640 1 -- 172.21.15.33:0/3792226631 learned_addr learned my addr 172.21.15.33:0/3792226631 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-12T01:54:33.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:54:33.141+0000 7f049cdfa640 1 --2- 172.21.15.33:0/3792226631 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0498102e10 0x7f049819e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:33.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:54:33.141+0000 7f049d5fb640 1 -- 172.21.15.33:0/3792226631 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0498102e10 msgr2=0x7f049819e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:54:33.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:54:33.141+0000 7f049d5fb640 1 --2- 172.21.15.33:0/3792226631 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0498102e10 0x7f049819e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:33.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:54:33.141+0000 7f049d5fb640 1 -- 172.21.15.33:0/3792226631 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04981037d0 msgr2=0x7f04981a5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:54:33.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:54:33.141+0000 7f049d5fb640 1 --2- 172.21.15.33:0/3792226631 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04981037d0 0x7f04981a5a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:33.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:54:33.141+0000 7f049d5fb640 1 -- 172.21.15.33:0/3792226631 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04981a60d0 con 0x7f0498101c10 2024-09-12T01:54:33.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:54:33.141+0000 7f049ddfc640 1 --2- 172.21.15.33:0/3792226631 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04981037d0 0x7f04981a5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:54:33.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:54:33.141+0000 7f049d5fb640 1 --2- 172.21.15.33:0/3792226631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0498101c10 0x7f049819e4a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f049400d950 tx=0x7f049400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:54:33.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:54:33.141+0000 7f048e7fc640 1 -- 172.21.15.33:0/3792226631 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0494017070 con 0x7f0498101c10 2024-09-12T01:54:33.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:54:33.142+0000 7f049f85f640 1 -- 172.21.15.33:0/3792226631 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04981a63c0 con 0x7f0498101c10 2024-09-12T01:54:33.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:54:33.142+0000 7f048e7fc640 1 -- 172.21.15.33:0/3792226631 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0494009c30 con 0x7f0498101c10 2024-09-12T01:54:33.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:54:33.142+0000 7f048e7fc640 1 -- 172.21.15.33:0/3792226631 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f049400f600 con 0x7f0498101c10 2024-09-12T01:54:33.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:54:33.142+0000 7f049f85f640 1 -- 172.21.15.33:0/3792226631 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04981a68b0 con 0x7f0498101c10 2024-09-12T01:54:33.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:54:33.143+0000 7f049f85f640 1 -- 172.21.15.33:0/3792226631 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0498106f30 con 0x7f0498101c10 2024-09-12T01:54:33.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:54:33.144+0000 7f048e7fc640 1 -- 172.21.15.33:0/3792226631 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7f049401a070 con 0x7f0498101c10 2024-09-12T01:54:33.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:54:33.144+0000 7f048e7fc640 1 --2- 172.21.15.33:0/3792226631 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f04840a2ba0 0x7f04840a5060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:33.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:54:33.145+0000 7f049cdfa640 1 --2- 172.21.15.33:0/3792226631 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f04840a2ba0 0x7f04840a5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:33.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:54:33.145+0000 7f048e7fc640 1 -- 172.21.15.33:0/3792226631 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1487+0+0 (secure 0 0 0) 0x7f04940ccf60 con 0x7f0498101c10 2024-09-12T01:54:33.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:54:33.145+0000 7f049cdfa640 1 --2- 172.21.15.33:0/3792226631 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f04840a2ba0 0x7f04840a5060 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f049819f9c0 tx=0x7f0488005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:54:33.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:54:33.148+0000 7f048e7fc640 1 -- 172.21.15.33:0/3792226631 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f049409b9b0 con 0x7f0498101c10 2024-09-12T01:54:33.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:54:33.334+0000 7f049f85f640 1 -- 172.21.15.33:0/3792226631 --> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f0498062630 con 0x7f04840a2ba0 2024-09-12T01:54:33.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:33 smithi033 ceph-mon[35332]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:54:33.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:33 smithi033 ceph-mon[35332]: from='osd.0 [v2:172.21.15.16:6802/4000187867,v1:172.21.15.16:6803/4000187867]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-09-12T01:54:33.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:33 smithi033 ceph-mon[35332]: osdmap e7: 1 total, 0 up, 1 in 2024-09-12T01:54:33.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:33 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T01:54:33.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:33 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T01:54:33.670 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:33 smithi033 ceph-mon[35332]: Detected new or changed devices on smithi016 2024-09-12T01:54:33.670 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:33 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:54:33.670 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:33 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:54:33.670 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:33 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:54:33.670 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:33 smithi033 ceph-mon[35332]: Adjusting osd_memory_target on smithi016 to 17092M 2024-09-12T01:54:33.670 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:33 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:54:33.670 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:33 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:54:33.670 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:33 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:54:33.670 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:33 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:54:33.670 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:33 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T01:54:33.670 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:33 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T01:54:33.670 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:33 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:54:33.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:33 smithi016 ceph-mon[30947]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:54:33.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:33 smithi016 ceph-mon[30947]: from='osd.0 [v2:172.21.15.16:6802/4000187867,v1:172.21.15.16:6803/4000187867]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-09-12T01:54:33.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:33 smithi016 ceph-mon[30947]: osdmap e7: 1 total, 0 up, 1 in 2024-09-12T01:54:33.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:33 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T01:54:33.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:33 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T01:54:33.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:33 smithi016 ceph-mon[30947]: Detected new or changed devices on smithi016 2024-09-12T01:54:33.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:33 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:54:33.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:33 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:54:33.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:33 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:54:33.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:33 smithi016 ceph-mon[30947]: Adjusting osd_memory_target on smithi016 to 17092M 2024-09-12T01:54:33.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:33 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:54:33.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:33 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:54:33.685 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:33 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:54:33.685 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:33 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:54:33.685 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:33 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T01:54:33.685 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:33 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T01:54:33.685 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:33 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:54:33.878 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:33 smithi184 ceph-mon[35593]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:54:33.878 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:33 smithi184 ceph-mon[35593]: from='osd.0 [v2:172.21.15.16:6802/4000187867,v1:172.21.15.16:6803/4000187867]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-09-12T01:54:33.878 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:33 smithi184 ceph-mon[35593]: osdmap e7: 1 total, 0 up, 1 in 2024-09-12T01:54:33.878 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:33 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T01:54:33.878 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:33 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T01:54:33.878 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:33 smithi184 ceph-mon[35593]: Detected new or changed devices on smithi016 2024-09-12T01:54:33.878 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:33 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:54:33.879 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:33 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:54:33.879 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:33 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:54:33.879 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:33 smithi184 ceph-mon[35593]: Adjusting osd_memory_target on smithi016 to 17092M 2024-09-12T01:54:33.879 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:33 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:54:33.879 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:33 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:54:33.879 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:33 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:54:33.879 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:33 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:54:33.879 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:33 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T01:54:33.879 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:33 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T01:54:33.879 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:33 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:54:34.795 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:34 smithi033 ceph-mon[35332]: purged_snaps scrub starts 2024-09-12T01:54:34.969 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:34 smithi033 ceph-mon[35332]: purged_snaps scrub ok 2024-09-12T01:54:34.969 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:34 smithi033 ceph-mon[35332]: from='client.14259 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:54:34.969 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:34 smithi033 ceph-mon[35332]: osd.0 [v2:172.21.15.16:6802/4000187867,v1:172.21.15.16:6803/4000187867] boot 2024-09-12T01:54:34.969 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:34 smithi033 ceph-mon[35332]: osdmap e8: 1 total, 1 up, 1 in 2024-09-12T01:54:34.969 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:34 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T01:54:34.970 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:34 smithi016 ceph-mon[30947]: purged_snaps scrub starts 2024-09-12T01:54:34.970 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:34 smithi016 ceph-mon[30947]: purged_snaps scrub ok 2024-09-12T01:54:34.970 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:34 smithi016 ceph-mon[30947]: from='client.14259 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:54:34.971 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:34 smithi016 ceph-mon[30947]: osd.0 [v2:172.21.15.16:6802/4000187867,v1:172.21.15.16:6803/4000187867] boot 2024-09-12T01:54:34.971 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:34 smithi016 ceph-mon[30947]: osdmap e8: 1 total, 1 up, 1 in 2024-09-12T01:54:34.971 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:34 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T01:54:34.971 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:34 smithi184 ceph-mon[35593]: purged_snaps scrub starts 2024-09-12T01:54:34.971 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:34 smithi184 ceph-mon[35593]: purged_snaps scrub ok 2024-09-12T01:54:34.971 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:34 smithi184 ceph-mon[35593]: from='client.14259 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:54:34.971 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:34 smithi184 ceph-mon[35593]: osd.0 [v2:172.21.15.16:6802/4000187867,v1:172.21.15.16:6803/4000187867] boot 2024-09-12T01:54:34.971 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:34 smithi184 ceph-mon[35593]: osdmap e8: 1 total, 1 up, 1 in 2024-09-12T01:54:34.971 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:34 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T01:54:35.787 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:35 smithi033 ceph-mon[35332]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:54:35.787 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:35 smithi033 ceph-mon[35332]: osdmap e9: 1 total, 1 up, 1 in 2024-09-12T01:54:35.787 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:35 smithi033 ceph-mon[35332]: from='client.? 172.21.15.33:0/3232454178' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "07bf63e7-70fa-41b2-af53-12a5fec3f33c"}]: dispatch 2024-09-12T01:54:35.787 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:35 smithi033 ceph-mon[35332]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "07bf63e7-70fa-41b2-af53-12a5fec3f33c"}]: dispatch 2024-09-12T01:54:35.787 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:35 smithi033 ceph-mon[35332]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "07bf63e7-70fa-41b2-af53-12a5fec3f33c"}]': finished 2024-09-12T01:54:35.787 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:35 smithi033 ceph-mon[35332]: osdmap e10: 2 total, 1 up, 2 in 2024-09-12T01:54:35.787 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:35 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T01:54:35.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:35 smithi184 ceph-mon[35593]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:54:35.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:35 smithi184 ceph-mon[35593]: osdmap e9: 1 total, 1 up, 1 in 2024-09-12T01:54:35.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:35 smithi184 ceph-mon[35593]: from='client.? 172.21.15.33:0/3232454178' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "07bf63e7-70fa-41b2-af53-12a5fec3f33c"}]: dispatch 2024-09-12T01:54:35.878 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:35 smithi184 ceph-mon[35593]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "07bf63e7-70fa-41b2-af53-12a5fec3f33c"}]: dispatch 2024-09-12T01:54:35.878 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:35 smithi184 ceph-mon[35593]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "07bf63e7-70fa-41b2-af53-12a5fec3f33c"}]': finished 2024-09-12T01:54:35.878 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:35 smithi184 ceph-mon[35593]: osdmap e10: 2 total, 1 up, 2 in 2024-09-12T01:54:35.878 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:35 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T01:54:35.934 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:35 smithi016 ceph-mon[30947]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:54:35.934 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:35 smithi016 ceph-mon[30947]: osdmap e9: 1 total, 1 up, 1 in 2024-09-12T01:54:35.934 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:35 smithi016 ceph-mon[30947]: from='client.? 172.21.15.33:0/3232454178' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "07bf63e7-70fa-41b2-af53-12a5fec3f33c"}]: dispatch 2024-09-12T01:54:35.934 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:35 smithi016 ceph-mon[30947]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "07bf63e7-70fa-41b2-af53-12a5fec3f33c"}]: dispatch 2024-09-12T01:54:35.934 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:35 smithi016 ceph-mon[30947]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "07bf63e7-70fa-41b2-af53-12a5fec3f33c"}]': finished 2024-09-12T01:54:35.934 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:35 smithi016 ceph-mon[30947]: osdmap e10: 2 total, 1 up, 2 in 2024-09-12T01:54:35.934 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:35 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T01:54:36.835 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:36 smithi016 ceph-mon[30947]: from='client.? 172.21.15.33:0/3831152028' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T01:54:36.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:36 smithi184 ceph-mon[35593]: from='client.? 172.21.15.33:0/3831152028' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T01:54:36.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:36 smithi033 ceph-mon[35332]: from='client.? 172.21.15.33:0/3831152028' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T01:54:37.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:37 smithi184 ceph-mon[35593]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T01:54:37.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:37 smithi033 ceph-mon[35332]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T01:54:37.933 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:37 smithi016 ceph-mon[30947]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T01:54:39.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:39 smithi184 ceph-mon[35593]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T01:54:39.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:39 smithi033 ceph-mon[35332]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T01:54:39.933 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:39 smithi016 ceph-mon[30947]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T01:54:41.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:41 smithi033 ceph-mon[35332]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T01:54:41.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:41 smithi184 ceph-mon[35593]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T01:54:41.936 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:41 smithi016 ceph-mon[30947]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T01:54:43.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:43 smithi184 ceph-mon[35593]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T01:54:43.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:43 smithi033 ceph-mon[35332]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T01:54:43.934 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:43 smithi016 ceph-mon[30947]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T01:54:44.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:44 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-12T01:54:44.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:44 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:54:44.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:44 smithi033 ceph-mon[35332]: Deploying daemon osd.1 on smithi033 2024-09-12T01:54:44.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:44 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-12T01:54:44.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:44 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:54:44.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:44 smithi184 ceph-mon[35593]: Deploying daemon osd.1 on smithi033 2024-09-12T01:54:44.933 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:44 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-12T01:54:44.934 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:44 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:54:44.934 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:44 smithi016 ceph-mon[30947]: Deploying daemon osd.1 on smithi033 2024-09-12T01:54:45.836 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:45 smithi033 ceph-mon[35332]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T01:54:45.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:45 smithi184 ceph-mon[35593]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T01:54:45.933 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:45 smithi016 ceph-mon[30947]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T01:54:47.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:47 smithi184 ceph-mon[35593]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T01:54:47.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:47 smithi033 ceph-mon[35332]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T01:54:47.933 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:47 smithi016 ceph-mon[30947]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T01:54:49.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:49 smithi033 ceph-mon[35332]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T01:54:49.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:49 smithi184 ceph-mon[35593]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T01:54:49.933 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:49 smithi016 ceph-mon[30947]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T01:54:50.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:50 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:54:50.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:50 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:54:50.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:50 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:54:50.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:50 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:54:50.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:50 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:54:50.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:50 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:54:50.933 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:50 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:54:50.933 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:50 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:54:50.934 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:50 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:54:51.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:51 smithi184 ceph-mon[35593]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T01:54:51.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:51 smithi033 ceph-mon[35332]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T01:54:51.933 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:51 smithi016 ceph-mon[30947]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T01:54:53.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:53 smithi184 ceph-mon[35593]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T01:54:53.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:53 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:54:53.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:53 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:54:53.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:53 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:54:53.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:53 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:54:53.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:53 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:54:53.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:53 smithi033 ceph-mon[35332]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T01:54:53.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:53 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:54:53.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:53 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:54:53.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:53 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:54:53.670 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:53 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:54:53.670 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:53 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:54:53.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:53 smithi016 ceph-mon[30947]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T01:54:53.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:53 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:54:53.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:53 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:54:53.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:53 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:54:53.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:53 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:54:53.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:53 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:54:54.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:54:54.386+0000 7f048e7fc640 1 -- 172.21.15.33:0/3792226631 <== mgr.14150 v2:172.21.15.16:6800/3887444820 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f0498062630 con 0x7f04840a2ba0 2024-09-12T01:54:54.387 INFO:teuthology.orchestra.run.smithi033.stdout:Created osd(s) 1 on host 'smithi033' 2024-09-12T01:54:54.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:54:54.388+0000 7f049f85f640 1 -- 172.21.15.33:0/3792226631 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f04840a2ba0 msgr2=0x7f04840a5060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:54:54.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:54:54.388+0000 7f049f85f640 1 --2- 172.21.15.33:0/3792226631 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f04840a2ba0 0x7f04840a5060 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f049819f9c0 tx=0x7f0488005f90 comp rx=0 tx=0).stop 2024-09-12T01:54:54.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:54:54.388+0000 7f049f85f640 1 -- 172.21.15.33:0/3792226631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0498101c10 msgr2=0x7f049819e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:54:54.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:54:54.388+0000 7f049f85f640 1 --2- 172.21.15.33:0/3792226631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0498101c10 0x7f049819e4a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f049400d950 tx=0x7f049400de10 comp rx=0 tx=0).stop 2024-09-12T01:54:54.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:54:54.388+0000 7f049f85f640 1 -- 172.21.15.33:0/3792226631 shutdown_connections 2024-09-12T01:54:54.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:54:54.388+0000 7f049f85f640 1 --2- 172.21.15.33:0/3792226631 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f04840a2ba0 0x7f04840a5060 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:54.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:54:54.388+0000 7f049f85f640 1 --2- 172.21.15.33:0/3792226631 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04981037d0 0x7f04981a5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:54.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:54:54.388+0000 7f049f85f640 1 --2- 172.21.15.33:0/3792226631 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0498102e10 0x7f049819e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:54.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:54:54.388+0000 7f049f85f640 1 --2- 172.21.15.33:0/3792226631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0498101c10 0x7f049819e4a0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:54.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:54:54.388+0000 7f049f85f640 1 -- 172.21.15.33:0/3792226631 >> 172.21.15.33:0/3792226631 conn(0x7f04980fd3c0 msgr2=0x7f0498104100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:54:54.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:54:54.388+0000 7f049f85f640 1 -- 172.21.15.33:0/3792226631 shutdown_connections 2024-09-12T01:54:54.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:54:54.388+0000 7f049f85f640 1 -- 172.21.15.33:0/3792226631 wait complete. 2024-09-12T01:54:54.954 DEBUG:teuthology.orchestra.run.smithi033:osd.1> sudo journalctl -f -n 0 -u ceph-879f3b56-70a9-11ef-bcea-c7b262605968@osd.1.service 2024-09-12T01:54:54.958 INFO:tasks.cephadm:Deploying osd.2 on smithi184 with /dev/vg_nvme/lv_4... 2024-09-12T01:54:54.958 DEBUG:teuthology.orchestra.run.smithi184:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-12T01:54:55.212 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T01:54:55.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:55 smithi184 ceph-mon[35593]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T01:54:55.628 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:55 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:54:55.628 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:55 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:54:55.628 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:55 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:54:55.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:55 smithi033 ceph-mon[35332]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T01:54:55.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:55 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:54:55.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:55 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:54:55.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:55 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:54:55.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:55 smithi016 ceph-mon[30947]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T01:54:55.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:55 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:54:55.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:55 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:54:55.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:55 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:54:56.669 INFO:journalctl@ceph.osd.1.smithi033.stdout:Sep 12 01:54:56 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-1[39829]: 2024-09-12T01:54:56.546+0000 7fe87fd1b640 -1 osd.1 0 log_to_monitors true 2024-09-12T01:54:57.567 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:57 smithi184 ceph-mon[35593]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T01:54:57.567 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:57 smithi184 ceph-mon[35593]: from='osd.1 [v2:172.21.15.33:6800/3862661148,v1:172.21.15.33:6801/3862661148]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-12T01:54:57.567 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:57 smithi184 ceph-mon[35593]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-12T01:54:57.571 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-12T01:54:57.588 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:57 smithi033 ceph-mon[35332]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T01:54:57.588 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:57 smithi033 ceph-mon[35332]: from='osd.1 [v2:172.21.15.33:6800/3862661148,v1:172.21.15.33:6801/3862661148]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-12T01:54:57.588 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:57 smithi033 ceph-mon[35332]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-12T01:54:57.604 DEBUG:teuthology.orchestra.run.smithi184:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi184:vg_nvme/lv_4 2024-09-12T01:54:57.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:57 smithi016 ceph-mon[30947]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T01:54:57.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:57 smithi016 ceph-mon[30947]: from='osd.1 [v2:172.21.15.33:6800/3862661148,v1:172.21.15.33:6801/3862661148]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-12T01:54:57.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:57 smithi016 ceph-mon[30947]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-12T01:54:57.852 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T01:54:58.329 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:54:58.314+0000 7fbf2f584640 1 -- 172.21.15.184:0/4207982588 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbf28101cd0 msgr2=0x7fbf28102150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:54:58.329 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:54:58.314+0000 7fbf2f584640 1 --2- 172.21.15.184:0/4207982588 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbf28101cd0 0x7fbf28102150 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c009900 tx=0x7fbf1c02a410 comp rx=0 tx=0).stop 2024-09-12T01:54:58.329 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:54:58.314+0000 7fbf2f584640 1 -- 172.21.15.184:0/4207982588 shutdown_connections 2024-09-12T01:54:58.329 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:54:58.314+0000 7fbf2f584640 1 --2- 172.21.15.184:0/4207982588 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf28102690 0x7fbf28108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:58.329 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:54:58.314+0000 7fbf2f584640 1 --2- 172.21.15.184:0/4207982588 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbf28101cd0 0x7fbf28102150 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:58.329 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:54:58.314+0000 7fbf2f584640 1 --2- 172.21.15.184:0/4207982588 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf28100ad0 0x7fbf28100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:58.329 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:54:58.314+0000 7fbf2f584640 1 -- 172.21.15.184:0/4207982588 >> 172.21.15.184:0/4207982588 conn(0x7fbf280fc280 msgr2=0x7fbf280fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:54:58.329 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:54:58.314+0000 7fbf2f584640 1 -- 172.21.15.184:0/4207982588 shutdown_connections 2024-09-12T01:54:58.330 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:54:58.314+0000 7fbf2f584640 1 -- 172.21.15.184:0/4207982588 wait complete. 2024-09-12T01:54:58.330 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:54:58.315+0000 7fbf2f584640 1 Processor -- start 2024-09-12T01:54:58.330 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:54:58.315+0000 7fbf2f584640 1 -- start start 2024-09-12T01:54:58.330 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:54:58.315+0000 7fbf2f584640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbf28100ad0 0x7fbf28197e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:58.330 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:54:58.315+0000 7fbf2f584640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf28101cd0 0x7fbf28198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:58.330 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:54:58.315+0000 7fbf2f584640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf28102690 0x7fbf2819f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:58.330 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:54:58.315+0000 7fbf2f584640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbf2810d8e0 con 0x7fbf28101cd0 2024-09-12T01:54:58.330 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:54:58.315+0000 7fbf2f584640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbf2810d760 con 0x7fbf28102690 2024-09-12T01:54:58.330 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:54:58.315+0000 7fbf2f584640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fbf2810da60 con 0x7fbf28100ad0 2024-09-12T01:54:58.330 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:54:58.315+0000 7fbf2d320640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbf28100ad0 0x7fbf28197e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:58.330 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:54:58.315+0000 7fbf2d320640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbf28100ad0 0x7fbf28197e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.184:36576/0 (socket says 172.21.15.184:36576) 2024-09-12T01:54:58.330 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:54:58.315+0000 7fbf2d320640 1 -- 172.21.15.184:0/326389990 learned_addr learned my addr 172.21.15.184:0/326389990 (peer_addr_for_me v2:172.21.15.184:0/0) 2024-09-12T01:54:58.331 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:54:58.315+0000 7fbf2cb1f640 1 --2- 172.21.15.184:0/326389990 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf28101cd0 0x7fbf28198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:58.331 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:54:58.315+0000 7fbf2d320640 1 -- 172.21.15.184:0/326389990 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf28102690 msgr2=0x7fbf2819f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:54:58.331 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:54:58.315+0000 7fbf2d320640 1 --2- 172.21.15.184:0/326389990 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf28102690 0x7fbf2819f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:58.331 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:54:58.315+0000 7fbf2d320640 1 -- 172.21.15.184:0/326389990 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf28101cd0 msgr2=0x7fbf28198390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:54:58.331 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:54:58.315+0000 7fbf2d320640 1 --2- 172.21.15.184:0/326389990 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf28101cd0 0x7fbf28198390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:58.331 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:54:58.315+0000 7fbf2d320640 1 -- 172.21.15.184:0/326389990 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf2819fa80 con 0x7fbf28100ad0 2024-09-12T01:54:58.331 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:54:58.315+0000 7fbf2cb1f640 1 --2- 172.21.15.184:0/326389990 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf28101cd0 0x7fbf28198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:54:58.331 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:54:58.315+0000 7fbf2d320640 1 --2- 172.21.15.184:0/326389990 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbf28100ad0 0x7fbf28197e50 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fbf1800ea80 tx=0x7fbf1800ef40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:54:58.331 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:54:58.315+0000 7fbf267fc640 1 -- 172.21.15.184:0/326389990 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf1800ce40 con 0x7fbf28100ad0 2024-09-12T01:54:58.331 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:54:58.316+0000 7fbf2f584640 1 -- 172.21.15.184:0/326389990 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf2819fd70 con 0x7fbf28100ad0 2024-09-12T01:54:58.331 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:54:58.316+0000 7fbf267fc640 1 -- 172.21.15.184:0/326389990 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf18011950 con 0x7fbf28100ad0 2024-09-12T01:54:58.331 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:54:58.316+0000 7fbf267fc640 1 -- 172.21.15.184:0/326389990 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf1801ab80 con 0x7fbf28100ad0 2024-09-12T01:54:58.332 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:54:58.316+0000 7fbf2f584640 1 -- 172.21.15.184:0/326389990 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf281a0210 con 0x7fbf28100ad0 2024-09-12T01:54:58.332 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:54:58.316+0000 7fbf2f584640 1 -- 172.21.15.184:0/326389990 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf28107e50 con 0x7fbf28100ad0 2024-09-12T01:54:58.332 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:54:58.318+0000 7fbf267fc640 1 -- 172.21.15.184:0/326389990 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7fbf18011b00 con 0x7fbf28100ad0 2024-09-12T01:54:58.332 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:54:58.318+0000 7fbf267fc640 1 --2- 172.21.15.184:0/326389990 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fbf100a2b20 0x7fbf100a4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:58.332 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:54:58.318+0000 7fbf267fc640 1 -- 172.21.15.184:0/326389990 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(11..11 src has 1..11) ==== 1903+0+0 (secure 0 0 0) 0x7fbf180cdcd0 con 0x7fbf28100ad0 2024-09-12T01:54:58.332 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:54:58.318+0000 7fbf2cb1f640 1 --2- 172.21.15.184:0/326389990 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fbf100a2b20 0x7fbf100a4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:58.332 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:54:58.319+0000 7fbf2cb1f640 1 --2- 172.21.15.184:0/326389990 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fbf100a2b20 0x7fbf100a4fe0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c006010 tx=0x7fbf1c000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:54:58.332 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:54:58.323+0000 7fbf267fc640 1 -- 172.21.15.184:0/326389990 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbf18016030 con 0x7fbf28100ad0 2024-09-12T01:54:58.497 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:54:58.496+0000 7fbf2f584640 1 -- 172.21.15.184:0/326389990 --> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi184:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fbf28062630 con 0x7fbf100a2b20 2024-09-12T01:54:58.497 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:58 smithi184 ceph-mon[35593]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-12T01:54:58.497 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:58 smithi184 ceph-mon[35593]: from='osd.1 [v2:172.21.15.33:6800/3862661148,v1:172.21.15.33:6801/3862661148]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-09-12T01:54:58.497 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:58 smithi184 ceph-mon[35593]: osdmap e11: 2 total, 1 up, 2 in 2024-09-12T01:54:58.497 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:58 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T01:54:58.497 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:58 smithi184 ceph-mon[35593]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-09-12T01:54:58.497 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:58 smithi184 ceph-mon[35593]: Detected new or changed devices on smithi033 2024-09-12T01:54:58.497 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:58 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:54:58.498 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:58 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:54:58.498 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:58 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:54:58.498 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:58 smithi184 ceph-mon[35593]: Adjusting osd_memory_target on smithi033 to 17092M 2024-09-12T01:54:58.498 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:58 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:54:58.498 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:58 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:54:58.498 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:58 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:54:58.498 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:58 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:54:58.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:58 smithi033 ceph-mon[35332]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-12T01:54:58.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:58 smithi033 ceph-mon[35332]: from='osd.1 [v2:172.21.15.33:6800/3862661148,v1:172.21.15.33:6801/3862661148]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-09-12T01:54:58.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:58 smithi033 ceph-mon[35332]: osdmap e11: 2 total, 1 up, 2 in 2024-09-12T01:54:58.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:58 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T01:54:58.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:58 smithi033 ceph-mon[35332]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-09-12T01:54:58.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:58 smithi033 ceph-mon[35332]: Detected new or changed devices on smithi033 2024-09-12T01:54:58.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:58 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:54:58.670 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:58 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:54:58.670 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:58 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:54:58.670 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:58 smithi033 ceph-mon[35332]: Adjusting osd_memory_target on smithi033 to 17092M 2024-09-12T01:54:58.670 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:58 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:54:58.670 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:58 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:54:58.670 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:58 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:54:58.670 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:58 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:54:58.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:58 smithi016 ceph-mon[30947]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-12T01:54:58.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:58 smithi016 ceph-mon[30947]: from='osd.1 [v2:172.21.15.33:6800/3862661148,v1:172.21.15.33:6801/3862661148]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-09-12T01:54:58.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:58 smithi016 ceph-mon[30947]: osdmap e11: 2 total, 1 up, 2 in 2024-09-12T01:54:58.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:58 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T01:54:58.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:58 smithi016 ceph-mon[30947]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-09-12T01:54:58.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:58 smithi016 ceph-mon[30947]: Detected new or changed devices on smithi033 2024-09-12T01:54:58.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:58 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:54:58.685 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:58 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:54:58.685 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:58 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:54:58.685 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:58 smithi016 ceph-mon[30947]: Adjusting osd_memory_target on smithi033 to 17092M 2024-09-12T01:54:58.685 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:58 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:54:58.685 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:58 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:54:58.685 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:58 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:54:58.685 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:58 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:54:59.419 INFO:journalctl@ceph.osd.1.smithi033.stdout:Sep 12 01:54:59 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-1[39829]: 2024-09-12T01:54:59.165+0000 7fe87b47b640 -1 osd.1 0 waiting for initial osdmap 2024-09-12T01:54:59.419 INFO:journalctl@ceph.osd.1.smithi033.stdout:Sep 12 01:54:59 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-1[39829]: 2024-09-12T01:54:59.176+0000 7fe873aad640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T01:54:59.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:59 smithi033 ceph-mon[35332]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T01:54:59.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:59 smithi033 ceph-mon[35332]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]': finished 2024-09-12T01:54:59.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:59 smithi033 ceph-mon[35332]: osdmap e12: 2 total, 1 up, 2 in 2024-09-12T01:54:59.420 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:59 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T01:54:59.420 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:59 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T01:54:59.420 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:59 smithi033 ceph-mon[35332]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi184:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:54:59.420 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:59 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T01:54:59.420 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:59 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T01:54:59.420 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:59 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:54:59.420 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:54:59 smithi033 ceph-mon[35332]: from='osd.1 ' entity='osd.1' 2024-09-12T01:54:59.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:59 smithi184 ceph-mon[35593]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T01:54:59.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:59 smithi184 ceph-mon[35593]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]': finished 2024-09-12T01:54:59.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:59 smithi184 ceph-mon[35593]: osdmap e12: 2 total, 1 up, 2 in 2024-09-12T01:54:59.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:59 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T01:54:59.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:59 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T01:54:59.628 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:59 smithi184 ceph-mon[35593]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi184:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:54:59.628 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:59 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T01:54:59.628 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:59 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T01:54:59.628 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:59 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:54:59.628 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:54:59 smithi184 ceph-mon[35593]: from='osd.1 ' entity='osd.1' 2024-09-12T01:54:59.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:59 smithi016 ceph-mon[30947]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T01:54:59.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:59 smithi016 ceph-mon[30947]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]': finished 2024-09-12T01:54:59.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:59 smithi016 ceph-mon[30947]: osdmap e12: 2 total, 1 up, 2 in 2024-09-12T01:54:59.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:59 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T01:54:59.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:59 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T01:54:59.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:59 smithi016 ceph-mon[30947]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi184:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:54:59.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:59 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T01:54:59.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:59 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T01:54:59.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:59 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:54:59.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:54:59 smithi016 ceph-mon[30947]: from='osd.1 ' entity='osd.1' 2024-09-12T01:55:00.553 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:00 smithi184 ceph-mon[35593]: purged_snaps scrub starts 2024-09-12T01:55:00.553 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:00 smithi184 ceph-mon[35593]: purged_snaps scrub ok 2024-09-12T01:55:00.553 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:00 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T01:55:00.554 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:00 smithi184 ceph-mon[35593]: osd.1 [v2:172.21.15.33:6800/3862661148,v1:172.21.15.33:6801/3862661148] boot 2024-09-12T01:55:00.554 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:00 smithi184 ceph-mon[35593]: osdmap e13: 2 total, 2 up, 2 in 2024-09-12T01:55:00.554 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:00 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T01:55:00.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:00 smithi033 ceph-mon[35332]: purged_snaps scrub starts 2024-09-12T01:55:00.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:00 smithi033 ceph-mon[35332]: purged_snaps scrub ok 2024-09-12T01:55:00.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:00 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T01:55:00.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:00 smithi033 ceph-mon[35332]: osd.1 [v2:172.21.15.33:6800/3862661148,v1:172.21.15.33:6801/3862661148] boot 2024-09-12T01:55:00.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:00 smithi033 ceph-mon[35332]: osdmap e13: 2 total, 2 up, 2 in 2024-09-12T01:55:00.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:00 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T01:55:00.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:00 smithi016 ceph-mon[30947]: purged_snaps scrub starts 2024-09-12T01:55:00.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:00 smithi016 ceph-mon[30947]: purged_snaps scrub ok 2024-09-12T01:55:00.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:00 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T01:55:00.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:00 smithi016 ceph-mon[30947]: osd.1 [v2:172.21.15.33:6800/3862661148,v1:172.21.15.33:6801/3862661148] boot 2024-09-12T01:55:00.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:00 smithi016 ceph-mon[30947]: osdmap e13: 2 total, 2 up, 2 in 2024-09-12T01:55:00.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:00 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T01:55:01.466 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:01 smithi184 ceph-mon[35593]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T01:55:01.467 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:01 smithi184 ceph-mon[35593]: from='client.? 172.21.15.184:0/2343918374' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "22e40cac-3bc0-4cce-b47b-77499b767f7e"}]: dispatch 2024-09-12T01:55:01.467 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:01 smithi184 ceph-mon[35593]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "22e40cac-3bc0-4cce-b47b-77499b767f7e"}]: dispatch 2024-09-12T01:55:01.467 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:01 smithi184 ceph-mon[35593]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "22e40cac-3bc0-4cce-b47b-77499b767f7e"}]': finished 2024-09-12T01:55:01.467 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:01 smithi184 ceph-mon[35593]: osdmap e14: 3 total, 2 up, 3 in 2024-09-12T01:55:01.467 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:01 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T01:55:01.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:01 smithi033 ceph-mon[35332]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T01:55:01.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:01 smithi033 ceph-mon[35332]: from='client.? 172.21.15.184:0/2343918374' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "22e40cac-3bc0-4cce-b47b-77499b767f7e"}]: dispatch 2024-09-12T01:55:01.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:01 smithi033 ceph-mon[35332]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "22e40cac-3bc0-4cce-b47b-77499b767f7e"}]: dispatch 2024-09-12T01:55:01.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:01 smithi033 ceph-mon[35332]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "22e40cac-3bc0-4cce-b47b-77499b767f7e"}]': finished 2024-09-12T01:55:01.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:01 smithi033 ceph-mon[35332]: osdmap e14: 3 total, 2 up, 3 in 2024-09-12T01:55:01.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:01 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T01:55:01.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:01 smithi016 ceph-mon[30947]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T01:55:01.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:01 smithi016 ceph-mon[30947]: from='client.? 172.21.15.184:0/2343918374' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "22e40cac-3bc0-4cce-b47b-77499b767f7e"}]: dispatch 2024-09-12T01:55:01.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:01 smithi016 ceph-mon[30947]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "22e40cac-3bc0-4cce-b47b-77499b767f7e"}]: dispatch 2024-09-12T01:55:01.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:01 smithi016 ceph-mon[30947]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "22e40cac-3bc0-4cce-b47b-77499b767f7e"}]': finished 2024-09-12T01:55:01.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:01 smithi016 ceph-mon[30947]: osdmap e14: 3 total, 2 up, 3 in 2024-09-12T01:55:01.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:01 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T01:55:02.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:02 smithi033 ceph-mon[35332]: from='client.? 172.21.15.184:0/3670467359' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T01:55:02.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:02 smithi016 ceph-mon[30947]: from='client.? 172.21.15.184:0/3670467359' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T01:55:02.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:02 smithi184 ceph-mon[35593]: from='client.? 172.21.15.184:0/3670467359' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T01:55:03.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:03 smithi033 ceph-mon[35332]: pgmap v61: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-12T01:55:03.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:03 smithi016 ceph-mon[30947]: pgmap v61: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-12T01:55:03.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:03 smithi184 ceph-mon[35593]: pgmap v61: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-12T01:55:05.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:05 smithi033 ceph-mon[35332]: pgmap v62: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-12T01:55:05.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:05 smithi016 ceph-mon[30947]: pgmap v62: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-12T01:55:05.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:05 smithi184 ceph-mon[35593]: pgmap v62: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-12T01:55:07.590 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:07 smithi184 ceph-mon[35593]: pgmap v63: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-12T01:55:07.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:07 smithi033 ceph-mon[35332]: pgmap v63: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-12T01:55:07.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:07 smithi016 ceph-mon[30947]: pgmap v63: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-12T01:55:09.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:09 smithi184 ceph-mon[35593]: pgmap v64: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-12T01:55:09.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:09 smithi033 ceph-mon[35332]: pgmap v64: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-12T01:55:09.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:09 smithi016 ceph-mon[30947]: pgmap v64: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-12T01:55:10.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:10 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-12T01:55:10.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:10 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:55:10.670 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:10 smithi033 ceph-mon[35332]: Deploying daemon osd.2 on smithi184 2024-09-12T01:55:10.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:10 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-12T01:55:10.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:10 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:55:10.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:10 smithi016 ceph-mon[30947]: Deploying daemon osd.2 on smithi184 2024-09-12T01:55:10.724 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:10 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-12T01:55:10.724 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:10 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:55:10.724 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:10 smithi184 ceph-mon[35593]: Deploying daemon osd.2 on smithi184 2024-09-12T01:55:11.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:11 smithi184 ceph-mon[35593]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-12T01:55:11.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:11 smithi033 ceph-mon[35332]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-12T01:55:11.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:11 smithi016 ceph-mon[30947]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-12T01:55:13.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:13 smithi033 ceph-mon[35332]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-12T01:55:13.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:13 smithi016 ceph-mon[30947]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-12T01:55:13.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:13 smithi184 ceph-mon[35593]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-12T01:55:15.543 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:15 smithi184 ceph-mon[35593]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-12T01:55:15.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:15 smithi033 ceph-mon[35332]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-12T01:55:15.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:15 smithi016 ceph-mon[30947]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-12T01:55:16.519 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:16 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:55:16.519 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:16 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:55:16.519 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:16 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:55:16.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:16 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:55:16.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:16 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:55:16.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:16 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:55:16.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:16 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:55:16.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:16 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:55:16.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:16 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:55:18.064 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:17 smithi184 ceph-mon[35593]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-12T01:55:18.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:17 smithi033 ceph-mon[35332]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-12T01:55:18.183 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:17 smithi016 ceph-mon[30947]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-12T01:55:19.572 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:19 smithi184 ceph-mon[35593]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-12T01:55:19.573 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:19 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:55:19.573 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:19 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:55:19.573 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:19 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:55:19.573 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:19 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:55:19.573 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:19 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:55:19.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:19 smithi033 ceph-mon[35332]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-12T01:55:19.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:19 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:55:19.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:19 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:55:19.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:19 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:55:19.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:19 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:55:19.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:19 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:55:19.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:19 smithi016 ceph-mon[30947]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-12T01:55:19.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:19 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:55:19.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:19 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:55:19.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:19 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:55:19.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:19 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:55:19.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:19 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:55:19.897 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:55:19.897+0000 7fbf267fc640 1 -- 172.21.15.184:0/326389990 <== mgr.14150 v2:172.21.15.16:6800/3887444820 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fbf28062630 con 0x7fbf100a2b20 2024-09-12T01:55:19.898 INFO:teuthology.orchestra.run.smithi184.stdout:Created osd(s) 2 on host 'smithi184' 2024-09-12T01:55:19.900 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:55:19.899+0000 7fbf2f584640 1 -- 172.21.15.184:0/326389990 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fbf100a2b20 msgr2=0x7fbf100a4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:55:19.900 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:55:19.899+0000 7fbf2f584640 1 --2- 172.21.15.184:0/326389990 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fbf100a2b20 0x7fbf100a4fe0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c006010 tx=0x7fbf1c000bc0 comp rx=0 tx=0).stop 2024-09-12T01:55:19.900 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:55:19.899+0000 7fbf2f584640 1 -- 172.21.15.184:0/326389990 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbf28100ad0 msgr2=0x7fbf28197e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:55:19.900 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:55:19.899+0000 7fbf2f584640 1 --2- 172.21.15.184:0/326389990 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbf28100ad0 0x7fbf28197e50 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fbf1800ea80 tx=0x7fbf1800ef40 comp rx=0 tx=0).stop 2024-09-12T01:55:19.900 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:55:19.899+0000 7fbf2f584640 1 -- 172.21.15.184:0/326389990 shutdown_connections 2024-09-12T01:55:19.900 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:55:19.899+0000 7fbf2f584640 1 --2- 172.21.15.184:0/326389990 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fbf100a2b20 0x7fbf100a4fe0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:55:19.900 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:55:19.899+0000 7fbf2f584640 1 --2- 172.21.15.184:0/326389990 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf28102690 0x7fbf2819f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:55:19.900 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:55:19.899+0000 7fbf2f584640 1 --2- 172.21.15.184:0/326389990 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf28101cd0 0x7fbf28198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:55:19.901 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:55:19.899+0000 7fbf2f584640 1 --2- 172.21.15.184:0/326389990 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbf28100ad0 0x7fbf28197e50 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:55:19.901 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:55:19.899+0000 7fbf2f584640 1 -- 172.21.15.184:0/326389990 >> 172.21.15.184:0/326389990 conn(0x7fbf280fc280 msgr2=0x7fbf28105020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:55:19.901 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:55:19.900+0000 7fbf2f584640 1 -- 172.21.15.184:0/326389990 shutdown_connections 2024-09-12T01:55:19.901 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:55:19.900+0000 7fbf2f584640 1 -- 172.21.15.184:0/326389990 wait complete. 2024-09-12T01:55:20.494 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:20 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:55:20.494 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:20 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:55:20.494 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:20 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:55:20.496 DEBUG:teuthology.orchestra.run.smithi184:osd.2> sudo journalctl -f -n 0 -u ceph-879f3b56-70a9-11ef-bcea-c7b262605968@osd.2.service 2024-09-12T01:55:20.498 INFO:tasks.cephadm:Deploying osd.3 on smithi016 with /dev/vg_nvme/lv_3... 2024-09-12T01:55:20.498 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-12T01:55:20.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:20 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:55:20.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:20 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:55:20.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:20 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:55:20.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:20 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:55:20.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:20 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:55:20.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:20 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:55:20.749 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:55:21.953 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:21 smithi184 ceph-mon[35593]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-12T01:55:22.132 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:21 smithi016 ceph-mon[30947]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-12T01:55:22.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:21 smithi033 ceph-mon[35332]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-12T01:55:22.213 INFO:journalctl@ceph.osd.2.smithi184.stdout:Sep 12 01:55:22 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-2[40596]: 2024-09-12T01:55:22.126+0000 7fbf88907640 -1 osd.2 0 log_to_monitors true 2024-09-12T01:55:23.057 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:22 smithi184 ceph-mon[35593]: from='osd.2 [v2:172.21.15.184:6800/3958753963,v1:172.21.15.184:6801/3958753963]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-12T01:55:23.057 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:22 smithi184 ceph-mon[35593]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-12T01:55:23.060 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:22 smithi016 ceph-mon[30947]: from='osd.2 [v2:172.21.15.184:6800/3958753963,v1:172.21.15.184:6801/3958753963]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-12T01:55:23.060 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:22 smithi016 ceph-mon[30947]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-12T01:55:23.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:22 smithi033 ceph-mon[35332]: from='osd.2 [v2:172.21.15.184:6800/3958753963,v1:172.21.15.184:6801/3958753963]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-12T01:55:23.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:22 smithi033 ceph-mon[35332]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-12T01:55:23.430 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:55:23.462 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi016:vg_nvme/lv_3 2024-09-12T01:55:23.715 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:55:24.138 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:23 smithi016 ceph-mon[30947]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-12T01:55:24.138 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:23 smithi016 ceph-mon[30947]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-12T01:55:24.138 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:23 smithi016 ceph-mon[30947]: osdmap e15: 3 total, 2 up, 3 in 2024-09-12T01:55:24.138 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:23 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T01:55:24.138 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:23 smithi016 ceph-mon[30947]: from='osd.2 [v2:172.21.15.184:6800/3958753963,v1:172.21.15.184:6801/3958753963]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2024-09-12T01:55:24.139 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:23 smithi016 ceph-mon[30947]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2024-09-12T01:55:24.139 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:23 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:55:24.139 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:23 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:55:24.139 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:23 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:55:24.139 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:23 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:55:24.139 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:23 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:55:24.139 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:23 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:55:24.139 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:23 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:55:24.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:55:24.149+0000 7fce489bf640 1 -- 172.21.15.16:0/3697212954 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce44101cf0 msgr2=0x7fce44102170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:55:24.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:55:24.149+0000 7fce489bf640 1 --2- 172.21.15.16:0/3697212954 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce44101cf0 0x7fce44102170 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fce38009900 tx=0x7fce3802a410 comp rx=0 tx=0).stop 2024-09-12T01:55:24.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:55:24.150+0000 7fce489bf640 1 -- 172.21.15.16:0/3697212954 shutdown_connections 2024-09-12T01:55:24.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:55:24.150+0000 7fce489bf640 1 --2- 172.21.15.16:0/3697212954 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce441026b0 0x7fce44108f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:55:24.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:55:24.150+0000 7fce489bf640 1 --2- 172.21.15.16:0/3697212954 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce44101cf0 0x7fce44102170 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:55:24.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:55:24.150+0000 7fce489bf640 1 --2- 172.21.15.16:0/3697212954 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fce44100af0 0x7fce44100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:55:24.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:55:24.150+0000 7fce489bf640 1 -- 172.21.15.16:0/3697212954 >> 172.21.15.16:0/3697212954 conn(0x7fce440fc280 msgr2=0x7fce440fe6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:55:24.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:55:24.150+0000 7fce489bf640 1 -- 172.21.15.16:0/3697212954 shutdown_connections 2024-09-12T01:55:24.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:55:24.150+0000 7fce489bf640 1 -- 172.21.15.16:0/3697212954 wait complete. 2024-09-12T01:55:24.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:55:24.150+0000 7fce489bf640 1 Processor -- start 2024-09-12T01:55:24.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:55:24.150+0000 7fce489bf640 1 -- start start 2024-09-12T01:55:24.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:55:24.150+0000 7fce489bf640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce44100af0 0x7fce44197f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:55:24.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:55:24.151+0000 7fce489bf640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce44101cf0 0x7fce44198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:55:24.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:55:24.151+0000 7fce489bf640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fce441026b0 0x7fce4419f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:55:24.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:55:24.151+0000 7fce489bf640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fce4410c750 con 0x7fce44101cf0 2024-09-12T01:55:24.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:55:24.151+0000 7fce489bf640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fce4410c5d0 con 0x7fce44100af0 2024-09-12T01:55:24.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:55:24.151+0000 7fce489bf640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fce4410c8d0 con 0x7fce441026b0 2024-09-12T01:55:24.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:55:24.151+0000 7fce41d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce44101cf0 0x7fce44198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:55:24.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:55:24.151+0000 7fce41d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce44101cf0 0x7fce44198470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58156/0 (socket says 172.21.15.16:58156) 2024-09-12T01:55:24.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:55:24.151+0000 7fce41d9b640 1 -- 172.21.15.16:0/1760098901 learned_addr learned my addr 172.21.15.16:0/1760098901 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:55:24.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:55:24.151+0000 7fce42d9d640 1 --2- 172.21.15.16:0/1760098901 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fce441026b0 0x7fce4419f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:55:24.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:55:24.151+0000 7fce4259c640 1 --2- 172.21.15.16:0/1760098901 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce44100af0 0x7fce44197f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:55:24.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:55:24.151+0000 7fce41d9b640 1 -- 172.21.15.16:0/1760098901 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fce441026b0 msgr2=0x7fce4419f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:55:24.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:55:24.151+0000 7fce41d9b640 1 --2- 172.21.15.16:0/1760098901 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fce441026b0 0x7fce4419f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:55:24.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:55:24.151+0000 7fce41d9b640 1 -- 172.21.15.16:0/1760098901 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce44100af0 msgr2=0x7fce44197f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:55:24.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:55:24.151+0000 7fce41d9b640 1 --2- 172.21.15.16:0/1760098901 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce44100af0 0x7fce44197f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:55:24.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:55:24.151+0000 7fce41d9b640 1 -- 172.21.15.16:0/1760098901 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce4419fb60 con 0x7fce44101cf0 2024-09-12T01:55:24.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:55:24.151+0000 7fce41d9b640 1 --2- 172.21.15.16:0/1760098901 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce44101cf0 0x7fce44198470 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fce38009230 tx=0x7fce38004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:55:24.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:55:24.151+0000 7fce337fe640 1 -- 172.21.15.16:0/1760098901 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce3803f030 con 0x7fce44101cf0 2024-09-12T01:55:24.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:55:24.151+0000 7fce337fe640 1 -- 172.21.15.16:0/1760098901 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce38003ce0 con 0x7fce44101cf0 2024-09-12T01:55:24.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:55:24.151+0000 7fce489bf640 1 -- 172.21.15.16:0/1760098901 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce4419fdf0 con 0x7fce44101cf0 2024-09-12T01:55:24.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:55:24.151+0000 7fce337fe640 1 -- 172.21.15.16:0/1760098901 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce38035070 con 0x7fce44101cf0 2024-09-12T01:55:24.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:55:24.152+0000 7fce489bf640 1 -- 172.21.15.16:0/1760098901 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce441a0270 con 0x7fce44101cf0 2024-09-12T01:55:24.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:55:24.152+0000 7fce489bf640 1 -- 172.21.15.16:0/1760098901 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce441a04a0 con 0x7fce44101cf0 2024-09-12T01:55:24.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:55:24.154+0000 7fce337fe640 1 -- 172.21.15.16:0/1760098901 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7fce38042030 con 0x7fce44101cf0 2024-09-12T01:55:24.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:55:24.155+0000 7fce337fe640 1 --2- 172.21.15.16:0/1760098901 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fce2c0a2b20 0x7fce2c0a4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:55:24.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:55:24.155+0000 7fce4259c640 1 --2- 172.21.15.16:0/1760098901 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fce2c0a2b20 0x7fce2c0a4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:55:24.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:55:24.155+0000 7fce337fe640 1 -- 172.21.15.16:0/1760098901 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 2571+0+0 (secure 0 0 0) 0x7fce380ede20 con 0x7fce44101cf0 2024-09-12T01:55:24.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:55:24.155+0000 7fce4259c640 1 --2- 172.21.15.16:0/1760098901 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fce2c0a2b20 0x7fce2c0a4fe0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fce44101b50 tx=0x7fce3400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:55:24.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:55:24.159+0000 7fce337fe640 1 -- 172.21.15.16:0/1760098901 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fce380bbe80 con 0x7fce44101cf0 2024-09-12T01:55:24.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:23 smithi033 ceph-mon[35332]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-12T01:55:24.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:23 smithi033 ceph-mon[35332]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-12T01:55:24.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:23 smithi033 ceph-mon[35332]: osdmap e15: 3 total, 2 up, 3 in 2024-09-12T01:55:24.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:23 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T01:55:24.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:23 smithi033 ceph-mon[35332]: from='osd.2 [v2:172.21.15.184:6800/3958753963,v1:172.21.15.184:6801/3958753963]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2024-09-12T01:55:24.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:23 smithi033 ceph-mon[35332]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2024-09-12T01:55:24.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:23 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:55:24.170 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:23 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:55:24.170 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:23 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:55:24.170 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:23 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:55:24.170 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:23 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:55:24.170 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:23 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:55:24.170 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:23 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:55:24.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:55:24.343+0000 7fce489bf640 1 -- 172.21.15.16:0/1760098901 --> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fce4406af10 con 0x7fce2c0a2b20 2024-09-12T01:55:24.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:23 smithi184 ceph-mon[35593]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-12T01:55:24.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:23 smithi184 ceph-mon[35593]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-12T01:55:24.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:23 smithi184 ceph-mon[35593]: osdmap e15: 3 total, 2 up, 3 in 2024-09-12T01:55:24.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:23 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T01:55:24.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:23 smithi184 ceph-mon[35593]: from='osd.2 [v2:172.21.15.184:6800/3958753963,v1:172.21.15.184:6801/3958753963]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2024-09-12T01:55:24.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:23 smithi184 ceph-mon[35593]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2024-09-12T01:55:24.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:23 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:55:24.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:23 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:55:24.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:23 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:55:24.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:23 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:55:24.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:23 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:55:24.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:23 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:55:24.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:23 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:55:25.069 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:24 smithi016 ceph-mon[30947]: Detected new or changed devices on smithi184 2024-09-12T01:55:25.069 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:24 smithi016 ceph-mon[30947]: Adjusting osd_memory_target on smithi184 to 17088M 2024-09-12T01:55:25.069 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:24 smithi016 ceph-mon[30947]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]': finished 2024-09-12T01:55:25.069 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:24 smithi016 ceph-mon[30947]: osdmap e16: 3 total, 2 up, 3 in 2024-09-12T01:55:25.070 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:24 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T01:55:25.070 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:24 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T01:55:25.070 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:24 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T01:55:25.070 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:24 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T01:55:25.070 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:24 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:55:25.070 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:24 smithi016 ceph-mon[30947]: from='osd.2 ' entity='osd.2' 2024-09-12T01:55:25.070 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:24 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T01:55:25.127 INFO:journalctl@ceph.osd.2.smithi184.stdout:Sep 12 01:55:24 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-2[40596]: 2024-09-12T01:55:24.711+0000 7fbf84067640 -1 osd.2 0 waiting for initial osdmap 2024-09-12T01:55:25.128 INFO:journalctl@ceph.osd.2.smithi184.stdout:Sep 12 01:55:24 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-2[40596]: 2024-09-12T01:55:24.722+0000 7fbf7c699640 -1 osd.2 16 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T01:55:25.128 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:24 smithi184 ceph-mon[35593]: Detected new or changed devices on smithi184 2024-09-12T01:55:25.128 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:24 smithi184 ceph-mon[35593]: Adjusting osd_memory_target on smithi184 to 17088M 2024-09-12T01:55:25.128 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:24 smithi184 ceph-mon[35593]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]': finished 2024-09-12T01:55:25.128 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:24 smithi184 ceph-mon[35593]: osdmap e16: 3 total, 2 up, 3 in 2024-09-12T01:55:25.128 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:24 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T01:55:25.128 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:24 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T01:55:25.128 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:24 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T01:55:25.128 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:24 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T01:55:25.128 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:24 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:55:25.129 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:24 smithi184 ceph-mon[35593]: from='osd.2 ' entity='osd.2' 2024-09-12T01:55:25.129 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:24 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T01:55:25.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:24 smithi033 ceph-mon[35332]: Detected new or changed devices on smithi184 2024-09-12T01:55:25.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:24 smithi033 ceph-mon[35332]: Adjusting osd_memory_target on smithi184 to 17088M 2024-09-12T01:55:25.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:24 smithi033 ceph-mon[35332]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]': finished 2024-09-12T01:55:25.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:24 smithi033 ceph-mon[35332]: osdmap e16: 3 total, 2 up, 3 in 2024-09-12T01:55:25.170 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:24 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T01:55:25.170 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:24 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T01:55:25.170 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:24 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T01:55:25.170 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:24 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T01:55:25.170 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:24 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:55:25.170 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:24 smithi033 ceph-mon[35332]: from='osd.2 ' entity='osd.2' 2024-09-12T01:55:25.170 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:24 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T01:55:26.052 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:25 smithi016 ceph-mon[30947]: purged_snaps scrub starts 2024-09-12T01:55:26.052 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:25 smithi016 ceph-mon[30947]: purged_snaps scrub ok 2024-09-12T01:55:26.052 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:25 smithi016 ceph-mon[30947]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-12T01:55:26.052 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:25 smithi016 ceph-mon[30947]: from='client.14295 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:55:26.052 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:25 smithi016 ceph-mon[30947]: osd.2 [v2:172.21.15.184:6800/3958753963,v1:172.21.15.184:6801/3958753963] boot 2024-09-12T01:55:26.053 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:25 smithi016 ceph-mon[30947]: osdmap e17: 3 total, 3 up, 3 in 2024-09-12T01:55:26.053 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:25 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T01:55:26.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:25 smithi033 ceph-mon[35332]: purged_snaps scrub starts 2024-09-12T01:55:26.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:25 smithi033 ceph-mon[35332]: purged_snaps scrub ok 2024-09-12T01:55:26.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:25 smithi033 ceph-mon[35332]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-12T01:55:26.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:25 smithi033 ceph-mon[35332]: from='client.14295 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:55:26.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:25 smithi033 ceph-mon[35332]: osd.2 [v2:172.21.15.184:6800/3958753963,v1:172.21.15.184:6801/3958753963] boot 2024-09-12T01:55:26.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:25 smithi033 ceph-mon[35332]: osdmap e17: 3 total, 3 up, 3 in 2024-09-12T01:55:26.170 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:25 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T01:55:26.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:25 smithi184 ceph-mon[35593]: purged_snaps scrub starts 2024-09-12T01:55:26.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:25 smithi184 ceph-mon[35593]: purged_snaps scrub ok 2024-09-12T01:55:26.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:25 smithi184 ceph-mon[35593]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-12T01:55:26.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:25 smithi184 ceph-mon[35593]: from='client.14295 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:55:26.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:25 smithi184 ceph-mon[35593]: osd.2 [v2:172.21.15.184:6800/3958753963,v1:172.21.15.184:6801/3958753963] boot 2024-09-12T01:55:26.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:25 smithi184 ceph-mon[35593]: osdmap e17: 3 total, 3 up, 3 in 2024-09-12T01:55:26.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:25 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T01:55:27.183 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:26 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-12T01:55:27.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:26 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-12T01:55:27.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:26 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-12T01:55:28.183 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:27 smithi016 ceph-mon[30947]: pgmap v76: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-12T01:55:28.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:27 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-12T01:55:28.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:27 smithi016 ceph-mon[30947]: osdmap e18: 3 total, 3 up, 3 in 2024-09-12T01:55:28.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:27 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/3688050755' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "834a07f5-2c24-49e1-8e4a-12c96cfbdc65"}]: dispatch 2024-09-12T01:55:28.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:27 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-12T01:55:28.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:27 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/3688050755' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "834a07f5-2c24-49e1-8e4a-12c96cfbdc65"}]': finished 2024-09-12T01:55:28.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:27 smithi016 ceph-mon[30947]: osdmap e19: 4 total, 3 up, 4 in 2024-09-12T01:55:28.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:27 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T01:55:28.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:27 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/3603765400' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T01:55:28.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:27 smithi184 ceph-mon[35593]: pgmap v76: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-12T01:55:28.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:27 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-12T01:55:28.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:27 smithi184 ceph-mon[35593]: osdmap e18: 3 total, 3 up, 3 in 2024-09-12T01:55:28.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:27 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/3688050755' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "834a07f5-2c24-49e1-8e4a-12c96cfbdc65"}]: dispatch 2024-09-12T01:55:28.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:27 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-12T01:55:28.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:27 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/3688050755' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "834a07f5-2c24-49e1-8e4a-12c96cfbdc65"}]': finished 2024-09-12T01:55:28.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:27 smithi184 ceph-mon[35593]: osdmap e19: 4 total, 3 up, 4 in 2024-09-12T01:55:28.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:27 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T01:55:28.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:27 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/3603765400' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T01:55:28.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:27 smithi033 ceph-mon[35332]: pgmap v76: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-12T01:55:28.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:27 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-12T01:55:28.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:27 smithi033 ceph-mon[35332]: osdmap e18: 3 total, 3 up, 3 in 2024-09-12T01:55:28.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:27 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/3688050755' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "834a07f5-2c24-49e1-8e4a-12c96cfbdc65"}]: dispatch 2024-09-12T01:55:28.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:27 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-12T01:55:28.420 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:27 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/3688050755' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "834a07f5-2c24-49e1-8e4a-12c96cfbdc65"}]': finished 2024-09-12T01:55:28.420 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:27 smithi033 ceph-mon[35332]: osdmap e19: 4 total, 3 up, 4 in 2024-09-12T01:55:28.420 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:27 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T01:55:28.420 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:27 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/3603765400' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T01:55:28.920 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 12 01:55:28 smithi016 sudo[49482]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-12T01:55:28.920 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 12 01:55:28 smithi016 sudo[49482]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T01:55:28.920 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 12 01:55:28 smithi016 sudo[49482]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T01:55:28.920 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 12 01:55:28 smithi016 sudo[49482]: pam_unix(sudo:session): session closed for user root 2024-09-12T01:55:28.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:55:28.941+0000 7fce337fe640 1 -- 172.21.15.16:0/1760098901 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fce380b84f0 con 0x7fce44101cf0 2024-09-12T01:55:29.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:28 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-12T01:55:29.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:28 smithi016 ceph-mon[30947]: osdmap e20: 4 total, 3 up, 4 in 2024-09-12T01:55:29.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:28 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T01:55:29.184 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 12 01:55:28 smithi016 sudo[49854]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-12T01:55:29.184 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 12 01:55:28 smithi016 sudo[49854]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T01:55:29.184 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 12 01:55:28 smithi016 sudo[49854]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T01:55:29.184 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 12 01:55:28 smithi016 sudo[49854]: pam_unix(sudo:session): session closed for user root 2024-09-12T01:55:29.209 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:28 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-12T01:55:29.209 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:28 smithi033 ceph-mon[35332]: osdmap e20: 4 total, 3 up, 4 in 2024-09-12T01:55:29.209 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:28 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T01:55:29.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:28 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-12T01:55:29.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:28 smithi184 ceph-mon[35593]: osdmap e20: 4 total, 3 up, 4 in 2024-09-12T01:55:29.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:28 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T01:55:29.478 INFO:journalctl@ceph.osd.1.smithi033.stdout:Sep 12 01:55:29 smithi033 sudo[43557]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-12T01:55:29.478 INFO:journalctl@ceph.osd.1.smithi033.stdout:Sep 12 01:55:29 smithi033 sudo[43557]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T01:55:29.478 INFO:journalctl@ceph.osd.1.smithi033.stdout:Sep 12 01:55:29 smithi033 sudo[43557]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T01:55:29.479 INFO:journalctl@ceph.osd.1.smithi033.stdout:Sep 12 01:55:29 smithi033 sudo[43557]: pam_unix(sudo:session): session closed for user root 2024-09-12T01:55:29.919 INFO:journalctl@ceph.osd.1.smithi033.stdout:Sep 12 01:55:29 smithi033 sudo[43560]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-12T01:55:29.919 INFO:journalctl@ceph.osd.1.smithi033.stdout:Sep 12 01:55:29 smithi033 sudo[43560]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T01:55:29.919 INFO:journalctl@ceph.osd.1.smithi033.stdout:Sep 12 01:55:29 smithi033 sudo[43560]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T01:55:29.919 INFO:journalctl@ceph.osd.1.smithi033.stdout:Sep 12 01:55:29 smithi033 sudo[43560]: pam_unix(sudo:session): session closed for user root 2024-09-12T01:55:30.006 INFO:journalctl@ceph.osd.2.smithi184.stdout:Sep 12 01:55:29 smithi184 sudo[44423]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-12T01:55:30.007 INFO:journalctl@ceph.osd.2.smithi184.stdout:Sep 12 01:55:29 smithi184 sudo[44423]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T01:55:30.007 INFO:journalctl@ceph.osd.2.smithi184.stdout:Sep 12 01:55:29 smithi184 sudo[44423]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T01:55:30.007 INFO:journalctl@ceph.osd.2.smithi184.stdout:Sep 12 01:55:29 smithi184 sudo[44423]: pam_unix(sudo:session): session closed for user root 2024-09-12T01:55:30.007 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:29 smithi184 ceph-mon[35593]: pgmap v80: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-12T01:55:30.007 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:29 smithi184 ceph-mon[35593]: osdmap e21: 4 total, 3 up, 4 in 2024-09-12T01:55:30.007 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:29 smithi184 ceph-mon[35593]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-12T01:55:30.007 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:29 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T01:55:30.007 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:29 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:55:30.007 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:29 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:55:30.381 INFO:journalctl@ceph.osd.2.smithi184.stdout:Sep 12 01:55:30 smithi184 sudo[44426]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-12T01:55:30.381 INFO:journalctl@ceph.osd.2.smithi184.stdout:Sep 12 01:55:30 smithi184 sudo[44426]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T01:55:30.381 INFO:journalctl@ceph.osd.2.smithi184.stdout:Sep 12 01:55:30 smithi184 sudo[44426]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T01:55:30.381 INFO:journalctl@ceph.osd.2.smithi184.stdout:Sep 12 01:55:30 smithi184 sudo[44426]: pam_unix(sudo:session): session closed for user root 2024-09-12T01:55:30.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:29 smithi033 ceph-mon[35332]: pgmap v80: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-12T01:55:30.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:29 smithi033 ceph-mon[35332]: osdmap e21: 4 total, 3 up, 4 in 2024-09-12T01:55:30.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:29 smithi033 ceph-mon[35332]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-12T01:55:30.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:29 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T01:55:30.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:29 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:55:30.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:29 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:55:30.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:29 smithi016 ceph-mon[30947]: pgmap v80: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-12T01:55:30.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:29 smithi016 ceph-mon[30947]: osdmap e21: 4 total, 3 up, 4 in 2024-09-12T01:55:30.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:29 smithi016 ceph-mon[30947]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-12T01:55:30.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:29 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T01:55:30.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:29 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:55:30.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:29 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:55:30.933 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:30 smithi016 sudo[50237]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-12T01:55:30.933 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:30 smithi016 sudo[50237]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T01:55:30.934 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:30 smithi016 sudo[50237]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T01:55:30.934 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:30 smithi016 sudo[50237]: pam_unix(sudo:session): session closed for user root 2024-09-12T01:55:31.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:30 smithi016 sudo[50243]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-12T01:55:31.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:30 smithi016 sudo[50243]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T01:55:31.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:30 smithi016 sudo[50243]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T01:55:31.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:30 smithi016 sudo[50243]: pam_unix(sudo:session): session closed for user root 2024-09-12T01:55:31.435 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:31 smithi184 sudo[44429]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-12T01:55:31.435 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:31 smithi184 sudo[44429]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T01:55:31.435 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:31 smithi184 sudo[44429]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T01:55:31.436 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:31 smithi184 sudo[44429]: pam_unix(sudo:session): session closed for user root 2024-09-12T01:55:31.741 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:31 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:55:31.741 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:31 smithi016 ceph-mon[30947]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-12T01:55:31.741 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:31 smithi016 ceph-mon[30947]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-12T01:55:31.741 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:31 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T01:55:31.741 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:31 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T01:55:31.742 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:31 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T01:55:31.742 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:31 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T01:55:31.742 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:31 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T01:55:31.742 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:31 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T01:55:31.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:31 smithi184 sudo[44432]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-12T01:55:31.878 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:31 smithi184 sudo[44432]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T01:55:31.878 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:31 smithi184 sudo[44432]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T01:55:31.878 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:31 smithi184 sudo[44432]: pam_unix(sudo:session): session closed for user root 2024-09-12T01:55:31.878 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:31 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:55:31.878 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:31 smithi184 ceph-mon[35593]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-12T01:55:31.878 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:31 smithi184 ceph-mon[35593]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-12T01:55:31.878 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:31 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T01:55:31.878 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:31 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T01:55:31.878 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:31 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T01:55:31.878 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:31 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T01:55:31.878 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:31 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T01:55:31.878 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:31 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T01:55:31.962 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:31 smithi033 sudo[43563]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-12T01:55:31.962 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:31 smithi033 sudo[43563]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T01:55:31.962 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:31 smithi033 sudo[43563]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T01:55:31.962 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:31 smithi033 sudo[43563]: pam_unix(sudo:session): session closed for user root 2024-09-12T01:55:32.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:31 smithi033 sudo[43566]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-12T01:55:32.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:31 smithi033 sudo[43566]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T01:55:32.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:31 smithi033 sudo[43566]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T01:55:32.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:31 smithi033 sudo[43566]: pam_unix(sudo:session): session closed for user root 2024-09-12T01:55:32.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:31 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:55:32.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:31 smithi033 ceph-mon[35332]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-12T01:55:32.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:31 smithi033 ceph-mon[35332]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-12T01:55:32.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:31 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T01:55:32.420 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:31 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T01:55:32.420 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:31 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T01:55:32.420 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:31 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T01:55:32.420 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:31 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T01:55:32.420 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:31 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T01:55:32.679 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:32 smithi016 ceph-mon[30947]: pgmap v82: 1 pgs: 1 unknown; 0 B data, 89 MiB used, 268 GiB / 268 GiB avail 2024-09-12T01:55:32.679 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:32 smithi016 ceph-mon[30947]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-12T01:55:32.679 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:32 smithi016 ceph-mon[30947]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-12T01:55:32.679 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:32 smithi016 ceph-mon[30947]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-12T01:55:32.679 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:32 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T01:55:32.679 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:32 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T01:55:32.679 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:32 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T01:55:32.679 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:32 smithi016 ceph-mon[30947]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-12T01:55:32.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:32 smithi184 ceph-mon[35593]: pgmap v82: 1 pgs: 1 unknown; 0 B data, 89 MiB used, 268 GiB / 268 GiB avail 2024-09-12T01:55:32.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:32 smithi184 ceph-mon[35593]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-12T01:55:32.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:32 smithi184 ceph-mon[35593]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-12T01:55:32.878 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:32 smithi184 ceph-mon[35593]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-12T01:55:32.878 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:32 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T01:55:32.878 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:32 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T01:55:32.878 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:32 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T01:55:32.878 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:32 smithi184 ceph-mon[35593]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-12T01:55:32.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:32 smithi033 ceph-mon[35332]: pgmap v82: 1 pgs: 1 unknown; 0 B data, 89 MiB used, 268 GiB / 268 GiB avail 2024-09-12T01:55:32.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:32 smithi033 ceph-mon[35332]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-12T01:55:32.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:32 smithi033 ceph-mon[35332]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-12T01:55:32.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:32 smithi033 ceph-mon[35332]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-12T01:55:32.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:32 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T01:55:32.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:32 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T01:55:32.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:32 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T01:55:32.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:32 smithi033 ceph-mon[35332]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-12T01:55:33.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:33 smithi184 ceph-mon[35593]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-09-12T01:55:33.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:33 smithi016 ceph-mon[30947]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-09-12T01:55:33.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:33 smithi033 ceph-mon[35332]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-09-12T01:55:35.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:35 smithi016 ceph-mon[30947]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-09-12T01:55:35.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:35 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-12T01:55:35.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:35 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:55:35.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:35 smithi184 ceph-mon[35593]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-09-12T01:55:35.878 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:35 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-12T01:55:35.878 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:35 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:55:35.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:35 smithi033 ceph-mon[35332]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-09-12T01:55:35.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:35 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-12T01:55:35.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:35 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:55:36.847 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:36 smithi016 ceph-mon[30947]: Deploying daemon osd.3 on smithi016 2024-09-12T01:55:36.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:36 smithi184 ceph-mon[35593]: Deploying daemon osd.3 on smithi016 2024-09-12T01:55:36.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:36 smithi033 ceph-mon[35332]: Deploying daemon osd.3 on smithi016 2024-09-12T01:55:37.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:37 smithi016 ceph-mon[30947]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 65 KiB/s wr, 5 op/s 2024-09-12T01:55:37.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:37 smithi184 ceph-mon[35593]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 65 KiB/s wr, 5 op/s 2024-09-12T01:55:37.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:37 smithi033 ceph-mon[35332]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 65 KiB/s wr, 5 op/s 2024-09-12T01:55:39.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:39 smithi184 ceph-mon[35593]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2024-09-12T01:55:39.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:39 smithi033 ceph-mon[35332]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2024-09-12T01:55:39.934 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:39 smithi016 ceph-mon[30947]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2024-09-12T01:55:41.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:41 smithi016 ceph-mon[30947]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2024-09-12T01:55:41.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:41 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:55:41.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:41 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:55:41.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:41 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:55:41.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:41 smithi184 ceph-mon[35593]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2024-09-12T01:55:41.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:41 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:55:41.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:41 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:55:41.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:41 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:55:41.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:41 smithi033 ceph-mon[35332]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2024-09-12T01:55:41.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:41 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:55:41.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:41 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:55:41.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:41 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:55:43.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:43 smithi016 ceph-mon[30947]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2024-09-12T01:55:43.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:43 smithi184 ceph-mon[35593]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2024-09-12T01:55:43.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:43 smithi033 ceph-mon[35332]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2024-09-12T01:55:45.165 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:44 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:55:45.166 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:44 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:55:45.166 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:44 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:55:45.166 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:44 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:55:45.166 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:44 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:55:45.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:55:45.205+0000 7fce337fe640 1 -- 172.21.15.16:0/1760098901 <== mgr.14150 v2:172.21.15.16:6800/3887444820 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fce4406af10 con 0x7fce2c0a2b20 2024-09-12T01:55:45.205 INFO:teuthology.orchestra.run.smithi016.stdout:Created osd(s) 3 on host 'smithi016' 2024-09-12T01:55:45.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:55:45.208+0000 7fce489bf640 1 -- 172.21.15.16:0/1760098901 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fce2c0a2b20 msgr2=0x7fce2c0a4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:55:45.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:55:45.208+0000 7fce489bf640 1 --2- 172.21.15.16:0/1760098901 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fce2c0a2b20 0x7fce2c0a4fe0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fce44101b50 tx=0x7fce3400b390 comp rx=0 tx=0).stop 2024-09-12T01:55:45.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:55:45.208+0000 7fce489bf640 1 -- 172.21.15.16:0/1760098901 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce44101cf0 msgr2=0x7fce44198470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:55:45.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:55:45.208+0000 7fce489bf640 1 --2- 172.21.15.16:0/1760098901 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce44101cf0 0x7fce44198470 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fce38009230 tx=0x7fce38004360 comp rx=0 tx=0).stop 2024-09-12T01:55:45.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:55:45.208+0000 7fce489bf640 1 -- 172.21.15.16:0/1760098901 shutdown_connections 2024-09-12T01:55:45.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:55:45.208+0000 7fce489bf640 1 --2- 172.21.15.16:0/1760098901 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fce2c0a2b20 0x7fce2c0a4fe0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:55:45.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:55:45.208+0000 7fce489bf640 1 --2- 172.21.15.16:0/1760098901 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fce441026b0 0x7fce4419f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:55:45.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:55:45.208+0000 7fce489bf640 1 --2- 172.21.15.16:0/1760098901 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce44101cf0 0x7fce44198470 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:55:45.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:55:45.208+0000 7fce489bf640 1 --2- 172.21.15.16:0/1760098901 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce44100af0 0x7fce44197f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:55:45.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:55:45.208+0000 7fce489bf640 1 -- 172.21.15.16:0/1760098901 >> 172.21.15.16:0/1760098901 conn(0x7fce440fc280 msgr2=0x7fce4406a800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:55:45.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:55:45.208+0000 7fce489bf640 1 -- 172.21.15.16:0/1760098901 shutdown_connections 2024-09-12T01:55:45.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:55:45.208+0000 7fce489bf640 1 -- 172.21.15.16:0/1760098901 wait complete. 2024-09-12T01:55:45.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:44 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:55:45.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:44 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:55:45.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:44 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:55:45.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:44 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:55:45.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:44 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:55:45.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:44 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:55:45.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:44 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:55:45.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:44 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:55:45.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:44 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:55:45.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:44 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:55:45.848 DEBUG:teuthology.orchestra.run.smithi016:osd.3> sudo journalctl -f -n 0 -u ceph-879f3b56-70a9-11ef-bcea-c7b262605968@osd.3.service 2024-09-12T01:55:45.850 INFO:tasks.cephadm:Deploying osd.4 on smithi033 with /dev/vg_nvme/lv_3... 2024-09-12T01:55:45.850 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-12T01:55:46.078 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:45 smithi016 ceph-mon[30947]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-09-12T01:55:46.078 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:45 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:55:46.078 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:45 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:55:46.078 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:45 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:55:46.110 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T01:55:46.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:45 smithi033 ceph-mon[35332]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-09-12T01:55:46.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:45 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:55:46.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:45 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:55:46.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:45 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:55:46.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:45 smithi184 ceph-mon[35593]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-09-12T01:55:46.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:45 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:55:46.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:45 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:55:46.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:45 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:55:47.934 INFO:journalctl@ceph.osd.3.smithi016.stdout:Sep 12 01:55:47 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-3[51462]: 2024-09-12T01:55:47.667+0000 7fdc508b1640 -1 osd.3 0 log_to_monitors true 2024-09-12T01:55:48.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:47 smithi033 ceph-mon[35332]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-09-12T01:55:48.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:47 smithi033 ceph-mon[35332]: from='osd.3 [v2:172.21.15.16:6810/3395182524,v1:172.21.15.16:6811/3395182524]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-12T01:55:48.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:47 smithi184 ceph-mon[35593]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-09-12T01:55:48.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:47 smithi184 ceph-mon[35593]: from='osd.3 [v2:172.21.15.16:6810/3395182524,v1:172.21.15.16:6811/3395182524]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-12T01:55:48.407 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:47 smithi016 ceph-mon[30947]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-09-12T01:55:48.407 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:47 smithi016 ceph-mon[30947]: from='osd.3 [v2:172.21.15.16:6810/3395182524,v1:172.21.15.16:6811/3395182524]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-12T01:55:48.433 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-12T01:55:48.462 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi033:vg_nvme/lv_3 2024-09-12T01:55:48.727 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T01:55:48.959 INFO:journalctl@ceph.osd.3.smithi016.stdout:Sep 12 01:55:48 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-3[51462]: 2024-09-12T01:55:48.952+0000 7fdc4d048640 -1 osd.3 0 waiting for initial osdmap 2024-09-12T01:55:48.959 INFO:journalctl@ceph.osd.3.smithi016.stdout:Sep 12 01:55:48 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-3[51462]: 2024-09-12T01:55:48.958+0000 7fdc4864b640 -1 osd.3 23 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T01:55:48.959 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:48 smithi016 ceph-mon[30947]: from='osd.3 [v2:172.21.15.16:6810/3395182524,v1:172.21.15.16:6811/3395182524]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-12T01:55:48.959 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:48 smithi016 ceph-mon[30947]: osdmap e22: 4 total, 3 up, 4 in 2024-09-12T01:55:48.959 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:48 smithi016 ceph-mon[30947]: from='osd.3 [v2:172.21.15.16:6810/3395182524,v1:172.21.15.16:6811/3395182524]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-09-12T01:55:48.959 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:48 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T01:55:48.960 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:48 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:55:48.960 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:48 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:55:48.960 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:48 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:55:48.960 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:48 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:55:48.960 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:48 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:55:48.960 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:48 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:55:48.960 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:48 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:55:48.960 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:48 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:55:48.960 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:48 smithi016 ceph-mon[30947]: from='osd.3 [v2:172.21.15.16:6810/3395182524,v1:172.21.15.16:6811/3395182524]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-09-12T01:55:49.076 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:48 smithi033 ceph-mon[35332]: from='osd.3 [v2:172.21.15.16:6810/3395182524,v1:172.21.15.16:6811/3395182524]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-12T01:55:49.076 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:48 smithi033 ceph-mon[35332]: osdmap e22: 4 total, 3 up, 4 in 2024-09-12T01:55:49.077 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:48 smithi033 ceph-mon[35332]: from='osd.3 [v2:172.21.15.16:6810/3395182524,v1:172.21.15.16:6811/3395182524]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-09-12T01:55:49.077 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:48 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T01:55:49.077 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:48 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:55:49.077 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:48 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:55:49.077 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:48 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:55:49.077 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:48 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:55:49.077 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:48 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:55:49.077 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:48 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:55:49.077 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:48 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:55:49.077 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:48 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:55:49.077 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:48 smithi033 ceph-mon[35332]: from='osd.3 [v2:172.21.15.16:6810/3395182524,v1:172.21.15.16:6811/3395182524]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-09-12T01:55:49.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:55:49.161+0000 7f7c1a8fe640 1 -- 172.21.15.33:0/1424669219 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c14107900 msgr2=0x7f7c14109d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:55:49.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:55:49.161+0000 7f7c1a8fe640 1 --2- 172.21.15.33:0/1424669219 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c14107900 0x7f7c14109d50 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f7c0c009ca0 tx=0x7f7c0c02a8c0 comp rx=0 tx=0).stop 2024-09-12T01:55:49.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:55:49.161+0000 7f7c1a8fe640 1 -- 172.21.15.33:0/1424669219 shutdown_connections 2024-09-12T01:55:49.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:55:49.161+0000 7f7c1a8fe640 1 --2- 172.21.15.33:0/1424669219 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c14107900 0x7f7c14109d50 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:55:49.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:55:49.161+0000 7f7c1a8fe640 1 --2- 172.21.15.33:0/1424669219 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7c14104fd0 0x7f7c141073c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:55:49.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:55:49.161+0000 7f7c1a8fe640 1 --2- 172.21.15.33:0/1424669219 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c141026a0 0x7f7c14104a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:55:49.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:55:49.161+0000 7f7c1a8fe640 1 -- 172.21.15.33:0/1424669219 >> 172.21.15.33:0/1424669219 conn(0x7f7c140fc2a0 msgr2=0x7f7c140fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:55:49.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:55:49.162+0000 7f7c1a8fe640 1 -- 172.21.15.33:0/1424669219 shutdown_connections 2024-09-12T01:55:49.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:55:49.162+0000 7f7c1a8fe640 1 -- 172.21.15.33:0/1424669219 wait complete. 2024-09-12T01:55:49.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:55:49.162+0000 7f7c1a8fe640 1 Processor -- start 2024-09-12T01:55:49.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:55:49.162+0000 7f7c1a8fe640 1 -- start start 2024-09-12T01:55:49.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:55:49.162+0000 7f7c1a8fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c141026a0 0x7f7c14102000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:55:49.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:55:49.162+0000 7f7c1a8fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c14104fd0 0x7f7c14100650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:55:49.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:55:49.162+0000 7f7c1a8fe640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7c14107900 0x7f7c14100b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:55:49.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:55:49.163+0000 7f7c1a8fe640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7c1410d8f0 con 0x7f7c141026a0 2024-09-12T01:55:49.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:55:49.163+0000 7f7c1a8fe640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7c1410d770 con 0x7f7c14104fd0 2024-09-12T01:55:49.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:55:49.163+0000 7f7c1a8fe640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7c1410da70 con 0x7f7c14107900 2024-09-12T01:55:49.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:55:49.163+0000 7f7c137fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c14104fd0 0x7f7c14100650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:55:49.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:55:49.163+0000 7f7c13fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c141026a0 0x7f7c14102000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:55:49.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:55:49.163+0000 7f7c137fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c14104fd0 0x7f7c14100650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58936/0 (socket says 172.21.15.33:58936) 2024-09-12T01:55:49.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:55:49.163+0000 7f7c137fe640 1 -- 172.21.15.33:0/1477013768 learned_addr learned my addr 172.21.15.33:0/1477013768 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-12T01:55:49.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:55:49.163+0000 7f7c18e9b640 1 --2- 172.21.15.33:0/1477013768 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7c14107900 0x7f7c14100b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:55:49.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:55:49.163+0000 7f7c137fe640 1 -- 172.21.15.33:0/1477013768 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7c14107900 msgr2=0x7f7c14100b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:55:49.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:55:49.163+0000 7f7c137fe640 1 --2- 172.21.15.33:0/1477013768 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7c14107900 0x7f7c14100b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:55:49.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:55:49.163+0000 7f7c137fe640 1 -- 172.21.15.33:0/1477013768 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c141026a0 msgr2=0x7f7c14102000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:55:49.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:55:49.163+0000 7f7c137fe640 1 --2- 172.21.15.33:0/1477013768 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c141026a0 0x7f7c14102000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:55:49.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:55:49.163+0000 7f7c137fe640 1 -- 172.21.15.33:0/1477013768 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c14101450 con 0x7f7c14104fd0 2024-09-12T01:55:49.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:55:49.163+0000 7f7c18e9b640 1 --2- 172.21.15.33:0/1477013768 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7c14107900 0x7f7c14100b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:55:49.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:55:49.163+0000 7f7c13fff640 1 --2- 172.21.15.33:0/1477013768 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c141026a0 0x7f7c14102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:55:49.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:55:49.163+0000 7f7c137fe640 1 --2- 172.21.15.33:0/1477013768 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c14104fd0 0x7f7c14100650 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f7c0800d770 tx=0x7f7c0800dc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:55:49.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:55:49.163+0000 7f7c117fa640 1 -- 172.21.15.33:0/1477013768 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c08017070 con 0x7f7c14104fd0 2024-09-12T01:55:49.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:55:49.163+0000 7f7c1a8fe640 1 -- 172.21.15.33:0/1477013768 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c141a2060 con 0x7f7c14104fd0 2024-09-12T01:55:49.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:55:49.163+0000 7f7c117fa640 1 -- 172.21.15.33:0/1477013768 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c08009c30 con 0x7f7c14104fd0 2024-09-12T01:55:49.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:55:49.163+0000 7f7c117fa640 1 -- 172.21.15.33:0/1477013768 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c0800f620 con 0x7f7c14104fd0 2024-09-12T01:55:49.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:55:49.163+0000 7f7c1a8fe640 1 -- 172.21.15.33:0/1477013768 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c141a25c0 con 0x7f7c14104fd0 2024-09-12T01:55:49.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:55:49.164+0000 7f7c1a8fe640 1 -- 172.21.15.33:0/1477013768 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c140fdd90 con 0x7f7c14104fd0 2024-09-12T01:55:49.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:55:49.166+0000 7f7c117fa640 1 -- 172.21.15.33:0/1477013768 <== mon.2 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f7c0801a070 con 0x7f7c14104fd0 2024-09-12T01:55:49.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:55:49.166+0000 7f7c117fa640 1 --2- 172.21.15.33:0/1477013768 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f7bfc0a2b80 0x7f7bfc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:55:49.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:55:49.166+0000 7f7c117fa640 1 -- 172.21.15.33:0/1477013768 <== mon.2 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 3389+0+0 (secure 0 0 0) 0x7f7c080cd790 con 0x7f7c14104fd0 2024-09-12T01:55:49.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:55:49.167+0000 7f7c13fff640 1 --2- 172.21.15.33:0/1477013768 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f7bfc0a2b80 0x7f7bfc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:55:49.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:55:49.167+0000 7f7c13fff640 1 --2- 172.21.15.33:0/1477013768 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f7bfc0a2b80 0x7f7bfc0a5040 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7c04009fc0 tx=0x7f7c04005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:55:49.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:55:49.170+0000 7f7c117fa640 1 -- 172.21.15.33:0/1477013768 <== mon.2 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7c08009de0 con 0x7f7c14104fd0 2024-09-12T01:55:49.279 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:48 smithi184 ceph-mon[35593]: from='osd.3 [v2:172.21.15.16:6810/3395182524,v1:172.21.15.16:6811/3395182524]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-12T01:55:49.279 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:48 smithi184 ceph-mon[35593]: osdmap e22: 4 total, 3 up, 4 in 2024-09-12T01:55:49.279 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:48 smithi184 ceph-mon[35593]: from='osd.3 [v2:172.21.15.16:6810/3395182524,v1:172.21.15.16:6811/3395182524]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-09-12T01:55:49.279 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:48 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T01:55:49.279 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:48 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:55:49.279 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:48 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:55:49.279 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:48 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:55:49.279 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:48 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:55:49.279 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:48 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:55:49.279 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:48 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:55:49.279 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:48 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:55:49.280 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:48 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:55:49.280 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:48 smithi184 ceph-mon[35593]: from='osd.3 [v2:172.21.15.16:6810/3395182524,v1:172.21.15.16:6811/3395182524]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-09-12T01:55:49.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:55:49.353+0000 7f7c1a8fe640 1 -- 172.21.15.33:0/1477013768 --> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f7c14062630 con 0x7f7bfc0a2b80 2024-09-12T01:55:50.054 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:49 smithi033 ceph-mon[35332]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-12T01:55:50.054 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:49 smithi033 ceph-mon[35332]: Detected new or changed devices on smithi016 2024-09-12T01:55:50.054 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:49 smithi033 ceph-mon[35332]: Adjusting osd_memory_target on smithi016 to 8546M 2024-09-12T01:55:50.054 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:49 smithi033 ceph-mon[35332]: osdmap e23: 4 total, 3 up, 4 in 2024-09-12T01:55:50.054 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:49 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T01:55:50.054 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:49 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T01:55:50.054 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:49 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T01:55:50.054 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:49 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T01:55:50.054 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:49 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:55:50.055 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:49 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T01:55:50.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:49 smithi184 ceph-mon[35593]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-12T01:55:50.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:49 smithi184 ceph-mon[35593]: Detected new or changed devices on smithi016 2024-09-12T01:55:50.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:49 smithi184 ceph-mon[35593]: Adjusting osd_memory_target on smithi016 to 8546M 2024-09-12T01:55:50.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:49 smithi184 ceph-mon[35593]: osdmap e23: 4 total, 3 up, 4 in 2024-09-12T01:55:50.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:49 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T01:55:50.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:49 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T01:55:50.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:49 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T01:55:50.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:49 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T01:55:50.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:49 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:55:50.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:49 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T01:55:50.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:49 smithi016 ceph-mon[30947]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-12T01:55:50.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:49 smithi016 ceph-mon[30947]: Detected new or changed devices on smithi016 2024-09-12T01:55:50.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:49 smithi016 ceph-mon[30947]: Adjusting osd_memory_target on smithi016 to 8546M 2024-09-12T01:55:50.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:49 smithi016 ceph-mon[30947]: osdmap e23: 4 total, 3 up, 4 in 2024-09-12T01:55:50.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:49 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T01:55:50.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:49 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T01:55:50.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:49 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T01:55:50.435 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:49 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T01:55:50.435 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:49 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:55:50.435 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:49 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T01:55:51.051 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:50 smithi033 ceph-mon[35332]: purged_snaps scrub starts 2024-09-12T01:55:51.052 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:50 smithi033 ceph-mon[35332]: purged_snaps scrub ok 2024-09-12T01:55:51.052 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:50 smithi033 ceph-mon[35332]: from='client.24206 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:55:51.052 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:50 smithi033 ceph-mon[35332]: osd.3 [v2:172.21.15.16:6810/3395182524,v1:172.21.15.16:6811/3395182524] boot 2024-09-12T01:55:51.052 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:50 smithi033 ceph-mon[35332]: osdmap e24: 4 total, 4 up, 4 in 2024-09-12T01:55:51.052 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:50 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T01:55:51.052 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:50 smithi033 ceph-mon[35332]: osdmap e25: 4 total, 4 up, 4 in 2024-09-12T01:55:51.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:50 smithi184 ceph-mon[35593]: purged_snaps scrub starts 2024-09-12T01:55:51.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:50 smithi184 ceph-mon[35593]: purged_snaps scrub ok 2024-09-12T01:55:51.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:50 smithi184 ceph-mon[35593]: from='client.24206 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:55:51.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:50 smithi184 ceph-mon[35593]: osd.3 [v2:172.21.15.16:6810/3395182524,v1:172.21.15.16:6811/3395182524] boot 2024-09-12T01:55:51.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:50 smithi184 ceph-mon[35593]: osdmap e24: 4 total, 4 up, 4 in 2024-09-12T01:55:51.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:50 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T01:55:51.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:50 smithi184 ceph-mon[35593]: osdmap e25: 4 total, 4 up, 4 in 2024-09-12T01:55:51.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:50 smithi016 ceph-mon[30947]: purged_snaps scrub starts 2024-09-12T01:55:51.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:50 smithi016 ceph-mon[30947]: purged_snaps scrub ok 2024-09-12T01:55:51.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:50 smithi016 ceph-mon[30947]: from='client.24206 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:55:51.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:50 smithi016 ceph-mon[30947]: osd.3 [v2:172.21.15.16:6810/3395182524,v1:172.21.15.16:6811/3395182524] boot 2024-09-12T01:55:51.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:50 smithi016 ceph-mon[30947]: osdmap e24: 4 total, 4 up, 4 in 2024-09-12T01:55:51.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:50 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T01:55:51.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:50 smithi016 ceph-mon[30947]: osdmap e25: 4 total, 4 up, 4 in 2024-09-12T01:55:52.149 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:51 smithi033 ceph-mon[35332]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-12T01:55:52.149 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:51 smithi033 ceph-mon[35332]: from='client.? 172.21.15.33:0/230245834' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3ca56e09-c194-4404-b9bd-2524d38367eb"}]: dispatch 2024-09-12T01:55:52.149 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:51 smithi033 ceph-mon[35332]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3ca56e09-c194-4404-b9bd-2524d38367eb"}]: dispatch 2024-09-12T01:55:52.150 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:51 smithi033 ceph-mon[35332]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3ca56e09-c194-4404-b9bd-2524d38367eb"}]': finished 2024-09-12T01:55:52.150 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:51 smithi033 ceph-mon[35332]: osdmap e26: 5 total, 4 up, 5 in 2024-09-12T01:55:52.150 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:51 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T01:55:52.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:51 smithi184 ceph-mon[35593]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-12T01:55:52.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:51 smithi184 ceph-mon[35593]: from='client.? 172.21.15.33:0/230245834' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3ca56e09-c194-4404-b9bd-2524d38367eb"}]: dispatch 2024-09-12T01:55:52.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:51 smithi184 ceph-mon[35593]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3ca56e09-c194-4404-b9bd-2524d38367eb"}]: dispatch 2024-09-12T01:55:52.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:51 smithi184 ceph-mon[35593]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3ca56e09-c194-4404-b9bd-2524d38367eb"}]': finished 2024-09-12T01:55:52.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:51 smithi184 ceph-mon[35593]: osdmap e26: 5 total, 4 up, 5 in 2024-09-12T01:55:52.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:51 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T01:55:52.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:51 smithi016 ceph-mon[30947]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-12T01:55:52.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:51 smithi016 ceph-mon[30947]: from='client.? 172.21.15.33:0/230245834' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3ca56e09-c194-4404-b9bd-2524d38367eb"}]: dispatch 2024-09-12T01:55:52.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:51 smithi016 ceph-mon[30947]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3ca56e09-c194-4404-b9bd-2524d38367eb"}]: dispatch 2024-09-12T01:55:52.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:51 smithi016 ceph-mon[30947]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3ca56e09-c194-4404-b9bd-2524d38367eb"}]': finished 2024-09-12T01:55:52.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:51 smithi016 ceph-mon[30947]: osdmap e26: 5 total, 4 up, 5 in 2024-09-12T01:55:52.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:51 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T01:55:53.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:52 smithi184 ceph-mon[35593]: from='client.? 172.21.15.33:0/2567282815' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T01:55:53.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:52 smithi184 ceph-mon[35593]: osdmap e27: 5 total, 4 up, 5 in 2024-09-12T01:55:53.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:52 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T01:55:53.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:52 smithi033 ceph-mon[35332]: from='client.? 172.21.15.33:0/2567282815' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T01:55:53.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:52 smithi033 ceph-mon[35332]: osdmap e27: 5 total, 4 up, 5 in 2024-09-12T01:55:53.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:52 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T01:55:53.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:52 smithi016 ceph-mon[30947]: from='client.? 172.21.15.33:0/2567282815' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T01:55:53.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:52 smithi016 ceph-mon[30947]: osdmap e27: 5 total, 4 up, 5 in 2024-09-12T01:55:53.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:52 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T01:55:54.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:53 smithi184 ceph-mon[35593]: pgmap v98: 1 pgs: 1 remapped+peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-12T01:55:54.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:53 smithi184 ceph-mon[35593]: osdmap e28: 5 total, 4 up, 5 in 2024-09-12T01:55:54.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:53 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T01:55:54.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:53 smithi033 ceph-mon[35332]: pgmap v98: 1 pgs: 1 remapped+peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-12T01:55:54.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:53 smithi033 ceph-mon[35332]: osdmap e28: 5 total, 4 up, 5 in 2024-09-12T01:55:54.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:53 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T01:55:54.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:53 smithi016 ceph-mon[30947]: pgmap v98: 1 pgs: 1 remapped+peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-12T01:55:54.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:53 smithi016 ceph-mon[30947]: osdmap e28: 5 total, 4 up, 5 in 2024-09-12T01:55:54.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:53 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T01:55:56.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:55 smithi184 ceph-mon[35593]: pgmap v101: 1 pgs: 1 remapped+peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-12T01:55:56.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:55 smithi033 ceph-mon[35332]: pgmap v101: 1 pgs: 1 remapped+peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-12T01:55:56.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:55 smithi016 ceph-mon[30947]: pgmap v101: 1 pgs: 1 remapped+peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-12T01:55:58.293 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:57 smithi033 ceph-mon[35332]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 117 MiB used, 357 GiB / 358 GiB avail; 109 KiB/s, 0 objects/s recovering 2024-09-12T01:55:58.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:57 smithi184 ceph-mon[35593]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 117 MiB used, 357 GiB / 358 GiB avail; 109 KiB/s, 0 objects/s recovering 2024-09-12T01:55:58.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:57 smithi016 ceph-mon[30947]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 117 MiB used, 357 GiB / 358 GiB avail; 109 KiB/s, 0 objects/s recovering 2024-09-12T01:56:00.070 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:55:59 smithi033 ceph-mon[35332]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 117 MiB used, 357 GiB / 358 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-09-12T01:56:00.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:55:59 smithi184 ceph-mon[35593]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 117 MiB used, 357 GiB / 358 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-09-12T01:56:00.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:55:59 smithi016 ceph-mon[30947]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 117 MiB used, 357 GiB / 358 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-09-12T01:56:01.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:56:00 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-12T01:56:01.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:56:00 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:56:01.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:56:00 smithi033 ceph-mon[35332]: Deploying daemon osd.4 on smithi033 2024-09-12T01:56:01.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:56:00 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-12T01:56:01.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:56:00 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:56:01.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:56:00 smithi184 ceph-mon[35593]: Deploying daemon osd.4 on smithi033 2024-09-12T01:56:01.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:56:00 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-12T01:56:01.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:56:00 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:56:01.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:56:00 smithi016 ceph-mon[30947]: Deploying daemon osd.4 on smithi033 2024-09-12T01:56:02.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:56:01 smithi184 ceph-mon[35593]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-12T01:56:02.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:56:01 smithi033 ceph-mon[35332]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-12T01:56:02.436 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:56:01 smithi016 ceph-mon[30947]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-12T01:56:04.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:56:03 smithi184 ceph-mon[35593]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-09-12T01:56:04.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:56:03 smithi033 ceph-mon[35332]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-09-12T01:56:04.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:56:03 smithi016 ceph-mon[30947]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-09-12T01:56:06.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:56:06 smithi033 ceph-mon[35332]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-09-12T01:56:06.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:56:06 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:56:06.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:56:06 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:56:06.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:56:06 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:56:06.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:56:05 smithi184 ceph-mon[35593]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-09-12T01:56:06.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:56:05 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:56:06.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:56:05 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:56:06.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:56:05 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:56:06.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:56:05 smithi016 ceph-mon[30947]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-09-12T01:56:06.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:56:05 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:56:06.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:56:05 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:56:06.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:56:05 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:56:08.129 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:56:08 smithi033 ceph-mon[35332]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-12T01:56:08.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:56:08 smithi184 ceph-mon[35593]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-12T01:56:08.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:56:08 smithi016 ceph-mon[30947]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-12T01:56:09.555 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:56:09 smithi016 ceph-mon[30947]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-12T01:56:09.555 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:56:09 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:56:09.555 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:56:09 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:56:09.555 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:56:09 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:56:09.555 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:56:09 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:56:09.555 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:56:09 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:56:09.574 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:56:09 smithi033 ceph-mon[35332]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-12T01:56:09.574 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:56:09 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:56:09.575 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:56:09 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:56:09.575 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:56:09 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:56:09.575 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:56:09 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:56:09.575 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:56:09 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:56:09.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:56:09.615+0000 7f7c117fa640 1 -- 172.21.15.33:0/1477013768 <== mgr.14150 v2:172.21.15.16:6800/3887444820 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f7c14062630 con 0x7f7bfc0a2b80 2024-09-12T01:56:09.615 INFO:teuthology.orchestra.run.smithi033.stdout:Created osd(s) 4 on host 'smithi033' 2024-09-12T01:56:09.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:56:09.617+0000 7f7c1a8fe640 1 -- 172.21.15.33:0/1477013768 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f7bfc0a2b80 msgr2=0x7f7bfc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:56:09.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:56:09.617+0000 7f7c1a8fe640 1 --2- 172.21.15.33:0/1477013768 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f7bfc0a2b80 0x7f7bfc0a5040 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7c04009fc0 tx=0x7f7c04005f90 comp rx=0 tx=0).stop 2024-09-12T01:56:09.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:56:09.617+0000 7f7c1a8fe640 1 -- 172.21.15.33:0/1477013768 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c14104fd0 msgr2=0x7f7c14100650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:56:09.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:56:09.617+0000 7f7c1a8fe640 1 --2- 172.21.15.33:0/1477013768 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c14104fd0 0x7f7c14100650 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f7c0800d770 tx=0x7f7c0800dc30 comp rx=0 tx=0).stop 2024-09-12T01:56:09.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:56:09.617+0000 7f7c1a8fe640 1 -- 172.21.15.33:0/1477013768 shutdown_connections 2024-09-12T01:56:09.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:56:09.617+0000 7f7c1a8fe640 1 --2- 172.21.15.33:0/1477013768 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f7bfc0a2b80 0x7f7bfc0a5040 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:09.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:56:09.617+0000 7f7c1a8fe640 1 --2- 172.21.15.33:0/1477013768 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7c14107900 0x7f7c14100b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:09.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:56:09.617+0000 7f7c1a8fe640 1 --2- 172.21.15.33:0/1477013768 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c14104fd0 0x7f7c14100650 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:09.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:56:09.617+0000 7f7c1a8fe640 1 --2- 172.21.15.33:0/1477013768 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c141026a0 0x7f7c14102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:09.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:56:09.617+0000 7f7c1a8fe640 1 -- 172.21.15.33:0/1477013768 >> 172.21.15.33:0/1477013768 conn(0x7f7c140fc2a0 msgr2=0x7f7c14102f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:56:09.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:56:09.617+0000 7f7c1a8fe640 1 -- 172.21.15.33:0/1477013768 shutdown_connections 2024-09-12T01:56:09.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:56:09.617+0000 7f7c1a8fe640 1 -- 172.21.15.33:0/1477013768 wait complete. 2024-09-12T01:56:09.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:56:09 smithi184 ceph-mon[35593]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-12T01:56:09.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:56:09 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:56:09.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:56:09 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:56:09.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:56:09 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:56:09.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:56:09 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:56:09.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:56:09 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:56:10.271 DEBUG:teuthology.orchestra.run.smithi033:osd.4> sudo journalctl -f -n 0 -u ceph-879f3b56-70a9-11ef-bcea-c7b262605968@osd.4.service 2024-09-12T01:56:10.273 INFO:tasks.cephadm:Deploying osd.5 on smithi184 with /dev/vg_nvme/lv_3... 2024-09-12T01:56:10.273 DEBUG:teuthology.orchestra.run.smithi184:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-12T01:56:10.529 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T01:56:10.551 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:56:10 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:56:10.551 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:56:10 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:56:10.551 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:56:10 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:56:10.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:56:10 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:56:10.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:56:10 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:56:10.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:56:10 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:56:10.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:56:10 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:56:10.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:56:10 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:56:10.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:56:10 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:56:11.787 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:56:11 smithi033 ceph-mon[35332]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-12T01:56:11.834 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:56:11 smithi184 ceph-mon[35593]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-12T01:56:11.933 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:56:11 smithi016 ceph-mon[30947]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-12T01:56:12.574 INFO:journalctl@ceph.osd.4.smithi033.stdout:Sep 12 01:56:12 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-4[46311]: 2024-09-12T01:56:12.338+0000 7f8041db9640 -1 osd.4 0 log_to_monitors true 2024-09-12T01:56:12.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:56:12 smithi184 ceph-mon[35593]: from='osd.4 [v2:172.21.15.33:6808/1355184322,v1:172.21.15.33:6809/1355184322]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-12T01:56:12.878 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:56:12 smithi184 ceph-mon[35593]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-12T01:56:12.887 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-12T01:56:12.899 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:56:12 smithi033 ceph-mon[35332]: from='osd.4 [v2:172.21.15.33:6808/1355184322,v1:172.21.15.33:6809/1355184322]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-12T01:56:12.899 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:56:12 smithi033 ceph-mon[35332]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-12T01:56:12.913 DEBUG:teuthology.orchestra.run.smithi184:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi184:vg_nvme/lv_3 2024-09-12T01:56:12.933 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:56:12 smithi016 ceph-mon[30947]: from='osd.4 [v2:172.21.15.33:6808/1355184322,v1:172.21.15.33:6809/1355184322]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-12T01:56:12.933 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:56:12 smithi016 ceph-mon[30947]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-12T01:56:13.167 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T01:56:13.646 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:56:13.614+0000 7f73edced640 1 -- 172.21.15.184:0/3514769431 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f73e8107900 msgr2=0x7f73e8109d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:56:13.646 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:56:13.614+0000 7f73edced640 1 --2- 172.21.15.184:0/3514769431 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f73e8107900 0x7f73e8109d50 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f73e0009ca0 tx=0x7f73e002a8c0 comp rx=0 tx=0).stop 2024-09-12T01:56:13.646 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:56:13.614+0000 7f73edced640 1 -- 172.21.15.184:0/3514769431 shutdown_connections 2024-09-12T01:56:13.646 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:56:13.614+0000 7f73edced640 1 --2- 172.21.15.184:0/3514769431 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f73e8107900 0x7f73e8109d50 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:13.647 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:56:13.614+0000 7f73edced640 1 --2- 172.21.15.184:0/3514769431 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73e8104fd0 0x7f73e81073c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:13.647 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:56:13.615+0000 7f73edced640 1 --2- 172.21.15.184:0/3514769431 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73e81026a0 0x7f73e8104a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:13.647 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:56:13.615+0000 7f73edced640 1 -- 172.21.15.184:0/3514769431 >> 172.21.15.184:0/3514769431 conn(0x7f73e80fc2a0 msgr2=0x7f73e80fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:56:13.647 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:56:13.615+0000 7f73edced640 1 -- 172.21.15.184:0/3514769431 shutdown_connections 2024-09-12T01:56:13.647 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:56:13.615+0000 7f73edced640 1 -- 172.21.15.184:0/3514769431 wait complete. 2024-09-12T01:56:13.647 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:56:13.615+0000 7f73edced640 1 Processor -- start 2024-09-12T01:56:13.647 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:56:13.615+0000 7f73edced640 1 -- start start 2024-09-12T01:56:13.647 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:56:13.616+0000 7f73edced640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73e81026a0 0x7f73e8197ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:13.647 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:56:13.616+0000 7f73edced640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73e8104fd0 0x7f73e8198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:13.647 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:56:13.616+0000 7f73edced640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f73e8107900 0x7f73e819f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:13.647 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:56:13.616+0000 7f73edced640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f73e810d8c0 con 0x7f73e8104fd0 2024-09-12T01:56:13.647 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:56:13.616+0000 7f73edced640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f73e810d740 con 0x7f73e81026a0 2024-09-12T01:56:13.648 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:56:13.616+0000 7f73edced640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f73e810da40 con 0x7f73e8107900 2024-09-12T01:56:13.648 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:56:13.616+0000 7f73e7fff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f73e8107900 0x7f73e819f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:13.648 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:56:13.616+0000 7f73e7fff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f73e8107900 0x7f73e819f4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.184:34802/0 (socket says 172.21.15.184:34802) 2024-09-12T01:56:13.648 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:56:13.616+0000 7f73e7fff640 1 -- 172.21.15.184:0/182949804 learned_addr learned my addr 172.21.15.184:0/182949804 (peer_addr_for_me v2:172.21.15.184:0/0) 2024-09-12T01:56:13.648 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:56:13.616+0000 7f73e77fe640 1 --2- 172.21.15.184:0/182949804 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73e81026a0 0x7f73e8197ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:13.648 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:56:13.616+0000 7f73deffd640 1 --2- 172.21.15.184:0/182949804 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73e8104fd0 0x7f73e8198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:13.648 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:56:13.616+0000 7f73e7fff640 1 -- 172.21.15.184:0/182949804 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73e81026a0 msgr2=0x7f73e8197ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:56:13.648 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:56:13.616+0000 7f73e7fff640 1 --2- 172.21.15.184:0/182949804 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73e81026a0 0x7f73e8197ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:13.648 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:56:13.616+0000 7f73e7fff640 1 -- 172.21.15.184:0/182949804 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73e8104fd0 msgr2=0x7f73e8198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:56:13.648 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:56:13.616+0000 7f73e7fff640 1 --2- 172.21.15.184:0/182949804 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73e8104fd0 0x7f73e8198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:13.648 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:56:13.616+0000 7f73e7fff640 1 -- 172.21.15.184:0/182949804 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73e819fb20 con 0x7f73e8107900 2024-09-12T01:56:13.648 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:56:13.616+0000 7f73e7fff640 1 --2- 172.21.15.184:0/182949804 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f73e8107900 0x7f73e819f4b0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f73e0005e30 tx=0x7f73e0031ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:56:13.649 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:56:13.616+0000 7f73e57fa640 1 -- 172.21.15.184:0/182949804 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73e0003c30 con 0x7f73e8107900 2024-09-12T01:56:13.649 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:56:13.616+0000 7f73edced640 1 -- 172.21.15.184:0/182949804 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73e819fdb0 con 0x7f73e8107900 2024-09-12T01:56:13.649 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:56:13.616+0000 7f73e57fa640 1 -- 172.21.15.184:0/182949804 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73e0003dd0 con 0x7f73e8107900 2024-09-12T01:56:13.649 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:56:13.616+0000 7f73e57fa640 1 -- 172.21.15.184:0/182949804 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73e002fde0 con 0x7f73e8107900 2024-09-12T01:56:13.649 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:56:13.616+0000 7f73edced640 1 -- 172.21.15.184:0/182949804 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73e81a02c0 con 0x7f73e8107900 2024-09-12T01:56:13.649 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:56:13.618+0000 7f73edced640 1 -- 172.21.15.184:0/182949804 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73e81053b0 con 0x7f73e8107900 2024-09-12T01:56:13.649 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:56:13.618+0000 7f73e57fa640 1 -- 172.21.15.184:0/182949804 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f73e0004050 con 0x7f73e8107900 2024-09-12T01:56:13.649 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:56:13.619+0000 7f73e57fa640 1 --2- 172.21.15.184:0/182949804 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f73d40a2bb0 0x7f73d40a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:13.649 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:56:13.619+0000 7f73e57fa640 1 -- 172.21.15.184:0/182949804 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 3805+0+0 (secure 0 0 0) 0x7f73e00ef220 con 0x7f73e8107900 2024-09-12T01:56:13.649 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:56:13.619+0000 7f73e77fe640 1 --2- 172.21.15.184:0/182949804 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f73d40a2bb0 0x7f73d40a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:13.649 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:56:13.620+0000 7f73e77fe640 1 --2- 172.21.15.184:0/182949804 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f73d40a2bb0 0x7f73d40a5070 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f73d80058c0 tx=0x7f73d800a250 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:56:13.649 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:56:13.624+0000 7f73e57fa640 1 -- 172.21.15.184:0/182949804 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f73e00bd300 con 0x7f73e8107900 2024-09-12T01:56:13.799 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:56:13.798+0000 7f73edced640 1 -- 172.21.15.184:0/182949804 --> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi184:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f73e8062630 con 0x7f73d40a2bb0 2024-09-12T01:56:13.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:56:13 smithi184 ceph-mon[35593]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-12T01:56:13.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:56:13 smithi184 ceph-mon[35593]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-12T01:56:13.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:56:13 smithi184 ceph-mon[35593]: osdmap e29: 5 total, 4 up, 5 in 2024-09-12T01:56:13.878 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:56:13 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T01:56:13.878 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:56:13 smithi184 ceph-mon[35593]: from='osd.4 [v2:172.21.15.33:6808/1355184322,v1:172.21.15.33:6809/1355184322]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-09-12T01:56:13.878 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:56:13 smithi184 ceph-mon[35593]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-09-12T01:56:13.878 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:56:13 smithi184 ceph-mon[35593]: Detected new or changed devices on smithi033 2024-09-12T01:56:13.878 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:56:13 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:56:13.878 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:56:13 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:56:13.878 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:56:13 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:56:13.878 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:56:13 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:56:13.878 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:56:13 smithi184 ceph-mon[35593]: Adjusting osd_memory_target on smithi033 to 8546M 2024-09-12T01:56:13.878 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:56:13 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:56:13.878 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:56:13 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:56:13.878 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:56:13 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:56:13.878 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:56:13 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:56:13.919 INFO:journalctl@ceph.osd.4.smithi033.stdout:Sep 12 01:56:13 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-4[46311]: 2024-09-12T01:56:13.632+0000 7f803d519640 -1 osd.4 0 waiting for initial osdmap 2024-09-12T01:56:13.919 INFO:journalctl@ceph.osd.4.smithi033.stdout:Sep 12 01:56:13 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-4[46311]: 2024-09-12T01:56:13.639+0000 7f8039b53640 -1 osd.4 30 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T01:56:13.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:56:13 smithi033 ceph-mon[35332]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-12T01:56:13.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:56:13 smithi033 ceph-mon[35332]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-12T01:56:13.920 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:56:13 smithi033 ceph-mon[35332]: osdmap e29: 5 total, 4 up, 5 in 2024-09-12T01:56:13.920 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:56:13 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T01:56:13.920 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:56:13 smithi033 ceph-mon[35332]: from='osd.4 [v2:172.21.15.33:6808/1355184322,v1:172.21.15.33:6809/1355184322]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-09-12T01:56:13.920 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:56:13 smithi033 ceph-mon[35332]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-09-12T01:56:13.920 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:56:13 smithi033 ceph-mon[35332]: Detected new or changed devices on smithi033 2024-09-12T01:56:13.920 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:56:13 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:56:13.920 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:56:13 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:56:13.920 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:56:13 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:56:13.920 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:56:13 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:56:13.920 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:56:13 smithi033 ceph-mon[35332]: Adjusting osd_memory_target on smithi033 to 8546M 2024-09-12T01:56:13.920 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:56:13 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:56:13.920 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:56:13 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:56:13.921 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:56:13 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:56:13.921 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:56:13 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:56:13.933 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:56:13 smithi016 ceph-mon[30947]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-12T01:56:13.934 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:56:13 smithi016 ceph-mon[30947]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-12T01:56:13.934 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:56:13 smithi016 ceph-mon[30947]: osdmap e29: 5 total, 4 up, 5 in 2024-09-12T01:56:13.934 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:56:13 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T01:56:13.934 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:56:13 smithi016 ceph-mon[30947]: from='osd.4 [v2:172.21.15.33:6808/1355184322,v1:172.21.15.33:6809/1355184322]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-09-12T01:56:13.934 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:56:13 smithi016 ceph-mon[30947]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-09-12T01:56:13.934 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:56:13 smithi016 ceph-mon[30947]: Detected new or changed devices on smithi033 2024-09-12T01:56:13.934 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:56:13 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:56:13.934 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:56:13 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:56:13.934 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:56:13 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:56:13.934 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:56:13 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:56:13.934 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:56:13 smithi016 ceph-mon[30947]: Adjusting osd_memory_target on smithi033 to 8546M 2024-09-12T01:56:13.934 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:56:13 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:56:13.935 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:56:13 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:56:13.935 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:56:13 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:56:13.935 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:56:13 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:56:14.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:56:14 smithi184 ceph-mon[35593]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]': finished 2024-09-12T01:56:14.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:56:14 smithi184 ceph-mon[35593]: osdmap e30: 5 total, 4 up, 5 in 2024-09-12T01:56:14.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:56:14 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T01:56:14.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:56:14 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T01:56:14.878 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:56:14 smithi184 ceph-mon[35593]: from='client.24235 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi184:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:56:14.878 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:56:14 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T01:56:14.878 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:56:14 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T01:56:14.878 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:56:14 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:56:14.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:56:14 smithi033 ceph-mon[35332]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]': finished 2024-09-12T01:56:14.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:56:14 smithi033 ceph-mon[35332]: osdmap e30: 5 total, 4 up, 5 in 2024-09-12T01:56:14.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:56:14 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T01:56:14.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:56:14 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T01:56:14.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:56:14 smithi033 ceph-mon[35332]: from='client.24235 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi184:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:56:14.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:56:14 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T01:56:14.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:56:14 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T01:56:14.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:56:14 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:56:14.933 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:56:14 smithi016 ceph-mon[30947]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]': finished 2024-09-12T01:56:14.933 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:56:14 smithi016 ceph-mon[30947]: osdmap e30: 5 total, 4 up, 5 in 2024-09-12T01:56:14.934 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:56:14 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T01:56:14.934 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:56:14 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T01:56:14.934 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:56:14 smithi016 ceph-mon[30947]: from='client.24235 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi184:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:56:14.934 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:56:14 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T01:56:14.934 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:56:14 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T01:56:14.934 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:56:14 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:56:15.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:56:15 smithi184 ceph-mon[35593]: purged_snaps scrub starts 2024-09-12T01:56:15.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:56:15 smithi184 ceph-mon[35593]: purged_snaps scrub ok 2024-09-12T01:56:15.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:56:15 smithi184 ceph-mon[35593]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-12T01:56:15.878 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:56:15 smithi184 ceph-mon[35593]: osd.4 [v2:172.21.15.33:6808/1355184322,v1:172.21.15.33:6809/1355184322] boot 2024-09-12T01:56:15.878 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:56:15 smithi184 ceph-mon[35593]: osdmap e31: 5 total, 5 up, 5 in 2024-09-12T01:56:15.878 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:56:15 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T01:56:15.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:56:15 smithi033 ceph-mon[35332]: purged_snaps scrub starts 2024-09-12T01:56:15.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:56:15 smithi033 ceph-mon[35332]: purged_snaps scrub ok 2024-09-12T01:56:15.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:56:15 smithi033 ceph-mon[35332]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-12T01:56:15.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:56:15 smithi033 ceph-mon[35332]: osd.4 [v2:172.21.15.33:6808/1355184322,v1:172.21.15.33:6809/1355184322] boot 2024-09-12T01:56:15.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:56:15 smithi033 ceph-mon[35332]: osdmap e31: 5 total, 5 up, 5 in 2024-09-12T01:56:15.920 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:56:15 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T01:56:15.933 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:56:15 smithi016 ceph-mon[30947]: purged_snaps scrub starts 2024-09-12T01:56:15.933 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:56:15 smithi016 ceph-mon[30947]: purged_snaps scrub ok 2024-09-12T01:56:15.934 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:56:15 smithi016 ceph-mon[30947]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-12T01:56:15.934 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:56:15 smithi016 ceph-mon[30947]: osd.4 [v2:172.21.15.33:6808/1355184322,v1:172.21.15.33:6809/1355184322] boot 2024-09-12T01:56:15.934 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:56:15 smithi016 ceph-mon[30947]: osdmap e31: 5 total, 5 up, 5 in 2024-09-12T01:56:15.934 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:56:15 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T01:56:16.789 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:56:16 smithi184 ceph-mon[35593]: osdmap e32: 5 total, 5 up, 5 in 2024-09-12T01:56:16.790 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:56:16 smithi184 ceph-mon[35593]: from='client.? 172.21.15.184:0/1595399448' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0b05ecdb-b987-461f-951c-6d76739bfb4c"}]: dispatch 2024-09-12T01:56:16.790 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:56:16 smithi184 ceph-mon[35593]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0b05ecdb-b987-461f-951c-6d76739bfb4c"}]: dispatch 2024-09-12T01:56:16.790 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:56:16 smithi184 ceph-mon[35593]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0b05ecdb-b987-461f-951c-6d76739bfb4c"}]': finished 2024-09-12T01:56:16.790 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:56:16 smithi184 ceph-mon[35593]: osdmap e33: 6 total, 5 up, 6 in 2024-09-12T01:56:16.790 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:56:16 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T01:56:16.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:56:16 smithi033 ceph-mon[35332]: osdmap e32: 5 total, 5 up, 5 in 2024-09-12T01:56:16.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:56:16 smithi033 ceph-mon[35332]: from='client.? 172.21.15.184:0/1595399448' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0b05ecdb-b987-461f-951c-6d76739bfb4c"}]: dispatch 2024-09-12T01:56:16.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:56:16 smithi033 ceph-mon[35332]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0b05ecdb-b987-461f-951c-6d76739bfb4c"}]: dispatch 2024-09-12T01:56:16.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:56:16 smithi033 ceph-mon[35332]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0b05ecdb-b987-461f-951c-6d76739bfb4c"}]': finished 2024-09-12T01:56:16.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:56:16 smithi033 ceph-mon[35332]: osdmap e33: 6 total, 5 up, 6 in 2024-09-12T01:56:16.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:56:16 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T01:56:16.933 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:56:16 smithi016 ceph-mon[30947]: osdmap e32: 5 total, 5 up, 5 in 2024-09-12T01:56:16.933 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:56:16 smithi016 ceph-mon[30947]: from='client.? 172.21.15.184:0/1595399448' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0b05ecdb-b987-461f-951c-6d76739bfb4c"}]: dispatch 2024-09-12T01:56:16.933 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:56:16 smithi016 ceph-mon[30947]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0b05ecdb-b987-461f-951c-6d76739bfb4c"}]: dispatch 2024-09-12T01:56:16.933 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:56:16 smithi016 ceph-mon[30947]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0b05ecdb-b987-461f-951c-6d76739bfb4c"}]': finished 2024-09-12T01:56:16.934 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:56:16 smithi016 ceph-mon[30947]: osdmap e33: 6 total, 5 up, 6 in 2024-09-12T01:56:16.934 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:56:16 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T01:56:17.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:56:17 smithi184 ceph-mon[35593]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-12T01:56:17.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:56:17 smithi184 ceph-mon[35593]: from='client.? 172.21.15.184:0/3948808131' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T01:56:17.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:56:17 smithi033 ceph-mon[35332]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-12T01:56:17.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:56:17 smithi033 ceph-mon[35332]: from='client.? 172.21.15.184:0/3948808131' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T01:56:17.933 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:56:17 smithi016 ceph-mon[30947]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-12T01:56:17.933 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:56:17 smithi016 ceph-mon[30947]: from='client.? 172.21.15.184:0/3948808131' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T01:56:19.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:56:19 smithi033 ceph-mon[35332]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-12T01:56:19.933 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:56:19 smithi016 ceph-mon[30947]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-12T01:56:20.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:56:19 smithi184 ceph-mon[35593]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-12T01:56:21.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:56:21 smithi184 ceph-mon[35593]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-12T01:56:21.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:56:21 smithi033 ceph-mon[35332]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-12T01:56:21.933 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:56:21 smithi016 ceph-mon[30947]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-12T01:56:23.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:56:23 smithi033 ceph-mon[35332]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-12T01:56:23.933 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:56:23 smithi016 ceph-mon[30947]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-12T01:56:23.970 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:56:23 smithi184 ceph-mon[35593]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-12T01:56:25.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:56:25 smithi033 ceph-mon[35332]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-12T01:56:25.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:56:25 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-12T01:56:25.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:56:25 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:56:25.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:56:25 smithi033 ceph-mon[35332]: Deploying daemon osd.5 on smithi184 2024-09-12T01:56:25.937 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:56:25 smithi016 ceph-mon[30947]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-12T01:56:25.937 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:56:25 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-12T01:56:25.937 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:56:25 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:56:25.937 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:56:25 smithi016 ceph-mon[30947]: Deploying daemon osd.5 on smithi184 2024-09-12T01:56:26.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:56:25 smithi184 ceph-mon[35593]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-12T01:56:26.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:56:25 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-12T01:56:26.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:56:25 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:56:26.128 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:56:25 smithi184 ceph-mon[35593]: Deploying daemon osd.5 on smithi184 2024-09-12T01:56:27.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:56:27 smithi033 ceph-mon[35332]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-12T01:56:27.933 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:56:27 smithi016 ceph-mon[30947]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-12T01:56:28.101 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:56:27 smithi184 ceph-mon[35593]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-12T01:56:29.878 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:56:29 smithi184 ceph-mon[35593]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-12T01:56:29.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:56:29 smithi033 ceph-mon[35332]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-12T01:56:29.933 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:56:29 smithi016 ceph-mon[30947]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-12T01:56:31.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:56:31 smithi016 ceph-mon[30947]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-12T01:56:31.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:56:31 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:56:31.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:56:31 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:56:31.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:56:31 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:56:31.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:56:31 smithi184 ceph-mon[35593]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-12T01:56:31.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:56:31 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:56:31.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:56:31 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:56:31.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:56:31 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:56:31.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:56:31 smithi033 ceph-mon[35332]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-12T01:56:31.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:56:31 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:56:31.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:56:31 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:56:31.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:56:31 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:56:33.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:56:33 smithi033 ceph-mon[35332]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-12T01:56:33.933 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:56:33 smithi016 ceph-mon[30947]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-12T01:56:34.073 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:56:33 smithi184 ceph-mon[35593]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-12T01:56:34.380 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:56:34.380+0000 7f73e57fa640 1 -- 172.21.15.184:0/182949804 <== mgr.14150 v2:172.21.15.16:6800/3887444820 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f73e8062630 con 0x7f73d40a2bb0 2024-09-12T01:56:34.380 INFO:teuthology.orchestra.run.smithi184.stdout:Created osd(s) 5 on host 'smithi184' 2024-09-12T01:56:34.382 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:56:34.382+0000 7f73edced640 1 -- 172.21.15.184:0/182949804 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f73d40a2bb0 msgr2=0x7f73d40a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:56:34.382 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:56:34.382+0000 7f73edced640 1 --2- 172.21.15.184:0/182949804 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f73d40a2bb0 0x7f73d40a5070 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f73d80058c0 tx=0x7f73d800a250 comp rx=0 tx=0).stop 2024-09-12T01:56:34.382 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:56:34.382+0000 7f73edced640 1 -- 172.21.15.184:0/182949804 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f73e8107900 msgr2=0x7f73e819f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:56:34.382 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:56:34.382+0000 7f73edced640 1 --2- 172.21.15.184:0/182949804 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f73e8107900 0x7f73e819f4b0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f73e0005e30 tx=0x7f73e0031ee0 comp rx=0 tx=0).stop 2024-09-12T01:56:34.382 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:56:34.382+0000 7f73edced640 1 -- 172.21.15.184:0/182949804 shutdown_connections 2024-09-12T01:56:34.383 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:56:34.382+0000 7f73edced640 1 --2- 172.21.15.184:0/182949804 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f73d40a2bb0 0x7f73d40a5070 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:34.383 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:56:34.382+0000 7f73edced640 1 --2- 172.21.15.184:0/182949804 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f73e8107900 0x7f73e819f4b0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:34.383 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:56:34.382+0000 7f73edced640 1 --2- 172.21.15.184:0/182949804 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73e8104fd0 0x7f73e8198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:34.383 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:56:34.382+0000 7f73edced640 1 --2- 172.21.15.184:0/182949804 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73e81026a0 0x7f73e8197ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:34.383 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:56:34.382+0000 7f73edced640 1 -- 172.21.15.184:0/182949804 >> 172.21.15.184:0/182949804 conn(0x7f73e80fc2a0 msgr2=0x7f73e8102f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:56:34.383 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:56:34.382+0000 7f73edced640 1 -- 172.21.15.184:0/182949804 shutdown_connections 2024-09-12T01:56:34.383 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:56:34.382+0000 7f73edced640 1 -- 172.21.15.184:0/182949804 wait complete. 2024-09-12T01:56:34.969 DEBUG:teuthology.orchestra.run.smithi184:osd.5> sudo journalctl -f -n 0 -u ceph-879f3b56-70a9-11ef-bcea-c7b262605968@osd.5.service 2024-09-12T01:56:34.971 INFO:tasks.cephadm:Deploying osd.6 on smithi016 with /dev/vg_nvme/lv_2... 2024-09-12T01:56:34.971 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-12T01:56:35.239 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:56:35.255 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:56:34 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:56:35.255 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:56:34 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:56:35.255 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:56:34 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:56:35.255 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:56:34 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:56:35.255 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:56:34 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:56:35.256 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:56:34 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:56:35.256 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:56:34 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:56:35.256 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:56:34 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:56:35.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:56:34 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:56:35.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:56:34 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:56:35.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:56:34 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:56:35.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:56:34 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:56:35.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:56:34 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:56:35.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:56:34 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:56:35.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:56:34 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:56:35.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:56:34 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:56:35.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:56:34 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:56:35.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:56:34 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:56:35.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:56:34 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:56:35.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:56:34 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:56:35.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:56:34 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:56:35.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:56:34 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:56:35.420 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:56:34 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:56:35.420 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:56:34 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:56:36.267 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:56:36 smithi184 ceph-mon[35593]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-12T01:56:36.278 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:56:36 smithi016 ceph-mon[30947]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-12T01:56:36.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:56:36 smithi033 ceph-mon[35332]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-12T01:56:37.378 INFO:journalctl@ceph.osd.5.smithi184.stdout:Sep 12 01:56:37 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-5[47266]: 2024-09-12T01:56:37.252+0000 7f6af60b8640 -1 osd.5 0 log_to_monitors true 2024-09-12T01:56:37.605 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:56:37.633 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi016:vg_nvme/lv_2 2024-09-12T01:56:37.889 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:56:38.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:56:38 smithi184 ceph-mon[35593]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-12T01:56:38.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:56:38 smithi184 ceph-mon[35593]: from='osd.5 [v2:172.21.15.184:6808/1533397276,v1:172.21.15.184:6809/1533397276]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-12T01:56:38.128 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:56:38 smithi184 ceph-mon[35593]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-12T01:56:38.128 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:56:38 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:56:38.128 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:56:38 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:56:38.128 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:56:38 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:56:38.128 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:56:38 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:56:38.128 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:56:38 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:56:38.128 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:56:38 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:56:38.128 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:56:38 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:56:38.128 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:56:38 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:56:38.334 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:56:38 smithi016 ceph-mon[30947]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-12T01:56:38.335 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:56:38 smithi016 ceph-mon[30947]: from='osd.5 [v2:172.21.15.184:6808/1533397276,v1:172.21.15.184:6809/1533397276]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-12T01:56:38.335 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:56:38 smithi016 ceph-mon[30947]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-12T01:56:38.335 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:56:38 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:56:38.335 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:56:38 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:56:38.335 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:56:38 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:56:38.335 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:56:38 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:56:38.335 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:56:38 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:56:38.335 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:56:38 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:56:38.335 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:56:38 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:56:38.335 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:56:38 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:56:38.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:56:38.334+0000 7fa243d99640 1 -- 172.21.15.16:0/3403606577 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa23c100dc0 msgr2=0x7fa23c101220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:56:38.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:56:38.334+0000 7fa243d99640 1 --2- 172.21.15.16:0/3403606577 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa23c100dc0 0x7fa23c101220 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fa238009900 tx=0x7fa23802a410 comp rx=0 tx=0).stop 2024-09-12T01:56:38.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:56:38.334+0000 7fa243d99640 1 -- 172.21.15.16:0/3403606577 shutdown_connections 2024-09-12T01:56:38.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:56:38.334+0000 7fa243d99640 1 --2- 172.21.15.16:0/3403606577 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa23c101760 0x7fa23c101c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:38.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:56:38.334+0000 7fa243d99640 1 --2- 172.21.15.16:0/3403606577 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa23c100dc0 0x7fa23c101220 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:38.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:56:38.334+0000 7fa243d99640 1 --2- 172.21.15.16:0/3403606577 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa23c100480 0x7fa23c100880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:38.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:56:38.334+0000 7fa243d99640 1 -- 172.21.15.16:0/3403606577 >> 172.21.15.16:0/3403606577 conn(0x7fa23c0fc270 msgr2=0x7fa23c0fe6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:56:38.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:56:38.334+0000 7fa243d99640 1 -- 172.21.15.16:0/3403606577 shutdown_connections 2024-09-12T01:56:38.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:56:38.334+0000 7fa243d99640 1 -- 172.21.15.16:0/3403606577 wait complete. 2024-09-12T01:56:38.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:56:38.335+0000 7fa243d99640 1 Processor -- start 2024-09-12T01:56:38.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:56:38.335+0000 7fa243d99640 1 -- start start 2024-09-12T01:56:38.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:56:38.335+0000 7fa243d99640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa23c100480 0x7fa23c19a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:38.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:56:38.335+0000 7fa243d99640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa23c100dc0 0x7fa23c19a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:38.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:56:38.335+0000 7fa243d99640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa23c101760 0x7fa23c1a16f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:38.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:56:38.335+0000 7fa243d99640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa23c10fa70 con 0x7fa23c101760 2024-09-12T01:56:38.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:56:38.335+0000 7fa243d99640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa23c10f8f0 con 0x7fa23c100dc0 2024-09-12T01:56:38.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:56:38.335+0000 7fa243d99640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa23c10fbf0 con 0x7fa23c100480 2024-09-12T01:56:38.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:56:38.335+0000 7fa242336640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa23c101760 0x7fa23c1a16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:38.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:56:38.335+0000 7fa242336640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa23c101760 0x7fa23c1a16f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52322/0 (socket says 172.21.15.16:52322) 2024-09-12T01:56:38.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:56:38.335+0000 7fa242336640 1 -- 172.21.15.16:0/321121855 learned_addr learned my addr 172.21.15.16:0/321121855 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:56:38.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:56:38.335+0000 7fa241b35640 1 --2- 172.21.15.16:0/321121855 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa23c100480 0x7fa23c19a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:38.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:56:38.335+0000 7fa241334640 1 --2- 172.21.15.16:0/321121855 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa23c100dc0 0x7fa23c19a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:38.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:56:38.336+0000 7fa242336640 1 -- 172.21.15.16:0/321121855 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa23c100480 msgr2=0x7fa23c19a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:56:38.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:56:38.336+0000 7fa242336640 1 --2- 172.21.15.16:0/321121855 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa23c100480 0x7fa23c19a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:38.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:56:38.336+0000 7fa242336640 1 -- 172.21.15.16:0/321121855 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa23c100dc0 msgr2=0x7fa23c19a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:56:38.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:56:38.336+0000 7fa242336640 1 --2- 172.21.15.16:0/321121855 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa23c100dc0 0x7fa23c19a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:38.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:56:38.336+0000 7fa242336640 1 -- 172.21.15.16:0/321121855 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa23c1a1d60 con 0x7fa23c101760 2024-09-12T01:56:38.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:56:38.336+0000 7fa241334640 1 --2- 172.21.15.16:0/321121855 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa23c100dc0 0x7fa23c19a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:56:38.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:56:38.336+0000 7fa242336640 1 --2- 172.21.15.16:0/321121855 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa23c101760 0x7fa23c1a16f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fa23400eaa0 tx=0x7fa23400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:56:38.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:56:38.336+0000 7fa232ffd640 1 -- 172.21.15.16:0/321121855 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa23400cdb0 con 0x7fa23c101760 2024-09-12T01:56:38.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:56:38.336+0000 7fa232ffd640 1 -- 172.21.15.16:0/321121855 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa234005020 con 0x7fa23c101760 2024-09-12T01:56:38.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:56:38.336+0000 7fa243d99640 1 -- 172.21.15.16:0/321121855 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa23c1a1ff0 con 0x7fa23c101760 2024-09-12T01:56:38.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:56:38.336+0000 7fa232ffd640 1 -- 172.21.15.16:0/321121855 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2340116b0 con 0x7fa23c101760 2024-09-12T01:56:38.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:56:38.336+0000 7fa243d99640 1 -- 172.21.15.16:0/321121855 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa23c1a2500 con 0x7fa23c101760 2024-09-12T01:56:38.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:56:38.337+0000 7fa243d99640 1 -- 172.21.15.16:0/321121855 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa23c10b3c0 con 0x7fa23c101760 2024-09-12T01:56:38.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:56:38.338+0000 7fa232ffd640 1 -- 172.21.15.16:0/321121855 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fa234005560 con 0x7fa23c101760 2024-09-12T01:56:38.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:56:38.339+0000 7fa232ffd640 1 --2- 172.21.15.16:0/321121855 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fa2280a2bb0 0x7fa2280a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:38.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:56:38.339+0000 7fa241b35640 1 --2- 172.21.15.16:0/321121855 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fa2280a2bb0 0x7fa2280a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:38.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:56:38.339+0000 7fa232ffd640 1 -- 172.21.15.16:0/321121855 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 4237+0+0 (secure 0 0 0) 0x7fa2340ce5e0 con 0x7fa23c101760 2024-09-12T01:56:38.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:56:38.340+0000 7fa241b35640 1 --2- 172.21.15.16:0/321121855 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fa2280a2bb0 0x7fa2280a5070 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa22c0058c0 tx=0x7fa22c00a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:56:38.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:56:38.343+0000 7fa232ffd640 1 -- 172.21.15.16:0/321121855 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa234016030 con 0x7fa23c101760 2024-09-12T01:56:38.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:56:38 smithi033 ceph-mon[35332]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-12T01:56:38.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:56:38 smithi033 ceph-mon[35332]: from='osd.5 [v2:172.21.15.184:6808/1533397276,v1:172.21.15.184:6809/1533397276]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-12T01:56:38.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:56:38 smithi033 ceph-mon[35332]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-12T01:56:38.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:56:38 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:56:38.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:56:38 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:56:38.420 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:56:38 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:56:38.420 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:56:38 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:56:38.420 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:56:38 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:56:38.420 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:56:38 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:56:38.420 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:56:38 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:56:38.420 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:56:38 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:56:38.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:56:38.529+0000 7fa243d99640 1 -- 172.21.15.16:0/321121855 --> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fa23c062630 con 0x7fa2280a2bb0 2024-09-12T01:56:39.225 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:56:39 smithi016 ceph-mon[30947]: Detected new or changed devices on smithi184 2024-09-12T01:56:39.225 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:56:39 smithi016 ceph-mon[30947]: Adjusting osd_memory_target on smithi184 to 8544M 2024-09-12T01:56:39.225 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:56:39 smithi016 ceph-mon[30947]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-12T01:56:39.225 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:56:39 smithi016 ceph-mon[30947]: osdmap e34: 6 total, 5 up, 6 in 2024-09-12T01:56:39.226 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:56:39 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T01:56:39.226 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:56:39 smithi016 ceph-mon[30947]: from='osd.5 [v2:172.21.15.184:6808/1533397276,v1:172.21.15.184:6809/1533397276]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2024-09-12T01:56:39.226 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:56:39 smithi016 ceph-mon[30947]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2024-09-12T01:56:39.226 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:56:39 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T01:56:39.226 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:56:39 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T01:56:39.226 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:56:39 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:56:39.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:56:39 smithi184 ceph-mon[35593]: Detected new or changed devices on smithi184 2024-09-12T01:56:39.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:56:39 smithi184 ceph-mon[35593]: Adjusting osd_memory_target on smithi184 to 8544M 2024-09-12T01:56:39.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:56:39 smithi184 ceph-mon[35593]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-12T01:56:39.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:56:39 smithi184 ceph-mon[35593]: osdmap e34: 6 total, 5 up, 6 in 2024-09-12T01:56:39.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:56:39 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T01:56:39.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:56:39 smithi184 ceph-mon[35593]: from='osd.5 [v2:172.21.15.184:6808/1533397276,v1:172.21.15.184:6809/1533397276]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2024-09-12T01:56:39.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:56:39 smithi184 ceph-mon[35593]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2024-09-12T01:56:39.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:56:39 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T01:56:39.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:56:39 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T01:56:39.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:56:39 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:56:39.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:56:39 smithi033 ceph-mon[35332]: Detected new or changed devices on smithi184 2024-09-12T01:56:39.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:56:39 smithi033 ceph-mon[35332]: Adjusting osd_memory_target on smithi184 to 8544M 2024-09-12T01:56:39.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:56:39 smithi033 ceph-mon[35332]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-12T01:56:39.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:56:39 smithi033 ceph-mon[35332]: osdmap e34: 6 total, 5 up, 6 in 2024-09-12T01:56:39.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:56:39 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T01:56:39.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:56:39 smithi033 ceph-mon[35332]: from='osd.5 [v2:172.21.15.184:6808/1533397276,v1:172.21.15.184:6809/1533397276]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2024-09-12T01:56:39.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:56:39 smithi033 ceph-mon[35332]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2024-09-12T01:56:39.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:56:39 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T01:56:39.420 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:56:39 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T01:56:39.420 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:56:39 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:56:40.127 INFO:journalctl@ceph.osd.5.smithi184.stdout:Sep 12 01:56:39 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-5[47266]: 2024-09-12T01:56:39.800+0000 7f6af1818640 -1 osd.5 0 waiting for initial osdmap 2024-09-12T01:56:40.127 INFO:journalctl@ceph.osd.5.smithi184.stdout:Sep 12 01:56:39 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-5[47266]: 2024-09-12T01:56:39.807+0000 7f6ae9e4a640 -1 osd.5 35 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T01:56:40.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:56:40 smithi184 ceph-mon[35593]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-12T01:56:40.128 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:56:40 smithi184 ceph-mon[35593]: from='client.14373 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:56:40.128 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:56:40 smithi184 ceph-mon[35593]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]': finished 2024-09-12T01:56:40.128 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:56:40 smithi184 ceph-mon[35593]: osdmap e35: 6 total, 5 up, 6 in 2024-09-12T01:56:40.128 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:56:40 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T01:56:40.128 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:56:40 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T01:56:40.128 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:56:40 smithi184 ceph-mon[35593]: from='osd.5 ' entity='osd.5' 2024-09-12T01:56:40.274 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:56:40 smithi016 ceph-mon[30947]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-12T01:56:40.274 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:56:40 smithi016 ceph-mon[30947]: from='client.14373 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:56:40.274 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:56:40 smithi016 ceph-mon[30947]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]': finished 2024-09-12T01:56:40.274 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:56:40 smithi016 ceph-mon[30947]: osdmap e35: 6 total, 5 up, 6 in 2024-09-12T01:56:40.274 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:56:40 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T01:56:40.274 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:56:40 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T01:56:40.274 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:56:40 smithi016 ceph-mon[30947]: from='osd.5 ' entity='osd.5' 2024-09-12T01:56:40.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:56:40 smithi033 ceph-mon[35332]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-12T01:56:40.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:56:40 smithi033 ceph-mon[35332]: from='client.14373 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:56:40.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:56:40 smithi033 ceph-mon[35332]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]': finished 2024-09-12T01:56:40.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:56:40 smithi033 ceph-mon[35332]: osdmap e35: 6 total, 5 up, 6 in 2024-09-12T01:56:40.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:56:40 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T01:56:40.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:56:40 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T01:56:40.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:56:40 smithi033 ceph-mon[35332]: from='osd.5 ' entity='osd.5' 2024-09-12T01:56:41.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:56:41 smithi016 ceph-mon[30947]: purged_snaps scrub starts 2024-09-12T01:56:41.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:56:41 smithi016 ceph-mon[30947]: purged_snaps scrub ok 2024-09-12T01:56:41.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:56:41 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T01:56:41.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:56:41 smithi016 ceph-mon[30947]: osd.5 [v2:172.21.15.184:6808/1533397276,v1:172.21.15.184:6809/1533397276] boot 2024-09-12T01:56:41.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:56:41 smithi016 ceph-mon[30947]: osdmap e36: 6 total, 6 up, 6 in 2024-09-12T01:56:41.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:56:41 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T01:56:41.362 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:56:41 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/3796963288' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b3936e21-e197-49fa-8441-d909b258b612"}]: dispatch 2024-09-12T01:56:41.362 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:56:41 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/3796963288' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b3936e21-e197-49fa-8441-d909b258b612"}]': finished 2024-09-12T01:56:41.362 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:56:41 smithi016 ceph-mon[30947]: osdmap e37: 7 total, 6 up, 7 in 2024-09-12T01:56:41.362 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:56:41 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T01:56:41.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:56:41 smithi184 ceph-mon[35593]: purged_snaps scrub starts 2024-09-12T01:56:41.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:56:41 smithi184 ceph-mon[35593]: purged_snaps scrub ok 2024-09-12T01:56:41.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:56:41 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T01:56:41.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:56:41 smithi184 ceph-mon[35593]: osd.5 [v2:172.21.15.184:6808/1533397276,v1:172.21.15.184:6809/1533397276] boot 2024-09-12T01:56:41.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:56:41 smithi184 ceph-mon[35593]: osdmap e36: 6 total, 6 up, 6 in 2024-09-12T01:56:41.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:56:41 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T01:56:41.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:56:41 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/3796963288' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b3936e21-e197-49fa-8441-d909b258b612"}]: dispatch 2024-09-12T01:56:41.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:56:41 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/3796963288' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b3936e21-e197-49fa-8441-d909b258b612"}]': finished 2024-09-12T01:56:41.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:56:41 smithi184 ceph-mon[35593]: osdmap e37: 7 total, 6 up, 7 in 2024-09-12T01:56:41.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:56:41 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T01:56:41.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:56:41 smithi033 ceph-mon[35332]: purged_snaps scrub starts 2024-09-12T01:56:41.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:56:41 smithi033 ceph-mon[35332]: purged_snaps scrub ok 2024-09-12T01:56:41.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:56:41 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T01:56:41.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:56:41 smithi033 ceph-mon[35332]: osd.5 [v2:172.21.15.184:6808/1533397276,v1:172.21.15.184:6809/1533397276] boot 2024-09-12T01:56:41.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:56:41 smithi033 ceph-mon[35332]: osdmap e36: 6 total, 6 up, 6 in 2024-09-12T01:56:41.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:56:41 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T01:56:41.420 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:56:41 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/3796963288' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b3936e21-e197-49fa-8441-d909b258b612"}]: dispatch 2024-09-12T01:56:41.420 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:56:41 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/3796963288' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b3936e21-e197-49fa-8441-d909b258b612"}]': finished 2024-09-12T01:56:41.420 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:56:41 smithi033 ceph-mon[35332]: osdmap e37: 7 total, 6 up, 7 in 2024-09-12T01:56:41.420 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:56:41 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T01:56:42.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:56:42 smithi184 ceph-mon[35593]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-12T01:56:42.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:56:42 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/2373934467' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T01:56:42.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:56:42 smithi184 ceph-mon[35593]: osdmap e38: 7 total, 6 up, 7 in 2024-09-12T01:56:42.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:56:42 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T01:56:42.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:56:42 smithi033 ceph-mon[35332]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-12T01:56:42.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:56:42 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/2373934467' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T01:56:42.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:56:42 smithi033 ceph-mon[35332]: osdmap e38: 7 total, 6 up, 7 in 2024-09-12T01:56:42.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:56:42 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T01:56:42.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:56:42 smithi016 ceph-mon[30947]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-12T01:56:42.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:56:42 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/2373934467' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T01:56:42.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:56:42 smithi016 ceph-mon[30947]: osdmap e38: 7 total, 6 up, 7 in 2024-09-12T01:56:42.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:56:42 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T01:56:44.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:56:43 smithi184 ceph-mon[35593]: pgmap v135: 1 pgs: 1 remapped+peering; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-12T01:56:44.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:56:43 smithi184 ceph-mon[35593]: osdmap e39: 7 total, 6 up, 7 in 2024-09-12T01:56:44.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:56:43 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T01:56:44.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:56:43 smithi033 ceph-mon[35332]: pgmap v135: 1 pgs: 1 remapped+peering; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-12T01:56:44.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:56:43 smithi033 ceph-mon[35332]: osdmap e39: 7 total, 6 up, 7 in 2024-09-12T01:56:44.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:56:43 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T01:56:44.183 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:56:43 smithi016 ceph-mon[30947]: pgmap v135: 1 pgs: 1 remapped+peering; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-12T01:56:44.183 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:56:43 smithi016 ceph-mon[30947]: osdmap e39: 7 total, 6 up, 7 in 2024-09-12T01:56:44.183 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:56:43 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T01:56:45.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:56:44 smithi184 ceph-mon[35593]: osdmap e40: 7 total, 6 up, 7 in 2024-09-12T01:56:45.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:56:44 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T01:56:45.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:56:44 smithi033 ceph-mon[35332]: osdmap e40: 7 total, 6 up, 7 in 2024-09-12T01:56:45.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:56:44 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T01:56:45.183 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:56:44 smithi016 ceph-mon[30947]: osdmap e40: 7 total, 6 up, 7 in 2024-09-12T01:56:45.183 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:56:44 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T01:56:45.934 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:56:45 smithi016 ceph-mon[30947]: pgmap v138: 1 pgs: 1 remapped+peering; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-12T01:56:46.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:56:45 smithi184 ceph-mon[35593]: pgmap v138: 1 pgs: 1 remapped+peering; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-12T01:56:46.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:56:45 smithi033 ceph-mon[35332]: pgmap v138: 1 pgs: 1 remapped+peering; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-12T01:56:48.115 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:56:47 smithi016 ceph-mon[30947]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 105 KiB/s, 0 objects/s recovering 2024-09-12T01:56:48.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:56:47 smithi184 ceph-mon[35593]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 105 KiB/s, 0 objects/s recovering 2024-09-12T01:56:48.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:56:47 smithi033 ceph-mon[35332]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 105 KiB/s, 0 objects/s recovering 2024-09-12T01:56:50.077 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:56:49 smithi016 ceph-mon[30947]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-09-12T01:56:50.077 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:56:49 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-12T01:56:50.077 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:56:49 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:56:50.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:56:49 smithi184 ceph-mon[35593]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-09-12T01:56:50.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:56:49 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-12T01:56:50.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:56:49 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:56:50.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:56:49 smithi033 ceph-mon[35332]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-09-12T01:56:50.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:56:49 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-12T01:56:50.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:56:49 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:56:51.011 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:56:50 smithi016 ceph-mon[30947]: Deploying daemon osd.6 on smithi016 2024-09-12T01:56:51.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:56:50 smithi184 ceph-mon[35593]: Deploying daemon osd.6 on smithi016 2024-09-12T01:56:51.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:56:50 smithi033 ceph-mon[35332]: Deploying daemon osd.6 on smithi016 2024-09-12T01:56:52.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:56:51 smithi184 ceph-mon[35593]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-12T01:56:52.160 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:56:51 smithi016 ceph-mon[30947]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-12T01:56:52.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:56:51 smithi033 ceph-mon[35332]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-12T01:56:53.983 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:56:53 smithi016 ceph-mon[30947]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-09-12T01:56:54.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:56:53 smithi184 ceph-mon[35593]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-09-12T01:56:54.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:56:53 smithi033 ceph-mon[35332]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-09-12T01:56:55.993 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:56:55 smithi016 ceph-mon[30947]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-09-12T01:56:55.993 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:56:55 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:56:55.993 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:56:55 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:56:55.993 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:56:55 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:56:56.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:56:55 smithi184 ceph-mon[35593]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-09-12T01:56:56.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:56:55 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:56:56.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:56:55 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:56:56.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:56:55 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:56:56.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:56:55 smithi033 ceph-mon[35332]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-09-12T01:56:56.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:56:55 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:56:56.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:56:55 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:56:56.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:56:55 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:56:58.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:56:57 smithi184 ceph-mon[35593]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-12T01:56:58.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:56:57 smithi033 ceph-mon[35332]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-12T01:56:58.183 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:56:57 smithi016 ceph-mon[30947]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-12T01:56:58.897 INFO:teuthology.orchestra.run.smithi016.stdout:Created osd(s) 6 on host 'smithi016' 2024-09-12T01:56:58.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:56:58.896+0000 7fa232ffd640 1 -- 172.21.15.16:0/321121855 <== mgr.14150 v2:172.21.15.16:6800/3887444820 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fa23c062630 con 0x7fa2280a2bb0 2024-09-12T01:56:58.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:56:58.899+0000 7fa243d99640 1 -- 172.21.15.16:0/321121855 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fa2280a2bb0 msgr2=0x7fa2280a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:56:58.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:56:58.899+0000 7fa243d99640 1 --2- 172.21.15.16:0/321121855 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fa2280a2bb0 0x7fa2280a5070 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa22c0058c0 tx=0x7fa22c00a2d0 comp rx=0 tx=0).stop 2024-09-12T01:56:58.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:56:58.899+0000 7fa243d99640 1 -- 172.21.15.16:0/321121855 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa23c101760 msgr2=0x7fa23c1a16f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:56:58.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:56:58.899+0000 7fa243d99640 1 --2- 172.21.15.16:0/321121855 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa23c101760 0x7fa23c1a16f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fa23400eaa0 tx=0x7fa23400ef60 comp rx=0 tx=0).stop 2024-09-12T01:56:58.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:56:58.900+0000 7fa243d99640 1 -- 172.21.15.16:0/321121855 shutdown_connections 2024-09-12T01:56:58.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:56:58.900+0000 7fa243d99640 1 --2- 172.21.15.16:0/321121855 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fa2280a2bb0 0x7fa2280a5070 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:58.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:56:58.900+0000 7fa243d99640 1 --2- 172.21.15.16:0/321121855 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa23c101760 0x7fa23c1a16f0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:58.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:56:58.900+0000 7fa243d99640 1 --2- 172.21.15.16:0/321121855 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa23c100dc0 0x7fa23c19a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:58.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:56:58.900+0000 7fa243d99640 1 --2- 172.21.15.16:0/321121855 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa23c100480 0x7fa23c19a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:58.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:56:58.900+0000 7fa243d99640 1 -- 172.21.15.16:0/321121855 >> 172.21.15.16:0/321121855 conn(0x7fa23c0fc270 msgr2=0x7fa23c0fda90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:56:58.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:56:58.900+0000 7fa243d99640 1 -- 172.21.15.16:0/321121855 shutdown_connections 2024-09-12T01:56:58.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:56:58.900+0000 7fa243d99640 1 -- 172.21.15.16:0/321121855 wait complete. 2024-09-12T01:56:59.183 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:56:58 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:56:59.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:56:58 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:56:59.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:56:58 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:56:59.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:56:58 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:56:59.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:56:58 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:56:59.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:56:58 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:56:59.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:56:58 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:56:59.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:56:58 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:56:59.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:56:58 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:56:59.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:56:58 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:56:59.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:56:58 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:56:59.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:56:58 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:56:59.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:56:58 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:56:59.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:56:58 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:56:59.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:56:58 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:56:59.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:56:58 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:56:59.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:56:58 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:56:59.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:56:58 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:56:59.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:56:58 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:56:59.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:56:58 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:56:59.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:56:58 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:56:59.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:56:58 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:56:59.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:56:58 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:56:59.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:56:58 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:56:59.563 DEBUG:teuthology.orchestra.run.smithi016:osd.6> sudo journalctl -f -n 0 -u ceph-879f3b56-70a9-11ef-bcea-c7b262605968@osd.6.service 2024-09-12T01:56:59.565 INFO:tasks.cephadm:Deploying osd.7 on smithi033 with /dev/vg_nvme/lv_2... 2024-09-12T01:56:59.565 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-12T01:56:59.826 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T01:57:00.183 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:56:59 smithi016 ceph-mon[30947]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-12T01:57:00.232 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:56:59 smithi033 ceph-mon[35332]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-12T01:57:00.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:56:59 smithi184 ceph-mon[35593]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-12T01:57:01.934 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 12 01:57:01 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-6[58414]: 2024-09-12T01:57:01.588+0000 7fa4b42d2640 -1 osd.6 0 log_to_monitors true 2024-09-12T01:57:02.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:01 smithi184 ceph-mon[35593]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-12T01:57:02.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:01 smithi184 ceph-mon[35593]: from='osd.6 [v2:172.21.15.16:6818/902466142,v1:172.21.15.16:6819/902466142]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-12T01:57:02.282 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:01 smithi016 ceph-mon[30947]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-12T01:57:02.282 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:01 smithi016 ceph-mon[30947]: from='osd.6 [v2:172.21.15.16:6818/902466142,v1:172.21.15.16:6819/902466142]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-12T01:57:02.376 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:01 smithi033 ceph-mon[35332]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-12T01:57:02.377 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:01 smithi033 ceph-mon[35332]: from='osd.6 [v2:172.21.15.16:6818/902466142,v1:172.21.15.16:6819/902466142]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-12T01:57:02.380 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-12T01:57:02.411 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi033:vg_nvme/lv_2 2024-09-12T01:57:02.669 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T01:57:03.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:57:03.148+0000 7f75fe8fa640 1 -- 172.21.15.33:0/912096164 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f75f8100ad0 msgr2=0x7f75f8100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:57:03.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:57:03.148+0000 7f75fe8fa640 1 --2- 172.21.15.33:0/912096164 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f75f8100ad0 0x7f75f8100ed0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f75e8009900 tx=0x7f75e802a410 comp rx=0 tx=0).stop 2024-09-12T01:57:03.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:57:03.148+0000 7f75fe8fa640 1 -- 172.21.15.33:0/912096164 shutdown_connections 2024-09-12T01:57:03.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:57:03.148+0000 7f75fe8fa640 1 --2- 172.21.15.33:0/912096164 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75f8102690 0x7f75f8108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:57:03.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:57:03.148+0000 7f75fe8fa640 1 --2- 172.21.15.33:0/912096164 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f75f8101cd0 0x7f75f8102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:57:03.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:57:03.148+0000 7f75fe8fa640 1 --2- 172.21.15.33:0/912096164 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f75f8100ad0 0x7f75f8100ed0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:57:03.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:57:03.148+0000 7f75fe8fa640 1 -- 172.21.15.33:0/912096164 >> 172.21.15.33:0/912096164 conn(0x7f75f80fc280 msgr2=0x7f75f80fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:57:03.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:57:03.148+0000 7f75fe8fa640 1 -- 172.21.15.33:0/912096164 shutdown_connections 2024-09-12T01:57:03.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:57:03.148+0000 7f75fe8fa640 1 -- 172.21.15.33:0/912096164 wait complete. 2024-09-12T01:57:03.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:57:03.149+0000 7f75fe8fa640 1 Processor -- start 2024-09-12T01:57:03.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:57:03.149+0000 7f75fe8fa640 1 -- start start 2024-09-12T01:57:03.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:57:03.149+0000 7f75fe8fa640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75f8100ad0 0x7f75f8197fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:57:03.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:57:03.149+0000 7f75fe8fa640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f75f8101cd0 0x7f75f8198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:57:03.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:57:03.149+0000 7f75fe8fa640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f75f8102690 0x7f75f819f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:57:03.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:57:03.149+0000 7f75fe8fa640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f75f810c7b0 con 0x7f75f8100ad0 2024-09-12T01:57:03.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:57:03.149+0000 7f75fe8fa640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f75f810c630 con 0x7f75f8101cd0 2024-09-12T01:57:03.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:57:03.149+0000 7f75fe8fa640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f75f810c930 con 0x7f75f8102690 2024-09-12T01:57:03.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:57:03.149+0000 7f75f77fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f75f8101cd0 0x7f75f8198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:57:03.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:57:03.149+0000 7f75f7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75f8100ad0 0x7f75f8197fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:57:03.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:57:03.149+0000 7f75f77fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f75f8101cd0 0x7f75f8198500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35724/0 (socket says 172.21.15.33:35724) 2024-09-12T01:57:03.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:57:03.149+0000 7f75f77fe640 1 -- 172.21.15.33:0/2922431527 learned_addr learned my addr 172.21.15.33:0/2922431527 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-12T01:57:03.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:57:03.149+0000 7f75f7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75f8100ad0 0x7f75f8197fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.33:51310/0 (socket says 172.21.15.33:51310) 2024-09-12T01:57:03.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:57:03.149+0000 7f75fce97640 1 --2- 172.21.15.33:0/2922431527 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f75f8102690 0x7f75f819f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:57:03.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:57:03.150+0000 7f75f7fff640 1 -- 172.21.15.33:0/2922431527 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f75f8102690 msgr2=0x7f75f819f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:57:03.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:57:03.150+0000 7f75f7fff640 1 --2- 172.21.15.33:0/2922431527 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f75f8102690 0x7f75f819f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:57:03.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:57:03.150+0000 7f75f7fff640 1 -- 172.21.15.33:0/2922431527 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f75f8101cd0 msgr2=0x7f75f8198500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:57:03.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:57:03.150+0000 7f75f7fff640 1 --2- 172.21.15.33:0/2922431527 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f75f8101cd0 0x7f75f8198500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:57:03.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:57:03.150+0000 7f75f7fff640 1 -- 172.21.15.33:0/2922431527 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75f819fbf0 con 0x7f75f8100ad0 2024-09-12T01:57:03.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:57:03.150+0000 7f75f77fe640 1 --2- 172.21.15.33:0/2922431527 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f75f8101cd0 0x7f75f8198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:57:03.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:57:03.150+0000 7f75f7fff640 1 --2- 172.21.15.33:0/2922431527 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75f8100ad0 0x7f75f8197fc0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f75e8009230 tx=0x7f75e8031b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:57:03.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:57:03.150+0000 7f75f57fa640 1 -- 172.21.15.33:0/2922431527 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75e803f070 con 0x7f75f8100ad0 2024-09-12T01:57:03.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:57:03.150+0000 7f75f57fa640 1 -- 172.21.15.33:0/2922431527 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75e8003cd0 con 0x7f75f8100ad0 2024-09-12T01:57:03.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:57:03.150+0000 7f75fe8fa640 1 -- 172.21.15.33:0/2922431527 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75f819fe80 con 0x7f75f8100ad0 2024-09-12T01:57:03.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:57:03.150+0000 7f75f57fa640 1 -- 172.21.15.33:0/2922431527 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75e8035070 con 0x7f75f8100ad0 2024-09-12T01:57:03.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:57:03.150+0000 7f75fe8fa640 1 -- 172.21.15.33:0/2922431527 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75f81a02f0 con 0x7f75f8100ad0 2024-09-12T01:57:03.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:57:03.151+0000 7f75fe8fa640 1 -- 172.21.15.33:0/2922431527 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75f81a05e0 con 0x7f75f8100ad0 2024-09-12T01:57:03.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:57:03.152+0000 7f75f57fa640 1 -- 172.21.15.33:0/2922431527 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f75e803b4f0 con 0x7f75f8100ad0 2024-09-12T01:57:03.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:57:03.153+0000 7f75f57fa640 1 --2- 172.21.15.33:0/2922431527 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f75e00a2bb0 0x7f75e00a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:57:03.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:57:03.153+0000 7f75f57fa640 1 -- 172.21.15.33:0/2922431527 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 4685+0+0 (secure 0 0 0) 0x7f75e803be00 con 0x7f75f8100ad0 2024-09-12T01:57:03.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:57:03.153+0000 7f75f77fe640 1 --2- 172.21.15.33:0/2922431527 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f75e00a2bb0 0x7f75e00a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:57:03.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:57:03.154+0000 7f75f77fe640 1 --2- 172.21.15.33:0/2922431527 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f75e00a2bb0 0x7f75e00a5070 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f75f81994e0 tx=0x7f75ec0091c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:57:03.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:57:03.158+0000 7f75f57fa640 1 -- 172.21.15.33:0/2922431527 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f75e80bbda0 con 0x7f75f8100ad0 2024-09-12T01:57:03.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:57:03.342+0000 7f75fe8fa640 1 -- 172.21.15.33:0/2922431527 --> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f75f806a6f0 con 0x7f75e00a2bb0 2024-09-12T01:57:03.344 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:02 smithi033 ceph-mon[35332]: from='osd.6 [v2:172.21.15.16:6818/902466142,v1:172.21.15.16:6819/902466142]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-12T01:57:03.344 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:02 smithi033 ceph-mon[35332]: osdmap e41: 7 total, 6 up, 7 in 2024-09-12T01:57:03.344 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:02 smithi033 ceph-mon[35332]: from='osd.6 [v2:172.21.15.16:6818/902466142,v1:172.21.15.16:6819/902466142]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-09-12T01:57:03.344 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:02 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T01:57:03.344 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:02 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:57:03.344 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:02 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:57:03.344 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:02 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:57:03.344 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:02 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:57:03.344 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:02 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:57:03.344 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:02 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:57:03.344 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:02 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:57:03.345 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:02 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:57:03.345 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:02 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:57:03.345 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:02 smithi033 ceph-mon[35332]: from='osd.6 [v2:172.21.15.16:6818/902466142,v1:172.21.15.16:6819/902466142]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-09-12T01:57:03.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:02 smithi184 ceph-mon[35593]: from='osd.6 [v2:172.21.15.16:6818/902466142,v1:172.21.15.16:6819/902466142]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-12T01:57:03.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:02 smithi184 ceph-mon[35593]: osdmap e41: 7 total, 6 up, 7 in 2024-09-12T01:57:03.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:02 smithi184 ceph-mon[35593]: from='osd.6 [v2:172.21.15.16:6818/902466142,v1:172.21.15.16:6819/902466142]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-09-12T01:57:03.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:02 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T01:57:03.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:02 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:57:03.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:02 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:57:03.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:02 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:57:03.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:02 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:57:03.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:02 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:57:03.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:02 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:57:03.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:02 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:57:03.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:02 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:57:03.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:02 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:57:03.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:02 smithi184 ceph-mon[35593]: from='osd.6 [v2:172.21.15.16:6818/902466142,v1:172.21.15.16:6819/902466142]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-09-12T01:57:03.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:02 smithi016 ceph-mon[30947]: from='osd.6 [v2:172.21.15.16:6818/902466142,v1:172.21.15.16:6819/902466142]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-12T01:57:03.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:02 smithi016 ceph-mon[30947]: osdmap e41: 7 total, 6 up, 7 in 2024-09-12T01:57:03.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:02 smithi016 ceph-mon[30947]: from='osd.6 [v2:172.21.15.16:6818/902466142,v1:172.21.15.16:6819/902466142]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-09-12T01:57:03.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:02 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T01:57:03.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:02 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:57:03.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:02 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:57:03.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:02 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:57:03.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:02 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:57:03.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:02 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:57:03.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:02 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:57:03.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:02 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:57:03.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:02 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:57:03.435 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:02 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:57:03.435 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:02 smithi016 ceph-mon[30947]: from='osd.6 [v2:172.21.15.16:6818/902466142,v1:172.21.15.16:6819/902466142]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-09-12T01:57:03.934 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 12 01:57:03 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-6[58414]: 2024-09-12T01:57:03.663+0000 7fa4afa32640 -1 osd.6 0 waiting for initial osdmap 2024-09-12T01:57:03.934 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 12 01:57:03 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-6[58414]: 2024-09-12T01:57:03.672+0000 7fa4a8064640 -1 osd.6 42 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T01:57:04.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:03 smithi184 ceph-mon[35593]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-12T01:57:04.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:03 smithi184 ceph-mon[35593]: Detected new or changed devices on smithi016 2024-09-12T01:57:04.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:03 smithi184 ceph-mon[35593]: Adjusting osd_memory_target on smithi016 to 5697M 2024-09-12T01:57:04.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:03 smithi184 ceph-mon[35593]: osdmap e42: 7 total, 6 up, 7 in 2024-09-12T01:57:04.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:03 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T01:57:04.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:03 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T01:57:04.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:03 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T01:57:04.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:03 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T01:57:04.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:03 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:57:04.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:03 smithi184 ceph-mon[35593]: from='osd.6 [v2:172.21.15.16:6818/902466142,v1:172.21.15.16:6819/902466142]' entity='osd.6' 2024-09-12T01:57:04.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:03 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T01:57:04.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:03 smithi033 ceph-mon[35332]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-12T01:57:04.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:03 smithi033 ceph-mon[35332]: Detected new or changed devices on smithi016 2024-09-12T01:57:04.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:03 smithi033 ceph-mon[35332]: Adjusting osd_memory_target on smithi016 to 5697M 2024-09-12T01:57:04.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:03 smithi033 ceph-mon[35332]: osdmap e42: 7 total, 6 up, 7 in 2024-09-12T01:57:04.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:03 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T01:57:04.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:03 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T01:57:04.420 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:03 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T01:57:04.420 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:03 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T01:57:04.420 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:03 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:57:04.420 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:03 smithi033 ceph-mon[35332]: from='osd.6 [v2:172.21.15.16:6818/902466142,v1:172.21.15.16:6819/902466142]' entity='osd.6' 2024-09-12T01:57:04.420 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:03 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T01:57:04.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:03 smithi016 ceph-mon[30947]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-12T01:57:04.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:03 smithi016 ceph-mon[30947]: Detected new or changed devices on smithi016 2024-09-12T01:57:04.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:03 smithi016 ceph-mon[30947]: Adjusting osd_memory_target on smithi016 to 5697M 2024-09-12T01:57:04.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:03 smithi016 ceph-mon[30947]: osdmap e42: 7 total, 6 up, 7 in 2024-09-12T01:57:04.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:03 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T01:57:04.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:03 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T01:57:04.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:03 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T01:57:04.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:03 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T01:57:04.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:03 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:57:04.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:03 smithi016 ceph-mon[30947]: from='osd.6 [v2:172.21.15.16:6818/902466142,v1:172.21.15.16:6819/902466142]' entity='osd.6' 2024-09-12T01:57:04.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:03 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T01:57:05.079 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:04 smithi033 ceph-mon[35332]: purged_snaps scrub starts 2024-09-12T01:57:05.079 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:04 smithi033 ceph-mon[35332]: purged_snaps scrub ok 2024-09-12T01:57:05.079 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:04 smithi033 ceph-mon[35332]: from='client.14400 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:57:05.080 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:04 smithi033 ceph-mon[35332]: osd.6 [v2:172.21.15.16:6818/902466142,v1:172.21.15.16:6819/902466142] boot 2024-09-12T01:57:05.080 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:04 smithi033 ceph-mon[35332]: osdmap e43: 7 total, 7 up, 7 in 2024-09-12T01:57:05.080 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:04 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T01:57:05.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:04 smithi184 ceph-mon[35593]: purged_snaps scrub starts 2024-09-12T01:57:05.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:04 smithi184 ceph-mon[35593]: purged_snaps scrub ok 2024-09-12T01:57:05.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:04 smithi184 ceph-mon[35593]: from='client.14400 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:57:05.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:04 smithi184 ceph-mon[35593]: osd.6 [v2:172.21.15.16:6818/902466142,v1:172.21.15.16:6819/902466142] boot 2024-09-12T01:57:05.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:04 smithi184 ceph-mon[35593]: osdmap e43: 7 total, 7 up, 7 in 2024-09-12T01:57:05.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:04 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T01:57:05.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:04 smithi016 ceph-mon[30947]: purged_snaps scrub starts 2024-09-12T01:57:05.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:04 smithi016 ceph-mon[30947]: purged_snaps scrub ok 2024-09-12T01:57:05.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:04 smithi016 ceph-mon[30947]: from='client.14400 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:57:05.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:04 smithi016 ceph-mon[30947]: osd.6 [v2:172.21.15.16:6818/902466142,v1:172.21.15.16:6819/902466142] boot 2024-09-12T01:57:05.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:04 smithi016 ceph-mon[30947]: osdmap e43: 7 total, 7 up, 7 in 2024-09-12T01:57:05.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:04 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T01:57:06.340 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:05 smithi033 ceph-mon[35332]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-12T01:57:06.340 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:05 smithi033 ceph-mon[35332]: from='client.? 172.21.15.33:0/4170931148' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4ddbe011-34c1-4b3f-a8eb-577422fb5426"}]: dispatch 2024-09-12T01:57:06.340 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:05 smithi033 ceph-mon[35332]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4ddbe011-34c1-4b3f-a8eb-577422fb5426"}]: dispatch 2024-09-12T01:57:06.341 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:05 smithi033 ceph-mon[35332]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4ddbe011-34c1-4b3f-a8eb-577422fb5426"}]': finished 2024-09-12T01:57:06.341 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:05 smithi033 ceph-mon[35332]: osdmap e44: 8 total, 7 up, 8 in 2024-09-12T01:57:06.341 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:05 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T01:57:06.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:05 smithi184 ceph-mon[35593]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-12T01:57:06.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:05 smithi184 ceph-mon[35593]: from='client.? 172.21.15.33:0/4170931148' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4ddbe011-34c1-4b3f-a8eb-577422fb5426"}]: dispatch 2024-09-12T01:57:06.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:05 smithi184 ceph-mon[35593]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4ddbe011-34c1-4b3f-a8eb-577422fb5426"}]: dispatch 2024-09-12T01:57:06.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:05 smithi184 ceph-mon[35593]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4ddbe011-34c1-4b3f-a8eb-577422fb5426"}]': finished 2024-09-12T01:57:06.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:05 smithi184 ceph-mon[35593]: osdmap e44: 8 total, 7 up, 8 in 2024-09-12T01:57:06.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:05 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T01:57:06.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:05 smithi016 ceph-mon[30947]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-12T01:57:06.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:05 smithi016 ceph-mon[30947]: from='client.? 172.21.15.33:0/4170931148' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4ddbe011-34c1-4b3f-a8eb-577422fb5426"}]: dispatch 2024-09-12T01:57:06.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:05 smithi016 ceph-mon[30947]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4ddbe011-34c1-4b3f-a8eb-577422fb5426"}]: dispatch 2024-09-12T01:57:06.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:05 smithi016 ceph-mon[30947]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4ddbe011-34c1-4b3f-a8eb-577422fb5426"}]': finished 2024-09-12T01:57:06.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:05 smithi016 ceph-mon[30947]: osdmap e44: 8 total, 7 up, 8 in 2024-09-12T01:57:06.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:05 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T01:57:07.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:07 smithi184 ceph-mon[35593]: from='client.? 172.21.15.33:0/3315521301' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T01:57:07.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:07 smithi033 ceph-mon[35332]: from='client.? 172.21.15.33:0/3315521301' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T01:57:07.436 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:07 smithi016 ceph-mon[30947]: from='client.? 172.21.15.33:0/3315521301' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T01:57:08.335 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:08 smithi016 ceph-mon[30947]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-09-12T01:57:08.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:08 smithi184 ceph-mon[35593]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-09-12T01:57:08.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:08 smithi033 ceph-mon[35332]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-09-12T01:57:10.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:10 smithi184 ceph-mon[35593]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-09-12T01:57:10.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:10 smithi033 ceph-mon[35332]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-09-12T01:57:10.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:10 smithi016 ceph-mon[30947]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-09-12T01:57:12.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:12 smithi184 ceph-mon[35593]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-12T01:57:12.384 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:12 smithi033 ceph-mon[35332]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-12T01:57:12.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:12 smithi016 ceph-mon[30947]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-12T01:57:14.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:14 smithi184 ceph-mon[35593]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-12T01:57:14.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:14 smithi033 ceph-mon[35332]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-12T01:57:14.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:14 smithi016 ceph-mon[30947]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-12T01:57:15.310 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:15 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-12T01:57:15.311 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:15 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:57:15.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:15 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-12T01:57:15.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:15 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:57:15.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:15 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-12T01:57:15.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:15 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:57:16.350 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:16 smithi033 ceph-mon[35332]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-12T01:57:16.350 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:16 smithi033 ceph-mon[35332]: Deploying daemon osd.7 on smithi033 2024-09-12T01:57:16.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:16 smithi184 ceph-mon[35593]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-12T01:57:16.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:16 smithi184 ceph-mon[35593]: Deploying daemon osd.7 on smithi033 2024-09-12T01:57:16.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:16 smithi016 ceph-mon[30947]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-12T01:57:16.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:16 smithi016 ceph-mon[30947]: Deploying daemon osd.7 on smithi033 2024-09-12T01:57:18.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:18 smithi033 ceph-mon[35332]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-12T01:57:18.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:18 smithi184 ceph-mon[35593]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-12T01:57:18.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:18 smithi016 ceph-mon[30947]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-12T01:57:20.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:20 smithi184 ceph-mon[35593]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-12T01:57:20.384 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:20 smithi033 ceph-mon[35332]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-12T01:57:20.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:20 smithi016 ceph-mon[30947]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-12T01:57:21.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:21 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:57:21.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:21 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:57:21.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:21 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:57:21.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:21 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:57:21.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:21 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:57:21.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:21 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:57:21.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:21 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:57:21.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:21 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:57:21.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:21 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:57:22.477 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:22 smithi033 ceph-mon[35332]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-12T01:57:22.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:22 smithi184 ceph-mon[35593]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-12T01:57:22.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:22 smithi016 ceph-mon[30947]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-12T01:57:23.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:57:23.704+0000 7f75f57fa640 1 -- 172.21.15.33:0/2922431527 <== mgr.14150 v2:172.21.15.16:6800/3887444820 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f75f806a6f0 con 0x7f75e00a2bb0 2024-09-12T01:57:23.705 INFO:teuthology.orchestra.run.smithi033.stdout:Created osd(s) 7 on host 'smithi033' 2024-09-12T01:57:23.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:57:23.706+0000 7f75fe8fa640 1 -- 172.21.15.33:0/2922431527 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f75e00a2bb0 msgr2=0x7f75e00a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:57:23.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:57:23.706+0000 7f75fe8fa640 1 --2- 172.21.15.33:0/2922431527 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f75e00a2bb0 0x7f75e00a5070 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f75f81994e0 tx=0x7f75ec0091c0 comp rx=0 tx=0).stop 2024-09-12T01:57:23.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:57:23.706+0000 7f75fe8fa640 1 -- 172.21.15.33:0/2922431527 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75f8100ad0 msgr2=0x7f75f8197fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:57:23.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:57:23.706+0000 7f75fe8fa640 1 --2- 172.21.15.33:0/2922431527 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75f8100ad0 0x7f75f8197fc0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f75e8009230 tx=0x7f75e8031b60 comp rx=0 tx=0).stop 2024-09-12T01:57:23.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:57:23.707+0000 7f75fe8fa640 1 -- 172.21.15.33:0/2922431527 shutdown_connections 2024-09-12T01:57:23.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:57:23.707+0000 7f75fe8fa640 1 --2- 172.21.15.33:0/2922431527 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f75e00a2bb0 0x7f75e00a5070 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:57:23.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:57:23.707+0000 7f75fe8fa640 1 --2- 172.21.15.33:0/2922431527 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f75f8102690 0x7f75f819f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:57:23.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:57:23.707+0000 7f75fe8fa640 1 --2- 172.21.15.33:0/2922431527 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f75f8101cd0 0x7f75f8198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:57:23.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:57:23.707+0000 7f75fe8fa640 1 --2- 172.21.15.33:0/2922431527 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75f8100ad0 0x7f75f8197fc0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:57:23.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:57:23.707+0000 7f75fe8fa640 1 -- 172.21.15.33:0/2922431527 >> 172.21.15.33:0/2922431527 conn(0x7f75f80fc280 msgr2=0x7f75f8069fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:57:23.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:57:23.707+0000 7f75fe8fa640 1 -- 172.21.15.33:0/2922431527 shutdown_connections 2024-09-12T01:57:23.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:57:23.707+0000 7f75fe8fa640 1 -- 172.21.15.33:0/2922431527 wait complete. 2024-09-12T01:57:23.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:23 smithi033 ceph-mon[35332]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-12T01:57:23.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:23 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:57:23.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:23 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:57:23.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:23 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:57:23.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:23 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:57:23.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:23 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:57:23.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:23 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:57:23.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:23 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:57:23.920 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:23 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:57:24.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:23 smithi184 ceph-mon[35593]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-12T01:57:24.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:23 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:57:24.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:23 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:57:24.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:23 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:57:24.128 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:23 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:57:24.128 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:23 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:57:24.128 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:23 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:57:24.128 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:23 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:57:24.128 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:23 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:57:24.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:23 smithi016 ceph-mon[30947]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-12T01:57:24.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:23 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:57:24.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:23 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:57:24.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:23 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:57:24.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:23 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:57:24.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:23 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:57:24.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:23 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:57:24.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:23 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:57:24.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:23 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:57:24.279 DEBUG:teuthology.orchestra.run.smithi033:osd.7> sudo journalctl -f -n 0 -u ceph-879f3b56-70a9-11ef-bcea-c7b262605968@osd.7.service 2024-09-12T01:57:24.282 INFO:tasks.cephadm:Deploying osd.8 on smithi184 with /dev/vg_nvme/lv_2... 2024-09-12T01:57:24.282 DEBUG:teuthology.orchestra.run.smithi184:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-12T01:57:24.538 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T01:57:25.936 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:25 smithi184 ceph-mon[35593]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-12T01:57:26.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:25 smithi033 ceph-mon[35332]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-12T01:57:26.183 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:25 smithi016 ceph-mon[30947]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-12T01:57:26.669 INFO:journalctl@ceph.osd.7.smithi033.stdout:Sep 12 01:57:26 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-7[53039]: 2024-09-12T01:57:26.477+0000 7f5a3f6a1640 -1 osd.7 0 log_to_monitors true 2024-09-12T01:57:27.068 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:26 smithi184 ceph-mon[35593]: from='osd.7 [v2:172.21.15.33:6816/4186032630,v1:172.21.15.33:6817/4186032630]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-12T01:57:27.068 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:26 smithi184 ceph-mon[35593]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-12T01:57:27.071 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-12T01:57:27.096 DEBUG:teuthology.orchestra.run.smithi184:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi184:vg_nvme/lv_2 2024-09-12T01:57:27.114 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:26 smithi033 ceph-mon[35332]: from='osd.7 [v2:172.21.15.33:6816/4186032630,v1:172.21.15.33:6817/4186032630]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-12T01:57:27.114 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:26 smithi033 ceph-mon[35332]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-12T01:57:27.183 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:26 smithi016 ceph-mon[30947]: from='osd.7 [v2:172.21.15.33:6816/4186032630,v1:172.21.15.33:6817/4186032630]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-12T01:57:27.183 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:26 smithi016 ceph-mon[30947]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-12T01:57:27.349 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T01:57:27.767 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:57:27.764+0000 7f8052227640 1 -- 172.21.15.184:0/1188240645 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f804c1026d0 msgr2=0x7f804c108f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:57:27.767 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:57:27.764+0000 7f8052227640 1 --2- 172.21.15.184:0/1188240645 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f804c1026d0 0x7f804c108f60 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f8044009ca0 tx=0x7f804402a8c0 comp rx=0 tx=0).stop 2024-09-12T01:57:27.767 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:57:27.764+0000 7f8052227640 1 -- 172.21.15.184:0/1188240645 shutdown_connections 2024-09-12T01:57:27.767 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:57:27.764+0000 7f8052227640 1 --2- 172.21.15.184:0/1188240645 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f804c1026d0 0x7f804c108f60 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:57:27.767 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:57:27.764+0000 7f8052227640 1 --2- 172.21.15.184:0/1188240645 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f804c101d10 0x7f804c102190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:57:27.767 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:57:27.764+0000 7f8052227640 1 --2- 172.21.15.184:0/1188240645 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f804c100b10 0x7f804c100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:57:27.767 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:57:27.764+0000 7f8052227640 1 -- 172.21.15.184:0/1188240645 >> 172.21.15.184:0/1188240645 conn(0x7f804c0fc2a0 msgr2=0x7f804c0fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:57:27.767 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:57:27.764+0000 7f8052227640 1 -- 172.21.15.184:0/1188240645 shutdown_connections 2024-09-12T01:57:27.768 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:57:27.764+0000 7f8052227640 1 -- 172.21.15.184:0/1188240645 wait complete. 2024-09-12T01:57:27.768 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:57:27.765+0000 7f8052227640 1 Processor -- start 2024-09-12T01:57:27.768 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:57:27.765+0000 7f8052227640 1 -- start start 2024-09-12T01:57:27.768 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:57:27.765+0000 7f8052227640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f804c100b10 0x7f804c06b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:57:27.768 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:57:27.765+0000 7f8052227640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f804c101d10 0x7f804c19eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:57:27.768 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:57:27.765+0000 7f8052227640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f804c1026d0 0x7f804c1a1540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:57:27.768 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:57:27.765+0000 7f8052227640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f804c10c770 con 0x7f804c101d10 2024-09-12T01:57:27.768 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:57:27.765+0000 7f8052227640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f804c10c5f0 con 0x7f804c1026d0 2024-09-12T01:57:27.768 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:57:27.765+0000 7f8052227640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f804c10c8f0 con 0x7f804c100b10 2024-09-12T01:57:27.768 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:57:27.765+0000 7f804b7fe640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f804c100b10 0x7f804c06b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:57:27.768 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:57:27.766+0000 7f804b7fe640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f804c100b10 0x7f804c06b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.184:46250/0 (socket says 172.21.15.184:46250) 2024-09-12T01:57:27.768 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:57:27.766+0000 7f804b7fe640 1 -- 172.21.15.184:0/2407805574 learned_addr learned my addr 172.21.15.184:0/2407805574 (peer_addr_for_me v2:172.21.15.184:0/0) 2024-09-12T01:57:27.768 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:57:27.766+0000 7f804affd640 1 --2- 172.21.15.184:0/2407805574 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f804c101d10 0x7f804c19eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:57:27.769 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:57:27.766+0000 7f804bfff640 1 --2- 172.21.15.184:0/2407805574 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f804c1026d0 0x7f804c1a1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:57:27.769 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:57:27.766+0000 7f804b7fe640 1 -- 172.21.15.184:0/2407805574 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f804c1026d0 msgr2=0x7f804c1a1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:57:27.769 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:57:27.766+0000 7f804b7fe640 1 --2- 172.21.15.184:0/2407805574 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f804c1026d0 0x7f804c1a1540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:57:27.769 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:57:27.766+0000 7f804b7fe640 1 -- 172.21.15.184:0/2407805574 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f804c101d10 msgr2=0x7f804c19eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:57:27.769 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:57:27.766+0000 7f804b7fe640 1 --2- 172.21.15.184:0/2407805574 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f804c101d10 0x7f804c19eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:57:27.769 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:57:27.766+0000 7f804b7fe640 1 -- 172.21.15.184:0/2407805574 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f804c1a1d30 con 0x7f804c100b10 2024-09-12T01:57:27.769 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:57:27.766+0000 7f804b7fe640 1 --2- 172.21.15.184:0/2407805574 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f804c100b10 0x7f804c06b2f0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f803c00d950 tx=0x7f803c00de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:57:27.769 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:57:27.766+0000 7f8048ff9640 1 -- 172.21.15.184:0/2407805574 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f803c017020 con 0x7f804c100b10 2024-09-12T01:57:27.769 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:57:27.766+0000 7f8048ff9640 1 -- 172.21.15.184:0/2407805574 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f803c009c30 con 0x7f804c100b10 2024-09-12T01:57:27.769 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:57:27.766+0000 7f8052227640 1 -- 172.21.15.184:0/2407805574 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f804c1a1fc0 con 0x7f804c100b10 2024-09-12T01:57:27.769 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:57:27.766+0000 7f8048ff9640 1 -- 172.21.15.184:0/2407805574 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f803c00f740 con 0x7f804c100b10 2024-09-12T01:57:27.769 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:57:27.766+0000 7f8052227640 1 -- 172.21.15.184:0/2407805574 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f804c1a2470 con 0x7f804c100b10 2024-09-12T01:57:27.770 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:57:27.767+0000 7f8052227640 1 -- 172.21.15.184:0/2407805574 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f804c107eb0 con 0x7f804c100b10 2024-09-12T01:57:27.770 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:57:27.769+0000 7f8048ff9640 1 -- 172.21.15.184:0/2407805574 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f803c01a020 con 0x7f804c100b10 2024-09-12T01:57:27.771 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:57:27.769+0000 7f8048ff9640 1 --2- 172.21.15.184:0/2407805574 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f80380a2bb0 0x7f80380a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:57:27.771 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:57:27.770+0000 7f8048ff9640 1 -- 172.21.15.184:0/2407805574 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5117+0+0 (secure 0 0 0) 0x7f803c0ce010 con 0x7f804c100b10 2024-09-12T01:57:27.771 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:57:27.770+0000 7f804affd640 1 --2- 172.21.15.184:0/2407805574 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f80380a2bb0 0x7f80380a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:57:27.771 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:57:27.770+0000 7f804affd640 1 --2- 172.21.15.184:0/2407805574 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f80380a2bb0 0x7f80380a5070 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f804c06b870 tx=0x7f803400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:57:27.773 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:57:27.773+0000 7f8048ff9640 1 -- 172.21.15.184:0/2407805574 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f803c021070 con 0x7f804c100b10 2024-09-12T01:57:27.949 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:57:27.948+0000 7f8052227640 1 -- 172.21.15.184:0/2407805574 --> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi184:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f804c062630 con 0x7f80380a2bb0 2024-09-12T01:57:28.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:27 smithi184 ceph-mon[35593]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-12T01:57:28.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:27 smithi184 ceph-mon[35593]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-12T01:57:28.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:27 smithi184 ceph-mon[35593]: osdmap e45: 8 total, 7 up, 8 in 2024-09-12T01:57:28.128 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:27 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T01:57:28.128 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:27 smithi184 ceph-mon[35593]: from='osd.7 [v2:172.21.15.33:6816/4186032630,v1:172.21.15.33:6817/4186032630]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-09-12T01:57:28.128 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:27 smithi184 ceph-mon[35593]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-09-12T01:57:28.128 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:27 smithi184 ceph-mon[35593]: Detected new or changed devices on smithi033 2024-09-12T01:57:28.128 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:27 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:57:28.128 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:27 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:57:28.128 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:27 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:57:28.128 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:27 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:57:28.128 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:27 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:57:28.128 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:27 smithi184 ceph-mon[35593]: Adjusting osd_memory_target on smithi033 to 5697M 2024-09-12T01:57:28.128 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:27 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:57:28.128 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:27 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:57:28.129 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:27 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:57:28.129 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:27 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:57:28.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:27 smithi033 ceph-mon[35332]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-12T01:57:28.170 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:27 smithi033 ceph-mon[35332]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-12T01:57:28.170 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:27 smithi033 ceph-mon[35332]: osdmap e45: 8 total, 7 up, 8 in 2024-09-12T01:57:28.170 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:27 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T01:57:28.170 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:27 smithi033 ceph-mon[35332]: from='osd.7 [v2:172.21.15.33:6816/4186032630,v1:172.21.15.33:6817/4186032630]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-09-12T01:57:28.170 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:27 smithi033 ceph-mon[35332]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-09-12T01:57:28.170 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:27 smithi033 ceph-mon[35332]: Detected new or changed devices on smithi033 2024-09-12T01:57:28.170 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:27 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:57:28.170 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:27 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:57:28.171 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:27 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:57:28.171 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:27 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:57:28.171 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:27 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:57:28.171 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:27 smithi033 ceph-mon[35332]: Adjusting osd_memory_target on smithi033 to 5697M 2024-09-12T01:57:28.171 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:27 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:57:28.171 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:27 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:57:28.171 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:27 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:57:28.172 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:27 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:57:28.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:27 smithi016 ceph-mon[30947]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-12T01:57:28.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:27 smithi016 ceph-mon[30947]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-12T01:57:28.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:27 smithi016 ceph-mon[30947]: osdmap e45: 8 total, 7 up, 8 in 2024-09-12T01:57:28.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:27 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T01:57:28.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:27 smithi016 ceph-mon[30947]: from='osd.7 [v2:172.21.15.33:6816/4186032630,v1:172.21.15.33:6817/4186032630]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-09-12T01:57:28.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:27 smithi016 ceph-mon[30947]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-09-12T01:57:28.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:27 smithi016 ceph-mon[30947]: Detected new or changed devices on smithi033 2024-09-12T01:57:28.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:27 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:57:28.185 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:27 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:57:28.185 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:27 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:57:28.185 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:27 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:57:28.185 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:27 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:57:28.185 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:27 smithi016 ceph-mon[30947]: Adjusting osd_memory_target on smithi033 to 5697M 2024-09-12T01:57:28.185 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:27 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:57:28.185 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:27 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:57:28.185 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:27 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:57:28.185 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:27 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:57:28.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:28 smithi033 ceph-mon[35332]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]': finished 2024-09-12T01:57:28.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:28 smithi033 ceph-mon[35332]: osdmap e46: 8 total, 7 up, 8 in 2024-09-12T01:57:28.920 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:28 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T01:57:28.920 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:28 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T01:57:28.920 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:28 smithi033 ceph-mon[35332]: osdmap e47: 8 total, 7 up, 8 in 2024-09-12T01:57:28.920 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:28 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T01:57:28.920 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:28 smithi033 ceph-mon[35332]: from='client.24307 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi184:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:57:28.920 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:28 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T01:57:28.920 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:28 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T01:57:28.920 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:28 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:57:28.920 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:28 smithi033 ceph-mon[35332]: from='osd.7 ' entity='osd.7' 2024-09-12T01:57:28.920 INFO:journalctl@ceph.osd.7.smithi033.stdout:Sep 12 01:57:28 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-7[53039]: 2024-09-12T01:57:28.485+0000 7f5a3be38640 -1 osd.7 0 waiting for initial osdmap 2024-09-12T01:57:28.920 INFO:journalctl@ceph.osd.7.smithi033.stdout:Sep 12 01:57:28 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-7[53039]: 2024-09-12T01:57:28.496+0000 7f5a33433640 -1 osd.7 47 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T01:57:29.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:28 smithi184 ceph-mon[35593]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]': finished 2024-09-12T01:57:29.128 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:28 smithi184 ceph-mon[35593]: osdmap e46: 8 total, 7 up, 8 in 2024-09-12T01:57:29.128 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:28 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T01:57:29.128 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:28 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T01:57:29.128 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:28 smithi184 ceph-mon[35593]: osdmap e47: 8 total, 7 up, 8 in 2024-09-12T01:57:29.128 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:28 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T01:57:29.128 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:28 smithi184 ceph-mon[35593]: from='client.24307 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi184:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:57:29.128 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:28 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T01:57:29.128 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:28 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T01:57:29.128 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:28 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:57:29.128 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:28 smithi184 ceph-mon[35593]: from='osd.7 ' entity='osd.7' 2024-09-12T01:57:29.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:28 smithi016 ceph-mon[30947]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]': finished 2024-09-12T01:57:29.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:28 smithi016 ceph-mon[30947]: osdmap e46: 8 total, 7 up, 8 in 2024-09-12T01:57:29.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:28 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T01:57:29.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:28 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T01:57:29.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:28 smithi016 ceph-mon[30947]: osdmap e47: 8 total, 7 up, 8 in 2024-09-12T01:57:29.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:28 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T01:57:29.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:28 smithi016 ceph-mon[30947]: from='client.24307 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi184:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:57:29.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:28 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T01:57:29.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:28 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T01:57:29.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:28 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:57:29.185 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:28 smithi016 ceph-mon[30947]: from='osd.7 ' entity='osd.7' 2024-09-12T01:57:29.864 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:29 smithi184 ceph-mon[35593]: purged_snaps scrub starts 2024-09-12T01:57:29.864 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:29 smithi184 ceph-mon[35593]: purged_snaps scrub ok 2024-09-12T01:57:29.864 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:29 smithi184 ceph-mon[35593]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-12T01:57:29.864 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:29 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T01:57:29.865 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:29 smithi184 ceph-mon[35593]: osd.7 [v2:172.21.15.33:6816/4186032630,v1:172.21.15.33:6817/4186032630] boot 2024-09-12T01:57:29.865 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:29 smithi184 ceph-mon[35593]: osdmap e48: 8 total, 8 up, 8 in 2024-09-12T01:57:29.865 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:29 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T01:57:30.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:29 smithi033 ceph-mon[35332]: purged_snaps scrub starts 2024-09-12T01:57:30.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:29 smithi033 ceph-mon[35332]: purged_snaps scrub ok 2024-09-12T01:57:30.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:29 smithi033 ceph-mon[35332]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-12T01:57:30.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:29 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T01:57:30.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:29 smithi033 ceph-mon[35332]: osd.7 [v2:172.21.15.33:6816/4186032630,v1:172.21.15.33:6817/4186032630] boot 2024-09-12T01:57:30.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:29 smithi033 ceph-mon[35332]: osdmap e48: 8 total, 8 up, 8 in 2024-09-12T01:57:30.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:29 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T01:57:30.183 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:29 smithi016 ceph-mon[30947]: purged_snaps scrub starts 2024-09-12T01:57:30.183 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:29 smithi016 ceph-mon[30947]: purged_snaps scrub ok 2024-09-12T01:57:30.183 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:29 smithi016 ceph-mon[30947]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-12T01:57:30.183 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:29 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T01:57:30.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:29 smithi016 ceph-mon[30947]: osd.7 [v2:172.21.15.33:6816/4186032630,v1:172.21.15.33:6817/4186032630] boot 2024-09-12T01:57:30.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:29 smithi016 ceph-mon[30947]: osdmap e48: 8 total, 8 up, 8 in 2024-09-12T01:57:30.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:29 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T01:57:30.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:30 smithi184 ceph-mon[35593]: osdmap e49: 8 total, 8 up, 8 in 2024-09-12T01:57:30.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:30 smithi184 ceph-mon[35593]: from='client.? 172.21.15.184:0/3941752543' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dca5bfb8-ac6a-47ce-905c-be7fcaf0b5ec"}]: dispatch 2024-09-12T01:57:30.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:30 smithi184 ceph-mon[35593]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dca5bfb8-ac6a-47ce-905c-be7fcaf0b5ec"}]: dispatch 2024-09-12T01:57:30.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:30 smithi184 ceph-mon[35593]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "dca5bfb8-ac6a-47ce-905c-be7fcaf0b5ec"}]': finished 2024-09-12T01:57:30.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:30 smithi184 ceph-mon[35593]: osdmap e50: 9 total, 8 up, 9 in 2024-09-12T01:57:30.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:30 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T01:57:31.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:30 smithi033 ceph-mon[35332]: osdmap e49: 8 total, 8 up, 8 in 2024-09-12T01:57:31.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:30 smithi033 ceph-mon[35332]: from='client.? 172.21.15.184:0/3941752543' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dca5bfb8-ac6a-47ce-905c-be7fcaf0b5ec"}]: dispatch 2024-09-12T01:57:31.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:30 smithi033 ceph-mon[35332]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dca5bfb8-ac6a-47ce-905c-be7fcaf0b5ec"}]: dispatch 2024-09-12T01:57:31.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:30 smithi033 ceph-mon[35332]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "dca5bfb8-ac6a-47ce-905c-be7fcaf0b5ec"}]': finished 2024-09-12T01:57:31.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:30 smithi033 ceph-mon[35332]: osdmap e50: 9 total, 8 up, 9 in 2024-09-12T01:57:31.170 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:30 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T01:57:31.183 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:30 smithi016 ceph-mon[30947]: osdmap e49: 8 total, 8 up, 8 in 2024-09-12T01:57:31.183 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:30 smithi016 ceph-mon[30947]: from='client.? 172.21.15.184:0/3941752543' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dca5bfb8-ac6a-47ce-905c-be7fcaf0b5ec"}]: dispatch 2024-09-12T01:57:31.183 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:30 smithi016 ceph-mon[30947]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dca5bfb8-ac6a-47ce-905c-be7fcaf0b5ec"}]: dispatch 2024-09-12T01:57:31.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:30 smithi016 ceph-mon[30947]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "dca5bfb8-ac6a-47ce-905c-be7fcaf0b5ec"}]': finished 2024-09-12T01:57:31.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:30 smithi016 ceph-mon[30947]: osdmap e50: 9 total, 8 up, 9 in 2024-09-12T01:57:31.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:30 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T01:57:32.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:31 smithi184 ceph-mon[35593]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-12T01:57:32.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:31 smithi184 ceph-mon[35593]: from='client.? 172.21.15.184:0/2819085447' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T01:57:32.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:31 smithi184 ceph-mon[35593]: osdmap e51: 9 total, 8 up, 9 in 2024-09-12T01:57:32.128 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:31 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T01:57:32.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:31 smithi033 ceph-mon[35332]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-12T01:57:32.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:31 smithi033 ceph-mon[35332]: from='client.? 172.21.15.184:0/2819085447' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T01:57:32.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:31 smithi033 ceph-mon[35332]: osdmap e51: 9 total, 8 up, 9 in 2024-09-12T01:57:32.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:31 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T01:57:32.183 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:31 smithi016 ceph-mon[30947]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-12T01:57:32.183 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:31 smithi016 ceph-mon[30947]: from='client.? 172.21.15.184:0/2819085447' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T01:57:32.183 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:31 smithi016 ceph-mon[30947]: osdmap e51: 9 total, 8 up, 9 in 2024-09-12T01:57:32.183 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:31 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T01:57:33.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:33 smithi033 ceph-mon[35332]: pgmap v173: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-12T01:57:33.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:33 smithi033 ceph-mon[35332]: osdmap e52: 9 total, 8 up, 9 in 2024-09-12T01:57:33.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:33 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T01:57:33.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:33 smithi016 ceph-mon[30947]: pgmap v173: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-12T01:57:33.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:33 smithi016 ceph-mon[30947]: osdmap e52: 9 total, 8 up, 9 in 2024-09-12T01:57:33.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:33 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T01:57:33.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:33 smithi184 ceph-mon[35593]: pgmap v173: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-12T01:57:33.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:33 smithi184 ceph-mon[35593]: osdmap e52: 9 total, 8 up, 9 in 2024-09-12T01:57:33.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:33 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T01:57:35.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:35 smithi033 ceph-mon[35332]: pgmap v175: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-12T01:57:35.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:35 smithi016 ceph-mon[30947]: pgmap v175: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-12T01:57:35.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:35 smithi184 ceph-mon[35593]: pgmap v175: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-12T01:57:37.659 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:37 smithi184 ceph-mon[35593]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-12T01:57:37.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:37 smithi033 ceph-mon[35332]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-12T01:57:37.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:37 smithi016 ceph-mon[30947]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-12T01:57:39.602 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:39 smithi184 ceph-mon[35593]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-12T01:57:39.602 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:39 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-12T01:57:39.602 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:39 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:57:39.602 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:39 smithi184 ceph-mon[35593]: Deploying daemon osd.8 on smithi184 2024-09-12T01:57:39.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:39 smithi033 ceph-mon[35332]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-12T01:57:39.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:39 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-12T01:57:39.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:39 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:57:39.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:39 smithi033 ceph-mon[35332]: Deploying daemon osd.8 on smithi184 2024-09-12T01:57:39.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:39 smithi016 ceph-mon[30947]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-12T01:57:39.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:39 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-12T01:57:39.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:39 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:57:39.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:39 smithi016 ceph-mon[30947]: Deploying daemon osd.8 on smithi184 2024-09-12T01:57:41.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:41 smithi184 ceph-mon[35593]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-12T01:57:41.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:41 smithi033 ceph-mon[35332]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-12T01:57:41.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:41 smithi016 ceph-mon[30947]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-12T01:57:43.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:43 smithi033 ceph-mon[35332]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-12T01:57:43.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:43 smithi016 ceph-mon[30947]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-12T01:57:43.731 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:43 smithi184 ceph-mon[35593]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-12T01:57:45.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:45 smithi033 ceph-mon[35332]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-12T01:57:45.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:45 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:57:45.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:45 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:57:45.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:45 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:57:45.670 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:45 smithi184 ceph-mon[35593]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-12T01:57:45.670 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:45 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:57:45.670 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:45 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:57:45.670 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:45 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:57:45.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:45 smithi016 ceph-mon[30947]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-12T01:57:45.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:45 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:57:45.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:45 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:57:45.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:45 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:57:47.541 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:47 smithi184 ceph-mon[35593]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-12T01:57:47.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:47 smithi033 ceph-mon[35332]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-12T01:57:47.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:47 smithi016 ceph-mon[30947]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-12T01:57:48.247 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:57:48.247+0000 7f8048ff9640 1 -- 172.21.15.184:0/2407805574 <== mgr.14150 v2:172.21.15.16:6800/3887444820 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f804c062630 con 0x7f80380a2bb0 2024-09-12T01:57:48.247 INFO:teuthology.orchestra.run.smithi184.stdout:Created osd(s) 8 on host 'smithi184' 2024-09-12T01:57:48.249 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:57:48.249+0000 7f8052227640 1 -- 172.21.15.184:0/2407805574 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f80380a2bb0 msgr2=0x7f80380a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:57:48.249 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:57:48.249+0000 7f8052227640 1 --2- 172.21.15.184:0/2407805574 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f80380a2bb0 0x7f80380a5070 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f804c06b870 tx=0x7f803400b390 comp rx=0 tx=0).stop 2024-09-12T01:57:48.249 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:57:48.249+0000 7f8052227640 1 -- 172.21.15.184:0/2407805574 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f804c100b10 msgr2=0x7f804c06b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:57:48.249 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:57:48.249+0000 7f8052227640 1 --2- 172.21.15.184:0/2407805574 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f804c100b10 0x7f804c06b2f0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f803c00d950 tx=0x7f803c00de10 comp rx=0 tx=0).stop 2024-09-12T01:57:48.249 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:57:48.249+0000 7f8052227640 1 -- 172.21.15.184:0/2407805574 shutdown_connections 2024-09-12T01:57:48.250 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:57:48.249+0000 7f8052227640 1 --2- 172.21.15.184:0/2407805574 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f80380a2bb0 0x7f80380a5070 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:57:48.250 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:57:48.249+0000 7f8052227640 1 --2- 172.21.15.184:0/2407805574 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f804c1026d0 0x7f804c1a1540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:57:48.250 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:57:48.249+0000 7f8052227640 1 --2- 172.21.15.184:0/2407805574 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f804c101d10 0x7f804c19eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:57:48.250 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:57:48.249+0000 7f8052227640 1 --2- 172.21.15.184:0/2407805574 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f804c100b10 0x7f804c06b2f0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:57:48.250 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:57:48.249+0000 7f8052227640 1 -- 172.21.15.184:0/2407805574 >> 172.21.15.184:0/2407805574 conn(0x7f804c0fc2a0 msgr2=0x7f804c105080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:57:48.250 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:57:48.249+0000 7f8052227640 1 -- 172.21.15.184:0/2407805574 shutdown_connections 2024-09-12T01:57:48.250 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:57:48.249+0000 7f8052227640 1 -- 172.21.15.184:0/2407805574 wait complete. 2024-09-12T01:57:48.906 DEBUG:teuthology.orchestra.run.smithi184:osd.8> sudo journalctl -f -n 0 -u ceph-879f3b56-70a9-11ef-bcea-c7b262605968@osd.8.service 2024-09-12T01:57:48.909 INFO:tasks.cephadm:Deploying osd.9 on smithi016 with /dev/vg_nvme/lv_1... 2024-09-12T01:57:48.909 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-12T01:57:49.179 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:57:49.501 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:49 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:57:49.502 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:49 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:57:49.502 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:49 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:57:49.502 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:49 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:57:49.537 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:49 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:57:49.537 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:49 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:57:49.537 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:49 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:57:49.537 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:49 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:57:49.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:49 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:57:49.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:49 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:57:49.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:49 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:57:49.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:49 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:57:50.528 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:50 smithi016 ceph-mon[30947]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-12T01:57:50.541 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:50 smithi184 ceph-mon[35593]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-12T01:57:50.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:50 smithi033 ceph-mon[35332]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-12T01:57:51.127 INFO:journalctl@ceph.osd.8.smithi184.stdout:Sep 12 01:57:51 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-8[54277]: 2024-09-12T01:57:51.022+0000 7fe98e51d640 -1 osd.8 0 log_to_monitors true 2024-09-12T01:57:51.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:51 smithi016 ceph-mon[30947]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-12T01:57:51.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:51 smithi016 ceph-mon[30947]: from='osd.8 [v2:172.21.15.184:6816/781851781,v1:172.21.15.184:6817/781851781]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-12T01:57:51.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:51 smithi016 ceph-mon[30947]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-12T01:57:51.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:51 smithi184 ceph-mon[35593]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-12T01:57:51.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:51 smithi184 ceph-mon[35593]: from='osd.8 [v2:172.21.15.184:6816/781851781,v1:172.21.15.184:6817/781851781]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-12T01:57:51.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:51 smithi184 ceph-mon[35593]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-12T01:57:51.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:51 smithi033 ceph-mon[35332]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-12T01:57:51.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:51 smithi033 ceph-mon[35332]: from='osd.8 [v2:172.21.15.184:6816/781851781,v1:172.21.15.184:6817/781851781]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-12T01:57:51.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:51 smithi033 ceph-mon[35332]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-12T01:57:51.767 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:57:51.793 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi016:vg_nvme/lv_1 2024-09-12T01:57:52.054 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:57:52.377 INFO:journalctl@ceph.osd.8.smithi184.stdout:Sep 12 01:57:52 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-8[54277]: 2024-09-12T01:57:52.252+0000 7fe989c7d640 -1 osd.8 0 waiting for initial osdmap 2024-09-12T01:57:52.377 INFO:journalctl@ceph.osd.8.smithi184.stdout:Sep 12 01:57:52 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-8[54277]: 2024-09-12T01:57:52.267+0000 7fe9862b7640 -1 osd.8 54 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T01:57:52.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:52 smithi184 ceph-mon[35593]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-12T01:57:52.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:52 smithi184 ceph-mon[35593]: osdmap e53: 9 total, 8 up, 9 in 2024-09-12T01:57:52.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:52 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T01:57:52.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:52 smithi184 ceph-mon[35593]: from='osd.8 [v2:172.21.15.184:6816/781851781,v1:172.21.15.184:6817/781851781]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2024-09-12T01:57:52.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:52 smithi184 ceph-mon[35593]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2024-09-12T01:57:52.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:52 smithi184 ceph-mon[35593]: Detected new or changed devices on smithi184 2024-09-12T01:57:52.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:52 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:57:52.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:52 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:57:52.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:52 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:57:52.379 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:52 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:57:52.379 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:52 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:57:52.379 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:52 smithi184 ceph-mon[35593]: Adjusting osd_memory_target on smithi184 to 5696M 2024-09-12T01:57:52.379 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:52 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:57:52.379 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:52 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:57:52.379 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:52 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:57:52.379 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:52 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:57:52.379 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:52 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:57:52.379 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:52 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:57:52.379 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:52 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:57:52.379 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:52 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:57:52.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:52 smithi016 ceph-mon[30947]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-12T01:57:52.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:52 smithi016 ceph-mon[30947]: osdmap e53: 9 total, 8 up, 9 in 2024-09-12T01:57:52.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:52 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T01:57:52.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:52 smithi016 ceph-mon[30947]: from='osd.8 [v2:172.21.15.184:6816/781851781,v1:172.21.15.184:6817/781851781]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2024-09-12T01:57:52.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:52 smithi016 ceph-mon[30947]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2024-09-12T01:57:52.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:52 smithi016 ceph-mon[30947]: Detected new or changed devices on smithi184 2024-09-12T01:57:52.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:52 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:57:52.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:52 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:57:52.435 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:52 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:57:52.435 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:52 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:57:52.435 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:52 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:57:52.435 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:52 smithi016 ceph-mon[30947]: Adjusting osd_memory_target on smithi184 to 5696M 2024-09-12T01:57:52.435 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:52 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:57:52.435 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:52 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:57:52.435 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:52 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:57:52.435 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:52 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:57:52.435 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:52 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:57:52.435 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:52 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:57:52.435 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:52 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:57:52.435 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:52 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:57:52.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:57:52.546+0000 7f2d3e008640 1 -- 172.21.15.16:0/2649899909 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d38105010 msgr2=0x7f2d38107400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:57:52.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:57:52.546+0000 7f2d3e008640 1 --2- 172.21.15.16:0/2649899909 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d38105010 0x7f2d38107400 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2d2c009900 tx=0x7f2d2c02a410 comp rx=0 tx=0).stop 2024-09-12T01:57:52.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:57:52.546+0000 7f2d3e008640 1 -- 172.21.15.16:0/2649899909 shutdown_connections 2024-09-12T01:57:52.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:57:52.546+0000 7f2d3e008640 1 --2- 172.21.15.16:0/2649899909 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2d38107940 0x7f2d38109d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:57:52.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:57:52.546+0000 7f2d3e008640 1 --2- 172.21.15.16:0/2649899909 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d38105010 0x7f2d38107400 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:57:52.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:57:52.546+0000 7f2d3e008640 1 --2- 172.21.15.16:0/2649899909 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d381026e0 0x7f2d38104ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:57:52.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:57:52.546+0000 7f2d3e008640 1 -- 172.21.15.16:0/2649899909 >> 172.21.15.16:0/2649899909 conn(0x7f2d380fc280 msgr2=0x7f2d380fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:57:52.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:57:52.546+0000 7f2d3e008640 1 -- 172.21.15.16:0/2649899909 shutdown_connections 2024-09-12T01:57:52.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:57:52.546+0000 7f2d3e008640 1 -- 172.21.15.16:0/2649899909 wait complete. 2024-09-12T01:57:52.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:57:52.546+0000 7f2d3e008640 1 Processor -- start 2024-09-12T01:57:52.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:57:52.546+0000 7f2d3e008640 1 -- start start 2024-09-12T01:57:52.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:57:52.547+0000 7f2d3e008640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2d381026e0 0x7f2d38197ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:57:52.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:57:52.547+0000 7f2d3e008640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d38105010 0x7f2d38198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:57:52.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:57:52.547+0000 7f2d3e008640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d38107940 0x7f2d3819f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:57:52.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:57:52.547+0000 7f2d3e008640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2d3810d880 con 0x7f2d38105010 2024-09-12T01:57:52.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:57:52.547+0000 7f2d3e008640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2d3810d700 con 0x7f2d38107940 2024-09-12T01:57:52.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:57:52.547+0000 7f2d3e008640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f2d3810da00 con 0x7f2d381026e0 2024-09-12T01:57:52.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:57:52.547+0000 7f2d36ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d38105010 0x7f2d38198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:57:52.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:57:52.547+0000 7f2d377fe640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2d381026e0 0x7f2d38197ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:57:52.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:57:52.547+0000 7f2d36ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d38105010 0x7f2d38198420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55508/0 (socket says 172.21.15.16:55508) 2024-09-12T01:57:52.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:57:52.547+0000 7f2d36ffd640 1 -- 172.21.15.16:0/505296413 learned_addr learned my addr 172.21.15.16:0/505296413 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:57:52.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:57:52.547+0000 7f2d37fff640 1 --2- 172.21.15.16:0/505296413 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d38107940 0x7f2d3819f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:57:52.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:57:52.547+0000 7f2d36ffd640 1 -- 172.21.15.16:0/505296413 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2d381026e0 msgr2=0x7f2d38197ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:57:52.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:57:52.547+0000 7f2d36ffd640 1 --2- 172.21.15.16:0/505296413 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2d381026e0 0x7f2d38197ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:57:52.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:57:52.547+0000 7f2d36ffd640 1 -- 172.21.15.16:0/505296413 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d38107940 msgr2=0x7f2d3819f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:57:52.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:57:52.547+0000 7f2d36ffd640 1 --2- 172.21.15.16:0/505296413 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d38107940 0x7f2d3819f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:57:52.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:57:52.547+0000 7f2d36ffd640 1 -- 172.21.15.16:0/505296413 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d3819fb10 con 0x7f2d38105010 2024-09-12T01:57:52.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:57:52.547+0000 7f2d377fe640 1 --2- 172.21.15.16:0/505296413 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2d381026e0 0x7f2d38197ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:57:52.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:57:52.547+0000 7f2d36ffd640 1 --2- 172.21.15.16:0/505296413 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d38105010 0x7f2d38198420 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f2d2c031440 tx=0x7f2d2c032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:57:52.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:57:52.547+0000 7f2d34ff9640 1 -- 172.21.15.16:0/505296413 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d2c03f030 con 0x7f2d38105010 2024-09-12T01:57:52.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:57:52.548+0000 7f2d34ff9640 1 -- 172.21.15.16:0/505296413 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d2c003ce0 con 0x7f2d38105010 2024-09-12T01:57:52.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:57:52.548+0000 7f2d3e008640 1 -- 172.21.15.16:0/505296413 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d3819fda0 con 0x7f2d38105010 2024-09-12T01:57:52.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:57:52.548+0000 7f2d34ff9640 1 -- 172.21.15.16:0/505296413 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d2c035070 con 0x7f2d38105010 2024-09-12T01:57:52.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:57:52.548+0000 7f2d3e008640 1 -- 172.21.15.16:0/505296413 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d381a0220 con 0x7f2d38105010 2024-09-12T01:57:52.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:57:52.549+0000 7f2d3e008640 1 -- 172.21.15.16:0/505296413 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d380fdd70 con 0x7f2d38105010 2024-09-12T01:57:52.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:57:52.550+0000 7f2d34ff9640 1 -- 172.21.15.16:0/505296413 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f2d2c042030 con 0x7f2d38105010 2024-09-12T01:57:52.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:57:52.550+0000 7f2d34ff9640 1 --2- 172.21.15.16:0/505296413 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f2d200a2bb0 0x7f2d200a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:57:52.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:57:52.551+0000 7f2d377fe640 1 --2- 172.21.15.16:0/505296413 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f2d200a2bb0 0x7f2d200a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:57:52.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:57:52.551+0000 7f2d34ff9640 1 -- 172.21.15.16:0/505296413 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 5549+0+0 (secure 0 0 0) 0x7f2d2c0eea80 con 0x7f2d38105010 2024-09-12T01:57:52.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:57:52.551+0000 7f2d377fe640 1 --2- 172.21.15.16:0/505296413 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f2d200a2bb0 0x7f2d200a5070 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f2d2800bfd0 tx=0x7f2d2800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:57:52.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:57:52.555+0000 7f2d34ff9640 1 -- 172.21.15.16:0/505296413 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2d2c0bbe60 con 0x7f2d38105010 2024-09-12T01:57:52.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:52 smithi033 ceph-mon[35332]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-12T01:57:52.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:52 smithi033 ceph-mon[35332]: osdmap e53: 9 total, 8 up, 9 in 2024-09-12T01:57:52.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:52 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T01:57:52.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:52 smithi033 ceph-mon[35332]: from='osd.8 [v2:172.21.15.184:6816/781851781,v1:172.21.15.184:6817/781851781]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2024-09-12T01:57:52.670 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:52 smithi033 ceph-mon[35332]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2024-09-12T01:57:52.670 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:52 smithi033 ceph-mon[35332]: Detected new or changed devices on smithi184 2024-09-12T01:57:52.670 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:52 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:57:52.670 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:52 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:57:52.670 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:52 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:57:52.670 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:52 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:57:52.670 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:52 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:57:52.670 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:52 smithi033 ceph-mon[35332]: Adjusting osd_memory_target on smithi184 to 5696M 2024-09-12T01:57:52.670 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:52 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:57:52.670 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:52 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:57:52.670 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:52 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:57:52.671 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:52 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:57:52.671 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:52 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:57:52.671 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:52 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:57:52.671 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:52 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:57:52.671 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:52 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:57:52.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:57:52.741+0000 7f2d3e008640 1 -- 172.21.15.16:0/505296413 --> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f2d38062630 con 0x7f2d200a2bb0 2024-09-12T01:57:53.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:53 smithi016 ceph-mon[30947]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]': finished 2024-09-12T01:57:53.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:53 smithi016 ceph-mon[30947]: osdmap e54: 9 total, 8 up, 9 in 2024-09-12T01:57:53.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:53 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T01:57:53.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:53 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T01:57:53.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:53 smithi016 ceph-mon[30947]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-12T01:57:53.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:53 smithi016 ceph-mon[30947]: from='client.14436 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:57:53.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:53 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T01:57:53.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:53 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T01:57:53.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:53 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:57:53.435 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:53 smithi016 ceph-mon[30947]: osd.8 [v2:172.21.15.184:6816/781851781,v1:172.21.15.184:6817/781851781] boot 2024-09-12T01:57:53.435 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:53 smithi016 ceph-mon[30947]: osdmap e55: 9 total, 9 up, 9 in 2024-09-12T01:57:53.435 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:53 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T01:57:53.628 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:53 smithi184 ceph-mon[35593]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]': finished 2024-09-12T01:57:53.628 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:53 smithi184 ceph-mon[35593]: osdmap e54: 9 total, 8 up, 9 in 2024-09-12T01:57:53.628 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:53 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T01:57:53.628 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:53 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T01:57:53.628 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:53 smithi184 ceph-mon[35593]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-12T01:57:53.628 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:53 smithi184 ceph-mon[35593]: from='client.14436 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:57:53.629 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:53 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T01:57:53.629 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:53 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T01:57:53.629 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:53 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:57:53.629 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:53 smithi184 ceph-mon[35593]: osd.8 [v2:172.21.15.184:6816/781851781,v1:172.21.15.184:6817/781851781] boot 2024-09-12T01:57:53.629 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:53 smithi184 ceph-mon[35593]: osdmap e55: 9 total, 9 up, 9 in 2024-09-12T01:57:53.629 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:53 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T01:57:53.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:53 smithi033 ceph-mon[35332]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]': finished 2024-09-12T01:57:53.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:53 smithi033 ceph-mon[35332]: osdmap e54: 9 total, 8 up, 9 in 2024-09-12T01:57:53.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:53 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T01:57:53.670 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:53 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T01:57:53.670 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:53 smithi033 ceph-mon[35332]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-12T01:57:53.670 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:53 smithi033 ceph-mon[35332]: from='client.14436 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:57:53.670 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:53 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T01:57:53.670 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:53 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T01:57:53.670 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:53 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:57:53.670 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:53 smithi033 ceph-mon[35332]: osd.8 [v2:172.21.15.184:6816/781851781,v1:172.21.15.184:6817/781851781] boot 2024-09-12T01:57:53.670 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:53 smithi033 ceph-mon[35332]: osdmap e55: 9 total, 9 up, 9 in 2024-09-12T01:57:53.670 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:53 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T01:57:54.910 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:54 smithi016 ceph-mon[30947]: purged_snaps scrub starts 2024-09-12T01:57:54.910 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:54 smithi016 ceph-mon[30947]: purged_snaps scrub ok 2024-09-12T01:57:54.910 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:54 smithi016 ceph-mon[30947]: osdmap e56: 9 total, 9 up, 9 in 2024-09-12T01:57:55.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:54 smithi184 ceph-mon[35593]: purged_snaps scrub starts 2024-09-12T01:57:55.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:54 smithi184 ceph-mon[35593]: purged_snaps scrub ok 2024-09-12T01:57:55.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:54 smithi184 ceph-mon[35593]: osdmap e56: 9 total, 9 up, 9 in 2024-09-12T01:57:55.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:54 smithi033 ceph-mon[35332]: purged_snaps scrub starts 2024-09-12T01:57:55.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:54 smithi033 ceph-mon[35332]: purged_snaps scrub ok 2024-09-12T01:57:55.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:54 smithi033 ceph-mon[35332]: osdmap e56: 9 total, 9 up, 9 in 2024-09-12T01:57:55.934 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:55 smithi016 ceph-mon[30947]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-09-12T01:57:55.934 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:55 smithi016 ceph-mon[30947]: osdmap e57: 9 total, 9 up, 9 in 2024-09-12T01:57:55.934 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:55 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/1514262990' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "23a05cb5-01d5-40f3-b66a-15fcccedbd6c"}]: dispatch 2024-09-12T01:57:55.934 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:55 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/1514262990' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "23a05cb5-01d5-40f3-b66a-15fcccedbd6c"}]': finished 2024-09-12T01:57:55.934 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:55 smithi016 ceph-mon[30947]: osdmap e58: 10 total, 9 up, 10 in 2024-09-12T01:57:55.934 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:55 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T01:57:56.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:55 smithi184 ceph-mon[35593]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-09-12T01:57:56.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:55 smithi184 ceph-mon[35593]: osdmap e57: 9 total, 9 up, 9 in 2024-09-12T01:57:56.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:55 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/1514262990' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "23a05cb5-01d5-40f3-b66a-15fcccedbd6c"}]: dispatch 2024-09-12T01:57:56.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:55 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/1514262990' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "23a05cb5-01d5-40f3-b66a-15fcccedbd6c"}]': finished 2024-09-12T01:57:56.128 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:55 smithi184 ceph-mon[35593]: osdmap e58: 10 total, 9 up, 10 in 2024-09-12T01:57:56.128 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:55 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T01:57:56.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:55 smithi033 ceph-mon[35332]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-09-12T01:57:56.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:55 smithi033 ceph-mon[35332]: osdmap e57: 9 total, 9 up, 9 in 2024-09-12T01:57:56.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:55 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/1514262990' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "23a05cb5-01d5-40f3-b66a-15fcccedbd6c"}]: dispatch 2024-09-12T01:57:56.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:55 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/1514262990' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "23a05cb5-01d5-40f3-b66a-15fcccedbd6c"}]': finished 2024-09-12T01:57:56.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:55 smithi033 ceph-mon[35332]: osdmap e58: 10 total, 9 up, 10 in 2024-09-12T01:57:56.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:55 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T01:57:57.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:56 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/3985362915' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T01:57:57.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:56 smithi184 ceph-mon[35593]: osdmap e59: 10 total, 9 up, 10 in 2024-09-12T01:57:57.128 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:56 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T01:57:57.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:56 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/3985362915' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T01:57:57.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:56 smithi033 ceph-mon[35332]: osdmap e59: 10 total, 9 up, 10 in 2024-09-12T01:57:57.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:56 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T01:57:57.183 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:56 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/3985362915' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T01:57:57.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:56 smithi016 ceph-mon[30947]: osdmap e59: 10 total, 9 up, 10 in 2024-09-12T01:57:57.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:56 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T01:57:58.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:57 smithi184 ceph-mon[35593]: pgmap v192: 1 pgs: 1 remapped+peering; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-12T01:57:58.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:57 smithi033 ceph-mon[35332]: pgmap v192: 1 pgs: 1 remapped+peering; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-12T01:57:58.183 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:57 smithi016 ceph-mon[30947]: pgmap v192: 1 pgs: 1 remapped+peering; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-12T01:58:00.128 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:57:59 smithi184 ceph-mon[35593]: pgmap v194: 1 pgs: 1 remapped+peering; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-12T01:58:00.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:57:59 smithi033 ceph-mon[35332]: pgmap v194: 1 pgs: 1 remapped+peering; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-12T01:58:00.183 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:57:59 smithi016 ceph-mon[30947]: pgmap v194: 1 pgs: 1 remapped+peering; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-12T01:58:02.085 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:58:01 smithi016 ceph-mon[30947]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-12T01:58:02.113 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:58:01 smithi184 ceph-mon[35593]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-12T01:58:02.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:58:01 smithi033 ceph-mon[35332]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-12T01:58:04.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:58:03 smithi184 ceph-mon[35593]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-09-12T01:58:04.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:58:03 smithi033 ceph-mon[35332]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-09-12T01:58:04.183 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:58:03 smithi016 ceph-mon[30947]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-09-12T01:58:04.934 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:58:04 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-12T01:58:04.934 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:58:04 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:58:04.934 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:58:04 smithi016 ceph-mon[30947]: Deploying daemon osd.9 on smithi016 2024-09-12T01:58:05.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:58:04 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-12T01:58:05.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:58:04 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:58:05.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:58:04 smithi184 ceph-mon[35593]: Deploying daemon osd.9 on smithi016 2024-09-12T01:58:05.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:58:04 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-12T01:58:05.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:58:04 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:58:05.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:58:04 smithi033 ceph-mon[35332]: Deploying daemon osd.9 on smithi016 2024-09-12T01:58:05.889 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:58:05 smithi016 ceph-mon[30947]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-09-12T01:58:06.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:58:05 smithi184 ceph-mon[35593]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-09-12T01:58:06.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:58:05 smithi033 ceph-mon[35332]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-09-12T01:58:07.857 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:58:07 smithi016 ceph-mon[30947]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-12T01:58:08.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:58:07 smithi184 ceph-mon[35593]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-12T01:58:08.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:58:07 smithi033 ceph-mon[35332]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-12T01:58:10.053 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:58:09 smithi016 ceph-mon[30947]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-09-12T01:58:10.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:58:09 smithi184 ceph-mon[35593]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-09-12T01:58:10.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:58:09 smithi033 ceph-mon[35332]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-09-12T01:58:10.934 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:58:10 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:58:10.934 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:58:10 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:58:10.934 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:58:10 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:58:11.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:58:10 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:58:11.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:58:10 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:58:11.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:58:10 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:58:11.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:58:10 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:58:11.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:58:10 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:58:11.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:58:10 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:58:12.303 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:58:12 smithi016 ceph-mon[30947]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-12T01:58:12.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:58:12 smithi184 ceph-mon[35593]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-12T01:58:12.420 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:58:12 smithi033 ceph-mon[35332]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-12T01:58:13.380 INFO:teuthology.orchestra.run.smithi016.stdout:Created osd(s) 9 on host 'smithi016' 2024-09-12T01:58:13.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:58:13.378+0000 7f2d34ff9640 1 -- 172.21.15.16:0/505296413 <== mgr.14150 v2:172.21.15.16:6800/3887444820 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f2d38062630 con 0x7f2d200a2bb0 2024-09-12T01:58:13.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:58:13.381+0000 7f2d3e008640 1 -- 172.21.15.16:0/505296413 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f2d200a2bb0 msgr2=0x7f2d200a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:58:13.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:58:13.381+0000 7f2d3e008640 1 --2- 172.21.15.16:0/505296413 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f2d200a2bb0 0x7f2d200a5070 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f2d2800bfd0 tx=0x7f2d2800b390 comp rx=0 tx=0).stop 2024-09-12T01:58:13.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:58:13.381+0000 7f2d3e008640 1 -- 172.21.15.16:0/505296413 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d38105010 msgr2=0x7f2d38198420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:58:13.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:58:13.381+0000 7f2d3e008640 1 --2- 172.21.15.16:0/505296413 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d38105010 0x7f2d38198420 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f2d2c031440 tx=0x7f2d2c032ce0 comp rx=0 tx=0).stop 2024-09-12T01:58:13.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:58:13.381+0000 7f2d3e008640 1 -- 172.21.15.16:0/505296413 shutdown_connections 2024-09-12T01:58:13.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:58:13.381+0000 7f2d3e008640 1 --2- 172.21.15.16:0/505296413 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f2d200a2bb0 0x7f2d200a5070 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:13.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:58:13.381+0000 7f2d3e008640 1 --2- 172.21.15.16:0/505296413 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d38107940 0x7f2d3819f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:13.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:58:13.381+0000 7f2d3e008640 1 --2- 172.21.15.16:0/505296413 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d38105010 0x7f2d38198420 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:13.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:58:13.381+0000 7f2d3e008640 1 --2- 172.21.15.16:0/505296413 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2d381026e0 0x7f2d38197ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:13.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:58:13.381+0000 7f2d3e008640 1 -- 172.21.15.16:0/505296413 >> 172.21.15.16:0/505296413 conn(0x7f2d380fc280 msgr2=0x7f2d38103080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:58:13.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:58:13.381+0000 7f2d3e008640 1 -- 172.21.15.16:0/505296413 shutdown_connections 2024-09-12T01:58:13.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:58:13.381+0000 7f2d3e008640 1 -- 172.21.15.16:0/505296413 wait complete. 2024-09-12T01:58:14.002 DEBUG:teuthology.orchestra.run.smithi016:osd.9> sudo journalctl -f -n 0 -u ceph-879f3b56-70a9-11ef-bcea-c7b262605968@osd.9.service 2024-09-12T01:58:14.005 INFO:tasks.cephadm:Deploying osd.10 on smithi033 with /dev/vg_nvme/lv_1... 2024-09-12T01:58:14.005 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-12T01:58:14.270 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T01:58:14.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:58:14 smithi184 ceph-mon[35593]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-12T01:58:14.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:58:14 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:58:14.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:58:14 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:58:14.390 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:58:14 smithi033 ceph-mon[35332]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-12T01:58:14.390 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:58:14 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:58:14.390 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:58:14 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:58:14.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:58:14 smithi016 ceph-mon[30947]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-12T01:58:14.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:58:14 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:58:14.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:58:14 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:58:15.495 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:58:15 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:58:15.495 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:58:15 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:58:15.601 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:58:15 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:58:15.601 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:58:15 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:58:15.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:58:15 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:58:15.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:58:15 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:58:16.482 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:58:16 smithi033 ceph-mon[35332]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-12T01:58:16.572 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-12T01:58:16.596 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:58:16 smithi016 ceph-mon[30947]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-12T01:58:16.596 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:58:16 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T01:58:16.391+0000 7fc3b9b3d640 -1 osd.9 0 log_to_monitors true 2024-09-12T01:58:16.598 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi033:vg_nvme/lv_1 2024-09-12T01:58:16.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:58:16 smithi184 ceph-mon[35593]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-12T01:58:16.850 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T01:58:17.274 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:58:17 smithi033 ceph-mon[35332]: from='osd.9 [v2:172.21.15.16:6826/1569578746,v1:172.21.15.16:6827/1569578746]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-12T01:58:17.382 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:58:17 smithi016 ceph-mon[30947]: from='osd.9 [v2:172.21.15.16:6826/1569578746,v1:172.21.15.16:6827/1569578746]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-12T01:58:17.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:58:17.491+0000 7fd08a91e640 1 -- 172.21.15.33:0/3812742943 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd084101d00 msgr2=0x7fd084102180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:58:17.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:58:17.491+0000 7fd08a91e640 1 --2- 172.21.15.33:0/3812742943 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd084101d00 0x7fd084102180 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fd078009900 tx=0x7fd07802a410 comp rx=0 tx=0).stop 2024-09-12T01:58:17.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:58:17.491+0000 7fd08a91e640 1 -- 172.21.15.33:0/3812742943 shutdown_connections 2024-09-12T01:58:17.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:58:17.491+0000 7fd08a91e640 1 --2- 172.21.15.33:0/3812742943 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd0841026c0 0x7fd084108f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:17.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:58:17.491+0000 7fd08a91e640 1 --2- 172.21.15.33:0/3812742943 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd084101d00 0x7fd084102180 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:17.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:58:17.491+0000 7fd08a91e640 1 --2- 172.21.15.33:0/3812742943 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd084100b00 0x7fd084100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:17.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:58:17.491+0000 7fd08a91e640 1 -- 172.21.15.33:0/3812742943 >> 172.21.15.33:0/3812742943 conn(0x7fd0840fc290 msgr2=0x7fd0840fe6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:58:17.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:58:17.491+0000 7fd08a91e640 1 -- 172.21.15.33:0/3812742943 shutdown_connections 2024-09-12T01:58:17.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:58:17.491+0000 7fd08a91e640 1 -- 172.21.15.33:0/3812742943 wait complete. 2024-09-12T01:58:17.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:58:17.492+0000 7fd08a91e640 1 Processor -- start 2024-09-12T01:58:17.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:58:17.492+0000 7fd08a91e640 1 -- start start 2024-09-12T01:58:17.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:58:17.492+0000 7fd08a91e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd084100b00 0x7fd08406b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:58:17.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:58:17.492+0000 7fd08a91e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd084101d00 0x7fd08419f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:58:17.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:58:17.492+0000 7fd08a91e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd0841026c0 0x7fd0841a1590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:58:17.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:58:17.492+0000 7fd08a91e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd08410c780 con 0x7fd084101d00 2024-09-12T01:58:17.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:58:17.492+0000 7fd08a91e640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd08410c600 con 0x7fd084100b00 2024-09-12T01:58:17.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:58:17.492+0000 7fd08a91e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd08410c900 con 0x7fd0841026c0 2024-09-12T01:58:17.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:58:17.492+0000 7fd083fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd084100b00 0x7fd08406b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:58:17.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:58:17.492+0000 7fd083fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd084100b00 0x7fd08406b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58792/0 (socket says 172.21.15.33:58792) 2024-09-12T01:58:17.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:58:17.492+0000 7fd083fff640 1 -- 172.21.15.33:0/3394518101 learned_addr learned my addr 172.21.15.33:0/3394518101 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-12T01:58:17.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:58:17.492+0000 7fd0837fe640 1 --2- 172.21.15.33:0/3394518101 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd084101d00 0x7fd08419f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:58:17.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:58:17.492+0000 7fd088ebb640 1 --2- 172.21.15.33:0/3394518101 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd0841026c0 0x7fd0841a1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:58:17.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:58:17.493+0000 7fd083fff640 1 -- 172.21.15.33:0/3394518101 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd0841026c0 msgr2=0x7fd0841a1590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:58:17.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:58:17.493+0000 7fd083fff640 1 --2- 172.21.15.33:0/3394518101 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd0841026c0 0x7fd0841a1590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:17.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:58:17.493+0000 7fd083fff640 1 -- 172.21.15.33:0/3394518101 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd084101d00 msgr2=0x7fd08419f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:58:17.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:58:17.493+0000 7fd083fff640 1 --2- 172.21.15.33:0/3394518101 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd084101d00 0x7fd08419f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:17.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:58:17.493+0000 7fd083fff640 1 -- 172.21.15.33:0/3394518101 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0841a1d80 con 0x7fd084100b00 2024-09-12T01:58:17.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:58:17.493+0000 7fd083fff640 1 --2- 172.21.15.33:0/3394518101 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd084100b00 0x7fd08406b2f0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd07400e9c0 tx=0x7fd07400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:58:17.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:58:17.493+0000 7fd0817fa640 1 -- 172.21.15.33:0/3394518101 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd07400cd80 con 0x7fd084100b00 2024-09-12T01:58:17.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:58:17.493+0000 7fd0817fa640 1 -- 172.21.15.33:0/3394518101 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd0740035f0 con 0x7fd084100b00 2024-09-12T01:58:17.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:58:17.493+0000 7fd08a91e640 1 -- 172.21.15.33:0/3394518101 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0841a1fb0 con 0x7fd084100b00 2024-09-12T01:58:17.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:58:17.493+0000 7fd0817fa640 1 -- 172.21.15.33:0/3394518101 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd074010690 con 0x7fd084100b00 2024-09-12T01:58:17.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:58:17.493+0000 7fd08a91e640 1 -- 172.21.15.33:0/3394518101 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0841a2460 con 0x7fd084100b00 2024-09-12T01:58:17.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:58:17.494+0000 7fd08a91e640 1 -- 172.21.15.33:0/3394518101 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd084100f00 con 0x7fd084100b00 2024-09-12T01:58:17.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:58:17.495+0000 7fd0817fa640 1 -- 172.21.15.33:0/3394518101 <== mon.2 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fd0740037a0 con 0x7fd084100b00 2024-09-12T01:58:17.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:58:17.496+0000 7fd0817fa640 1 --2- 172.21.15.33:0/3394518101 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fd06c0a2bb0 0x7fd06c0a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:58:17.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:58:17.496+0000 7fd0817fa640 1 -- 172.21.15.33:0/3394518101 <== mon.2 v2:172.21.15.33:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5965+0+0 (secure 0 0 0) 0x7fd0740ceb30 con 0x7fd084100b00 2024-09-12T01:58:17.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:58:17.496+0000 7fd0837fe640 1 --2- 172.21.15.33:0/3394518101 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fd06c0a2bb0 0x7fd06c0a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:58:17.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:58:17.497+0000 7fd0837fe640 1 --2- 172.21.15.33:0/3394518101 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fd06c0a2bb0 0x7fd06c0a5070 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd0780314e0 tx=0x7fd078000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:58:17.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:58:17.500+0000 7fd0817fa640 1 -- 172.21.15.33:0/3394518101 <== mon.2 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd07409c320 con 0x7fd084100b00 2024-09-12T01:58:17.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:58:17 smithi184 ceph-mon[35593]: from='osd.9 [v2:172.21.15.16:6826/1569578746,v1:172.21.15.16:6827/1569578746]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-12T01:58:17.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:58:17.685+0000 7fd08a91e640 1 -- 172.21.15.33:0/3394518101 --> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fd0840626f0 con 0x7fd06c0a2bb0 2024-09-12T01:58:18.375 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:58:18 smithi033 ceph-mon[35332]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-12T01:58:18.375 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:58:18 smithi033 ceph-mon[35332]: from='osd.9 [v2:172.21.15.16:6826/1569578746,v1:172.21.15.16:6827/1569578746]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-12T01:58:18.375 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:58:18 smithi033 ceph-mon[35332]: osdmap e60: 10 total, 9 up, 10 in 2024-09-12T01:58:18.375 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:58:18 smithi033 ceph-mon[35332]: from='osd.9 [v2:172.21.15.16:6826/1569578746,v1:172.21.15.16:6827/1569578746]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-09-12T01:58:18.376 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:58:18 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T01:58:18.376 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:58:18 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:58:18.376 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:58:18 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:58:18.376 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:58:18 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:58:18.376 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:58:18 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:58:18.376 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:58:18 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:58:18.376 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:58:18 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:58:18.376 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:58:18 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:58:18.376 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:58:18 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:58:18.376 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:58:18 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:58:18.376 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:58:18 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:58:18.377 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:58:18 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:58:18.377 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:58:18 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:58:18.377 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:58:18 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:58:18.377 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:58:18 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:58:18.377 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:58:18 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T01:58:18.377 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:58:18 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T01:58:18.377 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:58:18 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:58:18.628 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:58:18 smithi184 ceph-mon[35593]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-12T01:58:18.628 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:58:18 smithi184 ceph-mon[35593]: from='osd.9 [v2:172.21.15.16:6826/1569578746,v1:172.21.15.16:6827/1569578746]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-12T01:58:18.628 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:58:18 smithi184 ceph-mon[35593]: osdmap e60: 10 total, 9 up, 10 in 2024-09-12T01:58:18.628 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:58:18 smithi184 ceph-mon[35593]: from='osd.9 [v2:172.21.15.16:6826/1569578746,v1:172.21.15.16:6827/1569578746]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-09-12T01:58:18.628 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:58:18 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T01:58:18.628 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:58:18 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:58:18.628 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:58:18 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:58:18.628 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:58:18 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:58:18.628 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:58:18 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:58:18.628 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:58:18 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:58:18.629 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:58:18 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:58:18.629 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:58:18 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:58:18.629 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:58:18 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:58:18.629 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:58:18 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:58:18.629 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:58:18 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:58:18.629 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:58:18 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:58:18.629 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:58:18 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:58:18.629 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:58:18 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:58:18.629 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:58:18 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:58:18.629 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:58:18 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T01:58:18.629 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:58:18 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T01:58:18.629 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:58:18 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:58:18.683 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:58:18 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T01:58:18.222+0000 7fc3b529d640 -1 osd.9 0 waiting for initial osdmap 2024-09-12T01:58:18.684 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:58:18 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T01:58:18.236+0000 7fc3b18d7640 -1 osd.9 61 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T01:58:18.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:58:18 smithi016 ceph-mon[30947]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-12T01:58:18.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:58:18 smithi016 ceph-mon[30947]: from='osd.9 [v2:172.21.15.16:6826/1569578746,v1:172.21.15.16:6827/1569578746]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-12T01:58:18.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:58:18 smithi016 ceph-mon[30947]: osdmap e60: 10 total, 9 up, 10 in 2024-09-12T01:58:18.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:58:18 smithi016 ceph-mon[30947]: from='osd.9 [v2:172.21.15.16:6826/1569578746,v1:172.21.15.16:6827/1569578746]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-09-12T01:58:18.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:58:18 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T01:58:18.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:58:18 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:58:18.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:58:18 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:58:18.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:58:18 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:58:18.685 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:58:18 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:58:18.685 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:58:18 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:58:18.685 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:58:18 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:58:18.685 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:58:18 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:58:18.685 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:58:18 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:58:18.685 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:58:18 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:58:18.685 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:58:18 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:58:18.685 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:58:18 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:58:18.685 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:58:18 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:58:18.685 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:58:18 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:58:18.685 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:58:18 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:58:18.685 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:58:18 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T01:58:18.685 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:58:18 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T01:58:18.686 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:58:18 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:58:19.331 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:58:19 smithi033 ceph-mon[35332]: Detected new or changed devices on smithi016 2024-09-12T01:58:19.331 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:58:19 smithi033 ceph-mon[35332]: Adjusting osd_memory_target on smithi016 to 4273M 2024-09-12T01:58:19.331 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:58:19 smithi033 ceph-mon[35332]: from='client.24326 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:58:19.331 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:58:19 smithi033 ceph-mon[35332]: from='osd.9 [v2:172.21.15.16:6826/1569578746,v1:172.21.15.16:6827/1569578746]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-09-12T01:58:19.331 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:58:19 smithi033 ceph-mon[35332]: osdmap e61: 10 total, 9 up, 10 in 2024-09-12T01:58:19.331 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:58:19 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T01:58:19.332 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:58:19 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T01:58:19.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:58:19 smithi184 ceph-mon[35593]: Detected new or changed devices on smithi016 2024-09-12T01:58:19.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:58:19 smithi184 ceph-mon[35593]: Adjusting osd_memory_target on smithi016 to 4273M 2024-09-12T01:58:19.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:58:19 smithi184 ceph-mon[35593]: from='client.24326 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:58:19.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:58:19 smithi184 ceph-mon[35593]: from='osd.9 [v2:172.21.15.16:6826/1569578746,v1:172.21.15.16:6827/1569578746]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-09-12T01:58:19.628 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:58:19 smithi184 ceph-mon[35593]: osdmap e61: 10 total, 9 up, 10 in 2024-09-12T01:58:19.628 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:58:19 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T01:58:19.628 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:58:19 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T01:58:19.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:58:19 smithi016 ceph-mon[30947]: Detected new or changed devices on smithi016 2024-09-12T01:58:19.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:58:19 smithi016 ceph-mon[30947]: Adjusting osd_memory_target on smithi016 to 4273M 2024-09-12T01:58:19.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:58:19 smithi016 ceph-mon[30947]: from='client.24326 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:58:19.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:58:19 smithi016 ceph-mon[30947]: from='osd.9 [v2:172.21.15.16:6826/1569578746,v1:172.21.15.16:6827/1569578746]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-09-12T01:58:19.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:58:19 smithi016 ceph-mon[30947]: osdmap e61: 10 total, 9 up, 10 in 2024-09-12T01:58:19.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:58:19 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T01:58:19.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:58:19 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T01:58:20.529 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:58:20 smithi184 ceph-mon[35593]: purged_snaps scrub starts 2024-09-12T01:58:20.529 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:58:20 smithi184 ceph-mon[35593]: purged_snaps scrub ok 2024-09-12T01:58:20.529 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:58:20 smithi184 ceph-mon[35593]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-12T01:58:20.529 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:58:20 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T01:58:20.529 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:58:20 smithi184 ceph-mon[35593]: osd.9 [v2:172.21.15.16:6826/1569578746,v1:172.21.15.16:6827/1569578746] boot 2024-09-12T01:58:20.529 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:58:20 smithi184 ceph-mon[35593]: osdmap e62: 10 total, 10 up, 10 in 2024-09-12T01:58:20.529 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:58:20 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T01:58:20.529 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:58:20 smithi184 ceph-mon[35593]: from='client.? 172.21.15.33:0/1075587869' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e4bd6bce-3c69-44db-bc09-d264a0ad094a"}]: dispatch 2024-09-12T01:58:20.529 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:58:20 smithi184 ceph-mon[35593]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e4bd6bce-3c69-44db-bc09-d264a0ad094a"}]: dispatch 2024-09-12T01:58:20.530 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:58:20 smithi184 ceph-mon[35593]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e4bd6bce-3c69-44db-bc09-d264a0ad094a"}]': finished 2024-09-12T01:58:20.530 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:58:20 smithi184 ceph-mon[35593]: osdmap e63: 11 total, 10 up, 11 in 2024-09-12T01:58:20.530 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:58:20 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T01:58:20.588 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:58:20 smithi033 ceph-mon[35332]: purged_snaps scrub starts 2024-09-12T01:58:20.588 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:58:20 smithi033 ceph-mon[35332]: purged_snaps scrub ok 2024-09-12T01:58:20.588 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:58:20 smithi033 ceph-mon[35332]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-12T01:58:20.588 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:58:20 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T01:58:20.588 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:58:20 smithi033 ceph-mon[35332]: osd.9 [v2:172.21.15.16:6826/1569578746,v1:172.21.15.16:6827/1569578746] boot 2024-09-12T01:58:20.588 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:58:20 smithi033 ceph-mon[35332]: osdmap e62: 10 total, 10 up, 10 in 2024-09-12T01:58:20.589 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:58:20 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T01:58:20.589 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:58:20 smithi033 ceph-mon[35332]: from='client.? 172.21.15.33:0/1075587869' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e4bd6bce-3c69-44db-bc09-d264a0ad094a"}]: dispatch 2024-09-12T01:58:20.589 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:58:20 smithi033 ceph-mon[35332]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e4bd6bce-3c69-44db-bc09-d264a0ad094a"}]: dispatch 2024-09-12T01:58:20.589 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:58:20 smithi033 ceph-mon[35332]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e4bd6bce-3c69-44db-bc09-d264a0ad094a"}]': finished 2024-09-12T01:58:20.589 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:58:20 smithi033 ceph-mon[35332]: osdmap e63: 11 total, 10 up, 11 in 2024-09-12T01:58:20.589 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:58:20 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T01:58:20.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:58:20 smithi016 ceph-mon[30947]: purged_snaps scrub starts 2024-09-12T01:58:20.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:58:20 smithi016 ceph-mon[30947]: purged_snaps scrub ok 2024-09-12T01:58:20.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:58:20 smithi016 ceph-mon[30947]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-12T01:58:20.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:58:20 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T01:58:20.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:58:20 smithi016 ceph-mon[30947]: osd.9 [v2:172.21.15.16:6826/1569578746,v1:172.21.15.16:6827/1569578746] boot 2024-09-12T01:58:20.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:58:20 smithi016 ceph-mon[30947]: osdmap e62: 10 total, 10 up, 10 in 2024-09-12T01:58:20.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:58:20 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T01:58:20.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:58:20 smithi016 ceph-mon[30947]: from='client.? 172.21.15.33:0/1075587869' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e4bd6bce-3c69-44db-bc09-d264a0ad094a"}]: dispatch 2024-09-12T01:58:20.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:58:20 smithi016 ceph-mon[30947]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e4bd6bce-3c69-44db-bc09-d264a0ad094a"}]: dispatch 2024-09-12T01:58:20.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:58:20 smithi016 ceph-mon[30947]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e4bd6bce-3c69-44db-bc09-d264a0ad094a"}]': finished 2024-09-12T01:58:20.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:58:20 smithi016 ceph-mon[30947]: osdmap e63: 11 total, 10 up, 11 in 2024-09-12T01:58:20.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:58:20 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T01:58:21.546 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:58:21 smithi184 ceph-mon[35593]: from='client.? 172.21.15.33:0/1562482598' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T01:58:21.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:58:21 smithi033 ceph-mon[35332]: from='client.? 172.21.15.33:0/1562482598' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T01:58:21.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:58:21 smithi016 ceph-mon[30947]: from='client.? 172.21.15.33:0/1562482598' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T01:58:22.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:58:22 smithi184 ceph-mon[35593]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-12T01:58:22.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:58:22 smithi033 ceph-mon[35332]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-12T01:58:22.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:58:22 smithi016 ceph-mon[30947]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-12T01:58:23.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:58:23 smithi184 ceph-mon[35593]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-12T01:58:23.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:58:23 smithi033 ceph-mon[35332]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-12T01:58:23.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:58:23 smithi016 ceph-mon[30947]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-12T01:58:25.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:58:25 smithi184 ceph-mon[35593]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-12T01:58:25.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:58:25 smithi033 ceph-mon[35332]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-12T01:58:25.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:58:25 smithi016 ceph-mon[30947]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-12T01:58:27.585 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:58:27 smithi033 ceph-mon[35332]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-12T01:58:27.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:58:27 smithi184 ceph-mon[35593]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-12T01:58:27.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:58:27 smithi016 ceph-mon[30947]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-12T01:58:29.464 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:58:29 smithi033 ceph-mon[35332]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-12T01:58:29.465 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:58:29 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-12T01:58:29.465 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:58:29 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:58:29.465 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:58:29 smithi033 ceph-mon[35332]: Deploying daemon osd.10 on smithi033 2024-09-12T01:58:29.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:58:29 smithi184 ceph-mon[35593]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-12T01:58:29.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:58:29 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-12T01:58:29.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:58:29 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:58:29.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:58:29 smithi184 ceph-mon[35593]: Deploying daemon osd.10 on smithi033 2024-09-12T01:58:29.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:58:29 smithi016 ceph-mon[30947]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-12T01:58:29.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:58:29 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-12T01:58:29.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:58:29 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:58:29.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:58:29 smithi016 ceph-mon[30947]: Deploying daemon osd.10 on smithi033 2024-09-12T01:58:31.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:58:31 smithi184 ceph-mon[35593]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-12T01:58:31.629 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:58:31 smithi033 ceph-mon[35332]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-12T01:58:31.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:58:31 smithi016 ceph-mon[30947]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-12T01:58:33.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:58:33 smithi184 ceph-mon[35593]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-12T01:58:33.656 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:58:33 smithi033 ceph-mon[35332]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-12T01:58:33.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:58:33 smithi016 ceph-mon[30947]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-12T01:58:34.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:58:34 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:58:34.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:58:34 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:58:34.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:58:34 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:58:34.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:58:34 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:58:34.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:58:34 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:58:34.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:58:34 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:58:34.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:58:34 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:58:34.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:58:34 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:58:34.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:58:34 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:58:35.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:58:35 smithi184 ceph-mon[35593]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-12T01:58:35.640 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:58:35 smithi033 ceph-mon[35332]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-12T01:58:35.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:58:35 smithi016 ceph-mon[30947]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-12T01:58:37.651 INFO:teuthology.orchestra.run.smithi033.stdout:Created osd(s) 10 on host 'smithi033' 2024-09-12T01:58:37.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:58:37.651+0000 7fd0817fa640 1 -- 172.21.15.33:0/3394518101 <== mgr.14150 v2:172.21.15.16:6800/3887444820 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fd0840626f0 con 0x7fd06c0a2bb0 2024-09-12T01:58:37.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:58:37.653+0000 7fd08a91e640 1 -- 172.21.15.33:0/3394518101 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fd06c0a2bb0 msgr2=0x7fd06c0a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:58:37.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:58:37.653+0000 7fd08a91e640 1 --2- 172.21.15.33:0/3394518101 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fd06c0a2bb0 0x7fd06c0a5070 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd0780314e0 tx=0x7fd078000bc0 comp rx=0 tx=0).stop 2024-09-12T01:58:37.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:58:37.653+0000 7fd08a91e640 1 -- 172.21.15.33:0/3394518101 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd084100b00 msgr2=0x7fd08406b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:58:37.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:58:37.653+0000 7fd08a91e640 1 --2- 172.21.15.33:0/3394518101 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd084100b00 0x7fd08406b2f0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd07400e9c0 tx=0x7fd07400ee80 comp rx=0 tx=0).stop 2024-09-12T01:58:37.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:58:37.653+0000 7fd08a91e640 1 -- 172.21.15.33:0/3394518101 shutdown_connections 2024-09-12T01:58:37.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:58:37.653+0000 7fd08a91e640 1 --2- 172.21.15.33:0/3394518101 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fd06c0a2bb0 0x7fd06c0a5070 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:37.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:58:37.653+0000 7fd08a91e640 1 --2- 172.21.15.33:0/3394518101 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd0841026c0 0x7fd0841a1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:37.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:58:37.653+0000 7fd08a91e640 1 --2- 172.21.15.33:0/3394518101 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd084101d00 0x7fd08419f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:37.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:58:37.653+0000 7fd08a91e640 1 --2- 172.21.15.33:0/3394518101 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd084100b00 0x7fd08406b2f0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:37.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:58:37.653+0000 7fd08a91e640 1 -- 172.21.15.33:0/3394518101 >> 172.21.15.33:0/3394518101 conn(0x7fd0840fc290 msgr2=0x7fd0841050c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:58:37.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:58:37.653+0000 7fd08a91e640 1 -- 172.21.15.33:0/3394518101 shutdown_connections 2024-09-12T01:58:37.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:58:37.653+0000 7fd08a91e640 1 -- 172.21.15.33:0/3394518101 wait complete. 2024-09-12T01:58:38.230 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:58:38 smithi033 ceph-mon[35332]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-12T01:58:38.230 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:58:38 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:58:38.230 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:58:38 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:58:38.230 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:58:38 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:58:38.230 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:58:38 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:58:38.230 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:58:38 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:58:38.230 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:58:38 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:58:38.230 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:58:38 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:58:38.231 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:58:38 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:58:38.256 DEBUG:teuthology.orchestra.run.smithi033:osd.10> sudo journalctl -f -n 0 -u ceph-879f3b56-70a9-11ef-bcea-c7b262605968@osd.10.service 2024-09-12T01:58:38.259 INFO:tasks.cephadm:Deploying osd.11 on smithi184 with /dev/vg_nvme/lv_1... 2024-09-12T01:58:38.260 DEBUG:teuthology.orchestra.run.smithi184:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-12T01:58:38.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:58:38 smithi184 ceph-mon[35593]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-12T01:58:38.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:58:38 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:58:38.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:58:38 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:58:38.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:58:38 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:58:38.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:58:38 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:58:38.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:58:38 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:58:38.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:58:38 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:58:38.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:58:38 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:58:38.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:58:38 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:58:38.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:58:38 smithi016 ceph-mon[30947]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-12T01:58:38.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:58:38 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:58:38.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:58:38 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:58:38.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:58:38 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:58:38.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:58:38 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:58:38.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:58:38 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:58:38.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:58:38 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:58:38.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:58:38 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:58:38.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:58:38 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:58:38.518 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T01:58:40.359 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:58:40 smithi184 ceph-mon[35593]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-12T01:58:40.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:58:40 smithi033 ceph-mon[35332]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-12T01:58:40.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:58:40 smithi016 ceph-mon[30947]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-12T01:58:40.734 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 01:58:40 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T01:58:40.582+0000 7f78d7b3e640 -1 osd.10 0 log_to_monitors true 2024-09-12T01:58:41.192 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-12T01:58:41.226 DEBUG:teuthology.orchestra.run.smithi184:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi184:vg_nvme/lv_1 2024-09-12T01:58:41.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:58:41 smithi184 ceph-mon[35593]: from='osd.10 [v2:172.21.15.33:6824/469977847,v1:172.21.15.33:6825/469977847]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-12T01:58:41.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:58:41 smithi184 ceph-mon[35593]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-12T01:58:41.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:58:41 smithi033 ceph-mon[35332]: from='osd.10 [v2:172.21.15.33:6824/469977847,v1:172.21.15.33:6825/469977847]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-12T01:58:41.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:58:41 smithi033 ceph-mon[35332]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-12T01:58:41.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:58:41 smithi016 ceph-mon[30947]: from='osd.10 [v2:172.21.15.33:6824/469977847,v1:172.21.15.33:6825/469977847]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-12T01:58:41.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:58:41 smithi016 ceph-mon[30947]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-12T01:58:41.484 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T01:58:41.966 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:58:41.933+0000 7f5305369640 1 -- 172.21.15.184:0/4012234767 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5300102e00 msgr2=0x7f5300103280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:58:41.966 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:58:41.933+0000 7f5305369640 1 --2- 172.21.15.184:0/4012234767 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5300102e00 0x7f5300103280 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f52e8009900 tx=0x7f52e802a410 comp rx=0 tx=0).stop 2024-09-12T01:58:41.966 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:58:41.933+0000 7f5305369640 1 -- 172.21.15.184:0/4012234767 shutdown_connections 2024-09-12T01:58:41.966 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:58:41.933+0000 7f5305369640 1 --2- 172.21.15.184:0/4012234767 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53001037c0 0x7f530010a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:41.966 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:58:41.933+0000 7f5305369640 1 --2- 172.21.15.184:0/4012234767 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5300102e00 0x7f5300103280 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:41.966 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:58:41.933+0000 7f5305369640 1 --2- 172.21.15.184:0/4012234767 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5300101c00 0x7f5300102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:41.966 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:58:41.933+0000 7f5305369640 1 -- 172.21.15.184:0/4012234767 >> 172.21.15.184:0/4012234767 conn(0x7f53000fd390 msgr2=0x7f53000ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:58:41.966 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:58:41.933+0000 7f5305369640 1 -- 172.21.15.184:0/4012234767 shutdown_connections 2024-09-12T01:58:41.966 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:58:41.933+0000 7f5305369640 1 -- 172.21.15.184:0/4012234767 wait complete. 2024-09-12T01:58:41.967 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:58:41.933+0000 7f5305369640 1 Processor -- start 2024-09-12T01:58:41.967 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:58:41.933+0000 7f5305369640 1 -- start start 2024-09-12T01:58:41.967 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:58:41.934+0000 7f5305369640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5300101c00 0x7f5300075f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:58:41.967 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:58:41.934+0000 7f5305369640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5300102e00 0x7f53000745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:58:41.967 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:58:41.934+0000 7f5305369640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53001037c0 0x7f5300074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:58:41.967 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:58:41.934+0000 7f5305369640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f530010d880 con 0x7f53001037c0 2024-09-12T01:58:41.967 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:58:41.934+0000 7f5305369640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f530010d700 con 0x7f5300101c00 2024-09-12T01:58:41.967 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:58:41.934+0000 7f5305369640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f530010da00 con 0x7f5300102e00 2024-09-12T01:58:41.967 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:58:41.934+0000 7f52f67fc640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5300102e00 0x7f53000745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:58:41.967 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:58:41.934+0000 7f52f67fc640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5300102e00 0x7f53000745e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.184:53022/0 (socket says 172.21.15.184:53022) 2024-09-12T01:58:41.967 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:58:41.934+0000 7f52f67fc640 1 -- 172.21.15.184:0/3857538822 learned_addr learned my addr 172.21.15.184:0/3857538822 (peer_addr_for_me v2:172.21.15.184:0/0) 2024-09-12T01:58:41.967 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:58:41.934+0000 7f52feffd640 1 --2- 172.21.15.184:0/3857538822 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5300101c00 0x7f5300075f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:58:41.967 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:58:41.934+0000 7f52ff7fe640 1 --2- 172.21.15.184:0/3857538822 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53001037c0 0x7f5300074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:58:41.968 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:58:41.934+0000 7f52f67fc640 1 -- 172.21.15.184:0/3857538822 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5300101c00 msgr2=0x7f5300075f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:58:41.968 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:58:41.934+0000 7f52f67fc640 1 --2- 172.21.15.184:0/3857538822 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5300101c00 0x7f5300075f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:41.968 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:58:41.934+0000 7f52f67fc640 1 -- 172.21.15.184:0/3857538822 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53001037c0 msgr2=0x7f5300074b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:58:41.968 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:58:41.934+0000 7f52f67fc640 1 --2- 172.21.15.184:0/3857538822 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53001037c0 0x7f5300074b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:41.968 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:58:41.934+0000 7f52f67fc640 1 -- 172.21.15.184:0/3857538822 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53000753e0 con 0x7f5300102e00 2024-09-12T01:58:41.968 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:58:41.934+0000 7f52feffd640 1 --2- 172.21.15.184:0/3857538822 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5300101c00 0x7f5300075f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:58:41.968 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:58:41.934+0000 7f52ff7fe640 1 --2- 172.21.15.184:0/3857538822 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53001037c0 0x7f5300074b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:58:41.968 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:58:41.934+0000 7f52f67fc640 1 --2- 172.21.15.184:0/3857538822 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5300102e00 0x7f53000745e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f52e8009230 tx=0x7f52e8004360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:58:41.968 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:58:41.934+0000 7f52fcff9640 1 -- 172.21.15.184:0/3857538822 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52e803f030 con 0x7f5300102e00 2024-09-12T01:58:41.968 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:58:41.934+0000 7f5305369640 1 -- 172.21.15.184:0/3857538822 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53001a2030 con 0x7f5300102e00 2024-09-12T01:58:41.968 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:58:41.934+0000 7f52fcff9640 1 -- 172.21.15.184:0/3857538822 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f52e8003ce0 con 0x7f5300102e00 2024-09-12T01:58:41.968 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:58:41.934+0000 7f52fcff9640 1 -- 172.21.15.184:0/3857538822 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52e8035070 con 0x7f5300102e00 2024-09-12T01:58:41.968 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:58:41.934+0000 7f5305369640 1 -- 172.21.15.184:0/3857538822 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53001a2530 con 0x7f5300102e00 2024-09-12T01:58:41.969 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:58:41.935+0000 7f5305369640 1 -- 172.21.15.184:0/3857538822 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5300109000 con 0x7f5300102e00 2024-09-12T01:58:41.969 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:58:41.937+0000 7f52fcff9640 1 -- 172.21.15.184:0/3857538822 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f52e8042030 con 0x7f5300102e00 2024-09-12T01:58:41.969 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:58:41.937+0000 7f52fcff9640 1 --2- 172.21.15.184:0/3857538822 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f52ec0a2b80 0x7f52ec0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:58:41.969 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:58:41.937+0000 7f52fcff9640 1 -- 172.21.15.184:0/3857538822 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 6398+0+0 (secure 0 0 0) 0x7f52e80ef380 con 0x7f5300102e00 2024-09-12T01:58:41.969 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:58:41.938+0000 7f52feffd640 1 --2- 172.21.15.184:0/3857538822 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f52ec0a2b80 0x7f52ec0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:58:41.969 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:58:41.938+0000 7f52feffd640 1 --2- 172.21.15.184:0/3857538822 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f52ec0a2b80 0x7f52ec0a5040 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f5300102c60 tx=0x7f52f000a560 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:58:41.969 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:58:41.941+0000 7f52fcff9640 1 -- 172.21.15.184:0/3857538822 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f52e80bca60 con 0x7f5300102e00 2024-09-12T01:58:42.117 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:58:42.116+0000 7f5305369640 1 -- 172.21.15.184:0/3857538822 --> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi184:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f5300062630 con 0x7f52ec0a2b80 2024-09-12T01:58:42.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:58:42 smithi184 ceph-mon[35593]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-12T01:58:42.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:58:42 smithi184 ceph-mon[35593]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-12T01:58:42.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:58:42 smithi184 ceph-mon[35593]: osdmap e64: 11 total, 10 up, 11 in 2024-09-12T01:58:42.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:58:42 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T01:58:42.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:58:42 smithi184 ceph-mon[35593]: from='osd.10 [v2:172.21.15.33:6824/469977847,v1:172.21.15.33:6825/469977847]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-09-12T01:58:42.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:58:42 smithi184 ceph-mon[35593]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-09-12T01:58:42.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:58:42 smithi184 ceph-mon[35593]: Detected new or changed devices on smithi033 2024-09-12T01:58:42.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:58:42 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:58:42.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:58:42 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:58:42.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:58:42 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:58:42.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:58:42 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:58:42.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:58:42 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:58:42.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:58:42 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:58:42.379 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:58:42 smithi184 ceph-mon[35593]: Adjusting osd_memory_target on smithi033 to 4273M 2024-09-12T01:58:42.379 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:58:42 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:58:42.379 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:58:42 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:58:42.379 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:58:42 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:58:42.379 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:58:42 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:58:42.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:58:42 smithi033 ceph-mon[35332]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-12T01:58:42.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:58:42 smithi033 ceph-mon[35332]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-12T01:58:42.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:58:42 smithi033 ceph-mon[35332]: osdmap e64: 11 total, 10 up, 11 in 2024-09-12T01:58:42.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:58:42 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T01:58:42.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:58:42 smithi033 ceph-mon[35332]: from='osd.10 [v2:172.21.15.33:6824/469977847,v1:172.21.15.33:6825/469977847]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-09-12T01:58:42.420 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:58:42 smithi033 ceph-mon[35332]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-09-12T01:58:42.420 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:58:42 smithi033 ceph-mon[35332]: Detected new or changed devices on smithi033 2024-09-12T01:58:42.420 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:58:42 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:58:42.420 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:58:42 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:58:42.420 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:58:42 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:58:42.420 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:58:42 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:58:42.420 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:58:42 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:58:42.420 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:58:42 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:58:42.420 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:58:42 smithi033 ceph-mon[35332]: Adjusting osd_memory_target on smithi033 to 4273M 2024-09-12T01:58:42.420 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:58:42 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:58:42.420 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:58:42 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:58:42.420 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:58:42 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:58:42.420 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:58:42 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:58:42.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:58:42 smithi016 ceph-mon[30947]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-12T01:58:42.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:58:42 smithi016 ceph-mon[30947]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-12T01:58:42.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:58:42 smithi016 ceph-mon[30947]: osdmap e64: 11 total, 10 up, 11 in 2024-09-12T01:58:42.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:58:42 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T01:58:42.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:58:42 smithi016 ceph-mon[30947]: from='osd.10 [v2:172.21.15.33:6824/469977847,v1:172.21.15.33:6825/469977847]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-09-12T01:58:42.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:58:42 smithi016 ceph-mon[30947]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-09-12T01:58:42.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:58:42 smithi016 ceph-mon[30947]: Detected new or changed devices on smithi033 2024-09-12T01:58:42.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:58:42 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:58:42.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:58:42 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:58:42.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:58:42 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:58:42.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:58:42 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:58:42.435 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:58:42 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:58:42.435 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:58:42 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:58:42.435 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:58:42 smithi016 ceph-mon[30947]: Adjusting osd_memory_target on smithi033 to 4273M 2024-09-12T01:58:42.435 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:58:42 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:58:42.435 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:58:42 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:58:42.435 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:58:42 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:58:42.435 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:58:42 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:58:43.169 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 01:58:42 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T01:58:42.907+0000 7f78d329e640 -1 osd.10 0 waiting for initial osdmap 2024-09-12T01:58:43.169 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 01:58:42 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T01:58:42.928+0000 7f78cb8d0640 -1 osd.10 66 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T01:58:43.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:58:43 smithi033 ceph-mon[35332]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]': finished 2024-09-12T01:58:43.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:58:43 smithi033 ceph-mon[35332]: osdmap e65: 11 total, 10 up, 11 in 2024-09-12T01:58:43.170 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:58:43 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T01:58:43.170 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:58:43 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T01:58:43.170 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:58:43 smithi033 ceph-mon[35332]: from='client.24376 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi184:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:58:43.170 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:58:43 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T01:58:43.170 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:58:43 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T01:58:43.170 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:58:43 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:58:43.170 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:58:43 smithi033 ceph-mon[35332]: osdmap e66: 11 total, 10 up, 11 in 2024-09-12T01:58:43.170 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:58:43 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T01:58:43.170 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:58:43 smithi033 ceph-mon[35332]: from='osd.10 ' entity='osd.10' 2024-09-12T01:58:43.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:58:43 smithi184 ceph-mon[35593]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]': finished 2024-09-12T01:58:43.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:58:43 smithi184 ceph-mon[35593]: osdmap e65: 11 total, 10 up, 11 in 2024-09-12T01:58:43.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:58:43 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T01:58:43.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:58:43 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T01:58:43.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:58:43 smithi184 ceph-mon[35593]: from='client.24376 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi184:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:58:43.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:58:43 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T01:58:43.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:58:43 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T01:58:43.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:58:43 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:58:43.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:58:43 smithi184 ceph-mon[35593]: osdmap e66: 11 total, 10 up, 11 in 2024-09-12T01:58:43.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:58:43 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T01:58:43.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:58:43 smithi184 ceph-mon[35593]: from='osd.10 ' entity='osd.10' 2024-09-12T01:58:43.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:58:43 smithi016 ceph-mon[30947]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]': finished 2024-09-12T01:58:43.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:58:43 smithi016 ceph-mon[30947]: osdmap e65: 11 total, 10 up, 11 in 2024-09-12T01:58:43.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:58:43 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T01:58:43.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:58:43 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T01:58:43.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:58:43 smithi016 ceph-mon[30947]: from='client.24376 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi184:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:58:43.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:58:43 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T01:58:43.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:58:43 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T01:58:43.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:58:43 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:58:43.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:58:43 smithi016 ceph-mon[30947]: osdmap e66: 11 total, 10 up, 11 in 2024-09-12T01:58:43.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:58:43 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T01:58:43.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:58:43 smithi016 ceph-mon[30947]: from='osd.10 ' entity='osd.10' 2024-09-12T01:58:44.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:58:44 smithi184 ceph-mon[35593]: purged_snaps scrub starts 2024-09-12T01:58:44.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:58:44 smithi184 ceph-mon[35593]: purged_snaps scrub ok 2024-09-12T01:58:44.128 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:58:44 smithi184 ceph-mon[35593]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-12T01:58:44.128 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:58:44 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T01:58:44.128 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:58:44 smithi184 ceph-mon[35593]: osd.10 [v2:172.21.15.33:6824/469977847,v1:172.21.15.33:6825/469977847] boot 2024-09-12T01:58:44.128 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:58:44 smithi184 ceph-mon[35593]: osdmap e67: 11 total, 11 up, 11 in 2024-09-12T01:58:44.128 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:58:44 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T01:58:44.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:58:44 smithi033 ceph-mon[35332]: purged_snaps scrub starts 2024-09-12T01:58:44.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:58:44 smithi033 ceph-mon[35332]: purged_snaps scrub ok 2024-09-12T01:58:44.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:58:44 smithi033 ceph-mon[35332]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-12T01:58:44.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:58:44 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T01:58:44.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:58:44 smithi033 ceph-mon[35332]: osd.10 [v2:172.21.15.33:6824/469977847,v1:172.21.15.33:6825/469977847] boot 2024-09-12T01:58:44.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:58:44 smithi033 ceph-mon[35332]: osdmap e67: 11 total, 11 up, 11 in 2024-09-12T01:58:44.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:58:44 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T01:58:44.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:58:44 smithi016 ceph-mon[30947]: purged_snaps scrub starts 2024-09-12T01:58:44.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:58:44 smithi016 ceph-mon[30947]: purged_snaps scrub ok 2024-09-12T01:58:44.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:58:44 smithi016 ceph-mon[30947]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-12T01:58:44.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:58:44 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T01:58:44.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:58:44 smithi016 ceph-mon[30947]: osd.10 [v2:172.21.15.33:6824/469977847,v1:172.21.15.33:6825/469977847] boot 2024-09-12T01:58:44.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:58:44 smithi016 ceph-mon[30947]: osdmap e67: 11 total, 11 up, 11 in 2024-09-12T01:58:44.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:58:44 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T01:58:45.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:58:45 smithi184 ceph-mon[35593]: from='client.? 172.21.15.184:0/3790996656' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d60212f8-5a1d-45f7-9a1b-12294bb4c80d"}]: dispatch 2024-09-12T01:58:45.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:58:45 smithi184 ceph-mon[35593]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d60212f8-5a1d-45f7-9a1b-12294bb4c80d"}]: dispatch 2024-09-12T01:58:45.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:58:45 smithi184 ceph-mon[35593]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d60212f8-5a1d-45f7-9a1b-12294bb4c80d"}]': finished 2024-09-12T01:58:45.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:58:45 smithi184 ceph-mon[35593]: osdmap e68: 12 total, 11 up, 12 in 2024-09-12T01:58:45.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:58:45 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T01:58:45.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:58:45 smithi033 ceph-mon[35332]: from='client.? 172.21.15.184:0/3790996656' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d60212f8-5a1d-45f7-9a1b-12294bb4c80d"}]: dispatch 2024-09-12T01:58:45.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:58:45 smithi033 ceph-mon[35332]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d60212f8-5a1d-45f7-9a1b-12294bb4c80d"}]: dispatch 2024-09-12T01:58:45.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:58:45 smithi033 ceph-mon[35332]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d60212f8-5a1d-45f7-9a1b-12294bb4c80d"}]': finished 2024-09-12T01:58:45.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:58:45 smithi033 ceph-mon[35332]: osdmap e68: 12 total, 11 up, 12 in 2024-09-12T01:58:45.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:58:45 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T01:58:45.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:58:45 smithi016 ceph-mon[30947]: from='client.? 172.21.15.184:0/3790996656' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d60212f8-5a1d-45f7-9a1b-12294bb4c80d"}]: dispatch 2024-09-12T01:58:45.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:58:45 smithi016 ceph-mon[30947]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d60212f8-5a1d-45f7-9a1b-12294bb4c80d"}]: dispatch 2024-09-12T01:58:45.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:58:45 smithi016 ceph-mon[30947]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d60212f8-5a1d-45f7-9a1b-12294bb4c80d"}]': finished 2024-09-12T01:58:45.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:58:45 smithi016 ceph-mon[30947]: osdmap e68: 12 total, 11 up, 12 in 2024-09-12T01:58:45.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:58:45 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T01:58:46.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:58:46 smithi184 ceph-mon[35593]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-09-12T01:58:46.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:58:46 smithi184 ceph-mon[35593]: from='client.? 172.21.15.184:0/2197235720' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T01:58:46.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:58:46 smithi184 ceph-mon[35593]: osdmap e69: 12 total, 11 up, 12 in 2024-09-12T01:58:46.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:58:46 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T01:58:46.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:58:46 smithi033 ceph-mon[35332]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-09-12T01:58:46.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:58:46 smithi033 ceph-mon[35332]: from='client.? 172.21.15.184:0/2197235720' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T01:58:46.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:58:46 smithi033 ceph-mon[35332]: osdmap e69: 12 total, 11 up, 12 in 2024-09-12T01:58:46.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:58:46 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T01:58:46.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:58:46 smithi016 ceph-mon[30947]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-09-12T01:58:46.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:58:46 smithi016 ceph-mon[30947]: from='client.? 172.21.15.184:0/2197235720' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T01:58:46.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:58:46 smithi016 ceph-mon[30947]: osdmap e69: 12 total, 11 up, 12 in 2024-09-12T01:58:46.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:58:46 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T01:58:47.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:58:47 smithi184 ceph-mon[35593]: pgmap v228: 1 pgs: 1 remapped+peering; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-09-12T01:58:47.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:58:47 smithi184 ceph-mon[35593]: osdmap e70: 12 total, 11 up, 12 in 2024-09-12T01:58:47.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:58:47 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T01:58:47.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:58:47 smithi033 ceph-mon[35332]: pgmap v228: 1 pgs: 1 remapped+peering; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-09-12T01:58:47.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:58:47 smithi033 ceph-mon[35332]: osdmap e70: 12 total, 11 up, 12 in 2024-09-12T01:58:47.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:58:47 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T01:58:47.933 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:58:47 smithi016 ceph-mon[30947]: pgmap v228: 1 pgs: 1 remapped+peering; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-09-12T01:58:47.934 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:58:47 smithi016 ceph-mon[30947]: osdmap e70: 12 total, 11 up, 12 in 2024-09-12T01:58:47.934 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:58:47 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T01:58:48.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:58:48 smithi184 ceph-mon[35593]: osdmap e71: 12 total, 11 up, 12 in 2024-09-12T01:58:48.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:58:48 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T01:58:48.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:58:48 smithi033 ceph-mon[35332]: osdmap e71: 12 total, 11 up, 12 in 2024-09-12T01:58:48.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:58:48 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T01:58:48.933 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:58:48 smithi016 ceph-mon[30947]: osdmap e71: 12 total, 11 up, 12 in 2024-09-12T01:58:48.934 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:58:48 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T01:58:49.820 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:58:49 smithi033 ceph-mon[35332]: pgmap v231: 1 pgs: 1 remapped+peering; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-09-12T01:58:49.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:58:49 smithi184 ceph-mon[35593]: pgmap v231: 1 pgs: 1 remapped+peering; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-09-12T01:58:49.933 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:58:49 smithi016 ceph-mon[30947]: pgmap v231: 1 pgs: 1 remapped+peering; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-09-12T01:58:51.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:58:51 smithi184 ceph-mon[35593]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 100 KiB/s, 0 objects/s recovering 2024-09-12T01:58:51.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:58:51 smithi033 ceph-mon[35332]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 100 KiB/s, 0 objects/s recovering 2024-09-12T01:58:51.934 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:58:51 smithi016 ceph-mon[30947]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 100 KiB/s, 0 objects/s recovering 2024-09-12T01:58:53.628 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:58:53 smithi184 ceph-mon[35593]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-09-12T01:58:53.628 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:58:53 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-12T01:58:53.628 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:58:53 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:58:53.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:58:53 smithi033 ceph-mon[35332]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-09-12T01:58:53.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:58:53 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-12T01:58:53.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:58:53 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:58:53.933 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:58:53 smithi016 ceph-mon[30947]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-09-12T01:58:53.933 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:58:53 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-12T01:58:53.933 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:58:53 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:58:54.820 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:58:54 smithi184 ceph-mon[35593]: Deploying daemon osd.11 on smithi184 2024-09-12T01:58:54.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:58:54 smithi033 ceph-mon[35332]: Deploying daemon osd.11 on smithi184 2024-09-12T01:58:54.936 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:58:54 smithi016 ceph-mon[30947]: Deploying daemon osd.11 on smithi184 2024-09-12T01:58:55.743 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:58:55 smithi184 ceph-mon[35593]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-12T01:58:55.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:58:55 smithi033 ceph-mon[35332]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-12T01:58:55.933 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:58:55 smithi016 ceph-mon[30947]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-12T01:58:57.878 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:58:57 smithi184 ceph-mon[35593]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-09-12T01:58:57.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:58:57 smithi033 ceph-mon[35332]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-09-12T01:58:57.933 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:58:57 smithi016 ceph-mon[30947]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-09-12T01:58:59.878 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:58:59 smithi184 ceph-mon[35593]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-09-12T01:58:59.878 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:58:59 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:58:59.878 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:58:59 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:58:59.878 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:58:59 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:58:59.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:58:59 smithi033 ceph-mon[35332]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-09-12T01:58:59.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:58:59 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:58:59.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:58:59 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:58:59.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:58:59 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:58:59.933 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:58:59 smithi016 ceph-mon[30947]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-09-12T01:58:59.933 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:58:59 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:58:59.933 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:58:59 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:58:59.933 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:58:59 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:01.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:01 smithi184 ceph-mon[35593]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-12T01:59:01.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:01 smithi033 ceph-mon[35332]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-12T01:59:01.933 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:01 smithi016 ceph-mon[30947]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-12T01:59:02.162 INFO:teuthology.orchestra.run.smithi184.stdout:Created osd(s) 11 on host 'smithi184' 2024-09-12T01:59:02.163 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:59:02.162+0000 7f52fcff9640 1 -- 172.21.15.184:0/3857538822 <== mgr.14150 v2:172.21.15.16:6800/3887444820 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f5300062630 con 0x7f52ec0a2b80 2024-09-12T01:59:02.165 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:59:02.164+0000 7f5305369640 1 -- 172.21.15.184:0/3857538822 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f52ec0a2b80 msgr2=0x7f52ec0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:02.165 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:59:02.164+0000 7f5305369640 1 --2- 172.21.15.184:0/3857538822 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f52ec0a2b80 0x7f52ec0a5040 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f5300102c60 tx=0x7f52f000a560 comp rx=0 tx=0).stop 2024-09-12T01:59:02.165 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:59:02.164+0000 7f5305369640 1 -- 172.21.15.184:0/3857538822 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5300102e00 msgr2=0x7f53000745e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:02.165 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:59:02.164+0000 7f5305369640 1 --2- 172.21.15.184:0/3857538822 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5300102e00 0x7f53000745e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f52e8009230 tx=0x7f52e8004360 comp rx=0 tx=0).stop 2024-09-12T01:59:02.165 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:59:02.165+0000 7f5305369640 1 -- 172.21.15.184:0/3857538822 shutdown_connections 2024-09-12T01:59:02.165 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:59:02.165+0000 7f5305369640 1 --2- 172.21.15.184:0/3857538822 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f52ec0a2b80 0x7f52ec0a5040 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:02.166 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:59:02.165+0000 7f5305369640 1 --2- 172.21.15.184:0/3857538822 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53001037c0 0x7f5300074b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:02.166 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:59:02.165+0000 7f5305369640 1 --2- 172.21.15.184:0/3857538822 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5300102e00 0x7f53000745e0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:02.166 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:59:02.165+0000 7f5305369640 1 --2- 172.21.15.184:0/3857538822 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5300101c00 0x7f5300075f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:02.166 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:59:02.165+0000 7f5305369640 1 -- 172.21.15.184:0/3857538822 >> 172.21.15.184:0/3857538822 conn(0x7f53000fd390 msgr2=0x7f53001061d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:59:02.166 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:59:02.165+0000 7f5305369640 1 -- 172.21.15.184:0/3857538822 shutdown_connections 2024-09-12T01:59:02.166 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:59:02.165+0000 7f5305369640 1 -- 172.21.15.184:0/3857538822 wait complete. 2024-09-12T01:59:03.264 DEBUG:teuthology.orchestra.run.smithi184:osd.11> sudo journalctl -f -n 0 -u ceph-879f3b56-70a9-11ef-bcea-c7b262605968@osd.11.service 2024-09-12T01:59:03.267 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-09-12T01:59:03.267 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph osd stat -f json 2024-09-12T01:59:03.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:03 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:03.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:03 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:03.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:03 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:03.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:03 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:03.535 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:59:03.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:03 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:03.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:03 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:03.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:03.951+0000 7f376d24f640 1 -- 172.21.15.16:0/1719527791 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3768102e00 msgr2=0x7f3768103280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:03.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:03.951+0000 7f376d24f640 1 --2- 172.21.15.16:0/1719527791 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3768102e00 0x7f3768103280 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f3750009900 tx=0x7f375002a410 comp rx=0 tx=0).stop 2024-09-12T01:59:03.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:03.951+0000 7f376d24f640 1 -- 172.21.15.16:0/1719527791 shutdown_connections 2024-09-12T01:59:03.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:03.951+0000 7f376d24f640 1 --2- 172.21.15.16:0/1719527791 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f37681037c0 0x7f376810a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:03.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:03.951+0000 7f376d24f640 1 --2- 172.21.15.16:0/1719527791 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3768102e00 0x7f3768103280 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:03.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:03.951+0000 7f376d24f640 1 --2- 172.21.15.16:0/1719527791 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3768101c00 0x7f3768102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:03.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:03.951+0000 7f376d24f640 1 -- 172.21.15.16:0/1719527791 >> 172.21.15.16:0/1719527791 conn(0x7f37680fd390 msgr2=0x7f37680ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:59:03.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:03.951+0000 7f376d24f640 1 -- 172.21.15.16:0/1719527791 shutdown_connections 2024-09-12T01:59:03.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:03.951+0000 7f376d24f640 1 -- 172.21.15.16:0/1719527791 wait complete. 2024-09-12T01:59:03.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:03.952+0000 7f376d24f640 1 Processor -- start 2024-09-12T01:59:03.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:03.952+0000 7f376d24f640 1 -- start start 2024-09-12T01:59:03.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:03.952+0000 7f376d24f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3768101c00 0x7f3768075f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:03.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:03.952+0000 7f376d24f640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3768102e00 0x7f37680745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:03.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:03.952+0000 7f376d24f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f37681037c0 0x7f3768074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:03.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:03.952+0000 7f376d24f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f376810d880 con 0x7f3768101c00 2024-09-12T01:59:03.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:03.952+0000 7f376d24f640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f376810d700 con 0x7f37681037c0 2024-09-12T01:59:03.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:03.952+0000 7f376d24f640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f376810da00 con 0x7f3768102e00 2024-09-12T01:59:03.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:03.952+0000 7f3766d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3768101c00 0x7f3768075f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:03.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:03.952+0000 7f3766d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3768101c00 0x7f3768075f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35910/0 (socket says 172.21.15.16:35910) 2024-09-12T01:59:03.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:03.952+0000 7f3766d9d640 1 -- 172.21.15.16:0/2690005670 learned_addr learned my addr 172.21.15.16:0/2690005670 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:59:03.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:03.952+0000 7f375ffff640 1 --2- 172.21.15.16:0/2690005670 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3768102e00 0x7f37680745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:03.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:03.952+0000 7f376759e640 1 --2- 172.21.15.16:0/2690005670 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f37681037c0 0x7f3768074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:03.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:03.953+0000 7f3766d9d640 1 -- 172.21.15.16:0/2690005670 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3768102e00 msgr2=0x7f37680745e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:03.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:03.953+0000 7f3766d9d640 1 --2- 172.21.15.16:0/2690005670 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3768102e00 0x7f37680745e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:03.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:03.953+0000 7f3766d9d640 1 -- 172.21.15.16:0/2690005670 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f37681037c0 msgr2=0x7f3768074b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:03.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:03.953+0000 7f3766d9d640 1 --2- 172.21.15.16:0/2690005670 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f37681037c0 0x7f3768074b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:03.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:03.953+0000 7f3766d9d640 1 -- 172.21.15.16:0/2690005670 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37680753e0 con 0x7f3768101c00 2024-09-12T01:59:03.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:03.953+0000 7f375ffff640 1 --2- 172.21.15.16:0/2690005670 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3768102e00 0x7f37680745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:59:03.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:03.953+0000 7f3766d9d640 1 --2- 172.21.15.16:0/2690005670 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3768101c00 0x7f3768075f90 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f375800e9c0 tx=0x7f375800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:59:03.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:03.953+0000 7f3764d99640 1 -- 172.21.15.16:0/2690005670 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f375800cd80 con 0x7f3768101c00 2024-09-12T01:59:03.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:03.953+0000 7f376d24f640 1 -- 172.21.15.16:0/2690005670 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37681a20f0 con 0x7f3768101c00 2024-09-12T01:59:03.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:03.953+0000 7f3764d99640 1 -- 172.21.15.16:0/2690005670 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37580035f0 con 0x7f3768101c00 2024-09-12T01:59:03.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:03.953+0000 7f3764d99640 1 -- 172.21.15.16:0/2690005670 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3758010690 con 0x7f3768101c00 2024-09-12T01:59:03.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:03.953+0000 7f376d24f640 1 -- 172.21.15.16:0/2690005670 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37681a2600 con 0x7f3768101c00 2024-09-12T01:59:03.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:03.954+0000 7f376d24f640 1 -- 172.21.15.16:0/2690005670 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37480052c0 con 0x7f3768101c00 2024-09-12T01:59:03.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:03.956+0000 7f3764d99640 1 -- 172.21.15.16:0/2690005670 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f37580037a0 con 0x7f3768101c00 2024-09-12T01:59:03.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:03.957+0000 7f3764d99640 1 --2- 172.21.15.16:0/2690005670 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f37540a2b80 0x7f37540a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:03.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:03.957+0000 7f375ffff640 1 --2- 172.21.15.16:0/2690005670 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f37540a2b80 0x7f37540a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:03.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:03.957+0000 7f3764d99640 1 -- 172.21.15.16:0/2690005670 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 6809+0+0 (secure 0 0 0) 0x7f37580ceec0 con 0x7f3768101c00 2024-09-12T01:59:03.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:03.957+0000 7f375ffff640 1 --2- 172.21.15.16:0/2690005670 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f37540a2b80 0x7f37540a5040 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f37500314e0 tx=0x7f3750000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:59:03.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:03.964+0000 7f3764d99640 1 -- 172.21.15.16:0/2690005670 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f375809c360 con 0x7f3768101c00 2024-09-12T01:59:04.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:04.144+0000 7f376d24f640 1 -- 172.21.15.16:0/2690005670 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f3748005880 con 0x7f3768101c00 2024-09-12T01:59:04.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:04.144+0000 7f3764d99640 1 -- 172.21.15.16:0/2690005670 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v71) ==== 74+0+133 (secure 0 0 0) 0x7f375809bc60 con 0x7f3768101c00 2024-09-12T01:59:04.145 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:59:04.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:04.146+0000 7f376d24f640 1 -- 172.21.15.16:0/2690005670 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f37540a2b80 msgr2=0x7f37540a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:04.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:04.146+0000 7f376d24f640 1 --2- 172.21.15.16:0/2690005670 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f37540a2b80 0x7f37540a5040 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f37500314e0 tx=0x7f3750000bc0 comp rx=0 tx=0).stop 2024-09-12T01:59:04.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:04.147+0000 7f376d24f640 1 -- 172.21.15.16:0/2690005670 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3768101c00 msgr2=0x7f3768075f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:04.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:04.147+0000 7f376d24f640 1 --2- 172.21.15.16:0/2690005670 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3768101c00 0x7f3768075f90 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f375800e9c0 tx=0x7f375800ee80 comp rx=0 tx=0).stop 2024-09-12T01:59:04.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:04.147+0000 7f376d24f640 1 -- 172.21.15.16:0/2690005670 shutdown_connections 2024-09-12T01:59:04.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:04.147+0000 7f376d24f640 1 --2- 172.21.15.16:0/2690005670 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f37540a2b80 0x7f37540a5040 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:04.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:04.147+0000 7f376d24f640 1 --2- 172.21.15.16:0/2690005670 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f37681037c0 0x7f3768074b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:04.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:04.147+0000 7f376d24f640 1 --2- 172.21.15.16:0/2690005670 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3768102e00 0x7f37680745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:04.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:04.147+0000 7f376d24f640 1 --2- 172.21.15.16:0/2690005670 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3768101c00 0x7f3768075f90 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:04.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:04.147+0000 7f376d24f640 1 -- 172.21.15.16:0/2690005670 >> 172.21.15.16:0/2690005670 conn(0x7f37680fd390 msgr2=0x7f3768106290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:59:04.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:04.147+0000 7f376d24f640 1 -- 172.21.15.16:0/2690005670 shutdown_connections 2024-09-12T01:59:04.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:04.147+0000 7f376d24f640 1 -- 172.21.15.16:0/2690005670 wait complete. 2024-09-12T01:59:04.214 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:04 smithi016 ceph-mon[30947]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-09-12T01:59:04.214 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:04 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:04.215 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:04 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:04.215 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:04 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/2690005670' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-12T01:59:04.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:04 smithi033 ceph-mon[35332]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-09-12T01:59:04.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:04 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:04.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:04 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:04.420 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:04 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/2690005670' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-12T01:59:04.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:04 smithi184 ceph-mon[35593]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-09-12T01:59:04.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:04 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:04.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:04 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:04.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:04 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/2690005670' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-12T01:59:04.868 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":71,"num_osds":12,"num_up_osds":11,"osd_up_since":1726106323,"num_in_osds":12,"osd_in_since":1726106324,"num_remapped_pgs":0} 2024-09-12T01:59:05.378 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 01:59:05 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T01:59:05.280+0000 7f9d29287640 -1 osd.11 0 log_to_monitors true 2024-09-12T01:59:05.869 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph osd stat -f json 2024-09-12T01:59:06.139 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:59:06.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:06 smithi184 ceph-mon[35593]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-09-12T01:59:06.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:06 smithi184 ceph-mon[35593]: from='osd.11 [v2:172.21.15.184:6824/1619871346,v1:172.21.15.184:6825/1619871346]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-12T01:59:06.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:06 smithi184 ceph-mon[35593]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-12T01:59:06.424 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:06 smithi016 ceph-mon[30947]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-09-12T01:59:06.424 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:06 smithi016 ceph-mon[30947]: from='osd.11 [v2:172.21.15.184:6824/1619871346,v1:172.21.15.184:6825/1619871346]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-12T01:59:06.424 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:06 smithi016 ceph-mon[30947]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-12T01:59:06.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:06.612+0000 7f36ec065640 1 -- 172.21.15.16:0/389501535 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36e4101cd0 msgr2=0x7f36e4102150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:06.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:06.612+0000 7f36ec065640 1 --2- 172.21.15.16:0/389501535 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36e4101cd0 0x7f36e4102150 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f36d4009900 tx=0x7f36d402a410 comp rx=0 tx=0).stop 2024-09-12T01:59:06.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:06.612+0000 7f36ec065640 1 -- 172.21.15.16:0/389501535 shutdown_connections 2024-09-12T01:59:06.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:06.612+0000 7f36ec065640 1 --2- 172.21.15.16:0/389501535 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f36e4102690 0x7f36e4108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:06.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:06.612+0000 7f36ec065640 1 --2- 172.21.15.16:0/389501535 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36e4101cd0 0x7f36e4102150 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:06.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:06.612+0000 7f36ec065640 1 --2- 172.21.15.16:0/389501535 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f36e4100ad0 0x7f36e4100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:06.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:06.612+0000 7f36ec065640 1 -- 172.21.15.16:0/389501535 >> 172.21.15.16:0/389501535 conn(0x7f36e40fc280 msgr2=0x7f36e40fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:59:06.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:06.612+0000 7f36ec065640 1 -- 172.21.15.16:0/389501535 shutdown_connections 2024-09-12T01:59:06.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:06.612+0000 7f36ec065640 1 -- 172.21.15.16:0/389501535 wait complete. 2024-09-12T01:59:06.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:06.612+0000 7f36ec065640 1 Processor -- start 2024-09-12T01:59:06.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:06.613+0000 7f36ec065640 1 -- start start 2024-09-12T01:59:06.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:06.613+0000 7f36ec065640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36e4100ad0 0x7f36e4197f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:06.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:06.613+0000 7f36ec065640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f36e4101cd0 0x7f36e4198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:06.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:06.613+0000 7f36ec065640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f36e4102690 0x7f36e419f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:06.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:06.613+0000 7f36ec065640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f36e410c730 con 0x7f36e4100ad0 2024-09-12T01:59:06.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:06.613+0000 7f36ec065640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f36e410c5b0 con 0x7f36e4101cd0 2024-09-12T01:59:06.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:06.613+0000 7f36ec065640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f36e410c8b0 con 0x7f36e4102690 2024-09-12T01:59:06.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:06.613+0000 7f36e9e01640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36e4100ad0 0x7f36e4197f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:06.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:06.613+0000 7f36e9e01640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36e4100ad0 0x7f36e4197f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35932/0 (socket says 172.21.15.16:35932) 2024-09-12T01:59:06.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:06.613+0000 7f36e9e01640 1 -- 172.21.15.16:0/581239012 learned_addr learned my addr 172.21.15.16:0/581239012 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:59:06.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:06.613+0000 7f36ea602640 1 --2- 172.21.15.16:0/581239012 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f36e4102690 0x7f36e419f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:06.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:06.613+0000 7f36e9600640 1 --2- 172.21.15.16:0/581239012 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f36e4101cd0 0x7f36e4198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:06.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:06.613+0000 7f36e9e01640 1 -- 172.21.15.16:0/581239012 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f36e4102690 msgr2=0x7f36e419f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:06.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:06.613+0000 7f36e9e01640 1 --2- 172.21.15.16:0/581239012 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f36e4102690 0x7f36e419f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:06.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:06.613+0000 7f36e9e01640 1 -- 172.21.15.16:0/581239012 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f36e4101cd0 msgr2=0x7f36e4198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:06.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:06.613+0000 7f36e9e01640 1 --2- 172.21.15.16:0/581239012 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f36e4101cd0 0x7f36e4198450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:06.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:06.613+0000 7f36e9e01640 1 -- 172.21.15.16:0/581239012 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36e419fb40 con 0x7f36e4100ad0 2024-09-12T01:59:06.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:06.613+0000 7f36e9e01640 1 --2- 172.21.15.16:0/581239012 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36e4100ad0 0x7f36e4197f10 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f36e000d950 tx=0x7f36e000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:59:06.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:06.614+0000 7f36daffd640 1 -- 172.21.15.16:0/581239012 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36e0017020 con 0x7f36e4100ad0 2024-09-12T01:59:06.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:06.614+0000 7f36ec065640 1 -- 172.21.15.16:0/581239012 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36e419fe30 con 0x7f36e4100ad0 2024-09-12T01:59:06.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:06.614+0000 7f36daffd640 1 -- 172.21.15.16:0/581239012 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36e0009c30 con 0x7f36e4100ad0 2024-09-12T01:59:06.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:06.614+0000 7f36daffd640 1 -- 172.21.15.16:0/581239012 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36e000f740 con 0x7f36e4100ad0 2024-09-12T01:59:06.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:06.614+0000 7f36ec065640 1 -- 172.21.15.16:0/581239012 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36e41a0280 con 0x7f36e4100ad0 2024-09-12T01:59:06.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:06.615+0000 7f36ec065640 1 -- 172.21.15.16:0/581239012 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36e41a0510 con 0x7f36e4100ad0 2024-09-12T01:59:06.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:06.616+0000 7f36daffd640 1 -- 172.21.15.16:0/581239012 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f36e001a020 con 0x7f36e4100ad0 2024-09-12T01:59:06.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:06.617+0000 7f36daffd640 1 --2- 172.21.15.16:0/581239012 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f36d00a2bb0 0x7f36d00a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:06.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:06.617+0000 7f36daffd640 1 -- 172.21.15.16:0/581239012 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 6831+0+0 (secure 0 0 0) 0x7f36e00ce3d0 con 0x7f36e4100ad0 2024-09-12T01:59:06.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:06.617+0000 7f36e9600640 1 --2- 172.21.15.16:0/581239012 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f36d00a2bb0 0x7f36d00a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:06.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:06.617+0000 7f36e9600640 1 --2- 172.21.15.16:0/581239012 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f36d00a2bb0 0x7f36d00a5070 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f36d40314e0 tx=0x7f36d4000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:59:06.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:06.621+0000 7f36daffd640 1 -- 172.21.15.16:0/581239012 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f36e0021070 con 0x7f36e4100ad0 2024-09-12T01:59:06.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:06 smithi033 ceph-mon[35332]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-09-12T01:59:06.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:06 smithi033 ceph-mon[35332]: from='osd.11 [v2:172.21.15.184:6824/1619871346,v1:172.21.15.184:6825/1619871346]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-12T01:59:06.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:06 smithi033 ceph-mon[35332]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-12T01:59:06.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:06.798+0000 7f36ec065640 1 -- 172.21.15.16:0/581239012 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f36e41a0780 con 0x7f36e4100ad0 2024-09-12T01:59:06.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:06.798+0000 7f36daffd640 1 -- 172.21.15.16:0/581239012 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v72) ==== 74+0+133 (secure 0 0 0) 0x7f36e0009de0 con 0x7f36e4100ad0 2024-09-12T01:59:06.799 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:59:06.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:06.800+0000 7f36ec065640 1 -- 172.21.15.16:0/581239012 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f36d00a2bb0 msgr2=0x7f36d00a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:06.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:06.800+0000 7f36ec065640 1 --2- 172.21.15.16:0/581239012 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f36d00a2bb0 0x7f36d00a5070 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f36d40314e0 tx=0x7f36d4000bc0 comp rx=0 tx=0).stop 2024-09-12T01:59:06.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:06.800+0000 7f36ec065640 1 -- 172.21.15.16:0/581239012 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36e4100ad0 msgr2=0x7f36e4197f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:06.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:06.800+0000 7f36ec065640 1 --2- 172.21.15.16:0/581239012 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36e4100ad0 0x7f36e4197f10 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f36e000d950 tx=0x7f36e000de10 comp rx=0 tx=0).stop 2024-09-12T01:59:06.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:06.800+0000 7f36ec065640 1 -- 172.21.15.16:0/581239012 shutdown_connections 2024-09-12T01:59:06.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:06.800+0000 7f36ec065640 1 --2- 172.21.15.16:0/581239012 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f36d00a2bb0 0x7f36d00a5070 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:06.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:06.800+0000 7f36ec065640 1 --2- 172.21.15.16:0/581239012 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f36e4102690 0x7f36e419f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:06.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:06.800+0000 7f36ec065640 1 --2- 172.21.15.16:0/581239012 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f36e4101cd0 0x7f36e4198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:06.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:06.800+0000 7f36ec065640 1 --2- 172.21.15.16:0/581239012 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36e4100ad0 0x7f36e4197f10 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:06.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:06.800+0000 7f36ec065640 1 -- 172.21.15.16:0/581239012 >> 172.21.15.16:0/581239012 conn(0x7f36e40fc280 msgr2=0x7f36e4069f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:59:06.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:06.800+0000 7f36ec065640 1 -- 172.21.15.16:0/581239012 shutdown_connections 2024-09-12T01:59:06.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:06.800+0000 7f36ec065640 1 -- 172.21.15.16:0/581239012 wait complete. 2024-09-12T01:59:07.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:07 smithi016 ceph-mon[30947]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-12T01:59:07.435 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:07 smithi016 ceph-mon[30947]: osdmap e72: 12 total, 11 up, 12 in 2024-09-12T01:59:07.435 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:07 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T01:59:07.435 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:07 smithi016 ceph-mon[30947]: from='osd.11 [v2:172.21.15.184:6824/1619871346,v1:172.21.15.184:6825/1619871346]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2024-09-12T01:59:07.435 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:07 smithi016 ceph-mon[30947]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2024-09-12T01:59:07.435 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:07 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:07.435 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:07 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:07.436 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:07 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:59:07.436 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:07 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:59:07.436 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:07 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:59:07.436 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:07 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:59:07.436 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:07 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:07.436 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:07 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:59:07.436 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:07 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:59:07.436 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:07 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:07.436 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:07 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:59:07.436 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:07 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:59:07.436 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:07 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:59:07.437 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:07 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:07.437 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:07 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/581239012' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-12T01:59:07.526 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":72,"num_osds":12,"num_up_osds":11,"osd_up_since":1726106323,"num_in_osds":12,"osd_in_since":1726106324,"num_remapped_pgs":0} 2024-09-12T01:59:07.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:07 smithi184 ceph-mon[35593]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-12T01:59:07.628 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:07 smithi184 ceph-mon[35593]: osdmap e72: 12 total, 11 up, 12 in 2024-09-12T01:59:07.628 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:07 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T01:59:07.628 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:07 smithi184 ceph-mon[35593]: from='osd.11 [v2:172.21.15.184:6824/1619871346,v1:172.21.15.184:6825/1619871346]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2024-09-12T01:59:07.628 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:07 smithi184 ceph-mon[35593]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2024-09-12T01:59:07.628 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:07 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:07.628 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:07 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:07.628 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:07 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:59:07.628 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:07 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:59:07.628 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:07 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:59:07.628 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:07 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:59:07.628 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:07 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:07.628 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:07 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:59:07.629 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:07 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:59:07.629 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:07 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:07.629 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:07 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:59:07.629 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:07 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:59:07.629 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:07 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:59:07.629 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:07 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:07.629 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:07 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/581239012' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-12T01:59:07.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:07 smithi033 ceph-mon[35332]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-12T01:59:07.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:07 smithi033 ceph-mon[35332]: osdmap e72: 12 total, 11 up, 12 in 2024-09-12T01:59:07.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:07 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T01:59:07.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:07 smithi033 ceph-mon[35332]: from='osd.11 [v2:172.21.15.184:6824/1619871346,v1:172.21.15.184:6825/1619871346]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2024-09-12T01:59:07.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:07 smithi033 ceph-mon[35332]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2024-09-12T01:59:07.670 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:07 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:07.670 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:07 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:07.670 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:07 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:59:07.670 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:07 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:59:07.670 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:07 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:59:07.671 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:07 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:59:07.671 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:07 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:07.671 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:07 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:59:07.671 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:07 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:59:07.671 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:07 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:07.671 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:07 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:59:07.672 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:07 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:59:07.672 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:07 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:59:07.672 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:07 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:07.672 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:07 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/581239012' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-12T01:59:08.377 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 01:59:07 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T01:59:07.972+0000 7f9d249e7640 -1 osd.11 0 waiting for initial osdmap 2024-09-12T01:59:08.377 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 01:59:07 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T01:59:07.989+0000 7f9d1d019640 -1 osd.11 73 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T01:59:08.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:08 smithi184 ceph-mon[35593]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-09-12T01:59:08.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:08 smithi184 ceph-mon[35593]: Detected new or changed devices on smithi184 2024-09-12T01:59:08.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:08 smithi184 ceph-mon[35593]: Adjusting osd_memory_target on smithi184 to 4272M 2024-09-12T01:59:08.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:08 smithi184 ceph-mon[35593]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]': finished 2024-09-12T01:59:08.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:08 smithi184 ceph-mon[35593]: osdmap e73: 12 total, 11 up, 12 in 2024-09-12T01:59:08.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:08 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T01:59:08.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:08 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T01:59:08.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:08 smithi184 ceph-mon[35593]: from='osd.11 ' entity='osd.11' 2024-09-12T01:59:08.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:08 smithi016 ceph-mon[30947]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-09-12T01:59:08.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:08 smithi016 ceph-mon[30947]: Detected new or changed devices on smithi184 2024-09-12T01:59:08.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:08 smithi016 ceph-mon[30947]: Adjusting osd_memory_target on smithi184 to 4272M 2024-09-12T01:59:08.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:08 smithi016 ceph-mon[30947]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]': finished 2024-09-12T01:59:08.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:08 smithi016 ceph-mon[30947]: osdmap e73: 12 total, 11 up, 12 in 2024-09-12T01:59:08.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:08 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T01:59:08.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:08 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T01:59:08.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:08 smithi016 ceph-mon[30947]: from='osd.11 ' entity='osd.11' 2024-09-12T01:59:08.528 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph osd stat -f json 2024-09-12T01:59:08.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:08 smithi033 ceph-mon[35332]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-09-12T01:59:08.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:08 smithi033 ceph-mon[35332]: Detected new or changed devices on smithi184 2024-09-12T01:59:08.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:08 smithi033 ceph-mon[35332]: Adjusting osd_memory_target on smithi184 to 4272M 2024-09-12T01:59:08.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:08 smithi033 ceph-mon[35332]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]': finished 2024-09-12T01:59:08.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:08 smithi033 ceph-mon[35332]: osdmap e73: 12 total, 11 up, 12 in 2024-09-12T01:59:08.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:08 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T01:59:08.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:08 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T01:59:08.670 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:08 smithi033 ceph-mon[35332]: from='osd.11 ' entity='osd.11' 2024-09-12T01:59:08.796 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:59:09.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:09.280+0000 7fbef4a51640 1 -- 172.21.15.16:0/3121790389 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbef01037d0 msgr2=0x7fbef010a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:09.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:09.280+0000 7fbef4a51640 1 --2- 172.21.15.16:0/3121790389 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbef01037d0 0x7fbef010a060 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fbee8009ca0 tx=0x7fbee802a8c0 comp rx=0 tx=0).stop 2024-09-12T01:59:09.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:09.280+0000 7fbef4a51640 1 -- 172.21.15.16:0/3121790389 shutdown_connections 2024-09-12T01:59:09.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:09.280+0000 7fbef4a51640 1 --2- 172.21.15.16:0/3121790389 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbef01037d0 0x7fbef010a060 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:09.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:09.280+0000 7fbef4a51640 1 --2- 172.21.15.16:0/3121790389 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbef0102e10 0x7fbef0103290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:09.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:09.280+0000 7fbef4a51640 1 --2- 172.21.15.16:0/3121790389 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbef0101c10 0x7fbef0102010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:09.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:09.280+0000 7fbef4a51640 1 -- 172.21.15.16:0/3121790389 >> 172.21.15.16:0/3121790389 conn(0x7fbef00fd3c0 msgr2=0x7fbef00ff7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:59:09.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:09.280+0000 7fbef4a51640 1 -- 172.21.15.16:0/3121790389 shutdown_connections 2024-09-12T01:59:09.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:09.280+0000 7fbef4a51640 1 -- 172.21.15.16:0/3121790389 wait complete. 2024-09-12T01:59:09.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:09.281+0000 7fbef4a51640 1 Processor -- start 2024-09-12T01:59:09.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:09.281+0000 7fbef4a51640 1 -- start start 2024-09-12T01:59:09.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:09.281+0000 7fbef4a51640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbef0101c10 0x7fbef0197ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:09.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:09.281+0000 7fbef4a51640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbef0102e10 0x7fbef0198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:09.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:09.281+0000 7fbef4a51640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbef01037d0 0x7fbef019f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:09.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:09.281+0000 7fbef4a51640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbef010d890 con 0x7fbef01037d0 2024-09-12T01:59:09.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:09.281+0000 7fbef4a51640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbef010d710 con 0x7fbef0101c10 2024-09-12T01:59:09.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:09.281+0000 7fbef4a51640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fbef010da10 con 0x7fbef0102e10 2024-09-12T01:59:09.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:09.281+0000 7fbeeed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbef01037d0 0x7fbef019f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:09.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:09.281+0000 7fbeeed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbef01037d0 0x7fbef019f480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35948/0 (socket says 172.21.15.16:35948) 2024-09-12T01:59:09.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:09.281+0000 7fbeeed9d640 1 -- 172.21.15.16:0/798054929 learned_addr learned my addr 172.21.15.16:0/798054929 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:59:09.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:09.281+0000 7fbee7fff640 1 --2- 172.21.15.16:0/798054929 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbef0102e10 0x7fbef0198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:09.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:09.281+0000 7fbeee59c640 1 --2- 172.21.15.16:0/798054929 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbef0101c10 0x7fbef0197ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:09.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:09.281+0000 7fbeeed9d640 1 -- 172.21.15.16:0/798054929 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbef0102e10 msgr2=0x7fbef0198400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:09.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:09.281+0000 7fbeeed9d640 1 --2- 172.21.15.16:0/798054929 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbef0102e10 0x7fbef0198400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:09.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:09.281+0000 7fbeeed9d640 1 -- 172.21.15.16:0/798054929 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbef0101c10 msgr2=0x7fbef0197ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:09.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:09.281+0000 7fbeeed9d640 1 --2- 172.21.15.16:0/798054929 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbef0101c10 0x7fbef0197ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:09.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:09.281+0000 7fbeeed9d640 1 -- 172.21.15.16:0/798054929 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbef019faf0 con 0x7fbef01037d0 2024-09-12T01:59:09.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:09.282+0000 7fbeeed9d640 1 --2- 172.21.15.16:0/798054929 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbef01037d0 0x7fbef019f480 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fbee8005e30 tx=0x7fbee8031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:59:09.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:09.282+0000 7fbee77fe640 1 -- 172.21.15.16:0/798054929 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbee8003c30 con 0x7fbef01037d0 2024-09-12T01:59:09.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:09.282+0000 7fbee77fe640 1 -- 172.21.15.16:0/798054929 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbee8003dd0 con 0x7fbef01037d0 2024-09-12T01:59:09.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:09.282+0000 7fbef4a51640 1 -- 172.21.15.16:0/798054929 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbef019fd80 con 0x7fbef01037d0 2024-09-12T01:59:09.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:09.282+0000 7fbee77fe640 1 -- 172.21.15.16:0/798054929 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbee802fde0 con 0x7fbef01037d0 2024-09-12T01:59:09.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:09.282+0000 7fbef4a51640 1 -- 172.21.15.16:0/798054929 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbef01a0290 con 0x7fbef01037d0 2024-09-12T01:59:09.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:09.283+0000 7fbef4a51640 1 -- 172.21.15.16:0/798054929 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbef0108fc0 con 0x7fbef01037d0 2024-09-12T01:59:09.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:09.284+0000 7fbee77fe640 1 -- 172.21.15.16:0/798054929 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fbee8004050 con 0x7fbef01037d0 2024-09-12T01:59:09.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:09.285+0000 7fbee77fe640 1 --2- 172.21.15.16:0/798054929 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fbedc0a2cb0 0x7fbedc0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:09.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:09.285+0000 7fbeee59c640 1 --2- 172.21.15.16:0/798054929 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fbedc0a2cb0 0x7fbedc0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:09.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:09.285+0000 7fbee77fe640 1 -- 172.21.15.16:0/798054929 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7fbee80efd00 con 0x7fbef01037d0 2024-09-12T01:59:09.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:09.285+0000 7fbeee59c640 1 --2- 172.21.15.16:0/798054929 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fbedc0a2cb0 0x7fbedc0a5170 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fbef0102c70 tx=0x7fbee000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:59:09.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:09.289+0000 7fbee77fe640 1 -- 172.21.15.16:0/798054929 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbee80bd060 con 0x7fbef01037d0 2024-09-12T01:59:09.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:09.464+0000 7fbef4a51640 1 -- 172.21.15.16:0/798054929 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7fbef0062e90 con 0x7fbef01037d0 2024-09-12T01:59:09.465 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:09 smithi016 ceph-mon[30947]: purged_snaps scrub starts 2024-09-12T01:59:09.465 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:09 smithi016 ceph-mon[30947]: purged_snaps scrub ok 2024-09-12T01:59:09.465 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:09 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T01:59:09.465 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:09 smithi016 ceph-mon[30947]: osd.11 [v2:172.21.15.184:6824/1619871346,v1:172.21.15.184:6825/1619871346] boot 2024-09-12T01:59:09.465 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:09 smithi016 ceph-mon[30947]: osdmap e74: 12 total, 12 up, 12 in 2024-09-12T01:59:09.465 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:09 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T01:59:09.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:09.465+0000 7fbee77fe640 1 -- 172.21.15.16:0/798054929 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v74) ==== 74+0+133 (secure 0 0 0) 0x7fbee80bc960 con 0x7fbef01037d0 2024-09-12T01:59:09.466 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:59:09.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:09.467+0000 7fbef4a51640 1 -- 172.21.15.16:0/798054929 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fbedc0a2cb0 msgr2=0x7fbedc0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:09.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:09.467+0000 7fbef4a51640 1 --2- 172.21.15.16:0/798054929 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fbedc0a2cb0 0x7fbedc0a5170 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fbef0102c70 tx=0x7fbee000b390 comp rx=0 tx=0).stop 2024-09-12T01:59:09.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:09.467+0000 7fbef4a51640 1 -- 172.21.15.16:0/798054929 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbef01037d0 msgr2=0x7fbef019f480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:09.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:09.467+0000 7fbef4a51640 1 --2- 172.21.15.16:0/798054929 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbef01037d0 0x7fbef019f480 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fbee8005e30 tx=0x7fbee8031ee0 comp rx=0 tx=0).stop 2024-09-12T01:59:09.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:09.467+0000 7fbef4a51640 1 -- 172.21.15.16:0/798054929 shutdown_connections 2024-09-12T01:59:09.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:09.467+0000 7fbef4a51640 1 --2- 172.21.15.16:0/798054929 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fbedc0a2cb0 0x7fbedc0a5170 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:09.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:09.467+0000 7fbef4a51640 1 --2- 172.21.15.16:0/798054929 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbef01037d0 0x7fbef019f480 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:09.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:09.467+0000 7fbef4a51640 1 --2- 172.21.15.16:0/798054929 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbef0102e10 0x7fbef0198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:09.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:09.467+0000 7fbef4a51640 1 --2- 172.21.15.16:0/798054929 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbef0101c10 0x7fbef0197ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:09.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:09.467+0000 7fbef4a51640 1 -- 172.21.15.16:0/798054929 >> 172.21.15.16:0/798054929 conn(0x7fbef00fd3c0 msgr2=0x7fbef0106190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:59:09.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:09.467+0000 7fbef4a51640 1 -- 172.21.15.16:0/798054929 shutdown_connections 2024-09-12T01:59:09.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:09.467+0000 7fbef4a51640 1 -- 172.21.15.16:0/798054929 wait complete. 2024-09-12T01:59:09.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:09 smithi184 ceph-mon[35593]: purged_snaps scrub starts 2024-09-12T01:59:09.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:09 smithi184 ceph-mon[35593]: purged_snaps scrub ok 2024-09-12T01:59:09.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:09 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T01:59:09.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:09 smithi184 ceph-mon[35593]: osd.11 [v2:172.21.15.184:6824/1619871346,v1:172.21.15.184:6825/1619871346] boot 2024-09-12T01:59:09.628 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:09 smithi184 ceph-mon[35593]: osdmap e74: 12 total, 12 up, 12 in 2024-09-12T01:59:09.628 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:09 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T01:59:09.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:09 smithi033 ceph-mon[35332]: purged_snaps scrub starts 2024-09-12T01:59:09.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:09 smithi033 ceph-mon[35332]: purged_snaps scrub ok 2024-09-12T01:59:09.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:09 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T01:59:09.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:09 smithi033 ceph-mon[35332]: osd.11 [v2:172.21.15.184:6824/1619871346,v1:172.21.15.184:6825/1619871346] boot 2024-09-12T01:59:09.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:09 smithi033 ceph-mon[35332]: osdmap e74: 12 total, 12 up, 12 in 2024-09-12T01:59:09.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:09 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T01:59:10.104 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":74,"num_osds":12,"num_up_osds":12,"osd_up_since":1726106348,"num_in_osds":12,"osd_in_since":1726106324,"num_remapped_pgs":0} 2024-09-12T01:59:10.105 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T01:59:10.370 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:59:10.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:10 smithi016 ceph-mon[30947]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-09-12T01:59:10.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:10 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/798054929' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-12T01:59:10.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:10 smithi184 ceph-mon[35593]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-09-12T01:59:10.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:10 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/798054929' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-12T01:59:10.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:10 smithi033 ceph-mon[35332]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-09-12T01:59:10.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:10 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/798054929' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-12T01:59:10.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:10.838+0000 7f26eb2fd640 1 -- 172.21.15.16:0/628057792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26e4102d40 msgr2=0x7f26e4103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:10.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:10.838+0000 7f26eb2fd640 1 --2- 172.21.15.16:0/628057792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26e4102d40 0x7f26e4103120 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f26e0009900 tx=0x7f26e002a480 comp rx=0 tx=0).stop 2024-09-12T01:59:10.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:10.838+0000 7f26eb2fd640 1 -- 172.21.15.16:0/628057792 shutdown_connections 2024-09-12T01:59:10.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:10.838+0000 7f26eb2fd640 1 --2- 172.21.15.16:0/628057792 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f26e410c170 0x7f26e410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:10.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:10.838+0000 7f26eb2fd640 1 --2- 172.21.15.16:0/628057792 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26e4103660 0x7f26e410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:10.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:10.838+0000 7f26eb2fd640 1 --2- 172.21.15.16:0/628057792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26e4102d40 0x7f26e4103120 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:10.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:10.838+0000 7f26eb2fd640 1 -- 172.21.15.16:0/628057792 >> 172.21.15.16:0/628057792 conn(0x7f26e40fc7a0 msgr2=0x7f26e40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:59:10.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:10.838+0000 7f26eb2fd640 1 -- 172.21.15.16:0/628057792 shutdown_connections 2024-09-12T01:59:10.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:10.838+0000 7f26eb2fd640 1 -- 172.21.15.16:0/628057792 wait complete. 2024-09-12T01:59:10.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:10.839+0000 7f26eb2fd640 1 Processor -- start 2024-09-12T01:59:10.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:10.839+0000 7f26eb2fd640 1 -- start start 2024-09-12T01:59:10.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:10.839+0000 7f26eb2fd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26e4102d40 0x7f26e41984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:10.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:10.839+0000 7f26eb2fd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26e4103660 0x7f26e41989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:10.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:10.839+0000 7f26eb2fd640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f26e410c170 0x7f26e419cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:10.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:10.839+0000 7f26eb2fd640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f26e4111fd0 con 0x7f26e4102d40 2024-09-12T01:59:10.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:10.839+0000 7f26eb2fd640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f26e4111e50 con 0x7f26e4103660 2024-09-12T01:59:10.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:10.839+0000 7f26eb2fd640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f26e4112150 con 0x7f26e410c170 2024-09-12T01:59:10.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:10.839+0000 7f26e9099640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26e4102d40 0x7f26e41984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:10.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:10.839+0000 7f26e9099640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26e4102d40 0x7f26e41984a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35966/0 (socket says 172.21.15.16:35966) 2024-09-12T01:59:10.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:10.839+0000 7f26e9099640 1 -- 172.21.15.16:0/3131872929 learned_addr learned my addr 172.21.15.16:0/3131872929 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:59:10.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:10.839+0000 7f26e989a640 1 --2- 172.21.15.16:0/3131872929 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f26e410c170 0x7f26e419cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:10.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:10.840+0000 7f26e8898640 1 --2- 172.21.15.16:0/3131872929 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26e4103660 0x7f26e41989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:10.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:10.840+0000 7f26e9099640 1 -- 172.21.15.16:0/3131872929 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f26e410c170 msgr2=0x7f26e419cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:10.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:10.840+0000 7f26e9099640 1 --2- 172.21.15.16:0/3131872929 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f26e410c170 0x7f26e419cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:10.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:10.840+0000 7f26e9099640 1 -- 172.21.15.16:0/3131872929 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26e4103660 msgr2=0x7f26e41989e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:10.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:10.840+0000 7f26e9099640 1 --2- 172.21.15.16:0/3131872929 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26e4103660 0x7f26e41989e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:10.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:10.840+0000 7f26e9099640 1 -- 172.21.15.16:0/3131872929 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26e419d450 con 0x7f26e4102d40 2024-09-12T01:59:10.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:10.840+0000 7f26e8898640 1 --2- 172.21.15.16:0/3131872929 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26e4103660 0x7f26e41989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:59:10.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:10.840+0000 7f26e9099640 1 --2- 172.21.15.16:0/3131872929 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26e4102d40 0x7f26e41984a0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f26e0005ab0 tx=0x7f26e00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:59:10.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:10.840+0000 7f26da7fc640 1 -- 172.21.15.16:0/3131872929 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26e003f030 con 0x7f26e4102d40 2024-09-12T01:59:10.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:10.840+0000 7f26da7fc640 1 -- 172.21.15.16:0/3131872929 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26e0003ce0 con 0x7f26e4102d40 2024-09-12T01:59:10.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:10.840+0000 7f26eb2fd640 1 -- 172.21.15.16:0/3131872929 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26e419d6e0 con 0x7f26e4102d40 2024-09-12T01:59:10.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:10.840+0000 7f26da7fc640 1 -- 172.21.15.16:0/3131872929 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26e0035070 con 0x7f26e4102d40 2024-09-12T01:59:10.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:10.840+0000 7f26eb2fd640 1 -- 172.21.15.16:0/3131872929 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26e41a5020 con 0x7f26e4102d40 2024-09-12T01:59:10.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:10.841+0000 7f26eb2fd640 1 -- 172.21.15.16:0/3131872929 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26e406b070 con 0x7f26e4102d40 2024-09-12T01:59:10.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:10.842+0000 7f26da7fc640 1 -- 172.21.15.16:0/3131872929 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f26e0042030 con 0x7f26e4102d40 2024-09-12T01:59:10.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:10.843+0000 7f26da7fc640 1 --2- 172.21.15.16:0/3131872929 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f26d00a2b80 0x7f26d00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:10.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:10.843+0000 7f26e8898640 1 --2- 172.21.15.16:0/3131872929 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f26d00a2b80 0x7f26d00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:10.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:10.843+0000 7f26da7fc640 1 -- 172.21.15.16:0/3131872929 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7f26e00ef490 con 0x7f26e4102d40 2024-09-12T01:59:10.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:10.843+0000 7f26e8898640 1 --2- 172.21.15.16:0/3131872929 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f26d00a2b80 0x7f26d00a5040 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f26d4006fd0 tx=0x7f26d4008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:59:10.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:10.847+0000 7f26da7fc640 1 -- 172.21.15.16:0/3131872929 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f26e00bc890 con 0x7f26e4102d40 2024-09-12T01:59:11.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:11.023+0000 7f26eb2fd640 1 -- 172.21.15.16:0/3131872929 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f26e4107210 con 0x7f26e4102d40 2024-09-12T01:59:11.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:11.024+0000 7f26da7fc640 1 -- 172.21.15.16:0/3131872929 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) ==== 74+0+18434 (secure 0 0 0) 0x7f26e00bc190 con 0x7f26e4102d40 2024-09-12T01:59:11.025 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:59:11.025 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":75,"fsid":"879f3b56-70a9-11ef-bcea-c7b262605968","created":"2024-09-12T01:52:12.841045+0000","modified":"2024-09-12T01:59:10.185525+0000","last_up_change":"2024-09-12T01:59:08.975443+0000","last_in_change":"2024-09-12T01:58:44.530220+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T01:55:26.727564+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3ea41eb6-e384-4ddc-810f-0f103c127849","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4000187867},{"type":"v1","addr":"172.21.15.16:6803","nonce":4000187867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4000187867},{"type":"v1","addr":"172.21.15.16:6805","nonce":4000187867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":4000187867},{"type":"v1","addr":"172.21.15.16:6809","nonce":4000187867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4000187867},{"type":"v1","addr":"172.21.15.16:6807","nonce":4000187867}]},"public_addr":"172.21.15.16:6803/4000187867","cluster_addr":"172.21.15.16:6805/4000187867","heartbeat_back_addr":"172.21.15.16:6809/4000187867","heartbeat_front_addr":"172.21.15.16:6807/4000187867","state":["exists","up"]},{"osd":1,"uuid":"07bf63e7-70fa-41b2-af53-12a5fec3f33c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":3862661148},{"type":"v1","addr":"172.21.15.33:6801","nonce":3862661148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":3862661148},{"type":"v1","addr":"172.21.15.33:6803","nonce":3862661148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":3862661148},{"type":"v1","addr":"172.21.15.33:6807","nonce":3862661148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":3862661148},{"type":"v1","addr":"172.21.15.33:6805","nonce":3862661148}]},"public_addr":"172.21.15.33:6801/3862661148","cluster_addr":"172.21.15.33:6803/3862661148","heartbeat_back_addr":"172.21.15.33:6807/3862661148","heartbeat_front_addr":"172.21.15.33:6805/3862661148","state":["exists","up"]},{"osd":2,"uuid":"22e40cac-3bc0-4cce-b47b-77499b767f7e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":3958753963},{"type":"v1","addr":"172.21.15.184:6801","nonce":3958753963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":3958753963},{"type":"v1","addr":"172.21.15.184:6803","nonce":3958753963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":3958753963},{"type":"v1","addr":"172.21.15.184:6807","nonce":3958753963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":3958753963},{"type":"v1","addr":"172.21.15.184:6805","nonce":3958753963}]},"public_addr":"172.21.15.184:6801/3958753963","cluster_addr":"172.21.15.184:6803/3958753963","heartbeat_back_addr":"172.21.15.184:6807/3958753963","heartbeat_front_addr":"172.21.15.184:6805/3958753963","state":["exists","up"]},{"osd":3,"uuid":"834a07f5-2c24-49e1-8e4a-12c96cfbdc65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3395182524},{"type":"v1","addr":"172.21.15.16:6811","nonce":3395182524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3395182524},{"type":"v1","addr":"172.21.15.16:6813","nonce":3395182524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3395182524},{"type":"v1","addr":"172.21.15.16:6817","nonce":3395182524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3395182524},{"type":"v1","addr":"172.21.15.16:6815","nonce":3395182524}]},"public_addr":"172.21.15.16:6811/3395182524","cluster_addr":"172.21.15.16:6813/3395182524","heartbeat_back_addr":"172.21.15.16:6817/3395182524","heartbeat_front_addr":"172.21.15.16:6815/3395182524","state":["exists","up"]},{"osd":4,"uuid":"3ca56e09-c194-4404-b9bd-2524d38367eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":1355184322},{"type":"v1","addr":"172.21.15.33:6809","nonce":1355184322}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1355184322},{"type":"v1","addr":"172.21.15.33:6811","nonce":1355184322}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1355184322},{"type":"v1","addr":"172.21.15.33:6815","nonce":1355184322}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1355184322},{"type":"v1","addr":"172.21.15.33:6813","nonce":1355184322}]},"public_addr":"172.21.15.33:6809/1355184322","cluster_addr":"172.21.15.33:6811/1355184322","heartbeat_back_addr":"172.21.15.33:6815/1355184322","heartbeat_front_addr":"172.21.15.33:6813/1355184322","state":["exists","up"]},{"osd":5,"uuid":"0b05ecdb-b987-461f-951c-6d76739bfb4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":1533397276},{"type":"v1","addr":"172.21.15.184:6809","nonce":1533397276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":1533397276},{"type":"v1","addr":"172.21.15.184:6811","nonce":1533397276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":1533397276},{"type":"v1","addr":"172.21.15.184:6815","nonce":1533397276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":1533397276},{"type":"v1","addr":"172.21.15.184:6813","nonce":1533397276}]},"public_addr":"172.21.15.184:6809/1533397276","cluster_addr":"172.21.15.184:6811/1533397276","heartbeat_back_addr":"172.21.15.184:6815/1533397276","heartbeat_front_addr":"172.21.15.184:6813/1533397276","state":["exists","up"]},{"osd":6,"uuid":"b3936e21-e197-49fa-8441-d909b258b612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":902466142},{"type":"v1","addr":"172.21.15.16:6819","nonce":902466142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":902466142},{"type":"v1","addr":"172.21.15.16:6821","nonce":902466142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":902466142},{"type":"v1","addr":"172.21.15.16:6825","nonce":902466142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":902466142},{"type":"v1","addr":"172.21.15.16:6823","nonce":902466142}]},"public_addr":"172.21.15.16:6819/902466142","cluster_addr":"172.21.15.16:6821/902466142","heartbeat_back_addr":"172.21.15.16:6825/902466142","heartbeat_front_addr":"172.21.15.16:6823/902466142","state":["exists","up"]},{"osd":7,"uuid":"4ddbe011-34c1-4b3f-a8eb-577422fb5426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":4186032630},{"type":"v1","addr":"172.21.15.33:6817","nonce":4186032630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":4186032630},{"type":"v1","addr":"172.21.15.33:6819","nonce":4186032630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":4186032630},{"type":"v1","addr":"172.21.15.33:6823","nonce":4186032630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":4186032630},{"type":"v1","addr":"172.21.15.33:6821","nonce":4186032630}]},"public_addr":"172.21.15.33:6817/4186032630","cluster_addr":"172.21.15.33:6819/4186032630","heartbeat_back_addr":"172.21.15.33:6823/4186032630","heartbeat_front_addr":"172.21.15.33:6821/4186032630","state":["exists","up"]},{"osd":8,"uuid":"dca5bfb8-ac6a-47ce-905c-be7fcaf0b5ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":781851781},{"type":"v1","addr":"172.21.15.184:6817","nonce":781851781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":781851781},{"type":"v1","addr":"172.21.15.184:6819","nonce":781851781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":781851781},{"type":"v1","addr":"172.21.15.184:6823","nonce":781851781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":781851781},{"type":"v1","addr":"172.21.15.184:6821","nonce":781851781}]},"public_addr":"172.21.15.184:6817/781851781","cluster_addr":"172.21.15.184:6819/781851781","heartbeat_back_addr":"172.21.15.184:6823/781851781","heartbeat_front_addr":"172.21.15.184:6821/781851781","state":["exists","up"]},{"osd":9,"uuid":"23a05cb5-01d5-40f3-b66a-15fcccedbd6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":1569578746},{"type":"v1","addr":"172.21.15.16:6827","nonce":1569578746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":1569578746},{"type":"v1","addr":"172.21.15.16:6829","nonce":1569578746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1569578746},{"type":"v1","addr":"172.21.15.16:6833","nonce":1569578746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":1569578746},{"type":"v1","addr":"172.21.15.16:6831","nonce":1569578746}]},"public_addr":"172.21.15.16:6827/1569578746","cluster_addr":"172.21.15.16:6829/1569578746","heartbeat_back_addr":"172.21.15.16:6833/1569578746","heartbeat_front_addr":"172.21.15.16:6831/1569578746","state":["exists","up"]},{"osd":10,"uuid":"e4bd6bce-3c69-44db-bc09-d264a0ad094a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":469977847},{"type":"v1","addr":"172.21.15.33:6825","nonce":469977847}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":469977847},{"type":"v1","addr":"172.21.15.33:6827","nonce":469977847}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":469977847},{"type":"v1","addr":"172.21.15.33:6831","nonce":469977847}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":469977847},{"type":"v1","addr":"172.21.15.33:6829","nonce":469977847}]},"public_addr":"172.21.15.33:6825/469977847","cluster_addr":"172.21.15.33:6827/469977847","heartbeat_back_addr":"172.21.15.33:6831/469977847","heartbeat_front_addr":"172.21.15.33:6829/469977847","state":["exists","up"]},{"osd":11,"uuid":"d60212f8-5a1d-45f7-9a1b-12294bb4c80d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":1619871346},{"type":"v1","addr":"172.21.15.184:6825","nonce":1619871346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":1619871346},{"type":"v1","addr":"172.21.15.184:6827","nonce":1619871346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":1619871346},{"type":"v1","addr":"172.21.15.184:6831","nonce":1619871346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":1619871346},{"type":"v1","addr":"172.21.15.184:6829","nonce":1619871346}]},"public_addr":"172.21.15.184:6825/1619871346","cluster_addr":"172.21.15.184:6827/1619871346","heartbeat_back_addr":"172.21.15.184:6831/1619871346","heartbeat_front_addr":"172.21.15.184:6829/1619871346","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:54:32.057979+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:54:57.501562+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:55:23.137225+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:55:48.646155+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:56:13.311907+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:56:38.214188+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:57:02.570921+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:57:27.506097+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:57:52.038529+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:58:17.423300+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:58:41.579992+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:59:06.245817+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:0/3532565698":"2024-09-13T01:53:08.218475+0000","172.21.15.16:0/1822157179":"2024-09-13T01:52:44.194152+0000","172.21.15.16:0/3919664061":"2024-09-13T01:52:44.194152+0000","172.21.15.16:0/2269394774":"2024-09-13T01:52:44.194152+0000","172.21.15.16:0/2595308634":"2024-09-13T01:53:08.218475+0000","172.21.15.16:6801/2754331153":"2024-09-13T01:52:44.194152+0000","172.21.15.16:6801/2615861334":"2024-09-13T01:53:08.218475+0000","172.21.15.16:6800/2615861334":"2024-09-13T01:53:08.218475+0000","172.21.15.16:0/4199114771":"2024-09-13T01:53:08.218475+0000","172.21.15.16:6800/2754331153":"2024-09-13T01:52:44.194152+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T01:59:11.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:11.027+0000 7f26eb2fd640 1 -- 172.21.15.16:0/3131872929 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f26d00a2b80 msgr2=0x7f26d00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:11.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:11.027+0000 7f26eb2fd640 1 --2- 172.21.15.16:0/3131872929 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f26d00a2b80 0x7f26d00a5040 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f26d4006fd0 tx=0x7f26d4008040 comp rx=0 tx=0).stop 2024-09-12T01:59:11.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:11.027+0000 7f26eb2fd640 1 -- 172.21.15.16:0/3131872929 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26e4102d40 msgr2=0x7f26e41984a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:11.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:11.027+0000 7f26eb2fd640 1 --2- 172.21.15.16:0/3131872929 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26e4102d40 0x7f26e41984a0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f26e0005ab0 tx=0x7f26e00042f0 comp rx=0 tx=0).stop 2024-09-12T01:59:11.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:11.027+0000 7f26eb2fd640 1 -- 172.21.15.16:0/3131872929 shutdown_connections 2024-09-12T01:59:11.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:11.027+0000 7f26eb2fd640 1 --2- 172.21.15.16:0/3131872929 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f26d00a2b80 0x7f26d00a5040 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:11.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:11.027+0000 7f26eb2fd640 1 --2- 172.21.15.16:0/3131872929 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f26e410c170 0x7f26e419cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:11.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:11.027+0000 7f26eb2fd640 1 --2- 172.21.15.16:0/3131872929 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26e4103660 0x7f26e41989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:11.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:11.027+0000 7f26eb2fd640 1 --2- 172.21.15.16:0/3131872929 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26e4102d40 0x7f26e41984a0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:11.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:11.027+0000 7f26eb2fd640 1 -- 172.21.15.16:0/3131872929 >> 172.21.15.16:0/3131872929 conn(0x7f26e40fc7a0 msgr2=0x7f26e40fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:59:11.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:11.027+0000 7f26eb2fd640 1 -- 172.21.15.16:0/3131872929 shutdown_connections 2024-09-12T01:59:11.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:11.027+0000 7f26eb2fd640 1 -- 172.21.15.16:0/3131872929 wait complete. 2024-09-12T01:59:11.334 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:11 smithi016 ceph-mon[30947]: osdmap e75: 12 total, 12 up, 12 in 2024-09-12T01:59:11.334 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:11 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/3131872929' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T01:59:11.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:11 smithi184 ceph-mon[35593]: osdmap e75: 12 total, 12 up, 12 in 2024-09-12T01:59:11.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:11 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/3131872929' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T01:59:11.631 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-12T01:55:26.727564+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-12T01:59:11.632 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-09-12T01:59:11.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:11 smithi033 ceph-mon[35332]: osdmap e75: 12 total, 12 up, 12 in 2024-09-12T01:59:11.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:11 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/3131872929' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T01:59:11.884 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:59:12.304 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:12 smithi016 ceph-mon[30947]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:59:12.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:12.322+0000 7f27df439640 1 -- 172.21.15.16:0/2772185005 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27d810c2f0 msgr2=0x7f27d810e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:12.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:12.322+0000 7f27df439640 1 --2- 172.21.15.16:0/2772185005 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27d810c2f0 0x7f27d810e790 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f27d0009ca0 tx=0x7f27d002a8c0 comp rx=0 tx=0).stop 2024-09-12T01:59:12.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:12.322+0000 7f27df439640 1 -- 172.21.15.16:0/2772185005 shutdown_connections 2024-09-12T01:59:12.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:12.322+0000 7f27df439640 1 --2- 172.21.15.16:0/2772185005 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27d810c2f0 0x7f27d810e790 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:12.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:12.322+0000 7f27df439640 1 --2- 172.21.15.16:0/2772185005 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f27d81037a0 0x7f27d810bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:12.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:12.322+0000 7f27df439640 1 --2- 172.21.15.16:0/2772185005 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27d8102df0 0x7f27d81031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:12.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:12.322+0000 7f27df439640 1 -- 172.21.15.16:0/2772185005 >> 172.21.15.16:0/2772185005 conn(0x7f27d80fc7d0 msgr2=0x7f27d80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:59:12.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:12.322+0000 7f27df439640 1 -- 172.21.15.16:0/2772185005 shutdown_connections 2024-09-12T01:59:12.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:12.322+0000 7f27df439640 1 -- 172.21.15.16:0/2772185005 wait complete. 2024-09-12T01:59:12.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:12.322+0000 7f27df439640 1 Processor -- start 2024-09-12T01:59:12.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:12.323+0000 7f27df439640 1 -- start start 2024-09-12T01:59:12.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:12.323+0000 7f27df439640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27d8102df0 0x7f27d81985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:12.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:12.323+0000 7f27df439640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f27d81037a0 0x7f27d8198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:12.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:12.323+0000 7f27df439640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27d810c2f0 0x7f27d819ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:12.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:12.323+0000 7f27df439640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f27d8112110 con 0x7f27d810c2f0 2024-09-12T01:59:12.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:12.323+0000 7f27df439640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f27d8111f90 con 0x7f27d8102df0 2024-09-12T01:59:12.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:12.323+0000 7f27df439640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f27d8112290 con 0x7f27d81037a0 2024-09-12T01:59:12.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:12.323+0000 7f27dd9d6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27d810c2f0 0x7f27d819ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:12.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:12.323+0000 7f27dd9d6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27d810c2f0 0x7f27d819ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44004/0 (socket says 172.21.15.16:44004) 2024-09-12T01:59:12.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:12.323+0000 7f27dd9d6640 1 -- 172.21.15.16:0/2858755106 learned_addr learned my addr 172.21.15.16:0/2858755106 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:59:12.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:12.323+0000 7f27dc9d4640 1 --2- 172.21.15.16:0/2858755106 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f27d81037a0 0x7f27d8198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:12.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:12.323+0000 7f27dd1d5640 1 --2- 172.21.15.16:0/2858755106 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27d8102df0 0x7f27d81985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:12.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:12.323+0000 7f27dd9d6640 1 -- 172.21.15.16:0/2858755106 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f27d81037a0 msgr2=0x7f27d8198b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:12.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:12.323+0000 7f27dd9d6640 1 --2- 172.21.15.16:0/2858755106 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f27d81037a0 0x7f27d8198b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:12.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:12.323+0000 7f27dd9d6640 1 -- 172.21.15.16:0/2858755106 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27d8102df0 msgr2=0x7f27d81985c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:12.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:12.323+0000 7f27dd9d6640 1 --2- 172.21.15.16:0/2858755106 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27d8102df0 0x7f27d81985c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:12.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:12.323+0000 7f27dd9d6640 1 -- 172.21.15.16:0/2858755106 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27d819d570 con 0x7f27d810c2f0 2024-09-12T01:59:12.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:12.324+0000 7f27dd9d6640 1 --2- 172.21.15.16:0/2858755106 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27d810c2f0 0x7f27d819ce90 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f27d0031a20 tx=0x7f27d0031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:59:12.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:12.324+0000 7f27ce7fc640 1 -- 172.21.15.16:0/2858755106 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27d0003c30 con 0x7f27d810c2f0 2024-09-12T01:59:12.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:12.324+0000 7f27ce7fc640 1 -- 172.21.15.16:0/2858755106 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27d0003dd0 con 0x7f27d810c2f0 2024-09-12T01:59:12.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:12.324+0000 7f27df439640 1 -- 172.21.15.16:0/2858755106 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27d819d800 con 0x7f27d810c2f0 2024-09-12T01:59:12.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:12.324+0000 7f27ce7fc640 1 -- 172.21.15.16:0/2858755106 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27d002fcf0 con 0x7f27d810c2f0 2024-09-12T01:59:12.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:12.324+0000 7f27df439640 1 -- 172.21.15.16:0/2858755106 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27d81a5140 con 0x7f27d810c2f0 2024-09-12T01:59:12.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:12.325+0000 7f27df439640 1 -- 172.21.15.16:0/2858755106 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27d81031d0 con 0x7f27d810c2f0 2024-09-12T01:59:12.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:12.327+0000 7f27ce7fc640 1 -- 172.21.15.16:0/2858755106 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f27d0004050 con 0x7f27d810c2f0 2024-09-12T01:59:12.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:12.327+0000 7f27ce7fc640 1 --2- 172.21.15.16:0/2858755106 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f27c40a2c00 0x7f27c40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:12.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:12.327+0000 7f27ce7fc640 1 -- 172.21.15.16:0/2858755106 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7f27d00eff80 con 0x7f27d810c2f0 2024-09-12T01:59:12.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:12.327+0000 7f27dd1d5640 1 --2- 172.21.15.16:0/2858755106 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f27c40a2c00 0x7f27c40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:12.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:12.328+0000 7f27dd1d5640 1 --2- 172.21.15.16:0/2858755106 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f27c40a2c00 0x7f27c40a50c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f27d40058c0 tx=0x7f27d400a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:59:12.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:12.330+0000 7f27ce7fc640 1 -- 172.21.15.16:0/2858755106 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f27d00bd410 con 0x7f27d810c2f0 2024-09-12T01:59:12.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:12.506+0000 7f27df439640 1 -- 172.21.15.16:0/2858755106 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f27d81072a0 con 0x7f27d810c2f0 2024-09-12T01:59:12.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:12.506+0000 7f27ce7fc640 1 -- 172.21.15.16:0/2858755106 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v75) ==== 93+0+10 (secure 0 0 0) 0x7f27d002fe90 con 0x7f27d810c2f0 2024-09-12T01:59:12.506 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-09-12T01:59:12.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:12.508+0000 7f27df439640 1 -- 172.21.15.16:0/2858755106 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f27c40a2c00 msgr2=0x7f27c40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:12.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:12.508+0000 7f27df439640 1 --2- 172.21.15.16:0/2858755106 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f27c40a2c00 0x7f27c40a50c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f27d40058c0 tx=0x7f27d400a2d0 comp rx=0 tx=0).stop 2024-09-12T01:59:12.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:12.508+0000 7f27df439640 1 -- 172.21.15.16:0/2858755106 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27d810c2f0 msgr2=0x7f27d819ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:12.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:12.508+0000 7f27df439640 1 --2- 172.21.15.16:0/2858755106 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27d810c2f0 0x7f27d819ce90 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f27d0031a20 tx=0x7f27d0031a50 comp rx=0 tx=0).stop 2024-09-12T01:59:12.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:12.508+0000 7f27df439640 1 -- 172.21.15.16:0/2858755106 shutdown_connections 2024-09-12T01:59:12.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:12.508+0000 7f27df439640 1 --2- 172.21.15.16:0/2858755106 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f27c40a2c00 0x7f27c40a50c0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:12.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:12.508+0000 7f27df439640 1 --2- 172.21.15.16:0/2858755106 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27d810c2f0 0x7f27d819ce90 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:12.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:12.508+0000 7f27df439640 1 --2- 172.21.15.16:0/2858755106 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f27d81037a0 0x7f27d8198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:12.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:12.508+0000 7f27df439640 1 --2- 172.21.15.16:0/2858755106 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27d8102df0 0x7f27d81985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:12.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:12.508+0000 7f27df439640 1 -- 172.21.15.16:0/2858755106 >> 172.21.15.16:0/2858755106 conn(0x7f27d80fc7d0 msgr2=0x7f27d80fd470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:59:12.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:12.508+0000 7f27df439640 1 -- 172.21.15.16:0/2858755106 shutdown_connections 2024-09-12T01:59:12.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:12.508+0000 7f27df439640 1 -- 172.21.15.16:0/2858755106 wait complete. 2024-09-12T01:59:12.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:12 smithi184 ceph-mon[35593]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:59:12.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:12 smithi033 ceph-mon[35332]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:59:13.121 INFO:tasks.cephadm:Adding mds.a on smithi016 2024-09-12T01:59:13.121 INFO:tasks.cephadm:Adding mds.d on smithi016 2024-09-12T01:59:13.121 INFO:tasks.cephadm:Adding mds.g on smithi016 2024-09-12T01:59:13.121 INFO:tasks.cephadm:Adding mds.j on smithi016 2024-09-12T01:59:13.121 INFO:tasks.cephadm:Adding mds.b on smithi033 2024-09-12T01:59:13.121 INFO:tasks.cephadm:Adding mds.e on smithi033 2024-09-12T01:59:13.121 INFO:tasks.cephadm:Adding mds.h on smithi033 2024-09-12T01:59:13.121 INFO:tasks.cephadm:Adding mds.k on smithi033 2024-09-12T01:59:13.121 INFO:tasks.cephadm:Adding mds.c on smithi184 2024-09-12T01:59:13.121 INFO:tasks.cephadm:Adding mds.f on smithi184 2024-09-12T01:59:13.121 INFO:tasks.cephadm:Adding mds.i on smithi184 2024-09-12T01:59:13.122 INFO:tasks.cephadm:Adding mds.l on smithi184 2024-09-12T01:59:13.122 DEBUG:teuthology.orchestra.run.smithi184:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph orch apply mds all '12;smithi016=a;smithi016=d;smithi016=g;smithi016=j;smithi033=b;smithi033=e;smithi033=h;smithi033=k;smithi184=c;smithi184=f;smithi184=i;smithi184=l' 2024-09-12T01:59:13.385 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T01:59:13.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:13 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/2858755106' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-12T01:59:13.506 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:13 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/2858755106' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-12T01:59:13.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:13 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/2858755106' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-12T01:59:13.806 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:59:13.804+0000 7f7f66dcb640 1 -- 172.21.15.184:0/2485516126 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7f60101cd0 msgr2=0x7f7f60102150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:13.806 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:59:13.804+0000 7f7f66dcb640 1 --2- 172.21.15.184:0/2485516126 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7f60101cd0 0x7f7f60102150 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f7f48009900 tx=0x7f7f4802a410 comp rx=0 tx=0).stop 2024-09-12T01:59:13.806 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:59:13.804+0000 7f7f66dcb640 1 -- 172.21.15.184:0/2485516126 shutdown_connections 2024-09-12T01:59:13.807 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:59:13.804+0000 7f7f66dcb640 1 --2- 172.21.15.184:0/2485516126 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f60102690 0x7f7f60108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:13.807 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:59:13.804+0000 7f7f66dcb640 1 --2- 172.21.15.184:0/2485516126 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7f60101cd0 0x7f7f60102150 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:13.807 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:59:13.804+0000 7f7f66dcb640 1 --2- 172.21.15.184:0/2485516126 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7f60100ad0 0x7f7f60100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:13.807 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:59:13.804+0000 7f7f66dcb640 1 -- 172.21.15.184:0/2485516126 >> 172.21.15.184:0/2485516126 conn(0x7f7f600fc280 msgr2=0x7f7f600fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:59:13.807 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:59:13.804+0000 7f7f66dcb640 1 -- 172.21.15.184:0/2485516126 shutdown_connections 2024-09-12T01:59:13.807 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:59:13.804+0000 7f7f66dcb640 1 -- 172.21.15.184:0/2485516126 wait complete. 2024-09-12T01:59:13.807 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:59:13.804+0000 7f7f66dcb640 1 Processor -- start 2024-09-12T01:59:13.807 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:59:13.805+0000 7f7f66dcb640 1 -- start start 2024-09-12T01:59:13.807 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:59:13.805+0000 7f7f66dcb640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7f60100ad0 0x7f7f60198060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:13.807 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:59:13.805+0000 7f7f66dcb640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7f60101cd0 0x7f7f601985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:13.807 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:59:13.805+0000 7f7f66dcb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f60102690 0x7f7f6019f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:13.807 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:59:13.805+0000 7f7f66dcb640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7f6010c850 con 0x7f7f60102690 2024-09-12T01:59:13.807 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:59:13.805+0000 7f7f66dcb640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7f6010c6d0 con 0x7f7f60101cd0 2024-09-12T01:59:13.808 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:59:13.805+0000 7f7f66dcb640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7f6010c9d0 con 0x7f7f60100ad0 2024-09-12T01:59:13.808 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:59:13.805+0000 7f7f64b67640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7f60100ad0 0x7f7f60198060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:13.808 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:59:13.805+0000 7f7f64b67640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7f60100ad0 0x7f7f60198060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.184:33614/0 (socket says 172.21.15.184:33614) 2024-09-12T01:59:13.808 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:59:13.805+0000 7f7f64b67640 1 -- 172.21.15.184:0/2019418124 learned_addr learned my addr 172.21.15.184:0/2019418124 (peer_addr_for_me v2:172.21.15.184:0/0) 2024-09-12T01:59:13.808 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:59:13.805+0000 7f7f64b67640 1 -- 172.21.15.184:0/2019418124 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7f60101cd0 msgr2=0x7f7f601985a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:13.808 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:59:13.805+0000 7f7f65368640 1 --2- 172.21.15.184:0/2019418124 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f60102690 0x7f7f6019f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:13.808 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:59:13.805+0000 7f7f5ffff640 1 --2- 172.21.15.184:0/2019418124 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7f60101cd0 0x7f7f601985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:13.808 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:59:13.805+0000 7f7f64b67640 1 --2- 172.21.15.184:0/2019418124 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7f60101cd0 0x7f7f601985a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:13.808 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:59:13.805+0000 7f7f64b67640 1 -- 172.21.15.184:0/2019418124 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f60102690 msgr2=0x7f7f6019f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:13.808 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:59:13.805+0000 7f7f64b67640 1 --2- 172.21.15.184:0/2019418124 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f60102690 0x7f7f6019f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:13.808 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:59:13.805+0000 7f7f64b67640 1 -- 172.21.15.184:0/2019418124 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f6019fc90 con 0x7f7f60100ad0 2024-09-12T01:59:13.808 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:59:13.805+0000 7f7f65368640 1 --2- 172.21.15.184:0/2019418124 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f60102690 0x7f7f6019f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:59:13.808 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:59:13.805+0000 7f7f64b67640 1 --2- 172.21.15.184:0/2019418124 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7f60100ad0 0x7f7f60198060 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f7f5000b9c0 tx=0x7f7f5000be80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:59:13.809 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:59:13.805+0000 7f7f5dffb640 1 -- 172.21.15.184:0/2019418124 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f50019070 con 0x7f7f60100ad0 2024-09-12T01:59:13.809 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:59:13.805+0000 7f7f5dffb640 1 -- 172.21.15.184:0/2019418124 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f5000cba0 con 0x7f7f60100ad0 2024-09-12T01:59:13.809 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:59:13.805+0000 7f7f66dcb640 1 -- 172.21.15.184:0/2019418124 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f6019ff80 con 0x7f7f60100ad0 2024-09-12T01:59:13.809 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:59:13.805+0000 7f7f5dffb640 1 -- 172.21.15.184:0/2019418124 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f50012640 con 0x7f7f60100ad0 2024-09-12T01:59:13.809 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:59:13.806+0000 7f7f66dcb640 1 -- 172.21.15.184:0/2019418124 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f601a0400 con 0x7f7f60100ad0 2024-09-12T01:59:13.810 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:59:13.807+0000 7f7f66dcb640 1 -- 172.21.15.184:0/2019418124 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f601a06c0 con 0x7f7f60100ad0 2024-09-12T01:59:13.810 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:59:13.808+0000 7f7f5dffb640 1 -- 172.21.15.184:0/2019418124 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f7f5001c070 con 0x7f7f60100ad0 2024-09-12T01:59:13.810 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:59:13.808+0000 7f7f5dffb640 1 --2- 172.21.15.184:0/2019418124 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f7f4c0a2bb0 0x7f7f4c0a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:13.810 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:59:13.808+0000 7f7f5dffb640 1 -- 172.21.15.184:0/2019418124 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7f7f500ce1d0 con 0x7f7f60100ad0 2024-09-12T01:59:13.810 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:59:13.809+0000 7f7f5ffff640 1 --2- 172.21.15.184:0/2019418124 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f7f4c0a2bb0 0x7f7f4c0a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:13.810 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:59:13.809+0000 7f7f5ffff640 1 --2- 172.21.15.184:0/2019418124 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f7f4c0a2bb0 0x7f7f4c0a5070 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f7f48031610 tx=0x7f7f48000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:59:13.813 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:59:13.813+0000 7f7f5dffb640 1 -- 172.21.15.184:0/2019418124 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7f5009b5d0 con 0x7f7f60100ad0 2024-09-12T01:59:13.995 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:59:13.995+0000 7f7f66dcb640 1 -- 172.21.15.184:0/2019418124 --> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi016=a;smithi016=d;smithi016=g;smithi016=j;smithi033=b;smithi033=e;smithi033=h;smithi033=k;smithi184=c;smithi184=f;smithi184=i;smithi184=l", "target": ["mon-mgr", ""]}) -- 0x7f7f6006a6b0 con 0x7f7f4c0a2bb0 2024-09-12T01:59:14.003 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:59:14.003+0000 7f7f5dffb640 1 -- 172.21.15.184:0/2019418124 <== mgr.14150 v2:172.21.15.16:6800/3887444820 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f7f6006a6b0 con 0x7f7f4c0a2bb0 2024-09-12T01:59:14.003 INFO:teuthology.orchestra.run.smithi184.stdout:Scheduled mds.all update... 2024-09-12T01:59:14.006 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:59:14.006+0000 7f7f66dcb640 1 -- 172.21.15.184:0/2019418124 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f7f4c0a2bb0 msgr2=0x7f7f4c0a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:14.006 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:59:14.006+0000 7f7f66dcb640 1 --2- 172.21.15.184:0/2019418124 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f7f4c0a2bb0 0x7f7f4c0a5070 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f7f48031610 tx=0x7f7f48000bc0 comp rx=0 tx=0).stop 2024-09-12T01:59:14.006 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:59:14.006+0000 7f7f66dcb640 1 -- 172.21.15.184:0/2019418124 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7f60100ad0 msgr2=0x7f7f60198060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:14.006 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:59:14.006+0000 7f7f66dcb640 1 --2- 172.21.15.184:0/2019418124 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7f60100ad0 0x7f7f60198060 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f7f5000b9c0 tx=0x7f7f5000be80 comp rx=0 tx=0).stop 2024-09-12T01:59:14.006 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:59:14.006+0000 7f7f66dcb640 1 -- 172.21.15.184:0/2019418124 shutdown_connections 2024-09-12T01:59:14.007 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:59:14.006+0000 7f7f66dcb640 1 --2- 172.21.15.184:0/2019418124 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f7f4c0a2bb0 0x7f7f4c0a5070 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:14.007 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:59:14.006+0000 7f7f66dcb640 1 --2- 172.21.15.184:0/2019418124 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f60102690 0x7f7f6019f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:14.007 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:59:14.006+0000 7f7f66dcb640 1 --2- 172.21.15.184:0/2019418124 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7f60101cd0 0x7f7f601985a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:14.007 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:59:14.006+0000 7f7f66dcb640 1 --2- 172.21.15.184:0/2019418124 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7f60100ad0 0x7f7f60198060 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:14.007 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:59:14.006+0000 7f7f66dcb640 1 -- 172.21.15.184:0/2019418124 >> 172.21.15.184:0/2019418124 conn(0x7f7f600fc280 msgr2=0x7f7f60069fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:59:14.007 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:59:14.006+0000 7f7f66dcb640 1 -- 172.21.15.184:0/2019418124 shutdown_connections 2024-09-12T01:59:14.007 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-12T01:59:14.006+0000 7f7f66dcb640 1 -- 172.21.15.184:0/2019418124 wait complete. 2024-09-12T01:59:14.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:14 smithi184 ceph-mon[35593]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:59:14.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:14 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:14.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:14 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:59:14.624 DEBUG:teuthology.orchestra.run.smithi016:mds.a> sudo journalctl -f -n 0 -u ceph-879f3b56-70a9-11ef-bcea-c7b262605968@mds.a.service 2024-09-12T01:59:14.627 DEBUG:teuthology.orchestra.run.smithi016:mds.d> sudo journalctl -f -n 0 -u ceph-879f3b56-70a9-11ef-bcea-c7b262605968@mds.d.service 2024-09-12T01:59:14.653 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:14 smithi016 ceph-mon[30947]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:59:14.653 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:14 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:14.653 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:14 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:59:14.654 DEBUG:teuthology.orchestra.run.smithi016:mds.g> sudo journalctl -f -n 0 -u ceph-879f3b56-70a9-11ef-bcea-c7b262605968@mds.g.service 2024-09-12T01:59:14.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:14 smithi033 ceph-mon[35332]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:59:14.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:14 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:14.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:14 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:59:14.697 DEBUG:teuthology.orchestra.run.smithi016:mds.j> sudo journalctl -f -n 0 -u ceph-879f3b56-70a9-11ef-bcea-c7b262605968@mds.j.service 2024-09-12T01:59:14.740 DEBUG:teuthology.orchestra.run.smithi033:mds.b> sudo journalctl -f -n 0 -u ceph-879f3b56-70a9-11ef-bcea-c7b262605968@mds.b.service 2024-09-12T01:59:14.745 DEBUG:teuthology.orchestra.run.smithi033:mds.e> sudo journalctl -f -n 0 -u ceph-879f3b56-70a9-11ef-bcea-c7b262605968@mds.e.service 2024-09-12T01:59:14.790 DEBUG:teuthology.orchestra.run.smithi033:mds.h> sudo journalctl -f -n 0 -u ceph-879f3b56-70a9-11ef-bcea-c7b262605968@mds.h.service 2024-09-12T01:59:14.833 DEBUG:teuthology.orchestra.run.smithi033:mds.k> sudo journalctl -f -n 0 -u ceph-879f3b56-70a9-11ef-bcea-c7b262605968@mds.k.service 2024-09-12T01:59:14.876 DEBUG:teuthology.orchestra.run.smithi184:mds.c> sudo journalctl -f -n 0 -u ceph-879f3b56-70a9-11ef-bcea-c7b262605968@mds.c.service 2024-09-12T01:59:14.878 DEBUG:teuthology.orchestra.run.smithi184:mds.f> sudo journalctl -f -n 0 -u ceph-879f3b56-70a9-11ef-bcea-c7b262605968@mds.f.service 2024-09-12T01:59:14.921 DEBUG:teuthology.orchestra.run.smithi184:mds.i> sudo journalctl -f -n 0 -u ceph-879f3b56-70a9-11ef-bcea-c7b262605968@mds.i.service 2024-09-12T01:59:14.964 DEBUG:teuthology.orchestra.run.smithi184:mds.l> sudo journalctl -f -n 0 -u ceph-879f3b56-70a9-11ef-bcea-c7b262605968@mds.l.service 2024-09-12T01:59:15.007 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-09-12T01:59:15.007 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T01:59:15.269 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:59:15.376 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:15 smithi016 ceph-mon[30947]: from='client.24430 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi016=a;smithi016=d;smithi016=g;smithi016=j;smithi033=b;smithi033=e;smithi033=h;smithi033=k;smithi184=c;smithi184=f;smithi184=i;smithi184=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:59:15.376 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:15 smithi016 ceph-mon[30947]: Saving service mds.all spec with placement smithi016=a;smithi016=d;smithi016=g;smithi016=j;smithi033=b;smithi033=e;smithi033=h;smithi033=k;smithi184=c;smithi184=f;smithi184=i;smithi184=l;count:12 2024-09-12T01:59:15.376 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:15 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:59:15.376 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:15 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:59:15.376 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:15 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:15.376 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:15 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T01:59:15.377 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:15 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T01:59:15.377 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:15 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:59:15.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:15 smithi184 ceph-mon[35593]: from='client.24430 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi016=a;smithi016=d;smithi016=g;smithi016=j;smithi033=b;smithi033=e;smithi033=h;smithi033=k;smithi184=c;smithi184=f;smithi184=i;smithi184=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:59:15.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:15 smithi184 ceph-mon[35593]: Saving service mds.all spec with placement smithi016=a;smithi016=d;smithi016=g;smithi016=j;smithi033=b;smithi033=e;smithi033=h;smithi033=k;smithi184=c;smithi184=f;smithi184=i;smithi184=l;count:12 2024-09-12T01:59:15.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:15 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:59:15.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:15 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:59:15.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:15 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:15.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:15 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T01:59:15.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:15 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T01:59:15.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:15 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:59:15.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:15.669+0000 7f8eef506640 1 -- 172.21.15.16:0/97290252 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ee8106f10 msgr2=0x7f8ee810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:15.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:15.669+0000 7f8eef506640 1 --2- 172.21.15.16:0/97290252 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ee8106f10 0x7f8ee810ea90 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8ee0009900 tx=0x7f8ee002a410 comp rx=0 tx=0).stop 2024-09-12T01:59:15.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:15.669+0000 7f8eef506640 1 -- 172.21.15.16:0/97290252 shutdown_connections 2024-09-12T01:59:15.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:15.669+0000 7f8eef506640 1 --2- 172.21.15.16:0/97290252 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ee8106f10 0x7f8ee810ea90 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:15.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:15.669+0000 7f8eef506640 1 --2- 172.21.15.16:0/97290252 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8ee8102790 0x7f8ee81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:15.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:15.669+0000 7f8eef506640 1 --2- 172.21.15.16:0/97290252 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8ee8101de0 0x7f8ee81021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:15.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:15.669+0000 7f8eef506640 1 -- 172.21.15.16:0/97290252 >> 172.21.15.16:0/97290252 conn(0x7f8ee80fd8c0 msgr2=0x7f8ee80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:59:15.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:15.669+0000 7f8eef506640 1 -- 172.21.15.16:0/97290252 shutdown_connections 2024-09-12T01:59:15.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:15.669+0000 7f8eef506640 1 -- 172.21.15.16:0/97290252 wait complete. 2024-09-12T01:59:15.671 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:15 smithi033 ceph-mon[35332]: from='client.24430 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi016=a;smithi016=d;smithi016=g;smithi016=j;smithi033=b;smithi033=e;smithi033=h;smithi033=k;smithi184=c;smithi184=f;smithi184=i;smithi184=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:59:15.671 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:15 smithi033 ceph-mon[35332]: Saving service mds.all spec with placement smithi016=a;smithi016=d;smithi016=g;smithi016=j;smithi033=b;smithi033=e;smithi033=h;smithi033=k;smithi184=c;smithi184=f;smithi184=i;smithi184=l;count:12 2024-09-12T01:59:15.671 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:15 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:59:15.671 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:15 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:59:15.671 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:15 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:15.671 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:15 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T01:59:15.671 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:15 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T01:59:15.671 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:15 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:59:15.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:15.670+0000 7f8eef506640 1 Processor -- start 2024-09-12T01:59:15.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:15.670+0000 7f8eef506640 1 -- start start 2024-09-12T01:59:15.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:15.670+0000 7f8eef506640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8ee8101de0 0x7f8ee81983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:15.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:15.670+0000 7f8eef506640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8ee8102790 0x7f8ee8198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:15.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:15.670+0000 7f8eef506640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ee8106f10 0x7f8ee819cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:15.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:15.670+0000 7f8eef506640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8ee8111f70 con 0x7f8ee8106f10 2024-09-12T01:59:15.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:15.670+0000 7f8eef506640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8ee8111df0 con 0x7f8ee8102790 2024-09-12T01:59:15.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:15.670+0000 7f8eef506640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8ee81120f0 con 0x7f8ee8101de0 2024-09-12T01:59:15.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:15.670+0000 7f8eedaa3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ee8106f10 0x7f8ee819cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:15.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:15.670+0000 7f8eedaa3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ee8106f10 0x7f8ee819cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44012/0 (socket says 172.21.15.16:44012) 2024-09-12T01:59:15.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:15.670+0000 7f8eedaa3640 1 -- 172.21.15.16:0/3501303441 learned_addr learned my addr 172.21.15.16:0/3501303441 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:59:15.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:15.670+0000 7f8eed2a2640 1 --2- 172.21.15.16:0/3501303441 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8ee8101de0 0x7f8ee81983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:15.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:15.670+0000 7f8eecaa1640 1 --2- 172.21.15.16:0/3501303441 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8ee8102790 0x7f8ee8198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:15.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:15.671+0000 7f8eedaa3640 1 -- 172.21.15.16:0/3501303441 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8ee8101de0 msgr2=0x7f8ee81983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:15.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:15.671+0000 7f8eedaa3640 1 --2- 172.21.15.16:0/3501303441 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8ee8101de0 0x7f8ee81983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:15.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:15.671+0000 7f8eedaa3640 1 -- 172.21.15.16:0/3501303441 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8ee8102790 msgr2=0x7f8ee8198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:15.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:15.671+0000 7f8eedaa3640 1 --2- 172.21.15.16:0/3501303441 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8ee8102790 0x7f8ee8198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:15.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:15.671+0000 7f8eedaa3640 1 -- 172.21.15.16:0/3501303441 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ee819d380 con 0x7f8ee8106f10 2024-09-12T01:59:15.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:15.671+0000 7f8eecaa1640 1 --2- 172.21.15.16:0/3501303441 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8ee8102790 0x7f8ee8198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:59:15.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:15.671+0000 7f8eedaa3640 1 --2- 172.21.15.16:0/3501303441 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ee8106f10 0x7f8ee819cca0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f8ee0005ab0 tx=0x7f8ee00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:59:15.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:15.671+0000 7f8ee67fc640 1 -- 172.21.15.16:0/3501303441 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ee003f030 con 0x7f8ee8106f10 2024-09-12T01:59:15.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:15.671+0000 7f8eef506640 1 -- 172.21.15.16:0/3501303441 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ee819d610 con 0x7f8ee8106f10 2024-09-12T01:59:15.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:15.671+0000 7f8ee67fc640 1 -- 172.21.15.16:0/3501303441 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ee0003ce0 con 0x7f8ee8106f10 2024-09-12T01:59:15.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:15.671+0000 7f8ee67fc640 1 -- 172.21.15.16:0/3501303441 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ee0035070 con 0x7f8ee8106f10 2024-09-12T01:59:15.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:15.671+0000 7f8eef506640 1 -- 172.21.15.16:0/3501303441 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ee81a4f50 con 0x7f8ee8106f10 2024-09-12T01:59:15.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:15.672+0000 7f8eef506640 1 -- 172.21.15.16:0/3501303441 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ee81021c0 con 0x7f8ee8106f10 2024-09-12T01:59:15.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:15.673+0000 7f8ee67fc640 1 -- 172.21.15.16:0/3501303441 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f8ee0042030 con 0x7f8ee8106f10 2024-09-12T01:59:15.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:15.674+0000 7f8ee67fc640 1 --2- 172.21.15.16:0/3501303441 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f8ed00a2b80 0x7f8ed00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:15.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:15.674+0000 7f8eed2a2640 1 --2- 172.21.15.16:0/3501303441 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f8ed00a2b80 0x7f8ed00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:15.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:15.674+0000 7f8ee67fc640 1 -- 172.21.15.16:0/3501303441 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7f8ee00ef380 con 0x7f8ee8106f10 2024-09-12T01:59:15.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:15.674+0000 7f8eed2a2640 1 --2- 172.21.15.16:0/3501303441 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f8ed00a2b80 0x7f8ed00a5040 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f8ed8005c40 tx=0x7f8ed800a570 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:59:15.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:15.678+0000 7f8ee67fc640 1 -- 172.21.15.16:0/3501303441 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8ee00bc780 con 0x7f8ee8106f10 2024-09-12T01:59:15.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:15.889+0000 7f8eef506640 1 -- 172.21.15.16:0/3501303441 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8ee810a150 con 0x7f8ee8106f10 2024-09-12T01:59:15.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:15.890+0000 7f8ee67fc640 1 -- 172.21.15.16:0/3501303441 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+611 (secure 0 0 0) 0x7f8ee00bc080 con 0x7f8ee8106f10 2024-09-12T01:59:15.890 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:59:15.890 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":1,"btime":"2024-09-12T01:52:12:840866+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-09-12T01:59:15.890 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 1 2024-09-12T01:59:15.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:15.891+0000 7f8eef506640 1 -- 172.21.15.16:0/3501303441 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f8ed00a2b80 msgr2=0x7f8ed00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:15.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:15.891+0000 7f8eef506640 1 --2- 172.21.15.16:0/3501303441 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f8ed00a2b80 0x7f8ed00a5040 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f8ed8005c40 tx=0x7f8ed800a570 comp rx=0 tx=0).stop 2024-09-12T01:59:15.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:15.892+0000 7f8eef506640 1 -- 172.21.15.16:0/3501303441 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ee8106f10 msgr2=0x7f8ee819cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:15.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:15.892+0000 7f8eef506640 1 --2- 172.21.15.16:0/3501303441 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ee8106f10 0x7f8ee819cca0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f8ee0005ab0 tx=0x7f8ee00042f0 comp rx=0 tx=0).stop 2024-09-12T01:59:15.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:15.892+0000 7f8eef506640 1 -- 172.21.15.16:0/3501303441 shutdown_connections 2024-09-12T01:59:15.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:15.892+0000 7f8eef506640 1 --2- 172.21.15.16:0/3501303441 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f8ed00a2b80 0x7f8ed00a5040 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:15.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:15.892+0000 7f8eef506640 1 --2- 172.21.15.16:0/3501303441 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ee8106f10 0x7f8ee819cca0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:15.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:15.892+0000 7f8eef506640 1 --2- 172.21.15.16:0/3501303441 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8ee8102790 0x7f8ee8198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:15.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:15.892+0000 7f8eef506640 1 --2- 172.21.15.16:0/3501303441 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8ee8101de0 0x7f8ee81983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:15.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:15.892+0000 7f8eef506640 1 -- 172.21.15.16:0/3501303441 >> 172.21.15.16:0/3501303441 conn(0x7f8ee80fd8c0 msgr2=0x7f8ee80feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:59:15.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:15.892+0000 7f8eef506640 1 -- 172.21.15.16:0/3501303441 shutdown_connections 2024-09-12T01:59:15.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:15.892+0000 7f8eef506640 1 -- 172.21.15.16:0/3501303441 wait complete. 2024-09-12T01:59:16.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:16 smithi184 ceph-mon[35593]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:59:16.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:16 smithi184 ceph-mon[35593]: Deploying daemon mds.l on smithi184 2024-09-12T01:59:16.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:16 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/3501303441' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:59:16.378 INFO:journalctl@ceph.mds.l.smithi184.stdout:Sep 12 01:59:16 smithi184 systemd[1]: Starting Ceph mds.l for 879f3b56-70a9-11ef-bcea-c7b262605968... 2024-09-12T01:59:16.616 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:16 smithi016 ceph-mon[30947]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:59:16.616 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:16 smithi016 ceph-mon[30947]: Deploying daemon mds.l on smithi184 2024-09-12T01:59:16.616 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:16 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/3501303441' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:59:16.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:16 smithi033 ceph-mon[35332]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:59:16.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:16 smithi033 ceph-mon[35332]: Deploying daemon mds.l on smithi184 2024-09-12T01:59:16.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:16 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/3501303441' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:59:16.679 INFO:journalctl@ceph.mds.l.smithi184.stdout:Sep 12 01:59:16 smithi184 podman[65734]: 2024-09-12 01:59:16.425473311 +0000 UTC m=+0.096965514 container create 92d03507ab25f00d5299bb4ff7ffca1518a2bb5882f8288b3ca53201f0021122 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mds-l, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T01:59:16.679 INFO:journalctl@ceph.mds.l.smithi184.stdout:Sep 12 01:59:16 smithi184 podman[65734]: 2024-09-12 01:59:16.344323723 +0000 UTC m=+0.015815928 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T01:59:16.680 INFO:journalctl@ceph.mds.l.smithi184.stdout:Sep 12 01:59:16 smithi184 podman[65734]: 2024-09-12 01:59:16.653072169 +0000 UTC m=+0.324564385 container init 92d03507ab25f00d5299bb4ff7ffca1518a2bb5882f8288b3ca53201f0021122 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mds-l, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-09-12T01:59:16.680 INFO:journalctl@ceph.mds.l.smithi184.stdout:Sep 12 01:59:16 smithi184 podman[65734]: 2024-09-12 01:59:16.655925224 +0000 UTC m=+0.327417425 container start 92d03507ab25f00d5299bb4ff7ffca1518a2bb5882f8288b3ca53201f0021122 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mds-l, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T01:59:16.680 INFO:journalctl@ceph.mds.l.smithi184.stdout:Sep 12 01:59:16 smithi184 ceph-mds[65753]: -- 172.21.15.184:0/3857923835 <== mon.1 v2:172.21.15.184:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555623241860 con 0x55562321f000 2024-09-12T01:59:17.127 INFO:journalctl@ceph.mds.l.smithi184.stdout:Sep 12 01:59:16 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mds-l[65749]: starting mds.l at 2024-09-12T01:59:17.127 INFO:journalctl@ceph.mds.l.smithi184.stdout:Sep 12 01:59:16 smithi184 bash[65734]: 92d03507ab25f00d5299bb4ff7ffca1518a2bb5882f8288b3ca53201f0021122 2024-09-12T01:59:17.127 INFO:journalctl@ceph.mds.l.smithi184.stdout:Sep 12 01:59:16 smithi184 systemd[1]: Started Ceph mds.l for 879f3b56-70a9-11ef-bcea-c7b262605968. 2024-09-12T01:59:17.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:17 smithi033 ceph-mon[35332]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:59:17.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:17 smithi033 ceph-mon[35332]: mds.? [v2:172.21.15.184:6832/1563374106,v1:172.21.15.184:6833/1563374106] up:boot 2024-09-12T01:59:17.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:17 smithi033 ceph-mon[35332]: fsmap 1 up:standby 2024-09-12T01:59:17.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:17 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-12T01:59:17.920 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:17 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:17.920 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:17 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:17.920 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:17 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:17.920 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:17 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T01:59:17.920 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:17 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T01:59:17.920 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:17 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:59:17.920 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:17 smithi033 ceph-mon[35332]: Deploying daemon mds.k on smithi033 2024-09-12T01:59:18.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:17 smithi184 ceph-mon[35593]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:59:18.128 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:17 smithi184 ceph-mon[35593]: mds.? [v2:172.21.15.184:6832/1563374106,v1:172.21.15.184:6833/1563374106] up:boot 2024-09-12T01:59:18.128 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:17 smithi184 ceph-mon[35593]: fsmap 1 up:standby 2024-09-12T01:59:18.128 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:17 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-12T01:59:18.128 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:17 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:18.128 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:17 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:18.128 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:17 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:18.128 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:17 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T01:59:18.128 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:17 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T01:59:18.128 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:17 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:59:18.128 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:17 smithi184 ceph-mon[35593]: Deploying daemon mds.k on smithi033 2024-09-12T01:59:18.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:17 smithi016 ceph-mon[30947]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:59:18.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:17 smithi016 ceph-mon[30947]: mds.? [v2:172.21.15.184:6832/1563374106,v1:172.21.15.184:6833/1563374106] up:boot 2024-09-12T01:59:18.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:17 smithi016 ceph-mon[30947]: fsmap 1 up:standby 2024-09-12T01:59:18.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:17 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-12T01:59:18.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:17 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:18.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:17 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:18.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:17 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:18.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:17 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T01:59:18.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:17 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T01:59:18.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:17 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:59:18.185 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:17 smithi016 ceph-mon[30947]: Deploying daemon mds.k on smithi033 2024-09-12T01:59:18.648 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T01:59:18.669 INFO:journalctl@ceph.mds.k.smithi033.stdout:Sep 12 01:59:18 smithi033 systemd[1]: Starting Ceph mds.k for 879f3b56-70a9-11ef-bcea-c7b262605968... 2024-09-12T01:59:18.918 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:59:19.030 INFO:journalctl@ceph.mds.k.smithi033.stdout:Sep 12 01:59:18 smithi033 podman[64214]: 2024-09-12 01:59:18.75978827 +0000 UTC m=+0.092963828 container create 266f2fa2e5ecb5c485e3ab27d2df05088f0b3190b2d973d3678c6cf2370bf668 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mds-k, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, ceph=True) 2024-09-12T01:59:19.030 INFO:journalctl@ceph.mds.k.smithi033.stdout:Sep 12 01:59:18 smithi033 podman[64214]: 2024-09-12 01:59:18.682817814 +0000 UTC m=+0.015993382 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T01:59:19.030 INFO:journalctl@ceph.mds.k.smithi033.stdout:Sep 12 01:59:19 smithi033 podman[64214]: 2024-09-12 01:59:19.004054407 +0000 UTC m=+0.337229966 container init 266f2fa2e5ecb5c485e3ab27d2df05088f0b3190b2d973d3678c6cf2370bf668 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mds-k, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-09-12T01:59:19.030 INFO:journalctl@ceph.mds.k.smithi033.stdout:Sep 12 01:59:19 smithi033 podman[64214]: 2024-09-12 01:59:19.006895347 +0000 UTC m=+0.340070911 container start 266f2fa2e5ecb5c485e3ab27d2df05088f0b3190b2d973d3678c6cf2370bf668 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mds-k, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T01:59:19.419 INFO:journalctl@ceph.mds.k.smithi033.stdout:Sep 12 01:59:19 smithi033 ceph-mds[64233]: -- 172.21.15.33:0/3403211449 <== mon.2 v2:172.21.15.33:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559e2f2ff860 con 0x559e2f2dd000 2024-09-12T01:59:19.419 INFO:journalctl@ceph.mds.k.smithi033.stdout:Sep 12 01:59:19 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mds-k[64229]: starting mds.k at 2024-09-12T01:59:19.420 INFO:journalctl@ceph.mds.k.smithi033.stdout:Sep 12 01:59:19 smithi033 bash[64214]: 266f2fa2e5ecb5c485e3ab27d2df05088f0b3190b2d973d3678c6cf2370bf668 2024-09-12T01:59:19.420 INFO:journalctl@ceph.mds.k.smithi033.stdout:Sep 12 01:59:19 smithi033 systemd[1]: Started Ceph mds.k for 879f3b56-70a9-11ef-bcea-c7b262605968. 2024-09-12T01:59:19.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:19.443+0000 7fc09fd41640 1 -- 172.21.15.16:0/3666900065 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc098106f00 msgr2=0x7fc09810ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:19.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:19.443+0000 7fc09fd41640 1 --2- 172.21.15.16:0/3666900065 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc098106f00 0x7fc09810ea80 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc090009ca0 tx=0x7fc09002a8c0 comp rx=0 tx=0).stop 2024-09-12T01:59:19.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:19.443+0000 7fc09fd41640 1 -- 172.21.15.16:0/3666900065 shutdown_connections 2024-09-12T01:59:19.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:19.443+0000 7fc09fd41640 1 --2- 172.21.15.16:0/3666900065 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc098106f00 0x7fc09810ea80 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:19.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:19.443+0000 7fc09fd41640 1 --2- 172.21.15.16:0/3666900065 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc098102780 0x7fc0981067d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:19.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:19.443+0000 7fc09fd41640 1 --2- 172.21.15.16:0/3666900065 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc098101dd0 0x7fc0981021b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:19.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:19.443+0000 7fc09fd41640 1 -- 172.21.15.16:0/3666900065 >> 172.21.15.16:0/3666900065 conn(0x7fc0980fd890 msgr2=0x7fc0980ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:59:19.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:19.443+0000 7fc09fd41640 1 -- 172.21.15.16:0/3666900065 shutdown_connections 2024-09-12T01:59:19.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:19.443+0000 7fc09fd41640 1 -- 172.21.15.16:0/3666900065 wait complete. 2024-09-12T01:59:19.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:19.443+0000 7fc09fd41640 1 Processor -- start 2024-09-12T01:59:19.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:19.444+0000 7fc09fd41640 1 -- start start 2024-09-12T01:59:19.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:19.444+0000 7fc09fd41640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc098101dd0 0x7fc0980744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:19.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:19.444+0000 7fc09fd41640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc098102780 0x7fc098074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:19.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:19.444+0000 7fc09fd41640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc098106f00 0x7fc098074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:19.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:19.444+0000 7fc09fd41640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc098111f60 con 0x7fc098106f00 2024-09-12T01:59:19.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:19.444+0000 7fc09fd41640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc098111de0 con 0x7fc098102780 2024-09-12T01:59:19.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:19.444+0000 7fc09fd41640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc0981120e0 con 0x7fc098101dd0 2024-09-12T01:59:19.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:19.444+0000 7fc09e2de640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc098106f00 0x7fc098074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:19.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:19.444+0000 7fc09e2de640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc098106f00 0x7fc098074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44036/0 (socket says 172.21.15.16:44036) 2024-09-12T01:59:19.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:19.444+0000 7fc09e2de640 1 -- 172.21.15.16:0/1078220135 learned_addr learned my addr 172.21.15.16:0/1078220135 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:59:19.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:19.444+0000 7fc09d2dc640 1 --2- 172.21.15.16:0/1078220135 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc098102780 0x7fc098074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:19.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:19.444+0000 7fc09dadd640 1 --2- 172.21.15.16:0/1078220135 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc098101dd0 0x7fc0980744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:19.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:19.444+0000 7fc09e2de640 1 -- 172.21.15.16:0/1078220135 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc098101dd0 msgr2=0x7fc0980744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:19.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:19.444+0000 7fc09e2de640 1 --2- 172.21.15.16:0/1078220135 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc098101dd0 0x7fc0980744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:19.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:19.444+0000 7fc09e2de640 1 -- 172.21.15.16:0/1078220135 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc098102780 msgr2=0x7fc098074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:19.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:19.444+0000 7fc09e2de640 1 --2- 172.21.15.16:0/1078220135 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc098102780 0x7fc098074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:19.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:19.444+0000 7fc09e2de640 1 -- 172.21.15.16:0/1078220135 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0981a6870 con 0x7fc098106f00 2024-09-12T01:59:19.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:19.444+0000 7fc09dadd640 1 --2- 172.21.15.16:0/1078220135 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc098101dd0 0x7fc0980744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:59:19.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:19.444+0000 7fc09e2de640 1 --2- 172.21.15.16:0/1078220135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc098106f00 0x7fc098074f70 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fc090031a20 tx=0x7fc090031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:59:19.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:19.445+0000 7fc08effd640 1 -- 172.21.15.16:0/1078220135 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc090003c30 con 0x7fc098106f00 2024-09-12T01:59:19.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:19.445+0000 7fc08effd640 1 -- 172.21.15.16:0/1078220135 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc090003dd0 con 0x7fc098106f00 2024-09-12T01:59:19.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:19.445+0000 7fc09fd41640 1 -- 172.21.15.16:0/1078220135 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0981a6b00 con 0x7fc098106f00 2024-09-12T01:59:19.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:19.445+0000 7fc08effd640 1 -- 172.21.15.16:0/1078220135 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc09002fcf0 con 0x7fc098106f00 2024-09-12T01:59:19.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:19.445+0000 7fc09fd41640 1 -- 172.21.15.16:0/1078220135 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0981a7040 con 0x7fc098106f00 2024-09-12T01:59:19.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:19.446+0000 7fc09fd41640 1 -- 172.21.15.16:0/1078220135 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0981021b0 con 0x7fc098106f00 2024-09-12T01:59:19.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:19.447+0000 7fc08effd640 1 -- 172.21.15.16:0/1078220135 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fc090004050 con 0x7fc098106f00 2024-09-12T01:59:19.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:19.448+0000 7fc08effd640 1 --2- 172.21.15.16:0/1078220135 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc0840a2b80 0x7fc0840a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:19.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:19.448+0000 7fc08effd640 1 -- 172.21.15.16:0/1078220135 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7fc0900f01a0 con 0x7fc098106f00 2024-09-12T01:59:19.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:19.448+0000 7fc09dadd640 1 --2- 172.21.15.16:0/1078220135 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc0840a2b80 0x7fc0840a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:19.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:19.449+0000 7fc09dadd640 1 --2- 172.21.15.16:0/1078220135 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc0840a2b80 0x7fc0840a5040 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fc0940058c0 tx=0x7fc094005d30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:59:19.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:19.453+0000 7fc08effd640 1 -- 172.21.15.16:0/1078220135 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc0900bd590 con 0x7fc098106f00 2024-09-12T01:59:19.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:19.676+0000 7fc09fd41640 1 -- 172.21.15.16:0/1078220135 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc09810a0e0 con 0x7fc098106f00 2024-09-12T01:59:19.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:19.677+0000 7fc08effd640 1 -- 172.21.15.16:0/1078220135 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1438 (secure 0 0 0) 0x7fc09002fe90 con 0x7fc098106f00 2024-09-12T01:59:19.678 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:59:19.678 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":2,"btime":"2024-09-12T01:59:16:684678+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6833/1563374106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1563374106},{"type":"v1","addr":"172.21.15.184:6833","nonce":1563374106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-12T01:59:19.678 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 2 2024-09-12T01:59:19.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:19.679+0000 7fc09fd41640 1 -- 172.21.15.16:0/1078220135 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc0840a2b80 msgr2=0x7fc0840a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:19.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:19.679+0000 7fc09fd41640 1 --2- 172.21.15.16:0/1078220135 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc0840a2b80 0x7fc0840a5040 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fc0940058c0 tx=0x7fc094005d30 comp rx=0 tx=0).stop 2024-09-12T01:59:19.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:19.679+0000 7fc09fd41640 1 -- 172.21.15.16:0/1078220135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc098106f00 msgr2=0x7fc098074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:19.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:19.679+0000 7fc09fd41640 1 --2- 172.21.15.16:0/1078220135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc098106f00 0x7fc098074f70 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fc090031a20 tx=0x7fc090031a50 comp rx=0 tx=0).stop 2024-09-12T01:59:19.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:19.679+0000 7fc09fd41640 1 -- 172.21.15.16:0/1078220135 shutdown_connections 2024-09-12T01:59:19.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:19.679+0000 7fc09fd41640 1 --2- 172.21.15.16:0/1078220135 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc0840a2b80 0x7fc0840a5040 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:19.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:19.679+0000 7fc09fd41640 1 --2- 172.21.15.16:0/1078220135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc098106f00 0x7fc098074f70 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:19.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:19.679+0000 7fc09fd41640 1 --2- 172.21.15.16:0/1078220135 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc098102780 0x7fc098074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:19.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:19.679+0000 7fc09fd41640 1 --2- 172.21.15.16:0/1078220135 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc098101dd0 0x7fc0980744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:19.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:19.679+0000 7fc09fd41640 1 -- 172.21.15.16:0/1078220135 >> 172.21.15.16:0/1078220135 conn(0x7fc0980fd890 msgr2=0x7fc0980fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:59:19.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:19.679+0000 7fc09fd41640 1 -- 172.21.15.16:0/1078220135 shutdown_connections 2024-09-12T01:59:19.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:19.679+0000 7fc09fd41640 1 -- 172.21.15.16:0/1078220135 wait complete. 2024-09-12T01:59:20.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:20 smithi033 ceph-mon[35332]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:59:20.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:20 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:20.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:20 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:20.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:20 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:20.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:20 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T01:59:20.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:20 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T01:59:20.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:20 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:59:20.420 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:20 smithi033 ceph-mon[35332]: Deploying daemon mds.j on smithi016 2024-09-12T01:59:20.420 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:20 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/1078220135' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:59:20.420 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:20 smithi033 ceph-mon[35332]: mds.? [v2:172.21.15.33:6832/279674603,v1:172.21.15.33:6833/279674603] up:boot 2024-09-12T01:59:20.420 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:20 smithi033 ceph-mon[35332]: fsmap 2 up:standby 2024-09-12T01:59:20.420 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:20 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-12T01:59:20.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:20 smithi016 ceph-mon[30947]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:59:20.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:20 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:20.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:20 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:20.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:20 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:20.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:20 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T01:59:20.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:20 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T01:59:20.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:20 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:59:20.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:20 smithi016 ceph-mon[30947]: Deploying daemon mds.j on smithi016 2024-09-12T01:59:20.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:20 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/1078220135' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:59:20.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:20 smithi016 ceph-mon[30947]: mds.? [v2:172.21.15.33:6832/279674603,v1:172.21.15.33:6833/279674603] up:boot 2024-09-12T01:59:20.435 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:20 smithi016 ceph-mon[30947]: fsmap 2 up:standby 2024-09-12T01:59:20.435 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:20 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-12T01:59:20.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:20 smithi184 ceph-mon[35593]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:59:20.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:20 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:20.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:20 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:20.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:20 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:20.628 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:20 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T01:59:20.628 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:20 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T01:59:20.628 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:20 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:59:20.628 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:20 smithi184 ceph-mon[35593]: Deploying daemon mds.j on smithi016 2024-09-12T01:59:20.628 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:20 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/1078220135' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:59:20.628 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:20 smithi184 ceph-mon[35593]: mds.? [v2:172.21.15.33:6832/279674603,v1:172.21.15.33:6833/279674603] up:boot 2024-09-12T01:59:20.628 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:20 smithi184 ceph-mon[35593]: fsmap 2 up:standby 2024-09-12T01:59:20.628 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:20 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-12T01:59:22.122 INFO:journalctl@ceph.mds.j.smithi016.stdout:Sep 12 01:59:21 smithi016 systemd[1]: Starting Ceph mds.j for 879f3b56-70a9-11ef-bcea-c7b262605968... 2024-09-12T01:59:22.379 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:22 smithi016 ceph-mon[30947]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:59:22.379 INFO:journalctl@ceph.mds.j.smithi016.stdout:Sep 12 01:59:22 smithi016 podman[71007]: 2024-09-12 01:59:22.025372622 +0000 UTC m=+0.016754782 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T01:59:22.380 INFO:journalctl@ceph.mds.j.smithi016.stdout:Sep 12 01:59:22 smithi016 podman[71007]: 2024-09-12 01:59:22.133286958 +0000 UTC m=+0.124669106 container create 83e1c73e17f49f25005772cddb315a01007b657e86555f2ad9f91e04f8e5e1b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mds-j, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2) 2024-09-12T01:59:22.380 INFO:journalctl@ceph.mds.j.smithi016.stdout:Sep 12 01:59:22 smithi016 podman[71007]: 2024-09-12 01:59:22.352232223 +0000 UTC m=+0.343614372 container init 83e1c73e17f49f25005772cddb315a01007b657e86555f2ad9f91e04f8e5e1b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mds-j, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD) 2024-09-12T01:59:22.380 INFO:journalctl@ceph.mds.j.smithi016.stdout:Sep 12 01:59:22 smithi016 podman[71007]: 2024-09-12 01:59:22.355268037 +0000 UTC m=+0.346650186 container start 83e1c73e17f49f25005772cddb315a01007b657e86555f2ad9f91e04f8e5e1b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mds-j, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T01:59:22.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:22 smithi033 ceph-mon[35332]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:59:22.626 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T01:59:22.628 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:22 smithi184 ceph-mon[35593]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:59:22.650 INFO:journalctl@ceph.mds.j.smithi016.stdout:Sep 12 01:59:22 smithi016 ceph-mds[71026]: -- 172.21.15.16:0/1377588494 <== mon.0 v2:172.21.15.16:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56334b653860 con 0x56334b631000 2024-09-12T01:59:22.651 INFO:journalctl@ceph.mds.j.smithi016.stdout:Sep 12 01:59:22 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mds-j[71022]: starting mds.j at 2024-09-12T01:59:22.651 INFO:journalctl@ceph.mds.j.smithi016.stdout:Sep 12 01:59:22 smithi016 bash[71007]: 83e1c73e17f49f25005772cddb315a01007b657e86555f2ad9f91e04f8e5e1b6 2024-09-12T01:59:22.651 INFO:journalctl@ceph.mds.j.smithi016.stdout:Sep 12 01:59:22 smithi016 systemd[1]: Started Ceph mds.j for 879f3b56-70a9-11ef-bcea-c7b262605968. 2024-09-12T01:59:23.437 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:59:23.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:23 smithi016 ceph-mon[30947]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:59:23.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:23 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:23.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:23 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:23.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:23 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:23.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:23 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T01:59:23.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:23 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T01:59:23.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:23 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:59:23.685 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:23 smithi016 ceph-mon[30947]: Deploying daemon mds.d on smithi016 2024-09-12T01:59:23.685 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:23 smithi016 ceph-mon[30947]: mds.? [v2:172.21.15.16:6834/3279123279,v1:172.21.15.16:6835/3279123279] up:boot 2024-09-12T01:59:23.685 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:23 smithi016 ceph-mon[30947]: fsmap 3 up:standby 2024-09-12T01:59:23.685 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:23 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-12T01:59:23.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:23 smithi184 ceph-mon[35593]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:59:23.878 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:23 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:23.878 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:23 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:23.878 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:23 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:23.878 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:23 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T01:59:23.878 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:23 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T01:59:23.878 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:23 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:59:23.878 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:23 smithi184 ceph-mon[35593]: Deploying daemon mds.d on smithi016 2024-09-12T01:59:23.878 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:23 smithi184 ceph-mon[35593]: mds.? [v2:172.21.15.16:6834/3279123279,v1:172.21.15.16:6835/3279123279] up:boot 2024-09-12T01:59:23.878 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:23 smithi184 ceph-mon[35593]: fsmap 3 up:standby 2024-09-12T01:59:23.878 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:23 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-12T01:59:23.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:23 smithi033 ceph-mon[35332]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:59:23.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:23 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:23.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:23 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:23.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:23 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:23.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:23 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T01:59:23.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:23 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T01:59:23.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:23 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:59:23.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:23 smithi033 ceph-mon[35332]: Deploying daemon mds.d on smithi016 2024-09-12T01:59:23.920 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:23 smithi033 ceph-mon[35332]: mds.? [v2:172.21.15.16:6834/3279123279,v1:172.21.15.16:6835/3279123279] up:boot 2024-09-12T01:59:23.920 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:23 smithi033 ceph-mon[35332]: fsmap 3 up:standby 2024-09-12T01:59:23.920 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:23 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-12T01:59:24.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:24.296+0000 7fbae9386640 1 -- 172.21.15.16:0/1400131210 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbae4101440 msgr2=0x7fbae410df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:24.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:24.297+0000 7fbae9386640 1 --2- 172.21.15.16:0/1400131210 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbae4101440 0x7fbae410df30 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fbad4009900 tx=0x7fbad402a410 comp rx=0 tx=0).stop 2024-09-12T01:59:24.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:24.297+0000 7fbae9386640 1 -- 172.21.15.16:0/1400131210 shutdown_connections 2024-09-12T01:59:24.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:24.297+0000 7fbae9386640 1 --2- 172.21.15.16:0/1400131210 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbae410e500 0x7fbae4110910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:24.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:24.297+0000 7fbae9386640 1 --2- 172.21.15.16:0/1400131210 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbae4101440 0x7fbae410df30 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:24.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:24.297+0000 7fbae9386640 1 --2- 172.21.15.16:0/1400131210 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbae4100b20 0x7fbae4100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:24.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:24.297+0000 7fbae9386640 1 -- 172.21.15.16:0/1400131210 >> 172.21.15.16:0/1400131210 conn(0x7fbae40fc7d0 msgr2=0x7fbae40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:59:24.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:24.297+0000 7fbae9386640 1 -- 172.21.15.16:0/1400131210 shutdown_connections 2024-09-12T01:59:24.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:24.297+0000 7fbae9386640 1 -- 172.21.15.16:0/1400131210 wait complete. 2024-09-12T01:59:24.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:24.297+0000 7fbae9386640 1 Processor -- start 2024-09-12T01:59:24.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:24.297+0000 7fbae9386640 1 -- start start 2024-09-12T01:59:24.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:24.298+0000 7fbae9386640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbae4100b20 0x7fbae419a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:24.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:24.298+0000 7fbae9386640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbae4101440 0x7fbae419ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:24.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:24.298+0000 7fbae9386640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbae410e500 0x7fbae419efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:24.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:24.298+0000 7fbae9386640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbae41142c0 con 0x7fbae4101440 2024-09-12T01:59:24.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:24.298+0000 7fbae9386640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbae4114140 con 0x7fbae410e500 2024-09-12T01:59:24.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:24.298+0000 7fbae9386640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fbae4114440 con 0x7fbae4100b20 2024-09-12T01:59:24.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:24.298+0000 7fbae27fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbae4101440 0x7fbae419ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:24.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:24.298+0000 7fbae27fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbae4101440 0x7fbae419ac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52160/0 (socket says 172.21.15.16:52160) 2024-09-12T01:59:24.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:24.298+0000 7fbae27fc640 1 -- 172.21.15.16:0/482115546 learned_addr learned my addr 172.21.15.16:0/482115546 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:59:24.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:24.298+0000 7fbae37fe640 1 --2- 172.21.15.16:0/482115546 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbae410e500 0x7fbae419efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:24.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:24.298+0000 7fbae2ffd640 1 --2- 172.21.15.16:0/482115546 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbae4100b20 0x7fbae419a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:24.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:24.298+0000 7fbae27fc640 1 -- 172.21.15.16:0/482115546 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbae4100b20 msgr2=0x7fbae419a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:24.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:24.298+0000 7fbae27fc640 1 --2- 172.21.15.16:0/482115546 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbae4100b20 0x7fbae419a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:24.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:24.298+0000 7fbae27fc640 1 -- 172.21.15.16:0/482115546 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbae410e500 msgr2=0x7fbae419efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:24.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:24.298+0000 7fbae27fc640 1 --2- 172.21.15.16:0/482115546 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbae410e500 0x7fbae419efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:24.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:24.298+0000 7fbae27fc640 1 -- 172.21.15.16:0/482115546 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbae419f6b0 con 0x7fbae4101440 2024-09-12T01:59:24.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:24.298+0000 7fbae27fc640 1 --2- 172.21.15.16:0/482115546 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbae4101440 0x7fbae419ac40 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fbad402a930 tx=0x7fbad4003c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:59:24.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:24.298+0000 7fbadbfff640 1 -- 172.21.15.16:0/482115546 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbad403f070 con 0x7fbae4101440 2024-09-12T01:59:24.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:24.298+0000 7fbadbfff640 1 -- 172.21.15.16:0/482115546 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbad403b460 con 0x7fbae4101440 2024-09-12T01:59:24.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:24.298+0000 7fbae9386640 1 -- 172.21.15.16:0/482115546 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbae419f940 con 0x7fbae4101440 2024-09-12T01:59:24.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:24.298+0000 7fbadbfff640 1 -- 172.21.15.16:0/482115546 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbad4035030 con 0x7fbae4101440 2024-09-12T01:59:24.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:24.299+0000 7fbae9386640 1 -- 172.21.15.16:0/482115546 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbae41a7280 con 0x7fbae4101440 2024-09-12T01:59:24.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:24.299+0000 7fbae9386640 1 -- 172.21.15.16:0/482115546 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbae406a850 con 0x7fbae4101440 2024-09-12T01:59:24.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:24.301+0000 7fbadbfff640 1 -- 172.21.15.16:0/482115546 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fbad4042070 con 0x7fbae4101440 2024-09-12T01:59:24.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:24.302+0000 7fbadbfff640 1 --2- 172.21.15.16:0/482115546 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fbad00a2b80 0x7fbad00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:24.353 INFO:journalctl@ceph.mds.d.smithi016.stdout:Sep 12 01:59:24 smithi016 systemd[1]: Starting Ceph mds.d for 879f3b56-70a9-11ef-bcea-c7b262605968... 2024-09-12T01:59:24.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:24.302+0000 7fbae2ffd640 1 --2- 172.21.15.16:0/482115546 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fbad00a2b80 0x7fbad00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:24.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:24.302+0000 7fbadbfff640 1 -- 172.21.15.16:0/482115546 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7fbad40ef8f0 con 0x7fbae4101440 2024-09-12T01:59:24.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:24.302+0000 7fbae2ffd640 1 --2- 172.21.15.16:0/482115546 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fbad00a2b80 0x7fbad00a5040 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fbacc005fd0 tx=0x7fbacc005e20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:59:24.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:24.305+0000 7fbadbfff640 1 -- 172.21.15.16:0/482115546 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbad40bcc50 con 0x7fbae4101440 2024-09-12T01:59:24.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:24.527+0000 7fbae9386640 1 -- 172.21.15.16:0/482115546 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbae4100f00 con 0x7fbae4101440 2024-09-12T01:59:24.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:24.527+0000 7fbadbfff640 1 -- 172.21.15.16:0/482115546 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3085 (secure 0 0 0) 0x7fbae4100f00 con 0x7fbae4101440 2024-09-12T01:59:24.529 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:59:24.529 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":4,"btime":"2024-09-12T01:59:23:164094+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3279123279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3279123279},{"type":"v1","addr":"172.21.15.16:6835","nonce":3279123279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/279674603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":279674603},{"type":"v1","addr":"172.21.15.33:6833","nonce":279674603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6833/1563374106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1563374106},{"type":"v1","addr":"172.21.15.184:6833","nonce":1563374106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-12T01:59:24.529 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 4 2024-09-12T01:59:24.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:24.530+0000 7fbae9386640 1 -- 172.21.15.16:0/482115546 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fbad00a2b80 msgr2=0x7fbad00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:24.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:24.530+0000 7fbae9386640 1 --2- 172.21.15.16:0/482115546 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fbad00a2b80 0x7fbad00a5040 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fbacc005fd0 tx=0x7fbacc005e20 comp rx=0 tx=0).stop 2024-09-12T01:59:24.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:24.530+0000 7fbae9386640 1 -- 172.21.15.16:0/482115546 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbae4101440 msgr2=0x7fbae419ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:24.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:24.530+0000 7fbae9386640 1 --2- 172.21.15.16:0/482115546 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbae4101440 0x7fbae419ac40 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fbad402a930 tx=0x7fbad4003c00 comp rx=0 tx=0).stop 2024-09-12T01:59:24.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:24.530+0000 7fbae9386640 1 -- 172.21.15.16:0/482115546 shutdown_connections 2024-09-12T01:59:24.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:24.530+0000 7fbae9386640 1 --2- 172.21.15.16:0/482115546 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fbad00a2b80 0x7fbad00a5040 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:24.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:24.530+0000 7fbae9386640 1 --2- 172.21.15.16:0/482115546 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbae410e500 0x7fbae419efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:24.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:24.530+0000 7fbae9386640 1 --2- 172.21.15.16:0/482115546 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbae4101440 0x7fbae419ac40 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:24.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:24.530+0000 7fbae9386640 1 --2- 172.21.15.16:0/482115546 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbae4100b20 0x7fbae419a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:24.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:24.530+0000 7fbae9386640 1 -- 172.21.15.16:0/482115546 >> 172.21.15.16:0/482115546 conn(0x7fbae40fc7d0 msgr2=0x7fbae40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:59:24.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:24.530+0000 7fbae9386640 1 -- 172.21.15.16:0/482115546 shutdown_connections 2024-09-12T01:59:24.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:24.530+0000 7fbae9386640 1 -- 172.21.15.16:0/482115546 wait complete. 2024-09-12T01:59:24.645 INFO:journalctl@ceph.mds.d.smithi016.stdout:Sep 12 01:59:24 smithi016 podman[71411]: 2024-09-12 01:59:24.531982839 +0000 UTC m=+0.108516414 container create 0fcdf8c628453f89a0c0cbdc1791df24d93a3933bba94155544dcf9ca06e1ce0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mds-d, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD) 2024-09-12T01:59:24.645 INFO:journalctl@ceph.mds.d.smithi016.stdout:Sep 12 01:59:24 smithi016 podman[71411]: 2024-09-12 01:59:24.439875653 +0000 UTC m=+0.016409228 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T01:59:25.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:25 smithi016 ceph-mon[30947]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:59:25.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:25 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/482115546' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:59:25.684 INFO:journalctl@ceph.mds.d.smithi016.stdout:Sep 12 01:59:25 smithi016 podman[71411]: 2024-09-12 01:59:25.597587871 +0000 UTC m=+1.174121462 container init 0fcdf8c628453f89a0c0cbdc1791df24d93a3933bba94155544dcf9ca06e1ce0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mds-d, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-09-12T01:59:25.684 INFO:journalctl@ceph.mds.d.smithi016.stdout:Sep 12 01:59:25 smithi016 podman[71411]: 2024-09-12 01:59:25.601857695 +0000 UTC m=+1.178391270 container start 0fcdf8c628453f89a0c0cbdc1791df24d93a3933bba94155544dcf9ca06e1ce0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mds-d, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.license=GPLv2) 2024-09-12T01:59:25.684 INFO:journalctl@ceph.mds.d.smithi016.stdout:Sep 12 01:59:25 smithi016 ceph-mds[71447]: -- 172.21.15.16:0/1203050598 <== mon.0 v2:172.21.15.16:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558f70ab3860 con 0x558f70a91400 2024-09-12T01:59:25.684 INFO:journalctl@ceph.mds.d.smithi016.stdout:Sep 12 01:59:25 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mds-d[71440]: starting mds.d at 2024-09-12T01:59:25.684 INFO:journalctl@ceph.mds.d.smithi016.stdout:Sep 12 01:59:25 smithi016 bash[71411]: 0fcdf8c628453f89a0c0cbdc1791df24d93a3933bba94155544dcf9ca06e1ce0 2024-09-12T01:59:25.684 INFO:journalctl@ceph.mds.d.smithi016.stdout:Sep 12 01:59:25 smithi016 systemd[1]: Started Ceph mds.d for 879f3b56-70a9-11ef-bcea-c7b262605968. 2024-09-12T01:59:25.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:25 smithi184 ceph-mon[35593]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:59:25.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:25 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/482115546' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:59:25.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:25 smithi033 ceph-mon[35332]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:59:25.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:25 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/482115546' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:59:26.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:26 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:26.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:26 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:26.920 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:26 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:26.920 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:26 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T01:59:26.920 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:26 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T01:59:26.920 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:26 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:59:26.920 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:26 smithi033 ceph-mon[35332]: Deploying daemon mds.e on smithi033 2024-09-12T01:59:26.920 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:26 smithi033 ceph-mon[35332]: mds.? [v2:172.21.15.16:6836/4119452382,v1:172.21.15.16:6837/4119452382] up:boot 2024-09-12T01:59:26.920 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:26 smithi033 ceph-mon[35332]: fsmap 4 up:standby 2024-09-12T01:59:26.920 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:26 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-12T01:59:27.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:26 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:27.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:26 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:27.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:26 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:27.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:26 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T01:59:27.128 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:26 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T01:59:27.128 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:26 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:59:27.128 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:26 smithi184 ceph-mon[35593]: Deploying daemon mds.e on smithi033 2024-09-12T01:59:27.128 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:26 smithi184 ceph-mon[35593]: mds.? [v2:172.21.15.16:6836/4119452382,v1:172.21.15.16:6837/4119452382] up:boot 2024-09-12T01:59:27.128 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:26 smithi184 ceph-mon[35593]: fsmap 4 up:standby 2024-09-12T01:59:27.128 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:26 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-12T01:59:27.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:26 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:27.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:26 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:27.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:26 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:27.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:26 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T01:59:27.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:26 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T01:59:27.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:26 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:59:27.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:26 smithi016 ceph-mon[30947]: Deploying daemon mds.e on smithi033 2024-09-12T01:59:27.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:26 smithi016 ceph-mon[30947]: mds.? [v2:172.21.15.16:6836/4119452382,v1:172.21.15.16:6837/4119452382] up:boot 2024-09-12T01:59:27.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:26 smithi016 ceph-mon[30947]: fsmap 4 up:standby 2024-09-12T01:59:27.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:26 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-12T01:59:27.464 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T01:59:27.669 INFO:journalctl@ceph.mds.e.smithi033.stdout:Sep 12 01:59:27 smithi033 systemd[1]: Starting Ceph mds.e for 879f3b56-70a9-11ef-bcea-c7b262605968... 2024-09-12T01:59:27.737 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:59:27.859 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:27 smithi016 ceph-mon[30947]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:59:27.997 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:27 smithi033 ceph-mon[35332]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:59:27.997 INFO:journalctl@ceph.mds.e.smithi033.stdout:Sep 12 01:59:27 smithi033 podman[64529]: 2024-09-12 01:59:27.698820251 +0000 UTC m=+0.016159721 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T01:59:27.998 INFO:journalctl@ceph.mds.e.smithi033.stdout:Sep 12 01:59:27 smithi033 podman[64529]: 2024-09-12 01:59:27.813024676 +0000 UTC m=+0.130364134 container create ecbc94849c8c5af03e665944ba85e537b6bff8376f18bb0284fc6c5d4a11c6a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mds-e, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-09-12T01:59:27.998 INFO:journalctl@ceph.mds.e.smithi033.stdout:Sep 12 01:59:27 smithi033 podman[64529]: 2024-09-12 01:59:27.998129774 +0000 UTC m=+0.315469231 container init ecbc94849c8c5af03e665944ba85e537b6bff8376f18bb0284fc6c5d4a11c6a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mds-e, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3) 2024-09-12T01:59:28.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:27 smithi184 ceph-mon[35593]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:59:28.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:28.232+0000 7f7a7f098640 1 -- 172.21.15.16:0/1440476626 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a78102570 msgr2=0x7f7a781029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:28.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:28.232+0000 7f7a7f098640 1 --2- 172.21.15.16:0/1440476626 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a78102570 0x7f7a781029b0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f7a64009900 tx=0x7f7a6402a480 comp rx=0 tx=0).stop 2024-09-12T01:59:28.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:28.233+0000 7f7a7f098640 1 -- 172.21.15.16:0/1440476626 shutdown_connections 2024-09-12T01:59:28.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:28.233+0000 7f7a7f098640 1 --2- 172.21.15.16:0/1440476626 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a78102ef0 0x7f7a7810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:28.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:28.233+0000 7f7a7f098640 1 --2- 172.21.15.16:0/1440476626 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a78102570 0x7f7a781029b0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:28.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:28.233+0000 7f7a7f098640 1 --2- 172.21.15.16:0/1440476626 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7a78100e20 0x7f7a78101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:28.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:28.233+0000 7f7a7f098640 1 -- 172.21.15.16:0/1440476626 >> 172.21.15.16:0/1440476626 conn(0x7f7a780fc7d0 msgr2=0x7f7a780febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:59:28.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:28.233+0000 7f7a7f098640 1 -- 172.21.15.16:0/1440476626 shutdown_connections 2024-09-12T01:59:28.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:28.233+0000 7f7a7f098640 1 -- 172.21.15.16:0/1440476626 wait complete. 2024-09-12T01:59:28.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:28.233+0000 7f7a7f098640 1 Processor -- start 2024-09-12T01:59:28.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:28.233+0000 7f7a7f098640 1 -- start start 2024-09-12T01:59:28.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:28.233+0000 7f7a7f098640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a78100e20 0x7f7a7819a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:28.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:28.233+0000 7f7a7f098640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7a78102570 0x7f7a7819abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:28.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:28.233+0000 7f7a7f098640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a78102ef0 0x7f7a7819ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:28.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:28.233+0000 7f7a7f098640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7a78110f80 con 0x7f7a78100e20 2024-09-12T01:59:28.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:28.233+0000 7f7a7f098640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7a78110e00 con 0x7f7a78102ef0 2024-09-12T01:59:28.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:28.233+0000 7f7a7f098640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7a78111100 con 0x7f7a78102570 2024-09-12T01:59:28.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:28.234+0000 7f7a7ce34640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a78100e20 0x7f7a7819a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:28.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:28.234+0000 7f7a7ce34640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a78100e20 0x7f7a7819a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52198/0 (socket says 172.21.15.16:52198) 2024-09-12T01:59:28.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:28.234+0000 7f7a7ce34640 1 -- 172.21.15.16:0/601886386 learned_addr learned my addr 172.21.15.16:0/601886386 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:59:28.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:28.234+0000 7f7a7d635640 1 --2- 172.21.15.16:0/601886386 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a78102ef0 0x7f7a7819ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:28.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:28.234+0000 7f7a77fff640 1 --2- 172.21.15.16:0/601886386 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7a78102570 0x7f7a7819abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:28.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:28.234+0000 7f7a7ce34640 1 -- 172.21.15.16:0/601886386 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7a78102570 msgr2=0x7f7a7819abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:28.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:28.234+0000 7f7a7ce34640 1 --2- 172.21.15.16:0/601886386 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7a78102570 0x7f7a7819abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:28.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:28.234+0000 7f7a7ce34640 1 -- 172.21.15.16:0/601886386 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a78102ef0 msgr2=0x7f7a7819ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:28.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:28.234+0000 7f7a7ce34640 1 --2- 172.21.15.16:0/601886386 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a78102ef0 0x7f7a7819ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:28.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:28.234+0000 7f7a7ce34640 1 -- 172.21.15.16:0/601886386 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a7819f660 con 0x7f7a78100e20 2024-09-12T01:59:28.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:28.234+0000 7f7a7d635640 1 --2- 172.21.15.16:0/601886386 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a78102ef0 0x7f7a7819ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:59:28.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:28.234+0000 7f7a7ce34640 1 --2- 172.21.15.16:0/601886386 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a78100e20 0x7f7a7819a6b0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f7a6c00d950 tx=0x7f7a6c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:59:28.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:28.234+0000 7f7a75ffb640 1 -- 172.21.15.16:0/601886386 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a6c017070 con 0x7f7a78100e20 2024-09-12T01:59:28.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:28.234+0000 7f7a75ffb640 1 -- 172.21.15.16:0/601886386 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a6c009c30 con 0x7f7a78100e20 2024-09-12T01:59:28.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:28.234+0000 7f7a7f098640 1 -- 172.21.15.16:0/601886386 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a7819f950 con 0x7f7a78100e20 2024-09-12T01:59:28.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:28.234+0000 7f7a75ffb640 1 -- 172.21.15.16:0/601886386 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a6c00f6b0 con 0x7f7a78100e20 2024-09-12T01:59:28.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:28.235+0000 7f7a7f098640 1 -- 172.21.15.16:0/601886386 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a781a7230 con 0x7f7a78100e20 2024-09-12T01:59:28.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:28.235+0000 7f7a7f098640 1 -- 172.21.15.16:0/601886386 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a78101200 con 0x7f7a78100e20 2024-09-12T01:59:28.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:28.236+0000 7f7a75ffb640 1 -- 172.21.15.16:0/601886386 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f7a6c0109d0 con 0x7f7a78100e20 2024-09-12T01:59:28.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:28.237+0000 7f7a75ffb640 1 --2- 172.21.15.16:0/601886386 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f7a680a2b80 0x7f7a680a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:28.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:28.237+0000 7f7a75ffb640 1 -- 172.21.15.16:0/601886386 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7f7a6c0ce780 con 0x7f7a78100e20 2024-09-12T01:59:28.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:28.237+0000 7f7a77fff640 1 --2- 172.21.15.16:0/601886386 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f7a680a2b80 0x7f7a680a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:28.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:28.238+0000 7f7a77fff640 1 --2- 172.21.15.16:0/601886386 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f7a680a2b80 0x7f7a680a5040 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f7a64009230 tx=0x7f7a640058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:59:28.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:28.242+0000 7f7a75ffb640 1 -- 172.21.15.16:0/601886386 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7a6c009de0 con 0x7f7a78100e20 2024-09-12T01:59:28.419 INFO:journalctl@ceph.mds.e.smithi033.stdout:Sep 12 01:59:28 smithi033 podman[64529]: 2024-09-12 01:59:28.001179539 +0000 UTC m=+0.318518997 container start ecbc94849c8c5af03e665944ba85e537b6bff8376f18bb0284fc6c5d4a11c6a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mds-e, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS) 2024-09-12T01:59:28.419 INFO:journalctl@ceph.mds.e.smithi033.stdout:Sep 12 01:59:28 smithi033 ceph-mds[64549]: -- 172.21.15.33:0/4217351860 <== mon.2 v2:172.21.15.33:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5600bb9fb860 con 0x5600bb9d9000 2024-09-12T01:59:28.419 INFO:journalctl@ceph.mds.e.smithi033.stdout:Sep 12 01:59:28 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mds-e[64545]: starting mds.e at 2024-09-12T01:59:28.419 INFO:journalctl@ceph.mds.e.smithi033.stdout:Sep 12 01:59:28 smithi033 bash[64529]: ecbc94849c8c5af03e665944ba85e537b6bff8376f18bb0284fc6c5d4a11c6a5 2024-09-12T01:59:28.419 INFO:journalctl@ceph.mds.e.smithi033.stdout:Sep 12 01:59:28 smithi033 systemd[1]: Started Ceph mds.e for 879f3b56-70a9-11ef-bcea-c7b262605968. 2024-09-12T01:59:28.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:28.454+0000 7f7a7f098640 1 -- 172.21.15.16:0/601886386 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7a781a74c0 con 0x7f7a78100e20 2024-09-12T01:59:28.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:28.455+0000 7f7a75ffb640 1 -- 172.21.15.16:0/601886386 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3910 (secure 0 0 0) 0x7f7a6c09b530 con 0x7f7a78100e20 2024-09-12T01:59:28.456 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:59:28.456 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":5,"btime":"2024-09-12T01:59:26:514294+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3279123279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3279123279},{"type":"v1","addr":"172.21.15.16:6835","nonce":3279123279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4119452382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4119452382},{"type":"v1","addr":"172.21.15.16:6837","nonce":4119452382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/279674603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":279674603},{"type":"v1","addr":"172.21.15.33:6833","nonce":279674603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6833/1563374106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1563374106},{"type":"v1","addr":"172.21.15.184:6833","nonce":1563374106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-12T01:59:28.456 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 5 2024-09-12T01:59:28.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:28.457+0000 7f7a7f098640 1 -- 172.21.15.16:0/601886386 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f7a680a2b80 msgr2=0x7f7a680a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:28.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:28.457+0000 7f7a7f098640 1 --2- 172.21.15.16:0/601886386 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f7a680a2b80 0x7f7a680a5040 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f7a64009230 tx=0x7f7a640058d0 comp rx=0 tx=0).stop 2024-09-12T01:59:28.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:28.457+0000 7f7a7f098640 1 -- 172.21.15.16:0/601886386 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a78100e20 msgr2=0x7f7a7819a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:28.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:28.457+0000 7f7a7f098640 1 --2- 172.21.15.16:0/601886386 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a78100e20 0x7f7a7819a6b0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f7a6c00d950 tx=0x7f7a6c00de10 comp rx=0 tx=0).stop 2024-09-12T01:59:28.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:28.457+0000 7f7a7f098640 1 -- 172.21.15.16:0/601886386 shutdown_connections 2024-09-12T01:59:28.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:28.457+0000 7f7a7f098640 1 --2- 172.21.15.16:0/601886386 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f7a680a2b80 0x7f7a680a5040 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:28.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:28.457+0000 7f7a7f098640 1 --2- 172.21.15.16:0/601886386 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a78102ef0 0x7f7a7819ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:28.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:28.457+0000 7f7a7f098640 1 --2- 172.21.15.16:0/601886386 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7a78102570 0x7f7a7819abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:28.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:28.457+0000 7f7a7f098640 1 --2- 172.21.15.16:0/601886386 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a78100e20 0x7f7a7819a6b0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:28.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:28.457+0000 7f7a7f098640 1 -- 172.21.15.16:0/601886386 >> 172.21.15.16:0/601886386 conn(0x7f7a780fc7d0 msgr2=0x7f7a780ffc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:59:28.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:28.457+0000 7f7a7f098640 1 -- 172.21.15.16:0/601886386 shutdown_connections 2024-09-12T01:59:28.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:28.457+0000 7f7a7f098640 1 -- 172.21.15.16:0/601886386 wait complete. 2024-09-12T01:59:29.180 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:29 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:29.180 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:29 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:29.180 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:29 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:29.180 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:29 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T01:59:29.180 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:29 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T01:59:29.180 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:29 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:59:29.180 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:29 smithi016 ceph-mon[30947]: Deploying daemon mds.f on smithi184 2024-09-12T01:59:29.180 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:29 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/601886386' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:59:29.180 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:29 smithi016 ceph-mon[30947]: mds.? [v2:172.21.15.33:6834/3555728028,v1:172.21.15.33:6835/3555728028] up:boot 2024-09-12T01:59:29.180 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:29 smithi016 ceph-mon[30947]: fsmap 5 up:standby 2024-09-12T01:59:29.181 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:29 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-12T01:59:29.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:29 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:29.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:29 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:29.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:29 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:29.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:29 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T01:59:29.420 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:29 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T01:59:29.420 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:29 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:59:29.420 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:29 smithi033 ceph-mon[35332]: Deploying daemon mds.f on smithi184 2024-09-12T01:59:29.420 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:29 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/601886386' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:59:29.420 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:29 smithi033 ceph-mon[35332]: mds.? [v2:172.21.15.33:6834/3555728028,v1:172.21.15.33:6835/3555728028] up:boot 2024-09-12T01:59:29.420 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:29 smithi033 ceph-mon[35332]: fsmap 5 up:standby 2024-09-12T01:59:29.420 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:29 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-12T01:59:29.509 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:29 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:29.509 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:29 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:29.509 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:29 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:29.509 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:29 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T01:59:29.509 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:29 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T01:59:29.509 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:29 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:59:29.509 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:29 smithi184 ceph-mon[35593]: Deploying daemon mds.f on smithi184 2024-09-12T01:59:29.510 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:29 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/601886386' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:59:29.510 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:29 smithi184 ceph-mon[35593]: mds.? [v2:172.21.15.33:6834/3555728028,v1:172.21.15.33:6835/3555728028] up:boot 2024-09-12T01:59:29.510 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:29 smithi184 ceph-mon[35593]: fsmap 5 up:standby 2024-09-12T01:59:29.510 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:29 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-12T01:59:30.127 INFO:journalctl@ceph.mds.f.smithi184.stdout:Sep 12 01:59:29 smithi184 systemd[1]: Starting Ceph mds.f for 879f3b56-70a9-11ef-bcea-c7b262605968... 2024-09-12T01:59:30.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:30 smithi033 ceph-mon[35332]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:59:30.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:30 smithi016 ceph-mon[30947]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:59:30.480 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:30 smithi184 ceph-mon[35593]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:59:30.480 INFO:journalctl@ceph.mds.f.smithi184.stdout:Sep 12 01:59:30 smithi184 podman[66033]: 2024-09-12 01:59:30.103404687 +0000 UTC m=+0.015842051 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T01:59:30.480 INFO:journalctl@ceph.mds.f.smithi184.stdout:Sep 12 01:59:30 smithi184 podman[66033]: 2024-09-12 01:59:30.22807764 +0000 UTC m=+0.140514999 container create c4f79993c34fdb8c78d42cd77177487df31e421987d1b8304c11d5ca8ad7ac69 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mds-f, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, ceph=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-09-12T01:59:30.480 INFO:journalctl@ceph.mds.f.smithi184.stdout:Sep 12 01:59:30 smithi184 podman[66033]: 2024-09-12 01:59:30.480605867 +0000 UTC m=+0.393043219 container init c4f79993c34fdb8c78d42cd77177487df31e421987d1b8304c11d5ca8ad7ac69 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mds-f, ceph=True, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux ) 2024-09-12T01:59:30.877 INFO:journalctl@ceph.mds.f.smithi184.stdout:Sep 12 01:59:30 smithi184 podman[66033]: 2024-09-12 01:59:30.483632869 +0000 UTC m=+0.396070225 container start c4f79993c34fdb8c78d42cd77177487df31e421987d1b8304c11d5ca8ad7ac69 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mds-f, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, ceph=True) 2024-09-12T01:59:30.877 INFO:journalctl@ceph.mds.f.smithi184.stdout:Sep 12 01:59:30 smithi184 ceph-mds[66053]: -- 172.21.15.184:0/3393567191 <== mon.1 v2:172.21.15.184:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b2dd67d860 con 0x55b2dd65b000 2024-09-12T01:59:30.877 INFO:journalctl@ceph.mds.f.smithi184.stdout:Sep 12 01:59:30 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mds-f[66049]: starting mds.f at 2024-09-12T01:59:30.877 INFO:journalctl@ceph.mds.f.smithi184.stdout:Sep 12 01:59:30 smithi184 bash[66033]: c4f79993c34fdb8c78d42cd77177487df31e421987d1b8304c11d5ca8ad7ac69 2024-09-12T01:59:30.878 INFO:journalctl@ceph.mds.f.smithi184.stdout:Sep 12 01:59:30 smithi184 systemd[1]: Started Ceph mds.f for 879f3b56-70a9-11ef-bcea-c7b262605968. 2024-09-12T01:59:31.214 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T01:59:31.482 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:59:31.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:31 smithi184 ceph-mon[35593]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:59:31.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:31 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:31.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:31 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:31.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:31 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:31.878 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:31 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T01:59:31.878 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:31 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T01:59:31.878 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:31 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:59:31.878 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:31 smithi184 ceph-mon[35593]: Deploying daemon mds.c on smithi184 2024-09-12T01:59:31.878 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:31 smithi184 ceph-mon[35593]: mds.? [v2:172.21.15.184:6834/1457679686,v1:172.21.15.184:6835/1457679686] up:boot 2024-09-12T01:59:31.878 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:31 smithi184 ceph-mon[35593]: fsmap 6 up:standby 2024-09-12T01:59:31.878 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:31 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-12T01:59:31.882 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:31 smithi016 ceph-mon[30947]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:59:31.882 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:31 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:31.883 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:31 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:31.883 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:31 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:31.883 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:31 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T01:59:31.883 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:31 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T01:59:31.883 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:31 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:59:31.883 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:31 smithi016 ceph-mon[30947]: Deploying daemon mds.c on smithi184 2024-09-12T01:59:31.883 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:31 smithi016 ceph-mon[30947]: mds.? [v2:172.21.15.184:6834/1457679686,v1:172.21.15.184:6835/1457679686] up:boot 2024-09-12T01:59:31.883 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:31 smithi016 ceph-mon[30947]: fsmap 6 up:standby 2024-09-12T01:59:31.883 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:31 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-12T01:59:31.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:31 smithi033 ceph-mon[35332]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:59:31.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:31 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:31.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:31 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:31.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:31 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:31.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:31 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T01:59:31.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:31 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T01:59:31.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:31 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:59:31.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:31 smithi033 ceph-mon[35332]: Deploying daemon mds.c on smithi184 2024-09-12T01:59:31.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:31 smithi033 ceph-mon[35332]: mds.? [v2:172.21.15.184:6834/1457679686,v1:172.21.15.184:6835/1457679686] up:boot 2024-09-12T01:59:31.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:31 smithi033 ceph-mon[35332]: fsmap 6 up:standby 2024-09-12T01:59:31.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:31 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-12T01:59:31.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:31.964+0000 7f7d63797640 1 -- 172.21.15.16:0/2604596670 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d5c10c180 msgr2=0x7f7d5c10e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:31.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:31.964+0000 7f7d63797640 1 --2- 172.21.15.16:0/2604596670 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d5c10c180 0x7f7d5c10e600 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f7d54009ca0 tx=0x7f7d5402a950 comp rx=0 tx=0).stop 2024-09-12T01:59:31.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:31.964+0000 7f7d63797640 1 -- 172.21.15.16:0/2604596670 shutdown_connections 2024-09-12T01:59:31.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:31.964+0000 7f7d63797640 1 --2- 172.21.15.16:0/2604596670 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d5c10c180 0x7f7d5c10e600 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:31.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:31.964+0000 7f7d63797640 1 --2- 172.21.15.16:0/2604596670 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d5c1036c0 0x7f7d5c10bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:31.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:31.964+0000 7f7d63797640 1 --2- 172.21.15.16:0/2604596670 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7d5c102da0 0x7f7d5c103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:31.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:31.964+0000 7f7d63797640 1 -- 172.21.15.16:0/2604596670 >> 172.21.15.16:0/2604596670 conn(0x7f7d5c0fc7a0 msgr2=0x7f7d5c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:59:31.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:31.964+0000 7f7d63797640 1 -- 172.21.15.16:0/2604596670 shutdown_connections 2024-09-12T01:59:31.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:31.964+0000 7f7d63797640 1 -- 172.21.15.16:0/2604596670 wait complete. 2024-09-12T01:59:31.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:31.965+0000 7f7d63797640 1 Processor -- start 2024-09-12T01:59:31.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:31.965+0000 7f7d63797640 1 -- start start 2024-09-12T01:59:31.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:31.966+0000 7f7d63797640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7d5c102da0 0x7f7d5c198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:31.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:31.966+0000 7f7d63797640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d5c1036c0 0x7f7d5c198840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:31.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:31.966+0000 7f7d63797640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d5c10c180 0x7f7d5c19cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:31.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:31.966+0000 7f7d63797640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7d5c111f20 con 0x7f7d5c1036c0 2024-09-12T01:59:31.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:31.966+0000 7f7d63797640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7d5c111da0 con 0x7f7d5c10c180 2024-09-12T01:59:31.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:31.966+0000 7f7d63797640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7d5c1120a0 con 0x7f7d5c102da0 2024-09-12T01:59:31.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:31.966+0000 7f7d60d32640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d5c1036c0 0x7f7d5c198840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:31.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:31.966+0000 7f7d60d32640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d5c1036c0 0x7f7d5c198840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48110/0 (socket says 172.21.15.16:48110) 2024-09-12T01:59:31.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:31.966+0000 7f7d60d32640 1 -- 172.21.15.16:0/1624882056 learned_addr learned my addr 172.21.15.16:0/1624882056 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:59:31.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:31.966+0000 7f7d61d34640 1 --2- 172.21.15.16:0/1624882056 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d5c10c180 0x7f7d5c19cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:31.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:31.966+0000 7f7d61533640 1 --2- 172.21.15.16:0/1624882056 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7d5c102da0 0x7f7d5c198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:31.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:31.966+0000 7f7d60d32640 1 -- 172.21.15.16:0/1624882056 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7d5c102da0 msgr2=0x7f7d5c198300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:31.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:31.966+0000 7f7d60d32640 1 --2- 172.21.15.16:0/1624882056 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7d5c102da0 0x7f7d5c198300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:31.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:31.966+0000 7f7d60d32640 1 -- 172.21.15.16:0/1624882056 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d5c10c180 msgr2=0x7f7d5c19cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:31.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:31.966+0000 7f7d60d32640 1 --2- 172.21.15.16:0/1624882056 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d5c10c180 0x7f7d5c19cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:31.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:31.966+0000 7f7d60d32640 1 -- 172.21.15.16:0/1624882056 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d5c19d2b0 con 0x7f7d5c1036c0 2024-09-12T01:59:31.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:31.966+0000 7f7d60d32640 1 --2- 172.21.15.16:0/1624882056 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d5c1036c0 0x7f7d5c198840 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f7d4c00d950 tx=0x7f7d4c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:59:31.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:31.966+0000 7f7d527fc640 1 -- 172.21.15.16:0/1624882056 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d4c017070 con 0x7f7d5c1036c0 2024-09-12T01:59:31.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:31.966+0000 7f7d527fc640 1 -- 172.21.15.16:0/1624882056 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d4c009c30 con 0x7f7d5c1036c0 2024-09-12T01:59:31.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:31.966+0000 7f7d63797640 1 -- 172.21.15.16:0/1624882056 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d5c19d5a0 con 0x7f7d5c1036c0 2024-09-12T01:59:31.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:31.966+0000 7f7d527fc640 1 -- 172.21.15.16:0/1624882056 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d4c00f6b0 con 0x7f7d5c1036c0 2024-09-12T01:59:31.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:31.966+0000 7f7d63797640 1 -- 172.21.15.16:0/1624882056 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d5c1a4e80 con 0x7f7d5c1036c0 2024-09-12T01:59:31.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:31.968+0000 7f7d63797640 1 -- 172.21.15.16:0/1624882056 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d5c103180 con 0x7f7d5c1036c0 2024-09-12T01:59:31.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:31.969+0000 7f7d527fc640 1 -- 172.21.15.16:0/1624882056 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f7d4c0109d0 con 0x7f7d5c1036c0 2024-09-12T01:59:31.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:31.969+0000 7f7d527fc640 1 --2- 172.21.15.16:0/1624882056 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f7d480a2c00 0x7f7d480a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:31.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:31.970+0000 7f7d61533640 1 --2- 172.21.15.16:0/1624882056 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f7d480a2c00 0x7f7d480a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:31.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:31.970+0000 7f7d527fc640 1 -- 172.21.15.16:0/1624882056 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7f7d4c0ce6f0 con 0x7f7d5c1036c0 2024-09-12T01:59:31.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:31.970+0000 7f7d61533640 1 --2- 172.21.15.16:0/1624882056 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f7d480a2c00 0x7f7d480a50c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f7d58005e10 tx=0x7f7d58005da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:59:31.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:31.974+0000 7f7d527fc640 1 -- 172.21.15.16:0/1624882056 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7d4c009de0 con 0x7f7d5c1036c0 2024-09-12T01:59:32.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:32.184+0000 7f7d63797640 1 -- 172.21.15.16:0/1624882056 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7d5c1a5110 con 0x7f7d5c1036c0 2024-09-12T01:59:32.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:32.185+0000 7f7d527fc640 1 -- 172.21.15.16:0/1624882056 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5563 (secure 0 0 0) 0x7f7d4c09baf0 con 0x7f7d5c1036c0 2024-09-12T01:59:32.185 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:59:32.186 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":7,"btime":"2024-09-12T01:59:31:155117+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3279123279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3279123279},{"type":"v1","addr":"172.21.15.16:6835","nonce":3279123279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4119452382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4119452382},{"type":"v1","addr":"172.21.15.16:6837","nonce":4119452382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/279674603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":279674603},{"type":"v1","addr":"172.21.15.33:6833","nonce":279674603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3555728028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3555728028},{"type":"v1","addr":"172.21.15.33:6835","nonce":3555728028}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6833/1563374106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1563374106},{"type":"v1","addr":"172.21.15.184:6833","nonce":1563374106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24460,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/1457679686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1457679686},{"type":"v1","addr":"172.21.15.184:6835","nonce":1457679686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[]} 2024-09-12T01:59:32.186 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 7 2024-09-12T01:59:32.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:32.187+0000 7f7d63797640 1 -- 172.21.15.16:0/1624882056 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f7d480a2c00 msgr2=0x7f7d480a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:32.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:32.187+0000 7f7d63797640 1 --2- 172.21.15.16:0/1624882056 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f7d480a2c00 0x7f7d480a50c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f7d58005e10 tx=0x7f7d58005da0 comp rx=0 tx=0).stop 2024-09-12T01:59:32.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:32.187+0000 7f7d63797640 1 -- 172.21.15.16:0/1624882056 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d5c1036c0 msgr2=0x7f7d5c198840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:32.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:32.187+0000 7f7d63797640 1 --2- 172.21.15.16:0/1624882056 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d5c1036c0 0x7f7d5c198840 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f7d4c00d950 tx=0x7f7d4c00de10 comp rx=0 tx=0).stop 2024-09-12T01:59:32.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:32.187+0000 7f7d63797640 1 -- 172.21.15.16:0/1624882056 shutdown_connections 2024-09-12T01:59:32.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:32.187+0000 7f7d63797640 1 --2- 172.21.15.16:0/1624882056 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f7d480a2c00 0x7f7d480a50c0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:32.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:32.187+0000 7f7d63797640 1 --2- 172.21.15.16:0/1624882056 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d5c10c180 0x7f7d5c19cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:32.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:32.187+0000 7f7d63797640 1 --2- 172.21.15.16:0/1624882056 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d5c1036c0 0x7f7d5c198840 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:32.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:32.187+0000 7f7d63797640 1 --2- 172.21.15.16:0/1624882056 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7d5c102da0 0x7f7d5c198300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:32.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:32.187+0000 7f7d63797640 1 -- 172.21.15.16:0/1624882056 >> 172.21.15.16:0/1624882056 conn(0x7f7d5c0fc7a0 msgr2=0x7f7d5c0fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:59:32.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:32.187+0000 7f7d63797640 1 -- 172.21.15.16:0/1624882056 shutdown_connections 2024-09-12T01:59:32.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:32.187+0000 7f7d63797640 1 -- 172.21.15.16:0/1624882056 wait complete. 2024-09-12T01:59:32.579 INFO:journalctl@ceph.mds.c.smithi184.stdout:Sep 12 01:59:32 smithi184 systemd[1]: Starting Ceph mds.c for 879f3b56-70a9-11ef-bcea-c7b262605968... 2024-09-12T01:59:32.842 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:32 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/1624882056' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:59:32.842 INFO:journalctl@ceph.mds.c.smithi184.stdout:Sep 12 01:59:32 smithi184 podman[66332]: 2024-09-12 01:59:32.482208582 +0000 UTC m=+0.015783757 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T01:59:32.842 INFO:journalctl@ceph.mds.c.smithi184.stdout:Sep 12 01:59:32 smithi184 podman[66332]: 2024-09-12 01:59:32.598117824 +0000 UTC m=+0.131692995 container create 89511b817656cdabf3dd5099dca77de03f11c21ab58eba3be482c8b8bb72753b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mds-c, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.37.2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-12T01:59:32.842 INFO:journalctl@ceph.mds.c.smithi184.stdout:Sep 12 01:59:32 smithi184 podman[66332]: 2024-09-12 01:59:32.842530953 +0000 UTC m=+0.376106128 container init 89511b817656cdabf3dd5099dca77de03f11c21ab58eba3be482c8b8bb72753b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mds-c, GIT_CLEAN=True, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0) 2024-09-12T01:59:32.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:32 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/1624882056' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:59:32.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:32 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/1624882056' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:59:33.127 INFO:journalctl@ceph.mds.c.smithi184.stdout:Sep 12 01:59:32 smithi184 podman[66332]: 2024-09-12 01:59:32.845490315 +0000 UTC m=+0.379065478 container start 89511b817656cdabf3dd5099dca77de03f11c21ab58eba3be482c8b8bb72753b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mds-c, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3) 2024-09-12T01:59:33.127 INFO:journalctl@ceph.mds.c.smithi184.stdout:Sep 12 01:59:32 smithi184 ceph-mds[66351]: -- 172.21.15.184:0/934889367 <== mon.1 v2:172.21.15.184:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557751cf9c20 con 0x557751cd7000 2024-09-12T01:59:33.127 INFO:journalctl@ceph.mds.c.smithi184.stdout:Sep 12 01:59:32 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mds-c[66347]: starting mds.c at 2024-09-12T01:59:33.128 INFO:journalctl@ceph.mds.c.smithi184.stdout:Sep 12 01:59:32 smithi184 bash[66332]: 89511b817656cdabf3dd5099dca77de03f11c21ab58eba3be482c8b8bb72753b 2024-09-12T01:59:33.128 INFO:journalctl@ceph.mds.c.smithi184.stdout:Sep 12 01:59:32 smithi184 systemd[1]: Started Ceph mds.c for 879f3b56-70a9-11ef-bcea-c7b262605968. 2024-09-12T01:59:34.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:33 smithi033 ceph-mon[35332]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:59:34.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:33 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:34.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:33 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:34.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:33 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:34.170 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:33 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T01:59:34.170 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:33 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T01:59:34.170 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:33 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:59:34.170 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:33 smithi033 ceph-mon[35332]: Deploying daemon mds.h on smithi033 2024-09-12T01:59:34.170 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:33 smithi033 ceph-mon[35332]: mds.? [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] up:boot 2024-09-12T01:59:34.170 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:33 smithi033 ceph-mon[35332]: fsmap 7 up:standby 2024-09-12T01:59:34.170 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:33 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-12T01:59:34.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:33 smithi184 ceph-mon[35593]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:59:34.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:33 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:34.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:33 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:34.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:33 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:34.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:33 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T01:59:34.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:33 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T01:59:34.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:33 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:59:34.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:33 smithi184 ceph-mon[35593]: Deploying daemon mds.h on smithi033 2024-09-12T01:59:34.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:33 smithi184 ceph-mon[35593]: mds.? [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] up:boot 2024-09-12T01:59:34.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:33 smithi184 ceph-mon[35593]: fsmap 7 up:standby 2024-09-12T01:59:34.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:33 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-12T01:59:34.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:33 smithi016 ceph-mon[30947]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:59:34.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:33 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:34.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:33 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:34.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:33 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:34.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:33 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T01:59:34.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:33 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T01:59:34.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:33 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:59:34.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:33 smithi016 ceph-mon[30947]: Deploying daemon mds.h on smithi033 2024-09-12T01:59:34.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:33 smithi016 ceph-mon[30947]: mds.? [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] up:boot 2024-09-12T01:59:34.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:33 smithi016 ceph-mon[30947]: fsmap 7 up:standby 2024-09-12T01:59:34.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:33 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-12T01:59:34.896 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T01:59:35.124 INFO:journalctl@ceph.mds.h.smithi033.stdout:Sep 12 01:59:34 smithi033 systemd[1]: Starting Ceph mds.h for 879f3b56-70a9-11ef-bcea-c7b262605968... 2024-09-12T01:59:35.164 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:59:35.386 INFO:journalctl@ceph.mds.h.smithi033.stdout:Sep 12 01:59:35 smithi033 podman[64835]: 2024-09-12 01:59:35.026956422 +0000 UTC m=+0.016195820 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T01:59:35.387 INFO:journalctl@ceph.mds.h.smithi033.stdout:Sep 12 01:59:35 smithi033 podman[64835]: 2024-09-12 01:59:35.140893229 +0000 UTC m=+0.130132612 container create 56f3bcd2417377c23473b54258745dda0a9d32ea9943504666fdb154c289629d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mds-h, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3) 2024-09-12T01:59:35.387 INFO:journalctl@ceph.mds.h.smithi033.stdout:Sep 12 01:59:35 smithi033 podman[64835]: 2024-09-12 01:59:35.360052349 +0000 UTC m=+0.349291732 container init 56f3bcd2417377c23473b54258745dda0a9d32ea9943504666fdb154c289629d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mds-h, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0) 2024-09-12T01:59:35.387 INFO:journalctl@ceph.mds.h.smithi033.stdout:Sep 12 01:59:35 smithi033 podman[64835]: 2024-09-12 01:59:35.363028427 +0000 UTC m=+0.352267825 container start 56f3bcd2417377c23473b54258745dda0a9d32ea9943504666fdb154c289629d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mds-h, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-09-12T01:59:35.387 INFO:journalctl@ceph.mds.h.smithi033.stdout:Sep 12 01:59:35 smithi033 ceph-mds[64853]: -- 172.21.15.33:0/3442638490 <== mon.2 v2:172.21.15.33:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563b92a93860 con 0x563b92a71000 2024-09-12T01:59:35.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:35.613+0000 7fe4a922c640 1 -- 172.21.15.16:0/3781919881 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4a4074510 msgr2=0x7fe4a40748f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:35.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:35.613+0000 7fe4a922c640 1 --2- 172.21.15.16:0/3781919881 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4a4074510 0x7fe4a40748f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fe494009900 tx=0x7fe49402a480 comp rx=0 tx=0).stop 2024-09-12T01:59:35.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:35.613+0000 7fe4a922c640 1 -- 172.21.15.16:0/3781919881 shutdown_connections 2024-09-12T01:59:35.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:35.613+0000 7fe4a922c640 1 --2- 172.21.15.16:0/3781919881 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe4a410e450 0x7fe4a4110840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:35.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:35.613+0000 7fe4a922c640 1 --2- 172.21.15.16:0/3781919881 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe4a4074e30 0x7fe4a410df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:35.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:35.613+0000 7fe4a922c640 1 --2- 172.21.15.16:0/3781919881 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4a4074510 0x7fe4a40748f0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:35.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:35.613+0000 7fe4a922c640 1 -- 172.21.15.16:0/3781919881 >> 172.21.15.16:0/3781919881 conn(0x7fe4a40fd8f0 msgr2=0x7fe4a40ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:59:35.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:35.613+0000 7fe4a922c640 1 -- 172.21.15.16:0/3781919881 shutdown_connections 2024-09-12T01:59:35.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:35.613+0000 7fe4a922c640 1 -- 172.21.15.16:0/3781919881 wait complete. 2024-09-12T01:59:35.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:35.614+0000 7fe4a922c640 1 Processor -- start 2024-09-12T01:59:35.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:35.614+0000 7fe4a922c640 1 -- start start 2024-09-12T01:59:35.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:35.615+0000 7fe4a922c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe4a4074510 0x7fe4a419a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:35.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:35.615+0000 7fe4a922c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe4a4074e30 0x7fe4a419ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:35.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:35.615+0000 7fe4a922c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4a410e450 0x7fe4a419ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:35.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:35.615+0000 7fe4a922c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe4a41141b0 con 0x7fe4a410e450 2024-09-12T01:59:35.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:35.615+0000 7fe4a922c640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe4a4114030 con 0x7fe4a4074510 2024-09-12T01:59:35.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:35.615+0000 7fe4a922c640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe4a4114330 con 0x7fe4a4074e30 2024-09-12T01:59:35.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:35.615+0000 7fe4a359e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4a410e450 0x7fe4a419ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:35.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:35.615+0000 7fe4a359e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4a410e450 0x7fe4a419ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48126/0 (socket says 172.21.15.16:48126) 2024-09-12T01:59:35.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:35.615+0000 7fe4a359e640 1 -- 172.21.15.16:0/4052619854 learned_addr learned my addr 172.21.15.16:0/4052619854 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:59:35.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:35.615+0000 7fe4a259c640 1 --2- 172.21.15.16:0/4052619854 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe4a4074e30 0x7fe4a419ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:35.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:35.615+0000 7fe4a2d9d640 1 --2- 172.21.15.16:0/4052619854 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe4a4074510 0x7fe4a419a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:35.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:35.615+0000 7fe4a359e640 1 -- 172.21.15.16:0/4052619854 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe4a4074e30 msgr2=0x7fe4a419ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:35.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:35.615+0000 7fe4a359e640 1 --2- 172.21.15.16:0/4052619854 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe4a4074e30 0x7fe4a419ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:35.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:35.615+0000 7fe4a359e640 1 -- 172.21.15.16:0/4052619854 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe4a4074510 msgr2=0x7fe4a419a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:35.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:35.615+0000 7fe4a359e640 1 --2- 172.21.15.16:0/4052619854 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe4a4074510 0x7fe4a419a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:35.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:35.615+0000 7fe4a359e640 1 -- 172.21.15.16:0/4052619854 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4a419f600 con 0x7fe4a410e450 2024-09-12T01:59:35.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:35.615+0000 7fe4a359e640 1 --2- 172.21.15.16:0/4052619854 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4a410e450 0x7fe4a419ef20 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fe49c00e9d0 tx=0x7fe49c00ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:59:35.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:35.616+0000 7fe48bfff640 1 -- 172.21.15.16:0/4052619854 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe49c00cd80 con 0x7fe4a410e450 2024-09-12T01:59:35.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:35.616+0000 7fe4a922c640 1 -- 172.21.15.16:0/4052619854 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4a419f8f0 con 0x7fe4a410e450 2024-09-12T01:59:35.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:35.616+0000 7fe48bfff640 1 -- 172.21.15.16:0/4052619854 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe49c005020 con 0x7fe4a410e450 2024-09-12T01:59:35.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:35.616+0000 7fe48bfff640 1 -- 172.21.15.16:0/4052619854 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe49c0116d0 con 0x7fe4a410e450 2024-09-12T01:59:35.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:35.616+0000 7fe4a922c640 1 -- 172.21.15.16:0/4052619854 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4a41a71d0 con 0x7fe4a410e450 2024-09-12T01:59:35.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:35.617+0000 7fe4a922c640 1 -- 172.21.15.16:0/4052619854 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4a40748f0 con 0x7fe4a410e450 2024-09-12T01:59:35.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:35.618+0000 7fe48bfff640 1 -- 172.21.15.16:0/4052619854 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fe49c0051d0 con 0x7fe4a410e450 2024-09-12T01:59:35.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:35.619+0000 7fe48bfff640 1 --2- 172.21.15.16:0/4052619854 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fe48c0a2b80 0x7fe48c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:35.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:35.619+0000 7fe48bfff640 1 -- 172.21.15.16:0/4052619854 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7fe49c0cf080 con 0x7fe4a410e450 2024-09-12T01:59:35.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:35.619+0000 7fe4a2d9d640 1 --2- 172.21.15.16:0/4052619854 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fe48c0a2b80 0x7fe48c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:35.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:35.619+0000 7fe4a2d9d640 1 --2- 172.21.15.16:0/4052619854 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fe48c0a2b80 0x7fe48c0a5040 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fe494000c00 tx=0x7fe494005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:59:35.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:35.623+0000 7fe48bfff640 1 -- 172.21.15.16:0/4052619854 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe49c016030 con 0x7fe4a410e450 2024-09-12T01:59:35.669 INFO:journalctl@ceph.mds.h.smithi033.stdout:Sep 12 01:59:35 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mds-h[64849]: starting mds.h at 2024-09-12T01:59:35.669 INFO:journalctl@ceph.mds.h.smithi033.stdout:Sep 12 01:59:35 smithi033 bash[64835]: 56f3bcd2417377c23473b54258745dda0a9d32ea9943504666fdb154c289629d 2024-09-12T01:59:35.669 INFO:journalctl@ceph.mds.h.smithi033.stdout:Sep 12 01:59:35 smithi033 systemd[1]: Started Ceph mds.h for 879f3b56-70a9-11ef-bcea-c7b262605968. 2024-09-12T01:59:35.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:35.833+0000 7fe4a922c640 1 -- 172.21.15.16:0/4052619854 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe4a402d7c0 con 0x7fe4a410e450 2024-09-12T01:59:35.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:35.834+0000 7fe48bfff640 1 -- 172.21.15.16:0/4052619854 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6391 (secure 0 0 0) 0x7fe49c09be30 con 0x7fe4a410e450 2024-09-12T01:59:35.834 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:59:35.834 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":8,"btime":"2024-09-12T01:59:33:644337+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3279123279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3279123279},{"type":"v1","addr":"172.21.15.16:6835","nonce":3279123279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4119452382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4119452382},{"type":"v1","addr":"172.21.15.16:6837","nonce":4119452382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/279674603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":279674603},{"type":"v1","addr":"172.21.15.33:6833","nonce":279674603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3555728028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3555728028},{"type":"v1","addr":"172.21.15.33:6835","nonce":3555728028}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6833/1563374106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1563374106},{"type":"v1","addr":"172.21.15.184:6833","nonce":1563374106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24460,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/1457679686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1457679686},{"type":"v1","addr":"172.21.15.184:6835","nonce":1457679686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1793393087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":1793393087},{"type":"v1","addr":"172.21.15.184:6837","nonce":1793393087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-09-12T01:59:35.834 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 8 2024-09-12T01:59:35.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:35.836+0000 7fe4a922c640 1 -- 172.21.15.16:0/4052619854 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fe48c0a2b80 msgr2=0x7fe48c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:35.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:35.836+0000 7fe4a922c640 1 --2- 172.21.15.16:0/4052619854 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fe48c0a2b80 0x7fe48c0a5040 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fe494000c00 tx=0x7fe494005b50 comp rx=0 tx=0).stop 2024-09-12T01:59:35.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:35.836+0000 7fe4a922c640 1 -- 172.21.15.16:0/4052619854 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4a410e450 msgr2=0x7fe4a419ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:35.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:35.836+0000 7fe4a922c640 1 --2- 172.21.15.16:0/4052619854 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4a410e450 0x7fe4a419ef20 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fe49c00e9d0 tx=0x7fe49c00ee90 comp rx=0 tx=0).stop 2024-09-12T01:59:35.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:35.836+0000 7fe4a922c640 1 -- 172.21.15.16:0/4052619854 shutdown_connections 2024-09-12T01:59:35.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:35.836+0000 7fe4a922c640 1 --2- 172.21.15.16:0/4052619854 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fe48c0a2b80 0x7fe48c0a5040 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:35.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:35.836+0000 7fe4a922c640 1 --2- 172.21.15.16:0/4052619854 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4a410e450 0x7fe4a419ef20 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:35.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:35.836+0000 7fe4a922c640 1 --2- 172.21.15.16:0/4052619854 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe4a4074e30 0x7fe4a419ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:35.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:35.836+0000 7fe4a922c640 1 --2- 172.21.15.16:0/4052619854 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe4a4074510 0x7fe4a419a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:35.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:35.836+0000 7fe4a922c640 1 -- 172.21.15.16:0/4052619854 >> 172.21.15.16:0/4052619854 conn(0x7fe4a40fd8f0 msgr2=0x7fe4a40fe550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:59:35.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:35.836+0000 7fe4a922c640 1 -- 172.21.15.16:0/4052619854 shutdown_connections 2024-09-12T01:59:35.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:35.836+0000 7fe4a922c640 1 -- 172.21.15.16:0/4052619854 wait complete. 2024-09-12T01:59:36.130 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:35 smithi016 ceph-mon[30947]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:59:36.130 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:35 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:36.130 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:35 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:36.130 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:35 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:36.130 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:35 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T01:59:36.130 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:35 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T01:59:36.131 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:35 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:59:36.131 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:35 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/4052619854' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:59:36.237 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:35 smithi184 ceph-mon[35593]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:59:36.237 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:35 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:36.237 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:35 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:36.237 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:35 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:36.237 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:35 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T01:59:36.237 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:35 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T01:59:36.237 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:35 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:59:36.238 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:35 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/4052619854' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:59:36.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:35 smithi033 ceph-mon[35332]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:59:36.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:35 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:36.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:35 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:36.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:35 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:36.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:35 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T01:59:36.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:35 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T01:59:36.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:35 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:59:36.420 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:35 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/4052619854' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:59:37.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:36 smithi184 ceph-mon[35593]: Deploying daemon mds.i on smithi184 2024-09-12T01:59:37.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:37 smithi184 ceph-mon[35593]: mds.? [v2:172.21.15.33:6836/1892833868,v1:172.21.15.33:6837/1892833868] up:boot 2024-09-12T01:59:37.128 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:37 smithi184 ceph-mon[35593]: fsmap 8 up:standby 2024-09-12T01:59:37.128 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:37 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-12T01:59:37.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:36 smithi033 ceph-mon[35332]: Deploying daemon mds.i on smithi184 2024-09-12T01:59:37.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:36 smithi033 ceph-mon[35332]: mds.? [v2:172.21.15.33:6836/1892833868,v1:172.21.15.33:6837/1892833868] up:boot 2024-09-12T01:59:37.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:37 smithi033 ceph-mon[35332]: fsmap 8 up:standby 2024-09-12T01:59:37.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:37 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-12T01:59:37.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:36 smithi016 ceph-mon[30947]: Deploying daemon mds.i on smithi184 2024-09-12T01:59:37.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:36 smithi016 ceph-mon[30947]: mds.? [v2:172.21.15.33:6836/1892833868,v1:172.21.15.33:6837/1892833868] up:boot 2024-09-12T01:59:37.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:36 smithi016 ceph-mon[30947]: fsmap 8 up:standby 2024-09-12T01:59:37.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:36 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-12T01:59:37.545 INFO:journalctl@ceph.mds.i.smithi184.stdout:Sep 12 01:59:37 smithi184 systemd[1]: Starting Ceph mds.i for 879f3b56-70a9-11ef-bcea-c7b262605968... 2024-09-12T01:59:37.845 INFO:journalctl@ceph.mds.i.smithi184.stdout:Sep 12 01:59:37 smithi184 podman[66631]: 2024-09-12 01:59:37.447903851 +0000 UTC m=+0.015963212 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T01:59:37.845 INFO:journalctl@ceph.mds.i.smithi184.stdout:Sep 12 01:59:37 smithi184 podman[66631]: 2024-09-12 01:59:37.551416572 +0000 UTC m=+0.119475923 container create 8de64a118bb1603a3647adcee2ca31a315d8803f121227aae6e071336f53a2cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mds-i, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-09-12T01:59:37.845 INFO:journalctl@ceph.mds.i.smithi184.stdout:Sep 12 01:59:37 smithi184 podman[66631]: 2024-09-12 01:59:37.762062944 +0000 UTC m=+0.330122293 container init 8de64a118bb1603a3647adcee2ca31a315d8803f121227aae6e071336f53a2cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mds-i, RELEASE=squid-b53b6d3, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-09-12T01:59:37.845 INFO:journalctl@ceph.mds.i.smithi184.stdout:Sep 12 01:59:37 smithi184 podman[66631]: 2024-09-12 01:59:37.76483339 +0000 UTC m=+0.332892738 container start 8de64a118bb1603a3647adcee2ca31a315d8803f121227aae6e071336f53a2cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mds-i, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T01:59:37.845 INFO:journalctl@ceph.mds.i.smithi184.stdout:Sep 12 01:59:37 smithi184 ceph-mds[66650]: -- 172.21.15.184:0/1662294326 <== mon.1 v2:172.21.15.184:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5645f714d860 con 0x5645f712b400 2024-09-12T01:59:37.845 INFO:journalctl@ceph.mds.i.smithi184.stdout:Sep 12 01:59:37 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mds-i[66646]: starting mds.i at 2024-09-12T01:59:37.845 INFO:journalctl@ceph.mds.i.smithi184.stdout:Sep 12 01:59:37 smithi184 bash[66631]: 8de64a118bb1603a3647adcee2ca31a315d8803f121227aae6e071336f53a2cf 2024-09-12T01:59:38.127 INFO:journalctl@ceph.mds.i.smithi184.stdout:Sep 12 01:59:37 smithi184 systemd[1]: Started Ceph mds.i for 879f3b56-70a9-11ef-bcea-c7b262605968. 2024-09-12T01:59:38.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:38 smithi184 ceph-mon[35593]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:59:38.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:38 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:38.128 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:38 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:38.128 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:38 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:38.128 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:38 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T01:59:38.128 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:38 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T01:59:38.128 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:38 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:59:38.351 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:38 smithi016 ceph-mon[30947]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:59:38.351 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:38 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:38.351 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:38 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:38.351 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:38 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:38.351 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:38 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T01:59:38.351 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:38 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T01:59:38.351 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:38 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:59:38.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:38 smithi033 ceph-mon[35332]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:59:38.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:38 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:38.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:38 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:38.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:38 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:38.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:38 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T01:59:38.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:38 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T01:59:38.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:38 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:59:38.508 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T01:59:39.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:39 smithi184 ceph-mon[35593]: Deploying daemon mds.g on smithi016 2024-09-12T01:59:39.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:39 smithi184 ceph-mon[35593]: mds.? [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] up:boot 2024-09-12T01:59:39.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:39 smithi184 ceph-mon[35593]: fsmap 9 up:standby 2024-09-12T01:59:39.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:39 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-12T01:59:39.390 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:39 smithi016 ceph-mon[30947]: Deploying daemon mds.g on smithi016 2024-09-12T01:59:39.390 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:39 smithi016 ceph-mon[30947]: mds.? [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] up:boot 2024-09-12T01:59:39.391 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:39 smithi016 ceph-mon[30947]: fsmap 9 up:standby 2024-09-12T01:59:39.391 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:39 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-12T01:59:39.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:39 smithi033 ceph-mon[35332]: Deploying daemon mds.g on smithi016 2024-09-12T01:59:39.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:39 smithi033 ceph-mon[35332]: mds.? [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] up:boot 2024-09-12T01:59:39.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:39 smithi033 ceph-mon[35332]: fsmap 9 up:standby 2024-09-12T01:59:39.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:39 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-12T01:59:39.554 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:59:40.225 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:40 smithi016 ceph-mon[30947]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:59:40.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:40.224+0000 7f3a27059640 1 -- 172.21.15.16:0/472583036 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a20102ef0 msgr2=0x7f3a2010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:40.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:40.224+0000 7f3a27059640 1 --2- 172.21.15.16:0/472583036 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a20102ef0 0x7f3a2010da40 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f3a18009ca0 tx=0x7f3a1802a950 comp rx=0 tx=0).stop 2024-09-12T01:59:40.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:40.224+0000 7f3a27059640 1 -- 172.21.15.16:0/472583036 shutdown_connections 2024-09-12T01:59:40.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:40.224+0000 7f3a27059640 1 --2- 172.21.15.16:0/472583036 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a20102ef0 0x7f3a2010da40 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:40.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:40.224+0000 7f3a27059640 1 --2- 172.21.15.16:0/472583036 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3a20102570 0x7f3a201029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:40.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:40.224+0000 7f3a27059640 1 --2- 172.21.15.16:0/472583036 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a20100e20 0x7f3a20101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:40.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:40.224+0000 7f3a27059640 1 -- 172.21.15.16:0/472583036 >> 172.21.15.16:0/472583036 conn(0x7f3a200fc7d0 msgr2=0x7f3a200febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:59:40.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:40.224+0000 7f3a27059640 1 -- 172.21.15.16:0/472583036 shutdown_connections 2024-09-12T01:59:40.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:40.224+0000 7f3a27059640 1 -- 172.21.15.16:0/472583036 wait complete. 2024-09-12T01:59:40.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:40.225+0000 7f3a27059640 1 Processor -- start 2024-09-12T01:59:40.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:40.225+0000 7f3a27059640 1 -- start start 2024-09-12T01:59:40.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:40.225+0000 7f3a27059640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3a20100e20 0x7f3a20198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:40.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:40.225+0000 7f3a27059640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a20102570 0x7f3a20198bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:40.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:40.225+0000 7f3a27059640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a20102ef0 0x7f3a2019cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:40.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:40.225+0000 7f3a27059640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3a20111070 con 0x7f3a20102570 2024-09-12T01:59:40.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:40.225+0000 7f3a27059640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3a20110ef0 con 0x7f3a20102ef0 2024-09-12T01:59:40.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:40.225+0000 7f3a27059640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f3a201111f0 con 0x7f3a20100e20 2024-09-12T01:59:40.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:40.225+0000 7f3a1ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a20102570 0x7f3a20198bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:40.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:40.225+0000 7f3a1ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a20102570 0x7f3a20198bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48144/0 (socket says 172.21.15.16:48144) 2024-09-12T01:59:40.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:40.225+0000 7f3a1ffff640 1 -- 172.21.15.16:0/3585173147 learned_addr learned my addr 172.21.15.16:0/3585173147 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:59:40.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:40.225+0000 7f3a24df5640 1 --2- 172.21.15.16:0/3585173147 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3a20100e20 0x7f3a20198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:40.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:40.226+0000 7f3a255f6640 1 --2- 172.21.15.16:0/3585173147 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a20102ef0 0x7f3a2019cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:40.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:40.226+0000 7f3a1ffff640 1 -- 172.21.15.16:0/3585173147 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3a20100e20 msgr2=0x7f3a20198690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:40.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:40.226+0000 7f3a1ffff640 1 --2- 172.21.15.16:0/3585173147 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3a20100e20 0x7f3a20198690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:40.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:40.226+0000 7f3a1ffff640 1 -- 172.21.15.16:0/3585173147 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a20102ef0 msgr2=0x7f3a2019cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:40.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:40.226+0000 7f3a1ffff640 1 --2- 172.21.15.16:0/3585173147 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a20102ef0 0x7f3a2019cf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:40.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:40.226+0000 7f3a1ffff640 1 -- 172.21.15.16:0/3585173147 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a2019d640 con 0x7f3a20102570 2024-09-12T01:59:40.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:40.226+0000 7f3a24df5640 1 --2- 172.21.15.16:0/3585173147 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3a20100e20 0x7f3a20198690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:59:40.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:40.226+0000 7f3a1ffff640 1 --2- 172.21.15.16:0/3585173147 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a20102570 0x7f3a20198bd0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c00b4a0 tx=0x7f3a0c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:59:40.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:40.226+0000 7f3a1dffb640 1 -- 172.21.15.16:0/3585173147 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a0c014070 con 0x7f3a20102570 2024-09-12T01:59:40.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:40.226+0000 7f3a27059640 1 -- 172.21.15.16:0/3585173147 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a2019d930 con 0x7f3a20102570 2024-09-12T01:59:40.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:40.226+0000 7f3a1dffb640 1 -- 172.21.15.16:0/3585173147 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a0c010af0 con 0x7f3a20102570 2024-09-12T01:59:40.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:40.226+0000 7f3a1dffb640 1 -- 172.21.15.16:0/3585173147 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a0c017d10 con 0x7f3a20102570 2024-09-12T01:59:40.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:40.226+0000 7f3a27059640 1 -- 172.21.15.16:0/3585173147 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a201a5210 con 0x7f3a20102570 2024-09-12T01:59:40.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:40.227+0000 7f3a27059640 1 -- 172.21.15.16:0/3585173147 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a20101280 con 0x7f3a20102570 2024-09-12T01:59:40.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:40.228+0000 7f3a1dffb640 1 -- 172.21.15.16:0/3585173147 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f3a0c017070 con 0x7f3a20102570 2024-09-12T01:59:40.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:40.229+0000 7f3a1dffb640 1 --2- 172.21.15.16:0/3585173147 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f3a100a2b80 0x7f3a100a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:40.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:40.229+0000 7f3a24df5640 1 --2- 172.21.15.16:0/3585173147 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f3a100a2b80 0x7f3a100a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:40.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:40.229+0000 7f3a1dffb640 1 -- 172.21.15.16:0/3585173147 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7f3a0c0ce130 con 0x7f3a20102570 2024-09-12T01:59:40.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:40.230+0000 7f3a24df5640 1 --2- 172.21.15.16:0/3585173147 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f3a100a2b80 0x7f3a100a5040 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f3a14009930 tx=0x7f3a14008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:59:40.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:40.233+0000 7f3a1dffb640 1 -- 172.21.15.16:0/3585173147 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3a0c09b490 con 0x7f3a20102570 2024-09-12T01:59:40.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:40 smithi184 ceph-mon[35593]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:59:40.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:40 smithi033 ceph-mon[35332]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:59:40.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:40.461+0000 7f3a27059640 1 -- 172.21.15.16:0/3585173147 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3a2010a0a0 con 0x7f3a20102570 2024-09-12T01:59:40.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:40.462+0000 7f3a1dffb640 1 -- 172.21.15.16:0/3585173147 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8046 (secure 0 0 0) 0x7f3a0c09ad90 con 0x7f3a20102570 2024-09-12T01:59:40.462 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:59:40.462 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":10,"btime":"2024-09-12T01:59:37:999829+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3279123279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3279123279},{"type":"v1","addr":"172.21.15.16:6835","nonce":3279123279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4119452382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4119452382},{"type":"v1","addr":"172.21.15.16:6837","nonce":4119452382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/279674603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":279674603},{"type":"v1","addr":"172.21.15.33:6833","nonce":279674603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3555728028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3555728028},{"type":"v1","addr":"172.21.15.33:6835","nonce":3555728028}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6833/1563374106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1563374106},{"type":"v1","addr":"172.21.15.184:6833","nonce":1563374106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1892833868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1892833868},{"type":"v1","addr":"172.21.15.33:6837","nonce":1892833868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24460,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/1457679686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1457679686},{"type":"v1","addr":"172.21.15.184:6835","nonce":1457679686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1793393087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":1793393087},{"type":"v1","addr":"172.21.15.184:6837","nonce":1793393087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24481,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6839/4203347941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":4203347941},{"type":"v1","addr":"172.21.15.184:6839","nonce":4203347941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-09-12T01:59:40.463 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 10 2024-09-12T01:59:40.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:40.464+0000 7f3a27059640 1 -- 172.21.15.16:0/3585173147 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f3a100a2b80 msgr2=0x7f3a100a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:40.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:40.464+0000 7f3a27059640 1 --2- 172.21.15.16:0/3585173147 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f3a100a2b80 0x7f3a100a5040 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f3a14009930 tx=0x7f3a14008040 comp rx=0 tx=0).stop 2024-09-12T01:59:40.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:40.464+0000 7f3a27059640 1 -- 172.21.15.16:0/3585173147 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a20102570 msgr2=0x7f3a20198bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:40.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:40.464+0000 7f3a27059640 1 --2- 172.21.15.16:0/3585173147 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a20102570 0x7f3a20198bd0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c00b4a0 tx=0x7f3a0c00b960 comp rx=0 tx=0).stop 2024-09-12T01:59:40.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:40.464+0000 7f3a27059640 1 -- 172.21.15.16:0/3585173147 shutdown_connections 2024-09-12T01:59:40.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:40.464+0000 7f3a27059640 1 --2- 172.21.15.16:0/3585173147 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f3a100a2b80 0x7f3a100a5040 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:40.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:40.464+0000 7f3a27059640 1 --2- 172.21.15.16:0/3585173147 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a20102ef0 0x7f3a2019cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:40.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:40.464+0000 7f3a27059640 1 --2- 172.21.15.16:0/3585173147 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a20102570 0x7f3a20198bd0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:40.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:40.464+0000 7f3a27059640 1 --2- 172.21.15.16:0/3585173147 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3a20100e20 0x7f3a20198690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:40.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:40.464+0000 7f3a27059640 1 -- 172.21.15.16:0/3585173147 >> 172.21.15.16:0/3585173147 conn(0x7f3a200fc7d0 msgr2=0x7f3a2006a270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:59:40.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:40.464+0000 7f3a27059640 1 -- 172.21.15.16:0/3585173147 shutdown_connections 2024-09-12T01:59:40.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:40.464+0000 7f3a27059640 1 -- 172.21.15.16:0/3585173147 wait complete. 2024-09-12T01:59:41.255 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:41 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/3585173147' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:59:41.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:41 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/3585173147' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:59:41.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:41 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/3585173147' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:59:42.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:42 smithi016 ceph-mon[30947]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:59:42.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:42 smithi016 ceph-mon[30947]: mds.? [v2:172.21.15.16:6838/3263823165,v1:172.21.15.16:6839/3263823165] up:boot 2024-09-12T01:59:42.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:42 smithi016 ceph-mon[30947]: fsmap 10 up:standby 2024-09-12T01:59:42.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:42 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-12T01:59:42.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:42 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:42.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:42 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:42.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:42 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:42.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:42 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T01:59:42.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:42 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T01:59:42.185 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:42 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:59:42.185 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:42 smithi016 ceph-mon[30947]: Deploying daemon mds.a on smithi016 2024-09-12T01:59:42.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:42 smithi184 ceph-mon[35593]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:59:42.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:42 smithi184 ceph-mon[35593]: mds.? [v2:172.21.15.16:6838/3263823165,v1:172.21.15.16:6839/3263823165] up:boot 2024-09-12T01:59:42.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:42 smithi184 ceph-mon[35593]: fsmap 10 up:standby 2024-09-12T01:59:42.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:42 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-12T01:59:42.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:42 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:42.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:42 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:42.379 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:42 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:42.379 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:42 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T01:59:42.379 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:42 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T01:59:42.379 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:42 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:59:42.379 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:42 smithi184 ceph-mon[35593]: Deploying daemon mds.a on smithi016 2024-09-12T01:59:42.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:42 smithi033 ceph-mon[35332]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:59:42.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:42 smithi033 ceph-mon[35332]: mds.? [v2:172.21.15.16:6838/3263823165,v1:172.21.15.16:6839/3263823165] up:boot 2024-09-12T01:59:42.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:42 smithi033 ceph-mon[35332]: fsmap 10 up:standby 2024-09-12T01:59:42.420 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:42 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-12T01:59:42.420 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:42 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:42.420 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:42 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:42.420 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:42 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:42.420 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:42 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T01:59:42.420 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:42 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T01:59:42.420 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:42 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:59:42.421 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:42 smithi033 ceph-mon[35332]: Deploying daemon mds.a on smithi016 2024-09-12T01:59:43.390 INFO:journalctl@ceph.mds.a.smithi016.stdout:Sep 12 01:59:43 smithi016 systemd[1]: Starting Ceph mds.a for 879f3b56-70a9-11ef-bcea-c7b262605968... 2024-09-12T01:59:43.570 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T01:59:43.644 INFO:journalctl@ceph.mds.a.smithi016.stdout:Sep 12 01:59:43 smithi016 podman[72493]: 2024-09-12 01:59:43.293622518 +0000 UTC m=+0.016748095 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T01:59:43.644 INFO:journalctl@ceph.mds.a.smithi016.stdout:Sep 12 01:59:43 smithi016 podman[72493]: 2024-09-12 01:59:43.406607253 +0000 UTC m=+0.129732816 container create 14f2a6d3d79fb4df275519b336effe3a68ab8a4d720711b7d992f219e99cf9f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mds-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3) 2024-09-12T01:59:43.644 INFO:journalctl@ceph.mds.a.smithi016.stdout:Sep 12 01:59:43 smithi016 podman[72493]: 2024-09-12 01:59:43.617319447 +0000 UTC m=+0.340445010 container init 14f2a6d3d79fb4df275519b336effe3a68ab8a4d720711b7d992f219e99cf9f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mds-a, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-12T01:59:43.645 INFO:journalctl@ceph.mds.a.smithi016.stdout:Sep 12 01:59:43 smithi016 podman[72493]: 2024-09-12 01:59:43.620398314 +0000 UTC m=+0.343523887 container start 14f2a6d3d79fb4df275519b336effe3a68ab8a4d720711b7d992f219e99cf9f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mds-a, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.license=GPLv2) 2024-09-12T01:59:43.841 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:59:43.933 INFO:journalctl@ceph.mds.a.smithi016.stdout:Sep 12 01:59:43 smithi016 ceph-mds[72530]: -- 172.21.15.16:0/262017860 <== mon.0 v2:172.21.15.16:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5596c4005860 con 0x5596c3fe3000 2024-09-12T01:59:43.934 INFO:journalctl@ceph.mds.a.smithi016.stdout:Sep 12 01:59:43 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mds-a[72508]: starting mds.a at 2024-09-12T01:59:43.934 INFO:journalctl@ceph.mds.a.smithi016.stdout:Sep 12 01:59:43 smithi016 bash[72493]: 14f2a6d3d79fb4df275519b336effe3a68ab8a4d720711b7d992f219e99cf9f9 2024-09-12T01:59:43.934 INFO:journalctl@ceph.mds.a.smithi016.stdout:Sep 12 01:59:43 smithi016 systemd[1]: Started Ceph mds.a for 879f3b56-70a9-11ef-bcea-c7b262605968. 2024-09-12T01:59:44.262 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:44 smithi016 ceph-mon[30947]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:59:44.262 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:44 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:44.262 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:44 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:44.262 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:44 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:44.262 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:44 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T01:59:44.262 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:44 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T01:59:44.262 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:44 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:59:44.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:44.278+0000 7fd646df4640 1 -- 172.21.15.16:0/2991621725 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd640106f10 msgr2=0x7fd64010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:44.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:44.278+0000 7fd646df4640 1 --2- 172.21.15.16:0/2991621725 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd640106f10 0x7fd64010ea90 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fd638009ca0 tx=0x7fd63802a8c0 comp rx=0 tx=0).stop 2024-09-12T01:59:44.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:44.278+0000 7fd646df4640 1 -- 172.21.15.16:0/2991621725 shutdown_connections 2024-09-12T01:59:44.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:44.278+0000 7fd646df4640 1 --2- 172.21.15.16:0/2991621725 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd640106f10 0x7fd64010ea90 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:44.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:44.278+0000 7fd646df4640 1 --2- 172.21.15.16:0/2991621725 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd640102790 0x7fd6401067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:44.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:44.278+0000 7fd646df4640 1 --2- 172.21.15.16:0/2991621725 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd640101de0 0x7fd6401021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:44.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:44.278+0000 7fd646df4640 1 -- 172.21.15.16:0/2991621725 >> 172.21.15.16:0/2991621725 conn(0x7fd6400fd8c0 msgr2=0x7fd6400ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:59:44.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:44.278+0000 7fd646df4640 1 -- 172.21.15.16:0/2991621725 shutdown_connections 2024-09-12T01:59:44.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:44.278+0000 7fd646df4640 1 -- 172.21.15.16:0/2991621725 wait complete. 2024-09-12T01:59:44.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:44.279+0000 7fd646df4640 1 Processor -- start 2024-09-12T01:59:44.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:44.279+0000 7fd646df4640 1 -- start start 2024-09-12T01:59:44.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:44.279+0000 7fd646df4640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd640101de0 0x7fd6400744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:44.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:44.279+0000 7fd646df4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd640102790 0x7fd640074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:44.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:44.279+0000 7fd646df4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd640106f10 0x7fd640074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:44.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:44.279+0000 7fd646df4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd640111fc0 con 0x7fd640102790 2024-09-12T01:59:44.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:44.279+0000 7fd646df4640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd640111e40 con 0x7fd640106f10 2024-09-12T01:59:44.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:44.279+0000 7fd646df4640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd640112140 con 0x7fd640101de0 2024-09-12T01:59:44.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:44.279+0000 7fd63ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd640102790 0x7fd640074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:44.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:44.279+0000 7fd63ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd640102790 0x7fd640074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36990/0 (socket says 172.21.15.16:36990) 2024-09-12T01:59:44.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:44.279+0000 7fd63ffff640 1 -- 172.21.15.16:0/2298250449 learned_addr learned my addr 172.21.15.16:0/2298250449 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:59:44.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:44.279+0000 7fd645391640 1 --2- 172.21.15.16:0/2298250449 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd640106f10 0x7fd640074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:44.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:44.280+0000 7fd644b90640 1 --2- 172.21.15.16:0/2298250449 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd640101de0 0x7fd6400744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:44.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:44.280+0000 7fd63ffff640 1 -- 172.21.15.16:0/2298250449 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd640101de0 msgr2=0x7fd6400744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:44.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:44.280+0000 7fd63ffff640 1 --2- 172.21.15.16:0/2298250449 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd640101de0 0x7fd6400744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:44.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:44.280+0000 7fd63ffff640 1 -- 172.21.15.16:0/2298250449 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd640106f10 msgr2=0x7fd640074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:44.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:44.280+0000 7fd63ffff640 1 --2- 172.21.15.16:0/2298250449 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd640106f10 0x7fd640074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:44.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:44.280+0000 7fd63ffff640 1 -- 172.21.15.16:0/2298250449 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6401a6890 con 0x7fd640102790 2024-09-12T01:59:44.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:44.280+0000 7fd63ffff640 1 --2- 172.21.15.16:0/2298250449 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd640102790 0x7fd640074a30 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fd63000e9f0 tx=0x7fd63000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:59:44.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:44.280+0000 7fd63dffb640 1 -- 172.21.15.16:0/2298250449 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd63000cd80 con 0x7fd640102790 2024-09-12T01:59:44.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:44.280+0000 7fd646df4640 1 -- 172.21.15.16:0/2298250449 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6401a6b80 con 0x7fd640102790 2024-09-12T01:59:44.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:44.280+0000 7fd63dffb640 1 -- 172.21.15.16:0/2298250449 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd630010910 con 0x7fd640102790 2024-09-12T01:59:44.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:44.280+0000 7fd63dffb640 1 -- 172.21.15.16:0/2298250449 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6300188f0 con 0x7fd640102790 2024-09-12T01:59:44.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:44.280+0000 7fd646df4640 1 -- 172.21.15.16:0/2298250449 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6401a70c0 con 0x7fd640102790 2024-09-12T01:59:44.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:44.281+0000 7fd646df4640 1 -- 172.21.15.16:0/2298250449 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6401021c0 con 0x7fd640102790 2024-09-12T01:59:44.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:44.283+0000 7fd63dffb640 1 -- 172.21.15.16:0/2298250449 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fd630010ac0 con 0x7fd640102790 2024-09-12T01:59:44.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:44.284+0000 7fd63dffb640 1 --2- 172.21.15.16:0/2298250449 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fd62c0a2b80 0x7fd62c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:44.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:44.284+0000 7fd644b90640 1 --2- 172.21.15.16:0/2298250449 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fd62c0a2b80 0x7fd62c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:44.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:44.284+0000 7fd63dffb640 1 -- 172.21.15.16:0/2298250449 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7fd6300cf400 con 0x7fd640102790 2024-09-12T01:59:44.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:44.284+0000 7fd644b90640 1 --2- 172.21.15.16:0/2298250449 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fd62c0a2b80 0x7fd62c0a5040 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd628005fd0 tx=0x7fd628005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:59:44.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:44.287+0000 7fd63dffb640 1 -- 172.21.15.16:0/2298250449 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd63009c760 con 0x7fd640102790 2024-09-12T01:59:44.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:44 smithi184 ceph-mon[35593]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:59:44.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:44 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:44.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:44 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:44.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:44 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:44.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:44 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T01:59:44.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:44 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T01:59:44.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:44 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:59:44.380 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:44 smithi033 ceph-mon[35332]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:59:44.380 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:44 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:44.380 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:44 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:44.380 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:44 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:44.380 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:44 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T01:59:44.380 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:44 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T01:59:44.380 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:44 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:59:44.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:44.499+0000 7fd646df4640 1 -- 172.21.15.16:0/2298250449 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd640075a20 con 0x7fd640102790 2024-09-12T01:59:44.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:44.499+0000 7fd63dffb640 1 -- 172.21.15.16:0/2298250449 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9698 (secure 0 0 0) 0x7fd640075a20 con 0x7fd640102790 2024-09-12T01:59:44.501 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:59:44.501 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":12,"btime":"2024-09-12T01:59:44:023533+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3279123279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3279123279},{"type":"v1","addr":"172.21.15.16:6835","nonce":3279123279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4119452382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4119452382},{"type":"v1","addr":"172.21.15.16:6837","nonce":4119452382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/3263823165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3263823165},{"type":"v1","addr":"172.21.15.16:6839","nonce":3263823165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/1510290732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":1510290732},{"type":"v1","addr":"172.21.15.16:6841","nonce":1510290732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/279674603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":279674603},{"type":"v1","addr":"172.21.15.33:6833","nonce":279674603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3555728028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3555728028},{"type":"v1","addr":"172.21.15.33:6835","nonce":3555728028}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6833/1563374106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1563374106},{"type":"v1","addr":"172.21.15.184:6833","nonce":1563374106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1892833868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1892833868},{"type":"v1","addr":"172.21.15.33:6837","nonce":1892833868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24460,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/1457679686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1457679686},{"type":"v1","addr":"172.21.15.184:6835","nonce":1457679686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1793393087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":1793393087},{"type":"v1","addr":"172.21.15.184:6837","nonce":1793393087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24481,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6839/4203347941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":4203347941},{"type":"v1","addr":"172.21.15.184:6839","nonce":4203347941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-09-12T01:59:44.501 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 12 2024-09-12T01:59:44.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:44.502+0000 7fd646df4640 1 -- 172.21.15.16:0/2298250449 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fd62c0a2b80 msgr2=0x7fd62c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:44.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:44.502+0000 7fd646df4640 1 --2- 172.21.15.16:0/2298250449 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fd62c0a2b80 0x7fd62c0a5040 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd628005fd0 tx=0x7fd628005850 comp rx=0 tx=0).stop 2024-09-12T01:59:44.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:44.502+0000 7fd646df4640 1 -- 172.21.15.16:0/2298250449 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd640102790 msgr2=0x7fd640074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:44.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:44.502+0000 7fd646df4640 1 --2- 172.21.15.16:0/2298250449 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd640102790 0x7fd640074a30 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fd63000e9f0 tx=0x7fd63000eeb0 comp rx=0 tx=0).stop 2024-09-12T01:59:44.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:44.502+0000 7fd646df4640 1 -- 172.21.15.16:0/2298250449 shutdown_connections 2024-09-12T01:59:44.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:44.502+0000 7fd646df4640 1 --2- 172.21.15.16:0/2298250449 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fd62c0a2b80 0x7fd62c0a5040 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:44.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:44.502+0000 7fd646df4640 1 --2- 172.21.15.16:0/2298250449 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd640106f10 0x7fd640074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:44.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:44.502+0000 7fd646df4640 1 --2- 172.21.15.16:0/2298250449 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd640102790 0x7fd640074a30 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:44.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:44.502+0000 7fd646df4640 1 --2- 172.21.15.16:0/2298250449 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd640101de0 0x7fd6400744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:44.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:44.502+0000 7fd646df4640 1 -- 172.21.15.16:0/2298250449 >> 172.21.15.16:0/2298250449 conn(0x7fd6400fd8c0 msgr2=0x7fd6400fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:59:44.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:44.502+0000 7fd646df4640 1 -- 172.21.15.16:0/2298250449 shutdown_connections 2024-09-12T01:59:44.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:44.502+0000 7fd646df4640 1 -- 172.21.15.16:0/2298250449 wait complete. 2024-09-12T01:59:45.317 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:45 smithi033 ceph-mon[35332]: Deploying daemon mds.b on smithi033 2024-09-12T01:59:45.317 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:45 smithi033 ceph-mon[35332]: mds.? [v2:172.21.15.16:6840/1510290732,v1:172.21.15.16:6841/1510290732] up:boot 2024-09-12T01:59:45.317 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:45 smithi033 ceph-mon[35332]: fsmap 11 up:standby 2024-09-12T01:59:45.317 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:45 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-12T01:59:45.317 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:45 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/2298250449' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:59:45.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:45 smithi184 ceph-mon[35593]: Deploying daemon mds.b on smithi033 2024-09-12T01:59:45.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:45 smithi184 ceph-mon[35593]: mds.? [v2:172.21.15.16:6840/1510290732,v1:172.21.15.16:6841/1510290732] up:boot 2024-09-12T01:59:45.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:45 smithi184 ceph-mon[35593]: fsmap 11 up:standby 2024-09-12T01:59:45.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:45 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-12T01:59:45.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:45 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/2298250449' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:59:45.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:45 smithi016 ceph-mon[30947]: Deploying daemon mds.b on smithi033 2024-09-12T01:59:45.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:45 smithi016 ceph-mon[30947]: mds.? [v2:172.21.15.16:6840/1510290732,v1:172.21.15.16:6841/1510290732] up:boot 2024-09-12T01:59:45.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:45 smithi016 ceph-mon[30947]: fsmap 11 up:standby 2024-09-12T01:59:45.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:45 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-12T01:59:45.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:45 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/2298250449' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:59:45.920 INFO:journalctl@ceph.mds.b.smithi033.stdout:Sep 12 01:59:45 smithi033 systemd[1]: Starting Ceph mds.b for 879f3b56-70a9-11ef-bcea-c7b262605968... 2024-09-12T01:59:46.304 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:46 smithi033 ceph-mon[35332]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:59:46.305 INFO:journalctl@ceph.mds.b.smithi033.stdout:Sep 12 01:59:45 smithi033 podman[65136]: 2024-09-12 01:59:45.994609037 +0000 UTC m=+0.097979167 container create 4ddf0b353ec0366be99faa634728709f2c63fe2866dff0d54aa19d1bc53df8d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mds-b, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T01:59:46.305 INFO:journalctl@ceph.mds.b.smithi033.stdout:Sep 12 01:59:46 smithi033 podman[65136]: 2024-09-12 01:59:45.912725089 +0000 UTC m=+0.016095190 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T01:59:46.305 INFO:journalctl@ceph.mds.b.smithi033.stdout:Sep 12 01:59:46 smithi033 podman[65136]: 2024-09-12 01:59:46.213591942 +0000 UTC m=+0.316962031 container init 4ddf0b353ec0366be99faa634728709f2c63fe2866dff0d54aa19d1bc53df8d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mds-b, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-09-12T01:59:46.305 INFO:journalctl@ceph.mds.b.smithi033.stdout:Sep 12 01:59:46 smithi033 podman[65136]: 2024-09-12 01:59:46.216347332 +0000 UTC m=+0.319717423 container start 4ddf0b353ec0366be99faa634728709f2c63fe2866dff0d54aa19d1bc53df8d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mds-b, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-09-12T01:59:46.305 INFO:journalctl@ceph.mds.b.smithi033.stdout:Sep 12 01:59:46 smithi033 ceph-mds[65155]: -- 172.21.15.33:0/1354920575 <== mon.2 v2:172.21.15.33:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558aa1ff3860 con 0x558aa1fd1400 2024-09-12T01:59:46.305 INFO:journalctl@ceph.mds.b.smithi033.stdout:Sep 12 01:59:46 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mds-b[65151]: starting mds.b at 2024-09-12T01:59:46.305 INFO:journalctl@ceph.mds.b.smithi033.stdout:Sep 12 01:59:46 smithi033 bash[65136]: 4ddf0b353ec0366be99faa634728709f2c63fe2866dff0d54aa19d1bc53df8d2 2024-09-12T01:59:46.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:46 smithi184 ceph-mon[35593]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:59:46.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:46 smithi016 ceph-mon[30947]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:59:46.668 INFO:journalctl@ceph.mds.b.smithi033.stdout:Sep 12 01:59:46 smithi033 systemd[1]: Started Ceph mds.b for 879f3b56-70a9-11ef-bcea-c7b262605968. 2024-09-12T01:59:47.153 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T01:59:47.555 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:59:47.628 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:47 smithi184 ceph-mon[35593]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:59:47.628 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:47 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:47.628 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:47 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:47.628 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:47 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:47.628 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:47 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:47.628 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:47 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:47.628 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:47 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:59:47.628 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:47 smithi184 ceph-mon[35593]: mds.? [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] up:boot 2024-09-12T01:59:47.629 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:47 smithi184 ceph-mon[35593]: fsmap 12 up:standby 2024-09-12T01:59:47.629 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:47 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-12T01:59:47.668 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:47 smithi033 ceph-mon[35332]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:59:47.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:47 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:47.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:47 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:47.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:47 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:47.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:47 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:47.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:47 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:47.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:47 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:59:47.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:47 smithi033 ceph-mon[35332]: mds.? [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] up:boot 2024-09-12T01:59:47.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:47 smithi033 ceph-mon[35332]: fsmap 12 up:standby 2024-09-12T01:59:47.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:47 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-12T01:59:47.678 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:47 smithi016 ceph-mon[30947]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:59:47.678 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:47 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:47.678 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:47 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:47.678 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:47 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:47.679 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:47 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:47.679 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:47 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:47.679 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:47 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:59:47.679 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:47 smithi016 ceph-mon[30947]: mds.? [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] up:boot 2024-09-12T01:59:47.679 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:47 smithi016 ceph-mon[30947]: fsmap 12 up:standby 2024-09-12T01:59:47.679 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:47 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-12T01:59:48.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:48.014+0000 7f0d4b4e8640 1 -- 172.21.15.16:0/3060288994 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d441027a0 msgr2=0x7f0d441067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:48.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:48.014+0000 7f0d4b4e8640 1 --2- 172.21.15.16:0/3060288994 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d441027a0 0x7f0d441067f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f0d34009900 tx=0x7f0d3402a410 comp rx=0 tx=0).stop 2024-09-12T01:59:48.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:48.014+0000 7f0d4b4e8640 1 -- 172.21.15.16:0/3060288994 shutdown_connections 2024-09-12T01:59:48.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:48.014+0000 7f0d4b4e8640 1 --2- 172.21.15.16:0/3060288994 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0d44106f20 0x7f0d4410eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:48.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:48.014+0000 7f0d4b4e8640 1 --2- 172.21.15.16:0/3060288994 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d441027a0 0x7f0d441067f0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:48.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:48.014+0000 7f0d4b4e8640 1 --2- 172.21.15.16:0/3060288994 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0d44101df0 0x7f0d441021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:48.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:48.014+0000 7f0d4b4e8640 1 -- 172.21.15.16:0/3060288994 >> 172.21.15.16:0/3060288994 conn(0x7f0d440fd8f0 msgr2=0x7f0d440ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:59:48.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:48.014+0000 7f0d4b4e8640 1 -- 172.21.15.16:0/3060288994 shutdown_connections 2024-09-12T01:59:48.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:48.014+0000 7f0d4b4e8640 1 -- 172.21.15.16:0/3060288994 wait complete. 2024-09-12T01:59:48.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:48.015+0000 7f0d4b4e8640 1 Processor -- start 2024-09-12T01:59:48.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:48.015+0000 7f0d4b4e8640 1 -- start start 2024-09-12T01:59:48.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:48.015+0000 7f0d4b4e8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0d44101df0 0x7f0d441984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:48.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:48.015+0000 7f0d4b4e8640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0d441027a0 0x7f0d44198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:48.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:48.015+0000 7f0d4b4e8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d44106f20 0x7f0d4419cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:48.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:48.015+0000 7f0d4b4e8640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0d44111f80 con 0x7f0d44106f20 2024-09-12T01:59:48.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:48.015+0000 7f0d4b4e8640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0d44111e00 con 0x7f0d44101df0 2024-09-12T01:59:48.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:48.015+0000 7f0d4b4e8640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f0d44112100 con 0x7f0d441027a0 2024-09-12T01:59:48.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:48.015+0000 7f0d49a85640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d44106f20 0x7f0d4419cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:48.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:48.015+0000 7f0d49284640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0d44101df0 0x7f0d441984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:48.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:48.015+0000 7f0d49a85640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d44106f20 0x7f0d4419cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37010/0 (socket says 172.21.15.16:37010) 2024-09-12T01:59:48.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:48.015+0000 7f0d49a85640 1 -- 172.21.15.16:0/1691470177 learned_addr learned my addr 172.21.15.16:0/1691470177 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:59:48.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:48.016+0000 7f0d48a83640 1 --2- 172.21.15.16:0/1691470177 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0d441027a0 0x7f0d44198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:48.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:48.016+0000 7f0d49a85640 1 -- 172.21.15.16:0/1691470177 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0d441027a0 msgr2=0x7f0d44198a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:48.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:48.016+0000 7f0d49a85640 1 --2- 172.21.15.16:0/1691470177 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0d441027a0 0x7f0d44198a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:48.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:48.016+0000 7f0d49a85640 1 -- 172.21.15.16:0/1691470177 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0d44101df0 msgr2=0x7f0d441984e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:48.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:48.016+0000 7f0d49a85640 1 --2- 172.21.15.16:0/1691470177 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0d44101df0 0x7f0d441984e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:48.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:48.016+0000 7f0d49a85640 1 -- 172.21.15.16:0/1691470177 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d4419d490 con 0x7f0d44106f20 2024-09-12T01:59:48.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:48.016+0000 7f0d49284640 1 --2- 172.21.15.16:0/1691470177 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0d44101df0 0x7f0d441984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:59:48.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:48.016+0000 7f0d49a85640 1 --2- 172.21.15.16:0/1691470177 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d44106f20 0x7f0d4419cdb0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c00eb10 tx=0x7f0d3c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:59:48.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:48.016+0000 7f0d3a7fc640 1 -- 172.21.15.16:0/1691470177 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d3c004b20 con 0x7f0d44106f20 2024-09-12T01:59:48.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:48.016+0000 7f0d3a7fc640 1 -- 172.21.15.16:0/1691470177 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d3c004cc0 con 0x7f0d44106f20 2024-09-12T01:59:48.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:48.016+0000 7f0d4b4e8640 1 -- 172.21.15.16:0/1691470177 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d4419d780 con 0x7f0d44106f20 2024-09-12T01:59:48.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:48.016+0000 7f0d3a7fc640 1 -- 172.21.15.16:0/1691470177 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d3c010760 con 0x7f0d44106f20 2024-09-12T01:59:48.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:48.016+0000 7f0d4b4e8640 1 -- 172.21.15.16:0/1691470177 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d441a5060 con 0x7f0d44106f20 2024-09-12T01:59:48.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:48.017+0000 7f0d4b4e8640 1 -- 172.21.15.16:0/1691470177 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d441021d0 con 0x7f0d44106f20 2024-09-12T01:59:48.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:48.019+0000 7f0d3a7fc640 1 -- 172.21.15.16:0/1691470177 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f0d3c005000 con 0x7f0d44106f20 2024-09-12T01:59:48.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:48.019+0000 7f0d3a7fc640 1 --2- 172.21.15.16:0/1691470177 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f0d300a2b80 0x7f0d300a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:48.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:48.019+0000 7f0d3a7fc640 1 -- 172.21.15.16:0/1691470177 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7f0d3c0cf570 con 0x7f0d44106f20 2024-09-12T01:59:48.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:48.019+0000 7f0d49284640 1 --2- 172.21.15.16:0/1691470177 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f0d300a2b80 0x7f0d300a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:48.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:48.020+0000 7f0d49284640 1 --2- 172.21.15.16:0/1691470177 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f0d300a2b80 0x7f0d300a5040 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f0d4000bfd0 tx=0x7f0d4000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:59:48.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:48.022+0000 7f0d3a7fc640 1 -- 172.21.15.16:0/1691470177 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0d3c09c970 con 0x7f0d44106f20 2024-09-12T01:59:48.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:48.242+0000 7f0d4b4e8640 1 -- 172.21.15.16:0/1691470177 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0d4410a100 con 0x7f0d44106f20 2024-09-12T01:59:48.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:48.243+0000 7f0d3a7fc640 1 -- 172.21.15.16:0/1691470177 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10524 (secure 0 0 0) 0x7f0d3c09c270 con 0x7f0d44106f20 2024-09-12T01:59:48.244 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:59:48.245 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":13,"btime":"2024-09-12T01:59:47:037325+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3279123279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3279123279},{"type":"v1","addr":"172.21.15.16:6835","nonce":3279123279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4119452382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4119452382},{"type":"v1","addr":"172.21.15.16:6837","nonce":4119452382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/3263823165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3263823165},{"type":"v1","addr":"172.21.15.16:6839","nonce":3263823165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/1510290732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":1510290732},{"type":"v1","addr":"172.21.15.16:6841","nonce":1510290732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/279674603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":279674603},{"type":"v1","addr":"172.21.15.33:6833","nonce":279674603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3555728028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3555728028},{"type":"v1","addr":"172.21.15.33:6835","nonce":3555728028}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6833/1563374106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1563374106},{"type":"v1","addr":"172.21.15.184:6833","nonce":1563374106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1892833868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1892833868},{"type":"v1","addr":"172.21.15.33:6837","nonce":1892833868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24460,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/1457679686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1457679686},{"type":"v1","addr":"172.21.15.184:6835","nonce":1457679686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1793393087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":1793393087},{"type":"v1","addr":"172.21.15.184:6837","nonce":1793393087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24481,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6839/4203347941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":4203347941},{"type":"v1","addr":"172.21.15.184:6839","nonce":4203347941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24482,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/4136670204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":4136670204},{"type":"v1","addr":"172.21.15.33:6839","nonce":4136670204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-09-12T01:59:48.245 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 13 2024-09-12T01:59:48.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:48.245+0000 7f0d4b4e8640 1 -- 172.21.15.16:0/1691470177 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f0d300a2b80 msgr2=0x7f0d300a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:48.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:48.246+0000 7f0d4b4e8640 1 --2- 172.21.15.16:0/1691470177 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f0d300a2b80 0x7f0d300a5040 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f0d4000bfd0 tx=0x7f0d4000b390 comp rx=0 tx=0).stop 2024-09-12T01:59:48.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:48.246+0000 7f0d4b4e8640 1 -- 172.21.15.16:0/1691470177 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d44106f20 msgr2=0x7f0d4419cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:48.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:48.246+0000 7f0d4b4e8640 1 --2- 172.21.15.16:0/1691470177 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d44106f20 0x7f0d4419cdb0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c00eb10 tx=0x7f0d3c00c2d0 comp rx=0 tx=0).stop 2024-09-12T01:59:48.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:48.246+0000 7f0d4b4e8640 1 -- 172.21.15.16:0/1691470177 shutdown_connections 2024-09-12T01:59:48.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:48.246+0000 7f0d4b4e8640 1 --2- 172.21.15.16:0/1691470177 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f0d300a2b80 0x7f0d300a5040 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:48.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:48.246+0000 7f0d4b4e8640 1 --2- 172.21.15.16:0/1691470177 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d44106f20 0x7f0d4419cdb0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:48.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:48.246+0000 7f0d4b4e8640 1 --2- 172.21.15.16:0/1691470177 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0d441027a0 0x7f0d44198a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:48.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:48.246+0000 7f0d4b4e8640 1 --2- 172.21.15.16:0/1691470177 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0d44101df0 0x7f0d441984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:48.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:48.246+0000 7f0d4b4e8640 1 -- 172.21.15.16:0/1691470177 >> 172.21.15.16:0/1691470177 conn(0x7f0d440fd8f0 msgr2=0x7f0d440fec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:59:48.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:48.246+0000 7f0d4b4e8640 1 -- 172.21.15.16:0/1691470177 shutdown_connections 2024-09-12T01:59:48.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:48.246+0000 7f0d4b4e8640 1 -- 172.21.15.16:0/1691470177 wait complete. 2024-09-12T01:59:48.914 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-09-12T01:59:49.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:48 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:49.128 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:48 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:49.128 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:48 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:49.128 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:48 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:49.128 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:48 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/1691470177' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:59:49.128 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:48 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:49.128 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:48 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:49.128 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:48 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:49.128 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:48 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:59:49.128 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:48 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:59:49.129 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:48 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:49.129 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:48 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:59:49.168 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:48 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:49.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:48 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:49.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:48 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:49.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:48 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:49.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:48 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/1691470177' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:59:49.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:48 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:49.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:48 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:49.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:48 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:49.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:48 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:59:49.170 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:48 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:59:49.170 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:48 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:49.170 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:48 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:59:49.183 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:48 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:49.183 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:48 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:49.183 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:48 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:49.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:48 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:49.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:48 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/1691470177' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:59:49.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:48 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:49.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:48 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:49.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:48 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:49.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:48 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:59:49.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:48 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:59:49.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:48 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:49.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:48 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:59:49.678 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:59:49.858 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:49 smithi033 ceph-mon[35332]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:59:50.112 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:49 smithi016 ceph-mon[30947]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:59:50.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:49 smithi184 ceph-mon[35593]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:59:50.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:50.194+0000 7fd6ec3fd640 1 -- 172.21.15.16:0/1418544509 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6e4103660 msgr2=0x7fd6e410bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:50.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:50.194+0000 7fd6ec3fd640 1 --2- 172.21.15.16:0/1418544509 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6e4103660 0x7fd6e410bb50 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd6d4009900 tx=0x7fd6d402a480 comp rx=0 tx=0).stop 2024-09-12T01:59:50.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:50.194+0000 7fd6ec3fd640 1 -- 172.21.15.16:0/1418544509 shutdown_connections 2024-09-12T01:59:50.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:50.194+0000 7fd6ec3fd640 1 --2- 172.21.15.16:0/1418544509 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6e410c170 0x7fd6e410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:50.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:50.194+0000 7fd6ec3fd640 1 --2- 172.21.15.16:0/1418544509 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6e4103660 0x7fd6e410bb50 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:50.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:50.194+0000 7fd6ec3fd640 1 --2- 172.21.15.16:0/1418544509 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd6e4102d40 0x7fd6e4103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:50.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:50.194+0000 7fd6ec3fd640 1 -- 172.21.15.16:0/1418544509 >> 172.21.15.16:0/1418544509 conn(0x7fd6e40fc7a0 msgr2=0x7fd6e40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:59:50.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:50.194+0000 7fd6ec3fd640 1 -- 172.21.15.16:0/1418544509 shutdown_connections 2024-09-12T01:59:50.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:50.194+0000 7fd6ec3fd640 1 -- 172.21.15.16:0/1418544509 wait complete. 2024-09-12T01:59:50.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:50.195+0000 7fd6ec3fd640 1 Processor -- start 2024-09-12T01:59:50.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:50.195+0000 7fd6ec3fd640 1 -- start start 2024-09-12T01:59:50.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:50.195+0000 7fd6ec3fd640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd6e4102d40 0x7fd6e419a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:50.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:50.195+0000 7fd6ec3fd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6e4103660 0x7fd6e419ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:50.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:50.195+0000 7fd6ec3fd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6e410c170 0x7fd6e419ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:50.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:50.195+0000 7fd6ec3fd640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd6e4111f70 con 0x7fd6e410c170 2024-09-12T01:59:50.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:50.195+0000 7fd6ec3fd640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd6e4111df0 con 0x7fd6e4103660 2024-09-12T01:59:50.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:50.195+0000 7fd6ec3fd640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd6e41120f0 con 0x7fd6e4102d40 2024-09-12T01:59:50.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:50.195+0000 7fd6ea99a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6e410c170 0x7fd6e419ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:50.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:50.195+0000 7fd6ea99a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6e410c170 0x7fd6e419ef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37026/0 (socket says 172.21.15.16:37026) 2024-09-12T01:59:50.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:50.195+0000 7fd6ea99a640 1 -- 172.21.15.16:0/3752492955 learned_addr learned my addr 172.21.15.16:0/3752492955 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:59:50.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:50.195+0000 7fd6e9998640 1 --2- 172.21.15.16:0/3752492955 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6e4103660 0x7fd6e419ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:50.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:50.196+0000 7fd6ea199640 1 --2- 172.21.15.16:0/3752492955 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd6e4102d40 0x7fd6e419a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:50.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:50.196+0000 7fd6ea99a640 1 -- 172.21.15.16:0/3752492955 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd6e4102d40 msgr2=0x7fd6e419a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:50.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:50.196+0000 7fd6ea99a640 1 --2- 172.21.15.16:0/3752492955 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd6e4102d40 0x7fd6e419a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:50.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:50.196+0000 7fd6ea99a640 1 -- 172.21.15.16:0/3752492955 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6e4103660 msgr2=0x7fd6e419ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:50.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:50.196+0000 7fd6ea99a640 1 --2- 172.21.15.16:0/3752492955 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6e4103660 0x7fd6e419ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:50.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:50.196+0000 7fd6ea99a640 1 -- 172.21.15.16:0/3752492955 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6e419f660 con 0x7fd6e410c170 2024-09-12T01:59:50.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:50.196+0000 7fd6e9998640 1 --2- 172.21.15.16:0/3752492955 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6e4103660 0x7fd6e419ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:59:50.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:50.196+0000 7fd6ea99a640 1 --2- 172.21.15.16:0/3752492955 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6e410c170 0x7fd6e419ef80 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd6dc00efd0 tx=0x7fd6dc00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:59:50.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:50.196+0000 7fd6db7fe640 1 -- 172.21.15.16:0/3752492955 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6dc003b70 con 0x7fd6e410c170 2024-09-12T01:59:50.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:50.196+0000 7fd6db7fe640 1 -- 172.21.15.16:0/3752492955 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd6dc003d10 con 0x7fd6e410c170 2024-09-12T01:59:50.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:50.196+0000 7fd6ec3fd640 1 -- 172.21.15.16:0/3752492955 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6e419f950 con 0x7fd6e410c170 2024-09-12T01:59:50.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:50.196+0000 7fd6db7fe640 1 -- 172.21.15.16:0/3752492955 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6dc0109f0 con 0x7fd6e410c170 2024-09-12T01:59:50.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:50.196+0000 7fd6ec3fd640 1 -- 172.21.15.16:0/3752492955 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6e41a7230 con 0x7fd6e410c170 2024-09-12T01:59:50.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:50.197+0000 7fd6ec3fd640 1 -- 172.21.15.16:0/3752492955 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6e4103120 con 0x7fd6e410c170 2024-09-12T01:59:50.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:50.198+0000 7fd6db7fe640 1 -- 172.21.15.16:0/3752492955 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fd6dc004020 con 0x7fd6e410c170 2024-09-12T01:59:50.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:50.199+0000 7fd6db7fe640 1 --2- 172.21.15.16:0/3752492955 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fd6d00a2c00 0x7fd6d00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:50.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:50.199+0000 7fd6ea199640 1 --2- 172.21.15.16:0/3752492955 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fd6d00a2c00 0x7fd6d00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:50.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:50.199+0000 7fd6db7fe640 1 -- 172.21.15.16:0/3752492955 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7fd6dc0ceee0 con 0x7fd6e410c170 2024-09-12T01:59:50.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:50.200+0000 7fd6ea199640 1 --2- 172.21.15.16:0/3752492955 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fd6d00a2c00 0x7fd6d00a50c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fd6e0009bd0 tx=0x7fd6e0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:59:50.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:50.203+0000 7fd6db7fe640 1 -- 172.21.15.16:0/3752492955 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd6dc09c2e0 con 0x7fd6e410c170 2024-09-12T01:59:50.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:50.404+0000 7fd6ec3fd640 1 -- 172.21.15.16:0/3752492955 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fd6e41a7490 con 0x7fd6e410c170 2024-09-12T01:59:50.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:50.404+0000 7fd6db7fe640 1 -- 172.21.15.16:0/3752492955 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v75) ==== 84+0+65 (secure 0 0 0) 0x7fd6dc09bbe0 con 0x7fd6e410c170 2024-09-12T01:59:50.404 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:59:50.405 INFO:teuthology.orchestra.run.smithi016.stdout:[ 2024-09-12T01:59:50.405 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T01:59:50.405 INFO:teuthology.orchestra.run.smithi016.stdout: "poolnum": 1, 2024-09-12T01:59:50.405 INFO:teuthology.orchestra.run.smithi016.stdout: "poolname": ".mgr" 2024-09-12T01:59:50.405 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:59:50.405 INFO:teuthology.orchestra.run.smithi016.stdout:] 2024-09-12T01:59:50.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:50.406+0000 7fd6ec3fd640 1 -- 172.21.15.16:0/3752492955 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fd6d00a2c00 msgr2=0x7fd6d00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:50.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:50.406+0000 7fd6ec3fd640 1 --2- 172.21.15.16:0/3752492955 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fd6d00a2c00 0x7fd6d00a50c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fd6e0009bd0 tx=0x7fd6e0005f90 comp rx=0 tx=0).stop 2024-09-12T01:59:50.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:50.406+0000 7fd6ec3fd640 1 -- 172.21.15.16:0/3752492955 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6e410c170 msgr2=0x7fd6e419ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:50.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:50.406+0000 7fd6ec3fd640 1 --2- 172.21.15.16:0/3752492955 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6e410c170 0x7fd6e419ef80 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd6dc00efd0 tx=0x7fd6dc00c6a0 comp rx=0 tx=0).stop 2024-09-12T01:59:50.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:50.407+0000 7fd6ec3fd640 1 -- 172.21.15.16:0/3752492955 shutdown_connections 2024-09-12T01:59:50.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:50.407+0000 7fd6ec3fd640 1 --2- 172.21.15.16:0/3752492955 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fd6d00a2c00 0x7fd6d00a50c0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:50.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:50.407+0000 7fd6ec3fd640 1 --2- 172.21.15.16:0/3752492955 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6e410c170 0x7fd6e419ef80 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:50.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:50.407+0000 7fd6ec3fd640 1 --2- 172.21.15.16:0/3752492955 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6e4103660 0x7fd6e419ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:50.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:50.407+0000 7fd6ec3fd640 1 --2- 172.21.15.16:0/3752492955 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd6e4102d40 0x7fd6e419a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:50.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:50.407+0000 7fd6ec3fd640 1 -- 172.21.15.16:0/3752492955 >> 172.21.15.16:0/3752492955 conn(0x7fd6e40fc7a0 msgr2=0x7fd6e40fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:59:50.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:50.407+0000 7fd6ec3fd640 1 -- 172.21.15.16:0/3752492955 shutdown_connections 2024-09-12T01:59:50.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:50.407+0000 7fd6ec3fd640 1 -- 172.21.15.16:0/3752492955 wait complete. 2024-09-12T01:59:51.012 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-12T01:59:51.012 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-12T01:59:51.290 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:59:51.338 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:51 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:51.338 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:51 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:51.338 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:51 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:51.338 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:51 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:51.338 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:51 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/3752492955' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-12T01:59:51.411 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:51 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:51.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:51 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:51.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:51 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:51.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:51 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:51.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:51 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/3752492955' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-12T01:59:51.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:51 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:51.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:51 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:51.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:51 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:51.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:51 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:51.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:51 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/3752492955' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-12T01:59:51.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:51.780+0000 7f94873d9640 1 -- 172.21.15.16:0/1950263194 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9480102790 msgr2=0x7f94801067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:51.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:51.780+0000 7f94873d9640 1 --2- 172.21.15.16:0/1950263194 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9480102790 0x7f94801067e0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f9470009900 tx=0x7f947002a410 comp rx=0 tx=0).stop 2024-09-12T01:59:51.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:51.780+0000 7f94873d9640 1 -- 172.21.15.16:0/1950263194 shutdown_connections 2024-09-12T01:59:51.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:51.780+0000 7f94873d9640 1 --2- 172.21.15.16:0/1950263194 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9480106f10 0x7f948010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:51.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:51.780+0000 7f94873d9640 1 --2- 172.21.15.16:0/1950263194 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9480102790 0x7f94801067e0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:51.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:51.780+0000 7f94873d9640 1 --2- 172.21.15.16:0/1950263194 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9480101de0 0x7f94801021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:51.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:51.780+0000 7f94873d9640 1 -- 172.21.15.16:0/1950263194 >> 172.21.15.16:0/1950263194 conn(0x7f94800fd8c0 msgr2=0x7f94800ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:59:51.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:51.780+0000 7f94873d9640 1 -- 172.21.15.16:0/1950263194 shutdown_connections 2024-09-12T01:59:51.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:51.780+0000 7f94873d9640 1 -- 172.21.15.16:0/1950263194 wait complete. 2024-09-12T01:59:51.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:51.781+0000 7f94873d9640 1 Processor -- start 2024-09-12T01:59:51.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:51.781+0000 7f94873d9640 1 -- start start 2024-09-12T01:59:51.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:51.781+0000 7f94873d9640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9480101de0 0x7f94800744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:51.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:51.781+0000 7f94873d9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9480102790 0x7f9480074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:51.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:51.781+0000 7f94873d9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9480106f10 0x7f9480074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:51.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:51.781+0000 7f94873d9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9480111f90 con 0x7f9480106f10 2024-09-12T01:59:51.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:51.781+0000 7f94873d9640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9480111e10 con 0x7f9480102790 2024-09-12T01:59:51.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:51.781+0000 7f94873d9640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f9480112110 con 0x7f9480101de0 2024-09-12T01:59:51.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:51.781+0000 7f9485976640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9480106f10 0x7f9480074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:51.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:51.781+0000 7f9485976640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9480106f10 0x7f9480074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35246/0 (socket says 172.21.15.16:35246) 2024-09-12T01:59:51.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:51.781+0000 7f9485976640 1 -- 172.21.15.16:0/1050141155 learned_addr learned my addr 172.21.15.16:0/1050141155 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:59:51.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:51.781+0000 7f9484974640 1 --2- 172.21.15.16:0/1050141155 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9480102790 0x7f9480074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:51.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:51.781+0000 7f9485175640 1 --2- 172.21.15.16:0/1050141155 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9480101de0 0x7f94800744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:51.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:51.781+0000 7f9485976640 1 -- 172.21.15.16:0/1050141155 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9480101de0 msgr2=0x7f94800744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:51.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:51.781+0000 7f9485976640 1 --2- 172.21.15.16:0/1050141155 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9480101de0 0x7f94800744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:51.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:51.782+0000 7f9485976640 1 -- 172.21.15.16:0/1050141155 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9480102790 msgr2=0x7f9480074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:51.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:51.782+0000 7f9485976640 1 --2- 172.21.15.16:0/1050141155 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9480102790 0x7f9480074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:51.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:51.782+0000 7f9485976640 1 -- 172.21.15.16:0/1050141155 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94801a6950 con 0x7f9480106f10 2024-09-12T01:59:51.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:51.782+0000 7f9485175640 1 --2- 172.21.15.16:0/1050141155 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9480101de0 0x7f94800744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:59:51.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:51.782+0000 7f9485976640 1 --2- 172.21.15.16:0/1050141155 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9480106f10 0x7f9480074f70 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f947800eaa0 tx=0x7f947800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:59:51.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:51.782+0000 7f94767fc640 1 -- 172.21.15.16:0/1050141155 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f947800cdb0 con 0x7f9480106f10 2024-09-12T01:59:51.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:51.782+0000 7f94767fc640 1 -- 172.21.15.16:0/1050141155 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9478005020 con 0x7f9480106f10 2024-09-12T01:59:51.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:51.782+0000 7f94873d9640 1 -- 172.21.15.16:0/1050141155 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94801a6c40 con 0x7f9480106f10 2024-09-12T01:59:51.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:51.782+0000 7f94767fc640 1 -- 172.21.15.16:0/1050141155 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94780116b0 con 0x7f9480106f10 2024-09-12T01:59:51.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:51.782+0000 7f94873d9640 1 -- 172.21.15.16:0/1050141155 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94801a7180 con 0x7f9480106f10 2024-09-12T01:59:51.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:51.783+0000 7f94873d9640 1 -- 172.21.15.16:0/1050141155 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94801021c0 con 0x7f9480106f10 2024-09-12T01:59:51.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:51.784+0000 7f94767fc640 1 -- 172.21.15.16:0/1050141155 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f9478005560 con 0x7f9480106f10 2024-09-12T01:59:51.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:51.785+0000 7f94767fc640 1 --2- 172.21.15.16:0/1050141155 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f946c0a2c00 0x7f946c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:51.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:51.785+0000 7f9485175640 1 --2- 172.21.15.16:0/1050141155 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f946c0a2c00 0x7f946c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:51.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:51.785+0000 7f94767fc640 1 -- 172.21.15.16:0/1050141155 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7f94780cf260 con 0x7f9480106f10 2024-09-12T01:59:51.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:51.785+0000 7f9485175640 1 --2- 172.21.15.16:0/1050141155 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f946c0a2c00 0x7f946c0a50c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f947c006fd0 tx=0x7f947c008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:59:51.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:51.788+0000 7f94767fc640 1 -- 172.21.15.16:0/1050141155 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9478016030 con 0x7f9480106f10 2024-09-12T01:59:51.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:51.964+0000 7f94873d9640 1 -- 172.21.15.16:0/1050141155 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f948010a0f0 con 0x7f9480106f10 2024-09-12T01:59:52.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:52.453+0000 7f94767fc640 1 -- 172.21.15.16:0/1050141155 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v76) ==== 138+0+0 (secure 0 0 0) 0x7f947809c640 con 0x7f9480106f10 2024-09-12T01:59:52.453 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_metadata' created 2024-09-12T01:59:52.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:52.455+0000 7f94873d9640 1 -- 172.21.15.16:0/1050141155 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f946c0a2c00 msgr2=0x7f946c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:52.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:52.455+0000 7f94873d9640 1 --2- 172.21.15.16:0/1050141155 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f946c0a2c00 0x7f946c0a50c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f947c006fd0 tx=0x7f947c008040 comp rx=0 tx=0).stop 2024-09-12T01:59:52.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:52.455+0000 7f94873d9640 1 -- 172.21.15.16:0/1050141155 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9480106f10 msgr2=0x7f9480074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:52.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:52.455+0000 7f94873d9640 1 --2- 172.21.15.16:0/1050141155 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9480106f10 0x7f9480074f70 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f947800eaa0 tx=0x7f947800ef60 comp rx=0 tx=0).stop 2024-09-12T01:59:52.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:52.455+0000 7f94873d9640 1 -- 172.21.15.16:0/1050141155 shutdown_connections 2024-09-12T01:59:52.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:52.455+0000 7f94873d9640 1 --2- 172.21.15.16:0/1050141155 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f946c0a2c00 0x7f946c0a50c0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:52.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:52.455+0000 7f94873d9640 1 --2- 172.21.15.16:0/1050141155 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9480106f10 0x7f9480074f70 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:52.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:52.455+0000 7f94873d9640 1 --2- 172.21.15.16:0/1050141155 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9480102790 0x7f9480074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:52.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:52.455+0000 7f94873d9640 1 --2- 172.21.15.16:0/1050141155 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9480101de0 0x7f94800744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:52.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:52.455+0000 7f94873d9640 1 -- 172.21.15.16:0/1050141155 >> 172.21.15.16:0/1050141155 conn(0x7f94800fd8c0 msgr2=0x7f94800fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:59:52.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:52.455+0000 7f94873d9640 1 -- 172.21.15.16:0/1050141155 shutdown_connections 2024-09-12T01:59:52.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:52.455+0000 7f94873d9640 1 -- 172.21.15.16:0/1050141155 wait complete. 2024-09-12T01:59:52.668 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:52 smithi033 ceph-mon[35332]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:59:52.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:52 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:52.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:52 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:52.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:52 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:59:52.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:52 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:59:52.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:52 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:52.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:52 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/1050141155' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-12T01:59:52.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:52 smithi016 ceph-mon[30947]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:59:52.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:52 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:52.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:52 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:52.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:52 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:59:52.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:52 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:59:52.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:52 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:52.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:52 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/1050141155' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-12T01:59:52.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:52 smithi184 ceph-mon[35593]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:59:52.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:52 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:52.878 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:52 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:52.878 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:52 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:59:52.878 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:52 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:59:52.878 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:52 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T01:59:52.878 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:52 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/1050141155' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-12T01:59:53.086 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-12T01:59:53.342 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:59:53.751 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:53 smithi016 ceph-mon[30947]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:59:53.752 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:53 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/1050141155' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-12T01:59:53.752 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:53 smithi016 ceph-mon[30947]: osdmap e76: 12 total, 12 up, 12 in 2024-09-12T01:59:53.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:53.840+0000 7fe5c3068640 1 -- 172.21.15.16:0/1585364785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe5bc106a80 msgr2=0x7fe5bc106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:53.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:53.840+0000 7fe5c3068640 1 --2- 172.21.15.16:0/1585364785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe5bc106a80 0x7fe5bc106e60 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fe5b0009930 tx=0x7fe5b002a420 comp rx=0 tx=0).stop 2024-09-12T01:59:53.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:53.840+0000 7fe5c3068640 1 -- 172.21.15.16:0/1585364785 shutdown_connections 2024-09-12T01:59:53.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:53.840+0000 7fe5c3068640 1 --2- 172.21.15.16:0/1585364785 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5bc101420 0x7fe5bc10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:53.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:53.840+0000 7fe5c3068640 1 --2- 172.21.15.16:0/1585364785 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe5bc100a80 0x7fe5bc100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:53.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:53.840+0000 7fe5c3068640 1 --2- 172.21.15.16:0/1585364785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe5bc106a80 0x7fe5bc106e60 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:53.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:53.840+0000 7fe5c3068640 1 -- 172.21.15.16:0/1585364785 >> 172.21.15.16:0/1585364785 conn(0x7fe5bc0fc7a0 msgr2=0x7fe5bc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:59:53.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:53.840+0000 7fe5c3068640 1 -- 172.21.15.16:0/1585364785 shutdown_connections 2024-09-12T01:59:53.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:53.840+0000 7fe5c3068640 1 -- 172.21.15.16:0/1585364785 wait complete. 2024-09-12T01:59:53.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:53.841+0000 7fe5c3068640 1 Processor -- start 2024-09-12T01:59:53.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:53.841+0000 7fe5c3068640 1 -- start start 2024-09-12T01:59:53.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:53.841+0000 7fe5c3068640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe5bc100a80 0x7fe5bc199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:53.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:53.841+0000 7fe5c3068640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5bc101420 0x7fe5bc1997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:53.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:53.841+0000 7fe5c3068640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe5bc106a80 0x7fe5bc1a4310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:53.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:53.841+0000 7fe5c3068640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe5bc110e80 con 0x7fe5bc106a80 2024-09-12T01:59:53.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:53.841+0000 7fe5c3068640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe5bc110d00 con 0x7fe5bc101420 2024-09-12T01:59:53.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:53.841+0000 7fe5c3068640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe5bc111000 con 0x7fe5bc100a80 2024-09-12T01:59:53.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:53.841+0000 7fe5c1605640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe5bc106a80 0x7fe5bc1a4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:53.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:53.841+0000 7fe5c1605640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe5bc106a80 0x7fe5bc1a4310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35262/0 (socket says 172.21.15.16:35262) 2024-09-12T01:59:53.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:53.841+0000 7fe5c1605640 1 -- 172.21.15.16:0/3841815889 learned_addr learned my addr 172.21.15.16:0/3841815889 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:59:53.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:53.842+0000 7fe5bbfff640 1 --2- 172.21.15.16:0/3841815889 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5bc101420 0x7fe5bc1997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:53.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:53.842+0000 7fe5c1605640 1 -- 172.21.15.16:0/3841815889 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe5bc100a80 msgr2=0x7fe5bc199290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:53.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:53.842+0000 7fe5c0e04640 1 --2- 172.21.15.16:0/3841815889 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe5bc100a80 0x7fe5bc199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:53.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:53.842+0000 7fe5c1605640 1 --2- 172.21.15.16:0/3841815889 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe5bc100a80 0x7fe5bc199290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:53.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:53.842+0000 7fe5c1605640 1 -- 172.21.15.16:0/3841815889 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5bc101420 msgr2=0x7fe5bc1997d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:53.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:53.842+0000 7fe5c1605640 1 --2- 172.21.15.16:0/3841815889 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5bc101420 0x7fe5bc1997d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:53.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:53.842+0000 7fe5c1605640 1 -- 172.21.15.16:0/3841815889 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5bc1a4850 con 0x7fe5bc106a80 2024-09-12T01:59:53.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:53.842+0000 7fe5bbfff640 1 --2- 172.21.15.16:0/3841815889 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5bc101420 0x7fe5bc1997d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:59:53.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:53.842+0000 7fe5c0e04640 1 --2- 172.21.15.16:0/3841815889 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe5bc100a80 0x7fe5bc199290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:59:53.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:53.842+0000 7fe5c1605640 1 --2- 172.21.15.16:0/3841815889 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe5bc106a80 0x7fe5bc1a4310 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fe5b4009fd0 tx=0x7fe5b400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:59:53.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:53.842+0000 7fe5b9ffb640 1 -- 172.21.15.16:0/3841815889 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5b4017070 con 0x7fe5bc106a80 2024-09-12T01:59:53.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:53.842+0000 7fe5c3068640 1 -- 172.21.15.16:0/3841815889 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5bc1a4a20 con 0x7fe5bc106a80 2024-09-12T01:59:53.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:53.842+0000 7fe5b9ffb640 1 -- 172.21.15.16:0/3841815889 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5b4003e50 con 0x7fe5bc106a80 2024-09-12T01:59:53.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:53.842+0000 7fe5b9ffb640 1 -- 172.21.15.16:0/3841815889 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5b40106c0 con 0x7fe5bc106a80 2024-09-12T01:59:53.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:53.842+0000 7fe5c3068640 1 -- 172.21.15.16:0/3841815889 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5bc1a4f30 con 0x7fe5bc106a80 2024-09-12T01:59:53.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:53.843+0000 7fe5c3068640 1 -- 172.21.15.16:0/3841815889 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5bc100ee0 con 0x7fe5bc106a80 2024-09-12T01:59:53.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:53.844+0000 7fe5b9ffb640 1 -- 172.21.15.16:0/3841815889 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fe5b4003970 con 0x7fe5bc106a80 2024-09-12T01:59:53.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:53.845+0000 7fe5b9ffb640 1 --2- 172.21.15.16:0/3841815889 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fe5ac0a2b80 0x7fe5ac0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:53.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:53.845+0000 7fe5c0e04640 1 --2- 172.21.15.16:0/3841815889 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fe5ac0a2b80 0x7fe5ac0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:53.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:53.845+0000 7fe5b9ffb640 1 -- 172.21.15.16:0/3841815889 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7497+0+0 (secure 0 0 0) 0x7fe5b40ceaf0 con 0x7fe5bc106a80 2024-09-12T01:59:53.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:53.846+0000 7fe5c0e04640 1 --2- 172.21.15.16:0/3841815889 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fe5ac0a2b80 0x7fe5ac0a5040 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fe5b0000c00 tx=0x7fe5b0032c70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:59:53.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:53.849+0000 7fe5b9ffb640 1 -- 172.21.15.16:0/3841815889 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe5b409bd80 con 0x7fe5bc106a80 2024-09-12T01:59:53.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:53 smithi184 ceph-mon[35593]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:59:53.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:53 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/1050141155' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-12T01:59:53.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:53 smithi184 ceph-mon[35593]: osdmap e76: 12 total, 12 up, 12 in 2024-09-12T01:59:53.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:53 smithi033 ceph-mon[35332]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:59:53.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:53 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/1050141155' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-12T01:59:53.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:53 smithi033 ceph-mon[35332]: osdmap e76: 12 total, 12 up, 12 in 2024-09-12T01:59:54.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:54.027+0000 7fe5c3068640 1 -- 172.21.15.16:0/3841815889 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fe5bc108ff0 con 0x7fe5bc106a80 2024-09-12T01:59:54.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:54.471+0000 7fe5b9ffb640 1 -- 172.21.15.16:0/3841815889 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v78) ==== 170+0+0 (secure 0 0 0) 0x7fe5b409b680 con 0x7fe5bc106a80 2024-09-12T01:59:54.471 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data' created 2024-09-12T01:59:54.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:54.473+0000 7fe5c3068640 1 -- 172.21.15.16:0/3841815889 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fe5ac0a2b80 msgr2=0x7fe5ac0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:54.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:54.473+0000 7fe5c3068640 1 --2- 172.21.15.16:0/3841815889 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fe5ac0a2b80 0x7fe5ac0a5040 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fe5b0000c00 tx=0x7fe5b0032c70 comp rx=0 tx=0).stop 2024-09-12T01:59:54.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:54.473+0000 7fe5c3068640 1 -- 172.21.15.16:0/3841815889 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe5bc106a80 msgr2=0x7fe5bc1a4310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:54.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:54.473+0000 7fe5c3068640 1 --2- 172.21.15.16:0/3841815889 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe5bc106a80 0x7fe5bc1a4310 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fe5b4009fd0 tx=0x7fe5b400b260 comp rx=0 tx=0).stop 2024-09-12T01:59:54.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:54.473+0000 7fe5c3068640 1 -- 172.21.15.16:0/3841815889 shutdown_connections 2024-09-12T01:59:54.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:54.473+0000 7fe5c3068640 1 --2- 172.21.15.16:0/3841815889 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fe5ac0a2b80 0x7fe5ac0a5040 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:54.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:54.473+0000 7fe5c3068640 1 --2- 172.21.15.16:0/3841815889 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe5bc106a80 0x7fe5bc1a4310 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:54.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:54.473+0000 7fe5c3068640 1 --2- 172.21.15.16:0/3841815889 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5bc101420 0x7fe5bc1997d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:54.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:54.473+0000 7fe5c3068640 1 --2- 172.21.15.16:0/3841815889 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe5bc100a80 0x7fe5bc199290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:54.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:54.473+0000 7fe5c3068640 1 -- 172.21.15.16:0/3841815889 >> 172.21.15.16:0/3841815889 conn(0x7fe5bc0fc7a0 msgr2=0x7fe5bc0fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:59:54.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:54.473+0000 7fe5c3068640 1 -- 172.21.15.16:0/3841815889 shutdown_connections 2024-09-12T01:59:54.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:54.473+0000 7fe5c3068640 1 -- 172.21.15.16:0/3841815889 wait complete. 2024-09-12T01:59:54.806 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:54 smithi016 ceph-mon[30947]: osdmap e77: 12 total, 12 up, 12 in 2024-09-12T01:59:54.806 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:54 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/3841815889' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-12T01:59:54.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:54 smithi184 ceph-mon[35593]: osdmap e77: 12 total, 12 up, 12 in 2024-09-12T01:59:54.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:54 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/3841815889' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-12T01:59:54.918 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:54 smithi033 ceph-mon[35332]: osdmap e77: 12 total, 12 up, 12 in 2024-09-12T01:59:54.918 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:54 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/3841815889' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-12T01:59:55.151 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-12T01:59:55.408 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:59:55.809 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:55 smithi016 ceph-mon[30947]: pgmap v270: 33 pgs: 6 creating+peering, 23 unknown, 4 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:59:55.809 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:55 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/3841815889' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-12T01:59:55.809 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:55 smithi016 ceph-mon[30947]: osdmap e78: 12 total, 12 up, 12 in 2024-09-12T01:59:55.810 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:55 smithi016 ceph-mon[30947]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-12T01:59:55.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:55 smithi184 ceph-mon[35593]: pgmap v270: 33 pgs: 6 creating+peering, 23 unknown, 4 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:59:55.878 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:55 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/3841815889' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-12T01:59:55.878 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:55 smithi184 ceph-mon[35593]: osdmap e78: 12 total, 12 up, 12 in 2024-09-12T01:59:55.878 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:55 smithi184 ceph-mon[35593]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-12T01:59:55.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:55.894+0000 7f1f347d8640 1 -- 172.21.15.16:0/3683367111 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f2c103660 msgr2=0x7f1f2c10bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:55.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:55.894+0000 7f1f347d8640 1 --2- 172.21.15.16:0/3683367111 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f2c103660 0x7f1f2c10bb50 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f1f1c009900 tx=0x7f1f1c02a480 comp rx=0 tx=0).stop 2024-09-12T01:59:55.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:55.894+0000 7f1f347d8640 1 -- 172.21.15.16:0/3683367111 shutdown_connections 2024-09-12T01:59:55.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:55.894+0000 7f1f347d8640 1 --2- 172.21.15.16:0/3683367111 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f2c10c170 0x7f1f2c10e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:55.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:55.894+0000 7f1f347d8640 1 --2- 172.21.15.16:0/3683367111 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f2c103660 0x7f1f2c10bb50 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:55.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:55.894+0000 7f1f347d8640 1 --2- 172.21.15.16:0/3683367111 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1f2c102d40 0x7f1f2c103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:55.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:55.894+0000 7f1f347d8640 1 -- 172.21.15.16:0/3683367111 >> 172.21.15.16:0/3683367111 conn(0x7f1f2c0fc7a0 msgr2=0x7f1f2c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:59:55.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:55.894+0000 7f1f347d8640 1 -- 172.21.15.16:0/3683367111 shutdown_connections 2024-09-12T01:59:55.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:55.894+0000 7f1f347d8640 1 -- 172.21.15.16:0/3683367111 wait complete. 2024-09-12T01:59:55.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:55.895+0000 7f1f347d8640 1 Processor -- start 2024-09-12T01:59:55.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:55.895+0000 7f1f347d8640 1 -- start start 2024-09-12T01:59:55.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:55.895+0000 7f1f347d8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f2c102d40 0x7f1f2c1009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:55.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:55.895+0000 7f1f347d8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f2c103660 0x7f1f2c100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:55.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:55.895+0000 7f1f347d8640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1f2c10c170 0x7f1f2c101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:55.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:55.895+0000 7f1f347d8640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1f2c111f70 con 0x7f1f2c102d40 2024-09-12T01:59:55.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:55.895+0000 7f1f347d8640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1f2c111df0 con 0x7f1f2c103660 2024-09-12T01:59:55.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:55.895+0000 7f1f347d8640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1f2c1120f0 con 0x7f1f2c10c170 2024-09-12T01:59:55.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:55.895+0000 7f1f32574640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f2c102d40 0x7f1f2c1009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:55.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:55.895+0000 7f1f32574640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f2c102d40 0x7f1f2c1009a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35272/0 (socket says 172.21.15.16:35272) 2024-09-12T01:59:55.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:55.895+0000 7f1f32574640 1 -- 172.21.15.16:0/519418844 learned_addr learned my addr 172.21.15.16:0/519418844 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:59:55.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:55.896+0000 7f1f31d73640 1 --2- 172.21.15.16:0/519418844 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f2c103660 0x7f1f2c100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:55.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:55.896+0000 7f1f32574640 1 -- 172.21.15.16:0/519418844 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1f2c10c170 msgr2=0x7f1f2c101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:55.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:55.896+0000 7f1f32d75640 1 --2- 172.21.15.16:0/519418844 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1f2c10c170 0x7f1f2c101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:55.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:55.896+0000 7f1f32574640 1 --2- 172.21.15.16:0/519418844 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1f2c10c170 0x7f1f2c101420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:55.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:55.896+0000 7f1f32574640 1 -- 172.21.15.16:0/519418844 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f2c103660 msgr2=0x7f1f2c100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:55.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:55.896+0000 7f1f32574640 1 --2- 172.21.15.16:0/519418844 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f2c103660 0x7f1f2c100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:55.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:55.896+0000 7f1f32574640 1 -- 172.21.15.16:0/519418844 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f2c1a6970 con 0x7f1f2c102d40 2024-09-12T01:59:55.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:55.896+0000 7f1f32d75640 1 --2- 172.21.15.16:0/519418844 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1f2c10c170 0x7f1f2c101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:59:55.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:55.896+0000 7f1f32574640 1 --2- 172.21.15.16:0/519418844 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f2c102d40 0x7f1f2c1009a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f1f2800d770 tx=0x7f1f2800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:59:55.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:55.896+0000 7f1f237fe640 1 -- 172.21.15.16:0/519418844 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f28017070 con 0x7f1f2c102d40 2024-09-12T01:59:55.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:55.896+0000 7f1f237fe640 1 -- 172.21.15.16:0/519418844 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f28005500 con 0x7f1f2c102d40 2024-09-12T01:59:55.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:55.896+0000 7f1f347d8640 1 -- 172.21.15.16:0/519418844 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f2c1a6c60 con 0x7f1f2c102d40 2024-09-12T01:59:55.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:55.896+0000 7f1f237fe640 1 -- 172.21.15.16:0/519418844 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f28004e00 con 0x7f1f2c102d40 2024-09-12T01:59:55.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:55.896+0000 7f1f347d8640 1 -- 172.21.15.16:0/519418844 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f2c1a7170 con 0x7f1f2c102d40 2024-09-12T01:59:55.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:55.897+0000 7f1f347d8640 1 -- 172.21.15.16:0/519418844 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f2c103120 con 0x7f1f2c102d40 2024-09-12T01:59:55.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:55.898+0000 7f1f237fe640 1 -- 172.21.15.16:0/519418844 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f1f28004fa0 con 0x7f1f2c102d40 2024-09-12T01:59:55.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:55.899+0000 7f1f237fe640 1 --2- 172.21.15.16:0/519418844 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f1f180a2c00 0x7f1f180a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:55.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:55.900+0000 7f1f31d73640 1 --2- 172.21.15.16:0/519418844 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f1f180a2c00 0x7f1f180a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:55.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:55.900+0000 7f1f237fe640 1 -- 172.21.15.16:0/519418844 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7879+0+0 (secure 0 0 0) 0x7f1f280ced30 con 0x7f1f2c102d40 2024-09-12T01:59:55.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:55.900+0000 7f1f31d73640 1 --2- 172.21.15.16:0/519418844 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f1f180a2c00 0x7f1f180a50c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f1f1c009230 tx=0x7f1f1c0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:59:55.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:55.903+0000 7f1f237fe640 1 -- 172.21.15.16:0/519418844 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1f2809be20 con 0x7f1f2c102d40 2024-09-12T01:59:55.918 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:55 smithi033 ceph-mon[35332]: pgmap v270: 33 pgs: 6 creating+peering, 23 unknown, 4 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:59:55.918 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:55 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/3841815889' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-12T01:59:55.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:55 smithi033 ceph-mon[35332]: osdmap e78: 12 total, 12 up, 12 in 2024-09-12T01:59:55.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:55 smithi033 ceph-mon[35332]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-12T01:59:56.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:56.116+0000 7f1f347d8640 1 -- 172.21.15.16:0/519418844 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f1f2c101ed0 con 0x7f1f2c102d40 2024-09-12T01:59:56.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:56.490+0000 7f1f237fe640 1 -- 172.21.15.16:0/519418844 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f1f2809b720 con 0x7f1f2c102d40 2024-09-12T01:59:56.491 INFO:teuthology.orchestra.run.smithi016.stderr:new fs with metadata pool 2 and data pool 3 2024-09-12T01:59:56.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:56.493+0000 7f1f347d8640 1 -- 172.21.15.16:0/519418844 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f1f180a2c00 msgr2=0x7f1f180a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:56.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:56.493+0000 7f1f347d8640 1 --2- 172.21.15.16:0/519418844 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f1f180a2c00 0x7f1f180a50c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f1f1c009230 tx=0x7f1f1c0058d0 comp rx=0 tx=0).stop 2024-09-12T01:59:56.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:56.493+0000 7f1f347d8640 1 -- 172.21.15.16:0/519418844 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f2c102d40 msgr2=0x7f1f2c1009a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:56.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:56.493+0000 7f1f347d8640 1 --2- 172.21.15.16:0/519418844 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f2c102d40 0x7f1f2c1009a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f1f2800d770 tx=0x7f1f2800dc30 comp rx=0 tx=0).stop 2024-09-12T01:59:56.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:56.493+0000 7f1f347d8640 1 -- 172.21.15.16:0/519418844 shutdown_connections 2024-09-12T01:59:56.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:56.493+0000 7f1f347d8640 1 --2- 172.21.15.16:0/519418844 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f1f180a2c00 0x7f1f180a50c0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:56.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:56.493+0000 7f1f347d8640 1 --2- 172.21.15.16:0/519418844 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1f2c10c170 0x7f1f2c101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:56.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:56.493+0000 7f1f347d8640 1 --2- 172.21.15.16:0/519418844 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f2c103660 0x7f1f2c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:56.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:56.493+0000 7f1f347d8640 1 --2- 172.21.15.16:0/519418844 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f2c102d40 0x7f1f2c1009a0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:56.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:56.493+0000 7f1f347d8640 1 -- 172.21.15.16:0/519418844 >> 172.21.15.16:0/519418844 conn(0x7f1f2c0fc7a0 msgr2=0x7f1f2c0fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:59:56.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:56.493+0000 7f1f347d8640 1 -- 172.21.15.16:0/519418844 shutdown_connections 2024-09-12T01:59:56.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:56.493+0000 7f1f347d8640 1 -- 172.21.15.16:0/519418844 wait complete. 2024-09-12T01:59:56.796 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:56 smithi016 ceph-mon[30947]: osdmap e79: 12 total, 12 up, 12 in 2024-09-12T01:59:56.797 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:56 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/519418844' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-12T01:59:56.797 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:56 smithi016 ceph-mon[30947]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-12T01:59:56.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:56 smithi184 ceph-mon[35593]: osdmap e79: 12 total, 12 up, 12 in 2024-09-12T01:59:56.878 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:56 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/519418844' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-12T01:59:56.878 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:56 smithi184 ceph-mon[35593]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-12T01:59:56.918 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:56 smithi033 ceph-mon[35332]: osdmap e79: 12 total, 12 up, 12 in 2024-09-12T01:59:56.918 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:56 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/519418844' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-12T01:59:56.918 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:56 smithi033 ceph-mon[35332]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-12T01:59:57.166 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T01:59:57.429 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:59:57.817 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:57 smithi016 ceph-mon[30947]: pgmap v273: 97 pgs: 38 creating+peering, 15 unknown, 44 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:59:57.817 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:57 smithi016 ceph-mon[30947]: osdmap e80: 12 total, 12 up, 12 in 2024-09-12T01:59:57.817 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:57 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/519418844' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-12T01:59:57.817 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:57 smithi016 ceph-mon[30947]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-12T01:59:57.817 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:57 smithi016 ceph-mon[30947]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-12T01:59:57.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:57 smithi184 ceph-mon[35593]: pgmap v273: 97 pgs: 38 creating+peering, 15 unknown, 44 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:59:57.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:57 smithi184 ceph-mon[35593]: osdmap e80: 12 total, 12 up, 12 in 2024-09-12T01:59:57.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:57 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/519418844' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-12T01:59:57.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:57 smithi184 ceph-mon[35593]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-12T01:59:57.878 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:57 smithi184 ceph-mon[35593]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-12T01:59:57.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:57.897+0000 7fcd75cec640 1 -- 172.21.15.16:0/1013071543 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd70100a80 msgr2=0x7fcd70100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:57.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:57.897+0000 7fcd75cec640 1 --2- 172.21.15.16:0/1013071543 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd70100a80 0x7fcd70100ee0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fcd64009930 tx=0x7fcd6402a420 comp rx=0 tx=0).stop 2024-09-12T01:59:57.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:57.897+0000 7fcd75cec640 1 -- 172.21.15.16:0/1013071543 shutdown_connections 2024-09-12T01:59:57.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:57.897+0000 7fcd75cec640 1 --2- 172.21.15.16:0/1013071543 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd70101420 0x7fcd7010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:57.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:57.897+0000 7fcd75cec640 1 --2- 172.21.15.16:0/1013071543 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd70100a80 0x7fcd70100ee0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:57.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:57.897+0000 7fcd75cec640 1 --2- 172.21.15.16:0/1013071543 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcd70106a80 0x7fcd70106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:57.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:57.897+0000 7fcd75cec640 1 -- 172.21.15.16:0/1013071543 >> 172.21.15.16:0/1013071543 conn(0x7fcd700fc7a0 msgr2=0x7fcd700febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:59:57.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:57.897+0000 7fcd75cec640 1 -- 172.21.15.16:0/1013071543 shutdown_connections 2024-09-12T01:59:57.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:57.897+0000 7fcd75cec640 1 -- 172.21.15.16:0/1013071543 wait complete. 2024-09-12T01:59:57.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:57.898+0000 7fcd75cec640 1 Processor -- start 2024-09-12T01:59:57.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:57.898+0000 7fcd75cec640 1 -- start start 2024-09-12T01:59:57.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:57.898+0000 7fcd75cec640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd70100a80 0x7fcd701973e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:57.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:57.898+0000 7fcd75cec640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcd70101420 0x7fcd70197920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:57.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:57.898+0000 7fcd75cec640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd70106a80 0x7fcd7019bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:57.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:57.899+0000 7fcd75cec640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fcd70110ec0 con 0x7fcd70106a80 2024-09-12T01:59:57.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:57.899+0000 7fcd75cec640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fcd70110d40 con 0x7fcd70100a80 2024-09-12T01:59:57.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:57.899+0000 7fcd75cec640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fcd70111040 con 0x7fcd70101420 2024-09-12T01:59:57.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:57.899+0000 7fcd6ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd70106a80 0x7fcd7019bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:57.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:57.899+0000 7fcd6ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd70106a80 0x7fcd7019bcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35278/0 (socket says 172.21.15.16:35278) 2024-09-12T01:59:57.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:57.899+0000 7fcd6ffff640 1 -- 172.21.15.16:0/260924695 learned_addr learned my addr 172.21.15.16:0/260924695 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:59:57.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:57.899+0000 7fcd6effd640 1 --2- 172.21.15.16:0/260924695 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcd70101420 0x7fcd70197920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:57.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:57.899+0000 7fcd6f7fe640 1 --2- 172.21.15.16:0/260924695 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd70100a80 0x7fcd701973e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:57.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:57.899+0000 7fcd6ffff640 1 -- 172.21.15.16:0/260924695 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcd70101420 msgr2=0x7fcd70197920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:57.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:57.899+0000 7fcd6ffff640 1 --2- 172.21.15.16:0/260924695 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcd70101420 0x7fcd70197920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:57.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:57.899+0000 7fcd6ffff640 1 -- 172.21.15.16:0/260924695 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd70100a80 msgr2=0x7fcd701973e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:57.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:57.899+0000 7fcd6ffff640 1 --2- 172.21.15.16:0/260924695 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd70100a80 0x7fcd701973e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:57.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:57.899+0000 7fcd6ffff640 1 -- 172.21.15.16:0/260924695 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd7019c390 con 0x7fcd70106a80 2024-09-12T01:59:57.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:57.899+0000 7fcd6ffff640 1 --2- 172.21.15.16:0/260924695 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd70106a80 0x7fcd7019bcb0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fcd6800b970 tx=0x7fcd6800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:59:57.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:57.899+0000 7fcd6cff9640 1 -- 172.21.15.16:0/260924695 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd68019070 con 0x7fcd70106a80 2024-09-12T01:59:57.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:57.899+0000 7fcd6cff9640 1 -- 172.21.15.16:0/260924695 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd6800cb40 con 0x7fcd70106a80 2024-09-12T01:59:57.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:57.899+0000 7fcd75cec640 1 -- 172.21.15.16:0/260924695 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd7019c680 con 0x7fcd70106a80 2024-09-12T01:59:57.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:57.899+0000 7fcd6cff9640 1 -- 172.21.15.16:0/260924695 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd68007650 con 0x7fcd70106a80 2024-09-12T01:59:57.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:57.899+0000 7fcd75cec640 1 -- 172.21.15.16:0/260924695 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd7006a790 con 0x7fcd70106a80 2024-09-12T01:59:57.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:57.901+0000 7fcd75cec640 1 -- 172.21.15.16:0/260924695 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd701021c0 con 0x7fcd70106a80 2024-09-12T01:59:57.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:57.903+0000 7fcd6cff9640 1 -- 172.21.15.16:0/260924695 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fcd6801c070 con 0x7fcd70106a80 2024-09-12T01:59:57.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:57.903+0000 7fcd6cff9640 1 --2- 172.21.15.16:0/260924695 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fcd580a2c00 0x7fcd580a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:57.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:57.904+0000 7fcd6f7fe640 1 --2- 172.21.15.16:0/260924695 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fcd580a2c00 0x7fcd580a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:57.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:57.904+0000 7fcd6cff9640 1 -- 172.21.15.16:0/260924695 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7fcd680ce980 con 0x7fcd70106a80 2024-09-12T01:59:57.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:57.904+0000 7fcd6f7fe640 1 --2- 172.21.15.16:0/260924695 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fcd580a2c00 0x7fcd580a50c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fcd60005fd0 tx=0x7fcd6000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:59:57.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:57.907+0000 7fcd6cff9640 1 -- 172.21.15.16:0/260924695 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcd6809b340 con 0x7fcd70106a80 2024-09-12T01:59:57.918 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:57 smithi033 ceph-mon[35332]: pgmap v273: 97 pgs: 38 creating+peering, 15 unknown, 44 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:59:57.918 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:57 smithi033 ceph-mon[35332]: osdmap e80: 12 total, 12 up, 12 in 2024-09-12T01:59:57.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:57 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/519418844' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-12T01:59:57.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:57 smithi033 ceph-mon[35332]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-12T01:59:57.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:57 smithi033 ceph-mon[35332]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-12T01:59:58.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:58.084+0000 7fcd75cec640 1 -- 172.21.15.16:0/260924695 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcd70100ee0 con 0x7fcd70106a80 2024-09-12T01:59:58.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:58.085+0000 7fcd6cff9640 1 -- 172.21.15.16:0/260924695 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+22245 (secure 0 0 0) 0x7fcd70100ee0 con 0x7fcd70106a80 2024-09-12T01:59:58.086 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:59:58.086 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":80,"fsid":"879f3b56-70a9-11ef-bcea-c7b262605968","created":"2024-09-12T01:52:12.841045+0000","modified":"2024-09-12T01:59:56.117422+0000","last_up_change":"2024-09-12T01:59:08.975443+0000","last_in_change":"2024-09-12T01:58:44.530220+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T01:55:26.727564+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T01:59:52.402097+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T01:59:54.463627+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3ea41eb6-e384-4ddc-810f-0f103c127849","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4000187867},{"type":"v1","addr":"172.21.15.16:6803","nonce":4000187867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4000187867},{"type":"v1","addr":"172.21.15.16:6805","nonce":4000187867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":4000187867},{"type":"v1","addr":"172.21.15.16:6809","nonce":4000187867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4000187867},{"type":"v1","addr":"172.21.15.16:6807","nonce":4000187867}]},"public_addr":"172.21.15.16:6803/4000187867","cluster_addr":"172.21.15.16:6805/4000187867","heartbeat_back_addr":"172.21.15.16:6809/4000187867","heartbeat_front_addr":"172.21.15.16:6807/4000187867","state":["exists","up"]},{"osd":1,"uuid":"07bf63e7-70fa-41b2-af53-12a5fec3f33c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":3862661148},{"type":"v1","addr":"172.21.15.33:6801","nonce":3862661148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":3862661148},{"type":"v1","addr":"172.21.15.33:6803","nonce":3862661148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":3862661148},{"type":"v1","addr":"172.21.15.33:6807","nonce":3862661148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":3862661148},{"type":"v1","addr":"172.21.15.33:6805","nonce":3862661148}]},"public_addr":"172.21.15.33:6801/3862661148","cluster_addr":"172.21.15.33:6803/3862661148","heartbeat_back_addr":"172.21.15.33:6807/3862661148","heartbeat_front_addr":"172.21.15.33:6805/3862661148","state":["exists","up"]},{"osd":2,"uuid":"22e40cac-3bc0-4cce-b47b-77499b767f7e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":3958753963},{"type":"v1","addr":"172.21.15.184:6801","nonce":3958753963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":3958753963},{"type":"v1","addr":"172.21.15.184:6803","nonce":3958753963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":3958753963},{"type":"v1","addr":"172.21.15.184:6807","nonce":3958753963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":3958753963},{"type":"v1","addr":"172.21.15.184:6805","nonce":3958753963}]},"public_addr":"172.21.15.184:6801/3958753963","cluster_addr":"172.21.15.184:6803/3958753963","heartbeat_back_addr":"172.21.15.184:6807/3958753963","heartbeat_front_addr":"172.21.15.184:6805/3958753963","state":["exists","up"]},{"osd":3,"uuid":"834a07f5-2c24-49e1-8e4a-12c96cfbdc65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3395182524},{"type":"v1","addr":"172.21.15.16:6811","nonce":3395182524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3395182524},{"type":"v1","addr":"172.21.15.16:6813","nonce":3395182524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3395182524},{"type":"v1","addr":"172.21.15.16:6817","nonce":3395182524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3395182524},{"type":"v1","addr":"172.21.15.16:6815","nonce":3395182524}]},"public_addr":"172.21.15.16:6811/3395182524","cluster_addr":"172.21.15.16:6813/3395182524","heartbeat_back_addr":"172.21.15.16:6817/3395182524","heartbeat_front_addr":"172.21.15.16:6815/3395182524","state":["exists","up"]},{"osd":4,"uuid":"3ca56e09-c194-4404-b9bd-2524d38367eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":1355184322},{"type":"v1","addr":"172.21.15.33:6809","nonce":1355184322}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1355184322},{"type":"v1","addr":"172.21.15.33:6811","nonce":1355184322}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1355184322},{"type":"v1","addr":"172.21.15.33:6815","nonce":1355184322}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1355184322},{"type":"v1","addr":"172.21.15.33:6813","nonce":1355184322}]},"public_addr":"172.21.15.33:6809/1355184322","cluster_addr":"172.21.15.33:6811/1355184322","heartbeat_back_addr":"172.21.15.33:6815/1355184322","heartbeat_front_addr":"172.21.15.33:6813/1355184322","state":["exists","up"]},{"osd":5,"uuid":"0b05ecdb-b987-461f-951c-6d76739bfb4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":1533397276},{"type":"v1","addr":"172.21.15.184:6809","nonce":1533397276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":1533397276},{"type":"v1","addr":"172.21.15.184:6811","nonce":1533397276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":1533397276},{"type":"v1","addr":"172.21.15.184:6815","nonce":1533397276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":1533397276},{"type":"v1","addr":"172.21.15.184:6813","nonce":1533397276}]},"public_addr":"172.21.15.184:6809/1533397276","cluster_addr":"172.21.15.184:6811/1533397276","heartbeat_back_addr":"172.21.15.184:6815/1533397276","heartbeat_front_addr":"172.21.15.184:6813/1533397276","state":["exists","up"]},{"osd":6,"uuid":"b3936e21-e197-49fa-8441-d909b258b612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":902466142},{"type":"v1","addr":"172.21.15.16:6819","nonce":902466142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":902466142},{"type":"v1","addr":"172.21.15.16:6821","nonce":902466142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":902466142},{"type":"v1","addr":"172.21.15.16:6825","nonce":902466142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":902466142},{"type":"v1","addr":"172.21.15.16:6823","nonce":902466142}]},"public_addr":"172.21.15.16:6819/902466142","cluster_addr":"172.21.15.16:6821/902466142","heartbeat_back_addr":"172.21.15.16:6825/902466142","heartbeat_front_addr":"172.21.15.16:6823/902466142","state":["exists","up"]},{"osd":7,"uuid":"4ddbe011-34c1-4b3f-a8eb-577422fb5426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":4186032630},{"type":"v1","addr":"172.21.15.33:6817","nonce":4186032630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":4186032630},{"type":"v1","addr":"172.21.15.33:6819","nonce":4186032630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":4186032630},{"type":"v1","addr":"172.21.15.33:6823","nonce":4186032630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":4186032630},{"type":"v1","addr":"172.21.15.33:6821","nonce":4186032630}]},"public_addr":"172.21.15.33:6817/4186032630","cluster_addr":"172.21.15.33:6819/4186032630","heartbeat_back_addr":"172.21.15.33:6823/4186032630","heartbeat_front_addr":"172.21.15.33:6821/4186032630","state":["exists","up"]},{"osd":8,"uuid":"dca5bfb8-ac6a-47ce-905c-be7fcaf0b5ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":781851781},{"type":"v1","addr":"172.21.15.184:6817","nonce":781851781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":781851781},{"type":"v1","addr":"172.21.15.184:6819","nonce":781851781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":781851781},{"type":"v1","addr":"172.21.15.184:6823","nonce":781851781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":781851781},{"type":"v1","addr":"172.21.15.184:6821","nonce":781851781}]},"public_addr":"172.21.15.184:6817/781851781","cluster_addr":"172.21.15.184:6819/781851781","heartbeat_back_addr":"172.21.15.184:6823/781851781","heartbeat_front_addr":"172.21.15.184:6821/781851781","state":["exists","up"]},{"osd":9,"uuid":"23a05cb5-01d5-40f3-b66a-15fcccedbd6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":1569578746},{"type":"v1","addr":"172.21.15.16:6827","nonce":1569578746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":1569578746},{"type":"v1","addr":"172.21.15.16:6829","nonce":1569578746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1569578746},{"type":"v1","addr":"172.21.15.16:6833","nonce":1569578746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":1569578746},{"type":"v1","addr":"172.21.15.16:6831","nonce":1569578746}]},"public_addr":"172.21.15.16:6827/1569578746","cluster_addr":"172.21.15.16:6829/1569578746","heartbeat_back_addr":"172.21.15.16:6833/1569578746","heartbeat_front_addr":"172.21.15.16:6831/1569578746","state":["exists","up"]},{"osd":10,"uuid":"e4bd6bce-3c69-44db-bc09-d264a0ad094a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":469977847},{"type":"v1","addr":"172.21.15.33:6825","nonce":469977847}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":469977847},{"type":"v1","addr":"172.21.15.33:6827","nonce":469977847}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":469977847},{"type":"v1","addr":"172.21.15.33:6831","nonce":469977847}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":469977847},{"type":"v1","addr":"172.21.15.33:6829","nonce":469977847}]},"public_addr":"172.21.15.33:6825/469977847","cluster_addr":"172.21.15.33:6827/469977847","heartbeat_back_addr":"172.21.15.33:6831/469977847","heartbeat_front_addr":"172.21.15.33:6829/469977847","state":["exists","up"]},{"osd":11,"uuid":"d60212f8-5a1d-45f7-9a1b-12294bb4c80d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":1619871346},{"type":"v1","addr":"172.21.15.184:6825","nonce":1619871346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":1619871346},{"type":"v1","addr":"172.21.15.184:6827","nonce":1619871346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":1619871346},{"type":"v1","addr":"172.21.15.184:6831","nonce":1619871346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":1619871346},{"type":"v1","addr":"172.21.15.184:6829","nonce":1619871346}]},"public_addr":"172.21.15.184:6825/1619871346","cluster_addr":"172.21.15.184:6827/1619871346","heartbeat_back_addr":"172.21.15.184:6831/1619871346","heartbeat_front_addr":"172.21.15.184:6829/1619871346","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:54:32.057979+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:54:57.501562+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:55:23.137225+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:55:48.646155+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:56:13.311907+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:56:38.214188+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:57:02.570921+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:57:27.506097+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:57:52.038529+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:58:17.423300+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:58:41.579992+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:59:06.245817+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:0/3532565698":"2024-09-13T01:53:08.218475+0000","172.21.15.16:0/1822157179":"2024-09-13T01:52:44.194152+0000","172.21.15.16:0/3919664061":"2024-09-13T01:52:44.194152+0000","172.21.15.16:0/2269394774":"2024-09-13T01:52:44.194152+0000","172.21.15.16:0/2595308634":"2024-09-13T01:53:08.218475+0000","172.21.15.16:6801/2754331153":"2024-09-13T01:52:44.194152+0000","172.21.15.16:6801/2615861334":"2024-09-13T01:53:08.218475+0000","172.21.15.16:6800/2615861334":"2024-09-13T01:53:08.218475+0000","172.21.15.16:0/4199114771":"2024-09-13T01:53:08.218475+0000","172.21.15.16:6800/2754331153":"2024-09-13T01:52:44.194152+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T01:59:58.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:58.087+0000 7fcd75cec640 1 -- 172.21.15.16:0/260924695 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fcd580a2c00 msgr2=0x7fcd580a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:58.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:58.087+0000 7fcd75cec640 1 --2- 172.21.15.16:0/260924695 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fcd580a2c00 0x7fcd580a50c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fcd60005fd0 tx=0x7fcd6000b390 comp rx=0 tx=0).stop 2024-09-12T01:59:58.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:58.087+0000 7fcd75cec640 1 -- 172.21.15.16:0/260924695 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd70106a80 msgr2=0x7fcd7019bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:58.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:58.087+0000 7fcd75cec640 1 --2- 172.21.15.16:0/260924695 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd70106a80 0x7fcd7019bcb0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fcd6800b970 tx=0x7fcd6800be30 comp rx=0 tx=0).stop 2024-09-12T01:59:58.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:58.087+0000 7fcd75cec640 1 -- 172.21.15.16:0/260924695 shutdown_connections 2024-09-12T01:59:58.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:58.087+0000 7fcd75cec640 1 --2- 172.21.15.16:0/260924695 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fcd580a2c00 0x7fcd580a50c0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:58.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:58.087+0000 7fcd75cec640 1 --2- 172.21.15.16:0/260924695 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd70106a80 0x7fcd7019bcb0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:58.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:58.087+0000 7fcd75cec640 1 --2- 172.21.15.16:0/260924695 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcd70101420 0x7fcd70197920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:58.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:58.087+0000 7fcd75cec640 1 --2- 172.21.15.16:0/260924695 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd70100a80 0x7fcd701973e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:58.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:58.087+0000 7fcd75cec640 1 -- 172.21.15.16:0/260924695 >> 172.21.15.16:0/260924695 conn(0x7fcd700fc7a0 msgr2=0x7fcd700fcee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:59:58.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:58.088+0000 7fcd75cec640 1 -- 172.21.15.16:0/260924695 shutdown_connections 2024-09-12T01:59:58.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:58.088+0000 7fcd75cec640 1 -- 172.21.15.16:0/260924695 wait complete. 2024-09-12T01:59:58.705 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:58 smithi016 ceph-mon[30947]: mds.? [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] up:active 2024-09-12T01:59:58.705 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:58 smithi016 ceph-mon[30947]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-12T01:59:58.705 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:58 smithi016 ceph-mon[30947]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-12T01:59:58.705 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:58 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/260924695' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T01:59:58.737 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T01:59:58.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:58 smithi184 ceph-mon[35593]: mds.? [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] up:active 2024-09-12T01:59:58.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:58 smithi184 ceph-mon[35593]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-12T01:59:58.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:58 smithi184 ceph-mon[35593]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-12T01:59:58.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:58 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/260924695' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T01:59:58.918 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:58 smithi033 ceph-mon[35332]: mds.? [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] up:active 2024-09-12T01:59:58.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:58 smithi033 ceph-mon[35332]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-12T01:59:58.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:58 smithi033 ceph-mon[35332]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-12T01:59:58.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:58 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/260924695' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T01:59:58.998 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:59:59.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:59.491+0000 7fc7e02f9640 1 -- 172.21.15.16:0/361174287 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7d8100a80 msgr2=0x7fc7d8100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:59.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:59.491+0000 7fc7e02f9640 1 --2- 172.21.15.16:0/361174287 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7d8100a80 0x7fc7d8100ee0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fc7c8009930 tx=0x7fc7c802a420 comp rx=0 tx=0).stop 2024-09-12T01:59:59.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:59.491+0000 7fc7e02f9640 1 -- 172.21.15.16:0/361174287 shutdown_connections 2024-09-12T01:59:59.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:59.491+0000 7fc7e02f9640 1 --2- 172.21.15.16:0/361174287 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc7d8101420 0x7fc7d810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:59.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:59.491+0000 7fc7e02f9640 1 --2- 172.21.15.16:0/361174287 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7d8100a80 0x7fc7d8100ee0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:59.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:59.491+0000 7fc7e02f9640 1 --2- 172.21.15.16:0/361174287 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc7d8106a80 0x7fc7d8106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:59.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:59.491+0000 7fc7e02f9640 1 -- 172.21.15.16:0/361174287 >> 172.21.15.16:0/361174287 conn(0x7fc7d80fc7a0 msgr2=0x7fc7d80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:59:59.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:59.491+0000 7fc7e02f9640 1 -- 172.21.15.16:0/361174287 shutdown_connections 2024-09-12T01:59:59.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:59.491+0000 7fc7e02f9640 1 -- 172.21.15.16:0/361174287 wait complete. 2024-09-12T01:59:59.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:59.492+0000 7fc7e02f9640 1 Processor -- start 2024-09-12T01:59:59.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:59.492+0000 7fc7e02f9640 1 -- start start 2024-09-12T01:59:59.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:59.492+0000 7fc7e02f9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7d8100a80 0x7fc7d80fce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:59.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:59.492+0000 7fc7e02f9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc7d8101420 0x7fc7d80fd370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:59.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:59.492+0000 7fc7e02f9640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc7d8106a80 0x7fc7d80fd8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:59.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:59.492+0000 7fc7e02f9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc7d8110e80 con 0x7fc7d8100a80 2024-09-12T01:59:59.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:59.492+0000 7fc7e02f9640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc7d8110d00 con 0x7fc7d8101420 2024-09-12T01:59:59.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:59.492+0000 7fc7e02f9640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc7d8111000 con 0x7fc7d8106a80 2024-09-12T01:59:59.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:59.492+0000 7fc7de095640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7d8100a80 0x7fc7d80fce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:59.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:59.492+0000 7fc7de095640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7d8100a80 0x7fc7d80fce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35288/0 (socket says 172.21.15.16:35288) 2024-09-12T01:59:59.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:59.492+0000 7fc7de095640 1 -- 172.21.15.16:0/2021089600 learned_addr learned my addr 172.21.15.16:0/2021089600 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:59:59.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:59.492+0000 7fc7dd894640 1 --2- 172.21.15.16:0/2021089600 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc7d8101420 0x7fc7d80fd370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:59.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:59.492+0000 7fc7de896640 1 --2- 172.21.15.16:0/2021089600 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc7d8106a80 0x7fc7d80fd8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:59.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:59.493+0000 7fc7de095640 1 -- 172.21.15.16:0/2021089600 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc7d8106a80 msgr2=0x7fc7d80fd8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:59.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:59.493+0000 7fc7de095640 1 --2- 172.21.15.16:0/2021089600 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc7d8106a80 0x7fc7d80fd8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:59.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:59.493+0000 7fc7de095640 1 -- 172.21.15.16:0/2021089600 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc7d8101420 msgr2=0x7fc7d80fd370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:59.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:59.493+0000 7fc7de095640 1 --2- 172.21.15.16:0/2021089600 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc7d8101420 0x7fc7d80fd370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:59.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:59.493+0000 7fc7de095640 1 -- 172.21.15.16:0/2021089600 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7d81a6950 con 0x7fc7d8100a80 2024-09-12T01:59:59.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:59.493+0000 7fc7de896640 1 --2- 172.21.15.16:0/2021089600 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc7d8106a80 0x7fc7d80fd8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:59:59.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:59.493+0000 7fc7de095640 1 --2- 172.21.15.16:0/2021089600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7d8100a80 0x7fc7d80fce30 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fc7d400c890 tx=0x7fc7d400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:59:59.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:59.493+0000 7fc7cf7fe640 1 -- 172.21.15.16:0/2021089600 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7d4004220 con 0x7fc7d8100a80 2024-09-12T01:59:59.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:59.493+0000 7fc7cf7fe640 1 -- 172.21.15.16:0/2021089600 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc7d40043c0 con 0x7fc7d8100a80 2024-09-12T01:59:59.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:59.493+0000 7fc7e02f9640 1 -- 172.21.15.16:0/2021089600 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7d81a6c40 con 0x7fc7d8100a80 2024-09-12T01:59:59.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:59.493+0000 7fc7cf7fe640 1 -- 172.21.15.16:0/2021089600 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7d40049e0 con 0x7fc7d8100a80 2024-09-12T01:59:59.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:59.493+0000 7fc7e02f9640 1 -- 172.21.15.16:0/2021089600 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7d81a7150 con 0x7fc7d8100a80 2024-09-12T01:59:59.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:59.494+0000 7fc7e02f9640 1 -- 172.21.15.16:0/2021089600 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7d8100ee0 con 0x7fc7d8100a80 2024-09-12T01:59:59.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:59.495+0000 7fc7cf7fe640 1 -- 172.21.15.16:0/2021089600 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fc7d40079b0 con 0x7fc7d8100a80 2024-09-12T01:59:59.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:59.496+0000 7fc7cf7fe640 1 --2- 172.21.15.16:0/2021089600 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc7c40a2b80 0x7fc7c40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:59.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:59.496+0000 7fc7cf7fe640 1 -- 172.21.15.16:0/2021089600 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7fc7d40cef10 con 0x7fc7d8100a80 2024-09-12T01:59:59.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:59.496+0000 7fc7dd894640 1 --2- 172.21.15.16:0/2021089600 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc7c40a2b80 0x7fc7c40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:59.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:59.496+0000 7fc7dd894640 1 --2- 172.21.15.16:0/2021089600 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc7c40a2b80 0x7fc7c40a5040 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fc7c8009900 tx=0x7fc7c80058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:59:59.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:59.500+0000 7fc7cf7fe640 1 -- 172.21.15.16:0/2021089600 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc7d4014030 con 0x7fc7d8100a80 2024-09-12T01:59:59.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:59.677+0000 7fc7e02f9640 1 -- 172.21.15.16:0/2021089600 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc7d8109a70 con 0x7fc7d8100a80 2024-09-12T01:59:59.677 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:59 smithi016 ceph-mon[30947]: pgmap v275: 97 pgs: 32 creating+peering, 11 unknown, 54 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 211 B/s wr, 4 op/s 2024-09-12T01:59:59.677 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:59 smithi016 ceph-mon[30947]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-12T01:59:59.678 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:59:59 smithi016 ceph-mon[30947]: Cluster is now healthy 2024-09-12T01:59:59.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:59.678+0000 7fc7cf7fe640 1 -- 172.21.15.16:0/2021089600 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+22245 (secure 0 0 0) 0x7fc7d409bfc0 con 0x7fc7d8100a80 2024-09-12T01:59:59.679 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:59:59.679 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":80,"fsid":"879f3b56-70a9-11ef-bcea-c7b262605968","created":"2024-09-12T01:52:12.841045+0000","modified":"2024-09-12T01:59:56.117422+0000","last_up_change":"2024-09-12T01:59:08.975443+0000","last_in_change":"2024-09-12T01:58:44.530220+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T01:55:26.727564+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T01:59:52.402097+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T01:59:54.463627+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3ea41eb6-e384-4ddc-810f-0f103c127849","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4000187867},{"type":"v1","addr":"172.21.15.16:6803","nonce":4000187867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4000187867},{"type":"v1","addr":"172.21.15.16:6805","nonce":4000187867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":4000187867},{"type":"v1","addr":"172.21.15.16:6809","nonce":4000187867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4000187867},{"type":"v1","addr":"172.21.15.16:6807","nonce":4000187867}]},"public_addr":"172.21.15.16:6803/4000187867","cluster_addr":"172.21.15.16:6805/4000187867","heartbeat_back_addr":"172.21.15.16:6809/4000187867","heartbeat_front_addr":"172.21.15.16:6807/4000187867","state":["exists","up"]},{"osd":1,"uuid":"07bf63e7-70fa-41b2-af53-12a5fec3f33c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":3862661148},{"type":"v1","addr":"172.21.15.33:6801","nonce":3862661148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":3862661148},{"type":"v1","addr":"172.21.15.33:6803","nonce":3862661148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":3862661148},{"type":"v1","addr":"172.21.15.33:6807","nonce":3862661148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":3862661148},{"type":"v1","addr":"172.21.15.33:6805","nonce":3862661148}]},"public_addr":"172.21.15.33:6801/3862661148","cluster_addr":"172.21.15.33:6803/3862661148","heartbeat_back_addr":"172.21.15.33:6807/3862661148","heartbeat_front_addr":"172.21.15.33:6805/3862661148","state":["exists","up"]},{"osd":2,"uuid":"22e40cac-3bc0-4cce-b47b-77499b767f7e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":3958753963},{"type":"v1","addr":"172.21.15.184:6801","nonce":3958753963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":3958753963},{"type":"v1","addr":"172.21.15.184:6803","nonce":3958753963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":3958753963},{"type":"v1","addr":"172.21.15.184:6807","nonce":3958753963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":3958753963},{"type":"v1","addr":"172.21.15.184:6805","nonce":3958753963}]},"public_addr":"172.21.15.184:6801/3958753963","cluster_addr":"172.21.15.184:6803/3958753963","heartbeat_back_addr":"172.21.15.184:6807/3958753963","heartbeat_front_addr":"172.21.15.184:6805/3958753963","state":["exists","up"]},{"osd":3,"uuid":"834a07f5-2c24-49e1-8e4a-12c96cfbdc65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3395182524},{"type":"v1","addr":"172.21.15.16:6811","nonce":3395182524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3395182524},{"type":"v1","addr":"172.21.15.16:6813","nonce":3395182524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3395182524},{"type":"v1","addr":"172.21.15.16:6817","nonce":3395182524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3395182524},{"type":"v1","addr":"172.21.15.16:6815","nonce":3395182524}]},"public_addr":"172.21.15.16:6811/3395182524","cluster_addr":"172.21.15.16:6813/3395182524","heartbeat_back_addr":"172.21.15.16:6817/3395182524","heartbeat_front_addr":"172.21.15.16:6815/3395182524","state":["exists","up"]},{"osd":4,"uuid":"3ca56e09-c194-4404-b9bd-2524d38367eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":1355184322},{"type":"v1","addr":"172.21.15.33:6809","nonce":1355184322}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1355184322},{"type":"v1","addr":"172.21.15.33:6811","nonce":1355184322}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1355184322},{"type":"v1","addr":"172.21.15.33:6815","nonce":1355184322}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1355184322},{"type":"v1","addr":"172.21.15.33:6813","nonce":1355184322}]},"public_addr":"172.21.15.33:6809/1355184322","cluster_addr":"172.21.15.33:6811/1355184322","heartbeat_back_addr":"172.21.15.33:6815/1355184322","heartbeat_front_addr":"172.21.15.33:6813/1355184322","state":["exists","up"]},{"osd":5,"uuid":"0b05ecdb-b987-461f-951c-6d76739bfb4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":1533397276},{"type":"v1","addr":"172.21.15.184:6809","nonce":1533397276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":1533397276},{"type":"v1","addr":"172.21.15.184:6811","nonce":1533397276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":1533397276},{"type":"v1","addr":"172.21.15.184:6815","nonce":1533397276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":1533397276},{"type":"v1","addr":"172.21.15.184:6813","nonce":1533397276}]},"public_addr":"172.21.15.184:6809/1533397276","cluster_addr":"172.21.15.184:6811/1533397276","heartbeat_back_addr":"172.21.15.184:6815/1533397276","heartbeat_front_addr":"172.21.15.184:6813/1533397276","state":["exists","up"]},{"osd":6,"uuid":"b3936e21-e197-49fa-8441-d909b258b612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":902466142},{"type":"v1","addr":"172.21.15.16:6819","nonce":902466142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":902466142},{"type":"v1","addr":"172.21.15.16:6821","nonce":902466142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":902466142},{"type":"v1","addr":"172.21.15.16:6825","nonce":902466142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":902466142},{"type":"v1","addr":"172.21.15.16:6823","nonce":902466142}]},"public_addr":"172.21.15.16:6819/902466142","cluster_addr":"172.21.15.16:6821/902466142","heartbeat_back_addr":"172.21.15.16:6825/902466142","heartbeat_front_addr":"172.21.15.16:6823/902466142","state":["exists","up"]},{"osd":7,"uuid":"4ddbe011-34c1-4b3f-a8eb-577422fb5426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":4186032630},{"type":"v1","addr":"172.21.15.33:6817","nonce":4186032630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":4186032630},{"type":"v1","addr":"172.21.15.33:6819","nonce":4186032630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":4186032630},{"type":"v1","addr":"172.21.15.33:6823","nonce":4186032630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":4186032630},{"type":"v1","addr":"172.21.15.33:6821","nonce":4186032630}]},"public_addr":"172.21.15.33:6817/4186032630","cluster_addr":"172.21.15.33:6819/4186032630","heartbeat_back_addr":"172.21.15.33:6823/4186032630","heartbeat_front_addr":"172.21.15.33:6821/4186032630","state":["exists","up"]},{"osd":8,"uuid":"dca5bfb8-ac6a-47ce-905c-be7fcaf0b5ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":781851781},{"type":"v1","addr":"172.21.15.184:6817","nonce":781851781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":781851781},{"type":"v1","addr":"172.21.15.184:6819","nonce":781851781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":781851781},{"type":"v1","addr":"172.21.15.184:6823","nonce":781851781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":781851781},{"type":"v1","addr":"172.21.15.184:6821","nonce":781851781}]},"public_addr":"172.21.15.184:6817/781851781","cluster_addr":"172.21.15.184:6819/781851781","heartbeat_back_addr":"172.21.15.184:6823/781851781","heartbeat_front_addr":"172.21.15.184:6821/781851781","state":["exists","up"]},{"osd":9,"uuid":"23a05cb5-01d5-40f3-b66a-15fcccedbd6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":1569578746},{"type":"v1","addr":"172.21.15.16:6827","nonce":1569578746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":1569578746},{"type":"v1","addr":"172.21.15.16:6829","nonce":1569578746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1569578746},{"type":"v1","addr":"172.21.15.16:6833","nonce":1569578746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":1569578746},{"type":"v1","addr":"172.21.15.16:6831","nonce":1569578746}]},"public_addr":"172.21.15.16:6827/1569578746","cluster_addr":"172.21.15.16:6829/1569578746","heartbeat_back_addr":"172.21.15.16:6833/1569578746","heartbeat_front_addr":"172.21.15.16:6831/1569578746","state":["exists","up"]},{"osd":10,"uuid":"e4bd6bce-3c69-44db-bc09-d264a0ad094a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":469977847},{"type":"v1","addr":"172.21.15.33:6825","nonce":469977847}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":469977847},{"type":"v1","addr":"172.21.15.33:6827","nonce":469977847}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":469977847},{"type":"v1","addr":"172.21.15.33:6831","nonce":469977847}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":469977847},{"type":"v1","addr":"172.21.15.33:6829","nonce":469977847}]},"public_addr":"172.21.15.33:6825/469977847","cluster_addr":"172.21.15.33:6827/469977847","heartbeat_back_addr":"172.21.15.33:6831/469977847","heartbeat_front_addr":"172.21.15.33:6829/469977847","state":["exists","up"]},{"osd":11,"uuid":"d60212f8-5a1d-45f7-9a1b-12294bb4c80d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":1619871346},{"type":"v1","addr":"172.21.15.184:6825","nonce":1619871346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":1619871346},{"type":"v1","addr":"172.21.15.184:6827","nonce":1619871346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":1619871346},{"type":"v1","addr":"172.21.15.184:6831","nonce":1619871346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":1619871346},{"type":"v1","addr":"172.21.15.184:6829","nonce":1619871346}]},"public_addr":"172.21.15.184:6825/1619871346","cluster_addr":"172.21.15.184:6827/1619871346","heartbeat_back_addr":"172.21.15.184:6831/1619871346","heartbeat_front_addr":"172.21.15.184:6829/1619871346","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:54:32.057979+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:54:57.501562+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:55:23.137225+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:55:48.646155+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:56:13.311907+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:56:38.214188+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:57:02.570921+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:57:27.506097+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:57:52.038529+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:58:17.423300+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:58:41.579992+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:59:06.245817+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:0/3532565698":"2024-09-13T01:53:08.218475+0000","172.21.15.16:0/1822157179":"2024-09-13T01:52:44.194152+0000","172.21.15.16:0/3919664061":"2024-09-13T01:52:44.194152+0000","172.21.15.16:0/2269394774":"2024-09-13T01:52:44.194152+0000","172.21.15.16:0/2595308634":"2024-09-13T01:53:08.218475+0000","172.21.15.16:6801/2754331153":"2024-09-13T01:52:44.194152+0000","172.21.15.16:6801/2615861334":"2024-09-13T01:53:08.218475+0000","172.21.15.16:6800/2615861334":"2024-09-13T01:53:08.218475+0000","172.21.15.16:0/4199114771":"2024-09-13T01:53:08.218475+0000","172.21.15.16:6800/2754331153":"2024-09-13T01:52:44.194152+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T01:59:59.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:59.681+0000 7fc7e02f9640 1 -- 172.21.15.16:0/2021089600 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc7c40a2b80 msgr2=0x7fc7c40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:59.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:59.681+0000 7fc7e02f9640 1 --2- 172.21.15.16:0/2021089600 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc7c40a2b80 0x7fc7c40a5040 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fc7c8009900 tx=0x7fc7c80058d0 comp rx=0 tx=0).stop 2024-09-12T01:59:59.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:59.681+0000 7fc7e02f9640 1 -- 172.21.15.16:0/2021089600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7d8100a80 msgr2=0x7fc7d80fce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:59.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:59.681+0000 7fc7e02f9640 1 --2- 172.21.15.16:0/2021089600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7d8100a80 0x7fc7d80fce30 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fc7d400c890 tx=0x7fc7d400cd50 comp rx=0 tx=0).stop 2024-09-12T01:59:59.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:59.681+0000 7fc7e02f9640 1 -- 172.21.15.16:0/2021089600 shutdown_connections 2024-09-12T01:59:59.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:59.681+0000 7fc7e02f9640 1 --2- 172.21.15.16:0/2021089600 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc7c40a2b80 0x7fc7c40a5040 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:59.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:59.681+0000 7fc7e02f9640 1 --2- 172.21.15.16:0/2021089600 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc7d8106a80 0x7fc7d80fd8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:59.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:59.681+0000 7fc7e02f9640 1 --2- 172.21.15.16:0/2021089600 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc7d8101420 0x7fc7d80fd370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:59.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:59.681+0000 7fc7e02f9640 1 --2- 172.21.15.16:0/2021089600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7d8100a80 0x7fc7d80fce30 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:59.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:59.681+0000 7fc7e02f9640 1 -- 172.21.15.16:0/2021089600 >> 172.21.15.16:0/2021089600 conn(0x7fc7d80fc7a0 msgr2=0x7fc7d806aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:59:59.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:59.681+0000 7fc7e02f9640 1 -- 172.21.15.16:0/2021089600 shutdown_connections 2024-09-12T01:59:59.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:59:59.681+0000 7fc7e02f9640 1 -- 172.21.15.16:0/2021089600 wait complete. 2024-09-12T01:59:59.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:59 smithi184 ceph-mon[35593]: pgmap v275: 97 pgs: 32 creating+peering, 11 unknown, 54 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 211 B/s wr, 4 op/s 2024-09-12T01:59:59.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:59 smithi184 ceph-mon[35593]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-12T01:59:59.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 01:59:59 smithi184 ceph-mon[35593]: Cluster is now healthy 2024-09-12T01:59:59.918 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:59 smithi033 ceph-mon[35332]: pgmap v275: 97 pgs: 32 creating+peering, 11 unknown, 54 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 211 B/s wr, 4 op/s 2024-09-12T01:59:59.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:59 smithi033 ceph-mon[35332]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-12T01:59:59.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 01:59:59 smithi033 ceph-mon[35332]: Cluster is now healthy 2024-09-12T02:00:00.305 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-09-12T02:00:00.569 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:00:00.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:00:00 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/2021089600' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T02:00:00.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:00:00 smithi016 ceph-mon[30947]: overall HEALTH_OK 2024-09-12T02:00:00.846 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:00:00 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/2021089600' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T02:00:00.846 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:00:00 smithi184 ceph-mon[35593]: overall HEALTH_OK 2024-09-12T02:00:00.918 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:00:00 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/2021089600' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T02:00:00.918 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:00:00 smithi033 ceph-mon[35332]: overall HEALTH_OK 2024-09-12T02:00:01.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:01.042+0000 7fed76286640 1 -- 172.21.15.16:0/2402671268 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed70101de0 msgr2=0x7fed701021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:01.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:01.042+0000 7fed76286640 1 --2- 172.21.15.16:0/2402671268 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed70101de0 0x7fed701021c0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fed60009900 tx=0x7fed6002a410 comp rx=0 tx=0).stop 2024-09-12T02:00:01.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:01.043+0000 7fed76286640 1 -- 172.21.15.16:0/2402671268 shutdown_connections 2024-09-12T02:00:01.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:01.043+0000 7fed76286640 1 --2- 172.21.15.16:0/2402671268 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fed70106f10 0x7fed7010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:01.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:01.043+0000 7fed76286640 1 --2- 172.21.15.16:0/2402671268 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fed70102790 0x7fed701067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:01.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:01.043+0000 7fed76286640 1 --2- 172.21.15.16:0/2402671268 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed70101de0 0x7fed701021c0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:01.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:01.043+0000 7fed76286640 1 -- 172.21.15.16:0/2402671268 >> 172.21.15.16:0/2402671268 conn(0x7fed700fd8c0 msgr2=0x7fed700ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:00:01.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:01.043+0000 7fed76286640 1 -- 172.21.15.16:0/2402671268 shutdown_connections 2024-09-12T02:00:01.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:01.043+0000 7fed76286640 1 -- 172.21.15.16:0/2402671268 wait complete. 2024-09-12T02:00:01.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:01.043+0000 7fed76286640 1 Processor -- start 2024-09-12T02:00:01.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:01.043+0000 7fed76286640 1 -- start start 2024-09-12T02:00:01.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:01.044+0000 7fed76286640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fed70101de0 0x7fed701984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:01.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:01.044+0000 7fed76286640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed70102790 0x7fed70198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:01.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:01.044+0000 7fed76286640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fed70106f10 0x7fed7019cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:01.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:01.044+0000 7fed76286640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fed70111f70 con 0x7fed70102790 2024-09-12T02:00:01.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:01.044+0000 7fed76286640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fed70111df0 con 0x7fed70106f10 2024-09-12T02:00:01.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:01.044+0000 7fed76286640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fed701120f0 con 0x7fed70101de0 2024-09-12T02:00:01.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:01.044+0000 7fed6f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed70102790 0x7fed70198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:01.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:01.044+0000 7fed6f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed70102790 0x7fed70198a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35302/0 (socket says 172.21.15.16:35302) 2024-09-12T02:00:01.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:01.044+0000 7fed6f7fe640 1 -- 172.21.15.16:0/2688239777 learned_addr learned my addr 172.21.15.16:0/2688239777 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:00:01.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:01.044+0000 7fed6ffff640 1 --2- 172.21.15.16:0/2688239777 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fed70101de0 0x7fed701984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:01.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:01.044+0000 7fed74823640 1 --2- 172.21.15.16:0/2688239777 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fed70106f10 0x7fed7019cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:01.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:01.044+0000 7fed6f7fe640 1 -- 172.21.15.16:0/2688239777 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fed70101de0 msgr2=0x7fed701984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:01.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:01.044+0000 7fed6f7fe640 1 --2- 172.21.15.16:0/2688239777 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fed70101de0 0x7fed701984d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:01.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:01.044+0000 7fed6f7fe640 1 -- 172.21.15.16:0/2688239777 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fed70106f10 msgr2=0x7fed7019cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:01.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:01.044+0000 7fed6f7fe640 1 --2- 172.21.15.16:0/2688239777 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fed70106f10 0x7fed7019cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:01.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:01.044+0000 7fed6f7fe640 1 -- 172.21.15.16:0/2688239777 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed7019d480 con 0x7fed70102790 2024-09-12T02:00:01.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:01.044+0000 7fed6ffff640 1 --2- 172.21.15.16:0/2688239777 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fed70101de0 0x7fed701984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:00:01.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:01.044+0000 7fed74823640 1 --2- 172.21.15.16:0/2688239777 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fed70106f10 0x7fed7019cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:00:01.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:01.044+0000 7fed6f7fe640 1 --2- 172.21.15.16:0/2688239777 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed70102790 0x7fed70198a10 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fed6400d770 tx=0x7fed6400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:00:01.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:01.044+0000 7fed6d7fa640 1 -- 172.21.15.16:0/2688239777 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed64017070 con 0x7fed70102790 2024-09-12T02:00:01.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:01.044+0000 7fed76286640 1 -- 172.21.15.16:0/2688239777 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed7019d770 con 0x7fed70102790 2024-09-12T02:00:01.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:01.044+0000 7fed6d7fa640 1 -- 172.21.15.16:0/2688239777 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed64005500 con 0x7fed70102790 2024-09-12T02:00:01.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:01.044+0000 7fed6d7fa640 1 -- 172.21.15.16:0/2688239777 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed64004e00 con 0x7fed70102790 2024-09-12T02:00:01.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:01.044+0000 7fed76286640 1 -- 172.21.15.16:0/2688239777 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed701a5050 con 0x7fed70102790 2024-09-12T02:00:01.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:01.046+0000 7fed76286640 1 -- 172.21.15.16:0/2688239777 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed701021c0 con 0x7fed70102790 2024-09-12T02:00:01.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:01.047+0000 7fed6d7fa640 1 -- 172.21.15.16:0/2688239777 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fed64004fa0 con 0x7fed70102790 2024-09-12T02:00:01.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:01.048+0000 7fed6d7fa640 1 --2- 172.21.15.16:0/2688239777 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fed580a2b80 0x7fed580a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:01.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:01.048+0000 7fed6ffff640 1 --2- 172.21.15.16:0/2688239777 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fed580a2b80 0x7fed580a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:01.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:01.048+0000 7fed6d7fa640 1 -- 172.21.15.16:0/2688239777 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7fed640cee70 con 0x7fed70102790 2024-09-12T02:00:01.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:01.048+0000 7fed6ffff640 1 --2- 172.21.15.16:0/2688239777 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fed580a2b80 0x7fed580a5040 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fed60005ab0 tx=0x7fed60005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:00:01.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:01.051+0000 7fed6d7fa640 1 -- 172.21.15.16:0/2688239777 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fed6409bf00 con 0x7fed70102790 2024-09-12T02:00:01.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:01.267+0000 7fed76286640 1 -- 172.21.15.16:0/2688239777 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fed70199830 con 0x7fed70102790 2024-09-12T02:00:01.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:01.513+0000 7fed6d7fa640 1 -- 172.21.15.16:0/2688239777 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) ==== 118+0+0 (secure 0 0 0) 0x7fed70199830 con 0x7fed70102790 2024-09-12T02:00:01.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:01.515+0000 7fed76286640 1 -- 172.21.15.16:0/2688239777 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fed580a2b80 msgr2=0x7fed580a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:01.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:01.515+0000 7fed76286640 1 --2- 172.21.15.16:0/2688239777 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fed580a2b80 0x7fed580a5040 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fed60005ab0 tx=0x7fed60005a20 comp rx=0 tx=0).stop 2024-09-12T02:00:01.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:01.515+0000 7fed76286640 1 -- 172.21.15.16:0/2688239777 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed70102790 msgr2=0x7fed70198a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:01.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:01.515+0000 7fed76286640 1 --2- 172.21.15.16:0/2688239777 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed70102790 0x7fed70198a10 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fed6400d770 tx=0x7fed6400dc30 comp rx=0 tx=0).stop 2024-09-12T02:00:01.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:01.516+0000 7fed76286640 1 -- 172.21.15.16:0/2688239777 shutdown_connections 2024-09-12T02:00:01.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:01.516+0000 7fed76286640 1 --2- 172.21.15.16:0/2688239777 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fed580a2b80 0x7fed580a5040 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:01.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:01.516+0000 7fed76286640 1 --2- 172.21.15.16:0/2688239777 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fed70106f10 0x7fed7019cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:01.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:01.516+0000 7fed76286640 1 --2- 172.21.15.16:0/2688239777 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed70102790 0x7fed70198a10 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:01.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:01.516+0000 7fed76286640 1 --2- 172.21.15.16:0/2688239777 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fed70101de0 0x7fed701984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:01.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:01.516+0000 7fed76286640 1 -- 172.21.15.16:0/2688239777 >> 172.21.15.16:0/2688239777 conn(0x7fed700fd8c0 msgr2=0x7fed700fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:00:01.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:01.516+0000 7fed76286640 1 -- 172.21.15.16:0/2688239777 shutdown_connections 2024-09-12T02:00:01.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:01.516+0000 7fed76286640 1 -- 172.21.15.16:0/2688239777 wait complete. 2024-09-12T02:00:01.527 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:00:01 smithi016 ceph-mon[30947]: pgmap v276: 97 pgs: 6 creating+peering, 91 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s wr, 6 op/s 2024-09-12T02:00:01.527 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:00:01 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/2688239777' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-12T02:00:01.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:00:01 smithi184 ceph-mon[35593]: pgmap v276: 97 pgs: 6 creating+peering, 91 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s wr, 6 op/s 2024-09-12T02:00:01.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:00:01 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/2688239777' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-12T02:00:01.918 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:00:01 smithi033 ceph-mon[35332]: pgmap v276: 97 pgs: 6 creating+peering, 91 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s wr, 6 op/s 2024-09-12T02:00:01.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:00:01 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/2688239777' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-12T02:00:02.191 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-09-12T02:00:02.192 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph fs set cephfs max_mds 3 2024-09-12T02:00:02.419 INFO:journalctl@ceph.mds.b.smithi033.stdout:Sep 12 02:00:02 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mds-b[65151]: 2024-09-12T02:00:02.051+0000 7fe773e15640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-12T02:00:02.454 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:00:02.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:00:02 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/2688239777' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-12T02:00:02.878 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:00:02 smithi184 ceph-mon[35593]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-12T02:00:02.881 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:00:02 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/2688239777' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-12T02:00:02.881 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:00:02 smithi016 ceph-mon[30947]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-12T02:00:02.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:00:02 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/2688239777' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-12T02:00:02.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:00:02 smithi033 ceph-mon[35332]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-12T02:00:02.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:02.966+0000 7f736beec640 1 -- 172.21.15.16:0/4265056558 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73641008d0 msgr2=0x7f7364100cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:02.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:02.966+0000 7f736beec640 1 --2- 172.21.15.16:0/4265056558 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73641008d0 0x7f7364100cb0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f7360009900 tx=0x7f736002a410 comp rx=0 tx=0).stop 2024-09-12T02:00:02.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:02.966+0000 7f736beec640 1 -- 172.21.15.16:0/4265056558 shutdown_connections 2024-09-12T02:00:02.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:02.966+0000 7f736beec640 1 --2- 172.21.15.16:0/4265056558 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f736410e480 0x7f7364110870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:02.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:02.966+0000 7f736beec640 1 --2- 172.21.15.16:0/4265056558 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7364101280 0x7f736410df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:02.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:02.966+0000 7f736beec640 1 --2- 172.21.15.16:0/4265056558 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73641008d0 0x7f7364100cb0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:02.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:02.966+0000 7f736beec640 1 -- 172.21.15.16:0/4265056558 >> 172.21.15.16:0/4265056558 conn(0x7f73640fc7a0 msgr2=0x7f73640febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:00:02.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:02.966+0000 7f736beec640 1 -- 172.21.15.16:0/4265056558 shutdown_connections 2024-09-12T02:00:02.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:02.966+0000 7f736beec640 1 -- 172.21.15.16:0/4265056558 wait complete. 2024-09-12T02:00:02.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:02.967+0000 7f736beec640 1 Processor -- start 2024-09-12T02:00:02.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:02.967+0000 7f736beec640 1 -- start start 2024-09-12T02:00:02.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:02.967+0000 7f736beec640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73641008d0 0x7f736419a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:02.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:02.967+0000 7f736beec640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7364101280 0x7f736419ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:02.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:02.967+0000 7f736beec640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f736410e480 0x7f736419efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:02.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:02.967+0000 7f736beec640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f73641141f0 con 0x7f73641008d0 2024-09-12T02:00:02.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:02.967+0000 7f736beec640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7364114070 con 0x7f7364101280 2024-09-12T02:00:02.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:02.967+0000 7f736beec640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7364114370 con 0x7f736410e480 2024-09-12T02:00:02.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:02.967+0000 7f7369c88640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73641008d0 0x7f736419a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:02.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:02.967+0000 7f7369c88640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73641008d0 0x7f736419a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45038/0 (socket says 172.21.15.16:45038) 2024-09-12T02:00:02.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:02.967+0000 7f7369c88640 1 -- 172.21.15.16:0/3431559194 learned_addr learned my addr 172.21.15.16:0/3431559194 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:00:02.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:02.967+0000 7f7369487640 1 --2- 172.21.15.16:0/3431559194 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7364101280 0x7f736419ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:02.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:02.968+0000 7f7369c88640 1 -- 172.21.15.16:0/3431559194 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f736410e480 msgr2=0x7f736419efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:02.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:02.968+0000 7f736a489640 1 --2- 172.21.15.16:0/3431559194 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f736410e480 0x7f736419efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:02.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:02.968+0000 7f7369c88640 1 --2- 172.21.15.16:0/3431559194 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f736410e480 0x7f736419efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:02.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:02.968+0000 7f7369c88640 1 -- 172.21.15.16:0/3431559194 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7364101280 msgr2=0x7f736419ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:02.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:02.968+0000 7f7369c88640 1 --2- 172.21.15.16:0/3431559194 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7364101280 0x7f736419ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:02.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:02.968+0000 7f7369c88640 1 -- 172.21.15.16:0/3431559194 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f736419f680 con 0x7f73641008d0 2024-09-12T02:00:02.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:02.968+0000 7f736a489640 1 --2- 172.21.15.16:0/3431559194 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f736410e480 0x7f736419efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:00:02.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:02.968+0000 7f7369487640 1 --2- 172.21.15.16:0/3431559194 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7364101280 0x7f736419ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:00:02.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:02.968+0000 7f7369c88640 1 --2- 172.21.15.16:0/3431559194 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73641008d0 0x7f736419a6d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f7360006010 tx=0x7f73600042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:00:02.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:02.968+0000 7f735affd640 1 -- 172.21.15.16:0/3431559194 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f736003f030 con 0x7f73641008d0 2024-09-12T02:00:02.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:02.968+0000 7f735affd640 1 -- 172.21.15.16:0/3431559194 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7360003ce0 con 0x7f73641008d0 2024-09-12T02:00:02.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:02.968+0000 7f735affd640 1 -- 172.21.15.16:0/3431559194 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7360035070 con 0x7f73641008d0 2024-09-12T02:00:02.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:02.968+0000 7f736beec640 1 -- 172.21.15.16:0/3431559194 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f736419f910 con 0x7f73641008d0 2024-09-12T02:00:02.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:02.968+0000 7f736beec640 1 -- 172.21.15.16:0/3431559194 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73641a7250 con 0x7f73641008d0 2024-09-12T02:00:02.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:02.969+0000 7f736beec640 1 -- 172.21.15.16:0/3431559194 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7364100cb0 con 0x7f73641008d0 2024-09-12T02:00:02.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:02.971+0000 7f735affd640 1 -- 172.21.15.16:0/3431559194 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f7360042030 con 0x7f73641008d0 2024-09-12T02:00:02.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:02.971+0000 7f735affd640 1 --2- 172.21.15.16:0/3431559194 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f73500a2b80 0x7f73500a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:02.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:02.971+0000 7f7369487640 1 --2- 172.21.15.16:0/3431559194 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f73500a2b80 0x7f73500a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:02.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:02.971+0000 7f735affd640 1 -- 172.21.15.16:0/3431559194 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f73600ef7c0 con 0x7f73641008d0 2024-09-12T02:00:02.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:02.972+0000 7f7369487640 1 --2- 172.21.15.16:0/3431559194 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f73500a2b80 0x7f73500a5040 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f73540058c0 tx=0x7f7354005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:00:02.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:02.974+0000 7f735affd640 1 -- 172.21.15.16:0/3431559194 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f73600bc810 con 0x7f73641008d0 2024-09-12T02:00:03.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:03.187+0000 7f736beec640 1 -- 172.21.15.16:0/3431559194 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) -- 0x7f7364109610 con 0x7f73641008d0 2024-09-12T02:00:03.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:03.523+0000 7f735affd640 1 -- 172.21.15.16:0/3431559194 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v18) ==== 105+0+0 (secure 0 0 0) 0x7f73600bc110 con 0x7f73641008d0 2024-09-12T02:00:03.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:03.526+0000 7f736beec640 1 -- 172.21.15.16:0/3431559194 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f73500a2b80 msgr2=0x7f73500a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:03.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:03.526+0000 7f736beec640 1 --2- 172.21.15.16:0/3431559194 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f73500a2b80 0x7f73500a5040 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f73540058c0 tx=0x7f7354005850 comp rx=0 tx=0).stop 2024-09-12T02:00:03.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:03.526+0000 7f736beec640 1 -- 172.21.15.16:0/3431559194 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73641008d0 msgr2=0x7f736419a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:03.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:03.526+0000 7f736beec640 1 --2- 172.21.15.16:0/3431559194 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73641008d0 0x7f736419a6d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f7360006010 tx=0x7f73600042f0 comp rx=0 tx=0).stop 2024-09-12T02:00:03.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:03.526+0000 7f736beec640 1 -- 172.21.15.16:0/3431559194 shutdown_connections 2024-09-12T02:00:03.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:03.526+0000 7f736beec640 1 --2- 172.21.15.16:0/3431559194 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f73500a2b80 0x7f73500a5040 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:03.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:03.526+0000 7f736beec640 1 --2- 172.21.15.16:0/3431559194 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f736410e480 0x7f736419efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:03.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:03.526+0000 7f736beec640 1 --2- 172.21.15.16:0/3431559194 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7364101280 0x7f736419ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:03.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:03.526+0000 7f736beec640 1 --2- 172.21.15.16:0/3431559194 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73641008d0 0x7f736419a6d0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:03.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:03.526+0000 7f736beec640 1 -- 172.21.15.16:0/3431559194 >> 172.21.15.16:0/3431559194 conn(0x7f73640fc7a0 msgr2=0x7f73640feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:00:03.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:03.526+0000 7f736beec640 1 -- 172.21.15.16:0/3431559194 shutdown_connections 2024-09-12T02:00:03.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:03.526+0000 7f736beec640 1 -- 172.21.15.16:0/3431559194 wait complete. 2024-09-12T02:00:03.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:00:03 smithi184 ceph-mon[35593]: pgmap v277: 97 pgs: 97 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 5 op/s 2024-09-12T02:00:03.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:00:03 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/3431559194' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-09-12T02:00:03.918 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:00:03 smithi033 ceph-mon[35332]: pgmap v277: 97 pgs: 97 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 5 op/s 2024-09-12T02:00:03.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:00:03 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/3431559194' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-09-12T02:00:03.933 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:00:03 smithi016 ceph-mon[30947]: pgmap v277: 97 pgs: 97 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 5 op/s 2024-09-12T02:00:03.933 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:00:03 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/3431559194' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-09-12T02:00:04.414 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-09-12T02:00:04.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:00:04 smithi016 ceph-mon[30947]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-12T02:00:04.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:00:04 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/3431559194' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-09-12T02:00:04.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:00:04 smithi016 ceph-mon[30947]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-12T02:00:04.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:00:04 smithi016 ceph-mon[30947]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-12T02:00:04.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:00:04 smithi016 ceph-mon[30947]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-09-12T02:00:04.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:00:04 smithi016 ceph-mon[30947]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-09-12T02:00:04.687 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:00:04.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:00:04 smithi184 ceph-mon[35593]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-12T02:00:04.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:00:04 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/3431559194' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-09-12T02:00:04.878 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:00:04 smithi184 ceph-mon[35593]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-12T02:00:04.878 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:00:04 smithi184 ceph-mon[35593]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-12T02:00:04.878 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:00:04 smithi184 ceph-mon[35593]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-09-12T02:00:04.878 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:00:04 smithi184 ceph-mon[35593]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-09-12T02:00:04.918 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:00:04 smithi033 ceph-mon[35332]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-12T02:00:04.918 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:00:04 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/3431559194' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-09-12T02:00:04.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:00:04 smithi033 ceph-mon[35332]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-12T02:00:04.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:00:04 smithi033 ceph-mon[35332]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-12T02:00:04.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:00:04 smithi033 ceph-mon[35332]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-09-12T02:00:04.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:00:04 smithi033 ceph-mon[35332]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-09-12T02:00:05.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:05.150+0000 7f9baf989640 1 -- 172.21.15.16:0/3348262430 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ba8100a70 msgr2=0x7f9ba8100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:05.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:05.150+0000 7f9baf989640 1 --2- 172.21.15.16:0/3348262430 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ba8100a70 0x7f9ba8100ed0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f9b98009930 tx=0x7f9b9802a420 comp rx=0 tx=0).stop 2024-09-12T02:00:05.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:05.151+0000 7f9baf989640 1 -- 172.21.15.16:0/3348262430 shutdown_connections 2024-09-12T02:00:05.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:05.151+0000 7f9baf989640 1 --2- 172.21.15.16:0/3348262430 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ba8101410 0x7f9ba810d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:05.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:05.151+0000 7f9baf989640 1 --2- 172.21.15.16:0/3348262430 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ba8100a70 0x7f9ba8100ed0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:05.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:05.151+0000 7f9baf989640 1 --2- 172.21.15.16:0/3348262430 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9ba8106a70 0x7f9ba8106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:05.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:05.151+0000 7f9baf989640 1 -- 172.21.15.16:0/3348262430 >> 172.21.15.16:0/3348262430 conn(0x7f9ba80fc790 msgr2=0x7f9ba80febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:00:05.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:05.151+0000 7f9baf989640 1 -- 172.21.15.16:0/3348262430 shutdown_connections 2024-09-12T02:00:05.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:05.151+0000 7f9baf989640 1 -- 172.21.15.16:0/3348262430 wait complete. 2024-09-12T02:00:05.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:05.151+0000 7f9baf989640 1 Processor -- start 2024-09-12T02:00:05.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:05.151+0000 7f9baf989640 1 -- start start 2024-09-12T02:00:05.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:05.151+0000 7f9baf989640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ba8100a70 0x7f9ba8198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:05.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:05.152+0000 7f9baf989640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9ba8101410 0x7f9ba8198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:05.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:05.152+0000 7f9baf989640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ba8106a70 0x7f9ba819ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:05.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:05.152+0000 7f9baf989640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9ba8110e30 con 0x7f9ba8106a70 2024-09-12T02:00:05.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:05.152+0000 7f9baf989640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9ba8110cb0 con 0x7f9ba8100a70 2024-09-12T02:00:05.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:05.152+0000 7f9baf989640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f9ba8110fb0 con 0x7f9ba8101410 2024-09-12T02:00:05.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:05.152+0000 7f9badf26640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ba8106a70 0x7f9ba819ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:05.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:05.152+0000 7f9badf26640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ba8106a70 0x7f9ba819ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45058/0 (socket says 172.21.15.16:45058) 2024-09-12T02:00:05.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:05.152+0000 7f9badf26640 1 -- 172.21.15.16:0/1888916002 learned_addr learned my addr 172.21.15.16:0/1888916002 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:00:05.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:05.152+0000 7f9bad725640 1 --2- 172.21.15.16:0/1888916002 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ba8100a70 0x7f9ba8198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:05.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:05.152+0000 7f9bacf24640 1 --2- 172.21.15.16:0/1888916002 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9ba8101410 0x7f9ba8198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:05.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:05.152+0000 7f9badf26640 1 -- 172.21.15.16:0/1888916002 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9ba8101410 msgr2=0x7f9ba8198940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:05.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:05.152+0000 7f9badf26640 1 --2- 172.21.15.16:0/1888916002 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9ba8101410 0x7f9ba8198940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:05.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:05.152+0000 7f9badf26640 1 -- 172.21.15.16:0/1888916002 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ba8100a70 msgr2=0x7f9ba8198400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:05.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:05.152+0000 7f9badf26640 1 --2- 172.21.15.16:0/1888916002 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ba8100a70 0x7f9ba8198400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:05.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:05.152+0000 7f9badf26640 1 -- 172.21.15.16:0/1888916002 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ba819d3b0 con 0x7f9ba8106a70 2024-09-12T02:00:05.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:05.152+0000 7f9badf26640 1 --2- 172.21.15.16:0/1888916002 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ba8106a70 0x7f9ba819ccd0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f9ba0009fd0 tx=0x7f9ba000def0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:00:05.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:05.152+0000 7f9b9e7fc640 1 -- 172.21.15.16:0/1888916002 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ba0017070 con 0x7f9ba8106a70 2024-09-12T02:00:05.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:05.152+0000 7f9b9e7fc640 1 -- 172.21.15.16:0/1888916002 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ba0003e50 con 0x7f9ba8106a70 2024-09-12T02:00:05.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:05.152+0000 7f9baf989640 1 -- 172.21.15.16:0/1888916002 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ba819d6a0 con 0x7f9ba8106a70 2024-09-12T02:00:05.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:05.152+0000 7f9b9e7fc640 1 -- 172.21.15.16:0/1888916002 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ba0010460 con 0x7f9ba8106a70 2024-09-12T02:00:05.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:05.152+0000 7f9baf989640 1 -- 172.21.15.16:0/1888916002 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ba81a4f80 con 0x7f9ba8106a70 2024-09-12T02:00:05.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:05.153+0000 7f9baf989640 1 -- 172.21.15.16:0/1888916002 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ba81021b0 con 0x7f9ba8106a70 2024-09-12T02:00:05.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:05.155+0000 7f9b9e7fc640 1 -- 172.21.15.16:0/1888916002 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f9ba0003970 con 0x7f9ba8106a70 2024-09-12T02:00:05.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:05.156+0000 7f9b9e7fc640 1 --2- 172.21.15.16:0/1888916002 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f9b940a2b80 0x7f9b940a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:05.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:05.156+0000 7f9bad725640 1 --2- 172.21.15.16:0/1888916002 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f9b940a2b80 0x7f9b940a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:05.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:05.156+0000 7f9b9e7fc640 1 -- 172.21.15.16:0/1888916002 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f9ba00ceb90 con 0x7f9ba8106a70 2024-09-12T02:00:05.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:05.156+0000 7f9bad725640 1 --2- 172.21.15.16:0/1888916002 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f9b940a2b80 0x7f9b940a5040 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f9ba400ab60 tx=0x7f9ba400a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:00:05.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:05.159+0000 7f9b9e7fc640 1 -- 172.21.15.16:0/1888916002 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9ba009bc20 con 0x7f9ba8106a70 2024-09-12T02:00:05.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:05.374+0000 7f9baf989640 1 -- 172.21.15.16:0/1888916002 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f9ba8100ed0 con 0x7f9ba8106a70 2024-09-12T02:00:05.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:05.538+0000 7f9b9e7fc640 1 -- 172.21.15.16:0/1888916002 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v22) ==== 121+0+0 (secure 0 0 0) 0x7f9ba8100ed0 con 0x7f9ba8106a70 2024-09-12T02:00:05.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:05.541+0000 7f9baf989640 1 -- 172.21.15.16:0/1888916002 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f9b940a2b80 msgr2=0x7f9b940a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:05.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:05.541+0000 7f9baf989640 1 --2- 172.21.15.16:0/1888916002 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f9b940a2b80 0x7f9b940a5040 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f9ba400ab60 tx=0x7f9ba400a2d0 comp rx=0 tx=0).stop 2024-09-12T02:00:05.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:05.541+0000 7f9baf989640 1 -- 172.21.15.16:0/1888916002 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ba8106a70 msgr2=0x7f9ba819ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:05.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:05.541+0000 7f9baf989640 1 --2- 172.21.15.16:0/1888916002 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ba8106a70 0x7f9ba819ccd0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f9ba0009fd0 tx=0x7f9ba000def0 comp rx=0 tx=0).stop 2024-09-12T02:00:05.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:05.541+0000 7f9baf989640 1 -- 172.21.15.16:0/1888916002 shutdown_connections 2024-09-12T02:00:05.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:05.541+0000 7f9baf989640 1 --2- 172.21.15.16:0/1888916002 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f9b940a2b80 0x7f9b940a5040 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:05.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:05.541+0000 7f9baf989640 1 --2- 172.21.15.16:0/1888916002 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ba8106a70 0x7f9ba819ccd0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:05.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:05.541+0000 7f9baf989640 1 --2- 172.21.15.16:0/1888916002 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9ba8101410 0x7f9ba8198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:05.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:05.541+0000 7f9baf989640 1 --2- 172.21.15.16:0/1888916002 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ba8100a70 0x7f9ba8198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:05.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:05.541+0000 7f9baf989640 1 -- 172.21.15.16:0/1888916002 >> 172.21.15.16:0/1888916002 conn(0x7f9ba80fc790 msgr2=0x7f9ba8104840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:00:05.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:05.542+0000 7f9baf989640 1 -- 172.21.15.16:0/1888916002 shutdown_connections 2024-09-12T02:00:05.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:05.542+0000 7f9baf989640 1 -- 172.21.15.16:0/1888916002 wait complete. 2024-09-12T02:00:05.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:00:05 smithi016 ceph-mon[30947]: pgmap v278: 97 pgs: 97 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-09-12T02:00:05.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:00:05 smithi016 ceph-mon[30947]: mds.? [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] up:active 2024-09-12T02:00:05.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:00:05 smithi016 ceph-mon[30947]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-12T02:00:05.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:00:05 smithi016 ceph-mon[30947]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-12T02:00:05.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:00:05 smithi016 ceph-mon[30947]: Cluster is now healthy 2024-09-12T02:00:05.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:00:05 smithi016 ceph-mon[30947]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-09-12T02:00:05.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:00:05 smithi016 ceph-mon[30947]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2024-09-12T02:00:05.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:00:05 smithi016 ceph-mon[30947]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-09-12T02:00:05.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:00:05 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/1888916002' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-12T02:00:05.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:00:05 smithi184 ceph-mon[35593]: pgmap v278: 97 pgs: 97 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-09-12T02:00:05.878 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:00:05 smithi184 ceph-mon[35593]: mds.? [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] up:active 2024-09-12T02:00:05.878 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:00:05 smithi184 ceph-mon[35593]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-12T02:00:05.878 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:00:05 smithi184 ceph-mon[35593]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-12T02:00:05.878 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:00:05 smithi184 ceph-mon[35593]: Cluster is now healthy 2024-09-12T02:00:05.878 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:00:05 smithi184 ceph-mon[35593]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-09-12T02:00:05.878 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:00:05 smithi184 ceph-mon[35593]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2024-09-12T02:00:05.878 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:00:05 smithi184 ceph-mon[35593]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-09-12T02:00:05.878 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:00:05 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/1888916002' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-12T02:00:05.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:00:05 smithi033 ceph-mon[35332]: pgmap v278: 97 pgs: 97 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-09-12T02:00:05.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:00:05 smithi033 ceph-mon[35332]: mds.? [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] up:active 2024-09-12T02:00:05.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:00:05 smithi033 ceph-mon[35332]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-12T02:00:05.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:00:05 smithi033 ceph-mon[35332]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-12T02:00:05.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:00:05 smithi033 ceph-mon[35332]: Cluster is now healthy 2024-09-12T02:00:05.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:00:05 smithi033 ceph-mon[35332]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-09-12T02:00:05.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:00:05 smithi033 ceph-mon[35332]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2024-09-12T02:00:05.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:00:05 smithi033 ceph-mon[35332]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-09-12T02:00:05.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:00:05 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/1888916002' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-12T02:00:06.163 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-09-12T02:00:06.424 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:00:06.792 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:00:06 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/1888916002' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-12T02:00:06.793 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:00:06 smithi016 ceph-mon[30947]: mds.? [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] up:active 2024-09-12T02:00:06.793 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:00:06 smithi016 ceph-mon[30947]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 9 up:standby 2024-09-12T02:00:06.793 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:00:06 smithi016 ceph-mon[30947]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T02:00:06.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:00:06 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/1888916002' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-12T02:00:06.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:00:06 smithi184 ceph-mon[35593]: mds.? [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] up:active 2024-09-12T02:00:06.878 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:00:06 smithi184 ceph-mon[35593]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 9 up:standby 2024-09-12T02:00:06.878 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:00:06 smithi184 ceph-mon[35593]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T02:00:06.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:06.875+0000 7fb2d2121640 1 -- 172.21.15.16:0/2004865383 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb2cc10c180 msgr2=0x7fb2cc10e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:06.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:06.875+0000 7fb2d2121640 1 --2- 172.21.15.16:0/2004865383 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb2cc10c180 0x7fb2cc10e600 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb2c4009ca0 tx=0x7fb2c402a950 comp rx=0 tx=0).stop 2024-09-12T02:00:06.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:06.875+0000 7fb2d2121640 1 -- 172.21.15.16:0/2004865383 shutdown_connections 2024-09-12T02:00:06.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:06.875+0000 7fb2d2121640 1 --2- 172.21.15.16:0/2004865383 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb2cc10c180 0x7fb2cc10e600 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:06.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:06.875+0000 7fb2d2121640 1 --2- 172.21.15.16:0/2004865383 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2cc1036c0 0x7fb2cc10bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:06.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:06.875+0000 7fb2d2121640 1 --2- 172.21.15.16:0/2004865383 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb2cc102da0 0x7fb2cc103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:06.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:06.875+0000 7fb2d2121640 1 -- 172.21.15.16:0/2004865383 >> 172.21.15.16:0/2004865383 conn(0x7fb2cc0fc7a0 msgr2=0x7fb2cc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:00:06.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:06.875+0000 7fb2d2121640 1 -- 172.21.15.16:0/2004865383 shutdown_connections 2024-09-12T02:00:06.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:06.875+0000 7fb2d2121640 1 -- 172.21.15.16:0/2004865383 wait complete. 2024-09-12T02:00:06.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:06.875+0000 7fb2d2121640 1 Processor -- start 2024-09-12T02:00:06.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:06.876+0000 7fb2d2121640 1 -- start start 2024-09-12T02:00:06.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:06.876+0000 7fb2d2121640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb2cc102da0 0x7fb2cc100a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:06.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:06.876+0000 7fb2d2121640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb2cc1036c0 0x7fb2cc100f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:06.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:06.876+0000 7fb2d2121640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2cc10c180 0x7fb2cc101480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:06.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:06.876+0000 7fb2d2121640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb2cc111f60 con 0x7fb2cc10c180 2024-09-12T02:00:06.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:06.876+0000 7fb2d2121640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb2cc111de0 con 0x7fb2cc1036c0 2024-09-12T02:00:06.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:06.876+0000 7fb2d2121640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb2cc1120e0 con 0x7fb2cc102da0 2024-09-12T02:00:06.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:06.876+0000 7fb2cbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2cc10c180 0x7fb2cc101480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:06.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:06.876+0000 7fb2cbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2cc10c180 0x7fb2cc101480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45080/0 (socket says 172.21.15.16:45080) 2024-09-12T02:00:06.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:06.876+0000 7fb2cbfff640 1 -- 172.21.15.16:0/741534396 learned_addr learned my addr 172.21.15.16:0/741534396 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:00:06.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:06.876+0000 7fb2cb7fe640 1 --2- 172.21.15.16:0/741534396 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb2cc102da0 0x7fb2cc100a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:06.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:06.876+0000 7fb2caffd640 1 --2- 172.21.15.16:0/741534396 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb2cc1036c0 0x7fb2cc100f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:06.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:06.876+0000 7fb2cbfff640 1 -- 172.21.15.16:0/741534396 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb2cc102da0 msgr2=0x7fb2cc100a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:06.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:06.876+0000 7fb2cbfff640 1 --2- 172.21.15.16:0/741534396 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb2cc102da0 0x7fb2cc100a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:06.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:06.876+0000 7fb2cbfff640 1 -- 172.21.15.16:0/741534396 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb2cc1036c0 msgr2=0x7fb2cc100f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:06.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:06.876+0000 7fb2cbfff640 1 --2- 172.21.15.16:0/741534396 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb2cc1036c0 0x7fb2cc100f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:06.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:06.876+0000 7fb2cbfff640 1 -- 172.21.15.16:0/741534396 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2cc1a6880 con 0x7fb2cc10c180 2024-09-12T02:00:06.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:06.876+0000 7fb2cb7fe640 1 --2- 172.21.15.16:0/741534396 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb2cc102da0 0x7fb2cc100a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:00:06.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:06.876+0000 7fb2caffd640 1 --2- 172.21.15.16:0/741534396 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb2cc1036c0 0x7fb2cc100f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:00:06.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:06.877+0000 7fb2cbfff640 1 --2- 172.21.15.16:0/741534396 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2cc10c180 0x7fb2cc101480 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fb2c4004af0 tx=0x7fb2c402f3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:00:06.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:06.877+0000 7fb2c8ff9640 1 -- 172.21.15.16:0/741534396 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2c4034040 con 0x7fb2cc10c180 2024-09-12T02:00:06.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:06.877+0000 7fb2d2121640 1 -- 172.21.15.16:0/741534396 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2cc1a6b10 con 0x7fb2cc10c180 2024-09-12T02:00:06.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:06.877+0000 7fb2c8ff9640 1 -- 172.21.15.16:0/741534396 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb2c4030920 con 0x7fb2cc10c180 2024-09-12T02:00:06.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:06.877+0000 7fb2c8ff9640 1 -- 172.21.15.16:0/741534396 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2c4004670 con 0x7fb2cc10c180 2024-09-12T02:00:06.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:06.877+0000 7fb2d2121640 1 -- 172.21.15.16:0/741534396 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2cc1a6ff0 con 0x7fb2cc10c180 2024-09-12T02:00:06.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:06.878+0000 7fb2d2121640 1 -- 172.21.15.16:0/741534396 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2cc103180 con 0x7fb2cc10c180 2024-09-12T02:00:06.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:06.879+0000 7fb2c8ff9640 1 -- 172.21.15.16:0/741534396 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fb2c4037020 con 0x7fb2cc10c180 2024-09-12T02:00:06.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:06.880+0000 7fb2c8ff9640 1 --2- 172.21.15.16:0/741534396 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fb2b40a2b80 0x7fb2b40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:06.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:06.880+0000 7fb2cb7fe640 1 --2- 172.21.15.16:0/741534396 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fb2b40a2b80 0x7fb2b40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:06.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:06.880+0000 7fb2c8ff9640 1 -- 172.21.15.16:0/741534396 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7fb2c40efd10 con 0x7fb2cc10c180 2024-09-12T02:00:06.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:06.880+0000 7fb2cb7fe640 1 --2- 172.21.15.16:0/741534396 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fb2b40a2b80 0x7fb2b40a5040 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc009b50 tx=0x7fb2bc005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:00:06.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:06.884+0000 7fb2c8ff9640 1 -- 172.21.15.16:0/741534396 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb2c40bcdc0 con 0x7fb2cc10c180 2024-09-12T02:00:06.918 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:00:06 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/1888916002' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-12T02:00:06.918 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:00:06 smithi033 ceph-mon[35332]: mds.? [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] up:active 2024-09-12T02:00:06.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:00:06 smithi033 ceph-mon[35332]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 9 up:standby 2024-09-12T02:00:06.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:00:06 smithi033 ceph-mon[35332]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T02:00:07.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:07.096+0000 7fb2d2121640 1 -- 172.21.15.16:0/741534396 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7fb2cc107250 con 0x7fb2cc10c180 2024-09-12T02:00:07.419 INFO:journalctl@ceph.mds.b.smithi033.stdout:Sep 12 02:00:07 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mds-b[65151]: 2024-09-12T02:00:07.051+0000 7fe773e15640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-09-12T02:00:07.419 INFO:journalctl@ceph.mds.b.smithi033.stdout:Sep 12 02:00:07 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mds-b[65151]: 2024-09-12T02:00:07.051+0000 7fe773e15640 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-09-12T02:00:07.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:07.553+0000 7fb2c8ff9640 1 -- 172.21.15.16:0/741534396 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v24) ==== 115+0+0 (secure 0 0 0) 0x7fb2c40bc6c0 con 0x7fb2cc10c180 2024-09-12T02:00:07.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:07.557+0000 7fb2d2121640 1 -- 172.21.15.16:0/741534396 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fb2b40a2b80 msgr2=0x7fb2b40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:07.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:07.557+0000 7fb2d2121640 1 --2- 172.21.15.16:0/741534396 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fb2b40a2b80 0x7fb2b40a5040 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc009b50 tx=0x7fb2bc005f90 comp rx=0 tx=0).stop 2024-09-12T02:00:07.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:07.557+0000 7fb2d2121640 1 -- 172.21.15.16:0/741534396 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2cc10c180 msgr2=0x7fb2cc101480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:07.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:07.557+0000 7fb2d2121640 1 --2- 172.21.15.16:0/741534396 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2cc10c180 0x7fb2cc101480 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fb2c4004af0 tx=0x7fb2c402f3a0 comp rx=0 tx=0).stop 2024-09-12T02:00:07.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:07.557+0000 7fb2d2121640 1 -- 172.21.15.16:0/741534396 shutdown_connections 2024-09-12T02:00:07.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:07.557+0000 7fb2d2121640 1 --2- 172.21.15.16:0/741534396 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fb2b40a2b80 0x7fb2b40a5040 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:07.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:07.557+0000 7fb2d2121640 1 --2- 172.21.15.16:0/741534396 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2cc10c180 0x7fb2cc101480 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:07.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:07.557+0000 7fb2d2121640 1 --2- 172.21.15.16:0/741534396 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb2cc1036c0 0x7fb2cc100f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:07.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:07.557+0000 7fb2d2121640 1 --2- 172.21.15.16:0/741534396 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb2cc102da0 0x7fb2cc100a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:07.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:07.557+0000 7fb2d2121640 1 -- 172.21.15.16:0/741534396 >> 172.21.15.16:0/741534396 conn(0x7fb2cc0fc7a0 msgr2=0x7fb2cc0fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:00:07.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:07.557+0000 7fb2d2121640 1 -- 172.21.15.16:0/741534396 shutdown_connections 2024-09-12T02:00:07.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:07.557+0000 7fb2d2121640 1 -- 172.21.15.16:0/741534396 wait complete. 2024-09-12T02:00:07.874 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:00:07 smithi016 ceph-mon[30947]: pgmap v279: 97 pgs: 97 active+clean; 582 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 614 B/s rd, 2.9 KiB/s wr, 11 op/s 2024-09-12T02:00:07.874 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:00:07 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/741534396' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-12T02:00:07.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:00:07 smithi184 ceph-mon[35593]: pgmap v279: 97 pgs: 97 active+clean; 582 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 614 B/s rd, 2.9 KiB/s wr, 11 op/s 2024-09-12T02:00:07.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:00:07 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/741534396' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-12T02:00:07.918 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:00:07 smithi033 ceph-mon[35332]: pgmap v279: 97 pgs: 97 active+clean; 582 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 614 B/s rd, 2.9 KiB/s wr, 11 op/s 2024-09-12T02:00:07.918 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:00:07 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/741534396' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-12T02:00:08.201 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated'} subvols for filesystem 'cephfs' 2024-09-12T02:00:08.202 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-09-12T02:00:08.467 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:00:08.871 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:00:08 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/741534396' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-12T02:00:08.871 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:00:08 smithi016 ceph-mon[30947]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T02:00:08.871 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:00:08 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]: dispatch 2024-09-12T02:00:08.871 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:00:08 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]: dispatch 2024-09-12T02:00:08.871 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:00:08 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]: dispatch 2024-09-12T02:00:08.871 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:00:08 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2024-09-12T02:00:08.871 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:00:08 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]: dispatch 2024-09-12T02:00:08.871 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:00:08 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2024-09-12T02:00:08.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:00:08 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/741534396' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-12T02:00:08.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:00:08 smithi184 ceph-mon[35593]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T02:00:08.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:00:08 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]: dispatch 2024-09-12T02:00:08.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:00:08 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]: dispatch 2024-09-12T02:00:08.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:00:08 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]: dispatch 2024-09-12T02:00:08.878 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:00:08 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2024-09-12T02:00:08.878 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:00:08 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]: dispatch 2024-09-12T02:00:08.878 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:00:08 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2024-09-12T02:00:08.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:00:08 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/741534396' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-12T02:00:08.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:00:08 smithi033 ceph-mon[35332]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T02:00:08.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:00:08 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]: dispatch 2024-09-12T02:00:08.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:00:08 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]: dispatch 2024-09-12T02:00:08.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:00:08 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]: dispatch 2024-09-12T02:00:08.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:00:08 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2024-09-12T02:00:08.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:00:08 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]: dispatch 2024-09-12T02:00:08.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:00:08 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2024-09-12T02:00:08.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:08.955+0000 7fc6fc0bc640 1 -- 172.21.15.16:0/423670997 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6f4101de0 msgr2=0x7fc6f41021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:08.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:08.955+0000 7fc6fc0bc640 1 --2- 172.21.15.16:0/423670997 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6f4101de0 0x7fc6f41021c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fc6f0009900 tx=0x7fc6f002a410 comp rx=0 tx=0).stop 2024-09-12T02:00:08.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:08.955+0000 7fc6fc0bc640 1 -- 172.21.15.16:0/423670997 shutdown_connections 2024-09-12T02:00:08.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:08.955+0000 7fc6fc0bc640 1 --2- 172.21.15.16:0/423670997 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc6f4106f10 0x7fc6f410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:08.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:08.955+0000 7fc6fc0bc640 1 --2- 172.21.15.16:0/423670997 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc6f4102790 0x7fc6f41067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:08.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:08.955+0000 7fc6fc0bc640 1 --2- 172.21.15.16:0/423670997 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6f4101de0 0x7fc6f41021c0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:08.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:08.955+0000 7fc6fc0bc640 1 -- 172.21.15.16:0/423670997 >> 172.21.15.16:0/423670997 conn(0x7fc6f40fd8c0 msgr2=0x7fc6f40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:00:08.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:08.955+0000 7fc6fc0bc640 1 -- 172.21.15.16:0/423670997 shutdown_connections 2024-09-12T02:00:08.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:08.955+0000 7fc6fc0bc640 1 -- 172.21.15.16:0/423670997 wait complete. 2024-09-12T02:00:08.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:08.956+0000 7fc6fc0bc640 1 Processor -- start 2024-09-12T02:00:08.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:08.956+0000 7fc6fc0bc640 1 -- start start 2024-09-12T02:00:08.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:08.956+0000 7fc6fc0bc640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc6f4101de0 0x7fc6f4198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:08.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:08.956+0000 7fc6fc0bc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6f4102790 0x7fc6f41989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:08.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:08.956+0000 7fc6fc0bc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc6f4106f10 0x7fc6f419cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:08.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:08.956+0000 7fc6fc0bc640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc6f4111fc0 con 0x7fc6f4102790 2024-09-12T02:00:08.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:08.956+0000 7fc6fc0bc640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc6f4111e40 con 0x7fc6f4106f10 2024-09-12T02:00:08.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:08.956+0000 7fc6fc0bc640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc6f4112140 con 0x7fc6f4101de0 2024-09-12T02:00:08.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:08.956+0000 7fc6f9657640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6f4102790 0x7fc6f41989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:08.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:08.956+0000 7fc6f9657640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6f4102790 0x7fc6f41989a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45108/0 (socket says 172.21.15.16:45108) 2024-09-12T02:00:08.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:08.956+0000 7fc6f9657640 1 -- 172.21.15.16:0/2942067144 learned_addr learned my addr 172.21.15.16:0/2942067144 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:00:08.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:08.956+0000 7fc6fa659640 1 --2- 172.21.15.16:0/2942067144 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc6f4106f10 0x7fc6f419cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:08.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:08.956+0000 7fc6f9e58640 1 --2- 172.21.15.16:0/2942067144 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc6f4101de0 0x7fc6f4198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:08.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:08.957+0000 7fc6f9657640 1 -- 172.21.15.16:0/2942067144 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc6f4101de0 msgr2=0x7fc6f4198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:08.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:08.957+0000 7fc6f9657640 1 --2- 172.21.15.16:0/2942067144 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc6f4101de0 0x7fc6f4198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:08.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:08.957+0000 7fc6f9657640 1 -- 172.21.15.16:0/2942067144 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc6f4106f10 msgr2=0x7fc6f419cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:08.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:08.957+0000 7fc6f9657640 1 --2- 172.21.15.16:0/2942067144 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc6f4106f10 0x7fc6f419cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:08.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:08.957+0000 7fc6f9657640 1 -- 172.21.15.16:0/2942067144 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6f419d410 con 0x7fc6f4102790 2024-09-12T02:00:08.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:08.957+0000 7fc6f9e58640 1 --2- 172.21.15.16:0/2942067144 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc6f4101de0 0x7fc6f4198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:00:08.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:08.957+0000 7fc6f9657640 1 --2- 172.21.15.16:0/2942067144 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6f4102790 0x7fc6f41989a0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc6e400c970 tx=0x7fc6e400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:00:08.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:08.957+0000 7fc6eaffd640 1 -- 172.21.15.16:0/2942067144 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6e4007bb0 con 0x7fc6f4102790 2024-09-12T02:00:08.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:08.957+0000 7fc6eaffd640 1 -- 172.21.15.16:0/2942067144 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc6e4007d50 con 0x7fc6f4102790 2024-09-12T02:00:08.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:08.957+0000 7fc6fc0bc640 1 -- 172.21.15.16:0/2942067144 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6f419d700 con 0x7fc6f4102790 2024-09-12T02:00:08.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:08.957+0000 7fc6eaffd640 1 -- 172.21.15.16:0/2942067144 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6e400f450 con 0x7fc6f4102790 2024-09-12T02:00:08.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:08.957+0000 7fc6fc0bc640 1 -- 172.21.15.16:0/2942067144 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6f41a4fe0 con 0x7fc6f4102790 2024-09-12T02:00:08.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:08.958+0000 7fc6fc0bc640 1 -- 172.21.15.16:0/2942067144 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6f41021c0 con 0x7fc6f4102790 2024-09-12T02:00:08.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:08.959+0000 7fc6eaffd640 1 -- 172.21.15.16:0/2942067144 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fc6e4005020 con 0x7fc6f4102790 2024-09-12T02:00:08.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:08.960+0000 7fc6eaffd640 1 --2- 172.21.15.16:0/2942067144 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc6e00a2c00 0x7fc6e00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:08.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:08.960+0000 7fc6f9e58640 1 --2- 172.21.15.16:0/2942067144 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc6e00a2c00 0x7fc6e00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:08.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:08.960+0000 7fc6eaffd640 1 -- 172.21.15.16:0/2942067144 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8153+0+0 (secure 0 0 0) 0x7fc6e40ced00 con 0x7fc6f4102790 2024-09-12T02:00:08.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:08.960+0000 7fc6f9e58640 1 --2- 172.21.15.16:0/2942067144 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc6e00a2c00 0x7fc6e00a50c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fc6f0005ab0 tx=0x7fc6f0000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:00:08.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:08.964+0000 7fc6eaffd640 1 -- 172.21.15.16:0/2942067144 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc6e4014030 con 0x7fc6f4102790 2024-09-12T02:00:09.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:09.168+0000 7fc6fc0bc640 1 -- 172.21.15.16:0/2942067144 --> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fc6f4062630 con 0x7fc6e00a2c00 2024-09-12T02:00:09.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:09.193+0000 7fc6eaffd640 1 -- 172.21.15.16:0/2942067144 <== mgr.14150 v2:172.21.15.16:6800/3887444820 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fc6f4062630 con 0x7fc6e00a2c00 2024-09-12T02:00:09.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:09.196+0000 7fc6fc0bc640 1 -- 172.21.15.16:0/2942067144 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc6e00a2c00 msgr2=0x7fc6e00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:09.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:09.196+0000 7fc6fc0bc640 1 --2- 172.21.15.16:0/2942067144 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc6e00a2c00 0x7fc6e00a50c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fc6f0005ab0 tx=0x7fc6f0000bc0 comp rx=0 tx=0).stop 2024-09-12T02:00:09.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:09.196+0000 7fc6fc0bc640 1 -- 172.21.15.16:0/2942067144 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6f4102790 msgr2=0x7fc6f41989a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:09.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:09.196+0000 7fc6fc0bc640 1 --2- 172.21.15.16:0/2942067144 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6f4102790 0x7fc6f41989a0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc6e400c970 tx=0x7fc6e400ce30 comp rx=0 tx=0).stop 2024-09-12T02:00:09.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:09.196+0000 7fc6fc0bc640 1 -- 172.21.15.16:0/2942067144 shutdown_connections 2024-09-12T02:00:09.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:09.196+0000 7fc6fc0bc640 1 --2- 172.21.15.16:0/2942067144 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc6e00a2c00 0x7fc6e00a50c0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:09.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:09.196+0000 7fc6fc0bc640 1 --2- 172.21.15.16:0/2942067144 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc6f4106f10 0x7fc6f419cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:09.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:09.196+0000 7fc6fc0bc640 1 --2- 172.21.15.16:0/2942067144 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6f4102790 0x7fc6f41989a0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:09.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:09.196+0000 7fc6fc0bc640 1 --2- 172.21.15.16:0/2942067144 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc6f4101de0 0x7fc6f4198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:09.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:09.196+0000 7fc6fc0bc640 1 -- 172.21.15.16:0/2942067144 >> 172.21.15.16:0/2942067144 conn(0x7fc6f40fd8c0 msgr2=0x7fc6f40febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:00:09.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:09.196+0000 7fc6fc0bc640 1 -- 172.21.15.16:0/2942067144 shutdown_connections 2024-09-12T02:00:09.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:09.196+0000 7fc6fc0bc640 1 -- 172.21.15.16:0/2942067144 wait complete. 2024-09-12T02:00:09.434 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 02:00:09 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-x[31173]: 2024-09-12T02:00:09.173+0000 7f18e8f1b640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T02:00:09.434 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 02:00:09 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-x[31173]: 2024-09-12T02:00:09.173+0000 7f18e8f1b640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T02:00:09.434 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 02:00:09 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-x[31173]: 2024-09-12T02:00:09.173+0000 7f18e8f1b640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T02:00:09.434 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 02:00:09 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-x[31173]: 2024-09-12T02:00:09.173+0000 7f18e8f1b640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T02:00:09.434 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 02:00:09 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-x[31173]: 2024-09-12T02:00:09.173+0000 7f18e8f1b640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T02:00:09.800 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:00:09 smithi016 ceph-mon[30947]: pgmap v280: 97 pgs: 97 active+clean; 582 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 953 B/s rd, 3.2 KiB/s wr, 11 op/s 2024-09-12T02:00:09.800 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:00:09 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]': finished 2024-09-12T02:00:09.800 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:00:09 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]': finished 2024-09-12T02:00:09.800 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:00:09 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]': finished 2024-09-12T02:00:09.800 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:00:09 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2024-09-12T02:00:09.800 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:00:09 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]': finished 2024-09-12T02:00:09.800 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:00:09 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2024-09-12T02:00:09.800 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:00:09 smithi016 ceph-mon[30947]: osdmap e81: 12 total, 12 up, 12 in 2024-09-12T02:00:09.801 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:00:09 smithi016 ceph-mon[30947]: from='client.14691 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:00:09.829 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-09-12T02:00:09.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:00:09 smithi184 ceph-mon[35593]: pgmap v280: 97 pgs: 97 active+clean; 582 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 953 B/s rd, 3.2 KiB/s wr, 11 op/s 2024-09-12T02:00:09.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:00:09 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]': finished 2024-09-12T02:00:09.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:00:09 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]': finished 2024-09-12T02:00:09.878 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:00:09 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]': finished 2024-09-12T02:00:09.878 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:00:09 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2024-09-12T02:00:09.878 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:00:09 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]': finished 2024-09-12T02:00:09.878 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:00:09 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2024-09-12T02:00:09.878 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:00:09 smithi184 ceph-mon[35593]: osdmap e81: 12 total, 12 up, 12 in 2024-09-12T02:00:09.878 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:00:09 smithi184 ceph-mon[35593]: from='client.14691 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:00:09.918 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:00:09 smithi033 ceph-mon[35332]: pgmap v280: 97 pgs: 97 active+clean; 582 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 953 B/s rd, 3.2 KiB/s wr, 11 op/s 2024-09-12T02:00:09.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:00:09 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]': finished 2024-09-12T02:00:09.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:00:09 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]': finished 2024-09-12T02:00:09.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:00:09 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]': finished 2024-09-12T02:00:09.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:00:09 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2024-09-12T02:00:09.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:00:09 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]': finished 2024-09-12T02:00:09.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:00:09 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2024-09-12T02:00:09.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:00:09 smithi033 ceph-mon[35332]: osdmap e81: 12 total, 12 up, 12 in 2024-09-12T02:00:09.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:00:09 smithi033 ceph-mon[35332]: from='client.14691 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:00:10.089 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:00:10.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:10.566+0000 7f9decf97640 1 -- 172.21.15.16:0/2050340703 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9de8102790 msgr2=0x7f9de81067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:10.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:10.566+0000 7f9decf97640 1 --2- 172.21.15.16:0/2050340703 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9de8102790 0x7f9de81067e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f9dd0009900 tx=0x7f9dd002a410 comp rx=0 tx=0).stop 2024-09-12T02:00:10.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:10.566+0000 7f9decf97640 1 -- 172.21.15.16:0/2050340703 shutdown_connections 2024-09-12T02:00:10.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:10.566+0000 7f9decf97640 1 --2- 172.21.15.16:0/2050340703 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9de8106f10 0x7f9de810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:10.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:10.566+0000 7f9decf97640 1 --2- 172.21.15.16:0/2050340703 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9de8102790 0x7f9de81067e0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:10.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:10.566+0000 7f9decf97640 1 --2- 172.21.15.16:0/2050340703 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9de8101de0 0x7f9de81021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:10.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:10.566+0000 7f9decf97640 1 -- 172.21.15.16:0/2050340703 >> 172.21.15.16:0/2050340703 conn(0x7f9de80fd8e0 msgr2=0x7f9de80ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:00:10.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:10.566+0000 7f9decf97640 1 -- 172.21.15.16:0/2050340703 shutdown_connections 2024-09-12T02:00:10.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:10.566+0000 7f9decf97640 1 -- 172.21.15.16:0/2050340703 wait complete. 2024-09-12T02:00:10.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:10.567+0000 7f9decf97640 1 Processor -- start 2024-09-12T02:00:10.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:10.567+0000 7f9decf97640 1 -- start start 2024-09-12T02:00:10.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:10.567+0000 7f9decf97640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9de8101de0 0x7f9de8198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:10.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:10.567+0000 7f9decf97640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9de8102790 0x7f9de8198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:10.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:10.567+0000 7f9decf97640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9de8106f10 0x7f9de819cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:10.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:10.567+0000 7f9decf97640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9de8111f30 con 0x7f9de8102790 2024-09-12T02:00:10.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:10.567+0000 7f9decf97640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9de8111db0 con 0x7f9de8106f10 2024-09-12T02:00:10.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:10.567+0000 7f9decf97640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f9de81120b0 con 0x7f9de8101de0 2024-09-12T02:00:10.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:10.567+0000 7f9de5d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9de8102790 0x7f9de8198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:10.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:10.567+0000 7f9de5d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9de8102790 0x7f9de8198990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45164/0 (socket says 172.21.15.16:45164) 2024-09-12T02:00:10.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:10.567+0000 7f9de5d9b640 1 -- 172.21.15.16:0/2171381968 learned_addr learned my addr 172.21.15.16:0/2171381968 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:00:10.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:10.567+0000 7f9de659c640 1 --2- 172.21.15.16:0/2171381968 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9de8101de0 0x7f9de8198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:10.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:10.567+0000 7f9de6d9d640 1 --2- 172.21.15.16:0/2171381968 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9de8106f10 0x7f9de819cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:10.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:10.567+0000 7f9de5d9b640 1 -- 172.21.15.16:0/2171381968 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9de8101de0 msgr2=0x7f9de8198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:10.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:10.567+0000 7f9de5d9b640 1 --2- 172.21.15.16:0/2171381968 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9de8101de0 0x7f9de8198450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:10.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:10.567+0000 7f9de5d9b640 1 -- 172.21.15.16:0/2171381968 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9de8106f10 msgr2=0x7f9de819cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:10.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:10.567+0000 7f9de5d9b640 1 --2- 172.21.15.16:0/2171381968 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9de8106f10 0x7f9de819cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:10.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:10.567+0000 7f9de5d9b640 1 -- 172.21.15.16:0/2171381968 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9de819d400 con 0x7f9de8102790 2024-09-12T02:00:10.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:10.567+0000 7f9de659c640 1 --2- 172.21.15.16:0/2171381968 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9de8101de0 0x7f9de8198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:00:10.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:10.567+0000 7f9de6d9d640 1 --2- 172.21.15.16:0/2171381968 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9de8106f10 0x7f9de819cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:00:10.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:10.568+0000 7f9de5d9b640 1 --2- 172.21.15.16:0/2171381968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9de8102790 0x7f9de8198990 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f9dd0006010 tx=0x7f9dd00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:00:10.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:10.568+0000 7f9ddf7fe640 1 -- 172.21.15.16:0/2171381968 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dd003f030 con 0x7f9de8102790 2024-09-12T02:00:10.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:10.568+0000 7f9ddf7fe640 1 -- 172.21.15.16:0/2171381968 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9dd0003ce0 con 0x7f9de8102790 2024-09-12T02:00:10.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:10.568+0000 7f9decf97640 1 -- 172.21.15.16:0/2171381968 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9de819d690 con 0x7f9de8102790 2024-09-12T02:00:10.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:10.568+0000 7f9ddf7fe640 1 -- 172.21.15.16:0/2171381968 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dd0035070 con 0x7f9de8102790 2024-09-12T02:00:10.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:10.568+0000 7f9decf97640 1 -- 172.21.15.16:0/2171381968 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9de81a4fd0 con 0x7f9de8102790 2024-09-12T02:00:10.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:10.569+0000 7f9decf97640 1 -- 172.21.15.16:0/2171381968 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9de81021c0 con 0x7f9de8102790 2024-09-12T02:00:10.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:10.570+0000 7f9ddf7fe640 1 -- 172.21.15.16:0/2171381968 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f9dd0042030 con 0x7f9de8102790 2024-09-12T02:00:10.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:10.571+0000 7f9ddf7fe640 1 --2- 172.21.15.16:0/2171381968 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f9dd40a2b80 0x7f9dd40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:10.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:10.571+0000 7f9de659c640 1 --2- 172.21.15.16:0/2171381968 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f9dd40a2b80 0x7f9dd40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:10.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:10.571+0000 7f9ddf7fe640 1 -- 172.21.15.16:0/2171381968 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f9dd00ef970 con 0x7f9de8102790 2024-09-12T02:00:10.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:10.572+0000 7f9de659c640 1 --2- 172.21.15.16:0/2171381968 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f9dd40a2b80 0x7f9dd40a5040 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f9dd8005d00 tx=0x7f9dd8009550 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:00:10.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:10.574+0000 7f9ddf7fe640 1 -- 172.21.15.16:0/2171381968 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9dd00bca30 con 0x7f9de8102790 2024-09-12T02:00:10.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:10.592+0000 7f9ddf7fe640 1 -- 172.21.15.16:0/2171381968 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f9dd00b89a0 con 0x7f9de8102790 2024-09-12T02:00:10.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:10.783+0000 7f9decf97640 1 -- 172.21.15.16:0/2171381968 --> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f9de8062630 con 0x7f9dd40a2b80 2024-09-12T02:00:10.783 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:00:10 smithi016 ceph-mon[30947]: osdmap e82: 12 total, 12 up, 12 in 2024-09-12T02:00:10.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:10.807+0000 7f9ddf7fe640 1 -- 172.21.15.16:0/2171381968 <== mgr.14150 v2:172.21.15.16:6800/3887444820 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f9de8062630 con 0x7f9dd40a2b80 2024-09-12T02:00:10.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:10.809+0000 7f9decf97640 1 -- 172.21.15.16:0/2171381968 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f9dd40a2b80 msgr2=0x7f9dd40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:10.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:10.809+0000 7f9decf97640 1 --2- 172.21.15.16:0/2171381968 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f9dd40a2b80 0x7f9dd40a5040 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f9dd8005d00 tx=0x7f9dd8009550 comp rx=0 tx=0).stop 2024-09-12T02:00:10.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:10.809+0000 7f9decf97640 1 -- 172.21.15.16:0/2171381968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9de8102790 msgr2=0x7f9de8198990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:10.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:10.809+0000 7f9decf97640 1 --2- 172.21.15.16:0/2171381968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9de8102790 0x7f9de8198990 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f9dd0006010 tx=0x7f9dd00042f0 comp rx=0 tx=0).stop 2024-09-12T02:00:10.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:10.809+0000 7f9decf97640 1 -- 172.21.15.16:0/2171381968 shutdown_connections 2024-09-12T02:00:10.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:10.809+0000 7f9decf97640 1 --2- 172.21.15.16:0/2171381968 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f9dd40a2b80 0x7f9dd40a5040 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:10.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:10.809+0000 7f9decf97640 1 --2- 172.21.15.16:0/2171381968 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9de8106f10 0x7f9de819cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:10.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:10.809+0000 7f9decf97640 1 --2- 172.21.15.16:0/2171381968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9de8102790 0x7f9de8198990 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:10.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:10.809+0000 7f9decf97640 1 --2- 172.21.15.16:0/2171381968 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9de8101de0 0x7f9de8198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:10.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:10.809+0000 7f9decf97640 1 -- 172.21.15.16:0/2171381968 >> 172.21.15.16:0/2171381968 conn(0x7f9de80fd8e0 msgr2=0x7f9de80fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:00:10.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:10.810+0000 7f9decf97640 1 -- 172.21.15.16:0/2171381968 shutdown_connections 2024-09-12T02:00:10.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:10.810+0000 7f9decf97640 1 -- 172.21.15.16:0/2171381968 wait complete. 2024-09-12T02:00:10.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:00:10 smithi184 ceph-mon[35593]: osdmap e82: 12 total, 12 up, 12 in 2024-09-12T02:00:10.918 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:00:10 smithi033 ceph-mon[35332]: osdmap e82: 12 total, 12 up, 12 in 2024-09-12T02:00:11.551 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-09-12T02:00:11.815 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:00:11.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:00:11 smithi184 ceph-mon[35593]: pgmap v283: 97 pgs: 1 peering, 96 active+clean; 582 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 3.1 KiB/s wr, 15 op/s 2024-09-12T02:00:11.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:00:11 smithi184 ceph-mon[35593]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-12T02:00:11.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:00:11 smithi184 ceph-mon[35593]: from='client.14703 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:00:11.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:00:11 smithi033 ceph-mon[35332]: pgmap v283: 97 pgs: 1 peering, 96 active+clean; 582 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 3.1 KiB/s wr, 15 op/s 2024-09-12T02:00:11.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:00:11 smithi033 ceph-mon[35332]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-12T02:00:11.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:00:11 smithi033 ceph-mon[35332]: from='client.14703 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:00:11.933 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:00:11 smithi016 ceph-mon[30947]: pgmap v283: 97 pgs: 1 peering, 96 active+clean; 582 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 3.1 KiB/s wr, 15 op/s 2024-09-12T02:00:11.933 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:00:11 smithi016 ceph-mon[30947]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-12T02:00:11.934 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:00:11 smithi016 ceph-mon[30947]: from='client.14703 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:00:12.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:12.297+0000 7fae205b3640 1 -- 172.21.15.16:0/3304931760 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae18100e20 msgr2=0x7fae18101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:12.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:12.297+0000 7fae205b3640 1 --2- 172.21.15.16:0/3304931760 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae18100e20 0x7fae18101200 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fae14009900 tx=0x7fae1402a480 comp rx=0 tx=0).stop 2024-09-12T02:00:12.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:12.297+0000 7fae205b3640 1 -- 172.21.15.16:0/3304931760 shutdown_connections 2024-09-12T02:00:12.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:12.297+0000 7fae205b3640 1 --2- 172.21.15.16:0/3304931760 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fae18102ef0 0x7fae1810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:12.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:12.297+0000 7fae205b3640 1 --2- 172.21.15.16:0/3304931760 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fae18102570 0x7fae181029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:12.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:12.297+0000 7fae205b3640 1 --2- 172.21.15.16:0/3304931760 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae18100e20 0x7fae18101200 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:12.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:12.297+0000 7fae205b3640 1 -- 172.21.15.16:0/3304931760 >> 172.21.15.16:0/3304931760 conn(0x7fae180fc7d0 msgr2=0x7fae180febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:00:12.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:12.297+0000 7fae205b3640 1 -- 172.21.15.16:0/3304931760 shutdown_connections 2024-09-12T02:00:12.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:12.297+0000 7fae205b3640 1 -- 172.21.15.16:0/3304931760 wait complete. 2024-09-12T02:00:12.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:12.298+0000 7fae205b3640 1 Processor -- start 2024-09-12T02:00:12.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:12.298+0000 7fae205b3640 1 -- start start 2024-09-12T02:00:12.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:12.298+0000 7fae205b3640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fae18100e20 0x7fae18199440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:12.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:12.298+0000 7fae205b3640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fae18102570 0x7fae18199980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:12.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:12.298+0000 7fae205b3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae18102ef0 0x7fae181a44c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:12.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:12.298+0000 7fae205b3640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fae18110fa0 con 0x7fae18102ef0 2024-09-12T02:00:12.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:12.298+0000 7fae205b3640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fae18110e20 con 0x7fae18100e20 2024-09-12T02:00:12.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:12.298+0000 7fae205b3640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fae18111120 con 0x7fae18102570 2024-09-12T02:00:12.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:12.298+0000 7fae1eb50640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae18102ef0 0x7fae181a44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:12.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:12.298+0000 7fae1eb50640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae18102ef0 0x7fae181a44c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50100/0 (socket says 172.21.15.16:50100) 2024-09-12T02:00:12.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:12.298+0000 7fae1eb50640 1 -- 172.21.15.16:0/2224234041 learned_addr learned my addr 172.21.15.16:0/2224234041 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:00:12.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:12.298+0000 7fae1e34f640 1 --2- 172.21.15.16:0/2224234041 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fae18100e20 0x7fae18199440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:12.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:12.298+0000 7fae1db4e640 1 --2- 172.21.15.16:0/2224234041 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fae18102570 0x7fae18199980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:12.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:12.298+0000 7fae1eb50640 1 -- 172.21.15.16:0/2224234041 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fae18102570 msgr2=0x7fae18199980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:12.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:12.298+0000 7fae1eb50640 1 --2- 172.21.15.16:0/2224234041 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fae18102570 0x7fae18199980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:12.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:12.298+0000 7fae1eb50640 1 -- 172.21.15.16:0/2224234041 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fae18100e20 msgr2=0x7fae18199440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:12.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:12.298+0000 7fae1eb50640 1 --2- 172.21.15.16:0/2224234041 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fae18100e20 0x7fae18199440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:12.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:12.298+0000 7fae1eb50640 1 -- 172.21.15.16:0/2224234041 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae181a4a00 con 0x7fae18102ef0 2024-09-12T02:00:12.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:12.299+0000 7fae1db4e640 1 --2- 172.21.15.16:0/2224234041 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fae18102570 0x7fae18199980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:00:12.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:12.299+0000 7fae1eb50640 1 --2- 172.21.15.16:0/2224234041 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae18102ef0 0x7fae181a44c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fae1000eaa0 tx=0x7fae1000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:00:12.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:12.299+0000 7fae0f7fe640 1 -- 172.21.15.16:0/2224234041 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae1000cdb0 con 0x7fae18102ef0 2024-09-12T02:00:12.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:12.299+0000 7fae205b3640 1 -- 172.21.15.16:0/2224234041 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae181a4bd0 con 0x7fae18102ef0 2024-09-12T02:00:12.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:12.299+0000 7fae0f7fe640 1 -- 172.21.15.16:0/2224234041 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae10005020 con 0x7fae18102ef0 2024-09-12T02:00:12.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:12.299+0000 7fae0f7fe640 1 -- 172.21.15.16:0/2224234041 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae100116b0 con 0x7fae18102ef0 2024-09-12T02:00:12.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:12.299+0000 7fae205b3640 1 -- 172.21.15.16:0/2224234041 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae181a5090 con 0x7fae18102ef0 2024-09-12T02:00:12.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:12.300+0000 7fae205b3640 1 -- 172.21.15.16:0/2224234041 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae18101200 con 0x7fae18102ef0 2024-09-12T02:00:12.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:12.301+0000 7fae0f7fe640 1 -- 172.21.15.16:0/2224234041 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fae10005560 con 0x7fae18102ef0 2024-09-12T02:00:12.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:12.302+0000 7fae0f7fe640 1 --2- 172.21.15.16:0/2224234041 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fae040a2c10 0x7fae040a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:12.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:12.302+0000 7fae1e34f640 1 --2- 172.21.15.16:0/2224234041 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fae040a2c10 0x7fae040a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:12.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:12.302+0000 7fae0f7fe640 1 -- 172.21.15.16:0/2224234041 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fae100cf420 con 0x7fae18102ef0 2024-09-12T02:00:12.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:12.302+0000 7fae1e34f640 1 --2- 172.21.15.16:0/2224234041 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fae040a2c10 0x7fae040a50d0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fae14009230 tx=0x7fae140058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:00:12.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:12.306+0000 7fae0f7fe640 1 -- 172.21.15.16:0/2224234041 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fae10016030 con 0x7fae18102ef0 2024-09-12T02:00:12.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:12.513+0000 7fae205b3640 1 -- 172.21.15.16:0/2224234041 --> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7fae18062630 con 0x7fae040a2c10 2024-09-12T02:00:12.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:12.518+0000 7fae0f7fe640 1 -- 172.21.15.16:0/2224234041 <== mgr.14150 v2:172.21.15.16:6800/3887444820 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fae18062630 con 0x7fae040a2c10 2024-09-12T02:00:12.518 INFO:teuthology.orchestra.run.smithi016.stdout:/volumes/qa/sv_0/1302ab5a-bb1b-4617-a967-04d9affaf74a 2024-09-12T02:00:12.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:12.520+0000 7fae205b3640 1 -- 172.21.15.16:0/2224234041 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fae040a2c10 msgr2=0x7fae040a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:12.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:12.520+0000 7fae205b3640 1 --2- 172.21.15.16:0/2224234041 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fae040a2c10 0x7fae040a50d0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fae14009230 tx=0x7fae140058d0 comp rx=0 tx=0).stop 2024-09-12T02:00:12.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:12.520+0000 7fae205b3640 1 -- 172.21.15.16:0/2224234041 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae18102ef0 msgr2=0x7fae181a44c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:12.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:12.520+0000 7fae205b3640 1 --2- 172.21.15.16:0/2224234041 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae18102ef0 0x7fae181a44c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fae1000eaa0 tx=0x7fae1000ef60 comp rx=0 tx=0).stop 2024-09-12T02:00:12.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:12.520+0000 7fae205b3640 1 -- 172.21.15.16:0/2224234041 shutdown_connections 2024-09-12T02:00:12.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:12.520+0000 7fae205b3640 1 --2- 172.21.15.16:0/2224234041 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fae040a2c10 0x7fae040a50d0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:12.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:12.520+0000 7fae205b3640 1 --2- 172.21.15.16:0/2224234041 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae18102ef0 0x7fae181a44c0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:12.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:12.520+0000 7fae205b3640 1 --2- 172.21.15.16:0/2224234041 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fae18102570 0x7fae18199980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:12.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:12.520+0000 7fae205b3640 1 --2- 172.21.15.16:0/2224234041 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fae18100e20 0x7fae18199440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:12.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:12.521+0000 7fae205b3640 1 -- 172.21.15.16:0/2224234041 >> 172.21.15.16:0/2224234041 conn(0x7fae180fc7d0 msgr2=0x7fae180febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:00:12.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:12.521+0000 7fae205b3640 1 -- 172.21.15.16:0/2224234041 shutdown_connections 2024-09-12T02:00:12.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:12.521+0000 7fae205b3640 1 -- 172.21.15.16:0/2224234041 wait complete. 2024-09-12T02:00:13.165 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-09-12T02:00:13.427 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:00:13.860 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:00:13 smithi016 ceph-mon[30947]: pgmap v284: 97 pgs: 1 peering, 96 active+clean; 582 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 3.1 KiB/s wr, 15 op/s 2024-09-12T02:00:13.860 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:00:13 smithi016 ceph-mon[30947]: from='client.14709 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:00:13.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:00:13 smithi184 ceph-mon[35593]: pgmap v284: 97 pgs: 1 peering, 96 active+clean; 582 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 3.1 KiB/s wr, 15 op/s 2024-09-12T02:00:13.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:00:13 smithi184 ceph-mon[35593]: from='client.14709 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:00:13.918 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:00:13 smithi033 ceph-mon[35332]: pgmap v284: 97 pgs: 1 peering, 96 active+clean; 582 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 3.1 KiB/s wr, 15 op/s 2024-09-12T02:00:13.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:00:13 smithi033 ceph-mon[35332]: from='client.14709 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:00:13.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:13.947+0000 7f39e35f3640 1 -- 172.21.15.16:0/4129143478 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39dc100ca0 msgr2=0x7f39dc106840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:13.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:13.947+0000 7f39e35f3640 1 --2- 172.21.15.16:0/4129143478 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39dc100ca0 0x7f39dc106840 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f39cc009930 tx=0x7f39cc02a420 comp rx=0 tx=0).stop 2024-09-12T02:00:13.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:13.947+0000 7f39e35f3640 1 -- 172.21.15.16:0/4129143478 shutdown_connections 2024-09-12T02:00:13.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:13.947+0000 7f39e35f3640 1 --2- 172.21.15.16:0/4129143478 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39dc106f50 0x7f39dc10eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:13.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:13.947+0000 7f39e35f3640 1 --2- 172.21.15.16:0/4129143478 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39dc100ca0 0x7f39dc106840 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:13.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:13.947+0000 7f39e35f3640 1 --2- 172.21.15.16:0/4129143478 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f39dc1002f0 0x7f39dc1006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:13.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:13.947+0000 7f39e35f3640 1 -- 172.21.15.16:0/4129143478 >> 172.21.15.16:0/4129143478 conn(0x7f39dc077210 msgr2=0x7f39dc0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:00:13.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:13.947+0000 7f39e35f3640 1 -- 172.21.15.16:0/4129143478 shutdown_connections 2024-09-12T02:00:13.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:13.947+0000 7f39e35f3640 1 -- 172.21.15.16:0/4129143478 wait complete. 2024-09-12T02:00:13.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:13.948+0000 7f39e35f3640 1 Processor -- start 2024-09-12T02:00:13.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:13.948+0000 7f39e35f3640 1 -- start start 2024-09-12T02:00:13.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:13.948+0000 7f39e35f3640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f39dc1002f0 0x7f39dc101270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:13.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:13.948+0000 7f39e35f3640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39dc100ca0 0x7f39dc1017b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:13.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:13.948+0000 7f39e35f3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39dc106f50 0x7f39dc101cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:13.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:13.948+0000 7f39e35f3640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f39dc111fa0 con 0x7f39dc106f50 2024-09-12T02:00:13.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:13.948+0000 7f39e35f3640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f39dc111e20 con 0x7f39dc100ca0 2024-09-12T02:00:13.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:13.948+0000 7f39e35f3640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f39dc112120 con 0x7f39dc1002f0 2024-09-12T02:00:13.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:13.948+0000 7f39e1b90640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39dc106f50 0x7f39dc101cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:13.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:13.948+0000 7f39e1b90640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39dc106f50 0x7f39dc101cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50120/0 (socket says 172.21.15.16:50120) 2024-09-12T02:00:13.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:13.948+0000 7f39e1b90640 1 -- 172.21.15.16:0/2230951014 learned_addr learned my addr 172.21.15.16:0/2230951014 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:00:13.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:13.949+0000 7f39e138f640 1 --2- 172.21.15.16:0/2230951014 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f39dc1002f0 0x7f39dc101270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:13.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:13.949+0000 7f39e0b8e640 1 --2- 172.21.15.16:0/2230951014 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39dc100ca0 0x7f39dc1017b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:13.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:13.949+0000 7f39e1b90640 1 -- 172.21.15.16:0/2230951014 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f39dc1002f0 msgr2=0x7f39dc101270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:13.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:13.949+0000 7f39e1b90640 1 --2- 172.21.15.16:0/2230951014 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f39dc1002f0 0x7f39dc101270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:13.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:13.949+0000 7f39e1b90640 1 -- 172.21.15.16:0/2230951014 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39dc100ca0 msgr2=0x7f39dc1017b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:13.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:13.949+0000 7f39e1b90640 1 --2- 172.21.15.16:0/2230951014 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39dc100ca0 0x7f39dc1017b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:13.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:13.949+0000 7f39e1b90640 1 -- 172.21.15.16:0/2230951014 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39dc1a69c0 con 0x7f39dc106f50 2024-09-12T02:00:13.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:13.949+0000 7f39e0b8e640 1 --2- 172.21.15.16:0/2230951014 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39dc100ca0 0x7f39dc1017b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:00:13.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:13.949+0000 7f39e1b90640 1 --2- 172.21.15.16:0/2230951014 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39dc106f50 0x7f39dc101cf0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f39d400eaa0 tx=0x7f39d400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:00:13.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:13.949+0000 7f39d27fc640 1 -- 172.21.15.16:0/2230951014 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39d400cdb0 con 0x7f39dc106f50 2024-09-12T02:00:13.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:13.949+0000 7f39d27fc640 1 -- 172.21.15.16:0/2230951014 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39d4005020 con 0x7f39dc106f50 2024-09-12T02:00:13.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:13.949+0000 7f39e35f3640 1 -- 172.21.15.16:0/2230951014 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39dc1a6cb0 con 0x7f39dc106f50 2024-09-12T02:00:13.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:13.949+0000 7f39d27fc640 1 -- 172.21.15.16:0/2230951014 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39d40116b0 con 0x7f39dc106f50 2024-09-12T02:00:13.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:13.949+0000 7f39e35f3640 1 -- 172.21.15.16:0/2230951014 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39dc1a71c0 con 0x7f39dc106f50 2024-09-12T02:00:13.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:13.950+0000 7f39e35f3640 1 -- 172.21.15.16:0/2230951014 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39dc1006d0 con 0x7f39dc106f50 2024-09-12T02:00:13.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:13.952+0000 7f39d27fc640 1 -- 172.21.15.16:0/2230951014 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f39d4005560 con 0x7f39dc106f50 2024-09-12T02:00:13.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:13.952+0000 7f39d27fc640 1 --2- 172.21.15.16:0/2230951014 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f39c80a2c90 0x7f39c80a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:13.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:13.952+0000 7f39e138f640 1 --2- 172.21.15.16:0/2230951014 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f39c80a2c90 0x7f39c80a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:13.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:13.952+0000 7f39d27fc640 1 -- 172.21.15.16:0/2230951014 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f39d40cf580 con 0x7f39dc106f50 2024-09-12T02:00:13.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:13.953+0000 7f39e138f640 1 --2- 172.21.15.16:0/2230951014 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f39c80a2c90 0x7f39c80a5150 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f39d8005fd0 tx=0x7f39d800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:00:13.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:13.956+0000 7f39d27fc640 1 -- 172.21.15.16:0/2230951014 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f39d4016030 con 0x7f39dc106f50 2024-09-12T02:00:14.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:14.158+0000 7f39e35f3640 1 -- 172.21.15.16:0/2230951014 --> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f39dc062630 con 0x7f39c80a2c90 2024-09-12T02:00:14.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:14.183+0000 7f39d27fc640 1 -- 172.21.15.16:0/2230951014 <== mgr.14150 v2:172.21.15.16:6800/3887444820 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f39dc062630 con 0x7f39c80a2c90 2024-09-12T02:00:14.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:14.185+0000 7f39e35f3640 1 -- 172.21.15.16:0/2230951014 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f39c80a2c90 msgr2=0x7f39c80a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:14.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:14.185+0000 7f39e35f3640 1 --2- 172.21.15.16:0/2230951014 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f39c80a2c90 0x7f39c80a5150 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f39d8005fd0 tx=0x7f39d800b390 comp rx=0 tx=0).stop 2024-09-12T02:00:14.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:14.185+0000 7f39e35f3640 1 -- 172.21.15.16:0/2230951014 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39dc106f50 msgr2=0x7f39dc101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:14.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:14.185+0000 7f39e35f3640 1 --2- 172.21.15.16:0/2230951014 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39dc106f50 0x7f39dc101cf0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f39d400eaa0 tx=0x7f39d400ef60 comp rx=0 tx=0).stop 2024-09-12T02:00:14.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:14.185+0000 7f39e35f3640 1 -- 172.21.15.16:0/2230951014 shutdown_connections 2024-09-12T02:00:14.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:14.185+0000 7f39e35f3640 1 --2- 172.21.15.16:0/2230951014 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f39c80a2c90 0x7f39c80a5150 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:14.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:14.185+0000 7f39e35f3640 1 --2- 172.21.15.16:0/2230951014 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39dc106f50 0x7f39dc101cf0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:14.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:14.185+0000 7f39e35f3640 1 --2- 172.21.15.16:0/2230951014 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39dc100ca0 0x7f39dc1017b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:14.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:14.185+0000 7f39e35f3640 1 --2- 172.21.15.16:0/2230951014 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f39dc1002f0 0x7f39dc101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:14.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:14.185+0000 7f39e35f3640 1 -- 172.21.15.16:0/2230951014 >> 172.21.15.16:0/2230951014 conn(0x7f39dc077210 msgr2=0x7f39dc105270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:00:14.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:14.185+0000 7f39e35f3640 1 -- 172.21.15.16:0/2230951014 shutdown_connections 2024-09-12T02:00:14.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:14.185+0000 7f39e35f3640 1 -- 172.21.15.16:0/2230951014 wait complete. 2024-09-12T02:00:14.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:00:14 smithi016 ceph-mon[30947]: from='client.14715 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:00:14.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:00:14 smithi184 ceph-mon[35593]: from='client.14715 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:00:14.895 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-09-12T02:00:14.918 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:00:14 smithi033 ceph-mon[35332]: from='client.14715 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:00:15.155 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:00:15.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:15.646+0000 7f9835a37640 1 -- 172.21.15.16:0/3557541061 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98301036c0 msgr2=0x7f983010bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:15.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:15.646+0000 7f9835a37640 1 --2- 172.21.15.16:0/3557541061 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98301036c0 0x7f983010bbb0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f9824009900 tx=0x7f982402a480 comp rx=0 tx=0).stop 2024-09-12T02:00:15.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:15.646+0000 7f9835a37640 1 -- 172.21.15.16:0/3557541061 shutdown_connections 2024-09-12T02:00:15.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:15.646+0000 7f9835a37640 1 --2- 172.21.15.16:0/3557541061 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f983010c180 0x7f983010e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:15.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:15.646+0000 7f9835a37640 1 --2- 172.21.15.16:0/3557541061 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98301036c0 0x7f983010bbb0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:15.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:15.646+0000 7f9835a37640 1 --2- 172.21.15.16:0/3557541061 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9830102da0 0x7f9830103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:15.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:15.646+0000 7f9835a37640 1 -- 172.21.15.16:0/3557541061 >> 172.21.15.16:0/3557541061 conn(0x7f98300fc7a0 msgr2=0x7f98300febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:00:15.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:15.646+0000 7f9835a37640 1 -- 172.21.15.16:0/3557541061 shutdown_connections 2024-09-12T02:00:15.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:15.646+0000 7f9835a37640 1 -- 172.21.15.16:0/3557541061 wait complete. 2024-09-12T02:00:15.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:15.647+0000 7f9835a37640 1 Processor -- start 2024-09-12T02:00:15.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:15.647+0000 7f9835a37640 1 -- start start 2024-09-12T02:00:15.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:15.647+0000 7f9835a37640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9830102da0 0x7f9830100a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:15.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:15.647+0000 7f9835a37640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98301036c0 0x7f9830100f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:15.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:15.647+0000 7f9835a37640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f983010c180 0x7f9830101480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:15.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:15.647+0000 7f9835a37640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9830111ff0 con 0x7f983010c180 2024-09-12T02:00:15.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:15.647+0000 7f9835a37640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9830111e70 con 0x7f98301036c0 2024-09-12T02:00:15.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:15.647+0000 7f9835a37640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f9830112170 con 0x7f9830102da0 2024-09-12T02:00:15.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:15.647+0000 7f982f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f983010c180 0x7f9830101480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:15.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:15.647+0000 7f982f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f983010c180 0x7f9830101480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50132/0 (socket says 172.21.15.16:50132) 2024-09-12T02:00:15.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:15.647+0000 7f982f7fe640 1 -- 172.21.15.16:0/1184769930 learned_addr learned my addr 172.21.15.16:0/1184769930 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:00:15.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:15.648+0000 7f982e7fc640 1 --2- 172.21.15.16:0/1184769930 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98301036c0 0x7f9830100f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:15.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:15.648+0000 7f982effd640 1 --2- 172.21.15.16:0/1184769930 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9830102da0 0x7f9830100a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:15.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:15.648+0000 7f982f7fe640 1 -- 172.21.15.16:0/1184769930 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9830102da0 msgr2=0x7f9830100a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:15.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:15.648+0000 7f982f7fe640 1 --2- 172.21.15.16:0/1184769930 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9830102da0 0x7f9830100a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:15.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:15.648+0000 7f982f7fe640 1 -- 172.21.15.16:0/1184769930 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98301036c0 msgr2=0x7f9830100f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:15.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:15.648+0000 7f982f7fe640 1 --2- 172.21.15.16:0/1184769930 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98301036c0 0x7f9830100f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:15.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:15.648+0000 7f982f7fe640 1 -- 172.21.15.16:0/1184769930 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98301a69f0 con 0x7f983010c180 2024-09-12T02:00:15.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:15.648+0000 7f982effd640 1 --2- 172.21.15.16:0/1184769930 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9830102da0 0x7f9830100a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:00:15.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:15.648+0000 7f982e7fc640 1 --2- 172.21.15.16:0/1184769930 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98301036c0 0x7f9830100f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:00:15.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:15.648+0000 7f982f7fe640 1 --2- 172.21.15.16:0/1184769930 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f983010c180 0x7f9830101480 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f982800b970 tx=0x7f982800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:00:15.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:15.648+0000 7f9834a35640 1 -- 172.21.15.16:0/1184769930 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9828019070 con 0x7f983010c180 2024-09-12T02:00:15.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:15.648+0000 7f9835a37640 1 -- 172.21.15.16:0/1184769930 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98301a6ce0 con 0x7f983010c180 2024-09-12T02:00:15.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:15.648+0000 7f9834a35640 1 -- 172.21.15.16:0/1184769930 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f982800cb40 con 0x7f983010c180 2024-09-12T02:00:15.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:15.648+0000 7f9834a35640 1 -- 172.21.15.16:0/1184769930 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9828007650 con 0x7f983010c180 2024-09-12T02:00:15.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:15.648+0000 7f9835a37640 1 -- 172.21.15.16:0/1184769930 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98301a71f0 con 0x7f983010c180 2024-09-12T02:00:15.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:15.649+0000 7f9835a37640 1 -- 172.21.15.16:0/1184769930 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9830103180 con 0x7f983010c180 2024-09-12T02:00:15.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:15.651+0000 7f9834a35640 1 -- 172.21.15.16:0/1184769930 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f982801c070 con 0x7f983010c180 2024-09-12T02:00:15.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:15.652+0000 7f9834a35640 1 --2- 172.21.15.16:0/1184769930 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f98180a2c10 0x7f98180a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:15.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:15.652+0000 7f982effd640 1 --2- 172.21.15.16:0/1184769930 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f98180a2c10 0x7f98180a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:15.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:15.652+0000 7f9834a35640 1 -- 172.21.15.16:0/1184769930 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f98280ce710 con 0x7f983010c180 2024-09-12T02:00:15.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:15.652+0000 7f982effd640 1 --2- 172.21.15.16:0/1184769930 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f98180a2c10 0x7f98180a50d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f98200058c0 tx=0x7f9820005d30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:00:15.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:15.656+0000 7f9834a35640 1 -- 172.21.15.16:0/1184769930 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f982809b620 con 0x7f983010c180 2024-09-12T02:00:15.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:15.856+0000 7f9835a37640 1 -- 172.21.15.16:0/1184769930 --> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f9830062630 con 0x7f98180a2c10 2024-09-12T02:00:15.857 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:00:15 smithi016 ceph-mon[30947]: pgmap v285: 97 pgs: 97 active+clean; 607 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 4.9 KiB/s wr, 10 op/s; 13 B/s, 0 objects/s recovering 2024-09-12T02:00:15.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:15.861+0000 7f9834a35640 1 -- 172.21.15.16:0/1184769930 <== mgr.14150 v2:172.21.15.16:6800/3887444820 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f9830062630 con 0x7f98180a2c10 2024-09-12T02:00:15.861 INFO:teuthology.orchestra.run.smithi016.stdout:/volumes/qa/sv_1/a1673dc0-6b08-43d7-895c-025fe8df6ce5 2024-09-12T02:00:15.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:15.863+0000 7f9835a37640 1 -- 172.21.15.16:0/1184769930 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f98180a2c10 msgr2=0x7f98180a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:15.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:15.863+0000 7f9835a37640 1 --2- 172.21.15.16:0/1184769930 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f98180a2c10 0x7f98180a50d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f98200058c0 tx=0x7f9820005d30 comp rx=0 tx=0).stop 2024-09-12T02:00:15.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:15.863+0000 7f9835a37640 1 -- 172.21.15.16:0/1184769930 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f983010c180 msgr2=0x7f9830101480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:15.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:15.863+0000 7f9835a37640 1 --2- 172.21.15.16:0/1184769930 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f983010c180 0x7f9830101480 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f982800b970 tx=0x7f982800be30 comp rx=0 tx=0).stop 2024-09-12T02:00:15.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:15.863+0000 7f9835a37640 1 -- 172.21.15.16:0/1184769930 shutdown_connections 2024-09-12T02:00:15.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:15.863+0000 7f9835a37640 1 --2- 172.21.15.16:0/1184769930 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f98180a2c10 0x7f98180a50d0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:15.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:15.863+0000 7f9835a37640 1 --2- 172.21.15.16:0/1184769930 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f983010c180 0x7f9830101480 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:15.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:15.863+0000 7f9835a37640 1 --2- 172.21.15.16:0/1184769930 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98301036c0 0x7f9830100f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:15.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:15.863+0000 7f9835a37640 1 --2- 172.21.15.16:0/1184769930 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9830102da0 0x7f9830100a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:15.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:15.863+0000 7f9835a37640 1 -- 172.21.15.16:0/1184769930 >> 172.21.15.16:0/1184769930 conn(0x7f98300fc7a0 msgr2=0x7f98300fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:00:15.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:15.863+0000 7f9835a37640 1 -- 172.21.15.16:0/1184769930 shutdown_connections 2024-09-12T02:00:15.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:15.863+0000 7f9835a37640 1 -- 172.21.15.16:0/1184769930 wait complete. 2024-09-12T02:00:15.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:00:15 smithi184 ceph-mon[35593]: pgmap v285: 97 pgs: 97 active+clean; 607 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 4.9 KiB/s wr, 10 op/s; 13 B/s, 0 objects/s recovering 2024-09-12T02:00:15.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:00:15 smithi033 ceph-mon[35332]: pgmap v285: 97 pgs: 97 active+clean; 607 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 4.9 KiB/s wr, 10 op/s; 13 B/s, 0 objects/s recovering 2024-09-12T02:00:16.488 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:00:16.749 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:00:16.855 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:00:16 smithi016 ceph-mon[30947]: from='client.14721 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:00:16.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:00:16 smithi184 ceph-mon[35593]: from='client.14721 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:00:16.918 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:00:16 smithi033 ceph-mon[35332]: from='client.14721 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:00:17.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:17.222+0000 7f5be61df640 1 -- 172.21.15.16:0/1752529054 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5be010c180 msgr2=0x7f5be010e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:17.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:17.222+0000 7f5be61df640 1 --2- 172.21.15.16:0/1752529054 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5be010c180 0x7f5be010e600 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f5bd8009ca0 tx=0x7f5bd802a950 comp rx=0 tx=0).stop 2024-09-12T02:00:17.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:17.222+0000 7f5be61df640 1 -- 172.21.15.16:0/1752529054 shutdown_connections 2024-09-12T02:00:17.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:17.222+0000 7f5be61df640 1 --2- 172.21.15.16:0/1752529054 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5be010c180 0x7f5be010e600 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:17.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:17.222+0000 7f5be61df640 1 --2- 172.21.15.16:0/1752529054 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5be01036c0 0x7f5be010bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:17.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:17.222+0000 7f5be61df640 1 --2- 172.21.15.16:0/1752529054 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5be0102da0 0x7f5be0103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:17.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:17.222+0000 7f5be61df640 1 -- 172.21.15.16:0/1752529054 >> 172.21.15.16:0/1752529054 conn(0x7f5be00fc7a0 msgr2=0x7f5be00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:00:17.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:17.222+0000 7f5be61df640 1 -- 172.21.15.16:0/1752529054 shutdown_connections 2024-09-12T02:00:17.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:17.222+0000 7f5be61df640 1 -- 172.21.15.16:0/1752529054 wait complete. 2024-09-12T02:00:17.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:17.223+0000 7f5be61df640 1 Processor -- start 2024-09-12T02:00:17.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:17.223+0000 7f5be61df640 1 -- start start 2024-09-12T02:00:17.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:17.223+0000 7f5be61df640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5be0102da0 0x7f5be0100a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:17.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:17.223+0000 7f5be61df640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5be01036c0 0x7f5be0100f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:17.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:17.223+0000 7f5be61df640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5be010c180 0x7f5be0101480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:17.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:17.223+0000 7f5be61df640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5be0111fb0 con 0x7f5be0102da0 2024-09-12T02:00:17.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:17.223+0000 7f5be61df640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5be0111e30 con 0x7f5be010c180 2024-09-12T02:00:17.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:17.223+0000 7f5be61df640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5be0112130 con 0x7f5be01036c0 2024-09-12T02:00:17.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:17.223+0000 7f5bdf7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5be0102da0 0x7f5be0100a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:17.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:17.223+0000 7f5bdf7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5be0102da0 0x7f5be0100a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50150/0 (socket says 172.21.15.16:50150) 2024-09-12T02:00:17.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:17.224+0000 7f5bdf7fe640 1 -- 172.21.15.16:0/2416780268 learned_addr learned my addr 172.21.15.16:0/2416780268 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:00:17.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:17.224+0000 7f5bdeffd640 1 --2- 172.21.15.16:0/2416780268 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5be01036c0 0x7f5be0100f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:17.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:17.224+0000 7f5bdffff640 1 --2- 172.21.15.16:0/2416780268 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5be010c180 0x7f5be0101480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:17.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:17.224+0000 7f5bdf7fe640 1 -- 172.21.15.16:0/2416780268 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5be01036c0 msgr2=0x7f5be0100f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:17.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:17.224+0000 7f5bdf7fe640 1 --2- 172.21.15.16:0/2416780268 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5be01036c0 0x7f5be0100f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:17.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:17.224+0000 7f5bdf7fe640 1 -- 172.21.15.16:0/2416780268 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5be010c180 msgr2=0x7f5be0101480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:17.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:17.224+0000 7f5bdf7fe640 1 --2- 172.21.15.16:0/2416780268 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5be010c180 0x7f5be0101480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:17.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:17.224+0000 7f5bdf7fe640 1 -- 172.21.15.16:0/2416780268 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5be01a6890 con 0x7f5be0102da0 2024-09-12T02:00:17.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:17.224+0000 7f5bdf7fe640 1 --2- 172.21.15.16:0/2416780268 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5be0102da0 0x7f5be0100a00 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f5bd000d770 tx=0x7f5bd000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:00:17.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:17.224+0000 7f5bdcff9640 1 -- 172.21.15.16:0/2416780268 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bd0017070 con 0x7f5be0102da0 2024-09-12T02:00:17.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:17.224+0000 7f5bdcff9640 1 -- 172.21.15.16:0/2416780268 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5bd0005500 con 0x7f5be0102da0 2024-09-12T02:00:17.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:17.224+0000 7f5be61df640 1 -- 172.21.15.16:0/2416780268 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5be01a6b80 con 0x7f5be0102da0 2024-09-12T02:00:17.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:17.224+0000 7f5bdcff9640 1 -- 172.21.15.16:0/2416780268 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bd0004e00 con 0x7f5be0102da0 2024-09-12T02:00:17.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:17.224+0000 7f5be61df640 1 -- 172.21.15.16:0/2416780268 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5be01a7070 con 0x7f5be0102da0 2024-09-12T02:00:17.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:17.225+0000 7f5be61df640 1 -- 172.21.15.16:0/2416780268 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5be0103180 con 0x7f5be0102da0 2024-09-12T02:00:17.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:17.226+0000 7f5bdcff9640 1 -- 172.21.15.16:0/2416780268 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f5bd0004fa0 con 0x7f5be0102da0 2024-09-12T02:00:17.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:17.227+0000 7f5bdcff9640 1 --2- 172.21.15.16:0/2416780268 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f5bc80a2c10 0x7f5bc80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:17.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:17.227+0000 7f5bdeffd640 1 --2- 172.21.15.16:0/2416780268 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f5bc80a2c10 0x7f5bc80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:17.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:17.227+0000 7f5bdcff9640 1 -- 172.21.15.16:0/2416780268 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f5bd00ceaa0 con 0x7f5be0102da0 2024-09-12T02:00:17.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:17.228+0000 7f5bdeffd640 1 --2- 172.21.15.16:0/2416780268 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f5bc80a2c10 0x7f5bc80a50d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f5be0102490 tx=0x7f5bd400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:00:17.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:17.231+0000 7f5bdcff9640 1 -- 172.21.15.16:0/2416780268 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5bd009b350 con 0x7f5be0102da0 2024-09-12T02:00:17.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:17.441+0000 7f5be61df640 1 -- 172.21.15.16:0/2416780268 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5be0107220 con 0x7f5be0102da0 2024-09-12T02:00:17.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:17.442+0000 7f5bdcff9640 1 -- 172.21.15.16:0/2416780268 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11911 (secure 0 0 0) 0x7f5bd009f650 con 0x7f5be0102da0 2024-09-12T02:00:17.442 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T02:00:17.442 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":24,"btime":"2024-09-12T02:00:07:547152+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3279123279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3279123279},{"type":"v1","addr":"172.21.15.16:6835","nonce":3279123279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4119452382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4119452382},{"type":"v1","addr":"172.21.15.16:6837","nonce":4119452382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/3263823165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3263823165},{"type":"v1","addr":"172.21.15.16:6839","nonce":3263823165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/1510290732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":1510290732},{"type":"v1","addr":"172.21.15.16:6841","nonce":1510290732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/279674603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":279674603},{"type":"v1","addr":"172.21.15.33:6833","nonce":279674603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3555728028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3555728028},{"type":"v1","addr":"172.21.15.33:6835","nonce":3555728028}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T01:59:56.117608+0000","modified":"2024-09-12T02:00:07.097363+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24482,"mds_1":24481,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6833/1563374106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1563374106},{"type":"v1","addr":"172.21.15.184:6833","nonce":1563374106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6837/1892833868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1892833868},{"type":"v1","addr":"172.21.15.33:6837","nonce":1892833868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/1457679686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1457679686},{"type":"v1","addr":"172.21.15.184:6835","nonce":1457679686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.184:6837/1793393087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":1793393087},{"type":"v1","addr":"172.21.15.184:6837","nonce":1793393087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.184:6839/4203347941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":4203347941},{"type":"v1","addr":"172.21.15.184:6839","nonce":4203347941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.33:6839/4136670204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":4136670204},{"type":"v1","addr":"172.21.15.33:6839","nonce":4136670204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24469,24482,24481]},"id":1}]} 2024-09-12T02:00:17.442 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 24 2024-09-12T02:00:17.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:17.444+0000 7f5be61df640 1 -- 172.21.15.16:0/2416780268 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f5bc80a2c10 msgr2=0x7f5bc80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:17.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:17.444+0000 7f5be61df640 1 --2- 172.21.15.16:0/2416780268 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f5bc80a2c10 0x7f5bc80a50d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f5be0102490 tx=0x7f5bd400b390 comp rx=0 tx=0).stop 2024-09-12T02:00:17.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:17.444+0000 7f5be61df640 1 -- 172.21.15.16:0/2416780268 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5be0102da0 msgr2=0x7f5be0100a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:17.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:17.444+0000 7f5be61df640 1 --2- 172.21.15.16:0/2416780268 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5be0102da0 0x7f5be0100a00 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f5bd000d770 tx=0x7f5bd000dc30 comp rx=0 tx=0).stop 2024-09-12T02:00:17.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:17.444+0000 7f5be61df640 1 -- 172.21.15.16:0/2416780268 shutdown_connections 2024-09-12T02:00:17.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:17.444+0000 7f5be61df640 1 --2- 172.21.15.16:0/2416780268 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f5bc80a2c10 0x7f5bc80a50d0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:17.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:17.444+0000 7f5be61df640 1 --2- 172.21.15.16:0/2416780268 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5be010c180 0x7f5be0101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:17.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:17.444+0000 7f5be61df640 1 --2- 172.21.15.16:0/2416780268 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5be01036c0 0x7f5be0100f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:17.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:17.444+0000 7f5be61df640 1 --2- 172.21.15.16:0/2416780268 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5be0102da0 0x7f5be0100a00 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:17.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:17.444+0000 7f5be61df640 1 -- 172.21.15.16:0/2416780268 >> 172.21.15.16:0/2416780268 conn(0x7f5be00fc7a0 msgr2=0x7f5be00fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:00:17.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:17.444+0000 7f5be61df640 1 -- 172.21.15.16:0/2416780268 shutdown_connections 2024-09-12T02:00:17.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:17.444+0000 7f5be61df640 1 -- 172.21.15.16:0/2416780268 wait complete. 2024-09-12T02:00:17.858 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:00:17 smithi016 ceph-mon[30947]: pgmap v286: 97 pgs: 97 active+clean; 607 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 4.1 KiB/s wr, 10 op/s; 13 B/s, 0 objects/s recovering 2024-09-12T02:00:17.858 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:00:17 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/2416780268' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:00:17.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:00:17 smithi184 ceph-mon[35593]: pgmap v286: 97 pgs: 97 active+clean; 607 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 4.1 KiB/s wr, 10 op/s; 13 B/s, 0 objects/s recovering 2024-09-12T02:00:17.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:00:17 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/2416780268' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:00:17.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:00:17 smithi033 ceph-mon[35332]: pgmap v286: 97 pgs: 97 active+clean; 607 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 4.1 KiB/s wr, 10 op/s; 13 B/s, 0 objects/s recovering 2024-09-12T02:00:17.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:00:17 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/2416780268' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:00:18.152 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T02:00:18.410 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:00:18.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:18.847+0000 7f5b183db640 1 -- 172.21.15.16:0/2873249751 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b10106f10 msgr2=0x7f5b1010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:18.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:18.847+0000 7f5b183db640 1 --2- 172.21.15.16:0/2873249751 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b10106f10 0x7f5b1010ea90 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f5b08009ca0 tx=0x7f5b0802a8c0 comp rx=0 tx=0).stop 2024-09-12T02:00:18.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:18.847+0000 7f5b183db640 1 -- 172.21.15.16:0/2873249751 shutdown_connections 2024-09-12T02:00:18.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:18.847+0000 7f5b183db640 1 --2- 172.21.15.16:0/2873249751 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b10106f10 0x7f5b1010ea90 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:18.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:18.847+0000 7f5b183db640 1 --2- 172.21.15.16:0/2873249751 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b10102790 0x7f5b101067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:18.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:18.847+0000 7f5b183db640 1 --2- 172.21.15.16:0/2873249751 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5b10101de0 0x7f5b101021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:18.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:18.847+0000 7f5b183db640 1 -- 172.21.15.16:0/2873249751 >> 172.21.15.16:0/2873249751 conn(0x7f5b100fd8c0 msgr2=0x7f5b100ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:00:18.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:18.847+0000 7f5b183db640 1 -- 172.21.15.16:0/2873249751 shutdown_connections 2024-09-12T02:00:18.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:18.847+0000 7f5b183db640 1 -- 172.21.15.16:0/2873249751 wait complete. 2024-09-12T02:00:18.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:18.848+0000 7f5b183db640 1 Processor -- start 2024-09-12T02:00:18.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:18.848+0000 7f5b183db640 1 -- start start 2024-09-12T02:00:18.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:18.848+0000 7f5b183db640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b10101de0 0x7f5b100744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:18.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:18.848+0000 7f5b183db640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b10102790 0x7f5b10074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:18.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:18.848+0000 7f5b183db640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5b10106f10 0x7f5b10074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:18.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:18.848+0000 7f5b183db640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5b10111fb0 con 0x7f5b10102790 2024-09-12T02:00:18.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:18.848+0000 7f5b183db640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5b10111e30 con 0x7f5b10101de0 2024-09-12T02:00:18.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:18.848+0000 7f5b183db640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5b10112130 con 0x7f5b10106f10 2024-09-12T02:00:18.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:18.848+0000 7f5b15976640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b10102790 0x7f5b10074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:18.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:18.848+0000 7f5b16177640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b10101de0 0x7f5b100744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:18.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:18.848+0000 7f5b15976640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b10102790 0x7f5b10074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50162/0 (socket says 172.21.15.16:50162) 2024-09-12T02:00:18.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:18.848+0000 7f5b15976640 1 -- 172.21.15.16:0/1110408318 learned_addr learned my addr 172.21.15.16:0/1110408318 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:00:18.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:18.849+0000 7f5b16978640 1 --2- 172.21.15.16:0/1110408318 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5b10106f10 0x7f5b10074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:18.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:18.849+0000 7f5b15976640 1 -- 172.21.15.16:0/1110408318 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5b10106f10 msgr2=0x7f5b10074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:18.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:18.849+0000 7f5b15976640 1 --2- 172.21.15.16:0/1110408318 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5b10106f10 0x7f5b10074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:18.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:18.849+0000 7f5b15976640 1 -- 172.21.15.16:0/1110408318 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b10101de0 msgr2=0x7f5b100744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:18.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:18.849+0000 7f5b15976640 1 --2- 172.21.15.16:0/1110408318 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b10101de0 0x7f5b100744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:18.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:18.849+0000 7f5b15976640 1 -- 172.21.15.16:0/1110408318 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b101a6970 con 0x7f5b10102790 2024-09-12T02:00:18.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:18.849+0000 7f5b16177640 1 --2- 172.21.15.16:0/1110408318 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b10101de0 0x7f5b100744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:00:18.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:18.849+0000 7f5b15976640 1 --2- 172.21.15.16:0/1110408318 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b10102790 0x7f5b10074a30 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f5b0000b4a0 tx=0x7f5b0000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:00:18.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:18.849+0000 7f5b077fe640 1 -- 172.21.15.16:0/1110408318 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b00014070 con 0x7f5b10102790 2024-09-12T02:00:18.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:18.849+0000 7f5b183db640 1 -- 172.21.15.16:0/1110408318 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b101a6c60 con 0x7f5b10102790 2024-09-12T02:00:18.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:18.849+0000 7f5b077fe640 1 -- 172.21.15.16:0/1110408318 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b00010af0 con 0x7f5b10102790 2024-09-12T02:00:18.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:18.849+0000 7f5b077fe640 1 -- 172.21.15.16:0/1110408318 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b00017d10 con 0x7f5b10102790 2024-09-12T02:00:18.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:18.849+0000 7f5b183db640 1 -- 172.21.15.16:0/1110408318 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b101a71a0 con 0x7f5b10102790 2024-09-12T02:00:18.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:18.850+0000 7f5b183db640 1 -- 172.21.15.16:0/1110408318 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b101021c0 con 0x7f5b10102790 2024-09-12T02:00:18.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:18.851+0000 7f5b077fe640 1 -- 172.21.15.16:0/1110408318 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f5b00017070 con 0x7f5b10102790 2024-09-12T02:00:18.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:18.852+0000 7f5b077fe640 1 --2- 172.21.15.16:0/1110408318 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f5afc0a2c10 0x7f5afc0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:18.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:18.853+0000 7f5b16177640 1 --2- 172.21.15.16:0/1110408318 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f5afc0a2c10 0x7f5afc0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:18.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:18.853+0000 7f5b077fe640 1 -- 172.21.15.16:0/1110408318 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f5b000ce280 con 0x7f5b10102790 2024-09-12T02:00:18.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:18.853+0000 7f5b16177640 1 --2- 172.21.15.16:0/1110408318 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f5afc0a2c10 0x7f5afc0a50d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f5b0c005fd0 tx=0x7f5b0c00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:00:18.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:18.857+0000 7f5b077fe640 1 -- 172.21.15.16:0/1110408318 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5b0009ac40 con 0x7f5b10102790 2024-09-12T02:00:19.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:19.037+0000 7f5b183db640 1 -- 172.21.15.16:0/1110408318 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5b1010a150 con 0x7f5b10102790 2024-09-12T02:00:19.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:19.039+0000 7f5b077fe640 1 -- 172.21.15.16:0/1110408318 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+22584 (secure 0 0 0) 0x7f5b0009ef40 con 0x7f5b10102790 2024-09-12T02:00:19.039 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T02:00:19.040 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":82,"fsid":"879f3b56-70a9-11ef-bcea-c7b262605968","created":"2024-09-12T01:52:12.841045+0000","modified":"2024-09-12T02:00:09.559511+0000","last_up_change":"2024-09-12T01:59:08.975443+0000","last_in_change":"2024-09-12T01:58:44.530220+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T01:55:26.727564+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T01:59:52.402097+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T01:59:54.463627+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3ea41eb6-e384-4ddc-810f-0f103c127849","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4000187867},{"type":"v1","addr":"172.21.15.16:6803","nonce":4000187867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4000187867},{"type":"v1","addr":"172.21.15.16:6805","nonce":4000187867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":4000187867},{"type":"v1","addr":"172.21.15.16:6809","nonce":4000187867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4000187867},{"type":"v1","addr":"172.21.15.16:6807","nonce":4000187867}]},"public_addr":"172.21.15.16:6803/4000187867","cluster_addr":"172.21.15.16:6805/4000187867","heartbeat_back_addr":"172.21.15.16:6809/4000187867","heartbeat_front_addr":"172.21.15.16:6807/4000187867","state":["exists","up"]},{"osd":1,"uuid":"07bf63e7-70fa-41b2-af53-12a5fec3f33c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":3862661148},{"type":"v1","addr":"172.21.15.33:6801","nonce":3862661148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":3862661148},{"type":"v1","addr":"172.21.15.33:6803","nonce":3862661148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":3862661148},{"type":"v1","addr":"172.21.15.33:6807","nonce":3862661148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":3862661148},{"type":"v1","addr":"172.21.15.33:6805","nonce":3862661148}]},"public_addr":"172.21.15.33:6801/3862661148","cluster_addr":"172.21.15.33:6803/3862661148","heartbeat_back_addr":"172.21.15.33:6807/3862661148","heartbeat_front_addr":"172.21.15.33:6805/3862661148","state":["exists","up"]},{"osd":2,"uuid":"22e40cac-3bc0-4cce-b47b-77499b767f7e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":3958753963},{"type":"v1","addr":"172.21.15.184:6801","nonce":3958753963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":3958753963},{"type":"v1","addr":"172.21.15.184:6803","nonce":3958753963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":3958753963},{"type":"v1","addr":"172.21.15.184:6807","nonce":3958753963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":3958753963},{"type":"v1","addr":"172.21.15.184:6805","nonce":3958753963}]},"public_addr":"172.21.15.184:6801/3958753963","cluster_addr":"172.21.15.184:6803/3958753963","heartbeat_back_addr":"172.21.15.184:6807/3958753963","heartbeat_front_addr":"172.21.15.184:6805/3958753963","state":["exists","up"]},{"osd":3,"uuid":"834a07f5-2c24-49e1-8e4a-12c96cfbdc65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3395182524},{"type":"v1","addr":"172.21.15.16:6811","nonce":3395182524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3395182524},{"type":"v1","addr":"172.21.15.16:6813","nonce":3395182524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3395182524},{"type":"v1","addr":"172.21.15.16:6817","nonce":3395182524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3395182524},{"type":"v1","addr":"172.21.15.16:6815","nonce":3395182524}]},"public_addr":"172.21.15.16:6811/3395182524","cluster_addr":"172.21.15.16:6813/3395182524","heartbeat_back_addr":"172.21.15.16:6817/3395182524","heartbeat_front_addr":"172.21.15.16:6815/3395182524","state":["exists","up"]},{"osd":4,"uuid":"3ca56e09-c194-4404-b9bd-2524d38367eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":1355184322},{"type":"v1","addr":"172.21.15.33:6809","nonce":1355184322}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1355184322},{"type":"v1","addr":"172.21.15.33:6811","nonce":1355184322}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1355184322},{"type":"v1","addr":"172.21.15.33:6815","nonce":1355184322}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1355184322},{"type":"v1","addr":"172.21.15.33:6813","nonce":1355184322}]},"public_addr":"172.21.15.33:6809/1355184322","cluster_addr":"172.21.15.33:6811/1355184322","heartbeat_back_addr":"172.21.15.33:6815/1355184322","heartbeat_front_addr":"172.21.15.33:6813/1355184322","state":["exists","up"]},{"osd":5,"uuid":"0b05ecdb-b987-461f-951c-6d76739bfb4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":1533397276},{"type":"v1","addr":"172.21.15.184:6809","nonce":1533397276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":1533397276},{"type":"v1","addr":"172.21.15.184:6811","nonce":1533397276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":1533397276},{"type":"v1","addr":"172.21.15.184:6815","nonce":1533397276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":1533397276},{"type":"v1","addr":"172.21.15.184:6813","nonce":1533397276}]},"public_addr":"172.21.15.184:6809/1533397276","cluster_addr":"172.21.15.184:6811/1533397276","heartbeat_back_addr":"172.21.15.184:6815/1533397276","heartbeat_front_addr":"172.21.15.184:6813/1533397276","state":["exists","up"]},{"osd":6,"uuid":"b3936e21-e197-49fa-8441-d909b258b612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":902466142},{"type":"v1","addr":"172.21.15.16:6819","nonce":902466142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":902466142},{"type":"v1","addr":"172.21.15.16:6821","nonce":902466142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":902466142},{"type":"v1","addr":"172.21.15.16:6825","nonce":902466142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":902466142},{"type":"v1","addr":"172.21.15.16:6823","nonce":902466142}]},"public_addr":"172.21.15.16:6819/902466142","cluster_addr":"172.21.15.16:6821/902466142","heartbeat_back_addr":"172.21.15.16:6825/902466142","heartbeat_front_addr":"172.21.15.16:6823/902466142","state":["exists","up"]},{"osd":7,"uuid":"4ddbe011-34c1-4b3f-a8eb-577422fb5426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":4186032630},{"type":"v1","addr":"172.21.15.33:6817","nonce":4186032630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":4186032630},{"type":"v1","addr":"172.21.15.33:6819","nonce":4186032630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":4186032630},{"type":"v1","addr":"172.21.15.33:6823","nonce":4186032630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":4186032630},{"type":"v1","addr":"172.21.15.33:6821","nonce":4186032630}]},"public_addr":"172.21.15.33:6817/4186032630","cluster_addr":"172.21.15.33:6819/4186032630","heartbeat_back_addr":"172.21.15.33:6823/4186032630","heartbeat_front_addr":"172.21.15.33:6821/4186032630","state":["exists","up"]},{"osd":8,"uuid":"dca5bfb8-ac6a-47ce-905c-be7fcaf0b5ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":781851781},{"type":"v1","addr":"172.21.15.184:6817","nonce":781851781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":781851781},{"type":"v1","addr":"172.21.15.184:6819","nonce":781851781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":781851781},{"type":"v1","addr":"172.21.15.184:6823","nonce":781851781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":781851781},{"type":"v1","addr":"172.21.15.184:6821","nonce":781851781}]},"public_addr":"172.21.15.184:6817/781851781","cluster_addr":"172.21.15.184:6819/781851781","heartbeat_back_addr":"172.21.15.184:6823/781851781","heartbeat_front_addr":"172.21.15.184:6821/781851781","state":["exists","up"]},{"osd":9,"uuid":"23a05cb5-01d5-40f3-b66a-15fcccedbd6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":1569578746},{"type":"v1","addr":"172.21.15.16:6827","nonce":1569578746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":1569578746},{"type":"v1","addr":"172.21.15.16:6829","nonce":1569578746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1569578746},{"type":"v1","addr":"172.21.15.16:6833","nonce":1569578746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":1569578746},{"type":"v1","addr":"172.21.15.16:6831","nonce":1569578746}]},"public_addr":"172.21.15.16:6827/1569578746","cluster_addr":"172.21.15.16:6829/1569578746","heartbeat_back_addr":"172.21.15.16:6833/1569578746","heartbeat_front_addr":"172.21.15.16:6831/1569578746","state":["exists","up"]},{"osd":10,"uuid":"e4bd6bce-3c69-44db-bc09-d264a0ad094a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":469977847},{"type":"v1","addr":"172.21.15.33:6825","nonce":469977847}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":469977847},{"type":"v1","addr":"172.21.15.33:6827","nonce":469977847}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":469977847},{"type":"v1","addr":"172.21.15.33:6831","nonce":469977847}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":469977847},{"type":"v1","addr":"172.21.15.33:6829","nonce":469977847}]},"public_addr":"172.21.15.33:6825/469977847","cluster_addr":"172.21.15.33:6827/469977847","heartbeat_back_addr":"172.21.15.33:6831/469977847","heartbeat_front_addr":"172.21.15.33:6829/469977847","state":["exists","up"]},{"osd":11,"uuid":"d60212f8-5a1d-45f7-9a1b-12294bb4c80d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":1619871346},{"type":"v1","addr":"172.21.15.184:6825","nonce":1619871346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":1619871346},{"type":"v1","addr":"172.21.15.184:6827","nonce":1619871346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":1619871346},{"type":"v1","addr":"172.21.15.184:6831","nonce":1619871346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":1619871346},{"type":"v1","addr":"172.21.15.184:6829","nonce":1619871346}]},"public_addr":"172.21.15.184:6825/1619871346","cluster_addr":"172.21.15.184:6827/1619871346","heartbeat_back_addr":"172.21.15.184:6831/1619871346","heartbeat_front_addr":"172.21.15.184:6829/1619871346","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:54:32.057979+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:54:57.501562+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:55:23.137225+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:55:48.646155+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:56:13.311907+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:56:38.214188+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:57:02.570921+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:57:27.506097+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:57:52.038529+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:58:17.423300+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:58:41.579992+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:59:06.245817+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:0/3532565698":"2024-09-13T01:53:08.218475+0000","172.21.15.16:0/1822157179":"2024-09-13T01:52:44.194152+0000","172.21.15.16:0/3919664061":"2024-09-13T01:52:44.194152+0000","172.21.15.16:0/2269394774":"2024-09-13T01:52:44.194152+0000","172.21.15.16:0/2595308634":"2024-09-13T01:53:08.218475+0000","172.21.15.16:6801/2754331153":"2024-09-13T01:52:44.194152+0000","172.21.15.16:6801/2615861334":"2024-09-13T01:53:08.218475+0000","172.21.15.16:6800/2615861334":"2024-09-13T01:53:08.218475+0000","172.21.15.16:0/4199114771":"2024-09-13T01:53:08.218475+0000","172.21.15.16:6800/2754331153":"2024-09-13T01:52:44.194152+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T02:00:19.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:19.041+0000 7f5b183db640 1 -- 172.21.15.16:0/1110408318 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f5afc0a2c10 msgr2=0x7f5afc0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:19.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:19.041+0000 7f5b183db640 1 --2- 172.21.15.16:0/1110408318 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f5afc0a2c10 0x7f5afc0a50d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f5b0c005fd0 tx=0x7f5b0c00b390 comp rx=0 tx=0).stop 2024-09-12T02:00:19.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:19.041+0000 7f5b183db640 1 -- 172.21.15.16:0/1110408318 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b10102790 msgr2=0x7f5b10074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:19.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:19.041+0000 7f5b183db640 1 --2- 172.21.15.16:0/1110408318 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b10102790 0x7f5b10074a30 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f5b0000b4a0 tx=0x7f5b0000b960 comp rx=0 tx=0).stop 2024-09-12T02:00:19.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:19.041+0000 7f5b183db640 1 -- 172.21.15.16:0/1110408318 shutdown_connections 2024-09-12T02:00:19.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:19.041+0000 7f5b183db640 1 --2- 172.21.15.16:0/1110408318 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f5afc0a2c10 0x7f5afc0a50d0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:19.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:19.041+0000 7f5b183db640 1 --2- 172.21.15.16:0/1110408318 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5b10106f10 0x7f5b10074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:19.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:19.041+0000 7f5b183db640 1 --2- 172.21.15.16:0/1110408318 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b10102790 0x7f5b10074a30 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:19.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:19.041+0000 7f5b183db640 1 --2- 172.21.15.16:0/1110408318 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b10101de0 0x7f5b100744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:19.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:19.041+0000 7f5b183db640 1 -- 172.21.15.16:0/1110408318 >> 172.21.15.16:0/1110408318 conn(0x7f5b100fd8c0 msgr2=0x7f5b100fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:00:19.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:19.042+0000 7f5b183db640 1 -- 172.21.15.16:0/1110408318 shutdown_connections 2024-09-12T02:00:19.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:19.042+0000 7f5b183db640 1 -- 172.21.15.16:0/1110408318 wait complete. 2024-09-12T02:00:19.693 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-12T02:00:19.693 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-12T02:00:19.933 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:00:19 smithi016 ceph-mon[30947]: pgmap v287: 97 pgs: 97 active+clean; 607 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 3.4 KiB/s wr, 9 op/s; 11 B/s, 0 objects/s recovering 2024-09-12T02:00:19.933 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:00:19 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/1110408318' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T02:00:19.953 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:00:20.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:00:19 smithi184 ceph-mon[35593]: pgmap v287: 97 pgs: 97 active+clean; 607 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 3.4 KiB/s wr, 9 op/s; 11 B/s, 0 objects/s recovering 2024-09-12T02:00:20.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:00:19 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/1110408318' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T02:00:20.168 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:00:19 smithi033 ceph-mon[35332]: pgmap v287: 97 pgs: 97 active+clean; 607 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 3.4 KiB/s wr, 9 op/s; 11 B/s, 0 objects/s recovering 2024-09-12T02:00:20.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:00:19 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/1110408318' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T02:00:20.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:20.497+0000 7ff8a4585640 1 -- 172.21.15.16:0/462011988 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff89c1055b0 msgr2=0x7ff89c10d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:20.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:20.497+0000 7ff8a4585640 1 --2- 172.21.15.16:0/462011988 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff89c1055b0 0x7ff89c10d130 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7ff894009900 tx=0x7ff89402a410 comp rx=0 tx=0).stop 2024-09-12T02:00:20.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:20.497+0000 7ff8a4585640 1 -- 172.21.15.16:0/462011988 shutdown_connections 2024-09-12T02:00:20.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:20.497+0000 7ff8a4585640 1 --2- 172.21.15.16:0/462011988 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff89c1055b0 0x7ff89c10d130 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:20.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:20.497+0000 7ff8a4585640 1 --2- 172.21.15.16:0/462011988 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff89c100e30 0x7ff89c104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:20.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:20.497+0000 7ff8a4585640 1 --2- 172.21.15.16:0/462011988 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff89c100480 0x7ff89c100860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:20.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:20.497+0000 7ff8a4585640 1 -- 172.21.15.16:0/462011988 >> 172.21.15.16:0/462011988 conn(0x7ff89c077210 msgr2=0x7ff89c0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:00:20.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:20.497+0000 7ff8a4585640 1 -- 172.21.15.16:0/462011988 shutdown_connections 2024-09-12T02:00:20.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:20.497+0000 7ff8a4585640 1 -- 172.21.15.16:0/462011988 wait complete. 2024-09-12T02:00:20.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:20.498+0000 7ff8a4585640 1 Processor -- start 2024-09-12T02:00:20.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:20.498+0000 7ff8a4585640 1 -- start start 2024-09-12T02:00:20.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:20.498+0000 7ff8a4585640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff89c100480 0x7ff89c0fc690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:20.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:20.498+0000 7ff8a4585640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff89c100e30 0x7ff89c0fcbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:20.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:20.498+0000 7ff8a4585640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff89c1055b0 0x7ff89c0fd110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:20.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:20.498+0000 7ff8a4585640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff89c10fdd0 con 0x7ff89c100480 2024-09-12T02:00:20.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:20.498+0000 7ff8a4585640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff89c10fc50 con 0x7ff89c100e30 2024-09-12T02:00:20.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:20.498+0000 7ff8a4585640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff89c10ff50 con 0x7ff89c1055b0 2024-09-12T02:00:20.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:20.498+0000 7ff8a2321640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff89c100480 0x7ff89c0fc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:20.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:20.498+0000 7ff8a2321640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff89c100480 0x7ff89c0fc690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50174/0 (socket says 172.21.15.16:50174) 2024-09-12T02:00:20.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:20.498+0000 7ff8a2321640 1 -- 172.21.15.16:0/537429102 learned_addr learned my addr 172.21.15.16:0/537429102 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:00:20.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:20.498+0000 7ff8a2b22640 1 --2- 172.21.15.16:0/537429102 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff89c1055b0 0x7ff89c0fd110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:20.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:20.498+0000 7ff8a2321640 1 -- 172.21.15.16:0/537429102 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff89c1055b0 msgr2=0x7ff89c0fd110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:20.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:20.498+0000 7ff8a2321640 1 --2- 172.21.15.16:0/537429102 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff89c1055b0 0x7ff89c0fd110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:20.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:20.498+0000 7ff8a2321640 1 -- 172.21.15.16:0/537429102 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff89c100e30 msgr2=0x7ff89c0fcbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:20.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:20.498+0000 7ff8a2321640 1 --2- 172.21.15.16:0/537429102 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff89c100e30 0x7ff89c0fcbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:20.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:20.498+0000 7ff8a2321640 1 -- 172.21.15.16:0/537429102 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff89c1a6960 con 0x7ff89c100480 2024-09-12T02:00:20.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:20.499+0000 7ff8a2321640 1 --2- 172.21.15.16:0/537429102 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff89c100480 0x7ff89c0fc690 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ff89800ece0 tx=0x7ff89800c7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:00:20.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:20.499+0000 7ff88f7fe640 1 -- 172.21.15.16:0/537429102 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff898004d20 con 0x7ff89c100480 2024-09-12T02:00:20.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:20.499+0000 7ff8a4585640 1 -- 172.21.15.16:0/537429102 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff89c1a6c50 con 0x7ff89c100480 2024-09-12T02:00:20.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:20.499+0000 7ff88f7fe640 1 -- 172.21.15.16:0/537429102 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff89800ba50 con 0x7ff89c100480 2024-09-12T02:00:20.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:20.499+0000 7ff88f7fe640 1 -- 172.21.15.16:0/537429102 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff898005740 con 0x7ff89c100480 2024-09-12T02:00:20.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:20.499+0000 7ff8a4585640 1 -- 172.21.15.16:0/537429102 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff89c1a7140 con 0x7ff89c100480 2024-09-12T02:00:20.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:20.500+0000 7ff8a4585640 1 -- 172.21.15.16:0/537429102 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff89c100860 con 0x7ff89c100480 2024-09-12T02:00:20.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:20.501+0000 7ff88f7fe640 1 -- 172.21.15.16:0/537429102 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7ff8980058e0 con 0x7ff89c100480 2024-09-12T02:00:20.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:20.502+0000 7ff88f7fe640 1 --2- 172.21.15.16:0/537429102 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7ff8880a2c10 0x7ff8880a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:20.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:20.502+0000 7ff8a1b20640 1 --2- 172.21.15.16:0/537429102 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7ff8880a2c10 0x7ff8880a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:20.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:20.502+0000 7ff88f7fe640 1 -- 172.21.15.16:0/537429102 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7ff8980cf6f0 con 0x7ff89c100480 2024-09-12T02:00:20.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:20.503+0000 7ff8a1b20640 1 --2- 172.21.15.16:0/537429102 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7ff8880a2c10 0x7ff8880a50d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ff89c0fe120 tx=0x7ff890006c90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:00:20.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:20.507+0000 7ff88f7fe640 1 -- 172.21.15.16:0/537429102 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff89809c5f0 con 0x7ff89c100480 2024-09-12T02:00:20.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:20.679+0000 7ff8a4585640 1 -- 172.21.15.16:0/537429102 --> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff89c062630 con 0x7ff8880a2c10 2024-09-12T02:00:20.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:20.686+0000 7ff88f7fe640 1 -- 172.21.15.16:0/537429102 <== mgr.14150 v2:172.21.15.16:6800/3887444820 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258696 (secure 0 0 0) 0x7ff89c062630 con 0x7ff8880a2c10 2024-09-12T02:00:20.690 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T02:00:20.692 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-12T02:00:20.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:20.690+0000 7ff8a4585640 1 -- 172.21.15.16:0/537429102 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7ff8880a2c10 msgr2=0x7ff8880a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:20.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:20.690+0000 7ff8a4585640 1 --2- 172.21.15.16:0/537429102 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7ff8880a2c10 0x7ff8880a50d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ff89c0fe120 tx=0x7ff890006c90 comp rx=0 tx=0).stop 2024-09-12T02:00:20.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:20.691+0000 7ff8a4585640 1 -- 172.21.15.16:0/537429102 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff89c100480 msgr2=0x7ff89c0fc690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:20.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:20.691+0000 7ff8a4585640 1 --2- 172.21.15.16:0/537429102 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff89c100480 0x7ff89c0fc690 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ff89800ece0 tx=0x7ff89800c7a0 comp rx=0 tx=0).stop 2024-09-12T02:00:20.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:20.691+0000 7ff8a4585640 1 -- 172.21.15.16:0/537429102 shutdown_connections 2024-09-12T02:00:20.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:20.691+0000 7ff8a4585640 1 --2- 172.21.15.16:0/537429102 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7ff8880a2c10 0x7ff8880a50d0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:20.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:20.691+0000 7ff8a4585640 1 --2- 172.21.15.16:0/537429102 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff89c1055b0 0x7ff89c0fd110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:20.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:20.691+0000 7ff8a4585640 1 --2- 172.21.15.16:0/537429102 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff89c100e30 0x7ff89c0fcbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:20.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:20.691+0000 7ff8a4585640 1 --2- 172.21.15.16:0/537429102 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff89c100480 0x7ff89c0fc690 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:20.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:20.691+0000 7ff8a4585640 1 -- 172.21.15.16:0/537429102 >> 172.21.15.16:0/537429102 conn(0x7ff89c077210 msgr2=0x7ff89c103bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:00:20.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:20.691+0000 7ff8a4585640 1 -- 172.21.15.16:0/537429102 shutdown_connections 2024-09-12T02:00:20.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:20.691+0000 7ff8a4585640 1 -- 172.21.15.16:0/537429102 wait complete. 2024-09-12T02:00:21.460 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":288,"stamp":"2024-09-12T02:00:20.304575+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":644717,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":233,"num_read_kb":221,"num_write":337,"num_write_kb":3036,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":1180848,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":271,"ondisk_log_size":271,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":374240,"kb_used_data":12492,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124567584,"statfs":{"total":1151940427776,"available":1151557206016,"internally_reserved":0,"allocated":12791808,"data_stored":7138721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":370193723},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":49127,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":-8,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":82,"num_read_kb":92,"num_write":31,"num_write_kb":60,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.743464"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.572370+0000","last_change":"2024-09-12T01:59:55.503882+0000","last_active":"2024-09-12T02:00:09.572370+0000","last_peered":"2024-09-12T02:00:09.572370+0000","last_clean":"2024-09-12T02:00:09.572370+0000","last_became_active":"2024-09-12T01:59:55.503513+0000","last_became_peered":"2024-09-12T01:59:55.503513+0000","last_unstale":"2024-09-12T02:00:09.572370+0000","last_undegraded":"2024-09-12T02:00:09.572370+0000","last_fullsized":"2024-09-12T02:00:09.572370+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:43:13.098527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.573809+0000","last_change":"2024-09-12T01:59:55.492965+0000","last_active":"2024-09-12T02:00:09.573809+0000","last_peered":"2024-09-12T02:00:09.573809+0000","last_clean":"2024-09-12T02:00:09.573809+0000","last_became_active":"2024-09-12T01:59:55.492613+0000","last_became_peered":"2024-09-12T01:59:55.492613+0000","last_unstale":"2024-09-12T02:00:09.573809+0000","last_undegraded":"2024-09-12T02:00:09.573809+0000","last_fullsized":"2024-09-12T02:00:09.573809+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:58:20.889770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.573872+0000","last_change":"2024-09-12T01:59:55.497334+0000","last_active":"2024-09-12T02:00:09.573872+0000","last_peered":"2024-09-12T02:00:09.573872+0000","last_clean":"2024-09-12T02:00:09.573872+0000","last_became_active":"2024-09-12T01:59:55.497037+0000","last_became_peered":"2024-09-12T01:59:55.497037+0000","last_unstale":"2024-09-12T02:00:09.573872+0000","last_undegraded":"2024-09-12T02:00:09.573872+0000","last_fullsized":"2024-09-12T02:00:09.573872+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:43:04.082918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.563839+0000","last_change":"2024-09-12T01:59:55.495888+0000","last_active":"2024-09-12T02:00:09.563839+0000","last_peered":"2024-09-12T02:00:09.563839+0000","last_clean":"2024-09-12T02:00:09.563839+0000","last_became_active":"2024-09-12T01:59:55.492684+0000","last_became_peered":"2024-09-12T01:59:55.492684+0000","last_unstale":"2024-09-12T02:00:09.563839+0000","last_undegraded":"2024-09-12T02:00:09.563839+0000","last_fullsized":"2024-09-12T02:00:09.563839+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:15:00.500006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.574662+0000","last_change":"2024-09-12T01:59:55.494613+0000","last_active":"2024-09-12T02:00:09.574662+0000","last_peered":"2024-09-12T02:00:09.574662+0000","last_clean":"2024-09-12T02:00:09.574662+0000","last_became_active":"2024-09-12T01:59:55.494205+0000","last_became_peered":"2024-09-12T01:59:55.494205+0000","last_unstale":"2024-09-12T02:00:09.574662+0000","last_undegraded":"2024-09-12T02:00:09.574662+0000","last_fullsized":"2024-09-12T02:00:09.574662+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:39:14.573280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.622709+0000","last_change":"2024-09-12T01:59:55.496659+0000","last_active":"2024-09-12T02:00:09.622709+0000","last_peered":"2024-09-12T02:00:09.622709+0000","last_clean":"2024-09-12T02:00:09.622709+0000","last_became_active":"2024-09-12T01:59:55.496232+0000","last_became_peered":"2024-09-12T01:59:55.496232+0000","last_unstale":"2024-09-12T02:00:09.622709+0000","last_undegraded":"2024-09-12T02:00:09.622709+0000","last_fullsized":"2024-09-12T02:00:09.622709+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:48:21.544764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.622510+0000","last_change":"2024-09-12T01:59:55.486789+0000","last_active":"2024-09-12T02:00:09.622510+0000","last_peered":"2024-09-12T02:00:09.622510+0000","last_clean":"2024-09-12T02:00:09.622510+0000","last_became_active":"2024-09-12T01:59:55.485921+0000","last_became_peered":"2024-09-12T01:59:55.485921+0000","last_unstale":"2024-09-12T02:00:09.622510+0000","last_undegraded":"2024-09-12T02:00:09.622510+0000","last_fullsized":"2024-09-12T02:00:09.622510+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:21:16.071121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.621086+0000","last_change":"2024-09-12T01:59:55.494525+0000","last_active":"2024-09-12T02:00:09.621086+0000","last_peered":"2024-09-12T02:00:09.621086+0000","last_clean":"2024-09-12T02:00:09.621086+0000","last_became_active":"2024-09-12T01:59:55.493998+0000","last_became_peered":"2024-09-12T01:59:55.493998+0000","last_unstale":"2024-09-12T02:00:09.621086+0000","last_undegraded":"2024-09-12T02:00:09.621086+0000","last_fullsized":"2024-09-12T02:00:09.621086+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:59:24.653660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.622633+0000","last_change":"2024-09-12T01:59:55.501466+0000","last_active":"2024-09-12T02:00:09.622633+0000","last_peered":"2024-09-12T02:00:09.622633+0000","last_clean":"2024-09-12T02:00:09.622633+0000","last_became_active":"2024-09-12T01:59:55.500800+0000","last_became_peered":"2024-09-12T01:59:55.500800+0000","last_unstale":"2024-09-12T02:00:09.622633+0000","last_undegraded":"2024-09-12T02:00:09.622633+0000","last_fullsized":"2024-09-12T02:00:09.622633+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:55:13.704791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.623367+0000","last_change":"2024-09-12T01:59:55.497986+0000","last_active":"2024-09-12T02:00:09.623367+0000","last_peered":"2024-09-12T02:00:09.623367+0000","last_clean":"2024-09-12T02:00:09.623367+0000","last_became_active":"2024-09-12T01:59:55.497691+0000","last_became_peered":"2024-09-12T01:59:55.497691+0000","last_unstale":"2024-09-12T02:00:09.623367+0000","last_undegraded":"2024-09-12T02:00:09.623367+0000","last_fullsized":"2024-09-12T02:00:09.623367+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:37:07.702606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.622801+0000","last_change":"2024-09-12T01:59:55.501339+0000","last_active":"2024-09-12T02:00:09.622801+0000","last_peered":"2024-09-12T02:00:09.622801+0000","last_clean":"2024-09-12T02:00:09.622801+0000","last_became_active":"2024-09-12T01:59:55.500700+0000","last_became_peered":"2024-09-12T01:59:55.500700+0000","last_unstale":"2024-09-12T02:00:09.622801+0000","last_undegraded":"2024-09-12T02:00:09.622801+0000","last_fullsized":"2024-09-12T02:00:09.622801+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:12:15.879183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.563946+0000","last_change":"2024-09-12T01:59:55.495196+0000","last_active":"2024-09-12T02:00:09.563946+0000","last_peered":"2024-09-12T02:00:09.563946+0000","last_clean":"2024-09-12T02:00:09.563946+0000","last_became_active":"2024-09-12T01:59:55.494889+0000","last_became_peered":"2024-09-12T01:59:55.494889+0000","last_unstale":"2024-09-12T02:00:09.563946+0000","last_undegraded":"2024-09-12T02:00:09.563946+0000","last_fullsized":"2024-09-12T02:00:09.563946+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:32:30.621946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.562435+0000","last_change":"2024-09-12T01:59:55.492565+0000","last_active":"2024-09-12T02:00:09.562435+0000","last_peered":"2024-09-12T02:00:09.562435+0000","last_clean":"2024-09-12T02:00:09.562435+0000","last_became_active":"2024-09-12T01:59:55.492168+0000","last_became_peered":"2024-09-12T01:59:55.492168+0000","last_unstale":"2024-09-12T02:00:09.562435+0000","last_undegraded":"2024-09-12T02:00:09.562435+0000","last_fullsized":"2024-09-12T02:00:09.562435+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:39:48.630202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.573885+0000","last_change":"2024-09-12T01:59:55.493152+0000","last_active":"2024-09-12T02:00:09.573885+0000","last_peered":"2024-09-12T02:00:09.573885+0000","last_clean":"2024-09-12T02:00:09.573885+0000","last_became_active":"2024-09-12T01:59:55.492794+0000","last_became_peered":"2024-09-12T01:59:55.492794+0000","last_unstale":"2024-09-12T02:00:09.573885+0000","last_undegraded":"2024-09-12T02:00:09.573885+0000","last_fullsized":"2024-09-12T02:00:09.573885+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:24:27.270502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.573561+0000","last_change":"2024-09-12T01:59:55.494651+0000","last_active":"2024-09-12T02:00:09.573561+0000","last_peered":"2024-09-12T02:00:09.573561+0000","last_clean":"2024-09-12T02:00:09.573561+0000","last_became_active":"2024-09-12T01:59:55.494355+0000","last_became_peered":"2024-09-12T01:59:55.494355+0000","last_unstale":"2024-09-12T02:00:09.573561+0000","last_undegraded":"2024-09-12T02:00:09.573561+0000","last_fullsized":"2024-09-12T02:00:09.573561+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:14:44.364158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.622950+0000","last_change":"2024-09-12T01:59:55.495049+0000","last_active":"2024-09-12T02:00:09.622950+0000","last_peered":"2024-09-12T02:00:09.622950+0000","last_clean":"2024-09-12T02:00:09.622950+0000","last_became_active":"2024-09-12T01:59:55.494688+0000","last_became_peered":"2024-09-12T01:59:55.494688+0000","last_unstale":"2024-09-12T02:00:09.622950+0000","last_undegraded":"2024-09-12T02:00:09.622950+0000","last_fullsized":"2024-09-12T02:00:09.622950+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:13:05.944577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.572458+0000","last_change":"2024-09-12T01:59:55.504146+0000","last_active":"2024-09-12T02:00:09.572458+0000","last_peered":"2024-09-12T02:00:09.572458+0000","last_clean":"2024-09-12T02:00:09.572458+0000","last_became_active":"2024-09-12T01:59:55.503646+0000","last_became_peered":"2024-09-12T01:59:55.503646+0000","last_unstale":"2024-09-12T02:00:09.572458+0000","last_undegraded":"2024-09-12T02:00:09.572458+0000","last_fullsized":"2024-09-12T02:00:09.572458+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:23:51.780795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.565050+0000","last_change":"2024-09-12T01:59:55.489930+0000","last_active":"2024-09-12T02:00:09.565050+0000","last_peered":"2024-09-12T02:00:09.565050+0000","last_clean":"2024-09-12T02:00:09.565050+0000","last_became_active":"2024-09-12T01:59:55.489619+0000","last_became_peered":"2024-09-12T01:59:55.489619+0000","last_unstale":"2024-09-12T02:00:09.565050+0000","last_undegraded":"2024-09-12T02:00:09.565050+0000","last_fullsized":"2024-09-12T02:00:09.565050+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:04:57.307091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.569107+0000","last_change":"2024-09-12T01:59:55.492098+0000","last_active":"2024-09-12T02:00:09.569107+0000","last_peered":"2024-09-12T02:00:09.569107+0000","last_clean":"2024-09-12T02:00:09.569107+0000","last_became_active":"2024-09-12T01:59:55.491370+0000","last_became_peered":"2024-09-12T01:59:55.491370+0000","last_unstale":"2024-09-12T02:00:09.569107+0000","last_undegraded":"2024-09-12T02:00:09.569107+0000","last_fullsized":"2024-09-12T02:00:09.569107+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:25:06.307903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.571940+0000","last_change":"2024-09-12T01:59:55.499938+0000","last_active":"2024-09-12T02:00:09.571940+0000","last_peered":"2024-09-12T02:00:09.571940+0000","last_clean":"2024-09-12T02:00:09.571940+0000","last_became_active":"2024-09-12T01:59:55.499619+0000","last_became_peered":"2024-09-12T01:59:55.499619+0000","last_unstale":"2024-09-12T02:00:09.571940+0000","last_undegraded":"2024-09-12T02:00:09.571940+0000","last_fullsized":"2024-09-12T02:00:09.571940+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:00:17.536120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.562514+0000","last_change":"2024-09-12T01:59:55.486106+0000","last_active":"2024-09-12T02:00:09.562514+0000","last_peered":"2024-09-12T02:00:09.562514+0000","last_clean":"2024-09-12T02:00:09.562514+0000","last_became_active":"2024-09-12T01:59:55.485325+0000","last_became_peered":"2024-09-12T01:59:55.485325+0000","last_unstale":"2024-09-12T02:00:09.562514+0000","last_undegraded":"2024-09-12T02:00:09.562514+0000","last_fullsized":"2024-09-12T02:00:09.562514+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:35:46.446224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.622237+0000","last_change":"2024-09-12T01:59:55.496438+0000","last_active":"2024-09-12T02:00:09.622237+0000","last_peered":"2024-09-12T02:00:09.622237+0000","last_clean":"2024-09-12T02:00:09.622237+0000","last_became_active":"2024-09-12T01:59:55.495885+0000","last_became_peered":"2024-09-12T01:59:55.495885+0000","last_unstale":"2024-09-12T02:00:09.622237+0000","last_undegraded":"2024-09-12T02:00:09.622237+0000","last_fullsized":"2024-09-12T02:00:09.622237+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:15:14.911335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.621139+0000","last_change":"2024-09-12T01:59:55.489350+0000","last_active":"2024-09-12T02:00:09.621139+0000","last_peered":"2024-09-12T02:00:09.621139+0000","last_clean":"2024-09-12T02:00:09.621139+0000","last_became_active":"2024-09-12T01:59:55.489028+0000","last_became_peered":"2024-09-12T01:59:55.489028+0000","last_unstale":"2024-09-12T02:00:09.621139+0000","last_undegraded":"2024-09-12T02:00:09.621139+0000","last_fullsized":"2024-09-12T02:00:09.621139+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:29:23.769004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.566645+0000","last_change":"2024-09-12T01:59:55.494532+0000","last_active":"2024-09-12T02:00:09.566645+0000","last_peered":"2024-09-12T02:00:09.566645+0000","last_clean":"2024-09-12T02:00:09.566645+0000","last_became_active":"2024-09-12T01:59:55.490982+0000","last_became_peered":"2024-09-12T01:59:55.490982+0000","last_unstale":"2024-09-12T02:00:09.566645+0000","last_undegraded":"2024-09-12T02:00:09.566645+0000","last_fullsized":"2024-09-12T02:00:09.566645+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:04:50.633730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.621490+0000","last_change":"2024-09-12T01:59:55.496057+0000","last_active":"2024-09-12T02:00:09.621490+0000","last_peered":"2024-09-12T02:00:09.621490+0000","last_clean":"2024-09-12T02:00:09.621490+0000","last_became_active":"2024-09-12T01:59:55.495710+0000","last_became_peered":"2024-09-12T01:59:55.495710+0000","last_unstale":"2024-09-12T02:00:09.621490+0000","last_undegraded":"2024-09-12T02:00:09.621490+0000","last_fullsized":"2024-09-12T02:00:09.621490+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:49:01.962537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.569100+0000","last_change":"2024-09-12T01:59:55.491594+0000","last_active":"2024-09-12T02:00:09.569100+0000","last_peered":"2024-09-12T02:00:09.569100+0000","last_clean":"2024-09-12T02:00:09.569100+0000","last_became_active":"2024-09-12T01:59:55.491233+0000","last_became_peered":"2024-09-12T01:59:55.491233+0000","last_unstale":"2024-09-12T02:00:09.569100+0000","last_undegraded":"2024-09-12T02:00:09.569100+0000","last_fullsized":"2024-09-12T02:00:09.569100+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:24:22.066777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.572270+0000","last_change":"2024-09-12T01:59:55.498331+0000","last_active":"2024-09-12T02:00:09.572270+0000","last_peered":"2024-09-12T02:00:09.572270+0000","last_clean":"2024-09-12T02:00:09.572270+0000","last_became_active":"2024-09-12T01:59:55.497983+0000","last_became_peered":"2024-09-12T01:59:55.497983+0000","last_unstale":"2024-09-12T02:00:09.572270+0000","last_undegraded":"2024-09-12T02:00:09.572270+0000","last_fullsized":"2024-09-12T02:00:09.572270+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:37:00.735231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.575358+0000","last_change":"2024-09-12T02:00:09.575358+0000","last_active":"2024-09-12T02:00:09.575358+0000","last_peered":"2024-09-12T02:00:09.575358+0000","last_clean":"2024-09-12T02:00:09.575358+0000","last_became_active":"2024-09-12T02:00:09.574913+0000","last_became_peered":"2024-09-12T02:00:09.574913+0000","last_unstale":"2024-09-12T02:00:09.575358+0000","last_undegraded":"2024-09-12T02:00:09.575358+0000","last_fullsized":"2024-09-12T02:00:09.575358+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:19:53.630453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.573748+0000","last_change":"2024-09-12T01:59:55.492419+0000","last_active":"2024-09-12T02:00:09.573748+0000","last_peered":"2024-09-12T02:00:09.573748+0000","last_clean":"2024-09-12T02:00:09.573748+0000","last_became_active":"2024-09-12T01:59:55.492003+0000","last_became_peered":"2024-09-12T01:59:55.492003+0000","last_unstale":"2024-09-12T02:00:09.573748+0000","last_undegraded":"2024-09-12T02:00:09.573748+0000","last_fullsized":"2024-09-12T02:00:09.573748+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:25:45.774679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.568801+0000","last_change":"2024-09-12T01:59:55.490125+0000","last_active":"2024-09-12T02:00:09.568801+0000","last_peered":"2024-09-12T02:00:09.568801+0000","last_clean":"2024-09-12T02:00:09.568801+0000","last_became_active":"2024-09-12T01:59:55.489636+0000","last_became_peered":"2024-09-12T01:59:55.489636+0000","last_unstale":"2024-09-12T02:00:09.568801+0000","last_undegraded":"2024-09-12T02:00:09.568801+0000","last_fullsized":"2024-09-12T02:00:09.568801+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:41:30.922185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.569135+0000","last_change":"2024-09-12T01:59:55.496322+0000","last_active":"2024-09-12T02:00:09.569135+0000","last_peered":"2024-09-12T02:00:09.569135+0000","last_clean":"2024-09-12T02:00:09.569135+0000","last_became_active":"2024-09-12T01:59:55.496005+0000","last_became_peered":"2024-09-12T01:59:55.496005+0000","last_unstale":"2024-09-12T02:00:09.569135+0000","last_undegraded":"2024-09-12T02:00:09.569135+0000","last_fullsized":"2024-09-12T02:00:09.569135+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:53:39.248715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.572017+0000","last_change":"2024-09-12T01:59:55.497758+0000","last_active":"2024-09-12T02:00:09.572017+0000","last_peered":"2024-09-12T02:00:09.572017+0000","last_clean":"2024-09-12T02:00:09.572017+0000","last_became_active":"2024-09-12T01:59:55.497282+0000","last_became_peered":"2024-09-12T01:59:55.497282+0000","last_unstale":"2024-09-12T02:00:09.572017+0000","last_undegraded":"2024-09-12T02:00:09.572017+0000","last_fullsized":"2024-09-12T02:00:09.572017+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:08:25.509372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.574620+0000","last_change":"2024-09-12T01:59:55.497919+0000","last_active":"2024-09-12T02:00:09.574620+0000","last_peered":"2024-09-12T02:00:09.574620+0000","last_clean":"2024-09-12T02:00:09.574620+0000","last_became_active":"2024-09-12T01:59:55.497609+0000","last_became_peered":"2024-09-12T01:59:55.497609+0000","last_unstale":"2024-09-12T02:00:09.574620+0000","last_undegraded":"2024-09-12T02:00:09.574620+0000","last_fullsized":"2024-09-12T02:00:09.574620+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:10:22.281180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"81'6","reported_seq":42,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:14.567314+0000","last_change":"2024-09-12T01:59:53.473974+0000","last_active":"2024-09-12T02:00:14.567314+0000","last_peered":"2024-09-12T02:00:14.567314+0000","last_clean":"2024-09-12T02:00:14.567314+0000","last_became_active":"2024-09-12T01:59:53.473546+0000","last_became_peered":"2024-09-12T01:59:53.473546+0000","last_unstale":"2024-09-12T02:00:14.567314+0000","last_undegraded":"2024-09-12T02:00:14.567314+0000","last_fullsized":"2024-09-12T02:00:14.567314+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:28:03.532249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.567217+0000","last_change":"2024-09-12T01:59:55.493197+0000","last_active":"2024-09-12T02:00:09.567217+0000","last_peered":"2024-09-12T02:00:09.567217+0000","last_clean":"2024-09-12T02:00:09.567217+0000","last_became_active":"2024-09-12T01:59:55.492845+0000","last_became_peered":"2024-09-12T01:59:55.492845+0000","last_unstale":"2024-09-12T02:00:09.567217+0000","last_undegraded":"2024-09-12T02:00:09.567217+0000","last_fullsized":"2024-09-12T02:00:09.567217+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:42:50.790896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"80'3","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.622865+0000","last_change":"2024-09-12T01:59:53.471214+0000","last_active":"2024-09-12T02:00:09.622865+0000","last_peered":"2024-09-12T02:00:09.622865+0000","last_clean":"2024-09-12T02:00:09.622865+0000","last_became_active":"2024-09-12T01:59:53.470858+0000","last_became_peered":"2024-09-12T01:59:53.470858+0000","last_unstale":"2024-09-12T02:00:09.622865+0000","last_undegraded":"2024-09-12T02:00:09.622865+0000","last_fullsized":"2024-09-12T02:00:09.622865+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:34:48.730392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"80'2","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.562106+0000","last_change":"2024-09-12T01:59:53.464568+0000","last_active":"2024-09-12T02:00:09.562106+0000","last_peered":"2024-09-12T02:00:09.562106+0000","last_clean":"2024-09-12T02:00:09.562106+0000","last_became_active":"2024-09-12T01:59:53.464168+0000","last_became_peered":"2024-09-12T01:59:53.464168+0000","last_unstale":"2024-09-12T02:00:09.562106+0000","last_undegraded":"2024-09-12T02:00:09.562106+0000","last_fullsized":"2024-09-12T02:00:09.562106+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:41:48.135522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.621151+0000","last_change":"2024-09-12T01:59:55.487177+0000","last_active":"2024-09-12T02:00:09.621151+0000","last_peered":"2024-09-12T02:00:09.621151+0000","last_clean":"2024-09-12T02:00:09.621151+0000","last_became_active":"2024-09-12T01:59:55.486886+0000","last_became_peered":"2024-09-12T01:59:55.486886+0000","last_unstale":"2024-09-12T02:00:09.621151+0000","last_undegraded":"2024-09-12T02:00:09.621151+0000","last_fullsized":"2024-09-12T02:00:09.621151+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:32:40.150071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"80'1","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.567129+0000","last_change":"2024-09-12T01:59:53.474114+0000","last_active":"2024-09-12T02:00:09.567129+0000","last_peered":"2024-09-12T02:00:09.567129+0000","last_clean":"2024-09-12T02:00:09.567129+0000","last_became_active":"2024-09-12T01:59:53.473755+0000","last_became_peered":"2024-09-12T01:59:53.473755+0000","last_unstale":"2024-09-12T02:00:09.567129+0000","last_undegraded":"2024-09-12T02:00:09.567129+0000","last_fullsized":"2024-09-12T02:00:09.567129+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:07:07.021137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.564793+0000","last_change":"2024-09-12T01:59:55.489868+0000","last_active":"2024-09-12T02:00:09.564793+0000","last_peered":"2024-09-12T02:00:09.564793+0000","last_clean":"2024-09-12T02:00:09.564793+0000","last_became_active":"2024-09-12T01:59:55.489537+0000","last_became_peered":"2024-09-12T01:59:55.489537+0000","last_unstale":"2024-09-12T02:00:09.564793+0000","last_undegraded":"2024-09-12T02:00:09.564793+0000","last_fullsized":"2024-09-12T02:00:09.564793+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:18:49.417689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"80'3","reported_seq":39,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:17.571161+0000","last_change":"2024-09-12T01:59:53.471402+0000","last_active":"2024-09-12T02:00:17.571161+0000","last_peered":"2024-09-12T02:00:17.571161+0000","last_clean":"2024-09-12T02:00:17.571161+0000","last_became_active":"2024-09-12T01:59:53.470940+0000","last_became_peered":"2024-09-12T01:59:53.470940+0000","last_unstale":"2024-09-12T02:00:17.571161+0000","last_undegraded":"2024-09-12T02:00:17.571161+0000","last_fullsized":"2024-09-12T02:00:17.571161+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:41:16.853592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.623260+0000","last_change":"2024-09-12T01:59:55.499047+0000","last_active":"2024-09-12T02:00:09.623260+0000","last_peered":"2024-09-12T02:00:09.623260+0000","last_clean":"2024-09-12T02:00:09.623260+0000","last_became_active":"2024-09-12T01:59:55.498071+0000","last_became_peered":"2024-09-12T01:59:55.498071+0000","last_unstale":"2024-09-12T02:00:09.623260+0000","last_undegraded":"2024-09-12T02:00:09.623260+0000","last_fullsized":"2024-09-12T02:00:09.623260+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:36:40.106041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"80'2","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.622966+0000","last_change":"2024-09-12T01:59:53.473508+0000","last_active":"2024-09-12T02:00:09.622966+0000","last_peered":"2024-09-12T02:00:09.622966+0000","last_clean":"2024-09-12T02:00:09.622966+0000","last_became_active":"2024-09-12T01:59:53.473063+0000","last_became_peered":"2024-09-12T01:59:53.473063+0000","last_unstale":"2024-09-12T02:00:09.622966+0000","last_undegraded":"2024-09-12T02:00:09.622966+0000","last_fullsized":"2024-09-12T02:00:09.622966+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:55:32.161769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.573961+0000","last_change":"2024-09-12T01:59:55.491721+0000","last_active":"2024-09-12T02:00:09.573961+0000","last_peered":"2024-09-12T02:00:09.573961+0000","last_clean":"2024-09-12T02:00:09.573961+0000","last_became_active":"2024-09-12T01:59:55.491364+0000","last_became_peered":"2024-09-12T01:59:55.491364+0000","last_unstale":"2024-09-12T02:00:09.573961+0000","last_undegraded":"2024-09-12T02:00:09.573961+0000","last_fullsized":"2024-09-12T02:00:09.573961+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:21:20.795420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"80'4","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.567315+0000","last_change":"2024-09-12T01:59:53.921789+0000","last_active":"2024-09-12T02:00:09.567315+0000","last_peered":"2024-09-12T02:00:09.567315+0000","last_clean":"2024-09-12T02:00:09.567315+0000","last_became_active":"2024-09-12T01:59:53.921383+0000","last_became_peered":"2024-09-12T01:59:53.921383+0000","last_unstale":"2024-09-12T02:00:09.567315+0000","last_undegraded":"2024-09-12T02:00:09.567315+0000","last_fullsized":"2024-09-12T02:00:09.567315+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:21:58.792794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.622517+0000","last_change":"2024-09-12T01:59:55.503008+0000","last_active":"2024-09-12T02:00:09.622517+0000","last_peered":"2024-09-12T02:00:09.622517+0000","last_clean":"2024-09-12T02:00:09.622517+0000","last_became_active":"2024-09-12T01:59:55.502538+0000","last_became_peered":"2024-09-12T01:59:55.502538+0000","last_unstale":"2024-09-12T02:00:09.622517+0000","last_undegraded":"2024-09-12T02:00:09.622517+0000","last_fullsized":"2024-09-12T02:00:09.622517+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:51:36.735985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"82'25","reported_seq":81,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:18.575816+0000","last_change":"2024-09-12T01:59:53.470688+0000","last_active":"2024-09-12T02:00:18.575816+0000","last_peered":"2024-09-12T02:00:18.575816+0000","last_clean":"2024-09-12T02:00:18.575816+0000","last_became_active":"2024-09-12T01:59:53.470231+0000","last_became_peered":"2024-09-12T01:59:53.470231+0000","last_unstale":"2024-09-12T02:00:18.575816+0000","last_undegraded":"2024-09-12T02:00:18.575816+0000","last_fullsized":"2024-09-12T02:00:18.575816+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:53:54.439893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":50112,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":65,"num_write":26,"num_write_kb":56,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.575173+0000","last_change":"2024-09-12T01:59:55.495343+0000","last_active":"2024-09-12T02:00:09.575173+0000","last_peered":"2024-09-12T02:00:09.575173+0000","last_clean":"2024-09-12T02:00:09.575173+0000","last_became_active":"2024-09-12T01:59:55.495006+0000","last_became_peered":"2024-09-12T01:59:55.495006+0000","last_unstale":"2024-09-12T02:00:09.575173+0000","last_undegraded":"2024-09-12T02:00:09.575173+0000","last_fullsized":"2024-09-12T02:00:09.575173+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:32:20.932551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'6","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.569082+0000","last_change":"2024-09-12T01:59:53.468625+0000","last_active":"2024-09-12T02:00:09.569082+0000","last_peered":"2024-09-12T02:00:09.569082+0000","last_clean":"2024-09-12T02:00:09.569082+0000","last_became_active":"2024-09-12T01:59:53.468015+0000","last_became_peered":"2024-09-12T01:59:53.468015+0000","last_unstale":"2024-09-12T02:00:09.569082+0000","last_undegraded":"2024-09-12T02:00:09.569082+0000","last_fullsized":"2024-09-12T02:00:09.569082+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:49:34.842239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.562155+0000","last_change":"2024-09-12T01:59:55.487962+0000","last_active":"2024-09-12T02:00:09.562155+0000","last_peered":"2024-09-12T02:00:09.562155+0000","last_clean":"2024-09-12T02:00:09.562155+0000","last_became_active":"2024-09-12T01:59:55.487619+0000","last_became_peered":"2024-09-12T01:59:55.487619+0000","last_unstale":"2024-09-12T02:00:09.562155+0000","last_undegraded":"2024-09-12T02:00:09.562155+0000","last_fullsized":"2024-09-12T02:00:09.562155+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:32:16.882373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"80'3","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.567124+0000","last_change":"2024-09-12T01:59:53.474418+0000","last_active":"2024-09-12T02:00:09.567124+0000","last_peered":"2024-09-12T02:00:09.567124+0000","last_clean":"2024-09-12T02:00:09.567124+0000","last_became_active":"2024-09-12T01:59:53.474059+0000","last_became_peered":"2024-09-12T01:59:53.474059+0000","last_unstale":"2024-09-12T02:00:09.567124+0000","last_undegraded":"2024-09-12T02:00:09.567124+0000","last_fullsized":"2024-09-12T02:00:09.567124+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:10:54.963435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.622740+0000","last_change":"2024-09-12T01:59:55.493394+0000","last_active":"2024-09-12T02:00:09.622740+0000","last_peered":"2024-09-12T02:00:09.622740+0000","last_clean":"2024-09-12T02:00:09.622740+0000","last_became_active":"2024-09-12T01:59:55.491669+0000","last_became_peered":"2024-09-12T01:59:55.491669+0000","last_unstale":"2024-09-12T02:00:09.622740+0000","last_undegraded":"2024-09-12T02:00:09.622740+0000","last_fullsized":"2024-09-12T02:00:09.622740+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:14:40.871154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"80'3","reported_seq":42,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:18.573828+0000","last_change":"2024-09-12T01:59:53.471623+0000","last_active":"2024-09-12T02:00:18.573828+0000","last_peered":"2024-09-12T02:00:18.573828+0000","last_clean":"2024-09-12T02:00:18.573828+0000","last_became_active":"2024-09-12T01:59:53.471294+0000","last_became_peered":"2024-09-12T01:59:53.471294+0000","last_unstale":"2024-09-12T02:00:18.573828+0000","last_undegraded":"2024-09-12T02:00:18.573828+0000","last_fullsized":"2024-09-12T02:00:18.573828+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:51:54.130883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":18,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.621910+0000","last_change":"2024-09-12T01:59:55.493082+0000","last_active":"2024-09-12T02:00:09.621910+0000","last_peered":"2024-09-12T02:00:09.621910+0000","last_clean":"2024-09-12T02:00:09.621910+0000","last_became_active":"2024-09-12T01:59:55.492778+0000","last_became_peered":"2024-09-12T01:59:55.492778+0000","last_unstale":"2024-09-12T02:00:09.621910+0000","last_undegraded":"2024-09-12T02:00:09.621910+0000","last_fullsized":"2024-09-12T02:00:09.621910+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:59:56.466377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'15","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.588050+0000","last_change":"2024-09-12T02:00:09.588050+0000","last_active":"2024-09-12T02:00:09.588050+0000","last_peered":"2024-09-12T02:00:09.588050+0000","last_clean":"2024-09-12T02:00:09.588050+0000","last_became_active":"2024-09-12T02:00:09.575478+0000","last_became_peered":"2024-09-12T02:00:09.575478+0000","last_unstale":"2024-09-12T02:00:09.588050+0000","last_undegraded":"2024-09-12T02:00:09.588050+0000","last_fullsized":"2024-09-12T02:00:09.588050+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:38:45.484167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.566976+0000","last_change":"2024-09-12T01:59:55.497601+0000","last_active":"2024-09-12T02:00:09.566976+0000","last_peered":"2024-09-12T02:00:09.566976+0000","last_clean":"2024-09-12T02:00:09.566976+0000","last_became_active":"2024-09-12T01:59:55.497202+0000","last_became_peered":"2024-09-12T01:59:55.497202+0000","last_unstale":"2024-09-12T02:00:09.566976+0000","last_undegraded":"2024-09-12T02:00:09.566976+0000","last_fullsized":"2024-09-12T02:00:09.566976+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:22:22.850736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"80'2","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.575359+0000","last_change":"2024-09-12T01:59:53.467575+0000","last_active":"2024-09-12T02:00:09.575359+0000","last_peered":"2024-09-12T02:00:09.575359+0000","last_clean":"2024-09-12T02:00:09.575359+0000","last_became_active":"2024-09-12T01:59:53.467146+0000","last_became_peered":"2024-09-12T01:59:53.467146+0000","last_unstale":"2024-09-12T02:00:09.575359+0000","last_undegraded":"2024-09-12T02:00:09.575359+0000","last_fullsized":"2024-09-12T02:00:09.575359+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:07:25.962496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.572034+0000","last_change":"2024-09-12T01:59:55.498297+0000","last_active":"2024-09-12T02:00:09.572034+0000","last_peered":"2024-09-12T02:00:09.572034+0000","last_clean":"2024-09-12T02:00:09.572034+0000","last_became_active":"2024-09-12T01:59:55.497367+0000","last_became_peered":"2024-09-12T01:59:55.497367+0000","last_unstale":"2024-09-12T02:00:09.572034+0000","last_undegraded":"2024-09-12T02:00:09.572034+0000","last_fullsized":"2024-09-12T02:00:09.572034+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:58:32.717710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"80'2","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.562292+0000","last_change":"2024-09-12T01:59:53.466664+0000","last_active":"2024-09-12T02:00:09.562292+0000","last_peered":"2024-09-12T02:00:09.562292+0000","last_clean":"2024-09-12T02:00:09.562292+0000","last_became_active":"2024-09-12T01:59:53.466318+0000","last_became_peered":"2024-09-12T01:59:53.466318+0000","last_unstale":"2024-09-12T02:00:09.562292+0000","last_undegraded":"2024-09-12T02:00:09.562292+0000","last_fullsized":"2024-09-12T02:00:09.562292+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:12:13.541420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.622804+0000","last_change":"2024-09-12T01:59:55.490420+0000","last_active":"2024-09-12T02:00:09.622804+0000","last_peered":"2024-09-12T02:00:09.622804+0000","last_clean":"2024-09-12T02:00:09.622804+0000","last_became_active":"2024-09-12T01:59:55.489757+0000","last_became_peered":"2024-09-12T01:59:55.489757+0000","last_unstale":"2024-09-12T02:00:09.622804+0000","last_undegraded":"2024-09-12T02:00:09.622804+0000","last_fullsized":"2024-09-12T02:00:09.622804+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:34:43.117142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'1","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.567185+0000","last_change":"2024-09-12T01:59:53.466399+0000","last_active":"2024-09-12T02:00:09.567185+0000","last_peered":"2024-09-12T02:00:09.567185+0000","last_clean":"2024-09-12T02:00:09.567185+0000","last_became_active":"2024-09-12T01:59:53.465982+0000","last_became_peered":"2024-09-12T01:59:53.465982+0000","last_unstale":"2024-09-12T02:00:09.567185+0000","last_undegraded":"2024-09-12T02:00:09.567185+0000","last_fullsized":"2024-09-12T02:00:09.567185+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:18:02.015040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"82'1","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.622907+0000","last_change":"2024-09-12T01:59:55.491527+0000","last_active":"2024-09-12T02:00:09.622907+0000","last_peered":"2024-09-12T02:00:09.622907+0000","last_clean":"2024-09-12T02:00:09.622907+0000","last_became_active":"2024-09-12T01:59:55.491165+0000","last_became_peered":"2024-09-12T01:59:55.491165+0000","last_unstale":"2024-09-12T02:00:09.622907+0000","last_undegraded":"2024-09-12T02:00:09.622907+0000","last_fullsized":"2024-09-12T02:00:09.622907+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:57:05.506384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"80'3","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.569187+0000","last_change":"2024-09-12T01:59:54.075752+0000","last_active":"2024-09-12T02:00:09.569187+0000","last_peered":"2024-09-12T02:00:09.569187+0000","last_clean":"2024-09-12T02:00:09.569187+0000","last_became_active":"2024-09-12T01:59:54.075387+0000","last_became_peered":"2024-09-12T01:59:54.075387+0000","last_unstale":"2024-09-12T02:00:09.569187+0000","last_undegraded":"2024-09-12T02:00:09.569187+0000","last_fullsized":"2024-09-12T02:00:09.569187+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:21:15.980591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.622907+0000","last_change":"2024-09-12T01:59:55.500935+0000","last_active":"2024-09-12T02:00:09.622907+0000","last_peered":"2024-09-12T02:00:09.622907+0000","last_clean":"2024-09-12T02:00:09.622907+0000","last_became_active":"2024-09-12T01:59:55.500206+0000","last_became_peered":"2024-09-12T01:59:55.500206+0000","last_unstale":"2024-09-12T02:00:09.622907+0000","last_undegraded":"2024-09-12T02:00:09.622907+0000","last_fullsized":"2024-09-12T02:00:09.622907+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:19:59.937701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":333,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.623202+0000","last_change":"2024-09-12T01:58:47.553853+0000","last_active":"2024-09-12T02:00:09.623202+0000","last_peered":"2024-09-12T02:00:09.623202+0000","last_clean":"2024-09-12T02:00:09.623202+0000","last_became_active":"2024-09-12T01:58:47.553357+0000","last_became_peered":"2024-09-12T01:58:47.553357+0000","last_unstale":"2024-09-12T02:00:09.623202+0000","last_undegraded":"2024-09-12T02:00:09.623202+0000","last_fullsized":"2024-09-12T02:00:09.623202+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:55:26.917733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:55:26.917733+0000","last_clean_scrub_stamp":"2024-09-12T01:55:26.917733+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:58:26.081766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"80'3","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.563615+0000","last_change":"2024-09-12T01:59:53.467907+0000","last_active":"2024-09-12T02:00:09.563615+0000","last_peered":"2024-09-12T02:00:09.563615+0000","last_clean":"2024-09-12T02:00:09.563615+0000","last_became_active":"2024-09-12T01:59:53.467164+0000","last_became_peered":"2024-09-12T01:59:53.467164+0000","last_unstale":"2024-09-12T02:00:09.563615+0000","last_undegraded":"2024-09-12T02:00:09.563615+0000","last_fullsized":"2024-09-12T02:00:09.563615+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:09:35.242708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.621290+0000","last_change":"2024-09-12T01:59:55.495109+0000","last_active":"2024-09-12T02:00:09.621290+0000","last_peered":"2024-09-12T02:00:09.621290+0000","last_clean":"2024-09-12T02:00:09.621290+0000","last_became_active":"2024-09-12T01:59:55.494791+0000","last_became_peered":"2024-09-12T01:59:55.494791+0000","last_unstale":"2024-09-12T02:00:09.621290+0000","last_undegraded":"2024-09-12T02:00:09.621290+0000","last_fullsized":"2024-09-12T02:00:09.621290+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:22:43.882050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"80'3","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.623115+0000","last_change":"2024-09-12T01:59:53.473286+0000","last_active":"2024-09-12T02:00:09.623115+0000","last_peered":"2024-09-12T02:00:09.623115+0000","last_clean":"2024-09-12T02:00:09.623115+0000","last_became_active":"2024-09-12T01:59:53.472976+0000","last_became_peered":"2024-09-12T01:59:53.472976+0000","last_unstale":"2024-09-12T02:00:09.623115+0000","last_undegraded":"2024-09-12T02:00:09.623115+0000","last_fullsized":"2024-09-12T02:00:09.623115+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:32:39.604173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.571159+0000","last_change":"2024-09-12T02:00:09.571159+0000","last_active":"2024-09-12T02:00:09.571159+0000","last_peered":"2024-09-12T02:00:09.571159+0000","last_clean":"2024-09-12T02:00:09.571159+0000","last_became_active":"2024-09-12T02:00:09.570672+0000","last_became_peered":"2024-09-12T02:00:09.570672+0000","last_unstale":"2024-09-12T02:00:09.571159+0000","last_undegraded":"2024-09-12T02:00:09.571159+0000","last_fullsized":"2024-09-12T02:00:09.571159+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:04:10.808942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"80'4","reported_seq":41,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:18.572794+0000","last_change":"2024-09-12T01:59:53.921856+0000","last_active":"2024-09-12T02:00:18.572794+0000","last_peered":"2024-09-12T02:00:18.572794+0000","last_clean":"2024-09-12T02:00:18.572794+0000","last_became_active":"2024-09-12T01:59:53.920672+0000","last_became_peered":"2024-09-12T01:59:53.920672+0000","last_unstale":"2024-09-12T02:00:18.572794+0000","last_undegraded":"2024-09-12T02:00:18.572794+0000","last_fullsized":"2024-09-12T02:00:18.572794+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:42:33.876493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.571600+0000","last_change":"2024-09-12T02:00:09.571185+0000","last_active":"2024-09-12T02:00:09.571600+0000","last_peered":"2024-09-12T02:00:09.571600+0000","last_clean":"2024-09-12T02:00:09.571600+0000","last_became_active":"2024-09-12T02:00:09.570769+0000","last_became_peered":"2024-09-12T02:00:09.570769+0000","last_unstale":"2024-09-12T02:00:09.571600+0000","last_undegraded":"2024-09-12T02:00:09.571600+0000","last_fullsized":"2024-09-12T02:00:09.571600+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:34:17.230435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.2","version":"80'1","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.622691+0000","last_change":"2024-09-12T01:59:53.472072+0000","last_active":"2024-09-12T02:00:09.622691+0000","last_peered":"2024-09-12T02:00:09.622691+0000","last_clean":"2024-09-12T02:00:09.622691+0000","last_became_active":"2024-09-12T01:59:53.471088+0000","last_became_peered":"2024-09-12T01:59:53.471088+0000","last_unstale":"2024-09-12T02:00:09.622691+0000","last_undegraded":"2024-09-12T02:00:09.622691+0000","last_fullsized":"2024-09-12T02:00:09.622691+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:02:13.879131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.622413+0000","last_change":"2024-09-12T01:59:55.502843+0000","last_active":"2024-09-12T02:00:09.622413+0000","last_peered":"2024-09-12T02:00:09.622413+0000","last_clean":"2024-09-12T02:00:09.622413+0000","last_became_active":"2024-09-12T01:59:55.499296+0000","last_became_peered":"2024-09-12T01:59:55.499296+0000","last_unstale":"2024-09-12T02:00:09.622413+0000","last_undegraded":"2024-09-12T02:00:09.622413+0000","last_fullsized":"2024-09-12T02:00:09.622413+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:04:21.787370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.575061+0000","last_change":"2024-09-12T02:00:09.575061+0000","last_active":"2024-09-12T02:00:09.575061+0000","last_peered":"2024-09-12T02:00:09.575061+0000","last_clean":"2024-09-12T02:00:09.575061+0000","last_became_active":"2024-09-12T02:00:09.574653+0000","last_became_peered":"2024-09-12T02:00:09.574653+0000","last_unstale":"2024-09-12T02:00:09.575061+0000","last_undegraded":"2024-09-12T02:00:09.575061+0000","last_fullsized":"2024-09-12T02:00:09.575061+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:32:27.931211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"82'1","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.622315+0000","last_change":"2024-09-12T01:59:55.494162+0000","last_active":"2024-09-12T02:00:09.622315+0000","last_peered":"2024-09-12T02:00:09.622315+0000","last_clean":"2024-09-12T02:00:09.622315+0000","last_became_active":"2024-09-12T01:59:55.493802+0000","last_became_peered":"2024-09-12T01:59:55.493802+0000","last_unstale":"2024-09-12T02:00:09.622315+0000","last_undegraded":"2024-09-12T02:00:09.622315+0000","last_fullsized":"2024-09-12T02:00:09.622315+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:22:16.302683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"80'1","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.575278+0000","last_change":"2024-09-12T01:59:53.921669+0000","last_active":"2024-09-12T02:00:09.575278+0000","last_peered":"2024-09-12T02:00:09.575278+0000","last_clean":"2024-09-12T02:00:09.575278+0000","last_became_active":"2024-09-12T01:59:53.921328+0000","last_became_peered":"2024-09-12T01:59:53.921328+0000","last_unstale":"2024-09-12T02:00:09.575278+0000","last_undegraded":"2024-09-12T02:00:09.575278+0000","last_fullsized":"2024-09-12T02:00:09.575278+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:46:33.313726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.562329+0000","last_change":"2024-09-12T01:59:55.488986+0000","last_active":"2024-09-12T02:00:09.562329+0000","last_peered":"2024-09-12T02:00:09.562329+0000","last_clean":"2024-09-12T02:00:09.562329+0000","last_became_active":"2024-09-12T01:59:55.488632+0000","last_became_peered":"2024-09-12T01:59:55.488632+0000","last_unstale":"2024-09-12T02:00:09.562329+0000","last_undegraded":"2024-09-12T02:00:09.562329+0000","last_fullsized":"2024-09-12T02:00:09.562329+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:20:57.596976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"82'8","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:16.500951+0000","last_change":"2024-09-12T01:59:53.467648+0000","last_active":"2024-09-12T02:00:16.500951+0000","last_peered":"2024-09-12T02:00:16.500951+0000","last_clean":"2024-09-12T02:00:16.500951+0000","last_became_active":"2024-09-12T01:59:53.467255+0000","last_became_peered":"2024-09-12T01:59:53.467255+0000","last_unstale":"2024-09-12T02:00:16.500951+0000","last_undegraded":"2024-09-12T02:00:16.500951+0000","last_fullsized":"2024-09-12T02:00:16.500951+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:37:09.897388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.621489+0000","last_change":"2024-09-12T01:59:55.487603+0000","last_active":"2024-09-12T02:00:09.621489+0000","last_peered":"2024-09-12T02:00:09.621489+0000","last_clean":"2024-09-12T02:00:09.621489+0000","last_became_active":"2024-09-12T01:59:55.487334+0000","last_became_peered":"2024-09-12T01:59:55.487334+0000","last_unstale":"2024-09-12T02:00:09.621489+0000","last_undegraded":"2024-09-12T02:00:09.621489+0000","last_fullsized":"2024-09-12T02:00:09.621489+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:23:16.669515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'4","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.563870+0000","last_change":"2024-09-12T01:59:54.076156+0000","last_active":"2024-09-12T02:00:09.563870+0000","last_peered":"2024-09-12T02:00:09.563870+0000","last_clean":"2024-09-12T02:00:09.563870+0000","last_became_active":"2024-09-12T01:59:54.075794+0000","last_became_peered":"2024-09-12T01:59:54.075794+0000","last_unstale":"2024-09-12T02:00:09.563870+0000","last_undegraded":"2024-09-12T02:00:09.563870+0000","last_fullsized":"2024-09-12T02:00:09.563870+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:06:17.431696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.573860+0000","last_change":"2024-09-12T01:59:55.494604+0000","last_active":"2024-09-12T02:00:09.573860+0000","last_peered":"2024-09-12T02:00:09.573860+0000","last_clean":"2024-09-12T02:00:09.573860+0000","last_became_active":"2024-09-12T01:59:55.494300+0000","last_became_peered":"2024-09-12T01:59:55.494300+0000","last_unstale":"2024-09-12T02:00:09.573860+0000","last_undegraded":"2024-09-12T02:00:09.573860+0000","last_fullsized":"2024-09-12T02:00:09.573860+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:18:56.359299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.622205+0000","last_change":"2024-09-12T01:59:53.921832+0000","last_active":"2024-09-12T02:00:09.622205+0000","last_peered":"2024-09-12T02:00:09.622205+0000","last_clean":"2024-09-12T02:00:09.622205+0000","last_became_active":"2024-09-12T01:59:53.921379+0000","last_became_peered":"2024-09-12T01:59:53.921379+0000","last_unstale":"2024-09-12T02:00:09.622205+0000","last_undegraded":"2024-09-12T02:00:09.622205+0000","last_fullsized":"2024-09-12T02:00:09.622205+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:26:02.277149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.574438+0000","last_change":"2024-09-12T01:59:55.493198+0000","last_active":"2024-09-12T02:00:09.574438+0000","last_peered":"2024-09-12T02:00:09.574438+0000","last_clean":"2024-09-12T02:00:09.574438+0000","last_became_active":"2024-09-12T01:59:55.492834+0000","last_became_peered":"2024-09-12T01:59:55.492834+0000","last_unstale":"2024-09-12T02:00:09.574438+0000","last_undegraded":"2024-09-12T02:00:09.574438+0000","last_fullsized":"2024-09-12T02:00:09.574438+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:58:43.872921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"80'3","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.621258+0000","last_change":"2024-09-12T01:59:53.466528+0000","last_active":"2024-09-12T02:00:09.621258+0000","last_peered":"2024-09-12T02:00:09.621258+0000","last_clean":"2024-09-12T02:00:09.621258+0000","last_became_active":"2024-09-12T01:59:53.466182+0000","last_became_peered":"2024-09-12T01:59:53.466182+0000","last_unstale":"2024-09-12T02:00:09.621258+0000","last_undegraded":"2024-09-12T02:00:09.621258+0000","last_fullsized":"2024-09-12T02:00:09.621258+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:00:26.083539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.572154+0000","last_change":"2024-09-12T01:59:55.494511+0000","last_active":"2024-09-12T02:00:09.572154+0000","last_peered":"2024-09-12T02:00:09.572154+0000","last_clean":"2024-09-12T02:00:09.572154+0000","last_became_active":"2024-09-12T01:59:55.493607+0000","last_became_peered":"2024-09-12T01:59:55.493607+0000","last_unstale":"2024-09-12T02:00:09.572154+0000","last_undegraded":"2024-09-12T02:00:09.572154+0000","last_fullsized":"2024-09-12T02:00:09.572154+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:55:27.434447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"80'1","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.569211+0000","last_change":"2024-09-12T01:59:53.467385+0000","last_active":"2024-09-12T02:00:09.569211+0000","last_peered":"2024-09-12T02:00:09.569211+0000","last_clean":"2024-09-12T02:00:09.569211+0000","last_became_active":"2024-09-12T01:59:53.466997+0000","last_became_peered":"2024-09-12T01:59:53.466997+0000","last_unstale":"2024-09-12T02:00:09.569211+0000","last_undegraded":"2024-09-12T02:00:09.569211+0000","last_fullsized":"2024-09-12T02:00:09.569211+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:54:47.154036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.621293+0000","last_change":"2024-09-12T01:59:55.490757+0000","last_active":"2024-09-12T02:00:09.621293+0000","last_peered":"2024-09-12T02:00:09.621293+0000","last_clean":"2024-09-12T02:00:09.621293+0000","last_became_active":"2024-09-12T01:59:55.490451+0000","last_became_peered":"2024-09-12T01:59:55.490451+0000","last_unstale":"2024-09-12T02:00:09.621293+0000","last_undegraded":"2024-09-12T02:00:09.621293+0000","last_fullsized":"2024-09-12T02:00:09.621293+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:21:26.139049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.574833+0000","last_change":"2024-09-12T01:59:53.472898+0000","last_active":"2024-09-12T02:00:09.574833+0000","last_peered":"2024-09-12T02:00:09.574833+0000","last_clean":"2024-09-12T02:00:09.574833+0000","last_became_active":"2024-09-12T01:59:53.472561+0000","last_became_peered":"2024-09-12T01:59:53.472561+0000","last_unstale":"2024-09-12T02:00:09.574833+0000","last_undegraded":"2024-09-12T02:00:09.574833+0000","last_fullsized":"2024-09-12T02:00:09.574833+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:55:59.933804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.622500+0000","last_change":"2024-09-12T01:59:55.497731+0000","last_active":"2024-09-12T02:00:09.622500+0000","last_peered":"2024-09-12T02:00:09.622500+0000","last_clean":"2024-09-12T02:00:09.622500+0000","last_became_active":"2024-09-12T01:59:55.497356+0000","last_became_peered":"2024-09-12T01:59:55.497356+0000","last_unstale":"2024-09-12T02:00:09.622500+0000","last_undegraded":"2024-09-12T02:00:09.622500+0000","last_fullsized":"2024-09-12T02:00:09.622500+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:57:18.795865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.624453+0000","last_change":"2024-09-12T02:00:09.624453+0000","last_active":"2024-09-12T02:00:09.624453+0000","last_peered":"2024-09-12T02:00:09.624453+0000","last_clean":"2024-09-12T02:00:09.624453+0000","last_became_active":"2024-09-12T02:00:09.624049+0000","last_became_peered":"2024-09-12T02:00:09.624049+0000","last_unstale":"2024-09-12T02:00:09.624453+0000","last_undegraded":"2024-09-12T02:00:09.624453+0000","last_fullsized":"2024-09-12T02:00:09.624453+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:17:40.417275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"80'2","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.575060+0000","last_change":"2024-09-12T01:59:53.470194+0000","last_active":"2024-09-12T02:00:09.575060+0000","last_peered":"2024-09-12T02:00:09.575060+0000","last_clean":"2024-09-12T02:00:09.575060+0000","last_became_active":"2024-09-12T01:59:53.469772+0000","last_became_peered":"2024-09-12T01:59:53.469772+0000","last_unstale":"2024-09-12T02:00:09.575060+0000","last_undegraded":"2024-09-12T02:00:09.575060+0000","last_fullsized":"2024-09-12T02:00:09.575060+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:52:57.709596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.622423+0000","last_change":"2024-09-12T01:59:55.497693+0000","last_active":"2024-09-12T02:00:09.622423+0000","last_peered":"2024-09-12T02:00:09.622423+0000","last_clean":"2024-09-12T02:00:09.622423+0000","last_became_active":"2024-09-12T01:59:55.497380+0000","last_became_peered":"2024-09-12T01:59:55.497380+0000","last_unstale":"2024-09-12T02:00:09.622423+0000","last_undegraded":"2024-09-12T02:00:09.622423+0000","last_fullsized":"2024-09-12T02:00:09.622423+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:42:27.943530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"80'4","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.575274+0000","last_change":"2024-09-12T01:59:53.473044+0000","last_active":"2024-09-12T02:00:09.575274+0000","last_peered":"2024-09-12T02:00:09.575274+0000","last_clean":"2024-09-12T02:00:09.575274+0000","last_became_active":"2024-09-12T01:59:53.472569+0000","last_became_peered":"2024-09-12T01:59:53.472569+0000","last_unstale":"2024-09-12T02:00:09.575274+0000","last_undegraded":"2024-09-12T02:00:09.575274+0000","last_fullsized":"2024-09-12T02:00:09.575274+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:41:29.716836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.573956+0000","last_change":"2024-09-12T01:59:55.497377+0000","last_active":"2024-09-12T02:00:09.573956+0000","last_peered":"2024-09-12T02:00:09.573956+0000","last_clean":"2024-09-12T02:00:09.573956+0000","last_became_active":"2024-09-12T01:59:55.497078+0000","last_became_peered":"2024-09-12T01:59:55.497078+0000","last_unstale":"2024-09-12T02:00:09.573956+0000","last_undegraded":"2024-09-12T02:00:09.573956+0000","last_fullsized":"2024-09-12T02:00:09.573956+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:59:57.362350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.621555+0000","last_change":"2024-09-12T01:59:53.920868+0000","last_active":"2024-09-12T02:00:09.621555+0000","last_peered":"2024-09-12T02:00:09.621555+0000","last_clean":"2024-09-12T02:00:09.621555+0000","last_became_active":"2024-09-12T01:59:53.920481+0000","last_became_peered":"2024-09-12T01:59:53.920481+0000","last_unstale":"2024-09-12T02:00:09.621555+0000","last_undegraded":"2024-09-12T02:00:09.621555+0000","last_fullsized":"2024-09-12T02:00:09.621555+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:49:27.085388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.621438+0000","last_change":"2024-09-12T01:59:55.497703+0000","last_active":"2024-09-12T02:00:09.621438+0000","last_peered":"2024-09-12T02:00:09.621438+0000","last_clean":"2024-09-12T02:00:09.621438+0000","last_became_active":"2024-09-12T01:59:55.497397+0000","last_became_peered":"2024-09-12T01:59:55.497397+0000","last_unstale":"2024-09-12T02:00:09.621438+0000","last_undegraded":"2024-09-12T02:00:09.621438+0000","last_fullsized":"2024-09-12T02:00:09.621438+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:54:02.803203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"80'9","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.565145+0000","last_change":"2024-09-12T01:59:53.469978+0000","last_active":"2024-09-12T02:00:09.565145+0000","last_peered":"2024-09-12T02:00:09.565145+0000","last_clean":"2024-09-12T02:00:09.565145+0000","last_became_active":"2024-09-12T01:59:53.469648+0000","last_became_peered":"2024-09-12T01:59:53.469648+0000","last_unstale":"2024-09-12T02:00:09.565145+0000","last_undegraded":"2024-09-12T02:00:09.565145+0000","last_fullsized":"2024-09-12T02:00:09.565145+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:08:37.720452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":54115,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":107,"num_read_kb":112,"num_write":154,"num_write_kb":96,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":368640,"data_stored":144251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":132,"ondisk_log_size":132,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3563520,"data_stored":3542208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":74,"seq":317827579921,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32068,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713084,"statfs":{"total":95995035648,"available":95962198016,"internally_reserved":0,"allocated":897024,"data_stored":435916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":67,"seq":287762808854,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32724,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712428,"statfs":{"total":95995035648,"available":95961526272,"internally_reserved":0,"allocated":1576960,"data_stored":1077397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":62,"seq":266287972379,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":905216,"data_stored":436040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201311,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":897024,"data_stored":435848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430245,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32648,"kb_used_data":1464,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712504,"statfs":{"total":95995035648,"available":95961604096,"internally_reserved":0,"allocated":1499136,"data_stored":1027892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593770,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36216,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708936,"statfs":{"total":95995035648,"available":95957950464,"internally_reserved":0,"allocated":892928,"data_stored":435429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822702,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32660,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712492,"statfs":{"total":95995035648,"available":95961591808,"internally_reserved":0,"allocated":1511424,"data_stored":1027576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986227,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":897024,"data_stored":435866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215159,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":909312,"data_stored":435740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":17,"seq":73014444093,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36280,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708872,"statfs":{"total":95995035648,"available":95957884928,"internally_reserved":0,"allocated":958464,"data_stored":478030,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574914,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32096,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713056,"statfs":{"total":95995035648,"available":95962169344,"internally_reserved":0,"allocated":933888,"data_stored":476287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738440,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":913408,"data_stored":436700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":41065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":42808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":51690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T02:00:21.465 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-12T02:00:21.749 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:00:22.124 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:00:21 smithi016 ceph-mon[30947]: pgmap v288: 97 pgs: 97 active+clean; 630 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 5.6 KiB/s wr, 10 op/s; 10 B/s, 0 objects/s recovering 2024-09-12T02:00:22.124 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:00:21 smithi016 ceph-mon[30947]: from='client.14739 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T02:00:22.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:00:21 smithi184 ceph-mon[35593]: pgmap v288: 97 pgs: 97 active+clean; 630 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 5.6 KiB/s wr, 10 op/s; 10 B/s, 0 objects/s recovering 2024-09-12T02:00:22.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:00:21 smithi184 ceph-mon[35593]: from='client.14739 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T02:00:22.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:00:21 smithi033 ceph-mon[35332]: pgmap v288: 97 pgs: 97 active+clean; 630 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 5.6 KiB/s wr, 10 op/s; 10 B/s, 0 objects/s recovering 2024-09-12T02:00:22.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:00:21 smithi033 ceph-mon[35332]: from='client.14739 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T02:00:22.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:22.204+0000 7fb45897b640 1 -- 172.21.15.16:0/624104027 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb454101de0 msgr2=0x7fb4541021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:22.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:22.204+0000 7fb45897b640 1 --2- 172.21.15.16:0/624104027 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb454101de0 0x7fb4541021c0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fb444009900 tx=0x7fb44402a410 comp rx=0 tx=0).stop 2024-09-12T02:00:22.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:22.204+0000 7fb45897b640 1 -- 172.21.15.16:0/624104027 shutdown_connections 2024-09-12T02:00:22.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:22.204+0000 7fb45897b640 1 --2- 172.21.15.16:0/624104027 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb454106f10 0x7fb45410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:22.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:22.204+0000 7fb45897b640 1 --2- 172.21.15.16:0/624104027 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb454102790 0x7fb4541067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:22.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:22.204+0000 7fb45897b640 1 --2- 172.21.15.16:0/624104027 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb454101de0 0x7fb4541021c0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:22.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:22.205+0000 7fb45897b640 1 -- 172.21.15.16:0/624104027 >> 172.21.15.16:0/624104027 conn(0x7fb4540fd8c0 msgr2=0x7fb4540ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:00:22.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:22.205+0000 7fb45897b640 1 -- 172.21.15.16:0/624104027 shutdown_connections 2024-09-12T02:00:22.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:22.205+0000 7fb45897b640 1 -- 172.21.15.16:0/624104027 wait complete. 2024-09-12T02:00:22.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:22.205+0000 7fb45897b640 1 Processor -- start 2024-09-12T02:00:22.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:22.206+0000 7fb45897b640 1 -- start start 2024-09-12T02:00:22.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:22.206+0000 7fb45897b640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb454101de0 0x7fb4541984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:22.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:22.206+0000 7fb45897b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb454102790 0x7fb454198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:22.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:22.206+0000 7fb45897b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb454106f10 0x7fb45419cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:22.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:22.206+0000 7fb45897b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb454111fb0 con 0x7fb454102790 2024-09-12T02:00:22.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:22.206+0000 7fb45897b640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb454111e30 con 0x7fb454106f10 2024-09-12T02:00:22.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:22.206+0000 7fb45897b640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb454112130 con 0x7fb454101de0 2024-09-12T02:00:22.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:22.206+0000 7fb449d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb454102790 0x7fb454198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:22.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:22.206+0000 7fb45259c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb454101de0 0x7fb4541984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:22.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:22.206+0000 7fb45259c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb454101de0 0x7fb4541984f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:36254/0 (socket says 172.21.15.16:36254) 2024-09-12T02:00:22.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:22.206+0000 7fb45259c640 1 -- 172.21.15.16:0/3921473093 learned_addr learned my addr 172.21.15.16:0/3921473093 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:00:22.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:22.206+0000 7fb452d9d640 1 --2- 172.21.15.16:0/3921473093 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb454106f10 0x7fb45419cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:22.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:22.206+0000 7fb449d9b640 1 -- 172.21.15.16:0/3921473093 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb454101de0 msgr2=0x7fb4541984f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:22.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:22.206+0000 7fb449d9b640 1 --2- 172.21.15.16:0/3921473093 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb454101de0 0x7fb4541984f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:22.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:22.206+0000 7fb449d9b640 1 -- 172.21.15.16:0/3921473093 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb454106f10 msgr2=0x7fb45419cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:22.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:22.206+0000 7fb449d9b640 1 --2- 172.21.15.16:0/3921473093 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb454106f10 0x7fb45419cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:22.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:22.206+0000 7fb449d9b640 1 -- 172.21.15.16:0/3921473093 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb45419d4a0 con 0x7fb454102790 2024-09-12T02:00:22.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:22.206+0000 7fb449d9b640 1 --2- 172.21.15.16:0/3921473093 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb454102790 0x7fb454198a30 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb43c00d720 tx=0x7fb43c00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:00:22.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:22.207+0000 7fb44bfff640 1 -- 172.21.15.16:0/3921473093 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb43c017070 con 0x7fb454102790 2024-09-12T02:00:22.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:22.207+0000 7fb44bfff640 1 -- 172.21.15.16:0/3921473093 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb43c00de30 con 0x7fb454102790 2024-09-12T02:00:22.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:22.207+0000 7fb44bfff640 1 -- 172.21.15.16:0/3921473093 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb43c0047a0 con 0x7fb454102790 2024-09-12T02:00:22.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:22.207+0000 7fb45897b640 1 -- 172.21.15.16:0/3921473093 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb45419d790 con 0x7fb454102790 2024-09-12T02:00:22.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:22.207+0000 7fb45897b640 1 -- 172.21.15.16:0/3921473093 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4541a5070 con 0x7fb454102790 2024-09-12T02:00:22.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:22.208+0000 7fb45897b640 1 -- 172.21.15.16:0/3921473093 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4541021c0 con 0x7fb454102790 2024-09-12T02:00:22.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:22.209+0000 7fb44bfff640 1 -- 172.21.15.16:0/3921473093 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fb43c0099d0 con 0x7fb454102790 2024-09-12T02:00:22.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:22.210+0000 7fb44bfff640 1 --2- 172.21.15.16:0/3921473093 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fb4400a2c10 0x7fb4400a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:22.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:22.210+0000 7fb45259c640 1 --2- 172.21.15.16:0/3921473093 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fb4400a2c10 0x7fb4400a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:22.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:22.210+0000 7fb44bfff640 1 -- 172.21.15.16:0/3921473093 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fb43c0cedb0 con 0x7fb454102790 2024-09-12T02:00:22.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:22.210+0000 7fb45259c640 1 --2- 172.21.15.16:0/3921473093 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fb4400a2c10 0x7fb4400a50d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fb4440096c0 tx=0x7fb444005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:00:22.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:22.215+0000 7fb44bfff640 1 -- 172.21.15.16:0/3921473093 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb43c0d3070 con 0x7fb454102790 2024-09-12T02:00:22.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:22.388+0000 7fb45897b640 1 -- 172.21.15.16:0/3921473093 --> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb454062630 con 0x7fb4400a2c10 2024-09-12T02:00:22.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:22.394+0000 7fb44bfff640 1 -- 172.21.15.16:0/3921473093 <== mgr.14150 v2:172.21.15.16:6800/3887444820 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258696 (secure 0 0 0) 0x7fb454062630 con 0x7fb4400a2c10 2024-09-12T02:00:22.397 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T02:00:22.398 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-12T02:00:22.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:22.397+0000 7fb45897b640 1 -- 172.21.15.16:0/3921473093 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fb4400a2c10 msgr2=0x7fb4400a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:22.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:22.397+0000 7fb45897b640 1 --2- 172.21.15.16:0/3921473093 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fb4400a2c10 0x7fb4400a50d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fb4440096c0 tx=0x7fb444005a20 comp rx=0 tx=0).stop 2024-09-12T02:00:22.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:22.397+0000 7fb45897b640 1 -- 172.21.15.16:0/3921473093 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb454102790 msgr2=0x7fb454198a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:22.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:22.397+0000 7fb45897b640 1 --2- 172.21.15.16:0/3921473093 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb454102790 0x7fb454198a30 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb43c00d720 tx=0x7fb43c00dbe0 comp rx=0 tx=0).stop 2024-09-12T02:00:22.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:22.397+0000 7fb45897b640 1 -- 172.21.15.16:0/3921473093 shutdown_connections 2024-09-12T02:00:22.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:22.397+0000 7fb45897b640 1 --2- 172.21.15.16:0/3921473093 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fb4400a2c10 0x7fb4400a50d0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:22.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:22.397+0000 7fb45897b640 1 --2- 172.21.15.16:0/3921473093 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb454106f10 0x7fb45419cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:22.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:22.397+0000 7fb45897b640 1 --2- 172.21.15.16:0/3921473093 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb454102790 0x7fb454198a30 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:22.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:22.397+0000 7fb45897b640 1 --2- 172.21.15.16:0/3921473093 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb454101de0 0x7fb4541984f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:22.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:22.397+0000 7fb45897b640 1 -- 172.21.15.16:0/3921473093 >> 172.21.15.16:0/3921473093 conn(0x7fb4540fd8c0 msgr2=0x7fb4540febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:00:22.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:22.397+0000 7fb45897b640 1 -- 172.21.15.16:0/3921473093 shutdown_connections 2024-09-12T02:00:22.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:22.397+0000 7fb45897b640 1 -- 172.21.15.16:0/3921473093 wait complete. 2024-09-12T02:00:23.074 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":289,"stamp":"2024-09-12T02:00:22.305037+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":644717,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":238,"num_read_kb":221,"num_write":337,"num_write_kb":3036,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":1180848,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":271,"ondisk_log_size":271,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":374264,"kb_used_data":12516,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124567560,"statfs":{"total":1151940427776,"available":1151557181440,"internally_reserved":0,"allocated":12816384,"data_stored":7156815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":370193723},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":49127,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":-8,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":70,"num_read_kb":80,"num_write":29,"num_write_kb":59,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002886"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.572370+0000","last_change":"2024-09-12T01:59:55.503882+0000","last_active":"2024-09-12T02:00:09.572370+0000","last_peered":"2024-09-12T02:00:09.572370+0000","last_clean":"2024-09-12T02:00:09.572370+0000","last_became_active":"2024-09-12T01:59:55.503513+0000","last_became_peered":"2024-09-12T01:59:55.503513+0000","last_unstale":"2024-09-12T02:00:09.572370+0000","last_undegraded":"2024-09-12T02:00:09.572370+0000","last_fullsized":"2024-09-12T02:00:09.572370+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:43:13.098527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.573809+0000","last_change":"2024-09-12T01:59:55.492965+0000","last_active":"2024-09-12T02:00:09.573809+0000","last_peered":"2024-09-12T02:00:09.573809+0000","last_clean":"2024-09-12T02:00:09.573809+0000","last_became_active":"2024-09-12T01:59:55.492613+0000","last_became_peered":"2024-09-12T01:59:55.492613+0000","last_unstale":"2024-09-12T02:00:09.573809+0000","last_undegraded":"2024-09-12T02:00:09.573809+0000","last_fullsized":"2024-09-12T02:00:09.573809+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:58:20.889770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.573872+0000","last_change":"2024-09-12T01:59:55.497334+0000","last_active":"2024-09-12T02:00:09.573872+0000","last_peered":"2024-09-12T02:00:09.573872+0000","last_clean":"2024-09-12T02:00:09.573872+0000","last_became_active":"2024-09-12T01:59:55.497037+0000","last_became_peered":"2024-09-12T01:59:55.497037+0000","last_unstale":"2024-09-12T02:00:09.573872+0000","last_undegraded":"2024-09-12T02:00:09.573872+0000","last_fullsized":"2024-09-12T02:00:09.573872+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:43:04.082918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.563839+0000","last_change":"2024-09-12T01:59:55.495888+0000","last_active":"2024-09-12T02:00:09.563839+0000","last_peered":"2024-09-12T02:00:09.563839+0000","last_clean":"2024-09-12T02:00:09.563839+0000","last_became_active":"2024-09-12T01:59:55.492684+0000","last_became_peered":"2024-09-12T01:59:55.492684+0000","last_unstale":"2024-09-12T02:00:09.563839+0000","last_undegraded":"2024-09-12T02:00:09.563839+0000","last_fullsized":"2024-09-12T02:00:09.563839+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:15:00.500006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.574662+0000","last_change":"2024-09-12T01:59:55.494613+0000","last_active":"2024-09-12T02:00:09.574662+0000","last_peered":"2024-09-12T02:00:09.574662+0000","last_clean":"2024-09-12T02:00:09.574662+0000","last_became_active":"2024-09-12T01:59:55.494205+0000","last_became_peered":"2024-09-12T01:59:55.494205+0000","last_unstale":"2024-09-12T02:00:09.574662+0000","last_undegraded":"2024-09-12T02:00:09.574662+0000","last_fullsized":"2024-09-12T02:00:09.574662+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:39:14.573280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.622709+0000","last_change":"2024-09-12T01:59:55.496659+0000","last_active":"2024-09-12T02:00:09.622709+0000","last_peered":"2024-09-12T02:00:09.622709+0000","last_clean":"2024-09-12T02:00:09.622709+0000","last_became_active":"2024-09-12T01:59:55.496232+0000","last_became_peered":"2024-09-12T01:59:55.496232+0000","last_unstale":"2024-09-12T02:00:09.622709+0000","last_undegraded":"2024-09-12T02:00:09.622709+0000","last_fullsized":"2024-09-12T02:00:09.622709+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:48:21.544764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.622510+0000","last_change":"2024-09-12T01:59:55.486789+0000","last_active":"2024-09-12T02:00:09.622510+0000","last_peered":"2024-09-12T02:00:09.622510+0000","last_clean":"2024-09-12T02:00:09.622510+0000","last_became_active":"2024-09-12T01:59:55.485921+0000","last_became_peered":"2024-09-12T01:59:55.485921+0000","last_unstale":"2024-09-12T02:00:09.622510+0000","last_undegraded":"2024-09-12T02:00:09.622510+0000","last_fullsized":"2024-09-12T02:00:09.622510+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:21:16.071121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.621086+0000","last_change":"2024-09-12T01:59:55.494525+0000","last_active":"2024-09-12T02:00:09.621086+0000","last_peered":"2024-09-12T02:00:09.621086+0000","last_clean":"2024-09-12T02:00:09.621086+0000","last_became_active":"2024-09-12T01:59:55.493998+0000","last_became_peered":"2024-09-12T01:59:55.493998+0000","last_unstale":"2024-09-12T02:00:09.621086+0000","last_undegraded":"2024-09-12T02:00:09.621086+0000","last_fullsized":"2024-09-12T02:00:09.621086+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:59:24.653660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.622633+0000","last_change":"2024-09-12T01:59:55.501466+0000","last_active":"2024-09-12T02:00:09.622633+0000","last_peered":"2024-09-12T02:00:09.622633+0000","last_clean":"2024-09-12T02:00:09.622633+0000","last_became_active":"2024-09-12T01:59:55.500800+0000","last_became_peered":"2024-09-12T01:59:55.500800+0000","last_unstale":"2024-09-12T02:00:09.622633+0000","last_undegraded":"2024-09-12T02:00:09.622633+0000","last_fullsized":"2024-09-12T02:00:09.622633+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:55:13.704791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.623367+0000","last_change":"2024-09-12T01:59:55.497986+0000","last_active":"2024-09-12T02:00:09.623367+0000","last_peered":"2024-09-12T02:00:09.623367+0000","last_clean":"2024-09-12T02:00:09.623367+0000","last_became_active":"2024-09-12T01:59:55.497691+0000","last_became_peered":"2024-09-12T01:59:55.497691+0000","last_unstale":"2024-09-12T02:00:09.623367+0000","last_undegraded":"2024-09-12T02:00:09.623367+0000","last_fullsized":"2024-09-12T02:00:09.623367+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:37:07.702606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.622801+0000","last_change":"2024-09-12T01:59:55.501339+0000","last_active":"2024-09-12T02:00:09.622801+0000","last_peered":"2024-09-12T02:00:09.622801+0000","last_clean":"2024-09-12T02:00:09.622801+0000","last_became_active":"2024-09-12T01:59:55.500700+0000","last_became_peered":"2024-09-12T01:59:55.500700+0000","last_unstale":"2024-09-12T02:00:09.622801+0000","last_undegraded":"2024-09-12T02:00:09.622801+0000","last_fullsized":"2024-09-12T02:00:09.622801+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:12:15.879183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.563946+0000","last_change":"2024-09-12T01:59:55.495196+0000","last_active":"2024-09-12T02:00:09.563946+0000","last_peered":"2024-09-12T02:00:09.563946+0000","last_clean":"2024-09-12T02:00:09.563946+0000","last_became_active":"2024-09-12T01:59:55.494889+0000","last_became_peered":"2024-09-12T01:59:55.494889+0000","last_unstale":"2024-09-12T02:00:09.563946+0000","last_undegraded":"2024-09-12T02:00:09.563946+0000","last_fullsized":"2024-09-12T02:00:09.563946+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:32:30.621946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.562435+0000","last_change":"2024-09-12T01:59:55.492565+0000","last_active":"2024-09-12T02:00:09.562435+0000","last_peered":"2024-09-12T02:00:09.562435+0000","last_clean":"2024-09-12T02:00:09.562435+0000","last_became_active":"2024-09-12T01:59:55.492168+0000","last_became_peered":"2024-09-12T01:59:55.492168+0000","last_unstale":"2024-09-12T02:00:09.562435+0000","last_undegraded":"2024-09-12T02:00:09.562435+0000","last_fullsized":"2024-09-12T02:00:09.562435+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:39:48.630202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.573885+0000","last_change":"2024-09-12T01:59:55.493152+0000","last_active":"2024-09-12T02:00:09.573885+0000","last_peered":"2024-09-12T02:00:09.573885+0000","last_clean":"2024-09-12T02:00:09.573885+0000","last_became_active":"2024-09-12T01:59:55.492794+0000","last_became_peered":"2024-09-12T01:59:55.492794+0000","last_unstale":"2024-09-12T02:00:09.573885+0000","last_undegraded":"2024-09-12T02:00:09.573885+0000","last_fullsized":"2024-09-12T02:00:09.573885+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:24:27.270502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.573561+0000","last_change":"2024-09-12T01:59:55.494651+0000","last_active":"2024-09-12T02:00:09.573561+0000","last_peered":"2024-09-12T02:00:09.573561+0000","last_clean":"2024-09-12T02:00:09.573561+0000","last_became_active":"2024-09-12T01:59:55.494355+0000","last_became_peered":"2024-09-12T01:59:55.494355+0000","last_unstale":"2024-09-12T02:00:09.573561+0000","last_undegraded":"2024-09-12T02:00:09.573561+0000","last_fullsized":"2024-09-12T02:00:09.573561+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:14:44.364158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.622950+0000","last_change":"2024-09-12T01:59:55.495049+0000","last_active":"2024-09-12T02:00:09.622950+0000","last_peered":"2024-09-12T02:00:09.622950+0000","last_clean":"2024-09-12T02:00:09.622950+0000","last_became_active":"2024-09-12T01:59:55.494688+0000","last_became_peered":"2024-09-12T01:59:55.494688+0000","last_unstale":"2024-09-12T02:00:09.622950+0000","last_undegraded":"2024-09-12T02:00:09.622950+0000","last_fullsized":"2024-09-12T02:00:09.622950+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:13:05.944577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.572458+0000","last_change":"2024-09-12T01:59:55.504146+0000","last_active":"2024-09-12T02:00:09.572458+0000","last_peered":"2024-09-12T02:00:09.572458+0000","last_clean":"2024-09-12T02:00:09.572458+0000","last_became_active":"2024-09-12T01:59:55.503646+0000","last_became_peered":"2024-09-12T01:59:55.503646+0000","last_unstale":"2024-09-12T02:00:09.572458+0000","last_undegraded":"2024-09-12T02:00:09.572458+0000","last_fullsized":"2024-09-12T02:00:09.572458+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:23:51.780795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.565050+0000","last_change":"2024-09-12T01:59:55.489930+0000","last_active":"2024-09-12T02:00:09.565050+0000","last_peered":"2024-09-12T02:00:09.565050+0000","last_clean":"2024-09-12T02:00:09.565050+0000","last_became_active":"2024-09-12T01:59:55.489619+0000","last_became_peered":"2024-09-12T01:59:55.489619+0000","last_unstale":"2024-09-12T02:00:09.565050+0000","last_undegraded":"2024-09-12T02:00:09.565050+0000","last_fullsized":"2024-09-12T02:00:09.565050+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:04:57.307091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.569107+0000","last_change":"2024-09-12T01:59:55.492098+0000","last_active":"2024-09-12T02:00:09.569107+0000","last_peered":"2024-09-12T02:00:09.569107+0000","last_clean":"2024-09-12T02:00:09.569107+0000","last_became_active":"2024-09-12T01:59:55.491370+0000","last_became_peered":"2024-09-12T01:59:55.491370+0000","last_unstale":"2024-09-12T02:00:09.569107+0000","last_undegraded":"2024-09-12T02:00:09.569107+0000","last_fullsized":"2024-09-12T02:00:09.569107+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:25:06.307903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.571940+0000","last_change":"2024-09-12T01:59:55.499938+0000","last_active":"2024-09-12T02:00:09.571940+0000","last_peered":"2024-09-12T02:00:09.571940+0000","last_clean":"2024-09-12T02:00:09.571940+0000","last_became_active":"2024-09-12T01:59:55.499619+0000","last_became_peered":"2024-09-12T01:59:55.499619+0000","last_unstale":"2024-09-12T02:00:09.571940+0000","last_undegraded":"2024-09-12T02:00:09.571940+0000","last_fullsized":"2024-09-12T02:00:09.571940+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:00:17.536120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.562514+0000","last_change":"2024-09-12T01:59:55.486106+0000","last_active":"2024-09-12T02:00:09.562514+0000","last_peered":"2024-09-12T02:00:09.562514+0000","last_clean":"2024-09-12T02:00:09.562514+0000","last_became_active":"2024-09-12T01:59:55.485325+0000","last_became_peered":"2024-09-12T01:59:55.485325+0000","last_unstale":"2024-09-12T02:00:09.562514+0000","last_undegraded":"2024-09-12T02:00:09.562514+0000","last_fullsized":"2024-09-12T02:00:09.562514+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:35:46.446224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.622237+0000","last_change":"2024-09-12T01:59:55.496438+0000","last_active":"2024-09-12T02:00:09.622237+0000","last_peered":"2024-09-12T02:00:09.622237+0000","last_clean":"2024-09-12T02:00:09.622237+0000","last_became_active":"2024-09-12T01:59:55.495885+0000","last_became_peered":"2024-09-12T01:59:55.495885+0000","last_unstale":"2024-09-12T02:00:09.622237+0000","last_undegraded":"2024-09-12T02:00:09.622237+0000","last_fullsized":"2024-09-12T02:00:09.622237+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:15:14.911335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.621139+0000","last_change":"2024-09-12T01:59:55.489350+0000","last_active":"2024-09-12T02:00:09.621139+0000","last_peered":"2024-09-12T02:00:09.621139+0000","last_clean":"2024-09-12T02:00:09.621139+0000","last_became_active":"2024-09-12T01:59:55.489028+0000","last_became_peered":"2024-09-12T01:59:55.489028+0000","last_unstale":"2024-09-12T02:00:09.621139+0000","last_undegraded":"2024-09-12T02:00:09.621139+0000","last_fullsized":"2024-09-12T02:00:09.621139+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:29:23.769004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.566645+0000","last_change":"2024-09-12T01:59:55.494532+0000","last_active":"2024-09-12T02:00:09.566645+0000","last_peered":"2024-09-12T02:00:09.566645+0000","last_clean":"2024-09-12T02:00:09.566645+0000","last_became_active":"2024-09-12T01:59:55.490982+0000","last_became_peered":"2024-09-12T01:59:55.490982+0000","last_unstale":"2024-09-12T02:00:09.566645+0000","last_undegraded":"2024-09-12T02:00:09.566645+0000","last_fullsized":"2024-09-12T02:00:09.566645+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:04:50.633730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.621490+0000","last_change":"2024-09-12T01:59:55.496057+0000","last_active":"2024-09-12T02:00:09.621490+0000","last_peered":"2024-09-12T02:00:09.621490+0000","last_clean":"2024-09-12T02:00:09.621490+0000","last_became_active":"2024-09-12T01:59:55.495710+0000","last_became_peered":"2024-09-12T01:59:55.495710+0000","last_unstale":"2024-09-12T02:00:09.621490+0000","last_undegraded":"2024-09-12T02:00:09.621490+0000","last_fullsized":"2024-09-12T02:00:09.621490+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:49:01.962537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.569100+0000","last_change":"2024-09-12T01:59:55.491594+0000","last_active":"2024-09-12T02:00:09.569100+0000","last_peered":"2024-09-12T02:00:09.569100+0000","last_clean":"2024-09-12T02:00:09.569100+0000","last_became_active":"2024-09-12T01:59:55.491233+0000","last_became_peered":"2024-09-12T01:59:55.491233+0000","last_unstale":"2024-09-12T02:00:09.569100+0000","last_undegraded":"2024-09-12T02:00:09.569100+0000","last_fullsized":"2024-09-12T02:00:09.569100+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:24:22.066777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.572270+0000","last_change":"2024-09-12T01:59:55.498331+0000","last_active":"2024-09-12T02:00:09.572270+0000","last_peered":"2024-09-12T02:00:09.572270+0000","last_clean":"2024-09-12T02:00:09.572270+0000","last_became_active":"2024-09-12T01:59:55.497983+0000","last_became_peered":"2024-09-12T01:59:55.497983+0000","last_unstale":"2024-09-12T02:00:09.572270+0000","last_undegraded":"2024-09-12T02:00:09.572270+0000","last_fullsized":"2024-09-12T02:00:09.572270+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:37:00.735231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.575358+0000","last_change":"2024-09-12T02:00:09.575358+0000","last_active":"2024-09-12T02:00:09.575358+0000","last_peered":"2024-09-12T02:00:09.575358+0000","last_clean":"2024-09-12T02:00:09.575358+0000","last_became_active":"2024-09-12T02:00:09.574913+0000","last_became_peered":"2024-09-12T02:00:09.574913+0000","last_unstale":"2024-09-12T02:00:09.575358+0000","last_undegraded":"2024-09-12T02:00:09.575358+0000","last_fullsized":"2024-09-12T02:00:09.575358+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:19:53.630453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.573748+0000","last_change":"2024-09-12T01:59:55.492419+0000","last_active":"2024-09-12T02:00:09.573748+0000","last_peered":"2024-09-12T02:00:09.573748+0000","last_clean":"2024-09-12T02:00:09.573748+0000","last_became_active":"2024-09-12T01:59:55.492003+0000","last_became_peered":"2024-09-12T01:59:55.492003+0000","last_unstale":"2024-09-12T02:00:09.573748+0000","last_undegraded":"2024-09-12T02:00:09.573748+0000","last_fullsized":"2024-09-12T02:00:09.573748+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:25:45.774679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.568801+0000","last_change":"2024-09-12T01:59:55.490125+0000","last_active":"2024-09-12T02:00:09.568801+0000","last_peered":"2024-09-12T02:00:09.568801+0000","last_clean":"2024-09-12T02:00:09.568801+0000","last_became_active":"2024-09-12T01:59:55.489636+0000","last_became_peered":"2024-09-12T01:59:55.489636+0000","last_unstale":"2024-09-12T02:00:09.568801+0000","last_undegraded":"2024-09-12T02:00:09.568801+0000","last_fullsized":"2024-09-12T02:00:09.568801+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:41:30.922185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.569135+0000","last_change":"2024-09-12T01:59:55.496322+0000","last_active":"2024-09-12T02:00:09.569135+0000","last_peered":"2024-09-12T02:00:09.569135+0000","last_clean":"2024-09-12T02:00:09.569135+0000","last_became_active":"2024-09-12T01:59:55.496005+0000","last_became_peered":"2024-09-12T01:59:55.496005+0000","last_unstale":"2024-09-12T02:00:09.569135+0000","last_undegraded":"2024-09-12T02:00:09.569135+0000","last_fullsized":"2024-09-12T02:00:09.569135+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:53:39.248715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.572017+0000","last_change":"2024-09-12T01:59:55.497758+0000","last_active":"2024-09-12T02:00:09.572017+0000","last_peered":"2024-09-12T02:00:09.572017+0000","last_clean":"2024-09-12T02:00:09.572017+0000","last_became_active":"2024-09-12T01:59:55.497282+0000","last_became_peered":"2024-09-12T01:59:55.497282+0000","last_unstale":"2024-09-12T02:00:09.572017+0000","last_undegraded":"2024-09-12T02:00:09.572017+0000","last_fullsized":"2024-09-12T02:00:09.572017+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:08:25.509372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.574620+0000","last_change":"2024-09-12T01:59:55.497919+0000","last_active":"2024-09-12T02:00:09.574620+0000","last_peered":"2024-09-12T02:00:09.574620+0000","last_clean":"2024-09-12T02:00:09.574620+0000","last_became_active":"2024-09-12T01:59:55.497609+0000","last_became_peered":"2024-09-12T01:59:55.497609+0000","last_unstale":"2024-09-12T02:00:09.574620+0000","last_undegraded":"2024-09-12T02:00:09.574620+0000","last_fullsized":"2024-09-12T02:00:09.574620+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:10:22.281180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"81'6","reported_seq":47,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:19.574031+0000","last_change":"2024-09-12T01:59:53.473974+0000","last_active":"2024-09-12T02:00:19.574031+0000","last_peered":"2024-09-12T02:00:19.574031+0000","last_clean":"2024-09-12T02:00:19.574031+0000","last_became_active":"2024-09-12T01:59:53.473546+0000","last_became_peered":"2024-09-12T01:59:53.473546+0000","last_unstale":"2024-09-12T02:00:19.574031+0000","last_undegraded":"2024-09-12T02:00:19.574031+0000","last_fullsized":"2024-09-12T02:00:19.574031+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:28:03.532249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":19,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.567217+0000","last_change":"2024-09-12T01:59:55.493197+0000","last_active":"2024-09-12T02:00:09.567217+0000","last_peered":"2024-09-12T02:00:09.567217+0000","last_clean":"2024-09-12T02:00:09.567217+0000","last_became_active":"2024-09-12T01:59:55.492845+0000","last_became_peered":"2024-09-12T01:59:55.492845+0000","last_unstale":"2024-09-12T02:00:09.567217+0000","last_undegraded":"2024-09-12T02:00:09.567217+0000","last_fullsized":"2024-09-12T02:00:09.567217+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:42:50.790896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"80'3","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.622865+0000","last_change":"2024-09-12T01:59:53.471214+0000","last_active":"2024-09-12T02:00:09.622865+0000","last_peered":"2024-09-12T02:00:09.622865+0000","last_clean":"2024-09-12T02:00:09.622865+0000","last_became_active":"2024-09-12T01:59:53.470858+0000","last_became_peered":"2024-09-12T01:59:53.470858+0000","last_unstale":"2024-09-12T02:00:09.622865+0000","last_undegraded":"2024-09-12T02:00:09.622865+0000","last_fullsized":"2024-09-12T02:00:09.622865+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:34:48.730392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"80'2","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.562106+0000","last_change":"2024-09-12T01:59:53.464568+0000","last_active":"2024-09-12T02:00:09.562106+0000","last_peered":"2024-09-12T02:00:09.562106+0000","last_clean":"2024-09-12T02:00:09.562106+0000","last_became_active":"2024-09-12T01:59:53.464168+0000","last_became_peered":"2024-09-12T01:59:53.464168+0000","last_unstale":"2024-09-12T02:00:09.562106+0000","last_undegraded":"2024-09-12T02:00:09.562106+0000","last_fullsized":"2024-09-12T02:00:09.562106+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:41:48.135522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.621151+0000","last_change":"2024-09-12T01:59:55.487177+0000","last_active":"2024-09-12T02:00:09.621151+0000","last_peered":"2024-09-12T02:00:09.621151+0000","last_clean":"2024-09-12T02:00:09.621151+0000","last_became_active":"2024-09-12T01:59:55.486886+0000","last_became_peered":"2024-09-12T01:59:55.486886+0000","last_unstale":"2024-09-12T02:00:09.621151+0000","last_undegraded":"2024-09-12T02:00:09.621151+0000","last_fullsized":"2024-09-12T02:00:09.621151+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:32:40.150071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"80'1","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.567129+0000","last_change":"2024-09-12T01:59:53.474114+0000","last_active":"2024-09-12T02:00:09.567129+0000","last_peered":"2024-09-12T02:00:09.567129+0000","last_clean":"2024-09-12T02:00:09.567129+0000","last_became_active":"2024-09-12T01:59:53.473755+0000","last_became_peered":"2024-09-12T01:59:53.473755+0000","last_unstale":"2024-09-12T02:00:09.567129+0000","last_undegraded":"2024-09-12T02:00:09.567129+0000","last_fullsized":"2024-09-12T02:00:09.567129+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:07:07.021137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.564793+0000","last_change":"2024-09-12T01:59:55.489868+0000","last_active":"2024-09-12T02:00:09.564793+0000","last_peered":"2024-09-12T02:00:09.564793+0000","last_clean":"2024-09-12T02:00:09.564793+0000","last_became_active":"2024-09-12T01:59:55.489537+0000","last_became_peered":"2024-09-12T01:59:55.489537+0000","last_unstale":"2024-09-12T02:00:09.564793+0000","last_undegraded":"2024-09-12T02:00:09.564793+0000","last_fullsized":"2024-09-12T02:00:09.564793+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:18:49.417689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"80'3","reported_seq":39,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:17.571161+0000","last_change":"2024-09-12T01:59:53.471402+0000","last_active":"2024-09-12T02:00:17.571161+0000","last_peered":"2024-09-12T02:00:17.571161+0000","last_clean":"2024-09-12T02:00:17.571161+0000","last_became_active":"2024-09-12T01:59:53.470940+0000","last_became_peered":"2024-09-12T01:59:53.470940+0000","last_unstale":"2024-09-12T02:00:17.571161+0000","last_undegraded":"2024-09-12T02:00:17.571161+0000","last_fullsized":"2024-09-12T02:00:17.571161+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:41:16.853592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.623260+0000","last_change":"2024-09-12T01:59:55.499047+0000","last_active":"2024-09-12T02:00:09.623260+0000","last_peered":"2024-09-12T02:00:09.623260+0000","last_clean":"2024-09-12T02:00:09.623260+0000","last_became_active":"2024-09-12T01:59:55.498071+0000","last_became_peered":"2024-09-12T01:59:55.498071+0000","last_unstale":"2024-09-12T02:00:09.623260+0000","last_undegraded":"2024-09-12T02:00:09.623260+0000","last_fullsized":"2024-09-12T02:00:09.623260+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:36:40.106041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"80'2","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.622966+0000","last_change":"2024-09-12T01:59:53.473508+0000","last_active":"2024-09-12T02:00:09.622966+0000","last_peered":"2024-09-12T02:00:09.622966+0000","last_clean":"2024-09-12T02:00:09.622966+0000","last_became_active":"2024-09-12T01:59:53.473063+0000","last_became_peered":"2024-09-12T01:59:53.473063+0000","last_unstale":"2024-09-12T02:00:09.622966+0000","last_undegraded":"2024-09-12T02:00:09.622966+0000","last_fullsized":"2024-09-12T02:00:09.622966+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:55:32.161769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.573961+0000","last_change":"2024-09-12T01:59:55.491721+0000","last_active":"2024-09-12T02:00:09.573961+0000","last_peered":"2024-09-12T02:00:09.573961+0000","last_clean":"2024-09-12T02:00:09.573961+0000","last_became_active":"2024-09-12T01:59:55.491364+0000","last_became_peered":"2024-09-12T01:59:55.491364+0000","last_unstale":"2024-09-12T02:00:09.573961+0000","last_undegraded":"2024-09-12T02:00:09.573961+0000","last_fullsized":"2024-09-12T02:00:09.573961+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:21:20.795420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"80'4","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.567315+0000","last_change":"2024-09-12T01:59:53.921789+0000","last_active":"2024-09-12T02:00:09.567315+0000","last_peered":"2024-09-12T02:00:09.567315+0000","last_clean":"2024-09-12T02:00:09.567315+0000","last_became_active":"2024-09-12T01:59:53.921383+0000","last_became_peered":"2024-09-12T01:59:53.921383+0000","last_unstale":"2024-09-12T02:00:09.567315+0000","last_undegraded":"2024-09-12T02:00:09.567315+0000","last_fullsized":"2024-09-12T02:00:09.567315+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:21:58.792794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.622517+0000","last_change":"2024-09-12T01:59:55.503008+0000","last_active":"2024-09-12T02:00:09.622517+0000","last_peered":"2024-09-12T02:00:09.622517+0000","last_clean":"2024-09-12T02:00:09.622517+0000","last_became_active":"2024-09-12T01:59:55.502538+0000","last_became_peered":"2024-09-12T01:59:55.502538+0000","last_unstale":"2024-09-12T02:00:09.622517+0000","last_undegraded":"2024-09-12T02:00:09.622517+0000","last_fullsized":"2024-09-12T02:00:09.622517+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:51:36.735985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"82'25","reported_seq":81,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:18.575816+0000","last_change":"2024-09-12T01:59:53.470688+0000","last_active":"2024-09-12T02:00:18.575816+0000","last_peered":"2024-09-12T02:00:18.575816+0000","last_clean":"2024-09-12T02:00:18.575816+0000","last_became_active":"2024-09-12T01:59:53.470231+0000","last_became_peered":"2024-09-12T01:59:53.470231+0000","last_unstale":"2024-09-12T02:00:18.575816+0000","last_undegraded":"2024-09-12T02:00:18.575816+0000","last_fullsized":"2024-09-12T02:00:18.575816+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:53:54.439893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":50112,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":65,"num_write":26,"num_write_kb":56,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.575173+0000","last_change":"2024-09-12T01:59:55.495343+0000","last_active":"2024-09-12T02:00:09.575173+0000","last_peered":"2024-09-12T02:00:09.575173+0000","last_clean":"2024-09-12T02:00:09.575173+0000","last_became_active":"2024-09-12T01:59:55.495006+0000","last_became_peered":"2024-09-12T01:59:55.495006+0000","last_unstale":"2024-09-12T02:00:09.575173+0000","last_undegraded":"2024-09-12T02:00:09.575173+0000","last_fullsized":"2024-09-12T02:00:09.575173+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:32:20.932551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'6","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.569082+0000","last_change":"2024-09-12T01:59:53.468625+0000","last_active":"2024-09-12T02:00:09.569082+0000","last_peered":"2024-09-12T02:00:09.569082+0000","last_clean":"2024-09-12T02:00:09.569082+0000","last_became_active":"2024-09-12T01:59:53.468015+0000","last_became_peered":"2024-09-12T01:59:53.468015+0000","last_unstale":"2024-09-12T02:00:09.569082+0000","last_undegraded":"2024-09-12T02:00:09.569082+0000","last_fullsized":"2024-09-12T02:00:09.569082+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:49:34.842239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.562155+0000","last_change":"2024-09-12T01:59:55.487962+0000","last_active":"2024-09-12T02:00:09.562155+0000","last_peered":"2024-09-12T02:00:09.562155+0000","last_clean":"2024-09-12T02:00:09.562155+0000","last_became_active":"2024-09-12T01:59:55.487619+0000","last_became_peered":"2024-09-12T01:59:55.487619+0000","last_unstale":"2024-09-12T02:00:09.562155+0000","last_undegraded":"2024-09-12T02:00:09.562155+0000","last_fullsized":"2024-09-12T02:00:09.562155+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:32:16.882373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"80'3","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.567124+0000","last_change":"2024-09-12T01:59:53.474418+0000","last_active":"2024-09-12T02:00:09.567124+0000","last_peered":"2024-09-12T02:00:09.567124+0000","last_clean":"2024-09-12T02:00:09.567124+0000","last_became_active":"2024-09-12T01:59:53.474059+0000","last_became_peered":"2024-09-12T01:59:53.474059+0000","last_unstale":"2024-09-12T02:00:09.567124+0000","last_undegraded":"2024-09-12T02:00:09.567124+0000","last_fullsized":"2024-09-12T02:00:09.567124+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:10:54.963435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.622740+0000","last_change":"2024-09-12T01:59:55.493394+0000","last_active":"2024-09-12T02:00:09.622740+0000","last_peered":"2024-09-12T02:00:09.622740+0000","last_clean":"2024-09-12T02:00:09.622740+0000","last_became_active":"2024-09-12T01:59:55.491669+0000","last_became_peered":"2024-09-12T01:59:55.491669+0000","last_unstale":"2024-09-12T02:00:09.622740+0000","last_undegraded":"2024-09-12T02:00:09.622740+0000","last_fullsized":"2024-09-12T02:00:09.622740+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:14:40.871154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"80'3","reported_seq":42,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:18.573828+0000","last_change":"2024-09-12T01:59:53.471623+0000","last_active":"2024-09-12T02:00:18.573828+0000","last_peered":"2024-09-12T02:00:18.573828+0000","last_clean":"2024-09-12T02:00:18.573828+0000","last_became_active":"2024-09-12T01:59:53.471294+0000","last_became_peered":"2024-09-12T01:59:53.471294+0000","last_unstale":"2024-09-12T02:00:18.573828+0000","last_undegraded":"2024-09-12T02:00:18.573828+0000","last_fullsized":"2024-09-12T02:00:18.573828+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:51:54.130883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":18,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.621910+0000","last_change":"2024-09-12T01:59:55.493082+0000","last_active":"2024-09-12T02:00:09.621910+0000","last_peered":"2024-09-12T02:00:09.621910+0000","last_clean":"2024-09-12T02:00:09.621910+0000","last_became_active":"2024-09-12T01:59:55.492778+0000","last_became_peered":"2024-09-12T01:59:55.492778+0000","last_unstale":"2024-09-12T02:00:09.621910+0000","last_undegraded":"2024-09-12T02:00:09.621910+0000","last_fullsized":"2024-09-12T02:00:09.621910+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:59:56.466377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'15","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.588050+0000","last_change":"2024-09-12T02:00:09.588050+0000","last_active":"2024-09-12T02:00:09.588050+0000","last_peered":"2024-09-12T02:00:09.588050+0000","last_clean":"2024-09-12T02:00:09.588050+0000","last_became_active":"2024-09-12T02:00:09.575478+0000","last_became_peered":"2024-09-12T02:00:09.575478+0000","last_unstale":"2024-09-12T02:00:09.588050+0000","last_undegraded":"2024-09-12T02:00:09.588050+0000","last_fullsized":"2024-09-12T02:00:09.588050+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:38:45.484167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.566976+0000","last_change":"2024-09-12T01:59:55.497601+0000","last_active":"2024-09-12T02:00:09.566976+0000","last_peered":"2024-09-12T02:00:09.566976+0000","last_clean":"2024-09-12T02:00:09.566976+0000","last_became_active":"2024-09-12T01:59:55.497202+0000","last_became_peered":"2024-09-12T01:59:55.497202+0000","last_unstale":"2024-09-12T02:00:09.566976+0000","last_undegraded":"2024-09-12T02:00:09.566976+0000","last_fullsized":"2024-09-12T02:00:09.566976+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:22:22.850736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"80'2","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.575359+0000","last_change":"2024-09-12T01:59:53.467575+0000","last_active":"2024-09-12T02:00:09.575359+0000","last_peered":"2024-09-12T02:00:09.575359+0000","last_clean":"2024-09-12T02:00:09.575359+0000","last_became_active":"2024-09-12T01:59:53.467146+0000","last_became_peered":"2024-09-12T01:59:53.467146+0000","last_unstale":"2024-09-12T02:00:09.575359+0000","last_undegraded":"2024-09-12T02:00:09.575359+0000","last_fullsized":"2024-09-12T02:00:09.575359+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:07:25.962496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.572034+0000","last_change":"2024-09-12T01:59:55.498297+0000","last_active":"2024-09-12T02:00:09.572034+0000","last_peered":"2024-09-12T02:00:09.572034+0000","last_clean":"2024-09-12T02:00:09.572034+0000","last_became_active":"2024-09-12T01:59:55.497367+0000","last_became_peered":"2024-09-12T01:59:55.497367+0000","last_unstale":"2024-09-12T02:00:09.572034+0000","last_undegraded":"2024-09-12T02:00:09.572034+0000","last_fullsized":"2024-09-12T02:00:09.572034+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:58:32.717710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"80'2","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.562292+0000","last_change":"2024-09-12T01:59:53.466664+0000","last_active":"2024-09-12T02:00:09.562292+0000","last_peered":"2024-09-12T02:00:09.562292+0000","last_clean":"2024-09-12T02:00:09.562292+0000","last_became_active":"2024-09-12T01:59:53.466318+0000","last_became_peered":"2024-09-12T01:59:53.466318+0000","last_unstale":"2024-09-12T02:00:09.562292+0000","last_undegraded":"2024-09-12T02:00:09.562292+0000","last_fullsized":"2024-09-12T02:00:09.562292+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:12:13.541420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.622804+0000","last_change":"2024-09-12T01:59:55.490420+0000","last_active":"2024-09-12T02:00:09.622804+0000","last_peered":"2024-09-12T02:00:09.622804+0000","last_clean":"2024-09-12T02:00:09.622804+0000","last_became_active":"2024-09-12T01:59:55.489757+0000","last_became_peered":"2024-09-12T01:59:55.489757+0000","last_unstale":"2024-09-12T02:00:09.622804+0000","last_undegraded":"2024-09-12T02:00:09.622804+0000","last_fullsized":"2024-09-12T02:00:09.622804+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:34:43.117142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'1","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.567185+0000","last_change":"2024-09-12T01:59:53.466399+0000","last_active":"2024-09-12T02:00:09.567185+0000","last_peered":"2024-09-12T02:00:09.567185+0000","last_clean":"2024-09-12T02:00:09.567185+0000","last_became_active":"2024-09-12T01:59:53.465982+0000","last_became_peered":"2024-09-12T01:59:53.465982+0000","last_unstale":"2024-09-12T02:00:09.567185+0000","last_undegraded":"2024-09-12T02:00:09.567185+0000","last_fullsized":"2024-09-12T02:00:09.567185+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:18:02.015040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"82'1","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.622907+0000","last_change":"2024-09-12T01:59:55.491527+0000","last_active":"2024-09-12T02:00:09.622907+0000","last_peered":"2024-09-12T02:00:09.622907+0000","last_clean":"2024-09-12T02:00:09.622907+0000","last_became_active":"2024-09-12T01:59:55.491165+0000","last_became_peered":"2024-09-12T01:59:55.491165+0000","last_unstale":"2024-09-12T02:00:09.622907+0000","last_undegraded":"2024-09-12T02:00:09.622907+0000","last_fullsized":"2024-09-12T02:00:09.622907+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:57:05.506384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"80'3","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.569187+0000","last_change":"2024-09-12T01:59:54.075752+0000","last_active":"2024-09-12T02:00:09.569187+0000","last_peered":"2024-09-12T02:00:09.569187+0000","last_clean":"2024-09-12T02:00:09.569187+0000","last_became_active":"2024-09-12T01:59:54.075387+0000","last_became_peered":"2024-09-12T01:59:54.075387+0000","last_unstale":"2024-09-12T02:00:09.569187+0000","last_undegraded":"2024-09-12T02:00:09.569187+0000","last_fullsized":"2024-09-12T02:00:09.569187+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:21:15.980591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.622907+0000","last_change":"2024-09-12T01:59:55.500935+0000","last_active":"2024-09-12T02:00:09.622907+0000","last_peered":"2024-09-12T02:00:09.622907+0000","last_clean":"2024-09-12T02:00:09.622907+0000","last_became_active":"2024-09-12T01:59:55.500206+0000","last_became_peered":"2024-09-12T01:59:55.500206+0000","last_unstale":"2024-09-12T02:00:09.622907+0000","last_undegraded":"2024-09-12T02:00:09.622907+0000","last_fullsized":"2024-09-12T02:00:09.622907+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:19:59.937701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":333,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.623202+0000","last_change":"2024-09-12T01:58:47.553853+0000","last_active":"2024-09-12T02:00:09.623202+0000","last_peered":"2024-09-12T02:00:09.623202+0000","last_clean":"2024-09-12T02:00:09.623202+0000","last_became_active":"2024-09-12T01:58:47.553357+0000","last_became_peered":"2024-09-12T01:58:47.553357+0000","last_unstale":"2024-09-12T02:00:09.623202+0000","last_undegraded":"2024-09-12T02:00:09.623202+0000","last_fullsized":"2024-09-12T02:00:09.623202+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:55:26.917733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:55:26.917733+0000","last_clean_scrub_stamp":"2024-09-12T01:55:26.917733+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:58:26.081766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"80'3","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.563615+0000","last_change":"2024-09-12T01:59:53.467907+0000","last_active":"2024-09-12T02:00:09.563615+0000","last_peered":"2024-09-12T02:00:09.563615+0000","last_clean":"2024-09-12T02:00:09.563615+0000","last_became_active":"2024-09-12T01:59:53.467164+0000","last_became_peered":"2024-09-12T01:59:53.467164+0000","last_unstale":"2024-09-12T02:00:09.563615+0000","last_undegraded":"2024-09-12T02:00:09.563615+0000","last_fullsized":"2024-09-12T02:00:09.563615+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:09:35.242708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.621290+0000","last_change":"2024-09-12T01:59:55.495109+0000","last_active":"2024-09-12T02:00:09.621290+0000","last_peered":"2024-09-12T02:00:09.621290+0000","last_clean":"2024-09-12T02:00:09.621290+0000","last_became_active":"2024-09-12T01:59:55.494791+0000","last_became_peered":"2024-09-12T01:59:55.494791+0000","last_unstale":"2024-09-12T02:00:09.621290+0000","last_undegraded":"2024-09-12T02:00:09.621290+0000","last_fullsized":"2024-09-12T02:00:09.621290+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:22:43.882050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"80'3","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.623115+0000","last_change":"2024-09-12T01:59:53.473286+0000","last_active":"2024-09-12T02:00:09.623115+0000","last_peered":"2024-09-12T02:00:09.623115+0000","last_clean":"2024-09-12T02:00:09.623115+0000","last_became_active":"2024-09-12T01:59:53.472976+0000","last_became_peered":"2024-09-12T01:59:53.472976+0000","last_unstale":"2024-09-12T02:00:09.623115+0000","last_undegraded":"2024-09-12T02:00:09.623115+0000","last_fullsized":"2024-09-12T02:00:09.623115+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:32:39.604173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.571159+0000","last_change":"2024-09-12T02:00:09.571159+0000","last_active":"2024-09-12T02:00:09.571159+0000","last_peered":"2024-09-12T02:00:09.571159+0000","last_clean":"2024-09-12T02:00:09.571159+0000","last_became_active":"2024-09-12T02:00:09.570672+0000","last_became_peered":"2024-09-12T02:00:09.570672+0000","last_unstale":"2024-09-12T02:00:09.571159+0000","last_undegraded":"2024-09-12T02:00:09.571159+0000","last_fullsized":"2024-09-12T02:00:09.571159+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:04:10.808942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"80'4","reported_seq":41,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:18.572794+0000","last_change":"2024-09-12T01:59:53.921856+0000","last_active":"2024-09-12T02:00:18.572794+0000","last_peered":"2024-09-12T02:00:18.572794+0000","last_clean":"2024-09-12T02:00:18.572794+0000","last_became_active":"2024-09-12T01:59:53.920672+0000","last_became_peered":"2024-09-12T01:59:53.920672+0000","last_unstale":"2024-09-12T02:00:18.572794+0000","last_undegraded":"2024-09-12T02:00:18.572794+0000","last_fullsized":"2024-09-12T02:00:18.572794+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:42:33.876493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.571600+0000","last_change":"2024-09-12T02:00:09.571185+0000","last_active":"2024-09-12T02:00:09.571600+0000","last_peered":"2024-09-12T02:00:09.571600+0000","last_clean":"2024-09-12T02:00:09.571600+0000","last_became_active":"2024-09-12T02:00:09.570769+0000","last_became_peered":"2024-09-12T02:00:09.570769+0000","last_unstale":"2024-09-12T02:00:09.571600+0000","last_undegraded":"2024-09-12T02:00:09.571600+0000","last_fullsized":"2024-09-12T02:00:09.571600+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:34:17.230435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.2","version":"80'1","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.622691+0000","last_change":"2024-09-12T01:59:53.472072+0000","last_active":"2024-09-12T02:00:09.622691+0000","last_peered":"2024-09-12T02:00:09.622691+0000","last_clean":"2024-09-12T02:00:09.622691+0000","last_became_active":"2024-09-12T01:59:53.471088+0000","last_became_peered":"2024-09-12T01:59:53.471088+0000","last_unstale":"2024-09-12T02:00:09.622691+0000","last_undegraded":"2024-09-12T02:00:09.622691+0000","last_fullsized":"2024-09-12T02:00:09.622691+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:02:13.879131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.622413+0000","last_change":"2024-09-12T01:59:55.502843+0000","last_active":"2024-09-12T02:00:09.622413+0000","last_peered":"2024-09-12T02:00:09.622413+0000","last_clean":"2024-09-12T02:00:09.622413+0000","last_became_active":"2024-09-12T01:59:55.499296+0000","last_became_peered":"2024-09-12T01:59:55.499296+0000","last_unstale":"2024-09-12T02:00:09.622413+0000","last_undegraded":"2024-09-12T02:00:09.622413+0000","last_fullsized":"2024-09-12T02:00:09.622413+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:04:21.787370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.575061+0000","last_change":"2024-09-12T02:00:09.575061+0000","last_active":"2024-09-12T02:00:09.575061+0000","last_peered":"2024-09-12T02:00:09.575061+0000","last_clean":"2024-09-12T02:00:09.575061+0000","last_became_active":"2024-09-12T02:00:09.574653+0000","last_became_peered":"2024-09-12T02:00:09.574653+0000","last_unstale":"2024-09-12T02:00:09.575061+0000","last_undegraded":"2024-09-12T02:00:09.575061+0000","last_fullsized":"2024-09-12T02:00:09.575061+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:32:27.931211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"82'1","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.622315+0000","last_change":"2024-09-12T01:59:55.494162+0000","last_active":"2024-09-12T02:00:09.622315+0000","last_peered":"2024-09-12T02:00:09.622315+0000","last_clean":"2024-09-12T02:00:09.622315+0000","last_became_active":"2024-09-12T01:59:55.493802+0000","last_became_peered":"2024-09-12T01:59:55.493802+0000","last_unstale":"2024-09-12T02:00:09.622315+0000","last_undegraded":"2024-09-12T02:00:09.622315+0000","last_fullsized":"2024-09-12T02:00:09.622315+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:22:16.302683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"80'1","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.575278+0000","last_change":"2024-09-12T01:59:53.921669+0000","last_active":"2024-09-12T02:00:09.575278+0000","last_peered":"2024-09-12T02:00:09.575278+0000","last_clean":"2024-09-12T02:00:09.575278+0000","last_became_active":"2024-09-12T01:59:53.921328+0000","last_became_peered":"2024-09-12T01:59:53.921328+0000","last_unstale":"2024-09-12T02:00:09.575278+0000","last_undegraded":"2024-09-12T02:00:09.575278+0000","last_fullsized":"2024-09-12T02:00:09.575278+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:46:33.313726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.562329+0000","last_change":"2024-09-12T01:59:55.488986+0000","last_active":"2024-09-12T02:00:09.562329+0000","last_peered":"2024-09-12T02:00:09.562329+0000","last_clean":"2024-09-12T02:00:09.562329+0000","last_became_active":"2024-09-12T01:59:55.488632+0000","last_became_peered":"2024-09-12T01:59:55.488632+0000","last_unstale":"2024-09-12T02:00:09.562329+0000","last_undegraded":"2024-09-12T02:00:09.562329+0000","last_fullsized":"2024-09-12T02:00:09.562329+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:20:57.596976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"82'8","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:16.500951+0000","last_change":"2024-09-12T01:59:53.467648+0000","last_active":"2024-09-12T02:00:16.500951+0000","last_peered":"2024-09-12T02:00:16.500951+0000","last_clean":"2024-09-12T02:00:16.500951+0000","last_became_active":"2024-09-12T01:59:53.467255+0000","last_became_peered":"2024-09-12T01:59:53.467255+0000","last_unstale":"2024-09-12T02:00:16.500951+0000","last_undegraded":"2024-09-12T02:00:16.500951+0000","last_fullsized":"2024-09-12T02:00:16.500951+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:37:09.897388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.621489+0000","last_change":"2024-09-12T01:59:55.487603+0000","last_active":"2024-09-12T02:00:09.621489+0000","last_peered":"2024-09-12T02:00:09.621489+0000","last_clean":"2024-09-12T02:00:09.621489+0000","last_became_active":"2024-09-12T01:59:55.487334+0000","last_became_peered":"2024-09-12T01:59:55.487334+0000","last_unstale":"2024-09-12T02:00:09.621489+0000","last_undegraded":"2024-09-12T02:00:09.621489+0000","last_fullsized":"2024-09-12T02:00:09.621489+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:23:16.669515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'4","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.563870+0000","last_change":"2024-09-12T01:59:54.076156+0000","last_active":"2024-09-12T02:00:09.563870+0000","last_peered":"2024-09-12T02:00:09.563870+0000","last_clean":"2024-09-12T02:00:09.563870+0000","last_became_active":"2024-09-12T01:59:54.075794+0000","last_became_peered":"2024-09-12T01:59:54.075794+0000","last_unstale":"2024-09-12T02:00:09.563870+0000","last_undegraded":"2024-09-12T02:00:09.563870+0000","last_fullsized":"2024-09-12T02:00:09.563870+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:06:17.431696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.573860+0000","last_change":"2024-09-12T01:59:55.494604+0000","last_active":"2024-09-12T02:00:09.573860+0000","last_peered":"2024-09-12T02:00:09.573860+0000","last_clean":"2024-09-12T02:00:09.573860+0000","last_became_active":"2024-09-12T01:59:55.494300+0000","last_became_peered":"2024-09-12T01:59:55.494300+0000","last_unstale":"2024-09-12T02:00:09.573860+0000","last_undegraded":"2024-09-12T02:00:09.573860+0000","last_fullsized":"2024-09-12T02:00:09.573860+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:18:56.359299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.622205+0000","last_change":"2024-09-12T01:59:53.921832+0000","last_active":"2024-09-12T02:00:09.622205+0000","last_peered":"2024-09-12T02:00:09.622205+0000","last_clean":"2024-09-12T02:00:09.622205+0000","last_became_active":"2024-09-12T01:59:53.921379+0000","last_became_peered":"2024-09-12T01:59:53.921379+0000","last_unstale":"2024-09-12T02:00:09.622205+0000","last_undegraded":"2024-09-12T02:00:09.622205+0000","last_fullsized":"2024-09-12T02:00:09.622205+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:26:02.277149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.574438+0000","last_change":"2024-09-12T01:59:55.493198+0000","last_active":"2024-09-12T02:00:09.574438+0000","last_peered":"2024-09-12T02:00:09.574438+0000","last_clean":"2024-09-12T02:00:09.574438+0000","last_became_active":"2024-09-12T01:59:55.492834+0000","last_became_peered":"2024-09-12T01:59:55.492834+0000","last_unstale":"2024-09-12T02:00:09.574438+0000","last_undegraded":"2024-09-12T02:00:09.574438+0000","last_fullsized":"2024-09-12T02:00:09.574438+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:58:43.872921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"80'3","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.621258+0000","last_change":"2024-09-12T01:59:53.466528+0000","last_active":"2024-09-12T02:00:09.621258+0000","last_peered":"2024-09-12T02:00:09.621258+0000","last_clean":"2024-09-12T02:00:09.621258+0000","last_became_active":"2024-09-12T01:59:53.466182+0000","last_became_peered":"2024-09-12T01:59:53.466182+0000","last_unstale":"2024-09-12T02:00:09.621258+0000","last_undegraded":"2024-09-12T02:00:09.621258+0000","last_fullsized":"2024-09-12T02:00:09.621258+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:00:26.083539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.572154+0000","last_change":"2024-09-12T01:59:55.494511+0000","last_active":"2024-09-12T02:00:09.572154+0000","last_peered":"2024-09-12T02:00:09.572154+0000","last_clean":"2024-09-12T02:00:09.572154+0000","last_became_active":"2024-09-12T01:59:55.493607+0000","last_became_peered":"2024-09-12T01:59:55.493607+0000","last_unstale":"2024-09-12T02:00:09.572154+0000","last_undegraded":"2024-09-12T02:00:09.572154+0000","last_fullsized":"2024-09-12T02:00:09.572154+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:55:27.434447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"80'1","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.569211+0000","last_change":"2024-09-12T01:59:53.467385+0000","last_active":"2024-09-12T02:00:09.569211+0000","last_peered":"2024-09-12T02:00:09.569211+0000","last_clean":"2024-09-12T02:00:09.569211+0000","last_became_active":"2024-09-12T01:59:53.466997+0000","last_became_peered":"2024-09-12T01:59:53.466997+0000","last_unstale":"2024-09-12T02:00:09.569211+0000","last_undegraded":"2024-09-12T02:00:09.569211+0000","last_fullsized":"2024-09-12T02:00:09.569211+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:54:47.154036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.621293+0000","last_change":"2024-09-12T01:59:55.490757+0000","last_active":"2024-09-12T02:00:09.621293+0000","last_peered":"2024-09-12T02:00:09.621293+0000","last_clean":"2024-09-12T02:00:09.621293+0000","last_became_active":"2024-09-12T01:59:55.490451+0000","last_became_peered":"2024-09-12T01:59:55.490451+0000","last_unstale":"2024-09-12T02:00:09.621293+0000","last_undegraded":"2024-09-12T02:00:09.621293+0000","last_fullsized":"2024-09-12T02:00:09.621293+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:21:26.139049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.574833+0000","last_change":"2024-09-12T01:59:53.472898+0000","last_active":"2024-09-12T02:00:09.574833+0000","last_peered":"2024-09-12T02:00:09.574833+0000","last_clean":"2024-09-12T02:00:09.574833+0000","last_became_active":"2024-09-12T01:59:53.472561+0000","last_became_peered":"2024-09-12T01:59:53.472561+0000","last_unstale":"2024-09-12T02:00:09.574833+0000","last_undegraded":"2024-09-12T02:00:09.574833+0000","last_fullsized":"2024-09-12T02:00:09.574833+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:55:59.933804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.622500+0000","last_change":"2024-09-12T01:59:55.497731+0000","last_active":"2024-09-12T02:00:09.622500+0000","last_peered":"2024-09-12T02:00:09.622500+0000","last_clean":"2024-09-12T02:00:09.622500+0000","last_became_active":"2024-09-12T01:59:55.497356+0000","last_became_peered":"2024-09-12T01:59:55.497356+0000","last_unstale":"2024-09-12T02:00:09.622500+0000","last_undegraded":"2024-09-12T02:00:09.622500+0000","last_fullsized":"2024-09-12T02:00:09.622500+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:57:18.795865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.624453+0000","last_change":"2024-09-12T02:00:09.624453+0000","last_active":"2024-09-12T02:00:09.624453+0000","last_peered":"2024-09-12T02:00:09.624453+0000","last_clean":"2024-09-12T02:00:09.624453+0000","last_became_active":"2024-09-12T02:00:09.624049+0000","last_became_peered":"2024-09-12T02:00:09.624049+0000","last_unstale":"2024-09-12T02:00:09.624453+0000","last_undegraded":"2024-09-12T02:00:09.624453+0000","last_fullsized":"2024-09-12T02:00:09.624453+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:17:40.417275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"80'2","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.575060+0000","last_change":"2024-09-12T01:59:53.470194+0000","last_active":"2024-09-12T02:00:09.575060+0000","last_peered":"2024-09-12T02:00:09.575060+0000","last_clean":"2024-09-12T02:00:09.575060+0000","last_became_active":"2024-09-12T01:59:53.469772+0000","last_became_peered":"2024-09-12T01:59:53.469772+0000","last_unstale":"2024-09-12T02:00:09.575060+0000","last_undegraded":"2024-09-12T02:00:09.575060+0000","last_fullsized":"2024-09-12T02:00:09.575060+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:52:57.709596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.622423+0000","last_change":"2024-09-12T01:59:55.497693+0000","last_active":"2024-09-12T02:00:09.622423+0000","last_peered":"2024-09-12T02:00:09.622423+0000","last_clean":"2024-09-12T02:00:09.622423+0000","last_became_active":"2024-09-12T01:59:55.497380+0000","last_became_peered":"2024-09-12T01:59:55.497380+0000","last_unstale":"2024-09-12T02:00:09.622423+0000","last_undegraded":"2024-09-12T02:00:09.622423+0000","last_fullsized":"2024-09-12T02:00:09.622423+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:42:27.943530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"80'4","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.575274+0000","last_change":"2024-09-12T01:59:53.473044+0000","last_active":"2024-09-12T02:00:09.575274+0000","last_peered":"2024-09-12T02:00:09.575274+0000","last_clean":"2024-09-12T02:00:09.575274+0000","last_became_active":"2024-09-12T01:59:53.472569+0000","last_became_peered":"2024-09-12T01:59:53.472569+0000","last_unstale":"2024-09-12T02:00:09.575274+0000","last_undegraded":"2024-09-12T02:00:09.575274+0000","last_fullsized":"2024-09-12T02:00:09.575274+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:41:29.716836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.573956+0000","last_change":"2024-09-12T01:59:55.497377+0000","last_active":"2024-09-12T02:00:09.573956+0000","last_peered":"2024-09-12T02:00:09.573956+0000","last_clean":"2024-09-12T02:00:09.573956+0000","last_became_active":"2024-09-12T01:59:55.497078+0000","last_became_peered":"2024-09-12T01:59:55.497078+0000","last_unstale":"2024-09-12T02:00:09.573956+0000","last_undegraded":"2024-09-12T02:00:09.573956+0000","last_fullsized":"2024-09-12T02:00:09.573956+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:59:57.362350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.621555+0000","last_change":"2024-09-12T01:59:53.920868+0000","last_active":"2024-09-12T02:00:09.621555+0000","last_peered":"2024-09-12T02:00:09.621555+0000","last_clean":"2024-09-12T02:00:09.621555+0000","last_became_active":"2024-09-12T01:59:53.920481+0000","last_became_peered":"2024-09-12T01:59:53.920481+0000","last_unstale":"2024-09-12T02:00:09.621555+0000","last_undegraded":"2024-09-12T02:00:09.621555+0000","last_fullsized":"2024-09-12T02:00:09.621555+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:49:27.085388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.621438+0000","last_change":"2024-09-12T01:59:55.497703+0000","last_active":"2024-09-12T02:00:09.621438+0000","last_peered":"2024-09-12T02:00:09.621438+0000","last_clean":"2024-09-12T02:00:09.621438+0000","last_became_active":"2024-09-12T01:59:55.497397+0000","last_became_peered":"2024-09-12T01:59:55.497397+0000","last_unstale":"2024-09-12T02:00:09.621438+0000","last_undegraded":"2024-09-12T02:00:09.621438+0000","last_fullsized":"2024-09-12T02:00:09.621438+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:54:02.803203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"80'9","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.565145+0000","last_change":"2024-09-12T01:59:53.469978+0000","last_active":"2024-09-12T02:00:09.565145+0000","last_peered":"2024-09-12T02:00:09.565145+0000","last_clean":"2024-09-12T02:00:09.565145+0000","last_became_active":"2024-09-12T01:59:53.469648+0000","last_became_peered":"2024-09-12T01:59:53.469648+0000","last_unstale":"2024-09-12T02:00:09.565145+0000","last_undegraded":"2024-09-12T02:00:09.565145+0000","last_fullsized":"2024-09-12T02:00:09.565145+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:08:37.720452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":54115,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":112,"num_read_kb":112,"num_write":154,"num_write_kb":96,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":162345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":132,"ondisk_log_size":132,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3563520,"data_stored":3542208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":74,"seq":317827579921,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32068,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713084,"statfs":{"total":95995035648,"available":95962198016,"internally_reserved":0,"allocated":897024,"data_stored":435916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":67,"seq":287762808854,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32724,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712428,"statfs":{"total":95995035648,"available":95961526272,"internally_reserved":0,"allocated":1576960,"data_stored":1077397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":62,"seq":266287972379,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":905216,"data_stored":436040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201311,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":897024,"data_stored":435848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430245,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32648,"kb_used_data":1464,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712504,"statfs":{"total":95995035648,"available":95961604096,"internally_reserved":0,"allocated":1499136,"data_stored":1027892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593770,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36216,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708936,"statfs":{"total":95995035648,"available":95957950464,"internally_reserved":0,"allocated":892928,"data_stored":435429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822703,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32660,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712492,"statfs":{"total":95995035648,"available":95961591808,"internally_reserved":0,"allocated":1511424,"data_stored":1027576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986227,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":897024,"data_stored":435866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215160,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":909312,"data_stored":435740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":17,"seq":73014444094,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36292,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708860,"statfs":{"total":95995035648,"available":95957872640,"internally_reserved":0,"allocated":970752,"data_stored":487077,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574915,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32108,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713044,"statfs":{"total":95995035648,"available":95962157056,"internally_reserved":0,"allocated":946176,"data_stored":485334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738440,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":913408,"data_stored":436700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":50112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":51855,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":51690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T02:00:23.078 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-12T02:00:23.078 INFO:tasks.cephadm:Setting up client nodes... 2024-09-12T02:00:23.079 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-12T02:00:23.337 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:00:23.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:23.777+0000 7f8f716a7640 1 -- 172.21.15.16:0/2443190246 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f6c100ad0 msgr2=0x7f8f6c100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:23.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:23.777+0000 7f8f716a7640 1 --2- 172.21.15.16:0/2443190246 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f6c100ad0 0x7f8f6c100ed0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f8f5c009ca0 tx=0x7f8f5c02a8c0 comp rx=0 tx=0).stop 2024-09-12T02:00:23.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:23.777+0000 7f8f716a7640 1 -- 172.21.15.16:0/2443190246 shutdown_connections 2024-09-12T02:00:23.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:23.777+0000 7f8f716a7640 1 --2- 172.21.15.16:0/2443190246 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f6c102690 0x7f8f6c108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:23.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:23.777+0000 7f8f716a7640 1 --2- 172.21.15.16:0/2443190246 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8f6c101cd0 0x7f8f6c102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:23.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:23.777+0000 7f8f716a7640 1 --2- 172.21.15.16:0/2443190246 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f6c100ad0 0x7f8f6c100ed0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:23.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:23.777+0000 7f8f716a7640 1 -- 172.21.15.16:0/2443190246 >> 172.21.15.16:0/2443190246 conn(0x7f8f6c0fc280 msgr2=0x7f8f6c0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:00:23.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:23.777+0000 7f8f716a7640 1 -- 172.21.15.16:0/2443190246 shutdown_connections 2024-09-12T02:00:23.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:23.777+0000 7f8f716a7640 1 -- 172.21.15.16:0/2443190246 wait complete. 2024-09-12T02:00:23.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:23.778+0000 7f8f716a7640 1 Processor -- start 2024-09-12T02:00:23.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:23.778+0000 7f8f716a7640 1 -- start start 2024-09-12T02:00:23.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:23.778+0000 7f8f716a7640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f6c100ad0 0x7f8f6c06ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:23.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:23.778+0000 7f8f716a7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f6c101cd0 0x7f8f6c19cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:23.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:23.778+0000 7f8f716a7640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8f6c102690 0x7f8f6c19f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:23.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:23.778+0000 7f8f716a7640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8f6c10c800 con 0x7f8f6c101cd0 2024-09-12T02:00:23.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:23.778+0000 7f8f716a7640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8f6c10c680 con 0x7f8f6c100ad0 2024-09-12T02:00:23.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:23.778+0000 7f8f716a7640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8f6c10c980 con 0x7f8f6c102690 2024-09-12T02:00:23.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:23.778+0000 7f8f6a7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f6c101cd0 0x7f8f6c19cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:23.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:23.779+0000 7f8f6a7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f6c101cd0 0x7f8f6c19cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43150/0 (socket says 172.21.15.16:43150) 2024-09-12T02:00:23.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:23.779+0000 7f8f6a7fc640 1 -- 172.21.15.16:0/3079744912 learned_addr learned my addr 172.21.15.16:0/3079744912 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:00:23.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:23.779+0000 7f8f6affd640 1 --2- 172.21.15.16:0/3079744912 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f6c100ad0 0x7f8f6c06ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:23.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:23.779+0000 7f8f6b7fe640 1 --2- 172.21.15.16:0/3079744912 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8f6c102690 0x7f8f6c19f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:23.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:23.779+0000 7f8f6a7fc640 1 -- 172.21.15.16:0/3079744912 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8f6c102690 msgr2=0x7f8f6c19f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:23.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:23.779+0000 7f8f6a7fc640 1 --2- 172.21.15.16:0/3079744912 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8f6c102690 0x7f8f6c19f2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:23.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:23.779+0000 7f8f6a7fc640 1 -- 172.21.15.16:0/3079744912 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f6c100ad0 msgr2=0x7f8f6c06ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:23.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:23.779+0000 7f8f6a7fc640 1 --2- 172.21.15.16:0/3079744912 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f6c100ad0 0x7f8f6c06ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:23.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:23.779+0000 7f8f6a7fc640 1 -- 172.21.15.16:0/3079744912 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f6c19fb60 con 0x7f8f6c101cd0 2024-09-12T02:00:23.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:23.779+0000 7f8f6b7fe640 1 --2- 172.21.15.16:0/3079744912 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8f6c102690 0x7f8f6c19f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:00:23.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:23.779+0000 7f8f6affd640 1 --2- 172.21.15.16:0/3079744912 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f6c100ad0 0x7f8f6c06ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:00:23.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:23.779+0000 7f8f6a7fc640 1 --2- 172.21.15.16:0/3079744912 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f6c101cd0 0x7f8f6c19cc50 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f8f6000b4a0 tx=0x7f8f6000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:00:23.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:23.779+0000 7f8f53fff640 1 -- 172.21.15.16:0/3079744912 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f60014070 con 0x7f8f6c101cd0 2024-09-12T02:00:23.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:23.779+0000 7f8f53fff640 1 -- 172.21.15.16:0/3079744912 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f60010af0 con 0x7f8f6c101cd0 2024-09-12T02:00:23.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:23.779+0000 7f8f53fff640 1 -- 172.21.15.16:0/3079744912 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f60017d10 con 0x7f8f6c101cd0 2024-09-12T02:00:23.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:23.779+0000 7f8f716a7640 1 -- 172.21.15.16:0/3079744912 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f6c19fe50 con 0x7f8f6c101cd0 2024-09-12T02:00:23.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:23.779+0000 7f8f716a7640 1 -- 172.21.15.16:0/3079744912 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f6c1a0330 con 0x7f8f6c101cd0 2024-09-12T02:00:23.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:23.780+0000 7f8f716a7640 1 -- 172.21.15.16:0/3079744912 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f6c107e80 con 0x7f8f6c101cd0 2024-09-12T02:00:23.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:23.782+0000 7f8f53fff640 1 -- 172.21.15.16:0/3079744912 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f8f60017070 con 0x7f8f6c101cd0 2024-09-12T02:00:23.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:23.782+0000 7f8f53fff640 1 --2- 172.21.15.16:0/3079744912 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f8f540a2c90 0x7f8f540a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:23.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:23.782+0000 7f8f6affd640 1 --2- 172.21.15.16:0/3079744912 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f8f540a2c90 0x7f8f540a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:23.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:23.782+0000 7f8f53fff640 1 -- 172.21.15.16:0/3079744912 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f8f600ce3d0 con 0x7f8f6c101cd0 2024-09-12T02:00:23.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:23.783+0000 7f8f6affd640 1 --2- 172.21.15.16:0/3079744912 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f8f540a2c90 0x7f8f540a5150 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f8f6c06b050 tx=0x7f8f5c031bb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:00:23.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:23.787+0000 7f8f53fff640 1 -- 172.21.15.16:0/3079744912 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8f6009b3e0 con 0x7f8f6c101cd0 2024-09-12T02:00:24.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:24.030+0000 7f8f716a7640 1 -- 172.21.15.16:0/3079744912 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f8f6c02da10 con 0x7f8f6c101cd0 2024-09-12T02:00:24.032 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:00:23 smithi016 ceph-mon[30947]: pgmap v289: 97 pgs: 97 active+clean; 630 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 4.9 KiB/s wr, 8 op/s; 9 B/s, 0 objects/s recovering 2024-09-12T02:00:24.032 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:00:23 smithi016 ceph-mon[30947]: from='client.14745 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T02:00:24.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:24.033+0000 7f8f53fff640 1 -- 172.21.15.16:0/3079744912 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f8f6009ace0 con 0x7f8f6c101cd0 2024-09-12T02:00:24.034 INFO:teuthology.orchestra.run.smithi016.stdout:[client.0] 2024-09-12T02:00:24.034 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQA4S+JmqTD7ARAAHQAuarWQCyIhayRy+S5Vog== 2024-09-12T02:00:24.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:24.035+0000 7f8f716a7640 1 -- 172.21.15.16:0/3079744912 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f8f540a2c90 msgr2=0x7f8f540a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:24.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:24.035+0000 7f8f716a7640 1 --2- 172.21.15.16:0/3079744912 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f8f540a2c90 0x7f8f540a5150 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f8f6c06b050 tx=0x7f8f5c031bb0 comp rx=0 tx=0).stop 2024-09-12T02:00:24.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:24.035+0000 7f8f716a7640 1 -- 172.21.15.16:0/3079744912 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f6c101cd0 msgr2=0x7f8f6c19cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:24.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:24.035+0000 7f8f716a7640 1 --2- 172.21.15.16:0/3079744912 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f6c101cd0 0x7f8f6c19cc50 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f8f6000b4a0 tx=0x7f8f6000b960 comp rx=0 tx=0).stop 2024-09-12T02:00:24.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:24.035+0000 7f8f716a7640 1 -- 172.21.15.16:0/3079744912 shutdown_connections 2024-09-12T02:00:24.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:24.035+0000 7f8f716a7640 1 --2- 172.21.15.16:0/3079744912 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f8f540a2c90 0x7f8f540a5150 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:24.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:24.035+0000 7f8f716a7640 1 --2- 172.21.15.16:0/3079744912 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8f6c102690 0x7f8f6c19f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:24.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:24.035+0000 7f8f716a7640 1 --2- 172.21.15.16:0/3079744912 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f6c101cd0 0x7f8f6c19cc50 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:24.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:24.035+0000 7f8f716a7640 1 --2- 172.21.15.16:0/3079744912 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f6c100ad0 0x7f8f6c06ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:24.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:24.035+0000 7f8f716a7640 1 -- 172.21.15.16:0/3079744912 >> 172.21.15.16:0/3079744912 conn(0x7f8f6c0fc280 msgr2=0x7f8f6c105050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:00:24.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:24.035+0000 7f8f716a7640 1 -- 172.21.15.16:0/3079744912 shutdown_connections 2024-09-12T02:00:24.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:24.035+0000 7f8f716a7640 1 -- 172.21.15.16:0/3079744912 wait complete. 2024-09-12T02:00:24.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:00:23 smithi184 ceph-mon[35593]: pgmap v289: 97 pgs: 97 active+clean; 630 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 4.9 KiB/s wr, 8 op/s; 9 B/s, 0 objects/s recovering 2024-09-12T02:00:24.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:00:23 smithi184 ceph-mon[35593]: from='client.14745 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T02:00:24.168 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:00:23 smithi033 ceph-mon[35332]: pgmap v289: 97 pgs: 97 active+clean; 630 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 4.9 KiB/s wr, 8 op/s; 9 B/s, 0 objects/s recovering 2024-09-12T02:00:24.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:00:23 smithi033 ceph-mon[35332]: from='client.14745 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T02:00:24.693 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-12T02:00:24.693 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-09-12T02:00:24.694 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-12T02:00:24.734 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-12T02:00:24.735 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-09-12T02:00:24.735 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph mgr dump --format=json 2024-09-12T02:00:24.933 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:00:24 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/3079744912' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-12T02:00:24.934 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:00:24 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/3079744912' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-12T02:00:25.032 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:00:25.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:00:24 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/3079744912' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-12T02:00:25.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:00:24 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/3079744912' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-12T02:00:25.168 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:00:24 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/3079744912' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-12T02:00:25.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:00:24 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/3079744912' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-12T02:00:25.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:25.520+0000 7f172aadb640 1 -- 172.21.15.16:0/1157599838 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1724100500 msgr2=0x7f172410ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:25.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:25.520+0000 7f172aadb640 1 --2- 172.21.15.16:0/1157599838 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1724100500 0x7f172410ea20 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f171c009ca0 tx=0x7f171c02a8c0 comp rx=0 tx=0).stop 2024-09-12T02:00:25.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:25.520+0000 7f172aadb640 1 -- 172.21.15.16:0/1157599838 shutdown_connections 2024-09-12T02:00:25.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:25.520+0000 7f172aadb640 1 --2- 172.21.15.16:0/1157599838 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1724100500 0x7f172410ea20 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:25.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:25.520+0000 7f172aadb640 1 --2- 172.21.15.16:0/1157599838 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f17240ffb60 0x7f17240fffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:25.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:25.520+0000 7f172aadb640 1 --2- 172.21.15.16:0/1157599838 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1724106b80 0x7f1724106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:25.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:25.520+0000 7f172aadb640 1 -- 172.21.15.16:0/1157599838 >> 172.21.15.16:0/1157599838 conn(0x7f17240fba90 msgr2=0x7f17240fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:00:25.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:25.520+0000 7f172aadb640 1 -- 172.21.15.16:0/1157599838 shutdown_connections 2024-09-12T02:00:25.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:25.520+0000 7f172aadb640 1 -- 172.21.15.16:0/1157599838 wait complete. 2024-09-12T02:00:25.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:25.521+0000 7f172aadb640 1 Processor -- start 2024-09-12T02:00:25.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:25.521+0000 7f172aadb640 1 -- start start 2024-09-12T02:00:25.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:25.521+0000 7f172aadb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17240ffb60 0x7f172419a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:25.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:25.521+0000 7f172aadb640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1724100500 0x7f172419ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:25.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:25.521+0000 7f172aadb640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1724106b80 0x7f172419eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:25.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:25.521+0000 7f172aadb640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1724111fa0 con 0x7f17240ffb60 2024-09-12T02:00:25.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:25.521+0000 7f172aadb640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1724111e20 con 0x7f1724106b80 2024-09-12T02:00:25.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:25.521+0000 7f172aadb640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1724112120 con 0x7f1724100500 2024-09-12T02:00:25.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:25.521+0000 7f1728877640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17240ffb60 0x7f172419a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:25.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:25.521+0000 7f1728877640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17240ffb60 0x7f172419a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43164/0 (socket says 172.21.15.16:43164) 2024-09-12T02:00:25.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:25.521+0000 7f1728877640 1 -- 172.21.15.16:0/4265834014 learned_addr learned my addr 172.21.15.16:0/4265834014 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:00:25.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:25.521+0000 7f1729078640 1 --2- 172.21.15.16:0/4265834014 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1724106b80 0x7f172419eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:25.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:25.522+0000 7f1723fff640 1 --2- 172.21.15.16:0/4265834014 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1724100500 0x7f172419ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:25.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:25.522+0000 7f1728877640 1 -- 172.21.15.16:0/4265834014 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1724100500 msgr2=0x7f172419ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:25.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:25.522+0000 7f1728877640 1 --2- 172.21.15.16:0/4265834014 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1724100500 0x7f172419ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:25.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:25.522+0000 7f1728877640 1 -- 172.21.15.16:0/4265834014 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1724106b80 msgr2=0x7f172419eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:25.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:25.522+0000 7f1728877640 1 --2- 172.21.15.16:0/4265834014 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1724106b80 0x7f172419eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:25.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:25.522+0000 7f1728877640 1 -- 172.21.15.16:0/4265834014 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f172419f5c0 con 0x7f17240ffb60 2024-09-12T02:00:25.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:25.522+0000 7f1723fff640 1 --2- 172.21.15.16:0/4265834014 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1724100500 0x7f172419ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:00:25.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:25.522+0000 7f1728877640 1 --2- 172.21.15.16:0/4265834014 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17240ffb60 0x7f172419a610 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f171800b4a0 tx=0x7f171800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:00:25.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:25.522+0000 7f1721ffb640 1 -- 172.21.15.16:0/4265834014 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1718014070 con 0x7f17240ffb60 2024-09-12T02:00:25.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:25.522+0000 7f172aadb640 1 -- 172.21.15.16:0/4265834014 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f172419f8b0 con 0x7f17240ffb60 2024-09-12T02:00:25.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:25.522+0000 7f1721ffb640 1 -- 172.21.15.16:0/4265834014 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1718010af0 con 0x7f17240ffb60 2024-09-12T02:00:25.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:25.522+0000 7f1721ffb640 1 -- 172.21.15.16:0/4265834014 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1718017d10 con 0x7f17240ffb60 2024-09-12T02:00:25.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:25.522+0000 7f172aadb640 1 -- 172.21.15.16:0/4265834014 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17241a7190 con 0x7f17240ffb60 2024-09-12T02:00:25.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:25.523+0000 7f172aadb640 1 -- 172.21.15.16:0/4265834014 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17240fffc0 con 0x7f17240ffb60 2024-09-12T02:00:25.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:25.524+0000 7f1721ffb640 1 -- 172.21.15.16:0/4265834014 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f1718017070 con 0x7f17240ffb60 2024-09-12T02:00:25.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:25.525+0000 7f1721ffb640 1 --2- 172.21.15.16:0/4265834014 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f17140a2c10 0x7f17140a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:25.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:25.525+0000 7f1723fff640 1 --2- 172.21.15.16:0/4265834014 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f17140a2c10 0x7f17140a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:25.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:25.525+0000 7f1721ffb640 1 -- 172.21.15.16:0/4265834014 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f17180ce500 con 0x7f17240ffb60 2024-09-12T02:00:25.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:25.526+0000 7f1723fff640 1 --2- 172.21.15.16:0/4265834014 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f17140a2c10 0x7f17140a50d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f172410ec30 tx=0x7f1710005d30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:00:25.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:25.529+0000 7f1721ffb640 1 -- 172.21.15.16:0/4265834014 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f171809b410 con 0x7f17240ffb60 2024-09-12T02:00:25.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:25.753+0000 7f172aadb640 1 -- 172.21.15.16:0/4265834014 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f172419b970 con 0x7f17240ffb60 2024-09-12T02:00:25.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:25.758+0000 7f1721ffb640 1 -- 172.21.15.16:0/4265834014 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+263200 (secure 0 0 0) 0x7f171809ad10 con 0x7f17240ffb60 2024-09-12T02:00:25.762 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T02:00:25.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:25.762+0000 7f172aadb640 1 -- 172.21.15.16:0/4265834014 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f17140a2c10 msgr2=0x7f17140a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:25.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:25.762+0000 7f172aadb640 1 --2- 172.21.15.16:0/4265834014 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f17140a2c10 0x7f17140a50d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f172410ec30 tx=0x7f1710005d30 comp rx=0 tx=0).stop 2024-09-12T02:00:25.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:25.762+0000 7f172aadb640 1 -- 172.21.15.16:0/4265834014 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17240ffb60 msgr2=0x7f172419a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:25.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:25.762+0000 7f172aadb640 1 --2- 172.21.15.16:0/4265834014 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17240ffb60 0x7f172419a610 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f171800b4a0 tx=0x7f171800b960 comp rx=0 tx=0).stop 2024-09-12T02:00:25.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:25.762+0000 7f172aadb640 1 -- 172.21.15.16:0/4265834014 shutdown_connections 2024-09-12T02:00:25.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:25.762+0000 7f172aadb640 1 --2- 172.21.15.16:0/4265834014 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f17140a2c10 0x7f17140a50d0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:25.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:25.762+0000 7f172aadb640 1 --2- 172.21.15.16:0/4265834014 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1724106b80 0x7f172419eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:25.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:25.762+0000 7f172aadb640 1 --2- 172.21.15.16:0/4265834014 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1724100500 0x7f172419ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:25.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:25.762+0000 7f172aadb640 1 --2- 172.21.15.16:0/4265834014 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17240ffb60 0x7f172419a610 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:25.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:25.762+0000 7f172aadb640 1 -- 172.21.15.16:0/4265834014 >> 172.21.15.16:0/4265834014 conn(0x7f17240fba90 msgr2=0x7f172410d1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:00:25.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:25.762+0000 7f172aadb640 1 -- 172.21.15.16:0/4265834014 shutdown_connections 2024-09-12T02:00:25.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:25.762+0000 7f172aadb640 1 -- 172.21.15.16:0/4265834014 wait complete. 2024-09-12T02:00:26.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:00:26 smithi184 ceph-mon[35593]: pgmap v290: 97 pgs: 97 active+clean; 638 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.6 KiB/s wr, 9 op/s; 9 B/s, 0 objects/s recovering 2024-09-12T02:00:26.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:00:26 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/4265834014' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-12T02:00:26.418 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:00:26 smithi033 ceph-mon[35332]: pgmap v290: 97 pgs: 97 active+clean; 638 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.6 KiB/s wr, 9 op/s; 9 B/s, 0 objects/s recovering 2024-09-12T02:00:26.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:00:26 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/4265834014' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-12T02:00:26.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:00:26 smithi016 ceph-mon[30947]: pgmap v290: 97 pgs: 97 active+clean; 638 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.6 KiB/s wr, 9 op/s; 9 B/s, 0 objects/s recovering 2024-09-12T02:00:26.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:00:26 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/4265834014' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-12T02:00:26.462 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":17,"flags":0,"active_gid":14150,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3887444820},{"type":"v1","addr":"172.21.15.16:6801","nonce":3887444820}]},"active_addr":"172.21.15.16:6801/3887444820","active_change":"2024-09-12T01:53:08.218622+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24110,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.16:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.16:0","nonce":890192749}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.16:0","nonce":2939463879}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.16:0","nonce":3820405214}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.16:0","nonce":3431006499}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.16:0","nonce":4258858815}]}]} 2024-09-12T02:00:26.467 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-09-12T02:00:26.467 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-09-12T02:00:26.468 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T02:00:26.725 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:00:27.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:27.242+0000 7f87ed180640 1 -- 172.21.15.16:0/2955876047 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87e81082a0 msgr2=0x7f87e810ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:27.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:27.242+0000 7f87ed180640 1 --2- 172.21.15.16:0/2955876047 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87e81082a0 0x7f87e810ec40 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f87e0009930 tx=0x7f87e002a420 comp rx=0 tx=0).stop 2024-09-12T02:00:27.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:27.243+0000 7f87ed180640 1 -- 172.21.15.16:0/2955876047 shutdown_connections 2024-09-12T02:00:27.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:27.243+0000 7f87ed180640 1 --2- 172.21.15.16:0/2955876047 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87e81082a0 0x7f87e810ec40 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:27.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:27.243+0000 7f87ed180640 1 --2- 172.21.15.16:0/2955876047 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f87e81036e0 0x7f87e8107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:27.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:27.243+0000 7f87ed180640 1 --2- 172.21.15.16:0/2955876047 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87e8102d30 0x7f87e8103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:27.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:27.243+0000 7f87ed180640 1 -- 172.21.15.16:0/2955876047 >> 172.21.15.16:0/2955876047 conn(0x7f87e80fc7d0 msgr2=0x7f87e80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:00:27.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:27.243+0000 7f87ed180640 1 -- 172.21.15.16:0/2955876047 shutdown_connections 2024-09-12T02:00:27.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:27.243+0000 7f87ed180640 1 -- 172.21.15.16:0/2955876047 wait complete. 2024-09-12T02:00:27.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:27.243+0000 7f87ed180640 1 Processor -- start 2024-09-12T02:00:27.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:27.244+0000 7f87ed180640 1 -- start start 2024-09-12T02:00:27.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:27.244+0000 7f87ed180640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87e8102d30 0x7f87e81984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:27.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:27.244+0000 7f87ed180640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87e81036e0 0x7f87e8198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:27.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:27.244+0000 7f87ed180640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f87e81082a0 0x7f87e819cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:27.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:27.244+0000 7f87ed180640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f87e81120a0 con 0x7f87e81036e0 2024-09-12T02:00:27.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:27.244+0000 7f87ed180640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f87e8111f20 con 0x7f87e8102d30 2024-09-12T02:00:27.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:27.244+0000 7f87ed180640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f87e8112220 con 0x7f87e81082a0 2024-09-12T02:00:27.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:27.244+0000 7f87e659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87e81036e0 0x7f87e8198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:27.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:27.244+0000 7f87e659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87e81036e0 0x7f87e8198a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43180/0 (socket says 172.21.15.16:43180) 2024-09-12T02:00:27.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:27.244+0000 7f87e659c640 1 -- 172.21.15.16:0/1230033373 learned_addr learned my addr 172.21.15.16:0/1230033373 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:00:27.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:27.244+0000 7f87e759e640 1 --2- 172.21.15.16:0/1230033373 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f87e81082a0 0x7f87e819cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:27.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:27.244+0000 7f87e6d9d640 1 --2- 172.21.15.16:0/1230033373 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87e8102d30 0x7f87e81984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:27.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:27.244+0000 7f87e659c640 1 -- 172.21.15.16:0/1230033373 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f87e81082a0 msgr2=0x7f87e819cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:27.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:27.244+0000 7f87e659c640 1 --2- 172.21.15.16:0/1230033373 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f87e81082a0 0x7f87e819cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:27.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:27.244+0000 7f87e659c640 1 -- 172.21.15.16:0/1230033373 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87e8102d30 msgr2=0x7f87e81984c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:27.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:27.244+0000 7f87e659c640 1 --2- 172.21.15.16:0/1230033373 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87e8102d30 0x7f87e81984c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:27.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:27.244+0000 7f87e659c640 1 -- 172.21.15.16:0/1230033373 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87e819d470 con 0x7f87e81036e0 2024-09-12T02:00:27.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:27.244+0000 7f87e6d9d640 1 --2- 172.21.15.16:0/1230033373 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87e8102d30 0x7f87e81984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:00:27.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:27.244+0000 7f87e759e640 1 --2- 172.21.15.16:0/1230033373 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f87e81082a0 0x7f87e819cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:00:27.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:27.244+0000 7f87e659c640 1 --2- 172.21.15.16:0/1230033373 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87e81036e0 0x7f87e8198a00 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f87dc00c8c0 tx=0x7f87dc00cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:00:27.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:27.245+0000 7f87cffff640 1 -- 172.21.15.16:0/1230033373 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87dc007bb0 con 0x7f87e81036e0 2024-09-12T02:00:27.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:27.245+0000 7f87ed180640 1 -- 172.21.15.16:0/1230033373 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87e819d760 con 0x7f87e81036e0 2024-09-12T02:00:27.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:27.245+0000 7f87cffff640 1 -- 172.21.15.16:0/1230033373 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87dc007d50 con 0x7f87e81036e0 2024-09-12T02:00:27.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:27.245+0000 7f87cffff640 1 -- 172.21.15.16:0/1230033373 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87dc018830 con 0x7f87e81036e0 2024-09-12T02:00:27.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:27.245+0000 7f87ed180640 1 -- 172.21.15.16:0/1230033373 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87e81a5040 con 0x7f87e81036e0 2024-09-12T02:00:27.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:27.246+0000 7f87ed180640 1 -- 172.21.15.16:0/1230033373 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87e8103110 con 0x7f87e81036e0 2024-09-12T02:00:27.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:27.247+0000 7f87cffff640 1 -- 172.21.15.16:0/1230033373 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f87dc018a10 con 0x7f87e81036e0 2024-09-12T02:00:27.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:27.248+0000 7f87cffff640 1 --2- 172.21.15.16:0/1230033373 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f87d00a2c10 0x7f87d00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:27.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:27.248+0000 7f87e6d9d640 1 --2- 172.21.15.16:0/1230033373 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f87d00a2c10 0x7f87d00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:27.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:27.248+0000 7f87cffff640 1 -- 172.21.15.16:0/1230033373 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f87dc0cf220 con 0x7f87e81036e0 2024-09-12T02:00:27.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:27.248+0000 7f87e6d9d640 1 --2- 172.21.15.16:0/1230033373 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f87d00a2c10 0x7f87d00a50d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f87d8005c40 tx=0x7f87d800a430 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:00:27.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:27.252+0000 7f87cffff640 1 -- 172.21.15.16:0/1230033373 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f87dc014030 con 0x7f87e81036e0 2024-09-12T02:00:27.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:27.430+0000 7f87ed180640 1 -- 172.21.15.16:0/1230033373 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f87e810a2b0 con 0x7f87e81036e0 2024-09-12T02:00:27.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:27.432+0000 7f87cffff640 1 -- 172.21.15.16:0/1230033373 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+22584 (secure 0 0 0) 0x7f87dc01f300 con 0x7f87e81036e0 2024-09-12T02:00:27.434 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T02:00:27.434 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":82,"fsid":"879f3b56-70a9-11ef-bcea-c7b262605968","created":"2024-09-12T01:52:12.841045+0000","modified":"2024-09-12T02:00:09.559511+0000","last_up_change":"2024-09-12T01:59:08.975443+0000","last_in_change":"2024-09-12T01:58:44.530220+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T01:55:26.727564+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T01:59:52.402097+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T01:59:54.463627+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3ea41eb6-e384-4ddc-810f-0f103c127849","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4000187867},{"type":"v1","addr":"172.21.15.16:6803","nonce":4000187867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4000187867},{"type":"v1","addr":"172.21.15.16:6805","nonce":4000187867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":4000187867},{"type":"v1","addr":"172.21.15.16:6809","nonce":4000187867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4000187867},{"type":"v1","addr":"172.21.15.16:6807","nonce":4000187867}]},"public_addr":"172.21.15.16:6803/4000187867","cluster_addr":"172.21.15.16:6805/4000187867","heartbeat_back_addr":"172.21.15.16:6809/4000187867","heartbeat_front_addr":"172.21.15.16:6807/4000187867","state":["exists","up"]},{"osd":1,"uuid":"07bf63e7-70fa-41b2-af53-12a5fec3f33c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":3862661148},{"type":"v1","addr":"172.21.15.33:6801","nonce":3862661148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":3862661148},{"type":"v1","addr":"172.21.15.33:6803","nonce":3862661148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":3862661148},{"type":"v1","addr":"172.21.15.33:6807","nonce":3862661148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":3862661148},{"type":"v1","addr":"172.21.15.33:6805","nonce":3862661148}]},"public_addr":"172.21.15.33:6801/3862661148","cluster_addr":"172.21.15.33:6803/3862661148","heartbeat_back_addr":"172.21.15.33:6807/3862661148","heartbeat_front_addr":"172.21.15.33:6805/3862661148","state":["exists","up"]},{"osd":2,"uuid":"22e40cac-3bc0-4cce-b47b-77499b767f7e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":3958753963},{"type":"v1","addr":"172.21.15.184:6801","nonce":3958753963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":3958753963},{"type":"v1","addr":"172.21.15.184:6803","nonce":3958753963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":3958753963},{"type":"v1","addr":"172.21.15.184:6807","nonce":3958753963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":3958753963},{"type":"v1","addr":"172.21.15.184:6805","nonce":3958753963}]},"public_addr":"172.21.15.184:6801/3958753963","cluster_addr":"172.21.15.184:6803/3958753963","heartbeat_back_addr":"172.21.15.184:6807/3958753963","heartbeat_front_addr":"172.21.15.184:6805/3958753963","state":["exists","up"]},{"osd":3,"uuid":"834a07f5-2c24-49e1-8e4a-12c96cfbdc65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3395182524},{"type":"v1","addr":"172.21.15.16:6811","nonce":3395182524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3395182524},{"type":"v1","addr":"172.21.15.16:6813","nonce":3395182524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3395182524},{"type":"v1","addr":"172.21.15.16:6817","nonce":3395182524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3395182524},{"type":"v1","addr":"172.21.15.16:6815","nonce":3395182524}]},"public_addr":"172.21.15.16:6811/3395182524","cluster_addr":"172.21.15.16:6813/3395182524","heartbeat_back_addr":"172.21.15.16:6817/3395182524","heartbeat_front_addr":"172.21.15.16:6815/3395182524","state":["exists","up"]},{"osd":4,"uuid":"3ca56e09-c194-4404-b9bd-2524d38367eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":1355184322},{"type":"v1","addr":"172.21.15.33:6809","nonce":1355184322}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1355184322},{"type":"v1","addr":"172.21.15.33:6811","nonce":1355184322}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1355184322},{"type":"v1","addr":"172.21.15.33:6815","nonce":1355184322}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1355184322},{"type":"v1","addr":"172.21.15.33:6813","nonce":1355184322}]},"public_addr":"172.21.15.33:6809/1355184322","cluster_addr":"172.21.15.33:6811/1355184322","heartbeat_back_addr":"172.21.15.33:6815/1355184322","heartbeat_front_addr":"172.21.15.33:6813/1355184322","state":["exists","up"]},{"osd":5,"uuid":"0b05ecdb-b987-461f-951c-6d76739bfb4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":1533397276},{"type":"v1","addr":"172.21.15.184:6809","nonce":1533397276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":1533397276},{"type":"v1","addr":"172.21.15.184:6811","nonce":1533397276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":1533397276},{"type":"v1","addr":"172.21.15.184:6815","nonce":1533397276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":1533397276},{"type":"v1","addr":"172.21.15.184:6813","nonce":1533397276}]},"public_addr":"172.21.15.184:6809/1533397276","cluster_addr":"172.21.15.184:6811/1533397276","heartbeat_back_addr":"172.21.15.184:6815/1533397276","heartbeat_front_addr":"172.21.15.184:6813/1533397276","state":["exists","up"]},{"osd":6,"uuid":"b3936e21-e197-49fa-8441-d909b258b612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":902466142},{"type":"v1","addr":"172.21.15.16:6819","nonce":902466142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":902466142},{"type":"v1","addr":"172.21.15.16:6821","nonce":902466142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":902466142},{"type":"v1","addr":"172.21.15.16:6825","nonce":902466142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":902466142},{"type":"v1","addr":"172.21.15.16:6823","nonce":902466142}]},"public_addr":"172.21.15.16:6819/902466142","cluster_addr":"172.21.15.16:6821/902466142","heartbeat_back_addr":"172.21.15.16:6825/902466142","heartbeat_front_addr":"172.21.15.16:6823/902466142","state":["exists","up"]},{"osd":7,"uuid":"4ddbe011-34c1-4b3f-a8eb-577422fb5426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":4186032630},{"type":"v1","addr":"172.21.15.33:6817","nonce":4186032630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":4186032630},{"type":"v1","addr":"172.21.15.33:6819","nonce":4186032630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":4186032630},{"type":"v1","addr":"172.21.15.33:6823","nonce":4186032630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":4186032630},{"type":"v1","addr":"172.21.15.33:6821","nonce":4186032630}]},"public_addr":"172.21.15.33:6817/4186032630","cluster_addr":"172.21.15.33:6819/4186032630","heartbeat_back_addr":"172.21.15.33:6823/4186032630","heartbeat_front_addr":"172.21.15.33:6821/4186032630","state":["exists","up"]},{"osd":8,"uuid":"dca5bfb8-ac6a-47ce-905c-be7fcaf0b5ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":781851781},{"type":"v1","addr":"172.21.15.184:6817","nonce":781851781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":781851781},{"type":"v1","addr":"172.21.15.184:6819","nonce":781851781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":781851781},{"type":"v1","addr":"172.21.15.184:6823","nonce":781851781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":781851781},{"type":"v1","addr":"172.21.15.184:6821","nonce":781851781}]},"public_addr":"172.21.15.184:6817/781851781","cluster_addr":"172.21.15.184:6819/781851781","heartbeat_back_addr":"172.21.15.184:6823/781851781","heartbeat_front_addr":"172.21.15.184:6821/781851781","state":["exists","up"]},{"osd":9,"uuid":"23a05cb5-01d5-40f3-b66a-15fcccedbd6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":1569578746},{"type":"v1","addr":"172.21.15.16:6827","nonce":1569578746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":1569578746},{"type":"v1","addr":"172.21.15.16:6829","nonce":1569578746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1569578746},{"type":"v1","addr":"172.21.15.16:6833","nonce":1569578746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":1569578746},{"type":"v1","addr":"172.21.15.16:6831","nonce":1569578746}]},"public_addr":"172.21.15.16:6827/1569578746","cluster_addr":"172.21.15.16:6829/1569578746","heartbeat_back_addr":"172.21.15.16:6833/1569578746","heartbeat_front_addr":"172.21.15.16:6831/1569578746","state":["exists","up"]},{"osd":10,"uuid":"e4bd6bce-3c69-44db-bc09-d264a0ad094a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":469977847},{"type":"v1","addr":"172.21.15.33:6825","nonce":469977847}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":469977847},{"type":"v1","addr":"172.21.15.33:6827","nonce":469977847}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":469977847},{"type":"v1","addr":"172.21.15.33:6831","nonce":469977847}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":469977847},{"type":"v1","addr":"172.21.15.33:6829","nonce":469977847}]},"public_addr":"172.21.15.33:6825/469977847","cluster_addr":"172.21.15.33:6827/469977847","heartbeat_back_addr":"172.21.15.33:6831/469977847","heartbeat_front_addr":"172.21.15.33:6829/469977847","state":["exists","up"]},{"osd":11,"uuid":"d60212f8-5a1d-45f7-9a1b-12294bb4c80d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":1619871346},{"type":"v1","addr":"172.21.15.184:6825","nonce":1619871346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":1619871346},{"type":"v1","addr":"172.21.15.184:6827","nonce":1619871346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":1619871346},{"type":"v1","addr":"172.21.15.184:6831","nonce":1619871346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":1619871346},{"type":"v1","addr":"172.21.15.184:6829","nonce":1619871346}]},"public_addr":"172.21.15.184:6825/1619871346","cluster_addr":"172.21.15.184:6827/1619871346","heartbeat_back_addr":"172.21.15.184:6831/1619871346","heartbeat_front_addr":"172.21.15.184:6829/1619871346","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:54:32.057979+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:54:57.501562+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:55:23.137225+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:55:48.646155+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:56:13.311907+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:56:38.214188+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:57:02.570921+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:57:27.506097+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:57:52.038529+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:58:17.423300+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:58:41.579992+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:59:06.245817+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:0/3532565698":"2024-09-13T01:53:08.218475+0000","172.21.15.16:0/1822157179":"2024-09-13T01:52:44.194152+0000","172.21.15.16:0/3919664061":"2024-09-13T01:52:44.194152+0000","172.21.15.16:0/2269394774":"2024-09-13T01:52:44.194152+0000","172.21.15.16:0/2595308634":"2024-09-13T01:53:08.218475+0000","172.21.15.16:6801/2754331153":"2024-09-13T01:52:44.194152+0000","172.21.15.16:6801/2615861334":"2024-09-13T01:53:08.218475+0000","172.21.15.16:6800/2615861334":"2024-09-13T01:53:08.218475+0000","172.21.15.16:0/4199114771":"2024-09-13T01:53:08.218475+0000","172.21.15.16:6800/2754331153":"2024-09-13T01:52:44.194152+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T02:00:27.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:27.434+0000 7f87ed180640 1 -- 172.21.15.16:0/1230033373 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f87d00a2c10 msgr2=0x7f87d00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:27.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:27.434+0000 7f87ed180640 1 --2- 172.21.15.16:0/1230033373 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f87d00a2c10 0x7f87d00a50d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f87d8005c40 tx=0x7f87d800a430 comp rx=0 tx=0).stop 2024-09-12T02:00:27.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:27.434+0000 7f87ed180640 1 -- 172.21.15.16:0/1230033373 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87e81036e0 msgr2=0x7f87e8198a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:27.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:27.434+0000 7f87ed180640 1 --2- 172.21.15.16:0/1230033373 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87e81036e0 0x7f87e8198a00 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f87dc00c8c0 tx=0x7f87dc00cd80 comp rx=0 tx=0).stop 2024-09-12T02:00:27.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:27.434+0000 7f87ed180640 1 -- 172.21.15.16:0/1230033373 shutdown_connections 2024-09-12T02:00:27.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:27.434+0000 7f87ed180640 1 --2- 172.21.15.16:0/1230033373 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f87d00a2c10 0x7f87d00a50d0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:27.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:27.434+0000 7f87ed180640 1 --2- 172.21.15.16:0/1230033373 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f87e81082a0 0x7f87e819cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:27.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:27.434+0000 7f87ed180640 1 --2- 172.21.15.16:0/1230033373 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87e81036e0 0x7f87e8198a00 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:27.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:27.434+0000 7f87ed180640 1 --2- 172.21.15.16:0/1230033373 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87e8102d30 0x7f87e81984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:27.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:27.434+0000 7f87ed180640 1 -- 172.21.15.16:0/1230033373 >> 172.21.15.16:0/1230033373 conn(0x7f87e80fc7d0 msgr2=0x7f87e8106380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:00:27.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:27.434+0000 7f87ed180640 1 -- 172.21.15.16:0/1230033373 shutdown_connections 2024-09-12T02:00:27.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:27.434+0000 7f87ed180640 1 -- 172.21.15.16:0/1230033373 wait complete. 2024-09-12T02:00:28.047 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-09-12T02:00:28.047 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T02:00:28.306 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:00:28.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:00:28 smithi184 ceph-mon[35593]: pgmap v291: 97 pgs: 97 active+clean; 638 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 3.2 KiB/s wr, 8 op/s 2024-09-12T02:00:28.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:00:28 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/1230033373' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T02:00:28.418 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:00:28 smithi033 ceph-mon[35332]: pgmap v291: 97 pgs: 97 active+clean; 638 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 3.2 KiB/s wr, 8 op/s 2024-09-12T02:00:28.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:00:28 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/1230033373' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T02:00:28.427 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:00:28 smithi016 ceph-mon[30947]: pgmap v291: 97 pgs: 97 active+clean; 638 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 3.2 KiB/s wr, 8 op/s 2024-09-12T02:00:28.427 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:00:28 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/1230033373' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T02:00:28.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:28.781+0000 7f6e7f926640 1 -- 172.21.15.16:0/1560579466 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e78100e30 msgr2=0x7f6e78104e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:28.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:28.781+0000 7f6e7f926640 1 --2- 172.21.15.16:0/1560579466 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e78100e30 0x7f6e78104e80 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f6e6c00aa10 tx=0x7f6e6c02b720 comp rx=0 tx=0).stop 2024-09-12T02:00:28.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:28.781+0000 7f6e7f926640 1 -- 172.21.15.16:0/1560579466 shutdown_connections 2024-09-12T02:00:28.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:28.781+0000 7f6e7f926640 1 --2- 172.21.15.16:0/1560579466 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6e781055b0 0x7f6e7810d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:28.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:28.781+0000 7f6e7f926640 1 --2- 172.21.15.16:0/1560579466 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e78100e30 0x7f6e78104e80 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:28.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:28.781+0000 7f6e7f926640 1 --2- 172.21.15.16:0/1560579466 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6e78100480 0x7f6e78100860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:28.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:28.781+0000 7f6e7f926640 1 -- 172.21.15.16:0/1560579466 >> 172.21.15.16:0/1560579466 conn(0x7f6e78077210 msgr2=0x7f6e780fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:00:28.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:28.781+0000 7f6e7f926640 1 -- 172.21.15.16:0/1560579466 shutdown_connections 2024-09-12T02:00:28.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:28.781+0000 7f6e7f926640 1 -- 172.21.15.16:0/1560579466 wait complete. 2024-09-12T02:00:28.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:28.782+0000 7f6e7f926640 1 Processor -- start 2024-09-12T02:00:28.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:28.782+0000 7f6e7f926640 1 -- start start 2024-09-12T02:00:28.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:28.782+0000 7f6e7f926640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e78100480 0x7f6e78198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:28.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:28.782+0000 7f6e7f926640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6e78100e30 0x7f6e78198bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:28.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:28.782+0000 7f6e7f926640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6e781055b0 0x7f6e7819cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:28.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:28.782+0000 7f6e7f926640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6e7810fe30 con 0x7f6e78100480 2024-09-12T02:00:28.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:28.782+0000 7f6e7f926640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6e7810fcb0 con 0x7f6e78100e30 2024-09-12T02:00:28.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:28.782+0000 7f6e7f926640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f6e7810ffb0 con 0x7f6e781055b0 2024-09-12T02:00:28.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:28.782+0000 7f6e7d6c2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e78100480 0x7f6e78198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:28.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:28.782+0000 7f6e7d6c2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e78100480 0x7f6e78198670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43200/0 (socket says 172.21.15.16:43200) 2024-09-12T02:00:28.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:28.782+0000 7f6e7d6c2640 1 -- 172.21.15.16:0/1994251346 learned_addr learned my addr 172.21.15.16:0/1994251346 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:00:28.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:28.783+0000 7f6e7cec1640 1 --2- 172.21.15.16:0/1994251346 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6e78100e30 0x7f6e78198bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:28.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:28.783+0000 7f6e7d6c2640 1 -- 172.21.15.16:0/1994251346 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6e781055b0 msgr2=0x7f6e7819cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:28.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:28.783+0000 7f6e7d6c2640 1 --2- 172.21.15.16:0/1994251346 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6e781055b0 0x7f6e7819cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:28.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:28.783+0000 7f6e7d6c2640 1 -- 172.21.15.16:0/1994251346 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6e78100e30 msgr2=0x7f6e78198bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:28.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:28.783+0000 7f6e7d6c2640 1 --2- 172.21.15.16:0/1994251346 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6e78100e30 0x7f6e78198bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:28.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:28.783+0000 7f6e7d6c2640 1 -- 172.21.15.16:0/1994251346 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e7819d620 con 0x7f6e78100480 2024-09-12T02:00:28.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:28.783+0000 7f6e7d6c2640 1 --2- 172.21.15.16:0/1994251346 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e78100480 0x7f6e78198670 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f6e7400b4a0 tx=0x7f6e7400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:00:28.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:28.783+0000 7f6e6a7fc640 1 -- 172.21.15.16:0/1994251346 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e74014070 con 0x7f6e78100480 2024-09-12T02:00:28.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:28.783+0000 7f6e6a7fc640 1 -- 172.21.15.16:0/1994251346 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e74010af0 con 0x7f6e78100480 2024-09-12T02:00:28.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:28.783+0000 7f6e7f926640 1 -- 172.21.15.16:0/1994251346 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e7819d910 con 0x7f6e78100480 2024-09-12T02:00:28.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:28.783+0000 7f6e6a7fc640 1 -- 172.21.15.16:0/1994251346 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e74017d10 con 0x7f6e78100480 2024-09-12T02:00:28.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:28.783+0000 7f6e7f926640 1 -- 172.21.15.16:0/1994251346 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e781a51f0 con 0x7f6e78100480 2024-09-12T02:00:28.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:28.784+0000 7f6e7f926640 1 -- 172.21.15.16:0/1994251346 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e781008e0 con 0x7f6e78100480 2024-09-12T02:00:28.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:28.785+0000 7f6e6a7fc640 1 -- 172.21.15.16:0/1994251346 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f6e74017070 con 0x7f6e78100480 2024-09-12T02:00:28.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:28.786+0000 7f6e6a7fc640 1 --2- 172.21.15.16:0/1994251346 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f6e640a2c90 0x7f6e640a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:28.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:28.786+0000 7f6e7cec1640 1 --2- 172.21.15.16:0/1994251346 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f6e640a2c90 0x7f6e640a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:28.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:28.786+0000 7f6e6a7fc640 1 -- 172.21.15.16:0/1994251346 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f6e740ce7d0 con 0x7f6e78100480 2024-09-12T02:00:28.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:28.787+0000 7f6e7cec1640 1 --2- 172.21.15.16:0/1994251346 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f6e640a2c90 0x7f6e640a5150 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f6e78103030 tx=0x7f6e6c0305a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:00:28.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:28.789+0000 7f6e6a7fc640 1 -- 172.21.15.16:0/1994251346 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6e7409b7e0 con 0x7f6e78100480 2024-09-12T02:00:28.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:28.966+0000 7f6e7f926640 1 -- 172.21.15.16:0/1994251346 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6e781999d0 con 0x7f6e78100480 2024-09-12T02:00:28.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:28.968+0000 7f6e6a7fc640 1 -- 172.21.15.16:0/1994251346 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+22584 (secure 0 0 0) 0x7f6e7409b0e0 con 0x7f6e78100480 2024-09-12T02:00:28.969 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T02:00:28.969 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":82,"fsid":"879f3b56-70a9-11ef-bcea-c7b262605968","created":"2024-09-12T01:52:12.841045+0000","modified":"2024-09-12T02:00:09.559511+0000","last_up_change":"2024-09-12T01:59:08.975443+0000","last_in_change":"2024-09-12T01:58:44.530220+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T01:55:26.727564+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T01:59:52.402097+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T01:59:54.463627+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3ea41eb6-e384-4ddc-810f-0f103c127849","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4000187867},{"type":"v1","addr":"172.21.15.16:6803","nonce":4000187867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4000187867},{"type":"v1","addr":"172.21.15.16:6805","nonce":4000187867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":4000187867},{"type":"v1","addr":"172.21.15.16:6809","nonce":4000187867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4000187867},{"type":"v1","addr":"172.21.15.16:6807","nonce":4000187867}]},"public_addr":"172.21.15.16:6803/4000187867","cluster_addr":"172.21.15.16:6805/4000187867","heartbeat_back_addr":"172.21.15.16:6809/4000187867","heartbeat_front_addr":"172.21.15.16:6807/4000187867","state":["exists","up"]},{"osd":1,"uuid":"07bf63e7-70fa-41b2-af53-12a5fec3f33c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":3862661148},{"type":"v1","addr":"172.21.15.33:6801","nonce":3862661148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":3862661148},{"type":"v1","addr":"172.21.15.33:6803","nonce":3862661148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":3862661148},{"type":"v1","addr":"172.21.15.33:6807","nonce":3862661148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":3862661148},{"type":"v1","addr":"172.21.15.33:6805","nonce":3862661148}]},"public_addr":"172.21.15.33:6801/3862661148","cluster_addr":"172.21.15.33:6803/3862661148","heartbeat_back_addr":"172.21.15.33:6807/3862661148","heartbeat_front_addr":"172.21.15.33:6805/3862661148","state":["exists","up"]},{"osd":2,"uuid":"22e40cac-3bc0-4cce-b47b-77499b767f7e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":3958753963},{"type":"v1","addr":"172.21.15.184:6801","nonce":3958753963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":3958753963},{"type":"v1","addr":"172.21.15.184:6803","nonce":3958753963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":3958753963},{"type":"v1","addr":"172.21.15.184:6807","nonce":3958753963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":3958753963},{"type":"v1","addr":"172.21.15.184:6805","nonce":3958753963}]},"public_addr":"172.21.15.184:6801/3958753963","cluster_addr":"172.21.15.184:6803/3958753963","heartbeat_back_addr":"172.21.15.184:6807/3958753963","heartbeat_front_addr":"172.21.15.184:6805/3958753963","state":["exists","up"]},{"osd":3,"uuid":"834a07f5-2c24-49e1-8e4a-12c96cfbdc65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3395182524},{"type":"v1","addr":"172.21.15.16:6811","nonce":3395182524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3395182524},{"type":"v1","addr":"172.21.15.16:6813","nonce":3395182524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3395182524},{"type":"v1","addr":"172.21.15.16:6817","nonce":3395182524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3395182524},{"type":"v1","addr":"172.21.15.16:6815","nonce":3395182524}]},"public_addr":"172.21.15.16:6811/3395182524","cluster_addr":"172.21.15.16:6813/3395182524","heartbeat_back_addr":"172.21.15.16:6817/3395182524","heartbeat_front_addr":"172.21.15.16:6815/3395182524","state":["exists","up"]},{"osd":4,"uuid":"3ca56e09-c194-4404-b9bd-2524d38367eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":1355184322},{"type":"v1","addr":"172.21.15.33:6809","nonce":1355184322}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1355184322},{"type":"v1","addr":"172.21.15.33:6811","nonce":1355184322}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1355184322},{"type":"v1","addr":"172.21.15.33:6815","nonce":1355184322}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1355184322},{"type":"v1","addr":"172.21.15.33:6813","nonce":1355184322}]},"public_addr":"172.21.15.33:6809/1355184322","cluster_addr":"172.21.15.33:6811/1355184322","heartbeat_back_addr":"172.21.15.33:6815/1355184322","heartbeat_front_addr":"172.21.15.33:6813/1355184322","state":["exists","up"]},{"osd":5,"uuid":"0b05ecdb-b987-461f-951c-6d76739bfb4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":1533397276},{"type":"v1","addr":"172.21.15.184:6809","nonce":1533397276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":1533397276},{"type":"v1","addr":"172.21.15.184:6811","nonce":1533397276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":1533397276},{"type":"v1","addr":"172.21.15.184:6815","nonce":1533397276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":1533397276},{"type":"v1","addr":"172.21.15.184:6813","nonce":1533397276}]},"public_addr":"172.21.15.184:6809/1533397276","cluster_addr":"172.21.15.184:6811/1533397276","heartbeat_back_addr":"172.21.15.184:6815/1533397276","heartbeat_front_addr":"172.21.15.184:6813/1533397276","state":["exists","up"]},{"osd":6,"uuid":"b3936e21-e197-49fa-8441-d909b258b612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":902466142},{"type":"v1","addr":"172.21.15.16:6819","nonce":902466142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":902466142},{"type":"v1","addr":"172.21.15.16:6821","nonce":902466142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":902466142},{"type":"v1","addr":"172.21.15.16:6825","nonce":902466142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":902466142},{"type":"v1","addr":"172.21.15.16:6823","nonce":902466142}]},"public_addr":"172.21.15.16:6819/902466142","cluster_addr":"172.21.15.16:6821/902466142","heartbeat_back_addr":"172.21.15.16:6825/902466142","heartbeat_front_addr":"172.21.15.16:6823/902466142","state":["exists","up"]},{"osd":7,"uuid":"4ddbe011-34c1-4b3f-a8eb-577422fb5426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":4186032630},{"type":"v1","addr":"172.21.15.33:6817","nonce":4186032630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":4186032630},{"type":"v1","addr":"172.21.15.33:6819","nonce":4186032630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":4186032630},{"type":"v1","addr":"172.21.15.33:6823","nonce":4186032630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":4186032630},{"type":"v1","addr":"172.21.15.33:6821","nonce":4186032630}]},"public_addr":"172.21.15.33:6817/4186032630","cluster_addr":"172.21.15.33:6819/4186032630","heartbeat_back_addr":"172.21.15.33:6823/4186032630","heartbeat_front_addr":"172.21.15.33:6821/4186032630","state":["exists","up"]},{"osd":8,"uuid":"dca5bfb8-ac6a-47ce-905c-be7fcaf0b5ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":781851781},{"type":"v1","addr":"172.21.15.184:6817","nonce":781851781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":781851781},{"type":"v1","addr":"172.21.15.184:6819","nonce":781851781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":781851781},{"type":"v1","addr":"172.21.15.184:6823","nonce":781851781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":781851781},{"type":"v1","addr":"172.21.15.184:6821","nonce":781851781}]},"public_addr":"172.21.15.184:6817/781851781","cluster_addr":"172.21.15.184:6819/781851781","heartbeat_back_addr":"172.21.15.184:6823/781851781","heartbeat_front_addr":"172.21.15.184:6821/781851781","state":["exists","up"]},{"osd":9,"uuid":"23a05cb5-01d5-40f3-b66a-15fcccedbd6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":1569578746},{"type":"v1","addr":"172.21.15.16:6827","nonce":1569578746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":1569578746},{"type":"v1","addr":"172.21.15.16:6829","nonce":1569578746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1569578746},{"type":"v1","addr":"172.21.15.16:6833","nonce":1569578746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":1569578746},{"type":"v1","addr":"172.21.15.16:6831","nonce":1569578746}]},"public_addr":"172.21.15.16:6827/1569578746","cluster_addr":"172.21.15.16:6829/1569578746","heartbeat_back_addr":"172.21.15.16:6833/1569578746","heartbeat_front_addr":"172.21.15.16:6831/1569578746","state":["exists","up"]},{"osd":10,"uuid":"e4bd6bce-3c69-44db-bc09-d264a0ad094a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":469977847},{"type":"v1","addr":"172.21.15.33:6825","nonce":469977847}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":469977847},{"type":"v1","addr":"172.21.15.33:6827","nonce":469977847}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":469977847},{"type":"v1","addr":"172.21.15.33:6831","nonce":469977847}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":469977847},{"type":"v1","addr":"172.21.15.33:6829","nonce":469977847}]},"public_addr":"172.21.15.33:6825/469977847","cluster_addr":"172.21.15.33:6827/469977847","heartbeat_back_addr":"172.21.15.33:6831/469977847","heartbeat_front_addr":"172.21.15.33:6829/469977847","state":["exists","up"]},{"osd":11,"uuid":"d60212f8-5a1d-45f7-9a1b-12294bb4c80d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":1619871346},{"type":"v1","addr":"172.21.15.184:6825","nonce":1619871346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":1619871346},{"type":"v1","addr":"172.21.15.184:6827","nonce":1619871346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":1619871346},{"type":"v1","addr":"172.21.15.184:6831","nonce":1619871346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":1619871346},{"type":"v1","addr":"172.21.15.184:6829","nonce":1619871346}]},"public_addr":"172.21.15.184:6825/1619871346","cluster_addr":"172.21.15.184:6827/1619871346","heartbeat_back_addr":"172.21.15.184:6831/1619871346","heartbeat_front_addr":"172.21.15.184:6829/1619871346","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:54:32.057979+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:54:57.501562+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:55:23.137225+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:55:48.646155+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:56:13.311907+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:56:38.214188+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:57:02.570921+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:57:27.506097+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:57:52.038529+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:58:17.423300+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:58:41.579992+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:59:06.245817+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:0/3532565698":"2024-09-13T01:53:08.218475+0000","172.21.15.16:0/1822157179":"2024-09-13T01:52:44.194152+0000","172.21.15.16:0/3919664061":"2024-09-13T01:52:44.194152+0000","172.21.15.16:0/2269394774":"2024-09-13T01:52:44.194152+0000","172.21.15.16:0/2595308634":"2024-09-13T01:53:08.218475+0000","172.21.15.16:6801/2754331153":"2024-09-13T01:52:44.194152+0000","172.21.15.16:6801/2615861334":"2024-09-13T01:53:08.218475+0000","172.21.15.16:6800/2615861334":"2024-09-13T01:53:08.218475+0000","172.21.15.16:0/4199114771":"2024-09-13T01:53:08.218475+0000","172.21.15.16:6800/2754331153":"2024-09-13T01:52:44.194152+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T02:00:28.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:28.970+0000 7f6e7f926640 1 -- 172.21.15.16:0/1994251346 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f6e640a2c90 msgr2=0x7f6e640a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:28.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:28.970+0000 7f6e7f926640 1 --2- 172.21.15.16:0/1994251346 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f6e640a2c90 0x7f6e640a5150 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f6e78103030 tx=0x7f6e6c0305a0 comp rx=0 tx=0).stop 2024-09-12T02:00:28.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:28.970+0000 7f6e7f926640 1 -- 172.21.15.16:0/1994251346 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e78100480 msgr2=0x7f6e78198670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:28.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:28.970+0000 7f6e7f926640 1 --2- 172.21.15.16:0/1994251346 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e78100480 0x7f6e78198670 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f6e7400b4a0 tx=0x7f6e7400b960 comp rx=0 tx=0).stop 2024-09-12T02:00:28.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:28.970+0000 7f6e7f926640 1 -- 172.21.15.16:0/1994251346 shutdown_connections 2024-09-12T02:00:28.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:28.970+0000 7f6e7f926640 1 --2- 172.21.15.16:0/1994251346 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f6e640a2c90 0x7f6e640a5150 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:28.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:28.970+0000 7f6e7f926640 1 --2- 172.21.15.16:0/1994251346 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6e781055b0 0x7f6e7819cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:28.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:28.970+0000 7f6e7f926640 1 --2- 172.21.15.16:0/1994251346 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6e78100e30 0x7f6e78198bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:28.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:28.970+0000 7f6e7f926640 1 --2- 172.21.15.16:0/1994251346 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e78100480 0x7f6e78198670 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:28.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:28.970+0000 7f6e7f926640 1 -- 172.21.15.16:0/1994251346 >> 172.21.15.16:0/1994251346 conn(0x7f6e78077210 msgr2=0x7f6e780ff0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:00:28.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:28.970+0000 7f6e7f926640 1 -- 172.21.15.16:0/1994251346 shutdown_connections 2024-09-12T02:00:28.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:28.970+0000 7f6e7f926640 1 -- 172.21.15.16:0/1994251346 wait complete. 2024-09-12T02:00:29.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:00:29 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/1994251346' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T02:00:29.421 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:00:29 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/1994251346' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T02:00:29.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:00:29 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/1994251346' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T02:00:29.645 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-09-12T02:00:29.645 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-09-12T02:00:29.646 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-09-12T02:00:29.646 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-09-12T02:00:29.647 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-09-12T02:00:29.647 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-09-12T02:00:29.647 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-09-12T02:00:29.648 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-09-12T02:00:29.649 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-09-12T02:00:29.649 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-09-12T02:00:29.650 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-09-12T02:00:29.651 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-09-12T02:00:30.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:00:30 smithi184 ceph-mon[35593]: pgmap v292: 97 pgs: 97 active+clean; 638 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.0 KiB/s wr, 6 op/s 2024-09-12T02:00:30.395 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:00:30.407 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:00:30.418 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:00:30 smithi033 ceph-mon[35332]: pgmap v292: 97 pgs: 97 active+clean; 638 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.0 KiB/s wr, 6 op/s 2024-09-12T02:00:30.429 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:00:30.433 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:00:30.433 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:00:30.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:00:30 smithi016 ceph-mon[30947]: pgmap v292: 97 pgs: 97 active+clean; 638 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.0 KiB/s wr, 6 op/s 2024-09-12T02:00:30.436 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:00:30.440 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:00:30.449 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:00:30.453 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:00:30.454 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:00:30.461 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:00:30.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:30.951+0000 7f996a83b640 1 -- 172.21.15.16:0/564677146 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9964101200 msgr2=0x7f9964101660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:30.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:30.951+0000 7f996a83b640 1 --2- 172.21.15.16:0/564677146 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9964101200 0x7f9964101660 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f9958009900 tx=0x7f995802a410 comp rx=0 tx=0).stop 2024-09-12T02:00:30.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:30.951+0000 7f996a83b640 1 -- 172.21.15.16:0/564677146 shutdown_connections 2024-09-12T02:00:30.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:30.951+0000 7f996a83b640 1 --2- 172.21.15.16:0/564677146 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9964101ba0 0x7f996410d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:30.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:30.951+0000 7f996a83b640 1 --2- 172.21.15.16:0/564677146 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9964101200 0x7f9964101660 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:30.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:30.951+0000 7f996a83b640 1 --2- 172.21.15.16:0/564677146 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9964105ad0 0x7f9964105eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:30.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:30.951+0000 7f996a83b640 1 -- 172.21.15.16:0/564677146 >> 172.21.15.16:0/564677146 conn(0x7f9964077210 msgr2=0x7f99640fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:00:30.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:30.951+0000 7f996a83b640 1 -- 172.21.15.16:0/564677146 shutdown_connections 2024-09-12T02:00:30.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:30.951+0000 7f996a83b640 1 -- 172.21.15.16:0/564677146 wait complete. 2024-09-12T02:00:30.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:30.952+0000 7f996a83b640 1 Processor -- start 2024-09-12T02:00:30.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:30.952+0000 7f996a83b640 1 -- start start 2024-09-12T02:00:30.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:30.952+0000 7f996a83b640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9964101200 0x7f99641991e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:30.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:30.952+0000 7f996a83b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9964101ba0 0x7f9964199720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:30.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:30.952+0000 7f996a83b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9964105ad0 0x7f99641a4260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:30.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:30.952+0000 7f996a83b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9964110ef0 con 0x7f9964101ba0 2024-09-12T02:00:30.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:30.952+0000 7f996a83b640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9964110d70 con 0x7f9964105ad0 2024-09-12T02:00:30.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:30.952+0000 7f996a83b640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f9964111070 con 0x7f9964101200 2024-09-12T02:00:30.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:30.952+0000 7f99637fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9964101ba0 0x7f9964199720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:30.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:30.952+0000 7f99637fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9964101ba0 0x7f9964199720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43218/0 (socket says 172.21.15.16:43218) 2024-09-12T02:00:30.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:30.952+0000 7f99637fe640 1 -- 172.21.15.16:0/132017003 learned_addr learned my addr 172.21.15.16:0/132017003 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:00:30.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:30.952+0000 7f9963fff640 1 --2- 172.21.15.16:0/132017003 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9964101200 0x7f99641991e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:30.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:30.952+0000 7f9968dd8640 1 --2- 172.21.15.16:0/132017003 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9964105ad0 0x7f99641a4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:30.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:30.952+0000 7f99637fe640 1 -- 172.21.15.16:0/132017003 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9964101200 msgr2=0x7f99641991e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:30.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:30.952+0000 7f99637fe640 1 --2- 172.21.15.16:0/132017003 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9964101200 0x7f99641991e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:30.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:30.952+0000 7f99637fe640 1 -- 172.21.15.16:0/132017003 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9964105ad0 msgr2=0x7f99641a4260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:30.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:30.952+0000 7f99637fe640 1 --2- 172.21.15.16:0/132017003 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9964105ad0 0x7f99641a4260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:30.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:30.952+0000 7f99637fe640 1 -- 172.21.15.16:0/132017003 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99641a47a0 con 0x7f9964101ba0 2024-09-12T02:00:30.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:30.952+0000 7f9968dd8640 1 --2- 172.21.15.16:0/132017003 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9964105ad0 0x7f99641a4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:00:30.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:30.953+0000 7f99637fe640 1 --2- 172.21.15.16:0/132017003 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9964101ba0 0x7f9964199720 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f9958006010 tx=0x7f9958003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:00:30.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:30.953+0000 7f99617fa640 1 -- 172.21.15.16:0/132017003 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f995803f070 con 0x7f9964101ba0 2024-09-12T02:00:30.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:30.953+0000 7f99617fa640 1 -- 172.21.15.16:0/132017003 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9958031e30 con 0x7f9964101ba0 2024-09-12T02:00:30.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:30.953+0000 7f996a83b640 1 -- 172.21.15.16:0/132017003 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99641a4970 con 0x7f9964101ba0 2024-09-12T02:00:30.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:30.953+0000 7f99617fa640 1 -- 172.21.15.16:0/132017003 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9958035030 con 0x7f9964101ba0 2024-09-12T02:00:30.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:30.953+0000 7f996a83b640 1 -- 172.21.15.16:0/132017003 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99641a4e20 con 0x7f9964101ba0 2024-09-12T02:00:30.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:30.954+0000 7f996a83b640 1 -- 172.21.15.16:0/132017003 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f99641043a0 con 0x7f9964101ba0 2024-09-12T02:00:30.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:30.955+0000 7f99617fa640 1 -- 172.21.15.16:0/132017003 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f9958042070 con 0x7f9964101ba0 2024-09-12T02:00:30.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:30.956+0000 7f99617fa640 1 --2- 172.21.15.16:0/132017003 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f994c0a2c90 0x7f994c0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:30.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:30.956+0000 7f99617fa640 1 -- 172.21.15.16:0/132017003 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f99580efa50 con 0x7f9964101ba0 2024-09-12T02:00:30.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:30.956+0000 7f9963fff640 1 --2- 172.21.15.16:0/132017003 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f994c0a2c90 0x7f994c0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:30.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:30.956+0000 7f99617fa640 1 --2- 172.21.15.16:0/132017003 >> [v2:172.21.15.184:6816/781851781,v1:172.21.15.184:6817/781851781] conn(0x7f994c0ac740 0x7f994c0aebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:30.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:30.956+0000 7f99617fa640 1 -- 172.21.15.16:0/132017003 --> [v2:172.21.15.184:6816/781851781,v1:172.21.15.184:6817/781851781] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f995802fec0 con 0x7f994c0ac740 2024-09-12T02:00:30.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:30.956+0000 7f99617fa640 1 -- 172.21.15.16:0/132017003 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f99580f42c0 con 0x7f9964101ba0 2024-09-12T02:00:30.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:30.956+0000 7f9968dd8640 1 --2- 172.21.15.16:0/132017003 >> [v2:172.21.15.184:6816/781851781,v1:172.21.15.184:6817/781851781] conn(0x7f994c0ac740 0x7f994c0aebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:30.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:30.956+0000 7f9963fff640 1 --2- 172.21.15.16:0/132017003 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f994c0a2c90 0x7f994c0a5150 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f9954005d70 tx=0x7f9954005d00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:00:30.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:30.957+0000 7f9968dd8640 1 --2- 172.21.15.16:0/132017003 >> [v2:172.21.15.184:6816/781851781,v1:172.21.15.184:6817/781851781] conn(0x7f994c0ac740 0x7f994c0aebe0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:00:30.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:30.957+0000 7f99617fa640 1 -- 172.21.15.16:0/132017003 <== osd.8 v2:172.21.15.184:6816/781851781 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f995802fec0 con 0x7f994c0ac740 2024-09-12T02:00:30.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:30.972+0000 7f996a83b640 1 -- 172.21.15.16:0/132017003 --> [v2:172.21.15.184:6816/781851781,v1:172.21.15.184:6817/781851781] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9964102210 con 0x7f994c0ac740 2024-09-12T02:00:30.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:30.972+0000 7f99617fa640 1 -- 172.21.15.16:0/132017003 <== osd.8 v2:172.21.15.184:6816/781851781 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f9964102210 con 0x7f994c0ac740 2024-09-12T02:00:30.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:30.973+0000 7f996a83b640 1 -- 172.21.15.16:0/132017003 >> [v2:172.21.15.184:6816/781851781,v1:172.21.15.184:6817/781851781] conn(0x7f994c0ac740 msgr2=0x7f994c0aebe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:30.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:30.973+0000 7f996a83b640 1 --2- 172.21.15.16:0/132017003 >> [v2:172.21.15.184:6816/781851781,v1:172.21.15.184:6817/781851781] conn(0x7f994c0ac740 0x7f994c0aebe0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:30.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:30.973+0000 7f996a83b640 1 -- 172.21.15.16:0/132017003 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f994c0a2c90 msgr2=0x7f994c0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:30.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:30.973+0000 7f996a83b640 1 --2- 172.21.15.16:0/132017003 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f994c0a2c90 0x7f994c0a5150 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f9954005d70 tx=0x7f9954005d00 comp rx=0 tx=0).stop 2024-09-12T02:00:30.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:30.973+0000 7f996a83b640 1 -- 172.21.15.16:0/132017003 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9964101ba0 msgr2=0x7f9964199720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:30.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:30.973+0000 7f996a83b640 1 --2- 172.21.15.16:0/132017003 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9964101ba0 0x7f9964199720 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f9958006010 tx=0x7f9958003c80 comp rx=0 tx=0).stop 2024-09-12T02:00:30.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:30.973+0000 7f9968dd8640 1 -- 172.21.15.16:0/132017003 reap_dead start 2024-09-12T02:00:30.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:30.973+0000 7f996a83b640 1 -- 172.21.15.16:0/132017003 shutdown_connections 2024-09-12T02:00:30.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:30.973+0000 7f996a83b640 1 -- 172.21.15.16:0/132017003 >> 172.21.15.16:0/132017003 conn(0x7f9964077210 msgr2=0x7f9964103940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:00:30.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:30.973+0000 7f996a83b640 1 -- 172.21.15.16:0/132017003 shutdown_connections 2024-09-12T02:00:30.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:30.973+0000 7f996a83b640 1 -- 172.21.15.16:0/132017003 wait complete. 2024-09-12T02:00:30.976 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:00:31.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:31.794+0000 7f530ce4f640 1 -- 172.21.15.16:0/144633884 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5308102de0 msgr2=0x7f53081031c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:31.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:31.794+0000 7f530ce4f640 1 --2- 172.21.15.16:0/144633884 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5308102de0 0x7f53081031c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f52f8009900 tx=0x7f52f802a480 comp rx=0 tx=0).stop 2024-09-12T02:00:31.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:31.794+0000 7f530ce4f640 1 -- 172.21.15.16:0/144633884 shutdown_connections 2024-09-12T02:00:31.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:31.794+0000 7f530ce4f640 1 --2- 172.21.15.16:0/144633884 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f530810c180 0x7f530810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:31.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:31.794+0000 7f530ce4f640 1 --2- 172.21.15.16:0/144633884 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5308103700 0x7f530810bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:31.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:31.794+0000 7f530ce4f640 1 --2- 172.21.15.16:0/144633884 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5308102de0 0x7f53081031c0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:31.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:31.794+0000 7f530ce4f640 1 -- 172.21.15.16:0/144633884 >> 172.21.15.16:0/144633884 conn(0x7f53080fc7a0 msgr2=0x7f53080febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:00:31.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:31.795+0000 7f530ce4f640 1 -- 172.21.15.16:0/144633884 shutdown_connections 2024-09-12T02:00:31.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:31.795+0000 7f530ce4f640 1 -- 172.21.15.16:0/144633884 wait complete. 2024-09-12T02:00:31.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:31.795+0000 7f530ce4f640 1 Processor -- start 2024-09-12T02:00:31.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:31.795+0000 7f530ce4f640 1 -- start start 2024-09-12T02:00:31.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:31.796+0000 7f530ce4f640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5308102de0 0x7f5308198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:31.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:31.796+0000 7f530ce4f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5308103700 0x7f5308198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:31.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:31.796+0000 7f530ce4f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f530810c180 0x7f530819ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:31.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:31.796+0000 7f530ce4f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5308111fb0 con 0x7f530810c180 2024-09-12T02:00:31.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:31.796+0000 7f530ce4f640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5308111e30 con 0x7f5308103700 2024-09-12T02:00:31.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:31.796+0000 7f530ce4f640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5308112130 con 0x7f5308102de0 2024-09-12T02:00:31.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:31.796+0000 7f5306d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f530810c180 0x7f530819ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:31.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:31.796+0000 7f5306d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f530810c180 0x7f530819ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38566/0 (socket says 172.21.15.16:38566) 2024-09-12T02:00:31.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:31.796+0000 7f5306d9d640 1 -- 172.21.15.16:0/3741158014 learned_addr learned my addr 172.21.15.16:0/3741158014 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:00:31.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:31.796+0000 7f5305d9b640 1 --2- 172.21.15.16:0/3741158014 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5308103700 0x7f5308198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:31.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:31.796+0000 7f530659c640 1 --2- 172.21.15.16:0/3741158014 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5308102de0 0x7f5308198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:31.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:31.796+0000 7f5306d9d640 1 -- 172.21.15.16:0/3741158014 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5308102de0 msgr2=0x7f5308198590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:31.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:31.796+0000 7f5306d9d640 1 --2- 172.21.15.16:0/3741158014 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5308102de0 0x7f5308198590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:31.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:31.796+0000 7f5306d9d640 1 -- 172.21.15.16:0/3741158014 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5308103700 msgr2=0x7f5308198ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:31.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:31.796+0000 7f5306d9d640 1 --2- 172.21.15.16:0/3741158014 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5308103700 0x7f5308198ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:31.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:31.796+0000 7f5306d9d640 1 -- 172.21.15.16:0/3741158014 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f530819d540 con 0x7f530810c180 2024-09-12T02:00:31.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:31.796+0000 7f5306d9d640 1 --2- 172.21.15.16:0/3741158014 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f530810c180 0x7f530819ce60 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f530000eaa0 tx=0x7f530000eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:00:31.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:31.796+0000 7f52f77fe640 1 -- 172.21.15.16:0/3741158014 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5300004d20 con 0x7f530810c180 2024-09-12T02:00:31.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:31.796+0000 7f530ce4f640 1 -- 172.21.15.16:0/3741158014 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f530819d830 con 0x7f530810c180 2024-09-12T02:00:31.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:31.796+0000 7f52f77fe640 1 -- 172.21.15.16:0/3741158014 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5300005020 con 0x7f530810c180 2024-09-12T02:00:31.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:31.797+0000 7f52f77fe640 1 -- 172.21.15.16:0/3741158014 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53000116d0 con 0x7f530810c180 2024-09-12T02:00:31.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:31.797+0000 7f530ce4f640 1 -- 172.21.15.16:0/3741158014 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53081a5110 con 0x7f530810c180 2024-09-12T02:00:31.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:31.797+0000 7f530ce4f640 1 -- 172.21.15.16:0/3741158014 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5308062900 con 0x7f530810c180 2024-09-12T02:00:31.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:31.799+0000 7f52f77fe640 1 -- 172.21.15.16:0/3741158014 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f53000051d0 con 0x7f530810c180 2024-09-12T02:00:31.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:31.799+0000 7f52f77fe640 1 --2- 172.21.15.16:0/3741158014 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f52f00a2c10 0x7f52f00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:31.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:31.799+0000 7f530659c640 1 --2- 172.21.15.16:0/3741158014 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f52f00a2c10 0x7f52f00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:31.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:31.799+0000 7f52f77fe640 1 -- 172.21.15.16:0/3741158014 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f53000cf750 con 0x7f530810c180 2024-09-12T02:00:31.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:31.800+0000 7f52f77fe640 1 --2- 172.21.15.16:0/3741158014 >> [v2:172.21.15.33:6800/3862661148,v1:172.21.15.33:6801/3862661148] conn(0x7f52f00ac6c0 0x7f52f00aeb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:31.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:31.800+0000 7f52f77fe640 1 -- 172.21.15.16:0/3741158014 --> [v2:172.21.15.33:6800/3862661148,v1:172.21.15.33:6801/3862661148] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5300014ec0 con 0x7f52f00ac6c0 2024-09-12T02:00:31.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:31.800+0000 7f52f77fe640 1 -- 172.21.15.16:0/3741158014 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f53000cfc30 con 0x7f530810c180 2024-09-12T02:00:31.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:31.800+0000 7f530659c640 1 --2- 172.21.15.16:0/3741158014 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f52f00a2c10 0x7f52f00a50d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f52f8005ab0 tx=0x7f52f8005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:00:31.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:31.800+0000 7f5305d9b640 1 --2- 172.21.15.16:0/3741158014 >> [v2:172.21.15.33:6800/3862661148,v1:172.21.15.33:6801/3862661148] conn(0x7f52f00ac6c0 0x7f52f00aeb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:31.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:31.801+0000 7f5305d9b640 1 --2- 172.21.15.16:0/3741158014 >> [v2:172.21.15.33:6800/3862661148,v1:172.21.15.33:6801/3862661148] conn(0x7f52f00ac6c0 0x7f52f00aeb60 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:00:31.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:31.801+0000 7f52f77fe640 1 -- 172.21.15.16:0/3741158014 <== osd.1 v2:172.21.15.33:6800/3862661148 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5300014ec0 con 0x7f52f00ac6c0 2024-09-12T02:00:31.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:31.815+0000 7f530ce4f640 1 -- 172.21.15.16:0/3741158014 --> [v2:172.21.15.33:6800/3862661148,v1:172.21.15.33:6801/3862661148] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5308062d00 con 0x7f52f00ac6c0 2024-09-12T02:00:31.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:31.815+0000 7f52f77fe640 1 -- 172.21.15.16:0/3741158014 <== osd.1 v2:172.21.15.33:6800/3862661148 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f5308062d00 con 0x7f52f00ac6c0 2024-09-12T02:00:31.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:31.816+0000 7f530ce4f640 1 -- 172.21.15.16:0/3741158014 >> [v2:172.21.15.33:6800/3862661148,v1:172.21.15.33:6801/3862661148] conn(0x7f52f00ac6c0 msgr2=0x7f52f00aeb60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:31.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:31.816+0000 7f530ce4f640 1 --2- 172.21.15.16:0/3741158014 >> [v2:172.21.15.33:6800/3862661148,v1:172.21.15.33:6801/3862661148] conn(0x7f52f00ac6c0 0x7f52f00aeb60 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:31.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:31.816+0000 7f530ce4f640 1 -- 172.21.15.16:0/3741158014 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f52f00a2c10 msgr2=0x7f52f00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:31.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:31.816+0000 7f530ce4f640 1 --2- 172.21.15.16:0/3741158014 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f52f00a2c10 0x7f52f00a50d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f52f8005ab0 tx=0x7f52f8005a20 comp rx=0 tx=0).stop 2024-09-12T02:00:31.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:31.816+0000 7f530ce4f640 1 -- 172.21.15.16:0/3741158014 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f530810c180 msgr2=0x7f530819ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:31.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:31.816+0000 7f530ce4f640 1 --2- 172.21.15.16:0/3741158014 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f530810c180 0x7f530819ce60 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f530000eaa0 tx=0x7f530000eea0 comp rx=0 tx=0).stop 2024-09-12T02:00:31.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:31.816+0000 7f5306d9d640 1 -- 172.21.15.16:0/3741158014 reap_dead start 2024-09-12T02:00:31.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:31.816+0000 7f530ce4f640 1 -- 172.21.15.16:0/3741158014 shutdown_connections 2024-09-12T02:00:31.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:31.816+0000 7f530ce4f640 1 -- 172.21.15.16:0/3741158014 >> 172.21.15.16:0/3741158014 conn(0x7f53080fc7a0 msgr2=0x7f53080fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:00:31.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:31.816+0000 7f530ce4f640 1 -- 172.21.15.16:0/3741158014 shutdown_connections 2024-09-12T02:00:31.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:31.816+0000 7f530ce4f640 1 -- 172.21.15.16:0/3741158014 wait complete. 2024-09-12T02:00:32.066 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:00:32 smithi016 ceph-mon[30947]: pgmap v293: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 3.2 KiB/s wr, 9 op/s 2024-09-12T02:00:32.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:00:32 smithi184 ceph-mon[35593]: pgmap v293: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 3.2 KiB/s wr, 9 op/s 2024-09-12T02:00:32.418 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:00:32 smithi033 ceph-mon[35332]: pgmap v293: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 3.2 KiB/s wr, 9 op/s 2024-09-12T02:00:32.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:32.678+0000 7f3346a5e640 1 -- 172.21.15.16:0/2662897803 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3340106f10 msgr2=0x7f334010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:32.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:32.678+0000 7f3346a5e640 1 --2- 172.21.15.16:0/2662897803 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3340106f10 0x7f334010ea90 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f3338009ca0 tx=0x7f333802a8c0 comp rx=0 tx=0).stop 2024-09-12T02:00:32.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:32.678+0000 7f3346a5e640 1 -- 172.21.15.16:0/2662897803 shutdown_connections 2024-09-12T02:00:32.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:32.678+0000 7f3346a5e640 1 --2- 172.21.15.16:0/2662897803 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3340106f10 0x7f334010ea90 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:32.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:32.678+0000 7f3346a5e640 1 --2- 172.21.15.16:0/2662897803 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3340102790 0x7f33401067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:32.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:32.678+0000 7f3346a5e640 1 --2- 172.21.15.16:0/2662897803 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3340101de0 0x7f33401021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:32.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:32.678+0000 7f3346a5e640 1 -- 172.21.15.16:0/2662897803 >> 172.21.15.16:0/2662897803 conn(0x7f33400fd880 msgr2=0x7f33400ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:00:32.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:32.678+0000 7f3346a5e640 1 -- 172.21.15.16:0/2662897803 shutdown_connections 2024-09-12T02:00:32.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:32.678+0000 7f3346a5e640 1 -- 172.21.15.16:0/2662897803 wait complete. 2024-09-12T02:00:32.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:32.679+0000 7f3346a5e640 1 Processor -- start 2024-09-12T02:00:32.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:32.679+0000 7f3346a5e640 1 -- start start 2024-09-12T02:00:32.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:32.679+0000 7f3346a5e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3340101de0 0x7f33400744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:32.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:32.679+0000 7f3346a5e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3340102790 0x7f3340074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:32.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:32.679+0000 7f3346a5e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3340106f10 0x7f3340074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:32.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:32.679+0000 7f3346a5e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3340111f70 con 0x7f3340101de0 2024-09-12T02:00:32.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:32.679+0000 7f3346a5e640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3340111df0 con 0x7f3340102790 2024-09-12T02:00:32.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:32.679+0000 7f3346a5e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f33401120f0 con 0x7f3340106f10 2024-09-12T02:00:32.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:32.679+0000 7f333ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3340101de0 0x7f33400744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:32.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:32.679+0000 7f333ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3340101de0 0x7f33400744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38582/0 (socket says 172.21.15.16:38582) 2024-09-12T02:00:32.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:32.679+0000 7f333ffff640 1 -- 172.21.15.16:0/3342565826 learned_addr learned my addr 172.21.15.16:0/3342565826 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:00:32.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:32.679+0000 7f333f7fe640 1 --2- 172.21.15.16:0/3342565826 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3340102790 0x7f3340074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:32.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:32.679+0000 7f3344ffb640 1 --2- 172.21.15.16:0/3342565826 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3340106f10 0x7f3340074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:32.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:32.680+0000 7f333ffff640 1 -- 172.21.15.16:0/3342565826 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3340106f10 msgr2=0x7f3340074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:32.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:32.680+0000 7f333ffff640 1 --2- 172.21.15.16:0/3342565826 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3340106f10 0x7f3340074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:32.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:32.680+0000 7f333ffff640 1 -- 172.21.15.16:0/3342565826 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3340102790 msgr2=0x7f3340074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:32.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:32.680+0000 7f333ffff640 1 --2- 172.21.15.16:0/3342565826 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3340102790 0x7f3340074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:32.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:32.680+0000 7f333ffff640 1 -- 172.21.15.16:0/3342565826 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33401a6850 con 0x7f3340101de0 2024-09-12T02:00:32.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:32.680+0000 7f333ffff640 1 --2- 172.21.15.16:0/3342565826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3340101de0 0x7f33400744f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f333000d950 tx=0x7f333000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:00:32.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:32.680+0000 7f333d7fa640 1 -- 172.21.15.16:0/3342565826 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3330017070 con 0x7f3340101de0 2024-09-12T02:00:32.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:32.680+0000 7f333d7fa640 1 -- 172.21.15.16:0/3342565826 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3330009c30 con 0x7f3340101de0 2024-09-12T02:00:32.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:32.680+0000 7f333d7fa640 1 -- 172.21.15.16:0/3342565826 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f333000f6b0 con 0x7f3340101de0 2024-09-12T02:00:32.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:32.680+0000 7f3346a5e640 1 -- 172.21.15.16:0/3342565826 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33401a6b40 con 0x7f3340101de0 2024-09-12T02:00:32.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:32.680+0000 7f3346a5e640 1 -- 172.21.15.16:0/3342565826 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33401a70d0 con 0x7f3340101de0 2024-09-12T02:00:32.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:32.681+0000 7f3346a5e640 1 -- 172.21.15.16:0/3342565826 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f33400628c0 con 0x7f3340101de0 2024-09-12T02:00:32.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:32.682+0000 7f333d7fa640 1 -- 172.21.15.16:0/3342565826 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f33300109d0 con 0x7f3340101de0 2024-09-12T02:00:32.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:32.683+0000 7f333d7fa640 1 --2- 172.21.15.16:0/3342565826 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f33280a2c90 0x7f33280a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:32.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:32.683+0000 7f333f7fe640 1 --2- 172.21.15.16:0/3342565826 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f33280a2c90 0x7f33280a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:32.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:32.683+0000 7f333d7fa640 1 -- 172.21.15.16:0/3342565826 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f33300ceff0 con 0x7f3340101de0 2024-09-12T02:00:32.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:32.683+0000 7f333d7fa640 1 --2- 172.21.15.16:0/3342565826 >> [v2:172.21.15.16:6802/4000187867,v1:172.21.15.16:6803/4000187867] conn(0x7f33280ac740 0x7f33280aebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:32.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:32.683+0000 7f333d7fa640 1 -- 172.21.15.16:0/3342565826 --> [v2:172.21.15.16:6802/4000187867,v1:172.21.15.16:6803/4000187867] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f333001ce50 con 0x7f33280ac740 2024-09-12T02:00:32.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:32.683+0000 7f333d7fa640 1 -- 172.21.15.16:0/3342565826 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f33300cf4d0 con 0x7f3340101de0 2024-09-12T02:00:32.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:32.684+0000 7f3344ffb640 1 --2- 172.21.15.16:0/3342565826 >> [v2:172.21.15.16:6802/4000187867,v1:172.21.15.16:6803/4000187867] conn(0x7f33280ac740 0x7f33280aebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:32.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:32.684+0000 7f333f7fe640 1 --2- 172.21.15.16:0/3342565826 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f33280a2c90 0x7f33280a5150 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f3340075f80 tx=0x7f333400a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:00:32.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:32.684+0000 7f3344ffb640 1 --2- 172.21.15.16:0/3342565826 >> [v2:172.21.15.16:6802/4000187867,v1:172.21.15.16:6803/4000187867] conn(0x7f33280ac740 0x7f33280aebe0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:00:32.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:32.685+0000 7f333d7fa640 1 -- 172.21.15.16:0/3342565826 <== osd.0 v2:172.21.15.16:6802/4000187867 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f333001ce50 con 0x7f33280ac740 2024-09-12T02:00:32.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:32.698+0000 7f3346a5e640 1 -- 172.21.15.16:0/3342565826 --> [v2:172.21.15.16:6802/4000187867,v1:172.21.15.16:6803/4000187867] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3340062cc0 con 0x7f33280ac740 2024-09-12T02:00:32.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:32.699+0000 7f333d7fa640 1 -- 172.21.15.16:0/3342565826 <== osd.0 v2:172.21.15.16:6802/4000187867 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f3340062cc0 con 0x7f33280ac740 2024-09-12T02:00:32.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:32.699+0000 7f3346a5e640 1 -- 172.21.15.16:0/3342565826 >> [v2:172.21.15.16:6802/4000187867,v1:172.21.15.16:6803/4000187867] conn(0x7f33280ac740 msgr2=0x7f33280aebe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:32.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:32.699+0000 7f3346a5e640 1 --2- 172.21.15.16:0/3342565826 >> [v2:172.21.15.16:6802/4000187867,v1:172.21.15.16:6803/4000187867] conn(0x7f33280ac740 0x7f33280aebe0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:32.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:32.699+0000 7f3346a5e640 1 -- 172.21.15.16:0/3342565826 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f33280a2c90 msgr2=0x7f33280a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:32.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:32.699+0000 7f3346a5e640 1 --2- 172.21.15.16:0/3342565826 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f33280a2c90 0x7f33280a5150 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f3340075f80 tx=0x7f333400a2d0 comp rx=0 tx=0).stop 2024-09-12T02:00:32.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:32.699+0000 7f3346a5e640 1 -- 172.21.15.16:0/3342565826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3340101de0 msgr2=0x7f33400744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:32.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:32.699+0000 7f3346a5e640 1 --2- 172.21.15.16:0/3342565826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3340101de0 0x7f33400744f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f333000d950 tx=0x7f333000de10 comp rx=0 tx=0).stop 2024-09-12T02:00:32.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:32.699+0000 7f3344ffb640 1 -- 172.21.15.16:0/3342565826 reap_dead start 2024-09-12T02:00:32.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:32.699+0000 7f3346a5e640 1 -- 172.21.15.16:0/3342565826 shutdown_connections 2024-09-12T02:00:32.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:32.699+0000 7f3346a5e640 1 -- 172.21.15.16:0/3342565826 >> 172.21.15.16:0/3342565826 conn(0x7f33400fd880 msgr2=0x7f33400feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:00:32.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:32.699+0000 7f3346a5e640 1 -- 172.21.15.16:0/3342565826 shutdown_connections 2024-09-12T02:00:32.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:32.699+0000 7f3346a5e640 1 -- 172.21.15.16:0/3342565826 wait complete. 2024-09-12T02:00:33.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:33.645+0000 7f1314806640 1 -- 172.21.15.16:0/934426329 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1310101de0 msgr2=0x7f13101021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:33.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:33.645+0000 7f1314806640 1 --2- 172.21.15.16:0/934426329 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1310101de0 0x7f13101021c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f1300009ca0 tx=0x7f130002a8c0 comp rx=0 tx=0).stop 2024-09-12T02:00:33.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:33.645+0000 7f1314806640 1 -- 172.21.15.16:0/934426329 shutdown_connections 2024-09-12T02:00:33.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:33.645+0000 7f1314806640 1 --2- 172.21.15.16:0/934426329 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1310106f10 0x7f131010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:33.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:33.645+0000 7f1314806640 1 --2- 172.21.15.16:0/934426329 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1310102790 0x7f13101067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:33.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:33.645+0000 7f1314806640 1 --2- 172.21.15.16:0/934426329 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1310101de0 0x7f13101021c0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:33.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:33.645+0000 7f1314806640 1 -- 172.21.15.16:0/934426329 >> 172.21.15.16:0/934426329 conn(0x7f13100fd8c0 msgr2=0x7f13100ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:00:33.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:33.645+0000 7f1314806640 1 -- 172.21.15.16:0/934426329 shutdown_connections 2024-09-12T02:00:33.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:33.645+0000 7f1314806640 1 -- 172.21.15.16:0/934426329 wait complete. 2024-09-12T02:00:33.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:33.646+0000 7f1314806640 1 Processor -- start 2024-09-12T02:00:33.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:33.646+0000 7f1314806640 1 -- start start 2024-09-12T02:00:33.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:33.646+0000 7f1314806640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1310101de0 0x7f131019a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:33.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:33.646+0000 7f1314806640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1310102790 0x7f131019ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:33.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:33.646+0000 7f1314806640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1310106f10 0x7f131019efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:33.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:33.646+0000 7f1314806640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1310111f90 con 0x7f1310102790 2024-09-12T02:00:33.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:33.646+0000 7f1314806640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1310111e10 con 0x7f1310101de0 2024-09-12T02:00:33.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:33.646+0000 7f1314806640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1310112110 con 0x7f1310106f10 2024-09-12T02:00:33.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:33.646+0000 7f130dd9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1310102790 0x7f131019ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:33.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:33.646+0000 7f130dd9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1310102790 0x7f131019ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38616/0 (socket says 172.21.15.16:38616) 2024-09-12T02:00:33.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:33.646+0000 7f130dd9b640 1 -- 172.21.15.16:0/1794252436 learned_addr learned my addr 172.21.15.16:0/1794252436 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:00:33.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:33.646+0000 7f130ed9d640 1 --2- 172.21.15.16:0/1794252436 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1310106f10 0x7f131019efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:33.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:33.646+0000 7f130e59c640 1 --2- 172.21.15.16:0/1794252436 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1310101de0 0x7f131019a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:33.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:33.647+0000 7f130dd9b640 1 -- 172.21.15.16:0/1794252436 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1310106f10 msgr2=0x7f131019efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:33.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:33.647+0000 7f130dd9b640 1 --2- 172.21.15.16:0/1794252436 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1310106f10 0x7f131019efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:33.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:33.647+0000 7f130dd9b640 1 -- 172.21.15.16:0/1794252436 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1310101de0 msgr2=0x7f131019a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:33.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:33.647+0000 7f130dd9b640 1 --2- 172.21.15.16:0/1794252436 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1310101de0 0x7f131019a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:33.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:33.647+0000 7f130dd9b640 1 -- 172.21.15.16:0/1794252436 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f131019f6a0 con 0x7f1310102790 2024-09-12T02:00:33.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:33.647+0000 7f130dd9b640 1 --2- 172.21.15.16:0/1794252436 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1310102790 0x7f131019ac30 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f130400b4a0 tx=0x7f130400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:00:33.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:33.647+0000 7f12ff7fe640 1 -- 172.21.15.16:0/1794252436 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1304014070 con 0x7f1310102790 2024-09-12T02:00:33.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:33.647+0000 7f12ff7fe640 1 -- 172.21.15.16:0/1794252436 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1304010af0 con 0x7f1310102790 2024-09-12T02:00:33.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:33.647+0000 7f1314806640 1 -- 172.21.15.16:0/1794252436 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f131019f990 con 0x7f1310102790 2024-09-12T02:00:33.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:33.647+0000 7f12ff7fe640 1 -- 172.21.15.16:0/1794252436 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1304017d10 con 0x7f1310102790 2024-09-12T02:00:33.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:33.647+0000 7f1314806640 1 -- 172.21.15.16:0/1794252436 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13101a7270 con 0x7f1310102790 2024-09-12T02:00:33.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:33.648+0000 7f1314806640 1 -- 172.21.15.16:0/1794252436 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f131010e0d0 con 0x7f1310102790 2024-09-12T02:00:33.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:33.649+0000 7f12ff7fe640 1 -- 172.21.15.16:0/1794252436 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f1304017070 con 0x7f1310102790 2024-09-12T02:00:33.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:33.650+0000 7f12ff7fe640 1 --2- 172.21.15.16:0/1794252436 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f12f80a2c10 0x7f12f80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:33.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:33.650+0000 7f130e59c640 1 --2- 172.21.15.16:0/1794252436 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f12f80a2c10 0x7f12f80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:33.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:33.650+0000 7f12ff7fe640 1 -- 172.21.15.16:0/1794252436 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f13040ce910 con 0x7f1310102790 2024-09-12T02:00:33.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:33.650+0000 7f12ff7fe640 1 --2- 172.21.15.16:0/1794252436 >> [v2:172.21.15.184:6808/1533397276,v1:172.21.15.184:6809/1533397276] conn(0x7f12f80ac6c0 0x7f12f80aeb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:33.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:33.650+0000 7f12ff7fe640 1 -- 172.21.15.16:0/1794252436 --> [v2:172.21.15.184:6808/1533397276,v1:172.21.15.184:6809/1533397276] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f12f80af230 con 0x7f12f80ac6c0 2024-09-12T02:00:33.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:33.650+0000 7f12ff7fe640 1 -- 172.21.15.16:0/1794252436 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f13040cecc0 con 0x7f1310102790 2024-09-12T02:00:33.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:33.650+0000 7f130ed9d640 1 --2- 172.21.15.16:0/1794252436 >> [v2:172.21.15.184:6808/1533397276,v1:172.21.15.184:6809/1533397276] conn(0x7f12f80ac6c0 0x7f12f80aeb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:33.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:33.650+0000 7f130e59c640 1 --2- 172.21.15.16:0/1794252436 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f12f80a2c10 0x7f12f80a50d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f130002f540 tx=0x7f130002f310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:00:33.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:33.651+0000 7f130ed9d640 1 --2- 172.21.15.16:0/1794252436 >> [v2:172.21.15.184:6808/1533397276,v1:172.21.15.184:6809/1533397276] conn(0x7f12f80ac6c0 0x7f12f80aeb60 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:00:33.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:33.652+0000 7f12ff7fe640 1 -- 172.21.15.16:0/1794252436 <== osd.5 v2:172.21.15.184:6808/1533397276 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f12f80af230 con 0x7f12f80ac6c0 2024-09-12T02:00:33.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:33.666+0000 7f1314806640 1 -- 172.21.15.16:0/1794252436 --> [v2:172.21.15.184:6808/1533397276,v1:172.21.15.184:6809/1533397276] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f13101032c0 con 0x7f12f80ac6c0 2024-09-12T02:00:33.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:33.667+0000 7f12ff7fe640 1 -- 172.21.15.16:0/1794252436 <== osd.5 v2:172.21.15.184:6808/1533397276 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f13101032c0 con 0x7f12f80ac6c0 2024-09-12T02:00:33.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:33.667+0000 7f1314806640 1 -- 172.21.15.16:0/1794252436 >> [v2:172.21.15.184:6808/1533397276,v1:172.21.15.184:6809/1533397276] conn(0x7f12f80ac6c0 msgr2=0x7f12f80aeb60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:33.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:33.667+0000 7f1314806640 1 --2- 172.21.15.16:0/1794252436 >> [v2:172.21.15.184:6808/1533397276,v1:172.21.15.184:6809/1533397276] conn(0x7f12f80ac6c0 0x7f12f80aeb60 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:33.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:33.667+0000 7f1314806640 1 -- 172.21.15.16:0/1794252436 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f12f80a2c10 msgr2=0x7f12f80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:33.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:33.667+0000 7f1314806640 1 --2- 172.21.15.16:0/1794252436 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f12f80a2c10 0x7f12f80a50d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f130002f540 tx=0x7f130002f310 comp rx=0 tx=0).stop 2024-09-12T02:00:33.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:33.667+0000 7f1314806640 1 -- 172.21.15.16:0/1794252436 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1310102790 msgr2=0x7f131019ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:33.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:33.667+0000 7f1314806640 1 --2- 172.21.15.16:0/1794252436 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1310102790 0x7f131019ac30 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f130400b4a0 tx=0x7f130400b960 comp rx=0 tx=0).stop 2024-09-12T02:00:33.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:33.667+0000 7f130ed9d640 1 -- 172.21.15.16:0/1794252436 reap_dead start 2024-09-12T02:00:33.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:33.667+0000 7f1314806640 1 -- 172.21.15.16:0/1794252436 shutdown_connections 2024-09-12T02:00:33.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:33.667+0000 7f1314806640 1 -- 172.21.15.16:0/1794252436 >> 172.21.15.16:0/1794252436 conn(0x7f13100fd8c0 msgr2=0x7f13100febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:00:33.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:33.667+0000 7f1314806640 1 -- 172.21.15.16:0/1794252436 shutdown_connections 2024-09-12T02:00:33.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:33.667+0000 7f1314806640 1 -- 172.21.15.16:0/1794252436 wait complete. 2024-09-12T02:00:34.183 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:00:34 smithi016 ceph-mon[30947]: pgmap v294: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 938 B/s wr, 6 op/s 2024-09-12T02:00:34.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:00:34 smithi184 ceph-mon[35593]: pgmap v294: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 938 B/s wr, 6 op/s 2024-09-12T02:00:34.418 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:00:34 smithi033 ceph-mon[35332]: pgmap v294: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 938 B/s wr, 6 op/s 2024-09-12T02:00:34.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:33.309+0000 7f3de33c4640 1 -- 172.21.15.16:0/408699793 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ddc106f10 msgr2=0x7f3ddc10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:34.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:33.309+0000 7f3de33c4640 1 --2- 172.21.15.16:0/408699793 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ddc106f10 0x7f3ddc10ea90 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f3dd4009ca0 tx=0x7f3dd402a8c0 comp rx=0 tx=0).stop 2024-09-12T02:00:34.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:33.309+0000 7f3de33c4640 1 -- 172.21.15.16:0/408699793 shutdown_connections 2024-09-12T02:00:34.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:33.309+0000 7f3de33c4640 1 --2- 172.21.15.16:0/408699793 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ddc106f10 0x7f3ddc10ea90 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:34.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:33.309+0000 7f3de33c4640 1 --2- 172.21.15.16:0/408699793 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ddc102790 0x7f3ddc1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:34.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:33.309+0000 7f3de33c4640 1 --2- 172.21.15.16:0/408699793 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3ddc101de0 0x7f3ddc1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:34.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:33.309+0000 7f3de33c4640 1 -- 172.21.15.16:0/408699793 >> 172.21.15.16:0/408699793 conn(0x7f3ddc0fd8c0 msgr2=0x7f3ddc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:00:34.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:33.309+0000 7f3de33c4640 1 -- 172.21.15.16:0/408699793 shutdown_connections 2024-09-12T02:00:34.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:33.309+0000 7f3de33c4640 1 -- 172.21.15.16:0/408699793 wait complete. 2024-09-12T02:00:34.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:33.309+0000 7f3de33c4640 1 Processor -- start 2024-09-12T02:00:34.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:33.310+0000 7f3de33c4640 1 -- start start 2024-09-12T02:00:34.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:33.310+0000 7f3de33c4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ddc101de0 0x7f3ddc198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:34.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:33.310+0000 7f3de33c4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ddc102790 0x7f3ddc198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:34.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:33.310+0000 7f3de33c4640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3ddc106f10 0x7f3ddc19ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:34.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:33.310+0000 7f3de33c4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3ddc112010 con 0x7f3ddc102790 2024-09-12T02:00:34.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:33.310+0000 7f3de33c4640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3ddc111e90 con 0x7f3ddc101de0 2024-09-12T02:00:34.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:33.310+0000 7f3de33c4640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f3ddc112190 con 0x7f3ddc106f10 2024-09-12T02:00:34.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:33.310+0000 7f3de095f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ddc102790 0x7f3ddc198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:34.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:33.310+0000 7f3de095f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ddc102790 0x7f3ddc198ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38608/0 (socket says 172.21.15.16:38608) 2024-09-12T02:00:34.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:33.310+0000 7f3de095f640 1 -- 172.21.15.16:0/461183207 learned_addr learned my addr 172.21.15.16:0/461183207 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:00:34.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:33.310+0000 7f3de1961640 1 --2- 172.21.15.16:0/461183207 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3ddc106f10 0x7f3ddc19ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:34.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:33.310+0000 7f3de1160640 1 --2- 172.21.15.16:0/461183207 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ddc101de0 0x7f3ddc198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:34.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:33.310+0000 7f3de095f640 1 -- 172.21.15.16:0/461183207 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3ddc106f10 msgr2=0x7f3ddc19ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:34.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:33.310+0000 7f3de095f640 1 --2- 172.21.15.16:0/461183207 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3ddc106f10 0x7f3ddc19ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:34.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:33.310+0000 7f3de095f640 1 -- 172.21.15.16:0/461183207 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ddc101de0 msgr2=0x7f3ddc198570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:34.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:33.310+0000 7f3de095f640 1 --2- 172.21.15.16:0/461183207 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ddc101de0 0x7f3ddc198570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:34.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:33.310+0000 7f3de095f640 1 -- 172.21.15.16:0/461183207 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ddc19d520 con 0x7f3ddc102790 2024-09-12T02:00:34.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:33.310+0000 7f3de095f640 1 --2- 172.21.15.16:0/461183207 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ddc102790 0x7f3ddc198ab0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f3dcc00b4a0 tx=0x7f3dcc00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:00:34.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:33.311+0000 7f3dd27fc640 1 -- 172.21.15.16:0/461183207 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dcc014070 con 0x7f3ddc102790 2024-09-12T02:00:34.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:33.311+0000 7f3dd27fc640 1 -- 172.21.15.16:0/461183207 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3dcc010af0 con 0x7f3ddc102790 2024-09-12T02:00:34.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:33.311+0000 7f3de33c4640 1 -- 172.21.15.16:0/461183207 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ddc19d810 con 0x7f3ddc102790 2024-09-12T02:00:34.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:33.311+0000 7f3dd27fc640 1 -- 172.21.15.16:0/461183207 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dcc017d10 con 0x7f3ddc102790 2024-09-12T02:00:34.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:33.311+0000 7f3de33c4640 1 -- 172.21.15.16:0/461183207 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ddc1a50f0 con 0x7f3ddc102790 2024-09-12T02:00:34.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:33.311+0000 7f3de33c4640 1 -- 172.21.15.16:0/461183207 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3ddc062980 con 0x7f3ddc102790 2024-09-12T02:00:34.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:33.313+0000 7f3dd27fc640 1 -- 172.21.15.16:0/461183207 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f3dcc017070 con 0x7f3ddc102790 2024-09-12T02:00:34.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:33.313+0000 7f3dd27fc640 1 --2- 172.21.15.16:0/461183207 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f3dc80a2c10 0x7f3dc80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:34.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:33.314+0000 7f3de1160640 1 --2- 172.21.15.16:0/461183207 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f3dc80a2c10 0x7f3dc80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:34.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:33.314+0000 7f3dd27fc640 1 -- 172.21.15.16:0/461183207 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f3dcc0ce7e0 con 0x7f3ddc102790 2024-09-12T02:00:34.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:33.314+0000 7f3dd27fc640 1 --2- 172.21.15.16:0/461183207 >> [v2:172.21.15.184:6800/3958753963,v1:172.21.15.184:6801/3958753963] conn(0x7f3dc80ac6c0 0x7f3dc80aeb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:34.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:33.314+0000 7f3dd27fc640 1 -- 172.21.15.16:0/461183207 --> [v2:172.21.15.184:6800/3958753963,v1:172.21.15.184:6801/3958753963] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3dcc01ae50 con 0x7f3dc80ac6c0 2024-09-12T02:00:34.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:33.314+0000 7f3dd27fc640 1 -- 172.21.15.16:0/461183207 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f3dcc0cecc0 con 0x7f3ddc102790 2024-09-12T02:00:34.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:33.314+0000 7f3de1160640 1 --2- 172.21.15.16:0/461183207 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f3dc80a2c10 0x7f3dc80a50d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f3dd8009890 tx=0x7f3dd8008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:00:34.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:33.314+0000 7f3de1961640 1 --2- 172.21.15.16:0/461183207 >> [v2:172.21.15.184:6800/3958753963,v1:172.21.15.184:6801/3958753963] conn(0x7f3dc80ac6c0 0x7f3dc80aeb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:34.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:33.314+0000 7f3de1961640 1 --2- 172.21.15.16:0/461183207 >> [v2:172.21.15.184:6800/3958753963,v1:172.21.15.184:6801/3958753963] conn(0x7f3dc80ac6c0 0x7f3dc80aeb60 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:00:34.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:33.315+0000 7f3dd27fc640 1 -- 172.21.15.16:0/461183207 <== osd.2 v2:172.21.15.184:6800/3958753963 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3dcc01ae50 con 0x7f3dc80ac6c0 2024-09-12T02:00:34.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:33.329+0000 7f3de33c4640 1 -- 172.21.15.16:0/461183207 --> [v2:172.21.15.184:6800/3958753963,v1:172.21.15.184:6801/3958753963] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3ddc062e00 con 0x7f3dc80ac6c0 2024-09-12T02:00:34.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:33.330+0000 7f3dd27fc640 1 -- 172.21.15.16:0/461183207 <== osd.2 v2:172.21.15.184:6800/3958753963 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f3ddc062e00 con 0x7f3dc80ac6c0 2024-09-12T02:00:34.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:33.330+0000 7f3de33c4640 1 -- 172.21.15.16:0/461183207 >> [v2:172.21.15.184:6800/3958753963,v1:172.21.15.184:6801/3958753963] conn(0x7f3dc80ac6c0 msgr2=0x7f3dc80aeb60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:34.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:33.330+0000 7f3de33c4640 1 --2- 172.21.15.16:0/461183207 >> [v2:172.21.15.184:6800/3958753963,v1:172.21.15.184:6801/3958753963] conn(0x7f3dc80ac6c0 0x7f3dc80aeb60 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:34.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:33.330+0000 7f3de33c4640 1 -- 172.21.15.16:0/461183207 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f3dc80a2c10 msgr2=0x7f3dc80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:34.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:33.330+0000 7f3de33c4640 1 --2- 172.21.15.16:0/461183207 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f3dc80a2c10 0x7f3dc80a50d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f3dd8009890 tx=0x7f3dd8008040 comp rx=0 tx=0).stop 2024-09-12T02:00:34.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:33.330+0000 7f3de33c4640 1 -- 172.21.15.16:0/461183207 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ddc102790 msgr2=0x7f3ddc198ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:34.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:33.330+0000 7f3de33c4640 1 --2- 172.21.15.16:0/461183207 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ddc102790 0x7f3ddc198ab0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f3dcc00b4a0 tx=0x7f3dcc00b960 comp rx=0 tx=0).stop 2024-09-12T02:00:34.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:33.330+0000 7f3de1961640 1 -- 172.21.15.16:0/461183207 reap_dead start 2024-09-12T02:00:34.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:33.330+0000 7f3de33c4640 1 -- 172.21.15.16:0/461183207 shutdown_connections 2024-09-12T02:00:34.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:33.330+0000 7f3de33c4640 1 -- 172.21.15.16:0/461183207 >> 172.21.15.16:0/461183207 conn(0x7f3ddc0fd8c0 msgr2=0x7f3ddc0fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:00:34.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:33.330+0000 7f3de33c4640 1 -- 172.21.15.16:0/461183207 shutdown_connections 2024-09-12T02:00:34.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:33.330+0000 7f3de33c4640 1 -- 172.21.15.16:0/461183207 wait complete. 2024-09-12T02:00:35.268 INFO:teuthology.orchestra.run.smithi016.stdout:73014444097 2024-09-12T02:00:35.269 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-09-12T02:00:35.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:35.398+0000 7fe900fa1640 1 -- 172.21.15.16:0/1275964728 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8fc102dd0 msgr2=0x7fe8fc1031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:35.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:35.398+0000 7fe900fa1640 1 --2- 172.21.15.16:0/1275964728 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8fc102dd0 0x7fe8fc1031b0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fe8ec009930 tx=0x7fe8ec02a420 comp rx=0 tx=0).stop 2024-09-12T02:00:35.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:35.399+0000 7fe900fa1640 1 -- 172.21.15.16:0/1275964728 shutdown_connections 2024-09-12T02:00:35.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:35.399+0000 7fe900fa1640 1 --2- 172.21.15.16:0/1275964728 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe8fc108300 0x7fe8fc10ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:35.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:35.399+0000 7fe900fa1640 1 --2- 172.21.15.16:0/1275964728 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe8fc103780 0x7fe8fc107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:35.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:35.399+0000 7fe900fa1640 1 --2- 172.21.15.16:0/1275964728 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8fc102dd0 0x7fe8fc1031b0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:35.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:35.399+0000 7fe900fa1640 1 -- 172.21.15.16:0/1275964728 >> 172.21.15.16:0/1275964728 conn(0x7fe8fc0fc7d0 msgr2=0x7fe8fc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:00:35.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:35.399+0000 7fe900fa1640 1 -- 172.21.15.16:0/1275964728 shutdown_connections 2024-09-12T02:00:35.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:35.399+0000 7fe900fa1640 1 -- 172.21.15.16:0/1275964728 wait complete. 2024-09-12T02:00:35.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:35.399+0000 7fe900fa1640 1 Processor -- start 2024-09-12T02:00:35.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:35.399+0000 7fe900fa1640 1 -- start start 2024-09-12T02:00:35.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:35.400+0000 7fe900fa1640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe8fc102dd0 0x7fe8fc198610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:35.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:35.400+0000 7fe900fa1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8fc103780 0x7fe8fc198b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:35.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:35.400+0000 7fe900fa1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe8fc108300 0x7fe8fc19cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:35.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:35.400+0000 7fe900fa1640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe8fc112110 con 0x7fe8fc103780 2024-09-12T02:00:35.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:35.400+0000 7fe900fa1640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe8fc111f90 con 0x7fe8fc108300 2024-09-12T02:00:35.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:35.400+0000 7fe900fa1640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe8fc112290 con 0x7fe8fc102dd0 2024-09-12T02:00:35.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:35.400+0000 7fe8f9d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8fc103780 0x7fe8fc198b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:35.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:35.400+0000 7fe8f9d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8fc103780 0x7fe8fc198b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38630/0 (socket says 172.21.15.16:38630) 2024-09-12T02:00:35.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:35.400+0000 7fe8f9d9b640 1 -- 172.21.15.16:0/3933037764 learned_addr learned my addr 172.21.15.16:0/3933037764 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:00:35.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:35.400+0000 7fe8fa59c640 1 --2- 172.21.15.16:0/3933037764 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe8fc102dd0 0x7fe8fc198610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:35.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:35.400+0000 7fe8fad9d640 1 --2- 172.21.15.16:0/3933037764 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe8fc108300 0x7fe8fc19cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:35.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:35.400+0000 7fe8f9d9b640 1 -- 172.21.15.16:0/3933037764 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe8fc102dd0 msgr2=0x7fe8fc198610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:35.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:35.400+0000 7fe8f9d9b640 1 --2- 172.21.15.16:0/3933037764 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe8fc102dd0 0x7fe8fc198610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:35.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:35.400+0000 7fe8f9d9b640 1 -- 172.21.15.16:0/3933037764 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe8fc108300 msgr2=0x7fe8fc19cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:35.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:35.400+0000 7fe8f9d9b640 1 --2- 172.21.15.16:0/3933037764 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe8fc108300 0x7fe8fc19cee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:35.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:35.400+0000 7fe8f9d9b640 1 -- 172.21.15.16:0/3933037764 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8fc19d5c0 con 0x7fe8fc103780 2024-09-12T02:00:35.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:35.400+0000 7fe8fad9d640 1 --2- 172.21.15.16:0/3933037764 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe8fc108300 0x7fe8fc19cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:00:35.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:35.400+0000 7fe8f9d9b640 1 --2- 172.21.15.16:0/3933037764 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8fc103780 0x7fe8fc198b50 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fe8f000e9f0 tx=0x7fe8f000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:00:35.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:35.401+0000 7fe8eb7fe640 1 -- 172.21.15.16:0/3933037764 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8f000ccc0 con 0x7fe8fc103780 2024-09-12T02:00:35.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:35.401+0000 7fe900fa1640 1 -- 172.21.15.16:0/3933037764 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8fc19d8b0 con 0x7fe8fc103780 2024-09-12T02:00:35.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:35.401+0000 7fe8eb7fe640 1 -- 172.21.15.16:0/3933037764 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe8f0010910 con 0x7fe8fc103780 2024-09-12T02:00:35.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:35.401+0000 7fe8eb7fe640 1 -- 172.21.15.16:0/3933037764 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8f0018920 con 0x7fe8fc103780 2024-09-12T02:00:35.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:35.401+0000 7fe900fa1640 1 -- 172.21.15.16:0/3933037764 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8fc1a5190 con 0x7fe8fc103780 2024-09-12T02:00:35.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:35.402+0000 7fe900fa1640 1 -- 172.21.15.16:0/3933037764 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe8fc062980 con 0x7fe8fc103780 2024-09-12T02:00:35.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:35.403+0000 7fe8eb7fe640 1 -- 172.21.15.16:0/3933037764 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fe8f0010ac0 con 0x7fe8fc103780 2024-09-12T02:00:35.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:35.403+0000 7fe8eb7fe640 1 --2- 172.21.15.16:0/3933037764 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fe8e40a2c10 0x7fe8e40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:35.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:35.404+0000 7fe8fa59c640 1 --2- 172.21.15.16:0/3933037764 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fe8e40a2c10 0x7fe8e40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:35.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:35.404+0000 7fe8eb7fe640 1 -- 172.21.15.16:0/3933037764 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fe8f00cf420 con 0x7fe8fc103780 2024-09-12T02:00:35.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:35.404+0000 7fe8eb7fe640 1 --2- 172.21.15.16:0/3933037764 >> [v2:172.21.15.16:6810/3395182524,v1:172.21.15.16:6811/3395182524] conn(0x7fe8e40ac6c0 0x7fe8e40aeb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:35.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:35.404+0000 7fe8eb7fe640 1 -- 172.21.15.16:0/3933037764 --> [v2:172.21.15.16:6810/3395182524,v1:172.21.15.16:6811/3395182524] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe8e40af230 con 0x7fe8e40ac6c0 2024-09-12T02:00:35.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:35.404+0000 7fe8eb7fe640 1 -- 172.21.15.16:0/3933037764 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7fe8f00d6050 con 0x7fe8fc103780 2024-09-12T02:00:35.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:35.404+0000 7fe8fad9d640 1 --2- 172.21.15.16:0/3933037764 >> [v2:172.21.15.16:6810/3395182524,v1:172.21.15.16:6811/3395182524] conn(0x7fe8e40ac6c0 0x7fe8e40aeb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:35.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:35.404+0000 7fe8fa59c640 1 --2- 172.21.15.16:0/3933037764 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fe8e40a2c10 0x7fe8e40a50d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fe8ec009900 tx=0x7fe8ec0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:00:35.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:35.404+0000 7fe8fad9d640 1 --2- 172.21.15.16:0/3933037764 >> [v2:172.21.15.16:6810/3395182524,v1:172.21.15.16:6811/3395182524] conn(0x7fe8e40ac6c0 0x7fe8e40aeb60 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:00:35.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:35.405+0000 7fe8eb7fe640 1 -- 172.21.15.16:0/3933037764 <== osd.3 v2:172.21.15.16:6810/3395182524 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe8e40af230 con 0x7fe8e40ac6c0 2024-09-12T02:00:35.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:35.419+0000 7fe900fa1640 1 -- 172.21.15.16:0/3933037764 --> [v2:172.21.15.16:6810/3395182524,v1:172.21.15.16:6811/3395182524] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe8fc062db0 con 0x7fe8e40ac6c0 2024-09-12T02:00:35.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:35.419+0000 7fe8eb7fe640 1 -- 172.21.15.16:0/3933037764 <== osd.3 v2:172.21.15.16:6810/3395182524 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fe8fc062db0 con 0x7fe8e40ac6c0 2024-09-12T02:00:35.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:35.420+0000 7fe900fa1640 1 -- 172.21.15.16:0/3933037764 >> [v2:172.21.15.16:6810/3395182524,v1:172.21.15.16:6811/3395182524] conn(0x7fe8e40ac6c0 msgr2=0x7fe8e40aeb60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:35.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:35.420+0000 7fe900fa1640 1 --2- 172.21.15.16:0/3933037764 >> [v2:172.21.15.16:6810/3395182524,v1:172.21.15.16:6811/3395182524] conn(0x7fe8e40ac6c0 0x7fe8e40aeb60 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:35.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:35.420+0000 7fe900fa1640 1 -- 172.21.15.16:0/3933037764 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fe8e40a2c10 msgr2=0x7fe8e40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:35.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:35.420+0000 7fe900fa1640 1 --2- 172.21.15.16:0/3933037764 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fe8e40a2c10 0x7fe8e40a50d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fe8ec009900 tx=0x7fe8ec0058d0 comp rx=0 tx=0).stop 2024-09-12T02:00:35.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:35.420+0000 7fe900fa1640 1 -- 172.21.15.16:0/3933037764 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8fc103780 msgr2=0x7fe8fc198b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:35.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:35.420+0000 7fe900fa1640 1 --2- 172.21.15.16:0/3933037764 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8fc103780 0x7fe8fc198b50 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fe8f000e9f0 tx=0x7fe8f000eeb0 comp rx=0 tx=0).stop 2024-09-12T02:00:35.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:35.420+0000 7fe8fad9d640 1 -- 172.21.15.16:0/3933037764 reap_dead start 2024-09-12T02:00:35.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:35.420+0000 7fe900fa1640 1 -- 172.21.15.16:0/3933037764 shutdown_connections 2024-09-12T02:00:35.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:35.420+0000 7fe900fa1640 1 -- 172.21.15.16:0/3933037764 >> 172.21.15.16:0/3933037764 conn(0x7fe8fc0fc7d0 msgr2=0x7fe8fc105dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:00:35.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:35.420+0000 7fe900fa1640 1 -- 172.21.15.16:0/3933037764 shutdown_connections 2024-09-12T02:00:35.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:35.420+0000 7fe900fa1640 1 -- 172.21.15.16:0/3933037764 wait complete. 2024-09-12T02:00:36.183 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:00:36 smithi016 ceph-mon[30947]: pgmap v295: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 938 B/s wr, 7 op/s 2024-09-12T02:00:36.217 INFO:teuthology.orchestra.run.smithi016.stdout:236223201314 2024-09-12T02:00:36.218 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-09-12T02:00:36.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:00:36 smithi184 ceph-mon[35593]: pgmap v295: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 938 B/s wr, 7 op/s 2024-09-12T02:00:36.418 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:00:36 smithi033 ceph-mon[35332]: pgmap v295: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 938 B/s wr, 7 op/s 2024-09-12T02:00:36.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.797+0000 7fc9ed4cf640 1 -- 172.21.15.16:0/3664526048 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9e8106f10 msgr2=0x7fc9e810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:36.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.797+0000 7fc9ed4cf640 1 --2- 172.21.15.16:0/3664526048 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9e8106f10 0x7fc9e810ea90 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fc9e0009ca0 tx=0x7fc9e002a8c0 comp rx=0 tx=0).stop 2024-09-12T02:00:36.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.797+0000 7fc9ed4cf640 1 -- 172.21.15.16:0/3664526048 shutdown_connections 2024-09-12T02:00:36.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.797+0000 7fc9ed4cf640 1 --2- 172.21.15.16:0/3664526048 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9e8106f10 0x7fc9e810ea90 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:36.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.797+0000 7fc9ed4cf640 1 --2- 172.21.15.16:0/3664526048 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc9e8102790 0x7fc9e81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:36.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.797+0000 7fc9ed4cf640 1 --2- 172.21.15.16:0/3664526048 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc9e8101de0 0x7fc9e81021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:36.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.797+0000 7fc9ed4cf640 1 -- 172.21.15.16:0/3664526048 >> 172.21.15.16:0/3664526048 conn(0x7fc9e80fd8c0 msgr2=0x7fc9e80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:00:36.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.798+0000 7fc9ed4cf640 1 -- 172.21.15.16:0/3664526048 shutdown_connections 2024-09-12T02:00:36.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.798+0000 7fc9ed4cf640 1 -- 172.21.15.16:0/3664526048 wait complete. 2024-09-12T02:00:36.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.798+0000 7fc9ed4cf640 1 Processor -- start 2024-09-12T02:00:36.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.798+0000 7fc9ed4cf640 1 -- start start 2024-09-12T02:00:36.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.799+0000 7fc9ed4cf640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc9e8101de0 0x7fc9e81983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:36.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.799+0000 7fc9ed4cf640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc9e8102790 0x7fc9e8198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:36.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.799+0000 7fc9ed4cf640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9e8106f10 0x7fc9e819cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:36.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.799+0000 7fc9ed4cf640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc9e8111f70 con 0x7fc9e8106f10 2024-09-12T02:00:36.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.799+0000 7fc9ed4cf640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc9e8111df0 con 0x7fc9e8102790 2024-09-12T02:00:36.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.799+0000 7fc9ed4cf640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc9e81120f0 con 0x7fc9e8101de0 2024-09-12T02:00:36.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.799+0000 7fc9e77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9e8106f10 0x7fc9e819cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:36.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.799+0000 7fc9e77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9e8106f10 0x7fc9e819cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38700/0 (socket says 172.21.15.16:38700) 2024-09-12T02:00:36.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.799+0000 7fc9e77fe640 1 -- 172.21.15.16:0/3207521152 learned_addr learned my addr 172.21.15.16:0/3207521152 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:00:36.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.799+0000 7fc9e6ffd640 1 --2- 172.21.15.16:0/3207521152 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc9e8101de0 0x7fc9e81983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:36.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.799+0000 7fc9e67fc640 1 --2- 172.21.15.16:0/3207521152 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc9e8102790 0x7fc9e8198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:36.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.799+0000 7fc9e77fe640 1 -- 172.21.15.16:0/3207521152 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc9e8101de0 msgr2=0x7fc9e81983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:36.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.799+0000 7fc9e77fe640 1 --2- 172.21.15.16:0/3207521152 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc9e8101de0 0x7fc9e81983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:36.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.799+0000 7fc9e77fe640 1 -- 172.21.15.16:0/3207521152 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc9e8102790 msgr2=0x7fc9e8198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:36.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.799+0000 7fc9e77fe640 1 --2- 172.21.15.16:0/3207521152 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc9e8102790 0x7fc9e8198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:36.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.799+0000 7fc9e77fe640 1 -- 172.21.15.16:0/3207521152 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9e819d380 con 0x7fc9e8106f10 2024-09-12T02:00:36.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.799+0000 7fc9e77fe640 1 --2- 172.21.15.16:0/3207521152 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9e8106f10 0x7fc9e819cca0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fc9e0031a20 tx=0x7fc9e0031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:00:36.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.799+0000 7fc9cffff640 1 -- 172.21.15.16:0/3207521152 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9e0003c30 con 0x7fc9e8106f10 2024-09-12T02:00:36.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.799+0000 7fc9ed4cf640 1 -- 172.21.15.16:0/3207521152 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9e819d610 con 0x7fc9e8106f10 2024-09-12T02:00:36.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.799+0000 7fc9cffff640 1 -- 172.21.15.16:0/3207521152 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc9e0003dd0 con 0x7fc9e8106f10 2024-09-12T02:00:36.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.799+0000 7fc9cffff640 1 -- 172.21.15.16:0/3207521152 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9e002fc40 con 0x7fc9e8106f10 2024-09-12T02:00:36.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.800+0000 7fc9ed4cf640 1 -- 172.21.15.16:0/3207521152 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9e81a4f50 con 0x7fc9e8106f10 2024-09-12T02:00:36.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.800+0000 7fc9ed4cf640 1 -- 172.21.15.16:0/3207521152 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc9e8062980 con 0x7fc9e8106f10 2024-09-12T02:00:36.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.802+0000 7fc9cffff640 1 -- 172.21.15.16:0/3207521152 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fc9e0004050 con 0x7fc9e8106f10 2024-09-12T02:00:36.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.802+0000 7fc9cffff640 1 --2- 172.21.15.16:0/3207521152 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc9d00a2c10 0x7fc9d00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:36.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.802+0000 7fc9e6ffd640 1 --2- 172.21.15.16:0/3207521152 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc9d00a2c10 0x7fc9d00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:36.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.802+0000 7fc9cffff640 1 -- 172.21.15.16:0/3207521152 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fc9e00f0460 con 0x7fc9e8106f10 2024-09-12T02:00:36.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.802+0000 7fc9cffff640 1 --2- 172.21.15.16:0/3207521152 >> [v2:172.21.15.33:6816/4186032630,v1:172.21.15.33:6817/4186032630] conn(0x7fc9d00ac590 0x7fc9d00aea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:36.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.802+0000 7fc9cffff640 1 -- 172.21.15.16:0/3207521152 --> [v2:172.21.15.33:6816/4186032630,v1:172.21.15.33:6817/4186032630] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc9d00af100 con 0x7fc9d00ac590 2024-09-12T02:00:36.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.802+0000 7fc9cffff640 1 -- 172.21.15.16:0/3207521152 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7fc9e00f0940 con 0x7fc9e8106f10 2024-09-12T02:00:36.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.803+0000 7fc9e6ffd640 1 --2- 172.21.15.16:0/3207521152 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc9d00a2c10 0x7fc9d00a50d0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fc9d8009bd0 tx=0x7fc9d8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:00:36.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.803+0000 7fc9e67fc640 1 --2- 172.21.15.16:0/3207521152 >> [v2:172.21.15.33:6816/4186032630,v1:172.21.15.33:6817/4186032630] conn(0x7fc9d00ac590 0x7fc9d00aea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:36.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.803+0000 7fc9e67fc640 1 --2- 172.21.15.16:0/3207521152 >> [v2:172.21.15.33:6816/4186032630,v1:172.21.15.33:6817/4186032630] conn(0x7fc9d00ac590 0x7fc9d00aea30 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:00:36.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.804+0000 7fc9cffff640 1 -- 172.21.15.16:0/3207521152 <== osd.7 v2:172.21.15.33:6816/4186032630 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc9d00af100 con 0x7fc9d00ac590 2024-09-12T02:00:36.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.818+0000 7fc9ed4cf640 1 -- 172.21.15.16:0/3207521152 --> [v2:172.21.15.33:6816/4186032630,v1:172.21.15.33:6817/4186032630] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc9e8062d80 con 0x7fc9d00ac590 2024-09-12T02:00:36.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.818+0000 7fc9cffff640 1 -- 172.21.15.16:0/3207521152 <== osd.7 v2:172.21.15.33:6816/4186032630 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fc9e8062d80 con 0x7fc9d00ac590 2024-09-12T02:00:36.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.819+0000 7fc9ed4cf640 1 -- 172.21.15.16:0/3207521152 >> [v2:172.21.15.33:6816/4186032630,v1:172.21.15.33:6817/4186032630] conn(0x7fc9d00ac590 msgr2=0x7fc9d00aea30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:36.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.819+0000 7fc9ed4cf640 1 --2- 172.21.15.16:0/3207521152 >> [v2:172.21.15.33:6816/4186032630,v1:172.21.15.33:6817/4186032630] conn(0x7fc9d00ac590 0x7fc9d00aea30 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:36.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.819+0000 7fc9ed4cf640 1 -- 172.21.15.16:0/3207521152 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc9d00a2c10 msgr2=0x7fc9d00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:36.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.819+0000 7fc9ed4cf640 1 --2- 172.21.15.16:0/3207521152 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc9d00a2c10 0x7fc9d00a50d0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fc9d8009bd0 tx=0x7fc9d8005f90 comp rx=0 tx=0).stop 2024-09-12T02:00:36.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.819+0000 7fc9ed4cf640 1 -- 172.21.15.16:0/3207521152 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9e8106f10 msgr2=0x7fc9e819cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:36.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.819+0000 7fc9ed4cf640 1 --2- 172.21.15.16:0/3207521152 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9e8106f10 0x7fc9e819cca0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fc9e0031a20 tx=0x7fc9e0031a50 comp rx=0 tx=0).stop 2024-09-12T02:00:36.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.819+0000 7fc9e77fe640 1 -- 172.21.15.16:0/3207521152 reap_dead start 2024-09-12T02:00:36.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.819+0000 7fc9ed4cf640 1 -- 172.21.15.16:0/3207521152 shutdown_connections 2024-09-12T02:00:36.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.819+0000 7fc9ed4cf640 1 -- 172.21.15.16:0/3207521152 >> 172.21.15.16:0/3207521152 conn(0x7fc9e80fd8c0 msgr2=0x7fc9e80fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:00:36.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.819+0000 7fc9ed4cf640 1 -- 172.21.15.16:0/3207521152 shutdown_connections 2024-09-12T02:00:36.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.819+0000 7fc9ed4cf640 1 -- 172.21.15.16:0/3207521152 wait complete. 2024-09-12T02:00:37.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.987+0000 7fa8e54c5640 1 -- 172.21.15.16:0/3615393330 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8e0102d90 msgr2=0x7fa8e0103170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:37.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.987+0000 7fa8e54c5640 1 --2- 172.21.15.16:0/3615393330 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8e0102d90 0x7fa8e0103170 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fa8d0009930 tx=0x7fa8d002a420 comp rx=0 tx=0).stop 2024-09-12T02:00:37.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.987+0000 7fa8e54c5640 1 -- 172.21.15.16:0/3615393330 shutdown_connections 2024-09-12T02:00:37.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.987+0000 7fa8e54c5640 1 --2- 172.21.15.16:0/3615393330 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa8e0108290 0x7fa8e010ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:37.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.987+0000 7fa8e54c5640 1 --2- 172.21.15.16:0/3615393330 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8e0103740 0x7fa8e0107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:37.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.987+0000 7fa8e54c5640 1 --2- 172.21.15.16:0/3615393330 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8e0102d90 0x7fa8e0103170 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:37.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.987+0000 7fa8e54c5640 1 -- 172.21.15.16:0/3615393330 >> 172.21.15.16:0/3615393330 conn(0x7fa8e00fc7d0 msgr2=0x7fa8e00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:00:37.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.987+0000 7fa8e54c5640 1 -- 172.21.15.16:0/3615393330 shutdown_connections 2024-09-12T02:00:37.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.988+0000 7fa8e54c5640 1 -- 172.21.15.16:0/3615393330 wait complete. 2024-09-12T02:00:37.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.988+0000 7fa8e54c5640 1 Processor -- start 2024-09-12T02:00:37.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.988+0000 7fa8e54c5640 1 -- start start 2024-09-12T02:00:37.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.989+0000 7fa8e54c5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8e0102d90 0x7fa8e01985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:37.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.989+0000 7fa8e54c5640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa8e0103740 0x7fa8e0198b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:37.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.989+0000 7fa8e54c5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8e0108290 0x7fa8e019cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:37.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.989+0000 7fa8e54c5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa8e0112090 con 0x7fa8e0108290 2024-09-12T02:00:37.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.989+0000 7fa8e54c5640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa8e0111f10 con 0x7fa8e0102d90 2024-09-12T02:00:37.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.989+0000 7fa8e54c5640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa8e0112210 con 0x7fa8e0103740 2024-09-12T02:00:37.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.989+0000 7fa8df7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8e0108290 0x7fa8e019cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:37.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.989+0000 7fa8df7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8e0108290 0x7fa8e019cec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38720/0 (socket says 172.21.15.16:38720) 2024-09-12T02:00:37.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.989+0000 7fa8df7fe640 1 -- 172.21.15.16:0/1379423799 learned_addr learned my addr 172.21.15.16:0/1379423799 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:00:37.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.989+0000 7fa8de7fc640 1 --2- 172.21.15.16:0/1379423799 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa8e0103740 0x7fa8e0198b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:37.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.989+0000 7fa8deffd640 1 --2- 172.21.15.16:0/1379423799 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8e0102d90 0x7fa8e01985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:37.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.989+0000 7fa8df7fe640 1 -- 172.21.15.16:0/1379423799 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa8e0103740 msgr2=0x7fa8e0198b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:37.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.989+0000 7fa8df7fe640 1 --2- 172.21.15.16:0/1379423799 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa8e0103740 0x7fa8e0198b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:37.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.989+0000 7fa8df7fe640 1 -- 172.21.15.16:0/1379423799 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8e0102d90 msgr2=0x7fa8e01985f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:37.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.989+0000 7fa8df7fe640 1 --2- 172.21.15.16:0/1379423799 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8e0102d90 0x7fa8e01985f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:37.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.989+0000 7fa8df7fe640 1 -- 172.21.15.16:0/1379423799 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8e019d5a0 con 0x7fa8e0108290 2024-09-12T02:00:37.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.989+0000 7fa8df7fe640 1 --2- 172.21.15.16:0/1379423799 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8e0108290 0x7fa8e019cec0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fa8d800eaa0 tx=0x7fa8d800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:00:37.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.989+0000 7fa8c7fff640 1 -- 172.21.15.16:0/1379423799 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8d800cdb0 con 0x7fa8e0108290 2024-09-12T02:00:37.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.989+0000 7fa8e54c5640 1 -- 172.21.15.16:0/1379423799 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8e019d890 con 0x7fa8e0108290 2024-09-12T02:00:37.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.990+0000 7fa8c7fff640 1 -- 172.21.15.16:0/1379423799 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8d8005020 con 0x7fa8e0108290 2024-09-12T02:00:37.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.990+0000 7fa8c7fff640 1 -- 172.21.15.16:0/1379423799 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8d80116d0 con 0x7fa8e0108290 2024-09-12T02:00:37.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.990+0000 7fa8e54c5640 1 -- 172.21.15.16:0/1379423799 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8e01a5170 con 0x7fa8e0108290 2024-09-12T02:00:37.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.990+0000 7fa8e54c5640 1 -- 172.21.15.16:0/1379423799 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa8e0062980 con 0x7fa8e0108290 2024-09-12T02:00:37.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.992+0000 7fa8c7fff640 1 -- 172.21.15.16:0/1379423799 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fa8d80054d0 con 0x7fa8e0108290 2024-09-12T02:00:37.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.992+0000 7fa8c7fff640 1 --2- 172.21.15.16:0/1379423799 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fa8c80a2c90 0x7fa8c80a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:37.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.992+0000 7fa8deffd640 1 --2- 172.21.15.16:0/1379423799 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fa8c80a2c90 0x7fa8c80a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:37.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.992+0000 7fa8c7fff640 1 -- 172.21.15.16:0/1379423799 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fa8d80cf5e0 con 0x7fa8e0108290 2024-09-12T02:00:37.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.993+0000 7fa8c7fff640 1 --2- 172.21.15.16:0/1379423799 >> [v2:172.21.15.184:6824/1619871346,v1:172.21.15.184:6825/1619871346] conn(0x7fa8c80ac740 0x7fa8c80aebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:37.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.993+0000 7fa8c7fff640 1 -- 172.21.15.16:0/1379423799 --> [v2:172.21.15.184:6824/1619871346,v1:172.21.15.184:6825/1619871346] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa8d8014ec0 con 0x7fa8c80ac740 2024-09-12T02:00:37.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.993+0000 7fa8c7fff640 1 -- 172.21.15.16:0/1379423799 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7fa8d80cfac0 con 0x7fa8e0108290 2024-09-12T02:00:37.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.993+0000 7fa8deffd640 1 --2- 172.21.15.16:0/1379423799 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fa8c80a2c90 0x7fa8c80a5150 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fa8d0000c00 tx=0x7fa8d0005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:00:37.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.993+0000 7fa8de7fc640 1 --2- 172.21.15.16:0/1379423799 >> [v2:172.21.15.184:6824/1619871346,v1:172.21.15.184:6825/1619871346] conn(0x7fa8c80ac740 0x7fa8c80aebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:37.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.993+0000 7fa8de7fc640 1 --2- 172.21.15.16:0/1379423799 >> [v2:172.21.15.184:6824/1619871346,v1:172.21.15.184:6825/1619871346] conn(0x7fa8c80ac740 0x7fa8c80aebe0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:00:37.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.994+0000 7fa8c7fff640 1 -- 172.21.15.16:0/1379423799 <== osd.11 v2:172.21.15.184:6824/1619871346 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa8d8014ec0 con 0x7fa8c80ac740 2024-09-12T02:00:37.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:37.008+0000 7fa8e54c5640 1 -- 172.21.15.16:0/1379423799 --> [v2:172.21.15.184:6824/1619871346,v1:172.21.15.184:6825/1619871346] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa8e0062d80 con 0x7fa8c80ac740 2024-09-12T02:00:37.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:37.009+0000 7fa8c7fff640 1 -- 172.21.15.16:0/1379423799 <== osd.11 v2:172.21.15.184:6824/1619871346 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fa8e0062d80 con 0x7fa8c80ac740 2024-09-12T02:00:37.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:37.009+0000 7fa8e54c5640 1 -- 172.21.15.16:0/1379423799 >> [v2:172.21.15.184:6824/1619871346,v1:172.21.15.184:6825/1619871346] conn(0x7fa8c80ac740 msgr2=0x7fa8c80aebe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:37.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:37.009+0000 7fa8e54c5640 1 --2- 172.21.15.16:0/1379423799 >> [v2:172.21.15.184:6824/1619871346,v1:172.21.15.184:6825/1619871346] conn(0x7fa8c80ac740 0x7fa8c80aebe0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:37.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:37.009+0000 7fa8e54c5640 1 -- 172.21.15.16:0/1379423799 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fa8c80a2c90 msgr2=0x7fa8c80a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:37.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:37.009+0000 7fa8e54c5640 1 --2- 172.21.15.16:0/1379423799 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fa8c80a2c90 0x7fa8c80a5150 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fa8d0000c00 tx=0x7fa8d0005a20 comp rx=0 tx=0).stop 2024-09-12T02:00:37.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:37.009+0000 7fa8e54c5640 1 -- 172.21.15.16:0/1379423799 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8e0108290 msgr2=0x7fa8e019cec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:37.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:37.009+0000 7fa8e54c5640 1 --2- 172.21.15.16:0/1379423799 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8e0108290 0x7fa8e019cec0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fa8d800eaa0 tx=0x7fa8d800ef60 comp rx=0 tx=0).stop 2024-09-12T02:00:37.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:37.009+0000 7fa8df7fe640 1 -- 172.21.15.16:0/1379423799 reap_dead start 2024-09-12T02:00:37.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:37.009+0000 7fa8e54c5640 1 -- 172.21.15.16:0/1379423799 shutdown_connections 2024-09-12T02:00:37.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:37.009+0000 7fa8e54c5640 1 -- 172.21.15.16:0/1379423799 >> 172.21.15.16:0/1379423799 conn(0x7fa8e00fc7d0 msgr2=0x7fa8e0106370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:00:37.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:37.009+0000 7fa8e54c5640 1 -- 172.21.15.16:0/1379423799 shutdown_connections 2024-09-12T02:00:37.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:37.009+0000 7fa8e54c5640 1 -- 172.21.15.16:0/1379423799 wait complete. 2024-09-12T02:00:38.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.704+0000 7f0d50219640 1 -- 172.21.15.16:0/1671499473 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d48101280 msgr2=0x7f0d4810df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:38.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.704+0000 7f0d50219640 1 --2- 172.21.15.16:0/1671499473 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d48101280 0x7f0d4810df40 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f0d38009900 tx=0x7f0d3802a410 comp rx=0 tx=0).stop 2024-09-12T02:00:38.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.704+0000 7f0d50219640 1 -- 172.21.15.16:0/1671499473 shutdown_connections 2024-09-12T02:00:38.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.704+0000 7f0d50219640 1 --2- 172.21.15.16:0/1671499473 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0d4810e480 0x7f0d48110870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:38.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.704+0000 7f0d50219640 1 --2- 172.21.15.16:0/1671499473 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d48101280 0x7f0d4810df40 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:38.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.704+0000 7f0d50219640 1 --2- 172.21.15.16:0/1671499473 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0d481008d0 0x7f0d48100cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:38.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.704+0000 7f0d50219640 1 -- 172.21.15.16:0/1671499473 >> 172.21.15.16:0/1671499473 conn(0x7f0d480fc7a0 msgr2=0x7f0d480febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:00:38.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.704+0000 7f0d50219640 1 -- 172.21.15.16:0/1671499473 shutdown_connections 2024-09-12T02:00:38.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.704+0000 7f0d50219640 1 -- 172.21.15.16:0/1671499473 wait complete. 2024-09-12T02:00:38.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.705+0000 7f0d50219640 1 Processor -- start 2024-09-12T02:00:38.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.705+0000 7f0d50219640 1 -- start start 2024-09-12T02:00:38.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.705+0000 7f0d50219640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0d481008d0 0x7f0d4819a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:38.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.705+0000 7f0d50219640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0d48101280 0x7f0d4819aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:38.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.705+0000 7f0d50219640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d4810e480 0x7f0d4819ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:38.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.705+0000 7f0d50219640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0d481141a0 con 0x7f0d4810e480 2024-09-12T02:00:38.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.705+0000 7f0d50219640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0d48114020 con 0x7f0d481008d0 2024-09-12T02:00:38.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.705+0000 7f0d50219640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f0d48114320 con 0x7f0d48101280 2024-09-12T02:00:38.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.705+0000 7f0d4e7b6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d4810e480 0x7f0d4819ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:38.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.705+0000 7f0d4e7b6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d4810e480 0x7f0d4819ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38674/0 (socket says 172.21.15.16:38674) 2024-09-12T02:00:38.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.705+0000 7f0d4e7b6640 1 -- 172.21.15.16:0/2719828276 learned_addr learned my addr 172.21.15.16:0/2719828276 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:00:38.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.705+0000 7f0d4dfb5640 1 --2- 172.21.15.16:0/2719828276 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0d481008d0 0x7f0d4819a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:38.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.705+0000 7f0d4d7b4640 1 --2- 172.21.15.16:0/2719828276 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0d48101280 0x7f0d4819aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:38.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.706+0000 7f0d4e7b6640 1 -- 172.21.15.16:0/2719828276 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0d48101280 msgr2=0x7f0d4819aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:38.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.706+0000 7f0d4e7b6640 1 --2- 172.21.15.16:0/2719828276 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0d48101280 0x7f0d4819aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:38.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.706+0000 7f0d4e7b6640 1 -- 172.21.15.16:0/2719828276 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0d481008d0 msgr2=0x7f0d4819a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:38.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.706+0000 7f0d4e7b6640 1 --2- 172.21.15.16:0/2719828276 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0d481008d0 0x7f0d4819a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:38.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.706+0000 7f0d4e7b6640 1 -- 172.21.15.16:0/2719828276 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d4819f610 con 0x7f0d4810e480 2024-09-12T02:00:38.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.706+0000 7f0d4e7b6640 1 --2- 172.21.15.16:0/2719828276 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d4810e480 0x7f0d4819ef30 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f0d40009fd0 tx=0x7f0d4000eda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:00:38.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.706+0000 7f0d3effd640 1 -- 172.21.15.16:0/2719828276 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d40004d20 con 0x7f0d4810e480 2024-09-12T02:00:38.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.706+0000 7f0d3effd640 1 -- 172.21.15.16:0/2719828276 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d40005020 con 0x7f0d4810e480 2024-09-12T02:00:38.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.706+0000 7f0d50219640 1 -- 172.21.15.16:0/2719828276 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d4819f900 con 0x7f0d4810e480 2024-09-12T02:00:38.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.706+0000 7f0d3effd640 1 -- 172.21.15.16:0/2719828276 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d400116d0 con 0x7f0d4810e480 2024-09-12T02:00:38.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.706+0000 7f0d50219640 1 -- 172.21.15.16:0/2719828276 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d481a71e0 con 0x7f0d4810e480 2024-09-12T02:00:38.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.707+0000 7f0d50219640 1 -- 172.21.15.16:0/2719828276 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0d48062900 con 0x7f0d4810e480 2024-09-12T02:00:38.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.708+0000 7f0d3effd640 1 -- 172.21.15.16:0/2719828276 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f0d400051d0 con 0x7f0d4810e480 2024-09-12T02:00:38.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.709+0000 7f0d3effd640 1 --2- 172.21.15.16:0/2719828276 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f0d340a2c90 0x7f0d340a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:38.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.709+0000 7f0d4dfb5640 1 --2- 172.21.15.16:0/2719828276 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f0d340a2c90 0x7f0d340a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:38.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.709+0000 7f0d3effd640 1 -- 172.21.15.16:0/2719828276 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f0d400cf770 con 0x7f0d4810e480 2024-09-12T02:00:38.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.709+0000 7f0d3effd640 1 --2- 172.21.15.16:0/2719828276 >> [v2:172.21.15.33:6808/1355184322,v1:172.21.15.33:6809/1355184322] conn(0x7f0d340ac740 0x7f0d340aebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:38.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.709+0000 7f0d3effd640 1 -- 172.21.15.16:0/2719828276 --> [v2:172.21.15.33:6808/1355184322,v1:172.21.15.33:6809/1355184322] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0d40019e70 con 0x7f0d340ac740 2024-09-12T02:00:38.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.709+0000 7f0d3effd640 1 -- 172.21.15.16:0/2719828276 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f0d400cfc50 con 0x7f0d4810e480 2024-09-12T02:00:38.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.710+0000 7f0d4dfb5640 1 --2- 172.21.15.16:0/2719828276 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f0d340a2c90 0x7f0d340a5150 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f0d4400abe0 tx=0x7f0d4400a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:00:38.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.710+0000 7f0d4d7b4640 1 --2- 172.21.15.16:0/2719828276 >> [v2:172.21.15.33:6808/1355184322,v1:172.21.15.33:6809/1355184322] conn(0x7f0d340ac740 0x7f0d340aebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:38.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.710+0000 7f0d4d7b4640 1 --2- 172.21.15.16:0/2719828276 >> [v2:172.21.15.33:6808/1355184322,v1:172.21.15.33:6809/1355184322] conn(0x7f0d340ac740 0x7f0d340aebe0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:00:38.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.711+0000 7f0d3effd640 1 -- 172.21.15.16:0/2719828276 <== osd.4 v2:172.21.15.33:6808/1355184322 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0d40019e70 con 0x7f0d340ac740 2024-09-12T02:00:38.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.726+0000 7f0d50219640 1 -- 172.21.15.16:0/2719828276 --> [v2:172.21.15.33:6808/1355184322,v1:172.21.15.33:6809/1355184322] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0d48062d80 con 0x7f0d340ac740 2024-09-12T02:00:38.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.726+0000 7f0d3effd640 1 -- 172.21.15.16:0/2719828276 <== osd.4 v2:172.21.15.33:6808/1355184322 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f0d48062d80 con 0x7f0d340ac740 2024-09-12T02:00:38.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.727+0000 7f0d50219640 1 -- 172.21.15.16:0/2719828276 >> [v2:172.21.15.33:6808/1355184322,v1:172.21.15.33:6809/1355184322] conn(0x7f0d340ac740 msgr2=0x7f0d340aebe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:38.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.727+0000 7f0d50219640 1 --2- 172.21.15.16:0/2719828276 >> [v2:172.21.15.33:6808/1355184322,v1:172.21.15.33:6809/1355184322] conn(0x7f0d340ac740 0x7f0d340aebe0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:38.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.727+0000 7f0d50219640 1 -- 172.21.15.16:0/2719828276 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f0d340a2c90 msgr2=0x7f0d340a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:38.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.727+0000 7f0d50219640 1 --2- 172.21.15.16:0/2719828276 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f0d340a2c90 0x7f0d340a5150 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f0d4400abe0 tx=0x7f0d4400a2d0 comp rx=0 tx=0).stop 2024-09-12T02:00:38.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.727+0000 7f0d50219640 1 -- 172.21.15.16:0/2719828276 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d4810e480 msgr2=0x7f0d4819ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:38.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.727+0000 7f0d50219640 1 --2- 172.21.15.16:0/2719828276 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d4810e480 0x7f0d4819ef30 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f0d40009fd0 tx=0x7f0d4000eda0 comp rx=0 tx=0).stop 2024-09-12T02:00:38.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.727+0000 7f0d4e7b6640 1 -- 172.21.15.16:0/2719828276 reap_dead start 2024-09-12T02:00:38.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.727+0000 7f0d50219640 1 -- 172.21.15.16:0/2719828276 shutdown_connections 2024-09-12T02:00:38.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.727+0000 7f0d50219640 1 -- 172.21.15.16:0/2719828276 >> 172.21.15.16:0/2719828276 conn(0x7f0d480fc7a0 msgr2=0x7f0d480feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:00:38.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.727+0000 7f0d50219640 1 -- 172.21.15.16:0/2719828276 shutdown_connections 2024-09-12T02:00:38.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:36.727+0000 7f0d50219640 1 -- 172.21.15.16:0/2719828276 wait complete. 2024-09-12T02:00:38.318 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:00:38 smithi016 ceph-mon[30947]: pgmap v296: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 170 B/s wr, 6 op/s 2024-09-12T02:00:38.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:00:38 smithi184 ceph-mon[35593]: pgmap v296: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 170 B/s wr, 6 op/s 2024-09-12T02:00:38.418 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:00:38 smithi033 ceph-mon[35332]: pgmap v296: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 170 B/s wr, 6 op/s 2024-09-12T02:00:38.865 INFO:teuthology.orchestra.run.smithi016.stdout:55834574918 2024-09-12T02:00:38.865 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-09-12T02:00:39.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:35.506+0000 7fc95b008640 1 -- 172.21.15.16:0/1754293997 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc954101df0 msgr2=0x7fc9541021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:39.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:35.506+0000 7fc95b008640 1 --2- 172.21.15.16:0/1754293997 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc954101df0 0x7fc9541021d0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fc948009900 tx=0x7fc94802a410 comp rx=0 tx=0).stop 2024-09-12T02:00:39.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:35.506+0000 7fc95b008640 1 -- 172.21.15.16:0/1754293997 shutdown_connections 2024-09-12T02:00:39.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:35.506+0000 7fc95b008640 1 --2- 172.21.15.16:0/1754293997 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc954106f20 0x7fc95410eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:39.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:35.506+0000 7fc95b008640 1 --2- 172.21.15.16:0/1754293997 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc9541027a0 0x7fc9541067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:39.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:35.506+0000 7fc95b008640 1 --2- 172.21.15.16:0/1754293997 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc954101df0 0x7fc9541021d0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:39.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:35.506+0000 7fc95b008640 1 -- 172.21.15.16:0/1754293997 >> 172.21.15.16:0/1754293997 conn(0x7fc9540fd8f0 msgr2=0x7fc9540ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:00:39.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:35.506+0000 7fc95b008640 1 -- 172.21.15.16:0/1754293997 shutdown_connections 2024-09-12T02:00:39.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:35.506+0000 7fc95b008640 1 -- 172.21.15.16:0/1754293997 wait complete. 2024-09-12T02:00:39.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:35.506+0000 7fc95b008640 1 Processor -- start 2024-09-12T02:00:39.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:35.507+0000 7fc95b008640 1 -- start start 2024-09-12T02:00:39.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:35.507+0000 7fc95b008640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc954101df0 0x7fc954198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:39.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:35.507+0000 7fc95b008640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc9541027a0 0x7fc954198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:39.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:35.507+0000 7fc95b008640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc954106f20 0x7fc95419cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:39.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:35.507+0000 7fc95b008640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc954111f80 con 0x7fc954106f20 2024-09-12T02:00:39.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:35.507+0000 7fc95b008640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc954111e00 con 0x7fc9541027a0 2024-09-12T02:00:39.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:35.507+0000 7fc95b008640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc954112100 con 0x7fc954101df0 2024-09-12T02:00:39.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:35.507+0000 7fc958da4640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc954101df0 0x7fc954198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:39.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:35.507+0000 7fc9595a5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc954106f20 0x7fc95419cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:39.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:35.507+0000 7fc958da4640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc954101df0 0x7fc954198440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:53778/0 (socket says 172.21.15.16:53778) 2024-09-12T02:00:39.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:35.507+0000 7fc958da4640 1 -- 172.21.15.16:0/155461361 learned_addr learned my addr 172.21.15.16:0/155461361 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:00:39.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:35.507+0000 7fc953fff640 1 --2- 172.21.15.16:0/155461361 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc9541027a0 0x7fc954198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:39.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:35.507+0000 7fc9595a5640 1 -- 172.21.15.16:0/155461361 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc954101df0 msgr2=0x7fc954198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:39.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:35.507+0000 7fc9595a5640 1 --2- 172.21.15.16:0/155461361 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc954101df0 0x7fc954198440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:39.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:35.507+0000 7fc9595a5640 1 -- 172.21.15.16:0/155461361 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc9541027a0 msgr2=0x7fc954198980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:39.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:35.507+0000 7fc9595a5640 1 --2- 172.21.15.16:0/155461361 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc9541027a0 0x7fc954198980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:39.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:35.507+0000 7fc9595a5640 1 -- 172.21.15.16:0/155461361 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc95419d3f0 con 0x7fc954106f20 2024-09-12T02:00:39.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:35.508+0000 7fc9595a5640 1 --2- 172.21.15.16:0/155461361 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc954106f20 0x7fc95419cd10 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fc94c00da10 tx=0x7fc94c00ded0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:00:39.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:35.508+0000 7fc951ffb640 1 -- 172.21.15.16:0/155461361 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc94c00bd70 con 0x7fc954106f20 2024-09-12T02:00:39.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:35.508+0000 7fc951ffb640 1 -- 172.21.15.16:0/155461361 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc94c003e50 con 0x7fc954106f20 2024-09-12T02:00:39.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:35.508+0000 7fc951ffb640 1 -- 172.21.15.16:0/155461361 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc94c0106a0 con 0x7fc954106f20 2024-09-12T02:00:39.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:35.508+0000 7fc95b008640 1 -- 172.21.15.16:0/155461361 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc95419d6e0 con 0x7fc954106f20 2024-09-12T02:00:39.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:35.508+0000 7fc95b008640 1 -- 172.21.15.16:0/155461361 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc954074420 con 0x7fc954106f20 2024-09-12T02:00:39.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:35.509+0000 7fc95b008640 1 -- 172.21.15.16:0/155461361 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc954062980 con 0x7fc954106f20 2024-09-12T02:00:39.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:35.511+0000 7fc951ffb640 1 -- 172.21.15.16:0/155461361 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fc94c003970 con 0x7fc954106f20 2024-09-12T02:00:39.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:35.512+0000 7fc951ffb640 1 --2- 172.21.15.16:0/155461361 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc9440a70a0 0x7fc9440a9560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:39.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:35.512+0000 7fc958da4640 1 --2- 172.21.15.16:0/155461361 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc9440a70a0 0x7fc9440a9560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:39.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:35.512+0000 7fc951ffb640 1 -- 172.21.15.16:0/155461361 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fc94c0cf100 con 0x7fc954106f20 2024-09-12T02:00:39.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:35.512+0000 7fc951ffb640 1 --2- 172.21.15.16:0/155461361 >> [v2:172.21.15.16:6818/902466142,v1:172.21.15.16:6819/902466142] conn(0x7fc9440b0b50 0x7fc9440b2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:39.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:35.512+0000 7fc951ffb640 1 -- 172.21.15.16:0/155461361 --> [v2:172.21.15.16:6818/902466142,v1:172.21.15.16:6819/902466142] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc94c014ec0 con 0x7fc9440b0b50 2024-09-12T02:00:39.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:35.512+0000 7fc951ffb640 1 -- 172.21.15.16:0/155461361 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7fc94c0cf4b0 con 0x7fc954106f20 2024-09-12T02:00:39.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:35.512+0000 7fc953fff640 1 --2- 172.21.15.16:0/155461361 >> [v2:172.21.15.16:6818/902466142,v1:172.21.15.16:6819/902466142] conn(0x7fc9440b0b50 0x7fc9440b2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:39.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:35.512+0000 7fc958da4640 1 --2- 172.21.15.16:0/155461361 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc9440a70a0 0x7fc9440a9560 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fc9480096c0 tx=0x7fc948005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:00:39.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:35.513+0000 7fc953fff640 1 --2- 172.21.15.16:0/155461361 >> [v2:172.21.15.16:6818/902466142,v1:172.21.15.16:6819/902466142] conn(0x7fc9440b0b50 0x7fc9440b2ff0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:00:39.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:35.513+0000 7fc951ffb640 1 -- 172.21.15.16:0/155461361 <== osd.6 v2:172.21.15.16:6818/902466142 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc94c014ec0 con 0x7fc9440b0b50 2024-09-12T02:00:39.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:35.527+0000 7fc95b008640 1 -- 172.21.15.16:0/155461361 --> [v2:172.21.15.16:6818/902466142,v1:172.21.15.16:6819/902466142] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc954062c10 con 0x7fc9440b0b50 2024-09-12T02:00:39.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:35.528+0000 7fc951ffb640 1 -- 172.21.15.16:0/155461361 <== osd.6 v2:172.21.15.16:6818/902466142 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fc954062c10 con 0x7fc9440b0b50 2024-09-12T02:00:39.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:35.528+0000 7fc95b008640 1 -- 172.21.15.16:0/155461361 >> [v2:172.21.15.16:6818/902466142,v1:172.21.15.16:6819/902466142] conn(0x7fc9440b0b50 msgr2=0x7fc9440b2ff0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:39.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:35.528+0000 7fc95b008640 1 --2- 172.21.15.16:0/155461361 >> [v2:172.21.15.16:6818/902466142,v1:172.21.15.16:6819/902466142] conn(0x7fc9440b0b50 0x7fc9440b2ff0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:39.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:35.528+0000 7fc95b008640 1 -- 172.21.15.16:0/155461361 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc9440a70a0 msgr2=0x7fc9440a9560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:39.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:35.528+0000 7fc95b008640 1 --2- 172.21.15.16:0/155461361 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc9440a70a0 0x7fc9440a9560 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fc9480096c0 tx=0x7fc948005a20 comp rx=0 tx=0).stop 2024-09-12T02:00:39.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:35.528+0000 7fc95b008640 1 -- 172.21.15.16:0/155461361 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc954106f20 msgr2=0x7fc95419cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:39.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:35.528+0000 7fc95b008640 1 --2- 172.21.15.16:0/155461361 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc954106f20 0x7fc95419cd10 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fc94c00da10 tx=0x7fc94c00ded0 comp rx=0 tx=0).stop 2024-09-12T02:00:39.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:35.528+0000 7fc9595a5640 1 -- 172.21.15.16:0/155461361 reap_dead start 2024-09-12T02:00:39.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:35.528+0000 7fc95b008640 1 -- 172.21.15.16:0/155461361 shutdown_connections 2024-09-12T02:00:39.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:35.528+0000 7fc95b008640 1 -- 172.21.15.16:0/155461361 >> 172.21.15.16:0/155461361 conn(0x7fc9540fd8f0 msgr2=0x7fc9540fed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:00:39.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:35.528+0000 7fc95b008640 1 -- 172.21.15.16:0/155461361 shutdown_connections 2024-09-12T02:00:39.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:35.528+0000 7fc95b008640 1 -- 172.21.15.16:0/155461361 wait complete. 2024-09-12T02:00:39.731 INFO:teuthology.orchestra.run.smithi016.stdout:184683593774 2024-09-12T02:00:39.731 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-09-12T02:00:40.126 INFO:teuthology.orchestra.run.smithi016.stdout:317827579925 2024-09-12T02:00:40.127 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-09-12T02:00:40.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:00:40 smithi184 ceph-mon[35593]: pgmap v297: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-09-12T02:00:40.418 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:00:40 smithi033 ceph-mon[35332]: pgmap v297: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-09-12T02:00:40.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:00:40 smithi016 ceph-mon[30947]: pgmap v297: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-09-12T02:00:40.682 INFO:teuthology.orchestra.run.smithi016.stdout:206158430249 2024-09-12T02:00:40.682 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-09-12T02:00:40.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:40.867+0000 7f3455e88640 1 -- 172.21.15.16:0/2590378696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3450101de0 msgr2=0x7f34501021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:40.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:40.867+0000 7f3455e88640 1 --2- 172.21.15.16:0/2590378696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3450101de0 0x7f34501021c0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f3440009900 tx=0x7f344002a410 comp rx=0 tx=0).stop 2024-09-12T02:00:40.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:40.867+0000 7f3455e88640 1 -- 172.21.15.16:0/2590378696 shutdown_connections 2024-09-12T02:00:40.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:40.867+0000 7f3455e88640 1 --2- 172.21.15.16:0/2590378696 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3450106f10 0x7f345010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:40.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:40.867+0000 7f3455e88640 1 --2- 172.21.15.16:0/2590378696 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3450102790 0x7f34501067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:40.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:40.867+0000 7f3455e88640 1 --2- 172.21.15.16:0/2590378696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3450101de0 0x7f34501021c0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:40.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:40.867+0000 7f3455e88640 1 -- 172.21.15.16:0/2590378696 >> 172.21.15.16:0/2590378696 conn(0x7f34500fd8c0 msgr2=0x7f34500ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:00:40.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:40.867+0000 7f3455e88640 1 -- 172.21.15.16:0/2590378696 shutdown_connections 2024-09-12T02:00:40.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:40.867+0000 7f3455e88640 1 -- 172.21.15.16:0/2590378696 wait complete. 2024-09-12T02:00:40.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:40.868+0000 7f3455e88640 1 Processor -- start 2024-09-12T02:00:40.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:40.868+0000 7f3455e88640 1 -- start start 2024-09-12T02:00:40.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:40.868+0000 7f3455e88640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3450101de0 0x7f345019a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:40.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:40.868+0000 7f3455e88640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3450102790 0x7f345019abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:40.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:40.868+0000 7f3455e88640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3450106f10 0x7f345019ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:40.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:40.868+0000 7f3455e88640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3450111f90 con 0x7f3450106f10 2024-09-12T02:00:40.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:40.868+0000 7f3455e88640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3450111e10 con 0x7f3450102790 2024-09-12T02:00:40.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:40.868+0000 7f3455e88640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f3450112110 con 0x7f3450101de0 2024-09-12T02:00:40.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:40.868+0000 7f344ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3450106f10 0x7f345019ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:40.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:40.868+0000 7f344ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3450106f10 0x7f345019ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38748/0 (socket says 172.21.15.16:38748) 2024-09-12T02:00:40.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:40.868+0000 7f344ffff640 1 -- 172.21.15.16:0/3526547841 learned_addr learned my addr 172.21.15.16:0/3526547841 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:00:40.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:40.868+0000 7f3447fff640 1 --2- 172.21.15.16:0/3526547841 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3450102790 0x7f345019abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:40.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:40.868+0000 7f344f7fe640 1 --2- 172.21.15.16:0/3526547841 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3450101de0 0x7f345019a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:40.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:40.868+0000 7f344ffff640 1 -- 172.21.15.16:0/3526547841 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3450101de0 msgr2=0x7f345019a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:40.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:40.868+0000 7f344ffff640 1 --2- 172.21.15.16:0/3526547841 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3450101de0 0x7f345019a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:40.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:40.868+0000 7f344ffff640 1 -- 172.21.15.16:0/3526547841 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3450102790 msgr2=0x7f345019abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:40.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:40.868+0000 7f344ffff640 1 --2- 172.21.15.16:0/3526547841 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3450102790 0x7f345019abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:40.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:40.868+0000 7f344ffff640 1 -- 172.21.15.16:0/3526547841 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f345019f640 con 0x7f3450106f10 2024-09-12T02:00:40.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:40.869+0000 7f344ffff640 1 --2- 172.21.15.16:0/3526547841 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3450106f10 0x7f345019ef60 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f344800cce0 tx=0x7f344800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:00:40.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:40.869+0000 7f344d7fa640 1 -- 172.21.15.16:0/3526547841 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f344800f450 con 0x7f3450106f10 2024-09-12T02:00:40.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:40.869+0000 7f3455e88640 1 -- 172.21.15.16:0/3526547841 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f345019f930 con 0x7f3450106f10 2024-09-12T02:00:40.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:40.869+0000 7f344d7fa640 1 -- 172.21.15.16:0/3526547841 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f344800fad0 con 0x7f3450106f10 2024-09-12T02:00:40.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:40.869+0000 7f344d7fa640 1 -- 172.21.15.16:0/3526547841 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3448018b60 con 0x7f3450106f10 2024-09-12T02:00:40.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:40.869+0000 7f3455e88640 1 -- 172.21.15.16:0/3526547841 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34501a7210 con 0x7f3450106f10 2024-09-12T02:00:40.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:40.870+0000 7f3455e88640 1 -- 172.21.15.16:0/3526547841 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3450075700 con 0x7f3450106f10 2024-09-12T02:00:40.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:40.871+0000 7f344d7fa640 1 -- 172.21.15.16:0/3526547841 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f344800fc80 con 0x7f3450106f10 2024-09-12T02:00:40.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:40.872+0000 7f344d7fa640 1 --2- 172.21.15.16:0/3526547841 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f343c0a2c10 0x7f343c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:40.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:40.872+0000 7f344d7fa640 1 -- 172.21.15.16:0/3526547841 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f34480cfb20 con 0x7f3450106f10 2024-09-12T02:00:40.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:40.872+0000 7f344f7fe640 1 --2- 172.21.15.16:0/3526547841 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f343c0a2c10 0x7f343c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:40.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:40.872+0000 7f344d7fa640 1 --2- 172.21.15.16:0/3526547841 >> [v2:172.21.15.33:6824/469977847,v1:172.21.15.33:6825/469977847] conn(0x7f343c0ac6c0 0x7f343c0aeb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:40.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:40.872+0000 7f344d7fa640 1 -- 172.21.15.16:0/3526547841 --> [v2:172.21.15.33:6824/469977847,v1:172.21.15.33:6825/469977847] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3448012ec0 con 0x7f343c0ac6c0 2024-09-12T02:00:40.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:40.872+0000 7f344d7fa640 1 -- 172.21.15.16:0/3526547841 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f34480cff80 con 0x7f3450106f10 2024-09-12T02:00:40.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:40.873+0000 7f3447fff640 1 --2- 172.21.15.16:0/3526547841 >> [v2:172.21.15.33:6824/469977847,v1:172.21.15.33:6825/469977847] conn(0x7f343c0ac6c0 0x7f343c0aeb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:40.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:40.873+0000 7f344f7fe640 1 --2- 172.21.15.16:0/3526547841 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f343c0a2c10 0x7f343c0a50d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f3440000c00 tx=0x7f34400058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:00:40.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:40.873+0000 7f3447fff640 1 --2- 172.21.15.16:0/3526547841 >> [v2:172.21.15.33:6824/469977847,v1:172.21.15.33:6825/469977847] conn(0x7f343c0ac6c0 0x7f343c0aeb60 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:00:40.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:40.874+0000 7f344d7fa640 1 -- 172.21.15.16:0/3526547841 <== osd.10 v2:172.21.15.33:6824/469977847 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3448012ec0 con 0x7f343c0ac6c0 2024-09-12T02:00:40.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:40.888+0000 7f3455e88640 1 -- 172.21.15.16:0/3526547841 --> [v2:172.21.15.33:6824/469977847,v1:172.21.15.33:6825/469977847] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f34501032c0 con 0x7f343c0ac6c0 2024-09-12T02:00:40.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:40.889+0000 7f344d7fa640 1 -- 172.21.15.16:0/3526547841 <== osd.10 v2:172.21.15.33:6824/469977847 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f34501032c0 con 0x7f343c0ac6c0 2024-09-12T02:00:40.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:40.889+0000 7f3455e88640 1 -- 172.21.15.16:0/3526547841 >> [v2:172.21.15.33:6824/469977847,v1:172.21.15.33:6825/469977847] conn(0x7f343c0ac6c0 msgr2=0x7f343c0aeb60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:40.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:40.889+0000 7f3455e88640 1 --2- 172.21.15.16:0/3526547841 >> [v2:172.21.15.33:6824/469977847,v1:172.21.15.33:6825/469977847] conn(0x7f343c0ac6c0 0x7f343c0aeb60 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:40.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:40.889+0000 7f3455e88640 1 -- 172.21.15.16:0/3526547841 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f343c0a2c10 msgr2=0x7f343c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:40.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:40.889+0000 7f3455e88640 1 --2- 172.21.15.16:0/3526547841 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f343c0a2c10 0x7f343c0a50d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f3440000c00 tx=0x7f34400058d0 comp rx=0 tx=0).stop 2024-09-12T02:00:40.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:40.889+0000 7f3455e88640 1 -- 172.21.15.16:0/3526547841 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3450106f10 msgr2=0x7f345019ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:40.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:40.889+0000 7f3455e88640 1 --2- 172.21.15.16:0/3526547841 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3450106f10 0x7f345019ef60 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f344800cce0 tx=0x7f344800a620 comp rx=0 tx=0).stop 2024-09-12T02:00:40.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:40.889+0000 7f344ffff640 1 -- 172.21.15.16:0/3526547841 reap_dead start 2024-09-12T02:00:40.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:40.889+0000 7f3455e88640 1 -- 172.21.15.16:0/3526547841 shutdown_connections 2024-09-12T02:00:40.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:40.889+0000 7f3455e88640 1 -- 172.21.15.16:0/3526547841 >> 172.21.15.16:0/3526547841 conn(0x7f34500fd8c0 msgr2=0x7f34500feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:00:40.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:40.889+0000 7f3455e88640 1 -- 172.21.15.16:0/3526547841 shutdown_connections 2024-09-12T02:00:40.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:40.889+0000 7f3455e88640 1 -- 172.21.15.16:0/3526547841 wait complete. 2024-09-12T02:00:41.536 INFO:teuthology.orchestra.run.smithi016.stdout:287762808859 2024-09-12T02:00:41.537 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-09-12T02:00:41.888 INFO:teuthology.orchestra.run.smithi016.stdout:34359738443 2024-09-12T02:00:41.888 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-09-12T02:00:42.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:42.088+0000 7f6815f2c640 1 -- 172.21.15.16:0/2236250331 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6810100a80 msgr2=0x7f6810100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:42.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:42.088+0000 7f6815f2c640 1 --2- 172.21.15.16:0/2236250331 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6810100a80 0x7f6810100ee0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f67f8009930 tx=0x7f67f802a420 comp rx=0 tx=0).stop 2024-09-12T02:00:42.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:42.088+0000 7f6815f2c640 1 -- 172.21.15.16:0/2236250331 shutdown_connections 2024-09-12T02:00:42.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:42.088+0000 7f6815f2c640 1 --2- 172.21.15.16:0/2236250331 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6810101420 0x7f681010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:42.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:42.088+0000 7f6815f2c640 1 --2- 172.21.15.16:0/2236250331 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6810100a80 0x7f6810100ee0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:42.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:42.088+0000 7f6815f2c640 1 --2- 172.21.15.16:0/2236250331 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6810106a80 0x7f6810106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:42.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:42.088+0000 7f6815f2c640 1 -- 172.21.15.16:0/2236250331 >> 172.21.15.16:0/2236250331 conn(0x7f68100fc7a0 msgr2=0x7f68100febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:00:42.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:42.088+0000 7f6815f2c640 1 -- 172.21.15.16:0/2236250331 shutdown_connections 2024-09-12T02:00:42.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:42.088+0000 7f6815f2c640 1 -- 172.21.15.16:0/2236250331 wait complete. 2024-09-12T02:00:42.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:42.089+0000 7f6815f2c640 1 Processor -- start 2024-09-12T02:00:42.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:42.089+0000 7f6815f2c640 1 -- start start 2024-09-12T02:00:42.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:42.089+0000 7f6815f2c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6810100a80 0x7f6810198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:42.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:42.089+0000 7f6815f2c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6810101420 0x7f68101989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:42.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:42.089+0000 7f6815f2c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6810106a80 0x7f681019cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:42.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:42.089+0000 7f6815f2c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6810110ed0 con 0x7f6810101420 2024-09-12T02:00:42.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:42.089+0000 7f6815f2c640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6810110d50 con 0x7f6810100a80 2024-09-12T02:00:42.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:42.089+0000 7f6815f2c640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f6810111050 con 0x7f6810106a80 2024-09-12T02:00:42.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:42.089+0000 7f6806ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6810101420 0x7f68101989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:42.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:42.089+0000 7f6806ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6810101420 0x7f68101989d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51456/0 (socket says 172.21.15.16:51456) 2024-09-12T02:00:42.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:42.089+0000 7f6806ffd640 1 -- 172.21.15.16:0/1138004994 learned_addr learned my addr 172.21.15.16:0/1138004994 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:00:42.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:42.090+0000 7f680f7fe640 1 --2- 172.21.15.16:0/1138004994 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6810100a80 0x7f6810198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:42.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:42.090+0000 7f680ffff640 1 --2- 172.21.15.16:0/1138004994 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6810106a80 0x7f681019cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:42.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:42.090+0000 7f6806ffd640 1 -- 172.21.15.16:0/1138004994 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6810106a80 msgr2=0x7f681019cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:42.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:42.090+0000 7f6806ffd640 1 --2- 172.21.15.16:0/1138004994 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6810106a80 0x7f681019cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:42.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:42.090+0000 7f6806ffd640 1 -- 172.21.15.16:0/1138004994 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6810100a80 msgr2=0x7f6810198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:42.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:42.090+0000 7f6806ffd640 1 --2- 172.21.15.16:0/1138004994 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6810100a80 0x7f6810198490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:42.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:42.090+0000 7f6806ffd640 1 -- 172.21.15.16:0/1138004994 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f681019d440 con 0x7f6810101420 2024-09-12T02:00:42.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:42.090+0000 7f6806ffd640 1 --2- 172.21.15.16:0/1138004994 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6810101420 0x7f68101989d0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f67f802a940 tx=0x7f67f8003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:00:42.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:42.090+0000 7f680d7fa640 1 -- 172.21.15.16:0/1138004994 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67f803f070 con 0x7f6810101420 2024-09-12T02:00:42.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:42.090+0000 7f680d7fa640 1 -- 172.21.15.16:0/1138004994 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67f8031c90 con 0x7f6810101420 2024-09-12T02:00:42.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:42.090+0000 7f6815f2c640 1 -- 172.21.15.16:0/1138004994 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f681019d6d0 con 0x7f6810101420 2024-09-12T02:00:42.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:42.090+0000 7f680d7fa640 1 -- 172.21.15.16:0/1138004994 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67f8035030 con 0x7f6810101420 2024-09-12T02:00:42.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:42.090+0000 7f6815f2c640 1 -- 172.21.15.16:0/1138004994 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68101a5010 con 0x7f6810101420 2024-09-12T02:00:42.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:42.091+0000 7f6815f2c640 1 -- 172.21.15.16:0/1138004994 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6810062980 con 0x7f6810101420 2024-09-12T02:00:42.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:42.092+0000 7f680d7fa640 1 -- 172.21.15.16:0/1138004994 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f67f8042070 con 0x7f6810101420 2024-09-12T02:00:42.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:42.093+0000 7f680d7fa640 1 --2- 172.21.15.16:0/1138004994 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f67fc0a2c90 0x7f67fc0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:42.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:42.093+0000 7f680f7fe640 1 --2- 172.21.15.16:0/1138004994 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f67fc0a2c90 0x7f67fc0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:42.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:42.093+0000 7f680d7fa640 1 -- 172.21.15.16:0/1138004994 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f67f80efbf0 con 0x7f6810101420 2024-09-12T02:00:42.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:42.094+0000 7f680d7fa640 1 --2- 172.21.15.16:0/1138004994 >> [v2:172.21.15.16:6826/1569578746,v1:172.21.15.16:6827/1569578746] conn(0x7f67fc0ac740 0x7f67fc0aebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:42.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:42.094+0000 7f680d7fa640 1 -- 172.21.15.16:0/1138004994 --> [v2:172.21.15.16:6826/1569578746,v1:172.21.15.16:6827/1569578746] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f67f8038e90 con 0x7f67fc0ac740 2024-09-12T02:00:42.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:42.094+0000 7f680d7fa640 1 -- 172.21.15.16:0/1138004994 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f67f80effa0 con 0x7f6810101420 2024-09-12T02:00:42.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:42.094+0000 7f680ffff640 1 --2- 172.21.15.16:0/1138004994 >> [v2:172.21.15.16:6826/1569578746,v1:172.21.15.16:6827/1569578746] conn(0x7f67fc0ac740 0x7f67fc0aebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:42.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:42.094+0000 7f680f7fe640 1 --2- 172.21.15.16:0/1138004994 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f67fc0a2c90 0x7f67fc0a5150 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f680000bb60 tx=0x7f680000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:00:42.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:42.094+0000 7f680ffff640 1 --2- 172.21.15.16:0/1138004994 >> [v2:172.21.15.16:6826/1569578746,v1:172.21.15.16:6827/1569578746] conn(0x7f67fc0ac740 0x7f67fc0aebe0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:00:42.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:42.095+0000 7f680d7fa640 1 -- 172.21.15.16:0/1138004994 <== osd.9 v2:172.21.15.16:6826/1569578746 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f67f8038e90 con 0x7f67fc0ac740 2024-09-12T02:00:42.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:42.112+0000 7f6815f2c640 1 -- 172.21.15.16:0/1138004994 --> [v2:172.21.15.16:6826/1569578746,v1:172.21.15.16:6827/1569578746] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6810062d80 con 0x7f67fc0ac740 2024-09-12T02:00:42.112 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:00:42 smithi016 ceph-mon[30947]: pgmap v298: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 7 op/s 2024-09-12T02:00:42.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:42.112+0000 7f680d7fa640 1 -- 172.21.15.16:0/1138004994 <== osd.9 v2:172.21.15.16:6826/1569578746 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f6810062d80 con 0x7f67fc0ac740 2024-09-12T02:00:42.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:42.113+0000 7f6815f2c640 1 -- 172.21.15.16:0/1138004994 >> [v2:172.21.15.16:6826/1569578746,v1:172.21.15.16:6827/1569578746] conn(0x7f67fc0ac740 msgr2=0x7f67fc0aebe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:42.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:42.113+0000 7f6815f2c640 1 --2- 172.21.15.16:0/1138004994 >> [v2:172.21.15.16:6826/1569578746,v1:172.21.15.16:6827/1569578746] conn(0x7f67fc0ac740 0x7f67fc0aebe0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:42.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:42.113+0000 7f6815f2c640 1 -- 172.21.15.16:0/1138004994 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f67fc0a2c90 msgr2=0x7f67fc0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:42.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:42.113+0000 7f6815f2c640 1 --2- 172.21.15.16:0/1138004994 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f67fc0a2c90 0x7f67fc0a5150 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f680000bb60 tx=0x7f680000b390 comp rx=0 tx=0).stop 2024-09-12T02:00:42.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:42.113+0000 7f6815f2c640 1 -- 172.21.15.16:0/1138004994 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6810101420 msgr2=0x7f68101989d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:42.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:42.113+0000 7f6815f2c640 1 --2- 172.21.15.16:0/1138004994 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6810101420 0x7f68101989d0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f67f802a940 tx=0x7f67f8003b90 comp rx=0 tx=0).stop 2024-09-12T02:00:42.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:42.113+0000 7f680ffff640 1 -- 172.21.15.16:0/1138004994 reap_dead start 2024-09-12T02:00:42.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:42.113+0000 7f6815f2c640 1 -- 172.21.15.16:0/1138004994 shutdown_connections 2024-09-12T02:00:42.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:42.113+0000 7f6815f2c640 1 -- 172.21.15.16:0/1138004994 >> 172.21.15.16:0/1138004994 conn(0x7f68100fc7a0 msgr2=0x7f681006aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:00:42.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:42.113+0000 7f6815f2c640 1 -- 172.21.15.16:0/1138004994 shutdown_connections 2024-09-12T02:00:42.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:42.113+0000 7f6815f2c640 1 -- 172.21.15.16:0/1138004994 wait complete. 2024-09-12T02:00:42.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:00:42 smithi184 ceph-mon[35593]: pgmap v298: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 7 op/s 2024-09-12T02:00:42.418 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:00:42 smithi033 ceph-mon[35332]: pgmap v298: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 7 op/s 2024-09-12T02:00:42.652 INFO:teuthology.orchestra.run.smithi016.stdout:103079215163 2024-09-12T02:00:42.652 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-09-12T02:00:43.340 INFO:teuthology.orchestra.run.smithi016.stdout:266287972384 2024-09-12T02:00:43.341 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-09-12T02:00:43.440 INFO:teuthology.orchestra.run.smithi016.stdout:154618822706 2024-09-12T02:00:43.441 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-09-12T02:00:43.827 INFO:teuthology.orchestra.run.smithi016.stdout:133143986231 2024-09-12T02:00:43.828 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-09-12T02:00:43.932 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:00:43.932 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:00:43.934 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:00:43.934 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:00:43.939 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:00:43.944 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:00:43.945 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:00:43.953 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:00:43.957 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:00:44.005 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:00:44.344 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:00:44 smithi016 ceph-mon[30947]: pgmap v299: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-09-12T02:00:44.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:00:44 smithi184 ceph-mon[35593]: pgmap v299: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-09-12T02:00:44.418 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:00:44 smithi033 ceph-mon[35332]: pgmap v299: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-09-12T02:00:44.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:44.643+0000 7f371b047640 1 -- 172.21.15.16:0/1715547 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3714102ef0 msgr2=0x7f371410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:44.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:44.643+0000 7f371b047640 1 --2- 172.21.15.16:0/1715547 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3714102ef0 0x7f371410da40 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f370c009ca0 tx=0x7f370c02a950 comp rx=0 tx=0).stop 2024-09-12T02:00:44.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:44.643+0000 7f371b047640 1 -- 172.21.15.16:0/1715547 shutdown_connections 2024-09-12T02:00:44.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:44.643+0000 7f371b047640 1 --2- 172.21.15.16:0/1715547 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3714102ef0 0x7f371410da40 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:44.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:44.643+0000 7f371b047640 1 --2- 172.21.15.16:0/1715547 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3714102570 0x7f37141029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:44.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:44.643+0000 7f371b047640 1 --2- 172.21.15.16:0/1715547 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3714100e20 0x7f3714101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:44.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:44.643+0000 7f371b047640 1 -- 172.21.15.16:0/1715547 >> 172.21.15.16:0/1715547 conn(0x7f37140fc7d0 msgr2=0x7f37140febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:00:44.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:44.643+0000 7f371b047640 1 -- 172.21.15.16:0/1715547 shutdown_connections 2024-09-12T02:00:44.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:44.643+0000 7f371b047640 1 -- 172.21.15.16:0/1715547 wait complete. 2024-09-12T02:00:44.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:44.644+0000 7f371b047640 1 Processor -- start 2024-09-12T02:00:44.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:44.644+0000 7f371b047640 1 -- start start 2024-09-12T02:00:44.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:44.644+0000 7f371b047640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3714100e20 0x7f37141a4850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:44.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:44.644+0000 7f371b047640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3714102570 0x7f37141a4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:44.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:44.644+0000 7f371b047640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3714102ef0 0x7f371419ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:44.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:44.644+0000 7f371b047640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3714110f80 con 0x7f3714102570 2024-09-12T02:00:44.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:44.644+0000 7f371b047640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3714110e00 con 0x7f3714100e20 2024-09-12T02:00:44.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:44.644+0000 7f371b047640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f3714111100 con 0x7f3714102ef0 2024-09-12T02:00:44.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:44.644+0000 7f3713fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3714102570 0x7f37141a4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:44.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:44.644+0000 7f3713fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3714102570 0x7f37141a4d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51480/0 (socket says 172.21.15.16:51480) 2024-09-12T02:00:44.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:44.644+0000 7f3713fff640 1 -- 172.21.15.16:0/4149937162 learned_addr learned my addr 172.21.15.16:0/4149937162 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:00:44.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:44.644+0000 7f3718de3640 1 --2- 172.21.15.16:0/4149937162 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3714100e20 0x7f37141a4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:44.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:44.645+0000 7f37195e4640 1 --2- 172.21.15.16:0/4149937162 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3714102ef0 0x7f371419ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:44.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:44.645+0000 7f3713fff640 1 -- 172.21.15.16:0/4149937162 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3714102ef0 msgr2=0x7f371419ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:44.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:44.645+0000 7f3713fff640 1 --2- 172.21.15.16:0/4149937162 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3714102ef0 0x7f371419ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:44.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:44.645+0000 7f3713fff640 1 -- 172.21.15.16:0/4149937162 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3714100e20 msgr2=0x7f37141a4850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:44.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:44.645+0000 7f3713fff640 1 --2- 172.21.15.16:0/4149937162 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3714100e20 0x7f37141a4850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:44.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:44.645+0000 7f3713fff640 1 -- 172.21.15.16:0/4149937162 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f371419f4c0 con 0x7f3714102570 2024-09-12T02:00:44.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:44.645+0000 7f37195e4640 1 --2- 172.21.15.16:0/4149937162 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3714102ef0 0x7f371419ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:00:44.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:44.645+0000 7f3713fff640 1 --2- 172.21.15.16:0/4149937162 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3714102570 0x7f37141a4d90 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f370000d950 tx=0x7f370000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:00:44.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:44.645+0000 7f3711ffb640 1 -- 172.21.15.16:0/4149937162 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3700017070 con 0x7f3714102570 2024-09-12T02:00:44.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:44.645+0000 7f3711ffb640 1 -- 172.21.15.16:0/4149937162 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3700009c30 con 0x7f3714102570 2024-09-12T02:00:44.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:44.645+0000 7f3711ffb640 1 -- 172.21.15.16:0/4149937162 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f370000f6b0 con 0x7f3714102570 2024-09-12T02:00:44.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:44.645+0000 7f371b047640 1 -- 172.21.15.16:0/4149937162 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f371419f7b0 con 0x7f3714102570 2024-09-12T02:00:44.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:44.645+0000 7f371b047640 1 -- 172.21.15.16:0/4149937162 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37141ab780 con 0x7f3714102570 2024-09-12T02:00:44.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:44.646+0000 7f371b047640 1 -- 172.21.15.16:0/4149937162 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37141021b0 con 0x7f3714102570 2024-09-12T02:00:44.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:44.648+0000 7f3711ffb640 1 -- 172.21.15.16:0/4149937162 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f37000109d0 con 0x7f3714102570 2024-09-12T02:00:44.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:44.648+0000 7f3711ffb640 1 --2- 172.21.15.16:0/4149937162 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f37040a2c90 0x7f37040a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:44.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:44.648+0000 7f3711ffb640 1 -- 172.21.15.16:0/4149937162 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f37000ced90 con 0x7f3714102570 2024-09-12T02:00:44.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:44.648+0000 7f3718de3640 1 --2- 172.21.15.16:0/4149937162 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f37040a2c90 0x7f37040a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:44.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:44.649+0000 7f3718de3640 1 --2- 172.21.15.16:0/4149937162 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f37040a2c90 0x7f37040a5150 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f3708005fd0 tx=0x7f370800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:00:44.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:44.652+0000 7f3711ffb640 1 -- 172.21.15.16:0/4149937162 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3700009de0 con 0x7f3714102570 2024-09-12T02:00:44.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:44.835+0000 7f371b047640 1 -- 172.21.15.16:0/4149937162 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f371419fe60 con 0x7f3714102570 2024-09-12T02:00:44.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:44.835+0000 7f3711ffb640 1 -- 172.21.15.16:0/4149937162 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f371419fe60 con 0x7f3714102570 2024-09-12T02:00:44.837 INFO:teuthology.orchestra.run.smithi016.stdout:34359738445 2024-09-12T02:00:44.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:44.837+0000 7f371b047640 1 -- 172.21.15.16:0/4149937162 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f37040a2c90 msgr2=0x7f37040a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:44.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:44.837+0000 7f371b047640 1 --2- 172.21.15.16:0/4149937162 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f37040a2c90 0x7f37040a5150 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f3708005fd0 tx=0x7f370800b390 comp rx=0 tx=0).stop 2024-09-12T02:00:44.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:44.837+0000 7f371b047640 1 -- 172.21.15.16:0/4149937162 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3714102570 msgr2=0x7f37141a4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:44.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:44.837+0000 7f371b047640 1 --2- 172.21.15.16:0/4149937162 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3714102570 0x7f37141a4d90 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f370000d950 tx=0x7f370000de10 comp rx=0 tx=0).stop 2024-09-12T02:00:44.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:44.837+0000 7f371b047640 1 -- 172.21.15.16:0/4149937162 shutdown_connections 2024-09-12T02:00:44.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:44.838+0000 7f371b047640 1 --2- 172.21.15.16:0/4149937162 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f37040a2c90 0x7f37040a5150 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:44.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:44.838+0000 7f371b047640 1 --2- 172.21.15.16:0/4149937162 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3714102ef0 0x7f371419ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:44.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:44.838+0000 7f371b047640 1 --2- 172.21.15.16:0/4149937162 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3714102570 0x7f37141a4d90 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:44.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:44.838+0000 7f371b047640 1 --2- 172.21.15.16:0/4149937162 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3714100e20 0x7f37141a4850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:44.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:44.838+0000 7f371b047640 1 -- 172.21.15.16:0/4149937162 >> 172.21.15.16:0/4149937162 conn(0x7f37140fc7d0 msgr2=0x7f3714069fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:00:44.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:44.838+0000 7f371b047640 1 -- 172.21.15.16:0/4149937162 shutdown_connections 2024-09-12T02:00:44.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:44.838+0000 7f371b047640 1 -- 172.21.15.16:0/4149937162 wait complete. 2024-09-12T02:00:44.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:44.747+0000 7fdfeabf9640 1 -- 172.21.15.16:0/2027846301 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdfe4103650 msgr2=0x7fdfe410bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:44.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:44.747+0000 7fdfeabf9640 1 --2- 172.21.15.16:0/2027846301 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdfe4103650 0x7fdfe410bb40 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fdfd4009ca0 tx=0x7fdfd402a950 comp rx=0 tx=0).stop 2024-09-12T02:00:44.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:44.747+0000 7fdfeabf9640 1 -- 172.21.15.16:0/2027846301 shutdown_connections 2024-09-12T02:00:44.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:44.747+0000 7fdfeabf9640 1 --2- 172.21.15.16:0/2027846301 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdfe410c160 0x7fdfe410e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:44.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:44.747+0000 7fdfeabf9640 1 --2- 172.21.15.16:0/2027846301 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdfe4103650 0x7fdfe410bb40 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:44.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:44.747+0000 7fdfeabf9640 1 --2- 172.21.15.16:0/2027846301 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdfe4102d30 0x7fdfe4103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:44.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:44.747+0000 7fdfeabf9640 1 -- 172.21.15.16:0/2027846301 >> 172.21.15.16:0/2027846301 conn(0x7fdfe40fc7d0 msgr2=0x7fdfe40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:00:44.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:44.747+0000 7fdfeabf9640 1 -- 172.21.15.16:0/2027846301 shutdown_connections 2024-09-12T02:00:44.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:44.747+0000 7fdfeabf9640 1 -- 172.21.15.16:0/2027846301 wait complete. 2024-09-12T02:00:44.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:44.747+0000 7fdfeabf9640 1 Processor -- start 2024-09-12T02:00:44.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:44.748+0000 7fdfeabf9640 1 -- start start 2024-09-12T02:00:44.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:44.748+0000 7fdfeabf9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdfe4102d30 0x7fdfe4198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:44.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:44.748+0000 7fdfeabf9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdfe4103650 0x7fdfe41989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:44.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:44.748+0000 7fdfeabf9640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdfe410c160 0x7fdfe419cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:44.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:44.748+0000 7fdfeabf9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fdfe4111f60 con 0x7fdfe4102d30 2024-09-12T02:00:44.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:44.748+0000 7fdfeabf9640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fdfe4111de0 con 0x7fdfe4103650 2024-09-12T02:00:44.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:44.748+0000 7fdfe8995640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdfe4102d30 0x7fdfe4198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:44.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:44.748+0000 7fdfeabf9640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fdfe41120e0 con 0x7fdfe410c160 2024-09-12T02:00:44.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:44.748+0000 7fdfe8995640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdfe4102d30 0x7fdfe4198460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51490/0 (socket says 172.21.15.16:51490) 2024-09-12T02:00:44.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:44.748+0000 7fdfe8995640 1 -- 172.21.15.16:0/3124630663 learned_addr learned my addr 172.21.15.16:0/3124630663 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:00:44.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:44.748+0000 7fdfe9196640 1 --2- 172.21.15.16:0/3124630663 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdfe410c160 0x7fdfe419cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:44.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:44.748+0000 7fdfe3fff640 1 --2- 172.21.15.16:0/3124630663 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdfe4103650 0x7fdfe41989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:44.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:44.748+0000 7fdfe8995640 1 -- 172.21.15.16:0/3124630663 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdfe410c160 msgr2=0x7fdfe419cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:44.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:44.748+0000 7fdfe8995640 1 --2- 172.21.15.16:0/3124630663 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdfe410c160 0x7fdfe419cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:44.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:44.748+0000 7fdfe8995640 1 -- 172.21.15.16:0/3124630663 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdfe4103650 msgr2=0x7fdfe41989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:44.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:44.748+0000 7fdfe8995640 1 --2- 172.21.15.16:0/3124630663 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdfe4103650 0x7fdfe41989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:44.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:44.748+0000 7fdfe8995640 1 -- 172.21.15.16:0/3124630663 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfe419d410 con 0x7fdfe4102d30 2024-09-12T02:00:44.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:44.749+0000 7fdfe8995640 1 --2- 172.21.15.16:0/3124630663 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdfe4102d30 0x7fdfe4198460 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fdfcc00e9f0 tx=0x7fdfcc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:00:44.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:44.749+0000 7fdfe1ffb640 1 -- 172.21.15.16:0/3124630663 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfcc00cd80 con 0x7fdfe4102d30 2024-09-12T02:00:44.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:44.749+0000 7fdfeabf9640 1 -- 172.21.15.16:0/3124630663 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfe419d700 con 0x7fdfe4102d30 2024-09-12T02:00:44.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:44.749+0000 7fdfe1ffb640 1 -- 172.21.15.16:0/3124630663 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdfcc010910 con 0x7fdfe4102d30 2024-09-12T02:00:44.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:44.749+0000 7fdfe1ffb640 1 -- 172.21.15.16:0/3124630663 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfcc0188f0 con 0x7fdfe4102d30 2024-09-12T02:00:44.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:44.749+0000 7fdfeabf9640 1 -- 172.21.15.16:0/3124630663 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfe41a4fe0 con 0x7fdfe4102d30 2024-09-12T02:00:44.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:44.750+0000 7fdfeabf9640 1 -- 172.21.15.16:0/3124630663 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfe4103110 con 0x7fdfe4102d30 2024-09-12T02:00:44.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:44.751+0000 7fdfe1ffb640 1 -- 172.21.15.16:0/3124630663 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fdfcc010ac0 con 0x7fdfe4102d30 2024-09-12T02:00:44.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:44.752+0000 7fdfe1ffb640 1 --2- 172.21.15.16:0/3124630663 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fdfd00a2c10 0x7fdfd00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:44.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:44.752+0000 7fdfe3fff640 1 --2- 172.21.15.16:0/3124630663 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fdfd00a2c10 0x7fdfd00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:44.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:44.752+0000 7fdfe1ffb640 1 -- 172.21.15.16:0/3124630663 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fdfcc0cf560 con 0x7fdfe4102d30 2024-09-12T02:00:44.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:44.753+0000 7fdfe3fff640 1 --2- 172.21.15.16:0/3124630663 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fdfd00a2c10 0x7fdfd00a50d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fdfe410d6a0 tx=0x7fdfd4004a60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:00:44.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:44.756+0000 7fdfe1ffb640 1 -- 172.21.15.16:0/3124630663 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fdfcc09bd10 con 0x7fdfe4102d30 2024-09-12T02:00:44.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:44.937+0000 7fdfeabf9640 1 -- 172.21.15.16:0/3124630663 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fdfe41997c0 con 0x7fdfe4102d30 2024-09-12T02:00:44.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:44.938+0000 7fdfe1ffb640 1 -- 172.21.15.16:0/3124630663 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fdfcc0a0010 con 0x7fdfe4102d30 2024-09-12T02:00:44.938 INFO:teuthology.orchestra.run.smithi016.stdout:184683593775 2024-09-12T02:00:44.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:44.939+0000 7fdfeabf9640 1 -- 172.21.15.16:0/3124630663 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fdfd00a2c10 msgr2=0x7fdfd00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:44.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:44.939+0000 7fdfeabf9640 1 --2- 172.21.15.16:0/3124630663 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fdfd00a2c10 0x7fdfd00a50d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fdfe410d6a0 tx=0x7fdfd4004a60 comp rx=0 tx=0).stop 2024-09-12T02:00:44.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:44.939+0000 7fdfeabf9640 1 -- 172.21.15.16:0/3124630663 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdfe4102d30 msgr2=0x7fdfe4198460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:44.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:44.939+0000 7fdfeabf9640 1 --2- 172.21.15.16:0/3124630663 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdfe4102d30 0x7fdfe4198460 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fdfcc00e9f0 tx=0x7fdfcc00eeb0 comp rx=0 tx=0).stop 2024-09-12T02:00:44.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:44.940+0000 7fdfeabf9640 1 -- 172.21.15.16:0/3124630663 shutdown_connections 2024-09-12T02:00:44.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:44.940+0000 7fdfeabf9640 1 --2- 172.21.15.16:0/3124630663 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fdfd00a2c10 0x7fdfd00a50d0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:44.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:44.940+0000 7fdfeabf9640 1 --2- 172.21.15.16:0/3124630663 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdfe410c160 0x7fdfe419cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:44.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:44.940+0000 7fdfeabf9640 1 --2- 172.21.15.16:0/3124630663 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdfe4103650 0x7fdfe41989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:44.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:44.940+0000 7fdfeabf9640 1 --2- 172.21.15.16:0/3124630663 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdfe4102d30 0x7fdfe4198460 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:44.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:44.940+0000 7fdfeabf9640 1 -- 172.21.15.16:0/3124630663 >> 172.21.15.16:0/3124630663 conn(0x7fdfe40fc7d0 msgr2=0x7fdfe40fd4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:00:44.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:44.940+0000 7fdfeabf9640 1 -- 172.21.15.16:0/3124630663 shutdown_connections 2024-09-12T02:00:44.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:44.940+0000 7fdfeabf9640 1 -- 172.21.15.16:0/3124630663 wait complete. 2024-09-12T02:00:45.373 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:00:45 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/4149937162' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-12T02:00:45.373 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:00:45 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/3124630663' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-12T02:00:45.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:00:45 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/4149937162' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-12T02:00:45.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:00:45 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/3124630663' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-12T02:00:45.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:00:45 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/4149937162' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-12T02:00:45.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:00:45 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/3124630663' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-12T02:00:45.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:45.843+0000 7f928c4c7640 1 -- 172.21.15.16:0/232160688 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9284101de0 msgr2=0x7f92841021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:45.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:45.843+0000 7f928c4c7640 1 --2- 172.21.15.16:0/232160688 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9284101de0 0x7f92841021c0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f9280009900 tx=0x7f928002a410 comp rx=0 tx=0).stop 2024-09-12T02:00:45.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:45.843+0000 7f928c4c7640 1 -- 172.21.15.16:0/232160688 shutdown_connections 2024-09-12T02:00:45.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:45.843+0000 7f928c4c7640 1 --2- 172.21.15.16:0/232160688 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9284106f10 0x7f928410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:45.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:45.843+0000 7f928c4c7640 1 --2- 172.21.15.16:0/232160688 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9284102790 0x7f92841067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:45.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:45.843+0000 7f928c4c7640 1 --2- 172.21.15.16:0/232160688 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9284101de0 0x7f92841021c0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:45.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:45.843+0000 7f928c4c7640 1 -- 172.21.15.16:0/232160688 >> 172.21.15.16:0/232160688 conn(0x7f92840fd8c0 msgr2=0x7f92840ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:00:45.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:45.843+0000 7f928c4c7640 1 -- 172.21.15.16:0/232160688 shutdown_connections 2024-09-12T02:00:45.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:45.843+0000 7f928c4c7640 1 -- 172.21.15.16:0/232160688 wait complete. 2024-09-12T02:00:45.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:45.844+0000 7f928c4c7640 1 Processor -- start 2024-09-12T02:00:45.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:45.844+0000 7f928c4c7640 1 -- start start 2024-09-12T02:00:45.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:45.844+0000 7f928c4c7640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9284101de0 0x7f92841984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:45.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:45.844+0000 7f928c4c7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9284102790 0x7f92841989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:45.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:45.844+0000 7f928c4c7640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9284106f10 0x7f928419cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:45.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:45.844+0000 7f928c4c7640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9284111f90 con 0x7f9284102790 2024-09-12T02:00:45.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:45.844+0000 7f928c4c7640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9284111e10 con 0x7f9284106f10 2024-09-12T02:00:45.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:45.844+0000 7f928c4c7640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f9284112110 con 0x7f9284101de0 2024-09-12T02:00:45.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:45.844+0000 7f9289a62640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9284102790 0x7f92841989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:45.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:45.844+0000 7f9289a62640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9284102790 0x7f92841989f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51504/0 (socket says 172.21.15.16:51504) 2024-09-12T02:00:45.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:45.844+0000 7f9289a62640 1 -- 172.21.15.16:0/3654276859 learned_addr learned my addr 172.21.15.16:0/3654276859 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:00:45.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:45.844+0000 7f928aa64640 1 --2- 172.21.15.16:0/3654276859 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9284106f10 0x7f928419cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:45.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:45.844+0000 7f928a263640 1 --2- 172.21.15.16:0/3654276859 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9284101de0 0x7f92841984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:45.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:45.844+0000 7f9289a62640 1 -- 172.21.15.16:0/3654276859 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9284101de0 msgr2=0x7f92841984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:45.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:45.844+0000 7f9289a62640 1 --2- 172.21.15.16:0/3654276859 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9284101de0 0x7f92841984b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:45.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:45.844+0000 7f9289a62640 1 -- 172.21.15.16:0/3654276859 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9284106f10 msgr2=0x7f928419cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:45.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:45.844+0000 7f9289a62640 1 --2- 172.21.15.16:0/3654276859 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9284106f10 0x7f928419cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:45.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:45.844+0000 7f9289a62640 1 -- 172.21.15.16:0/3654276859 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f928419d460 con 0x7f9284102790 2024-09-12T02:00:45.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:45.844+0000 7f928aa64640 1 --2- 172.21.15.16:0/3654276859 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9284106f10 0x7f928419cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:00:45.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:45.845+0000 7f9289a62640 1 --2- 172.21.15.16:0/3654276859 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9284102790 0x7f92841989f0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f927400c890 tx=0x7f927400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:00:45.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:45.845+0000 7f927b7fe640 1 -- 172.21.15.16:0/3654276859 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9274004220 con 0x7f9284102790 2024-09-12T02:00:45.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:45.845+0000 7f928c4c7640 1 -- 172.21.15.16:0/3654276859 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f928419d750 con 0x7f9284102790 2024-09-12T02:00:45.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:45.845+0000 7f927b7fe640 1 -- 172.21.15.16:0/3654276859 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f92740043c0 con 0x7f9284102790 2024-09-12T02:00:45.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:45.845+0000 7f927b7fe640 1 -- 172.21.15.16:0/3654276859 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92740049e0 con 0x7f9284102790 2024-09-12T02:00:45.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:45.845+0000 7f928c4c7640 1 -- 172.21.15.16:0/3654276859 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92841a5030 con 0x7f9284102790 2024-09-12T02:00:45.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:45.846+0000 7f928c4c7640 1 -- 172.21.15.16:0/3654276859 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92841021c0 con 0x7f9284102790 2024-09-12T02:00:45.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:45.847+0000 7f927b7fe640 1 -- 172.21.15.16:0/3654276859 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f92740079b0 con 0x7f9284102790 2024-09-12T02:00:45.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:45.848+0000 7f927b7fe640 1 --2- 172.21.15.16:0/3654276859 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f92700a2c10 0x7f92700a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:45.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:45.848+0000 7f928a263640 1 --2- 172.21.15.16:0/3654276859 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f92700a2c10 0x7f92700a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:45.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:45.848+0000 7f927b7fe640 1 -- 172.21.15.16:0/3654276859 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f92740cf1c0 con 0x7f9284102790 2024-09-12T02:00:45.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:45.848+0000 7f928a263640 1 --2- 172.21.15.16:0/3654276859 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f92700a2c10 0x7f92700a50d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f92800096c0 tx=0x7f9280000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:00:45.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:45.852+0000 7f927b7fe640 1 -- 172.21.15.16:0/3654276859 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9274014030 con 0x7f9284102790 2024-09-12T02:00:46.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:46.007+0000 7fa694bc7640 1 -- 172.21.15.16:0/1808545340 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa690100e20 msgr2=0x7fa690101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:46.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:46.007+0000 7fa694bc7640 1 --2- 172.21.15.16:0/1808545340 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa690100e20 0x7fa690101200 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fa680009900 tx=0x7fa68002a480 comp rx=0 tx=0).stop 2024-09-12T02:00:46.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:46.007+0000 7fa694bc7640 1 -- 172.21.15.16:0/1808545340 shutdown_connections 2024-09-12T02:00:46.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:46.007+0000 7fa694bc7640 1 --2- 172.21.15.16:0/1808545340 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa690102ef0 0x7fa69010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:46.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:46.007+0000 7fa694bc7640 1 --2- 172.21.15.16:0/1808545340 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa690102570 0x7fa6901029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:46.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:46.007+0000 7fa694bc7640 1 --2- 172.21.15.16:0/1808545340 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa690100e20 0x7fa690101200 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:46.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:46.007+0000 7fa694bc7640 1 -- 172.21.15.16:0/1808545340 >> 172.21.15.16:0/1808545340 conn(0x7fa6900fc7d0 msgr2=0x7fa6900febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:00:46.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:46.007+0000 7fa694bc7640 1 -- 172.21.15.16:0/1808545340 shutdown_connections 2024-09-12T02:00:46.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:46.007+0000 7fa694bc7640 1 -- 172.21.15.16:0/1808545340 wait complete. 2024-09-12T02:00:46.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:46.008+0000 7fa694bc7640 1 Processor -- start 2024-09-12T02:00:46.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:46.008+0000 7fa694bc7640 1 -- start start 2024-09-12T02:00:46.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:46.008+0000 7fa694bc7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa690100e20 0x7fa6901984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:46.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:46.008+0000 7fa694bc7640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa690102570 0x7fa690198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:46.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:46.008+0000 7fa694bc7640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa690102ef0 0x7fa69019cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:46.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:46.008+0000 7fa694bc7640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa690110fa0 con 0x7fa690100e20 2024-09-12T02:00:46.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:46.008+0000 7fa694bc7640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa690110e20 con 0x7fa690102ef0 2024-09-12T02:00:46.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:46.008+0000 7fa694bc7640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa690111120 con 0x7fa690102570 2024-09-12T02:00:46.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:46.008+0000 7fa68e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa690100e20 0x7fa6901984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:46.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:46.008+0000 7fa68e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa690100e20 0x7fa6901984c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51524/0 (socket says 172.21.15.16:51524) 2024-09-12T02:00:46.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:46.008+0000 7fa68e59c640 1 -- 172.21.15.16:0/2197664695 learned_addr learned my addr 172.21.15.16:0/2197664695 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:00:46.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:46.008+0000 7fa68dd9b640 1 --2- 172.21.15.16:0/2197664695 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa690102570 0x7fa690198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:46.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:46.008+0000 7fa68ed9d640 1 --2- 172.21.15.16:0/2197664695 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa690102ef0 0x7fa69019cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:46.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:46.008+0000 7fa68e59c640 1 -- 172.21.15.16:0/2197664695 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa690102570 msgr2=0x7fa690198a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:46.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:46.008+0000 7fa68e59c640 1 --2- 172.21.15.16:0/2197664695 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa690102570 0x7fa690198a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:46.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:46.008+0000 7fa68e59c640 1 -- 172.21.15.16:0/2197664695 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa690102ef0 msgr2=0x7fa69019cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:46.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:46.008+0000 7fa68e59c640 1 --2- 172.21.15.16:0/2197664695 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa690102ef0 0x7fa69019cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:46.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:46.008+0000 7fa68e59c640 1 -- 172.21.15.16:0/2197664695 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa69019d470 con 0x7fa690100e20 2024-09-12T02:00:46.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:46.008+0000 7fa68e59c640 1 --2- 172.21.15.16:0/2197664695 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa690100e20 0x7fa6901984c0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fa680006010 tx=0x7fa680003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:00:46.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:46.009+0000 7fa67f7fe640 1 -- 172.21.15.16:0/2197664695 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa68003f070 con 0x7fa690100e20 2024-09-12T02:00:46.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:46.009+0000 7fa694bc7640 1 -- 172.21.15.16:0/2197664695 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa69019d700 con 0x7fa690100e20 2024-09-12T02:00:46.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:46.033+0000 7f928c4c7640 1 -- 172.21.15.16:0/3654276859 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f928402d7c0 con 0x7f9284102790 2024-09-12T02:00:46.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:46.009+0000 7fa67f7fe640 1 -- 172.21.15.16:0/2197664695 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa680031e30 con 0x7fa690100e20 2024-09-12T02:00:46.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:46.009+0000 7fa67f7fe640 1 -- 172.21.15.16:0/2197664695 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa680035030 con 0x7fa690100e20 2024-09-12T02:00:46.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:46.009+0000 7fa694bc7640 1 -- 172.21.15.16:0/2197664695 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6901a5040 con 0x7fa690100e20 2024-09-12T02:00:46.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:46.010+0000 7fa694bc7640 1 -- 172.21.15.16:0/2197664695 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa690101200 con 0x7fa690100e20 2024-09-12T02:00:46.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:46.011+0000 7fa67f7fe640 1 -- 172.21.15.16:0/2197664695 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fa680042070 con 0x7fa690100e20 2024-09-12T02:00:46.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:46.012+0000 7fa67f7fe640 1 --2- 172.21.15.16:0/2197664695 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fa6780a2c10 0x7fa6780a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:46.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:46.012+0000 7fa68dd9b640 1 --2- 172.21.15.16:0/2197664695 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fa6780a2c10 0x7fa6780a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:46.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:46.012+0000 7fa67f7fe640 1 -- 172.21.15.16:0/2197664695 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fa6800f0200 con 0x7fa690100e20 2024-09-12T02:00:46.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:46.012+0000 7fa68dd9b640 1 --2- 172.21.15.16:0/2197664695 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fa6780a2c10 0x7fa6780a50d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fa6900fef70 tx=0x7fa68400a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:00:46.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:46.015+0000 7fa67f7fe640 1 -- 172.21.15.16:0/2197664695 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa6800bd100 con 0x7fa690100e20 2024-09-12T02:00:46.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:46.033+0000 7f927b7fe640 1 -- 172.21.15.16:0/3654276859 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f927409c0c0 con 0x7f9284102790 2024-09-12T02:00:46.037 INFO:teuthology.orchestra.run.smithi016.stdout:206158430250 2024-09-12T02:00:46.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:46.036+0000 7f928c4c7640 1 -- 172.21.15.16:0/3654276859 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f92700a2c10 msgr2=0x7f92700a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:46.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:46.036+0000 7f928c4c7640 1 --2- 172.21.15.16:0/3654276859 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f92700a2c10 0x7f92700a50d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f92800096c0 tx=0x7f9280000bc0 comp rx=0 tx=0).stop 2024-09-12T02:00:46.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:46.036+0000 7f928c4c7640 1 -- 172.21.15.16:0/3654276859 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9284102790 msgr2=0x7f92841989f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:46.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:46.036+0000 7f928c4c7640 1 --2- 172.21.15.16:0/3654276859 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9284102790 0x7f92841989f0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f927400c890 tx=0x7f927400cd50 comp rx=0 tx=0).stop 2024-09-12T02:00:46.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:46.036+0000 7f928c4c7640 1 -- 172.21.15.16:0/3654276859 shutdown_connections 2024-09-12T02:00:46.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:46.036+0000 7f928c4c7640 1 --2- 172.21.15.16:0/3654276859 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f92700a2c10 0x7f92700a50d0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:46.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:46.036+0000 7f928c4c7640 1 --2- 172.21.15.16:0/3654276859 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9284106f10 0x7f928419cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:46.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:46.036+0000 7f928c4c7640 1 --2- 172.21.15.16:0/3654276859 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9284102790 0x7f92841989f0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:46.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:46.036+0000 7f928c4c7640 1 --2- 172.21.15.16:0/3654276859 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9284101de0 0x7f92841984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:46.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:46.036+0000 7f928c4c7640 1 -- 172.21.15.16:0/3654276859 >> 172.21.15.16:0/3654276859 conn(0x7f92840fd8c0 msgr2=0x7f92840fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:00:46.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:46.036+0000 7f928c4c7640 1 -- 172.21.15.16:0/3654276859 shutdown_connections 2024-09-12T02:00:46.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:46.036+0000 7f928c4c7640 1 -- 172.21.15.16:0/3654276859 wait complete. 2024-09-12T02:00:46.183 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:00:46 smithi016 ceph-mon[30947]: pgmap v300: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-12T02:00:46.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:00:46 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/3654276859' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-12T02:00:46.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:46.197+0000 7fa694bc7640 1 -- 172.21.15.16:0/2197664695 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fa69010a0a0 con 0x7fa690100e20 2024-09-12T02:00:46.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:46.198+0000 7fa67f7fe640 1 -- 172.21.15.16:0/2197664695 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fa6800bca00 con 0x7fa690100e20 2024-09-12T02:00:46.198 INFO:teuthology.orchestra.run.smithi016.stdout:154618822708 2024-09-12T02:00:46.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:46.200+0000 7fa694bc7640 1 -- 172.21.15.16:0/2197664695 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fa6780a2c10 msgr2=0x7fa6780a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:46.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:46.200+0000 7fa694bc7640 1 --2- 172.21.15.16:0/2197664695 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fa6780a2c10 0x7fa6780a50d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fa6900fef70 tx=0x7fa68400a2d0 comp rx=0 tx=0).stop 2024-09-12T02:00:46.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:46.200+0000 7fa694bc7640 1 -- 172.21.15.16:0/2197664695 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa690100e20 msgr2=0x7fa6901984c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:46.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:46.200+0000 7fa694bc7640 1 --2- 172.21.15.16:0/2197664695 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa690100e20 0x7fa6901984c0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fa680006010 tx=0x7fa680003c80 comp rx=0 tx=0).stop 2024-09-12T02:00:46.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:46.200+0000 7fa694bc7640 1 -- 172.21.15.16:0/2197664695 shutdown_connections 2024-09-12T02:00:46.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:46.200+0000 7fa694bc7640 1 --2- 172.21.15.16:0/2197664695 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fa6780a2c10 0x7fa6780a50d0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:46.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:46.200+0000 7fa694bc7640 1 --2- 172.21.15.16:0/2197664695 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa690102ef0 0x7fa69019cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:46.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:46.200+0000 7fa694bc7640 1 --2- 172.21.15.16:0/2197664695 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa690102570 0x7fa690198a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:46.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:46.200+0000 7fa694bc7640 1 --2- 172.21.15.16:0/2197664695 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa690100e20 0x7fa6901984c0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:46.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:46.200+0000 7fa694bc7640 1 -- 172.21.15.16:0/2197664695 >> 172.21.15.16:0/2197664695 conn(0x7fa6900fc7d0 msgr2=0x7fa69006a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:00:46.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:46.200+0000 7fa694bc7640 1 -- 172.21.15.16:0/2197664695 shutdown_connections 2024-09-12T02:00:46.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:46.200+0000 7fa694bc7640 1 -- 172.21.15.16:0/2197664695 wait complete. 2024-09-12T02:00:46.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:00:46 smithi184 ceph-mon[35593]: pgmap v300: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-12T02:00:46.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:00:46 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/3654276859' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-12T02:00:46.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:00:46 smithi033 ceph-mon[35332]: pgmap v300: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-12T02:00:46.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:00:46 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/3654276859' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-12T02:00:47.215 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:00:47 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/2197664695' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-12T02:00:47.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:00:47 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/2197664695' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-12T02:00:47.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:47.368+0000 7f8190b26640 1 -- 172.21.15.16:0/635401775 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f818c1082a0 msgr2=0x7f818c10ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:47.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:47.368+0000 7f8190b26640 1 --2- 172.21.15.16:0/635401775 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f818c1082a0 0x7f818c10ec40 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f817c009930 tx=0x7f817c02a420 comp rx=0 tx=0).stop 2024-09-12T02:00:47.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:47.368+0000 7f8190b26640 1 -- 172.21.15.16:0/635401775 shutdown_connections 2024-09-12T02:00:47.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:47.368+0000 7f8190b26640 1 --2- 172.21.15.16:0/635401775 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f818c1082a0 0x7f818c10ec40 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:47.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:47.368+0000 7f8190b26640 1 --2- 172.21.15.16:0/635401775 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f818c1036e0 0x7f818c107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:47.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:47.368+0000 7f8190b26640 1 --2- 172.21.15.16:0/635401775 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f818c102d30 0x7f818c103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:47.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:47.368+0000 7f8190b26640 1 -- 172.21.15.16:0/635401775 >> 172.21.15.16:0/635401775 conn(0x7f818c0fc7d0 msgr2=0x7f818c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:00:47.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:47.368+0000 7f8190b26640 1 -- 172.21.15.16:0/635401775 shutdown_connections 2024-09-12T02:00:47.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:47.368+0000 7f8190b26640 1 -- 172.21.15.16:0/635401775 wait complete. 2024-09-12T02:00:47.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:47.368+0000 7f8190b26640 1 Processor -- start 2024-09-12T02:00:47.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:47.369+0000 7f8190b26640 1 -- start start 2024-09-12T02:00:47.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:47.369+0000 7f8190b26640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f818c102d30 0x7f818c198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:47.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:47.369+0000 7f8190b26640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f818c1036e0 0x7f818c198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:47.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:47.369+0000 7f8190b26640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f818c1082a0 0x7f818c19cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:47.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:47.369+0000 7f8190b26640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f818c1120a0 con 0x7f818c1036e0 2024-09-12T02:00:47.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:47.369+0000 7f8190b26640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f818c111f20 con 0x7f818c1082a0 2024-09-12T02:00:47.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:47.369+0000 7f8190b26640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f818c112220 con 0x7f818c102d30 2024-09-12T02:00:47.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:47.369+0000 7f8189d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f818c1036e0 0x7f818c198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:47.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:47.369+0000 7f8189d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f818c1036e0 0x7f818c198a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51548/0 (socket says 172.21.15.16:51548) 2024-09-12T02:00:47.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:47.369+0000 7f8189d9b640 1 -- 172.21.15.16:0/213249044 learned_addr learned my addr 172.21.15.16:0/213249044 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:00:47.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:47.369+0000 7f818ad9d640 1 --2- 172.21.15.16:0/213249044 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f818c1082a0 0x7f818c19cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:47.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:47.369+0000 7f818a59c640 1 --2- 172.21.15.16:0/213249044 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f818c102d30 0x7f818c198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:47.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:47.369+0000 7f8189d9b640 1 -- 172.21.15.16:0/213249044 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f818c102d30 msgr2=0x7f818c198510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:47.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:47.369+0000 7f8189d9b640 1 --2- 172.21.15.16:0/213249044 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f818c102d30 0x7f818c198510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:47.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:47.369+0000 7f8189d9b640 1 -- 172.21.15.16:0/213249044 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f818c1082a0 msgr2=0x7f818c19cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:47.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:47.369+0000 7f8189d9b640 1 --2- 172.21.15.16:0/213249044 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f818c1082a0 0x7f818c19cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:47.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:47.369+0000 7f8189d9b640 1 -- 172.21.15.16:0/213249044 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f818c19d4c0 con 0x7f818c1036e0 2024-09-12T02:00:47.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:47.369+0000 7f8189d9b640 1 --2- 172.21.15.16:0/213249044 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f818c1036e0 0x7f818c198a50 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f817400d740 tx=0x7f817400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:00:47.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:47.370+0000 7f81837fe640 1 -- 172.21.15.16:0/213249044 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8174017070 con 0x7f818c1036e0 2024-09-12T02:00:47.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:47.370+0000 7f81837fe640 1 -- 172.21.15.16:0/213249044 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f817400de50 con 0x7f818c1036e0 2024-09-12T02:00:47.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:47.370+0000 7f8190b26640 1 -- 172.21.15.16:0/213249044 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f818c19d7b0 con 0x7f818c1036e0 2024-09-12T02:00:47.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:47.370+0000 7f81837fe640 1 -- 172.21.15.16:0/213249044 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f817400f6b0 con 0x7f818c1036e0 2024-09-12T02:00:47.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:47.370+0000 7f8190b26640 1 -- 172.21.15.16:0/213249044 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f818c1a5090 con 0x7f818c1036e0 2024-09-12T02:00:47.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:47.371+0000 7f8190b26640 1 -- 172.21.15.16:0/213249044 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f818c103110 con 0x7f818c1036e0 2024-09-12T02:00:47.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:47.372+0000 7f81837fe640 1 -- 172.21.15.16:0/213249044 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f817401a070 con 0x7f818c1036e0 2024-09-12T02:00:47.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:47.373+0000 7f81837fe640 1 --2- 172.21.15.16:0/213249044 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f81780a2c90 0x7f81780a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:47.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:47.373+0000 7f818a59c640 1 --2- 172.21.15.16:0/213249044 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f81780a2c90 0x7f81780a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:47.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:47.373+0000 7f81837fe640 1 -- 172.21.15.16:0/213249044 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f81740cea10 con 0x7f818c1036e0 2024-09-12T02:00:47.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:47.373+0000 7f818a59c640 1 --2- 172.21.15.16:0/213249044 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f81780a2c90 0x7f81780a5150 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f8184005fd0 tx=0x7f8184009870 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:00:47.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:47.377+0000 7f81837fe640 1 -- 172.21.15.16:0/213249044 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f817409b910 con 0x7f818c1036e0 2024-09-12T02:00:47.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:00:47 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/2197664695' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-12T02:00:47.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:47.555+0000 7f8190b26640 1 -- 172.21.15.16:0/213249044 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f818c10a2e0 con 0x7f818c1036e0 2024-09-12T02:00:47.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:47.555+0000 7f81837fe640 1 -- 172.21.15.16:0/213249044 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f817409b210 con 0x7f818c1036e0 2024-09-12T02:00:47.555 INFO:teuthology.orchestra.run.smithi016.stdout:103079215166 2024-09-12T02:00:47.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:47.557+0000 7f8190b26640 1 -- 172.21.15.16:0/213249044 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f81780a2c90 msgr2=0x7f81780a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:47.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:47.557+0000 7f8190b26640 1 --2- 172.21.15.16:0/213249044 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f81780a2c90 0x7f81780a5150 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f8184005fd0 tx=0x7f8184009870 comp rx=0 tx=0).stop 2024-09-12T02:00:47.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:47.557+0000 7f8190b26640 1 -- 172.21.15.16:0/213249044 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f818c1036e0 msgr2=0x7f818c198a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:47.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:47.557+0000 7f8190b26640 1 --2- 172.21.15.16:0/213249044 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f818c1036e0 0x7f818c198a50 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f817400d740 tx=0x7f817400dc00 comp rx=0 tx=0).stop 2024-09-12T02:00:47.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:47.557+0000 7f8190b26640 1 -- 172.21.15.16:0/213249044 shutdown_connections 2024-09-12T02:00:47.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:47.557+0000 7f8190b26640 1 --2- 172.21.15.16:0/213249044 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f81780a2c90 0x7f81780a5150 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:47.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:47.557+0000 7f8190b26640 1 --2- 172.21.15.16:0/213249044 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f818c1082a0 0x7f818c19cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:47.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:47.557+0000 7f8190b26640 1 --2- 172.21.15.16:0/213249044 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f818c1036e0 0x7f818c198a50 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:47.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:47.557+0000 7f8190b26640 1 --2- 172.21.15.16:0/213249044 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f818c102d30 0x7f818c198510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:47.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:47.557+0000 7f8190b26640 1 -- 172.21.15.16:0/213249044 >> 172.21.15.16:0/213249044 conn(0x7f818c0fc7d0 msgr2=0x7f818c105d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:00:47.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:47.557+0000 7f8190b26640 1 -- 172.21.15.16:0/213249044 shutdown_connections 2024-09-12T02:00:47.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:47.557+0000 7f8190b26640 1 -- 172.21.15.16:0/213249044 wait complete. 2024-09-12T02:00:48.331 INFO:tasks.cephadm.ceph_manager.ceph:need seq 103079215163 got 103079215166 for osd.3 2024-09-12T02:00:48.331 DEBUG:teuthology.parallel:result is None 2024-09-12T02:00:48.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:00:48 smithi184 ceph-mon[35593]: pgmap v301: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-12T02:00:48.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:00:48 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/213249044' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-12T02:00:48.385 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:00:48 smithi016 ceph-mon[30947]: pgmap v301: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-12T02:00:48.385 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:00:48 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/213249044' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-12T02:00:48.418 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:00:48 smithi033 ceph-mon[35332]: pgmap v301: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-12T02:00:48.418 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:00:48 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/213249044' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-12T02:00:48.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:48.501+0000 7fd2e8e68640 1 -- 172.21.15.16:0/59910814 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2e4102ef0 msgr2=0x7fd2e410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:48.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:48.501+0000 7fd2e8e68640 1 --2- 172.21.15.16:0/59910814 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2e4102ef0 0x7fd2e410da40 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fd2dc009ca0 tx=0x7fd2dc02a950 comp rx=0 tx=0).stop 2024-09-12T02:00:48.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:48.501+0000 7fd2e8e68640 1 -- 172.21.15.16:0/59910814 shutdown_connections 2024-09-12T02:00:48.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:48.501+0000 7fd2e8e68640 1 --2- 172.21.15.16:0/59910814 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2e4102ef0 0x7fd2e410da40 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:48.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:48.501+0000 7fd2e8e68640 1 --2- 172.21.15.16:0/59910814 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd2e4102570 0x7fd2e41029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:48.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:48.501+0000 7fd2e8e68640 1 --2- 172.21.15.16:0/59910814 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2e4100e20 0x7fd2e4101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:48.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:48.501+0000 7fd2e8e68640 1 -- 172.21.15.16:0/59910814 >> 172.21.15.16:0/59910814 conn(0x7fd2e40fc7d0 msgr2=0x7fd2e40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:00:48.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:48.501+0000 7fd2e8e68640 1 -- 172.21.15.16:0/59910814 shutdown_connections 2024-09-12T02:00:48.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:48.501+0000 7fd2e8e68640 1 -- 172.21.15.16:0/59910814 wait complete. 2024-09-12T02:00:48.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:48.502+0000 7fd2e8e68640 1 Processor -- start 2024-09-12T02:00:48.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:48.502+0000 7fd2e8e68640 1 -- start start 2024-09-12T02:00:48.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:48.502+0000 7fd2e8e68640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2e4100e20 0x7fd2e4198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:48.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:48.502+0000 7fd2e8e68640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd2e4102570 0x7fd2e4198bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:48.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:48.502+0000 7fd2e8e68640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2e4102ef0 0x7fd2e419cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:48.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:48.502+0000 7fd2e8e68640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd2e4111020 con 0x7fd2e4100e20 2024-09-12T02:00:48.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:48.502+0000 7fd2e8e68640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd2e4110ea0 con 0x7fd2e4102ef0 2024-09-12T02:00:48.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:48.502+0000 7fd2e8e68640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd2e41111a0 con 0x7fd2e4102570 2024-09-12T02:00:48.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:48.502+0000 7fd2e259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2e4100e20 0x7fd2e4198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:48.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:48.502+0000 7fd2e259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2e4100e20 0x7fd2e4198670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51558/0 (socket says 172.21.15.16:51558) 2024-09-12T02:00:48.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:48.502+0000 7fd2e259c640 1 -- 172.21.15.16:0/2862868599 learned_addr learned my addr 172.21.15.16:0/2862868599 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:00:48.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:48.502+0000 7fd2e2d9d640 1 --2- 172.21.15.16:0/2862868599 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2e4102ef0 0x7fd2e419cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:48.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:48.502+0000 7fd2e1d9b640 1 --2- 172.21.15.16:0/2862868599 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd2e4102570 0x7fd2e4198bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:48.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:48.503+0000 7fd2e259c640 1 -- 172.21.15.16:0/2862868599 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd2e4102570 msgr2=0x7fd2e4198bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:48.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:48.503+0000 7fd2e259c640 1 --2- 172.21.15.16:0/2862868599 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd2e4102570 0x7fd2e4198bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:48.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:48.503+0000 7fd2e259c640 1 -- 172.21.15.16:0/2862868599 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2e4102ef0 msgr2=0x7fd2e419cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:48.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:48.503+0000 7fd2e259c640 1 --2- 172.21.15.16:0/2862868599 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2e4102ef0 0x7fd2e419cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:48.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:48.503+0000 7fd2e259c640 1 -- 172.21.15.16:0/2862868599 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2e419d620 con 0x7fd2e4100e20 2024-09-12T02:00:48.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:48.503+0000 7fd2e2d9d640 1 --2- 172.21.15.16:0/2862868599 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2e4102ef0 0x7fd2e419cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:00:48.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:48.503+0000 7fd2e1d9b640 1 --2- 172.21.15.16:0/2862868599 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd2e4102570 0x7fd2e4198bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:00:48.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:48.503+0000 7fd2e259c640 1 --2- 172.21.15.16:0/2862868599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2e4100e20 0x7fd2e4198670 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fd2d400b4a0 tx=0x7fd2d400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:00:48.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:48.503+0000 7fd2d37fe640 1 -- 172.21.15.16:0/2862868599 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2d4014070 con 0x7fd2e4100e20 2024-09-12T02:00:48.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:48.503+0000 7fd2d37fe640 1 -- 172.21.15.16:0/2862868599 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd2d4010af0 con 0x7fd2e4100e20 2024-09-12T02:00:48.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:48.503+0000 7fd2e8e68640 1 -- 172.21.15.16:0/2862868599 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2e419d910 con 0x7fd2e4100e20 2024-09-12T02:00:48.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:48.503+0000 7fd2d37fe640 1 -- 172.21.15.16:0/2862868599 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2d4017d10 con 0x7fd2e4100e20 2024-09-12T02:00:48.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:48.503+0000 7fd2e8e68640 1 -- 172.21.15.16:0/2862868599 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2e41a51f0 con 0x7fd2e4100e20 2024-09-12T02:00:48.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:48.504+0000 7fd2e8e68640 1 -- 172.21.15.16:0/2862868599 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2e4101280 con 0x7fd2e4100e20 2024-09-12T02:00:48.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:48.505+0000 7fd2d37fe640 1 -- 172.21.15.16:0/2862868599 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fd2d4017070 con 0x7fd2e4100e20 2024-09-12T02:00:48.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:48.506+0000 7fd2d37fe640 1 --2- 172.21.15.16:0/2862868599 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fd2cc0a2c90 0x7fd2cc0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:48.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:48.506+0000 7fd2e1d9b640 1 --2- 172.21.15.16:0/2862868599 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fd2cc0a2c90 0x7fd2cc0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:48.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:48.506+0000 7fd2d37fe640 1 -- 172.21.15.16:0/2862868599 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fd2d40ce460 con 0x7fd2e4100e20 2024-09-12T02:00:48.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:48.507+0000 7fd2e1d9b640 1 --2- 172.21.15.16:0/2862868599 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fd2cc0a2c90 0x7fd2cc0a5150 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd2d8006fd0 tx=0x7fd2d8008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:00:48.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:48.510+0000 7fd2d37fe640 1 -- 172.21.15.16:0/2862868599 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd2d409b370 con 0x7fd2e4100e20 2024-09-12T02:00:48.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:48.694+0000 7fd2e8e68640 1 -- 172.21.15.16:0/2862868599 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7fd2e410a0a0 con 0x7fd2e4100e20 2024-09-12T02:00:48.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:48.694+0000 7fd2d37fe640 1 -- 172.21.15.16:0/2862868599 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fd2d409ac70 con 0x7fd2e4100e20 2024-09-12T02:00:48.694 INFO:teuthology.orchestra.run.smithi016.stdout:287762808860 2024-09-12T02:00:48.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:48.696+0000 7fd2e8e68640 1 -- 172.21.15.16:0/2862868599 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fd2cc0a2c90 msgr2=0x7fd2cc0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:48.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:48.696+0000 7fd2e8e68640 1 --2- 172.21.15.16:0/2862868599 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fd2cc0a2c90 0x7fd2cc0a5150 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd2d8006fd0 tx=0x7fd2d8008040 comp rx=0 tx=0).stop 2024-09-12T02:00:48.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:48.696+0000 7fd2e8e68640 1 -- 172.21.15.16:0/2862868599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2e4100e20 msgr2=0x7fd2e4198670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:48.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:48.696+0000 7fd2e8e68640 1 --2- 172.21.15.16:0/2862868599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2e4100e20 0x7fd2e4198670 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fd2d400b4a0 tx=0x7fd2d400b960 comp rx=0 tx=0).stop 2024-09-12T02:00:48.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:48.696+0000 7fd2e8e68640 1 -- 172.21.15.16:0/2862868599 shutdown_connections 2024-09-12T02:00:48.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:48.696+0000 7fd2e8e68640 1 --2- 172.21.15.16:0/2862868599 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fd2cc0a2c90 0x7fd2cc0a5150 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:48.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:48.696+0000 7fd2e8e68640 1 --2- 172.21.15.16:0/2862868599 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2e4102ef0 0x7fd2e419cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:48.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:48.696+0000 7fd2e8e68640 1 --2- 172.21.15.16:0/2862868599 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd2e4102570 0x7fd2e4198bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:48.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:48.696+0000 7fd2e8e68640 1 --2- 172.21.15.16:0/2862868599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2e4100e20 0x7fd2e4198670 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:48.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:48.696+0000 7fd2e8e68640 1 -- 172.21.15.16:0/2862868599 >> 172.21.15.16:0/2862868599 conn(0x7fd2e40fc7d0 msgr2=0x7fd2e406a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:00:48.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:48.696+0000 7fd2e8e68640 1 -- 172.21.15.16:0/2862868599 shutdown_connections 2024-09-12T02:00:48.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:48.696+0000 7fd2e8e68640 1 -- 172.21.15.16:0/2862868599 wait complete. 2024-09-12T02:00:49.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:49.106+0000 7f184a775640 1 -- 172.21.15.16:0/1870606795 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1844103640 msgr2=0x7f184410bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:49.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:49.106+0000 7f184a775640 1 --2- 172.21.15.16:0/1870606795 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1844103640 0x7f184410bb30 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f1838009900 tx=0x7f183802a480 comp rx=0 tx=0).stop 2024-09-12T02:00:49.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:49.107+0000 7f184a775640 1 -- 172.21.15.16:0/1870606795 shutdown_connections 2024-09-12T02:00:49.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:49.107+0000 7f184a775640 1 --2- 172.21.15.16:0/1870606795 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f184410c150 0x7f184410e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:49.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:49.107+0000 7f184a775640 1 --2- 172.21.15.16:0/1870606795 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1844103640 0x7f184410bb30 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:49.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:49.107+0000 7f184a775640 1 --2- 172.21.15.16:0/1870606795 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1844102d20 0x7f1844103100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:49.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:49.107+0000 7f184a775640 1 -- 172.21.15.16:0/1870606795 >> 172.21.15.16:0/1870606795 conn(0x7f18440fc780 msgr2=0x7f18440feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:00:49.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:49.107+0000 7f184a775640 1 -- 172.21.15.16:0/1870606795 shutdown_connections 2024-09-12T02:00:49.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:49.107+0000 7f184a775640 1 -- 172.21.15.16:0/1870606795 wait complete. 2024-09-12T02:00:49.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:49.107+0000 7f184a775640 1 Processor -- start 2024-09-12T02:00:49.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:49.107+0000 7f184a775640 1 -- start start 2024-09-12T02:00:49.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:49.108+0000 7f184a775640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1844102d20 0x7f1844198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:49.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:49.108+0000 7f184a775640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1844103640 0x7f1844198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:49.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:49.108+0000 7f184a775640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f184410c150 0x7f184419cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:49.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:49.108+0000 7f184a775640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1844111f90 con 0x7f184410c150 2024-09-12T02:00:49.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:49.108+0000 7f184a775640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1844111e10 con 0x7f1844102d20 2024-09-12T02:00:49.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:49.108+0000 7f184a775640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1844112110 con 0x7f1844103640 2024-09-12T02:00:49.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:49.108+0000 7f1848d12640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f184410c150 0x7f184419cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:49.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:49.108+0000 7f1848d12640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f184410c150 0x7f184419cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51588/0 (socket says 172.21.15.16:51588) 2024-09-12T02:00:49.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:49.108+0000 7f1848d12640 1 -- 172.21.15.16:0/3178576980 learned_addr learned my addr 172.21.15.16:0/3178576980 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:00:49.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:49.108+0000 7f1843fff640 1 --2- 172.21.15.16:0/3178576980 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1844102d20 0x7f1844198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:49.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:49.108+0000 7f18437fe640 1 --2- 172.21.15.16:0/3178576980 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1844103640 0x7f1844198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:49.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:49.108+0000 7f1848d12640 1 -- 172.21.15.16:0/3178576980 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1844103640 msgr2=0x7f1844198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:49.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:49.108+0000 7f1848d12640 1 --2- 172.21.15.16:0/3178576980 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1844103640 0x7f1844198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:49.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:49.108+0000 7f1848d12640 1 -- 172.21.15.16:0/3178576980 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1844102d20 msgr2=0x7f1844198500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:49.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:49.108+0000 7f1848d12640 1 --2- 172.21.15.16:0/3178576980 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1844102d20 0x7f1844198500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:49.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:49.108+0000 7f1848d12640 1 -- 172.21.15.16:0/3178576980 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f184419d4b0 con 0x7f184410c150 2024-09-12T02:00:49.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:49.108+0000 7f1848d12640 1 --2- 172.21.15.16:0/3178576980 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f184410c150 0x7f184419cdd0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f183c00cad0 tx=0x7f183c00ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:00:49.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:49.108+0000 7f18417fa640 1 -- 172.21.15.16:0/3178576980 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f183c00ad60 con 0x7f184410c150 2024-09-12T02:00:49.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:49.108+0000 7f184a775640 1 -- 172.21.15.16:0/3178576980 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f184419d7a0 con 0x7f184410c150 2024-09-12T02:00:49.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:49.108+0000 7f184a775640 1 -- 172.21.15.16:0/3178576980 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18441a5080 con 0x7f184410c150 2024-09-12T02:00:49.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:49.109+0000 7f18417fa640 1 -- 172.21.15.16:0/3178576980 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f183c005020 con 0x7f184410c150 2024-09-12T02:00:49.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:49.109+0000 7f18417fa640 1 -- 172.21.15.16:0/3178576980 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f183c01d060 con 0x7f184410c150 2024-09-12T02:00:49.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:49.110+0000 7f184a775640 1 -- 172.21.15.16:0/3178576980 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1830005290 con 0x7f184410c150 2024-09-12T02:00:49.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:49.111+0000 7f18417fa640 1 -- 172.21.15.16:0/3178576980 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f183c004a40 con 0x7f184410c150 2024-09-12T02:00:49.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:49.112+0000 7f18417fa640 1 --2- 172.21.15.16:0/3178576980 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f182c0a2c10 0x7f182c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:49.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:49.112+0000 7f1843fff640 1 --2- 172.21.15.16:0/3178576980 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f182c0a2c10 0x7f182c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:49.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:49.112+0000 7f18417fa640 1 -- 172.21.15.16:0/3178576980 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f183c0cf4c0 con 0x7f184410c150 2024-09-12T02:00:49.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:49.112+0000 7f1843fff640 1 --2- 172.21.15.16:0/3178576980 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f182c0a2c10 0x7f182c0a50d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f1834009990 tx=0x7f1834009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:00:49.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:49.117+0000 7f18417fa640 1 -- 172.21.15.16:0/3178576980 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f183c014030 con 0x7f184410c150 2024-09-12T02:00:49.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:49.298+0000 7f184a775640 1 -- 172.21.15.16:0/3178576980 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f1830005850 con 0x7f184410c150 2024-09-12T02:00:49.299 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:00:49 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/2862868599' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-12T02:00:49.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:49.298+0000 7f18417fa640 1 -- 172.21.15.16:0/3178576980 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f183c09c2c0 con 0x7f184410c150 2024-09-12T02:00:49.299 INFO:teuthology.orchestra.run.smithi016.stdout:236223201318 2024-09-12T02:00:49.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:49.301+0000 7f184a775640 1 -- 172.21.15.16:0/3178576980 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f182c0a2c10 msgr2=0x7f182c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:49.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:49.301+0000 7f184a775640 1 --2- 172.21.15.16:0/3178576980 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f182c0a2c10 0x7f182c0a50d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f1834009990 tx=0x7f1834009110 comp rx=0 tx=0).stop 2024-09-12T02:00:49.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:49.301+0000 7f184a775640 1 -- 172.21.15.16:0/3178576980 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f184410c150 msgr2=0x7f184419cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:49.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:49.301+0000 7f184a775640 1 --2- 172.21.15.16:0/3178576980 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f184410c150 0x7f184419cdd0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f183c00cad0 tx=0x7f183c00ced0 comp rx=0 tx=0).stop 2024-09-12T02:00:49.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:49.301+0000 7f184a775640 1 -- 172.21.15.16:0/3178576980 shutdown_connections 2024-09-12T02:00:49.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:49.301+0000 7f184a775640 1 --2- 172.21.15.16:0/3178576980 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f182c0a2c10 0x7f182c0a50d0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:49.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:49.301+0000 7f184a775640 1 --2- 172.21.15.16:0/3178576980 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f184410c150 0x7f184419cdd0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:49.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:49.301+0000 7f184a775640 1 --2- 172.21.15.16:0/3178576980 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1844103640 0x7f1844198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:49.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:49.301+0000 7f184a775640 1 --2- 172.21.15.16:0/3178576980 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1844102d20 0x7f1844198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:49.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:49.301+0000 7f184a775640 1 -- 172.21.15.16:0/3178576980 >> 172.21.15.16:0/3178576980 conn(0x7f18440fc780 msgr2=0x7f18440fe020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:00:49.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:49.301+0000 7f184a775640 1 -- 172.21.15.16:0/3178576980 shutdown_connections 2024-09-12T02:00:49.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:49.301+0000 7f184a775640 1 -- 172.21.15.16:0/3178576980 wait complete. 2024-09-12T02:00:49.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:00:49 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/2862868599' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-12T02:00:49.418 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:00:49 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/2862868599' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-12T02:00:49.589 INFO:tasks.cephadm.ceph_manager.ceph:need seq 184683593774 got 184683593775 for osd.6 2024-09-12T02:00:49.590 DEBUG:teuthology.parallel:result is None 2024-09-12T02:00:49.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:49.727+0000 7fb7805e0640 1 -- 172.21.15.16:0/2872358646 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb778106f10 msgr2=0x7fb77810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:49.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:49.727+0000 7fb7805e0640 1 --2- 172.21.15.16:0/2872358646 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb778106f10 0x7fb77810ea90 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fb770009ca0 tx=0x7fb77002a8c0 comp rx=0 tx=0).stop 2024-09-12T02:00:49.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:49.727+0000 7fb7805e0640 1 -- 172.21.15.16:0/2872358646 shutdown_connections 2024-09-12T02:00:49.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:49.727+0000 7fb7805e0640 1 --2- 172.21.15.16:0/2872358646 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb778106f10 0x7fb77810ea90 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:49.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:49.727+0000 7fb7805e0640 1 --2- 172.21.15.16:0/2872358646 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb778102790 0x7fb7781067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:49.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:49.727+0000 7fb7805e0640 1 --2- 172.21.15.16:0/2872358646 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb778101de0 0x7fb7781021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:49.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:49.727+0000 7fb7805e0640 1 -- 172.21.15.16:0/2872358646 >> 172.21.15.16:0/2872358646 conn(0x7fb7780fd8c0 msgr2=0x7fb7780ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:00:49.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:49.727+0000 7fb7805e0640 1 -- 172.21.15.16:0/2872358646 shutdown_connections 2024-09-12T02:00:49.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:49.727+0000 7fb7805e0640 1 -- 172.21.15.16:0/2872358646 wait complete. 2024-09-12T02:00:49.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:49.728+0000 7fb7805e0640 1 Processor -- start 2024-09-12T02:00:49.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:49.728+0000 7fb7805e0640 1 -- start start 2024-09-12T02:00:49.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:49.728+0000 7fb7805e0640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb778101de0 0x7fb7780744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:49.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:49.728+0000 7fb7805e0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb778102790 0x7fb778074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:49.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:49.728+0000 7fb7805e0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb778106f10 0x7fb778074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:49.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:49.728+0000 7fb7805e0640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb778111f70 con 0x7fb778102790 2024-09-12T02:00:49.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:49.728+0000 7fb7805e0640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb778111df0 con 0x7fb778106f10 2024-09-12T02:00:49.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:49.728+0000 7fb7805e0640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb7781120f0 con 0x7fb778101de0 2024-09-12T02:00:49.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:49.728+0000 7fb77db7b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb778102790 0x7fb778074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:49.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:49.728+0000 7fb77db7b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb778102790 0x7fb778074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51606/0 (socket says 172.21.15.16:51606) 2024-09-12T02:00:49.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:49.728+0000 7fb77db7b640 1 -- 172.21.15.16:0/1271406707 learned_addr learned my addr 172.21.15.16:0/1271406707 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:00:49.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:49.729+0000 7fb77eb7d640 1 --2- 172.21.15.16:0/1271406707 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb778106f10 0x7fb778074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:49.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:49.729+0000 7fb77e37c640 1 --2- 172.21.15.16:0/1271406707 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb778101de0 0x7fb7780744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:49.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:49.729+0000 7fb77db7b640 1 -- 172.21.15.16:0/1271406707 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb778101de0 msgr2=0x7fb7780744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:49.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:49.729+0000 7fb77db7b640 1 --2- 172.21.15.16:0/1271406707 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb778101de0 0x7fb7780744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:49.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:49.729+0000 7fb77db7b640 1 -- 172.21.15.16:0/1271406707 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb778106f10 msgr2=0x7fb778074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:49.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:49.729+0000 7fb77db7b640 1 --2- 172.21.15.16:0/1271406707 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb778106f10 0x7fb778074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:49.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:49.729+0000 7fb77db7b640 1 -- 172.21.15.16:0/1271406707 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7781a6850 con 0x7fb778102790 2024-09-12T02:00:49.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:49.729+0000 7fb77e37c640 1 --2- 172.21.15.16:0/1271406707 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb778101de0 0x7fb7780744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:00:49.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:49.729+0000 7fb77db7b640 1 --2- 172.21.15.16:0/1271406707 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb778102790 0x7fb778074a30 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fb76800c920 tx=0x7fb76800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:00:49.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:49.729+0000 7fb76f7fe640 1 -- 172.21.15.16:0/1271406707 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb768007c70 con 0x7fb778102790 2024-09-12T02:00:49.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:49.729+0000 7fb76f7fe640 1 -- 172.21.15.16:0/1271406707 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb768007e10 con 0x7fb778102790 2024-09-12T02:00:49.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:49.729+0000 7fb7805e0640 1 -- 172.21.15.16:0/1271406707 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7781a6b40 con 0x7fb778102790 2024-09-12T02:00:49.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:49.729+0000 7fb76f7fe640 1 -- 172.21.15.16:0/1271406707 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb768018890 con 0x7fb778102790 2024-09-12T02:00:49.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:49.729+0000 7fb7805e0640 1 -- 172.21.15.16:0/1271406707 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7781a70d0 con 0x7fb778102790 2024-09-12T02:00:49.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:49.730+0000 7fb7805e0640 1 -- 172.21.15.16:0/1271406707 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7781021c0 con 0x7fb778102790 2024-09-12T02:00:49.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:49.731+0000 7fb76f7fe640 1 -- 172.21.15.16:0/1271406707 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fb768018a70 con 0x7fb778102790 2024-09-12T02:00:49.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:49.732+0000 7fb76f7fe640 1 --2- 172.21.15.16:0/1271406707 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fb7640a2c90 0x7fb7640a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:49.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:49.732+0000 7fb77e37c640 1 --2- 172.21.15.16:0/1271406707 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fb7640a2c90 0x7fb7640a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:49.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:49.732+0000 7fb76f7fe640 1 -- 172.21.15.16:0/1271406707 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fb7680cf2d0 con 0x7fb778102790 2024-09-12T02:00:49.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:49.733+0000 7fb77e37c640 1 --2- 172.21.15.16:0/1271406707 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fb7640a2c90 0x7fb7640a5150 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb774009b40 tx=0x7fb774005e50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:00:49.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:49.737+0000 7fb76f7fe640 1 -- 172.21.15.16:0/1271406707 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb768014030 con 0x7fb778102790 2024-09-12T02:00:49.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:49.917+0000 7fb7805e0640 1 -- 172.21.15.16:0/1271406707 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7fb778075a20 con 0x7fb778102790 2024-09-12T02:00:49.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:49.918+0000 7fb76f7fe640 1 -- 172.21.15.16:0/1271406707 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fb76800de20 con 0x7fb778102790 2024-09-12T02:00:49.918 INFO:teuthology.orchestra.run.smithi016.stdout:317827579927 2024-09-12T02:00:49.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:49.920+0000 7fb7805e0640 1 -- 172.21.15.16:0/1271406707 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fb7640a2c90 msgr2=0x7fb7640a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:49.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:49.920+0000 7fb7805e0640 1 --2- 172.21.15.16:0/1271406707 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fb7640a2c90 0x7fb7640a5150 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb774009b40 tx=0x7fb774005e50 comp rx=0 tx=0).stop 2024-09-12T02:00:49.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:49.920+0000 7fb7805e0640 1 -- 172.21.15.16:0/1271406707 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb778102790 msgr2=0x7fb778074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:49.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:49.920+0000 7fb7805e0640 1 --2- 172.21.15.16:0/1271406707 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb778102790 0x7fb778074a30 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fb76800c920 tx=0x7fb76800cde0 comp rx=0 tx=0).stop 2024-09-12T02:00:49.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:49.920+0000 7fb7805e0640 1 -- 172.21.15.16:0/1271406707 shutdown_connections 2024-09-12T02:00:49.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:49.920+0000 7fb7805e0640 1 --2- 172.21.15.16:0/1271406707 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fb7640a2c90 0x7fb7640a5150 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:49.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:49.920+0000 7fb7805e0640 1 --2- 172.21.15.16:0/1271406707 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb778106f10 0x7fb778074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:49.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:49.920+0000 7fb7805e0640 1 --2- 172.21.15.16:0/1271406707 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb778102790 0x7fb778074a30 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:49.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:49.920+0000 7fb7805e0640 1 --2- 172.21.15.16:0/1271406707 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb778101de0 0x7fb7780744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:49.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:49.920+0000 7fb7805e0640 1 -- 172.21.15.16:0/1271406707 >> 172.21.15.16:0/1271406707 conn(0x7fb7780fd8c0 msgr2=0x7fb7780fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:00:49.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:49.920+0000 7fb7805e0640 1 -- 172.21.15.16:0/1271406707 shutdown_connections 2024-09-12T02:00:49.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:49.920+0000 7fb7805e0640 1 -- 172.21.15.16:0/1271406707 wait complete. 2024-09-12T02:00:50.183 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:00:50 smithi016 ceph-mon[30947]: pgmap v302: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T02:00:50.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:00:50 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/3178576980' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-12T02:00:50.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:00:50 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/1271406707' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-12T02:00:50.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:00:50 smithi184 ceph-mon[35593]: pgmap v302: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T02:00:50.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:00:50 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/3178576980' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-12T02:00:50.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:00:50 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/1271406707' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-12T02:00:50.418 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:00:50 smithi033 ceph-mon[35332]: pgmap v302: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T02:00:50.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:00:50 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/3178576980' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-12T02:00:50.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:00:50 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/1271406707' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-12T02:00:50.563 INFO:tasks.cephadm.ceph_manager.ceph:need seq 317827579925 got 317827579927 for osd.11 2024-09-12T02:00:50.563 DEBUG:teuthology.parallel:result is None 2024-09-12T02:00:51.173 INFO:tasks.cephadm.ceph_manager.ceph:need seq 236223201314 got 236223201318 for osd.8 2024-09-12T02:00:51.174 DEBUG:teuthology.parallel:result is None 2024-09-12T02:00:51.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:51.610+0000 7f41b22b9640 1 -- 172.21.15.16:0/3209454391 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41ac1036e0 msgr2=0x7f41ac107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:51.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:51.610+0000 7f41b22b9640 1 --2- 172.21.15.16:0/3209454391 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41ac1036e0 0x7f41ac107b90 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f4194009930 tx=0x7f419402a420 comp rx=0 tx=0).stop 2024-09-12T02:00:51.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:51.610+0000 7f41b22b9640 1 -- 172.21.15.16:0/3209454391 shutdown_connections 2024-09-12T02:00:51.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:51.610+0000 7f41b22b9640 1 --2- 172.21.15.16:0/3209454391 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f41ac1082a0 0x7f41ac10ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:51.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:51.610+0000 7f41b22b9640 1 --2- 172.21.15.16:0/3209454391 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41ac1036e0 0x7f41ac107b90 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:51.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:51.610+0000 7f41b22b9640 1 --2- 172.21.15.16:0/3209454391 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f41ac102d30 0x7f41ac103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:51.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:51.610+0000 7f41b22b9640 1 -- 172.21.15.16:0/3209454391 >> 172.21.15.16:0/3209454391 conn(0x7f41ac0fc7d0 msgr2=0x7f41ac0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:00:51.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:51.610+0000 7f41b22b9640 1 -- 172.21.15.16:0/3209454391 shutdown_connections 2024-09-12T02:00:51.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:51.610+0000 7f41b22b9640 1 -- 172.21.15.16:0/3209454391 wait complete. 2024-09-12T02:00:51.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:51.610+0000 7f41b22b9640 1 Processor -- start 2024-09-12T02:00:51.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:51.611+0000 7f41b22b9640 1 -- start start 2024-09-12T02:00:51.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:51.611+0000 7f41b22b9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41ac102d30 0x7f41ac198620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:51.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:51.611+0000 7f41b22b9640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f41ac1036e0 0x7f41ac198b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:51.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:51.611+0000 7f41b22b9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f41ac1082a0 0x7f41ac19cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:51.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:51.611+0000 7f41b22b9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f41ac1120a0 con 0x7f41ac102d30 2024-09-12T02:00:51.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:51.611+0000 7f41b22b9640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f41ac111f20 con 0x7f41ac1082a0 2024-09-12T02:00:51.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:51.611+0000 7f41b22b9640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f41ac112220 con 0x7f41ac1036e0 2024-09-12T02:00:51.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:51.611+0000 7f41abfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41ac102d30 0x7f41ac198620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:51.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:51.611+0000 7f41abfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41ac102d30 0x7f41ac198620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39586/0 (socket says 172.21.15.16:39586) 2024-09-12T02:00:51.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:51.611+0000 7f41abfff640 1 -- 172.21.15.16:0/2489787948 learned_addr learned my addr 172.21.15.16:0/2489787948 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:00:51.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:51.611+0000 7f41a3fff640 1 --2- 172.21.15.16:0/2489787948 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f41ac1036e0 0x7f41ac198b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:51.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:51.611+0000 7f41b0856640 1 --2- 172.21.15.16:0/2489787948 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f41ac1082a0 0x7f41ac19cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:51.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:51.611+0000 7f41abfff640 1 -- 172.21.15.16:0/2489787948 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f41ac1036e0 msgr2=0x7f41ac198b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:51.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:51.611+0000 7f41abfff640 1 --2- 172.21.15.16:0/2489787948 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f41ac1036e0 0x7f41ac198b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:51.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:51.611+0000 7f41abfff640 1 -- 172.21.15.16:0/2489787948 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f41ac1082a0 msgr2=0x7f41ac19cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:51.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:51.611+0000 7f41abfff640 1 --2- 172.21.15.16:0/2489787948 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f41ac1082a0 0x7f41ac19cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:51.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:51.611+0000 7f41abfff640 1 -- 172.21.15.16:0/2489787948 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41ac19d5d0 con 0x7f41ac102d30 2024-09-12T02:00:51.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:51.611+0000 7f41abfff640 1 --2- 172.21.15.16:0/2489787948 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41ac102d30 0x7f41ac198620 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f419c00e9f0 tx=0x7f419c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:00:51.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:51.612+0000 7f41a9ffb640 1 -- 172.21.15.16:0/2489787948 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f419c00cd80 con 0x7f41ac102d30 2024-09-12T02:00:51.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:51.612+0000 7f41a9ffb640 1 -- 172.21.15.16:0/2489787948 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f419c005500 con 0x7f41ac102d30 2024-09-12T02:00:51.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:51.612+0000 7f41b22b9640 1 -- 172.21.15.16:0/2489787948 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41ac19d8c0 con 0x7f41ac102d30 2024-09-12T02:00:51.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:51.612+0000 7f41a9ffb640 1 -- 172.21.15.16:0/2489787948 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f419c010690 con 0x7f41ac102d30 2024-09-12T02:00:51.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:51.612+0000 7f41b22b9640 1 -- 172.21.15.16:0/2489787948 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41ac1a51a0 con 0x7f41ac102d30 2024-09-12T02:00:51.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:51.613+0000 7f41b22b9640 1 -- 172.21.15.16:0/2489787948 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41ac103110 con 0x7f41ac102d30 2024-09-12T02:00:51.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:51.614+0000 7f41a9ffb640 1 -- 172.21.15.16:0/2489787948 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f419c0046c0 con 0x7f41ac102d30 2024-09-12T02:00:51.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:51.615+0000 7f41a9ffb640 1 --2- 172.21.15.16:0/2489787948 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f41980a2c90 0x7f41980a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:51.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:51.615+0000 7f41a3fff640 1 --2- 172.21.15.16:0/2489787948 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f41980a2c90 0x7f41980a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:51.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:51.615+0000 7f41a9ffb640 1 -- 172.21.15.16:0/2489787948 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f419c0cf450 con 0x7f41ac102d30 2024-09-12T02:00:51.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:51.615+0000 7f41a3fff640 1 --2- 172.21.15.16:0/2489787948 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f41980a2c90 0x7f41980a5150 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f4194005940 tx=0x7f41940058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:00:51.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:51.619+0000 7f41a9ffb640 1 -- 172.21.15.16:0/2489787948 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f419c09c350 con 0x7f41ac102d30 2024-09-12T02:00:51.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:51.796+0000 7f41b22b9640 1 -- 172.21.15.16:0/2489787948 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f41ac10a2b0 con 0x7f41ac102d30 2024-09-12T02:00:51.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:51.797+0000 7f41a9ffb640 1 -- 172.21.15.16:0/2489787948 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f419c09bc50 con 0x7f41ac102d30 2024-09-12T02:00:51.797 INFO:teuthology.orchestra.run.smithi016.stdout:55834574921 2024-09-12T02:00:51.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:51.799+0000 7f41b22b9640 1 -- 172.21.15.16:0/2489787948 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f41980a2c90 msgr2=0x7f41980a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:51.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:51.799+0000 7f41b22b9640 1 --2- 172.21.15.16:0/2489787948 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f41980a2c90 0x7f41980a5150 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f4194005940 tx=0x7f41940058d0 comp rx=0 tx=0).stop 2024-09-12T02:00:51.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:51.799+0000 7f41b22b9640 1 -- 172.21.15.16:0/2489787948 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41ac102d30 msgr2=0x7f41ac198620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:51.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:51.799+0000 7f41b22b9640 1 --2- 172.21.15.16:0/2489787948 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41ac102d30 0x7f41ac198620 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f419c00e9f0 tx=0x7f419c00edf0 comp rx=0 tx=0).stop 2024-09-12T02:00:51.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:51.799+0000 7f41b22b9640 1 -- 172.21.15.16:0/2489787948 shutdown_connections 2024-09-12T02:00:51.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:51.799+0000 7f41b22b9640 1 --2- 172.21.15.16:0/2489787948 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f41980a2c90 0x7f41980a5150 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:51.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:51.799+0000 7f41b22b9640 1 --2- 172.21.15.16:0/2489787948 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f41ac1082a0 0x7f41ac19cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:51.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:51.799+0000 7f41b22b9640 1 --2- 172.21.15.16:0/2489787948 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f41ac1036e0 0x7f41ac198b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:51.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:51.799+0000 7f41b22b9640 1 --2- 172.21.15.16:0/2489787948 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41ac102d30 0x7f41ac198620 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:51.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:51.799+0000 7f41b22b9640 1 -- 172.21.15.16:0/2489787948 >> 172.21.15.16:0/2489787948 conn(0x7f41ac0fc7d0 msgr2=0x7f41ac105e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:00:51.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:51.799+0000 7f41b22b9640 1 -- 172.21.15.16:0/2489787948 shutdown_connections 2024-09-12T02:00:51.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:51.799+0000 7f41b22b9640 1 -- 172.21.15.16:0/2489787948 wait complete. 2024-09-12T02:00:52.047 INFO:tasks.cephadm.ceph_manager.ceph:need seq 287762808859 got 287762808860 for osd.10 2024-09-12T02:00:52.047 DEBUG:teuthology.parallel:result is None 2024-09-12T02:00:52.148 INFO:tasks.cephadm.ceph_manager.ceph:need seq 154618822706 got 154618822708 for osd.5 2024-09-12T02:00:52.148 DEBUG:teuthology.parallel:result is None 2024-09-12T02:00:52.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:00:52 smithi184 ceph-mon[35593]: pgmap v303: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-12T02:00:52.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:00:52 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:00:52.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:00:52 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/2489787948' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-12T02:00:52.418 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:00:52 smithi033 ceph-mon[35332]: pgmap v303: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-12T02:00:52.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:00:52 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:00:52.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:00:52 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/2489787948' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-12T02:00:52.432 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:00:52 smithi016 ceph-mon[30947]: pgmap v303: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-12T02:00:52.432 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:00:52 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:00:52.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:00:52 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/2489787948' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-12T02:00:52.759 INFO:tasks.cephadm.ceph_manager.ceph:need seq 206158430249 got 206158430250 for osd.7 2024-09-12T02:00:52.759 DEBUG:teuthology.parallel:result is None 2024-09-12T02:00:53.512 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738443 got 34359738445 for osd.0 2024-09-12T02:00:53.513 DEBUG:teuthology.parallel:result is None 2024-09-12T02:00:53.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:53.908+0000 7efcdf043640 1 -- 172.21.15.16:0/3762506949 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efcd8101de0 msgr2=0x7efcd81021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:53.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:53.908+0000 7efcdf043640 1 --2- 172.21.15.16:0/3762506949 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efcd8101de0 0x7efcd81021c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7efccc009900 tx=0x7efccc02a410 comp rx=0 tx=0).stop 2024-09-12T02:00:53.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:53.908+0000 7efcdf043640 1 -- 172.21.15.16:0/3762506949 shutdown_connections 2024-09-12T02:00:53.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:53.908+0000 7efcdf043640 1 --2- 172.21.15.16:0/3762506949 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efcd8106f10 0x7efcd810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:53.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:53.908+0000 7efcdf043640 1 --2- 172.21.15.16:0/3762506949 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efcd8102790 0x7efcd81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:53.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:53.908+0000 7efcdf043640 1 --2- 172.21.15.16:0/3762506949 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efcd8101de0 0x7efcd81021c0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:53.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:53.908+0000 7efcdf043640 1 -- 172.21.15.16:0/3762506949 >> 172.21.15.16:0/3762506949 conn(0x7efcd80fd8c0 msgr2=0x7efcd80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:00:53.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:53.908+0000 7efcdf043640 1 -- 172.21.15.16:0/3762506949 shutdown_connections 2024-09-12T02:00:53.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:53.908+0000 7efcdf043640 1 -- 172.21.15.16:0/3762506949 wait complete. 2024-09-12T02:00:53.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:53.908+0000 7efcdf043640 1 Processor -- start 2024-09-12T02:00:53.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:53.908+0000 7efcdf043640 1 -- start start 2024-09-12T02:00:53.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:53.909+0000 7efcdf043640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efcd8101de0 0x7efcd8198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:53.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:53.909+0000 7efcdf043640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efcd8102790 0x7efcd8198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:53.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:53.909+0000 7efcdf043640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efcd8106f10 0x7efcd819cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:53.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:53.909+0000 7efcdf043640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7efcd8111f90 con 0x7efcd8101de0 2024-09-12T02:00:53.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:53.909+0000 7efcdf043640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7efcd8111e10 con 0x7efcd8106f10 2024-09-12T02:00:53.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:53.909+0000 7efcdf043640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7efcd8112110 con 0x7efcd8102790 2024-09-12T02:00:53.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:53.909+0000 7efcdcddf640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efcd8101de0 0x7efcd8198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:53.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:53.909+0000 7efcdcddf640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efcd8101de0 0x7efcd8198510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39610/0 (socket says 172.21.15.16:39610) 2024-09-12T02:00:53.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:53.909+0000 7efcdcddf640 1 -- 172.21.15.16:0/2466466380 learned_addr learned my addr 172.21.15.16:0/2466466380 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:00:53.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:53.909+0000 7efcdd5e0640 1 --2- 172.21.15.16:0/2466466380 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efcd8106f10 0x7efcd819cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:53.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:53.909+0000 7efcd7fff640 1 --2- 172.21.15.16:0/2466466380 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efcd8102790 0x7efcd8198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:53.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:53.909+0000 7efcdcddf640 1 -- 172.21.15.16:0/2466466380 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efcd8102790 msgr2=0x7efcd8198a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:53.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:53.909+0000 7efcdcddf640 1 --2- 172.21.15.16:0/2466466380 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efcd8102790 0x7efcd8198a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:53.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:53.909+0000 7efcdcddf640 1 -- 172.21.15.16:0/2466466380 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efcd8106f10 msgr2=0x7efcd819cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:53.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:53.909+0000 7efcdcddf640 1 --2- 172.21.15.16:0/2466466380 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efcd8106f10 0x7efcd819cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:53.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:53.909+0000 7efcdcddf640 1 -- 172.21.15.16:0/2466466380 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcd819d4c0 con 0x7efcd8101de0 2024-09-12T02:00:53.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:53.909+0000 7efcd7fff640 1 --2- 172.21.15.16:0/2466466380 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efcd8102790 0x7efcd8198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:00:53.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:53.909+0000 7efcdcddf640 1 --2- 172.21.15.16:0/2466466380 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efcd8101de0 0x7efcd8198510 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7efccc006010 tx=0x7efccc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:00:53.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:53.909+0000 7efcd5ffb640 1 -- 172.21.15.16:0/2466466380 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efccc03f030 con 0x7efcd8101de0 2024-09-12T02:00:53.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:53.910+0000 7efcdf043640 1 -- 172.21.15.16:0/2466466380 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcd819d750 con 0x7efcd8101de0 2024-09-12T02:00:53.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:53.910+0000 7efcd5ffb640 1 -- 172.21.15.16:0/2466466380 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efccc003ce0 con 0x7efcd8101de0 2024-09-12T02:00:53.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:53.910+0000 7efcd5ffb640 1 -- 172.21.15.16:0/2466466380 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efccc035070 con 0x7efcd8101de0 2024-09-12T02:00:53.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:53.910+0000 7efcdf043640 1 -- 172.21.15.16:0/2466466380 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efcd81a5090 con 0x7efcd8101de0 2024-09-12T02:00:53.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:53.911+0000 7efcdf043640 1 -- 172.21.15.16:0/2466466380 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efcd81021c0 con 0x7efcd8101de0 2024-09-12T02:00:53.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:53.912+0000 7efcd5ffb640 1 -- 172.21.15.16:0/2466466380 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7efccc042030 con 0x7efcd8101de0 2024-09-12T02:00:53.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:53.912+0000 7efcd5ffb640 1 --2- 172.21.15.16:0/2466466380 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7efcc80a2c10 0x7efcc80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:53.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:53.913+0000 7efcd7fff640 1 --2- 172.21.15.16:0/2466466380 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7efcc80a2c10 0x7efcc80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:53.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:53.913+0000 7efcd5ffb640 1 -- 172.21.15.16:0/2466466380 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7efccc0ef700 con 0x7efcd8101de0 2024-09-12T02:00:53.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:53.913+0000 7efcd7fff640 1 --2- 172.21.15.16:0/2466466380 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7efcc80a2c10 0x7efcc80a50d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7efcc4006fd0 tx=0x7efcc4008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:00:53.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:53.917+0000 7efcd5ffb640 1 -- 172.21.15.16:0/2466466380 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7efccc0bc6c0 con 0x7efcd8101de0 2024-09-12T02:00:54.042 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:00:53 smithi016 ceph-mon[30947]: pgmap v304: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-12T02:00:54.042 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:00:53 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T02:00:54.042 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:00:53 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T02:00:54.042 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:00:53 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T02:00:54.042 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:00:53 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T02:00:54.050 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:00:54.088 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574918 got 55834574921 for osd.1 2024-09-12T02:00:54.089 DEBUG:teuthology.parallel:result is None 2024-09-12T02:00:54.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:54.112+0000 7efcdf043640 1 -- 172.21.15.16:0/2466466380 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7efcd810a0f0 con 0x7efcd8101de0 2024-09-12T02:00:54.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:54.112+0000 7efcd5ffb640 1 -- 172.21.15.16:0/2466466380 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7efccc0bbfc0 con 0x7efcd8101de0 2024-09-12T02:00:54.113 INFO:teuthology.orchestra.run.smithi016.stdout:73014444101 2024-09-12T02:00:54.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:54.114+0000 7efcdf043640 1 -- 172.21.15.16:0/2466466380 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7efcc80a2c10 msgr2=0x7efcc80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:54.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:54.114+0000 7efcdf043640 1 --2- 172.21.15.16:0/2466466380 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7efcc80a2c10 0x7efcc80a50d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7efcc4006fd0 tx=0x7efcc4008040 comp rx=0 tx=0).stop 2024-09-12T02:00:54.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:54.114+0000 7efcdf043640 1 -- 172.21.15.16:0/2466466380 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efcd8101de0 msgr2=0x7efcd8198510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:54.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:54.114+0000 7efcdf043640 1 --2- 172.21.15.16:0/2466466380 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efcd8101de0 0x7efcd8198510 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7efccc006010 tx=0x7efccc0042f0 comp rx=0 tx=0).stop 2024-09-12T02:00:54.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:54.114+0000 7efcdf043640 1 -- 172.21.15.16:0/2466466380 shutdown_connections 2024-09-12T02:00:54.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:54.114+0000 7efcdf043640 1 --2- 172.21.15.16:0/2466466380 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7efcc80a2c10 0x7efcc80a50d0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:54.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:54.114+0000 7efcdf043640 1 --2- 172.21.15.16:0/2466466380 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efcd8106f10 0x7efcd819cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:54.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:54.114+0000 7efcdf043640 1 --2- 172.21.15.16:0/2466466380 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efcd8102790 0x7efcd8198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:54.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:54.114+0000 7efcdf043640 1 --2- 172.21.15.16:0/2466466380 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efcd8101de0 0x7efcd8198510 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:54.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:54.114+0000 7efcdf043640 1 -- 172.21.15.16:0/2466466380 >> 172.21.15.16:0/2466466380 conn(0x7efcd80fd8c0 msgr2=0x7efcd80fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:00:54.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:54.114+0000 7efcdf043640 1 -- 172.21.15.16:0/2466466380 shutdown_connections 2024-09-12T02:00:54.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:54.115+0000 7efcdf043640 1 -- 172.21.15.16:0/2466466380 wait complete. 2024-09-12T02:00:54.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:00:53 smithi184 ceph-mon[35593]: pgmap v304: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-12T02:00:54.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:00:53 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T02:00:54.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:00:53 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T02:00:54.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:00:53 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T02:00:54.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:00:53 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T02:00:54.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:00:53 smithi033 ceph-mon[35332]: pgmap v304: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-12T02:00:54.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:00:53 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T02:00:54.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:00:53 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T02:00:54.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:00:53 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T02:00:54.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:00:53 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T02:00:54.905 INFO:tasks.cephadm.ceph_manager.ceph:need seq 73014444097 got 73014444101 for osd.2 2024-09-12T02:00:54.905 DEBUG:teuthology.parallel:result is None 2024-09-12T02:00:55.052 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:00:54 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/2466466380' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-12T02:00:55.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:00:54 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/2466466380' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-12T02:00:55.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:55.133+0000 7f3b04cc7640 1 -- 172.21.15.16:0/663993323 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b00106f10 msgr2=0x7f3b0010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:55.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:55.133+0000 7f3b04cc7640 1 --2- 172.21.15.16:0/663993323 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b00106f10 0x7f3b0010ea90 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f3af8009ca0 tx=0x7f3af802a8c0 comp rx=0 tx=0).stop 2024-09-12T02:00:55.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:55.133+0000 7f3b04cc7640 1 -- 172.21.15.16:0/663993323 shutdown_connections 2024-09-12T02:00:55.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:55.133+0000 7f3b04cc7640 1 --2- 172.21.15.16:0/663993323 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b00106f10 0x7f3b0010ea90 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:55.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:55.133+0000 7f3b04cc7640 1 --2- 172.21.15.16:0/663993323 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3b00102790 0x7f3b001067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:55.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:55.133+0000 7f3b04cc7640 1 --2- 172.21.15.16:0/663993323 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3b00101de0 0x7f3b001021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:55.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:55.133+0000 7f3b04cc7640 1 -- 172.21.15.16:0/663993323 >> 172.21.15.16:0/663993323 conn(0x7f3b000fd8c0 msgr2=0x7f3b000ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:00:55.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:55.134+0000 7f3b04cc7640 1 -- 172.21.15.16:0/663993323 shutdown_connections 2024-09-12T02:00:55.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:55.134+0000 7f3b04cc7640 1 -- 172.21.15.16:0/663993323 wait complete. 2024-09-12T02:00:55.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:55.134+0000 7f3b04cc7640 1 Processor -- start 2024-09-12T02:00:55.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:55.134+0000 7f3b04cc7640 1 -- start start 2024-09-12T02:00:55.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:55.135+0000 7f3b04cc7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b00101de0 0x7f3b000744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:55.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:55.135+0000 7f3b04cc7640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3b00102790 0x7f3b00074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:55.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:55.135+0000 7f3b04cc7640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3b00106f10 0x7f3b00074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:55.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:55.135+0000 7f3b04cc7640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3b00111fb0 con 0x7f3b00101de0 2024-09-12T02:00:55.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:55.135+0000 7f3b04cc7640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3b00111e30 con 0x7f3b00102790 2024-09-12T02:00:55.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:55.135+0000 7f3b04cc7640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f3b00112130 con 0x7f3b00106f10 2024-09-12T02:00:55.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:55.135+0000 7f3afe59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b00101de0 0x7f3b000744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:55.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:55.135+0000 7f3afe59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b00101de0 0x7f3b000744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39620/0 (socket says 172.21.15.16:39620) 2024-09-12T02:00:55.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:55.135+0000 7f3afe59c640 1 -- 172.21.15.16:0/2158565631 learned_addr learned my addr 172.21.15.16:0/2158565631 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:00:55.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:55.135+0000 7f3afed9d640 1 --2- 172.21.15.16:0/2158565631 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3b00106f10 0x7f3b00074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:55.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:55.135+0000 7f3afdd9b640 1 --2- 172.21.15.16:0/2158565631 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3b00102790 0x7f3b00074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:55.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:55.135+0000 7f3afe59c640 1 -- 172.21.15.16:0/2158565631 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3b00106f10 msgr2=0x7f3b00074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:55.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:55.135+0000 7f3afe59c640 1 --2- 172.21.15.16:0/2158565631 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3b00106f10 0x7f3b00074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:55.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:55.135+0000 7f3afe59c640 1 -- 172.21.15.16:0/2158565631 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3b00102790 msgr2=0x7f3b00074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:55.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:55.135+0000 7f3afe59c640 1 --2- 172.21.15.16:0/2158565631 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3b00102790 0x7f3b00074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:55.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:55.135+0000 7f3afe59c640 1 -- 172.21.15.16:0/2158565631 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b001a6930 con 0x7f3b00101de0 2024-09-12T02:00:55.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:55.135+0000 7f3afe59c640 1 --2- 172.21.15.16:0/2158565631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b00101de0 0x7f3b000744f0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f3af000b4a0 tx=0x7f3af000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:00:55.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:55.135+0000 7f3aef7fe640 1 -- 172.21.15.16:0/2158565631 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3af0014070 con 0x7f3b00101de0 2024-09-12T02:00:55.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:55.135+0000 7f3aef7fe640 1 -- 172.21.15.16:0/2158565631 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3af0010af0 con 0x7f3b00101de0 2024-09-12T02:00:55.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:55.136+0000 7f3b04cc7640 1 -- 172.21.15.16:0/2158565631 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b001a6c20 con 0x7f3b00101de0 2024-09-12T02:00:55.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:55.136+0000 7f3b04cc7640 1 -- 172.21.15.16:0/2158565631 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b001a71b0 con 0x7f3b00101de0 2024-09-12T02:00:55.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:55.136+0000 7f3aef7fe640 1 -- 172.21.15.16:0/2158565631 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3af0017d10 con 0x7f3b00101de0 2024-09-12T02:00:55.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:55.137+0000 7f3b04cc7640 1 -- 172.21.15.16:0/2158565631 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b001021c0 con 0x7f3b00101de0 2024-09-12T02:00:55.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:55.138+0000 7f3aef7fe640 1 -- 172.21.15.16:0/2158565631 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f3af0017070 con 0x7f3b00101de0 2024-09-12T02:00:55.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:55.139+0000 7f3aef7fe640 1 --2- 172.21.15.16:0/2158565631 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f3ae80a2c90 0x7f3ae80a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:55.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:55.139+0000 7f3afdd9b640 1 --2- 172.21.15.16:0/2158565631 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f3ae80a2c90 0x7f3ae80a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:55.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:55.139+0000 7f3aef7fe640 1 -- 172.21.15.16:0/2158565631 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f3af00ce690 con 0x7f3b00101de0 2024-09-12T02:00:55.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:55.139+0000 7f3afdd9b640 1 --2- 172.21.15.16:0/2158565631 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f3ae80a2c90 0x7f3ae80a5150 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f3af4005fd0 tx=0x7f3af4005e20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:00:55.141 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:00:55.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:55.142+0000 7f3aef7fe640 1 -- 172.21.15.16:0/2158565631 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3af009b6a0 con 0x7f3b00101de0 2024-09-12T02:00:55.168 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:00:54 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/2466466380' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-12T02:00:55.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:55.336+0000 7f3b04cc7640 1 -- 172.21.15.16:0/2158565631 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f3b0002d7c0 con 0x7f3b00101de0 2024-09-12T02:00:55.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:55.336+0000 7f3aef7fe640 1 -- 172.21.15.16:0/2158565631 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f3af009afa0 con 0x7f3b00101de0 2024-09-12T02:00:55.337 INFO:teuthology.orchestra.run.smithi016.stdout:133143986234 2024-09-12T02:00:55.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:55.338+0000 7f3b04cc7640 1 -- 172.21.15.16:0/2158565631 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f3ae80a2c90 msgr2=0x7f3ae80a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:55.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:55.338+0000 7f3b04cc7640 1 --2- 172.21.15.16:0/2158565631 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f3ae80a2c90 0x7f3ae80a5150 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f3af4005fd0 tx=0x7f3af4005e20 comp rx=0 tx=0).stop 2024-09-12T02:00:55.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:55.338+0000 7f3b04cc7640 1 -- 172.21.15.16:0/2158565631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b00101de0 msgr2=0x7f3b000744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:55.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:55.338+0000 7f3b04cc7640 1 --2- 172.21.15.16:0/2158565631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b00101de0 0x7f3b000744f0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f3af000b4a0 tx=0x7f3af000b960 comp rx=0 tx=0).stop 2024-09-12T02:00:55.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:55.338+0000 7f3b04cc7640 1 -- 172.21.15.16:0/2158565631 shutdown_connections 2024-09-12T02:00:55.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:55.338+0000 7f3b04cc7640 1 --2- 172.21.15.16:0/2158565631 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f3ae80a2c90 0x7f3ae80a5150 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:55.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:55.338+0000 7f3b04cc7640 1 --2- 172.21.15.16:0/2158565631 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3b00106f10 0x7f3b00074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:55.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:55.338+0000 7f3b04cc7640 1 --2- 172.21.15.16:0/2158565631 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3b00102790 0x7f3b00074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:55.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:55.338+0000 7f3b04cc7640 1 --2- 172.21.15.16:0/2158565631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b00101de0 0x7f3b000744f0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:55.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:55.338+0000 7f3b04cc7640 1 -- 172.21.15.16:0/2158565631 >> 172.21.15.16:0/2158565631 conn(0x7f3b000fd8c0 msgr2=0x7f3b000fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:00:55.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:55.338+0000 7f3b04cc7640 1 -- 172.21.15.16:0/2158565631 shutdown_connections 2024-09-12T02:00:55.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:55.338+0000 7f3b04cc7640 1 -- 172.21.15.16:0/2158565631 wait complete. 2024-09-12T02:00:55.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:55.805+0000 7fe89fe29640 1 -- 172.21.15.16:0/1325772395 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe898102790 msgr2=0x7fe8981067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:55.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:55.805+0000 7fe89fe29640 1 --2- 172.21.15.16:0/1325772395 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe898102790 0x7fe8981067e0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fe888009900 tx=0x7fe88802a410 comp rx=0 tx=0).stop 2024-09-12T02:00:55.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:55.805+0000 7fe89fe29640 1 -- 172.21.15.16:0/1325772395 shutdown_connections 2024-09-12T02:00:55.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:55.805+0000 7fe89fe29640 1 --2- 172.21.15.16:0/1325772395 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe898106f10 0x7fe89810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:55.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:55.805+0000 7fe89fe29640 1 --2- 172.21.15.16:0/1325772395 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe898102790 0x7fe8981067e0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:55.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:55.805+0000 7fe89fe29640 1 --2- 172.21.15.16:0/1325772395 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe898101de0 0x7fe8981021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:55.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:55.805+0000 7fe89fe29640 1 -- 172.21.15.16:0/1325772395 >> 172.21.15.16:0/1325772395 conn(0x7fe8980fd8c0 msgr2=0x7fe8980ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:00:55.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:55.805+0000 7fe89fe29640 1 -- 172.21.15.16:0/1325772395 shutdown_connections 2024-09-12T02:00:55.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:55.805+0000 7fe89fe29640 1 -- 172.21.15.16:0/1325772395 wait complete. 2024-09-12T02:00:55.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:55.806+0000 7fe89fe29640 1 Processor -- start 2024-09-12T02:00:55.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:55.806+0000 7fe89fe29640 1 -- start start 2024-09-12T02:00:55.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:55.806+0000 7fe89fe29640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe898101de0 0x7fe8981984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:55.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:55.806+0000 7fe89fe29640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe898102790 0x7fe8981989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:55.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:55.806+0000 7fe89fe29640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe898106f10 0x7fe89819cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:55.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:55.806+0000 7fe89fe29640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe898111f70 con 0x7fe898102790 2024-09-12T02:00:55.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:55.806+0000 7fe89fe29640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe898111df0 con 0x7fe898101de0 2024-09-12T02:00:55.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:55.806+0000 7fe89fe29640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe8981120f0 con 0x7fe898106f10 2024-09-12T02:00:55.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:55.806+0000 7fe89d3c4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe898102790 0x7fe8981989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:55.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:55.806+0000 7fe89d3c4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe898102790 0x7fe8981989f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39630/0 (socket says 172.21.15.16:39630) 2024-09-12T02:00:55.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:55.806+0000 7fe89d3c4640 1 -- 172.21.15.16:0/745554847 learned_addr learned my addr 172.21.15.16:0/745554847 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:00:55.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:55.807+0000 7fe89dbc5640 1 --2- 172.21.15.16:0/745554847 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe898101de0 0x7fe8981984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:55.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:55.807+0000 7fe89e3c6640 1 --2- 172.21.15.16:0/745554847 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe898106f10 0x7fe89819cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:55.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:55.807+0000 7fe89d3c4640 1 -- 172.21.15.16:0/745554847 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe898106f10 msgr2=0x7fe89819cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:55.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:55.807+0000 7fe89d3c4640 1 --2- 172.21.15.16:0/745554847 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe898106f10 0x7fe89819cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:55.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:55.807+0000 7fe89d3c4640 1 -- 172.21.15.16:0/745554847 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe898101de0 msgr2=0x7fe8981984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:55.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:55.807+0000 7fe89d3c4640 1 --2- 172.21.15.16:0/745554847 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe898101de0 0x7fe8981984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:55.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:55.807+0000 7fe89d3c4640 1 -- 172.21.15.16:0/745554847 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe89819d460 con 0x7fe898102790 2024-09-12T02:00:55.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:55.807+0000 7fe89d3c4640 1 --2- 172.21.15.16:0/745554847 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe898102790 0x7fe8981989f0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fe888006010 tx=0x7fe8880042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:00:55.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:55.807+0000 7fe88effd640 1 -- 172.21.15.16:0/745554847 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe88803f030 con 0x7fe898102790 2024-09-12T02:00:55.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:55.807+0000 7fe88effd640 1 -- 172.21.15.16:0/745554847 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe888003ce0 con 0x7fe898102790 2024-09-12T02:00:55.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:55.807+0000 7fe89fe29640 1 -- 172.21.15.16:0/745554847 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe89819d6f0 con 0x7fe898102790 2024-09-12T02:00:55.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:55.807+0000 7fe88effd640 1 -- 172.21.15.16:0/745554847 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe888035070 con 0x7fe898102790 2024-09-12T02:00:55.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:55.807+0000 7fe89fe29640 1 -- 172.21.15.16:0/745554847 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8981a5030 con 0x7fe898102790 2024-09-12T02:00:55.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:55.808+0000 7fe89fe29640 1 -- 172.21.15.16:0/745554847 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8981021c0 con 0x7fe898102790 2024-09-12T02:00:55.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:55.809+0000 7fe88effd640 1 -- 172.21.15.16:0/745554847 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fe888042030 con 0x7fe898102790 2024-09-12T02:00:55.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:55.810+0000 7fe88effd640 1 --2- 172.21.15.16:0/745554847 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fe8840a2c90 0x7fe8840a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:55.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:55.811+0000 7fe89dbc5640 1 --2- 172.21.15.16:0/745554847 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fe8840a2c90 0x7fe8840a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:55.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:55.811+0000 7fe88effd640 1 -- 172.21.15.16:0/745554847 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fe8880efa90 con 0x7fe898102790 2024-09-12T02:00:55.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:55.811+0000 7fe89dbc5640 1 --2- 172.21.15.16:0/745554847 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fe8840a2c90 0x7fe8840a5150 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fe894005fd0 tx=0x7fe89400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:00:55.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:55.814+0000 7fe88effd640 1 -- 172.21.15.16:0/745554847 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe8880bc9d0 con 0x7fe898102790 2024-09-12T02:00:55.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:55.996+0000 7fe89fe29640 1 -- 172.21.15.16:0/745554847 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7fe89810a150 con 0x7fe898102790 2024-09-12T02:00:55.996 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:00:55 smithi016 ceph-mon[30947]: pgmap v305: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-09-12T02:00:55.996 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:00:55 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/2158565631' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-12T02:00:55.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:55.996+0000 7fe88effd640 1 -- 172.21.15.16:0/745554847 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fe8880bc2d0 con 0x7fe898102790 2024-09-12T02:00:55.997 INFO:teuthology.orchestra.run.smithi016.stdout:266287972386 2024-09-12T02:00:55.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:55.998+0000 7fe89fe29640 1 -- 172.21.15.16:0/745554847 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fe8840a2c90 msgr2=0x7fe8840a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:55.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:55.998+0000 7fe89fe29640 1 --2- 172.21.15.16:0/745554847 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fe8840a2c90 0x7fe8840a5150 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fe894005fd0 tx=0x7fe89400b390 comp rx=0 tx=0).stop 2024-09-12T02:00:55.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:55.998+0000 7fe89fe29640 1 -- 172.21.15.16:0/745554847 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe898102790 msgr2=0x7fe8981989f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:55.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:55.998+0000 7fe89fe29640 1 --2- 172.21.15.16:0/745554847 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe898102790 0x7fe8981989f0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fe888006010 tx=0x7fe8880042f0 comp rx=0 tx=0).stop 2024-09-12T02:00:55.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:55.998+0000 7fe89fe29640 1 -- 172.21.15.16:0/745554847 shutdown_connections 2024-09-12T02:00:55.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:55.998+0000 7fe89fe29640 1 --2- 172.21.15.16:0/745554847 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fe8840a2c90 0x7fe8840a5150 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:55.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:55.998+0000 7fe89fe29640 1 --2- 172.21.15.16:0/745554847 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe898106f10 0x7fe89819cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:55.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:55.998+0000 7fe89fe29640 1 --2- 172.21.15.16:0/745554847 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe898102790 0x7fe8981989f0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:55.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:55.998+0000 7fe89fe29640 1 --2- 172.21.15.16:0/745554847 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe898101de0 0x7fe8981984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:55.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:55.998+0000 7fe89fe29640 1 -- 172.21.15.16:0/745554847 >> 172.21.15.16:0/745554847 conn(0x7fe8980fd8c0 msgr2=0x7fe8980fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:00:55.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:55.999+0000 7fe89fe29640 1 -- 172.21.15.16:0/745554847 shutdown_connections 2024-09-12T02:00:55.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:55.999+0000 7fe89fe29640 1 -- 172.21.15.16:0/745554847 wait complete. 2024-09-12T02:00:56.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:00:55 smithi184 ceph-mon[35593]: pgmap v305: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-09-12T02:00:56.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:00:55 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/2158565631' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-12T02:00:56.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:00:55 smithi033 ceph-mon[35332]: pgmap v305: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-09-12T02:00:56.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:00:55 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/2158565631' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-12T02:00:56.752 INFO:tasks.cephadm.ceph_manager.ceph:need seq 133143986231 got 133143986234 for osd.4 2024-09-12T02:00:56.752 DEBUG:teuthology.parallel:result is None 2024-09-12T02:00:57.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:00:56 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/745554847' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-12T02:00:57.158 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:00:56 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/745554847' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-12T02:00:57.168 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:00:56 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/745554847' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-12T02:00:57.463 INFO:tasks.cephadm.ceph_manager.ceph:need seq 266287972384 got 266287972386 for osd.9 2024-09-12T02:00:57.463 DEBUG:teuthology.parallel:result is None 2024-09-12T02:00:57.463 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-12T02:00:57.464 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-12T02:00:57.762 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:00:58.050 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:00:57 smithi016 ceph-mon[30947]: pgmap v306: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-12T02:00:58.129 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:00:57 smithi184 ceph-mon[35593]: pgmap v306: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-12T02:00:58.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:00:57 smithi033 ceph-mon[35332]: pgmap v306: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-12T02:00:58.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:58.280+0000 7fbd9698c640 1 -- 172.21.15.16:0/38782022 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd90100e20 msgr2=0x7fbd90101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:58.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:58.280+0000 7fbd9698c640 1 --2- 172.21.15.16:0/38782022 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd90100e20 0x7fbd90101200 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fbd80009900 tx=0x7fbd8002a480 comp rx=0 tx=0).stop 2024-09-12T02:00:58.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:58.280+0000 7fbd9698c640 1 -- 172.21.15.16:0/38782022 shutdown_connections 2024-09-12T02:00:58.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:58.280+0000 7fbd9698c640 1 --2- 172.21.15.16:0/38782022 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd90102ef0 0x7fbd9010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:58.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:58.280+0000 7fbd9698c640 1 --2- 172.21.15.16:0/38782022 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbd90102570 0x7fbd901029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:58.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:58.280+0000 7fbd9698c640 1 --2- 172.21.15.16:0/38782022 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd90100e20 0x7fbd90101200 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:58.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:58.281+0000 7fbd9698c640 1 -- 172.21.15.16:0/38782022 >> 172.21.15.16:0/38782022 conn(0x7fbd900fc7d0 msgr2=0x7fbd900febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:00:58.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:58.281+0000 7fbd9698c640 1 -- 172.21.15.16:0/38782022 shutdown_connections 2024-09-12T02:00:58.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:58.281+0000 7fbd9698c640 1 -- 172.21.15.16:0/38782022 wait complete. 2024-09-12T02:00:58.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:58.281+0000 7fbd9698c640 1 Processor -- start 2024-09-12T02:00:58.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:58.281+0000 7fbd9698c640 1 -- start start 2024-09-12T02:00:58.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:58.281+0000 7fbd9698c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd90100e20 0x7fbd901984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:58.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:58.281+0000 7fbd9698c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbd90102570 0x7fbd901989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:58.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:58.281+0000 7fbd9698c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd90102ef0 0x7fbd9019cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:58.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:58.281+0000 7fbd9698c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbd90110f80 con 0x7fbd90102ef0 2024-09-12T02:00:58.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:58.281+0000 7fbd9698c640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbd90110e00 con 0x7fbd90100e20 2024-09-12T02:00:58.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:58.281+0000 7fbd9698c640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fbd90111100 con 0x7fbd90102570 2024-09-12T02:00:58.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:58.282+0000 7fbd94f29640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd90102ef0 0x7fbd9019cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:58.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:58.282+0000 7fbd94f29640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd90102ef0 0x7fbd9019cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39656/0 (socket says 172.21.15.16:39656) 2024-09-12T02:00:58.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:58.282+0000 7fbd94f29640 1 -- 172.21.15.16:0/1492614824 learned_addr learned my addr 172.21.15.16:0/1492614824 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:00:58.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:58.282+0000 7fbd87fff640 1 --2- 172.21.15.16:0/1492614824 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbd90102570 0x7fbd901989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:58.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:58.282+0000 7fbd8ffff640 1 --2- 172.21.15.16:0/1492614824 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd90100e20 0x7fbd901984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:58.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:58.282+0000 7fbd94f29640 1 -- 172.21.15.16:0/1492614824 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbd90102570 msgr2=0x7fbd901989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:58.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:58.282+0000 7fbd94f29640 1 --2- 172.21.15.16:0/1492614824 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbd90102570 0x7fbd901989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:58.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:58.282+0000 7fbd94f29640 1 -- 172.21.15.16:0/1492614824 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd90100e20 msgr2=0x7fbd901984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:58.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:58.282+0000 7fbd94f29640 1 --2- 172.21.15.16:0/1492614824 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd90100e20 0x7fbd901984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:58.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:58.282+0000 7fbd94f29640 1 -- 172.21.15.16:0/1492614824 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd9019d460 con 0x7fbd90102ef0 2024-09-12T02:00:58.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:58.282+0000 7fbd87fff640 1 --2- 172.21.15.16:0/1492614824 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbd90102570 0x7fbd901989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:00:58.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:58.282+0000 7fbd94f29640 1 --2- 172.21.15.16:0/1492614824 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd90102ef0 0x7fbd9019cd80 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fbd8800eb10 tx=0x7fbd8800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:00:58.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:58.282+0000 7fbd8dffb640 1 -- 172.21.15.16:0/1492614824 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd88004b20 con 0x7fbd90102ef0 2024-09-12T02:00:58.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:58.282+0000 7fbd9698c640 1 -- 172.21.15.16:0/1492614824 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd9019d750 con 0x7fbd90102ef0 2024-09-12T02:00:58.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:58.283+0000 7fbd8dffb640 1 -- 172.21.15.16:0/1492614824 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd88004cc0 con 0x7fbd90102ef0 2024-09-12T02:00:58.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:58.283+0000 7fbd8dffb640 1 -- 172.21.15.16:0/1492614824 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd88010760 con 0x7fbd90102ef0 2024-09-12T02:00:58.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:58.283+0000 7fbd9698c640 1 -- 172.21.15.16:0/1492614824 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd901a5030 con 0x7fbd90102ef0 2024-09-12T02:00:58.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:58.284+0000 7fbd9698c640 1 -- 172.21.15.16:0/1492614824 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd901021b0 con 0x7fbd90102ef0 2024-09-12T02:00:58.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:58.285+0000 7fbd8dffb640 1 -- 172.21.15.16:0/1492614824 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fbd88005000 con 0x7fbd90102ef0 2024-09-12T02:00:58.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:58.285+0000 7fbd8dffb640 1 --2- 172.21.15.16:0/1492614824 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fbd7c0a2b70 0x7fbd7c0a5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:58.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:58.286+0000 7fbd8ffff640 1 --2- 172.21.15.16:0/1492614824 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fbd7c0a2b70 0x7fbd7c0a5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:58.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:58.286+0000 7fbd8dffb640 1 -- 172.21.15.16:0/1492614824 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fbd880cf630 con 0x7fbd90102ef0 2024-09-12T02:00:58.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:58.286+0000 7fbd8ffff640 1 --2- 172.21.15.16:0/1492614824 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fbd7c0a2b70 0x7fbd7c0a5030 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fbd80009230 tx=0x7fbd800058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:00:58.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:58.290+0000 7fbd8dffb640 1 -- 172.21.15.16:0/1492614824 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbd8809c530 con 0x7fbd90102ef0 2024-09-12T02:00:58.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:58.465+0000 7fbd9698c640 1 -- 172.21.15.16:0/1492614824 --> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fbd90199810 con 0x7fbd7c0a2b70 2024-09-12T02:00:58.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:58.471+0000 7fbd8dffb640 1 -- 172.21.15.16:0/1492614824 <== mgr.14150 v2:172.21.15.16:6800/3887444820 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258695 (secure 0 0 0) 0x7fbd90199810 con 0x7fbd7c0a2b70 2024-09-12T02:00:58.474 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T02:00:58.475 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-12T02:00:58.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:58.475+0000 7fbd9698c640 1 -- 172.21.15.16:0/1492614824 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fbd7c0a2b70 msgr2=0x7fbd7c0a5030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:58.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:58.475+0000 7fbd9698c640 1 --2- 172.21.15.16:0/1492614824 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fbd7c0a2b70 0x7fbd7c0a5030 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fbd80009230 tx=0x7fbd800058d0 comp rx=0 tx=0).stop 2024-09-12T02:00:58.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:58.475+0000 7fbd9698c640 1 -- 172.21.15.16:0/1492614824 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd90102ef0 msgr2=0x7fbd9019cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:58.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:58.475+0000 7fbd9698c640 1 --2- 172.21.15.16:0/1492614824 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd90102ef0 0x7fbd9019cd80 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fbd8800eb10 tx=0x7fbd8800c2d0 comp rx=0 tx=0).stop 2024-09-12T02:00:58.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:58.475+0000 7fbd9698c640 1 -- 172.21.15.16:0/1492614824 shutdown_connections 2024-09-12T02:00:58.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:58.475+0000 7fbd9698c640 1 --2- 172.21.15.16:0/1492614824 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fbd7c0a2b70 0x7fbd7c0a5030 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:58.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:58.475+0000 7fbd9698c640 1 --2- 172.21.15.16:0/1492614824 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd90102ef0 0x7fbd9019cd80 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:58.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:58.475+0000 7fbd9698c640 1 --2- 172.21.15.16:0/1492614824 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbd90102570 0x7fbd901989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:58.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:58.475+0000 7fbd9698c640 1 --2- 172.21.15.16:0/1492614824 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd90100e20 0x7fbd901984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:58.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:58.475+0000 7fbd9698c640 1 -- 172.21.15.16:0/1492614824 >> 172.21.15.16:0/1492614824 conn(0x7fbd900fc7d0 msgr2=0x7fbd9006a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:00:58.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:58.475+0000 7fbd9698c640 1 -- 172.21.15.16:0/1492614824 shutdown_connections 2024-09-12T02:00:58.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:58.475+0000 7fbd9698c640 1 -- 172.21.15.16:0/1492614824 wait complete. 2024-09-12T02:00:59.123 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":307,"stamp":"2024-09-12T02:00:58.313792+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":654369,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":450,"num_read_kb":337,"num_write":343,"num_write_kb":3047,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":1180848,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":275,"ondisk_log_size":275,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":374288,"kb_used_data":12540,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124567536,"statfs":{"total":1151940427776,"available":1151557156864,"internally_reserved":0,"allocated":12840960,"data_stored":7185771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":370193723},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":60,"num_read_kb":30,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002661"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.572370+0000","last_change":"2024-09-12T01:59:55.503882+0000","last_active":"2024-09-12T02:00:09.572370+0000","last_peered":"2024-09-12T02:00:09.572370+0000","last_clean":"2024-09-12T02:00:09.572370+0000","last_became_active":"2024-09-12T01:59:55.503513+0000","last_became_peered":"2024-09-12T01:59:55.503513+0000","last_unstale":"2024-09-12T02:00:09.572370+0000","last_undegraded":"2024-09-12T02:00:09.572370+0000","last_fullsized":"2024-09-12T02:00:09.572370+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:43:13.098527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.573809+0000","last_change":"2024-09-12T01:59:55.492965+0000","last_active":"2024-09-12T02:00:09.573809+0000","last_peered":"2024-09-12T02:00:09.573809+0000","last_clean":"2024-09-12T02:00:09.573809+0000","last_became_active":"2024-09-12T01:59:55.492613+0000","last_became_peered":"2024-09-12T01:59:55.492613+0000","last_unstale":"2024-09-12T02:00:09.573809+0000","last_undegraded":"2024-09-12T02:00:09.573809+0000","last_fullsized":"2024-09-12T02:00:09.573809+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:58:20.889770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.573872+0000","last_change":"2024-09-12T01:59:55.497334+0000","last_active":"2024-09-12T02:00:09.573872+0000","last_peered":"2024-09-12T02:00:09.573872+0000","last_clean":"2024-09-12T02:00:09.573872+0000","last_became_active":"2024-09-12T01:59:55.497037+0000","last_became_peered":"2024-09-12T01:59:55.497037+0000","last_unstale":"2024-09-12T02:00:09.573872+0000","last_undegraded":"2024-09-12T02:00:09.573872+0000","last_fullsized":"2024-09-12T02:00:09.573872+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:43:04.082918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.563839+0000","last_change":"2024-09-12T01:59:55.495888+0000","last_active":"2024-09-12T02:00:09.563839+0000","last_peered":"2024-09-12T02:00:09.563839+0000","last_clean":"2024-09-12T02:00:09.563839+0000","last_became_active":"2024-09-12T01:59:55.492684+0000","last_became_peered":"2024-09-12T01:59:55.492684+0000","last_unstale":"2024-09-12T02:00:09.563839+0000","last_undegraded":"2024-09-12T02:00:09.563839+0000","last_fullsized":"2024-09-12T02:00:09.563839+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:15:00.500006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.574662+0000","last_change":"2024-09-12T01:59:55.494613+0000","last_active":"2024-09-12T02:00:09.574662+0000","last_peered":"2024-09-12T02:00:09.574662+0000","last_clean":"2024-09-12T02:00:09.574662+0000","last_became_active":"2024-09-12T01:59:55.494205+0000","last_became_peered":"2024-09-12T01:59:55.494205+0000","last_unstale":"2024-09-12T02:00:09.574662+0000","last_undegraded":"2024-09-12T02:00:09.574662+0000","last_fullsized":"2024-09-12T02:00:09.574662+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:39:14.573280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.622709+0000","last_change":"2024-09-12T01:59:55.496659+0000","last_active":"2024-09-12T02:00:09.622709+0000","last_peered":"2024-09-12T02:00:09.622709+0000","last_clean":"2024-09-12T02:00:09.622709+0000","last_became_active":"2024-09-12T01:59:55.496232+0000","last_became_peered":"2024-09-12T01:59:55.496232+0000","last_unstale":"2024-09-12T02:00:09.622709+0000","last_undegraded":"2024-09-12T02:00:09.622709+0000","last_fullsized":"2024-09-12T02:00:09.622709+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:48:21.544764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.622510+0000","last_change":"2024-09-12T01:59:55.486789+0000","last_active":"2024-09-12T02:00:09.622510+0000","last_peered":"2024-09-12T02:00:09.622510+0000","last_clean":"2024-09-12T02:00:09.622510+0000","last_became_active":"2024-09-12T01:59:55.485921+0000","last_became_peered":"2024-09-12T01:59:55.485921+0000","last_unstale":"2024-09-12T02:00:09.622510+0000","last_undegraded":"2024-09-12T02:00:09.622510+0000","last_fullsized":"2024-09-12T02:00:09.622510+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:21:16.071121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.621086+0000","last_change":"2024-09-12T01:59:55.494525+0000","last_active":"2024-09-12T02:00:09.621086+0000","last_peered":"2024-09-12T02:00:09.621086+0000","last_clean":"2024-09-12T02:00:09.621086+0000","last_became_active":"2024-09-12T01:59:55.493998+0000","last_became_peered":"2024-09-12T01:59:55.493998+0000","last_unstale":"2024-09-12T02:00:09.621086+0000","last_undegraded":"2024-09-12T02:00:09.621086+0000","last_fullsized":"2024-09-12T02:00:09.621086+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:59:24.653660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.622633+0000","last_change":"2024-09-12T01:59:55.501466+0000","last_active":"2024-09-12T02:00:09.622633+0000","last_peered":"2024-09-12T02:00:09.622633+0000","last_clean":"2024-09-12T02:00:09.622633+0000","last_became_active":"2024-09-12T01:59:55.500800+0000","last_became_peered":"2024-09-12T01:59:55.500800+0000","last_unstale":"2024-09-12T02:00:09.622633+0000","last_undegraded":"2024-09-12T02:00:09.622633+0000","last_fullsized":"2024-09-12T02:00:09.622633+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:55:13.704791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.623367+0000","last_change":"2024-09-12T01:59:55.497986+0000","last_active":"2024-09-12T02:00:09.623367+0000","last_peered":"2024-09-12T02:00:09.623367+0000","last_clean":"2024-09-12T02:00:09.623367+0000","last_became_active":"2024-09-12T01:59:55.497691+0000","last_became_peered":"2024-09-12T01:59:55.497691+0000","last_unstale":"2024-09-12T02:00:09.623367+0000","last_undegraded":"2024-09-12T02:00:09.623367+0000","last_fullsized":"2024-09-12T02:00:09.623367+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:37:07.702606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.622801+0000","last_change":"2024-09-12T01:59:55.501339+0000","last_active":"2024-09-12T02:00:09.622801+0000","last_peered":"2024-09-12T02:00:09.622801+0000","last_clean":"2024-09-12T02:00:09.622801+0000","last_became_active":"2024-09-12T01:59:55.500700+0000","last_became_peered":"2024-09-12T01:59:55.500700+0000","last_unstale":"2024-09-12T02:00:09.622801+0000","last_undegraded":"2024-09-12T02:00:09.622801+0000","last_fullsized":"2024-09-12T02:00:09.622801+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:12:15.879183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.563946+0000","last_change":"2024-09-12T01:59:55.495196+0000","last_active":"2024-09-12T02:00:09.563946+0000","last_peered":"2024-09-12T02:00:09.563946+0000","last_clean":"2024-09-12T02:00:09.563946+0000","last_became_active":"2024-09-12T01:59:55.494889+0000","last_became_peered":"2024-09-12T01:59:55.494889+0000","last_unstale":"2024-09-12T02:00:09.563946+0000","last_undegraded":"2024-09-12T02:00:09.563946+0000","last_fullsized":"2024-09-12T02:00:09.563946+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:32:30.621946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.562435+0000","last_change":"2024-09-12T01:59:55.492565+0000","last_active":"2024-09-12T02:00:09.562435+0000","last_peered":"2024-09-12T02:00:09.562435+0000","last_clean":"2024-09-12T02:00:09.562435+0000","last_became_active":"2024-09-12T01:59:55.492168+0000","last_became_peered":"2024-09-12T01:59:55.492168+0000","last_unstale":"2024-09-12T02:00:09.562435+0000","last_undegraded":"2024-09-12T02:00:09.562435+0000","last_fullsized":"2024-09-12T02:00:09.562435+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:39:48.630202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.573885+0000","last_change":"2024-09-12T01:59:55.493152+0000","last_active":"2024-09-12T02:00:09.573885+0000","last_peered":"2024-09-12T02:00:09.573885+0000","last_clean":"2024-09-12T02:00:09.573885+0000","last_became_active":"2024-09-12T01:59:55.492794+0000","last_became_peered":"2024-09-12T01:59:55.492794+0000","last_unstale":"2024-09-12T02:00:09.573885+0000","last_undegraded":"2024-09-12T02:00:09.573885+0000","last_fullsized":"2024-09-12T02:00:09.573885+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:24:27.270502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.573561+0000","last_change":"2024-09-12T01:59:55.494651+0000","last_active":"2024-09-12T02:00:09.573561+0000","last_peered":"2024-09-12T02:00:09.573561+0000","last_clean":"2024-09-12T02:00:09.573561+0000","last_became_active":"2024-09-12T01:59:55.494355+0000","last_became_peered":"2024-09-12T01:59:55.494355+0000","last_unstale":"2024-09-12T02:00:09.573561+0000","last_undegraded":"2024-09-12T02:00:09.573561+0000","last_fullsized":"2024-09-12T02:00:09.573561+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:14:44.364158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.622950+0000","last_change":"2024-09-12T01:59:55.495049+0000","last_active":"2024-09-12T02:00:09.622950+0000","last_peered":"2024-09-12T02:00:09.622950+0000","last_clean":"2024-09-12T02:00:09.622950+0000","last_became_active":"2024-09-12T01:59:55.494688+0000","last_became_peered":"2024-09-12T01:59:55.494688+0000","last_unstale":"2024-09-12T02:00:09.622950+0000","last_undegraded":"2024-09-12T02:00:09.622950+0000","last_fullsized":"2024-09-12T02:00:09.622950+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:13:05.944577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.572458+0000","last_change":"2024-09-12T01:59:55.504146+0000","last_active":"2024-09-12T02:00:09.572458+0000","last_peered":"2024-09-12T02:00:09.572458+0000","last_clean":"2024-09-12T02:00:09.572458+0000","last_became_active":"2024-09-12T01:59:55.503646+0000","last_became_peered":"2024-09-12T01:59:55.503646+0000","last_unstale":"2024-09-12T02:00:09.572458+0000","last_undegraded":"2024-09-12T02:00:09.572458+0000","last_fullsized":"2024-09-12T02:00:09.572458+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:23:51.780795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.565050+0000","last_change":"2024-09-12T01:59:55.489930+0000","last_active":"2024-09-12T02:00:09.565050+0000","last_peered":"2024-09-12T02:00:09.565050+0000","last_clean":"2024-09-12T02:00:09.565050+0000","last_became_active":"2024-09-12T01:59:55.489619+0000","last_became_peered":"2024-09-12T01:59:55.489619+0000","last_unstale":"2024-09-12T02:00:09.565050+0000","last_undegraded":"2024-09-12T02:00:09.565050+0000","last_fullsized":"2024-09-12T02:00:09.565050+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:04:57.307091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.569107+0000","last_change":"2024-09-12T01:59:55.492098+0000","last_active":"2024-09-12T02:00:09.569107+0000","last_peered":"2024-09-12T02:00:09.569107+0000","last_clean":"2024-09-12T02:00:09.569107+0000","last_became_active":"2024-09-12T01:59:55.491370+0000","last_became_peered":"2024-09-12T01:59:55.491370+0000","last_unstale":"2024-09-12T02:00:09.569107+0000","last_undegraded":"2024-09-12T02:00:09.569107+0000","last_fullsized":"2024-09-12T02:00:09.569107+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:25:06.307903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.571940+0000","last_change":"2024-09-12T01:59:55.499938+0000","last_active":"2024-09-12T02:00:09.571940+0000","last_peered":"2024-09-12T02:00:09.571940+0000","last_clean":"2024-09-12T02:00:09.571940+0000","last_became_active":"2024-09-12T01:59:55.499619+0000","last_became_peered":"2024-09-12T01:59:55.499619+0000","last_unstale":"2024-09-12T02:00:09.571940+0000","last_undegraded":"2024-09-12T02:00:09.571940+0000","last_fullsized":"2024-09-12T02:00:09.571940+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:00:17.536120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.562514+0000","last_change":"2024-09-12T01:59:55.486106+0000","last_active":"2024-09-12T02:00:09.562514+0000","last_peered":"2024-09-12T02:00:09.562514+0000","last_clean":"2024-09-12T02:00:09.562514+0000","last_became_active":"2024-09-12T01:59:55.485325+0000","last_became_peered":"2024-09-12T01:59:55.485325+0000","last_unstale":"2024-09-12T02:00:09.562514+0000","last_undegraded":"2024-09-12T02:00:09.562514+0000","last_fullsized":"2024-09-12T02:00:09.562514+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:35:46.446224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.622237+0000","last_change":"2024-09-12T01:59:55.496438+0000","last_active":"2024-09-12T02:00:09.622237+0000","last_peered":"2024-09-12T02:00:09.622237+0000","last_clean":"2024-09-12T02:00:09.622237+0000","last_became_active":"2024-09-12T01:59:55.495885+0000","last_became_peered":"2024-09-12T01:59:55.495885+0000","last_unstale":"2024-09-12T02:00:09.622237+0000","last_undegraded":"2024-09-12T02:00:09.622237+0000","last_fullsized":"2024-09-12T02:00:09.622237+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:15:14.911335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.621139+0000","last_change":"2024-09-12T01:59:55.489350+0000","last_active":"2024-09-12T02:00:09.621139+0000","last_peered":"2024-09-12T02:00:09.621139+0000","last_clean":"2024-09-12T02:00:09.621139+0000","last_became_active":"2024-09-12T01:59:55.489028+0000","last_became_peered":"2024-09-12T01:59:55.489028+0000","last_unstale":"2024-09-12T02:00:09.621139+0000","last_undegraded":"2024-09-12T02:00:09.621139+0000","last_fullsized":"2024-09-12T02:00:09.621139+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:29:23.769004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.566645+0000","last_change":"2024-09-12T01:59:55.494532+0000","last_active":"2024-09-12T02:00:09.566645+0000","last_peered":"2024-09-12T02:00:09.566645+0000","last_clean":"2024-09-12T02:00:09.566645+0000","last_became_active":"2024-09-12T01:59:55.490982+0000","last_became_peered":"2024-09-12T01:59:55.490982+0000","last_unstale":"2024-09-12T02:00:09.566645+0000","last_undegraded":"2024-09-12T02:00:09.566645+0000","last_fullsized":"2024-09-12T02:00:09.566645+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:04:50.633730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.621490+0000","last_change":"2024-09-12T01:59:55.496057+0000","last_active":"2024-09-12T02:00:09.621490+0000","last_peered":"2024-09-12T02:00:09.621490+0000","last_clean":"2024-09-12T02:00:09.621490+0000","last_became_active":"2024-09-12T01:59:55.495710+0000","last_became_peered":"2024-09-12T01:59:55.495710+0000","last_unstale":"2024-09-12T02:00:09.621490+0000","last_undegraded":"2024-09-12T02:00:09.621490+0000","last_fullsized":"2024-09-12T02:00:09.621490+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:49:01.962537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.569100+0000","last_change":"2024-09-12T01:59:55.491594+0000","last_active":"2024-09-12T02:00:09.569100+0000","last_peered":"2024-09-12T02:00:09.569100+0000","last_clean":"2024-09-12T02:00:09.569100+0000","last_became_active":"2024-09-12T01:59:55.491233+0000","last_became_peered":"2024-09-12T01:59:55.491233+0000","last_unstale":"2024-09-12T02:00:09.569100+0000","last_undegraded":"2024-09-12T02:00:09.569100+0000","last_fullsized":"2024-09-12T02:00:09.569100+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:24:22.066777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.572270+0000","last_change":"2024-09-12T01:59:55.498331+0000","last_active":"2024-09-12T02:00:09.572270+0000","last_peered":"2024-09-12T02:00:09.572270+0000","last_clean":"2024-09-12T02:00:09.572270+0000","last_became_active":"2024-09-12T01:59:55.497983+0000","last_became_peered":"2024-09-12T01:59:55.497983+0000","last_unstale":"2024-09-12T02:00:09.572270+0000","last_undegraded":"2024-09-12T02:00:09.572270+0000","last_fullsized":"2024-09-12T02:00:09.572270+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:37:00.735231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.575358+0000","last_change":"2024-09-12T02:00:09.575358+0000","last_active":"2024-09-12T02:00:09.575358+0000","last_peered":"2024-09-12T02:00:09.575358+0000","last_clean":"2024-09-12T02:00:09.575358+0000","last_became_active":"2024-09-12T02:00:09.574913+0000","last_became_peered":"2024-09-12T02:00:09.574913+0000","last_unstale":"2024-09-12T02:00:09.575358+0000","last_undegraded":"2024-09-12T02:00:09.575358+0000","last_fullsized":"2024-09-12T02:00:09.575358+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:19:53.630453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.573748+0000","last_change":"2024-09-12T01:59:55.492419+0000","last_active":"2024-09-12T02:00:09.573748+0000","last_peered":"2024-09-12T02:00:09.573748+0000","last_clean":"2024-09-12T02:00:09.573748+0000","last_became_active":"2024-09-12T01:59:55.492003+0000","last_became_peered":"2024-09-12T01:59:55.492003+0000","last_unstale":"2024-09-12T02:00:09.573748+0000","last_undegraded":"2024-09-12T02:00:09.573748+0000","last_fullsized":"2024-09-12T02:00:09.573748+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:25:45.774679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.568801+0000","last_change":"2024-09-12T01:59:55.490125+0000","last_active":"2024-09-12T02:00:09.568801+0000","last_peered":"2024-09-12T02:00:09.568801+0000","last_clean":"2024-09-12T02:00:09.568801+0000","last_became_active":"2024-09-12T01:59:55.489636+0000","last_became_peered":"2024-09-12T01:59:55.489636+0000","last_unstale":"2024-09-12T02:00:09.568801+0000","last_undegraded":"2024-09-12T02:00:09.568801+0000","last_fullsized":"2024-09-12T02:00:09.568801+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:41:30.922185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.569135+0000","last_change":"2024-09-12T01:59:55.496322+0000","last_active":"2024-09-12T02:00:09.569135+0000","last_peered":"2024-09-12T02:00:09.569135+0000","last_clean":"2024-09-12T02:00:09.569135+0000","last_became_active":"2024-09-12T01:59:55.496005+0000","last_became_peered":"2024-09-12T01:59:55.496005+0000","last_unstale":"2024-09-12T02:00:09.569135+0000","last_undegraded":"2024-09-12T02:00:09.569135+0000","last_fullsized":"2024-09-12T02:00:09.569135+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:53:39.248715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.572017+0000","last_change":"2024-09-12T01:59:55.497758+0000","last_active":"2024-09-12T02:00:09.572017+0000","last_peered":"2024-09-12T02:00:09.572017+0000","last_clean":"2024-09-12T02:00:09.572017+0000","last_became_active":"2024-09-12T01:59:55.497282+0000","last_became_peered":"2024-09-12T01:59:55.497282+0000","last_unstale":"2024-09-12T02:00:09.572017+0000","last_undegraded":"2024-09-12T02:00:09.572017+0000","last_fullsized":"2024-09-12T02:00:09.572017+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:08:25.509372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.574620+0000","last_change":"2024-09-12T01:59:55.497919+0000","last_active":"2024-09-12T02:00:09.574620+0000","last_peered":"2024-09-12T02:00:09.574620+0000","last_clean":"2024-09-12T02:00:09.574620+0000","last_became_active":"2024-09-12T01:59:55.497609+0000","last_became_peered":"2024-09-12T01:59:55.497609+0000","last_unstale":"2024-09-12T02:00:09.574620+0000","last_undegraded":"2024-09-12T02:00:09.574620+0000","last_fullsized":"2024-09-12T02:00:09.574620+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:10:22.281180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"81'6","reported_seq":82,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:54.624408+0000","last_change":"2024-09-12T01:59:53.473974+0000","last_active":"2024-09-12T02:00:54.624408+0000","last_peered":"2024-09-12T02:00:54.624408+0000","last_clean":"2024-09-12T02:00:54.624408+0000","last_became_active":"2024-09-12T01:59:53.473546+0000","last_became_peered":"2024-09-12T01:59:53.473546+0000","last_unstale":"2024-09-12T02:00:54.624408+0000","last_undegraded":"2024-09-12T02:00:54.624408+0000","last_fullsized":"2024-09-12T02:00:54.624408+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:28:03.532249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":54,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.567217+0000","last_change":"2024-09-12T01:59:55.493197+0000","last_active":"2024-09-12T02:00:09.567217+0000","last_peered":"2024-09-12T02:00:09.567217+0000","last_clean":"2024-09-12T02:00:09.567217+0000","last_became_active":"2024-09-12T01:59:55.492845+0000","last_became_peered":"2024-09-12T01:59:55.492845+0000","last_unstale":"2024-09-12T02:00:09.567217+0000","last_undegraded":"2024-09-12T02:00:09.567217+0000","last_fullsized":"2024-09-12T02:00:09.567217+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:42:50.790896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"80'3","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.622865+0000","last_change":"2024-09-12T01:59:53.471214+0000","last_active":"2024-09-12T02:00:09.622865+0000","last_peered":"2024-09-12T02:00:09.622865+0000","last_clean":"2024-09-12T02:00:09.622865+0000","last_became_active":"2024-09-12T01:59:53.470858+0000","last_became_peered":"2024-09-12T01:59:53.470858+0000","last_unstale":"2024-09-12T02:00:09.622865+0000","last_undegraded":"2024-09-12T02:00:09.622865+0000","last_fullsized":"2024-09-12T02:00:09.622865+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:34:48.730392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"80'2","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.562106+0000","last_change":"2024-09-12T01:59:53.464568+0000","last_active":"2024-09-12T02:00:09.562106+0000","last_peered":"2024-09-12T02:00:09.562106+0000","last_clean":"2024-09-12T02:00:09.562106+0000","last_became_active":"2024-09-12T01:59:53.464168+0000","last_became_peered":"2024-09-12T01:59:53.464168+0000","last_unstale":"2024-09-12T02:00:09.562106+0000","last_undegraded":"2024-09-12T02:00:09.562106+0000","last_fullsized":"2024-09-12T02:00:09.562106+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:41:48.135522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.621151+0000","last_change":"2024-09-12T01:59:55.487177+0000","last_active":"2024-09-12T02:00:09.621151+0000","last_peered":"2024-09-12T02:00:09.621151+0000","last_clean":"2024-09-12T02:00:09.621151+0000","last_became_active":"2024-09-12T01:59:55.486886+0000","last_became_peered":"2024-09-12T01:59:55.486886+0000","last_unstale":"2024-09-12T02:00:09.621151+0000","last_undegraded":"2024-09-12T02:00:09.621151+0000","last_fullsized":"2024-09-12T02:00:09.621151+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:32:40.150071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"80'1","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.567129+0000","last_change":"2024-09-12T01:59:53.474114+0000","last_active":"2024-09-12T02:00:09.567129+0000","last_peered":"2024-09-12T02:00:09.567129+0000","last_clean":"2024-09-12T02:00:09.567129+0000","last_became_active":"2024-09-12T01:59:53.473755+0000","last_became_peered":"2024-09-12T01:59:53.473755+0000","last_unstale":"2024-09-12T02:00:09.567129+0000","last_undegraded":"2024-09-12T02:00:09.567129+0000","last_fullsized":"2024-09-12T02:00:09.567129+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:07:07.021137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.564793+0000","last_change":"2024-09-12T01:59:55.489868+0000","last_active":"2024-09-12T02:00:09.564793+0000","last_peered":"2024-09-12T02:00:09.564793+0000","last_clean":"2024-09-12T02:00:09.564793+0000","last_became_active":"2024-09-12T01:59:55.489537+0000","last_became_peered":"2024-09-12T01:59:55.489537+0000","last_unstale":"2024-09-12T02:00:09.564793+0000","last_undegraded":"2024-09-12T02:00:09.564793+0000","last_fullsized":"2024-09-12T02:00:09.564793+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:18:49.417689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"80'3","reported_seq":74,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:52.621151+0000","last_change":"2024-09-12T01:59:53.471402+0000","last_active":"2024-09-12T02:00:52.621151+0000","last_peered":"2024-09-12T02:00:52.621151+0000","last_clean":"2024-09-12T02:00:52.621151+0000","last_became_active":"2024-09-12T01:59:53.470940+0000","last_became_peered":"2024-09-12T01:59:53.470940+0000","last_unstale":"2024-09-12T02:00:52.621151+0000","last_undegraded":"2024-09-12T02:00:52.621151+0000","last_fullsized":"2024-09-12T02:00:52.621151+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:41:16.853592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":49,"num_read_kb":49,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.623260+0000","last_change":"2024-09-12T01:59:55.499047+0000","last_active":"2024-09-12T02:00:09.623260+0000","last_peered":"2024-09-12T02:00:09.623260+0000","last_clean":"2024-09-12T02:00:09.623260+0000","last_became_active":"2024-09-12T01:59:55.498071+0000","last_became_peered":"2024-09-12T01:59:55.498071+0000","last_unstale":"2024-09-12T02:00:09.623260+0000","last_undegraded":"2024-09-12T02:00:09.623260+0000","last_fullsized":"2024-09-12T02:00:09.623260+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:36:40.106041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"80'2","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.622966+0000","last_change":"2024-09-12T01:59:53.473508+0000","last_active":"2024-09-12T02:00:09.622966+0000","last_peered":"2024-09-12T02:00:09.622966+0000","last_clean":"2024-09-12T02:00:09.622966+0000","last_became_active":"2024-09-12T01:59:53.473063+0000","last_became_peered":"2024-09-12T01:59:53.473063+0000","last_unstale":"2024-09-12T02:00:09.622966+0000","last_undegraded":"2024-09-12T02:00:09.622966+0000","last_fullsized":"2024-09-12T02:00:09.622966+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:55:32.161769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.573961+0000","last_change":"2024-09-12T01:59:55.491721+0000","last_active":"2024-09-12T02:00:09.573961+0000","last_peered":"2024-09-12T02:00:09.573961+0000","last_clean":"2024-09-12T02:00:09.573961+0000","last_became_active":"2024-09-12T01:59:55.491364+0000","last_became_peered":"2024-09-12T01:59:55.491364+0000","last_unstale":"2024-09-12T02:00:09.573961+0000","last_undegraded":"2024-09-12T02:00:09.573961+0000","last_fullsized":"2024-09-12T02:00:09.573961+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:21:20.795420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"80'4","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.567315+0000","last_change":"2024-09-12T01:59:53.921789+0000","last_active":"2024-09-12T02:00:09.567315+0000","last_peered":"2024-09-12T02:00:09.567315+0000","last_clean":"2024-09-12T02:00:09.567315+0000","last_became_active":"2024-09-12T01:59:53.921383+0000","last_became_peered":"2024-09-12T01:59:53.921383+0000","last_unstale":"2024-09-12T02:00:09.567315+0000","last_undegraded":"2024-09-12T02:00:09.567315+0000","last_fullsized":"2024-09-12T02:00:09.567315+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:21:58.792794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.622517+0000","last_change":"2024-09-12T01:59:55.503008+0000","last_active":"2024-09-12T02:00:09.622517+0000","last_peered":"2024-09-12T02:00:09.622517+0000","last_clean":"2024-09-12T02:00:09.622517+0000","last_became_active":"2024-09-12T01:59:55.502538+0000","last_became_peered":"2024-09-12T01:59:55.502538+0000","last_unstale":"2024-09-12T02:00:09.622517+0000","last_undegraded":"2024-09-12T02:00:09.622517+0000","last_fullsized":"2024-09-12T02:00:09.622517+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:51:36.735985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"82'27","reported_seq":155,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:53.622631+0000","last_change":"2024-09-12T01:59:53.470688+0000","last_active":"2024-09-12T02:00:53.622631+0000","last_peered":"2024-09-12T02:00:53.622631+0000","last_clean":"2024-09-12T02:00:53.622631+0000","last_became_active":"2024-09-12T01:59:53.470231+0000","last_became_peered":"2024-09-12T01:59:53.470231+0000","last_unstale":"2024-09-12T02:00:53.622631+0000","last_undegraded":"2024-09-12T02:00:53.622631+0000","last_fullsized":"2024-09-12T02:00:53.622631+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:53:54.439893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59764,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":106,"num_read_kb":111,"num_write":28,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.575173+0000","last_change":"2024-09-12T01:59:55.495343+0000","last_active":"2024-09-12T02:00:09.575173+0000","last_peered":"2024-09-12T02:00:09.575173+0000","last_clean":"2024-09-12T02:00:09.575173+0000","last_became_active":"2024-09-12T01:59:55.495006+0000","last_became_peered":"2024-09-12T01:59:55.495006+0000","last_unstale":"2024-09-12T02:00:09.575173+0000","last_undegraded":"2024-09-12T02:00:09.575173+0000","last_fullsized":"2024-09-12T02:00:09.575173+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:32:20.932551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'6","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.569082+0000","last_change":"2024-09-12T01:59:53.468625+0000","last_active":"2024-09-12T02:00:09.569082+0000","last_peered":"2024-09-12T02:00:09.569082+0000","last_clean":"2024-09-12T02:00:09.569082+0000","last_became_active":"2024-09-12T01:59:53.468015+0000","last_became_peered":"2024-09-12T01:59:53.468015+0000","last_unstale":"2024-09-12T02:00:09.569082+0000","last_undegraded":"2024-09-12T02:00:09.569082+0000","last_fullsized":"2024-09-12T02:00:09.569082+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:49:34.842239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.562155+0000","last_change":"2024-09-12T01:59:55.487962+0000","last_active":"2024-09-12T02:00:09.562155+0000","last_peered":"2024-09-12T02:00:09.562155+0000","last_clean":"2024-09-12T02:00:09.562155+0000","last_became_active":"2024-09-12T01:59:55.487619+0000","last_became_peered":"2024-09-12T01:59:55.487619+0000","last_unstale":"2024-09-12T02:00:09.562155+0000","last_undegraded":"2024-09-12T02:00:09.562155+0000","last_fullsized":"2024-09-12T02:00:09.562155+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:32:16.882373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"80'3","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.567124+0000","last_change":"2024-09-12T01:59:53.474418+0000","last_active":"2024-09-12T02:00:09.567124+0000","last_peered":"2024-09-12T02:00:09.567124+0000","last_clean":"2024-09-12T02:00:09.567124+0000","last_became_active":"2024-09-12T01:59:53.474059+0000","last_became_peered":"2024-09-12T01:59:53.474059+0000","last_unstale":"2024-09-12T02:00:09.567124+0000","last_undegraded":"2024-09-12T02:00:09.567124+0000","last_fullsized":"2024-09-12T02:00:09.567124+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:10:54.963435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.622740+0000","last_change":"2024-09-12T01:59:55.493394+0000","last_active":"2024-09-12T02:00:09.622740+0000","last_peered":"2024-09-12T02:00:09.622740+0000","last_clean":"2024-09-12T02:00:09.622740+0000","last_became_active":"2024-09-12T01:59:55.491669+0000","last_became_peered":"2024-09-12T01:59:55.491669+0000","last_unstale":"2024-09-12T02:00:09.622740+0000","last_undegraded":"2024-09-12T02:00:09.622740+0000","last_fullsized":"2024-09-12T02:00:09.622740+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:14:40.871154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"80'3","reported_seq":77,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:53.623170+0000","last_change":"2024-09-12T01:59:53.471623+0000","last_active":"2024-09-12T02:00:53.623170+0000","last_peered":"2024-09-12T02:00:53.623170+0000","last_clean":"2024-09-12T02:00:53.623170+0000","last_became_active":"2024-09-12T01:59:53.471294+0000","last_became_peered":"2024-09-12T01:59:53.471294+0000","last_unstale":"2024-09-12T02:00:53.623170+0000","last_undegraded":"2024-09-12T02:00:53.623170+0000","last_fullsized":"2024-09-12T02:00:53.623170+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:51:54.130883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":53,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.621910+0000","last_change":"2024-09-12T01:59:55.493082+0000","last_active":"2024-09-12T02:00:09.621910+0000","last_peered":"2024-09-12T02:00:09.621910+0000","last_clean":"2024-09-12T02:00:09.621910+0000","last_became_active":"2024-09-12T01:59:55.492778+0000","last_became_peered":"2024-09-12T01:59:55.492778+0000","last_unstale":"2024-09-12T02:00:09.621910+0000","last_undegraded":"2024-09-12T02:00:09.621910+0000","last_fullsized":"2024-09-12T02:00:09.621910+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:59:56.466377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'15","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.588050+0000","last_change":"2024-09-12T02:00:09.588050+0000","last_active":"2024-09-12T02:00:09.588050+0000","last_peered":"2024-09-12T02:00:09.588050+0000","last_clean":"2024-09-12T02:00:09.588050+0000","last_became_active":"2024-09-12T02:00:09.575478+0000","last_became_peered":"2024-09-12T02:00:09.575478+0000","last_unstale":"2024-09-12T02:00:09.588050+0000","last_undegraded":"2024-09-12T02:00:09.588050+0000","last_fullsized":"2024-09-12T02:00:09.588050+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:38:45.484167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.566976+0000","last_change":"2024-09-12T01:59:55.497601+0000","last_active":"2024-09-12T02:00:09.566976+0000","last_peered":"2024-09-12T02:00:09.566976+0000","last_clean":"2024-09-12T02:00:09.566976+0000","last_became_active":"2024-09-12T01:59:55.497202+0000","last_became_peered":"2024-09-12T01:59:55.497202+0000","last_unstale":"2024-09-12T02:00:09.566976+0000","last_undegraded":"2024-09-12T02:00:09.566976+0000","last_fullsized":"2024-09-12T02:00:09.566976+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:22:22.850736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"80'2","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.575359+0000","last_change":"2024-09-12T01:59:53.467575+0000","last_active":"2024-09-12T02:00:09.575359+0000","last_peered":"2024-09-12T02:00:09.575359+0000","last_clean":"2024-09-12T02:00:09.575359+0000","last_became_active":"2024-09-12T01:59:53.467146+0000","last_became_peered":"2024-09-12T01:59:53.467146+0000","last_unstale":"2024-09-12T02:00:09.575359+0000","last_undegraded":"2024-09-12T02:00:09.575359+0000","last_fullsized":"2024-09-12T02:00:09.575359+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:07:25.962496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.572034+0000","last_change":"2024-09-12T01:59:55.498297+0000","last_active":"2024-09-12T02:00:09.572034+0000","last_peered":"2024-09-12T02:00:09.572034+0000","last_clean":"2024-09-12T02:00:09.572034+0000","last_became_active":"2024-09-12T01:59:55.497367+0000","last_became_peered":"2024-09-12T01:59:55.497367+0000","last_unstale":"2024-09-12T02:00:09.572034+0000","last_undegraded":"2024-09-12T02:00:09.572034+0000","last_fullsized":"2024-09-12T02:00:09.572034+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:58:32.717710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"80'2","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.562292+0000","last_change":"2024-09-12T01:59:53.466664+0000","last_active":"2024-09-12T02:00:09.562292+0000","last_peered":"2024-09-12T02:00:09.562292+0000","last_clean":"2024-09-12T02:00:09.562292+0000","last_became_active":"2024-09-12T01:59:53.466318+0000","last_became_peered":"2024-09-12T01:59:53.466318+0000","last_unstale":"2024-09-12T02:00:09.562292+0000","last_undegraded":"2024-09-12T02:00:09.562292+0000","last_fullsized":"2024-09-12T02:00:09.562292+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:12:13.541420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.622804+0000","last_change":"2024-09-12T01:59:55.490420+0000","last_active":"2024-09-12T02:00:09.622804+0000","last_peered":"2024-09-12T02:00:09.622804+0000","last_clean":"2024-09-12T02:00:09.622804+0000","last_became_active":"2024-09-12T01:59:55.489757+0000","last_became_peered":"2024-09-12T01:59:55.489757+0000","last_unstale":"2024-09-12T02:00:09.622804+0000","last_undegraded":"2024-09-12T02:00:09.622804+0000","last_fullsized":"2024-09-12T02:00:09.622804+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:34:43.117142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'1","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.567185+0000","last_change":"2024-09-12T01:59:53.466399+0000","last_active":"2024-09-12T02:00:09.567185+0000","last_peered":"2024-09-12T02:00:09.567185+0000","last_clean":"2024-09-12T02:00:09.567185+0000","last_became_active":"2024-09-12T01:59:53.465982+0000","last_became_peered":"2024-09-12T01:59:53.465982+0000","last_unstale":"2024-09-12T02:00:09.567185+0000","last_undegraded":"2024-09-12T02:00:09.567185+0000","last_fullsized":"2024-09-12T02:00:09.567185+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:18:02.015040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"82'1","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.622907+0000","last_change":"2024-09-12T01:59:55.491527+0000","last_active":"2024-09-12T02:00:09.622907+0000","last_peered":"2024-09-12T02:00:09.622907+0000","last_clean":"2024-09-12T02:00:09.622907+0000","last_became_active":"2024-09-12T01:59:55.491165+0000","last_became_peered":"2024-09-12T01:59:55.491165+0000","last_unstale":"2024-09-12T02:00:09.622907+0000","last_undegraded":"2024-09-12T02:00:09.622907+0000","last_fullsized":"2024-09-12T02:00:09.622907+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:57:05.506384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"80'3","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.569187+0000","last_change":"2024-09-12T01:59:54.075752+0000","last_active":"2024-09-12T02:00:09.569187+0000","last_peered":"2024-09-12T02:00:09.569187+0000","last_clean":"2024-09-12T02:00:09.569187+0000","last_became_active":"2024-09-12T01:59:54.075387+0000","last_became_peered":"2024-09-12T01:59:54.075387+0000","last_unstale":"2024-09-12T02:00:09.569187+0000","last_undegraded":"2024-09-12T02:00:09.569187+0000","last_fullsized":"2024-09-12T02:00:09.569187+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:21:15.980591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.622907+0000","last_change":"2024-09-12T01:59:55.500935+0000","last_active":"2024-09-12T02:00:09.622907+0000","last_peered":"2024-09-12T02:00:09.622907+0000","last_clean":"2024-09-12T02:00:09.622907+0000","last_became_active":"2024-09-12T01:59:55.500206+0000","last_became_peered":"2024-09-12T01:59:55.500206+0000","last_unstale":"2024-09-12T02:00:09.622907+0000","last_undegraded":"2024-09-12T02:00:09.622907+0000","last_fullsized":"2024-09-12T02:00:09.622907+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:19:59.937701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":333,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.623202+0000","last_change":"2024-09-12T01:58:47.553853+0000","last_active":"2024-09-12T02:00:09.623202+0000","last_peered":"2024-09-12T02:00:09.623202+0000","last_clean":"2024-09-12T02:00:09.623202+0000","last_became_active":"2024-09-12T01:58:47.553357+0000","last_became_peered":"2024-09-12T01:58:47.553357+0000","last_unstale":"2024-09-12T02:00:09.623202+0000","last_undegraded":"2024-09-12T02:00:09.623202+0000","last_fullsized":"2024-09-12T02:00:09.623202+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:55:26.917733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:55:26.917733+0000","last_clean_scrub_stamp":"2024-09-12T01:55:26.917733+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:58:26.081766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"80'3","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.563615+0000","last_change":"2024-09-12T01:59:53.467907+0000","last_active":"2024-09-12T02:00:09.563615+0000","last_peered":"2024-09-12T02:00:09.563615+0000","last_clean":"2024-09-12T02:00:09.563615+0000","last_became_active":"2024-09-12T01:59:53.467164+0000","last_became_peered":"2024-09-12T01:59:53.467164+0000","last_unstale":"2024-09-12T02:00:09.563615+0000","last_undegraded":"2024-09-12T02:00:09.563615+0000","last_fullsized":"2024-09-12T02:00:09.563615+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:09:35.242708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.621290+0000","last_change":"2024-09-12T01:59:55.495109+0000","last_active":"2024-09-12T02:00:09.621290+0000","last_peered":"2024-09-12T02:00:09.621290+0000","last_clean":"2024-09-12T02:00:09.621290+0000","last_became_active":"2024-09-12T01:59:55.494791+0000","last_became_peered":"2024-09-12T01:59:55.494791+0000","last_unstale":"2024-09-12T02:00:09.621290+0000","last_undegraded":"2024-09-12T02:00:09.621290+0000","last_fullsized":"2024-09-12T02:00:09.621290+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:22:43.882050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"80'3","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.623115+0000","last_change":"2024-09-12T01:59:53.473286+0000","last_active":"2024-09-12T02:00:09.623115+0000","last_peered":"2024-09-12T02:00:09.623115+0000","last_clean":"2024-09-12T02:00:09.623115+0000","last_became_active":"2024-09-12T01:59:53.472976+0000","last_became_peered":"2024-09-12T01:59:53.472976+0000","last_unstale":"2024-09-12T02:00:09.623115+0000","last_undegraded":"2024-09-12T02:00:09.623115+0000","last_fullsized":"2024-09-12T02:00:09.623115+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:32:39.604173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.571159+0000","last_change":"2024-09-12T02:00:09.571159+0000","last_active":"2024-09-12T02:00:09.571159+0000","last_peered":"2024-09-12T02:00:09.571159+0000","last_clean":"2024-09-12T02:00:09.571159+0000","last_became_active":"2024-09-12T02:00:09.570672+0000","last_became_peered":"2024-09-12T02:00:09.570672+0000","last_unstale":"2024-09-12T02:00:09.571159+0000","last_undegraded":"2024-09-12T02:00:09.571159+0000","last_fullsized":"2024-09-12T02:00:09.571159+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:04:10.808942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"80'4","reported_seq":76,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:53.622299+0000","last_change":"2024-09-12T01:59:53.921856+0000","last_active":"2024-09-12T02:00:53.622299+0000","last_peered":"2024-09-12T02:00:53.622299+0000","last_clean":"2024-09-12T02:00:53.622299+0000","last_became_active":"2024-09-12T01:59:53.920672+0000","last_became_peered":"2024-09-12T01:59:53.920672+0000","last_unstale":"2024-09-12T02:00:53.622299+0000","last_undegraded":"2024-09-12T02:00:53.622299+0000","last_fullsized":"2024-09-12T02:00:53.622299+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:42:33.876493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":50,"num_read_kb":50,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.571600+0000","last_change":"2024-09-12T02:00:09.571185+0000","last_active":"2024-09-12T02:00:09.571600+0000","last_peered":"2024-09-12T02:00:09.571600+0000","last_clean":"2024-09-12T02:00:09.571600+0000","last_became_active":"2024-09-12T02:00:09.570769+0000","last_became_peered":"2024-09-12T02:00:09.570769+0000","last_unstale":"2024-09-12T02:00:09.571600+0000","last_undegraded":"2024-09-12T02:00:09.571600+0000","last_fullsized":"2024-09-12T02:00:09.571600+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:34:17.230435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.2","version":"80'1","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.622691+0000","last_change":"2024-09-12T01:59:53.472072+0000","last_active":"2024-09-12T02:00:09.622691+0000","last_peered":"2024-09-12T02:00:09.622691+0000","last_clean":"2024-09-12T02:00:09.622691+0000","last_became_active":"2024-09-12T01:59:53.471088+0000","last_became_peered":"2024-09-12T01:59:53.471088+0000","last_unstale":"2024-09-12T02:00:09.622691+0000","last_undegraded":"2024-09-12T02:00:09.622691+0000","last_fullsized":"2024-09-12T02:00:09.622691+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:02:13.879131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.622413+0000","last_change":"2024-09-12T01:59:55.502843+0000","last_active":"2024-09-12T02:00:09.622413+0000","last_peered":"2024-09-12T02:00:09.622413+0000","last_clean":"2024-09-12T02:00:09.622413+0000","last_became_active":"2024-09-12T01:59:55.499296+0000","last_became_peered":"2024-09-12T01:59:55.499296+0000","last_unstale":"2024-09-12T02:00:09.622413+0000","last_undegraded":"2024-09-12T02:00:09.622413+0000","last_fullsized":"2024-09-12T02:00:09.622413+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:04:21.787370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.575061+0000","last_change":"2024-09-12T02:00:09.575061+0000","last_active":"2024-09-12T02:00:09.575061+0000","last_peered":"2024-09-12T02:00:09.575061+0000","last_clean":"2024-09-12T02:00:09.575061+0000","last_became_active":"2024-09-12T02:00:09.574653+0000","last_became_peered":"2024-09-12T02:00:09.574653+0000","last_unstale":"2024-09-12T02:00:09.575061+0000","last_undegraded":"2024-09-12T02:00:09.575061+0000","last_fullsized":"2024-09-12T02:00:09.575061+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:32:27.931211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"82'1","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.622315+0000","last_change":"2024-09-12T01:59:55.494162+0000","last_active":"2024-09-12T02:00:09.622315+0000","last_peered":"2024-09-12T02:00:09.622315+0000","last_clean":"2024-09-12T02:00:09.622315+0000","last_became_active":"2024-09-12T01:59:55.493802+0000","last_became_peered":"2024-09-12T01:59:55.493802+0000","last_unstale":"2024-09-12T02:00:09.622315+0000","last_undegraded":"2024-09-12T02:00:09.622315+0000","last_fullsized":"2024-09-12T02:00:09.622315+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:22:16.302683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"80'1","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.575278+0000","last_change":"2024-09-12T01:59:53.921669+0000","last_active":"2024-09-12T02:00:09.575278+0000","last_peered":"2024-09-12T02:00:09.575278+0000","last_clean":"2024-09-12T02:00:09.575278+0000","last_became_active":"2024-09-12T01:59:53.921328+0000","last_became_peered":"2024-09-12T01:59:53.921328+0000","last_unstale":"2024-09-12T02:00:09.575278+0000","last_undegraded":"2024-09-12T02:00:09.575278+0000","last_fullsized":"2024-09-12T02:00:09.575278+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:46:33.313726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.562329+0000","last_change":"2024-09-12T01:59:55.488986+0000","last_active":"2024-09-12T02:00:09.562329+0000","last_peered":"2024-09-12T02:00:09.562329+0000","last_clean":"2024-09-12T02:00:09.562329+0000","last_became_active":"2024-09-12T01:59:55.488632+0000","last_became_peered":"2024-09-12T01:59:55.488632+0000","last_unstale":"2024-09-12T02:00:09.562329+0000","last_undegraded":"2024-09-12T02:00:09.562329+0000","last_fullsized":"2024-09-12T02:00:09.562329+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:20:57.596976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"82'10","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:20.501181+0000","last_change":"2024-09-12T01:59:53.467648+0000","last_active":"2024-09-12T02:00:20.501181+0000","last_peered":"2024-09-12T02:00:20.501181+0000","last_clean":"2024-09-12T02:00:20.501181+0000","last_became_active":"2024-09-12T01:59:53.467255+0000","last_became_peered":"2024-09-12T01:59:53.467255+0000","last_unstale":"2024-09-12T02:00:20.501181+0000","last_undegraded":"2024-09-12T02:00:20.501181+0000","last_fullsized":"2024-09-12T02:00:20.501181+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:37:09.897388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.621489+0000","last_change":"2024-09-12T01:59:55.487603+0000","last_active":"2024-09-12T02:00:09.621489+0000","last_peered":"2024-09-12T02:00:09.621489+0000","last_clean":"2024-09-12T02:00:09.621489+0000","last_became_active":"2024-09-12T01:59:55.487334+0000","last_became_peered":"2024-09-12T01:59:55.487334+0000","last_unstale":"2024-09-12T02:00:09.621489+0000","last_undegraded":"2024-09-12T02:00:09.621489+0000","last_fullsized":"2024-09-12T02:00:09.621489+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:23:16.669515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'4","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.563870+0000","last_change":"2024-09-12T01:59:54.076156+0000","last_active":"2024-09-12T02:00:09.563870+0000","last_peered":"2024-09-12T02:00:09.563870+0000","last_clean":"2024-09-12T02:00:09.563870+0000","last_became_active":"2024-09-12T01:59:54.075794+0000","last_became_peered":"2024-09-12T01:59:54.075794+0000","last_unstale":"2024-09-12T02:00:09.563870+0000","last_undegraded":"2024-09-12T02:00:09.563870+0000","last_fullsized":"2024-09-12T02:00:09.563870+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:06:17.431696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.573860+0000","last_change":"2024-09-12T01:59:55.494604+0000","last_active":"2024-09-12T02:00:09.573860+0000","last_peered":"2024-09-12T02:00:09.573860+0000","last_clean":"2024-09-12T02:00:09.573860+0000","last_became_active":"2024-09-12T01:59:55.494300+0000","last_became_peered":"2024-09-12T01:59:55.494300+0000","last_unstale":"2024-09-12T02:00:09.573860+0000","last_undegraded":"2024-09-12T02:00:09.573860+0000","last_fullsized":"2024-09-12T02:00:09.573860+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:18:56.359299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.622205+0000","last_change":"2024-09-12T01:59:53.921832+0000","last_active":"2024-09-12T02:00:09.622205+0000","last_peered":"2024-09-12T02:00:09.622205+0000","last_clean":"2024-09-12T02:00:09.622205+0000","last_became_active":"2024-09-12T01:59:53.921379+0000","last_became_peered":"2024-09-12T01:59:53.921379+0000","last_unstale":"2024-09-12T02:00:09.622205+0000","last_undegraded":"2024-09-12T02:00:09.622205+0000","last_fullsized":"2024-09-12T02:00:09.622205+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:26:02.277149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.574438+0000","last_change":"2024-09-12T01:59:55.493198+0000","last_active":"2024-09-12T02:00:09.574438+0000","last_peered":"2024-09-12T02:00:09.574438+0000","last_clean":"2024-09-12T02:00:09.574438+0000","last_became_active":"2024-09-12T01:59:55.492834+0000","last_became_peered":"2024-09-12T01:59:55.492834+0000","last_unstale":"2024-09-12T02:00:09.574438+0000","last_undegraded":"2024-09-12T02:00:09.574438+0000","last_fullsized":"2024-09-12T02:00:09.574438+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:58:43.872921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"80'3","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.621258+0000","last_change":"2024-09-12T01:59:53.466528+0000","last_active":"2024-09-12T02:00:09.621258+0000","last_peered":"2024-09-12T02:00:09.621258+0000","last_clean":"2024-09-12T02:00:09.621258+0000","last_became_active":"2024-09-12T01:59:53.466182+0000","last_became_peered":"2024-09-12T01:59:53.466182+0000","last_unstale":"2024-09-12T02:00:09.621258+0000","last_undegraded":"2024-09-12T02:00:09.621258+0000","last_fullsized":"2024-09-12T02:00:09.621258+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:00:26.083539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.572154+0000","last_change":"2024-09-12T01:59:55.494511+0000","last_active":"2024-09-12T02:00:09.572154+0000","last_peered":"2024-09-12T02:00:09.572154+0000","last_clean":"2024-09-12T02:00:09.572154+0000","last_became_active":"2024-09-12T01:59:55.493607+0000","last_became_peered":"2024-09-12T01:59:55.493607+0000","last_unstale":"2024-09-12T02:00:09.572154+0000","last_undegraded":"2024-09-12T02:00:09.572154+0000","last_fullsized":"2024-09-12T02:00:09.572154+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:55:27.434447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"80'1","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.569211+0000","last_change":"2024-09-12T01:59:53.467385+0000","last_active":"2024-09-12T02:00:09.569211+0000","last_peered":"2024-09-12T02:00:09.569211+0000","last_clean":"2024-09-12T02:00:09.569211+0000","last_became_active":"2024-09-12T01:59:53.466997+0000","last_became_peered":"2024-09-12T01:59:53.466997+0000","last_unstale":"2024-09-12T02:00:09.569211+0000","last_undegraded":"2024-09-12T02:00:09.569211+0000","last_fullsized":"2024-09-12T02:00:09.569211+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:54:47.154036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.621293+0000","last_change":"2024-09-12T01:59:55.490757+0000","last_active":"2024-09-12T02:00:09.621293+0000","last_peered":"2024-09-12T02:00:09.621293+0000","last_clean":"2024-09-12T02:00:09.621293+0000","last_became_active":"2024-09-12T01:59:55.490451+0000","last_became_peered":"2024-09-12T01:59:55.490451+0000","last_unstale":"2024-09-12T02:00:09.621293+0000","last_undegraded":"2024-09-12T02:00:09.621293+0000","last_fullsized":"2024-09-12T02:00:09.621293+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:21:26.139049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.574833+0000","last_change":"2024-09-12T01:59:53.472898+0000","last_active":"2024-09-12T02:00:09.574833+0000","last_peered":"2024-09-12T02:00:09.574833+0000","last_clean":"2024-09-12T02:00:09.574833+0000","last_became_active":"2024-09-12T01:59:53.472561+0000","last_became_peered":"2024-09-12T01:59:53.472561+0000","last_unstale":"2024-09-12T02:00:09.574833+0000","last_undegraded":"2024-09-12T02:00:09.574833+0000","last_fullsized":"2024-09-12T02:00:09.574833+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:55:59.933804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.622500+0000","last_change":"2024-09-12T01:59:55.497731+0000","last_active":"2024-09-12T02:00:09.622500+0000","last_peered":"2024-09-12T02:00:09.622500+0000","last_clean":"2024-09-12T02:00:09.622500+0000","last_became_active":"2024-09-12T01:59:55.497356+0000","last_became_peered":"2024-09-12T01:59:55.497356+0000","last_unstale":"2024-09-12T02:00:09.622500+0000","last_undegraded":"2024-09-12T02:00:09.622500+0000","last_fullsized":"2024-09-12T02:00:09.622500+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:57:18.795865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.624453+0000","last_change":"2024-09-12T02:00:09.624453+0000","last_active":"2024-09-12T02:00:09.624453+0000","last_peered":"2024-09-12T02:00:09.624453+0000","last_clean":"2024-09-12T02:00:09.624453+0000","last_became_active":"2024-09-12T02:00:09.624049+0000","last_became_peered":"2024-09-12T02:00:09.624049+0000","last_unstale":"2024-09-12T02:00:09.624453+0000","last_undegraded":"2024-09-12T02:00:09.624453+0000","last_fullsized":"2024-09-12T02:00:09.624453+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:17:40.417275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"80'2","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.575060+0000","last_change":"2024-09-12T01:59:53.470194+0000","last_active":"2024-09-12T02:00:09.575060+0000","last_peered":"2024-09-12T02:00:09.575060+0000","last_clean":"2024-09-12T02:00:09.575060+0000","last_became_active":"2024-09-12T01:59:53.469772+0000","last_became_peered":"2024-09-12T01:59:53.469772+0000","last_unstale":"2024-09-12T02:00:09.575060+0000","last_undegraded":"2024-09-12T02:00:09.575060+0000","last_fullsized":"2024-09-12T02:00:09.575060+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:52:57.709596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.622423+0000","last_change":"2024-09-12T01:59:55.497693+0000","last_active":"2024-09-12T02:00:09.622423+0000","last_peered":"2024-09-12T02:00:09.622423+0000","last_clean":"2024-09-12T02:00:09.622423+0000","last_became_active":"2024-09-12T01:59:55.497380+0000","last_became_peered":"2024-09-12T01:59:55.497380+0000","last_unstale":"2024-09-12T02:00:09.622423+0000","last_undegraded":"2024-09-12T02:00:09.622423+0000","last_fullsized":"2024-09-12T02:00:09.622423+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:42:27.943530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"80'4","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.575274+0000","last_change":"2024-09-12T01:59:53.473044+0000","last_active":"2024-09-12T02:00:09.575274+0000","last_peered":"2024-09-12T02:00:09.575274+0000","last_clean":"2024-09-12T02:00:09.575274+0000","last_became_active":"2024-09-12T01:59:53.472569+0000","last_became_peered":"2024-09-12T01:59:53.472569+0000","last_unstale":"2024-09-12T02:00:09.575274+0000","last_undegraded":"2024-09-12T02:00:09.575274+0000","last_fullsized":"2024-09-12T02:00:09.575274+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:41:29.716836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.573956+0000","last_change":"2024-09-12T01:59:55.497377+0000","last_active":"2024-09-12T02:00:09.573956+0000","last_peered":"2024-09-12T02:00:09.573956+0000","last_clean":"2024-09-12T02:00:09.573956+0000","last_became_active":"2024-09-12T01:59:55.497078+0000","last_became_peered":"2024-09-12T01:59:55.497078+0000","last_unstale":"2024-09-12T02:00:09.573956+0000","last_undegraded":"2024-09-12T02:00:09.573956+0000","last_fullsized":"2024-09-12T02:00:09.573956+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:59:57.362350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.621555+0000","last_change":"2024-09-12T01:59:53.920868+0000","last_active":"2024-09-12T02:00:09.621555+0000","last_peered":"2024-09-12T02:00:09.621555+0000","last_clean":"2024-09-12T02:00:09.621555+0000","last_became_active":"2024-09-12T01:59:53.920481+0000","last_became_peered":"2024-09-12T01:59:53.920481+0000","last_unstale":"2024-09-12T02:00:09.621555+0000","last_undegraded":"2024-09-12T02:00:09.621555+0000","last_fullsized":"2024-09-12T02:00:09.621555+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:49:27.085388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.621438+0000","last_change":"2024-09-12T01:59:55.497703+0000","last_active":"2024-09-12T02:00:09.621438+0000","last_peered":"2024-09-12T02:00:09.621438+0000","last_clean":"2024-09-12T02:00:09.621438+0000","last_became_active":"2024-09-12T01:59:55.497397+0000","last_became_peered":"2024-09-12T01:59:55.497397+0000","last_unstale":"2024-09-12T02:00:09.621438+0000","last_undegraded":"2024-09-12T02:00:09.621438+0000","last_fullsized":"2024-09-12T02:00:09.621438+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:54:02.803203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"80'9","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.565145+0000","last_change":"2024-09-12T01:59:53.469978+0000","last_active":"2024-09-12T02:00:09.565145+0000","last_peered":"2024-09-12T02:00:09.565145+0000","last_clean":"2024-09-12T02:00:09.565145+0000","last_became_active":"2024-09-12T01:59:53.469648+0000","last_became_peered":"2024-09-12T01:59:53.469648+0000","last_unstale":"2024-09-12T02:00:09.565145+0000","last_undegraded":"2024-09-12T02:00:09.565145+0000","last_fullsized":"2024-09-12T02:00:09.565145+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:08:37.720452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":63767,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":324,"num_read_kb":228,"num_write":160,"num_write_kb":107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":191301,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":136,"ondisk_log_size":136,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3563520,"data_stored":3542208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":74,"seq":317827579929,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32068,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713084,"statfs":{"total":95995035648,"available":95962198016,"internally_reserved":0,"allocated":897024,"data_stored":435916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":67,"seq":287762808862,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32732,"kb_used_data":1548,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712420,"statfs":{"total":95995035648,"available":95961518080,"internally_reserved":0,"allocated":1585152,"data_stored":1087049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":62,"seq":266287972387,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":905216,"data_stored":436040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201320,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":897024,"data_stored":435848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430253,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32648,"kb_used_data":1464,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712504,"statfs":{"total":95995035648,"available":95961604096,"internally_reserved":0,"allocated":1499136,"data_stored":1027892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593778,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36216,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708936,"statfs":{"total":95995035648,"available":95957950464,"internally_reserved":0,"allocated":892928,"data_stored":435429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822711,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32660,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712492,"statfs":{"total":95995035648,"available":95961591808,"internally_reserved":0,"allocated":1511424,"data_stored":1027576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986235,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":897024,"data_stored":435866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215168,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":909312,"data_stored":435740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":17,"seq":73014444102,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36300,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708852,"statfs":{"total":95995035648,"available":95957864448,"internally_reserved":0,"allocated":978944,"data_stored":496729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574923,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32116,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713036,"statfs":{"total":95995035648,"available":95962148864,"internally_reserved":0,"allocated":954368,"data_stored":494986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738448,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":913408,"data_stored":436700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":59764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":61507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":61342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T02:00:59.128 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-12T02:00:59.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:00:59 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T02:00:59.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:00:59 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T02:00:59.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:00:59 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:00:59.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:00:59 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:00:59.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:00:59 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T02:00:59.384 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:00:59.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:00:59 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T02:00:59.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:00:59 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T02:00:59.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:00:59 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:00:59.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:00:59 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:00:59.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:00:59 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T02:00:59.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:00:59 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T02:00:59.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:00:59 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T02:00:59.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:00:59 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:00:59.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:00:59 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:00:59.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:00:59 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T02:00:59.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:59.877+0000 7fa6092b5640 1 -- 172.21.15.16:0/2860665093 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa604106a80 msgr2=0x7fa604106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:59.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:59.877+0000 7fa6092b5640 1 --2- 172.21.15.16:0/2860665093 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa604106a80 0x7fa604106e60 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fa5f4009930 tx=0x7fa5f402a420 comp rx=0 tx=0).stop 2024-09-12T02:00:59.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:59.877+0000 7fa6092b5640 1 -- 172.21.15.16:0/2860665093 shutdown_connections 2024-09-12T02:00:59.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:59.877+0000 7fa6092b5640 1 --2- 172.21.15.16:0/2860665093 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa604101420 0x7fa60410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:59.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:59.877+0000 7fa6092b5640 1 --2- 172.21.15.16:0/2860665093 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa604100a80 0x7fa604100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:59.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:59.877+0000 7fa6092b5640 1 --2- 172.21.15.16:0/2860665093 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa604106a80 0x7fa604106e60 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:59.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:59.877+0000 7fa6092b5640 1 -- 172.21.15.16:0/2860665093 >> 172.21.15.16:0/2860665093 conn(0x7fa6040fc7a0 msgr2=0x7fa6040febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:00:59.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:59.877+0000 7fa6092b5640 1 -- 172.21.15.16:0/2860665093 shutdown_connections 2024-09-12T02:00:59.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:59.877+0000 7fa6092b5640 1 -- 172.21.15.16:0/2860665093 wait complete. 2024-09-12T02:00:59.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:59.878+0000 7fa6092b5640 1 Processor -- start 2024-09-12T02:00:59.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:59.878+0000 7fa6092b5640 1 -- start start 2024-09-12T02:00:59.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:59.878+0000 7fa6092b5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa604100a80 0x7fa604198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:59.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:59.878+0000 7fa6092b5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa604101420 0x7fa604198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:59.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:59.878+0000 7fa6092b5640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa604106a80 0x7fa60419cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:59.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:59.878+0000 7fa6092b5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa604111fd0 con 0x7fa604100a80 2024-09-12T02:00:59.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:59.878+0000 7fa6092b5640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa604111e50 con 0x7fa604101420 2024-09-12T02:00:59.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:59.878+0000 7fa6092b5640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa604112150 con 0x7fa604106a80 2024-09-12T02:00:59.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:59.878+0000 7fa602ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa604100a80 0x7fa604198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:59.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:59.878+0000 7fa602ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa604100a80 0x7fa604198510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39668/0 (socket says 172.21.15.16:39668) 2024-09-12T02:00:59.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:59.878+0000 7fa602ffd640 1 -- 172.21.15.16:0/3720078343 learned_addr learned my addr 172.21.15.16:0/3720078343 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:00:59.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:59.878+0000 7fa6037fe640 1 --2- 172.21.15.16:0/3720078343 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa604106a80 0x7fa60419cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:59.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:59.878+0000 7fa5fbfff640 1 --2- 172.21.15.16:0/3720078343 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa604101420 0x7fa604198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:59.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:59.878+0000 7fa602ffd640 1 -- 172.21.15.16:0/3720078343 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa604106a80 msgr2=0x7fa60419cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:59.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:59.878+0000 7fa602ffd640 1 --2- 172.21.15.16:0/3720078343 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa604106a80 0x7fa60419cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:59.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:59.878+0000 7fa602ffd640 1 -- 172.21.15.16:0/3720078343 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa604101420 msgr2=0x7fa604198a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:59.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:59.878+0000 7fa602ffd640 1 --2- 172.21.15.16:0/3720078343 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa604101420 0x7fa604198a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:59.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:59.878+0000 7fa602ffd640 1 -- 172.21.15.16:0/3720078343 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa60419d4c0 con 0x7fa604100a80 2024-09-12T02:00:59.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:59.878+0000 7fa6037fe640 1 --2- 172.21.15.16:0/3720078343 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa604106a80 0x7fa60419cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:00:59.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:59.879+0000 7fa602ffd640 1 --2- 172.21.15.16:0/3720078343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa604100a80 0x7fa604198510 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fa5f4009230 tx=0x7fa5f4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:00:59.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:59.879+0000 7fa600ff9640 1 -- 172.21.15.16:0/3720078343 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5f403f030 con 0x7fa604100a80 2024-09-12T02:00:59.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:59.879+0000 7fa600ff9640 1 -- 172.21.15.16:0/3720078343 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5f4003ad0 con 0x7fa604100a80 2024-09-12T02:00:59.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:59.879+0000 7fa6092b5640 1 -- 172.21.15.16:0/3720078343 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa60419d750 con 0x7fa604100a80 2024-09-12T02:00:59.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:59.879+0000 7fa600ff9640 1 -- 172.21.15.16:0/3720078343 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5f4035070 con 0x7fa604100a80 2024-09-12T02:00:59.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:59.879+0000 7fa6092b5640 1 -- 172.21.15.16:0/3720078343 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6041a5090 con 0x7fa604100a80 2024-09-12T02:00:59.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:59.880+0000 7fa6092b5640 1 -- 172.21.15.16:0/3720078343 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa604100ee0 con 0x7fa604100a80 2024-09-12T02:00:59.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:59.881+0000 7fa600ff9640 1 -- 172.21.15.16:0/3720078343 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fa5f4042030 con 0x7fa604100a80 2024-09-12T02:00:59.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:59.882+0000 7fa600ff9640 1 --2- 172.21.15.16:0/3720078343 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fa5f00a2c90 0x7fa5f00a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:59.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:59.882+0000 7fa5fbfff640 1 --2- 172.21.15.16:0/3720078343 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fa5f00a2c90 0x7fa5f00a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:59.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:59.882+0000 7fa600ff9640 1 -- 172.21.15.16:0/3720078343 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fa5f40ef610 con 0x7fa604100a80 2024-09-12T02:00:59.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:59.882+0000 7fa5fbfff640 1 --2- 172.21.15.16:0/3720078343 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fa5f00a2c90 0x7fa5f00a5150 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa5ec005fd0 tx=0x7fa5ec005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:00:59.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:00:59.886+0000 7fa600ff9640 1 -- 172.21.15.16:0/3720078343 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa5f40bc510 con 0x7fa604100a80 2024-09-12T02:01:00.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:00.057+0000 7fa6092b5640 1 -- 172.21.15.16:0/3720078343 --> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa604062630 con 0x7fa5f00a2c90 2024-09-12T02:01:00.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:00.064+0000 7fa600ff9640 1 -- 172.21.15.16:0/3720078343 <== mgr.14150 v2:172.21.15.16:6800/3887444820 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258695 (secure 0 0 0) 0x7fa604062630 con 0x7fa5f00a2c90 2024-09-12T02:01:00.066 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T02:01:00.066 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-12T02:01:00.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:00.066+0000 7fa6092b5640 1 -- 172.21.15.16:0/3720078343 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fa5f00a2c90 msgr2=0x7fa5f00a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:00.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:00.066+0000 7fa6092b5640 1 --2- 172.21.15.16:0/3720078343 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fa5f00a2c90 0x7fa5f00a5150 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa5ec005fd0 tx=0x7fa5ec005850 comp rx=0 tx=0).stop 2024-09-12T02:01:00.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:00.066+0000 7fa6092b5640 1 -- 172.21.15.16:0/3720078343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa604100a80 msgr2=0x7fa604198510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:00.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:00.066+0000 7fa6092b5640 1 --2- 172.21.15.16:0/3720078343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa604100a80 0x7fa604198510 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fa5f4009230 tx=0x7fa5f4004270 comp rx=0 tx=0).stop 2024-09-12T02:01:00.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:00.067+0000 7fa6092b5640 1 -- 172.21.15.16:0/3720078343 shutdown_connections 2024-09-12T02:01:00.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:00.067+0000 7fa6092b5640 1 --2- 172.21.15.16:0/3720078343 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fa5f00a2c90 0x7fa5f00a5150 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:00.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:00.067+0000 7fa6092b5640 1 --2- 172.21.15.16:0/3720078343 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa604106a80 0x7fa60419cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:00.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:00.067+0000 7fa6092b5640 1 --2- 172.21.15.16:0/3720078343 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa604101420 0x7fa604198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:00.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:00.067+0000 7fa6092b5640 1 --2- 172.21.15.16:0/3720078343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa604100a80 0x7fa604198510 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:00.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:00.067+0000 7fa6092b5640 1 -- 172.21.15.16:0/3720078343 >> 172.21.15.16:0/3720078343 conn(0x7fa6040fc7a0 msgr2=0x7fa6040fd0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:01:00.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:00.067+0000 7fa6092b5640 1 -- 172.21.15.16:0/3720078343 shutdown_connections 2024-09-12T02:01:00.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:00.067+0000 7fa6092b5640 1 -- 172.21.15.16:0/3720078343 wait complete. 2024-09-12T02:01:00.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:01:00 smithi184 ceph-mon[35593]: pgmap v307: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T02:01:00.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:01:00 smithi184 ceph-mon[35593]: from='client.14919 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T02:01:00.418 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:01:00 smithi033 ceph-mon[35332]: pgmap v307: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T02:01:00.418 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:01:00 smithi033 ceph-mon[35332]: from='client.14919 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T02:01:00.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:01:00 smithi016 ceph-mon[30947]: pgmap v307: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T02:01:00.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:01:00 smithi016 ceph-mon[30947]: from='client.14919 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T02:01:00.739 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":307,"stamp":"2024-09-12T02:00:58.313792+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":654369,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":450,"num_read_kb":337,"num_write":343,"num_write_kb":3047,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":1180848,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":275,"ondisk_log_size":275,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":374288,"kb_used_data":12540,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124567536,"statfs":{"total":1151940427776,"available":1151557156864,"internally_reserved":0,"allocated":12840960,"data_stored":7185771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":370193723},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":60,"num_read_kb":30,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002661"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.572370+0000","last_change":"2024-09-12T01:59:55.503882+0000","last_active":"2024-09-12T02:00:09.572370+0000","last_peered":"2024-09-12T02:00:09.572370+0000","last_clean":"2024-09-12T02:00:09.572370+0000","last_became_active":"2024-09-12T01:59:55.503513+0000","last_became_peered":"2024-09-12T01:59:55.503513+0000","last_unstale":"2024-09-12T02:00:09.572370+0000","last_undegraded":"2024-09-12T02:00:09.572370+0000","last_fullsized":"2024-09-12T02:00:09.572370+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:43:13.098527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.573809+0000","last_change":"2024-09-12T01:59:55.492965+0000","last_active":"2024-09-12T02:00:09.573809+0000","last_peered":"2024-09-12T02:00:09.573809+0000","last_clean":"2024-09-12T02:00:09.573809+0000","last_became_active":"2024-09-12T01:59:55.492613+0000","last_became_peered":"2024-09-12T01:59:55.492613+0000","last_unstale":"2024-09-12T02:00:09.573809+0000","last_undegraded":"2024-09-12T02:00:09.573809+0000","last_fullsized":"2024-09-12T02:00:09.573809+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:58:20.889770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.573872+0000","last_change":"2024-09-12T01:59:55.497334+0000","last_active":"2024-09-12T02:00:09.573872+0000","last_peered":"2024-09-12T02:00:09.573872+0000","last_clean":"2024-09-12T02:00:09.573872+0000","last_became_active":"2024-09-12T01:59:55.497037+0000","last_became_peered":"2024-09-12T01:59:55.497037+0000","last_unstale":"2024-09-12T02:00:09.573872+0000","last_undegraded":"2024-09-12T02:00:09.573872+0000","last_fullsized":"2024-09-12T02:00:09.573872+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:43:04.082918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.563839+0000","last_change":"2024-09-12T01:59:55.495888+0000","last_active":"2024-09-12T02:00:09.563839+0000","last_peered":"2024-09-12T02:00:09.563839+0000","last_clean":"2024-09-12T02:00:09.563839+0000","last_became_active":"2024-09-12T01:59:55.492684+0000","last_became_peered":"2024-09-12T01:59:55.492684+0000","last_unstale":"2024-09-12T02:00:09.563839+0000","last_undegraded":"2024-09-12T02:00:09.563839+0000","last_fullsized":"2024-09-12T02:00:09.563839+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:15:00.500006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.574662+0000","last_change":"2024-09-12T01:59:55.494613+0000","last_active":"2024-09-12T02:00:09.574662+0000","last_peered":"2024-09-12T02:00:09.574662+0000","last_clean":"2024-09-12T02:00:09.574662+0000","last_became_active":"2024-09-12T01:59:55.494205+0000","last_became_peered":"2024-09-12T01:59:55.494205+0000","last_unstale":"2024-09-12T02:00:09.574662+0000","last_undegraded":"2024-09-12T02:00:09.574662+0000","last_fullsized":"2024-09-12T02:00:09.574662+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:39:14.573280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.622709+0000","last_change":"2024-09-12T01:59:55.496659+0000","last_active":"2024-09-12T02:00:09.622709+0000","last_peered":"2024-09-12T02:00:09.622709+0000","last_clean":"2024-09-12T02:00:09.622709+0000","last_became_active":"2024-09-12T01:59:55.496232+0000","last_became_peered":"2024-09-12T01:59:55.496232+0000","last_unstale":"2024-09-12T02:00:09.622709+0000","last_undegraded":"2024-09-12T02:00:09.622709+0000","last_fullsized":"2024-09-12T02:00:09.622709+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:48:21.544764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.622510+0000","last_change":"2024-09-12T01:59:55.486789+0000","last_active":"2024-09-12T02:00:09.622510+0000","last_peered":"2024-09-12T02:00:09.622510+0000","last_clean":"2024-09-12T02:00:09.622510+0000","last_became_active":"2024-09-12T01:59:55.485921+0000","last_became_peered":"2024-09-12T01:59:55.485921+0000","last_unstale":"2024-09-12T02:00:09.622510+0000","last_undegraded":"2024-09-12T02:00:09.622510+0000","last_fullsized":"2024-09-12T02:00:09.622510+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:21:16.071121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.621086+0000","last_change":"2024-09-12T01:59:55.494525+0000","last_active":"2024-09-12T02:00:09.621086+0000","last_peered":"2024-09-12T02:00:09.621086+0000","last_clean":"2024-09-12T02:00:09.621086+0000","last_became_active":"2024-09-12T01:59:55.493998+0000","last_became_peered":"2024-09-12T01:59:55.493998+0000","last_unstale":"2024-09-12T02:00:09.621086+0000","last_undegraded":"2024-09-12T02:00:09.621086+0000","last_fullsized":"2024-09-12T02:00:09.621086+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:59:24.653660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.622633+0000","last_change":"2024-09-12T01:59:55.501466+0000","last_active":"2024-09-12T02:00:09.622633+0000","last_peered":"2024-09-12T02:00:09.622633+0000","last_clean":"2024-09-12T02:00:09.622633+0000","last_became_active":"2024-09-12T01:59:55.500800+0000","last_became_peered":"2024-09-12T01:59:55.500800+0000","last_unstale":"2024-09-12T02:00:09.622633+0000","last_undegraded":"2024-09-12T02:00:09.622633+0000","last_fullsized":"2024-09-12T02:00:09.622633+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:55:13.704791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.623367+0000","last_change":"2024-09-12T01:59:55.497986+0000","last_active":"2024-09-12T02:00:09.623367+0000","last_peered":"2024-09-12T02:00:09.623367+0000","last_clean":"2024-09-12T02:00:09.623367+0000","last_became_active":"2024-09-12T01:59:55.497691+0000","last_became_peered":"2024-09-12T01:59:55.497691+0000","last_unstale":"2024-09-12T02:00:09.623367+0000","last_undegraded":"2024-09-12T02:00:09.623367+0000","last_fullsized":"2024-09-12T02:00:09.623367+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:37:07.702606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.622801+0000","last_change":"2024-09-12T01:59:55.501339+0000","last_active":"2024-09-12T02:00:09.622801+0000","last_peered":"2024-09-12T02:00:09.622801+0000","last_clean":"2024-09-12T02:00:09.622801+0000","last_became_active":"2024-09-12T01:59:55.500700+0000","last_became_peered":"2024-09-12T01:59:55.500700+0000","last_unstale":"2024-09-12T02:00:09.622801+0000","last_undegraded":"2024-09-12T02:00:09.622801+0000","last_fullsized":"2024-09-12T02:00:09.622801+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:12:15.879183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.563946+0000","last_change":"2024-09-12T01:59:55.495196+0000","last_active":"2024-09-12T02:00:09.563946+0000","last_peered":"2024-09-12T02:00:09.563946+0000","last_clean":"2024-09-12T02:00:09.563946+0000","last_became_active":"2024-09-12T01:59:55.494889+0000","last_became_peered":"2024-09-12T01:59:55.494889+0000","last_unstale":"2024-09-12T02:00:09.563946+0000","last_undegraded":"2024-09-12T02:00:09.563946+0000","last_fullsized":"2024-09-12T02:00:09.563946+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:32:30.621946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.562435+0000","last_change":"2024-09-12T01:59:55.492565+0000","last_active":"2024-09-12T02:00:09.562435+0000","last_peered":"2024-09-12T02:00:09.562435+0000","last_clean":"2024-09-12T02:00:09.562435+0000","last_became_active":"2024-09-12T01:59:55.492168+0000","last_became_peered":"2024-09-12T01:59:55.492168+0000","last_unstale":"2024-09-12T02:00:09.562435+0000","last_undegraded":"2024-09-12T02:00:09.562435+0000","last_fullsized":"2024-09-12T02:00:09.562435+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:39:48.630202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.573885+0000","last_change":"2024-09-12T01:59:55.493152+0000","last_active":"2024-09-12T02:00:09.573885+0000","last_peered":"2024-09-12T02:00:09.573885+0000","last_clean":"2024-09-12T02:00:09.573885+0000","last_became_active":"2024-09-12T01:59:55.492794+0000","last_became_peered":"2024-09-12T01:59:55.492794+0000","last_unstale":"2024-09-12T02:00:09.573885+0000","last_undegraded":"2024-09-12T02:00:09.573885+0000","last_fullsized":"2024-09-12T02:00:09.573885+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:24:27.270502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.573561+0000","last_change":"2024-09-12T01:59:55.494651+0000","last_active":"2024-09-12T02:00:09.573561+0000","last_peered":"2024-09-12T02:00:09.573561+0000","last_clean":"2024-09-12T02:00:09.573561+0000","last_became_active":"2024-09-12T01:59:55.494355+0000","last_became_peered":"2024-09-12T01:59:55.494355+0000","last_unstale":"2024-09-12T02:00:09.573561+0000","last_undegraded":"2024-09-12T02:00:09.573561+0000","last_fullsized":"2024-09-12T02:00:09.573561+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:14:44.364158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.622950+0000","last_change":"2024-09-12T01:59:55.495049+0000","last_active":"2024-09-12T02:00:09.622950+0000","last_peered":"2024-09-12T02:00:09.622950+0000","last_clean":"2024-09-12T02:00:09.622950+0000","last_became_active":"2024-09-12T01:59:55.494688+0000","last_became_peered":"2024-09-12T01:59:55.494688+0000","last_unstale":"2024-09-12T02:00:09.622950+0000","last_undegraded":"2024-09-12T02:00:09.622950+0000","last_fullsized":"2024-09-12T02:00:09.622950+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:13:05.944577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.572458+0000","last_change":"2024-09-12T01:59:55.504146+0000","last_active":"2024-09-12T02:00:09.572458+0000","last_peered":"2024-09-12T02:00:09.572458+0000","last_clean":"2024-09-12T02:00:09.572458+0000","last_became_active":"2024-09-12T01:59:55.503646+0000","last_became_peered":"2024-09-12T01:59:55.503646+0000","last_unstale":"2024-09-12T02:00:09.572458+0000","last_undegraded":"2024-09-12T02:00:09.572458+0000","last_fullsized":"2024-09-12T02:00:09.572458+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:23:51.780795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.565050+0000","last_change":"2024-09-12T01:59:55.489930+0000","last_active":"2024-09-12T02:00:09.565050+0000","last_peered":"2024-09-12T02:00:09.565050+0000","last_clean":"2024-09-12T02:00:09.565050+0000","last_became_active":"2024-09-12T01:59:55.489619+0000","last_became_peered":"2024-09-12T01:59:55.489619+0000","last_unstale":"2024-09-12T02:00:09.565050+0000","last_undegraded":"2024-09-12T02:00:09.565050+0000","last_fullsized":"2024-09-12T02:00:09.565050+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:04:57.307091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.569107+0000","last_change":"2024-09-12T01:59:55.492098+0000","last_active":"2024-09-12T02:00:09.569107+0000","last_peered":"2024-09-12T02:00:09.569107+0000","last_clean":"2024-09-12T02:00:09.569107+0000","last_became_active":"2024-09-12T01:59:55.491370+0000","last_became_peered":"2024-09-12T01:59:55.491370+0000","last_unstale":"2024-09-12T02:00:09.569107+0000","last_undegraded":"2024-09-12T02:00:09.569107+0000","last_fullsized":"2024-09-12T02:00:09.569107+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:25:06.307903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.571940+0000","last_change":"2024-09-12T01:59:55.499938+0000","last_active":"2024-09-12T02:00:09.571940+0000","last_peered":"2024-09-12T02:00:09.571940+0000","last_clean":"2024-09-12T02:00:09.571940+0000","last_became_active":"2024-09-12T01:59:55.499619+0000","last_became_peered":"2024-09-12T01:59:55.499619+0000","last_unstale":"2024-09-12T02:00:09.571940+0000","last_undegraded":"2024-09-12T02:00:09.571940+0000","last_fullsized":"2024-09-12T02:00:09.571940+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:00:17.536120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.562514+0000","last_change":"2024-09-12T01:59:55.486106+0000","last_active":"2024-09-12T02:00:09.562514+0000","last_peered":"2024-09-12T02:00:09.562514+0000","last_clean":"2024-09-12T02:00:09.562514+0000","last_became_active":"2024-09-12T01:59:55.485325+0000","last_became_peered":"2024-09-12T01:59:55.485325+0000","last_unstale":"2024-09-12T02:00:09.562514+0000","last_undegraded":"2024-09-12T02:00:09.562514+0000","last_fullsized":"2024-09-12T02:00:09.562514+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:35:46.446224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.622237+0000","last_change":"2024-09-12T01:59:55.496438+0000","last_active":"2024-09-12T02:00:09.622237+0000","last_peered":"2024-09-12T02:00:09.622237+0000","last_clean":"2024-09-12T02:00:09.622237+0000","last_became_active":"2024-09-12T01:59:55.495885+0000","last_became_peered":"2024-09-12T01:59:55.495885+0000","last_unstale":"2024-09-12T02:00:09.622237+0000","last_undegraded":"2024-09-12T02:00:09.622237+0000","last_fullsized":"2024-09-12T02:00:09.622237+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:15:14.911335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.621139+0000","last_change":"2024-09-12T01:59:55.489350+0000","last_active":"2024-09-12T02:00:09.621139+0000","last_peered":"2024-09-12T02:00:09.621139+0000","last_clean":"2024-09-12T02:00:09.621139+0000","last_became_active":"2024-09-12T01:59:55.489028+0000","last_became_peered":"2024-09-12T01:59:55.489028+0000","last_unstale":"2024-09-12T02:00:09.621139+0000","last_undegraded":"2024-09-12T02:00:09.621139+0000","last_fullsized":"2024-09-12T02:00:09.621139+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:29:23.769004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.566645+0000","last_change":"2024-09-12T01:59:55.494532+0000","last_active":"2024-09-12T02:00:09.566645+0000","last_peered":"2024-09-12T02:00:09.566645+0000","last_clean":"2024-09-12T02:00:09.566645+0000","last_became_active":"2024-09-12T01:59:55.490982+0000","last_became_peered":"2024-09-12T01:59:55.490982+0000","last_unstale":"2024-09-12T02:00:09.566645+0000","last_undegraded":"2024-09-12T02:00:09.566645+0000","last_fullsized":"2024-09-12T02:00:09.566645+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:04:50.633730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.621490+0000","last_change":"2024-09-12T01:59:55.496057+0000","last_active":"2024-09-12T02:00:09.621490+0000","last_peered":"2024-09-12T02:00:09.621490+0000","last_clean":"2024-09-12T02:00:09.621490+0000","last_became_active":"2024-09-12T01:59:55.495710+0000","last_became_peered":"2024-09-12T01:59:55.495710+0000","last_unstale":"2024-09-12T02:00:09.621490+0000","last_undegraded":"2024-09-12T02:00:09.621490+0000","last_fullsized":"2024-09-12T02:00:09.621490+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:49:01.962537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.569100+0000","last_change":"2024-09-12T01:59:55.491594+0000","last_active":"2024-09-12T02:00:09.569100+0000","last_peered":"2024-09-12T02:00:09.569100+0000","last_clean":"2024-09-12T02:00:09.569100+0000","last_became_active":"2024-09-12T01:59:55.491233+0000","last_became_peered":"2024-09-12T01:59:55.491233+0000","last_unstale":"2024-09-12T02:00:09.569100+0000","last_undegraded":"2024-09-12T02:00:09.569100+0000","last_fullsized":"2024-09-12T02:00:09.569100+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:24:22.066777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.572270+0000","last_change":"2024-09-12T01:59:55.498331+0000","last_active":"2024-09-12T02:00:09.572270+0000","last_peered":"2024-09-12T02:00:09.572270+0000","last_clean":"2024-09-12T02:00:09.572270+0000","last_became_active":"2024-09-12T01:59:55.497983+0000","last_became_peered":"2024-09-12T01:59:55.497983+0000","last_unstale":"2024-09-12T02:00:09.572270+0000","last_undegraded":"2024-09-12T02:00:09.572270+0000","last_fullsized":"2024-09-12T02:00:09.572270+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:37:00.735231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.575358+0000","last_change":"2024-09-12T02:00:09.575358+0000","last_active":"2024-09-12T02:00:09.575358+0000","last_peered":"2024-09-12T02:00:09.575358+0000","last_clean":"2024-09-12T02:00:09.575358+0000","last_became_active":"2024-09-12T02:00:09.574913+0000","last_became_peered":"2024-09-12T02:00:09.574913+0000","last_unstale":"2024-09-12T02:00:09.575358+0000","last_undegraded":"2024-09-12T02:00:09.575358+0000","last_fullsized":"2024-09-12T02:00:09.575358+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:19:53.630453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.573748+0000","last_change":"2024-09-12T01:59:55.492419+0000","last_active":"2024-09-12T02:00:09.573748+0000","last_peered":"2024-09-12T02:00:09.573748+0000","last_clean":"2024-09-12T02:00:09.573748+0000","last_became_active":"2024-09-12T01:59:55.492003+0000","last_became_peered":"2024-09-12T01:59:55.492003+0000","last_unstale":"2024-09-12T02:00:09.573748+0000","last_undegraded":"2024-09-12T02:00:09.573748+0000","last_fullsized":"2024-09-12T02:00:09.573748+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:25:45.774679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.568801+0000","last_change":"2024-09-12T01:59:55.490125+0000","last_active":"2024-09-12T02:00:09.568801+0000","last_peered":"2024-09-12T02:00:09.568801+0000","last_clean":"2024-09-12T02:00:09.568801+0000","last_became_active":"2024-09-12T01:59:55.489636+0000","last_became_peered":"2024-09-12T01:59:55.489636+0000","last_unstale":"2024-09-12T02:00:09.568801+0000","last_undegraded":"2024-09-12T02:00:09.568801+0000","last_fullsized":"2024-09-12T02:00:09.568801+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:41:30.922185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.569135+0000","last_change":"2024-09-12T01:59:55.496322+0000","last_active":"2024-09-12T02:00:09.569135+0000","last_peered":"2024-09-12T02:00:09.569135+0000","last_clean":"2024-09-12T02:00:09.569135+0000","last_became_active":"2024-09-12T01:59:55.496005+0000","last_became_peered":"2024-09-12T01:59:55.496005+0000","last_unstale":"2024-09-12T02:00:09.569135+0000","last_undegraded":"2024-09-12T02:00:09.569135+0000","last_fullsized":"2024-09-12T02:00:09.569135+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:53:39.248715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.572017+0000","last_change":"2024-09-12T01:59:55.497758+0000","last_active":"2024-09-12T02:00:09.572017+0000","last_peered":"2024-09-12T02:00:09.572017+0000","last_clean":"2024-09-12T02:00:09.572017+0000","last_became_active":"2024-09-12T01:59:55.497282+0000","last_became_peered":"2024-09-12T01:59:55.497282+0000","last_unstale":"2024-09-12T02:00:09.572017+0000","last_undegraded":"2024-09-12T02:00:09.572017+0000","last_fullsized":"2024-09-12T02:00:09.572017+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:08:25.509372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.574620+0000","last_change":"2024-09-12T01:59:55.497919+0000","last_active":"2024-09-12T02:00:09.574620+0000","last_peered":"2024-09-12T02:00:09.574620+0000","last_clean":"2024-09-12T02:00:09.574620+0000","last_became_active":"2024-09-12T01:59:55.497609+0000","last_became_peered":"2024-09-12T01:59:55.497609+0000","last_unstale":"2024-09-12T02:00:09.574620+0000","last_undegraded":"2024-09-12T02:00:09.574620+0000","last_fullsized":"2024-09-12T02:00:09.574620+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:10:22.281180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"81'6","reported_seq":82,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:54.624408+0000","last_change":"2024-09-12T01:59:53.473974+0000","last_active":"2024-09-12T02:00:54.624408+0000","last_peered":"2024-09-12T02:00:54.624408+0000","last_clean":"2024-09-12T02:00:54.624408+0000","last_became_active":"2024-09-12T01:59:53.473546+0000","last_became_peered":"2024-09-12T01:59:53.473546+0000","last_unstale":"2024-09-12T02:00:54.624408+0000","last_undegraded":"2024-09-12T02:00:54.624408+0000","last_fullsized":"2024-09-12T02:00:54.624408+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:28:03.532249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":54,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.567217+0000","last_change":"2024-09-12T01:59:55.493197+0000","last_active":"2024-09-12T02:00:09.567217+0000","last_peered":"2024-09-12T02:00:09.567217+0000","last_clean":"2024-09-12T02:00:09.567217+0000","last_became_active":"2024-09-12T01:59:55.492845+0000","last_became_peered":"2024-09-12T01:59:55.492845+0000","last_unstale":"2024-09-12T02:00:09.567217+0000","last_undegraded":"2024-09-12T02:00:09.567217+0000","last_fullsized":"2024-09-12T02:00:09.567217+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:42:50.790896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"80'3","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.622865+0000","last_change":"2024-09-12T01:59:53.471214+0000","last_active":"2024-09-12T02:00:09.622865+0000","last_peered":"2024-09-12T02:00:09.622865+0000","last_clean":"2024-09-12T02:00:09.622865+0000","last_became_active":"2024-09-12T01:59:53.470858+0000","last_became_peered":"2024-09-12T01:59:53.470858+0000","last_unstale":"2024-09-12T02:00:09.622865+0000","last_undegraded":"2024-09-12T02:00:09.622865+0000","last_fullsized":"2024-09-12T02:00:09.622865+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:34:48.730392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"80'2","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.562106+0000","last_change":"2024-09-12T01:59:53.464568+0000","last_active":"2024-09-12T02:00:09.562106+0000","last_peered":"2024-09-12T02:00:09.562106+0000","last_clean":"2024-09-12T02:00:09.562106+0000","last_became_active":"2024-09-12T01:59:53.464168+0000","last_became_peered":"2024-09-12T01:59:53.464168+0000","last_unstale":"2024-09-12T02:00:09.562106+0000","last_undegraded":"2024-09-12T02:00:09.562106+0000","last_fullsized":"2024-09-12T02:00:09.562106+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:41:48.135522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.621151+0000","last_change":"2024-09-12T01:59:55.487177+0000","last_active":"2024-09-12T02:00:09.621151+0000","last_peered":"2024-09-12T02:00:09.621151+0000","last_clean":"2024-09-12T02:00:09.621151+0000","last_became_active":"2024-09-12T01:59:55.486886+0000","last_became_peered":"2024-09-12T01:59:55.486886+0000","last_unstale":"2024-09-12T02:00:09.621151+0000","last_undegraded":"2024-09-12T02:00:09.621151+0000","last_fullsized":"2024-09-12T02:00:09.621151+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:32:40.150071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"80'1","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.567129+0000","last_change":"2024-09-12T01:59:53.474114+0000","last_active":"2024-09-12T02:00:09.567129+0000","last_peered":"2024-09-12T02:00:09.567129+0000","last_clean":"2024-09-12T02:00:09.567129+0000","last_became_active":"2024-09-12T01:59:53.473755+0000","last_became_peered":"2024-09-12T01:59:53.473755+0000","last_unstale":"2024-09-12T02:00:09.567129+0000","last_undegraded":"2024-09-12T02:00:09.567129+0000","last_fullsized":"2024-09-12T02:00:09.567129+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:07:07.021137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.564793+0000","last_change":"2024-09-12T01:59:55.489868+0000","last_active":"2024-09-12T02:00:09.564793+0000","last_peered":"2024-09-12T02:00:09.564793+0000","last_clean":"2024-09-12T02:00:09.564793+0000","last_became_active":"2024-09-12T01:59:55.489537+0000","last_became_peered":"2024-09-12T01:59:55.489537+0000","last_unstale":"2024-09-12T02:00:09.564793+0000","last_undegraded":"2024-09-12T02:00:09.564793+0000","last_fullsized":"2024-09-12T02:00:09.564793+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:18:49.417689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"80'3","reported_seq":74,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:52.621151+0000","last_change":"2024-09-12T01:59:53.471402+0000","last_active":"2024-09-12T02:00:52.621151+0000","last_peered":"2024-09-12T02:00:52.621151+0000","last_clean":"2024-09-12T02:00:52.621151+0000","last_became_active":"2024-09-12T01:59:53.470940+0000","last_became_peered":"2024-09-12T01:59:53.470940+0000","last_unstale":"2024-09-12T02:00:52.621151+0000","last_undegraded":"2024-09-12T02:00:52.621151+0000","last_fullsized":"2024-09-12T02:00:52.621151+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:41:16.853592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":49,"num_read_kb":49,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.623260+0000","last_change":"2024-09-12T01:59:55.499047+0000","last_active":"2024-09-12T02:00:09.623260+0000","last_peered":"2024-09-12T02:00:09.623260+0000","last_clean":"2024-09-12T02:00:09.623260+0000","last_became_active":"2024-09-12T01:59:55.498071+0000","last_became_peered":"2024-09-12T01:59:55.498071+0000","last_unstale":"2024-09-12T02:00:09.623260+0000","last_undegraded":"2024-09-12T02:00:09.623260+0000","last_fullsized":"2024-09-12T02:00:09.623260+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:36:40.106041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"80'2","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.622966+0000","last_change":"2024-09-12T01:59:53.473508+0000","last_active":"2024-09-12T02:00:09.622966+0000","last_peered":"2024-09-12T02:00:09.622966+0000","last_clean":"2024-09-12T02:00:09.622966+0000","last_became_active":"2024-09-12T01:59:53.473063+0000","last_became_peered":"2024-09-12T01:59:53.473063+0000","last_unstale":"2024-09-12T02:00:09.622966+0000","last_undegraded":"2024-09-12T02:00:09.622966+0000","last_fullsized":"2024-09-12T02:00:09.622966+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:55:32.161769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.573961+0000","last_change":"2024-09-12T01:59:55.491721+0000","last_active":"2024-09-12T02:00:09.573961+0000","last_peered":"2024-09-12T02:00:09.573961+0000","last_clean":"2024-09-12T02:00:09.573961+0000","last_became_active":"2024-09-12T01:59:55.491364+0000","last_became_peered":"2024-09-12T01:59:55.491364+0000","last_unstale":"2024-09-12T02:00:09.573961+0000","last_undegraded":"2024-09-12T02:00:09.573961+0000","last_fullsized":"2024-09-12T02:00:09.573961+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:21:20.795420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"80'4","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.567315+0000","last_change":"2024-09-12T01:59:53.921789+0000","last_active":"2024-09-12T02:00:09.567315+0000","last_peered":"2024-09-12T02:00:09.567315+0000","last_clean":"2024-09-12T02:00:09.567315+0000","last_became_active":"2024-09-12T01:59:53.921383+0000","last_became_peered":"2024-09-12T01:59:53.921383+0000","last_unstale":"2024-09-12T02:00:09.567315+0000","last_undegraded":"2024-09-12T02:00:09.567315+0000","last_fullsized":"2024-09-12T02:00:09.567315+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:21:58.792794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.622517+0000","last_change":"2024-09-12T01:59:55.503008+0000","last_active":"2024-09-12T02:00:09.622517+0000","last_peered":"2024-09-12T02:00:09.622517+0000","last_clean":"2024-09-12T02:00:09.622517+0000","last_became_active":"2024-09-12T01:59:55.502538+0000","last_became_peered":"2024-09-12T01:59:55.502538+0000","last_unstale":"2024-09-12T02:00:09.622517+0000","last_undegraded":"2024-09-12T02:00:09.622517+0000","last_fullsized":"2024-09-12T02:00:09.622517+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:51:36.735985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"82'27","reported_seq":155,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:53.622631+0000","last_change":"2024-09-12T01:59:53.470688+0000","last_active":"2024-09-12T02:00:53.622631+0000","last_peered":"2024-09-12T02:00:53.622631+0000","last_clean":"2024-09-12T02:00:53.622631+0000","last_became_active":"2024-09-12T01:59:53.470231+0000","last_became_peered":"2024-09-12T01:59:53.470231+0000","last_unstale":"2024-09-12T02:00:53.622631+0000","last_undegraded":"2024-09-12T02:00:53.622631+0000","last_fullsized":"2024-09-12T02:00:53.622631+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:53:54.439893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59764,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":106,"num_read_kb":111,"num_write":28,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.575173+0000","last_change":"2024-09-12T01:59:55.495343+0000","last_active":"2024-09-12T02:00:09.575173+0000","last_peered":"2024-09-12T02:00:09.575173+0000","last_clean":"2024-09-12T02:00:09.575173+0000","last_became_active":"2024-09-12T01:59:55.495006+0000","last_became_peered":"2024-09-12T01:59:55.495006+0000","last_unstale":"2024-09-12T02:00:09.575173+0000","last_undegraded":"2024-09-12T02:00:09.575173+0000","last_fullsized":"2024-09-12T02:00:09.575173+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:32:20.932551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'6","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.569082+0000","last_change":"2024-09-12T01:59:53.468625+0000","last_active":"2024-09-12T02:00:09.569082+0000","last_peered":"2024-09-12T02:00:09.569082+0000","last_clean":"2024-09-12T02:00:09.569082+0000","last_became_active":"2024-09-12T01:59:53.468015+0000","last_became_peered":"2024-09-12T01:59:53.468015+0000","last_unstale":"2024-09-12T02:00:09.569082+0000","last_undegraded":"2024-09-12T02:00:09.569082+0000","last_fullsized":"2024-09-12T02:00:09.569082+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:49:34.842239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.562155+0000","last_change":"2024-09-12T01:59:55.487962+0000","last_active":"2024-09-12T02:00:09.562155+0000","last_peered":"2024-09-12T02:00:09.562155+0000","last_clean":"2024-09-12T02:00:09.562155+0000","last_became_active":"2024-09-12T01:59:55.487619+0000","last_became_peered":"2024-09-12T01:59:55.487619+0000","last_unstale":"2024-09-12T02:00:09.562155+0000","last_undegraded":"2024-09-12T02:00:09.562155+0000","last_fullsized":"2024-09-12T02:00:09.562155+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:32:16.882373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"80'3","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.567124+0000","last_change":"2024-09-12T01:59:53.474418+0000","last_active":"2024-09-12T02:00:09.567124+0000","last_peered":"2024-09-12T02:00:09.567124+0000","last_clean":"2024-09-12T02:00:09.567124+0000","last_became_active":"2024-09-12T01:59:53.474059+0000","last_became_peered":"2024-09-12T01:59:53.474059+0000","last_unstale":"2024-09-12T02:00:09.567124+0000","last_undegraded":"2024-09-12T02:00:09.567124+0000","last_fullsized":"2024-09-12T02:00:09.567124+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:10:54.963435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.622740+0000","last_change":"2024-09-12T01:59:55.493394+0000","last_active":"2024-09-12T02:00:09.622740+0000","last_peered":"2024-09-12T02:00:09.622740+0000","last_clean":"2024-09-12T02:00:09.622740+0000","last_became_active":"2024-09-12T01:59:55.491669+0000","last_became_peered":"2024-09-12T01:59:55.491669+0000","last_unstale":"2024-09-12T02:00:09.622740+0000","last_undegraded":"2024-09-12T02:00:09.622740+0000","last_fullsized":"2024-09-12T02:00:09.622740+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:14:40.871154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"80'3","reported_seq":77,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:53.623170+0000","last_change":"2024-09-12T01:59:53.471623+0000","last_active":"2024-09-12T02:00:53.623170+0000","last_peered":"2024-09-12T02:00:53.623170+0000","last_clean":"2024-09-12T02:00:53.623170+0000","last_became_active":"2024-09-12T01:59:53.471294+0000","last_became_peered":"2024-09-12T01:59:53.471294+0000","last_unstale":"2024-09-12T02:00:53.623170+0000","last_undegraded":"2024-09-12T02:00:53.623170+0000","last_fullsized":"2024-09-12T02:00:53.623170+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:51:54.130883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":53,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.621910+0000","last_change":"2024-09-12T01:59:55.493082+0000","last_active":"2024-09-12T02:00:09.621910+0000","last_peered":"2024-09-12T02:00:09.621910+0000","last_clean":"2024-09-12T02:00:09.621910+0000","last_became_active":"2024-09-12T01:59:55.492778+0000","last_became_peered":"2024-09-12T01:59:55.492778+0000","last_unstale":"2024-09-12T02:00:09.621910+0000","last_undegraded":"2024-09-12T02:00:09.621910+0000","last_fullsized":"2024-09-12T02:00:09.621910+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:59:56.466377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'15","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.588050+0000","last_change":"2024-09-12T02:00:09.588050+0000","last_active":"2024-09-12T02:00:09.588050+0000","last_peered":"2024-09-12T02:00:09.588050+0000","last_clean":"2024-09-12T02:00:09.588050+0000","last_became_active":"2024-09-12T02:00:09.575478+0000","last_became_peered":"2024-09-12T02:00:09.575478+0000","last_unstale":"2024-09-12T02:00:09.588050+0000","last_undegraded":"2024-09-12T02:00:09.588050+0000","last_fullsized":"2024-09-12T02:00:09.588050+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:38:45.484167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.566976+0000","last_change":"2024-09-12T01:59:55.497601+0000","last_active":"2024-09-12T02:00:09.566976+0000","last_peered":"2024-09-12T02:00:09.566976+0000","last_clean":"2024-09-12T02:00:09.566976+0000","last_became_active":"2024-09-12T01:59:55.497202+0000","last_became_peered":"2024-09-12T01:59:55.497202+0000","last_unstale":"2024-09-12T02:00:09.566976+0000","last_undegraded":"2024-09-12T02:00:09.566976+0000","last_fullsized":"2024-09-12T02:00:09.566976+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:22:22.850736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"80'2","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.575359+0000","last_change":"2024-09-12T01:59:53.467575+0000","last_active":"2024-09-12T02:00:09.575359+0000","last_peered":"2024-09-12T02:00:09.575359+0000","last_clean":"2024-09-12T02:00:09.575359+0000","last_became_active":"2024-09-12T01:59:53.467146+0000","last_became_peered":"2024-09-12T01:59:53.467146+0000","last_unstale":"2024-09-12T02:00:09.575359+0000","last_undegraded":"2024-09-12T02:00:09.575359+0000","last_fullsized":"2024-09-12T02:00:09.575359+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:07:25.962496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.572034+0000","last_change":"2024-09-12T01:59:55.498297+0000","last_active":"2024-09-12T02:00:09.572034+0000","last_peered":"2024-09-12T02:00:09.572034+0000","last_clean":"2024-09-12T02:00:09.572034+0000","last_became_active":"2024-09-12T01:59:55.497367+0000","last_became_peered":"2024-09-12T01:59:55.497367+0000","last_unstale":"2024-09-12T02:00:09.572034+0000","last_undegraded":"2024-09-12T02:00:09.572034+0000","last_fullsized":"2024-09-12T02:00:09.572034+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:58:32.717710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"80'2","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.562292+0000","last_change":"2024-09-12T01:59:53.466664+0000","last_active":"2024-09-12T02:00:09.562292+0000","last_peered":"2024-09-12T02:00:09.562292+0000","last_clean":"2024-09-12T02:00:09.562292+0000","last_became_active":"2024-09-12T01:59:53.466318+0000","last_became_peered":"2024-09-12T01:59:53.466318+0000","last_unstale":"2024-09-12T02:00:09.562292+0000","last_undegraded":"2024-09-12T02:00:09.562292+0000","last_fullsized":"2024-09-12T02:00:09.562292+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:12:13.541420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.622804+0000","last_change":"2024-09-12T01:59:55.490420+0000","last_active":"2024-09-12T02:00:09.622804+0000","last_peered":"2024-09-12T02:00:09.622804+0000","last_clean":"2024-09-12T02:00:09.622804+0000","last_became_active":"2024-09-12T01:59:55.489757+0000","last_became_peered":"2024-09-12T01:59:55.489757+0000","last_unstale":"2024-09-12T02:00:09.622804+0000","last_undegraded":"2024-09-12T02:00:09.622804+0000","last_fullsized":"2024-09-12T02:00:09.622804+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:34:43.117142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'1","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.567185+0000","last_change":"2024-09-12T01:59:53.466399+0000","last_active":"2024-09-12T02:00:09.567185+0000","last_peered":"2024-09-12T02:00:09.567185+0000","last_clean":"2024-09-12T02:00:09.567185+0000","last_became_active":"2024-09-12T01:59:53.465982+0000","last_became_peered":"2024-09-12T01:59:53.465982+0000","last_unstale":"2024-09-12T02:00:09.567185+0000","last_undegraded":"2024-09-12T02:00:09.567185+0000","last_fullsized":"2024-09-12T02:00:09.567185+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:18:02.015040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"82'1","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.622907+0000","last_change":"2024-09-12T01:59:55.491527+0000","last_active":"2024-09-12T02:00:09.622907+0000","last_peered":"2024-09-12T02:00:09.622907+0000","last_clean":"2024-09-12T02:00:09.622907+0000","last_became_active":"2024-09-12T01:59:55.491165+0000","last_became_peered":"2024-09-12T01:59:55.491165+0000","last_unstale":"2024-09-12T02:00:09.622907+0000","last_undegraded":"2024-09-12T02:00:09.622907+0000","last_fullsized":"2024-09-12T02:00:09.622907+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:57:05.506384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"80'3","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.569187+0000","last_change":"2024-09-12T01:59:54.075752+0000","last_active":"2024-09-12T02:00:09.569187+0000","last_peered":"2024-09-12T02:00:09.569187+0000","last_clean":"2024-09-12T02:00:09.569187+0000","last_became_active":"2024-09-12T01:59:54.075387+0000","last_became_peered":"2024-09-12T01:59:54.075387+0000","last_unstale":"2024-09-12T02:00:09.569187+0000","last_undegraded":"2024-09-12T02:00:09.569187+0000","last_fullsized":"2024-09-12T02:00:09.569187+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:21:15.980591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.622907+0000","last_change":"2024-09-12T01:59:55.500935+0000","last_active":"2024-09-12T02:00:09.622907+0000","last_peered":"2024-09-12T02:00:09.622907+0000","last_clean":"2024-09-12T02:00:09.622907+0000","last_became_active":"2024-09-12T01:59:55.500206+0000","last_became_peered":"2024-09-12T01:59:55.500206+0000","last_unstale":"2024-09-12T02:00:09.622907+0000","last_undegraded":"2024-09-12T02:00:09.622907+0000","last_fullsized":"2024-09-12T02:00:09.622907+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:19:59.937701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":333,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.623202+0000","last_change":"2024-09-12T01:58:47.553853+0000","last_active":"2024-09-12T02:00:09.623202+0000","last_peered":"2024-09-12T02:00:09.623202+0000","last_clean":"2024-09-12T02:00:09.623202+0000","last_became_active":"2024-09-12T01:58:47.553357+0000","last_became_peered":"2024-09-12T01:58:47.553357+0000","last_unstale":"2024-09-12T02:00:09.623202+0000","last_undegraded":"2024-09-12T02:00:09.623202+0000","last_fullsized":"2024-09-12T02:00:09.623202+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:55:26.917733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:55:26.917733+0000","last_clean_scrub_stamp":"2024-09-12T01:55:26.917733+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:58:26.081766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"80'3","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.563615+0000","last_change":"2024-09-12T01:59:53.467907+0000","last_active":"2024-09-12T02:00:09.563615+0000","last_peered":"2024-09-12T02:00:09.563615+0000","last_clean":"2024-09-12T02:00:09.563615+0000","last_became_active":"2024-09-12T01:59:53.467164+0000","last_became_peered":"2024-09-12T01:59:53.467164+0000","last_unstale":"2024-09-12T02:00:09.563615+0000","last_undegraded":"2024-09-12T02:00:09.563615+0000","last_fullsized":"2024-09-12T02:00:09.563615+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:09:35.242708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.621290+0000","last_change":"2024-09-12T01:59:55.495109+0000","last_active":"2024-09-12T02:00:09.621290+0000","last_peered":"2024-09-12T02:00:09.621290+0000","last_clean":"2024-09-12T02:00:09.621290+0000","last_became_active":"2024-09-12T01:59:55.494791+0000","last_became_peered":"2024-09-12T01:59:55.494791+0000","last_unstale":"2024-09-12T02:00:09.621290+0000","last_undegraded":"2024-09-12T02:00:09.621290+0000","last_fullsized":"2024-09-12T02:00:09.621290+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:22:43.882050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"80'3","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.623115+0000","last_change":"2024-09-12T01:59:53.473286+0000","last_active":"2024-09-12T02:00:09.623115+0000","last_peered":"2024-09-12T02:00:09.623115+0000","last_clean":"2024-09-12T02:00:09.623115+0000","last_became_active":"2024-09-12T01:59:53.472976+0000","last_became_peered":"2024-09-12T01:59:53.472976+0000","last_unstale":"2024-09-12T02:00:09.623115+0000","last_undegraded":"2024-09-12T02:00:09.623115+0000","last_fullsized":"2024-09-12T02:00:09.623115+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:32:39.604173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.571159+0000","last_change":"2024-09-12T02:00:09.571159+0000","last_active":"2024-09-12T02:00:09.571159+0000","last_peered":"2024-09-12T02:00:09.571159+0000","last_clean":"2024-09-12T02:00:09.571159+0000","last_became_active":"2024-09-12T02:00:09.570672+0000","last_became_peered":"2024-09-12T02:00:09.570672+0000","last_unstale":"2024-09-12T02:00:09.571159+0000","last_undegraded":"2024-09-12T02:00:09.571159+0000","last_fullsized":"2024-09-12T02:00:09.571159+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:04:10.808942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"80'4","reported_seq":76,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:53.622299+0000","last_change":"2024-09-12T01:59:53.921856+0000","last_active":"2024-09-12T02:00:53.622299+0000","last_peered":"2024-09-12T02:00:53.622299+0000","last_clean":"2024-09-12T02:00:53.622299+0000","last_became_active":"2024-09-12T01:59:53.920672+0000","last_became_peered":"2024-09-12T01:59:53.920672+0000","last_unstale":"2024-09-12T02:00:53.622299+0000","last_undegraded":"2024-09-12T02:00:53.622299+0000","last_fullsized":"2024-09-12T02:00:53.622299+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:42:33.876493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":50,"num_read_kb":50,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.571600+0000","last_change":"2024-09-12T02:00:09.571185+0000","last_active":"2024-09-12T02:00:09.571600+0000","last_peered":"2024-09-12T02:00:09.571600+0000","last_clean":"2024-09-12T02:00:09.571600+0000","last_became_active":"2024-09-12T02:00:09.570769+0000","last_became_peered":"2024-09-12T02:00:09.570769+0000","last_unstale":"2024-09-12T02:00:09.571600+0000","last_undegraded":"2024-09-12T02:00:09.571600+0000","last_fullsized":"2024-09-12T02:00:09.571600+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:34:17.230435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.2","version":"80'1","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.622691+0000","last_change":"2024-09-12T01:59:53.472072+0000","last_active":"2024-09-12T02:00:09.622691+0000","last_peered":"2024-09-12T02:00:09.622691+0000","last_clean":"2024-09-12T02:00:09.622691+0000","last_became_active":"2024-09-12T01:59:53.471088+0000","last_became_peered":"2024-09-12T01:59:53.471088+0000","last_unstale":"2024-09-12T02:00:09.622691+0000","last_undegraded":"2024-09-12T02:00:09.622691+0000","last_fullsized":"2024-09-12T02:00:09.622691+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:02:13.879131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.622413+0000","last_change":"2024-09-12T01:59:55.502843+0000","last_active":"2024-09-12T02:00:09.622413+0000","last_peered":"2024-09-12T02:00:09.622413+0000","last_clean":"2024-09-12T02:00:09.622413+0000","last_became_active":"2024-09-12T01:59:55.499296+0000","last_became_peered":"2024-09-12T01:59:55.499296+0000","last_unstale":"2024-09-12T02:00:09.622413+0000","last_undegraded":"2024-09-12T02:00:09.622413+0000","last_fullsized":"2024-09-12T02:00:09.622413+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:04:21.787370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.575061+0000","last_change":"2024-09-12T02:00:09.575061+0000","last_active":"2024-09-12T02:00:09.575061+0000","last_peered":"2024-09-12T02:00:09.575061+0000","last_clean":"2024-09-12T02:00:09.575061+0000","last_became_active":"2024-09-12T02:00:09.574653+0000","last_became_peered":"2024-09-12T02:00:09.574653+0000","last_unstale":"2024-09-12T02:00:09.575061+0000","last_undegraded":"2024-09-12T02:00:09.575061+0000","last_fullsized":"2024-09-12T02:00:09.575061+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:32:27.931211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"82'1","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.622315+0000","last_change":"2024-09-12T01:59:55.494162+0000","last_active":"2024-09-12T02:00:09.622315+0000","last_peered":"2024-09-12T02:00:09.622315+0000","last_clean":"2024-09-12T02:00:09.622315+0000","last_became_active":"2024-09-12T01:59:55.493802+0000","last_became_peered":"2024-09-12T01:59:55.493802+0000","last_unstale":"2024-09-12T02:00:09.622315+0000","last_undegraded":"2024-09-12T02:00:09.622315+0000","last_fullsized":"2024-09-12T02:00:09.622315+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:22:16.302683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"80'1","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.575278+0000","last_change":"2024-09-12T01:59:53.921669+0000","last_active":"2024-09-12T02:00:09.575278+0000","last_peered":"2024-09-12T02:00:09.575278+0000","last_clean":"2024-09-12T02:00:09.575278+0000","last_became_active":"2024-09-12T01:59:53.921328+0000","last_became_peered":"2024-09-12T01:59:53.921328+0000","last_unstale":"2024-09-12T02:00:09.575278+0000","last_undegraded":"2024-09-12T02:00:09.575278+0000","last_fullsized":"2024-09-12T02:00:09.575278+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:46:33.313726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.562329+0000","last_change":"2024-09-12T01:59:55.488986+0000","last_active":"2024-09-12T02:00:09.562329+0000","last_peered":"2024-09-12T02:00:09.562329+0000","last_clean":"2024-09-12T02:00:09.562329+0000","last_became_active":"2024-09-12T01:59:55.488632+0000","last_became_peered":"2024-09-12T01:59:55.488632+0000","last_unstale":"2024-09-12T02:00:09.562329+0000","last_undegraded":"2024-09-12T02:00:09.562329+0000","last_fullsized":"2024-09-12T02:00:09.562329+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:20:57.596976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"82'10","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:20.501181+0000","last_change":"2024-09-12T01:59:53.467648+0000","last_active":"2024-09-12T02:00:20.501181+0000","last_peered":"2024-09-12T02:00:20.501181+0000","last_clean":"2024-09-12T02:00:20.501181+0000","last_became_active":"2024-09-12T01:59:53.467255+0000","last_became_peered":"2024-09-12T01:59:53.467255+0000","last_unstale":"2024-09-12T02:00:20.501181+0000","last_undegraded":"2024-09-12T02:00:20.501181+0000","last_fullsized":"2024-09-12T02:00:20.501181+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:37:09.897388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.621489+0000","last_change":"2024-09-12T01:59:55.487603+0000","last_active":"2024-09-12T02:00:09.621489+0000","last_peered":"2024-09-12T02:00:09.621489+0000","last_clean":"2024-09-12T02:00:09.621489+0000","last_became_active":"2024-09-12T01:59:55.487334+0000","last_became_peered":"2024-09-12T01:59:55.487334+0000","last_unstale":"2024-09-12T02:00:09.621489+0000","last_undegraded":"2024-09-12T02:00:09.621489+0000","last_fullsized":"2024-09-12T02:00:09.621489+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:23:16.669515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'4","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.563870+0000","last_change":"2024-09-12T01:59:54.076156+0000","last_active":"2024-09-12T02:00:09.563870+0000","last_peered":"2024-09-12T02:00:09.563870+0000","last_clean":"2024-09-12T02:00:09.563870+0000","last_became_active":"2024-09-12T01:59:54.075794+0000","last_became_peered":"2024-09-12T01:59:54.075794+0000","last_unstale":"2024-09-12T02:00:09.563870+0000","last_undegraded":"2024-09-12T02:00:09.563870+0000","last_fullsized":"2024-09-12T02:00:09.563870+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:06:17.431696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.573860+0000","last_change":"2024-09-12T01:59:55.494604+0000","last_active":"2024-09-12T02:00:09.573860+0000","last_peered":"2024-09-12T02:00:09.573860+0000","last_clean":"2024-09-12T02:00:09.573860+0000","last_became_active":"2024-09-12T01:59:55.494300+0000","last_became_peered":"2024-09-12T01:59:55.494300+0000","last_unstale":"2024-09-12T02:00:09.573860+0000","last_undegraded":"2024-09-12T02:00:09.573860+0000","last_fullsized":"2024-09-12T02:00:09.573860+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:18:56.359299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.622205+0000","last_change":"2024-09-12T01:59:53.921832+0000","last_active":"2024-09-12T02:00:09.622205+0000","last_peered":"2024-09-12T02:00:09.622205+0000","last_clean":"2024-09-12T02:00:09.622205+0000","last_became_active":"2024-09-12T01:59:53.921379+0000","last_became_peered":"2024-09-12T01:59:53.921379+0000","last_unstale":"2024-09-12T02:00:09.622205+0000","last_undegraded":"2024-09-12T02:00:09.622205+0000","last_fullsized":"2024-09-12T02:00:09.622205+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:26:02.277149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.574438+0000","last_change":"2024-09-12T01:59:55.493198+0000","last_active":"2024-09-12T02:00:09.574438+0000","last_peered":"2024-09-12T02:00:09.574438+0000","last_clean":"2024-09-12T02:00:09.574438+0000","last_became_active":"2024-09-12T01:59:55.492834+0000","last_became_peered":"2024-09-12T01:59:55.492834+0000","last_unstale":"2024-09-12T02:00:09.574438+0000","last_undegraded":"2024-09-12T02:00:09.574438+0000","last_fullsized":"2024-09-12T02:00:09.574438+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:58:43.872921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"80'3","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.621258+0000","last_change":"2024-09-12T01:59:53.466528+0000","last_active":"2024-09-12T02:00:09.621258+0000","last_peered":"2024-09-12T02:00:09.621258+0000","last_clean":"2024-09-12T02:00:09.621258+0000","last_became_active":"2024-09-12T01:59:53.466182+0000","last_became_peered":"2024-09-12T01:59:53.466182+0000","last_unstale":"2024-09-12T02:00:09.621258+0000","last_undegraded":"2024-09-12T02:00:09.621258+0000","last_fullsized":"2024-09-12T02:00:09.621258+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:00:26.083539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.572154+0000","last_change":"2024-09-12T01:59:55.494511+0000","last_active":"2024-09-12T02:00:09.572154+0000","last_peered":"2024-09-12T02:00:09.572154+0000","last_clean":"2024-09-12T02:00:09.572154+0000","last_became_active":"2024-09-12T01:59:55.493607+0000","last_became_peered":"2024-09-12T01:59:55.493607+0000","last_unstale":"2024-09-12T02:00:09.572154+0000","last_undegraded":"2024-09-12T02:00:09.572154+0000","last_fullsized":"2024-09-12T02:00:09.572154+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:55:27.434447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"80'1","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.569211+0000","last_change":"2024-09-12T01:59:53.467385+0000","last_active":"2024-09-12T02:00:09.569211+0000","last_peered":"2024-09-12T02:00:09.569211+0000","last_clean":"2024-09-12T02:00:09.569211+0000","last_became_active":"2024-09-12T01:59:53.466997+0000","last_became_peered":"2024-09-12T01:59:53.466997+0000","last_unstale":"2024-09-12T02:00:09.569211+0000","last_undegraded":"2024-09-12T02:00:09.569211+0000","last_fullsized":"2024-09-12T02:00:09.569211+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:54:47.154036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.621293+0000","last_change":"2024-09-12T01:59:55.490757+0000","last_active":"2024-09-12T02:00:09.621293+0000","last_peered":"2024-09-12T02:00:09.621293+0000","last_clean":"2024-09-12T02:00:09.621293+0000","last_became_active":"2024-09-12T01:59:55.490451+0000","last_became_peered":"2024-09-12T01:59:55.490451+0000","last_unstale":"2024-09-12T02:00:09.621293+0000","last_undegraded":"2024-09-12T02:00:09.621293+0000","last_fullsized":"2024-09-12T02:00:09.621293+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:21:26.139049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.574833+0000","last_change":"2024-09-12T01:59:53.472898+0000","last_active":"2024-09-12T02:00:09.574833+0000","last_peered":"2024-09-12T02:00:09.574833+0000","last_clean":"2024-09-12T02:00:09.574833+0000","last_became_active":"2024-09-12T01:59:53.472561+0000","last_became_peered":"2024-09-12T01:59:53.472561+0000","last_unstale":"2024-09-12T02:00:09.574833+0000","last_undegraded":"2024-09-12T02:00:09.574833+0000","last_fullsized":"2024-09-12T02:00:09.574833+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:55:59.933804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.622500+0000","last_change":"2024-09-12T01:59:55.497731+0000","last_active":"2024-09-12T02:00:09.622500+0000","last_peered":"2024-09-12T02:00:09.622500+0000","last_clean":"2024-09-12T02:00:09.622500+0000","last_became_active":"2024-09-12T01:59:55.497356+0000","last_became_peered":"2024-09-12T01:59:55.497356+0000","last_unstale":"2024-09-12T02:00:09.622500+0000","last_undegraded":"2024-09-12T02:00:09.622500+0000","last_fullsized":"2024-09-12T02:00:09.622500+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:57:18.795865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.624453+0000","last_change":"2024-09-12T02:00:09.624453+0000","last_active":"2024-09-12T02:00:09.624453+0000","last_peered":"2024-09-12T02:00:09.624453+0000","last_clean":"2024-09-12T02:00:09.624453+0000","last_became_active":"2024-09-12T02:00:09.624049+0000","last_became_peered":"2024-09-12T02:00:09.624049+0000","last_unstale":"2024-09-12T02:00:09.624453+0000","last_undegraded":"2024-09-12T02:00:09.624453+0000","last_fullsized":"2024-09-12T02:00:09.624453+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:17:40.417275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"80'2","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.575060+0000","last_change":"2024-09-12T01:59:53.470194+0000","last_active":"2024-09-12T02:00:09.575060+0000","last_peered":"2024-09-12T02:00:09.575060+0000","last_clean":"2024-09-12T02:00:09.575060+0000","last_became_active":"2024-09-12T01:59:53.469772+0000","last_became_peered":"2024-09-12T01:59:53.469772+0000","last_unstale":"2024-09-12T02:00:09.575060+0000","last_undegraded":"2024-09-12T02:00:09.575060+0000","last_fullsized":"2024-09-12T02:00:09.575060+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:52:57.709596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.622423+0000","last_change":"2024-09-12T01:59:55.497693+0000","last_active":"2024-09-12T02:00:09.622423+0000","last_peered":"2024-09-12T02:00:09.622423+0000","last_clean":"2024-09-12T02:00:09.622423+0000","last_became_active":"2024-09-12T01:59:55.497380+0000","last_became_peered":"2024-09-12T01:59:55.497380+0000","last_unstale":"2024-09-12T02:00:09.622423+0000","last_undegraded":"2024-09-12T02:00:09.622423+0000","last_fullsized":"2024-09-12T02:00:09.622423+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:42:27.943530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"80'4","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.575274+0000","last_change":"2024-09-12T01:59:53.473044+0000","last_active":"2024-09-12T02:00:09.575274+0000","last_peered":"2024-09-12T02:00:09.575274+0000","last_clean":"2024-09-12T02:00:09.575274+0000","last_became_active":"2024-09-12T01:59:53.472569+0000","last_became_peered":"2024-09-12T01:59:53.472569+0000","last_unstale":"2024-09-12T02:00:09.575274+0000","last_undegraded":"2024-09-12T02:00:09.575274+0000","last_fullsized":"2024-09-12T02:00:09.575274+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:41:29.716836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.573956+0000","last_change":"2024-09-12T01:59:55.497377+0000","last_active":"2024-09-12T02:00:09.573956+0000","last_peered":"2024-09-12T02:00:09.573956+0000","last_clean":"2024-09-12T02:00:09.573956+0000","last_became_active":"2024-09-12T01:59:55.497078+0000","last_became_peered":"2024-09-12T01:59:55.497078+0000","last_unstale":"2024-09-12T02:00:09.573956+0000","last_undegraded":"2024-09-12T02:00:09.573956+0000","last_fullsized":"2024-09-12T02:00:09.573956+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:59:57.362350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.621555+0000","last_change":"2024-09-12T01:59:53.920868+0000","last_active":"2024-09-12T02:00:09.621555+0000","last_peered":"2024-09-12T02:00:09.621555+0000","last_clean":"2024-09-12T02:00:09.621555+0000","last_became_active":"2024-09-12T01:59:53.920481+0000","last_became_peered":"2024-09-12T01:59:53.920481+0000","last_unstale":"2024-09-12T02:00:09.621555+0000","last_undegraded":"2024-09-12T02:00:09.621555+0000","last_fullsized":"2024-09-12T02:00:09.621555+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:49:27.085388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.621438+0000","last_change":"2024-09-12T01:59:55.497703+0000","last_active":"2024-09-12T02:00:09.621438+0000","last_peered":"2024-09-12T02:00:09.621438+0000","last_clean":"2024-09-12T02:00:09.621438+0000","last_became_active":"2024-09-12T01:59:55.497397+0000","last_became_peered":"2024-09-12T01:59:55.497397+0000","last_unstale":"2024-09-12T02:00:09.621438+0000","last_undegraded":"2024-09-12T02:00:09.621438+0000","last_fullsized":"2024-09-12T02:00:09.621438+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:54:02.803203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"80'9","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.565145+0000","last_change":"2024-09-12T01:59:53.469978+0000","last_active":"2024-09-12T02:00:09.565145+0000","last_peered":"2024-09-12T02:00:09.565145+0000","last_clean":"2024-09-12T02:00:09.565145+0000","last_became_active":"2024-09-12T01:59:53.469648+0000","last_became_peered":"2024-09-12T01:59:53.469648+0000","last_unstale":"2024-09-12T02:00:09.565145+0000","last_undegraded":"2024-09-12T02:00:09.565145+0000","last_fullsized":"2024-09-12T02:00:09.565145+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:08:37.720452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":63767,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":324,"num_read_kb":228,"num_write":160,"num_write_kb":107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":191301,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":136,"ondisk_log_size":136,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3563520,"data_stored":3542208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":74,"seq":317827579929,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32068,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713084,"statfs":{"total":95995035648,"available":95962198016,"internally_reserved":0,"allocated":897024,"data_stored":435916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":67,"seq":287762808862,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32732,"kb_used_data":1548,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712420,"statfs":{"total":95995035648,"available":95961518080,"internally_reserved":0,"allocated":1585152,"data_stored":1087049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":62,"seq":266287972387,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":905216,"data_stored":436040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201320,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":897024,"data_stored":435848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430253,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32648,"kb_used_data":1464,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712504,"statfs":{"total":95995035648,"available":95961604096,"internally_reserved":0,"allocated":1499136,"data_stored":1027892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593778,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36216,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708936,"statfs":{"total":95995035648,"available":95957950464,"internally_reserved":0,"allocated":892928,"data_stored":435429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822711,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32660,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712492,"statfs":{"total":95995035648,"available":95961591808,"internally_reserved":0,"allocated":1511424,"data_stored":1027576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986235,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":897024,"data_stored":435866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215168,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":909312,"data_stored":435740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":17,"seq":73014444102,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36300,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708852,"statfs":{"total":95995035648,"available":95957864448,"internally_reserved":0,"allocated":978944,"data_stored":496729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574923,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32116,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713036,"statfs":{"total":95995035648,"available":95962148864,"internally_reserved":0,"allocated":954368,"data_stored":494986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738448,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":913408,"data_stored":436700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":59764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":61507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":61342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T02:01:00.743 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-12T02:01:00.743 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-12T02:01:00.743 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-09-12T02:01:00.743 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph health --format=json 2024-09-12T02:01:01.000 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:01:01.372 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:01:01 smithi016 ceph-mon[30947]: from='client.14925 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T02:01:01.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:01:01 smithi184 ceph-mon[35593]: from='client.14925 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T02:01:01.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:01:01 smithi033 ceph-mon[35332]: from='client.14925 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T02:01:01.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:01.454+0000 7fe399a61640 1 -- 172.21.15.16:0/1450465531 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe394102570 msgr2=0x7fe3941029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:01.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:01.454+0000 7fe399a61640 1 --2- 172.21.15.16:0/1450465531 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe394102570 0x7fe3941029b0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fe37c009900 tx=0x7fe37c02a480 comp rx=0 tx=0).stop 2024-09-12T02:01:01.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:01.455+0000 7fe399a61640 1 -- 172.21.15.16:0/1450465531 shutdown_connections 2024-09-12T02:01:01.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:01.455+0000 7fe399a61640 1 --2- 172.21.15.16:0/1450465531 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe394102ef0 0x7fe39410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:01.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:01.455+0000 7fe399a61640 1 --2- 172.21.15.16:0/1450465531 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe394102570 0x7fe3941029b0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:01.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:01.455+0000 7fe399a61640 1 --2- 172.21.15.16:0/1450465531 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe394100e20 0x7fe394101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:01.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:01.455+0000 7fe399a61640 1 -- 172.21.15.16:0/1450465531 >> 172.21.15.16:0/1450465531 conn(0x7fe3940fc7d0 msgr2=0x7fe3940febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:01:01.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:01.455+0000 7fe399a61640 1 -- 172.21.15.16:0/1450465531 shutdown_connections 2024-09-12T02:01:01.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:01.455+0000 7fe399a61640 1 -- 172.21.15.16:0/1450465531 wait complete. 2024-09-12T02:01:01.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:01.455+0000 7fe399a61640 1 Processor -- start 2024-09-12T02:01:01.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:01.455+0000 7fe399a61640 1 -- start start 2024-09-12T02:01:01.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:01.456+0000 7fe399a61640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe394100e20 0x7fe39419a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:01.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:01.456+0000 7fe399a61640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe394102570 0x7fe39419ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:01.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:01.456+0000 7fe399a61640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe394102ef0 0x7fe39419efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:01.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:01.456+0000 7fe399a61640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe394110f50 con 0x7fe394100e20 2024-09-12T02:01:01.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:01.456+0000 7fe399a61640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe394110dd0 con 0x7fe394102ef0 2024-09-12T02:01:01.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:01.456+0000 7fe399a61640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe3941110d0 con 0x7fe394102570 2024-09-12T02:01:01.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:01.456+0000 7fe392ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe394100e20 0x7fe39419a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:01.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:01.456+0000 7fe392ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe394100e20 0x7fe39419a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38578/0 (socket says 172.21.15.16:38578) 2024-09-12T02:01:01.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:01.456+0000 7fe392ffd640 1 -- 172.21.15.16:0/131373972 learned_addr learned my addr 172.21.15.16:0/131373972 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:01:01.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:01.456+0000 7fe3937fe640 1 --2- 172.21.15.16:0/131373972 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe394102ef0 0x7fe39419efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:01.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:01.456+0000 7fe3927fc640 1 --2- 172.21.15.16:0/131373972 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe394102570 0x7fe39419ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:01.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:01.456+0000 7fe392ffd640 1 -- 172.21.15.16:0/131373972 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe394102570 msgr2=0x7fe39419ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:01.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:01.456+0000 7fe392ffd640 1 --2- 172.21.15.16:0/131373972 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe394102570 0x7fe39419ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:01.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:01.456+0000 7fe392ffd640 1 -- 172.21.15.16:0/131373972 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe394102ef0 msgr2=0x7fe39419efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:01.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:01.456+0000 7fe392ffd640 1 --2- 172.21.15.16:0/131373972 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe394102ef0 0x7fe39419efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:01.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:01.456+0000 7fe392ffd640 1 -- 172.21.15.16:0/131373972 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe39419f6b0 con 0x7fe394100e20 2024-09-12T02:01:01.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:01.456+0000 7fe3937fe640 1 --2- 172.21.15.16:0/131373972 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe394102ef0 0x7fe39419efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:01:01.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:01.456+0000 7fe392ffd640 1 --2- 172.21.15.16:0/131373972 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe394100e20 0x7fe39419a700 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fe38c00c920 tx=0x7fe38c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:01:01.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:01.456+0000 7fe398a5f640 1 -- 172.21.15.16:0/131373972 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe38c007c70 con 0x7fe394100e20 2024-09-12T02:01:01.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:01.456+0000 7fe398a5f640 1 -- 172.21.15.16:0/131373972 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe38c007e10 con 0x7fe394100e20 2024-09-12T02:01:01.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:01.456+0000 7fe399a61640 1 -- 172.21.15.16:0/131373972 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe39419f9a0 con 0x7fe394100e20 2024-09-12T02:01:01.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:01.456+0000 7fe398a5f640 1 -- 172.21.15.16:0/131373972 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe38c018890 con 0x7fe394100e20 2024-09-12T02:01:01.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:01.457+0000 7fe399a61640 1 -- 172.21.15.16:0/131373972 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3941a7280 con 0x7fe394100e20 2024-09-12T02:01:01.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:01.457+0000 7fe399a61640 1 -- 172.21.15.16:0/131373972 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe394101200 con 0x7fe394100e20 2024-09-12T02:01:01.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:01.459+0000 7fe398a5f640 1 -- 172.21.15.16:0/131373972 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fe38c018a50 con 0x7fe394100e20 2024-09-12T02:01:01.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:01.460+0000 7fe398a5f640 1 --2- 172.21.15.16:0/131373972 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fe3800a2c90 0x7fe3800a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:01.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:01.460+0000 7fe3927fc640 1 --2- 172.21.15.16:0/131373972 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fe3800a2c90 0x7fe3800a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:01.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:01.460+0000 7fe398a5f640 1 -- 172.21.15.16:0/131373972 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fe38c0cf5c0 con 0x7fe394100e20 2024-09-12T02:01:01.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:01.461+0000 7fe3927fc640 1 --2- 172.21.15.16:0/131373972 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fe3800a2c90 0x7fe3800a5150 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fe39410de80 tx=0x7fe37c005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:01:01.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:01.463+0000 7fe398a5f640 1 -- 172.21.15.16:0/131373972 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe38c014030 con 0x7fe394100e20 2024-09-12T02:01:01.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:01.677+0000 7fe399a61640 1 -- 172.21.15.16:0/131373972 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fe39419ba60 con 0x7fe394100e20 2024-09-12T02:01:01.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:01.678+0000 7fe398a5f640 1 -- 172.21.15.16:0/131373972 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7fe38c09c4d0 con 0x7fe394100e20 2024-09-12T02:01:01.678 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T02:01:01.678 INFO:teuthology.orchestra.run.smithi016.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-12T02:01:01.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:01.680+0000 7fe399a61640 1 -- 172.21.15.16:0/131373972 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fe3800a2c90 msgr2=0x7fe3800a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:01.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:01.680+0000 7fe399a61640 1 --2- 172.21.15.16:0/131373972 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fe3800a2c90 0x7fe3800a5150 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fe39410de80 tx=0x7fe37c005a20 comp rx=0 tx=0).stop 2024-09-12T02:01:01.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:01.680+0000 7fe399a61640 1 -- 172.21.15.16:0/131373972 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe394100e20 msgr2=0x7fe39419a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:01.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:01.680+0000 7fe399a61640 1 --2- 172.21.15.16:0/131373972 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe394100e20 0x7fe39419a700 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fe38c00c920 tx=0x7fe38c00cde0 comp rx=0 tx=0).stop 2024-09-12T02:01:01.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:01.680+0000 7fe399a61640 1 -- 172.21.15.16:0/131373972 shutdown_connections 2024-09-12T02:01:01.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:01.680+0000 7fe399a61640 1 --2- 172.21.15.16:0/131373972 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fe3800a2c90 0x7fe3800a5150 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:01.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:01.680+0000 7fe399a61640 1 --2- 172.21.15.16:0/131373972 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe394102ef0 0x7fe39419efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:01.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:01.680+0000 7fe399a61640 1 --2- 172.21.15.16:0/131373972 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe394102570 0x7fe39419ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:01.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:01.680+0000 7fe399a61640 1 --2- 172.21.15.16:0/131373972 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe394100e20 0x7fe39419a700 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:01.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:01.680+0000 7fe399a61640 1 -- 172.21.15.16:0/131373972 >> 172.21.15.16:0/131373972 conn(0x7fe3940fc7d0 msgr2=0x7fe394069f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:01:01.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:01.680+0000 7fe399a61640 1 -- 172.21.15.16:0/131373972 shutdown_connections 2024-09-12T02:01:01.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:01.680+0000 7fe399a61640 1 -- 172.21.15.16:0/131373972 wait complete. 2024-09-12T02:01:02.226 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:01:02 smithi016 ceph-mon[30947]: pgmap v308: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-12T02:01:02.226 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:01:02 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/131373972' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-12T02:01:02.256 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-09-12T02:01:02.257 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:01:02.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:01:02 smithi184 ceph-mon[35593]: pgmap v308: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-12T02:01:02.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:01:02 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/131373972' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-12T02:01:02.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:01:02 smithi033 ceph-mon[35332]: pgmap v308: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-12T02:01:02.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:01:02 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/131373972' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-12T02:01:02.517 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:01:02.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:02.937+0000 7fd2c968e640 1 -- 172.21.15.16:0/286514358 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2c41008d0 msgr2=0x7fd2c4100cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:02.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:02.937+0000 7fd2c968e640 1 --2- 172.21.15.16:0/286514358 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2c41008d0 0x7fd2c4100cb0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fd2b4009900 tx=0x7fd2b402a410 comp rx=0 tx=0).stop 2024-09-12T02:01:02.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:02.937+0000 7fd2c968e640 1 -- 172.21.15.16:0/286514358 shutdown_connections 2024-09-12T02:01:02.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:02.937+0000 7fd2c968e640 1 --2- 172.21.15.16:0/286514358 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2c410e480 0x7fd2c4110870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:02.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:02.937+0000 7fd2c968e640 1 --2- 172.21.15.16:0/286514358 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd2c4101280 0x7fd2c410df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:02.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:02.937+0000 7fd2c968e640 1 --2- 172.21.15.16:0/286514358 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2c41008d0 0x7fd2c4100cb0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:02.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:02.937+0000 7fd2c968e640 1 -- 172.21.15.16:0/286514358 >> 172.21.15.16:0/286514358 conn(0x7fd2c40fc7a0 msgr2=0x7fd2c40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:01:02.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:02.937+0000 7fd2c968e640 1 -- 172.21.15.16:0/286514358 shutdown_connections 2024-09-12T02:01:02.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:02.937+0000 7fd2c968e640 1 -- 172.21.15.16:0/286514358 wait complete. 2024-09-12T02:01:02.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:02.937+0000 7fd2c968e640 1 Processor -- start 2024-09-12T02:01:02.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:02.938+0000 7fd2c968e640 1 -- start start 2024-09-12T02:01:02.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:02.939+0000 7fd2c968e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd2c41008d0 0x7fd2c41a0580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:02.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:02.939+0000 7fd2c968e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2c4101280 0x7fd2c41a0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:02.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:02.939+0000 7fd2c968e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2c410e480 0x7fd2c419a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:02.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:02.939+0000 7fd2c968e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd2c4114260 con 0x7fd2c410e480 2024-09-12T02:01:02.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:02.939+0000 7fd2c968e640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd2c41140e0 con 0x7fd2c4101280 2024-09-12T02:01:02.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:02.939+0000 7fd2c968e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd2c41143e0 con 0x7fd2c41008d0 2024-09-12T02:01:02.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:02.939+0000 7fd2c37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2c410e480 0x7fd2c419a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:02.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:02.939+0000 7fd2c37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2c410e480 0x7fd2c419a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38594/0 (socket says 172.21.15.16:38594) 2024-09-12T02:01:02.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:02.939+0000 7fd2c37fe640 1 -- 172.21.15.16:0/2221545583 learned_addr learned my addr 172.21.15.16:0/2221545583 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:01:02.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:02.939+0000 7fd2bbfff640 1 --2- 172.21.15.16:0/2221545583 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2c4101280 0x7fd2c41a0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:02.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:02.939+0000 7fd2c2ffd640 1 --2- 172.21.15.16:0/2221545583 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd2c41008d0 0x7fd2c41a0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:02.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:02.939+0000 7fd2c37fe640 1 -- 172.21.15.16:0/2221545583 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd2c41008d0 msgr2=0x7fd2c41a0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:02.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:02.939+0000 7fd2c37fe640 1 --2- 172.21.15.16:0/2221545583 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd2c41008d0 0x7fd2c41a0580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:02.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:02.939+0000 7fd2c37fe640 1 -- 172.21.15.16:0/2221545583 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2c4101280 msgr2=0x7fd2c41a0ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:02.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:02.939+0000 7fd2c37fe640 1 --2- 172.21.15.16:0/2221545583 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2c4101280 0x7fd2c41a0ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:02.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:02.939+0000 7fd2c37fe640 1 -- 172.21.15.16:0/2221545583 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2c419aec0 con 0x7fd2c410e480 2024-09-12T02:01:02.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:02.939+0000 7fd2c37fe640 1 --2- 172.21.15.16:0/2221545583 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2c410e480 0x7fd2c419a650 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc00b9e0 tx=0x7fd2bc00bde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:01:02.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:02.940+0000 7fd2c0ff9640 1 -- 172.21.15.16:0/2221545583 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2bc019070 con 0x7fd2c410e480 2024-09-12T02:01:02.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:02.940+0000 7fd2c0ff9640 1 -- 172.21.15.16:0/2221545583 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd2bc014440 con 0x7fd2c410e480 2024-09-12T02:01:02.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:02.940+0000 7fd2c968e640 1 -- 172.21.15.16:0/2221545583 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2c419b1b0 con 0x7fd2c410e480 2024-09-12T02:01:02.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:02.940+0000 7fd2c0ff9640 1 -- 172.21.15.16:0/2221545583 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2bc0125a0 con 0x7fd2c410e480 2024-09-12T02:01:02.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:02.940+0000 7fd2c968e640 1 -- 172.21.15.16:0/2221545583 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2c41a7500 con 0x7fd2c410e480 2024-09-12T02:01:02.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:02.941+0000 7fd2c968e640 1 -- 172.21.15.16:0/2221545583 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2c406b070 con 0x7fd2c410e480 2024-09-12T02:01:02.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:02.942+0000 7fd2c0ff9640 1 -- 172.21.15.16:0/2221545583 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fd2bc01c070 con 0x7fd2c410e480 2024-09-12T02:01:02.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:02.943+0000 7fd2c0ff9640 1 --2- 172.21.15.16:0/2221545583 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fd2b00a2c90 0x7fd2b00a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:02.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:02.943+0000 7fd2c2ffd640 1 --2- 172.21.15.16:0/2221545583 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fd2b00a2c90 0x7fd2b00a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:02.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:02.943+0000 7fd2c0ff9640 1 -- 172.21.15.16:0/2221545583 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fd2bc0ce460 con 0x7fd2c410e480 2024-09-12T02:01:02.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:02.943+0000 7fd2c2ffd640 1 --2- 172.21.15.16:0/2221545583 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fd2b00a2c90 0x7fd2b00a5150 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fd2b4000c00 tx=0x7fd2b40058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:01:02.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:02.947+0000 7fd2c0ff9640 1 -- 172.21.15.16:0/2221545583 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd2bc09ae20 con 0x7fd2c410e480 2024-09-12T02:01:03.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:03.156+0000 7fd2c968e640 1 -- 172.21.15.16:0/2221545583 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd2c4109610 con 0x7fd2c410e480 2024-09-12T02:01:03.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:03.157+0000 7fd2c0ff9640 1 -- 172.21.15.16:0/2221545583 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11911 (secure 0 0 0) 0x7fd2bc09f120 con 0x7fd2c410e480 2024-09-12T02:01:03.157 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T02:01:03.157 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":24,"btime":"2024-09-12T02:00:07:547152+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3279123279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3279123279},{"type":"v1","addr":"172.21.15.16:6835","nonce":3279123279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4119452382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4119452382},{"type":"v1","addr":"172.21.15.16:6837","nonce":4119452382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/3263823165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3263823165},{"type":"v1","addr":"172.21.15.16:6839","nonce":3263823165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/1510290732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":1510290732},{"type":"v1","addr":"172.21.15.16:6841","nonce":1510290732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/279674603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":279674603},{"type":"v1","addr":"172.21.15.33:6833","nonce":279674603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3555728028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3555728028},{"type":"v1","addr":"172.21.15.33:6835","nonce":3555728028}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T01:59:56.117608+0000","modified":"2024-09-12T02:00:07.097363+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24482,"mds_1":24481,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6833/1563374106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1563374106},{"type":"v1","addr":"172.21.15.184:6833","nonce":1563374106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6837/1892833868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1892833868},{"type":"v1","addr":"172.21.15.33:6837","nonce":1892833868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/1457679686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1457679686},{"type":"v1","addr":"172.21.15.184:6835","nonce":1457679686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.184:6837/1793393087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":1793393087},{"type":"v1","addr":"172.21.15.184:6837","nonce":1793393087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.184:6839/4203347941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":4203347941},{"type":"v1","addr":"172.21.15.184:6839","nonce":4203347941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.33:6839/4136670204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":4136670204},{"type":"v1","addr":"172.21.15.33:6839","nonce":4136670204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24469,24482,24481]},"id":1}]} 2024-09-12T02:01:03.157 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 24 2024-09-12T02:01:03.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:03.159+0000 7fd2c968e640 1 -- 172.21.15.16:0/2221545583 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fd2b00a2c90 msgr2=0x7fd2b00a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:03.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:03.159+0000 7fd2c968e640 1 --2- 172.21.15.16:0/2221545583 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fd2b00a2c90 0x7fd2b00a5150 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fd2b4000c00 tx=0x7fd2b40058d0 comp rx=0 tx=0).stop 2024-09-12T02:01:03.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:03.159+0000 7fd2c968e640 1 -- 172.21.15.16:0/2221545583 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2c410e480 msgr2=0x7fd2c419a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:03.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:03.159+0000 7fd2c968e640 1 --2- 172.21.15.16:0/2221545583 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2c410e480 0x7fd2c419a650 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc00b9e0 tx=0x7fd2bc00bde0 comp rx=0 tx=0).stop 2024-09-12T02:01:03.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:03.159+0000 7fd2c968e640 1 -- 172.21.15.16:0/2221545583 shutdown_connections 2024-09-12T02:01:03.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:03.159+0000 7fd2c968e640 1 --2- 172.21.15.16:0/2221545583 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fd2b00a2c90 0x7fd2b00a5150 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:03.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:03.159+0000 7fd2c968e640 1 --2- 172.21.15.16:0/2221545583 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2c410e480 0x7fd2c419a650 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:03.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:03.159+0000 7fd2c968e640 1 --2- 172.21.15.16:0/2221545583 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2c4101280 0x7fd2c41a0ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:03.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:03.159+0000 7fd2c968e640 1 --2- 172.21.15.16:0/2221545583 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd2c41008d0 0x7fd2c41a0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:03.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:03.159+0000 7fd2c968e640 1 -- 172.21.15.16:0/2221545583 >> 172.21.15.16:0/2221545583 conn(0x7fd2c40fc7a0 msgr2=0x7fd2c40feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:01:03.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:03.159+0000 7fd2c968e640 1 -- 172.21.15.16:0/2221545583 shutdown_connections 2024-09-12T02:01:03.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:03.159+0000 7fd2c968e640 1 -- 172.21.15.16:0/2221545583 wait complete. 2024-09-12T02:01:03.789 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:01:04.048 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:01:04.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:01:04 smithi184 ceph-mon[35593]: pgmap v309: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-12T02:01:04.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:01:04 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/2221545583' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:01:04.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:01:04 smithi033 ceph-mon[35332]: pgmap v309: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-12T02:01:04.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:01:04 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/2221545583' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:01:04.429 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:01:04 smithi016 ceph-mon[30947]: pgmap v309: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-12T02:01:04.429 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:01:04 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/2221545583' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:01:04.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:04.511+0000 7f448a51f640 1 -- 172.21.15.16:0/1942664154 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4484100e20 msgr2=0x7f4484101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:04.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:04.511+0000 7f448a51f640 1 --2- 172.21.15.16:0/1942664154 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4484100e20 0x7f4484101200 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f4474009900 tx=0x7f447402a480 comp rx=0 tx=0).stop 2024-09-12T02:01:04.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:04.511+0000 7f448a51f640 1 -- 172.21.15.16:0/1942664154 shutdown_connections 2024-09-12T02:01:04.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:04.511+0000 7f448a51f640 1 --2- 172.21.15.16:0/1942664154 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4484102ef0 0x7f448410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:04.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:04.511+0000 7f448a51f640 1 --2- 172.21.15.16:0/1942664154 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4484102570 0x7f44841029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:04.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:04.511+0000 7f448a51f640 1 --2- 172.21.15.16:0/1942664154 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4484100e20 0x7f4484101200 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:04.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:04.511+0000 7f448a51f640 1 -- 172.21.15.16:0/1942664154 >> 172.21.15.16:0/1942664154 conn(0x7f44840fc7d0 msgr2=0x7f44840febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:01:04.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:04.511+0000 7f448a51f640 1 -- 172.21.15.16:0/1942664154 shutdown_connections 2024-09-12T02:01:04.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:04.511+0000 7f448a51f640 1 -- 172.21.15.16:0/1942664154 wait complete. 2024-09-12T02:01:04.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:04.512+0000 7f448a51f640 1 Processor -- start 2024-09-12T02:01:04.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:04.512+0000 7f448a51f640 1 -- start start 2024-09-12T02:01:04.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:04.512+0000 7f448a51f640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4484100e20 0x7f4484103750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:04.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:04.512+0000 7f448a51f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4484102570 0x7f4484103c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:04.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:04.512+0000 7f448a51f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4484102ef0 0x7f44841041d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:04.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:04.512+0000 7f448a51f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4484110f80 con 0x7f4484102570 2024-09-12T02:01:04.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:04.512+0000 7f448a51f640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4484110e00 con 0x7f4484102ef0 2024-09-12T02:01:04.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:04.512+0000 7f448a51f640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f4484111100 con 0x7f4484100e20 2024-09-12T02:01:04.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:04.512+0000 7f447bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4484102570 0x7f4484103c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:04.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:04.512+0000 7f447bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4484102570 0x7f4484103c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38606/0 (socket says 172.21.15.16:38606) 2024-09-12T02:01:04.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:04.512+0000 7f447bfff640 1 -- 172.21.15.16:0/1831589094 learned_addr learned my addr 172.21.15.16:0/1831589094 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:01:04.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:04.512+0000 7f4488abc640 1 --2- 172.21.15.16:0/1831589094 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4484102ef0 0x7f44841041d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:04.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:04.512+0000 7f4483fff640 1 --2- 172.21.15.16:0/1831589094 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4484100e20 0x7f4484103750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:04.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:04.512+0000 7f447bfff640 1 -- 172.21.15.16:0/1831589094 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4484100e20 msgr2=0x7f4484103750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:04.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:04.512+0000 7f447bfff640 1 --2- 172.21.15.16:0/1831589094 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4484100e20 0x7f4484103750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:04.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:04.512+0000 7f447bfff640 1 -- 172.21.15.16:0/1831589094 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4484102ef0 msgr2=0x7f44841041d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:04.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:04.512+0000 7f447bfff640 1 --2- 172.21.15.16:0/1831589094 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4484102ef0 0x7f44841041d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:04.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:04.512+0000 7f447bfff640 1 -- 172.21.15.16:0/1831589094 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44841a6940 con 0x7f4484102570 2024-09-12T02:01:04.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:04.513+0000 7f447bfff640 1 --2- 172.21.15.16:0/1831589094 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4484102570 0x7f4484103c90 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f446c00c890 tx=0x7f446c00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:01:04.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:04.513+0000 7f4481ffb640 1 -- 172.21.15.16:0/1831589094 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f446c004220 con 0x7f4484102570 2024-09-12T02:01:04.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:04.513+0000 7f4481ffb640 1 -- 172.21.15.16:0/1831589094 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f446c0043c0 con 0x7f4484102570 2024-09-12T02:01:04.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:04.513+0000 7f4481ffb640 1 -- 172.21.15.16:0/1831589094 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f446c0049e0 con 0x7f4484102570 2024-09-12T02:01:04.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:04.513+0000 7f448a51f640 1 -- 172.21.15.16:0/1831589094 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44841a6c30 con 0x7f4484102570 2024-09-12T02:01:04.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:04.513+0000 7f448a51f640 1 -- 172.21.15.16:0/1831589094 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44841a7120 con 0x7f4484102570 2024-09-12T02:01:04.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:04.514+0000 7f448a51f640 1 -- 172.21.15.16:0/1831589094 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4484101200 con 0x7f4484102570 2024-09-12T02:01:04.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:04.515+0000 7f4481ffb640 1 -- 172.21.15.16:0/1831589094 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f446c0079b0 con 0x7f4484102570 2024-09-12T02:01:04.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:04.516+0000 7f4481ffb640 1 --2- 172.21.15.16:0/1831589094 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f44700a2c10 0x7f44700a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:04.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:04.516+0000 7f4481ffb640 1 -- 172.21.15.16:0/1831589094 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f446c0cf130 con 0x7f4484102570 2024-09-12T02:01:04.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:04.516+0000 7f4483fff640 1 --2- 172.21.15.16:0/1831589094 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f44700a2c10 0x7f44700a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:04.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:04.516+0000 7f4483fff640 1 --2- 172.21.15.16:0/1831589094 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f44700a2c10 0x7f44700a50d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f4474005ab0 tx=0x7f4474005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:01:04.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:04.520+0000 7f4481ffb640 1 -- 172.21.15.16:0/1831589094 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f446c014030 con 0x7f4484102570 2024-09-12T02:01:04.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:04.730+0000 7f448a51f640 1 -- 172.21.15.16:0/1831589094 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4484104c80 con 0x7f4484102570 2024-09-12T02:01:04.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:04.731+0000 7f4481ffb640 1 -- 172.21.15.16:0/1831589094 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11911 (secure 0 0 0) 0x7f4484104c80 con 0x7f4484102570 2024-09-12T02:01:04.731 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T02:01:04.731 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":24,"btime":"2024-09-12T02:00:07:547152+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3279123279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3279123279},{"type":"v1","addr":"172.21.15.16:6835","nonce":3279123279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4119452382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4119452382},{"type":"v1","addr":"172.21.15.16:6837","nonce":4119452382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/3263823165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3263823165},{"type":"v1","addr":"172.21.15.16:6839","nonce":3263823165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/1510290732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":1510290732},{"type":"v1","addr":"172.21.15.16:6841","nonce":1510290732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/279674603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":279674603},{"type":"v1","addr":"172.21.15.33:6833","nonce":279674603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3555728028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3555728028},{"type":"v1","addr":"172.21.15.33:6835","nonce":3555728028}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T01:59:56.117608+0000","modified":"2024-09-12T02:00:07.097363+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24482,"mds_1":24481,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6833/1563374106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1563374106},{"type":"v1","addr":"172.21.15.184:6833","nonce":1563374106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6837/1892833868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1892833868},{"type":"v1","addr":"172.21.15.33:6837","nonce":1892833868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/1457679686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1457679686},{"type":"v1","addr":"172.21.15.184:6835","nonce":1457679686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.184:6837/1793393087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":1793393087},{"type":"v1","addr":"172.21.15.184:6837","nonce":1793393087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.184:6839/4203347941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":4203347941},{"type":"v1","addr":"172.21.15.184:6839","nonce":4203347941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.33:6839/4136670204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":4136670204},{"type":"v1","addr":"172.21.15.33:6839","nonce":4136670204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24469,24482,24481]},"id":1}]} 2024-09-12T02:01:04.731 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 24 2024-09-12T02:01:04.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:04.733+0000 7f448a51f640 1 -- 172.21.15.16:0/1831589094 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f44700a2c10 msgr2=0x7f44700a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:04.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:04.733+0000 7f448a51f640 1 --2- 172.21.15.16:0/1831589094 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f44700a2c10 0x7f44700a50d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f4474005ab0 tx=0x7f4474005a20 comp rx=0 tx=0).stop 2024-09-12T02:01:04.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:04.733+0000 7f448a51f640 1 -- 172.21.15.16:0/1831589094 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4484102570 msgr2=0x7f4484103c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:04.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:04.733+0000 7f448a51f640 1 --2- 172.21.15.16:0/1831589094 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4484102570 0x7f4484103c90 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f446c00c890 tx=0x7f446c00cd50 comp rx=0 tx=0).stop 2024-09-12T02:01:04.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:04.733+0000 7f448a51f640 1 -- 172.21.15.16:0/1831589094 shutdown_connections 2024-09-12T02:01:04.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:04.733+0000 7f448a51f640 1 --2- 172.21.15.16:0/1831589094 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f44700a2c10 0x7f44700a50d0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:04.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:04.733+0000 7f448a51f640 1 --2- 172.21.15.16:0/1831589094 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4484102ef0 0x7f44841041d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:04.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:04.733+0000 7f448a51f640 1 --2- 172.21.15.16:0/1831589094 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4484102570 0x7f4484103c90 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:04.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:04.733+0000 7f448a51f640 1 --2- 172.21.15.16:0/1831589094 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4484100e20 0x7f4484103750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:04.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:04.733+0000 7f448a51f640 1 -- 172.21.15.16:0/1831589094 >> 172.21.15.16:0/1831589094 conn(0x7f44840fc7d0 msgr2=0x7f448406a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:01:04.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:04.733+0000 7f448a51f640 1 -- 172.21.15.16:0/1831589094 shutdown_connections 2024-09-12T02:01:04.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:04.733+0000 7f448a51f640 1 -- 172.21.15.16:0/1831589094 wait complete. 2024-09-12T02:01:05.325 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:01:05 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/1831589094' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:01:05.354 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T02:01:05.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:01:05 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/1831589094' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:01:05.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:01:05 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/1831589094' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:01:05.615 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:01:06.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:06.205+0000 7f5ffd112640 1 -- 172.21.15.16:0/2271644631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ff8102ef0 msgr2=0x7f5ff810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:06.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:06.205+0000 7f5ffd112640 1 --2- 172.21.15.16:0/2271644631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ff8102ef0 0x7f5ff810da40 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f5ff0009900 tx=0x7f5ff002a480 comp rx=0 tx=0).stop 2024-09-12T02:01:06.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:06.205+0000 7f5ffd112640 1 -- 172.21.15.16:0/2271644631 shutdown_connections 2024-09-12T02:01:06.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:06.205+0000 7f5ffd112640 1 --2- 172.21.15.16:0/2271644631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ff8102ef0 0x7f5ff810da40 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:06.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:06.205+0000 7f5ffd112640 1 --2- 172.21.15.16:0/2271644631 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5ff8102570 0x7f5ff81029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:06.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:06.205+0000 7f5ffd112640 1 --2- 172.21.15.16:0/2271644631 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ff8100e20 0x7f5ff8101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:06.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:06.205+0000 7f5ffd112640 1 -- 172.21.15.16:0/2271644631 >> 172.21.15.16:0/2271644631 conn(0x7f5ff80fc7d0 msgr2=0x7f5ff80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:01:06.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:06.205+0000 7f5ffd112640 1 -- 172.21.15.16:0/2271644631 shutdown_connections 2024-09-12T02:01:06.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:06.205+0000 7f5ffd112640 1 -- 172.21.15.16:0/2271644631 wait complete. 2024-09-12T02:01:06.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:06.206+0000 7f5ffd112640 1 Processor -- start 2024-09-12T02:01:06.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:06.206+0000 7f5ffd112640 1 -- start start 2024-09-12T02:01:06.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:06.206+0000 7f5ffd112640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ff8100e20 0x7f5ff80fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:06.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:06.206+0000 7f5ffd112640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5ff8102570 0x7f5ff80fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:06.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:06.206+0000 7f5ffd112640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ff8102ef0 0x7f5ff80fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:06.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:06.206+0000 7f5ffd112640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5ff8110f80 con 0x7f5ff8102ef0 2024-09-12T02:01:06.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:06.206+0000 7f5ffd112640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5ff8110e00 con 0x7f5ff8100e20 2024-09-12T02:01:06.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:06.206+0000 7f5ffd112640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5ff8111100 con 0x7f5ff8102570 2024-09-12T02:01:06.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:06.206+0000 7f5ff759e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ff8102ef0 0x7f5ff80fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:06.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:06.206+0000 7f5ff6d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ff8100e20 0x7f5ff80fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:06.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:06.207+0000 7f5ff759e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ff8102ef0 0x7f5ff80fd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38624/0 (socket says 172.21.15.16:38624) 2024-09-12T02:01:06.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:06.207+0000 7f5ff6d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ff8100e20 0x7f5ff80fcc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.16:39506/0 (socket says 172.21.15.16:39506) 2024-09-12T02:01:06.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:06.207+0000 7f5ff759e640 1 -- 172.21.15.16:0/2164048963 learned_addr learned my addr 172.21.15.16:0/2164048963 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:01:06.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:06.207+0000 7f5ff659c640 1 --2- 172.21.15.16:0/2164048963 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5ff8102570 0x7f5ff80fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:06.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:06.207+0000 7f5ff759e640 1 -- 172.21.15.16:0/2164048963 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5ff8102570 msgr2=0x7f5ff80fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:06.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:06.207+0000 7f5ff759e640 1 --2- 172.21.15.16:0/2164048963 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5ff8102570 0x7f5ff80fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:06.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:06.207+0000 7f5ff759e640 1 -- 172.21.15.16:0/2164048963 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ff8100e20 msgr2=0x7f5ff80fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:06.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:06.207+0000 7f5ff759e640 1 --2- 172.21.15.16:0/2164048963 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ff8100e20 0x7f5ff80fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:06.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:06.207+0000 7f5ff759e640 1 -- 172.21.15.16:0/2164048963 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ff81a6950 con 0x7f5ff8102ef0 2024-09-12T02:01:06.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:06.207+0000 7f5ff6d9d640 1 --2- 172.21.15.16:0/2164048963 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ff8100e20 0x7f5ff80fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:01:06.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:06.207+0000 7f5ff759e640 1 --2- 172.21.15.16:0/2164048963 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ff8102ef0 0x7f5ff80fd700 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f5ff0004770 tx=0x7f5ff0003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:01:06.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:06.207+0000 7f5fdffff640 1 -- 172.21.15.16:0/2164048963 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ff003f070 con 0x7f5ff8102ef0 2024-09-12T02:01:06.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:06.207+0000 7f5fdffff640 1 -- 172.21.15.16:0/2164048963 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ff0031c30 con 0x7f5ff8102ef0 2024-09-12T02:01:06.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:06.207+0000 7f5ffd112640 1 -- 172.21.15.16:0/2164048963 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ff81a6be0 con 0x7f5ff8102ef0 2024-09-12T02:01:06.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:06.207+0000 7f5fdffff640 1 -- 172.21.15.16:0/2164048963 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ff0035030 con 0x7f5ff8102ef0 2024-09-12T02:01:06.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:06.207+0000 7f5ffd112640 1 -- 172.21.15.16:0/2164048963 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ff81a70c0 con 0x7f5ff8102ef0 2024-09-12T02:01:06.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:06.208+0000 7f5ffd112640 1 -- 172.21.15.16:0/2164048963 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ff8101200 con 0x7f5ff8102ef0 2024-09-12T02:01:06.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:06.210+0000 7f5fdffff640 1 -- 172.21.15.16:0/2164048963 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f5ff0042070 con 0x7f5ff8102ef0 2024-09-12T02:01:06.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:06.211+0000 7f5fdffff640 1 --2- 172.21.15.16:0/2164048963 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f5fe00a2c10 0x7f5fe00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:06.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:06.211+0000 7f5ff6d9d640 1 --2- 172.21.15.16:0/2164048963 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f5fe00a2c10 0x7f5fe00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:06.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:06.211+0000 7f5fdffff640 1 -- 172.21.15.16:0/2164048963 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f5ff00f00f0 con 0x7f5ff8102ef0 2024-09-12T02:01:06.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:06.211+0000 7f5ff6d9d640 1 --2- 172.21.15.16:0/2164048963 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f5fe00a2c10 0x7f5fe00a50d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f5fe8009fc0 tx=0x7f5fe8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:01:06.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:06.214+0000 7f5fdffff640 1 -- 172.21.15.16:0/2164048963 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5ff00bc8a0 con 0x7f5ff8102ef0 2024-09-12T02:01:06.271 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:01:06 smithi016 ceph-mon[30947]: pgmap v310: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-12T02:01:06.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:01:06 smithi184 ceph-mon[35593]: pgmap v310: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-12T02:01:06.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:06.392+0000 7f5ffd112640 1 -- 172.21.15.16:0/2164048963 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5ff810a0a0 con 0x7f5ff8102ef0 2024-09-12T02:01:06.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:06.394+0000 7f5fdffff640 1 -- 172.21.15.16:0/2164048963 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+22584 (secure 0 0 0) 0x7f5ff00c0ba0 con 0x7f5ff8102ef0 2024-09-12T02:01:06.394 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T02:01:06.394 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":82,"fsid":"879f3b56-70a9-11ef-bcea-c7b262605968","created":"2024-09-12T01:52:12.841045+0000","modified":"2024-09-12T02:00:09.559511+0000","last_up_change":"2024-09-12T01:59:08.975443+0000","last_in_change":"2024-09-12T01:58:44.530220+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T01:55:26.727564+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T01:59:52.402097+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T01:59:54.463627+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3ea41eb6-e384-4ddc-810f-0f103c127849","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4000187867},{"type":"v1","addr":"172.21.15.16:6803","nonce":4000187867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4000187867},{"type":"v1","addr":"172.21.15.16:6805","nonce":4000187867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":4000187867},{"type":"v1","addr":"172.21.15.16:6809","nonce":4000187867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4000187867},{"type":"v1","addr":"172.21.15.16:6807","nonce":4000187867}]},"public_addr":"172.21.15.16:6803/4000187867","cluster_addr":"172.21.15.16:6805/4000187867","heartbeat_back_addr":"172.21.15.16:6809/4000187867","heartbeat_front_addr":"172.21.15.16:6807/4000187867","state":["exists","up"]},{"osd":1,"uuid":"07bf63e7-70fa-41b2-af53-12a5fec3f33c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":3862661148},{"type":"v1","addr":"172.21.15.33:6801","nonce":3862661148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":3862661148},{"type":"v1","addr":"172.21.15.33:6803","nonce":3862661148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":3862661148},{"type":"v1","addr":"172.21.15.33:6807","nonce":3862661148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":3862661148},{"type":"v1","addr":"172.21.15.33:6805","nonce":3862661148}]},"public_addr":"172.21.15.33:6801/3862661148","cluster_addr":"172.21.15.33:6803/3862661148","heartbeat_back_addr":"172.21.15.33:6807/3862661148","heartbeat_front_addr":"172.21.15.33:6805/3862661148","state":["exists","up"]},{"osd":2,"uuid":"22e40cac-3bc0-4cce-b47b-77499b767f7e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":3958753963},{"type":"v1","addr":"172.21.15.184:6801","nonce":3958753963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":3958753963},{"type":"v1","addr":"172.21.15.184:6803","nonce":3958753963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":3958753963},{"type":"v1","addr":"172.21.15.184:6807","nonce":3958753963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":3958753963},{"type":"v1","addr":"172.21.15.184:6805","nonce":3958753963}]},"public_addr":"172.21.15.184:6801/3958753963","cluster_addr":"172.21.15.184:6803/3958753963","heartbeat_back_addr":"172.21.15.184:6807/3958753963","heartbeat_front_addr":"172.21.15.184:6805/3958753963","state":["exists","up"]},{"osd":3,"uuid":"834a07f5-2c24-49e1-8e4a-12c96cfbdc65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3395182524},{"type":"v1","addr":"172.21.15.16:6811","nonce":3395182524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3395182524},{"type":"v1","addr":"172.21.15.16:6813","nonce":3395182524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3395182524},{"type":"v1","addr":"172.21.15.16:6817","nonce":3395182524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3395182524},{"type":"v1","addr":"172.21.15.16:6815","nonce":3395182524}]},"public_addr":"172.21.15.16:6811/3395182524","cluster_addr":"172.21.15.16:6813/3395182524","heartbeat_back_addr":"172.21.15.16:6817/3395182524","heartbeat_front_addr":"172.21.15.16:6815/3395182524","state":["exists","up"]},{"osd":4,"uuid":"3ca56e09-c194-4404-b9bd-2524d38367eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":1355184322},{"type":"v1","addr":"172.21.15.33:6809","nonce":1355184322}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1355184322},{"type":"v1","addr":"172.21.15.33:6811","nonce":1355184322}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1355184322},{"type":"v1","addr":"172.21.15.33:6815","nonce":1355184322}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1355184322},{"type":"v1","addr":"172.21.15.33:6813","nonce":1355184322}]},"public_addr":"172.21.15.33:6809/1355184322","cluster_addr":"172.21.15.33:6811/1355184322","heartbeat_back_addr":"172.21.15.33:6815/1355184322","heartbeat_front_addr":"172.21.15.33:6813/1355184322","state":["exists","up"]},{"osd":5,"uuid":"0b05ecdb-b987-461f-951c-6d76739bfb4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":1533397276},{"type":"v1","addr":"172.21.15.184:6809","nonce":1533397276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":1533397276},{"type":"v1","addr":"172.21.15.184:6811","nonce":1533397276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":1533397276},{"type":"v1","addr":"172.21.15.184:6815","nonce":1533397276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":1533397276},{"type":"v1","addr":"172.21.15.184:6813","nonce":1533397276}]},"public_addr":"172.21.15.184:6809/1533397276","cluster_addr":"172.21.15.184:6811/1533397276","heartbeat_back_addr":"172.21.15.184:6815/1533397276","heartbeat_front_addr":"172.21.15.184:6813/1533397276","state":["exists","up"]},{"osd":6,"uuid":"b3936e21-e197-49fa-8441-d909b258b612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":902466142},{"type":"v1","addr":"172.21.15.16:6819","nonce":902466142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":902466142},{"type":"v1","addr":"172.21.15.16:6821","nonce":902466142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":902466142},{"type":"v1","addr":"172.21.15.16:6825","nonce":902466142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":902466142},{"type":"v1","addr":"172.21.15.16:6823","nonce":902466142}]},"public_addr":"172.21.15.16:6819/902466142","cluster_addr":"172.21.15.16:6821/902466142","heartbeat_back_addr":"172.21.15.16:6825/902466142","heartbeat_front_addr":"172.21.15.16:6823/902466142","state":["exists","up"]},{"osd":7,"uuid":"4ddbe011-34c1-4b3f-a8eb-577422fb5426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":4186032630},{"type":"v1","addr":"172.21.15.33:6817","nonce":4186032630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":4186032630},{"type":"v1","addr":"172.21.15.33:6819","nonce":4186032630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":4186032630},{"type":"v1","addr":"172.21.15.33:6823","nonce":4186032630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":4186032630},{"type":"v1","addr":"172.21.15.33:6821","nonce":4186032630}]},"public_addr":"172.21.15.33:6817/4186032630","cluster_addr":"172.21.15.33:6819/4186032630","heartbeat_back_addr":"172.21.15.33:6823/4186032630","heartbeat_front_addr":"172.21.15.33:6821/4186032630","state":["exists","up"]},{"osd":8,"uuid":"dca5bfb8-ac6a-47ce-905c-be7fcaf0b5ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":781851781},{"type":"v1","addr":"172.21.15.184:6817","nonce":781851781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":781851781},{"type":"v1","addr":"172.21.15.184:6819","nonce":781851781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":781851781},{"type":"v1","addr":"172.21.15.184:6823","nonce":781851781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":781851781},{"type":"v1","addr":"172.21.15.184:6821","nonce":781851781}]},"public_addr":"172.21.15.184:6817/781851781","cluster_addr":"172.21.15.184:6819/781851781","heartbeat_back_addr":"172.21.15.184:6823/781851781","heartbeat_front_addr":"172.21.15.184:6821/781851781","state":["exists","up"]},{"osd":9,"uuid":"23a05cb5-01d5-40f3-b66a-15fcccedbd6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":1569578746},{"type":"v1","addr":"172.21.15.16:6827","nonce":1569578746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":1569578746},{"type":"v1","addr":"172.21.15.16:6829","nonce":1569578746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1569578746},{"type":"v1","addr":"172.21.15.16:6833","nonce":1569578746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":1569578746},{"type":"v1","addr":"172.21.15.16:6831","nonce":1569578746}]},"public_addr":"172.21.15.16:6827/1569578746","cluster_addr":"172.21.15.16:6829/1569578746","heartbeat_back_addr":"172.21.15.16:6833/1569578746","heartbeat_front_addr":"172.21.15.16:6831/1569578746","state":["exists","up"]},{"osd":10,"uuid":"e4bd6bce-3c69-44db-bc09-d264a0ad094a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":469977847},{"type":"v1","addr":"172.21.15.33:6825","nonce":469977847}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":469977847},{"type":"v1","addr":"172.21.15.33:6827","nonce":469977847}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":469977847},{"type":"v1","addr":"172.21.15.33:6831","nonce":469977847}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":469977847},{"type":"v1","addr":"172.21.15.33:6829","nonce":469977847}]},"public_addr":"172.21.15.33:6825/469977847","cluster_addr":"172.21.15.33:6827/469977847","heartbeat_back_addr":"172.21.15.33:6831/469977847","heartbeat_front_addr":"172.21.15.33:6829/469977847","state":["exists","up"]},{"osd":11,"uuid":"d60212f8-5a1d-45f7-9a1b-12294bb4c80d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":1619871346},{"type":"v1","addr":"172.21.15.184:6825","nonce":1619871346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":1619871346},{"type":"v1","addr":"172.21.15.184:6827","nonce":1619871346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":1619871346},{"type":"v1","addr":"172.21.15.184:6831","nonce":1619871346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":1619871346},{"type":"v1","addr":"172.21.15.184:6829","nonce":1619871346}]},"public_addr":"172.21.15.184:6825/1619871346","cluster_addr":"172.21.15.184:6827/1619871346","heartbeat_back_addr":"172.21.15.184:6831/1619871346","heartbeat_front_addr":"172.21.15.184:6829/1619871346","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:54:32.057979+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:54:57.501562+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:55:23.137225+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:55:48.646155+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:56:13.311907+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:56:38.214188+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:57:02.570921+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:57:27.506097+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:57:52.038529+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:58:17.423300+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:58:41.579992+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:59:06.245817+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:0/3532565698":"2024-09-13T01:53:08.218475+0000","172.21.15.16:0/1822157179":"2024-09-13T01:52:44.194152+0000","172.21.15.16:0/3919664061":"2024-09-13T01:52:44.194152+0000","172.21.15.16:0/2269394774":"2024-09-13T01:52:44.194152+0000","172.21.15.16:0/2595308634":"2024-09-13T01:53:08.218475+0000","172.21.15.16:6801/2754331153":"2024-09-13T01:52:44.194152+0000","172.21.15.16:6801/2615861334":"2024-09-13T01:53:08.218475+0000","172.21.15.16:6800/2615861334":"2024-09-13T01:53:08.218475+0000","172.21.15.16:0/4199114771":"2024-09-13T01:53:08.218475+0000","172.21.15.16:6800/2754331153":"2024-09-13T01:52:44.194152+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T02:01:06.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:06.396+0000 7f5ffd112640 1 -- 172.21.15.16:0/2164048963 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f5fe00a2c10 msgr2=0x7f5fe00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:06.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:06.396+0000 7f5ffd112640 1 --2- 172.21.15.16:0/2164048963 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f5fe00a2c10 0x7f5fe00a50d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f5fe8009fc0 tx=0x7f5fe8005f90 comp rx=0 tx=0).stop 2024-09-12T02:01:06.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:06.396+0000 7f5ffd112640 1 -- 172.21.15.16:0/2164048963 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ff8102ef0 msgr2=0x7f5ff80fd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:06.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:06.396+0000 7f5ffd112640 1 --2- 172.21.15.16:0/2164048963 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ff8102ef0 0x7f5ff80fd700 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f5ff0004770 tx=0x7f5ff0003c10 comp rx=0 tx=0).stop 2024-09-12T02:01:06.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:06.396+0000 7f5ffd112640 1 -- 172.21.15.16:0/2164048963 shutdown_connections 2024-09-12T02:01:06.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:06.396+0000 7f5ffd112640 1 --2- 172.21.15.16:0/2164048963 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f5fe00a2c10 0x7f5fe00a50d0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:06.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:06.396+0000 7f5ffd112640 1 --2- 172.21.15.16:0/2164048963 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ff8102ef0 0x7f5ff80fd700 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:06.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:06.396+0000 7f5ffd112640 1 --2- 172.21.15.16:0/2164048963 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5ff8102570 0x7f5ff80fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:06.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:06.396+0000 7f5ffd112640 1 --2- 172.21.15.16:0/2164048963 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ff8100e20 0x7f5ff80fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:06.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:06.396+0000 7f5ffd112640 1 -- 172.21.15.16:0/2164048963 >> 172.21.15.16:0/2164048963 conn(0x7f5ff80fc7d0 msgr2=0x7f5ff806a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:01:06.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:06.396+0000 7f5ffd112640 1 -- 172.21.15.16:0/2164048963 shutdown_connections 2024-09-12T02:01:06.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:06.396+0000 7f5ffd112640 1 -- 172.21.15.16:0/2164048963 wait complete. 2024-09-12T02:01:06.418 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:01:06 smithi033 ceph-mon[35332]: pgmap v310: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-12T02:01:06.977 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:01:07.247 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:01:07.311 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:01:07 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/2164048963' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T02:01:07.418 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:01:07 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/2164048963' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T02:01:07.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:01:07 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/2164048963' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T02:01:07.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:07.744+0000 7fa7b98ff640 1 -- 172.21.15.16:0/1895245969 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7b41036e0 msgr2=0x7fa7b4107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:07.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:07.744+0000 7fa7b98ff640 1 --2- 172.21.15.16:0/1895245969 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7b41036e0 0x7fa7b4107b90 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fa79c009930 tx=0x7fa79c02a420 comp rx=0 tx=0).stop 2024-09-12T02:01:07.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:07.744+0000 7fa7b98ff640 1 -- 172.21.15.16:0/1895245969 shutdown_connections 2024-09-12T02:01:07.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:07.744+0000 7fa7b98ff640 1 --2- 172.21.15.16:0/1895245969 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa7b41082a0 0x7fa7b410ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:07.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:07.744+0000 7fa7b98ff640 1 --2- 172.21.15.16:0/1895245969 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7b41036e0 0x7fa7b4107b90 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:07.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:07.744+0000 7fa7b98ff640 1 --2- 172.21.15.16:0/1895245969 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa7b4102d30 0x7fa7b4103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:07.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:07.744+0000 7fa7b98ff640 1 -- 172.21.15.16:0/1895245969 >> 172.21.15.16:0/1895245969 conn(0x7fa7b40fc7d0 msgr2=0x7fa7b40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:01:07.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:07.744+0000 7fa7b98ff640 1 -- 172.21.15.16:0/1895245969 shutdown_connections 2024-09-12T02:01:07.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:07.744+0000 7fa7b98ff640 1 -- 172.21.15.16:0/1895245969 wait complete. 2024-09-12T02:01:07.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:07.744+0000 7fa7b98ff640 1 Processor -- start 2024-09-12T02:01:07.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:07.745+0000 7fa7b98ff640 1 -- start start 2024-09-12T02:01:07.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:07.745+0000 7fa7b98ff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa7b4102d30 0x7fa7b4198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:07.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:07.745+0000 7fa7b98ff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7b41036e0 0x7fa7b4198b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:07.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:07.745+0000 7fa7b98ff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa7b41082a0 0x7fa7b419cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:07.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:07.745+0000 7fa7b98ff640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa7b41120c0 con 0x7fa7b41036e0 2024-09-12T02:01:07.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:07.745+0000 7fa7b98ff640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa7b4111f40 con 0x7fa7b41082a0 2024-09-12T02:01:07.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:07.745+0000 7fa7b98ff640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa7b4112240 con 0x7fa7b4102d30 2024-09-12T02:01:07.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:07.745+0000 7fa7b27fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7b41036e0 0x7fa7b4198b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:07.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:07.745+0000 7fa7b27fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7b41036e0 0x7fa7b4198b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38648/0 (socket says 172.21.15.16:38648) 2024-09-12T02:01:07.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:07.745+0000 7fa7b27fc640 1 -- 172.21.15.16:0/2987813430 learned_addr learned my addr 172.21.15.16:0/2987813430 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:01:07.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:07.745+0000 7fa7b2ffd640 1 --2- 172.21.15.16:0/2987813430 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa7b4102d30 0x7fa7b4198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:07.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:07.745+0000 7fa7b37fe640 1 --2- 172.21.15.16:0/2987813430 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa7b41082a0 0x7fa7b419cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:07.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:07.745+0000 7fa7b27fc640 1 -- 172.21.15.16:0/2987813430 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa7b4102d30 msgr2=0x7fa7b4198640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:07.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:07.745+0000 7fa7b27fc640 1 --2- 172.21.15.16:0/2987813430 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa7b4102d30 0x7fa7b4198640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:07.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:07.745+0000 7fa7b27fc640 1 -- 172.21.15.16:0/2987813430 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa7b41082a0 msgr2=0x7fa7b419cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:07.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:07.745+0000 7fa7b27fc640 1 --2- 172.21.15.16:0/2987813430 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa7b41082a0 0x7fa7b419cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:07.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:07.745+0000 7fa7b27fc640 1 -- 172.21.15.16:0/2987813430 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7b419d5f0 con 0x7fa7b41036e0 2024-09-12T02:01:07.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:07.746+0000 7fa7b27fc640 1 --2- 172.21.15.16:0/2987813430 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7b41036e0 0x7fa7b4198b80 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fa79c005940 tx=0x7fa79c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:01:07.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:07.746+0000 7fa7b88fd640 1 -- 172.21.15.16:0/2987813430 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa79c03f030 con 0x7fa7b41036e0 2024-09-12T02:01:07.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:07.746+0000 7fa7b88fd640 1 -- 172.21.15.16:0/2987813430 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa79c003ce0 con 0x7fa7b41036e0 2024-09-12T02:01:07.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:07.746+0000 7fa7b98ff640 1 -- 172.21.15.16:0/2987813430 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7b419d880 con 0x7fa7b41036e0 2024-09-12T02:01:07.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:07.746+0000 7fa7b88fd640 1 -- 172.21.15.16:0/2987813430 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa79c035070 con 0x7fa7b41036e0 2024-09-12T02:01:07.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:07.746+0000 7fa7b98ff640 1 -- 172.21.15.16:0/2987813430 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7b41a51c0 con 0x7fa7b41036e0 2024-09-12T02:01:07.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:07.747+0000 7fa7b98ff640 1 -- 172.21.15.16:0/2987813430 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7b406a850 con 0x7fa7b41036e0 2024-09-12T02:01:07.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:07.748+0000 7fa7b88fd640 1 -- 172.21.15.16:0/2987813430 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fa79c042030 con 0x7fa7b41036e0 2024-09-12T02:01:07.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:07.749+0000 7fa7b88fd640 1 --2- 172.21.15.16:0/2987813430 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fa7a00a2c10 0x7fa7a00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:07.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:07.749+0000 7fa7b88fd640 1 -- 172.21.15.16:0/2987813430 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fa79c0ef740 con 0x7fa7b41036e0 2024-09-12T02:01:07.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:07.749+0000 7fa7b2ffd640 1 --2- 172.21.15.16:0/2987813430 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fa7a00a2c10 0x7fa7a00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:07.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:07.750+0000 7fa7b2ffd640 1 --2- 172.21.15.16:0/2987813430 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fa7a00a2c10 0x7fa7a00a50d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa7a400a400 tx=0x7fa7a400d040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:01:07.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:07.752+0000 7fa7b88fd640 1 -- 172.21.15.16:0/2987813430 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa79c0bbff0 con 0x7fa7b41036e0 2024-09-12T02:01:07.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:07.966+0000 7fa7b98ff640 1 -- 172.21.15.16:0/2987813430 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa7b410a2b0 con 0x7fa7b41036e0 2024-09-12T02:01:07.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:07.966+0000 7fa7b88fd640 1 -- 172.21.15.16:0/2987813430 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11911 (secure 0 0 0) 0x7fa79c0c02f0 con 0x7fa7b41036e0 2024-09-12T02:01:07.966 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T02:01:07.967 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":24,"btime":"2024-09-12T02:00:07:547152+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3279123279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3279123279},{"type":"v1","addr":"172.21.15.16:6835","nonce":3279123279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4119452382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4119452382},{"type":"v1","addr":"172.21.15.16:6837","nonce":4119452382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/3263823165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3263823165},{"type":"v1","addr":"172.21.15.16:6839","nonce":3263823165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/1510290732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":1510290732},{"type":"v1","addr":"172.21.15.16:6841","nonce":1510290732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/279674603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":279674603},{"type":"v1","addr":"172.21.15.33:6833","nonce":279674603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3555728028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3555728028},{"type":"v1","addr":"172.21.15.33:6835","nonce":3555728028}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T01:59:56.117608+0000","modified":"2024-09-12T02:00:07.097363+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24482,"mds_1":24481,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6833/1563374106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1563374106},{"type":"v1","addr":"172.21.15.184:6833","nonce":1563374106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6837/1892833868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1892833868},{"type":"v1","addr":"172.21.15.33:6837","nonce":1892833868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/1457679686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1457679686},{"type":"v1","addr":"172.21.15.184:6835","nonce":1457679686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.184:6837/1793393087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":1793393087},{"type":"v1","addr":"172.21.15.184:6837","nonce":1793393087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.184:6839/4203347941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":4203347941},{"type":"v1","addr":"172.21.15.184:6839","nonce":4203347941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.33:6839/4136670204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":4136670204},{"type":"v1","addr":"172.21.15.33:6839","nonce":4136670204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24469,24482,24481]},"id":1}]} 2024-09-12T02:01:07.968 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 24 2024-09-12T02:01:07.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:07.968+0000 7fa7b98ff640 1 -- 172.21.15.16:0/2987813430 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fa7a00a2c10 msgr2=0x7fa7a00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:07.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:07.968+0000 7fa7b98ff640 1 --2- 172.21.15.16:0/2987813430 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fa7a00a2c10 0x7fa7a00a50d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa7a400a400 tx=0x7fa7a400d040 comp rx=0 tx=0).stop 2024-09-12T02:01:07.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:07.968+0000 7fa7b98ff640 1 -- 172.21.15.16:0/2987813430 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7b41036e0 msgr2=0x7fa7b4198b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:07.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:07.968+0000 7fa7b98ff640 1 --2- 172.21.15.16:0/2987813430 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7b41036e0 0x7fa7b4198b80 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fa79c005940 tx=0x7fa79c0042f0 comp rx=0 tx=0).stop 2024-09-12T02:01:07.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:07.968+0000 7fa7b98ff640 1 -- 172.21.15.16:0/2987813430 shutdown_connections 2024-09-12T02:01:07.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:07.968+0000 7fa7b98ff640 1 --2- 172.21.15.16:0/2987813430 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fa7a00a2c10 0x7fa7a00a50d0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:07.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:07.968+0000 7fa7b98ff640 1 --2- 172.21.15.16:0/2987813430 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa7b41082a0 0x7fa7b419cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:07.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:07.968+0000 7fa7b98ff640 1 --2- 172.21.15.16:0/2987813430 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7b41036e0 0x7fa7b4198b80 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:07.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:07.969+0000 7fa7b98ff640 1 --2- 172.21.15.16:0/2987813430 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa7b4102d30 0x7fa7b4198640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:07.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:07.969+0000 7fa7b98ff640 1 -- 172.21.15.16:0/2987813430 >> 172.21.15.16:0/2987813430 conn(0x7fa7b40fc7d0 msgr2=0x7fa7b4105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:01:07.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:07.969+0000 7fa7b98ff640 1 -- 172.21.15.16:0/2987813430 shutdown_connections 2024-09-12T02:01:07.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:07.969+0000 7fa7b98ff640 1 -- 172.21.15.16:0/2987813430 wait complete. 2024-09-12T02:01:08.280 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:01:08 smithi016 ceph-mon[30947]: pgmap v311: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-12T02:01:08.281 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:01:08 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/2987813430' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:01:08.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:01:08 smithi184 ceph-mon[35593]: pgmap v311: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-12T02:01:08.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:01:08 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/2987813430' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:01:08.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:01:08 smithi033 ceph-mon[35332]: pgmap v311: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-12T02:01:08.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:01:08 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/2987813430' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:01:08.586 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-12T01:59:56.117608+0000', 'modified': '2024-09-12T02:00:07.097363+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24482, 'mds_1': 24481, 'mds_2': 24469}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24439': {'gid': 24439, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.184:6833/1563374106', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6832', 'nonce': 1563374106}, {'type': 'v1', 'addr': '172.21.15.184:6833', 'nonce': 1563374106}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24452': {'gid': 24452, 'name': 'h', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.33:6837/1892833868', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6836', 'nonce': 1892833868}, {'type': 'v1', 'addr': '172.21.15.33:6837', 'nonce': 1892833868}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24460': {'gid': 24460, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.184:6835/1457679686', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6834', 'nonce': 1457679686}, {'type': 'v1', 'addr': '172.21.15.184:6835', 'nonce': 1457679686}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24469': {'gid': 24469, 'name': 'c', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.184:6837/1793393087', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6836', 'nonce': 1793393087}, {'type': 'v1', 'addr': '172.21.15.184:6837', 'nonce': 1793393087}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24481': {'gid': 24481, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.184:6839/4203347941', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6838', 'nonce': 4203347941}, {'type': 'v1', 'addr': '172.21.15.184:6839', 'nonce': 4203347941}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24482': {'gid': 24482, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.33:6839/4136670204', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6838', 'nonce': 4136670204}, {'type': 'v1', 'addr': '172.21.15.33:6839', 'nonce': 4136670204}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24482, 'qdb_cluster': [24469, 24482, 24481]} 2024-09-12T02:01:08.586 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-09-12T02:01:08.586 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.c status 2024-09-12T02:01:08.844 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:01:09.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.309+0000 7f83323d7640 1 -- 172.21.15.16:0/2078229623 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f832c103660 msgr2=0x7f832c10bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:09.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.309+0000 7f83323d7640 1 --2- 172.21.15.16:0/2078229623 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f832c103660 0x7f832c10bb50 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f8320009900 tx=0x7f832002a480 comp rx=0 tx=0).stop 2024-09-12T02:01:09.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.309+0000 7f83323d7640 1 -- 172.21.15.16:0/2078229623 shutdown_connections 2024-09-12T02:01:09.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.309+0000 7f83323d7640 1 --2- 172.21.15.16:0/2078229623 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f832c10c170 0x7f832c10e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:09.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.309+0000 7f83323d7640 1 --2- 172.21.15.16:0/2078229623 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f832c103660 0x7f832c10bb50 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:09.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.309+0000 7f83323d7640 1 --2- 172.21.15.16:0/2078229623 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f832c102d40 0x7f832c103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:09.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.309+0000 7f83323d7640 1 -- 172.21.15.16:0/2078229623 >> 172.21.15.16:0/2078229623 conn(0x7f832c0fc7a0 msgr2=0x7f832c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:01:09.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.309+0000 7f83323d7640 1 -- 172.21.15.16:0/2078229623 shutdown_connections 2024-09-12T02:01:09.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.309+0000 7f83323d7640 1 -- 172.21.15.16:0/2078229623 wait complete. 2024-09-12T02:01:09.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.310+0000 7f83323d7640 1 Processor -- start 2024-09-12T02:01:09.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.310+0000 7f83323d7640 1 -- start start 2024-09-12T02:01:09.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.310+0000 7f83323d7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f832c102d40 0x7f832c198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:09.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.310+0000 7f83323d7640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f832c103660 0x7f832c1989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:09.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.310+0000 7f83323d7640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f832c10c170 0x7f832c19cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:09.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.311+0000 7f83323d7640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f832c111fb0 con 0x7f832c102d40 2024-09-12T02:01:09.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.311+0000 7f83323d7640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f832c111e30 con 0x7f832c103660 2024-09-12T02:01:09.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.311+0000 7f83323d7640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f832c112130 con 0x7f832c10c170 2024-09-12T02:01:09.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.311+0000 7f832bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f832c102d40 0x7f832c198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:09.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.311+0000 7f832bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f832c102d40 0x7f832c198470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38668/0 (socket says 172.21.15.16:38668) 2024-09-12T02:01:09.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.311+0000 7f832bfff640 1 -- 172.21.15.16:0/1225179996 learned_addr learned my addr 172.21.15.16:0/1225179996 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:01:09.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.311+0000 7f832b7fe640 1 --2- 172.21.15.16:0/1225179996 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f832c103660 0x7f832c1989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:09.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.311+0000 7f8330974640 1 --2- 172.21.15.16:0/1225179996 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f832c10c170 0x7f832c19cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:09.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.311+0000 7f832bfff640 1 -- 172.21.15.16:0/1225179996 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f832c10c170 msgr2=0x7f832c19cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:09.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.311+0000 7f832bfff640 1 --2- 172.21.15.16:0/1225179996 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f832c10c170 0x7f832c19cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:09.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.311+0000 7f832bfff640 1 -- 172.21.15.16:0/1225179996 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f832c103660 msgr2=0x7f832c1989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:09.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.311+0000 7f832bfff640 1 --2- 172.21.15.16:0/1225179996 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f832c103660 0x7f832c1989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:09.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.311+0000 7f832bfff640 1 -- 172.21.15.16:0/1225179996 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f832c19d420 con 0x7f832c102d40 2024-09-12T02:01:09.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.311+0000 7f832bfff640 1 --2- 172.21.15.16:0/1225179996 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f832c102d40 0x7f832c198470 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f831c00d950 tx=0x7f831c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:01:09.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.311+0000 7f83297fa640 1 -- 172.21.15.16:0/1225179996 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f831c017070 con 0x7f832c102d40 2024-09-12T02:01:09.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.311+0000 7f83297fa640 1 -- 172.21.15.16:0/1225179996 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f831c009c30 con 0x7f832c102d40 2024-09-12T02:01:09.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.311+0000 7f83323d7640 1 -- 172.21.15.16:0/1225179996 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f832c19d710 con 0x7f832c102d40 2024-09-12T02:01:09.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.311+0000 7f83297fa640 1 -- 172.21.15.16:0/1225179996 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f831c00f6b0 con 0x7f832c102d40 2024-09-12T02:01:09.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.311+0000 7f83323d7640 1 -- 172.21.15.16:0/1225179996 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f832c1a4ff0 con 0x7f832c102d40 2024-09-12T02:01:09.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.314+0000 7f83297fa640 1 -- 172.21.15.16:0/1225179996 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f831c0109d0 con 0x7f832c102d40 2024-09-12T02:01:09.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.314+0000 7f83347b0740 1 Processor -- start 2024-09-12T02:01:09.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.314+0000 7f83347b0740 1 -- start start 2024-09-12T02:01:09.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.314+0000 7f83347b0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ea5e4b19d0 0x55ea5e4b1db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:09.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.314+0000 7f83347b0740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ea5e4b22f0 0x55ea5e4bc670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:09.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.314+0000 7f83347b0740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ea5e4bcbb0 0x55ea5e4befa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:09.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.314+0000 7f83347b0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55ea5e418090 con 0x55ea5e4b19d0 2024-09-12T02:01:09.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.314+0000 7f83347b0740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55ea5e3bfa70 con 0x55ea5e4b22f0 2024-09-12T02:01:09.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.314+0000 7f83347b0740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55ea5e414050 con 0x55ea5e4bcbb0 2024-09-12T02:01:09.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.315+0000 7f8330974640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ea5e4b19d0 0x55ea5e4b1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:09.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.315+0000 7f83297fa640 1 --2- 172.21.15.16:0/1225179996 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f83140a2c10 0x7f83140a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:09.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.315+0000 7f8330974640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ea5e4b19d0 0x55ea5e4b1db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38672/0 (socket says 172.21.15.16:38672) 2024-09-12T02:01:09.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.315+0000 7f8330974640 1 -- 172.21.15.16:0/3599858811 learned_addr learned my addr 172.21.15.16:0/3599858811 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:01:09.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.315+0000 7f832bfff640 1 --2- 172.21.15.16:0/3599858811 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ea5e4b22f0 0x55ea5e4bc670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:09.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.315+0000 7f8330974640 1 --2- 172.21.15.16:0/1225179996 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f83140a2c10 0x7f83140a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:09.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.315+0000 7f832b7fe640 1 --2- 172.21.15.16:0/3599858811 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ea5e4bcbb0 0x55ea5e4befa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:09.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.315+0000 7f8330974640 1 -- 172.21.15.16:0/3599858811 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ea5e4bcbb0 msgr2=0x55ea5e4befa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:01:09.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.315+0000 7f83297fa640 1 -- 172.21.15.16:0/1225179996 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f831c0cee20 con 0x7f832c102d40 2024-09-12T02:01:09.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.315+0000 7f8330974640 1 --2- 172.21.15.16:0/3599858811 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ea5e4bcbb0 0x55ea5e4befa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:09.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.315+0000 7f8330974640 1 -- 172.21.15.16:0/3599858811 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ea5e4b22f0 msgr2=0x55ea5e4bc670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:01:09.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.315+0000 7f8330974640 1 --2- 172.21.15.16:0/3599858811 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ea5e4b22f0 0x55ea5e4bc670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:09.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.315+0000 7f8330974640 1 -- 172.21.15.16:0/3599858811 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea5e3cc130 con 0x55ea5e4b19d0 2024-09-12T02:01:09.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.315+0000 7f832b7fe640 1 --2- 172.21.15.16:0/3599858811 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ea5e4bcbb0 0x55ea5e4befa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:01:09.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.315+0000 7f8330974640 1 --2- 172.21.15.16:0/3599858811 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ea5e4b19d0 0x55ea5e4b1db0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f832400cb20 tx=0x7f83240072b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d9528be60f7a290 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:01:09.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.315+0000 7f8330974640 1 --2- 172.21.15.16:0/1225179996 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f83140a2c10 0x7f83140a50d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f832400c000 tx=0x7f832401b000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:01:09.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.315+0000 7f83127fc640 1 -- 172.21.15.16:0/3599858811 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8324015070 con 0x55ea5e4b19d0 2024-09-12T02:01:09.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.316+0000 7f83127fc640 1 -- 172.21.15.16:0/3599858811 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f832401e070 con 0x55ea5e4b19d0 2024-09-12T02:01:09.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.316+0000 7f83127fc640 1 -- 172.21.15.16:0/3599858811 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f832400a6f0 con 0x55ea5e4b19d0 2024-09-12T02:01:09.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.316+0000 7f83347b0740 1 -- 172.21.15.16:0/3599858811 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ea5e4b19d0 msgr2=0x55ea5e4b1db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:09.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.316+0000 7f83347b0740 1 --2- 172.21.15.16:0/3599858811 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ea5e4b19d0 0x55ea5e4b1db0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f832400cb20 tx=0x7f83240072b0 comp rx=0 tx=0).stop 2024-09-12T02:01:09.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.316+0000 7f83347b0740 1 -- 172.21.15.16:0/3599858811 shutdown_connections 2024-09-12T02:01:09.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.316+0000 7f83347b0740 1 --2- 172.21.15.16:0/3599858811 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ea5e4bcbb0 0x55ea5e4befa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:09.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.316+0000 7f83347b0740 1 --2- 172.21.15.16:0/3599858811 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ea5e4b22f0 0x55ea5e4bc670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:09.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.316+0000 7f83347b0740 1 --2- 172.21.15.16:0/3599858811 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ea5e4b19d0 0x55ea5e4b1db0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:09.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.316+0000 7f83347b0740 1 -- 172.21.15.16:0/3599858811 >> 172.21.15.16:0/3599858811 conn(0x55ea5e4adf60 msgr2=0x55ea5e4ae340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:01:09.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.316+0000 7f83347b0740 1 -- 172.21.15.16:0/3599858811 shutdown_connections 2024-09-12T02:01:09.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.316+0000 7f83347b0740 1 -- 172.21.15.16:0/3599858811 wait complete. 2024-09-12T02:01:09.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.317+0000 7f83347b0740 1 -- start start 2024-09-12T02:01:09.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.317+0000 7f83347b0740 1 Processor -- start 2024-09-12T02:01:09.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.317+0000 7f83347b0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ea5e4b19d0 0x55ea5e4bc4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:09.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.317+0000 7f83347b0740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ea5e4b22f0 0x55ea5e4b97f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:09.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.317+0000 7f83347b0740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ea5e4bcbb0 0x55ea5e4b9d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:09.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.317+0000 7f83347b0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55ea5e4bf780 con 0x55ea5e4b19d0 2024-09-12T02:01:09.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.317+0000 7f83347b0740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55ea5e3d2f20 con 0x55ea5e4bcbb0 2024-09-12T02:01:09.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.317+0000 7f83347b0740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55ea5e4bf900 con 0x55ea5e4b22f0 2024-09-12T02:01:09.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.317+0000 7f83347b0740 10 client.0 _ll_register_callbacks cb 0x55ea5e39c030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:01:09.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.317+0000 7f832bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ea5e4b19d0 0x55ea5e4bc4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:09.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.317+0000 7f832bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ea5e4b19d0 0x55ea5e4bc4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38688/0 (socket says 172.21.15.16:38688) 2024-09-12T02:01:09.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.317+0000 7f832bfff640 1 -- 172.21.15.16:0/3024095912 learned_addr learned my addr 172.21.15.16:0/3024095912 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:01:09.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.317+0000 7f8330974640 1 --2- 172.21.15.16:0/3024095912 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ea5e4bcbb0 0x55ea5e4b9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:09.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.318+0000 7f832b7fe640 1 --2- 172.21.15.16:0/3024095912 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ea5e4b22f0 0x55ea5e4b97f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:09.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.318+0000 7f832bfff640 1 -- 172.21.15.16:0/3024095912 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ea5e4b22f0 msgr2=0x55ea5e4b97f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:01:09.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.318+0000 7f832bfff640 1 --2- 172.21.15.16:0/3024095912 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ea5e4b22f0 0x55ea5e4b97f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:09.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.318+0000 7f832bfff640 1 -- 172.21.15.16:0/3024095912 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ea5e4bcbb0 msgr2=0x55ea5e4b9d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:01:09.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.318+0000 7f832bfff640 1 --2- 172.21.15.16:0/3024095912 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ea5e4bcbb0 0x55ea5e4b9d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:09.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.318+0000 7f832bfff640 1 -- 172.21.15.16:0/3024095912 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea5e1892b0 con 0x55ea5e4b19d0 2024-09-12T02:01:09.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.318+0000 7f8330974640 1 --2- 172.21.15.16:0/3024095912 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ea5e4bcbb0 0x55ea5e4b9d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:01:09.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.318+0000 7f832b7fe640 1 --2- 172.21.15.16:0/3024095912 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ea5e4b22f0 0x55ea5e4b97f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:01:09.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.318+0000 7f832bfff640 1 --2- 172.21.15.16:0/3024095912 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ea5e4b19d0 0x55ea5e4bc4a0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f831c097980 tx=0x7f831c06b3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ae1b91ca6fb71bd server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:01:09.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.318+0000 7f830b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:01:09.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.318+0000 7f830b7fe640 1 -- 172.21.15.16:0/3024095912 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f831c020040 con 0x55ea5e4b19d0 2024-09-12T02:01:09.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.318+0000 7f830b7fe640 1 -- 172.21.15.16:0/3024095912 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f831c06d0b0 con 0x55ea5e4b19d0 2024-09-12T02:01:09.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.318+0000 7f83347b0740 1 -- 172.21.15.16:0/3024095912 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f832401e070 con 0x55ea5e4b19d0 2024-09-12T02:01:09.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.318+0000 7f830b7fe640 1 -- 172.21.15.16:0/3024095912 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f831c009de0 con 0x55ea5e4b19d0 2024-09-12T02:01:09.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.318+0000 7f83347b0740 10 client.14967 fetch_fsmap learned FSMap version 24 2024-09-12T02:01:09.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.318+0000 7f83347b0740 1 -- 172.21.15.16:0/3024095912 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55ea5e4cb060 con 0x55ea5e4b19d0 2024-09-12T02:01:09.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.318+0000 7f830b7fe640 1 -- 172.21.15.16:0/3024095912 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f832401e070 con 0x55ea5e4b19d0 2024-09-12T02:01:09.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.319+0000 7f830b7fe640 1 -- 172.21.15.16:0/3024095912 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f831c06a0c0 con 0x55ea5e4b19d0 2024-09-12T02:01:09.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.319+0000 7f83347b0740 10 client.14967 fetch_fsmap finished waiting for FSMap version 24 2024-09-12T02:01:09.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.319+0000 7f83347b0740 10 client.14967 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-12T02:01:09.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.319+0000 7f83347b0740 20 client.14967 populate_metadata read hostname 'smithi016' 2024-09-12T02:01:09.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.319+0000 7f83347b0740 1 --2- 172.21.15.16:0/3024095912 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55ea5e4cd640 0x55ea5e4ceab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:09.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.319+0000 7f83347b0740 4 client.14967 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:01:09.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.319+0000 7f83347b0740 1 -- 172.21.15.16:0/3024095912 --> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ea5e425600 con 0x55ea5e4cd640 2024-09-12T02:01:09.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.319+0000 7f832b7fe640 1 --2- 172.21.15.16:0/3024095912 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55ea5e4cd640 0x55ea5e4ceab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:09.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.319+0000 7f832b7fe640 1 --2- 172.21.15.16:0/3024095912 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55ea5e4cd640 0x55ea5e4ceab0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cbe0154799ad1ec8 server_cookie=cfc08839936032b6 in_seq=0 out_seq=0 2024-09-12T02:01:09.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.319+0000 7f830b7fe640 10 client.14967 ms_handle_connect on v2:172.21.15.184:6836/1793393087 2024-09-12T02:01:09.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.320+0000 7f830b7fe640 1 -- 172.21.15.16:0/3024095912 <== mds.2 v2:172.21.15.184:6836/1793393087 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ea5e425600 con 0x55ea5e4cd640 2024-09-12T02:01:09.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.320+0000 7f830b7fe640 10 client.14967 handle_command_reply: tid=0 2024-09-12T02:01:09.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.320+0000 7f83347b0740 1 client.14967 shutdown 2024-09-12T02:01:09.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.321+0000 7f83347b0740 1 -- 172.21.15.16:0/3024095912 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ea5e4b19d0 msgr2=0x55ea5e4bc4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:09.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.321+0000 7f83347b0740 1 --2- 172.21.15.16:0/3024095912 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ea5e4b19d0 0x55ea5e4bc4a0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f831c097980 tx=0x7f831c06b3d0 comp rx=0 tx=0).stop 2024-09-12T02:01:09.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.321+0000 7f83347b0740 1 -- 172.21.15.16:0/3024095912 shutdown_connections 2024-09-12T02:01:09.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.321+0000 7f83347b0740 1 --2- 172.21.15.16:0/3024095912 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ea5e4bcbb0 0x55ea5e4b9d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:09.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.321+0000 7f83347b0740 1 --2- 172.21.15.16:0/3024095912 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ea5e4b22f0 0x55ea5e4b97f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:09.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.321+0000 7f83347b0740 1 --2- 172.21.15.16:0/3024095912 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55ea5e4cd640 0x55ea5e4ceab0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:09.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.321+0000 7f83347b0740 1 --2- 172.21.15.16:0/3024095912 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ea5e4b19d0 0x55ea5e4bc4a0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:09.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.321+0000 7f830b7fe640 0 client.14967 ms_handle_reset on v2:172.21.15.184:6836/1793393087 2024-09-12T02:01:09.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.321+0000 7f83347b0740 1 -- 172.21.15.16:0/3024095912 >> 172.21.15.16:0/3024095912 conn(0x55ea5e4adf60 msgr2=0x55ea5e4b3c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:01:09.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.321+0000 7f83347b0740 1 -- 172.21.15.16:0/3024095912 shutdown_connections 2024-09-12T02:01:09.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.321+0000 7f83347b0740 1 -- 172.21.15.16:0/3024095912 wait complete. 2024-09-12T02:01:09.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.321+0000 7f83347b0740 20 client.14967 trim_cache size 0 max 16384 2024-09-12T02:01:09.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.338+0000 7f83347b0740 1 Processor -- start 2024-09-12T02:01:09.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.339+0000 7f83347b0740 1 -- start start 2024-09-12T02:01:09.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.339+0000 7f83347b0740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ea5e4b19d0 0x55ea5e4cdd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:09.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.339+0000 7f83347b0740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ea5e4b22f0 0x55ea5e4b97b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:09.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.339+0000 7f83347b0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ea5e4bcbb0 0x55ea5e4b9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:09.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.339+0000 7f83347b0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55ea5e4ccdf0 con 0x55ea5e4bcbb0 2024-09-12T02:01:09.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.339+0000 7f83347b0740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55ea5e4cd4c0 con 0x55ea5e4b22f0 2024-09-12T02:01:09.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.339+0000 7f83347b0740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55ea5e4cdb90 con 0x55ea5e4b19d0 2024-09-12T02:01:09.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.339+0000 7f8330974640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ea5e4bcbb0 0x55ea5e4b9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:09.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.339+0000 7f8330974640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ea5e4bcbb0 0x55ea5e4b9cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38692/0 (socket says 172.21.15.16:38692) 2024-09-12T02:01:09.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.339+0000 7f8330974640 1 -- 172.21.15.16:0/2818502604 learned_addr learned my addr 172.21.15.16:0/2818502604 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:01:09.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.339+0000 7f832b7fe640 1 --2- 172.21.15.16:0/2818502604 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ea5e4b22f0 0x55ea5e4b97b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:09.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.339+0000 7f832bfff640 1 --2- 172.21.15.16:0/2818502604 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ea5e4b19d0 0x55ea5e4cdd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:09.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.339+0000 7f8330974640 1 -- 172.21.15.16:0/2818502604 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ea5e4b19d0 msgr2=0x55ea5e4cdd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:01:09.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.339+0000 7f8330974640 1 --2- 172.21.15.16:0/2818502604 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ea5e4b19d0 0x55ea5e4cdd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:09.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.339+0000 7f8330974640 1 -- 172.21.15.16:0/2818502604 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ea5e4b22f0 msgr2=0x55ea5e4b97b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:01:09.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.339+0000 7f8330974640 1 --2- 172.21.15.16:0/2818502604 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ea5e4b22f0 0x55ea5e4b97b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:09.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.339+0000 7f8330974640 1 -- 172.21.15.16:0/2818502604 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea5e4bf890 con 0x55ea5e4bcbb0 2024-09-12T02:01:09.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.339+0000 7f832bfff640 1 --2- 172.21.15.16:0/2818502604 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ea5e4b19d0 0x55ea5e4cdd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:01:09.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.339+0000 7f8330974640 1 --2- 172.21.15.16:0/2818502604 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ea5e4bcbb0 0x55ea5e4b9cf0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f832400cb20 tx=0x7f832400c9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3982d16e10fc9216 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:01:09.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.340+0000 7f8311ffb640 1 -- 172.21.15.16:0/2818502604 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8324031070 con 0x55ea5e4bcbb0 2024-09-12T02:01:09.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.340+0000 7f8311ffb640 1 -- 172.21.15.16:0/2818502604 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f832401e050 con 0x55ea5e4bcbb0 2024-09-12T02:01:09.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.340+0000 7f8311ffb640 1 -- 172.21.15.16:0/2818502604 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8324015040 con 0x55ea5e4bcbb0 2024-09-12T02:01:09.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.340+0000 7f83347b0740 1 -- 172.21.15.16:0/2818502604 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ea5e4bcbb0 msgr2=0x55ea5e4b9cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:09.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.340+0000 7f83347b0740 1 --2- 172.21.15.16:0/2818502604 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ea5e4bcbb0 0x55ea5e4b9cf0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f832400cb20 tx=0x7f832400c9d0 comp rx=0 tx=0).stop 2024-09-12T02:01:09.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.340+0000 7f83347b0740 1 -- 172.21.15.16:0/2818502604 shutdown_connections 2024-09-12T02:01:09.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.340+0000 7f83347b0740 1 --2- 172.21.15.16:0/2818502604 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ea5e4bcbb0 0x55ea5e4b9cf0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:09.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.340+0000 7f83347b0740 1 --2- 172.21.15.16:0/2818502604 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ea5e4b22f0 0x55ea5e4b97b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:09.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.340+0000 7f83347b0740 1 --2- 172.21.15.16:0/2818502604 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ea5e4b19d0 0x55ea5e4cdd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:09.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.340+0000 7f83347b0740 1 -- 172.21.15.16:0/2818502604 >> 172.21.15.16:0/2818502604 conn(0x55ea5e4adf60 msgr2=0x55ea5e4b8d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:01:09.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.340+0000 7f83347b0740 1 -- 172.21.15.16:0/2818502604 shutdown_connections 2024-09-12T02:01:09.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.340+0000 7f83347b0740 1 -- 172.21.15.16:0/2818502604 wait complete. 2024-09-12T02:01:09.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.341+0000 7f83347b0740 1 -- start start 2024-09-12T02:01:09.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.341+0000 7f83347b0740 1 Processor -- start 2024-09-12T02:01:09.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.341+0000 7f83347b0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ea5e4b19d0 0x55ea5e4d1d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:09.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.341+0000 7f83347b0740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ea5e4b22f0 0x55ea5e4d22b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:09.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.341+0000 7f83347b0740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ea5e4bcbb0 0x55ea5e4d27f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:09.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.341+0000 7f83347b0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55ea5e4ce930 con 0x55ea5e4b19d0 2024-09-12T02:01:09.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.341+0000 7f83347b0740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55ea5e4ce260 con 0x55ea5e4b22f0 2024-09-12T02:01:09.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.341+0000 7f83347b0740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55ea5e4cf000 con 0x55ea5e4bcbb0 2024-09-12T02:01:09.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.341+0000 7f83347b0740 10 client.0 _ll_register_callbacks cb 0x7f831c00df40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:01:09.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.341+0000 7f832bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ea5e4b19d0 0x55ea5e4d1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:09.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.341+0000 7f832bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ea5e4b19d0 0x55ea5e4d1d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38694/0 (socket says 172.21.15.16:38694) 2024-09-12T02:01:09.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.341+0000 7f832bfff640 1 -- 172.21.15.16:0/4157837968 learned_addr learned my addr 172.21.15.16:0/4157837968 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:01:09.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.341+0000 7f8330974640 1 --2- 172.21.15.16:0/4157837968 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ea5e4bcbb0 0x55ea5e4d27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:09.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.341+0000 7f832b7fe640 1 --2- 172.21.15.16:0/4157837968 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ea5e4b22f0 0x55ea5e4d22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:09.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.341+0000 7f832bfff640 1 -- 172.21.15.16:0/4157837968 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ea5e4bcbb0 msgr2=0x55ea5e4d27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:01:09.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.341+0000 7f832bfff640 1 --2- 172.21.15.16:0/4157837968 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ea5e4bcbb0 0x55ea5e4d27f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:09.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.342+0000 7f832bfff640 1 -- 172.21.15.16:0/4157837968 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ea5e4b22f0 msgr2=0x55ea5e4d22b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:01:09.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.342+0000 7f832bfff640 1 --2- 172.21.15.16:0/4157837968 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ea5e4b22f0 0x55ea5e4d22b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:09.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.342+0000 7f832bfff640 1 -- 172.21.15.16:0/4157837968 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea5e535f90 con 0x55ea5e4b19d0 2024-09-12T02:01:09.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.342+0000 7f832b7fe640 1 --2- 172.21.15.16:0/4157837968 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ea5e4b22f0 0x55ea5e4d22b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:01:09.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.342+0000 7f8330974640 1 --2- 172.21.15.16:0/4157837968 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ea5e4bcbb0 0x55ea5e4d27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:01:09.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.342+0000 7f832bfff640 1 --2- 172.21.15.16:0/4157837968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ea5e4b19d0 0x55ea5e4d1d70 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f831c09a990 tx=0x7f831c09ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=335ff8cd760fcbe6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:01:09.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.342+0000 7f830b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:01:09.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.342+0000 7f830b7fe640 1 -- 172.21.15.16:0/4157837968 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f831c020030 con 0x55ea5e4b19d0 2024-09-12T02:01:09.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.342+0000 7f830b7fe640 1 -- 172.21.15.16:0/4157837968 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f831c097480 con 0x55ea5e4b19d0 2024-09-12T02:01:09.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.342+0000 7f83347b0740 1 -- 172.21.15.16:0/4157837968 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f832401e050 con 0x55ea5e4b19d0 2024-09-12T02:01:09.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.342+0000 7f830b7fe640 1 -- 172.21.15.16:0/4157837968 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f831c09b060 con 0x55ea5e4b19d0 2024-09-12T02:01:09.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.342+0000 7f83347b0740 10 client.14973 fetch_fsmap learned FSMap version 24 2024-09-12T02:01:09.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.342+0000 7f83347b0740 1 -- 172.21.15.16:0/4157837968 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55ea5e537290 con 0x55ea5e4b19d0 2024-09-12T02:01:09.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.342+0000 7f830b7fe640 1 -- 172.21.15.16:0/4157837968 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f832401e050 con 0x55ea5e4b19d0 2024-09-12T02:01:09.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.342+0000 7f830b7fe640 1 -- 172.21.15.16:0/4157837968 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f831c0b4260 con 0x55ea5e4b19d0 2024-09-12T02:01:09.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.342+0000 7f83347b0740 10 client.14973 fetch_fsmap finished waiting for FSMap version 24 2024-09-12T02:01:09.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.342+0000 7f83347b0740 10 client.14973 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-12T02:01:09.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.342+0000 7f83347b0740 20 client.14973 populate_metadata read hostname 'smithi016' 2024-09-12T02:01:09.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.342+0000 7f83347b0740 1 --2- 172.21.15.16:0/4157837968 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55ea5e4cd640 0x55ea5e53aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:09.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.343+0000 7f83347b0740 4 client.14973 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T02:01:09.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.343+0000 7f83347b0740 1 -- 172.21.15.16:0/4157837968 --> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] -- command(tid 0: {"prefix": "status"}) -- 0x55ea5e41fb10 con 0x55ea5e4cd640 2024-09-12T02:01:09.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.343+0000 7f832b7fe640 1 --2- 172.21.15.16:0/4157837968 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55ea5e4cd640 0x55ea5e53aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:09.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.343+0000 7f830b7fe640 10 client.14973 ms_handle_connect on v2:172.21.15.184:6836/1793393087 2024-09-12T02:01:09.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.343+0000 7f832b7fe640 1 --2- 172.21.15.16:0/4157837968 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55ea5e4cd640 0x55ea5e53aba0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=23a2b8651165a40c server_cookie=a7657ef3534fc423 in_seq=0 out_seq=0 2024-09-12T02:01:09.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.344+0000 7f830b7fe640 1 -- 172.21.15.16:0/4157837968 <== mds.2 v2:172.21.15.184:6836/1793393087 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55ea5e41fb10 con 0x55ea5e4cd640 2024-09-12T02:01:09.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.344+0000 7f830b7fe640 10 client.14973 handle_command_reply: tid=0 2024-09-12T02:01:09.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.344+0000 7f83347b0740 1 client.14973 shutdown 2024-09-12T02:01:09.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.344+0000 7f83347b0740 1 -- 172.21.15.16:0/4157837968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ea5e4b19d0 msgr2=0x55ea5e4d1d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:09.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.344+0000 7f83347b0740 1 --2- 172.21.15.16:0/4157837968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ea5e4b19d0 0x55ea5e4d1d70 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f831c09a990 tx=0x7f831c09ad60 comp rx=0 tx=0).stop 2024-09-12T02:01:09.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.344+0000 7f83347b0740 1 -- 172.21.15.16:0/4157837968 shutdown_connections 2024-09-12T02:01:09.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.344+0000 7f83347b0740 1 --2- 172.21.15.16:0/4157837968 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ea5e4bcbb0 0x55ea5e4d27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:09.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.344+0000 7f83347b0740 1 --2- 172.21.15.16:0/4157837968 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ea5e4b22f0 0x55ea5e4d22b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:09.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.344+0000 7f83347b0740 1 --2- 172.21.15.16:0/4157837968 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55ea5e4cd640 0x55ea5e53aba0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:09.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.344+0000 7f83347b0740 1 --2- 172.21.15.16:0/4157837968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ea5e4b19d0 0x55ea5e4d1d70 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:09.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.344+0000 7f830b7fe640 0 client.14973 ms_handle_reset on v2:172.21.15.184:6836/1793393087 2024-09-12T02:01:09.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.344+0000 7f83347b0740 1 -- 172.21.15.16:0/4157837968 >> 172.21.15.16:0/4157837968 conn(0x55ea5e4adf60 msgr2=0x55ea5e4b97b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:01:09.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.344+0000 7f83347b0740 1 -- 172.21.15.16:0/4157837968 shutdown_connections 2024-09-12T02:01:09.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.344+0000 7f83347b0740 1 -- 172.21.15.16:0/4157837968 wait complete. 2024-09-12T02:01:09.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.344+0000 7f83347b0740 20 client.14973 trim_cache size 0 max 16384 2024-09-12T02:01:09.355 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T02:01:09.355 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "879f3b56-70a9-11ef-bcea-c7b262605968", 2024-09-12T02:01:09.355 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 2, 2024-09-12T02:01:09.355 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 24469, 2024-09-12T02:01:09.355 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-09-12T02:01:09.356 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-09-12T02:01:09.356 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-09-12T02:01:09.356 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 64.805659305000006, 2024-09-12T02:01:09.356 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 24, 2024-09-12T02:01:09.356 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 82, 2024-09-12T02:01:09.356 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 80, 2024-09-12T02:01:09.356 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 96.473462741999995 2024-09-12T02:01:09.356 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T02:01:09.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.345+0000 7f83323d7640 1 -- 172.21.15.16:0/1225179996 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f83140a2c10 msgr2=0x7f83140a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:09.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.345+0000 7f83323d7640 1 --2- 172.21.15.16:0/1225179996 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f83140a2c10 0x7f83140a50d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f832400c000 tx=0x7f832401b000 comp rx=0 tx=0).stop 2024-09-12T02:01:09.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.345+0000 7f83323d7640 1 -- 172.21.15.16:0/1225179996 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f832c102d40 msgr2=0x7f832c198470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:09.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.345+0000 7f83323d7640 1 --2- 172.21.15.16:0/1225179996 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f832c102d40 0x7f832c198470 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f831c00d950 tx=0x7f831c00de10 comp rx=0 tx=0).stop 2024-09-12T02:01:09.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.345+0000 7f83323d7640 1 -- 172.21.15.16:0/1225179996 shutdown_connections 2024-09-12T02:01:09.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.345+0000 7f83323d7640 1 --2- 172.21.15.16:0/1225179996 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f83140a2c10 0x7f83140a50d0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:09.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.345+0000 7f83323d7640 1 --2- 172.21.15.16:0/1225179996 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f832c10c170 0x7f832c19cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:09.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.345+0000 7f83323d7640 1 --2- 172.21.15.16:0/1225179996 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f832c103660 0x7f832c1989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:09.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.345+0000 7f83323d7640 1 --2- 172.21.15.16:0/1225179996 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f832c102d40 0x7f832c198470 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:09.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.345+0000 7f83323d7640 1 -- 172.21.15.16:0/1225179996 >> 172.21.15.16:0/1225179996 conn(0x7f832c0fc7a0 msgr2=0x7f832c0fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:01:09.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.345+0000 7f83323d7640 1 -- 172.21.15.16:0/1225179996 shutdown_connections 2024-09-12T02:01:09.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:09.345+0000 7f83323d7640 1 -- 172.21.15.16:0/1225179996 wait complete. 2024-09-12T02:01:09.974 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.i status 2024-09-12T02:01:10.231 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:01:10.353 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:01:10 smithi016 ceph-mon[30947]: pgmap v312: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T02:01:10.418 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:01:10 smithi033 ceph-mon[35332]: pgmap v312: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T02:01:10.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:01:10 smithi184 ceph-mon[35593]: pgmap v312: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T02:01:10.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.708+0000 7f54ad83b640 1 -- 172.21.15.16:0/3304263617 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54a8106a80 msgr2=0x7f54a8106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:10.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.708+0000 7f54ad83b640 1 --2- 172.21.15.16:0/3304263617 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54a8106a80 0x7f54a8106e60 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f5498009930 tx=0x7f549802a420 comp rx=0 tx=0).stop 2024-09-12T02:01:10.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.708+0000 7f54ad83b640 1 -- 172.21.15.16:0/3304263617 shutdown_connections 2024-09-12T02:01:10.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.708+0000 7f54ad83b640 1 --2- 172.21.15.16:0/3304263617 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f54a8101420 0x7f54a810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:10.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.708+0000 7f54ad83b640 1 --2- 172.21.15.16:0/3304263617 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f54a8100a80 0x7f54a8100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:10.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.708+0000 7f54ad83b640 1 --2- 172.21.15.16:0/3304263617 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54a8106a80 0x7f54a8106e60 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:10.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.708+0000 7f54ad83b640 1 -- 172.21.15.16:0/3304263617 >> 172.21.15.16:0/3304263617 conn(0x7f54a80fc7a0 msgr2=0x7f54a80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:01:10.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.708+0000 7f54ad83b640 1 -- 172.21.15.16:0/3304263617 shutdown_connections 2024-09-12T02:01:10.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.708+0000 7f54ad83b640 1 -- 172.21.15.16:0/3304263617 wait complete. 2024-09-12T02:01:10.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.709+0000 7f54ad83b640 1 Processor -- start 2024-09-12T02:01:10.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.709+0000 7f54ad83b640 1 -- start start 2024-09-12T02:01:10.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.709+0000 7f54ad83b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54a8100a80 0x7f54a8197310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:10.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.709+0000 7f54ad83b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f54a8101420 0x7f54a8197850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:10.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.709+0000 7f54ad83b640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f54a8106a80 0x7f54a819bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:10.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.709+0000 7f54ad83b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f54a8110e40 con 0x7f54a8100a80 2024-09-12T02:01:10.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.709+0000 7f54ad83b640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f54a8110cc0 con 0x7f54a8101420 2024-09-12T02:01:10.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.709+0000 7f54ad83b640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f54a8110fc0 con 0x7f54a8106a80 2024-09-12T02:01:10.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.709+0000 7f54a6ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54a8100a80 0x7f54a8197310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:10.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.709+0000 7f54a6ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54a8100a80 0x7f54a8197310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38722/0 (socket says 172.21.15.16:38722) 2024-09-12T02:01:10.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.710+0000 7f54a6ffd640 1 -- 172.21.15.16:0/3974832717 learned_addr learned my addr 172.21.15.16:0/3974832717 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:01:10.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.710+0000 7f549ffff640 1 --2- 172.21.15.16:0/3974832717 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f54a8101420 0x7f54a8197850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:10.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.710+0000 7f54a77fe640 1 --2- 172.21.15.16:0/3974832717 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f54a8106a80 0x7f54a819bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:10.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.710+0000 7f54a6ffd640 1 -- 172.21.15.16:0/3974832717 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f54a8106a80 msgr2=0x7f54a819bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:10.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.710+0000 7f54a6ffd640 1 --2- 172.21.15.16:0/3974832717 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f54a8106a80 0x7f54a819bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:10.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.710+0000 7f54a6ffd640 1 -- 172.21.15.16:0/3974832717 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f54a8101420 msgr2=0x7f54a8197850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:10.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.710+0000 7f54a6ffd640 1 --2- 172.21.15.16:0/3974832717 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f54a8101420 0x7f54a8197850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:10.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.710+0000 7f54a6ffd640 1 -- 172.21.15.16:0/3974832717 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54a819c2c0 con 0x7f54a8100a80 2024-09-12T02:01:10.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.710+0000 7f549ffff640 1 --2- 172.21.15.16:0/3974832717 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f54a8101420 0x7f54a8197850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:01:10.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.710+0000 7f54a77fe640 1 --2- 172.21.15.16:0/3974832717 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f54a8106a80 0x7f54a819bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:01:10.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.710+0000 7f54a6ffd640 1 --2- 172.21.15.16:0/3974832717 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54a8100a80 0x7f54a8197310 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f5498009230 tx=0x7f5498004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:01:10.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.710+0000 7f54ac839640 1 -- 172.21.15.16:0/3974832717 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f549803f030 con 0x7f54a8100a80 2024-09-12T02:01:10.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.710+0000 7f54ad83b640 1 -- 172.21.15.16:0/3974832717 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54a819c550 con 0x7f54a8100a80 2024-09-12T02:01:10.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.710+0000 7f54ac839640 1 -- 172.21.15.16:0/3974832717 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5498003ad0 con 0x7f54a8100a80 2024-09-12T02:01:10.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.710+0000 7f54ac839640 1 -- 172.21.15.16:0/3974832717 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5498035070 con 0x7f54a8100a80 2024-09-12T02:01:10.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.710+0000 7f54ad83b640 1 -- 172.21.15.16:0/3974832717 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54a81a3e90 con 0x7f54a8100a80 2024-09-12T02:01:10.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.713+0000 7f54ac839640 1 -- 172.21.15.16:0/3974832717 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f5498042030 con 0x7f54a8100a80 2024-09-12T02:01:10.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.713+0000 7f54afc14740 1 Processor -- start 2024-09-12T02:01:10.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.713+0000 7f54afc14740 1 -- start start 2024-09-12T02:01:10.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.713+0000 7f54afc14740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5574861149d0 0x557486114db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:10.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.713+0000 7f54afc14740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5574861152f0 0x55748611f670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:10.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.713+0000 7f54afc14740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55748611fbb0 0x557486121fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:10.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.714+0000 7f54afc14740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55748607b090 con 0x55748611fbb0 2024-09-12T02:01:10.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.714+0000 7f54afc14740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x557486022a70 con 0x5574861149d0 2024-09-12T02:01:10.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.714+0000 7f54afc14740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x557486077050 con 0x5574861152f0 2024-09-12T02:01:10.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.714+0000 7f549ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55748611fbb0 0x557486121fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:10.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.714+0000 7f549ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55748611fbb0 0x557486121fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38724/0 (socket says 172.21.15.16:38724) 2024-09-12T02:01:10.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.714+0000 7f549ffff640 1 -- 172.21.15.16:0/2187551989 learned_addr learned my addr 172.21.15.16:0/2187551989 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:01:10.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.714+0000 7f54a77fe640 1 --2- 172.21.15.16:0/2187551989 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5574861149d0 0x557486114db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:10.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.714+0000 7f54ac839640 1 --2- 172.21.15.16:0/3974832717 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f54940a2c10 0x7f54940a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:10.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.714+0000 7f54a6ffd640 1 --2- 172.21.15.16:0/2187551989 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5574861152f0 0x55748611f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:10.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.714+0000 7f549ffff640 1 -- 172.21.15.16:0/2187551989 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5574861152f0 msgr2=0x55748611f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:01:10.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.714+0000 7f549ffff640 1 --2- 172.21.15.16:0/2187551989 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5574861152f0 0x55748611f670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:10.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.714+0000 7f549ffff640 1 -- 172.21.15.16:0/2187551989 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5574861149d0 msgr2=0x557486114db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:01:10.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.714+0000 7f549ffff640 1 --2- 172.21.15.16:0/2187551989 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5574861149d0 0x557486114db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:10.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.714+0000 7f549ffff640 1 -- 172.21.15.16:0/2187551989 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55748602f130 con 0x55748611fbb0 2024-09-12T02:01:10.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.714+0000 7f54a77fe640 1 --2- 172.21.15.16:0/2187551989 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5574861149d0 0x557486114db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:01:10.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.714+0000 7f54a77fe640 1 --2- 172.21.15.16:0/3974832717 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f54940a2c10 0x7f54940a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:10.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.714+0000 7f54ac839640 1 -- 172.21.15.16:0/3974832717 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f54980ef780 con 0x7f54a8100a80 2024-09-12T02:01:10.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.714+0000 7f549ffff640 1 --2- 172.21.15.16:0/2187551989 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55748611fbb0 0x557486121fa0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f5490008c10 tx=0x7f549000e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=143522e808392571 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:01:10.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.714+0000 7f549dffb640 1 -- 172.21.15.16:0/2187551989 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5490014070 con 0x55748611fbb0 2024-09-12T02:01:10.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.714+0000 7f549dffb640 1 -- 172.21.15.16:0/2187551989 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54900047a0 con 0x55748611fbb0 2024-09-12T02:01:10.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.714+0000 7f549dffb640 1 -- 172.21.15.16:0/2187551989 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5490010760 con 0x55748611fbb0 2024-09-12T02:01:10.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.715+0000 7f54afc14740 1 -- 172.21.15.16:0/2187551989 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55748611fbb0 msgr2=0x557486121fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:10.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.715+0000 7f54afc14740 1 --2- 172.21.15.16:0/2187551989 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55748611fbb0 0x557486121fa0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f5490008c10 tx=0x7f549000e720 comp rx=0 tx=0).stop 2024-09-12T02:01:10.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.715+0000 7f54a77fe640 1 --2- 172.21.15.16:0/3974832717 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f54940a2c10 0x7f54940a50d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f54a0005fd0 tx=0x7f54a0004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:01:10.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.715+0000 7f54afc14740 1 -- 172.21.15.16:0/2187551989 shutdown_connections 2024-09-12T02:01:10.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.715+0000 7f54afc14740 1 --2- 172.21.15.16:0/2187551989 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55748611fbb0 0x557486121fa0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:10.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.715+0000 7f54afc14740 1 --2- 172.21.15.16:0/2187551989 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5574861152f0 0x55748611f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:10.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.715+0000 7f54afc14740 1 --2- 172.21.15.16:0/2187551989 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5574861149d0 0x557486114db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:10.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.715+0000 7f54afc14740 1 -- 172.21.15.16:0/2187551989 >> 172.21.15.16:0/2187551989 conn(0x557486110f60 msgr2=0x557486111340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:01:10.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.715+0000 7f54afc14740 1 -- 172.21.15.16:0/2187551989 shutdown_connections 2024-09-12T02:01:10.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.715+0000 7f54afc14740 1 -- 172.21.15.16:0/2187551989 wait complete. 2024-09-12T02:01:10.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.715+0000 7f54afc14740 1 -- start start 2024-09-12T02:01:10.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.716+0000 7f54afc14740 1 Processor -- start 2024-09-12T02:01:10.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.716+0000 7f54afc14740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5574861149d0 0x55748611f410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:10.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.716+0000 7f54afc14740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5574861152f0 0x55748611c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:10.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.716+0000 7f54afc14740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55748611fbb0 0x55748611cca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:10.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.716+0000 7f54afc14740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x557486122780 con 0x55748611fbb0 2024-09-12T02:01:10.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.716+0000 7f54afc14740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x557486035f20 con 0x5574861149d0 2024-09-12T02:01:10.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.716+0000 7f54afc14740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x557486122900 con 0x5574861152f0 2024-09-12T02:01:10.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.716+0000 7f54afc14740 10 client.0 _ll_register_callbacks cb 0x557485fff030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:01:10.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.716+0000 7f54a77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55748611fbb0 0x55748611cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:10.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.716+0000 7f54a77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55748611fbb0 0x55748611cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38734/0 (socket says 172.21.15.16:38734) 2024-09-12T02:01:10.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.716+0000 7f54a77fe640 1 -- 172.21.15.16:0/2576797450 learned_addr learned my addr 172.21.15.16:0/2576797450 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:01:10.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.716+0000 7f549ffff640 1 --2- 172.21.15.16:0/2576797450 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5574861152f0 0x55748611c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:10.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.716+0000 7f54a6ffd640 1 --2- 172.21.15.16:0/2576797450 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5574861149d0 0x55748611f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:10.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.716+0000 7f54a77fe640 1 -- 172.21.15.16:0/2576797450 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5574861152f0 msgr2=0x55748611c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:01:10.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.716+0000 7f54a77fe640 1 --2- 172.21.15.16:0/2576797450 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5574861152f0 0x55748611c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:10.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.716+0000 7f54a77fe640 1 -- 172.21.15.16:0/2576797450 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5574861149d0 msgr2=0x55748611f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:01:10.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.716+0000 7f54a77fe640 1 --2- 172.21.15.16:0/2576797450 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5574861149d0 0x55748611f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:10.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.716+0000 7f54a77fe640 1 -- 172.21.15.16:0/2576797450 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557485dec2b0 con 0x55748611fbb0 2024-09-12T02:01:10.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.717+0000 7f54a6ffd640 1 --2- 172.21.15.16:0/2576797450 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5574861149d0 0x55748611f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:01:10.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.717+0000 7f54a77fe640 1 --2- 172.21.15.16:0/2576797450 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55748611fbb0 0x55748611cca0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f54a0016010 tx=0x7f54a0004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b44af83cfab5bec2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:01:10.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.717+0000 7f5486ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:01:10.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.717+0000 7f5486ffd640 1 -- 172.21.15.16:0/2576797450 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54a002c060 con 0x55748611fbb0 2024-09-12T02:01:10.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.717+0000 7f5486ffd640 1 -- 172.21.15.16:0/2576797450 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f54a0019070 con 0x55748611fbb0 2024-09-12T02:01:10.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.717+0000 7f54afc14740 1 -- 172.21.15.16:0/2576797450 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54900047a0 con 0x55748611fbb0 2024-09-12T02:01:10.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.717+0000 7f5486ffd640 1 -- 172.21.15.16:0/2576797450 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f54a001e940 con 0x55748611fbb0 2024-09-12T02:01:10.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.717+0000 7f54afc14740 10 client.14985 fetch_fsmap learned FSMap version 24 2024-09-12T02:01:10.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.717+0000 7f54afc14740 1 -- 172.21.15.16:0/2576797450 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55748612e0d0 con 0x55748611fbb0 2024-09-12T02:01:10.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.718+0000 7f5486ffd640 1 -- 172.21.15.16:0/2576797450 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54900047a0 con 0x55748611fbb0 2024-09-12T02:01:10.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.718+0000 7f5486ffd640 1 -- 172.21.15.16:0/2576797450 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f54a0031070 con 0x55748611fbb0 2024-09-12T02:01:10.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.718+0000 7f54afc14740 10 client.14985 fetch_fsmap finished waiting for FSMap version 24 2024-09-12T02:01:10.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.718+0000 7f54afc14740 10 client.14985 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-12T02:01:10.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.718+0000 7f54afc14740 20 client.14985 populate_metadata read hostname 'smithi016' 2024-09-12T02:01:10.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.718+0000 7f54afc14740 1 --2- 172.21.15.16:0/2576797450 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x557486130710 0x557486131b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:10.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.718+0000 7f54afc14740 4 client.14985 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:01:10.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.718+0000 7f54afc14740 1 -- 172.21.15.16:0/2576797450 --> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557486088600 con 0x557486130710 2024-09-12T02:01:10.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.719+0000 7f54a6ffd640 1 --2- 172.21.15.16:0/2576797450 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x557486130710 0x557486131b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:10.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.719+0000 7f54a6ffd640 1 --2- 172.21.15.16:0/2576797450 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x557486130710 0x557486131b80 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e8da544b8842c14f server_cookie=3e73efae612a574d in_seq=0 out_seq=0 2024-09-12T02:01:10.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.719+0000 7f5486ffd640 10 client.14985 ms_handle_connect on v2:172.21.15.184:6838/4203347941 2024-09-12T02:01:10.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.720+0000 7f5486ffd640 1 -- 172.21.15.16:0/2576797450 <== mds.1 v2:172.21.15.184:6838/4203347941 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557486088600 con 0x557486130710 2024-09-12T02:01:10.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.720+0000 7f5486ffd640 10 client.14985 handle_command_reply: tid=0 2024-09-12T02:01:10.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.720+0000 7f54afc14740 1 client.14985 shutdown 2024-09-12T02:01:10.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.720+0000 7f54afc14740 1 -- 172.21.15.16:0/2576797450 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55748611fbb0 msgr2=0x55748611cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:10.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.720+0000 7f54afc14740 1 --2- 172.21.15.16:0/2576797450 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55748611fbb0 0x55748611cca0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f54a0016010 tx=0x7f54a0004ab0 comp rx=0 tx=0).stop 2024-09-12T02:01:10.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.720+0000 7f54afc14740 1 -- 172.21.15.16:0/2576797450 shutdown_connections 2024-09-12T02:01:10.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.720+0000 7f54afc14740 1 --2- 172.21.15.16:0/2576797450 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x557486130710 0x557486131b80 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:10.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.720+0000 7f54afc14740 1 --2- 172.21.15.16:0/2576797450 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55748611fbb0 0x55748611cca0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:10.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.720+0000 7f54afc14740 1 --2- 172.21.15.16:0/2576797450 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5574861152f0 0x55748611c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:10.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.720+0000 7f54afc14740 1 --2- 172.21.15.16:0/2576797450 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5574861149d0 0x55748611f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:10.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.720+0000 7f5486ffd640 0 client.14985 ms_handle_reset on v2:172.21.15.184:6838/4203347941 2024-09-12T02:01:10.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.720+0000 7f54afc14740 1 -- 172.21.15.16:0/2576797450 >> 172.21.15.16:0/2576797450 conn(0x557486110f60 msgr2=0x557486116c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:01:10.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.720+0000 7f54afc14740 1 -- 172.21.15.16:0/2576797450 shutdown_connections 2024-09-12T02:01:10.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.720+0000 7f54afc14740 1 -- 172.21.15.16:0/2576797450 wait complete. 2024-09-12T02:01:10.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.720+0000 7f54afc14740 20 client.14985 trim_cache size 0 max 16384 2024-09-12T02:01:10.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.738+0000 7f54afc14740 1 Processor -- start 2024-09-12T02:01:10.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.738+0000 7f54afc14740 1 -- start start 2024-09-12T02:01:10.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.738+0000 7f54afc14740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5574861149d0 0x557486131b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:10.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.738+0000 7f54afc14740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5574861152f0 0x55748611c720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:10.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.738+0000 7f54afc14740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55748611fbb0 0x55748611cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:10.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.738+0000 7f54afc14740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55748612fec0 con 0x5574861149d0 2024-09-12T02:01:10.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.738+0000 7f54afc14740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x557486130590 con 0x55748611fbb0 2024-09-12T02:01:10.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.738+0000 7f54afc14740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x557486130c60 con 0x5574861152f0 2024-09-12T02:01:10.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.739+0000 7f54a6ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5574861149d0 0x557486131b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:10.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.739+0000 7f54a6ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5574861149d0 0x557486131b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38746/0 (socket says 172.21.15.16:38746) 2024-09-12T02:01:10.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.739+0000 7f54a6ffd640 1 -- 172.21.15.16:0/4090866360 learned_addr learned my addr 172.21.15.16:0/4090866360 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:01:10.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.739+0000 7f54a77fe640 1 --2- 172.21.15.16:0/4090866360 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55748611fbb0 0x55748611cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:10.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.739+0000 7f54a6ffd640 1 -- 172.21.15.16:0/4090866360 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5574861152f0 msgr2=0x55748611c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:01:10.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.739+0000 7f54a6ffd640 1 --2- 172.21.15.16:0/4090866360 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5574861152f0 0x55748611c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:10.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.739+0000 7f54a6ffd640 1 -- 172.21.15.16:0/4090866360 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55748611fbb0 msgr2=0x55748611cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:01:10.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.739+0000 7f54a6ffd640 1 --2- 172.21.15.16:0/4090866360 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55748611fbb0 0x55748611cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:10.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.739+0000 7f54a6ffd640 1 -- 172.21.15.16:0/4090866360 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557486110810 con 0x5574861149d0 2024-09-12T02:01:10.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.739+0000 7f54a77fe640 1 --2- 172.21.15.16:0/4090866360 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55748611fbb0 0x55748611cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:01:10.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.739+0000 7f54a6ffd640 1 --2- 172.21.15.16:0/4090866360 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5574861149d0 0x557486131b80 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f54980bbb30 tx=0x7f54980045f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f61edac66ee29f79 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:01:10.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.739+0000 7f549d7fa640 1 -- 172.21.15.16:0/4090866360 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5498049040 con 0x5574861149d0 2024-09-12T02:01:10.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.739+0000 7f549d7fa640 1 -- 172.21.15.16:0/4090866360 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54980b9f20 con 0x5574861149d0 2024-09-12T02:01:10.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.739+0000 7f549d7fa640 1 -- 172.21.15.16:0/4090866360 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54980d0e60 con 0x5574861149d0 2024-09-12T02:01:10.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.740+0000 7f54afc14740 1 -- 172.21.15.16:0/4090866360 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5574861149d0 msgr2=0x557486131b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:10.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.740+0000 7f54afc14740 1 --2- 172.21.15.16:0/4090866360 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5574861149d0 0x557486131b80 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f54980bbb30 tx=0x7f54980045f0 comp rx=0 tx=0).stop 2024-09-12T02:01:10.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.740+0000 7f54afc14740 1 -- 172.21.15.16:0/4090866360 shutdown_connections 2024-09-12T02:01:10.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.740+0000 7f54afc14740 1 --2- 172.21.15.16:0/4090866360 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55748611fbb0 0x55748611cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:10.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.740+0000 7f54afc14740 1 --2- 172.21.15.16:0/4090866360 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5574861152f0 0x55748611c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:10.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.740+0000 7f54afc14740 1 --2- 172.21.15.16:0/4090866360 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5574861149d0 0x557486131b80 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:10.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.740+0000 7f54afc14740 1 -- 172.21.15.16:0/4090866360 >> 172.21.15.16:0/4090866360 conn(0x557486110f60 msgr2=0x55748611bca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:01:10.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.740+0000 7f54afc14740 1 -- 172.21.15.16:0/4090866360 shutdown_connections 2024-09-12T02:01:10.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.740+0000 7f54afc14740 1 -- 172.21.15.16:0/4090866360 wait complete. 2024-09-12T02:01:10.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.740+0000 7f54afc14740 1 -- start start 2024-09-12T02:01:10.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.741+0000 7f54afc14740 1 Processor -- start 2024-09-12T02:01:10.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.741+0000 7f54afc14740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5574861149d0 0x557486134ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:10.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.741+0000 7f54afc14740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5574861152f0 0x557486135220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:10.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.741+0000 7f54afc14740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55748611fbb0 0x557486135760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:10.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.741+0000 7f54afc14740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x557486131a00 con 0x5574861152f0 2024-09-12T02:01:10.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.741+0000 7f54afc14740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x557486131330 con 0x5574861149d0 2024-09-12T02:01:10.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.741+0000 7f54afc14740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5574861320d0 con 0x55748611fbb0 2024-09-12T02:01:10.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.741+0000 7f54afc14740 10 client.0 _ll_register_callbacks cb 0x7f54a0009c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:01:10.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.741+0000 7f549ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5574861152f0 0x557486135220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:10.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.741+0000 7f549ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5574861152f0 0x557486135220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38754/0 (socket says 172.21.15.16:38754) 2024-09-12T02:01:10.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.741+0000 7f549ffff640 1 -- 172.21.15.16:0/1439980037 learned_addr learned my addr 172.21.15.16:0/1439980037 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:01:10.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.741+0000 7f54a6ffd640 1 --2- 172.21.15.16:0/1439980037 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5574861149d0 0x557486134ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:10.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.741+0000 7f54a77fe640 1 --2- 172.21.15.16:0/1439980037 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55748611fbb0 0x557486135760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:10.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.741+0000 7f549ffff640 1 -- 172.21.15.16:0/1439980037 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55748611fbb0 msgr2=0x557486135760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:01:10.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.741+0000 7f549ffff640 1 --2- 172.21.15.16:0/1439980037 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55748611fbb0 0x557486135760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:10.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.741+0000 7f549ffff640 1 -- 172.21.15.16:0/1439980037 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5574861149d0 msgr2=0x557486134ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:01:10.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.741+0000 7f549ffff640 1 --2- 172.21.15.16:0/1439980037 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5574861149d0 0x557486134ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:10.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.741+0000 7f549ffff640 1 -- 172.21.15.16:0/1439980037 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55748619d270 con 0x5574861152f0 2024-09-12T02:01:10.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.741+0000 7f54a77fe640 1 --2- 172.21.15.16:0/1439980037 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55748611fbb0 0x557486135760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:01:10.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.741+0000 7f549ffff640 1 --2- 172.21.15.16:0/1439980037 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5574861152f0 0x557486135220 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x5574861122e0 tx=0x7f5490005c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d481ae9b49d975eb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:01:10.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.741+0000 7f5486ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:01:10.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.742+0000 7f5486ffd640 1 -- 172.21.15.16:0/1439980037 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5490014040 con 0x5574861152f0 2024-09-12T02:01:10.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.742+0000 7f5486ffd640 1 -- 172.21.15.16:0/1439980037 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f5490003800 con 0x5574861152f0 2024-09-12T02:01:10.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.742+0000 7f54afc14740 1 -- 172.21.15.16:0/1439980037 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54980b9f20 con 0x5574861152f0 2024-09-12T02:01:10.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.742+0000 7f5486ffd640 1 -- 172.21.15.16:0/1439980037 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f5490003a00 con 0x5574861152f0 2024-09-12T02:01:10.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.742+0000 7f54afc14740 10 client.14991 fetch_fsmap learned FSMap version 24 2024-09-12T02:01:10.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.742+0000 7f54afc14740 1 -- 172.21.15.16:0/1439980037 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55748619e520 con 0x5574861152f0 2024-09-12T02:01:10.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.742+0000 7f5486ffd640 1 -- 172.21.15.16:0/1439980037 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54980b9f20 con 0x5574861152f0 2024-09-12T02:01:10.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.742+0000 7f5486ffd640 1 -- 172.21.15.16:0/1439980037 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f5490010c70 con 0x5574861152f0 2024-09-12T02:01:10.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.742+0000 7f54afc14740 10 client.14991 fetch_fsmap finished waiting for FSMap version 24 2024-09-12T02:01:10.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.742+0000 7f54afc14740 10 client.14991 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-12T02:01:10.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.742+0000 7f54afc14740 20 client.14991 populate_metadata read hostname 'smithi016' 2024-09-12T02:01:10.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.742+0000 7f54afc14740 1 --2- 172.21.15.16:0/1439980037 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x5574861a2f50 0x5574861a1c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:10.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.742+0000 7f54afc14740 4 client.14991 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T02:01:10.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.742+0000 7f54afc14740 1 -- 172.21.15.16:0/1439980037 --> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] -- command(tid 0: {"prefix": "status"}) -- 0x557486082b10 con 0x5574861a2f50 2024-09-12T02:01:10.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.743+0000 7f54a6ffd640 1 --2- 172.21.15.16:0/1439980037 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x5574861a2f50 0x5574861a1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:10.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.743+0000 7f5486ffd640 10 client.14991 ms_handle_connect on v2:172.21.15.184:6838/4203347941 2024-09-12T02:01:10.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.743+0000 7f54a6ffd640 1 --2- 172.21.15.16:0/1439980037 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x5574861a2f50 0x5574861a1c60 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=afdf63cf26a8886d server_cookie=eaf527965e233826 in_seq=0 out_seq=0 2024-09-12T02:01:10.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.743+0000 7f5486ffd640 1 -- 172.21.15.16:0/1439980037 <== mds.1 v2:172.21.15.184:6838/4203347941 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x557486082b10 con 0x5574861a2f50 2024-09-12T02:01:10.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.743+0000 7f5486ffd640 10 client.14991 handle_command_reply: tid=0 2024-09-12T02:01:10.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.743+0000 7f54afc14740 1 client.14991 shutdown 2024-09-12T02:01:10.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.743+0000 7f54afc14740 1 -- 172.21.15.16:0/1439980037 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5574861152f0 msgr2=0x557486135220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:10.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.743+0000 7f54afc14740 1 --2- 172.21.15.16:0/1439980037 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5574861152f0 0x557486135220 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x5574861122e0 tx=0x7f5490005c20 comp rx=0 tx=0).stop 2024-09-12T02:01:10.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.744+0000 7f54afc14740 1 -- 172.21.15.16:0/1439980037 shutdown_connections 2024-09-12T02:01:10.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.744+0000 7f54afc14740 1 --2- 172.21.15.16:0/1439980037 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x5574861a2f50 0x5574861a1c60 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:10.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.744+0000 7f54afc14740 1 --2- 172.21.15.16:0/1439980037 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55748611fbb0 0x557486135760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:10.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.744+0000 7f54afc14740 1 --2- 172.21.15.16:0/1439980037 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5574861152f0 0x557486135220 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:10.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.744+0000 7f5486ffd640 0 client.14991 ms_handle_reset on v2:172.21.15.184:6838/4203347941 2024-09-12T02:01:10.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.744+0000 7f54afc14740 1 --2- 172.21.15.16:0/1439980037 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5574861149d0 0x557486134ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:10.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.744+0000 7f54afc14740 1 -- 172.21.15.16:0/1439980037 >> 172.21.15.16:0/1439980037 conn(0x557486110f60 msgr2=0x55748611c720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:01:10.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.744+0000 7f54afc14740 1 -- 172.21.15.16:0/1439980037 shutdown_connections 2024-09-12T02:01:10.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.744+0000 7f54afc14740 1 -- 172.21.15.16:0/1439980037 wait complete. 2024-09-12T02:01:10.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.744+0000 7f54afc14740 20 client.14991 trim_cache size 0 max 16384 2024-09-12T02:01:10.750 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T02:01:10.750 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "879f3b56-70a9-11ef-bcea-c7b262605968", 2024-09-12T02:01:10.750 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 1, 2024-09-12T02:01:10.750 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 24481, 2024-09-12T02:01:10.750 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-09-12T02:01:10.751 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-09-12T02:01:10.751 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-09-12T02:01:10.751 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 67.213546586999996, 2024-09-12T02:01:10.751 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 24, 2024-09-12T02:01:10.751 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 82, 2024-09-12T02:01:10.751 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 80, 2024-09-12T02:01:10.751 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 92.953884457000001 2024-09-12T02:01:10.751 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T02:01:10.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.744+0000 7f54ad83b640 1 -- 172.21.15.16:0/3974832717 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f54940a2c10 msgr2=0x7f54940a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:10.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.744+0000 7f54ad83b640 1 --2- 172.21.15.16:0/3974832717 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f54940a2c10 0x7f54940a50d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f54a0005fd0 tx=0x7f54a0004a40 comp rx=0 tx=0).stop 2024-09-12T02:01:10.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.744+0000 7f54ad83b640 1 -- 172.21.15.16:0/3974832717 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54a8100a80 msgr2=0x7f54a8197310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:10.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.744+0000 7f54ad83b640 1 --2- 172.21.15.16:0/3974832717 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54a8100a80 0x7f54a8197310 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f5498009230 tx=0x7f5498004270 comp rx=0 tx=0).stop 2024-09-12T02:01:10.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.744+0000 7f54ad83b640 1 -- 172.21.15.16:0/3974832717 shutdown_connections 2024-09-12T02:01:10.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.744+0000 7f54ad83b640 1 --2- 172.21.15.16:0/3974832717 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f54940a2c10 0x7f54940a50d0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:10.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.744+0000 7f54ad83b640 1 --2- 172.21.15.16:0/3974832717 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f54a8106a80 0x7f54a819bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:10.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.744+0000 7f54ad83b640 1 --2- 172.21.15.16:0/3974832717 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f54a8101420 0x7f54a8197850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:10.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.744+0000 7f54ad83b640 1 --2- 172.21.15.16:0/3974832717 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54a8100a80 0x7f54a8197310 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:10.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.744+0000 7f54ad83b640 1 -- 172.21.15.16:0/3974832717 >> 172.21.15.16:0/3974832717 conn(0x7f54a80fc7a0 msgr2=0x7f54a80fd0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:01:10.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.745+0000 7f54ad83b640 1 -- 172.21.15.16:0/3974832717 shutdown_connections 2024-09-12T02:01:10.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:10.745+0000 7f54ad83b640 1 -- 172.21.15.16:0/3974832717 wait complete. 2024-09-12T02:01:11.367 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.b status 2024-09-12T02:01:11.623 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:01:12.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.061+0000 7ff72762c640 1 -- 172.21.15.16:0/2215741254 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7201008d0 msgr2=0x7ff720100cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:12.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.061+0000 7ff72762c640 1 --2- 172.21.15.16:0/2215741254 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7201008d0 0x7ff720100cb0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7ff71c009900 tx=0x7ff71c02a410 comp rx=0 tx=0).stop 2024-09-12T02:01:12.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.061+0000 7ff72762c640 1 -- 172.21.15.16:0/2215741254 shutdown_connections 2024-09-12T02:01:12.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.061+0000 7ff72762c640 1 --2- 172.21.15.16:0/2215741254 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff72010e480 0x7ff720110870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:12.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.061+0000 7ff72762c640 1 --2- 172.21.15.16:0/2215741254 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff720101280 0x7ff72010df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:12.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.061+0000 7ff72762c640 1 --2- 172.21.15.16:0/2215741254 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7201008d0 0x7ff720100cb0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:12.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.061+0000 7ff72762c640 1 -- 172.21.15.16:0/2215741254 >> 172.21.15.16:0/2215741254 conn(0x7ff7200fc7a0 msgr2=0x7ff7200febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:01:12.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.061+0000 7ff72762c640 1 -- 172.21.15.16:0/2215741254 shutdown_connections 2024-09-12T02:01:12.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.061+0000 7ff72762c640 1 -- 172.21.15.16:0/2215741254 wait complete. 2024-09-12T02:01:12.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.062+0000 7ff72762c640 1 Processor -- start 2024-09-12T02:01:12.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.062+0000 7ff72762c640 1 -- start start 2024-09-12T02:01:12.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.062+0000 7ff72762c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7201008d0 0x7ff7201a05d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:12.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.062+0000 7ff72762c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff720101280 0x7ff7201a0b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:12.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.062+0000 7ff72762c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff72010e480 0x7ff72019a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:12.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.062+0000 7ff72762c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff7201142b0 con 0x7ff7201008d0 2024-09-12T02:01:12.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.062+0000 7ff72762c640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff720114130 con 0x7ff72010e480 2024-09-12T02:01:12.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.062+0000 7ff72762c640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff720114430 con 0x7ff720101280 2024-09-12T02:01:12.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.062+0000 7ff7253c8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7201008d0 0x7ff7201a05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:12.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.062+0000 7ff7253c8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7201008d0 0x7ff7201a05d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35140/0 (socket says 172.21.15.16:35140) 2024-09-12T02:01:12.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.062+0000 7ff7253c8640 1 -- 172.21.15.16:0/3330795348 learned_addr learned my addr 172.21.15.16:0/3330795348 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:01:12.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.063+0000 7ff725bc9640 1 --2- 172.21.15.16:0/3330795348 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff72010e480 0x7ff72019a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:12.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.063+0000 7ff724bc7640 1 --2- 172.21.15.16:0/3330795348 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff720101280 0x7ff7201a0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:12.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.063+0000 7ff7253c8640 1 -- 172.21.15.16:0/3330795348 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff720101280 msgr2=0x7ff7201a0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:12.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.063+0000 7ff7253c8640 1 --2- 172.21.15.16:0/3330795348 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff720101280 0x7ff7201a0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:12.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.063+0000 7ff7253c8640 1 -- 172.21.15.16:0/3330795348 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff72010e480 msgr2=0x7ff72019a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:12.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.063+0000 7ff7253c8640 1 --2- 172.21.15.16:0/3330795348 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff72010e480 0x7ff72019a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:12.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.063+0000 7ff7253c8640 1 -- 172.21.15.16:0/3330795348 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff72019af10 con 0x7ff7201008d0 2024-09-12T02:01:12.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.063+0000 7ff724bc7640 1 --2- 172.21.15.16:0/3330795348 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff720101280 0x7ff7201a0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:01:12.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.063+0000 7ff725bc9640 1 --2- 172.21.15.16:0/3330795348 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff72010e480 0x7ff72019a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:01:12.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.063+0000 7ff7253c8640 1 --2- 172.21.15.16:0/3330795348 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7201008d0 0x7ff7201a05d0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7ff71c02aa00 tx=0x7ff71c003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:01:12.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.063+0000 7ff7167fc640 1 -- 172.21.15.16:0/3330795348 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff71c03f070 con 0x7ff7201008d0 2024-09-12T02:01:12.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.063+0000 7ff72762c640 1 -- 172.21.15.16:0/3330795348 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff72019b1a0 con 0x7ff7201008d0 2024-09-12T02:01:12.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.063+0000 7ff7167fc640 1 -- 172.21.15.16:0/3330795348 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff71c03b460 con 0x7ff7201008d0 2024-09-12T02:01:12.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.063+0000 7ff7167fc640 1 -- 172.21.15.16:0/3330795348 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff71c035030 con 0x7ff7201008d0 2024-09-12T02:01:12.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.063+0000 7ff72762c640 1 -- 172.21.15.16:0/3330795348 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7201a7380 con 0x7ff7201008d0 2024-09-12T02:01:12.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.065+0000 7ff7167fc640 1 -- 172.21.15.16:0/3330795348 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7ff71c042070 con 0x7ff7201008d0 2024-09-12T02:01:12.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.066+0000 7ff729a05740 1 Processor -- start 2024-09-12T02:01:12.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.066+0000 7ff729a05740 1 -- start start 2024-09-12T02:01:12.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.066+0000 7ff729a05740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559bc66c59d0 0x559bc66c5db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:12.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.066+0000 7ff729a05740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559bc66c62f0 0x559bc66d0670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:12.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.066+0000 7ff729a05740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559bc66d0bb0 0x559bc66d2fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:12.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.066+0000 7ff729a05740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x559bc662c090 con 0x559bc66c62f0 2024-09-12T02:01:12.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.066+0000 7ff729a05740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x559bc65d3a70 con 0x559bc66d0bb0 2024-09-12T02:01:12.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.066+0000 7ff729a05740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x559bc6628050 con 0x559bc66c59d0 2024-09-12T02:01:12.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.066+0000 7ff7167fc640 1 --2- 172.21.15.16:0/3330795348 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7ff70c0a2c10 0x7ff70c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:12.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.066+0000 7ff7253c8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559bc66c62f0 0x559bc66d0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:12.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.066+0000 7ff7253c8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559bc66c62f0 0x559bc66d0670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35146/0 (socket says 172.21.15.16:35146) 2024-09-12T02:01:12.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.066+0000 7ff7253c8640 1 -- 172.21.15.16:0/1018207477 learned_addr learned my addr 172.21.15.16:0/1018207477 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:01:12.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.066+0000 7ff725bc9640 1 --2- 172.21.15.16:0/1018207477 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559bc66c59d0 0x559bc66c5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:12.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.067+0000 7ff724bc7640 1 --2- 172.21.15.16:0/1018207477 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559bc66d0bb0 0x559bc66d2fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:12.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.067+0000 7ff725bc9640 1 --2- 172.21.15.16:0/3330795348 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7ff70c0a2c10 0x7ff70c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:12.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.067+0000 7ff7167fc640 1 -- 172.21.15.16:0/3330795348 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7ff71c0efcd0 con 0x7ff7201008d0 2024-09-12T02:01:12.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.067+0000 7ff7253c8640 1 -- 172.21.15.16:0/1018207477 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559bc66c59d0 msgr2=0x559bc66c5db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:01:12.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.067+0000 7ff7253c8640 1 --2- 172.21.15.16:0/1018207477 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559bc66c59d0 0x559bc66c5db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:12.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.067+0000 7ff7253c8640 1 -- 172.21.15.16:0/1018207477 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559bc66d0bb0 msgr2=0x559bc66d2fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:01:12.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.067+0000 7ff7253c8640 1 --2- 172.21.15.16:0/1018207477 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559bc66d0bb0 0x559bc66d2fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:12.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.067+0000 7ff7253c8640 1 -- 172.21.15.16:0/1018207477 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559bc65e0130 con 0x559bc66c62f0 2024-09-12T02:01:12.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.067+0000 7ff724bc7640 1 --2- 172.21.15.16:0/1018207477 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559bc66d0bb0 0x559bc66d2fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:01:12.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.067+0000 7ff7253c8640 1 --2- 172.21.15.16:0/1018207477 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559bc66c62f0 0x559bc66d0670 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7ff71c0ed620 tx=0x7ff71c047000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9953aa40a250e29c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:01:12.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.067+0000 7ff725bc9640 1 --2- 172.21.15.16:0/3330795348 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7ff70c0a2c10 0x7ff70c0a50d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ff718009440 tx=0x7ff7180093d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:01:12.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.067+0000 7ff70b7fe640 1 -- 172.21.15.16:0/1018207477 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff71c049070 con 0x559bc66c62f0 2024-09-12T02:01:12.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.067+0000 7ff70b7fe640 1 -- 172.21.15.16:0/1018207477 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff71c0bcad0 con 0x559bc66c62f0 2024-09-12T02:01:12.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.067+0000 7ff70b7fe640 1 -- 172.21.15.16:0/1018207477 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff71c0b8a40 con 0x559bc66c62f0 2024-09-12T02:01:12.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.068+0000 7ff729a05740 1 -- 172.21.15.16:0/1018207477 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559bc66c62f0 msgr2=0x559bc66d0670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:12.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.068+0000 7ff729a05740 1 --2- 172.21.15.16:0/1018207477 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559bc66c62f0 0x559bc66d0670 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7ff71c0ed620 tx=0x7ff71c047000 comp rx=0 tx=0).stop 2024-09-12T02:01:12.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.068+0000 7ff729a05740 1 -- 172.21.15.16:0/1018207477 shutdown_connections 2024-09-12T02:01:12.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.068+0000 7ff729a05740 1 --2- 172.21.15.16:0/1018207477 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559bc66d0bb0 0x559bc66d2fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:12.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.068+0000 7ff729a05740 1 --2- 172.21.15.16:0/1018207477 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559bc66c62f0 0x559bc66d0670 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:12.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.068+0000 7ff729a05740 1 --2- 172.21.15.16:0/1018207477 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559bc66c59d0 0x559bc66c5db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:12.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.068+0000 7ff729a05740 1 -- 172.21.15.16:0/1018207477 >> 172.21.15.16:0/1018207477 conn(0x559bc66c1f60 msgr2=0x559bc66c2340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:01:12.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.068+0000 7ff729a05740 1 -- 172.21.15.16:0/1018207477 shutdown_connections 2024-09-12T02:01:12.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.068+0000 7ff729a05740 1 -- 172.21.15.16:0/1018207477 wait complete. 2024-09-12T02:01:12.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.068+0000 7ff729a05740 1 -- start start 2024-09-12T02:01:12.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.069+0000 7ff729a05740 1 Processor -- start 2024-09-12T02:01:12.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.069+0000 7ff729a05740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559bc66c59d0 0x559bc66d0480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:12.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.069+0000 7ff729a05740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559bc66c62f0 0x559bc66cd7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:12.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.069+0000 7ff729a05740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559bc66d0bb0 0x559bc66cdd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:12.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.069+0000 7ff729a05740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x559bc66d3540 con 0x559bc66c59d0 2024-09-12T02:01:12.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.069+0000 7ff729a05740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x559bc65e6f20 con 0x559bc66c62f0 2024-09-12T02:01:12.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.069+0000 7ff729a05740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x559bc66d36c0 con 0x559bc66d0bb0 2024-09-12T02:01:12.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.069+0000 7ff729a05740 10 client.0 _ll_register_callbacks cb 0x559bc65b0030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:01:12.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.069+0000 7ff7253c8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559bc66c59d0 0x559bc66d0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:12.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.069+0000 7ff7253c8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559bc66c59d0 0x559bc66d0480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35158/0 (socket says 172.21.15.16:35158) 2024-09-12T02:01:12.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.069+0000 7ff7253c8640 1 -- 172.21.15.16:0/4170306462 learned_addr learned my addr 172.21.15.16:0/4170306462 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:01:12.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.069+0000 7ff725bc9640 1 --2- 172.21.15.16:0/4170306462 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559bc66d0bb0 0x559bc66cdd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:12.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.069+0000 7ff724bc7640 1 --2- 172.21.15.16:0/4170306462 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559bc66c62f0 0x559bc66cd7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:12.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.069+0000 7ff7253c8640 1 -- 172.21.15.16:0/4170306462 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559bc66d0bb0 msgr2=0x559bc66cdd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:01:12.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.069+0000 7ff7253c8640 1 --2- 172.21.15.16:0/4170306462 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559bc66d0bb0 0x559bc66cdd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:12.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.069+0000 7ff7253c8640 1 -- 172.21.15.16:0/4170306462 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559bc66c62f0 msgr2=0x559bc66cd7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:01:12.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.069+0000 7ff7253c8640 1 --2- 172.21.15.16:0/4170306462 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559bc66c62f0 0x559bc66cd7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:12.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.069+0000 7ff7253c8640 1 -- 172.21.15.16:0/4170306462 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559bc639d2b0 con 0x559bc66c59d0 2024-09-12T02:01:12.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.069+0000 7ff724bc7640 1 --2- 172.21.15.16:0/4170306462 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559bc66c62f0 0x559bc66cd7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:01:12.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.070+0000 7ff7253c8640 1 --2- 172.21.15.16:0/4170306462 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559bc66c59d0 0x559bc66d0480 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7ff71c03bd00 tx=0x7ff71c0b8670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2785893f41d5950c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:01:12.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.070+0000 7ff708ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:01:12.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.070+0000 7ff708ff9640 1 -- 172.21.15.16:0/4170306462 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff71c0d4020 con 0x559bc66c59d0 2024-09-12T02:01:12.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.070+0000 7ff708ff9640 1 -- 172.21.15.16:0/4170306462 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7ff71c0b8a40 con 0x559bc66c59d0 2024-09-12T02:01:12.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.070+0000 7ff729a05740 1 -- 172.21.15.16:0/4170306462 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff71c0bcad0 con 0x559bc66c59d0 2024-09-12T02:01:12.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.070+0000 7ff708ff9640 1 -- 172.21.15.16:0/4170306462 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7ff71c049070 con 0x559bc66c59d0 2024-09-12T02:01:12.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.070+0000 7ff729a05740 10 client.15003 fetch_fsmap learned FSMap version 24 2024-09-12T02:01:12.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.070+0000 7ff729a05740 1 -- 172.21.15.16:0/4170306462 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x559bc66d3540 con 0x559bc66c59d0 2024-09-12T02:01:12.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.070+0000 7ff708ff9640 1 -- 172.21.15.16:0/4170306462 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff71c0bcad0 con 0x559bc66c59d0 2024-09-12T02:01:12.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.070+0000 7ff708ff9640 1 -- 172.21.15.16:0/4170306462 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7ff71c0c4bb0 con 0x559bc66c59d0 2024-09-12T02:01:12.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.070+0000 7ff729a05740 10 client.15003 fetch_fsmap finished waiting for FSMap version 24 2024-09-12T02:01:12.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.070+0000 7ff729a05740 10 client.15003 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-12T02:01:12.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.070+0000 7ff729a05740 20 client.15003 populate_metadata read hostname 'smithi016' 2024-09-12T02:01:12.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.070+0000 7ff729a05740 1 --2- 172.21.15.16:0/4170306462 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x559bc66e1230 0x559bc66e26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:12.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.070+0000 7ff729a05740 4 client.15003 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:01:12.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.070+0000 7ff729a05740 1 -- 172.21.15.16:0/4170306462 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559bc6639600 con 0x559bc66e1230 2024-09-12T02:01:12.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.071+0000 7ff724bc7640 1 --2- 172.21.15.16:0/4170306462 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x559bc66e1230 0x559bc66e26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:12.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.071+0000 7ff708ff9640 10 client.15003 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:01:12.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.071+0000 7ff724bc7640 1 --2- 172.21.15.16:0/4170306462 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x559bc66e1230 0x559bc66e26a0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f21ba5b021e39f4 server_cookie=39fdfc753b9bd940 in_seq=0 out_seq=0 2024-09-12T02:01:12.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.072+0000 7ff708ff9640 1 -- 172.21.15.16:0/4170306462 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559bc6639600 con 0x559bc66e1230 2024-09-12T02:01:12.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.072+0000 7ff708ff9640 10 client.15003 handle_command_reply: tid=0 2024-09-12T02:01:12.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.072+0000 7ff729a05740 1 client.15003 shutdown 2024-09-12T02:01:12.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.072+0000 7ff729a05740 1 -- 172.21.15.16:0/4170306462 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559bc66c59d0 msgr2=0x559bc66d0480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:12.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.072+0000 7ff729a05740 1 --2- 172.21.15.16:0/4170306462 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559bc66c59d0 0x559bc66d0480 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7ff71c03bd00 tx=0x7ff71c0b8670 comp rx=0 tx=0).stop 2024-09-12T02:01:12.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.072+0000 7ff729a05740 1 -- 172.21.15.16:0/4170306462 shutdown_connections 2024-09-12T02:01:12.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.072+0000 7ff729a05740 1 --2- 172.21.15.16:0/4170306462 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x559bc66e1230 0x559bc66e26a0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:12.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.072+0000 7ff708ff9640 0 client.15003 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:01:12.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.072+0000 7ff729a05740 1 --2- 172.21.15.16:0/4170306462 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559bc66d0bb0 0x559bc66cdd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:12.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.072+0000 7ff729a05740 1 --2- 172.21.15.16:0/4170306462 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559bc66c62f0 0x559bc66cd7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:12.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.072+0000 7ff729a05740 1 --2- 172.21.15.16:0/4170306462 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559bc66c59d0 0x559bc66d0480 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:12.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.072+0000 7ff729a05740 1 -- 172.21.15.16:0/4170306462 >> 172.21.15.16:0/4170306462 conn(0x559bc66c1f60 msgr2=0x559bc66c7c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:01:12.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.072+0000 7ff729a05740 1 -- 172.21.15.16:0/4170306462 shutdown_connections 2024-09-12T02:01:12.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.073+0000 7ff729a05740 1 -- 172.21.15.16:0/4170306462 wait complete. 2024-09-12T02:01:12.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.073+0000 7ff729a05740 20 client.15003 trim_cache size 0 max 16384 2024-09-12T02:01:12.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.090+0000 7ff729a05740 1 Processor -- start 2024-09-12T02:01:12.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.090+0000 7ff729a05740 1 -- start start 2024-09-12T02:01:12.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.091+0000 7ff729a05740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559bc66c59d0 0x559bc66dff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:12.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.091+0000 7ff729a05740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559bc66c62f0 0x559bc66ea980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:12.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.091+0000 7ff729a05740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559bc66d0bb0 0x559bc66eaec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:12.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.091+0000 7ff729a05740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x559bc66e09e0 con 0x559bc66c59d0 2024-09-12T02:01:12.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.091+0000 7ff729a05740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x559bc66e10b0 con 0x559bc66d0bb0 2024-09-12T02:01:12.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.091+0000 7ff729a05740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x559bc66e1780 con 0x559bc66c62f0 2024-09-12T02:01:12.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.091+0000 7ff7253c8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559bc66c59d0 0x559bc66dff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:12.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.091+0000 7ff7253c8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559bc66c59d0 0x559bc66dff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35174/0 (socket says 172.21.15.16:35174) 2024-09-12T02:01:12.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.091+0000 7ff7253c8640 1 -- 172.21.15.16:0/1250483677 learned_addr learned my addr 172.21.15.16:0/1250483677 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:01:12.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.091+0000 7ff724bc7640 1 --2- 172.21.15.16:0/1250483677 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559bc66c62f0 0x559bc66ea980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:12.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.091+0000 7ff7253c8640 1 -- 172.21.15.16:0/1250483677 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559bc66c62f0 msgr2=0x559bc66ea980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:01:12.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.091+0000 7ff7253c8640 1 --2- 172.21.15.16:0/1250483677 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559bc66c62f0 0x559bc66ea980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:12.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.091+0000 7ff7253c8640 1 -- 172.21.15.16:0/1250483677 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559bc66d0bb0 msgr2=0x559bc66eaec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:01:12.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.091+0000 7ff725bc9640 1 --2- 172.21.15.16:0/1250483677 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559bc66d0bb0 0x559bc66eaec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:12.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.091+0000 7ff7253c8640 1 --2- 172.21.15.16:0/1250483677 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559bc66d0bb0 0x559bc66eaec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:12.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.091+0000 7ff7253c8640 1 -- 172.21.15.16:0/1250483677 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559bc66c5800 con 0x559bc66c59d0 2024-09-12T02:01:12.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.091+0000 7ff7253c8640 1 --2- 172.21.15.16:0/1250483677 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559bc66c59d0 0x559bc66dff40 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7ff71c0b8830 tx=0x7ff71c0ba8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8965ddb057e57d5a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:01:12.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.091+0000 7ff70affd640 1 -- 172.21.15.16:0/1250483677 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff71c0d4070 con 0x559bc66c59d0 2024-09-12T02:01:12.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.091+0000 7ff70affd640 1 -- 172.21.15.16:0/1250483677 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff71c0bcad0 con 0x559bc66c59d0 2024-09-12T02:01:12.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.092+0000 7ff70affd640 1 -- 172.21.15.16:0/1250483677 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff71c0bba00 con 0x559bc66c59d0 2024-09-12T02:01:12.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.092+0000 7ff729a05740 1 -- 172.21.15.16:0/1250483677 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559bc66c59d0 msgr2=0x559bc66dff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:12.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.092+0000 7ff729a05740 1 --2- 172.21.15.16:0/1250483677 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559bc66c59d0 0x559bc66dff40 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7ff71c0b8830 tx=0x7ff71c0ba8e0 comp rx=0 tx=0).stop 2024-09-12T02:01:12.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.092+0000 7ff729a05740 1 -- 172.21.15.16:0/1250483677 shutdown_connections 2024-09-12T02:01:12.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.092+0000 7ff729a05740 1 --2- 172.21.15.16:0/1250483677 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559bc66d0bb0 0x559bc66eaec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:12.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.092+0000 7ff729a05740 1 --2- 172.21.15.16:0/1250483677 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559bc66c62f0 0x559bc66ea980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:12.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.092+0000 7ff729a05740 1 --2- 172.21.15.16:0/1250483677 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559bc66c59d0 0x559bc66dff40 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:12.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.092+0000 7ff729a05740 1 -- 172.21.15.16:0/1250483677 >> 172.21.15.16:0/1250483677 conn(0x559bc66c1f60 msgr2=0x559bc66cc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:01:12.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.092+0000 7ff729a05740 1 -- 172.21.15.16:0/1250483677 shutdown_connections 2024-09-12T02:01:12.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.092+0000 7ff729a05740 1 -- 172.21.15.16:0/1250483677 wait complete. 2024-09-12T02:01:12.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.093+0000 7ff729a05740 1 -- start start 2024-09-12T02:01:12.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.093+0000 7ff729a05740 1 Processor -- start 2024-09-12T02:01:12.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.093+0000 7ff729a05740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559bc66c59d0 0x559bc66e5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:12.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.093+0000 7ff729a05740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559bc66c62f0 0x559bc66e62e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:12.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.093+0000 7ff729a05740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559bc66d0bb0 0x559bc6749930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:12.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.093+0000 7ff729a05740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x559bc66e2520 con 0x559bc66c62f0 2024-09-12T02:01:12.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.093+0000 7ff729a05740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x559bc66e1e50 con 0x559bc66d0bb0 2024-09-12T02:01:12.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.093+0000 7ff729a05740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x559bc66e2bf0 con 0x559bc66c59d0 2024-09-12T02:01:12.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.093+0000 7ff729a05740 10 client.0 _ll_register_callbacks cb 0x7ff7100071a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:01:12.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.093+0000 7ff724bc7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559bc66c62f0 0x559bc66e62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:12.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.093+0000 7ff724bc7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559bc66c62f0 0x559bc66e62e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35182/0 (socket says 172.21.15.16:35182) 2024-09-12T02:01:12.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.093+0000 7ff724bc7640 1 -- 172.21.15.16:0/269687285 learned_addr learned my addr 172.21.15.16:0/269687285 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:01:12.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.093+0000 7ff7253c8640 1 --2- 172.21.15.16:0/269687285 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559bc66c59d0 0x559bc66e5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:12.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.093+0000 7ff725bc9640 1 --2- 172.21.15.16:0/269687285 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559bc66d0bb0 0x559bc6749930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:12.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.093+0000 7ff724bc7640 1 -- 172.21.15.16:0/269687285 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559bc66c59d0 msgr2=0x559bc66e5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:01:12.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.094+0000 7ff724bc7640 1 --2- 172.21.15.16:0/269687285 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559bc66c59d0 0x559bc66e5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:12.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.094+0000 7ff724bc7640 1 -- 172.21.15.16:0/269687285 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559bc66d0bb0 msgr2=0x559bc6749930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:01:12.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.094+0000 7ff724bc7640 1 --2- 172.21.15.16:0/269687285 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559bc66d0bb0 0x559bc6749930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:12.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.094+0000 7ff724bc7640 1 -- 172.21.15.16:0/269687285 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559bc6749e70 con 0x559bc66c62f0 2024-09-12T02:01:12.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.094+0000 7ff7253c8640 1 --2- 172.21.15.16:0/269687285 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559bc66c59d0 0x559bc66e5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:01:12.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.094+0000 7ff724bc7640 1 --2- 172.21.15.16:0/269687285 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559bc66c62f0 0x559bc66e62e0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7ff710004810 tx=0x7ff71001a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff1467a9acf5cced server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:01:12.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.094+0000 7ff708ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:01:12.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.094+0000 7ff708ff9640 1 -- 172.21.15.16:0/269687285 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff710018820 con 0x559bc66c62f0 2024-09-12T02:01:12.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.094+0000 7ff708ff9640 1 -- 172.21.15.16:0/269687285 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7ff7100189c0 con 0x559bc66c62f0 2024-09-12T02:01:12.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.094+0000 7ff729a05740 1 -- 172.21.15.16:0/269687285 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff71c0bcad0 con 0x559bc66c62f0 2024-09-12T02:01:12.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.094+0000 7ff708ff9640 1 -- 172.21.15.16:0/269687285 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7ff710018bc0 con 0x559bc66c62f0 2024-09-12T02:01:12.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.094+0000 7ff729a05740 10 client.15009 fetch_fsmap learned FSMap version 24 2024-09-12T02:01:12.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.094+0000 7ff729a05740 1 -- 172.21.15.16:0/269687285 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x559bc674b110 con 0x559bc66c62f0 2024-09-12T02:01:12.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.094+0000 7ff708ff9640 1 -- 172.21.15.16:0/269687285 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff71c0bcad0 con 0x559bc66c62f0 2024-09-12T02:01:12.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.094+0000 7ff708ff9640 1 -- 172.21.15.16:0/269687285 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7ff710016040 con 0x559bc66c62f0 2024-09-12T02:01:12.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.094+0000 7ff729a05740 10 client.15009 fetch_fsmap finished waiting for FSMap version 24 2024-09-12T02:01:12.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.095+0000 7ff729a05740 10 client.15009 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-12T02:01:12.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.095+0000 7ff729a05740 20 client.15009 populate_metadata read hostname 'smithi016' 2024-09-12T02:01:12.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.095+0000 7ff729a05740 1 --2- 172.21.15.16:0/269687285 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x559bc66e1230 0x559bc674eae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:12.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.095+0000 7ff729a05740 4 client.15009 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T02:01:12.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.095+0000 7ff729a05740 1 -- 172.21.15.16:0/269687285 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "status"}) -- 0x559bc6633b10 con 0x559bc66e1230 2024-09-12T02:01:12.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.095+0000 7ff7253c8640 1 --2- 172.21.15.16:0/269687285 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x559bc66e1230 0x559bc674eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:12.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.095+0000 7ff7253c8640 1 --2- 172.21.15.16:0/269687285 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x559bc66e1230 0x559bc674eae0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b6f969b08533799 server_cookie=a271d33533fde023 in_seq=0 out_seq=0 2024-09-12T02:01:12.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.095+0000 7ff708ff9640 10 client.15009 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:01:12.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.095+0000 7ff708ff9640 1 -- 172.21.15.16:0/269687285 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x559bc6633b10 con 0x559bc66e1230 2024-09-12T02:01:12.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.095+0000 7ff708ff9640 10 client.15009 handle_command_reply: tid=0 2024-09-12T02:01:12.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.095+0000 7ff729a05740 1 client.15009 shutdown 2024-09-12T02:01:12.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.096+0000 7ff729a05740 1 -- 172.21.15.16:0/269687285 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559bc66c62f0 msgr2=0x559bc66e62e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:12.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.096+0000 7ff729a05740 1 --2- 172.21.15.16:0/269687285 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559bc66c62f0 0x559bc66e62e0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7ff710004810 tx=0x7ff71001a7d0 comp rx=0 tx=0).stop 2024-09-12T02:01:12.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.096+0000 7ff729a05740 1 -- 172.21.15.16:0/269687285 shutdown_connections 2024-09-12T02:01:12.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.096+0000 7ff729a05740 1 --2- 172.21.15.16:0/269687285 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x559bc66e1230 0x559bc674eae0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:12.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.096+0000 7ff729a05740 1 --2- 172.21.15.16:0/269687285 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559bc66d0bb0 0x559bc6749930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:12.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.096+0000 7ff729a05740 1 --2- 172.21.15.16:0/269687285 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559bc66c62f0 0x559bc66e62e0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:12.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.096+0000 7ff708ff9640 0 client.15009 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:01:12.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.096+0000 7ff729a05740 1 --2- 172.21.15.16:0/269687285 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559bc66c59d0 0x559bc66e5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:12.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.096+0000 7ff729a05740 1 -- 172.21.15.16:0/269687285 >> 172.21.15.16:0/269687285 conn(0x559bc66c1f60 msgr2=0x559bc66c07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:01:12.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.096+0000 7ff729a05740 1 -- 172.21.15.16:0/269687285 shutdown_connections 2024-09-12T02:01:12.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.096+0000 7ff729a05740 1 -- 172.21.15.16:0/269687285 wait complete. 2024-09-12T02:01:12.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.096+0000 7ff729a05740 20 client.15009 trim_cache size 0 max 16384 2024-09-12T02:01:12.102 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T02:01:12.102 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "879f3b56-70a9-11ef-bcea-c7b262605968", 2024-09-12T02:01:12.102 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 0, 2024-09-12T02:01:12.102 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 24482, 2024-09-12T02:01:12.102 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-09-12T02:01:12.102 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-09-12T02:01:12.102 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-09-12T02:01:12.102 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 75.603408196000004, 2024-09-12T02:01:12.102 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 24, 2024-09-12T02:01:12.103 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 82, 2024-09-12T02:01:12.103 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 80, 2024-09-12T02:01:12.103 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 85.854143719999996 2024-09-12T02:01:12.103 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T02:01:12.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.096+0000 7ff72762c640 1 -- 172.21.15.16:0/3330795348 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7ff70c0a2c10 msgr2=0x7ff70c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:12.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.096+0000 7ff72762c640 1 --2- 172.21.15.16:0/3330795348 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7ff70c0a2c10 0x7ff70c0a50d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ff718009440 tx=0x7ff7180093d0 comp rx=0 tx=0).stop 2024-09-12T02:01:12.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.096+0000 7ff72762c640 1 -- 172.21.15.16:0/3330795348 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7201008d0 msgr2=0x7ff7201a05d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:12.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.096+0000 7ff72762c640 1 --2- 172.21.15.16:0/3330795348 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7201008d0 0x7ff7201a05d0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7ff71c02aa00 tx=0x7ff71c003d50 comp rx=0 tx=0).stop 2024-09-12T02:01:12.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.096+0000 7ff72762c640 1 -- 172.21.15.16:0/3330795348 shutdown_connections 2024-09-12T02:01:12.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.096+0000 7ff72762c640 1 --2- 172.21.15.16:0/3330795348 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7ff70c0a2c10 0x7ff70c0a50d0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:12.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.096+0000 7ff72762c640 1 --2- 172.21.15.16:0/3330795348 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff72010e480 0x7ff72019a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:12.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.096+0000 7ff72762c640 1 --2- 172.21.15.16:0/3330795348 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff720101280 0x7ff7201a0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:12.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.096+0000 7ff72762c640 1 --2- 172.21.15.16:0/3330795348 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7201008d0 0x7ff7201a05d0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:12.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.096+0000 7ff72762c640 1 -- 172.21.15.16:0/3330795348 >> 172.21.15.16:0/3330795348 conn(0x7ff7200fc7a0 msgr2=0x7ff7200feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:01:12.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.097+0000 7ff72762c640 1 -- 172.21.15.16:0/3330795348 shutdown_connections 2024-09-12T02:01:12.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:12.097+0000 7ff72762c640 1 -- 172.21.15.16:0/3330795348 wait complete. 2024-09-12T02:01:12.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:01:12 smithi033 ceph-mon[35332]: pgmap v313: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-12T02:01:12.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:01:12 smithi016 ceph-mon[30947]: pgmap v313: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-12T02:01:12.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:01:12 smithi184 ceph-mon[35593]: pgmap v313: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-12T02:01:12.810 INFO:tasks.cephadm:Setup complete, yielding 2024-09-12T02:01:12.810 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-12T02:01:12.821 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi016.front.sepia.ceph.com 2024-09-12T02:01:12.821 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- bash -c 'ceph orch status' 2024-09-12T02:01:13.080 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:01:13.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:13.576+0000 7feb09096640 1 -- 172.21.15.16:0/2660015938 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb04102e00 msgr2=0x7feb04103280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:13.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:13.576+0000 7feb09096640 1 --2- 172.21.15.16:0/2660015938 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb04102e00 0x7feb04103280 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7feaf4009900 tx=0x7feaf402a410 comp rx=0 tx=0).stop 2024-09-12T02:01:13.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:13.576+0000 7feb09096640 1 -- 172.21.15.16:0/2660015938 shutdown_connections 2024-09-12T02:01:13.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:13.576+0000 7feb09096640 1 --2- 172.21.15.16:0/2660015938 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feb041037c0 0x7feb0410a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:13.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:13.576+0000 7feb09096640 1 --2- 172.21.15.16:0/2660015938 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb04102e00 0x7feb04103280 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:13.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:13.576+0000 7feb09096640 1 --2- 172.21.15.16:0/2660015938 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb04101c00 0x7feb04102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:13.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:13.576+0000 7feb09096640 1 -- 172.21.15.16:0/2660015938 >> 172.21.15.16:0/2660015938 conn(0x7feb040fd390 msgr2=0x7feb040ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:01:13.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:13.576+0000 7feb09096640 1 -- 172.21.15.16:0/2660015938 shutdown_connections 2024-09-12T02:01:13.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:13.576+0000 7feb09096640 1 -- 172.21.15.16:0/2660015938 wait complete. 2024-09-12T02:01:13.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:13.577+0000 7feb09096640 1 Processor -- start 2024-09-12T02:01:13.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:13.577+0000 7feb09096640 1 -- start start 2024-09-12T02:01:13.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:13.578+0000 7feb09096640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feb04101c00 0x7feb04075f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:13.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:13.578+0000 7feb09096640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb04102e00 0x7feb040745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:13.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:13.578+0000 7feb09096640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb041037c0 0x7feb04074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:13.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:13.578+0000 7feb09096640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7feb0410d8d0 con 0x7feb041037c0 2024-09-12T02:01:13.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:13.578+0000 7feb09096640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7feb0410d750 con 0x7feb04102e00 2024-09-12T02:01:13.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:13.578+0000 7feb09096640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7feb0410da50 con 0x7feb04101c00 2024-09-12T02:01:13.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:13.578+0000 7feb0359e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb041037c0 0x7feb04074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:13.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:13.578+0000 7feb0359e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb041037c0 0x7feb04074b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35194/0 (socket says 172.21.15.16:35194) 2024-09-12T02:01:13.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:13.578+0000 7feb02d9d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feb04101c00 0x7feb04075f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:13.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:13.578+0000 7feb0359e640 1 -- 172.21.15.16:0/2931750873 learned_addr learned my addr 172.21.15.16:0/2931750873 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:01:13.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:13.578+0000 7feb0259c640 1 --2- 172.21.15.16:0/2931750873 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb04102e00 0x7feb040745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:13.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:13.578+0000 7feb02d9d640 1 -- 172.21.15.16:0/2931750873 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb04102e00 msgr2=0x7feb040745e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:13.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:13.578+0000 7feb02d9d640 1 --2- 172.21.15.16:0/2931750873 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb04102e00 0x7feb040745e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:13.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:13.578+0000 7feb02d9d640 1 -- 172.21.15.16:0/2931750873 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb041037c0 msgr2=0x7feb04074b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:13.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:13.578+0000 7feb02d9d640 1 --2- 172.21.15.16:0/2931750873 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb041037c0 0x7feb04074b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:13.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:13.578+0000 7feb02d9d640 1 -- 172.21.15.16:0/2931750873 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb040753e0 con 0x7feb04101c00 2024-09-12T02:01:13.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:13.579+0000 7feb02d9d640 1 --2- 172.21.15.16:0/2931750873 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feb04101c00 0x7feb04075f90 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7feaec00d6a0 tx=0x7feaec00db60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:01:13.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:13.579+0000 7feafbfff640 1 -- 172.21.15.16:0/2931750873 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaec017070 con 0x7feb04101c00 2024-09-12T02:01:13.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:13.579+0000 7feb09096640 1 -- 172.21.15.16:0/2931750873 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb041a1ff0 con 0x7feb04101c00 2024-09-12T02:01:13.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:13.579+0000 7feafbfff640 1 -- 172.21.15.16:0/2931750873 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feaec009d00 con 0x7feb04101c00 2024-09-12T02:01:13.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:13.579+0000 7feafbfff640 1 -- 172.21.15.16:0/2931750873 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaec003c60 con 0x7feb04101c00 2024-09-12T02:01:13.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:13.579+0000 7feb09096640 1 -- 172.21.15.16:0/2931750873 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb041a2450 con 0x7feb04101c00 2024-09-12T02:01:13.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:13.580+0000 7feb09096640 1 -- 172.21.15.16:0/2931750873 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb04108fb0 con 0x7feb04101c00 2024-09-12T02:01:13.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:13.581+0000 7feafbfff640 1 -- 172.21.15.16:0/2931750873 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7feaec009820 con 0x7feb04101c00 2024-09-12T02:01:13.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:13.582+0000 7feafbfff640 1 --2- 172.21.15.16:0/2931750873 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7feaf00a2c10 0x7feaf00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:13.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:13.582+0000 7feb0259c640 1 --2- 172.21.15.16:0/2931750873 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7feaf00a2c10 0x7feaf00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:13.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:13.582+0000 7feafbfff640 1 -- 172.21.15.16:0/2931750873 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7feaec0cedb0 con 0x7feb04101c00 2024-09-12T02:01:13.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:13.583+0000 7feb0259c640 1 --2- 172.21.15.16:0/2931750873 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7feaf00a2c10 0x7feaf00a50d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7feaf4031440 tx=0x7feaf40315b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:01:13.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:13.585+0000 7feafbfff640 1 -- 172.21.15.16:0/2931750873 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7feaec09b660 con 0x7feb04101c00 2024-09-12T02:01:13.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:13.775+0000 7feb09096640 1 -- 172.21.15.16:0/2931750873 --> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7feb04062630 con 0x7feaf00a2c10 2024-09-12T02:01:13.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:13.776+0000 7feafbfff640 1 -- 172.21.15.16:0/2931750873 <== mgr.14150 v2:172.21.15.16:6800/3887444820 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7feb04062630 con 0x7feaf00a2c10 2024-09-12T02:01:13.776 INFO:teuthology.orchestra.run.smithi016.stdout:Backend: cephadm 2024-09-12T02:01:13.776 INFO:teuthology.orchestra.run.smithi016.stdout:Available: Yes 2024-09-12T02:01:13.776 INFO:teuthology.orchestra.run.smithi016.stdout:Paused: No 2024-09-12T02:01:13.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:13.779+0000 7feb09096640 1 -- 172.21.15.16:0/2931750873 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7feaf00a2c10 msgr2=0x7feaf00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:13.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:13.779+0000 7feb09096640 1 --2- 172.21.15.16:0/2931750873 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7feaf00a2c10 0x7feaf00a50d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7feaf4031440 tx=0x7feaf40315b0 comp rx=0 tx=0).stop 2024-09-12T02:01:13.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:13.779+0000 7feb09096640 1 -- 172.21.15.16:0/2931750873 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feb04101c00 msgr2=0x7feb04075f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:13.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:13.779+0000 7feb09096640 1 --2- 172.21.15.16:0/2931750873 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feb04101c00 0x7feb04075f90 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7feaec00d6a0 tx=0x7feaec00db60 comp rx=0 tx=0).stop 2024-09-12T02:01:13.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:13.779+0000 7feb09096640 1 -- 172.21.15.16:0/2931750873 shutdown_connections 2024-09-12T02:01:13.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:13.779+0000 7feb09096640 1 --2- 172.21.15.16:0/2931750873 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7feaf00a2c10 0x7feaf00a50d0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:13.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:13.779+0000 7feb09096640 1 --2- 172.21.15.16:0/2931750873 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb041037c0 0x7feb04074b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:13.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:13.779+0000 7feb09096640 1 --2- 172.21.15.16:0/2931750873 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb04102e00 0x7feb040745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:13.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:13.779+0000 7feb09096640 1 --2- 172.21.15.16:0/2931750873 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feb04101c00 0x7feb04075f90 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:13.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:13.779+0000 7feb09096640 1 -- 172.21.15.16:0/2931750873 >> 172.21.15.16:0/2931750873 conn(0x7feb040fd390 msgr2=0x7feb04106180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:01:13.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:13.779+0000 7feb09096640 1 -- 172.21.15.16:0/2931750873 shutdown_connections 2024-09-12T02:01:13.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:13.779+0000 7feb09096640 1 -- 172.21.15.16:0/2931750873 wait complete. 2024-09-12T02:01:14.396 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:01:14 smithi016 ceph-mon[30947]: pgmap v314: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-12T02:01:14.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:01:14 smithi033 ceph-mon[35332]: pgmap v314: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-12T02:01:14.428 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- bash -c 'ceph orch ps' 2024-09-12T02:01:14.628 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:01:14 smithi184 ceph-mon[35593]: pgmap v314: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-12T02:01:14.689 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:01:15.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:15.124+0000 7fd9de707640 1 -- 172.21.15.16:0/2564250492 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9d8101c00 msgr2=0x7fd9d8102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:15.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:15.124+0000 7fd9de707640 1 --2- 172.21.15.16:0/2564250492 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9d8101c00 0x7fd9d8102000 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fd9c8009900 tx=0x7fd9c802a410 comp rx=0 tx=0).stop 2024-09-12T02:01:15.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:15.124+0000 7fd9de707640 1 -- 172.21.15.16:0/2564250492 shutdown_connections 2024-09-12T02:01:15.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:15.124+0000 7fd9de707640 1 --2- 172.21.15.16:0/2564250492 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd9d81037c0 0x7fd9d810a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:15.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:15.124+0000 7fd9de707640 1 --2- 172.21.15.16:0/2564250492 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9d8102e00 0x7fd9d8103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:15.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:15.124+0000 7fd9de707640 1 --2- 172.21.15.16:0/2564250492 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9d8101c00 0x7fd9d8102000 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:15.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:15.124+0000 7fd9de707640 1 -- 172.21.15.16:0/2564250492 >> 172.21.15.16:0/2564250492 conn(0x7fd9d80fd390 msgr2=0x7fd9d80ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:01:15.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:15.124+0000 7fd9de707640 1 -- 172.21.15.16:0/2564250492 shutdown_connections 2024-09-12T02:01:15.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:15.124+0000 7fd9de707640 1 -- 172.21.15.16:0/2564250492 wait complete. 2024-09-12T02:01:15.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:15.125+0000 7fd9de707640 1 Processor -- start 2024-09-12T02:01:15.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:15.125+0000 7fd9de707640 1 -- start start 2024-09-12T02:01:15.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:15.125+0000 7fd9de707640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd9d8101c00 0x7fd9d819a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:15.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:15.125+0000 7fd9de707640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9d8102e00 0x7fd9d819a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:15.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:15.125+0000 7fd9de707640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9d81037c0 0x7fd9d81a1730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:15.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:15.125+0000 7fd9de707640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd9d810d8e0 con 0x7fd9d8102e00 2024-09-12T02:01:15.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:15.125+0000 7fd9de707640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd9d810d760 con 0x7fd9d81037c0 2024-09-12T02:01:15.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:15.125+0000 7fd9de707640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd9d810da60 con 0x7fd9d8101c00 2024-09-12T02:01:15.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:15.125+0000 7fd9cffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9d8102e00 0x7fd9d819a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:15.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:15.125+0000 7fd9d7fff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd9d8101c00 0x7fd9d819a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:15.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:15.125+0000 7fd9cffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9d8102e00 0x7fd9d819a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35202/0 (socket says 172.21.15.16:35202) 2024-09-12T02:01:15.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:15.126+0000 7fd9cffff640 1 -- 172.21.15.16:0/1327054627 learned_addr learned my addr 172.21.15.16:0/1327054627 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:01:15.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:15.126+0000 7fd9dcca4640 1 --2- 172.21.15.16:0/1327054627 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9d81037c0 0x7fd9d81a1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:15.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:15.126+0000 7fd9cffff640 1 -- 172.21.15.16:0/1327054627 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd9d8101c00 msgr2=0x7fd9d819a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:15.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:15.126+0000 7fd9cffff640 1 --2- 172.21.15.16:0/1327054627 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd9d8101c00 0x7fd9d819a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:15.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:15.126+0000 7fd9cffff640 1 -- 172.21.15.16:0/1327054627 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9d81037c0 msgr2=0x7fd9d81a1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:15.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:15.126+0000 7fd9cffff640 1 --2- 172.21.15.16:0/1327054627 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9d81037c0 0x7fd9d81a1730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:15.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:15.126+0000 7fd9cffff640 1 -- 172.21.15.16:0/1327054627 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9d81a1da0 con 0x7fd9d8102e00 2024-09-12T02:01:15.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:15.126+0000 7fd9cffff640 1 --2- 172.21.15.16:0/1327054627 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9d8102e00 0x7fd9d819a6b0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fd9c000c890 tx=0x7fd9c000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:01:15.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:15.126+0000 7fd9d5ffb640 1 -- 172.21.15.16:0/1327054627 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9c0004220 con 0x7fd9d8102e00 2024-09-12T02:01:15.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:15.126+0000 7fd9d5ffb640 1 -- 172.21.15.16:0/1327054627 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd9c00043c0 con 0x7fd9d8102e00 2024-09-12T02:01:15.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:15.126+0000 7fd9de707640 1 -- 172.21.15.16:0/1327054627 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9d81a2090 con 0x7fd9d8102e00 2024-09-12T02:01:15.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:15.126+0000 7fd9d5ffb640 1 -- 172.21.15.16:0/1327054627 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9c00049e0 con 0x7fd9d8102e00 2024-09-12T02:01:15.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:15.126+0000 7fd9de707640 1 -- 172.21.15.16:0/1327054627 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9d81a2650 con 0x7fd9d8102e00 2024-09-12T02:01:15.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:15.127+0000 7fd9de707640 1 -- 172.21.15.16:0/1327054627 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9d81a2910 con 0x7fd9d8102e00 2024-09-12T02:01:15.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:15.128+0000 7fd9d5ffb640 1 -- 172.21.15.16:0/1327054627 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fd9c00079b0 con 0x7fd9d8102e00 2024-09-12T02:01:15.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:15.129+0000 7fd9d5ffb640 1 --2- 172.21.15.16:0/1327054627 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fd9c40a2c40 0x7fd9c40a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:15.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:15.129+0000 7fd9d7fff640 1 --2- 172.21.15.16:0/1327054627 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fd9c40a2c40 0x7fd9c40a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:15.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:15.129+0000 7fd9d5ffb640 1 -- 172.21.15.16:0/1327054627 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fd9c00cf070 con 0x7fd9d8102e00 2024-09-12T02:01:15.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:15.130+0000 7fd9d7fff640 1 --2- 172.21.15.16:0/1327054627 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fd9c40a2c40 0x7fd9c40a5100 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fd9c8005ab0 tx=0x7fd9c8000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:01:15.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:15.133+0000 7fd9d5ffb640 1 -- 172.21.15.16:0/1327054627 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd9c0014030 con 0x7fd9d8102e00 2024-09-12T02:01:15.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:15.317+0000 7fd9de707640 1 -- 172.21.15.16:0/1327054627 --> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7fd9d8109d10 con 0x7fd9c40a2c40 2024-09-12T02:01:15.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:15.329+0000 7fd9d5ffb640 1 -- 172.21.15.16:0/1327054627 <== mgr.14150 v2:172.21.15.16:6800/3887444820 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4154 (secure 0 0 0) 0x7fd9d8109d10 con 0x7fd9c40a2c40 2024-09-12T02:01:15.330 INFO:teuthology.orchestra.run.smithi016.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-12T02:01:15.330 INFO:teuthology.orchestra.run.smithi016.stdout:mds.a smithi016 running (92s) 17s ago 91s 16.6M - 19.1.1-298-gb53b6d3e 54467abc55f8 14f2a6d3d79f 2024-09-12T02:01:15.330 INFO:teuthology.orchestra.run.smithi016.stdout:mds.b smithi033 running (89s) 22s ago 89s 28.6M - 19.1.1-298-gb53b6d3e 54467abc55f8 4ddf0b353ec0 2024-09-12T02:01:15.330 INFO:teuthology.orchestra.run.smithi016.stdout:mds.c smithi184 running (102s) 22s ago 102s 17.2M - 19.1.1-298-gb53b6d3e 54467abc55f8 89511b817656 2024-09-12T02:01:15.330 INFO:teuthology.orchestra.run.smithi016.stdout:mds.d smithi016 running (110s) 17s ago 109s 15.8M - 19.1.1-298-gb53b6d3e 54467abc55f8 0fcdf8c62845 2024-09-12T02:01:15.330 INFO:teuthology.orchestra.run.smithi016.stdout:mds.e smithi033 running (107s) 22s ago 107s 16.5M - 19.1.1-298-gb53b6d3e 54467abc55f8 ecbc94849c8c 2024-09-12T02:01:15.330 INFO:teuthology.orchestra.run.smithi016.stdout:mds.f smithi184 running (105s) 22s ago 104s 19.5M - 19.1.1-298-gb53b6d3e 54467abc55f8 c4f79993c34f 2024-09-12T02:01:15.331 INFO:teuthology.orchestra.run.smithi016.stdout:mds.g smithi016 running (94s) 17s ago 94s 14.3M - 19.1.1-298-gb53b6d3e 54467abc55f8 a1f810e1d0cc 2024-09-12T02:01:15.331 INFO:teuthology.orchestra.run.smithi016.stdout:mds.h smithi033 running (100s) 22s ago 99s 18.4M - 19.1.1-298-gb53b6d3e 54467abc55f8 56f3bcd24173 2024-09-12T02:01:15.331 INFO:teuthology.orchestra.run.smithi016.stdout:mds.i smithi184 running (97s) 22s ago 97s 18.4M - 19.1.1-298-gb53b6d3e 54467abc55f8 8de64a118bb1 2024-09-12T02:01:15.331 INFO:teuthology.orchestra.run.smithi016.stdout:mds.j smithi016 running (113s) 17s ago 112s 14.2M - 19.1.1-298-gb53b6d3e 54467abc55f8 83e1c73e17f4 2024-09-12T02:01:15.331 INFO:teuthology.orchestra.run.smithi016.stdout:mds.k smithi033 running (116s) 22s ago 116s 13.5M - 19.1.1-298-gb53b6d3e 54467abc55f8 266f2fa2e5ec 2024-09-12T02:01:15.331 INFO:teuthology.orchestra.run.smithi016.stdout:mds.l smithi184 running (118s) 22s ago 118s 16.9M - 19.1.1-298-gb53b6d3e 54467abc55f8 92d03507ab25 2024-09-12T02:01:15.331 INFO:teuthology.orchestra.run.smithi016.stdout:mgr.x smithi016 *:9283,8765 running (8m) 17s ago 8m 507M - 19.1.1-298-gb53b6d3e 54467abc55f8 ea29060771e8 2024-09-12T02:01:15.331 INFO:teuthology.orchestra.run.smithi016.stdout:mgr.y smithi033 *:8443,8765 running (7m) 22s ago 7m 444M - 19.1.1-298-gb53b6d3e 54467abc55f8 c98e15cd9dd5 2024-09-12T02:01:15.331 INFO:teuthology.orchestra.run.smithi016.stdout:mgr.z smithi184 *:8443,8765 running (7m) 22s ago 7m 441M - 19.1.1-298-gb53b6d3e 54467abc55f8 13b9ff9064fa 2024-09-12T02:01:15.331 INFO:teuthology.orchestra.run.smithi016.stdout:mon.a smithi016 running (8m) 17s ago 9m 50.1M 2048M 19.1.1-298-gb53b6d3e 54467abc55f8 ab3eea3e12fe 2024-09-12T02:01:15.331 INFO:teuthology.orchestra.run.smithi016.stdout:mon.b smithi033 running (7m) 22s ago 7m 43.5M 2048M 19.1.1-298-gb53b6d3e 54467abc55f8 0883b7e46f06 2024-09-12T02:01:15.331 INFO:teuthology.orchestra.run.smithi016.stdout:mon.c smithi184 running (7m) 22s ago 7m 40.0M 2048M 19.1.1-298-gb53b6d3e 54467abc55f8 4c7277924f9d 2024-09-12T02:01:15.331 INFO:teuthology.orchestra.run.smithi016.stdout:osd.0 smithi016 running (6m) 17s ago 6m 47.4M 4273M 19.1.1-298-gb53b6d3e 54467abc55f8 67143cd1611c 2024-09-12T02:01:15.331 INFO:teuthology.orchestra.run.smithi016.stdout:osd.1 smithi033 running (6m) 22s ago 6m 58.5M 4273M 19.1.1-298-gb53b6d3e 54467abc55f8 3b7a464851d7 2024-09-12T02:01:15.332 INFO:teuthology.orchestra.run.smithi016.stdout:osd.2 smithi184 running (6m) 22s ago 5m 56.2M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 2a1230457f20 2024-09-12T02:01:15.332 INFO:teuthology.orchestra.run.smithi016.stdout:osd.3 smithi016 running (5m) 17s ago 5m 46.7M 4273M 19.1.1-298-gb53b6d3e 54467abc55f8 00cf949c9eda 2024-09-12T02:01:15.332 INFO:teuthology.orchestra.run.smithi016.stdout:osd.4 smithi033 running (5m) 22s ago 5m 45.7M 4273M 19.1.1-298-gb53b6d3e 54467abc55f8 fb5bdfe221f9 2024-09-12T02:01:15.332 INFO:teuthology.orchestra.run.smithi016.stdout:osd.5 smithi184 running (4m) 22s ago 4m 54.5M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 bea475ed1b6c 2024-09-12T02:01:15.332 INFO:teuthology.orchestra.run.smithi016.stdout:osd.6 smithi016 running (4m) 17s ago 4m 52.2M 4273M 19.1.1-298-gb53b6d3e 54467abc55f8 de82b2ecdc5f 2024-09-12T02:01:15.332 INFO:teuthology.orchestra.run.smithi016.stdout:osd.7 smithi033 running (3m) 22s ago 3m 55.7M 4273M 19.1.1-298-gb53b6d3e 54467abc55f8 6f36b51894a9 2024-09-12T02:01:15.332 INFO:teuthology.orchestra.run.smithi016.stdout:osd.8 smithi184 running (3m) 22s ago 3m 44.4M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 dfe554856d3c 2024-09-12T02:01:15.332 INFO:teuthology.orchestra.run.smithi016.stdout:osd.9 smithi016 running (3m) 17s ago 3m 46.8M 4273M 19.1.1-298-gb53b6d3e 54467abc55f8 8480dcd4fd51 2024-09-12T02:01:15.332 INFO:teuthology.orchestra.run.smithi016.stdout:osd.10 smithi033 running (2m) 22s ago 2m 55.7M 4273M 19.1.1-298-gb53b6d3e 54467abc55f8 a3ae324e478e 2024-09-12T02:01:15.332 INFO:teuthology.orchestra.run.smithi016.stdout:osd.11 smithi184 running (2m) 22s ago 2m 51.1M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 f8be75c18662 2024-09-12T02:01:15.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:15.331+0000 7fd9de707640 1 -- 172.21.15.16:0/1327054627 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fd9c40a2c40 msgr2=0x7fd9c40a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:15.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:15.331+0000 7fd9de707640 1 --2- 172.21.15.16:0/1327054627 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fd9c40a2c40 0x7fd9c40a5100 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fd9c8005ab0 tx=0x7fd9c8000bc0 comp rx=0 tx=0).stop 2024-09-12T02:01:15.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:15.331+0000 7fd9de707640 1 -- 172.21.15.16:0/1327054627 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9d8102e00 msgr2=0x7fd9d819a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:15.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:15.331+0000 7fd9de707640 1 --2- 172.21.15.16:0/1327054627 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9d8102e00 0x7fd9d819a6b0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fd9c000c890 tx=0x7fd9c000cd50 comp rx=0 tx=0).stop 2024-09-12T02:01:15.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:15.331+0000 7fd9de707640 1 -- 172.21.15.16:0/1327054627 shutdown_connections 2024-09-12T02:01:15.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:15.331+0000 7fd9de707640 1 --2- 172.21.15.16:0/1327054627 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fd9c40a2c40 0x7fd9c40a5100 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:15.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:15.331+0000 7fd9de707640 1 --2- 172.21.15.16:0/1327054627 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9d81037c0 0x7fd9d81a1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:15.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:15.331+0000 7fd9de707640 1 --2- 172.21.15.16:0/1327054627 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9d8102e00 0x7fd9d819a6b0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:15.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:15.332+0000 7fd9de707640 1 --2- 172.21.15.16:0/1327054627 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd9d8101c00 0x7fd9d819a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:15.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:15.332+0000 7fd9de707640 1 -- 172.21.15.16:0/1327054627 >> 172.21.15.16:0/1327054627 conn(0x7fd9d80fd390 msgr2=0x7fd9d81085f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:01:15.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:15.332+0000 7fd9de707640 1 -- 172.21.15.16:0/1327054627 shutdown_connections 2024-09-12T02:01:15.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:15.332+0000 7fd9de707640 1 -- 172.21.15.16:0/1327054627 wait complete. 2024-09-12T02:01:15.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:01:15 smithi033 ceph-mon[35332]: from='client.24739 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:01:15.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:01:15 smithi016 ceph-mon[30947]: from='client.24739 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:01:15.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:01:15 smithi184 ceph-mon[35593]: from='client.24739 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:01:16.007 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- bash -c 'ceph orch ls' 2024-09-12T02:01:16.267 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:01:16.388 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:01:16 smithi016 ceph-mon[30947]: pgmap v315: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-12T02:01:16.418 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:01:16 smithi033 ceph-mon[35332]: pgmap v315: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-12T02:01:16.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:01:16 smithi184 ceph-mon[35593]: pgmap v315: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-12T02:01:16.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:16.722+0000 7f1c4d91a640 1 -- 172.21.15.16:0/3348157240 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c48104f90 msgr2=0x7f1c48107380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:16.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:16.722+0000 7f1c4d91a640 1 --2- 172.21.15.16:0/3348157240 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c48104f90 0x7f1c48107380 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f1c3c009900 tx=0x7f1c3c02a410 comp rx=0 tx=0).stop 2024-09-12T02:01:16.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:16.722+0000 7f1c4d91a640 1 -- 172.21.15.16:0/3348157240 shutdown_connections 2024-09-12T02:01:16.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:16.722+0000 7f1c4d91a640 1 --2- 172.21.15.16:0/3348157240 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1c481078c0 0x7f1c48109d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:16.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:16.722+0000 7f1c4d91a640 1 --2- 172.21.15.16:0/3348157240 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c48104f90 0x7f1c48107380 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:16.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:16.722+0000 7f1c4d91a640 1 --2- 172.21.15.16:0/3348157240 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1c48102660 0x7f1c48104a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:16.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:16.722+0000 7f1c4d91a640 1 -- 172.21.15.16:0/3348157240 >> 172.21.15.16:0/3348157240 conn(0x7f1c480fc280 msgr2=0x7f1c480fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:01:16.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:16.722+0000 7f1c4d91a640 1 -- 172.21.15.16:0/3348157240 shutdown_connections 2024-09-12T02:01:16.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:16.722+0000 7f1c4d91a640 1 -- 172.21.15.16:0/3348157240 wait complete. 2024-09-12T02:01:16.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:16.723+0000 7f1c4d91a640 1 Processor -- start 2024-09-12T02:01:16.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:16.723+0000 7f1c4d91a640 1 -- start start 2024-09-12T02:01:16.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:16.723+0000 7f1c4d91a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1c48102660 0x7f1c48197f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:16.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:16.723+0000 7f1c4d91a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1c48104f90 0x7f1c481984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:16.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:16.723+0000 7f1c4d91a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c481078c0 0x7f1c4819f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:16.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:16.723+0000 7f1c4d91a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1c4810d900 con 0x7f1c481078c0 2024-09-12T02:01:16.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:16.723+0000 7f1c4d91a640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1c4810d780 con 0x7f1c48102660 2024-09-12T02:01:16.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:16.723+0000 7f1c4d91a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1c4810da80 con 0x7f1c48104f90 2024-09-12T02:01:16.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:16.724+0000 7f1c477fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c481078c0 0x7f1c4819f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:16.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:16.724+0000 7f1c46ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1c48102660 0x7f1c48197f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:16.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:16.724+0000 7f1c46ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1c48102660 0x7f1c48197f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.16:49006/0 (socket says 172.21.15.16:49006) 2024-09-12T02:01:16.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:16.724+0000 7f1c46ffd640 1 -- 172.21.15.16:0/1098134414 learned_addr learned my addr 172.21.15.16:0/1098134414 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:01:16.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:16.724+0000 7f1c477fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c481078c0 0x7f1c4819f540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35214/0 (socket says 172.21.15.16:35214) 2024-09-12T02:01:16.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:16.724+0000 7f1c467fc640 1 --2- 172.21.15.16:0/1098134414 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1c48104f90 0x7f1c481984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:16.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:16.724+0000 7f1c477fe640 1 -- 172.21.15.16:0/1098134414 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1c48104f90 msgr2=0x7f1c481984c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:16.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:16.724+0000 7f1c477fe640 1 --2- 172.21.15.16:0/1098134414 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1c48104f90 0x7f1c481984c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:16.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:16.724+0000 7f1c477fe640 1 -- 172.21.15.16:0/1098134414 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1c48102660 msgr2=0x7f1c48197f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:16.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:16.724+0000 7f1c477fe640 1 --2- 172.21.15.16:0/1098134414 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1c48102660 0x7f1c48197f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:16.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:16.724+0000 7f1c477fe640 1 -- 172.21.15.16:0/1098134414 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c4819fbb0 con 0x7f1c481078c0 2024-09-12T02:01:16.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:16.724+0000 7f1c467fc640 1 --2- 172.21.15.16:0/1098134414 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1c48104f90 0x7f1c481984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:01:16.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:16.724+0000 7f1c477fe640 1 --2- 172.21.15.16:0/1098134414 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c481078c0 0x7f1c4819f540 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f1c4000be20 tx=0x7f1c4000bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:01:16.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:16.724+0000 7f1c4c918640 1 -- 172.21.15.16:0/1098134414 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c40019070 con 0x7f1c481078c0 2024-09-12T02:01:16.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:16.724+0000 7f1c4c918640 1 -- 172.21.15.16:0/1098134414 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c40014920 con 0x7f1c481078c0 2024-09-12T02:01:16.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:16.724+0000 7f1c4d91a640 1 -- 172.21.15.16:0/1098134414 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c4819fea0 con 0x7f1c481078c0 2024-09-12T02:01:16.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:16.724+0000 7f1c4c918640 1 -- 172.21.15.16:0/1098134414 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c40012920 con 0x7f1c481078c0 2024-09-12T02:01:16.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:16.725+0000 7f1c4d91a640 1 -- 172.21.15.16:0/1098134414 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c481a03d0 con 0x7f1c481078c0 2024-09-12T02:01:16.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:16.725+0000 7f1c4d91a640 1 -- 172.21.15.16:0/1098134414 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c480fdd70 con 0x7f1c481078c0 2024-09-12T02:01:16.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:16.727+0000 7f1c4c918640 1 -- 172.21.15.16:0/1098134414 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f1c4001c070 con 0x7f1c481078c0 2024-09-12T02:01:16.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:16.727+0000 7f1c4c918640 1 --2- 172.21.15.16:0/1098134414 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f1c300a2c40 0x7f1c300a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:16.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:16.727+0000 7f1c46ffd640 1 --2- 172.21.15.16:0/1098134414 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f1c300a2c40 0x7f1c300a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:16.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:16.727+0000 7f1c4c918640 1 -- 172.21.15.16:0/1098134414 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f1c400ce790 con 0x7f1c481078c0 2024-09-12T02:01:16.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:16.728+0000 7f1c46ffd640 1 --2- 172.21.15.16:0/1098134414 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f1c300a2c40 0x7f1c300a5100 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f1c38005fd0 tx=0x7f1c3800b480 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:01:16.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:16.731+0000 7f1c4c918640 1 -- 172.21.15.16:0/1098134414 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1c4009b150 con 0x7f1c481078c0 2024-09-12T02:01:16.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:16.915+0000 7f1c4d91a640 1 -- 172.21.15.16:0/1098134414 --> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f1c48062630 con 0x7f1c300a2c40 2024-09-12T02:01:16.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:16.921+0000 7f1c4c918640 1 -- 172.21.15.16:0/1098134414 <== mgr.14150 v2:172.21.15.16:6800/3887444820 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f1c48062630 con 0x7f1c300a2c40 2024-09-12T02:01:16.921 INFO:teuthology.orchestra.run.smithi016.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-09-12T02:01:16.921 INFO:teuthology.orchestra.run.smithi016.stdout:mds.all 12/12 24s ago 2m smithi016=a;smithi016=d;smithi016=g;smithi016=j;smithi033=b;smithi033=e;smithi033=h;smithi033=k;smithi184=c;smithi184=f;smithi184=i;smithi184=l;count:12 2024-09-12T02:01:16.921 INFO:teuthology.orchestra.run.smithi016.stdout:mgr 3/3 24s ago 7m smithi016=x;smithi033=y;smithi184=z;count:3 2024-09-12T02:01:16.921 INFO:teuthology.orchestra.run.smithi016.stdout:mon 3/3 24s ago 7m smithi016:172.21.15.16=a;smithi033:172.21.15.33=b;smithi184:172.21.15.184=c;count:3 2024-09-12T02:01:16.921 INFO:teuthology.orchestra.run.smithi016.stdout:osd 12 24s ago - 2024-09-12T02:01:16.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:16.923+0000 7f1c4d91a640 1 -- 172.21.15.16:0/1098134414 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f1c300a2c40 msgr2=0x7f1c300a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:16.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:16.923+0000 7f1c4d91a640 1 --2- 172.21.15.16:0/1098134414 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f1c300a2c40 0x7f1c300a5100 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f1c38005fd0 tx=0x7f1c3800b480 comp rx=0 tx=0).stop 2024-09-12T02:01:16.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:16.923+0000 7f1c4d91a640 1 -- 172.21.15.16:0/1098134414 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c481078c0 msgr2=0x7f1c4819f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:16.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:16.923+0000 7f1c4d91a640 1 --2- 172.21.15.16:0/1098134414 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c481078c0 0x7f1c4819f540 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f1c4000be20 tx=0x7f1c4000bf10 comp rx=0 tx=0).stop 2024-09-12T02:01:16.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:16.923+0000 7f1c4d91a640 1 -- 172.21.15.16:0/1098134414 shutdown_connections 2024-09-12T02:01:16.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:16.923+0000 7f1c4d91a640 1 --2- 172.21.15.16:0/1098134414 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f1c300a2c40 0x7f1c300a5100 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:16.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:16.923+0000 7f1c4d91a640 1 --2- 172.21.15.16:0/1098134414 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c481078c0 0x7f1c4819f540 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:16.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:16.923+0000 7f1c4d91a640 1 --2- 172.21.15.16:0/1098134414 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1c48104f90 0x7f1c481984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:16.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:16.923+0000 7f1c4d91a640 1 --2- 172.21.15.16:0/1098134414 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1c48102660 0x7f1c48197f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:16.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:16.923+0000 7f1c4d91a640 1 -- 172.21.15.16:0/1098134414 >> 172.21.15.16:0/1098134414 conn(0x7f1c480fc280 msgr2=0x7f1c48102f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:01:16.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:16.923+0000 7f1c4d91a640 1 -- 172.21.15.16:0/1098134414 shutdown_connections 2024-09-12T02:01:16.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:16.923+0000 7f1c4d91a640 1 -- 172.21.15.16:0/1098134414 wait complete. 2024-09-12T02:01:17.183 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:01:17 smithi016 ceph-mon[30947]: from='client.15021 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:01:17.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:01:17 smithi033 ceph-mon[35332]: from='client.15021 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:01:17.479 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- bash -c 'ceph orch host ls' 2024-09-12T02:01:17.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:01:17 smithi184 ceph-mon[35593]: from='client.15021 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:01:17.733 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:01:18.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:18.146+0000 7fdcbd770640 1 -- 172.21.15.16:0/1856951947 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdcb8107960 msgr2=0x7fdcb8109d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:18.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:18.146+0000 7fdcbd770640 1 --2- 172.21.15.16:0/1856951947 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdcb8107960 0x7fdcb8109d50 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fdcb0009ca0 tx=0x7fdcb002a8c0 comp rx=0 tx=0).stop 2024-09-12T02:01:18.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:18.146+0000 7fdcbd770640 1 -- 172.21.15.16:0/1856951947 shutdown_connections 2024-09-12T02:01:18.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:18.146+0000 7fdcbd770640 1 --2- 172.21.15.16:0/1856951947 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdcb8107960 0x7fdcb8109d50 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:18.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:18.146+0000 7fdcbd770640 1 --2- 172.21.15.16:0/1856951947 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdcb8105030 0x7fdcb8107420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:18.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:18.146+0000 7fdcbd770640 1 --2- 172.21.15.16:0/1856951947 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdcb8102700 0x7fdcb8104af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:18.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:18.146+0000 7fdcbd770640 1 -- 172.21.15.16:0/1856951947 >> 172.21.15.16:0/1856951947 conn(0x7fdcb80fc2a0 msgr2=0x7fdcb80fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:01:18.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:18.146+0000 7fdcbd770640 1 -- 172.21.15.16:0/1856951947 shutdown_connections 2024-09-12T02:01:18.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:18.146+0000 7fdcbd770640 1 -- 172.21.15.16:0/1856951947 wait complete. 2024-09-12T02:01:18.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:18.147+0000 7fdcbd770640 1 Processor -- start 2024-09-12T02:01:18.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:18.147+0000 7fdcbd770640 1 -- start start 2024-09-12T02:01:18.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:18.147+0000 7fdcbd770640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdcb8102700 0x7fdcb819a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:18.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:18.147+0000 7fdcbd770640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdcb8105030 0x7fdcb819a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:18.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:18.147+0000 7fdcbd770640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdcb8107960 0x7fdcb81a16e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:18.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:18.147+0000 7fdcbd770640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fdcb810d910 con 0x7fdcb8107960 2024-09-12T02:01:18.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:18.147+0000 7fdcbd770640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fdcb810d790 con 0x7fdcb8105030 2024-09-12T02:01:18.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:18.147+0000 7fdcbd770640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fdcb810da90 con 0x7fdcb8102700 2024-09-12T02:01:18.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:18.147+0000 7fdcb77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdcb8107960 0x7fdcb81a16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:18.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:18.147+0000 7fdcb77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdcb8107960 0x7fdcb81a16e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35236/0 (socket says 172.21.15.16:35236) 2024-09-12T02:01:18.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:18.147+0000 7fdcb77fe640 1 -- 172.21.15.16:0/4084451027 learned_addr learned my addr 172.21.15.16:0/4084451027 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:01:18.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:18.147+0000 7fdcb67fc640 1 --2- 172.21.15.16:0/4084451027 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdcb8105030 0x7fdcb819a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:18.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:18.148+0000 7fdcb6ffd640 1 --2- 172.21.15.16:0/4084451027 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdcb8102700 0x7fdcb819a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:18.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:18.148+0000 7fdcb77fe640 1 -- 172.21.15.16:0/4084451027 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdcb8102700 msgr2=0x7fdcb819a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:18.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:18.148+0000 7fdcb77fe640 1 --2- 172.21.15.16:0/4084451027 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdcb8102700 0x7fdcb819a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:18.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:18.148+0000 7fdcb77fe640 1 -- 172.21.15.16:0/4084451027 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdcb8105030 msgr2=0x7fdcb819a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:18.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:18.148+0000 7fdcb77fe640 1 --2- 172.21.15.16:0/4084451027 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdcb8105030 0x7fdcb819a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:18.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:18.148+0000 7fdcb77fe640 1 -- 172.21.15.16:0/4084451027 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcb81a1d50 con 0x7fdcb8107960 2024-09-12T02:01:18.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:18.148+0000 7fdcb77fe640 1 --2- 172.21.15.16:0/4084451027 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdcb8107960 0x7fdcb81a16e0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fdcb00319f0 tx=0x7fdcb002f450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:01:18.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:18.148+0000 7fdc9ffff640 1 -- 172.21.15.16:0/4084451027 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcb0003c70 con 0x7fdcb8107960 2024-09-12T02:01:18.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:18.148+0000 7fdcbd770640 1 -- 172.21.15.16:0/4084451027 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcb81a1fe0 con 0x7fdcb8107960 2024-09-12T02:01:18.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:18.148+0000 7fdc9ffff640 1 -- 172.21.15.16:0/4084451027 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdcb0003e10 con 0x7fdcb8107960 2024-09-12T02:01:18.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:18.148+0000 7fdc9ffff640 1 -- 172.21.15.16:0/4084451027 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcb002fd30 con 0x7fdcb8107960 2024-09-12T02:01:18.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:18.148+0000 7fdcbd770640 1 -- 172.21.15.16:0/4084451027 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcb81a24e0 con 0x7fdcb8107960 2024-09-12T02:01:18.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:18.149+0000 7fdcbd770640 1 -- 172.21.15.16:0/4084451027 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdcb8109450 con 0x7fdcb8107960 2024-09-12T02:01:18.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:18.150+0000 7fdc9ffff640 1 -- 172.21.15.16:0/4084451027 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fdcb0004b40 con 0x7fdcb8107960 2024-09-12T02:01:18.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:18.151+0000 7fdc9ffff640 1 --2- 172.21.15.16:0/4084451027 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fdca00a2c40 0x7fdca00a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:18.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:18.151+0000 7fdcb6ffd640 1 --2- 172.21.15.16:0/4084451027 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fdca00a2c40 0x7fdca00a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:18.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:18.151+0000 7fdc9ffff640 1 -- 172.21.15.16:0/4084451027 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fdcb00f0730 con 0x7fdcb8107960 2024-09-12T02:01:18.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:18.152+0000 7fdcb6ffd640 1 --2- 172.21.15.16:0/4084451027 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fdca00a2c40 0x7fdca00a5100 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fdca8005fd0 tx=0x7fdca800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:01:18.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:18.155+0000 7fdc9ffff640 1 -- 172.21.15.16:0/4084451027 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fdcb00bd640 con 0x7fdcb8107960 2024-09-12T02:01:18.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:18.347+0000 7fdcbd770640 1 -- 172.21.15.16:0/4084451027 --> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7fdcb8062630 con 0x7fdca00a2c40 2024-09-12T02:01:18.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:18.348+0000 7fdc9ffff640 1 -- 172.21.15.16:0/4084451027 <== mgr.14150 v2:172.21.15.16:6800/3887444820 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7fdcb8062630 con 0x7fdca00a2c40 2024-09-12T02:01:18.348 INFO:teuthology.orchestra.run.smithi016.stdout:HOST ADDR LABELS STATUS 2024-09-12T02:01:18.348 INFO:teuthology.orchestra.run.smithi016.stdout:smithi016 172.21.15.16 2024-09-12T02:01:18.348 INFO:teuthology.orchestra.run.smithi016.stdout:smithi033 172.21.15.33 2024-09-12T02:01:18.348 INFO:teuthology.orchestra.run.smithi016.stdout:smithi184 172.21.15.184 2024-09-12T02:01:18.349 INFO:teuthology.orchestra.run.smithi016.stdout:3 hosts in cluster 2024-09-12T02:01:18.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:18.351+0000 7fdcbd770640 1 -- 172.21.15.16:0/4084451027 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fdca00a2c40 msgr2=0x7fdca00a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:18.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:18.351+0000 7fdcbd770640 1 --2- 172.21.15.16:0/4084451027 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fdca00a2c40 0x7fdca00a5100 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fdca8005fd0 tx=0x7fdca800b390 comp rx=0 tx=0).stop 2024-09-12T02:01:18.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:18.351+0000 7fdcbd770640 1 -- 172.21.15.16:0/4084451027 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdcb8107960 msgr2=0x7fdcb81a16e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:18.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:18.351+0000 7fdcbd770640 1 --2- 172.21.15.16:0/4084451027 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdcb8107960 0x7fdcb81a16e0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fdcb00319f0 tx=0x7fdcb002f450 comp rx=0 tx=0).stop 2024-09-12T02:01:18.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:18.351+0000 7fdcbd770640 1 -- 172.21.15.16:0/4084451027 shutdown_connections 2024-09-12T02:01:18.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:18.351+0000 7fdcbd770640 1 --2- 172.21.15.16:0/4084451027 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fdca00a2c40 0x7fdca00a5100 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:18.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:18.351+0000 7fdcbd770640 1 --2- 172.21.15.16:0/4084451027 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdcb8107960 0x7fdcb81a16e0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:18.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:18.351+0000 7fdcbd770640 1 --2- 172.21.15.16:0/4084451027 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdcb8105030 0x7fdcb819a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:18.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:18.351+0000 7fdcbd770640 1 --2- 172.21.15.16:0/4084451027 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdcb8102700 0x7fdcb819a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:18.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:18.351+0000 7fdcbd770640 1 -- 172.21.15.16:0/4084451027 >> 172.21.15.16:0/4084451027 conn(0x7fdcb80fc2a0 msgr2=0x7fdcb8101ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:01:18.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:18.351+0000 7fdcbd770640 1 -- 172.21.15.16:0/4084451027 shutdown_connections 2024-09-12T02:01:18.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:18.352+0000 7fdcbd770640 1 -- 172.21.15.16:0/4084451027 wait complete. 2024-09-12T02:01:18.418 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:01:18 smithi033 ceph-mon[35332]: pgmap v316: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-12T02:01:18.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:01:18 smithi033 ceph-mon[35332]: from='client.15027 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:01:18.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:01:18 smithi016 ceph-mon[30947]: pgmap v316: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-12T02:01:18.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:01:18 smithi016 ceph-mon[30947]: from='client.15027 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:01:18.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:01:18 smithi184 ceph-mon[35593]: pgmap v316: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-12T02:01:18.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:01:18 smithi184 ceph-mon[35593]: from='client.15027 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:01:18.942 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- bash -c 'ceph orch device ls' 2024-09-12T02:01:19.203 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:01:19.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:19.646+0000 7f77a898d640 1 -- 172.21.15.16:0/2332669281 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77a4102e00 msgr2=0x7f77a4103280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:19.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:19.646+0000 7f77a898d640 1 --2- 172.21.15.16:0/2332669281 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77a4102e00 0x7f77a4103280 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f7798009900 tx=0x7f779802a410 comp rx=0 tx=0).stop 2024-09-12T02:01:19.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:19.646+0000 7f77a898d640 1 -- 172.21.15.16:0/2332669281 shutdown_connections 2024-09-12T02:01:19.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:19.646+0000 7f77a898d640 1 --2- 172.21.15.16:0/2332669281 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f77a41037c0 0x7f77a410a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:19.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:19.646+0000 7f77a898d640 1 --2- 172.21.15.16:0/2332669281 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77a4102e00 0x7f77a4103280 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:19.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:19.646+0000 7f77a898d640 1 --2- 172.21.15.16:0/2332669281 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77a4101c00 0x7f77a4102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:19.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:19.646+0000 7f77a898d640 1 -- 172.21.15.16:0/2332669281 >> 172.21.15.16:0/2332669281 conn(0x7f77a40fd390 msgr2=0x7f77a40ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:01:19.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:19.646+0000 7f77a898d640 1 -- 172.21.15.16:0/2332669281 shutdown_connections 2024-09-12T02:01:19.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:19.646+0000 7f77a898d640 1 -- 172.21.15.16:0/2332669281 wait complete. 2024-09-12T02:01:19.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:19.647+0000 7f77a898d640 1 Processor -- start 2024-09-12T02:01:19.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:19.647+0000 7f77a898d640 1 -- start start 2024-09-12T02:01:19.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:19.647+0000 7f77a898d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f77a4101c00 0x7f77a4197e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:19.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:19.647+0000 7f77a898d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77a4102e00 0x7f77a41983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:19.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:19.647+0000 7f77a898d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77a41037c0 0x7f77a419f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:19.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:19.647+0000 7f77a898d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f77a410d8b0 con 0x7f77a4102e00 2024-09-12T02:01:19.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:19.647+0000 7f77a898d640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f77a410d730 con 0x7f77a41037c0 2024-09-12T02:01:19.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:19.647+0000 7f77a898d640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f77a410da30 con 0x7f77a4101c00 2024-09-12T02:01:19.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:19.647+0000 7f77a1d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77a4102e00 0x7f77a41983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:19.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:19.647+0000 7f77a1d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77a4102e00 0x7f77a41983c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35260/0 (socket says 172.21.15.16:35260) 2024-09-12T02:01:19.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:19.647+0000 7f77a1d9b640 1 -- 172.21.15.16:0/3809982744 learned_addr learned my addr 172.21.15.16:0/3809982744 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:01:19.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:19.647+0000 7f77a259c640 1 --2- 172.21.15.16:0/3809982744 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f77a4101c00 0x7f77a4197e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:19.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:19.647+0000 7f77a2d9d640 1 --2- 172.21.15.16:0/3809982744 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77a41037c0 0x7f77a419f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:19.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:19.648+0000 7f77a1d9b640 1 -- 172.21.15.16:0/3809982744 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f77a4101c00 msgr2=0x7f77a4197e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:19.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:19.648+0000 7f77a1d9b640 1 --2- 172.21.15.16:0/3809982744 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f77a4101c00 0x7f77a4197e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:19.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:19.648+0000 7f77a1d9b640 1 -- 172.21.15.16:0/3809982744 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77a41037c0 msgr2=0x7f77a419f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:19.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:19.648+0000 7f77a1d9b640 1 --2- 172.21.15.16:0/3809982744 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77a41037c0 0x7f77a419f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:19.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:19.648+0000 7f77a1d9b640 1 -- 172.21.15.16:0/3809982744 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77a419fab0 con 0x7f77a4102e00 2024-09-12T02:01:19.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:19.648+0000 7f77a1d9b640 1 --2- 172.21.15.16:0/3809982744 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77a4102e00 0x7f77a41983c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f7798031440 tx=0x7f7798032e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:01:19.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:19.648+0000 7f77937fe640 1 -- 172.21.15.16:0/3809982744 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f779803f030 con 0x7f77a4102e00 2024-09-12T02:01:19.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:19.648+0000 7f77937fe640 1 -- 172.21.15.16:0/3809982744 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7798003ce0 con 0x7f77a4102e00 2024-09-12T02:01:19.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:19.648+0000 7f77a898d640 1 -- 172.21.15.16:0/3809982744 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77a419fd40 con 0x7f77a4102e00 2024-09-12T02:01:19.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:19.648+0000 7f77937fe640 1 -- 172.21.15.16:0/3809982744 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7798035070 con 0x7f77a4102e00 2024-09-12T02:01:19.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:19.648+0000 7f77a898d640 1 -- 172.21.15.16:0/3809982744 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77a41a01c0 con 0x7f77a4102e00 2024-09-12T02:01:19.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:19.649+0000 7f77a898d640 1 -- 172.21.15.16:0/3809982744 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77a4108fb0 con 0x7f77a4102e00 2024-09-12T02:01:19.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:19.650+0000 7f77937fe640 1 -- 172.21.15.16:0/3809982744 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f7798042030 con 0x7f77a4102e00 2024-09-12T02:01:19.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:19.651+0000 7f77937fe640 1 --2- 172.21.15.16:0/3809982744 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f778c0a2c40 0x7f778c0a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:19.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:19.651+0000 7f77937fe640 1 -- 172.21.15.16:0/3809982744 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f77980efc00 con 0x7f77a4102e00 2024-09-12T02:01:19.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:19.651+0000 7f77a259c640 1 --2- 172.21.15.16:0/3809982744 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f778c0a2c40 0x7f778c0a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:19.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:19.651+0000 7f77a259c640 1 --2- 172.21.15.16:0/3809982744 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f778c0a2c40 0x7f778c0a5100 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f77a4102c60 tx=0x7f7794005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:01:19.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:19.654+0000 7f77937fe640 1 -- 172.21.15.16:0/3809982744 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f77980bca00 con 0x7f77a4102e00 2024-09-12T02:01:19.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:19.835+0000 7f77a898d640 1 -- 172.21.15.16:0/3809982744 --> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f77a4062630 con 0x7f778c0a2c40 2024-09-12T02:01:19.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:19.837+0000 7f77937fe640 1 -- 172.21.15.16:0/3809982744 <== mgr.14150 v2:172.21.15.16:6800/3887444820 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f77a4062630 con 0x7f778c0a2c40 2024-09-12T02:01:19.837 INFO:teuthology.orchestra.run.smithi016.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-09-12T02:01:19.837 INFO:teuthology.orchestra.run.smithi016.stdout:smithi016 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB7513002C480DGN 447G No 3m ago Has a FileSystem, LVM detected 2024-09-12T02:01:19.837 INFO:teuthology.orchestra.run.smithi016.stdout:smithi033 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT5330008Q400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-09-12T02:01:19.837 INFO:teuthology.orchestra.run.smithi016.stdout:smithi184 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT626500CE400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-09-12T02:01:19.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:19.838+0000 7f77a898d640 1 -- 172.21.15.16:0/3809982744 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f778c0a2c40 msgr2=0x7f778c0a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:19.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:19.838+0000 7f77a898d640 1 --2- 172.21.15.16:0/3809982744 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f778c0a2c40 0x7f778c0a5100 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f77a4102c60 tx=0x7f7794005f90 comp rx=0 tx=0).stop 2024-09-12T02:01:19.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:19.838+0000 7f77a898d640 1 -- 172.21.15.16:0/3809982744 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77a4102e00 msgr2=0x7f77a41983c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:19.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:19.838+0000 7f77a898d640 1 --2- 172.21.15.16:0/3809982744 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77a4102e00 0x7f77a41983c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f7798031440 tx=0x7f7798032e20 comp rx=0 tx=0).stop 2024-09-12T02:01:19.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:19.838+0000 7f77a898d640 1 -- 172.21.15.16:0/3809982744 shutdown_connections 2024-09-12T02:01:19.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:19.839+0000 7f77a898d640 1 --2- 172.21.15.16:0/3809982744 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f778c0a2c40 0x7f778c0a5100 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:19.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:19.839+0000 7f77a898d640 1 --2- 172.21.15.16:0/3809982744 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77a41037c0 0x7f77a419f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:19.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:19.839+0000 7f77a898d640 1 --2- 172.21.15.16:0/3809982744 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77a4102e00 0x7f77a41983c0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:19.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:19.839+0000 7f77a898d640 1 --2- 172.21.15.16:0/3809982744 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f77a4101c00 0x7f77a4197e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:19.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:19.839+0000 7f77a898d640 1 -- 172.21.15.16:0/3809982744 >> 172.21.15.16:0/3809982744 conn(0x7f77a40fd390 msgr2=0x7f77a4106180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:01:19.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:19.839+0000 7f77a898d640 1 -- 172.21.15.16:0/3809982744 shutdown_connections 2024-09-12T02:01:19.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:19.839+0000 7f77a898d640 1 -- 172.21.15.16:0/3809982744 wait complete. 2024-09-12T02:01:20.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:01:20 smithi033 ceph-mon[35332]: pgmap v317: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T02:01:20.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:01:20 smithi033 ceph-mon[35332]: from='client.15033 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:01:20.429 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:01:20 smithi184 ceph-mon[35593]: pgmap v317: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T02:01:20.429 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:01:20 smithi184 ceph-mon[35593]: from='client.15033 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:01:20.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:01:20 smithi016 ceph-mon[30947]: pgmap v317: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T02:01:20.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:01:20 smithi016 ceph-mon[30947]: from='client.15033 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:01:20.520 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-12T02:01:20.530 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi016.front.sepia.ceph.com 2024-09-12T02:01:20.530 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- bash -c 'ceph fs dump' 2024-09-12T02:01:20.789 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:01:21.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:21.203+0000 7f8e17640640 1 -- 172.21.15.16:0/2018915525 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e10101d10 msgr2=0x7f8e10102190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:21.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:21.203+0000 7f8e17640640 1 --2- 172.21.15.16:0/2018915525 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e10101d10 0x7f8e10102190 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f8e00009900 tx=0x7f8e0002a410 comp rx=0 tx=0).stop 2024-09-12T02:01:21.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:21.203+0000 7f8e17640640 1 -- 172.21.15.16:0/2018915525 shutdown_connections 2024-09-12T02:01:21.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:21.203+0000 7f8e17640640 1 --2- 172.21.15.16:0/2018915525 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e101026d0 0x7f8e10108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:21.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:21.203+0000 7f8e17640640 1 --2- 172.21.15.16:0/2018915525 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e10101d10 0x7f8e10102190 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:21.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:21.203+0000 7f8e17640640 1 --2- 172.21.15.16:0/2018915525 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8e10100b10 0x7f8e10100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:21.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:21.203+0000 7f8e17640640 1 -- 172.21.15.16:0/2018915525 >> 172.21.15.16:0/2018915525 conn(0x7f8e100fc2a0 msgr2=0x7f8e100fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:01:21.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:21.204+0000 7f8e17640640 1 -- 172.21.15.16:0/2018915525 shutdown_connections 2024-09-12T02:01:21.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:21.204+0000 7f8e17640640 1 -- 172.21.15.16:0/2018915525 wait complete. 2024-09-12T02:01:21.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:21.204+0000 7f8e17640640 1 Processor -- start 2024-09-12T02:01:21.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:21.204+0000 7f8e17640640 1 -- start start 2024-09-12T02:01:21.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:21.204+0000 7f8e17640640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e10100b10 0x7f8e10197f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:21.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:21.204+0000 7f8e17640640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e10101d10 0x7f8e10198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:21.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:21.204+0000 7f8e17640640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8e101026d0 0x7f8e1019f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:21.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:21.204+0000 7f8e17640640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8e1010d8c0 con 0x7f8e10100b10 2024-09-12T02:01:21.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:21.204+0000 7f8e17640640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8e1010d740 con 0x7f8e10101d10 2024-09-12T02:01:21.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:21.204+0000 7f8e17640640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8e1010da40 con 0x7f8e101026d0 2024-09-12T02:01:21.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:21.204+0000 7f8e153dc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e10100b10 0x7f8e10197f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:21.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:21.204+0000 7f8e153dc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e10100b10 0x7f8e10197f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42962/0 (socket says 172.21.15.16:42962) 2024-09-12T02:01:21.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:21.204+0000 7f8e153dc640 1 -- 172.21.15.16:0/897862557 learned_addr learned my addr 172.21.15.16:0/897862557 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:01:21.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:21.205+0000 7f8e15bdd640 1 --2- 172.21.15.16:0/897862557 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8e101026d0 0x7f8e1019f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:21.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:21.205+0000 7f8e14bdb640 1 --2- 172.21.15.16:0/897862557 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e10101d10 0x7f8e10198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:21.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:21.205+0000 7f8e153dc640 1 -- 172.21.15.16:0/897862557 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8e101026d0 msgr2=0x7f8e1019f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:21.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:21.205+0000 7f8e153dc640 1 --2- 172.21.15.16:0/897862557 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8e101026d0 0x7f8e1019f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:21.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:21.205+0000 7f8e153dc640 1 -- 172.21.15.16:0/897862557 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e10101d10 msgr2=0x7f8e10198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:21.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:21.205+0000 7f8e153dc640 1 --2- 172.21.15.16:0/897862557 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e10101d10 0x7f8e10198450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:21.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:21.205+0000 7f8e153dc640 1 -- 172.21.15.16:0/897862557 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e1019fb40 con 0x7f8e10100b10 2024-09-12T02:01:21.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:21.205+0000 7f8e14bdb640 1 --2- 172.21.15.16:0/897862557 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e10101d10 0x7f8e10198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:01:21.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:21.205+0000 7f8e153dc640 1 --2- 172.21.15.16:0/897862557 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e10100b10 0x7f8e10197f10 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f8e0c00e9c0 tx=0x7f8e0c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:01:21.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:21.205+0000 7f8e067fc640 1 -- 172.21.15.16:0/897862557 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e0c00cd80 con 0x7f8e10100b10 2024-09-12T02:01:21.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:21.205+0000 7f8e17640640 1 -- 172.21.15.16:0/897862557 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e1019fe30 con 0x7f8e10100b10 2024-09-12T02:01:21.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:21.205+0000 7f8e067fc640 1 -- 172.21.15.16:0/897862557 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e0c0035f0 con 0x7f8e10100b10 2024-09-12T02:01:21.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:21.205+0000 7f8e067fc640 1 -- 172.21.15.16:0/897862557 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e0c010690 con 0x7f8e10100b10 2024-09-12T02:01:21.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:21.205+0000 7f8e17640640 1 -- 172.21.15.16:0/897862557 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e101a0290 con 0x7f8e10100b10 2024-09-12T02:01:21.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:21.206+0000 7f8e17640640 1 -- 172.21.15.16:0/897862557 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e10107f00 con 0x7f8e10100b10 2024-09-12T02:01:21.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:21.208+0000 7f8e067fc640 1 -- 172.21.15.16:0/897862557 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f8e0c0037a0 con 0x7f8e10100b10 2024-09-12T02:01:21.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:21.208+0000 7f8e067fc640 1 --2- 172.21.15.16:0/897862557 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f8dfc0a2c40 0x7f8dfc0a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:21.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:21.208+0000 7f8e14bdb640 1 --2- 172.21.15.16:0/897862557 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f8dfc0a2c40 0x7f8dfc0a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:21.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:21.208+0000 7f8e067fc640 1 -- 172.21.15.16:0/897862557 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f8e0c0cf570 con 0x7f8e10100b10 2024-09-12T02:01:21.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:21.209+0000 7f8e14bdb640 1 --2- 172.21.15.16:0/897862557 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f8dfc0a2c40 0x7f8dfc0a5100 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f8e00000c00 tx=0x7f8e000095f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:01:21.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:21.211+0000 7f8e067fc640 1 -- 172.21.15.16:0/897862557 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8e0c09c500 con 0x7f8e10100b10 2024-09-12T02:01:21.418 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:01:21 smithi033 ceph-mon[35332]: from='client.15039 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:01:21.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:21.422+0000 7f8e17640640 1 -- 172.21.15.16:0/897862557 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f8e1002d9d0 con 0x7f8e10100b10 2024-09-12T02:01:21.422 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:01:21 smithi016 ceph-mon[30947]: from='client.15039 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:01:21.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:21.423+0000 7f8e067fc640 1 -- 172.21.15.16:0/897862557 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 24 v24) ==== 76+0+3086 (secure 0 0 0) 0x7f8e0c09be00 con 0x7f8e10100b10 2024-09-12T02:01:21.423 INFO:teuthology.orchestra.run.smithi016.stdout:e24 2024-09-12T02:01:21.423 INFO:teuthology.orchestra.run.smithi016.stdout:btime 2024-09-12T02:00:07:547152+0000 2024-09-12T02:01:21.423 INFO:teuthology.orchestra.run.smithi016.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-12T02:01:21.423 INFO:teuthology.orchestra.run.smithi016.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-12T02:01:21.423 INFO:teuthology.orchestra.run.smithi016.stdout:legacy client fscid: 1 2024-09-12T02:01:21.424 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T02:01:21.424 INFO:teuthology.orchestra.run.smithi016.stdout:Filesystem 'cephfs' (1) 2024-09-12T02:01:21.424 INFO:teuthology.orchestra.run.smithi016.stdout:fs_name cephfs 2024-09-12T02:01:21.424 INFO:teuthology.orchestra.run.smithi016.stdout:epoch 24 2024-09-12T02:01:21.424 INFO:teuthology.orchestra.run.smithi016.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-09-12T02:01:21.424 INFO:teuthology.orchestra.run.smithi016.stdout:created 2024-09-12T01:59:56.117608+0000 2024-09-12T02:01:21.424 INFO:teuthology.orchestra.run.smithi016.stdout:modified 2024-09-12T02:00:07.097363+0000 2024-09-12T02:01:21.424 INFO:teuthology.orchestra.run.smithi016.stdout:tableserver 0 2024-09-12T02:01:21.424 INFO:teuthology.orchestra.run.smithi016.stdout:root 0 2024-09-12T02:01:21.424 INFO:teuthology.orchestra.run.smithi016.stdout:session_timeout 300 2024-09-12T02:01:21.424 INFO:teuthology.orchestra.run.smithi016.stdout:session_autoclose 300 2024-09-12T02:01:21.424 INFO:teuthology.orchestra.run.smithi016.stdout:max_file_size 1099511627776 2024-09-12T02:01:21.424 INFO:teuthology.orchestra.run.smithi016.stdout:max_xattr_size 65536 2024-09-12T02:01:21.425 INFO:teuthology.orchestra.run.smithi016.stdout:required_client_features {} 2024-09-12T02:01:21.425 INFO:teuthology.orchestra.run.smithi016.stdout:last_failure 0 2024-09-12T02:01:21.425 INFO:teuthology.orchestra.run.smithi016.stdout:last_failure_osd_epoch 0 2024-09-12T02:01:21.425 INFO:teuthology.orchestra.run.smithi016.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-12T02:01:21.425 INFO:teuthology.orchestra.run.smithi016.stdout:max_mds 3 2024-09-12T02:01:21.425 INFO:teuthology.orchestra.run.smithi016.stdout:in 0,1,2 2024-09-12T02:01:21.425 INFO:teuthology.orchestra.run.smithi016.stdout:up {0=24482,1=24481,2=24469} 2024-09-12T02:01:21.425 INFO:teuthology.orchestra.run.smithi016.stdout:failed 2024-09-12T02:01:21.425 INFO:teuthology.orchestra.run.smithi016.stdout:damaged 2024-09-12T02:01:21.425 INFO:teuthology.orchestra.run.smithi016.stdout:stopped 2024-09-12T02:01:21.425 INFO:teuthology.orchestra.run.smithi016.stdout:data_pools [3] 2024-09-12T02:01:21.425 INFO:teuthology.orchestra.run.smithi016.stdout:metadata_pool 2 2024-09-12T02:01:21.425 INFO:teuthology.orchestra.run.smithi016.stdout:inline_data disabled 2024-09-12T02:01:21.426 INFO:teuthology.orchestra.run.smithi016.stdout:balancer 2024-09-12T02:01:21.426 INFO:teuthology.orchestra.run.smithi016.stdout:bal_rank_mask -1 2024-09-12T02:01:21.426 INFO:teuthology.orchestra.run.smithi016.stdout:standby_count_wanted 0 2024-09-12T02:01:21.426 INFO:teuthology.orchestra.run.smithi016.stdout:qdb_cluster leader: 24482 members: 24469,24482,24481 2024-09-12T02:01:21.426 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.b{0:24482} state up:active seq 4 addr [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T02:01:21.426 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.f{0:24460} state up:standby-replay seq 1 addr [v2:172.21.15.184:6834/1457679686,v1:172.21.15.184:6835/1457679686] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T02:01:21.426 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.i{1:24481} state up:active seq 8 addr [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T02:01:21.426 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.h{1:24452} state up:standby-replay seq 1 addr [v2:172.21.15.33:6836/1892833868,v1:172.21.15.33:6837/1892833868] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T02:01:21.426 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.c{2:24469} state up:active seq 9 addr [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T02:01:21.426 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.l{2:24439} state up:standby-replay seq 1 addr [v2:172.21.15.184:6832/1563374106,v1:172.21.15.184:6833/1563374106] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T02:01:21.426 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T02:01:21.426 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T02:01:21.426 INFO:teuthology.orchestra.run.smithi016.stdout:Standby daemons: 2024-09-12T02:01:21.427 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T02:01:21.427 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.j{-1:14541} state up:standby seq 1 addr [v2:172.21.15.16:6834/3279123279,v1:172.21.15.16:6835/3279123279] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T02:01:21.427 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.d{-1:14553} state up:standby seq 1 addr [v2:172.21.15.16:6836/4119452382,v1:172.21.15.16:6837/4119452382] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T02:01:21.427 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.g{-1:14601} state up:standby seq 1 addr [v2:172.21.15.16:6838/3263823165,v1:172.21.15.16:6839/3263823165] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T02:01:21.427 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.a{-1:14607} state up:standby seq 1 addr [v2:172.21.15.16:6840/1510290732,v1:172.21.15.16:6841/1510290732] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T02:01:21.427 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.k{-1:24401} state up:standby seq 1 addr [v2:172.21.15.33:6832/279674603,v1:172.21.15.33:6833/279674603] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T02:01:21.427 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.e{-1:24425} state up:standby seq 1 addr [v2:172.21.15.33:6834/3555728028,v1:172.21.15.33:6835/3555728028] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T02:01:21.428 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 24 2024-09-12T02:01:21.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:21.425+0000 7f8e17640640 1 -- 172.21.15.16:0/897862557 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f8dfc0a2c40 msgr2=0x7f8dfc0a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:21.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:21.425+0000 7f8e17640640 1 --2- 172.21.15.16:0/897862557 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f8dfc0a2c40 0x7f8dfc0a5100 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f8e00000c00 tx=0x7f8e000095f0 comp rx=0 tx=0).stop 2024-09-12T02:01:21.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:21.425+0000 7f8e17640640 1 -- 172.21.15.16:0/897862557 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e10100b10 msgr2=0x7f8e10197f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:21.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:21.425+0000 7f8e17640640 1 --2- 172.21.15.16:0/897862557 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e10100b10 0x7f8e10197f10 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f8e0c00e9c0 tx=0x7f8e0c00ee80 comp rx=0 tx=0).stop 2024-09-12T02:01:21.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:21.426+0000 7f8e17640640 1 -- 172.21.15.16:0/897862557 shutdown_connections 2024-09-12T02:01:21.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:21.426+0000 7f8e17640640 1 --2- 172.21.15.16:0/897862557 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f8dfc0a2c40 0x7f8dfc0a5100 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:21.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:21.426+0000 7f8e17640640 1 --2- 172.21.15.16:0/897862557 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8e101026d0 0x7f8e1019f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:21.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:21.426+0000 7f8e17640640 1 --2- 172.21.15.16:0/897862557 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e10101d10 0x7f8e10198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:21.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:21.426+0000 7f8e17640640 1 --2- 172.21.15.16:0/897862557 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e10100b10 0x7f8e10197f10 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:21.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:21.426+0000 7f8e17640640 1 -- 172.21.15.16:0/897862557 >> 172.21.15.16:0/897862557 conn(0x7f8e100fc2a0 msgr2=0x7f8e101050d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:01:21.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:21.426+0000 7f8e17640640 1 -- 172.21.15.16:0/897862557 shutdown_connections 2024-09-12T02:01:21.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:21.426+0000 7f8e17640640 1 -- 172.21.15.16:0/897862557 wait complete. 2024-09-12T02:01:21.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:01:21 smithi184 ceph-mon[35593]: from='client.15039 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:01:22.010 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- bash -c 'ceph osd dump' 2024-09-12T02:01:22.266 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:01:22.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:01:22 smithi033 ceph-mon[35332]: pgmap v318: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-12T02:01:22.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:01:22 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/897862557' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-12T02:01:22.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:01:22 smithi016 ceph-mon[30947]: pgmap v318: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-12T02:01:22.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:01:22 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/897862557' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-12T02:01:22.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:01:22 smithi184 ceph-mon[35593]: pgmap v318: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-12T02:01:22.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:01:22 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/897862557' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-12T02:01:22.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:22.682+0000 7fa5206f1640 1 -- 172.21.15.16:0/2341729424 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa518101c00 msgr2=0x7fa518102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:22.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:22.682+0000 7fa5206f1640 1 --2- 172.21.15.16:0/2341729424 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa518101c00 0x7fa518102000 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fa514009900 tx=0x7fa51402a410 comp rx=0 tx=0).stop 2024-09-12T02:01:22.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:22.682+0000 7fa5206f1640 1 -- 172.21.15.16:0/2341729424 shutdown_connections 2024-09-12T02:01:22.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:22.682+0000 7fa5206f1640 1 --2- 172.21.15.16:0/2341729424 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa5181037c0 0x7fa51810a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:22.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:22.682+0000 7fa5206f1640 1 --2- 172.21.15.16:0/2341729424 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa518102e00 0x7fa518103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:22.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:22.682+0000 7fa5206f1640 1 --2- 172.21.15.16:0/2341729424 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa518101c00 0x7fa518102000 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:22.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:22.682+0000 7fa5206f1640 1 -- 172.21.15.16:0/2341729424 >> 172.21.15.16:0/2341729424 conn(0x7fa5180fd390 msgr2=0x7fa5180ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:01:22.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:22.683+0000 7fa5206f1640 1 -- 172.21.15.16:0/2341729424 shutdown_connections 2024-09-12T02:01:22.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:22.683+0000 7fa5206f1640 1 -- 172.21.15.16:0/2341729424 wait complete. 2024-09-12T02:01:22.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:22.683+0000 7fa5206f1640 1 Processor -- start 2024-09-12T02:01:22.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:22.683+0000 7fa5206f1640 1 -- start start 2024-09-12T02:01:22.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:22.683+0000 7fa5206f1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa518101c00 0x7fa518197e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:22.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:22.683+0000 7fa5206f1640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa518102e00 0x7fa5181983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:22.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:22.683+0000 7fa5206f1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa5181037c0 0x7fa51819f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:22.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:22.684+0000 7fa51e48d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa518101c00 0x7fa518197e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:22.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:22.684+0000 7fa5206f1640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa51810d860 con 0x7fa518101c00 2024-09-12T02:01:22.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:22.684+0000 7fa5206f1640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa51810d6e0 con 0x7fa5181037c0 2024-09-12T02:01:22.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:22.684+0000 7fa5206f1640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa51810d9e0 con 0x7fa518102e00 2024-09-12T02:01:22.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:22.684+0000 7fa51e48d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa518101c00 0x7fa518197e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42984/0 (socket says 172.21.15.16:42984) 2024-09-12T02:01:22.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:22.684+0000 7fa51e48d640 1 -- 172.21.15.16:0/819217003 learned_addr learned my addr 172.21.15.16:0/819217003 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:01:22.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:22.684+0000 7fa51dc8c640 1 --2- 172.21.15.16:0/819217003 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa518102e00 0x7fa5181983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:22.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:22.684+0000 7fa51ec8e640 1 --2- 172.21.15.16:0/819217003 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa5181037c0 0x7fa51819f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:22.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:22.684+0000 7fa51e48d640 1 -- 172.21.15.16:0/819217003 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa518102e00 msgr2=0x7fa5181983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:22.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:22.684+0000 7fa51e48d640 1 --2- 172.21.15.16:0/819217003 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa518102e00 0x7fa5181983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:22.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:22.684+0000 7fa51e48d640 1 -- 172.21.15.16:0/819217003 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa5181037c0 msgr2=0x7fa51819f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:22.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:22.684+0000 7fa51e48d640 1 --2- 172.21.15.16:0/819217003 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa5181037c0 0x7fa51819f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:22.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:22.684+0000 7fa51e48d640 1 -- 172.21.15.16:0/819217003 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa51819fab0 con 0x7fa518101c00 2024-09-12T02:01:22.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:22.684+0000 7fa51dc8c640 1 --2- 172.21.15.16:0/819217003 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa518102e00 0x7fa5181983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:01:22.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:22.684+0000 7fa51ec8e640 1 --2- 172.21.15.16:0/819217003 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa5181037c0 0x7fa51819f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:01:22.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:22.684+0000 7fa51e48d640 1 --2- 172.21.15.16:0/819217003 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa518101c00 0x7fa518197e80 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fa514031610 tx=0x7fa5140043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:01:22.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:22.684+0000 7fa50f7fe640 1 -- 172.21.15.16:0/819217003 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa51403f030 con 0x7fa518101c00 2024-09-12T02:01:22.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:22.684+0000 7fa50f7fe640 1 -- 172.21.15.16:0/819217003 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa514003ce0 con 0x7fa518101c00 2024-09-12T02:01:22.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:22.684+0000 7fa50f7fe640 1 -- 172.21.15.16:0/819217003 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa514035070 con 0x7fa518101c00 2024-09-12T02:01:22.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:22.684+0000 7fa5206f1640 1 -- 172.21.15.16:0/819217003 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa51819fd40 con 0x7fa518101c00 2024-09-12T02:01:22.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:22.685+0000 7fa5206f1640 1 -- 172.21.15.16:0/819217003 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5181a0170 con 0x7fa518101c00 2024-09-12T02:01:22.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:22.685+0000 7fa5206f1640 1 -- 172.21.15.16:0/819217003 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa518108fa0 con 0x7fa518101c00 2024-09-12T02:01:22.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:22.687+0000 7fa50f7fe640 1 -- 172.21.15.16:0/819217003 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fa514042030 con 0x7fa518101c00 2024-09-12T02:01:22.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:22.687+0000 7fa50f7fe640 1 --2- 172.21.15.16:0/819217003 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fa5040a2cc0 0x7fa5040a5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:22.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:22.687+0000 7fa51dc8c640 1 --2- 172.21.15.16:0/819217003 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fa5040a2cc0 0x7fa5040a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:22.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:22.687+0000 7fa50f7fe640 1 -- 172.21.15.16:0/819217003 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fa5140ef780 con 0x7fa518101c00 2024-09-12T02:01:22.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:22.688+0000 7fa51dc8c640 1 --2- 172.21.15.16:0/819217003 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fa5040a2cc0 0x7fa5040a5180 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fa5181993a0 tx=0x7fa508005d90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:01:22.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:22.691+0000 7fa50f7fe640 1 -- 172.21.15.16:0/819217003 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa5140bc680 con 0x7fa518101c00 2024-09-12T02:01:22.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:22.869+0000 7fa5206f1640 1 -- 172.21.15.16:0/819217003 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7fa518062f30 con 0x7fa518101c00 2024-09-12T02:01:22.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:22.870+0000 7fa50f7fe640 1 -- 172.21.15.16:0/819217003 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v82) ==== 56+0+5217 (secure 0 0 0) 0x7fa5140bbf80 con 0x7fa518101c00 2024-09-12T02:01:22.870 INFO:teuthology.orchestra.run.smithi016.stdout:epoch 82 2024-09-12T02:01:22.870 INFO:teuthology.orchestra.run.smithi016.stdout:fsid 879f3b56-70a9-11ef-bcea-c7b262605968 2024-09-12T02:01:22.870 INFO:teuthology.orchestra.run.smithi016.stdout:created 2024-09-12T01:52:12.841045+0000 2024-09-12T02:01:22.870 INFO:teuthology.orchestra.run.smithi016.stdout:modified 2024-09-12T02:00:09.559511+0000 2024-09-12T02:01:22.870 INFO:teuthology.orchestra.run.smithi016.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-12T02:01:22.870 INFO:teuthology.orchestra.run.smithi016.stdout:crush_version 26 2024-09-12T02:01:22.870 INFO:teuthology.orchestra.run.smithi016.stdout:full_ratio 0.95 2024-09-12T02:01:22.870 INFO:teuthology.orchestra.run.smithi016.stdout:backfillfull_ratio 0.9 2024-09-12T02:01:22.871 INFO:teuthology.orchestra.run.smithi016.stdout:nearfull_ratio 0.85 2024-09-12T02:01:22.871 INFO:teuthology.orchestra.run.smithi016.stdout:require_min_compat_client luminous 2024-09-12T02:01:22.871 INFO:teuthology.orchestra.run.smithi016.stdout:min_compat_client luminous 2024-09-12T02:01:22.871 INFO:teuthology.orchestra.run.smithi016.stdout:require_osd_release squid 2024-09-12T02:01:22.871 INFO:teuthology.orchestra.run.smithi016.stdout:stretch_mode_enabled false 2024-09-12T02:01:22.871 INFO:teuthology.orchestra.run.smithi016.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-09-12T02:01:22.871 INFO:teuthology.orchestra.run.smithi016.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 80 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-09-12T02:01:22.871 INFO:teuthology.orchestra.run.smithi016.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 80 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-09-12T02:01:22.871 INFO:teuthology.orchestra.run.smithi016.stdout:max_osd 12 2024-09-12T02:01:22.871 INFO:teuthology.orchestra.run.smithi016.stdout:osd.0 up in weight 1 up_from 8 up_thru 78 down_at 0 last_clean_interval [0,0) [v2:172.21.15.16:6802/4000187867,v1:172.21.15.16:6803/4000187867] [v2:172.21.15.16:6804/4000187867,v1:172.21.15.16:6805/4000187867] exists,up 3ea41eb6-e384-4ddc-810f-0f103c127849 2024-09-12T02:01:22.871 INFO:teuthology.orchestra.run.smithi016.stdout:osd.1 up in weight 1 up_from 13 up_thru 78 down_at 0 last_clean_interval [0,0) [v2:172.21.15.33:6800/3862661148,v1:172.21.15.33:6801/3862661148] [v2:172.21.15.33:6802/3862661148,v1:172.21.15.33:6803/3862661148] exists,up 07bf63e7-70fa-41b2-af53-12a5fec3f33c 2024-09-12T02:01:22.871 INFO:teuthology.orchestra.run.smithi016.stdout:osd.2 up in weight 1 up_from 17 up_thru 78 down_at 0 last_clean_interval [0,0) [v2:172.21.15.184:6800/3958753963,v1:172.21.15.184:6801/3958753963] [v2:172.21.15.184:6802/3958753963,v1:172.21.15.184:6803/3958753963] exists,up 22e40cac-3bc0-4cce-b47b-77499b767f7e 2024-09-12T02:01:22.871 INFO:teuthology.orchestra.run.smithi016.stdout:osd.3 up in weight 1 up_from 24 up_thru 78 down_at 0 last_clean_interval [0,0) [v2:172.21.15.16:6810/3395182524,v1:172.21.15.16:6811/3395182524] [v2:172.21.15.16:6812/3395182524,v1:172.21.15.16:6813/3395182524] exists,up 834a07f5-2c24-49e1-8e4a-12c96cfbdc65 2024-09-12T02:01:22.871 INFO:teuthology.orchestra.run.smithi016.stdout:osd.4 up in weight 1 up_from 31 up_thru 78 down_at 0 last_clean_interval [0,0) [v2:172.21.15.33:6808/1355184322,v1:172.21.15.33:6809/1355184322] [v2:172.21.15.33:6810/1355184322,v1:172.21.15.33:6811/1355184322] exists,up 3ca56e09-c194-4404-b9bd-2524d38367eb 2024-09-12T02:01:22.872 INFO:teuthology.orchestra.run.smithi016.stdout:osd.5 up in weight 1 up_from 36 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.184:6808/1533397276,v1:172.21.15.184:6809/1533397276] [v2:172.21.15.184:6810/1533397276,v1:172.21.15.184:6811/1533397276] exists,up 0b05ecdb-b987-461f-951c-6d76739bfb4c 2024-09-12T02:01:22.872 INFO:teuthology.orchestra.run.smithi016.stdout:osd.6 up in weight 1 up_from 43 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.16:6818/902466142,v1:172.21.15.16:6819/902466142] [v2:172.21.15.16:6820/902466142,v1:172.21.15.16:6821/902466142] exists,up b3936e21-e197-49fa-8441-d909b258b612 2024-09-12T02:01:22.872 INFO:teuthology.orchestra.run.smithi016.stdout:osd.7 up in weight 1 up_from 48 up_thru 78 down_at 0 last_clean_interval [0,0) [v2:172.21.15.33:6816/4186032630,v1:172.21.15.33:6817/4186032630] [v2:172.21.15.33:6818/4186032630,v1:172.21.15.33:6819/4186032630] exists,up 4ddbe011-34c1-4b3f-a8eb-577422fb5426 2024-09-12T02:01:22.872 INFO:teuthology.orchestra.run.smithi016.stdout:osd.8 up in weight 1 up_from 55 up_thru 78 down_at 0 last_clean_interval [0,0) [v2:172.21.15.184:6816/781851781,v1:172.21.15.184:6817/781851781] [v2:172.21.15.184:6818/781851781,v1:172.21.15.184:6819/781851781] exists,up dca5bfb8-ac6a-47ce-905c-be7fcaf0b5ec 2024-09-12T02:01:22.872 INFO:teuthology.orchestra.run.smithi016.stdout:osd.9 up in weight 1 up_from 62 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.16:6826/1569578746,v1:172.21.15.16:6827/1569578746] [v2:172.21.15.16:6828/1569578746,v1:172.21.15.16:6829/1569578746] exists,up 23a05cb5-01d5-40f3-b66a-15fcccedbd6c 2024-09-12T02:01:22.872 INFO:teuthology.orchestra.run.smithi016.stdout:osd.10 up in weight 1 up_from 67 up_thru 78 down_at 0 last_clean_interval [0,0) [v2:172.21.15.33:6824/469977847,v1:172.21.15.33:6825/469977847] [v2:172.21.15.33:6826/469977847,v1:172.21.15.33:6827/469977847] exists,up e4bd6bce-3c69-44db-bc09-d264a0ad094a 2024-09-12T02:01:22.872 INFO:teuthology.orchestra.run.smithi016.stdout:osd.11 up in weight 1 up_from 74 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.184:6824/1619871346,v1:172.21.15.184:6825/1619871346] [v2:172.21.15.184:6826/1619871346,v1:172.21.15.184:6827/1619871346] exists,up d60212f8-5a1d-45f7-9a1b-12294bb4c80d 2024-09-12T02:01:22.872 INFO:teuthology.orchestra.run.smithi016.stdout:pg_upmap_items 2.4 [2,11] 2024-09-12T02:01:22.872 INFO:teuthology.orchestra.run.smithi016.stdout:pg_upmap_items 2.10 [8,11] 2024-09-12T02:01:22.872 INFO:teuthology.orchestra.run.smithi016.stdout:pg_upmap_items 3.0 [5,9] 2024-09-12T02:01:22.872 INFO:teuthology.orchestra.run.smithi016.stdout:pg_upmap_items 3.1 [4,9] 2024-09-12T02:01:22.872 INFO:teuthology.orchestra.run.smithi016.stdout:pg_upmap_items 3.1d [4,8] 2024-09-12T02:01:22.872 INFO:teuthology.orchestra.run.smithi016.stdout:pg_upmap_items 3.24 [4,9] 2024-09-12T02:01:22.873 INFO:teuthology.orchestra.run.smithi016.stdout:blocklist 172.21.15.16:0/3532565698 expires 2024-09-13T01:53:08.218475+0000 2024-09-12T02:01:22.873 INFO:teuthology.orchestra.run.smithi016.stdout:blocklist 172.21.15.16:0/1822157179 expires 2024-09-13T01:52:44.194152+0000 2024-09-12T02:01:22.873 INFO:teuthology.orchestra.run.smithi016.stdout:blocklist 172.21.15.16:0/3919664061 expires 2024-09-13T01:52:44.194152+0000 2024-09-12T02:01:22.873 INFO:teuthology.orchestra.run.smithi016.stdout:blocklist 172.21.15.16:0/2269394774 expires 2024-09-13T01:52:44.194152+0000 2024-09-12T02:01:22.874 INFO:teuthology.orchestra.run.smithi016.stdout:blocklist 172.21.15.16:0/2595308634 expires 2024-09-13T01:53:08.218475+0000 2024-09-12T02:01:22.874 INFO:teuthology.orchestra.run.smithi016.stdout:blocklist 172.21.15.16:6801/2754331153 expires 2024-09-13T01:52:44.194152+0000 2024-09-12T02:01:22.874 INFO:teuthology.orchestra.run.smithi016.stdout:blocklist 172.21.15.16:6801/2615861334 expires 2024-09-13T01:53:08.218475+0000 2024-09-12T02:01:22.874 INFO:teuthology.orchestra.run.smithi016.stdout:blocklist 172.21.15.16:6800/2615861334 expires 2024-09-13T01:53:08.218475+0000 2024-09-12T02:01:22.874 INFO:teuthology.orchestra.run.smithi016.stdout:blocklist 172.21.15.16:0/4199114771 expires 2024-09-13T01:53:08.218475+0000 2024-09-12T02:01:22.874 INFO:teuthology.orchestra.run.smithi016.stdout:blocklist 172.21.15.16:6800/2754331153 expires 2024-09-13T01:52:44.194152+0000 2024-09-12T02:01:22.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:22.872+0000 7fa5206f1640 1 -- 172.21.15.16:0/819217003 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fa5040a2cc0 msgr2=0x7fa5040a5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:22.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:22.872+0000 7fa5206f1640 1 --2- 172.21.15.16:0/819217003 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fa5040a2cc0 0x7fa5040a5180 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fa5181993a0 tx=0x7fa508005d90 comp rx=0 tx=0).stop 2024-09-12T02:01:22.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:22.872+0000 7fa5206f1640 1 -- 172.21.15.16:0/819217003 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa518101c00 msgr2=0x7fa518197e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:22.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:22.872+0000 7fa5206f1640 1 --2- 172.21.15.16:0/819217003 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa518101c00 0x7fa518197e80 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fa514031610 tx=0x7fa5140043b0 comp rx=0 tx=0).stop 2024-09-12T02:01:22.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:22.872+0000 7fa5206f1640 1 -- 172.21.15.16:0/819217003 shutdown_connections 2024-09-12T02:01:22.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:22.872+0000 7fa5206f1640 1 --2- 172.21.15.16:0/819217003 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fa5040a2cc0 0x7fa5040a5180 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:22.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:22.872+0000 7fa5206f1640 1 --2- 172.21.15.16:0/819217003 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa5181037c0 0x7fa51819f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:22.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:22.872+0000 7fa5206f1640 1 --2- 172.21.15.16:0/819217003 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa518102e00 0x7fa5181983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:22.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:22.872+0000 7fa5206f1640 1 --2- 172.21.15.16:0/819217003 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa518101c00 0x7fa518197e80 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:22.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:22.872+0000 7fa5206f1640 1 -- 172.21.15.16:0/819217003 >> 172.21.15.16:0/819217003 conn(0x7fa5180fd390 msgr2=0x7fa518106170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:01:22.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:22.872+0000 7fa5206f1640 1 -- 172.21.15.16:0/819217003 shutdown_connections 2024-09-12T02:01:22.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:22.872+0000 7fa5206f1640 1 -- 172.21.15.16:0/819217003 wait complete. 2024-09-12T02:01:23.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:01:23 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/819217003' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-12T02:01:23.556 INFO:teuthology.run_tasks:Running task fs.ready... 2024-09-12T02:01:23.569 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:01:23.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:01:23 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/819217003' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-12T02:01:23.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:01:23 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/819217003' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-12T02:01:23.831 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:01:24.183 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:01:24 smithi016 ceph-mon[30947]: pgmap v319: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-12T02:01:24.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:24.247+0000 7f2cb88ad640 1 -- 172.21.15.16:0/3121625968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cb4102570 msgr2=0x7f2cb41029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:24.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:24.247+0000 7f2cb88ad640 1 --2- 172.21.15.16:0/3121625968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cb4102570 0x7f2cb41029b0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c009900 tx=0x7f2c9c02a480 comp rx=0 tx=0).stop 2024-09-12T02:01:24.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:24.247+0000 7f2cb88ad640 1 -- 172.21.15.16:0/3121625968 shutdown_connections 2024-09-12T02:01:24.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:24.247+0000 7f2cb88ad640 1 --2- 172.21.15.16:0/3121625968 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2cb4102ef0 0x7f2cb410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:24.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:24.247+0000 7f2cb88ad640 1 --2- 172.21.15.16:0/3121625968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cb4102570 0x7f2cb41029b0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:24.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:24.247+0000 7f2cb88ad640 1 --2- 172.21.15.16:0/3121625968 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2cb4100e20 0x7f2cb4101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:24.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:24.247+0000 7f2cb88ad640 1 -- 172.21.15.16:0/3121625968 >> 172.21.15.16:0/3121625968 conn(0x7f2cb40fc7d0 msgr2=0x7f2cb40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:01:24.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:24.247+0000 7f2cb88ad640 1 -- 172.21.15.16:0/3121625968 shutdown_connections 2024-09-12T02:01:24.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:24.247+0000 7f2cb88ad640 1 -- 172.21.15.16:0/3121625968 wait complete. 2024-09-12T02:01:24.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:24.248+0000 7f2cb88ad640 1 Processor -- start 2024-09-12T02:01:24.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:24.248+0000 7f2cb88ad640 1 -- start start 2024-09-12T02:01:24.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:24.248+0000 7f2cb88ad640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2cb4100e20 0x7f2cb41984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:24.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:24.248+0000 7f2cb88ad640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2cb4102570 0x7f2cb41989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:24.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:24.248+0000 7f2cb88ad640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cb4102ef0 0x7f2cb419cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:24.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:24.248+0000 7f2cb88ad640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2cb4110f80 con 0x7f2cb4102ef0 2024-09-12T02:01:24.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:24.248+0000 7f2cb88ad640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2cb4110e00 con 0x7f2cb4100e20 2024-09-12T02:01:24.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:24.248+0000 7f2cb88ad640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f2cb4111100 con 0x7f2cb4102570 2024-09-12T02:01:24.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:24.248+0000 7f2cb2d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cb4102ef0 0x7f2cb419cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:24.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:24.248+0000 7f2cb2d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cb4102ef0 0x7f2cb419cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43002/0 (socket says 172.21.15.16:43002) 2024-09-12T02:01:24.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:24.248+0000 7f2cb2d9d640 1 -- 172.21.15.16:0/4271221826 learned_addr learned my addr 172.21.15.16:0/4271221826 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:01:24.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:24.248+0000 7f2cb259c640 1 --2- 172.21.15.16:0/4271221826 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2cb4100e20 0x7f2cb41984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:24.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:24.248+0000 7f2cabfff640 1 --2- 172.21.15.16:0/4271221826 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2cb4102570 0x7f2cb41989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:24.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:24.248+0000 7f2cb2d9d640 1 -- 172.21.15.16:0/4271221826 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2cb4102570 msgr2=0x7f2cb41989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:24.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:24.248+0000 7f2cb2d9d640 1 --2- 172.21.15.16:0/4271221826 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2cb4102570 0x7f2cb41989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:24.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:24.248+0000 7f2cb2d9d640 1 -- 172.21.15.16:0/4271221826 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2cb4100e20 msgr2=0x7f2cb41984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:24.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:24.248+0000 7f2cb2d9d640 1 --2- 172.21.15.16:0/4271221826 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2cb4100e20 0x7f2cb41984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:24.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:24.248+0000 7f2cb2d9d640 1 -- 172.21.15.16:0/4271221826 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cb419d460 con 0x7f2cb4102ef0 2024-09-12T02:01:24.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:24.249+0000 7f2cb2d9d640 1 --2- 172.21.15.16:0/4271221826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cb4102ef0 0x7f2cb419cd80 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f2cac00eaa0 tx=0x7f2cac00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:01:24.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:24.249+0000 7f2cab7fe640 1 -- 172.21.15.16:0/4271221826 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cac00cdb0 con 0x7f2cb4102ef0 2024-09-12T02:01:24.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:24.249+0000 7f2cb88ad640 1 -- 172.21.15.16:0/4271221826 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cb419d750 con 0x7f2cb4102ef0 2024-09-12T02:01:24.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:24.249+0000 7f2cab7fe640 1 -- 172.21.15.16:0/4271221826 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2cac005020 con 0x7f2cb4102ef0 2024-09-12T02:01:24.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:24.249+0000 7f2cab7fe640 1 -- 172.21.15.16:0/4271221826 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cac0116b0 con 0x7f2cb4102ef0 2024-09-12T02:01:24.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:24.249+0000 7f2cb88ad640 1 -- 172.21.15.16:0/4271221826 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cb41a5030 con 0x7f2cb4102ef0 2024-09-12T02:01:24.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:24.250+0000 7f2cb88ad640 1 -- 172.21.15.16:0/4271221826 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2cb4101200 con 0x7f2cb4102ef0 2024-09-12T02:01:24.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:24.251+0000 7f2cab7fe640 1 -- 172.21.15.16:0/4271221826 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f2cac005560 con 0x7f2cb4102ef0 2024-09-12T02:01:24.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:24.252+0000 7f2cab7fe640 1 --2- 172.21.15.16:0/4271221826 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f2ca00a2c10 0x7f2ca00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:24.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:24.252+0000 7f2cb259c640 1 --2- 172.21.15.16:0/4271221826 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f2ca00a2c10 0x7f2ca00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:24.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:24.252+0000 7f2cab7fe640 1 -- 172.21.15.16:0/4271221826 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f2cac0cf450 con 0x7f2cb4102ef0 2024-09-12T02:01:24.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:24.252+0000 7f2cb259c640 1 --2- 172.21.15.16:0/4271221826 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f2ca00a2c10 0x7f2ca00a50d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f2ca40058c0 tx=0x7f2ca400a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:01:24.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:24.256+0000 7f2cab7fe640 1 -- 172.21.15.16:0/4271221826 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2cac016030 con 0x7f2cb4102ef0 2024-09-12T02:01:24.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:24.469+0000 7f2cb88ad640 1 -- 172.21.15.16:0/4271221826 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2cb410a0a0 con 0x7f2cb4102ef0 2024-09-12T02:01:24.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:24.470+0000 7f2cab7fe640 1 -- 172.21.15.16:0/4271221826 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11911 (secure 0 0 0) 0x7f2cac09c250 con 0x7f2cb4102ef0 2024-09-12T02:01:24.471 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T02:01:24.471 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":24,"btime":"2024-09-12T02:00:07:547152+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3279123279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3279123279},{"type":"v1","addr":"172.21.15.16:6835","nonce":3279123279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4119452382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4119452382},{"type":"v1","addr":"172.21.15.16:6837","nonce":4119452382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/3263823165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3263823165},{"type":"v1","addr":"172.21.15.16:6839","nonce":3263823165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/1510290732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":1510290732},{"type":"v1","addr":"172.21.15.16:6841","nonce":1510290732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/279674603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":279674603},{"type":"v1","addr":"172.21.15.33:6833","nonce":279674603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3555728028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3555728028},{"type":"v1","addr":"172.21.15.33:6835","nonce":3555728028}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T01:59:56.117608+0000","modified":"2024-09-12T02:00:07.097363+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24482,"mds_1":24481,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6833/1563374106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1563374106},{"type":"v1","addr":"172.21.15.184:6833","nonce":1563374106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6837/1892833868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1892833868},{"type":"v1","addr":"172.21.15.33:6837","nonce":1892833868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/1457679686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1457679686},{"type":"v1","addr":"172.21.15.184:6835","nonce":1457679686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.184:6837/1793393087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":1793393087},{"type":"v1","addr":"172.21.15.184:6837","nonce":1793393087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.184:6839/4203347941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":4203347941},{"type":"v1","addr":"172.21.15.184:6839","nonce":4203347941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.33:6839/4136670204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":4136670204},{"type":"v1","addr":"172.21.15.33:6839","nonce":4136670204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24469,24482,24481]},"id":1}]} 2024-09-12T02:01:24.473 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 24 2024-09-12T02:01:24.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:24.472+0000 7f2cb88ad640 1 -- 172.21.15.16:0/4271221826 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f2ca00a2c10 msgr2=0x7f2ca00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:24.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:24.472+0000 7f2cb88ad640 1 --2- 172.21.15.16:0/4271221826 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f2ca00a2c10 0x7f2ca00a50d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f2ca40058c0 tx=0x7f2ca400a2d0 comp rx=0 tx=0).stop 2024-09-12T02:01:24.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:24.472+0000 7f2cb88ad640 1 -- 172.21.15.16:0/4271221826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cb4102ef0 msgr2=0x7f2cb419cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:24.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:24.472+0000 7f2cb88ad640 1 --2- 172.21.15.16:0/4271221826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cb4102ef0 0x7f2cb419cd80 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f2cac00eaa0 tx=0x7f2cac00ef60 comp rx=0 tx=0).stop 2024-09-12T02:01:24.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:24.472+0000 7f2cb88ad640 1 -- 172.21.15.16:0/4271221826 shutdown_connections 2024-09-12T02:01:24.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:24.472+0000 7f2cb88ad640 1 --2- 172.21.15.16:0/4271221826 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f2ca00a2c10 0x7f2ca00a50d0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:24.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:24.472+0000 7f2cb88ad640 1 --2- 172.21.15.16:0/4271221826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cb4102ef0 0x7f2cb419cd80 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:24.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:24.472+0000 7f2cb88ad640 1 --2- 172.21.15.16:0/4271221826 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2cb4102570 0x7f2cb41989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:24.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:24.472+0000 7f2cb88ad640 1 --2- 172.21.15.16:0/4271221826 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2cb4100e20 0x7f2cb41984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:24.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:24.472+0000 7f2cb88ad640 1 -- 172.21.15.16:0/4271221826 >> 172.21.15.16:0/4271221826 conn(0x7f2cb40fc7d0 msgr2=0x7f2cb406a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:01:24.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:24.472+0000 7f2cb88ad640 1 -- 172.21.15.16:0/4271221826 shutdown_connections 2024-09-12T02:01:24.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:24.472+0000 7f2cb88ad640 1 -- 172.21.15.16:0/4271221826 wait complete. 2024-09-12T02:01:24.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:01:24 smithi184 ceph-mon[35593]: pgmap v319: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-12T02:01:24.668 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:01:24 smithi033 ceph-mon[35332]: pgmap v319: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-12T02:01:25.115 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:01:25.374 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:01:25.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:01:25 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/4271221826' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:01:25.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:01:25 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/4271221826' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:01:25.668 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:01:25 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/4271221826' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:01:25.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:25.862+0000 7fae9bee6640 1 -- 172.21.15.16:0/12428530 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae94101de0 msgr2=0x7fae941021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:25.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:25.862+0000 7fae9bee6640 1 --2- 172.21.15.16:0/12428530 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae94101de0 0x7fae941021c0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fae88009900 tx=0x7fae8802a410 comp rx=0 tx=0).stop 2024-09-12T02:01:25.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:25.862+0000 7fae9bee6640 1 -- 172.21.15.16:0/12428530 shutdown_connections 2024-09-12T02:01:25.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:25.862+0000 7fae9bee6640 1 --2- 172.21.15.16:0/12428530 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fae94106f10 0x7fae9410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:25.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:25.862+0000 7fae9bee6640 1 --2- 172.21.15.16:0/12428530 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fae94102790 0x7fae941067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:25.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:25.862+0000 7fae9bee6640 1 --2- 172.21.15.16:0/12428530 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae94101de0 0x7fae941021c0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:25.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:25.862+0000 7fae9bee6640 1 -- 172.21.15.16:0/12428530 >> 172.21.15.16:0/12428530 conn(0x7fae940fd8c0 msgr2=0x7fae940ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:01:25.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:25.862+0000 7fae9bee6640 1 -- 172.21.15.16:0/12428530 shutdown_connections 2024-09-12T02:01:25.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:25.862+0000 7fae9bee6640 1 -- 172.21.15.16:0/12428530 wait complete. 2024-09-12T02:01:25.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:25.863+0000 7fae9bee6640 1 Processor -- start 2024-09-12T02:01:25.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:25.863+0000 7fae9bee6640 1 -- start start 2024-09-12T02:01:25.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:25.863+0000 7fae9bee6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae94101de0 0x7fae94198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:25.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:25.863+0000 7fae9bee6640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fae94102790 0x7fae94198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:25.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:25.863+0000 7fae9bee6640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fae94106f10 0x7fae9419cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:25.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:25.863+0000 7fae9bee6640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fae94111f70 con 0x7fae94101de0 2024-09-12T02:01:25.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:25.863+0000 7fae9bee6640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fae94111df0 con 0x7fae94106f10 2024-09-12T02:01:25.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:25.863+0000 7fae9bee6640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fae941120f0 con 0x7fae94102790 2024-09-12T02:01:25.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:25.863+0000 7fae99c82640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae94101de0 0x7fae94198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:25.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:25.863+0000 7fae99c82640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae94101de0 0x7fae94198450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43014/0 (socket says 172.21.15.16:43014) 2024-09-12T02:01:25.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:25.863+0000 7fae99c82640 1 -- 172.21.15.16:0/1941003084 learned_addr learned my addr 172.21.15.16:0/1941003084 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:01:25.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:25.863+0000 7fae99481640 1 --2- 172.21.15.16:0/1941003084 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fae94102790 0x7fae94198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:25.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:25.863+0000 7fae9a483640 1 --2- 172.21.15.16:0/1941003084 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fae94106f10 0x7fae9419cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:25.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:25.863+0000 7fae99c82640 1 -- 172.21.15.16:0/1941003084 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fae94102790 msgr2=0x7fae94198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:25.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:25.863+0000 7fae99c82640 1 --2- 172.21.15.16:0/1941003084 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fae94102790 0x7fae94198990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:25.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:25.863+0000 7fae99c82640 1 -- 172.21.15.16:0/1941003084 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fae94106f10 msgr2=0x7fae9419cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:25.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:25.863+0000 7fae99c82640 1 --2- 172.21.15.16:0/1941003084 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fae94106f10 0x7fae9419cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:25.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:25.863+0000 7fae99c82640 1 -- 172.21.15.16:0/1941003084 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae9419d400 con 0x7fae94101de0 2024-09-12T02:01:25.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:25.864+0000 7fae99481640 1 --2- 172.21.15.16:0/1941003084 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fae94102790 0x7fae94198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:01:25.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:25.864+0000 7fae9a483640 1 --2- 172.21.15.16:0/1941003084 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fae94106f10 0x7fae9419cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:01:25.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:25.864+0000 7fae99c82640 1 --2- 172.21.15.16:0/1941003084 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae94101de0 0x7fae94198450 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fae88006010 tx=0x7fae880042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:01:25.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:25.864+0000 7fae86ffd640 1 -- 172.21.15.16:0/1941003084 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae8803f030 con 0x7fae94101de0 2024-09-12T02:01:25.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:25.864+0000 7fae9bee6640 1 -- 172.21.15.16:0/1941003084 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae9419d690 con 0x7fae94101de0 2024-09-12T02:01:25.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:25.864+0000 7fae86ffd640 1 -- 172.21.15.16:0/1941003084 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae88003ce0 con 0x7fae94101de0 2024-09-12T02:01:25.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:25.864+0000 7fae86ffd640 1 -- 172.21.15.16:0/1941003084 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae88035070 con 0x7fae94101de0 2024-09-12T02:01:25.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:25.864+0000 7fae9bee6640 1 -- 172.21.15.16:0/1941003084 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae941a4fd0 con 0x7fae94101de0 2024-09-12T02:01:25.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:25.865+0000 7fae9bee6640 1 -- 172.21.15.16:0/1941003084 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae941032c0 con 0x7fae94101de0 2024-09-12T02:01:25.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:25.866+0000 7fae86ffd640 1 -- 172.21.15.16:0/1941003084 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fae88042030 con 0x7fae94101de0 2024-09-12T02:01:25.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:25.867+0000 7fae86ffd640 1 --2- 172.21.15.16:0/1941003084 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fae800a2c10 0x7fae800a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:25.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:25.867+0000 7fae99481640 1 --2- 172.21.15.16:0/1941003084 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fae800a2c10 0x7fae800a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:25.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:25.867+0000 7fae86ffd640 1 -- 172.21.15.16:0/1941003084 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fae880ef790 con 0x7fae94101de0 2024-09-12T02:01:25.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:25.867+0000 7fae99481640 1 --2- 172.21.15.16:0/1941003084 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fae800a2c10 0x7fae800a50d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fae900058c0 tx=0x7fae90005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:01:25.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:25.871+0000 7fae86ffd640 1 -- 172.21.15.16:0/1941003084 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fae880bc650 con 0x7fae94101de0 2024-09-12T02:01:26.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:26.084+0000 7fae9bee6640 1 -- 172.21.15.16:0/1941003084 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fae9410a0f0 con 0x7fae94101de0 2024-09-12T02:01:26.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:26.084+0000 7fae86ffd640 1 -- 172.21.15.16:0/1941003084 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11911 (secure 0 0 0) 0x7fae880bbf50 con 0x7fae94101de0 2024-09-12T02:01:26.084 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T02:01:26.085 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":24,"btime":"2024-09-12T02:00:07:547152+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3279123279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3279123279},{"type":"v1","addr":"172.21.15.16:6835","nonce":3279123279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4119452382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4119452382},{"type":"v1","addr":"172.21.15.16:6837","nonce":4119452382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/3263823165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3263823165},{"type":"v1","addr":"172.21.15.16:6839","nonce":3263823165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/1510290732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":1510290732},{"type":"v1","addr":"172.21.15.16:6841","nonce":1510290732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/279674603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":279674603},{"type":"v1","addr":"172.21.15.33:6833","nonce":279674603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3555728028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3555728028},{"type":"v1","addr":"172.21.15.33:6835","nonce":3555728028}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T01:59:56.117608+0000","modified":"2024-09-12T02:00:07.097363+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24482,"mds_1":24481,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6833/1563374106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1563374106},{"type":"v1","addr":"172.21.15.184:6833","nonce":1563374106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6837/1892833868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1892833868},{"type":"v1","addr":"172.21.15.33:6837","nonce":1892833868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/1457679686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1457679686},{"type":"v1","addr":"172.21.15.184:6835","nonce":1457679686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.184:6837/1793393087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":1793393087},{"type":"v1","addr":"172.21.15.184:6837","nonce":1793393087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.184:6839/4203347941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":4203347941},{"type":"v1","addr":"172.21.15.184:6839","nonce":4203347941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.33:6839/4136670204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":4136670204},{"type":"v1","addr":"172.21.15.33:6839","nonce":4136670204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24469,24482,24481]},"id":1}]} 2024-09-12T02:01:26.085 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 24 2024-09-12T02:01:26.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:26.087+0000 7fae9bee6640 1 -- 172.21.15.16:0/1941003084 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fae800a2c10 msgr2=0x7fae800a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:26.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:26.087+0000 7fae9bee6640 1 --2- 172.21.15.16:0/1941003084 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fae800a2c10 0x7fae800a50d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fae900058c0 tx=0x7fae90005850 comp rx=0 tx=0).stop 2024-09-12T02:01:26.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:26.087+0000 7fae9bee6640 1 -- 172.21.15.16:0/1941003084 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae94101de0 msgr2=0x7fae94198450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:26.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:26.087+0000 7fae9bee6640 1 --2- 172.21.15.16:0/1941003084 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae94101de0 0x7fae94198450 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fae88006010 tx=0x7fae880042f0 comp rx=0 tx=0).stop 2024-09-12T02:01:26.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:26.087+0000 7fae9bee6640 1 -- 172.21.15.16:0/1941003084 shutdown_connections 2024-09-12T02:01:26.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:26.087+0000 7fae9bee6640 1 --2- 172.21.15.16:0/1941003084 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fae800a2c10 0x7fae800a50d0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:26.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:26.087+0000 7fae9bee6640 1 --2- 172.21.15.16:0/1941003084 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fae94106f10 0x7fae9419cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:26.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:26.087+0000 7fae9bee6640 1 --2- 172.21.15.16:0/1941003084 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fae94102790 0x7fae94198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:26.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:26.087+0000 7fae9bee6640 1 --2- 172.21.15.16:0/1941003084 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae94101de0 0x7fae94198450 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:26.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:26.087+0000 7fae9bee6640 1 -- 172.21.15.16:0/1941003084 >> 172.21.15.16:0/1941003084 conn(0x7fae940fd8c0 msgr2=0x7fae940fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:01:26.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:26.087+0000 7fae9bee6640 1 -- 172.21.15.16:0/1941003084 shutdown_connections 2024-09-12T02:01:26.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:26.088+0000 7fae9bee6640 1 -- 172.21.15.16:0/1941003084 wait complete. 2024-09-12T02:01:26.400 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:01:26 smithi016 ceph-mon[30947]: pgmap v320: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-12T02:01:26.400 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:01:26 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/1941003084' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:01:26.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:01:26 smithi184 ceph-mon[35593]: pgmap v320: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-12T02:01:26.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:01:26 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/1941003084' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:01:26.668 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:01:26 smithi033 ceph-mon[35332]: pgmap v320: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-12T02:01:26.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:01:26 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/1941003084' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:01:26.730 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T02:01:26.988 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:01:27.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:27.428+0000 7f1b105c0640 1 -- 172.21.15.16:0/2887782505 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b08102de0 msgr2=0x7f1b081031c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:27.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:27.428+0000 7f1b105c0640 1 --2- 172.21.15.16:0/2887782505 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b08102de0 0x7f1b081031c0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f1b04009900 tx=0x7f1b0402a480 comp rx=0 tx=0).stop 2024-09-12T02:01:27.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:27.428+0000 7f1b105c0640 1 -- 172.21.15.16:0/2887782505 shutdown_connections 2024-09-12T02:01:27.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:27.428+0000 7f1b105c0640 1 --2- 172.21.15.16:0/2887782505 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b0810c180 0x7f1b0810e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:27.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:27.428+0000 7f1b105c0640 1 --2- 172.21.15.16:0/2887782505 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1b08103700 0x7f1b0810bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:27.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:27.428+0000 7f1b105c0640 1 --2- 172.21.15.16:0/2887782505 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b08102de0 0x7f1b081031c0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:27.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:27.428+0000 7f1b105c0640 1 -- 172.21.15.16:0/2887782505 >> 172.21.15.16:0/2887782505 conn(0x7f1b080fc780 msgr2=0x7f1b080feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:01:27.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:27.428+0000 7f1b105c0640 1 -- 172.21.15.16:0/2887782505 shutdown_connections 2024-09-12T02:01:27.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:27.428+0000 7f1b105c0640 1 -- 172.21.15.16:0/2887782505 wait complete. 2024-09-12T02:01:27.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:27.428+0000 7f1b105c0640 1 Processor -- start 2024-09-12T02:01:27.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:27.428+0000 7f1b105c0640 1 -- start start 2024-09-12T02:01:27.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:27.429+0000 7f1b105c0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b08102de0 0x7f1b08198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:27.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:27.429+0000 7f1b105c0640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1b08103700 0x7f1b081989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:27.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:27.429+0000 7f1b105c0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b0810c180 0x7f1b0819cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:27.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:27.429+0000 7f1b105c0640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1b08111f50 con 0x7f1b08102de0 2024-09-12T02:01:27.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:27.429+0000 7f1b105c0640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1b08111dd0 con 0x7f1b0810c180 2024-09-12T02:01:27.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:27.429+0000 7f1b105c0640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1b081120d0 con 0x7f1b08103700 2024-09-12T02:01:27.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:27.429+0000 7f1b0e35c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b08102de0 0x7f1b08198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:27.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:27.429+0000 7f1b0e35c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b08102de0 0x7f1b08198470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43034/0 (socket says 172.21.15.16:43034) 2024-09-12T02:01:27.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:27.429+0000 7f1b0e35c640 1 -- 172.21.15.16:0/2260100460 learned_addr learned my addr 172.21.15.16:0/2260100460 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:01:27.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:27.429+0000 7f1b0eb5d640 1 --2- 172.21.15.16:0/2260100460 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b0810c180 0x7f1b0819cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:27.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:27.429+0000 7f1b0db5b640 1 --2- 172.21.15.16:0/2260100460 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1b08103700 0x7f1b081989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:27.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:27.429+0000 7f1b0e35c640 1 -- 172.21.15.16:0/2260100460 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1b08103700 msgr2=0x7f1b081989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:27.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:27.429+0000 7f1b0e35c640 1 --2- 172.21.15.16:0/2260100460 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1b08103700 0x7f1b081989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:27.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:27.429+0000 7f1b0e35c640 1 -- 172.21.15.16:0/2260100460 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b0810c180 msgr2=0x7f1b0819cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:27.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:27.429+0000 7f1b0e35c640 1 --2- 172.21.15.16:0/2260100460 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b0810c180 0x7f1b0819cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:27.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:27.429+0000 7f1b0e35c640 1 -- 172.21.15.16:0/2260100460 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b0819d420 con 0x7f1b08102de0 2024-09-12T02:01:27.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:27.429+0000 7f1b0db5b640 1 --2- 172.21.15.16:0/2260100460 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1b08103700 0x7f1b081989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:01:27.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:27.429+0000 7f1b0eb5d640 1 --2- 172.21.15.16:0/2260100460 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b0810c180 0x7f1b0819cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:01:27.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:27.429+0000 7f1b0e35c640 1 --2- 172.21.15.16:0/2260100460 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b08102de0 0x7f1b08198470 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f1b04006010 tx=0x7f1b040042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:01:27.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:27.429+0000 7f1aff7fe640 1 -- 172.21.15.16:0/2260100460 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b0403f030 con 0x7f1b08102de0 2024-09-12T02:01:27.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:27.429+0000 7f1aff7fe640 1 -- 172.21.15.16:0/2260100460 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b04003ce0 con 0x7f1b08102de0 2024-09-12T02:01:27.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:27.429+0000 7f1b105c0640 1 -- 172.21.15.16:0/2260100460 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b0819d6b0 con 0x7f1b08102de0 2024-09-12T02:01:27.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:27.429+0000 7f1aff7fe640 1 -- 172.21.15.16:0/2260100460 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b04035070 con 0x7f1b08102de0 2024-09-12T02:01:27.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:27.430+0000 7f1b105c0640 1 -- 172.21.15.16:0/2260100460 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b081a4ff0 con 0x7f1b08102de0 2024-09-12T02:01:27.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:27.430+0000 7f1b105c0640 1 -- 172.21.15.16:0/2260100460 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b081031c0 con 0x7f1b08102de0 2024-09-12T02:01:27.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:27.432+0000 7f1aff7fe640 1 -- 172.21.15.16:0/2260100460 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f1b04042030 con 0x7f1b08102de0 2024-09-12T02:01:27.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:27.432+0000 7f1aff7fe640 1 --2- 172.21.15.16:0/2260100460 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f1af40a2c90 0x7f1af40a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:27.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:27.433+0000 7f1b0db5b640 1 --2- 172.21.15.16:0/2260100460 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f1af40a2c90 0x7f1af40a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:27.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:27.433+0000 7f1aff7fe640 1 -- 172.21.15.16:0/2260100460 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f1b040ef960 con 0x7f1b08102de0 2024-09-12T02:01:27.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:27.433+0000 7f1b0db5b640 1 --2- 172.21.15.16:0/2260100460 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f1af40a2c90 0x7f1af40a5150 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f1b0810d620 tx=0x7f1af8005d90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:01:27.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:27.436+0000 7f1aff7fe640 1 -- 172.21.15.16:0/2260100460 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1b040bc920 con 0x7f1b08102de0 2024-09-12T02:01:27.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:27.612+0000 7f1b105c0640 1 -- 172.21.15.16:0/2260100460 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1b08107260 con 0x7f1b08102de0 2024-09-12T02:01:27.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:27.613+0000 7f1aff7fe640 1 -- 172.21.15.16:0/2260100460 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+22584 (secure 0 0 0) 0x7f1b040bc220 con 0x7f1b08102de0 2024-09-12T02:01:27.613 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T02:01:27.613 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":82,"fsid":"879f3b56-70a9-11ef-bcea-c7b262605968","created":"2024-09-12T01:52:12.841045+0000","modified":"2024-09-12T02:00:09.559511+0000","last_up_change":"2024-09-12T01:59:08.975443+0000","last_in_change":"2024-09-12T01:58:44.530220+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T01:55:26.727564+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T01:59:52.402097+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T01:59:54.463627+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3ea41eb6-e384-4ddc-810f-0f103c127849","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4000187867},{"type":"v1","addr":"172.21.15.16:6803","nonce":4000187867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4000187867},{"type":"v1","addr":"172.21.15.16:6805","nonce":4000187867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":4000187867},{"type":"v1","addr":"172.21.15.16:6809","nonce":4000187867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4000187867},{"type":"v1","addr":"172.21.15.16:6807","nonce":4000187867}]},"public_addr":"172.21.15.16:6803/4000187867","cluster_addr":"172.21.15.16:6805/4000187867","heartbeat_back_addr":"172.21.15.16:6809/4000187867","heartbeat_front_addr":"172.21.15.16:6807/4000187867","state":["exists","up"]},{"osd":1,"uuid":"07bf63e7-70fa-41b2-af53-12a5fec3f33c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":3862661148},{"type":"v1","addr":"172.21.15.33:6801","nonce":3862661148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":3862661148},{"type":"v1","addr":"172.21.15.33:6803","nonce":3862661148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":3862661148},{"type":"v1","addr":"172.21.15.33:6807","nonce":3862661148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":3862661148},{"type":"v1","addr":"172.21.15.33:6805","nonce":3862661148}]},"public_addr":"172.21.15.33:6801/3862661148","cluster_addr":"172.21.15.33:6803/3862661148","heartbeat_back_addr":"172.21.15.33:6807/3862661148","heartbeat_front_addr":"172.21.15.33:6805/3862661148","state":["exists","up"]},{"osd":2,"uuid":"22e40cac-3bc0-4cce-b47b-77499b767f7e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":3958753963},{"type":"v1","addr":"172.21.15.184:6801","nonce":3958753963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":3958753963},{"type":"v1","addr":"172.21.15.184:6803","nonce":3958753963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":3958753963},{"type":"v1","addr":"172.21.15.184:6807","nonce":3958753963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":3958753963},{"type":"v1","addr":"172.21.15.184:6805","nonce":3958753963}]},"public_addr":"172.21.15.184:6801/3958753963","cluster_addr":"172.21.15.184:6803/3958753963","heartbeat_back_addr":"172.21.15.184:6807/3958753963","heartbeat_front_addr":"172.21.15.184:6805/3958753963","state":["exists","up"]},{"osd":3,"uuid":"834a07f5-2c24-49e1-8e4a-12c96cfbdc65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3395182524},{"type":"v1","addr":"172.21.15.16:6811","nonce":3395182524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3395182524},{"type":"v1","addr":"172.21.15.16:6813","nonce":3395182524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3395182524},{"type":"v1","addr":"172.21.15.16:6817","nonce":3395182524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3395182524},{"type":"v1","addr":"172.21.15.16:6815","nonce":3395182524}]},"public_addr":"172.21.15.16:6811/3395182524","cluster_addr":"172.21.15.16:6813/3395182524","heartbeat_back_addr":"172.21.15.16:6817/3395182524","heartbeat_front_addr":"172.21.15.16:6815/3395182524","state":["exists","up"]},{"osd":4,"uuid":"3ca56e09-c194-4404-b9bd-2524d38367eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":1355184322},{"type":"v1","addr":"172.21.15.33:6809","nonce":1355184322}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1355184322},{"type":"v1","addr":"172.21.15.33:6811","nonce":1355184322}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1355184322},{"type":"v1","addr":"172.21.15.33:6815","nonce":1355184322}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1355184322},{"type":"v1","addr":"172.21.15.33:6813","nonce":1355184322}]},"public_addr":"172.21.15.33:6809/1355184322","cluster_addr":"172.21.15.33:6811/1355184322","heartbeat_back_addr":"172.21.15.33:6815/1355184322","heartbeat_front_addr":"172.21.15.33:6813/1355184322","state":["exists","up"]},{"osd":5,"uuid":"0b05ecdb-b987-461f-951c-6d76739bfb4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":1533397276},{"type":"v1","addr":"172.21.15.184:6809","nonce":1533397276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":1533397276},{"type":"v1","addr":"172.21.15.184:6811","nonce":1533397276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":1533397276},{"type":"v1","addr":"172.21.15.184:6815","nonce":1533397276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":1533397276},{"type":"v1","addr":"172.21.15.184:6813","nonce":1533397276}]},"public_addr":"172.21.15.184:6809/1533397276","cluster_addr":"172.21.15.184:6811/1533397276","heartbeat_back_addr":"172.21.15.184:6815/1533397276","heartbeat_front_addr":"172.21.15.184:6813/1533397276","state":["exists","up"]},{"osd":6,"uuid":"b3936e21-e197-49fa-8441-d909b258b612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":902466142},{"type":"v1","addr":"172.21.15.16:6819","nonce":902466142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":902466142},{"type":"v1","addr":"172.21.15.16:6821","nonce":902466142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":902466142},{"type":"v1","addr":"172.21.15.16:6825","nonce":902466142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":902466142},{"type":"v1","addr":"172.21.15.16:6823","nonce":902466142}]},"public_addr":"172.21.15.16:6819/902466142","cluster_addr":"172.21.15.16:6821/902466142","heartbeat_back_addr":"172.21.15.16:6825/902466142","heartbeat_front_addr":"172.21.15.16:6823/902466142","state":["exists","up"]},{"osd":7,"uuid":"4ddbe011-34c1-4b3f-a8eb-577422fb5426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":4186032630},{"type":"v1","addr":"172.21.15.33:6817","nonce":4186032630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":4186032630},{"type":"v1","addr":"172.21.15.33:6819","nonce":4186032630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":4186032630},{"type":"v1","addr":"172.21.15.33:6823","nonce":4186032630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":4186032630},{"type":"v1","addr":"172.21.15.33:6821","nonce":4186032630}]},"public_addr":"172.21.15.33:6817/4186032630","cluster_addr":"172.21.15.33:6819/4186032630","heartbeat_back_addr":"172.21.15.33:6823/4186032630","heartbeat_front_addr":"172.21.15.33:6821/4186032630","state":["exists","up"]},{"osd":8,"uuid":"dca5bfb8-ac6a-47ce-905c-be7fcaf0b5ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":781851781},{"type":"v1","addr":"172.21.15.184:6817","nonce":781851781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":781851781},{"type":"v1","addr":"172.21.15.184:6819","nonce":781851781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":781851781},{"type":"v1","addr":"172.21.15.184:6823","nonce":781851781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":781851781},{"type":"v1","addr":"172.21.15.184:6821","nonce":781851781}]},"public_addr":"172.21.15.184:6817/781851781","cluster_addr":"172.21.15.184:6819/781851781","heartbeat_back_addr":"172.21.15.184:6823/781851781","heartbeat_front_addr":"172.21.15.184:6821/781851781","state":["exists","up"]},{"osd":9,"uuid":"23a05cb5-01d5-40f3-b66a-15fcccedbd6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":1569578746},{"type":"v1","addr":"172.21.15.16:6827","nonce":1569578746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":1569578746},{"type":"v1","addr":"172.21.15.16:6829","nonce":1569578746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1569578746},{"type":"v1","addr":"172.21.15.16:6833","nonce":1569578746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":1569578746},{"type":"v1","addr":"172.21.15.16:6831","nonce":1569578746}]},"public_addr":"172.21.15.16:6827/1569578746","cluster_addr":"172.21.15.16:6829/1569578746","heartbeat_back_addr":"172.21.15.16:6833/1569578746","heartbeat_front_addr":"172.21.15.16:6831/1569578746","state":["exists","up"]},{"osd":10,"uuid":"e4bd6bce-3c69-44db-bc09-d264a0ad094a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":469977847},{"type":"v1","addr":"172.21.15.33:6825","nonce":469977847}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":469977847},{"type":"v1","addr":"172.21.15.33:6827","nonce":469977847}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":469977847},{"type":"v1","addr":"172.21.15.33:6831","nonce":469977847}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":469977847},{"type":"v1","addr":"172.21.15.33:6829","nonce":469977847}]},"public_addr":"172.21.15.33:6825/469977847","cluster_addr":"172.21.15.33:6827/469977847","heartbeat_back_addr":"172.21.15.33:6831/469977847","heartbeat_front_addr":"172.21.15.33:6829/469977847","state":["exists","up"]},{"osd":11,"uuid":"d60212f8-5a1d-45f7-9a1b-12294bb4c80d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":1619871346},{"type":"v1","addr":"172.21.15.184:6825","nonce":1619871346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":1619871346},{"type":"v1","addr":"172.21.15.184:6827","nonce":1619871346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":1619871346},{"type":"v1","addr":"172.21.15.184:6831","nonce":1619871346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":1619871346},{"type":"v1","addr":"172.21.15.184:6829","nonce":1619871346}]},"public_addr":"172.21.15.184:6825/1619871346","cluster_addr":"172.21.15.184:6827/1619871346","heartbeat_back_addr":"172.21.15.184:6831/1619871346","heartbeat_front_addr":"172.21.15.184:6829/1619871346","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:54:32.057979+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:54:57.501562+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:55:23.137225+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:55:48.646155+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:56:13.311907+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:56:38.214188+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:57:02.570921+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:57:27.506097+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:57:52.038529+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:58:17.423300+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:58:41.579992+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:59:06.245817+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:0/3532565698":"2024-09-13T01:53:08.218475+0000","172.21.15.16:0/1822157179":"2024-09-13T01:52:44.194152+0000","172.21.15.16:0/3919664061":"2024-09-13T01:52:44.194152+0000","172.21.15.16:0/2269394774":"2024-09-13T01:52:44.194152+0000","172.21.15.16:0/2595308634":"2024-09-13T01:53:08.218475+0000","172.21.15.16:6801/2754331153":"2024-09-13T01:52:44.194152+0000","172.21.15.16:6801/2615861334":"2024-09-13T01:53:08.218475+0000","172.21.15.16:6800/2615861334":"2024-09-13T01:53:08.218475+0000","172.21.15.16:0/4199114771":"2024-09-13T01:53:08.218475+0000","172.21.15.16:6800/2754331153":"2024-09-13T01:52:44.194152+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T02:01:27.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:27.615+0000 7f1b105c0640 1 -- 172.21.15.16:0/2260100460 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f1af40a2c90 msgr2=0x7f1af40a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:27.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:27.615+0000 7f1b105c0640 1 --2- 172.21.15.16:0/2260100460 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f1af40a2c90 0x7f1af40a5150 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f1b0810d620 tx=0x7f1af8005d90 comp rx=0 tx=0).stop 2024-09-12T02:01:27.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:27.615+0000 7f1b105c0640 1 -- 172.21.15.16:0/2260100460 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b08102de0 msgr2=0x7f1b08198470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:27.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:27.615+0000 7f1b105c0640 1 --2- 172.21.15.16:0/2260100460 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b08102de0 0x7f1b08198470 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f1b04006010 tx=0x7f1b040042f0 comp rx=0 tx=0).stop 2024-09-12T02:01:27.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:27.615+0000 7f1b105c0640 1 -- 172.21.15.16:0/2260100460 shutdown_connections 2024-09-12T02:01:27.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:27.615+0000 7f1b105c0640 1 --2- 172.21.15.16:0/2260100460 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f1af40a2c90 0x7f1af40a5150 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:27.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:27.615+0000 7f1b105c0640 1 --2- 172.21.15.16:0/2260100460 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b0810c180 0x7f1b0819cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:27.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:27.615+0000 7f1b105c0640 1 --2- 172.21.15.16:0/2260100460 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1b08103700 0x7f1b081989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:27.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:27.615+0000 7f1b105c0640 1 --2- 172.21.15.16:0/2260100460 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b08102de0 0x7f1b08198470 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:27.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:27.615+0000 7f1b105c0640 1 -- 172.21.15.16:0/2260100460 >> 172.21.15.16:0/2260100460 conn(0x7f1b080fc780 msgr2=0x7f1b080fd480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:01:27.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:27.615+0000 7f1b105c0640 1 -- 172.21.15.16:0/2260100460 shutdown_connections 2024-09-12T02:01:27.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:27.615+0000 7f1b105c0640 1 -- 172.21.15.16:0/2260100460 wait complete. 2024-09-12T02:01:28.188 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-12T01:59:56.117608+0000', 'modified': '2024-09-12T02:00:07.097363+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24482, 'mds_1': 24481, 'mds_2': 24469}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24439': {'gid': 24439, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.184:6833/1563374106', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6832', 'nonce': 1563374106}, {'type': 'v1', 'addr': '172.21.15.184:6833', 'nonce': 1563374106}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24452': {'gid': 24452, 'name': 'h', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.33:6837/1892833868', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6836', 'nonce': 1892833868}, {'type': 'v1', 'addr': '172.21.15.33:6837', 'nonce': 1892833868}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24460': {'gid': 24460, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.184:6835/1457679686', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6834', 'nonce': 1457679686}, {'type': 'v1', 'addr': '172.21.15.184:6835', 'nonce': 1457679686}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24469': {'gid': 24469, 'name': 'c', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.184:6837/1793393087', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6836', 'nonce': 1793393087}, {'type': 'v1', 'addr': '172.21.15.184:6837', 'nonce': 1793393087}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24481': {'gid': 24481, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.184:6839/4203347941', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6838', 'nonce': 4203347941}, {'type': 'v1', 'addr': '172.21.15.184:6839', 'nonce': 4203347941}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24482': {'gid': 24482, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.33:6839/4136670204', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6838', 'nonce': 4136670204}, {'type': 'v1', 'addr': '172.21.15.33:6839', 'nonce': 4136670204}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24482, 'qdb_cluster': [24469, 24482, 24481]} 2024-09-12T02:01:28.188 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-09-12T02:01:28.189 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.c status 2024-09-12T02:01:28.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:01:28 smithi016 ceph-mon[30947]: pgmap v321: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-12T02:01:28.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:01:28 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/2260100460' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T02:01:28.444 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:01:28.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:01:28 smithi184 ceph-mon[35593]: pgmap v321: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-12T02:01:28.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:01:28 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/2260100460' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T02:01:28.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:01:28 smithi033 ceph-mon[35332]: pgmap v321: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-12T02:01:28.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:01:28 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/2260100460' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T02:01:28.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.869+0000 7f82b33d1640 1 -- 172.21.15.16:0/4293142562 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82ac106f10 msgr2=0x7f82ac10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:28.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.869+0000 7f82b33d1640 1 --2- 172.21.15.16:0/4293142562 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82ac106f10 0x7f82ac10ea90 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f82a4009900 tx=0x7f82a402a410 comp rx=0 tx=0).stop 2024-09-12T02:01:28.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.869+0000 7f82b33d1640 1 -- 172.21.15.16:0/4293142562 shutdown_connections 2024-09-12T02:01:28.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.869+0000 7f82b33d1640 1 --2- 172.21.15.16:0/4293142562 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82ac106f10 0x7f82ac10ea90 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:28.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.869+0000 7f82b33d1640 1 --2- 172.21.15.16:0/4293142562 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f82ac102790 0x7f82ac1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:28.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.869+0000 7f82b33d1640 1 --2- 172.21.15.16:0/4293142562 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f82ac101de0 0x7f82ac1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:28.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.869+0000 7f82b33d1640 1 -- 172.21.15.16:0/4293142562 >> 172.21.15.16:0/4293142562 conn(0x7f82ac0fd8c0 msgr2=0x7f82ac0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:01:28.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.870+0000 7f82b33d1640 1 -- 172.21.15.16:0/4293142562 shutdown_connections 2024-09-12T02:01:28.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.870+0000 7f82b33d1640 1 -- 172.21.15.16:0/4293142562 wait complete. 2024-09-12T02:01:28.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.871+0000 7f82b33d1640 1 Processor -- start 2024-09-12T02:01:28.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.871+0000 7f82b33d1640 1 -- start start 2024-09-12T02:01:28.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.871+0000 7f82b33d1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82ac101de0 0x7f82ac0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:28.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.871+0000 7f82b33d1640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f82ac102790 0x7f82ac074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:28.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.871+0000 7f82b33d1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f82ac106f10 0x7f82ac074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:28.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.871+0000 7f82b33d1640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f82ac111f70 con 0x7f82ac101de0 2024-09-12T02:01:28.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.871+0000 7f82b33d1640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f82ac111df0 con 0x7f82ac106f10 2024-09-12T02:01:28.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.871+0000 7f82b33d1640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f82ac1120f0 con 0x7f82ac102790 2024-09-12T02:01:28.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.871+0000 7f82b116d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82ac101de0 0x7f82ac0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:28.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.871+0000 7f82b116d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82ac101de0 0x7f82ac0744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43050/0 (socket says 172.21.15.16:43050) 2024-09-12T02:01:28.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.871+0000 7f82b116d640 1 -- 172.21.15.16:0/3269338655 learned_addr learned my addr 172.21.15.16:0/3269338655 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:01:28.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.871+0000 7f82b096c640 1 --2- 172.21.15.16:0/3269338655 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f82ac102790 0x7f82ac074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:28.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.871+0000 7f82b196e640 1 --2- 172.21.15.16:0/3269338655 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f82ac106f10 0x7f82ac074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:28.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.872+0000 7f82b116d640 1 -- 172.21.15.16:0/3269338655 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f82ac102790 msgr2=0x7f82ac074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:28.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.872+0000 7f82b116d640 1 --2- 172.21.15.16:0/3269338655 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f82ac102790 0x7f82ac074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:28.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.872+0000 7f82b116d640 1 -- 172.21.15.16:0/3269338655 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f82ac106f10 msgr2=0x7f82ac074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:28.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.872+0000 7f82b116d640 1 --2- 172.21.15.16:0/3269338655 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f82ac106f10 0x7f82ac074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:28.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.872+0000 7f82b116d640 1 -- 172.21.15.16:0/3269338655 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82ac1a6870 con 0x7f82ac101de0 2024-09-12T02:01:28.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.872+0000 7f82b116d640 1 --2- 172.21.15.16:0/3269338655 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82ac101de0 0x7f82ac0744f0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f829c00c890 tx=0x7f829c00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:01:28.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.872+0000 7f82aa7fc640 1 -- 172.21.15.16:0/3269338655 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f829c004220 con 0x7f82ac101de0 2024-09-12T02:01:28.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.872+0000 7f82aa7fc640 1 -- 172.21.15.16:0/3269338655 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f829c0043c0 con 0x7f82ac101de0 2024-09-12T02:01:28.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.872+0000 7f82b33d1640 1 -- 172.21.15.16:0/3269338655 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82ac1a6b60 con 0x7f82ac101de0 2024-09-12T02:01:28.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.872+0000 7f82aa7fc640 1 -- 172.21.15.16:0/3269338655 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f829c004a00 con 0x7f82ac101de0 2024-09-12T02:01:28.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.872+0000 7f82b33d1640 1 -- 172.21.15.16:0/3269338655 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82ac1a70a0 con 0x7f82ac101de0 2024-09-12T02:01:28.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.874+0000 7f82aa7fc640 1 -- 172.21.15.16:0/3269338655 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f829c0079b0 con 0x7f82ac101de0 2024-09-12T02:01:28.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.875+0000 7f82b57aa740 1 Processor -- start 2024-09-12T02:01:28.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.875+0000 7f82b57aa740 1 -- start start 2024-09-12T02:01:28.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.875+0000 7f82b57aa740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55863a6a99d0 0x55863a6a9db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:28.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.875+0000 7f82b57aa740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55863a6aa2f0 0x55863a6b4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:28.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.875+0000 7f82b57aa740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55863a6b4bb0 0x55863a6b6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:28.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.875+0000 7f82b57aa740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55863a610090 con 0x55863a6a99d0 2024-09-12T02:01:28.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.875+0000 7f82b57aa740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55863a5b7a70 con 0x55863a6b4bb0 2024-09-12T02:01:28.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.875+0000 7f82b57aa740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55863a60c050 con 0x55863a6aa2f0 2024-09-12T02:01:28.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.875+0000 7f82b196e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55863a6a99d0 0x55863a6a9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:28.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.875+0000 7f82aa7fc640 1 --2- 172.21.15.16:0/3269338655 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f82940a2c10 0x7f82940a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:28.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.875+0000 7f82b196e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55863a6a99d0 0x55863a6a9db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43056/0 (socket says 172.21.15.16:43056) 2024-09-12T02:01:28.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.875+0000 7f82b196e640 1 -- 172.21.15.16:0/1430533082 learned_addr learned my addr 172.21.15.16:0/1430533082 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:01:28.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.875+0000 7f82b116d640 1 --2- 172.21.15.16:0/1430533082 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55863a6aa2f0 0x55863a6b4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:28.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.876+0000 7f82b096c640 1 --2- 172.21.15.16:0/1430533082 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55863a6b4bb0 0x55863a6b6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:28.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.876+0000 7f82b196e640 1 --2- 172.21.15.16:0/3269338655 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f82940a2c10 0x7f82940a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:28.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.876+0000 7f82b196e640 1 -- 172.21.15.16:0/1430533082 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55863a6aa2f0 msgr2=0x55863a6b4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:01:28.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.876+0000 7f82b196e640 1 --2- 172.21.15.16:0/1430533082 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55863a6aa2f0 0x55863a6b4670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:28.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.876+0000 7f82b196e640 1 -- 172.21.15.16:0/1430533082 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55863a6b4bb0 msgr2=0x55863a6b6fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:01:28.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.876+0000 7f82b196e640 1 --2- 172.21.15.16:0/1430533082 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55863a6b4bb0 0x55863a6b6fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:28.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.876+0000 7f82b196e640 1 -- 172.21.15.16:0/1430533082 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55863a5c4130 con 0x55863a6a99d0 2024-09-12T02:01:28.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.876+0000 7f82aa7fc640 1 -- 172.21.15.16:0/3269338655 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f829c0ced20 con 0x7f82ac101de0 2024-09-12T02:01:28.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.876+0000 7f82b196e640 1 --2- 172.21.15.16:0/1430533082 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55863a6a99d0 0x55863a6a9db0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f82a4033000 tx=0x7f82a40040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a9c4427080cfb67 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:01:28.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.876+0000 7f82937fe640 1 -- 172.21.15.16:0/1430533082 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82a4035040 con 0x55863a6a99d0 2024-09-12T02:01:28.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.876+0000 7f82937fe640 1 -- 172.21.15.16:0/1430533082 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82a404d070 con 0x55863a6a99d0 2024-09-12T02:01:28.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.876+0000 7f82937fe640 1 -- 172.21.15.16:0/1430533082 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82a40487d0 con 0x55863a6a99d0 2024-09-12T02:01:28.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.877+0000 7f82b57aa740 1 -- 172.21.15.16:0/1430533082 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55863a6a99d0 msgr2=0x55863a6a9db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:28.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.877+0000 7f82b57aa740 1 --2- 172.21.15.16:0/1430533082 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55863a6a99d0 0x55863a6a9db0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f82a4033000 tx=0x7f82a40040c0 comp rx=0 tx=0).stop 2024-09-12T02:01:28.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.877+0000 7f82b196e640 1 --2- 172.21.15.16:0/3269338655 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f82940a2c10 0x7f82940a50d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f82a404c000 tx=0x7f82a404a000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:01:28.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.877+0000 7f82b57aa740 1 -- 172.21.15.16:0/1430533082 shutdown_connections 2024-09-12T02:01:28.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.877+0000 7f82b57aa740 1 --2- 172.21.15.16:0/1430533082 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55863a6b4bb0 0x55863a6b6fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:28.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.877+0000 7f82b57aa740 1 --2- 172.21.15.16:0/1430533082 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55863a6aa2f0 0x55863a6b4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:28.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.877+0000 7f82b57aa740 1 --2- 172.21.15.16:0/1430533082 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55863a6a99d0 0x55863a6a9db0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:28.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.877+0000 7f82b57aa740 1 -- 172.21.15.16:0/1430533082 >> 172.21.15.16:0/1430533082 conn(0x55863a6a5f60 msgr2=0x55863a6a6340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:01:28.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.877+0000 7f82b57aa740 1 -- 172.21.15.16:0/1430533082 shutdown_connections 2024-09-12T02:01:28.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.877+0000 7f82b57aa740 1 -- 172.21.15.16:0/1430533082 wait complete. 2024-09-12T02:01:28.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.877+0000 7f82b57aa740 1 -- start start 2024-09-12T02:01:28.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.878+0000 7f82b57aa740 1 Processor -- start 2024-09-12T02:01:28.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.878+0000 7f82b57aa740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55863a6a99d0 0x55863a6b43b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:28.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.878+0000 7f82b57aa740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55863a6aa2f0 0x55863a6b1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:28.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.878+0000 7f82b57aa740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55863a6b4bb0 0x55863a6b1c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:28.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.878+0000 7f82b57aa740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55863a6b7780 con 0x55863a6aa2f0 2024-09-12T02:01:28.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.878+0000 7f82b57aa740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55863a5caf20 con 0x55863a6b4bb0 2024-09-12T02:01:28.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.878+0000 7f82b57aa740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55863a6b7900 con 0x55863a6a99d0 2024-09-12T02:01:28.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.878+0000 7f82b57aa740 10 client.0 _ll_register_callbacks cb 0x55863a594030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:01:28.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.878+0000 7f82b096c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55863a6aa2f0 0x55863a6b1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:28.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.878+0000 7f82b096c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55863a6aa2f0 0x55863a6b1750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43068/0 (socket says 172.21.15.16:43068) 2024-09-12T02:01:28.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.878+0000 7f82b096c640 1 -- 172.21.15.16:0/3772676464 learned_addr learned my addr 172.21.15.16:0/3772676464 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:01:28.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.878+0000 7f82b196e640 1 --2- 172.21.15.16:0/3772676464 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55863a6b4bb0 0x55863a6b1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:28.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.878+0000 7f82b116d640 1 --2- 172.21.15.16:0/3772676464 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55863a6a99d0 0x55863a6b43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:28.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.878+0000 7f82b096c640 1 -- 172.21.15.16:0/3772676464 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55863a6a99d0 msgr2=0x55863a6b43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:01:28.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.878+0000 7f82b096c640 1 --2- 172.21.15.16:0/3772676464 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55863a6a99d0 0x55863a6b43b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:28.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.878+0000 7f82b096c640 1 -- 172.21.15.16:0/3772676464 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55863a6b4bb0 msgr2=0x55863a6b1c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:01:28.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.879+0000 7f82b096c640 1 --2- 172.21.15.16:0/3772676464 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55863a6b4bb0 0x55863a6b1c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:28.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.879+0000 7f82b096c640 1 -- 172.21.15.16:0/3772676464 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55863a3812b0 con 0x55863a6aa2f0 2024-09-12T02:01:28.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.879+0000 7f82b096c640 1 --2- 172.21.15.16:0/3772676464 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55863a6aa2f0 0x55863a6b1750 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f82a0010a90 tx=0x7f82a0010f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f002ae18bfe0a4e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:01:28.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.879+0000 7f8290ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:01:28.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.879+0000 7f8290ff9640 1 -- 172.21.15.16:0/3772676464 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82a000fdb0 con 0x55863a6aa2f0 2024-09-12T02:01:28.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.879+0000 7f8290ff9640 1 -- 172.21.15.16:0/3772676464 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f82a0004b20 con 0x55863a6aa2f0 2024-09-12T02:01:28.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.879+0000 7f82b57aa740 1 -- 172.21.15.16:0/3772676464 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f82a404d070 con 0x55863a6aa2f0 2024-09-12T02:01:28.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.879+0000 7f8290ff9640 1 -- 172.21.15.16:0/3772676464 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f82a0004d20 con 0x55863a6aa2f0 2024-09-12T02:01:28.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.879+0000 7f82b57aa740 10 client.15081 fetch_fsmap learned FSMap version 24 2024-09-12T02:01:28.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.879+0000 7f82b57aa740 1 -- 172.21.15.16:0/3772676464 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55863a6c30a0 con 0x55863a6aa2f0 2024-09-12T02:01:28.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.879+0000 7f8290ff9640 1 -- 172.21.15.16:0/3772676464 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82a404d070 con 0x55863a6aa2f0 2024-09-12T02:01:28.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.879+0000 7f8290ff9640 1 -- 172.21.15.16:0/3772676464 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f82a000e570 con 0x55863a6aa2f0 2024-09-12T02:01:28.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.880+0000 7f82b57aa740 10 client.15081 fetch_fsmap finished waiting for FSMap version 24 2024-09-12T02:01:28.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.880+0000 7f82b57aa740 10 client.15081 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-12T02:01:28.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.880+0000 7f82b57aa740 20 client.15081 populate_metadata read hostname 'smithi016' 2024-09-12T02:01:28.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.880+0000 7f82b57aa740 1 --2- 172.21.15.16:0/3772676464 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55863a6c5760 0x55863a6c6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:28.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.880+0000 7f82b57aa740 4 client.15081 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:01:28.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.880+0000 7f82b57aa740 1 -- 172.21.15.16:0/3772676464 --> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55863a61d600 con 0x55863a6c5760 2024-09-12T02:01:28.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.880+0000 7f82b116d640 1 --2- 172.21.15.16:0/3772676464 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55863a6c5760 0x55863a6c6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:28.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.880+0000 7f8290ff9640 10 client.15081 ms_handle_connect on v2:172.21.15.184:6836/1793393087 2024-09-12T02:01:28.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.880+0000 7f82b116d640 1 --2- 172.21.15.16:0/3772676464 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55863a6c5760 0x55863a6c6bd0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=142fd1e3ce3aae44 server_cookie=11c2fab8e8a97859 in_seq=0 out_seq=0 2024-09-12T02:01:28.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.881+0000 7f8290ff9640 1 -- 172.21.15.16:0/3772676464 <== mds.2 v2:172.21.15.184:6836/1793393087 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55863a61d600 con 0x55863a6c5760 2024-09-12T02:01:28.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.881+0000 7f8290ff9640 10 client.15081 handle_command_reply: tid=0 2024-09-12T02:01:28.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.881+0000 7f82b57aa740 1 client.15081 shutdown 2024-09-12T02:01:28.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.881+0000 7f82b57aa740 1 -- 172.21.15.16:0/3772676464 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55863a6aa2f0 msgr2=0x55863a6b1750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:28.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.881+0000 7f82b57aa740 1 --2- 172.21.15.16:0/3772676464 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55863a6aa2f0 0x55863a6b1750 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f82a0010a90 tx=0x7f82a0010f50 comp rx=0 tx=0).stop 2024-09-12T02:01:28.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.881+0000 7f82b57aa740 1 -- 172.21.15.16:0/3772676464 shutdown_connections 2024-09-12T02:01:28.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.881+0000 7f82b57aa740 1 --2- 172.21.15.16:0/3772676464 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55863a6b4bb0 0x55863a6b1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:28.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.881+0000 7f82b57aa740 1 --2- 172.21.15.16:0/3772676464 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55863a6c5760 0x55863a6c6bd0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:28.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.881+0000 7f82b57aa740 1 --2- 172.21.15.16:0/3772676464 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55863a6aa2f0 0x55863a6b1750 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:28.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.881+0000 7f82b57aa740 1 --2- 172.21.15.16:0/3772676464 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55863a6a99d0 0x55863a6b43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:28.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.881+0000 7f82b57aa740 1 -- 172.21.15.16:0/3772676464 >> 172.21.15.16:0/3772676464 conn(0x55863a6a5f60 msgr2=0x55863a6abc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:01:28.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.882+0000 7f8290ff9640 0 client.15081 ms_handle_reset on v2:172.21.15.184:6836/1793393087 2024-09-12T02:01:28.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.882+0000 7f82b57aa740 1 -- 172.21.15.16:0/3772676464 shutdown_connections 2024-09-12T02:01:28.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.882+0000 7f82b57aa740 1 -- 172.21.15.16:0/3772676464 wait complete. 2024-09-12T02:01:28.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.882+0000 7f82b57aa740 20 client.15081 trim_cache size 0 max 16384 2024-09-12T02:01:28.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.900+0000 7f82b57aa740 1 Processor -- start 2024-09-12T02:01:28.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.900+0000 7f82b57aa740 1 -- start start 2024-09-12T02:01:28.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.900+0000 7f82b57aa740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55863a6a99d0 0x55863a6c5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:28.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.900+0000 7f82b57aa740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55863a6aa2f0 0x55863a6c6500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:28.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.900+0000 7f82b57aa740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55863a6b4bb0 0x55863a6ceeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:28.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.900+0000 7f82b57aa740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55863a6c4f10 con 0x55863a6aa2f0 2024-09-12T02:01:28.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.900+0000 7f82b57aa740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55863a6c55e0 con 0x55863a6a99d0 2024-09-12T02:01:28.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.900+0000 7f82b57aa740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55863a6c5cb0 con 0x55863a6b4bb0 2024-09-12T02:01:28.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.900+0000 7f82b096c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55863a6aa2f0 0x55863a6c6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:28.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.900+0000 7f82b096c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55863a6aa2f0 0x55863a6c6500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43080/0 (socket says 172.21.15.16:43080) 2024-09-12T02:01:28.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.900+0000 7f82b096c640 1 -- 172.21.15.16:0/1255648169 learned_addr learned my addr 172.21.15.16:0/1255648169 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:01:28.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.900+0000 7f82b116d640 1 --2- 172.21.15.16:0/1255648169 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55863a6a99d0 0x55863a6c5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:28.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.900+0000 7f82b196e640 1 --2- 172.21.15.16:0/1255648169 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55863a6b4bb0 0x55863a6ceeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:28.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.900+0000 7f82b096c640 1 -- 172.21.15.16:0/1255648169 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55863a6b4bb0 msgr2=0x55863a6ceeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:01:28.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.900+0000 7f82b096c640 1 --2- 172.21.15.16:0/1255648169 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55863a6b4bb0 0x55863a6ceeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:28.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.900+0000 7f82b096c640 1 -- 172.21.15.16:0/1255648169 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55863a6a99d0 msgr2=0x55863a6c5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:01:28.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.900+0000 7f82b096c640 1 --2- 172.21.15.16:0/1255648169 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55863a6a99d0 0x55863a6c5090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:28.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.900+0000 7f82b096c640 1 -- 172.21.15.16:0/1255648169 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55863a6b7890 con 0x55863a6aa2f0 2024-09-12T02:01:28.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.900+0000 7f82b096c640 1 --2- 172.21.15.16:0/1255648169 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55863a6aa2f0 0x55863a6c6500 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f82a0002230 tx=0x7f82a000a790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b9f03dfdaf42354 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:01:28.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.901+0000 7f8292ffd640 1 -- 172.21.15.16:0/1255648169 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82a0028040 con 0x55863a6aa2f0 2024-09-12T02:01:28.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.901+0000 7f8292ffd640 1 -- 172.21.15.16:0/1255648169 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82a000fdb0 con 0x55863a6aa2f0 2024-09-12T02:01:28.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.901+0000 7f8292ffd640 1 -- 172.21.15.16:0/1255648169 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82a002d5a0 con 0x55863a6aa2f0 2024-09-12T02:01:28.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.901+0000 7f82b57aa740 1 -- 172.21.15.16:0/1255648169 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55863a6aa2f0 msgr2=0x55863a6c6500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:28.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.901+0000 7f82b57aa740 1 --2- 172.21.15.16:0/1255648169 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55863a6aa2f0 0x55863a6c6500 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f82a0002230 tx=0x7f82a000a790 comp rx=0 tx=0).stop 2024-09-12T02:01:28.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.901+0000 7f82b57aa740 1 -- 172.21.15.16:0/1255648169 shutdown_connections 2024-09-12T02:01:28.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.901+0000 7f82b57aa740 1 --2- 172.21.15.16:0/1255648169 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55863a6b4bb0 0x55863a6ceeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:28.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.901+0000 7f82b57aa740 1 --2- 172.21.15.16:0/1255648169 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55863a6aa2f0 0x55863a6c6500 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:28.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.901+0000 7f82b57aa740 1 --2- 172.21.15.16:0/1255648169 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55863a6a99d0 0x55863a6c5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:28.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.901+0000 7f82b57aa740 1 -- 172.21.15.16:0/1255648169 >> 172.21.15.16:0/1255648169 conn(0x55863a6a5f60 msgr2=0x55863a6b0020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:01:28.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.901+0000 7f82b57aa740 1 -- 172.21.15.16:0/1255648169 shutdown_connections 2024-09-12T02:01:28.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.901+0000 7f82b57aa740 1 -- 172.21.15.16:0/1255648169 wait complete. 2024-09-12T02:01:28.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.902+0000 7f82b57aa740 1 -- start start 2024-09-12T02:01:28.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.902+0000 7f82b57aa740 1 Processor -- start 2024-09-12T02:01:28.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.902+0000 7f82b57aa740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55863a6a99d0 0x55863a6af420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:28.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.902+0000 7f82b57aa740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55863a6aa2f0 0x55863a6ad9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:28.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.902+0000 7f82b57aa740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55863a6b4bb0 0x55863a6ae130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:28.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.902+0000 7f82b57aa740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55863a6c6a50 con 0x55863a6aa2f0 2024-09-12T02:01:28.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.902+0000 7f82b57aa740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55863a6c6380 con 0x55863a6a99d0 2024-09-12T02:01:28.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.902+0000 7f82b57aa740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55863a6c7120 con 0x55863a6b4bb0 2024-09-12T02:01:28.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.902+0000 7f82b57aa740 10 client.0 _ll_register_callbacks cb 0x7f82a000aa50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:01:28.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.902+0000 7f82b096c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55863a6aa2f0 0x55863a6ad9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:28.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.902+0000 7f82b096c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55863a6aa2f0 0x55863a6ad9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43084/0 (socket says 172.21.15.16:43084) 2024-09-12T02:01:28.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.902+0000 7f82b096c640 1 -- 172.21.15.16:0/1122751529 learned_addr learned my addr 172.21.15.16:0/1122751529 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:01:28.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.902+0000 7f82b116d640 1 --2- 172.21.15.16:0/1122751529 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55863a6a99d0 0x55863a6af420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:28.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.903+0000 7f82b196e640 1 --2- 172.21.15.16:0/1122751529 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55863a6b4bb0 0x55863a6ae130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:28.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.903+0000 7f82b096c640 1 -- 172.21.15.16:0/1122751529 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55863a6b4bb0 msgr2=0x55863a6ae130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:01:28.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.903+0000 7f82b096c640 1 --2- 172.21.15.16:0/1122751529 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55863a6b4bb0 0x55863a6ae130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:28.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.903+0000 7f82b096c640 1 -- 172.21.15.16:0/1122751529 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55863a6a99d0 msgr2=0x55863a6af420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:01:28.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.903+0000 7f82b096c640 1 --2- 172.21.15.16:0/1122751529 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55863a6a99d0 0x55863a6af420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:28.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.903+0000 7f82b096c640 1 -- 172.21.15.16:0/1122751529 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55863a6cde20 con 0x55863a6aa2f0 2024-09-12T02:01:28.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.903+0000 7f82b196e640 1 --2- 172.21.15.16:0/1122751529 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55863a6b4bb0 0x55863a6ae130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:01:28.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.903+0000 7f82b116d640 1 --2- 172.21.15.16:0/1122751529 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55863a6a99d0 0x55863a6af420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:01:28.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.903+0000 7f82b096c640 1 --2- 172.21.15.16:0/1122751529 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55863a6aa2f0 0x55863a6ad9c0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f82a000ead0 tx=0x7f82a000a920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0e52d70c8836cd7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:01:28.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.903+0000 7f8290ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:01:28.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.903+0000 7f8290ff9640 1 -- 172.21.15.16:0/1122751529 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82a0028040 con 0x55863a6aa2f0 2024-09-12T02:01:28.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.903+0000 7f8290ff9640 1 -- 172.21.15.16:0/1122751529 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f82a000e5c0 con 0x55863a6aa2f0 2024-09-12T02:01:28.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.903+0000 7f82b57aa740 1 -- 172.21.15.16:0/1122751529 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f82a000fdb0 con 0x55863a6aa2f0 2024-09-12T02:01:28.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.903+0000 7f8290ff9640 1 -- 172.21.15.16:0/1122751529 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f82a003d050 con 0x55863a6aa2f0 2024-09-12T02:01:28.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.903+0000 7f82b57aa740 10 client.15087 fetch_fsmap learned FSMap version 24 2024-09-12T02:01:28.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.903+0000 7f82b57aa740 1 -- 172.21.15.16:0/1122751529 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55863a6cf120 con 0x55863a6aa2f0 2024-09-12T02:01:28.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.903+0000 7f8290ff9640 1 -- 172.21.15.16:0/1122751529 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82a000fdb0 con 0x55863a6aa2f0 2024-09-12T02:01:28.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.904+0000 7f8290ff9640 1 -- 172.21.15.16:0/1122751529 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f82a0036700 con 0x55863a6aa2f0 2024-09-12T02:01:28.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.904+0000 7f82b57aa740 10 client.15087 fetch_fsmap finished waiting for FSMap version 24 2024-09-12T02:01:28.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.904+0000 7f82b57aa740 10 client.15087 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-12T02:01:28.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.904+0000 7f82b57aa740 20 client.15087 populate_metadata read hostname 'smithi016' 2024-09-12T02:01:28.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.904+0000 7f82b57aa740 1 --2- 172.21.15.16:0/1122751529 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55863a7338d0 0x55863a7325e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:28.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.904+0000 7f82b57aa740 4 client.15087 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T02:01:28.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.904+0000 7f82b57aa740 1 -- 172.21.15.16:0/1122751529 --> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] -- command(tid 0: {"prefix": "status"}) -- 0x55863a617b10 con 0x55863a7338d0 2024-09-12T02:01:28.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.904+0000 7f82b116d640 1 --2- 172.21.15.16:0/1122751529 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55863a7338d0 0x55863a7325e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:28.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.904+0000 7f82b116d640 1 --2- 172.21.15.16:0/1122751529 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55863a7338d0 0x55863a7325e0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1ca8dd8eb2309763 server_cookie=66c16204291012cd in_seq=0 out_seq=0 2024-09-12T02:01:28.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.904+0000 7f8290ff9640 10 client.15087 ms_handle_connect on v2:172.21.15.184:6836/1793393087 2024-09-12T02:01:28.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.905+0000 7f8290ff9640 1 -- 172.21.15.16:0/1122751529 <== mds.2 v2:172.21.15.184:6836/1793393087 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55863a617b10 con 0x55863a7338d0 2024-09-12T02:01:28.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.905+0000 7f8290ff9640 10 client.15087 handle_command_reply: tid=0 2024-09-12T02:01:28.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.905+0000 7f82b57aa740 1 client.15087 shutdown 2024-09-12T02:01:28.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.905+0000 7f82b57aa740 1 -- 172.21.15.16:0/1122751529 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55863a6aa2f0 msgr2=0x55863a6ad9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:28.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.905+0000 7f82b57aa740 1 --2- 172.21.15.16:0/1122751529 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55863a6aa2f0 0x55863a6ad9c0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f82a000ead0 tx=0x7f82a000a920 comp rx=0 tx=0).stop 2024-09-12T02:01:28.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.905+0000 7f82b57aa740 1 -- 172.21.15.16:0/1122751529 shutdown_connections 2024-09-12T02:01:28.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.905+0000 7f82b57aa740 1 --2- 172.21.15.16:0/1122751529 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55863a6b4bb0 0x55863a6ae130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:28.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.905+0000 7f82b57aa740 1 --2- 172.21.15.16:0/1122751529 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55863a7338d0 0x55863a7325e0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:28.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.905+0000 7f82b57aa740 1 --2- 172.21.15.16:0/1122751529 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55863a6aa2f0 0x55863a6ad9c0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:28.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.905+0000 7f82b57aa740 1 --2- 172.21.15.16:0/1122751529 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55863a6a99d0 0x55863a6af420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:28.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.905+0000 7f8290ff9640 0 client.15087 ms_handle_reset on v2:172.21.15.184:6836/1793393087 2024-09-12T02:01:28.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.905+0000 7f82b57aa740 1 -- 172.21.15.16:0/1122751529 >> 172.21.15.16:0/1122751529 conn(0x55863a6a5f60 msgr2=0x55863a6ab420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:01:28.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.905+0000 7f82b57aa740 1 -- 172.21.15.16:0/1122751529 shutdown_connections 2024-09-12T02:01:28.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.905+0000 7f82b57aa740 1 -- 172.21.15.16:0/1122751529 wait complete. 2024-09-12T02:01:28.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.905+0000 7f82b57aa740 20 client.15087 trim_cache size 0 max 16384 2024-09-12T02:01:28.912 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T02:01:28.912 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "879f3b56-70a9-11ef-bcea-c7b262605968", 2024-09-12T02:01:28.912 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 2, 2024-09-12T02:01:28.912 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 24469, 2024-09-12T02:01:28.912 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-09-12T02:01:28.913 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-09-12T02:01:28.913 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-09-12T02:01:28.913 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 84.366605155000002, 2024-09-12T02:01:28.913 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 24, 2024-09-12T02:01:28.913 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 82, 2024-09-12T02:01:28.913 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 80, 2024-09-12T02:01:28.913 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 116.03440734599999 2024-09-12T02:01:28.913 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T02:01:28.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.905+0000 7f82b33d1640 1 -- 172.21.15.16:0/3269338655 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f82940a2c10 msgr2=0x7f82940a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:28.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.905+0000 7f82b33d1640 1 --2- 172.21.15.16:0/3269338655 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f82940a2c10 0x7f82940a50d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f82a404c000 tx=0x7f82a404a000 comp rx=0 tx=0).stop 2024-09-12T02:01:28.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.905+0000 7f82b33d1640 1 -- 172.21.15.16:0/3269338655 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82ac101de0 msgr2=0x7f82ac0744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:28.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.905+0000 7f82b33d1640 1 --2- 172.21.15.16:0/3269338655 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82ac101de0 0x7f82ac0744f0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f829c00c890 tx=0x7f829c00cd50 comp rx=0 tx=0).stop 2024-09-12T02:01:28.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.906+0000 7f82b33d1640 1 -- 172.21.15.16:0/3269338655 shutdown_connections 2024-09-12T02:01:28.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.906+0000 7f82b33d1640 1 --2- 172.21.15.16:0/3269338655 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f82940a2c10 0x7f82940a50d0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:28.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.906+0000 7f82b33d1640 1 --2- 172.21.15.16:0/3269338655 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f82ac106f10 0x7f82ac074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:28.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.906+0000 7f82b33d1640 1 --2- 172.21.15.16:0/3269338655 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f82ac102790 0x7f82ac074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:28.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.906+0000 7f82b33d1640 1 --2- 172.21.15.16:0/3269338655 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82ac101de0 0x7f82ac0744f0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:28.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.906+0000 7f82b33d1640 1 -- 172.21.15.16:0/3269338655 >> 172.21.15.16:0/3269338655 conn(0x7f82ac0fd8c0 msgr2=0x7f82ac0fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:01:28.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.906+0000 7f82b33d1640 1 -- 172.21.15.16:0/3269338655 shutdown_connections 2024-09-12T02:01:28.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:28.906+0000 7f82b33d1640 1 -- 172.21.15.16:0/3269338655 wait complete. 2024-09-12T02:01:29.587 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.i status 2024-09-12T02:01:29.845 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:01:30.237 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:01:30 smithi016 ceph-mon[30947]: pgmap v322: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T02:01:30.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.320+0000 7fb059e8d640 1 -- 172.21.15.16:0/1471778741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb054101420 msgr2=0x7fb05410d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:30.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.320+0000 7fb059e8d640 1 --2- 172.21.15.16:0/1471778741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb054101420 0x7fb05410d950 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fb04c009cd0 tx=0x7fb04c02a8e0 comp rx=0 tx=0).stop 2024-09-12T02:01:30.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.320+0000 7fb059e8d640 1 -- 172.21.15.16:0/1471778741 shutdown_connections 2024-09-12T02:01:30.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.320+0000 7fb059e8d640 1 --2- 172.21.15.16:0/1471778741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb054101420 0x7fb05410d950 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:30.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.320+0000 7fb059e8d640 1 --2- 172.21.15.16:0/1471778741 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb054100a80 0x7fb054100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:30.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.320+0000 7fb059e8d640 1 --2- 172.21.15.16:0/1471778741 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb054106a80 0x7fb054106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:30.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.320+0000 7fb059e8d640 1 -- 172.21.15.16:0/1471778741 >> 172.21.15.16:0/1471778741 conn(0x7fb0540fc7a0 msgr2=0x7fb0540febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:01:30.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.320+0000 7fb059e8d640 1 -- 172.21.15.16:0/1471778741 shutdown_connections 2024-09-12T02:01:30.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.320+0000 7fb059e8d640 1 -- 172.21.15.16:0/1471778741 wait complete. 2024-09-12T02:01:30.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.321+0000 7fb059e8d640 1 Processor -- start 2024-09-12T02:01:30.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.321+0000 7fb059e8d640 1 -- start start 2024-09-12T02:01:30.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.321+0000 7fb059e8d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb054100a80 0x7fb054198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:30.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.321+0000 7fb059e8d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb054101420 0x7fb0541989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:30.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.321+0000 7fb059e8d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb054106a80 0x7fb05419cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:30.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.321+0000 7fb059e8d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb054110ea0 con 0x7fb054106a80 2024-09-12T02:01:30.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.321+0000 7fb059e8d640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb054110d20 con 0x7fb054101420 2024-09-12T02:01:30.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.321+0000 7fb059e8d640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb054111020 con 0x7fb054100a80 2024-09-12T02:01:30.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.321+0000 7fb053fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb054106a80 0x7fb05419cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:30.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.321+0000 7fb053fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb054106a80 0x7fb05419cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43098/0 (socket says 172.21.15.16:43098) 2024-09-12T02:01:30.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.321+0000 7fb053fff640 1 -- 172.21.15.16:0/509065595 learned_addr learned my addr 172.21.15.16:0/509065595 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:01:30.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.321+0000 7fb052ffd640 1 --2- 172.21.15.16:0/509065595 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb054101420 0x7fb0541989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:30.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.322+0000 7fb0537fe640 1 --2- 172.21.15.16:0/509065595 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb054100a80 0x7fb054198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:30.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.322+0000 7fb053fff640 1 -- 172.21.15.16:0/509065595 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb054100a80 msgr2=0x7fb054198480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:30.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.322+0000 7fb053fff640 1 --2- 172.21.15.16:0/509065595 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb054100a80 0x7fb054198480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:30.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.322+0000 7fb053fff640 1 -- 172.21.15.16:0/509065595 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb054101420 msgr2=0x7fb0541989c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:30.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.322+0000 7fb053fff640 1 --2- 172.21.15.16:0/509065595 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb054101420 0x7fb0541989c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:30.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.322+0000 7fb053fff640 1 -- 172.21.15.16:0/509065595 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb05419d430 con 0x7fb054106a80 2024-09-12T02:01:30.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.322+0000 7fb053fff640 1 --2- 172.21.15.16:0/509065595 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb054106a80 0x7fb05419cd50 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fb04c005cc0 tx=0x7fb04c003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:01:30.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.322+0000 7fb050ff9640 1 -- 172.21.15.16:0/509065595 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb04c004040 con 0x7fb054106a80 2024-09-12T02:01:30.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.322+0000 7fb050ff9640 1 -- 172.21.15.16:0/509065595 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb04c0046c0 con 0x7fb054106a80 2024-09-12T02:01:30.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.322+0000 7fb059e8d640 1 -- 172.21.15.16:0/509065595 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb05419d6c0 con 0x7fb054106a80 2024-09-12T02:01:30.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.322+0000 7fb050ff9640 1 -- 172.21.15.16:0/509065595 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb04c046480 con 0x7fb054106a80 2024-09-12T02:01:30.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.322+0000 7fb059e8d640 1 -- 172.21.15.16:0/509065595 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0541a5000 con 0x7fb054106a80 2024-09-12T02:01:30.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.324+0000 7fb050ff9640 1 -- 172.21.15.16:0/509065595 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fb04c0305a0 con 0x7fb054106a80 2024-09-12T02:01:30.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.325+0000 7fb05c266740 1 Processor -- start 2024-09-12T02:01:30.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.325+0000 7fb05c266740 1 -- start start 2024-09-12T02:01:30.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.325+0000 7fb05c266740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557545d53de0 0x557545d541c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:30.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.325+0000 7fb05c266740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557545d54700 0x557545d5ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:30.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.325+0000 7fb05c266740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557545d5efc0 0x557545d613b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:30.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.325+0000 7fb05c266740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x557545cb6090 con 0x557545d54700 2024-09-12T02:01:30.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.325+0000 7fb05c266740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x557545c5da70 con 0x557545d5efc0 2024-09-12T02:01:30.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.325+0000 7fb05c266740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x557545cb2050 con 0x557545d53de0 2024-09-12T02:01:30.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.325+0000 7fb0537fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557545d54700 0x557545d5ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:30.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.325+0000 7fb050ff9640 1 --2- 172.21.15.16:0/509065595 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fb03c0a2c10 0x7fb03c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:30.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.325+0000 7fb0537fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557545d54700 0x557545d5ea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43104/0 (socket says 172.21.15.16:43104) 2024-09-12T02:01:30.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.325+0000 7fb0537fe640 1 -- 172.21.15.16:0/3501160070 learned_addr learned my addr 172.21.15.16:0/3501160070 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:01:30.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.325+0000 7fb052ffd640 1 --2- 172.21.15.16:0/3501160070 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557545d53de0 0x557545d541c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:30.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.325+0000 7fb053fff640 1 --2- 172.21.15.16:0/509065595 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fb03c0a2c10 0x7fb03c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:30.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.325+0000 7fb052ffd640 1 --2- 172.21.15.16:0/3501160070 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557545d5efc0 0x557545d613b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:30.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.325+0000 7fb0537fe640 1 -- 172.21.15.16:0/3501160070 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557545d53de0 msgr2=0x557545d541c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:01:30.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.325+0000 7fb0537fe640 1 --2- 172.21.15.16:0/3501160070 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557545d53de0 0x557545d541c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:30.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.325+0000 7fb0537fe640 1 -- 172.21.15.16:0/3501160070 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557545d5efc0 msgr2=0x557545d613b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:01:30.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.325+0000 7fb050ff9640 1 -- 172.21.15.16:0/509065595 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fb04c02e030 con 0x7fb054106a80 2024-09-12T02:01:30.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.325+0000 7fb0537fe640 1 --2- 172.21.15.16:0/3501160070 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557545d5efc0 0x557545d613b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:30.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.325+0000 7fb0537fe640 1 -- 172.21.15.16:0/3501160070 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557545c6a130 con 0x557545d54700 2024-09-12T02:01:30.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.325+0000 7fb052ffd640 1 --2- 172.21.15.16:0/3501160070 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557545d5efc0 0x557545d613b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:01:30.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.326+0000 7fb0537fe640 1 --2- 172.21.15.16:0/3501160070 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557545d54700 0x557545d5ea80 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fb04400b820 tx=0x7fb04400bce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba2b221de2f002d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:01:30.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.326+0000 7fb053fff640 1 --2- 172.21.15.16:0/509065595 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fb03c0a2c10 0x7fb03c0a50d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fb04c0bd090 tx=0x7fb04c034f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:01:30.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.326+0000 7fb039ffb640 1 -- 172.21.15.16:0/3501160070 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb04400d950 con 0x557545d54700 2024-09-12T02:01:30.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.326+0000 7fb039ffb640 1 -- 172.21.15.16:0/3501160070 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb044019070 con 0x557545d54700 2024-09-12T02:01:30.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.326+0000 7fb039ffb640 1 -- 172.21.15.16:0/3501160070 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0440123e0 con 0x557545d54700 2024-09-12T02:01:30.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.327+0000 7fb05c266740 1 -- 172.21.15.16:0/3501160070 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557545d54700 msgr2=0x557545d5ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:30.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.327+0000 7fb05c266740 1 --2- 172.21.15.16:0/3501160070 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557545d54700 0x557545d5ea80 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fb04400b820 tx=0x7fb04400bce0 comp rx=0 tx=0).stop 2024-09-12T02:01:30.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.327+0000 7fb05c266740 1 -- 172.21.15.16:0/3501160070 shutdown_connections 2024-09-12T02:01:30.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.327+0000 7fb05c266740 1 --2- 172.21.15.16:0/3501160070 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557545d5efc0 0x557545d613b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:30.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.327+0000 7fb05c266740 1 --2- 172.21.15.16:0/3501160070 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557545d54700 0x557545d5ea80 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:30.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.327+0000 7fb05c266740 1 --2- 172.21.15.16:0/3501160070 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557545d53de0 0x557545d541c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:30.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.327+0000 7fb05c266740 1 -- 172.21.15.16:0/3501160070 >> 172.21.15.16:0/3501160070 conn(0x557545d50370 msgr2=0x557545d50750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:01:30.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.327+0000 7fb05c266740 1 -- 172.21.15.16:0/3501160070 shutdown_connections 2024-09-12T02:01:30.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.327+0000 7fb05c266740 1 -- 172.21.15.16:0/3501160070 wait complete. 2024-09-12T02:01:30.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.327+0000 7fb05c266740 1 -- start start 2024-09-12T02:01:30.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.327+0000 7fb05c266740 1 Processor -- start 2024-09-12T02:01:30.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.328+0000 7fb05c266740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557545d53de0 0x557545d5e840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:30.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.328+0000 7fb05c266740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557545d54700 0x557545d5bb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:30.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.328+0000 7fb05c266740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557545d5efc0 0x557545d5c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:30.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.328+0000 7fb05c266740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x557545d61b90 con 0x557545d5efc0 2024-09-12T02:01:30.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.328+0000 7fb05c266740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x557545c70f20 con 0x557545d54700 2024-09-12T02:01:30.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.328+0000 7fb05c266740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x557545d61d10 con 0x557545d53de0 2024-09-12T02:01:30.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.328+0000 7fb05c266740 10 client.0 _ll_register_callbacks cb 0x557545c3a030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:01:30.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.328+0000 7fb052ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557545d5efc0 0x557545d5c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:30.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.328+0000 7fb052ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557545d5efc0 0x557545d5c0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43112/0 (socket says 172.21.15.16:43112) 2024-09-12T02:01:30.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.328+0000 7fb052ffd640 1 -- 172.21.15.16:0/1034913437 learned_addr learned my addr 172.21.15.16:0/1034913437 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:01:30.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.328+0000 7fb0537fe640 1 --2- 172.21.15.16:0/1034913437 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557545d54700 0x557545d5bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:30.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.328+0000 7fb052ffd640 1 --2- 172.21.15.16:0/1034913437 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557545d53de0 0x557545d5e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:30.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.328+0000 7fb052ffd640 1 -- 172.21.15.16:0/1034913437 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557545d53de0 msgr2=0x557545d5e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:01:30.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.328+0000 7fb052ffd640 1 --2- 172.21.15.16:0/1034913437 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557545d53de0 0x557545d5e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:30.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.328+0000 7fb052ffd640 1 -- 172.21.15.16:0/1034913437 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557545d54700 msgr2=0x557545d5bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:01:30.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.328+0000 7fb052ffd640 1 --2- 172.21.15.16:0/1034913437 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557545d54700 0x557545d5bb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:30.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.328+0000 7fb052ffd640 1 -- 172.21.15.16:0/1034913437 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557545a272b0 con 0x557545d5efc0 2024-09-12T02:01:30.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.328+0000 7fb052ffd640 1 --2- 172.21.15.16:0/1034913437 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557545d5efc0 0x557545d5c0d0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fb048004420 tx=0x7fb0480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb229a0e4cd9c76e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:01:30.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.328+0000 7fb032ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:01:30.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.328+0000 7fb032ffd640 1 -- 172.21.15.16:0/1034913437 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb048005220 con 0x557545d5efc0 2024-09-12T02:01:30.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.328+0000 7fb032ffd640 1 -- 172.21.15.16:0/1034913437 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fb0480053c0 con 0x557545d5efc0 2024-09-12T02:01:30.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.328+0000 7fb05c266740 1 -- 172.21.15.16:0/1034913437 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb044019070 con 0x557545d5efc0 2024-09-12T02:01:30.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.329+0000 7fb032ffd640 1 -- 172.21.15.16:0/1034913437 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fb048027050 con 0x557545d5efc0 2024-09-12T02:01:30.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.329+0000 7fb05c266740 10 client.15099 fetch_fsmap learned FSMap version 24 2024-09-12T02:01:30.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.329+0000 7fb05c266740 1 -- 172.21.15.16:0/1034913437 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x557545d6d4f0 con 0x557545d5efc0 2024-09-12T02:01:30.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.329+0000 7fb032ffd640 1 -- 172.21.15.16:0/1034913437 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb044019070 con 0x557545d5efc0 2024-09-12T02:01:30.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.329+0000 7fb032ffd640 1 -- 172.21.15.16:0/1034913437 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fb048011890 con 0x557545d5efc0 2024-09-12T02:01:30.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.329+0000 7fb05c266740 10 client.15099 fetch_fsmap finished waiting for FSMap version 24 2024-09-12T02:01:30.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.329+0000 7fb05c266740 10 client.15099 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-12T02:01:30.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.329+0000 7fb05c266740 20 client.15099 populate_metadata read hostname 'smithi016' 2024-09-12T02:01:30.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.329+0000 7fb05c266740 1 --2- 172.21.15.16:0/1034913437 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x557545d6fac0 0x557545d70f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:30.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.329+0000 7fb05c266740 4 client.15099 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:01:30.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.329+0000 7fb05c266740 1 -- 172.21.15.16:0/1034913437 --> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557545cc3600 con 0x557545d6fac0 2024-09-12T02:01:30.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.330+0000 7fb0537fe640 1 --2- 172.21.15.16:0/1034913437 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x557545d6fac0 0x557545d70f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:30.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.330+0000 7fb032ffd640 10 client.15099 ms_handle_connect on v2:172.21.15.184:6838/4203347941 2024-09-12T02:01:30.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.330+0000 7fb0537fe640 1 --2- 172.21.15.16:0/1034913437 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x557545d6fac0 0x557545d70f30 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5a53063e98336e76 server_cookie=51d820f341577b70 in_seq=0 out_seq=0 2024-09-12T02:01:30.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.331+0000 7fb032ffd640 1 -- 172.21.15.16:0/1034913437 <== mds.1 v2:172.21.15.184:6838/4203347941 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557545cc3600 con 0x557545d6fac0 2024-09-12T02:01:30.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.331+0000 7fb032ffd640 10 client.15099 handle_command_reply: tid=0 2024-09-12T02:01:30.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.331+0000 7fb05c266740 1 client.15099 shutdown 2024-09-12T02:01:30.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.331+0000 7fb05c266740 1 -- 172.21.15.16:0/1034913437 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557545d5efc0 msgr2=0x557545d5c0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:30.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.331+0000 7fb05c266740 1 --2- 172.21.15.16:0/1034913437 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557545d5efc0 0x557545d5c0d0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fb048004420 tx=0x7fb0480046c0 comp rx=0 tx=0).stop 2024-09-12T02:01:30.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.331+0000 7fb05c266740 1 -- 172.21.15.16:0/1034913437 shutdown_connections 2024-09-12T02:01:30.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.331+0000 7fb05c266740 1 --2- 172.21.15.16:0/1034913437 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x557545d6fac0 0x557545d70f30 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:30.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.331+0000 7fb05c266740 1 --2- 172.21.15.16:0/1034913437 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557545d5efc0 0x557545d5c0d0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:30.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.331+0000 7fb05c266740 1 --2- 172.21.15.16:0/1034913437 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557545d54700 0x557545d5bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:30.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.331+0000 7fb05c266740 1 --2- 172.21.15.16:0/1034913437 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557545d53de0 0x557545d5e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:30.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.331+0000 7fb05c266740 1 -- 172.21.15.16:0/1034913437 >> 172.21.15.16:0/1034913437 conn(0x557545d50370 msgr2=0x557545d4b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:01:30.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.331+0000 7fb032ffd640 0 client.15099 ms_handle_reset on v2:172.21.15.184:6838/4203347941 2024-09-12T02:01:30.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.331+0000 7fb05c266740 1 -- 172.21.15.16:0/1034913437 shutdown_connections 2024-09-12T02:01:30.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.331+0000 7fb05c266740 1 -- 172.21.15.16:0/1034913437 wait complete. 2024-09-12T02:01:30.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.331+0000 7fb05c266740 20 client.15099 trim_cache size 0 max 16384 2024-09-12T02:01:30.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.349+0000 7fb05c266740 1 Processor -- start 2024-09-12T02:01:30.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.349+0000 7fb05c266740 1 -- start start 2024-09-12T02:01:30.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.349+0000 7fb05c266740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557545d53de0 0x557545d70190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:30.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.349+0000 7fb05c266740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557545d54700 0x557545d5bb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:30.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.349+0000 7fb05c266740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557545d5efc0 0x557545d5c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:30.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.349+0000 7fb05c266740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x557545d6f270 con 0x557545d5efc0 2024-09-12T02:01:30.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.349+0000 7fb05c266740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x557545d6f940 con 0x557545d53de0 2024-09-12T02:01:30.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.349+0000 7fb05c266740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x557545d70010 con 0x557545d54700 2024-09-12T02:01:30.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.349+0000 7fb052ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557545d5efc0 0x557545d5c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:30.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.350+0000 7fb052ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557545d5efc0 0x557545d5c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43124/0 (socket says 172.21.15.16:43124) 2024-09-12T02:01:30.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.350+0000 7fb052ffd640 1 -- 172.21.15.16:0/1019458356 learned_addr learned my addr 172.21.15.16:0/1019458356 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:01:30.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.350+0000 7fb052ffd640 1 -- 172.21.15.16:0/1019458356 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557545d54700 msgr2=0x557545d5bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:01:30.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.350+0000 7fb0537fe640 1 --2- 172.21.15.16:0/1019458356 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557545d54700 0x557545d5bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:30.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.350+0000 7fb052ffd640 1 --2- 172.21.15.16:0/1019458356 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557545d54700 0x557545d5bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:30.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.350+0000 7fb052ffd640 1 -- 172.21.15.16:0/1019458356 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557545d53de0 msgr2=0x557545d70190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:01:30.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.350+0000 7fb052ffd640 1 --2- 172.21.15.16:0/1019458356 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557545d53de0 0x557545d70190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:30.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.350+0000 7fb052ffd640 1 -- 172.21.15.16:0/1019458356 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557545d53c10 con 0x557545d5efc0 2024-09-12T02:01:30.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.350+0000 7fb0537fe640 1 --2- 172.21.15.16:0/1019458356 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557545d54700 0x557545d5bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:01:30.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.350+0000 7fb052ffd640 1 --2- 172.21.15.16:0/1019458356 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557545d5efc0 0x557545d5c090 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fb048001eb0 tx=0x7fb048004290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6db26c3002f18ea4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:01:30.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.350+0000 7fb0397fa640 1 -- 172.21.15.16:0/1019458356 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb04802f020 con 0x557545d5efc0 2024-09-12T02:01:30.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.350+0000 7fb0397fa640 1 -- 172.21.15.16:0/1019458356 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb048011e80 con 0x557545d5efc0 2024-09-12T02:01:30.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.350+0000 7fb0397fa640 1 -- 172.21.15.16:0/1019458356 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb048016070 con 0x557545d5efc0 2024-09-12T02:01:30.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.351+0000 7fb05c266740 1 -- 172.21.15.16:0/1019458356 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557545d5efc0 msgr2=0x557545d5c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:30.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.351+0000 7fb05c266740 1 --2- 172.21.15.16:0/1019458356 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557545d5efc0 0x557545d5c090 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fb048001eb0 tx=0x7fb048004290 comp rx=0 tx=0).stop 2024-09-12T02:01:30.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.351+0000 7fb05c266740 1 -- 172.21.15.16:0/1019458356 shutdown_connections 2024-09-12T02:01:30.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.351+0000 7fb05c266740 1 --2- 172.21.15.16:0/1019458356 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557545d5efc0 0x557545d5c090 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:30.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.351+0000 7fb05c266740 1 --2- 172.21.15.16:0/1019458356 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557545d54700 0x557545d5bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:30.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.351+0000 7fb05c266740 1 --2- 172.21.15.16:0/1019458356 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557545d53de0 0x557545d70190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:30.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.351+0000 7fb05c266740 1 -- 172.21.15.16:0/1019458356 >> 172.21.15.16:0/1019458356 conn(0x557545d50370 msgr2=0x557545d50750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:01:30.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.351+0000 7fb05c266740 1 -- 172.21.15.16:0/1019458356 shutdown_connections 2024-09-12T02:01:30.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.351+0000 7fb05c266740 1 -- 172.21.15.16:0/1019458356 wait complete. 2024-09-12T02:01:30.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.351+0000 7fb05c266740 1 -- start start 2024-09-12T02:01:30.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.351+0000 7fb05c266740 1 Processor -- start 2024-09-12T02:01:30.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.352+0000 7fb05c266740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557545d53de0 0x557545d740c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:30.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.352+0000 7fb05c266740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557545d54700 0x557545d74600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:30.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.352+0000 7fb05c266740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557545d5efc0 0x557545d74b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:30.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.352+0000 7fb05c266740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x557545d70db0 con 0x557545d53de0 2024-09-12T02:01:30.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.352+0000 7fb05c266740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x557545d706e0 con 0x557545d5efc0 2024-09-12T02:01:30.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.352+0000 7fb05c266740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x557545d71480 con 0x557545d54700 2024-09-12T02:01:30.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.352+0000 7fb05c266740 10 client.0 _ll_register_callbacks cb 0x7fb04800bd00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:01:30.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.352+0000 7fb052ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557545d53de0 0x557545d740c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:30.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.352+0000 7fb052ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557545d53de0 0x557545d740c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43136/0 (socket says 172.21.15.16:43136) 2024-09-12T02:01:30.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.352+0000 7fb052ffd640 1 -- 172.21.15.16:0/1657368344 learned_addr learned my addr 172.21.15.16:0/1657368344 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:01:30.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.352+0000 7fb0537fe640 1 --2- 172.21.15.16:0/1657368344 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557545d54700 0x557545d74600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:30.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.352+0000 7fb052ffd640 1 --2- 172.21.15.16:0/1657368344 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557545d5efc0 0x557545d74b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:30.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.352+0000 7fb052ffd640 1 -- 172.21.15.16:0/1657368344 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557545d54700 msgr2=0x557545d74600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:01:30.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.352+0000 7fb052ffd640 1 --2- 172.21.15.16:0/1657368344 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557545d54700 0x557545d74600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:30.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.352+0000 7fb052ffd640 1 -- 172.21.15.16:0/1657368344 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557545d5efc0 msgr2=0x557545d74b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:01:30.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.352+0000 7fb052ffd640 1 --2- 172.21.15.16:0/1657368344 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557545d5efc0 0x557545d74b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:30.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.352+0000 7fb052ffd640 1 -- 172.21.15.16:0/1657368344 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557545dd82f0 con 0x557545d53de0 2024-09-12T02:01:30.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.352+0000 7fb052ffd640 1 --2- 172.21.15.16:0/1657368344 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557545d53de0 0x557545d740c0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fb048011c90 tx=0x7fb04800b410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8abb6ff7047c1c3f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:01:30.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.352+0000 7fb032ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:01:30.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.352+0000 7fb032ffd640 1 -- 172.21.15.16:0/1657368344 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb04803c060 con 0x557545d53de0 2024-09-12T02:01:30.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.352+0000 7fb032ffd640 1 -- 172.21.15.16:0/1657368344 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fb048012860 con 0x557545d53de0 2024-09-12T02:01:30.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.353+0000 7fb05c266740 1 -- 172.21.15.16:0/1657368344 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x557545cbdb10 con 0x557545d53de0 2024-09-12T02:01:30.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.353+0000 7fb032ffd640 1 -- 172.21.15.16:0/1657368344 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fb048016070 con 0x557545d53de0 2024-09-12T02:01:30.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.353+0000 7fb05c266740 10 client.15105 fetch_fsmap learned FSMap version 24 2024-09-12T02:01:30.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.353+0000 7fb05c266740 1 -- 172.21.15.16:0/1657368344 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x557545dd9520 con 0x557545d53de0 2024-09-12T02:01:30.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.353+0000 7fb032ffd640 1 -- 172.21.15.16:0/1657368344 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x557545cbdb10 con 0x557545d53de0 2024-09-12T02:01:30.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.353+0000 7fb032ffd640 1 -- 172.21.15.16:0/1657368344 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fb048034040 con 0x557545d53de0 2024-09-12T02:01:30.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.353+0000 7fb05c266740 10 client.15105 fetch_fsmap finished waiting for FSMap version 24 2024-09-12T02:01:30.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.353+0000 7fb05c266740 10 client.15105 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-12T02:01:30.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.353+0000 7fb05c266740 20 client.15105 populate_metadata read hostname 'smithi016' 2024-09-12T02:01:30.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.353+0000 7fb05c266740 1 --2- 172.21.15.16:0/1657368344 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x557545dddd90 0x557545ddcaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:30.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.353+0000 7fb05c266740 4 client.15105 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T02:01:30.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.353+0000 7fb05c266740 1 -- 172.21.15.16:0/1657368344 --> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] -- command(tid 0: {"prefix": "status"}) -- 0x557545c3a0e0 con 0x557545dddd90 2024-09-12T02:01:30.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.354+0000 7fb0537fe640 1 --2- 172.21.15.16:0/1657368344 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x557545dddd90 0x557545ddcaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:30.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.354+0000 7fb032ffd640 10 client.15105 ms_handle_connect on v2:172.21.15.184:6838/4203347941 2024-09-12T02:01:30.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.354+0000 7fb0537fe640 1 --2- 172.21.15.16:0/1657368344 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x557545dddd90 0x557545ddcaa0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=997d3c68741f58b1 server_cookie=bfe856badd8d9518 in_seq=0 out_seq=0 2024-09-12T02:01:30.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.354+0000 7fb032ffd640 1 -- 172.21.15.16:0/1657368344 <== mds.1 v2:172.21.15.184:6838/4203347941 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x557545c3a0e0 con 0x557545dddd90 2024-09-12T02:01:30.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.354+0000 7fb032ffd640 10 client.15105 handle_command_reply: tid=0 2024-09-12T02:01:30.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.354+0000 7fb05c266740 1 client.15105 shutdown 2024-09-12T02:01:30.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.354+0000 7fb05c266740 1 -- 172.21.15.16:0/1657368344 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557545d53de0 msgr2=0x557545d740c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:30.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.354+0000 7fb05c266740 1 --2- 172.21.15.16:0/1657368344 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557545d53de0 0x557545d740c0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fb048011c90 tx=0x7fb04800b410 comp rx=0 tx=0).stop 2024-09-12T02:01:30.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.355+0000 7fb05c266740 1 -- 172.21.15.16:0/1657368344 shutdown_connections 2024-09-12T02:01:30.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.355+0000 7fb05c266740 1 --2- 172.21.15.16:0/1657368344 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x557545dddd90 0x557545ddcaa0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:30.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.355+0000 7fb05c266740 1 --2- 172.21.15.16:0/1657368344 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557545d5efc0 0x557545d74b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:30.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.355+0000 7fb05c266740 1 --2- 172.21.15.16:0/1657368344 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557545d54700 0x557545d74600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:30.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.355+0000 7fb05c266740 1 --2- 172.21.15.16:0/1657368344 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557545d53de0 0x557545d740c0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:30.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.355+0000 7fb05c266740 1 -- 172.21.15.16:0/1657368344 >> 172.21.15.16:0/1657368344 conn(0x557545d50370 msgr2=0x557545d5bb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:01:30.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.355+0000 7fb032ffd640 0 client.15105 ms_handle_reset on v2:172.21.15.184:6838/4203347941 2024-09-12T02:01:30.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.355+0000 7fb05c266740 1 -- 172.21.15.16:0/1657368344 shutdown_connections 2024-09-12T02:01:30.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.355+0000 7fb05c266740 1 -- 172.21.15.16:0/1657368344 wait complete. 2024-09-12T02:01:30.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.355+0000 7fb05c266740 20 client.15105 trim_cache size 0 max 16384 2024-09-12T02:01:30.360 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T02:01:30.361 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "879f3b56-70a9-11ef-bcea-c7b262605968", 2024-09-12T02:01:30.361 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 1, 2024-09-12T02:01:30.361 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 24481, 2024-09-12T02:01:30.361 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-09-12T02:01:30.361 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-09-12T02:01:30.361 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-09-12T02:01:30.361 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 86.824522701999996, 2024-09-12T02:01:30.361 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 24, 2024-09-12T02:01:30.361 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 82, 2024-09-12T02:01:30.361 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 80, 2024-09-12T02:01:30.361 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 112.564861046 2024-09-12T02:01:30.361 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T02:01:30.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.355+0000 7fb059e8d640 1 -- 172.21.15.16:0/509065595 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fb03c0a2c10 msgr2=0x7fb03c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:30.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.355+0000 7fb059e8d640 1 --2- 172.21.15.16:0/509065595 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fb03c0a2c10 0x7fb03c0a50d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fb04c0bd090 tx=0x7fb04c034f90 comp rx=0 tx=0).stop 2024-09-12T02:01:30.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.355+0000 7fb059e8d640 1 -- 172.21.15.16:0/509065595 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb054106a80 msgr2=0x7fb05419cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:30.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.355+0000 7fb059e8d640 1 --2- 172.21.15.16:0/509065595 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb054106a80 0x7fb05419cd50 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fb04c005cc0 tx=0x7fb04c003970 comp rx=0 tx=0).stop 2024-09-12T02:01:30.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.355+0000 7fb059e8d640 1 -- 172.21.15.16:0/509065595 shutdown_connections 2024-09-12T02:01:30.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.355+0000 7fb059e8d640 1 --2- 172.21.15.16:0/509065595 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fb03c0a2c10 0x7fb03c0a50d0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:30.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.355+0000 7fb059e8d640 1 --2- 172.21.15.16:0/509065595 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb054106a80 0x7fb05419cd50 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:30.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.355+0000 7fb059e8d640 1 --2- 172.21.15.16:0/509065595 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb054101420 0x7fb0541989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:30.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.355+0000 7fb059e8d640 1 --2- 172.21.15.16:0/509065595 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb054100a80 0x7fb054198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:30.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.355+0000 7fb059e8d640 1 -- 172.21.15.16:0/509065595 >> 172.21.15.16:0/509065595 conn(0x7fb0540fc7a0 msgr2=0x7fb05406a850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:01:30.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.355+0000 7fb059e8d640 1 -- 172.21.15.16:0/509065595 shutdown_connections 2024-09-12T02:01:30.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:30.356+0000 7fb059e8d640 1 -- 172.21.15.16:0/509065595 wait complete. 2024-09-12T02:01:30.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:01:30 smithi184 ceph-mon[35593]: pgmap v322: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T02:01:30.668 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:01:30 smithi033 ceph-mon[35332]: pgmap v322: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T02:01:31.053 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.b status 2024-09-12T02:01:31.313 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:01:31.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.724+0000 7f8ecf632640 1 -- 172.21.15.16:0/16574201 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ec8101420 msgr2=0x7f8ec810d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:31.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.724+0000 7f8ecf632640 1 --2- 172.21.15.16:0/16574201 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ec8101420 0x7f8ec810d950 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f8ec0009cd0 tx=0x7f8ec002a8e0 comp rx=0 tx=0).stop 2024-09-12T02:01:31.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.724+0000 7f8ecf632640 1 -- 172.21.15.16:0/16574201 shutdown_connections 2024-09-12T02:01:31.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.724+0000 7f8ecf632640 1 --2- 172.21.15.16:0/16574201 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ec8101420 0x7f8ec810d950 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:31.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.724+0000 7f8ecf632640 1 --2- 172.21.15.16:0/16574201 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8ec8100a80 0x7f8ec8100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:31.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.724+0000 7f8ecf632640 1 --2- 172.21.15.16:0/16574201 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8ec8106a80 0x7f8ec8106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:31.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.724+0000 7f8ecf632640 1 -- 172.21.15.16:0/16574201 >> 172.21.15.16:0/16574201 conn(0x7f8ec80fc7a0 msgr2=0x7f8ec80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:01:31.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.724+0000 7f8ecf632640 1 -- 172.21.15.16:0/16574201 shutdown_connections 2024-09-12T02:01:31.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.724+0000 7f8ecf632640 1 -- 172.21.15.16:0/16574201 wait complete. 2024-09-12T02:01:31.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.725+0000 7f8ecf632640 1 Processor -- start 2024-09-12T02:01:31.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.725+0000 7f8ecf632640 1 -- start start 2024-09-12T02:01:31.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.725+0000 7f8ecf632640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8ec8100a80 0x7f8ec8199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:31.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.725+0000 7f8ecf632640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ec8101420 0x7f8ec81997b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:31.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.725+0000 7f8ecf632640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8ec8106a80 0x7f8ec81a42f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:31.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.725+0000 7f8ecf632640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8ec8110e80 con 0x7f8ec8101420 2024-09-12T02:01:31.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.725+0000 7f8ecf632640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8ec8110d00 con 0x7f8ec8100a80 2024-09-12T02:01:31.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.725+0000 7f8ecf632640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8ec8111000 con 0x7f8ec8106a80 2024-09-12T02:01:31.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.725+0000 7f8eccbcd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ec8101420 0x7f8ec81997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:31.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.725+0000 7f8eccbcd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ec8101420 0x7f8ec81997b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42572/0 (socket says 172.21.15.16:42572) 2024-09-12T02:01:31.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.725+0000 7f8eccbcd640 1 -- 172.21.15.16:0/3187249868 learned_addr learned my addr 172.21.15.16:0/3187249868 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:01:31.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.725+0000 7f8ecdbcf640 1 --2- 172.21.15.16:0/3187249868 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8ec8106a80 0x7f8ec81a42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:31.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.725+0000 7f8ecd3ce640 1 --2- 172.21.15.16:0/3187249868 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8ec8100a80 0x7f8ec8199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:31.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.726+0000 7f8eccbcd640 1 -- 172.21.15.16:0/3187249868 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8ec8106a80 msgr2=0x7f8ec81a42f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:31.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.726+0000 7f8eccbcd640 1 --2- 172.21.15.16:0/3187249868 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8ec8106a80 0x7f8ec81a42f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:31.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.726+0000 7f8eccbcd640 1 -- 172.21.15.16:0/3187249868 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8ec8100a80 msgr2=0x7f8ec8199270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:31.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.726+0000 7f8eccbcd640 1 --2- 172.21.15.16:0/3187249868 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8ec8100a80 0x7f8ec8199270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:31.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.726+0000 7f8eccbcd640 1 -- 172.21.15.16:0/3187249868 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ec81a4830 con 0x7f8ec8101420 2024-09-12T02:01:31.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.726+0000 7f8ecd3ce640 1 --2- 172.21.15.16:0/3187249868 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8ec8100a80 0x7f8ec8199270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:01:31.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.726+0000 7f8eccbcd640 1 --2- 172.21.15.16:0/3187249868 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ec8101420 0x7f8ec81997b0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f8eb800c890 tx=0x7f8eb800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:01:31.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.726+0000 7f8ebe7fc640 1 -- 172.21.15.16:0/3187249868 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eb8004220 con 0x7f8ec8101420 2024-09-12T02:01:31.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.726+0000 7f8ebe7fc640 1 -- 172.21.15.16:0/3187249868 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8eb80043c0 con 0x7f8ec8101420 2024-09-12T02:01:31.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.726+0000 7f8ecf632640 1 -- 172.21.15.16:0/3187249868 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ec81a4a00 con 0x7f8ec8101420 2024-09-12T02:01:31.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.726+0000 7f8ebe7fc640 1 -- 172.21.15.16:0/3187249868 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eb80049e0 con 0x7f8ec8101420 2024-09-12T02:01:31.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.726+0000 7f8ecf632640 1 -- 172.21.15.16:0/3187249868 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ec81a4f10 con 0x7f8ec8101420 2024-09-12T02:01:31.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.728+0000 7f8ebe7fc640 1 -- 172.21.15.16:0/3187249868 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f8eb80079b0 con 0x7f8ec8101420 2024-09-12T02:01:31.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.729+0000 7f8ed1a0b740 1 Processor -- start 2024-09-12T02:01:31.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.729+0000 7f8ed1a0b740 1 -- start start 2024-09-12T02:01:31.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.729+0000 7f8ebe7fc640 1 --2- 172.21.15.16:0/3187249868 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f8eb40a2c10 0x7f8eb40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:31.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.729+0000 7f8ed1a0b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a4e30ad8b0 0x55a4e30adc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:31.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.729+0000 7f8ed1a0b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a4e30ae1d0 0x55a4e30bca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:31.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.729+0000 7f8ed1a0b740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a4e30bcfc0 0x55a4e30bf3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:31.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.729+0000 7f8ed1a0b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55a4e3014090 con 0x55a4e30ae1d0 2024-09-12T02:01:31.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.729+0000 7f8ed1a0b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55a4e2fbba70 con 0x55a4e30ad8b0 2024-09-12T02:01:31.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.729+0000 7f8ed1a0b740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55a4e3010050 con 0x55a4e30bcfc0 2024-09-12T02:01:31.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.729+0000 7f8ebe7fc640 1 -- 172.21.15.16:0/3187249868 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f8eb80cf230 con 0x7f8ec8101420 2024-09-12T02:01:31.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.729+0000 7f8ecdbcf640 1 --2- 172.21.15.16:0/3187249868 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f8eb40a2c10 0x7f8eb40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:31.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.729+0000 7f8eccbcd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a4e30ae1d0 0x55a4e30bca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:31.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.729+0000 7f8eccbcd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a4e30ae1d0 0x55a4e30bca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42582/0 (socket says 172.21.15.16:42582) 2024-09-12T02:01:31.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.729+0000 7f8eccbcd640 1 -- 172.21.15.16:0/1301027442 learned_addr learned my addr 172.21.15.16:0/1301027442 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:01:31.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.730+0000 7f8ecdbcf640 1 --2- 172.21.15.16:0/1301027442 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a4e30bcfc0 0x55a4e30bf3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:31.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.730+0000 7f8ecd3ce640 1 --2- 172.21.15.16:0/1301027442 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a4e30ad8b0 0x55a4e30adc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:31.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.730+0000 7f8eccbcd640 1 -- 172.21.15.16:0/1301027442 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a4e30bcfc0 msgr2=0x55a4e30bf3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:01:31.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.730+0000 7f8eccbcd640 1 --2- 172.21.15.16:0/1301027442 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a4e30bcfc0 0x55a4e30bf3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:31.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.730+0000 7f8eccbcd640 1 -- 172.21.15.16:0/1301027442 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a4e30ad8b0 msgr2=0x55a4e30adc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:01:31.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.730+0000 7f8eccbcd640 1 --2- 172.21.15.16:0/1301027442 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a4e30ad8b0 0x55a4e30adc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:31.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.730+0000 7f8eccbcd640 1 -- 172.21.15.16:0/1301027442 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4e2fc8130 con 0x55a4e30ae1d0 2024-09-12T02:01:31.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.730+0000 7f8ecdbcf640 1 --2- 172.21.15.16:0/3187249868 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f8eb40a2c10 0x7f8eb40a50d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f8ec0004ab0 tx=0x7f8ec00317a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:01:31.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.730+0000 7f8eccbcd640 1 --2- 172.21.15.16:0/1301027442 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a4e30ae1d0 0x55a4e30bca80 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f8eb8098300 tx=0x7f8eb804a320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8b1b5203310757a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:01:31.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.730+0000 7f8eb37fe640 1 -- 172.21.15.16:0/1301027442 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eb801d040 con 0x55a4e30ae1d0 2024-09-12T02:01:31.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.730+0000 7f8eb37fe640 1 -- 172.21.15.16:0/1301027442 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8eb809b930 con 0x55a4e30ae1d0 2024-09-12T02:01:31.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.730+0000 7f8eb37fe640 1 -- 172.21.15.16:0/1301027442 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eb8086110 con 0x55a4e30ae1d0 2024-09-12T02:01:31.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.731+0000 7f8ed1a0b740 1 -- 172.21.15.16:0/1301027442 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a4e30ae1d0 msgr2=0x55a4e30bca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:31.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.731+0000 7f8ed1a0b740 1 --2- 172.21.15.16:0/1301027442 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a4e30ae1d0 0x55a4e30bca80 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f8eb8098300 tx=0x7f8eb804a320 comp rx=0 tx=0).stop 2024-09-12T02:01:31.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.731+0000 7f8ed1a0b740 1 -- 172.21.15.16:0/1301027442 shutdown_connections 2024-09-12T02:01:31.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.731+0000 7f8ed1a0b740 1 --2- 172.21.15.16:0/1301027442 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a4e30bcfc0 0x55a4e30bf3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:31.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.731+0000 7f8ed1a0b740 1 --2- 172.21.15.16:0/1301027442 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a4e30ae1d0 0x55a4e30bca80 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:31.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.731+0000 7f8ed1a0b740 1 --2- 172.21.15.16:0/1301027442 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a4e30ad8b0 0x55a4e30adc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:31.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.731+0000 7f8ed1a0b740 1 -- 172.21.15.16:0/1301027442 >> 172.21.15.16:0/1301027442 conn(0x55a4e30a9f60 msgr2=0x55a4e30aa340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:01:31.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.731+0000 7f8ed1a0b740 1 -- 172.21.15.16:0/1301027442 shutdown_connections 2024-09-12T02:01:31.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.731+0000 7f8ed1a0b740 1 -- 172.21.15.16:0/1301027442 wait complete. 2024-09-12T02:01:31.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.731+0000 7f8ed1a0b740 1 -- start start 2024-09-12T02:01:31.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.731+0000 7f8ed1a0b740 1 Processor -- start 2024-09-12T02:01:31.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.732+0000 7f8ed1a0b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a4e30ad8b0 0x55a4e30bc890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:31.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.732+0000 7f8ed1a0b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a4e30ae1d0 0x55a4e30b9be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:31.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.732+0000 7f8ed1a0b740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a4e30bcfc0 0x55a4e30ba120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:31.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.732+0000 7f8ed1a0b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55a4e30bfb90 con 0x55a4e30ae1d0 2024-09-12T02:01:31.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.732+0000 7f8ed1a0b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55a4e2fcef20 con 0x55a4e30ad8b0 2024-09-12T02:01:31.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.732+0000 7f8ed1a0b740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55a4e30bfd10 con 0x55a4e30bcfc0 2024-09-12T02:01:31.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.732+0000 7f8ed1a0b740 10 client.0 _ll_register_callbacks cb 0x55a4e2f98030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:01:31.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.732+0000 7f8eccbcd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a4e30ae1d0 0x55a4e30b9be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:31.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.732+0000 7f8eccbcd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a4e30ae1d0 0x55a4e30b9be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42594/0 (socket says 172.21.15.16:42594) 2024-09-12T02:01:31.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.732+0000 7f8eccbcd640 1 -- 172.21.15.16:0/1316450214 learned_addr learned my addr 172.21.15.16:0/1316450214 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:01:31.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.732+0000 7f8ecd3ce640 1 --2- 172.21.15.16:0/1316450214 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a4e30ad8b0 0x55a4e30bc890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:31.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.732+0000 7f8ecdbcf640 1 --2- 172.21.15.16:0/1316450214 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a4e30bcfc0 0x55a4e30ba120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:31.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.732+0000 7f8eccbcd640 1 -- 172.21.15.16:0/1316450214 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a4e30bcfc0 msgr2=0x55a4e30ba120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:01:31.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.732+0000 7f8eccbcd640 1 --2- 172.21.15.16:0/1316450214 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a4e30bcfc0 0x55a4e30ba120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:31.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.732+0000 7f8eccbcd640 1 -- 172.21.15.16:0/1316450214 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a4e30ad8b0 msgr2=0x55a4e30bc890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:01:31.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.732+0000 7f8eccbcd640 1 --2- 172.21.15.16:0/1316450214 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a4e30ad8b0 0x55a4e30bc890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:31.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.732+0000 7f8eccbcd640 1 -- 172.21.15.16:0/1316450214 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a4e2d852b0 con 0x55a4e30ae1d0 2024-09-12T02:01:31.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.732+0000 7f8eccbcd640 1 --2- 172.21.15.16:0/1316450214 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a4e30ae1d0 0x55a4e30b9be0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f8eb8097d90 tx=0x7f8eb8098180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32de146958a49a79 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:01:31.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.732+0000 7f8eb0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:01:31.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.732+0000 7f8eb0ff9640 1 -- 172.21.15.16:0/1316450214 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eb801d040 con 0x55a4e30ae1d0 2024-09-12T02:01:31.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.732+0000 7f8eb0ff9640 1 -- 172.21.15.16:0/1316450214 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f8eb809bb90 con 0x55a4e30ae1d0 2024-09-12T02:01:31.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.732+0000 7f8ed1a0b740 1 -- 172.21.15.16:0/1316450214 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8eb809b930 con 0x55a4e30ae1d0 2024-09-12T02:01:31.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.733+0000 7f8eb0ff9640 1 -- 172.21.15.16:0/1316450214 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f8eb80a45a0 con 0x55a4e30ae1d0 2024-09-12T02:01:31.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.733+0000 7f8ed1a0b740 10 client.15117 fetch_fsmap learned FSMap version 24 2024-09-12T02:01:31.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.733+0000 7f8ed1a0b740 1 -- 172.21.15.16:0/1316450214 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55a4e30cb520 con 0x55a4e30ae1d0 2024-09-12T02:01:31.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.733+0000 7f8eb0ff9640 1 -- 172.21.15.16:0/1316450214 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8eb809b930 con 0x55a4e30ae1d0 2024-09-12T02:01:31.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.733+0000 7f8eb0ff9640 1 -- 172.21.15.16:0/1316450214 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f8eb809ad80 con 0x55a4e30ae1d0 2024-09-12T02:01:31.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.733+0000 7f8ed1a0b740 10 client.15117 fetch_fsmap finished waiting for FSMap version 24 2024-09-12T02:01:31.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.733+0000 7f8ed1a0b740 10 client.15117 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-12T02:01:31.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.733+0000 7f8ed1a0b740 20 client.15117 populate_metadata read hostname 'smithi016' 2024-09-12T02:01:31.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.733+0000 7f8ed1a0b740 1 --2- 172.21.15.16:0/1316450214 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55a4e30cc870 0x55a4e30cea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:31.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.733+0000 7f8ed1a0b740 4 client.15117 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:01:31.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.733+0000 7f8ed1a0b740 1 -- 172.21.15.16:0/1316450214 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a4e3021600 con 0x55a4e30cc870 2024-09-12T02:01:31.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.733+0000 7f8ecd3ce640 1 --2- 172.21.15.16:0/1316450214 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55a4e30cc870 0x55a4e30cea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:31.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.734+0000 7f8eb0ff9640 10 client.15117 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:01:31.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.734+0000 7f8ecd3ce640 1 --2- 172.21.15.16:0/1316450214 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55a4e30cc870 0x55a4e30cea10 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ec3b05f87d48a26 server_cookie=c4e3cfce7d3417a4 in_seq=0 out_seq=0 2024-09-12T02:01:31.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.734+0000 7f8eb0ff9640 1 -- 172.21.15.16:0/1316450214 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a4e3021600 con 0x55a4e30cc870 2024-09-12T02:01:31.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.734+0000 7f8eb0ff9640 10 client.15117 handle_command_reply: tid=0 2024-09-12T02:01:31.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.735+0000 7f8ed1a0b740 1 client.15117 shutdown 2024-09-12T02:01:31.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.735+0000 7f8ed1a0b740 1 -- 172.21.15.16:0/1316450214 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a4e30ae1d0 msgr2=0x55a4e30b9be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:31.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.735+0000 7f8ed1a0b740 1 --2- 172.21.15.16:0/1316450214 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a4e30ae1d0 0x55a4e30b9be0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f8eb8097d90 tx=0x7f8eb8098180 comp rx=0 tx=0).stop 2024-09-12T02:01:31.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.735+0000 7f8ed1a0b740 1 -- 172.21.15.16:0/1316450214 shutdown_connections 2024-09-12T02:01:31.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.735+0000 7f8ed1a0b740 1 --2- 172.21.15.16:0/1316450214 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55a4e30cc870 0x55a4e30cea10 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:31.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.735+0000 7f8ed1a0b740 1 --2- 172.21.15.16:0/1316450214 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a4e30bcfc0 0x55a4e30ba120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:31.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.735+0000 7f8ed1a0b740 1 --2- 172.21.15.16:0/1316450214 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a4e30ae1d0 0x55a4e30b9be0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:31.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.735+0000 7f8eb0ff9640 0 client.15117 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:01:31.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.735+0000 7f8ed1a0b740 1 --2- 172.21.15.16:0/1316450214 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a4e30ad8b0 0x55a4e30bc890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:31.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.735+0000 7f8ed1a0b740 1 -- 172.21.15.16:0/1316450214 >> 172.21.15.16:0/1316450214 conn(0x55a4e30a9f60 msgr2=0x55a4e30afcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:01:31.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.735+0000 7f8ed1a0b740 1 -- 172.21.15.16:0/1316450214 shutdown_connections 2024-09-12T02:01:31.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.735+0000 7f8ed1a0b740 1 -- 172.21.15.16:0/1316450214 wait complete. 2024-09-12T02:01:31.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.735+0000 7f8ed1a0b740 20 client.15117 trim_cache size 0 max 16384 2024-09-12T02:01:31.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.753+0000 7f8ed1a0b740 1 Processor -- start 2024-09-12T02:01:31.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.753+0000 7f8ed1a0b740 1 -- start start 2024-09-12T02:01:31.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.753+0000 7f8ed1a0b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a4e30ad8b0 0x55a4e30cced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:31.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.753+0000 7f8ed1a0b740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a4e30ae1d0 0x55a4e30cdc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:31.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.753+0000 7f8ed1a0b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a4e30bcfc0 0x55a4e30b5130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:31.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.753+0000 7f8ed1a0b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55a4e30ccd50 con 0x55a4e30bcfc0 2024-09-12T02:01:31.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.753+0000 7f8ed1a0b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55a4e30cd420 con 0x55a4e30ad8b0 2024-09-12T02:01:31.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.753+0000 7f8ed1a0b740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55a4e30cdaf0 con 0x55a4e30ae1d0 2024-09-12T02:01:31.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.753+0000 7f8ecdbcf640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a4e30bcfc0 0x55a4e30b5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:31.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.753+0000 7f8ecdbcf640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a4e30bcfc0 0x55a4e30b5130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42610/0 (socket says 172.21.15.16:42610) 2024-09-12T02:01:31.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.753+0000 7f8ecdbcf640 1 -- 172.21.15.16:0/806952640 learned_addr learned my addr 172.21.15.16:0/806952640 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:01:31.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.754+0000 7f8eccbcd640 1 --2- 172.21.15.16:0/806952640 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a4e30ae1d0 0x55a4e30cdc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:31.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.754+0000 7f8ecd3ce640 1 --2- 172.21.15.16:0/806952640 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a4e30ad8b0 0x55a4e30cced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:31.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.754+0000 7f8ecdbcf640 1 -- 172.21.15.16:0/806952640 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a4e30ae1d0 msgr2=0x55a4e30cdc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:01:31.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.754+0000 7f8ecdbcf640 1 --2- 172.21.15.16:0/806952640 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a4e30ae1d0 0x55a4e30cdc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:31.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.754+0000 7f8ecdbcf640 1 -- 172.21.15.16:0/806952640 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a4e30ad8b0 msgr2=0x55a4e30cced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:01:31.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.754+0000 7f8ecdbcf640 1 --2- 172.21.15.16:0/806952640 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a4e30ad8b0 0x55a4e30cced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:31.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.754+0000 7f8ecdbcf640 1 -- 172.21.15.16:0/806952640 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4e30a97f0 con 0x55a4e30bcfc0 2024-09-12T02:01:31.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.754+0000 7f8ecdbcf640 1 --2- 172.21.15.16:0/806952640 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a4e30bcfc0 0x55a4e30b5130 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f8ec0009cd0 tx=0x7f8ec0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56244e0d7a70d75c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:01:31.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.754+0000 7f8eb2ffd640 1 -- 172.21.15.16:0/806952640 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec0004690 con 0x55a4e30bcfc0 2024-09-12T02:01:31.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.754+0000 7f8eb2ffd640 1 -- 172.21.15.16:0/806952640 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ec0036070 con 0x55a4e30bcfc0 2024-09-12T02:01:31.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.754+0000 7f8eb2ffd640 1 -- 172.21.15.16:0/806952640 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec003d770 con 0x55a4e30bcfc0 2024-09-12T02:01:31.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.755+0000 7f8ed1a0b740 1 -- 172.21.15.16:0/806952640 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a4e30bcfc0 msgr2=0x55a4e30b5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:31.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.755+0000 7f8ed1a0b740 1 --2- 172.21.15.16:0/806952640 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a4e30bcfc0 0x55a4e30b5130 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f8ec0009cd0 tx=0x7f8ec0004a40 comp rx=0 tx=0).stop 2024-09-12T02:01:31.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.755+0000 7f8ed1a0b740 1 -- 172.21.15.16:0/806952640 shutdown_connections 2024-09-12T02:01:31.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.755+0000 7f8ed1a0b740 1 --2- 172.21.15.16:0/806952640 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a4e30bcfc0 0x55a4e30b5130 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:31.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.755+0000 7f8ed1a0b740 1 --2- 172.21.15.16:0/806952640 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a4e30ae1d0 0x55a4e30cdc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:31.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.755+0000 7f8ed1a0b740 1 --2- 172.21.15.16:0/806952640 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a4e30ad8b0 0x55a4e30cced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:31.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.755+0000 7f8ed1a0b740 1 -- 172.21.15.16:0/806952640 >> 172.21.15.16:0/806952640 conn(0x55a4e30a9f60 msgr2=0x55a4e30b4a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:01:31.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.755+0000 7f8ed1a0b740 1 -- 172.21.15.16:0/806952640 shutdown_connections 2024-09-12T02:01:31.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.755+0000 7f8ed1a0b740 1 -- 172.21.15.16:0/806952640 wait complete. 2024-09-12T02:01:31.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.755+0000 7f8ed1a0b740 1 -- start start 2024-09-12T02:01:31.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.755+0000 7f8ed1a0b740 1 Processor -- start 2024-09-12T02:01:31.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.756+0000 7f8ed1a0b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a4e30ad8b0 0x55a4e30d75c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:31.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.756+0000 7f8ed1a0b740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a4e30ae1d0 0x55a4e30d09c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:31.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.756+0000 7f8ed1a0b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a4e30bcfc0 0x55a4e30d1100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:31.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.756+0000 7f8ed1a0b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55a4e30ce890 con 0x55a4e30ad8b0 2024-09-12T02:01:31.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.756+0000 7f8ed1a0b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55a4e30ce1c0 con 0x55a4e30bcfc0 2024-09-12T02:01:31.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.756+0000 7f8ed1a0b740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55a4e30cef60 con 0x55a4e30ae1d0 2024-09-12T02:01:31.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.756+0000 7f8ed1a0b740 10 client.0 _ll_register_callbacks cb 0x7f8eb809b520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:01:31.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.756+0000 7f8ecd3ce640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a4e30ad8b0 0x55a4e30d75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:31.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.756+0000 7f8ecd3ce640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a4e30ad8b0 0x55a4e30d75c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42622/0 (socket says 172.21.15.16:42622) 2024-09-12T02:01:31.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.756+0000 7f8ecd3ce640 1 -- 172.21.15.16:0/4230110482 learned_addr learned my addr 172.21.15.16:0/4230110482 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:01:31.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.756+0000 7f8eccbcd640 1 --2- 172.21.15.16:0/4230110482 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a4e30ae1d0 0x55a4e30d09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:31.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.756+0000 7f8ecdbcf640 1 --2- 172.21.15.16:0/4230110482 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a4e30bcfc0 0x55a4e30d1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:31.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.756+0000 7f8ecd3ce640 1 -- 172.21.15.16:0/4230110482 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a4e30ae1d0 msgr2=0x55a4e30d09c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:01:31.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.756+0000 7f8ecd3ce640 1 --2- 172.21.15.16:0/4230110482 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a4e30ae1d0 0x55a4e30d09c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:31.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.756+0000 7f8ecd3ce640 1 -- 172.21.15.16:0/4230110482 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a4e30bcfc0 msgr2=0x55a4e30d1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:01:31.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.756+0000 7f8ecd3ce640 1 --2- 172.21.15.16:0/4230110482 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a4e30bcfc0 0x55a4e30d1100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:31.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.756+0000 7f8ecd3ce640 1 -- 172.21.15.16:0/4230110482 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a4e30d50a0 con 0x55a4e30ad8b0 2024-09-12T02:01:31.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.756+0000 7f8ecdbcf640 1 --2- 172.21.15.16:0/4230110482 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a4e30bcfc0 0x55a4e30d1100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:01:31.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.756+0000 7f8ecd3ce640 1 --2- 172.21.15.16:0/4230110482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a4e30ad8b0 0x55a4e30d75c0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x55a4e2fcf070 tx=0x7f8ec401e9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e50b2dc0edc73f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:01:31.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.756+0000 7f8eb0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:01:31.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.757+0000 7f8eb0ff9640 1 -- 172.21.15.16:0/4230110482 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec4014b40 con 0x55a4e30ad8b0 2024-09-12T02:01:31.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.757+0000 7f8eb0ff9640 1 -- 172.21.15.16:0/4230110482 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f8ec4014ce0 con 0x55a4e30ad8b0 2024-09-12T02:01:31.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.757+0000 7f8ed1a0b740 1 -- 172.21.15.16:0/4230110482 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ec0036070 con 0x55a4e30ad8b0 2024-09-12T02:01:31.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.757+0000 7f8eb0ff9640 1 -- 172.21.15.16:0/4230110482 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f8ec401ec90 con 0x55a4e30ad8b0 2024-09-12T02:01:31.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.757+0000 7f8ed1a0b740 10 client.15123 fetch_fsmap learned FSMap version 24 2024-09-12T02:01:31.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.757+0000 7f8ed1a0b740 1 -- 172.21.15.16:0/4230110482 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55a4e3137e00 con 0x55a4e30ad8b0 2024-09-12T02:01:31.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.757+0000 7f8eb0ff9640 1 -- 172.21.15.16:0/4230110482 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ec0036070 con 0x55a4e30ad8b0 2024-09-12T02:01:31.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.757+0000 7f8eb0ff9640 1 -- 172.21.15.16:0/4230110482 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f8ec4004bb0 con 0x55a4e30ad8b0 2024-09-12T02:01:31.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.757+0000 7f8ed1a0b740 10 client.15123 fetch_fsmap finished waiting for FSMap version 24 2024-09-12T02:01:31.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.757+0000 7f8ed1a0b740 10 client.15123 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-12T02:01:31.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.757+0000 7f8ed1a0b740 20 client.15123 populate_metadata read hostname 'smithi016' 2024-09-12T02:01:31.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.757+0000 7f8ed1a0b740 1 --2- 172.21.15.16:0/4230110482 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55a4e313ca00 0x55a4e313b710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:31.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.757+0000 7f8ed1a0b740 4 client.15123 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T02:01:31.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.757+0000 7f8ed1a0b740 1 -- 172.21.15.16:0/4230110482 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "status"}) -- 0x55a4e301bb10 con 0x55a4e313ca00 2024-09-12T02:01:31.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.758+0000 7f8eccbcd640 1 --2- 172.21.15.16:0/4230110482 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55a4e313ca00 0x55a4e313b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:31.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.758+0000 7f8eccbcd640 1 --2- 172.21.15.16:0/4230110482 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55a4e313ca00 0x55a4e313b710 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33dcbb2a71959248 server_cookie=29b327b93b1dd48c in_seq=0 out_seq=0 2024-09-12T02:01:31.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.758+0000 7f8eb0ff9640 10 client.15123 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:01:31.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.758+0000 7f8eb0ff9640 1 -- 172.21.15.16:0/4230110482 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55a4e301bb10 con 0x55a4e313ca00 2024-09-12T02:01:31.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.758+0000 7f8eb0ff9640 10 client.15123 handle_command_reply: tid=0 2024-09-12T02:01:31.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.758+0000 7f8ed1a0b740 1 client.15123 shutdown 2024-09-12T02:01:31.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.758+0000 7f8ed1a0b740 1 -- 172.21.15.16:0/4230110482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a4e30ad8b0 msgr2=0x55a4e30d75c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:31.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.758+0000 7f8ed1a0b740 1 --2- 172.21.15.16:0/4230110482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a4e30ad8b0 0x55a4e30d75c0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x55a4e2fcf070 tx=0x7f8ec401e9e0 comp rx=0 tx=0).stop 2024-09-12T02:01:31.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.758+0000 7f8ed1a0b740 1 -- 172.21.15.16:0/4230110482 shutdown_connections 2024-09-12T02:01:31.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.758+0000 7f8ed1a0b740 1 --2- 172.21.15.16:0/4230110482 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55a4e313ca00 0x55a4e313b710 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:31.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.758+0000 7f8ed1a0b740 1 --2- 172.21.15.16:0/4230110482 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a4e30bcfc0 0x55a4e30d1100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:31.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.758+0000 7f8ed1a0b740 1 --2- 172.21.15.16:0/4230110482 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a4e30ae1d0 0x55a4e30d09c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:31.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.758+0000 7f8ed1a0b740 1 --2- 172.21.15.16:0/4230110482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a4e30ad8b0 0x55a4e30d75c0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:31.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.758+0000 7f8ed1a0b740 1 -- 172.21.15.16:0/4230110482 >> 172.21.15.16:0/4230110482 conn(0x55a4e30a9f60 msgr2=0x55a4e30b5240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:01:31.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.758+0000 7f8eb0ff9640 0 client.15123 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:01:31.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.759+0000 7f8ed1a0b740 1 -- 172.21.15.16:0/4230110482 shutdown_connections 2024-09-12T02:01:31.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.759+0000 7f8ed1a0b740 1 -- 172.21.15.16:0/4230110482 wait complete. 2024-09-12T02:01:31.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.759+0000 7f8ed1a0b740 20 client.15123 trim_cache size 0 max 16384 2024-09-12T02:01:31.764 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T02:01:31.764 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "879f3b56-70a9-11ef-bcea-c7b262605968", 2024-09-12T02:01:31.764 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 0, 2024-09-12T02:01:31.764 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 24482, 2024-09-12T02:01:31.764 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-09-12T02:01:31.764 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-09-12T02:01:31.764 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-09-12T02:01:31.764 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 95.266237798000006, 2024-09-12T02:01:31.764 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 24, 2024-09-12T02:01:31.764 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 82, 2024-09-12T02:01:31.764 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 80, 2024-09-12T02:01:31.764 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 105.516976228 2024-09-12T02:01:31.765 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T02:01:31.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.759+0000 7f8ecf632640 1 -- 172.21.15.16:0/3187249868 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f8eb40a2c10 msgr2=0x7f8eb40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:31.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.759+0000 7f8ecf632640 1 --2- 172.21.15.16:0/3187249868 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f8eb40a2c10 0x7f8eb40a50d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f8ec0004ab0 tx=0x7f8ec00317a0 comp rx=0 tx=0).stop 2024-09-12T02:01:31.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.759+0000 7f8ecf632640 1 -- 172.21.15.16:0/3187249868 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ec8101420 msgr2=0x7f8ec81997b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:31.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.759+0000 7f8ecf632640 1 --2- 172.21.15.16:0/3187249868 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ec8101420 0x7f8ec81997b0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f8eb800c890 tx=0x7f8eb800cd50 comp rx=0 tx=0).stop 2024-09-12T02:01:31.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.759+0000 7f8ecf632640 1 -- 172.21.15.16:0/3187249868 shutdown_connections 2024-09-12T02:01:31.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.759+0000 7f8ecf632640 1 --2- 172.21.15.16:0/3187249868 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f8eb40a2c10 0x7f8eb40a50d0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:31.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.759+0000 7f8ecf632640 1 --2- 172.21.15.16:0/3187249868 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8ec8106a80 0x7f8ec81a42f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:31.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.759+0000 7f8ecf632640 1 --2- 172.21.15.16:0/3187249868 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ec8101420 0x7f8ec81997b0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:31.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.759+0000 7f8ecf632640 1 --2- 172.21.15.16:0/3187249868 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8ec8100a80 0x7f8ec8199270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:31.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.759+0000 7f8ecf632640 1 -- 172.21.15.16:0/3187249868 >> 172.21.15.16:0/3187249868 conn(0x7f8ec80fc7a0 msgr2=0x7f8ec80fcec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:01:31.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.759+0000 7f8ecf632640 1 -- 172.21.15.16:0/3187249868 shutdown_connections 2024-09-12T02:01:31.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:31.759+0000 7f8ecf632640 1 -- 172.21.15.16:0/3187249868 wait complete. 2024-09-12T02:01:32.418 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:01:32 smithi033 ceph-mon[35332]: pgmap v323: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-12T02:01:32.420 INFO:teuthology.run_tasks:Running task sequential... 2024-09-12T02:01:32.432 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-09-12T02:01:32.432 INFO:teuthology.task.sequential:In sequential, running task print... 2024-09-12T02:01:32.433 INFO:teuthology.task.print:Enabling mgr modules 2024-09-12T02:01:32.433 INFO:teuthology.run_tasks:Running task kclient... 2024-09-12T02:01:32.535 INFO:tasks.kclient:Mounting kernel clients... 2024-09-12T02:01:32.535 INFO:tasks.kclient:config is {'client.0': {}} 2024-09-12T02:01:32.535 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T02:01:32.535 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2024-09-12T02:01:32.554 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T02:01:32.554 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link delete ceph-brx 2024-09-12T02:01:32.618 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:01:32 smithi016 ceph-mon[30947]: pgmap v323: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-12T02:01:32.625 INFO:teuthology.orchestra.run.smithi016.stderr:Cannot find device "ceph-brx" 2024-09-12T02:01:32.626 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T02:01:32.627 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=secure', 'wsync'], 'syntax': 'v1', 'mount_subvol_num': 1} 2024-09-12T02:01:32.627 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-12T02:01:32.627 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-12T02:01:32.627 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph fs ls 2024-09-12T02:01:32.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:01:32 smithi184 ceph-mon[35593]: pgmap v323: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-12T02:01:32.926 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:01:33.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:33.391+0000 7f320eec2640 1 -- 172.21.15.16:0/419688142 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3208100e20 msgr2=0x7f3208101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:33.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:33.391+0000 7f320eec2640 1 --2- 172.21.15.16:0/419688142 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3208100e20 0x7f3208101200 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f31fc009900 tx=0x7f31fc02a480 comp rx=0 tx=0).stop 2024-09-12T02:01:33.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:33.391+0000 7f320eec2640 1 -- 172.21.15.16:0/419688142 shutdown_connections 2024-09-12T02:01:33.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:33.391+0000 7f320eec2640 1 --2- 172.21.15.16:0/419688142 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3208102ef0 0x7f320810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:33.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:33.391+0000 7f320eec2640 1 --2- 172.21.15.16:0/419688142 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3208102570 0x7f32081029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:33.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:33.391+0000 7f320eec2640 1 --2- 172.21.15.16:0/419688142 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3208100e20 0x7f3208101200 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:33.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:33.391+0000 7f320eec2640 1 -- 172.21.15.16:0/419688142 >> 172.21.15.16:0/419688142 conn(0x7f32080fc7d0 msgr2=0x7f32080febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:01:33.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:33.391+0000 7f320eec2640 1 -- 172.21.15.16:0/419688142 shutdown_connections 2024-09-12T02:01:33.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:33.391+0000 7f320eec2640 1 -- 172.21.15.16:0/419688142 wait complete. 2024-09-12T02:01:33.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:33.392+0000 7f320eec2640 1 Processor -- start 2024-09-12T02:01:33.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:33.392+0000 7f320eec2640 1 -- start start 2024-09-12T02:01:33.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:33.392+0000 7f320eec2640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3208100e20 0x7f3208198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:33.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:33.392+0000 7f320eec2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3208102570 0x7f3208198bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:33.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:33.392+0000 7f320eec2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3208102ef0 0x7f320819cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:33.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:33.392+0000 7f320eec2640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3208111020 con 0x7f3208102570 2024-09-12T02:01:33.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:33.392+0000 7f320eec2640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3208110ea0 con 0x7f3208102ef0 2024-09-12T02:01:33.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:33.392+0000 7f320eec2640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f32081111a0 con 0x7f3208100e20 2024-09-12T02:01:33.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:33.392+0000 7f3207fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3208102570 0x7f3208198bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:33.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:33.392+0000 7f3207fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3208102570 0x7f3208198bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42630/0 (socket says 172.21.15.16:42630) 2024-09-12T02:01:33.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:33.392+0000 7f3207fff640 1 -- 172.21.15.16:0/2054920573 learned_addr learned my addr 172.21.15.16:0/2054920573 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:01:33.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:33.392+0000 7f320d45f640 1 --2- 172.21.15.16:0/2054920573 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3208102ef0 0x7f320819cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:33.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:33.393+0000 7f320cc5e640 1 --2- 172.21.15.16:0/2054920573 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3208100e20 0x7f3208198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:33.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:33.393+0000 7f3207fff640 1 -- 172.21.15.16:0/2054920573 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3208100e20 msgr2=0x7f3208198670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:33.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:33.393+0000 7f3207fff640 1 --2- 172.21.15.16:0/2054920573 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3208100e20 0x7f3208198670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:33.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:33.393+0000 7f3207fff640 1 -- 172.21.15.16:0/2054920573 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3208102ef0 msgr2=0x7f320819cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:33.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:33.393+0000 7f3207fff640 1 --2- 172.21.15.16:0/2054920573 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3208102ef0 0x7f320819cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:33.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:33.393+0000 7f3207fff640 1 -- 172.21.15.16:0/2054920573 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f320819d620 con 0x7f3208102570 2024-09-12T02:01:33.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:33.393+0000 7f3207fff640 1 --2- 172.21.15.16:0/2054920573 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3208102570 0x7f3208198bb0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f31f400b4a0 tx=0x7f31f400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:01:33.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:33.393+0000 7f3205ffb640 1 -- 172.21.15.16:0/2054920573 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31f4014070 con 0x7f3208102570 2024-09-12T02:01:33.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:33.393+0000 7f3205ffb640 1 -- 172.21.15.16:0/2054920573 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31f4010af0 con 0x7f3208102570 2024-09-12T02:01:33.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:33.393+0000 7f320eec2640 1 -- 172.21.15.16:0/2054920573 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f320819d910 con 0x7f3208102570 2024-09-12T02:01:33.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:33.393+0000 7f3205ffb640 1 -- 172.21.15.16:0/2054920573 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31f4017d10 con 0x7f3208102570 2024-09-12T02:01:33.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:33.393+0000 7f320eec2640 1 -- 172.21.15.16:0/2054920573 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32081a51f0 con 0x7f3208102570 2024-09-12T02:01:33.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:33.394+0000 7f320eec2640 1 -- 172.21.15.16:0/2054920573 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3208101280 con 0x7f3208102570 2024-09-12T02:01:33.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:33.395+0000 7f3205ffb640 1 -- 172.21.15.16:0/2054920573 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f31f4017070 con 0x7f3208102570 2024-09-12T02:01:33.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:33.396+0000 7f3205ffb640 1 --2- 172.21.15.16:0/2054920573 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f31f80a2c90 0x7f31f80a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:33.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:33.396+0000 7f320cc5e640 1 --2- 172.21.15.16:0/2054920573 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f31f80a2c90 0x7f31f80a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:33.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:33.397+0000 7f3205ffb640 1 -- 172.21.15.16:0/2054920573 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f31f40ce2b0 con 0x7f3208102570 2024-09-12T02:01:33.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:33.397+0000 7f320cc5e640 1 --2- 172.21.15.16:0/2054920573 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f31f80a2c90 0x7f31f80a5150 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f31fc005ab0 tx=0x7f31fc005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:01:33.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:33.401+0000 7f3205ffb640 1 -- 172.21.15.16:0/2054920573 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f31f409b2c0 con 0x7f3208102570 2024-09-12T02:01:33.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:33.615+0000 7f320eec2640 1 -- 172.21.15.16:0/2054920573 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f320810a0d0 con 0x7f3208102570 2024-09-12T02:01:33.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:33.615+0000 7f3205ffb640 1 -- 172.21.15.16:0/2054920573 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v24) ==== 53+0+73 (secure 0 0 0) 0x7f31f409abc0 con 0x7f3208102570 2024-09-12T02:01:33.615 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-12T02:01:33.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:33.618+0000 7f320eec2640 1 -- 172.21.15.16:0/2054920573 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f31f80a2c90 msgr2=0x7f31f80a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:33.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:33.618+0000 7f320eec2640 1 --2- 172.21.15.16:0/2054920573 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f31f80a2c90 0x7f31f80a5150 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f31fc005ab0 tx=0x7f31fc005a20 comp rx=0 tx=0).stop 2024-09-12T02:01:33.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:33.618+0000 7f320eec2640 1 -- 172.21.15.16:0/2054920573 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3208102570 msgr2=0x7f3208198bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:33.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:33.618+0000 7f320eec2640 1 --2- 172.21.15.16:0/2054920573 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3208102570 0x7f3208198bb0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f31f400b4a0 tx=0x7f31f400b960 comp rx=0 tx=0).stop 2024-09-12T02:01:33.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:33.618+0000 7f320eec2640 1 -- 172.21.15.16:0/2054920573 shutdown_connections 2024-09-12T02:01:33.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:33.618+0000 7f320eec2640 1 --2- 172.21.15.16:0/2054920573 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f31f80a2c90 0x7f31f80a5150 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:33.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:33.618+0000 7f320eec2640 1 --2- 172.21.15.16:0/2054920573 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3208102ef0 0x7f320819cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:33.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:33.618+0000 7f320eec2640 1 --2- 172.21.15.16:0/2054920573 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3208102570 0x7f3208198bb0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:33.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:33.618+0000 7f320eec2640 1 --2- 172.21.15.16:0/2054920573 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3208100e20 0x7f3208198670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:33.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:33.618+0000 7f320eec2640 1 -- 172.21.15.16:0/2054920573 >> 172.21.15.16:0/2054920573 conn(0x7f32080fc7d0 msgr2=0x7f320806a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:01:33.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:33.618+0000 7f320eec2640 1 -- 172.21.15.16:0/2054920573 shutdown_connections 2024-09-12T02:01:33.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:33.618+0000 7f320eec2640 1 -- 172.21.15.16:0/2054920573 wait complete. 2024-09-12T02:01:34.203 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-12T02:01:34.204 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-12T02:01:34.204 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi016.front.sepia.ceph.com 2024-09-12T02:01:34.204 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-12T02:01:34.204 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-12T02:01:34.204 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-12T02:01:34.204 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-12T02:01:34.204 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-12T02:01:34.204 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-12T02:01:34.204 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T02:01:34.204 DEBUG:teuthology.orchestra.run.smithi016:> ip addr 2024-09-12T02:01:34.224 INFO:teuthology.orchestra.run.smithi016.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-12T02:01:34.224 INFO:teuthology.orchestra.run.smithi016.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-12T02:01:34.225 INFO:teuthology.orchestra.run.smithi016.stdout: inet 127.0.0.1/8 scope host lo 2024-09-12T02:01:34.225 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-12T02:01:34.225 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 ::1/128 scope host 2024-09-12T02:01:34.225 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-12T02:01:34.225 INFO:teuthology.orchestra.run.smithi016.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-12T02:01:34.225 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e0 brd ff:ff:ff:ff:ff:ff 2024-09-12T02:01:34.225 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f0 2024-09-12T02:01:34.225 INFO:teuthology.orchestra.run.smithi016.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-12T02:01:34.225 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e1 brd ff:ff:ff:ff:ff:ff 2024-09-12T02:01:34.225 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f1 2024-09-12T02:01:34.225 INFO:teuthology.orchestra.run.smithi016.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-12T02:01:34.225 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:64 brd ff:ff:ff:ff:ff:ff 2024-09-12T02:01:34.225 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f0 2024-09-12T02:01:34.225 INFO:teuthology.orchestra.run.smithi016.stdout: inet 172.21.15.16/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-12T02:01:34.226 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft 42223sec preferred_lft 42223sec 2024-09-12T02:01:34.226 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::ec4:7aff:febd:1564/64 scope link 2024-09-12T02:01:34.226 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-12T02:01:34.226 INFO:teuthology.orchestra.run.smithi016.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-12T02:01:34.226 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:65 brd ff:ff:ff:ff:ff:ff 2024-09-12T02:01:34.226 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f1 2024-09-12T02:01:34.226 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-12T02:01:34.227 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T02:01:34.227 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-09-12T02:01:34.227 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link add name ceph-brx type bridge 2024-09-12T02:01:34.227 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip addr flush dev ceph-brx 2024-09-12T02:01:34.227 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set ceph-brx up 2024-09-12T02:01:34.227 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-12T02:01:34.227 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-09-12T02:01:34.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:34 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T02:01:34.428 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:01:34 smithi016 ceph-mon[30947]: pgmap v324: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-12T02:01:34.428 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:01:34 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/2054920573' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-12T02:01:34.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T02:01:34.439 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T02:01:34.439 DEBUG:teuthology.orchestra.run.smithi016:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-12T02:01:34.465 INFO:teuthology.orchestra.run.smithi016.stdout:1 2024-09-12T02:01:34.466 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T02:01:34.467 DEBUG:teuthology.orchestra.run.smithi016:> route 2024-09-12T02:01:34.545 INFO:teuthology.orchestra.run.smithi016.stdout:Kernel IP routing table 2024-09-12T02:01:34.545 INFO:teuthology.orchestra.run.smithi016.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-12T02:01:34.545 INFO:teuthology.orchestra.run.smithi016.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-12T02:01:34.545 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-12T02:01:34.546 INFO:teuthology.orchestra.run.smithi016.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-12T02:01:34.546 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T02:01:34.546 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-09-12T02:01:34.546 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-12T02:01:34.546 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-12T02:01:34.546 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-12T02:01:34.546 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-09-12T02:01:34.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:01:34 smithi184 ceph-mon[35593]: pgmap v324: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-12T02:01:34.628 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:01:34 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/2054920573' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-12T02:01:34.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:34 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T02:01:34.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:01:34 smithi033 ceph-mon[35332]: pgmap v324: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-12T02:01:34.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:01:34 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/2054920573' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-12T02:01:34.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T02:01:34.830 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T02:01:34.830 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2024-09-12T02:01:34.848 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T02:01:34.848 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list-id 2024-09-12T02:01:34.906 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T02:01:34.906 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-09-12T02:01:34.906 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-12T02:01:34.906 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-09-12T02:01:34.906 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-09-12T02:01:34.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:34 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T02:01:35.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T02:01:35.029 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-09-12T02:01:35.030 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T02:01:35.030 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-09-12T02:01:35.030 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-09-12T02:01:35.030 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-12T02:01:35.030 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-09-12T02:01:35.031 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-09-12T02:01:35.031 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-09-12T02:01:35.031 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-09-12T02:01:35.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:35 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T02:01:35.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T02:01:35.164 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T02:01:35.164 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-09-12T02:01:35.164 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set brx.0 up 2024-09-12T02:01:35.164 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set dev brx.0 master ceph-brx 2024-09-12T02:01:35.165 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-09-12T02:01:35.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:35 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T02:01:35.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T02:01:35.244 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-12T02:01:35.244 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-12T02:01:35.260 INFO:teuthology.orchestra.run.smithi016.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-12T02:01:35.260 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-12T02:01:35.260 DEBUG:teuthology.orchestra.run.smithi016:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-12T02:01:35.317 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-09-12T02:01:35.317 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-09-12T02:01:35.317 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/qa/sv_1/a1673dc0-6b08-43d7-895c-025fe8df6ce5 2024-09-12T02:01:35.317 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T02:01:35.317 DEBUG:teuthology.orchestra.run.smithi016:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/qa/sv_1/a1673dc0-6b08-43d7-895c-025fe8df6ce5 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync 2024-09-12T02:01:35.425 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync 2024-09-12T02:01:35.425 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync". 2024-09-12T02:01:35.425 INFO:teuthology.orchestra.run.smithi016.stdout:invalid new device string format 2024-09-12T02:01:35.552 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync 2024-09-12T02:01:35.553 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync". 2024-09-12T02:01:35.553 INFO:teuthology.orchestra.run.smithi016.stdout:invalid new device string format 2024-09-12T02:01:35.553 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: resolved to: "172.21.15.16:3300,172.21.15.33:3300,172.21.15.184:3300" 2024-09-12T02:01:35.553 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.16:3300,172.21.15.33:3300,172.21.15.184:3300:/volumes/qa/sv_1/a1673dc0-6b08-43d7-895c-025fe8df6ce5 2024-09-12T02:01:35.553 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync,key=0,fsid=879f3b56-70a9-11ef-bcea-c7b262605968" will pass to kernel 2024-09-12T02:01:35.554 INFO:teuthology.orchestra.run.smithi016.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-12T02:01:35.554 INFO:teuthology.orchestra.run.smithi016.stdout: You just mounted a file system that supports labels which does not 2024-09-12T02:01:35.554 INFO:teuthology.orchestra.run.smithi016.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-12T02:01:35.554 INFO:teuthology.orchestra.run.smithi016.stdout: applications will generate AVC messages and not be allowed access to 2024-09-12T02:01:35.554 INFO:teuthology.orchestra.run.smithi016.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-12T02:01:35.556 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-12T02:01:35.556 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T02:01:35.556 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-12T02:01:35.584 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-12T02:01:35.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:35 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-12T02:01:35.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T02:01:35.690 INFO:teuthology.orchestra.run.smithi016.stdout:client15132 2024-09-12T02:01:35.691 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-12T02:01:35.691 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T02:01:35.709 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-12T02:01:35.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:35 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-12T02:01:35.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T02:01:35.813 INFO:teuthology.orchestra.run.smithi016.stdout:client15132 2024-09-12T02:01:35.814 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-12T02:01:35.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:35 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-12T02:01:35.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T02:01:35.878 INFO:teuthology.orchestra.run.smithi016.stdout:client15132 2024-09-12T02:01:35.879 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/879f3b56-70a9-11ef-bcea-c7b262605968.client15132/status') 2024-09-12T02:01:35.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:35 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T02:01:35.944 INFO:teuthology.orchestra.run.smithi016.stdout:instance: client.15132 (3)192.168.0.1:0/1436690633 2024-09-12T02:01:35.944 INFO:teuthology.orchestra.run.smithi016.stdout:blocklisted: false 2024-09-12T02:01:35.944 INFO:teuthology.orchestra.run.smithi016.stderr:0+1 records in 2024-09-12T02:01:35.944 INFO:teuthology.orchestra.run.smithi016.stderr:0+1 records out 2024-09-12T02:01:35.945 INFO:teuthology.orchestra.run.smithi016.stderr:70 bytes copied, 2.4308e-05 s, 2.9 MB/s 2024-09-12T02:01:35.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T02:01:35.952 INFO:teuthology.run_tasks:Running task exec... 2024-09-12T02:01:35.962 INFO:teuthology.task.exec:Executing custom commands... 2024-09-12T02:01:35.962 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi016.front.sepia.ceph.com 2024-09-12T02:01:35.962 DEBUG:teuthology.orchestra.run.smithi016:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate true' 2024-09-12T02:01:36.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:36.067+0000 7ff879331640 1 -- 172.21.15.16:0/2144087715 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff874101710 msgr2=0x7ff874101bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:36.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:36.067+0000 7ff879331640 1 --2- 172.21.15.16:0/2144087715 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff874101710 0x7ff874101bc0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7ff86c009900 tx=0x7ff86c02a480 comp rx=0 tx=0).stop 2024-09-12T02:01:36.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:36.067+0000 7ff879331640 1 -- 172.21.15.16:0/2144087715 shutdown_connections 2024-09-12T02:01:36.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:36.067+0000 7ff879331640 1 --2- 172.21.15.16:0/2144087715 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff874101710 0x7ff874101bc0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:36.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:36.067+0000 7ff879331640 1 --2- 172.21.15.16:0/2144087715 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff874100d70 0x7ff8741011d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:36.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:36.067+0000 7ff879331640 1 --2- 172.21.15.16:0/2144087715 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff874100430 0x7ff874100830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:36.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:36.067+0000 7ff879331640 1 -- 172.21.15.16:0/2144087715 >> 172.21.15.16:0/2144087715 conn(0x7ff8740fc1e0 msgr2=0x7ff8740fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:01:36.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:36.067+0000 7ff879331640 1 -- 172.21.15.16:0/2144087715 shutdown_connections 2024-09-12T02:01:36.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:36.067+0000 7ff879331640 1 -- 172.21.15.16:0/2144087715 wait complete. 2024-09-12T02:01:36.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:36.067+0000 7ff879331640 1 Processor -- start 2024-09-12T02:01:36.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:36.068+0000 7ff879331640 1 -- start start 2024-09-12T02:01:36.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:36.068+0000 7ff879331640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff874100430 0x7ff874199fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:36.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:36.068+0000 7ff879331640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff874100d70 0x7ff87419a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:36.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:36.068+0000 7ff879331640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff874101710 0x7ff8741a1560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:36.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:36.068+0000 7ff879331640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff87410f9c0 con 0x7ff874101710 2024-09-12T02:01:36.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:36.068+0000 7ff879331640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff87410f840 con 0x7ff874100430 2024-09-12T02:01:36.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:36.068+0000 7ff879331640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff87410fb40 con 0x7ff874100d70 2024-09-12T02:01:36.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:36.068+0000 7ff8737fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff874101710 0x7ff8741a1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:36.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:36.068+0000 7ff8737fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff874101710 0x7ff8741a1560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42650/0 (socket says 172.21.15.16:42650) 2024-09-12T02:01:36.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:36.068+0000 7ff8737fe640 1 -- 172.21.15.16:0/800813287 learned_addr learned my addr 172.21.15.16:0/800813287 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:01:36.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:36.068+0000 7ff8727fc640 1 --2- 172.21.15.16:0/800813287 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff874100d70 0x7ff87419a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:36.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:36.068+0000 7ff872ffd640 1 --2- 172.21.15.16:0/800813287 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff874100430 0x7ff874199fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:36.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:36.068+0000 7ff8737fe640 1 -- 172.21.15.16:0/800813287 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff874100d70 msgr2=0x7ff87419a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:36.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:36.068+0000 7ff8737fe640 1 --2- 172.21.15.16:0/800813287 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff874100d70 0x7ff87419a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:36.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:36.068+0000 7ff8737fe640 1 -- 172.21.15.16:0/800813287 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff874100430 msgr2=0x7ff874199fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:36.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:36.068+0000 7ff8737fe640 1 --2- 172.21.15.16:0/800813287 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff874100430 0x7ff874199fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:36.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:36.068+0000 7ff8737fe640 1 -- 172.21.15.16:0/800813287 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8741a1bd0 con 0x7ff874101710 2024-09-12T02:01:36.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:36.069+0000 7ff8737fe640 1 --2- 172.21.15.16:0/800813287 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff874101710 0x7ff8741a1560 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7ff86c0314e0 tx=0x7ff86c004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:01:36.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:36.069+0000 7ff85bfff640 1 -- 172.21.15.16:0/800813287 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff86c03f030 con 0x7ff874101710 2024-09-12T02:01:36.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:36.069+0000 7ff85bfff640 1 -- 172.21.15.16:0/800813287 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff86c003ce0 con 0x7ff874101710 2024-09-12T02:01:36.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:36.069+0000 7ff879331640 1 -- 172.21.15.16:0/800813287 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8741a1e60 con 0x7ff874101710 2024-09-12T02:01:36.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:36.069+0000 7ff85bfff640 1 -- 172.21.15.16:0/800813287 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff86c035070 con 0x7ff874101710 2024-09-12T02:01:36.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:36.069+0000 7ff879331640 1 -- 172.21.15.16:0/800813287 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8741a22c0 con 0x7ff874101710 2024-09-12T02:01:36.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:36.070+0000 7ff879331640 1 -- 172.21.15.16:0/800813287 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff87410b180 con 0x7ff874101710 2024-09-12T02:01:36.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:36.071+0000 7ff85bfff640 1 -- 172.21.15.16:0/800813287 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7ff86c042030 con 0x7ff874101710 2024-09-12T02:01:36.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:36.072+0000 7ff85bfff640 1 --2- 172.21.15.16:0/800813287 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7ff85c0a2cd0 0x7ff85c0a5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:36.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:36.072+0000 7ff872ffd640 1 --2- 172.21.15.16:0/800813287 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7ff85c0a2cd0 0x7ff85c0a5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:36.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:36.072+0000 7ff85bfff640 1 -- 172.21.15.16:0/800813287 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7ff86c0efbc0 con 0x7ff874101710 2024-09-12T02:01:36.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:36.072+0000 7ff872ffd640 1 --2- 172.21.15.16:0/800813287 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7ff85c0a2cd0 0x7ff85c0a5190 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ff864005fd0 tx=0x7ff864009870 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:01:36.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:36.075+0000 7ff85bfff640 1 -- 172.21.15.16:0/800813287 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff86c0bc9c0 con 0x7ff874101710 2024-09-12T02:01:36.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:36.293+0000 7ff879331640 1 -- 172.21.15.16:0/800813287 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} v 0) -- 0x7ff87402d7c0 con 0x7ff874101710 2024-09-12T02:01:36.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:01:36 smithi016 ceph-mon[30947]: pgmap v325: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-12T02:01:36.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:01:36 smithi184 ceph-mon[35593]: pgmap v325: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-12T02:01:36.668 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:01:36 smithi033 ceph-mon[35332]: pgmap v325: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-12T02:01:37.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:37.198+0000 7ff85bfff640 1 -- 172.21.15.16:0/800813287 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]=0 v25) ==== 117+0+0 (secure 0 0 0) 0x7ff86c0bc2c0 con 0x7ff874101710 2024-09-12T02:01:37.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:37.201+0000 7ff879331640 1 -- 172.21.15.16:0/800813287 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7ff85c0a2cd0 msgr2=0x7ff85c0a5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:37.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:37.201+0000 7ff879331640 1 --2- 172.21.15.16:0/800813287 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7ff85c0a2cd0 0x7ff85c0a5190 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ff864005fd0 tx=0x7ff864009870 comp rx=0 tx=0).stop 2024-09-12T02:01:37.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:37.201+0000 7ff879331640 1 -- 172.21.15.16:0/800813287 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff874101710 msgr2=0x7ff8741a1560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:37.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:37.201+0000 7ff879331640 1 --2- 172.21.15.16:0/800813287 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff874101710 0x7ff8741a1560 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7ff86c0314e0 tx=0x7ff86c004360 comp rx=0 tx=0).stop 2024-09-12T02:01:37.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:37.201+0000 7ff879331640 1 -- 172.21.15.16:0/800813287 shutdown_connections 2024-09-12T02:01:37.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:37.201+0000 7ff879331640 1 --2- 172.21.15.16:0/800813287 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7ff85c0a2cd0 0x7ff85c0a5190 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:37.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:37.201+0000 7ff879331640 1 --2- 172.21.15.16:0/800813287 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff874101710 0x7ff8741a1560 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:37.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:37.201+0000 7ff879331640 1 --2- 172.21.15.16:0/800813287 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff874100d70 0x7ff87419a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:37.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:37.201+0000 7ff879331640 1 --2- 172.21.15.16:0/800813287 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff874100430 0x7ff874199fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:37.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:37.202+0000 7ff879331640 1 -- 172.21.15.16:0/800813287 >> 172.21.15.16:0/800813287 conn(0x7ff8740fc1e0 msgr2=0x7ff8740fdc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:01:37.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:37.202+0000 7ff879331640 1 -- 172.21.15.16:0/800813287 shutdown_connections 2024-09-12T02:01:37.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:37.202+0000 7ff879331640 1 -- 172.21.15.16:0/800813287 wait complete. 2024-09-12T02:01:37.215 INFO:teuthology.run_tasks:Running task check-counter... 2024-09-12T02:01:37.225 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-09-12T02:01:37.422 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T02:01:37.450 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:01:37 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/800813287' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-09-12T02:01:37.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:37.541+0000 7fe88f3b7640 1 -- 172.21.15.16:0/2603471204 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe888103780 msgr2=0x7fe88810a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:37.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:37.541+0000 7fe88f3b7640 1 --2- 172.21.15.16:0/2603471204 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe888103780 0x7fe88810a010 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fe880009900 tx=0x7fe88002a410 comp rx=0 tx=0).stop 2024-09-12T02:01:37.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:37.541+0000 7fe88f3b7640 1 -- 172.21.15.16:0/2603471204 shutdown_connections 2024-09-12T02:01:37.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:37.541+0000 7fe88f3b7640 1 --2- 172.21.15.16:0/2603471204 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe888103780 0x7fe88810a010 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:37.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:37.541+0000 7fe88f3b7640 1 --2- 172.21.15.16:0/2603471204 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe888102dc0 0x7fe888103240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:37.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:37.541+0000 7fe88f3b7640 1 --2- 172.21.15.16:0/2603471204 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe8880ffdf0 0x7fe8881001f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:37.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:37.541+0000 7fe88f3b7640 1 -- 172.21.15.16:0/2603471204 >> 172.21.15.16:0/2603471204 conn(0x7fe8880fb910 msgr2=0x7fe8880fdd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:01:37.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:37.541+0000 7fe88f3b7640 1 -- 172.21.15.16:0/2603471204 shutdown_connections 2024-09-12T02:01:37.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:37.541+0000 7fe88f3b7640 1 -- 172.21.15.16:0/2603471204 wait complete. 2024-09-12T02:01:37.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:37.542+0000 7fe88f3b7640 1 Processor -- start 2024-09-12T02:01:37.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:37.542+0000 7fe88f3b7640 1 -- start start 2024-09-12T02:01:37.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:37.542+0000 7fe88f3b7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8880ffdf0 0x7fe888197e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:37.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:37.542+0000 7fe88f3b7640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe888102dc0 0x7fe888198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:37.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:37.542+0000 7fe88f3b7640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe888103780 0x7fe88819f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:37.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:37.542+0000 7fe88f3b7640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe88810d820 con 0x7fe8880ffdf0 2024-09-12T02:01:37.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:37.542+0000 7fe88f3b7640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe88810d6a0 con 0x7fe888102dc0 2024-09-12T02:01:37.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:37.542+0000 7fe88f3b7640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe88810d9a0 con 0x7fe888103780 2024-09-12T02:01:37.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:37.542+0000 7fe88d153640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8880ffdf0 0x7fe888197e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:37.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:37.542+0000 7fe88d153640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8880ffdf0 0x7fe888197e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42668/0 (socket says 172.21.15.16:42668) 2024-09-12T02:01:37.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:37.542+0000 7fe88d153640 1 -- 172.21.15.16:0/1887785245 learned_addr learned my addr 172.21.15.16:0/1887785245 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:01:37.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:37.542+0000 7fe88d954640 1 --2- 172.21.15.16:0/1887785245 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe888103780 0x7fe88819f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:37.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:37.542+0000 7fe88c952640 1 --2- 172.21.15.16:0/1887785245 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe888102dc0 0x7fe888198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:37.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:37.542+0000 7fe88d153640 1 -- 172.21.15.16:0/1887785245 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe888103780 msgr2=0x7fe88819f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:37.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:37.542+0000 7fe88d153640 1 --2- 172.21.15.16:0/1887785245 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe888103780 0x7fe88819f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:37.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:37.542+0000 7fe88d153640 1 -- 172.21.15.16:0/1887785245 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe888102dc0 msgr2=0x7fe888198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:37.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:37.542+0000 7fe88d153640 1 --2- 172.21.15.16:0/1887785245 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe888102dc0 0x7fe888198380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:37.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:37.542+0000 7fe88d153640 1 -- 172.21.15.16:0/1887785245 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe88819fa60 con 0x7fe8880ffdf0 2024-09-12T02:01:37.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:37.542+0000 7fe88c952640 1 --2- 172.21.15.16:0/1887785245 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe888102dc0 0x7fe888198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:01:37.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:37.543+0000 7fe88d153640 1 --2- 172.21.15.16:0/1887785245 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8880ffdf0 0x7fe888197e40 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fe88400ccb0 tx=0x7fe88400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:01:37.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:37.543+0000 7fe87e7fc640 1 -- 172.21.15.16:0/1887785245 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe884003b70 con 0x7fe8880ffdf0 2024-09-12T02:01:37.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:37.543+0000 7fe87e7fc640 1 -- 172.21.15.16:0/1887785245 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe88400ae50 con 0x7fe8880ffdf0 2024-09-12T02:01:37.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:37.543+0000 7fe88f3b7640 1 -- 172.21.15.16:0/1887785245 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe88819fd50 con 0x7fe8880ffdf0 2024-09-12T02:01:37.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:37.543+0000 7fe87e7fc640 1 -- 172.21.15.16:0/1887785245 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe88400f810 con 0x7fe8880ffdf0 2024-09-12T02:01:37.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:37.543+0000 7fe88f3b7640 1 -- 172.21.15.16:0/1887785245 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8881a0290 con 0x7fe8880ffdf0 2024-09-12T02:01:37.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:37.544+0000 7fe88f3b7640 1 -- 172.21.15.16:0/1887785245 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe888108fe0 con 0x7fe8880ffdf0 2024-09-12T02:01:37.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:37.546+0000 7fe87e7fc640 1 -- 172.21.15.16:0/1887785245 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fe88400f9b0 con 0x7fe8880ffdf0 2024-09-12T02:01:37.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:37.546+0000 7fe87e7fc640 1 --2- 172.21.15.16:0/1887785245 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fe8740a2ca0 0x7fe8740a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:37.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:37.547+0000 7fe88c952640 1 --2- 172.21.15.16:0/1887785245 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fe8740a2ca0 0x7fe8740a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:37.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:37.547+0000 7fe87e7fc640 1 -- 172.21.15.16:0/1887785245 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fe8840cf690 con 0x7fe8880ffdf0 2024-09-12T02:01:37.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:37.547+0000 7fe88c952640 1 --2- 172.21.15.16:0/1887785245 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fe8740a2ca0 0x7fe8740a5160 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fe878006fd0 tx=0x7fe878008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:01:37.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:37.550+0000 7fe87e7fc640 1 -- 172.21.15.16:0/1887785245 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe884014070 con 0x7fe8880ffdf0 2024-09-12T02:01:37.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:01:37 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/800813287' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-09-12T02:01:37.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:01:37 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/800813287' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-09-12T02:01:37.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:37.728+0000 7fe88f3b7640 1 -- 172.21.15.16:0/1887785245 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe88802da70 con 0x7fe8880ffdf0 2024-09-12T02:01:37.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:37.730+0000 7fe87e7fc640 1 -- 172.21.15.16:0/1887785245 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+22584 (secure 0 0 0) 0x7fe88409c510 con 0x7fe8880ffdf0 2024-09-12T02:01:37.730 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T02:01:37.730 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":82,"fsid":"879f3b56-70a9-11ef-bcea-c7b262605968","created":"2024-09-12T01:52:12.841045+0000","modified":"2024-09-12T02:00:09.559511+0000","last_up_change":"2024-09-12T01:59:08.975443+0000","last_in_change":"2024-09-12T01:58:44.530220+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T01:55:26.727564+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T01:59:52.402097+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T01:59:54.463627+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3ea41eb6-e384-4ddc-810f-0f103c127849","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4000187867},{"type":"v1","addr":"172.21.15.16:6803","nonce":4000187867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4000187867},{"type":"v1","addr":"172.21.15.16:6805","nonce":4000187867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":4000187867},{"type":"v1","addr":"172.21.15.16:6809","nonce":4000187867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4000187867},{"type":"v1","addr":"172.21.15.16:6807","nonce":4000187867}]},"public_addr":"172.21.15.16:6803/4000187867","cluster_addr":"172.21.15.16:6805/4000187867","heartbeat_back_addr":"172.21.15.16:6809/4000187867","heartbeat_front_addr":"172.21.15.16:6807/4000187867","state":["exists","up"]},{"osd":1,"uuid":"07bf63e7-70fa-41b2-af53-12a5fec3f33c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":3862661148},{"type":"v1","addr":"172.21.15.33:6801","nonce":3862661148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":3862661148},{"type":"v1","addr":"172.21.15.33:6803","nonce":3862661148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":3862661148},{"type":"v1","addr":"172.21.15.33:6807","nonce":3862661148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":3862661148},{"type":"v1","addr":"172.21.15.33:6805","nonce":3862661148}]},"public_addr":"172.21.15.33:6801/3862661148","cluster_addr":"172.21.15.33:6803/3862661148","heartbeat_back_addr":"172.21.15.33:6807/3862661148","heartbeat_front_addr":"172.21.15.33:6805/3862661148","state":["exists","up"]},{"osd":2,"uuid":"22e40cac-3bc0-4cce-b47b-77499b767f7e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":3958753963},{"type":"v1","addr":"172.21.15.184:6801","nonce":3958753963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":3958753963},{"type":"v1","addr":"172.21.15.184:6803","nonce":3958753963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":3958753963},{"type":"v1","addr":"172.21.15.184:6807","nonce":3958753963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":3958753963},{"type":"v1","addr":"172.21.15.184:6805","nonce":3958753963}]},"public_addr":"172.21.15.184:6801/3958753963","cluster_addr":"172.21.15.184:6803/3958753963","heartbeat_back_addr":"172.21.15.184:6807/3958753963","heartbeat_front_addr":"172.21.15.184:6805/3958753963","state":["exists","up"]},{"osd":3,"uuid":"834a07f5-2c24-49e1-8e4a-12c96cfbdc65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3395182524},{"type":"v1","addr":"172.21.15.16:6811","nonce":3395182524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3395182524},{"type":"v1","addr":"172.21.15.16:6813","nonce":3395182524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3395182524},{"type":"v1","addr":"172.21.15.16:6817","nonce":3395182524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3395182524},{"type":"v1","addr":"172.21.15.16:6815","nonce":3395182524}]},"public_addr":"172.21.15.16:6811/3395182524","cluster_addr":"172.21.15.16:6813/3395182524","heartbeat_back_addr":"172.21.15.16:6817/3395182524","heartbeat_front_addr":"172.21.15.16:6815/3395182524","state":["exists","up"]},{"osd":4,"uuid":"3ca56e09-c194-4404-b9bd-2524d38367eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":1355184322},{"type":"v1","addr":"172.21.15.33:6809","nonce":1355184322}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1355184322},{"type":"v1","addr":"172.21.15.33:6811","nonce":1355184322}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1355184322},{"type":"v1","addr":"172.21.15.33:6815","nonce":1355184322}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1355184322},{"type":"v1","addr":"172.21.15.33:6813","nonce":1355184322}]},"public_addr":"172.21.15.33:6809/1355184322","cluster_addr":"172.21.15.33:6811/1355184322","heartbeat_back_addr":"172.21.15.33:6815/1355184322","heartbeat_front_addr":"172.21.15.33:6813/1355184322","state":["exists","up"]},{"osd":5,"uuid":"0b05ecdb-b987-461f-951c-6d76739bfb4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":1533397276},{"type":"v1","addr":"172.21.15.184:6809","nonce":1533397276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":1533397276},{"type":"v1","addr":"172.21.15.184:6811","nonce":1533397276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":1533397276},{"type":"v1","addr":"172.21.15.184:6815","nonce":1533397276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":1533397276},{"type":"v1","addr":"172.21.15.184:6813","nonce":1533397276}]},"public_addr":"172.21.15.184:6809/1533397276","cluster_addr":"172.21.15.184:6811/1533397276","heartbeat_back_addr":"172.21.15.184:6815/1533397276","heartbeat_front_addr":"172.21.15.184:6813/1533397276","state":["exists","up"]},{"osd":6,"uuid":"b3936e21-e197-49fa-8441-d909b258b612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":902466142},{"type":"v1","addr":"172.21.15.16:6819","nonce":902466142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":902466142},{"type":"v1","addr":"172.21.15.16:6821","nonce":902466142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":902466142},{"type":"v1","addr":"172.21.15.16:6825","nonce":902466142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":902466142},{"type":"v1","addr":"172.21.15.16:6823","nonce":902466142}]},"public_addr":"172.21.15.16:6819/902466142","cluster_addr":"172.21.15.16:6821/902466142","heartbeat_back_addr":"172.21.15.16:6825/902466142","heartbeat_front_addr":"172.21.15.16:6823/902466142","state":["exists","up"]},{"osd":7,"uuid":"4ddbe011-34c1-4b3f-a8eb-577422fb5426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":4186032630},{"type":"v1","addr":"172.21.15.33:6817","nonce":4186032630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":4186032630},{"type":"v1","addr":"172.21.15.33:6819","nonce":4186032630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":4186032630},{"type":"v1","addr":"172.21.15.33:6823","nonce":4186032630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":4186032630},{"type":"v1","addr":"172.21.15.33:6821","nonce":4186032630}]},"public_addr":"172.21.15.33:6817/4186032630","cluster_addr":"172.21.15.33:6819/4186032630","heartbeat_back_addr":"172.21.15.33:6823/4186032630","heartbeat_front_addr":"172.21.15.33:6821/4186032630","state":["exists","up"]},{"osd":8,"uuid":"dca5bfb8-ac6a-47ce-905c-be7fcaf0b5ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":781851781},{"type":"v1","addr":"172.21.15.184:6817","nonce":781851781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":781851781},{"type":"v1","addr":"172.21.15.184:6819","nonce":781851781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":781851781},{"type":"v1","addr":"172.21.15.184:6823","nonce":781851781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":781851781},{"type":"v1","addr":"172.21.15.184:6821","nonce":781851781}]},"public_addr":"172.21.15.184:6817/781851781","cluster_addr":"172.21.15.184:6819/781851781","heartbeat_back_addr":"172.21.15.184:6823/781851781","heartbeat_front_addr":"172.21.15.184:6821/781851781","state":["exists","up"]},{"osd":9,"uuid":"23a05cb5-01d5-40f3-b66a-15fcccedbd6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":1569578746},{"type":"v1","addr":"172.21.15.16:6827","nonce":1569578746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":1569578746},{"type":"v1","addr":"172.21.15.16:6829","nonce":1569578746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1569578746},{"type":"v1","addr":"172.21.15.16:6833","nonce":1569578746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":1569578746},{"type":"v1","addr":"172.21.15.16:6831","nonce":1569578746}]},"public_addr":"172.21.15.16:6827/1569578746","cluster_addr":"172.21.15.16:6829/1569578746","heartbeat_back_addr":"172.21.15.16:6833/1569578746","heartbeat_front_addr":"172.21.15.16:6831/1569578746","state":["exists","up"]},{"osd":10,"uuid":"e4bd6bce-3c69-44db-bc09-d264a0ad094a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":469977847},{"type":"v1","addr":"172.21.15.33:6825","nonce":469977847}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":469977847},{"type":"v1","addr":"172.21.15.33:6827","nonce":469977847}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":469977847},{"type":"v1","addr":"172.21.15.33:6831","nonce":469977847}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":469977847},{"type":"v1","addr":"172.21.15.33:6829","nonce":469977847}]},"public_addr":"172.21.15.33:6825/469977847","cluster_addr":"172.21.15.33:6827/469977847","heartbeat_back_addr":"172.21.15.33:6831/469977847","heartbeat_front_addr":"172.21.15.33:6829/469977847","state":["exists","up"]},{"osd":11,"uuid":"d60212f8-5a1d-45f7-9a1b-12294bb4c80d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":1619871346},{"type":"v1","addr":"172.21.15.184:6825","nonce":1619871346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":1619871346},{"type":"v1","addr":"172.21.15.184:6827","nonce":1619871346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":1619871346},{"type":"v1","addr":"172.21.15.184:6831","nonce":1619871346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":1619871346},{"type":"v1","addr":"172.21.15.184:6829","nonce":1619871346}]},"public_addr":"172.21.15.184:6825/1619871346","cluster_addr":"172.21.15.184:6827/1619871346","heartbeat_back_addr":"172.21.15.184:6831/1619871346","heartbeat_front_addr":"172.21.15.184:6829/1619871346","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:54:32.057979+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:54:57.501562+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:55:23.137225+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:55:48.646155+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:56:13.311907+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:56:38.214188+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:57:02.570921+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:57:27.506097+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:57:52.038529+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:58:17.423300+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:58:41.579992+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:59:06.245817+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:0/3532565698":"2024-09-13T01:53:08.218475+0000","172.21.15.16:0/1822157179":"2024-09-13T01:52:44.194152+0000","172.21.15.16:0/3919664061":"2024-09-13T01:52:44.194152+0000","172.21.15.16:0/2269394774":"2024-09-13T01:52:44.194152+0000","172.21.15.16:0/2595308634":"2024-09-13T01:53:08.218475+0000","172.21.15.16:6801/2754331153":"2024-09-13T01:52:44.194152+0000","172.21.15.16:6801/2615861334":"2024-09-13T01:53:08.218475+0000","172.21.15.16:6800/2615861334":"2024-09-13T01:53:08.218475+0000","172.21.15.16:0/4199114771":"2024-09-13T01:53:08.218475+0000","172.21.15.16:6800/2754331153":"2024-09-13T01:52:44.194152+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T02:01:37.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:37.732+0000 7fe88f3b7640 1 -- 172.21.15.16:0/1887785245 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fe8740a2ca0 msgr2=0x7fe8740a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:37.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:37.732+0000 7fe88f3b7640 1 --2- 172.21.15.16:0/1887785245 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fe8740a2ca0 0x7fe8740a5160 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fe878006fd0 tx=0x7fe878008040 comp rx=0 tx=0).stop 2024-09-12T02:01:37.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:37.732+0000 7fe88f3b7640 1 -- 172.21.15.16:0/1887785245 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8880ffdf0 msgr2=0x7fe888197e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:37.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:37.732+0000 7fe88f3b7640 1 --2- 172.21.15.16:0/1887785245 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8880ffdf0 0x7fe888197e40 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fe88400ccb0 tx=0x7fe88400a620 comp rx=0 tx=0).stop 2024-09-12T02:01:37.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:37.732+0000 7fe88f3b7640 1 -- 172.21.15.16:0/1887785245 shutdown_connections 2024-09-12T02:01:37.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:37.732+0000 7fe88f3b7640 1 --2- 172.21.15.16:0/1887785245 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fe8740a2ca0 0x7fe8740a5160 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:37.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:37.732+0000 7fe88f3b7640 1 --2- 172.21.15.16:0/1887785245 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe888103780 0x7fe88819f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:37.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:37.732+0000 7fe88f3b7640 1 --2- 172.21.15.16:0/1887785245 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe888102dc0 0x7fe888198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:37.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:37.732+0000 7fe88f3b7640 1 --2- 172.21.15.16:0/1887785245 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8880ffdf0 0x7fe888197e40 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:37.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:37.732+0000 7fe88f3b7640 1 -- 172.21.15.16:0/1887785245 >> 172.21.15.16:0/1887785245 conn(0x7fe8880fb910 msgr2=0x7fe8881061b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:01:37.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:37.732+0000 7fe88f3b7640 1 -- 172.21.15.16:0/1887785245 shutdown_connections 2024-09-12T02:01:37.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:37.733+0000 7fe88f3b7640 1 -- 172.21.15.16:0/1887785245 wait complete. 2024-09-12T02:01:37.744 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-12T01:55:26.727564+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-12T01:59:52.402097+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-12T01:59:54.463627+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-12T02:01:37.745 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-12T02:01:37.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:37.891+0000 7f4db6996640 1 -- 172.21.15.16:0/2879534278 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4db0104ee0 msgr2=0x7f4db01072d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:37.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:37.891+0000 7f4db6996640 1 --2- 172.21.15.16:0/2879534278 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4db0104ee0 0x7f4db01072d0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f4da4009900 tx=0x7f4da402a410 comp rx=0 tx=0).stop 2024-09-12T02:01:37.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:37.891+0000 7f4db6996640 1 -- 172.21.15.16:0/2879534278 shutdown_connections 2024-09-12T02:01:37.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:37.891+0000 7f4db6996640 1 --2- 172.21.15.16:0/2879534278 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4db0107810 0x7f4db0109c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:37.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:37.891+0000 7f4db6996640 1 --2- 172.21.15.16:0/2879534278 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4db0104ee0 0x7f4db01072d0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:37.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:37.891+0000 7f4db6996640 1 --2- 172.21.15.16:0/2879534278 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4db01025b0 0x7f4db01049a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:37.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:37.891+0000 7f4db6996640 1 -- 172.21.15.16:0/2879534278 >> 172.21.15.16:0/2879534278 conn(0x7f4db00fc1b0 msgr2=0x7f4db00fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:01:37.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:37.891+0000 7f4db6996640 1 -- 172.21.15.16:0/2879534278 shutdown_connections 2024-09-12T02:01:37.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:37.891+0000 7f4db6996640 1 -- 172.21.15.16:0/2879534278 wait complete. 2024-09-12T02:01:37.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:37.892+0000 7f4db6996640 1 Processor -- start 2024-09-12T02:01:37.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:37.892+0000 7f4db6996640 1 -- start start 2024-09-12T02:01:37.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:37.892+0000 7f4db6996640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4db01025b0 0x7f4db0101f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:37.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:37.892+0000 7f4db6996640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4db0104ee0 0x7f4db0100560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:37.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:37.892+0000 7f4db6996640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4db0107810 0x7f4db0100aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:37.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:37.892+0000 7f4db6996640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4db010d7c0 con 0x7f4db0104ee0 2024-09-12T02:01:37.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:37.892+0000 7f4db6996640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4db010d640 con 0x7f4db0107810 2024-09-12T02:01:37.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:37.892+0000 7f4db6996640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f4db010d940 con 0x7f4db01025b0 2024-09-12T02:01:37.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:37.892+0000 7f4daf7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4db0104ee0 0x7f4db0100560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:37.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:37.892+0000 7f4daf7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4db0104ee0 0x7f4db0100560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42686/0 (socket says 172.21.15.16:42686) 2024-09-12T02:01:37.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:37.892+0000 7f4daf7fe640 1 -- 172.21.15.16:0/3458726702 learned_addr learned my addr 172.21.15.16:0/3458726702 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:01:37.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:37.892+0000 7f4daffff640 1 --2- 172.21.15.16:0/3458726702 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4db01025b0 0x7f4db0101f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:37.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:37.892+0000 7f4db4f33640 1 --2- 172.21.15.16:0/3458726702 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4db0107810 0x7f4db0100aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:37.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:37.893+0000 7f4daf7fe640 1 -- 172.21.15.16:0/3458726702 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4db01025b0 msgr2=0x7f4db0101f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:37.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:37.893+0000 7f4daf7fe640 1 --2- 172.21.15.16:0/3458726702 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4db01025b0 0x7f4db0101f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:37.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:37.893+0000 7f4daf7fe640 1 -- 172.21.15.16:0/3458726702 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4db0107810 msgr2=0x7f4db0100aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:37.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:37.893+0000 7f4daf7fe640 1 --2- 172.21.15.16:0/3458726702 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4db0107810 0x7f4db0100aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:37.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:37.893+0000 7f4daf7fe640 1 -- 172.21.15.16:0/3458726702 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4db0101330 con 0x7f4db0104ee0 2024-09-12T02:01:37.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:37.893+0000 7f4daf7fe640 1 --2- 172.21.15.16:0/3458726702 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4db0104ee0 0x7f4db0100560 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f4da4006010 tx=0x7f4da4031b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:01:37.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:37.893+0000 7f4dad7fa640 1 -- 172.21.15.16:0/3458726702 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4da403f030 con 0x7f4db0104ee0 2024-09-12T02:01:37.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:37.893+0000 7f4dad7fa640 1 -- 172.21.15.16:0/3458726702 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4da40044a0 con 0x7f4db0104ee0 2024-09-12T02:01:37.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:37.893+0000 7f4db6996640 1 -- 172.21.15.16:0/3458726702 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4db01a1f90 con 0x7f4db0104ee0 2024-09-12T02:01:37.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:37.893+0000 7f4dad7fa640 1 -- 172.21.15.16:0/3458726702 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4da4035070 con 0x7f4db0104ee0 2024-09-12T02:01:37.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:37.893+0000 7f4db6996640 1 -- 172.21.15.16:0/3458726702 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4db01a23f0 con 0x7f4db0104ee0 2024-09-12T02:01:37.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:37.894+0000 7f4db6996640 1 -- 172.21.15.16:0/3458726702 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4db00fdca0 con 0x7f4db0104ee0 2024-09-12T02:01:37.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:37.895+0000 7f4dad7fa640 1 -- 172.21.15.16:0/3458726702 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f4da4042030 con 0x7f4db0104ee0 2024-09-12T02:01:37.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:37.896+0000 7f4dad7fa640 1 --2- 172.21.15.16:0/3458726702 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f4d980a2c20 0x7f4d980a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:37.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:37.896+0000 7f4daffff640 1 --2- 172.21.15.16:0/3458726702 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f4d980a2c20 0x7f4d980a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:37.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:37.897+0000 7f4dad7fa640 1 -- 172.21.15.16:0/3458726702 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f4da40ef7d0 con 0x7f4db0104ee0 2024-09-12T02:01:37.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:37.897+0000 7f4daffff640 1 --2- 172.21.15.16:0/3458726702 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f4d980a2c20 0x7f4d980a50e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f4da000abe0 tx=0x7f4da000a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:01:37.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:37.900+0000 7f4dad7fa640 1 -- 172.21.15.16:0/3458726702 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4da40bc650 con 0x7f4db0104ee0 2024-09-12T02:01:38.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:38.077+0000 7f4db6996640 1 -- 172.21.15.16:0/3458726702 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f4db002da10 con 0x7f4db0104ee0 2024-09-12T02:01:38.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:38.077+0000 7f4dad7fa640 1 -- 172.21.15.16:0/3458726702 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7f4da40bbf50 con 0x7f4db0104ee0 2024-09-12T02:01:38.078 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-09-12T02:01:38.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:38.079+0000 7f4db6996640 1 -- 172.21.15.16:0/3458726702 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f4d980a2c20 msgr2=0x7f4d980a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:38.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:38.079+0000 7f4db6996640 1 --2- 172.21.15.16:0/3458726702 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f4d980a2c20 0x7f4d980a50e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f4da000abe0 tx=0x7f4da000a2d0 comp rx=0 tx=0).stop 2024-09-12T02:01:38.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:38.079+0000 7f4db6996640 1 -- 172.21.15.16:0/3458726702 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4db0104ee0 msgr2=0x7f4db0100560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:38.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:38.079+0000 7f4db6996640 1 --2- 172.21.15.16:0/3458726702 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4db0104ee0 0x7f4db0100560 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f4da4006010 tx=0x7f4da4031b60 comp rx=0 tx=0).stop 2024-09-12T02:01:38.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:38.079+0000 7f4db6996640 1 -- 172.21.15.16:0/3458726702 shutdown_connections 2024-09-12T02:01:38.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:38.079+0000 7f4db6996640 1 --2- 172.21.15.16:0/3458726702 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f4d980a2c20 0x7f4d980a50e0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:38.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:38.079+0000 7f4db6996640 1 --2- 172.21.15.16:0/3458726702 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4db0107810 0x7f4db0100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:38.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:38.079+0000 7f4db6996640 1 --2- 172.21.15.16:0/3458726702 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4db0104ee0 0x7f4db0100560 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:38.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:38.079+0000 7f4db6996640 1 --2- 172.21.15.16:0/3458726702 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4db01025b0 0x7f4db0101f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:38.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:38.079+0000 7f4db6996640 1 -- 172.21.15.16:0/3458726702 >> 172.21.15.16:0/3458726702 conn(0x7f4db00fc1b0 msgr2=0x7f4db0102f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:01:38.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:38.079+0000 7f4db6996640 1 -- 172.21.15.16:0/3458726702 shutdown_connections 2024-09-12T02:01:38.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:38.079+0000 7f4db6996640 1 -- 172.21.15.16:0/3458726702 wait complete. 2024-09-12T02:01:38.090 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-12T02:01:38.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:38.196+0000 7f89bec8d640 1 -- 172.21.15.16:0/1938368362 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89b80ffbc0 msgr2=0x7f89b8100040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:38.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:38.196+0000 7f89bec8d640 1 --2- 172.21.15.16:0/1938368362 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89b80ffbc0 0x7f89b8100040 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f89ac009900 tx=0x7f89ac02a410 comp rx=0 tx=0).stop 2024-09-12T02:01:38.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:38.196+0000 7f89bec8d640 1 -- 172.21.15.16:0/1938368362 shutdown_connections 2024-09-12T02:01:38.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:38.196+0000 7f89bec8d640 1 --2- 172.21.15.16:0/1938368362 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f89b8100580 0x7f89b810a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:38.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:38.196+0000 7f89bec8d640 1 --2- 172.21.15.16:0/1938368362 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89b80ffbc0 0x7f89b8100040 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:38.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:38.196+0000 7f89bec8d640 1 --2- 172.21.15.16:0/1938368362 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f89b8103150 0x7f89b8103550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:38.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:38.196+0000 7f89bec8d640 1 -- 172.21.15.16:0/1938368362 >> 172.21.15.16:0/1938368362 conn(0x7f89b80fb950 msgr2=0x7f89b80fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:01:38.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:38.197+0000 7f89bec8d640 1 -- 172.21.15.16:0/1938368362 shutdown_connections 2024-09-12T02:01:38.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:38.197+0000 7f89bec8d640 1 -- 172.21.15.16:0/1938368362 wait complete. 2024-09-12T02:01:38.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:38.197+0000 7f89bec8d640 1 Processor -- start 2024-09-12T02:01:38.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:38.197+0000 7f89bec8d640 1 -- start start 2024-09-12T02:01:38.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:38.197+0000 7f89bec8d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f89b80ffbc0 0x7f89b8197e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:38.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:38.197+0000 7f89bec8d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f89b8100580 0x7f89b8198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:38.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:38.197+0000 7f89bec8d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89b8103150 0x7f89b819f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:38.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:38.197+0000 7f89bec8d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f89b810d7f0 con 0x7f89b8103150 2024-09-12T02:01:38.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:38.197+0000 7f89bec8d640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f89b810d670 con 0x7f89b8100580 2024-09-12T02:01:38.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:38.197+0000 7f89bec8d640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f89b810d970 con 0x7f89b80ffbc0 2024-09-12T02:01:38.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:38.197+0000 7f89bd22a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89b8103150 0x7f89b819f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:38.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:38.197+0000 7f89bd22a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89b8103150 0x7f89b819f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42714/0 (socket says 172.21.15.16:42714) 2024-09-12T02:01:38.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:38.197+0000 7f89bd22a640 1 -- 172.21.15.16:0/2556489438 learned_addr learned my addr 172.21.15.16:0/2556489438 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:01:38.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:38.198+0000 7f89bca29640 1 --2- 172.21.15.16:0/2556489438 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f89b80ffbc0 0x7f89b8197e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:38.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:38.198+0000 7f89b7fff640 1 --2- 172.21.15.16:0/2556489438 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f89b8100580 0x7f89b8198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:38.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:38.198+0000 7f89bd22a640 1 -- 172.21.15.16:0/2556489438 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f89b80ffbc0 msgr2=0x7f89b8197e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:38.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:38.198+0000 7f89bd22a640 1 --2- 172.21.15.16:0/2556489438 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f89b80ffbc0 0x7f89b8197e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:38.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:38.198+0000 7f89bd22a640 1 -- 172.21.15.16:0/2556489438 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f89b8100580 msgr2=0x7f89b8198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:38.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:38.198+0000 7f89bd22a640 1 --2- 172.21.15.16:0/2556489438 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f89b8100580 0x7f89b8198380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:38.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:38.198+0000 7f89bd22a640 1 -- 172.21.15.16:0/2556489438 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89b819fa60 con 0x7f89b8103150 2024-09-12T02:01:38.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:38.198+0000 7f89b7fff640 1 --2- 172.21.15.16:0/2556489438 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f89b8100580 0x7f89b8198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:01:38.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:38.198+0000 7f89bd22a640 1 --2- 172.21.15.16:0/2556489438 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89b8103150 0x7f89b819f3f0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f89b000cad0 tx=0x7f89b000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:01:38.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:38.198+0000 7f89b5ffb640 1 -- 172.21.15.16:0/2556489438 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b000ae20 con 0x7f89b8103150 2024-09-12T02:01:38.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:38.198+0000 7f89b5ffb640 1 -- 172.21.15.16:0/2556489438 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89b0005020 con 0x7f89b8103150 2024-09-12T02:01:38.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:38.198+0000 7f89bec8d640 1 -- 172.21.15.16:0/2556489438 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89b819fd50 con 0x7f89b8103150 2024-09-12T02:01:38.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:38.198+0000 7f89b5ffb640 1 -- 172.21.15.16:0/2556489438 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b000f6f0 con 0x7f89b8103150 2024-09-12T02:01:38.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:38.199+0000 7f89bec8d640 1 -- 172.21.15.16:0/2556489438 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89b81a0240 con 0x7f89b8103150 2024-09-12T02:01:38.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:38.200+0000 7f89bec8d640 1 -- 172.21.15.16:0/2556489438 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89b8109130 con 0x7f89b8103150 2024-09-12T02:01:38.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:38.201+0000 7f89b5ffb640 1 -- 172.21.15.16:0/2556489438 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f89b00051d0 con 0x7f89b8103150 2024-09-12T02:01:38.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:38.202+0000 7f89b5ffb640 1 --2- 172.21.15.16:0/2556489438 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f89a00a2ca0 0x7f89a00a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:38.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:38.202+0000 7f89bca29640 1 --2- 172.21.15.16:0/2556489438 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f89a00a2ca0 0x7f89a00a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:38.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:38.202+0000 7f89b5ffb640 1 -- 172.21.15.16:0/2556489438 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f89b00cf1f0 con 0x7f89b8103150 2024-09-12T02:01:38.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:38.202+0000 7f89bca29640 1 --2- 172.21.15.16:0/2556489438 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f89a00a2ca0 0x7f89a00a5160 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f89b806a040 tx=0x7f89a8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:01:38.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:38.207+0000 7f89b5ffb640 1 -- 172.21.15.16:0/2556489438 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f89b0014030 con 0x7f89b8103150 2024-09-12T02:01:38.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:38.410+0000 7f89bec8d640 1 -- 172.21.15.16:0/2556489438 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f89b8062e90 con 0x7f89b8103150 2024-09-12T02:01:38.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:38.410+0000 7f89b5ffb640 1 -- 172.21.15.16:0/2556489438 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v82) ==== 104+0+11 (secure 0 0 0) 0x7f89b009bff0 con 0x7f89b8103150 2024-09-12T02:01:38.410 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2024-09-12T02:01:38.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:38.412+0000 7f89bec8d640 1 -- 172.21.15.16:0/2556489438 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f89a00a2ca0 msgr2=0x7f89a00a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:38.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:38.412+0000 7f89bec8d640 1 --2- 172.21.15.16:0/2556489438 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f89a00a2ca0 0x7f89a00a5160 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f89b806a040 tx=0x7f89a8005f90 comp rx=0 tx=0).stop 2024-09-12T02:01:38.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:38.413+0000 7f89bec8d640 1 -- 172.21.15.16:0/2556489438 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89b8103150 msgr2=0x7f89b819f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:38.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:38.413+0000 7f89bec8d640 1 --2- 172.21.15.16:0/2556489438 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89b8103150 0x7f89b819f3f0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f89b000cad0 tx=0x7f89b000cf90 comp rx=0 tx=0).stop 2024-09-12T02:01:38.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:38.413+0000 7f89bec8d640 1 -- 172.21.15.16:0/2556489438 shutdown_connections 2024-09-12T02:01:38.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:38.413+0000 7f89bec8d640 1 --2- 172.21.15.16:0/2556489438 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f89a00a2ca0 0x7f89a00a5160 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:38.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:38.413+0000 7f89bec8d640 1 --2- 172.21.15.16:0/2556489438 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89b8103150 0x7f89b819f3f0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:38.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:38.413+0000 7f89bec8d640 1 --2- 172.21.15.16:0/2556489438 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f89b8100580 0x7f89b8198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:38.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:38.413+0000 7f89bec8d640 1 --2- 172.21.15.16:0/2556489438 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f89b80ffbc0 0x7f89b8197e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:38.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:38.413+0000 7f89bec8d640 1 -- 172.21.15.16:0/2556489438 >> 172.21.15.16:0/2556489438 conn(0x7f89b80fb950 msgr2=0x7f89b8106300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:01:38.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:38.413+0000 7f89bec8d640 1 -- 172.21.15.16:0/2556489438 shutdown_connections 2024-09-12T02:01:38.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:38.413+0000 7f89bec8d640 1 -- 172.21.15.16:0/2556489438 wait complete. 2024-09-12T02:01:38.423 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:01:38 smithi016 ceph-mon[30947]: pgmap v326: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-12T02:01:38.423 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:01:38 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/800813287' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-09-12T02:01:38.423 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:01:38 smithi016 ceph-mon[30947]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T02:01:38.423 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:01:38 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/1887785245' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T02:01:38.423 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:01:38 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/3458726702' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-12T02:01:38.424 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-12T02:01:38.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:38.531+0000 7fca9cc3c640 1 -- 172.21.15.16:0/3738261082 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca98104ee0 msgr2=0x7fca981072d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:38.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:38.531+0000 7fca9cc3c640 1 --2- 172.21.15.16:0/3738261082 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca98104ee0 0x7fca981072d0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fca80009900 tx=0x7fca8002a410 comp rx=0 tx=0).stop 2024-09-12T02:01:38.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:38.531+0000 7fca9cc3c640 1 -- 172.21.15.16:0/3738261082 shutdown_connections 2024-09-12T02:01:38.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:38.531+0000 7fca9cc3c640 1 --2- 172.21.15.16:0/3738261082 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fca98107810 0x7fca98109c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:38.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:38.531+0000 7fca9cc3c640 1 --2- 172.21.15.16:0/3738261082 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca98104ee0 0x7fca981072d0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:38.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:38.531+0000 7fca9cc3c640 1 --2- 172.21.15.16:0/3738261082 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca981025b0 0x7fca981049a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:38.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:38.531+0000 7fca9cc3c640 1 -- 172.21.15.16:0/3738261082 >> 172.21.15.16:0/3738261082 conn(0x7fca980fc1b0 msgr2=0x7fca980fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:01:38.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:38.531+0000 7fca9cc3c640 1 -- 172.21.15.16:0/3738261082 shutdown_connections 2024-09-12T02:01:38.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:38.531+0000 7fca9cc3c640 1 -- 172.21.15.16:0/3738261082 wait complete. 2024-09-12T02:01:38.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:38.532+0000 7fca9cc3c640 1 Processor -- start 2024-09-12T02:01:38.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:38.532+0000 7fca9cc3c640 1 -- start start 2024-09-12T02:01:38.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:38.532+0000 7fca9cc3c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca981025b0 0x7fca98197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:38.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:38.532+0000 7fca9cc3c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fca98104ee0 0x7fca981984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:38.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:38.532+0000 7fca9cc3c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca98107810 0x7fca9819f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:38.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:38.532+0000 7fca9cc3c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fca9810d890 con 0x7fca98107810 2024-09-12T02:01:38.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:38.532+0000 7fca9cc3c640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fca9810d710 con 0x7fca981025b0 2024-09-12T02:01:38.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:38.532+0000 7fca9cc3c640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fca9810da10 con 0x7fca98104ee0 2024-09-12T02:01:38.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:38.532+0000 7fca96d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca98107810 0x7fca9819f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:38.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:38.532+0000 7fca96d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca98107810 0x7fca9819f550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42726/0 (socket says 172.21.15.16:42726) 2024-09-12T02:01:38.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:38.532+0000 7fca96d9d640 1 -- 172.21.15.16:0/52669776 learned_addr learned my addr 172.21.15.16:0/52669776 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:01:38.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:38.532+0000 7fca9659c640 1 --2- 172.21.15.16:0/52669776 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca981025b0 0x7fca98197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:38.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:38.532+0000 7fca8ffff640 1 --2- 172.21.15.16:0/52669776 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fca98104ee0 0x7fca981984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:38.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:38.532+0000 7fca96d9d640 1 -- 172.21.15.16:0/52669776 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fca98104ee0 msgr2=0x7fca981984e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:38.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:38.532+0000 7fca96d9d640 1 --2- 172.21.15.16:0/52669776 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fca98104ee0 0x7fca981984e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:38.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:38.532+0000 7fca96d9d640 1 -- 172.21.15.16:0/52669776 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca981025b0 msgr2=0x7fca98197fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:38.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:38.532+0000 7fca96d9d640 1 --2- 172.21.15.16:0/52669776 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca981025b0 0x7fca98197fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:38.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:38.533+0000 7fca96d9d640 1 -- 172.21.15.16:0/52669776 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca9819fbc0 con 0x7fca98107810 2024-09-12T02:01:38.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:38.533+0000 7fca96d9d640 1 --2- 172.21.15.16:0/52669776 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca98107810 0x7fca9819f550 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fca9000b9e0 tx=0x7fca9000bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:01:38.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:38.533+0000 7fca8f7fe640 1 -- 172.21.15.16:0/52669776 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca90019070 con 0x7fca98107810 2024-09-12T02:01:38.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:38.533+0000 7fca8f7fe640 1 -- 172.21.15.16:0/52669776 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca90014440 con 0x7fca98107810 2024-09-12T02:01:38.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:38.533+0000 7fca9cc3c640 1 -- 172.21.15.16:0/52669776 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca9819feb0 con 0x7fca98107810 2024-09-12T02:01:38.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:38.533+0000 7fca8f7fe640 1 -- 172.21.15.16:0/52669776 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca900125e0 con 0x7fca98107810 2024-09-12T02:01:38.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:38.533+0000 7fca9cc3c640 1 -- 172.21.15.16:0/52669776 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca981a03f0 con 0x7fca98107810 2024-09-12T02:01:38.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:38.534+0000 7fca9cc3c640 1 -- 172.21.15.16:0/52669776 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca980fdca0 con 0x7fca98107810 2024-09-12T02:01:38.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:38.535+0000 7fca8f7fe640 1 -- 172.21.15.16:0/52669776 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fca9001c070 con 0x7fca98107810 2024-09-12T02:01:38.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:38.536+0000 7fca8f7fe640 1 --2- 172.21.15.16:0/52669776 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fca840a2ca0 0x7fca840a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:38.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:38.536+0000 7fca8f7fe640 1 -- 172.21.15.16:0/52669776 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fca900ceb40 con 0x7fca98107810 2024-09-12T02:01:38.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:38.536+0000 7fca9659c640 1 --2- 172.21.15.16:0/52669776 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fca840a2ca0 0x7fca840a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:38.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:38.536+0000 7fca9659c640 1 --2- 172.21.15.16:0/52669776 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fca840a2ca0 0x7fca840a5160 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fca8800bbe0 tx=0x7fca8800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:01:38.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:38.539+0000 7fca8f7fe640 1 -- 172.21.15.16:0/52669776 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fca9009b500 con 0x7fca98107810 2024-09-12T02:01:38.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:01:38 smithi184 ceph-mon[35593]: pgmap v326: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-12T02:01:38.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:01:38 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/800813287' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-09-12T02:01:38.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:01:38 smithi184 ceph-mon[35593]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T02:01:38.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:01:38 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/1887785245' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T02:01:38.628 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:01:38 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/3458726702' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-12T02:01:38.668 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:01:38 smithi033 ceph-mon[35332]: pgmap v326: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-12T02:01:38.668 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:01:38 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/800813287' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-09-12T02:01:38.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:01:38 smithi033 ceph-mon[35332]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T02:01:38.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:01:38 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/1887785245' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T02:01:38.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:01:38 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/3458726702' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-12T02:01:38.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:38.713+0000 7fca9cc3c640 1 -- 172.21.15.16:0/52669776 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fca9802d9c0 con 0x7fca98107810 2024-09-12T02:01:38.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:38.713+0000 7fca8f7fe640 1 -- 172.21.15.16:0/52669776 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v82) ==== 100+0+11 (secure 0 0 0) 0x7fca9009f800 con 0x7fca98107810 2024-09-12T02:01:38.714 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2024-09-12T02:01:38.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:38.716+0000 7fca9cc3c640 1 -- 172.21.15.16:0/52669776 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fca840a2ca0 msgr2=0x7fca840a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:38.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:38.716+0000 7fca9cc3c640 1 --2- 172.21.15.16:0/52669776 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fca840a2ca0 0x7fca840a5160 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fca8800bbe0 tx=0x7fca8800b390 comp rx=0 tx=0).stop 2024-09-12T02:01:38.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:38.716+0000 7fca9cc3c640 1 -- 172.21.15.16:0/52669776 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca98107810 msgr2=0x7fca9819f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:38.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:38.716+0000 7fca9cc3c640 1 --2- 172.21.15.16:0/52669776 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca98107810 0x7fca9819f550 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fca9000b9e0 tx=0x7fca9000bea0 comp rx=0 tx=0).stop 2024-09-12T02:01:38.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:38.716+0000 7fca9cc3c640 1 -- 172.21.15.16:0/52669776 shutdown_connections 2024-09-12T02:01:38.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:38.716+0000 7fca9cc3c640 1 --2- 172.21.15.16:0/52669776 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fca840a2ca0 0x7fca840a5160 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:38.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:38.716+0000 7fca9cc3c640 1 --2- 172.21.15.16:0/52669776 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca98107810 0x7fca9819f550 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:38.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:38.716+0000 7fca9cc3c640 1 --2- 172.21.15.16:0/52669776 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fca98104ee0 0x7fca981984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:38.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:38.716+0000 7fca9cc3c640 1 --2- 172.21.15.16:0/52669776 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca981025b0 0x7fca98197fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:38.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:38.716+0000 7fca9cc3c640 1 -- 172.21.15.16:0/52669776 >> 172.21.15.16:0/52669776 conn(0x7fca980fc1b0 msgr2=0x7fca98102ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:01:38.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:38.716+0000 7fca9cc3c640 1 -- 172.21.15.16:0/52669776 shutdown_connections 2024-09-12T02:01:38.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:38.716+0000 7fca9cc3c640 1 -- 172.21.15.16:0/52669776 wait complete. 2024-09-12T02:01:38.726 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-09-12T02:01:38.727 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:01:38.984 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:01:39.369 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:01:39 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/2556489438' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-12T02:01:39.369 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:01:39 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/52669776' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-12T02:01:39.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:39.452+0000 7f69bb18b640 1 -- 172.21.15.16:0/293260820 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69b4102d40 msgr2=0x7f69b4103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:39.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:39.452+0000 7f69bb18b640 1 --2- 172.21.15.16:0/293260820 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69b4102d40 0x7f69b4103120 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f69a4009900 tx=0x7f69a402a480 comp rx=0 tx=0).stop 2024-09-12T02:01:39.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:39.452+0000 7f69bb18b640 1 -- 172.21.15.16:0/293260820 shutdown_connections 2024-09-12T02:01:39.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:39.452+0000 7f69bb18b640 1 --2- 172.21.15.16:0/293260820 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f69b410c170 0x7f69b410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:39.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:39.452+0000 7f69bb18b640 1 --2- 172.21.15.16:0/293260820 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f69b4103660 0x7f69b410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:39.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:39.452+0000 7f69bb18b640 1 --2- 172.21.15.16:0/293260820 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69b4102d40 0x7f69b4103120 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:39.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:39.452+0000 7f69bb18b640 1 -- 172.21.15.16:0/293260820 >> 172.21.15.16:0/293260820 conn(0x7f69b40fc7a0 msgr2=0x7f69b40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:01:39.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:39.452+0000 7f69bb18b640 1 -- 172.21.15.16:0/293260820 shutdown_connections 2024-09-12T02:01:39.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:39.452+0000 7f69bb18b640 1 -- 172.21.15.16:0/293260820 wait complete. 2024-09-12T02:01:39.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:39.453+0000 7f69bb18b640 1 Processor -- start 2024-09-12T02:01:39.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:39.453+0000 7f69bb18b640 1 -- start start 2024-09-12T02:01:39.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:39.453+0000 7f69bb18b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f69b4102d40 0x7f69b4198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:39.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:39.453+0000 7f69bb18b640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f69b4103660 0x7f69b41989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:39.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:39.453+0000 7f69bb18b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69b410c170 0x7f69b419cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:39.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:39.453+0000 7f69bb18b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f69b4112000 con 0x7f69b410c170 2024-09-12T02:01:39.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:39.453+0000 7f69bb18b640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f69b4111e80 con 0x7f69b4102d40 2024-09-12T02:01:39.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:39.453+0000 7f69bb18b640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f69b4112180 con 0x7f69b4103660 2024-09-12T02:01:39.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:39.453+0000 7f69b9728640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69b410c170 0x7f69b419cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:39.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:39.453+0000 7f69b9728640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69b410c170 0x7f69b419cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42742/0 (socket says 172.21.15.16:42742) 2024-09-12T02:01:39.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:39.453+0000 7f69b9728640 1 -- 172.21.15.16:0/1824604566 learned_addr learned my addr 172.21.15.16:0/1824604566 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:01:39.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:39.453+0000 7f69b8f27640 1 --2- 172.21.15.16:0/1824604566 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f69b4102d40 0x7f69b4198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:39.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:39.453+0000 7f69b3fff640 1 --2- 172.21.15.16:0/1824604566 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f69b4103660 0x7f69b41989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:39.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:39.453+0000 7f69b9728640 1 -- 172.21.15.16:0/1824604566 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f69b4103660 msgr2=0x7f69b41989c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:39.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:39.453+0000 7f69b9728640 1 --2- 172.21.15.16:0/1824604566 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f69b4103660 0x7f69b41989c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:39.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:39.453+0000 7f69b9728640 1 -- 172.21.15.16:0/1824604566 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f69b4102d40 msgr2=0x7f69b4198480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:39.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:39.453+0000 7f69b9728640 1 --2- 172.21.15.16:0/1824604566 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f69b4102d40 0x7f69b4198480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:39.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:39.454+0000 7f69b9728640 1 -- 172.21.15.16:0/1824604566 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69b419d430 con 0x7f69b410c170 2024-09-12T02:01:39.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:39.454+0000 7f69b9728640 1 --2- 172.21.15.16:0/1824604566 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69b410c170 0x7f69b419cd50 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f69ac00b970 tx=0x7f69ac00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:01:39.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:39.454+0000 7f69b1ffb640 1 -- 172.21.15.16:0/1824604566 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69ac019070 con 0x7f69b410c170 2024-09-12T02:01:39.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:39.454+0000 7f69b1ffb640 1 -- 172.21.15.16:0/1824604566 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69ac00cb40 con 0x7f69b410c170 2024-09-12T02:01:39.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:39.454+0000 7f69bb18b640 1 -- 172.21.15.16:0/1824604566 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69b419d720 con 0x7f69b410c170 2024-09-12T02:01:39.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:39.454+0000 7f69b1ffb640 1 -- 172.21.15.16:0/1824604566 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69ac007650 con 0x7f69b410c170 2024-09-12T02:01:39.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:39.454+0000 7f69bb18b640 1 -- 172.21.15.16:0/1824604566 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69b41a5000 con 0x7f69b410c170 2024-09-12T02:01:39.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:39.455+0000 7f69bb18b640 1 -- 172.21.15.16:0/1824604566 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69b4103120 con 0x7f69b410c170 2024-09-12T02:01:39.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:39.456+0000 7f69b1ffb640 1 -- 172.21.15.16:0/1824604566 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f69ac01c070 con 0x7f69b410c170 2024-09-12T02:01:39.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:39.457+0000 7f69b1ffb640 1 --2- 172.21.15.16:0/1824604566 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f69a00a2b80 0x7f69a00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:39.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:39.457+0000 7f69b1ffb640 1 -- 172.21.15.16:0/1824604566 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f69ac0ce5b0 con 0x7f69b410c170 2024-09-12T02:01:39.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:39.457+0000 7f69b8f27640 1 --2- 172.21.15.16:0/1824604566 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f69a00a2b80 0x7f69a00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:39.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:39.458+0000 7f69b8f27640 1 --2- 172.21.15.16:0/1824604566 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f69a00a2b80 0x7f69a00a5040 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f69a4000c00 tx=0x7f69a4005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:01:39.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:39.461+0000 7f69b1ffb640 1 -- 172.21.15.16:0/1824604566 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f69ac09b5b0 con 0x7f69b410c170 2024-09-12T02:01:39.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:01:39 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/2556489438' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-12T02:01:39.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:01:39 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/52669776' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-12T02:01:39.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:01:39 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/2556489438' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-12T02:01:39.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:01:39 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/52669776' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-12T02:01:39.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:39.677+0000 7f69bb18b640 1 -- 172.21.15.16:0/1824604566 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f69b402d7c0 con 0x7f69b410c170 2024-09-12T02:01:39.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:39.678+0000 7f69b1ffb640 1 -- 172.21.15.16:0/1824604566 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11911 (secure 0 0 0) 0x7f69ac09aeb0 con 0x7f69b410c170 2024-09-12T02:01:39.679 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T02:01:39.679 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":25,"btime":"2024-09-12T02:01:37:192021+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3279123279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3279123279},{"type":"v1","addr":"172.21.15.16:6835","nonce":3279123279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4119452382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4119452382},{"type":"v1","addr":"172.21.15.16:6837","nonce":4119452382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/3263823165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3263823165},{"type":"v1","addr":"172.21.15.16:6839","nonce":3263823165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/1510290732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":1510290732},{"type":"v1","addr":"172.21.15.16:6841","nonce":1510290732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/279674603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":279674603},{"type":"v1","addr":"172.21.15.33:6833","nonce":279674603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3555728028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3555728028},{"type":"v1","addr":"172.21.15.33:6835","nonce":3555728028}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T01:59:56.117608+0000","modified":"2024-09-12T02:01:36.294760+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24482,"mds_1":24481,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6833/1563374106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1563374106},{"type":"v1","addr":"172.21.15.184:6833","nonce":1563374106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6837/1892833868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1892833868},{"type":"v1","addr":"172.21.15.33:6837","nonce":1892833868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/1457679686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1457679686},{"type":"v1","addr":"172.21.15.184:6835","nonce":1457679686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.184:6837/1793393087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":1793393087},{"type":"v1","addr":"172.21.15.184:6837","nonce":1793393087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.184:6839/4203347941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":4203347941},{"type":"v1","addr":"172.21.15.184:6839","nonce":4203347941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.33:6839/4136670204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":4136670204},{"type":"v1","addr":"172.21.15.33:6839","nonce":4136670204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24481,24482,24469]},"id":1}]} 2024-09-12T02:01:39.679 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 25 2024-09-12T02:01:39.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:39.680+0000 7f69bb18b640 1 -- 172.21.15.16:0/1824604566 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f69a00a2b80 msgr2=0x7f69a00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:39.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:39.680+0000 7f69bb18b640 1 --2- 172.21.15.16:0/1824604566 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f69a00a2b80 0x7f69a00a5040 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f69a4000c00 tx=0x7f69a4005a20 comp rx=0 tx=0).stop 2024-09-12T02:01:39.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:39.680+0000 7f69bb18b640 1 -- 172.21.15.16:0/1824604566 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69b410c170 msgr2=0x7f69b419cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:39.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:39.680+0000 7f69bb18b640 1 --2- 172.21.15.16:0/1824604566 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69b410c170 0x7f69b419cd50 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f69ac00b970 tx=0x7f69ac00be30 comp rx=0 tx=0).stop 2024-09-12T02:01:39.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:39.680+0000 7f69bb18b640 1 -- 172.21.15.16:0/1824604566 shutdown_connections 2024-09-12T02:01:39.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:39.680+0000 7f69bb18b640 1 --2- 172.21.15.16:0/1824604566 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f69a00a2b80 0x7f69a00a5040 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:39.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:39.680+0000 7f69bb18b640 1 --2- 172.21.15.16:0/1824604566 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69b410c170 0x7f69b419cd50 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:39.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:39.680+0000 7f69bb18b640 1 --2- 172.21.15.16:0/1824604566 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f69b4103660 0x7f69b41989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:39.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:39.680+0000 7f69bb18b640 1 --2- 172.21.15.16:0/1824604566 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f69b4102d40 0x7f69b4198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:39.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:39.680+0000 7f69bb18b640 1 -- 172.21.15.16:0/1824604566 >> 172.21.15.16:0/1824604566 conn(0x7f69b40fc7a0 msgr2=0x7f69b40fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:01:39.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:39.680+0000 7f69bb18b640 1 -- 172.21.15.16:0/1824604566 shutdown_connections 2024-09-12T02:01:39.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:39.680+0000 7f69bb18b640 1 -- 172.21.15.16:0/1824604566 wait complete. 2024-09-12T02:01:40.340 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:01:40 smithi016 ceph-mon[30947]: pgmap v327: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T02:01:40.340 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:01:40 smithi016 ceph-mon[30947]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-09-12T02:01:40.340 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:01:40 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/1824604566' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:01:40.368 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-09-12T02:01:40.368 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-09-12T02:01:40.368 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T02:01:40.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:40.477+0000 7f4a67f18640 1 -- 172.21.15.16:0/2699414634 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a60107810 msgr2=0x7f4a60109c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:40.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:40.477+0000 7f4a67f18640 1 --2- 172.21.15.16:0/2699414634 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a60107810 0x7f4a60109c80 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f4a50009ca0 tx=0x7f4a5002a8c0 comp rx=0 tx=0).stop 2024-09-12T02:01:40.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:40.477+0000 7f4a67f18640 1 -- 172.21.15.16:0/2699414634 shutdown_connections 2024-09-12T02:01:40.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:40.477+0000 7f4a67f18640 1 --2- 172.21.15.16:0/2699414634 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a60107810 0x7f4a60109c80 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:40.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:40.477+0000 7f4a67f18640 1 --2- 172.21.15.16:0/2699414634 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4a60104ee0 0x7f4a601072d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:40.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:40.477+0000 7f4a67f18640 1 --2- 172.21.15.16:0/2699414634 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4a601025b0 0x7f4a601049a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:40.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:40.477+0000 7f4a67f18640 1 -- 172.21.15.16:0/2699414634 >> 172.21.15.16:0/2699414634 conn(0x7f4a600fc190 msgr2=0x7f4a600fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:01:40.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:40.477+0000 7f4a67f18640 1 -- 172.21.15.16:0/2699414634 shutdown_connections 2024-09-12T02:01:40.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:40.477+0000 7f4a67f18640 1 -- 172.21.15.16:0/2699414634 wait complete. 2024-09-12T02:01:40.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:40.478+0000 7f4a67f18640 1 Processor -- start 2024-09-12T02:01:40.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:40.478+0000 7f4a67f18640 1 -- start start 2024-09-12T02:01:40.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:40.478+0000 7f4a67f18640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4a601025b0 0x7f4a60101f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:40.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:40.478+0000 7f4a67f18640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a60104ee0 0x7f4a60100560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:40.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:40.478+0000 7f4a67f18640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4a60107810 0x7f4a60100aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:40.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:40.478+0000 7f4a67f18640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4a6010d840 con 0x7f4a60104ee0 2024-09-12T02:01:40.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:40.478+0000 7f4a67f18640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4a6010d6c0 con 0x7f4a60107810 2024-09-12T02:01:40.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:40.478+0000 7f4a67f18640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f4a6010d9c0 con 0x7f4a601025b0 2024-09-12T02:01:40.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:40.478+0000 7f4a654b3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a60104ee0 0x7f4a60100560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:40.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:40.478+0000 7f4a654b3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a60104ee0 0x7f4a60100560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42758/0 (socket says 172.21.15.16:42758) 2024-09-12T02:01:40.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:40.478+0000 7f4a654b3640 1 -- 172.21.15.16:0/4235525903 learned_addr learned my addr 172.21.15.16:0/4235525903 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:01:40.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:40.478+0000 7f4a65cb4640 1 --2- 172.21.15.16:0/4235525903 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4a601025b0 0x7f4a60101f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:40.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:40.478+0000 7f4a664b5640 1 --2- 172.21.15.16:0/4235525903 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4a60107810 0x7f4a60100aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:40.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:40.479+0000 7f4a654b3640 1 -- 172.21.15.16:0/4235525903 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4a601025b0 msgr2=0x7f4a60101f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:40.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:40.479+0000 7f4a654b3640 1 --2- 172.21.15.16:0/4235525903 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4a601025b0 0x7f4a60101f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:40.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:40.479+0000 7f4a654b3640 1 -- 172.21.15.16:0/4235525903 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4a60107810 msgr2=0x7f4a60100aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:40.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:40.479+0000 7f4a654b3640 1 --2- 172.21.15.16:0/4235525903 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4a60107810 0x7f4a60100aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:40.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:40.479+0000 7f4a654b3640 1 -- 172.21.15.16:0/4235525903 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a60101330 con 0x7f4a60104ee0 2024-09-12T02:01:40.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:40.479+0000 7f4a664b5640 1 --2- 172.21.15.16:0/4235525903 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4a60107810 0x7f4a60100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:01:40.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:40.479+0000 7f4a654b3640 1 --2- 172.21.15.16:0/4235525903 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a60104ee0 0x7f4a60100560 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f4a5400e9f0 tx=0x7f4a5400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:01:40.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:40.479+0000 7f4a5effd640 1 -- 172.21.15.16:0/4235525903 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a5400ccc0 con 0x7f4a60104ee0 2024-09-12T02:01:40.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:40.479+0000 7f4a5effd640 1 -- 172.21.15.16:0/4235525903 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a54005500 con 0x7f4a60104ee0 2024-09-12T02:01:40.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:40.479+0000 7f4a67f18640 1 -- 172.21.15.16:0/4235525903 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a601a1fc0 con 0x7f4a60104ee0 2024-09-12T02:01:40.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:40.479+0000 7f4a5effd640 1 -- 172.21.15.16:0/4235525903 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a54010690 con 0x7f4a60104ee0 2024-09-12T02:01:40.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:40.479+0000 7f4a67f18640 1 -- 172.21.15.16:0/4235525903 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a601a2430 con 0x7f4a60104ee0 2024-09-12T02:01:40.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:40.480+0000 7f4a67f18640 1 -- 172.21.15.16:0/4235525903 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a600fdc80 con 0x7f4a60104ee0 2024-09-12T02:01:40.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:40.481+0000 7f4a5effd640 1 -- 172.21.15.16:0/4235525903 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f4a540046c0 con 0x7f4a60104ee0 2024-09-12T02:01:40.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:40.482+0000 7f4a5effd640 1 --2- 172.21.15.16:0/4235525903 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f4a480a2b90 0x7f4a480a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:40.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:40.482+0000 7f4a65cb4640 1 --2- 172.21.15.16:0/4235525903 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f4a480a2b90 0x7f4a480a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:40.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:40.482+0000 7f4a5effd640 1 -- 172.21.15.16:0/4235525903 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f4a540cf380 con 0x7f4a60104ee0 2024-09-12T02:01:40.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:40.482+0000 7f4a65cb4640 1 --2- 172.21.15.16:0/4235525903 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f4a480a2b90 0x7f4a480a5050 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f4a58005d00 tx=0x7f4a5800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:01:40.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:40.490+0000 7f4a5effd640 1 -- 172.21.15.16:0/4235525903 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4a5409c380 con 0x7f4a60104ee0 2024-09-12T02:01:40.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:01:40 smithi184 ceph-mon[35593]: pgmap v327: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T02:01:40.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:01:40 smithi184 ceph-mon[35593]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-09-12T02:01:40.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:01:40 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/1824604566' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:01:40.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:40.667+0000 7f4a67f18640 1 -- 172.21.15.16:0/4235525903 --> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4a60062630 con 0x7f4a480a2b90 2024-09-12T02:01:40.668 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:01:40 smithi033 ceph-mon[35332]: pgmap v327: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T02:01:40.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:01:40 smithi033 ceph-mon[35332]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-09-12T02:01:40.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:01:40 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/1824604566' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:01:40.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:40.673+0000 7f4a5effd640 1 -- 172.21.15.16:0/4235525903 <== mgr.14150 v2:172.21.15.16:6800/3887444820 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258833 (secure 0 0 0) 0x7f4a60062630 con 0x7f4a480a2b90 2024-09-12T02:01:40.677 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T02:01:40.678 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-12T02:01:40.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:40.676+0000 7f4a67f18640 1 -- 172.21.15.16:0/4235525903 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f4a480a2b90 msgr2=0x7f4a480a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:40.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:40.676+0000 7f4a67f18640 1 --2- 172.21.15.16:0/4235525903 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f4a480a2b90 0x7f4a480a5050 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f4a58005d00 tx=0x7f4a5800b410 comp rx=0 tx=0).stop 2024-09-12T02:01:40.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:40.676+0000 7f4a67f18640 1 -- 172.21.15.16:0/4235525903 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a60104ee0 msgr2=0x7f4a60100560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:40.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:40.676+0000 7f4a67f18640 1 --2- 172.21.15.16:0/4235525903 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a60104ee0 0x7f4a60100560 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f4a5400e9f0 tx=0x7f4a5400eeb0 comp rx=0 tx=0).stop 2024-09-12T02:01:40.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:40.676+0000 7f4a67f18640 1 -- 172.21.15.16:0/4235525903 shutdown_connections 2024-09-12T02:01:40.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:40.676+0000 7f4a67f18640 1 --2- 172.21.15.16:0/4235525903 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f4a480a2b90 0x7f4a480a5050 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:40.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:40.676+0000 7f4a67f18640 1 --2- 172.21.15.16:0/4235525903 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4a60107810 0x7f4a60100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:40.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:40.676+0000 7f4a67f18640 1 --2- 172.21.15.16:0/4235525903 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a60104ee0 0x7f4a60100560 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:40.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:40.676+0000 7f4a67f18640 1 --2- 172.21.15.16:0/4235525903 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4a601025b0 0x7f4a60101f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:40.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:40.676+0000 7f4a67f18640 1 -- 172.21.15.16:0/4235525903 >> 172.21.15.16:0/4235525903 conn(0x7f4a600fc190 msgr2=0x7f4a60102f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:01:40.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:40.677+0000 7f4a67f18640 1 -- 172.21.15.16:0/4235525903 shutdown_connections 2024-09-12T02:01:40.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:40.677+0000 7f4a67f18640 1 -- 172.21.15.16:0/4235525903 wait complete. 2024-09-12T02:01:40.688 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":328,"stamp":"2024-09-12T02:01:40.323810+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":659240,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":717,"num_read_kb":479,"num_write":352,"num_write_kb":3058,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":1180848,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":573,"ondisk_log_size":573,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":374280,"kb_used_data":12544,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124567544,"statfs":{"total":1151940427776,"available":1151557165056,"internally_reserved":0,"allocated":12845056,"data_stored":7190707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":370193723},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4871,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":87,"num_read_kb":52,"num_write":9,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002929"},"pg_stats":[{"pgid":"3.3f","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.318917+0000","last_change":"2024-09-12T01:59:55.503882+0000","last_active":"2024-09-12T02:01:38.318917+0000","last_peered":"2024-09-12T02:01:38.318917+0000","last_clean":"2024-09-12T02:01:38.318917+0000","last_became_active":"2024-09-12T01:59:55.503513+0000","last_became_peered":"2024-09-12T01:59:55.503513+0000","last_unstale":"2024-09-12T02:01:38.318917+0000","last_undegraded":"2024-09-12T02:01:38.318917+0000","last_fullsized":"2024-09-12T02:01:38.318917+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:43:13.098527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"82'4","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.322981+0000","last_change":"2024-09-12T01:59:55.492965+0000","last_active":"2024-09-12T02:01:38.322981+0000","last_peered":"2024-09-12T02:01:38.322981+0000","last_clean":"2024-09-12T02:01:38.322981+0000","last_became_active":"2024-09-12T01:59:55.492613+0000","last_became_peered":"2024-09-12T01:59:55.492613+0000","last_unstale":"2024-09-12T02:01:38.322981+0000","last_undegraded":"2024-09-12T02:01:38.322981+0000","last_fullsized":"2024-09-12T02:01:38.322981+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:58:20.889770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.332184+0000","last_change":"2024-09-12T01:59:55.497334+0000","last_active":"2024-09-12T02:01:38.332184+0000","last_peered":"2024-09-12T02:01:38.332184+0000","last_clean":"2024-09-12T02:01:38.332184+0000","last_became_active":"2024-09-12T01:59:55.497037+0000","last_became_peered":"2024-09-12T01:59:55.497037+0000","last_unstale":"2024-09-12T02:01:38.332184+0000","last_undegraded":"2024-09-12T02:01:38.332184+0000","last_fullsized":"2024-09-12T02:01:38.332184+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:43:04.082918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.331162+0000","last_change":"2024-09-12T01:59:55.495888+0000","last_active":"2024-09-12T02:01:38.331162+0000","last_peered":"2024-09-12T02:01:38.331162+0000","last_clean":"2024-09-12T02:01:38.331162+0000","last_became_active":"2024-09-12T01:59:55.492684+0000","last_became_peered":"2024-09-12T01:59:55.492684+0000","last_unstale":"2024-09-12T02:01:38.331162+0000","last_undegraded":"2024-09-12T02:01:38.331162+0000","last_fullsized":"2024-09-12T02:01:38.331162+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:15:00.500006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.574662+0000","last_change":"2024-09-12T01:59:55.494613+0000","last_active":"2024-09-12T02:00:09.574662+0000","last_peered":"2024-09-12T02:00:09.574662+0000","last_clean":"2024-09-12T02:00:09.574662+0000","last_became_active":"2024-09-12T01:59:55.494205+0000","last_became_peered":"2024-09-12T01:59:55.494205+0000","last_unstale":"2024-09-12T02:00:09.574662+0000","last_undegraded":"2024-09-12T02:00:09.574662+0000","last_fullsized":"2024-09-12T02:00:09.574662+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:39:14.573280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.331517+0000","last_change":"2024-09-12T01:59:55.496659+0000","last_active":"2024-09-12T02:01:38.331517+0000","last_peered":"2024-09-12T02:01:38.331517+0000","last_clean":"2024-09-12T02:01:38.331517+0000","last_became_active":"2024-09-12T01:59:55.496232+0000","last_became_peered":"2024-09-12T01:59:55.496232+0000","last_unstale":"2024-09-12T02:01:38.331517+0000","last_undegraded":"2024-09-12T02:01:38.331517+0000","last_fullsized":"2024-09-12T02:01:38.331517+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:48:21.544764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"82'5","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.317177+0000","last_change":"2024-09-12T01:59:55.486789+0000","last_active":"2024-09-12T02:01:38.317177+0000","last_peered":"2024-09-12T02:01:38.317177+0000","last_clean":"2024-09-12T02:01:38.317177+0000","last_became_active":"2024-09-12T01:59:55.485921+0000","last_became_peered":"2024-09-12T01:59:55.485921+0000","last_unstale":"2024-09-12T02:01:38.317177+0000","last_undegraded":"2024-09-12T02:01:38.317177+0000","last_fullsized":"2024-09-12T02:01:38.317177+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:21:16.071121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.621086+0000","last_change":"2024-09-12T01:59:55.494525+0000","last_active":"2024-09-12T02:00:09.621086+0000","last_peered":"2024-09-12T02:00:09.621086+0000","last_clean":"2024-09-12T02:00:09.621086+0000","last_became_active":"2024-09-12T01:59:55.493998+0000","last_became_peered":"2024-09-12T01:59:55.493998+0000","last_unstale":"2024-09-12T02:00:09.621086+0000","last_undegraded":"2024-09-12T02:00:09.621086+0000","last_fullsized":"2024-09-12T02:00:09.621086+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:59:24.653660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.622633+0000","last_change":"2024-09-12T01:59:55.501466+0000","last_active":"2024-09-12T02:00:09.622633+0000","last_peered":"2024-09-12T02:00:09.622633+0000","last_clean":"2024-09-12T02:00:09.622633+0000","last_became_active":"2024-09-12T01:59:55.500800+0000","last_became_peered":"2024-09-12T01:59:55.500800+0000","last_unstale":"2024-09-12T02:00:09.622633+0000","last_undegraded":"2024-09-12T02:00:09.622633+0000","last_fullsized":"2024-09-12T02:00:09.622633+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:55:13.704791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"82'16","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.331670+0000","last_change":"2024-09-12T01:59:55.497986+0000","last_active":"2024-09-12T02:01:38.331670+0000","last_peered":"2024-09-12T02:01:38.331670+0000","last_clean":"2024-09-12T02:01:38.331670+0000","last_became_active":"2024-09-12T01:59:55.497691+0000","last_became_peered":"2024-09-12T01:59:55.497691+0000","last_unstale":"2024-09-12T02:01:38.331670+0000","last_undegraded":"2024-09-12T02:01:38.331670+0000","last_fullsized":"2024-09-12T02:01:38.331670+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:37:07.702606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.329986+0000","last_change":"2024-09-12T01:59:55.501339+0000","last_active":"2024-09-12T02:01:38.329986+0000","last_peered":"2024-09-12T02:01:38.329986+0000","last_clean":"2024-09-12T02:01:38.329986+0000","last_became_active":"2024-09-12T01:59:55.500700+0000","last_became_peered":"2024-09-12T01:59:55.500700+0000","last_unstale":"2024-09-12T02:01:38.329986+0000","last_undegraded":"2024-09-12T02:01:38.329986+0000","last_fullsized":"2024-09-12T02:01:38.329986+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:12:15.879183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.330871+0000","last_change":"2024-09-12T01:59:55.495196+0000","last_active":"2024-09-12T02:01:38.330871+0000","last_peered":"2024-09-12T02:01:38.330871+0000","last_clean":"2024-09-12T02:01:38.330871+0000","last_became_active":"2024-09-12T01:59:55.494889+0000","last_became_peered":"2024-09-12T01:59:55.494889+0000","last_unstale":"2024-09-12T02:01:38.330871+0000","last_undegraded":"2024-09-12T02:01:38.330871+0000","last_fullsized":"2024-09-12T02:01:38.330871+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:32:30.621946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"82'10","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.315946+0000","last_change":"2024-09-12T01:59:55.492565+0000","last_active":"2024-09-12T02:01:38.315946+0000","last_peered":"2024-09-12T02:01:38.315946+0000","last_clean":"2024-09-12T02:01:38.315946+0000","last_became_active":"2024-09-12T01:59:55.492168+0000","last_became_peered":"2024-09-12T01:59:55.492168+0000","last_unstale":"2024-09-12T02:01:38.315946+0000","last_undegraded":"2024-09-12T02:01:38.315946+0000","last_fullsized":"2024-09-12T02:01:38.315946+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:39:48.630202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.323015+0000","last_change":"2024-09-12T01:59:55.493152+0000","last_active":"2024-09-12T02:01:38.323015+0000","last_peered":"2024-09-12T02:01:38.323015+0000","last_clean":"2024-09-12T02:01:38.323015+0000","last_became_active":"2024-09-12T01:59:55.492794+0000","last_became_peered":"2024-09-12T01:59:55.492794+0000","last_unstale":"2024-09-12T02:01:38.323015+0000","last_undegraded":"2024-09-12T02:01:38.323015+0000","last_fullsized":"2024-09-12T02:01:38.323015+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:24:27.270502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.333374+0000","last_change":"2024-09-12T01:59:55.494651+0000","last_active":"2024-09-12T02:01:38.333374+0000","last_peered":"2024-09-12T02:01:38.333374+0000","last_clean":"2024-09-12T02:01:38.333374+0000","last_became_active":"2024-09-12T01:59:55.494355+0000","last_became_peered":"2024-09-12T01:59:55.494355+0000","last_unstale":"2024-09-12T02:01:38.333374+0000","last_undegraded":"2024-09-12T02:01:38.333374+0000","last_fullsized":"2024-09-12T02:01:38.333374+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:14:44.364158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.622950+0000","last_change":"2024-09-12T01:59:55.495049+0000","last_active":"2024-09-12T02:00:09.622950+0000","last_peered":"2024-09-12T02:00:09.622950+0000","last_clean":"2024-09-12T02:00:09.622950+0000","last_became_active":"2024-09-12T01:59:55.494688+0000","last_became_peered":"2024-09-12T01:59:55.494688+0000","last_unstale":"2024-09-12T02:00:09.622950+0000","last_undegraded":"2024-09-12T02:00:09.622950+0000","last_fullsized":"2024-09-12T02:00:09.622950+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:13:05.944577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.319055+0000","last_change":"2024-09-12T01:59:55.504146+0000","last_active":"2024-09-12T02:01:38.319055+0000","last_peered":"2024-09-12T02:01:38.319055+0000","last_clean":"2024-09-12T02:01:38.319055+0000","last_became_active":"2024-09-12T01:59:55.503646+0000","last_became_peered":"2024-09-12T01:59:55.503646+0000","last_unstale":"2024-09-12T02:01:38.319055+0000","last_undegraded":"2024-09-12T02:01:38.319055+0000","last_fullsized":"2024-09-12T02:01:38.319055+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:23:51.780795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"82'10","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.325350+0000","last_change":"2024-09-12T01:59:55.489930+0000","last_active":"2024-09-12T02:01:38.325350+0000","last_peered":"2024-09-12T02:01:38.325350+0000","last_clean":"2024-09-12T02:01:38.325350+0000","last_became_active":"2024-09-12T01:59:55.489619+0000","last_became_peered":"2024-09-12T01:59:55.489619+0000","last_unstale":"2024-09-12T02:01:38.325350+0000","last_undegraded":"2024-09-12T02:01:38.325350+0000","last_fullsized":"2024-09-12T02:01:38.325350+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:04:57.307091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.569107+0000","last_change":"2024-09-12T01:59:55.492098+0000","last_active":"2024-09-12T02:00:09.569107+0000","last_peered":"2024-09-12T02:00:09.569107+0000","last_clean":"2024-09-12T02:00:09.569107+0000","last_became_active":"2024-09-12T01:59:55.491370+0000","last_became_peered":"2024-09-12T01:59:55.491370+0000","last_unstale":"2024-09-12T02:00:09.569107+0000","last_undegraded":"2024-09-12T02:00:09.569107+0000","last_fullsized":"2024-09-12T02:00:09.569107+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:25:06.307903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.314843+0000","last_change":"2024-09-12T01:59:55.499938+0000","last_active":"2024-09-12T02:01:38.314843+0000","last_peered":"2024-09-12T02:01:38.314843+0000","last_clean":"2024-09-12T02:01:38.314843+0000","last_became_active":"2024-09-12T01:59:55.499619+0000","last_became_peered":"2024-09-12T01:59:55.499619+0000","last_unstale":"2024-09-12T02:01:38.314843+0000","last_undegraded":"2024-09-12T02:01:38.314843+0000","last_fullsized":"2024-09-12T02:01:38.314843+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:00:17.536120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"82'11","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.321068+0000","last_change":"2024-09-12T01:59:55.486106+0000","last_active":"2024-09-12T02:01:38.321068+0000","last_peered":"2024-09-12T02:01:38.321068+0000","last_clean":"2024-09-12T02:01:38.321068+0000","last_became_active":"2024-09-12T01:59:55.485325+0000","last_became_peered":"2024-09-12T01:59:55.485325+0000","last_unstale":"2024-09-12T02:01:38.321068+0000","last_undegraded":"2024-09-12T02:01:38.321068+0000","last_fullsized":"2024-09-12T02:01:38.321068+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:35:46.446224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.622237+0000","last_change":"2024-09-12T01:59:55.496438+0000","last_active":"2024-09-12T02:00:09.622237+0000","last_peered":"2024-09-12T02:00:09.622237+0000","last_clean":"2024-09-12T02:00:09.622237+0000","last_became_active":"2024-09-12T01:59:55.495885+0000","last_became_peered":"2024-09-12T01:59:55.495885+0000","last_unstale":"2024-09-12T02:00:09.622237+0000","last_undegraded":"2024-09-12T02:00:09.622237+0000","last_fullsized":"2024-09-12T02:00:09.622237+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:15:14.911335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.621139+0000","last_change":"2024-09-12T01:59:55.489350+0000","last_active":"2024-09-12T02:00:09.621139+0000","last_peered":"2024-09-12T02:00:09.621139+0000","last_clean":"2024-09-12T02:00:09.621139+0000","last_became_active":"2024-09-12T01:59:55.489028+0000","last_became_peered":"2024-09-12T01:59:55.489028+0000","last_unstale":"2024-09-12T02:00:09.621139+0000","last_undegraded":"2024-09-12T02:00:09.621139+0000","last_fullsized":"2024-09-12T02:00:09.621139+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:29:23.769004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.566645+0000","last_change":"2024-09-12T01:59:55.494532+0000","last_active":"2024-09-12T02:00:09.566645+0000","last_peered":"2024-09-12T02:00:09.566645+0000","last_clean":"2024-09-12T02:00:09.566645+0000","last_became_active":"2024-09-12T01:59:55.490982+0000","last_became_peered":"2024-09-12T01:59:55.490982+0000","last_unstale":"2024-09-12T02:00:09.566645+0000","last_undegraded":"2024-09-12T02:00:09.566645+0000","last_fullsized":"2024-09-12T02:00:09.566645+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:04:50.633730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.621490+0000","last_change":"2024-09-12T01:59:55.496057+0000","last_active":"2024-09-12T02:00:09.621490+0000","last_peered":"2024-09-12T02:00:09.621490+0000","last_clean":"2024-09-12T02:00:09.621490+0000","last_became_active":"2024-09-12T01:59:55.495710+0000","last_became_peered":"2024-09-12T01:59:55.495710+0000","last_unstale":"2024-09-12T02:00:09.621490+0000","last_undegraded":"2024-09-12T02:00:09.621490+0000","last_fullsized":"2024-09-12T02:00:09.621490+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:49:01.962537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.569100+0000","last_change":"2024-09-12T01:59:55.491594+0000","last_active":"2024-09-12T02:00:09.569100+0000","last_peered":"2024-09-12T02:00:09.569100+0000","last_clean":"2024-09-12T02:00:09.569100+0000","last_became_active":"2024-09-12T01:59:55.491233+0000","last_became_peered":"2024-09-12T01:59:55.491233+0000","last_unstale":"2024-09-12T02:00:09.569100+0000","last_undegraded":"2024-09-12T02:00:09.569100+0000","last_fullsized":"2024-09-12T02:00:09.569100+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:24:22.066777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.309442+0000","last_change":"2024-09-12T01:59:55.498331+0000","last_active":"2024-09-12T02:01:38.309442+0000","last_peered":"2024-09-12T02:01:38.309442+0000","last_clean":"2024-09-12T02:01:38.309442+0000","last_became_active":"2024-09-12T01:59:55.497983+0000","last_became_peered":"2024-09-12T01:59:55.497983+0000","last_unstale":"2024-09-12T02:01:38.309442+0000","last_undegraded":"2024-09-12T02:01:38.309442+0000","last_fullsized":"2024-09-12T02:01:38.309442+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:37:00.735231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"82'4","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.315811+0000","last_change":"2024-09-12T02:00:09.575358+0000","last_active":"2024-09-12T02:01:38.315811+0000","last_peered":"2024-09-12T02:01:38.315811+0000","last_clean":"2024-09-12T02:01:38.315811+0000","last_became_active":"2024-09-12T02:00:09.574913+0000","last_became_peered":"2024-09-12T02:00:09.574913+0000","last_unstale":"2024-09-12T02:01:38.315811+0000","last_undegraded":"2024-09-12T02:01:38.315811+0000","last_fullsized":"2024-09-12T02:01:38.315811+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:19:53.630453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.333406+0000","last_change":"2024-09-12T01:59:55.492419+0000","last_active":"2024-09-12T02:01:38.333406+0000","last_peered":"2024-09-12T02:01:38.333406+0000","last_clean":"2024-09-12T02:01:38.333406+0000","last_became_active":"2024-09-12T01:59:55.492003+0000","last_became_peered":"2024-09-12T01:59:55.492003+0000","last_unstale":"2024-09-12T02:01:38.333406+0000","last_undegraded":"2024-09-12T02:01:38.333406+0000","last_fullsized":"2024-09-12T02:01:38.333406+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:25:45.774679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.568801+0000","last_change":"2024-09-12T01:59:55.490125+0000","last_active":"2024-09-12T02:00:09.568801+0000","last_peered":"2024-09-12T02:00:09.568801+0000","last_clean":"2024-09-12T02:00:09.568801+0000","last_became_active":"2024-09-12T01:59:55.489636+0000","last_became_peered":"2024-09-12T01:59:55.489636+0000","last_unstale":"2024-09-12T02:00:09.568801+0000","last_undegraded":"2024-09-12T02:00:09.568801+0000","last_fullsized":"2024-09-12T02:00:09.568801+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:41:30.922185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.569135+0000","last_change":"2024-09-12T01:59:55.496322+0000","last_active":"2024-09-12T02:00:09.569135+0000","last_peered":"2024-09-12T02:00:09.569135+0000","last_clean":"2024-09-12T02:00:09.569135+0000","last_became_active":"2024-09-12T01:59:55.496005+0000","last_became_peered":"2024-09-12T01:59:55.496005+0000","last_unstale":"2024-09-12T02:00:09.569135+0000","last_undegraded":"2024-09-12T02:00:09.569135+0000","last_fullsized":"2024-09-12T02:00:09.569135+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:53:39.248715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.310438+0000","last_change":"2024-09-12T01:59:55.497758+0000","last_active":"2024-09-12T02:01:38.310438+0000","last_peered":"2024-09-12T02:01:38.310438+0000","last_clean":"2024-09-12T02:01:38.310438+0000","last_became_active":"2024-09-12T01:59:55.497282+0000","last_became_peered":"2024-09-12T01:59:55.497282+0000","last_unstale":"2024-09-12T02:01:38.310438+0000","last_undegraded":"2024-09-12T02:01:38.310438+0000","last_fullsized":"2024-09-12T02:01:38.310438+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:08:25.509372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.574620+0000","last_change":"2024-09-12T01:59:55.497919+0000","last_active":"2024-09-12T02:00:09.574620+0000","last_peered":"2024-09-12T02:00:09.574620+0000","last_clean":"2024-09-12T02:00:09.574620+0000","last_became_active":"2024-09-12T01:59:55.497609+0000","last_became_peered":"2024-09-12T01:59:55.497609+0000","last_unstale":"2024-09-12T02:00:09.574620+0000","last_undegraded":"2024-09-12T02:00:09.574620+0000","last_fullsized":"2024-09-12T02:00:09.574620+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:10:22.281180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"81'6","reported_seq":122,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:34.680658+0000","last_change":"2024-09-12T01:59:53.473974+0000","last_active":"2024-09-12T02:01:34.680658+0000","last_peered":"2024-09-12T02:01:34.680658+0000","last_clean":"2024-09-12T02:01:34.680658+0000","last_became_active":"2024-09-12T01:59:53.473546+0000","last_became_peered":"2024-09-12T01:59:53.473546+0000","last_unstale":"2024-09-12T02:01:34.680658+0000","last_undegraded":"2024-09-12T02:01:34.680658+0000","last_fullsized":"2024-09-12T02:01:34.680658+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:28:03.532249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":94,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.567217+0000","last_change":"2024-09-12T01:59:55.493197+0000","last_active":"2024-09-12T02:00:09.567217+0000","last_peered":"2024-09-12T02:00:09.567217+0000","last_clean":"2024-09-12T02:00:09.567217+0000","last_became_active":"2024-09-12T01:59:55.492845+0000","last_became_peered":"2024-09-12T01:59:55.492845+0000","last_unstale":"2024-09-12T02:00:09.567217+0000","last_undegraded":"2024-09-12T02:00:09.567217+0000","last_fullsized":"2024-09-12T02:00:09.567217+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:42:50.790896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"80'3","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.622865+0000","last_change":"2024-09-12T01:59:53.471214+0000","last_active":"2024-09-12T02:00:09.622865+0000","last_peered":"2024-09-12T02:00:09.622865+0000","last_clean":"2024-09-12T02:00:09.622865+0000","last_became_active":"2024-09-12T01:59:53.470858+0000","last_became_peered":"2024-09-12T01:59:53.470858+0000","last_unstale":"2024-09-12T02:00:09.622865+0000","last_undegraded":"2024-09-12T02:00:09.622865+0000","last_fullsized":"2024-09-12T02:00:09.622865+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:34:48.730392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"80'2","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.562106+0000","last_change":"2024-09-12T01:59:53.464568+0000","last_active":"2024-09-12T02:00:09.562106+0000","last_peered":"2024-09-12T02:00:09.562106+0000","last_clean":"2024-09-12T02:00:09.562106+0000","last_became_active":"2024-09-12T01:59:53.464168+0000","last_became_peered":"2024-09-12T01:59:53.464168+0000","last_unstale":"2024-09-12T02:00:09.562106+0000","last_undegraded":"2024-09-12T02:00:09.562106+0000","last_fullsized":"2024-09-12T02:00:09.562106+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:41:48.135522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.621151+0000","last_change":"2024-09-12T01:59:55.487177+0000","last_active":"2024-09-12T02:00:09.621151+0000","last_peered":"2024-09-12T02:00:09.621151+0000","last_clean":"2024-09-12T02:00:09.621151+0000","last_became_active":"2024-09-12T01:59:55.486886+0000","last_became_peered":"2024-09-12T01:59:55.486886+0000","last_unstale":"2024-09-12T02:00:09.621151+0000","last_undegraded":"2024-09-12T02:00:09.621151+0000","last_fullsized":"2024-09-12T02:00:09.621151+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:32:40.150071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"80'1","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.567129+0000","last_change":"2024-09-12T01:59:53.474114+0000","last_active":"2024-09-12T02:00:09.567129+0000","last_peered":"2024-09-12T02:00:09.567129+0000","last_clean":"2024-09-12T02:00:09.567129+0000","last_became_active":"2024-09-12T01:59:53.473755+0000","last_became_peered":"2024-09-12T01:59:53.473755+0000","last_unstale":"2024-09-12T02:00:09.567129+0000","last_undegraded":"2024-09-12T02:00:09.567129+0000","last_fullsized":"2024-09-12T02:00:09.567129+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:07:07.021137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"82'11","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.323623+0000","last_change":"2024-09-12T01:59:55.489868+0000","last_active":"2024-09-12T02:01:38.323623+0000","last_peered":"2024-09-12T02:01:38.323623+0000","last_clean":"2024-09-12T02:01:38.323623+0000","last_became_active":"2024-09-12T01:59:55.489537+0000","last_became_peered":"2024-09-12T01:59:55.489537+0000","last_unstale":"2024-09-12T02:01:38.323623+0000","last_undegraded":"2024-09-12T02:01:38.323623+0000","last_fullsized":"2024-09-12T02:01:38.323623+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:18:49.417689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"82'4","reported_seq":120,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.276658+0000","last_change":"2024-09-12T01:59:53.471402+0000","last_active":"2024-09-12T02:01:38.276658+0000","last_peered":"2024-09-12T02:01:38.276658+0000","last_clean":"2024-09-12T02:01:38.276658+0000","last_became_active":"2024-09-12T01:59:53.470940+0000","last_became_peered":"2024-09-12T01:59:53.470940+0000","last_unstale":"2024-09-12T02:01:38.276658+0000","last_undegraded":"2024-09-12T02:01:38.276658+0000","last_fullsized":"2024-09-12T02:01:38.276658+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:41:16.853592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":94,"num_read_kb":94,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.623260+0000","last_change":"2024-09-12T01:59:55.499047+0000","last_active":"2024-09-12T02:00:09.623260+0000","last_peered":"2024-09-12T02:00:09.623260+0000","last_clean":"2024-09-12T02:00:09.623260+0000","last_became_active":"2024-09-12T01:59:55.498071+0000","last_became_peered":"2024-09-12T01:59:55.498071+0000","last_unstale":"2024-09-12T02:00:09.623260+0000","last_undegraded":"2024-09-12T02:00:09.623260+0000","last_fullsized":"2024-09-12T02:00:09.623260+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:36:40.106041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"80'2","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.622966+0000","last_change":"2024-09-12T01:59:53.473508+0000","last_active":"2024-09-12T02:00:09.622966+0000","last_peered":"2024-09-12T02:00:09.622966+0000","last_clean":"2024-09-12T02:00:09.622966+0000","last_became_active":"2024-09-12T01:59:53.473063+0000","last_became_peered":"2024-09-12T01:59:53.473063+0000","last_unstale":"2024-09-12T02:00:09.622966+0000","last_undegraded":"2024-09-12T02:00:09.622966+0000","last_fullsized":"2024-09-12T02:00:09.622966+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:55:32.161769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.333032+0000","last_change":"2024-09-12T01:59:55.491721+0000","last_active":"2024-09-12T02:01:38.333032+0000","last_peered":"2024-09-12T02:01:38.333032+0000","last_clean":"2024-09-12T02:01:38.333032+0000","last_became_active":"2024-09-12T01:59:55.491364+0000","last_became_peered":"2024-09-12T01:59:55.491364+0000","last_unstale":"2024-09-12T02:01:38.333032+0000","last_undegraded":"2024-09-12T02:01:38.333032+0000","last_fullsized":"2024-09-12T02:01:38.333032+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:21:20.795420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"80'4","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.567315+0000","last_change":"2024-09-12T01:59:53.921789+0000","last_active":"2024-09-12T02:00:09.567315+0000","last_peered":"2024-09-12T02:00:09.567315+0000","last_clean":"2024-09-12T02:00:09.567315+0000","last_became_active":"2024-09-12T01:59:53.921383+0000","last_became_peered":"2024-09-12T01:59:53.921383+0000","last_unstale":"2024-09-12T02:00:09.567315+0000","last_undegraded":"2024-09-12T02:00:09.567315+0000","last_fullsized":"2024-09-12T02:00:09.567315+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:21:58.792794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.622517+0000","last_change":"2024-09-12T01:59:55.503008+0000","last_active":"2024-09-12T02:00:09.622517+0000","last_peered":"2024-09-12T02:00:09.622517+0000","last_clean":"2024-09-12T02:00:09.622517+0000","last_became_active":"2024-09-12T01:59:55.502538+0000","last_became_peered":"2024-09-12T01:59:55.502538+0000","last_unstale":"2024-09-12T02:00:09.622517+0000","last_undegraded":"2024-09-12T02:00:09.622517+0000","last_fullsized":"2024-09-12T02:00:09.622517+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:51:36.735985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"82'32","reported_seq":253,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.686996+0000","last_change":"2024-09-12T01:59:53.470688+0000","last_active":"2024-09-12T02:01:38.686996+0000","last_peered":"2024-09-12T02:01:38.686996+0000","last_clean":"2024-09-12T02:01:38.686996+0000","last_became_active":"2024-09-12T01:59:53.470231+0000","last_became_peered":"2024-09-12T01:59:53.470231+0000","last_unstale":"2024-09-12T02:01:38.686996+0000","last_undegraded":"2024-09-12T02:01:38.686996+0000","last_fullsized":"2024-09-12T02:01:38.686996+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:53:54.439893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64635,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":199,"num_read_kb":163,"num_write":33,"num_write_kb":76,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.575173+0000","last_change":"2024-09-12T01:59:55.495343+0000","last_active":"2024-09-12T02:00:09.575173+0000","last_peered":"2024-09-12T02:00:09.575173+0000","last_clean":"2024-09-12T02:00:09.575173+0000","last_became_active":"2024-09-12T01:59:55.495006+0000","last_became_peered":"2024-09-12T01:59:55.495006+0000","last_unstale":"2024-09-12T02:00:09.575173+0000","last_undegraded":"2024-09-12T02:00:09.575173+0000","last_fullsized":"2024-09-12T02:00:09.575173+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:32:20.932551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'6","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.569082+0000","last_change":"2024-09-12T01:59:53.468625+0000","last_active":"2024-09-12T02:00:09.569082+0000","last_peered":"2024-09-12T02:00:09.569082+0000","last_clean":"2024-09-12T02:00:09.569082+0000","last_became_active":"2024-09-12T01:59:53.468015+0000","last_became_peered":"2024-09-12T01:59:53.468015+0000","last_unstale":"2024-09-12T02:00:09.569082+0000","last_undegraded":"2024-09-12T02:00:09.569082+0000","last_fullsized":"2024-09-12T02:00:09.569082+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:49:34.842239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.319752+0000","last_change":"2024-09-12T01:59:55.487962+0000","last_active":"2024-09-12T02:01:38.319752+0000","last_peered":"2024-09-12T02:01:38.319752+0000","last_clean":"2024-09-12T02:01:38.319752+0000","last_became_active":"2024-09-12T01:59:55.487619+0000","last_became_peered":"2024-09-12T01:59:55.487619+0000","last_unstale":"2024-09-12T02:01:38.319752+0000","last_undegraded":"2024-09-12T02:01:38.319752+0000","last_fullsized":"2024-09-12T02:01:38.319752+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:32:16.882373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"80'3","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.567124+0000","last_change":"2024-09-12T01:59:53.474418+0000","last_active":"2024-09-12T02:00:09.567124+0000","last_peered":"2024-09-12T02:00:09.567124+0000","last_clean":"2024-09-12T02:00:09.567124+0000","last_became_active":"2024-09-12T01:59:53.474059+0000","last_became_peered":"2024-09-12T01:59:53.474059+0000","last_unstale":"2024-09-12T02:00:09.567124+0000","last_undegraded":"2024-09-12T02:00:09.567124+0000","last_fullsized":"2024-09-12T02:00:09.567124+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:10:54.963435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"82'15","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.333121+0000","last_change":"2024-09-12T01:59:55.493394+0000","last_active":"2024-09-12T02:01:38.333121+0000","last_peered":"2024-09-12T02:01:38.333121+0000","last_clean":"2024-09-12T02:01:38.333121+0000","last_became_active":"2024-09-12T01:59:55.491669+0000","last_became_peered":"2024-09-12T01:59:55.491669+0000","last_unstale":"2024-09-12T02:01:38.333121+0000","last_undegraded":"2024-09-12T02:01:38.333121+0000","last_fullsized":"2024-09-12T02:01:38.333121+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:14:40.871154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"80'3","reported_seq":121,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:37.685132+0000","last_change":"2024-09-12T01:59:53.471623+0000","last_active":"2024-09-12T02:01:37.685132+0000","last_peered":"2024-09-12T02:01:37.685132+0000","last_clean":"2024-09-12T02:01:37.685132+0000","last_became_active":"2024-09-12T01:59:53.471294+0000","last_became_peered":"2024-09-12T01:59:53.471294+0000","last_unstale":"2024-09-12T02:01:37.685132+0000","last_undegraded":"2024-09-12T02:01:37.685132+0000","last_fullsized":"2024-09-12T02:01:37.685132+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:51:54.130883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":97,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.322144+0000","last_change":"2024-09-12T01:59:55.493082+0000","last_active":"2024-09-12T02:01:38.322144+0000","last_peered":"2024-09-12T02:01:38.322144+0000","last_clean":"2024-09-12T02:01:38.322144+0000","last_became_active":"2024-09-12T01:59:55.492778+0000","last_became_peered":"2024-09-12T01:59:55.492778+0000","last_unstale":"2024-09-12T02:01:38.322144+0000","last_undegraded":"2024-09-12T02:01:38.322144+0000","last_fullsized":"2024-09-12T02:01:38.322144+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:59:56.466377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'15","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.588050+0000","last_change":"2024-09-12T02:00:09.588050+0000","last_active":"2024-09-12T02:00:09.588050+0000","last_peered":"2024-09-12T02:00:09.588050+0000","last_clean":"2024-09-12T02:00:09.588050+0000","last_became_active":"2024-09-12T02:00:09.575478+0000","last_became_peered":"2024-09-12T02:00:09.575478+0000","last_unstale":"2024-09-12T02:00:09.588050+0000","last_undegraded":"2024-09-12T02:00:09.588050+0000","last_fullsized":"2024-09-12T02:00:09.588050+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:38:45.484167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.566976+0000","last_change":"2024-09-12T01:59:55.497601+0000","last_active":"2024-09-12T02:00:09.566976+0000","last_peered":"2024-09-12T02:00:09.566976+0000","last_clean":"2024-09-12T02:00:09.566976+0000","last_became_active":"2024-09-12T01:59:55.497202+0000","last_became_peered":"2024-09-12T01:59:55.497202+0000","last_unstale":"2024-09-12T02:00:09.566976+0000","last_undegraded":"2024-09-12T02:00:09.566976+0000","last_fullsized":"2024-09-12T02:00:09.566976+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:22:22.850736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"80'2","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.575359+0000","last_change":"2024-09-12T01:59:53.467575+0000","last_active":"2024-09-12T02:00:09.575359+0000","last_peered":"2024-09-12T02:00:09.575359+0000","last_clean":"2024-09-12T02:00:09.575359+0000","last_became_active":"2024-09-12T01:59:53.467146+0000","last_became_peered":"2024-09-12T01:59:53.467146+0000","last_unstale":"2024-09-12T02:00:09.575359+0000","last_undegraded":"2024-09-12T02:00:09.575359+0000","last_fullsized":"2024-09-12T02:00:09.575359+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:07:25.962496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"82'5","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.314382+0000","last_change":"2024-09-12T01:59:55.498297+0000","last_active":"2024-09-12T02:01:38.314382+0000","last_peered":"2024-09-12T02:01:38.314382+0000","last_clean":"2024-09-12T02:01:38.314382+0000","last_became_active":"2024-09-12T01:59:55.497367+0000","last_became_peered":"2024-09-12T01:59:55.497367+0000","last_unstale":"2024-09-12T02:01:38.314382+0000","last_undegraded":"2024-09-12T02:01:38.314382+0000","last_fullsized":"2024-09-12T02:01:38.314382+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:58:32.717710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"80'2","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.562292+0000","last_change":"2024-09-12T01:59:53.466664+0000","last_active":"2024-09-12T02:00:09.562292+0000","last_peered":"2024-09-12T02:00:09.562292+0000","last_clean":"2024-09-12T02:00:09.562292+0000","last_became_active":"2024-09-12T01:59:53.466318+0000","last_became_peered":"2024-09-12T01:59:53.466318+0000","last_unstale":"2024-09-12T02:00:09.562292+0000","last_undegraded":"2024-09-12T02:00:09.562292+0000","last_fullsized":"2024-09-12T02:00:09.562292+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:12:13.541420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.313289+0000","last_change":"2024-09-12T01:59:55.490420+0000","last_active":"2024-09-12T02:01:38.313289+0000","last_peered":"2024-09-12T02:01:38.313289+0000","last_clean":"2024-09-12T02:01:38.313289+0000","last_became_active":"2024-09-12T01:59:55.489757+0000","last_became_peered":"2024-09-12T01:59:55.489757+0000","last_unstale":"2024-09-12T02:01:38.313289+0000","last_undegraded":"2024-09-12T02:01:38.313289+0000","last_fullsized":"2024-09-12T02:01:38.313289+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:34:43.117142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'1","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.567185+0000","last_change":"2024-09-12T01:59:53.466399+0000","last_active":"2024-09-12T02:00:09.567185+0000","last_peered":"2024-09-12T02:00:09.567185+0000","last_clean":"2024-09-12T02:00:09.567185+0000","last_became_active":"2024-09-12T01:59:53.465982+0000","last_became_peered":"2024-09-12T01:59:53.465982+0000","last_unstale":"2024-09-12T02:00:09.567185+0000","last_undegraded":"2024-09-12T02:00:09.567185+0000","last_fullsized":"2024-09-12T02:00:09.567185+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:18:02.015040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.326034+0000","last_change":"2024-09-12T01:59:55.491527+0000","last_active":"2024-09-12T02:01:38.326034+0000","last_peered":"2024-09-12T02:01:38.326034+0000","last_clean":"2024-09-12T02:01:38.326034+0000","last_became_active":"2024-09-12T01:59:55.491165+0000","last_became_peered":"2024-09-12T01:59:55.491165+0000","last_unstale":"2024-09-12T02:01:38.326034+0000","last_undegraded":"2024-09-12T02:01:38.326034+0000","last_fullsized":"2024-09-12T02:01:38.326034+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:57:05.506384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"80'3","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.569187+0000","last_change":"2024-09-12T01:59:54.075752+0000","last_active":"2024-09-12T02:00:09.569187+0000","last_peered":"2024-09-12T02:00:09.569187+0000","last_clean":"2024-09-12T02:00:09.569187+0000","last_became_active":"2024-09-12T01:59:54.075387+0000","last_became_peered":"2024-09-12T01:59:54.075387+0000","last_unstale":"2024-09-12T02:00:09.569187+0000","last_undegraded":"2024-09-12T02:00:09.569187+0000","last_fullsized":"2024-09-12T02:00:09.569187+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:21:15.980591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.330203+0000","last_change":"2024-09-12T01:59:55.500935+0000","last_active":"2024-09-12T02:01:38.330203+0000","last_peered":"2024-09-12T02:01:38.330203+0000","last_clean":"2024-09-12T02:01:38.330203+0000","last_became_active":"2024-09-12T01:59:55.500206+0000","last_became_peered":"2024-09-12T01:59:55.500206+0000","last_unstale":"2024-09-12T02:01:38.330203+0000","last_undegraded":"2024-09-12T02:01:38.330203+0000","last_fullsized":"2024-09-12T02:01:38.330203+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:19:59.937701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":334,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.623254+0000","last_change":"2024-09-12T01:58:47.553853+0000","last_active":"2024-09-12T02:00:09.623254+0000","last_peered":"2024-09-12T02:00:09.623254+0000","last_clean":"2024-09-12T02:00:09.623254+0000","last_became_active":"2024-09-12T01:58:47.553357+0000","last_became_peered":"2024-09-12T01:58:47.553357+0000","last_unstale":"2024-09-12T02:00:09.623254+0000","last_undegraded":"2024-09-12T02:00:09.623254+0000","last_fullsized":"2024-09-12T02:00:09.623254+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:55:26.917733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:55:26.917733+0000","last_clean_scrub_stamp":"2024-09-12T01:55:26.917733+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:58:26.081766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"80'3","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.563615+0000","last_change":"2024-09-12T01:59:53.467907+0000","last_active":"2024-09-12T02:00:09.563615+0000","last_peered":"2024-09-12T02:00:09.563615+0000","last_clean":"2024-09-12T02:00:09.563615+0000","last_became_active":"2024-09-12T01:59:53.467164+0000","last_became_peered":"2024-09-12T01:59:53.467164+0000","last_unstale":"2024-09-12T02:00:09.563615+0000","last_undegraded":"2024-09-12T02:00:09.563615+0000","last_fullsized":"2024-09-12T02:00:09.563615+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:09:35.242708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.621290+0000","last_change":"2024-09-12T01:59:55.495109+0000","last_active":"2024-09-12T02:00:09.621290+0000","last_peered":"2024-09-12T02:00:09.621290+0000","last_clean":"2024-09-12T02:00:09.621290+0000","last_became_active":"2024-09-12T01:59:55.494791+0000","last_became_peered":"2024-09-12T01:59:55.494791+0000","last_unstale":"2024-09-12T02:00:09.621290+0000","last_undegraded":"2024-09-12T02:00:09.621290+0000","last_fullsized":"2024-09-12T02:00:09.621290+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:22:43.882050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"80'3","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.623115+0000","last_change":"2024-09-12T01:59:53.473286+0000","last_active":"2024-09-12T02:00:09.623115+0000","last_peered":"2024-09-12T02:00:09.623115+0000","last_clean":"2024-09-12T02:00:09.623115+0000","last_became_active":"2024-09-12T01:59:53.472976+0000","last_became_peered":"2024-09-12T01:59:53.472976+0000","last_unstale":"2024-09-12T02:00:09.623115+0000","last_undegraded":"2024-09-12T02:00:09.623115+0000","last_fullsized":"2024-09-12T02:00:09.623115+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:32:39.604173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"82'5","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.313749+0000","last_change":"2024-09-12T02:00:09.571159+0000","last_active":"2024-09-12T02:01:38.313749+0000","last_peered":"2024-09-12T02:01:38.313749+0000","last_clean":"2024-09-12T02:01:38.313749+0000","last_became_active":"2024-09-12T02:00:09.570672+0000","last_became_peered":"2024-09-12T02:00:09.570672+0000","last_unstale":"2024-09-12T02:01:38.313749+0000","last_undegraded":"2024-09-12T02:01:38.313749+0000","last_fullsized":"2024-09-12T02:01:38.313749+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:04:10.808942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"80'4","reported_seq":121,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.685685+0000","last_change":"2024-09-12T01:59:53.921856+0000","last_active":"2024-09-12T02:01:38.685685+0000","last_peered":"2024-09-12T02:01:38.685685+0000","last_clean":"2024-09-12T02:01:38.685685+0000","last_became_active":"2024-09-12T01:59:53.920672+0000","last_became_peered":"2024-09-12T01:59:53.920672+0000","last_unstale":"2024-09-12T02:01:38.685685+0000","last_undegraded":"2024-09-12T02:01:38.685685+0000","last_fullsized":"2024-09-12T02:01:38.685685+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:42:33.876493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":95,"num_read_kb":95,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"82'8","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.309498+0000","last_change":"2024-09-12T02:00:09.571185+0000","last_active":"2024-09-12T02:01:38.309498+0000","last_peered":"2024-09-12T02:01:38.309498+0000","last_clean":"2024-09-12T02:01:38.309498+0000","last_became_active":"2024-09-12T02:00:09.570769+0000","last_became_peered":"2024-09-12T02:00:09.570769+0000","last_unstale":"2024-09-12T02:01:38.309498+0000","last_undegraded":"2024-09-12T02:01:38.309498+0000","last_fullsized":"2024-09-12T02:01:38.309498+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:34:17.230435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.2","version":"80'1","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.622691+0000","last_change":"2024-09-12T01:59:53.472072+0000","last_active":"2024-09-12T02:00:09.622691+0000","last_peered":"2024-09-12T02:00:09.622691+0000","last_clean":"2024-09-12T02:00:09.622691+0000","last_became_active":"2024-09-12T01:59:53.471088+0000","last_became_peered":"2024-09-12T01:59:53.471088+0000","last_unstale":"2024-09-12T02:00:09.622691+0000","last_undegraded":"2024-09-12T02:00:09.622691+0000","last_fullsized":"2024-09-12T02:00:09.622691+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:02:13.879131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.330778+0000","last_change":"2024-09-12T01:59:55.502843+0000","last_active":"2024-09-12T02:01:38.330778+0000","last_peered":"2024-09-12T02:01:38.330778+0000","last_clean":"2024-09-12T02:01:38.330778+0000","last_became_active":"2024-09-12T01:59:55.499296+0000","last_became_peered":"2024-09-12T01:59:55.499296+0000","last_unstale":"2024-09-12T02:01:38.330778+0000","last_undegraded":"2024-09-12T02:01:38.330778+0000","last_fullsized":"2024-09-12T02:01:38.330778+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:04:21.787370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.575061+0000","last_change":"2024-09-12T02:00:09.575061+0000","last_active":"2024-09-12T02:00:09.575061+0000","last_peered":"2024-09-12T02:00:09.575061+0000","last_clean":"2024-09-12T02:00:09.575061+0000","last_became_active":"2024-09-12T02:00:09.574653+0000","last_became_peered":"2024-09-12T02:00:09.574653+0000","last_unstale":"2024-09-12T02:00:09.575061+0000","last_undegraded":"2024-09-12T02:00:09.575061+0000","last_fullsized":"2024-09-12T02:00:09.575061+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:32:27.931211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"82'5","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.320660+0000","last_change":"2024-09-12T01:59:55.494162+0000","last_active":"2024-09-12T02:01:38.320660+0000","last_peered":"2024-09-12T02:01:38.320660+0000","last_clean":"2024-09-12T02:01:38.320660+0000","last_became_active":"2024-09-12T01:59:55.493802+0000","last_became_peered":"2024-09-12T01:59:55.493802+0000","last_unstale":"2024-09-12T02:01:38.320660+0000","last_undegraded":"2024-09-12T02:01:38.320660+0000","last_fullsized":"2024-09-12T02:01:38.320660+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:22:16.302683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"80'1","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.575278+0000","last_change":"2024-09-12T01:59:53.921669+0000","last_active":"2024-09-12T02:00:09.575278+0000","last_peered":"2024-09-12T02:00:09.575278+0000","last_clean":"2024-09-12T02:00:09.575278+0000","last_became_active":"2024-09-12T01:59:53.921328+0000","last_became_peered":"2024-09-12T01:59:53.921328+0000","last_unstale":"2024-09-12T02:00:09.575278+0000","last_undegraded":"2024-09-12T02:00:09.575278+0000","last_fullsized":"2024-09-12T02:00:09.575278+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:46:33.313726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.311067+0000","last_change":"2024-09-12T01:59:55.488986+0000","last_active":"2024-09-12T02:01:38.311067+0000","last_peered":"2024-09-12T02:01:38.311067+0000","last_clean":"2024-09-12T02:01:38.311067+0000","last_became_active":"2024-09-12T01:59:55.488632+0000","last_became_peered":"2024-09-12T01:59:55.488632+0000","last_unstale":"2024-09-12T02:01:38.311067+0000","last_undegraded":"2024-09-12T02:01:38.311067+0000","last_fullsized":"2024-09-12T02:01:38.311067+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:20:57.596976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"82'11","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:21.501527+0000","last_change":"2024-09-12T01:59:53.467648+0000","last_active":"2024-09-12T02:00:21.501527+0000","last_peered":"2024-09-12T02:00:21.501527+0000","last_clean":"2024-09-12T02:00:21.501527+0000","last_became_active":"2024-09-12T01:59:53.467255+0000","last_became_peered":"2024-09-12T01:59:53.467255+0000","last_unstale":"2024-09-12T02:00:21.501527+0000","last_undegraded":"2024-09-12T02:00:21.501527+0000","last_fullsized":"2024-09-12T02:00:21.501527+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:37:09.897388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.621489+0000","last_change":"2024-09-12T01:59:55.487603+0000","last_active":"2024-09-12T02:00:09.621489+0000","last_peered":"2024-09-12T02:00:09.621489+0000","last_clean":"2024-09-12T02:00:09.621489+0000","last_became_active":"2024-09-12T01:59:55.487334+0000","last_became_peered":"2024-09-12T01:59:55.487334+0000","last_unstale":"2024-09-12T02:00:09.621489+0000","last_undegraded":"2024-09-12T02:00:09.621489+0000","last_fullsized":"2024-09-12T02:00:09.621489+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:23:16.669515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'4","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.563870+0000","last_change":"2024-09-12T01:59:54.076156+0000","last_active":"2024-09-12T02:00:09.563870+0000","last_peered":"2024-09-12T02:00:09.563870+0000","last_clean":"2024-09-12T02:00:09.563870+0000","last_became_active":"2024-09-12T01:59:54.075794+0000","last_became_peered":"2024-09-12T01:59:54.075794+0000","last_unstale":"2024-09-12T02:00:09.563870+0000","last_undegraded":"2024-09-12T02:00:09.563870+0000","last_fullsized":"2024-09-12T02:00:09.563870+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:06:17.431696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.333603+0000","last_change":"2024-09-12T01:59:55.494604+0000","last_active":"2024-09-12T02:01:38.333603+0000","last_peered":"2024-09-12T02:01:38.333603+0000","last_clean":"2024-09-12T02:01:38.333603+0000","last_became_active":"2024-09-12T01:59:55.494300+0000","last_became_peered":"2024-09-12T01:59:55.494300+0000","last_unstale":"2024-09-12T02:01:38.333603+0000","last_undegraded":"2024-09-12T02:01:38.333603+0000","last_fullsized":"2024-09-12T02:01:38.333603+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:18:56.359299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.622205+0000","last_change":"2024-09-12T01:59:53.921832+0000","last_active":"2024-09-12T02:00:09.622205+0000","last_peered":"2024-09-12T02:00:09.622205+0000","last_clean":"2024-09-12T02:00:09.622205+0000","last_became_active":"2024-09-12T01:59:53.921379+0000","last_became_peered":"2024-09-12T01:59:53.921379+0000","last_unstale":"2024-09-12T02:00:09.622205+0000","last_undegraded":"2024-09-12T02:00:09.622205+0000","last_fullsized":"2024-09-12T02:00:09.622205+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:26:02.277149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.574438+0000","last_change":"2024-09-12T01:59:55.493198+0000","last_active":"2024-09-12T02:00:09.574438+0000","last_peered":"2024-09-12T02:00:09.574438+0000","last_clean":"2024-09-12T02:00:09.574438+0000","last_became_active":"2024-09-12T01:59:55.492834+0000","last_became_peered":"2024-09-12T01:59:55.492834+0000","last_unstale":"2024-09-12T02:00:09.574438+0000","last_undegraded":"2024-09-12T02:00:09.574438+0000","last_fullsized":"2024-09-12T02:00:09.574438+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:58:43.872921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"80'3","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.621258+0000","last_change":"2024-09-12T01:59:53.466528+0000","last_active":"2024-09-12T02:00:09.621258+0000","last_peered":"2024-09-12T02:00:09.621258+0000","last_clean":"2024-09-12T02:00:09.621258+0000","last_became_active":"2024-09-12T01:59:53.466182+0000","last_became_peered":"2024-09-12T01:59:53.466182+0000","last_unstale":"2024-09-12T02:00:09.621258+0000","last_undegraded":"2024-09-12T02:00:09.621258+0000","last_fullsized":"2024-09-12T02:00:09.621258+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:00:26.083539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.316786+0000","last_change":"2024-09-12T01:59:55.494511+0000","last_active":"2024-09-12T02:01:38.316786+0000","last_peered":"2024-09-12T02:01:38.316786+0000","last_clean":"2024-09-12T02:01:38.316786+0000","last_became_active":"2024-09-12T01:59:55.493607+0000","last_became_peered":"2024-09-12T01:59:55.493607+0000","last_unstale":"2024-09-12T02:01:38.316786+0000","last_undegraded":"2024-09-12T02:01:38.316786+0000","last_fullsized":"2024-09-12T02:01:38.316786+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:55:27.434447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"80'1","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.569211+0000","last_change":"2024-09-12T01:59:53.467385+0000","last_active":"2024-09-12T02:00:09.569211+0000","last_peered":"2024-09-12T02:00:09.569211+0000","last_clean":"2024-09-12T02:00:09.569211+0000","last_became_active":"2024-09-12T01:59:53.466997+0000","last_became_peered":"2024-09-12T01:59:53.466997+0000","last_unstale":"2024-09-12T02:00:09.569211+0000","last_undegraded":"2024-09-12T02:00:09.569211+0000","last_fullsized":"2024-09-12T02:00:09.569211+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:54:47.154036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.621293+0000","last_change":"2024-09-12T01:59:55.490757+0000","last_active":"2024-09-12T02:00:09.621293+0000","last_peered":"2024-09-12T02:00:09.621293+0000","last_clean":"2024-09-12T02:00:09.621293+0000","last_became_active":"2024-09-12T01:59:55.490451+0000","last_became_peered":"2024-09-12T01:59:55.490451+0000","last_unstale":"2024-09-12T02:00:09.621293+0000","last_undegraded":"2024-09-12T02:00:09.621293+0000","last_fullsized":"2024-09-12T02:00:09.621293+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:21:26.139049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.574833+0000","last_change":"2024-09-12T01:59:53.472898+0000","last_active":"2024-09-12T02:00:09.574833+0000","last_peered":"2024-09-12T02:00:09.574833+0000","last_clean":"2024-09-12T02:00:09.574833+0000","last_became_active":"2024-09-12T01:59:53.472561+0000","last_became_peered":"2024-09-12T01:59:53.472561+0000","last_unstale":"2024-09-12T02:00:09.574833+0000","last_undegraded":"2024-09-12T02:00:09.574833+0000","last_fullsized":"2024-09-12T02:00:09.574833+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:55:59.933804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"82'5","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.328406+0000","last_change":"2024-09-12T01:59:55.497731+0000","last_active":"2024-09-12T02:01:38.328406+0000","last_peered":"2024-09-12T02:01:38.328406+0000","last_clean":"2024-09-12T02:01:38.328406+0000","last_became_active":"2024-09-12T01:59:55.497356+0000","last_became_peered":"2024-09-12T01:59:55.497356+0000","last_unstale":"2024-09-12T02:01:38.328406+0000","last_undegraded":"2024-09-12T02:01:38.328406+0000","last_fullsized":"2024-09-12T02:01:38.328406+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:57:18.795865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.624453+0000","last_change":"2024-09-12T02:00:09.624453+0000","last_active":"2024-09-12T02:00:09.624453+0000","last_peered":"2024-09-12T02:00:09.624453+0000","last_clean":"2024-09-12T02:00:09.624453+0000","last_became_active":"2024-09-12T02:00:09.624049+0000","last_became_peered":"2024-09-12T02:00:09.624049+0000","last_unstale":"2024-09-12T02:00:09.624453+0000","last_undegraded":"2024-09-12T02:00:09.624453+0000","last_fullsized":"2024-09-12T02:00:09.624453+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:17:40.417275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"80'2","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.575060+0000","last_change":"2024-09-12T01:59:53.470194+0000","last_active":"2024-09-12T02:00:09.575060+0000","last_peered":"2024-09-12T02:00:09.575060+0000","last_clean":"2024-09-12T02:00:09.575060+0000","last_became_active":"2024-09-12T01:59:53.469772+0000","last_became_peered":"2024-09-12T01:59:53.469772+0000","last_unstale":"2024-09-12T02:00:09.575060+0000","last_undegraded":"2024-09-12T02:00:09.575060+0000","last_fullsized":"2024-09-12T02:00:09.575060+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:52:57.709596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.622423+0000","last_change":"2024-09-12T01:59:55.497693+0000","last_active":"2024-09-12T02:00:09.622423+0000","last_peered":"2024-09-12T02:00:09.622423+0000","last_clean":"2024-09-12T02:00:09.622423+0000","last_became_active":"2024-09-12T01:59:55.497380+0000","last_became_peered":"2024-09-12T01:59:55.497380+0000","last_unstale":"2024-09-12T02:00:09.622423+0000","last_undegraded":"2024-09-12T02:00:09.622423+0000","last_fullsized":"2024-09-12T02:00:09.622423+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:42:27.943530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"80'4","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.575274+0000","last_change":"2024-09-12T01:59:53.473044+0000","last_active":"2024-09-12T02:00:09.575274+0000","last_peered":"2024-09-12T02:00:09.575274+0000","last_clean":"2024-09-12T02:00:09.575274+0000","last_became_active":"2024-09-12T01:59:53.472569+0000","last_became_peered":"2024-09-12T01:59:53.472569+0000","last_unstale":"2024-09-12T02:00:09.575274+0000","last_undegraded":"2024-09-12T02:00:09.575274+0000","last_fullsized":"2024-09-12T02:00:09.575274+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:41:29.716836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.328172+0000","last_change":"2024-09-12T01:59:55.497377+0000","last_active":"2024-09-12T02:01:38.328172+0000","last_peered":"2024-09-12T02:01:38.328172+0000","last_clean":"2024-09-12T02:01:38.328172+0000","last_became_active":"2024-09-12T01:59:55.497078+0000","last_became_peered":"2024-09-12T01:59:55.497078+0000","last_unstale":"2024-09-12T02:01:38.328172+0000","last_undegraded":"2024-09-12T02:01:38.328172+0000","last_fullsized":"2024-09-12T02:01:38.328172+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:59:57.362350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.621555+0000","last_change":"2024-09-12T01:59:53.920868+0000","last_active":"2024-09-12T02:00:09.621555+0000","last_peered":"2024-09-12T02:00:09.621555+0000","last_clean":"2024-09-12T02:00:09.621555+0000","last_became_active":"2024-09-12T01:59:53.920481+0000","last_became_peered":"2024-09-12T01:59:53.920481+0000","last_unstale":"2024-09-12T02:00:09.621555+0000","last_undegraded":"2024-09-12T02:00:09.621555+0000","last_fullsized":"2024-09-12T02:00:09.621555+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:49:27.085388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.621438+0000","last_change":"2024-09-12T01:59:55.497703+0000","last_active":"2024-09-12T02:00:09.621438+0000","last_peered":"2024-09-12T02:00:09.621438+0000","last_clean":"2024-09-12T02:00:09.621438+0000","last_became_active":"2024-09-12T01:59:55.497397+0000","last_became_peered":"2024-09-12T01:59:55.497397+0000","last_unstale":"2024-09-12T02:00:09.621438+0000","last_undegraded":"2024-09-12T02:00:09.621438+0000","last_fullsized":"2024-09-12T02:00:09.621438+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:54:02.803203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"80'9","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.565145+0000","last_change":"2024-09-12T01:59:53.469978+0000","last_active":"2024-09-12T02:00:09.565145+0000","last_peered":"2024-09-12T02:00:09.565145+0000","last_clean":"2024-09-12T02:00:09.565145+0000","last_became_active":"2024-09-12T01:59:53.469648+0000","last_became_peered":"2024-09-12T01:59:53.469648+0000","last_unstale":"2024-09-12T02:00:09.565145+0000","last_undegraded":"2024-09-12T02:00:09.565145+0000","last_fullsized":"2024-09-12T02:00:09.565145+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:08:37.720452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":293,"ondisk_log_size":293,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":68638,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":591,"num_read_kb":370,"num_write":169,"num_write_kb":118,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":421888,"data_stored":196621,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":143,"ondisk_log_size":143,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3563520,"data_stored":3542208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":74,"seq":317827579938,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32068,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713084,"statfs":{"total":95995035648,"available":95962198016,"internally_reserved":0,"allocated":897024,"data_stored":435916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":67,"seq":287762808871,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32736,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712416,"statfs":{"total":95995035648,"available":95961513984,"internally_reserved":0,"allocated":1589248,"data_stored":1091844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":9,"up_from":62,"seq":266287972396,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":905216,"data_stored":436040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201328,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":897024,"data_stored":435848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430262,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32648,"kb_used_data":1464,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712504,"statfs":{"total":95995035648,"available":95961604096,"internally_reserved":0,"allocated":1499136,"data_stored":1028033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593787,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36216,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708936,"statfs":{"total":95995035648,"available":95957950464,"internally_reserved":0,"allocated":892928,"data_stored":435429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822719,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32660,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712492,"statfs":{"total":95995035648,"available":95961591808,"internally_reserved":0,"allocated":1511424,"data_stored":1027576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986244,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":897024,"data_stored":435866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215176,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":909312,"data_stored":435740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":17,"seq":73014444110,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36300,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708852,"statfs":{"total":95995035648,"available":95957864448,"internally_reserved":0,"allocated":978944,"data_stored":496729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574931,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32116,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713036,"statfs":{"total":95995035648,"available":95962148864,"internally_reserved":0,"allocated":954368,"data_stored":494986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738457,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":913408,"data_stored":436700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":59764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":61815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":66213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T02:01:40.693 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T02:01:40.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:40.832+0000 7fd1799f6640 1 -- 172.21.15.16:0/2194307311 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd174101b40 msgr2=0x7fd174101f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:40.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:40.832+0000 7fd1799f6640 1 --2- 172.21.15.16:0/2194307311 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd174101b40 0x7fd174101f40 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fd164009900 tx=0x7fd16402a410 comp rx=0 tx=0).stop 2024-09-12T02:01:40.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:40.832+0000 7fd1799f6640 1 -- 172.21.15.16:0/2194307311 shutdown_connections 2024-09-12T02:01:40.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:40.832+0000 7fd1799f6640 1 --2- 172.21.15.16:0/2194307311 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd1741036f0 0x7fd174109f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:40.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:40.832+0000 7fd1799f6640 1 --2- 172.21.15.16:0/2194307311 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd174102d30 0x7fd1741031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:40.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:40.832+0000 7fd1799f6640 1 --2- 172.21.15.16:0/2194307311 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd174101b40 0x7fd174101f40 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:40.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:40.832+0000 7fd1799f6640 1 -- 172.21.15.16:0/2194307311 >> 172.21.15.16:0/2194307311 conn(0x7fd1740fd2e0 msgr2=0x7fd1740ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:01:40.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:40.832+0000 7fd1799f6640 1 -- 172.21.15.16:0/2194307311 shutdown_connections 2024-09-12T02:01:40.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:40.832+0000 7fd1799f6640 1 -- 172.21.15.16:0/2194307311 wait complete. 2024-09-12T02:01:40.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:40.833+0000 7fd1799f6640 1 Processor -- start 2024-09-12T02:01:40.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:40.833+0000 7fd1799f6640 1 -- start start 2024-09-12T02:01:40.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:40.833+0000 7fd1799f6640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd174101b40 0x7fd174197ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:40.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:40.833+0000 7fd1799f6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd174102d30 0x7fd1741983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:40.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:40.833+0000 7fd1799f6640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd1741036f0 0x7fd17419f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:40.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:40.833+0000 7fd1799f6640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd17410d810 con 0x7fd174102d30 2024-09-12T02:01:40.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:40.833+0000 7fd1799f6640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd17410d690 con 0x7fd1741036f0 2024-09-12T02:01:40.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:40.833+0000 7fd1799f6640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd17410d990 con 0x7fd174101b40 2024-09-12T02:01:40.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:40.833+0000 7fd1727fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd174102d30 0x7fd1741983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:40.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:40.833+0000 7fd1727fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd174102d30 0x7fd1741983e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42782/0 (socket says 172.21.15.16:42782) 2024-09-12T02:01:40.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:40.833+0000 7fd1727fc640 1 -- 172.21.15.16:0/4097884842 learned_addr learned my addr 172.21.15.16:0/4097884842 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:01:40.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:40.833+0000 7fd172ffd640 1 --2- 172.21.15.16:0/4097884842 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd174101b40 0x7fd174197ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:40.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:40.833+0000 7fd1737fe640 1 --2- 172.21.15.16:0/4097884842 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd1741036f0 0x7fd17419f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:40.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:40.833+0000 7fd1727fc640 1 -- 172.21.15.16:0/4097884842 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd174101b40 msgr2=0x7fd174197ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:40.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:40.833+0000 7fd1727fc640 1 --2- 172.21.15.16:0/4097884842 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd174101b40 0x7fd174197ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:40.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:40.833+0000 7fd1727fc640 1 -- 172.21.15.16:0/4097884842 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd1741036f0 msgr2=0x7fd17419f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:40.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:40.833+0000 7fd1727fc640 1 --2- 172.21.15.16:0/4097884842 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd1741036f0 0x7fd17419f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:40.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:40.833+0000 7fd1727fc640 1 -- 172.21.15.16:0/4097884842 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd17419fac0 con 0x7fd174102d30 2024-09-12T02:01:40.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:40.833+0000 7fd172ffd640 1 --2- 172.21.15.16:0/4097884842 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd174101b40 0x7fd174197ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:01:40.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:40.833+0000 7fd1737fe640 1 --2- 172.21.15.16:0/4097884842 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd1741036f0 0x7fd17419f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:01:40.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:40.834+0000 7fd1727fc640 1 --2- 172.21.15.16:0/4097884842 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd174102d30 0x7fd1741983e0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fd15c00e9f0 tx=0x7fd15c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:01:40.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:40.834+0000 7fd1789f4640 1 -- 172.21.15.16:0/4097884842 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd15c00cd80 con 0x7fd174102d30 2024-09-12T02:01:40.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:40.834+0000 7fd1799f6640 1 -- 172.21.15.16:0/4097884842 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd17419fdb0 con 0x7fd174102d30 2024-09-12T02:01:40.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:40.834+0000 7fd1789f4640 1 -- 172.21.15.16:0/4097884842 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd15c010910 con 0x7fd174102d30 2024-09-12T02:01:40.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:40.834+0000 7fd1789f4640 1 -- 172.21.15.16:0/4097884842 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd15c0188f0 con 0x7fd174102d30 2024-09-12T02:01:40.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:40.834+0000 7fd1799f6640 1 -- 172.21.15.16:0/4097884842 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1741a0270 con 0x7fd174102d30 2024-09-12T02:01:40.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:40.835+0000 7fd1799f6640 1 -- 172.21.15.16:0/4097884842 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd174108f80 con 0x7fd174102d30 2024-09-12T02:01:40.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:40.836+0000 7fd1789f4640 1 -- 172.21.15.16:0/4097884842 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fd15c010ac0 con 0x7fd174102d30 2024-09-12T02:01:40.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:40.837+0000 7fd1789f4640 1 --2- 172.21.15.16:0/4097884842 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fd1600a2b90 0x7fd1600a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:40.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:40.837+0000 7fd172ffd640 1 --2- 172.21.15.16:0/4097884842 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fd1600a2b90 0x7fd1600a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:40.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:40.837+0000 7fd1789f4640 1 -- 172.21.15.16:0/4097884842 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fd15c0cf460 con 0x7fd174102d30 2024-09-12T02:01:40.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:40.837+0000 7fd172ffd640 1 --2- 172.21.15.16:0/4097884842 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fd1600a2b90 0x7fd1600a5050 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fd16402aa00 tx=0x7fd164000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:01:40.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:40.845+0000 7fd1789f4640 1 -- 172.21.15.16:0/4097884842 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd15c09c460 con 0x7fd174102d30 2024-09-12T02:01:41.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:41.018+0000 7fd1799f6640 1 -- 172.21.15.16:0/4097884842 --> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd174062630 con 0x7fd1600a2b90 2024-09-12T02:01:41.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:41.025+0000 7fd1789f4640 1 -- 172.21.15.16:0/4097884842 <== mgr.14150 v2:172.21.15.16:6800/3887444820 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258833 (secure 0 0 0) 0x7fd174062630 con 0x7fd1600a2b90 2024-09-12T02:01:41.029 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T02:01:41.030 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-12T02:01:41.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:41.027+0000 7fd1799f6640 1 -- 172.21.15.16:0/4097884842 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fd1600a2b90 msgr2=0x7fd1600a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:41.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:41.027+0000 7fd1799f6640 1 --2- 172.21.15.16:0/4097884842 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fd1600a2b90 0x7fd1600a5050 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fd16402aa00 tx=0x7fd164000bc0 comp rx=0 tx=0).stop 2024-09-12T02:01:41.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:41.028+0000 7fd1799f6640 1 -- 172.21.15.16:0/4097884842 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd174102d30 msgr2=0x7fd1741983e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:41.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:41.028+0000 7fd1799f6640 1 --2- 172.21.15.16:0/4097884842 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd174102d30 0x7fd1741983e0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fd15c00e9f0 tx=0x7fd15c00eeb0 comp rx=0 tx=0).stop 2024-09-12T02:01:41.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:41.028+0000 7fd1799f6640 1 -- 172.21.15.16:0/4097884842 shutdown_connections 2024-09-12T02:01:41.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:41.028+0000 7fd1799f6640 1 --2- 172.21.15.16:0/4097884842 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fd1600a2b90 0x7fd1600a5050 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:41.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:41.028+0000 7fd1799f6640 1 --2- 172.21.15.16:0/4097884842 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd1741036f0 0x7fd17419f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:41.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:41.028+0000 7fd1799f6640 1 --2- 172.21.15.16:0/4097884842 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd174102d30 0x7fd1741983e0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:41.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:41.028+0000 7fd1799f6640 1 --2- 172.21.15.16:0/4097884842 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd174101b40 0x7fd174197ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:41.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:41.028+0000 7fd1799f6640 1 -- 172.21.15.16:0/4097884842 >> 172.21.15.16:0/4097884842 conn(0x7fd1740fd2e0 msgr2=0x7fd174106150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:01:41.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:41.028+0000 7fd1799f6640 1 -- 172.21.15.16:0/4097884842 shutdown_connections 2024-09-12T02:01:41.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:41.028+0000 7fd1799f6640 1 -- 172.21.15.16:0/4097884842 wait complete. 2024-09-12T02:01:41.040 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":328,"stamp":"2024-09-12T02:01:40.323810+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":659240,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":717,"num_read_kb":479,"num_write":352,"num_write_kb":3058,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":1180848,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":573,"ondisk_log_size":573,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":374280,"kb_used_data":12544,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124567544,"statfs":{"total":1151940427776,"available":1151557165056,"internally_reserved":0,"allocated":12845056,"data_stored":7190707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":370193723},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4871,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":87,"num_read_kb":52,"num_write":9,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002929"},"pg_stats":[{"pgid":"3.3f","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.318917+0000","last_change":"2024-09-12T01:59:55.503882+0000","last_active":"2024-09-12T02:01:38.318917+0000","last_peered":"2024-09-12T02:01:38.318917+0000","last_clean":"2024-09-12T02:01:38.318917+0000","last_became_active":"2024-09-12T01:59:55.503513+0000","last_became_peered":"2024-09-12T01:59:55.503513+0000","last_unstale":"2024-09-12T02:01:38.318917+0000","last_undegraded":"2024-09-12T02:01:38.318917+0000","last_fullsized":"2024-09-12T02:01:38.318917+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:43:13.098527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"82'4","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.322981+0000","last_change":"2024-09-12T01:59:55.492965+0000","last_active":"2024-09-12T02:01:38.322981+0000","last_peered":"2024-09-12T02:01:38.322981+0000","last_clean":"2024-09-12T02:01:38.322981+0000","last_became_active":"2024-09-12T01:59:55.492613+0000","last_became_peered":"2024-09-12T01:59:55.492613+0000","last_unstale":"2024-09-12T02:01:38.322981+0000","last_undegraded":"2024-09-12T02:01:38.322981+0000","last_fullsized":"2024-09-12T02:01:38.322981+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:58:20.889770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.332184+0000","last_change":"2024-09-12T01:59:55.497334+0000","last_active":"2024-09-12T02:01:38.332184+0000","last_peered":"2024-09-12T02:01:38.332184+0000","last_clean":"2024-09-12T02:01:38.332184+0000","last_became_active":"2024-09-12T01:59:55.497037+0000","last_became_peered":"2024-09-12T01:59:55.497037+0000","last_unstale":"2024-09-12T02:01:38.332184+0000","last_undegraded":"2024-09-12T02:01:38.332184+0000","last_fullsized":"2024-09-12T02:01:38.332184+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:43:04.082918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.331162+0000","last_change":"2024-09-12T01:59:55.495888+0000","last_active":"2024-09-12T02:01:38.331162+0000","last_peered":"2024-09-12T02:01:38.331162+0000","last_clean":"2024-09-12T02:01:38.331162+0000","last_became_active":"2024-09-12T01:59:55.492684+0000","last_became_peered":"2024-09-12T01:59:55.492684+0000","last_unstale":"2024-09-12T02:01:38.331162+0000","last_undegraded":"2024-09-12T02:01:38.331162+0000","last_fullsized":"2024-09-12T02:01:38.331162+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:15:00.500006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.574662+0000","last_change":"2024-09-12T01:59:55.494613+0000","last_active":"2024-09-12T02:00:09.574662+0000","last_peered":"2024-09-12T02:00:09.574662+0000","last_clean":"2024-09-12T02:00:09.574662+0000","last_became_active":"2024-09-12T01:59:55.494205+0000","last_became_peered":"2024-09-12T01:59:55.494205+0000","last_unstale":"2024-09-12T02:00:09.574662+0000","last_undegraded":"2024-09-12T02:00:09.574662+0000","last_fullsized":"2024-09-12T02:00:09.574662+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:39:14.573280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.331517+0000","last_change":"2024-09-12T01:59:55.496659+0000","last_active":"2024-09-12T02:01:38.331517+0000","last_peered":"2024-09-12T02:01:38.331517+0000","last_clean":"2024-09-12T02:01:38.331517+0000","last_became_active":"2024-09-12T01:59:55.496232+0000","last_became_peered":"2024-09-12T01:59:55.496232+0000","last_unstale":"2024-09-12T02:01:38.331517+0000","last_undegraded":"2024-09-12T02:01:38.331517+0000","last_fullsized":"2024-09-12T02:01:38.331517+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:48:21.544764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"82'5","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.317177+0000","last_change":"2024-09-12T01:59:55.486789+0000","last_active":"2024-09-12T02:01:38.317177+0000","last_peered":"2024-09-12T02:01:38.317177+0000","last_clean":"2024-09-12T02:01:38.317177+0000","last_became_active":"2024-09-12T01:59:55.485921+0000","last_became_peered":"2024-09-12T01:59:55.485921+0000","last_unstale":"2024-09-12T02:01:38.317177+0000","last_undegraded":"2024-09-12T02:01:38.317177+0000","last_fullsized":"2024-09-12T02:01:38.317177+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:21:16.071121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.621086+0000","last_change":"2024-09-12T01:59:55.494525+0000","last_active":"2024-09-12T02:00:09.621086+0000","last_peered":"2024-09-12T02:00:09.621086+0000","last_clean":"2024-09-12T02:00:09.621086+0000","last_became_active":"2024-09-12T01:59:55.493998+0000","last_became_peered":"2024-09-12T01:59:55.493998+0000","last_unstale":"2024-09-12T02:00:09.621086+0000","last_undegraded":"2024-09-12T02:00:09.621086+0000","last_fullsized":"2024-09-12T02:00:09.621086+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:59:24.653660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.622633+0000","last_change":"2024-09-12T01:59:55.501466+0000","last_active":"2024-09-12T02:00:09.622633+0000","last_peered":"2024-09-12T02:00:09.622633+0000","last_clean":"2024-09-12T02:00:09.622633+0000","last_became_active":"2024-09-12T01:59:55.500800+0000","last_became_peered":"2024-09-12T01:59:55.500800+0000","last_unstale":"2024-09-12T02:00:09.622633+0000","last_undegraded":"2024-09-12T02:00:09.622633+0000","last_fullsized":"2024-09-12T02:00:09.622633+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:55:13.704791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"82'16","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.331670+0000","last_change":"2024-09-12T01:59:55.497986+0000","last_active":"2024-09-12T02:01:38.331670+0000","last_peered":"2024-09-12T02:01:38.331670+0000","last_clean":"2024-09-12T02:01:38.331670+0000","last_became_active":"2024-09-12T01:59:55.497691+0000","last_became_peered":"2024-09-12T01:59:55.497691+0000","last_unstale":"2024-09-12T02:01:38.331670+0000","last_undegraded":"2024-09-12T02:01:38.331670+0000","last_fullsized":"2024-09-12T02:01:38.331670+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:37:07.702606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.329986+0000","last_change":"2024-09-12T01:59:55.501339+0000","last_active":"2024-09-12T02:01:38.329986+0000","last_peered":"2024-09-12T02:01:38.329986+0000","last_clean":"2024-09-12T02:01:38.329986+0000","last_became_active":"2024-09-12T01:59:55.500700+0000","last_became_peered":"2024-09-12T01:59:55.500700+0000","last_unstale":"2024-09-12T02:01:38.329986+0000","last_undegraded":"2024-09-12T02:01:38.329986+0000","last_fullsized":"2024-09-12T02:01:38.329986+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:12:15.879183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.330871+0000","last_change":"2024-09-12T01:59:55.495196+0000","last_active":"2024-09-12T02:01:38.330871+0000","last_peered":"2024-09-12T02:01:38.330871+0000","last_clean":"2024-09-12T02:01:38.330871+0000","last_became_active":"2024-09-12T01:59:55.494889+0000","last_became_peered":"2024-09-12T01:59:55.494889+0000","last_unstale":"2024-09-12T02:01:38.330871+0000","last_undegraded":"2024-09-12T02:01:38.330871+0000","last_fullsized":"2024-09-12T02:01:38.330871+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:32:30.621946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"82'10","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.315946+0000","last_change":"2024-09-12T01:59:55.492565+0000","last_active":"2024-09-12T02:01:38.315946+0000","last_peered":"2024-09-12T02:01:38.315946+0000","last_clean":"2024-09-12T02:01:38.315946+0000","last_became_active":"2024-09-12T01:59:55.492168+0000","last_became_peered":"2024-09-12T01:59:55.492168+0000","last_unstale":"2024-09-12T02:01:38.315946+0000","last_undegraded":"2024-09-12T02:01:38.315946+0000","last_fullsized":"2024-09-12T02:01:38.315946+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:39:48.630202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.323015+0000","last_change":"2024-09-12T01:59:55.493152+0000","last_active":"2024-09-12T02:01:38.323015+0000","last_peered":"2024-09-12T02:01:38.323015+0000","last_clean":"2024-09-12T02:01:38.323015+0000","last_became_active":"2024-09-12T01:59:55.492794+0000","last_became_peered":"2024-09-12T01:59:55.492794+0000","last_unstale":"2024-09-12T02:01:38.323015+0000","last_undegraded":"2024-09-12T02:01:38.323015+0000","last_fullsized":"2024-09-12T02:01:38.323015+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:24:27.270502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.333374+0000","last_change":"2024-09-12T01:59:55.494651+0000","last_active":"2024-09-12T02:01:38.333374+0000","last_peered":"2024-09-12T02:01:38.333374+0000","last_clean":"2024-09-12T02:01:38.333374+0000","last_became_active":"2024-09-12T01:59:55.494355+0000","last_became_peered":"2024-09-12T01:59:55.494355+0000","last_unstale":"2024-09-12T02:01:38.333374+0000","last_undegraded":"2024-09-12T02:01:38.333374+0000","last_fullsized":"2024-09-12T02:01:38.333374+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:14:44.364158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.622950+0000","last_change":"2024-09-12T01:59:55.495049+0000","last_active":"2024-09-12T02:00:09.622950+0000","last_peered":"2024-09-12T02:00:09.622950+0000","last_clean":"2024-09-12T02:00:09.622950+0000","last_became_active":"2024-09-12T01:59:55.494688+0000","last_became_peered":"2024-09-12T01:59:55.494688+0000","last_unstale":"2024-09-12T02:00:09.622950+0000","last_undegraded":"2024-09-12T02:00:09.622950+0000","last_fullsized":"2024-09-12T02:00:09.622950+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:13:05.944577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.319055+0000","last_change":"2024-09-12T01:59:55.504146+0000","last_active":"2024-09-12T02:01:38.319055+0000","last_peered":"2024-09-12T02:01:38.319055+0000","last_clean":"2024-09-12T02:01:38.319055+0000","last_became_active":"2024-09-12T01:59:55.503646+0000","last_became_peered":"2024-09-12T01:59:55.503646+0000","last_unstale":"2024-09-12T02:01:38.319055+0000","last_undegraded":"2024-09-12T02:01:38.319055+0000","last_fullsized":"2024-09-12T02:01:38.319055+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:23:51.780795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"82'10","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.325350+0000","last_change":"2024-09-12T01:59:55.489930+0000","last_active":"2024-09-12T02:01:38.325350+0000","last_peered":"2024-09-12T02:01:38.325350+0000","last_clean":"2024-09-12T02:01:38.325350+0000","last_became_active":"2024-09-12T01:59:55.489619+0000","last_became_peered":"2024-09-12T01:59:55.489619+0000","last_unstale":"2024-09-12T02:01:38.325350+0000","last_undegraded":"2024-09-12T02:01:38.325350+0000","last_fullsized":"2024-09-12T02:01:38.325350+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:04:57.307091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.569107+0000","last_change":"2024-09-12T01:59:55.492098+0000","last_active":"2024-09-12T02:00:09.569107+0000","last_peered":"2024-09-12T02:00:09.569107+0000","last_clean":"2024-09-12T02:00:09.569107+0000","last_became_active":"2024-09-12T01:59:55.491370+0000","last_became_peered":"2024-09-12T01:59:55.491370+0000","last_unstale":"2024-09-12T02:00:09.569107+0000","last_undegraded":"2024-09-12T02:00:09.569107+0000","last_fullsized":"2024-09-12T02:00:09.569107+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:25:06.307903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.314843+0000","last_change":"2024-09-12T01:59:55.499938+0000","last_active":"2024-09-12T02:01:38.314843+0000","last_peered":"2024-09-12T02:01:38.314843+0000","last_clean":"2024-09-12T02:01:38.314843+0000","last_became_active":"2024-09-12T01:59:55.499619+0000","last_became_peered":"2024-09-12T01:59:55.499619+0000","last_unstale":"2024-09-12T02:01:38.314843+0000","last_undegraded":"2024-09-12T02:01:38.314843+0000","last_fullsized":"2024-09-12T02:01:38.314843+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:00:17.536120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"82'11","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.321068+0000","last_change":"2024-09-12T01:59:55.486106+0000","last_active":"2024-09-12T02:01:38.321068+0000","last_peered":"2024-09-12T02:01:38.321068+0000","last_clean":"2024-09-12T02:01:38.321068+0000","last_became_active":"2024-09-12T01:59:55.485325+0000","last_became_peered":"2024-09-12T01:59:55.485325+0000","last_unstale":"2024-09-12T02:01:38.321068+0000","last_undegraded":"2024-09-12T02:01:38.321068+0000","last_fullsized":"2024-09-12T02:01:38.321068+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:35:46.446224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.622237+0000","last_change":"2024-09-12T01:59:55.496438+0000","last_active":"2024-09-12T02:00:09.622237+0000","last_peered":"2024-09-12T02:00:09.622237+0000","last_clean":"2024-09-12T02:00:09.622237+0000","last_became_active":"2024-09-12T01:59:55.495885+0000","last_became_peered":"2024-09-12T01:59:55.495885+0000","last_unstale":"2024-09-12T02:00:09.622237+0000","last_undegraded":"2024-09-12T02:00:09.622237+0000","last_fullsized":"2024-09-12T02:00:09.622237+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:15:14.911335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.621139+0000","last_change":"2024-09-12T01:59:55.489350+0000","last_active":"2024-09-12T02:00:09.621139+0000","last_peered":"2024-09-12T02:00:09.621139+0000","last_clean":"2024-09-12T02:00:09.621139+0000","last_became_active":"2024-09-12T01:59:55.489028+0000","last_became_peered":"2024-09-12T01:59:55.489028+0000","last_unstale":"2024-09-12T02:00:09.621139+0000","last_undegraded":"2024-09-12T02:00:09.621139+0000","last_fullsized":"2024-09-12T02:00:09.621139+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:29:23.769004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.566645+0000","last_change":"2024-09-12T01:59:55.494532+0000","last_active":"2024-09-12T02:00:09.566645+0000","last_peered":"2024-09-12T02:00:09.566645+0000","last_clean":"2024-09-12T02:00:09.566645+0000","last_became_active":"2024-09-12T01:59:55.490982+0000","last_became_peered":"2024-09-12T01:59:55.490982+0000","last_unstale":"2024-09-12T02:00:09.566645+0000","last_undegraded":"2024-09-12T02:00:09.566645+0000","last_fullsized":"2024-09-12T02:00:09.566645+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:04:50.633730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.621490+0000","last_change":"2024-09-12T01:59:55.496057+0000","last_active":"2024-09-12T02:00:09.621490+0000","last_peered":"2024-09-12T02:00:09.621490+0000","last_clean":"2024-09-12T02:00:09.621490+0000","last_became_active":"2024-09-12T01:59:55.495710+0000","last_became_peered":"2024-09-12T01:59:55.495710+0000","last_unstale":"2024-09-12T02:00:09.621490+0000","last_undegraded":"2024-09-12T02:00:09.621490+0000","last_fullsized":"2024-09-12T02:00:09.621490+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:49:01.962537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.569100+0000","last_change":"2024-09-12T01:59:55.491594+0000","last_active":"2024-09-12T02:00:09.569100+0000","last_peered":"2024-09-12T02:00:09.569100+0000","last_clean":"2024-09-12T02:00:09.569100+0000","last_became_active":"2024-09-12T01:59:55.491233+0000","last_became_peered":"2024-09-12T01:59:55.491233+0000","last_unstale":"2024-09-12T02:00:09.569100+0000","last_undegraded":"2024-09-12T02:00:09.569100+0000","last_fullsized":"2024-09-12T02:00:09.569100+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:24:22.066777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.309442+0000","last_change":"2024-09-12T01:59:55.498331+0000","last_active":"2024-09-12T02:01:38.309442+0000","last_peered":"2024-09-12T02:01:38.309442+0000","last_clean":"2024-09-12T02:01:38.309442+0000","last_became_active":"2024-09-12T01:59:55.497983+0000","last_became_peered":"2024-09-12T01:59:55.497983+0000","last_unstale":"2024-09-12T02:01:38.309442+0000","last_undegraded":"2024-09-12T02:01:38.309442+0000","last_fullsized":"2024-09-12T02:01:38.309442+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:37:00.735231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"82'4","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.315811+0000","last_change":"2024-09-12T02:00:09.575358+0000","last_active":"2024-09-12T02:01:38.315811+0000","last_peered":"2024-09-12T02:01:38.315811+0000","last_clean":"2024-09-12T02:01:38.315811+0000","last_became_active":"2024-09-12T02:00:09.574913+0000","last_became_peered":"2024-09-12T02:00:09.574913+0000","last_unstale":"2024-09-12T02:01:38.315811+0000","last_undegraded":"2024-09-12T02:01:38.315811+0000","last_fullsized":"2024-09-12T02:01:38.315811+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:19:53.630453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.333406+0000","last_change":"2024-09-12T01:59:55.492419+0000","last_active":"2024-09-12T02:01:38.333406+0000","last_peered":"2024-09-12T02:01:38.333406+0000","last_clean":"2024-09-12T02:01:38.333406+0000","last_became_active":"2024-09-12T01:59:55.492003+0000","last_became_peered":"2024-09-12T01:59:55.492003+0000","last_unstale":"2024-09-12T02:01:38.333406+0000","last_undegraded":"2024-09-12T02:01:38.333406+0000","last_fullsized":"2024-09-12T02:01:38.333406+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:25:45.774679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.568801+0000","last_change":"2024-09-12T01:59:55.490125+0000","last_active":"2024-09-12T02:00:09.568801+0000","last_peered":"2024-09-12T02:00:09.568801+0000","last_clean":"2024-09-12T02:00:09.568801+0000","last_became_active":"2024-09-12T01:59:55.489636+0000","last_became_peered":"2024-09-12T01:59:55.489636+0000","last_unstale":"2024-09-12T02:00:09.568801+0000","last_undegraded":"2024-09-12T02:00:09.568801+0000","last_fullsized":"2024-09-12T02:00:09.568801+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:41:30.922185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.569135+0000","last_change":"2024-09-12T01:59:55.496322+0000","last_active":"2024-09-12T02:00:09.569135+0000","last_peered":"2024-09-12T02:00:09.569135+0000","last_clean":"2024-09-12T02:00:09.569135+0000","last_became_active":"2024-09-12T01:59:55.496005+0000","last_became_peered":"2024-09-12T01:59:55.496005+0000","last_unstale":"2024-09-12T02:00:09.569135+0000","last_undegraded":"2024-09-12T02:00:09.569135+0000","last_fullsized":"2024-09-12T02:00:09.569135+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:53:39.248715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.310438+0000","last_change":"2024-09-12T01:59:55.497758+0000","last_active":"2024-09-12T02:01:38.310438+0000","last_peered":"2024-09-12T02:01:38.310438+0000","last_clean":"2024-09-12T02:01:38.310438+0000","last_became_active":"2024-09-12T01:59:55.497282+0000","last_became_peered":"2024-09-12T01:59:55.497282+0000","last_unstale":"2024-09-12T02:01:38.310438+0000","last_undegraded":"2024-09-12T02:01:38.310438+0000","last_fullsized":"2024-09-12T02:01:38.310438+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:08:25.509372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.574620+0000","last_change":"2024-09-12T01:59:55.497919+0000","last_active":"2024-09-12T02:00:09.574620+0000","last_peered":"2024-09-12T02:00:09.574620+0000","last_clean":"2024-09-12T02:00:09.574620+0000","last_became_active":"2024-09-12T01:59:55.497609+0000","last_became_peered":"2024-09-12T01:59:55.497609+0000","last_unstale":"2024-09-12T02:00:09.574620+0000","last_undegraded":"2024-09-12T02:00:09.574620+0000","last_fullsized":"2024-09-12T02:00:09.574620+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:10:22.281180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"81'6","reported_seq":122,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:34.680658+0000","last_change":"2024-09-12T01:59:53.473974+0000","last_active":"2024-09-12T02:01:34.680658+0000","last_peered":"2024-09-12T02:01:34.680658+0000","last_clean":"2024-09-12T02:01:34.680658+0000","last_became_active":"2024-09-12T01:59:53.473546+0000","last_became_peered":"2024-09-12T01:59:53.473546+0000","last_unstale":"2024-09-12T02:01:34.680658+0000","last_undegraded":"2024-09-12T02:01:34.680658+0000","last_fullsized":"2024-09-12T02:01:34.680658+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:28:03.532249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":94,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.567217+0000","last_change":"2024-09-12T01:59:55.493197+0000","last_active":"2024-09-12T02:00:09.567217+0000","last_peered":"2024-09-12T02:00:09.567217+0000","last_clean":"2024-09-12T02:00:09.567217+0000","last_became_active":"2024-09-12T01:59:55.492845+0000","last_became_peered":"2024-09-12T01:59:55.492845+0000","last_unstale":"2024-09-12T02:00:09.567217+0000","last_undegraded":"2024-09-12T02:00:09.567217+0000","last_fullsized":"2024-09-12T02:00:09.567217+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:42:50.790896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"80'3","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.622865+0000","last_change":"2024-09-12T01:59:53.471214+0000","last_active":"2024-09-12T02:00:09.622865+0000","last_peered":"2024-09-12T02:00:09.622865+0000","last_clean":"2024-09-12T02:00:09.622865+0000","last_became_active":"2024-09-12T01:59:53.470858+0000","last_became_peered":"2024-09-12T01:59:53.470858+0000","last_unstale":"2024-09-12T02:00:09.622865+0000","last_undegraded":"2024-09-12T02:00:09.622865+0000","last_fullsized":"2024-09-12T02:00:09.622865+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:34:48.730392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"80'2","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.562106+0000","last_change":"2024-09-12T01:59:53.464568+0000","last_active":"2024-09-12T02:00:09.562106+0000","last_peered":"2024-09-12T02:00:09.562106+0000","last_clean":"2024-09-12T02:00:09.562106+0000","last_became_active":"2024-09-12T01:59:53.464168+0000","last_became_peered":"2024-09-12T01:59:53.464168+0000","last_unstale":"2024-09-12T02:00:09.562106+0000","last_undegraded":"2024-09-12T02:00:09.562106+0000","last_fullsized":"2024-09-12T02:00:09.562106+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:41:48.135522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.621151+0000","last_change":"2024-09-12T01:59:55.487177+0000","last_active":"2024-09-12T02:00:09.621151+0000","last_peered":"2024-09-12T02:00:09.621151+0000","last_clean":"2024-09-12T02:00:09.621151+0000","last_became_active":"2024-09-12T01:59:55.486886+0000","last_became_peered":"2024-09-12T01:59:55.486886+0000","last_unstale":"2024-09-12T02:00:09.621151+0000","last_undegraded":"2024-09-12T02:00:09.621151+0000","last_fullsized":"2024-09-12T02:00:09.621151+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:32:40.150071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"80'1","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.567129+0000","last_change":"2024-09-12T01:59:53.474114+0000","last_active":"2024-09-12T02:00:09.567129+0000","last_peered":"2024-09-12T02:00:09.567129+0000","last_clean":"2024-09-12T02:00:09.567129+0000","last_became_active":"2024-09-12T01:59:53.473755+0000","last_became_peered":"2024-09-12T01:59:53.473755+0000","last_unstale":"2024-09-12T02:00:09.567129+0000","last_undegraded":"2024-09-12T02:00:09.567129+0000","last_fullsized":"2024-09-12T02:00:09.567129+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:07:07.021137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"82'11","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.323623+0000","last_change":"2024-09-12T01:59:55.489868+0000","last_active":"2024-09-12T02:01:38.323623+0000","last_peered":"2024-09-12T02:01:38.323623+0000","last_clean":"2024-09-12T02:01:38.323623+0000","last_became_active":"2024-09-12T01:59:55.489537+0000","last_became_peered":"2024-09-12T01:59:55.489537+0000","last_unstale":"2024-09-12T02:01:38.323623+0000","last_undegraded":"2024-09-12T02:01:38.323623+0000","last_fullsized":"2024-09-12T02:01:38.323623+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:18:49.417689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"82'4","reported_seq":120,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.276658+0000","last_change":"2024-09-12T01:59:53.471402+0000","last_active":"2024-09-12T02:01:38.276658+0000","last_peered":"2024-09-12T02:01:38.276658+0000","last_clean":"2024-09-12T02:01:38.276658+0000","last_became_active":"2024-09-12T01:59:53.470940+0000","last_became_peered":"2024-09-12T01:59:53.470940+0000","last_unstale":"2024-09-12T02:01:38.276658+0000","last_undegraded":"2024-09-12T02:01:38.276658+0000","last_fullsized":"2024-09-12T02:01:38.276658+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:41:16.853592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":94,"num_read_kb":94,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.623260+0000","last_change":"2024-09-12T01:59:55.499047+0000","last_active":"2024-09-12T02:00:09.623260+0000","last_peered":"2024-09-12T02:00:09.623260+0000","last_clean":"2024-09-12T02:00:09.623260+0000","last_became_active":"2024-09-12T01:59:55.498071+0000","last_became_peered":"2024-09-12T01:59:55.498071+0000","last_unstale":"2024-09-12T02:00:09.623260+0000","last_undegraded":"2024-09-12T02:00:09.623260+0000","last_fullsized":"2024-09-12T02:00:09.623260+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:36:40.106041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"80'2","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.622966+0000","last_change":"2024-09-12T01:59:53.473508+0000","last_active":"2024-09-12T02:00:09.622966+0000","last_peered":"2024-09-12T02:00:09.622966+0000","last_clean":"2024-09-12T02:00:09.622966+0000","last_became_active":"2024-09-12T01:59:53.473063+0000","last_became_peered":"2024-09-12T01:59:53.473063+0000","last_unstale":"2024-09-12T02:00:09.622966+0000","last_undegraded":"2024-09-12T02:00:09.622966+0000","last_fullsized":"2024-09-12T02:00:09.622966+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:55:32.161769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.333032+0000","last_change":"2024-09-12T01:59:55.491721+0000","last_active":"2024-09-12T02:01:38.333032+0000","last_peered":"2024-09-12T02:01:38.333032+0000","last_clean":"2024-09-12T02:01:38.333032+0000","last_became_active":"2024-09-12T01:59:55.491364+0000","last_became_peered":"2024-09-12T01:59:55.491364+0000","last_unstale":"2024-09-12T02:01:38.333032+0000","last_undegraded":"2024-09-12T02:01:38.333032+0000","last_fullsized":"2024-09-12T02:01:38.333032+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:21:20.795420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"80'4","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.567315+0000","last_change":"2024-09-12T01:59:53.921789+0000","last_active":"2024-09-12T02:00:09.567315+0000","last_peered":"2024-09-12T02:00:09.567315+0000","last_clean":"2024-09-12T02:00:09.567315+0000","last_became_active":"2024-09-12T01:59:53.921383+0000","last_became_peered":"2024-09-12T01:59:53.921383+0000","last_unstale":"2024-09-12T02:00:09.567315+0000","last_undegraded":"2024-09-12T02:00:09.567315+0000","last_fullsized":"2024-09-12T02:00:09.567315+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:21:58.792794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.622517+0000","last_change":"2024-09-12T01:59:55.503008+0000","last_active":"2024-09-12T02:00:09.622517+0000","last_peered":"2024-09-12T02:00:09.622517+0000","last_clean":"2024-09-12T02:00:09.622517+0000","last_became_active":"2024-09-12T01:59:55.502538+0000","last_became_peered":"2024-09-12T01:59:55.502538+0000","last_unstale":"2024-09-12T02:00:09.622517+0000","last_undegraded":"2024-09-12T02:00:09.622517+0000","last_fullsized":"2024-09-12T02:00:09.622517+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:51:36.735985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"82'32","reported_seq":253,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.686996+0000","last_change":"2024-09-12T01:59:53.470688+0000","last_active":"2024-09-12T02:01:38.686996+0000","last_peered":"2024-09-12T02:01:38.686996+0000","last_clean":"2024-09-12T02:01:38.686996+0000","last_became_active":"2024-09-12T01:59:53.470231+0000","last_became_peered":"2024-09-12T01:59:53.470231+0000","last_unstale":"2024-09-12T02:01:38.686996+0000","last_undegraded":"2024-09-12T02:01:38.686996+0000","last_fullsized":"2024-09-12T02:01:38.686996+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:53:54.439893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64635,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":199,"num_read_kb":163,"num_write":33,"num_write_kb":76,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.575173+0000","last_change":"2024-09-12T01:59:55.495343+0000","last_active":"2024-09-12T02:00:09.575173+0000","last_peered":"2024-09-12T02:00:09.575173+0000","last_clean":"2024-09-12T02:00:09.575173+0000","last_became_active":"2024-09-12T01:59:55.495006+0000","last_became_peered":"2024-09-12T01:59:55.495006+0000","last_unstale":"2024-09-12T02:00:09.575173+0000","last_undegraded":"2024-09-12T02:00:09.575173+0000","last_fullsized":"2024-09-12T02:00:09.575173+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:32:20.932551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'6","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.569082+0000","last_change":"2024-09-12T01:59:53.468625+0000","last_active":"2024-09-12T02:00:09.569082+0000","last_peered":"2024-09-12T02:00:09.569082+0000","last_clean":"2024-09-12T02:00:09.569082+0000","last_became_active":"2024-09-12T01:59:53.468015+0000","last_became_peered":"2024-09-12T01:59:53.468015+0000","last_unstale":"2024-09-12T02:00:09.569082+0000","last_undegraded":"2024-09-12T02:00:09.569082+0000","last_fullsized":"2024-09-12T02:00:09.569082+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:49:34.842239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.319752+0000","last_change":"2024-09-12T01:59:55.487962+0000","last_active":"2024-09-12T02:01:38.319752+0000","last_peered":"2024-09-12T02:01:38.319752+0000","last_clean":"2024-09-12T02:01:38.319752+0000","last_became_active":"2024-09-12T01:59:55.487619+0000","last_became_peered":"2024-09-12T01:59:55.487619+0000","last_unstale":"2024-09-12T02:01:38.319752+0000","last_undegraded":"2024-09-12T02:01:38.319752+0000","last_fullsized":"2024-09-12T02:01:38.319752+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:32:16.882373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"80'3","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.567124+0000","last_change":"2024-09-12T01:59:53.474418+0000","last_active":"2024-09-12T02:00:09.567124+0000","last_peered":"2024-09-12T02:00:09.567124+0000","last_clean":"2024-09-12T02:00:09.567124+0000","last_became_active":"2024-09-12T01:59:53.474059+0000","last_became_peered":"2024-09-12T01:59:53.474059+0000","last_unstale":"2024-09-12T02:00:09.567124+0000","last_undegraded":"2024-09-12T02:00:09.567124+0000","last_fullsized":"2024-09-12T02:00:09.567124+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:10:54.963435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"82'15","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.333121+0000","last_change":"2024-09-12T01:59:55.493394+0000","last_active":"2024-09-12T02:01:38.333121+0000","last_peered":"2024-09-12T02:01:38.333121+0000","last_clean":"2024-09-12T02:01:38.333121+0000","last_became_active":"2024-09-12T01:59:55.491669+0000","last_became_peered":"2024-09-12T01:59:55.491669+0000","last_unstale":"2024-09-12T02:01:38.333121+0000","last_undegraded":"2024-09-12T02:01:38.333121+0000","last_fullsized":"2024-09-12T02:01:38.333121+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:14:40.871154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"80'3","reported_seq":121,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:37.685132+0000","last_change":"2024-09-12T01:59:53.471623+0000","last_active":"2024-09-12T02:01:37.685132+0000","last_peered":"2024-09-12T02:01:37.685132+0000","last_clean":"2024-09-12T02:01:37.685132+0000","last_became_active":"2024-09-12T01:59:53.471294+0000","last_became_peered":"2024-09-12T01:59:53.471294+0000","last_unstale":"2024-09-12T02:01:37.685132+0000","last_undegraded":"2024-09-12T02:01:37.685132+0000","last_fullsized":"2024-09-12T02:01:37.685132+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:51:54.130883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":97,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.322144+0000","last_change":"2024-09-12T01:59:55.493082+0000","last_active":"2024-09-12T02:01:38.322144+0000","last_peered":"2024-09-12T02:01:38.322144+0000","last_clean":"2024-09-12T02:01:38.322144+0000","last_became_active":"2024-09-12T01:59:55.492778+0000","last_became_peered":"2024-09-12T01:59:55.492778+0000","last_unstale":"2024-09-12T02:01:38.322144+0000","last_undegraded":"2024-09-12T02:01:38.322144+0000","last_fullsized":"2024-09-12T02:01:38.322144+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:59:56.466377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'15","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.588050+0000","last_change":"2024-09-12T02:00:09.588050+0000","last_active":"2024-09-12T02:00:09.588050+0000","last_peered":"2024-09-12T02:00:09.588050+0000","last_clean":"2024-09-12T02:00:09.588050+0000","last_became_active":"2024-09-12T02:00:09.575478+0000","last_became_peered":"2024-09-12T02:00:09.575478+0000","last_unstale":"2024-09-12T02:00:09.588050+0000","last_undegraded":"2024-09-12T02:00:09.588050+0000","last_fullsized":"2024-09-12T02:00:09.588050+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:38:45.484167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.566976+0000","last_change":"2024-09-12T01:59:55.497601+0000","last_active":"2024-09-12T02:00:09.566976+0000","last_peered":"2024-09-12T02:00:09.566976+0000","last_clean":"2024-09-12T02:00:09.566976+0000","last_became_active":"2024-09-12T01:59:55.497202+0000","last_became_peered":"2024-09-12T01:59:55.497202+0000","last_unstale":"2024-09-12T02:00:09.566976+0000","last_undegraded":"2024-09-12T02:00:09.566976+0000","last_fullsized":"2024-09-12T02:00:09.566976+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:22:22.850736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"80'2","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.575359+0000","last_change":"2024-09-12T01:59:53.467575+0000","last_active":"2024-09-12T02:00:09.575359+0000","last_peered":"2024-09-12T02:00:09.575359+0000","last_clean":"2024-09-12T02:00:09.575359+0000","last_became_active":"2024-09-12T01:59:53.467146+0000","last_became_peered":"2024-09-12T01:59:53.467146+0000","last_unstale":"2024-09-12T02:00:09.575359+0000","last_undegraded":"2024-09-12T02:00:09.575359+0000","last_fullsized":"2024-09-12T02:00:09.575359+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:07:25.962496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"82'5","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.314382+0000","last_change":"2024-09-12T01:59:55.498297+0000","last_active":"2024-09-12T02:01:38.314382+0000","last_peered":"2024-09-12T02:01:38.314382+0000","last_clean":"2024-09-12T02:01:38.314382+0000","last_became_active":"2024-09-12T01:59:55.497367+0000","last_became_peered":"2024-09-12T01:59:55.497367+0000","last_unstale":"2024-09-12T02:01:38.314382+0000","last_undegraded":"2024-09-12T02:01:38.314382+0000","last_fullsized":"2024-09-12T02:01:38.314382+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:58:32.717710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"80'2","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.562292+0000","last_change":"2024-09-12T01:59:53.466664+0000","last_active":"2024-09-12T02:00:09.562292+0000","last_peered":"2024-09-12T02:00:09.562292+0000","last_clean":"2024-09-12T02:00:09.562292+0000","last_became_active":"2024-09-12T01:59:53.466318+0000","last_became_peered":"2024-09-12T01:59:53.466318+0000","last_unstale":"2024-09-12T02:00:09.562292+0000","last_undegraded":"2024-09-12T02:00:09.562292+0000","last_fullsized":"2024-09-12T02:00:09.562292+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:12:13.541420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.313289+0000","last_change":"2024-09-12T01:59:55.490420+0000","last_active":"2024-09-12T02:01:38.313289+0000","last_peered":"2024-09-12T02:01:38.313289+0000","last_clean":"2024-09-12T02:01:38.313289+0000","last_became_active":"2024-09-12T01:59:55.489757+0000","last_became_peered":"2024-09-12T01:59:55.489757+0000","last_unstale":"2024-09-12T02:01:38.313289+0000","last_undegraded":"2024-09-12T02:01:38.313289+0000","last_fullsized":"2024-09-12T02:01:38.313289+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:34:43.117142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'1","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.567185+0000","last_change":"2024-09-12T01:59:53.466399+0000","last_active":"2024-09-12T02:00:09.567185+0000","last_peered":"2024-09-12T02:00:09.567185+0000","last_clean":"2024-09-12T02:00:09.567185+0000","last_became_active":"2024-09-12T01:59:53.465982+0000","last_became_peered":"2024-09-12T01:59:53.465982+0000","last_unstale":"2024-09-12T02:00:09.567185+0000","last_undegraded":"2024-09-12T02:00:09.567185+0000","last_fullsized":"2024-09-12T02:00:09.567185+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:18:02.015040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.326034+0000","last_change":"2024-09-12T01:59:55.491527+0000","last_active":"2024-09-12T02:01:38.326034+0000","last_peered":"2024-09-12T02:01:38.326034+0000","last_clean":"2024-09-12T02:01:38.326034+0000","last_became_active":"2024-09-12T01:59:55.491165+0000","last_became_peered":"2024-09-12T01:59:55.491165+0000","last_unstale":"2024-09-12T02:01:38.326034+0000","last_undegraded":"2024-09-12T02:01:38.326034+0000","last_fullsized":"2024-09-12T02:01:38.326034+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:57:05.506384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"80'3","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.569187+0000","last_change":"2024-09-12T01:59:54.075752+0000","last_active":"2024-09-12T02:00:09.569187+0000","last_peered":"2024-09-12T02:00:09.569187+0000","last_clean":"2024-09-12T02:00:09.569187+0000","last_became_active":"2024-09-12T01:59:54.075387+0000","last_became_peered":"2024-09-12T01:59:54.075387+0000","last_unstale":"2024-09-12T02:00:09.569187+0000","last_undegraded":"2024-09-12T02:00:09.569187+0000","last_fullsized":"2024-09-12T02:00:09.569187+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:21:15.980591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.330203+0000","last_change":"2024-09-12T01:59:55.500935+0000","last_active":"2024-09-12T02:01:38.330203+0000","last_peered":"2024-09-12T02:01:38.330203+0000","last_clean":"2024-09-12T02:01:38.330203+0000","last_became_active":"2024-09-12T01:59:55.500206+0000","last_became_peered":"2024-09-12T01:59:55.500206+0000","last_unstale":"2024-09-12T02:01:38.330203+0000","last_undegraded":"2024-09-12T02:01:38.330203+0000","last_fullsized":"2024-09-12T02:01:38.330203+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:19:59.937701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":334,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.623254+0000","last_change":"2024-09-12T01:58:47.553853+0000","last_active":"2024-09-12T02:00:09.623254+0000","last_peered":"2024-09-12T02:00:09.623254+0000","last_clean":"2024-09-12T02:00:09.623254+0000","last_became_active":"2024-09-12T01:58:47.553357+0000","last_became_peered":"2024-09-12T01:58:47.553357+0000","last_unstale":"2024-09-12T02:00:09.623254+0000","last_undegraded":"2024-09-12T02:00:09.623254+0000","last_fullsized":"2024-09-12T02:00:09.623254+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:55:26.917733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:55:26.917733+0000","last_clean_scrub_stamp":"2024-09-12T01:55:26.917733+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:58:26.081766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"80'3","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.563615+0000","last_change":"2024-09-12T01:59:53.467907+0000","last_active":"2024-09-12T02:00:09.563615+0000","last_peered":"2024-09-12T02:00:09.563615+0000","last_clean":"2024-09-12T02:00:09.563615+0000","last_became_active":"2024-09-12T01:59:53.467164+0000","last_became_peered":"2024-09-12T01:59:53.467164+0000","last_unstale":"2024-09-12T02:00:09.563615+0000","last_undegraded":"2024-09-12T02:00:09.563615+0000","last_fullsized":"2024-09-12T02:00:09.563615+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:09:35.242708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.621290+0000","last_change":"2024-09-12T01:59:55.495109+0000","last_active":"2024-09-12T02:00:09.621290+0000","last_peered":"2024-09-12T02:00:09.621290+0000","last_clean":"2024-09-12T02:00:09.621290+0000","last_became_active":"2024-09-12T01:59:55.494791+0000","last_became_peered":"2024-09-12T01:59:55.494791+0000","last_unstale":"2024-09-12T02:00:09.621290+0000","last_undegraded":"2024-09-12T02:00:09.621290+0000","last_fullsized":"2024-09-12T02:00:09.621290+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:22:43.882050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"80'3","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.623115+0000","last_change":"2024-09-12T01:59:53.473286+0000","last_active":"2024-09-12T02:00:09.623115+0000","last_peered":"2024-09-12T02:00:09.623115+0000","last_clean":"2024-09-12T02:00:09.623115+0000","last_became_active":"2024-09-12T01:59:53.472976+0000","last_became_peered":"2024-09-12T01:59:53.472976+0000","last_unstale":"2024-09-12T02:00:09.623115+0000","last_undegraded":"2024-09-12T02:00:09.623115+0000","last_fullsized":"2024-09-12T02:00:09.623115+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:32:39.604173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"82'5","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.313749+0000","last_change":"2024-09-12T02:00:09.571159+0000","last_active":"2024-09-12T02:01:38.313749+0000","last_peered":"2024-09-12T02:01:38.313749+0000","last_clean":"2024-09-12T02:01:38.313749+0000","last_became_active":"2024-09-12T02:00:09.570672+0000","last_became_peered":"2024-09-12T02:00:09.570672+0000","last_unstale":"2024-09-12T02:01:38.313749+0000","last_undegraded":"2024-09-12T02:01:38.313749+0000","last_fullsized":"2024-09-12T02:01:38.313749+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:04:10.808942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"80'4","reported_seq":121,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.685685+0000","last_change":"2024-09-12T01:59:53.921856+0000","last_active":"2024-09-12T02:01:38.685685+0000","last_peered":"2024-09-12T02:01:38.685685+0000","last_clean":"2024-09-12T02:01:38.685685+0000","last_became_active":"2024-09-12T01:59:53.920672+0000","last_became_peered":"2024-09-12T01:59:53.920672+0000","last_unstale":"2024-09-12T02:01:38.685685+0000","last_undegraded":"2024-09-12T02:01:38.685685+0000","last_fullsized":"2024-09-12T02:01:38.685685+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:42:33.876493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":95,"num_read_kb":95,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"82'8","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.309498+0000","last_change":"2024-09-12T02:00:09.571185+0000","last_active":"2024-09-12T02:01:38.309498+0000","last_peered":"2024-09-12T02:01:38.309498+0000","last_clean":"2024-09-12T02:01:38.309498+0000","last_became_active":"2024-09-12T02:00:09.570769+0000","last_became_peered":"2024-09-12T02:00:09.570769+0000","last_unstale":"2024-09-12T02:01:38.309498+0000","last_undegraded":"2024-09-12T02:01:38.309498+0000","last_fullsized":"2024-09-12T02:01:38.309498+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:34:17.230435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.2","version":"80'1","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.622691+0000","last_change":"2024-09-12T01:59:53.472072+0000","last_active":"2024-09-12T02:00:09.622691+0000","last_peered":"2024-09-12T02:00:09.622691+0000","last_clean":"2024-09-12T02:00:09.622691+0000","last_became_active":"2024-09-12T01:59:53.471088+0000","last_became_peered":"2024-09-12T01:59:53.471088+0000","last_unstale":"2024-09-12T02:00:09.622691+0000","last_undegraded":"2024-09-12T02:00:09.622691+0000","last_fullsized":"2024-09-12T02:00:09.622691+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:02:13.879131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.330778+0000","last_change":"2024-09-12T01:59:55.502843+0000","last_active":"2024-09-12T02:01:38.330778+0000","last_peered":"2024-09-12T02:01:38.330778+0000","last_clean":"2024-09-12T02:01:38.330778+0000","last_became_active":"2024-09-12T01:59:55.499296+0000","last_became_peered":"2024-09-12T01:59:55.499296+0000","last_unstale":"2024-09-12T02:01:38.330778+0000","last_undegraded":"2024-09-12T02:01:38.330778+0000","last_fullsized":"2024-09-12T02:01:38.330778+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:04:21.787370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.575061+0000","last_change":"2024-09-12T02:00:09.575061+0000","last_active":"2024-09-12T02:00:09.575061+0000","last_peered":"2024-09-12T02:00:09.575061+0000","last_clean":"2024-09-12T02:00:09.575061+0000","last_became_active":"2024-09-12T02:00:09.574653+0000","last_became_peered":"2024-09-12T02:00:09.574653+0000","last_unstale":"2024-09-12T02:00:09.575061+0000","last_undegraded":"2024-09-12T02:00:09.575061+0000","last_fullsized":"2024-09-12T02:00:09.575061+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:32:27.931211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"82'5","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.320660+0000","last_change":"2024-09-12T01:59:55.494162+0000","last_active":"2024-09-12T02:01:38.320660+0000","last_peered":"2024-09-12T02:01:38.320660+0000","last_clean":"2024-09-12T02:01:38.320660+0000","last_became_active":"2024-09-12T01:59:55.493802+0000","last_became_peered":"2024-09-12T01:59:55.493802+0000","last_unstale":"2024-09-12T02:01:38.320660+0000","last_undegraded":"2024-09-12T02:01:38.320660+0000","last_fullsized":"2024-09-12T02:01:38.320660+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:22:16.302683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"80'1","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.575278+0000","last_change":"2024-09-12T01:59:53.921669+0000","last_active":"2024-09-12T02:00:09.575278+0000","last_peered":"2024-09-12T02:00:09.575278+0000","last_clean":"2024-09-12T02:00:09.575278+0000","last_became_active":"2024-09-12T01:59:53.921328+0000","last_became_peered":"2024-09-12T01:59:53.921328+0000","last_unstale":"2024-09-12T02:00:09.575278+0000","last_undegraded":"2024-09-12T02:00:09.575278+0000","last_fullsized":"2024-09-12T02:00:09.575278+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:46:33.313726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.311067+0000","last_change":"2024-09-12T01:59:55.488986+0000","last_active":"2024-09-12T02:01:38.311067+0000","last_peered":"2024-09-12T02:01:38.311067+0000","last_clean":"2024-09-12T02:01:38.311067+0000","last_became_active":"2024-09-12T01:59:55.488632+0000","last_became_peered":"2024-09-12T01:59:55.488632+0000","last_unstale":"2024-09-12T02:01:38.311067+0000","last_undegraded":"2024-09-12T02:01:38.311067+0000","last_fullsized":"2024-09-12T02:01:38.311067+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:20:57.596976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"82'11","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:21.501527+0000","last_change":"2024-09-12T01:59:53.467648+0000","last_active":"2024-09-12T02:00:21.501527+0000","last_peered":"2024-09-12T02:00:21.501527+0000","last_clean":"2024-09-12T02:00:21.501527+0000","last_became_active":"2024-09-12T01:59:53.467255+0000","last_became_peered":"2024-09-12T01:59:53.467255+0000","last_unstale":"2024-09-12T02:00:21.501527+0000","last_undegraded":"2024-09-12T02:00:21.501527+0000","last_fullsized":"2024-09-12T02:00:21.501527+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:37:09.897388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.621489+0000","last_change":"2024-09-12T01:59:55.487603+0000","last_active":"2024-09-12T02:00:09.621489+0000","last_peered":"2024-09-12T02:00:09.621489+0000","last_clean":"2024-09-12T02:00:09.621489+0000","last_became_active":"2024-09-12T01:59:55.487334+0000","last_became_peered":"2024-09-12T01:59:55.487334+0000","last_unstale":"2024-09-12T02:00:09.621489+0000","last_undegraded":"2024-09-12T02:00:09.621489+0000","last_fullsized":"2024-09-12T02:00:09.621489+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:23:16.669515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'4","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.563870+0000","last_change":"2024-09-12T01:59:54.076156+0000","last_active":"2024-09-12T02:00:09.563870+0000","last_peered":"2024-09-12T02:00:09.563870+0000","last_clean":"2024-09-12T02:00:09.563870+0000","last_became_active":"2024-09-12T01:59:54.075794+0000","last_became_peered":"2024-09-12T01:59:54.075794+0000","last_unstale":"2024-09-12T02:00:09.563870+0000","last_undegraded":"2024-09-12T02:00:09.563870+0000","last_fullsized":"2024-09-12T02:00:09.563870+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:06:17.431696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.333603+0000","last_change":"2024-09-12T01:59:55.494604+0000","last_active":"2024-09-12T02:01:38.333603+0000","last_peered":"2024-09-12T02:01:38.333603+0000","last_clean":"2024-09-12T02:01:38.333603+0000","last_became_active":"2024-09-12T01:59:55.494300+0000","last_became_peered":"2024-09-12T01:59:55.494300+0000","last_unstale":"2024-09-12T02:01:38.333603+0000","last_undegraded":"2024-09-12T02:01:38.333603+0000","last_fullsized":"2024-09-12T02:01:38.333603+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:18:56.359299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.622205+0000","last_change":"2024-09-12T01:59:53.921832+0000","last_active":"2024-09-12T02:00:09.622205+0000","last_peered":"2024-09-12T02:00:09.622205+0000","last_clean":"2024-09-12T02:00:09.622205+0000","last_became_active":"2024-09-12T01:59:53.921379+0000","last_became_peered":"2024-09-12T01:59:53.921379+0000","last_unstale":"2024-09-12T02:00:09.622205+0000","last_undegraded":"2024-09-12T02:00:09.622205+0000","last_fullsized":"2024-09-12T02:00:09.622205+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:26:02.277149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.574438+0000","last_change":"2024-09-12T01:59:55.493198+0000","last_active":"2024-09-12T02:00:09.574438+0000","last_peered":"2024-09-12T02:00:09.574438+0000","last_clean":"2024-09-12T02:00:09.574438+0000","last_became_active":"2024-09-12T01:59:55.492834+0000","last_became_peered":"2024-09-12T01:59:55.492834+0000","last_unstale":"2024-09-12T02:00:09.574438+0000","last_undegraded":"2024-09-12T02:00:09.574438+0000","last_fullsized":"2024-09-12T02:00:09.574438+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:58:43.872921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"80'3","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.621258+0000","last_change":"2024-09-12T01:59:53.466528+0000","last_active":"2024-09-12T02:00:09.621258+0000","last_peered":"2024-09-12T02:00:09.621258+0000","last_clean":"2024-09-12T02:00:09.621258+0000","last_became_active":"2024-09-12T01:59:53.466182+0000","last_became_peered":"2024-09-12T01:59:53.466182+0000","last_unstale":"2024-09-12T02:00:09.621258+0000","last_undegraded":"2024-09-12T02:00:09.621258+0000","last_fullsized":"2024-09-12T02:00:09.621258+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:00:26.083539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.316786+0000","last_change":"2024-09-12T01:59:55.494511+0000","last_active":"2024-09-12T02:01:38.316786+0000","last_peered":"2024-09-12T02:01:38.316786+0000","last_clean":"2024-09-12T02:01:38.316786+0000","last_became_active":"2024-09-12T01:59:55.493607+0000","last_became_peered":"2024-09-12T01:59:55.493607+0000","last_unstale":"2024-09-12T02:01:38.316786+0000","last_undegraded":"2024-09-12T02:01:38.316786+0000","last_fullsized":"2024-09-12T02:01:38.316786+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:55:27.434447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"80'1","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.569211+0000","last_change":"2024-09-12T01:59:53.467385+0000","last_active":"2024-09-12T02:00:09.569211+0000","last_peered":"2024-09-12T02:00:09.569211+0000","last_clean":"2024-09-12T02:00:09.569211+0000","last_became_active":"2024-09-12T01:59:53.466997+0000","last_became_peered":"2024-09-12T01:59:53.466997+0000","last_unstale":"2024-09-12T02:00:09.569211+0000","last_undegraded":"2024-09-12T02:00:09.569211+0000","last_fullsized":"2024-09-12T02:00:09.569211+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:54:47.154036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.621293+0000","last_change":"2024-09-12T01:59:55.490757+0000","last_active":"2024-09-12T02:00:09.621293+0000","last_peered":"2024-09-12T02:00:09.621293+0000","last_clean":"2024-09-12T02:00:09.621293+0000","last_became_active":"2024-09-12T01:59:55.490451+0000","last_became_peered":"2024-09-12T01:59:55.490451+0000","last_unstale":"2024-09-12T02:00:09.621293+0000","last_undegraded":"2024-09-12T02:00:09.621293+0000","last_fullsized":"2024-09-12T02:00:09.621293+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:21:26.139049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.574833+0000","last_change":"2024-09-12T01:59:53.472898+0000","last_active":"2024-09-12T02:00:09.574833+0000","last_peered":"2024-09-12T02:00:09.574833+0000","last_clean":"2024-09-12T02:00:09.574833+0000","last_became_active":"2024-09-12T01:59:53.472561+0000","last_became_peered":"2024-09-12T01:59:53.472561+0000","last_unstale":"2024-09-12T02:00:09.574833+0000","last_undegraded":"2024-09-12T02:00:09.574833+0000","last_fullsized":"2024-09-12T02:00:09.574833+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:55:59.933804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"82'5","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.328406+0000","last_change":"2024-09-12T01:59:55.497731+0000","last_active":"2024-09-12T02:01:38.328406+0000","last_peered":"2024-09-12T02:01:38.328406+0000","last_clean":"2024-09-12T02:01:38.328406+0000","last_became_active":"2024-09-12T01:59:55.497356+0000","last_became_peered":"2024-09-12T01:59:55.497356+0000","last_unstale":"2024-09-12T02:01:38.328406+0000","last_undegraded":"2024-09-12T02:01:38.328406+0000","last_fullsized":"2024-09-12T02:01:38.328406+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:57:18.795865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.624453+0000","last_change":"2024-09-12T02:00:09.624453+0000","last_active":"2024-09-12T02:00:09.624453+0000","last_peered":"2024-09-12T02:00:09.624453+0000","last_clean":"2024-09-12T02:00:09.624453+0000","last_became_active":"2024-09-12T02:00:09.624049+0000","last_became_peered":"2024-09-12T02:00:09.624049+0000","last_unstale":"2024-09-12T02:00:09.624453+0000","last_undegraded":"2024-09-12T02:00:09.624453+0000","last_fullsized":"2024-09-12T02:00:09.624453+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:17:40.417275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"80'2","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.575060+0000","last_change":"2024-09-12T01:59:53.470194+0000","last_active":"2024-09-12T02:00:09.575060+0000","last_peered":"2024-09-12T02:00:09.575060+0000","last_clean":"2024-09-12T02:00:09.575060+0000","last_became_active":"2024-09-12T01:59:53.469772+0000","last_became_peered":"2024-09-12T01:59:53.469772+0000","last_unstale":"2024-09-12T02:00:09.575060+0000","last_undegraded":"2024-09-12T02:00:09.575060+0000","last_fullsized":"2024-09-12T02:00:09.575060+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:52:57.709596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.622423+0000","last_change":"2024-09-12T01:59:55.497693+0000","last_active":"2024-09-12T02:00:09.622423+0000","last_peered":"2024-09-12T02:00:09.622423+0000","last_clean":"2024-09-12T02:00:09.622423+0000","last_became_active":"2024-09-12T01:59:55.497380+0000","last_became_peered":"2024-09-12T01:59:55.497380+0000","last_unstale":"2024-09-12T02:00:09.622423+0000","last_undegraded":"2024-09-12T02:00:09.622423+0000","last_fullsized":"2024-09-12T02:00:09.622423+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:42:27.943530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"80'4","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.575274+0000","last_change":"2024-09-12T01:59:53.473044+0000","last_active":"2024-09-12T02:00:09.575274+0000","last_peered":"2024-09-12T02:00:09.575274+0000","last_clean":"2024-09-12T02:00:09.575274+0000","last_became_active":"2024-09-12T01:59:53.472569+0000","last_became_peered":"2024-09-12T01:59:53.472569+0000","last_unstale":"2024-09-12T02:00:09.575274+0000","last_undegraded":"2024-09-12T02:00:09.575274+0000","last_fullsized":"2024-09-12T02:00:09.575274+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:41:29.716836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.328172+0000","last_change":"2024-09-12T01:59:55.497377+0000","last_active":"2024-09-12T02:01:38.328172+0000","last_peered":"2024-09-12T02:01:38.328172+0000","last_clean":"2024-09-12T02:01:38.328172+0000","last_became_active":"2024-09-12T01:59:55.497078+0000","last_became_peered":"2024-09-12T01:59:55.497078+0000","last_unstale":"2024-09-12T02:01:38.328172+0000","last_undegraded":"2024-09-12T02:01:38.328172+0000","last_fullsized":"2024-09-12T02:01:38.328172+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:59:57.362350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.621555+0000","last_change":"2024-09-12T01:59:53.920868+0000","last_active":"2024-09-12T02:00:09.621555+0000","last_peered":"2024-09-12T02:00:09.621555+0000","last_clean":"2024-09-12T02:00:09.621555+0000","last_became_active":"2024-09-12T01:59:53.920481+0000","last_became_peered":"2024-09-12T01:59:53.920481+0000","last_unstale":"2024-09-12T02:00:09.621555+0000","last_undegraded":"2024-09-12T02:00:09.621555+0000","last_fullsized":"2024-09-12T02:00:09.621555+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:49:27.085388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.621438+0000","last_change":"2024-09-12T01:59:55.497703+0000","last_active":"2024-09-12T02:00:09.621438+0000","last_peered":"2024-09-12T02:00:09.621438+0000","last_clean":"2024-09-12T02:00:09.621438+0000","last_became_active":"2024-09-12T01:59:55.497397+0000","last_became_peered":"2024-09-12T01:59:55.497397+0000","last_unstale":"2024-09-12T02:00:09.621438+0000","last_undegraded":"2024-09-12T02:00:09.621438+0000","last_fullsized":"2024-09-12T02:00:09.621438+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:54:02.803203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"80'9","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.565145+0000","last_change":"2024-09-12T01:59:53.469978+0000","last_active":"2024-09-12T02:00:09.565145+0000","last_peered":"2024-09-12T02:00:09.565145+0000","last_clean":"2024-09-12T02:00:09.565145+0000","last_became_active":"2024-09-12T01:59:53.469648+0000","last_became_peered":"2024-09-12T01:59:53.469648+0000","last_unstale":"2024-09-12T02:00:09.565145+0000","last_undegraded":"2024-09-12T02:00:09.565145+0000","last_fullsized":"2024-09-12T02:00:09.565145+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:08:37.720452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":293,"ondisk_log_size":293,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":68638,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":591,"num_read_kb":370,"num_write":169,"num_write_kb":118,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":421888,"data_stored":196621,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":143,"ondisk_log_size":143,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3563520,"data_stored":3542208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":74,"seq":317827579938,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32068,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713084,"statfs":{"total":95995035648,"available":95962198016,"internally_reserved":0,"allocated":897024,"data_stored":435916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":67,"seq":287762808871,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32736,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712416,"statfs":{"total":95995035648,"available":95961513984,"internally_reserved":0,"allocated":1589248,"data_stored":1091844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":9,"up_from":62,"seq":266287972396,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":905216,"data_stored":436040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201328,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":897024,"data_stored":435848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430262,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32648,"kb_used_data":1464,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712504,"statfs":{"total":95995035648,"available":95961604096,"internally_reserved":0,"allocated":1499136,"data_stored":1028033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593787,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36216,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708936,"statfs":{"total":95995035648,"available":95957950464,"internally_reserved":0,"allocated":892928,"data_stored":435429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822719,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32660,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712492,"statfs":{"total":95995035648,"available":95961591808,"internally_reserved":0,"allocated":1511424,"data_stored":1027576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986244,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":897024,"data_stored":435866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215176,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":909312,"data_stored":435740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":17,"seq":73014444110,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36300,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708852,"statfs":{"total":95995035648,"available":95957864448,"internally_reserved":0,"allocated":978944,"data_stored":496729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574931,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32116,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713036,"statfs":{"total":95995035648,"available":95962148864,"internally_reserved":0,"allocated":954368,"data_stored":494986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738457,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":913408,"data_stored":436700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":59764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":61815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":66213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T02:01:41.043 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-09-12T02:01:41.044 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T02:01:41.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:41.182+0000 7ff4505bd640 1 -- 172.21.15.16:0/2807280379 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4481016b0 msgr2=0x7ff448108f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:41.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:41.182+0000 7ff4505bd640 1 --2- 172.21.15.16:0/2807280379 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4481016b0 0x7ff448108f50 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7ff440009ca0 tx=0x7ff44002a8c0 comp rx=0 tx=0).stop 2024-09-12T02:01:41.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:41.183+0000 7ff4505bd640 1 -- 172.21.15.16:0/2807280379 shutdown_connections 2024-09-12T02:01:41.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:41.183+0000 7ff4505bd640 1 --2- 172.21.15.16:0/2807280379 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4481016b0 0x7ff448108f50 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:41.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:41.183+0000 7ff4505bd640 1 --2- 172.21.15.16:0/2807280379 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff448100cf0 0x7ff448101170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:41.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:41.183+0000 7ff4505bd640 1 --2- 172.21.15.16:0/2807280379 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff448101fc0 0x7ff4481023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:41.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:41.183+0000 7ff4505bd640 1 -- 172.21.15.16:0/2807280379 >> 172.21.15.16:0/2807280379 conn(0x7ff4480fb950 msgr2=0x7ff4480fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:01:41.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:41.183+0000 7ff4505bd640 1 -- 172.21.15.16:0/2807280379 shutdown_connections 2024-09-12T02:01:41.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:41.183+0000 7ff4505bd640 1 -- 172.21.15.16:0/2807280379 wait complete. 2024-09-12T02:01:41.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:41.183+0000 7ff4505bd640 1 Processor -- start 2024-09-12T02:01:41.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:41.183+0000 7ff4505bd640 1 -- start start 2024-09-12T02:01:41.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:41.183+0000 7ff4505bd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff448100cf0 0x7ff448197db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:41.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:41.183+0000 7ff4505bd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4481016b0 0x7ff4481982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:41.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:41.183+0000 7ff4505bd640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff448101fc0 0x7ff44819f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:41.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:41.183+0000 7ff4505bd640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff44810d810 con 0x7ff448100cf0 2024-09-12T02:01:41.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:41.183+0000 7ff4505bd640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff44810d690 con 0x7ff4481016b0 2024-09-12T02:01:41.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:41.183+0000 7ff4505bd640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff44810d990 con 0x7ff448101fc0 2024-09-12T02:01:41.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:41.183+0000 7ff44e359640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff448100cf0 0x7ff448197db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:41.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:41.183+0000 7ff44e359640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff448100cf0 0x7ff448197db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46520/0 (socket says 172.21.15.16:46520) 2024-09-12T02:01:41.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:41.183+0000 7ff44e359640 1 -- 172.21.15.16:0/1621091065 learned_addr learned my addr 172.21.15.16:0/1621091065 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:01:41.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:41.184+0000 7ff44db58640 1 --2- 172.21.15.16:0/1621091065 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4481016b0 0x7ff4481982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:41.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:41.184+0000 7ff44e359640 1 -- 172.21.15.16:0/1621091065 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff448101fc0 msgr2=0x7ff44819f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:41.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:41.184+0000 7ff44e359640 1 --2- 172.21.15.16:0/1621091065 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff448101fc0 0x7ff44819f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:41.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:41.184+0000 7ff44e359640 1 -- 172.21.15.16:0/1621091065 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4481016b0 msgr2=0x7ff4481982f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:41.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:41.184+0000 7ff44e359640 1 --2- 172.21.15.16:0/1621091065 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4481016b0 0x7ff4481982f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:41.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:41.184+0000 7ff44e359640 1 -- 172.21.15.16:0/1621091065 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff44819f9d0 con 0x7ff448100cf0 2024-09-12T02:01:41.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:41.184+0000 7ff44e359640 1 --2- 172.21.15.16:0/1621091065 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff448100cf0 0x7ff448197db0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7ff44400d770 tx=0x7ff44400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:01:41.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:41.184+0000 7ff43f7fe640 1 -- 172.21.15.16:0/1621091065 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff444017070 con 0x7ff448100cf0 2024-09-12T02:01:41.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:41.184+0000 7ff4505bd640 1 -- 172.21.15.16:0/1621091065 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff44819fcc0 con 0x7ff448100cf0 2024-09-12T02:01:41.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:41.184+0000 7ff43f7fe640 1 -- 172.21.15.16:0/1621091065 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff444009c30 con 0x7ff448100cf0 2024-09-12T02:01:41.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:41.184+0000 7ff43f7fe640 1 -- 172.21.15.16:0/1621091065 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff444003dd0 con 0x7ff448100cf0 2024-09-12T02:01:41.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:41.184+0000 7ff4505bd640 1 -- 172.21.15.16:0/1621091065 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4481a01c0 con 0x7ff448100cf0 2024-09-12T02:01:41.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:41.185+0000 7ff4505bd640 1 -- 172.21.15.16:0/1621091065 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff448107e30 con 0x7ff448100cf0 2024-09-12T02:01:41.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:41.186+0000 7ff43f7fe640 1 -- 172.21.15.16:0/1621091065 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff444003f70 con 0x7ff448100cf0 2024-09-12T02:01:41.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:41.187+0000 7ff43f7fe640 1 --2- 172.21.15.16:0/1621091065 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7ff4340a2b90 0x7ff4340a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:41.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:41.187+0000 7ff44db58640 1 --2- 172.21.15.16:0/1621091065 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7ff4340a2b90 0x7ff4340a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:41.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:41.187+0000 7ff43f7fe640 1 -- 172.21.15.16:0/1621091065 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7ff4440ced60 con 0x7ff448100cf0 2024-09-12T02:01:41.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:41.188+0000 7ff44db58640 1 --2- 172.21.15.16:0/1621091065 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7ff4340a2b90 0x7ff4340a5050 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff4481992c0 tx=0x7ff43800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:01:41.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:41.190+0000 7ff43f7fe640 1 -- 172.21.15.16:0/1621091065 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff444009de0 con 0x7ff448100cf0 2024-09-12T02:01:41.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:41.362+0000 7ff4505bd640 1 -- 172.21.15.16:0/1621091065 --> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff448062630 con 0x7ff4340a2b90 2024-09-12T02:01:41.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:41.368+0000 7ff43f7fe640 1 -- 172.21.15.16:0/1621091065 <== mgr.14150 v2:172.21.15.16:6800/3887444820 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258833 (secure 0 0 0) 0x7ff448062630 con 0x7ff4340a2b90 2024-09-12T02:01:41.372 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T02:01:41.373 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-12T02:01:41.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:41.371+0000 7ff4505bd640 1 -- 172.21.15.16:0/1621091065 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7ff4340a2b90 msgr2=0x7ff4340a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:41.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:41.371+0000 7ff4505bd640 1 --2- 172.21.15.16:0/1621091065 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7ff4340a2b90 0x7ff4340a5050 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff4481992c0 tx=0x7ff43800b390 comp rx=0 tx=0).stop 2024-09-12T02:01:41.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:41.371+0000 7ff4505bd640 1 -- 172.21.15.16:0/1621091065 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff448100cf0 msgr2=0x7ff448197db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:41.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:41.371+0000 7ff4505bd640 1 --2- 172.21.15.16:0/1621091065 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff448100cf0 0x7ff448197db0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7ff44400d770 tx=0x7ff44400dc30 comp rx=0 tx=0).stop 2024-09-12T02:01:41.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:41.371+0000 7ff4505bd640 1 -- 172.21.15.16:0/1621091065 shutdown_connections 2024-09-12T02:01:41.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:41.371+0000 7ff4505bd640 1 --2- 172.21.15.16:0/1621091065 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7ff4340a2b90 0x7ff4340a5050 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:41.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:41.371+0000 7ff4505bd640 1 --2- 172.21.15.16:0/1621091065 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff448101fc0 0x7ff44819f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:41.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:41.371+0000 7ff4505bd640 1 --2- 172.21.15.16:0/1621091065 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4481016b0 0x7ff4481982f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:41.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:41.371+0000 7ff4505bd640 1 --2- 172.21.15.16:0/1621091065 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff448100cf0 0x7ff448197db0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:41.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:41.371+0000 7ff4505bd640 1 -- 172.21.15.16:0/1621091065 >> 172.21.15.16:0/1621091065 conn(0x7ff4480fb950 msgr2=0x7ff448105000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:01:41.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:41.371+0000 7ff4505bd640 1 -- 172.21.15.16:0/1621091065 shutdown_connections 2024-09-12T02:01:41.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:41.371+0000 7ff4505bd640 1 -- 172.21.15.16:0/1621091065 wait complete. 2024-09-12T02:01:41.383 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":328,"stamp":"2024-09-12T02:01:40.323810+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":659240,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":717,"num_read_kb":479,"num_write":352,"num_write_kb":3058,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":1180848,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":573,"ondisk_log_size":573,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":374280,"kb_used_data":12544,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124567544,"statfs":{"total":1151940427776,"available":1151557165056,"internally_reserved":0,"allocated":12845056,"data_stored":7190707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":370193723},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4871,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":87,"num_read_kb":52,"num_write":9,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002929"},"pg_stats":[{"pgid":"3.3f","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.318917+0000","last_change":"2024-09-12T01:59:55.503882+0000","last_active":"2024-09-12T02:01:38.318917+0000","last_peered":"2024-09-12T02:01:38.318917+0000","last_clean":"2024-09-12T02:01:38.318917+0000","last_became_active":"2024-09-12T01:59:55.503513+0000","last_became_peered":"2024-09-12T01:59:55.503513+0000","last_unstale":"2024-09-12T02:01:38.318917+0000","last_undegraded":"2024-09-12T02:01:38.318917+0000","last_fullsized":"2024-09-12T02:01:38.318917+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:43:13.098527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"82'4","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.322981+0000","last_change":"2024-09-12T01:59:55.492965+0000","last_active":"2024-09-12T02:01:38.322981+0000","last_peered":"2024-09-12T02:01:38.322981+0000","last_clean":"2024-09-12T02:01:38.322981+0000","last_became_active":"2024-09-12T01:59:55.492613+0000","last_became_peered":"2024-09-12T01:59:55.492613+0000","last_unstale":"2024-09-12T02:01:38.322981+0000","last_undegraded":"2024-09-12T02:01:38.322981+0000","last_fullsized":"2024-09-12T02:01:38.322981+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:58:20.889770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.332184+0000","last_change":"2024-09-12T01:59:55.497334+0000","last_active":"2024-09-12T02:01:38.332184+0000","last_peered":"2024-09-12T02:01:38.332184+0000","last_clean":"2024-09-12T02:01:38.332184+0000","last_became_active":"2024-09-12T01:59:55.497037+0000","last_became_peered":"2024-09-12T01:59:55.497037+0000","last_unstale":"2024-09-12T02:01:38.332184+0000","last_undegraded":"2024-09-12T02:01:38.332184+0000","last_fullsized":"2024-09-12T02:01:38.332184+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:43:04.082918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.331162+0000","last_change":"2024-09-12T01:59:55.495888+0000","last_active":"2024-09-12T02:01:38.331162+0000","last_peered":"2024-09-12T02:01:38.331162+0000","last_clean":"2024-09-12T02:01:38.331162+0000","last_became_active":"2024-09-12T01:59:55.492684+0000","last_became_peered":"2024-09-12T01:59:55.492684+0000","last_unstale":"2024-09-12T02:01:38.331162+0000","last_undegraded":"2024-09-12T02:01:38.331162+0000","last_fullsized":"2024-09-12T02:01:38.331162+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:15:00.500006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.574662+0000","last_change":"2024-09-12T01:59:55.494613+0000","last_active":"2024-09-12T02:00:09.574662+0000","last_peered":"2024-09-12T02:00:09.574662+0000","last_clean":"2024-09-12T02:00:09.574662+0000","last_became_active":"2024-09-12T01:59:55.494205+0000","last_became_peered":"2024-09-12T01:59:55.494205+0000","last_unstale":"2024-09-12T02:00:09.574662+0000","last_undegraded":"2024-09-12T02:00:09.574662+0000","last_fullsized":"2024-09-12T02:00:09.574662+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:39:14.573280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.331517+0000","last_change":"2024-09-12T01:59:55.496659+0000","last_active":"2024-09-12T02:01:38.331517+0000","last_peered":"2024-09-12T02:01:38.331517+0000","last_clean":"2024-09-12T02:01:38.331517+0000","last_became_active":"2024-09-12T01:59:55.496232+0000","last_became_peered":"2024-09-12T01:59:55.496232+0000","last_unstale":"2024-09-12T02:01:38.331517+0000","last_undegraded":"2024-09-12T02:01:38.331517+0000","last_fullsized":"2024-09-12T02:01:38.331517+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:48:21.544764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"82'5","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.317177+0000","last_change":"2024-09-12T01:59:55.486789+0000","last_active":"2024-09-12T02:01:38.317177+0000","last_peered":"2024-09-12T02:01:38.317177+0000","last_clean":"2024-09-12T02:01:38.317177+0000","last_became_active":"2024-09-12T01:59:55.485921+0000","last_became_peered":"2024-09-12T01:59:55.485921+0000","last_unstale":"2024-09-12T02:01:38.317177+0000","last_undegraded":"2024-09-12T02:01:38.317177+0000","last_fullsized":"2024-09-12T02:01:38.317177+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:21:16.071121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.621086+0000","last_change":"2024-09-12T01:59:55.494525+0000","last_active":"2024-09-12T02:00:09.621086+0000","last_peered":"2024-09-12T02:00:09.621086+0000","last_clean":"2024-09-12T02:00:09.621086+0000","last_became_active":"2024-09-12T01:59:55.493998+0000","last_became_peered":"2024-09-12T01:59:55.493998+0000","last_unstale":"2024-09-12T02:00:09.621086+0000","last_undegraded":"2024-09-12T02:00:09.621086+0000","last_fullsized":"2024-09-12T02:00:09.621086+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:59:24.653660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.622633+0000","last_change":"2024-09-12T01:59:55.501466+0000","last_active":"2024-09-12T02:00:09.622633+0000","last_peered":"2024-09-12T02:00:09.622633+0000","last_clean":"2024-09-12T02:00:09.622633+0000","last_became_active":"2024-09-12T01:59:55.500800+0000","last_became_peered":"2024-09-12T01:59:55.500800+0000","last_unstale":"2024-09-12T02:00:09.622633+0000","last_undegraded":"2024-09-12T02:00:09.622633+0000","last_fullsized":"2024-09-12T02:00:09.622633+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:55:13.704791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"82'16","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.331670+0000","last_change":"2024-09-12T01:59:55.497986+0000","last_active":"2024-09-12T02:01:38.331670+0000","last_peered":"2024-09-12T02:01:38.331670+0000","last_clean":"2024-09-12T02:01:38.331670+0000","last_became_active":"2024-09-12T01:59:55.497691+0000","last_became_peered":"2024-09-12T01:59:55.497691+0000","last_unstale":"2024-09-12T02:01:38.331670+0000","last_undegraded":"2024-09-12T02:01:38.331670+0000","last_fullsized":"2024-09-12T02:01:38.331670+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:37:07.702606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.329986+0000","last_change":"2024-09-12T01:59:55.501339+0000","last_active":"2024-09-12T02:01:38.329986+0000","last_peered":"2024-09-12T02:01:38.329986+0000","last_clean":"2024-09-12T02:01:38.329986+0000","last_became_active":"2024-09-12T01:59:55.500700+0000","last_became_peered":"2024-09-12T01:59:55.500700+0000","last_unstale":"2024-09-12T02:01:38.329986+0000","last_undegraded":"2024-09-12T02:01:38.329986+0000","last_fullsized":"2024-09-12T02:01:38.329986+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:12:15.879183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.330871+0000","last_change":"2024-09-12T01:59:55.495196+0000","last_active":"2024-09-12T02:01:38.330871+0000","last_peered":"2024-09-12T02:01:38.330871+0000","last_clean":"2024-09-12T02:01:38.330871+0000","last_became_active":"2024-09-12T01:59:55.494889+0000","last_became_peered":"2024-09-12T01:59:55.494889+0000","last_unstale":"2024-09-12T02:01:38.330871+0000","last_undegraded":"2024-09-12T02:01:38.330871+0000","last_fullsized":"2024-09-12T02:01:38.330871+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:32:30.621946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"82'10","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.315946+0000","last_change":"2024-09-12T01:59:55.492565+0000","last_active":"2024-09-12T02:01:38.315946+0000","last_peered":"2024-09-12T02:01:38.315946+0000","last_clean":"2024-09-12T02:01:38.315946+0000","last_became_active":"2024-09-12T01:59:55.492168+0000","last_became_peered":"2024-09-12T01:59:55.492168+0000","last_unstale":"2024-09-12T02:01:38.315946+0000","last_undegraded":"2024-09-12T02:01:38.315946+0000","last_fullsized":"2024-09-12T02:01:38.315946+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:39:48.630202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.323015+0000","last_change":"2024-09-12T01:59:55.493152+0000","last_active":"2024-09-12T02:01:38.323015+0000","last_peered":"2024-09-12T02:01:38.323015+0000","last_clean":"2024-09-12T02:01:38.323015+0000","last_became_active":"2024-09-12T01:59:55.492794+0000","last_became_peered":"2024-09-12T01:59:55.492794+0000","last_unstale":"2024-09-12T02:01:38.323015+0000","last_undegraded":"2024-09-12T02:01:38.323015+0000","last_fullsized":"2024-09-12T02:01:38.323015+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:24:27.270502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.333374+0000","last_change":"2024-09-12T01:59:55.494651+0000","last_active":"2024-09-12T02:01:38.333374+0000","last_peered":"2024-09-12T02:01:38.333374+0000","last_clean":"2024-09-12T02:01:38.333374+0000","last_became_active":"2024-09-12T01:59:55.494355+0000","last_became_peered":"2024-09-12T01:59:55.494355+0000","last_unstale":"2024-09-12T02:01:38.333374+0000","last_undegraded":"2024-09-12T02:01:38.333374+0000","last_fullsized":"2024-09-12T02:01:38.333374+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:14:44.364158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.622950+0000","last_change":"2024-09-12T01:59:55.495049+0000","last_active":"2024-09-12T02:00:09.622950+0000","last_peered":"2024-09-12T02:00:09.622950+0000","last_clean":"2024-09-12T02:00:09.622950+0000","last_became_active":"2024-09-12T01:59:55.494688+0000","last_became_peered":"2024-09-12T01:59:55.494688+0000","last_unstale":"2024-09-12T02:00:09.622950+0000","last_undegraded":"2024-09-12T02:00:09.622950+0000","last_fullsized":"2024-09-12T02:00:09.622950+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:13:05.944577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.319055+0000","last_change":"2024-09-12T01:59:55.504146+0000","last_active":"2024-09-12T02:01:38.319055+0000","last_peered":"2024-09-12T02:01:38.319055+0000","last_clean":"2024-09-12T02:01:38.319055+0000","last_became_active":"2024-09-12T01:59:55.503646+0000","last_became_peered":"2024-09-12T01:59:55.503646+0000","last_unstale":"2024-09-12T02:01:38.319055+0000","last_undegraded":"2024-09-12T02:01:38.319055+0000","last_fullsized":"2024-09-12T02:01:38.319055+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:23:51.780795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"82'10","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.325350+0000","last_change":"2024-09-12T01:59:55.489930+0000","last_active":"2024-09-12T02:01:38.325350+0000","last_peered":"2024-09-12T02:01:38.325350+0000","last_clean":"2024-09-12T02:01:38.325350+0000","last_became_active":"2024-09-12T01:59:55.489619+0000","last_became_peered":"2024-09-12T01:59:55.489619+0000","last_unstale":"2024-09-12T02:01:38.325350+0000","last_undegraded":"2024-09-12T02:01:38.325350+0000","last_fullsized":"2024-09-12T02:01:38.325350+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:04:57.307091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.569107+0000","last_change":"2024-09-12T01:59:55.492098+0000","last_active":"2024-09-12T02:00:09.569107+0000","last_peered":"2024-09-12T02:00:09.569107+0000","last_clean":"2024-09-12T02:00:09.569107+0000","last_became_active":"2024-09-12T01:59:55.491370+0000","last_became_peered":"2024-09-12T01:59:55.491370+0000","last_unstale":"2024-09-12T02:00:09.569107+0000","last_undegraded":"2024-09-12T02:00:09.569107+0000","last_fullsized":"2024-09-12T02:00:09.569107+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:25:06.307903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.314843+0000","last_change":"2024-09-12T01:59:55.499938+0000","last_active":"2024-09-12T02:01:38.314843+0000","last_peered":"2024-09-12T02:01:38.314843+0000","last_clean":"2024-09-12T02:01:38.314843+0000","last_became_active":"2024-09-12T01:59:55.499619+0000","last_became_peered":"2024-09-12T01:59:55.499619+0000","last_unstale":"2024-09-12T02:01:38.314843+0000","last_undegraded":"2024-09-12T02:01:38.314843+0000","last_fullsized":"2024-09-12T02:01:38.314843+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:00:17.536120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"82'11","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.321068+0000","last_change":"2024-09-12T01:59:55.486106+0000","last_active":"2024-09-12T02:01:38.321068+0000","last_peered":"2024-09-12T02:01:38.321068+0000","last_clean":"2024-09-12T02:01:38.321068+0000","last_became_active":"2024-09-12T01:59:55.485325+0000","last_became_peered":"2024-09-12T01:59:55.485325+0000","last_unstale":"2024-09-12T02:01:38.321068+0000","last_undegraded":"2024-09-12T02:01:38.321068+0000","last_fullsized":"2024-09-12T02:01:38.321068+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:35:46.446224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.622237+0000","last_change":"2024-09-12T01:59:55.496438+0000","last_active":"2024-09-12T02:00:09.622237+0000","last_peered":"2024-09-12T02:00:09.622237+0000","last_clean":"2024-09-12T02:00:09.622237+0000","last_became_active":"2024-09-12T01:59:55.495885+0000","last_became_peered":"2024-09-12T01:59:55.495885+0000","last_unstale":"2024-09-12T02:00:09.622237+0000","last_undegraded":"2024-09-12T02:00:09.622237+0000","last_fullsized":"2024-09-12T02:00:09.622237+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:15:14.911335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.621139+0000","last_change":"2024-09-12T01:59:55.489350+0000","last_active":"2024-09-12T02:00:09.621139+0000","last_peered":"2024-09-12T02:00:09.621139+0000","last_clean":"2024-09-12T02:00:09.621139+0000","last_became_active":"2024-09-12T01:59:55.489028+0000","last_became_peered":"2024-09-12T01:59:55.489028+0000","last_unstale":"2024-09-12T02:00:09.621139+0000","last_undegraded":"2024-09-12T02:00:09.621139+0000","last_fullsized":"2024-09-12T02:00:09.621139+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:29:23.769004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.566645+0000","last_change":"2024-09-12T01:59:55.494532+0000","last_active":"2024-09-12T02:00:09.566645+0000","last_peered":"2024-09-12T02:00:09.566645+0000","last_clean":"2024-09-12T02:00:09.566645+0000","last_became_active":"2024-09-12T01:59:55.490982+0000","last_became_peered":"2024-09-12T01:59:55.490982+0000","last_unstale":"2024-09-12T02:00:09.566645+0000","last_undegraded":"2024-09-12T02:00:09.566645+0000","last_fullsized":"2024-09-12T02:00:09.566645+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:04:50.633730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.621490+0000","last_change":"2024-09-12T01:59:55.496057+0000","last_active":"2024-09-12T02:00:09.621490+0000","last_peered":"2024-09-12T02:00:09.621490+0000","last_clean":"2024-09-12T02:00:09.621490+0000","last_became_active":"2024-09-12T01:59:55.495710+0000","last_became_peered":"2024-09-12T01:59:55.495710+0000","last_unstale":"2024-09-12T02:00:09.621490+0000","last_undegraded":"2024-09-12T02:00:09.621490+0000","last_fullsized":"2024-09-12T02:00:09.621490+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:49:01.962537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.569100+0000","last_change":"2024-09-12T01:59:55.491594+0000","last_active":"2024-09-12T02:00:09.569100+0000","last_peered":"2024-09-12T02:00:09.569100+0000","last_clean":"2024-09-12T02:00:09.569100+0000","last_became_active":"2024-09-12T01:59:55.491233+0000","last_became_peered":"2024-09-12T01:59:55.491233+0000","last_unstale":"2024-09-12T02:00:09.569100+0000","last_undegraded":"2024-09-12T02:00:09.569100+0000","last_fullsized":"2024-09-12T02:00:09.569100+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:24:22.066777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.309442+0000","last_change":"2024-09-12T01:59:55.498331+0000","last_active":"2024-09-12T02:01:38.309442+0000","last_peered":"2024-09-12T02:01:38.309442+0000","last_clean":"2024-09-12T02:01:38.309442+0000","last_became_active":"2024-09-12T01:59:55.497983+0000","last_became_peered":"2024-09-12T01:59:55.497983+0000","last_unstale":"2024-09-12T02:01:38.309442+0000","last_undegraded":"2024-09-12T02:01:38.309442+0000","last_fullsized":"2024-09-12T02:01:38.309442+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:37:00.735231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"82'4","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.315811+0000","last_change":"2024-09-12T02:00:09.575358+0000","last_active":"2024-09-12T02:01:38.315811+0000","last_peered":"2024-09-12T02:01:38.315811+0000","last_clean":"2024-09-12T02:01:38.315811+0000","last_became_active":"2024-09-12T02:00:09.574913+0000","last_became_peered":"2024-09-12T02:00:09.574913+0000","last_unstale":"2024-09-12T02:01:38.315811+0000","last_undegraded":"2024-09-12T02:01:38.315811+0000","last_fullsized":"2024-09-12T02:01:38.315811+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:19:53.630453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.333406+0000","last_change":"2024-09-12T01:59:55.492419+0000","last_active":"2024-09-12T02:01:38.333406+0000","last_peered":"2024-09-12T02:01:38.333406+0000","last_clean":"2024-09-12T02:01:38.333406+0000","last_became_active":"2024-09-12T01:59:55.492003+0000","last_became_peered":"2024-09-12T01:59:55.492003+0000","last_unstale":"2024-09-12T02:01:38.333406+0000","last_undegraded":"2024-09-12T02:01:38.333406+0000","last_fullsized":"2024-09-12T02:01:38.333406+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:25:45.774679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.568801+0000","last_change":"2024-09-12T01:59:55.490125+0000","last_active":"2024-09-12T02:00:09.568801+0000","last_peered":"2024-09-12T02:00:09.568801+0000","last_clean":"2024-09-12T02:00:09.568801+0000","last_became_active":"2024-09-12T01:59:55.489636+0000","last_became_peered":"2024-09-12T01:59:55.489636+0000","last_unstale":"2024-09-12T02:00:09.568801+0000","last_undegraded":"2024-09-12T02:00:09.568801+0000","last_fullsized":"2024-09-12T02:00:09.568801+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:41:30.922185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.569135+0000","last_change":"2024-09-12T01:59:55.496322+0000","last_active":"2024-09-12T02:00:09.569135+0000","last_peered":"2024-09-12T02:00:09.569135+0000","last_clean":"2024-09-12T02:00:09.569135+0000","last_became_active":"2024-09-12T01:59:55.496005+0000","last_became_peered":"2024-09-12T01:59:55.496005+0000","last_unstale":"2024-09-12T02:00:09.569135+0000","last_undegraded":"2024-09-12T02:00:09.569135+0000","last_fullsized":"2024-09-12T02:00:09.569135+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:53:39.248715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.310438+0000","last_change":"2024-09-12T01:59:55.497758+0000","last_active":"2024-09-12T02:01:38.310438+0000","last_peered":"2024-09-12T02:01:38.310438+0000","last_clean":"2024-09-12T02:01:38.310438+0000","last_became_active":"2024-09-12T01:59:55.497282+0000","last_became_peered":"2024-09-12T01:59:55.497282+0000","last_unstale":"2024-09-12T02:01:38.310438+0000","last_undegraded":"2024-09-12T02:01:38.310438+0000","last_fullsized":"2024-09-12T02:01:38.310438+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:08:25.509372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.574620+0000","last_change":"2024-09-12T01:59:55.497919+0000","last_active":"2024-09-12T02:00:09.574620+0000","last_peered":"2024-09-12T02:00:09.574620+0000","last_clean":"2024-09-12T02:00:09.574620+0000","last_became_active":"2024-09-12T01:59:55.497609+0000","last_became_peered":"2024-09-12T01:59:55.497609+0000","last_unstale":"2024-09-12T02:00:09.574620+0000","last_undegraded":"2024-09-12T02:00:09.574620+0000","last_fullsized":"2024-09-12T02:00:09.574620+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:10:22.281180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"81'6","reported_seq":122,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:34.680658+0000","last_change":"2024-09-12T01:59:53.473974+0000","last_active":"2024-09-12T02:01:34.680658+0000","last_peered":"2024-09-12T02:01:34.680658+0000","last_clean":"2024-09-12T02:01:34.680658+0000","last_became_active":"2024-09-12T01:59:53.473546+0000","last_became_peered":"2024-09-12T01:59:53.473546+0000","last_unstale":"2024-09-12T02:01:34.680658+0000","last_undegraded":"2024-09-12T02:01:34.680658+0000","last_fullsized":"2024-09-12T02:01:34.680658+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:28:03.532249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":94,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.567217+0000","last_change":"2024-09-12T01:59:55.493197+0000","last_active":"2024-09-12T02:00:09.567217+0000","last_peered":"2024-09-12T02:00:09.567217+0000","last_clean":"2024-09-12T02:00:09.567217+0000","last_became_active":"2024-09-12T01:59:55.492845+0000","last_became_peered":"2024-09-12T01:59:55.492845+0000","last_unstale":"2024-09-12T02:00:09.567217+0000","last_undegraded":"2024-09-12T02:00:09.567217+0000","last_fullsized":"2024-09-12T02:00:09.567217+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:42:50.790896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"80'3","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.622865+0000","last_change":"2024-09-12T01:59:53.471214+0000","last_active":"2024-09-12T02:00:09.622865+0000","last_peered":"2024-09-12T02:00:09.622865+0000","last_clean":"2024-09-12T02:00:09.622865+0000","last_became_active":"2024-09-12T01:59:53.470858+0000","last_became_peered":"2024-09-12T01:59:53.470858+0000","last_unstale":"2024-09-12T02:00:09.622865+0000","last_undegraded":"2024-09-12T02:00:09.622865+0000","last_fullsized":"2024-09-12T02:00:09.622865+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:34:48.730392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"80'2","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.562106+0000","last_change":"2024-09-12T01:59:53.464568+0000","last_active":"2024-09-12T02:00:09.562106+0000","last_peered":"2024-09-12T02:00:09.562106+0000","last_clean":"2024-09-12T02:00:09.562106+0000","last_became_active":"2024-09-12T01:59:53.464168+0000","last_became_peered":"2024-09-12T01:59:53.464168+0000","last_unstale":"2024-09-12T02:00:09.562106+0000","last_undegraded":"2024-09-12T02:00:09.562106+0000","last_fullsized":"2024-09-12T02:00:09.562106+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:41:48.135522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.621151+0000","last_change":"2024-09-12T01:59:55.487177+0000","last_active":"2024-09-12T02:00:09.621151+0000","last_peered":"2024-09-12T02:00:09.621151+0000","last_clean":"2024-09-12T02:00:09.621151+0000","last_became_active":"2024-09-12T01:59:55.486886+0000","last_became_peered":"2024-09-12T01:59:55.486886+0000","last_unstale":"2024-09-12T02:00:09.621151+0000","last_undegraded":"2024-09-12T02:00:09.621151+0000","last_fullsized":"2024-09-12T02:00:09.621151+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:32:40.150071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"80'1","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.567129+0000","last_change":"2024-09-12T01:59:53.474114+0000","last_active":"2024-09-12T02:00:09.567129+0000","last_peered":"2024-09-12T02:00:09.567129+0000","last_clean":"2024-09-12T02:00:09.567129+0000","last_became_active":"2024-09-12T01:59:53.473755+0000","last_became_peered":"2024-09-12T01:59:53.473755+0000","last_unstale":"2024-09-12T02:00:09.567129+0000","last_undegraded":"2024-09-12T02:00:09.567129+0000","last_fullsized":"2024-09-12T02:00:09.567129+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:07:07.021137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"82'11","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.323623+0000","last_change":"2024-09-12T01:59:55.489868+0000","last_active":"2024-09-12T02:01:38.323623+0000","last_peered":"2024-09-12T02:01:38.323623+0000","last_clean":"2024-09-12T02:01:38.323623+0000","last_became_active":"2024-09-12T01:59:55.489537+0000","last_became_peered":"2024-09-12T01:59:55.489537+0000","last_unstale":"2024-09-12T02:01:38.323623+0000","last_undegraded":"2024-09-12T02:01:38.323623+0000","last_fullsized":"2024-09-12T02:01:38.323623+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:18:49.417689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"82'4","reported_seq":120,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.276658+0000","last_change":"2024-09-12T01:59:53.471402+0000","last_active":"2024-09-12T02:01:38.276658+0000","last_peered":"2024-09-12T02:01:38.276658+0000","last_clean":"2024-09-12T02:01:38.276658+0000","last_became_active":"2024-09-12T01:59:53.470940+0000","last_became_peered":"2024-09-12T01:59:53.470940+0000","last_unstale":"2024-09-12T02:01:38.276658+0000","last_undegraded":"2024-09-12T02:01:38.276658+0000","last_fullsized":"2024-09-12T02:01:38.276658+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:41:16.853592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":94,"num_read_kb":94,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.623260+0000","last_change":"2024-09-12T01:59:55.499047+0000","last_active":"2024-09-12T02:00:09.623260+0000","last_peered":"2024-09-12T02:00:09.623260+0000","last_clean":"2024-09-12T02:00:09.623260+0000","last_became_active":"2024-09-12T01:59:55.498071+0000","last_became_peered":"2024-09-12T01:59:55.498071+0000","last_unstale":"2024-09-12T02:00:09.623260+0000","last_undegraded":"2024-09-12T02:00:09.623260+0000","last_fullsized":"2024-09-12T02:00:09.623260+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:36:40.106041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"80'2","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.622966+0000","last_change":"2024-09-12T01:59:53.473508+0000","last_active":"2024-09-12T02:00:09.622966+0000","last_peered":"2024-09-12T02:00:09.622966+0000","last_clean":"2024-09-12T02:00:09.622966+0000","last_became_active":"2024-09-12T01:59:53.473063+0000","last_became_peered":"2024-09-12T01:59:53.473063+0000","last_unstale":"2024-09-12T02:00:09.622966+0000","last_undegraded":"2024-09-12T02:00:09.622966+0000","last_fullsized":"2024-09-12T02:00:09.622966+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:55:32.161769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.333032+0000","last_change":"2024-09-12T01:59:55.491721+0000","last_active":"2024-09-12T02:01:38.333032+0000","last_peered":"2024-09-12T02:01:38.333032+0000","last_clean":"2024-09-12T02:01:38.333032+0000","last_became_active":"2024-09-12T01:59:55.491364+0000","last_became_peered":"2024-09-12T01:59:55.491364+0000","last_unstale":"2024-09-12T02:01:38.333032+0000","last_undegraded":"2024-09-12T02:01:38.333032+0000","last_fullsized":"2024-09-12T02:01:38.333032+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:21:20.795420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"80'4","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.567315+0000","last_change":"2024-09-12T01:59:53.921789+0000","last_active":"2024-09-12T02:00:09.567315+0000","last_peered":"2024-09-12T02:00:09.567315+0000","last_clean":"2024-09-12T02:00:09.567315+0000","last_became_active":"2024-09-12T01:59:53.921383+0000","last_became_peered":"2024-09-12T01:59:53.921383+0000","last_unstale":"2024-09-12T02:00:09.567315+0000","last_undegraded":"2024-09-12T02:00:09.567315+0000","last_fullsized":"2024-09-12T02:00:09.567315+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:21:58.792794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.622517+0000","last_change":"2024-09-12T01:59:55.503008+0000","last_active":"2024-09-12T02:00:09.622517+0000","last_peered":"2024-09-12T02:00:09.622517+0000","last_clean":"2024-09-12T02:00:09.622517+0000","last_became_active":"2024-09-12T01:59:55.502538+0000","last_became_peered":"2024-09-12T01:59:55.502538+0000","last_unstale":"2024-09-12T02:00:09.622517+0000","last_undegraded":"2024-09-12T02:00:09.622517+0000","last_fullsized":"2024-09-12T02:00:09.622517+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:51:36.735985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"82'32","reported_seq":253,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.686996+0000","last_change":"2024-09-12T01:59:53.470688+0000","last_active":"2024-09-12T02:01:38.686996+0000","last_peered":"2024-09-12T02:01:38.686996+0000","last_clean":"2024-09-12T02:01:38.686996+0000","last_became_active":"2024-09-12T01:59:53.470231+0000","last_became_peered":"2024-09-12T01:59:53.470231+0000","last_unstale":"2024-09-12T02:01:38.686996+0000","last_undegraded":"2024-09-12T02:01:38.686996+0000","last_fullsized":"2024-09-12T02:01:38.686996+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:53:54.439893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64635,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":199,"num_read_kb":163,"num_write":33,"num_write_kb":76,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.575173+0000","last_change":"2024-09-12T01:59:55.495343+0000","last_active":"2024-09-12T02:00:09.575173+0000","last_peered":"2024-09-12T02:00:09.575173+0000","last_clean":"2024-09-12T02:00:09.575173+0000","last_became_active":"2024-09-12T01:59:55.495006+0000","last_became_peered":"2024-09-12T01:59:55.495006+0000","last_unstale":"2024-09-12T02:00:09.575173+0000","last_undegraded":"2024-09-12T02:00:09.575173+0000","last_fullsized":"2024-09-12T02:00:09.575173+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:32:20.932551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'6","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.569082+0000","last_change":"2024-09-12T01:59:53.468625+0000","last_active":"2024-09-12T02:00:09.569082+0000","last_peered":"2024-09-12T02:00:09.569082+0000","last_clean":"2024-09-12T02:00:09.569082+0000","last_became_active":"2024-09-12T01:59:53.468015+0000","last_became_peered":"2024-09-12T01:59:53.468015+0000","last_unstale":"2024-09-12T02:00:09.569082+0000","last_undegraded":"2024-09-12T02:00:09.569082+0000","last_fullsized":"2024-09-12T02:00:09.569082+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:49:34.842239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.319752+0000","last_change":"2024-09-12T01:59:55.487962+0000","last_active":"2024-09-12T02:01:38.319752+0000","last_peered":"2024-09-12T02:01:38.319752+0000","last_clean":"2024-09-12T02:01:38.319752+0000","last_became_active":"2024-09-12T01:59:55.487619+0000","last_became_peered":"2024-09-12T01:59:55.487619+0000","last_unstale":"2024-09-12T02:01:38.319752+0000","last_undegraded":"2024-09-12T02:01:38.319752+0000","last_fullsized":"2024-09-12T02:01:38.319752+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:32:16.882373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"80'3","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.567124+0000","last_change":"2024-09-12T01:59:53.474418+0000","last_active":"2024-09-12T02:00:09.567124+0000","last_peered":"2024-09-12T02:00:09.567124+0000","last_clean":"2024-09-12T02:00:09.567124+0000","last_became_active":"2024-09-12T01:59:53.474059+0000","last_became_peered":"2024-09-12T01:59:53.474059+0000","last_unstale":"2024-09-12T02:00:09.567124+0000","last_undegraded":"2024-09-12T02:00:09.567124+0000","last_fullsized":"2024-09-12T02:00:09.567124+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:10:54.963435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"82'15","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.333121+0000","last_change":"2024-09-12T01:59:55.493394+0000","last_active":"2024-09-12T02:01:38.333121+0000","last_peered":"2024-09-12T02:01:38.333121+0000","last_clean":"2024-09-12T02:01:38.333121+0000","last_became_active":"2024-09-12T01:59:55.491669+0000","last_became_peered":"2024-09-12T01:59:55.491669+0000","last_unstale":"2024-09-12T02:01:38.333121+0000","last_undegraded":"2024-09-12T02:01:38.333121+0000","last_fullsized":"2024-09-12T02:01:38.333121+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:14:40.871154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"80'3","reported_seq":121,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:37.685132+0000","last_change":"2024-09-12T01:59:53.471623+0000","last_active":"2024-09-12T02:01:37.685132+0000","last_peered":"2024-09-12T02:01:37.685132+0000","last_clean":"2024-09-12T02:01:37.685132+0000","last_became_active":"2024-09-12T01:59:53.471294+0000","last_became_peered":"2024-09-12T01:59:53.471294+0000","last_unstale":"2024-09-12T02:01:37.685132+0000","last_undegraded":"2024-09-12T02:01:37.685132+0000","last_fullsized":"2024-09-12T02:01:37.685132+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:51:54.130883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":97,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.322144+0000","last_change":"2024-09-12T01:59:55.493082+0000","last_active":"2024-09-12T02:01:38.322144+0000","last_peered":"2024-09-12T02:01:38.322144+0000","last_clean":"2024-09-12T02:01:38.322144+0000","last_became_active":"2024-09-12T01:59:55.492778+0000","last_became_peered":"2024-09-12T01:59:55.492778+0000","last_unstale":"2024-09-12T02:01:38.322144+0000","last_undegraded":"2024-09-12T02:01:38.322144+0000","last_fullsized":"2024-09-12T02:01:38.322144+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:59:56.466377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'15","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.588050+0000","last_change":"2024-09-12T02:00:09.588050+0000","last_active":"2024-09-12T02:00:09.588050+0000","last_peered":"2024-09-12T02:00:09.588050+0000","last_clean":"2024-09-12T02:00:09.588050+0000","last_became_active":"2024-09-12T02:00:09.575478+0000","last_became_peered":"2024-09-12T02:00:09.575478+0000","last_unstale":"2024-09-12T02:00:09.588050+0000","last_undegraded":"2024-09-12T02:00:09.588050+0000","last_fullsized":"2024-09-12T02:00:09.588050+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:38:45.484167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.566976+0000","last_change":"2024-09-12T01:59:55.497601+0000","last_active":"2024-09-12T02:00:09.566976+0000","last_peered":"2024-09-12T02:00:09.566976+0000","last_clean":"2024-09-12T02:00:09.566976+0000","last_became_active":"2024-09-12T01:59:55.497202+0000","last_became_peered":"2024-09-12T01:59:55.497202+0000","last_unstale":"2024-09-12T02:00:09.566976+0000","last_undegraded":"2024-09-12T02:00:09.566976+0000","last_fullsized":"2024-09-12T02:00:09.566976+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:22:22.850736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"80'2","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.575359+0000","last_change":"2024-09-12T01:59:53.467575+0000","last_active":"2024-09-12T02:00:09.575359+0000","last_peered":"2024-09-12T02:00:09.575359+0000","last_clean":"2024-09-12T02:00:09.575359+0000","last_became_active":"2024-09-12T01:59:53.467146+0000","last_became_peered":"2024-09-12T01:59:53.467146+0000","last_unstale":"2024-09-12T02:00:09.575359+0000","last_undegraded":"2024-09-12T02:00:09.575359+0000","last_fullsized":"2024-09-12T02:00:09.575359+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:07:25.962496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"82'5","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.314382+0000","last_change":"2024-09-12T01:59:55.498297+0000","last_active":"2024-09-12T02:01:38.314382+0000","last_peered":"2024-09-12T02:01:38.314382+0000","last_clean":"2024-09-12T02:01:38.314382+0000","last_became_active":"2024-09-12T01:59:55.497367+0000","last_became_peered":"2024-09-12T01:59:55.497367+0000","last_unstale":"2024-09-12T02:01:38.314382+0000","last_undegraded":"2024-09-12T02:01:38.314382+0000","last_fullsized":"2024-09-12T02:01:38.314382+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:58:32.717710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"80'2","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.562292+0000","last_change":"2024-09-12T01:59:53.466664+0000","last_active":"2024-09-12T02:00:09.562292+0000","last_peered":"2024-09-12T02:00:09.562292+0000","last_clean":"2024-09-12T02:00:09.562292+0000","last_became_active":"2024-09-12T01:59:53.466318+0000","last_became_peered":"2024-09-12T01:59:53.466318+0000","last_unstale":"2024-09-12T02:00:09.562292+0000","last_undegraded":"2024-09-12T02:00:09.562292+0000","last_fullsized":"2024-09-12T02:00:09.562292+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:12:13.541420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.313289+0000","last_change":"2024-09-12T01:59:55.490420+0000","last_active":"2024-09-12T02:01:38.313289+0000","last_peered":"2024-09-12T02:01:38.313289+0000","last_clean":"2024-09-12T02:01:38.313289+0000","last_became_active":"2024-09-12T01:59:55.489757+0000","last_became_peered":"2024-09-12T01:59:55.489757+0000","last_unstale":"2024-09-12T02:01:38.313289+0000","last_undegraded":"2024-09-12T02:01:38.313289+0000","last_fullsized":"2024-09-12T02:01:38.313289+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:34:43.117142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'1","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.567185+0000","last_change":"2024-09-12T01:59:53.466399+0000","last_active":"2024-09-12T02:00:09.567185+0000","last_peered":"2024-09-12T02:00:09.567185+0000","last_clean":"2024-09-12T02:00:09.567185+0000","last_became_active":"2024-09-12T01:59:53.465982+0000","last_became_peered":"2024-09-12T01:59:53.465982+0000","last_unstale":"2024-09-12T02:00:09.567185+0000","last_undegraded":"2024-09-12T02:00:09.567185+0000","last_fullsized":"2024-09-12T02:00:09.567185+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:18:02.015040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.326034+0000","last_change":"2024-09-12T01:59:55.491527+0000","last_active":"2024-09-12T02:01:38.326034+0000","last_peered":"2024-09-12T02:01:38.326034+0000","last_clean":"2024-09-12T02:01:38.326034+0000","last_became_active":"2024-09-12T01:59:55.491165+0000","last_became_peered":"2024-09-12T01:59:55.491165+0000","last_unstale":"2024-09-12T02:01:38.326034+0000","last_undegraded":"2024-09-12T02:01:38.326034+0000","last_fullsized":"2024-09-12T02:01:38.326034+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:57:05.506384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"80'3","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.569187+0000","last_change":"2024-09-12T01:59:54.075752+0000","last_active":"2024-09-12T02:00:09.569187+0000","last_peered":"2024-09-12T02:00:09.569187+0000","last_clean":"2024-09-12T02:00:09.569187+0000","last_became_active":"2024-09-12T01:59:54.075387+0000","last_became_peered":"2024-09-12T01:59:54.075387+0000","last_unstale":"2024-09-12T02:00:09.569187+0000","last_undegraded":"2024-09-12T02:00:09.569187+0000","last_fullsized":"2024-09-12T02:00:09.569187+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:21:15.980591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.330203+0000","last_change":"2024-09-12T01:59:55.500935+0000","last_active":"2024-09-12T02:01:38.330203+0000","last_peered":"2024-09-12T02:01:38.330203+0000","last_clean":"2024-09-12T02:01:38.330203+0000","last_became_active":"2024-09-12T01:59:55.500206+0000","last_became_peered":"2024-09-12T01:59:55.500206+0000","last_unstale":"2024-09-12T02:01:38.330203+0000","last_undegraded":"2024-09-12T02:01:38.330203+0000","last_fullsized":"2024-09-12T02:01:38.330203+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:19:59.937701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":334,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.623254+0000","last_change":"2024-09-12T01:58:47.553853+0000","last_active":"2024-09-12T02:00:09.623254+0000","last_peered":"2024-09-12T02:00:09.623254+0000","last_clean":"2024-09-12T02:00:09.623254+0000","last_became_active":"2024-09-12T01:58:47.553357+0000","last_became_peered":"2024-09-12T01:58:47.553357+0000","last_unstale":"2024-09-12T02:00:09.623254+0000","last_undegraded":"2024-09-12T02:00:09.623254+0000","last_fullsized":"2024-09-12T02:00:09.623254+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:55:26.917733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:55:26.917733+0000","last_clean_scrub_stamp":"2024-09-12T01:55:26.917733+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:58:26.081766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"80'3","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.563615+0000","last_change":"2024-09-12T01:59:53.467907+0000","last_active":"2024-09-12T02:00:09.563615+0000","last_peered":"2024-09-12T02:00:09.563615+0000","last_clean":"2024-09-12T02:00:09.563615+0000","last_became_active":"2024-09-12T01:59:53.467164+0000","last_became_peered":"2024-09-12T01:59:53.467164+0000","last_unstale":"2024-09-12T02:00:09.563615+0000","last_undegraded":"2024-09-12T02:00:09.563615+0000","last_fullsized":"2024-09-12T02:00:09.563615+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:09:35.242708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.621290+0000","last_change":"2024-09-12T01:59:55.495109+0000","last_active":"2024-09-12T02:00:09.621290+0000","last_peered":"2024-09-12T02:00:09.621290+0000","last_clean":"2024-09-12T02:00:09.621290+0000","last_became_active":"2024-09-12T01:59:55.494791+0000","last_became_peered":"2024-09-12T01:59:55.494791+0000","last_unstale":"2024-09-12T02:00:09.621290+0000","last_undegraded":"2024-09-12T02:00:09.621290+0000","last_fullsized":"2024-09-12T02:00:09.621290+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:22:43.882050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"80'3","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.623115+0000","last_change":"2024-09-12T01:59:53.473286+0000","last_active":"2024-09-12T02:00:09.623115+0000","last_peered":"2024-09-12T02:00:09.623115+0000","last_clean":"2024-09-12T02:00:09.623115+0000","last_became_active":"2024-09-12T01:59:53.472976+0000","last_became_peered":"2024-09-12T01:59:53.472976+0000","last_unstale":"2024-09-12T02:00:09.623115+0000","last_undegraded":"2024-09-12T02:00:09.623115+0000","last_fullsized":"2024-09-12T02:00:09.623115+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:32:39.604173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"82'5","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.313749+0000","last_change":"2024-09-12T02:00:09.571159+0000","last_active":"2024-09-12T02:01:38.313749+0000","last_peered":"2024-09-12T02:01:38.313749+0000","last_clean":"2024-09-12T02:01:38.313749+0000","last_became_active":"2024-09-12T02:00:09.570672+0000","last_became_peered":"2024-09-12T02:00:09.570672+0000","last_unstale":"2024-09-12T02:01:38.313749+0000","last_undegraded":"2024-09-12T02:01:38.313749+0000","last_fullsized":"2024-09-12T02:01:38.313749+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:04:10.808942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"80'4","reported_seq":121,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.685685+0000","last_change":"2024-09-12T01:59:53.921856+0000","last_active":"2024-09-12T02:01:38.685685+0000","last_peered":"2024-09-12T02:01:38.685685+0000","last_clean":"2024-09-12T02:01:38.685685+0000","last_became_active":"2024-09-12T01:59:53.920672+0000","last_became_peered":"2024-09-12T01:59:53.920672+0000","last_unstale":"2024-09-12T02:01:38.685685+0000","last_undegraded":"2024-09-12T02:01:38.685685+0000","last_fullsized":"2024-09-12T02:01:38.685685+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:42:33.876493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":95,"num_read_kb":95,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"82'8","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.309498+0000","last_change":"2024-09-12T02:00:09.571185+0000","last_active":"2024-09-12T02:01:38.309498+0000","last_peered":"2024-09-12T02:01:38.309498+0000","last_clean":"2024-09-12T02:01:38.309498+0000","last_became_active":"2024-09-12T02:00:09.570769+0000","last_became_peered":"2024-09-12T02:00:09.570769+0000","last_unstale":"2024-09-12T02:01:38.309498+0000","last_undegraded":"2024-09-12T02:01:38.309498+0000","last_fullsized":"2024-09-12T02:01:38.309498+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:34:17.230435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.2","version":"80'1","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.622691+0000","last_change":"2024-09-12T01:59:53.472072+0000","last_active":"2024-09-12T02:00:09.622691+0000","last_peered":"2024-09-12T02:00:09.622691+0000","last_clean":"2024-09-12T02:00:09.622691+0000","last_became_active":"2024-09-12T01:59:53.471088+0000","last_became_peered":"2024-09-12T01:59:53.471088+0000","last_unstale":"2024-09-12T02:00:09.622691+0000","last_undegraded":"2024-09-12T02:00:09.622691+0000","last_fullsized":"2024-09-12T02:00:09.622691+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:02:13.879131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.330778+0000","last_change":"2024-09-12T01:59:55.502843+0000","last_active":"2024-09-12T02:01:38.330778+0000","last_peered":"2024-09-12T02:01:38.330778+0000","last_clean":"2024-09-12T02:01:38.330778+0000","last_became_active":"2024-09-12T01:59:55.499296+0000","last_became_peered":"2024-09-12T01:59:55.499296+0000","last_unstale":"2024-09-12T02:01:38.330778+0000","last_undegraded":"2024-09-12T02:01:38.330778+0000","last_fullsized":"2024-09-12T02:01:38.330778+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:04:21.787370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.575061+0000","last_change":"2024-09-12T02:00:09.575061+0000","last_active":"2024-09-12T02:00:09.575061+0000","last_peered":"2024-09-12T02:00:09.575061+0000","last_clean":"2024-09-12T02:00:09.575061+0000","last_became_active":"2024-09-12T02:00:09.574653+0000","last_became_peered":"2024-09-12T02:00:09.574653+0000","last_unstale":"2024-09-12T02:00:09.575061+0000","last_undegraded":"2024-09-12T02:00:09.575061+0000","last_fullsized":"2024-09-12T02:00:09.575061+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:32:27.931211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"82'5","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.320660+0000","last_change":"2024-09-12T01:59:55.494162+0000","last_active":"2024-09-12T02:01:38.320660+0000","last_peered":"2024-09-12T02:01:38.320660+0000","last_clean":"2024-09-12T02:01:38.320660+0000","last_became_active":"2024-09-12T01:59:55.493802+0000","last_became_peered":"2024-09-12T01:59:55.493802+0000","last_unstale":"2024-09-12T02:01:38.320660+0000","last_undegraded":"2024-09-12T02:01:38.320660+0000","last_fullsized":"2024-09-12T02:01:38.320660+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:22:16.302683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"80'1","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.575278+0000","last_change":"2024-09-12T01:59:53.921669+0000","last_active":"2024-09-12T02:00:09.575278+0000","last_peered":"2024-09-12T02:00:09.575278+0000","last_clean":"2024-09-12T02:00:09.575278+0000","last_became_active":"2024-09-12T01:59:53.921328+0000","last_became_peered":"2024-09-12T01:59:53.921328+0000","last_unstale":"2024-09-12T02:00:09.575278+0000","last_undegraded":"2024-09-12T02:00:09.575278+0000","last_fullsized":"2024-09-12T02:00:09.575278+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:46:33.313726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.311067+0000","last_change":"2024-09-12T01:59:55.488986+0000","last_active":"2024-09-12T02:01:38.311067+0000","last_peered":"2024-09-12T02:01:38.311067+0000","last_clean":"2024-09-12T02:01:38.311067+0000","last_became_active":"2024-09-12T01:59:55.488632+0000","last_became_peered":"2024-09-12T01:59:55.488632+0000","last_unstale":"2024-09-12T02:01:38.311067+0000","last_undegraded":"2024-09-12T02:01:38.311067+0000","last_fullsized":"2024-09-12T02:01:38.311067+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:20:57.596976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"82'11","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:21.501527+0000","last_change":"2024-09-12T01:59:53.467648+0000","last_active":"2024-09-12T02:00:21.501527+0000","last_peered":"2024-09-12T02:00:21.501527+0000","last_clean":"2024-09-12T02:00:21.501527+0000","last_became_active":"2024-09-12T01:59:53.467255+0000","last_became_peered":"2024-09-12T01:59:53.467255+0000","last_unstale":"2024-09-12T02:00:21.501527+0000","last_undegraded":"2024-09-12T02:00:21.501527+0000","last_fullsized":"2024-09-12T02:00:21.501527+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:37:09.897388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.621489+0000","last_change":"2024-09-12T01:59:55.487603+0000","last_active":"2024-09-12T02:00:09.621489+0000","last_peered":"2024-09-12T02:00:09.621489+0000","last_clean":"2024-09-12T02:00:09.621489+0000","last_became_active":"2024-09-12T01:59:55.487334+0000","last_became_peered":"2024-09-12T01:59:55.487334+0000","last_unstale":"2024-09-12T02:00:09.621489+0000","last_undegraded":"2024-09-12T02:00:09.621489+0000","last_fullsized":"2024-09-12T02:00:09.621489+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:23:16.669515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'4","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.563870+0000","last_change":"2024-09-12T01:59:54.076156+0000","last_active":"2024-09-12T02:00:09.563870+0000","last_peered":"2024-09-12T02:00:09.563870+0000","last_clean":"2024-09-12T02:00:09.563870+0000","last_became_active":"2024-09-12T01:59:54.075794+0000","last_became_peered":"2024-09-12T01:59:54.075794+0000","last_unstale":"2024-09-12T02:00:09.563870+0000","last_undegraded":"2024-09-12T02:00:09.563870+0000","last_fullsized":"2024-09-12T02:00:09.563870+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:06:17.431696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.333603+0000","last_change":"2024-09-12T01:59:55.494604+0000","last_active":"2024-09-12T02:01:38.333603+0000","last_peered":"2024-09-12T02:01:38.333603+0000","last_clean":"2024-09-12T02:01:38.333603+0000","last_became_active":"2024-09-12T01:59:55.494300+0000","last_became_peered":"2024-09-12T01:59:55.494300+0000","last_unstale":"2024-09-12T02:01:38.333603+0000","last_undegraded":"2024-09-12T02:01:38.333603+0000","last_fullsized":"2024-09-12T02:01:38.333603+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:18:56.359299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.622205+0000","last_change":"2024-09-12T01:59:53.921832+0000","last_active":"2024-09-12T02:00:09.622205+0000","last_peered":"2024-09-12T02:00:09.622205+0000","last_clean":"2024-09-12T02:00:09.622205+0000","last_became_active":"2024-09-12T01:59:53.921379+0000","last_became_peered":"2024-09-12T01:59:53.921379+0000","last_unstale":"2024-09-12T02:00:09.622205+0000","last_undegraded":"2024-09-12T02:00:09.622205+0000","last_fullsized":"2024-09-12T02:00:09.622205+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:26:02.277149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.574438+0000","last_change":"2024-09-12T01:59:55.493198+0000","last_active":"2024-09-12T02:00:09.574438+0000","last_peered":"2024-09-12T02:00:09.574438+0000","last_clean":"2024-09-12T02:00:09.574438+0000","last_became_active":"2024-09-12T01:59:55.492834+0000","last_became_peered":"2024-09-12T01:59:55.492834+0000","last_unstale":"2024-09-12T02:00:09.574438+0000","last_undegraded":"2024-09-12T02:00:09.574438+0000","last_fullsized":"2024-09-12T02:00:09.574438+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:58:43.872921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"80'3","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.621258+0000","last_change":"2024-09-12T01:59:53.466528+0000","last_active":"2024-09-12T02:00:09.621258+0000","last_peered":"2024-09-12T02:00:09.621258+0000","last_clean":"2024-09-12T02:00:09.621258+0000","last_became_active":"2024-09-12T01:59:53.466182+0000","last_became_peered":"2024-09-12T01:59:53.466182+0000","last_unstale":"2024-09-12T02:00:09.621258+0000","last_undegraded":"2024-09-12T02:00:09.621258+0000","last_fullsized":"2024-09-12T02:00:09.621258+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:00:26.083539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.316786+0000","last_change":"2024-09-12T01:59:55.494511+0000","last_active":"2024-09-12T02:01:38.316786+0000","last_peered":"2024-09-12T02:01:38.316786+0000","last_clean":"2024-09-12T02:01:38.316786+0000","last_became_active":"2024-09-12T01:59:55.493607+0000","last_became_peered":"2024-09-12T01:59:55.493607+0000","last_unstale":"2024-09-12T02:01:38.316786+0000","last_undegraded":"2024-09-12T02:01:38.316786+0000","last_fullsized":"2024-09-12T02:01:38.316786+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:55:27.434447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"80'1","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.569211+0000","last_change":"2024-09-12T01:59:53.467385+0000","last_active":"2024-09-12T02:00:09.569211+0000","last_peered":"2024-09-12T02:00:09.569211+0000","last_clean":"2024-09-12T02:00:09.569211+0000","last_became_active":"2024-09-12T01:59:53.466997+0000","last_became_peered":"2024-09-12T01:59:53.466997+0000","last_unstale":"2024-09-12T02:00:09.569211+0000","last_undegraded":"2024-09-12T02:00:09.569211+0000","last_fullsized":"2024-09-12T02:00:09.569211+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:54:47.154036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.621293+0000","last_change":"2024-09-12T01:59:55.490757+0000","last_active":"2024-09-12T02:00:09.621293+0000","last_peered":"2024-09-12T02:00:09.621293+0000","last_clean":"2024-09-12T02:00:09.621293+0000","last_became_active":"2024-09-12T01:59:55.490451+0000","last_became_peered":"2024-09-12T01:59:55.490451+0000","last_unstale":"2024-09-12T02:00:09.621293+0000","last_undegraded":"2024-09-12T02:00:09.621293+0000","last_fullsized":"2024-09-12T02:00:09.621293+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:21:26.139049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.574833+0000","last_change":"2024-09-12T01:59:53.472898+0000","last_active":"2024-09-12T02:00:09.574833+0000","last_peered":"2024-09-12T02:00:09.574833+0000","last_clean":"2024-09-12T02:00:09.574833+0000","last_became_active":"2024-09-12T01:59:53.472561+0000","last_became_peered":"2024-09-12T01:59:53.472561+0000","last_unstale":"2024-09-12T02:00:09.574833+0000","last_undegraded":"2024-09-12T02:00:09.574833+0000","last_fullsized":"2024-09-12T02:00:09.574833+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:55:59.933804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"82'5","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.328406+0000","last_change":"2024-09-12T01:59:55.497731+0000","last_active":"2024-09-12T02:01:38.328406+0000","last_peered":"2024-09-12T02:01:38.328406+0000","last_clean":"2024-09-12T02:01:38.328406+0000","last_became_active":"2024-09-12T01:59:55.497356+0000","last_became_peered":"2024-09-12T01:59:55.497356+0000","last_unstale":"2024-09-12T02:01:38.328406+0000","last_undegraded":"2024-09-12T02:01:38.328406+0000","last_fullsized":"2024-09-12T02:01:38.328406+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:57:18.795865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.624453+0000","last_change":"2024-09-12T02:00:09.624453+0000","last_active":"2024-09-12T02:00:09.624453+0000","last_peered":"2024-09-12T02:00:09.624453+0000","last_clean":"2024-09-12T02:00:09.624453+0000","last_became_active":"2024-09-12T02:00:09.624049+0000","last_became_peered":"2024-09-12T02:00:09.624049+0000","last_unstale":"2024-09-12T02:00:09.624453+0000","last_undegraded":"2024-09-12T02:00:09.624453+0000","last_fullsized":"2024-09-12T02:00:09.624453+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:17:40.417275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"80'2","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.575060+0000","last_change":"2024-09-12T01:59:53.470194+0000","last_active":"2024-09-12T02:00:09.575060+0000","last_peered":"2024-09-12T02:00:09.575060+0000","last_clean":"2024-09-12T02:00:09.575060+0000","last_became_active":"2024-09-12T01:59:53.469772+0000","last_became_peered":"2024-09-12T01:59:53.469772+0000","last_unstale":"2024-09-12T02:00:09.575060+0000","last_undegraded":"2024-09-12T02:00:09.575060+0000","last_fullsized":"2024-09-12T02:00:09.575060+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:52:57.709596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.622423+0000","last_change":"2024-09-12T01:59:55.497693+0000","last_active":"2024-09-12T02:00:09.622423+0000","last_peered":"2024-09-12T02:00:09.622423+0000","last_clean":"2024-09-12T02:00:09.622423+0000","last_became_active":"2024-09-12T01:59:55.497380+0000","last_became_peered":"2024-09-12T01:59:55.497380+0000","last_unstale":"2024-09-12T02:00:09.622423+0000","last_undegraded":"2024-09-12T02:00:09.622423+0000","last_fullsized":"2024-09-12T02:00:09.622423+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:42:27.943530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"80'4","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.575274+0000","last_change":"2024-09-12T01:59:53.473044+0000","last_active":"2024-09-12T02:00:09.575274+0000","last_peered":"2024-09-12T02:00:09.575274+0000","last_clean":"2024-09-12T02:00:09.575274+0000","last_became_active":"2024-09-12T01:59:53.472569+0000","last_became_peered":"2024-09-12T01:59:53.472569+0000","last_unstale":"2024-09-12T02:00:09.575274+0000","last_undegraded":"2024-09-12T02:00:09.575274+0000","last_fullsized":"2024-09-12T02:00:09.575274+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:41:29.716836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.328172+0000","last_change":"2024-09-12T01:59:55.497377+0000","last_active":"2024-09-12T02:01:38.328172+0000","last_peered":"2024-09-12T02:01:38.328172+0000","last_clean":"2024-09-12T02:01:38.328172+0000","last_became_active":"2024-09-12T01:59:55.497078+0000","last_became_peered":"2024-09-12T01:59:55.497078+0000","last_unstale":"2024-09-12T02:01:38.328172+0000","last_undegraded":"2024-09-12T02:01:38.328172+0000","last_fullsized":"2024-09-12T02:01:38.328172+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:59:57.362350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.621555+0000","last_change":"2024-09-12T01:59:53.920868+0000","last_active":"2024-09-12T02:00:09.621555+0000","last_peered":"2024-09-12T02:00:09.621555+0000","last_clean":"2024-09-12T02:00:09.621555+0000","last_became_active":"2024-09-12T01:59:53.920481+0000","last_became_peered":"2024-09-12T01:59:53.920481+0000","last_unstale":"2024-09-12T02:00:09.621555+0000","last_undegraded":"2024-09-12T02:00:09.621555+0000","last_fullsized":"2024-09-12T02:00:09.621555+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:49:27.085388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.621438+0000","last_change":"2024-09-12T01:59:55.497703+0000","last_active":"2024-09-12T02:00:09.621438+0000","last_peered":"2024-09-12T02:00:09.621438+0000","last_clean":"2024-09-12T02:00:09.621438+0000","last_became_active":"2024-09-12T01:59:55.497397+0000","last_became_peered":"2024-09-12T01:59:55.497397+0000","last_unstale":"2024-09-12T02:00:09.621438+0000","last_undegraded":"2024-09-12T02:00:09.621438+0000","last_fullsized":"2024-09-12T02:00:09.621438+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:54:02.803203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"80'9","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.565145+0000","last_change":"2024-09-12T01:59:53.469978+0000","last_active":"2024-09-12T02:00:09.565145+0000","last_peered":"2024-09-12T02:00:09.565145+0000","last_clean":"2024-09-12T02:00:09.565145+0000","last_became_active":"2024-09-12T01:59:53.469648+0000","last_became_peered":"2024-09-12T01:59:53.469648+0000","last_unstale":"2024-09-12T02:00:09.565145+0000","last_undegraded":"2024-09-12T02:00:09.565145+0000","last_fullsized":"2024-09-12T02:00:09.565145+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:08:37.720452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":293,"ondisk_log_size":293,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":68638,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":591,"num_read_kb":370,"num_write":169,"num_write_kb":118,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":421888,"data_stored":196621,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":143,"ondisk_log_size":143,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3563520,"data_stored":3542208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":74,"seq":317827579938,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32068,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713084,"statfs":{"total":95995035648,"available":95962198016,"internally_reserved":0,"allocated":897024,"data_stored":435916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":67,"seq":287762808871,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32736,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712416,"statfs":{"total":95995035648,"available":95961513984,"internally_reserved":0,"allocated":1589248,"data_stored":1091844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":9,"up_from":62,"seq":266287972396,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":905216,"data_stored":436040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201328,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":897024,"data_stored":435848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430262,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32648,"kb_used_data":1464,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712504,"statfs":{"total":95995035648,"available":95961604096,"internally_reserved":0,"allocated":1499136,"data_stored":1028033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593787,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36216,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708936,"statfs":{"total":95995035648,"available":95957950464,"internally_reserved":0,"allocated":892928,"data_stored":435429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822719,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32660,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712492,"statfs":{"total":95995035648,"available":95961591808,"internally_reserved":0,"allocated":1511424,"data_stored":1027576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986244,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":897024,"data_stored":435866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215176,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":909312,"data_stored":435740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":17,"seq":73014444110,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36300,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708852,"statfs":{"total":95995035648,"available":95957864448,"internally_reserved":0,"allocated":978944,"data_stored":496729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574931,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32116,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713036,"statfs":{"total":95995035648,"available":95962148864,"internally_reserved":0,"allocated":954368,"data_stored":494986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738457,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":913408,"data_stored":436700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":59764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":61815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":66213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T02:01:41.387 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:01:41.680 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:01:42.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:42.108+0000 7f955fdd3640 1 -- 172.21.15.16:0/2023688783 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9558103780 msgr2=0x7f9558107bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:42.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:42.108+0000 7f955fdd3640 1 --2- 172.21.15.16:0/2023688783 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9558103780 0x7f9558107bc0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f954c009930 tx=0x7f954c02a420 comp rx=0 tx=0).stop 2024-09-12T02:01:42.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:42.108+0000 7f955fdd3640 1 -- 172.21.15.16:0/2023688783 shutdown_connections 2024-09-12T02:01:42.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:42.108+0000 7f955fdd3640 1 --2- 172.21.15.16:0/2023688783 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9558108300 0x7f955810ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:42.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:42.108+0000 7f955fdd3640 1 --2- 172.21.15.16:0/2023688783 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9558103780 0x7f9558107bc0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:42.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:42.108+0000 7f955fdd3640 1 --2- 172.21.15.16:0/2023688783 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9558102dd0 0x7f95581031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:42.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:42.108+0000 7f955fdd3640 1 -- 172.21.15.16:0/2023688783 >> 172.21.15.16:0/2023688783 conn(0x7f95580fc7d0 msgr2=0x7f95580febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:01:42.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:42.108+0000 7f955fdd3640 1 -- 172.21.15.16:0/2023688783 shutdown_connections 2024-09-12T02:01:42.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:42.109+0000 7f955fdd3640 1 -- 172.21.15.16:0/2023688783 wait complete. 2024-09-12T02:01:42.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:42.109+0000 7f955fdd3640 1 Processor -- start 2024-09-12T02:01:42.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:42.109+0000 7f955fdd3640 1 -- start start 2024-09-12T02:01:42.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:42.109+0000 7f955fdd3640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9558102dd0 0x7f95581985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:42.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:42.109+0000 7f955fdd3640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9558103780 0x7f9558198b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:42.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:42.109+0000 7f955fdd3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9558108300 0x7f955819cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:42.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:42.110+0000 7f955fdd3640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9558112070 con 0x7f9558108300 2024-09-12T02:01:42.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:42.110+0000 7f955fdd3640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9558111ef0 con 0x7f9558102dd0 2024-09-12T02:01:42.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:42.110+0000 7f955fdd3640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f95581121f0 con 0x7f9558103780 2024-09-12T02:01:42.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:42.110+0000 7f955e370640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9558108300 0x7f955819cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:42.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:42.110+0000 7f955e370640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9558108300 0x7f955819cec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46532/0 (socket says 172.21.15.16:46532) 2024-09-12T02:01:42.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:42.110+0000 7f955e370640 1 -- 172.21.15.16:0/2820973412 learned_addr learned my addr 172.21.15.16:0/2820973412 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:01:42.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:42.110+0000 7f955db6f640 1 --2- 172.21.15.16:0/2820973412 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9558102dd0 0x7f95581985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:42.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:42.110+0000 7f955d36e640 1 --2- 172.21.15.16:0/2820973412 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9558103780 0x7f9558198b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:42.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:42.110+0000 7f955e370640 1 -- 172.21.15.16:0/2820973412 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9558103780 msgr2=0x7f9558198b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:42.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:42.110+0000 7f955e370640 1 --2- 172.21.15.16:0/2820973412 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9558103780 0x7f9558198b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:42.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:42.110+0000 7f955e370640 1 -- 172.21.15.16:0/2820973412 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9558102dd0 msgr2=0x7f95581985f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:42.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:42.110+0000 7f955e370640 1 --2- 172.21.15.16:0/2820973412 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9558102dd0 0x7f95581985f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:42.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:42.110+0000 7f955e370640 1 -- 172.21.15.16:0/2820973412 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f955819d5a0 con 0x7f9558108300 2024-09-12T02:01:42.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:42.110+0000 7f955db6f640 1 --2- 172.21.15.16:0/2820973412 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9558102dd0 0x7f95581985f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:01:42.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:42.110+0000 7f955e370640 1 --2- 172.21.15.16:0/2820973412 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9558108300 0x7f955819cec0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f955000d750 tx=0x7f955000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:01:42.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:42.110+0000 7f9556ffd640 1 -- 172.21.15.16:0/2820973412 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9550017070 con 0x7f9558108300 2024-09-12T02:01:42.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:42.111+0000 7f955fdd3640 1 -- 172.21.15.16:0/2820973412 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f955819d890 con 0x7f9558108300 2024-09-12T02:01:42.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:42.111+0000 7f9556ffd640 1 -- 172.21.15.16:0/2820973412 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9550005500 con 0x7f9558108300 2024-09-12T02:01:42.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:42.111+0000 7f9556ffd640 1 -- 172.21.15.16:0/2820973412 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9550004d90 con 0x7f9558108300 2024-09-12T02:01:42.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:42.111+0000 7f955fdd3640 1 -- 172.21.15.16:0/2820973412 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95581a5170 con 0x7f9558108300 2024-09-12T02:01:42.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:42.112+0000 7f955fdd3640 1 -- 172.21.15.16:0/2820973412 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95581031b0 con 0x7f9558108300 2024-09-12T02:01:42.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:42.113+0000 7f9556ffd640 1 -- 172.21.15.16:0/2820973412 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f9550005020 con 0x7f9558108300 2024-09-12T02:01:42.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:42.114+0000 7f9556ffd640 1 --2- 172.21.15.16:0/2820973412 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f95400a2c00 0x7f95400a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:42.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:42.114+0000 7f955db6f640 1 --2- 172.21.15.16:0/2820973412 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f95400a2c00 0x7f95400a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:42.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:42.114+0000 7f9556ffd640 1 -- 172.21.15.16:0/2820973412 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f95500ce870 con 0x7f9558108300 2024-09-12T02:01:42.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:42.114+0000 7f955db6f640 1 --2- 172.21.15.16:0/2820973412 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f95400a2c00 0x7f95400a50c0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f9548009fd0 tx=0x7f9548009680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:01:42.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:42.118+0000 7f9556ffd640 1 -- 172.21.15.16:0/2820973412 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f955009b180 con 0x7f9558108300 2024-09-12T02:01:42.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:42.330+0000 7f955fdd3640 1 -- 172.21.15.16:0/2820973412 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f955810a290 con 0x7f9558108300 2024-09-12T02:01:42.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:42.331+0000 7f9556ffd640 1 -- 172.21.15.16:0/2820973412 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11911 (secure 0 0 0) 0x7f955009f480 con 0x7f9558108300 2024-09-12T02:01:42.332 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T02:01:42.332 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":25,"btime":"2024-09-12T02:01:37:192021+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3279123279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3279123279},{"type":"v1","addr":"172.21.15.16:6835","nonce":3279123279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4119452382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4119452382},{"type":"v1","addr":"172.21.15.16:6837","nonce":4119452382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/3263823165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3263823165},{"type":"v1","addr":"172.21.15.16:6839","nonce":3263823165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/1510290732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":1510290732},{"type":"v1","addr":"172.21.15.16:6841","nonce":1510290732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/279674603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":279674603},{"type":"v1","addr":"172.21.15.33:6833","nonce":279674603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3555728028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3555728028},{"type":"v1","addr":"172.21.15.33:6835","nonce":3555728028}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T01:59:56.117608+0000","modified":"2024-09-12T02:01:36.294760+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24482,"mds_1":24481,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6833/1563374106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1563374106},{"type":"v1","addr":"172.21.15.184:6833","nonce":1563374106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6837/1892833868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1892833868},{"type":"v1","addr":"172.21.15.33:6837","nonce":1892833868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/1457679686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1457679686},{"type":"v1","addr":"172.21.15.184:6835","nonce":1457679686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.184:6837/1793393087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":1793393087},{"type":"v1","addr":"172.21.15.184:6837","nonce":1793393087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.184:6839/4203347941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":4203347941},{"type":"v1","addr":"172.21.15.184:6839","nonce":4203347941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.33:6839/4136670204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":4136670204},{"type":"v1","addr":"172.21.15.33:6839","nonce":4136670204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24481,24482,24469]},"id":1}]} 2024-09-12T02:01:42.332 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 25 2024-09-12T02:01:42.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:42.333+0000 7f955fdd3640 1 -- 172.21.15.16:0/2820973412 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f95400a2c00 msgr2=0x7f95400a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:42.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:42.333+0000 7f955fdd3640 1 --2- 172.21.15.16:0/2820973412 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f95400a2c00 0x7f95400a50c0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f9548009fd0 tx=0x7f9548009680 comp rx=0 tx=0).stop 2024-09-12T02:01:42.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:42.333+0000 7f955fdd3640 1 -- 172.21.15.16:0/2820973412 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9558108300 msgr2=0x7f955819cec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:42.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:42.333+0000 7f955fdd3640 1 --2- 172.21.15.16:0/2820973412 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9558108300 0x7f955819cec0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f955000d750 tx=0x7f955000dc10 comp rx=0 tx=0).stop 2024-09-12T02:01:42.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:42.333+0000 7f955fdd3640 1 -- 172.21.15.16:0/2820973412 shutdown_connections 2024-09-12T02:01:42.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:42.333+0000 7f955fdd3640 1 --2- 172.21.15.16:0/2820973412 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f95400a2c00 0x7f95400a50c0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:42.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:42.333+0000 7f955fdd3640 1 --2- 172.21.15.16:0/2820973412 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9558108300 0x7f955819cec0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:42.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:42.333+0000 7f955fdd3640 1 --2- 172.21.15.16:0/2820973412 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9558103780 0x7f9558198b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:42.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:42.333+0000 7f955fdd3640 1 --2- 172.21.15.16:0/2820973412 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9558102dd0 0x7f95581985f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:42.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:42.333+0000 7f955fdd3640 1 -- 172.21.15.16:0/2820973412 >> 172.21.15.16:0/2820973412 conn(0x7f95580fc7d0 msgr2=0x7f9558105ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:01:42.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:42.333+0000 7f955fdd3640 1 -- 172.21.15.16:0/2820973412 shutdown_connections 2024-09-12T02:01:42.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:42.333+0000 7f955fdd3640 1 -- 172.21.15.16:0/2820973412 wait complete. 2024-09-12T02:01:42.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:01:42 smithi016 ceph-mon[30947]: pgmap v328: 97 pgs: 97 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 938 B/s wr, 7 op/s 2024-09-12T02:01:42.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:01:42 smithi016 ceph-mon[30947]: from='client.15174 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T02:01:42.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:01:42 smithi016 ceph-mon[30947]: from='client.15180 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T02:01:42.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:01:42 smithi184 ceph-mon[35593]: pgmap v328: 97 pgs: 97 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 938 B/s wr, 7 op/s 2024-09-12T02:01:42.628 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:01:42 smithi184 ceph-mon[35593]: from='client.15174 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T02:01:42.628 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:01:42 smithi184 ceph-mon[35593]: from='client.15180 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T02:01:42.668 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:01:42 smithi033 ceph-mon[35332]: pgmap v328: 97 pgs: 97 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 938 B/s wr, 7 op/s 2024-09-12T02:01:42.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:01:42 smithi033 ceph-mon[35332]: from='client.15174 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T02:01:42.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:01:42 smithi033 ceph-mon[35332]: from='client.15180 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T02:01:42.943 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T02:01:43.202 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:01:43.467 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:01:43 smithi016 ceph-mon[30947]: from='client.15186 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T02:01:43.468 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:01:43 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/2820973412' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:01:43.628 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:01:43 smithi184 ceph-mon[35593]: from='client.15186 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T02:01:43.628 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:01:43 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/2820973412' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:01:43.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:01:43 smithi033 ceph-mon[35332]: from='client.15186 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T02:01:43.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:01:43 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/2820973412' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:01:43.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:43.664+0000 7f9a1fec6640 1 -- 172.21.15.16:0/1218207592 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a18101420 msgr2=0x7f9a1810d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:43.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:43.664+0000 7f9a1fec6640 1 --2- 172.21.15.16:0/1218207592 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a18101420 0x7f9a1810d950 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f9a10009cd0 tx=0x7f9a1002a8e0 comp rx=0 tx=0).stop 2024-09-12T02:01:43.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:43.664+0000 7f9a1fec6640 1 -- 172.21.15.16:0/1218207592 shutdown_connections 2024-09-12T02:01:43.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:43.664+0000 7f9a1fec6640 1 --2- 172.21.15.16:0/1218207592 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a18101420 0x7f9a1810d950 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:43.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:43.664+0000 7f9a1fec6640 1 --2- 172.21.15.16:0/1218207592 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9a18100a80 0x7f9a18100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:43.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:43.664+0000 7f9a1fec6640 1 --2- 172.21.15.16:0/1218207592 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a18106a80 0x7f9a18106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:43.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:43.664+0000 7f9a1fec6640 1 -- 172.21.15.16:0/1218207592 >> 172.21.15.16:0/1218207592 conn(0x7f9a180fc7a0 msgr2=0x7f9a180febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:01:43.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:43.664+0000 7f9a1fec6640 1 -- 172.21.15.16:0/1218207592 shutdown_connections 2024-09-12T02:01:43.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:43.664+0000 7f9a1fec6640 1 -- 172.21.15.16:0/1218207592 wait complete. 2024-09-12T02:01:43.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:43.664+0000 7f9a1fec6640 1 Processor -- start 2024-09-12T02:01:43.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:43.664+0000 7f9a1fec6640 1 -- start start 2024-09-12T02:01:43.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:43.665+0000 7f9a1fec6640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9a18100a80 0x7f9a1819a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:43.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:43.665+0000 7f9a1fec6640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a18101420 0x7f9a1819aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:43.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:43.665+0000 7f9a1fec6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a18106a80 0x7f9a1819ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:43.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:43.665+0000 7f9a1fec6640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9a18110e80 con 0x7f9a18106a80 2024-09-12T02:01:43.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:43.665+0000 7f9a1fec6640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9a18110d00 con 0x7f9a18101420 2024-09-12T02:01:43.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:43.665+0000 7f9a1fec6640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f9a18111000 con 0x7f9a18100a80 2024-09-12T02:01:43.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:43.665+0000 7f9a1e463640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a18106a80 0x7f9a1819ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:43.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:43.665+0000 7f9a1e463640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a18106a80 0x7f9a1819ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46546/0 (socket says 172.21.15.16:46546) 2024-09-12T02:01:43.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:43.665+0000 7f9a1e463640 1 -- 172.21.15.16:0/3215947739 learned_addr learned my addr 172.21.15.16:0/3215947739 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:01:43.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:43.665+0000 7f9a1dc62640 1 --2- 172.21.15.16:0/3215947739 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9a18100a80 0x7f9a1819a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:43.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:43.665+0000 7f9a1d461640 1 --2- 172.21.15.16:0/3215947739 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a18101420 0x7f9a1819aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:43.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:43.665+0000 7f9a1e463640 1 -- 172.21.15.16:0/3215947739 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9a18100a80 msgr2=0x7f9a1819a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:43.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:43.665+0000 7f9a1e463640 1 --2- 172.21.15.16:0/3215947739 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9a18100a80 0x7f9a1819a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:43.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:43.665+0000 7f9a1e463640 1 -- 172.21.15.16:0/3215947739 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a18101420 msgr2=0x7f9a1819aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:43.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:43.665+0000 7f9a1e463640 1 --2- 172.21.15.16:0/3215947739 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a18101420 0x7f9a1819aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:43.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:43.665+0000 7f9a1e463640 1 -- 172.21.15.16:0/3215947739 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a1819f560 con 0x7f9a18106a80 2024-09-12T02:01:43.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:43.665+0000 7f9a1e463640 1 --2- 172.21.15.16:0/3215947739 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a18106a80 0x7f9a1819ee80 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f9a10005cc0 tx=0x7f9a10003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:01:43.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:43.665+0000 7f9a0effd640 1 -- 172.21.15.16:0/3215947739 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a10004040 con 0x7f9a18106a80 2024-09-12T02:01:43.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:43.665+0000 7f9a1fec6640 1 -- 172.21.15.16:0/3215947739 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a1819f7f0 con 0x7f9a18106a80 2024-09-12T02:01:43.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:43.665+0000 7f9a0effd640 1 -- 172.21.15.16:0/3215947739 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a100046c0 con 0x7f9a18106a80 2024-09-12T02:01:43.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:43.665+0000 7f9a0effd640 1 -- 172.21.15.16:0/3215947739 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a10030e70 con 0x7f9a18106a80 2024-09-12T02:01:43.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:43.666+0000 7f9a1fec6640 1 -- 172.21.15.16:0/3215947739 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a181a7130 con 0x7f9a18106a80 2024-09-12T02:01:43.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:43.666+0000 7f9a1fec6640 1 -- 172.21.15.16:0/3215947739 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a18100ee0 con 0x7f9a18106a80 2024-09-12T02:01:43.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:43.668+0000 7f9a0effd640 1 -- 172.21.15.16:0/3215947739 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f9a100305a0 con 0x7f9a18106a80 2024-09-12T02:01:43.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:43.668+0000 7f9a0effd640 1 --2- 172.21.15.16:0/3215947739 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f9a040a2b80 0x7f9a040a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:43.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:43.669+0000 7f9a1dc62640 1 --2- 172.21.15.16:0/3215947739 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f9a040a2b80 0x7f9a040a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:43.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:43.669+0000 7f9a0effd640 1 -- 172.21.15.16:0/3215947739 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f9a1002e030 con 0x7f9a18106a80 2024-09-12T02:01:43.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:43.669+0000 7f9a1dc62640 1 --2- 172.21.15.16:0/3215947739 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f9a040a2b80 0x7f9a040a5040 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f9a1400ab60 tx=0x7f9a1400a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:01:43.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:43.672+0000 7f9a0effd640 1 -- 172.21.15.16:0/3215947739 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9a100bd600 con 0x7f9a18106a80 2024-09-12T02:01:43.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:43.848+0000 7f9a1fec6640 1 -- 172.21.15.16:0/3215947739 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9a181a7390 con 0x7f9a18106a80 2024-09-12T02:01:43.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:43.850+0000 7f9a0effd640 1 -- 172.21.15.16:0/3215947739 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+22584 (secure 0 0 0) 0x7f9a100098b0 con 0x7f9a18106a80 2024-09-12T02:01:43.851 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T02:01:43.851 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":82,"fsid":"879f3b56-70a9-11ef-bcea-c7b262605968","created":"2024-09-12T01:52:12.841045+0000","modified":"2024-09-12T02:00:09.559511+0000","last_up_change":"2024-09-12T01:59:08.975443+0000","last_in_change":"2024-09-12T01:58:44.530220+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T01:55:26.727564+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T01:59:52.402097+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T01:59:54.463627+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3ea41eb6-e384-4ddc-810f-0f103c127849","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4000187867},{"type":"v1","addr":"172.21.15.16:6803","nonce":4000187867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4000187867},{"type":"v1","addr":"172.21.15.16:6805","nonce":4000187867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":4000187867},{"type":"v1","addr":"172.21.15.16:6809","nonce":4000187867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4000187867},{"type":"v1","addr":"172.21.15.16:6807","nonce":4000187867}]},"public_addr":"172.21.15.16:6803/4000187867","cluster_addr":"172.21.15.16:6805/4000187867","heartbeat_back_addr":"172.21.15.16:6809/4000187867","heartbeat_front_addr":"172.21.15.16:6807/4000187867","state":["exists","up"]},{"osd":1,"uuid":"07bf63e7-70fa-41b2-af53-12a5fec3f33c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":3862661148},{"type":"v1","addr":"172.21.15.33:6801","nonce":3862661148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":3862661148},{"type":"v1","addr":"172.21.15.33:6803","nonce":3862661148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":3862661148},{"type":"v1","addr":"172.21.15.33:6807","nonce":3862661148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":3862661148},{"type":"v1","addr":"172.21.15.33:6805","nonce":3862661148}]},"public_addr":"172.21.15.33:6801/3862661148","cluster_addr":"172.21.15.33:6803/3862661148","heartbeat_back_addr":"172.21.15.33:6807/3862661148","heartbeat_front_addr":"172.21.15.33:6805/3862661148","state":["exists","up"]},{"osd":2,"uuid":"22e40cac-3bc0-4cce-b47b-77499b767f7e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":3958753963},{"type":"v1","addr":"172.21.15.184:6801","nonce":3958753963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":3958753963},{"type":"v1","addr":"172.21.15.184:6803","nonce":3958753963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":3958753963},{"type":"v1","addr":"172.21.15.184:6807","nonce":3958753963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":3958753963},{"type":"v1","addr":"172.21.15.184:6805","nonce":3958753963}]},"public_addr":"172.21.15.184:6801/3958753963","cluster_addr":"172.21.15.184:6803/3958753963","heartbeat_back_addr":"172.21.15.184:6807/3958753963","heartbeat_front_addr":"172.21.15.184:6805/3958753963","state":["exists","up"]},{"osd":3,"uuid":"834a07f5-2c24-49e1-8e4a-12c96cfbdc65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3395182524},{"type":"v1","addr":"172.21.15.16:6811","nonce":3395182524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3395182524},{"type":"v1","addr":"172.21.15.16:6813","nonce":3395182524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3395182524},{"type":"v1","addr":"172.21.15.16:6817","nonce":3395182524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3395182524},{"type":"v1","addr":"172.21.15.16:6815","nonce":3395182524}]},"public_addr":"172.21.15.16:6811/3395182524","cluster_addr":"172.21.15.16:6813/3395182524","heartbeat_back_addr":"172.21.15.16:6817/3395182524","heartbeat_front_addr":"172.21.15.16:6815/3395182524","state":["exists","up"]},{"osd":4,"uuid":"3ca56e09-c194-4404-b9bd-2524d38367eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":1355184322},{"type":"v1","addr":"172.21.15.33:6809","nonce":1355184322}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1355184322},{"type":"v1","addr":"172.21.15.33:6811","nonce":1355184322}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1355184322},{"type":"v1","addr":"172.21.15.33:6815","nonce":1355184322}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1355184322},{"type":"v1","addr":"172.21.15.33:6813","nonce":1355184322}]},"public_addr":"172.21.15.33:6809/1355184322","cluster_addr":"172.21.15.33:6811/1355184322","heartbeat_back_addr":"172.21.15.33:6815/1355184322","heartbeat_front_addr":"172.21.15.33:6813/1355184322","state":["exists","up"]},{"osd":5,"uuid":"0b05ecdb-b987-461f-951c-6d76739bfb4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":1533397276},{"type":"v1","addr":"172.21.15.184:6809","nonce":1533397276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":1533397276},{"type":"v1","addr":"172.21.15.184:6811","nonce":1533397276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":1533397276},{"type":"v1","addr":"172.21.15.184:6815","nonce":1533397276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":1533397276},{"type":"v1","addr":"172.21.15.184:6813","nonce":1533397276}]},"public_addr":"172.21.15.184:6809/1533397276","cluster_addr":"172.21.15.184:6811/1533397276","heartbeat_back_addr":"172.21.15.184:6815/1533397276","heartbeat_front_addr":"172.21.15.184:6813/1533397276","state":["exists","up"]},{"osd":6,"uuid":"b3936e21-e197-49fa-8441-d909b258b612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":902466142},{"type":"v1","addr":"172.21.15.16:6819","nonce":902466142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":902466142},{"type":"v1","addr":"172.21.15.16:6821","nonce":902466142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":902466142},{"type":"v1","addr":"172.21.15.16:6825","nonce":902466142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":902466142},{"type":"v1","addr":"172.21.15.16:6823","nonce":902466142}]},"public_addr":"172.21.15.16:6819/902466142","cluster_addr":"172.21.15.16:6821/902466142","heartbeat_back_addr":"172.21.15.16:6825/902466142","heartbeat_front_addr":"172.21.15.16:6823/902466142","state":["exists","up"]},{"osd":7,"uuid":"4ddbe011-34c1-4b3f-a8eb-577422fb5426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":4186032630},{"type":"v1","addr":"172.21.15.33:6817","nonce":4186032630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":4186032630},{"type":"v1","addr":"172.21.15.33:6819","nonce":4186032630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":4186032630},{"type":"v1","addr":"172.21.15.33:6823","nonce":4186032630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":4186032630},{"type":"v1","addr":"172.21.15.33:6821","nonce":4186032630}]},"public_addr":"172.21.15.33:6817/4186032630","cluster_addr":"172.21.15.33:6819/4186032630","heartbeat_back_addr":"172.21.15.33:6823/4186032630","heartbeat_front_addr":"172.21.15.33:6821/4186032630","state":["exists","up"]},{"osd":8,"uuid":"dca5bfb8-ac6a-47ce-905c-be7fcaf0b5ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":781851781},{"type":"v1","addr":"172.21.15.184:6817","nonce":781851781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":781851781},{"type":"v1","addr":"172.21.15.184:6819","nonce":781851781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":781851781},{"type":"v1","addr":"172.21.15.184:6823","nonce":781851781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":781851781},{"type":"v1","addr":"172.21.15.184:6821","nonce":781851781}]},"public_addr":"172.21.15.184:6817/781851781","cluster_addr":"172.21.15.184:6819/781851781","heartbeat_back_addr":"172.21.15.184:6823/781851781","heartbeat_front_addr":"172.21.15.184:6821/781851781","state":["exists","up"]},{"osd":9,"uuid":"23a05cb5-01d5-40f3-b66a-15fcccedbd6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":1569578746},{"type":"v1","addr":"172.21.15.16:6827","nonce":1569578746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":1569578746},{"type":"v1","addr":"172.21.15.16:6829","nonce":1569578746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1569578746},{"type":"v1","addr":"172.21.15.16:6833","nonce":1569578746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":1569578746},{"type":"v1","addr":"172.21.15.16:6831","nonce":1569578746}]},"public_addr":"172.21.15.16:6827/1569578746","cluster_addr":"172.21.15.16:6829/1569578746","heartbeat_back_addr":"172.21.15.16:6833/1569578746","heartbeat_front_addr":"172.21.15.16:6831/1569578746","state":["exists","up"]},{"osd":10,"uuid":"e4bd6bce-3c69-44db-bc09-d264a0ad094a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":469977847},{"type":"v1","addr":"172.21.15.33:6825","nonce":469977847}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":469977847},{"type":"v1","addr":"172.21.15.33:6827","nonce":469977847}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":469977847},{"type":"v1","addr":"172.21.15.33:6831","nonce":469977847}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":469977847},{"type":"v1","addr":"172.21.15.33:6829","nonce":469977847}]},"public_addr":"172.21.15.33:6825/469977847","cluster_addr":"172.21.15.33:6827/469977847","heartbeat_back_addr":"172.21.15.33:6831/469977847","heartbeat_front_addr":"172.21.15.33:6829/469977847","state":["exists","up"]},{"osd":11,"uuid":"d60212f8-5a1d-45f7-9a1b-12294bb4c80d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":1619871346},{"type":"v1","addr":"172.21.15.184:6825","nonce":1619871346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":1619871346},{"type":"v1","addr":"172.21.15.184:6827","nonce":1619871346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":1619871346},{"type":"v1","addr":"172.21.15.184:6831","nonce":1619871346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":1619871346},{"type":"v1","addr":"172.21.15.184:6829","nonce":1619871346}]},"public_addr":"172.21.15.184:6825/1619871346","cluster_addr":"172.21.15.184:6827/1619871346","heartbeat_back_addr":"172.21.15.184:6831/1619871346","heartbeat_front_addr":"172.21.15.184:6829/1619871346","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:54:32.057979+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:54:57.501562+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:55:23.137225+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:55:48.646155+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:56:13.311907+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:56:38.214188+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:57:02.570921+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:57:27.506097+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:57:52.038529+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:58:17.423300+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:58:41.579992+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:59:06.245817+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:0/3532565698":"2024-09-13T01:53:08.218475+0000","172.21.15.16:0/1822157179":"2024-09-13T01:52:44.194152+0000","172.21.15.16:0/3919664061":"2024-09-13T01:52:44.194152+0000","172.21.15.16:0/2269394774":"2024-09-13T01:52:44.194152+0000","172.21.15.16:0/2595308634":"2024-09-13T01:53:08.218475+0000","172.21.15.16:6801/2754331153":"2024-09-13T01:52:44.194152+0000","172.21.15.16:6801/2615861334":"2024-09-13T01:53:08.218475+0000","172.21.15.16:6800/2615861334":"2024-09-13T01:53:08.218475+0000","172.21.15.16:0/4199114771":"2024-09-13T01:53:08.218475+0000","172.21.15.16:6800/2754331153":"2024-09-13T01:52:44.194152+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T02:01:43.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:43.852+0000 7f9a1fec6640 1 -- 172.21.15.16:0/3215947739 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f9a040a2b80 msgr2=0x7f9a040a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:43.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:43.852+0000 7f9a1fec6640 1 --2- 172.21.15.16:0/3215947739 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f9a040a2b80 0x7f9a040a5040 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f9a1400ab60 tx=0x7f9a1400a2d0 comp rx=0 tx=0).stop 2024-09-12T02:01:43.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:43.852+0000 7f9a1fec6640 1 -- 172.21.15.16:0/3215947739 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a18106a80 msgr2=0x7f9a1819ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:43.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:43.852+0000 7f9a1fec6640 1 --2- 172.21.15.16:0/3215947739 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a18106a80 0x7f9a1819ee80 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f9a10005cc0 tx=0x7f9a10003970 comp rx=0 tx=0).stop 2024-09-12T02:01:43.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:43.852+0000 7f9a1fec6640 1 -- 172.21.15.16:0/3215947739 shutdown_connections 2024-09-12T02:01:43.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:43.852+0000 7f9a1fec6640 1 --2- 172.21.15.16:0/3215947739 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f9a040a2b80 0x7f9a040a5040 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:43.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:43.852+0000 7f9a1fec6640 1 --2- 172.21.15.16:0/3215947739 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a18106a80 0x7f9a1819ee80 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:43.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:43.852+0000 7f9a1fec6640 1 --2- 172.21.15.16:0/3215947739 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a18101420 0x7f9a1819aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:43.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:43.852+0000 7f9a1fec6640 1 --2- 172.21.15.16:0/3215947739 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9a18100a80 0x7f9a1819a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:43.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:43.853+0000 7f9a1fec6640 1 -- 172.21.15.16:0/3215947739 >> 172.21.15.16:0/3215947739 conn(0x7f9a180fc7a0 msgr2=0x7f9a1810c670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:01:43.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:43.853+0000 7f9a1fec6640 1 -- 172.21.15.16:0/3215947739 shutdown_connections 2024-09-12T02:01:43.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:43.853+0000 7f9a1fec6640 1 -- 172.21.15.16:0/3215947739 wait complete. 2024-09-12T02:01:44.469 DEBUG:tasks.fwd_scrub:Yielding 2024-09-12T02:01:44.469 INFO:teuthology.run_tasks:Running task background_exec... 2024-09-12T02:01:44.479 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi016.front.sepia.ceph.com 2024-09-12T02:01:44.479 DEBUG:teuthology.orchestra.run.smithi016:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-09-12T02:01:44.479 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-09-12T02:01:44.479 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-12T02:01:44.480 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-12T02:01:44.481 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:01:44 smithi016 ceph-mon[30947]: pgmap v329: 97 pgs: 97 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1023 B/s wr, 6 op/s 2024-09-12T02:01:44.481 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:01:44 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/3215947739' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T02:01:44.483 INFO:teuthology.run_tasks:Running task quiescer... 2024-09-12T02:01:44.660 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-12T02:01:44.660 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-12T02:01:44.661 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:01:44 smithi184 ceph-mon[35593]: pgmap v329: 97 pgs: 97 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1023 B/s wr, 6 op/s 2024-09-12T02:01:44.662 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:01:44 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/3215947739' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T02:01:44.668 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:01:44 smithi033 ceph-mon[35332]: pgmap v329: 97 pgs: 97 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1023 B/s wr, 6 op/s 2024-09-12T02:01:44.668 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:01:44 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/3215947739' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T02:01:44.755 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:01:45.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.369+0000 7f2e68ecb640 1 -- 172.21.15.16:0/3648466632 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e64106a60 msgr2=0x7f2e64106e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:45.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.369+0000 7f2e68ecb640 1 --2- 172.21.15.16:0/3648466632 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e64106a60 0x7f2e64106e40 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f2e54009930 tx=0x7f2e5402a420 comp rx=0 tx=0).stop 2024-09-12T02:01:45.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.369+0000 7f2e68ecb640 1 -- 172.21.15.16:0/3648466632 shutdown_connections 2024-09-12T02:01:45.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.369+0000 7f2e68ecb640 1 --2- 172.21.15.16:0/3648466632 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2e64101400 0x7f2e6410d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:45.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.369+0000 7f2e68ecb640 1 --2- 172.21.15.16:0/3648466632 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2e64100a60 0x7f2e64100ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:45.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.369+0000 7f2e68ecb640 1 --2- 172.21.15.16:0/3648466632 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e64106a60 0x7f2e64106e40 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:45.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.369+0000 7f2e68ecb640 1 -- 172.21.15.16:0/3648466632 >> 172.21.15.16:0/3648466632 conn(0x7f2e640fc780 msgr2=0x7f2e640feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:01:45.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.369+0000 7f2e68ecb640 1 -- 172.21.15.16:0/3648466632 shutdown_connections 2024-09-12T02:01:45.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.369+0000 7f2e68ecb640 1 -- 172.21.15.16:0/3648466632 wait complete. 2024-09-12T02:01:45.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.370+0000 7f2e68ecb640 1 Processor -- start 2024-09-12T02:01:45.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.370+0000 7f2e68ecb640 1 -- start start 2024-09-12T02:01:45.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.370+0000 7f2e68ecb640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2e64100a60 0x7f2e640fce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:45.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.370+0000 7f2e68ecb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e64101400 0x7f2e640fd350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:45.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.370+0000 7f2e68ecb640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2e64106a60 0x7f2e640fd890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:45.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.370+0000 7f2e68ecb640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2e64110e80 con 0x7f2e64101400 2024-09-12T02:01:45.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.370+0000 7f2e68ecb640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2e64110d00 con 0x7f2e64106a60 2024-09-12T02:01:45.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.370+0000 7f2e68ecb640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f2e64111000 con 0x7f2e64100a60 2024-09-12T02:01:45.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.370+0000 7f2e6259c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2e64100a60 0x7f2e640fce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:45.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.370+0000 7f2e6259c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2e64100a60 0x7f2e640fce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:49122/0 (socket says 172.21.15.16:49122) 2024-09-12T02:01:45.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.370+0000 7f2e6259c640 1 -- 172.21.15.16:0/4129447548 learned_addr learned my addr 172.21.15.16:0/4129447548 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:01:45.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.370+0000 7f2e62d9d640 1 --2- 172.21.15.16:0/4129447548 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2e64106a60 0x7f2e640fd890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:45.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.370+0000 7f2e5bfff640 1 --2- 172.21.15.16:0/4129447548 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e64101400 0x7f2e640fd350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:45.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.370+0000 7f2e6259c640 1 -- 172.21.15.16:0/4129447548 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2e64106a60 msgr2=0x7f2e640fd890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:45.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.370+0000 7f2e6259c640 1 --2- 172.21.15.16:0/4129447548 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2e64106a60 0x7f2e640fd890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:45.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.370+0000 7f2e6259c640 1 -- 172.21.15.16:0/4129447548 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e64101400 msgr2=0x7f2e640fd350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:45.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.370+0000 7f2e6259c640 1 --2- 172.21.15.16:0/4129447548 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e64101400 0x7f2e640fd350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:45.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.370+0000 7f2e6259c640 1 -- 172.21.15.16:0/4129447548 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e641a6930 con 0x7f2e64100a60 2024-09-12T02:01:45.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.371+0000 7f2e5bfff640 1 --2- 172.21.15.16:0/4129447548 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e64101400 0x7f2e640fd350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:01:45.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.371+0000 7f2e6259c640 1 --2- 172.21.15.16:0/4129447548 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2e64100a60 0x7f2e640fce10 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f2e54009230 tx=0x7f2e54004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:01:45.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.371+0000 7f2e5b7fe640 1 -- 172.21.15.16:0/4129447548 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e5403f030 con 0x7f2e64100a60 2024-09-12T02:01:45.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.371+0000 7f2e68ecb640 1 -- 172.21.15.16:0/4129447548 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e641a6bc0 con 0x7f2e64100a60 2024-09-12T02:01:45.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.371+0000 7f2e5b7fe640 1 -- 172.21.15.16:0/4129447548 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e54003ad0 con 0x7f2e64100a60 2024-09-12T02:01:45.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.371+0000 7f2e5b7fe640 1 -- 172.21.15.16:0/4129447548 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e54035070 con 0x7f2e64100a60 2024-09-12T02:01:45.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.371+0000 7f2e68ecb640 1 -- 172.21.15.16:0/4129447548 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e641a70a0 con 0x7f2e64100a60 2024-09-12T02:01:45.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.374+0000 7f2e5b7fe640 1 -- 172.21.15.16:0/4129447548 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f2e54042030 con 0x7f2e64100a60 2024-09-12T02:01:45.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.374+0000 7f2e6b2a4740 1 Processor -- start 2024-09-12T02:01:45.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.374+0000 7f2e6b2a4740 1 -- start start 2024-09-12T02:01:45.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.374+0000 7f2e6b2a4740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564c1cc23b60 0x564c1cc23f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:45.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.374+0000 7f2e6b2a4740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564c1cc24480 0x564c1cc2e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:45.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.375+0000 7f2e6b2a4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c1cc2ed40 0x564c1cc31130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:45.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.375+0000 7f2e6b2a4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x564c1cb89280 con 0x564c1cc2ed40 2024-09-12T02:01:45.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.375+0000 7f2e6b2a4740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x564c1cb30b10 con 0x564c1cc24480 2024-09-12T02:01:45.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.375+0000 7f2e6b2a4740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x564c1cb85240 con 0x564c1cc23b60 2024-09-12T02:01:45.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.375+0000 7f2e5bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c1cc2ed40 0x564c1cc31130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:45.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.375+0000 7f2e5bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c1cc2ed40 0x564c1cc31130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46560/0 (socket says 172.21.15.16:46560) 2024-09-12T02:01:45.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.375+0000 7f2e62d9d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564c1cc23b60 0x564c1cc23f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:45.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.375+0000 7f2e6259c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564c1cc24480 0x564c1cc2e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:45.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.375+0000 7f2e5bfff640 1 -- 172.21.15.16:0/1775095721 learned_addr learned my addr 172.21.15.16:0/1775095721 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:01:45.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.375+0000 7f2e5bfff640 1 -- 172.21.15.16:0/1775095721 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564c1cc23b60 msgr2=0x564c1cc23f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:01:45.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.375+0000 7f2e5bfff640 1 --2- 172.21.15.16:0/1775095721 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564c1cc23b60 0x564c1cc23f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:45.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.375+0000 7f2e5bfff640 1 -- 172.21.15.16:0/1775095721 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564c1cc24480 msgr2=0x564c1cc2e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:01:45.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.375+0000 7f2e5b7fe640 1 --2- 172.21.15.16:0/4129447548 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f2e500a2b80 0x7f2e500a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:45.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.375+0000 7f2e5bfff640 1 --2- 172.21.15.16:0/1775095721 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564c1cc24480 0x564c1cc2e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:45.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.375+0000 7f2e5bfff640 1 -- 172.21.15.16:0/1775095721 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c1cb3d1f0 con 0x564c1cc2ed40 2024-09-12T02:01:45.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.375+0000 7f2e6259c640 1 --2- 172.21.15.16:0/1775095721 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564c1cc24480 0x564c1cc2e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:01:45.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.375+0000 7f2e62d9d640 1 --2- 172.21.15.16:0/4129447548 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f2e500a2b80 0x7f2e500a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:45.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.375+0000 7f2e5b7fe640 1 -- 172.21.15.16:0/4129447548 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f2e540ef3f0 con 0x7f2e64100a60 2024-09-12T02:01:45.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.375+0000 7f2e5bfff640 1 --2- 172.21.15.16:0/1775095721 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c1cc2ed40 0x564c1cc31130 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c00bfd0 tx=0x7f2e4c0108e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6f88a5992a799d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:01:45.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.376+0000 7f2e58ff9640 1 -- 172.21.15.16:0/1775095721 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e4c005220 con 0x564c1cc2ed40 2024-09-12T02:01:45.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.376+0000 7f2e58ff9640 1 -- 172.21.15.16:0/1775095721 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e4c004c80 con 0x564c1cc2ed40 2024-09-12T02:01:45.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.376+0000 7f2e58ff9640 1 -- 172.21.15.16:0/1775095721 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e4c011c40 con 0x564c1cc2ed40 2024-09-12T02:01:45.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.376+0000 7f2e6b2a4740 1 -- 172.21.15.16:0/1775095721 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c1cc2ed40 msgr2=0x564c1cc31130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:45.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.376+0000 7f2e6b2a4740 1 --2- 172.21.15.16:0/1775095721 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c1cc2ed40 0x564c1cc31130 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c00bfd0 tx=0x7f2e4c0108e0 comp rx=0 tx=0).stop 2024-09-12T02:01:45.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.376+0000 7f2e62d9d640 1 --2- 172.21.15.16:0/4129447548 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f2e500a2b80 0x7f2e500a5040 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2e5c009fd0 tx=0x7f2e5c00d040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:01:45.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.376+0000 7f2e6b2a4740 1 -- 172.21.15.16:0/1775095721 shutdown_connections 2024-09-12T02:01:45.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.376+0000 7f2e6b2a4740 1 --2- 172.21.15.16:0/1775095721 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c1cc2ed40 0x564c1cc31130 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:45.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.376+0000 7f2e6b2a4740 1 --2- 172.21.15.16:0/1775095721 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564c1cc24480 0x564c1cc2e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:45.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.376+0000 7f2e6b2a4740 1 --2- 172.21.15.16:0/1775095721 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564c1cc23b60 0x564c1cc23f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:45.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.376+0000 7f2e6b2a4740 1 -- 172.21.15.16:0/1775095721 >> 172.21.15.16:0/1775095721 conn(0x564c1cc200f0 msgr2=0x564c1cc204d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:01:45.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.376+0000 7f2e6b2a4740 1 -- 172.21.15.16:0/1775095721 shutdown_connections 2024-09-12T02:01:45.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.376+0000 7f2e6b2a4740 1 -- 172.21.15.16:0/1775095721 wait complete. 2024-09-12T02:01:45.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.377+0000 7f2e6b2a4740 1 -- start start 2024-09-12T02:01:45.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.377+0000 7f2e6b2a4740 1 Processor -- start 2024-09-12T02:01:45.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.377+0000 7f2e6b2a4740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564c1cc23b60 0x564c1cc2bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:45.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.377+0000 7f2e6b2a4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c1cc24480 0x564c1cc2c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:45.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.377+0000 7f2e6b2a4740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564c1cc2ed40 0x564c1cc2c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:45.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.377+0000 7f2e6b2a4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x564c1cc31910 con 0x564c1cc24480 2024-09-12T02:01:45.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.377+0000 7f2e6b2a4740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x564c1cb43f10 con 0x564c1cc23b60 2024-09-12T02:01:45.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.377+0000 7f2e6b2a4740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x564c1cc31a90 con 0x564c1cc2ed40 2024-09-12T02:01:45.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.377+0000 7f2e6b2a4740 10 client.0 _ll_register_callbacks cb 0x564c1cb0ca40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:01:45.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.377+0000 7f2e5bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c1cc24480 0x564c1cc2c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:45.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.377+0000 7f2e5bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c1cc24480 0x564c1cc2c1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46572/0 (socket says 172.21.15.16:46572) 2024-09-12T02:01:45.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.377+0000 7f2e5bfff640 1 -- 172.21.15.16:0/1416475663 learned_addr learned my addr 172.21.15.16:0/1416475663 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:01:45.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.377+0000 7f2e62d9d640 1 --2- 172.21.15.16:0/1416475663 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564c1cc2ed40 0x564c1cc2c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:45.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.377+0000 7f2e6259c640 1 --2- 172.21.15.16:0/1416475663 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564c1cc23b60 0x564c1cc2bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:45.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.378+0000 7f2e5bfff640 1 -- 172.21.15.16:0/1416475663 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564c1cc2ed40 msgr2=0x564c1cc2c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:01:45.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.378+0000 7f2e5bfff640 1 --2- 172.21.15.16:0/1416475663 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564c1cc2ed40 0x564c1cc2c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:45.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.378+0000 7f2e5bfff640 1 -- 172.21.15.16:0/1416475663 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564c1cc23b60 msgr2=0x564c1cc2bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:01:45.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.378+0000 7f2e5bfff640 1 --2- 172.21.15.16:0/1416475663 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564c1cc23b60 0x564c1cc2bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:45.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.378+0000 7f2e5bfff640 1 -- 172.21.15.16:0/1416475663 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564c1c8fa6a0 con 0x564c1cc24480 2024-09-12T02:01:45.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.378+0000 7f2e6259c640 1 --2- 172.21.15.16:0/1416475663 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564c1cc23b60 0x564c1cc2bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:01:45.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.378+0000 7f2e5bfff640 1 --2- 172.21.15.16:0/1416475663 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c1cc24480 0x564c1cc2c1f0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c0048f0 tx=0x7f2e4c004920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=704e9841aea342e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:01:45.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.378+0000 7f2e49ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:01:45.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.378+0000 7f2e49ffb640 1 -- 172.21.15.16:0/1416475663 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e4c004ac0 con 0x564c1cc24480 2024-09-12T02:01:45.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.378+0000 7f2e49ffb640 1 -- 172.21.15.16:0/1416475663 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f2e4c005210 con 0x564c1cc24480 2024-09-12T02:01:45.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.378+0000 7f2e6b2a4740 1 -- 172.21.15.16:0/1416475663 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e4c004c80 con 0x564c1cc24480 2024-09-12T02:01:45.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.378+0000 7f2e49ffb640 1 -- 172.21.15.16:0/1416475663 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2e4c00bc50 con 0x564c1cc24480 2024-09-12T02:01:45.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.378+0000 7f2e6b2a4740 10 client.15210 fetch_fsmap learned FSMap version 25 2024-09-12T02:01:45.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.378+0000 7f2e6b2a4740 1 -- 172.21.15.16:0/1416475663 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x564c1cc3d630 con 0x564c1cc24480 2024-09-12T02:01:45.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.378+0000 7f2e49ffb640 1 -- 172.21.15.16:0/1416475663 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e4c004c80 con 0x564c1cc24480 2024-09-12T02:01:45.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.378+0000 7f2e49ffb640 1 -- 172.21.15.16:0/1416475663 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f2e4c01b730 con 0x564c1cc24480 2024-09-12T02:01:45.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.379+0000 7f2e6b2a4740 10 client.15210 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:01:45.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.379+0000 7f2e6b2a4740 10 client.15210 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:01:45.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.379+0000 7f2e6b2a4740 20 client.15210 populate_metadata read hostname 'smithi016' 2024-09-12T02:01:45.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.379+0000 7f2e6b2a4740 1 --2- 172.21.15.16:0/1416475663 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x564c1cc3ea40 0x564c1cc40e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:45.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.379+0000 7f2e6b2a4740 4 client.15210 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:01:45.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.379+0000 7f2e6b2a4740 1 -- 172.21.15.16:0/1416475663 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564c1cb967f0 con 0x564c1cc3ea40 2024-09-12T02:01:45.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.379+0000 7f2e6259c640 1 --2- 172.21.15.16:0/1416475663 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x564c1cc3ea40 0x564c1cc40e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:45.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.379+0000 7f2e6259c640 1 --2- 172.21.15.16:0/1416475663 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x564c1cc3ea40 0x564c1cc40e30 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=13d076fe6726f70b server_cookie=a7a3d8cdcd508d0c in_seq=0 out_seq=0 2024-09-12T02:01:45.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.379+0000 7f2e49ffb640 10 client.15210 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:01:45.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.380+0000 7f2e49ffb640 1 -- 172.21.15.16:0/1416475663 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564c1cb967f0 con 0x564c1cc3ea40 2024-09-12T02:01:45.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.380+0000 7f2e49ffb640 10 client.15210 handle_command_reply: tid=0 2024-09-12T02:01:45.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.380+0000 7f2e6b2a4740 1 client.15210 shutdown 2024-09-12T02:01:45.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.380+0000 7f2e6b2a4740 1 -- 172.21.15.16:0/1416475663 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c1cc24480 msgr2=0x564c1cc2c1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:45.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.380+0000 7f2e6b2a4740 1 --2- 172.21.15.16:0/1416475663 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c1cc24480 0x564c1cc2c1f0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c0048f0 tx=0x7f2e4c004920 comp rx=0 tx=0).stop 2024-09-12T02:01:45.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.380+0000 7f2e6b2a4740 1 -- 172.21.15.16:0/1416475663 shutdown_connections 2024-09-12T02:01:45.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.380+0000 7f2e6b2a4740 1 --2- 172.21.15.16:0/1416475663 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x564c1cc3ea40 0x564c1cc40e30 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:45.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.380+0000 7f2e6b2a4740 1 --2- 172.21.15.16:0/1416475663 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564c1cc2ed40 0x564c1cc2c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:45.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.380+0000 7f2e6b2a4740 1 --2- 172.21.15.16:0/1416475663 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c1cc24480 0x564c1cc2c1f0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:45.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.380+0000 7f2e6b2a4740 1 --2- 172.21.15.16:0/1416475663 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564c1cc23b60 0x564c1cc2bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:45.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.380+0000 7f2e6b2a4740 1 -- 172.21.15.16:0/1416475663 >> 172.21.15.16:0/1416475663 conn(0x564c1cc200f0 msgr2=0x564c1cc25dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:01:45.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.380+0000 7f2e49ffb640 0 client.15210 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:01:45.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.380+0000 7f2e6b2a4740 1 -- 172.21.15.16:0/1416475663 shutdown_connections 2024-09-12T02:01:45.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.380+0000 7f2e6b2a4740 1 -- 172.21.15.16:0/1416475663 wait complete. 2024-09-12T02:01:45.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.380+0000 7f2e6b2a4740 20 client.15210 trim_cache size 0 max 16384 2024-09-12T02:01:45.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.404+0000 7f2e6b2a4740 1 Processor -- start 2024-09-12T02:01:45.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.404+0000 7f2e6b2a4740 1 -- start start 2024-09-12T02:01:45.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.404+0000 7f2e6b2a4740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564c1cc23b60 0x564c1cc465c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:45.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.404+0000 7f2e6b2a4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c1cc24480 0x564c1cc46b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:45.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.404+0000 7f2e6b2a4740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564c1cc2ed40 0x564c1cc2d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:45.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.404+0000 7f2e6b2a4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2e641b1e40 con 0x564c1cc24480 2024-09-12T02:01:45.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.404+0000 7f2e6b2a4740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2e641af710 con 0x564c1cc2ed40 2024-09-12T02:01:45.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.404+0000 7f2e6b2a4740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f2e641ad560 con 0x564c1cc23b60 2024-09-12T02:01:45.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.404+0000 7f2e5bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c1cc24480 0x564c1cc46b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:45.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.404+0000 7f2e5bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c1cc24480 0x564c1cc46b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46588/0 (socket says 172.21.15.16:46588) 2024-09-12T02:01:45.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.404+0000 7f2e5bfff640 1 -- 172.21.15.16:0/3029841218 learned_addr learned my addr 172.21.15.16:0/3029841218 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:01:45.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.404+0000 7f2e6259c640 1 --2- 172.21.15.16:0/3029841218 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564c1cc23b60 0x564c1cc465c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:45.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.404+0000 7f2e62d9d640 1 --2- 172.21.15.16:0/3029841218 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564c1cc2ed40 0x564c1cc2d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:45.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.404+0000 7f2e5bfff640 1 -- 172.21.15.16:0/3029841218 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564c1cc23b60 msgr2=0x564c1cc465c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:01:45.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.405+0000 7f2e5bfff640 1 --2- 172.21.15.16:0/3029841218 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564c1cc23b60 0x564c1cc465c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:45.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.405+0000 7f2e5bfff640 1 -- 172.21.15.16:0/3029841218 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564c1cc2ed40 msgr2=0x564c1cc2d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:01:45.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.405+0000 7f2e5bfff640 1 --2- 172.21.15.16:0/3029841218 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564c1cc2ed40 0x564c1cc2d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:45.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.405+0000 7f2e5bfff640 1 -- 172.21.15.16:0/3029841218 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c1cc1fb90 con 0x564c1cc24480 2024-09-12T02:01:45.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.405+0000 7f2e5bfff640 1 --2- 172.21.15.16:0/3029841218 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c1cc24480 0x564c1cc46b00 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c001eb0 tx=0x7f2e4c0108e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c540120d4dc53993 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:01:45.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.405+0000 7f2e4bfff640 1 -- 172.21.15.16:0/3029841218 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e4c035070 con 0x564c1cc24480 2024-09-12T02:01:45.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.405+0000 7f2e4bfff640 1 -- 172.21.15.16:0/3029841218 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e4c004990 con 0x564c1cc24480 2024-09-12T02:01:45.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.405+0000 7f2e4bfff640 1 -- 172.21.15.16:0/3029841218 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e4c011810 con 0x564c1cc24480 2024-09-12T02:01:45.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.406+0000 7f2e6b2a4740 1 -- 172.21.15.16:0/3029841218 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c1cc24480 msgr2=0x564c1cc46b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:45.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.406+0000 7f2e6b2a4740 1 --2- 172.21.15.16:0/3029841218 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c1cc24480 0x564c1cc46b00 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c001eb0 tx=0x7f2e4c0108e0 comp rx=0 tx=0).stop 2024-09-12T02:01:45.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.406+0000 7f2e6b2a4740 1 -- 172.21.15.16:0/3029841218 shutdown_connections 2024-09-12T02:01:45.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.406+0000 7f2e6b2a4740 1 --2- 172.21.15.16:0/3029841218 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564c1cc2ed40 0x564c1cc2d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:45.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.406+0000 7f2e6b2a4740 1 --2- 172.21.15.16:0/3029841218 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c1cc24480 0x564c1cc46b00 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:45.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.406+0000 7f2e6b2a4740 1 --2- 172.21.15.16:0/3029841218 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564c1cc23b60 0x564c1cc465c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:45.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.406+0000 7f2e6b2a4740 1 -- 172.21.15.16:0/3029841218 >> 172.21.15.16:0/3029841218 conn(0x564c1cc200f0 msgr2=0x564c1cc2ada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:01:45.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.406+0000 7f2e6b2a4740 1 -- 172.21.15.16:0/3029841218 shutdown_connections 2024-09-12T02:01:45.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.406+0000 7f2e6b2a4740 1 -- 172.21.15.16:0/3029841218 wait complete. 2024-09-12T02:01:45.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.407+0000 7f2e6b2a4740 1 -- start start 2024-09-12T02:01:45.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.407+0000 7f2e6b2a4740 1 Processor -- start 2024-09-12T02:01:45.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.407+0000 7f2e6b2a4740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564c1cc23b60 0x564c1cc37020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:45.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.407+0000 7f2e6b2a4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c1cc24480 0x564c1cc37560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:45.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.407+0000 7f2e6b2a4740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564c1cc2ed40 0x564c1cc323e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:45.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.407+0000 7f2e6b2a4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2e641adbd0 con 0x564c1cc24480 2024-09-12T02:01:45.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.407+0000 7f2e6b2a4740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2e641af040 con 0x564c1cc23b60 2024-09-12T02:01:45.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.407+0000 7f2e6b2a4740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f2e641ae2a0 con 0x564c1cc2ed40 2024-09-12T02:01:45.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.407+0000 7f2e5bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c1cc24480 0x564c1cc37560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:45.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.407+0000 7f2e5bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c1cc24480 0x564c1cc37560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46598/0 (socket says 172.21.15.16:46598) 2024-09-12T02:01:45.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.407+0000 7f2e5bfff640 1 -- 172.21.15.16:0/462721025 learned_addr learned my addr 172.21.15.16:0/462721025 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:01:45.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.407+0000 7f2e6b2a4740 10 client.0 _ll_register_callbacks cb 0x7f2e540f2de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:01:45.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.408+0000 7f2e62d9d640 1 --2- 172.21.15.16:0/462721025 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564c1cc2ed40 0x564c1cc323e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:45.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.408+0000 7f2e6259c640 1 --2- 172.21.15.16:0/462721025 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564c1cc23b60 0x564c1cc37020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:45.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.408+0000 7f2e5bfff640 1 -- 172.21.15.16:0/462721025 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564c1cc2ed40 msgr2=0x564c1cc323e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:01:45.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.408+0000 7f2e5bfff640 1 --2- 172.21.15.16:0/462721025 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564c1cc2ed40 0x564c1cc323e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:45.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.408+0000 7f2e5bfff640 1 -- 172.21.15.16:0/462721025 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564c1cc23b60 msgr2=0x564c1cc37020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:01:45.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.408+0000 7f2e5bfff640 1 --2- 172.21.15.16:0/462721025 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564c1cc23b60 0x564c1cc37020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:45.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.408+0000 7f2e5bfff640 1 -- 172.21.15.16:0/462721025 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564c1cc32c10 con 0x564c1cc24480 2024-09-12T02:01:45.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.408+0000 7f2e5bfff640 1 --2- 172.21.15.16:0/462721025 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c1cc24480 0x564c1cc37560 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c005bf0 tx=0x7f2e4c004d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b90e0624ab3580b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:01:45.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.408+0000 7f2e49ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:01:45.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.408+0000 7f2e49ffb640 1 -- 172.21.15.16:0/462721025 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e4c03e030 con 0x564c1cc24480 2024-09-12T02:01:45.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.408+0000 7f2e49ffb640 1 -- 172.21.15.16:0/462721025 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f2e4c005420 con 0x564c1cc24480 2024-09-12T02:01:45.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.408+0000 7f2e6b2a4740 1 -- 172.21.15.16:0/462721025 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e4c004990 con 0x564c1cc24480 2024-09-12T02:01:45.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.409+0000 7f2e49ffb640 1 -- 172.21.15.16:0/462721025 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2e4c035050 con 0x564c1cc24480 2024-09-12T02:01:45.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.409+0000 7f2e6b2a4740 10 client.15216 fetch_fsmap learned FSMap version 25 2024-09-12T02:01:45.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.409+0000 7f2e6b2a4740 1 -- 172.21.15.16:0/462721025 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x564c1cca9170 con 0x564c1cc24480 2024-09-12T02:01:45.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.409+0000 7f2e49ffb640 1 -- 172.21.15.16:0/462721025 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e4c004990 con 0x564c1cc24480 2024-09-12T02:01:45.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.409+0000 7f2e49ffb640 1 -- 172.21.15.16:0/462721025 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f2e4c011810 con 0x564c1cc24480 2024-09-12T02:01:45.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.409+0000 7f2e6b2a4740 10 client.15216 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:01:45.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.409+0000 7f2e6b2a4740 10 client.15216 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:01:45.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.409+0000 7f2e6b2a4740 20 client.15216 populate_metadata read hostname 'smithi016' 2024-09-12T02:01:45.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.409+0000 7f2e6b2a4740 1 --2- 172.21.15.16:0/462721025 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x564c1cca98d0 0x564c1ccabcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:45.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.409+0000 7f2e6b2a4740 4 client.15216 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-12T02:01:45.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.409+0000 7f2e6b2a4740 1 -- 172.21.15.16:0/462721025 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x564c1cb90d00 con 0x564c1cca98d0 2024-09-12T02:01:45.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.409+0000 7f2e6259c640 1 --2- 172.21.15.16:0/462721025 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x564c1cca98d0 0x564c1ccabcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:45.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.410+0000 7f2e49ffb640 10 client.15216 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:01:45.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.410+0000 7f2e6259c640 1 --2- 172.21.15.16:0/462721025 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x564c1cca98d0 0x564c1ccabcc0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b282c6de0593da78 server_cookie=bbfd7cce8e268d36 in_seq=0 out_seq=0 2024-09-12T02:01:45.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.411+0000 7f2e49ffb640 1 -- 172.21.15.16:0/462721025 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x564c1cb90d00 con 0x564c1cca98d0 2024-09-12T02:01:45.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.411+0000 7f2e49ffb640 10 client.15216 handle_command_reply: tid=0 2024-09-12T02:01:45.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.411+0000 7f2e6b2a4740 1 client.15216 shutdown 2024-09-12T02:01:45.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.411+0000 7f2e6b2a4740 1 -- 172.21.15.16:0/462721025 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c1cc24480 msgr2=0x564c1cc37560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:45.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.411+0000 7f2e6b2a4740 1 --2- 172.21.15.16:0/462721025 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c1cc24480 0x564c1cc37560 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c005bf0 tx=0x7f2e4c004d70 comp rx=0 tx=0).stop 2024-09-12T02:01:45.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.411+0000 7f2e6b2a4740 1 -- 172.21.15.16:0/462721025 shutdown_connections 2024-09-12T02:01:45.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.411+0000 7f2e6b2a4740 1 --2- 172.21.15.16:0/462721025 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x564c1cca98d0 0x564c1ccabcc0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:45.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.411+0000 7f2e6b2a4740 1 --2- 172.21.15.16:0/462721025 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564c1cc2ed40 0x564c1cc323e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:45.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.411+0000 7f2e6b2a4740 1 --2- 172.21.15.16:0/462721025 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c1cc24480 0x564c1cc37560 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:45.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.411+0000 7f2e6b2a4740 1 --2- 172.21.15.16:0/462721025 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564c1cc23b60 0x564c1cc37020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:45.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.411+0000 7f2e49ffb640 0 client.15216 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:01:45.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.411+0000 7f2e6b2a4740 1 -- 172.21.15.16:0/462721025 >> 172.21.15.16:0/462721025 conn(0x564c1cc200f0 msgr2=0x564c1cc1ea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:01:45.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.411+0000 7f2e6b2a4740 1 -- 172.21.15.16:0/462721025 shutdown_connections 2024-09-12T02:01:45.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.411+0000 7f2e6b2a4740 1 -- 172.21.15.16:0/462721025 wait complete. 2024-09-12T02:01:45.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.411+0000 7f2e6b2a4740 20 client.15216 trim_cache size 0 max 16384 2024-09-12T02:01:45.420 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T02:01:45.420 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-09-12T02:01:45.420 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "ca0d1578-6d6d-43f3-900a-7bd6aef3bd62", 2024-09-12T02:01:45.420 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-09-12T02:01:45.420 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T02:01:45.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.412+0000 7f2e68ecb640 1 -- 172.21.15.16:0/4129447548 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f2e500a2b80 msgr2=0x7f2e500a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:45.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.412+0000 7f2e68ecb640 1 --2- 172.21.15.16:0/4129447548 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f2e500a2b80 0x7f2e500a5040 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2e5c009fd0 tx=0x7f2e5c00d040 comp rx=0 tx=0).stop 2024-09-12T02:01:45.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.412+0000 7f2e68ecb640 1 -- 172.21.15.16:0/4129447548 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2e64100a60 msgr2=0x7f2e640fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:45.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.412+0000 7f2e68ecb640 1 --2- 172.21.15.16:0/4129447548 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2e64100a60 0x7f2e640fce10 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f2e54009230 tx=0x7f2e54004270 comp rx=0 tx=0).stop 2024-09-12T02:01:45.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.412+0000 7f2e68ecb640 1 -- 172.21.15.16:0/4129447548 shutdown_connections 2024-09-12T02:01:45.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.412+0000 7f2e68ecb640 1 --2- 172.21.15.16:0/4129447548 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f2e500a2b80 0x7f2e500a5040 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:45.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.412+0000 7f2e68ecb640 1 --2- 172.21.15.16:0/4129447548 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2e64106a60 0x7f2e640fd890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:45.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.412+0000 7f2e68ecb640 1 --2- 172.21.15.16:0/4129447548 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e64101400 0x7f2e640fd350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:45.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.412+0000 7f2e68ecb640 1 --2- 172.21.15.16:0/4129447548 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2e64100a60 0x7f2e640fce10 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:45.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.412+0000 7f2e68ecb640 1 -- 172.21.15.16:0/4129447548 >> 172.21.15.16:0/4129447548 conn(0x7f2e640fc780 msgr2=0x7f2e6406a9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:01:45.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.412+0000 7f2e68ecb640 1 -- 172.21.15.16:0/4129447548 shutdown_connections 2024-09-12T02:01:45.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:45.412+0000 7f2e68ecb640 1 -- 172.21.15.16:0/4129447548 wait complete. 2024-09-12T02:01:45.621 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:01:46.111 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T02:01:46.408 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:01:46 smithi016 ceph-mon[30947]: pgmap v330: 97 pgs: 97 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1023 B/s wr, 7 op/s 2024-09-12T02:01:46.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:01:46 smithi184 ceph-mon[35593]: pgmap v330: 97 pgs: 97 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1023 B/s wr, 7 op/s 2024-09-12T02:01:46.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:46.629+0000 7fac70eb1640 1 -- 172.21.15.16:0/2940097434 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac6c102790 msgr2=0x7fac6c1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:46.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:46.629+0000 7fac70eb1640 1 --2- 172.21.15.16:0/2940097434 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac6c102790 0x7fac6c1067e0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fac60009900 tx=0x7fac6002a410 comp rx=0 tx=0).stop 2024-09-12T02:01:46.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:46.629+0000 7fac70eb1640 1 -- 172.21.15.16:0/2940097434 shutdown_connections 2024-09-12T02:01:46.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:46.629+0000 7fac70eb1640 1 --2- 172.21.15.16:0/2940097434 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac6c106f10 0x7fac6c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:46.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:46.629+0000 7fac70eb1640 1 --2- 172.21.15.16:0/2940097434 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac6c102790 0x7fac6c1067e0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:46.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:46.629+0000 7fac70eb1640 1 --2- 172.21.15.16:0/2940097434 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fac6c101de0 0x7fac6c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:46.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:46.629+0000 7fac70eb1640 1 -- 172.21.15.16:0/2940097434 >> 172.21.15.16:0/2940097434 conn(0x7fac6c0fd8c0 msgr2=0x7fac6c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:01:46.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:46.629+0000 7fac70eb1640 1 -- 172.21.15.16:0/2940097434 shutdown_connections 2024-09-12T02:01:46.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:46.629+0000 7fac70eb1640 1 -- 172.21.15.16:0/2940097434 wait complete. 2024-09-12T02:01:46.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:46.630+0000 7fac70eb1640 1 Processor -- start 2024-09-12T02:01:46.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:46.630+0000 7fac70eb1640 1 -- start start 2024-09-12T02:01:46.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:46.630+0000 7fac70eb1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac6c101de0 0x7fac6c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:46.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:46.630+0000 7fac70eb1640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fac6c102790 0x7fac6c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:46.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:46.630+0000 7fac70eb1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac6c106f10 0x7fac6c074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:46.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:46.630+0000 7fac70eb1640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fac6c111f70 con 0x7fac6c101de0 2024-09-12T02:01:46.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:46.630+0000 7fac70eb1640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fac6c111df0 con 0x7fac6c106f10 2024-09-12T02:01:46.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:46.630+0000 7fac70eb1640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fac6c1120f0 con 0x7fac6c102790 2024-09-12T02:01:46.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:46.631+0000 7fac6a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac6c101de0 0x7fac6c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:46.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:46.631+0000 7fac6a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac6c101de0 0x7fac6c0744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46614/0 (socket says 172.21.15.16:46614) 2024-09-12T02:01:46.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:46.631+0000 7fac6a59c640 1 -- 172.21.15.16:0/1364821363 learned_addr learned my addr 172.21.15.16:0/1364821363 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:01:46.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:46.631+0000 7fac6ad9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac6c106f10 0x7fac6c074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:46.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:46.631+0000 7fac69d9b640 1 --2- 172.21.15.16:0/1364821363 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fac6c102790 0x7fac6c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:46.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:46.631+0000 7fac6a59c640 1 -- 172.21.15.16:0/1364821363 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fac6c102790 msgr2=0x7fac6c074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:46.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:46.631+0000 7fac6a59c640 1 --2- 172.21.15.16:0/1364821363 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fac6c102790 0x7fac6c074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:46.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:46.631+0000 7fac6a59c640 1 -- 172.21.15.16:0/1364821363 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac6c106f10 msgr2=0x7fac6c074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:46.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:46.631+0000 7fac6a59c640 1 --2- 172.21.15.16:0/1364821363 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac6c106f10 0x7fac6c074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:46.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:46.631+0000 7fac6a59c640 1 -- 172.21.15.16:0/1364821363 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac6c1aade0 con 0x7fac6c101de0 2024-09-12T02:01:46.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:46.631+0000 7fac6a59c640 1 --2- 172.21.15.16:0/1364821363 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac6c101de0 0x7fac6c0744f0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fac5c00d770 tx=0x7fac5c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:01:46.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:46.631+0000 7fac5b7fe640 1 -- 172.21.15.16:0/1364821363 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac5c017070 con 0x7fac6c101de0 2024-09-12T02:01:46.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:46.631+0000 7fac5b7fe640 1 -- 172.21.15.16:0/1364821363 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac5c005500 con 0x7fac6c101de0 2024-09-12T02:01:46.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:46.631+0000 7fac70eb1640 1 -- 172.21.15.16:0/1364821363 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac6c1aafb0 con 0x7fac6c101de0 2024-09-12T02:01:46.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:46.631+0000 7fac5b7fe640 1 -- 172.21.15.16:0/1364821363 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac5c004e00 con 0x7fac6c101de0 2024-09-12T02:01:46.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:46.631+0000 7fac70eb1640 1 -- 172.21.15.16:0/1364821363 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac6c1ab460 con 0x7fac6c101de0 2024-09-12T02:01:46.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:46.632+0000 7fac70eb1640 1 -- 172.21.15.16:0/1364821363 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac6c1032c0 con 0x7fac6c101de0 2024-09-12T02:01:46.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:46.633+0000 7fac5b7fe640 1 -- 172.21.15.16:0/1364821363 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fac5c005000 con 0x7fac6c101de0 2024-09-12T02:01:46.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:46.634+0000 7fac5b7fe640 1 --2- 172.21.15.16:0/1364821363 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fac540a2c00 0x7fac540a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:46.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:46.634+0000 7fac5b7fe640 1 -- 172.21.15.16:0/1364821363 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fac5c0ce990 con 0x7fac6c101de0 2024-09-12T02:01:46.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:46.634+0000 7fac69d9b640 1 --2- 172.21.15.16:0/1364821363 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fac540a2c00 0x7fac540a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:46.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:46.635+0000 7fac69d9b640 1 --2- 172.21.15.16:0/1364821363 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fac540a2c00 0x7fac540a50c0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fac60006010 tx=0x7fac60000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:01:46.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:46.638+0000 7fac5b7fe640 1 -- 172.21.15.16:0/1364821363 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fac5c09b990 con 0x7fac6c101de0 2024-09-12T02:01:46.668 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:01:46 smithi033 ceph-mon[35332]: pgmap v330: 97 pgs: 97 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1023 B/s wr, 7 op/s 2024-09-12T02:01:46.706 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:01:46.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:46.817+0000 7fac70eb1640 1 -- 172.21.15.16:0/1364821363 --> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fac6c062630 con 0x7fac540a2c00 2024-09-12T02:01:46.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:46.823+0000 7fac5b7fe640 1 -- 172.21.15.16:0/1364821363 <== mgr.14150 v2:172.21.15.16:6800/3887444820 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258837 (secure 0 0 0) 0x7fac6c062630 con 0x7fac540a2c00 2024-09-12T02:01:46.827 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T02:01:46.828 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-12T02:01:46.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:46.826+0000 7fac70eb1640 1 -- 172.21.15.16:0/1364821363 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fac540a2c00 msgr2=0x7fac540a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:46.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:46.826+0000 7fac70eb1640 1 --2- 172.21.15.16:0/1364821363 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fac540a2c00 0x7fac540a50c0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fac60006010 tx=0x7fac60000bc0 comp rx=0 tx=0).stop 2024-09-12T02:01:46.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:46.827+0000 7fac70eb1640 1 -- 172.21.15.16:0/1364821363 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac6c101de0 msgr2=0x7fac6c0744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:46.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:46.827+0000 7fac70eb1640 1 --2- 172.21.15.16:0/1364821363 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac6c101de0 0x7fac6c0744f0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fac5c00d770 tx=0x7fac5c00dc30 comp rx=0 tx=0).stop 2024-09-12T02:01:46.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:46.827+0000 7fac70eb1640 1 -- 172.21.15.16:0/1364821363 shutdown_connections 2024-09-12T02:01:46.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:46.827+0000 7fac70eb1640 1 --2- 172.21.15.16:0/1364821363 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fac540a2c00 0x7fac540a50c0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:46.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:46.827+0000 7fac70eb1640 1 --2- 172.21.15.16:0/1364821363 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac6c106f10 0x7fac6c074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:46.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:46.827+0000 7fac70eb1640 1 --2- 172.21.15.16:0/1364821363 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fac6c102790 0x7fac6c074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:46.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:46.827+0000 7fac70eb1640 1 --2- 172.21.15.16:0/1364821363 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac6c101de0 0x7fac6c0744f0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:46.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:46.827+0000 7fac70eb1640 1 -- 172.21.15.16:0/1364821363 >> 172.21.15.16:0/1364821363 conn(0x7fac6c0fd8c0 msgr2=0x7fac6c0fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:01:46.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:46.827+0000 7fac70eb1640 1 -- 172.21.15.16:0/1364821363 shutdown_connections 2024-09-12T02:01:46.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:46.827+0000 7fac70eb1640 1 -- 172.21.15.16:0/1364821363 wait complete. 2024-09-12T02:01:47.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:01:47 smithi184 ceph-mon[35593]: scrub queued for path: / 2024-09-12T02:01:47.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:01:47 smithi184 ceph-mon[35593]: scrub summary: idle+waiting paths [/] 2024-09-12T02:01:47.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:01:47 smithi184 ceph-mon[35593]: scrub summary: active paths [/] 2024-09-12T02:01:47.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:01:47 smithi184 ceph-mon[35593]: scrub summary: idle+waiting paths [/] 2024-09-12T02:01:47.631 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:01:47 smithi016 ceph-mon[30947]: scrub queued for path: / 2024-09-12T02:01:47.631 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:01:47 smithi016 ceph-mon[30947]: scrub summary: idle+waiting paths [/] 2024-09-12T02:01:47.631 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:01:47 smithi016 ceph-mon[30947]: scrub summary: active paths [/] 2024-09-12T02:01:47.631 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:01:47 smithi016 ceph-mon[30947]: scrub summary: idle+waiting paths [/] 2024-09-12T02:01:47.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:01:47 smithi033 ceph-mon[35332]: scrub queued for path: / 2024-09-12T02:01:47.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:01:47 smithi033 ceph-mon[35332]: scrub summary: idle+waiting paths [/] 2024-09-12T02:01:47.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:01:47 smithi033 ceph-mon[35332]: scrub summary: active paths [/] 2024-09-12T02:01:47.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:01:47 smithi033 ceph-mon[35332]: scrub summary: idle+waiting paths [/] 2024-09-12T02:01:47.674 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":331,"stamp":"2024-09-12T02:01:46.325053+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":659381,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":753,"num_read_kb":495,"num_write":353,"num_write_kb":3059,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":1180848,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":783,"ondisk_log_size":783,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":374280,"kb_used_data":12552,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124567544,"statfs":{"total":1151940427776,"available":1151557165056,"internally_reserved":0,"allocated":12853248,"data_stored":7200807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":370193723},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5012,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":79,"num_read_kb":43,"num_write":10,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002801"},"pg_stats":[{"pgid":"3.3f","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.318917+0000","last_change":"2024-09-12T01:59:55.503882+0000","last_active":"2024-09-12T02:01:38.318917+0000","last_peered":"2024-09-12T02:01:38.318917+0000","last_clean":"2024-09-12T02:01:38.318917+0000","last_became_active":"2024-09-12T01:59:55.503513+0000","last_became_peered":"2024-09-12T01:59:55.503513+0000","last_unstale":"2024-09-12T02:01:38.318917+0000","last_undegraded":"2024-09-12T02:01:38.318917+0000","last_fullsized":"2024-09-12T02:01:38.318917+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:43:13.098527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"82'4","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.322981+0000","last_change":"2024-09-12T01:59:55.492965+0000","last_active":"2024-09-12T02:01:38.322981+0000","last_peered":"2024-09-12T02:01:38.322981+0000","last_clean":"2024-09-12T02:01:38.322981+0000","last_became_active":"2024-09-12T01:59:55.492613+0000","last_became_peered":"2024-09-12T01:59:55.492613+0000","last_unstale":"2024-09-12T02:01:38.322981+0000","last_undegraded":"2024-09-12T02:01:38.322981+0000","last_fullsized":"2024-09-12T02:01:38.322981+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:58:20.889770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.332184+0000","last_change":"2024-09-12T01:59:55.497334+0000","last_active":"2024-09-12T02:01:38.332184+0000","last_peered":"2024-09-12T02:01:38.332184+0000","last_clean":"2024-09-12T02:01:38.332184+0000","last_became_active":"2024-09-12T01:59:55.497037+0000","last_became_peered":"2024-09-12T01:59:55.497037+0000","last_unstale":"2024-09-12T02:01:38.332184+0000","last_undegraded":"2024-09-12T02:01:38.332184+0000","last_fullsized":"2024-09-12T02:01:38.332184+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:43:04.082918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.331162+0000","last_change":"2024-09-12T01:59:55.495888+0000","last_active":"2024-09-12T02:01:38.331162+0000","last_peered":"2024-09-12T02:01:38.331162+0000","last_clean":"2024-09-12T02:01:38.331162+0000","last_became_active":"2024-09-12T01:59:55.492684+0000","last_became_peered":"2024-09-12T01:59:55.492684+0000","last_unstale":"2024-09-12T02:01:38.331162+0000","last_undegraded":"2024-09-12T02:01:38.331162+0000","last_fullsized":"2024-09-12T02:01:38.331162+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:15:00.500006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"82'4","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.301731+0000","last_change":"2024-09-12T01:59:55.494613+0000","last_active":"2024-09-12T02:01:38.301731+0000","last_peered":"2024-09-12T02:01:38.301731+0000","last_clean":"2024-09-12T02:01:38.301731+0000","last_became_active":"2024-09-12T01:59:55.494205+0000","last_became_peered":"2024-09-12T01:59:55.494205+0000","last_unstale":"2024-09-12T02:01:38.301731+0000","last_undegraded":"2024-09-12T02:01:38.301731+0000","last_fullsized":"2024-09-12T02:01:38.301731+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:39:14.573280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.331517+0000","last_change":"2024-09-12T01:59:55.496659+0000","last_active":"2024-09-12T02:01:38.331517+0000","last_peered":"2024-09-12T02:01:38.331517+0000","last_clean":"2024-09-12T02:01:38.331517+0000","last_became_active":"2024-09-12T01:59:55.496232+0000","last_became_peered":"2024-09-12T01:59:55.496232+0000","last_unstale":"2024-09-12T02:01:38.331517+0000","last_undegraded":"2024-09-12T02:01:38.331517+0000","last_fullsized":"2024-09-12T02:01:38.331517+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:48:21.544764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"82'5","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.317177+0000","last_change":"2024-09-12T01:59:55.486789+0000","last_active":"2024-09-12T02:01:38.317177+0000","last_peered":"2024-09-12T02:01:38.317177+0000","last_clean":"2024-09-12T02:01:38.317177+0000","last_became_active":"2024-09-12T01:59:55.485921+0000","last_became_peered":"2024-09-12T01:59:55.485921+0000","last_unstale":"2024-09-12T02:01:38.317177+0000","last_undegraded":"2024-09-12T02:01:38.317177+0000","last_fullsized":"2024-09-12T02:01:38.317177+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:21:16.071121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.321850+0000","last_change":"2024-09-12T01:59:55.494525+0000","last_active":"2024-09-12T02:01:38.321850+0000","last_peered":"2024-09-12T02:01:38.321850+0000","last_clean":"2024-09-12T02:01:38.321850+0000","last_became_active":"2024-09-12T01:59:55.493998+0000","last_became_peered":"2024-09-12T01:59:55.493998+0000","last_unstale":"2024-09-12T02:01:38.321850+0000","last_undegraded":"2024-09-12T02:01:38.321850+0000","last_fullsized":"2024-09-12T02:01:38.321850+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:59:24.653660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"82'11","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.320666+0000","last_change":"2024-09-12T01:59:55.501466+0000","last_active":"2024-09-12T02:01:38.320666+0000","last_peered":"2024-09-12T02:01:38.320666+0000","last_clean":"2024-09-12T02:01:38.320666+0000","last_became_active":"2024-09-12T01:59:55.500800+0000","last_became_peered":"2024-09-12T01:59:55.500800+0000","last_unstale":"2024-09-12T02:01:38.320666+0000","last_undegraded":"2024-09-12T02:01:38.320666+0000","last_fullsized":"2024-09-12T02:01:38.320666+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:55:13.704791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"82'16","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.331670+0000","last_change":"2024-09-12T01:59:55.497986+0000","last_active":"2024-09-12T02:01:38.331670+0000","last_peered":"2024-09-12T02:01:38.331670+0000","last_clean":"2024-09-12T02:01:38.331670+0000","last_became_active":"2024-09-12T01:59:55.497691+0000","last_became_peered":"2024-09-12T01:59:55.497691+0000","last_unstale":"2024-09-12T02:01:38.331670+0000","last_undegraded":"2024-09-12T02:01:38.331670+0000","last_fullsized":"2024-09-12T02:01:38.331670+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:37:07.702606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.329986+0000","last_change":"2024-09-12T01:59:55.501339+0000","last_active":"2024-09-12T02:01:38.329986+0000","last_peered":"2024-09-12T02:01:38.329986+0000","last_clean":"2024-09-12T02:01:38.329986+0000","last_became_active":"2024-09-12T01:59:55.500700+0000","last_became_peered":"2024-09-12T01:59:55.500700+0000","last_unstale":"2024-09-12T02:01:38.329986+0000","last_undegraded":"2024-09-12T02:01:38.329986+0000","last_fullsized":"2024-09-12T02:01:38.329986+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:12:15.879183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.330871+0000","last_change":"2024-09-12T01:59:55.495196+0000","last_active":"2024-09-12T02:01:38.330871+0000","last_peered":"2024-09-12T02:01:38.330871+0000","last_clean":"2024-09-12T02:01:38.330871+0000","last_became_active":"2024-09-12T01:59:55.494889+0000","last_became_peered":"2024-09-12T01:59:55.494889+0000","last_unstale":"2024-09-12T02:01:38.330871+0000","last_undegraded":"2024-09-12T02:01:38.330871+0000","last_fullsized":"2024-09-12T02:01:38.330871+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:32:30.621946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"82'10","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.315946+0000","last_change":"2024-09-12T01:59:55.492565+0000","last_active":"2024-09-12T02:01:38.315946+0000","last_peered":"2024-09-12T02:01:38.315946+0000","last_clean":"2024-09-12T02:01:38.315946+0000","last_became_active":"2024-09-12T01:59:55.492168+0000","last_became_peered":"2024-09-12T01:59:55.492168+0000","last_unstale":"2024-09-12T02:01:38.315946+0000","last_undegraded":"2024-09-12T02:01:38.315946+0000","last_fullsized":"2024-09-12T02:01:38.315946+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:39:48.630202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.323015+0000","last_change":"2024-09-12T01:59:55.493152+0000","last_active":"2024-09-12T02:01:38.323015+0000","last_peered":"2024-09-12T02:01:38.323015+0000","last_clean":"2024-09-12T02:01:38.323015+0000","last_became_active":"2024-09-12T01:59:55.492794+0000","last_became_peered":"2024-09-12T01:59:55.492794+0000","last_unstale":"2024-09-12T02:01:38.323015+0000","last_undegraded":"2024-09-12T02:01:38.323015+0000","last_fullsized":"2024-09-12T02:01:38.323015+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:24:27.270502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.333374+0000","last_change":"2024-09-12T01:59:55.494651+0000","last_active":"2024-09-12T02:01:38.333374+0000","last_peered":"2024-09-12T02:01:38.333374+0000","last_clean":"2024-09-12T02:01:38.333374+0000","last_became_active":"2024-09-12T01:59:55.494355+0000","last_became_peered":"2024-09-12T01:59:55.494355+0000","last_unstale":"2024-09-12T02:01:38.333374+0000","last_undegraded":"2024-09-12T02:01:38.333374+0000","last_fullsized":"2024-09-12T02:01:38.333374+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:14:44.364158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.330799+0000","last_change":"2024-09-12T01:59:55.495049+0000","last_active":"2024-09-12T02:01:38.330799+0000","last_peered":"2024-09-12T02:01:38.330799+0000","last_clean":"2024-09-12T02:01:38.330799+0000","last_became_active":"2024-09-12T01:59:55.494688+0000","last_became_peered":"2024-09-12T01:59:55.494688+0000","last_unstale":"2024-09-12T02:01:38.330799+0000","last_undegraded":"2024-09-12T02:01:38.330799+0000","last_fullsized":"2024-09-12T02:01:38.330799+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:13:05.944577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.319055+0000","last_change":"2024-09-12T01:59:55.504146+0000","last_active":"2024-09-12T02:01:38.319055+0000","last_peered":"2024-09-12T02:01:38.319055+0000","last_clean":"2024-09-12T02:01:38.319055+0000","last_became_active":"2024-09-12T01:59:55.503646+0000","last_became_peered":"2024-09-12T01:59:55.503646+0000","last_unstale":"2024-09-12T02:01:38.319055+0000","last_undegraded":"2024-09-12T02:01:38.319055+0000","last_fullsized":"2024-09-12T02:01:38.319055+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:23:51.780795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"82'10","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.325350+0000","last_change":"2024-09-12T01:59:55.489930+0000","last_active":"2024-09-12T02:01:38.325350+0000","last_peered":"2024-09-12T02:01:38.325350+0000","last_clean":"2024-09-12T02:01:38.325350+0000","last_became_active":"2024-09-12T01:59:55.489619+0000","last_became_peered":"2024-09-12T01:59:55.489619+0000","last_unstale":"2024-09-12T02:01:38.325350+0000","last_undegraded":"2024-09-12T02:01:38.325350+0000","last_fullsized":"2024-09-12T02:01:38.325350+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:04:57.307091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.303800+0000","last_change":"2024-09-12T01:59:55.492098+0000","last_active":"2024-09-12T02:01:38.303800+0000","last_peered":"2024-09-12T02:01:38.303800+0000","last_clean":"2024-09-12T02:01:38.303800+0000","last_became_active":"2024-09-12T01:59:55.491370+0000","last_became_peered":"2024-09-12T01:59:55.491370+0000","last_unstale":"2024-09-12T02:01:38.303800+0000","last_undegraded":"2024-09-12T02:01:38.303800+0000","last_fullsized":"2024-09-12T02:01:38.303800+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:25:06.307903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.314843+0000","last_change":"2024-09-12T01:59:55.499938+0000","last_active":"2024-09-12T02:01:38.314843+0000","last_peered":"2024-09-12T02:01:38.314843+0000","last_clean":"2024-09-12T02:01:38.314843+0000","last_became_active":"2024-09-12T01:59:55.499619+0000","last_became_peered":"2024-09-12T01:59:55.499619+0000","last_unstale":"2024-09-12T02:01:38.314843+0000","last_undegraded":"2024-09-12T02:01:38.314843+0000","last_fullsized":"2024-09-12T02:01:38.314843+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:00:17.536120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"82'11","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.321068+0000","last_change":"2024-09-12T01:59:55.486106+0000","last_active":"2024-09-12T02:01:38.321068+0000","last_peered":"2024-09-12T02:01:38.321068+0000","last_clean":"2024-09-12T02:01:38.321068+0000","last_became_active":"2024-09-12T01:59:55.485325+0000","last_became_peered":"2024-09-12T01:59:55.485325+0000","last_unstale":"2024-09-12T02:01:38.321068+0000","last_undegraded":"2024-09-12T02:01:38.321068+0000","last_fullsized":"2024-09-12T02:01:38.321068+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:35:46.446224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"82'10","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.331650+0000","last_change":"2024-09-12T01:59:55.496438+0000","last_active":"2024-09-12T02:01:38.331650+0000","last_peered":"2024-09-12T02:01:38.331650+0000","last_clean":"2024-09-12T02:01:38.331650+0000","last_became_active":"2024-09-12T01:59:55.495885+0000","last_became_peered":"2024-09-12T01:59:55.495885+0000","last_unstale":"2024-09-12T02:01:38.331650+0000","last_undegraded":"2024-09-12T02:01:38.331650+0000","last_fullsized":"2024-09-12T02:01:38.331650+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:15:14.911335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.322674+0000","last_change":"2024-09-12T01:59:55.489350+0000","last_active":"2024-09-12T02:01:38.322674+0000","last_peered":"2024-09-12T02:01:38.322674+0000","last_clean":"2024-09-12T02:01:38.322674+0000","last_became_active":"2024-09-12T01:59:55.489028+0000","last_became_peered":"2024-09-12T01:59:55.489028+0000","last_unstale":"2024-09-12T02:01:38.322674+0000","last_undegraded":"2024-09-12T02:01:38.322674+0000","last_fullsized":"2024-09-12T02:01:38.322674+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:29:23.769004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.316094+0000","last_change":"2024-09-12T01:59:55.494532+0000","last_active":"2024-09-12T02:01:38.316094+0000","last_peered":"2024-09-12T02:01:38.316094+0000","last_clean":"2024-09-12T02:01:38.316094+0000","last_became_active":"2024-09-12T01:59:55.490982+0000","last_became_peered":"2024-09-12T01:59:55.490982+0000","last_unstale":"2024-09-12T02:01:38.316094+0000","last_undegraded":"2024-09-12T02:01:38.316094+0000","last_fullsized":"2024-09-12T02:01:38.316094+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:04:50.633730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"82'2","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.316143+0000","last_change":"2024-09-12T01:59:55.496057+0000","last_active":"2024-09-12T02:01:38.316143+0000","last_peered":"2024-09-12T02:01:38.316143+0000","last_clean":"2024-09-12T02:01:38.316143+0000","last_became_active":"2024-09-12T01:59:55.495710+0000","last_became_peered":"2024-09-12T01:59:55.495710+0000","last_unstale":"2024-09-12T02:01:38.316143+0000","last_undegraded":"2024-09-12T02:01:38.316143+0000","last_fullsized":"2024-09-12T02:01:38.316143+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:49:01.962537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"82'15","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.311246+0000","last_change":"2024-09-12T01:59:55.491594+0000","last_active":"2024-09-12T02:01:38.311246+0000","last_peered":"2024-09-12T02:01:38.311246+0000","last_clean":"2024-09-12T02:01:38.311246+0000","last_became_active":"2024-09-12T01:59:55.491233+0000","last_became_peered":"2024-09-12T01:59:55.491233+0000","last_unstale":"2024-09-12T02:01:38.311246+0000","last_undegraded":"2024-09-12T02:01:38.311246+0000","last_fullsized":"2024-09-12T02:01:38.311246+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:24:22.066777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.309442+0000","last_change":"2024-09-12T01:59:55.498331+0000","last_active":"2024-09-12T02:01:38.309442+0000","last_peered":"2024-09-12T02:01:38.309442+0000","last_clean":"2024-09-12T02:01:38.309442+0000","last_became_active":"2024-09-12T01:59:55.497983+0000","last_became_peered":"2024-09-12T01:59:55.497983+0000","last_unstale":"2024-09-12T02:01:38.309442+0000","last_undegraded":"2024-09-12T02:01:38.309442+0000","last_fullsized":"2024-09-12T02:01:38.309442+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:37:00.735231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"82'4","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.315811+0000","last_change":"2024-09-12T02:00:09.575358+0000","last_active":"2024-09-12T02:01:38.315811+0000","last_peered":"2024-09-12T02:01:38.315811+0000","last_clean":"2024-09-12T02:01:38.315811+0000","last_became_active":"2024-09-12T02:00:09.574913+0000","last_became_peered":"2024-09-12T02:00:09.574913+0000","last_unstale":"2024-09-12T02:01:38.315811+0000","last_undegraded":"2024-09-12T02:01:38.315811+0000","last_fullsized":"2024-09-12T02:01:38.315811+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:19:53.630453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.333406+0000","last_change":"2024-09-12T01:59:55.492419+0000","last_active":"2024-09-12T02:01:38.333406+0000","last_peered":"2024-09-12T02:01:38.333406+0000","last_clean":"2024-09-12T02:01:38.333406+0000","last_became_active":"2024-09-12T01:59:55.492003+0000","last_became_peered":"2024-09-12T01:59:55.492003+0000","last_unstale":"2024-09-12T02:01:38.333406+0000","last_undegraded":"2024-09-12T02:01:38.333406+0000","last_fullsized":"2024-09-12T02:01:38.333406+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:25:45.774679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.307126+0000","last_change":"2024-09-12T01:59:55.490125+0000","last_active":"2024-09-12T02:01:38.307126+0000","last_peered":"2024-09-12T02:01:38.307126+0000","last_clean":"2024-09-12T02:01:38.307126+0000","last_became_active":"2024-09-12T01:59:55.489636+0000","last_became_peered":"2024-09-12T01:59:55.489636+0000","last_unstale":"2024-09-12T02:01:38.307126+0000","last_undegraded":"2024-09-12T02:01:38.307126+0000","last_fullsized":"2024-09-12T02:01:38.307126+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:41:30.922185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"82'12","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.309816+0000","last_change":"2024-09-12T01:59:55.496322+0000","last_active":"2024-09-12T02:01:38.309816+0000","last_peered":"2024-09-12T02:01:38.309816+0000","last_clean":"2024-09-12T02:01:38.309816+0000","last_became_active":"2024-09-12T01:59:55.496005+0000","last_became_peered":"2024-09-12T01:59:55.496005+0000","last_unstale":"2024-09-12T02:01:38.309816+0000","last_undegraded":"2024-09-12T02:01:38.309816+0000","last_fullsized":"2024-09-12T02:01:38.309816+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:53:39.248715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.310438+0000","last_change":"2024-09-12T01:59:55.497758+0000","last_active":"2024-09-12T02:01:38.310438+0000","last_peered":"2024-09-12T02:01:38.310438+0000","last_clean":"2024-09-12T02:01:38.310438+0000","last_became_active":"2024-09-12T01:59:55.497282+0000","last_became_peered":"2024-09-12T01:59:55.497282+0000","last_unstale":"2024-09-12T02:01:38.310438+0000","last_undegraded":"2024-09-12T02:01:38.310438+0000","last_fullsized":"2024-09-12T02:01:38.310438+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:08:25.509372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.301389+0000","last_change":"2024-09-12T01:59:55.497919+0000","last_active":"2024-09-12T02:01:38.301389+0000","last_peered":"2024-09-12T02:01:38.301389+0000","last_clean":"2024-09-12T02:01:38.301389+0000","last_became_active":"2024-09-12T01:59:55.497609+0000","last_became_peered":"2024-09-12T01:59:55.497609+0000","last_unstale":"2024-09-12T02:01:38.301389+0000","last_undegraded":"2024-09-12T02:01:38.301389+0000","last_fullsized":"2024-09-12T02:01:38.301389+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:10:22.281180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"82'7","reported_seq":134,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:44.695380+0000","last_change":"2024-09-12T01:59:53.473974+0000","last_active":"2024-09-12T02:01:44.695380+0000","last_peered":"2024-09-12T02:01:44.695380+0000","last_clean":"2024-09-12T02:01:44.695380+0000","last_became_active":"2024-09-12T01:59:53.473546+0000","last_became_peered":"2024-09-12T01:59:53.473546+0000","last_unstale":"2024-09-12T02:01:44.695380+0000","last_undegraded":"2024-09-12T02:01:44.695380+0000","last_fullsized":"2024-09-12T02:01:44.695380+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:28:03.532249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1156,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":105,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.328613+0000","last_change":"2024-09-12T01:59:55.493197+0000","last_active":"2024-09-12T02:01:38.328613+0000","last_peered":"2024-09-12T02:01:38.328613+0000","last_clean":"2024-09-12T02:01:38.328613+0000","last_became_active":"2024-09-12T01:59:55.492845+0000","last_became_peered":"2024-09-12T01:59:55.492845+0000","last_unstale":"2024-09-12T02:01:38.328613+0000","last_undegraded":"2024-09-12T02:01:38.328613+0000","last_fullsized":"2024-09-12T02:01:38.328613+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:42:50.790896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"80'3","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.622865+0000","last_change":"2024-09-12T01:59:53.471214+0000","last_active":"2024-09-12T02:00:09.622865+0000","last_peered":"2024-09-12T02:00:09.622865+0000","last_clean":"2024-09-12T02:00:09.622865+0000","last_became_active":"2024-09-12T01:59:53.470858+0000","last_became_peered":"2024-09-12T01:59:53.470858+0000","last_unstale":"2024-09-12T02:00:09.622865+0000","last_undegraded":"2024-09-12T02:00:09.622865+0000","last_fullsized":"2024-09-12T02:00:09.622865+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:34:48.730392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"80'2","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.562106+0000","last_change":"2024-09-12T01:59:53.464568+0000","last_active":"2024-09-12T02:00:09.562106+0000","last_peered":"2024-09-12T02:00:09.562106+0000","last_clean":"2024-09-12T02:00:09.562106+0000","last_became_active":"2024-09-12T01:59:53.464168+0000","last_became_peered":"2024-09-12T01:59:53.464168+0000","last_unstale":"2024-09-12T02:00:09.562106+0000","last_undegraded":"2024-09-12T02:00:09.562106+0000","last_fullsized":"2024-09-12T02:00:09.562106+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:41:48.135522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.318601+0000","last_change":"2024-09-12T01:59:55.487177+0000","last_active":"2024-09-12T02:01:38.318601+0000","last_peered":"2024-09-12T02:01:38.318601+0000","last_clean":"2024-09-12T02:01:38.318601+0000","last_became_active":"2024-09-12T01:59:55.486886+0000","last_became_peered":"2024-09-12T01:59:55.486886+0000","last_unstale":"2024-09-12T02:01:38.318601+0000","last_undegraded":"2024-09-12T02:01:38.318601+0000","last_fullsized":"2024-09-12T02:01:38.318601+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:32:40.150071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"80'1","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.567129+0000","last_change":"2024-09-12T01:59:53.474114+0000","last_active":"2024-09-12T02:00:09.567129+0000","last_peered":"2024-09-12T02:00:09.567129+0000","last_clean":"2024-09-12T02:00:09.567129+0000","last_became_active":"2024-09-12T01:59:53.473755+0000","last_became_peered":"2024-09-12T01:59:53.473755+0000","last_unstale":"2024-09-12T02:00:09.567129+0000","last_undegraded":"2024-09-12T02:00:09.567129+0000","last_fullsized":"2024-09-12T02:00:09.567129+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:07:07.021137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"82'11","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.323623+0000","last_change":"2024-09-12T01:59:55.489868+0000","last_active":"2024-09-12T02:01:38.323623+0000","last_peered":"2024-09-12T02:01:38.323623+0000","last_clean":"2024-09-12T02:01:38.323623+0000","last_became_active":"2024-09-12T01:59:55.489537+0000","last_became_peered":"2024-09-12T01:59:55.489537+0000","last_unstale":"2024-09-12T02:01:38.323623+0000","last_undegraded":"2024-09-12T02:01:38.323623+0000","last_fullsized":"2024-09-12T02:01:38.323623+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:18:49.417689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"82'4","reported_seq":125,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:42.692210+0000","last_change":"2024-09-12T01:59:53.471402+0000","last_active":"2024-09-12T02:01:42.692210+0000","last_peered":"2024-09-12T02:01:42.692210+0000","last_clean":"2024-09-12T02:01:42.692210+0000","last_became_active":"2024-09-12T01:59:53.470940+0000","last_became_peered":"2024-09-12T01:59:53.470940+0000","last_unstale":"2024-09-12T02:01:42.692210+0000","last_undegraded":"2024-09-12T02:01:42.692210+0000","last_fullsized":"2024-09-12T02:01:42.692210+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:41:16.853592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":99,"num_read_kb":99,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"82'10","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.330030+0000","last_change":"2024-09-12T01:59:55.499047+0000","last_active":"2024-09-12T02:01:38.330030+0000","last_peered":"2024-09-12T02:01:38.330030+0000","last_clean":"2024-09-12T02:01:38.330030+0000","last_became_active":"2024-09-12T01:59:55.498071+0000","last_became_peered":"2024-09-12T01:59:55.498071+0000","last_unstale":"2024-09-12T02:01:38.330030+0000","last_undegraded":"2024-09-12T02:01:38.330030+0000","last_fullsized":"2024-09-12T02:01:38.330030+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:36:40.106041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"80'2","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.622966+0000","last_change":"2024-09-12T01:59:53.473508+0000","last_active":"2024-09-12T02:00:09.622966+0000","last_peered":"2024-09-12T02:00:09.622966+0000","last_clean":"2024-09-12T02:00:09.622966+0000","last_became_active":"2024-09-12T01:59:53.473063+0000","last_became_peered":"2024-09-12T01:59:53.473063+0000","last_unstale":"2024-09-12T02:00:09.622966+0000","last_undegraded":"2024-09-12T02:00:09.622966+0000","last_fullsized":"2024-09-12T02:00:09.622966+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:55:32.161769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.333032+0000","last_change":"2024-09-12T01:59:55.491721+0000","last_active":"2024-09-12T02:01:38.333032+0000","last_peered":"2024-09-12T02:01:38.333032+0000","last_clean":"2024-09-12T02:01:38.333032+0000","last_became_active":"2024-09-12T01:59:55.491364+0000","last_became_peered":"2024-09-12T01:59:55.491364+0000","last_unstale":"2024-09-12T02:01:38.333032+0000","last_undegraded":"2024-09-12T02:01:38.333032+0000","last_fullsized":"2024-09-12T02:01:38.333032+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:21:20.795420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"80'4","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.567315+0000","last_change":"2024-09-12T01:59:53.921789+0000","last_active":"2024-09-12T02:00:09.567315+0000","last_peered":"2024-09-12T02:00:09.567315+0000","last_clean":"2024-09-12T02:00:09.567315+0000","last_became_active":"2024-09-12T01:59:53.921383+0000","last_became_peered":"2024-09-12T01:59:53.921383+0000","last_unstale":"2024-09-12T02:00:09.567315+0000","last_undegraded":"2024-09-12T02:00:09.567315+0000","last_fullsized":"2024-09-12T02:00:09.567315+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:21:58.792794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"82'11","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.331576+0000","last_change":"2024-09-12T01:59:55.503008+0000","last_active":"2024-09-12T02:01:38.331576+0000","last_peered":"2024-09-12T02:01:38.331576+0000","last_clean":"2024-09-12T02:01:38.331576+0000","last_became_active":"2024-09-12T01:59:55.502538+0000","last_became_peered":"2024-09-12T01:59:55.502538+0000","last_unstale":"2024-09-12T02:01:38.331576+0000","last_undegraded":"2024-09-12T02:01:38.331576+0000","last_fullsized":"2024-09-12T02:01:38.331576+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:51:36.735985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"82'32","reported_seq":263,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:43.693503+0000","last_change":"2024-09-12T01:59:53.470688+0000","last_active":"2024-09-12T02:01:43.693503+0000","last_peered":"2024-09-12T02:01:43.693503+0000","last_clean":"2024-09-12T02:01:43.693503+0000","last_became_active":"2024-09-12T01:59:53.470231+0000","last_became_peered":"2024-09-12T01:59:53.470231+0000","last_unstale":"2024-09-12T02:01:43.693503+0000","last_undegraded":"2024-09-12T02:01:43.693503+0000","last_fullsized":"2024-09-12T02:01:43.693503+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:53:54.439893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64635,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":209,"num_read_kb":168,"num_write":33,"num_write_kb":76,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.310987+0000","last_change":"2024-09-12T01:59:55.495343+0000","last_active":"2024-09-12T02:01:38.310987+0000","last_peered":"2024-09-12T02:01:38.310987+0000","last_clean":"2024-09-12T02:01:38.310987+0000","last_became_active":"2024-09-12T01:59:55.495006+0000","last_became_peered":"2024-09-12T01:59:55.495006+0000","last_unstale":"2024-09-12T02:01:38.310987+0000","last_undegraded":"2024-09-12T02:01:38.310987+0000","last_fullsized":"2024-09-12T02:01:38.310987+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:32:20.932551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'6","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.569082+0000","last_change":"2024-09-12T01:59:53.468625+0000","last_active":"2024-09-12T02:00:09.569082+0000","last_peered":"2024-09-12T02:00:09.569082+0000","last_clean":"2024-09-12T02:00:09.569082+0000","last_became_active":"2024-09-12T01:59:53.468015+0000","last_became_peered":"2024-09-12T01:59:53.468015+0000","last_unstale":"2024-09-12T02:00:09.569082+0000","last_undegraded":"2024-09-12T02:00:09.569082+0000","last_fullsized":"2024-09-12T02:00:09.569082+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:49:34.842239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.319752+0000","last_change":"2024-09-12T01:59:55.487962+0000","last_active":"2024-09-12T02:01:38.319752+0000","last_peered":"2024-09-12T02:01:38.319752+0000","last_clean":"2024-09-12T02:01:38.319752+0000","last_became_active":"2024-09-12T01:59:55.487619+0000","last_became_peered":"2024-09-12T01:59:55.487619+0000","last_unstale":"2024-09-12T02:01:38.319752+0000","last_undegraded":"2024-09-12T02:01:38.319752+0000","last_fullsized":"2024-09-12T02:01:38.319752+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:32:16.882373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"80'3","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.567124+0000","last_change":"2024-09-12T01:59:53.474418+0000","last_active":"2024-09-12T02:00:09.567124+0000","last_peered":"2024-09-12T02:00:09.567124+0000","last_clean":"2024-09-12T02:00:09.567124+0000","last_became_active":"2024-09-12T01:59:53.474059+0000","last_became_peered":"2024-09-12T01:59:53.474059+0000","last_unstale":"2024-09-12T02:00:09.567124+0000","last_undegraded":"2024-09-12T02:00:09.567124+0000","last_fullsized":"2024-09-12T02:00:09.567124+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:10:54.963435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"82'15","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.333121+0000","last_change":"2024-09-12T01:59:55.493394+0000","last_active":"2024-09-12T02:01:38.333121+0000","last_peered":"2024-09-12T02:01:38.333121+0000","last_clean":"2024-09-12T02:01:38.333121+0000","last_became_active":"2024-09-12T01:59:55.491669+0000","last_became_peered":"2024-09-12T01:59:55.491669+0000","last_unstale":"2024-09-12T02:01:38.333121+0000","last_undegraded":"2024-09-12T02:01:38.333121+0000","last_fullsized":"2024-09-12T02:01:38.333121+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:14:40.871154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"80'3","reported_seq":126,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:42.692234+0000","last_change":"2024-09-12T01:59:53.471623+0000","last_active":"2024-09-12T02:01:42.692234+0000","last_peered":"2024-09-12T02:01:42.692234+0000","last_clean":"2024-09-12T02:01:42.692234+0000","last_became_active":"2024-09-12T01:59:53.471294+0000","last_became_peered":"2024-09-12T01:59:53.471294+0000","last_unstale":"2024-09-12T02:01:42.692234+0000","last_undegraded":"2024-09-12T02:01:42.692234+0000","last_fullsized":"2024-09-12T02:01:42.692234+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:51:54.130883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":102,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.322144+0000","last_change":"2024-09-12T01:59:55.493082+0000","last_active":"2024-09-12T02:01:38.322144+0000","last_peered":"2024-09-12T02:01:38.322144+0000","last_clean":"2024-09-12T02:01:38.322144+0000","last_became_active":"2024-09-12T01:59:55.492778+0000","last_became_peered":"2024-09-12T01:59:55.492778+0000","last_unstale":"2024-09-12T02:01:38.322144+0000","last_undegraded":"2024-09-12T02:01:38.322144+0000","last_fullsized":"2024-09-12T02:01:38.322144+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:59:56.466377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'15","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.588050+0000","last_change":"2024-09-12T02:00:09.588050+0000","last_active":"2024-09-12T02:00:09.588050+0000","last_peered":"2024-09-12T02:00:09.588050+0000","last_clean":"2024-09-12T02:00:09.588050+0000","last_became_active":"2024-09-12T02:00:09.575478+0000","last_became_peered":"2024-09-12T02:00:09.575478+0000","last_unstale":"2024-09-12T02:00:09.588050+0000","last_undegraded":"2024-09-12T02:00:09.588050+0000","last_fullsized":"2024-09-12T02:00:09.588050+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:38:45.484167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.313876+0000","last_change":"2024-09-12T01:59:55.497601+0000","last_active":"2024-09-12T02:01:38.313876+0000","last_peered":"2024-09-12T02:01:38.313876+0000","last_clean":"2024-09-12T02:01:38.313876+0000","last_became_active":"2024-09-12T01:59:55.497202+0000","last_became_peered":"2024-09-12T01:59:55.497202+0000","last_unstale":"2024-09-12T02:01:38.313876+0000","last_undegraded":"2024-09-12T02:01:38.313876+0000","last_fullsized":"2024-09-12T02:01:38.313876+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:22:22.850736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"80'2","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.575359+0000","last_change":"2024-09-12T01:59:53.467575+0000","last_active":"2024-09-12T02:00:09.575359+0000","last_peered":"2024-09-12T02:00:09.575359+0000","last_clean":"2024-09-12T02:00:09.575359+0000","last_became_active":"2024-09-12T01:59:53.467146+0000","last_became_peered":"2024-09-12T01:59:53.467146+0000","last_unstale":"2024-09-12T02:00:09.575359+0000","last_undegraded":"2024-09-12T02:00:09.575359+0000","last_fullsized":"2024-09-12T02:00:09.575359+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:07:25.962496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"82'5","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.314382+0000","last_change":"2024-09-12T01:59:55.498297+0000","last_active":"2024-09-12T02:01:38.314382+0000","last_peered":"2024-09-12T02:01:38.314382+0000","last_clean":"2024-09-12T02:01:38.314382+0000","last_became_active":"2024-09-12T01:59:55.497367+0000","last_became_peered":"2024-09-12T01:59:55.497367+0000","last_unstale":"2024-09-12T02:01:38.314382+0000","last_undegraded":"2024-09-12T02:01:38.314382+0000","last_fullsized":"2024-09-12T02:01:38.314382+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:58:32.717710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"80'2","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.562292+0000","last_change":"2024-09-12T01:59:53.466664+0000","last_active":"2024-09-12T02:00:09.562292+0000","last_peered":"2024-09-12T02:00:09.562292+0000","last_clean":"2024-09-12T02:00:09.562292+0000","last_became_active":"2024-09-12T01:59:53.466318+0000","last_became_peered":"2024-09-12T01:59:53.466318+0000","last_unstale":"2024-09-12T02:00:09.562292+0000","last_undegraded":"2024-09-12T02:00:09.562292+0000","last_fullsized":"2024-09-12T02:00:09.562292+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:12:13.541420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.313289+0000","last_change":"2024-09-12T01:59:55.490420+0000","last_active":"2024-09-12T02:01:38.313289+0000","last_peered":"2024-09-12T02:01:38.313289+0000","last_clean":"2024-09-12T02:01:38.313289+0000","last_became_active":"2024-09-12T01:59:55.489757+0000","last_became_peered":"2024-09-12T01:59:55.489757+0000","last_unstale":"2024-09-12T02:01:38.313289+0000","last_undegraded":"2024-09-12T02:01:38.313289+0000","last_fullsized":"2024-09-12T02:01:38.313289+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:34:43.117142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'1","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.567185+0000","last_change":"2024-09-12T01:59:53.466399+0000","last_active":"2024-09-12T02:00:09.567185+0000","last_peered":"2024-09-12T02:00:09.567185+0000","last_clean":"2024-09-12T02:00:09.567185+0000","last_became_active":"2024-09-12T01:59:53.465982+0000","last_became_peered":"2024-09-12T01:59:53.465982+0000","last_unstale":"2024-09-12T02:00:09.567185+0000","last_undegraded":"2024-09-12T02:00:09.567185+0000","last_fullsized":"2024-09-12T02:00:09.567185+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:18:02.015040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.326034+0000","last_change":"2024-09-12T01:59:55.491527+0000","last_active":"2024-09-12T02:01:38.326034+0000","last_peered":"2024-09-12T02:01:38.326034+0000","last_clean":"2024-09-12T02:01:38.326034+0000","last_became_active":"2024-09-12T01:59:55.491165+0000","last_became_peered":"2024-09-12T01:59:55.491165+0000","last_unstale":"2024-09-12T02:01:38.326034+0000","last_undegraded":"2024-09-12T02:01:38.326034+0000","last_fullsized":"2024-09-12T02:01:38.326034+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:57:05.506384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"80'3","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.569187+0000","last_change":"2024-09-12T01:59:54.075752+0000","last_active":"2024-09-12T02:00:09.569187+0000","last_peered":"2024-09-12T02:00:09.569187+0000","last_clean":"2024-09-12T02:00:09.569187+0000","last_became_active":"2024-09-12T01:59:54.075387+0000","last_became_peered":"2024-09-12T01:59:54.075387+0000","last_unstale":"2024-09-12T02:00:09.569187+0000","last_undegraded":"2024-09-12T02:00:09.569187+0000","last_fullsized":"2024-09-12T02:00:09.569187+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:21:15.980591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.330203+0000","last_change":"2024-09-12T01:59:55.500935+0000","last_active":"2024-09-12T02:01:38.330203+0000","last_peered":"2024-09-12T02:01:38.330203+0000","last_clean":"2024-09-12T02:01:38.330203+0000","last_became_active":"2024-09-12T01:59:55.500206+0000","last_became_peered":"2024-09-12T01:59:55.500206+0000","last_unstale":"2024-09-12T02:01:38.330203+0000","last_undegraded":"2024-09-12T02:01:38.330203+0000","last_fullsized":"2024-09-12T02:01:38.330203+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:19:59.937701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":334,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.623254+0000","last_change":"2024-09-12T01:58:47.553853+0000","last_active":"2024-09-12T02:00:09.623254+0000","last_peered":"2024-09-12T02:00:09.623254+0000","last_clean":"2024-09-12T02:00:09.623254+0000","last_became_active":"2024-09-12T01:58:47.553357+0000","last_became_peered":"2024-09-12T01:58:47.553357+0000","last_unstale":"2024-09-12T02:00:09.623254+0000","last_undegraded":"2024-09-12T02:00:09.623254+0000","last_fullsized":"2024-09-12T02:00:09.623254+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:55:26.917733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:55:26.917733+0000","last_clean_scrub_stamp":"2024-09-12T01:55:26.917733+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:58:26.081766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"80'3","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.563615+0000","last_change":"2024-09-12T01:59:53.467907+0000","last_active":"2024-09-12T02:00:09.563615+0000","last_peered":"2024-09-12T02:00:09.563615+0000","last_clean":"2024-09-12T02:00:09.563615+0000","last_became_active":"2024-09-12T01:59:53.467164+0000","last_became_peered":"2024-09-12T01:59:53.467164+0000","last_unstale":"2024-09-12T02:00:09.563615+0000","last_undegraded":"2024-09-12T02:00:09.563615+0000","last_fullsized":"2024-09-12T02:00:09.563615+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:09:35.242708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.327715+0000","last_change":"2024-09-12T01:59:55.495109+0000","last_active":"2024-09-12T02:01:38.327715+0000","last_peered":"2024-09-12T02:01:38.327715+0000","last_clean":"2024-09-12T02:01:38.327715+0000","last_became_active":"2024-09-12T01:59:55.494791+0000","last_became_peered":"2024-09-12T01:59:55.494791+0000","last_unstale":"2024-09-12T02:01:38.327715+0000","last_undegraded":"2024-09-12T02:01:38.327715+0000","last_fullsized":"2024-09-12T02:01:38.327715+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:22:43.882050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"80'3","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.623115+0000","last_change":"2024-09-12T01:59:53.473286+0000","last_active":"2024-09-12T02:00:09.623115+0000","last_peered":"2024-09-12T02:00:09.623115+0000","last_clean":"2024-09-12T02:00:09.623115+0000","last_became_active":"2024-09-12T01:59:53.472976+0000","last_became_peered":"2024-09-12T01:59:53.472976+0000","last_unstale":"2024-09-12T02:00:09.623115+0000","last_undegraded":"2024-09-12T02:00:09.623115+0000","last_fullsized":"2024-09-12T02:00:09.623115+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:32:39.604173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"82'5","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.313749+0000","last_change":"2024-09-12T02:00:09.571159+0000","last_active":"2024-09-12T02:01:38.313749+0000","last_peered":"2024-09-12T02:01:38.313749+0000","last_clean":"2024-09-12T02:01:38.313749+0000","last_became_active":"2024-09-12T02:00:09.570672+0000","last_became_peered":"2024-09-12T02:00:09.570672+0000","last_unstale":"2024-09-12T02:01:38.313749+0000","last_undegraded":"2024-09-12T02:01:38.313749+0000","last_fullsized":"2024-09-12T02:01:38.313749+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:04:10.808942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"80'4","reported_seq":126,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:43.692856+0000","last_change":"2024-09-12T01:59:53.921856+0000","last_active":"2024-09-12T02:01:43.692856+0000","last_peered":"2024-09-12T02:01:43.692856+0000","last_clean":"2024-09-12T02:01:43.692856+0000","last_became_active":"2024-09-12T01:59:53.920672+0000","last_became_peered":"2024-09-12T01:59:53.920672+0000","last_unstale":"2024-09-12T02:01:43.692856+0000","last_undegraded":"2024-09-12T02:01:43.692856+0000","last_fullsized":"2024-09-12T02:01:43.692856+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:42:33.876493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":100,"num_read_kb":100,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"82'8","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.309498+0000","last_change":"2024-09-12T02:00:09.571185+0000","last_active":"2024-09-12T02:01:38.309498+0000","last_peered":"2024-09-12T02:01:38.309498+0000","last_clean":"2024-09-12T02:01:38.309498+0000","last_became_active":"2024-09-12T02:00:09.570769+0000","last_became_peered":"2024-09-12T02:00:09.570769+0000","last_unstale":"2024-09-12T02:01:38.309498+0000","last_undegraded":"2024-09-12T02:01:38.309498+0000","last_fullsized":"2024-09-12T02:01:38.309498+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:34:17.230435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.2","version":"80'1","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.622691+0000","last_change":"2024-09-12T01:59:53.472072+0000","last_active":"2024-09-12T02:00:09.622691+0000","last_peered":"2024-09-12T02:00:09.622691+0000","last_clean":"2024-09-12T02:00:09.622691+0000","last_became_active":"2024-09-12T01:59:53.471088+0000","last_became_peered":"2024-09-12T01:59:53.471088+0000","last_unstale":"2024-09-12T02:00:09.622691+0000","last_undegraded":"2024-09-12T02:00:09.622691+0000","last_fullsized":"2024-09-12T02:00:09.622691+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:02:13.879131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.330778+0000","last_change":"2024-09-12T01:59:55.502843+0000","last_active":"2024-09-12T02:01:38.330778+0000","last_peered":"2024-09-12T02:01:38.330778+0000","last_clean":"2024-09-12T02:01:38.330778+0000","last_became_active":"2024-09-12T01:59:55.499296+0000","last_became_peered":"2024-09-12T01:59:55.499296+0000","last_unstale":"2024-09-12T02:01:38.330778+0000","last_undegraded":"2024-09-12T02:01:38.330778+0000","last_fullsized":"2024-09-12T02:01:38.330778+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:04:21.787370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.575061+0000","last_change":"2024-09-12T02:00:09.575061+0000","last_active":"2024-09-12T02:00:09.575061+0000","last_peered":"2024-09-12T02:00:09.575061+0000","last_clean":"2024-09-12T02:00:09.575061+0000","last_became_active":"2024-09-12T02:00:09.574653+0000","last_became_peered":"2024-09-12T02:00:09.574653+0000","last_unstale":"2024-09-12T02:00:09.575061+0000","last_undegraded":"2024-09-12T02:00:09.575061+0000","last_fullsized":"2024-09-12T02:00:09.575061+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:32:27.931211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"82'5","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.320660+0000","last_change":"2024-09-12T01:59:55.494162+0000","last_active":"2024-09-12T02:01:38.320660+0000","last_peered":"2024-09-12T02:01:38.320660+0000","last_clean":"2024-09-12T02:01:38.320660+0000","last_became_active":"2024-09-12T01:59:55.493802+0000","last_became_peered":"2024-09-12T01:59:55.493802+0000","last_unstale":"2024-09-12T02:01:38.320660+0000","last_undegraded":"2024-09-12T02:01:38.320660+0000","last_fullsized":"2024-09-12T02:01:38.320660+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:22:16.302683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"80'1","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.575278+0000","last_change":"2024-09-12T01:59:53.921669+0000","last_active":"2024-09-12T02:00:09.575278+0000","last_peered":"2024-09-12T02:00:09.575278+0000","last_clean":"2024-09-12T02:00:09.575278+0000","last_became_active":"2024-09-12T01:59:53.921328+0000","last_became_peered":"2024-09-12T01:59:53.921328+0000","last_unstale":"2024-09-12T02:00:09.575278+0000","last_undegraded":"2024-09-12T02:00:09.575278+0000","last_fullsized":"2024-09-12T02:00:09.575278+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:46:33.313726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.311067+0000","last_change":"2024-09-12T01:59:55.488986+0000","last_active":"2024-09-12T02:01:38.311067+0000","last_peered":"2024-09-12T02:01:38.311067+0000","last_clean":"2024-09-12T02:01:38.311067+0000","last_became_active":"2024-09-12T01:59:55.488632+0000","last_became_peered":"2024-09-12T01:59:55.488632+0000","last_unstale":"2024-09-12T02:01:38.311067+0000","last_undegraded":"2024-09-12T02:01:38.311067+0000","last_fullsized":"2024-09-12T02:01:38.311067+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:20:57.596976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"82'11","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:21.501527+0000","last_change":"2024-09-12T01:59:53.467648+0000","last_active":"2024-09-12T02:00:21.501527+0000","last_peered":"2024-09-12T02:00:21.501527+0000","last_clean":"2024-09-12T02:00:21.501527+0000","last_became_active":"2024-09-12T01:59:53.467255+0000","last_became_peered":"2024-09-12T01:59:53.467255+0000","last_unstale":"2024-09-12T02:00:21.501527+0000","last_undegraded":"2024-09-12T02:00:21.501527+0000","last_fullsized":"2024-09-12T02:00:21.501527+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:37:09.897388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"82'5","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.328722+0000","last_change":"2024-09-12T01:59:55.487603+0000","last_active":"2024-09-12T02:01:38.328722+0000","last_peered":"2024-09-12T02:01:38.328722+0000","last_clean":"2024-09-12T02:01:38.328722+0000","last_became_active":"2024-09-12T01:59:55.487334+0000","last_became_peered":"2024-09-12T01:59:55.487334+0000","last_unstale":"2024-09-12T02:01:38.328722+0000","last_undegraded":"2024-09-12T02:01:38.328722+0000","last_fullsized":"2024-09-12T02:01:38.328722+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:23:16.669515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'4","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.563870+0000","last_change":"2024-09-12T01:59:54.076156+0000","last_active":"2024-09-12T02:00:09.563870+0000","last_peered":"2024-09-12T02:00:09.563870+0000","last_clean":"2024-09-12T02:00:09.563870+0000","last_became_active":"2024-09-12T01:59:54.075794+0000","last_became_peered":"2024-09-12T01:59:54.075794+0000","last_unstale":"2024-09-12T02:00:09.563870+0000","last_undegraded":"2024-09-12T02:00:09.563870+0000","last_fullsized":"2024-09-12T02:00:09.563870+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:06:17.431696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.333603+0000","last_change":"2024-09-12T01:59:55.494604+0000","last_active":"2024-09-12T02:01:38.333603+0000","last_peered":"2024-09-12T02:01:38.333603+0000","last_clean":"2024-09-12T02:01:38.333603+0000","last_became_active":"2024-09-12T01:59:55.494300+0000","last_became_peered":"2024-09-12T01:59:55.494300+0000","last_unstale":"2024-09-12T02:01:38.333603+0000","last_undegraded":"2024-09-12T02:01:38.333603+0000","last_fullsized":"2024-09-12T02:01:38.333603+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:18:56.359299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.622205+0000","last_change":"2024-09-12T01:59:53.921832+0000","last_active":"2024-09-12T02:00:09.622205+0000","last_peered":"2024-09-12T02:00:09.622205+0000","last_clean":"2024-09-12T02:00:09.622205+0000","last_became_active":"2024-09-12T01:59:53.921379+0000","last_became_peered":"2024-09-12T01:59:53.921379+0000","last_unstale":"2024-09-12T02:00:09.622205+0000","last_undegraded":"2024-09-12T02:00:09.622205+0000","last_fullsized":"2024-09-12T02:00:09.622205+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:26:02.277149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"82'2","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.281554+0000","last_change":"2024-09-12T01:59:55.493198+0000","last_active":"2024-09-12T02:01:38.281554+0000","last_peered":"2024-09-12T02:01:38.281554+0000","last_clean":"2024-09-12T02:01:38.281554+0000","last_became_active":"2024-09-12T01:59:55.492834+0000","last_became_peered":"2024-09-12T01:59:55.492834+0000","last_unstale":"2024-09-12T02:01:38.281554+0000","last_undegraded":"2024-09-12T02:01:38.281554+0000","last_fullsized":"2024-09-12T02:01:38.281554+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:58:43.872921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"80'3","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.621258+0000","last_change":"2024-09-12T01:59:53.466528+0000","last_active":"2024-09-12T02:00:09.621258+0000","last_peered":"2024-09-12T02:00:09.621258+0000","last_clean":"2024-09-12T02:00:09.621258+0000","last_became_active":"2024-09-12T01:59:53.466182+0000","last_became_peered":"2024-09-12T01:59:53.466182+0000","last_unstale":"2024-09-12T02:00:09.621258+0000","last_undegraded":"2024-09-12T02:00:09.621258+0000","last_fullsized":"2024-09-12T02:00:09.621258+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:00:26.083539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.316786+0000","last_change":"2024-09-12T01:59:55.494511+0000","last_active":"2024-09-12T02:01:38.316786+0000","last_peered":"2024-09-12T02:01:38.316786+0000","last_clean":"2024-09-12T02:01:38.316786+0000","last_became_active":"2024-09-12T01:59:55.493607+0000","last_became_peered":"2024-09-12T01:59:55.493607+0000","last_unstale":"2024-09-12T02:01:38.316786+0000","last_undegraded":"2024-09-12T02:01:38.316786+0000","last_fullsized":"2024-09-12T02:01:38.316786+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:55:27.434447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"80'1","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.569211+0000","last_change":"2024-09-12T01:59:53.467385+0000","last_active":"2024-09-12T02:00:09.569211+0000","last_peered":"2024-09-12T02:00:09.569211+0000","last_clean":"2024-09-12T02:00:09.569211+0000","last_became_active":"2024-09-12T01:59:53.466997+0000","last_became_peered":"2024-09-12T01:59:53.466997+0000","last_unstale":"2024-09-12T02:00:09.569211+0000","last_undegraded":"2024-09-12T02:00:09.569211+0000","last_fullsized":"2024-09-12T02:00:09.569211+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:54:47.154036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.316513+0000","last_change":"2024-09-12T01:59:55.490757+0000","last_active":"2024-09-12T02:01:38.316513+0000","last_peered":"2024-09-12T02:01:38.316513+0000","last_clean":"2024-09-12T02:01:38.316513+0000","last_became_active":"2024-09-12T01:59:55.490451+0000","last_became_peered":"2024-09-12T01:59:55.490451+0000","last_unstale":"2024-09-12T02:01:38.316513+0000","last_undegraded":"2024-09-12T02:01:38.316513+0000","last_fullsized":"2024-09-12T02:01:38.316513+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:21:26.139049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.574833+0000","last_change":"2024-09-12T01:59:53.472898+0000","last_active":"2024-09-12T02:00:09.574833+0000","last_peered":"2024-09-12T02:00:09.574833+0000","last_clean":"2024-09-12T02:00:09.574833+0000","last_became_active":"2024-09-12T01:59:53.472561+0000","last_became_peered":"2024-09-12T01:59:53.472561+0000","last_unstale":"2024-09-12T02:00:09.574833+0000","last_undegraded":"2024-09-12T02:00:09.574833+0000","last_fullsized":"2024-09-12T02:00:09.574833+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:55:59.933804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"82'5","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.328406+0000","last_change":"2024-09-12T01:59:55.497731+0000","last_active":"2024-09-12T02:01:38.328406+0000","last_peered":"2024-09-12T02:01:38.328406+0000","last_clean":"2024-09-12T02:01:38.328406+0000","last_became_active":"2024-09-12T01:59:55.497356+0000","last_became_peered":"2024-09-12T01:59:55.497356+0000","last_unstale":"2024-09-12T02:01:38.328406+0000","last_undegraded":"2024-09-12T02:01:38.328406+0000","last_fullsized":"2024-09-12T02:01:38.328406+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:57:18.795865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"82'15","reported_seq":38,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.318747+0000","last_change":"2024-09-12T02:00:09.624453+0000","last_active":"2024-09-12T02:01:38.318747+0000","last_peered":"2024-09-12T02:01:38.318747+0000","last_clean":"2024-09-12T02:01:38.318747+0000","last_became_active":"2024-09-12T02:00:09.624049+0000","last_became_peered":"2024-09-12T02:00:09.624049+0000","last_unstale":"2024-09-12T02:01:38.318747+0000","last_undegraded":"2024-09-12T02:01:38.318747+0000","last_fullsized":"2024-09-12T02:01:38.318747+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:17:40.417275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"80'2","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.575060+0000","last_change":"2024-09-12T01:59:53.470194+0000","last_active":"2024-09-12T02:00:09.575060+0000","last_peered":"2024-09-12T02:00:09.575060+0000","last_clean":"2024-09-12T02:00:09.575060+0000","last_became_active":"2024-09-12T01:59:53.469772+0000","last_became_peered":"2024-09-12T01:59:53.469772+0000","last_unstale":"2024-09-12T02:00:09.575060+0000","last_undegraded":"2024-09-12T02:00:09.575060+0000","last_fullsized":"2024-09-12T02:00:09.575060+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:52:57.709596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.329782+0000","last_change":"2024-09-12T01:59:55.497693+0000","last_active":"2024-09-12T02:01:38.329782+0000","last_peered":"2024-09-12T02:01:38.329782+0000","last_clean":"2024-09-12T02:01:38.329782+0000","last_became_active":"2024-09-12T01:59:55.497380+0000","last_became_peered":"2024-09-12T01:59:55.497380+0000","last_unstale":"2024-09-12T02:01:38.329782+0000","last_undegraded":"2024-09-12T02:01:38.329782+0000","last_fullsized":"2024-09-12T02:01:38.329782+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:42:27.943530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"80'4","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.575274+0000","last_change":"2024-09-12T01:59:53.473044+0000","last_active":"2024-09-12T02:00:09.575274+0000","last_peered":"2024-09-12T02:00:09.575274+0000","last_clean":"2024-09-12T02:00:09.575274+0000","last_became_active":"2024-09-12T01:59:53.472569+0000","last_became_peered":"2024-09-12T01:59:53.472569+0000","last_unstale":"2024-09-12T02:00:09.575274+0000","last_undegraded":"2024-09-12T02:00:09.575274+0000","last_fullsized":"2024-09-12T02:00:09.575274+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:41:29.716836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.328172+0000","last_change":"2024-09-12T01:59:55.497377+0000","last_active":"2024-09-12T02:01:38.328172+0000","last_peered":"2024-09-12T02:01:38.328172+0000","last_clean":"2024-09-12T02:01:38.328172+0000","last_became_active":"2024-09-12T01:59:55.497078+0000","last_became_peered":"2024-09-12T01:59:55.497078+0000","last_unstale":"2024-09-12T02:01:38.328172+0000","last_undegraded":"2024-09-12T02:01:38.328172+0000","last_fullsized":"2024-09-12T02:01:38.328172+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:59:57.362350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.621555+0000","last_change":"2024-09-12T01:59:53.920868+0000","last_active":"2024-09-12T02:00:09.621555+0000","last_peered":"2024-09-12T02:00:09.621555+0000","last_clean":"2024-09-12T02:00:09.621555+0000","last_became_active":"2024-09-12T01:59:53.920481+0000","last_became_peered":"2024-09-12T01:59:53.920481+0000","last_unstale":"2024-09-12T02:00:09.621555+0000","last_undegraded":"2024-09-12T02:00:09.621555+0000","last_fullsized":"2024-09-12T02:00:09.621555+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:49:27.085388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"82'5","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.315986+0000","last_change":"2024-09-12T01:59:55.497703+0000","last_active":"2024-09-12T02:01:38.315986+0000","last_peered":"2024-09-12T02:01:38.315986+0000","last_clean":"2024-09-12T02:01:38.315986+0000","last_became_active":"2024-09-12T01:59:55.497397+0000","last_became_peered":"2024-09-12T01:59:55.497397+0000","last_unstale":"2024-09-12T02:01:38.315986+0000","last_undegraded":"2024-09-12T02:01:38.315986+0000","last_fullsized":"2024-09-12T02:01:38.315986+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:54:02.803203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"80'9","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.565145+0000","last_change":"2024-09-12T01:59:53.469978+0000","last_active":"2024-09-12T02:00:09.565145+0000","last_peered":"2024-09-12T02:00:09.565145+0000","last_clean":"2024-09-12T02:00:09.565145+0000","last_became_active":"2024-09-12T01:59:53.469648+0000","last_became_peered":"2024-09-12T01:59:53.469648+0000","last_unstale":"2024-09-12T02:00:09.565145+0000","last_undegraded":"2024-09-12T02:00:09.565145+0000","last_fullsized":"2024-09-12T02:00:09.565145+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:08:37.720452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":68779,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":627,"num_read_kb":386,"num_write":170,"num_write_kb":119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":206337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":144,"ondisk_log_size":144,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3563520,"data_stored":3542208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":74,"seq":317827579939,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32060,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713092,"statfs":{"total":95995035648,"available":95962206208,"internally_reserved":0,"allocated":897024,"data_stored":435916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":67,"seq":287762808872,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32736,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712416,"statfs":{"total":95995035648,"available":95961513984,"internally_reserved":0,"allocated":1589248,"data_stored":1091920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":62,"seq":266287972397,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":905216,"data_stored":436040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201329,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":897024,"data_stored":435848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430263,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32648,"kb_used_data":1464,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712504,"statfs":{"total":95995035648,"available":95961604096,"internally_reserved":0,"allocated":1499136,"data_stored":1028033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593788,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36216,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708936,"statfs":{"total":95995035648,"available":95957950464,"internally_reserved":0,"allocated":892928,"data_stored":435429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822721,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32660,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712492,"statfs":{"total":95995035648,"available":95961591808,"internally_reserved":0,"allocated":1511424,"data_stored":1027717,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986245,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":897024,"data_stored":435866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215178,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":909312,"data_stored":435740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":17,"seq":73014444112,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36304,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708848,"statfs":{"total":95995035648,"available":95957860352,"internally_reserved":0,"allocated":983040,"data_stored":501741,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574933,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32120,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713032,"statfs":{"total":95995035648,"available":95962144768,"internally_reserved":0,"allocated":958464,"data_stored":499857,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738458,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":913408,"data_stored":436700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":64635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":66519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":66213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T02:01:47.678 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-12T02:01:47.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.823+0000 7f9878934640 1 -- 172.21.15.16:0/2298701090 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9874101410 msgr2=0x7f987410d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:47.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.823+0000 7f9878934640 1 --2- 172.21.15.16:0/2298701090 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9874101410 0x7f987410d940 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f986c009cd0 tx=0x7f986c02a8e0 comp rx=0 tx=0).stop 2024-09-12T02:01:47.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.824+0000 7f9878934640 1 -- 172.21.15.16:0/2298701090 shutdown_connections 2024-09-12T02:01:47.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.824+0000 7f9878934640 1 --2- 172.21.15.16:0/2298701090 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9874101410 0x7f987410d940 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:47.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.824+0000 7f9878934640 1 --2- 172.21.15.16:0/2298701090 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9874100a70 0x7f9874100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:47.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.824+0000 7f9878934640 1 --2- 172.21.15.16:0/2298701090 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9874106a70 0x7f9874106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:47.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.824+0000 7f9878934640 1 -- 172.21.15.16:0/2298701090 >> 172.21.15.16:0/2298701090 conn(0x7f98740fc7d0 msgr2=0x7f98740febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:01:47.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.824+0000 7f9878934640 1 -- 172.21.15.16:0/2298701090 shutdown_connections 2024-09-12T02:01:47.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.824+0000 7f9878934640 1 -- 172.21.15.16:0/2298701090 wait complete. 2024-09-12T02:01:47.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.824+0000 7f9878934640 1 Processor -- start 2024-09-12T02:01:47.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.824+0000 7f9878934640 1 -- start start 2024-09-12T02:01:47.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.825+0000 7f9878934640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9874100a70 0x7f9874199160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:47.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.825+0000 7f9878934640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9874101410 0x7f98741996a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:47.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.825+0000 7f9878934640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9874106a70 0x7f98741a41e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:47.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.825+0000 7f9878934640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9874110e70 con 0x7f9874100a70 2024-09-12T02:01:47.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.825+0000 7f9878934640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9874110cf0 con 0x7f9874106a70 2024-09-12T02:01:47.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.825+0000 7f9878934640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f9874110ff0 con 0x7f9874101410 2024-09-12T02:01:47.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.825+0000 7f987259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9874100a70 0x7f9874199160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:47.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.825+0000 7f987259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9874100a70 0x7f9874199160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46642/0 (socket says 172.21.15.16:46642) 2024-09-12T02:01:47.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.825+0000 7f987259c640 1 -- 172.21.15.16:0/490299353 learned_addr learned my addr 172.21.15.16:0/490299353 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:01:47.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.825+0000 7f9872d9d640 1 --2- 172.21.15.16:0/490299353 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9874106a70 0x7f98741a41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:47.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.825+0000 7f9871d9b640 1 --2- 172.21.15.16:0/490299353 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9874101410 0x7f98741996a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:47.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.825+0000 7f987259c640 1 -- 172.21.15.16:0/490299353 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9874101410 msgr2=0x7f98741996a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:47.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.825+0000 7f987259c640 1 --2- 172.21.15.16:0/490299353 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9874101410 0x7f98741996a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:47.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.825+0000 7f987259c640 1 -- 172.21.15.16:0/490299353 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9874106a70 msgr2=0x7f98741a41e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:47.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.825+0000 7f987259c640 1 --2- 172.21.15.16:0/490299353 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9874106a70 0x7f98741a41e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:47.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.825+0000 7f987259c640 1 -- 172.21.15.16:0/490299353 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98741a4720 con 0x7f9874100a70 2024-09-12T02:01:47.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.825+0000 7f9871d9b640 1 --2- 172.21.15.16:0/490299353 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9874101410 0x7f98741996a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:01:47.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.825+0000 7f987259c640 1 --2- 172.21.15.16:0/490299353 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9874100a70 0x7f9874199160 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f986400c920 tx=0x7f986400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:01:47.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.826+0000 7f98637fe640 1 -- 172.21.15.16:0/490299353 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9864007c70 con 0x7f9874100a70 2024-09-12T02:01:47.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.826+0000 7f98637fe640 1 -- 172.21.15.16:0/490299353 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9864007e10 con 0x7f9874100a70 2024-09-12T02:01:47.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.826+0000 7f9878934640 1 -- 172.21.15.16:0/490299353 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98741a48f0 con 0x7f9874100a70 2024-09-12T02:01:47.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.826+0000 7f98637fe640 1 -- 172.21.15.16:0/490299353 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9864018890 con 0x7f9874100a70 2024-09-12T02:01:47.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.826+0000 7f9878934640 1 -- 172.21.15.16:0/490299353 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98741a4e30 con 0x7f9874100a70 2024-09-12T02:01:47.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.829+0000 7f987ad0d740 1 Processor -- start 2024-09-12T02:01:47.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.829+0000 7f987ad0d740 1 -- start start 2024-09-12T02:01:47.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.829+0000 7f98637fe640 1 -- 172.21.15.16:0/490299353 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f9864018b30 con 0x7f9874100a70 2024-09-12T02:01:47.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.829+0000 7f987ad0d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab75e674a0 0x55ab75e67880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:47.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.829+0000 7f987ad0d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab75e67dc0 0x55ab75e72140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:47.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.829+0000 7f987ad0d740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ab75e72680 0x55ab75e74a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:47.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.829+0000 7f987ad0d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55ab75dcd2e0 con 0x55ab75e67dc0 2024-09-12T02:01:47.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.829+0000 7f987ad0d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55ab75d74ce0 con 0x55ab75e674a0 2024-09-12T02:01:47.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.829+0000 7f987ad0d740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55ab75dc92a0 con 0x55ab75e72680 2024-09-12T02:01:47.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.830+0000 7f987259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab75e67dc0 0x55ab75e72140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:47.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.830+0000 7f9872d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab75e674a0 0x55ab75e67880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:47.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.830+0000 7f9872d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab75e674a0 0x55ab75e67880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.16:34088/0 (socket says 172.21.15.16:34088) 2024-09-12T02:01:47.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.830+0000 7f9872d9d640 1 -- 172.21.15.16:0/3276219179 learned_addr learned my addr 172.21.15.16:0/3276219179 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:01:47.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.830+0000 7f9871d9b640 1 --2- 172.21.15.16:0/3276219179 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ab75e72680 0x55ab75e74a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:47.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.830+0000 7f9872d9d640 1 -- 172.21.15.16:0/3276219179 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ab75e72680 msgr2=0x55ab75e74a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:01:47.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.830+0000 7f9872d9d640 1 --2- 172.21.15.16:0/3276219179 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ab75e72680 0x55ab75e74a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:47.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.830+0000 7f9872d9d640 1 -- 172.21.15.16:0/3276219179 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab75e67dc0 msgr2=0x55ab75e72140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:01:47.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.830+0000 7f9872d9d640 1 --2- 172.21.15.16:0/3276219179 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab75e67dc0 0x55ab75e72140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:47.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.830+0000 7f9872d9d640 1 -- 172.21.15.16:0/3276219179 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab75d59440 con 0x55ab75e674a0 2024-09-12T02:01:47.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.830+0000 7f9872d9d640 1 --2- 172.21.15.16:0/3276219179 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab75e674a0 0x55ab75e67880 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f986c035000 tx=0x7f986c031700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd7978e4adfd94dd server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:01:47.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.830+0000 7f9860ff9640 1 -- 172.21.15.16:0/3276219179 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f986c048070 con 0x55ab75e674a0 2024-09-12T02:01:47.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.831+0000 7f9860ff9640 1 -- 172.21.15.16:0/3276219179 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f986c036070 con 0x55ab75e674a0 2024-09-12T02:01:47.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.831+0000 7f9860ff9640 1 -- 172.21.15.16:0/3276219179 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f986c004740 con 0x55ab75e674a0 2024-09-12T02:01:47.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.831+0000 7f98637fe640 1 --2- 172.21.15.16:0/490299353 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f985c0a2c00 0x7f985c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:47.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.831+0000 7f987259c640 1 --2- 172.21.15.16:0/490299353 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f985c0a2c00 0x7f985c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:47.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.831+0000 7f98637fe640 1 -- 172.21.15.16:0/490299353 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f98640ceff0 con 0x7f9874100a70 2024-09-12T02:01:47.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.831+0000 7f987ad0d740 1 -- 172.21.15.16:0/3276219179 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab75e674a0 msgr2=0x55ab75e67880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:47.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.831+0000 7f987ad0d740 1 --2- 172.21.15.16:0/3276219179 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab75e674a0 0x55ab75e67880 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f986c035000 tx=0x7f986c031700 comp rx=0 tx=0).stop 2024-09-12T02:01:47.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.831+0000 7f987ad0d740 1 -- 172.21.15.16:0/3276219179 shutdown_connections 2024-09-12T02:01:47.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.831+0000 7f987ad0d740 1 --2- 172.21.15.16:0/3276219179 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ab75e72680 0x55ab75e74a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:47.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.831+0000 7f987ad0d740 1 --2- 172.21.15.16:0/3276219179 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab75e67dc0 0x55ab75e72140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:47.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.831+0000 7f987ad0d740 1 --2- 172.21.15.16:0/3276219179 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab75e674a0 0x55ab75e67880 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:47.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.831+0000 7f987ad0d740 1 -- 172.21.15.16:0/3276219179 >> 172.21.15.16:0/3276219179 conn(0x55ab75e63a30 msgr2=0x55ab75e63e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:01:47.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.832+0000 7f987ad0d740 1 -- 172.21.15.16:0/3276219179 shutdown_connections 2024-09-12T02:01:47.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.832+0000 7f987259c640 1 --2- 172.21.15.16:0/490299353 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f985c0a2c00 0x7f985c0a50c0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f98640a4030 tx=0x7f9864086040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:01:47.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.832+0000 7f987ad0d740 1 -- 172.21.15.16:0/3276219179 wait complete. 2024-09-12T02:01:47.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.832+0000 7f987ad0d740 1 -- start start 2024-09-12T02:01:47.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.832+0000 7f987ad0d740 1 Processor -- start 2024-09-12T02:01:47.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.833+0000 7f987ad0d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab75e674a0 0x55ab75e71f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:47.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.833+0000 7f987ad0d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab75e67dc0 0x55ab75e6f240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:47.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.833+0000 7f987ad0d740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ab75e72680 0x55ab75e6f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:47.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.833+0000 7f987ad0d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55ab75e79660 con 0x55ab75e67dc0 2024-09-12T02:01:47.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.833+0000 7f987ad0d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55ab75d88050 con 0x55ab75e674a0 2024-09-12T02:01:47.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.833+0000 7f987ad0d740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55ab75e797e0 con 0x55ab75e72680 2024-09-12T02:01:47.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.833+0000 7f987ad0d740 10 client.0 _ll_register_callbacks cb 0x55ab75dc8160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:01:47.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.833+0000 7f9871d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab75e67dc0 0x55ab75e6f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:47.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.833+0000 7f9872d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab75e674a0 0x55ab75e71f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:47.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.833+0000 7f9871d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab75e67dc0 0x55ab75e6f240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46672/0 (socket says 172.21.15.16:46672) 2024-09-12T02:01:47.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.833+0000 7f9872d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab75e674a0 0x55ab75e71f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.16:34098/0 (socket says 172.21.15.16:34098) 2024-09-12T02:01:47.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.833+0000 7f9871d9b640 1 -- 172.21.15.16:0/2186532845 learned_addr learned my addr 172.21.15.16:0/2186532845 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:01:47.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.833+0000 7f9872d9d640 1 --2- 172.21.15.16:0/2186532845 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ab75e72680 0x55ab75e6f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:47.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.833+0000 7f9871d9b640 1 -- 172.21.15.16:0/2186532845 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ab75e72680 msgr2=0x55ab75e6f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:01:47.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.833+0000 7f9871d9b640 1 --2- 172.21.15.16:0/2186532845 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ab75e72680 0x55ab75e6f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:47.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.833+0000 7f9871d9b640 1 -- 172.21.15.16:0/2186532845 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab75e674a0 msgr2=0x55ab75e71f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:01:47.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.833+0000 7f9871d9b640 1 --2- 172.21.15.16:0/2186532845 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab75e674a0 0x55ab75e71f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:47.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.833+0000 7f9871d9b640 1 -- 172.21.15.16:0/2186532845 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab75b3e3c0 con 0x55ab75e67dc0 2024-09-12T02:01:47.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.833+0000 7f9872d9d640 1 --2- 172.21.15.16:0/2186532845 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab75e674a0 0x55ab75e71f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:01:47.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.833+0000 7f9871d9b640 1 --2- 172.21.15.16:0/2186532845 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab75e67dc0 0x55ab75e6f240 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f9868010740 tx=0x7f9868010c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7eab0ca624eca5ce server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:01:47.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.833+0000 7f9859ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:01:47.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.834+0000 7f9859ffb640 1 -- 172.21.15.16:0/2186532845 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f986800ed40 con 0x55ab75e67dc0 2024-09-12T02:01:47.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.834+0000 7f9859ffb640 1 -- 172.21.15.16:0/2186532845 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f9868004020 con 0x55ab75e67dc0 2024-09-12T02:01:47.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.834+0000 7f987ad0d740 1 -- 172.21.15.16:0/2186532845 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f986c036070 con 0x55ab75e67dc0 2024-09-12T02:01:47.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.834+0000 7f9859ffb640 1 -- 172.21.15.16:0/2186532845 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9868010e50 con 0x55ab75e67dc0 2024-09-12T02:01:47.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.834+0000 7f987ad0d740 10 client.15234 fetch_fsmap learned FSMap version 25 2024-09-12T02:01:47.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.834+0000 7f987ad0d740 1 -- 172.21.15.16:0/2186532845 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ab75e85090 con 0x55ab75e67dc0 2024-09-12T02:01:47.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.834+0000 7f9859ffb640 1 -- 172.21.15.16:0/2186532845 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f986c036070 con 0x55ab75e67dc0 2024-09-12T02:01:47.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.834+0000 7f9859ffb640 1 -- 172.21.15.16:0/2186532845 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f9868011440 con 0x55ab75e67dc0 2024-09-12T02:01:47.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.834+0000 7f987ad0d740 10 client.15234 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:01:47.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.834+0000 7f987ad0d740 10 client.15234 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:01:47.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.834+0000 7f987ad0d740 20 client.15234 populate_metadata read hostname 'smithi016' 2024-09-12T02:01:47.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.834+0000 7f987ad0d740 1 --2- 172.21.15.16:0/2186532845 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55ab75e86550 0x55ab75e88940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:47.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.834+0000 7f987ad0d740 4 client.15234 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:01:47.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.834+0000 7f987ad0d740 1 -- 172.21.15.16:0/2186532845 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ab75dda850 con 0x55ab75e86550 2024-09-12T02:01:47.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.835+0000 7f9872d9d640 1 --2- 172.21.15.16:0/2186532845 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55ab75e86550 0x55ab75e88940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:47.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.835+0000 7f9872d9d640 1 --2- 172.21.15.16:0/2186532845 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55ab75e86550 0x55ab75e88940 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=55d8297f6db99c02 server_cookie=22419c4ff9f4c590 in_seq=0 out_seq=0 2024-09-12T02:01:47.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.835+0000 7f9859ffb640 10 client.15234 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:01:47.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.836+0000 7f9859ffb640 1 -- 172.21.15.16:0/2186532845 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ab75dda850 con 0x55ab75e86550 2024-09-12T02:01:47.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.836+0000 7f9859ffb640 10 client.15234 handle_command_reply: tid=0 2024-09-12T02:01:47.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.836+0000 7f987ad0d740 1 client.15234 shutdown 2024-09-12T02:01:47.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.836+0000 7f987ad0d740 1 -- 172.21.15.16:0/2186532845 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab75e67dc0 msgr2=0x55ab75e6f240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:47.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.836+0000 7f987ad0d740 1 --2- 172.21.15.16:0/2186532845 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab75e67dc0 0x55ab75e6f240 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f9868010740 tx=0x7f9868010c00 comp rx=0 tx=0).stop 2024-09-12T02:01:47.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.836+0000 7f987ad0d740 1 -- 172.21.15.16:0/2186532845 shutdown_connections 2024-09-12T02:01:47.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.836+0000 7f987ad0d740 1 --2- 172.21.15.16:0/2186532845 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55ab75e86550 0x55ab75e88940 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:47.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.836+0000 7f987ad0d740 1 --2- 172.21.15.16:0/2186532845 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ab75e72680 0x55ab75e6f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:47.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.836+0000 7f987ad0d740 1 --2- 172.21.15.16:0/2186532845 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab75e67dc0 0x55ab75e6f240 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:47.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.836+0000 7f987ad0d740 1 --2- 172.21.15.16:0/2186532845 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab75e674a0 0x55ab75e71f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:47.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.836+0000 7f987ad0d740 1 -- 172.21.15.16:0/2186532845 >> 172.21.15.16:0/2186532845 conn(0x55ab75e63a30 msgr2=0x55ab75e62b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:01:47.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.836+0000 7f9859ffb640 0 client.15234 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:01:47.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.836+0000 7f987ad0d740 1 -- 172.21.15.16:0/2186532845 shutdown_connections 2024-09-12T02:01:47.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.836+0000 7f987ad0d740 1 -- 172.21.15.16:0/2186532845 wait complete. 2024-09-12T02:01:47.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.836+0000 7f987ad0d740 20 client.15234 trim_cache size 0 max 16384 2024-09-12T02:01:47.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.853+0000 7f987ad0d740 1 Processor -- start 2024-09-12T02:01:47.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.853+0000 7f987ad0d740 1 -- start start 2024-09-12T02:01:47.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.854+0000 7f987ad0d740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ab75e674a0 0x55ab75e828c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:47.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.854+0000 7f987ad0d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab75e67dc0 0x55ab75e6db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:47.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.854+0000 7f987ad0d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab75e72680 0x55ab75e711b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:47.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.854+0000 7f987ad0d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f98741ad860 con 0x55ab75e72680 2024-09-12T02:01:47.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.854+0000 7f987ad0d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f98741ab130 con 0x55ab75e67dc0 2024-09-12T02:01:47.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.854+0000 7f987ad0d740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f98741a8fa0 con 0x55ab75e674a0 2024-09-12T02:01:47.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.854+0000 7f9872d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab75e72680 0x55ab75e711b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:47.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.854+0000 7f9872d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab75e72680 0x55ab75e711b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46688/0 (socket says 172.21.15.16:46688) 2024-09-12T02:01:47.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.854+0000 7f9872d9d640 1 -- 172.21.15.16:0/2701047249 learned_addr learned my addr 172.21.15.16:0/2701047249 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:01:47.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.854+0000 7f9871d9b640 1 --2- 172.21.15.16:0/2701047249 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab75e67dc0 0x55ab75e6db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:47.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.854+0000 7f9872d9d640 1 --2- 172.21.15.16:0/2701047249 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ab75e674a0 0x55ab75e828c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:47.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.854+0000 7f9872d9d640 1 -- 172.21.15.16:0/2701047249 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ab75e674a0 msgr2=0x55ab75e828c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:01:47.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.854+0000 7f9872d9d640 1 --2- 172.21.15.16:0/2701047249 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ab75e674a0 0x55ab75e828c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:47.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.854+0000 7f9872d9d640 1 -- 172.21.15.16:0/2701047249 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab75e67dc0 msgr2=0x55ab75e6db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:01:47.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.854+0000 7f9872d9d640 1 --2- 172.21.15.16:0/2701047249 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab75e67dc0 0x55ab75e6db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:47.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.854+0000 7f9872d9d640 1 -- 172.21.15.16:0/2701047249 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab75e6e1f0 con 0x55ab75e72680 2024-09-12T02:01:47.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.854+0000 7f9871d9b640 1 --2- 172.21.15.16:0/2701047249 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab75e67dc0 0x55ab75e6db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:01:47.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.854+0000 7f9872d9d640 1 --2- 172.21.15.16:0/2701047249 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab75e72680 0x55ab75e711b0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f986c000f80 tx=0x7f986c005fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a0d9617fbbe1480 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:01:47.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.855+0000 7f985bfff640 1 -- 172.21.15.16:0/2701047249 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f986c048040 con 0x55ab75e72680 2024-09-12T02:01:47.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.855+0000 7f985bfff640 1 -- 172.21.15.16:0/2701047249 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f986c036070 con 0x55ab75e72680 2024-09-12T02:01:47.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.855+0000 7f985bfff640 1 -- 172.21.15.16:0/2701047249 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f986c0305a0 con 0x55ab75e72680 2024-09-12T02:01:47.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.856+0000 7f987ad0d740 1 -- 172.21.15.16:0/2701047249 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab75e72680 msgr2=0x55ab75e711b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:47.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.856+0000 7f987ad0d740 1 --2- 172.21.15.16:0/2701047249 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab75e72680 0x55ab75e711b0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f986c000f80 tx=0x7f986c005fa0 comp rx=0 tx=0).stop 2024-09-12T02:01:47.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.856+0000 7f987ad0d740 1 -- 172.21.15.16:0/2701047249 shutdown_connections 2024-09-12T02:01:47.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.856+0000 7f987ad0d740 1 --2- 172.21.15.16:0/2701047249 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab75e72680 0x55ab75e711b0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:47.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.856+0000 7f987ad0d740 1 --2- 172.21.15.16:0/2701047249 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab75e67dc0 0x55ab75e6db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:47.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.856+0000 7f987ad0d740 1 --2- 172.21.15.16:0/2701047249 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ab75e674a0 0x55ab75e828c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:47.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.856+0000 7f987ad0d740 1 -- 172.21.15.16:0/2701047249 >> 172.21.15.16:0/2701047249 conn(0x55ab75e63a30 msgr2=0x55ab75e70aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:01:47.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.856+0000 7f987ad0d740 1 -- 172.21.15.16:0/2701047249 shutdown_connections 2024-09-12T02:01:47.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.856+0000 7f987ad0d740 1 -- 172.21.15.16:0/2701047249 wait complete. 2024-09-12T02:01:47.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.856+0000 7f987ad0d740 1 -- start start 2024-09-12T02:01:47.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.856+0000 7f987ad0d740 1 Processor -- start 2024-09-12T02:01:47.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.857+0000 7f987ad0d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab75e674a0 0x55ab75e79f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:47.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.857+0000 7f987ad0d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab75e67dc0 0x55ab75e7a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:47.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.857+0000 7f987ad0d740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ab75e72680 0x55ab75eef1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:47.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.857+0000 7f987ad0d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f98741a95f0 con 0x55ab75e674a0 2024-09-12T02:01:47.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.857+0000 7f987ad0d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f98741aaa60 con 0x55ab75e67dc0 2024-09-12T02:01:47.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.857+0000 7f987ad0d740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f98741a9cc0 con 0x55ab75e72680 2024-09-12T02:01:47.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.857+0000 7f987ad0d740 10 client.0 _ll_register_callbacks cb 0x7f986800c410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:01:47.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.857+0000 7f9872d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab75e674a0 0x55ab75e79f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:47.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.857+0000 7f9872d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab75e674a0 0x55ab75e79f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46704/0 (socket says 172.21.15.16:46704) 2024-09-12T02:01:47.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.857+0000 7f9872d9d640 1 -- 172.21.15.16:0/1192746535 learned_addr learned my addr 172.21.15.16:0/1192746535 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:01:47.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.857+0000 7f9871d9b640 1 --2- 172.21.15.16:0/1192746535 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab75e67dc0 0x55ab75e7a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:47.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.857+0000 7f9872d9d640 1 --2- 172.21.15.16:0/1192746535 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ab75e72680 0x55ab75eef1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:47.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.857+0000 7f9872d9d640 1 -- 172.21.15.16:0/1192746535 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ab75e72680 msgr2=0x55ab75eef1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:01:47.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.857+0000 7f9872d9d640 1 --2- 172.21.15.16:0/1192746535 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ab75e72680 0x55ab75eef1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:47.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.857+0000 7f9872d9d640 1 -- 172.21.15.16:0/1192746535 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab75e67dc0 msgr2=0x55ab75e7a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:01:47.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.857+0000 7f9872d9d640 1 --2- 172.21.15.16:0/1192746535 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab75e67dc0 0x55ab75e7a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:47.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.857+0000 7f9872d9d640 1 -- 172.21.15.16:0/1192746535 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab75eef730 con 0x55ab75e674a0 2024-09-12T02:01:47.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.857+0000 7f9871d9b640 1 --2- 172.21.15.16:0/1192746535 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab75e67dc0 0x55ab75e7a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:01:47.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.857+0000 7f9872d9d640 1 --2- 172.21.15.16:0/1192746535 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab75e674a0 0x55ab75e79f90 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f986c02ae00 tx=0x7f986c003b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14d0e9446bc1d24e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:01:47.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.857+0000 7f9859ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:01:47.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.858+0000 7f9859ffb640 1 -- 172.21.15.16:0/1192746535 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f986c048070 con 0x55ab75e674a0 2024-09-12T02:01:47.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.858+0000 7f987ad0d740 1 -- 172.21.15.16:0/1192746535 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f986c036070 con 0x55ab75e674a0 2024-09-12T02:01:47.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.858+0000 7f9859ffb640 1 -- 172.21.15.16:0/1192746535 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f986c02e040 con 0x55ab75e674a0 2024-09-12T02:01:47.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.858+0000 7f9859ffb640 1 -- 172.21.15.16:0/1192746535 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f986c05e040 con 0x55ab75e674a0 2024-09-12T02:01:47.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.858+0000 7f987ad0d740 10 client.15240 fetch_fsmap learned FSMap version 25 2024-09-12T02:01:47.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.858+0000 7f987ad0d740 1 -- 172.21.15.16:0/1192746535 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ab75ef0a10 con 0x55ab75e674a0 2024-09-12T02:01:47.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.858+0000 7f9859ffb640 1 -- 172.21.15.16:0/1192746535 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f986c036070 con 0x55ab75e674a0 2024-09-12T02:01:47.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.858+0000 7f9859ffb640 1 -- 172.21.15.16:0/1192746535 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f986c042030 con 0x55ab75e674a0 2024-09-12T02:01:47.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.858+0000 7f987ad0d740 10 client.15240 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:01:47.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.858+0000 7f987ad0d740 10 client.15240 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:01:47.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.858+0000 7f987ad0d740 20 client.15240 populate_metadata read hostname 'smithi016' 2024-09-12T02:01:47.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.858+0000 7f987ad0d740 1 --2- 172.21.15.16:0/1192746535 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55ab75ef0c40 0x55ab75ef3030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:47.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.859+0000 7f987ad0d740 4 client.15240 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T02:01:47.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.859+0000 7f987ad0d740 1 -- 172.21.15.16:0/1192746535 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ab75dd4d60 con 0x55ab75ef0c40 2024-09-12T02:01:47.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.859+0000 7f9871d9b640 1 --2- 172.21.15.16:0/1192746535 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55ab75ef0c40 0x55ab75ef3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:47.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.859+0000 7f9859ffb640 10 client.15240 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:01:47.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.859+0000 7f9871d9b640 1 --2- 172.21.15.16:0/1192746535 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55ab75ef0c40 0x55ab75ef3030 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e74ec7fdad20472d server_cookie=966d49f0e8701882 in_seq=0 out_seq=0 2024-09-12T02:01:47.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.860+0000 7f9859ffb640 1 -- 172.21.15.16:0/1192746535 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55ab75dd4d60 con 0x55ab75ef0c40 2024-09-12T02:01:47.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.860+0000 7f9859ffb640 10 client.15240 handle_command_reply: tid=0 2024-09-12T02:01:47.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.860+0000 7f987ad0d740 1 client.15240 shutdown 2024-09-12T02:01:47.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.860+0000 7f987ad0d740 1 -- 172.21.15.16:0/1192746535 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab75e674a0 msgr2=0x55ab75e79f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:47.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.860+0000 7f987ad0d740 1 --2- 172.21.15.16:0/1192746535 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab75e674a0 0x55ab75e79f90 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f986c02ae00 tx=0x7f986c003b20 comp rx=0 tx=0).stop 2024-09-12T02:01:47.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.860+0000 7f987ad0d740 1 -- 172.21.15.16:0/1192746535 shutdown_connections 2024-09-12T02:01:47.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.860+0000 7f987ad0d740 1 --2- 172.21.15.16:0/1192746535 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55ab75ef0c40 0x55ab75ef3030 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:47.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.860+0000 7f987ad0d740 1 --2- 172.21.15.16:0/1192746535 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ab75e72680 0x55ab75eef1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:47.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.860+0000 7f987ad0d740 1 --2- 172.21.15.16:0/1192746535 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab75e67dc0 0x55ab75e7a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:47.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.860+0000 7f987ad0d740 1 --2- 172.21.15.16:0/1192746535 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab75e674a0 0x55ab75e79f90 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:47.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.860+0000 7f9859ffb640 0 client.15240 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:01:47.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.860+0000 7f987ad0d740 1 -- 172.21.15.16:0/1192746535 >> 172.21.15.16:0/1192746535 conn(0x55ab75e63a30 msgr2=0x55ab75e6f550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:01:47.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.860+0000 7f987ad0d740 1 -- 172.21.15.16:0/1192746535 shutdown_connections 2024-09-12T02:01:47.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.860+0000 7f987ad0d740 1 -- 172.21.15.16:0/1192746535 wait complete. 2024-09-12T02:01:47.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.860+0000 7f987ad0d740 20 client.15240 trim_cache size 0 max 16384 2024-09-12T02:01:47.873 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T02:01:47.873 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-12T02:01:47.873 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": { 2024-09-12T02:01:47.873 INFO:teuthology.orchestra.run.smithi016.stdout: "ca0d1578-6d6d-43f3-900a-7bd6aef3bd62": { 2024-09-12T02:01:47.873 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-12T02:01:47.873 INFO:teuthology.orchestra.run.smithi016.stdout: "tag": "ca0d1578-6d6d-43f3-900a-7bd6aef3bd62", 2024-09-12T02:01:47.873 INFO:teuthology.orchestra.run.smithi016.stdout: "options": "recursive,force" 2024-09-12T02:01:47.873 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:01:47.873 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:01:47.873 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T02:01:47.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.861+0000 7f9878934640 1 -- 172.21.15.16:0/490299353 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f985c0a2c00 msgr2=0x7f985c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:47.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.861+0000 7f9878934640 1 --2- 172.21.15.16:0/490299353 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f985c0a2c00 0x7f985c0a50c0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f98640a4030 tx=0x7f9864086040 comp rx=0 tx=0).stop 2024-09-12T02:01:47.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.861+0000 7f9878934640 1 -- 172.21.15.16:0/490299353 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9874100a70 msgr2=0x7f9874199160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:47.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.861+0000 7f9878934640 1 --2- 172.21.15.16:0/490299353 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9874100a70 0x7f9874199160 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f986400c920 tx=0x7f986400cde0 comp rx=0 tx=0).stop 2024-09-12T02:01:47.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.861+0000 7f9878934640 1 -- 172.21.15.16:0/490299353 shutdown_connections 2024-09-12T02:01:47.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.861+0000 7f9878934640 1 --2- 172.21.15.16:0/490299353 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f985c0a2c00 0x7f985c0a50c0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:47.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.861+0000 7f9878934640 1 --2- 172.21.15.16:0/490299353 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9874106a70 0x7f98741a41e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:47.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.861+0000 7f9878934640 1 --2- 172.21.15.16:0/490299353 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9874101410 0x7f98741996a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:47.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.861+0000 7f9878934640 1 --2- 172.21.15.16:0/490299353 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9874100a70 0x7f9874199160 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:47.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.861+0000 7f9878934640 1 -- 172.21.15.16:0/490299353 >> 172.21.15.16:0/490299353 conn(0x7f98740fc7d0 msgr2=0x7f98740fcd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:01:47.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.861+0000 7f9878934640 1 -- 172.21.15.16:0/490299353 shutdown_connections 2024-09-12T02:01:47.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:47.861+0000 7f9878934640 1 -- 172.21.15.16:0/490299353 wait complete. 2024-09-12T02:01:48.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:01:48 smithi184 ceph-mon[35593]: pgmap v331: 97 pgs: 97 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1023 B/s wr, 7 op/s 2024-09-12T02:01:48.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:01:48 smithi184 ceph-mon[35593]: from='client.15222 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T02:01:48.631 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:01:48 smithi016 ceph-mon[30947]: pgmap v331: 97 pgs: 97 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1023 B/s wr, 7 op/s 2024-09-12T02:01:48.631 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:01:48 smithi016 ceph-mon[30947]: from='client.15222 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T02:01:48.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:01:48 smithi033 ceph-mon[35332]: pgmap v331: 97 pgs: 97 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1023 B/s wr, 7 op/s 2024-09-12T02:01:48.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:01:48 smithi033 ceph-mon[35332]: from='client.15222 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T02:01:48.709 INFO:tasks.cephfs.filesystem:scrub status for tag:ca0d1578-6d6d-43f3-900a-7bd6aef3bd62 - {'path': '/', 'tag': 'ca0d1578-6d6d-43f3-900a-7bd6aef3bd62', 'options': 'recursive,force'} 2024-09-12T02:01:48.813 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:01:49.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:49.253+0000 7ff6a0d86640 1 -- 172.21.15.16:0/323377786 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff69c106a70 msgr2=0x7ff69c106e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:49.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:49.253+0000 7ff6a0d86640 1 --2- 172.21.15.16:0/323377786 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff69c106a70 0x7ff69c106e50 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7ff68c009930 tx=0x7ff68c02a420 comp rx=0 tx=0).stop 2024-09-12T02:01:49.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:49.253+0000 7ff6a0d86640 1 -- 172.21.15.16:0/323377786 shutdown_connections 2024-09-12T02:01:49.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:49.253+0000 7ff6a0d86640 1 --2- 172.21.15.16:0/323377786 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff69c101410 0x7ff69c10d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:49.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:49.253+0000 7ff6a0d86640 1 --2- 172.21.15.16:0/323377786 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff69c100a70 0x7ff69c100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:49.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:49.253+0000 7ff6a0d86640 1 --2- 172.21.15.16:0/323377786 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff69c106a70 0x7ff69c106e50 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:49.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:49.253+0000 7ff6a0d86640 1 -- 172.21.15.16:0/323377786 >> 172.21.15.16:0/323377786 conn(0x7ff69c0fc790 msgr2=0x7ff69c0febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:01:49.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:49.253+0000 7ff6a0d86640 1 -- 172.21.15.16:0/323377786 shutdown_connections 2024-09-12T02:01:49.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:49.253+0000 7ff6a0d86640 1 -- 172.21.15.16:0/323377786 wait complete. 2024-09-12T02:01:49.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:49.254+0000 7ff6a0d86640 1 Processor -- start 2024-09-12T02:01:49.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:49.254+0000 7ff6a0d86640 1 -- start start 2024-09-12T02:01:49.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:49.254+0000 7ff6a0d86640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff69c100a70 0x7ff69c198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:49.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:49.254+0000 7ff6a0d86640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff69c101410 0x7ff69c1989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:49.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:49.254+0000 7ff6a0d86640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff69c106a70 0x7ff69c19cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:49.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:49.254+0000 7ff6a0d86640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff69c110e70 con 0x7ff69c101410 2024-09-12T02:01:49.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:49.254+0000 7ff6a0d86640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff69c110cf0 con 0x7ff69c100a70 2024-09-12T02:01:49.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:49.254+0000 7ff6a0d86640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff69c110ff0 con 0x7ff69c106a70 2024-09-12T02:01:49.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:49.255+0000 7ff699d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff69c101410 0x7ff69c1989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:49.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:49.255+0000 7ff699d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff69c101410 0x7ff69c1989a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46726/0 (socket says 172.21.15.16:46726) 2024-09-12T02:01:49.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:49.255+0000 7ff699d9b640 1 -- 172.21.15.16:0/3586374712 learned_addr learned my addr 172.21.15.16:0/3586374712 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:01:49.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:49.255+0000 7ff69a59c640 1 --2- 172.21.15.16:0/3586374712 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff69c100a70 0x7ff69c198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:49.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:49.255+0000 7ff69ad9d640 1 --2- 172.21.15.16:0/3586374712 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff69c106a70 0x7ff69c19cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:49.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:49.255+0000 7ff699d9b640 1 -- 172.21.15.16:0/3586374712 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff69c106a70 msgr2=0x7ff69c19cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:49.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:49.255+0000 7ff699d9b640 1 --2- 172.21.15.16:0/3586374712 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff69c106a70 0x7ff69c19cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:49.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:49.255+0000 7ff699d9b640 1 -- 172.21.15.16:0/3586374712 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff69c100a70 msgr2=0x7ff69c198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:49.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:49.255+0000 7ff699d9b640 1 --2- 172.21.15.16:0/3586374712 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff69c100a70 0x7ff69c198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:49.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:49.255+0000 7ff699d9b640 1 -- 172.21.15.16:0/3586374712 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff69c19d410 con 0x7ff69c101410 2024-09-12T02:01:49.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:49.255+0000 7ff69ad9d640 1 --2- 172.21.15.16:0/3586374712 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff69c106a70 0x7ff69c19cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:01:49.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:49.255+0000 7ff699d9b640 1 --2- 172.21.15.16:0/3586374712 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff69c101410 0x7ff69c1989a0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7ff68400d950 tx=0x7ff68400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:01:49.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:49.255+0000 7ff6937fe640 1 -- 172.21.15.16:0/3586374712 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff684017070 con 0x7ff69c101410 2024-09-12T02:01:49.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:49.255+0000 7ff6937fe640 1 -- 172.21.15.16:0/3586374712 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff684009c30 con 0x7ff69c101410 2024-09-12T02:01:49.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:49.255+0000 7ff6a0d86640 1 -- 172.21.15.16:0/3586374712 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff69c19d700 con 0x7ff69c101410 2024-09-12T02:01:49.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:49.255+0000 7ff6937fe640 1 -- 172.21.15.16:0/3586374712 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff68400f6b0 con 0x7ff69c101410 2024-09-12T02:01:49.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:49.255+0000 7ff6a0d86640 1 -- 172.21.15.16:0/3586374712 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff69c1a4fe0 con 0x7ff69c101410 2024-09-12T02:01:49.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:49.256+0000 7ff6a0d86640 1 -- 172.21.15.16:0/3586374712 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff69c100ed0 con 0x7ff69c101410 2024-09-12T02:01:49.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:49.258+0000 7ff6937fe640 1 -- 172.21.15.16:0/3586374712 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff6840109d0 con 0x7ff69c101410 2024-09-12T02:01:49.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:49.258+0000 7ff6937fe640 1 --2- 172.21.15.16:0/3586374712 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7ff6880a2b80 0x7ff6880a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:49.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:49.258+0000 7ff69a59c640 1 --2- 172.21.15.16:0/3586374712 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7ff6880a2b80 0x7ff6880a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:49.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:49.258+0000 7ff6937fe640 1 -- 172.21.15.16:0/3586374712 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7ff6840cebf0 con 0x7ff69c101410 2024-09-12T02:01:49.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:49.259+0000 7ff69a59c640 1 --2- 172.21.15.16:0/3586374712 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7ff6880a2b80 0x7ff6880a5040 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ff68c009900 tx=0x7ff68c0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:01:49.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:49.262+0000 7ff6937fe640 1 -- 172.21.15.16:0/3586374712 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff684009de0 con 0x7ff69c101410 2024-09-12T02:01:49.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:49.435+0000 7ff6a0d86640 1 -- 172.21.15.16:0/3586374712 --> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff69c0fd670 con 0x7ff6880a2b80 2024-09-12T02:01:49.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:49.443+0000 7ff6937fe640 1 -- 172.21.15.16:0/3586374712 <== mgr.14150 v2:172.21.15.16:6800/3887444820 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258813 (secure 0 0 0) 0x7ff69c0fd670 con 0x7ff6880a2b80 2024-09-12T02:01:49.445 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T02:01:49.446 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-12T02:01:49.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:49.444+0000 7ff6a0d86640 1 -- 172.21.15.16:0/3586374712 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7ff6880a2b80 msgr2=0x7ff6880a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:49.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:49.444+0000 7ff6a0d86640 1 --2- 172.21.15.16:0/3586374712 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7ff6880a2b80 0x7ff6880a5040 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ff68c009900 tx=0x7ff68c0058d0 comp rx=0 tx=0).stop 2024-09-12T02:01:49.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:49.444+0000 7ff6a0d86640 1 -- 172.21.15.16:0/3586374712 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff69c101410 msgr2=0x7ff69c1989a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:49.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:49.444+0000 7ff6a0d86640 1 --2- 172.21.15.16:0/3586374712 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff69c101410 0x7ff69c1989a0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7ff68400d950 tx=0x7ff68400de10 comp rx=0 tx=0).stop 2024-09-12T02:01:49.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:49.444+0000 7ff6a0d86640 1 -- 172.21.15.16:0/3586374712 shutdown_connections 2024-09-12T02:01:49.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:49.444+0000 7ff6a0d86640 1 --2- 172.21.15.16:0/3586374712 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7ff6880a2b80 0x7ff6880a5040 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:49.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:49.444+0000 7ff6a0d86640 1 --2- 172.21.15.16:0/3586374712 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff69c106a70 0x7ff69c19cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:49.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:49.444+0000 7ff6a0d86640 1 --2- 172.21.15.16:0/3586374712 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff69c101410 0x7ff69c1989a0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:49.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:49.444+0000 7ff6a0d86640 1 --2- 172.21.15.16:0/3586374712 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff69c100a70 0x7ff69c198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:49.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:49.444+0000 7ff6a0d86640 1 -- 172.21.15.16:0/3586374712 >> 172.21.15.16:0/3586374712 conn(0x7ff69c0fc790 msgr2=0x7ff69c0fcf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:01:49.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:49.445+0000 7ff6a0d86640 1 -- 172.21.15.16:0/3586374712 shutdown_connections 2024-09-12T02:01:49.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:49.445+0000 7ff6a0d86640 1 -- 172.21.15.16:0/3586374712 wait complete. 2024-09-12T02:01:50.101 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":332,"stamp":"2024-09-12T02:01:48.325541+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":659381,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":753,"num_read_kb":495,"num_write":353,"num_write_kb":3059,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":1180848,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":783,"ondisk_log_size":783,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":374280,"kb_used_data":12552,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124567544,"statfs":{"total":1151940427776,"available":1151557165056,"internally_reserved":0,"allocated":12853248,"data_stored":7200807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":370193723},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5012,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":64,"num_read_kb":38,"num_write":10,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002831"},"pg_stats":[{"pgid":"3.3f","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.318917+0000","last_change":"2024-09-12T01:59:55.503882+0000","last_active":"2024-09-12T02:01:38.318917+0000","last_peered":"2024-09-12T02:01:38.318917+0000","last_clean":"2024-09-12T02:01:38.318917+0000","last_became_active":"2024-09-12T01:59:55.503513+0000","last_became_peered":"2024-09-12T01:59:55.503513+0000","last_unstale":"2024-09-12T02:01:38.318917+0000","last_undegraded":"2024-09-12T02:01:38.318917+0000","last_fullsized":"2024-09-12T02:01:38.318917+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:43:13.098527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"82'4","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.322981+0000","last_change":"2024-09-12T01:59:55.492965+0000","last_active":"2024-09-12T02:01:38.322981+0000","last_peered":"2024-09-12T02:01:38.322981+0000","last_clean":"2024-09-12T02:01:38.322981+0000","last_became_active":"2024-09-12T01:59:55.492613+0000","last_became_peered":"2024-09-12T01:59:55.492613+0000","last_unstale":"2024-09-12T02:01:38.322981+0000","last_undegraded":"2024-09-12T02:01:38.322981+0000","last_fullsized":"2024-09-12T02:01:38.322981+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:58:20.889770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.332184+0000","last_change":"2024-09-12T01:59:55.497334+0000","last_active":"2024-09-12T02:01:38.332184+0000","last_peered":"2024-09-12T02:01:38.332184+0000","last_clean":"2024-09-12T02:01:38.332184+0000","last_became_active":"2024-09-12T01:59:55.497037+0000","last_became_peered":"2024-09-12T01:59:55.497037+0000","last_unstale":"2024-09-12T02:01:38.332184+0000","last_undegraded":"2024-09-12T02:01:38.332184+0000","last_fullsized":"2024-09-12T02:01:38.332184+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:43:04.082918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.331162+0000","last_change":"2024-09-12T01:59:55.495888+0000","last_active":"2024-09-12T02:01:38.331162+0000","last_peered":"2024-09-12T02:01:38.331162+0000","last_clean":"2024-09-12T02:01:38.331162+0000","last_became_active":"2024-09-12T01:59:55.492684+0000","last_became_peered":"2024-09-12T01:59:55.492684+0000","last_unstale":"2024-09-12T02:01:38.331162+0000","last_undegraded":"2024-09-12T02:01:38.331162+0000","last_fullsized":"2024-09-12T02:01:38.331162+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:15:00.500006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"82'4","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.301731+0000","last_change":"2024-09-12T01:59:55.494613+0000","last_active":"2024-09-12T02:01:38.301731+0000","last_peered":"2024-09-12T02:01:38.301731+0000","last_clean":"2024-09-12T02:01:38.301731+0000","last_became_active":"2024-09-12T01:59:55.494205+0000","last_became_peered":"2024-09-12T01:59:55.494205+0000","last_unstale":"2024-09-12T02:01:38.301731+0000","last_undegraded":"2024-09-12T02:01:38.301731+0000","last_fullsized":"2024-09-12T02:01:38.301731+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:39:14.573280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.331517+0000","last_change":"2024-09-12T01:59:55.496659+0000","last_active":"2024-09-12T02:01:38.331517+0000","last_peered":"2024-09-12T02:01:38.331517+0000","last_clean":"2024-09-12T02:01:38.331517+0000","last_became_active":"2024-09-12T01:59:55.496232+0000","last_became_peered":"2024-09-12T01:59:55.496232+0000","last_unstale":"2024-09-12T02:01:38.331517+0000","last_undegraded":"2024-09-12T02:01:38.331517+0000","last_fullsized":"2024-09-12T02:01:38.331517+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:48:21.544764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"82'5","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.317177+0000","last_change":"2024-09-12T01:59:55.486789+0000","last_active":"2024-09-12T02:01:38.317177+0000","last_peered":"2024-09-12T02:01:38.317177+0000","last_clean":"2024-09-12T02:01:38.317177+0000","last_became_active":"2024-09-12T01:59:55.485921+0000","last_became_peered":"2024-09-12T01:59:55.485921+0000","last_unstale":"2024-09-12T02:01:38.317177+0000","last_undegraded":"2024-09-12T02:01:38.317177+0000","last_fullsized":"2024-09-12T02:01:38.317177+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:21:16.071121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.321850+0000","last_change":"2024-09-12T01:59:55.494525+0000","last_active":"2024-09-12T02:01:38.321850+0000","last_peered":"2024-09-12T02:01:38.321850+0000","last_clean":"2024-09-12T02:01:38.321850+0000","last_became_active":"2024-09-12T01:59:55.493998+0000","last_became_peered":"2024-09-12T01:59:55.493998+0000","last_unstale":"2024-09-12T02:01:38.321850+0000","last_undegraded":"2024-09-12T02:01:38.321850+0000","last_fullsized":"2024-09-12T02:01:38.321850+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:59:24.653660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"82'11","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.320666+0000","last_change":"2024-09-12T01:59:55.501466+0000","last_active":"2024-09-12T02:01:38.320666+0000","last_peered":"2024-09-12T02:01:38.320666+0000","last_clean":"2024-09-12T02:01:38.320666+0000","last_became_active":"2024-09-12T01:59:55.500800+0000","last_became_peered":"2024-09-12T01:59:55.500800+0000","last_unstale":"2024-09-12T02:01:38.320666+0000","last_undegraded":"2024-09-12T02:01:38.320666+0000","last_fullsized":"2024-09-12T02:01:38.320666+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:55:13.704791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"82'16","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.331670+0000","last_change":"2024-09-12T01:59:55.497986+0000","last_active":"2024-09-12T02:01:38.331670+0000","last_peered":"2024-09-12T02:01:38.331670+0000","last_clean":"2024-09-12T02:01:38.331670+0000","last_became_active":"2024-09-12T01:59:55.497691+0000","last_became_peered":"2024-09-12T01:59:55.497691+0000","last_unstale":"2024-09-12T02:01:38.331670+0000","last_undegraded":"2024-09-12T02:01:38.331670+0000","last_fullsized":"2024-09-12T02:01:38.331670+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:37:07.702606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.329986+0000","last_change":"2024-09-12T01:59:55.501339+0000","last_active":"2024-09-12T02:01:38.329986+0000","last_peered":"2024-09-12T02:01:38.329986+0000","last_clean":"2024-09-12T02:01:38.329986+0000","last_became_active":"2024-09-12T01:59:55.500700+0000","last_became_peered":"2024-09-12T01:59:55.500700+0000","last_unstale":"2024-09-12T02:01:38.329986+0000","last_undegraded":"2024-09-12T02:01:38.329986+0000","last_fullsized":"2024-09-12T02:01:38.329986+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:12:15.879183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.330871+0000","last_change":"2024-09-12T01:59:55.495196+0000","last_active":"2024-09-12T02:01:38.330871+0000","last_peered":"2024-09-12T02:01:38.330871+0000","last_clean":"2024-09-12T02:01:38.330871+0000","last_became_active":"2024-09-12T01:59:55.494889+0000","last_became_peered":"2024-09-12T01:59:55.494889+0000","last_unstale":"2024-09-12T02:01:38.330871+0000","last_undegraded":"2024-09-12T02:01:38.330871+0000","last_fullsized":"2024-09-12T02:01:38.330871+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:32:30.621946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"82'10","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.315946+0000","last_change":"2024-09-12T01:59:55.492565+0000","last_active":"2024-09-12T02:01:38.315946+0000","last_peered":"2024-09-12T02:01:38.315946+0000","last_clean":"2024-09-12T02:01:38.315946+0000","last_became_active":"2024-09-12T01:59:55.492168+0000","last_became_peered":"2024-09-12T01:59:55.492168+0000","last_unstale":"2024-09-12T02:01:38.315946+0000","last_undegraded":"2024-09-12T02:01:38.315946+0000","last_fullsized":"2024-09-12T02:01:38.315946+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:39:48.630202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.323015+0000","last_change":"2024-09-12T01:59:55.493152+0000","last_active":"2024-09-12T02:01:38.323015+0000","last_peered":"2024-09-12T02:01:38.323015+0000","last_clean":"2024-09-12T02:01:38.323015+0000","last_became_active":"2024-09-12T01:59:55.492794+0000","last_became_peered":"2024-09-12T01:59:55.492794+0000","last_unstale":"2024-09-12T02:01:38.323015+0000","last_undegraded":"2024-09-12T02:01:38.323015+0000","last_fullsized":"2024-09-12T02:01:38.323015+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:24:27.270502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.333374+0000","last_change":"2024-09-12T01:59:55.494651+0000","last_active":"2024-09-12T02:01:38.333374+0000","last_peered":"2024-09-12T02:01:38.333374+0000","last_clean":"2024-09-12T02:01:38.333374+0000","last_became_active":"2024-09-12T01:59:55.494355+0000","last_became_peered":"2024-09-12T01:59:55.494355+0000","last_unstale":"2024-09-12T02:01:38.333374+0000","last_undegraded":"2024-09-12T02:01:38.333374+0000","last_fullsized":"2024-09-12T02:01:38.333374+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:14:44.364158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.330799+0000","last_change":"2024-09-12T01:59:55.495049+0000","last_active":"2024-09-12T02:01:38.330799+0000","last_peered":"2024-09-12T02:01:38.330799+0000","last_clean":"2024-09-12T02:01:38.330799+0000","last_became_active":"2024-09-12T01:59:55.494688+0000","last_became_peered":"2024-09-12T01:59:55.494688+0000","last_unstale":"2024-09-12T02:01:38.330799+0000","last_undegraded":"2024-09-12T02:01:38.330799+0000","last_fullsized":"2024-09-12T02:01:38.330799+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:13:05.944577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.319055+0000","last_change":"2024-09-12T01:59:55.504146+0000","last_active":"2024-09-12T02:01:38.319055+0000","last_peered":"2024-09-12T02:01:38.319055+0000","last_clean":"2024-09-12T02:01:38.319055+0000","last_became_active":"2024-09-12T01:59:55.503646+0000","last_became_peered":"2024-09-12T01:59:55.503646+0000","last_unstale":"2024-09-12T02:01:38.319055+0000","last_undegraded":"2024-09-12T02:01:38.319055+0000","last_fullsized":"2024-09-12T02:01:38.319055+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:23:51.780795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"82'10","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.325350+0000","last_change":"2024-09-12T01:59:55.489930+0000","last_active":"2024-09-12T02:01:38.325350+0000","last_peered":"2024-09-12T02:01:38.325350+0000","last_clean":"2024-09-12T02:01:38.325350+0000","last_became_active":"2024-09-12T01:59:55.489619+0000","last_became_peered":"2024-09-12T01:59:55.489619+0000","last_unstale":"2024-09-12T02:01:38.325350+0000","last_undegraded":"2024-09-12T02:01:38.325350+0000","last_fullsized":"2024-09-12T02:01:38.325350+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:04:57.307091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.303800+0000","last_change":"2024-09-12T01:59:55.492098+0000","last_active":"2024-09-12T02:01:38.303800+0000","last_peered":"2024-09-12T02:01:38.303800+0000","last_clean":"2024-09-12T02:01:38.303800+0000","last_became_active":"2024-09-12T01:59:55.491370+0000","last_became_peered":"2024-09-12T01:59:55.491370+0000","last_unstale":"2024-09-12T02:01:38.303800+0000","last_undegraded":"2024-09-12T02:01:38.303800+0000","last_fullsized":"2024-09-12T02:01:38.303800+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:25:06.307903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.314843+0000","last_change":"2024-09-12T01:59:55.499938+0000","last_active":"2024-09-12T02:01:38.314843+0000","last_peered":"2024-09-12T02:01:38.314843+0000","last_clean":"2024-09-12T02:01:38.314843+0000","last_became_active":"2024-09-12T01:59:55.499619+0000","last_became_peered":"2024-09-12T01:59:55.499619+0000","last_unstale":"2024-09-12T02:01:38.314843+0000","last_undegraded":"2024-09-12T02:01:38.314843+0000","last_fullsized":"2024-09-12T02:01:38.314843+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:00:17.536120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"82'11","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.321068+0000","last_change":"2024-09-12T01:59:55.486106+0000","last_active":"2024-09-12T02:01:38.321068+0000","last_peered":"2024-09-12T02:01:38.321068+0000","last_clean":"2024-09-12T02:01:38.321068+0000","last_became_active":"2024-09-12T01:59:55.485325+0000","last_became_peered":"2024-09-12T01:59:55.485325+0000","last_unstale":"2024-09-12T02:01:38.321068+0000","last_undegraded":"2024-09-12T02:01:38.321068+0000","last_fullsized":"2024-09-12T02:01:38.321068+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:35:46.446224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"82'10","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.331650+0000","last_change":"2024-09-12T01:59:55.496438+0000","last_active":"2024-09-12T02:01:38.331650+0000","last_peered":"2024-09-12T02:01:38.331650+0000","last_clean":"2024-09-12T02:01:38.331650+0000","last_became_active":"2024-09-12T01:59:55.495885+0000","last_became_peered":"2024-09-12T01:59:55.495885+0000","last_unstale":"2024-09-12T02:01:38.331650+0000","last_undegraded":"2024-09-12T02:01:38.331650+0000","last_fullsized":"2024-09-12T02:01:38.331650+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:15:14.911335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.322674+0000","last_change":"2024-09-12T01:59:55.489350+0000","last_active":"2024-09-12T02:01:38.322674+0000","last_peered":"2024-09-12T02:01:38.322674+0000","last_clean":"2024-09-12T02:01:38.322674+0000","last_became_active":"2024-09-12T01:59:55.489028+0000","last_became_peered":"2024-09-12T01:59:55.489028+0000","last_unstale":"2024-09-12T02:01:38.322674+0000","last_undegraded":"2024-09-12T02:01:38.322674+0000","last_fullsized":"2024-09-12T02:01:38.322674+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:29:23.769004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.316094+0000","last_change":"2024-09-12T01:59:55.494532+0000","last_active":"2024-09-12T02:01:38.316094+0000","last_peered":"2024-09-12T02:01:38.316094+0000","last_clean":"2024-09-12T02:01:38.316094+0000","last_became_active":"2024-09-12T01:59:55.490982+0000","last_became_peered":"2024-09-12T01:59:55.490982+0000","last_unstale":"2024-09-12T02:01:38.316094+0000","last_undegraded":"2024-09-12T02:01:38.316094+0000","last_fullsized":"2024-09-12T02:01:38.316094+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:04:50.633730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"82'2","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.316143+0000","last_change":"2024-09-12T01:59:55.496057+0000","last_active":"2024-09-12T02:01:38.316143+0000","last_peered":"2024-09-12T02:01:38.316143+0000","last_clean":"2024-09-12T02:01:38.316143+0000","last_became_active":"2024-09-12T01:59:55.495710+0000","last_became_peered":"2024-09-12T01:59:55.495710+0000","last_unstale":"2024-09-12T02:01:38.316143+0000","last_undegraded":"2024-09-12T02:01:38.316143+0000","last_fullsized":"2024-09-12T02:01:38.316143+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:49:01.962537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"82'15","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.311246+0000","last_change":"2024-09-12T01:59:55.491594+0000","last_active":"2024-09-12T02:01:38.311246+0000","last_peered":"2024-09-12T02:01:38.311246+0000","last_clean":"2024-09-12T02:01:38.311246+0000","last_became_active":"2024-09-12T01:59:55.491233+0000","last_became_peered":"2024-09-12T01:59:55.491233+0000","last_unstale":"2024-09-12T02:01:38.311246+0000","last_undegraded":"2024-09-12T02:01:38.311246+0000","last_fullsized":"2024-09-12T02:01:38.311246+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:24:22.066777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.309442+0000","last_change":"2024-09-12T01:59:55.498331+0000","last_active":"2024-09-12T02:01:38.309442+0000","last_peered":"2024-09-12T02:01:38.309442+0000","last_clean":"2024-09-12T02:01:38.309442+0000","last_became_active":"2024-09-12T01:59:55.497983+0000","last_became_peered":"2024-09-12T01:59:55.497983+0000","last_unstale":"2024-09-12T02:01:38.309442+0000","last_undegraded":"2024-09-12T02:01:38.309442+0000","last_fullsized":"2024-09-12T02:01:38.309442+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:37:00.735231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"82'4","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.315811+0000","last_change":"2024-09-12T02:00:09.575358+0000","last_active":"2024-09-12T02:01:38.315811+0000","last_peered":"2024-09-12T02:01:38.315811+0000","last_clean":"2024-09-12T02:01:38.315811+0000","last_became_active":"2024-09-12T02:00:09.574913+0000","last_became_peered":"2024-09-12T02:00:09.574913+0000","last_unstale":"2024-09-12T02:01:38.315811+0000","last_undegraded":"2024-09-12T02:01:38.315811+0000","last_fullsized":"2024-09-12T02:01:38.315811+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:19:53.630453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.333406+0000","last_change":"2024-09-12T01:59:55.492419+0000","last_active":"2024-09-12T02:01:38.333406+0000","last_peered":"2024-09-12T02:01:38.333406+0000","last_clean":"2024-09-12T02:01:38.333406+0000","last_became_active":"2024-09-12T01:59:55.492003+0000","last_became_peered":"2024-09-12T01:59:55.492003+0000","last_unstale":"2024-09-12T02:01:38.333406+0000","last_undegraded":"2024-09-12T02:01:38.333406+0000","last_fullsized":"2024-09-12T02:01:38.333406+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:25:45.774679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.307126+0000","last_change":"2024-09-12T01:59:55.490125+0000","last_active":"2024-09-12T02:01:38.307126+0000","last_peered":"2024-09-12T02:01:38.307126+0000","last_clean":"2024-09-12T02:01:38.307126+0000","last_became_active":"2024-09-12T01:59:55.489636+0000","last_became_peered":"2024-09-12T01:59:55.489636+0000","last_unstale":"2024-09-12T02:01:38.307126+0000","last_undegraded":"2024-09-12T02:01:38.307126+0000","last_fullsized":"2024-09-12T02:01:38.307126+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:41:30.922185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"82'12","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.309816+0000","last_change":"2024-09-12T01:59:55.496322+0000","last_active":"2024-09-12T02:01:38.309816+0000","last_peered":"2024-09-12T02:01:38.309816+0000","last_clean":"2024-09-12T02:01:38.309816+0000","last_became_active":"2024-09-12T01:59:55.496005+0000","last_became_peered":"2024-09-12T01:59:55.496005+0000","last_unstale":"2024-09-12T02:01:38.309816+0000","last_undegraded":"2024-09-12T02:01:38.309816+0000","last_fullsized":"2024-09-12T02:01:38.309816+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:53:39.248715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.310438+0000","last_change":"2024-09-12T01:59:55.497758+0000","last_active":"2024-09-12T02:01:38.310438+0000","last_peered":"2024-09-12T02:01:38.310438+0000","last_clean":"2024-09-12T02:01:38.310438+0000","last_became_active":"2024-09-12T01:59:55.497282+0000","last_became_peered":"2024-09-12T01:59:55.497282+0000","last_unstale":"2024-09-12T02:01:38.310438+0000","last_undegraded":"2024-09-12T02:01:38.310438+0000","last_fullsized":"2024-09-12T02:01:38.310438+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:08:25.509372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.301389+0000","last_change":"2024-09-12T01:59:55.497919+0000","last_active":"2024-09-12T02:01:38.301389+0000","last_peered":"2024-09-12T02:01:38.301389+0000","last_clean":"2024-09-12T02:01:38.301389+0000","last_became_active":"2024-09-12T01:59:55.497609+0000","last_became_peered":"2024-09-12T01:59:55.497609+0000","last_unstale":"2024-09-12T02:01:38.301389+0000","last_undegraded":"2024-09-12T02:01:38.301389+0000","last_fullsized":"2024-09-12T02:01:38.301389+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:10:22.281180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"82'7","reported_seq":134,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:44.695380+0000","last_change":"2024-09-12T01:59:53.473974+0000","last_active":"2024-09-12T02:01:44.695380+0000","last_peered":"2024-09-12T02:01:44.695380+0000","last_clean":"2024-09-12T02:01:44.695380+0000","last_became_active":"2024-09-12T01:59:53.473546+0000","last_became_peered":"2024-09-12T01:59:53.473546+0000","last_unstale":"2024-09-12T02:01:44.695380+0000","last_undegraded":"2024-09-12T02:01:44.695380+0000","last_fullsized":"2024-09-12T02:01:44.695380+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:28:03.532249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1156,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":105,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.328613+0000","last_change":"2024-09-12T01:59:55.493197+0000","last_active":"2024-09-12T02:01:38.328613+0000","last_peered":"2024-09-12T02:01:38.328613+0000","last_clean":"2024-09-12T02:01:38.328613+0000","last_became_active":"2024-09-12T01:59:55.492845+0000","last_became_peered":"2024-09-12T01:59:55.492845+0000","last_unstale":"2024-09-12T02:01:38.328613+0000","last_undegraded":"2024-09-12T02:01:38.328613+0000","last_fullsized":"2024-09-12T02:01:38.328613+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:42:50.790896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"80'3","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.622865+0000","last_change":"2024-09-12T01:59:53.471214+0000","last_active":"2024-09-12T02:00:09.622865+0000","last_peered":"2024-09-12T02:00:09.622865+0000","last_clean":"2024-09-12T02:00:09.622865+0000","last_became_active":"2024-09-12T01:59:53.470858+0000","last_became_peered":"2024-09-12T01:59:53.470858+0000","last_unstale":"2024-09-12T02:00:09.622865+0000","last_undegraded":"2024-09-12T02:00:09.622865+0000","last_fullsized":"2024-09-12T02:00:09.622865+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:34:48.730392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"80'2","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.562106+0000","last_change":"2024-09-12T01:59:53.464568+0000","last_active":"2024-09-12T02:00:09.562106+0000","last_peered":"2024-09-12T02:00:09.562106+0000","last_clean":"2024-09-12T02:00:09.562106+0000","last_became_active":"2024-09-12T01:59:53.464168+0000","last_became_peered":"2024-09-12T01:59:53.464168+0000","last_unstale":"2024-09-12T02:00:09.562106+0000","last_undegraded":"2024-09-12T02:00:09.562106+0000","last_fullsized":"2024-09-12T02:00:09.562106+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:41:48.135522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.318601+0000","last_change":"2024-09-12T01:59:55.487177+0000","last_active":"2024-09-12T02:01:38.318601+0000","last_peered":"2024-09-12T02:01:38.318601+0000","last_clean":"2024-09-12T02:01:38.318601+0000","last_became_active":"2024-09-12T01:59:55.486886+0000","last_became_peered":"2024-09-12T01:59:55.486886+0000","last_unstale":"2024-09-12T02:01:38.318601+0000","last_undegraded":"2024-09-12T02:01:38.318601+0000","last_fullsized":"2024-09-12T02:01:38.318601+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:32:40.150071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"80'1","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.567129+0000","last_change":"2024-09-12T01:59:53.474114+0000","last_active":"2024-09-12T02:00:09.567129+0000","last_peered":"2024-09-12T02:00:09.567129+0000","last_clean":"2024-09-12T02:00:09.567129+0000","last_became_active":"2024-09-12T01:59:53.473755+0000","last_became_peered":"2024-09-12T01:59:53.473755+0000","last_unstale":"2024-09-12T02:00:09.567129+0000","last_undegraded":"2024-09-12T02:00:09.567129+0000","last_fullsized":"2024-09-12T02:00:09.567129+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:07:07.021137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"82'11","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.323623+0000","last_change":"2024-09-12T01:59:55.489868+0000","last_active":"2024-09-12T02:01:38.323623+0000","last_peered":"2024-09-12T02:01:38.323623+0000","last_clean":"2024-09-12T02:01:38.323623+0000","last_became_active":"2024-09-12T01:59:55.489537+0000","last_became_peered":"2024-09-12T01:59:55.489537+0000","last_unstale":"2024-09-12T02:01:38.323623+0000","last_undegraded":"2024-09-12T02:01:38.323623+0000","last_fullsized":"2024-09-12T02:01:38.323623+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:18:49.417689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"82'4","reported_seq":125,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:42.692210+0000","last_change":"2024-09-12T01:59:53.471402+0000","last_active":"2024-09-12T02:01:42.692210+0000","last_peered":"2024-09-12T02:01:42.692210+0000","last_clean":"2024-09-12T02:01:42.692210+0000","last_became_active":"2024-09-12T01:59:53.470940+0000","last_became_peered":"2024-09-12T01:59:53.470940+0000","last_unstale":"2024-09-12T02:01:42.692210+0000","last_undegraded":"2024-09-12T02:01:42.692210+0000","last_fullsized":"2024-09-12T02:01:42.692210+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:41:16.853592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":99,"num_read_kb":99,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"82'10","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.330030+0000","last_change":"2024-09-12T01:59:55.499047+0000","last_active":"2024-09-12T02:01:38.330030+0000","last_peered":"2024-09-12T02:01:38.330030+0000","last_clean":"2024-09-12T02:01:38.330030+0000","last_became_active":"2024-09-12T01:59:55.498071+0000","last_became_peered":"2024-09-12T01:59:55.498071+0000","last_unstale":"2024-09-12T02:01:38.330030+0000","last_undegraded":"2024-09-12T02:01:38.330030+0000","last_fullsized":"2024-09-12T02:01:38.330030+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:36:40.106041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"80'2","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.622966+0000","last_change":"2024-09-12T01:59:53.473508+0000","last_active":"2024-09-12T02:00:09.622966+0000","last_peered":"2024-09-12T02:00:09.622966+0000","last_clean":"2024-09-12T02:00:09.622966+0000","last_became_active":"2024-09-12T01:59:53.473063+0000","last_became_peered":"2024-09-12T01:59:53.473063+0000","last_unstale":"2024-09-12T02:00:09.622966+0000","last_undegraded":"2024-09-12T02:00:09.622966+0000","last_fullsized":"2024-09-12T02:00:09.622966+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:55:32.161769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.333032+0000","last_change":"2024-09-12T01:59:55.491721+0000","last_active":"2024-09-12T02:01:38.333032+0000","last_peered":"2024-09-12T02:01:38.333032+0000","last_clean":"2024-09-12T02:01:38.333032+0000","last_became_active":"2024-09-12T01:59:55.491364+0000","last_became_peered":"2024-09-12T01:59:55.491364+0000","last_unstale":"2024-09-12T02:01:38.333032+0000","last_undegraded":"2024-09-12T02:01:38.333032+0000","last_fullsized":"2024-09-12T02:01:38.333032+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:21:20.795420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"80'4","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.567315+0000","last_change":"2024-09-12T01:59:53.921789+0000","last_active":"2024-09-12T02:00:09.567315+0000","last_peered":"2024-09-12T02:00:09.567315+0000","last_clean":"2024-09-12T02:00:09.567315+0000","last_became_active":"2024-09-12T01:59:53.921383+0000","last_became_peered":"2024-09-12T01:59:53.921383+0000","last_unstale":"2024-09-12T02:00:09.567315+0000","last_undegraded":"2024-09-12T02:00:09.567315+0000","last_fullsized":"2024-09-12T02:00:09.567315+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:21:58.792794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"82'11","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.331576+0000","last_change":"2024-09-12T01:59:55.503008+0000","last_active":"2024-09-12T02:01:38.331576+0000","last_peered":"2024-09-12T02:01:38.331576+0000","last_clean":"2024-09-12T02:01:38.331576+0000","last_became_active":"2024-09-12T01:59:55.502538+0000","last_became_peered":"2024-09-12T01:59:55.502538+0000","last_unstale":"2024-09-12T02:01:38.331576+0000","last_undegraded":"2024-09-12T02:01:38.331576+0000","last_fullsized":"2024-09-12T02:01:38.331576+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:51:36.735985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"82'32","reported_seq":263,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:43.693503+0000","last_change":"2024-09-12T01:59:53.470688+0000","last_active":"2024-09-12T02:01:43.693503+0000","last_peered":"2024-09-12T02:01:43.693503+0000","last_clean":"2024-09-12T02:01:43.693503+0000","last_became_active":"2024-09-12T01:59:53.470231+0000","last_became_peered":"2024-09-12T01:59:53.470231+0000","last_unstale":"2024-09-12T02:01:43.693503+0000","last_undegraded":"2024-09-12T02:01:43.693503+0000","last_fullsized":"2024-09-12T02:01:43.693503+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:53:54.439893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64635,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":209,"num_read_kb":168,"num_write":33,"num_write_kb":76,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.310987+0000","last_change":"2024-09-12T01:59:55.495343+0000","last_active":"2024-09-12T02:01:38.310987+0000","last_peered":"2024-09-12T02:01:38.310987+0000","last_clean":"2024-09-12T02:01:38.310987+0000","last_became_active":"2024-09-12T01:59:55.495006+0000","last_became_peered":"2024-09-12T01:59:55.495006+0000","last_unstale":"2024-09-12T02:01:38.310987+0000","last_undegraded":"2024-09-12T02:01:38.310987+0000","last_fullsized":"2024-09-12T02:01:38.310987+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:32:20.932551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'6","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.569082+0000","last_change":"2024-09-12T01:59:53.468625+0000","last_active":"2024-09-12T02:00:09.569082+0000","last_peered":"2024-09-12T02:00:09.569082+0000","last_clean":"2024-09-12T02:00:09.569082+0000","last_became_active":"2024-09-12T01:59:53.468015+0000","last_became_peered":"2024-09-12T01:59:53.468015+0000","last_unstale":"2024-09-12T02:00:09.569082+0000","last_undegraded":"2024-09-12T02:00:09.569082+0000","last_fullsized":"2024-09-12T02:00:09.569082+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:49:34.842239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.319752+0000","last_change":"2024-09-12T01:59:55.487962+0000","last_active":"2024-09-12T02:01:38.319752+0000","last_peered":"2024-09-12T02:01:38.319752+0000","last_clean":"2024-09-12T02:01:38.319752+0000","last_became_active":"2024-09-12T01:59:55.487619+0000","last_became_peered":"2024-09-12T01:59:55.487619+0000","last_unstale":"2024-09-12T02:01:38.319752+0000","last_undegraded":"2024-09-12T02:01:38.319752+0000","last_fullsized":"2024-09-12T02:01:38.319752+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:32:16.882373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"80'3","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.567124+0000","last_change":"2024-09-12T01:59:53.474418+0000","last_active":"2024-09-12T02:00:09.567124+0000","last_peered":"2024-09-12T02:00:09.567124+0000","last_clean":"2024-09-12T02:00:09.567124+0000","last_became_active":"2024-09-12T01:59:53.474059+0000","last_became_peered":"2024-09-12T01:59:53.474059+0000","last_unstale":"2024-09-12T02:00:09.567124+0000","last_undegraded":"2024-09-12T02:00:09.567124+0000","last_fullsized":"2024-09-12T02:00:09.567124+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:10:54.963435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"82'15","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.333121+0000","last_change":"2024-09-12T01:59:55.493394+0000","last_active":"2024-09-12T02:01:38.333121+0000","last_peered":"2024-09-12T02:01:38.333121+0000","last_clean":"2024-09-12T02:01:38.333121+0000","last_became_active":"2024-09-12T01:59:55.491669+0000","last_became_peered":"2024-09-12T01:59:55.491669+0000","last_unstale":"2024-09-12T02:01:38.333121+0000","last_undegraded":"2024-09-12T02:01:38.333121+0000","last_fullsized":"2024-09-12T02:01:38.333121+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:14:40.871154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"80'3","reported_seq":126,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:42.692234+0000","last_change":"2024-09-12T01:59:53.471623+0000","last_active":"2024-09-12T02:01:42.692234+0000","last_peered":"2024-09-12T02:01:42.692234+0000","last_clean":"2024-09-12T02:01:42.692234+0000","last_became_active":"2024-09-12T01:59:53.471294+0000","last_became_peered":"2024-09-12T01:59:53.471294+0000","last_unstale":"2024-09-12T02:01:42.692234+0000","last_undegraded":"2024-09-12T02:01:42.692234+0000","last_fullsized":"2024-09-12T02:01:42.692234+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:51:54.130883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":102,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.322144+0000","last_change":"2024-09-12T01:59:55.493082+0000","last_active":"2024-09-12T02:01:38.322144+0000","last_peered":"2024-09-12T02:01:38.322144+0000","last_clean":"2024-09-12T02:01:38.322144+0000","last_became_active":"2024-09-12T01:59:55.492778+0000","last_became_peered":"2024-09-12T01:59:55.492778+0000","last_unstale":"2024-09-12T02:01:38.322144+0000","last_undegraded":"2024-09-12T02:01:38.322144+0000","last_fullsized":"2024-09-12T02:01:38.322144+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:59:56.466377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'15","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.588050+0000","last_change":"2024-09-12T02:00:09.588050+0000","last_active":"2024-09-12T02:00:09.588050+0000","last_peered":"2024-09-12T02:00:09.588050+0000","last_clean":"2024-09-12T02:00:09.588050+0000","last_became_active":"2024-09-12T02:00:09.575478+0000","last_became_peered":"2024-09-12T02:00:09.575478+0000","last_unstale":"2024-09-12T02:00:09.588050+0000","last_undegraded":"2024-09-12T02:00:09.588050+0000","last_fullsized":"2024-09-12T02:00:09.588050+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:38:45.484167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.313876+0000","last_change":"2024-09-12T01:59:55.497601+0000","last_active":"2024-09-12T02:01:38.313876+0000","last_peered":"2024-09-12T02:01:38.313876+0000","last_clean":"2024-09-12T02:01:38.313876+0000","last_became_active":"2024-09-12T01:59:55.497202+0000","last_became_peered":"2024-09-12T01:59:55.497202+0000","last_unstale":"2024-09-12T02:01:38.313876+0000","last_undegraded":"2024-09-12T02:01:38.313876+0000","last_fullsized":"2024-09-12T02:01:38.313876+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:22:22.850736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"80'2","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.575359+0000","last_change":"2024-09-12T01:59:53.467575+0000","last_active":"2024-09-12T02:00:09.575359+0000","last_peered":"2024-09-12T02:00:09.575359+0000","last_clean":"2024-09-12T02:00:09.575359+0000","last_became_active":"2024-09-12T01:59:53.467146+0000","last_became_peered":"2024-09-12T01:59:53.467146+0000","last_unstale":"2024-09-12T02:00:09.575359+0000","last_undegraded":"2024-09-12T02:00:09.575359+0000","last_fullsized":"2024-09-12T02:00:09.575359+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:07:25.962496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"82'5","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.314382+0000","last_change":"2024-09-12T01:59:55.498297+0000","last_active":"2024-09-12T02:01:38.314382+0000","last_peered":"2024-09-12T02:01:38.314382+0000","last_clean":"2024-09-12T02:01:38.314382+0000","last_became_active":"2024-09-12T01:59:55.497367+0000","last_became_peered":"2024-09-12T01:59:55.497367+0000","last_unstale":"2024-09-12T02:01:38.314382+0000","last_undegraded":"2024-09-12T02:01:38.314382+0000","last_fullsized":"2024-09-12T02:01:38.314382+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:58:32.717710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"80'2","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.562292+0000","last_change":"2024-09-12T01:59:53.466664+0000","last_active":"2024-09-12T02:00:09.562292+0000","last_peered":"2024-09-12T02:00:09.562292+0000","last_clean":"2024-09-12T02:00:09.562292+0000","last_became_active":"2024-09-12T01:59:53.466318+0000","last_became_peered":"2024-09-12T01:59:53.466318+0000","last_unstale":"2024-09-12T02:00:09.562292+0000","last_undegraded":"2024-09-12T02:00:09.562292+0000","last_fullsized":"2024-09-12T02:00:09.562292+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:12:13.541420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.313289+0000","last_change":"2024-09-12T01:59:55.490420+0000","last_active":"2024-09-12T02:01:38.313289+0000","last_peered":"2024-09-12T02:01:38.313289+0000","last_clean":"2024-09-12T02:01:38.313289+0000","last_became_active":"2024-09-12T01:59:55.489757+0000","last_became_peered":"2024-09-12T01:59:55.489757+0000","last_unstale":"2024-09-12T02:01:38.313289+0000","last_undegraded":"2024-09-12T02:01:38.313289+0000","last_fullsized":"2024-09-12T02:01:38.313289+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:34:43.117142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'1","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.567185+0000","last_change":"2024-09-12T01:59:53.466399+0000","last_active":"2024-09-12T02:00:09.567185+0000","last_peered":"2024-09-12T02:00:09.567185+0000","last_clean":"2024-09-12T02:00:09.567185+0000","last_became_active":"2024-09-12T01:59:53.465982+0000","last_became_peered":"2024-09-12T01:59:53.465982+0000","last_unstale":"2024-09-12T02:00:09.567185+0000","last_undegraded":"2024-09-12T02:00:09.567185+0000","last_fullsized":"2024-09-12T02:00:09.567185+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:18:02.015040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.326034+0000","last_change":"2024-09-12T01:59:55.491527+0000","last_active":"2024-09-12T02:01:38.326034+0000","last_peered":"2024-09-12T02:01:38.326034+0000","last_clean":"2024-09-12T02:01:38.326034+0000","last_became_active":"2024-09-12T01:59:55.491165+0000","last_became_peered":"2024-09-12T01:59:55.491165+0000","last_unstale":"2024-09-12T02:01:38.326034+0000","last_undegraded":"2024-09-12T02:01:38.326034+0000","last_fullsized":"2024-09-12T02:01:38.326034+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:57:05.506384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"80'3","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.569187+0000","last_change":"2024-09-12T01:59:54.075752+0000","last_active":"2024-09-12T02:00:09.569187+0000","last_peered":"2024-09-12T02:00:09.569187+0000","last_clean":"2024-09-12T02:00:09.569187+0000","last_became_active":"2024-09-12T01:59:54.075387+0000","last_became_peered":"2024-09-12T01:59:54.075387+0000","last_unstale":"2024-09-12T02:00:09.569187+0000","last_undegraded":"2024-09-12T02:00:09.569187+0000","last_fullsized":"2024-09-12T02:00:09.569187+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:21:15.980591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.330203+0000","last_change":"2024-09-12T01:59:55.500935+0000","last_active":"2024-09-12T02:01:38.330203+0000","last_peered":"2024-09-12T02:01:38.330203+0000","last_clean":"2024-09-12T02:01:38.330203+0000","last_became_active":"2024-09-12T01:59:55.500206+0000","last_became_peered":"2024-09-12T01:59:55.500206+0000","last_unstale":"2024-09-12T02:01:38.330203+0000","last_undegraded":"2024-09-12T02:01:38.330203+0000","last_fullsized":"2024-09-12T02:01:38.330203+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:19:59.937701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":334,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.623254+0000","last_change":"2024-09-12T01:58:47.553853+0000","last_active":"2024-09-12T02:00:09.623254+0000","last_peered":"2024-09-12T02:00:09.623254+0000","last_clean":"2024-09-12T02:00:09.623254+0000","last_became_active":"2024-09-12T01:58:47.553357+0000","last_became_peered":"2024-09-12T01:58:47.553357+0000","last_unstale":"2024-09-12T02:00:09.623254+0000","last_undegraded":"2024-09-12T02:00:09.623254+0000","last_fullsized":"2024-09-12T02:00:09.623254+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:55:26.917733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:55:26.917733+0000","last_clean_scrub_stamp":"2024-09-12T01:55:26.917733+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:58:26.081766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"80'3","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.563615+0000","last_change":"2024-09-12T01:59:53.467907+0000","last_active":"2024-09-12T02:00:09.563615+0000","last_peered":"2024-09-12T02:00:09.563615+0000","last_clean":"2024-09-12T02:00:09.563615+0000","last_became_active":"2024-09-12T01:59:53.467164+0000","last_became_peered":"2024-09-12T01:59:53.467164+0000","last_unstale":"2024-09-12T02:00:09.563615+0000","last_undegraded":"2024-09-12T02:00:09.563615+0000","last_fullsized":"2024-09-12T02:00:09.563615+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:09:35.242708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.327715+0000","last_change":"2024-09-12T01:59:55.495109+0000","last_active":"2024-09-12T02:01:38.327715+0000","last_peered":"2024-09-12T02:01:38.327715+0000","last_clean":"2024-09-12T02:01:38.327715+0000","last_became_active":"2024-09-12T01:59:55.494791+0000","last_became_peered":"2024-09-12T01:59:55.494791+0000","last_unstale":"2024-09-12T02:01:38.327715+0000","last_undegraded":"2024-09-12T02:01:38.327715+0000","last_fullsized":"2024-09-12T02:01:38.327715+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:22:43.882050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"80'3","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.623115+0000","last_change":"2024-09-12T01:59:53.473286+0000","last_active":"2024-09-12T02:00:09.623115+0000","last_peered":"2024-09-12T02:00:09.623115+0000","last_clean":"2024-09-12T02:00:09.623115+0000","last_became_active":"2024-09-12T01:59:53.472976+0000","last_became_peered":"2024-09-12T01:59:53.472976+0000","last_unstale":"2024-09-12T02:00:09.623115+0000","last_undegraded":"2024-09-12T02:00:09.623115+0000","last_fullsized":"2024-09-12T02:00:09.623115+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:32:39.604173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"82'5","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.313749+0000","last_change":"2024-09-12T02:00:09.571159+0000","last_active":"2024-09-12T02:01:38.313749+0000","last_peered":"2024-09-12T02:01:38.313749+0000","last_clean":"2024-09-12T02:01:38.313749+0000","last_became_active":"2024-09-12T02:00:09.570672+0000","last_became_peered":"2024-09-12T02:00:09.570672+0000","last_unstale":"2024-09-12T02:01:38.313749+0000","last_undegraded":"2024-09-12T02:01:38.313749+0000","last_fullsized":"2024-09-12T02:01:38.313749+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:04:10.808942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"80'4","reported_seq":126,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:43.692856+0000","last_change":"2024-09-12T01:59:53.921856+0000","last_active":"2024-09-12T02:01:43.692856+0000","last_peered":"2024-09-12T02:01:43.692856+0000","last_clean":"2024-09-12T02:01:43.692856+0000","last_became_active":"2024-09-12T01:59:53.920672+0000","last_became_peered":"2024-09-12T01:59:53.920672+0000","last_unstale":"2024-09-12T02:01:43.692856+0000","last_undegraded":"2024-09-12T02:01:43.692856+0000","last_fullsized":"2024-09-12T02:01:43.692856+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:42:33.876493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":100,"num_read_kb":100,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"82'8","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.309498+0000","last_change":"2024-09-12T02:00:09.571185+0000","last_active":"2024-09-12T02:01:38.309498+0000","last_peered":"2024-09-12T02:01:38.309498+0000","last_clean":"2024-09-12T02:01:38.309498+0000","last_became_active":"2024-09-12T02:00:09.570769+0000","last_became_peered":"2024-09-12T02:00:09.570769+0000","last_unstale":"2024-09-12T02:01:38.309498+0000","last_undegraded":"2024-09-12T02:01:38.309498+0000","last_fullsized":"2024-09-12T02:01:38.309498+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:34:17.230435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.2","version":"80'1","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.622691+0000","last_change":"2024-09-12T01:59:53.472072+0000","last_active":"2024-09-12T02:00:09.622691+0000","last_peered":"2024-09-12T02:00:09.622691+0000","last_clean":"2024-09-12T02:00:09.622691+0000","last_became_active":"2024-09-12T01:59:53.471088+0000","last_became_peered":"2024-09-12T01:59:53.471088+0000","last_unstale":"2024-09-12T02:00:09.622691+0000","last_undegraded":"2024-09-12T02:00:09.622691+0000","last_fullsized":"2024-09-12T02:00:09.622691+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:02:13.879131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.330778+0000","last_change":"2024-09-12T01:59:55.502843+0000","last_active":"2024-09-12T02:01:38.330778+0000","last_peered":"2024-09-12T02:01:38.330778+0000","last_clean":"2024-09-12T02:01:38.330778+0000","last_became_active":"2024-09-12T01:59:55.499296+0000","last_became_peered":"2024-09-12T01:59:55.499296+0000","last_unstale":"2024-09-12T02:01:38.330778+0000","last_undegraded":"2024-09-12T02:01:38.330778+0000","last_fullsized":"2024-09-12T02:01:38.330778+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:04:21.787370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.575061+0000","last_change":"2024-09-12T02:00:09.575061+0000","last_active":"2024-09-12T02:00:09.575061+0000","last_peered":"2024-09-12T02:00:09.575061+0000","last_clean":"2024-09-12T02:00:09.575061+0000","last_became_active":"2024-09-12T02:00:09.574653+0000","last_became_peered":"2024-09-12T02:00:09.574653+0000","last_unstale":"2024-09-12T02:00:09.575061+0000","last_undegraded":"2024-09-12T02:00:09.575061+0000","last_fullsized":"2024-09-12T02:00:09.575061+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:32:27.931211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"82'5","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.320660+0000","last_change":"2024-09-12T01:59:55.494162+0000","last_active":"2024-09-12T02:01:38.320660+0000","last_peered":"2024-09-12T02:01:38.320660+0000","last_clean":"2024-09-12T02:01:38.320660+0000","last_became_active":"2024-09-12T01:59:55.493802+0000","last_became_peered":"2024-09-12T01:59:55.493802+0000","last_unstale":"2024-09-12T02:01:38.320660+0000","last_undegraded":"2024-09-12T02:01:38.320660+0000","last_fullsized":"2024-09-12T02:01:38.320660+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:22:16.302683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"80'1","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.575278+0000","last_change":"2024-09-12T01:59:53.921669+0000","last_active":"2024-09-12T02:00:09.575278+0000","last_peered":"2024-09-12T02:00:09.575278+0000","last_clean":"2024-09-12T02:00:09.575278+0000","last_became_active":"2024-09-12T01:59:53.921328+0000","last_became_peered":"2024-09-12T01:59:53.921328+0000","last_unstale":"2024-09-12T02:00:09.575278+0000","last_undegraded":"2024-09-12T02:00:09.575278+0000","last_fullsized":"2024-09-12T02:00:09.575278+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:46:33.313726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.311067+0000","last_change":"2024-09-12T01:59:55.488986+0000","last_active":"2024-09-12T02:01:38.311067+0000","last_peered":"2024-09-12T02:01:38.311067+0000","last_clean":"2024-09-12T02:01:38.311067+0000","last_became_active":"2024-09-12T01:59:55.488632+0000","last_became_peered":"2024-09-12T01:59:55.488632+0000","last_unstale":"2024-09-12T02:01:38.311067+0000","last_undegraded":"2024-09-12T02:01:38.311067+0000","last_fullsized":"2024-09-12T02:01:38.311067+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:20:57.596976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"82'11","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:21.501527+0000","last_change":"2024-09-12T01:59:53.467648+0000","last_active":"2024-09-12T02:00:21.501527+0000","last_peered":"2024-09-12T02:00:21.501527+0000","last_clean":"2024-09-12T02:00:21.501527+0000","last_became_active":"2024-09-12T01:59:53.467255+0000","last_became_peered":"2024-09-12T01:59:53.467255+0000","last_unstale":"2024-09-12T02:00:21.501527+0000","last_undegraded":"2024-09-12T02:00:21.501527+0000","last_fullsized":"2024-09-12T02:00:21.501527+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:37:09.897388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"82'5","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.328722+0000","last_change":"2024-09-12T01:59:55.487603+0000","last_active":"2024-09-12T02:01:38.328722+0000","last_peered":"2024-09-12T02:01:38.328722+0000","last_clean":"2024-09-12T02:01:38.328722+0000","last_became_active":"2024-09-12T01:59:55.487334+0000","last_became_peered":"2024-09-12T01:59:55.487334+0000","last_unstale":"2024-09-12T02:01:38.328722+0000","last_undegraded":"2024-09-12T02:01:38.328722+0000","last_fullsized":"2024-09-12T02:01:38.328722+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:23:16.669515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'4","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.563870+0000","last_change":"2024-09-12T01:59:54.076156+0000","last_active":"2024-09-12T02:00:09.563870+0000","last_peered":"2024-09-12T02:00:09.563870+0000","last_clean":"2024-09-12T02:00:09.563870+0000","last_became_active":"2024-09-12T01:59:54.075794+0000","last_became_peered":"2024-09-12T01:59:54.075794+0000","last_unstale":"2024-09-12T02:00:09.563870+0000","last_undegraded":"2024-09-12T02:00:09.563870+0000","last_fullsized":"2024-09-12T02:00:09.563870+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:06:17.431696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.333603+0000","last_change":"2024-09-12T01:59:55.494604+0000","last_active":"2024-09-12T02:01:38.333603+0000","last_peered":"2024-09-12T02:01:38.333603+0000","last_clean":"2024-09-12T02:01:38.333603+0000","last_became_active":"2024-09-12T01:59:55.494300+0000","last_became_peered":"2024-09-12T01:59:55.494300+0000","last_unstale":"2024-09-12T02:01:38.333603+0000","last_undegraded":"2024-09-12T02:01:38.333603+0000","last_fullsized":"2024-09-12T02:01:38.333603+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:18:56.359299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.622205+0000","last_change":"2024-09-12T01:59:53.921832+0000","last_active":"2024-09-12T02:00:09.622205+0000","last_peered":"2024-09-12T02:00:09.622205+0000","last_clean":"2024-09-12T02:00:09.622205+0000","last_became_active":"2024-09-12T01:59:53.921379+0000","last_became_peered":"2024-09-12T01:59:53.921379+0000","last_unstale":"2024-09-12T02:00:09.622205+0000","last_undegraded":"2024-09-12T02:00:09.622205+0000","last_fullsized":"2024-09-12T02:00:09.622205+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:26:02.277149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"82'2","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.281554+0000","last_change":"2024-09-12T01:59:55.493198+0000","last_active":"2024-09-12T02:01:38.281554+0000","last_peered":"2024-09-12T02:01:38.281554+0000","last_clean":"2024-09-12T02:01:38.281554+0000","last_became_active":"2024-09-12T01:59:55.492834+0000","last_became_peered":"2024-09-12T01:59:55.492834+0000","last_unstale":"2024-09-12T02:01:38.281554+0000","last_undegraded":"2024-09-12T02:01:38.281554+0000","last_fullsized":"2024-09-12T02:01:38.281554+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:58:43.872921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"80'3","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.621258+0000","last_change":"2024-09-12T01:59:53.466528+0000","last_active":"2024-09-12T02:00:09.621258+0000","last_peered":"2024-09-12T02:00:09.621258+0000","last_clean":"2024-09-12T02:00:09.621258+0000","last_became_active":"2024-09-12T01:59:53.466182+0000","last_became_peered":"2024-09-12T01:59:53.466182+0000","last_unstale":"2024-09-12T02:00:09.621258+0000","last_undegraded":"2024-09-12T02:00:09.621258+0000","last_fullsized":"2024-09-12T02:00:09.621258+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:00:26.083539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.316786+0000","last_change":"2024-09-12T01:59:55.494511+0000","last_active":"2024-09-12T02:01:38.316786+0000","last_peered":"2024-09-12T02:01:38.316786+0000","last_clean":"2024-09-12T02:01:38.316786+0000","last_became_active":"2024-09-12T01:59:55.493607+0000","last_became_peered":"2024-09-12T01:59:55.493607+0000","last_unstale":"2024-09-12T02:01:38.316786+0000","last_undegraded":"2024-09-12T02:01:38.316786+0000","last_fullsized":"2024-09-12T02:01:38.316786+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:55:27.434447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"80'1","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.569211+0000","last_change":"2024-09-12T01:59:53.467385+0000","last_active":"2024-09-12T02:00:09.569211+0000","last_peered":"2024-09-12T02:00:09.569211+0000","last_clean":"2024-09-12T02:00:09.569211+0000","last_became_active":"2024-09-12T01:59:53.466997+0000","last_became_peered":"2024-09-12T01:59:53.466997+0000","last_unstale":"2024-09-12T02:00:09.569211+0000","last_undegraded":"2024-09-12T02:00:09.569211+0000","last_fullsized":"2024-09-12T02:00:09.569211+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:54:47.154036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.316513+0000","last_change":"2024-09-12T01:59:55.490757+0000","last_active":"2024-09-12T02:01:38.316513+0000","last_peered":"2024-09-12T02:01:38.316513+0000","last_clean":"2024-09-12T02:01:38.316513+0000","last_became_active":"2024-09-12T01:59:55.490451+0000","last_became_peered":"2024-09-12T01:59:55.490451+0000","last_unstale":"2024-09-12T02:01:38.316513+0000","last_undegraded":"2024-09-12T02:01:38.316513+0000","last_fullsized":"2024-09-12T02:01:38.316513+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:21:26.139049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.574833+0000","last_change":"2024-09-12T01:59:53.472898+0000","last_active":"2024-09-12T02:00:09.574833+0000","last_peered":"2024-09-12T02:00:09.574833+0000","last_clean":"2024-09-12T02:00:09.574833+0000","last_became_active":"2024-09-12T01:59:53.472561+0000","last_became_peered":"2024-09-12T01:59:53.472561+0000","last_unstale":"2024-09-12T02:00:09.574833+0000","last_undegraded":"2024-09-12T02:00:09.574833+0000","last_fullsized":"2024-09-12T02:00:09.574833+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:55:59.933804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"82'5","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.328406+0000","last_change":"2024-09-12T01:59:55.497731+0000","last_active":"2024-09-12T02:01:38.328406+0000","last_peered":"2024-09-12T02:01:38.328406+0000","last_clean":"2024-09-12T02:01:38.328406+0000","last_became_active":"2024-09-12T01:59:55.497356+0000","last_became_peered":"2024-09-12T01:59:55.497356+0000","last_unstale":"2024-09-12T02:01:38.328406+0000","last_undegraded":"2024-09-12T02:01:38.328406+0000","last_fullsized":"2024-09-12T02:01:38.328406+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:57:18.795865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"82'15","reported_seq":38,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.318747+0000","last_change":"2024-09-12T02:00:09.624453+0000","last_active":"2024-09-12T02:01:38.318747+0000","last_peered":"2024-09-12T02:01:38.318747+0000","last_clean":"2024-09-12T02:01:38.318747+0000","last_became_active":"2024-09-12T02:00:09.624049+0000","last_became_peered":"2024-09-12T02:00:09.624049+0000","last_unstale":"2024-09-12T02:01:38.318747+0000","last_undegraded":"2024-09-12T02:01:38.318747+0000","last_fullsized":"2024-09-12T02:01:38.318747+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:17:40.417275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"80'2","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.575060+0000","last_change":"2024-09-12T01:59:53.470194+0000","last_active":"2024-09-12T02:00:09.575060+0000","last_peered":"2024-09-12T02:00:09.575060+0000","last_clean":"2024-09-12T02:00:09.575060+0000","last_became_active":"2024-09-12T01:59:53.469772+0000","last_became_peered":"2024-09-12T01:59:53.469772+0000","last_unstale":"2024-09-12T02:00:09.575060+0000","last_undegraded":"2024-09-12T02:00:09.575060+0000","last_fullsized":"2024-09-12T02:00:09.575060+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:52:57.709596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.329782+0000","last_change":"2024-09-12T01:59:55.497693+0000","last_active":"2024-09-12T02:01:38.329782+0000","last_peered":"2024-09-12T02:01:38.329782+0000","last_clean":"2024-09-12T02:01:38.329782+0000","last_became_active":"2024-09-12T01:59:55.497380+0000","last_became_peered":"2024-09-12T01:59:55.497380+0000","last_unstale":"2024-09-12T02:01:38.329782+0000","last_undegraded":"2024-09-12T02:01:38.329782+0000","last_fullsized":"2024-09-12T02:01:38.329782+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:42:27.943530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"80'4","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.575274+0000","last_change":"2024-09-12T01:59:53.473044+0000","last_active":"2024-09-12T02:00:09.575274+0000","last_peered":"2024-09-12T02:00:09.575274+0000","last_clean":"2024-09-12T02:00:09.575274+0000","last_became_active":"2024-09-12T01:59:53.472569+0000","last_became_peered":"2024-09-12T01:59:53.472569+0000","last_unstale":"2024-09-12T02:00:09.575274+0000","last_undegraded":"2024-09-12T02:00:09.575274+0000","last_fullsized":"2024-09-12T02:00:09.575274+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:41:29.716836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.328172+0000","last_change":"2024-09-12T01:59:55.497377+0000","last_active":"2024-09-12T02:01:38.328172+0000","last_peered":"2024-09-12T02:01:38.328172+0000","last_clean":"2024-09-12T02:01:38.328172+0000","last_became_active":"2024-09-12T01:59:55.497078+0000","last_became_peered":"2024-09-12T01:59:55.497078+0000","last_unstale":"2024-09-12T02:01:38.328172+0000","last_undegraded":"2024-09-12T02:01:38.328172+0000","last_fullsized":"2024-09-12T02:01:38.328172+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:59:57.362350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.621555+0000","last_change":"2024-09-12T01:59:53.920868+0000","last_active":"2024-09-12T02:00:09.621555+0000","last_peered":"2024-09-12T02:00:09.621555+0000","last_clean":"2024-09-12T02:00:09.621555+0000","last_became_active":"2024-09-12T01:59:53.920481+0000","last_became_peered":"2024-09-12T01:59:53.920481+0000","last_unstale":"2024-09-12T02:00:09.621555+0000","last_undegraded":"2024-09-12T02:00:09.621555+0000","last_fullsized":"2024-09-12T02:00:09.621555+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:49:27.085388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"82'5","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.315986+0000","last_change":"2024-09-12T01:59:55.497703+0000","last_active":"2024-09-12T02:01:38.315986+0000","last_peered":"2024-09-12T02:01:38.315986+0000","last_clean":"2024-09-12T02:01:38.315986+0000","last_became_active":"2024-09-12T01:59:55.497397+0000","last_became_peered":"2024-09-12T01:59:55.497397+0000","last_unstale":"2024-09-12T02:01:38.315986+0000","last_undegraded":"2024-09-12T02:01:38.315986+0000","last_fullsized":"2024-09-12T02:01:38.315986+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:54:02.803203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"80'9","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.565145+0000","last_change":"2024-09-12T01:59:53.469978+0000","last_active":"2024-09-12T02:00:09.565145+0000","last_peered":"2024-09-12T02:00:09.565145+0000","last_clean":"2024-09-12T02:00:09.565145+0000","last_became_active":"2024-09-12T01:59:53.469648+0000","last_became_peered":"2024-09-12T01:59:53.469648+0000","last_unstale":"2024-09-12T02:00:09.565145+0000","last_undegraded":"2024-09-12T02:00:09.565145+0000","last_fullsized":"2024-09-12T02:00:09.565145+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:08:37.720452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":68779,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":627,"num_read_kb":386,"num_write":170,"num_write_kb":119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":206337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":144,"ondisk_log_size":144,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3563520,"data_stored":3542208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":74,"seq":317827579939,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32060,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713092,"statfs":{"total":95995035648,"available":95962206208,"internally_reserved":0,"allocated":897024,"data_stored":435916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":67,"seq":287762808872,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32736,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712416,"statfs":{"total":95995035648,"available":95961513984,"internally_reserved":0,"allocated":1589248,"data_stored":1091920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":62,"seq":266287972397,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":905216,"data_stored":436040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201330,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":897024,"data_stored":435848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430263,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32648,"kb_used_data":1464,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712504,"statfs":{"total":95995035648,"available":95961604096,"internally_reserved":0,"allocated":1499136,"data_stored":1028033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593788,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36216,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708936,"statfs":{"total":95995035648,"available":95957950464,"internally_reserved":0,"allocated":892928,"data_stored":435429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822721,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32660,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712492,"statfs":{"total":95995035648,"available":95961591808,"internally_reserved":0,"allocated":1511424,"data_stored":1027717,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986245,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":897024,"data_stored":435866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215178,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":909312,"data_stored":435740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":17,"seq":73014444112,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36304,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708848,"statfs":{"total":95995035648,"available":95957860352,"internally_reserved":0,"allocated":983040,"data_stored":501741,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574933,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32120,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713032,"statfs":{"total":95995035648,"available":95962144768,"internally_reserved":0,"allocated":958464,"data_stored":499857,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738458,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":913408,"data_stored":436700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":64635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":66519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":66213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T02:01:50.105 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-12T02:01:50.105 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-12T02:01:50.366 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:01:50.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:01:50 smithi016 ceph-mon[30947]: pgmap v332: 97 pgs: 97 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1023 B/s wr, 6 op/s 2024-09-12T02:01:50.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:01:50 smithi184 ceph-mon[35593]: pgmap v332: 97 pgs: 97 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1023 B/s wr, 6 op/s 2024-09-12T02:01:50.668 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:01:50 smithi033 ceph-mon[35332]: pgmap v332: 97 pgs: 97 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1023 B/s wr, 6 op/s 2024-09-12T02:01:50.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:50.824+0000 7ff26bcb3640 1 -- 172.21.15.16:0/2154593376 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff264102d30 msgr2=0x7ff264103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:50.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:50.824+0000 7ff26bcb3640 1 --2- 172.21.15.16:0/2154593376 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff264102d30 0x7ff264103110 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7ff260009930 tx=0x7ff26002a420 comp rx=0 tx=0).stop 2024-09-12T02:01:50.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:50.824+0000 7ff26bcb3640 1 -- 172.21.15.16:0/2154593376 shutdown_connections 2024-09-12T02:01:50.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:50.824+0000 7ff26bcb3640 1 --2- 172.21.15.16:0/2154593376 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2641082a0 0x7ff26410ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:50.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:50.824+0000 7ff26bcb3640 1 --2- 172.21.15.16:0/2154593376 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff2641036e0 0x7ff264107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:50.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:50.824+0000 7ff26bcb3640 1 --2- 172.21.15.16:0/2154593376 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff264102d30 0x7ff264103110 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:50.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:50.824+0000 7ff26bcb3640 1 -- 172.21.15.16:0/2154593376 >> 172.21.15.16:0/2154593376 conn(0x7ff2640fc7d0 msgr2=0x7ff2640febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:01:50.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:50.824+0000 7ff26bcb3640 1 -- 172.21.15.16:0/2154593376 shutdown_connections 2024-09-12T02:01:50.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:50.824+0000 7ff26bcb3640 1 -- 172.21.15.16:0/2154593376 wait complete. 2024-09-12T02:01:50.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:50.824+0000 7ff26bcb3640 1 Processor -- start 2024-09-12T02:01:50.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:50.824+0000 7ff26bcb3640 1 -- start start 2024-09-12T02:01:50.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:50.825+0000 7ff26bcb3640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff264102d30 0x7ff264100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:50.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:50.825+0000 7ff26bcb3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2641036e0 0x7ff264100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:50.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:50.825+0000 7ff26bcb3640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2641082a0 0x7ff264101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:50.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:50.825+0000 7ff26bcb3640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff2641120a0 con 0x7ff2641036e0 2024-09-12T02:01:50.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:50.825+0000 7ff26bcb3640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff264111f20 con 0x7ff2641082a0 2024-09-12T02:01:50.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:50.825+0000 7ff26bcb3640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff264112220 con 0x7ff264102d30 2024-09-12T02:01:50.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:50.825+0000 7ff26924e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2641036e0 0x7ff264100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:50.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:50.825+0000 7ff26924e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2641036e0 0x7ff264100ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46732/0 (socket says 172.21.15.16:46732) 2024-09-12T02:01:50.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:50.825+0000 7ff26924e640 1 -- 172.21.15.16:0/3731649132 learned_addr learned my addr 172.21.15.16:0/3731649132 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:01:50.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:50.825+0000 7ff269a4f640 1 --2- 172.21.15.16:0/3731649132 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff264102d30 0x7ff264100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:50.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:50.825+0000 7ff26a250640 1 --2- 172.21.15.16:0/3731649132 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2641082a0 0x7ff264101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:50.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:50.825+0000 7ff26924e640 1 -- 172.21.15.16:0/3731649132 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff264102d30 msgr2=0x7ff264100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:50.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:50.825+0000 7ff26924e640 1 --2- 172.21.15.16:0/3731649132 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff264102d30 0x7ff264100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:50.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:50.825+0000 7ff26924e640 1 -- 172.21.15.16:0/3731649132 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2641082a0 msgr2=0x7ff264101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:50.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:50.825+0000 7ff26924e640 1 --2- 172.21.15.16:0/3731649132 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2641082a0 0x7ff264101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:50.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:50.825+0000 7ff26924e640 1 -- 172.21.15.16:0/3731649132 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2641a6a60 con 0x7ff2641036e0 2024-09-12T02:01:50.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:50.825+0000 7ff26a250640 1 --2- 172.21.15.16:0/3731649132 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2641082a0 0x7ff264101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:01:50.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:50.825+0000 7ff26924e640 1 --2- 172.21.15.16:0/3731649132 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2641036e0 0x7ff264100ed0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7ff25400c8c0 tx=0x7ff25400cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:01:50.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:50.825+0000 7ff25affd640 1 -- 172.21.15.16:0/3731649132 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff254007bb0 con 0x7ff2641036e0 2024-09-12T02:01:50.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:50.826+0000 7ff25affd640 1 -- 172.21.15.16:0/3731649132 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff254007d50 con 0x7ff2641036e0 2024-09-12T02:01:50.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:50.826+0000 7ff26bcb3640 1 -- 172.21.15.16:0/3731649132 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2641a6d50 con 0x7ff2641036e0 2024-09-12T02:01:50.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:50.826+0000 7ff25affd640 1 -- 172.21.15.16:0/3731649132 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff254018830 con 0x7ff2641036e0 2024-09-12T02:01:50.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:50.826+0000 7ff26bcb3640 1 -- 172.21.15.16:0/3731649132 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2641a7230 con 0x7ff2641036e0 2024-09-12T02:01:50.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:50.827+0000 7ff26bcb3640 1 -- 172.21.15.16:0/3731649132 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff26406a850 con 0x7ff2641036e0 2024-09-12T02:01:50.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:50.828+0000 7ff25affd640 1 -- 172.21.15.16:0/3731649132 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff254018ab0 con 0x7ff2641036e0 2024-09-12T02:01:50.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:50.829+0000 7ff25affd640 1 --2- 172.21.15.16:0/3731649132 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7ff2500a2b80 0x7ff2500a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:50.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:50.829+0000 7ff269a4f640 1 --2- 172.21.15.16:0/3731649132 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7ff2500a2b80 0x7ff2500a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:50.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:50.829+0000 7ff25affd640 1 -- 172.21.15.16:0/3731649132 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7ff2540cf1f0 con 0x7ff2641036e0 2024-09-12T02:01:50.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:50.829+0000 7ff269a4f640 1 --2- 172.21.15.16:0/3731649132 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7ff2500a2b80 0x7ff2500a5040 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ff260000c00 tx=0x7ff260005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:01:50.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:50.833+0000 7ff25affd640 1 -- 172.21.15.16:0/3731649132 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff254014030 con 0x7ff2641036e0 2024-09-12T02:01:51.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:51.006+0000 7ff26bcb3640 1 -- 172.21.15.16:0/3731649132 --> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff264062630 con 0x7ff2500a2b80 2024-09-12T02:01:51.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:51.012+0000 7ff25affd640 1 -- 172.21.15.16:0/3731649132 <== mgr.14150 v2:172.21.15.16:6800/3887444820 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258815 (secure 0 0 0) 0x7ff264062630 con 0x7ff2500a2b80 2024-09-12T02:01:51.015 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T02:01:51.015 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-12T02:01:51.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:51.016+0000 7ff26bcb3640 1 -- 172.21.15.16:0/3731649132 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7ff2500a2b80 msgr2=0x7ff2500a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:51.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:51.016+0000 7ff26bcb3640 1 --2- 172.21.15.16:0/3731649132 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7ff2500a2b80 0x7ff2500a5040 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ff260000c00 tx=0x7ff260005a20 comp rx=0 tx=0).stop 2024-09-12T02:01:51.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:51.016+0000 7ff26bcb3640 1 -- 172.21.15.16:0/3731649132 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2641036e0 msgr2=0x7ff264100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:51.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:51.016+0000 7ff26bcb3640 1 --2- 172.21.15.16:0/3731649132 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2641036e0 0x7ff264100ed0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7ff25400c8c0 tx=0x7ff25400cd80 comp rx=0 tx=0).stop 2024-09-12T02:01:51.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:51.016+0000 7ff26bcb3640 1 -- 172.21.15.16:0/3731649132 shutdown_connections 2024-09-12T02:01:51.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:51.016+0000 7ff26bcb3640 1 --2- 172.21.15.16:0/3731649132 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7ff2500a2b80 0x7ff2500a5040 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:51.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:51.016+0000 7ff26bcb3640 1 --2- 172.21.15.16:0/3731649132 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2641082a0 0x7ff264101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:51.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:51.016+0000 7ff26bcb3640 1 --2- 172.21.15.16:0/3731649132 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2641036e0 0x7ff264100ed0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:51.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:51.016+0000 7ff26bcb3640 1 --2- 172.21.15.16:0/3731649132 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff264102d30 0x7ff264100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:51.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:51.016+0000 7ff26bcb3640 1 -- 172.21.15.16:0/3731649132 >> 172.21.15.16:0/3731649132 conn(0x7ff2640fc7d0 msgr2=0x7ff264105d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:01:51.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:51.016+0000 7ff26bcb3640 1 -- 172.21.15.16:0/3731649132 shutdown_connections 2024-09-12T02:01:51.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:51.016+0000 7ff26bcb3640 1 -- 172.21.15.16:0/3731649132 wait complete. 2024-09-12T02:01:51.345 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:01:51 smithi016 ceph-mon[30947]: from='client.15246 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T02:01:51.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:01:51 smithi184 ceph-mon[35593]: from='client.15246 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T02:01:51.668 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:01:51 smithi033 ceph-mon[35332]: from='client.15246 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T02:01:51.769 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":333,"stamp":"2024-09-12T02:01:50.325897+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":659381,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":780,"num_read_kb":512,"num_write":353,"num_write_kb":3059,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":1180848,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":783,"ondisk_log_size":783,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":374280,"kb_used_data":12552,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124567544,"statfs":{"total":1151940427776,"available":1151557165056,"internally_reserved":0,"allocated":12853248,"data_stored":7200807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":370193723},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5012,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":91,"num_read_kb":55,"num_write":10,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002529"},"pg_stats":[{"pgid":"3.3f","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.318917+0000","last_change":"2024-09-12T01:59:55.503882+0000","last_active":"2024-09-12T02:01:38.318917+0000","last_peered":"2024-09-12T02:01:38.318917+0000","last_clean":"2024-09-12T02:01:38.318917+0000","last_became_active":"2024-09-12T01:59:55.503513+0000","last_became_peered":"2024-09-12T01:59:55.503513+0000","last_unstale":"2024-09-12T02:01:38.318917+0000","last_undegraded":"2024-09-12T02:01:38.318917+0000","last_fullsized":"2024-09-12T02:01:38.318917+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:43:13.098527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"82'4","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.322981+0000","last_change":"2024-09-12T01:59:55.492965+0000","last_active":"2024-09-12T02:01:38.322981+0000","last_peered":"2024-09-12T02:01:38.322981+0000","last_clean":"2024-09-12T02:01:38.322981+0000","last_became_active":"2024-09-12T01:59:55.492613+0000","last_became_peered":"2024-09-12T01:59:55.492613+0000","last_unstale":"2024-09-12T02:01:38.322981+0000","last_undegraded":"2024-09-12T02:01:38.322981+0000","last_fullsized":"2024-09-12T02:01:38.322981+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:58:20.889770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.332184+0000","last_change":"2024-09-12T01:59:55.497334+0000","last_active":"2024-09-12T02:01:38.332184+0000","last_peered":"2024-09-12T02:01:38.332184+0000","last_clean":"2024-09-12T02:01:38.332184+0000","last_became_active":"2024-09-12T01:59:55.497037+0000","last_became_peered":"2024-09-12T01:59:55.497037+0000","last_unstale":"2024-09-12T02:01:38.332184+0000","last_undegraded":"2024-09-12T02:01:38.332184+0000","last_fullsized":"2024-09-12T02:01:38.332184+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:43:04.082918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.331162+0000","last_change":"2024-09-12T01:59:55.495888+0000","last_active":"2024-09-12T02:01:38.331162+0000","last_peered":"2024-09-12T02:01:38.331162+0000","last_clean":"2024-09-12T02:01:38.331162+0000","last_became_active":"2024-09-12T01:59:55.492684+0000","last_became_peered":"2024-09-12T01:59:55.492684+0000","last_unstale":"2024-09-12T02:01:38.331162+0000","last_undegraded":"2024-09-12T02:01:38.331162+0000","last_fullsized":"2024-09-12T02:01:38.331162+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:15:00.500006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"82'4","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.301731+0000","last_change":"2024-09-12T01:59:55.494613+0000","last_active":"2024-09-12T02:01:38.301731+0000","last_peered":"2024-09-12T02:01:38.301731+0000","last_clean":"2024-09-12T02:01:38.301731+0000","last_became_active":"2024-09-12T01:59:55.494205+0000","last_became_peered":"2024-09-12T01:59:55.494205+0000","last_unstale":"2024-09-12T02:01:38.301731+0000","last_undegraded":"2024-09-12T02:01:38.301731+0000","last_fullsized":"2024-09-12T02:01:38.301731+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:39:14.573280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.331517+0000","last_change":"2024-09-12T01:59:55.496659+0000","last_active":"2024-09-12T02:01:38.331517+0000","last_peered":"2024-09-12T02:01:38.331517+0000","last_clean":"2024-09-12T02:01:38.331517+0000","last_became_active":"2024-09-12T01:59:55.496232+0000","last_became_peered":"2024-09-12T01:59:55.496232+0000","last_unstale":"2024-09-12T02:01:38.331517+0000","last_undegraded":"2024-09-12T02:01:38.331517+0000","last_fullsized":"2024-09-12T02:01:38.331517+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:48:21.544764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"82'5","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.317177+0000","last_change":"2024-09-12T01:59:55.486789+0000","last_active":"2024-09-12T02:01:38.317177+0000","last_peered":"2024-09-12T02:01:38.317177+0000","last_clean":"2024-09-12T02:01:38.317177+0000","last_became_active":"2024-09-12T01:59:55.485921+0000","last_became_peered":"2024-09-12T01:59:55.485921+0000","last_unstale":"2024-09-12T02:01:38.317177+0000","last_undegraded":"2024-09-12T02:01:38.317177+0000","last_fullsized":"2024-09-12T02:01:38.317177+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:21:16.071121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.321850+0000","last_change":"2024-09-12T01:59:55.494525+0000","last_active":"2024-09-12T02:01:38.321850+0000","last_peered":"2024-09-12T02:01:38.321850+0000","last_clean":"2024-09-12T02:01:38.321850+0000","last_became_active":"2024-09-12T01:59:55.493998+0000","last_became_peered":"2024-09-12T01:59:55.493998+0000","last_unstale":"2024-09-12T02:01:38.321850+0000","last_undegraded":"2024-09-12T02:01:38.321850+0000","last_fullsized":"2024-09-12T02:01:38.321850+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:59:24.653660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"82'11","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.320666+0000","last_change":"2024-09-12T01:59:55.501466+0000","last_active":"2024-09-12T02:01:38.320666+0000","last_peered":"2024-09-12T02:01:38.320666+0000","last_clean":"2024-09-12T02:01:38.320666+0000","last_became_active":"2024-09-12T01:59:55.500800+0000","last_became_peered":"2024-09-12T01:59:55.500800+0000","last_unstale":"2024-09-12T02:01:38.320666+0000","last_undegraded":"2024-09-12T02:01:38.320666+0000","last_fullsized":"2024-09-12T02:01:38.320666+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:55:13.704791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"82'16","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.331670+0000","last_change":"2024-09-12T01:59:55.497986+0000","last_active":"2024-09-12T02:01:38.331670+0000","last_peered":"2024-09-12T02:01:38.331670+0000","last_clean":"2024-09-12T02:01:38.331670+0000","last_became_active":"2024-09-12T01:59:55.497691+0000","last_became_peered":"2024-09-12T01:59:55.497691+0000","last_unstale":"2024-09-12T02:01:38.331670+0000","last_undegraded":"2024-09-12T02:01:38.331670+0000","last_fullsized":"2024-09-12T02:01:38.331670+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:37:07.702606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.329986+0000","last_change":"2024-09-12T01:59:55.501339+0000","last_active":"2024-09-12T02:01:38.329986+0000","last_peered":"2024-09-12T02:01:38.329986+0000","last_clean":"2024-09-12T02:01:38.329986+0000","last_became_active":"2024-09-12T01:59:55.500700+0000","last_became_peered":"2024-09-12T01:59:55.500700+0000","last_unstale":"2024-09-12T02:01:38.329986+0000","last_undegraded":"2024-09-12T02:01:38.329986+0000","last_fullsized":"2024-09-12T02:01:38.329986+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:12:15.879183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.330871+0000","last_change":"2024-09-12T01:59:55.495196+0000","last_active":"2024-09-12T02:01:38.330871+0000","last_peered":"2024-09-12T02:01:38.330871+0000","last_clean":"2024-09-12T02:01:38.330871+0000","last_became_active":"2024-09-12T01:59:55.494889+0000","last_became_peered":"2024-09-12T01:59:55.494889+0000","last_unstale":"2024-09-12T02:01:38.330871+0000","last_undegraded":"2024-09-12T02:01:38.330871+0000","last_fullsized":"2024-09-12T02:01:38.330871+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:32:30.621946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"82'10","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.315946+0000","last_change":"2024-09-12T01:59:55.492565+0000","last_active":"2024-09-12T02:01:38.315946+0000","last_peered":"2024-09-12T02:01:38.315946+0000","last_clean":"2024-09-12T02:01:38.315946+0000","last_became_active":"2024-09-12T01:59:55.492168+0000","last_became_peered":"2024-09-12T01:59:55.492168+0000","last_unstale":"2024-09-12T02:01:38.315946+0000","last_undegraded":"2024-09-12T02:01:38.315946+0000","last_fullsized":"2024-09-12T02:01:38.315946+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:39:48.630202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.323015+0000","last_change":"2024-09-12T01:59:55.493152+0000","last_active":"2024-09-12T02:01:38.323015+0000","last_peered":"2024-09-12T02:01:38.323015+0000","last_clean":"2024-09-12T02:01:38.323015+0000","last_became_active":"2024-09-12T01:59:55.492794+0000","last_became_peered":"2024-09-12T01:59:55.492794+0000","last_unstale":"2024-09-12T02:01:38.323015+0000","last_undegraded":"2024-09-12T02:01:38.323015+0000","last_fullsized":"2024-09-12T02:01:38.323015+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:24:27.270502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.333374+0000","last_change":"2024-09-12T01:59:55.494651+0000","last_active":"2024-09-12T02:01:38.333374+0000","last_peered":"2024-09-12T02:01:38.333374+0000","last_clean":"2024-09-12T02:01:38.333374+0000","last_became_active":"2024-09-12T01:59:55.494355+0000","last_became_peered":"2024-09-12T01:59:55.494355+0000","last_unstale":"2024-09-12T02:01:38.333374+0000","last_undegraded":"2024-09-12T02:01:38.333374+0000","last_fullsized":"2024-09-12T02:01:38.333374+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:14:44.364158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.330799+0000","last_change":"2024-09-12T01:59:55.495049+0000","last_active":"2024-09-12T02:01:38.330799+0000","last_peered":"2024-09-12T02:01:38.330799+0000","last_clean":"2024-09-12T02:01:38.330799+0000","last_became_active":"2024-09-12T01:59:55.494688+0000","last_became_peered":"2024-09-12T01:59:55.494688+0000","last_unstale":"2024-09-12T02:01:38.330799+0000","last_undegraded":"2024-09-12T02:01:38.330799+0000","last_fullsized":"2024-09-12T02:01:38.330799+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:13:05.944577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.319055+0000","last_change":"2024-09-12T01:59:55.504146+0000","last_active":"2024-09-12T02:01:38.319055+0000","last_peered":"2024-09-12T02:01:38.319055+0000","last_clean":"2024-09-12T02:01:38.319055+0000","last_became_active":"2024-09-12T01:59:55.503646+0000","last_became_peered":"2024-09-12T01:59:55.503646+0000","last_unstale":"2024-09-12T02:01:38.319055+0000","last_undegraded":"2024-09-12T02:01:38.319055+0000","last_fullsized":"2024-09-12T02:01:38.319055+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:23:51.780795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"82'10","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.325350+0000","last_change":"2024-09-12T01:59:55.489930+0000","last_active":"2024-09-12T02:01:38.325350+0000","last_peered":"2024-09-12T02:01:38.325350+0000","last_clean":"2024-09-12T02:01:38.325350+0000","last_became_active":"2024-09-12T01:59:55.489619+0000","last_became_peered":"2024-09-12T01:59:55.489619+0000","last_unstale":"2024-09-12T02:01:38.325350+0000","last_undegraded":"2024-09-12T02:01:38.325350+0000","last_fullsized":"2024-09-12T02:01:38.325350+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:04:57.307091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.303800+0000","last_change":"2024-09-12T01:59:55.492098+0000","last_active":"2024-09-12T02:01:38.303800+0000","last_peered":"2024-09-12T02:01:38.303800+0000","last_clean":"2024-09-12T02:01:38.303800+0000","last_became_active":"2024-09-12T01:59:55.491370+0000","last_became_peered":"2024-09-12T01:59:55.491370+0000","last_unstale":"2024-09-12T02:01:38.303800+0000","last_undegraded":"2024-09-12T02:01:38.303800+0000","last_fullsized":"2024-09-12T02:01:38.303800+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:25:06.307903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.314843+0000","last_change":"2024-09-12T01:59:55.499938+0000","last_active":"2024-09-12T02:01:38.314843+0000","last_peered":"2024-09-12T02:01:38.314843+0000","last_clean":"2024-09-12T02:01:38.314843+0000","last_became_active":"2024-09-12T01:59:55.499619+0000","last_became_peered":"2024-09-12T01:59:55.499619+0000","last_unstale":"2024-09-12T02:01:38.314843+0000","last_undegraded":"2024-09-12T02:01:38.314843+0000","last_fullsized":"2024-09-12T02:01:38.314843+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:00:17.536120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"82'11","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.321068+0000","last_change":"2024-09-12T01:59:55.486106+0000","last_active":"2024-09-12T02:01:38.321068+0000","last_peered":"2024-09-12T02:01:38.321068+0000","last_clean":"2024-09-12T02:01:38.321068+0000","last_became_active":"2024-09-12T01:59:55.485325+0000","last_became_peered":"2024-09-12T01:59:55.485325+0000","last_unstale":"2024-09-12T02:01:38.321068+0000","last_undegraded":"2024-09-12T02:01:38.321068+0000","last_fullsized":"2024-09-12T02:01:38.321068+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:35:46.446224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"82'10","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.331650+0000","last_change":"2024-09-12T01:59:55.496438+0000","last_active":"2024-09-12T02:01:38.331650+0000","last_peered":"2024-09-12T02:01:38.331650+0000","last_clean":"2024-09-12T02:01:38.331650+0000","last_became_active":"2024-09-12T01:59:55.495885+0000","last_became_peered":"2024-09-12T01:59:55.495885+0000","last_unstale":"2024-09-12T02:01:38.331650+0000","last_undegraded":"2024-09-12T02:01:38.331650+0000","last_fullsized":"2024-09-12T02:01:38.331650+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:15:14.911335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.322674+0000","last_change":"2024-09-12T01:59:55.489350+0000","last_active":"2024-09-12T02:01:38.322674+0000","last_peered":"2024-09-12T02:01:38.322674+0000","last_clean":"2024-09-12T02:01:38.322674+0000","last_became_active":"2024-09-12T01:59:55.489028+0000","last_became_peered":"2024-09-12T01:59:55.489028+0000","last_unstale":"2024-09-12T02:01:38.322674+0000","last_undegraded":"2024-09-12T02:01:38.322674+0000","last_fullsized":"2024-09-12T02:01:38.322674+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:29:23.769004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.316094+0000","last_change":"2024-09-12T01:59:55.494532+0000","last_active":"2024-09-12T02:01:38.316094+0000","last_peered":"2024-09-12T02:01:38.316094+0000","last_clean":"2024-09-12T02:01:38.316094+0000","last_became_active":"2024-09-12T01:59:55.490982+0000","last_became_peered":"2024-09-12T01:59:55.490982+0000","last_unstale":"2024-09-12T02:01:38.316094+0000","last_undegraded":"2024-09-12T02:01:38.316094+0000","last_fullsized":"2024-09-12T02:01:38.316094+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:04:50.633730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"82'2","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.316143+0000","last_change":"2024-09-12T01:59:55.496057+0000","last_active":"2024-09-12T02:01:38.316143+0000","last_peered":"2024-09-12T02:01:38.316143+0000","last_clean":"2024-09-12T02:01:38.316143+0000","last_became_active":"2024-09-12T01:59:55.495710+0000","last_became_peered":"2024-09-12T01:59:55.495710+0000","last_unstale":"2024-09-12T02:01:38.316143+0000","last_undegraded":"2024-09-12T02:01:38.316143+0000","last_fullsized":"2024-09-12T02:01:38.316143+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:49:01.962537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"82'15","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.311246+0000","last_change":"2024-09-12T01:59:55.491594+0000","last_active":"2024-09-12T02:01:38.311246+0000","last_peered":"2024-09-12T02:01:38.311246+0000","last_clean":"2024-09-12T02:01:38.311246+0000","last_became_active":"2024-09-12T01:59:55.491233+0000","last_became_peered":"2024-09-12T01:59:55.491233+0000","last_unstale":"2024-09-12T02:01:38.311246+0000","last_undegraded":"2024-09-12T02:01:38.311246+0000","last_fullsized":"2024-09-12T02:01:38.311246+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:24:22.066777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.309442+0000","last_change":"2024-09-12T01:59:55.498331+0000","last_active":"2024-09-12T02:01:38.309442+0000","last_peered":"2024-09-12T02:01:38.309442+0000","last_clean":"2024-09-12T02:01:38.309442+0000","last_became_active":"2024-09-12T01:59:55.497983+0000","last_became_peered":"2024-09-12T01:59:55.497983+0000","last_unstale":"2024-09-12T02:01:38.309442+0000","last_undegraded":"2024-09-12T02:01:38.309442+0000","last_fullsized":"2024-09-12T02:01:38.309442+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:37:00.735231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"82'4","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.315811+0000","last_change":"2024-09-12T02:00:09.575358+0000","last_active":"2024-09-12T02:01:38.315811+0000","last_peered":"2024-09-12T02:01:38.315811+0000","last_clean":"2024-09-12T02:01:38.315811+0000","last_became_active":"2024-09-12T02:00:09.574913+0000","last_became_peered":"2024-09-12T02:00:09.574913+0000","last_unstale":"2024-09-12T02:01:38.315811+0000","last_undegraded":"2024-09-12T02:01:38.315811+0000","last_fullsized":"2024-09-12T02:01:38.315811+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:19:53.630453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.333406+0000","last_change":"2024-09-12T01:59:55.492419+0000","last_active":"2024-09-12T02:01:38.333406+0000","last_peered":"2024-09-12T02:01:38.333406+0000","last_clean":"2024-09-12T02:01:38.333406+0000","last_became_active":"2024-09-12T01:59:55.492003+0000","last_became_peered":"2024-09-12T01:59:55.492003+0000","last_unstale":"2024-09-12T02:01:38.333406+0000","last_undegraded":"2024-09-12T02:01:38.333406+0000","last_fullsized":"2024-09-12T02:01:38.333406+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:25:45.774679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.307126+0000","last_change":"2024-09-12T01:59:55.490125+0000","last_active":"2024-09-12T02:01:38.307126+0000","last_peered":"2024-09-12T02:01:38.307126+0000","last_clean":"2024-09-12T02:01:38.307126+0000","last_became_active":"2024-09-12T01:59:55.489636+0000","last_became_peered":"2024-09-12T01:59:55.489636+0000","last_unstale":"2024-09-12T02:01:38.307126+0000","last_undegraded":"2024-09-12T02:01:38.307126+0000","last_fullsized":"2024-09-12T02:01:38.307126+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:41:30.922185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"82'12","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.309816+0000","last_change":"2024-09-12T01:59:55.496322+0000","last_active":"2024-09-12T02:01:38.309816+0000","last_peered":"2024-09-12T02:01:38.309816+0000","last_clean":"2024-09-12T02:01:38.309816+0000","last_became_active":"2024-09-12T01:59:55.496005+0000","last_became_peered":"2024-09-12T01:59:55.496005+0000","last_unstale":"2024-09-12T02:01:38.309816+0000","last_undegraded":"2024-09-12T02:01:38.309816+0000","last_fullsized":"2024-09-12T02:01:38.309816+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:53:39.248715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.310438+0000","last_change":"2024-09-12T01:59:55.497758+0000","last_active":"2024-09-12T02:01:38.310438+0000","last_peered":"2024-09-12T02:01:38.310438+0000","last_clean":"2024-09-12T02:01:38.310438+0000","last_became_active":"2024-09-12T01:59:55.497282+0000","last_became_peered":"2024-09-12T01:59:55.497282+0000","last_unstale":"2024-09-12T02:01:38.310438+0000","last_undegraded":"2024-09-12T02:01:38.310438+0000","last_fullsized":"2024-09-12T02:01:38.310438+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:08:25.509372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.301389+0000","last_change":"2024-09-12T01:59:55.497919+0000","last_active":"2024-09-12T02:01:38.301389+0000","last_peered":"2024-09-12T02:01:38.301389+0000","last_clean":"2024-09-12T02:01:38.301389+0000","last_became_active":"2024-09-12T01:59:55.497609+0000","last_became_peered":"2024-09-12T01:59:55.497609+0000","last_unstale":"2024-09-12T02:01:38.301389+0000","last_undegraded":"2024-09-12T02:01:38.301389+0000","last_fullsized":"2024-09-12T02:01:38.301389+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:10:22.281180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"82'7","reported_seq":134,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:44.695380+0000","last_change":"2024-09-12T01:59:53.473974+0000","last_active":"2024-09-12T02:01:44.695380+0000","last_peered":"2024-09-12T02:01:44.695380+0000","last_clean":"2024-09-12T02:01:44.695380+0000","last_became_active":"2024-09-12T01:59:53.473546+0000","last_became_peered":"2024-09-12T01:59:53.473546+0000","last_unstale":"2024-09-12T02:01:44.695380+0000","last_undegraded":"2024-09-12T02:01:44.695380+0000","last_fullsized":"2024-09-12T02:01:44.695380+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:28:03.532249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1156,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":105,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.328613+0000","last_change":"2024-09-12T01:59:55.493197+0000","last_active":"2024-09-12T02:01:38.328613+0000","last_peered":"2024-09-12T02:01:38.328613+0000","last_clean":"2024-09-12T02:01:38.328613+0000","last_became_active":"2024-09-12T01:59:55.492845+0000","last_became_peered":"2024-09-12T01:59:55.492845+0000","last_unstale":"2024-09-12T02:01:38.328613+0000","last_undegraded":"2024-09-12T02:01:38.328613+0000","last_fullsized":"2024-09-12T02:01:38.328613+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:42:50.790896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"80'3","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.622865+0000","last_change":"2024-09-12T01:59:53.471214+0000","last_active":"2024-09-12T02:00:09.622865+0000","last_peered":"2024-09-12T02:00:09.622865+0000","last_clean":"2024-09-12T02:00:09.622865+0000","last_became_active":"2024-09-12T01:59:53.470858+0000","last_became_peered":"2024-09-12T01:59:53.470858+0000","last_unstale":"2024-09-12T02:00:09.622865+0000","last_undegraded":"2024-09-12T02:00:09.622865+0000","last_fullsized":"2024-09-12T02:00:09.622865+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:34:48.730392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"80'2","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.562106+0000","last_change":"2024-09-12T01:59:53.464568+0000","last_active":"2024-09-12T02:00:09.562106+0000","last_peered":"2024-09-12T02:00:09.562106+0000","last_clean":"2024-09-12T02:00:09.562106+0000","last_became_active":"2024-09-12T01:59:53.464168+0000","last_became_peered":"2024-09-12T01:59:53.464168+0000","last_unstale":"2024-09-12T02:00:09.562106+0000","last_undegraded":"2024-09-12T02:00:09.562106+0000","last_fullsized":"2024-09-12T02:00:09.562106+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:41:48.135522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.318601+0000","last_change":"2024-09-12T01:59:55.487177+0000","last_active":"2024-09-12T02:01:38.318601+0000","last_peered":"2024-09-12T02:01:38.318601+0000","last_clean":"2024-09-12T02:01:38.318601+0000","last_became_active":"2024-09-12T01:59:55.486886+0000","last_became_peered":"2024-09-12T01:59:55.486886+0000","last_unstale":"2024-09-12T02:01:38.318601+0000","last_undegraded":"2024-09-12T02:01:38.318601+0000","last_fullsized":"2024-09-12T02:01:38.318601+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:32:40.150071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"80'1","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.567129+0000","last_change":"2024-09-12T01:59:53.474114+0000","last_active":"2024-09-12T02:00:09.567129+0000","last_peered":"2024-09-12T02:00:09.567129+0000","last_clean":"2024-09-12T02:00:09.567129+0000","last_became_active":"2024-09-12T01:59:53.473755+0000","last_became_peered":"2024-09-12T01:59:53.473755+0000","last_unstale":"2024-09-12T02:00:09.567129+0000","last_undegraded":"2024-09-12T02:00:09.567129+0000","last_fullsized":"2024-09-12T02:00:09.567129+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:07:07.021137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"82'11","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.323623+0000","last_change":"2024-09-12T01:59:55.489868+0000","last_active":"2024-09-12T02:01:38.323623+0000","last_peered":"2024-09-12T02:01:38.323623+0000","last_clean":"2024-09-12T02:01:38.323623+0000","last_became_active":"2024-09-12T01:59:55.489537+0000","last_became_peered":"2024-09-12T01:59:55.489537+0000","last_unstale":"2024-09-12T02:01:38.323623+0000","last_undegraded":"2024-09-12T02:01:38.323623+0000","last_fullsized":"2024-09-12T02:01:38.323623+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:18:49.417689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"82'4","reported_seq":130,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:47.699204+0000","last_change":"2024-09-12T01:59:53.471402+0000","last_active":"2024-09-12T02:01:47.699204+0000","last_peered":"2024-09-12T02:01:47.699204+0000","last_clean":"2024-09-12T02:01:47.699204+0000","last_became_active":"2024-09-12T01:59:53.470940+0000","last_became_peered":"2024-09-12T01:59:53.470940+0000","last_unstale":"2024-09-12T02:01:47.699204+0000","last_undegraded":"2024-09-12T02:01:47.699204+0000","last_fullsized":"2024-09-12T02:01:47.699204+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:41:16.853592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":104,"num_read_kb":104,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"82'10","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.330030+0000","last_change":"2024-09-12T01:59:55.499047+0000","last_active":"2024-09-12T02:01:38.330030+0000","last_peered":"2024-09-12T02:01:38.330030+0000","last_clean":"2024-09-12T02:01:38.330030+0000","last_became_active":"2024-09-12T01:59:55.498071+0000","last_became_peered":"2024-09-12T01:59:55.498071+0000","last_unstale":"2024-09-12T02:01:38.330030+0000","last_undegraded":"2024-09-12T02:01:38.330030+0000","last_fullsized":"2024-09-12T02:01:38.330030+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:36:40.106041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"80'2","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.622966+0000","last_change":"2024-09-12T01:59:53.473508+0000","last_active":"2024-09-12T02:00:09.622966+0000","last_peered":"2024-09-12T02:00:09.622966+0000","last_clean":"2024-09-12T02:00:09.622966+0000","last_became_active":"2024-09-12T01:59:53.473063+0000","last_became_peered":"2024-09-12T01:59:53.473063+0000","last_unstale":"2024-09-12T02:00:09.622966+0000","last_undegraded":"2024-09-12T02:00:09.622966+0000","last_fullsized":"2024-09-12T02:00:09.622966+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:55:32.161769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.333032+0000","last_change":"2024-09-12T01:59:55.491721+0000","last_active":"2024-09-12T02:01:38.333032+0000","last_peered":"2024-09-12T02:01:38.333032+0000","last_clean":"2024-09-12T02:01:38.333032+0000","last_became_active":"2024-09-12T01:59:55.491364+0000","last_became_peered":"2024-09-12T01:59:55.491364+0000","last_unstale":"2024-09-12T02:01:38.333032+0000","last_undegraded":"2024-09-12T02:01:38.333032+0000","last_fullsized":"2024-09-12T02:01:38.333032+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:21:20.795420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"80'4","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.567315+0000","last_change":"2024-09-12T01:59:53.921789+0000","last_active":"2024-09-12T02:00:09.567315+0000","last_peered":"2024-09-12T02:00:09.567315+0000","last_clean":"2024-09-12T02:00:09.567315+0000","last_became_active":"2024-09-12T01:59:53.921383+0000","last_became_peered":"2024-09-12T01:59:53.921383+0000","last_unstale":"2024-09-12T02:00:09.567315+0000","last_undegraded":"2024-09-12T02:00:09.567315+0000","last_fullsized":"2024-09-12T02:00:09.567315+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:21:58.792794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"82'11","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.331576+0000","last_change":"2024-09-12T01:59:55.503008+0000","last_active":"2024-09-12T02:01:38.331576+0000","last_peered":"2024-09-12T02:01:38.331576+0000","last_clean":"2024-09-12T02:01:38.331576+0000","last_became_active":"2024-09-12T01:59:55.502538+0000","last_became_peered":"2024-09-12T01:59:55.502538+0000","last_unstale":"2024-09-12T02:01:38.331576+0000","last_undegraded":"2024-09-12T02:01:38.331576+0000","last_fullsized":"2024-09-12T02:01:38.331576+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:51:36.735985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"82'32","reported_seq":274,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:48.700435+0000","last_change":"2024-09-12T01:59:53.470688+0000","last_active":"2024-09-12T02:01:48.700435+0000","last_peered":"2024-09-12T02:01:48.700435+0000","last_clean":"2024-09-12T02:01:48.700435+0000","last_became_active":"2024-09-12T01:59:53.470231+0000","last_became_peered":"2024-09-12T01:59:53.470231+0000","last_unstale":"2024-09-12T02:01:48.700435+0000","last_undegraded":"2024-09-12T02:01:48.700435+0000","last_fullsized":"2024-09-12T02:01:48.700435+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:53:54.439893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64635,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":220,"num_read_kb":174,"num_write":33,"num_write_kb":76,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.310987+0000","last_change":"2024-09-12T01:59:55.495343+0000","last_active":"2024-09-12T02:01:38.310987+0000","last_peered":"2024-09-12T02:01:38.310987+0000","last_clean":"2024-09-12T02:01:38.310987+0000","last_became_active":"2024-09-12T01:59:55.495006+0000","last_became_peered":"2024-09-12T01:59:55.495006+0000","last_unstale":"2024-09-12T02:01:38.310987+0000","last_undegraded":"2024-09-12T02:01:38.310987+0000","last_fullsized":"2024-09-12T02:01:38.310987+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:32:20.932551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'6","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.569082+0000","last_change":"2024-09-12T01:59:53.468625+0000","last_active":"2024-09-12T02:00:09.569082+0000","last_peered":"2024-09-12T02:00:09.569082+0000","last_clean":"2024-09-12T02:00:09.569082+0000","last_became_active":"2024-09-12T01:59:53.468015+0000","last_became_peered":"2024-09-12T01:59:53.468015+0000","last_unstale":"2024-09-12T02:00:09.569082+0000","last_undegraded":"2024-09-12T02:00:09.569082+0000","last_fullsized":"2024-09-12T02:00:09.569082+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:49:34.842239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.319752+0000","last_change":"2024-09-12T01:59:55.487962+0000","last_active":"2024-09-12T02:01:38.319752+0000","last_peered":"2024-09-12T02:01:38.319752+0000","last_clean":"2024-09-12T02:01:38.319752+0000","last_became_active":"2024-09-12T01:59:55.487619+0000","last_became_peered":"2024-09-12T01:59:55.487619+0000","last_unstale":"2024-09-12T02:01:38.319752+0000","last_undegraded":"2024-09-12T02:01:38.319752+0000","last_fullsized":"2024-09-12T02:01:38.319752+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:32:16.882373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"80'3","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.567124+0000","last_change":"2024-09-12T01:59:53.474418+0000","last_active":"2024-09-12T02:00:09.567124+0000","last_peered":"2024-09-12T02:00:09.567124+0000","last_clean":"2024-09-12T02:00:09.567124+0000","last_became_active":"2024-09-12T01:59:53.474059+0000","last_became_peered":"2024-09-12T01:59:53.474059+0000","last_unstale":"2024-09-12T02:00:09.567124+0000","last_undegraded":"2024-09-12T02:00:09.567124+0000","last_fullsized":"2024-09-12T02:00:09.567124+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:10:54.963435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"82'15","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.333121+0000","last_change":"2024-09-12T01:59:55.493394+0000","last_active":"2024-09-12T02:01:38.333121+0000","last_peered":"2024-09-12T02:01:38.333121+0000","last_clean":"2024-09-12T02:01:38.333121+0000","last_became_active":"2024-09-12T01:59:55.491669+0000","last_became_peered":"2024-09-12T01:59:55.491669+0000","last_unstale":"2024-09-12T02:01:38.333121+0000","last_undegraded":"2024-09-12T02:01:38.333121+0000","last_fullsized":"2024-09-12T02:01:38.333121+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:14:40.871154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"80'3","reported_seq":131,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:47.699557+0000","last_change":"2024-09-12T01:59:53.471623+0000","last_active":"2024-09-12T02:01:47.699557+0000","last_peered":"2024-09-12T02:01:47.699557+0000","last_clean":"2024-09-12T02:01:47.699557+0000","last_became_active":"2024-09-12T01:59:53.471294+0000","last_became_peered":"2024-09-12T01:59:53.471294+0000","last_unstale":"2024-09-12T02:01:47.699557+0000","last_undegraded":"2024-09-12T02:01:47.699557+0000","last_fullsized":"2024-09-12T02:01:47.699557+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:51:54.130883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":107,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.322144+0000","last_change":"2024-09-12T01:59:55.493082+0000","last_active":"2024-09-12T02:01:38.322144+0000","last_peered":"2024-09-12T02:01:38.322144+0000","last_clean":"2024-09-12T02:01:38.322144+0000","last_became_active":"2024-09-12T01:59:55.492778+0000","last_became_peered":"2024-09-12T01:59:55.492778+0000","last_unstale":"2024-09-12T02:01:38.322144+0000","last_undegraded":"2024-09-12T02:01:38.322144+0000","last_fullsized":"2024-09-12T02:01:38.322144+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:59:56.466377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'15","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.588050+0000","last_change":"2024-09-12T02:00:09.588050+0000","last_active":"2024-09-12T02:00:09.588050+0000","last_peered":"2024-09-12T02:00:09.588050+0000","last_clean":"2024-09-12T02:00:09.588050+0000","last_became_active":"2024-09-12T02:00:09.575478+0000","last_became_peered":"2024-09-12T02:00:09.575478+0000","last_unstale":"2024-09-12T02:00:09.588050+0000","last_undegraded":"2024-09-12T02:00:09.588050+0000","last_fullsized":"2024-09-12T02:00:09.588050+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:38:45.484167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.313876+0000","last_change":"2024-09-12T01:59:55.497601+0000","last_active":"2024-09-12T02:01:38.313876+0000","last_peered":"2024-09-12T02:01:38.313876+0000","last_clean":"2024-09-12T02:01:38.313876+0000","last_became_active":"2024-09-12T01:59:55.497202+0000","last_became_peered":"2024-09-12T01:59:55.497202+0000","last_unstale":"2024-09-12T02:01:38.313876+0000","last_undegraded":"2024-09-12T02:01:38.313876+0000","last_fullsized":"2024-09-12T02:01:38.313876+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:22:22.850736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"80'2","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.575359+0000","last_change":"2024-09-12T01:59:53.467575+0000","last_active":"2024-09-12T02:00:09.575359+0000","last_peered":"2024-09-12T02:00:09.575359+0000","last_clean":"2024-09-12T02:00:09.575359+0000","last_became_active":"2024-09-12T01:59:53.467146+0000","last_became_peered":"2024-09-12T01:59:53.467146+0000","last_unstale":"2024-09-12T02:00:09.575359+0000","last_undegraded":"2024-09-12T02:00:09.575359+0000","last_fullsized":"2024-09-12T02:00:09.575359+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:07:25.962496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"82'5","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.314382+0000","last_change":"2024-09-12T01:59:55.498297+0000","last_active":"2024-09-12T02:01:38.314382+0000","last_peered":"2024-09-12T02:01:38.314382+0000","last_clean":"2024-09-12T02:01:38.314382+0000","last_became_active":"2024-09-12T01:59:55.497367+0000","last_became_peered":"2024-09-12T01:59:55.497367+0000","last_unstale":"2024-09-12T02:01:38.314382+0000","last_undegraded":"2024-09-12T02:01:38.314382+0000","last_fullsized":"2024-09-12T02:01:38.314382+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:58:32.717710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"80'2","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.562292+0000","last_change":"2024-09-12T01:59:53.466664+0000","last_active":"2024-09-12T02:00:09.562292+0000","last_peered":"2024-09-12T02:00:09.562292+0000","last_clean":"2024-09-12T02:00:09.562292+0000","last_became_active":"2024-09-12T01:59:53.466318+0000","last_became_peered":"2024-09-12T01:59:53.466318+0000","last_unstale":"2024-09-12T02:00:09.562292+0000","last_undegraded":"2024-09-12T02:00:09.562292+0000","last_fullsized":"2024-09-12T02:00:09.562292+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:12:13.541420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.313289+0000","last_change":"2024-09-12T01:59:55.490420+0000","last_active":"2024-09-12T02:01:38.313289+0000","last_peered":"2024-09-12T02:01:38.313289+0000","last_clean":"2024-09-12T02:01:38.313289+0000","last_became_active":"2024-09-12T01:59:55.489757+0000","last_became_peered":"2024-09-12T01:59:55.489757+0000","last_unstale":"2024-09-12T02:01:38.313289+0000","last_undegraded":"2024-09-12T02:01:38.313289+0000","last_fullsized":"2024-09-12T02:01:38.313289+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:34:43.117142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'1","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.567185+0000","last_change":"2024-09-12T01:59:53.466399+0000","last_active":"2024-09-12T02:00:09.567185+0000","last_peered":"2024-09-12T02:00:09.567185+0000","last_clean":"2024-09-12T02:00:09.567185+0000","last_became_active":"2024-09-12T01:59:53.465982+0000","last_became_peered":"2024-09-12T01:59:53.465982+0000","last_unstale":"2024-09-12T02:00:09.567185+0000","last_undegraded":"2024-09-12T02:00:09.567185+0000","last_fullsized":"2024-09-12T02:00:09.567185+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:18:02.015040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.326034+0000","last_change":"2024-09-12T01:59:55.491527+0000","last_active":"2024-09-12T02:01:38.326034+0000","last_peered":"2024-09-12T02:01:38.326034+0000","last_clean":"2024-09-12T02:01:38.326034+0000","last_became_active":"2024-09-12T01:59:55.491165+0000","last_became_peered":"2024-09-12T01:59:55.491165+0000","last_unstale":"2024-09-12T02:01:38.326034+0000","last_undegraded":"2024-09-12T02:01:38.326034+0000","last_fullsized":"2024-09-12T02:01:38.326034+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:57:05.506384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"80'3","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.569187+0000","last_change":"2024-09-12T01:59:54.075752+0000","last_active":"2024-09-12T02:00:09.569187+0000","last_peered":"2024-09-12T02:00:09.569187+0000","last_clean":"2024-09-12T02:00:09.569187+0000","last_became_active":"2024-09-12T01:59:54.075387+0000","last_became_peered":"2024-09-12T01:59:54.075387+0000","last_unstale":"2024-09-12T02:00:09.569187+0000","last_undegraded":"2024-09-12T02:00:09.569187+0000","last_fullsized":"2024-09-12T02:00:09.569187+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:21:15.980591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.330203+0000","last_change":"2024-09-12T01:59:55.500935+0000","last_active":"2024-09-12T02:01:38.330203+0000","last_peered":"2024-09-12T02:01:38.330203+0000","last_clean":"2024-09-12T02:01:38.330203+0000","last_became_active":"2024-09-12T01:59:55.500206+0000","last_became_peered":"2024-09-12T01:59:55.500206+0000","last_unstale":"2024-09-12T02:01:38.330203+0000","last_undegraded":"2024-09-12T02:01:38.330203+0000","last_fullsized":"2024-09-12T02:01:38.330203+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:19:59.937701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":334,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.623254+0000","last_change":"2024-09-12T01:58:47.553853+0000","last_active":"2024-09-12T02:00:09.623254+0000","last_peered":"2024-09-12T02:00:09.623254+0000","last_clean":"2024-09-12T02:00:09.623254+0000","last_became_active":"2024-09-12T01:58:47.553357+0000","last_became_peered":"2024-09-12T01:58:47.553357+0000","last_unstale":"2024-09-12T02:00:09.623254+0000","last_undegraded":"2024-09-12T02:00:09.623254+0000","last_fullsized":"2024-09-12T02:00:09.623254+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:55:26.917733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:55:26.917733+0000","last_clean_scrub_stamp":"2024-09-12T01:55:26.917733+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:58:26.081766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"80'3","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.563615+0000","last_change":"2024-09-12T01:59:53.467907+0000","last_active":"2024-09-12T02:00:09.563615+0000","last_peered":"2024-09-12T02:00:09.563615+0000","last_clean":"2024-09-12T02:00:09.563615+0000","last_became_active":"2024-09-12T01:59:53.467164+0000","last_became_peered":"2024-09-12T01:59:53.467164+0000","last_unstale":"2024-09-12T02:00:09.563615+0000","last_undegraded":"2024-09-12T02:00:09.563615+0000","last_fullsized":"2024-09-12T02:00:09.563615+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:09:35.242708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.327715+0000","last_change":"2024-09-12T01:59:55.495109+0000","last_active":"2024-09-12T02:01:38.327715+0000","last_peered":"2024-09-12T02:01:38.327715+0000","last_clean":"2024-09-12T02:01:38.327715+0000","last_became_active":"2024-09-12T01:59:55.494791+0000","last_became_peered":"2024-09-12T01:59:55.494791+0000","last_unstale":"2024-09-12T02:01:38.327715+0000","last_undegraded":"2024-09-12T02:01:38.327715+0000","last_fullsized":"2024-09-12T02:01:38.327715+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:22:43.882050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"80'3","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.623115+0000","last_change":"2024-09-12T01:59:53.473286+0000","last_active":"2024-09-12T02:00:09.623115+0000","last_peered":"2024-09-12T02:00:09.623115+0000","last_clean":"2024-09-12T02:00:09.623115+0000","last_became_active":"2024-09-12T01:59:53.472976+0000","last_became_peered":"2024-09-12T01:59:53.472976+0000","last_unstale":"2024-09-12T02:00:09.623115+0000","last_undegraded":"2024-09-12T02:00:09.623115+0000","last_fullsized":"2024-09-12T02:00:09.623115+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:32:39.604173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"82'5","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.313749+0000","last_change":"2024-09-12T02:00:09.571159+0000","last_active":"2024-09-12T02:01:38.313749+0000","last_peered":"2024-09-12T02:01:38.313749+0000","last_clean":"2024-09-12T02:01:38.313749+0000","last_became_active":"2024-09-12T02:00:09.570672+0000","last_became_peered":"2024-09-12T02:00:09.570672+0000","last_unstale":"2024-09-12T02:01:38.313749+0000","last_undegraded":"2024-09-12T02:01:38.313749+0000","last_fullsized":"2024-09-12T02:01:38.313749+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:04:10.808942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"80'4","reported_seq":131,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:48.700207+0000","last_change":"2024-09-12T01:59:53.921856+0000","last_active":"2024-09-12T02:01:48.700207+0000","last_peered":"2024-09-12T02:01:48.700207+0000","last_clean":"2024-09-12T02:01:48.700207+0000","last_became_active":"2024-09-12T01:59:53.920672+0000","last_became_peered":"2024-09-12T01:59:53.920672+0000","last_unstale":"2024-09-12T02:01:48.700207+0000","last_undegraded":"2024-09-12T02:01:48.700207+0000","last_fullsized":"2024-09-12T02:01:48.700207+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:42:33.876493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":105,"num_read_kb":105,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"82'8","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.309498+0000","last_change":"2024-09-12T02:00:09.571185+0000","last_active":"2024-09-12T02:01:38.309498+0000","last_peered":"2024-09-12T02:01:38.309498+0000","last_clean":"2024-09-12T02:01:38.309498+0000","last_became_active":"2024-09-12T02:00:09.570769+0000","last_became_peered":"2024-09-12T02:00:09.570769+0000","last_unstale":"2024-09-12T02:01:38.309498+0000","last_undegraded":"2024-09-12T02:01:38.309498+0000","last_fullsized":"2024-09-12T02:01:38.309498+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:34:17.230435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.2","version":"80'1","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.622691+0000","last_change":"2024-09-12T01:59:53.472072+0000","last_active":"2024-09-12T02:00:09.622691+0000","last_peered":"2024-09-12T02:00:09.622691+0000","last_clean":"2024-09-12T02:00:09.622691+0000","last_became_active":"2024-09-12T01:59:53.471088+0000","last_became_peered":"2024-09-12T01:59:53.471088+0000","last_unstale":"2024-09-12T02:00:09.622691+0000","last_undegraded":"2024-09-12T02:00:09.622691+0000","last_fullsized":"2024-09-12T02:00:09.622691+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:02:13.879131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.330778+0000","last_change":"2024-09-12T01:59:55.502843+0000","last_active":"2024-09-12T02:01:38.330778+0000","last_peered":"2024-09-12T02:01:38.330778+0000","last_clean":"2024-09-12T02:01:38.330778+0000","last_became_active":"2024-09-12T01:59:55.499296+0000","last_became_peered":"2024-09-12T01:59:55.499296+0000","last_unstale":"2024-09-12T02:01:38.330778+0000","last_undegraded":"2024-09-12T02:01:38.330778+0000","last_fullsized":"2024-09-12T02:01:38.330778+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:04:21.787370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.575061+0000","last_change":"2024-09-12T02:00:09.575061+0000","last_active":"2024-09-12T02:00:09.575061+0000","last_peered":"2024-09-12T02:00:09.575061+0000","last_clean":"2024-09-12T02:00:09.575061+0000","last_became_active":"2024-09-12T02:00:09.574653+0000","last_became_peered":"2024-09-12T02:00:09.574653+0000","last_unstale":"2024-09-12T02:00:09.575061+0000","last_undegraded":"2024-09-12T02:00:09.575061+0000","last_fullsized":"2024-09-12T02:00:09.575061+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:32:27.931211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"82'5","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.320660+0000","last_change":"2024-09-12T01:59:55.494162+0000","last_active":"2024-09-12T02:01:38.320660+0000","last_peered":"2024-09-12T02:01:38.320660+0000","last_clean":"2024-09-12T02:01:38.320660+0000","last_became_active":"2024-09-12T01:59:55.493802+0000","last_became_peered":"2024-09-12T01:59:55.493802+0000","last_unstale":"2024-09-12T02:01:38.320660+0000","last_undegraded":"2024-09-12T02:01:38.320660+0000","last_fullsized":"2024-09-12T02:01:38.320660+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:22:16.302683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"80'1","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.575278+0000","last_change":"2024-09-12T01:59:53.921669+0000","last_active":"2024-09-12T02:00:09.575278+0000","last_peered":"2024-09-12T02:00:09.575278+0000","last_clean":"2024-09-12T02:00:09.575278+0000","last_became_active":"2024-09-12T01:59:53.921328+0000","last_became_peered":"2024-09-12T01:59:53.921328+0000","last_unstale":"2024-09-12T02:00:09.575278+0000","last_undegraded":"2024-09-12T02:00:09.575278+0000","last_fullsized":"2024-09-12T02:00:09.575278+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:46:33.313726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.311067+0000","last_change":"2024-09-12T01:59:55.488986+0000","last_active":"2024-09-12T02:01:38.311067+0000","last_peered":"2024-09-12T02:01:38.311067+0000","last_clean":"2024-09-12T02:01:38.311067+0000","last_became_active":"2024-09-12T01:59:55.488632+0000","last_became_peered":"2024-09-12T01:59:55.488632+0000","last_unstale":"2024-09-12T02:01:38.311067+0000","last_undegraded":"2024-09-12T02:01:38.311067+0000","last_fullsized":"2024-09-12T02:01:38.311067+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:20:57.596976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"82'11","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:21.501527+0000","last_change":"2024-09-12T01:59:53.467648+0000","last_active":"2024-09-12T02:00:21.501527+0000","last_peered":"2024-09-12T02:00:21.501527+0000","last_clean":"2024-09-12T02:00:21.501527+0000","last_became_active":"2024-09-12T01:59:53.467255+0000","last_became_peered":"2024-09-12T01:59:53.467255+0000","last_unstale":"2024-09-12T02:00:21.501527+0000","last_undegraded":"2024-09-12T02:00:21.501527+0000","last_fullsized":"2024-09-12T02:00:21.501527+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:37:09.897388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"82'5","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.328722+0000","last_change":"2024-09-12T01:59:55.487603+0000","last_active":"2024-09-12T02:01:38.328722+0000","last_peered":"2024-09-12T02:01:38.328722+0000","last_clean":"2024-09-12T02:01:38.328722+0000","last_became_active":"2024-09-12T01:59:55.487334+0000","last_became_peered":"2024-09-12T01:59:55.487334+0000","last_unstale":"2024-09-12T02:01:38.328722+0000","last_undegraded":"2024-09-12T02:01:38.328722+0000","last_fullsized":"2024-09-12T02:01:38.328722+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:23:16.669515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'4","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.563870+0000","last_change":"2024-09-12T01:59:54.076156+0000","last_active":"2024-09-12T02:00:09.563870+0000","last_peered":"2024-09-12T02:00:09.563870+0000","last_clean":"2024-09-12T02:00:09.563870+0000","last_became_active":"2024-09-12T01:59:54.075794+0000","last_became_peered":"2024-09-12T01:59:54.075794+0000","last_unstale":"2024-09-12T02:00:09.563870+0000","last_undegraded":"2024-09-12T02:00:09.563870+0000","last_fullsized":"2024-09-12T02:00:09.563870+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:06:17.431696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.333603+0000","last_change":"2024-09-12T01:59:55.494604+0000","last_active":"2024-09-12T02:01:38.333603+0000","last_peered":"2024-09-12T02:01:38.333603+0000","last_clean":"2024-09-12T02:01:38.333603+0000","last_became_active":"2024-09-12T01:59:55.494300+0000","last_became_peered":"2024-09-12T01:59:55.494300+0000","last_unstale":"2024-09-12T02:01:38.333603+0000","last_undegraded":"2024-09-12T02:01:38.333603+0000","last_fullsized":"2024-09-12T02:01:38.333603+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:18:56.359299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.622205+0000","last_change":"2024-09-12T01:59:53.921832+0000","last_active":"2024-09-12T02:00:09.622205+0000","last_peered":"2024-09-12T02:00:09.622205+0000","last_clean":"2024-09-12T02:00:09.622205+0000","last_became_active":"2024-09-12T01:59:53.921379+0000","last_became_peered":"2024-09-12T01:59:53.921379+0000","last_unstale":"2024-09-12T02:00:09.622205+0000","last_undegraded":"2024-09-12T02:00:09.622205+0000","last_fullsized":"2024-09-12T02:00:09.622205+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:26:02.277149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"82'2","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.281554+0000","last_change":"2024-09-12T01:59:55.493198+0000","last_active":"2024-09-12T02:01:38.281554+0000","last_peered":"2024-09-12T02:01:38.281554+0000","last_clean":"2024-09-12T02:01:38.281554+0000","last_became_active":"2024-09-12T01:59:55.492834+0000","last_became_peered":"2024-09-12T01:59:55.492834+0000","last_unstale":"2024-09-12T02:01:38.281554+0000","last_undegraded":"2024-09-12T02:01:38.281554+0000","last_fullsized":"2024-09-12T02:01:38.281554+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:58:43.872921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"80'3","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.621258+0000","last_change":"2024-09-12T01:59:53.466528+0000","last_active":"2024-09-12T02:00:09.621258+0000","last_peered":"2024-09-12T02:00:09.621258+0000","last_clean":"2024-09-12T02:00:09.621258+0000","last_became_active":"2024-09-12T01:59:53.466182+0000","last_became_peered":"2024-09-12T01:59:53.466182+0000","last_unstale":"2024-09-12T02:00:09.621258+0000","last_undegraded":"2024-09-12T02:00:09.621258+0000","last_fullsized":"2024-09-12T02:00:09.621258+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:00:26.083539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.316786+0000","last_change":"2024-09-12T01:59:55.494511+0000","last_active":"2024-09-12T02:01:38.316786+0000","last_peered":"2024-09-12T02:01:38.316786+0000","last_clean":"2024-09-12T02:01:38.316786+0000","last_became_active":"2024-09-12T01:59:55.493607+0000","last_became_peered":"2024-09-12T01:59:55.493607+0000","last_unstale":"2024-09-12T02:01:38.316786+0000","last_undegraded":"2024-09-12T02:01:38.316786+0000","last_fullsized":"2024-09-12T02:01:38.316786+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:55:27.434447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"80'1","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.569211+0000","last_change":"2024-09-12T01:59:53.467385+0000","last_active":"2024-09-12T02:00:09.569211+0000","last_peered":"2024-09-12T02:00:09.569211+0000","last_clean":"2024-09-12T02:00:09.569211+0000","last_became_active":"2024-09-12T01:59:53.466997+0000","last_became_peered":"2024-09-12T01:59:53.466997+0000","last_unstale":"2024-09-12T02:00:09.569211+0000","last_undegraded":"2024-09-12T02:00:09.569211+0000","last_fullsized":"2024-09-12T02:00:09.569211+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:54:47.154036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.316513+0000","last_change":"2024-09-12T01:59:55.490757+0000","last_active":"2024-09-12T02:01:38.316513+0000","last_peered":"2024-09-12T02:01:38.316513+0000","last_clean":"2024-09-12T02:01:38.316513+0000","last_became_active":"2024-09-12T01:59:55.490451+0000","last_became_peered":"2024-09-12T01:59:55.490451+0000","last_unstale":"2024-09-12T02:01:38.316513+0000","last_undegraded":"2024-09-12T02:01:38.316513+0000","last_fullsized":"2024-09-12T02:01:38.316513+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:21:26.139049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.574833+0000","last_change":"2024-09-12T01:59:53.472898+0000","last_active":"2024-09-12T02:00:09.574833+0000","last_peered":"2024-09-12T02:00:09.574833+0000","last_clean":"2024-09-12T02:00:09.574833+0000","last_became_active":"2024-09-12T01:59:53.472561+0000","last_became_peered":"2024-09-12T01:59:53.472561+0000","last_unstale":"2024-09-12T02:00:09.574833+0000","last_undegraded":"2024-09-12T02:00:09.574833+0000","last_fullsized":"2024-09-12T02:00:09.574833+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:55:59.933804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"82'5","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.328406+0000","last_change":"2024-09-12T01:59:55.497731+0000","last_active":"2024-09-12T02:01:38.328406+0000","last_peered":"2024-09-12T02:01:38.328406+0000","last_clean":"2024-09-12T02:01:38.328406+0000","last_became_active":"2024-09-12T01:59:55.497356+0000","last_became_peered":"2024-09-12T01:59:55.497356+0000","last_unstale":"2024-09-12T02:01:38.328406+0000","last_undegraded":"2024-09-12T02:01:38.328406+0000","last_fullsized":"2024-09-12T02:01:38.328406+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:57:18.795865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"82'15","reported_seq":38,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.318747+0000","last_change":"2024-09-12T02:00:09.624453+0000","last_active":"2024-09-12T02:01:38.318747+0000","last_peered":"2024-09-12T02:01:38.318747+0000","last_clean":"2024-09-12T02:01:38.318747+0000","last_became_active":"2024-09-12T02:00:09.624049+0000","last_became_peered":"2024-09-12T02:00:09.624049+0000","last_unstale":"2024-09-12T02:01:38.318747+0000","last_undegraded":"2024-09-12T02:01:38.318747+0000","last_fullsized":"2024-09-12T02:01:38.318747+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:17:40.417275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"80'2","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.575060+0000","last_change":"2024-09-12T01:59:53.470194+0000","last_active":"2024-09-12T02:00:09.575060+0000","last_peered":"2024-09-12T02:00:09.575060+0000","last_clean":"2024-09-12T02:00:09.575060+0000","last_became_active":"2024-09-12T01:59:53.469772+0000","last_became_peered":"2024-09-12T01:59:53.469772+0000","last_unstale":"2024-09-12T02:00:09.575060+0000","last_undegraded":"2024-09-12T02:00:09.575060+0000","last_fullsized":"2024-09-12T02:00:09.575060+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:52:57.709596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.329782+0000","last_change":"2024-09-12T01:59:55.497693+0000","last_active":"2024-09-12T02:01:38.329782+0000","last_peered":"2024-09-12T02:01:38.329782+0000","last_clean":"2024-09-12T02:01:38.329782+0000","last_became_active":"2024-09-12T01:59:55.497380+0000","last_became_peered":"2024-09-12T01:59:55.497380+0000","last_unstale":"2024-09-12T02:01:38.329782+0000","last_undegraded":"2024-09-12T02:01:38.329782+0000","last_fullsized":"2024-09-12T02:01:38.329782+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:42:27.943530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"80'4","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.575274+0000","last_change":"2024-09-12T01:59:53.473044+0000","last_active":"2024-09-12T02:00:09.575274+0000","last_peered":"2024-09-12T02:00:09.575274+0000","last_clean":"2024-09-12T02:00:09.575274+0000","last_became_active":"2024-09-12T01:59:53.472569+0000","last_became_peered":"2024-09-12T01:59:53.472569+0000","last_unstale":"2024-09-12T02:00:09.575274+0000","last_undegraded":"2024-09-12T02:00:09.575274+0000","last_fullsized":"2024-09-12T02:00:09.575274+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:41:29.716836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.328172+0000","last_change":"2024-09-12T01:59:55.497377+0000","last_active":"2024-09-12T02:01:38.328172+0000","last_peered":"2024-09-12T02:01:38.328172+0000","last_clean":"2024-09-12T02:01:38.328172+0000","last_became_active":"2024-09-12T01:59:55.497078+0000","last_became_peered":"2024-09-12T01:59:55.497078+0000","last_unstale":"2024-09-12T02:01:38.328172+0000","last_undegraded":"2024-09-12T02:01:38.328172+0000","last_fullsized":"2024-09-12T02:01:38.328172+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:59:57.362350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.621555+0000","last_change":"2024-09-12T01:59:53.920868+0000","last_active":"2024-09-12T02:00:09.621555+0000","last_peered":"2024-09-12T02:00:09.621555+0000","last_clean":"2024-09-12T02:00:09.621555+0000","last_became_active":"2024-09-12T01:59:53.920481+0000","last_became_peered":"2024-09-12T01:59:53.920481+0000","last_unstale":"2024-09-12T02:00:09.621555+0000","last_undegraded":"2024-09-12T02:00:09.621555+0000","last_fullsized":"2024-09-12T02:00:09.621555+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:49:27.085388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"82'5","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:01:38.315986+0000","last_change":"2024-09-12T01:59:55.497703+0000","last_active":"2024-09-12T02:01:38.315986+0000","last_peered":"2024-09-12T02:01:38.315986+0000","last_clean":"2024-09-12T02:01:38.315986+0000","last_became_active":"2024-09-12T01:59:55.497397+0000","last_became_peered":"2024-09-12T01:59:55.497397+0000","last_unstale":"2024-09-12T02:01:38.315986+0000","last_undegraded":"2024-09-12T02:01:38.315986+0000","last_fullsized":"2024-09-12T02:01:38.315986+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:54.464059+0000","last_clean_scrub_stamp":"2024-09-12T01:59:54.464059+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:54:02.803203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"80'9","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T02:00:09.565198+0000","last_change":"2024-09-12T01:59:53.469978+0000","last_active":"2024-09-12T02:00:09.565198+0000","last_peered":"2024-09-12T02:00:09.565198+0000","last_clean":"2024-09-12T02:00:09.565198+0000","last_became_active":"2024-09-12T01:59:53.469648+0000","last_became_peered":"2024-09-12T01:59:53.469648+0000","last_unstale":"2024-09-12T02:00:09.565198+0000","last_undegraded":"2024-09-12T02:00:09.565198+0000","last_fullsized":"2024-09-12T02:00:09.565198+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:59:52.452296+0000","last_clean_scrub_stamp":"2024-09-12T01:59:52.452296+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:08:37.720452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":68779,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":654,"num_read_kb":403,"num_write":170,"num_write_kb":119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":206337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":144,"ondisk_log_size":144,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3563520,"data_stored":3542208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":74,"seq":317827579940,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32060,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713092,"statfs":{"total":95995035648,"available":95962206208,"internally_reserved":0,"allocated":897024,"data_stored":435916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":67,"seq":287762808873,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32736,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712416,"statfs":{"total":95995035648,"available":95961513984,"internally_reserved":0,"allocated":1589248,"data_stored":1091920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":62,"seq":266287972398,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":905216,"data_stored":436040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201330,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":897024,"data_stored":435848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430264,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32648,"kb_used_data":1464,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712504,"statfs":{"total":95995035648,"available":95961604096,"internally_reserved":0,"allocated":1499136,"data_stored":1028033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593789,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36216,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708936,"statfs":{"total":95995035648,"available":95957950464,"internally_reserved":0,"allocated":892928,"data_stored":435429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822721,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32660,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712492,"statfs":{"total":95995035648,"available":95961591808,"internally_reserved":0,"allocated":1511424,"data_stored":1027717,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986246,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":897024,"data_stored":435866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215178,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":909312,"data_stored":435740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":17,"seq":73014444112,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36304,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708848,"statfs":{"total":95995035648,"available":95957860352,"internally_reserved":0,"allocated":983040,"data_stored":501741,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574933,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32120,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713032,"statfs":{"total":95995035648,"available":95962144768,"internally_reserved":0,"allocated":958464,"data_stored":499857,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738459,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":913408,"data_stored":436700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":64635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":66519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":66213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T02:01:51.773 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:01:52.032 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:01:52.436 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:01:52 smithi016 ceph-mon[30947]: pgmap v333: 97 pgs: 97 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1023 B/s wr, 8 op/s 2024-09-12T02:01:52.436 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:01:52 smithi016 ceph-mon[30947]: from='client.15252 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T02:01:52.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:52.456+0000 7fa3dcf5d640 1 -- 172.21.15.16:0/2561171855 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3d8106f10 msgr2=0x7fa3d810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:52.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:52.456+0000 7fa3dcf5d640 1 --2- 172.21.15.16:0/2561171855 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3d8106f10 0x7fa3d810ea90 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fa3d0009ca0 tx=0x7fa3d002a8c0 comp rx=0 tx=0).stop 2024-09-12T02:01:52.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:52.456+0000 7fa3dcf5d640 1 -- 172.21.15.16:0/2561171855 shutdown_connections 2024-09-12T02:01:52.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:52.456+0000 7fa3dcf5d640 1 --2- 172.21.15.16:0/2561171855 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3d8106f10 0x7fa3d810ea90 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:52.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:52.456+0000 7fa3dcf5d640 1 --2- 172.21.15.16:0/2561171855 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3d8102790 0x7fa3d81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:52.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:52.456+0000 7fa3dcf5d640 1 --2- 172.21.15.16:0/2561171855 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa3d8101de0 0x7fa3d81021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:52.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:52.456+0000 7fa3dcf5d640 1 -- 172.21.15.16:0/2561171855 >> 172.21.15.16:0/2561171855 conn(0x7fa3d80fd8c0 msgr2=0x7fa3d80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:01:52.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:52.456+0000 7fa3dcf5d640 1 -- 172.21.15.16:0/2561171855 shutdown_connections 2024-09-12T02:01:52.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:52.456+0000 7fa3dcf5d640 1 -- 172.21.15.16:0/2561171855 wait complete. 2024-09-12T02:01:52.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:52.457+0000 7fa3dcf5d640 1 Processor -- start 2024-09-12T02:01:52.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:52.457+0000 7fa3dcf5d640 1 -- start start 2024-09-12T02:01:52.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:52.457+0000 7fa3dcf5d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3d8101de0 0x7fa3d80744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:52.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:52.457+0000 7fa3dcf5d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3d8102790 0x7fa3d8074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:52.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:52.457+0000 7fa3dcf5d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa3d8106f10 0x7fa3d8074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:52.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:52.457+0000 7fa3dcf5d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa3d8111f70 con 0x7fa3d8102790 2024-09-12T02:01:52.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:52.457+0000 7fa3dcf5d640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa3d8111df0 con 0x7fa3d8101de0 2024-09-12T02:01:52.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:52.457+0000 7fa3dcf5d640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa3d81120f0 con 0x7fa3d8106f10 2024-09-12T02:01:52.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:52.458+0000 7fa3d5d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3d8102790 0x7fa3d8074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:52.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:52.458+0000 7fa3d5d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3d8102790 0x7fa3d8074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58112/0 (socket says 172.21.15.16:58112) 2024-09-12T02:01:52.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:52.458+0000 7fa3d5d9b640 1 -- 172.21.15.16:0/1394325379 learned_addr learned my addr 172.21.15.16:0/1394325379 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:01:52.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:52.458+0000 7fa3d659c640 1 --2- 172.21.15.16:0/1394325379 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3d8101de0 0x7fa3d80744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:52.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:52.458+0000 7fa3d6d9d640 1 --2- 172.21.15.16:0/1394325379 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa3d8106f10 0x7fa3d8074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:52.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:52.458+0000 7fa3d5d9b640 1 -- 172.21.15.16:0/1394325379 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa3d8106f10 msgr2=0x7fa3d8074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:52.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:52.458+0000 7fa3d5d9b640 1 --2- 172.21.15.16:0/1394325379 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa3d8106f10 0x7fa3d8074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:52.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:52.458+0000 7fa3d5d9b640 1 -- 172.21.15.16:0/1394325379 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3d8101de0 msgr2=0x7fa3d80744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:52.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:52.458+0000 7fa3d5d9b640 1 --2- 172.21.15.16:0/1394325379 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3d8101de0 0x7fa3d80744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:52.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:52.458+0000 7fa3d5d9b640 1 -- 172.21.15.16:0/1394325379 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3d81a6890 con 0x7fa3d8102790 2024-09-12T02:01:52.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:52.458+0000 7fa3d5d9b640 1 --2- 172.21.15.16:0/1394325379 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3d8102790 0x7fa3d8074a30 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc00d950 tx=0x7fa3cc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:01:52.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:52.458+0000 7fa3c77fe640 1 -- 172.21.15.16:0/1394325379 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3cc017070 con 0x7fa3d8102790 2024-09-12T02:01:52.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:52.458+0000 7fa3dcf5d640 1 -- 172.21.15.16:0/1394325379 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3d81a6b80 con 0x7fa3d8102790 2024-09-12T02:01:52.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:52.458+0000 7fa3c77fe640 1 -- 172.21.15.16:0/1394325379 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa3cc009c30 con 0x7fa3d8102790 2024-09-12T02:01:52.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:52.458+0000 7fa3c77fe640 1 -- 172.21.15.16:0/1394325379 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3cc00f6b0 con 0x7fa3d8102790 2024-09-12T02:01:52.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:52.458+0000 7fa3dcf5d640 1 -- 172.21.15.16:0/1394325379 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3d81a7110 con 0x7fa3d8102790 2024-09-12T02:01:52.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:52.459+0000 7fa3dcf5d640 1 -- 172.21.15.16:0/1394325379 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3d81021c0 con 0x7fa3d8102790 2024-09-12T02:01:52.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:52.461+0000 7fa3c77fe640 1 -- 172.21.15.16:0/1394325379 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa3cc0109d0 con 0x7fa3d8102790 2024-09-12T02:01:52.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:52.461+0000 7fa3c77fe640 1 --2- 172.21.15.16:0/1394325379 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fa3c00a2b80 0x7fa3c00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:52.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:52.461+0000 7fa3d659c640 1 --2- 172.21.15.16:0/1394325379 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fa3c00a2b80 0x7fa3c00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:52.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:52.461+0000 7fa3c77fe640 1 -- 172.21.15.16:0/1394325379 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fa3cc0cebf0 con 0x7fa3d8102790 2024-09-12T02:01:52.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:52.462+0000 7fa3d659c640 1 --2- 172.21.15.16:0/1394325379 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fa3c00a2b80 0x7fa3c00a5040 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fa3c80058c0 tx=0x7fa3c800a250 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:01:52.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:52.464+0000 7fa3c77fe640 1 -- 172.21.15.16:0/1394325379 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa3cc009de0 con 0x7fa3d8102790 2024-09-12T02:01:52.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:01:52 smithi184 ceph-mon[35593]: pgmap v333: 97 pgs: 97 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1023 B/s wr, 8 op/s 2024-09-12T02:01:52.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:01:52 smithi184 ceph-mon[35593]: from='client.15252 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T02:01:52.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:01:52 smithi033 ceph-mon[35332]: pgmap v333: 97 pgs: 97 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1023 B/s wr, 8 op/s 2024-09-12T02:01:52.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:01:52 smithi033 ceph-mon[35332]: from='client.15252 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T02:01:52.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:52.678+0000 7fa3dcf5d640 1 -- 172.21.15.16:0/1394325379 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa3d802d7c0 con 0x7fa3d8102790 2024-09-12T02:01:52.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:52.678+0000 7fa3c77fe640 1 -- 172.21.15.16:0/1394325379 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11911 (secure 0 0 0) 0x7fa3cc09bb50 con 0x7fa3d8102790 2024-09-12T02:01:52.678 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T02:01:52.679 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":25,"btime":"2024-09-12T02:01:37:192021+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3279123279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3279123279},{"type":"v1","addr":"172.21.15.16:6835","nonce":3279123279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4119452382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4119452382},{"type":"v1","addr":"172.21.15.16:6837","nonce":4119452382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/3263823165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3263823165},{"type":"v1","addr":"172.21.15.16:6839","nonce":3263823165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/1510290732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":1510290732},{"type":"v1","addr":"172.21.15.16:6841","nonce":1510290732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/279674603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":279674603},{"type":"v1","addr":"172.21.15.33:6833","nonce":279674603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3555728028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3555728028},{"type":"v1","addr":"172.21.15.33:6835","nonce":3555728028}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T01:59:56.117608+0000","modified":"2024-09-12T02:01:36.294760+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24482,"mds_1":24481,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6833/1563374106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1563374106},{"type":"v1","addr":"172.21.15.184:6833","nonce":1563374106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6837/1892833868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1892833868},{"type":"v1","addr":"172.21.15.33:6837","nonce":1892833868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/1457679686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1457679686},{"type":"v1","addr":"172.21.15.184:6835","nonce":1457679686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.184:6837/1793393087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":1793393087},{"type":"v1","addr":"172.21.15.184:6837","nonce":1793393087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.184:6839/4203347941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":4203347941},{"type":"v1","addr":"172.21.15.184:6839","nonce":4203347941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.33:6839/4136670204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":4136670204},{"type":"v1","addr":"172.21.15.33:6839","nonce":4136670204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24481,24482,24469]},"id":1}]} 2024-09-12T02:01:52.679 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 25 2024-09-12T02:01:52.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:52.681+0000 7fa3dcf5d640 1 -- 172.21.15.16:0/1394325379 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fa3c00a2b80 msgr2=0x7fa3c00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:52.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:52.681+0000 7fa3dcf5d640 1 --2- 172.21.15.16:0/1394325379 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fa3c00a2b80 0x7fa3c00a5040 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fa3c80058c0 tx=0x7fa3c800a250 comp rx=0 tx=0).stop 2024-09-12T02:01:52.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:52.681+0000 7fa3dcf5d640 1 -- 172.21.15.16:0/1394325379 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3d8102790 msgr2=0x7fa3d8074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:52.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:52.681+0000 7fa3dcf5d640 1 --2- 172.21.15.16:0/1394325379 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3d8102790 0x7fa3d8074a30 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc00d950 tx=0x7fa3cc00de10 comp rx=0 tx=0).stop 2024-09-12T02:01:52.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:52.681+0000 7fa3dcf5d640 1 -- 172.21.15.16:0/1394325379 shutdown_connections 2024-09-12T02:01:52.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:52.681+0000 7fa3dcf5d640 1 --2- 172.21.15.16:0/1394325379 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fa3c00a2b80 0x7fa3c00a5040 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:52.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:52.681+0000 7fa3dcf5d640 1 --2- 172.21.15.16:0/1394325379 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa3d8106f10 0x7fa3d8074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:52.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:52.681+0000 7fa3dcf5d640 1 --2- 172.21.15.16:0/1394325379 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3d8102790 0x7fa3d8074a30 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:52.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:52.681+0000 7fa3dcf5d640 1 --2- 172.21.15.16:0/1394325379 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3d8101de0 0x7fa3d80744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:52.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:52.681+0000 7fa3dcf5d640 1 -- 172.21.15.16:0/1394325379 >> 172.21.15.16:0/1394325379 conn(0x7fa3d80fd8c0 msgr2=0x7fa3d80feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:01:52.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:52.681+0000 7fa3dcf5d640 1 -- 172.21.15.16:0/1394325379 shutdown_connections 2024-09-12T02:01:52.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:52.681+0000 7fa3dcf5d640 1 -- 172.21.15.16:0/1394325379 wait complete. 2024-09-12T02:01:53.361 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:01:53.622 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:01:53.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:01:53 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/1394325379' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:01:53.668 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:01:53 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/1394325379' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:01:53.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:01:53 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/1394325379' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:01:54.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:54.082+0000 7f46b6f0e640 1 -- 172.21.15.16:0/440309811 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46b0101de0 msgr2=0x7f46b01021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:54.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:54.082+0000 7f46b6f0e640 1 --2- 172.21.15.16:0/440309811 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46b0101de0 0x7f46b01021c0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f46a0009900 tx=0x7f46a002a410 comp rx=0 tx=0).stop 2024-09-12T02:01:54.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:54.082+0000 7f46b6f0e640 1 -- 172.21.15.16:0/440309811 shutdown_connections 2024-09-12T02:01:54.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:54.082+0000 7f46b6f0e640 1 --2- 172.21.15.16:0/440309811 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46b0106f10 0x7f46b010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:54.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:54.082+0000 7f46b6f0e640 1 --2- 172.21.15.16:0/440309811 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f46b0102790 0x7f46b01067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:54.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:54.082+0000 7f46b6f0e640 1 --2- 172.21.15.16:0/440309811 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46b0101de0 0x7f46b01021c0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:54.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:54.082+0000 7f46b6f0e640 1 -- 172.21.15.16:0/440309811 >> 172.21.15.16:0/440309811 conn(0x7f46b00fd8c0 msgr2=0x7f46b00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:01:54.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:54.082+0000 7f46b6f0e640 1 -- 172.21.15.16:0/440309811 shutdown_connections 2024-09-12T02:01:54.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:54.082+0000 7f46b6f0e640 1 -- 172.21.15.16:0/440309811 wait complete. 2024-09-12T02:01:54.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:54.083+0000 7f46b6f0e640 1 Processor -- start 2024-09-12T02:01:54.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:54.083+0000 7f46b6f0e640 1 -- start start 2024-09-12T02:01:54.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:54.083+0000 7f46b6f0e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46b0101de0 0x7f46b019a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:54.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:54.083+0000 7f46b6f0e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f46b0102790 0x7f46b019ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:54.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:54.083+0000 7f46b6f0e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46b0106f10 0x7f46b019efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:54.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:54.083+0000 7f46b6f0e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f46b0111f30 con 0x7f46b0101de0 2024-09-12T02:01:54.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:54.083+0000 7f46b6f0e640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f46b0111db0 con 0x7f46b0106f10 2024-09-12T02:01:54.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:54.083+0000 7f46b6f0e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f46b01120b0 con 0x7f46b0102790 2024-09-12T02:01:54.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:54.083+0000 7f46b4caa640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46b0101de0 0x7f46b019a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:54.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:54.083+0000 7f46b4caa640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46b0101de0 0x7f46b019a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58120/0 (socket says 172.21.15.16:58120) 2024-09-12T02:01:54.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:54.083+0000 7f46b4caa640 1 -- 172.21.15.16:0/107880912 learned_addr learned my addr 172.21.15.16:0/107880912 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:01:54.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:54.083+0000 7f46affff640 1 --2- 172.21.15.16:0/107880912 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f46b0102790 0x7f46b019ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:54.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:54.083+0000 7f46b54ab640 1 --2- 172.21.15.16:0/107880912 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46b0106f10 0x7f46b019efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:54.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:54.083+0000 7f46b4caa640 1 -- 172.21.15.16:0/107880912 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f46b0102790 msgr2=0x7f46b019ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:54.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:54.083+0000 7f46b4caa640 1 --2- 172.21.15.16:0/107880912 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f46b0102790 0x7f46b019ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:54.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:54.083+0000 7f46b4caa640 1 -- 172.21.15.16:0/107880912 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46b0106f10 msgr2=0x7f46b019efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:54.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:54.083+0000 7f46b4caa640 1 --2- 172.21.15.16:0/107880912 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46b0106f10 0x7f46b019efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:54.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:54.083+0000 7f46b4caa640 1 -- 172.21.15.16:0/107880912 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46b019f680 con 0x7f46b0101de0 2024-09-12T02:01:54.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:54.083+0000 7f46b54ab640 1 --2- 172.21.15.16:0/107880912 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46b0106f10 0x7f46b019efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:01:54.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:54.083+0000 7f46b4caa640 1 --2- 172.21.15.16:0/107880912 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46b0101de0 0x7f46b019a6d0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f46a0006010 tx=0x7f46a00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:01:54.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:54.084+0000 7f46adffb640 1 -- 172.21.15.16:0/107880912 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46a003f030 con 0x7f46b0101de0 2024-09-12T02:01:54.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:54.084+0000 7f46b6f0e640 1 -- 172.21.15.16:0/107880912 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46b019f910 con 0x7f46b0101de0 2024-09-12T02:01:54.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:54.084+0000 7f46adffb640 1 -- 172.21.15.16:0/107880912 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46a0003ce0 con 0x7f46b0101de0 2024-09-12T02:01:54.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:54.084+0000 7f46adffb640 1 -- 172.21.15.16:0/107880912 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46a0035070 con 0x7f46b0101de0 2024-09-12T02:01:54.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:54.084+0000 7f46b6f0e640 1 -- 172.21.15.16:0/107880912 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46b01a7250 con 0x7f46b0101de0 2024-09-12T02:01:54.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:54.085+0000 7f46b6f0e640 1 -- 172.21.15.16:0/107880912 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46b01021c0 con 0x7f46b0101de0 2024-09-12T02:01:54.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:54.086+0000 7f46adffb640 1 -- 172.21.15.16:0/107880912 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f46a0042030 con 0x7f46b0101de0 2024-09-12T02:01:54.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:54.087+0000 7f46adffb640 1 --2- 172.21.15.16:0/107880912 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f46980a2b80 0x7f46980a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:54.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:54.087+0000 7f46affff640 1 --2- 172.21.15.16:0/107880912 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f46980a2b80 0x7f46980a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:54.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:54.087+0000 7f46adffb640 1 -- 172.21.15.16:0/107880912 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f46a00ef970 con 0x7f46b0101de0 2024-09-12T02:01:54.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:54.088+0000 7f46affff640 1 --2- 172.21.15.16:0/107880912 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f46980a2b80 0x7f46980a5040 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f46b00fdea0 tx=0x7f46a400b590 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:01:54.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:54.091+0000 7f46adffb640 1 -- 172.21.15.16:0/107880912 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f46a00bca30 con 0x7f46b0101de0 2024-09-12T02:01:54.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:54.301+0000 7f46b6f0e640 1 -- 172.21.15.16:0/107880912 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f46b01a7420 con 0x7f46b0101de0 2024-09-12T02:01:54.302 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:01:54 smithi016 ceph-mon[30947]: pgmap v334: 97 pgs: 97 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-12T02:01:54.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:54.302+0000 7f46adffb640 1 -- 172.21.15.16:0/107880912 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11911 (secure 0 0 0) 0x7f46a00bc330 con 0x7f46b0101de0 2024-09-12T02:01:54.303 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T02:01:54.303 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":25,"btime":"2024-09-12T02:01:37:192021+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3279123279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3279123279},{"type":"v1","addr":"172.21.15.16:6835","nonce":3279123279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4119452382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4119452382},{"type":"v1","addr":"172.21.15.16:6837","nonce":4119452382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/3263823165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3263823165},{"type":"v1","addr":"172.21.15.16:6839","nonce":3263823165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/1510290732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":1510290732},{"type":"v1","addr":"172.21.15.16:6841","nonce":1510290732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/279674603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":279674603},{"type":"v1","addr":"172.21.15.33:6833","nonce":279674603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3555728028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3555728028},{"type":"v1","addr":"172.21.15.33:6835","nonce":3555728028}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T01:59:56.117608+0000","modified":"2024-09-12T02:01:36.294760+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24482,"mds_1":24481,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6833/1563374106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1563374106},{"type":"v1","addr":"172.21.15.184:6833","nonce":1563374106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6837/1892833868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1892833868},{"type":"v1","addr":"172.21.15.33:6837","nonce":1892833868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/1457679686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1457679686},{"type":"v1","addr":"172.21.15.184:6835","nonce":1457679686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.184:6837/1793393087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":1793393087},{"type":"v1","addr":"172.21.15.184:6837","nonce":1793393087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.184:6839/4203347941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":4203347941},{"type":"v1","addr":"172.21.15.184:6839","nonce":4203347941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.33:6839/4136670204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":4136670204},{"type":"v1","addr":"172.21.15.33:6839","nonce":4136670204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24481,24482,24469]},"id":1}]} 2024-09-12T02:01:54.303 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 25 2024-09-12T02:01:54.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:54.304+0000 7f46b6f0e640 1 -- 172.21.15.16:0/107880912 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f46980a2b80 msgr2=0x7f46980a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:54.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:54.304+0000 7f46b6f0e640 1 --2- 172.21.15.16:0/107880912 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f46980a2b80 0x7f46980a5040 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f46b00fdea0 tx=0x7f46a400b590 comp rx=0 tx=0).stop 2024-09-12T02:01:54.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:54.304+0000 7f46b6f0e640 1 -- 172.21.15.16:0/107880912 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46b0101de0 msgr2=0x7f46b019a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:54.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:54.304+0000 7f46b6f0e640 1 --2- 172.21.15.16:0/107880912 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46b0101de0 0x7f46b019a6d0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f46a0006010 tx=0x7f46a00042f0 comp rx=0 tx=0).stop 2024-09-12T02:01:54.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:54.304+0000 7f46b6f0e640 1 -- 172.21.15.16:0/107880912 shutdown_connections 2024-09-12T02:01:54.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:54.304+0000 7f46b6f0e640 1 --2- 172.21.15.16:0/107880912 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f46980a2b80 0x7f46980a5040 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:54.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:54.304+0000 7f46b6f0e640 1 --2- 172.21.15.16:0/107880912 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46b0106f10 0x7f46b019efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:54.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:54.304+0000 7f46b6f0e640 1 --2- 172.21.15.16:0/107880912 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f46b0102790 0x7f46b019ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:54.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:54.304+0000 7f46b6f0e640 1 --2- 172.21.15.16:0/107880912 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46b0101de0 0x7f46b019a6d0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:54.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:54.304+0000 7f46b6f0e640 1 -- 172.21.15.16:0/107880912 >> 172.21.15.16:0/107880912 conn(0x7f46b00fd8c0 msgr2=0x7f46b00fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:01:54.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:54.305+0000 7f46b6f0e640 1 -- 172.21.15.16:0/107880912 shutdown_connections 2024-09-12T02:01:54.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:54.305+0000 7f46b6f0e640 1 -- 172.21.15.16:0/107880912 wait complete. 2024-09-12T02:01:54.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:01:54 smithi184 ceph-mon[35593]: pgmap v334: 97 pgs: 97 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-12T02:01:54.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:01:54 smithi033 ceph-mon[35332]: pgmap v334: 97 pgs: 97 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-12T02:01:55.051 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T02:01:55.309 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:01:55.429 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:01:55 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/107880912' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:01:55.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:01:55 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/107880912' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:01:55.668 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:01:55 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/107880912' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:01:55.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:55.725+0000 7f230e9b9640 1 -- 172.21.15.16:0/2709780283 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2308102570 msgr2=0x7f23081029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:55.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:55.725+0000 7f230e9b9640 1 --2- 172.21.15.16:0/2709780283 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2308102570 0x7f23081029b0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f22fc009900 tx=0x7f22fc02a480 comp rx=0 tx=0).stop 2024-09-12T02:01:55.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:55.725+0000 7f230e9b9640 1 -- 172.21.15.16:0/2709780283 shutdown_connections 2024-09-12T02:01:55.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:55.725+0000 7f230e9b9640 1 --2- 172.21.15.16:0/2709780283 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2308102ef0 0x7f230810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:55.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:55.725+0000 7f230e9b9640 1 --2- 172.21.15.16:0/2709780283 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2308102570 0x7f23081029b0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:55.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:55.725+0000 7f230e9b9640 1 --2- 172.21.15.16:0/2709780283 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2308100e20 0x7f2308101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:55.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:55.725+0000 7f230e9b9640 1 -- 172.21.15.16:0/2709780283 >> 172.21.15.16:0/2709780283 conn(0x7f23080fc7d0 msgr2=0x7f23080febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:01:55.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:55.725+0000 7f230e9b9640 1 -- 172.21.15.16:0/2709780283 shutdown_connections 2024-09-12T02:01:55.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:55.725+0000 7f230e9b9640 1 -- 172.21.15.16:0/2709780283 wait complete. 2024-09-12T02:01:55.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:55.725+0000 7f230e9b9640 1 Processor -- start 2024-09-12T02:01:55.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:55.725+0000 7f230e9b9640 1 -- start start 2024-09-12T02:01:55.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:55.726+0000 7f230e9b9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2308100e20 0x7f230819a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:55.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:55.726+0000 7f230e9b9640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2308102570 0x7f230819ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:55.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:55.726+0000 7f230e9b9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2308102ef0 0x7f230819efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:55.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:55.726+0000 7f230e9b9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2308110f80 con 0x7f2308102ef0 2024-09-12T02:01:55.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:55.726+0000 7f230e9b9640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2308110e00 con 0x7f2308100e20 2024-09-12T02:01:55.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:55.726+0000 7f230e9b9640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f2308111100 con 0x7f2308102570 2024-09-12T02:01:55.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:55.726+0000 7f230cf56640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2308102ef0 0x7f230819efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:55.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:55.726+0000 7f23077fe640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2308102570 0x7f230819ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:55.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:55.726+0000 7f230cf56640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2308102ef0 0x7f230819efe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58142/0 (socket says 172.21.15.16:58142) 2024-09-12T02:01:55.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:55.726+0000 7f230cf56640 1 -- 172.21.15.16:0/3799157867 learned_addr learned my addr 172.21.15.16:0/3799157867 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:01:55.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:55.726+0000 7f2307fff640 1 --2- 172.21.15.16:0/3799157867 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2308100e20 0x7f230819a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:55.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:55.726+0000 7f230cf56640 1 -- 172.21.15.16:0/3799157867 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2308102570 msgr2=0x7f230819ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:55.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:55.726+0000 7f230cf56640 1 --2- 172.21.15.16:0/3799157867 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2308102570 0x7f230819ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:55.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:55.726+0000 7f230cf56640 1 -- 172.21.15.16:0/3799157867 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2308100e20 msgr2=0x7f230819a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:55.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:55.726+0000 7f230cf56640 1 --2- 172.21.15.16:0/3799157867 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2308100e20 0x7f230819a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:55.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:55.726+0000 7f230cf56640 1 -- 172.21.15.16:0/3799157867 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f230819f6c0 con 0x7f2308102ef0 2024-09-12T02:01:55.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:55.726+0000 7f23077fe640 1 --2- 172.21.15.16:0/3799157867 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2308102570 0x7f230819ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:01:55.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:55.727+0000 7f230cf56640 1 --2- 172.21.15.16:0/3799157867 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2308102ef0 0x7f230819efe0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f230000eaa0 tx=0x7f230000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:01:55.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:55.727+0000 7f23057fa640 1 -- 172.21.15.16:0/3799157867 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f230000cdb0 con 0x7f2308102ef0 2024-09-12T02:01:55.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:55.727+0000 7f23057fa640 1 -- 172.21.15.16:0/3799157867 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2300005020 con 0x7f2308102ef0 2024-09-12T02:01:55.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:55.727+0000 7f230e9b9640 1 -- 172.21.15.16:0/3799157867 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f230819f9b0 con 0x7f2308102ef0 2024-09-12T02:01:55.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:55.727+0000 7f23057fa640 1 -- 172.21.15.16:0/3799157867 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23000116b0 con 0x7f2308102ef0 2024-09-12T02:01:55.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:55.727+0000 7f230e9b9640 1 -- 172.21.15.16:0/3799157867 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23081a7290 con 0x7f2308102ef0 2024-09-12T02:01:55.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:55.728+0000 7f230e9b9640 1 -- 172.21.15.16:0/3799157867 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2308101200 con 0x7f2308102ef0 2024-09-12T02:01:55.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:55.730+0000 7f23057fa640 1 -- 172.21.15.16:0/3799157867 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f2300005560 con 0x7f2308102ef0 2024-09-12T02:01:55.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:55.730+0000 7f23057fa640 1 --2- 172.21.15.16:0/3799157867 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f22f00a2b80 0x7f22f00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:55.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:55.731+0000 7f2307fff640 1 --2- 172.21.15.16:0/3799157867 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f22f00a2b80 0x7f22f00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:55.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:55.731+0000 7f23057fa640 1 -- 172.21.15.16:0/3799157867 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f23000cf530 con 0x7f2308102ef0 2024-09-12T02:01:55.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:55.731+0000 7f2307fff640 1 --2- 172.21.15.16:0/3799157867 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f22f00a2b80 0x7f22f00a5040 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f22f8005fd0 tx=0x7f22f800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:01:55.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:55.734+0000 7f23057fa640 1 -- 172.21.15.16:0/3799157867 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2300016030 con 0x7f2308102ef0 2024-09-12T02:01:55.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:55.908+0000 7f230e9b9640 1 -- 172.21.15.16:0/3799157867 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f23081a7460 con 0x7f2308102ef0 2024-09-12T02:01:55.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:55.910+0000 7f23057fa640 1 -- 172.21.15.16:0/3799157867 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+22584 (secure 0 0 0) 0x7f230009bee0 con 0x7f2308102ef0 2024-09-12T02:01:55.910 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T02:01:55.910 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":82,"fsid":"879f3b56-70a9-11ef-bcea-c7b262605968","created":"2024-09-12T01:52:12.841045+0000","modified":"2024-09-12T02:00:09.559511+0000","last_up_change":"2024-09-12T01:59:08.975443+0000","last_in_change":"2024-09-12T01:58:44.530220+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T01:55:26.727564+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T01:59:52.402097+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T01:59:54.463627+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3ea41eb6-e384-4ddc-810f-0f103c127849","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4000187867},{"type":"v1","addr":"172.21.15.16:6803","nonce":4000187867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4000187867},{"type":"v1","addr":"172.21.15.16:6805","nonce":4000187867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":4000187867},{"type":"v1","addr":"172.21.15.16:6809","nonce":4000187867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4000187867},{"type":"v1","addr":"172.21.15.16:6807","nonce":4000187867}]},"public_addr":"172.21.15.16:6803/4000187867","cluster_addr":"172.21.15.16:6805/4000187867","heartbeat_back_addr":"172.21.15.16:6809/4000187867","heartbeat_front_addr":"172.21.15.16:6807/4000187867","state":["exists","up"]},{"osd":1,"uuid":"07bf63e7-70fa-41b2-af53-12a5fec3f33c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":3862661148},{"type":"v1","addr":"172.21.15.33:6801","nonce":3862661148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":3862661148},{"type":"v1","addr":"172.21.15.33:6803","nonce":3862661148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":3862661148},{"type":"v1","addr":"172.21.15.33:6807","nonce":3862661148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":3862661148},{"type":"v1","addr":"172.21.15.33:6805","nonce":3862661148}]},"public_addr":"172.21.15.33:6801/3862661148","cluster_addr":"172.21.15.33:6803/3862661148","heartbeat_back_addr":"172.21.15.33:6807/3862661148","heartbeat_front_addr":"172.21.15.33:6805/3862661148","state":["exists","up"]},{"osd":2,"uuid":"22e40cac-3bc0-4cce-b47b-77499b767f7e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":3958753963},{"type":"v1","addr":"172.21.15.184:6801","nonce":3958753963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":3958753963},{"type":"v1","addr":"172.21.15.184:6803","nonce":3958753963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":3958753963},{"type":"v1","addr":"172.21.15.184:6807","nonce":3958753963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":3958753963},{"type":"v1","addr":"172.21.15.184:6805","nonce":3958753963}]},"public_addr":"172.21.15.184:6801/3958753963","cluster_addr":"172.21.15.184:6803/3958753963","heartbeat_back_addr":"172.21.15.184:6807/3958753963","heartbeat_front_addr":"172.21.15.184:6805/3958753963","state":["exists","up"]},{"osd":3,"uuid":"834a07f5-2c24-49e1-8e4a-12c96cfbdc65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3395182524},{"type":"v1","addr":"172.21.15.16:6811","nonce":3395182524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3395182524},{"type":"v1","addr":"172.21.15.16:6813","nonce":3395182524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3395182524},{"type":"v1","addr":"172.21.15.16:6817","nonce":3395182524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3395182524},{"type":"v1","addr":"172.21.15.16:6815","nonce":3395182524}]},"public_addr":"172.21.15.16:6811/3395182524","cluster_addr":"172.21.15.16:6813/3395182524","heartbeat_back_addr":"172.21.15.16:6817/3395182524","heartbeat_front_addr":"172.21.15.16:6815/3395182524","state":["exists","up"]},{"osd":4,"uuid":"3ca56e09-c194-4404-b9bd-2524d38367eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":1355184322},{"type":"v1","addr":"172.21.15.33:6809","nonce":1355184322}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1355184322},{"type":"v1","addr":"172.21.15.33:6811","nonce":1355184322}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1355184322},{"type":"v1","addr":"172.21.15.33:6815","nonce":1355184322}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1355184322},{"type":"v1","addr":"172.21.15.33:6813","nonce":1355184322}]},"public_addr":"172.21.15.33:6809/1355184322","cluster_addr":"172.21.15.33:6811/1355184322","heartbeat_back_addr":"172.21.15.33:6815/1355184322","heartbeat_front_addr":"172.21.15.33:6813/1355184322","state":["exists","up"]},{"osd":5,"uuid":"0b05ecdb-b987-461f-951c-6d76739bfb4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":1533397276},{"type":"v1","addr":"172.21.15.184:6809","nonce":1533397276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":1533397276},{"type":"v1","addr":"172.21.15.184:6811","nonce":1533397276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":1533397276},{"type":"v1","addr":"172.21.15.184:6815","nonce":1533397276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":1533397276},{"type":"v1","addr":"172.21.15.184:6813","nonce":1533397276}]},"public_addr":"172.21.15.184:6809/1533397276","cluster_addr":"172.21.15.184:6811/1533397276","heartbeat_back_addr":"172.21.15.184:6815/1533397276","heartbeat_front_addr":"172.21.15.184:6813/1533397276","state":["exists","up"]},{"osd":6,"uuid":"b3936e21-e197-49fa-8441-d909b258b612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":902466142},{"type":"v1","addr":"172.21.15.16:6819","nonce":902466142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":902466142},{"type":"v1","addr":"172.21.15.16:6821","nonce":902466142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":902466142},{"type":"v1","addr":"172.21.15.16:6825","nonce":902466142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":902466142},{"type":"v1","addr":"172.21.15.16:6823","nonce":902466142}]},"public_addr":"172.21.15.16:6819/902466142","cluster_addr":"172.21.15.16:6821/902466142","heartbeat_back_addr":"172.21.15.16:6825/902466142","heartbeat_front_addr":"172.21.15.16:6823/902466142","state":["exists","up"]},{"osd":7,"uuid":"4ddbe011-34c1-4b3f-a8eb-577422fb5426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":4186032630},{"type":"v1","addr":"172.21.15.33:6817","nonce":4186032630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":4186032630},{"type":"v1","addr":"172.21.15.33:6819","nonce":4186032630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":4186032630},{"type":"v1","addr":"172.21.15.33:6823","nonce":4186032630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":4186032630},{"type":"v1","addr":"172.21.15.33:6821","nonce":4186032630}]},"public_addr":"172.21.15.33:6817/4186032630","cluster_addr":"172.21.15.33:6819/4186032630","heartbeat_back_addr":"172.21.15.33:6823/4186032630","heartbeat_front_addr":"172.21.15.33:6821/4186032630","state":["exists","up"]},{"osd":8,"uuid":"dca5bfb8-ac6a-47ce-905c-be7fcaf0b5ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":781851781},{"type":"v1","addr":"172.21.15.184:6817","nonce":781851781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":781851781},{"type":"v1","addr":"172.21.15.184:6819","nonce":781851781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":781851781},{"type":"v1","addr":"172.21.15.184:6823","nonce":781851781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":781851781},{"type":"v1","addr":"172.21.15.184:6821","nonce":781851781}]},"public_addr":"172.21.15.184:6817/781851781","cluster_addr":"172.21.15.184:6819/781851781","heartbeat_back_addr":"172.21.15.184:6823/781851781","heartbeat_front_addr":"172.21.15.184:6821/781851781","state":["exists","up"]},{"osd":9,"uuid":"23a05cb5-01d5-40f3-b66a-15fcccedbd6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":1569578746},{"type":"v1","addr":"172.21.15.16:6827","nonce":1569578746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":1569578746},{"type":"v1","addr":"172.21.15.16:6829","nonce":1569578746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1569578746},{"type":"v1","addr":"172.21.15.16:6833","nonce":1569578746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":1569578746},{"type":"v1","addr":"172.21.15.16:6831","nonce":1569578746}]},"public_addr":"172.21.15.16:6827/1569578746","cluster_addr":"172.21.15.16:6829/1569578746","heartbeat_back_addr":"172.21.15.16:6833/1569578746","heartbeat_front_addr":"172.21.15.16:6831/1569578746","state":["exists","up"]},{"osd":10,"uuid":"e4bd6bce-3c69-44db-bc09-d264a0ad094a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":469977847},{"type":"v1","addr":"172.21.15.33:6825","nonce":469977847}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":469977847},{"type":"v1","addr":"172.21.15.33:6827","nonce":469977847}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":469977847},{"type":"v1","addr":"172.21.15.33:6831","nonce":469977847}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":469977847},{"type":"v1","addr":"172.21.15.33:6829","nonce":469977847}]},"public_addr":"172.21.15.33:6825/469977847","cluster_addr":"172.21.15.33:6827/469977847","heartbeat_back_addr":"172.21.15.33:6831/469977847","heartbeat_front_addr":"172.21.15.33:6829/469977847","state":["exists","up"]},{"osd":11,"uuid":"d60212f8-5a1d-45f7-9a1b-12294bb4c80d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":1619871346},{"type":"v1","addr":"172.21.15.184:6825","nonce":1619871346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":1619871346},{"type":"v1","addr":"172.21.15.184:6827","nonce":1619871346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":1619871346},{"type":"v1","addr":"172.21.15.184:6831","nonce":1619871346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":1619871346},{"type":"v1","addr":"172.21.15.184:6829","nonce":1619871346}]},"public_addr":"172.21.15.184:6825/1619871346","cluster_addr":"172.21.15.184:6827/1619871346","heartbeat_back_addr":"172.21.15.184:6831/1619871346","heartbeat_front_addr":"172.21.15.184:6829/1619871346","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:54:32.057979+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:54:57.501562+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:55:23.137225+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:55:48.646155+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:56:13.311907+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:56:38.214188+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:57:02.570921+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:57:27.506097+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:57:52.038529+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:58:17.423300+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:58:41.579992+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:59:06.245817+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:0/3532565698":"2024-09-13T01:53:08.218475+0000","172.21.15.16:0/1822157179":"2024-09-13T01:52:44.194152+0000","172.21.15.16:0/3919664061":"2024-09-13T01:52:44.194152+0000","172.21.15.16:0/2269394774":"2024-09-13T01:52:44.194152+0000","172.21.15.16:0/2595308634":"2024-09-13T01:53:08.218475+0000","172.21.15.16:6801/2754331153":"2024-09-13T01:52:44.194152+0000","172.21.15.16:6801/2615861334":"2024-09-13T01:53:08.218475+0000","172.21.15.16:6800/2615861334":"2024-09-13T01:53:08.218475+0000","172.21.15.16:0/4199114771":"2024-09-13T01:53:08.218475+0000","172.21.15.16:6800/2754331153":"2024-09-13T01:52:44.194152+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T02:01:55.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:55.912+0000 7f230e9b9640 1 -- 172.21.15.16:0/3799157867 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f22f00a2b80 msgr2=0x7f22f00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:55.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:55.912+0000 7f230e9b9640 1 --2- 172.21.15.16:0/3799157867 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f22f00a2b80 0x7f22f00a5040 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f22f8005fd0 tx=0x7f22f800b390 comp rx=0 tx=0).stop 2024-09-12T02:01:55.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:55.912+0000 7f230e9b9640 1 -- 172.21.15.16:0/3799157867 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2308102ef0 msgr2=0x7f230819efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:55.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:55.912+0000 7f230e9b9640 1 --2- 172.21.15.16:0/3799157867 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2308102ef0 0x7f230819efe0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f230000eaa0 tx=0x7f230000ef60 comp rx=0 tx=0).stop 2024-09-12T02:01:55.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:55.912+0000 7f230e9b9640 1 -- 172.21.15.16:0/3799157867 shutdown_connections 2024-09-12T02:01:55.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:55.912+0000 7f230e9b9640 1 --2- 172.21.15.16:0/3799157867 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f22f00a2b80 0x7f22f00a5040 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:55.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:55.912+0000 7f230e9b9640 1 --2- 172.21.15.16:0/3799157867 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2308102ef0 0x7f230819efe0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:55.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:55.912+0000 7f230e9b9640 1 --2- 172.21.15.16:0/3799157867 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2308102570 0x7f230819ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:55.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:55.912+0000 7f230e9b9640 1 --2- 172.21.15.16:0/3799157867 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2308100e20 0x7f230819a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:55.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:55.912+0000 7f230e9b9640 1 -- 172.21.15.16:0/3799157867 >> 172.21.15.16:0/3799157867 conn(0x7f23080fc7d0 msgr2=0x7f23080ffc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:01:55.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:55.912+0000 7f230e9b9640 1 -- 172.21.15.16:0/3799157867 shutdown_connections 2024-09-12T02:01:55.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:55.912+0000 7f230e9b9640 1 -- 172.21.15.16:0/3799157867 wait complete. 2024-09-12T02:01:56.547 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:01:56 smithi016 ceph-mon[30947]: pgmap v335: 97 pgs: 97 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-09-12T02:01:56.547 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:01:56 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/3799157867' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T02:01:56.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:01:56 smithi184 ceph-mon[35593]: pgmap v335: 97 pgs: 97 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-09-12T02:01:56.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:01:56 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/3799157867' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T02:01:56.668 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:01:56 smithi033 ceph-mon[35332]: pgmap v335: 97 pgs: 97 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-09-12T02:01:56.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:01:56 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/3799157867' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T02:01:56.713 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph daemon mon.a --format=json config get run_dir 2024-09-12T02:01:56.974 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:01:57.375 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:01:57 smithi016 ceph-mon[30947]: scrub summary: idle 2024-09-12T02:01:57.625 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.621+0000 7f2cb895f640 1 -- 172.21.15.16:0/1774449138 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cb4104f10 msgr2=0x7f2cb4107300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:57.625 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.621+0000 7f2cb895f640 1 --2- 172.21.15.16:0/1774449138 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cb4104f10 0x7f2cb4107300 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f2ca8009900 tx=0x7f2ca802a410 comp rx=0 tx=0).stop 2024-09-12T02:01:57.625 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.621+0000 7f2cb895f640 1 -- 172.21.15.16:0/1774449138 shutdown_connections 2024-09-12T02:01:57.626 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.621+0000 7f2cb895f640 1 --2- 172.21.15.16:0/1774449138 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2cb4107840 0x7f2cb4109cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:57.626 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.621+0000 7f2cb895f640 1 --2- 172.21.15.16:0/1774449138 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cb4104f10 0x7f2cb4107300 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:57.626 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.621+0000 7f2cb895f640 1 --2- 172.21.15.16:0/1774449138 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2cb41025e0 0x7f2cb41049d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:57.626 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.621+0000 7f2cb895f640 1 -- 172.21.15.16:0/1774449138 >> 172.21.15.16:0/1774449138 conn(0x7f2cb40fc1e0 msgr2=0x7f2cb40fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:01:57.626 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.622+0000 7f2cb895f640 1 -- 172.21.15.16:0/1774449138 shutdown_connections 2024-09-12T02:01:57.626 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.622+0000 7f2cb895f640 1 -- 172.21.15.16:0/1774449138 wait complete. 2024-09-12T02:01:57.627 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.622+0000 7f2cb895f640 1 Processor -- start 2024-09-12T02:01:57.627 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.622+0000 7f2cb895f640 1 -- start start 2024-09-12T02:01:57.627 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.622+0000 7f2cb895f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cb41025e0 0x7f2cb4197e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:57.627 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.622+0000 7f2cb895f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2cb4104f10 0x7f2cb4198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:57.627 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.622+0000 7f2cb895f640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2cb4107840 0x7f2cb419f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:57.627 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.622+0000 7f2cb895f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2cb410d820 con 0x7f2cb41025e0 2024-09-12T02:01:57.628 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.622+0000 7f2cb895f640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2cb410d6a0 con 0x7f2cb4104f10 2024-09-12T02:01:57.628 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.622+0000 7f2cb895f640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f2cb410d9a0 con 0x7f2cb4107840 2024-09-12T02:01:57.628 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.623+0000 7f2cb259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cb41025e0 0x7f2cb4197e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:57.628 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.623+0000 7f2cb259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cb41025e0 0x7f2cb4197e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58162/0 (socket says 172.21.15.16:58162) 2024-09-12T02:01:57.628 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.623+0000 7f2cb259c640 1 -- 172.21.15.16:0/2273572012 learned_addr learned my addr 172.21.15.16:0/2273572012 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:01:57.628 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.623+0000 7f2cb1d9b640 1 --2- 172.21.15.16:0/2273572012 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2cb4104f10 0x7f2cb4198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:57.628 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.623+0000 7f2cb259c640 1 -- 172.21.15.16:0/2273572012 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2cb4107840 msgr2=0x7f2cb419f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:57.628 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.623+0000 7f2cb259c640 1 --2- 172.21.15.16:0/2273572012 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2cb4107840 0x7f2cb419f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:57.628 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.623+0000 7f2cb259c640 1 -- 172.21.15.16:0/2273572012 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2cb4104f10 msgr2=0x7f2cb4198390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:57.628 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.623+0000 7f2cb259c640 1 --2- 172.21.15.16:0/2273572012 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2cb4104f10 0x7f2cb4198390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:57.628 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.623+0000 7f2cb259c640 1 -- 172.21.15.16:0/2273572012 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cb419fa70 con 0x7f2cb41025e0 2024-09-12T02:01:57.628 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.623+0000 7f2cb259c640 1 --2- 172.21.15.16:0/2273572012 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cb41025e0 0x7f2cb4197e50 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f2ca400d950 tx=0x7f2ca400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:01:57.629 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.623+0000 7f2ca37fe640 1 -- 172.21.15.16:0/2273572012 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ca4017070 con 0x7f2cb41025e0 2024-09-12T02:01:57.629 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.623+0000 7f2ca37fe640 1 -- 172.21.15.16:0/2273572012 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ca4009c30 con 0x7f2cb41025e0 2024-09-12T02:01:57.629 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.623+0000 7f2cb895f640 1 -- 172.21.15.16:0/2273572012 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cb419fd60 con 0x7f2cb41025e0 2024-09-12T02:01:57.629 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.623+0000 7f2ca37fe640 1 -- 172.21.15.16:0/2273572012 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ca400f6b0 con 0x7f2cb41025e0 2024-09-12T02:01:57.629 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.623+0000 7f2cb895f640 1 -- 172.21.15.16:0/2273572012 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cb41a0210 con 0x7f2cb41025e0 2024-09-12T02:01:57.630 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:01:57 smithi184 ceph-mon[35593]: scrub summary: idle 2024-09-12T02:01:57.636 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.626+0000 7f2ca37fe640 1 -- 172.21.15.16:0/2273572012 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f2ca40109d0 con 0x7f2cb41025e0 2024-09-12T02:01:57.636 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.626+0000 7f2cbb7d7740 1 Processor -- start 2024-09-12T02:01:57.636 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.626+0000 7f2cbb7d7740 1 -- start start 2024-09-12T02:01:57.636 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.626+0000 7f2cbb7d7740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5595b4d579a0 0x5595b4d57da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:57.636 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.626+0000 7f2cbb7d7740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5595b4d582e0 0x5595b4d5cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:57.636 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.626+0000 7f2cbb7d7740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5595b4d5d370 0x5595b4d5f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:57.636 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.626+0000 7f2cbb7d7740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5595b4cbb2f0 con 0x5595b4d582e0 2024-09-12T02:01:57.637 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.626+0000 7f2cbb7d7740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5595b4c63270 con 0x5595b4d579a0 2024-09-12T02:01:57.637 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.626+0000 7f2cbb7d7740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5595b4cb72b0 con 0x5595b4d5d370 2024-09-12T02:01:57.637 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.627+0000 7f2cb259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5595b4d582e0 0x5595b4d5cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:57.637 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.627+0000 7f2cb259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5595b4d582e0 0x5595b4d5cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58168/0 (socket says 172.21.15.16:58168) 2024-09-12T02:01:57.637 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.627+0000 7f2cb259c640 1 -- 172.21.15.16:0/3050572748 learned_addr learned my addr 172.21.15.16:0/3050572748 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:01:57.637 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.627+0000 7f2cb2d9d640 1 --2- 172.21.15.16:0/3050572748 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5595b4d579a0 0x5595b4d57da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:57.637 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.627+0000 7f2ca37fe640 1 --2- 172.21.15.16:0/2273572012 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f2c9c0a2cb0 0x7f2c9c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:57.637 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.627+0000 7f2cb1d9b640 1 --2- 172.21.15.16:0/3050572748 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5595b4d5d370 0x5595b4d5f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:57.637 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.627+0000 7f2cb259c640 1 -- 172.21.15.16:0/3050572748 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5595b4d5d370 msgr2=0x5595b4d5f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:01:57.637 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.627+0000 7f2cb259c640 1 --2- 172.21.15.16:0/3050572748 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5595b4d5d370 0x5595b4d5f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:57.637 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.627+0000 7f2cb259c640 1 -- 172.21.15.16:0/3050572748 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5595b4d579a0 msgr2=0x5595b4d57da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:01:57.637 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.627+0000 7f2cb259c640 1 --2- 172.21.15.16:0/3050572748 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5595b4d579a0 0x5595b4d57da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:57.638 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.627+0000 7f2ca37fe640 1 -- 172.21.15.16:0/2273572012 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f2ca40cec30 con 0x7f2cb41025e0 2024-09-12T02:01:57.638 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.627+0000 7f2cb259c640 1 -- 172.21.15.16:0/3050572748 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595b4c62340 con 0x5595b4d582e0 2024-09-12T02:01:57.638 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.627+0000 7f2cb2d9d640 1 --2- 172.21.15.16:0/3050572748 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5595b4d579a0 0x5595b4d57da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:01:57.638 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.627+0000 7f2cb2d9d640 1 --2- 172.21.15.16:0/2273572012 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f2c9c0a2cb0 0x7f2c9c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:57.638 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.627+0000 7f2cb259c640 1 --2- 172.21.15.16:0/3050572748 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5595b4d582e0 0x5595b4d5cda0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f2ca4097e60 tx=0x7f2ca4050d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6c863c6a101269 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:01:57.638 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.627+0000 7f2ca0ff9640 1 -- 172.21.15.16:0/3050572748 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ca4020070 con 0x5595b4d582e0 2024-09-12T02:01:57.639 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.627+0000 7f2ca0ff9640 1 -- 172.21.15.16:0/3050572748 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ca4009de0 con 0x5595b4d582e0 2024-09-12T02:01:57.639 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.627+0000 7f2ca0ff9640 1 -- 172.21.15.16:0/3050572748 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ca40cc840 con 0x5595b4d582e0 2024-09-12T02:01:57.639 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.628+0000 7f2cbb7d7740 1 -- 172.21.15.16:0/3050572748 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5595b4d582e0 msgr2=0x5595b4d5cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:57.639 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.628+0000 7f2cb2d9d640 1 --2- 172.21.15.16:0/2273572012 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f2c9c0a2cb0 0x7f2c9c0a5170 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f2cac009270 tx=0x7f2cac004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:01:57.639 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.628+0000 7f2cbb7d7740 1 --2- 172.21.15.16:0/3050572748 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5595b4d582e0 0x5595b4d5cda0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f2ca4097e60 tx=0x7f2ca4050d40 comp rx=0 tx=0).stop 2024-09-12T02:01:57.639 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.628+0000 7f2cbb7d7740 1 -- 172.21.15.16:0/3050572748 shutdown_connections 2024-09-12T02:01:57.639 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.628+0000 7f2cbb7d7740 1 --2- 172.21.15.16:0/3050572748 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5595b4d5d370 0x5595b4d5f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:57.639 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.628+0000 7f2cbb7d7740 1 --2- 172.21.15.16:0/3050572748 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5595b4d582e0 0x5595b4d5cda0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:57.639 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.628+0000 7f2cbb7d7740 1 --2- 172.21.15.16:0/3050572748 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5595b4d579a0 0x5595b4d57da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:57.639 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.628+0000 7f2cbb7d7740 1 -- 172.21.15.16:0/3050572748 >> 172.21.15.16:0/3050572748 conn(0x5595b4d53010 msgr2=0x5595b4d53410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:01:57.640 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.628+0000 7f2cbb7d7740 1 -- 172.21.15.16:0/3050572748 shutdown_connections 2024-09-12T02:01:57.640 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.628+0000 7f2cbb7d7740 1 -- 172.21.15.16:0/3050572748 wait complete. 2024-09-12T02:01:57.640 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.629+0000 7f2cbb7d7740 1 -- start start 2024-09-12T02:01:57.640 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.629+0000 7f2cbb7d7740 1 Processor -- start 2024-09-12T02:01:57.640 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.629+0000 7f2cbb7d7740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5595b4d579a0 0x5595b4d61b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:57.640 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.629+0000 7f2cbb7d7740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5595b4d582e0 0x5595b4d62050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:57.640 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.629+0000 7f2cbb7d7740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5595b4d5d370 0x5595b4d69090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:57.640 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.629+0000 7f2cbb7d7740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5595b4d602e0 con 0x5595b4d579a0 2024-09-12T02:01:57.640 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.629+0000 7f2cbb7d7740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5595b4c44290 con 0x5595b4d582e0 2024-09-12T02:01:57.641 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.629+0000 7f2cbb7d7740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5595b4d60460 con 0x5595b4d5d370 2024-09-12T02:01:57.641 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.629+0000 7f2cbb7d7740 10 client.0 _ll_register_callbacks cb 0x5595b4c75140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:01:57.641 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.629+0000 7f2cb259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5595b4d579a0 0x5595b4d61b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:57.641 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.629+0000 7f2cb259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5595b4d579a0 0x5595b4d61b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58184/0 (socket says 172.21.15.16:58184) 2024-09-12T02:01:57.641 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.629+0000 7f2cb259c640 1 -- 172.21.15.16:0/187014319 learned_addr learned my addr 172.21.15.16:0/187014319 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:01:57.641 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.629+0000 7f2cb2d9d640 1 --2- 172.21.15.16:0/187014319 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5595b4d5d370 0x5595b4d69090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:57.641 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.629+0000 7f2cb1d9b640 1 --2- 172.21.15.16:0/187014319 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5595b4d582e0 0x5595b4d62050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:57.641 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.629+0000 7f2cb259c640 1 -- 172.21.15.16:0/187014319 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5595b4d5d370 msgr2=0x5595b4d69090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:01:57.642 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.629+0000 7f2cb259c640 1 --2- 172.21.15.16:0/187014319 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5595b4d5d370 0x5595b4d69090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:57.642 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.629+0000 7f2cb259c640 1 -- 172.21.15.16:0/187014319 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5595b4d582e0 msgr2=0x5595b4d62050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:01:57.642 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.629+0000 7f2cb259c640 1 --2- 172.21.15.16:0/187014319 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5595b4d582e0 0x5595b4d62050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:57.642 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.629+0000 7f2cb259c640 1 -- 172.21.15.16:0/187014319 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595b49ce210 con 0x5595b4d579a0 2024-09-12T02:01:57.642 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.629+0000 7f2cb2d9d640 1 --2- 172.21.15.16:0/187014319 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5595b4d5d370 0x5595b4d69090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:01:57.642 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.629+0000 7f2cb259c640 1 --2- 172.21.15.16:0/187014319 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5595b4d579a0 0x5595b4d61b10 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f2ca40cef20 tx=0x7f2ca4000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b774709b9d66dc8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:01:57.642 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.629+0000 7f2c99ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:01:57.642 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.630+0000 7f2c99ffb640 1 -- 172.21.15.16:0/187014319 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ca4020040 con 0x5595b4d579a0 2024-09-12T02:01:57.642 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.630+0000 7f2c99ffb640 1 -- 172.21.15.16:0/187014319 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f2ca4097b00 con 0x5595b4d579a0 2024-09-12T02:01:57.642 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.630+0000 7f2cbb7d7740 1 -- 172.21.15.16:0/187014319 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ca4009de0 con 0x5595b4d579a0 2024-09-12T02:01:57.642 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.630+0000 7f2c99ffb640 1 -- 172.21.15.16:0/187014319 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2ca409bb90 con 0x5595b4d579a0 2024-09-12T02:01:57.643 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.630+0000 7f2cbb7d7740 10 client.15282 fetch_fsmap learned FSMap version 25 2024-09-12T02:01:57.643 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.630+0000 7f2cbb7d7740 1 -- 172.21.15.16:0/187014319 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5595b4a9a620 con 0x5595b4d579a0 2024-09-12T02:01:57.643 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.630+0000 7f2c99ffb640 1 -- 172.21.15.16:0/187014319 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ca4009de0 con 0x5595b4d579a0 2024-09-12T02:01:57.643 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.630+0000 7f2c99ffb640 1 -- 172.21.15.16:0/187014319 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f2ca40ccb50 con 0x5595b4d579a0 2024-09-12T02:01:57.643 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.630+0000 7f2cbb7d7740 10 client.15282 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:01:57.643 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.630+0000 7f2cbb7d7740 10 client.15282 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:01:57.643 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.630+0000 7f2cbb7d7740 20 client.15282 populate_metadata read hostname 'smithi016' 2024-09-12T02:01:57.643 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.630+0000 7f2cbb7d7740 1 --2- 172.21.15.16:0/187014319 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5595b4d6d390 0x5595b4d6f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:57.644 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.630+0000 7f2cbb7d7740 4 client.15282 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:01:57.644 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.630+0000 7f2cbb7d7740 1 -- 172.21.15.16:0/187014319 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5595b4cc8a20 con 0x5595b4d6d390 2024-09-12T02:01:57.644 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.631+0000 7f2cb1d9b640 1 --2- 172.21.15.16:0/187014319 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5595b4d6d390 0x5595b4d6f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:57.644 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.631+0000 7f2cb1d9b640 1 --2- 172.21.15.16:0/187014319 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5595b4d6d390 0x5595b4d6f780 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b9c369a3318dff80 server_cookie=6fc54916c36fa08b in_seq=0 out_seq=0 2024-09-12T02:01:57.644 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.631+0000 7f2c99ffb640 10 client.15282 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:01:57.644 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.632+0000 7f2c99ffb640 1 -- 172.21.15.16:0/187014319 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5595b4cc8a20 con 0x5595b4d6d390 2024-09-12T02:01:57.645 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.632+0000 7f2c99ffb640 10 client.15282 handle_command_reply: tid=0 2024-09-12T02:01:57.645 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.632+0000 7f2cbb7d7740 1 client.15282 shutdown 2024-09-12T02:01:57.645 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.632+0000 7f2cbb7d7740 1 -- 172.21.15.16:0/187014319 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5595b4d579a0 msgr2=0x5595b4d61b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:57.645 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.632+0000 7f2cbb7d7740 1 --2- 172.21.15.16:0/187014319 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5595b4d579a0 0x5595b4d61b10 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f2ca40cef20 tx=0x7f2ca4000bc0 comp rx=0 tx=0).stop 2024-09-12T02:01:57.645 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.632+0000 7f2cbb7d7740 1 -- 172.21.15.16:0/187014319 shutdown_connections 2024-09-12T02:01:57.645 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.632+0000 7f2cbb7d7740 1 --2- 172.21.15.16:0/187014319 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5595b4d6d390 0x5595b4d6f780 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:57.645 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.632+0000 7f2cbb7d7740 1 --2- 172.21.15.16:0/187014319 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5595b4d5d370 0x5595b4d69090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:57.645 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.632+0000 7f2cbb7d7740 1 --2- 172.21.15.16:0/187014319 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5595b4d582e0 0x5595b4d62050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:57.645 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.632+0000 7f2c99ffb640 0 client.15282 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:01:57.646 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.632+0000 7f2cbb7d7740 1 --2- 172.21.15.16:0/187014319 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5595b4d579a0 0x5595b4d61b10 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:57.646 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.632+0000 7f2cbb7d7740 1 -- 172.21.15.16:0/187014319 >> 172.21.15.16:0/187014319 conn(0x5595b4d53010 msgr2=0x5595b4d54820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:01:57.646 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.632+0000 7f2cbb7d7740 1 -- 172.21.15.16:0/187014319 shutdown_connections 2024-09-12T02:01:57.646 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.632+0000 7f2cbb7d7740 1 -- 172.21.15.16:0/187014319 wait complete. 2024-09-12T02:01:57.646 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.632+0000 7f2cbb7d7740 20 client.15282 trim_cache size 0 max 16384 2024-09-12T02:01:57.651 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.646+0000 7f2cbb7d7740 1 Processor -- start 2024-09-12T02:01:57.651 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.646+0000 7f2cbb7d7740 1 -- start start 2024-09-12T02:01:57.651 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.646+0000 7f2cbb7d7740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5595b4d579a0 0x5595b4d620e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:57.651 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.646+0000 7f2cbb7d7740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5595b4d582e0 0x5595b4d54690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:57.651 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.646+0000 7f2cbb7d7740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5595b4d5d370 0x5595b4d73e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:57.651 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.646+0000 7f2cbb7d7740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2cb41ab740 con 0x5595b4d5d370 2024-09-12T02:01:57.652 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.646+0000 7f2cbb7d7740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2cb41a9010 con 0x5595b4d579a0 2024-09-12T02:01:57.652 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.646+0000 7f2cbb7d7740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f2cb41a6e60 con 0x5595b4d582e0 2024-09-12T02:01:57.652 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.647+0000 7f2cb2d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5595b4d5d370 0x5595b4d73e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:57.652 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.647+0000 7f2cb2d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5595b4d5d370 0x5595b4d73e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58186/0 (socket says 172.21.15.16:58186) 2024-09-12T02:01:57.652 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.647+0000 7f2cb2d9d640 1 -- 172.21.15.16:0/2115385462 learned_addr learned my addr 172.21.15.16:0/2115385462 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:01:57.652 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.647+0000 7f2cb259c640 1 --2- 172.21.15.16:0/2115385462 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5595b4d579a0 0x5595b4d620e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:57.652 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.647+0000 7f2cb1d9b640 1 --2- 172.21.15.16:0/2115385462 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5595b4d582e0 0x5595b4d54690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:57.652 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.647+0000 7f2cb2d9d640 1 -- 172.21.15.16:0/2115385462 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5595b4d582e0 msgr2=0x5595b4d54690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:01:57.652 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.647+0000 7f2cb2d9d640 1 --2- 172.21.15.16:0/2115385462 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5595b4d582e0 0x5595b4d54690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:57.653 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.647+0000 7f2cb2d9d640 1 -- 172.21.15.16:0/2115385462 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5595b4d579a0 msgr2=0x5595b4d620e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:01:57.653 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.647+0000 7f2cb2d9d640 1 --2- 172.21.15.16:0/2115385462 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5595b4d579a0 0x5595b4d620e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:57.653 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.647+0000 7f2cb2d9d640 1 -- 172.21.15.16:0/2115385462 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595b4d61650 con 0x5595b4d5d370 2024-09-12T02:01:57.653 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.647+0000 7f2cb259c640 1 --2- 172.21.15.16:0/2115385462 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5595b4d579a0 0x5595b4d620e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:01:57.653 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.647+0000 7f2cb2d9d640 1 --2- 172.21.15.16:0/2115385462 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5595b4d5d370 0x5595b4d73e30 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f2cac01d530 tx=0x7f2cac013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c85c7ab1bd87a9d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:01:57.653 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.647+0000 7f2c9bfff640 1 -- 172.21.15.16:0/2115385462 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cac0124b0 con 0x5595b4d5d370 2024-09-12T02:01:57.653 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.647+0000 7f2c9bfff640 1 -- 172.21.15.16:0/2115385462 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2cac012b30 con 0x5595b4d5d370 2024-09-12T02:01:57.653 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.647+0000 7f2c9bfff640 1 -- 172.21.15.16:0/2115385462 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cac01bcc0 con 0x5595b4d5d370 2024-09-12T02:01:57.653 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.648+0000 7f2cbb7d7740 1 -- 172.21.15.16:0/2115385462 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5595b4d5d370 msgr2=0x5595b4d73e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:57.653 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.648+0000 7f2cbb7d7740 1 --2- 172.21.15.16:0/2115385462 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5595b4d5d370 0x5595b4d73e30 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f2cac01d530 tx=0x7f2cac013000 comp rx=0 tx=0).stop 2024-09-12T02:01:57.653 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.648+0000 7f2cbb7d7740 1 -- 172.21.15.16:0/2115385462 shutdown_connections 2024-09-12T02:01:57.653 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.648+0000 7f2cbb7d7740 1 --2- 172.21.15.16:0/2115385462 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5595b4d5d370 0x5595b4d73e30 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:57.653 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.648+0000 7f2cbb7d7740 1 --2- 172.21.15.16:0/2115385462 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5595b4d582e0 0x5595b4d54690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:57.654 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.648+0000 7f2cbb7d7740 1 --2- 172.21.15.16:0/2115385462 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5595b4d579a0 0x5595b4d620e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:57.654 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.648+0000 7f2cbb7d7740 1 -- 172.21.15.16:0/2115385462 >> 172.21.15.16:0/2115385462 conn(0x5595b4d53010 msgr2=0x5595b4d53f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:01:57.654 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.648+0000 7f2cbb7d7740 1 -- 172.21.15.16:0/2115385462 shutdown_connections 2024-09-12T02:01:57.654 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.648+0000 7f2cbb7d7740 1 -- 172.21.15.16:0/2115385462 wait complete. 2024-09-12T02:01:57.654 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.648+0000 7f2cbb7d7740 1 -- start start 2024-09-12T02:01:57.654 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.648+0000 7f2cbb7d7740 1 Processor -- start 2024-09-12T02:01:57.654 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.648+0000 7f2cbb7d7740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5595b4d579a0 0x5595b4dd31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:57.654 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.648+0000 7f2cbb7d7740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5595b4d582e0 0x5595b4dd3730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:57.654 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.649+0000 7f2cbb7d7740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5595b4d5d370 0x5595b4dda770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:57.654 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.649+0000 7f2cbb7d7740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2cb41a74d0 con 0x5595b4d582e0 2024-09-12T02:01:57.654 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.649+0000 7f2cbb7d7740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2cb41a8940 con 0x5595b4d5d370 2024-09-12T02:01:57.654 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.649+0000 7f2cbb7d7740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f2cb41a7ba0 con 0x5595b4d579a0 2024-09-12T02:01:57.655 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.649+0000 7f2cbb7d7740 10 client.0 _ll_register_callbacks cb 0x7f2ca8005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:01:57.655 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.649+0000 7f2cb1d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5595b4d582e0 0x5595b4dd3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:57.655 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.649+0000 7f2cb1d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5595b4d582e0 0x5595b4dd3730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58200/0 (socket says 172.21.15.16:58200) 2024-09-12T02:01:57.655 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.649+0000 7f2cb1d9b640 1 -- 172.21.15.16:0/2201484271 learned_addr learned my addr 172.21.15.16:0/2201484271 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:01:57.655 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.649+0000 7f2cb259c640 1 --2- 172.21.15.16:0/2201484271 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5595b4d579a0 0x5595b4dd31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:57.655 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.649+0000 7f2cb2d9d640 1 --2- 172.21.15.16:0/2201484271 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5595b4d5d370 0x5595b4dda770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:57.655 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.649+0000 7f2cb1d9b640 1 -- 172.21.15.16:0/2201484271 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5595b4d579a0 msgr2=0x5595b4dd31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:01:57.655 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.649+0000 7f2cb1d9b640 1 --2- 172.21.15.16:0/2201484271 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5595b4d579a0 0x5595b4dd31f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:57.655 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.649+0000 7f2cb1d9b640 1 -- 172.21.15.16:0/2201484271 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5595b4d5d370 msgr2=0x5595b4dda770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:01:57.655 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.649+0000 7f2cb1d9b640 1 --2- 172.21.15.16:0/2201484271 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5595b4d5d370 0x5595b4dda770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:57.655 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.649+0000 7f2cb1d9b640 1 -- 172.21.15.16:0/2201484271 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595b4a9a620 con 0x5595b4d582e0 2024-09-12T02:01:57.656 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.649+0000 7f2cb2d9d640 1 --2- 172.21.15.16:0/2201484271 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5595b4d5d370 0x5595b4dda770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:01:57.656 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.649+0000 7f2cb1d9b640 1 --2- 172.21.15.16:0/2201484271 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5595b4d582e0 0x5595b4dd3730 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f2ca8005ab0 tx=0x7f2ca8004130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd3cdf9f5a0a11e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:01:57.656 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.649+0000 7f2c99ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:01:57.656 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.649+0000 7f2c99ffb640 1 -- 172.21.15.16:0/2201484271 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ca8035040 con 0x5595b4d582e0 2024-09-12T02:01:57.656 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.649+0000 7f2cbb7d7740 1 -- 172.21.15.16:0/2201484271 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2cac012b30 con 0x5595b4d582e0 2024-09-12T02:01:57.656 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.649+0000 7f2c99ffb640 1 -- 172.21.15.16:0/2201484271 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f2ca8005240 con 0x5595b4d582e0 2024-09-12T02:01:57.656 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.649+0000 7f2c99ffb640 1 -- 172.21.15.16:0/2201484271 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2ca8053050 con 0x5595b4d582e0 2024-09-12T02:01:57.656 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.649+0000 7f2cbb7d7740 10 client.15288 fetch_fsmap learned FSMap version 25 2024-09-12T02:01:57.656 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.649+0000 7f2cbb7d7740 1 -- 172.21.15.16:0/2201484271 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5595b4ddbf90 con 0x5595b4d582e0 2024-09-12T02:01:57.657 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.650+0000 7f2c99ffb640 1 -- 172.21.15.16:0/2201484271 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2cac012b30 con 0x5595b4d582e0 2024-09-12T02:01:57.657 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.650+0000 7f2c99ffb640 1 -- 172.21.15.16:0/2201484271 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f2ca8031720 con 0x5595b4d582e0 2024-09-12T02:01:57.657 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.650+0000 7f2cbb7d7740 10 client.15288 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:01:57.657 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.650+0000 7f2cbb7d7740 10 client.15288 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:01:57.657 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.650+0000 7f2cbb7d7740 20 client.15288 populate_metadata read hostname 'smithi016' 2024-09-12T02:01:57.657 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.650+0000 7f2cbb7d7740 1 --2- 172.21.15.16:0/2201484271 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5595b4ddc940 0x5595b4dded60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:57.657 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.650+0000 7f2cbb7d7740 4 client.15288 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:01:57.657 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.650+0000 7f2cbb7d7740 1 -- 172.21.15.16:0/2201484271 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5595b4cc2d70 con 0x5595b4ddc940 2024-09-12T02:01:57.658 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.650+0000 7f2cb259c640 1 --2- 172.21.15.16:0/2201484271 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5595b4ddc940 0x5595b4dded60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:57.658 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.650+0000 7f2cb259c640 1 --2- 172.21.15.16:0/2201484271 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5595b4ddc940 0x5595b4dded60 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a387d63e47e85cda server_cookie=f106f2673a109024 in_seq=0 out_seq=0 2024-09-12T02:01:57.658 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.650+0000 7f2c99ffb640 10 client.15288 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:01:57.664 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.663+0000 7f2c99ffb640 1 -- 172.21.15.16:0/2201484271 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5595b4cc2d70 con 0x5595b4ddc940 2024-09-12T02:01:57.665 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.663+0000 7f2c99ffb640 10 client.15288 handle_command_reply: tid=0 2024-09-12T02:01:57.665 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.663+0000 7f2cbb7d7740 1 client.15288 shutdown 2024-09-12T02:01:57.665 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.663+0000 7f2cbb7d7740 1 -- 172.21.15.16:0/2201484271 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5595b4d582e0 msgr2=0x5595b4dd3730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:57.665 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.663+0000 7f2cbb7d7740 1 --2- 172.21.15.16:0/2201484271 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5595b4d582e0 0x5595b4dd3730 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f2ca8005ab0 tx=0x7f2ca8004130 comp rx=0 tx=0).stop 2024-09-12T02:01:57.665 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.663+0000 7f2cbb7d7740 1 -- 172.21.15.16:0/2201484271 shutdown_connections 2024-09-12T02:01:57.665 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.663+0000 7f2cbb7d7740 1 --2- 172.21.15.16:0/2201484271 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5595b4ddc940 0x5595b4dded60 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:57.665 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.663+0000 7f2cbb7d7740 1 --2- 172.21.15.16:0/2201484271 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5595b4d5d370 0x5595b4dda770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:57.666 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.663+0000 7f2cbb7d7740 1 --2- 172.21.15.16:0/2201484271 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5595b4d582e0 0x5595b4dd3730 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:57.666 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.663+0000 7f2c99ffb640 0 client.15288 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:01:57.666 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.663+0000 7f2cbb7d7740 1 --2- 172.21.15.16:0/2201484271 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5595b4d579a0 0x5595b4dd31f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:57.666 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.663+0000 7f2cbb7d7740 1 -- 172.21.15.16:0/2201484271 >> 172.21.15.16:0/2201484271 conn(0x5595b4d53010 msgr2=0x5595b4d519a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:01:57.666 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.663+0000 7f2cbb7d7740 1 -- 172.21.15.16:0/2201484271 shutdown_connections 2024-09-12T02:01:57.667 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.663+0000 7f2cbb7d7740 1 -- 172.21.15.16:0/2201484271 wait complete. 2024-09-12T02:01:57.667 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.663+0000 7f2cbb7d7740 20 client.15288 trim_cache size 0 max 16384 2024-09-12T02:01:57.667 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-09-12T02:01:57.668 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-09-12T02:01:57.668 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-09-12T02:01:57.668 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-09-12T02:01:57.669 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.664+0000 7f2cb895f640 1 -- 172.21.15.16:0/2273572012 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f2c9c0a2cb0 msgr2=0x7f2c9c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:57.670 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.664+0000 7f2cb895f640 1 --2- 172.21.15.16:0/2273572012 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f2c9c0a2cb0 0x7f2c9c0a5170 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f2cac009270 tx=0x7f2cac004a40 comp rx=0 tx=0).stop 2024-09-12T02:01:57.670 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.664+0000 7f2cb895f640 1 -- 172.21.15.16:0/2273572012 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cb41025e0 msgr2=0x7f2cb4197e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:57.670 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.664+0000 7f2cb895f640 1 --2- 172.21.15.16:0/2273572012 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cb41025e0 0x7f2cb4197e50 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f2ca400d950 tx=0x7f2ca400de10 comp rx=0 tx=0).stop 2024-09-12T02:01:57.670 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.664+0000 7f2cb895f640 1 -- 172.21.15.16:0/2273572012 shutdown_connections 2024-09-12T02:01:57.670 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.664+0000 7f2cb895f640 1 --2- 172.21.15.16:0/2273572012 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f2c9c0a2cb0 0x7f2c9c0a5170 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:57.670 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.664+0000 7f2cb895f640 1 --2- 172.21.15.16:0/2273572012 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2cb4107840 0x7f2cb419f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:57.671 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.664+0000 7f2cb895f640 1 --2- 172.21.15.16:0/2273572012 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2cb4104f10 0x7f2cb4198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:57.671 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.664+0000 7f2cb895f640 1 --2- 172.21.15.16:0/2273572012 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cb41025e0 0x7f2cb4197e50 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:57.671 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.664+0000 7f2cb895f640 1 -- 172.21.15.16:0/2273572012 >> 172.21.15.16:0/2273572012 conn(0x7f2cb40fc1e0 msgr2=0x7f2cb4102fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:01:57.671 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.664+0000 7f2cb895f640 1 -- 172.21.15.16:0/2273572012 shutdown_connections 2024-09-12T02:01:57.671 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:01:57.664+0000 7f2cb895f640 1 -- 172.21.15.16:0/2273572012 wait complete. 2024-09-12T02:01:57.672 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:01:57 smithi033 ceph-mon[35332]: scrub summary: idle 2024-09-12T02:01:58.015 INFO:teuthology.orchestra.run.smithi016.stdout:{"run_dir":"/var/run/ceph"} 2024-09-12T02:01:58.015 DEBUG:tasks.cephfs.filesystem:_json_asok output { "run_dir": "/var/run/ceph" } 2024-09-12T02:01:58.016 INFO:tasks.quiescer.fs.[cephfs]:Initializing Quiescer with seed 481 2024-09-12T02:01:58.016 DEBUG:tasks.quiescer:Yielding 2024-09-12T02:01:58.016 INFO:teuthology.run_tasks:Running task workunit... 2024-09-12T02:01:58.031 INFO:tasks.workunit:Pulling workunits from ref 086c737fa75c22d521a3932c59b31c5e1d1c9ef4 2024-09-12T02:01:58.032 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-09-12T02:01:58.032 INFO:tasks.workunit:timeout=3h 2024-09-12T02:01:58.032 INFO:tasks.workunit:cleanup=True 2024-09-12T02:01:58.032 DEBUG:teuthology.orchestra.run.smithi016:> stat -- /home/ubuntu/cephtest/mnt.0 2024-09-12T02:01:58.033 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:01:58.051 INFO:teuthology.orchestra.run.smithi016.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-09-12T02:01:58.051 INFO:teuthology.orchestra.run.smithi016.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-09-12T02:01:58.051 INFO:teuthology.orchestra.run.smithi016.stdout:Device: 78h/120d Inode: 1099511628283 Links: 2 2024-09-12T02:01:58.051 INFO:teuthology.orchestra.run.smithi016.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T02:01:58.051 INFO:teuthology.orchestra.run.smithi016.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-09-12T02:01:58.051 INFO:teuthology.orchestra.run.smithi016.stdout:Access: 2024-09-12 02:00:14.164070814 +0000 2024-09-12T02:01:58.051 INFO:teuthology.orchestra.run.smithi016.stdout:Modify: 2024-09-12 02:00:14.164070814 +0000 2024-09-12T02:01:58.051 INFO:teuthology.orchestra.run.smithi016.stdout:Change: 2024-09-12 02:01:35.582050804 +0000 2024-09-12T02:01:58.051 INFO:teuthology.orchestra.run.smithi016.stdout: Birth: 2024-09-12 02:00:14.164070814 +0000 2024-09-12T02:01:58.051 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-09-12T02:01:58.052 DEBUG:teuthology.orchestra.run.smithi016:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-09-12T02:01:58.122 DEBUG:teuthology.orchestra.run.smithi016:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 086c737fa75c22d521a3932c59b31c5e1d1c9ef4 2024-09-12T02:01:58.178 INFO:tasks.workunit.client.0.smithi016.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-09-12T02:01:58.293 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:01:58.527 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:01:58 smithi016 ceph-mon[30947]: pgmap v336: 97 pgs: 97 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-09-12T02:01:58.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:01:58 smithi184 ceph-mon[35593]: pgmap v336: 97 pgs: 97 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-09-12T02:01:58.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:01:58 smithi033 ceph-mon[35332]: pgmap v336: 97 pgs: 97 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-09-12T02:01:58.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:58.744+0000 7f908a505640 1 -- 172.21.15.16:0/4052097537 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9084100b20 msgr2=0x7f9084100f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:58.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:58.744+0000 7f908a505640 1 --2- 172.21.15.16:0/4052097537 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9084100b20 0x7f9084100f00 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f9074009900 tx=0x7f907402a410 comp rx=0 tx=0).stop 2024-09-12T02:01:58.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:58.744+0000 7f908a505640 1 -- 172.21.15.16:0/4052097537 shutdown_connections 2024-09-12T02:01:58.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:58.744+0000 7f908a505640 1 --2- 172.21.15.16:0/4052097537 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f908410e500 0x7f9084110910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:58.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:58.744+0000 7f908a505640 1 --2- 172.21.15.16:0/4052097537 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9084101440 0x7f908410df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:58.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:58.744+0000 7f908a505640 1 --2- 172.21.15.16:0/4052097537 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9084100b20 0x7f9084100f00 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:58.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:58.744+0000 7f908a505640 1 -- 172.21.15.16:0/4052097537 >> 172.21.15.16:0/4052097537 conn(0x7f90840fc7d0 msgr2=0x7f90840febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:01:58.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:58.745+0000 7f908a505640 1 -- 172.21.15.16:0/4052097537 shutdown_connections 2024-09-12T02:01:58.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:58.745+0000 7f908a505640 1 -- 172.21.15.16:0/4052097537 wait complete. 2024-09-12T02:01:58.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:58.745+0000 7f908a505640 1 Processor -- start 2024-09-12T02:01:58.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:58.746+0000 7f908a505640 1 -- start start 2024-09-12T02:01:58.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:58.746+0000 7f908a505640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9084100b20 0x7f908419a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:58.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:58.746+0000 7f908a505640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9084101440 0x7f908419ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:58.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:58.746+0000 7f90837fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9084101440 0x7f908419ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:58.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:58.746+0000 7f9083fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9084100b20 0x7f908419a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:58.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:58.746+0000 7f908a505640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f908410e500 0x7f908419f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:58.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:58.746+0000 7f90837fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9084101440 0x7f908419ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58224/0 (socket says 172.21.15.16:58224) 2024-09-12T02:01:58.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:58.746+0000 7f9083fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9084100b20 0x7f908419a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.16:40908/0 (socket says 172.21.15.16:40908) 2024-09-12T02:01:58.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:58.746+0000 7f90837fe640 1 -- 172.21.15.16:0/3600268665 learned_addr learned my addr 172.21.15.16:0/3600268665 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:01:58.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:58.746+0000 7f908a505640 1 -- 172.21.15.16:0/3600268665 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f90841142b0 con 0x7f9084101440 2024-09-12T02:01:58.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:58.746+0000 7f908a505640 1 -- 172.21.15.16:0/3600268665 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9084114130 con 0x7f9084100b20 2024-09-12T02:01:58.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:58.746+0000 7f908a505640 1 -- 172.21.15.16:0/3600268665 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f9084114430 con 0x7f908410e500 2024-09-12T02:01:58.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:58.747+0000 7f9088aa2640 1 --2- 172.21.15.16:0/3600268665 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f908410e500 0x7f908419f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:58.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:58.747+0000 7f90837fe640 1 -- 172.21.15.16:0/3600268665 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f908410e500 msgr2=0x7f908419f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:58.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:58.747+0000 7f90837fe640 1 --2- 172.21.15.16:0/3600268665 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f908410e500 0x7f908419f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:58.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:58.747+0000 7f90837fe640 1 -- 172.21.15.16:0/3600268665 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9084100b20 msgr2=0x7f908419a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:58.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:58.747+0000 7f90837fe640 1 --2- 172.21.15.16:0/3600268665 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9084100b20 0x7f908419a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:58.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:58.747+0000 7f90837fe640 1 -- 172.21.15.16:0/3600268665 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f908419f700 con 0x7f9084101440 2024-09-12T02:01:58.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:58.747+0000 7f9083fff640 1 --2- 172.21.15.16:0/3600268665 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9084100b20 0x7f908419a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:01:58.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:58.747+0000 7f90837fe640 1 --2- 172.21.15.16:0/3600268665 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9084101440 0x7f908419ac90 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f907800e9c0 tx=0x7f907800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:01:58.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:58.747+0000 7f90817fa640 1 -- 172.21.15.16:0/3600268665 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f907800cd80 con 0x7f9084101440 2024-09-12T02:01:58.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:58.747+0000 7f908a505640 1 -- 172.21.15.16:0/3600268665 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f908419f9f0 con 0x7f9084101440 2024-09-12T02:01:58.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:58.747+0000 7f90817fa640 1 -- 172.21.15.16:0/3600268665 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f90780035f0 con 0x7f9084101440 2024-09-12T02:01:58.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:58.747+0000 7f90817fa640 1 -- 172.21.15.16:0/3600268665 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9078010690 con 0x7f9084101440 2024-09-12T02:01:58.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:58.747+0000 7f908a505640 1 -- 172.21.15.16:0/3600268665 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90841a72d0 con 0x7f9084101440 2024-09-12T02:01:58.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:58.749+0000 7f908a505640 1 -- 172.21.15.16:0/3600268665 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f908406a850 con 0x7f9084101440 2024-09-12T02:01:58.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:58.751+0000 7f90817fa640 1 -- 172.21.15.16:0/3600268665 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f90780037a0 con 0x7f9084101440 2024-09-12T02:01:58.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:58.752+0000 7f90817fa640 1 --2- 172.21.15.16:0/3600268665 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f906c0a2b80 0x7f906c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:58.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:58.752+0000 7f90817fa640 1 -- 172.21.15.16:0/3600268665 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f90780cf410 con 0x7f9084101440 2024-09-12T02:01:58.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:58.752+0000 7f9083fff640 1 --2- 172.21.15.16:0/3600268665 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f906c0a2b80 0x7f906c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:58.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:58.752+0000 7f9083fff640 1 --2- 172.21.15.16:0/3600268665 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f906c0a2b80 0x7f906c0a5040 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f9074000c00 tx=0x7f9074031510 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:01:58.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:58.754+0000 7f90817fa640 1 -- 172.21.15.16:0/3600268665 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f907809c4a0 con 0x7f9084101440 2024-09-12T02:01:58.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:58.996+0000 7f908a505640 1 -- 172.21.15.16:0/3600268665 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9084100f00 con 0x7f9084101440 2024-09-12T02:01:58.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:58.997+0000 7f90817fa640 1 -- 172.21.15.16:0/3600268665 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11911 (secure 0 0 0) 0x7f907809bda0 con 0x7f9084101440 2024-09-12T02:01:58.998 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T02:01:58.998 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":25,"btime":"2024-09-12T02:01:37:192021+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3279123279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3279123279},{"type":"v1","addr":"172.21.15.16:6835","nonce":3279123279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4119452382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4119452382},{"type":"v1","addr":"172.21.15.16:6837","nonce":4119452382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/3263823165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3263823165},{"type":"v1","addr":"172.21.15.16:6839","nonce":3263823165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/1510290732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":1510290732},{"type":"v1","addr":"172.21.15.16:6841","nonce":1510290732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/279674603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":279674603},{"type":"v1","addr":"172.21.15.33:6833","nonce":279674603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3555728028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3555728028},{"type":"v1","addr":"172.21.15.33:6835","nonce":3555728028}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T01:59:56.117608+0000","modified":"2024-09-12T02:01:36.294760+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24482,"mds_1":24481,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6833/1563374106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1563374106},{"type":"v1","addr":"172.21.15.184:6833","nonce":1563374106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6837/1892833868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1892833868},{"type":"v1","addr":"172.21.15.33:6837","nonce":1892833868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/1457679686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1457679686},{"type":"v1","addr":"172.21.15.184:6835","nonce":1457679686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.184:6837/1793393087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":1793393087},{"type":"v1","addr":"172.21.15.184:6837","nonce":1793393087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.184:6839/4203347941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":4203347941},{"type":"v1","addr":"172.21.15.184:6839","nonce":4203347941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.33:6839/4136670204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":4136670204},{"type":"v1","addr":"172.21.15.33:6839","nonce":4136670204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24481,24482,24469]},"id":1}]} 2024-09-12T02:01:58.998 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 25 2024-09-12T02:01:59.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:58.999+0000 7f908a505640 1 -- 172.21.15.16:0/3600268665 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f906c0a2b80 msgr2=0x7f906c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:59.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:58.999+0000 7f908a505640 1 --2- 172.21.15.16:0/3600268665 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f906c0a2b80 0x7f906c0a5040 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f9074000c00 tx=0x7f9074031510 comp rx=0 tx=0).stop 2024-09-12T02:01:59.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:58.999+0000 7f908a505640 1 -- 172.21.15.16:0/3600268665 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9084101440 msgr2=0x7f908419ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:59.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:58.999+0000 7f908a505640 1 --2- 172.21.15.16:0/3600268665 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9084101440 0x7f908419ac90 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f907800e9c0 tx=0x7f907800ee80 comp rx=0 tx=0).stop 2024-09-12T02:01:59.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:58.999+0000 7f908a505640 1 -- 172.21.15.16:0/3600268665 shutdown_connections 2024-09-12T02:01:59.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:58.999+0000 7f908a505640 1 --2- 172.21.15.16:0/3600268665 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f906c0a2b80 0x7f906c0a5040 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:59.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:58.999+0000 7f908a505640 1 --2- 172.21.15.16:0/3600268665 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f908410e500 0x7f908419f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:59.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:58.999+0000 7f908a505640 1 --2- 172.21.15.16:0/3600268665 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9084101440 0x7f908419ac90 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:59.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:58.999+0000 7f908a505640 1 --2- 172.21.15.16:0/3600268665 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9084100b20 0x7f908419a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:59.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:58.999+0000 7f908a505640 1 -- 172.21.15.16:0/3600268665 >> 172.21.15.16:0/3600268665 conn(0x7f90840fc7d0 msgr2=0x7f90840febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:01:59.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:58.999+0000 7f908a505640 1 -- 172.21.15.16:0/3600268665 shutdown_connections 2024-09-12T02:01:59.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:01:58.999+0000 7f908a505640 1 -- 172.21.15.16:0/3600268665 wait complete. 2024-09-12T02:01:59.453 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:01:59 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:01:59.453 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:01:59 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/3600268665' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:01:59.668 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:01:59 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:01:59.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:01:59 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/3600268665' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:01:59.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:01:59 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:01:59.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:01:59 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/3600268665' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:01:59.816 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 25, 'flags': 306, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': True}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-12T01:59:56.117608+0000', 'modified': '2024-09-12T02:01:36.294760+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24482, 'mds_1': 24481, 'mds_2': 24469}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24439': {'gid': 24439, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.184:6833/1563374106', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6832', 'nonce': 1563374106}, {'type': 'v1', 'addr': '172.21.15.184:6833', 'nonce': 1563374106}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24452': {'gid': 24452, 'name': 'h', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.33:6837/1892833868', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6836', 'nonce': 1892833868}, {'type': 'v1', 'addr': '172.21.15.33:6837', 'nonce': 1892833868}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24460': {'gid': 24460, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.184:6835/1457679686', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6834', 'nonce': 1457679686}, {'type': 'v1', 'addr': '172.21.15.184:6835', 'nonce': 1457679686}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24469': {'gid': 24469, 'name': 'c', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.184:6837/1793393087', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6836', 'nonce': 1793393087}, {'type': 'v1', 'addr': '172.21.15.184:6837', 'nonce': 1793393087}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24481': {'gid': 24481, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.184:6839/4203347941', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6838', 'nonce': 4203347941}, {'type': 'v1', 'addr': '172.21.15.184:6839', 'nonce': 4203347941}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24482': {'gid': 24482, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.33:6839/4136670204', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6838', 'nonce': 4136670204}, {'type': 'v1', 'addr': '172.21.15.33:6839', 'nonce': 4136670204}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24482, 'qdb_cluster': [24481, 24482, 24469]} 2024-09-12T02:01:59.816 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-09-12T02:01:59.816 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.c status 2024-09-12T02:02:00.437 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:02:00 smithi016 ceph-mon[30947]: pgmap v337: 97 pgs: 97 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-12T02:02:00.699 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:02:00 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T02:02:00.699 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:02:00 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T02:02:00.699 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:02:00 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T02:02:00.700 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:02:00 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T02:02:00.701 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:02:00 smithi033 ceph-mon[35332]: pgmap v337: 97 pgs: 97 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-12T02:02:00.701 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:02:00 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T02:02:00.701 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:02:00 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T02:02:00.701 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:02:00 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T02:02:00.701 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:02:00 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T02:02:00.701 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:02:00 smithi184 ceph-mon[35593]: pgmap v337: 97 pgs: 97 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-12T02:02:00.702 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:02:00 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T02:02:00.702 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:02:00 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T02:02:00.702 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:02:00 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T02:02:00.702 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:02:00 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T02:02:00.702 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:02:01.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.012+0000 7fee3b6cd640 1 -- 172.21.15.16:0/2933727692 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee341082a0 msgr2=0x7fee3410ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:01.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.012+0000 7fee3b6cd640 1 --2- 172.21.15.16:0/2933727692 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee341082a0 0x7fee3410ec40 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fee2c009cd0 tx=0x7fee2c02a8e0 comp rx=0 tx=0).stop 2024-09-12T02:02:01.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.012+0000 7fee3b6cd640 1 -- 172.21.15.16:0/2933727692 shutdown_connections 2024-09-12T02:02:01.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.012+0000 7fee3b6cd640 1 --2- 172.21.15.16:0/2933727692 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee341082a0 0x7fee3410ec40 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:01.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.012+0000 7fee3b6cd640 1 --2- 172.21.15.16:0/2933727692 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fee341036e0 0x7fee34107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:01.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.012+0000 7fee3b6cd640 1 --2- 172.21.15.16:0/2933727692 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fee34102d30 0x7fee34103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:01.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.012+0000 7fee3b6cd640 1 -- 172.21.15.16:0/2933727692 >> 172.21.15.16:0/2933727692 conn(0x7fee340fc7d0 msgr2=0x7fee340febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:02:01.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.012+0000 7fee3b6cd640 1 -- 172.21.15.16:0/2933727692 shutdown_connections 2024-09-12T02:02:01.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.012+0000 7fee3b6cd640 1 -- 172.21.15.16:0/2933727692 wait complete. 2024-09-12T02:02:01.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.012+0000 7fee3b6cd640 1 Processor -- start 2024-09-12T02:02:01.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.012+0000 7fee3b6cd640 1 -- start start 2024-09-12T02:02:01.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.013+0000 7fee3b6cd640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fee34102d30 0x7fee34198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:01.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.013+0000 7fee3b6cd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee341036e0 0x7fee34198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:01.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.013+0000 7fee3b6cd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fee341082a0 0x7fee3419ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:01.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.013+0000 7fee3b6cd640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fee341120d0 con 0x7fee341036e0 2024-09-12T02:02:01.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.013+0000 7fee3b6cd640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fee34111f50 con 0x7fee341082a0 2024-09-12T02:02:01.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.013+0000 7fee3b6cd640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fee34112250 con 0x7fee34102d30 2024-09-12T02:02:01.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.013+0000 7fee38c68640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee341036e0 0x7fee34198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:01.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.013+0000 7fee39469640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fee34102d30 0x7fee34198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:01.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.013+0000 7fee38c68640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee341036e0 0x7fee34198a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58242/0 (socket says 172.21.15.16:58242) 2024-09-12T02:02:01.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.013+0000 7fee38c68640 1 -- 172.21.15.16:0/885878401 learned_addr learned my addr 172.21.15.16:0/885878401 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:02:01.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.013+0000 7fee39c6a640 1 --2- 172.21.15.16:0/885878401 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fee341082a0 0x7fee3419ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:01.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.013+0000 7fee38c68640 1 -- 172.21.15.16:0/885878401 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fee34102d30 msgr2=0x7fee34198530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:01.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.013+0000 7fee38c68640 1 --2- 172.21.15.16:0/885878401 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fee34102d30 0x7fee34198530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:01.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.013+0000 7fee38c68640 1 -- 172.21.15.16:0/885878401 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fee341082a0 msgr2=0x7fee3419ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:01.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.013+0000 7fee38c68640 1 --2- 172.21.15.16:0/885878401 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fee341082a0 0x7fee3419ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:01.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.013+0000 7fee38c68640 1 -- 172.21.15.16:0/885878401 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee3419d4e0 con 0x7fee341036e0 2024-09-12T02:02:01.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.013+0000 7fee39469640 1 --2- 172.21.15.16:0/885878401 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fee34102d30 0x7fee34198530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:02:01.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.013+0000 7fee38c68640 1 --2- 172.21.15.16:0/885878401 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee341036e0 0x7fee34198a70 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fee2400d740 tx=0x7fee2400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:02:01.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.014+0000 7fee2a7fc640 1 -- 172.21.15.16:0/885878401 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee24017070 con 0x7fee341036e0 2024-09-12T02:02:01.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.014+0000 7fee3b6cd640 1 -- 172.21.15.16:0/885878401 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee3419d7d0 con 0x7fee341036e0 2024-09-12T02:02:01.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.014+0000 7fee2a7fc640 1 -- 172.21.15.16:0/885878401 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee2400de50 con 0x7fee341036e0 2024-09-12T02:02:01.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.014+0000 7fee2a7fc640 1 -- 172.21.15.16:0/885878401 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee2400f6b0 con 0x7fee341036e0 2024-09-12T02:02:01.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.014+0000 7fee3b6cd640 1 -- 172.21.15.16:0/885878401 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee341a50b0 con 0x7fee341036e0 2024-09-12T02:02:01.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.016+0000 7fee2a7fc640 1 -- 172.21.15.16:0/885878401 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fee2401a070 con 0x7fee341036e0 2024-09-12T02:02:01.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.016+0000 7fee3daa6740 1 Processor -- start 2024-09-12T02:02:01.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.016+0000 7fee3daa6740 1 -- start start 2024-09-12T02:02:01.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.017+0000 7fee3daa6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dff0a29890 0x55dff0a29c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:01.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.017+0000 7fee3daa6740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dff0a2a1b0 0x55dff0a38a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:01.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.017+0000 7fee3daa6740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dff0a38fc0 0x55dff0a3b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:01.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.017+0000 7fee3daa6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55dff0990090 con 0x55dff0a29890 2024-09-12T02:02:01.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.017+0000 7fee3daa6740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55dff0937a70 con 0x55dff0a38fc0 2024-09-12T02:02:01.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.017+0000 7fee3daa6740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55dff098c050 con 0x55dff0a2a1b0 2024-09-12T02:02:01.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.017+0000 7fee2a7fc640 1 --2- 172.21.15.16:0/885878401 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fee200a2b80 0x7fee200a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:01.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.017+0000 7fee39c6a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dff0a29890 0x55dff0a29c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:01.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.017+0000 7fee39469640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dff0a2a1b0 0x55dff0a38a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:01.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.017+0000 7fee39469640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dff0a2a1b0 0x55dff0a38a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:54148/0 (socket says 172.21.15.16:54148) 2024-09-12T02:02:01.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.017+0000 7fee39c6a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dff0a29890 0x55dff0a29c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58254/0 (socket says 172.21.15.16:58254) 2024-09-12T02:02:01.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.017+0000 7fee39469640 1 -- 172.21.15.16:0/1157902410 learned_addr learned my addr 172.21.15.16:0/1157902410 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:02:01.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.017+0000 7fee39c6a640 1 --2- 172.21.15.16:0/885878401 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fee200a2b80 0x7fee200a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:01.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.017+0000 7fee2a7fc640 1 -- 172.21.15.16:0/885878401 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fee240ce9f0 con 0x7fee341036e0 2024-09-12T02:02:01.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.017+0000 7fee38c68640 1 --2- 172.21.15.16:0/1157902410 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dff0a38fc0 0x55dff0a3b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:01.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.017+0000 7fee39469640 1 -- 172.21.15.16:0/1157902410 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dff0a38fc0 msgr2=0x55dff0a3b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:01.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.017+0000 7fee39469640 1 --2- 172.21.15.16:0/1157902410 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dff0a38fc0 0x55dff0a3b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:01.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.017+0000 7fee39469640 1 -- 172.21.15.16:0/1157902410 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dff0a29890 msgr2=0x55dff0a29c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:01.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.017+0000 7fee39469640 1 --2- 172.21.15.16:0/1157902410 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dff0a29890 0x55dff0a29c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:01.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.017+0000 7fee39469640 1 -- 172.21.15.16:0/1157902410 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dff0944130 con 0x55dff0a2a1b0 2024-09-12T02:02:01.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.018+0000 7fee39469640 1 --2- 172.21.15.16:0/1157902410 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dff0a2a1b0 0x55dff0a38a80 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fee300104d0 tx=0x7fee30010990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f1905083acee5a6e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:02:01.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.018+0000 7fee39c6a640 1 --2- 172.21.15.16:0/885878401 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fee200a2b80 0x7fee200a5040 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fee2c009430 tx=0x7fee2c0095c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:02:01.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.018+0000 7fee1f7fe640 1 -- 172.21.15.16:0/1157902410 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee30004220 con 0x55dff0a2a1b0 2024-09-12T02:02:01.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.018+0000 7fee1f7fe640 1 -- 172.21.15.16:0/1157902410 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee300043c0 con 0x55dff0a2a1b0 2024-09-12T02:02:01.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.018+0000 7fee1f7fe640 1 -- 172.21.15.16:0/1157902410 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee30011a70 con 0x55dff0a2a1b0 2024-09-12T02:02:01.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.019+0000 7fee3daa6740 1 -- 172.21.15.16:0/1157902410 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dff0a2a1b0 msgr2=0x55dff0a38a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:01.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.019+0000 7fee3daa6740 1 --2- 172.21.15.16:0/1157902410 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dff0a2a1b0 0x55dff0a38a80 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fee300104d0 tx=0x7fee30010990 comp rx=0 tx=0).stop 2024-09-12T02:02:01.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.019+0000 7fee3daa6740 1 -- 172.21.15.16:0/1157902410 shutdown_connections 2024-09-12T02:02:01.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.019+0000 7fee3daa6740 1 --2- 172.21.15.16:0/1157902410 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dff0a38fc0 0x55dff0a3b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:01.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.019+0000 7fee3daa6740 1 --2- 172.21.15.16:0/1157902410 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dff0a2a1b0 0x55dff0a38a80 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:01.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.019+0000 7fee3daa6740 1 --2- 172.21.15.16:0/1157902410 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dff0a29890 0x55dff0a29c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:01.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.019+0000 7fee3daa6740 1 -- 172.21.15.16:0/1157902410 >> 172.21.15.16:0/1157902410 conn(0x55dff0a25f60 msgr2=0x55dff0a26340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:02:01.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.019+0000 7fee3daa6740 1 -- 172.21.15.16:0/1157902410 shutdown_connections 2024-09-12T02:02:01.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.019+0000 7fee3daa6740 1 -- 172.21.15.16:0/1157902410 wait complete. 2024-09-12T02:02:01.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.019+0000 7fee3daa6740 1 -- start start 2024-09-12T02:02:01.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.019+0000 7fee3daa6740 1 Processor -- start 2024-09-12T02:02:01.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.019+0000 7fee3daa6740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dff0a29890 0x55dff0a38800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:01.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.019+0000 7fee3daa6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dff0a2a1b0 0x55dff0a35ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:01.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.019+0000 7fee3daa6740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dff0a38fc0 0x55dff0a360e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:01.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.020+0000 7fee3daa6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55dff0a3bb90 con 0x55dff0a2a1b0 2024-09-12T02:02:01.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.020+0000 7fee3daa6740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55dff094af20 con 0x55dff0a38fc0 2024-09-12T02:02:01.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.020+0000 7fee3daa6740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55dff0a3bd10 con 0x55dff0a29890 2024-09-12T02:02:01.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.020+0000 7fee3daa6740 10 client.0 _ll_register_callbacks cb 0x55dff0914030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:02:01.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.020+0000 7fee38c68640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dff0a2a1b0 0x55dff0a35ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:01.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.020+0000 7fee38c68640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dff0a2a1b0 0x55dff0a35ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58262/0 (socket says 172.21.15.16:58262) 2024-09-12T02:02:01.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.020+0000 7fee38c68640 1 -- 172.21.15.16:0/524792672 learned_addr learned my addr 172.21.15.16:0/524792672 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:02:01.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.020+0000 7fee39469640 1 --2- 172.21.15.16:0/524792672 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dff0a29890 0x55dff0a38800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:01.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.020+0000 7fee39c6a640 1 --2- 172.21.15.16:0/524792672 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dff0a38fc0 0x55dff0a360e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:01.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.020+0000 7fee39469640 1 -- 172.21.15.16:0/524792672 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dff0a38fc0 msgr2=0x55dff0a360e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:01.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.020+0000 7fee39469640 1 --2- 172.21.15.16:0/524792672 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dff0a38fc0 0x55dff0a360e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:01.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.020+0000 7fee39469640 1 -- 172.21.15.16:0/524792672 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dff0a2a1b0 msgr2=0x55dff0a35ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:01.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.020+0000 7fee39469640 1 --2- 172.21.15.16:0/524792672 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dff0a2a1b0 0x55dff0a35ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:01.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.020+0000 7fee39469640 1 -- 172.21.15.16:0/524792672 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dff07012b0 con 0x55dff0a29890 2024-09-12T02:02:01.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.020+0000 7fee38c68640 1 --2- 172.21.15.16:0/524792672 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dff0a2a1b0 0x55dff0a35ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:02:01.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.020+0000 7fee39469640 1 --2- 172.21.15.16:0/524792672 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dff0a29890 0x55dff0a38800 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fee300116d0 tx=0x7fee30003ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b16e16481f3c0c00 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:02:01.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.020+0000 7fee1cff9640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-09-12T02:02:01.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.020+0000 7fee1cff9640 1 -- 172.21.15.16:0/524792672 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee30029060 con 0x55dff0a29890 2024-09-12T02:02:01.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.020+0000 7fee3daa6740 1 -- 172.21.15.16:0/524792672 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee300043c0 con 0x55dff0a29890 2024-09-12T02:02:01.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.020+0000 7fee1cff9640 1 -- 172.21.15.16:0/524792672 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fee30005040 con 0x55dff0a29890 2024-09-12T02:02:01.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.021+0000 7fee1cff9640 1 -- 172.21.15.16:0/524792672 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fee30005240 con 0x55dff0a29890 2024-09-12T02:02:01.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.021+0000 7fee3daa6740 10 client.24952 fetch_fsmap learned FSMap version 25 2024-09-12T02:02:01.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.021+0000 7fee3daa6740 1 -- 172.21.15.16:0/524792672 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55dff0a474c0 con 0x55dff0a29890 2024-09-12T02:02:01.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.021+0000 7fee1cff9640 1 -- 172.21.15.16:0/524792672 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee300043c0 con 0x55dff0a29890 2024-09-12T02:02:01.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.021+0000 7fee1cff9640 1 -- 172.21.15.16:0/524792672 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fee3002e070 con 0x55dff0a29890 2024-09-12T02:02:01.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.021+0000 7fee3daa6740 10 client.24952 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:02:01.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.021+0000 7fee3daa6740 10 client.24952 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-12T02:02:01.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.021+0000 7fee3daa6740 20 client.24952 populate_metadata read hostname 'smithi016' 2024-09-12T02:02:01.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.021+0000 7fee3daa6740 1 --2- 172.21.15.16:0/524792672 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55dff0a48920 0x55dff0a4ab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:01.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.021+0000 7fee3daa6740 4 client.24952 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:02:01.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.021+0000 7fee3daa6740 1 -- 172.21.15.16:0/524792672 --> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dff099d600 con 0x55dff0a48920 2024-09-12T02:02:01.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.021+0000 7fee38c68640 1 --2- 172.21.15.16:0/524792672 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55dff0a48920 0x55dff0a4ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:01.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.022+0000 7fee38c68640 1 --2- 172.21.15.16:0/524792672 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55dff0a48920 0x55dff0a4ab20 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3fc44187dc444e05 server_cookie=dc986cc9c3bec68 in_seq=0 out_seq=0 2024-09-12T02:02:01.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.022+0000 7fee1cff9640 10 client.24952 ms_handle_connect on v2:172.21.15.184:6836/1793393087 2024-09-12T02:02:01.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.022+0000 7fee1cff9640 1 -- 172.21.15.16:0/524792672 <== mds.2 v2:172.21.15.184:6836/1793393087 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55dff099d600 con 0x55dff0a48920 2024-09-12T02:02:01.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.022+0000 7fee1cff9640 10 client.24952 handle_command_reply: tid=0 2024-09-12T02:02:01.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.022+0000 7fee3daa6740 1 client.24952 shutdown 2024-09-12T02:02:01.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.022+0000 7fee3daa6740 1 -- 172.21.15.16:0/524792672 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dff0a29890 msgr2=0x55dff0a38800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:01.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.022+0000 7fee3daa6740 1 --2- 172.21.15.16:0/524792672 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dff0a29890 0x55dff0a38800 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fee300116d0 tx=0x7fee30003ef0 comp rx=0 tx=0).stop 2024-09-12T02:02:01.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.022+0000 7fee3daa6740 1 -- 172.21.15.16:0/524792672 shutdown_connections 2024-09-12T02:02:01.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.022+0000 7fee3daa6740 1 --2- 172.21.15.16:0/524792672 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dff0a38fc0 0x55dff0a360e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:01.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.022+0000 7fee3daa6740 1 --2- 172.21.15.16:0/524792672 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55dff0a48920 0x55dff0a4ab20 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:01.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.022+0000 7fee3daa6740 1 --2- 172.21.15.16:0/524792672 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dff0a2a1b0 0x55dff0a35ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:01.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.022+0000 7fee3daa6740 1 --2- 172.21.15.16:0/524792672 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dff0a29890 0x55dff0a38800 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:01.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.022+0000 7fee1cff9640 0 client.24952 ms_handle_reset on v2:172.21.15.184:6836/1793393087 2024-09-12T02:02:01.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.022+0000 7fee3daa6740 1 -- 172.21.15.16:0/524792672 >> 172.21.15.16:0/524792672 conn(0x55dff0a25f60 msgr2=0x55dff0a2bcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:02:01.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.023+0000 7fee3daa6740 1 -- 172.21.15.16:0/524792672 shutdown_connections 2024-09-12T02:02:01.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.023+0000 7fee3daa6740 1 -- 172.21.15.16:0/524792672 wait complete. 2024-09-12T02:02:01.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.023+0000 7fee3daa6740 20 client.24952 trim_cache size 0 max 16384 2024-09-12T02:02:01.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.041+0000 7fee3daa6740 1 Processor -- start 2024-09-12T02:02:01.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.041+0000 7fee3daa6740 1 -- start start 2024-09-12T02:02:01.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.042+0000 7fee3daa6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dff0a29890 0x55dff0a4ab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:01.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.042+0000 7fee3daa6740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dff0a2a1b0 0x55dff0a32e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:01.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.042+0000 7fee3daa6740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dff0a38fc0 0x55dff0a33340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:01.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.042+0000 7fee3daa6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55dff0a48e60 con 0x55dff0a29890 2024-09-12T02:02:01.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.042+0000 7fee3daa6740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55dff0a49530 con 0x55dff0a2a1b0 2024-09-12T02:02:01.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.042+0000 7fee3daa6740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55dff0a49c00 con 0x55dff0a38fc0 2024-09-12T02:02:01.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.042+0000 7fee39469640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dff0a29890 0x55dff0a4ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:01.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.042+0000 7fee39469640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dff0a29890 0x55dff0a4ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58272/0 (socket says 172.21.15.16:58272) 2024-09-12T02:02:01.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.042+0000 7fee39469640 1 -- 172.21.15.16:0/2698414300 learned_addr learned my addr 172.21.15.16:0/2698414300 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:02:01.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.042+0000 7fee38c68640 1 --2- 172.21.15.16:0/2698414300 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dff0a2a1b0 0x55dff0a32e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:01.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.042+0000 7fee39c6a640 1 --2- 172.21.15.16:0/2698414300 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dff0a38fc0 0x55dff0a33340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:01.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.042+0000 7fee38c68640 1 -- 172.21.15.16:0/2698414300 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dff0a38fc0 msgr2=0x55dff0a33340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:01.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.042+0000 7fee38c68640 1 --2- 172.21.15.16:0/2698414300 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dff0a38fc0 0x55dff0a33340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:01.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.042+0000 7fee38c68640 1 -- 172.21.15.16:0/2698414300 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dff0a29890 msgr2=0x55dff0a4ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:01.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.042+0000 7fee38c68640 1 --2- 172.21.15.16:0/2698414300 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dff0a29890 0x55dff0a4ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:01.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.042+0000 7fee38c68640 1 -- 172.21.15.16:0/2698414300 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dff0a38b70 con 0x55dff0a2a1b0 2024-09-12T02:02:01.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.042+0000 7fee39469640 1 --2- 172.21.15.16:0/2698414300 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dff0a29890 0x55dff0a4ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:02:01.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.042+0000 7fee38c68640 1 --2- 172.21.15.16:0/2698414300 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dff0a2a1b0 0x55dff0a32e00 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fee240cc4f0 tx=0x7fee240cc3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fbddcd064e0b6bde server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:02:01.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.043+0000 7fee1effd640 1 -- 172.21.15.16:0/2698414300 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee24021040 con 0x55dff0a2a1b0 2024-09-12T02:02:01.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.043+0000 7fee1effd640 1 -- 172.21.15.16:0/2698414300 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee240a3ec0 con 0x55dff0a2a1b0 2024-09-12T02:02:01.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.043+0000 7fee1effd640 1 -- 172.21.15.16:0/2698414300 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee240bb310 con 0x55dff0a2a1b0 2024-09-12T02:02:01.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.043+0000 7fee3daa6740 1 -- 172.21.15.16:0/2698414300 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dff0a2a1b0 msgr2=0x55dff0a32e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:01.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.043+0000 7fee3daa6740 1 --2- 172.21.15.16:0/2698414300 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dff0a2a1b0 0x55dff0a32e00 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fee240cc4f0 tx=0x7fee240cc3c0 comp rx=0 tx=0).stop 2024-09-12T02:02:01.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.044+0000 7fee3daa6740 1 -- 172.21.15.16:0/2698414300 shutdown_connections 2024-09-12T02:02:01.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.044+0000 7fee3daa6740 1 --2- 172.21.15.16:0/2698414300 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dff0a38fc0 0x55dff0a33340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:01.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.044+0000 7fee3daa6740 1 --2- 172.21.15.16:0/2698414300 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dff0a2a1b0 0x55dff0a32e00 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:01.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.044+0000 7fee3daa6740 1 --2- 172.21.15.16:0/2698414300 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dff0a29890 0x55dff0a4ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:01.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.044+0000 7fee3daa6740 1 -- 172.21.15.16:0/2698414300 >> 172.21.15.16:0/2698414300 conn(0x55dff0a25f60 msgr2=0x55dff0a31240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:02:01.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.044+0000 7fee3daa6740 1 -- 172.21.15.16:0/2698414300 shutdown_connections 2024-09-12T02:02:01.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.044+0000 7fee3daa6740 1 -- 172.21.15.16:0/2698414300 wait complete. 2024-09-12T02:02:01.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.044+0000 7fee3daa6740 1 -- start start 2024-09-12T02:02:01.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.044+0000 7fee3daa6740 1 Processor -- start 2024-09-12T02:02:01.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.044+0000 7fee3daa6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dff0a29890 0x55dff0a53040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:01.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.044+0000 7fee3daa6740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dff0a2a1b0 0x55dff0a53580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:01.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.044+0000 7fee3daa6740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dff0a38fc0 0x55dff0a4c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:01.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.044+0000 7fee3daa6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55dff0a4a9a0 con 0x55dff0a29890 2024-09-12T02:02:01.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.045+0000 7fee3daa6740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55dff0a4a2d0 con 0x55dff0a2a1b0 2024-09-12T02:02:01.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.045+0000 7fee3daa6740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55dff0a4b070 con 0x55dff0a38fc0 2024-09-12T02:02:01.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.045+0000 7fee3daa6740 10 client.0 _ll_register_callbacks cb 0x7fee2c009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:02:01.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.045+0000 7fee39469640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dff0a29890 0x55dff0a53040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:01.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.045+0000 7fee39469640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dff0a29890 0x55dff0a53040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58286/0 (socket says 172.21.15.16:58286) 2024-09-12T02:02:01.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.045+0000 7fee39469640 1 -- 172.21.15.16:0/2648073293 learned_addr learned my addr 172.21.15.16:0/2648073293 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:02:01.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.045+0000 7fee39c6a640 1 --2- 172.21.15.16:0/2648073293 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dff0a38fc0 0x55dff0a4c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:01.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.045+0000 7fee38c68640 1 --2- 172.21.15.16:0/2648073293 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dff0a2a1b0 0x55dff0a53580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:01.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.045+0000 7fee39469640 1 -- 172.21.15.16:0/2648073293 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dff0a38fc0 msgr2=0x55dff0a4c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:01.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.045+0000 7fee39469640 1 --2- 172.21.15.16:0/2648073293 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dff0a38fc0 0x55dff0a4c370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:01.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.045+0000 7fee39469640 1 -- 172.21.15.16:0/2648073293 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dff0a2a1b0 msgr2=0x55dff0a53580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:01.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.045+0000 7fee39469640 1 --2- 172.21.15.16:0/2648073293 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dff0a2a1b0 0x55dff0a53580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:01.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.045+0000 7fee39469640 1 -- 172.21.15.16:0/2648073293 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dff0a4cba0 con 0x55dff0a29890 2024-09-12T02:02:01.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.045+0000 7fee39c6a640 1 --2- 172.21.15.16:0/2648073293 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dff0a38fc0 0x55dff0a4c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:02:01.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.045+0000 7fee39469640 1 --2- 172.21.15.16:0/2648073293 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dff0a29890 0x55dff0a53040 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fee30011a70 tx=0x7fee30003e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=712dd417aac911a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:02:01.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.045+0000 7fee1cff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:02:01.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.046+0000 7fee1cff9640 1 -- 172.21.15.16:0/2648073293 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee30034070 con 0x55dff0a29890 2024-09-12T02:02:01.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.046+0000 7fee1cff9640 1 -- 172.21.15.16:0/2648073293 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fee300053e0 con 0x55dff0a29890 2024-09-12T02:02:01.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.046+0000 7fee3daa6740 1 -- 172.21.15.16:0/2648073293 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee240a3ec0 con 0x55dff0a29890 2024-09-12T02:02:01.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.046+0000 7fee1cff9640 1 -- 172.21.15.16:0/2648073293 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fee300055e0 con 0x55dff0a29890 2024-09-12T02:02:01.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.046+0000 7fee3daa6740 10 client.15312 fetch_fsmap learned FSMap version 25 2024-09-12T02:02:01.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.046+0000 7fee3daa6740 1 -- 172.21.15.16:0/2648073293 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55dff0a51680 con 0x55dff0a29890 2024-09-12T02:02:01.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.046+0000 7fee1cff9640 1 -- 172.21.15.16:0/2648073293 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee240a3ec0 con 0x55dff0a29890 2024-09-12T02:02:01.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.046+0000 7fee1cff9640 1 -- 172.21.15.16:0/2648073293 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fee3002e040 con 0x55dff0a29890 2024-09-12T02:02:01.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.046+0000 7fee3daa6740 10 client.15312 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:02:01.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.046+0000 7fee3daa6740 10 client.15312 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-12T02:02:01.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.046+0000 7fee3daa6740 20 client.15312 populate_metadata read hostname 'smithi016' 2024-09-12T02:02:01.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.046+0000 7fee3daa6740 1 --2- 172.21.15.16:0/2648073293 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55dff0ab83a0 0x55dff0ab70b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:01.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.046+0000 7fee3daa6740 4 client.15312 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T02:02:01.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.046+0000 7fee3daa6740 1 -- 172.21.15.16:0/2648073293 --> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] -- command(tid 0: {"prefix": "status"}) -- 0x55dff0997b10 con 0x55dff0ab83a0 2024-09-12T02:02:01.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.047+0000 7fee38c68640 1 --2- 172.21.15.16:0/2648073293 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55dff0ab83a0 0x55dff0ab70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:01.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.047+0000 7fee1cff9640 10 client.15312 ms_handle_connect on v2:172.21.15.184:6836/1793393087 2024-09-12T02:02:01.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.047+0000 7fee38c68640 1 --2- 172.21.15.16:0/2648073293 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55dff0ab83a0 0x55dff0ab70b0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fb526b1852a45283 server_cookie=9d5aed9703df5f99 in_seq=0 out_seq=0 2024-09-12T02:02:01.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.047+0000 7fee1cff9640 1 -- 172.21.15.16:0/2648073293 <== mds.2 v2:172.21.15.184:6836/1793393087 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55dff0997b10 con 0x55dff0ab83a0 2024-09-12T02:02:01.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.047+0000 7fee1cff9640 10 client.15312 handle_command_reply: tid=0 2024-09-12T02:02:01.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.047+0000 7fee3daa6740 1 client.15312 shutdown 2024-09-12T02:02:01.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.047+0000 7fee3daa6740 1 -- 172.21.15.16:0/2648073293 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dff0a29890 msgr2=0x55dff0a53040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:01.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.047+0000 7fee3daa6740 1 --2- 172.21.15.16:0/2648073293 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dff0a29890 0x55dff0a53040 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fee30011a70 tx=0x7fee30003e50 comp rx=0 tx=0).stop 2024-09-12T02:02:01.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.048+0000 7fee3daa6740 1 -- 172.21.15.16:0/2648073293 shutdown_connections 2024-09-12T02:02:01.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.048+0000 7fee3daa6740 1 --2- 172.21.15.16:0/2648073293 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dff0a38fc0 0x55dff0a4c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:01.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.048+0000 7fee3daa6740 1 --2- 172.21.15.16:0/2648073293 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dff0a2a1b0 0x55dff0a53580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:01.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.048+0000 7fee3daa6740 1 --2- 172.21.15.16:0/2648073293 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55dff0ab83a0 0x55dff0ab70b0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:01.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.048+0000 7fee3daa6740 1 --2- 172.21.15.16:0/2648073293 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dff0a29890 0x55dff0a53040 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:01.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.048+0000 7fee1cff9640 0 client.15312 ms_handle_reset on v2:172.21.15.184:6836/1793393087 2024-09-12T02:02:01.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.048+0000 7fee3daa6740 1 -- 172.21.15.16:0/2648073293 >> 172.21.15.16:0/2648073293 conn(0x55dff0a25f60 msgr2=0x55dff0a371e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:02:01.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.048+0000 7fee3daa6740 1 -- 172.21.15.16:0/2648073293 shutdown_connections 2024-09-12T02:02:01.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.048+0000 7fee3daa6740 1 -- 172.21.15.16:0/2648073293 wait complete. 2024-09-12T02:02:01.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.048+0000 7fee3daa6740 20 client.15312 trim_cache size 0 max 16384 2024-09-12T02:02:01.075 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T02:02:01.075 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "879f3b56-70a9-11ef-bcea-c7b262605968", 2024-09-12T02:02:01.075 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 2, 2024-09-12T02:02:01.075 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 24469, 2024-09-12T02:02:01.076 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-09-12T02:02:01.076 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-09-12T02:02:01.076 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-09-12T02:02:01.077 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 116.509542318, 2024-09-12T02:02:01.077 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 25, 2024-09-12T02:02:01.077 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 82, 2024-09-12T02:02:01.077 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 82, 2024-09-12T02:02:01.077 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 148.17734329699999 2024-09-12T02:02:01.078 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T02:02:01.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.048+0000 7fee3b6cd640 1 -- 172.21.15.16:0/885878401 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fee200a2b80 msgr2=0x7fee200a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:01.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.048+0000 7fee3b6cd640 1 --2- 172.21.15.16:0/885878401 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fee200a2b80 0x7fee200a5040 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fee2c009430 tx=0x7fee2c0095c0 comp rx=0 tx=0).stop 2024-09-12T02:02:01.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.048+0000 7fee3b6cd640 1 -- 172.21.15.16:0/885878401 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee341036e0 msgr2=0x7fee34198a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:01.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.048+0000 7fee3b6cd640 1 --2- 172.21.15.16:0/885878401 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee341036e0 0x7fee34198a70 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fee2400d740 tx=0x7fee2400dc00 comp rx=0 tx=0).stop 2024-09-12T02:02:01.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.048+0000 7fee3b6cd640 1 -- 172.21.15.16:0/885878401 shutdown_connections 2024-09-12T02:02:01.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.048+0000 7fee3b6cd640 1 --2- 172.21.15.16:0/885878401 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fee200a2b80 0x7fee200a5040 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:01.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.048+0000 7fee3b6cd640 1 --2- 172.21.15.16:0/885878401 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fee341082a0 0x7fee3419ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:01.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.048+0000 7fee3b6cd640 1 --2- 172.21.15.16:0/885878401 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee341036e0 0x7fee34198a70 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:01.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.048+0000 7fee3b6cd640 1 --2- 172.21.15.16:0/885878401 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fee34102d30 0x7fee34198530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:01.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.048+0000 7fee3b6cd640 1 -- 172.21.15.16:0/885878401 >> 172.21.15.16:0/885878401 conn(0x7fee340fc7d0 msgr2=0x7fee34106380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:02:01.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.049+0000 7fee3b6cd640 1 -- 172.21.15.16:0/885878401 shutdown_connections 2024-09-12T02:02:01.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:01.049+0000 7fee3b6cd640 1 -- 172.21.15.16:0/885878401 wait complete. 2024-09-12T02:02:01.815 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.i status 2024-09-12T02:02:02.083 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:02:02.486 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:02:02 smithi016 ceph-mon[30947]: pgmap v338: 97 pgs: 97 active+clean; 656 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 2.3 KiB/s wr, 10 op/s 2024-09-12T02:02:02.487 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:02:02 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T02:02:02.487 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:02:02 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T02:02:02.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:02:02 smithi184 ceph-mon[35593]: pgmap v338: 97 pgs: 97 active+clean; 656 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 2.3 KiB/s wr, 10 op/s 2024-09-12T02:02:02.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:02:02 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T02:02:02.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:02:02 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T02:02:02.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.572+0000 7f4308233640 1 -- 172.21.15.16:0/2598715609 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4300100e20 msgr2=0x7f4300101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:02.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.572+0000 7f4308233640 1 --2- 172.21.15.16:0/2598715609 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4300100e20 0x7f4300101200 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f42f0009900 tx=0x7f42f002a480 comp rx=0 tx=0).stop 2024-09-12T02:02:02.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.572+0000 7f4308233640 1 -- 172.21.15.16:0/2598715609 shutdown_connections 2024-09-12T02:02:02.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.572+0000 7f4308233640 1 --2- 172.21.15.16:0/2598715609 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4300102ef0 0x7f430010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:02.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.572+0000 7f4308233640 1 --2- 172.21.15.16:0/2598715609 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4300102570 0x7f43001029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:02.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.572+0000 7f4308233640 1 --2- 172.21.15.16:0/2598715609 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4300100e20 0x7f4300101200 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:02.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.572+0000 7f4308233640 1 -- 172.21.15.16:0/2598715609 >> 172.21.15.16:0/2598715609 conn(0x7f43000fc7d0 msgr2=0x7f43000febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:02:02.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.572+0000 7f4308233640 1 -- 172.21.15.16:0/2598715609 shutdown_connections 2024-09-12T02:02:02.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.572+0000 7f4308233640 1 -- 172.21.15.16:0/2598715609 wait complete. 2024-09-12T02:02:02.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.573+0000 7f4308233640 1 Processor -- start 2024-09-12T02:02:02.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.573+0000 7f4308233640 1 -- start start 2024-09-12T02:02:02.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.573+0000 7f4308233640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4300100e20 0x7f43000fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:02.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.573+0000 7f4308233640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4300102570 0x7f43000fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:02.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.573+0000 7f4308233640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4300102ef0 0x7f43000fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:02.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.573+0000 7f4308233640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4300110f50 con 0x7f4300100e20 2024-09-12T02:02:02.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.574+0000 7f4305fcf640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4300100e20 0x7f43000fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:02.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.573+0000 7f4308233640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4300110dd0 con 0x7f4300102ef0 2024-09-12T02:02:02.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.574+0000 7f4308233640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f43001110d0 con 0x7f4300102570 2024-09-12T02:02:02.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.574+0000 7f4305fcf640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4300100e20 0x7f43000fcc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43072/0 (socket says 172.21.15.16:43072) 2024-09-12T02:02:02.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.574+0000 7f4305fcf640 1 -- 172.21.15.16:0/1594441618 learned_addr learned my addr 172.21.15.16:0/1594441618 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:02:02.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.574+0000 7f43067d0640 1 --2- 172.21.15.16:0/1594441618 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4300102ef0 0x7f43000fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:02.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.574+0000 7f43057ce640 1 --2- 172.21.15.16:0/1594441618 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4300102570 0x7f43000fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:02.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.574+0000 7f4305fcf640 1 -- 172.21.15.16:0/1594441618 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4300102570 msgr2=0x7f43000fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:02.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.574+0000 7f4305fcf640 1 --2- 172.21.15.16:0/1594441618 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4300102570 0x7f43000fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:02.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.574+0000 7f4305fcf640 1 -- 172.21.15.16:0/1594441618 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4300102ef0 msgr2=0x7f43000fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:02.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.574+0000 7f4305fcf640 1 --2- 172.21.15.16:0/1594441618 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4300102ef0 0x7f43000fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:02.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.574+0000 7f4305fcf640 1 -- 172.21.15.16:0/1594441618 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43001a6950 con 0x7f4300100e20 2024-09-12T02:02:02.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.574+0000 7f4305fcf640 1 --2- 172.21.15.16:0/1594441618 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4300100e20 0x7f43000fcc80 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f42f0006010 tx=0x7f42f0003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:02:02.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.574+0000 7f42feffd640 1 -- 172.21.15.16:0/1594441618 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42f003f070 con 0x7f4300100e20 2024-09-12T02:02:02.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.574+0000 7f4308233640 1 -- 172.21.15.16:0/1594441618 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43001a6be0 con 0x7f4300100e20 2024-09-12T02:02:02.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.574+0000 7f42feffd640 1 -- 172.21.15.16:0/1594441618 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f42f0031e30 con 0x7f4300100e20 2024-09-12T02:02:02.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.574+0000 7f42feffd640 1 -- 172.21.15.16:0/1594441618 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42f0035030 con 0x7f4300100e20 2024-09-12T02:02:02.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.575+0000 7f4308233640 1 -- 172.21.15.16:0/1594441618 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43001a70c0 con 0x7f4300100e20 2024-09-12T02:02:02.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.577+0000 7f42feffd640 1 -- 172.21.15.16:0/1594441618 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f42f0042070 con 0x7f4300100e20 2024-09-12T02:02:02.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.577+0000 7f430a60c740 1 Processor -- start 2024-09-12T02:02:02.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.578+0000 7f430a60c740 1 -- start start 2024-09-12T02:02:02.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.578+0000 7f430a60c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c9c35aa9d0 0x55c9c35aadb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:02.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.578+0000 7f430a60c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c9c35ab2f0 0x55c9c35b5670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:02.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.578+0000 7f430a60c740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c9c35b5bb0 0x55c9c35b7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:02.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.578+0000 7f430a60c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55c9c3511090 con 0x55c9c35ab2f0 2024-09-12T02:02:02.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.578+0000 7f430a60c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55c9c34b8a70 con 0x55c9c35aa9d0 2024-09-12T02:02:02.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.578+0000 7f430a60c740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55c9c350d050 con 0x55c9c35b5bb0 2024-09-12T02:02:02.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.578+0000 7f4305fcf640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c9c35ab2f0 0x55c9c35b5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:02.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.578+0000 7f42feffd640 1 --2- 172.21.15.16:0/1594441618 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f42e80a2b80 0x7f42e80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:02.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.578+0000 7f4305fcf640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c9c35ab2f0 0x55c9c35b5670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43086/0 (socket says 172.21.15.16:43086) 2024-09-12T02:02:02.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.578+0000 7f4305fcf640 1 -- 172.21.15.16:0/322368209 learned_addr learned my addr 172.21.15.16:0/322368209 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:02:02.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.578+0000 7f43067d0640 1 --2- 172.21.15.16:0/322368209 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c9c35aa9d0 0x55c9c35aadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:02.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.578+0000 7f43057ce640 1 --2- 172.21.15.16:0/322368209 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c9c35b5bb0 0x55c9c35b7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:02.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.578+0000 7f43067d0640 1 --2- 172.21.15.16:0/1594441618 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f42e80a2b80 0x7f42e80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:02.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.578+0000 7f42feffd640 1 -- 172.21.15.16:0/1594441618 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f42f00ef990 con 0x7f4300100e20 2024-09-12T02:02:02.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.579+0000 7f4305fcf640 1 -- 172.21.15.16:0/322368209 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c9c35b5bb0 msgr2=0x55c9c35b7fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:02.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.579+0000 7f4305fcf640 1 --2- 172.21.15.16:0/322368209 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c9c35b5bb0 0x55c9c35b7fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:02.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.579+0000 7f4305fcf640 1 -- 172.21.15.16:0/322368209 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c9c35aa9d0 msgr2=0x55c9c35aadb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:02.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.579+0000 7f4305fcf640 1 --2- 172.21.15.16:0/322368209 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c9c35aa9d0 0x55c9c35aadb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:02.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.579+0000 7f4305fcf640 1 -- 172.21.15.16:0/322368209 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9c34c5130 con 0x55c9c35ab2f0 2024-09-12T02:02:02.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.579+0000 7f43067d0640 1 --2- 172.21.15.16:0/322368209 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c9c35aa9d0 0x55c9c35aadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:02:02.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.579+0000 7f43057ce640 1 --2- 172.21.15.16:0/322368209 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c9c35b5bb0 0x55c9c35b7fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:02:02.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.579+0000 7f4305fcf640 1 --2- 172.21.15.16:0/322368209 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c9c35ab2f0 0x55c9c35b5670 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f42f00b8650 tx=0x7f42f0034000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=673dcb6576f36dfc server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:02:02.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.579+0000 7f43067d0640 1 --2- 172.21.15.16:0/1594441618 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f42e80a2b80 0x7f42e80a5040 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f42f8007fd0 tx=0x7f42f8007520 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:02:02.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.579+0000 7f42e7fff640 1 -- 172.21.15.16:0/322368209 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42f0049070 con 0x55c9c35ab2f0 2024-09-12T02:02:02.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.579+0000 7f42e7fff640 1 -- 172.21.15.16:0/322368209 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f42f00bc8f0 con 0x55c9c35ab2f0 2024-09-12T02:02:02.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.579+0000 7f42e7fff640 1 -- 172.21.15.16:0/322368209 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42f003b5f0 con 0x55c9c35ab2f0 2024-09-12T02:02:02.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.580+0000 7f430a60c740 1 -- 172.21.15.16:0/322368209 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c9c35ab2f0 msgr2=0x55c9c35b5670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:02.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.580+0000 7f430a60c740 1 --2- 172.21.15.16:0/322368209 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c9c35ab2f0 0x55c9c35b5670 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f42f00b8650 tx=0x7f42f0034000 comp rx=0 tx=0).stop 2024-09-12T02:02:02.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.580+0000 7f430a60c740 1 -- 172.21.15.16:0/322368209 shutdown_connections 2024-09-12T02:02:02.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.580+0000 7f430a60c740 1 --2- 172.21.15.16:0/322368209 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c9c35b5bb0 0x55c9c35b7fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:02.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.580+0000 7f430a60c740 1 --2- 172.21.15.16:0/322368209 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c9c35ab2f0 0x55c9c35b5670 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:02.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.580+0000 7f430a60c740 1 --2- 172.21.15.16:0/322368209 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c9c35aa9d0 0x55c9c35aadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:02.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.580+0000 7f430a60c740 1 -- 172.21.15.16:0/322368209 >> 172.21.15.16:0/322368209 conn(0x55c9c35a6f60 msgr2=0x55c9c35a7340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:02:02.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.580+0000 7f430a60c740 1 -- 172.21.15.16:0/322368209 shutdown_connections 2024-09-12T02:02:02.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.580+0000 7f430a60c740 1 -- 172.21.15.16:0/322368209 wait complete. 2024-09-12T02:02:02.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.580+0000 7f430a60c740 1 -- start start 2024-09-12T02:02:02.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.580+0000 7f430a60c740 1 Processor -- start 2024-09-12T02:02:02.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.581+0000 7f430a60c740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c9c35aa9d0 0x55c9c35b54a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:02.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.581+0000 7f430a60c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c9c35ab2f0 0x55c9c35b27f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:02.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.581+0000 7f430a60c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c9c35b5bb0 0x55c9c35b2d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:02.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.581+0000 7f430a60c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55c9c35b8540 con 0x55c9c35b5bb0 2024-09-12T02:02:02.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.581+0000 7f430a60c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55c9c34cbf20 con 0x55c9c35ab2f0 2024-09-12T02:02:02.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.581+0000 7f430a60c740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55c9c35b86c0 con 0x55c9c35aa9d0 2024-09-12T02:02:02.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.581+0000 7f430a60c740 10 client.0 _ll_register_callbacks cb 0x55c9c3495030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:02:02.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.581+0000 7f43067d0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c9c35b5bb0 0x55c9c35b2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:02.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.581+0000 7f43067d0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c9c35b5bb0 0x55c9c35b2d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43102/0 (socket says 172.21.15.16:43102) 2024-09-12T02:02:02.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.581+0000 7f43067d0640 1 -- 172.21.15.16:0/1966386713 learned_addr learned my addr 172.21.15.16:0/1966386713 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:02:02.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.581+0000 7f4305fcf640 1 --2- 172.21.15.16:0/1966386713 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c9c35aa9d0 0x55c9c35b54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:02.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.581+0000 7f43057ce640 1 --2- 172.21.15.16:0/1966386713 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c9c35ab2f0 0x55c9c35b27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:02.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.581+0000 7f43067d0640 1 -- 172.21.15.16:0/1966386713 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c9c35aa9d0 msgr2=0x55c9c35b54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:02.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.581+0000 7f43067d0640 1 --2- 172.21.15.16:0/1966386713 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c9c35aa9d0 0x55c9c35b54a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:02.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.581+0000 7f43067d0640 1 -- 172.21.15.16:0/1966386713 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c9c35ab2f0 msgr2=0x55c9c35b27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:02.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.581+0000 7f43067d0640 1 --2- 172.21.15.16:0/1966386713 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c9c35ab2f0 0x55c9c35b27f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:02.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.581+0000 7f43067d0640 1 -- 172.21.15.16:0/1966386713 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c9c32822b0 con 0x55c9c35b5bb0 2024-09-12T02:02:02.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.582+0000 7f43067d0640 1 --2- 172.21.15.16:0/1966386713 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c9c35b5bb0 0x55c9c35b2d30 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f42f8003ed0 tx=0x7f42f8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a71877bf525c12cc server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:02:02.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.582+0000 7f42e57fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:02:02.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.582+0000 7f42e57fa640 1 -- 172.21.15.16:0/1966386713 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42f8025070 con 0x55c9c35b5bb0 2024-09-12T02:02:02.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.582+0000 7f42e57fa640 1 -- 172.21.15.16:0/1966386713 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f42f8004d20 con 0x55c9c35b5bb0 2024-09-12T02:02:02.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.582+0000 7f430a60c740 1 -- 172.21.15.16:0/1966386713 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f42f00bc8f0 con 0x55c9c35b5bb0 2024-09-12T02:02:02.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.582+0000 7f42e57fa640 1 -- 172.21.15.16:0/1966386713 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f42f8030050 con 0x55c9c35b5bb0 2024-09-12T02:02:02.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.582+0000 7f430a60c740 10 client.15324 fetch_fsmap learned FSMap version 25 2024-09-12T02:02:02.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.582+0000 7f430a60c740 1 -- 172.21.15.16:0/1966386713 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c9c35c3e50 con 0x55c9c35b5bb0 2024-09-12T02:02:02.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.582+0000 7f42e57fa640 1 -- 172.21.15.16:0/1966386713 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f42f00bc8f0 con 0x55c9c35b5bb0 2024-09-12T02:02:02.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.583+0000 7f42e57fa640 1 -- 172.21.15.16:0/1966386713 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f42f80051e0 con 0x55c9c35b5bb0 2024-09-12T02:02:02.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.583+0000 7f430a60c740 10 client.15324 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:02:02.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.583+0000 7f430a60c740 10 client.15324 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-12T02:02:02.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.583+0000 7f430a60c740 20 client.15324 populate_metadata read hostname 'smithi016' 2024-09-12T02:02:02.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.583+0000 7f430a60c740 1 --2- 172.21.15.16:0/1966386713 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x55c9c35c6520 0x55c9c35c7990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:02.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.583+0000 7f430a60c740 4 client.15324 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:02:02.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.583+0000 7f430a60c740 1 -- 172.21.15.16:0/1966386713 --> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c9c351e600 con 0x55c9c35c6520 2024-09-12T02:02:02.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.583+0000 7f4305fcf640 1 --2- 172.21.15.16:0/1966386713 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x55c9c35c6520 0x55c9c35c7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:02.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.583+0000 7f4305fcf640 1 --2- 172.21.15.16:0/1966386713 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x55c9c35c6520 0x55c9c35c7990 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8288851697934d71 server_cookie=260ed3dc96b505e in_seq=0 out_seq=0 2024-09-12T02:02:02.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.583+0000 7f42e57fa640 10 client.15324 ms_handle_connect on v2:172.21.15.184:6838/4203347941 2024-09-12T02:02:02.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.584+0000 7f42e57fa640 1 -- 172.21.15.16:0/1966386713 <== mds.1 v2:172.21.15.184:6838/4203347941 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c9c351e600 con 0x55c9c35c6520 2024-09-12T02:02:02.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.584+0000 7f42e57fa640 10 client.15324 handle_command_reply: tid=0 2024-09-12T02:02:02.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.584+0000 7f430a60c740 1 client.15324 shutdown 2024-09-12T02:02:02.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.584+0000 7f430a60c740 1 -- 172.21.15.16:0/1966386713 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c9c35b5bb0 msgr2=0x55c9c35b2d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:02.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.584+0000 7f430a60c740 1 --2- 172.21.15.16:0/1966386713 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c9c35b5bb0 0x55c9c35b2d30 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f42f8003ed0 tx=0x7f42f8004a40 comp rx=0 tx=0).stop 2024-09-12T02:02:02.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.584+0000 7f430a60c740 1 -- 172.21.15.16:0/1966386713 shutdown_connections 2024-09-12T02:02:02.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.584+0000 7f430a60c740 1 --2- 172.21.15.16:0/1966386713 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x55c9c35c6520 0x55c9c35c7990 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:02.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.584+0000 7f430a60c740 1 --2- 172.21.15.16:0/1966386713 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c9c35b5bb0 0x55c9c35b2d30 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:02.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.584+0000 7f430a60c740 1 --2- 172.21.15.16:0/1966386713 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c9c35ab2f0 0x55c9c35b27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:02.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.584+0000 7f42e57fa640 0 client.15324 ms_handle_reset on v2:172.21.15.184:6838/4203347941 2024-09-12T02:02:02.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.584+0000 7f430a60c740 1 --2- 172.21.15.16:0/1966386713 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c9c35aa9d0 0x55c9c35b54a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:02.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.584+0000 7f430a60c740 1 -- 172.21.15.16:0/1966386713 >> 172.21.15.16:0/1966386713 conn(0x55c9c35a6f60 msgr2=0x55c9c35acc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:02:02.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.585+0000 7f430a60c740 1 -- 172.21.15.16:0/1966386713 shutdown_connections 2024-09-12T02:02:02.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.585+0000 7f430a60c740 1 -- 172.21.15.16:0/1966386713 wait complete. 2024-09-12T02:02:02.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.585+0000 7f430a60c740 20 client.15324 trim_cache size 0 max 16384 2024-09-12T02:02:02.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.602+0000 7f430a60c740 1 Processor -- start 2024-09-12T02:02:02.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.603+0000 7f430a60c740 1 -- start start 2024-09-12T02:02:02.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.603+0000 7f430a60c740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c9c35aa9d0 0x55c9c35b20b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:02.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.603+0000 7f430a60c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c9c35ab2f0 0x55c9c35c11a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:02.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.603+0000 7f430a60c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c9c35b5bb0 0x55c9c35c16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:02.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.603+0000 7f430a60c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55c9c35c5cd0 con 0x55c9c35b5bb0 2024-09-12T02:02:02.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.603+0000 7f430a60c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55c9c35c63a0 con 0x55c9c35ab2f0 2024-09-12T02:02:02.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.603+0000 7f430a60c740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55c9c35c6a70 con 0x55c9c35aa9d0 2024-09-12T02:02:02.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.603+0000 7f43067d0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c9c35b5bb0 0x55c9c35c16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:02.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.603+0000 7f43067d0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c9c35b5bb0 0x55c9c35c16e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43114/0 (socket says 172.21.15.16:43114) 2024-09-12T02:02:02.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.603+0000 7f43067d0640 1 -- 172.21.15.16:0/1229225304 learned_addr learned my addr 172.21.15.16:0/1229225304 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:02:02.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.603+0000 7f43057ce640 1 --2- 172.21.15.16:0/1229225304 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c9c35ab2f0 0x55c9c35c11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:02.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.603+0000 7f4305fcf640 1 --2- 172.21.15.16:0/1229225304 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c9c35aa9d0 0x55c9c35b20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:02.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.603+0000 7f43067d0640 1 -- 172.21.15.16:0/1229225304 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c9c35aa9d0 msgr2=0x55c9c35b20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:02.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.603+0000 7f43067d0640 1 --2- 172.21.15.16:0/1229225304 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c9c35aa9d0 0x55c9c35b20b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:02.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.603+0000 7f43067d0640 1 -- 172.21.15.16:0/1229225304 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c9c35ab2f0 msgr2=0x55c9c35c11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:02.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.603+0000 7f43067d0640 1 --2- 172.21.15.16:0/1229225304 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c9c35ab2f0 0x55c9c35c11a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:02.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.603+0000 7f43067d0640 1 -- 172.21.15.16:0/1229225304 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9c35c5ae0 con 0x55c9c35b5bb0 2024-09-12T02:02:02.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.604+0000 7f43067d0640 1 --2- 172.21.15.16:0/1229225304 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c9c35b5bb0 0x55c9c35c16e0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f42f8007a70 tx=0x7f42f8004890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e55edf13d9bca44 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:02:02.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.604+0000 7f42e77fe640 1 -- 172.21.15.16:0/1229225304 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42f8039070 con 0x55c9c35b5bb0 2024-09-12T02:02:02.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.604+0000 7f42e77fe640 1 -- 172.21.15.16:0/1229225304 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f42f800fb70 con 0x55c9c35b5bb0 2024-09-12T02:02:02.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.604+0000 7f42e77fe640 1 -- 172.21.15.16:0/1229225304 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42f8025040 con 0x55c9c35b5bb0 2024-09-12T02:02:02.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.604+0000 7f430a60c740 1 -- 172.21.15.16:0/1229225304 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c9c35b5bb0 msgr2=0x55c9c35c16e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:02.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.604+0000 7f430a60c740 1 --2- 172.21.15.16:0/1229225304 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c9c35b5bb0 0x55c9c35c16e0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f42f8007a70 tx=0x7f42f8004890 comp rx=0 tx=0).stop 2024-09-12T02:02:02.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.605+0000 7f430a60c740 1 -- 172.21.15.16:0/1229225304 shutdown_connections 2024-09-12T02:02:02.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.605+0000 7f430a60c740 1 --2- 172.21.15.16:0/1229225304 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c9c35b5bb0 0x55c9c35c16e0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:02.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.605+0000 7f430a60c740 1 --2- 172.21.15.16:0/1229225304 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c9c35ab2f0 0x55c9c35c11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:02.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.605+0000 7f430a60c740 1 --2- 172.21.15.16:0/1229225304 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c9c35aa9d0 0x55c9c35b20b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:02.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.605+0000 7f430a60c740 1 -- 172.21.15.16:0/1229225304 >> 172.21.15.16:0/1229225304 conn(0x55c9c35a6f60 msgr2=0x55c9c35b27b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:02:02.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.605+0000 7f430a60c740 1 -- 172.21.15.16:0/1229225304 shutdown_connections 2024-09-12T02:02:02.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.605+0000 7f430a60c740 1 -- 172.21.15.16:0/1229225304 wait complete. 2024-09-12T02:02:02.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.605+0000 7f430a60c740 1 -- start start 2024-09-12T02:02:02.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.605+0000 7f430a60c740 1 Processor -- start 2024-09-12T02:02:02.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.605+0000 7f430a60c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c9c35aa9d0 0x55c9c35cacd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:02.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.605+0000 7f430a60c740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c9c35ab2f0 0x55c9c35cb210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:02.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.605+0000 7f430a60c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c9c35b5bb0 0x55c9c362ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:02.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.605+0000 7f430a60c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55c9c35c7810 con 0x55c9c35aa9d0 2024-09-12T02:02:02.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.605+0000 7f430a60c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55c9c35c7140 con 0x55c9c35b5bb0 2024-09-12T02:02:02.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.605+0000 7f430a60c740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55c9c35c7ee0 con 0x55c9c35ab2f0 2024-09-12T02:02:02.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.605+0000 7f430a60c740 10 client.0 _ll_register_callbacks cb 0x7f42f8007c40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:02:02.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.606+0000 7f4305fcf640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c9c35aa9d0 0x55c9c35cacd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:02.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.606+0000 7f4305fcf640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c9c35aa9d0 0x55c9c35cacd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43126/0 (socket says 172.21.15.16:43126) 2024-09-12T02:02:02.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.606+0000 7f4305fcf640 1 -- 172.21.15.16:0/2364768310 learned_addr learned my addr 172.21.15.16:0/2364768310 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:02:02.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.606+0000 7f43067d0640 1 --2- 172.21.15.16:0/2364768310 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c9c35b5bb0 0x55c9c362ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:02.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.606+0000 7f43057ce640 1 --2- 172.21.15.16:0/2364768310 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c9c35ab2f0 0x55c9c35cb210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:02.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.606+0000 7f4305fcf640 1 -- 172.21.15.16:0/2364768310 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c9c35ab2f0 msgr2=0x55c9c35cb210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:02.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.606+0000 7f4305fcf640 1 --2- 172.21.15.16:0/2364768310 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c9c35ab2f0 0x55c9c35cb210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:02.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.606+0000 7f4305fcf640 1 -- 172.21.15.16:0/2364768310 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c9c35b5bb0 msgr2=0x55c9c362ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:02.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.606+0000 7f4305fcf640 1 --2- 172.21.15.16:0/2364768310 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c9c35b5bb0 0x55c9c362ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:02.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.606+0000 7f4305fcf640 1 -- 172.21.15.16:0/2364768310 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c9c35cbac0 con 0x55c9c35aa9d0 2024-09-12T02:02:02.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.606+0000 7f4305fcf640 1 --2- 172.21.15.16:0/2364768310 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c9c35aa9d0 0x55c9c35cacd0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f42f00bc180 tx=0x7f42f002a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ee968c9801b2e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:02:02.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.606+0000 7f42e57fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:02:02.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.606+0000 7f42e57fa640 1 -- 172.21.15.16:0/2364768310 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42f00d4070 con 0x55c9c35aa9d0 2024-09-12T02:02:02.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.606+0000 7f42e57fa640 1 -- 172.21.15.16:0/2364768310 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f42f003b5f0 con 0x55c9c35aa9d0 2024-09-12T02:02:02.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.606+0000 7f430a60c740 1 -- 172.21.15.16:0/2364768310 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f42f800fb70 con 0x55c9c35aa9d0 2024-09-12T02:02:02.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.606+0000 7f42e57fa640 1 -- 172.21.15.16:0/2364768310 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f42f0049040 con 0x55c9c35aa9d0 2024-09-12T02:02:02.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.606+0000 7f430a60c740 10 client.15330 fetch_fsmap learned FSMap version 25 2024-09-12T02:02:02.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.606+0000 7f430a60c740 1 -- 172.21.15.16:0/2364768310 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c9c3630240 con 0x55c9c35aa9d0 2024-09-12T02:02:02.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.607+0000 7f42e57fa640 1 -- 172.21.15.16:0/2364768310 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f42f800fb70 con 0x55c9c35aa9d0 2024-09-12T02:02:02.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.607+0000 7f42e57fa640 1 -- 172.21.15.16:0/2364768310 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f42f00dc080 con 0x55c9c35aa9d0 2024-09-12T02:02:02.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.607+0000 7f430a60c740 10 client.15330 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:02:02.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.607+0000 7f430a60c740 10 client.15330 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-12T02:02:02.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.607+0000 7f430a60c740 20 client.15330 populate_metadata read hostname 'smithi016' 2024-09-12T02:02:02.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.607+0000 7f430a60c740 1 --2- 172.21.15.16:0/2364768310 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x55c9c3634ad0 0x55c9c36337e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:02.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.607+0000 7f430a60c740 4 client.15330 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T02:02:02.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.607+0000 7f430a60c740 1 -- 172.21.15.16:0/2364768310 --> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] -- command(tid 0: {"prefix": "status"}) -- 0x55c9c3518b10 con 0x55c9c3634ad0 2024-09-12T02:02:02.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.607+0000 7f43057ce640 1 --2- 172.21.15.16:0/2364768310 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x55c9c3634ad0 0x55c9c36337e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:02.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.608+0000 7f43057ce640 1 --2- 172.21.15.16:0/2364768310 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x55c9c3634ad0 0x55c9c36337e0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e92d70c927dd366b server_cookie=bcacdd407f32c4de in_seq=0 out_seq=0 2024-09-12T02:02:02.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.608+0000 7f42e57fa640 10 client.15330 ms_handle_connect on v2:172.21.15.184:6838/4203347941 2024-09-12T02:02:02.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.608+0000 7f42e57fa640 1 -- 172.21.15.16:0/2364768310 <== mds.1 v2:172.21.15.184:6838/4203347941 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x55c9c3518b10 con 0x55c9c3634ad0 2024-09-12T02:02:02.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.608+0000 7f42e57fa640 10 client.15330 handle_command_reply: tid=0 2024-09-12T02:02:02.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.608+0000 7f430a60c740 1 client.15330 shutdown 2024-09-12T02:02:02.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.608+0000 7f430a60c740 1 -- 172.21.15.16:0/2364768310 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c9c35aa9d0 msgr2=0x55c9c35cacd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:02.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.608+0000 7f430a60c740 1 --2- 172.21.15.16:0/2364768310 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c9c35aa9d0 0x55c9c35cacd0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f42f00bc180 tx=0x7f42f002a480 comp rx=0 tx=0).stop 2024-09-12T02:02:02.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.608+0000 7f430a60c740 1 -- 172.21.15.16:0/2364768310 shutdown_connections 2024-09-12T02:02:02.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.608+0000 7f430a60c740 1 --2- 172.21.15.16:0/2364768310 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x55c9c3634ad0 0x55c9c36337e0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:02.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.608+0000 7f430a60c740 1 --2- 172.21.15.16:0/2364768310 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c9c35b5bb0 0x55c9c362ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:02.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.608+0000 7f430a60c740 1 --2- 172.21.15.16:0/2364768310 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c9c35ab2f0 0x55c9c35cb210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:02.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.608+0000 7f430a60c740 1 --2- 172.21.15.16:0/2364768310 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c9c35aa9d0 0x55c9c35cacd0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:02.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.608+0000 7f42e57fa640 0 client.15330 ms_handle_reset on v2:172.21.15.184:6838/4203347941 2024-09-12T02:02:02.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.608+0000 7f430a60c740 1 -- 172.21.15.16:0/2364768310 >> 172.21.15.16:0/2364768310 conn(0x55c9c35a6f60 msgr2=0x55c9c35b0c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:02:02.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.608+0000 7f430a60c740 1 -- 172.21.15.16:0/2364768310 shutdown_connections 2024-09-12T02:02:02.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.608+0000 7f430a60c740 1 -- 172.21.15.16:0/2364768310 wait complete. 2024-09-12T02:02:02.647 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T02:02:02.647 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "879f3b56-70a9-11ef-bcea-c7b262605968", 2024-09-12T02:02:02.647 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 1, 2024-09-12T02:02:02.647 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 24481, 2024-09-12T02:02:02.647 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-09-12T02:02:02.647 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-09-12T02:02:02.647 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-09-12T02:02:02.647 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 119.078246021, 2024-09-12T02:02:02.647 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 25, 2024-09-12T02:02:02.647 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 82, 2024-09-12T02:02:02.647 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 82, 2024-09-12T02:02:02.647 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 144.818583967 2024-09-12T02:02:02.647 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T02:02:02.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.608+0000 7f430a60c740 20 client.15330 trim_cache size 0 max 16384 2024-09-12T02:02:02.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.609+0000 7f4308233640 1 -- 172.21.15.16:0/1594441618 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f42e80a2b80 msgr2=0x7f42e80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:02.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.609+0000 7f4308233640 1 --2- 172.21.15.16:0/1594441618 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f42e80a2b80 0x7f42e80a5040 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f42f8007fd0 tx=0x7f42f8007520 comp rx=0 tx=0).stop 2024-09-12T02:02:02.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.609+0000 7f4308233640 1 -- 172.21.15.16:0/1594441618 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4300100e20 msgr2=0x7f43000fcc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:02.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.609+0000 7f4308233640 1 --2- 172.21.15.16:0/1594441618 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4300100e20 0x7f43000fcc80 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f42f0006010 tx=0x7f42f0003c80 comp rx=0 tx=0).stop 2024-09-12T02:02:02.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.609+0000 7f4308233640 1 -- 172.21.15.16:0/1594441618 shutdown_connections 2024-09-12T02:02:02.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.609+0000 7f4308233640 1 --2- 172.21.15.16:0/1594441618 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f42e80a2b80 0x7f42e80a5040 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:02.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.609+0000 7f4308233640 1 --2- 172.21.15.16:0/1594441618 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4300102ef0 0x7f43000fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:02.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.609+0000 7f4308233640 1 --2- 172.21.15.16:0/1594441618 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4300102570 0x7f43000fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:02.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.609+0000 7f4308233640 1 --2- 172.21.15.16:0/1594441618 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4300100e20 0x7f43000fcc80 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:02.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.609+0000 7f4308233640 1 -- 172.21.15.16:0/1594441618 >> 172.21.15.16:0/1594441618 conn(0x7f43000fc7d0 msgr2=0x7f430006a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:02:02.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.609+0000 7f4308233640 1 -- 172.21.15.16:0/1594441618 shutdown_connections 2024-09-12T02:02:02.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:02.609+0000 7f4308233640 1 -- 172.21.15.16:0/1594441618 wait complete. 2024-09-12T02:02:02.668 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:02:02 smithi033 ceph-mon[35332]: pgmap v338: 97 pgs: 97 active+clean; 656 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 2.3 KiB/s wr, 10 op/s 2024-09-12T02:02:02.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:02:02 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T02:02:02.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:02:02 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T02:02:03.388 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.b status 2024-09-12T02:02:03.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:02:03 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:02:03.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:02:03 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:02:03.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:02:03 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T02:02:03.647 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:02:03.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:02:03 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:02:03.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:02:03 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:02:03.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:02:03 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T02:02:03.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:02:03 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:02:03.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:02:03 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:02:03.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:02:03 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T02:02:04.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.119+0000 7f159855b640 1 -- 172.21.15.16:0/3174878650 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1590102dd0 msgr2=0x7f15901031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:04.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.119+0000 7f159855b640 1 --2- 172.21.15.16:0/3174878650 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1590102dd0 0x7f15901031b0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f1580009cd0 tx=0x7f158002a8d0 comp rx=0 tx=0).stop 2024-09-12T02:02:04.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.119+0000 7f159855b640 1 -- 172.21.15.16:0/3174878650 shutdown_connections 2024-09-12T02:02:04.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.119+0000 7f159855b640 1 --2- 172.21.15.16:0/3174878650 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1590108300 0x7f159010ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:04.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.119+0000 7f159855b640 1 --2- 172.21.15.16:0/3174878650 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1590103780 0x7f1590107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:04.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.119+0000 7f159855b640 1 --2- 172.21.15.16:0/3174878650 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1590102dd0 0x7f15901031b0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:04.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.119+0000 7f159855b640 1 -- 172.21.15.16:0/3174878650 >> 172.21.15.16:0/3174878650 conn(0x7f15900fc7d0 msgr2=0x7f15900febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:02:04.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.119+0000 7f159855b640 1 -- 172.21.15.16:0/3174878650 shutdown_connections 2024-09-12T02:02:04.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.119+0000 7f159855b640 1 -- 172.21.15.16:0/3174878650 wait complete. 2024-09-12T02:02:04.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.119+0000 7f159855b640 1 Processor -- start 2024-09-12T02:02:04.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.120+0000 7f159855b640 1 -- start start 2024-09-12T02:02:04.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.120+0000 7f159855b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1590102dd0 0x7f1590100a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:04.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.120+0000 7f159855b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1590103780 0x7f1590100f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:04.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.120+0000 7f159855b640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1590108300 0x7f15901014b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:04.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.120+0000 7f159855b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1590112090 con 0x7f1590103780 2024-09-12T02:02:04.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.120+0000 7f159855b640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1590111f10 con 0x7f1590102dd0 2024-09-12T02:02:04.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.120+0000 7f159855b640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1590112210 con 0x7f1590108300 2024-09-12T02:02:04.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.120+0000 7f1595af6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1590103780 0x7f1590100f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:04.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.120+0000 7f1595af6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1590103780 0x7f1590100f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43146/0 (socket says 172.21.15.16:43146) 2024-09-12T02:02:04.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.120+0000 7f1595af6640 1 -- 172.21.15.16:0/2664943125 learned_addr learned my addr 172.21.15.16:0/2664943125 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:02:04.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.120+0000 7f1596af8640 1 --2- 172.21.15.16:0/2664943125 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1590108300 0x7f15901014b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:04.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.120+0000 7f15962f7640 1 --2- 172.21.15.16:0/2664943125 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1590102dd0 0x7f1590100a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:04.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.120+0000 7f1595af6640 1 -- 172.21.15.16:0/2664943125 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1590108300 msgr2=0x7f15901014b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:04.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.120+0000 7f1595af6640 1 --2- 172.21.15.16:0/2664943125 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1590108300 0x7f15901014b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:04.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.120+0000 7f1595af6640 1 -- 172.21.15.16:0/2664943125 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1590102dd0 msgr2=0x7f1590100a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:04.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.120+0000 7f1595af6640 1 --2- 172.21.15.16:0/2664943125 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1590102dd0 0x7f1590100a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:04.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.120+0000 7f1595af6640 1 -- 172.21.15.16:0/2664943125 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15901a6b30 con 0x7f1590103780 2024-09-12T02:02:04.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.120+0000 7f15962f7640 1 --2- 172.21.15.16:0/2664943125 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1590102dd0 0x7f1590100a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:02:04.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.120+0000 7f1595af6640 1 --2- 172.21.15.16:0/2664943125 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1590103780 0x7f1590100f70 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f158400d950 tx=0x7f158400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:02:04.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.121+0000 7f158f7fe640 1 -- 172.21.15.16:0/2664943125 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1584017070 con 0x7f1590103780 2024-09-12T02:02:04.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.121+0000 7f158f7fe640 1 -- 172.21.15.16:0/2664943125 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1584009c30 con 0x7f1590103780 2024-09-12T02:02:04.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.121+0000 7f159855b640 1 -- 172.21.15.16:0/2664943125 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15901a6e20 con 0x7f1590103780 2024-09-12T02:02:04.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.121+0000 7f158f7fe640 1 -- 172.21.15.16:0/2664943125 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15840047a0 con 0x7f1590103780 2024-09-12T02:02:04.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.121+0000 7f159855b640 1 -- 172.21.15.16:0/2664943125 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15901a7320 con 0x7f1590103780 2024-09-12T02:02:04.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.123+0000 7f158f7fe640 1 -- 172.21.15.16:0/2664943125 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f1584004020 con 0x7f1590103780 2024-09-12T02:02:04.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.123+0000 7f159a934740 1 Processor -- start 2024-09-12T02:02:04.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.123+0000 7f159a934740 1 -- start start 2024-09-12T02:02:04.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.124+0000 7f159a934740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5606083b19d0 0x5606083b1db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:04.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.124+0000 7f159a934740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606083b22f0 0x5606083bc670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:04.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.124+0000 7f159a934740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5606083bcbb0 0x5606083befa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:04.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.124+0000 7f159a934740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x560608318090 con 0x5606083b19d0 2024-09-12T02:02:04.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.124+0000 7f159a934740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5606082bfa70 con 0x5606083b22f0 2024-09-12T02:02:04.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.124+0000 7f159a934740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x560608314050 con 0x5606083bcbb0 2024-09-12T02:02:04.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.124+0000 7f158f7fe640 1 --2- 172.21.15.16:0/2664943125 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f15780a2c00 0x7f15780a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:04.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.124+0000 7f1596af8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5606083b19d0 0x5606083b1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:04.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.124+0000 7f1596af8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5606083b19d0 0x5606083b1db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43160/0 (socket says 172.21.15.16:43160) 2024-09-12T02:02:04.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.124+0000 7f1596af8640 1 -- 172.21.15.16:0/50491148 learned_addr learned my addr 172.21.15.16:0/50491148 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:02:04.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.124+0000 7f1596af8640 1 --2- 172.21.15.16:0/2664943125 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f15780a2c00 0x7f15780a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:04.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.124+0000 7f15962f7640 1 --2- 172.21.15.16:0/50491148 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606083b22f0 0x5606083bc670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:04.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.124+0000 7f1595af6640 1 --2- 172.21.15.16:0/50491148 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5606083bcbb0 0x5606083befa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:04.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.124+0000 7f158f7fe640 1 -- 172.21.15.16:0/2664943125 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f15840cebb0 con 0x7f1590103780 2024-09-12T02:02:04.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.124+0000 7f1596af8640 1 -- 172.21.15.16:0/50491148 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5606083bcbb0 msgr2=0x5606083befa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:04.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.124+0000 7f1596af8640 1 --2- 172.21.15.16:0/50491148 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5606083bcbb0 0x5606083befa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:04.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.124+0000 7f1596af8640 1 -- 172.21.15.16:0/50491148 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606083b22f0 msgr2=0x5606083bc670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:04.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.124+0000 7f1596af8640 1 --2- 172.21.15.16:0/50491148 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606083b22f0 0x5606083bc670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:04.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.124+0000 7f1596af8640 1 -- 172.21.15.16:0/50491148 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5606082cc130 con 0x5606083b19d0 2024-09-12T02:02:04.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.125+0000 7f1596af8640 1 --2- 172.21.15.16:0/50491148 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5606083b19d0 0x5606083b1db0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f1588005d50 tx=0x7f15880113e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7157c0ecab18865e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:02:04.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.125+0000 7f1596af8640 1 --2- 172.21.15.16:0/2664943125 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f15780a2c00 0x7f15780a50c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f15880118e0 tx=0x7f158801c000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:02:04.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.125+0000 7f158cff9640 1 -- 172.21.15.16:0/50491148 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f158800ccc0 con 0x5606083b19d0 2024-09-12T02:02:04.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.125+0000 7f158cff9640 1 -- 172.21.15.16:0/50491148 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f158800a8e0 con 0x5606083b19d0 2024-09-12T02:02:04.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.125+0000 7f158cff9640 1 -- 172.21.15.16:0/50491148 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f158801e670 con 0x5606083b19d0 2024-09-12T02:02:04.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.125+0000 7f159a934740 1 -- 172.21.15.16:0/50491148 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5606083b19d0 msgr2=0x5606083b1db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:04.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.125+0000 7f159a934740 1 --2- 172.21.15.16:0/50491148 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5606083b19d0 0x5606083b1db0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f1588005d50 tx=0x7f15880113e0 comp rx=0 tx=0).stop 2024-09-12T02:02:04.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.125+0000 7f159a934740 1 -- 172.21.15.16:0/50491148 shutdown_connections 2024-09-12T02:02:04.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.125+0000 7f159a934740 1 --2- 172.21.15.16:0/50491148 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5606083bcbb0 0x5606083befa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:04.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.125+0000 7f159a934740 1 --2- 172.21.15.16:0/50491148 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606083b22f0 0x5606083bc670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:04.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.125+0000 7f159a934740 1 --2- 172.21.15.16:0/50491148 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5606083b19d0 0x5606083b1db0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:04.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.125+0000 7f159a934740 1 -- 172.21.15.16:0/50491148 >> 172.21.15.16:0/50491148 conn(0x5606083adf60 msgr2=0x5606083ae340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:02:04.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.126+0000 7f159a934740 1 -- 172.21.15.16:0/50491148 shutdown_connections 2024-09-12T02:02:04.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.126+0000 7f159a934740 1 -- 172.21.15.16:0/50491148 wait complete. 2024-09-12T02:02:04.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.126+0000 7f159a934740 1 -- start start 2024-09-12T02:02:04.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.126+0000 7f159a934740 1 Processor -- start 2024-09-12T02:02:04.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.126+0000 7f159a934740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5606083b19d0 0x5606083bc4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:04.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.126+0000 7f159a934740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5606083b22f0 0x5606083b97f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:04.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.126+0000 7f159a934740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606083bcbb0 0x5606083b9d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:04.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.126+0000 7f159a934740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5606083bf780 con 0x5606083b19d0 2024-09-12T02:02:04.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.126+0000 7f159a934740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5606082d2f20 con 0x5606083bcbb0 2024-09-12T02:02:04.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.126+0000 7f159a934740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5606083bf900 con 0x5606083b22f0 2024-09-12T02:02:04.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.126+0000 7f159a934740 10 client.0 _ll_register_callbacks cb 0x56060829c030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:02:04.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.126+0000 7f15962f7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5606083b19d0 0x5606083bc4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:04.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.126+0000 7f15962f7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5606083b19d0 0x5606083bc4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43170/0 (socket says 172.21.15.16:43170) 2024-09-12T02:02:04.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.126+0000 7f15962f7640 1 -- 172.21.15.16:0/2656706896 learned_addr learned my addr 172.21.15.16:0/2656706896 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:02:04.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.126+0000 7f1596af8640 1 --2- 172.21.15.16:0/2656706896 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606083bcbb0 0x5606083b9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:04.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.127+0000 7f1595af6640 1 --2- 172.21.15.16:0/2656706896 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5606083b22f0 0x5606083b97f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:04.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.127+0000 7f15962f7640 1 -- 172.21.15.16:0/2656706896 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5606083b22f0 msgr2=0x5606083b97f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:04.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.127+0000 7f15962f7640 1 --2- 172.21.15.16:0/2656706896 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5606083b22f0 0x5606083b97f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:04.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.127+0000 7f15962f7640 1 -- 172.21.15.16:0/2656706896 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606083bcbb0 msgr2=0x5606083b9d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:04.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.127+0000 7f15962f7640 1 --2- 172.21.15.16:0/2656706896 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606083bcbb0 0x5606083b9d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:04.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.127+0000 7f15962f7640 1 -- 172.21.15.16:0/2656706896 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5606080892b0 con 0x5606083b19d0 2024-09-12T02:02:04.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.127+0000 7f15962f7640 1 --2- 172.21.15.16:0/2656706896 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5606083b19d0 0x5606083bc4a0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f1580009cd0 tx=0x7f1580031ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbf57a5fb297f1dd server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:02:04.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.127+0000 7f1575ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:02:04.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.127+0000 7f1575ffb640 1 -- 172.21.15.16:0/2656706896 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15800098d0 con 0x5606083b19d0 2024-09-12T02:02:04.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.127+0000 7f1575ffb640 1 -- 172.21.15.16:0/2656706896 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f1580031630 con 0x5606083b19d0 2024-09-12T02:02:04.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.127+0000 7f159a934740 1 -- 172.21.15.16:0/2656706896 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f158800a8e0 con 0x5606083b19d0 2024-09-12T02:02:04.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.127+0000 7f1575ffb640 1 -- 172.21.15.16:0/2656706896 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1580031830 con 0x5606083b19d0 2024-09-12T02:02:04.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.127+0000 7f159a934740 10 client.15342 fetch_fsmap learned FSMap version 25 2024-09-12T02:02:04.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.127+0000 7f159a934740 1 -- 172.21.15.16:0/2656706896 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5606083cb140 con 0x5606083b19d0 2024-09-12T02:02:04.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.128+0000 7f1575ffb640 1 -- 172.21.15.16:0/2656706896 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f158800a8e0 con 0x5606083b19d0 2024-09-12T02:02:04.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.128+0000 7f1575ffb640 1 -- 172.21.15.16:0/2656706896 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f158004e020 con 0x5606083b19d0 2024-09-12T02:02:04.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.128+0000 7f159a934740 10 client.15342 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:02:04.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.128+0000 7f159a934740 10 client.15342 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-12T02:02:04.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.128+0000 7f159a934740 20 client.15342 populate_metadata read hostname 'smithi016' 2024-09-12T02:02:04.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.128+0000 7f159a934740 1 --2- 172.21.15.16:0/2656706896 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5606083cd720 0x5606083ceb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:04.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.128+0000 7f159a934740 4 client.15342 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:02:04.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.128+0000 7f159a934740 1 -- 172.21.15.16:0/2656706896 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560608325600 con 0x5606083cd720 2024-09-12T02:02:04.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.128+0000 7f1595af6640 1 --2- 172.21.15.16:0/2656706896 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5606083cd720 0x5606083ceb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:04.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.129+0000 7f1595af6640 1 --2- 172.21.15.16:0/2656706896 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5606083cd720 0x5606083ceb90 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e66770950886888b server_cookie=2b72f73263bafbf7 in_seq=0 out_seq=0 2024-09-12T02:02:04.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.129+0000 7f1575ffb640 10 client.15342 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:02:04.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.129+0000 7f1575ffb640 1 -- 172.21.15.16:0/2656706896 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560608325600 con 0x5606083cd720 2024-09-12T02:02:04.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.129+0000 7f1575ffb640 10 client.15342 handle_command_reply: tid=0 2024-09-12T02:02:04.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.129+0000 7f159a934740 1 client.15342 shutdown 2024-09-12T02:02:04.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.130+0000 7f159a934740 1 -- 172.21.15.16:0/2656706896 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5606083b19d0 msgr2=0x5606083bc4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:04.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.130+0000 7f159a934740 1 --2- 172.21.15.16:0/2656706896 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5606083b19d0 0x5606083bc4a0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f1580009cd0 tx=0x7f1580031ce0 comp rx=0 tx=0).stop 2024-09-12T02:02:04.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.130+0000 7f159a934740 1 -- 172.21.15.16:0/2656706896 shutdown_connections 2024-09-12T02:02:04.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.130+0000 7f159a934740 1 --2- 172.21.15.16:0/2656706896 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5606083cd720 0x5606083ceb90 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:04.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.130+0000 7f159a934740 1 --2- 172.21.15.16:0/2656706896 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606083bcbb0 0x5606083b9d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:04.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.130+0000 7f159a934740 1 --2- 172.21.15.16:0/2656706896 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5606083b22f0 0x5606083b97f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:04.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.130+0000 7f159a934740 1 --2- 172.21.15.16:0/2656706896 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5606083b19d0 0x5606083bc4a0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:04.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.130+0000 7f159a934740 1 -- 172.21.15.16:0/2656706896 >> 172.21.15.16:0/2656706896 conn(0x5606083adf60 msgr2=0x5606083b3c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:02:04.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.130+0000 7f1575ffb640 0 client.15342 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:02:04.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.130+0000 7f159a934740 1 -- 172.21.15.16:0/2656706896 shutdown_connections 2024-09-12T02:02:04.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.130+0000 7f159a934740 1 -- 172.21.15.16:0/2656706896 wait complete. 2024-09-12T02:02:04.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.130+0000 7f159a934740 20 client.15342 trim_cache size 0 max 16384 2024-09-12T02:02:04.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.148+0000 7f159a934740 1 Processor -- start 2024-09-12T02:02:04.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.148+0000 7f159a934740 1 -- start start 2024-09-12T02:02:04.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.148+0000 7f159a934740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5606083b19d0 0x5606083cd720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:04.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.148+0000 7f159a934740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5606083b22f0 0x5606083a8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:04.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.148+0000 7f159a934740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606083bcbb0 0x5606083b3b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:04.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.148+0000 7f159a934740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5606083cced0 con 0x5606083b22f0 2024-09-12T02:02:04.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.148+0000 7f159a934740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5606083cd5a0 con 0x5606083bcbb0 2024-09-12T02:02:04.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.148+0000 7f159a934740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5606083cdc70 con 0x5606083b19d0 2024-09-12T02:02:04.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.148+0000 7f1595af6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5606083b22f0 0x5606083a8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:04.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.148+0000 7f1595af6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5606083b22f0 0x5606083a8ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43186/0 (socket says 172.21.15.16:43186) 2024-09-12T02:02:04.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.148+0000 7f1595af6640 1 -- 172.21.15.16:0/1757045543 learned_addr learned my addr 172.21.15.16:0/1757045543 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:02:04.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.148+0000 7f15962f7640 1 --2- 172.21.15.16:0/1757045543 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5606083b19d0 0x5606083cd720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:04.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.148+0000 7f1595af6640 1 -- 172.21.15.16:0/1757045543 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5606083b19d0 msgr2=0x5606083cd720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:04.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.148+0000 7f1596af8640 1 --2- 172.21.15.16:0/1757045543 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606083bcbb0 0x5606083b3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:04.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.148+0000 7f1595af6640 1 --2- 172.21.15.16:0/1757045543 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5606083b19d0 0x5606083cd720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:04.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.148+0000 7f1595af6640 1 -- 172.21.15.16:0/1757045543 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606083bcbb0 msgr2=0x5606083b3b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:04.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.148+0000 7f1595af6640 1 --2- 172.21.15.16:0/1757045543 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606083bcbb0 0x5606083b3b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:04.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.148+0000 7f1595af6640 1 -- 172.21.15.16:0/1757045543 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5606083b87e0 con 0x5606083b22f0 2024-09-12T02:02:04.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.148+0000 7f1596af8640 1 --2- 172.21.15.16:0/1757045543 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606083bcbb0 0x5606083b3b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:02:04.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.149+0000 7f1595af6640 1 --2- 172.21.15.16:0/1757045543 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5606083b22f0 0x5606083a8ce0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f158409b440 tx=0x7f158401d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb356e7a74fd6c21 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:02:04.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.149+0000 7f1577fff640 1 -- 172.21.15.16:0/1757045543 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f158409b4b0 con 0x5606083b22f0 2024-09-12T02:02:04.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.149+0000 7f1577fff640 1 -- 172.21.15.16:0/1757045543 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f158409f860 con 0x5606083b22f0 2024-09-12T02:02:04.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.149+0000 7f1577fff640 1 -- 172.21.15.16:0/1757045543 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1584097b20 con 0x5606083b22f0 2024-09-12T02:02:04.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.149+0000 7f159a934740 1 -- 172.21.15.16:0/1757045543 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5606083b22f0 msgr2=0x5606083a8ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:04.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.149+0000 7f159a934740 1 --2- 172.21.15.16:0/1757045543 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5606083b22f0 0x5606083a8ce0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f158409b440 tx=0x7f158401d040 comp rx=0 tx=0).stop 2024-09-12T02:02:04.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.149+0000 7f159a934740 1 -- 172.21.15.16:0/1757045543 shutdown_connections 2024-09-12T02:02:04.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.149+0000 7f159a934740 1 --2- 172.21.15.16:0/1757045543 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606083bcbb0 0x5606083b3b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:04.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.149+0000 7f159a934740 1 --2- 172.21.15.16:0/1757045543 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5606083b22f0 0x5606083a8ce0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:04.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.149+0000 7f159a934740 1 --2- 172.21.15.16:0/1757045543 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5606083b19d0 0x5606083cd720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:04.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.149+0000 7f159a934740 1 -- 172.21.15.16:0/1757045543 >> 172.21.15.16:0/1757045543 conn(0x5606083adf60 msgr2=0x5606083b3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:02:04.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.149+0000 7f159a934740 1 -- 172.21.15.16:0/1757045543 shutdown_connections 2024-09-12T02:02:04.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.150+0000 7f159a934740 1 -- 172.21.15.16:0/1757045543 wait complete. 2024-09-12T02:02:04.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.150+0000 7f159a934740 1 -- start start 2024-09-12T02:02:04.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.150+0000 7f159a934740 1 Processor -- start 2024-09-12T02:02:04.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.150+0000 7f159a934740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5606083b19d0 0x5606083d7080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:04.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.150+0000 7f159a934740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5606083b22f0 0x5606083d75c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:04.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.150+0000 7f159a934740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606083bcbb0 0x5606083b6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:04.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.150+0000 7f159a934740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5606083cea10 con 0x5606083b19d0 2024-09-12T02:02:04.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.150+0000 7f159a934740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5606083ce340 con 0x5606083bcbb0 2024-09-12T02:02:04.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.150+0000 7f159a934740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5606083cf0e0 con 0x5606083b22f0 2024-09-12T02:02:04.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.150+0000 7f159a934740 10 client.0 _ll_register_callbacks cb 0x7f1580009800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:02:04.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.150+0000 7f15962f7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5606083b19d0 0x5606083d7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:04.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.150+0000 7f15962f7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5606083b19d0 0x5606083d7080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43200/0 (socket says 172.21.15.16:43200) 2024-09-12T02:02:04.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.150+0000 7f15962f7640 1 -- 172.21.15.16:0/1829302807 learned_addr learned my addr 172.21.15.16:0/1829302807 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:02:04.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.150+0000 7f1596af8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606083bcbb0 0x5606083b6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:04.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.151+0000 7f1595af6640 1 --2- 172.21.15.16:0/1829302807 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5606083b22f0 0x5606083d75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:04.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.151+0000 7f15962f7640 1 -- 172.21.15.16:0/1829302807 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5606083b22f0 msgr2=0x5606083d75c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:04.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.151+0000 7f15962f7640 1 --2- 172.21.15.16:0/1829302807 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5606083b22f0 0x5606083d75c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:04.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.151+0000 7f15962f7640 1 -- 172.21.15.16:0/1829302807 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606083bcbb0 msgr2=0x5606083b6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:04.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.151+0000 7f15962f7640 1 --2- 172.21.15.16:0/1829302807 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606083bcbb0 0x5606083b6100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:04.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.151+0000 7f15962f7640 1 -- 172.21.15.16:0/1829302807 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5606083b6980 con 0x5606083b19d0 2024-09-12T02:02:04.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.151+0000 7f15962f7640 1 --2- 172.21.15.16:0/1829302807 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5606083b19d0 0x5606083d7080 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f1580005d50 tx=0x7f15800047e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=940166e50011508e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:02:04.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.151+0000 7f1575ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:02:04.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.151+0000 7f1575ffb640 1 -- 172.21.15.16:0/1829302807 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15800098b0 con 0x5606083b19d0 2024-09-12T02:02:04.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.151+0000 7f1575ffb640 1 -- 172.21.15.16:0/1829302807 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f1580031630 con 0x5606083b19d0 2024-09-12T02:02:04.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.151+0000 7f159a934740 1 -- 172.21.15.16:0/1829302807 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f158409f860 con 0x5606083b19d0 2024-09-12T02:02:04.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.151+0000 7f1575ffb640 1 -- 172.21.15.16:0/1829302807 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1580004d50 con 0x5606083b19d0 2024-09-12T02:02:04.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.151+0000 7f159a934740 10 client.15348 fetch_fsmap learned FSMap version 25 2024-09-12T02:02:04.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.151+0000 7f159a934740 1 -- 172.21.15.16:0/1829302807 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5606083d6880 con 0x5606083b19d0 2024-09-12T02:02:04.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.152+0000 7f1575ffb640 1 -- 172.21.15.16:0/1829302807 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f158409f860 con 0x5606083b19d0 2024-09-12T02:02:04.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.152+0000 7f1575ffb640 1 -- 172.21.15.16:0/1829302807 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f158004e030 con 0x5606083b19d0 2024-09-12T02:02:04.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.152+0000 7f159a934740 10 client.15348 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:02:04.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.152+0000 7f159a934740 10 client.15348 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-12T02:02:04.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.152+0000 7f159a934740 20 client.15348 populate_metadata read hostname 'smithi016' 2024-09-12T02:02:04.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.152+0000 7f159a934740 1 --2- 172.21.15.16:0/1829302807 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x56060843c4c0 0x56060843b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:04.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.152+0000 7f159a934740 4 client.15348 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T02:02:04.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.152+0000 7f159a934740 1 -- 172.21.15.16:0/1829302807 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "status"}) -- 0x56060831fb10 con 0x56060843c4c0 2024-09-12T02:02:04.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.152+0000 7f1595af6640 1 --2- 172.21.15.16:0/1829302807 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x56060843c4c0 0x56060843b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:04.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.153+0000 7f1595af6640 1 --2- 172.21.15.16:0/1829302807 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x56060843c4c0 0x56060843b1d0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=35257674887294fd server_cookie=792f371430d72dfa in_seq=0 out_seq=0 2024-09-12T02:02:04.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.153+0000 7f1575ffb640 10 client.15348 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:02:04.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.153+0000 7f1575ffb640 1 -- 172.21.15.16:0/1829302807 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x56060831fb10 con 0x56060843c4c0 2024-09-12T02:02:04.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.153+0000 7f1575ffb640 10 client.15348 handle_command_reply: tid=0 2024-09-12T02:02:04.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.153+0000 7f159a934740 1 client.15348 shutdown 2024-09-12T02:02:04.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.153+0000 7f159a934740 1 -- 172.21.15.16:0/1829302807 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5606083b19d0 msgr2=0x5606083d7080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:04.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.153+0000 7f159a934740 1 --2- 172.21.15.16:0/1829302807 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5606083b19d0 0x5606083d7080 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f1580005d50 tx=0x7f15800047e0 comp rx=0 tx=0).stop 2024-09-12T02:02:04.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.153+0000 7f159a934740 1 -- 172.21.15.16:0/1829302807 shutdown_connections 2024-09-12T02:02:04.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.153+0000 7f159a934740 1 --2- 172.21.15.16:0/1829302807 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x56060843c4c0 0x56060843b1d0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:04.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.153+0000 7f159a934740 1 --2- 172.21.15.16:0/1829302807 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606083bcbb0 0x5606083b6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:04.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.153+0000 7f159a934740 1 --2- 172.21.15.16:0/1829302807 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5606083b22f0 0x5606083d75c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:04.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.153+0000 7f1575ffb640 0 client.15348 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:02:04.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.153+0000 7f159a934740 1 --2- 172.21.15.16:0/1829302807 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5606083b19d0 0x5606083d7080 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:04.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.153+0000 7f159a934740 1 -- 172.21.15.16:0/1829302807 >> 172.21.15.16:0/1829302807 conn(0x5606083adf60 msgr2=0x5606083b3c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:02:04.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.153+0000 7f159a934740 1 -- 172.21.15.16:0/1829302807 shutdown_connections 2024-09-12T02:02:04.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.153+0000 7f159a934740 1 -- 172.21.15.16:0/1829302807 wait complete. 2024-09-12T02:02:04.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.153+0000 7f159a934740 20 client.15348 trim_cache size 0 max 16384 2024-09-12T02:02:04.161 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T02:02:04.161 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "879f3b56-70a9-11ef-bcea-c7b262605968", 2024-09-12T02:02:04.161 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 0, 2024-09-12T02:02:04.161 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 24482, 2024-09-12T02:02:04.161 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-09-12T02:02:04.161 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-09-12T02:02:04.161 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-09-12T02:02:04.161 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 127.660893681, 2024-09-12T02:02:04.161 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 25, 2024-09-12T02:02:04.161 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 82, 2024-09-12T02:02:04.162 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 82, 2024-09-12T02:02:04.162 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 137.91162918800001 2024-09-12T02:02:04.162 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T02:02:04.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.154+0000 7f159855b640 1 -- 172.21.15.16:0/2664943125 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f15780a2c00 msgr2=0x7f15780a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:04.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.154+0000 7f159855b640 1 --2- 172.21.15.16:0/2664943125 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f15780a2c00 0x7f15780a50c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f15880118e0 tx=0x7f158801c000 comp rx=0 tx=0).stop 2024-09-12T02:02:04.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.154+0000 7f159855b640 1 -- 172.21.15.16:0/2664943125 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1590103780 msgr2=0x7f1590100f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:04.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.154+0000 7f159855b640 1 --2- 172.21.15.16:0/2664943125 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1590103780 0x7f1590100f70 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f158400d950 tx=0x7f158400de10 comp rx=0 tx=0).stop 2024-09-12T02:02:04.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.154+0000 7f159855b640 1 -- 172.21.15.16:0/2664943125 shutdown_connections 2024-09-12T02:02:04.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.154+0000 7f159855b640 1 --2- 172.21.15.16:0/2664943125 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f15780a2c00 0x7f15780a50c0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:04.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.154+0000 7f159855b640 1 --2- 172.21.15.16:0/2664943125 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1590108300 0x7f15901014b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:04.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.154+0000 7f159855b640 1 --2- 172.21.15.16:0/2664943125 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1590103780 0x7f1590100f70 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:04.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.154+0000 7f159855b640 1 --2- 172.21.15.16:0/2664943125 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1590102dd0 0x7f1590100a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:04.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.154+0000 7f159855b640 1 -- 172.21.15.16:0/2664943125 >> 172.21.15.16:0/2664943125 conn(0x7f15900fc7d0 msgr2=0x7f1590105ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:02:04.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.154+0000 7f159855b640 1 -- 172.21.15.16:0/2664943125 shutdown_connections 2024-09-12T02:02:04.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:04.154+0000 7f159855b640 1 -- 172.21.15.16:0/2664943125 wait complete. 2024-09-12T02:02:04.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:02:04 smithi016 ceph-mon[30947]: pgmap v339: 97 pgs: 97 active+clean; 656 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 2.6 KiB/s wr, 9 op/s 2024-09-12T02:02:04.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:02:04 smithi184 ceph-mon[35593]: pgmap v339: 97 pgs: 97 active+clean; 656 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 2.6 KiB/s wr, 9 op/s 2024-09-12T02:02:04.668 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:02:04 smithi033 ceph-mon[35332]: pgmap v339: 97 pgs: 97 active+clean; 656 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 2.6 KiB/s wr, 9 op/s 2024-09-12T02:02:04.885 INFO:tasks.quiescer:Ready to start quiesce thrashing; initial delay: 15 sec 2024-09-12T02:02:06.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:02:06 smithi184 ceph-mon[35593]: pgmap v340: 97 pgs: 97 active+clean; 656 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 2.6 KiB/s wr, 10 op/s 2024-09-12T02:02:06.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:02:06 smithi033 ceph-mon[35332]: pgmap v340: 97 pgs: 97 active+clean; 656 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 2.6 KiB/s wr, 10 op/s 2024-09-12T02:02:06.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:02:06 smithi016 ceph-mon[30947]: pgmap v340: 97 pgs: 97 active+clean; 656 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 2.6 KiB/s wr, 10 op/s 2024-09-12T02:02:08.629 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:02:08 smithi184 ceph-mon[35593]: pgmap v341: 97 pgs: 97 active+clean; 656 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 2.6 KiB/s wr, 10 op/s 2024-09-12T02:02:08.668 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:02:08 smithi033 ceph-mon[35332]: pgmap v341: 97 pgs: 97 active+clean; 656 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 2.6 KiB/s wr, 10 op/s 2024-09-12T02:02:08.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:02:08 smithi016 ceph-mon[30947]: pgmap v341: 97 pgs: 97 active+clean; 656 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 2.6 KiB/s wr, 10 op/s 2024-09-12T02:02:10.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:02:10 smithi184 ceph-mon[35593]: pgmap v342: 97 pgs: 97 active+clean; 656 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 2.6 KiB/s wr, 9 op/s 2024-09-12T02:02:10.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:02:10 smithi033 ceph-mon[35332]: pgmap v342: 97 pgs: 97 active+clean; 656 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 2.6 KiB/s wr, 9 op/s 2024-09-12T02:02:10.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:02:10 smithi016 ceph-mon[30947]: pgmap v342: 97 pgs: 97 active+clean; 656 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 2.6 KiB/s wr, 9 op/s 2024-09-12T02:02:10.765 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.764+0000 7fc9b22e6640 1 -- 172.21.15.16:0/1210802807 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9ac102d50 msgr2=0x7fc9ac1031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:10.765 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.764+0000 7fc9b22e6640 1 --2- 172.21.15.16:0/1210802807 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9ac102d50 0x7fc9ac1031d0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fc9a0009900 tx=0x7fc9a002a410 comp rx=0 tx=0).stop 2024-09-12T02:02:10.765 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.764+0000 7fc9b22e6640 1 -- 172.21.15.16:0/1210802807 shutdown_connections 2024-09-12T02:02:10.765 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.764+0000 7fc9b22e6640 1 --2- 172.21.15.16:0/1210802807 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc9ac103710 0x7fc9ac109fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:10.765 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.764+0000 7fc9b22e6640 1 --2- 172.21.15.16:0/1210802807 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9ac102d50 0x7fc9ac1031d0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:10.765 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.764+0000 7fc9b22e6640 1 --2- 172.21.15.16:0/1210802807 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc9ac101b60 0x7fc9ac101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:10.765 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.764+0000 7fc9b22e6640 1 -- 172.21.15.16:0/1210802807 >> 172.21.15.16:0/1210802807 conn(0x7fc9ac0fd2e0 msgr2=0x7fc9ac0ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:02:10.766 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.764+0000 7fc9b22e6640 1 -- 172.21.15.16:0/1210802807 shutdown_connections 2024-09-12T02:02:10.766 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.764+0000 7fc9b22e6640 1 -- 172.21.15.16:0/1210802807 wait complete. 2024-09-12T02:02:10.766 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.765+0000 7fc9b22e6640 1 Processor -- start 2024-09-12T02:02:10.766 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.765+0000 7fc9b22e6640 1 -- start start 2024-09-12T02:02:10.768 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.765+0000 7fc9b22e6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9ac101b60 0x7fc9ac197dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:10.768 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.766+0000 7fc9b22e6640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc9ac102d50 0x7fc9ac198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:10.768 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.766+0000 7fc9b22e6640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc9ac103710 0x7fc9ac19f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:10.768 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.766+0000 7fc9b22e6640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc9ac10d7e0 con 0x7fc9ac101b60 2024-09-12T02:02:10.768 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.766+0000 7fc9b22e6640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc9ac10d660 con 0x7fc9ac102d50 2024-09-12T02:02:10.768 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.766+0000 7fc9b22e6640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc9ac10d960 con 0x7fc9ac103710 2024-09-12T02:02:10.768 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.766+0000 7fc9abfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9ac101b60 0x7fc9ac197dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:10.768 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.766+0000 7fc9abfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9ac101b60 0x7fc9ac197dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43216/0 (socket says 172.21.15.16:43216) 2024-09-12T02:02:10.768 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.766+0000 7fc9abfff640 1 -- 172.21.15.16:0/1115426900 learned_addr learned my addr 172.21.15.16:0/1115426900 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:02:10.768 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.766+0000 7fc9ab7fe640 1 --2- 172.21.15.16:0/1115426900 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc9ac102d50 0x7fc9ac198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:10.769 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.766+0000 7fc9b0883640 1 --2- 172.21.15.16:0/1115426900 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc9ac103710 0x7fc9ac19f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:10.769 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.766+0000 7fc9abfff640 1 -- 172.21.15.16:0/1115426900 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc9ac103710 msgr2=0x7fc9ac19f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:10.769 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.766+0000 7fc9abfff640 1 --2- 172.21.15.16:0/1115426900 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc9ac103710 0x7fc9ac19f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:10.769 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.766+0000 7fc9abfff640 1 -- 172.21.15.16:0/1115426900 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc9ac102d50 msgr2=0x7fc9ac198310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:10.769 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.766+0000 7fc9abfff640 1 --2- 172.21.15.16:0/1115426900 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc9ac102d50 0x7fc9ac198310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:10.769 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.766+0000 7fc9abfff640 1 -- 172.21.15.16:0/1115426900 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9ac19f9f0 con 0x7fc9ac101b60 2024-09-12T02:02:10.769 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.766+0000 7fc9abfff640 1 --2- 172.21.15.16:0/1115426900 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9ac101b60 0x7fc9ac197dd0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fc99c00d950 tx=0x7fc99c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:02:10.769 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.766+0000 7fc9a97fa640 1 -- 172.21.15.16:0/1115426900 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc99c017020 con 0x7fc9ac101b60 2024-09-12T02:02:10.769 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.767+0000 7fc9a97fa640 1 -- 172.21.15.16:0/1115426900 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc99c009c30 con 0x7fc9ac101b60 2024-09-12T02:02:10.769 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.767+0000 7fc9b22e6640 1 -- 172.21.15.16:0/1115426900 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9ac19fce0 con 0x7fc9ac101b60 2024-09-12T02:02:10.769 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.767+0000 7fc9a97fa640 1 -- 172.21.15.16:0/1115426900 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc99c00f740 con 0x7fc9ac101b60 2024-09-12T02:02:10.769 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.767+0000 7fc9b22e6640 1 -- 172.21.15.16:0/1115426900 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9ac1a01b0 con 0x7fc9ac101b60 2024-09-12T02:02:10.776 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.769+0000 7fc9a97fa640 1 -- 172.21.15.16:0/1115426900 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fc99c01a020 con 0x7fc9ac101b60 2024-09-12T02:02:10.776 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.769+0000 7fc9b5082740 1 Processor -- start 2024-09-12T02:02:10.776 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.769+0000 7fc9b5082740 1 -- start start 2024-09-12T02:02:10.777 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.770+0000 7fc9b5082740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bfe73009a0 0x55bfe7300da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:10.777 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.770+0000 7fc9b5082740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bfe73012e0 0x55bfe7305da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:10.777 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.770+0000 7fc9b5082740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bfe7306370 0x55bfe7308830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:10.777 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.770+0000 7fc9b5082740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55bfe72642f0 con 0x55bfe73012e0 2024-09-12T02:02:10.777 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.770+0000 7fc9b5082740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55bfe720c270 con 0x55bfe73009a0 2024-09-12T02:02:10.777 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.770+0000 7fc9b5082740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55bfe72602b0 con 0x55bfe7306370 2024-09-12T02:02:10.777 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.770+0000 7fc9abfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bfe73012e0 0x55bfe7305da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:10.777 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.770+0000 7fc9abfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bfe73012e0 0x55bfe7305da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43220/0 (socket says 172.21.15.16:43220) 2024-09-12T02:02:10.777 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.770+0000 7fc9abfff640 1 -- 172.21.15.16:0/276795510 learned_addr learned my addr 172.21.15.16:0/276795510 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:02:10.777 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.770+0000 7fc9ab7fe640 1 --2- 172.21.15.16:0/276795510 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bfe7306370 0x55bfe7308830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:10.777 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.770+0000 7fc9b0883640 1 --2- 172.21.15.16:0/276795510 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bfe73009a0 0x55bfe7300da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:10.777 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.770+0000 7fc9a97fa640 1 --2- 172.21.15.16:0/1115426900 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc9940a2c30 0x7fc9940a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:10.778 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.770+0000 7fc9b0883640 1 --2- 172.21.15.16:0/1115426900 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc9940a2c30 0x7fc9940a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:10.778 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.770+0000 7fc9abfff640 1 -- 172.21.15.16:0/276795510 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bfe7306370 msgr2=0x55bfe7308830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:10.778 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.770+0000 7fc9abfff640 1 --2- 172.21.15.16:0/276795510 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bfe7306370 0x55bfe7308830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:10.778 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.770+0000 7fc9abfff640 1 -- 172.21.15.16:0/276795510 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bfe73009a0 msgr2=0x55bfe7300da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:10.778 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.770+0000 7fc9abfff640 1 --2- 172.21.15.16:0/276795510 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bfe73009a0 0x55bfe7300da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:10.778 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.770+0000 7fc9abfff640 1 -- 172.21.15.16:0/276795510 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bfe720b340 con 0x55bfe73012e0 2024-09-12T02:02:10.778 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.770+0000 7fc9ab7fe640 1 --2- 172.21.15.16:0/276795510 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bfe7306370 0x55bfe7308830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:02:10.778 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.770+0000 7fc9a97fa640 1 -- 172.21.15.16:0/1115426900 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fc99c0ce9b0 con 0x7fc9ac101b60 2024-09-12T02:02:10.778 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.771+0000 7fc9abfff640 1 --2- 172.21.15.16:0/276795510 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bfe73012e0 0x55bfe7305da0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fc99c097be0 tx=0x7fc99c010e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=503d82a8aba5571c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:02:10.778 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.771+0000 7fc9b0883640 1 --2- 172.21.15.16:0/1115426900 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc9940a2c30 0x7fc9940a50f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fc9a4009fd0 tx=0x7fc9a4004ab0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:02:10.778 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.771+0000 7fc9927fc640 1 -- 172.21.15.16:0/276795510 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc99c009de0 con 0x55bfe73012e0 2024-09-12T02:02:10.778 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.771+0000 7fc9927fc640 1 -- 172.21.15.16:0/276795510 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc99c0b9e50 con 0x55bfe73012e0 2024-09-12T02:02:10.778 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.771+0000 7fc9927fc640 1 -- 172.21.15.16:0/276795510 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc99c012210 con 0x55bfe73012e0 2024-09-12T02:02:10.779 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.772+0000 7fc9b5082740 1 -- 172.21.15.16:0/276795510 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bfe73012e0 msgr2=0x55bfe7305da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:10.779 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.772+0000 7fc9b5082740 1 --2- 172.21.15.16:0/276795510 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bfe73012e0 0x55bfe7305da0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fc99c097be0 tx=0x7fc99c010e10 comp rx=0 tx=0).stop 2024-09-12T02:02:10.779 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.772+0000 7fc9b5082740 1 -- 172.21.15.16:0/276795510 shutdown_connections 2024-09-12T02:02:10.779 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.772+0000 7fc9b5082740 1 --2- 172.21.15.16:0/276795510 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bfe7306370 0x55bfe7308830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:10.779 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.772+0000 7fc9b5082740 1 --2- 172.21.15.16:0/276795510 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bfe73012e0 0x55bfe7305da0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:10.779 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.772+0000 7fc9b5082740 1 --2- 172.21.15.16:0/276795510 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bfe73009a0 0x55bfe7300da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:10.779 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.772+0000 7fc9b5082740 1 -- 172.21.15.16:0/276795510 >> 172.21.15.16:0/276795510 conn(0x55bfe72fc010 msgr2=0x55bfe72fc410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:02:10.779 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.772+0000 7fc9b5082740 1 -- 172.21.15.16:0/276795510 shutdown_connections 2024-09-12T02:02:10.779 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.772+0000 7fc9b5082740 1 -- 172.21.15.16:0/276795510 wait complete. 2024-09-12T02:02:10.779 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.772+0000 7fc9b5082740 1 -- start start 2024-09-12T02:02:10.779 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.772+0000 7fc9b5082740 1 Processor -- start 2024-09-12T02:02:10.779 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.773+0000 7fc9b5082740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bfe73009a0 0x55bfe730ab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:10.779 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.773+0000 7fc9b5082740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bfe73012e0 0x55bfe730b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:10.780 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.773+0000 7fc9b5082740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bfe7306370 0x55bfe73120a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:10.780 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.773+0000 7fc9b5082740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55bfe73092e0 con 0x55bfe73012e0 2024-09-12T02:02:10.780 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.773+0000 7fc9b5082740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55bfe71ed290 con 0x55bfe7306370 2024-09-12T02:02:10.780 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.773+0000 7fc9b5082740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55bfe7309460 con 0x55bfe73009a0 2024-09-12T02:02:10.780 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.773+0000 7fc9b5082740 10 client.0 _ll_register_callbacks cb 0x55bfe721e140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:02:10.780 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.773+0000 7fc9ab7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bfe73012e0 0x55bfe730b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:10.780 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.773+0000 7fc9ab7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bfe73012e0 0x55bfe730b0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43234/0 (socket says 172.21.15.16:43234) 2024-09-12T02:02:10.780 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.773+0000 7fc9ab7fe640 1 -- 172.21.15.16:0/2585634741 learned_addr learned my addr 172.21.15.16:0/2585634741 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:02:10.780 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.773+0000 7fc9abfff640 1 --2- 172.21.15.16:0/2585634741 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bfe73009a0 0x55bfe730ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:10.780 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.773+0000 7fc9b0883640 1 --2- 172.21.15.16:0/2585634741 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bfe7306370 0x55bfe73120a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:10.780 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.773+0000 7fc9ab7fe640 1 -- 172.21.15.16:0/2585634741 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bfe73009a0 msgr2=0x55bfe730ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:10.780 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.773+0000 7fc9ab7fe640 1 --2- 172.21.15.16:0/2585634741 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bfe73009a0 0x55bfe730ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:10.781 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.773+0000 7fc9ab7fe640 1 -- 172.21.15.16:0/2585634741 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bfe7306370 msgr2=0x55bfe73120a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:10.781 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.773+0000 7fc9ab7fe640 1 --2- 172.21.15.16:0/2585634741 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bfe7306370 0x55bfe73120a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:10.781 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.773+0000 7fc9ab7fe640 1 -- 172.21.15.16:0/2585634741 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bfe6f77210 con 0x55bfe73012e0 2024-09-12T02:02:10.781 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.773+0000 7fc9b0883640 1 --2- 172.21.15.16:0/2585634741 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bfe7306370 0x55bfe73120a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:02:10.781 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.773+0000 7fc9ab7fe640 1 --2- 172.21.15.16:0/2585634741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bfe73012e0 0x55bfe730b0d0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fc9a002a930 tx=0x7fc9a00318e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fbfe91c21c5466 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:02:10.781 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.773+0000 7fc98b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:02:10.781 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.774+0000 7fc98b7fe640 1 -- 172.21.15.16:0/2585634741 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9a0035040 con 0x55bfe73012e0 2024-09-12T02:02:10.781 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.774+0000 7fc98b7fe640 1 -- 172.21.15.16:0/2585634741 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fc9a0004040 con 0x55bfe73012e0 2024-09-12T02:02:10.781 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.774+0000 7fc9b5082740 1 -- 172.21.15.16:0/2585634741 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc99c0b9e50 con 0x55bfe73012e0 2024-09-12T02:02:10.781 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.774+0000 7fc98b7fe640 1 -- 172.21.15.16:0/2585634741 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc9a0004240 con 0x55bfe73012e0 2024-09-12T02:02:10.782 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.774+0000 7fc9b5082740 10 client.15360 fetch_fsmap learned FSMap version 25 2024-09-12T02:02:10.782 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.774+0000 7fc9b5082740 1 -- 172.21.15.16:0/2585634741 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55bfe7043620 con 0x55bfe73012e0 2024-09-12T02:02:10.782 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.774+0000 7fc98b7fe640 1 -- 172.21.15.16:0/2585634741 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc99c0b9e50 con 0x55bfe73012e0 2024-09-12T02:02:10.782 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.774+0000 7fc98b7fe640 1 -- 172.21.15.16:0/2585634741 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fc9a0004ae0 con 0x55bfe73012e0 2024-09-12T02:02:10.782 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.774+0000 7fc9b5082740 10 client.15360 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:02:10.782 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.774+0000 7fc9b5082740 10 client.15360 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:02:10.782 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.774+0000 7fc9b5082740 20 client.15360 populate_metadata read hostname 'smithi016' 2024-09-12T02:02:10.782 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.774+0000 7fc9b5082740 1 --2- 172.21.15.16:0/2585634741 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55bfe7316790 0x55bfe7318b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:10.782 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.774+0000 7fc9b5082740 4 client.15360 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:02:10.782 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.774+0000 7fc9b5082740 1 -- 172.21.15.16:0/2585634741 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bfe7271a20 con 0x55bfe7316790 2024-09-12T02:02:10.782 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.775+0000 7fc9abfff640 1 --2- 172.21.15.16:0/2585634741 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55bfe7316790 0x55bfe7318b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:10.782 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.775+0000 7fc98b7fe640 10 client.15360 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:02:10.782 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.775+0000 7fc9abfff640 1 --2- 172.21.15.16:0/2585634741 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55bfe7316790 0x55bfe7318b80 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8e406cae16746a5 server_cookie=4f0b74f4362a0235 in_seq=0 out_seq=0 2024-09-12T02:02:10.783 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.776+0000 7fc98b7fe640 1 -- 172.21.15.16:0/2585634741 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bfe7271a20 con 0x55bfe7316790 2024-09-12T02:02:10.783 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.776+0000 7fc98b7fe640 10 client.15360 handle_command_reply: tid=0 2024-09-12T02:02:10.783 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.776+0000 7fc9b5082740 1 client.15360 shutdown 2024-09-12T02:02:10.783 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.776+0000 7fc9b5082740 1 -- 172.21.15.16:0/2585634741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bfe73012e0 msgr2=0x55bfe730b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:10.783 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.776+0000 7fc9b5082740 1 --2- 172.21.15.16:0/2585634741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bfe73012e0 0x55bfe730b0d0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fc9a002a930 tx=0x7fc9a00318e0 comp rx=0 tx=0).stop 2024-09-12T02:02:10.783 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.776+0000 7fc9b5082740 1 -- 172.21.15.16:0/2585634741 shutdown_connections 2024-09-12T02:02:10.783 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.776+0000 7fc9b5082740 1 --2- 172.21.15.16:0/2585634741 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55bfe7316790 0x55bfe7318b80 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:10.783 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.776+0000 7fc9b5082740 1 --2- 172.21.15.16:0/2585634741 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bfe7306370 0x55bfe73120a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:10.783 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.776+0000 7fc9b5082740 1 --2- 172.21.15.16:0/2585634741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bfe73012e0 0x55bfe730b0d0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:10.783 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.776+0000 7fc98b7fe640 0 client.15360 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:02:10.783 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.776+0000 7fc9b5082740 1 --2- 172.21.15.16:0/2585634741 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bfe73009a0 0x55bfe730ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:10.783 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.776+0000 7fc9b5082740 1 -- 172.21.15.16:0/2585634741 >> 172.21.15.16:0/2585634741 conn(0x55bfe72fc010 msgr2=0x55bfe72fd820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:02:10.784 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.776+0000 7fc9b5082740 1 -- 172.21.15.16:0/2585634741 shutdown_connections 2024-09-12T02:02:10.784 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.776+0000 7fc9b5082740 1 -- 172.21.15.16:0/2585634741 wait complete. 2024-09-12T02:02:10.784 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.776+0000 7fc9b5082740 20 client.15360 trim_cache size 0 max 16384 2024-09-12T02:02:10.793 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.791+0000 7fc9b5082740 1 Processor -- start 2024-09-12T02:02:10.793 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.791+0000 7fc9b5082740 1 -- start start 2024-09-12T02:02:10.793 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.791+0000 7fc9b5082740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bfe73009a0 0x55bfe72fd400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:10.793 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.791+0000 7fc9b5082740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bfe73012e0 0x55bfe7312060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:10.793 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.791+0000 7fc9b5082740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bfe7306370 0x55bfe72fd940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:10.793 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.791+0000 7fc9b5082740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc9ac1ab6a0 con 0x55bfe7306370 2024-09-12T02:02:10.793 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.791+0000 7fc9b5082740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc9ac1a8f70 con 0x55bfe73009a0 2024-09-12T02:02:10.793 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.791+0000 7fc9b5082740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc9ac1a6dc0 con 0x55bfe73012e0 2024-09-12T02:02:10.793 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.791+0000 7fc9b0883640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bfe7306370 0x55bfe72fd940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:10.793 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.791+0000 7fc9b0883640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bfe7306370 0x55bfe72fd940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43244/0 (socket says 172.21.15.16:43244) 2024-09-12T02:02:10.793 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.791+0000 7fc9b0883640 1 -- 172.21.15.16:0/126830529 learned_addr learned my addr 172.21.15.16:0/126830529 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:02:10.794 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.791+0000 7fc9abfff640 1 --2- 172.21.15.16:0/126830529 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bfe73009a0 0x55bfe72fd400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:10.794 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.791+0000 7fc9ab7fe640 1 --2- 172.21.15.16:0/126830529 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bfe73012e0 0x55bfe7312060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:10.794 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.791+0000 7fc9b0883640 1 -- 172.21.15.16:0/126830529 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bfe73012e0 msgr2=0x55bfe7312060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:10.794 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.791+0000 7fc9b0883640 1 --2- 172.21.15.16:0/126830529 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bfe73012e0 0x55bfe7312060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:10.794 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.791+0000 7fc9b0883640 1 -- 172.21.15.16:0/126830529 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bfe73009a0 msgr2=0x55bfe72fd400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:10.794 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.791+0000 7fc9b0883640 1 --2- 172.21.15.16:0/126830529 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bfe73009a0 0x55bfe72fd400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:10.794 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.791+0000 7fc9b0883640 1 -- 172.21.15.16:0/126830529 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bfe72fbe30 con 0x55bfe7306370 2024-09-12T02:02:10.794 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.792+0000 7fc9b0883640 1 --2- 172.21.15.16:0/126830529 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bfe7306370 0x55bfe72fd940 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fc9a4013620 tx=0x7fc9a4009c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dd91f0ffba8ed5d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:02:10.794 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.792+0000 7fc991ffb640 1 -- 172.21.15.16:0/126830529 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9a4010bb0 con 0x55bfe7306370 2024-09-12T02:02:10.794 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.792+0000 7fc991ffb640 1 -- 172.21.15.16:0/126830529 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc9a4010d50 con 0x55bfe7306370 2024-09-12T02:02:10.794 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.792+0000 7fc991ffb640 1 -- 172.21.15.16:0/126830529 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9a40227a0 con 0x55bfe7306370 2024-09-12T02:02:10.794 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.792+0000 7fc9b5082740 1 -- 172.21.15.16:0/126830529 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bfe7306370 msgr2=0x55bfe72fd940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:10.794 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.792+0000 7fc9b5082740 1 --2- 172.21.15.16:0/126830529 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bfe7306370 0x55bfe72fd940 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fc9a4013620 tx=0x7fc9a4009c80 comp rx=0 tx=0).stop 2024-09-12T02:02:10.795 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.792+0000 7fc9b5082740 1 -- 172.21.15.16:0/126830529 shutdown_connections 2024-09-12T02:02:10.795 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.792+0000 7fc9b5082740 1 --2- 172.21.15.16:0/126830529 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bfe7306370 0x55bfe72fd940 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:10.795 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.792+0000 7fc9b5082740 1 --2- 172.21.15.16:0/126830529 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bfe73012e0 0x55bfe7312060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:10.795 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.792+0000 7fc9b5082740 1 --2- 172.21.15.16:0/126830529 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bfe73009a0 0x55bfe72fd400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:10.795 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.792+0000 7fc9b5082740 1 -- 172.21.15.16:0/126830529 >> 172.21.15.16:0/126830529 conn(0x55bfe72fc010 msgr2=0x55bfe72fb1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:02:10.795 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.792+0000 7fc9b5082740 1 -- 172.21.15.16:0/126830529 shutdown_connections 2024-09-12T02:02:10.795 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.793+0000 7fc9b5082740 1 -- 172.21.15.16:0/126830529 wait complete. 2024-09-12T02:02:10.795 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.793+0000 7fc9b5082740 1 -- start start 2024-09-12T02:02:10.795 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.793+0000 7fc9b5082740 1 Processor -- start 2024-09-12T02:02:10.798 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.793+0000 7fc9b5082740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bfe73009a0 0x55bfe730fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:10.799 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.793+0000 7fc9b5082740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bfe73012e0 0x55bfe730ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:10.799 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.793+0000 7fc9b5082740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bfe7306370 0x55bfe7310f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:10.799 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.793+0000 7fc9b5082740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc9ac1a7430 con 0x55bfe7306370 2024-09-12T02:02:10.799 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.793+0000 7fc9b5082740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc9ac1a88a0 con 0x55bfe73012e0 2024-09-12T02:02:10.799 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.793+0000 7fc9b5082740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc9ac1a7b00 con 0x55bfe73009a0 2024-09-12T02:02:10.799 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.793+0000 7fc9b5082740 10 client.0 _ll_register_callbacks cb 0x7fc9a0009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:02:10.799 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.793+0000 7fc9b0883640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bfe7306370 0x55bfe7310f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:10.799 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.793+0000 7fc9b0883640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bfe7306370 0x55bfe7310f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43254/0 (socket says 172.21.15.16:43254) 2024-09-12T02:02:10.799 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.793+0000 7fc9b0883640 1 -- 172.21.15.16:0/1946355314 learned_addr learned my addr 172.21.15.16:0/1946355314 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:02:10.799 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.794+0000 7fc9ab7fe640 1 --2- 172.21.15.16:0/1946355314 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bfe73012e0 0x55bfe730ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:10.799 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.794+0000 7fc9abfff640 1 --2- 172.21.15.16:0/1946355314 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bfe73009a0 0x55bfe730fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:10.800 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.794+0000 7fc9b0883640 1 -- 172.21.15.16:0/1946355314 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bfe73009a0 msgr2=0x55bfe730fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:10.800 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.794+0000 7fc9b0883640 1 --2- 172.21.15.16:0/1946355314 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bfe73009a0 0x55bfe730fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:10.800 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.794+0000 7fc9b0883640 1 -- 172.21.15.16:0/1946355314 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bfe73012e0 msgr2=0x55bfe730ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:10.800 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.794+0000 7fc9b0883640 1 --2- 172.21.15.16:0/1946355314 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bfe73012e0 0x55bfe730ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:10.800 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.794+0000 7fc9b0883640 1 -- 172.21.15.16:0/1946355314 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bfe7311540 con 0x55bfe7306370 2024-09-12T02:02:10.800 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.794+0000 7fc9abfff640 1 --2- 172.21.15.16:0/1946355314 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bfe73009a0 0x55bfe730fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:02:10.800 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.794+0000 7fc9b0883640 1 --2- 172.21.15.16:0/1946355314 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bfe7306370 0x55bfe7310f30 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fc9a4006010 tx=0x7fc9a4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9b5f7fe2c8364b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:02:10.800 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.794+0000 7fc98b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:02:10.800 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.794+0000 7fc98b7fe640 1 -- 172.21.15.16:0/1946355314 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9a403b060 con 0x55bfe7306370 2024-09-12T02:02:10.800 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.794+0000 7fc98b7fe640 1 -- 172.21.15.16:0/1946355314 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fc9a4003970 con 0x55bfe7306370 2024-09-12T02:02:10.800 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.794+0000 7fc9b5082740 1 -- 172.21.15.16:0/1946355314 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc9a4010d50 con 0x55bfe7306370 2024-09-12T02:02:10.800 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.794+0000 7fc98b7fe640 1 -- 172.21.15.16:0/1946355314 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc9a4003b70 con 0x55bfe7306370 2024-09-12T02:02:10.801 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.794+0000 7fc9b5082740 10 client.15366 fetch_fsmap learned FSMap version 25 2024-09-12T02:02:10.801 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.794+0000 7fc9b5082740 1 -- 172.21.15.16:0/1946355314 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55bfe73816d0 con 0x55bfe7306370 2024-09-12T02:02:10.801 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.795+0000 7fc98b7fe640 1 -- 172.21.15.16:0/1946355314 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc9a4010d50 con 0x55bfe7306370 2024-09-12T02:02:10.801 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.795+0000 7fc98b7fe640 1 -- 172.21.15.16:0/1946355314 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fc9a4040070 con 0x55bfe7306370 2024-09-12T02:02:10.801 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.795+0000 7fc9b5082740 10 client.15366 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:02:10.801 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.795+0000 7fc9b5082740 10 client.15366 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:02:10.801 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.795+0000 7fc9b5082740 20 client.15366 populate_metadata read hostname 'smithi016' 2024-09-12T02:02:10.801 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.795+0000 7fc9b5082740 1 --2- 172.21.15.16:0/1946355314 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55bfe7382030 0x55bfe7384450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:10.801 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.795+0000 7fc9b5082740 4 client.15366 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:02:10.801 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.795+0000 7fc9b5082740 1 -- 172.21.15.16:0/1946355314 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bfe726bd70 con 0x55bfe7382030 2024-09-12T02:02:10.801 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.795+0000 7fc9abfff640 1 --2- 172.21.15.16:0/1946355314 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55bfe7382030 0x55bfe7384450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:10.801 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.795+0000 7fc98b7fe640 10 client.15366 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:02:10.801 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.795+0000 7fc9abfff640 1 --2- 172.21.15.16:0/1946355314 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55bfe7382030 0x55bfe7384450 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d06fc64a72cad754 server_cookie=cadf7156fb99a8bc in_seq=0 out_seq=0 2024-09-12T02:02:10.804 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.804+0000 7fc98b7fe640 1 -- 172.21.15.16:0/1946355314 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55bfe726bd70 con 0x55bfe7382030 2024-09-12T02:02:10.804 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.804+0000 7fc98b7fe640 10 client.15366 handle_command_reply: tid=0 2024-09-12T02:02:10.805 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.804+0000 7fc9b5082740 1 client.15366 shutdown 2024-09-12T02:02:10.805 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.804+0000 7fc9b5082740 1 -- 172.21.15.16:0/1946355314 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bfe7306370 msgr2=0x55bfe7310f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:10.805 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.804+0000 7fc9b5082740 1 --2- 172.21.15.16:0/1946355314 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bfe7306370 0x55bfe7310f30 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fc9a4006010 tx=0x7fc9a4004a40 comp rx=0 tx=0).stop 2024-09-12T02:02:10.805 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.804+0000 7fc9b5082740 1 -- 172.21.15.16:0/1946355314 shutdown_connections 2024-09-12T02:02:10.805 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.804+0000 7fc9b5082740 1 --2- 172.21.15.16:0/1946355314 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55bfe7382030 0x55bfe7384450 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:10.805 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.804+0000 7fc9b5082740 1 --2- 172.21.15.16:0/1946355314 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bfe7306370 0x55bfe7310f30 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:10.805 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.804+0000 7fc9b5082740 1 --2- 172.21.15.16:0/1946355314 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bfe73012e0 0x55bfe730ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:10.805 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.804+0000 7fc98b7fe640 0 client.15366 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:02:10.805 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.804+0000 7fc9b5082740 1 --2- 172.21.15.16:0/1946355314 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bfe73009a0 0x55bfe730fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:10.805 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.804+0000 7fc9b5082740 1 -- 172.21.15.16:0/1946355314 >> 172.21.15.16:0/1946355314 conn(0x55bfe72fc010 msgr2=0x55bfe72fc3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:02:10.805 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.804+0000 7fc9b5082740 1 -- 172.21.15.16:0/1946355314 shutdown_connections 2024-09-12T02:02:10.805 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.804+0000 7fc9b5082740 1 -- 172.21.15.16:0/1946355314 wait complete. 2024-09-12T02:02:10.805 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.804+0000 7fc9b5082740 20 client.15366 trim_cache size 0 max 16384 2024-09-12T02:02:10.806 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-09-12T02:02:10.806 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-09-12T02:02:10.806 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-09-12T02:02:10.806 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-09-12T02:02:10.807 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.805+0000 7fc9b22e6640 1 -- 172.21.15.16:0/1115426900 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc9940a2c30 msgr2=0x7fc9940a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:10.807 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.805+0000 7fc9b22e6640 1 --2- 172.21.15.16:0/1115426900 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc9940a2c30 0x7fc9940a50f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fc9a4009fd0 tx=0x7fc9a4004ab0 comp rx=0 tx=0).stop 2024-09-12T02:02:10.807 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.805+0000 7fc9b22e6640 1 -- 172.21.15.16:0/1115426900 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9ac101b60 msgr2=0x7fc9ac197dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:10.807 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.805+0000 7fc9b22e6640 1 --2- 172.21.15.16:0/1115426900 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9ac101b60 0x7fc9ac197dd0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fc99c00d950 tx=0x7fc99c00de10 comp rx=0 tx=0).stop 2024-09-12T02:02:10.807 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.805+0000 7fc9b22e6640 1 -- 172.21.15.16:0/1115426900 shutdown_connections 2024-09-12T02:02:10.807 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.805+0000 7fc9b22e6640 1 --2- 172.21.15.16:0/1115426900 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc9940a2c30 0x7fc9940a50f0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:10.807 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.805+0000 7fc9b22e6640 1 --2- 172.21.15.16:0/1115426900 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc9ac103710 0x7fc9ac19f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:10.807 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.805+0000 7fc9b22e6640 1 --2- 172.21.15.16:0/1115426900 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc9ac102d50 0x7fc9ac198310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:10.807 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.805+0000 7fc9b22e6640 1 --2- 172.21.15.16:0/1115426900 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9ac101b60 0x7fc9ac197dd0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:10.808 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.805+0000 7fc9b22e6640 1 -- 172.21.15.16:0/1115426900 >> 172.21.15.16:0/1115426900 conn(0x7fc9ac0fd2e0 msgr2=0x7fc9ac1060a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:02:10.808 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.805+0000 7fc9b22e6640 1 -- 172.21.15.16:0/1115426900 shutdown_connections 2024-09-12T02:02:10.808 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:10.805+0000 7fc9b22e6640 1 -- 172.21.15.16:0/1115426900 wait complete. 2024-09-12T02:02:12.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:02:12 smithi184 ceph-mon[35593]: pgmap v343: 97 pgs: 97 active+clean; 657 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 2.7 KiB/s wr, 11 op/s 2024-09-12T02:02:12.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:02:12 smithi033 ceph-mon[35332]: pgmap v343: 97 pgs: 97 active+clean; 657 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 2.7 KiB/s wr, 11 op/s 2024-09-12T02:02:12.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:02:12 smithi016 ceph-mon[30947]: pgmap v343: 97 pgs: 97 active+clean; 657 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 2.7 KiB/s wr, 11 op/s 2024-09-12T02:02:14.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:02:14 smithi184 ceph-mon[35593]: pgmap v344: 97 pgs: 97 active+clean; 657 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 7 op/s 2024-09-12T02:02:14.668 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:02:14 smithi033 ceph-mon[35332]: pgmap v344: 97 pgs: 97 active+clean; 657 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 7 op/s 2024-09-12T02:02:14.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:02:14 smithi016 ceph-mon[30947]: pgmap v344: 97 pgs: 97 active+clean; 657 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 7 op/s 2024-09-12T02:02:16.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:02:16 smithi184 ceph-mon[35593]: pgmap v345: 97 pgs: 97 active+clean; 660 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1023 B/s wr, 7 op/s 2024-09-12T02:02:16.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:02:16 smithi033 ceph-mon[35332]: pgmap v345: 97 pgs: 97 active+clean; 660 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1023 B/s wr, 7 op/s 2024-09-12T02:02:16.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:02:16 smithi016 ceph-mon[30947]: pgmap v345: 97 pgs: 97 active+clean; 660 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1023 B/s wr, 7 op/s 2024-09-12T02:02:18.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:02:18 smithi184 ceph-mon[35593]: pgmap v346: 97 pgs: 97 active+clean; 660 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-09-12T02:02:18.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:02:18 smithi033 ceph-mon[35332]: pgmap v346: 97 pgs: 97 active+clean; 660 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-09-12T02:02:18.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:02:18 smithi016 ceph-mon[30947]: pgmap v346: 97 pgs: 97 active+clean; 660 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-09-12T02:02:18.710 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T02:02:18.974 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:02:19.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.465+0000 7f2a03f62640 1 -- 172.21.15.16:0/855466252 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29fc102d30 msgr2=0x7f29fc103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:19.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.465+0000 7f2a03f62640 1 --2- 172.21.15.16:0/855466252 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29fc102d30 0x7f29fc103110 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f29f8009930 tx=0x7f29f802a420 comp rx=0 tx=0).stop 2024-09-12T02:02:19.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.465+0000 7f2a03f62640 1 -- 172.21.15.16:0/855466252 shutdown_connections 2024-09-12T02:02:19.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.465+0000 7f2a03f62640 1 --2- 172.21.15.16:0/855466252 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f29fc1082a0 0x7f29fc10ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:19.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.465+0000 7f2a03f62640 1 --2- 172.21.15.16:0/855466252 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f29fc1036e0 0x7f29fc107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:19.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.465+0000 7f2a03f62640 1 --2- 172.21.15.16:0/855466252 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29fc102d30 0x7f29fc103110 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:19.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.465+0000 7f2a03f62640 1 -- 172.21.15.16:0/855466252 >> 172.21.15.16:0/855466252 conn(0x7f29fc0fc7d0 msgr2=0x7f29fc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:02:19.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.465+0000 7f2a03f62640 1 -- 172.21.15.16:0/855466252 shutdown_connections 2024-09-12T02:02:19.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.465+0000 7f2a03f62640 1 -- 172.21.15.16:0/855466252 wait complete. 2024-09-12T02:02:19.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.466+0000 7f2a03f62640 1 Processor -- start 2024-09-12T02:02:19.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.466+0000 7f2a03f62640 1 -- start start 2024-09-12T02:02:19.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.466+0000 7f2a03f62640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f29fc102d30 0x7f29fc100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:19.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.466+0000 7f2a03f62640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f29fc1036e0 0x7f29fc100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:19.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.466+0000 7f2a03f62640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29fc1082a0 0x7f29fc101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:19.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.466+0000 7f2a03f62640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f29fc1120a0 con 0x7f29fc1082a0 2024-09-12T02:02:19.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.466+0000 7f2a03f62640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f29fc111f20 con 0x7f29fc102d30 2024-09-12T02:02:19.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.466+0000 7f2a03f62640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f29fc112220 con 0x7f29fc1036e0 2024-09-12T02:02:19.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.466+0000 7f2a024ff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29fc1082a0 0x7f29fc101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:19.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.466+0000 7f2a024ff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29fc1082a0 0x7f29fc101410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33896/0 (socket says 172.21.15.16:33896) 2024-09-12T02:02:19.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.466+0000 7f2a024ff640 1 -- 172.21.15.16:0/2606684729 learned_addr learned my addr 172.21.15.16:0/2606684729 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:02:19.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.466+0000 7f2a014fd640 1 --2- 172.21.15.16:0/2606684729 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f29fc1036e0 0x7f29fc100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:19.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.466+0000 7f2a01cfe640 1 --2- 172.21.15.16:0/2606684729 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f29fc102d30 0x7f29fc100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:19.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.466+0000 7f2a024ff640 1 -- 172.21.15.16:0/2606684729 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f29fc1036e0 msgr2=0x7f29fc100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:19.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.466+0000 7f2a024ff640 1 --2- 172.21.15.16:0/2606684729 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f29fc1036e0 0x7f29fc100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:19.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.466+0000 7f2a024ff640 1 -- 172.21.15.16:0/2606684729 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f29fc102d30 msgr2=0x7f29fc100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:19.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.466+0000 7f2a024ff640 1 --2- 172.21.15.16:0/2606684729 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f29fc102d30 0x7f29fc100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:19.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.466+0000 7f2a024ff640 1 -- 172.21.15.16:0/2606684729 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29fc1a6a80 con 0x7f29fc1082a0 2024-09-12T02:02:19.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.466+0000 7f2a014fd640 1 --2- 172.21.15.16:0/2606684729 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f29fc1036e0 0x7f29fc100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:02:19.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.467+0000 7f2a024ff640 1 --2- 172.21.15.16:0/2606684729 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29fc1082a0 0x7f29fc101410 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f29f4009fd0 tx=0x7f29f400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:02:19.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.467+0000 7f29f2ffd640 1 -- 172.21.15.16:0/2606684729 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29f4017070 con 0x7f29fc1082a0 2024-09-12T02:02:19.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.467+0000 7f29f2ffd640 1 -- 172.21.15.16:0/2606684729 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f29f4003e50 con 0x7f29fc1082a0 2024-09-12T02:02:19.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.467+0000 7f2a03f62640 1 -- 172.21.15.16:0/2606684729 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29fc1a6d70 con 0x7f29fc1082a0 2024-09-12T02:02:19.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.467+0000 7f29f2ffd640 1 -- 172.21.15.16:0/2606684729 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29f40106c0 con 0x7f29fc1082a0 2024-09-12T02:02:19.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.467+0000 7f2a03f62640 1 -- 172.21.15.16:0/2606684729 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29fc1a72b0 con 0x7f29fc1082a0 2024-09-12T02:02:19.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.470+0000 7f2a0633b740 1 Processor -- start 2024-09-12T02:02:19.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.470+0000 7f29f2ffd640 1 -- 172.21.15.16:0/2606684729 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f29f4003970 con 0x7f29fc1082a0 2024-09-12T02:02:19.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.470+0000 7f2a0633b740 1 -- start start 2024-09-12T02:02:19.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.471+0000 7f2a0633b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5631e497f4a0 0x5631e497f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:19.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.471+0000 7f2a0633b740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5631e497fdc0 0x5631e498a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:19.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.471+0000 7f2a0633b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5631e498a680 0x5631e498ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:19.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.471+0000 7f2a0633b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5631e48e52e0 con 0x5631e498a680 2024-09-12T02:02:19.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.471+0000 7f2a0633b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5631e488cce0 con 0x5631e497f4a0 2024-09-12T02:02:19.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.471+0000 7f2a0633b740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5631e48e12a0 con 0x5631e497fdc0 2024-09-12T02:02:19.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.471+0000 7f2a014fd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5631e498a680 0x5631e498ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:19.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.471+0000 7f2a014fd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5631e498a680 0x5631e498ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33902/0 (socket says 172.21.15.16:33902) 2024-09-12T02:02:19.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.471+0000 7f2a014fd640 1 -- 172.21.15.16:0/855495638 learned_addr learned my addr 172.21.15.16:0/855495638 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:02:19.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.471+0000 7f2a014fd640 1 --2- 172.21.15.16:0/855495638 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5631e497f4a0 0x5631e497f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:19.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.471+0000 7f2a01cfe640 1 --2- 172.21.15.16:0/855495638 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5631e497fdc0 0x5631e498a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:19.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.471+0000 7f29f2ffd640 1 --2- 172.21.15.16:0/2606684729 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f29e80a2c00 0x7f29e80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:19.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.471+0000 7f2a014fd640 1 -- 172.21.15.16:0/855495638 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5631e497fdc0 msgr2=0x5631e498a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:19.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.471+0000 7f2a014fd640 1 --2- 172.21.15.16:0/855495638 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5631e497fdc0 0x5631e498a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:19.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.471+0000 7f2a014fd640 1 -- 172.21.15.16:0/855495638 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5631e497f4a0 msgr2=0x5631e497f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:19.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.471+0000 7f2a014fd640 1 --2- 172.21.15.16:0/855495638 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5631e497f4a0 0x5631e497f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:19.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.471+0000 7f2a014fd640 1 -- 172.21.15.16:0/855495638 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5631e4871440 con 0x5631e498a680 2024-09-12T02:02:19.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.471+0000 7f2a024ff640 1 --2- 172.21.15.16:0/2606684729 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f29e80a2c00 0x7f29e80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:19.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.471+0000 7f29f2ffd640 1 -- 172.21.15.16:0/2606684729 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f29f40ce9f0 con 0x7f29fc1082a0 2024-09-12T02:02:19.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.471+0000 7f2a014fd640 1 --2- 172.21.15.16:0/855495638 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5631e498a680 0x5631e498ca70 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f29ec005fd0 tx=0x7f29ec00b460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89db3ac308731e51 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:02:19.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.472+0000 7f29e7fff640 1 -- 172.21.15.16:0/855495638 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29ec004020 con 0x5631e498a680 2024-09-12T02:02:19.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.472+0000 7f29e7fff640 1 -- 172.21.15.16:0/855495638 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f29ec00ee10 con 0x5631e498a680 2024-09-12T02:02:19.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.472+0000 7f29e7fff640 1 -- 172.21.15.16:0/855495638 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29ec003950 con 0x5631e498a680 2024-09-12T02:02:19.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.472+0000 7f2a0633b740 1 -- 172.21.15.16:0/855495638 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5631e498a680 msgr2=0x5631e498ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:19.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.472+0000 7f2a024ff640 1 --2- 172.21.15.16:0/2606684729 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f29e80a2c00 0x7f29e80a50c0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f29f4009230 tx=0x7f29f40bb040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:02:19.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.472+0000 7f2a0633b740 1 --2- 172.21.15.16:0/855495638 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5631e498a680 0x5631e498ca70 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f29ec005fd0 tx=0x7f29ec00b460 comp rx=0 tx=0).stop 2024-09-12T02:02:19.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.473+0000 7f2a0633b740 1 -- 172.21.15.16:0/855495638 shutdown_connections 2024-09-12T02:02:19.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.473+0000 7f2a0633b740 1 --2- 172.21.15.16:0/855495638 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5631e498a680 0x5631e498ca70 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:19.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.473+0000 7f2a0633b740 1 --2- 172.21.15.16:0/855495638 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5631e497fdc0 0x5631e498a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:19.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.473+0000 7f2a0633b740 1 --2- 172.21.15.16:0/855495638 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5631e497f4a0 0x5631e497f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:19.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.473+0000 7f2a0633b740 1 -- 172.21.15.16:0/855495638 >> 172.21.15.16:0/855495638 conn(0x5631e497ba30 msgr2=0x5631e497be10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:02:19.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.473+0000 7f2a0633b740 1 -- 172.21.15.16:0/855495638 shutdown_connections 2024-09-12T02:02:19.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.473+0000 7f2a0633b740 1 -- 172.21.15.16:0/855495638 wait complete. 2024-09-12T02:02:19.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.473+0000 7f2a0633b740 1 -- start start 2024-09-12T02:02:19.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.473+0000 7f2a0633b740 1 Processor -- start 2024-09-12T02:02:19.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.474+0000 7f2a0633b740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5631e497f4a0 0x5631e498a050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:19.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.474+0000 7f2a0633b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5631e497fdc0 0x5631e4987300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:19.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.474+0000 7f2a0633b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5631e498a680 0x5631e49878f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:19.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.474+0000 7f2a0633b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5631e498d250 con 0x5631e497fdc0 2024-09-12T02:02:19.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.474+0000 7f2a0633b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5631e48a0050 con 0x5631e498a680 2024-09-12T02:02:19.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.474+0000 7f2a0633b740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5631e498d3d0 con 0x5631e497f4a0 2024-09-12T02:02:19.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.474+0000 7f2a0633b740 10 client.0 _ll_register_callbacks cb 0x5631e48e0160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:02:19.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.474+0000 7f2a01cfe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5631e497fdc0 0x5631e4987300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:19.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.474+0000 7f2a01cfe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5631e497fdc0 0x5631e4987300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33904/0 (socket says 172.21.15.16:33904) 2024-09-12T02:02:19.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.474+0000 7f2a01cfe640 1 -- 172.21.15.16:0/879198924 learned_addr learned my addr 172.21.15.16:0/879198924 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:02:19.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.474+0000 7f2a014fd640 1 --2- 172.21.15.16:0/879198924 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5631e498a680 0x5631e49878f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:19.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.474+0000 7f2a014fd640 1 --2- 172.21.15.16:0/879198924 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5631e497f4a0 0x5631e498a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:19.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.474+0000 7f2a01cfe640 1 -- 172.21.15.16:0/879198924 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5631e497f4a0 msgr2=0x5631e498a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:19.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.474+0000 7f2a01cfe640 1 --2- 172.21.15.16:0/879198924 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5631e497f4a0 0x5631e498a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:19.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.474+0000 7f2a01cfe640 1 -- 172.21.15.16:0/879198924 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5631e498a680 msgr2=0x5631e49878f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:19.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.474+0000 7f2a01cfe640 1 --2- 172.21.15.16:0/879198924 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5631e498a680 0x5631e49878f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:19.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.474+0000 7f2a01cfe640 1 -- 172.21.15.16:0/879198924 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5631e46563c0 con 0x5631e497fdc0 2024-09-12T02:02:19.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.474+0000 7f2a01cfe640 1 --2- 172.21.15.16:0/879198924 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5631e497fdc0 0x5631e4987300 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f29f8031550 tx=0x7f29f8031860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=586941e36b25d78c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:02:19.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.474+0000 7f29e57fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:02:19.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.475+0000 7f29e57fa640 1 -- 172.21.15.16:0/879198924 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29f8035070 con 0x5631e497fdc0 2024-09-12T02:02:19.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.475+0000 7f29e57fa640 1 -- 172.21.15.16:0/879198924 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f29f8004040 con 0x5631e497fdc0 2024-09-12T02:02:19.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.475+0000 7f2a0633b740 1 -- 172.21.15.16:0/879198924 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f29ec00ee10 con 0x5631e497fdc0 2024-09-12T02:02:19.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.475+0000 7f29e57fa640 1 -- 172.21.15.16:0/879198924 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f29f8004240 con 0x5631e497fdc0 2024-09-12T02:02:19.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.475+0000 7f2a0633b740 10 client.15378 fetch_fsmap learned FSMap version 25 2024-09-12T02:02:19.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.475+0000 7f2a0633b740 1 -- 172.21.15.16:0/879198924 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5631e4998c80 con 0x5631e497fdc0 2024-09-12T02:02:19.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.475+0000 7f29e57fa640 1 -- 172.21.15.16:0/879198924 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f29ec00ee10 con 0x5631e497fdc0 2024-09-12T02:02:19.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.475+0000 7f29e57fa640 1 -- 172.21.15.16:0/879198924 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f29f8004ae0 con 0x5631e497fdc0 2024-09-12T02:02:19.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.475+0000 7f2a0633b740 10 client.15378 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:02:19.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.475+0000 7f2a0633b740 10 client.15378 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:02:19.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.475+0000 7f2a0633b740 20 client.15378 populate_metadata read hostname 'smithi016' 2024-09-12T02:02:19.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.475+0000 7f2a0633b740 1 --2- 172.21.15.16:0/879198924 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5631e499a140 0x5631e499c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:19.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.475+0000 7f2a0633b740 4 client.15378 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:02:19.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.475+0000 7f2a0633b740 1 -- 172.21.15.16:0/879198924 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5631e48f2850 con 0x5631e499a140 2024-09-12T02:02:19.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.476+0000 7f2a014fd640 1 --2- 172.21.15.16:0/879198924 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5631e499a140 0x5631e499c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:19.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.476+0000 7f29e57fa640 10 client.15378 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:02:19.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.476+0000 7f2a014fd640 1 --2- 172.21.15.16:0/879198924 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5631e499a140 0x5631e499c5a0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4bb0637b23e606ca server_cookie=8e31d5501a91f686 in_seq=0 out_seq=0 2024-09-12T02:02:19.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.477+0000 7f29e57fa640 1 -- 172.21.15.16:0/879198924 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5631e48f2850 con 0x5631e499a140 2024-09-12T02:02:19.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.477+0000 7f29e57fa640 10 client.15378 handle_command_reply: tid=0 2024-09-12T02:02:19.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.477+0000 7f2a0633b740 1 client.15378 shutdown 2024-09-12T02:02:19.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.477+0000 7f2a0633b740 1 -- 172.21.15.16:0/879198924 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5631e497fdc0 msgr2=0x5631e4987300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:19.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.477+0000 7f2a0633b740 1 --2- 172.21.15.16:0/879198924 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5631e497fdc0 0x5631e4987300 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f29f8031550 tx=0x7f29f8031860 comp rx=0 tx=0).stop 2024-09-12T02:02:19.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.477+0000 7f2a0633b740 1 -- 172.21.15.16:0/879198924 shutdown_connections 2024-09-12T02:02:19.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.477+0000 7f2a0633b740 1 --2- 172.21.15.16:0/879198924 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5631e499a140 0x5631e499c5a0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:19.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.477+0000 7f2a0633b740 1 --2- 172.21.15.16:0/879198924 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5631e498a680 0x5631e49878f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:19.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.477+0000 7f2a0633b740 1 --2- 172.21.15.16:0/879198924 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5631e497fdc0 0x5631e4987300 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:19.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.477+0000 7f29e57fa640 0 client.15378 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:02:19.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.477+0000 7f2a0633b740 1 --2- 172.21.15.16:0/879198924 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5631e497f4a0 0x5631e498a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:19.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.477+0000 7f2a0633b740 1 -- 172.21.15.16:0/879198924 >> 172.21.15.16:0/879198924 conn(0x5631e497ba30 msgr2=0x5631e4981710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:02:19.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.477+0000 7f2a0633b740 1 -- 172.21.15.16:0/879198924 shutdown_connections 2024-09-12T02:02:19.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.477+0000 7f2a0633b740 1 -- 172.21.15.16:0/879198924 wait complete. 2024-09-12T02:02:19.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.477+0000 7f2a0633b740 20 client.15378 trim_cache size 0 max 16384 2024-09-12T02:02:19.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.498+0000 7f2a0633b740 1 Processor -- start 2024-09-12T02:02:19.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.498+0000 7f2a0633b740 1 -- start start 2024-09-12T02:02:19.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.498+0000 7f2a0633b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5631e497f4a0 0x5631e4982b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:19.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.498+0000 7f2a0633b740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5631e497fdc0 0x5631e49964b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:19.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.498+0000 7f2a0633b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5631e498a680 0x5631e49a1e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:19.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.498+0000 7f2a0633b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f29fc1b1f50 con 0x5631e497f4a0 2024-09-12T02:02:19.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.498+0000 7f2a0633b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f29fc1af820 con 0x5631e498a680 2024-09-12T02:02:19.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.498+0000 7f2a0633b740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f29fc1ad670 con 0x5631e497fdc0 2024-09-12T02:02:19.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.498+0000 7f2a014fd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5631e497f4a0 0x5631e4982b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:19.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.498+0000 7f2a01cfe640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5631e497fdc0 0x5631e49964b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:19.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.498+0000 7f2a014fd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5631e497f4a0 0x5631e4982b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33908/0 (socket says 172.21.15.16:33908) 2024-09-12T02:02:19.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.499+0000 7f2a014fd640 1 -- 172.21.15.16:0/1457343180 learned_addr learned my addr 172.21.15.16:0/1457343180 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:02:19.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.499+0000 7f2a014fd640 1 --2- 172.21.15.16:0/1457343180 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5631e498a680 0x5631e49a1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:19.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.499+0000 7f2a014fd640 1 -- 172.21.15.16:0/1457343180 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5631e497fdc0 msgr2=0x5631e49964b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:19.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.499+0000 7f2a014fd640 1 --2- 172.21.15.16:0/1457343180 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5631e497fdc0 0x5631e49964b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:19.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.499+0000 7f2a014fd640 1 -- 172.21.15.16:0/1457343180 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5631e498a680 msgr2=0x5631e49a1e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:19.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.499+0000 7f2a014fd640 1 --2- 172.21.15.16:0/1457343180 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5631e498a680 0x5631e49a1e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:19.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.499+0000 7f2a014fd640 1 -- 172.21.15.16:0/1457343180 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5631e4985850 con 0x5631e497f4a0 2024-09-12T02:02:19.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.499+0000 7f2a01cfe640 1 --2- 172.21.15.16:0/1457343180 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5631e497fdc0 0x5631e49964b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:02:19.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.499+0000 7f2a014fd640 1 --2- 172.21.15.16:0/1457343180 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5631e497f4a0 0x5631e4982b60 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x5631e497cdb0 tx=0x7f29ec004280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=636ed99ef768f333 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:02:19.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.500+0000 7f29e77fe640 1 -- 172.21.15.16:0/1457343180 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29ec0043f0 con 0x5631e497f4a0 2024-09-12T02:02:19.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.500+0000 7f29e77fe640 1 -- 172.21.15.16:0/1457343180 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f29ec00ee10 con 0x5631e497f4a0 2024-09-12T02:02:19.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.500+0000 7f29e77fe640 1 -- 172.21.15.16:0/1457343180 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29ec02f790 con 0x5631e497f4a0 2024-09-12T02:02:19.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.500+0000 7f2a0633b740 1 -- 172.21.15.16:0/1457343180 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5631e497f4a0 msgr2=0x5631e4982b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:19.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.500+0000 7f2a0633b740 1 --2- 172.21.15.16:0/1457343180 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5631e497f4a0 0x5631e4982b60 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x5631e497cdb0 tx=0x7f29ec004280 comp rx=0 tx=0).stop 2024-09-12T02:02:19.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.500+0000 7f2a0633b740 1 -- 172.21.15.16:0/1457343180 shutdown_connections 2024-09-12T02:02:19.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.500+0000 7f2a0633b740 1 --2- 172.21.15.16:0/1457343180 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5631e498a680 0x5631e49a1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:19.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.500+0000 7f2a0633b740 1 --2- 172.21.15.16:0/1457343180 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5631e497fdc0 0x5631e49964b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:19.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.500+0000 7f2a0633b740 1 --2- 172.21.15.16:0/1457343180 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5631e497f4a0 0x5631e4982b60 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:19.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.500+0000 7f2a0633b740 1 -- 172.21.15.16:0/1457343180 >> 172.21.15.16:0/1457343180 conn(0x5631e497ba30 msgr2=0x5631e4989380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:02:19.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.501+0000 7f2a0633b740 1 -- 172.21.15.16:0/1457343180 shutdown_connections 2024-09-12T02:02:19.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.501+0000 7f2a0633b740 1 -- 172.21.15.16:0/1457343180 wait complete. 2024-09-12T02:02:19.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.501+0000 7f2a0633b740 1 -- start start 2024-09-12T02:02:19.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.501+0000 7f2a0633b740 1 Processor -- start 2024-09-12T02:02:19.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.501+0000 7f2a0633b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5631e497f4a0 0x5631e4984370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:19.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.501+0000 7f2a0633b740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5631e497fdc0 0x5631e49848b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:19.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.501+0000 7f2a0633b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5631e498a680 0x5631e4984df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:19.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.501+0000 7f2a0633b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f29fc1adce0 con 0x5631e497f4a0 2024-09-12T02:02:19.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.501+0000 7f2a0633b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f29fc1af150 con 0x5631e498a680 2024-09-12T02:02:19.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.501+0000 7f2a0633b740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f29fc1ae3b0 con 0x5631e497fdc0 2024-09-12T02:02:19.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.501+0000 7f2a0633b740 10 client.0 _ll_register_callbacks cb 0x7f29f8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:02:19.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.502+0000 7f2a01cfe640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5631e497fdc0 0x5631e49848b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:19.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.502+0000 7f2a01cfe640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5631e497fdc0 0x5631e49848b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:47188/0 (socket says 172.21.15.16:47188) 2024-09-12T02:02:19.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.502+0000 7f2a01cfe640 1 -- 172.21.15.16:0/3304982425 learned_addr learned my addr 172.21.15.16:0/3304982425 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:02:19.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.502+0000 7f2a014fd640 1 --2- 172.21.15.16:0/3304982425 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5631e497f4a0 0x5631e4984370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:19.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.502+0000 7f2a014fd640 1 --2- 172.21.15.16:0/3304982425 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5631e498a680 0x5631e4984df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:19.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.502+0000 7f2a01cfe640 1 -- 172.21.15.16:0/3304982425 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5631e498a680 msgr2=0x5631e4984df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:19.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.502+0000 7f2a01cfe640 1 --2- 172.21.15.16:0/3304982425 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5631e498a680 0x5631e4984df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:19.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.502+0000 7f2a01cfe640 1 -- 172.21.15.16:0/3304982425 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5631e497f4a0 msgr2=0x5631e4984370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:19.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.502+0000 7f2a01cfe640 1 --2- 172.21.15.16:0/3304982425 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5631e497f4a0 0x5631e4984370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:19.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.502+0000 7f2a01cfe640 1 -- 172.21.15.16:0/3304982425 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5631e4a03e90 con 0x5631e497fdc0 2024-09-12T02:02:19.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.502+0000 7f2a01cfe640 1 --2- 172.21.15.16:0/3304982425 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5631e497fdc0 0x5631e49848b0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f29f8003640 tx=0x7f29f8004610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d88c92488780d112 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:02:19.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.502+0000 7f29e57fa640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-09-12T02:02:19.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.503+0000 7f29e57fa640 1 -- 172.21.15.16:0/3304982425 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29f8035040 con 0x5631e497fdc0 2024-09-12T02:02:19.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.503+0000 7f29e57fa640 1 -- 172.21.15.16:0/3304982425 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f29f8003d60 con 0x5631e497fdc0 2024-09-12T02:02:19.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.503+0000 7f2a0633b740 1 -- 172.21.15.16:0/3304982425 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f29ec00ee10 con 0x5631e497fdc0 2024-09-12T02:02:19.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.503+0000 7f29e57fa640 1 -- 172.21.15.16:0/3304982425 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f29f8060050 con 0x5631e497fdc0 2024-09-12T02:02:19.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.503+0000 7f2a0633b740 10 client.25021 fetch_fsmap learned FSMap version 25 2024-09-12T02:02:19.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.503+0000 7f2a0633b740 1 -- 172.21.15.16:0/3304982425 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5631e4a051d0 con 0x5631e497fdc0 2024-09-12T02:02:19.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.503+0000 7f29e57fa640 1 -- 172.21.15.16:0/3304982425 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f29f8003f80 con 0x5631e497fdc0 2024-09-12T02:02:19.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.503+0000 7f29e57fa640 1 -- 172.21.15.16:0/3304982425 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f29f8005290 con 0x5631e497fdc0 2024-09-12T02:02:19.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.503+0000 7f2a0633b740 10 client.25021 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:02:19.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.503+0000 7f2a0633b740 10 client.25021 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:02:19.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.503+0000 7f2a0633b740 20 client.25021 populate_metadata read hostname 'smithi016' 2024-09-12T02:02:19.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.503+0000 7f2a0633b740 1 --2- 172.21.15.16:0/3304982425 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5631e4a05980 0x5631e4a07da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:19.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.503+0000 7f2a0633b740 4 client.25021 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T02:02:19.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.503+0000 7f2a0633b740 1 -- 172.21.15.16:0/3304982425 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f29ec00ee10 con 0x5631e4a05980 2024-09-12T02:02:19.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.504+0000 7f2a014fd640 1 --2- 172.21.15.16:0/3304982425 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5631e4a05980 0x5631e4a07da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:19.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.504+0000 7f2a014fd640 1 --2- 172.21.15.16:0/3304982425 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5631e4a05980 0x5631e4a07da0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba0284c68eaec0de server_cookie=3db3d9085e0cd88c in_seq=0 out_seq=0 2024-09-12T02:02:19.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.504+0000 7f29e57fa640 10 client.25021 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:02:19.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.504+0000 7f29e57fa640 1 -- 172.21.15.16:0/3304982425 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7f29ec00ee10 con 0x5631e4a05980 2024-09-12T02:02:19.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.504+0000 7f29e57fa640 10 client.25021 handle_command_reply: tid=0 2024-09-12T02:02:19.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.504+0000 7f2a0633b740 1 client.25021 shutdown 2024-09-12T02:02:19.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.504+0000 7f2a0633b740 1 -- 172.21.15.16:0/3304982425 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5631e497fdc0 msgr2=0x5631e49848b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:19.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.504+0000 7f2a0633b740 1 --2- 172.21.15.16:0/3304982425 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5631e497fdc0 0x5631e49848b0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f29f8003640 tx=0x7f29f8004610 comp rx=0 tx=0).stop 2024-09-12T02:02:19.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.504+0000 7f2a0633b740 1 -- 172.21.15.16:0/3304982425 shutdown_connections 2024-09-12T02:02:19.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.504+0000 7f2a0633b740 1 --2- 172.21.15.16:0/3304982425 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5631e4a05980 0x5631e4a07da0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:19.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.505+0000 7f2a0633b740 1 --2- 172.21.15.16:0/3304982425 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5631e498a680 0x5631e4984df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:19.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.505+0000 7f2a0633b740 1 --2- 172.21.15.16:0/3304982425 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5631e497fdc0 0x5631e49848b0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:19.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.505+0000 7f29e57fa640 0 client.25021 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:02:19.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.505+0000 7f2a0633b740 1 --2- 172.21.15.16:0/3304982425 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5631e497f4a0 0x5631e4984370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:19.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.505+0000 7f2a0633b740 1 -- 172.21.15.16:0/3304982425 >> 172.21.15.16:0/3304982425 conn(0x5631e497ba30 msgr2=0x5631e497a350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:02:19.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.505+0000 7f2a0633b740 1 -- 172.21.15.16:0/3304982425 shutdown_connections 2024-09-12T02:02:19.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.505+0000 7f2a0633b740 1 -- 172.21.15.16:0/3304982425 wait complete. 2024-09-12T02:02:19.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.505+0000 7f2a0633b740 20 client.25021 trim_cache size 0 max 16384 2024-09-12T02:02:19.514 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T02:02:19.514 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-09-12T02:02:19.514 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {} 2024-09-12T02:02:19.514 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T02:02:19.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.505+0000 7f2a03f62640 1 -- 172.21.15.16:0/2606684729 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f29e80a2c00 msgr2=0x7f29e80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:19.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.505+0000 7f2a03f62640 1 --2- 172.21.15.16:0/2606684729 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f29e80a2c00 0x7f29e80a50c0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f29f4009230 tx=0x7f29f40bb040 comp rx=0 tx=0).stop 2024-09-12T02:02:19.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.505+0000 7f2a03f62640 1 -- 172.21.15.16:0/2606684729 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29fc1082a0 msgr2=0x7f29fc101410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:19.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.505+0000 7f2a03f62640 1 --2- 172.21.15.16:0/2606684729 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29fc1082a0 0x7f29fc101410 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f29f4009fd0 tx=0x7f29f400b260 comp rx=0 tx=0).stop 2024-09-12T02:02:19.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.506+0000 7f2a03f62640 1 -- 172.21.15.16:0/2606684729 shutdown_connections 2024-09-12T02:02:19.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.506+0000 7f2a03f62640 1 --2- 172.21.15.16:0/2606684729 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f29e80a2c00 0x7f29e80a50c0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:19.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.506+0000 7f2a03f62640 1 --2- 172.21.15.16:0/2606684729 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29fc1082a0 0x7f29fc101410 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:19.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.506+0000 7f2a03f62640 1 --2- 172.21.15.16:0/2606684729 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f29fc1036e0 0x7f29fc100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:19.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.506+0000 7f2a03f62640 1 --2- 172.21.15.16:0/2606684729 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f29fc102d30 0x7f29fc100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:19.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.506+0000 7f2a03f62640 1 -- 172.21.15.16:0/2606684729 >> 172.21.15.16:0/2606684729 conn(0x7f29fc0fc7d0 msgr2=0x7f29fc105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:02:19.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.506+0000 7f2a03f62640 1 -- 172.21.15.16:0/2606684729 shutdown_connections 2024-09-12T02:02:19.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:19.506+0000 7f2a03f62640 1 -- 172.21.15.16:0/2606684729 wait complete. 2024-09-12T02:02:19.886 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 24.5 2024-09-12T02:02:19.887 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:02:20.279 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-12T02:02:20.280 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:02:20.326 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:02:20.518 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:02:20 smithi016 ceph-mon[30947]: pgmap v347: 97 pgs: 97 active+clean; 660 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-09-12T02:02:20.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:02:20 smithi184 ceph-mon[35593]: pgmap v347: 97 pgs: 97 active+clean; 660 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-09-12T02:02:20.668 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:02:20 smithi033 ceph-mon[35332]: pgmap v347: 97 pgs: 97 active+clean; 660 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-09-12T02:02:20.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:20.855+0000 7fddd31d4640 1 -- 172.21.15.16:0/322851828 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fddcc101df0 msgr2=0x7fddcc1021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:20.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:20.855+0000 7fddd31d4640 1 --2- 172.21.15.16:0/322851828 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fddcc101df0 0x7fddcc1021d0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fddc0009900 tx=0x7fddc002a410 comp rx=0 tx=0).stop 2024-09-12T02:02:20.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:20.855+0000 7fddd31d4640 1 -- 172.21.15.16:0/322851828 shutdown_connections 2024-09-12T02:02:20.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:20.855+0000 7fddd31d4640 1 --2- 172.21.15.16:0/322851828 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fddcc106f20 0x7fddcc10eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:20.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:20.855+0000 7fddd31d4640 1 --2- 172.21.15.16:0/322851828 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fddcc1027a0 0x7fddcc1067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:20.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:20.855+0000 7fddd31d4640 1 --2- 172.21.15.16:0/322851828 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fddcc101df0 0x7fddcc1021d0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:20.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:20.855+0000 7fddd31d4640 1 -- 172.21.15.16:0/322851828 >> 172.21.15.16:0/322851828 conn(0x7fddcc0fd8b0 msgr2=0x7fddcc0ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:02:20.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:20.855+0000 7fddd31d4640 1 -- 172.21.15.16:0/322851828 shutdown_connections 2024-09-12T02:02:20.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:20.855+0000 7fddd31d4640 1 -- 172.21.15.16:0/322851828 wait complete. 2024-09-12T02:02:20.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:20.856+0000 7fddd31d4640 1 Processor -- start 2024-09-12T02:02:20.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:20.856+0000 7fddd31d4640 1 -- start start 2024-09-12T02:02:20.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:20.856+0000 7fddd31d4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fddcc101df0 0x7fddcc198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:20.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:20.856+0000 7fddd31d4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fddcc1027a0 0x7fddcc198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:20.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:20.856+0000 7fddd31d4640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fddcc106f20 0x7fddcc19ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:20.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:20.856+0000 7fddd31d4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fddcc111f80 con 0x7fddcc1027a0 2024-09-12T02:02:20.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:20.856+0000 7fddd31d4640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fddcc111e00 con 0x7fddcc101df0 2024-09-12T02:02:20.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:20.857+0000 7fddd31d4640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fddcc112100 con 0x7fddcc106f20 2024-09-12T02:02:20.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:20.857+0000 7fddcbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fddcc1027a0 0x7fddcc198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:20.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:20.857+0000 7fddd0f70640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fddcc101df0 0x7fddcc198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:20.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:20.857+0000 7fddcbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fddcc1027a0 0x7fddcc198940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33922/0 (socket says 172.21.15.16:33922) 2024-09-12T02:02:20.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:20.857+0000 7fddcbfff640 1 -- 172.21.15.16:0/1539203657 learned_addr learned my addr 172.21.15.16:0/1539203657 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:02:20.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:20.857+0000 7fddd1771640 1 --2- 172.21.15.16:0/1539203657 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fddcc106f20 0x7fddcc19ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:20.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:20.857+0000 7fddcbfff640 1 -- 172.21.15.16:0/1539203657 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fddcc106f20 msgr2=0x7fddcc19ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:20.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:20.857+0000 7fddcbfff640 1 --2- 172.21.15.16:0/1539203657 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fddcc106f20 0x7fddcc19ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:20.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:20.857+0000 7fddcbfff640 1 -- 172.21.15.16:0/1539203657 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fddcc101df0 msgr2=0x7fddcc198400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:20.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:20.857+0000 7fddcbfff640 1 --2- 172.21.15.16:0/1539203657 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fddcc101df0 0x7fddcc198400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:20.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:20.857+0000 7fddcbfff640 1 -- 172.21.15.16:0/1539203657 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddcc19d3b0 con 0x7fddcc1027a0 2024-09-12T02:02:20.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:20.857+0000 7fddcbfff640 1 --2- 172.21.15.16:0/1539203657 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fddcc1027a0 0x7fddcc198940 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fddb800e9f0 tx=0x7fddb800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:02:20.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:20.857+0000 7fddc9ffb640 1 -- 172.21.15.16:0/1539203657 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddb800cd80 con 0x7fddcc1027a0 2024-09-12T02:02:20.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:20.857+0000 7fddc9ffb640 1 -- 172.21.15.16:0/1539203657 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fddb8010910 con 0x7fddcc1027a0 2024-09-12T02:02:20.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:20.857+0000 7fddd31d4640 1 -- 172.21.15.16:0/1539203657 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddcc19d6a0 con 0x7fddcc1027a0 2024-09-12T02:02:20.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:20.857+0000 7fddc9ffb640 1 -- 172.21.15.16:0/1539203657 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddb8018990 con 0x7fddcc1027a0 2024-09-12T02:02:20.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:20.858+0000 7fddd31d4640 1 -- 172.21.15.16:0/1539203657 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddcc1a4f80 con 0x7fddcc1027a0 2024-09-12T02:02:20.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:20.859+0000 7fddd31d4640 1 -- 172.21.15.16:0/1539203657 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fddcc1021d0 con 0x7fddcc1027a0 2024-09-12T02:02:20.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:20.860+0000 7fddc9ffb640 1 -- 172.21.15.16:0/1539203657 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fddb8010ac0 con 0x7fddcc1027a0 2024-09-12T02:02:20.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:20.861+0000 7fddc9ffb640 1 --2- 172.21.15.16:0/1539203657 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fddbc0a2c00 0x7fddbc0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:20.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:20.861+0000 7fddd0f70640 1 --2- 172.21.15.16:0/1539203657 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fddbc0a2c00 0x7fddbc0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:20.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:20.861+0000 7fddc9ffb640 1 -- 172.21.15.16:0/1539203657 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fddb80cf210 con 0x7fddcc1027a0 2024-09-12T02:02:20.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:20.862+0000 7fddd0f70640 1 --2- 172.21.15.16:0/1539203657 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fddbc0a2c00 0x7fddbc0a50c0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fddc0009230 tx=0x7fddc00058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:02:20.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:20.865+0000 7fddc9ffb640 1 -- 172.21.15.16:0/1539203657 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fddb809bbc0 con 0x7fddcc1027a0 2024-09-12T02:02:20.907 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:02:21.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:21.105+0000 7fddd31d4640 1 -- 172.21.15.16:0/1539203657 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fddcc02d7c0 con 0x7fddcc1027a0 2024-09-12T02:02:21.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:21.106+0000 7fddc9ffb640 1 -- 172.21.15.16:0/1539203657 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11911 (secure 0 0 0) 0x7fddb809fec0 con 0x7fddcc1027a0 2024-09-12T02:02:21.106 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T02:02:21.107 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":25,"btime":"2024-09-12T02:01:37:192021+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3279123279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3279123279},{"type":"v1","addr":"172.21.15.16:6835","nonce":3279123279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4119452382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4119452382},{"type":"v1","addr":"172.21.15.16:6837","nonce":4119452382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/3263823165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3263823165},{"type":"v1","addr":"172.21.15.16:6839","nonce":3263823165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/1510290732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":1510290732},{"type":"v1","addr":"172.21.15.16:6841","nonce":1510290732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/279674603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":279674603},{"type":"v1","addr":"172.21.15.33:6833","nonce":279674603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3555728028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3555728028},{"type":"v1","addr":"172.21.15.33:6835","nonce":3555728028}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T01:59:56.117608+0000","modified":"2024-09-12T02:01:36.294760+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24482,"mds_1":24481,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6833/1563374106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1563374106},{"type":"v1","addr":"172.21.15.184:6833","nonce":1563374106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6837/1892833868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1892833868},{"type":"v1","addr":"172.21.15.33:6837","nonce":1892833868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/1457679686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1457679686},{"type":"v1","addr":"172.21.15.184:6835","nonce":1457679686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.184:6837/1793393087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":1793393087},{"type":"v1","addr":"172.21.15.184:6837","nonce":1793393087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.184:6839/4203347941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":4203347941},{"type":"v1","addr":"172.21.15.184:6839","nonce":4203347941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.33:6839/4136670204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":4136670204},{"type":"v1","addr":"172.21.15.33:6839","nonce":4136670204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24481,24482,24469]},"id":1}]} 2024-09-12T02:02:21.107 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 25 2024-09-12T02:02:21.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:21.108+0000 7fddd31d4640 1 -- 172.21.15.16:0/1539203657 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fddbc0a2c00 msgr2=0x7fddbc0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:21.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:21.108+0000 7fddd31d4640 1 --2- 172.21.15.16:0/1539203657 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fddbc0a2c00 0x7fddbc0a50c0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fddc0009230 tx=0x7fddc00058d0 comp rx=0 tx=0).stop 2024-09-12T02:02:21.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:21.108+0000 7fddd31d4640 1 -- 172.21.15.16:0/1539203657 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fddcc1027a0 msgr2=0x7fddcc198940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:21.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:21.108+0000 7fddd31d4640 1 --2- 172.21.15.16:0/1539203657 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fddcc1027a0 0x7fddcc198940 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fddb800e9f0 tx=0x7fddb800eeb0 comp rx=0 tx=0).stop 2024-09-12T02:02:21.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:21.108+0000 7fddd31d4640 1 -- 172.21.15.16:0/1539203657 shutdown_connections 2024-09-12T02:02:21.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:21.108+0000 7fddd31d4640 1 --2- 172.21.15.16:0/1539203657 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fddbc0a2c00 0x7fddbc0a50c0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:21.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:21.108+0000 7fddd31d4640 1 --2- 172.21.15.16:0/1539203657 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fddcc106f20 0x7fddcc19ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:21.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:21.108+0000 7fddd31d4640 1 --2- 172.21.15.16:0/1539203657 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fddcc1027a0 0x7fddcc198940 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:21.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:21.108+0000 7fddd31d4640 1 --2- 172.21.15.16:0/1539203657 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fddcc101df0 0x7fddcc198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:21.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:21.108+0000 7fddd31d4640 1 -- 172.21.15.16:0/1539203657 >> 172.21.15.16:0/1539203657 conn(0x7fddcc0fd8b0 msgr2=0x7fddcc0feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:02:21.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:21.108+0000 7fddd31d4640 1 -- 172.21.15.16:0/1539203657 shutdown_connections 2024-09-12T02:02:21.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:21.109+0000 7fddd31d4640 1 -- 172.21.15.16:0/1539203657 wait complete. 2024-09-12T02:02:21.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:02:21 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/1539203657' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:02:21.668 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:02:21 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/1539203657' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:02:21.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:02:21 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/1539203657' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:02:21.840 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24482 quiesce db / --timeout 90 --expiration 144.5 --await-for 45.0' 2024-09-12T02:02:21.841 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24482 quiesce db / --timeout 90 --expiration 144.5 --await-for 45.0 2024-09-12T02:02:22.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:22.013+0000 7fb78dcd5640 1 -- 172.21.15.16:0/1260363056 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb788102ef0 msgr2=0x7fb78810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:22.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:22.013+0000 7fb78dcd5640 1 --2- 172.21.15.16:0/1260363056 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb788102ef0 0x7fb78810da40 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb780009cd0 tx=0x7fb78002a970 comp rx=0 tx=0).stop 2024-09-12T02:02:22.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:22.013+0000 7fb78dcd5640 1 -- 172.21.15.16:0/1260363056 shutdown_connections 2024-09-12T02:02:22.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:22.013+0000 7fb78dcd5640 1 --2- 172.21.15.16:0/1260363056 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb788102ef0 0x7fb78810da40 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:22.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:22.013+0000 7fb78dcd5640 1 --2- 172.21.15.16:0/1260363056 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb788102570 0x7fb7881029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:22.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:22.013+0000 7fb78dcd5640 1 --2- 172.21.15.16:0/1260363056 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb788100e20 0x7fb788101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:22.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:22.013+0000 7fb78dcd5640 1 -- 172.21.15.16:0/1260363056 >> 172.21.15.16:0/1260363056 conn(0x7fb7880fc7d0 msgr2=0x7fb7880febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:02:22.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:22.013+0000 7fb78dcd5640 1 -- 172.21.15.16:0/1260363056 shutdown_connections 2024-09-12T02:02:22.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:22.013+0000 7fb78dcd5640 1 -- 172.21.15.16:0/1260363056 wait complete. 2024-09-12T02:02:22.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:22.013+0000 7fb78dcd5640 1 Processor -- start 2024-09-12T02:02:22.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:22.013+0000 7fb78dcd5640 1 -- start start 2024-09-12T02:02:22.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:22.014+0000 7fb78dcd5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb788100e20 0x7fb788198610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:22.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:22.014+0000 7fb78dcd5640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb788102570 0x7fb788198b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:22.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:22.014+0000 7fb78dcd5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb788102ef0 0x7fb78819cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:22.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:22.014+0000 7fb78dcd5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb788110f80 con 0x7fb788100e20 2024-09-12T02:02:22.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:22.014+0000 7fb78dcd5640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb788110e00 con 0x7fb788102ef0 2024-09-12T02:02:22.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:22.014+0000 7fb78dcd5640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb788111100 con 0x7fb788102570 2024-09-12T02:02:22.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:22.014+0000 7fb7877fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb788100e20 0x7fb788198610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:22.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:22.014+0000 7fb7877fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb788100e20 0x7fb788198610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40454/0 (socket says 172.21.15.16:40454) 2024-09-12T02:02:22.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:22.014+0000 7fb7877fe640 1 -- 172.21.15.16:0/2649071618 learned_addr learned my addr 172.21.15.16:0/2649071618 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:02:22.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:22.014+0000 7fb787fff640 1 --2- 172.21.15.16:0/2649071618 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb788102ef0 0x7fb78819cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:22.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:22.014+0000 7fb786ffd640 1 --2- 172.21.15.16:0/2649071618 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb788102570 0x7fb788198b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:22.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:22.014+0000 7fb7877fe640 1 -- 172.21.15.16:0/2649071618 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb788102570 msgr2=0x7fb788198b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:22.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:22.014+0000 7fb7877fe640 1 --2- 172.21.15.16:0/2649071618 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb788102570 0x7fb788198b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:22.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:22.014+0000 7fb7877fe640 1 -- 172.21.15.16:0/2649071618 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb788102ef0 msgr2=0x7fb78819cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:22.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:22.014+0000 7fb7877fe640 1 --2- 172.21.15.16:0/2649071618 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb788102ef0 0x7fb78819cee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:22.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:22.014+0000 7fb7877fe640 1 -- 172.21.15.16:0/2649071618 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb78819d5c0 con 0x7fb788100e20 2024-09-12T02:02:22.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:22.014+0000 7fb787fff640 1 --2- 172.21.15.16:0/2649071618 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb788102ef0 0x7fb78819cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:02:22.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:22.015+0000 7fb7877fe640 1 --2- 172.21.15.16:0/2649071618 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb788100e20 0x7fb788198610 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fb77800ea60 tx=0x7fb77800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:02:22.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:22.015+0000 7fb784ff9640 1 -- 172.21.15.16:0/2649071618 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb77800cde0 con 0x7fb788100e20 2024-09-12T02:02:22.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:22.015+0000 7fb78dcd5640 1 -- 172.21.15.16:0/2649071618 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb78819d8b0 con 0x7fb788100e20 2024-09-12T02:02:22.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:22.015+0000 7fb784ff9640 1 -- 172.21.15.16:0/2649071618 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7780035f0 con 0x7fb788100e20 2024-09-12T02:02:22.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:22.016+0000 7fb784ff9640 1 -- 172.21.15.16:0/2649071618 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb778010670 con 0x7fb788100e20 2024-09-12T02:02:22.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:22.016+0000 7fb78dcd5640 1 -- 172.21.15.16:0/2649071618 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7881a5190 con 0x7fb788100e20 2024-09-12T02:02:22.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:22.017+0000 7fb78dcd5640 1 -- 172.21.15.16:0/2649071618 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7881021b0 con 0x7fb788100e20 2024-09-12T02:02:22.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:22.018+0000 7fb784ff9640 1 -- 172.21.15.16:0/2649071618 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fb778004000 con 0x7fb788100e20 2024-09-12T02:02:22.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:22.018+0000 7fb784ff9640 1 --2- 172.21.15.16:0/2649071618 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fb7700a2ad0 0x7fb7700a4f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:22.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:22.019+0000 7fb784ff9640 1 -- 172.21.15.16:0/2649071618 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fb7780cf1e0 con 0x7fb788100e20 2024-09-12T02:02:22.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:22.019+0000 7fb786ffd640 1 --2- 172.21.15.16:0/2649071618 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fb7700a2ad0 0x7fb7700a4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:22.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:22.019+0000 7fb786ffd640 1 --2- 172.21.15.16:0/2649071618 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fb7700a2ad0 0x7fb7700a4f90 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fb7881063d0 tx=0x7fb77c00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:02:22.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:22.027+0000 7fb784ff9640 1 -- 172.21.15.16:0/2649071618 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb77809c1e0 con 0x7fb788100e20 2024-09-12T02:02:22.126 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:02:22.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:22.263+0000 7fb78dcd5640 1 -- 172.21.15.16:0/2649071618 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb788199970 con 0x7fb788100e20 2024-09-12T02:02:22.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:22.264+0000 7fb784ff9640 1 -- 172.21.15.16:0/2649071618 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11911 (secure 0 0 0) 0x7fb77809bae0 con 0x7fb788100e20 2024-09-12T02:02:22.264 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T02:02:22.264 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":25,"btime":"2024-09-12T02:01:37:192021+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3279123279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3279123279},{"type":"v1","addr":"172.21.15.16:6835","nonce":3279123279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4119452382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4119452382},{"type":"v1","addr":"172.21.15.16:6837","nonce":4119452382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/3263823165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3263823165},{"type":"v1","addr":"172.21.15.16:6839","nonce":3263823165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/1510290732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":1510290732},{"type":"v1","addr":"172.21.15.16:6841","nonce":1510290732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/279674603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":279674603},{"type":"v1","addr":"172.21.15.33:6833","nonce":279674603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3555728028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3555728028},{"type":"v1","addr":"172.21.15.33:6835","nonce":3555728028}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T01:59:56.117608+0000","modified":"2024-09-12T02:01:36.294760+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24482,"mds_1":24481,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6833/1563374106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1563374106},{"type":"v1","addr":"172.21.15.184:6833","nonce":1563374106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6837/1892833868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1892833868},{"type":"v1","addr":"172.21.15.33:6837","nonce":1892833868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/1457679686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1457679686},{"type":"v1","addr":"172.21.15.184:6835","nonce":1457679686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.184:6837/1793393087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":1793393087},{"type":"v1","addr":"172.21.15.184:6837","nonce":1793393087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.184:6839/4203347941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":4203347941},{"type":"v1","addr":"172.21.15.184:6839","nonce":4203347941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.33:6839/4136670204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":4136670204},{"type":"v1","addr":"172.21.15.33:6839","nonce":4136670204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24481,24482,24469]},"id":1}]} 2024-09-12T02:02:22.265 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 25 2024-09-12T02:02:22.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:22.266+0000 7fb78dcd5640 1 -- 172.21.15.16:0/2649071618 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fb7700a2ad0 msgr2=0x7fb7700a4f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:22.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:22.266+0000 7fb78dcd5640 1 --2- 172.21.15.16:0/2649071618 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fb7700a2ad0 0x7fb7700a4f90 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fb7881063d0 tx=0x7fb77c00b390 comp rx=0 tx=0).stop 2024-09-12T02:02:22.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:22.266+0000 7fb78dcd5640 1 -- 172.21.15.16:0/2649071618 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb788100e20 msgr2=0x7fb788198610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:22.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:22.266+0000 7fb78dcd5640 1 --2- 172.21.15.16:0/2649071618 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb788100e20 0x7fb788198610 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fb77800ea60 tx=0x7fb77800ef20 comp rx=0 tx=0).stop 2024-09-12T02:02:22.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:22.266+0000 7fb78dcd5640 1 -- 172.21.15.16:0/2649071618 shutdown_connections 2024-09-12T02:02:22.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:22.266+0000 7fb78dcd5640 1 --2- 172.21.15.16:0/2649071618 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fb7700a2ad0 0x7fb7700a4f90 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:22.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:22.266+0000 7fb78dcd5640 1 --2- 172.21.15.16:0/2649071618 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb788102ef0 0x7fb78819cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:22.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:22.266+0000 7fb78dcd5640 1 --2- 172.21.15.16:0/2649071618 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb788102570 0x7fb788198b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:22.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:22.266+0000 7fb78dcd5640 1 --2- 172.21.15.16:0/2649071618 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb788100e20 0x7fb788198610 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:22.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:22.266+0000 7fb78dcd5640 1 -- 172.21.15.16:0/2649071618 >> 172.21.15.16:0/2649071618 conn(0x7fb7880fc7d0 msgr2=0x7fb78806a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:02:22.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:22.266+0000 7fb78dcd5640 1 -- 172.21.15.16:0/2649071618 shutdown_connections 2024-09-12T02:02:22.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:22.266+0000 7fb78dcd5640 1 -- 172.21.15.16:0/2649071618 wait complete. 2024-09-12T02:02:22.397 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:02:22 smithi016 ceph-mon[30947]: pgmap v348: 97 pgs: 97 active+clean; 660 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-09-12T02:02:22.397 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:02:22 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/2649071618' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:02:22.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:02:22 smithi184 ceph-mon[35593]: pgmap v348: 97 pgs: 97 active+clean; 660 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-09-12T02:02:22.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:02:22 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/2649071618' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:02:22.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:02:22 smithi033 ceph-mon[35332]: pgmap v348: 97 pgs: 97 active+clean; 660 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-09-12T02:02:22.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:02:22 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/2649071618' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:02:23.896 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.896+0000 7f48752d0640 1 -- 172.21.15.16:0/326086298 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4870101c40 msgr2=0x7f48701020c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:23.897 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.896+0000 7f48752d0640 1 --2- 172.21.15.16:0/326086298 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4870101c40 0x7f48701020c0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f4858009900 tx=0x7f485802a410 comp rx=0 tx=0).stop 2024-09-12T02:02:23.897 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.896+0000 7f48752d0640 1 -- 172.21.15.16:0/326086298 shutdown_connections 2024-09-12T02:02:23.897 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.896+0000 7f48752d0640 1 --2- 172.21.15.16:0/326086298 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4870102600 0x7f4870108e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:23.897 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.896+0000 7f48752d0640 1 --2- 172.21.15.16:0/326086298 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4870101c40 0x7f48701020c0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:23.897 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.896+0000 7f48752d0640 1 --2- 172.21.15.16:0/326086298 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4870100a50 0x7f4870100e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:23.897 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.896+0000 7f48752d0640 1 -- 172.21.15.16:0/326086298 >> 172.21.15.16:0/326086298 conn(0x7f48700fc1b0 msgr2=0x7f48700fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:02:23.897 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.896+0000 7f48752d0640 1 -- 172.21.15.16:0/326086298 shutdown_connections 2024-09-12T02:02:23.897 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.896+0000 7f48752d0640 1 -- 172.21.15.16:0/326086298 wait complete. 2024-09-12T02:02:23.898 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.897+0000 7f48752d0640 1 Processor -- start 2024-09-12T02:02:23.898 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.897+0000 7f48752d0640 1 -- start start 2024-09-12T02:02:23.913 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.897+0000 7f48752d0640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4870100a50 0x7f4870196d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:23.913 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.897+0000 7f48752d0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4870101c40 0x7f4870197270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:23.913 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.897+0000 7f48752d0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4870102600 0x7f487019e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:23.913 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.897+0000 7f48752d0640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f487010c720 con 0x7f4870101c40 2024-09-12T02:02:23.914 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.897+0000 7f48752d0640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f487010c5a0 con 0x7f4870102600 2024-09-12T02:02:23.914 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.897+0000 7f48752d0640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f487010c8a0 con 0x7f4870100a50 2024-09-12T02:02:23.915 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.897+0000 7f4867fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4870101c40 0x7f4870197270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:23.915 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.897+0000 7f4867fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4870101c40 0x7f4870197270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40464/0 (socket says 172.21.15.16:40464) 2024-09-12T02:02:23.915 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.897+0000 7f4867fff640 1 -- 172.21.15.16:0/3458853402 learned_addr learned my addr 172.21.15.16:0/3458853402 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:02:23.915 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.897+0000 7f486f7fe640 1 --2- 172.21.15.16:0/3458853402 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4870102600 0x7f487019e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:23.915 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.898+0000 7f486effd640 1 --2- 172.21.15.16:0/3458853402 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4870100a50 0x7f4870196d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:23.915 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.898+0000 7f4867fff640 1 -- 172.21.15.16:0/3458853402 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4870100a50 msgr2=0x7f4870196d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:23.915 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.898+0000 7f4867fff640 1 --2- 172.21.15.16:0/3458853402 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4870100a50 0x7f4870196d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:23.915 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.898+0000 7f4867fff640 1 -- 172.21.15.16:0/3458853402 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4870102600 msgr2=0x7f487019e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:23.915 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.898+0000 7f4867fff640 1 --2- 172.21.15.16:0/3458853402 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4870102600 0x7f487019e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:23.915 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.898+0000 7f4867fff640 1 -- 172.21.15.16:0/3458853402 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f487019e940 con 0x7f4870101c40 2024-09-12T02:02:23.915 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.898+0000 7f486f7fe640 1 --2- 172.21.15.16:0/3458853402 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4870102600 0x7f487019e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:02:23.915 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.898+0000 7f4867fff640 1 --2- 172.21.15.16:0/3458853402 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4870101c40 0x7f4870197270 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f4858031440 tx=0x7f4858032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:02:23.916 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.898+0000 7f486cff9640 1 -- 172.21.15.16:0/3458853402 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f485803f030 con 0x7f4870101c40 2024-09-12T02:02:23.916 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.898+0000 7f48752d0640 1 -- 172.21.15.16:0/3458853402 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f487019ebd0 con 0x7f4870101c40 2024-09-12T02:02:23.916 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.898+0000 7f486cff9640 1 -- 172.21.15.16:0/3458853402 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4858003ce0 con 0x7f4870101c40 2024-09-12T02:02:23.916 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.898+0000 7f486cff9640 1 -- 172.21.15.16:0/3458853402 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4858035070 con 0x7f4870101c40 2024-09-12T02:02:23.916 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.898+0000 7f48752d0640 1 -- 172.21.15.16:0/3458853402 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f487019f0c0 con 0x7f4870101c40 2024-09-12T02:02:23.916 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.901+0000 7f486cff9640 1 -- 172.21.15.16:0/3458853402 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f4858042030 con 0x7f4870101c40 2024-09-12T02:02:23.916 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.901+0000 7f487806b740 1 Processor -- start 2024-09-12T02:02:23.916 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.901+0000 7f487806b740 1 -- start start 2024-09-12T02:02:23.916 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.901+0000 7f487806b740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557ee90b3bb0 0x557ee90b3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:23.916 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.901+0000 7f487806b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ee90b44f0 0x557ee90b8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:23.917 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.901+0000 7f487806b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557ee90b9580 0x557ee90bba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:23.917 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.901+0000 7f487806b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x557ee90152f0 con 0x557ee90b44f0 2024-09-12T02:02:23.917 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.901+0000 7f487806b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x557ee8fbd270 con 0x557ee90b9580 2024-09-12T02:02:23.917 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.901+0000 7f487806b740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x557ee90112b0 con 0x557ee90b3bb0 2024-09-12T02:02:23.917 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.902+0000 7f486effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ee90b44f0 0x557ee90b8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:23.917 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.902+0000 7f486effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ee90b44f0 0x557ee90b8fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40476/0 (socket says 172.21.15.16:40476) 2024-09-12T02:02:23.917 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.902+0000 7f486effd640 1 -- 172.21.15.16:0/1340867934 learned_addr learned my addr 172.21.15.16:0/1340867934 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:02:23.917 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.902+0000 7f4867fff640 1 --2- 172.21.15.16:0/1340867934 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557ee90b9580 0x557ee90bba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:23.917 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.902+0000 7f486f7fe640 1 --2- 172.21.15.16:0/1340867934 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557ee90b3bb0 0x557ee90b3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:23.917 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.902+0000 7f486effd640 1 -- 172.21.15.16:0/1340867934 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557ee90b3bb0 msgr2=0x557ee90b3fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:23.918 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.902+0000 7f486effd640 1 --2- 172.21.15.16:0/1340867934 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557ee90b3bb0 0x557ee90b3fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:23.918 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.902+0000 7f486effd640 1 -- 172.21.15.16:0/1340867934 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557ee90b9580 msgr2=0x557ee90bba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:23.918 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.902+0000 7f486effd640 1 --2- 172.21.15.16:0/1340867934 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557ee90b9580 0x557ee90bba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:23.918 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.902+0000 7f486cff9640 1 --2- 172.21.15.16:0/3458853402 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f485c0a2c30 0x7f485c0a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:23.918 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.902+0000 7f486effd640 1 -- 172.21.15.16:0/1340867934 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ee8fbc340 con 0x557ee90b44f0 2024-09-12T02:02:23.918 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.902+0000 7f486cff9640 1 -- 172.21.15.16:0/3458853402 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f48580ef6f0 con 0x7f4870101c40 2024-09-12T02:02:23.918 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.902+0000 7f486f7fe640 1 --2- 172.21.15.16:0/3458853402 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f485c0a2c30 0x7f485c0a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:23.918 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.902+0000 7f486effd640 1 --2- 172.21.15.16:0/1340867934 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ee90b44f0 0x557ee90b8fb0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f486000bc60 tx=0x7f486000f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b598e2753a165dee server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:02:23.918 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.903+0000 7f48657fa640 1 -- 172.21.15.16:0/1340867934 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4860015070 con 0x557ee90b44f0 2024-09-12T02:02:23.918 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.903+0000 7f48657fa640 1 -- 172.21.15.16:0/1340867934 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f486001a940 con 0x557ee90b44f0 2024-09-12T02:02:23.918 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.903+0000 7f48657fa640 1 -- 172.21.15.16:0/1340867934 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48600119f0 con 0x557ee90b44f0 2024-09-12T02:02:23.918 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.903+0000 7f487806b740 1 -- 172.21.15.16:0/1340867934 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ee90b44f0 msgr2=0x557ee90b8fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:23.918 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.903+0000 7f487806b740 1 --2- 172.21.15.16:0/1340867934 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ee90b44f0 0x557ee90b8fb0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f486000bc60 tx=0x7f486000f730 comp rx=0 tx=0).stop 2024-09-12T02:02:23.919 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.903+0000 7f486f7fe640 1 --2- 172.21.15.16:0/3458853402 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f485c0a2c30 0x7f485c0a50f0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f4868005fd0 tx=0x7f4868009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:02:23.919 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.903+0000 7f487806b740 1 -- 172.21.15.16:0/1340867934 shutdown_connections 2024-09-12T02:02:23.919 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.903+0000 7f487806b740 1 --2- 172.21.15.16:0/1340867934 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557ee90b9580 0x557ee90bba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:23.919 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.903+0000 7f487806b740 1 --2- 172.21.15.16:0/1340867934 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ee90b44f0 0x557ee90b8fb0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:23.919 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.903+0000 7f487806b740 1 --2- 172.21.15.16:0/1340867934 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557ee90b3bb0 0x557ee90b3fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:23.919 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.904+0000 7f487806b740 1 -- 172.21.15.16:0/1340867934 >> 172.21.15.16:0/1340867934 conn(0x557ee90ad010 msgr2=0x557ee90ad410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:02:23.919 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.904+0000 7f487806b740 1 -- 172.21.15.16:0/1340867934 shutdown_connections 2024-09-12T02:02:23.919 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.904+0000 7f487806b740 1 -- 172.21.15.16:0/1340867934 wait complete. 2024-09-12T02:02:23.919 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.904+0000 7f487806b740 1 -- start start 2024-09-12T02:02:23.919 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.904+0000 7f487806b740 1 Processor -- start 2024-09-12T02:02:23.919 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.904+0000 7f487806b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557ee90b3bb0 0x557ee90bde00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:23.919 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.904+0000 7f487806b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ee90b44f0 0x557ee90be340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:23.919 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.904+0000 7f487806b740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557ee90b9580 0x557ee90c5380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:23.919 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.904+0000 7f487806b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x557ee90bc550 con 0x557ee90b44f0 2024-09-12T02:02:23.920 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.904+0000 7f487806b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x557ee8f9e290 con 0x557ee90b3bb0 2024-09-12T02:02:23.920 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.904+0000 7f487806b740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x557ee90bc6d0 con 0x557ee90b9580 2024-09-12T02:02:23.920 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.904+0000 7f487806b740 10 client.0 _ll_register_callbacks cb 0x557ee8fcf140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:02:23.920 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.904+0000 7f4867fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ee90b44f0 0x557ee90be340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:23.920 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.904+0000 7f4867fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ee90b44f0 0x557ee90be340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40488/0 (socket says 172.21.15.16:40488) 2024-09-12T02:02:23.920 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.904+0000 7f4867fff640 1 -- 172.21.15.16:0/2055096149 learned_addr learned my addr 172.21.15.16:0/2055096149 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:02:23.920 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.904+0000 7f486effd640 1 --2- 172.21.15.16:0/2055096149 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557ee90b3bb0 0x557ee90bde00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:23.920 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.905+0000 7f486f7fe640 1 --2- 172.21.15.16:0/2055096149 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557ee90b9580 0x557ee90c5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:23.920 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.905+0000 7f4867fff640 1 -- 172.21.15.16:0/2055096149 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557ee90b9580 msgr2=0x557ee90c5380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:23.920 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.905+0000 7f4867fff640 1 --2- 172.21.15.16:0/2055096149 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557ee90b9580 0x557ee90c5380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:23.920 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.905+0000 7f4867fff640 1 -- 172.21.15.16:0/2055096149 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557ee90b3bb0 msgr2=0x557ee90bde00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:23.920 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.905+0000 7f4867fff640 1 --2- 172.21.15.16:0/2055096149 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557ee90b3bb0 0x557ee90bde00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:23.920 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.905+0000 7f4867fff640 1 -- 172.21.15.16:0/2055096149 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ee8d28210 con 0x557ee90b44f0 2024-09-12T02:02:23.921 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.905+0000 7f486effd640 1 --2- 172.21.15.16:0/2055096149 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557ee90b3bb0 0x557ee90bde00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:02:23.921 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.905+0000 7f4867fff640 1 --2- 172.21.15.16:0/2055096149 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ee90b44f0 0x557ee90be340 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f48580b8930 tx=0x7f4858009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=544fa8a622550f10 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:02:23.921 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.905+0000 7f48567fc640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:02:23.921 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.905+0000 7f48567fc640 1 -- 172.21.15.16:0/2055096149 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4858049050 con 0x557ee90b44f0 2024-09-12T02:02:23.921 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.905+0000 7f48567fc640 1 -- 172.21.15.16:0/2055096149 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f485803b460 con 0x557ee90b44f0 2024-09-12T02:02:23.921 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.905+0000 7f487806b740 1 -- 172.21.15.16:0/2055096149 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f486001a940 con 0x557ee90b44f0 2024-09-12T02:02:23.921 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.905+0000 7f48567fc640 1 -- 172.21.15.16:0/2055096149 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f48580bb950 con 0x557ee90b44f0 2024-09-12T02:02:23.921 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.905+0000 7f487806b740 10 client.15408 fetch_fsmap learned FSMap version 25 2024-09-12T02:02:23.921 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.905+0000 7f487806b740 1 -- 172.21.15.16:0/2055096149 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x557ee8df4620 con 0x557ee90b44f0 2024-09-12T02:02:23.921 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.906+0000 7f48567fc640 1 -- 172.21.15.16:0/2055096149 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f486001a940 con 0x557ee90b44f0 2024-09-12T02:02:23.921 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.906+0000 7f48567fc640 1 -- 172.21.15.16:0/2055096149 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f48580bc250 con 0x557ee90b44f0 2024-09-12T02:02:23.921 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.906+0000 7f487806b740 10 client.15408 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:02:23.921 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.906+0000 7f487806b740 10 client.15408 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:02:23.922 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.906+0000 7f487806b740 20 client.15408 populate_metadata read hostname 'smithi016' 2024-09-12T02:02:23.922 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.906+0000 7f487806b740 1 --2- 172.21.15.16:0/2055096149 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x557ee90c99f0 0x557ee90cbe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:23.922 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.906+0000 7f487806b740 4 client.15408 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:02:23.922 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.906+0000 7f487806b740 1 -- 172.21.15.16:0/2055096149 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557ee9022a20 con 0x557ee90c99f0 2024-09-12T02:02:23.922 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.906+0000 7f486effd640 1 --2- 172.21.15.16:0/2055096149 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x557ee90c99f0 0x557ee90cbe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:23.922 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.907+0000 7f48567fc640 10 client.15408 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:02:23.922 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.907+0000 7f486effd640 1 --2- 172.21.15.16:0/2055096149 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x557ee90c99f0 0x557ee90cbe20 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10e89104e56fcd71 server_cookie=87af4842eaa78317 in_seq=0 out_seq=0 2024-09-12T02:02:23.922 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.907+0000 7f48567fc640 1 -- 172.21.15.16:0/2055096149 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557ee9022a20 con 0x557ee90c99f0 2024-09-12T02:02:23.922 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.907+0000 7f48567fc640 10 client.15408 handle_command_reply: tid=0 2024-09-12T02:02:23.922 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.907+0000 7f487806b740 1 client.15408 shutdown 2024-09-12T02:02:23.922 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.907+0000 7f487806b740 1 -- 172.21.15.16:0/2055096149 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ee90b44f0 msgr2=0x557ee90be340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:23.922 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.907+0000 7f487806b740 1 --2- 172.21.15.16:0/2055096149 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ee90b44f0 0x557ee90be340 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f48580b8930 tx=0x7f4858009560 comp rx=0 tx=0).stop 2024-09-12T02:02:23.923 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.907+0000 7f487806b740 1 -- 172.21.15.16:0/2055096149 shutdown_connections 2024-09-12T02:02:23.923 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.907+0000 7f487806b740 1 --2- 172.21.15.16:0/2055096149 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x557ee90c99f0 0x557ee90cbe20 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:23.923 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.907+0000 7f487806b740 1 --2- 172.21.15.16:0/2055096149 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557ee90b9580 0x557ee90c5380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:23.923 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.907+0000 7f487806b740 1 --2- 172.21.15.16:0/2055096149 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ee90b44f0 0x557ee90be340 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:23.923 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.907+0000 7f48567fc640 0 client.15408 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:02:23.923 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.907+0000 7f487806b740 1 --2- 172.21.15.16:0/2055096149 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557ee90b3bb0 0x557ee90bde00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:23.923 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.907+0000 7f487806b740 1 -- 172.21.15.16:0/2055096149 >> 172.21.15.16:0/2055096149 conn(0x557ee90ad010 msgr2=0x557ee90ae820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:02:23.923 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.908+0000 7f487806b740 1 -- 172.21.15.16:0/2055096149 shutdown_connections 2024-09-12T02:02:23.923 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.908+0000 7f487806b740 1 -- 172.21.15.16:0/2055096149 wait complete. 2024-09-12T02:02:23.923 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.908+0000 7f487806b740 20 client.15408 trim_cache size 0 max 16384 2024-09-12T02:02:23.929 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.922+0000 7f487806b740 1 Processor -- start 2024-09-12T02:02:23.929 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.922+0000 7f487806b740 1 -- start start 2024-09-12T02:02:23.929 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.923+0000 7f487806b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ee90b3bb0 0x557ee90ac0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:23.929 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.923+0000 7f487806b740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557ee90b44f0 0x557ee90c5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:23.929 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.923+0000 7f487806b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557ee90b9580 0x557ee90d0720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:23.930 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.923+0000 7f487806b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f48701a94a0 con 0x557ee90b3bb0 2024-09-12T02:02:23.930 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.923+0000 7f487806b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f48701a6d70 con 0x557ee90b9580 2024-09-12T02:02:23.930 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.923+0000 7f487806b740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f48701a4bc0 con 0x557ee90b44f0 2024-09-12T02:02:23.930 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.923+0000 7f486effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ee90b3bb0 0x557ee90ac0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:23.930 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.923+0000 7f486effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ee90b3bb0 0x557ee90ac0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40490/0 (socket says 172.21.15.16:40490) 2024-09-12T02:02:23.930 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.923+0000 7f486effd640 1 -- 172.21.15.16:0/610446026 learned_addr learned my addr 172.21.15.16:0/610446026 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:02:23.930 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.923+0000 7f4867fff640 1 --2- 172.21.15.16:0/610446026 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557ee90b44f0 0x557ee90c5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:23.930 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.923+0000 7f486f7fe640 1 --2- 172.21.15.16:0/610446026 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557ee90b9580 0x557ee90d0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:23.930 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.923+0000 7f486effd640 1 -- 172.21.15.16:0/610446026 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557ee90b44f0 msgr2=0x557ee90c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:23.930 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.923+0000 7f486effd640 1 --2- 172.21.15.16:0/610446026 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557ee90b44f0 0x557ee90c5340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:23.930 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.923+0000 7f486effd640 1 -- 172.21.15.16:0/610446026 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557ee90b9580 msgr2=0x557ee90d0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:23.930 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.923+0000 7f486effd640 1 --2- 172.21.15.16:0/610446026 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557ee90b9580 0x557ee90d0720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:23.930 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.923+0000 7f486effd640 1 -- 172.21.15.16:0/610446026 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ee90b49d0 con 0x557ee90b3bb0 2024-09-12T02:02:23.931 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.923+0000 7f486effd640 1 --2- 172.21.15.16:0/610446026 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ee90b3bb0 0x557ee90ac0b0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f486000bc60 tx=0x7f486000b5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4504c2d063120d84 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:02:23.931 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.924+0000 7f4864ff9640 1 -- 172.21.15.16:0/610446026 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4860015040 con 0x557ee90b3bb0 2024-09-12T02:02:23.931 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.924+0000 7f4864ff9640 1 -- 172.21.15.16:0/610446026 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4860005520 con 0x557ee90b3bb0 2024-09-12T02:02:23.931 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.924+0000 7f4864ff9640 1 -- 172.21.15.16:0/610446026 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4860011970 con 0x557ee90b3bb0 2024-09-12T02:02:23.931 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.924+0000 7f487806b740 1 -- 172.21.15.16:0/610446026 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ee90b3bb0 msgr2=0x557ee90ac0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:23.931 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.924+0000 7f487806b740 1 --2- 172.21.15.16:0/610446026 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ee90b3bb0 0x557ee90ac0b0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f486000bc60 tx=0x7f486000b5a0 comp rx=0 tx=0).stop 2024-09-12T02:02:23.931 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.924+0000 7f487806b740 1 -- 172.21.15.16:0/610446026 shutdown_connections 2024-09-12T02:02:23.931 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.924+0000 7f487806b740 1 --2- 172.21.15.16:0/610446026 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557ee90b9580 0x557ee90d0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:23.931 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.924+0000 7f487806b740 1 --2- 172.21.15.16:0/610446026 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557ee90b44f0 0x557ee90c5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:23.931 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.924+0000 7f487806b740 1 --2- 172.21.15.16:0/610446026 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ee90b3bb0 0x557ee90ac0b0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:23.931 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.924+0000 7f487806b740 1 -- 172.21.15.16:0/610446026 >> 172.21.15.16:0/610446026 conn(0x557ee90ad010 msgr2=0x557ee90ab9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:02:23.931 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.925+0000 7f487806b740 1 -- 172.21.15.16:0/610446026 shutdown_connections 2024-09-12T02:02:23.931 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.925+0000 7f487806b740 1 -- 172.21.15.16:0/610446026 wait complete. 2024-09-12T02:02:23.932 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.925+0000 7f487806b740 1 -- start start 2024-09-12T02:02:23.932 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.925+0000 7f487806b740 1 Processor -- start 2024-09-12T02:02:23.932 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.925+0000 7f487806b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ee90b3bb0 0x557ee90c2b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:23.932 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.925+0000 7f487806b740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557ee90b44f0 0x557ee90c3060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:23.932 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.925+0000 7f487806b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557ee90b9580 0x557ee90c4070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:23.932 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.925+0000 7f487806b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f48701a5230 con 0x557ee90b3bb0 2024-09-12T02:02:23.932 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.925+0000 7f487806b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f48701a66a0 con 0x557ee90b9580 2024-09-12T02:02:23.932 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.925+0000 7f487806b740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f48701a5900 con 0x557ee90b44f0 2024-09-12T02:02:23.932 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.925+0000 7f487806b740 10 client.0 _ll_register_callbacks cb 0x7f4858005b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:02:23.932 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.925+0000 7f486effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ee90b3bb0 0x557ee90c2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:23.933 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.925+0000 7f486effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ee90b3bb0 0x557ee90c2b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40498/0 (socket says 172.21.15.16:40498) 2024-09-12T02:02:23.933 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.925+0000 7f486effd640 1 -- 172.21.15.16:0/865011371 learned_addr learned my addr 172.21.15.16:0/865011371 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:02:23.933 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.926+0000 7f486f7fe640 1 --2- 172.21.15.16:0/865011371 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557ee90b9580 0x557ee90c4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:23.933 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.926+0000 7f4867fff640 1 --2- 172.21.15.16:0/865011371 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557ee90b44f0 0x557ee90c3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:23.933 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.926+0000 7f486effd640 1 -- 172.21.15.16:0/865011371 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557ee90b44f0 msgr2=0x557ee90c3060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:23.933 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.926+0000 7f486effd640 1 --2- 172.21.15.16:0/865011371 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557ee90b44f0 0x557ee90c3060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:23.933 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.926+0000 7f486effd640 1 -- 172.21.15.16:0/865011371 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557ee90b9580 msgr2=0x557ee90c4070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:23.934 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.926+0000 7f486effd640 1 --2- 172.21.15.16:0/865011371 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557ee90b9580 0x557ee90c4070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:23.934 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.926+0000 7f486effd640 1 -- 172.21.15.16:0/865011371 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ee90c4680 con 0x557ee90b3bb0 2024-09-12T02:02:23.934 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.926+0000 7f486f7fe640 1 --2- 172.21.15.16:0/865011371 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557ee90b9580 0x557ee90c4070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:02:23.934 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.926+0000 7f486effd640 1 --2- 172.21.15.16:0/865011371 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ee90b3bb0 0x557ee90c2b20 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f486000b4c0 tx=0x7f486000bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1a7c4d8743d17a0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:02:23.934 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.926+0000 7f48567fc640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:02:23.934 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.926+0000 7f48567fc640 1 -- 172.21.15.16:0/865011371 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f486003d060 con 0x557ee90b3bb0 2024-09-12T02:02:23.934 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.926+0000 7f487806b740 1 -- 172.21.15.16:0/865011371 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4860005520 con 0x557ee90b3bb0 2024-09-12T02:02:23.934 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.926+0000 7f48567fc640 1 -- 172.21.15.16:0/865011371 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f4860011b00 con 0x557ee90b3bb0 2024-09-12T02:02:23.934 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.926+0000 7f48567fc640 1 -- 172.21.15.16:0/865011371 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4860011d40 con 0x557ee90b3bb0 2024-09-12T02:02:23.934 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.926+0000 7f487806b740 10 client.15414 fetch_fsmap learned FSMap version 25 2024-09-12T02:02:23.934 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.926+0000 7f487806b740 1 -- 172.21.15.16:0/865011371 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x557ee91347c0 con 0x557ee90b3bb0 2024-09-12T02:02:23.934 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.927+0000 7f48567fc640 1 -- 172.21.15.16:0/865011371 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4860005520 con 0x557ee90b3bb0 2024-09-12T02:02:23.934 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.927+0000 7f48567fc640 1 -- 172.21.15.16:0/865011371 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f4860042070 con 0x557ee90b3bb0 2024-09-12T02:02:23.935 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.927+0000 7f487806b740 10 client.15414 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:02:23.935 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.927+0000 7f487806b740 10 client.15414 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:02:23.935 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.927+0000 7f487806b740 20 client.15414 populate_metadata read hostname 'smithi016' 2024-09-12T02:02:23.935 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.927+0000 7f487806b740 1 --2- 172.21.15.16:0/865011371 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x557ee9134fa0 0x557ee91373c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:23.935 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.927+0000 7f487806b740 4 client.15414 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:02:23.935 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.927+0000 7f487806b740 1 -- 172.21.15.16:0/865011371 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557ee901cd70 con 0x557ee9134fa0 2024-09-12T02:02:23.935 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.927+0000 7f4867fff640 1 --2- 172.21.15.16:0/865011371 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x557ee9134fa0 0x557ee91373c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:23.935 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.927+0000 7f48567fc640 10 client.15414 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:02:23.935 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.927+0000 7f4867fff640 1 --2- 172.21.15.16:0/865011371 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x557ee9134fa0 0x557ee91373c0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cca3909cda4b86c8 server_cookie=490f4bc87234d2ff in_seq=0 out_seq=0 2024-09-12T02:02:23.936 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.934+0000 7f48567fc640 1 -- 172.21.15.16:0/865011371 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557ee901cd70 con 0x557ee9134fa0 2024-09-12T02:02:23.937 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.934+0000 7f48567fc640 10 client.15414 handle_command_reply: tid=0 2024-09-12T02:02:23.937 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.934+0000 7f487806b740 1 client.15414 shutdown 2024-09-12T02:02:23.937 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.934+0000 7f487806b740 1 -- 172.21.15.16:0/865011371 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ee90b3bb0 msgr2=0x557ee90c2b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:23.937 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.934+0000 7f487806b740 1 --2- 172.21.15.16:0/865011371 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ee90b3bb0 0x557ee90c2b20 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f486000b4c0 tx=0x7f486000bf10 comp rx=0 tx=0).stop 2024-09-12T02:02:23.937 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.934+0000 7f487806b740 1 -- 172.21.15.16:0/865011371 shutdown_connections 2024-09-12T02:02:23.937 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.934+0000 7f487806b740 1 --2- 172.21.15.16:0/865011371 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x557ee9134fa0 0x557ee91373c0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:23.937 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.934+0000 7f487806b740 1 --2- 172.21.15.16:0/865011371 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557ee90b9580 0x557ee90c4070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:23.937 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.934+0000 7f487806b740 1 --2- 172.21.15.16:0/865011371 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557ee90b44f0 0x557ee90c3060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:23.937 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.934+0000 7f487806b740 1 --2- 172.21.15.16:0/865011371 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ee90b3bb0 0x557ee90c2b20 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:23.937 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.934+0000 7f487806b740 1 -- 172.21.15.16:0/865011371 >> 172.21.15.16:0/865011371 conn(0x557ee90ad010 msgr2=0x557ee90ac1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:02:23.937 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.934+0000 7f48567fc640 0 client.15414 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:02:23.938 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.934+0000 7f487806b740 1 -- 172.21.15.16:0/865011371 shutdown_connections 2024-09-12T02:02:23.938 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.934+0000 7f487806b740 1 -- 172.21.15.16:0/865011371 wait complete. 2024-09-12T02:02:23.938 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.934+0000 7f487806b740 20 client.15414 trim_cache size 0 max 16384 2024-09-12T02:02:23.938 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-09-12T02:02:23.938 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-09-12T02:02:23.938 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-09-12T02:02:23.938 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-09-12T02:02:23.939 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.935+0000 7f48752d0640 1 -- 172.21.15.16:0/3458853402 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f485c0a2c30 msgr2=0x7f485c0a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:23.939 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.935+0000 7f48752d0640 1 --2- 172.21.15.16:0/3458853402 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f485c0a2c30 0x7f485c0a50f0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f4868005fd0 tx=0x7f4868009500 comp rx=0 tx=0).stop 2024-09-12T02:02:23.940 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.935+0000 7f48752d0640 1 -- 172.21.15.16:0/3458853402 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4870101c40 msgr2=0x7f4870197270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:23.940 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.935+0000 7f48752d0640 1 --2- 172.21.15.16:0/3458853402 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4870101c40 0x7f4870197270 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f4858031440 tx=0x7f4858032ce0 comp rx=0 tx=0).stop 2024-09-12T02:02:23.940 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.935+0000 7f48752d0640 1 -- 172.21.15.16:0/3458853402 shutdown_connections 2024-09-12T02:02:23.940 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.935+0000 7f48752d0640 1 --2- 172.21.15.16:0/3458853402 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f485c0a2c30 0x7f485c0a50f0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:23.940 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.935+0000 7f48752d0640 1 --2- 172.21.15.16:0/3458853402 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4870102600 0x7f487019e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:23.940 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.935+0000 7f48752d0640 1 --2- 172.21.15.16:0/3458853402 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4870101c40 0x7f4870197270 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:23.940 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.935+0000 7f48752d0640 1 --2- 172.21.15.16:0/3458853402 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4870100a50 0x7f4870196d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:23.940 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.935+0000 7f48752d0640 1 -- 172.21.15.16:0/3458853402 >> 172.21.15.16:0/3458853402 conn(0x7f48700fc1b0 msgr2=0x7f4870104f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:02:23.940 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.935+0000 7f48752d0640 1 -- 172.21.15.16:0/3458853402 shutdown_connections 2024-09-12T02:02:23.940 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:23.935+0000 7f48752d0640 1 -- 172.21.15.16:0/3458853402 wait complete. 2024-09-12T02:02:24.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:02:24 smithi184 ceph-mon[35593]: pgmap v349: 97 pgs: 97 active+clean; 660 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-09-12T02:02:24.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:02:24 smithi033 ceph-mon[35332]: pgmap v349: 97 pgs: 97 active+clean; 660 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-09-12T02:02:24.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:02:24 smithi016 ceph-mon[30947]: pgmap v349: 97 pgs: 97 active+clean; 660 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-09-12T02:02:25.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:02:25 smithi184 ceph-mon[35593]: pgmap v350: 97 pgs: 97 active+clean; 661 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-09-12T02:02:25.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:02:25 smithi033 ceph-mon[35332]: pgmap v350: 97 pgs: 97 active+clean; 661 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-09-12T02:02:25.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:02:25 smithi016 ceph-mon[30947]: pgmap v350: 97 pgs: 97 active+clean; 661 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-09-12T02:02:26.609 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-12T02:02:26.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.869+0000 7f54de55e640 1 -- 172.21.15.16:0/3215898131 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54d8102ef0 msgr2=0x7f54d810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:26.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.869+0000 7f54de55e640 1 --2- 172.21.15.16:0/3215898131 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54d8102ef0 0x7f54d810da40 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f54d0009ca0 tx=0x7f54d002a950 comp rx=0 tx=0).stop 2024-09-12T02:02:26.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.869+0000 7f54de55e640 1 -- 172.21.15.16:0/3215898131 shutdown_connections 2024-09-12T02:02:26.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.869+0000 7f54de55e640 1 --2- 172.21.15.16:0/3215898131 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54d8102ef0 0x7f54d810da40 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:26.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.869+0000 7f54de55e640 1 --2- 172.21.15.16:0/3215898131 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f54d8102570 0x7f54d81029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:26.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.869+0000 7f54de55e640 1 --2- 172.21.15.16:0/3215898131 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f54d8100e20 0x7f54d8101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:26.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.869+0000 7f54de55e640 1 -- 172.21.15.16:0/3215898131 >> 172.21.15.16:0/3215898131 conn(0x7f54d80fc7d0 msgr2=0x7f54d80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:02:26.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.869+0000 7f54de55e640 1 -- 172.21.15.16:0/3215898131 shutdown_connections 2024-09-12T02:02:26.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.869+0000 7f54de55e640 1 -- 172.21.15.16:0/3215898131 wait complete. 2024-09-12T02:02:26.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.870+0000 7f54de55e640 1 Processor -- start 2024-09-12T02:02:26.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.870+0000 7f54de55e640 1 -- start start 2024-09-12T02:02:26.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.871+0000 7f54de55e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f54d8100e20 0x7f54d8107ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:26.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.871+0000 7f54de55e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54d8102570 0x7f54d8108020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:26.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.871+0000 7f54de55e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f54d8102ef0 0x7f54d8108560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:26.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.871+0000 7f54de55e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f54d8110f80 con 0x7f54d8102570 2024-09-12T02:02:26.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.871+0000 7f54de55e640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f54d8110e00 con 0x7f54d8102ef0 2024-09-12T02:02:26.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.871+0000 7f54de55e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f54d8111100 con 0x7f54d8100e20 2024-09-12T02:02:26.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.871+0000 7f54d77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54d8102570 0x7f54d8108020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:26.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.871+0000 7f54d77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54d8102570 0x7f54d8108020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40512/0 (socket says 172.21.15.16:40512) 2024-09-12T02:02:26.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.871+0000 7f54d77fe640 1 -- 172.21.15.16:0/2429712225 learned_addr learned my addr 172.21.15.16:0/2429712225 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:02:26.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.871+0000 7f54d7fff640 1 --2- 172.21.15.16:0/2429712225 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f54d8100e20 0x7f54d8107ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:26.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.871+0000 7f54dcafb640 1 --2- 172.21.15.16:0/2429712225 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f54d8102ef0 0x7f54d8108560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:26.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.871+0000 7f54d77fe640 1 -- 172.21.15.16:0/2429712225 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f54d8100e20 msgr2=0x7f54d8107ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:26.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.871+0000 7f54d77fe640 1 --2- 172.21.15.16:0/2429712225 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f54d8100e20 0x7f54d8107ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:26.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.871+0000 7f54d77fe640 1 -- 172.21.15.16:0/2429712225 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f54d8102ef0 msgr2=0x7f54d8108560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:26.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.871+0000 7f54d77fe640 1 --2- 172.21.15.16:0/2429712225 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f54d8102ef0 0x7f54d8108560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:26.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.871+0000 7f54d77fe640 1 -- 172.21.15.16:0/2429712225 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54d81a6870 con 0x7f54d8102570 2024-09-12T02:02:26.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.871+0000 7f54d77fe640 1 --2- 172.21.15.16:0/2429712225 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54d8102570 0x7f54d8108020 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f54cc00d950 tx=0x7f54cc00dd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:02:26.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.872+0000 7f54d57fa640 1 -- 172.21.15.16:0/2429712225 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54cc017070 con 0x7f54d8102570 2024-09-12T02:02:26.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.872+0000 7f54d57fa640 1 -- 172.21.15.16:0/2429712225 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54cc00de80 con 0x7f54d8102570 2024-09-12T02:02:26.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.872+0000 7f54de55e640 1 -- 172.21.15.16:0/2429712225 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54d81a6b60 con 0x7f54d8102570 2024-09-12T02:02:26.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.872+0000 7f54d57fa640 1 -- 172.21.15.16:0/2429712225 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54cc00f620 con 0x7f54d8102570 2024-09-12T02:02:26.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.872+0000 7f54de55e640 1 -- 172.21.15.16:0/2429712225 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54d81a70a0 con 0x7f54d8102570 2024-09-12T02:02:26.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.875+0000 7f54d57fa640 1 -- 172.21.15.16:0/2429712225 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f54cc009c30 con 0x7f54d8102570 2024-09-12T02:02:26.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.875+0000 7f54e0937740 1 Processor -- start 2024-09-12T02:02:26.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.875+0000 7f54e0937740 1 -- start start 2024-09-12T02:02:26.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.876+0000 7f54d57fa640 1 --2- 172.21.15.16:0/2429712225 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f54c00a2b80 0x7f54c00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:26.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.876+0000 7f54e0937740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562218cc9760 0x562218cc9b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:26.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.876+0000 7f54e0937740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562218cca080 0x562218cd4400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:26.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.876+0000 7f54e0937740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562218cd4940 0x562218cd6d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:26.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.876+0000 7f54e0937740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x562218c2f2a0 con 0x562218cd4940 2024-09-12T02:02:26.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.876+0000 7f54e0937740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x562218bd6b10 con 0x562218cca080 2024-09-12T02:02:26.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.876+0000 7f54e0937740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x562218c2b260 con 0x562218cc9760 2024-09-12T02:02:26.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.876+0000 7f54d57fa640 1 -- 172.21.15.16:0/2429712225 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f54cc0ceaf0 con 0x7f54d8102570 2024-09-12T02:02:26.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.876+0000 7f54dcafb640 1 --2- 172.21.15.16:0/2429712225 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f54c00a2b80 0x7f54c00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:26.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.876+0000 7f54d7fff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562218cc9760 0x562218cc9b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:26.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.876+0000 7f54d77fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562218cca080 0x562218cd4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:26.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.876+0000 7f54d7fff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562218cc9760 0x562218cc9b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:38454/0 (socket says 172.21.15.16:38454) 2024-09-12T02:02:26.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.876+0000 7f54d7fff640 1 -- 172.21.15.16:0/4075455898 learned_addr learned my addr 172.21.15.16:0/4075455898 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:02:26.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.876+0000 7f54dcafb640 1 --2- 172.21.15.16:0/4075455898 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562218cd4940 0x562218cd6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:26.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.876+0000 7f54d7fff640 1 -- 172.21.15.16:0/4075455898 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562218cca080 msgr2=0x562218cd4400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:26.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.876+0000 7f54d7fff640 1 --2- 172.21.15.16:0/4075455898 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562218cca080 0x562218cd4400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:26.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.876+0000 7f54d7fff640 1 -- 172.21.15.16:0/4075455898 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562218cd4940 msgr2=0x562218cd6d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:26.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.876+0000 7f54d7fff640 1 --2- 172.21.15.16:0/4075455898 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562218cd4940 0x562218cd6d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:26.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.876+0000 7f54d7fff640 1 -- 172.21.15.16:0/4075455898 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622189a0b90 con 0x562218cc9760 2024-09-12T02:02:26.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.876+0000 7f54dcafb640 1 --2- 172.21.15.16:0/4075455898 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562218cd4940 0x562218cd6d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:02:26.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.877+0000 7f54dcafb640 1 --2- 172.21.15.16:0/2429712225 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f54c00a2b80 0x7f54c00a5040 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f54d0005e30 tx=0x7f54d002fab0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:02:26.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.877+0000 7f54d7fff640 1 --2- 172.21.15.16:0/4075455898 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562218cc9760 0x562218cc9b40 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f54c800ad00 tx=0x7f54c800f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=72fbdc1c1e71b248 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:02:26.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.877+0000 7f54be7fc640 1 -- 172.21.15.16:0/4075455898 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54c801d070 con 0x562218cc9760 2024-09-12T02:02:26.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.877+0000 7f54be7fc640 1 -- 172.21.15.16:0/4075455898 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54c8018b20 con 0x562218cc9760 2024-09-12T02:02:26.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.877+0000 7f54be7fc640 1 -- 172.21.15.16:0/4075455898 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54c8010ba0 con 0x562218cc9760 2024-09-12T02:02:26.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.878+0000 7f54e0937740 1 -- 172.21.15.16:0/4075455898 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562218cc9760 msgr2=0x562218cc9b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:26.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.878+0000 7f54e0937740 1 --2- 172.21.15.16:0/4075455898 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562218cc9760 0x562218cc9b40 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f54c800ad00 tx=0x7f54c800f790 comp rx=0 tx=0).stop 2024-09-12T02:02:26.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.878+0000 7f54e0937740 1 -- 172.21.15.16:0/4075455898 shutdown_connections 2024-09-12T02:02:26.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.878+0000 7f54e0937740 1 --2- 172.21.15.16:0/4075455898 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562218cd4940 0x562218cd6d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:26.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.878+0000 7f54e0937740 1 --2- 172.21.15.16:0/4075455898 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562218cca080 0x562218cd4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:26.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.878+0000 7f54e0937740 1 --2- 172.21.15.16:0/4075455898 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562218cc9760 0x562218cc9b40 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:26.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.878+0000 7f54e0937740 1 -- 172.21.15.16:0/4075455898 >> 172.21.15.16:0/4075455898 conn(0x562218cc5cf0 msgr2=0x562218cc60d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:02:26.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.878+0000 7f54e0937740 1 -- 172.21.15.16:0/4075455898 shutdown_connections 2024-09-12T02:02:26.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.878+0000 7f54e0937740 1 -- 172.21.15.16:0/4075455898 wait complete. 2024-09-12T02:02:26.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.878+0000 7f54e0937740 1 -- start start 2024-09-12T02:02:26.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.879+0000 7f54e0937740 1 Processor -- start 2024-09-12T02:02:26.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.879+0000 7f54e0937740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562218cc9760 0x562218cd1770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:26.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.879+0000 7f54e0937740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562218cca080 0x562218cd1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:26.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.879+0000 7f54e0937740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562218cd4940 0x562218cd21f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:26.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.879+0000 7f54e0937740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x562218cd7510 con 0x562218cc9760 2024-09-12T02:02:26.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.879+0000 7f54e0937740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x562218bb76e0 con 0x562218cca080 2024-09-12T02:02:26.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.879+0000 7f54e0937740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x562218cd7690 con 0x562218cd4940 2024-09-12T02:02:26.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.879+0000 7f54d7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562218cc9760 0x562218cd1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:26.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.879+0000 7f54e0937740 10 client.0 _ll_register_callbacks cb 0x562218bdba80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:02:26.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.879+0000 7f54d7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562218cc9760 0x562218cd1770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40536/0 (socket says 172.21.15.16:40536) 2024-09-12T02:02:26.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.879+0000 7f54d7fff640 1 -- 172.21.15.16:0/1215172181 learned_addr learned my addr 172.21.15.16:0/1215172181 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:02:26.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.879+0000 7f54d77fe640 1 --2- 172.21.15.16:0/1215172181 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562218cca080 0x562218cd1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:26.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.879+0000 7f54dcafb640 1 --2- 172.21.15.16:0/1215172181 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562218cd4940 0x562218cd21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:26.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.880+0000 7f54d7fff640 1 -- 172.21.15.16:0/1215172181 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562218cd4940 msgr2=0x562218cd21f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:26.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.880+0000 7f54d7fff640 1 --2- 172.21.15.16:0/1215172181 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562218cd4940 0x562218cd21f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:26.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.880+0000 7f54d7fff640 1 -- 172.21.15.16:0/1215172181 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562218cca080 msgr2=0x562218cd1cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:26.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.880+0000 7f54d7fff640 1 --2- 172.21.15.16:0/1215172181 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562218cca080 0x562218cd1cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:26.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.880+0000 7f54d7fff640 1 -- 172.21.15.16:0/1215172181 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562218ce0980 con 0x562218cc9760 2024-09-12T02:02:26.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.880+0000 7f54d7fff640 1 --2- 172.21.15.16:0/1215172181 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562218cc9760 0x562218cd1770 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f54c801b040 tx=0x7f54c8018670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e61a314a6acf1d33 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:02:26.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.880+0000 7f54b77fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:02:26.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.880+0000 7f54b77fe640 1 -- 172.21.15.16:0/1215172181 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54c801d040 con 0x562218cc9760 2024-09-12T02:02:26.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.880+0000 7f54b77fe640 1 -- 172.21.15.16:0/1215172181 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f54c80108e0 con 0x562218cc9760 2024-09-12T02:02:26.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.880+0000 7f54e0937740 1 -- 172.21.15.16:0/1215172181 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54c8018b20 con 0x562218cc9760 2024-09-12T02:02:26.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.880+0000 7f54b77fe640 1 -- 172.21.15.16:0/1215172181 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f54c800a910 con 0x562218cc9760 2024-09-12T02:02:26.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.880+0000 7f54e0937740 10 client.15426 fetch_fsmap learned FSMap version 25 2024-09-12T02:02:26.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.880+0000 7f54e0937740 1 -- 172.21.15.16:0/1215172181 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x562218ce32e0 con 0x562218cc9760 2024-09-12T02:02:26.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.881+0000 7f54b77fe640 1 -- 172.21.15.16:0/1215172181 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54c8018b20 con 0x562218cc9760 2024-09-12T02:02:26.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.881+0000 7f54b77fe640 1 -- 172.21.15.16:0/1215172181 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f54c8020030 con 0x562218cc9760 2024-09-12T02:02:26.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.881+0000 7f54e0937740 10 client.15426 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:02:26.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.881+0000 7f54e0937740 10 client.15426 resolve_mds: validated gid 24482 aka daemon mds.b 2024-09-12T02:02:26.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.881+0000 7f54e0937740 20 client.15426 populate_metadata read hostname 'smithi016' 2024-09-12T02:02:26.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.881+0000 7f54e0937740 1 --2- 172.21.15.16:0/1215172181 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x562218ce4bd0 0x562218ce7050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:26.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.881+0000 7f54e0937740 4 client.15426 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:02:26.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.881+0000 7f54e0937740 1 -- 172.21.15.16:0/1215172181 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562218c3c9d0 con 0x562218ce4bd0 2024-09-12T02:02:26.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.882+0000 7f54d77fe640 1 --2- 172.21.15.16:0/1215172181 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x562218ce4bd0 0x562218ce7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:26.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.882+0000 7f54d77fe640 1 --2- 172.21.15.16:0/1215172181 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x562218ce4bd0 0x562218ce7050 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=41c325a1deae1e2 server_cookie=9d91c04f218f433 in_seq=0 out_seq=0 2024-09-12T02:02:26.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.882+0000 7f54b77fe640 10 client.15426 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:02:26.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.883+0000 7f54b77fe640 1 -- 172.21.15.16:0/1215172181 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562218c3c9d0 con 0x562218ce4bd0 2024-09-12T02:02:26.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.883+0000 7f54b77fe640 10 client.15426 handle_command_reply: tid=0 2024-09-12T02:02:26.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.883+0000 7f54e0937740 1 client.15426 shutdown 2024-09-12T02:02:26.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.883+0000 7f54e0937740 1 -- 172.21.15.16:0/1215172181 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562218cc9760 msgr2=0x562218cd1770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:26.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.883+0000 7f54e0937740 1 --2- 172.21.15.16:0/1215172181 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562218cc9760 0x562218cd1770 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f54c801b040 tx=0x7f54c8018670 comp rx=0 tx=0).stop 2024-09-12T02:02:26.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.883+0000 7f54e0937740 1 -- 172.21.15.16:0/1215172181 shutdown_connections 2024-09-12T02:02:26.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.883+0000 7f54e0937740 1 --2- 172.21.15.16:0/1215172181 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x562218ce4bd0 0x562218ce7050 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:26.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.883+0000 7f54e0937740 1 --2- 172.21.15.16:0/1215172181 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562218cd4940 0x562218cd21f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:26.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.883+0000 7f54e0937740 1 --2- 172.21.15.16:0/1215172181 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562218cca080 0x562218cd1cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:26.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.883+0000 7f54e0937740 1 --2- 172.21.15.16:0/1215172181 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562218cc9760 0x562218cd1770 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:26.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.883+0000 7f54e0937740 1 -- 172.21.15.16:0/1215172181 >> 172.21.15.16:0/1215172181 conn(0x562218cc5cf0 msgr2=0x562218cc4e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:02:26.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.883+0000 7f54b77fe640 0 client.15426 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:02:26.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.883+0000 7f54e0937740 1 -- 172.21.15.16:0/1215172181 shutdown_connections 2024-09-12T02:02:26.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.883+0000 7f54e0937740 1 -- 172.21.15.16:0/1215172181 wait complete. 2024-09-12T02:02:26.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.883+0000 7f54e0937740 20 client.15426 trim_cache size 0 max 16384 2024-09-12T02:02:26.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.904+0000 7f54e0937740 1 Processor -- start 2024-09-12T02:02:26.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.904+0000 7f54e0937740 1 -- start start 2024-09-12T02:02:26.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.904+0000 7f54e0937740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562218cc5cf0 0x562218ccc1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:26.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.904+0000 7f54e0937740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562218cc9760 0x562218cec960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:26.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.904+0000 7f54e0937740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562218cca080 0x562218cecea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:26.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.904+0000 7f54e0937740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f54d81b1da0 con 0x562218cca080 2024-09-12T02:02:26.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.905+0000 7f54e0937740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f54d81af670 con 0x562218cc5cf0 2024-09-12T02:02:26.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.905+0000 7f54e0937740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f54d81ad4c0 con 0x562218cc9760 2024-09-12T02:02:26.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.905+0000 7f54dcafb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562218cca080 0x562218cecea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:26.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.905+0000 7f54dcafb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562218cca080 0x562218cecea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40552/0 (socket says 172.21.15.16:40552) 2024-09-12T02:02:26.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.905+0000 7f54dcafb640 1 -- 172.21.15.16:0/123929094 learned_addr learned my addr 172.21.15.16:0/123929094 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:02:26.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.905+0000 7f54d7fff640 1 --2- 172.21.15.16:0/123929094 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562218cc5cf0 0x562218ccc1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:26.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.905+0000 7f54d77fe640 1 --2- 172.21.15.16:0/123929094 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562218cc9760 0x562218cec960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:26.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.905+0000 7f54dcafb640 1 -- 172.21.15.16:0/123929094 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562218cc9760 msgr2=0x562218cec960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:26.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.905+0000 7f54dcafb640 1 --2- 172.21.15.16:0/123929094 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562218cc9760 0x562218cec960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:26.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.905+0000 7f54dcafb640 1 -- 172.21.15.16:0/123929094 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562218cc5cf0 msgr2=0x562218ccc1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:26.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.905+0000 7f54dcafb640 1 --2- 172.21.15.16:0/123929094 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562218cc5cf0 0x562218ccc1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:26.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.905+0000 7f54dcafb640 1 -- 172.21.15.16:0/123929094 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562218cd2af0 con 0x562218cca080 2024-09-12T02:02:26.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.905+0000 7f54dcafb640 1 --2- 172.21.15.16:0/123929094 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562218cca080 0x562218cecea0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f54d0035000 tx=0x7f54d0049000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b608122f3fa65824 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:02:26.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.906+0000 7f54bdffb640 1 -- 172.21.15.16:0/123929094 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54d0034040 con 0x562218cca080 2024-09-12T02:02:26.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.906+0000 7f54bdffb640 1 -- 172.21.15.16:0/123929094 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54d0044040 con 0x562218cca080 2024-09-12T02:02:26.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.906+0000 7f54bdffb640 1 -- 172.21.15.16:0/123929094 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54d0053470 con 0x562218cca080 2024-09-12T02:02:26.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.907+0000 7f54e0937740 1 -- 172.21.15.16:0/123929094 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562218cca080 msgr2=0x562218cecea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:26.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.907+0000 7f54e0937740 1 --2- 172.21.15.16:0/123929094 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562218cca080 0x562218cecea0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f54d0035000 tx=0x7f54d0049000 comp rx=0 tx=0).stop 2024-09-12T02:02:26.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.907+0000 7f54e0937740 1 -- 172.21.15.16:0/123929094 shutdown_connections 2024-09-12T02:02:26.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.907+0000 7f54e0937740 1 --2- 172.21.15.16:0/123929094 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562218cca080 0x562218cecea0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:26.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.907+0000 7f54e0937740 1 --2- 172.21.15.16:0/123929094 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562218cc9760 0x562218cec960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:26.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.907+0000 7f54e0937740 1 --2- 172.21.15.16:0/123929094 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562218cc5cf0 0x562218ccc1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:26.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.907+0000 7f54e0937740 1 -- 172.21.15.16:0/123929094 >> 172.21.15.16:0/123929094 conn(0x562218ccd160 msgr2=0x562218cd0040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:02:26.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.907+0000 7f54e0937740 1 -- 172.21.15.16:0/123929094 shutdown_connections 2024-09-12T02:02:26.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.907+0000 7f54e0937740 1 -- 172.21.15.16:0/123929094 wait complete. 2024-09-12T02:02:26.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.907+0000 7f54e0937740 1 -- start start 2024-09-12T02:02:26.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.908+0000 7f54e0937740 1 Processor -- start 2024-09-12T02:02:26.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.908+0000 7f54e0937740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562218cc5cf0 0x562218cdcd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:26.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.908+0000 7f54e0937740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562218cc9760 0x562218cd8060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:26.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.908+0000 7f54e0937740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562218cca080 0x562218cd85a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:26.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.908+0000 7f54e0937740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f54d81adb30 con 0x562218cca080 2024-09-12T02:02:26.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.908+0000 7f54e0937740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f54d81aefa0 con 0x562218cc9760 2024-09-12T02:02:26.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.908+0000 7f54e0937740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f54d81ae200 con 0x562218cc5cf0 2024-09-12T02:02:26.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.908+0000 7f54e0937740 10 client.0 _ll_register_callbacks cb 0x7f54c800ab80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:02:26.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.908+0000 7f54dcafb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562218cca080 0x562218cd85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:26.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.908+0000 7f54d77fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562218cc9760 0x562218cd8060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:26.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.908+0000 7f54d77fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562218cc9760 0x562218cd8060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.16:48202/0 (socket says 172.21.15.16:48202) 2024-09-12T02:02:26.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.908+0000 7f54d77fe640 1 -- 172.21.15.16:0/1004113160 learned_addr learned my addr 172.21.15.16:0/1004113160 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:02:26.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.909+0000 7f54d7fff640 1 --2- 172.21.15.16:0/1004113160 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562218cc5cf0 0x562218cdcd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:26.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.909+0000 7f54d77fe640 1 -- 172.21.15.16:0/1004113160 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562218cc5cf0 msgr2=0x562218cdcd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:26.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.909+0000 7f54d77fe640 1 --2- 172.21.15.16:0/1004113160 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562218cc5cf0 0x562218cdcd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:26.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.909+0000 7f54d77fe640 1 -- 172.21.15.16:0/1004113160 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562218cca080 msgr2=0x562218cd85a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:26.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.909+0000 7f54d77fe640 1 --2- 172.21.15.16:0/1004113160 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562218cca080 0x562218cd85a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:26.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.909+0000 7f54d77fe640 1 -- 172.21.15.16:0/1004113160 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562218cd8ae0 con 0x562218cc9760 2024-09-12T02:02:26.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.909+0000 7f54d77fe640 1 --2- 172.21.15.16:0/1004113160 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562218cc9760 0x562218cd8060 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f54cc097dc0 tx=0x7f54cc000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cdedcb0a35a6329c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:02:26.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.909+0000 7f54b77fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-12T02:02:26.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.909+0000 7f54b77fe640 1 -- 172.21.15.16:0/1004113160 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54cc0b3020 con 0x562218cc9760 2024-09-12T02:02:26.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.909+0000 7f54b77fe640 1 -- 172.21.15.16:0/1004113160 <== mon.2 v2:172.21.15.33:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f54cc0b4220 con 0x562218cc9760 2024-09-12T02:02:26.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.909+0000 7f54e0937740 1 -- 172.21.15.16:0/1004113160 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54d0044040 con 0x562218cc9760 2024-09-12T02:02:26.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.909+0000 7f54b77fe640 1 -- 172.21.15.16:0/1004113160 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f54cc083070 con 0x562218cc9760 2024-09-12T02:02:26.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.909+0000 7f54e0937740 10 client.25109 fetch_fsmap learned FSMap version 25 2024-09-12T02:02:26.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.909+0000 7f54e0937740 1 -- 172.21.15.16:0/1004113160 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x562218d4f600 con 0x562218cc9760 2024-09-12T02:02:26.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.910+0000 7f54b77fe640 1 -- 172.21.15.16:0/1004113160 <== mon.2 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54d0044040 con 0x562218cc9760 2024-09-12T02:02:26.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.910+0000 7f54b77fe640 1 -- 172.21.15.16:0/1004113160 <== mon.2 v2:172.21.15.33:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f54c8019ce0 con 0x562218cc9760 2024-09-12T02:02:26.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.910+0000 7f54e0937740 10 client.25109 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:02:26.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.910+0000 7f54e0937740 10 client.25109 resolve_mds: validated gid 24482 aka daemon mds.b 2024-09-12T02:02:26.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.910+0000 7f54e0937740 20 client.25109 populate_metadata read hostname 'smithi016' 2024-09-12T02:02:26.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.910+0000 7f54e0937740 1 --2- 172.21.15.16:0/1004113160 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x562218cd4940 0x562218d51de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:26.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.910+0000 7f54e0937740 4 client.25109 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 144.5, "await_for": 45.0}] 2024-09-12T02:02:26.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.910+0000 7f54e0937740 1 -- 172.21.15.16:0/1004113160 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 144.5, "await_for": 45.0}) -- 0x562218c36d20 con 0x562218cd4940 2024-09-12T02:02:26.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.911+0000 7f54d7fff640 1 --2- 172.21.15.16:0/1004113160 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x562218cd4940 0x562218d51de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:26.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.911+0000 7f54d7fff640 1 --2- 172.21.15.16:0/1004113160 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x562218cd4940 0x562218d51de0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=527e1352dc566a35 server_cookie=8b94d3648133e345 in_seq=0 out_seq=0 2024-09-12T02:02:26.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.911+0000 7f54b77fe640 10 client.25109 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:02:26.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.921+0000 7f54b77fe640 1 -- 172.21.15.16:0/1004113160 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x562218c36d20 con 0x562218cd4940 2024-09-12T02:02:26.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.921+0000 7f54b77fe640 10 client.25109 handle_command_reply: tid=0 2024-09-12T02:02:26.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.921+0000 7f54e0937740 1 client.25109 shutdown 2024-09-12T02:02:26.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.921+0000 7f54e0937740 1 -- 172.21.15.16:0/1004113160 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562218cc9760 msgr2=0x562218cd8060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:26.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.921+0000 7f54e0937740 1 --2- 172.21.15.16:0/1004113160 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562218cc9760 0x562218cd8060 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f54cc097dc0 tx=0x7f54cc000bc0 comp rx=0 tx=0).stop 2024-09-12T02:02:26.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.921+0000 7f54e0937740 1 -- 172.21.15.16:0/1004113160 shutdown_connections 2024-09-12T02:02:26.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.921+0000 7f54e0937740 1 --2- 172.21.15.16:0/1004113160 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x562218cd4940 0x562218d51de0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:26.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.921+0000 7f54e0937740 1 --2- 172.21.15.16:0/1004113160 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562218cca080 0x562218cd85a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:26.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.921+0000 7f54e0937740 1 --2- 172.21.15.16:0/1004113160 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562218cc9760 0x562218cd8060 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:26.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.921+0000 7f54e0937740 1 --2- 172.21.15.16:0/1004113160 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562218cc5cf0 0x562218cdcd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:26.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.921+0000 7f54e0937740 1 -- 172.21.15.16:0/1004113160 >> 172.21.15.16:0/1004113160 conn(0x562218ccd160 msgr2=0x562218cc4650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:02:26.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.921+0000 7f54b77fe640 0 client.25109 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:02:26.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.922+0000 7f54e0937740 1 -- 172.21.15.16:0/1004113160 shutdown_connections 2024-09-12T02:02:26.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.922+0000 7f54e0937740 1 -- 172.21.15.16:0/1004113160 wait complete. 2024-09-12T02:02:26.923 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T02:02:26.923 INFO:teuthology.orchestra.run.smithi016.stdout: "epoch": 25, 2024-09-12T02:02:26.924 INFO:teuthology.orchestra.run.smithi016.stdout: "leader": 24482, 2024-09-12T02:02:26.924 INFO:teuthology.orchestra.run.smithi016.stdout: "set_version": 2, 2024-09-12T02:02:26.924 INFO:teuthology.orchestra.run.smithi016.stdout: "sets": { 2024-09-12T02:02:26.924 INFO:teuthology.orchestra.run.smithi016.stdout: "72b7f958": { 2024-09-12T02:02:26.924 INFO:teuthology.orchestra.run.smithi016.stdout: "version": 2, 2024-09-12T02:02:26.924 INFO:teuthology.orchestra.run.smithi016.stdout: "age_ref": 0.0, 2024-09-12T02:02:26.924 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-12T02:02:26.924 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "QUIESCED", 2024-09-12T02:02:26.924 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-12T02:02:26.924 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:02:26.924 INFO:teuthology.orchestra.run.smithi016.stdout: "timeout": 90.0, 2024-09-12T02:02:26.924 INFO:teuthology.orchestra.run.smithi016.stdout: "expiration": 144.5, 2024-09-12T02:02:26.924 INFO:teuthology.orchestra.run.smithi016.stdout: "members": { 2024-09-12T02:02:26.925 INFO:teuthology.orchestra.run.smithi016.stdout: "file:/": { 2024-09-12T02:02:26.925 INFO:teuthology.orchestra.run.smithi016.stdout: "excluded": false, 2024-09-12T02:02:26.925 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-12T02:02:26.925 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "QUIESCED", 2024-09-12T02:02:26.925 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-12T02:02:26.925 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:02:26.925 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:02:26.925 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:02:26.925 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:02:26.925 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:02:26.925 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T02:02:26.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.922+0000 7f54e0937740 20 client.25109 trim_cache size 0 max 16384 2024-09-12T02:02:26.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.922+0000 7f54de55e640 1 -- 172.21.15.16:0/2429712225 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f54c00a2b80 msgr2=0x7f54c00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:26.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.922+0000 7f54de55e640 1 --2- 172.21.15.16:0/2429712225 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f54c00a2b80 0x7f54c00a5040 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f54d0005e30 tx=0x7f54d002fab0 comp rx=0 tx=0).stop 2024-09-12T02:02:26.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.922+0000 7f54de55e640 1 -- 172.21.15.16:0/2429712225 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54d8102570 msgr2=0x7f54d8108020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:26.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.922+0000 7f54de55e640 1 --2- 172.21.15.16:0/2429712225 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54d8102570 0x7f54d8108020 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f54cc00d950 tx=0x7f54cc00dd50 comp rx=0 tx=0).stop 2024-09-12T02:02:26.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.923+0000 7f54de55e640 1 -- 172.21.15.16:0/2429712225 shutdown_connections 2024-09-12T02:02:26.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.923+0000 7f54de55e640 1 --2- 172.21.15.16:0/2429712225 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f54c00a2b80 0x7f54c00a5040 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:26.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.923+0000 7f54de55e640 1 --2- 172.21.15.16:0/2429712225 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f54d8102ef0 0x7f54d8108560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:26.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.923+0000 7f54de55e640 1 --2- 172.21.15.16:0/2429712225 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54d8102570 0x7f54d8108020 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:26.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.923+0000 7f54de55e640 1 --2- 172.21.15.16:0/2429712225 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f54d8100e20 0x7f54d8107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:26.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.923+0000 7f54de55e640 1 -- 172.21.15.16:0/2429712225 >> 172.21.15.16:0/2429712225 conn(0x7f54d80fc7d0 msgr2=0x7f54d81036a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:02:26.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.923+0000 7f54de55e640 1 -- 172.21.15.16:0/2429712225 shutdown_connections 2024-09-12T02:02:26.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:26.923+0000 7f54de55e640 1 -- 172.21.15.16:0/2429712225 wait complete. 2024-09-12T02:02:27.146 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:02:27.628 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:02:27 smithi016 ceph-mon[30947]: pgmap v351: 97 pgs: 97 active+clean; 661 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 682 B/s wr, 7 op/s 2024-09-12T02:02:27.656 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '72b7f958', quiesce took 7.8 seconds. Will release after: 16.7 2024-09-12T02:02:27.668 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:02:27 smithi033 ceph-mon[35332]: pgmap v351: 97 pgs: 97 active+clean; 661 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 682 B/s wr, 7 op/s 2024-09-12T02:02:27.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:02:27 smithi184 ceph-mon[35593]: pgmap v351: 97 pgs: 97 active+clean; 661 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 682 B/s wr, 7 op/s 2024-09-12T02:02:28.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.183+0000 7fc374536640 1 -- 172.21.15.16:0/869675870 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc36c102570 msgr2=0x7fc36c1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:28.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.183+0000 7fc374536640 1 --2- 172.21.15.16:0/869675870 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc36c102570 0x7fc36c1029b0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fc35c009900 tx=0x7fc35c02a480 comp rx=0 tx=0).stop 2024-09-12T02:02:28.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.184+0000 7fc374536640 1 -- 172.21.15.16:0/869675870 shutdown_connections 2024-09-12T02:02:28.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.184+0000 7fc374536640 1 --2- 172.21.15.16:0/869675870 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc36c102ef0 0x7fc36c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:28.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.184+0000 7fc374536640 1 --2- 172.21.15.16:0/869675870 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc36c102570 0x7fc36c1029b0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:28.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.184+0000 7fc374536640 1 --2- 172.21.15.16:0/869675870 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc36c100e20 0x7fc36c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:28.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.184+0000 7fc374536640 1 -- 172.21.15.16:0/869675870 >> 172.21.15.16:0/869675870 conn(0x7fc36c0fc7d0 msgr2=0x7fc36c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:02:28.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.184+0000 7fc374536640 1 -- 172.21.15.16:0/869675870 shutdown_connections 2024-09-12T02:02:28.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.184+0000 7fc374536640 1 -- 172.21.15.16:0/869675870 wait complete. 2024-09-12T02:02:28.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.185+0000 7fc374536640 1 Processor -- start 2024-09-12T02:02:28.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.185+0000 7fc374536640 1 -- start start 2024-09-12T02:02:28.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.185+0000 7fc374536640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc36c100e20 0x7fc36c198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:28.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.185+0000 7fc374536640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc36c102570 0x7fc36c198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:28.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.185+0000 7fc374536640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc36c102ef0 0x7fc36c19ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:28.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.185+0000 7fc374536640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc36c110f80 con 0x7fc36c102ef0 2024-09-12T02:02:28.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.185+0000 7fc374536640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc36c110e00 con 0x7fc36c102570 2024-09-12T02:02:28.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.185+0000 7fc374536640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc36c111100 con 0x7fc36c100e20 2024-09-12T02:02:28.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.185+0000 7fc3722d2640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc36c100e20 0x7fc36c198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:28.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.185+0000 7fc3722d2640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc36c100e20 0x7fc36c198560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:38506/0 (socket says 172.21.15.16:38506) 2024-09-12T02:02:28.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.185+0000 7fc3722d2640 1 -- 172.21.15.16:0/956457415 learned_addr learned my addr 172.21.15.16:0/956457415 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:02:28.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.185+0000 7fc371ad1640 1 --2- 172.21.15.16:0/956457415 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc36c102570 0x7fc36c198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:28.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.185+0000 7fc372ad3640 1 --2- 172.21.15.16:0/956457415 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc36c102ef0 0x7fc36c19ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:28.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.186+0000 7fc371ad1640 1 -- 172.21.15.16:0/956457415 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc36c100e20 msgr2=0x7fc36c198560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:28.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.186+0000 7fc371ad1640 1 --2- 172.21.15.16:0/956457415 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc36c100e20 0x7fc36c198560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:28.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.186+0000 7fc371ad1640 1 -- 172.21.15.16:0/956457415 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc36c102ef0 msgr2=0x7fc36c19ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:28.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.186+0000 7fc371ad1640 1 --2- 172.21.15.16:0/956457415 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc36c102ef0 0x7fc36c19ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:28.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.186+0000 7fc371ad1640 1 -- 172.21.15.16:0/956457415 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc36c19d510 con 0x7fc36c102570 2024-09-12T02:02:28.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.186+0000 7fc372ad3640 1 --2- 172.21.15.16:0/956457415 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc36c102ef0 0x7fc36c19ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:02:28.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.186+0000 7fc3722d2640 1 --2- 172.21.15.16:0/956457415 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc36c100e20 0x7fc36c198560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:02:28.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.186+0000 7fc371ad1640 1 --2- 172.21.15.16:0/956457415 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc36c102570 0x7fc36c198aa0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc35c005940 tx=0x7fc35c003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:02:28.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.186+0000 7fc3637fe640 1 -- 172.21.15.16:0/956457415 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc35c03f070 con 0x7fc36c102570 2024-09-12T02:02:28.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.186+0000 7fc374536640 1 -- 172.21.15.16:0/956457415 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc36c19d7a0 con 0x7fc36c102570 2024-09-12T02:02:28.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.187+0000 7fc3637fe640 1 -- 172.21.15.16:0/956457415 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc35c031c30 con 0x7fc36c102570 2024-09-12T02:02:28.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.187+0000 7fc374536640 1 -- 172.21.15.16:0/956457415 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc36c1a50e0 con 0x7fc36c102570 2024-09-12T02:02:28.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.187+0000 7fc3637fe640 1 -- 172.21.15.16:0/956457415 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc35c035030 con 0x7fc36c102570 2024-09-12T02:02:28.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.189+0000 7fc3637fe640 1 -- 172.21.15.16:0/956457415 <== mon.2 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fc35c042070 con 0x7fc36c102570 2024-09-12T02:02:28.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.190+0000 7fc3637fe640 1 --2- 172.21.15.16:0/956457415 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc3580a2c00 0x7fc3580a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:28.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.190+0000 7fc3637fe640 1 -- 172.21.15.16:0/956457415 <== mon.2 v2:172.21.15.33:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fc35c0ef550 con 0x7fc36c102570 2024-09-12T02:02:28.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.190+0000 7fc3722d2640 1 --2- 172.21.15.16:0/956457415 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc3580a2c00 0x7fc3580a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:28.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.190+0000 7fc37690f740 1 Processor -- start 2024-09-12T02:02:28.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.191+0000 7fc37690f740 1 -- start start 2024-09-12T02:02:28.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.191+0000 7fc3722d2640 1 --2- 172.21.15.16:0/956457415 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc3580a2c00 0x7fc3580a50c0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fc368005fd0 tx=0x7fc36800b4a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:02:28.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.191+0000 7fc37690f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565dd881b40 0x5565dd880090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:28.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.191+0000 7fc37690f740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5565dd8805d0 0x5565dd880a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:28.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.191+0000 7fc37690f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5565dd88f780 0x5565dd891b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:28.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.191+0000 7fc37690f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5565dd7e62f0 con 0x5565dd881b40 2024-09-12T02:02:28.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.191+0000 7fc37690f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5565dd78dcf0 con 0x5565dd88f780 2024-09-12T02:02:28.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.191+0000 7fc37690f740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5565dd7e22b0 con 0x5565dd8805d0 2024-09-12T02:02:28.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.191+0000 7fc3722d2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565dd881b40 0x5565dd880090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:28.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.191+0000 7fc3722d2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565dd881b40 0x5565dd880090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40580/0 (socket says 172.21.15.16:40580) 2024-09-12T02:02:28.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.191+0000 7fc3722d2640 1 -- 172.21.15.16:0/2694069511 learned_addr learned my addr 172.21.15.16:0/2694069511 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:02:28.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.191+0000 7fc372ad3640 1 --2- 172.21.15.16:0/2694069511 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5565dd88f780 0x5565dd891b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:28.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.191+0000 7fc371ad1640 1 --2- 172.21.15.16:0/2694069511 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5565dd8805d0 0x5565dd880a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:28.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.191+0000 7fc3722d2640 1 -- 172.21.15.16:0/2694069511 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5565dd8805d0 msgr2=0x5565dd880a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:28.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.191+0000 7fc3722d2640 1 --2- 172.21.15.16:0/2694069511 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5565dd8805d0 0x5565dd880a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:28.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.191+0000 7fc3722d2640 1 -- 172.21.15.16:0/2694069511 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5565dd88f780 msgr2=0x5565dd891b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:28.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.191+0000 7fc3722d2640 1 --2- 172.21.15.16:0/2694069511 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5565dd88f780 0x5565dd891b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:28.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.191+0000 7fc3722d2640 1 -- 172.21.15.16:0/2694069511 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565dd7723a0 con 0x5565dd881b40 2024-09-12T02:02:28.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.192+0000 7fc3722d2640 1 --2- 172.21.15.16:0/2694069511 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565dd881b40 0x5565dd880090 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fc3680118e0 tx=0x7fc368013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b944ec89d306184 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:02:28.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.192+0000 7fc360ff9640 1 -- 172.21.15.16:0/2694069511 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc368011e60 con 0x5565dd881b40 2024-09-12T02:02:28.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.192+0000 7fc360ff9640 1 -- 172.21.15.16:0/2694069511 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc36801e420 con 0x5565dd881b40 2024-09-12T02:02:28.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.192+0000 7fc360ff9640 1 -- 172.21.15.16:0/2694069511 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc36801c610 con 0x5565dd881b40 2024-09-12T02:02:28.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.193+0000 7fc37690f740 1 -- 172.21.15.16:0/2694069511 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565dd881b40 msgr2=0x5565dd880090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:28.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.193+0000 7fc37690f740 1 --2- 172.21.15.16:0/2694069511 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565dd881b40 0x5565dd880090 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fc3680118e0 tx=0x7fc368013000 comp rx=0 tx=0).stop 2024-09-12T02:02:28.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.193+0000 7fc37690f740 1 -- 172.21.15.16:0/2694069511 shutdown_connections 2024-09-12T02:02:28.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.193+0000 7fc37690f740 1 --2- 172.21.15.16:0/2694069511 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5565dd88f780 0x5565dd891b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:28.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.193+0000 7fc37690f740 1 --2- 172.21.15.16:0/2694069511 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5565dd8805d0 0x5565dd880a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:28.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.193+0000 7fc37690f740 1 --2- 172.21.15.16:0/2694069511 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565dd881b40 0x5565dd880090 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:28.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.193+0000 7fc37690f740 1 -- 172.21.15.16:0/2694069511 >> 172.21.15.16:0/2694069511 conn(0x5565dd849310 msgr2=0x5565dd87cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:02:28.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.194+0000 7fc37690f740 1 -- 172.21.15.16:0/2694069511 shutdown_connections 2024-09-12T02:02:28.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.194+0000 7fc37690f740 1 -- 172.21.15.16:0/2694069511 wait complete. 2024-09-12T02:02:28.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.194+0000 7fc37690f740 1 -- start start 2024-09-12T02:02:28.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.194+0000 7fc37690f740 1 Processor -- start 2024-09-12T02:02:28.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.194+0000 7fc37690f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565dd8805d0 0x5565dd88f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:28.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.194+0000 7fc37690f740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5565dd881b40 0x5565dd88c380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:28.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.195+0000 7fc37690f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5565dd88f780 0x5565dd88c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:28.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.195+0000 7fc37690f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5565dd8924d0 con 0x5565dd8805d0 2024-09-12T02:02:28.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.195+0000 7fc37690f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5565dd7a1060 con 0x5565dd88f780 2024-09-12T02:02:28.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.195+0000 7fc37690f740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5565dd892650 con 0x5565dd881b40 2024-09-12T02:02:28.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.195+0000 7fc37690f740 10 client.0 _ll_register_callbacks cb 0x5565dd4f48c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:02:28.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.195+0000 7fc371ad1640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5565dd881b40 0x5565dd88c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:28.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.195+0000 7fc372ad3640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5565dd88f780 0x5565dd88c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:28.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.195+0000 7fc3722d2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565dd8805d0 0x5565dd88f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:28.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.195+0000 7fc371ad1640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5565dd881b40 0x5565dd88c380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:38516/0 (socket says 172.21.15.16:38516) 2024-09-12T02:02:28.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.195+0000 7fc371ad1640 1 -- 172.21.15.16:0/636720544 learned_addr learned my addr 172.21.15.16:0/636720544 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:02:28.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.195+0000 7fc372ad3640 1 -- 172.21.15.16:0/636720544 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5565dd881b40 msgr2=0x5565dd88c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:28.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.195+0000 7fc372ad3640 1 --2- 172.21.15.16:0/636720544 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5565dd881b40 0x5565dd88c380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:28.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.195+0000 7fc372ad3640 1 -- 172.21.15.16:0/636720544 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565dd8805d0 msgr2=0x5565dd88f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:28.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.195+0000 7fc372ad3640 1 --2- 172.21.15.16:0/636720544 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565dd8805d0 0x5565dd88f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:28.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.195+0000 7fc372ad3640 1 -- 172.21.15.16:0/636720544 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565dd5573c0 con 0x5565dd88f780 2024-09-12T02:02:28.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.195+0000 7fc372ad3640 1 --2- 172.21.15.16:0/636720544 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5565dd88f780 0x5565dd88c8e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc364005fd0 tx=0x7fc3640117b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=117b4a102489ec35 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:02:28.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.195+0000 7fc355ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-12T02:02:28.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.196+0000 7fc355ffb640 1 -- 172.21.15.16:0/636720544 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc364013020 con 0x5565dd88f780 2024-09-12T02:02:28.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.196+0000 7fc355ffb640 1 -- 172.21.15.16:0/636720544 <== mon.2 v2:172.21.15.33:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fc3640131c0 con 0x5565dd88f780 2024-09-12T02:02:28.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.196+0000 7fc37690f740 1 -- 172.21.15.16:0/636720544 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc36801e420 con 0x5565dd88f780 2024-09-12T02:02:28.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.196+0000 7fc355ffb640 1 -- 172.21.15.16:0/636720544 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc3640106a0 con 0x5565dd88f780 2024-09-12T02:02:28.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.196+0000 7fc37690f740 10 client.25121 fetch_fsmap learned FSMap version 25 2024-09-12T02:02:28.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.196+0000 7fc37690f740 1 -- 172.21.15.16:0/636720544 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5565dd89dfa0 con 0x5565dd88f780 2024-09-12T02:02:28.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.196+0000 7fc355ffb640 1 -- 172.21.15.16:0/636720544 <== mon.2 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc36801e420 con 0x5565dd88f780 2024-09-12T02:02:28.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.197+0000 7fc355ffb640 1 -- 172.21.15.16:0/636720544 <== mon.2 v2:172.21.15.33:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fc36401a070 con 0x5565dd88f780 2024-09-12T02:02:28.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.197+0000 7fc37690f740 10 client.25121 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:02:28.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.197+0000 7fc37690f740 10 client.25121 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-12T02:02:28.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.197+0000 7fc37690f740 20 client.25121 populate_metadata read hostname 'smithi016' 2024-09-12T02:02:28.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.197+0000 7fc37690f740 1 --2- 172.21.15.16:0/636720544 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x5565dd89f470 0x5565dd8a18f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:28.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.197+0000 7fc37690f740 4 client.25121 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:02:28.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.197+0000 7fc37690f740 1 -- 172.21.15.16:0/636720544 --> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5565dd7f3860 con 0x5565dd89f470 2024-09-12T02:02:28.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.197+0000 7fc3722d2640 1 --2- 172.21.15.16:0/636720544 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x5565dd89f470 0x5565dd8a18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:28.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.198+0000 7fc3722d2640 1 --2- 172.21.15.16:0/636720544 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x5565dd89f470 0x5565dd8a18f0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=530f8754f4bef95e server_cookie=7a0eb4a44eb22a7f in_seq=0 out_seq=0 2024-09-12T02:02:28.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.198+0000 7fc355ffb640 10 client.25121 ms_handle_connect on v2:172.21.15.184:6836/1793393087 2024-09-12T02:02:28.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.198+0000 7fc355ffb640 1 -- 172.21.15.16:0/636720544 <== mds.2 v2:172.21.15.184:6836/1793393087 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5565dd7f3860 con 0x5565dd89f470 2024-09-12T02:02:28.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.198+0000 7fc355ffb640 10 client.25121 handle_command_reply: tid=0 2024-09-12T02:02:28.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.199+0000 7fc37690f740 1 client.25121 shutdown 2024-09-12T02:02:28.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.199+0000 7fc37690f740 1 -- 172.21.15.16:0/636720544 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5565dd88f780 msgr2=0x5565dd88c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:28.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.199+0000 7fc37690f740 1 --2- 172.21.15.16:0/636720544 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5565dd88f780 0x5565dd88c8e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc364005fd0 tx=0x7fc3640117b0 comp rx=0 tx=0).stop 2024-09-12T02:02:28.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.199+0000 7fc37690f740 1 -- 172.21.15.16:0/636720544 shutdown_connections 2024-09-12T02:02:28.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.199+0000 7fc37690f740 1 --2- 172.21.15.16:0/636720544 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5565dd88f780 0x5565dd88c8e0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:28.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.199+0000 7fc37690f740 1 --2- 172.21.15.16:0/636720544 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5565dd881b40 0x5565dd88c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:28.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.199+0000 7fc37690f740 1 --2- 172.21.15.16:0/636720544 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x5565dd89f470 0x5565dd8a18f0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:28.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.199+0000 7fc37690f740 1 --2- 172.21.15.16:0/636720544 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565dd8805d0 0x5565dd88f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:28.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.199+0000 7fc37690f740 1 -- 172.21.15.16:0/636720544 >> 172.21.15.16:0/636720544 conn(0x5565dd849310 msgr2=0x5565dd882930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:02:28.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.199+0000 7fc355ffb640 0 client.25121 ms_handle_reset on v2:172.21.15.184:6836/1793393087 2024-09-12T02:02:28.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.199+0000 7fc37690f740 1 -- 172.21.15.16:0/636720544 shutdown_connections 2024-09-12T02:02:28.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.199+0000 7fc37690f740 1 -- 172.21.15.16:0/636720544 wait complete. 2024-09-12T02:02:28.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.199+0000 7fc37690f740 20 client.25121 trim_cache size 0 max 16384 2024-09-12T02:02:28.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.224+0000 7fc37690f740 1 Processor -- start 2024-09-12T02:02:28.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.224+0000 7fc37690f740 1 -- start start 2024-09-12T02:02:28.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.224+0000 7fc37690f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5565dd8805d0 0x5565dd89b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:28.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.224+0000 7fc37690f740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5565dd881b40 0x5565dd88b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:28.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.224+0000 7fc37690f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565dd88f780 0x5565dd8a71c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:28.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.224+0000 7fc37690f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc350010bc0 con 0x5565dd88f780 2024-09-12T02:02:28.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.224+0000 7fc37690f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc35000e490 con 0x5565dd8805d0 2024-09-12T02:02:28.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.224+0000 7fc37690f740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc35000c2e0 con 0x5565dd881b40 2024-09-12T02:02:28.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.224+0000 7fc372ad3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565dd88f780 0x5565dd8a71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:28.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.224+0000 7fc372ad3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565dd88f780 0x5565dd8a71c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40590/0 (socket says 172.21.15.16:40590) 2024-09-12T02:02:28.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.225+0000 7fc372ad3640 1 -- 172.21.15.16:0/2092610386 learned_addr learned my addr 172.21.15.16:0/2092610386 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:02:28.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.225+0000 7fc3722d2640 1 --2- 172.21.15.16:0/2092610386 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5565dd8805d0 0x5565dd89b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:28.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.225+0000 7fc371ad1640 1 --2- 172.21.15.16:0/2092610386 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5565dd881b40 0x5565dd88b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:28.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.225+0000 7fc372ad3640 1 -- 172.21.15.16:0/2092610386 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5565dd881b40 msgr2=0x5565dd88b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:28.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.225+0000 7fc372ad3640 1 --2- 172.21.15.16:0/2092610386 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5565dd881b40 0x5565dd88b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:28.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.225+0000 7fc372ad3640 1 -- 172.21.15.16:0/2092610386 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5565dd8805d0 msgr2=0x5565dd89b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:28.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.225+0000 7fc372ad3640 1 --2- 172.21.15.16:0/2092610386 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5565dd8805d0 0x5565dd89b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:28.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.225+0000 7fc372ad3640 1 -- 172.21.15.16:0/2092610386 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565dd88cc50 con 0x5565dd88f780 2024-09-12T02:02:28.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.225+0000 7fc3722d2640 1 --2- 172.21.15.16:0/2092610386 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5565dd8805d0 0x5565dd89b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:02:28.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.225+0000 7fc372ad3640 1 --2- 172.21.15.16:0/2092610386 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565dd88f780 0x5565dd8a71c0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fc3640138c0 tx=0x7fc364005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b39ad1bb24161679 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:02:28.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.225+0000 7fc357fff640 1 -- 172.21.15.16:0/2092610386 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3640049a0 con 0x5565dd88f780 2024-09-12T02:02:28.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.225+0000 7fc357fff640 1 -- 172.21.15.16:0/2092610386 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc36402a070 con 0x5565dd88f780 2024-09-12T02:02:28.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.226+0000 7fc357fff640 1 -- 172.21.15.16:0/2092610386 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc364005290 con 0x5565dd88f780 2024-09-12T02:02:28.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.226+0000 7fc37690f740 1 -- 172.21.15.16:0/2092610386 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565dd88f780 msgr2=0x5565dd8a71c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:28.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.226+0000 7fc37690f740 1 --2- 172.21.15.16:0/2092610386 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565dd88f780 0x5565dd8a71c0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fc3640138c0 tx=0x7fc364005c50 comp rx=0 tx=0).stop 2024-09-12T02:02:28.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.226+0000 7fc37690f740 1 -- 172.21.15.16:0/2092610386 shutdown_connections 2024-09-12T02:02:28.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.226+0000 7fc37690f740 1 --2- 172.21.15.16:0/2092610386 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565dd88f780 0x5565dd8a71c0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:28.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.226+0000 7fc37690f740 1 --2- 172.21.15.16:0/2092610386 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5565dd881b40 0x5565dd88b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:28.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.226+0000 7fc37690f740 1 --2- 172.21.15.16:0/2092610386 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5565dd8805d0 0x5565dd89b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:28.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.226+0000 7fc37690f740 1 -- 172.21.15.16:0/2092610386 >> 172.21.15.16:0/2092610386 conn(0x5565dd849310 msgr2=0x5565dd88ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:02:28.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.226+0000 7fc37690f740 1 -- 172.21.15.16:0/2092610386 shutdown_connections 2024-09-12T02:02:28.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.226+0000 7fc37690f740 1 -- 172.21.15.16:0/2092610386 wait complete. 2024-09-12T02:02:28.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.227+0000 7fc37690f740 1 -- start start 2024-09-12T02:02:28.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.227+0000 7fc37690f740 1 Processor -- start 2024-09-12T02:02:28.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.227+0000 7fc37690f740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5565dd8805d0 0x5565dd884120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:28.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.227+0000 7fc37690f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565dd881b40 0x5565dd884660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:28.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.227+0000 7fc37690f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5565dd88f780 0x5565dd884ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:28.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.227+0000 7fc37690f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc35000c950 con 0x5565dd881b40 2024-09-12T02:02:28.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.227+0000 7fc37690f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc35000ddc0 con 0x5565dd88f780 2024-09-12T02:02:28.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.227+0000 7fc37690f740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc35000d020 con 0x5565dd8805d0 2024-09-12T02:02:28.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.227+0000 7fc37690f740 10 client.0 _ll_register_callbacks cb 0x7fc364013980 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:02:28.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.227+0000 7fc371ad1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565dd881b40 0x5565dd884660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:28.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.227+0000 7fc371ad1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565dd881b40 0x5565dd884660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40594/0 (socket says 172.21.15.16:40594) 2024-09-12T02:02:28.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.227+0000 7fc371ad1640 1 -- 172.21.15.16:0/829250014 learned_addr learned my addr 172.21.15.16:0/829250014 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:02:28.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.228+0000 7fc372ad3640 1 --2- 172.21.15.16:0/829250014 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5565dd88f780 0x5565dd884ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:28.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.228+0000 7fc3722d2640 1 --2- 172.21.15.16:0/829250014 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5565dd8805d0 0x5565dd884120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:28.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.228+0000 7fc371ad1640 1 -- 172.21.15.16:0/829250014 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5565dd8805d0 msgr2=0x5565dd884120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:28.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.228+0000 7fc371ad1640 1 --2- 172.21.15.16:0/829250014 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5565dd8805d0 0x5565dd884120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:28.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.228+0000 7fc371ad1640 1 -- 172.21.15.16:0/829250014 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5565dd88f780 msgr2=0x5565dd884ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:28.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.228+0000 7fc371ad1640 1 --2- 172.21.15.16:0/829250014 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5565dd88f780 0x5565dd884ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:28.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.228+0000 7fc371ad1640 1 -- 172.21.15.16:0/829250014 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565dd8969e0 con 0x5565dd881b40 2024-09-12T02:02:28.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.228+0000 7fc3722d2640 1 --2- 172.21.15.16:0/829250014 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5565dd8805d0 0x5565dd884120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:02:28.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.228+0000 7fc371ad1640 1 --2- 172.21.15.16:0/829250014 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565dd881b40 0x5565dd884660 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fc35c0bbdf0 tx=0x7fc35c0c0160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0022823fb2cc823 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:02:28.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.228+0000 7fc355ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:02:28.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.228+0000 7fc355ffb640 1 -- 172.21.15.16:0/829250014 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc35c049040 con 0x5565dd881b40 2024-09-12T02:02:28.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.228+0000 7fc355ffb640 1 -- 172.21.15.16:0/829250014 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fc35c0c4790 con 0x5565dd881b40 2024-09-12T02:02:28.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.228+0000 7fc37690f740 1 -- 172.21.15.16:0/829250014 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc36402a070 con 0x5565dd881b40 2024-09-12T02:02:28.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.228+0000 7fc355ffb640 1 -- 172.21.15.16:0/829250014 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc35c0ba220 con 0x5565dd881b40 2024-09-12T02:02:28.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.228+0000 7fc37690f740 10 client.15450 fetch_fsmap learned FSMap version 25 2024-09-12T02:02:28.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.228+0000 7fc37690f740 1 -- 172.21.15.16:0/829250014 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5565dd897d40 con 0x5565dd881b40 2024-09-12T02:02:28.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.229+0000 7fc355ffb640 1 -- 172.21.15.16:0/829250014 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc35c0bf030 con 0x5565dd881b40 2024-09-12T02:02:28.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.229+0000 7fc355ffb640 1 -- 172.21.15.16:0/829250014 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fc35c0bb2b0 con 0x5565dd881b40 2024-09-12T02:02:28.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.229+0000 7fc37690f740 10 client.15450 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:02:28.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.229+0000 7fc37690f740 10 client.15450 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-12T02:02:28.234 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T02:02:28.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.229+0000 7fc37690f740 20 client.15450 populate_metadata read hostname 'smithi016' 2024-09-12T02:02:28.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.229+0000 7fc37690f740 1 --2- 172.21.15.16:0/829250014 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x5565dd90e980 0x5565dd910d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:28.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.229+0000 7fc37690f740 4 client.15450 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T02:02:28.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.229+0000 7fc37690f740 1 -- 172.21.15.16:0/829250014 --> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fc36402a070 con 0x5565dd90e980 2024-09-12T02:02:28.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.229+0000 7fc3722d2640 1 --2- 172.21.15.16:0/829250014 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x5565dd90e980 0x5565dd910d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:28.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.230+0000 7fc355ffb640 10 client.15450 ms_handle_connect on v2:172.21.15.184:6836/1793393087 2024-09-12T02:02:28.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.230+0000 7fc3722d2640 1 --2- 172.21.15.16:0/829250014 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x5565dd90e980 0x5565dd910d70 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b860667f2053a9ac server_cookie=377282e4a0157fb9 in_seq=0 out_seq=0 2024-09-12T02:02:28.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.230+0000 7fc355ffb640 1 -- 172.21.15.16:0/829250014 <== mds.2 v2:172.21.15.184:6836/1793393087 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fc36402a070 con 0x5565dd90e980 2024-09-12T02:02:28.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.230+0000 7fc355ffb640 10 client.15450 handle_command_reply: tid=0 2024-09-12T02:02:28.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.230+0000 7fc37690f740 1 client.15450 shutdown 2024-09-12T02:02:28.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.230+0000 7fc37690f740 1 -- 172.21.15.16:0/829250014 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565dd881b40 msgr2=0x5565dd884660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:28.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.230+0000 7fc37690f740 1 --2- 172.21.15.16:0/829250014 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565dd881b40 0x5565dd884660 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fc35c0bbdf0 tx=0x7fc35c0c0160 comp rx=0 tx=0).stop 2024-09-12T02:02:28.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.230+0000 7fc37690f740 1 -- 172.21.15.16:0/829250014 shutdown_connections 2024-09-12T02:02:28.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.230+0000 7fc37690f740 1 --2- 172.21.15.16:0/829250014 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5565dd88f780 0x5565dd884ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:28.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.230+0000 7fc37690f740 1 --2- 172.21.15.16:0/829250014 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x5565dd90e980 0x5565dd910d70 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:28.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.230+0000 7fc355ffb640 0 client.15450 ms_handle_reset on v2:172.21.15.184:6836/1793393087 2024-09-12T02:02:28.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.230+0000 7fc37690f740 1 --2- 172.21.15.16:0/829250014 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565dd881b40 0x5565dd884660 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:28.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.230+0000 7fc37690f740 1 --2- 172.21.15.16:0/829250014 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5565dd8805d0 0x5565dd884120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:28.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.230+0000 7fc37690f740 1 -- 172.21.15.16:0/829250014 >> 172.21.15.16:0/829250014 conn(0x5565dd849310 msgr2=0x5565dd88b490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:02:28.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.231+0000 7fc37690f740 1 -- 172.21.15.16:0/829250014 shutdown_connections 2024-09-12T02:02:28.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.231+0000 7fc37690f740 1 -- 172.21.15.16:0/829250014 wait complete. 2024-09-12T02:02:28.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.231+0000 7fc37690f740 20 client.15450 trim_cache size 0 max 16384 2024-09-12T02:02:28.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.231+0000 7fc374536640 1 -- 172.21.15.16:0/956457415 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc3580a2c00 msgr2=0x7fc3580a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:28.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.231+0000 7fc374536640 1 --2- 172.21.15.16:0/956457415 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc3580a2c00 0x7fc3580a50c0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fc368005fd0 tx=0x7fc36800b4a0 comp rx=0 tx=0).stop 2024-09-12T02:02:28.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.231+0000 7fc374536640 1 -- 172.21.15.16:0/956457415 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc36c102570 msgr2=0x7fc36c198aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:28.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.231+0000 7fc374536640 1 --2- 172.21.15.16:0/956457415 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc36c102570 0x7fc36c198aa0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc35c005940 tx=0x7fc35c003c10 comp rx=0 tx=0).stop 2024-09-12T02:02:28.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.231+0000 7fc374536640 1 -- 172.21.15.16:0/956457415 shutdown_connections 2024-09-12T02:02:28.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.231+0000 7fc374536640 1 --2- 172.21.15.16:0/956457415 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc3580a2c00 0x7fc3580a50c0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:28.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.231+0000 7fc374536640 1 --2- 172.21.15.16:0/956457415 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc36c102ef0 0x7fc36c19ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:28.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.231+0000 7fc374536640 1 --2- 172.21.15.16:0/956457415 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc36c102570 0x7fc36c198aa0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:28.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.231+0000 7fc374536640 1 --2- 172.21.15.16:0/956457415 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc36c100e20 0x7fc36c198560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:28.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.231+0000 7fc374536640 1 -- 172.21.15.16:0/956457415 >> 172.21.15.16:0/956457415 conn(0x7fc36c0fc7d0 msgr2=0x7fc36c06a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:02:28.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.232+0000 7fc374536640 1 -- 172.21.15.16:0/956457415 shutdown_connections 2024-09-12T02:02:28.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:28.232+0000 7fc374536640 1 -- 172.21.15.16:0/956457415 wait complete. 2024-09-12T02:02:28.925 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-12T02:02:29.216 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:02:29.627 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:02:29 smithi016 ceph-mon[30947]: pgmap v352: 97 pgs: 97 active+clean; 661 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 426 B/s wr, 5 op/s 2024-09-12T02:02:29.668 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:02:29 smithi033 ceph-mon[35332]: pgmap v352: 97 pgs: 97 active+clean; 661 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 426 B/s wr, 5 op/s 2024-09-12T02:02:29.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.682+0000 7ff2fa82d640 1 -- 172.21.15.16:0/2943980497 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2f4100e20 msgr2=0x7ff2f4101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:29.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.682+0000 7ff2fa82d640 1 --2- 172.21.15.16:0/2943980497 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2f4100e20 0x7ff2f4101200 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7ff2e4009900 tx=0x7ff2e402a480 comp rx=0 tx=0).stop 2024-09-12T02:02:29.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.682+0000 7ff2fa82d640 1 -- 172.21.15.16:0/2943980497 shutdown_connections 2024-09-12T02:02:29.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.682+0000 7ff2fa82d640 1 --2- 172.21.15.16:0/2943980497 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff2f4102ef0 0x7ff2f410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:29.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.682+0000 7ff2fa82d640 1 --2- 172.21.15.16:0/2943980497 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2f4102570 0x7ff2f41029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:29.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.682+0000 7ff2fa82d640 1 --2- 172.21.15.16:0/2943980497 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2f4100e20 0x7ff2f4101200 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:29.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.682+0000 7ff2fa82d640 1 -- 172.21.15.16:0/2943980497 >> 172.21.15.16:0/2943980497 conn(0x7ff2f40fc7d0 msgr2=0x7ff2f40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:02:29.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.682+0000 7ff2fa82d640 1 -- 172.21.15.16:0/2943980497 shutdown_connections 2024-09-12T02:02:29.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.683+0000 7ff2fa82d640 1 -- 172.21.15.16:0/2943980497 wait complete. 2024-09-12T02:02:29.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.683+0000 7ff2fa82d640 1 Processor -- start 2024-09-12T02:02:29.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.683+0000 7ff2fa82d640 1 -- start start 2024-09-12T02:02:29.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.684+0000 7ff2fa82d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2f4100e20 0x7ff2f419a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:29.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.684+0000 7ff2fa82d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff2f4102570 0x7ff2f419abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:29.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.684+0000 7ff2fa82d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2f4102ef0 0x7ff2f419ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:29.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.684+0000 7ff2f8dca640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2f4102ef0 0x7ff2f419ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:29.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.684+0000 7ff2fa82d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff2f4110fa0 con 0x7ff2f4102ef0 2024-09-12T02:02:29.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.684+0000 7ff2fa82d640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff2f4110e20 con 0x7ff2f4100e20 2024-09-12T02:02:29.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.684+0000 7ff2fa82d640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff2f4111120 con 0x7ff2f4102570 2024-09-12T02:02:29.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.684+0000 7ff2f8dca640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2f4102ef0 0x7ff2f419ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40600/0 (socket says 172.21.15.16:40600) 2024-09-12T02:02:29.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.684+0000 7ff2f8dca640 1 -- 172.21.15.16:0/1972738362 learned_addr learned my addr 172.21.15.16:0/1972738362 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:02:29.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.684+0000 7ff2f37fe640 1 --2- 172.21.15.16:0/1972738362 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff2f4102570 0x7ff2f419abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:29.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.684+0000 7ff2f3fff640 1 --2- 172.21.15.16:0/1972738362 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2f4100e20 0x7ff2f419a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:29.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.684+0000 7ff2f8dca640 1 -- 172.21.15.16:0/1972738362 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff2f4102570 msgr2=0x7ff2f419abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:29.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.684+0000 7ff2f8dca640 1 --2- 172.21.15.16:0/1972738362 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff2f4102570 0x7ff2f419abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:29.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.684+0000 7ff2f8dca640 1 -- 172.21.15.16:0/1972738362 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2f4100e20 msgr2=0x7ff2f419a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:29.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.684+0000 7ff2f8dca640 1 --2- 172.21.15.16:0/1972738362 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2f4100e20 0x7ff2f419a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:29.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.684+0000 7ff2f8dca640 1 -- 172.21.15.16:0/1972738362 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2f419f650 con 0x7ff2f4102ef0 2024-09-12T02:02:29.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.685+0000 7ff2f8dca640 1 --2- 172.21.15.16:0/1972738362 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2f4102ef0 0x7ff2f419ef70 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7ff2ec00eb10 tx=0x7ff2ec00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:02:29.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.685+0000 7ff2f17fa640 1 -- 172.21.15.16:0/1972738362 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2ec004b20 con 0x7ff2f4102ef0 2024-09-12T02:02:29.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.685+0000 7ff2f17fa640 1 -- 172.21.15.16:0/1972738362 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2ec004cc0 con 0x7ff2f4102ef0 2024-09-12T02:02:29.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.685+0000 7ff2f17fa640 1 -- 172.21.15.16:0/1972738362 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2ec010760 con 0x7ff2f4102ef0 2024-09-12T02:02:29.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.685+0000 7ff2fa82d640 1 -- 172.21.15.16:0/1972738362 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2f419f940 con 0x7ff2f4102ef0 2024-09-12T02:02:29.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.685+0000 7ff2fa82d640 1 -- 172.21.15.16:0/1972738362 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2f41a7220 con 0x7ff2f4102ef0 2024-09-12T02:02:29.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.688+0000 7ff2f17fa640 1 -- 172.21.15.16:0/1972738362 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff2ec005020 con 0x7ff2f4102ef0 2024-09-12T02:02:29.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.689+0000 7ff2fcc06740 1 Processor -- start 2024-09-12T02:02:29.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.689+0000 7ff2f17fa640 1 --2- 172.21.15.16:0/1972738362 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7ff2dc0a2c00 0x7ff2dc0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:29.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.689+0000 7ff2fcc06740 1 -- start start 2024-09-12T02:02:29.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.689+0000 7ff2f17fa640 1 -- 172.21.15.16:0/1972738362 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7ff2ec0cf620 con 0x7ff2f4102ef0 2024-09-12T02:02:29.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.689+0000 7ff2fcc06740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56340cfdbb40 0x56340cfda090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:29.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.689+0000 7ff2fcc06740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56340cfda5d0 0x56340cfdaa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:29.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.689+0000 7ff2fcc06740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56340cfedb80 0x56340cfeff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:29.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.689+0000 7ff2fcc06740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56340cf402f0 con 0x56340cfda5d0 2024-09-12T02:02:29.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.689+0000 7ff2fcc06740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56340cee7cf0 con 0x56340cfedb80 2024-09-12T02:02:29.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.689+0000 7ff2fcc06740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x56340cf3c2b0 con 0x56340cfdbb40 2024-09-12T02:02:29.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.689+0000 7ff2f37fe640 1 --2- 172.21.15.16:0/1972738362 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7ff2dc0a2c00 0x7ff2dc0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:29.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.689+0000 7ff2f8dca640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56340cfedb80 0x56340cfeff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:29.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.690+0000 7ff2f8dca640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56340cfedb80 0x56340cfeff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.16:48292/0 (socket says 172.21.15.16:48292) 2024-09-12T02:02:29.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.690+0000 7ff2f8dca640 1 -- 172.21.15.16:0/2218426288 learned_addr learned my addr 172.21.15.16:0/2218426288 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:02:29.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.690+0000 7ff2f37fe640 1 --2- 172.21.15.16:0/2218426288 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56340cfda5d0 0x56340cfdaa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:29.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.690+0000 7ff2f3fff640 1 --2- 172.21.15.16:0/2218426288 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56340cfdbb40 0x56340cfda090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:29.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.690+0000 7ff2f8dca640 1 -- 172.21.15.16:0/2218426288 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56340cfdbb40 msgr2=0x56340cfda090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:29.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.690+0000 7ff2f8dca640 1 --2- 172.21.15.16:0/2218426288 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56340cfdbb40 0x56340cfda090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:29.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.690+0000 7ff2f8dca640 1 -- 172.21.15.16:0/2218426288 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56340cfda5d0 msgr2=0x56340cfdaa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:29.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.690+0000 7ff2f8dca640 1 --2- 172.21.15.16:0/2218426288 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56340cfda5d0 0x56340cfdaa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:29.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.690+0000 7ff2f8dca640 1 -- 172.21.15.16:0/2218426288 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56340cecc3a0 con 0x56340cfedb80 2024-09-12T02:02:29.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.690+0000 7ff2f37fe640 1 --2- 172.21.15.16:0/1972738362 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7ff2dc0a2c00 0x7ff2dc0a50c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7ff2f4104bb0 tx=0x7ff2e8009f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:02:29.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.690+0000 7ff2f8dca640 1 --2- 172.21.15.16:0/2218426288 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56340cfedb80 0x56340cfeff70 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff2ec098850 tx=0x7ff2ec09b9c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d7bb37bd29ab5bd server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:02:29.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.690+0000 7ff2da7fc640 1 -- 172.21.15.16:0/2218426288 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2ec09be80 con 0x56340cfedb80 2024-09-12T02:02:29.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.690+0000 7ff2da7fc640 1 -- 172.21.15.16:0/2218426288 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2ec0a0230 con 0x56340cfedb80 2024-09-12T02:02:29.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.691+0000 7ff2da7fc640 1 -- 172.21.15.16:0/2218426288 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2ec0a48a0 con 0x56340cfedb80 2024-09-12T02:02:29.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.691+0000 7ff2fcc06740 1 -- 172.21.15.16:0/2218426288 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56340cfedb80 msgr2=0x56340cfeff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:29.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.691+0000 7ff2fcc06740 1 --2- 172.21.15.16:0/2218426288 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56340cfedb80 0x56340cfeff70 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff2ec098850 tx=0x7ff2ec09b9c0 comp rx=0 tx=0).stop 2024-09-12T02:02:29.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.692+0000 7ff2fcc06740 1 -- 172.21.15.16:0/2218426288 shutdown_connections 2024-09-12T02:02:29.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.692+0000 7ff2fcc06740 1 --2- 172.21.15.16:0/2218426288 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56340cfedb80 0x56340cfeff70 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:29.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.692+0000 7ff2fcc06740 1 --2- 172.21.15.16:0/2218426288 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56340cfda5d0 0x56340cfdaa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:29.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.692+0000 7ff2fcc06740 1 --2- 172.21.15.16:0/2218426288 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56340cfdbb40 0x56340cfda090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:29.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.692+0000 7ff2fcc06740 1 -- 172.21.15.16:0/2218426288 >> 172.21.15.16:0/2218426288 conn(0x56340cfa3310 msgr2=0x56340cfd6c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:02:29.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.692+0000 7ff2fcc06740 1 -- 172.21.15.16:0/2218426288 shutdown_connections 2024-09-12T02:02:29.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.692+0000 7ff2fcc06740 1 -- 172.21.15.16:0/2218426288 wait complete. 2024-09-12T02:02:29.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.692+0000 7ff2fcc06740 1 -- start start 2024-09-12T02:02:29.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.692+0000 7ff2fcc06740 1 Processor -- start 2024-09-12T02:02:29.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.693+0000 7ff2fcc06740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56340cfda5d0 0x56340cfed570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:29.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.693+0000 7ff2fcc06740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56340cfdbb40 0x56340cfea820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:29.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.693+0000 7ff2fcc06740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56340cfedb80 0x56340cfeae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:29.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.693+0000 7ff2fcc06740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56340cff0880 con 0x56340cfedb80 2024-09-12T02:02:29.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.693+0000 7ff2fcc06740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56340cefb060 con 0x56340cfda5d0 2024-09-12T02:02:29.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.693+0000 7ff2fcc06740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x56340cff0a00 con 0x56340cfdbb40 2024-09-12T02:02:29.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.693+0000 7ff2fcc06740 10 client.0 _ll_register_callbacks cb 0x56340cc4e8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:02:29.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.693+0000 7ff2f8dca640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56340cfedb80 0x56340cfeae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:29.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.693+0000 7ff2f8dca640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56340cfedb80 0x56340cfeae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40626/0 (socket says 172.21.15.16:40626) 2024-09-12T02:02:29.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.693+0000 7ff2f8dca640 1 -- 172.21.15.16:0/1286258803 learned_addr learned my addr 172.21.15.16:0/1286258803 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:02:29.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.693+0000 7ff2f37fe640 1 --2- 172.21.15.16:0/1286258803 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56340cfdbb40 0x56340cfea820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:29.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.693+0000 7ff2f3fff640 1 --2- 172.21.15.16:0/1286258803 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56340cfda5d0 0x56340cfed570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:29.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.693+0000 7ff2f8dca640 1 -- 172.21.15.16:0/1286258803 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56340cfdbb40 msgr2=0x56340cfea820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:29.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.693+0000 7ff2f8dca640 1 --2- 172.21.15.16:0/1286258803 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56340cfdbb40 0x56340cfea820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:29.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.693+0000 7ff2f8dca640 1 -- 172.21.15.16:0/1286258803 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56340cfda5d0 msgr2=0x56340cfed570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:29.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.693+0000 7ff2f8dca640 1 --2- 172.21.15.16:0/1286258803 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56340cfda5d0 0x56340cfed570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:29.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.693+0000 7ff2f8dca640 1 -- 172.21.15.16:0/1286258803 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56340ccb13c0 con 0x56340cfedb80 2024-09-12T02:02:29.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.694+0000 7ff2f8dca640 1 --2- 172.21.15.16:0/1286258803 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56340cfedb80 0x56340cfeae10 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7ff2ec0a45c0 tx=0x7ff2ec09c640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8dbe1b2f410a18d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:02:29.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.694+0000 7ff2d37fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:02:29.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.694+0000 7ff2d37fe640 1 -- 172.21.15.16:0/1286258803 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2ec005510 con 0x56340cfedb80 2024-09-12T02:02:29.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.694+0000 7ff2d37fe640 1 -- 172.21.15.16:0/1286258803 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7ff2ec0cf110 con 0x56340cfedb80 2024-09-12T02:02:29.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.694+0000 7ff2fcc06740 1 -- 172.21.15.16:0/1286258803 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2ec0a0230 con 0x56340cfedb80 2024-09-12T02:02:29.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.694+0000 7ff2d37fe640 1 -- 172.21.15.16:0/1286258803 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff2ec09be80 con 0x56340cfedb80 2024-09-12T02:02:29.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.694+0000 7ff2fcc06740 10 client.15462 fetch_fsmap learned FSMap version 25 2024-09-12T02:02:29.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.694+0000 7ff2fcc06740 1 -- 172.21.15.16:0/1286258803 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56340cffc320 con 0x56340cfedb80 2024-09-12T02:02:29.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.694+0000 7ff2d37fe640 1 -- 172.21.15.16:0/1286258803 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2ec0a0230 con 0x56340cfedb80 2024-09-12T02:02:29.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.695+0000 7ff2d37fe640 1 -- 172.21.15.16:0/1286258803 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7ff2ec0baac0 con 0x56340cfedb80 2024-09-12T02:02:29.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.695+0000 7ff2fcc06740 10 client.15462 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:02:29.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.695+0000 7ff2fcc06740 10 client.15462 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T02:02:29.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.695+0000 7ff2fcc06740 20 client.15462 populate_metadata read hostname 'smithi016' 2024-09-12T02:02:29.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.695+0000 7ff2fcc06740 1 --2- 172.21.15.16:0/1286258803 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x56340cffd670 0x56340cfffa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:29.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.695+0000 7ff2fcc06740 4 client.15462 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:02:29.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.695+0000 7ff2fcc06740 1 -- 172.21.15.16:0/1286258803 --> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56340cf4d860 con 0x56340cffd670 2024-09-12T02:02:29.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.695+0000 7ff2f3fff640 1 --2- 172.21.15.16:0/1286258803 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x56340cffd670 0x56340cfffa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:29.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.696+0000 7ff2f3fff640 1 --2- 172.21.15.16:0/1286258803 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x56340cffd670 0x56340cfffa60 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=809d41c9a6217037 server_cookie=bdb978ebc0808805 in_seq=0 out_seq=0 2024-09-12T02:02:29.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.696+0000 7ff2d37fe640 10 client.15462 ms_handle_connect on v2:172.21.15.184:6838/4203347941 2024-09-12T02:02:29.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.696+0000 7ff2d37fe640 1 -- 172.21.15.16:0/1286258803 <== mds.1 v2:172.21.15.184:6838/4203347941 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56340cf4d860 con 0x56340cffd670 2024-09-12T02:02:29.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.696+0000 7ff2d37fe640 10 client.15462 handle_command_reply: tid=0 2024-09-12T02:02:29.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.696+0000 7ff2fcc06740 1 client.15462 shutdown 2024-09-12T02:02:29.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.697+0000 7ff2fcc06740 1 -- 172.21.15.16:0/1286258803 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56340cfedb80 msgr2=0x56340cfeae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:29.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.697+0000 7ff2fcc06740 1 --2- 172.21.15.16:0/1286258803 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56340cfedb80 0x56340cfeae10 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7ff2ec0a45c0 tx=0x7ff2ec09c640 comp rx=0 tx=0).stop 2024-09-12T02:02:29.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.697+0000 7ff2fcc06740 1 -- 172.21.15.16:0/1286258803 shutdown_connections 2024-09-12T02:02:29.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.697+0000 7ff2fcc06740 1 --2- 172.21.15.16:0/1286258803 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x56340cffd670 0x56340cfffa60 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:29.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.697+0000 7ff2fcc06740 1 --2- 172.21.15.16:0/1286258803 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56340cfedb80 0x56340cfeae10 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:29.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.697+0000 7ff2fcc06740 1 --2- 172.21.15.16:0/1286258803 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56340cfdbb40 0x56340cfea820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:29.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.697+0000 7ff2d37fe640 0 client.15462 ms_handle_reset on v2:172.21.15.184:6838/4203347941 2024-09-12T02:02:29.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.697+0000 7ff2fcc06740 1 --2- 172.21.15.16:0/1286258803 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56340cfda5d0 0x56340cfed570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:29.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.697+0000 7ff2fcc06740 1 -- 172.21.15.16:0/1286258803 >> 172.21.15.16:0/1286258803 conn(0x56340cfa3310 msgr2=0x56340cfdc9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:02:29.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.697+0000 7ff2fcc06740 1 -- 172.21.15.16:0/1286258803 shutdown_connections 2024-09-12T02:02:29.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.697+0000 7ff2fcc06740 1 -- 172.21.15.16:0/1286258803 wait complete. 2024-09-12T02:02:29.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.697+0000 7ff2fcc06740 20 client.15462 trim_cache size 0 max 16384 2024-09-12T02:02:29.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.722+0000 7ff2fcc06740 1 Processor -- start 2024-09-12T02:02:29.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.722+0000 7ff2fcc06740 1 -- start start 2024-09-12T02:02:29.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.722+0000 7ff2fcc06740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56340cfda5d0 0x56340cfd6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:29.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.722+0000 7ff2fcc06740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56340cfdbb40 0x56340cfea0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:29.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.722+0000 7ff2fcc06740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56340cfedb80 0x56340d002a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:29.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.722+0000 7ff2fcc06740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff2f41adb90 con 0x56340cfdbb40 2024-09-12T02:02:29.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.722+0000 7ff2fcc06740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff2f41ab460 con 0x56340cfedb80 2024-09-12T02:02:29.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.722+0000 7ff2fcc06740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff2f41a92b0 con 0x56340cfda5d0 2024-09-12T02:02:29.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.722+0000 7ff2f37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56340cfdbb40 0x56340cfea0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:29.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.723+0000 7ff2f37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56340cfdbb40 0x56340cfea0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40638/0 (socket says 172.21.15.16:40638) 2024-09-12T02:02:29.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.723+0000 7ff2f37fe640 1 -- 172.21.15.16:0/1098239140 learned_addr learned my addr 172.21.15.16:0/1098239140 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:02:29.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.723+0000 7ff2f8dca640 1 --2- 172.21.15.16:0/1098239140 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56340cfedb80 0x56340d002a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:29.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.723+0000 7ff2f37fe640 1 -- 172.21.15.16:0/1098239140 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56340cfda5d0 msgr2=0x56340cfd6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:29.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.723+0000 7ff2f37fe640 1 --2- 172.21.15.16:0/1098239140 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56340cfda5d0 0x56340cfd6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:29.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.723+0000 7ff2f37fe640 1 -- 172.21.15.16:0/1098239140 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56340cfedb80 msgr2=0x56340d002a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:29.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.723+0000 7ff2f37fe640 1 --2- 172.21.15.16:0/1098239140 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56340cfedb80 0x56340d002a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:29.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.723+0000 7ff2f37fe640 1 -- 172.21.15.16:0/1098239140 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56340cfe97f0 con 0x56340cfdbb40 2024-09-12T02:02:29.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.723+0000 7ff2f37fe640 1 --2- 172.21.15.16:0/1098239140 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56340cfdbb40 0x56340cfea0e0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x56340cfd7ac0 tx=0x7ff2e801f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1a6c565713b6761 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:02:29.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.724+0000 7ff2d9ffb640 1 -- 172.21.15.16:0/1098239140 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2e8004020 con 0x56340cfdbb40 2024-09-12T02:02:29.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.724+0000 7ff2d9ffb640 1 -- 172.21.15.16:0/1098239140 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2e80041c0 con 0x56340cfdbb40 2024-09-12T02:02:29.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.724+0000 7ff2d9ffb640 1 -- 172.21.15.16:0/1098239140 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2e800f900 con 0x56340cfdbb40 2024-09-12T02:02:29.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.724+0000 7ff2fcc06740 1 -- 172.21.15.16:0/1098239140 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56340cfdbb40 msgr2=0x56340cfea0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:29.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.724+0000 7ff2fcc06740 1 --2- 172.21.15.16:0/1098239140 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56340cfdbb40 0x56340cfea0e0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x56340cfd7ac0 tx=0x7ff2e801f000 comp rx=0 tx=0).stop 2024-09-12T02:02:29.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.725+0000 7ff2fcc06740 1 -- 172.21.15.16:0/1098239140 shutdown_connections 2024-09-12T02:02:29.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.725+0000 7ff2fcc06740 1 --2- 172.21.15.16:0/1098239140 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56340cfedb80 0x56340d002a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:29.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.725+0000 7ff2fcc06740 1 --2- 172.21.15.16:0/1098239140 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56340cfdbb40 0x56340cfea0e0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:29.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.725+0000 7ff2fcc06740 1 --2- 172.21.15.16:0/1098239140 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56340cfda5d0 0x56340cfd6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:29.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.725+0000 7ff2fcc06740 1 -- 172.21.15.16:0/1098239140 >> 172.21.15.16:0/1098239140 conn(0x56340cfa3310 msgr2=0x56340cfd6700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:02:29.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.725+0000 7ff2fcc06740 1 -- 172.21.15.16:0/1098239140 shutdown_connections 2024-09-12T02:02:29.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.725+0000 7ff2fcc06740 1 -- 172.21.15.16:0/1098239140 wait complete. 2024-09-12T02:02:29.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.725+0000 7ff2fcc06740 1 -- start start 2024-09-12T02:02:29.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.726+0000 7ff2fcc06740 1 Processor -- start 2024-09-12T02:02:29.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.726+0000 7ff2fcc06740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56340cfda5d0 0x56340d000b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:29.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.726+0000 7ff2fcc06740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56340cfdbb40 0x56340d0010a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:29.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.726+0000 7ff2fcc06740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56340cfedb80 0x56340d066fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:29.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.726+0000 7ff2fcc06740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff2f41a9920 con 0x56340cfdbb40 2024-09-12T02:02:29.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.726+0000 7ff2fcc06740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff2f41aad90 con 0x56340cfedb80 2024-09-12T02:02:29.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.726+0000 7ff2f3fff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56340cfda5d0 0x56340d000b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:29.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.726+0000 7ff2fcc06740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff2f41a9ff0 con 0x56340cfda5d0 2024-09-12T02:02:29.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.726+0000 7ff2f3fff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56340cfda5d0 0x56340d000b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:38598/0 (socket says 172.21.15.16:38598) 2024-09-12T02:02:29.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.726+0000 7ff2f3fff640 1 -- 172.21.15.16:0/1488773689 learned_addr learned my addr 172.21.15.16:0/1488773689 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:02:29.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.726+0000 7ff2f37fe640 1 --2- 172.21.15.16:0/1488773689 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56340cfdbb40 0x56340d0010a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:29.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.726+0000 7ff2f8dca640 1 --2- 172.21.15.16:0/1488773689 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56340cfedb80 0x56340d066fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:29.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.727+0000 7ff2fcc06740 10 client.0 _ll_register_callbacks cb 0x7ff2ec0b6070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:02:29.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.727+0000 7ff2f37fe640 1 -- 172.21.15.16:0/1488773689 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56340cfda5d0 msgr2=0x56340d000b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:29.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.727+0000 7ff2f37fe640 1 --2- 172.21.15.16:0/1488773689 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56340cfda5d0 0x56340d000b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:29.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.727+0000 7ff2f37fe640 1 -- 172.21.15.16:0/1488773689 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56340cfedb80 msgr2=0x56340d066fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:29.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.727+0000 7ff2f37fe640 1 --2- 172.21.15.16:0/1488773689 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56340cfedb80 0x56340d066fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:29.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.727+0000 7ff2f37fe640 1 -- 172.21.15.16:0/1488773689 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56340cfed8e0 con 0x56340cfdbb40 2024-09-12T02:02:29.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.727+0000 7ff2f8dca640 1 --2- 172.21.15.16:0/1488773689 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56340cfedb80 0x56340d066fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:02:29.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.727+0000 7ff2f37fe640 1 --2- 172.21.15.16:0/1488773689 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56340cfdbb40 0x56340d0010a0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7ff2e8009800 tx=0x7ff2e8003c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2cda186091b9b0b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:02:29.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.727+0000 7ff2d37fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:02:29.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.728+0000 7ff2d37fe640 1 -- 172.21.15.16:0/1488773689 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2e8039030 con 0x56340cfdbb40 2024-09-12T02:02:29.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.728+0000 7ff2d37fe640 1 -- 172.21.15.16:0/1488773689 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7ff2e8026050 con 0x56340cfdbb40 2024-09-12T02:02:29.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.728+0000 7ff2fcc06740 1 -- 172.21.15.16:0/1488773689 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2e80041c0 con 0x56340cfdbb40 2024-09-12T02:02:29.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.728+0000 7ff2d37fe640 1 -- 172.21.15.16:0/1488773689 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff2e8005040 con 0x56340cfdbb40 2024-09-12T02:02:29.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.728+0000 7ff2fcc06740 10 client.15468 fetch_fsmap learned FSMap version 25 2024-09-12T02:02:29.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.728+0000 7ff2fcc06740 1 -- 172.21.15.16:0/1488773689 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56340d0683e0 con 0x56340cfdbb40 2024-09-12T02:02:29.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.729+0000 7ff2d37fe640 1 -- 172.21.15.16:0/1488773689 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2e80041c0 con 0x56340cfdbb40 2024-09-12T02:02:29.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.729+0000 7ff2d37fe640 1 -- 172.21.15.16:0/1488773689 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7ff2e801f070 con 0x56340cfdbb40 2024-09-12T02:02:29.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.729+0000 7ff2fcc06740 10 client.15468 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:02:29.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.729+0000 7ff2fcc06740 10 client.15468 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T02:02:29.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.729+0000 7ff2fcc06740 20 client.15468 populate_metadata read hostname 'smithi016' 2024-09-12T02:02:29.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.729+0000 7ff2fcc06740 1 --2- 172.21.15.16:0/1488773689 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x56340cffd670 0x56340d06aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:29.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.729+0000 7ff2fcc06740 4 client.15468 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T02:02:29.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.729+0000 7ff2fcc06740 1 -- 172.21.15.16:0/1488773689 --> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56340cf47d70 con 0x56340cffd670 2024-09-12T02:02:29.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.729+0000 7ff2f3fff640 1 --2- 172.21.15.16:0/1488773689 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x56340cffd670 0x56340d06aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:29.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.730+0000 7ff2d37fe640 10 client.15468 ms_handle_connect on v2:172.21.15.184:6838/4203347941 2024-09-12T02:02:29.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.730+0000 7ff2f3fff640 1 --2- 172.21.15.16:0/1488773689 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x56340cffd670 0x56340d06aba0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=37db43bf590d5073 server_cookie=72641d7f4c0a82f0 in_seq=0 out_seq=0 2024-09-12T02:02:29.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.730+0000 7ff2d37fe640 1 -- 172.21.15.16:0/1488773689 <== mds.1 v2:172.21.15.184:6838/4203347941 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56340cf47d70 con 0x56340cffd670 2024-09-12T02:02:29.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.730+0000 7ff2d37fe640 10 client.15468 handle_command_reply: tid=0 2024-09-12T02:02:29.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.730+0000 7ff2fcc06740 1 client.15468 shutdown 2024-09-12T02:02:29.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.730+0000 7ff2fcc06740 1 -- 172.21.15.16:0/1488773689 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56340cfdbb40 msgr2=0x56340d0010a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:29.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.730+0000 7ff2fcc06740 1 --2- 172.21.15.16:0/1488773689 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56340cfdbb40 0x56340d0010a0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7ff2e8009800 tx=0x7ff2e8003c50 comp rx=0 tx=0).stop 2024-09-12T02:02:29.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.730+0000 7ff2fcc06740 1 -- 172.21.15.16:0/1488773689 shutdown_connections 2024-09-12T02:02:29.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.730+0000 7ff2fcc06740 1 --2- 172.21.15.16:0/1488773689 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x56340cffd670 0x56340d06aba0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:29.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.730+0000 7ff2fcc06740 1 --2- 172.21.15.16:0/1488773689 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56340cfedb80 0x56340d066fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:29.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.730+0000 7ff2fcc06740 1 --2- 172.21.15.16:0/1488773689 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56340cfdbb40 0x56340d0010a0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:29.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.730+0000 7ff2fcc06740 1 --2- 172.21.15.16:0/1488773689 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56340cfda5d0 0x56340d000b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:29.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.730+0000 7ff2fcc06740 1 -- 172.21.15.16:0/1488773689 >> 172.21.15.16:0/1488773689 conn(0x56340cfa3310 msgr2=0x56340cfdc110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:02:29.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.730+0000 7ff2d37fe640 0 client.15468 ms_handle_reset on v2:172.21.15.184:6838/4203347941 2024-09-12T02:02:29.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.731+0000 7ff2fcc06740 1 -- 172.21.15.16:0/1488773689 shutdown_connections 2024-09-12T02:02:29.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.731+0000 7ff2fcc06740 1 -- 172.21.15.16:0/1488773689 wait complete. 2024-09-12T02:02:29.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.731+0000 7ff2fcc06740 20 client.15468 trim_cache size 0 max 16384 2024-09-12T02:02:29.735 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T02:02:29.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.732+0000 7ff2fa82d640 1 -- 172.21.15.16:0/1972738362 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7ff2dc0a2c00 msgr2=0x7ff2dc0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:29.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.732+0000 7ff2fa82d640 1 --2- 172.21.15.16:0/1972738362 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7ff2dc0a2c00 0x7ff2dc0a50c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7ff2f4104bb0 tx=0x7ff2e8009f90 comp rx=0 tx=0).stop 2024-09-12T02:02:29.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.732+0000 7ff2fa82d640 1 -- 172.21.15.16:0/1972738362 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2f4102ef0 msgr2=0x7ff2f419ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:29.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.732+0000 7ff2fa82d640 1 --2- 172.21.15.16:0/1972738362 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2f4102ef0 0x7ff2f419ef70 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7ff2ec00eb10 tx=0x7ff2ec00c2d0 comp rx=0 tx=0).stop 2024-09-12T02:02:29.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.732+0000 7ff2fa82d640 1 -- 172.21.15.16:0/1972738362 shutdown_connections 2024-09-12T02:02:29.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.732+0000 7ff2fa82d640 1 --2- 172.21.15.16:0/1972738362 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7ff2dc0a2c00 0x7ff2dc0a50c0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:29.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.732+0000 7ff2fa82d640 1 --2- 172.21.15.16:0/1972738362 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2f4102ef0 0x7ff2f419ef70 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:29.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.732+0000 7ff2fa82d640 1 --2- 172.21.15.16:0/1972738362 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff2f4102570 0x7ff2f419abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:29.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.732+0000 7ff2fa82d640 1 --2- 172.21.15.16:0/1972738362 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2f4100e20 0x7ff2f419a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:29.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.732+0000 7ff2fa82d640 1 -- 172.21.15.16:0/1972738362 >> 172.21.15.16:0/1972738362 conn(0x7ff2f40fc7d0 msgr2=0x7ff2f406a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:02:29.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.733+0000 7ff2fa82d640 1 -- 172.21.15.16:0/1972738362 shutdown_connections 2024-09-12T02:02:29.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:29.733+0000 7ff2fa82d640 1 -- 172.21.15.16:0/1972738362 wait complete. 2024-09-12T02:02:29.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:02:29 smithi184 ceph-mon[35593]: pgmap v352: 97 pgs: 97 active+clean; 661 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 426 B/s wr, 5 op/s 2024-09-12T02:02:30.386 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-12T02:02:30.744 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:02:31.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.280+0000 7f1b1d8e9640 1 -- 172.21.15.16:0/1866260016 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b181002f0 msgr2=0x7f1b181006d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:31.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.280+0000 7f1b1d8e9640 1 --2- 172.21.15.16:0/1866260016 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b181002f0 0x7f1b181006d0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f1b08009930 tx=0x7f1b0802a420 comp rx=0 tx=0).stop 2024-09-12T02:02:31.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.280+0000 7f1b1d8e9640 1 -- 172.21.15.16:0/1866260016 shutdown_connections 2024-09-12T02:02:31.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.280+0000 7f1b1d8e9640 1 --2- 172.21.15.16:0/1866260016 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b18106f50 0x7f1b1810eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:31.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.280+0000 7f1b1d8e9640 1 --2- 172.21.15.16:0/1866260016 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1b18100ca0 0x7f1b18106840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:31.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.280+0000 7f1b1d8e9640 1 --2- 172.21.15.16:0/1866260016 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b181002f0 0x7f1b181006d0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:31.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.280+0000 7f1b1d8e9640 1 -- 172.21.15.16:0/1866260016 >> 172.21.15.16:0/1866260016 conn(0x7f1b18077210 msgr2=0x7f1b180fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:02:31.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.280+0000 7f1b1d8e9640 1 -- 172.21.15.16:0/1866260016 shutdown_connections 2024-09-12T02:02:31.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.280+0000 7f1b1d8e9640 1 -- 172.21.15.16:0/1866260016 wait complete. 2024-09-12T02:02:31.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.281+0000 7f1b1d8e9640 1 Processor -- start 2024-09-12T02:02:31.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.281+0000 7f1b1d8e9640 1 -- start start 2024-09-12T02:02:31.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.281+0000 7f1b1d8e9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b181002f0 0x7f1b18198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:31.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.281+0000 7f1b1d8e9640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1b18100ca0 0x7f1b18198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:31.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.281+0000 7f1b1d8e9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b18106f50 0x7f1b1819cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:31.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.281+0000 7f1b1d8e9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1b18111fa0 con 0x7f1b181002f0 2024-09-12T02:02:31.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.281+0000 7f1b1d8e9640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1b18111e20 con 0x7f1b18106f50 2024-09-12T02:02:31.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.281+0000 7f1b1d8e9640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1b18112120 con 0x7f1b18100ca0 2024-09-12T02:02:31.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.281+0000 7f1b16ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b181002f0 0x7f1b18198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:31.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.281+0000 7f1b16ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b181002f0 0x7f1b18198500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37826/0 (socket says 172.21.15.16:37826) 2024-09-12T02:02:31.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.281+0000 7f1b16ffd640 1 -- 172.21.15.16:0/2043494449 learned_addr learned my addr 172.21.15.16:0/2043494449 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:02:31.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.282+0000 7f1b177fe640 1 --2- 172.21.15.16:0/2043494449 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b18106f50 0x7f1b1819cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:31.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.282+0000 7f1b0ffff640 1 --2- 172.21.15.16:0/2043494449 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1b18100ca0 0x7f1b18198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:31.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.282+0000 7f1b16ffd640 1 -- 172.21.15.16:0/2043494449 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1b18100ca0 msgr2=0x7f1b18198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:31.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.282+0000 7f1b16ffd640 1 --2- 172.21.15.16:0/2043494449 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1b18100ca0 0x7f1b18198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:31.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.282+0000 7f1b16ffd640 1 -- 172.21.15.16:0/2043494449 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b18106f50 msgr2=0x7f1b1819cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:31.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.282+0000 7f1b16ffd640 1 --2- 172.21.15.16:0/2043494449 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b18106f50 0x7f1b1819cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:31.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.282+0000 7f1b16ffd640 1 -- 172.21.15.16:0/2043494449 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b1819d4b0 con 0x7f1b181002f0 2024-09-12T02:02:31.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.282+0000 7f1b177fe640 1 --2- 172.21.15.16:0/2043494449 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b18106f50 0x7f1b1819cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:02:31.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.282+0000 7f1b0ffff640 1 --2- 172.21.15.16:0/2043494449 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1b18100ca0 0x7f1b18198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:02:31.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.282+0000 7f1b16ffd640 1 --2- 172.21.15.16:0/2043494449 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b181002f0 0x7f1b18198500 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f1b08009230 tx=0x7f1b08004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:02:31.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.283+0000 7f1b1c8e7640 1 -- 172.21.15.16:0/2043494449 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b0803f030 con 0x7f1b181002f0 2024-09-12T02:02:31.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.283+0000 7f1b1c8e7640 1 -- 172.21.15.16:0/2043494449 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b08004440 con 0x7f1b181002f0 2024-09-12T02:02:31.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.283+0000 7f1b1c8e7640 1 -- 172.21.15.16:0/2043494449 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b08035070 con 0x7f1b181002f0 2024-09-12T02:02:31.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.283+0000 7f1b1d8e9640 1 -- 172.21.15.16:0/2043494449 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b1819d740 con 0x7f1b181002f0 2024-09-12T02:02:31.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.283+0000 7f1b1d8e9640 1 -- 172.21.15.16:0/2043494449 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b1819dbd0 con 0x7f1b181002f0 2024-09-12T02:02:31.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.287+0000 7f1b1c8e7640 1 -- 172.21.15.16:0/2043494449 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f1b08042030 con 0x7f1b181002f0 2024-09-12T02:02:31.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.287+0000 7f1b1fcc2740 1 Processor -- start 2024-09-12T02:02:31.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.287+0000 7f1b1fcc2740 1 -- start start 2024-09-12T02:02:31.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.287+0000 7f1b1fcc2740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5559c9662b40 0x5559c9661090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:31.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.287+0000 7f1b1fcc2740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5559c96615d0 0x5559c9661a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:31.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.287+0000 7f1b1fcc2740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559c966c370 0x5559c966e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:31.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.287+0000 7f1b1fcc2740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5559c95c72f0 con 0x5559c966c370 2024-09-12T02:02:31.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.287+0000 7f1b1fcc2740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5559c956ecf0 con 0x5559c96615d0 2024-09-12T02:02:31.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.287+0000 7f1b1fcc2740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5559c95c32b0 con 0x5559c9662b40 2024-09-12T02:02:31.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.287+0000 7f1b0ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559c966c370 0x5559c966e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:31.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.287+0000 7f1b16ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5559c96615d0 0x5559c9661a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:31.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.287+0000 7f1b0ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559c966c370 0x5559c966e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37828/0 (socket says 172.21.15.16:37828) 2024-09-12T02:02:31.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.288+0000 7f1b0ffff640 1 -- 172.21.15.16:0/2945108440 learned_addr learned my addr 172.21.15.16:0/2945108440 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:02:31.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.288+0000 7f1b177fe640 1 --2- 172.21.15.16:0/2945108440 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5559c9662b40 0x5559c9661090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:31.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.288+0000 7f1b16ffd640 1 -- 172.21.15.16:0/2945108440 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5559c9662b40 msgr2=0x5559c9661090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:31.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.288+0000 7f1b16ffd640 1 --2- 172.21.15.16:0/2945108440 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5559c9662b40 0x5559c9661090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:31.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.288+0000 7f1b16ffd640 1 -- 172.21.15.16:0/2945108440 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559c966c370 msgr2=0x5559c966e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:31.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.288+0000 7f1b1c8e7640 1 --2- 172.21.15.16:0/2043494449 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f1b040a2c00 0x7f1b040a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:31.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.288+0000 7f1b16ffd640 1 --2- 172.21.15.16:0/2945108440 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559c966c370 0x5559c966e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:31.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.288+0000 7f1b16ffd640 1 -- 172.21.15.16:0/2945108440 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5559c95533a0 con 0x5559c96615d0 2024-09-12T02:02:31.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.288+0000 7f1b0ffff640 1 --2- 172.21.15.16:0/2945108440 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559c966c370 0x5559c966e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:02:31.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.288+0000 7f1b1c8e7640 1 -- 172.21.15.16:0/2043494449 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f1b080ef2c0 con 0x7f1b181002f0 2024-09-12T02:02:31.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.288+0000 7f1b16ffd640 1 --2- 172.21.15.16:0/2945108440 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5559c96615d0 0x5559c9661a10 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1b080ece10 tx=0x7f1b080b7c70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=46e22f993bbe5c88 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:02:31.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.289+0000 7f1b177fe640 1 --2- 172.21.15.16:0/2043494449 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f1b040a2c00 0x7f1b040a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:31.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.289+0000 7f1b0dffb640 1 -- 172.21.15.16:0/2945108440 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b08049030 con 0x5559c96615d0 2024-09-12T02:02:31.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.289+0000 7f1b0dffb640 1 -- 172.21.15.16:0/2945108440 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b080bc2a0 con 0x5559c96615d0 2024-09-12T02:02:31.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.289+0000 7f1b0dffb640 1 -- 172.21.15.16:0/2945108440 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b080bb970 con 0x5559c96615d0 2024-09-12T02:02:31.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.290+0000 7f1b1fcc2740 1 -- 172.21.15.16:0/2945108440 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5559c96615d0 msgr2=0x5559c9661a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:31.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.290+0000 7f1b177fe640 1 --2- 172.21.15.16:0/2043494449 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f1b040a2c00 0x7f1b040a50c0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f1b10005fd0 tx=0x7f1b10009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:02:31.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.290+0000 7f1b1fcc2740 1 --2- 172.21.15.16:0/2945108440 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5559c96615d0 0x5559c9661a10 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1b080ece10 tx=0x7f1b080b7c70 comp rx=0 tx=0).stop 2024-09-12T02:02:31.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.290+0000 7f1b1fcc2740 1 -- 172.21.15.16:0/2945108440 shutdown_connections 2024-09-12T02:02:31.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.290+0000 7f1b1fcc2740 1 --2- 172.21.15.16:0/2945108440 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559c966c370 0x5559c966e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:31.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.290+0000 7f1b1fcc2740 1 --2- 172.21.15.16:0/2945108440 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5559c96615d0 0x5559c9661a10 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:31.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.290+0000 7f1b1fcc2740 1 --2- 172.21.15.16:0/2945108440 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5559c9662b40 0x5559c9661090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:31.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.290+0000 7f1b1fcc2740 1 -- 172.21.15.16:0/2945108440 >> 172.21.15.16:0/2945108440 conn(0x5559c962a310 msgr2=0x5559c965dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:02:31.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.290+0000 7f1b1fcc2740 1 -- 172.21.15.16:0/2945108440 shutdown_connections 2024-09-12T02:02:31.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.290+0000 7f1b1fcc2740 1 -- 172.21.15.16:0/2945108440 wait complete. 2024-09-12T02:02:31.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.290+0000 7f1b1fcc2740 1 -- start start 2024-09-12T02:02:31.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.291+0000 7f1b1fcc2740 1 Processor -- start 2024-09-12T02:02:31.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.291+0000 7f1b1fcc2740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5559c96615d0 0x5559c966bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:31.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.291+0000 7f1b1fcc2740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559c9662b40 0x5559c9669060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:31.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.291+0000 7f1b1fcc2740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5559c966c370 0x5559c96697f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:31.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.291+0000 7f1b1fcc2740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5559c956ecf0 con 0x5559c9662b40 2024-09-12T02:02:31.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.291+0000 7f1b1fcc2740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5559c9582060 con 0x5559c96615d0 2024-09-12T02:02:31.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.291+0000 7f1b1fcc2740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5559c9673240 con 0x5559c966c370 2024-09-12T02:02:31.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.291+0000 7f1b1fcc2740 10 client.0 _ll_register_callbacks cb 0x5559c92d58c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:02:31.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.291+0000 7f1b16ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5559c96615d0 0x5559c966bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:31.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.291+0000 7f1b0ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559c9662b40 0x5559c9669060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:31.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.291+0000 7f1b177fe640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5559c966c370 0x5559c96697f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:31.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.291+0000 7f1b16ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5559c96615d0 0x5559c966bdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.16:49696/0 (socket says 172.21.15.16:49696) 2024-09-12T02:02:31.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.291+0000 7f1b16ffd640 1 -- 172.21.15.16:0/2516592610 learned_addr learned my addr 172.21.15.16:0/2516592610 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:02:31.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.292+0000 7f1b16ffd640 1 -- 172.21.15.16:0/2516592610 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5559c966c370 msgr2=0x5559c96697f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:31.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.292+0000 7f1b16ffd640 1 --2- 172.21.15.16:0/2516592610 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5559c966c370 0x5559c96697f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:31.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.292+0000 7f1b16ffd640 1 -- 172.21.15.16:0/2516592610 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559c9662b40 msgr2=0x5559c9669060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:31.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.292+0000 7f1b16ffd640 1 --2- 172.21.15.16:0/2516592610 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559c9662b40 0x5559c9669060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:31.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.292+0000 7f1b16ffd640 1 -- 172.21.15.16:0/2516592610 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5559c93383c0 con 0x5559c96615d0 2024-09-12T02:02:31.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.292+0000 7f1b0ffff640 1 --2- 172.21.15.16:0/2516592610 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559c9662b40 0x5559c9669060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:02:31.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.292+0000 7f1b177fe640 1 --2- 172.21.15.16:0/2516592610 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5559c966c370 0x5559c96697f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:02:31.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.292+0000 7f1b16ffd640 1 --2- 172.21.15.16:0/2516592610 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5559c96615d0 0x5559c966bdb0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1b080b8000 tx=0x7f1b08009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5fdd7cf50cf92866 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:02:31.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.292+0000 7f1af6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-12T02:02:31.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.292+0000 7f1af6ffd640 1 -- 172.21.15.16:0/2516592610 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b08049070 con 0x5559c96615d0 2024-09-12T02:02:31.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.293+0000 7f1b1fcc2740 1 -- 172.21.15.16:0/2516592610 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b080bc2a0 con 0x5559c96615d0 2024-09-12T02:02:31.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.293+0000 7f1af6ffd640 1 -- 172.21.15.16:0/2516592610 <== mon.2 v2:172.21.15.33:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f1b080b7970 con 0x5559c96615d0 2024-09-12T02:02:31.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.293+0000 7f1af6ffd640 1 -- 172.21.15.16:0/2516592610 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1b080bb500 con 0x5559c96615d0 2024-09-12T02:02:31.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.293+0000 7f1b1fcc2740 10 client.25157 fetch_fsmap learned FSMap version 25 2024-09-12T02:02:31.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.293+0000 7f1b1fcc2740 1 -- 172.21.15.16:0/2516592610 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5559c967ebb0 con 0x5559c96615d0 2024-09-12T02:02:31.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.293+0000 7f1af6ffd640 1 -- 172.21.15.16:0/2516592610 <== mon.2 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b080bc2a0 con 0x5559c96615d0 2024-09-12T02:02:31.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.293+0000 7f1af6ffd640 1 -- 172.21.15.16:0/2516592610 <== mon.2 v2:172.21.15.33:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f1b080ec610 con 0x5559c96615d0 2024-09-12T02:02:31.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.293+0000 7f1b1fcc2740 10 client.25157 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:02:31.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.293+0000 7f1b1fcc2740 10 client.25157 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:02:31.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.293+0000 7f1b1fcc2740 20 client.25157 populate_metadata read hostname 'smithi016' 2024-09-12T02:02:31.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.293+0000 7f1b1fcc2740 1 --2- 172.21.15.16:0/2516592610 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5559c967ff00 0x5559c96822f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:31.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.293+0000 7f1b1fcc2740 4 client.25157 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:02:31.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.293+0000 7f1b1fcc2740 1 -- 172.21.15.16:0/2516592610 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5559c95d4860 con 0x5559c967ff00 2024-09-12T02:02:31.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.293+0000 7f1b0ffff640 1 --2- 172.21.15.16:0/2516592610 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5559c967ff00 0x5559c96822f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:31.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.294+0000 7f1af6ffd640 10 client.25157 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:02:31.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.294+0000 7f1b0ffff640 1 --2- 172.21.15.16:0/2516592610 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5559c967ff00 0x5559c96822f0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=651b269a1e9d721 server_cookie=d800fcf2f1bce54c in_seq=0 out_seq=0 2024-09-12T02:02:31.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.295+0000 7f1af6ffd640 1 -- 172.21.15.16:0/2516592610 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5559c95d4860 con 0x5559c967ff00 2024-09-12T02:02:31.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.295+0000 7f1af6ffd640 10 client.25157 handle_command_reply: tid=0 2024-09-12T02:02:31.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.295+0000 7f1b1fcc2740 1 client.25157 shutdown 2024-09-12T02:02:31.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.295+0000 7f1b1fcc2740 1 -- 172.21.15.16:0/2516592610 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5559c96615d0 msgr2=0x5559c966bdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:31.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.295+0000 7f1b1fcc2740 1 --2- 172.21.15.16:0/2516592610 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5559c96615d0 0x5559c966bdb0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1b080b8000 tx=0x7f1b08009260 comp rx=0 tx=0).stop 2024-09-12T02:02:31.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.295+0000 7f1b1fcc2740 1 -- 172.21.15.16:0/2516592610 shutdown_connections 2024-09-12T02:02:31.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.295+0000 7f1b1fcc2740 1 --2- 172.21.15.16:0/2516592610 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5559c967ff00 0x5559c96822f0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:31.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.295+0000 7f1b1fcc2740 1 --2- 172.21.15.16:0/2516592610 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5559c966c370 0x5559c96697f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:31.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.295+0000 7f1b1fcc2740 1 --2- 172.21.15.16:0/2516592610 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559c9662b40 0x5559c9669060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:31.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.295+0000 7f1b1fcc2740 1 --2- 172.21.15.16:0/2516592610 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5559c96615d0 0x5559c966bdb0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:31.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.295+0000 7f1b1fcc2740 1 -- 172.21.15.16:0/2516592610 >> 172.21.15.16:0/2516592610 conn(0x5559c962a310 msgr2=0x5559c9663960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:02:31.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.295+0000 7f1af6ffd640 0 client.25157 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:02:31.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.295+0000 7f1b1fcc2740 1 -- 172.21.15.16:0/2516592610 shutdown_connections 2024-09-12T02:02:31.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.295+0000 7f1b1fcc2740 1 -- 172.21.15.16:0/2516592610 wait complete. 2024-09-12T02:02:31.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.296+0000 7f1b1fcc2740 20 client.25157 trim_cache size 0 max 16384 2024-09-12T02:02:31.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.309+0000 7f1b1fcc2740 1 Processor -- start 2024-09-12T02:02:31.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.309+0000 7f1b1fcc2740 1 -- start start 2024-09-12T02:02:31.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.309+0000 7f1b1fcc2740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5559c96615d0 0x5559c965de10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:31.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.309+0000 7f1b1fcc2740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559c9662b40 0x5559c9687ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:31.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.309+0000 7f1b1fcc2740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5559c966c370 0x5559c9687ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:31.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.309+0000 7f1b1fcc2740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1b181adb80 con 0x5559c9662b40 2024-09-12T02:02:31.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.309+0000 7f1b1fcc2740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1b181ab450 con 0x5559c96615d0 2024-09-12T02:02:31.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.309+0000 7f1b1fcc2740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1b181a9300 con 0x5559c966c370 2024-09-12T02:02:31.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.309+0000 7f1b0ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559c9662b40 0x5559c9687ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:31.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.309+0000 7f1b0ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559c9662b40 0x5559c9687ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37842/0 (socket says 172.21.15.16:37842) 2024-09-12T02:02:31.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.309+0000 7f1b0ffff640 1 -- 172.21.15.16:0/765870792 learned_addr learned my addr 172.21.15.16:0/765870792 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:02:31.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.309+0000 7f1b16ffd640 1 --2- 172.21.15.16:0/765870792 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5559c96615d0 0x5559c965de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:31.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.309+0000 7f1b177fe640 1 --2- 172.21.15.16:0/765870792 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5559c966c370 0x5559c9687ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:31.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.309+0000 7f1b0ffff640 1 -- 172.21.15.16:0/765870792 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5559c966c370 msgr2=0x5559c9687ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:31.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.310+0000 7f1b0ffff640 1 --2- 172.21.15.16:0/765870792 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5559c966c370 0x5559c9687ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:31.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.310+0000 7f1b0ffff640 1 -- 172.21.15.16:0/765870792 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5559c96615d0 msgr2=0x5559c965de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:31.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.310+0000 7f1b0ffff640 1 --2- 172.21.15.16:0/765870792 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5559c96615d0 0x5559c965de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:31.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.310+0000 7f1b0ffff640 1 -- 172.21.15.16:0/765870792 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5559c965e960 con 0x5559c9662b40 2024-09-12T02:02:31.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.310+0000 7f1b0ffff640 1 --2- 172.21.15.16:0/765870792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559c9662b40 0x5559c9687ab0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x5559c96688f0 tx=0x7f1b000054d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d70f1adce545356 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:02:31.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.310+0000 7f1b0d7fa640 1 -- 172.21.15.16:0/765870792 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b000056c0 con 0x5559c9662b40 2024-09-12T02:02:31.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.310+0000 7f1b0d7fa640 1 -- 172.21.15.16:0/765870792 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b00013be0 con 0x5559c9662b40 2024-09-12T02:02:31.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.310+0000 7f1b0d7fa640 1 -- 172.21.15.16:0/765870792 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b0001ec40 con 0x5559c9662b40 2024-09-12T02:02:31.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.311+0000 7f1b1fcc2740 1 -- 172.21.15.16:0/765870792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559c9662b40 msgr2=0x5559c9687ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:31.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.311+0000 7f1b1fcc2740 1 --2- 172.21.15.16:0/765870792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559c9662b40 0x5559c9687ab0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x5559c96688f0 tx=0x7f1b000054d0 comp rx=0 tx=0).stop 2024-09-12T02:02:31.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.311+0000 7f1b1fcc2740 1 -- 172.21.15.16:0/765870792 shutdown_connections 2024-09-12T02:02:31.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.311+0000 7f1b1fcc2740 1 --2- 172.21.15.16:0/765870792 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5559c966c370 0x5559c9687ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:31.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.311+0000 7f1b1fcc2740 1 --2- 172.21.15.16:0/765870792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559c9662b40 0x5559c9687ab0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:31.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.311+0000 7f1b1fcc2740 1 --2- 172.21.15.16:0/765870792 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5559c96615d0 0x5559c965de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:31.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.311+0000 7f1b1fcc2740 1 -- 172.21.15.16:0/765870792 >> 172.21.15.16:0/765870792 conn(0x5559c962a310 msgr2=0x5559c965d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:02:31.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.311+0000 7f1b1fcc2740 1 -- 172.21.15.16:0/765870792 shutdown_connections 2024-09-12T02:02:31.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.311+0000 7f1b1fcc2740 1 -- 172.21.15.16:0/765870792 wait complete. 2024-09-12T02:02:31.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.312+0000 7f1b1fcc2740 1 -- start start 2024-09-12T02:02:31.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.312+0000 7f1b1fcc2740 1 Processor -- start 2024-09-12T02:02:31.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.312+0000 7f1b1fcc2740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5559c96615d0 0x5559c9673a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:31.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.312+0000 7f1b1fcc2740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5559c9662b40 0x5559c9673f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:31.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.312+0000 7f1b1fcc2740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559c966c370 0x5559c96e8a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:31.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.312+0000 7f1b1fcc2740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1b181a9950 con 0x5559c966c370 2024-09-12T02:02:31.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.312+0000 7f1b1fcc2740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1b181aad80 con 0x5559c9662b40 2024-09-12T02:02:31.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.312+0000 7f1b1fcc2740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1b181a9fe0 con 0x5559c96615d0 2024-09-12T02:02:31.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.313+0000 7f1b1fcc2740 10 client.0 _ll_register_callbacks cb 0x7f1b00009650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:02:31.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.313+0000 7f1b177fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559c966c370 0x5559c96e8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:31.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.313+0000 7f1b0ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5559c9662b40 0x5559c9673f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:31.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.313+0000 7f1b0ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5559c9662b40 0x5559c9673f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.16:49714/0 (socket says 172.21.15.16:49714) 2024-09-12T02:02:31.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.313+0000 7f1b177fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559c966c370 0x5559c96e8a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37854/0 (socket says 172.21.15.16:37854) 2024-09-12T02:02:31.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.313+0000 7f1b0ffff640 1 -- 172.21.15.16:0/2180325861 learned_addr learned my addr 172.21.15.16:0/2180325861 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:02:31.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.313+0000 7f1b16ffd640 1 --2- 172.21.15.16:0/2180325861 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5559c96615d0 0x5559c9673a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:31.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.313+0000 7f1b177fe640 1 -- 172.21.15.16:0/2180325861 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5559c96615d0 msgr2=0x5559c9673a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:31.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.313+0000 7f1b177fe640 1 --2- 172.21.15.16:0/2180325861 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5559c96615d0 0x5559c9673a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:31.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.313+0000 7f1b177fe640 1 -- 172.21.15.16:0/2180325861 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5559c9662b40 msgr2=0x5559c9673f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:31.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.313+0000 7f1b177fe640 1 --2- 172.21.15.16:0/2180325861 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5559c9662b40 0x5559c9673f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:31.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.313+0000 7f1b177fe640 1 -- 172.21.15.16:0/2180325861 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5559c96e9080 con 0x5559c966c370 2024-09-12T02:02:31.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.313+0000 7f1b16ffd640 1 --2- 172.21.15.16:0/2180325861 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5559c96615d0 0x5559c9673a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:02:31.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.313+0000 7f1b177fe640 1 --2- 172.21.15.16:0/2180325861 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559c966c370 0x5559c96e8a70 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f1b10012660 tx=0x7f1b10004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e53466ce91c5206f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:02:31.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.313+0000 7f1af6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:02:31.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.314+0000 7f1af6ffd640 1 -- 172.21.15.16:0/2180325861 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b1002e060 con 0x5559c966c370 2024-09-12T02:02:31.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.314+0000 7f1af6ffd640 1 -- 172.21.15.16:0/2180325861 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f1b10004720 con 0x5559c966c370 2024-09-12T02:02:31.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.314+0000 7f1b1fcc2740 1 -- 172.21.15.16:0/2180325861 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b00013be0 con 0x5559c966c370 2024-09-12T02:02:31.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.314+0000 7f1af6ffd640 1 -- 172.21.15.16:0/2180325861 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1b10029430 con 0x5559c966c370 2024-09-12T02:02:31.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.314+0000 7f1b1fcc2740 10 client.15486 fetch_fsmap learned FSMap version 25 2024-09-12T02:02:31.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.314+0000 7f1b1fcc2740 1 -- 172.21.15.16:0/2180325861 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5559c96ea3d0 con 0x5559c966c370 2024-09-12T02:02:31.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.314+0000 7f1af6ffd640 1 -- 172.21.15.16:0/2180325861 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b00013be0 con 0x5559c966c370 2024-09-12T02:02:31.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.315+0000 7f1af6ffd640 1 -- 172.21.15.16:0/2180325861 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f1b10033070 con 0x5559c966c370 2024-09-12T02:02:31.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.315+0000 7f1b1fcc2740 10 client.15486 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:02:31.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.315+0000 7f1b1fcc2740 10 client.15486 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:02:31.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.315+0000 7f1b1fcc2740 20 client.15486 populate_metadata read hostname 'smithi016' 2024-09-12T02:02:31.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.315+0000 7f1b1fcc2740 1 --2- 172.21.15.16:0/2180325861 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5559c96eabb0 0x5559c96ecfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:31.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.315+0000 7f1b1fcc2740 4 client.15486 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T02:02:31.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.315+0000 7f1b1fcc2740 1 -- 172.21.15.16:0/2180325861 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5559c95ced70 con 0x5559c96eabb0 2024-09-12T02:02:31.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.315+0000 7f1b16ffd640 1 --2- 172.21.15.16:0/2180325861 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5559c96eabb0 0x5559c96ecfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:31.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.316+0000 7f1af6ffd640 10 client.15486 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:02:31.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.316+0000 7f1b16ffd640 1 --2- 172.21.15.16:0/2180325861 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5559c96eabb0 0x5559c96ecfd0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=963ef9ea2819a2f2 server_cookie=bb4bc106e7af2e55 in_seq=0 out_seq=0 2024-09-12T02:02:31.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.316+0000 7f1af6ffd640 1 -- 172.21.15.16:0/2180325861 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5559c95ced70 con 0x5559c96eabb0 2024-09-12T02:02:31.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.316+0000 7f1af6ffd640 10 client.15486 handle_command_reply: tid=0 2024-09-12T02:02:31.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.316+0000 7f1b1fcc2740 1 client.15486 shutdown 2024-09-12T02:02:31.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.316+0000 7f1b1fcc2740 1 -- 172.21.15.16:0/2180325861 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559c966c370 msgr2=0x5559c96e8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:31.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.316+0000 7f1b1fcc2740 1 --2- 172.21.15.16:0/2180325861 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559c966c370 0x5559c96e8a70 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f1b10012660 tx=0x7f1b10004ab0 comp rx=0 tx=0).stop 2024-09-12T02:02:31.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.316+0000 7f1b1fcc2740 1 -- 172.21.15.16:0/2180325861 shutdown_connections 2024-09-12T02:02:31.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.316+0000 7f1b1fcc2740 1 --2- 172.21.15.16:0/2180325861 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5559c96eabb0 0x5559c96ecfd0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:31.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.316+0000 7f1b1fcc2740 1 --2- 172.21.15.16:0/2180325861 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559c966c370 0x5559c96e8a70 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:31.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.316+0000 7f1b1fcc2740 1 --2- 172.21.15.16:0/2180325861 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5559c9662b40 0x5559c9673f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:31.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.316+0000 7f1b1fcc2740 1 --2- 172.21.15.16:0/2180325861 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5559c96615d0 0x5559c9673a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:31.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.316+0000 7f1af6ffd640 0 client.15486 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:02:31.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.316+0000 7f1b1fcc2740 1 -- 172.21.15.16:0/2180325861 >> 172.21.15.16:0/2180325861 conn(0x5559c962a310 msgr2=0x5559c965dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:02:31.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.317+0000 7f1b1fcc2740 1 -- 172.21.15.16:0/2180325861 shutdown_connections 2024-09-12T02:02:31.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.317+0000 7f1b1fcc2740 1 -- 172.21.15.16:0/2180325861 wait complete. 2024-09-12T02:02:31.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.317+0000 7f1b1fcc2740 20 client.15486 trim_cache size 0 max 16384 2024-09-12T02:02:31.320 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T02:02:31.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.317+0000 7f1b1d8e9640 1 -- 172.21.15.16:0/2043494449 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f1b040a2c00 msgr2=0x7f1b040a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:31.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.317+0000 7f1b1d8e9640 1 --2- 172.21.15.16:0/2043494449 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f1b040a2c00 0x7f1b040a50c0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f1b10005fd0 tx=0x7f1b10009500 comp rx=0 tx=0).stop 2024-09-12T02:02:31.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.317+0000 7f1b1d8e9640 1 -- 172.21.15.16:0/2043494449 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b181002f0 msgr2=0x7f1b18198500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:31.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.317+0000 7f1b1d8e9640 1 --2- 172.21.15.16:0/2043494449 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b181002f0 0x7f1b18198500 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f1b08009230 tx=0x7f1b08004040 comp rx=0 tx=0).stop 2024-09-12T02:02:31.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.318+0000 7f1b1d8e9640 1 -- 172.21.15.16:0/2043494449 shutdown_connections 2024-09-12T02:02:31.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.318+0000 7f1b1d8e9640 1 --2- 172.21.15.16:0/2043494449 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f1b040a2c00 0x7f1b040a50c0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:31.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.318+0000 7f1b1d8e9640 1 --2- 172.21.15.16:0/2043494449 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b18106f50 0x7f1b1819cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:31.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.318+0000 7f1b1d8e9640 1 --2- 172.21.15.16:0/2043494449 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1b18100ca0 0x7f1b18198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:31.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.318+0000 7f1b1d8e9640 1 --2- 172.21.15.16:0/2043494449 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b181002f0 0x7f1b18198500 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:31.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.318+0000 7f1b1d8e9640 1 -- 172.21.15.16:0/2043494449 >> 172.21.15.16:0/2043494449 conn(0x7f1b18077210 msgr2=0x7f1b181051d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:02:31.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.318+0000 7f1b1d8e9640 1 -- 172.21.15.16:0/2043494449 shutdown_connections 2024-09-12T02:02:31.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:31.318+0000 7f1b1d8e9640 1 -- 172.21.15.16:0/2043494449 wait complete. 2024-09-12T02:02:31.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:02:31 smithi016 ceph-mon[30947]: pgmap v353: 97 pgs: 97 active+clean; 666 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 938 B/s wr, 8 op/s 2024-09-12T02:02:31.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:02:31 smithi033 ceph-mon[35332]: pgmap v353: 97 pgs: 97 active+clean; 666 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 938 B/s wr, 8 op/s 2024-09-12T02:02:31.897 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:02:31 smithi184 ceph-mon[35593]: pgmap v353: 97 pgs: 97 active+clean; 666 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 938 B/s wr, 8 op/s 2024-09-12T02:02:32.970 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-12T02:02:32.971 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-12T02:02:33.310 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:02:33.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:02:33 smithi033 ceph-mon[35332]: pgmap v354: 97 pgs: 97 active+clean; 666 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 938 B/s wr, 6 op/s 2024-09-12T02:02:33.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:02:33 smithi016 ceph-mon[30947]: pgmap v354: 97 pgs: 97 active+clean; 666 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 938 B/s wr, 6 op/s 2024-09-12T02:02:33.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:02:33 smithi184 ceph-mon[35593]: pgmap v354: 97 pgs: 97 active+clean; 666 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 938 B/s wr, 6 op/s 2024-09-12T02:02:33.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.885+0000 7fbccef1d640 1 -- 172.21.15.16:0/2802417397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbcc8106f00 msgr2=0x7fbcc810ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:33.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.885+0000 7fbccef1d640 1 --2- 172.21.15.16:0/2802417397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbcc8106f00 0x7fbcc810ea80 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fbcc0009ca0 tx=0x7fbcc002a8c0 comp rx=0 tx=0).stop 2024-09-12T02:02:33.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.885+0000 7fbccef1d640 1 -- 172.21.15.16:0/2802417397 shutdown_connections 2024-09-12T02:02:33.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.885+0000 7fbccef1d640 1 --2- 172.21.15.16:0/2802417397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbcc8106f00 0x7fbcc810ea80 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:33.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.885+0000 7fbccef1d640 1 --2- 172.21.15.16:0/2802417397 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbcc8102780 0x7fbcc81067d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:33.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.885+0000 7fbccef1d640 1 --2- 172.21.15.16:0/2802417397 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbcc8101dd0 0x7fbcc81021b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:33.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.885+0000 7fbccef1d640 1 -- 172.21.15.16:0/2802417397 >> 172.21.15.16:0/2802417397 conn(0x7fbcc80fd890 msgr2=0x7fbcc80ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:02:33.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.885+0000 7fbccef1d640 1 -- 172.21.15.16:0/2802417397 shutdown_connections 2024-09-12T02:02:33.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.885+0000 7fbccef1d640 1 -- 172.21.15.16:0/2802417397 wait complete. 2024-09-12T02:02:33.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.886+0000 7fbccef1d640 1 Processor -- start 2024-09-12T02:02:33.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.886+0000 7fbccef1d640 1 -- start start 2024-09-12T02:02:33.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.886+0000 7fbccef1d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbcc8101dd0 0x7fbcc8198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:33.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.886+0000 7fbccef1d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbcc8102780 0x7fbcc8198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:33.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.887+0000 7fbccef1d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbcc8106f00 0x7fbcc819ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:33.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.887+0000 7fbccef1d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbcc8111f60 con 0x7fbcc8101dd0 2024-09-12T02:02:33.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.887+0000 7fbccef1d640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbcc8111de0 con 0x7fbcc8102780 2024-09-12T02:02:33.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.887+0000 7fbccef1d640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fbcc81120e0 con 0x7fbcc8106f00 2024-09-12T02:02:33.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.887+0000 7fbccccb9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbcc8101dd0 0x7fbcc8198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:33.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.887+0000 7fbccccb9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbcc8101dd0 0x7fbcc8198400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37876/0 (socket says 172.21.15.16:37876) 2024-09-12T02:02:33.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.887+0000 7fbccccb9640 1 -- 172.21.15.16:0/1972110206 learned_addr learned my addr 172.21.15.16:0/1972110206 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:02:33.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.887+0000 7fbcc7fff640 1 --2- 172.21.15.16:0/1972110206 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbcc8102780 0x7fbcc8198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:33.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.887+0000 7fbccd4ba640 1 --2- 172.21.15.16:0/1972110206 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbcc8106f00 0x7fbcc819ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:33.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.887+0000 7fbccccb9640 1 -- 172.21.15.16:0/1972110206 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbcc8106f00 msgr2=0x7fbcc819ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:33.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.887+0000 7fbccccb9640 1 --2- 172.21.15.16:0/1972110206 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbcc8106f00 0x7fbcc819ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:33.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.887+0000 7fbccccb9640 1 -- 172.21.15.16:0/1972110206 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbcc8102780 msgr2=0x7fbcc8198940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:33.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.887+0000 7fbccccb9640 1 --2- 172.21.15.16:0/1972110206 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbcc8102780 0x7fbcc8198940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:33.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.887+0000 7fbccccb9640 1 -- 172.21.15.16:0/1972110206 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcc819d3b0 con 0x7fbcc8101dd0 2024-09-12T02:02:33.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.887+0000 7fbcc7fff640 1 --2- 172.21.15.16:0/1972110206 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbcc8102780 0x7fbcc8198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:02:33.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.887+0000 7fbccccb9640 1 --2- 172.21.15.16:0/1972110206 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbcc8101dd0 0x7fbcc8198400 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc00e9f0 tx=0x7fbcbc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:02:33.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.888+0000 7fbcc5ffb640 1 -- 172.21.15.16:0/1972110206 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcbc00cd80 con 0x7fbcc8101dd0 2024-09-12T02:02:33.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.888+0000 7fbccef1d640 1 -- 172.21.15.16:0/1972110206 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcc819d6a0 con 0x7fbcc8101dd0 2024-09-12T02:02:33.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.888+0000 7fbcc5ffb640 1 -- 172.21.15.16:0/1972110206 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbcbc005500 con 0x7fbcc8101dd0 2024-09-12T02:02:33.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.888+0000 7fbcc5ffb640 1 -- 172.21.15.16:0/1972110206 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcbc010690 con 0x7fbcc8101dd0 2024-09-12T02:02:33.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.888+0000 7fbccef1d640 1 -- 172.21.15.16:0/1972110206 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcc8074420 con 0x7fbcc8101dd0 2024-09-12T02:02:33.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.891+0000 7fbcd12f6740 1 Processor -- start 2024-09-12T02:02:33.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.891+0000 7fbcd12f6740 1 -- start start 2024-09-12T02:02:33.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.891+0000 7fbcd12f6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56334a9b6b60 0x56334a9b6f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:33.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.891+0000 7fbcd12f6740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56334a9b7480 0x56334a9c1800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:33.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.891+0000 7fbcd12f6740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56334a9c1d40 0x56334a9c4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:33.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.891+0000 7fbccd4ba640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56334a9b6b60 0x56334a9b6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:33.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.891+0000 7fbcd12f6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56334a91c280 con 0x56334a9b6b60 2024-09-12T02:02:33.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.891+0000 7fbcd12f6740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56334a8c3b10 con 0x56334a9c1d40 2024-09-12T02:02:33.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.891+0000 7fbcd12f6740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x56334a918240 con 0x56334a9b7480 2024-09-12T02:02:33.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.891+0000 7fbccd4ba640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56334a9b6b60 0x56334a9b6f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37892/0 (socket says 172.21.15.16:37892) 2024-09-12T02:02:33.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.891+0000 7fbccd4ba640 1 -- 172.21.15.16:0/407038263 learned_addr learned my addr 172.21.15.16:0/407038263 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:02:33.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.891+0000 7fbcc5ffb640 1 -- 172.21.15.16:0/1972110206 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fbcbc0046c0 con 0x7fbcc8101dd0 2024-09-12T02:02:33.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.892+0000 7fbcc7fff640 1 --2- 172.21.15.16:0/407038263 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56334a9c1d40 0x56334a9c4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:33.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.892+0000 7fbccccb9640 1 --2- 172.21.15.16:0/407038263 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56334a9b7480 0x56334a9c1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:33.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.892+0000 7fbccd4ba640 1 -- 172.21.15.16:0/407038263 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56334a9b7480 msgr2=0x56334a9c1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:33.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.892+0000 7fbccd4ba640 1 --2- 172.21.15.16:0/407038263 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56334a9b7480 0x56334a9c1800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:33.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.892+0000 7fbccd4ba640 1 -- 172.21.15.16:0/407038263 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56334a9c1d40 msgr2=0x56334a9c4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:33.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.892+0000 7fbccd4ba640 1 --2- 172.21.15.16:0/407038263 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56334a9c1d40 0x56334a9c4130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:33.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.892+0000 7fbccd4ba640 1 -- 172.21.15.16:0/407038263 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56334a8d01f0 con 0x56334a9b6b60 2024-09-12T02:02:33.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.892+0000 7fbccccb9640 1 --2- 172.21.15.16:0/407038263 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56334a9b7480 0x56334a9c1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:02:33.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.892+0000 7fbccd4ba640 1 --2- 172.21.15.16:0/407038263 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56334a9b6b60 0x56334a9b6f40 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fbcc0009ca0 tx=0x7fbcc0009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2f213a09a0792a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:02:33.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.893+0000 7fbcc5ffb640 1 --2- 172.21.15.16:0/1972110206 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fbcb80a2c00 0x7fbcb80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:33.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.893+0000 7fbcb2ffd640 1 -- 172.21.15.16:0/407038263 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcc0031db0 con 0x56334a9b6b60 2024-09-12T02:02:33.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.893+0000 7fbcb2ffd640 1 -- 172.21.15.16:0/407038263 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbcc0036040 con 0x56334a9b6b60 2024-09-12T02:02:33.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.893+0000 7fbcb2ffd640 1 -- 172.21.15.16:0/407038263 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcc00043c0 con 0x56334a9b6b60 2024-09-12T02:02:33.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.893+0000 7fbccccb9640 1 --2- 172.21.15.16:0/1972110206 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fbcb80a2c00 0x7fbcb80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:33.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.893+0000 7fbcc5ffb640 1 -- 172.21.15.16:0/1972110206 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fbcbc0cf380 con 0x7fbcc8101dd0 2024-09-12T02:02:33.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.894+0000 7fbcd12f6740 1 -- 172.21.15.16:0/407038263 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56334a9b6b60 msgr2=0x56334a9b6f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:33.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.894+0000 7fbccccb9640 1 --2- 172.21.15.16:0/1972110206 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fbcb80a2c00 0x7fbcb80a50c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc00c6a0 tx=0x7fbcbc004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:02:33.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.894+0000 7fbcd12f6740 1 --2- 172.21.15.16:0/407038263 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56334a9b6b60 0x56334a9b6f40 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fbcc0009ca0 tx=0x7fbcc0009800 comp rx=0 tx=0).stop 2024-09-12T02:02:33.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.894+0000 7fbcd12f6740 1 -- 172.21.15.16:0/407038263 shutdown_connections 2024-09-12T02:02:33.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.894+0000 7fbcd12f6740 1 --2- 172.21.15.16:0/407038263 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56334a9c1d40 0x56334a9c4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:33.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.894+0000 7fbcd12f6740 1 --2- 172.21.15.16:0/407038263 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56334a9b7480 0x56334a9c1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:33.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.894+0000 7fbcd12f6740 1 --2- 172.21.15.16:0/407038263 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56334a9b6b60 0x56334a9b6f40 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:33.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.894+0000 7fbcd12f6740 1 -- 172.21.15.16:0/407038263 >> 172.21.15.16:0/407038263 conn(0x56334a9b30f0 msgr2=0x56334a9b34d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:02:33.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.894+0000 7fbcd12f6740 1 -- 172.21.15.16:0/407038263 shutdown_connections 2024-09-12T02:02:33.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.894+0000 7fbcd12f6740 1 -- 172.21.15.16:0/407038263 wait complete. 2024-09-12T02:02:33.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.894+0000 7fbcd12f6740 1 -- start start 2024-09-12T02:02:33.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.895+0000 7fbcd12f6740 1 Processor -- start 2024-09-12T02:02:33.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.895+0000 7fbcd12f6740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56334a9b6b60 0x56334a9bec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:33.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.895+0000 7fbcd12f6740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56334a9b7480 0x56334a9bf150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:33.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.895+0000 7fbcd12f6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56334a9c1d40 0x56334a9bf690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:33.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.895+0000 7fbcd12f6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56334a9c4790 con 0x56334a9c1d40 2024-09-12T02:02:33.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.895+0000 7fbcd12f6740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56334a8d6f10 con 0x56334a9b6b60 2024-09-12T02:02:33.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.895+0000 7fbcd12f6740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x56334a9c4910 con 0x56334a9b7480 2024-09-12T02:02:33.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.895+0000 7fbcd12f6740 10 client.0 _ll_register_callbacks cb 0x56334a89fa40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:02:33.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.895+0000 7fbccd4ba640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56334a9c1d40 0x56334a9bf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:33.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.895+0000 7fbccd4ba640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56334a9c1d40 0x56334a9bf690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37908/0 (socket says 172.21.15.16:37908) 2024-09-12T02:02:33.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.895+0000 7fbccd4ba640 1 -- 172.21.15.16:0/1273470057 learned_addr learned my addr 172.21.15.16:0/1273470057 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:02:33.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.895+0000 7fbccd4ba640 1 --2- 172.21.15.16:0/1273470057 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56334a9b6b60 0x56334a9bec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:33.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.895+0000 7fbcc7fff640 1 --2- 172.21.15.16:0/1273470057 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56334a9b7480 0x56334a9bf150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:33.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.896+0000 7fbccd4ba640 1 -- 172.21.15.16:0/1273470057 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56334a9b7480 msgr2=0x56334a9bf150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:33.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.896+0000 7fbccd4ba640 1 --2- 172.21.15.16:0/1273470057 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56334a9b7480 0x56334a9bf150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:33.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.896+0000 7fbccd4ba640 1 -- 172.21.15.16:0/1273470057 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56334a9b6b60 msgr2=0x56334a9bec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:33.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.896+0000 7fbccd4ba640 1 --2- 172.21.15.16:0/1273470057 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56334a9b6b60 0x56334a9bec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:33.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.896+0000 7fbccd4ba640 1 -- 172.21.15.16:0/1273470057 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56334a68d6a0 con 0x56334a9c1d40 2024-09-12T02:02:33.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.896+0000 7fbccd4ba640 1 --2- 172.21.15.16:0/1273470057 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56334a9c1d40 0x56334a9bf690 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fbcc00319d0 tx=0x7fbcc0031ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ca8981dae94c8ba server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:02:33.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.896+0000 7fbca7fff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:02:33.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.896+0000 7fbca7fff640 1 -- 172.21.15.16:0/1273470057 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcc0003b80 con 0x56334a9c1d40 2024-09-12T02:02:33.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.896+0000 7fbca7fff640 1 -- 172.21.15.16:0/1273470057 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fbcc0003d20 con 0x56334a9c1d40 2024-09-12T02:02:33.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.896+0000 7fbcd12f6740 1 -- 172.21.15.16:0/1273470057 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbcc0036040 con 0x56334a9c1d40 2024-09-12T02:02:33.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.897+0000 7fbca7fff640 1 -- 172.21.15.16:0/1273470057 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fbcc0004b40 con 0x56334a9c1d40 2024-09-12T02:02:33.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.897+0000 7fbcd12f6740 10 client.15498 fetch_fsmap learned FSMap version 25 2024-09-12T02:02:33.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.897+0000 7fbcd12f6740 1 -- 172.21.15.16:0/1273470057 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56334a9c4790 con 0x56334a9c1d40 2024-09-12T02:02:33.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.897+0000 7fbca7fff640 1 -- 172.21.15.16:0/1273470057 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbcc0036040 con 0x56334a9c1d40 2024-09-12T02:02:33.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.897+0000 7fbca7fff640 1 -- 172.21.15.16:0/1273470057 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fbcc0050040 con 0x56334a9c1d40 2024-09-12T02:02:33.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.897+0000 7fbcd12f6740 10 client.15498 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:02:33.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.897+0000 7fbcd12f6740 10 client.15498 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:02:33.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.897+0000 7fbcd12f6740 20 client.15498 populate_metadata read hostname 'smithi016' 2024-09-12T02:02:33.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.897+0000 7fbcd12f6740 1 --2- 172.21.15.16:0/1273470057 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x56334a9d1460 0x56334a9d3880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:33.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.897+0000 7fbcd12f6740 4 client.15498 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:02:33.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.897+0000 7fbcd12f6740 1 -- 172.21.15.16:0/1273470057 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56334a9297f0 con 0x56334a9d1460 2024-09-12T02:02:33.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.898+0000 7fbcc7fff640 1 --2- 172.21.15.16:0/1273470057 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x56334a9d1460 0x56334a9d3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:33.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.898+0000 7fbca7fff640 10 client.15498 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:02:33.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.898+0000 7fbcc7fff640 1 --2- 172.21.15.16:0/1273470057 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x56334a9d1460 0x56334a9d3880 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bbaed4c2b76c1713 server_cookie=954c9a50db1e731a in_seq=0 out_seq=0 2024-09-12T02:02:33.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.899+0000 7fbca7fff640 1 -- 172.21.15.16:0/1273470057 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56334a9297f0 con 0x56334a9d1460 2024-09-12T02:02:33.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.899+0000 7fbca7fff640 10 client.15498 handle_command_reply: tid=0 2024-09-12T02:02:33.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.899+0000 7fbcd12f6740 1 client.15498 shutdown 2024-09-12T02:02:33.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.899+0000 7fbcd12f6740 1 -- 172.21.15.16:0/1273470057 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56334a9c1d40 msgr2=0x56334a9bf690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:33.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.899+0000 7fbcd12f6740 1 --2- 172.21.15.16:0/1273470057 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56334a9c1d40 0x56334a9bf690 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fbcc00319d0 tx=0x7fbcc0031ad0 comp rx=0 tx=0).stop 2024-09-12T02:02:33.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.899+0000 7fbcd12f6740 1 -- 172.21.15.16:0/1273470057 shutdown_connections 2024-09-12T02:02:33.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.899+0000 7fbcd12f6740 1 --2- 172.21.15.16:0/1273470057 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x56334a9d1460 0x56334a9d3880 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:33.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.899+0000 7fbcd12f6740 1 --2- 172.21.15.16:0/1273470057 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56334a9c1d40 0x56334a9bf690 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:33.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.899+0000 7fbcd12f6740 1 --2- 172.21.15.16:0/1273470057 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56334a9b7480 0x56334a9bf150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:33.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.899+0000 7fbcd12f6740 1 --2- 172.21.15.16:0/1273470057 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56334a9b6b60 0x56334a9bec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:33.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.899+0000 7fbcd12f6740 1 -- 172.21.15.16:0/1273470057 >> 172.21.15.16:0/1273470057 conn(0x56334a9b30f0 msgr2=0x56334a9b2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:02:33.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.899+0000 7fbca7fff640 0 client.15498 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:02:33.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.899+0000 7fbcd12f6740 1 -- 172.21.15.16:0/1273470057 shutdown_connections 2024-09-12T02:02:33.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.899+0000 7fbcd12f6740 1 -- 172.21.15.16:0/1273470057 wait complete. 2024-09-12T02:02:33.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.899+0000 7fbcd12f6740 20 client.15498 trim_cache size 0 max 16384 2024-09-12T02:02:33.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.934+0000 7fbcd12f6740 1 Processor -- start 2024-09-12T02:02:33.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.934+0000 7fbcd12f6740 1 -- start start 2024-09-12T02:02:33.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.934+0000 7fbcd12f6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56334a9b6b60 0x56334a9d9130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:33.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.934+0000 7fbcd12f6740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56334a9b7480 0x56334a9d9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:33.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.934+0000 7fbcd12f6740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56334a9c1d40 0x56334a9b1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:33.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.934+0000 7fbcd12f6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbcc81ad990 con 0x56334a9b6b60 2024-09-12T02:02:33.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.934+0000 7fbcd12f6740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbcc81ab260 con 0x56334a9b7480 2024-09-12T02:02:33.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.934+0000 7fbcd12f6740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fbcc81a90b0 con 0x56334a9c1d40 2024-09-12T02:02:33.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.935+0000 7fbccd4ba640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56334a9b6b60 0x56334a9d9130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:33.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.935+0000 7fbccd4ba640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56334a9b6b60 0x56334a9d9130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37918/0 (socket says 172.21.15.16:37918) 2024-09-12T02:02:33.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.935+0000 7fbccd4ba640 1 -- 172.21.15.16:0/2607048775 learned_addr learned my addr 172.21.15.16:0/2607048775 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:02:33.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.935+0000 7fbcc7fff640 1 --2- 172.21.15.16:0/2607048775 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56334a9b7480 0x56334a9d9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:33.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.935+0000 7fbccd4ba640 1 -- 172.21.15.16:0/2607048775 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56334a9c1d40 msgr2=0x56334a9b1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:33.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.935+0000 7fbccd4ba640 1 --2- 172.21.15.16:0/2607048775 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56334a9c1d40 0x56334a9b1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:33.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.935+0000 7fbccd4ba640 1 -- 172.21.15.16:0/2607048775 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56334a9b7480 msgr2=0x56334a9d9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:33.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.935+0000 7fbccd4ba640 1 --2- 172.21.15.16:0/2607048775 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56334a9b7480 0x56334a9d9670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:33.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.935+0000 7fbccd4ba640 1 -- 172.21.15.16:0/2607048775 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56334a9d3e60 con 0x56334a9b6b60 2024-09-12T02:02:33.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.935+0000 7fbccd4ba640 1 --2- 172.21.15.16:0/2607048775 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56334a9b6b60 0x56334a9d9130 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fbcc0055040 tx=0x7fbcc0009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b263ab5c9192ca5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:02:33.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.935+0000 7fbcb27fc640 1 -- 172.21.15.16:0/2607048775 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcc003f030 con 0x56334a9b6b60 2024-09-12T02:02:33.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.935+0000 7fbcb27fc640 1 -- 172.21.15.16:0/2607048775 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbcc0036070 con 0x56334a9b6b60 2024-09-12T02:02:33.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.936+0000 7fbcb27fc640 1 -- 172.21.15.16:0/2607048775 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcc002fd00 con 0x56334a9b6b60 2024-09-12T02:02:33.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.936+0000 7fbcd12f6740 1 -- 172.21.15.16:0/2607048775 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56334a9b6b60 msgr2=0x56334a9d9130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:33.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.936+0000 7fbcd12f6740 1 --2- 172.21.15.16:0/2607048775 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56334a9b6b60 0x56334a9d9130 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fbcc0055040 tx=0x7fbcc0009b20 comp rx=0 tx=0).stop 2024-09-12T02:02:33.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.937+0000 7fbcd12f6740 1 -- 172.21.15.16:0/2607048775 shutdown_connections 2024-09-12T02:02:33.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.937+0000 7fbcd12f6740 1 --2- 172.21.15.16:0/2607048775 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56334a9c1d40 0x56334a9b1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:33.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.937+0000 7fbcd12f6740 1 --2- 172.21.15.16:0/2607048775 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56334a9b7480 0x56334a9d9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:33.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.937+0000 7fbcd12f6740 1 --2- 172.21.15.16:0/2607048775 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56334a9b6b60 0x56334a9d9130 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:33.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.937+0000 7fbcd12f6740 1 -- 172.21.15.16:0/2607048775 >> 172.21.15.16:0/2607048775 conn(0x56334a9b30f0 msgr2=0x56334a9c0b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:02:33.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.937+0000 7fbcd12f6740 1 -- 172.21.15.16:0/2607048775 shutdown_connections 2024-09-12T02:02:33.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.937+0000 7fbcd12f6740 1 -- 172.21.15.16:0/2607048775 wait complete. 2024-09-12T02:02:33.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.937+0000 7fbcd12f6740 1 -- start start 2024-09-12T02:02:33.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.937+0000 7fbcd12f6740 1 Processor -- start 2024-09-12T02:02:33.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.938+0000 7fbcd12f6740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56334a9b6b60 0x56334a9ca1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:33.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.938+0000 7fbcd12f6740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56334a9b7480 0x56334a9c5500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:33.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.938+0000 7fbcd12f6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56334a9c1d40 0x56334a9c5a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:33.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.938+0000 7fbcd12f6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbcc81a9720 con 0x56334a9c1d40 2024-09-12T02:02:33.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.938+0000 7fbcd12f6740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbcc81aab90 con 0x56334a9b7480 2024-09-12T02:02:33.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.938+0000 7fbcd12f6740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fbcc81a9df0 con 0x56334a9b6b60 2024-09-12T02:02:33.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.938+0000 7fbccd4ba640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56334a9b6b60 0x56334a9ca1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:33.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.938+0000 7fbccd4ba640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56334a9b6b60 0x56334a9ca1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:36430/0 (socket says 172.21.15.16:36430) 2024-09-12T02:02:33.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.938+0000 7fbcc7fff640 1 --2- 172.21.15.16:0/2267717965 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56334a9b7480 0x56334a9c5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:33.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.938+0000 7fbccd4ba640 1 -- 172.21.15.16:0/2267717965 learned_addr learned my addr 172.21.15.16:0/2267717965 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:02:33.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.938+0000 7fbccd4ba640 1 --2- 172.21.15.16:0/2267717965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56334a9c1d40 0x56334a9c5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:33.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.939+0000 7fbccd4ba640 1 -- 172.21.15.16:0/2267717965 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56334a9b6b60 msgr2=0x56334a9ca1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:33.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.939+0000 7fbccd4ba640 1 --2- 172.21.15.16:0/2267717965 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56334a9b6b60 0x56334a9ca1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:33.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.939+0000 7fbccd4ba640 1 -- 172.21.15.16:0/2267717965 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56334a9b7480 msgr2=0x56334a9c5500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:33.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.939+0000 7fbccd4ba640 1 --2- 172.21.15.16:0/2267717965 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56334a9b7480 0x56334a9c5500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:33.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.939+0000 7fbccd4ba640 1 -- 172.21.15.16:0/2267717965 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56334a9c6270 con 0x56334a9c1d40 2024-09-12T02:02:33.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.939+0000 7fbcd12f6740 10 client.0 _ll_register_callbacks cb 0x7fbcb400fe50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:02:33.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.939+0000 7fbcd12f6740 1 -- 172.21.15.16:0/2267717965 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbcc0036070 con 0x56334a9c1d40 2024-09-12T02:02:33.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.939+0000 7fbccd4ba640 1 --2- 172.21.15.16:0/2267717965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56334a9c1d40 0x56334a9c5a40 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fbcc002f750 tx=0x7fbcc002a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99a6b7923a237100 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:02:33.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.939+0000 7fbca7fff640 10 client.15504 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:02:33.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.939+0000 7fbca7fff640 1 -- 172.21.15.16:0/2267717965 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcc0060030 con 0x56334a9c1d40 2024-09-12T02:02:33.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.939+0000 7fbca7fff640 1 -- 172.21.15.16:0/2267717965 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fbcc0002030 con 0x56334a9c1d40 2024-09-12T02:02:33.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.939+0000 7fbca7fff640 1 -- 172.21.15.16:0/2267717965 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fbcc003f070 con 0x56334a9c1d40 2024-09-12T02:02:33.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.939+0000 7fbcd12f6740 10 client.15504 fetch_fsmap learned FSMap version 25 2024-09-12T02:02:33.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.939+0000 7fbcd12f6740 1 -- 172.21.15.16:0/2267717965 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({fsmap=25}) -- 0x56334aa44590 con 0x56334a9c1d40 2024-09-12T02:02:33.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.940+0000 7fbca7fff640 1 -- 172.21.15.16:0/2267717965 <== mon.0 v2:172.21.15.16:3300/0 4 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fbcc0043070 con 0x56334a9c1d40 2024-09-12T02:02:33.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.940+0000 7fbcd12f6740 10 client.15504 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:02:33.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.940+0000 7fbcd12f6740 10 client.15504 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:02:33.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.940+0000 7fbcd12f6740 20 client.15504 populate_metadata read hostname 'smithi016' 2024-09-12T02:02:33.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.940+0000 7fbcd12f6740 1 --2- 172.21.15.16:0/2267717965 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x56334aa44980 0x56334aa46da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:33.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.940+0000 7fbcd12f6740 4 client.15504 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-12T02:02:33.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.940+0000 7fbcd12f6740 1 -- 172.21.15.16:0/2267717965 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56334a923d00 con 0x56334aa44980 2024-09-12T02:02:33.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.940+0000 7fbcc7fff640 1 --2- 172.21.15.16:0/2267717965 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x56334aa44980 0x56334aa46da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:33.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.941+0000 7fbca7fff640 10 client.15504 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:02:33.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.941+0000 7fbcc7fff640 1 --2- 172.21.15.16:0/2267717965 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x56334aa44980 0x56334aa46da0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d63981a3d9f9d1a3 server_cookie=38d6c87fd6b0329a in_seq=0 out_seq=0 2024-09-12T02:02:33.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.942+0000 7fbca7fff640 1 -- 172.21.15.16:0/2267717965 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56334a923d00 con 0x56334aa44980 2024-09-12T02:02:33.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.942+0000 7fbca7fff640 10 client.15504 handle_command_reply: tid=0 2024-09-12T02:02:33.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.942+0000 7fbcd12f6740 1 client.15504 shutdown 2024-09-12T02:02:33.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.942+0000 7fbcd12f6740 1 -- 172.21.15.16:0/2267717965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56334a9c1d40 msgr2=0x56334a9c5a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:33.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.942+0000 7fbcd12f6740 1 --2- 172.21.15.16:0/2267717965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56334a9c1d40 0x56334a9c5a40 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fbcc002f750 tx=0x7fbcc002a8c0 comp rx=0 tx=0).stop 2024-09-12T02:02:33.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.942+0000 7fbcd12f6740 1 -- 172.21.15.16:0/2267717965 shutdown_connections 2024-09-12T02:02:33.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.942+0000 7fbcd12f6740 1 --2- 172.21.15.16:0/2267717965 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x56334aa44980 0x56334aa46da0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:33.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.942+0000 7fbca7fff640 0 client.15504 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:02:33.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.942+0000 7fbcd12f6740 1 --2- 172.21.15.16:0/2267717965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56334a9c1d40 0x56334a9c5a40 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:33.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.942+0000 7fbcd12f6740 1 --2- 172.21.15.16:0/2267717965 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56334a9b7480 0x56334a9c5500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:33.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.942+0000 7fbcd12f6740 1 --2- 172.21.15.16:0/2267717965 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56334a9b6b60 0x56334a9ca1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:33.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.942+0000 7fbcd12f6740 1 -- 172.21.15.16:0/2267717965 >> 172.21.15.16:0/2267717965 conn(0x56334a9b30f0 msgr2=0x56334a9bd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:02:33.945 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T02:02:33.945 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-09-12T02:02:33.945 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "c341ae6d-5d8d-40f8-840b-45294fef76f8", 2024-09-12T02:02:33.946 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-09-12T02:02:33.946 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T02:02:33.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.942+0000 7fbcd12f6740 1 -- 172.21.15.16:0/2267717965 shutdown_connections 2024-09-12T02:02:33.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.942+0000 7fbcd12f6740 1 -- 172.21.15.16:0/2267717965 wait complete. 2024-09-12T02:02:33.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.942+0000 7fbcd12f6740 20 client.15504 trim_cache size 0 max 16384 2024-09-12T02:02:33.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.943+0000 7fbccef1d640 1 -- 172.21.15.16:0/1972110206 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fbcb80a2c00 msgr2=0x7fbcb80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:33.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.943+0000 7fbccef1d640 1 --2- 172.21.15.16:0/1972110206 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fbcb80a2c00 0x7fbcb80a50c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc00c6a0 tx=0x7fbcbc004a40 comp rx=0 tx=0).stop 2024-09-12T02:02:33.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.943+0000 7fbccef1d640 1 -- 172.21.15.16:0/1972110206 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbcc8101dd0 msgr2=0x7fbcc8198400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:33.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.943+0000 7fbccef1d640 1 --2- 172.21.15.16:0/1972110206 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbcc8101dd0 0x7fbcc8198400 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc00e9f0 tx=0x7fbcbc00edf0 comp rx=0 tx=0).stop 2024-09-12T02:02:33.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.943+0000 7fbccef1d640 1 -- 172.21.15.16:0/1972110206 shutdown_connections 2024-09-12T02:02:33.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.943+0000 7fbccef1d640 1 --2- 172.21.15.16:0/1972110206 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fbcb80a2c00 0x7fbcb80a50c0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:33.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.943+0000 7fbccef1d640 1 --2- 172.21.15.16:0/1972110206 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbcc8106f00 0x7fbcc819ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:33.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.943+0000 7fbccef1d640 1 --2- 172.21.15.16:0/1972110206 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbcc8102780 0x7fbcc8198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:33.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.943+0000 7fbccef1d640 1 --2- 172.21.15.16:0/1972110206 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbcc8101dd0 0x7fbcc8198400 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:33.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.943+0000 7fbccef1d640 1 -- 172.21.15.16:0/1972110206 >> 172.21.15.16:0/1972110206 conn(0x7fbcc80fd890 msgr2=0x7fbcc80febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:02:33.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.944+0000 7fbccef1d640 1 -- 172.21.15.16:0/1972110206 shutdown_connections 2024-09-12T02:02:33.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:33.944+0000 7fbccef1d640 1 -- 172.21.15.16:0/1972110206 wait complete. 2024-09-12T02:02:34.651 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:02:34 smithi016 ceph-mon[30947]: scrub queued for path: / 2024-09-12T02:02:34.651 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:02:34 smithi016 ceph-mon[30947]: scrub summary: idle+waiting paths [/] 2024-09-12T02:02:34.651 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:02:34 smithi016 ceph-mon[30947]: scrub summary: active paths [/] 2024-09-12T02:02:34.651 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:02:34 smithi016 ceph-mon[30947]: scrub summary: idle+waiting paths [/] 2024-09-12T02:02:34.652 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T02:02:34.668 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:02:34 smithi033 ceph-mon[35332]: scrub queued for path: / 2024-09-12T02:02:34.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:02:34 smithi033 ceph-mon[35332]: scrub summary: idle+waiting paths [/] 2024-09-12T02:02:34.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:02:34 smithi033 ceph-mon[35332]: scrub summary: active paths [/] 2024-09-12T02:02:34.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:02:34 smithi033 ceph-mon[35332]: scrub summary: idle+waiting paths [/] 2024-09-12T02:02:34.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:02:34 smithi184 ceph-mon[35593]: scrub queued for path: / 2024-09-12T02:02:34.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:02:34 smithi184 ceph-mon[35593]: scrub summary: idle+waiting paths [/] 2024-09-12T02:02:34.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:02:34 smithi184 ceph-mon[35593]: scrub summary: active paths [/] 2024-09-12T02:02:34.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:02:34 smithi184 ceph-mon[35593]: scrub summary: idle+waiting paths [/] 2024-09-12T02:02:34.936 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:02:35.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.445+0000 7fe16d427640 1 -- 172.21.15.16:0/56818111 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe168101430 msgr2=0x7fe16810d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:35.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.445+0000 7fe16d427640 1 --2- 172.21.15.16:0/56818111 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe168101430 0x7fe16810d960 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fe160009cd0 tx=0x7fe16002a910 comp rx=0 tx=0).stop 2024-09-12T02:02:35.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.445+0000 7fe16d427640 1 -- 172.21.15.16:0/56818111 shutdown_connections 2024-09-12T02:02:35.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.445+0000 7fe16d427640 1 --2- 172.21.15.16:0/56818111 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe168101430 0x7fe16810d960 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:35.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.445+0000 7fe16d427640 1 --2- 172.21.15.16:0/56818111 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe168100a90 0x7fe168100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:35.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.445+0000 7fe16d427640 1 --2- 172.21.15.16:0/56818111 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe168106a90 0x7fe168106e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:35.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.445+0000 7fe16d427640 1 -- 172.21.15.16:0/56818111 >> 172.21.15.16:0/56818111 conn(0x7fe1680fc7d0 msgr2=0x7fe1680febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:02:35.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.445+0000 7fe16d427640 1 -- 172.21.15.16:0/56818111 shutdown_connections 2024-09-12T02:02:35.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.445+0000 7fe16d427640 1 -- 172.21.15.16:0/56818111 wait complete. 2024-09-12T02:02:35.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.447+0000 7fe16d427640 1 Processor -- start 2024-09-12T02:02:35.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.447+0000 7fe16d427640 1 -- start start 2024-09-12T02:02:35.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.447+0000 7fe16d427640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe168100a90 0x7fe168197320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:35.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.447+0000 7fe16d427640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe168101430 0x7fe168197860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:35.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.448+0000 7fe16d427640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe168106a90 0x7fe16819bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:35.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.448+0000 7fe166ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe168100a90 0x7fe168197320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:35.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.448+0000 7fe166ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe168100a90 0x7fe168197320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.16:49778/0 (socket says 172.21.15.16:49778) 2024-09-12T02:02:35.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.448+0000 7fe1667fc640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe168101430 0x7fe168197860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:35.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.448+0000 7fe1667fc640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe168101430 0x7fe168197860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:36444/0 (socket says 172.21.15.16:36444) 2024-09-12T02:02:35.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.448+0000 7fe1677fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe168106a90 0x7fe16819bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:35.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.448+0000 7fe1677fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe168106a90 0x7fe16819bbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37936/0 (socket says 172.21.15.16:37936) 2024-09-12T02:02:35.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.448+0000 7fe166ffd640 1 -- 172.21.15.16:0/2675332932 learned_addr learned my addr 172.21.15.16:0/2675332932 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:02:35.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.448+0000 7fe16d427640 1 -- 172.21.15.16:0/2675332932 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe168110ee0 con 0x7fe168106a90 2024-09-12T02:02:35.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.448+0000 7fe16d427640 1 -- 172.21.15.16:0/2675332932 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe168110d60 con 0x7fe168100a90 2024-09-12T02:02:35.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.448+0000 7fe16d427640 1 -- 172.21.15.16:0/2675332932 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe168111060 con 0x7fe168101430 2024-09-12T02:02:35.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.449+0000 7fe166ffd640 1 -- 172.21.15.16:0/2675332932 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe168101430 msgr2=0x7fe168197860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:35.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.449+0000 7fe166ffd640 1 --2- 172.21.15.16:0/2675332932 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe168101430 0x7fe168197860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:35.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.449+0000 7fe166ffd640 1 -- 172.21.15.16:0/2675332932 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe168106a90 msgr2=0x7fe16819bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:35.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.449+0000 7fe166ffd640 1 --2- 172.21.15.16:0/2675332932 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe168106a90 0x7fe16819bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:35.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.449+0000 7fe166ffd640 1 -- 172.21.15.16:0/2675332932 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe16819c2d0 con 0x7fe168100a90 2024-09-12T02:02:35.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.449+0000 7fe1677fe640 1 --2- 172.21.15.16:0/2675332932 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe168106a90 0x7fe16819bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:02:35.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.449+0000 7fe166ffd640 1 --2- 172.21.15.16:0/2675332932 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe168100a90 0x7fe168197320 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fe15800e9f0 tx=0x7fe15800edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:02:35.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.450+0000 7fe14ffff640 1 -- 172.21.15.16:0/2675332932 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe15800cd80 con 0x7fe168100a90 2024-09-12T02:02:35.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.450+0000 7fe16d427640 1 -- 172.21.15.16:0/2675332932 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe16819c5c0 con 0x7fe168100a90 2024-09-12T02:02:35.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.450+0000 7fe14ffff640 1 -- 172.21.15.16:0/2675332932 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe158005500 con 0x7fe168100a90 2024-09-12T02:02:35.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.450+0000 7fe14ffff640 1 -- 172.21.15.16:0/2675332932 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe158010690 con 0x7fe168100a90 2024-09-12T02:02:35.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.450+0000 7fe16d427640 1 -- 172.21.15.16:0/2675332932 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe16806a790 con 0x7fe168100a90 2024-09-12T02:02:35.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.453+0000 7fe14ffff640 1 -- 172.21.15.16:0/2675332932 <== mon.2 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fe158004f60 con 0x7fe168100a90 2024-09-12T02:02:35.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.453+0000 7fe16f800740 1 Processor -- start 2024-09-12T02:02:35.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.454+0000 7fe16f800740 1 -- start start 2024-09-12T02:02:35.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.454+0000 7fe16f800740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ba3b67e360 0x55ba3b67e740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:35.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.454+0000 7fe16f800740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ba3b67ec80 0x55ba3b68d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:35.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.454+0000 7fe16f800740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ba3b68da90 0x55ba3b68fe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:35.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.454+0000 7fe16f800740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55ba3b5e42e0 con 0x55ba3b67ec80 2024-09-12T02:02:35.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.454+0000 7fe16f800740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55ba3b58bce0 con 0x55ba3b68da90 2024-09-12T02:02:35.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.454+0000 7fe16f800740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55ba3b5e02a0 con 0x55ba3b67e360 2024-09-12T02:02:35.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.454+0000 7fe166ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ba3b67ec80 0x55ba3b68d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:35.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.454+0000 7fe166ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ba3b67ec80 0x55ba3b68d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37946/0 (socket says 172.21.15.16:37946) 2024-09-12T02:02:35.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.454+0000 7fe166ffd640 1 -- 172.21.15.16:0/2873885610 learned_addr learned my addr 172.21.15.16:0/2873885610 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:02:35.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.455+0000 7fe1667fc640 1 --2- 172.21.15.16:0/2873885610 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ba3b68da90 0x55ba3b68fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:35.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.455+0000 7fe1677fe640 1 --2- 172.21.15.16:0/2873885610 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ba3b67e360 0x55ba3b67e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:35.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.455+0000 7fe14ffff640 1 --2- 172.21.15.16:0/2675332932 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fe1500a2b90 0x7fe1500a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:35.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.455+0000 7fe166ffd640 1 -- 172.21.15.16:0/2873885610 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ba3b67e360 msgr2=0x55ba3b67e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:35.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.455+0000 7fe166ffd640 1 --2- 172.21.15.16:0/2873885610 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ba3b67e360 0x55ba3b67e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:35.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.455+0000 7fe166ffd640 1 -- 172.21.15.16:0/2873885610 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ba3b68da90 msgr2=0x55ba3b68fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:35.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.455+0000 7fe166ffd640 1 --2- 172.21.15.16:0/2873885610 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ba3b68da90 0x55ba3b68fe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:35.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.455+0000 7fe166ffd640 1 -- 172.21.15.16:0/2873885610 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba3b570440 con 0x55ba3b67ec80 2024-09-12T02:02:35.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.455+0000 7fe1667fc640 1 --2- 172.21.15.16:0/2873885610 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ba3b68da90 0x55ba3b68fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:02:35.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.455+0000 7fe1677fe640 1 --2- 172.21.15.16:0/2873885610 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ba3b67e360 0x55ba3b67e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:02:35.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.455+0000 7fe14ffff640 1 -- 172.21.15.16:0/2675332932 <== mon.2 v2:172.21.15.33:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fe1580cf360 con 0x7fe168100a90 2024-09-12T02:02:35.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.456+0000 7fe1677fe640 1 --2- 172.21.15.16:0/2675332932 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fe1500a2b90 0x7fe1500a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:35.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.456+0000 7fe166ffd640 1 --2- 172.21.15.16:0/2873885610 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ba3b67ec80 0x55ba3b68d550 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fe1580952c0 tx=0x7fe158091140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8777185b19e5f1cd server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:02:35.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.456+0000 7fe14d7fa640 1 -- 172.21.15.16:0/2873885610 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe15809b7a0 con 0x55ba3b67ec80 2024-09-12T02:02:35.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.456+0000 7fe14d7fa640 1 -- 172.21.15.16:0/2873885610 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe15806ee50 con 0x55ba3b67ec80 2024-09-12T02:02:35.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.456+0000 7fe14d7fa640 1 -- 172.21.15.16:0/2873885610 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe15807bea0 con 0x55ba3b67ec80 2024-09-12T02:02:35.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.457+0000 7fe16f800740 1 -- 172.21.15.16:0/2873885610 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ba3b67ec80 msgr2=0x55ba3b68d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:35.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.457+0000 7fe16f800740 1 --2- 172.21.15.16:0/2873885610 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ba3b67ec80 0x55ba3b68d550 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fe1580952c0 tx=0x7fe158091140 comp rx=0 tx=0).stop 2024-09-12T02:02:35.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.457+0000 7fe1677fe640 1 --2- 172.21.15.16:0/2675332932 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fe1500a2b90 0x7fe1500a5050 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fe160004af0 tx=0x7fe16003f040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:02:35.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.457+0000 7fe16f800740 1 -- 172.21.15.16:0/2873885610 shutdown_connections 2024-09-12T02:02:35.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.457+0000 7fe16f800740 1 --2- 172.21.15.16:0/2873885610 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ba3b68da90 0x55ba3b68fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:35.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.457+0000 7fe16f800740 1 --2- 172.21.15.16:0/2873885610 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ba3b67ec80 0x55ba3b68d550 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:35.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.457+0000 7fe16f800740 1 --2- 172.21.15.16:0/2873885610 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ba3b67e360 0x55ba3b67e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:35.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.457+0000 7fe16f800740 1 -- 172.21.15.16:0/2873885610 >> 172.21.15.16:0/2873885610 conn(0x55ba3b67aa30 msgr2=0x55ba3b67ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:02:35.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.457+0000 7fe16f800740 1 -- 172.21.15.16:0/2873885610 shutdown_connections 2024-09-12T02:02:35.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.457+0000 7fe16f800740 1 -- 172.21.15.16:0/2873885610 wait complete. 2024-09-12T02:02:35.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.458+0000 7fe16f800740 1 -- start start 2024-09-12T02:02:35.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.458+0000 7fe16f800740 1 Processor -- start 2024-09-12T02:02:35.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.458+0000 7fe16f800740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ba3b67e360 0x55ba3b68d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:35.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.458+0000 7fe16f800740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ba3b67ec80 0x55ba3b68a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:35.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.458+0000 7fe16f800740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ba3b68da90 0x55ba3b68ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:35.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.459+0000 7fe1667fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ba3b67ec80 0x55ba3b68a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:35.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.459+0000 7fe1667fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ba3b67ec80 0x55ba3b68a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37950/0 (socket says 172.21.15.16:37950) 2024-09-12T02:02:35.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.459+0000 7fe1667fc640 1 -- 172.21.15.16:0/4212589347 learned_addr learned my addr 172.21.15.16:0/4212589347 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:02:35.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.459+0000 7fe166ffd640 1 --2- 172.21.15.16:0/4212589347 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ba3b67e360 0x55ba3b68d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:35.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.459+0000 7fe1677fe640 1 --2- 172.21.15.16:0/4212589347 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ba3b68da90 0x55ba3b68ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:35.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.459+0000 7fe16f800740 1 -- 172.21.15.16:0/4212589347 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55ba3b690600 con 0x55ba3b67ec80 2024-09-12T02:02:35.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.459+0000 7fe16f800740 1 -- 172.21.15.16:0/4212589347 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55ba3b59f050 con 0x55ba3b68da90 2024-09-12T02:02:35.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.459+0000 7fe16f800740 1 -- 172.21.15.16:0/4212589347 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55ba3b690780 con 0x55ba3b67e360 2024-09-12T02:02:35.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.459+0000 7fe16f800740 10 client.0 _ll_register_callbacks cb 0x55ba3b5df160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:02:35.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.459+0000 7fe1667fc640 1 -- 172.21.15.16:0/4212589347 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ba3b67e360 msgr2=0x55ba3b68d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:35.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.459+0000 7fe1667fc640 1 --2- 172.21.15.16:0/4212589347 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ba3b67e360 0x55ba3b68d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:35.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.459+0000 7fe1667fc640 1 -- 172.21.15.16:0/4212589347 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ba3b68da90 msgr2=0x55ba3b68ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:35.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.459+0000 7fe1667fc640 1 --2- 172.21.15.16:0/4212589347 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ba3b68da90 0x55ba3b68ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:35.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.459+0000 7fe1667fc640 1 -- 172.21.15.16:0/4212589347 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba3b3553c0 con 0x55ba3b67ec80 2024-09-12T02:02:35.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.459+0000 7fe1677fe640 1 --2- 172.21.15.16:0/4212589347 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ba3b68da90 0x55ba3b68ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:02:35.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.459+0000 7fe166ffd640 1 --2- 172.21.15.16:0/4212589347 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ba3b67e360 0x55ba3b68d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:02:35.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.460+0000 7fe1667fc640 1 --2- 172.21.15.16:0/4212589347 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ba3b67ec80 0x55ba3b68a730 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fe15c005bc0 tx=0x7fe15c00bf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8c255f35a2c4430 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:02:35.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.460+0000 7fe14aff5640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:02:35.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.460+0000 7fe14aff5640 1 -- 172.21.15.16:0/4212589347 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe15c01b440 con 0x55ba3b67ec80 2024-09-12T02:02:35.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.460+0000 7fe16f800740 1 -- 172.21.15.16:0/4212589347 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe15806ee50 con 0x55ba3b67ec80 2024-09-12T02:02:35.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.460+0000 7fe14aff5640 1 -- 172.21.15.16:0/4212589347 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fe15c011440 con 0x55ba3b67ec80 2024-09-12T02:02:35.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.460+0000 7fe14aff5640 1 -- 172.21.15.16:0/4212589347 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe15c011640 con 0x55ba3b67ec80 2024-09-12T02:02:35.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.460+0000 7fe16f800740 10 client.15516 fetch_fsmap learned FSMap version 25 2024-09-12T02:02:35.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.460+0000 7fe16f800740 1 -- 172.21.15.16:0/4212589347 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ba3b6a0530 con 0x55ba3b67ec80 2024-09-12T02:02:35.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.460+0000 7fe14aff5640 1 -- 172.21.15.16:0/4212589347 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe15806ee50 con 0x55ba3b67ec80 2024-09-12T02:02:35.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.461+0000 7fe14aff5640 1 -- 172.21.15.16:0/4212589347 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fe15c019020 con 0x55ba3b67ec80 2024-09-12T02:02:35.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.461+0000 7fe16f800740 10 client.15516 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:02:35.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.461+0000 7fe16f800740 10 client.15516 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:02:35.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.461+0000 7fe16f800740 20 client.15516 populate_metadata read hostname 'smithi016' 2024-09-12T02:02:35.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.461+0000 7fe16f800740 1 --2- 172.21.15.16:0/4212589347 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55ba3b6a1940 0x55ba3b6a3d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:35.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.462+0000 7fe16f800740 4 client.15516 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:02:35.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.462+0000 7fe16f800740 1 -- 172.21.15.16:0/4212589347 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ba3b5f1850 con 0x55ba3b6a1940 2024-09-12T02:02:35.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.462+0000 7fe166ffd640 1 --2- 172.21.15.16:0/4212589347 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55ba3b6a1940 0x55ba3b6a3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:35.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.462+0000 7fe166ffd640 1 --2- 172.21.15.16:0/4212589347 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55ba3b6a1940 0x55ba3b6a3d30 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f846a441ea99059 server_cookie=9bf0478908123419 in_seq=0 out_seq=0 2024-09-12T02:02:35.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.462+0000 7fe14aff5640 10 client.15516 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:02:35.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.463+0000 7fe14aff5640 1 -- 172.21.15.16:0/4212589347 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ba3b5f1850 con 0x55ba3b6a1940 2024-09-12T02:02:35.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.463+0000 7fe14aff5640 10 client.15516 handle_command_reply: tid=0 2024-09-12T02:02:35.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.463+0000 7fe16f800740 1 client.15516 shutdown 2024-09-12T02:02:35.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.463+0000 7fe16f800740 1 -- 172.21.15.16:0/4212589347 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ba3b67ec80 msgr2=0x55ba3b68a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:35.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.463+0000 7fe16f800740 1 --2- 172.21.15.16:0/4212589347 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ba3b67ec80 0x55ba3b68a730 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fe15c005bc0 tx=0x7fe15c00bf00 comp rx=0 tx=0).stop 2024-09-12T02:02:35.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.463+0000 7fe16f800740 1 -- 172.21.15.16:0/4212589347 shutdown_connections 2024-09-12T02:02:35.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.463+0000 7fe16f800740 1 --2- 172.21.15.16:0/4212589347 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55ba3b6a1940 0x55ba3b6a3d30 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:35.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.463+0000 7fe16f800740 1 --2- 172.21.15.16:0/4212589347 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ba3b68da90 0x55ba3b68ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:35.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.463+0000 7fe16f800740 1 --2- 172.21.15.16:0/4212589347 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ba3b67ec80 0x55ba3b68a730 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:35.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.463+0000 7fe16f800740 1 --2- 172.21.15.16:0/4212589347 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ba3b67e360 0x55ba3b68d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:35.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.463+0000 7fe14aff5640 0 client.15516 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:02:35.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.463+0000 7fe16f800740 1 -- 172.21.15.16:0/4212589347 >> 172.21.15.16:0/4212589347 conn(0x55ba3b67aa30 msgr2=0x55ba3b680790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:02:35.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.464+0000 7fe16f800740 1 -- 172.21.15.16:0/4212589347 shutdown_connections 2024-09-12T02:02:35.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.464+0000 7fe16f800740 1 -- 172.21.15.16:0/4212589347 wait complete. 2024-09-12T02:02:35.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.464+0000 7fe16f800740 20 client.15516 trim_cache size 0 max 16384 2024-09-12T02:02:35.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.498+0000 7fe16f800740 1 Processor -- start 2024-09-12T02:02:35.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.499+0000 7fe16f800740 1 -- start start 2024-09-12T02:02:35.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.499+0000 7fe16f800740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ba3b67e360 0x55ba3b6a95b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:35.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.499+0000 7fe16f800740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ba3b67ec80 0x55ba3b6a9af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:35.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.499+0000 7fe16f800740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ba3b68da90 0x55ba3b679350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:35.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.499+0000 7fe16f800740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe1681adaf0 con 0x55ba3b67e360 2024-09-12T02:02:35.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.499+0000 7fe16f800740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe1681ab3c0 con 0x55ba3b68da90 2024-09-12T02:02:35.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.499+0000 7fe16f800740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe1681a9210 con 0x55ba3b67ec80 2024-09-12T02:02:35.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.499+0000 7fe1677fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ba3b68da90 0x55ba3b679350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:35.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.499+0000 7fe1677fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ba3b68da90 0x55ba3b679350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.16:49794/0 (socket says 172.21.15.16:49794) 2024-09-12T02:02:35.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.499+0000 7fe1677fe640 1 -- 172.21.15.16:0/3180895297 learned_addr learned my addr 172.21.15.16:0/3180895297 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:02:35.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.499+0000 7fe1667fc640 1 --2- 172.21.15.16:0/3180895297 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ba3b67ec80 0x55ba3b6a9af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:35.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.500+0000 7fe1677fe640 1 -- 172.21.15.16:0/3180895297 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ba3b67ec80 msgr2=0x55ba3b6a9af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:35.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.500+0000 7fe1677fe640 1 --2- 172.21.15.16:0/3180895297 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ba3b67ec80 0x55ba3b6a9af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:35.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.500+0000 7fe1677fe640 1 -- 172.21.15.16:0/3180895297 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ba3b67e360 msgr2=0x55ba3b6a95b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:35.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.500+0000 7fe1677fe640 1 --2- 172.21.15.16:0/3180895297 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ba3b67e360 0x55ba3b6a95b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:35.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.500+0000 7fe1677fe640 1 -- 172.21.15.16:0/3180895297 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba3b67a4d0 con 0x55ba3b68da90 2024-09-12T02:02:35.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.500+0000 7fe1677fe640 1 --2- 172.21.15.16:0/3180895297 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ba3b68da90 0x55ba3b679350 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe160009cd0 tx=0x7fe160042000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f5148f5c5d5acf2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:02:35.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.501+0000 7fe14cff9640 1 -- 172.21.15.16:0/3180895297 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1600318f0 con 0x55ba3b68da90 2024-09-12T02:02:35.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.501+0000 7fe14cff9640 1 -- 172.21.15.16:0/3180895297 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe160036040 con 0x55ba3b68da90 2024-09-12T02:02:35.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.501+0000 7fe14cff9640 1 -- 172.21.15.16:0/3180895297 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe160046460 con 0x55ba3b68da90 2024-09-12T02:02:35.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.502+0000 7fe16f800740 1 -- 172.21.15.16:0/3180895297 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ba3b68da90 msgr2=0x55ba3b679350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:35.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.502+0000 7fe16f800740 1 --2- 172.21.15.16:0/3180895297 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ba3b68da90 0x55ba3b679350 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe160009cd0 tx=0x7fe160042000 comp rx=0 tx=0).stop 2024-09-12T02:02:35.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.502+0000 7fe16f800740 1 -- 172.21.15.16:0/3180895297 shutdown_connections 2024-09-12T02:02:35.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.502+0000 7fe16f800740 1 --2- 172.21.15.16:0/3180895297 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ba3b68da90 0x55ba3b679350 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:35.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.502+0000 7fe16f800740 1 --2- 172.21.15.16:0/3180895297 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ba3b67ec80 0x55ba3b6a9af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:35.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.502+0000 7fe16f800740 1 --2- 172.21.15.16:0/3180895297 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ba3b67e360 0x55ba3b6a95b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:35.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.502+0000 7fe16f800740 1 -- 172.21.15.16:0/3180895297 >> 172.21.15.16:0/3180895297 conn(0x55ba3b67aa30 msgr2=0x55ba3b68c7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:02:35.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.502+0000 7fe16f800740 1 -- 172.21.15.16:0/3180895297 shutdown_connections 2024-09-12T02:02:35.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.502+0000 7fe16f800740 1 -- 172.21.15.16:0/3180895297 wait complete. 2024-09-12T02:02:35.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.502+0000 7fe16f800740 1 -- start start 2024-09-12T02:02:35.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.503+0000 7fe16f800740 1 Processor -- start 2024-09-12T02:02:35.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.503+0000 7fe16f800740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ba3b67e360 0x55ba3b69e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:35.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.503+0000 7fe16f800740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ba3b67ec80 0x55ba3b699c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:35.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.503+0000 7fe16f800740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ba3b68da90 0x55ba3b69a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:35.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.503+0000 7fe16f800740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe1681a9880 con 0x55ba3b67e360 2024-09-12T02:02:35.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.503+0000 7fe16f800740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe1681aacf0 con 0x55ba3b67ec80 2024-09-12T02:02:35.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.503+0000 7fe16f800740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe1681a9f50 con 0x55ba3b68da90 2024-09-12T02:02:35.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.503+0000 7fe1667fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ba3b67ec80 0x55ba3b699c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:35.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.503+0000 7fe1667fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ba3b67ec80 0x55ba3b699c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.16:49800/0 (socket says 172.21.15.16:49800) 2024-09-12T02:02:35.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.503+0000 7fe1667fc640 1 -- 172.21.15.16:0/1007413911 learned_addr learned my addr 172.21.15.16:0/1007413911 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:02:35.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.503+0000 7fe16f800740 10 client.0 _ll_register_callbacks cb 0x7fe15c005870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:02:35.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.503+0000 7fe1667fc640 1 -- 172.21.15.16:0/1007413911 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ba3b68da90 msgr2=0x55ba3b69a160 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T02:02:35.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.503+0000 7fe166ffd640 1 --2- 172.21.15.16:0/1007413911 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ba3b67e360 0x55ba3b69e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:35.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.503+0000 7fe1667fc640 1 --2- 172.21.15.16:0/1007413911 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ba3b68da90 0x55ba3b69a160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:35.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.504+0000 7fe1667fc640 1 -- 172.21.15.16:0/1007413911 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ba3b67e360 msgr2=0x55ba3b69e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:35.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.504+0000 7fe1667fc640 1 --2- 172.21.15.16:0/1007413911 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ba3b67e360 0x55ba3b69e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:35.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.504+0000 7fe1667fc640 1 -- 172.21.15.16:0/1007413911 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba3b69a6a0 con 0x55ba3b67ec80 2024-09-12T02:02:35.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.504+0000 7fe166ffd640 1 --2- 172.21.15.16:0/1007413911 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ba3b67e360 0x55ba3b69e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:02:35.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.504+0000 7fe1667fc640 1 --2- 172.21.15.16:0/1007413911 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ba3b67ec80 0x55ba3b699c20 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x55ba3b68bd70 tx=0x7fe15c0055f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b2fb65b2e1b8c386 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:02:35.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.504+0000 7fe14aff5640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-12T02:02:35.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.504+0000 7fe14aff5640 1 -- 172.21.15.16:0/1007413911 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe15c005bf0 con 0x55ba3b67ec80 2024-09-12T02:02:35.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.504+0000 7fe14aff5640 1 -- 172.21.15.16:0/1007413911 <== mon.2 v2:172.21.15.33:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fe15c011440 con 0x55ba3b67ec80 2024-09-12T02:02:35.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.504+0000 7fe16f800740 1 -- 172.21.15.16:0/1007413911 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe160036040 con 0x55ba3b67ec80 2024-09-12T02:02:35.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.505+0000 7fe14aff5640 1 -- 172.21.15.16:0/1007413911 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe15c01b440 con 0x55ba3b67ec80 2024-09-12T02:02:35.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.505+0000 7fe16f800740 10 client.25196 fetch_fsmap learned FSMap version 25 2024-09-12T02:02:35.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.505+0000 7fe16f800740 1 -- 172.21.15.16:0/1007413911 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ba3b70c2a0 con 0x55ba3b67ec80 2024-09-12T02:02:35.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.505+0000 7fe14aff5640 1 -- 172.21.15.16:0/1007413911 <== mon.2 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe160036040 con 0x55ba3b67ec80 2024-09-12T02:02:35.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.505+0000 7fe14aff5640 1 -- 172.21.15.16:0/1007413911 <== mon.2 v2:172.21.15.33:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fe15c019040 con 0x55ba3b67ec80 2024-09-12T02:02:35.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.505+0000 7fe16f800740 10 client.25196 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:02:35.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.505+0000 7fe16f800740 10 client.25196 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:02:35.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.505+0000 7fe16f800740 20 client.25196 populate_metadata read hostname 'smithi016' 2024-09-12T02:02:35.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.505+0000 7fe16f800740 1 --2- 172.21.15.16:0/1007413911 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55ba3b6a1940 0x55ba3b70ea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:35.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.506+0000 7fe16f800740 4 client.25196 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T02:02:35.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.506+0000 7fe16f800740 1 -- 172.21.15.16:0/1007413911 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ba3b5ebd60 con 0x55ba3b6a1940 2024-09-12T02:02:35.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.506+0000 7fe166ffd640 1 --2- 172.21.15.16:0/1007413911 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55ba3b6a1940 0x55ba3b70ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:35.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.506+0000 7fe166ffd640 1 --2- 172.21.15.16:0/1007413911 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55ba3b6a1940 0x55ba3b70ea90 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=87c115628bc3ecc4 server_cookie=ea0285d4ef3a70be in_seq=0 out_seq=0 2024-09-12T02:02:35.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.506+0000 7fe14aff5640 10 client.25196 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:02:35.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.506+0000 7fe14aff5640 1 -- 172.21.15.16:0/1007413911 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55ba3b5ebd60 con 0x55ba3b6a1940 2024-09-12T02:02:35.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.506+0000 7fe14aff5640 10 client.25196 handle_command_reply: tid=0 2024-09-12T02:02:35.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.507+0000 7fe16f800740 1 client.25196 shutdown 2024-09-12T02:02:35.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.507+0000 7fe16f800740 1 -- 172.21.15.16:0/1007413911 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ba3b67ec80 msgr2=0x55ba3b699c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:35.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.507+0000 7fe16f800740 1 --2- 172.21.15.16:0/1007413911 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ba3b67ec80 0x55ba3b699c20 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x55ba3b68bd70 tx=0x7fe15c0055f0 comp rx=0 tx=0).stop 2024-09-12T02:02:35.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.507+0000 7fe16f800740 1 -- 172.21.15.16:0/1007413911 shutdown_connections 2024-09-12T02:02:35.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.507+0000 7fe16f800740 1 --2- 172.21.15.16:0/1007413911 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55ba3b6a1940 0x55ba3b70ea90 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:35.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.507+0000 7fe14aff5640 0 client.25196 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:02:35.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.507+0000 7fe16f800740 1 --2- 172.21.15.16:0/1007413911 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ba3b68da90 0x55ba3b69a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:35.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.507+0000 7fe16f800740 1 --2- 172.21.15.16:0/1007413911 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ba3b67ec80 0x55ba3b699c20 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:35.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.507+0000 7fe16f800740 1 --2- 172.21.15.16:0/1007413911 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ba3b67e360 0x55ba3b69e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:35.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.507+0000 7fe16f800740 1 -- 172.21.15.16:0/1007413911 >> 172.21.15.16:0/1007413911 conn(0x55ba3b67aa30 msgr2=0x55ba3b685430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:02:35.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.507+0000 7fe16f800740 1 -- 172.21.15.16:0/1007413911 shutdown_connections 2024-09-12T02:02:35.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.507+0000 7fe16f800740 1 -- 172.21.15.16:0/1007413911 wait complete. 2024-09-12T02:02:35.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.508+0000 7fe16f800740 20 client.25196 trim_cache size 0 max 16384 2024-09-12T02:02:35.510 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T02:02:35.510 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-12T02:02:35.510 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": { 2024-09-12T02:02:35.510 INFO:teuthology.orchestra.run.smithi016.stdout: "c341ae6d-5d8d-40f8-840b-45294fef76f8": { 2024-09-12T02:02:35.510 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-12T02:02:35.511 INFO:teuthology.orchestra.run.smithi016.stdout: "tag": "c341ae6d-5d8d-40f8-840b-45294fef76f8", 2024-09-12T02:02:35.511 INFO:teuthology.orchestra.run.smithi016.stdout: "options": "recursive,force" 2024-09-12T02:02:35.511 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:02:35.511 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:02:35.511 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T02:02:35.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.508+0000 7fe16d427640 1 -- 172.21.15.16:0/2675332932 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fe1500a2b90 msgr2=0x7fe1500a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:35.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.508+0000 7fe16d427640 1 --2- 172.21.15.16:0/2675332932 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fe1500a2b90 0x7fe1500a5050 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fe160004af0 tx=0x7fe16003f040 comp rx=0 tx=0).stop 2024-09-12T02:02:35.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.508+0000 7fe16d427640 1 -- 172.21.15.16:0/2675332932 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe168100a90 msgr2=0x7fe168197320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:35.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.508+0000 7fe16d427640 1 --2- 172.21.15.16:0/2675332932 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe168100a90 0x7fe168197320 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fe15800e9f0 tx=0x7fe15800edf0 comp rx=0 tx=0).stop 2024-09-12T02:02:35.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.509+0000 7fe16d427640 1 -- 172.21.15.16:0/2675332932 shutdown_connections 2024-09-12T02:02:35.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.509+0000 7fe16d427640 1 --2- 172.21.15.16:0/2675332932 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fe1500a2b90 0x7fe1500a5050 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:35.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.509+0000 7fe16d427640 1 --2- 172.21.15.16:0/2675332932 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe168106a90 0x7fe16819bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:35.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.509+0000 7fe16d427640 1 --2- 172.21.15.16:0/2675332932 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe168101430 0x7fe168197860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:35.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.509+0000 7fe16d427640 1 --2- 172.21.15.16:0/2675332932 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe168100a90 0x7fe168197320 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:35.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.509+0000 7fe16d427640 1 -- 172.21.15.16:0/2675332932 >> 172.21.15.16:0/2675332932 conn(0x7fe1680fc7d0 msgr2=0x7fe1680fce80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:02:35.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.509+0000 7fe16d427640 1 -- 172.21.15.16:0/2675332932 shutdown_connections 2024-09-12T02:02:35.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:35.509+0000 7fe16d427640 1 -- 172.21.15.16:0/2675332932 wait complete. 2024-09-12T02:02:35.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:02:35 smithi033 ceph-mon[35332]: pgmap v355: 97 pgs: 97 active+clean; 666 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 938 B/s wr, 7 op/s 2024-09-12T02:02:35.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:02:35 smithi016 ceph-mon[30947]: pgmap v355: 97 pgs: 97 active+clean; 666 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 938 B/s wr, 7 op/s 2024-09-12T02:02:35.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:02:35 smithi184 ceph-mon[35593]: pgmap v355: 97 pgs: 97 active+clean; 666 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 938 B/s wr, 7 op/s 2024-09-12T02:02:36.195 INFO:tasks.cephfs.filesystem:scrub status for tag:c341ae6d-5d8d-40f8-840b-45294fef76f8 - {'path': '/', 'tag': 'c341ae6d-5d8d-40f8-840b-45294fef76f8', 'options': 'recursive,force'} 2024-09-12T02:02:36.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:02:36 smithi033 ceph-mon[35332]: scrub summary: idle 2024-09-12T02:02:36.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:02:36 smithi016 ceph-mon[30947]: scrub summary: idle 2024-09-12T02:02:36.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:02:36 smithi184 ceph-mon[35593]: scrub summary: idle 2024-09-12T02:02:37.044 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.044+0000 7ffa94d6d640 1 -- 172.21.15.16:0/2257134287 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa901078c0 msgr2=0x7ffa90109cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:37.044 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.044+0000 7ffa94d6d640 1 --2- 172.21.15.16:0/2257134287 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa901078c0 0x7ffa90109cb0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7ffa88009ca0 tx=0x7ffa8802a8c0 comp rx=0 tx=0).stop 2024-09-12T02:02:37.044 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.044+0000 7ffa94d6d640 1 -- 172.21.15.16:0/2257134287 shutdown_connections 2024-09-12T02:02:37.044 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.044+0000 7ffa94d6d640 1 --2- 172.21.15.16:0/2257134287 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa901078c0 0x7ffa90109cb0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:37.044 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.044+0000 7ffa94d6d640 1 --2- 172.21.15.16:0/2257134287 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffa90104f90 0x7ffa90107380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:37.044 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.044+0000 7ffa94d6d640 1 --2- 172.21.15.16:0/2257134287 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa90102660 0x7ffa90104a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:37.044 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.044+0000 7ffa94d6d640 1 -- 172.21.15.16:0/2257134287 >> 172.21.15.16:0/2257134287 conn(0x7ffa900fc1e0 msgr2=0x7ffa900fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:02:37.044 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.044+0000 7ffa94d6d640 1 -- 172.21.15.16:0/2257134287 shutdown_connections 2024-09-12T02:02:37.044 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.044+0000 7ffa94d6d640 1 -- 172.21.15.16:0/2257134287 wait complete. 2024-09-12T02:02:37.046 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.045+0000 7ffa94d6d640 1 Processor -- start 2024-09-12T02:02:37.046 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.045+0000 7ffa94d6d640 1 -- start start 2024-09-12T02:02:37.046 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.045+0000 7ffa94d6d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa90102660 0x7ffa90197dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:37.046 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.045+0000 7ffa94d6d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa90104f90 0x7ffa90198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:37.046 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.045+0000 7ffa94d6d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffa901078c0 0x7ffa9019f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:37.047 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.045+0000 7ffa8dd9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa90104f90 0x7ffa90198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:37.047 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.045+0000 7ffa8dd9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa90104f90 0x7ffa90198310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37992/0 (socket says 172.21.15.16:37992) 2024-09-12T02:02:37.047 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.045+0000 7ffa8dd9b640 1 -- 172.21.15.16:0/1406346154 learned_addr learned my addr 172.21.15.16:0/1406346154 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:02:37.047 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.045+0000 7ffa94d6d640 1 -- 172.21.15.16:0/1406346154 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ffa9010d830 con 0x7ffa90104f90 2024-09-12T02:02:37.047 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.045+0000 7ffa94d6d640 1 -- 172.21.15.16:0/1406346154 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ffa9010d6b0 con 0x7ffa90102660 2024-09-12T02:02:37.047 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.045+0000 7ffa94d6d640 1 -- 172.21.15.16:0/1406346154 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ffa9010d9b0 con 0x7ffa901078c0 2024-09-12T02:02:37.047 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.045+0000 7ffa8e59c640 1 --2- 172.21.15.16:0/1406346154 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa90102660 0x7ffa90197dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:37.047 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.045+0000 7ffa8ed9d640 1 --2- 172.21.15.16:0/1406346154 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffa901078c0 0x7ffa9019f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:37.047 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.046+0000 7ffa8dd9b640 1 -- 172.21.15.16:0/1406346154 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffa901078c0 msgr2=0x7ffa9019f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:37.047 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.046+0000 7ffa8dd9b640 1 --2- 172.21.15.16:0/1406346154 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffa901078c0 0x7ffa9019f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:37.048 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.046+0000 7ffa8dd9b640 1 -- 172.21.15.16:0/1406346154 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa90102660 msgr2=0x7ffa90197dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:37.048 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.046+0000 7ffa8dd9b640 1 --2- 172.21.15.16:0/1406346154 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa90102660 0x7ffa90197dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:37.048 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.046+0000 7ffa8dd9b640 1 -- 172.21.15.16:0/1406346154 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa9019f9f0 con 0x7ffa90104f90 2024-09-12T02:02:37.048 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.046+0000 7ffa8dd9b640 1 --2- 172.21.15.16:0/1406346154 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa90104f90 0x7ffa90198310 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7ffa8400d770 tx=0x7ffa8400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:02:37.048 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.046+0000 7ffa7f7fe640 1 -- 172.21.15.16:0/1406346154 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa84017070 con 0x7ffa90104f90 2024-09-12T02:02:37.048 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.046+0000 7ffa7f7fe640 1 -- 172.21.15.16:0/1406346154 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa84009c30 con 0x7ffa90104f90 2024-09-12T02:02:37.048 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.046+0000 7ffa94d6d640 1 -- 172.21.15.16:0/1406346154 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa9019fce0 con 0x7ffa90104f90 2024-09-12T02:02:37.048 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.046+0000 7ffa7f7fe640 1 -- 172.21.15.16:0/1406346154 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa8400f620 con 0x7ffa90104f90 2024-09-12T02:02:37.049 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.046+0000 7ffa94d6d640 1 -- 172.21.15.16:0/1406346154 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa901a01c0 con 0x7ffa90104f90 2024-09-12T02:02:37.049 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.050+0000 7ffa7f7fe640 1 -- 172.21.15.16:0/1406346154 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ffa8401a070 con 0x7ffa90104f90 2024-09-12T02:02:37.049 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.050+0000 7ffa97bea740 1 Processor -- start 2024-09-12T02:02:37.051 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.050+0000 7ffa97bea740 1 -- start start 2024-09-12T02:02:37.051 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.050+0000 7ffa97bea740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a6a03259a0 0x55a6a0325da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:37.051 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.050+0000 7ffa97bea740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a6a03262e0 0x55a6a032ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:37.051 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.050+0000 7ffa97bea740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a6a032b370 0x55a6a032d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:37.051 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.050+0000 7ffa97bea740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55a6a02892f0 con 0x55a6a03262e0 2024-09-12T02:02:37.051 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.050+0000 7ffa97bea740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55a6a0231270 con 0x55a6a032b370 2024-09-12T02:02:37.051 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.050+0000 7ffa97bea740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55a6a02852b0 con 0x55a6a03259a0 2024-09-12T02:02:37.052 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.050+0000 7ffa8e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a6a03262e0 0x55a6a032ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:37.052 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.050+0000 7ffa8e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a6a03262e0 0x55a6a032ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38008/0 (socket says 172.21.15.16:38008) 2024-09-12T02:02:37.052 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.050+0000 7ffa8ed9d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a6a03259a0 0x55a6a0325da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:37.052 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.050+0000 7ffa8e59c640 1 -- 172.21.15.16:0/4269156089 learned_addr learned my addr 172.21.15.16:0/4269156089 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:02:37.052 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.050+0000 7ffa8dd9b640 1 --2- 172.21.15.16:0/4269156089 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a6a032b370 0x55a6a032d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:37.052 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.051+0000 7ffa8e59c640 1 -- 172.21.15.16:0/4269156089 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a6a03259a0 msgr2=0x55a6a0325da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:37.052 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.051+0000 7ffa7f7fe640 1 --2- 172.21.15.16:0/1406346154 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7ffa780a2cb0 0x7ffa780a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:37.052 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.051+0000 7ffa8e59c640 1 --2- 172.21.15.16:0/4269156089 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a6a03259a0 0x55a6a0325da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:37.052 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.051+0000 7ffa8e59c640 1 -- 172.21.15.16:0/4269156089 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a6a032b370 msgr2=0x55a6a032d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:37.052 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.051+0000 7ffa8e59c640 1 --2- 172.21.15.16:0/4269156089 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a6a032b370 0x55a6a032d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:37.053 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.051+0000 7ffa8e59c640 1 -- 172.21.15.16:0/4269156089 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a6a0230340 con 0x55a6a03262e0 2024-09-12T02:02:37.053 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.051+0000 7ffa8dd9b640 1 --2- 172.21.15.16:0/4269156089 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a6a032b370 0x55a6a032d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:02:37.053 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.051+0000 7ffa8ed9d640 1 --2- 172.21.15.16:0/4269156089 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a6a03259a0 0x55a6a0325da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:02:37.053 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.051+0000 7ffa7f7fe640 1 -- 172.21.15.16:0/1406346154 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7ffa840ceda0 con 0x7ffa90104f90 2024-09-12T02:02:37.053 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.051+0000 7ffa8ed9d640 1 --2- 172.21.15.16:0/1406346154 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7ffa780a2cb0 0x7ffa780a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:37.053 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.051+0000 7ffa8e59c640 1 --2- 172.21.15.16:0/4269156089 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a6a03262e0 0x55a6a032ada0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7ffa8000bd10 tx=0x7ffa800107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53d204d5333002cb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:02:37.056 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.052+0000 7ffa7cff9640 1 -- 172.21.15.16:0/4269156089 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa80004030 con 0x55a6a03262e0 2024-09-12T02:02:37.056 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.052+0000 7ffa8ed9d640 1 --2- 172.21.15.16:0/1406346154 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7ffa780a2cb0 0x7ffa780a5170 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7ffa88006010 tx=0x7ffa880095c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:02:37.056 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.052+0000 7ffa7cff9640 1 -- 172.21.15.16:0/4269156089 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa800041d0 con 0x55a6a03262e0 2024-09-12T02:02:37.056 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.052+0000 7ffa7cff9640 1 -- 172.21.15.16:0/4269156089 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa80004030 con 0x55a6a03262e0 2024-09-12T02:02:37.056 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.052+0000 7ffa97bea740 1 -- 172.21.15.16:0/4269156089 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a6a03262e0 msgr2=0x55a6a032ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:37.056 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.052+0000 7ffa97bea740 1 --2- 172.21.15.16:0/4269156089 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a6a03262e0 0x55a6a032ada0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7ffa8000bd10 tx=0x7ffa800107a0 comp rx=0 tx=0).stop 2024-09-12T02:02:37.056 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.052+0000 7ffa97bea740 1 -- 172.21.15.16:0/4269156089 shutdown_connections 2024-09-12T02:02:37.056 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.053+0000 7ffa97bea740 1 --2- 172.21.15.16:0/4269156089 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a6a032b370 0x55a6a032d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:37.056 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.053+0000 7ffa97bea740 1 --2- 172.21.15.16:0/4269156089 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a6a03262e0 0x55a6a032ada0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:37.057 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.053+0000 7ffa97bea740 1 --2- 172.21.15.16:0/4269156089 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a6a03259a0 0x55a6a0325da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:37.057 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.053+0000 7ffa97bea740 1 -- 172.21.15.16:0/4269156089 >> 172.21.15.16:0/4269156089 conn(0x55a6a0321010 msgr2=0x55a6a0321410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:02:37.057 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.053+0000 7ffa97bea740 1 -- 172.21.15.16:0/4269156089 shutdown_connections 2024-09-12T02:02:37.057 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.053+0000 7ffa97bea740 1 -- 172.21.15.16:0/4269156089 wait complete. 2024-09-12T02:02:37.057 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.053+0000 7ffa97bea740 1 -- start start 2024-09-12T02:02:37.057 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.053+0000 7ffa97bea740 1 Processor -- start 2024-09-12T02:02:37.057 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.053+0000 7ffa97bea740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a6a03259a0 0x55a6a032fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:37.057 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.053+0000 7ffa97bea740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a6a03262e0 0x55a6a0330110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:37.057 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.054+0000 7ffa97bea740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a6a032b370 0x55a6a0337150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:37.057 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.054+0000 7ffa97bea740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55a6a032e340 con 0x55a6a03259a0 2024-09-12T02:02:37.057 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.054+0000 7ffa97bea740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55a6a0212290 con 0x55a6a03262e0 2024-09-12T02:02:37.057 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.054+0000 7ffa97bea740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55a6a032e4c0 con 0x55a6a032b370 2024-09-12T02:02:37.057 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.054+0000 7ffa97bea740 10 client.0 _ll_register_callbacks cb 0x55a6a0243140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:02:37.058 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.054+0000 7ffa8ed9d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a6a032b370 0x55a6a0337150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:37.058 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.054+0000 7ffa8dd9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a6a03262e0 0x55a6a0330110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:37.058 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.054+0000 7ffa8ed9d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a6a032b370 0x55a6a0337150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:36518/0 (socket says 172.21.15.16:36518) 2024-09-12T02:02:37.058 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.054+0000 7ffa8dd9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a6a03262e0 0x55a6a0330110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.16:49834/0 (socket says 172.21.15.16:49834) 2024-09-12T02:02:37.058 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.054+0000 7ffa8ed9d640 1 -- 172.21.15.16:0/3952434405 learned_addr learned my addr 172.21.15.16:0/3952434405 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:02:37.058 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.054+0000 7ffa8ed9d640 1 -- 172.21.15.16:0/3952434405 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a6a03262e0 msgr2=0x55a6a0330110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:37.058 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.054+0000 7ffa8ed9d640 1 --2- 172.21.15.16:0/3952434405 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a6a03262e0 0x55a6a0330110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:37.058 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.054+0000 7ffa8ed9d640 1 -- 172.21.15.16:0/3952434405 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a6a03259a0 msgr2=0x55a6a032fbd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-12T02:02:37.058 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.054+0000 7ffa8ed9d640 1 --2- 172.21.15.16:0/3952434405 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a6a03259a0 0x55a6a032fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:37.058 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.054+0000 7ffa8ed9d640 1 -- 172.21.15.16:0/3952434405 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a69ff9c210 con 0x55a6a032b370 2024-09-12T02:02:37.058 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.054+0000 7ffa8dd9b640 1 --2- 172.21.15.16:0/3952434405 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a6a03262e0 0x55a6a0330110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:02:37.058 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.055+0000 7ffa8ed9d640 1 --2- 172.21.15.16:0/3952434405 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a6a032b370 0x55a6a0337150 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ffa88004af0 tx=0x7ffa8802a8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=634ba53ee96e6da9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:02:37.058 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.055+0000 7ffa75ffb640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-09-12T02:02:37.059 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.055+0000 7ffa75ffb640 1 -- 172.21.15.16:0/3952434405 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa88003b70 con 0x55a6a032b370 2024-09-12T02:02:37.059 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.055+0000 7ffa97bea740 1 -- 172.21.15.16:0/3952434405 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa800041d0 con 0x55a6a032b370 2024-09-12T02:02:37.059 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.055+0000 7ffa75ffb640 1 -- 172.21.15.16:0/3952434405 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7ffa88003d10 con 0x55a6a032b370 2024-09-12T02:02:37.059 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.055+0000 7ffa75ffb640 1 -- 172.21.15.16:0/3952434405 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ffa880307d0 con 0x55a6a032b370 2024-09-12T02:02:37.059 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.055+0000 7ffa97bea740 10 client.25147 fetch_fsmap learned FSMap version 25 2024-09-12T02:02:37.059 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.055+0000 7ffa97bea740 1 -- 172.21.15.16:0/3952434405 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a6a0068620 con 0x55a6a032b370 2024-09-12T02:02:37.059 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.055+0000 7ffa75ffb640 1 -- 172.21.15.16:0/3952434405 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa800041d0 con 0x55a6a032b370 2024-09-12T02:02:37.059 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.056+0000 7ffa75ffb640 1 -- 172.21.15.16:0/3952434405 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7ffa88045070 con 0x55a6a032b370 2024-09-12T02:02:37.059 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.056+0000 7ffa97bea740 10 client.25147 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:02:37.059 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.056+0000 7ffa97bea740 10 client.25147 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:02:37.059 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.056+0000 7ffa97bea740 20 client.25147 populate_metadata read hostname 'smithi016' 2024-09-12T02:02:37.059 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.056+0000 7ffa97bea740 1 --2- 172.21.15.16:0/3952434405 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55a6a033b890 0x55a6a033dd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:37.059 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.056+0000 7ffa97bea740 4 client.25147 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:02:37.060 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.056+0000 7ffa97bea740 1 -- 172.21.15.16:0/3952434405 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a6a0296a20 con 0x55a6a033b890 2024-09-12T02:02:37.061 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.056+0000 7ffa8e59c640 1 --2- 172.21.15.16:0/3952434405 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55a6a033b890 0x55a6a033dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:37.061 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.057+0000 7ffa75ffb640 10 client.25147 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:02:37.061 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.057+0000 7ffa8e59c640 1 --2- 172.21.15.16:0/3952434405 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55a6a033b890 0x55a6a033dd20 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6da92fbdc6577e01 server_cookie=1e1574362cabdbc4 in_seq=0 out_seq=0 2024-09-12T02:02:37.061 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.058+0000 7ffa75ffb640 1 -- 172.21.15.16:0/3952434405 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a6a0296a20 con 0x55a6a033b890 2024-09-12T02:02:37.061 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.058+0000 7ffa75ffb640 10 client.25147 handle_command_reply: tid=0 2024-09-12T02:02:37.061 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.058+0000 7ffa97bea740 1 client.25147 shutdown 2024-09-12T02:02:37.061 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.058+0000 7ffa97bea740 1 -- 172.21.15.16:0/3952434405 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a6a032b370 msgr2=0x55a6a0337150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:37.061 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.058+0000 7ffa97bea740 1 --2- 172.21.15.16:0/3952434405 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a6a032b370 0x55a6a0337150 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ffa88004af0 tx=0x7ffa8802a8c0 comp rx=0 tx=0).stop 2024-09-12T02:02:37.062 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.058+0000 7ffa97bea740 1 -- 172.21.15.16:0/3952434405 shutdown_connections 2024-09-12T02:02:37.062 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.058+0000 7ffa97bea740 1 --2- 172.21.15.16:0/3952434405 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55a6a033b890 0x55a6a033dd20 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:37.062 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.058+0000 7ffa97bea740 1 --2- 172.21.15.16:0/3952434405 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a6a032b370 0x55a6a0337150 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:37.062 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.058+0000 7ffa97bea740 1 --2- 172.21.15.16:0/3952434405 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a6a03262e0 0x55a6a0330110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:37.062 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.058+0000 7ffa97bea740 1 --2- 172.21.15.16:0/3952434405 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a6a03259a0 0x55a6a032fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:37.062 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.058+0000 7ffa97bea740 1 -- 172.21.15.16:0/3952434405 >> 172.21.15.16:0/3952434405 conn(0x55a6a0321010 msgr2=0x55a6a0322820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:02:37.062 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.058+0000 7ffa75ffb640 0 client.25147 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:02:37.062 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.058+0000 7ffa97bea740 1 -- 172.21.15.16:0/3952434405 shutdown_connections 2024-09-12T02:02:37.062 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.058+0000 7ffa97bea740 1 -- 172.21.15.16:0/3952434405 wait complete. 2024-09-12T02:02:37.062 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.058+0000 7ffa97bea740 20 client.25147 trim_cache size 0 max 16384 2024-09-12T02:02:37.086 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.086+0000 7ffa97bea740 1 Processor -- start 2024-09-12T02:02:37.087 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.086+0000 7ffa97bea740 1 -- start start 2024-09-12T02:02:37.087 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.087+0000 7ffa97bea740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a6a03259a0 0x55a6a0321ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:37.087 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.087+0000 7ffa97bea740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a6a03262e0 0x55a6a0337110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:37.087 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.087+0000 7ffa97bea740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a6a032b370 0x55a6a0322400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:37.087 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.087+0000 7ffa97bea740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ffa901ad9d0 con 0x55a6a03259a0 2024-09-12T02:02:37.087 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.087+0000 7ffa97bea740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ffa901ab2a0 con 0x55a6a03262e0 2024-09-12T02:02:37.087 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.087+0000 7ffa97bea740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ffa901a9110 con 0x55a6a032b370 2024-09-12T02:02:37.087 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.087+0000 7ffa8e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a6a03259a0 0x55a6a0321ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:37.087 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.087+0000 7ffa8e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a6a03259a0 0x55a6a0321ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38026/0 (socket says 172.21.15.16:38026) 2024-09-12T02:02:37.087 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.087+0000 7ffa8e59c640 1 -- 172.21.15.16:0/47970836 learned_addr learned my addr 172.21.15.16:0/47970836 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:02:37.087 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.087+0000 7ffa8dd9b640 1 --2- 172.21.15.16:0/47970836 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a6a03262e0 0x55a6a0337110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:37.089 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.087+0000 7ffa8e59c640 1 -- 172.21.15.16:0/47970836 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a6a032b370 msgr2=0x55a6a0322400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:37.089 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.087+0000 7ffa8e59c640 1 --2- 172.21.15.16:0/47970836 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a6a032b370 0x55a6a0322400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:37.089 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.087+0000 7ffa8e59c640 1 -- 172.21.15.16:0/47970836 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a6a03262e0 msgr2=0x55a6a0337110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:37.089 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.087+0000 7ffa8e59c640 1 --2- 172.21.15.16:0/47970836 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a6a03262e0 0x55a6a0337110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:37.089 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.087+0000 7ffa8e59c640 1 -- 172.21.15.16:0/47970836 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a6a0326f70 con 0x55a6a03259a0 2024-09-12T02:02:37.089 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.087+0000 7ffa8e59c640 1 --2- 172.21.15.16:0/47970836 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a6a03259a0 0x55a6a0321ec0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7ffa8000bce0 tx=0x7ffa8000bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41bd05d23a9f6ebb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:02:37.089 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.088+0000 7ffa77fff640 1 -- 172.21.15.16:0/47970836 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa80005240 con 0x55a6a03259a0 2024-09-12T02:02:37.089 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.088+0000 7ffa77fff640 1 -- 172.21.15.16:0/47970836 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa800053e0 con 0x55a6a03259a0 2024-09-12T02:02:37.089 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.088+0000 7ffa77fff640 1 -- 172.21.15.16:0/47970836 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa80026690 con 0x55a6a03259a0 2024-09-12T02:02:37.090 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.089+0000 7ffa97bea740 1 -- 172.21.15.16:0/47970836 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a6a03259a0 msgr2=0x55a6a0321ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:37.090 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.089+0000 7ffa97bea740 1 --2- 172.21.15.16:0/47970836 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a6a03259a0 0x55a6a0321ec0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7ffa8000bce0 tx=0x7ffa8000bf90 comp rx=0 tx=0).stop 2024-09-12T02:02:37.090 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.089+0000 7ffa97bea740 1 -- 172.21.15.16:0/47970836 shutdown_connections 2024-09-12T02:02:37.090 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.089+0000 7ffa97bea740 1 --2- 172.21.15.16:0/47970836 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a6a032b370 0x55a6a0322400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:37.090 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.089+0000 7ffa97bea740 1 --2- 172.21.15.16:0/47970836 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a6a03262e0 0x55a6a0337110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:37.090 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.089+0000 7ffa97bea740 1 --2- 172.21.15.16:0/47970836 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a6a03259a0 0x55a6a0321ec0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:37.090 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.089+0000 7ffa97bea740 1 -- 172.21.15.16:0/47970836 >> 172.21.15.16:0/47970836 conn(0x55a6a0321010 msgr2=0x55a6a032fbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:02:37.090 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.089+0000 7ffa97bea740 1 -- 172.21.15.16:0/47970836 shutdown_connections 2024-09-12T02:02:37.090 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.089+0000 7ffa97bea740 1 -- 172.21.15.16:0/47970836 wait complete. 2024-09-12T02:02:37.090 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.089+0000 7ffa97bea740 1 -- start start 2024-09-12T02:02:37.090 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.089+0000 7ffa97bea740 1 Processor -- start 2024-09-12T02:02:37.092 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.090+0000 7ffa97bea740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a6a03259a0 0x55a6a03349f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:37.093 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.090+0000 7ffa97bea740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a6a03262e0 0x55a6a0334f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:37.093 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.090+0000 7ffa97bea740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a6a032b370 0x55a6a0335f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:37.093 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.090+0000 7ffa97bea740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ffa901a9760 con 0x55a6a03259a0 2024-09-12T02:02:37.093 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.090+0000 7ffa97bea740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ffa901aabd0 con 0x55a6a032b370 2024-09-12T02:02:37.093 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.090+0000 7ffa97bea740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ffa901a9e30 con 0x55a6a03262e0 2024-09-12T02:02:37.093 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.090+0000 7ffa97bea740 10 client.0 _ll_register_callbacks cb 0x7ffa88009360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:02:37.093 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.090+0000 7ffa8e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a6a03259a0 0x55a6a03349f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:37.093 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.090+0000 7ffa8e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a6a03259a0 0x55a6a03349f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38034/0 (socket says 172.21.15.16:38034) 2024-09-12T02:02:37.093 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.090+0000 7ffa8e59c640 1 -- 172.21.15.16:0/1235809861 learned_addr learned my addr 172.21.15.16:0/1235809861 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:02:37.093 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.090+0000 7ffa8ed9d640 1 --2- 172.21.15.16:0/1235809861 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a6a032b370 0x55a6a0335f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:37.093 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.090+0000 7ffa8ed9d640 1 -- 172.21.15.16:0/1235809861 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a6a03262e0 msgr2=0x55a6a0334f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:37.093 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.090+0000 7ffa8dd9b640 1 --2- 172.21.15.16:0/1235809861 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a6a03262e0 0x55a6a0334f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:37.093 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.090+0000 7ffa8ed9d640 1 --2- 172.21.15.16:0/1235809861 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a6a03262e0 0x55a6a0334f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:37.094 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.090+0000 7ffa8ed9d640 1 -- 172.21.15.16:0/1235809861 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a6a03259a0 msgr2=0x55a6a03349f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:37.094 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.090+0000 7ffa8ed9d640 1 --2- 172.21.15.16:0/1235809861 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a6a03259a0 0x55a6a03349f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:37.094 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.090+0000 7ffa8ed9d640 1 -- 172.21.15.16:0/1235809861 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a6a032ebc0 con 0x55a6a032b370 2024-09-12T02:02:37.094 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.090+0000 7ffa8dd9b640 1 --2- 172.21.15.16:0/1235809861 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a6a03262e0 0x55a6a0334f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:02:37.094 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.090+0000 7ffa8e59c640 1 --2- 172.21.15.16:0/1235809861 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a6a03259a0 0x55a6a03349f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:02:37.094 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.090+0000 7ffa8ed9d640 1 --2- 172.21.15.16:0/1235809861 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a6a032b370 0x55a6a0335f40 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ffa88004560 tx=0x7ffa88049410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f16522d488367f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:02:37.094 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.090+0000 7ffa75ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-12T02:02:37.094 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.091+0000 7ffa75ffb640 1 -- 172.21.15.16:0/1235809861 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa880098c0 con 0x55a6a032b370 2024-09-12T02:02:37.094 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.091+0000 7ffa75ffb640 1 -- 172.21.15.16:0/1235809861 <== mon.2 v2:172.21.15.33:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7ffa88049570 con 0x55a6a032b370 2024-09-12T02:02:37.094 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.091+0000 7ffa97bea740 1 -- 172.21.15.16:0/1235809861 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa800053e0 con 0x55a6a032b370 2024-09-12T02:02:37.094 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.091+0000 7ffa75ffb640 1 -- 172.21.15.16:0/1235809861 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ffa88065050 con 0x55a6a032b370 2024-09-12T02:02:37.094 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.091+0000 7ffa97bea740 10 client.25214 fetch_fsmap learned FSMap version 25 2024-09-12T02:02:37.094 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.091+0000 7ffa97bea740 1 -- 172.21.15.16:0/1235809861 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a6a03a6660 con 0x55a6a032b370 2024-09-12T02:02:37.095 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.091+0000 7ffa75ffb640 1 -- 172.21.15.16:0/1235809861 <== mon.2 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa88036040 con 0x55a6a032b370 2024-09-12T02:02:37.095 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.091+0000 7ffa75ffb640 1 -- 172.21.15.16:0/1235809861 <== mon.2 v2:172.21.15.33:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7ffa88045040 con 0x55a6a032b370 2024-09-12T02:02:37.095 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.091+0000 7ffa97bea740 10 client.25214 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:02:37.095 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.092+0000 7ffa97bea740 10 client.25214 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:02:37.095 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.092+0000 7ffa97bea740 20 client.25214 populate_metadata read hostname 'smithi016' 2024-09-12T02:02:37.095 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.092+0000 7ffa97bea740 1 --2- 172.21.15.16:0/1235809861 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55a6a03a6f50 0x55a6a03a9340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:37.095 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.092+0000 7ffa97bea740 4 client.25214 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:02:37.095 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.092+0000 7ffa97bea740 1 -- 172.21.15.16:0/1235809861 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7ffa800053e0 con 0x55a6a03a6f50 2024-09-12T02:02:37.095 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.092+0000 7ffa8e59c640 1 --2- 172.21.15.16:0/1235809861 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55a6a03a6f50 0x55a6a03a9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:37.095 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.092+0000 7ffa75ffb640 10 client.25214 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:02:37.095 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.092+0000 7ffa8e59c640 1 --2- 172.21.15.16:0/1235809861 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55a6a03a6f50 0x55a6a03a9340 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=73fff41c01e8a353 server_cookie=3450b912ee0af174 in_seq=0 out_seq=0 2024-09-12T02:02:37.100 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.100+0000 7ffa75ffb640 1 -- 172.21.15.16:0/1235809861 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7ffa800053e0 con 0x55a6a03a6f50 2024-09-12T02:02:37.100 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.100+0000 7ffa75ffb640 10 client.25214 handle_command_reply: tid=0 2024-09-12T02:02:37.100 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.100+0000 7ffa97bea740 1 client.25214 shutdown 2024-09-12T02:02:37.100 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.100+0000 7ffa97bea740 1 -- 172.21.15.16:0/1235809861 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a6a032b370 msgr2=0x55a6a0335f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:37.100 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.100+0000 7ffa97bea740 1 --2- 172.21.15.16:0/1235809861 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a6a032b370 0x55a6a0335f40 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ffa88004560 tx=0x7ffa88049410 comp rx=0 tx=0).stop 2024-09-12T02:02:37.101 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.100+0000 7ffa97bea740 1 -- 172.21.15.16:0/1235809861 shutdown_connections 2024-09-12T02:02:37.101 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.100+0000 7ffa97bea740 1 --2- 172.21.15.16:0/1235809861 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55a6a03a6f50 0x55a6a03a9340 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:37.101 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.100+0000 7ffa97bea740 1 --2- 172.21.15.16:0/1235809861 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a6a032b370 0x55a6a0335f40 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:37.101 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.100+0000 7ffa97bea740 1 --2- 172.21.15.16:0/1235809861 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a6a03262e0 0x55a6a0334f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:37.101 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.100+0000 7ffa97bea740 1 --2- 172.21.15.16:0/1235809861 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a6a03259a0 0x55a6a03349f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:37.101 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.100+0000 7ffa97bea740 1 -- 172.21.15.16:0/1235809861 >> 172.21.15.16:0/1235809861 conn(0x55a6a0321010 msgr2=0x55a6a031f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:02:37.101 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.100+0000 7ffa75ffb640 0 client.25214 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:02:37.101 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.100+0000 7ffa97bea740 1 -- 172.21.15.16:0/1235809861 shutdown_connections 2024-09-12T02:02:37.101 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.100+0000 7ffa97bea740 1 -- 172.21.15.16:0/1235809861 wait complete. 2024-09-12T02:02:37.101 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.100+0000 7ffa97bea740 20 client.25214 trim_cache size 0 max 16384 2024-09-12T02:02:37.102 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-09-12T02:02:37.102 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-09-12T02:02:37.102 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-09-12T02:02:37.102 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-09-12T02:02:37.102 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.101+0000 7ffa94d6d640 1 -- 172.21.15.16:0/1406346154 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7ffa780a2cb0 msgr2=0x7ffa780a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:37.103 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.101+0000 7ffa94d6d640 1 --2- 172.21.15.16:0/1406346154 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7ffa780a2cb0 0x7ffa780a5170 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7ffa88006010 tx=0x7ffa880095c0 comp rx=0 tx=0).stop 2024-09-12T02:02:37.103 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.101+0000 7ffa94d6d640 1 -- 172.21.15.16:0/1406346154 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa90104f90 msgr2=0x7ffa90198310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:37.103 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.101+0000 7ffa94d6d640 1 --2- 172.21.15.16:0/1406346154 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa90104f90 0x7ffa90198310 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7ffa8400d770 tx=0x7ffa8400dc30 comp rx=0 tx=0).stop 2024-09-12T02:02:37.103 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.101+0000 7ffa94d6d640 1 -- 172.21.15.16:0/1406346154 shutdown_connections 2024-09-12T02:02:37.103 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.101+0000 7ffa94d6d640 1 --2- 172.21.15.16:0/1406346154 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7ffa780a2cb0 0x7ffa780a5170 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:37.103 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.101+0000 7ffa94d6d640 1 --2- 172.21.15.16:0/1406346154 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffa901078c0 0x7ffa9019f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:37.103 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.101+0000 7ffa94d6d640 1 --2- 172.21.15.16:0/1406346154 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa90104f90 0x7ffa90198310 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:37.103 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.101+0000 7ffa94d6d640 1 --2- 172.21.15.16:0/1406346154 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa90102660 0x7ffa90197dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:37.103 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.101+0000 7ffa94d6d640 1 -- 172.21.15.16:0/1406346154 >> 172.21.15.16:0/1406346154 conn(0x7ffa900fc1e0 msgr2=0x7ffa90102fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:02:37.103 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.101+0000 7ffa94d6d640 1 -- 172.21.15.16:0/1406346154 shutdown_connections 2024-09-12T02:02:37.103 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:37.101+0000 7ffa94d6d640 1 -- 172.21.15.16:0/1406346154 wait complete. 2024-09-12T02:02:37.668 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:02:37 smithi033 ceph-mon[35332]: pgmap v356: 97 pgs: 97 active+clean; 666 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 597 B/s wr, 7 op/s 2024-09-12T02:02:37.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:02:37 smithi016 ceph-mon[30947]: pgmap v356: 97 pgs: 97 active+clean; 666 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 597 B/s wr, 7 op/s 2024-09-12T02:02:37.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:02:37 smithi184 ceph-mon[35593]: pgmap v356: 97 pgs: 97 active+clean; 666 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 597 B/s wr, 7 op/s 2024-09-12T02:02:39.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:02:39 smithi033 ceph-mon[35332]: pgmap v357: 97 pgs: 97 active+clean; 666 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 511 B/s wr, 6 op/s 2024-09-12T02:02:39.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:02:39 smithi016 ceph-mon[30947]: pgmap v357: 97 pgs: 97 active+clean; 666 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 511 B/s wr, 6 op/s 2024-09-12T02:02:39.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:02:39 smithi184 ceph-mon[35593]: pgmap v357: 97 pgs: 97 active+clean; 666 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 511 B/s wr, 6 op/s 2024-09-12T02:02:41.668 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:02:41 smithi033 ceph-mon[35332]: pgmap v358: 97 pgs: 97 active+clean; 667 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-09-12T02:02:41.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:02:41 smithi016 ceph-mon[30947]: pgmap v358: 97 pgs: 97 active+clean; 667 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-09-12T02:02:41.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:02:41 smithi184 ceph-mon[35593]: pgmap v358: 97 pgs: 97 active+clean; 667 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-09-12T02:02:43.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:02:43 smithi033 ceph-mon[35332]: pgmap v359: 97 pgs: 97 active+clean; 667 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 853 B/s wr, 8 op/s 2024-09-12T02:02:43.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:02:43 smithi016 ceph-mon[30947]: pgmap v359: 97 pgs: 97 active+clean; 667 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 853 B/s wr, 8 op/s 2024-09-12T02:02:43.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:02:43 smithi184 ceph-mon[35593]: pgmap v359: 97 pgs: 97 active+clean; 667 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 853 B/s wr, 8 op/s 2024-09-12T02:02:44.357 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:02:44.736 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:02:45.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:45.244+0000 7f0290188640 1 -- 172.21.15.16:0/555134694 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0288106f40 msgr2=0x7f028810eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:45.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:45.244+0000 7f0290188640 1 --2- 172.21.15.16:0/555134694 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0288106f40 0x7f028810eac0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f0280009ca0 tx=0x7f028002a8c0 comp rx=0 tx=0).stop 2024-09-12T02:02:45.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:45.244+0000 7f0290188640 1 -- 172.21.15.16:0/555134694 shutdown_connections 2024-09-12T02:02:45.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:45.244+0000 7f0290188640 1 --2- 172.21.15.16:0/555134694 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0288106f40 0x7f028810eac0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:45.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:45.244+0000 7f0290188640 1 --2- 172.21.15.16:0/555134694 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f02881027c0 0x7f0288106810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:45.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:45.244+0000 7f0290188640 1 --2- 172.21.15.16:0/555134694 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0288101e10 0x7f02881021f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:45.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:45.244+0000 7f0290188640 1 -- 172.21.15.16:0/555134694 >> 172.21.15.16:0/555134694 conn(0x7f02880fd8d0 msgr2=0x7f02880ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:02:45.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:45.244+0000 7f0290188640 1 -- 172.21.15.16:0/555134694 shutdown_connections 2024-09-12T02:02:45.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:45.244+0000 7f0290188640 1 -- 172.21.15.16:0/555134694 wait complete. 2024-09-12T02:02:45.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:45.245+0000 7f0290188640 1 Processor -- start 2024-09-12T02:02:45.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:45.245+0000 7f0290188640 1 -- start start 2024-09-12T02:02:45.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:45.246+0000 7f0290188640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0288101e10 0x7f0288198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:45.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:45.246+0000 7f0290188640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f02881027c0 0x7f0288198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:45.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:45.246+0000 7f0290188640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0288106f40 0x7f028819ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:45.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:45.246+0000 7f0290188640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0288111fa0 con 0x7f0288101e10 2024-09-12T02:02:45.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:45.246+0000 7f0290188640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0288111e20 con 0x7f0288106f40 2024-09-12T02:02:45.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:45.246+0000 7f0290188640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f0288112120 con 0x7f02881027c0 2024-09-12T02:02:45.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:45.246+0000 7f028df24640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0288101e10 0x7f0288198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:45.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:45.246+0000 7f028df24640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0288101e10 0x7f0288198400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57798/0 (socket says 172.21.15.16:57798) 2024-09-12T02:02:45.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:45.246+0000 7f028df24640 1 -- 172.21.15.16:0/3887590797 learned_addr learned my addr 172.21.15.16:0/3887590797 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:02:45.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:45.246+0000 7f028e725640 1 --2- 172.21.15.16:0/3887590797 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0288106f40 0x7f028819ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:45.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:45.246+0000 7f028df24640 1 -- 172.21.15.16:0/3887590797 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f02881027c0 msgr2=0x7f0288198940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:45.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:45.246+0000 7f028df24640 1 --2- 172.21.15.16:0/3887590797 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f02881027c0 0x7f0288198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:45.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:45.246+0000 7f028df24640 1 -- 172.21.15.16:0/3887590797 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0288106f40 msgr2=0x7f028819ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:45.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:45.246+0000 7f028df24640 1 --2- 172.21.15.16:0/3887590797 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0288106f40 0x7f028819ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:45.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:45.246+0000 7f028df24640 1 -- 172.21.15.16:0/3887590797 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f028819d3b0 con 0x7f0288101e10 2024-09-12T02:02:45.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:45.247+0000 7f028df24640 1 --2- 172.21.15.16:0/3887590797 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0288101e10 0x7f0288198400 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f028400d770 tx=0x7f028400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:02:45.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:45.247+0000 7f027effd640 1 -- 172.21.15.16:0/3887590797 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f028401c070 con 0x7f0288101e10 2024-09-12T02:02:45.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:45.247+0000 7f027effd640 1 -- 172.21.15.16:0/3887590797 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0284005500 con 0x7f0288101e10 2024-09-12T02:02:45.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:45.247+0000 7f027effd640 1 -- 172.21.15.16:0/3887590797 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0284004a00 con 0x7f0288101e10 2024-09-12T02:02:45.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:45.247+0000 7f0290188640 1 -- 172.21.15.16:0/3887590797 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f028819d6a0 con 0x7f0288101e10 2024-09-12T02:02:45.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:45.247+0000 7f0290188640 1 -- 172.21.15.16:0/3887590797 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0288074420 con 0x7f0288101e10 2024-09-12T02:02:45.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:45.248+0000 7f0290188640 1 -- 172.21.15.16:0/3887590797 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02881032f0 con 0x7f0288101e10 2024-09-12T02:02:45.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:45.250+0000 7f027effd640 1 -- 172.21.15.16:0/3887590797 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f0284004ba0 con 0x7f0288101e10 2024-09-12T02:02:45.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:45.251+0000 7f027effd640 1 --2- 172.21.15.16:0/3887590797 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f02740a2c00 0x7f02740a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:45.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:45.252+0000 7f028d723640 1 --2- 172.21.15.16:0/3887590797 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f02740a2c00 0x7f02740a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:45.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:45.252+0000 7f027effd640 1 -- 172.21.15.16:0/3887590797 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f02840d7090 con 0x7f0288101e10 2024-09-12T02:02:45.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:45.252+0000 7f028d723640 1 --2- 172.21.15.16:0/3887590797 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f02740a2c00 0x7f02740a50c0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f02880ff7f0 tx=0x7f0278009590 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:02:45.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:45.256+0000 7f027effd640 1 -- 172.21.15.16:0/3887590797 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f02840a4090 con 0x7f0288101e10 2024-09-12T02:02:45.419 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:02:45 smithi016 ceph-mon[30947]: pgmap v360: 97 pgs: 97 active+clean; 667 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 853 B/s wr, 9 op/s 2024-09-12T02:02:45.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:45.556+0000 7f0290188640 1 -- 172.21.15.16:0/3887590797 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f028810a180 con 0x7f0288101e10 2024-09-12T02:02:45.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:45.557+0000 7f027effd640 1 -- 172.21.15.16:0/3887590797 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11911 (secure 0 0 0) 0x7f02840a3990 con 0x7f0288101e10 2024-09-12T02:02:45.559 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T02:02:45.559 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":25,"btime":"2024-09-12T02:01:37:192021+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3279123279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3279123279},{"type":"v1","addr":"172.21.15.16:6835","nonce":3279123279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4119452382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4119452382},{"type":"v1","addr":"172.21.15.16:6837","nonce":4119452382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/3263823165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3263823165},{"type":"v1","addr":"172.21.15.16:6839","nonce":3263823165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/1510290732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":1510290732},{"type":"v1","addr":"172.21.15.16:6841","nonce":1510290732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/279674603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":279674603},{"type":"v1","addr":"172.21.15.33:6833","nonce":279674603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3555728028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3555728028},{"type":"v1","addr":"172.21.15.33:6835","nonce":3555728028}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T01:59:56.117608+0000","modified":"2024-09-12T02:01:36.294760+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24482,"mds_1":24481,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6833/1563374106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1563374106},{"type":"v1","addr":"172.21.15.184:6833","nonce":1563374106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6837/1892833868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1892833868},{"type":"v1","addr":"172.21.15.33:6837","nonce":1892833868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/1457679686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1457679686},{"type":"v1","addr":"172.21.15.184:6835","nonce":1457679686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.184:6837/1793393087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":1793393087},{"type":"v1","addr":"172.21.15.184:6837","nonce":1793393087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.184:6839/4203347941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":4203347941},{"type":"v1","addr":"172.21.15.184:6839","nonce":4203347941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.33:6839/4136670204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":4136670204},{"type":"v1","addr":"172.21.15.33:6839","nonce":4136670204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24481,24482,24469]},"id":1}]} 2024-09-12T02:02:45.559 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 25 2024-09-12T02:02:45.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:45.560+0000 7f0290188640 1 -- 172.21.15.16:0/3887590797 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f02740a2c00 msgr2=0x7f02740a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:45.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:45.560+0000 7f0290188640 1 --2- 172.21.15.16:0/3887590797 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f02740a2c00 0x7f02740a50c0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f02880ff7f0 tx=0x7f0278009590 comp rx=0 tx=0).stop 2024-09-12T02:02:45.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:45.560+0000 7f0290188640 1 -- 172.21.15.16:0/3887590797 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0288101e10 msgr2=0x7f0288198400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:45.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:45.560+0000 7f0290188640 1 --2- 172.21.15.16:0/3887590797 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0288101e10 0x7f0288198400 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f028400d770 tx=0x7f028400dc30 comp rx=0 tx=0).stop 2024-09-12T02:02:45.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:45.560+0000 7f0290188640 1 -- 172.21.15.16:0/3887590797 shutdown_connections 2024-09-12T02:02:45.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:45.560+0000 7f0290188640 1 --2- 172.21.15.16:0/3887590797 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f02740a2c00 0x7f02740a50c0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:45.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:45.560+0000 7f0290188640 1 --2- 172.21.15.16:0/3887590797 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0288106f40 0x7f028819ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:45.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:45.560+0000 7f0290188640 1 --2- 172.21.15.16:0/3887590797 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f02881027c0 0x7f0288198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:45.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:45.560+0000 7f0290188640 1 --2- 172.21.15.16:0/3887590797 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0288101e10 0x7f0288198400 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:45.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:45.560+0000 7f0290188640 1 -- 172.21.15.16:0/3887590797 >> 172.21.15.16:0/3887590797 conn(0x7f02880fd8d0 msgr2=0x7f02880fec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:02:45.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:45.560+0000 7f0290188640 1 -- 172.21.15.16:0/3887590797 shutdown_connections 2024-09-12T02:02:45.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:45.560+0000 7f0290188640 1 -- 172.21.15.16:0/3887590797 wait complete. 2024-09-12T02:02:45.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:02:45 smithi184 ceph-mon[35593]: pgmap v360: 97 pgs: 97 active+clean; 667 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 853 B/s wr, 9 op/s 2024-09-12T02:02:45.918 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:02:45 smithi033 ceph-mon[35332]: pgmap v360: 97 pgs: 97 active+clean; 667 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 853 B/s wr, 9 op/s 2024-09-12T02:02:46.250 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24482 quiesce db --set-id 72b7f958 --release --await' 2024-09-12T02:02:46.250 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24482 quiesce db --set-id 72b7f958 --release --await 2024-09-12T02:02:46.595 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:02:46.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:02:46 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/3887590797' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:02:46.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:02:46 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/3887590797' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:02:46.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:02:46 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/3887590797' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:02:47.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.047+0000 7f879c9bd640 1 -- 172.21.15.16:0/2648153330 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8798101df0 msgr2=0x7f87981021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:47.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.047+0000 7f879c9bd640 1 --2- 172.21.15.16:0/2648153330 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8798101df0 0x7f87981021d0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f8788009900 tx=0x7f878802a410 comp rx=0 tx=0).stop 2024-09-12T02:02:47.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.048+0000 7f879c9bd640 1 -- 172.21.15.16:0/2648153330 shutdown_connections 2024-09-12T02:02:47.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.048+0000 7f879c9bd640 1 --2- 172.21.15.16:0/2648153330 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8798106f20 0x7f879810eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:47.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.048+0000 7f879c9bd640 1 --2- 172.21.15.16:0/2648153330 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f87981027a0 0x7f87981067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:47.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.048+0000 7f879c9bd640 1 --2- 172.21.15.16:0/2648153330 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8798101df0 0x7f87981021d0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:47.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.048+0000 7f879c9bd640 1 -- 172.21.15.16:0/2648153330 >> 172.21.15.16:0/2648153330 conn(0x7f87980fd8f0 msgr2=0x7f87980ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:02:47.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.048+0000 7f879c9bd640 1 -- 172.21.15.16:0/2648153330 shutdown_connections 2024-09-12T02:02:47.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.048+0000 7f879c9bd640 1 -- 172.21.15.16:0/2648153330 wait complete. 2024-09-12T02:02:47.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.049+0000 7f879c9bd640 1 Processor -- start 2024-09-12T02:02:47.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.049+0000 7f879c9bd640 1 -- start start 2024-09-12T02:02:47.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.049+0000 7f879c9bd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8798101df0 0x7f8798198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:47.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.049+0000 7f879c9bd640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f87981027a0 0x7f8798198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:47.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.049+0000 7f879c9bd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8798106f20 0x7f879819cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:47.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.049+0000 7f879c9bd640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8798111fd0 con 0x7f8798106f20 2024-09-12T02:02:47.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.049+0000 7f879c9bd640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8798111e50 con 0x7f8798101df0 2024-09-12T02:02:47.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.049+0000 7f879c9bd640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8798112150 con 0x7f87981027a0 2024-09-12T02:02:47.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.049+0000 7f8796d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8798106f20 0x7f879819cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:47.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.049+0000 7f8796d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8798106f20 0x7f879819cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57814/0 (socket says 172.21.15.16:57814) 2024-09-12T02:02:47.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.049+0000 7f8796d9d640 1 -- 172.21.15.16:0/2344316799 learned_addr learned my addr 172.21.15.16:0/2344316799 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:02:47.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.049+0000 7f8795d9b640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f87981027a0 0x7f8798198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:47.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.049+0000 7f879659c640 1 --2- 172.21.15.16:0/2344316799 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8798101df0 0x7f8798198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:47.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.050+0000 7f8796d9d640 1 -- 172.21.15.16:0/2344316799 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f87981027a0 msgr2=0x7f8798198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:47.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.050+0000 7f8796d9d640 1 --2- 172.21.15.16:0/2344316799 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f87981027a0 0x7f8798198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:47.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.050+0000 7f8796d9d640 1 -- 172.21.15.16:0/2344316799 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8798101df0 msgr2=0x7f8798198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:47.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.050+0000 7f8796d9d640 1 --2- 172.21.15.16:0/2344316799 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8798101df0 0x7f8798198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:47.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.050+0000 7f8796d9d640 1 -- 172.21.15.16:0/2344316799 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f879819d3e0 con 0x7f8798106f20 2024-09-12T02:02:47.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.050+0000 7f879659c640 1 --2- 172.21.15.16:0/2344316799 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8798101df0 0x7f8798198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:02:47.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.050+0000 7f8796d9d640 1 --2- 172.21.15.16:0/2344316799 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8798106f20 0x7f879819cd00 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f8790009fd0 tx=0x7f879000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:02:47.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.050+0000 7f87877fe640 1 -- 172.21.15.16:0/2344316799 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8790017070 con 0x7f8798106f20 2024-09-12T02:02:47.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.051+0000 7f879c9bd640 1 -- 172.21.15.16:0/2344316799 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f879819d6d0 con 0x7f8798106f20 2024-09-12T02:02:47.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.051+0000 7f87877fe640 1 -- 172.21.15.16:0/2344316799 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8790003e50 con 0x7f8798106f20 2024-09-12T02:02:47.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.051+0000 7f87877fe640 1 -- 172.21.15.16:0/2344316799 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87900106c0 con 0x7f8798106f20 2024-09-12T02:02:47.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.051+0000 7f879c9bd640 1 -- 172.21.15.16:0/2344316799 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87981a4fb0 con 0x7f8798106f20 2024-09-12T02:02:47.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.053+0000 7f87877fe640 1 -- 172.21.15.16:0/2344316799 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f8790003970 con 0x7f8798106f20 2024-09-12T02:02:47.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.054+0000 7f879ed96740 1 Processor -- start 2024-09-12T02:02:47.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.054+0000 7f879ed96740 1 -- start start 2024-09-12T02:02:47.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.054+0000 7f87877fe640 1 --2- 172.21.15.16:0/2344316799 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f87800a2c40 0x7f87800a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:47.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.055+0000 7f879ed96740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b320998060 0x55b320998440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:47.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.055+0000 7f879ed96740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b320998980 0x55b3209a2d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:47.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.055+0000 7f879ed96740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b3209a3240 0x55b3209a5630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:47.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.055+0000 7f879ed96740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55b3208fd620 con 0x55b320998980 2024-09-12T02:02:47.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.055+0000 7f879ed96740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55b3208a4e50 con 0x55b3209a3240 2024-09-12T02:02:47.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.055+0000 7f87877fe640 1 -- 172.21.15.16:0/2344316799 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f87900cec50 con 0x7f8798106f20 2024-09-12T02:02:47.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.055+0000 7f879ed96740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55b3208f95e0 con 0x55b320998060 2024-09-12T02:02:47.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.055+0000 7f879659c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b320998060 0x55b320998440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:47.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.055+0000 7f879659c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b320998060 0x55b320998440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:41608/0 (socket says 172.21.15.16:41608) 2024-09-12T02:02:47.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.055+0000 7f879659c640 1 -- 172.21.15.16:0/1121345005 learned_addr learned my addr 172.21.15.16:0/1121345005 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:02:47.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.055+0000 7f8795d9b640 1 --2- 172.21.15.16:0/2344316799 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f87800a2c40 0x7f87800a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:47.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.055+0000 7f879659c640 1 -- 172.21.15.16:0/1121345005 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b3209a3240 msgr2=0x55b3209a5630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:47.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.055+0000 7f879659c640 1 --2- 172.21.15.16:0/1121345005 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b3209a3240 0x55b3209a5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:47.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.055+0000 7f879659c640 1 -- 172.21.15.16:0/1121345005 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b320998980 msgr2=0x55b3209a2d00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-12T02:02:47.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.055+0000 7f879659c640 1 --2- 172.21.15.16:0/1121345005 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b320998980 0x55b3209a2d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:47.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.055+0000 7f879659c640 1 -- 172.21.15.16:0/1121345005 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b320895d40 con 0x55b320998060 2024-09-12T02:02:47.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.056+0000 7f879659c640 1 --2- 172.21.15.16:0/1121345005 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b320998060 0x55b320998440 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f8788005dc0 tx=0x7f87880040c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6b12666a5dbbd05c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:02:47.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.056+0000 7f8784ff9640 1 -- 172.21.15.16:0/1121345005 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8788035030 con 0x55b320998060 2024-09-12T02:02:47.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.056+0000 7f8795d9b640 1 --2- 172.21.15.16:0/2344316799 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f87800a2c40 0x7f87800a5100 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f87980fdd80 tx=0x7f878c00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:02:47.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.056+0000 7f8784ff9640 1 -- 172.21.15.16:0/1121345005 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87880044b0 con 0x55b320998060 2024-09-12T02:02:47.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.056+0000 7f8784ff9640 1 -- 172.21.15.16:0/1121345005 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87880315b0 con 0x55b320998060 2024-09-12T02:02:47.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.057+0000 7f879ed96740 1 -- 172.21.15.16:0/1121345005 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b320998060 msgr2=0x55b320998440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:47.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.057+0000 7f879ed96740 1 --2- 172.21.15.16:0/1121345005 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b320998060 0x55b320998440 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f8788005dc0 tx=0x7f87880040c0 comp rx=0 tx=0).stop 2024-09-12T02:02:47.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.057+0000 7f879ed96740 1 -- 172.21.15.16:0/1121345005 shutdown_connections 2024-09-12T02:02:47.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.057+0000 7f879ed96740 1 --2- 172.21.15.16:0/1121345005 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b3209a3240 0x55b3209a5630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:47.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.057+0000 7f879ed96740 1 --2- 172.21.15.16:0/1121345005 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b320998980 0x55b3209a2d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:47.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.057+0000 7f879ed96740 1 --2- 172.21.15.16:0/1121345005 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b320998060 0x55b320998440 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:47.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.057+0000 7f879ed96740 1 -- 172.21.15.16:0/1121345005 >> 172.21.15.16:0/1121345005 conn(0x55b3209945f0 msgr2=0x55b3209949d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:02:47.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.057+0000 7f879ed96740 1 -- 172.21.15.16:0/1121345005 shutdown_connections 2024-09-12T02:02:47.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.057+0000 7f879ed96740 1 -- 172.21.15.16:0/1121345005 wait complete. 2024-09-12T02:02:47.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.058+0000 7f879ed96740 1 -- start start 2024-09-12T02:02:47.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.058+0000 7f879ed96740 1 Processor -- start 2024-09-12T02:02:47.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.058+0000 7f879ed96740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b320998060 0x55b3209a0120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:47.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.058+0000 7f879ed96740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b320998980 0x55b3209a0660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:47.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.058+0000 7f879ed96740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b3209a3240 0x55b3209a0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:47.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.058+0000 7f879ed96740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55b3209a5e10 con 0x55b320998980 2024-09-12T02:02:47.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.058+0000 7f879ed96740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55b320885f10 con 0x55b320998060 2024-09-12T02:02:47.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.058+0000 7f879ed96740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55b3209a5f90 con 0x55b3209a3240 2024-09-12T02:02:47.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.058+0000 7f879ed96740 10 client.0 _ll_register_callbacks cb 0x55b3208a9db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:02:47.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.058+0000 7f8795d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b320998980 0x55b3209a0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:47.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.058+0000 7f8795d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b320998980 0x55b3209a0660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57816/0 (socket says 172.21.15.16:57816) 2024-09-12T02:02:47.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.058+0000 7f8795d9b640 1 -- 172.21.15.16:0/4282873197 learned_addr learned my addr 172.21.15.16:0/4282873197 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:02:47.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.059+0000 7f8796d9d640 1 --2- 172.21.15.16:0/4282873197 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b3209a3240 0x55b3209a0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:47.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.059+0000 7f879659c640 1 --2- 172.21.15.16:0/4282873197 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b320998060 0x55b3209a0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:47.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.059+0000 7f8796d9d640 1 -- 172.21.15.16:0/4282873197 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b320998060 msgr2=0x55b3209a0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:47.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.059+0000 7f8796d9d640 1 --2- 172.21.15.16:0/4282873197 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b320998060 0x55b3209a0120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:47.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.059+0000 7f8796d9d640 1 -- 172.21.15.16:0/4282873197 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b320998980 msgr2=0x55b3209a0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:47.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.059+0000 7f8796d9d640 1 --2- 172.21.15.16:0/4282873197 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b320998980 0x55b3209a0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:47.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.059+0000 7f8796d9d640 1 -- 172.21.15.16:0/4282873197 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b32060c580 con 0x55b3209a3240 2024-09-12T02:02:47.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.059+0000 7f8795d9b640 1 --2- 172.21.15.16:0/4282873197 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b320998980 0x55b3209a0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:02:47.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.059+0000 7f879659c640 1 --2- 172.21.15.16:0/4282873197 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b320998060 0x55b3209a0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:02:47.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.059+0000 7f8796d9d640 1 --2- 172.21.15.16:0/4282873197 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b3209a3240 0x55b3209a0ba0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f87900cc840 tx=0x7f8790004940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d7b2a63e44b9774f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:02:47.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.059+0000 7f877dffb640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-09-12T02:02:47.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.059+0000 7f877dffb640 1 -- 172.21.15.16:0/4282873197 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8790020040 con 0x55b3209a3240 2024-09-12T02:02:47.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.059+0000 7f877dffb640 1 -- 172.21.15.16:0/4282873197 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f8790097bc0 con 0x55b3209a3240 2024-09-12T02:02:47.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.060+0000 7f879ed96740 1 -- 172.21.15.16:0/4282873197 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f87880044b0 con 0x55b3209a3240 2024-09-12T02:02:47.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.060+0000 7f877dffb640 1 -- 172.21.15.16:0/4282873197 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f879009bc50 con 0x55b3209a3240 2024-09-12T02:02:47.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.060+0000 7f879ed96740 10 client.25162 fetch_fsmap learned FSMap version 25 2024-09-12T02:02:47.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.060+0000 7f879ed96740 1 -- 172.21.15.16:0/4282873197 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b3206d6960 con 0x55b3209a3240 2024-09-12T02:02:47.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.060+0000 7f877dffb640 1 -- 172.21.15.16:0/4282873197 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87880044b0 con 0x55b3209a3240 2024-09-12T02:02:47.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.060+0000 7f877dffb640 1 -- 172.21.15.16:0/4282873197 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f879009a9a0 con 0x55b3209a3240 2024-09-12T02:02:47.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.061+0000 7f879ed96740 10 client.25162 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:02:47.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.061+0000 7f879ed96740 10 client.25162 resolve_mds: validated gid 24482 aka daemon mds.b 2024-09-12T02:02:47.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.061+0000 7f879ed96740 20 client.25162 populate_metadata read hostname 'smithi016' 2024-09-12T02:02:47.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.061+0000 7f879ed96740 1 --2- 172.21.15.16:0/4282873197 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55b3209bb610 0x55b3209bda20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:47.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.061+0000 7f879ed96740 4 client.25162 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:02:47.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.061+0000 7f879ed96740 1 -- 172.21.15.16:0/4282873197 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b32090ab90 con 0x55b3209bb610 2024-09-12T02:02:47.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.061+0000 7f879659c640 1 --2- 172.21.15.16:0/4282873197 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55b3209bb610 0x55b3209bda20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:47.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.062+0000 7f879659c640 1 --2- 172.21.15.16:0/4282873197 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55b3209bb610 0x55b3209bda20 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6d19ac9bacf372c server_cookie=999eed08bb40545 in_seq=0 out_seq=0 2024-09-12T02:02:47.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.062+0000 7f877dffb640 10 client.25162 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:02:47.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.062+0000 7f877dffb640 1 -- 172.21.15.16:0/4282873197 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b32090ab90 con 0x55b3209bb610 2024-09-12T02:02:47.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.062+0000 7f877dffb640 10 client.25162 handle_command_reply: tid=0 2024-09-12T02:02:47.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.062+0000 7f879ed96740 1 client.25162 shutdown 2024-09-12T02:02:47.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.063+0000 7f879ed96740 1 -- 172.21.15.16:0/4282873197 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b3209a3240 msgr2=0x55b3209a0ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:47.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.063+0000 7f879ed96740 1 --2- 172.21.15.16:0/4282873197 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b3209a3240 0x55b3209a0ba0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f87900cc840 tx=0x7f8790004940 comp rx=0 tx=0).stop 2024-09-12T02:02:47.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.063+0000 7f879ed96740 1 -- 172.21.15.16:0/4282873197 shutdown_connections 2024-09-12T02:02:47.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.063+0000 7f879ed96740 1 --2- 172.21.15.16:0/4282873197 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55b3209bb610 0x55b3209bda20 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:47.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.063+0000 7f877dffb640 0 client.25162 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:02:47.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.063+0000 7f879ed96740 1 --2- 172.21.15.16:0/4282873197 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b3209a3240 0x55b3209a0ba0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:47.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.063+0000 7f879ed96740 1 --2- 172.21.15.16:0/4282873197 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b320998980 0x55b3209a0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:47.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.063+0000 7f879ed96740 1 --2- 172.21.15.16:0/4282873197 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b320998060 0x55b3209a0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:47.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.063+0000 7f879ed96740 1 -- 172.21.15.16:0/4282873197 >> 172.21.15.16:0/4282873197 conn(0x55b3209945f0 msgr2=0x55b32099a320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:02:47.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.063+0000 7f879ed96740 1 -- 172.21.15.16:0/4282873197 shutdown_connections 2024-09-12T02:02:47.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.063+0000 7f879ed96740 1 -- 172.21.15.16:0/4282873197 wait complete. 2024-09-12T02:02:47.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.063+0000 7f879ed96740 20 client.25162 trim_cache size 0 max 16384 2024-09-12T02:02:47.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.081+0000 7f879ed96740 1 Processor -- start 2024-09-12T02:02:47.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.081+0000 7f879ed96740 1 -- start start 2024-09-12T02:02:47.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.081+0000 7f879ed96740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b320998060 0x55b320993640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:47.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.081+0000 7f879ed96740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b320998980 0x55b3209c32f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:47.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.081+0000 7f879ed96740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b3209a3240 0x55b3209c3830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:47.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.081+0000 7f879ed96740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f87981ada70 con 0x55b320998060 2024-09-12T02:02:47.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.081+0000 7f879ed96740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f87981ab340 con 0x55b320998980 2024-09-12T02:02:47.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.081+0000 7f879ed96740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f87981a9190 con 0x55b3209a3240 2024-09-12T02:02:47.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.081+0000 7f879659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b320998060 0x55b320993640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:47.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.081+0000 7f879659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b320998060 0x55b320993640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57822/0 (socket says 172.21.15.16:57822) 2024-09-12T02:02:47.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.081+0000 7f879659c640 1 -- 172.21.15.16:0/1969748729 learned_addr learned my addr 172.21.15.16:0/1969748729 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:02:47.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.081+0000 7f8796d9d640 1 --2- 172.21.15.16:0/1969748729 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b3209a3240 0x55b3209c3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:47.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.082+0000 7f8795d9b640 1 --2- 172.21.15.16:0/1969748729 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b320998980 0x55b3209c32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:47.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.082+0000 7f879659c640 1 -- 172.21.15.16:0/1969748729 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b3209a3240 msgr2=0x55b3209c3830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:47.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.082+0000 7f879659c640 1 --2- 172.21.15.16:0/1969748729 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b3209a3240 0x55b3209c3830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:47.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.082+0000 7f879659c640 1 -- 172.21.15.16:0/1969748729 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b320998980 msgr2=0x55b3209c32f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:47.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.082+0000 7f879659c640 1 --2- 172.21.15.16:0/1969748729 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b320998980 0x55b3209c32f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:47.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.082+0000 7f879659c640 1 -- 172.21.15.16:0/1969748729 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b320992bb0 con 0x55b320998060 2024-09-12T02:02:47.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.082+0000 7f8796d9d640 1 --2- 172.21.15.16:0/1969748729 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b3209a3240 0x55b3209c3830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:02:47.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.082+0000 7f8795d9b640 1 --2- 172.21.15.16:0/1969748729 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b320998980 0x55b3209c32f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:02:47.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.082+0000 7f879659c640 1 --2- 172.21.15.16:0/1969748729 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b320998060 0x55b320993640 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f8788009900 tx=0x7f878802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3f449d1066715c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:02:47.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.082+0000 7f877ffff640 1 -- 172.21.15.16:0/1969748729 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8788035040 con 0x55b320998060 2024-09-12T02:02:47.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.082+0000 7f877ffff640 1 -- 172.21.15.16:0/1969748729 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8788031c90 con 0x55b320998060 2024-09-12T02:02:47.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.083+0000 7f877ffff640 1 -- 172.21.15.16:0/1969748729 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8788003ef0 con 0x55b320998060 2024-09-12T02:02:47.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.083+0000 7f879ed96740 1 -- 172.21.15.16:0/1969748729 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b320998060 msgr2=0x55b320993640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:47.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.083+0000 7f879ed96740 1 --2- 172.21.15.16:0/1969748729 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b320998060 0x55b320993640 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f8788009900 tx=0x7f878802a410 comp rx=0 tx=0).stop 2024-09-12T02:02:47.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.083+0000 7f879ed96740 1 -- 172.21.15.16:0/1969748729 shutdown_connections 2024-09-12T02:02:47.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.083+0000 7f879ed96740 1 --2- 172.21.15.16:0/1969748729 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b3209a3240 0x55b3209c3830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:47.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.083+0000 7f879ed96740 1 --2- 172.21.15.16:0/1969748729 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b320998980 0x55b3209c32f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:47.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.083+0000 7f879ed96740 1 --2- 172.21.15.16:0/1969748729 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b320998060 0x55b320993640 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:47.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.083+0000 7f879ed96740 1 -- 172.21.15.16:0/1969748729 >> 172.21.15.16:0/1969748729 conn(0x55b3209945f0 msgr2=0x55b320992f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:02:47.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.084+0000 7f879ed96740 1 -- 172.21.15.16:0/1969748729 shutdown_connections 2024-09-12T02:02:47.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.084+0000 7f879ed96740 1 -- 172.21.15.16:0/1969748729 wait complete. 2024-09-12T02:02:47.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.084+0000 7f879ed96740 1 -- start start 2024-09-12T02:02:47.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.084+0000 7f879ed96740 1 Processor -- start 2024-09-12T02:02:47.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.084+0000 7f879ed96740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b320998060 0x55b3209b0730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:47.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.084+0000 7f879ed96740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b320998980 0x55b3209b0c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:47.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.084+0000 7f879ed96740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b3209a3240 0x55b3209b11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:47.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.084+0000 7f879ed96740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f87981a9800 con 0x55b320998060 2024-09-12T02:02:47.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.085+0000 7f879ed96740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f87981aac70 con 0x55b3209a3240 2024-09-12T02:02:47.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.085+0000 7f879ed96740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f87981a9ed0 con 0x55b320998980 2024-09-12T02:02:47.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.085+0000 7f879ed96740 10 client.0 _ll_register_callbacks cb 0x7f879009b320 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:02:47.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.085+0000 7f879659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b320998060 0x55b3209b0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:47.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.085+0000 7f879659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b320998060 0x55b3209b0730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57838/0 (socket says 172.21.15.16:57838) 2024-09-12T02:02:47.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.085+0000 7f879659c640 1 -- 172.21.15.16:0/3190452583 learned_addr learned my addr 172.21.15.16:0/3190452583 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:02:47.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.085+0000 7f8795d9b640 1 --2- 172.21.15.16:0/3190452583 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b320998980 0x55b3209b0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:47.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.085+0000 7f879659c640 1 -- 172.21.15.16:0/3190452583 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b320998980 msgr2=0x55b3209b0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:47.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.085+0000 7f879659c640 1 --2- 172.21.15.16:0/3190452583 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b320998980 0x55b3209b0c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:47.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.085+0000 7f8796d9d640 1 --2- 172.21.15.16:0/3190452583 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b3209a3240 0x55b3209b11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:47.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.085+0000 7f879659c640 1 -- 172.21.15.16:0/3190452583 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b3209a3240 msgr2=0x55b3209b11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:47.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.085+0000 7f879659c640 1 --2- 172.21.15.16:0/3190452583 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b3209a3240 0x55b3209b11b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:47.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.085+0000 7f879659c640 1 -- 172.21.15.16:0/3190452583 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b320a262d0 con 0x55b320998060 2024-09-12T02:02:47.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.085+0000 7f879659c640 1 --2- 172.21.15.16:0/3190452583 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b320998060 0x55b3209b0730 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f8788009850 tx=0x7f8788031e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=842bd923a4cc41e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:02:47.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.085+0000 7f877dffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:02:47.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.086+0000 7f877dffb640 1 -- 172.21.15.16:0/3190452583 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f878805d060 con 0x55b320998060 2024-09-12T02:02:47.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.086+0000 7f877dffb640 1 -- 172.21.15.16:0/3190452583 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f87880035f0 con 0x55b320998060 2024-09-12T02:02:47.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.086+0000 7f879ed96740 1 -- 172.21.15.16:0/3190452583 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8788031c90 con 0x55b320998060 2024-09-12T02:02:47.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.086+0000 7f877dffb640 1 -- 172.21.15.16:0/3190452583 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f87880037f0 con 0x55b320998060 2024-09-12T02:02:47.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.086+0000 7f879ed96740 10 client.15552 fetch_fsmap learned FSMap version 25 2024-09-12T02:02:47.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.086+0000 7f879ed96740 1 -- 172.21.15.16:0/3190452583 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b320a27550 con 0x55b320998060 2024-09-12T02:02:47.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.086+0000 7f877dffb640 1 -- 172.21.15.16:0/3190452583 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8788031c90 con 0x55b320998060 2024-09-12T02:02:47.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.086+0000 7f877dffb640 1 -- 172.21.15.16:0/3190452583 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f8788062070 con 0x55b320998060 2024-09-12T02:02:47.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.086+0000 7f879ed96740 10 client.15552 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:02:47.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.086+0000 7f879ed96740 10 client.15552 resolve_mds: validated gid 24482 aka daemon mds.b 2024-09-12T02:02:47.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.086+0000 7f879ed96740 20 client.15552 populate_metadata read hostname 'smithi016' 2024-09-12T02:02:47.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.086+0000 7f879ed96740 1 --2- 172.21.15.16:0/3190452583 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55b3209bb610 0x55b320a29730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:47.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.086+0000 7f879ed96740 4 client.15552 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "72b7f958", "release": true, "await": true}] 2024-09-12T02:02:47.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.086+0000 7f879ed96740 1 -- 172.21.15.16:0/3190452583 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "quiesce db", "set_id": "72b7f958", "release": true, "await": true}) -- 0x55b3209050a0 con 0x55b3209bb610 2024-09-12T02:02:47.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.087+0000 7f8795d9b640 1 --2- 172.21.15.16:0/3190452583 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55b3209bb610 0x55b320a29730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:47.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.088+0000 7f8795d9b640 1 --2- 172.21.15.16:0/3190452583 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55b3209bb610 0x55b320a29730 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=256db02d72dcdf58 server_cookie=ae98154acade48f6 in_seq=0 out_seq=0 2024-09-12T02:02:47.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.088+0000 7f877dffb640 10 client.15552 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:02:47.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.089+0000 7f877dffb640 1 -- 172.21.15.16:0/3190452583 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x55b3209050a0 con 0x55b3209bb610 2024-09-12T02:02:47.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.089+0000 7f877dffb640 10 client.15552 handle_command_reply: tid=0 2024-09-12T02:02:47.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.089+0000 7f879ed96740 1 client.15552 shutdown 2024-09-12T02:02:47.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.089+0000 7f879ed96740 1 -- 172.21.15.16:0/3190452583 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b320998060 msgr2=0x55b3209b0730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:47.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.089+0000 7f879ed96740 1 --2- 172.21.15.16:0/3190452583 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b320998060 0x55b3209b0730 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f8788009850 tx=0x7f8788031e40 comp rx=0 tx=0).stop 2024-09-12T02:02:47.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.089+0000 7f879ed96740 1 -- 172.21.15.16:0/3190452583 shutdown_connections 2024-09-12T02:02:47.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.089+0000 7f879ed96740 1 --2- 172.21.15.16:0/3190452583 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55b3209bb610 0x55b320a29730 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:47.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.089+0000 7f877dffb640 0 client.15552 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:02:47.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.089+0000 7f879ed96740 1 --2- 172.21.15.16:0/3190452583 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b3209a3240 0x55b3209b11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:47.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.089+0000 7f879ed96740 1 --2- 172.21.15.16:0/3190452583 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b320998980 0x55b3209b0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:47.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.089+0000 7f879ed96740 1 --2- 172.21.15.16:0/3190452583 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b320998060 0x55b3209b0730 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:47.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.089+0000 7f879ed96740 1 -- 172.21.15.16:0/3190452583 >> 172.21.15.16:0/3190452583 conn(0x55b3209945f0 msgr2=0x55b32099a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:02:47.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.090+0000 7f879ed96740 1 -- 172.21.15.16:0/3190452583 shutdown_connections 2024-09-12T02:02:47.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.090+0000 7f879ed96740 1 -- 172.21.15.16:0/3190452583 wait complete. 2024-09-12T02:02:47.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.090+0000 7f879ed96740 20 client.15552 trim_cache size 0 max 16384 2024-09-12T02:02:47.093 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T02:02:47.093 INFO:teuthology.orchestra.run.smithi016.stdout: "epoch": 25, 2024-09-12T02:02:47.093 INFO:teuthology.orchestra.run.smithi016.stdout: "leader": 24482, 2024-09-12T02:02:47.093 INFO:teuthology.orchestra.run.smithi016.stdout: "set_version": 4, 2024-09-12T02:02:47.093 INFO:teuthology.orchestra.run.smithi016.stdout: "sets": { 2024-09-12T02:02:47.093 INFO:teuthology.orchestra.run.smithi016.stdout: "72b7f958": { 2024-09-12T02:02:47.094 INFO:teuthology.orchestra.run.smithi016.stdout: "version": 4, 2024-09-12T02:02:47.094 INFO:teuthology.orchestra.run.smithi016.stdout: "age_ref": 0.0, 2024-09-12T02:02:47.094 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-12T02:02:47.094 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RELEASED", 2024-09-12T02:02:47.094 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-12T02:02:47.094 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:02:47.094 INFO:teuthology.orchestra.run.smithi016.stdout: "timeout": 90.0, 2024-09-12T02:02:47.094 INFO:teuthology.orchestra.run.smithi016.stdout: "expiration": 144.5, 2024-09-12T02:02:47.094 INFO:teuthology.orchestra.run.smithi016.stdout: "members": { 2024-09-12T02:02:47.094 INFO:teuthology.orchestra.run.smithi016.stdout: "file:/": { 2024-09-12T02:02:47.094 INFO:teuthology.orchestra.run.smithi016.stdout: "excluded": false, 2024-09-12T02:02:47.094 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-12T02:02:47.094 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RELEASED", 2024-09-12T02:02:47.094 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-12T02:02:47.094 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:02:47.095 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:02:47.095 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:02:47.095 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:02:47.095 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:02:47.095 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T02:02:47.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.091+0000 7f879c9bd640 1 -- 172.21.15.16:0/2344316799 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f87800a2c40 msgr2=0x7f87800a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:47.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.091+0000 7f879c9bd640 1 --2- 172.21.15.16:0/2344316799 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f87800a2c40 0x7f87800a5100 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f87980fdd80 tx=0x7f878c00b390 comp rx=0 tx=0).stop 2024-09-12T02:02:47.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.091+0000 7f879c9bd640 1 -- 172.21.15.16:0/2344316799 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8798106f20 msgr2=0x7f879819cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:47.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.091+0000 7f879c9bd640 1 --2- 172.21.15.16:0/2344316799 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8798106f20 0x7f879819cd00 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f8790009fd0 tx=0x7f879000b260 comp rx=0 tx=0).stop 2024-09-12T02:02:47.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.091+0000 7f879c9bd640 1 -- 172.21.15.16:0/2344316799 shutdown_connections 2024-09-12T02:02:47.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.091+0000 7f879c9bd640 1 --2- 172.21.15.16:0/2344316799 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f87800a2c40 0x7f87800a5100 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:47.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.091+0000 7f879c9bd640 1 --2- 172.21.15.16:0/2344316799 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8798106f20 0x7f879819cd00 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:47.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.091+0000 7f879c9bd640 1 --2- 172.21.15.16:0/2344316799 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f87981027a0 0x7f8798198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:47.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.091+0000 7f879c9bd640 1 --2- 172.21.15.16:0/2344316799 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8798101df0 0x7f8798198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:47.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.091+0000 7f879c9bd640 1 -- 172.21.15.16:0/2344316799 >> 172.21.15.16:0/2344316799 conn(0x7f87980fd8f0 msgr2=0x7f87980fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:02:47.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.091+0000 7f879c9bd640 1 -- 172.21.15.16:0/2344316799 shutdown_connections 2024-09-12T02:02:47.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:02:47.091+0000 7f879c9bd640 1 -- 172.21.15.16:0/2344316799 wait complete. 2024-09-12T02:02:47.427 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:02:47 smithi016 ceph-mon[30947]: pgmap v361: 97 pgs: 97 active+clean; 667 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 853 B/s wr, 8 op/s 2024-09-12T02:02:47.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:02:47 smithi184 ceph-mon[35593]: pgmap v361: 97 pgs: 97 active+clean; 667 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 853 B/s wr, 8 op/s 2024-09-12T02:02:47.880 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '72b7f958', total seconds elapsed: 28.0 2024-09-12T02:02:47.918 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:02:47 smithi033 ceph-mon[35332]: pgmap v361: 97 pgs: 97 active+clean; 667 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 853 B/s wr, 8 op/s 2024-09-12T02:02:49.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:02:49 smithi016 ceph-mon[30947]: pgmap v362: 97 pgs: 97 active+clean; 667 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 853 B/s wr, 6 op/s 2024-09-12T02:02:49.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:02:49 smithi184 ceph-mon[35593]: pgmap v362: 97 pgs: 97 active+clean; 667 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 853 B/s wr, 6 op/s 2024-09-12T02:02:49.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:02:49 smithi033 ceph-mon[35332]: pgmap v362: 97 pgs: 97 active+clean; 667 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 853 B/s wr, 6 op/s 2024-09-12T02:02:50.236 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.234+0000 7f418b2ac640 1 -- 172.21.15.16:0/596160483 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4184100a40 msgr2=0x7f4184100e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:50.236 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.234+0000 7f418b2ac640 1 --2- 172.21.15.16:0/596160483 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4184100a40 0x7f4184100e40 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f4180009900 tx=0x7f418002e820 comp rx=0 tx=0).stop 2024-09-12T02:02:50.236 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.235+0000 7f418b2ac640 1 -- 172.21.15.16:0/596160483 shutdown_connections 2024-09-12T02:02:50.236 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.235+0000 7f418b2ac640 1 --2- 172.21.15.16:0/596160483 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f41841025f0 0x7f4184108e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:50.236 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.235+0000 7f418b2ac640 1 --2- 172.21.15.16:0/596160483 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4184101c30 0x7f41841020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:50.236 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.235+0000 7f418b2ac640 1 --2- 172.21.15.16:0/596160483 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4184100a40 0x7f4184100e40 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:50.236 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.235+0000 7f418b2ac640 1 -- 172.21.15.16:0/596160483 >> 172.21.15.16:0/596160483 conn(0x7f41840fc1e0 msgr2=0x7f41840fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:02:50.237 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.235+0000 7f418b2ac640 1 -- 172.21.15.16:0/596160483 shutdown_connections 2024-09-12T02:02:50.237 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.235+0000 7f418b2ac640 1 -- 172.21.15.16:0/596160483 wait complete. 2024-09-12T02:02:50.239 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.235+0000 7f418b2ac640 1 Processor -- start 2024-09-12T02:02:50.239 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.236+0000 7f418b2ac640 1 -- start start 2024-09-12T02:02:50.239 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.236+0000 7f418b2ac640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4184100a40 0x7f4184195b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:50.239 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.236+0000 7f418b2ac640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4184101c30 0x7f41841960d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:50.239 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.236+0000 7f418b2ac640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41841025f0 0x7f418419d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:50.239 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.236+0000 7f418b2ac640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f418410b5f0 con 0x7f41841025f0 2024-09-12T02:02:50.239 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.236+0000 7f418b2ac640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f418410b470 con 0x7f4184100a40 2024-09-12T02:02:50.239 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.236+0000 7f418b2ac640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f418410b770 con 0x7f4184101c30 2024-09-12T02:02:50.239 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.236+0000 7f4188847640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4184101c30 0x7f41841960d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:50.239 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.236+0000 7f4189849640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41841025f0 0x7f418419d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:50.239 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.236+0000 7f4188847640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4184101c30 0x7f41841960d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:41646/0 (socket says 172.21.15.16:41646) 2024-09-12T02:02:50.239 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.236+0000 7f4188847640 1 -- 172.21.15.16:0/265156578 learned_addr learned my addr 172.21.15.16:0/265156578 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:02:50.239 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.236+0000 7f4189048640 1 --2- 172.21.15.16:0/265156578 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4184100a40 0x7f4184195b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:50.240 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.236+0000 7f4189849640 1 -- 172.21.15.16:0/265156578 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4184101c30 msgr2=0x7f41841960d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:50.240 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.236+0000 7f4189849640 1 --2- 172.21.15.16:0/265156578 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4184101c30 0x7f41841960d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:50.240 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.236+0000 7f4189849640 1 -- 172.21.15.16:0/265156578 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4184100a40 msgr2=0x7f4184195b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:50.240 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.236+0000 7f4189849640 1 --2- 172.21.15.16:0/265156578 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4184100a40 0x7f4184195b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:50.240 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.236+0000 7f4189849640 1 -- 172.21.15.16:0/265156578 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f418419d7b0 con 0x7f41841025f0 2024-09-12T02:02:50.240 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.237+0000 7f4189849640 1 --2- 172.21.15.16:0/265156578 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41841025f0 0x7f418419d140 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f417c00e940 tx=0x7f417c00ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:02:50.240 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.237+0000 7f417a7fc640 1 -- 172.21.15.16:0/265156578 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f417c004d20 con 0x7f41841025f0 2024-09-12T02:02:50.240 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.237+0000 7f417a7fc640 1 -- 172.21.15.16:0/265156578 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f417c005020 con 0x7f41841025f0 2024-09-12T02:02:50.240 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.237+0000 7f418b2ac640 1 -- 172.21.15.16:0/265156578 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f418419daa0 con 0x7f41841025f0 2024-09-12T02:02:50.240 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.237+0000 7f418b2ac640 1 -- 172.21.15.16:0/265156578 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f418419dff0 con 0x7f41841025f0 2024-09-12T02:02:50.241 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.237+0000 7f417a7fc640 1 -- 172.21.15.16:0/265156578 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f417c0116b0 con 0x7f41841025f0 2024-09-12T02:02:50.241 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.241+0000 7f418e04d740 1 Processor -- start 2024-09-12T02:02:50.242 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.241+0000 7f417a7fc640 1 -- 172.21.15.16:0/265156578 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f417c0054d0 con 0x7f41841025f0 2024-09-12T02:02:50.242 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.241+0000 7f418e04d740 1 -- start start 2024-09-12T02:02:50.242 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.241+0000 7f418e04d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f437b479a0 0x55f437b47da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:50.243 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.241+0000 7f418e04d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f437b482e0 0x55f437b4cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:50.244 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.241+0000 7f418e04d740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f437b4d370 0x55f437b4f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:50.244 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.241+0000 7f418e04d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55f437aab2f0 con 0x55f437b482e0 2024-09-12T02:02:50.244 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.241+0000 7f418e04d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55f437a53270 con 0x55f437b479a0 2024-09-12T02:02:50.244 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.241+0000 7f418e04d740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55f437aa72b0 con 0x55f437b4d370 2024-09-12T02:02:50.244 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.241+0000 7f4189048640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f437b482e0 0x55f437b4cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:50.244 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.241+0000 7f4189048640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f437b482e0 0x55f437b4cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57854/0 (socket says 172.21.15.16:57854) 2024-09-12T02:02:50.244 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.241+0000 7f4189048640 1 -- 172.21.15.16:0/2612834953 learned_addr learned my addr 172.21.15.16:0/2612834953 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:02:50.244 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.241+0000 7f4188847640 1 --2- 172.21.15.16:0/2612834953 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f437b479a0 0x55f437b47da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:50.244 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.241+0000 7f4188847640 1 --2- 172.21.15.16:0/2612834953 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f437b4d370 0x55f437b4f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:50.244 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.242+0000 7f4188847640 1 -- 172.21.15.16:0/2612834953 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f437b4d370 msgr2=0x55f437b4f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:50.244 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.242+0000 7f4188847640 1 --2- 172.21.15.16:0/2612834953 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f437b4d370 0x55f437b4f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:50.244 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.242+0000 7f4188847640 1 -- 172.21.15.16:0/2612834953 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f437b482e0 msgr2=0x55f437b4cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:50.245 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.242+0000 7f4188847640 1 --2- 172.21.15.16:0/2612834953 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f437b482e0 0x55f437b4cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:50.245 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.242+0000 7f4188847640 1 -- 172.21.15.16:0/2612834953 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f437a52340 con 0x55f437b479a0 2024-09-12T02:02:50.245 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.242+0000 7f4189048640 1 --2- 172.21.15.16:0/2612834953 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f437b482e0 0x55f437b4cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:02:50.245 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.242+0000 7f4188847640 1 --2- 172.21.15.16:0/2612834953 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f437b479a0 0x55f437b47da0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f41841970a0 tx=0x7f4174009530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6bcb484c2ec380c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:02:50.245 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.242+0000 7f417a7fc640 1 --2- 172.21.15.16:0/265156578 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f41700a2cb0 0x7f41700a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:50.245 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.242+0000 7f416f7fe640 1 -- 172.21.15.16:0/2612834953 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4174004030 con 0x55f437b479a0 2024-09-12T02:02:50.246 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.242+0000 7f416f7fe640 1 -- 172.21.15.16:0/2612834953 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4174005500 con 0x55f437b479a0 2024-09-12T02:02:50.246 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.243+0000 7f417a7fc640 1 -- 172.21.15.16:0/265156578 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f417c0cec30 con 0x7f41841025f0 2024-09-12T02:02:50.246 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.243+0000 7f4189048640 1 --2- 172.21.15.16:0/265156578 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f41700a2cb0 0x7f41700a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:50.252 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.243+0000 7f418e04d740 1 -- 172.21.15.16:0/2612834953 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f437b479a0 msgr2=0x55f437b47da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:50.252 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.243+0000 7f418e04d740 1 --2- 172.21.15.16:0/2612834953 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f437b479a0 0x55f437b47da0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f41841970a0 tx=0x7f4174009530 comp rx=0 tx=0).stop 2024-09-12T02:02:50.252 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.243+0000 7f4189048640 1 --2- 172.21.15.16:0/265156578 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f41700a2cb0 0x7f41700a5170 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x55f437b48d10 tx=0x7f4180005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:02:50.252 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.243+0000 7f418e04d740 1 -- 172.21.15.16:0/2612834953 shutdown_connections 2024-09-12T02:02:50.252 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.243+0000 7f418e04d740 1 --2- 172.21.15.16:0/2612834953 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f437b4d370 0x55f437b4f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:50.252 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.243+0000 7f418e04d740 1 --2- 172.21.15.16:0/2612834953 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f437b482e0 0x55f437b4cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:50.252 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.243+0000 7f418e04d740 1 --2- 172.21.15.16:0/2612834953 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f437b479a0 0x55f437b47da0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:50.252 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.243+0000 7f418e04d740 1 -- 172.21.15.16:0/2612834953 >> 172.21.15.16:0/2612834953 conn(0x55f437b43010 msgr2=0x55f437b43410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:02:50.253 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.244+0000 7f418e04d740 1 -- 172.21.15.16:0/2612834953 shutdown_connections 2024-09-12T02:02:50.253 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.244+0000 7f418e04d740 1 -- 172.21.15.16:0/2612834953 wait complete. 2024-09-12T02:02:50.253 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.244+0000 7f418e04d740 1 -- start start 2024-09-12T02:02:50.253 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.244+0000 7f418e04d740 1 Processor -- start 2024-09-12T02:02:50.253 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.245+0000 7f418e04d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f437b479a0 0x55f437b51bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:50.253 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.245+0000 7f418e04d740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f437b482e0 0x55f437b52110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:50.253 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.245+0000 7f418e04d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f437b4d370 0x55f437b59150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:50.253 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.245+0000 7f418e04d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55f437b50390 con 0x55f437b479a0 2024-09-12T02:02:50.253 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.245+0000 7f418e04d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55f437a34290 con 0x55f437b4d370 2024-09-12T02:02:50.253 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.245+0000 7f418e04d740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55f437b50510 con 0x55f437b482e0 2024-09-12T02:02:50.253 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.245+0000 7f418e04d740 10 client.0 _ll_register_callbacks cb 0x55f437a65140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:02:50.253 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.245+0000 7f4189048640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f437b479a0 0x55f437b51bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:50.253 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.245+0000 7f4188847640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f437b482e0 0x55f437b52110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:50.253 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.245+0000 7f4189048640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f437b479a0 0x55f437b51bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57860/0 (socket says 172.21.15.16:57860) 2024-09-12T02:02:50.254 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.245+0000 7f4188847640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f437b482e0 0x55f437b52110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:41656/0 (socket says 172.21.15.16:41656) 2024-09-12T02:02:50.254 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.245+0000 7f4189048640 1 -- 172.21.15.16:0/1413539318 learned_addr learned my addr 172.21.15.16:0/1413539318 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:02:50.254 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.245+0000 7f4189849640 1 --2- 172.21.15.16:0/1413539318 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f437b4d370 0x55f437b59150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:50.254 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.245+0000 7f4189048640 1 -- 172.21.15.16:0/1413539318 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f437b482e0 msgr2=0x55f437b52110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:50.254 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.245+0000 7f4189048640 1 --2- 172.21.15.16:0/1413539318 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f437b482e0 0x55f437b52110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:50.254 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.245+0000 7f4189048640 1 -- 172.21.15.16:0/1413539318 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f437b4d370 msgr2=0x55f437b59150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:50.254 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.245+0000 7f4189048640 1 --2- 172.21.15.16:0/1413539318 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f437b4d370 0x55f437b59150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:50.254 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.245+0000 7f4189048640 1 -- 172.21.15.16:0/1413539318 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4377be210 con 0x55f437b479a0 2024-09-12T02:02:50.254 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.246+0000 7f4189048640 1 --2- 172.21.15.16:0/1413539318 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f437b479a0 0x55f437b51bd0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f4180038040 tx=0x7f4180009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7018763aeb941948 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:02:50.254 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.246+0000 7f416cff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:02:50.254 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.246+0000 7f416cff9640 1 -- 172.21.15.16:0/1413539318 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4180003ac0 con 0x55f437b479a0 2024-09-12T02:02:50.254 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.246+0000 7f418e04d740 1 -- 172.21.15.16:0/1413539318 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4174005500 con 0x55f437b479a0 2024-09-12T02:02:50.254 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.246+0000 7f416cff9640 1 -- 172.21.15.16:0/1413539318 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f4180033040 con 0x55f437b479a0 2024-09-12T02:02:50.254 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.246+0000 7f416cff9640 1 -- 172.21.15.16:0/1413539318 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4180003c60 con 0x55f437b479a0 2024-09-12T02:02:50.254 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.246+0000 7f418e04d740 10 client.15564 fetch_fsmap learned FSMap version 25 2024-09-12T02:02:50.255 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.246+0000 7f418e04d740 1 -- 172.21.15.16:0/1413539318 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f43788a620 con 0x55f437b479a0 2024-09-12T02:02:50.255 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.246+0000 7f416cff9640 1 -- 172.21.15.16:0/1413539318 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4174005500 con 0x55f437b479a0 2024-09-12T02:02:50.255 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.247+0000 7f416cff9640 1 -- 172.21.15.16:0/1413539318 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f418004b410 con 0x55f437b479a0 2024-09-12T02:02:50.255 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.247+0000 7f418e04d740 10 client.15564 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:02:50.255 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.247+0000 7f418e04d740 10 client.15564 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:02:50.255 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.247+0000 7f418e04d740 20 client.15564 populate_metadata read hostname 'smithi016' 2024-09-12T02:02:50.255 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.247+0000 7f418e04d740 1 --2- 172.21.15.16:0/1413539318 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55f437b5db70 0x55f437b5ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:50.255 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.248+0000 7f418e04d740 4 client.15564 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:02:50.255 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.248+0000 7f418e04d740 1 -- 172.21.15.16:0/1413539318 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f437ab8a20 con 0x55f437b5db70 2024-09-12T02:02:50.255 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.248+0000 7f4188847640 1 --2- 172.21.15.16:0/1413539318 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55f437b5db70 0x55f437b5ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:50.255 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.248+0000 7f4188847640 1 --2- 172.21.15.16:0/1413539318 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55f437b5db70 0x55f437b5ffd0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4dd116e77d2a6bd6 server_cookie=cca0bd1f67412bfe in_seq=0 out_seq=0 2024-09-12T02:02:50.255 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.248+0000 7f416cff9640 10 client.15564 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:02:50.255 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.249+0000 7f416cff9640 1 -- 172.21.15.16:0/1413539318 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f437ab8a20 con 0x55f437b5db70 2024-09-12T02:02:50.255 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.249+0000 7f416cff9640 10 client.15564 handle_command_reply: tid=0 2024-09-12T02:02:50.256 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.249+0000 7f418e04d740 1 client.15564 shutdown 2024-09-12T02:02:50.256 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.249+0000 7f418e04d740 1 -- 172.21.15.16:0/1413539318 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f437b479a0 msgr2=0x55f437b51bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:50.256 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.249+0000 7f418e04d740 1 --2- 172.21.15.16:0/1413539318 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f437b479a0 0x55f437b51bd0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f4180038040 tx=0x7f4180009260 comp rx=0 tx=0).stop 2024-09-12T02:02:50.256 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.249+0000 7f418e04d740 1 -- 172.21.15.16:0/1413539318 shutdown_connections 2024-09-12T02:02:50.256 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.249+0000 7f418e04d740 1 --2- 172.21.15.16:0/1413539318 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55f437b5db70 0x55f437b5ffd0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:50.256 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.249+0000 7f418e04d740 1 --2- 172.21.15.16:0/1413539318 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f437b4d370 0x55f437b59150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:50.256 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.249+0000 7f418e04d740 1 --2- 172.21.15.16:0/1413539318 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f437b482e0 0x55f437b52110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:50.256 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.249+0000 7f418e04d740 1 --2- 172.21.15.16:0/1413539318 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f437b479a0 0x55f437b51bd0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:50.256 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.249+0000 7f418e04d740 1 -- 172.21.15.16:0/1413539318 >> 172.21.15.16:0/1413539318 conn(0x55f437b43010 msgr2=0x55f437b44820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:02:50.256 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.249+0000 7f416cff9640 0 client.15564 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:02:50.256 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.249+0000 7f418e04d740 1 -- 172.21.15.16:0/1413539318 shutdown_connections 2024-09-12T02:02:50.256 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.250+0000 7f418e04d740 1 -- 172.21.15.16:0/1413539318 wait complete. 2024-09-12T02:02:50.256 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.250+0000 7f418e04d740 20 client.15564 trim_cache size 0 max 16384 2024-09-12T02:02:50.281 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.279+0000 7f418e04d740 1 Processor -- start 2024-09-12T02:02:50.281 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.279+0000 7f418e04d740 1 -- start start 2024-09-12T02:02:50.281 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.279+0000 7f418e04d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f437b479a0 0x55f437b52090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:50.281 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.279+0000 7f418e04d740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f437b482e0 0x55f437b59110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:50.281 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.279+0000 7f418e04d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f437b4d370 0x55f437b433f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:50.282 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.279+0000 7f418e04d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f41841a9320 con 0x55f437b4d370 2024-09-12T02:02:50.282 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.280+0000 7f418e04d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f41841a6bf0 con 0x55f437b479a0 2024-09-12T02:02:50.282 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.280+0000 7f4189048640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f437b479a0 0x55f437b52090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:50.282 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.280+0000 7f4189849640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f437b4d370 0x55f437b433f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:50.282 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.280+0000 7f4189048640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f437b479a0 0x55f437b52090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.16:52492/0 (socket says 172.21.15.16:52492) 2024-09-12T02:02:50.282 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.280+0000 7f4189048640 1 -- 172.21.15.16:0/4221094808 learned_addr learned my addr 172.21.15.16:0/4221094808 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:02:50.282 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.280+0000 7f4188847640 1 --2- 172.21.15.16:0/4221094808 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f437b482e0 0x55f437b59110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:50.282 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.280+0000 7f418e04d740 1 -- 172.21.15.16:0/4221094808 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f41841a4b10 con 0x55f437b482e0 2024-09-12T02:02:50.282 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.280+0000 7f4189849640 1 -- 172.21.15.16:0/4221094808 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f437b482e0 msgr2=0x55f437b59110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:50.282 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.280+0000 7f4189849640 1 --2- 172.21.15.16:0/4221094808 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f437b482e0 0x55f437b59110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:50.282 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.280+0000 7f4189849640 1 -- 172.21.15.16:0/4221094808 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f437b479a0 msgr2=0x55f437b52090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:50.282 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.280+0000 7f4189849640 1 --2- 172.21.15.16:0/4221094808 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f437b479a0 0x55f437b52090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:50.282 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.280+0000 7f4189849640 1 -- 172.21.15.16:0/4221094808 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f437b487c0 con 0x55f437b4d370 2024-09-12T02:02:50.284 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.281+0000 7f4189849640 1 --2- 172.21.15.16:0/4221094808 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f437b4d370 0x55f437b433f0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f417c0cc820 tx=0x7f417c01d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b52cfd6f64b417c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:02:50.284 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.281+0000 7f416effd640 1 -- 172.21.15.16:0/4221094808 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f417c0bb040 con 0x55f437b4d370 2024-09-12T02:02:50.284 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.281+0000 7f416effd640 1 -- 172.21.15.16:0/4221094808 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f417c0a41a0 con 0x55f437b4d370 2024-09-12T02:02:50.285 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.281+0000 7f416effd640 1 -- 172.21.15.16:0/4221094808 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f417c01f040 con 0x55f437b4d370 2024-09-12T02:02:50.285 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.282+0000 7f418e04d740 1 -- 172.21.15.16:0/4221094808 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f437b4d370 msgr2=0x55f437b433f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:50.285 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.282+0000 7f418e04d740 1 --2- 172.21.15.16:0/4221094808 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f437b4d370 0x55f437b433f0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f417c0cc820 tx=0x7f417c01d000 comp rx=0 tx=0).stop 2024-09-12T02:02:50.285 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.282+0000 7f418e04d740 1 -- 172.21.15.16:0/4221094808 shutdown_connections 2024-09-12T02:02:50.285 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.282+0000 7f418e04d740 1 --2- 172.21.15.16:0/4221094808 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f437b4d370 0x55f437b433f0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:50.285 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.282+0000 7f418e04d740 1 --2- 172.21.15.16:0/4221094808 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f437b482e0 0x55f437b59110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:50.285 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.282+0000 7f418e04d740 1 --2- 172.21.15.16:0/4221094808 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f437b479a0 0x55f437b52090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:50.285 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.282+0000 7f418e04d740 1 -- 172.21.15.16:0/4221094808 >> 172.21.15.16:0/4221094808 conn(0x55f437b43010 msgr2=0x55f437b51980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:02:50.285 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.283+0000 7f418e04d740 1 -- 172.21.15.16:0/4221094808 shutdown_connections 2024-09-12T02:02:50.285 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.283+0000 7f418e04d740 1 -- 172.21.15.16:0/4221094808 wait complete. 2024-09-12T02:02:50.285 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.283+0000 7f418e04d740 1 -- start start 2024-09-12T02:02:50.285 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.283+0000 7f418e04d740 1 Processor -- start 2024-09-12T02:02:50.289 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.284+0000 7f418e04d740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f437b479a0 0x55f437b55be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:50.289 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.284+0000 7f418e04d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f437b482e0 0x55f437b56120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:50.289 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.284+0000 7f418e04d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f437b4d370 0x55f437bc6860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:50.290 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.284+0000 7f418e04d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f41841a50d0 con 0x55f437b4d370 2024-09-12T02:02:50.290 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.284+0000 7f418e04d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f41841a6520 con 0x55f437b482e0 2024-09-12T02:02:50.290 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.284+0000 7f418e04d740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f41841a5780 con 0x55f437b479a0 2024-09-12T02:02:50.290 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.284+0000 7f418e04d740 10 client.0 _ll_register_callbacks cb 0x7f4180009600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:02:50.290 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.284+0000 7f4189048640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f437b479a0 0x55f437b55be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:50.290 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.284+0000 7f4189048640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f437b479a0 0x55f437b55be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:41676/0 (socket says 172.21.15.16:41676) 2024-09-12T02:02:50.290 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.284+0000 7f4189048640 1 -- 172.21.15.16:0/842244445 learned_addr learned my addr 172.21.15.16:0/842244445 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:02:50.290 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.285+0000 7f4189849640 1 --2- 172.21.15.16:0/842244445 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f437b4d370 0x55f437bc6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:50.290 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.285+0000 7f4188847640 1 --2- 172.21.15.16:0/842244445 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f437b482e0 0x55f437b56120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:50.291 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.285+0000 7f4189048640 1 -- 172.21.15.16:0/842244445 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f437b482e0 msgr2=0x55f437b56120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:50.291 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.285+0000 7f4189048640 1 --2- 172.21.15.16:0/842244445 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f437b482e0 0x55f437b56120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:50.291 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.285+0000 7f4189048640 1 -- 172.21.15.16:0/842244445 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f437b4d370 msgr2=0x55f437bc6860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:50.291 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.285+0000 7f4189048640 1 --2- 172.21.15.16:0/842244445 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f437b4d370 0x55f437bc6860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:50.291 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.285+0000 7f4189048640 1 -- 172.21.15.16:0/842244445 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f437b59210 con 0x55f437b479a0 2024-09-12T02:02:50.291 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.285+0000 7f4189849640 1 --2- 172.21.15.16:0/842244445 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f437b4d370 0x55f437bc6860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:02:50.291 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.285+0000 7f4189048640 1 --2- 172.21.15.16:0/842244445 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f437b479a0 0x55f437b55be0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f4180036480 tx=0x7f4180036a70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=45a223dc19393f9e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:02:50.291 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.285+0000 7f416cff9640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-09-12T02:02:50.291 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.286+0000 7f416cff9640 1 -- 172.21.15.16:0/842244445 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4180064060 con 0x55f437b479a0 2024-09-12T02:02:50.291 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.286+0000 7f416cff9640 1 -- 172.21.15.16:0/842244445 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f418004ba00 con 0x55f437b479a0 2024-09-12T02:02:50.291 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.286+0000 7f418e04d740 1 -- 172.21.15.16:0/842244445 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f417c0a41a0 con 0x55f437b479a0 2024-09-12T02:02:50.292 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.286+0000 7f416cff9640 1 -- 172.21.15.16:0/842244445 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f418004b610 con 0x55f437b479a0 2024-09-12T02:02:50.292 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.286+0000 7f418e04d740 10 client.25183 fetch_fsmap learned FSMap version 25 2024-09-12T02:02:50.292 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.286+0000 7f418e04d740 1 -- 172.21.15.16:0/842244445 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f437bc7d40 con 0x55f437b479a0 2024-09-12T02:02:50.292 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.286+0000 7f416cff9640 1 -- 172.21.15.16:0/842244445 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f417c0a41a0 con 0x55f437b479a0 2024-09-12T02:02:50.292 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.287+0000 7f416cff9640 1 -- 172.21.15.16:0/842244445 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f4180069070 con 0x55f437b479a0 2024-09-12T02:02:50.292 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.288+0000 7f418e04d740 10 client.25183 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:02:50.292 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.288+0000 7f418e04d740 10 client.25183 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:02:50.292 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.288+0000 7f418e04d740 20 client.25183 populate_metadata read hostname 'smithi016' 2024-09-12T02:02:50.292 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.288+0000 7f418e04d740 1 --2- 172.21.15.16:0/842244445 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55f437bc8710 0x55f437bcab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:50.292 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.288+0000 7f418e04d740 4 client.25183 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:02:50.292 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.288+0000 7f418e04d740 1 -- 172.21.15.16:0/842244445 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f437ab2d70 con 0x55f437bc8710 2024-09-12T02:02:50.292 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.288+0000 7f4188847640 1 --2- 172.21.15.16:0/842244445 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55f437bc8710 0x55f437bcab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:50.293 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.288+0000 7f416cff9640 10 client.25183 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:02:50.293 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.288+0000 7f4188847640 1 --2- 172.21.15.16:0/842244445 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55f437bc8710 0x55f437bcab30 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d5fa53a2f347b42 server_cookie=730a89eb25a0dd2d in_seq=0 out_seq=0 2024-09-12T02:02:50.294 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.292+0000 7f416cff9640 1 -- 172.21.15.16:0/842244445 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f437ab2d70 con 0x55f437bc8710 2024-09-12T02:02:50.294 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.292+0000 7f416cff9640 10 client.25183 handle_command_reply: tid=0 2024-09-12T02:02:50.294 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.292+0000 7f418e04d740 1 client.25183 shutdown 2024-09-12T02:02:50.294 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.292+0000 7f418e04d740 1 -- 172.21.15.16:0/842244445 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f437b479a0 msgr2=0x55f437b55be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:50.294 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.292+0000 7f418e04d740 1 --2- 172.21.15.16:0/842244445 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f437b479a0 0x55f437b55be0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f4180036480 tx=0x7f4180036a70 comp rx=0 tx=0).stop 2024-09-12T02:02:50.294 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.292+0000 7f418e04d740 1 -- 172.21.15.16:0/842244445 shutdown_connections 2024-09-12T02:02:50.294 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.292+0000 7f418e04d740 1 --2- 172.21.15.16:0/842244445 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55f437bc8710 0x55f437bcab30 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:50.294 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.292+0000 7f418e04d740 1 --2- 172.21.15.16:0/842244445 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f437b4d370 0x55f437bc6860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:50.294 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.292+0000 7f416cff9640 0 client.25183 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:02:50.295 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.292+0000 7f418e04d740 1 --2- 172.21.15.16:0/842244445 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f437b482e0 0x55f437b56120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:50.295 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.292+0000 7f418e04d740 1 --2- 172.21.15.16:0/842244445 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f437b479a0 0x55f437b55be0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:50.295 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.292+0000 7f418e04d740 1 -- 172.21.15.16:0/842244445 >> 172.21.15.16:0/842244445 conn(0x55f437b43010 msgr2=0x55f437b419a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:02:50.295 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.293+0000 7f418e04d740 1 -- 172.21.15.16:0/842244445 shutdown_connections 2024-09-12T02:02:50.295 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.293+0000 7f418e04d740 1 -- 172.21.15.16:0/842244445 wait complete. 2024-09-12T02:02:50.295 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.293+0000 7f418e04d740 20 client.25183 trim_cache size 0 max 16384 2024-09-12T02:02:50.295 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-09-12T02:02:50.295 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-09-12T02:02:50.295 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-09-12T02:02:50.295 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-09-12T02:02:50.296 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.294+0000 7f418b2ac640 1 -- 172.21.15.16:0/265156578 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f41700a2cb0 msgr2=0x7f41700a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:50.296 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.294+0000 7f418b2ac640 1 --2- 172.21.15.16:0/265156578 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f41700a2cb0 0x7f41700a5170 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x55f437b48d10 tx=0x7f4180005fd0 comp rx=0 tx=0).stop 2024-09-12T02:02:50.296 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.294+0000 7f418b2ac640 1 -- 172.21.15.16:0/265156578 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41841025f0 msgr2=0x7f418419d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:50.297 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.294+0000 7f418b2ac640 1 --2- 172.21.15.16:0/265156578 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41841025f0 0x7f418419d140 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f417c00e940 tx=0x7f417c00ee00 comp rx=0 tx=0).stop 2024-09-12T02:02:50.297 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.294+0000 7f418b2ac640 1 -- 172.21.15.16:0/265156578 shutdown_connections 2024-09-12T02:02:50.297 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.294+0000 7f418b2ac640 1 --2- 172.21.15.16:0/265156578 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f41700a2cb0 0x7f41700a5170 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:50.297 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.294+0000 7f418b2ac640 1 --2- 172.21.15.16:0/265156578 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41841025f0 0x7f418419d140 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:50.297 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.294+0000 7f418b2ac640 1 --2- 172.21.15.16:0/265156578 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4184101c30 0x7f41841960d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:50.297 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.294+0000 7f418b2ac640 1 --2- 172.21.15.16:0/265156578 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4184100a40 0x7f4184195b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:50.297 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.294+0000 7f418b2ac640 1 -- 172.21.15.16:0/265156578 >> 172.21.15.16:0/265156578 conn(0x7f41840fc1e0 msgr2=0x7f4184104f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:02:50.297 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.294+0000 7f418b2ac640 1 -- 172.21.15.16:0/265156578 shutdown_connections 2024-09-12T02:02:50.297 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:02:50.294+0000 7f418b2ac640 1 -- 172.21.15.16:0/265156578 wait complete. 2024-09-12T02:02:51.680 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:02:51 smithi184 ceph-mon[35593]: pgmap v363: 97 pgs: 97 active+clean; 667 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 853 B/s wr, 8 op/s 2024-09-12T02:02:51.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:02:51 smithi016 ceph-mon[30947]: pgmap v363: 97 pgs: 97 active+clean; 667 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 853 B/s wr, 8 op/s 2024-09-12T02:02:51.721 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:02:51 smithi033 ceph-mon[35332]: pgmap v363: 97 pgs: 97 active+clean; 667 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 853 B/s wr, 8 op/s 2024-09-12T02:02:53.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:02:53 smithi016 ceph-mon[30947]: pgmap v364: 97 pgs: 97 active+clean; 667 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-12T02:02:53.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:02:53 smithi184 ceph-mon[35593]: pgmap v364: 97 pgs: 97 active+clean; 667 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-12T02:02:53.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:02:53 smithi033 ceph-mon[35332]: pgmap v364: 97 pgs: 97 active+clean; 667 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-12T02:02:55.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:02:55 smithi184 ceph-mon[35593]: pgmap v365: 97 pgs: 97 active+clean; 668 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-09-12T02:02:55.918 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:02:55 smithi033 ceph-mon[35332]: pgmap v365: 97 pgs: 97 active+clean; 668 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-09-12T02:02:55.933 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:02:55 smithi016 ceph-mon[30947]: pgmap v365: 97 pgs: 97 active+clean; 668 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-09-12T02:02:57.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:02:57 smithi184 ceph-mon[35593]: pgmap v366: 97 pgs: 97 active+clean; 668 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-09-12T02:02:57.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:02:57 smithi033 ceph-mon[35332]: pgmap v366: 97 pgs: 97 active+clean; 668 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-09-12T02:02:57.933 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:02:57 smithi016 ceph-mon[30947]: pgmap v366: 97 pgs: 97 active+clean; 668 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-09-12T02:02:58.445 INFO:tasks.workunit.client.0.smithi016.stderr:Updating files: 92% (11898/12903) Updating files: 93% (12000/12903) Updating files: 94% (12129/12903) Updating files: 95% (12258/12903) Updating files: 96% (12387/12903) Updating files: 97% (12516/12903) Updating files: 98% (12645/12903) Updating files: 99% (12774/12903) Updating files: 100% (12903/12903) Updating files: 100% (12903/12903), done. 2024-09-12T02:02:59.182 INFO:tasks.workunit.client.0.smithi016.stderr:Note: switching to '086c737fa75c22d521a3932c59b31c5e1d1c9ef4'. 2024-09-12T02:02:59.182 INFO:tasks.workunit.client.0.smithi016.stderr: 2024-09-12T02:02:59.182 INFO:tasks.workunit.client.0.smithi016.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-09-12T02:02:59.182 INFO:tasks.workunit.client.0.smithi016.stderr:changes and commit them, and you can discard any commits you make in this 2024-09-12T02:02:59.182 INFO:tasks.workunit.client.0.smithi016.stderr:state without impacting any branches by switching back to a branch. 2024-09-12T02:02:59.182 INFO:tasks.workunit.client.0.smithi016.stderr: 2024-09-12T02:02:59.182 INFO:tasks.workunit.client.0.smithi016.stderr:If you want to create a new branch to retain commits you create, you may 2024-09-12T02:02:59.182 INFO:tasks.workunit.client.0.smithi016.stderr:do so (now or later) by using -c with the switch command. Example: 2024-09-12T02:02:59.182 INFO:tasks.workunit.client.0.smithi016.stderr: 2024-09-12T02:02:59.182 INFO:tasks.workunit.client.0.smithi016.stderr: git switch -c 2024-09-12T02:02:59.183 INFO:tasks.workunit.client.0.smithi016.stderr: 2024-09-12T02:02:59.183 INFO:tasks.workunit.client.0.smithi016.stderr:Or undo this operation with: 2024-09-12T02:02:59.183 INFO:tasks.workunit.client.0.smithi016.stderr: 2024-09-12T02:02:59.183 INFO:tasks.workunit.client.0.smithi016.stderr: git switch - 2024-09-12T02:02:59.183 INFO:tasks.workunit.client.0.smithi016.stderr: 2024-09-12T02:02:59.183 INFO:tasks.workunit.client.0.smithi016.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-09-12T02:02:59.183 INFO:tasks.workunit.client.0.smithi016.stderr: 2024-09-12T02:02:59.183 INFO:tasks.workunit.client.0.smithi016.stderr:HEAD is now at 086c737fa75 Merge pull request #59733 from zdover23/wip-doc-2024-09-11-backport-59635-to-squid 2024-09-12T02:02:59.188 DEBUG:teuthology.orchestra.run.smithi016:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-09-12T02:02:59.254 INFO:tasks.workunit.client.0.smithi016.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-09-12T02:02:59.257 INFO:tasks.workunit.client.0.smithi016.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-12T02:02:59.257 INFO:tasks.workunit.client.0.smithi016.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-09-12T02:02:59.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:02:59 smithi184 ceph-mon[35593]: pgmap v367: 97 pgs: 97 active+clean; 668 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-12T02:02:59.918 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:02:59 smithi033 ceph-mon[35332]: pgmap v367: 97 pgs: 97 active+clean; 668 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-12T02:02:59.933 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:02:59 smithi016 ceph-mon[30947]: pgmap v367: 97 pgs: 97 active+clean; 668 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-12T02:02:59.990 INFO:tasks.workunit.client.0.smithi016.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-09-12T02:03:00.065 INFO:tasks.workunit.client.0.smithi016.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-09-12T02:03:00.105 INFO:tasks.workunit.client.0.smithi016.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-12T02:03:00.107 INFO:tasks.workunit.client.0.smithi016.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-12T02:03:00.107 INFO:tasks.workunit.client.0.smithi016.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-09-12T02:03:00.147 INFO:tasks.workunit.client.0.smithi016.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-12T02:03:00.151 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-12T02:03:00.151 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-09-12T02:03:00.207 INFO:tasks.workunit:Running workunits matching fs/test_o_trunc.sh on client.0... 2024-09-12T02:03:00.208 INFO:tasks.workunit:Running workunit fs/test_o_trunc.sh... 2024-09-12T02:03:00.209 DEBUG:teuthology.orchestra.run.smithi016:workunit test fs/test_o_trunc.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=086c737fa75c22d521a3932c59b31c5e1d1c9ef4 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/test_o_trunc.sh 2024-09-12T02:03:00.272 INFO:tasks.workunit.client.0.smithi016.stderr:++ dirname /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/test_o_trunc.sh 2024-09-12T02:03:00.273 INFO:tasks.workunit.client.0.smithi016.stderr:+ mydir=/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs 2024-09-12T02:03:00.273 INFO:tasks.workunit.client.0.smithi016.stderr:+ /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/test_o_trunc trunc.foo 600 2024-09-12T02:03:01.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:03:01 smithi184 ceph-mon[35593]: pgmap v368: 97 pgs: 97 active+clean; 668 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-09-12T02:03:01.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:03:01 smithi033 ceph-mon[35332]: pgmap v368: 97 pgs: 97 active+clean; 668 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-09-12T02:03:01.933 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:03:01 smithi016 ceph-mon[30947]: pgmap v368: 97 pgs: 97 active+clean; 668 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-09-12T02:03:03.390 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.388+0000 7febeccf1640 1 -- 172.21.15.16:0/12481842 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febe8101c40 msgr2=0x7febe81020c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:03.391 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.388+0000 7febeccf1640 1 --2- 172.21.15.16:0/12481842 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febe8101c40 0x7febe81020c0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7febdc009900 tx=0x7febdc02a410 comp rx=0 tx=0).stop 2024-09-12T02:03:03.391 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.388+0000 7febeccf1640 1 -- 172.21.15.16:0/12481842 shutdown_connections 2024-09-12T02:03:03.391 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.388+0000 7febeccf1640 1 --2- 172.21.15.16:0/12481842 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7febe8102600 0x7febe8108e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:03.391 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.388+0000 7febeccf1640 1 --2- 172.21.15.16:0/12481842 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febe8101c40 0x7febe81020c0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:03.391 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.388+0000 7febeccf1640 1 --2- 172.21.15.16:0/12481842 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7febe8100a50 0x7febe8100e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:03.391 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.388+0000 7febeccf1640 1 -- 172.21.15.16:0/12481842 >> 172.21.15.16:0/12481842 conn(0x7febe80fc1b0 msgr2=0x7febe80fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:03.391 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.388+0000 7febeccf1640 1 -- 172.21.15.16:0/12481842 shutdown_connections 2024-09-12T02:03:03.391 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.388+0000 7febeccf1640 1 -- 172.21.15.16:0/12481842 wait complete. 2024-09-12T02:03:03.391 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.389+0000 7febeccf1640 1 Processor -- start 2024-09-12T02:03:03.391 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.389+0000 7febeccf1640 1 -- start start 2024-09-12T02:03:03.391 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.389+0000 7febeccf1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febe8100a50 0x7febe8197dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:03.392 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.389+0000 7febeccf1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7febe8101c40 0x7febe8198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:03.392 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.389+0000 7febeccf1640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7febe8102600 0x7febe819f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:03.392 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.389+0000 7febeccf1640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7febe810c6b0 con 0x7febe8100a50 2024-09-12T02:03:03.392 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.389+0000 7febeccf1640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7febe810c530 con 0x7febe8101c40 2024-09-12T02:03:03.392 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.389+0000 7febeccf1640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7febe810c830 con 0x7febe8102600 2024-09-12T02:03:03.392 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.389+0000 7febe659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febe8100a50 0x7febe8197dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:03.392 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.389+0000 7febe659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febe8100a50 0x7febe8197dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45980/0 (socket says 172.21.15.16:45980) 2024-09-12T02:03:03.392 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.389+0000 7febe659c640 1 -- 172.21.15.16:0/3725426405 learned_addr learned my addr 172.21.15.16:0/3725426405 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:03:03.392 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.389+0000 7febe5d9b640 1 --2- 172.21.15.16:0/3725426405 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7febe8101c40 0x7febe8198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:03.392 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.389+0000 7febe6d9d640 1 --2- 172.21.15.16:0/3725426405 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7febe8102600 0x7febe819f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:03.392 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.390+0000 7febe659c640 1 -- 172.21.15.16:0/3725426405 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7febe8102600 msgr2=0x7febe819f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:03.392 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.390+0000 7febe659c640 1 --2- 172.21.15.16:0/3725426405 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7febe8102600 0x7febe819f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:03.393 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.390+0000 7febe659c640 1 -- 172.21.15.16:0/3725426405 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7febe8101c40 msgr2=0x7febe8198310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:03.393 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.390+0000 7febe659c640 1 --2- 172.21.15.16:0/3725426405 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7febe8101c40 0x7febe8198310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:03.393 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.390+0000 7febe659c640 1 -- 172.21.15.16:0/3725426405 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febe819f9f0 con 0x7febe8100a50 2024-09-12T02:03:03.393 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.390+0000 7febe5d9b640 1 --2- 172.21.15.16:0/3725426405 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7febe8101c40 0x7febe8198310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:03:03.393 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.390+0000 7febe659c640 1 --2- 172.21.15.16:0/3725426405 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febe8100a50 0x7febe8197dd0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7febd800d950 tx=0x7febd800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:03.393 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.390+0000 7febd77fe640 1 -- 172.21.15.16:0/3725426405 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febd8017020 con 0x7febe8100a50 2024-09-12T02:03:03.393 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.390+0000 7febeccf1640 1 -- 172.21.15.16:0/3725426405 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febe819fce0 con 0x7febe8100a50 2024-09-12T02:03:03.393 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.390+0000 7febd77fe640 1 -- 172.21.15.16:0/3725426405 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7febd8009c30 con 0x7febe8100a50 2024-09-12T02:03:03.393 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.390+0000 7febd77fe640 1 -- 172.21.15.16:0/3725426405 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febd800f6b0 con 0x7febe8100a50 2024-09-12T02:03:03.393 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.390+0000 7febeccf1640 1 -- 172.21.15.16:0/3725426405 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febe81a01a0 con 0x7febe8100a50 2024-09-12T02:03:03.399 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.393+0000 7febd77fe640 1 -- 172.21.15.16:0/3725426405 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7febd801a020 con 0x7febe8100a50 2024-09-12T02:03:03.399 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.393+0000 7febefa8b740 1 Processor -- start 2024-09-12T02:03:03.400 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.393+0000 7febefa8b740 1 -- start start 2024-09-12T02:03:03.400 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.393+0000 7febefa8b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5604383e3990 0x5604383e3d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:03.400 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.393+0000 7febefa8b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5604383e42d0 0x5604383e8d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:03.400 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.393+0000 7febefa8b740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5604383e9360 0x5604383eb820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:03.400 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.393+0000 7febefa8b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5604383472f0 con 0x5604383e42d0 2024-09-12T02:03:03.400 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.393+0000 7febefa8b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5604382ef270 con 0x5604383e3990 2024-09-12T02:03:03.400 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.393+0000 7febefa8b740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5604383432b0 con 0x5604383e9360 2024-09-12T02:03:03.400 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.393+0000 7febe659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5604383e42d0 0x5604383e8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:03.400 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.394+0000 7febe6d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5604383e3990 0x5604383e3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:03.400 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.394+0000 7febe659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5604383e42d0 0x5604383e8d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45996/0 (socket says 172.21.15.16:45996) 2024-09-12T02:03:03.400 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.394+0000 7febe659c640 1 -- 172.21.15.16:0/451244274 learned_addr learned my addr 172.21.15.16:0/451244274 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:03:03.400 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.394+0000 7febe5d9b640 1 --2- 172.21.15.16:0/451244274 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5604383e9360 0x5604383eb820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:03.400 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.394+0000 7febd77fe640 1 --2- 172.21.15.16:0/3725426405 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7febd00a2c30 0x7febd00a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:03.401 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.394+0000 7febe6d9d640 1 -- 172.21.15.16:0/451244274 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5604383e9360 msgr2=0x5604383eb820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:03.401 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.394+0000 7febe6d9d640 1 --2- 172.21.15.16:0/451244274 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5604383e9360 0x5604383eb820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:03.401 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.394+0000 7febe6d9d640 1 -- 172.21.15.16:0/451244274 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5604383e42d0 msgr2=0x5604383e8d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:03.401 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.394+0000 7febe6d9d640 1 --2- 172.21.15.16:0/451244274 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5604383e42d0 0x5604383e8d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:03.401 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.394+0000 7febe6d9d640 1 -- 172.21.15.16:0/451244274 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604382ee340 con 0x5604383e3990 2024-09-12T02:03:03.401 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.394+0000 7febd77fe640 1 -- 172.21.15.16:0/3725426405 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7febd80cef20 con 0x7febe8100a50 2024-09-12T02:03:03.401 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.394+0000 7febe6d9d640 1 --2- 172.21.15.16:0/451244274 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5604383e3990 0x5604383e3d90 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7febe0009eb0 tx=0x7febe0010740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5dfcb7b37d0f5b64 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:03.401 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.394+0000 7febe6d9d640 1 --2- 172.21.15.16:0/3725426405 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7febd00a2c30 0x7febd00a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:03.401 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.394+0000 7febd4ff9640 1 -- 172.21.15.16:0/451244274 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febe0016070 con 0x5604383e3990 2024-09-12T02:03:03.401 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.394+0000 7febd4ff9640 1 -- 172.21.15.16:0/451244274 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7febe0012450 con 0x5604383e3990 2024-09-12T02:03:03.401 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.394+0000 7febd4ff9640 1 -- 172.21.15.16:0/451244274 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febe00116c0 con 0x5604383e3990 2024-09-12T02:03:03.401 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.395+0000 7febefa8b740 1 -- 172.21.15.16:0/451244274 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5604383e3990 msgr2=0x5604383e3d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:03.402 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.395+0000 7febe6d9d640 1 --2- 172.21.15.16:0/3725426405 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7febd00a2c30 0x7febd00a50f0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7febe0011010 tx=0x7febe0011db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:03.402 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.395+0000 7febefa8b740 1 --2- 172.21.15.16:0/451244274 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5604383e3990 0x5604383e3d90 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7febe0009eb0 tx=0x7febe0010740 comp rx=0 tx=0).stop 2024-09-12T02:03:03.402 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.395+0000 7febefa8b740 1 -- 172.21.15.16:0/451244274 shutdown_connections 2024-09-12T02:03:03.402 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.395+0000 7febefa8b740 1 --2- 172.21.15.16:0/451244274 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5604383e9360 0x5604383eb820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:03.402 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.395+0000 7febefa8b740 1 --2- 172.21.15.16:0/451244274 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5604383e42d0 0x5604383e8d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:03.402 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.395+0000 7febefa8b740 1 --2- 172.21.15.16:0/451244274 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5604383e3990 0x5604383e3d90 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:03.402 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.395+0000 7febefa8b740 1 -- 172.21.15.16:0/451244274 >> 172.21.15.16:0/451244274 conn(0x5604383df000 msgr2=0x5604383df400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:03.402 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.395+0000 7febefa8b740 1 -- 172.21.15.16:0/451244274 shutdown_connections 2024-09-12T02:03:03.402 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.395+0000 7febefa8b740 1 -- 172.21.15.16:0/451244274 wait complete. 2024-09-12T02:03:03.402 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.396+0000 7febefa8b740 1 -- start start 2024-09-12T02:03:03.402 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.396+0000 7febefa8b740 1 Processor -- start 2024-09-12T02:03:03.402 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.396+0000 7febefa8b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5604383e3990 0x5604383edb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:03.403 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.396+0000 7febefa8b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5604383e42d0 0x5604383ee090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:03.403 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.396+0000 7febefa8b740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5604383e9360 0x5604383f50d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:03.403 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.396+0000 7febefa8b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5604383ec330 con 0x5604383e42d0 2024-09-12T02:03:03.403 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.396+0000 7febefa8b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5604382d0290 con 0x5604383e3990 2024-09-12T02:03:03.403 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.396+0000 7febefa8b740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5604383ec4b0 con 0x5604383e9360 2024-09-12T02:03:03.403 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.396+0000 7febefa8b740 10 client.0 _ll_register_callbacks cb 0x560438301140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:03:03.403 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.396+0000 7febe5d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5604383e42d0 0x5604383ee090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:03.403 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.396+0000 7febe659c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5604383e3990 0x5604383edb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:03.403 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.396+0000 7febe5d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5604383e42d0 0x5604383ee090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45998/0 (socket says 172.21.15.16:45998) 2024-09-12T02:03:03.403 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.396+0000 7febe5d9b640 1 -- 172.21.15.16:0/2671439502 learned_addr learned my addr 172.21.15.16:0/2671439502 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:03:03.403 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.396+0000 7febe6d9d640 1 --2- 172.21.15.16:0/2671439502 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5604383e9360 0x5604383f50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:03.403 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.397+0000 7febe659c640 1 -- 172.21.15.16:0/2671439502 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5604383e9360 msgr2=0x5604383f50d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:03.403 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.397+0000 7febe659c640 1 --2- 172.21.15.16:0/2671439502 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5604383e9360 0x5604383f50d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:03.404 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.397+0000 7febe659c640 1 -- 172.21.15.16:0/2671439502 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5604383e42d0 msgr2=0x5604383ee090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:03.404 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.397+0000 7febe659c640 1 --2- 172.21.15.16:0/2671439502 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5604383e42d0 0x5604383ee090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:03.404 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.397+0000 7febe659c640 1 -- 172.21.15.16:0/2671439502 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56043805a210 con 0x5604383e3990 2024-09-12T02:03:03.404 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.397+0000 7febe659c640 1 --2- 172.21.15.16:0/2671439502 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5604383e3990 0x5604383edb50 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7febd80ccb10 tx=0x7febd8010ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc5bc164e67f6498 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:03.404 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.397+0000 7febcdffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-12T02:03:03.404 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.397+0000 7febcdffb640 1 -- 172.21.15.16:0/2671439502 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febd809b360 con 0x5604383e3990 2024-09-12T02:03:03.404 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.397+0000 7febcdffb640 1 -- 172.21.15.16:0/2671439502 <== mon.2 v2:172.21.15.33:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7febd80109d0 con 0x5604383e3990 2024-09-12T02:03:03.404 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.397+0000 7febefa8b740 1 -- 172.21.15.16:0/2671439502 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7febe0012450 con 0x5604383e3990 2024-09-12T02:03:03.404 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.397+0000 7febcdffb640 1 -- 172.21.15.16:0/2671439502 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7febd809b180 con 0x5604383e3990 2024-09-12T02:03:03.404 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.397+0000 7febefa8b740 10 client.25262 fetch_fsmap learned FSMap version 25 2024-09-12T02:03:03.404 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.397+0000 7febefa8b740 1 -- 172.21.15.16:0/2671439502 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x560438126620 con 0x5604383e3990 2024-09-12T02:03:03.404 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.397+0000 7febcdffb640 1 -- 172.21.15.16:0/2671439502 <== mon.2 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7febe0012450 con 0x5604383e3990 2024-09-12T02:03:03.404 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.397+0000 7febcdffb640 1 -- 172.21.15.16:0/2671439502 <== mon.2 v2:172.21.15.33:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7febd8097e90 con 0x5604383e3990 2024-09-12T02:03:03.405 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.398+0000 7febefa8b740 10 client.25262 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:03:03.405 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.398+0000 7febefa8b740 10 client.25262 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:03:03.405 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.398+0000 7febefa8b740 20 client.25262 populate_metadata read hostname 'smithi016' 2024-09-12T02:03:03.405 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.398+0000 7febefa8b740 1 --2- 172.21.15.16:0/2671439502 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5604383f9820 0x5604383fbc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:03.405 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.398+0000 7febefa8b740 4 client.25262 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:03:03.405 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.398+0000 7febefa8b740 1 -- 172.21.15.16:0/2671439502 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560438354a20 con 0x5604383f9820 2024-09-12T02:03:03.405 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.398+0000 7febe5d9b640 1 --2- 172.21.15.16:0/2671439502 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5604383f9820 0x5604383fbc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:03.405 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.398+0000 7febe5d9b640 1 --2- 172.21.15.16:0/2671439502 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5604383f9820 0x5604383fbc90 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=caccb8526182dfdc server_cookie=984df829ec89a1e9 in_seq=0 out_seq=0 2024-09-12T02:03:03.405 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.398+0000 7febcdffb640 10 client.25262 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:03:03.405 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.399+0000 7febcdffb640 1 -- 172.21.15.16:0/2671439502 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560438354a20 con 0x5604383f9820 2024-09-12T02:03:03.405 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.399+0000 7febcdffb640 10 client.25262 handle_command_reply: tid=0 2024-09-12T02:03:03.405 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.399+0000 7febefa8b740 1 client.25262 shutdown 2024-09-12T02:03:03.406 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.399+0000 7febefa8b740 1 -- 172.21.15.16:0/2671439502 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5604383e3990 msgr2=0x5604383edb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:03.406 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.399+0000 7febefa8b740 1 --2- 172.21.15.16:0/2671439502 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5604383e3990 0x5604383edb50 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7febd80ccb10 tx=0x7febd8010ca0 comp rx=0 tx=0).stop 2024-09-12T02:03:03.406 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.399+0000 7febefa8b740 1 -- 172.21.15.16:0/2671439502 shutdown_connections 2024-09-12T02:03:03.406 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.399+0000 7febefa8b740 1 --2- 172.21.15.16:0/2671439502 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5604383f9820 0x5604383fbc90 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:03.406 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.399+0000 7febefa8b740 1 --2- 172.21.15.16:0/2671439502 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5604383e9360 0x5604383f50d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:03.406 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.399+0000 7febefa8b740 1 --2- 172.21.15.16:0/2671439502 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5604383e42d0 0x5604383ee090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:03.406 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.399+0000 7febcdffb640 0 client.25262 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:03:03.406 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.399+0000 7febefa8b740 1 --2- 172.21.15.16:0/2671439502 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5604383e3990 0x5604383edb50 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:03.406 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.399+0000 7febefa8b740 1 -- 172.21.15.16:0/2671439502 >> 172.21.15.16:0/2671439502 conn(0x5604383df000 msgr2=0x5604383e0810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:03.406 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.399+0000 7febefa8b740 1 -- 172.21.15.16:0/2671439502 shutdown_connections 2024-09-12T02:03:03.406 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.399+0000 7febefa8b740 1 -- 172.21.15.16:0/2671439502 wait complete. 2024-09-12T02:03:03.406 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.399+0000 7febefa8b740 20 client.25262 trim_cache size 0 max 16384 2024-09-12T02:03:03.419 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.414+0000 7febefa8b740 1 Processor -- start 2024-09-12T02:03:03.419 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.414+0000 7febefa8b740 1 -- start start 2024-09-12T02:03:03.419 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.414+0000 7febefa8b740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5604383e3990 0x5604383ee010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:03.419 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.414+0000 7febefa8b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5604383e42d0 0x5604383f5060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:03.419 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.414+0000 7febefa8b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5604383e9360 0x5604383dff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:03.419 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.414+0000 7febefa8b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7febe81ab750 con 0x5604383e9360 2024-09-12T02:03:03.420 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.414+0000 7febefa8b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7febe81a9020 con 0x5604383e42d0 2024-09-12T02:03:03.420 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.414+0000 7febefa8b740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7febe81a6e70 con 0x5604383e3990 2024-09-12T02:03:03.420 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.414+0000 7febe6d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5604383e9360 0x5604383dff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:03.420 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.414+0000 7febe6d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5604383e9360 0x5604383dff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46014/0 (socket says 172.21.15.16:46014) 2024-09-12T02:03:03.420 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.414+0000 7febe6d9d640 1 -- 172.21.15.16:0/213926263 learned_addr learned my addr 172.21.15.16:0/213926263 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:03:03.420 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.415+0000 7febe659c640 1 --2- 172.21.15.16:0/213926263 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5604383e3990 0x5604383ee010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:03.420 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.415+0000 7febe5d9b640 1 --2- 172.21.15.16:0/213926263 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5604383e42d0 0x5604383f5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:03.420 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.415+0000 7febe6d9d640 1 -- 172.21.15.16:0/213926263 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5604383e3990 msgr2=0x5604383ee010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:03.421 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.415+0000 7febe6d9d640 1 --2- 172.21.15.16:0/213926263 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5604383e3990 0x5604383ee010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:03.421 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.415+0000 7febe6d9d640 1 -- 172.21.15.16:0/213926263 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5604383e42d0 msgr2=0x5604383f5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:03.421 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.415+0000 7febe6d9d640 1 --2- 172.21.15.16:0/213926263 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5604383e42d0 0x5604383f5060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:03.421 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.415+0000 7febe6d9d640 1 -- 172.21.15.16:0/213926263 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604383e8700 con 0x5604383e9360 2024-09-12T02:03:03.421 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.415+0000 7febe6d9d640 1 --2- 172.21.15.16:0/213926263 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5604383e9360 0x5604383dff10 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7febe0009a80 tx=0x7febe0027000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=715d2076ec8a03ed server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:03.421 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.415+0000 7febcffff640 1 -- 172.21.15.16:0/213926263 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febe002d040 con 0x5604383e9360 2024-09-12T02:03:03.421 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.415+0000 7febcffff640 1 -- 172.21.15.16:0/213926263 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7febe0004980 con 0x5604383e9360 2024-09-12T02:03:03.421 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.415+0000 7febcffff640 1 -- 172.21.15.16:0/213926263 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febe0016040 con 0x5604383e9360 2024-09-12T02:03:03.421 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.416+0000 7febefa8b740 1 -- 172.21.15.16:0/213926263 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5604383e9360 msgr2=0x5604383dff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:03.421 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.416+0000 7febefa8b740 1 --2- 172.21.15.16:0/213926263 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5604383e9360 0x5604383dff10 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7febe0009a80 tx=0x7febe0027000 comp rx=0 tx=0).stop 2024-09-12T02:03:03.421 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.416+0000 7febefa8b740 1 -- 172.21.15.16:0/213926263 shutdown_connections 2024-09-12T02:03:03.421 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.416+0000 7febefa8b740 1 --2- 172.21.15.16:0/213926263 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5604383e9360 0x5604383dff10 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:03.421 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.416+0000 7febefa8b740 1 --2- 172.21.15.16:0/213926263 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5604383e42d0 0x5604383f5060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:03.422 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.416+0000 7febefa8b740 1 --2- 172.21.15.16:0/213926263 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5604383e3990 0x5604383ee010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:03.422 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.416+0000 7febefa8b740 1 -- 172.21.15.16:0/213926263 >> 172.21.15.16:0/213926263 conn(0x5604383df000 msgr2=0x5604383ed900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:03.422 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.416+0000 7febefa8b740 1 -- 172.21.15.16:0/213926263 shutdown_connections 2024-09-12T02:03:03.422 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.416+0000 7febefa8b740 1 -- 172.21.15.16:0/213926263 wait complete. 2024-09-12T02:03:03.422 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.416+0000 7febefa8b740 1 -- start start 2024-09-12T02:03:03.422 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.417+0000 7febefa8b740 1 Processor -- start 2024-09-12T02:03:03.422 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.417+0000 7febefa8b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5604383e3990 0x5604383f24a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:03.422 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.417+0000 7febefa8b740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5604383e42d0 0x5604383f29e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:03.422 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.417+0000 7febefa8b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5604383e9360 0x5604383f39a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:03.422 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.417+0000 7febefa8b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7febe81a74e0 con 0x5604383e3990 2024-09-12T02:03:03.422 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.417+0000 7febefa8b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7febe81a8950 con 0x5604383e9360 2024-09-12T02:03:03.422 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.417+0000 7febefa8b740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7febe81a7bb0 con 0x5604383e42d0 2024-09-12T02:03:03.423 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.417+0000 7febefa8b740 10 client.0 _ll_register_callbacks cb 0x7febd80046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:03:03.423 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.417+0000 7febe659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5604383e3990 0x5604383f24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:03.423 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.417+0000 7febe6d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5604383e9360 0x5604383f39a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:03.423 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.417+0000 7febe659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5604383e3990 0x5604383f24a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46024/0 (socket says 172.21.15.16:46024) 2024-09-12T02:03:03.423 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.417+0000 7febe6d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5604383e9360 0x5604383f39a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.16:33998/0 (socket says 172.21.15.16:33998) 2024-09-12T02:03:03.423 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.417+0000 7febe659c640 1 -- 172.21.15.16:0/2939655528 learned_addr learned my addr 172.21.15.16:0/2939655528 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:03:03.423 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.417+0000 7febe5d9b640 1 --2- 172.21.15.16:0/2939655528 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5604383e42d0 0x5604383f29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:03.423 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.417+0000 7febe659c640 1 -- 172.21.15.16:0/2939655528 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5604383e42d0 msgr2=0x5604383f29e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:03.423 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.417+0000 7febe659c640 1 --2- 172.21.15.16:0/2939655528 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5604383e42d0 0x5604383f29e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:03.423 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.417+0000 7febe659c640 1 -- 172.21.15.16:0/2939655528 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5604383e9360 msgr2=0x5604383f39a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:03.424 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.417+0000 7febe659c640 1 --2- 172.21.15.16:0/2939655528 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5604383e9360 0x5604383f39a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:03.424 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.417+0000 7febe659c640 1 -- 172.21.15.16:0/2939655528 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5604383f3fb0 con 0x5604383e3990 2024-09-12T02:03:03.424 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.417+0000 7febe6d9d640 1 --2- 172.21.15.16:0/2939655528 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5604383e9360 0x5604383f39a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:03:03.424 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.418+0000 7febe659c640 1 --2- 172.21.15.16:0/2939655528 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5604383e3990 0x5604383f24a0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7febd8097c80 tx=0x7febd800bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81246727f55157ed server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:03.424 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.418+0000 7febcdffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:03:03.424 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.418+0000 7febcdffb640 1 -- 172.21.15.16:0/2939655528 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febd8014070 con 0x5604383e3990 2024-09-12T02:03:03.424 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.418+0000 7febcdffb640 1 -- 172.21.15.16:0/2939655528 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7febd809cd50 con 0x5604383e3990 2024-09-12T02:03:03.424 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.418+0000 7febefa8b740 1 -- 172.21.15.16:0/2939655528 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7febe0004980 con 0x5604383e3990 2024-09-12T02:03:03.424 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.418+0000 7febcdffb640 1 -- 172.21.15.16:0/2939655528 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7febd809b360 con 0x5604383e3990 2024-09-12T02:03:03.424 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.418+0000 7febefa8b740 10 client.15588 fetch_fsmap learned FSMap version 25 2024-09-12T02:03:03.424 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.418+0000 7febefa8b740 1 -- 172.21.15.16:0/2939655528 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5604383f5330 con 0x5604383e3990 2024-09-12T02:03:03.424 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.418+0000 7febcdffb640 1 -- 172.21.15.16:0/2939655528 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7febe0004980 con 0x5604383e3990 2024-09-12T02:03:03.424 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.418+0000 7febcdffb640 1 -- 172.21.15.16:0/2939655528 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7febd809ac70 con 0x5604383e3990 2024-09-12T02:03:03.425 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.418+0000 7febefa8b740 10 client.15588 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:03:03.425 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.418+0000 7febefa8b740 10 client.15588 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:03:03.425 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.418+0000 7febefa8b740 20 client.15588 populate_metadata read hostname 'smithi016' 2024-09-12T02:03:03.425 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.418+0000 7febefa8b740 1 --2- 172.21.15.16:0/2939655528 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5604383f9820 0x560438466b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:03.425 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.419+0000 7febefa8b740 4 client.15588 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:03:03.425 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.419+0000 7febefa8b740 1 -- 172.21.15.16:0/2939655528 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56043834ed70 con 0x5604383f9820 2024-09-12T02:03:03.425 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.419+0000 7febe5d9b640 1 --2- 172.21.15.16:0/2939655528 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5604383f9820 0x560438466b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:03.425 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.419+0000 7febcdffb640 10 client.15588 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:03:03.425 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.419+0000 7febe5d9b640 1 --2- 172.21.15.16:0/2939655528 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5604383f9820 0x560438466b30 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be8be83dac4bac91 server_cookie=e87177d1412055fa in_seq=0 out_seq=0 2024-09-12T02:03:03.427 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.426+0000 7febcdffb640 1 -- 172.21.15.16:0/2939655528 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56043834ed70 con 0x5604383f9820 2024-09-12T02:03:03.427 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.426+0000 7febcdffb640 10 client.15588 handle_command_reply: tid=0 2024-09-12T02:03:03.427 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.426+0000 7febefa8b740 1 client.15588 shutdown 2024-09-12T02:03:03.427 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.426+0000 7febefa8b740 1 -- 172.21.15.16:0/2939655528 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5604383e3990 msgr2=0x5604383f24a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:03.427 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.426+0000 7febefa8b740 1 --2- 172.21.15.16:0/2939655528 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5604383e3990 0x5604383f24a0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7febd8097c80 tx=0x7febd800bf60 comp rx=0 tx=0).stop 2024-09-12T02:03:03.427 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.426+0000 7febefa8b740 1 -- 172.21.15.16:0/2939655528 shutdown_connections 2024-09-12T02:03:03.427 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.426+0000 7febefa8b740 1 --2- 172.21.15.16:0/2939655528 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5604383f9820 0x560438466b30 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:03.427 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.426+0000 7febefa8b740 1 --2- 172.21.15.16:0/2939655528 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5604383e9360 0x5604383f39a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:03.427 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.426+0000 7febcdffb640 0 client.15588 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:03:03.427 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.426+0000 7febefa8b740 1 --2- 172.21.15.16:0/2939655528 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5604383e42d0 0x5604383f29e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:03.427 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.426+0000 7febefa8b740 1 --2- 172.21.15.16:0/2939655528 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5604383e3990 0x5604383f24a0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:03.427 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.426+0000 7febefa8b740 1 -- 172.21.15.16:0/2939655528 >> 172.21.15.16:0/2939655528 conn(0x5604383df000 msgr2=0x5604383dd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:03.428 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.426+0000 7febefa8b740 1 -- 172.21.15.16:0/2939655528 shutdown_connections 2024-09-12T02:03:03.428 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.426+0000 7febefa8b740 1 -- 172.21.15.16:0/2939655528 wait complete. 2024-09-12T02:03:03.428 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.426+0000 7febefa8b740 20 client.15588 trim_cache size 0 max 16384 2024-09-12T02:03:03.428 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-09-12T02:03:03.428 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-09-12T02:03:03.428 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-09-12T02:03:03.428 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-09-12T02:03:03.429 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.427+0000 7febeccf1640 1 -- 172.21.15.16:0/3725426405 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7febd00a2c30 msgr2=0x7febd00a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:03.429 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.427+0000 7febeccf1640 1 --2- 172.21.15.16:0/3725426405 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7febd00a2c30 0x7febd00a50f0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7febe0011010 tx=0x7febe0011db0 comp rx=0 tx=0).stop 2024-09-12T02:03:03.429 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.427+0000 7febeccf1640 1 -- 172.21.15.16:0/3725426405 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febe8100a50 msgr2=0x7febe8197dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:03.429 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.427+0000 7febeccf1640 1 --2- 172.21.15.16:0/3725426405 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febe8100a50 0x7febe8197dd0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7febd800d950 tx=0x7febd800de10 comp rx=0 tx=0).stop 2024-09-12T02:03:03.429 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.427+0000 7febeccf1640 1 -- 172.21.15.16:0/3725426405 shutdown_connections 2024-09-12T02:03:03.429 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.427+0000 7febeccf1640 1 --2- 172.21.15.16:0/3725426405 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7febd00a2c30 0x7febd00a50f0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:03.430 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.427+0000 7febeccf1640 1 --2- 172.21.15.16:0/3725426405 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7febe8102600 0x7febe819f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:03.430 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.427+0000 7febeccf1640 1 --2- 172.21.15.16:0/3725426405 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7febe8101c40 0x7febe8198310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:03.430 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.427+0000 7febeccf1640 1 --2- 172.21.15.16:0/3725426405 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febe8100a50 0x7febe8197dd0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:03.430 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.427+0000 7febeccf1640 1 -- 172.21.15.16:0/3725426405 >> 172.21.15.16:0/3725426405 conn(0x7febe80fc1b0 msgr2=0x7febe80fdc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:03.430 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.427+0000 7febeccf1640 1 -- 172.21.15.16:0/3725426405 shutdown_connections 2024-09-12T02:03:03.430 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:03.427+0000 7febeccf1640 1 -- 172.21.15.16:0/3725426405 wait complete. 2024-09-12T02:03:03.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:03:03 smithi184 ceph-mon[35593]: pgmap v369: 97 pgs: 97 active+clean; 668 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-12T02:03:03.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:03:03 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:03:03.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:03:03 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T02:03:03.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:03:03 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T02:03:03.878 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:03:03 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:03:03.878 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:03:03 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:03:03.878 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:03:03 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T02:03:03.918 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:03:03 smithi033 ceph-mon[35332]: pgmap v369: 97 pgs: 97 active+clean; 668 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-12T02:03:03.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:03:03 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:03:03.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:03:03 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T02:03:03.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:03:03 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T02:03:03.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:03:03 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:03:03.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:03:03 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:03:03.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:03:03 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T02:03:03.933 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:03:03 smithi016 ceph-mon[30947]: pgmap v369: 97 pgs: 97 active+clean; 668 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-12T02:03:03.933 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:03:03 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:03:03.933 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:03:03 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T02:03:03.933 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:03:03 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T02:03:03.933 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:03:03 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:03:03.933 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:03:03 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:03:03.933 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:03:03 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T02:03:05.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:03:05 smithi184 ceph-mon[35593]: pgmap v370: 97 pgs: 97 active+clean; 719 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 5.4 KiB/s wr, 8 op/s 2024-09-12T02:03:05.918 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:03:05 smithi033 ceph-mon[35332]: pgmap v370: 97 pgs: 97 active+clean; 719 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 5.4 KiB/s wr, 8 op/s 2024-09-12T02:03:05.934 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:03:05 smithi016 ceph-mon[30947]: pgmap v370: 97 pgs: 97 active+clean; 719 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 5.4 KiB/s wr, 8 op/s 2024-09-12T02:03:06.197 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T02:03:06.470 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:03:06.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.927+0000 7f19de499640 1 -- 172.21.15.16:0/1709586911 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19d8102570 msgr2=0x7f19d81029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:06.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.927+0000 7f19de499640 1 --2- 172.21.15.16:0/1709586911 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19d8102570 0x7f19d81029b0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f19cc009900 tx=0x7f19cc02a480 comp rx=0 tx=0).stop 2024-09-12T02:03:06.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.927+0000 7f19de499640 1 -- 172.21.15.16:0/1709586911 shutdown_connections 2024-09-12T02:03:06.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.927+0000 7f19de499640 1 --2- 172.21.15.16:0/1709586911 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f19d8102ef0 0x7f19d810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:06.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.927+0000 7f19de499640 1 --2- 172.21.15.16:0/1709586911 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19d8102570 0x7f19d81029b0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:06.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.927+0000 7f19de499640 1 --2- 172.21.15.16:0/1709586911 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f19d8100e20 0x7f19d8101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:06.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.927+0000 7f19de499640 1 -- 172.21.15.16:0/1709586911 >> 172.21.15.16:0/1709586911 conn(0x7f19d80fc7d0 msgr2=0x7f19d80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:06.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.927+0000 7f19de499640 1 -- 172.21.15.16:0/1709586911 shutdown_connections 2024-09-12T02:03:06.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.927+0000 7f19de499640 1 -- 172.21.15.16:0/1709586911 wait complete. 2024-09-12T02:03:06.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.927+0000 7f19de499640 1 Processor -- start 2024-09-12T02:03:06.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.928+0000 7f19de499640 1 -- start start 2024-09-12T02:03:06.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.928+0000 7f19de499640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f19d8100e20 0x7f19d80fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:06.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.928+0000 7f19de499640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19d8102570 0x7f19d80fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:06.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.928+0000 7f19de499640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f19d8102ef0 0x7f19d80fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:06.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.928+0000 7f19de499640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f19d8110f50 con 0x7f19d8102570 2024-09-12T02:03:06.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.928+0000 7f19de499640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f19d8110dd0 con 0x7f19d8102ef0 2024-09-12T02:03:06.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.928+0000 7f19de499640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f19d81110d0 con 0x7f19d8100e20 2024-09-12T02:03:06.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.928+0000 7f19d77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19d8102570 0x7f19d80fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:06.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.928+0000 7f19d77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19d8102570 0x7f19d80fd1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46048/0 (socket says 172.21.15.16:46048) 2024-09-12T02:03:06.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.928+0000 7f19d77fe640 1 -- 172.21.15.16:0/2470708668 learned_addr learned my addr 172.21.15.16:0/2470708668 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:03:06.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.928+0000 7f19dca36640 1 --2- 172.21.15.16:0/2470708668 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f19d8102ef0 0x7f19d80fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:06.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.928+0000 7f19d7fff640 1 --2- 172.21.15.16:0/2470708668 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f19d8100e20 0x7f19d80fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:06.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.928+0000 7f19d77fe640 1 -- 172.21.15.16:0/2470708668 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f19d8100e20 msgr2=0x7f19d80fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:06.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.928+0000 7f19d77fe640 1 --2- 172.21.15.16:0/2470708668 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f19d8100e20 0x7f19d80fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:06.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.928+0000 7f19d77fe640 1 -- 172.21.15.16:0/2470708668 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f19d8102ef0 msgr2=0x7f19d80fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:06.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.928+0000 7f19d77fe640 1 --2- 172.21.15.16:0/2470708668 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f19d8102ef0 0x7f19d80fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:06.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.928+0000 7f19d77fe640 1 -- 172.21.15.16:0/2470708668 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19d81a6940 con 0x7f19d8102570 2024-09-12T02:03:06.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.929+0000 7f19d77fe640 1 --2- 172.21.15.16:0/2470708668 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19d8102570 0x7f19d80fd1c0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f19cc006010 tx=0x7f19cc003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:06.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.929+0000 7f19d57fa640 1 -- 172.21.15.16:0/2470708668 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19cc03f070 con 0x7f19d8102570 2024-09-12T02:03:06.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.929+0000 7f19de499640 1 -- 172.21.15.16:0/2470708668 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19d81a6bd0 con 0x7f19d8102570 2024-09-12T02:03:06.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.929+0000 7f19d57fa640 1 -- 172.21.15.16:0/2470708668 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f19cc031c30 con 0x7f19d8102570 2024-09-12T02:03:06.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.929+0000 7f19d57fa640 1 -- 172.21.15.16:0/2470708668 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19cc035030 con 0x7f19d8102570 2024-09-12T02:03:06.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.929+0000 7f19de499640 1 -- 172.21.15.16:0/2470708668 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19d81a70b0 con 0x7f19d8102570 2024-09-12T02:03:06.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.931+0000 7f19d57fa640 1 -- 172.21.15.16:0/2470708668 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f19cc042070 con 0x7f19d8102570 2024-09-12T02:03:06.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.932+0000 7f19e0872740 1 Processor -- start 2024-09-12T02:03:06.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.932+0000 7f19e0872740 1 -- start start 2024-09-12T02:03:06.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.932+0000 7f19e0872740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56500c9314a0 0x56500c931880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:06.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.932+0000 7f19e0872740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56500c931dc0 0x56500c93c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:06.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.932+0000 7f19e0872740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56500c93c680 0x56500c93ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:06.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.932+0000 7f19e0872740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56500c8972e0 con 0x56500c93c680 2024-09-12T02:03:06.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.932+0000 7f19e0872740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56500c83ece0 con 0x56500c9314a0 2024-09-12T02:03:06.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.932+0000 7f19e0872740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x56500c8932a0 con 0x56500c931dc0 2024-09-12T02:03:06.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.932+0000 7f19d77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56500c93c680 0x56500c93ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:06.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.932+0000 7f19d77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56500c93c680 0x56500c93ea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46054/0 (socket says 172.21.15.16:46054) 2024-09-12T02:03:06.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.932+0000 7f19dca36640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56500c9314a0 0x56500c931880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:06.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.932+0000 7f19d77fe640 1 -- 172.21.15.16:0/2031192156 learned_addr learned my addr 172.21.15.16:0/2031192156 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:03:06.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.932+0000 7f19d7fff640 1 --2- 172.21.15.16:0/2031192156 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56500c931dc0 0x56500c93c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:06.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.932+0000 7f19d57fa640 1 --2- 172.21.15.16:0/2470708668 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f19c00a2b80 0x7f19c00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:06.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.933+0000 7f19dca36640 1 -- 172.21.15.16:0/2031192156 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56500c931dc0 msgr2=0x56500c93c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:06.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.933+0000 7f19dca36640 1 --2- 172.21.15.16:0/2031192156 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56500c931dc0 0x56500c93c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:06.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.933+0000 7f19dca36640 1 -- 172.21.15.16:0/2031192156 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56500c93c680 msgr2=0x56500c93ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:06.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.933+0000 7f19dca36640 1 --2- 172.21.15.16:0/2031192156 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56500c93c680 0x56500c93ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:06.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.933+0000 7f19dca36640 1 -- 172.21.15.16:0/2031192156 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56500c823440 con 0x56500c9314a0 2024-09-12T02:03:06.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.933+0000 7f19dca36640 1 --2- 172.21.15.16:0/2470708668 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f19c00a2b80 0x7f19c00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:06.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.933+0000 7f19d57fa640 1 -- 172.21.15.16:0/2470708668 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f19cc0ef5e0 con 0x7f19d8102570 2024-09-12T02:03:06.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.933+0000 7f19dca36640 1 --2- 172.21.15.16:0/2031192156 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56500c9314a0 0x56500c931880 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f19d0009fa0 tx=0x7f19d00118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fdc3eb409c081a3a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:06.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.933+0000 7f19be7fc640 1 -- 172.21.15.16:0/2031192156 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19d0017050 con 0x56500c9314a0 2024-09-12T02:03:06.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.933+0000 7f19be7fc640 1 -- 172.21.15.16:0/2031192156 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f19d0003bb0 con 0x56500c9314a0 2024-09-12T02:03:06.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.933+0000 7f19be7fc640 1 -- 172.21.15.16:0/2031192156 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19d001b990 con 0x56500c9314a0 2024-09-12T02:03:06.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.933+0000 7f19dca36640 1 --2- 172.21.15.16:0/2470708668 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f19c00a2b80 0x7f19c00a5040 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f19d0022000 tx=0x7f19d0023000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:06.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.934+0000 7f19e0872740 1 -- 172.21.15.16:0/2031192156 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56500c9314a0 msgr2=0x56500c931880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:06.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.934+0000 7f19e0872740 1 --2- 172.21.15.16:0/2031192156 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56500c9314a0 0x56500c931880 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f19d0009fa0 tx=0x7f19d00118d0 comp rx=0 tx=0).stop 2024-09-12T02:03:06.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.934+0000 7f19e0872740 1 -- 172.21.15.16:0/2031192156 shutdown_connections 2024-09-12T02:03:06.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.934+0000 7f19e0872740 1 --2- 172.21.15.16:0/2031192156 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56500c93c680 0x56500c93ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:06.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.934+0000 7f19e0872740 1 --2- 172.21.15.16:0/2031192156 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56500c931dc0 0x56500c93c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:06.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.934+0000 7f19e0872740 1 --2- 172.21.15.16:0/2031192156 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56500c9314a0 0x56500c931880 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:06.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.934+0000 7f19e0872740 1 -- 172.21.15.16:0/2031192156 >> 172.21.15.16:0/2031192156 conn(0x56500c92da30 msgr2=0x56500c92de10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:06.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.934+0000 7f19e0872740 1 -- 172.21.15.16:0/2031192156 shutdown_connections 2024-09-12T02:03:06.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.934+0000 7f19e0872740 1 -- 172.21.15.16:0/2031192156 wait complete. 2024-09-12T02:03:06.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.934+0000 7f19e0872740 1 -- start start 2024-09-12T02:03:06.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.935+0000 7f19e0872740 1 Processor -- start 2024-09-12T02:03:06.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.935+0000 7f19e0872740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56500c9314a0 0x56500c93bf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:06.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.935+0000 7f19e0872740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56500c931dc0 0x56500c939210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:06.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.935+0000 7f19e0872740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56500c93c680 0x56500c939750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:06.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.935+0000 7f19e0872740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56500c93f250 con 0x56500c93c680 2024-09-12T02:03:06.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.935+0000 7f19e0872740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56500c852050 con 0x56500c931dc0 2024-09-12T02:03:06.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.935+0000 7f19e0872740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x56500c93f3d0 con 0x56500c9314a0 2024-09-12T02:03:06.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.935+0000 7f19e0872740 10 client.0 _ll_register_callbacks cb 0x56500c892160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:03:06.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.935+0000 7f19dca36640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56500c93c680 0x56500c939750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:06.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.935+0000 7f19dca36640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56500c93c680 0x56500c939750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46058/0 (socket says 172.21.15.16:46058) 2024-09-12T02:03:06.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.935+0000 7f19dca36640 1 -- 172.21.15.16:0/1318295765 learned_addr learned my addr 172.21.15.16:0/1318295765 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:03:06.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.935+0000 7f19d7fff640 1 --2- 172.21.15.16:0/1318295765 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56500c9314a0 0x56500c93bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:06.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.935+0000 7f19d77fe640 1 --2- 172.21.15.16:0/1318295765 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56500c931dc0 0x56500c939210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:06.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.935+0000 7f19dca36640 1 -- 172.21.15.16:0/1318295765 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56500c9314a0 msgr2=0x56500c93bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:06.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.935+0000 7f19dca36640 1 --2- 172.21.15.16:0/1318295765 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56500c9314a0 0x56500c93bf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:06.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.935+0000 7f19dca36640 1 -- 172.21.15.16:0/1318295765 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56500c931dc0 msgr2=0x56500c939210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:06.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.935+0000 7f19dca36640 1 --2- 172.21.15.16:0/1318295765 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56500c931dc0 0x56500c939210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:06.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.935+0000 7f19dca36640 1 -- 172.21.15.16:0/1318295765 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56500c6083c0 con 0x56500c93c680 2024-09-12T02:03:06.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.936+0000 7f19dca36640 1 --2- 172.21.15.16:0/1318295765 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56500c93c680 0x56500c939750 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f19d001b510 tx=0x7f19d001b940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cff20d62f8f09f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:06.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.936+0000 7f19b77fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:03:06.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.936+0000 7f19b77fe640 1 -- 172.21.15.16:0/1318295765 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19d0004c10 con 0x56500c93c680 2024-09-12T02:03:06.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.936+0000 7f19b77fe640 1 -- 172.21.15.16:0/1318295765 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f19d0004db0 con 0x56500c93c680 2024-09-12T02:03:06.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.936+0000 7f19e0872740 1 -- 172.21.15.16:0/1318295765 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56500c8a4850 con 0x56500c93c680 2024-09-12T02:03:06.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.936+0000 7f19b77fe640 1 -- 172.21.15.16:0/1318295765 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f19d000f4e0 con 0x56500c93c680 2024-09-12T02:03:06.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.936+0000 7f19e0872740 10 client.15600 fetch_fsmap learned FSMap version 25 2024-09-12T02:03:06.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.936+0000 7f19e0872740 1 -- 172.21.15.16:0/1318295765 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56500c94ac40 con 0x56500c93c680 2024-09-12T02:03:06.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.936+0000 7f19b77fe640 1 -- 172.21.15.16:0/1318295765 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x56500c8a4850 con 0x56500c93c680 2024-09-12T02:03:06.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.936+0000 7f19b77fe640 1 -- 172.21.15.16:0/1318295765 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f19d001a070 con 0x56500c93c680 2024-09-12T02:03:06.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.936+0000 7f19e0872740 10 client.15600 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:03:06.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.936+0000 7f19e0872740 10 client.15600 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:03:06.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.937+0000 7f19e0872740 20 client.15600 populate_metadata read hostname 'smithi016' 2024-09-12T02:03:06.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.937+0000 7f19e0872740 1 --2- 172.21.15.16:0/1318295765 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x56500c94bff0 0x56500c94e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:06.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.937+0000 7f19e0872740 4 client.15600 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:03:06.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.937+0000 7f19e0872740 1 -- 172.21.15.16:0/1318295765 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56500c89ed60 con 0x56500c94bff0 2024-09-12T02:03:06.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.937+0000 7f19d7fff640 1 --2- 172.21.15.16:0/1318295765 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x56500c94bff0 0x56500c94e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:06.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.937+0000 7f19d7fff640 1 --2- 172.21.15.16:0/1318295765 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x56500c94bff0 0x56500c94e3e0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c02574dab5a32df server_cookie=cc2e40bfb08417c8 in_seq=0 out_seq=0 2024-09-12T02:03:06.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.937+0000 7f19b77fe640 10 client.15600 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:03:06.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.938+0000 7f19b77fe640 1 -- 172.21.15.16:0/1318295765 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56500c89ed60 con 0x56500c94bff0 2024-09-12T02:03:06.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.938+0000 7f19b77fe640 10 client.15600 handle_command_reply: tid=0 2024-09-12T02:03:06.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.938+0000 7f19e0872740 1 client.15600 shutdown 2024-09-12T02:03:06.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.938+0000 7f19e0872740 1 -- 172.21.15.16:0/1318295765 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56500c93c680 msgr2=0x56500c939750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:06.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.938+0000 7f19e0872740 1 --2- 172.21.15.16:0/1318295765 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56500c93c680 0x56500c939750 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f19d001b510 tx=0x7f19d001b940 comp rx=0 tx=0).stop 2024-09-12T02:03:06.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.938+0000 7f19e0872740 1 -- 172.21.15.16:0/1318295765 shutdown_connections 2024-09-12T02:03:06.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.938+0000 7f19e0872740 1 --2- 172.21.15.16:0/1318295765 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x56500c94bff0 0x56500c94e3e0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:06.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.938+0000 7f19e0872740 1 --2- 172.21.15.16:0/1318295765 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56500c93c680 0x56500c939750 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:06.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.938+0000 7f19e0872740 1 --2- 172.21.15.16:0/1318295765 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56500c931dc0 0x56500c939210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:06.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.938+0000 7f19b77fe640 0 client.15600 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:03:06.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.938+0000 7f19e0872740 1 --2- 172.21.15.16:0/1318295765 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56500c9314a0 0x56500c93bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:06.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.938+0000 7f19e0872740 1 -- 172.21.15.16:0/1318295765 >> 172.21.15.16:0/1318295765 conn(0x56500c92da30 msgr2=0x56500c933710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:06.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.938+0000 7f19e0872740 1 -- 172.21.15.16:0/1318295765 shutdown_connections 2024-09-12T02:03:06.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.938+0000 7f19e0872740 1 -- 172.21.15.16:0/1318295765 wait complete. 2024-09-12T02:03:06.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.938+0000 7f19e0872740 20 client.15600 trim_cache size 0 max 16384 2024-09-12T02:03:06.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.956+0000 7f19e0872740 1 Processor -- start 2024-09-12T02:03:06.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.956+0000 7f19e0872740 1 -- start start 2024-09-12T02:03:06.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.956+0000 7f19e0872740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56500c9314a0 0x56500c9484e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:06.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.956+0000 7f19e0872740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56500c931dc0 0x56500c953ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:06.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.956+0000 7f19e0872740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56500c93c680 0x56500c9540e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:06.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.956+0000 7f19e0872740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f19d81b1dd0 con 0x56500c931dc0 2024-09-12T02:03:06.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.956+0000 7f19e0872740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f19d81af6a0 con 0x56500c93c680 2024-09-12T02:03:06.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.956+0000 7f19e0872740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f19d81ad510 con 0x56500c9314a0 2024-09-12T02:03:06.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.956+0000 7f19d77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56500c931dc0 0x56500c953ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:06.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.956+0000 7f19d77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56500c931dc0 0x56500c953ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46072/0 (socket says 172.21.15.16:46072) 2024-09-12T02:03:06.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.956+0000 7f19d77fe640 1 -- 172.21.15.16:0/3661716430 learned_addr learned my addr 172.21.15.16:0/3661716430 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:03:06.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.956+0000 7f19d7fff640 1 --2- 172.21.15.16:0/3661716430 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56500c9314a0 0x56500c9484e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:06.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.957+0000 7f19dca36640 1 --2- 172.21.15.16:0/3661716430 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56500c93c680 0x56500c9540e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:06.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.957+0000 7f19d77fe640 1 -- 172.21.15.16:0/3661716430 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56500c9314a0 msgr2=0x56500c9484e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:06.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.957+0000 7f19d77fe640 1 --2- 172.21.15.16:0/3661716430 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56500c9314a0 0x56500c9484e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:06.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.957+0000 7f19d77fe640 1 -- 172.21.15.16:0/3661716430 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56500c93c680 msgr2=0x56500c9540e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:06.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.957+0000 7f19d77fe640 1 --2- 172.21.15.16:0/3661716430 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56500c93c680 0x56500c9540e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:06.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.957+0000 7f19d77fe640 1 -- 172.21.15.16:0/3661716430 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56500c94e750 con 0x56500c931dc0 2024-09-12T02:03:06.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.957+0000 7f19d77fe640 1 --2- 172.21.15.16:0/3661716430 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56500c931dc0 0x56500c953ba0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f19cc000c00 tx=0x7f19cc005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3226b46c4d84639b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:06.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.957+0000 7f19bdffb640 1 -- 172.21.15.16:0/3661716430 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19cc049040 con 0x56500c931dc0 2024-09-12T02:03:06.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.957+0000 7f19bdffb640 1 -- 172.21.15.16:0/3661716430 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f19cc0bc5e0 con 0x56500c931dc0 2024-09-12T02:03:06.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.957+0000 7f19bdffb640 1 -- 172.21.15.16:0/3661716430 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19cc0c0290 con 0x56500c931dc0 2024-09-12T02:03:06.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.958+0000 7f19e0872740 1 -- 172.21.15.16:0/3661716430 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56500c931dc0 msgr2=0x56500c953ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:06.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.958+0000 7f19e0872740 1 --2- 172.21.15.16:0/3661716430 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56500c931dc0 0x56500c953ba0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f19cc000c00 tx=0x7f19cc005c00 comp rx=0 tx=0).stop 2024-09-12T02:03:06.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.958+0000 7f19e0872740 1 -- 172.21.15.16:0/3661716430 shutdown_connections 2024-09-12T02:03:06.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.958+0000 7f19e0872740 1 --2- 172.21.15.16:0/3661716430 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56500c93c680 0x56500c9540e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:06.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.958+0000 7f19e0872740 1 --2- 172.21.15.16:0/3661716430 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56500c931dc0 0x56500c953ba0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:06.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.958+0000 7f19e0872740 1 --2- 172.21.15.16:0/3661716430 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56500c9314a0 0x56500c9484e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:06.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.958+0000 7f19e0872740 1 -- 172.21.15.16:0/3661716430 >> 172.21.15.16:0/3661716430 conn(0x56500c92da30 msgr2=0x56500c938300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:06.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.958+0000 7f19e0872740 1 -- 172.21.15.16:0/3661716430 shutdown_connections 2024-09-12T02:03:06.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.958+0000 7f19e0872740 1 -- 172.21.15.16:0/3661716430 wait complete. 2024-09-12T02:03:06.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.958+0000 7f19e0872740 1 -- start start 2024-09-12T02:03:06.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.958+0000 7f19e0872740 1 Processor -- start 2024-09-12T02:03:06.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.959+0000 7f19e0872740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56500c9314a0 0x56500c941ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:06.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.959+0000 7f19e0872740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56500c931dc0 0x56500c942000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:06.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.959+0000 7f19e0872740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56500c93c680 0x56500c9b4b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:06.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.959+0000 7f19e0872740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f19d81adb60 con 0x56500c93c680 2024-09-12T02:03:06.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.959+0000 7f19e0872740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f19d81aefd0 con 0x56500c931dc0 2024-09-12T02:03:06.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.959+0000 7f19e0872740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f19d81ae230 con 0x56500c9314a0 2024-09-12T02:03:06.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.959+0000 7f19e0872740 10 client.0 _ll_register_callbacks cb 0x7f19cc0f5400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:03:06.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.959+0000 7f19dca36640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56500c93c680 0x56500c9b4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:06.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.959+0000 7f19d7fff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56500c9314a0 0x56500c941ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:06.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.959+0000 7f19dca36640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56500c93c680 0x56500c9b4b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46080/0 (socket says 172.21.15.16:46080) 2024-09-12T02:03:06.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.959+0000 7f19dca36640 1 -- 172.21.15.16:0/3982670668 learned_addr learned my addr 172.21.15.16:0/3982670668 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:03:06.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.959+0000 7f19d77fe640 1 --2- 172.21.15.16:0/3982670668 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56500c931dc0 0x56500c942000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:06.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.959+0000 7f19dca36640 1 -- 172.21.15.16:0/3982670668 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56500c9314a0 msgr2=0x56500c941ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:06.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.959+0000 7f19dca36640 1 --2- 172.21.15.16:0/3982670668 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56500c9314a0 0x56500c941ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:06.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.959+0000 7f19dca36640 1 -- 172.21.15.16:0/3982670668 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56500c931dc0 msgr2=0x56500c942000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:06.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.959+0000 7f19dca36640 1 --2- 172.21.15.16:0/3982670668 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56500c931dc0 0x56500c942000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:06.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.959+0000 7f19dca36640 1 -- 172.21.15.16:0/3982670668 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56500c9b50b0 con 0x56500c93c680 2024-09-12T02:03:06.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.959+0000 7f19d7fff640 1 --2- 172.21.15.16:0/3982670668 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56500c9314a0 0x56500c941ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:03:06.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.959+0000 7f19dca36640 1 --2- 172.21.15.16:0/3982670668 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56500c93c680 0x56500c9b4b70 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f19d0004000 tx=0x7f19d001b9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bef452e0391ee4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:06.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.959+0000 7f19b77fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:03:06.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.960+0000 7f19b77fe640 1 -- 172.21.15.16:0/3982670668 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19d0017040 con 0x56500c93c680 2024-09-12T02:03:06.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.960+0000 7f19b77fe640 1 -- 172.21.15.16:0/3982670668 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f19d0005830 con 0x56500c93c680 2024-09-12T02:03:06.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.960+0000 7f19e0872740 1 -- 172.21.15.16:0/3982670668 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f19cc0bc5e0 con 0x56500c93c680 2024-09-12T02:03:06.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.960+0000 7f19b77fe640 1 -- 172.21.15.16:0/3982670668 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f19d0009b30 con 0x56500c93c680 2024-09-12T02:03:06.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.960+0000 7f19e0872740 10 client.15606 fetch_fsmap learned FSMap version 25 2024-09-12T02:03:06.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.960+0000 7f19e0872740 1 -- 172.21.15.16:0/3982670668 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56500c9b6390 con 0x56500c93c680 2024-09-12T02:03:06.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.960+0000 7f19b77fe640 1 -- 172.21.15.16:0/3982670668 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f19cc0bc5e0 con 0x56500c93c680 2024-09-12T02:03:06.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.960+0000 7f19b77fe640 1 -- 172.21.15.16:0/3982670668 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f19d001a070 con 0x56500c93c680 2024-09-12T02:03:06.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.960+0000 7f19e0872740 10 client.15606 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:03:06.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.960+0000 7f19e0872740 10 client.15606 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:03:06.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.960+0000 7f19e0872740 20 client.15606 populate_metadata read hostname 'smithi016' 2024-09-12T02:03:06.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.960+0000 7f19e0872740 1 --2- 172.21.15.16:0/3982670668 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x56500c9b6a50 0x56500c9b8e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:06.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.960+0000 7f19e0872740 4 client.15606 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T02:03:06.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.960+0000 7f19e0872740 1 -- 172.21.15.16:0/3982670668 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56500c81b210 con 0x56500c9b6a50 2024-09-12T02:03:06.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.961+0000 7f19d7fff640 1 --2- 172.21.15.16:0/3982670668 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x56500c9b6a50 0x56500c9b8e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:06.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.961+0000 7f19d7fff640 1 --2- 172.21.15.16:0/3982670668 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x56500c9b6a50 0x56500c9b8e70 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1430b00ec0628d7 server_cookie=442dbc1e1c7d0734 in_seq=0 out_seq=0 2024-09-12T02:03:06.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.961+0000 7f19b77fe640 10 client.15606 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:03:06.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.961+0000 7f19b77fe640 1 -- 172.21.15.16:0/3982670668 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56500c81b210 con 0x56500c9b6a50 2024-09-12T02:03:06.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.961+0000 7f19b77fe640 10 client.15606 handle_command_reply: tid=0 2024-09-12T02:03:06.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.961+0000 7f19e0872740 1 client.15606 shutdown 2024-09-12T02:03:06.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.961+0000 7f19e0872740 1 -- 172.21.15.16:0/3982670668 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56500c93c680 msgr2=0x56500c9b4b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:06.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.961+0000 7f19e0872740 1 --2- 172.21.15.16:0/3982670668 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56500c93c680 0x56500c9b4b70 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f19d0004000 tx=0x7f19d001b9b0 comp rx=0 tx=0).stop 2024-09-12T02:03:06.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.961+0000 7f19e0872740 1 -- 172.21.15.16:0/3982670668 shutdown_connections 2024-09-12T02:03:06.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.961+0000 7f19e0872740 1 --2- 172.21.15.16:0/3982670668 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x56500c9b6a50 0x56500c9b8e70 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:06.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.961+0000 7f19e0872740 1 --2- 172.21.15.16:0/3982670668 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56500c93c680 0x56500c9b4b70 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:06.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.961+0000 7f19e0872740 1 --2- 172.21.15.16:0/3982670668 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56500c931dc0 0x56500c942000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:06.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.961+0000 7f19e0872740 1 --2- 172.21.15.16:0/3982670668 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56500c9314a0 0x56500c941ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:06.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.961+0000 7f19b77fe640 0 client.15606 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:03:06.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.961+0000 7f19e0872740 1 -- 172.21.15.16:0/3982670668 >> 172.21.15.16:0/3982670668 conn(0x56500c92da30 msgr2=0x56500c92c350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:06.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.961+0000 7f19e0872740 1 -- 172.21.15.16:0/3982670668 shutdown_connections 2024-09-12T02:03:06.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.962+0000 7f19e0872740 1 -- 172.21.15.16:0/3982670668 wait complete. 2024-09-12T02:03:06.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.962+0000 7f19e0872740 20 client.15606 trim_cache size 0 max 16384 2024-09-12T02:03:06.968 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T02:03:06.968 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-09-12T02:03:06.968 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {} 2024-09-12T02:03:06.968 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T02:03:06.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.962+0000 7f19de499640 1 -- 172.21.15.16:0/2470708668 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f19c00a2b80 msgr2=0x7f19c00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:06.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.962+0000 7f19de499640 1 --2- 172.21.15.16:0/2470708668 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f19c00a2b80 0x7f19c00a5040 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f19d0022000 tx=0x7f19d0023000 comp rx=0 tx=0).stop 2024-09-12T02:03:06.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.962+0000 7f19de499640 1 -- 172.21.15.16:0/2470708668 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19d8102570 msgr2=0x7f19d80fd1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:06.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.962+0000 7f19de499640 1 --2- 172.21.15.16:0/2470708668 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19d8102570 0x7f19d80fd1c0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f19cc006010 tx=0x7f19cc003c10 comp rx=0 tx=0).stop 2024-09-12T02:03:06.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.962+0000 7f19de499640 1 -- 172.21.15.16:0/2470708668 shutdown_connections 2024-09-12T02:03:06.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.962+0000 7f19de499640 1 --2- 172.21.15.16:0/2470708668 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f19c00a2b80 0x7f19c00a5040 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:06.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.962+0000 7f19de499640 1 --2- 172.21.15.16:0/2470708668 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f19d8102ef0 0x7f19d80fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:06.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.962+0000 7f19de499640 1 --2- 172.21.15.16:0/2470708668 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19d8102570 0x7f19d80fd1c0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:06.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.962+0000 7f19de499640 1 --2- 172.21.15.16:0/2470708668 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f19d8100e20 0x7f19d80fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:06.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.962+0000 7f19de499640 1 -- 172.21.15.16:0/2470708668 >> 172.21.15.16:0/2470708668 conn(0x7f19d80fc7d0 msgr2=0x7f19d806a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:06.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.962+0000 7f19de499640 1 -- 172.21.15.16:0/2470708668 shutdown_connections 2024-09-12T02:03:06.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:06.962+0000 7f19de499640 1 -- 172.21.15.16:0/2470708668 wait complete. 2024-09-12T02:03:07.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:03:07 smithi016 ceph-mon[30947]: pgmap v371: 97 pgs: 97 active+clean; 719 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 5.8 KiB/s wr, 10 op/s 2024-09-12T02:03:07.776 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-12T02:03:07.777 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:03:07.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:03:07 smithi184 ceph-mon[35593]: pgmap v371: 97 pgs: 97 active+clean; 719 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 5.8 KiB/s wr, 10 op/s 2024-09-12T02:03:07.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:03:07 smithi033 ceph-mon[35332]: pgmap v371: 97 pgs: 97 active+clean; 719 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 5.8 KiB/s wr, 10 op/s 2024-09-12T02:03:08.045 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:03:08.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:08.511+0000 7fdd9b719640 1 -- 172.21.15.16:0/857140625 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd94103660 msgr2=0x7fdd9410bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:08.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:08.511+0000 7fdd9b719640 1 --2- 172.21.15.16:0/857140625 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd94103660 0x7fdd9410bb50 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fdd84009900 tx=0x7fdd8402a480 comp rx=0 tx=0).stop 2024-09-12T02:03:08.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:08.511+0000 7fdd9b719640 1 -- 172.21.15.16:0/857140625 shutdown_connections 2024-09-12T02:03:08.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:08.511+0000 7fdd9b719640 1 --2- 172.21.15.16:0/857140625 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdd9410c170 0x7fdd9410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:08.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:08.511+0000 7fdd9b719640 1 --2- 172.21.15.16:0/857140625 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd94103660 0x7fdd9410bb50 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:08.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:08.511+0000 7fdd9b719640 1 --2- 172.21.15.16:0/857140625 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdd94102d40 0x7fdd94103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:08.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:08.511+0000 7fdd9b719640 1 -- 172.21.15.16:0/857140625 >> 172.21.15.16:0/857140625 conn(0x7fdd940fc7a0 msgr2=0x7fdd940febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:08.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:08.511+0000 7fdd9b719640 1 -- 172.21.15.16:0/857140625 shutdown_connections 2024-09-12T02:03:08.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:08.511+0000 7fdd9b719640 1 -- 172.21.15.16:0/857140625 wait complete. 2024-09-12T02:03:08.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:08.512+0000 7fdd9b719640 1 Processor -- start 2024-09-12T02:03:08.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:08.512+0000 7fdd9b719640 1 -- start start 2024-09-12T02:03:08.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:08.512+0000 7fdd9b719640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdd94102d40 0x7fdd941009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:08.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:08.512+0000 7fdd9b719640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdd94103660 0x7fdd94100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:08.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:08.512+0000 7fdd9b719640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd9410c170 0x7fdd94101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:08.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:08.512+0000 7fdd9b719640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fdd94111f70 con 0x7fdd9410c170 2024-09-12T02:03:08.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:08.512+0000 7fdd9b719640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fdd94111df0 con 0x7fdd94102d40 2024-09-12T02:03:08.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:08.512+0000 7fdd9b719640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fdd941120f0 con 0x7fdd94103660 2024-09-12T02:03:08.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:08.512+0000 7fdd99cb6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd9410c170 0x7fdd94101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:08.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:08.512+0000 7fdd99cb6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd9410c170 0x7fdd94101420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46106/0 (socket says 172.21.15.16:46106) 2024-09-12T02:03:08.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:08.512+0000 7fdd99cb6640 1 -- 172.21.15.16:0/637175502 learned_addr learned my addr 172.21.15.16:0/637175502 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:03:08.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:08.512+0000 7fdd994b5640 1 --2- 172.21.15.16:0/637175502 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdd94102d40 0x7fdd941009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:08.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:08.513+0000 7fdd98cb4640 1 --2- 172.21.15.16:0/637175502 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdd94103660 0x7fdd94100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:08.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:08.513+0000 7fdd99cb6640 1 -- 172.21.15.16:0/637175502 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdd94103660 msgr2=0x7fdd94100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:08.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:08.513+0000 7fdd99cb6640 1 --2- 172.21.15.16:0/637175502 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdd94103660 0x7fdd94100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:08.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:08.513+0000 7fdd99cb6640 1 -- 172.21.15.16:0/637175502 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdd94102d40 msgr2=0x7fdd941009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:08.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:08.513+0000 7fdd99cb6640 1 --2- 172.21.15.16:0/637175502 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdd94102d40 0x7fdd941009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:08.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:08.513+0000 7fdd99cb6640 1 -- 172.21.15.16:0/637175502 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd941a68a0 con 0x7fdd9410c170 2024-09-12T02:03:08.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:08.513+0000 7fdd98cb4640 1 --2- 172.21.15.16:0/637175502 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdd94103660 0x7fdd94100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:03:08.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:08.513+0000 7fdd99cb6640 1 --2- 172.21.15.16:0/637175502 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd9410c170 0x7fdd94101420 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fdd8c00cce0 tx=0x7fdd8c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:08.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:08.513+0000 7fdd8a7fc640 1 -- 172.21.15.16:0/637175502 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd8c00f450 con 0x7fdd9410c170 2024-09-12T02:03:08.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:08.513+0000 7fdd9b719640 1 -- 172.21.15.16:0/637175502 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd941a6b90 con 0x7fdd9410c170 2024-09-12T02:03:08.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:08.513+0000 7fdd8a7fc640 1 -- 172.21.15.16:0/637175502 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd8c00fad0 con 0x7fdd9410c170 2024-09-12T02:03:08.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:08.513+0000 7fdd8a7fc640 1 -- 172.21.15.16:0/637175502 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd8c018c10 con 0x7fdd9410c170 2024-09-12T02:03:08.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:08.513+0000 7fdd9b719640 1 -- 172.21.15.16:0/637175502 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd941a70d0 con 0x7fdd9410c170 2024-09-12T02:03:08.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:08.514+0000 7fdd9b719640 1 -- 172.21.15.16:0/637175502 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd94103120 con 0x7fdd9410c170 2024-09-12T02:03:08.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:08.515+0000 7fdd8a7fc640 1 -- 172.21.15.16:0/637175502 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fdd8c00f5f0 con 0x7fdd9410c170 2024-09-12T02:03:08.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:08.516+0000 7fdd8a7fc640 1 --2- 172.21.15.16:0/637175502 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fdd800a2c00 0x7fdd800a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:08.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:08.516+0000 7fdd994b5640 1 --2- 172.21.15.16:0/637175502 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fdd800a2c00 0x7fdd800a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:08.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:08.516+0000 7fdd8a7fc640 1 -- 172.21.15.16:0/637175502 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fdd8c0cf640 con 0x7fdd9410c170 2024-09-12T02:03:08.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:08.517+0000 7fdd994b5640 1 --2- 172.21.15.16:0/637175502 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fdd800a2c00 0x7fdd800a50c0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fdd9000bbe0 tx=0x7fdd9000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:08.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:08.519+0000 7fdd8a7fc640 1 -- 172.21.15.16:0/637175502 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fdd8c014030 con 0x7fdd9410c170 2024-09-12T02:03:08.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:08.733+0000 7fdd9b719640 1 -- 172.21.15.16:0/637175502 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdd94101ed0 con 0x7fdd9410c170 2024-09-12T02:03:08.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:08.733+0000 7fdd8a7fc640 1 -- 172.21.15.16:0/637175502 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11911 (secure 0 0 0) 0x7fdd94101ed0 con 0x7fdd9410c170 2024-09-12T02:03:08.734 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T02:03:08.734 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":25,"btime":"2024-09-12T02:01:37:192021+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3279123279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3279123279},{"type":"v1","addr":"172.21.15.16:6835","nonce":3279123279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4119452382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4119452382},{"type":"v1","addr":"172.21.15.16:6837","nonce":4119452382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/3263823165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3263823165},{"type":"v1","addr":"172.21.15.16:6839","nonce":3263823165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/1510290732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":1510290732},{"type":"v1","addr":"172.21.15.16:6841","nonce":1510290732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/279674603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":279674603},{"type":"v1","addr":"172.21.15.33:6833","nonce":279674603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3555728028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3555728028},{"type":"v1","addr":"172.21.15.33:6835","nonce":3555728028}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T01:59:56.117608+0000","modified":"2024-09-12T02:01:36.294760+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24482,"mds_1":24481,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6833/1563374106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1563374106},{"type":"v1","addr":"172.21.15.184:6833","nonce":1563374106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6837/1892833868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1892833868},{"type":"v1","addr":"172.21.15.33:6837","nonce":1892833868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/1457679686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1457679686},{"type":"v1","addr":"172.21.15.184:6835","nonce":1457679686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.184:6837/1793393087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":1793393087},{"type":"v1","addr":"172.21.15.184:6837","nonce":1793393087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.184:6839/4203347941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":4203347941},{"type":"v1","addr":"172.21.15.184:6839","nonce":4203347941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.33:6839/4136670204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":4136670204},{"type":"v1","addr":"172.21.15.33:6839","nonce":4136670204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24481,24482,24469]},"id":1}]} 2024-09-12T02:03:08.734 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 25 2024-09-12T02:03:08.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:08.736+0000 7fdd9b719640 1 -- 172.21.15.16:0/637175502 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fdd800a2c00 msgr2=0x7fdd800a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:08.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:08.736+0000 7fdd9b719640 1 --2- 172.21.15.16:0/637175502 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fdd800a2c00 0x7fdd800a50c0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fdd9000bbe0 tx=0x7fdd9000b390 comp rx=0 tx=0).stop 2024-09-12T02:03:08.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:08.736+0000 7fdd9b719640 1 -- 172.21.15.16:0/637175502 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd9410c170 msgr2=0x7fdd94101420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:08.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:08.736+0000 7fdd9b719640 1 --2- 172.21.15.16:0/637175502 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd9410c170 0x7fdd94101420 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fdd8c00cce0 tx=0x7fdd8c00a620 comp rx=0 tx=0).stop 2024-09-12T02:03:08.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:08.736+0000 7fdd9b719640 1 -- 172.21.15.16:0/637175502 shutdown_connections 2024-09-12T02:03:08.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:08.736+0000 7fdd9b719640 1 --2- 172.21.15.16:0/637175502 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fdd800a2c00 0x7fdd800a50c0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:08.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:08.736+0000 7fdd9b719640 1 --2- 172.21.15.16:0/637175502 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd9410c170 0x7fdd94101420 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:08.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:08.736+0000 7fdd9b719640 1 --2- 172.21.15.16:0/637175502 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdd94103660 0x7fdd94100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:08.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:08.736+0000 7fdd9b719640 1 --2- 172.21.15.16:0/637175502 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdd94102d40 0x7fdd941009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:08.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:08.736+0000 7fdd9b719640 1 -- 172.21.15.16:0/637175502 >> 172.21.15.16:0/637175502 conn(0x7fdd940fc7a0 msgr2=0x7fdd940fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:08.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:08.736+0000 7fdd9b719640 1 -- 172.21.15.16:0/637175502 shutdown_connections 2024-09-12T02:03:08.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:08.736+0000 7fdd9b719640 1 -- 172.21.15.16:0/637175502 wait complete. 2024-09-12T02:03:09.446 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-12T02:03:09.708 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:03:09.829 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:03:09 smithi016 ceph-mon[30947]: pgmap v372: 97 pgs: 97 active+clean; 719 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 6.0 KiB/s wr, 9 op/s 2024-09-12T02:03:09.829 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:03:09 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/637175502' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:03:09.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:03:09 smithi184 ceph-mon[35593]: pgmap v372: 97 pgs: 97 active+clean; 719 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 6.0 KiB/s wr, 9 op/s 2024-09-12T02:03:09.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:03:09 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/637175502' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:03:09.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:03:09 smithi033 ceph-mon[35332]: pgmap v372: 97 pgs: 97 active+clean; 719 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 6.0 KiB/s wr, 9 op/s 2024-09-12T02:03:09.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:03:09 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/637175502' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:03:10.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.201+0000 7fc9e4454640 1 -- 172.21.15.16:0/2962253973 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9dc102d40 msgr2=0x7fc9dc103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:10.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.201+0000 7fc9e4454640 1 --2- 172.21.15.16:0/2962253973 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9dc102d40 0x7fc9dc103120 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fc9cc009ca0 tx=0x7fc9cc02a950 comp rx=0 tx=0).stop 2024-09-12T02:03:10.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.202+0000 7fc9e4454640 1 -- 172.21.15.16:0/2962253973 shutdown_connections 2024-09-12T02:03:10.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.202+0000 7fc9e4454640 1 --2- 172.21.15.16:0/2962253973 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc9dc10c170 0x7fc9dc10e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:10.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.202+0000 7fc9e4454640 1 --2- 172.21.15.16:0/2962253973 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc9dc103660 0x7fc9dc10bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:10.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.202+0000 7fc9e4454640 1 --2- 172.21.15.16:0/2962253973 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9dc102d40 0x7fc9dc103120 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:10.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.202+0000 7fc9e4454640 1 -- 172.21.15.16:0/2962253973 >> 172.21.15.16:0/2962253973 conn(0x7fc9dc0fc7a0 msgr2=0x7fc9dc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:10.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.202+0000 7fc9e4454640 1 -- 172.21.15.16:0/2962253973 shutdown_connections 2024-09-12T02:03:10.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.202+0000 7fc9e4454640 1 -- 172.21.15.16:0/2962253973 wait complete. 2024-09-12T02:03:10.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.202+0000 7fc9e4454640 1 Processor -- start 2024-09-12T02:03:10.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.202+0000 7fc9e4454640 1 -- start start 2024-09-12T02:03:10.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.202+0000 7fc9e4454640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9dc102d40 0x7fc9dc19a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:10.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.202+0000 7fc9e4454640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc9dc103660 0x7fc9dc19ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:10.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.202+0000 7fc9e4454640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc9dc10c170 0x7fc9dc19ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:10.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.202+0000 7fc9e4454640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc9dc111f70 con 0x7fc9dc102d40 2024-09-12T02:03:10.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.202+0000 7fc9e4454640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc9dc111df0 con 0x7fc9dc103660 2024-09-12T02:03:10.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.202+0000 7fc9e4454640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc9dc1120f0 con 0x7fc9dc10c170 2024-09-12T02:03:10.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.203+0000 7fc9e21f0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9dc102d40 0x7fc9dc19a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:10.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.203+0000 7fc9e21f0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9dc102d40 0x7fc9dc19a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46126/0 (socket says 172.21.15.16:46126) 2024-09-12T02:03:10.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.203+0000 7fc9e21f0640 1 -- 172.21.15.16:0/769019112 learned_addr learned my addr 172.21.15.16:0/769019112 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:03:10.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.203+0000 7fc9e19ef640 1 --2- 172.21.15.16:0/769019112 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc9dc103660 0x7fc9dc19ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:10.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.203+0000 7fc9e29f1640 1 --2- 172.21.15.16:0/769019112 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc9dc10c170 0x7fc9dc19ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:10.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.203+0000 7fc9e21f0640 1 -- 172.21.15.16:0/769019112 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc9dc10c170 msgr2=0x7fc9dc19ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:10.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.203+0000 7fc9e21f0640 1 --2- 172.21.15.16:0/769019112 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc9dc10c170 0x7fc9dc19ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:10.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.203+0000 7fc9e21f0640 1 -- 172.21.15.16:0/769019112 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc9dc103660 msgr2=0x7fc9dc19ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:10.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.203+0000 7fc9e21f0640 1 --2- 172.21.15.16:0/769019112 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc9dc103660 0x7fc9dc19ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:10.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.203+0000 7fc9e21f0640 1 -- 172.21.15.16:0/769019112 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9dc19f5f0 con 0x7fc9dc102d40 2024-09-12T02:03:10.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.203+0000 7fc9e21f0640 1 --2- 172.21.15.16:0/769019112 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9dc102d40 0x7fc9dc19a640 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fc9cc005cc0 tx=0x7fc9cc02f630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:10.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.203+0000 7fc9db7fe640 1 -- 172.21.15.16:0/769019112 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9cc034040 con 0x7fc9dc102d40 2024-09-12T02:03:10.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.203+0000 7fc9db7fe640 1 -- 172.21.15.16:0/769019112 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc9cc030920 con 0x7fc9dc102d40 2024-09-12T02:03:10.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.203+0000 7fc9e4454640 1 -- 172.21.15.16:0/769019112 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9dc19f880 con 0x7fc9dc102d40 2024-09-12T02:03:10.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.203+0000 7fc9db7fe640 1 -- 172.21.15.16:0/769019112 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9cc004830 con 0x7fc9dc102d40 2024-09-12T02:03:10.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.203+0000 7fc9e4454640 1 -- 172.21.15.16:0/769019112 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9dc1a71c0 con 0x7fc9dc102d40 2024-09-12T02:03:10.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.206+0000 7fc9db7fe640 1 -- 172.21.15.16:0/769019112 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fc9cc037020 con 0x7fc9dc102d40 2024-09-12T02:03:10.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.206+0000 7fc9e682d740 1 Processor -- start 2024-09-12T02:03:10.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.207+0000 7fc9e682d740 1 -- start start 2024-09-12T02:03:10.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.207+0000 7fc9e682d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a1c1468b40 0x55a1c1467090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:10.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.207+0000 7fc9e682d740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a1c14675d0 0x55a1c1467a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:10.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.207+0000 7fc9e682d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a1c1472370 0x55a1c1474760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:10.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.207+0000 7fc9e682d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55a1c13cd2f0 con 0x55a1c1472370 2024-09-12T02:03:10.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.207+0000 7fc9e682d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55a1c1374cf0 con 0x55a1c1468b40 2024-09-12T02:03:10.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.207+0000 7fc9e682d740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55a1c13c92b0 con 0x55a1c14675d0 2024-09-12T02:03:10.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.207+0000 7fc9e19ef640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a1c1472370 0x55a1c1474760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:10.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.207+0000 7fc9e19ef640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a1c1472370 0x55a1c1474760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46142/0 (socket says 172.21.15.16:46142) 2024-09-12T02:03:10.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.207+0000 7fc9e19ef640 1 -- 172.21.15.16:0/2602313027 learned_addr learned my addr 172.21.15.16:0/2602313027 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:03:10.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.207+0000 7fc9e21f0640 1 --2- 172.21.15.16:0/2602313027 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a1c14675d0 0x55a1c1467a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:10.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.207+0000 7fc9e29f1640 1 --2- 172.21.15.16:0/2602313027 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a1c1468b40 0x55a1c1467090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:10.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.207+0000 7fc9db7fe640 1 --2- 172.21.15.16:0/769019112 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc9c40a2b80 0x7fc9c40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:10.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.207+0000 7fc9e19ef640 1 -- 172.21.15.16:0/2602313027 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a1c14675d0 msgr2=0x55a1c1467a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:10.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.207+0000 7fc9e19ef640 1 --2- 172.21.15.16:0/2602313027 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a1c14675d0 0x55a1c1467a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:10.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.207+0000 7fc9e19ef640 1 -- 172.21.15.16:0/2602313027 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a1c1468b40 msgr2=0x55a1c1467090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:10.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.207+0000 7fc9e19ef640 1 --2- 172.21.15.16:0/2602313027 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a1c1468b40 0x55a1c1467090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:10.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.208+0000 7fc9e19ef640 1 -- 172.21.15.16:0/2602313027 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1c13593a0 con 0x55a1c1472370 2024-09-12T02:03:10.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.208+0000 7fc9e29f1640 1 --2- 172.21.15.16:0/2602313027 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a1c1468b40 0x55a1c1467090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:03:10.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.208+0000 7fc9e29f1640 1 --2- 172.21.15.16:0/769019112 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc9c40a2b80 0x7fc9c40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:10.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.208+0000 7fc9db7fe640 1 -- 172.21.15.16:0/769019112 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fc9cc0eff90 con 0x7fc9dc102d40 2024-09-12T02:03:10.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.208+0000 7fc9e19ef640 1 --2- 172.21.15.16:0/2602313027 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a1c1472370 0x55a1c1474760 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fc9d000bca0 tx=0x7fc9d00107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=147283bc02839ef8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:10.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.208+0000 7fc9d8dea640 1 -- 172.21.15.16:0/2602313027 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9d0004030 con 0x55a1c1472370 2024-09-12T02:03:10.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.208+0000 7fc9d8dea640 1 -- 172.21.15.16:0/2602313027 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc9d00041d0 con 0x55a1c1472370 2024-09-12T02:03:10.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.208+0000 7fc9d8dea640 1 -- 172.21.15.16:0/2602313027 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9d0011840 con 0x55a1c1472370 2024-09-12T02:03:10.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.209+0000 7fc9e682d740 1 -- 172.21.15.16:0/2602313027 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a1c1472370 msgr2=0x55a1c1474760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:10.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.209+0000 7fc9e29f1640 1 --2- 172.21.15.16:0/769019112 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc9c40a2b80 0x7fc9c40a5040 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fc9d40045e0 tx=0x7fc9d40046c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:10.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.209+0000 7fc9e682d740 1 --2- 172.21.15.16:0/2602313027 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a1c1472370 0x55a1c1474760 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fc9d000bca0 tx=0x7fc9d00107a0 comp rx=0 tx=0).stop 2024-09-12T02:03:10.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.209+0000 7fc9e682d740 1 -- 172.21.15.16:0/2602313027 shutdown_connections 2024-09-12T02:03:10.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.209+0000 7fc9e682d740 1 --2- 172.21.15.16:0/2602313027 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a1c1472370 0x55a1c1474760 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:10.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.209+0000 7fc9e682d740 1 --2- 172.21.15.16:0/2602313027 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a1c14675d0 0x55a1c1467a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:10.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.209+0000 7fc9e682d740 1 --2- 172.21.15.16:0/2602313027 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a1c1468b40 0x55a1c1467090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:10.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.209+0000 7fc9e682d740 1 -- 172.21.15.16:0/2602313027 >> 172.21.15.16:0/2602313027 conn(0x55a1c1430310 msgr2=0x55a1c1463c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:10.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.209+0000 7fc9e682d740 1 -- 172.21.15.16:0/2602313027 shutdown_connections 2024-09-12T02:03:10.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.209+0000 7fc9e682d740 1 -- 172.21.15.16:0/2602313027 wait complete. 2024-09-12T02:03:10.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.209+0000 7fc9e682d740 1 -- start start 2024-09-12T02:03:10.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.209+0000 7fc9e682d740 1 Processor -- start 2024-09-12T02:03:10.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.210+0000 7fc9e682d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a1c14675d0 0x55a1c1471da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:10.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.210+0000 7fc9e682d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a1c1468b40 0x55a1c146f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:10.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.210+0000 7fc9e682d740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a1c1472370 0x55a1c146f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:10.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.210+0000 7fc9e682d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55a1c1475070 con 0x55a1c14675d0 2024-09-12T02:03:10.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.210+0000 7fc9e682d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55a1c1388060 con 0x55a1c1468b40 2024-09-12T02:03:10.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.210+0000 7fc9e682d740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55a1c14751f0 con 0x55a1c1472370 2024-09-12T02:03:10.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.210+0000 7fc9e21f0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a1c14675d0 0x55a1c1471da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:10.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.210+0000 7fc9e21f0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a1c14675d0 0x55a1c1471da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46144/0 (socket says 172.21.15.16:46144) 2024-09-12T02:03:10.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.210+0000 7fc9e682d740 10 client.0 _ll_register_callbacks cb 0x55a1c10db8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:03:10.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.210+0000 7fc9e21f0640 1 -- 172.21.15.16:0/1995935622 learned_addr learned my addr 172.21.15.16:0/1995935622 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:03:10.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.210+0000 7fc9e19ef640 1 --2- 172.21.15.16:0/1995935622 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a1c1468b40 0x55a1c146f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:10.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.210+0000 7fc9e29f1640 1 --2- 172.21.15.16:0/1995935622 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a1c1472370 0x55a1c146f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:10.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.210+0000 7fc9e21f0640 1 -- 172.21.15.16:0/1995935622 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a1c1472370 msgr2=0x55a1c146f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:10.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.210+0000 7fc9e21f0640 1 --2- 172.21.15.16:0/1995935622 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a1c1472370 0x55a1c146f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:10.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.210+0000 7fc9e21f0640 1 -- 172.21.15.16:0/1995935622 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a1c1468b40 msgr2=0x55a1c146f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:10.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.210+0000 7fc9e21f0640 1 --2- 172.21.15.16:0/1995935622 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a1c1468b40 0x55a1c146f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:10.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.210+0000 7fc9e21f0640 1 -- 172.21.15.16:0/1995935622 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1c113e3c0 con 0x55a1c14675d0 2024-09-12T02:03:10.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.210+0000 7fc9e19ef640 1 --2- 172.21.15.16:0/1995935622 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a1c1468b40 0x55a1c146f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:03:10.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.210+0000 7fc9e29f1640 1 --2- 172.21.15.16:0/1995935622 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a1c1472370 0x55a1c146f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:03:10.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.210+0000 7fc9e21f0640 1 --2- 172.21.15.16:0/1995935622 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a1c14675d0 0x55a1c1471da0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fc9cc0049d0 tx=0x7fc9cc0bc720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c312beb5fb059b32 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:10.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.210+0000 7fc9c1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:03:10.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.210+0000 7fc9c1ffb640 1 -- 172.21.15.16:0/1995935622 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9cc0bcf90 con 0x55a1c14675d0 2024-09-12T02:03:10.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.210+0000 7fc9c1ffb640 1 -- 172.21.15.16:0/1995935622 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fc9cc004040 con 0x55a1c14675d0 2024-09-12T02:03:10.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.210+0000 7fc9e682d740 1 -- 172.21.15.16:0/1995935622 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc9d00041d0 con 0x55a1c14675d0 2024-09-12T02:03:10.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.211+0000 7fc9c1ffb640 1 -- 172.21.15.16:0/1995935622 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc9cc0c5500 con 0x55a1c14675d0 2024-09-12T02:03:10.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.211+0000 7fc9e682d740 10 client.15624 fetch_fsmap learned FSMap version 25 2024-09-12T02:03:10.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.211+0000 7fc9e682d740 1 -- 172.21.15.16:0/1995935622 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a1c1475070 con 0x55a1c14675d0 2024-09-12T02:03:10.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.211+0000 7fc9c1ffb640 1 -- 172.21.15.16:0/1995935622 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc9d00041d0 con 0x55a1c14675d0 2024-09-12T02:03:10.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.211+0000 7fc9c1ffb640 1 -- 172.21.15.16:0/1995935622 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fc9cc04d030 con 0x55a1c14675d0 2024-09-12T02:03:10.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.211+0000 7fc9e682d740 10 client.15624 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:03:10.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.211+0000 7fc9e682d740 10 client.15624 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-12T02:03:10.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.211+0000 7fc9e682d740 20 client.15624 populate_metadata read hostname 'smithi016' 2024-09-12T02:03:10.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.211+0000 7fc9e682d740 1 --2- 172.21.15.16:0/1995935622 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55a1c1481ed0 0x55a1c1484360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:10.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.211+0000 7fc9e682d740 4 client.15624 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:03:10.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.211+0000 7fc9e682d740 1 -- 172.21.15.16:0/1995935622 --> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a1c13da860 con 0x55a1c1481ed0 2024-09-12T02:03:10.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.211+0000 7fc9e19ef640 1 --2- 172.21.15.16:0/1995935622 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55a1c1481ed0 0x55a1c1484360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:10.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.212+0000 7fc9e19ef640 1 --2- 172.21.15.16:0/1995935622 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55a1c1481ed0 0x55a1c1484360 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b0c70ae5741f595c server_cookie=fbf71945475fb32e in_seq=0 out_seq=0 2024-09-12T02:03:10.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.212+0000 7fc9c1ffb640 10 client.15624 ms_handle_connect on v2:172.21.15.184:6836/1793393087 2024-09-12T02:03:10.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.212+0000 7fc9c1ffb640 1 -- 172.21.15.16:0/1995935622 <== mds.2 v2:172.21.15.184:6836/1793393087 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a1c13da860 con 0x55a1c1481ed0 2024-09-12T02:03:10.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.212+0000 7fc9c1ffb640 10 client.15624 handle_command_reply: tid=0 2024-09-12T02:03:10.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.212+0000 7fc9e682d740 1 client.15624 shutdown 2024-09-12T02:03:10.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.213+0000 7fc9e682d740 1 -- 172.21.15.16:0/1995935622 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a1c14675d0 msgr2=0x55a1c1471da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:10.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.213+0000 7fc9e682d740 1 --2- 172.21.15.16:0/1995935622 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a1c14675d0 0x55a1c1471da0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fc9cc0049d0 tx=0x7fc9cc0bc720 comp rx=0 tx=0).stop 2024-09-12T02:03:10.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.213+0000 7fc9e682d740 1 -- 172.21.15.16:0/1995935622 shutdown_connections 2024-09-12T02:03:10.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.213+0000 7fc9e682d740 1 --2- 172.21.15.16:0/1995935622 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a1c1472370 0x55a1c146f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:10.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.213+0000 7fc9e682d740 1 --2- 172.21.15.16:0/1995935622 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a1c1468b40 0x55a1c146f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:10.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.213+0000 7fc9e682d740 1 --2- 172.21.15.16:0/1995935622 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55a1c1481ed0 0x55a1c1484360 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:10.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.213+0000 7fc9e682d740 1 --2- 172.21.15.16:0/1995935622 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a1c14675d0 0x55a1c1471da0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:10.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.213+0000 7fc9c1ffb640 0 client.15624 ms_handle_reset on v2:172.21.15.184:6836/1793393087 2024-09-12T02:03:10.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.213+0000 7fc9e682d740 1 -- 172.21.15.16:0/1995935622 >> 172.21.15.16:0/1995935622 conn(0x55a1c1430310 msgr2=0x55a1c1469960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:10.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.213+0000 7fc9e682d740 1 -- 172.21.15.16:0/1995935622 shutdown_connections 2024-09-12T02:03:10.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.213+0000 7fc9e682d740 1 -- 172.21.15.16:0/1995935622 wait complete. 2024-09-12T02:03:10.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.213+0000 7fc9e682d740 20 client.15624 trim_cache size 0 max 16384 2024-09-12T02:03:10.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.226+0000 7fc9e682d740 1 Processor -- start 2024-09-12T02:03:10.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.226+0000 7fc9e682d740 1 -- start start 2024-09-12T02:03:10.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.226+0000 7fc9e682d740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a1c14675d0 0x55a1c146e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:10.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.226+0000 7fc9e682d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a1c1468b40 0x55a1c147da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:10.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.226+0000 7fc9e682d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a1c1472370 0x55a1c147df50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:10.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.226+0000 7fc9e682d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc9dc1b1e40 con 0x55a1c1468b40 2024-09-12T02:03:10.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.226+0000 7fc9e682d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc9dc1af710 con 0x55a1c1472370 2024-09-12T02:03:10.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.226+0000 7fc9e682d740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc9dc1ad560 con 0x55a1c14675d0 2024-09-12T02:03:10.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.226+0000 7fc9e19ef640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a1c1468b40 0x55a1c147da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:10.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.226+0000 7fc9e19ef640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a1c1468b40 0x55a1c147da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46158/0 (socket says 172.21.15.16:46158) 2024-09-12T02:03:10.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.226+0000 7fc9e19ef640 1 -- 172.21.15.16:0/923837828 learned_addr learned my addr 172.21.15.16:0/923837828 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:03:10.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.227+0000 7fc9e21f0640 1 --2- 172.21.15.16:0/923837828 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a1c14675d0 0x55a1c146e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:10.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.227+0000 7fc9e19ef640 1 -- 172.21.15.16:0/923837828 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a1c14675d0 msgr2=0x55a1c146e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:10.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.227+0000 7fc9e19ef640 1 --2- 172.21.15.16:0/923837828 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a1c14675d0 0x55a1c146e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:10.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.227+0000 7fc9e29f1640 1 --2- 172.21.15.16:0/923837828 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a1c1472370 0x55a1c147df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:10.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.227+0000 7fc9e19ef640 1 -- 172.21.15.16:0/923837828 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a1c1472370 msgr2=0x55a1c147df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:10.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.227+0000 7fc9e19ef640 1 --2- 172.21.15.16:0/923837828 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a1c1472370 0x55a1c147df50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:10.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.227+0000 7fc9e19ef640 1 -- 172.21.15.16:0/923837828 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1c146e000 con 0x55a1c1468b40 2024-09-12T02:03:10.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.227+0000 7fc9e29f1640 1 --2- 172.21.15.16:0/923837828 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a1c1472370 0x55a1c147df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:03:10.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.227+0000 7fc9e19ef640 1 --2- 172.21.15.16:0/923837828 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a1c1468b40 0x55a1c147da10 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fc9d000bbc0 tx=0x7fc9d000bf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf95f4a5bd8f9ff server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:10.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.227+0000 7fc9c3fff640 1 -- 172.21.15.16:0/923837828 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9d0005240 con 0x55a1c1468b40 2024-09-12T02:03:10.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.227+0000 7fc9c3fff640 1 -- 172.21.15.16:0/923837828 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc9d0003ad0 con 0x55a1c1468b40 2024-09-12T02:03:10.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.227+0000 7fc9c3fff640 1 -- 172.21.15.16:0/923837828 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9d0011860 con 0x55a1c1468b40 2024-09-12T02:03:10.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.228+0000 7fc9e682d740 1 -- 172.21.15.16:0/923837828 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a1c1468b40 msgr2=0x55a1c147da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:10.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.228+0000 7fc9e682d740 1 --2- 172.21.15.16:0/923837828 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a1c1468b40 0x55a1c147da10 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fc9d000bbc0 tx=0x7fc9d000bf20 comp rx=0 tx=0).stop 2024-09-12T02:03:10.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.228+0000 7fc9e682d740 1 -- 172.21.15.16:0/923837828 shutdown_connections 2024-09-12T02:03:10.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.228+0000 7fc9e682d740 1 --2- 172.21.15.16:0/923837828 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a1c1472370 0x55a1c147df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:10.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.228+0000 7fc9e682d740 1 --2- 172.21.15.16:0/923837828 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a1c1468b40 0x55a1c147da10 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:10.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.228+0000 7fc9e682d740 1 --2- 172.21.15.16:0/923837828 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a1c14675d0 0x55a1c146e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:10.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.228+0000 7fc9e682d740 1 -- 172.21.15.16:0/923837828 >> 172.21.15.16:0/923837828 conn(0x55a1c1430310 msgr2=0x55a1c147d300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:10.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.228+0000 7fc9e682d740 1 -- 172.21.15.16:0/923837828 shutdown_connections 2024-09-12T02:03:10.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.228+0000 7fc9e682d740 1 -- 172.21.15.16:0/923837828 wait complete. 2024-09-12T02:03:10.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.228+0000 7fc9e682d740 1 -- start start 2024-09-12T02:03:10.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.228+0000 7fc9e682d740 1 Processor -- start 2024-09-12T02:03:10.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.229+0000 7fc9e682d740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a1c14675d0 0x55a1c147abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:10.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.229+0000 7fc9e682d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a1c1468b40 0x55a1c1475f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:10.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.229+0000 7fc9e682d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a1c1472370 0x55a1c1476450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:10.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.229+0000 7fc9e682d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc9dc1adbd0 con 0x55a1c1472370 2024-09-12T02:03:10.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.229+0000 7fc9e682d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc9dc1af040 con 0x55a1c1468b40 2024-09-12T02:03:10.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.229+0000 7fc9e682d740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc9dc1ae2a0 con 0x55a1c14675d0 2024-09-12T02:03:10.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.229+0000 7fc9e682d740 10 client.0 _ll_register_callbacks cb 0x7fc9cc005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:03:10.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.229+0000 7fc9e29f1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a1c1472370 0x55a1c1476450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:10.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.229+0000 7fc9e29f1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a1c1472370 0x55a1c1476450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46162/0 (socket says 172.21.15.16:46162) 2024-09-12T02:03:10.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.229+0000 7fc9e29f1640 1 -- 172.21.15.16:0/746937463 learned_addr learned my addr 172.21.15.16:0/746937463 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:03:10.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.229+0000 7fc9e21f0640 1 --2- 172.21.15.16:0/746937463 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a1c14675d0 0x55a1c147abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:10.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.229+0000 7fc9e19ef640 1 --2- 172.21.15.16:0/746937463 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a1c1468b40 0x55a1c1475f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:10.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.229+0000 7fc9e29f1640 1 -- 172.21.15.16:0/746937463 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a1c14675d0 msgr2=0x55a1c147abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:10.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.229+0000 7fc9e29f1640 1 --2- 172.21.15.16:0/746937463 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a1c14675d0 0x55a1c147abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:10.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.229+0000 7fc9e29f1640 1 -- 172.21.15.16:0/746937463 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a1c1468b40 msgr2=0x55a1c1475f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:10.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.229+0000 7fc9e29f1640 1 --2- 172.21.15.16:0/746937463 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a1c1468b40 0x55a1c1475f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:10.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.229+0000 7fc9e29f1640 1 -- 172.21.15.16:0/746937463 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1c1476c30 con 0x55a1c1472370 2024-09-12T02:03:10.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.229+0000 7fc9e19ef640 1 --2- 172.21.15.16:0/746937463 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a1c1468b40 0x55a1c1475f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:03:10.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.229+0000 7fc9e29f1640 1 --2- 172.21.15.16:0/746937463 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a1c1472370 0x55a1c1476450 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fc9d4005d80 tx=0x7fc9d4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa4b87cae054dd8d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:10.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.229+0000 7fc9c1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:03:10.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.230+0000 7fc9c1ffb640 1 -- 172.21.15.16:0/746937463 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9d401f600 con 0x55a1c1472370 2024-09-12T02:03:10.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.230+0000 7fc9c1ffb640 1 -- 172.21.15.16:0/746937463 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fc9d401f7a0 con 0x55a1c1472370 2024-09-12T02:03:10.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.230+0000 7fc9e682d740 1 -- 172.21.15.16:0/746937463 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc9d0003ad0 con 0x55a1c1472370 2024-09-12T02:03:10.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.230+0000 7fc9c1ffb640 1 -- 172.21.15.16:0/746937463 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc9d4003ea0 con 0x55a1c1472370 2024-09-12T02:03:10.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.230+0000 7fc9e682d740 10 client.15630 fetch_fsmap learned FSMap version 25 2024-09-12T02:03:10.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.230+0000 7fc9e682d740 1 -- 172.21.15.16:0/746937463 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a1c14ecbc0 con 0x55a1c1472370 2024-09-12T02:03:10.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.230+0000 7fc9c1ffb640 1 -- 172.21.15.16:0/746937463 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc9d0003ad0 con 0x55a1c1472370 2024-09-12T02:03:10.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.230+0000 7fc9c1ffb640 1 -- 172.21.15.16:0/746937463 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fc9d401b070 con 0x55a1c1472370 2024-09-12T02:03:10.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.230+0000 7fc9e682d740 10 client.15630 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:03:10.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.230+0000 7fc9e682d740 10 client.15630 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-12T02:03:10.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.230+0000 7fc9e682d740 20 client.15630 populate_metadata read hostname 'smithi016' 2024-09-12T02:03:10.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.230+0000 7fc9e682d740 1 --2- 172.21.15.16:0/746937463 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55a1c14ed390 0x55a1c14ef7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:10.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.230+0000 7fc9e682d740 4 client.15630 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T02:03:10.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.230+0000 7fc9e682d740 1 -- 172.21.15.16:0/746937463 --> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a1c13d4d70 con 0x55a1c14ed390 2024-09-12T02:03:10.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.230+0000 7fc9e21f0640 1 --2- 172.21.15.16:0/746937463 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55a1c14ed390 0x55a1c14ef7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:10.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.231+0000 7fc9e21f0640 1 --2- 172.21.15.16:0/746937463 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55a1c14ed390 0x55a1c14ef7b0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3428cc1d743ed1d4 server_cookie=8c72b70dccd72e02 in_seq=0 out_seq=0 2024-09-12T02:03:10.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.231+0000 7fc9c1ffb640 10 client.15630 ms_handle_connect on v2:172.21.15.184:6836/1793393087 2024-09-12T02:03:10.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.231+0000 7fc9c1ffb640 1 -- 172.21.15.16:0/746937463 <== mds.2 v2:172.21.15.184:6836/1793393087 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a1c13d4d70 con 0x55a1c14ed390 2024-09-12T02:03:10.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.231+0000 7fc9c1ffb640 10 client.15630 handle_command_reply: tid=0 2024-09-12T02:03:10.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.231+0000 7fc9e682d740 1 client.15630 shutdown 2024-09-12T02:03:10.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.231+0000 7fc9e682d740 1 -- 172.21.15.16:0/746937463 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a1c1472370 msgr2=0x55a1c1476450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:10.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.231+0000 7fc9e682d740 1 --2- 172.21.15.16:0/746937463 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a1c1472370 0x55a1c1476450 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fc9d4005d80 tx=0x7fc9d4004730 comp rx=0 tx=0).stop 2024-09-12T02:03:10.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.231+0000 7fc9e682d740 1 -- 172.21.15.16:0/746937463 shutdown_connections 2024-09-12T02:03:10.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.231+0000 7fc9e682d740 1 --2- 172.21.15.16:0/746937463 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55a1c14ed390 0x55a1c14ef7b0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:10.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.231+0000 7fc9e682d740 1 --2- 172.21.15.16:0/746937463 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a1c1472370 0x55a1c1476450 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:10.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.231+0000 7fc9e682d740 1 --2- 172.21.15.16:0/746937463 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a1c1468b40 0x55a1c1475f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:10.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.231+0000 7fc9e682d740 1 --2- 172.21.15.16:0/746937463 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a1c14675d0 0x55a1c147abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:10.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.231+0000 7fc9c1ffb640 0 client.15630 ms_handle_reset on v2:172.21.15.184:6836/1793393087 2024-09-12T02:03:10.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.231+0000 7fc9e682d740 1 -- 172.21.15.16:0/746937463 >> 172.21.15.16:0/746937463 conn(0x55a1c1430310 msgr2=0x55a1c14708b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:10.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.231+0000 7fc9e682d740 1 -- 172.21.15.16:0/746937463 shutdown_connections 2024-09-12T02:03:10.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.231+0000 7fc9e682d740 1 -- 172.21.15.16:0/746937463 wait complete. 2024-09-12T02:03:10.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.231+0000 7fc9e682d740 20 client.15630 trim_cache size 0 max 16384 2024-09-12T02:03:10.249 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T02:03:10.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.232+0000 7fc9e4454640 1 -- 172.21.15.16:0/769019112 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc9c40a2b80 msgr2=0x7fc9c40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:10.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.232+0000 7fc9e4454640 1 --2- 172.21.15.16:0/769019112 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc9c40a2b80 0x7fc9c40a5040 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fc9d40045e0 tx=0x7fc9d40046c0 comp rx=0 tx=0).stop 2024-09-12T02:03:10.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.232+0000 7fc9e4454640 1 -- 172.21.15.16:0/769019112 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9dc102d40 msgr2=0x7fc9dc19a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:10.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.232+0000 7fc9e4454640 1 --2- 172.21.15.16:0/769019112 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9dc102d40 0x7fc9dc19a640 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fc9cc005cc0 tx=0x7fc9cc02f630 comp rx=0 tx=0).stop 2024-09-12T02:03:10.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.232+0000 7fc9e4454640 1 -- 172.21.15.16:0/769019112 shutdown_connections 2024-09-12T02:03:10.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.232+0000 7fc9e4454640 1 --2- 172.21.15.16:0/769019112 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc9c40a2b80 0x7fc9c40a5040 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:10.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.232+0000 7fc9e4454640 1 --2- 172.21.15.16:0/769019112 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc9dc10c170 0x7fc9dc19ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:10.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.232+0000 7fc9e4454640 1 --2- 172.21.15.16:0/769019112 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc9dc103660 0x7fc9dc19ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:10.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.232+0000 7fc9e4454640 1 --2- 172.21.15.16:0/769019112 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9dc102d40 0x7fc9dc19a640 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:10.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.232+0000 7fc9e4454640 1 -- 172.21.15.16:0/769019112 >> 172.21.15.16:0/769019112 conn(0x7fc9dc0fc7a0 msgr2=0x7fc9dc10d4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:10.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.232+0000 7fc9e4454640 1 -- 172.21.15.16:0/769019112 shutdown_connections 2024-09-12T02:03:10.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:10.232+0000 7fc9e4454640 1 -- 172.21.15.16:0/769019112 wait complete. 2024-09-12T02:03:10.960 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-12T02:03:11.218 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:03:11.456 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:03:11 smithi016 ceph-mon[30947]: pgmap v373: 97 pgs: 97 active+clean; 772 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 13 op/s 2024-09-12T02:03:11.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.692+0000 7fdc5a293640 1 -- 172.21.15.16:0/150537917 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc54102d40 msgr2=0x7fdc54103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:11.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.692+0000 7fdc5a293640 1 --2- 172.21.15.16:0/150537917 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc54102d40 0x7fdc54103120 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fdc44009900 tx=0x7fdc4402a480 comp rx=0 tx=0).stop 2024-09-12T02:03:11.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.692+0000 7fdc5a293640 1 -- 172.21.15.16:0/150537917 shutdown_connections 2024-09-12T02:03:11.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.692+0000 7fdc5a293640 1 --2- 172.21.15.16:0/150537917 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdc5410c170 0x7fdc5410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:11.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.692+0000 7fdc5a293640 1 --2- 172.21.15.16:0/150537917 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdc54103660 0x7fdc5410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:11.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.692+0000 7fdc5a293640 1 --2- 172.21.15.16:0/150537917 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc54102d40 0x7fdc54103120 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:11.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.692+0000 7fdc5a293640 1 -- 172.21.15.16:0/150537917 >> 172.21.15.16:0/150537917 conn(0x7fdc540fc7a0 msgr2=0x7fdc540febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:11.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.692+0000 7fdc5a293640 1 -- 172.21.15.16:0/150537917 shutdown_connections 2024-09-12T02:03:11.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.692+0000 7fdc5a293640 1 -- 172.21.15.16:0/150537917 wait complete. 2024-09-12T02:03:11.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.693+0000 7fdc5a293640 1 Processor -- start 2024-09-12T02:03:11.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.693+0000 7fdc5a293640 1 -- start start 2024-09-12T02:03:11.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.693+0000 7fdc5a293640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdc54102d40 0x7fdc541009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:11.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.694+0000 7fdc5a293640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc54103660 0x7fdc54100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:11.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.694+0000 7fdc5a293640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdc5410c170 0x7fdc54101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:11.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.694+0000 7fdc5a293640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fdc54111f70 con 0x7fdc54103660 2024-09-12T02:03:11.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.694+0000 7fdc5a293640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fdc54111df0 con 0x7fdc54102d40 2024-09-12T02:03:11.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.694+0000 7fdc5a293640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fdc541120f0 con 0x7fdc5410c170 2024-09-12T02:03:11.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.694+0000 7fdc537fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc54103660 0x7fdc54100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:11.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.694+0000 7fdc537fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc54103660 0x7fdc54100ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49442/0 (socket says 172.21.15.16:49442) 2024-09-12T02:03:11.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.694+0000 7fdc537fe640 1 -- 172.21.15.16:0/1120447133 learned_addr learned my addr 172.21.15.16:0/1120447133 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:03:11.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.694+0000 7fdc53fff640 1 --2- 172.21.15.16:0/1120447133 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdc54102d40 0x7fdc541009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:11.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.694+0000 7fdc58830640 1 --2- 172.21.15.16:0/1120447133 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdc5410c170 0x7fdc54101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:11.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.694+0000 7fdc537fe640 1 -- 172.21.15.16:0/1120447133 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdc5410c170 msgr2=0x7fdc54101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:11.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.694+0000 7fdc537fe640 1 --2- 172.21.15.16:0/1120447133 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdc5410c170 0x7fdc54101420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:11.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.694+0000 7fdc537fe640 1 -- 172.21.15.16:0/1120447133 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdc54102d40 msgr2=0x7fdc541009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:11.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.694+0000 7fdc537fe640 1 --2- 172.21.15.16:0/1120447133 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdc54102d40 0x7fdc541009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:11.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.694+0000 7fdc537fe640 1 -- 172.21.15.16:0/1120447133 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc541a68a0 con 0x7fdc54103660 2024-09-12T02:03:11.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.694+0000 7fdc537fe640 1 --2- 172.21.15.16:0/1120447133 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc54103660 0x7fdc54100ee0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fdc4800e9f0 tx=0x7fdc4800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:11.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.694+0000 7fdc517fa640 1 -- 172.21.15.16:0/1120447133 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc4800ccc0 con 0x7fdc54103660 2024-09-12T02:03:11.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.694+0000 7fdc517fa640 1 -- 172.21.15.16:0/1120447133 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc48010910 con 0x7fdc54103660 2024-09-12T02:03:11.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.694+0000 7fdc5a293640 1 -- 172.21.15.16:0/1120447133 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc541a6b90 con 0x7fdc54103660 2024-09-12T02:03:11.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.694+0000 7fdc517fa640 1 -- 172.21.15.16:0/1120447133 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc48018920 con 0x7fdc54103660 2024-09-12T02:03:11.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.694+0000 7fdc5a293640 1 -- 172.21.15.16:0/1120447133 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc541a70d0 con 0x7fdc54103660 2024-09-12T02:03:11.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.697+0000 7fdc517fa640 1 -- 172.21.15.16:0/1120447133 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fdc48010430 con 0x7fdc54103660 2024-09-12T02:03:11.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.697+0000 7fdc5c66c740 1 Processor -- start 2024-09-12T02:03:11.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.697+0000 7fdc5c66c740 1 -- start start 2024-09-12T02:03:11.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.697+0000 7fdc5c66c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf6e263b40 0x55bf6e262090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:11.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.697+0000 7fdc5c66c740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bf6e2625d0 0x55bf6e262a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:11.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.697+0000 7fdc5c66c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bf6e26d370 0x55bf6e26f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:11.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.697+0000 7fdc5c66c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55bf6e1c82f0 con 0x55bf6e26d370 2024-09-12T02:03:11.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.697+0000 7fdc5c66c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55bf6e16fcf0 con 0x55bf6e263b40 2024-09-12T02:03:11.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.697+0000 7fdc5c66c740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55bf6e1c42b0 con 0x55bf6e2625d0 2024-09-12T02:03:11.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.697+0000 7fdc537fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bf6e26d370 0x55bf6e26f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:11.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.697+0000 7fdc58830640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf6e263b40 0x55bf6e262090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:11.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.697+0000 7fdc537fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bf6e26d370 0x55bf6e26f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49456/0 (socket says 172.21.15.16:49456) 2024-09-12T02:03:11.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.697+0000 7fdc517fa640 1 --2- 172.21.15.16:0/1120447133 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fdc3c0a2b80 0x7fdc3c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:11.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.697+0000 7fdc537fe640 1 -- 172.21.15.16:0/1844041086 learned_addr learned my addr 172.21.15.16:0/1844041086 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:03:11.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.698+0000 7fdc53fff640 1 --2- 172.21.15.16:0/1844041086 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bf6e2625d0 0x55bf6e262a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:11.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.698+0000 7fdc58830640 1 --2- 172.21.15.16:0/1120447133 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fdc3c0a2b80 0x7fdc3c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:11.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.698+0000 7fdc517fa640 1 -- 172.21.15.16:0/1120447133 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fdc480cf270 con 0x7fdc54103660 2024-09-12T02:03:11.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.698+0000 7fdc537fe640 1 -- 172.21.15.16:0/1844041086 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bf6e2625d0 msgr2=0x55bf6e262a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:11.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.698+0000 7fdc537fe640 1 --2- 172.21.15.16:0/1844041086 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bf6e2625d0 0x55bf6e262a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:11.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.698+0000 7fdc537fe640 1 -- 172.21.15.16:0/1844041086 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf6e263b40 msgr2=0x55bf6e262090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:11.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.698+0000 7fdc537fe640 1 --2- 172.21.15.16:0/1844041086 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf6e263b40 0x55bf6e262090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:11.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.698+0000 7fdc537fe640 1 -- 172.21.15.16:0/1844041086 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf6e1543a0 con 0x55bf6e26d370 2024-09-12T02:03:11.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.698+0000 7fdc537fe640 1 --2- 172.21.15.16:0/1844041086 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bf6e26d370 0x55bf6e26f760 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fdc48097fd0 tx=0x7fdc48009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18fca7b502da749a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:11.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.698+0000 7fdc3a7fc640 1 -- 172.21.15.16:0/1844041086 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc4809bb70 con 0x55bf6e26d370 2024-09-12T02:03:11.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.698+0000 7fdc58830640 1 --2- 172.21.15.16:0/1120447133 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fdc3c0a2b80 0x7fdc3c0a5040 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fdc4c005fd0 tx=0x7fdc4c004ab0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:11.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.698+0000 7fdc3a7fc640 1 -- 172.21.15.16:0/1844041086 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc4809ff20 con 0x55bf6e26d370 2024-09-12T02:03:11.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.698+0000 7fdc3a7fc640 1 -- 172.21.15.16:0/1844041086 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc4809afc0 con 0x55bf6e26d370 2024-09-12T02:03:11.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.699+0000 7fdc5c66c740 1 -- 172.21.15.16:0/1844041086 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bf6e26d370 msgr2=0x55bf6e26f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:11.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.699+0000 7fdc5c66c740 1 --2- 172.21.15.16:0/1844041086 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bf6e26d370 0x55bf6e26f760 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fdc48097fd0 tx=0x7fdc48009f90 comp rx=0 tx=0).stop 2024-09-12T02:03:11.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.699+0000 7fdc5c66c740 1 -- 172.21.15.16:0/1844041086 shutdown_connections 2024-09-12T02:03:11.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.699+0000 7fdc5c66c740 1 --2- 172.21.15.16:0/1844041086 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bf6e26d370 0x55bf6e26f760 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:11.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.699+0000 7fdc5c66c740 1 --2- 172.21.15.16:0/1844041086 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bf6e2625d0 0x55bf6e262a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:11.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.699+0000 7fdc5c66c740 1 --2- 172.21.15.16:0/1844041086 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf6e263b40 0x55bf6e262090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:11.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.699+0000 7fdc5c66c740 1 -- 172.21.15.16:0/1844041086 >> 172.21.15.16:0/1844041086 conn(0x55bf6e22b310 msgr2=0x55bf6e25ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:11.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.699+0000 7fdc5c66c740 1 -- 172.21.15.16:0/1844041086 shutdown_connections 2024-09-12T02:03:11.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.699+0000 7fdc5c66c740 1 -- 172.21.15.16:0/1844041086 wait complete. 2024-09-12T02:03:11.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.699+0000 7fdc5c66c740 1 -- start start 2024-09-12T02:03:11.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.700+0000 7fdc5c66c740 1 Processor -- start 2024-09-12T02:03:11.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.700+0000 7fdc5c66c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf6e2625d0 0x55bf6e26cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:11.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.700+0000 7fdc5c66c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bf6e263b40 0x55bf6e26a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:11.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.700+0000 7fdc5c66c740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bf6e26d370 0x55bf6e26a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:11.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.700+0000 7fdc5c66c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55bf6e270010 con 0x55bf6e263b40 2024-09-12T02:03:11.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.700+0000 7fdc5c66c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55bf6e183060 con 0x55bf6e2625d0 2024-09-12T02:03:11.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.700+0000 7fdc5c66c740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55bf6e270190 con 0x55bf6e26d370 2024-09-12T02:03:11.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.700+0000 7fdc5c66c740 10 client.0 _ll_register_callbacks cb 0x55bf6ded68c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:03:11.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.700+0000 7fdc537fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bf6e263b40 0x55bf6e26a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:11.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.700+0000 7fdc537fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bf6e263b40 0x55bf6e26a060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49468/0 (socket says 172.21.15.16:49468) 2024-09-12T02:03:11.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.700+0000 7fdc537fe640 1 -- 172.21.15.16:0/2994187163 learned_addr learned my addr 172.21.15.16:0/2994187163 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:03:11.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.700+0000 7fdc53fff640 1 --2- 172.21.15.16:0/2994187163 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf6e2625d0 0x55bf6e26cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:11.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.700+0000 7fdc58830640 1 --2- 172.21.15.16:0/2994187163 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bf6e26d370 0x55bf6e26a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:11.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.700+0000 7fdc537fe640 1 -- 172.21.15.16:0/2994187163 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bf6e26d370 msgr2=0x55bf6e26a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:11.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.700+0000 7fdc537fe640 1 --2- 172.21.15.16:0/2994187163 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bf6e26d370 0x55bf6e26a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:11.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.700+0000 7fdc537fe640 1 -- 172.21.15.16:0/2994187163 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf6e2625d0 msgr2=0x55bf6e26cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:11.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.700+0000 7fdc537fe640 1 --2- 172.21.15.16:0/2994187163 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf6e2625d0 0x55bf6e26cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:11.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.700+0000 7fdc537fe640 1 -- 172.21.15.16:0/2994187163 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf6df393c0 con 0x55bf6e263b40 2024-09-12T02:03:11.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.701+0000 7fdc537fe640 1 --2- 172.21.15.16:0/2994187163 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bf6e263b40 0x55bf6e26a060 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fdc48009700 tx=0x7fdc48097e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9ed30011e24776f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:11.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.701+0000 7fdc337fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:03:11.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.701+0000 7fdc337fe640 1 -- 172.21.15.16:0/2994187163 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc48097ac0 con 0x55bf6e263b40 2024-09-12T02:03:11.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.701+0000 7fdc337fe640 1 -- 172.21.15.16:0/2994187163 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fdc48018ac0 con 0x55bf6e263b40 2024-09-12T02:03:11.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.701+0000 7fdc5c66c740 1 -- 172.21.15.16:0/2994187163 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc4809ff20 con 0x55bf6e263b40 2024-09-12T02:03:11.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.701+0000 7fdc337fe640 1 -- 172.21.15.16:0/2994187163 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fdc48010720 con 0x55bf6e263b40 2024-09-12T02:03:11.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.701+0000 7fdc5c66c740 10 client.15642 fetch_fsmap learned FSMap version 25 2024-09-12T02:03:11.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.701+0000 7fdc5c66c740 1 -- 172.21.15.16:0/2994187163 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55bf6e27bb00 con 0x55bf6e263b40 2024-09-12T02:03:11.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.701+0000 7fdc337fe640 1 -- 172.21.15.16:0/2994187163 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc4809ff20 con 0x55bf6e263b40 2024-09-12T02:03:11.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.701+0000 7fdc337fe640 1 -- 172.21.15.16:0/2994187163 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fdc48086170 con 0x55bf6e263b40 2024-09-12T02:03:11.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.701+0000 7fdc5c66c740 10 client.15642 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:03:11.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.701+0000 7fdc5c66c740 10 client.15642 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T02:03:11.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.701+0000 7fdc5c66c740 20 client.15642 populate_metadata read hostname 'smithi016' 2024-09-12T02:03:11.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.701+0000 7fdc5c66c740 1 --2- 172.21.15.16:0/2994187163 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x55bf6e27ce50 0x55bf6e27f240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:11.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.701+0000 7fdc5c66c740 4 client.15642 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:03:11.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.701+0000 7fdc5c66c740 1 -- 172.21.15.16:0/2994187163 --> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bf6e1d5860 con 0x55bf6e27ce50 2024-09-12T02:03:11.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.702+0000 7fdc53fff640 1 --2- 172.21.15.16:0/2994187163 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x55bf6e27ce50 0x55bf6e27f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:11.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.702+0000 7fdc53fff640 1 --2- 172.21.15.16:0/2994187163 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x55bf6e27ce50 0x55bf6e27f240 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cda0d0aabb46f432 server_cookie=b1c2aa9dff96ef33 in_seq=0 out_seq=0 2024-09-12T02:03:11.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.702+0000 7fdc337fe640 10 client.15642 ms_handle_connect on v2:172.21.15.184:6838/4203347941 2024-09-12T02:03:11.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.703+0000 7fdc337fe640 1 -- 172.21.15.16:0/2994187163 <== mds.1 v2:172.21.15.184:6838/4203347941 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bf6e1d5860 con 0x55bf6e27ce50 2024-09-12T02:03:11.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.703+0000 7fdc337fe640 10 client.15642 handle_command_reply: tid=0 2024-09-12T02:03:11.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.703+0000 7fdc5c66c740 1 client.15642 shutdown 2024-09-12T02:03:11.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.703+0000 7fdc5c66c740 1 -- 172.21.15.16:0/2994187163 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bf6e263b40 msgr2=0x55bf6e26a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:11.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.703+0000 7fdc5c66c740 1 --2- 172.21.15.16:0/2994187163 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bf6e263b40 0x55bf6e26a060 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fdc48009700 tx=0x7fdc48097e10 comp rx=0 tx=0).stop 2024-09-12T02:03:11.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.703+0000 7fdc5c66c740 1 -- 172.21.15.16:0/2994187163 shutdown_connections 2024-09-12T02:03:11.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.703+0000 7fdc5c66c740 1 --2- 172.21.15.16:0/2994187163 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x55bf6e27ce50 0x55bf6e27f240 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:11.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.703+0000 7fdc5c66c740 1 --2- 172.21.15.16:0/2994187163 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bf6e26d370 0x55bf6e26a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:11.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.703+0000 7fdc5c66c740 1 --2- 172.21.15.16:0/2994187163 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bf6e263b40 0x55bf6e26a060 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:11.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.703+0000 7fdc5c66c740 1 --2- 172.21.15.16:0/2994187163 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf6e2625d0 0x55bf6e26cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:11.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.703+0000 7fdc5c66c740 1 -- 172.21.15.16:0/2994187163 >> 172.21.15.16:0/2994187163 conn(0x55bf6e22b310 msgr2=0x55bf6e264960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:11.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.703+0000 7fdc337fe640 0 client.15642 ms_handle_reset on v2:172.21.15.184:6838/4203347941 2024-09-12T02:03:11.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.703+0000 7fdc5c66c740 1 -- 172.21.15.16:0/2994187163 shutdown_connections 2024-09-12T02:03:11.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.703+0000 7fdc5c66c740 1 -- 172.21.15.16:0/2994187163 wait complete. 2024-09-12T02:03:11.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.703+0000 7fdc5c66c740 20 client.15642 trim_cache size 0 max 16384 2024-09-12T02:03:11.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.716+0000 7fdc5c66c740 1 Processor -- start 2024-09-12T02:03:11.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.716+0000 7fdc5c66c740 1 -- start start 2024-09-12T02:03:11.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.716+0000 7fdc5c66c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bf6e2625d0 0x55bf6e25edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:11.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.716+0000 7fdc5c66c740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bf6e26d370 0x55bf6e2672c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:11.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.716+0000 7fdc5c66c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf6e267800 0x55bf6e267c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:11.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.717+0000 7fdc5c66c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fdc541b1ee0 con 0x55bf6e2625d0 2024-09-12T02:03:11.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.717+0000 7fdc5c66c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fdc541af7b0 con 0x55bf6e267800 2024-09-12T02:03:11.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.717+0000 7fdc5c66c740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fdc541ad600 con 0x55bf6e26d370 2024-09-12T02:03:11.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.717+0000 7fdc53fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bf6e2625d0 0x55bf6e25edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:11.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.717+0000 7fdc53fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bf6e2625d0 0x55bf6e25edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49482/0 (socket says 172.21.15.16:49482) 2024-09-12T02:03:11.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.717+0000 7fdc53fff640 1 -- 172.21.15.16:0/3075997521 learned_addr learned my addr 172.21.15.16:0/3075997521 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:03:11.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.717+0000 7fdc58830640 1 --2- 172.21.15.16:0/3075997521 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf6e267800 0x55bf6e267c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:11.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.717+0000 7fdc537fe640 1 --2- 172.21.15.16:0/3075997521 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bf6e26d370 0x55bf6e2672c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:11.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.717+0000 7fdc53fff640 1 -- 172.21.15.16:0/3075997521 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bf6e26d370 msgr2=0x55bf6e2672c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:11.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.717+0000 7fdc53fff640 1 --2- 172.21.15.16:0/3075997521 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bf6e26d370 0x55bf6e2672c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:11.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.717+0000 7fdc53fff640 1 -- 172.21.15.16:0/3075997521 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf6e267800 msgr2=0x55bf6e267c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:11.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.717+0000 7fdc53fff640 1 --2- 172.21.15.16:0/3075997521 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf6e267800 0x55bf6e267c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:11.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.717+0000 7fdc53fff640 1 -- 172.21.15.16:0/3075997521 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf6e25f960 con 0x55bf6e2625d0 2024-09-12T02:03:11.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.717+0000 7fdc53fff640 1 --2- 172.21.15.16:0/3075997521 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bf6e2625d0 0x55bf6e25edf0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fdc44009900 tx=0x7fdc44000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a7bacbef1281637 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:11.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.717+0000 7fdc39ffb640 1 -- 172.21.15.16:0/3075997521 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc44035040 con 0x55bf6e2625d0 2024-09-12T02:03:11.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.718+0000 7fdc39ffb640 1 -- 172.21.15.16:0/3075997521 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc44031a70 con 0x55bf6e2625d0 2024-09-12T02:03:11.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.718+0000 7fdc39ffb640 1 -- 172.21.15.16:0/3075997521 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc44004d00 con 0x55bf6e2625d0 2024-09-12T02:03:11.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.718+0000 7fdc5c66c740 1 -- 172.21.15.16:0/3075997521 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bf6e2625d0 msgr2=0x55bf6e25edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:11.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.718+0000 7fdc5c66c740 1 --2- 172.21.15.16:0/3075997521 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bf6e2625d0 0x55bf6e25edf0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fdc44009900 tx=0x7fdc44000bc0 comp rx=0 tx=0).stop 2024-09-12T02:03:11.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.718+0000 7fdc5c66c740 1 -- 172.21.15.16:0/3075997521 shutdown_connections 2024-09-12T02:03:11.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.718+0000 7fdc5c66c740 1 --2- 172.21.15.16:0/3075997521 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf6e267800 0x55bf6e267c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:11.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.718+0000 7fdc5c66c740 1 --2- 172.21.15.16:0/3075997521 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bf6e26d370 0x55bf6e2672c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:11.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.718+0000 7fdc5c66c740 1 --2- 172.21.15.16:0/3075997521 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bf6e2625d0 0x55bf6e25edf0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:11.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.718+0000 7fdc5c66c740 1 -- 172.21.15.16:0/3075997521 >> 172.21.15.16:0/3075997521 conn(0x55bf6e22b310 msgr2=0x55bf6e25e6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:11.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.718+0000 7fdc5c66c740 1 -- 172.21.15.16:0/3075997521 shutdown_connections 2024-09-12T02:03:11.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.718+0000 7fdc5c66c740 1 -- 172.21.15.16:0/3075997521 wait complete. 2024-09-12T02:03:11.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.719+0000 7fdc5c66c740 1 -- start start 2024-09-12T02:03:11.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.719+0000 7fdc5c66c740 1 Processor -- start 2024-09-12T02:03:11.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.719+0000 7fdc5c66c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bf6e2625d0 0x55bf6e27f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:11.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.719+0000 7fdc5c66c740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bf6e267800 0x55bf6e27fc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:11.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.719+0000 7fdc5c66c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf6e26d370 0x55bf6e2de740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:11.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.719+0000 7fdc5c66c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fdc541adc70 con 0x55bf6e2625d0 2024-09-12T02:03:11.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.719+0000 7fdc5c66c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fdc541af0e0 con 0x55bf6e26d370 2024-09-12T02:03:11.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.719+0000 7fdc5c66c740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fdc541ae340 con 0x55bf6e267800 2024-09-12T02:03:11.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.719+0000 7fdc5c66c740 10 client.0 _ll_register_callbacks cb 0x7fdc4c00b4e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:03:11.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.719+0000 7fdc53fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bf6e2625d0 0x55bf6e27f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:11.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.719+0000 7fdc53fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bf6e2625d0 0x55bf6e27f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49498/0 (socket says 172.21.15.16:49498) 2024-09-12T02:03:11.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.719+0000 7fdc53fff640 1 -- 172.21.15.16:0/1844827234 learned_addr learned my addr 172.21.15.16:0/1844827234 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:03:11.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.719+0000 7fdc58830640 1 --2- 172.21.15.16:0/1844827234 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf6e26d370 0x55bf6e2de740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:11.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.719+0000 7fdc537fe640 1 --2- 172.21.15.16:0/1844827234 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bf6e267800 0x55bf6e27fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:11.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.720+0000 7fdc53fff640 1 -- 172.21.15.16:0/1844827234 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bf6e267800 msgr2=0x55bf6e27fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:11.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.720+0000 7fdc53fff640 1 --2- 172.21.15.16:0/1844827234 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bf6e267800 0x55bf6e27fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:11.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.720+0000 7fdc53fff640 1 -- 172.21.15.16:0/1844827234 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf6e26d370 msgr2=0x55bf6e2de740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:11.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.720+0000 7fdc53fff640 1 --2- 172.21.15.16:0/1844827234 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf6e26d370 0x55bf6e2de740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:11.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.720+0000 7fdc53fff640 1 -- 172.21.15.16:0/1844827234 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf6e2dede0 con 0x55bf6e2625d0 2024-09-12T02:03:11.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.720+0000 7fdc58830640 1 --2- 172.21.15.16:0/1844827234 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf6e26d370 0x55bf6e2de740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:03:11.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.720+0000 7fdc53fff640 1 --2- 172.21.15.16:0/1844827234 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bf6e2625d0 0x55bf6e27f740 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fdc44004620 tx=0x7fdc44031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f61eec8f78019e93 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:11.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.720+0000 7fdc337fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:03:11.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.720+0000 7fdc337fe640 1 -- 172.21.15.16:0/1844827234 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc44055070 con 0x55bf6e2625d0 2024-09-12T02:03:11.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.720+0000 7fdc337fe640 1 -- 172.21.15.16:0/1844827234 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fdc44047670 con 0x55bf6e2625d0 2024-09-12T02:03:11.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.720+0000 7fdc5c66c740 1 -- 172.21.15.16:0/1844827234 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc44031a70 con 0x55bf6e2625d0 2024-09-12T02:03:11.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.720+0000 7fdc337fe640 1 -- 172.21.15.16:0/1844827234 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fdc44035070 con 0x55bf6e2625d0 2024-09-12T02:03:11.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.720+0000 7fdc5c66c740 10 client.15648 fetch_fsmap learned FSMap version 25 2024-09-12T02:03:11.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.720+0000 7fdc5c66c740 1 -- 172.21.15.16:0/1844827234 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55bf6e2e7370 con 0x55bf6e2625d0 2024-09-12T02:03:11.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.721+0000 7fdc337fe640 1 -- 172.21.15.16:0/1844827234 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc44031a70 con 0x55bf6e2625d0 2024-09-12T02:03:11.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.721+0000 7fdc337fe640 1 -- 172.21.15.16:0/1844827234 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fdc44005360 con 0x55bf6e2625d0 2024-09-12T02:03:11.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.721+0000 7fdc5c66c740 10 client.15648 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:03:11.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.721+0000 7fdc5c66c740 10 client.15648 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T02:03:11.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.721+0000 7fdc5c66c740 20 client.15648 populate_metadata read hostname 'smithi016' 2024-09-12T02:03:11.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.721+0000 7fdc5c66c740 1 --2- 172.21.15.16:0/1844827234 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x55bf6e2e7a10 0x55bf6e2e9e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:11.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.721+0000 7fdc5c66c740 4 client.15648 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T02:03:11.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.721+0000 7fdc5c66c740 1 -- 172.21.15.16:0/1844827234 --> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bf6e1cfd70 con 0x55bf6e2e7a10 2024-09-12T02:03:11.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.721+0000 7fdc537fe640 1 --2- 172.21.15.16:0/1844827234 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x55bf6e2e7a10 0x55bf6e2e9e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:11.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.722+0000 7fdc337fe640 10 client.15648 ms_handle_connect on v2:172.21.15.184:6838/4203347941 2024-09-12T02:03:11.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.722+0000 7fdc537fe640 1 --2- 172.21.15.16:0/1844827234 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x55bf6e2e7a10 0x55bf6e2e9e30 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=49f3780ff6d23714 server_cookie=78c68fa404e83589 in_seq=0 out_seq=0 2024-09-12T02:03:11.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.722+0000 7fdc337fe640 1 -- 172.21.15.16:0/1844827234 <== mds.1 v2:172.21.15.184:6838/4203347941 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bf6e1cfd70 con 0x55bf6e2e7a10 2024-09-12T02:03:11.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.722+0000 7fdc337fe640 10 client.15648 handle_command_reply: tid=0 2024-09-12T02:03:11.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.722+0000 7fdc5c66c740 1 client.15648 shutdown 2024-09-12T02:03:11.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.722+0000 7fdc5c66c740 1 -- 172.21.15.16:0/1844827234 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bf6e2625d0 msgr2=0x55bf6e27f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:11.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.722+0000 7fdc5c66c740 1 --2- 172.21.15.16:0/1844827234 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bf6e2625d0 0x55bf6e27f740 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fdc44004620 tx=0x7fdc44031400 comp rx=0 tx=0).stop 2024-09-12T02:03:11.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.722+0000 7fdc5c66c740 1 -- 172.21.15.16:0/1844827234 shutdown_connections 2024-09-12T02:03:11.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.722+0000 7fdc5c66c740 1 --2- 172.21.15.16:0/1844827234 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x55bf6e2e7a10 0x55bf6e2e9e30 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:11.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.722+0000 7fdc5c66c740 1 --2- 172.21.15.16:0/1844827234 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf6e26d370 0x55bf6e2de740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:11.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.722+0000 7fdc5c66c740 1 --2- 172.21.15.16:0/1844827234 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bf6e267800 0x55bf6e27fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:11.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.722+0000 7fdc5c66c740 1 --2- 172.21.15.16:0/1844827234 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bf6e2625d0 0x55bf6e27f740 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:11.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.722+0000 7fdc337fe640 0 client.15648 ms_handle_reset on v2:172.21.15.184:6838/4203347941 2024-09-12T02:03:11.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.722+0000 7fdc5c66c740 1 -- 172.21.15.16:0/1844827234 >> 172.21.15.16:0/1844827234 conn(0x55bf6e22b310 msgr2=0x55bf6e278420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:11.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.722+0000 7fdc5c66c740 1 -- 172.21.15.16:0/1844827234 shutdown_connections 2024-09-12T02:03:11.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.722+0000 7fdc5c66c740 1 -- 172.21.15.16:0/1844827234 wait complete. 2024-09-12T02:03:11.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.723+0000 7fdc5c66c740 20 client.15648 trim_cache size 0 max 16384 2024-09-12T02:03:11.728 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T02:03:11.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.723+0000 7fdc5a293640 1 -- 172.21.15.16:0/1120447133 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fdc3c0a2b80 msgr2=0x7fdc3c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:11.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.723+0000 7fdc5a293640 1 --2- 172.21.15.16:0/1120447133 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fdc3c0a2b80 0x7fdc3c0a5040 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fdc4c005fd0 tx=0x7fdc4c004ab0 comp rx=0 tx=0).stop 2024-09-12T02:03:11.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.723+0000 7fdc5a293640 1 -- 172.21.15.16:0/1120447133 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc54103660 msgr2=0x7fdc54100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:11.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.723+0000 7fdc5a293640 1 --2- 172.21.15.16:0/1120447133 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc54103660 0x7fdc54100ee0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fdc4800e9f0 tx=0x7fdc4800eeb0 comp rx=0 tx=0).stop 2024-09-12T02:03:11.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.723+0000 7fdc5a293640 1 -- 172.21.15.16:0/1120447133 shutdown_connections 2024-09-12T02:03:11.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.723+0000 7fdc5a293640 1 --2- 172.21.15.16:0/1120447133 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fdc3c0a2b80 0x7fdc3c0a5040 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:11.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.723+0000 7fdc5a293640 1 --2- 172.21.15.16:0/1120447133 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdc5410c170 0x7fdc54101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:11.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.723+0000 7fdc5a293640 1 --2- 172.21.15.16:0/1120447133 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc54103660 0x7fdc54100ee0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:11.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.723+0000 7fdc5a293640 1 --2- 172.21.15.16:0/1120447133 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdc54102d40 0x7fdc541009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:11.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.723+0000 7fdc5a293640 1 -- 172.21.15.16:0/1120447133 >> 172.21.15.16:0/1120447133 conn(0x7fdc540fc7a0 msgr2=0x7fdc540fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:11.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.723+0000 7fdc5a293640 1 -- 172.21.15.16:0/1120447133 shutdown_connections 2024-09-12T02:03:11.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:11.723+0000 7fdc5a293640 1 -- 172.21.15.16:0/1120447133 wait complete. 2024-09-12T02:03:11.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:03:11 smithi184 ceph-mon[35593]: pgmap v373: 97 pgs: 97 active+clean; 772 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 13 op/s 2024-09-12T02:03:11.918 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:03:11 smithi033 ceph-mon[35332]: pgmap v373: 97 pgs: 97 active+clean; 772 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 13 op/s 2024-09-12T02:03:12.448 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-12T02:03:12.715 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:03:13.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.133+0000 7f4e616bb640 1 -- 172.21.15.16:0/2978320180 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e5c1036e0 msgr2=0x7f4e5c107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:13.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.133+0000 7f4e616bb640 1 --2- 172.21.15.16:0/2978320180 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e5c1036e0 0x7f4e5c107b90 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f4e44009930 tx=0x7f4e4402a420 comp rx=0 tx=0).stop 2024-09-12T02:03:13.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.133+0000 7f4e616bb640 1 -- 172.21.15.16:0/2978320180 shutdown_connections 2024-09-12T02:03:13.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.133+0000 7f4e616bb640 1 --2- 172.21.15.16:0/2978320180 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4e5c1082a0 0x7f4e5c10ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:13.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.133+0000 7f4e616bb640 1 --2- 172.21.15.16:0/2978320180 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e5c1036e0 0x7f4e5c107b90 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:13.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.133+0000 7f4e616bb640 1 --2- 172.21.15.16:0/2978320180 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e5c102d30 0x7f4e5c103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:13.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.133+0000 7f4e616bb640 1 -- 172.21.15.16:0/2978320180 >> 172.21.15.16:0/2978320180 conn(0x7f4e5c0fc7d0 msgr2=0x7f4e5c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:13.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.133+0000 7f4e616bb640 1 -- 172.21.15.16:0/2978320180 shutdown_connections 2024-09-12T02:03:13.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.133+0000 7f4e616bb640 1 -- 172.21.15.16:0/2978320180 wait complete. 2024-09-12T02:03:13.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.133+0000 7f4e616bb640 1 Processor -- start 2024-09-12T02:03:13.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.134+0000 7f4e616bb640 1 -- start start 2024-09-12T02:03:13.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.134+0000 7f4e616bb640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4e5c102d30 0x7f4e5c198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:13.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.134+0000 7f4e616bb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e5c1036e0 0x7f4e5c198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:13.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.134+0000 7f4e616bb640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e5c1082a0 0x7f4e5c19cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:13.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.134+0000 7f4e616bb640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4e5c112060 con 0x7f4e5c1036e0 2024-09-12T02:03:13.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.134+0000 7f4e616bb640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4e5c111ee0 con 0x7f4e5c1082a0 2024-09-12T02:03:13.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.134+0000 7f4e616bb640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f4e5c1121e0 con 0x7f4e5c102d30 2024-09-12T02:03:13.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.134+0000 7f4e5a7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e5c1036e0 0x7f4e5c198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:13.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.134+0000 7f4e5a7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e5c1036e0 0x7f4e5c198a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49508/0 (socket says 172.21.15.16:49508) 2024-09-12T02:03:13.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.134+0000 7f4e5a7fc640 1 -- 172.21.15.16:0/1401911983 learned_addr learned my addr 172.21.15.16:0/1401911983 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:03:13.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.134+0000 7f4e5affd640 1 --2- 172.21.15.16:0/1401911983 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4e5c102d30 0x7f4e5c198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:13.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.134+0000 7f4e5b7fe640 1 --2- 172.21.15.16:0/1401911983 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e5c1082a0 0x7f4e5c19cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:13.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.134+0000 7f4e5a7fc640 1 -- 172.21.15.16:0/1401911983 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4e5c102d30 msgr2=0x7f4e5c198500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:13.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.134+0000 7f4e5a7fc640 1 --2- 172.21.15.16:0/1401911983 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4e5c102d30 0x7f4e5c198500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:13.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.134+0000 7f4e5a7fc640 1 -- 172.21.15.16:0/1401911983 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e5c1082a0 msgr2=0x7f4e5c19cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:13.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.134+0000 7f4e5a7fc640 1 --2- 172.21.15.16:0/1401911983 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e5c1082a0 0x7f4e5c19cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:13.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.135+0000 7f4e5a7fc640 1 -- 172.21.15.16:0/1401911983 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e5c19d4b0 con 0x7f4e5c1036e0 2024-09-12T02:03:13.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.135+0000 7f4e5b7fe640 1 --2- 172.21.15.16:0/1401911983 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e5c1082a0 0x7f4e5c19cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:03:13.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.135+0000 7f4e5a7fc640 1 --2- 172.21.15.16:0/1401911983 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e5c1036e0 0x7f4e5c198a40 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f4e44005940 tx=0x7f4e440042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:13.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.135+0000 7f4e53fff640 1 -- 172.21.15.16:0/1401911983 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e4403f030 con 0x7f4e5c1036e0 2024-09-12T02:03:13.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.135+0000 7f4e53fff640 1 -- 172.21.15.16:0/1401911983 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e44003ce0 con 0x7f4e5c1036e0 2024-09-12T02:03:13.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.135+0000 7f4e616bb640 1 -- 172.21.15.16:0/1401911983 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e5c19d740 con 0x7f4e5c1036e0 2024-09-12T02:03:13.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.135+0000 7f4e53fff640 1 -- 172.21.15.16:0/1401911983 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e44035070 con 0x7f4e5c1036e0 2024-09-12T02:03:13.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.135+0000 7f4e616bb640 1 -- 172.21.15.16:0/1401911983 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e5c1a5080 con 0x7f4e5c1036e0 2024-09-12T02:03:13.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.137+0000 7f4e53fff640 1 -- 172.21.15.16:0/1401911983 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f4e44042030 con 0x7f4e5c1036e0 2024-09-12T02:03:13.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.137+0000 7f4e63a94740 1 Processor -- start 2024-09-12T02:03:13.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.138+0000 7f4e63a94740 1 -- start start 2024-09-12T02:03:13.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.138+0000 7f4e63a94740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d2614afb40 0x55d2614ae090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:13.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.138+0000 7f4e63a94740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2614ae5d0 0x55d2614aea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:13.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.138+0000 7f4e63a94740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d2614bd780 0x55d2614bfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:13.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.138+0000 7f4e63a94740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55d2614142f0 con 0x55d2614ae5d0 2024-09-12T02:03:13.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.138+0000 7f4e63a94740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55d2613bbcf0 con 0x55d2614afb40 2024-09-12T02:03:13.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.138+0000 7f4e63a94740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55d2614102b0 con 0x55d2614bd780 2024-09-12T02:03:13.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.138+0000 7f4e5affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2614ae5d0 0x55d2614aea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:13.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.138+0000 7f4e5affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2614ae5d0 0x55d2614aea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49510/0 (socket says 172.21.15.16:49510) 2024-09-12T02:03:13.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.138+0000 7f4e5affd640 1 -- 172.21.15.16:0/870926639 learned_addr learned my addr 172.21.15.16:0/870926639 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:03:13.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.138+0000 7f4e53fff640 1 --2- 172.21.15.16:0/1401911983 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f4e480a2c00 0x7f4e480a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:13.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.138+0000 7f4e5b7fe640 1 --2- 172.21.15.16:0/870926639 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d2614afb40 0x55d2614ae090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:13.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.138+0000 7f4e5a7fc640 1 --2- 172.21.15.16:0/870926639 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d2614bd780 0x55d2614bfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:13.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.138+0000 7f4e5b7fe640 1 --2- 172.21.15.16:0/1401911983 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f4e480a2c00 0x7f4e480a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:13.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.138+0000 7f4e5affd640 1 -- 172.21.15.16:0/870926639 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d2614bd780 msgr2=0x55d2614bfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:13.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.138+0000 7f4e5affd640 1 --2- 172.21.15.16:0/870926639 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d2614bd780 0x55d2614bfb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:13.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.139+0000 7f4e5affd640 1 -- 172.21.15.16:0/870926639 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d2614afb40 msgr2=0x55d2614ae090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:13.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.139+0000 7f4e5affd640 1 --2- 172.21.15.16:0/870926639 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d2614afb40 0x55d2614ae090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:13.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.139+0000 7f4e53fff640 1 -- 172.21.15.16:0/1401911983 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f4e440ef3f0 con 0x7f4e5c1036e0 2024-09-12T02:03:13.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.139+0000 7f4e5affd640 1 -- 172.21.15.16:0/870926639 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2613a03a0 con 0x55d2614ae5d0 2024-09-12T02:03:13.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.139+0000 7f4e5affd640 1 --2- 172.21.15.16:0/870926639 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2614ae5d0 0x55d2614aea10 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f4e54007f90 tx=0x7f4e5400f850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50633c1b8472cb81 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:13.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.139+0000 7f4e517fa640 1 -- 172.21.15.16:0/870926639 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e54004d20 con 0x55d2614ae5d0 2024-09-12T02:03:13.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.139+0000 7f4e5b7fe640 1 --2- 172.21.15.16:0/1401911983 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f4e480a2c00 0x7f4e480a50c0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c005fd0 tx=0x7f4e4c005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:13.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.139+0000 7f4e517fa640 1 -- 172.21.15.16:0/870926639 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e54010be0 con 0x55d2614ae5d0 2024-09-12T02:03:13.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.139+0000 7f4e517fa640 1 -- 172.21.15.16:0/870926639 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e540057a0 con 0x55d2614ae5d0 2024-09-12T02:03:13.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.140+0000 7f4e63a94740 1 -- 172.21.15.16:0/870926639 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2614ae5d0 msgr2=0x55d2614aea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:13.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.140+0000 7f4e63a94740 1 --2- 172.21.15.16:0/870926639 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2614ae5d0 0x55d2614aea10 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f4e54007f90 tx=0x7f4e5400f850 comp rx=0 tx=0).stop 2024-09-12T02:03:13.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.140+0000 7f4e63a94740 1 -- 172.21.15.16:0/870926639 shutdown_connections 2024-09-12T02:03:13.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.140+0000 7f4e63a94740 1 --2- 172.21.15.16:0/870926639 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d2614bd780 0x55d2614bfb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:13.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.140+0000 7f4e63a94740 1 --2- 172.21.15.16:0/870926639 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2614ae5d0 0x55d2614aea10 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:13.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.140+0000 7f4e63a94740 1 --2- 172.21.15.16:0/870926639 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d2614afb40 0x55d2614ae090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:13.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.140+0000 7f4e63a94740 1 -- 172.21.15.16:0/870926639 >> 172.21.15.16:0/870926639 conn(0x55d261477310 msgr2=0x55d2614aac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:13.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.140+0000 7f4e63a94740 1 -- 172.21.15.16:0/870926639 shutdown_connections 2024-09-12T02:03:13.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.140+0000 7f4e63a94740 1 -- 172.21.15.16:0/870926639 wait complete. 2024-09-12T02:03:13.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.140+0000 7f4e63a94740 1 -- start start 2024-09-12T02:03:13.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.140+0000 7f4e63a94740 1 Processor -- start 2024-09-12T02:03:13.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.140+0000 7f4e63a94740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d2614ae5d0 0x55d2614bd0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:13.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.140+0000 7f4e63a94740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d2614afb40 0x55d2614ba370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:13.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.140+0000 7f4e63a94740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2614bd780 0x55d2614ba8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:13.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.141+0000 7f4e63a94740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55d2614c0480 con 0x55d2614bd780 2024-09-12T02:03:13.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.141+0000 7f4e63a94740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55d2613cf060 con 0x55d2614ae5d0 2024-09-12T02:03:13.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.141+0000 7f4e63a94740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55d2614c0600 con 0x55d2614afb40 2024-09-12T02:03:13.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.141+0000 7f4e63a94740 10 client.0 _ll_register_callbacks cb 0x55d2611228c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:03:13.176 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T02:03:13.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.141+0000 7f4e5b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2614bd780 0x55d2614ba8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:13.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.141+0000 7f4e5b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2614bd780 0x55d2614ba8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49524/0 (socket says 172.21.15.16:49524) 2024-09-12T02:03:13.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.141+0000 7f4e5b7fe640 1 -- 172.21.15.16:0/4062621979 learned_addr learned my addr 172.21.15.16:0/4062621979 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:03:13.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.141+0000 7f4e5affd640 1 --2- 172.21.15.16:0/4062621979 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d2614ae5d0 0x55d2614bd0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:13.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.141+0000 7f4e5a7fc640 1 --2- 172.21.15.16:0/4062621979 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d2614afb40 0x55d2614ba370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:13.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.141+0000 7f4e5b7fe640 1 -- 172.21.15.16:0/4062621979 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d2614afb40 msgr2=0x55d2614ba370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:13.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.141+0000 7f4e5b7fe640 1 --2- 172.21.15.16:0/4062621979 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d2614afb40 0x55d2614ba370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:13.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.141+0000 7f4e5b7fe640 1 -- 172.21.15.16:0/4062621979 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d2614ae5d0 msgr2=0x55d2614bd0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:13.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.141+0000 7f4e5b7fe640 1 --2- 172.21.15.16:0/4062621979 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d2614ae5d0 0x55d2614bd0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:13.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.141+0000 7f4e5b7fe640 1 -- 172.21.15.16:0/4062621979 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2611853c0 con 0x55d2614bd780 2024-09-12T02:03:13.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.141+0000 7f4e5affd640 1 --2- 172.21.15.16:0/4062621979 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d2614ae5d0 0x55d2614bd0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:03:13.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.141+0000 7f4e5b7fe640 1 --2- 172.21.15.16:0/4062621979 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2614bd780 0x55d2614ba8b0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c00fca0 tx=0x7f4e4c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f6918585bd6b4f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:13.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.141+0000 7f4e427fc640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:03:13.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.141+0000 7f4e427fc640 1 -- 172.21.15.16:0/4062621979 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e4c023070 con 0x55d2614bd780 2024-09-12T02:03:13.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.141+0000 7f4e427fc640 1 -- 172.21.15.16:0/4062621979 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f4e4c01f400 con 0x55d2614bd780 2024-09-12T02:03:13.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.141+0000 7f4e63a94740 1 -- 172.21.15.16:0/4062621979 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4e54010be0 con 0x55d2614bd780 2024-09-12T02:03:13.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.142+0000 7f4e427fc640 1 -- 172.21.15.16:0/4062621979 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4e4c02f050 con 0x55d2614bd780 2024-09-12T02:03:13.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.142+0000 7f4e63a94740 10 client.15660 fetch_fsmap learned FSMap version 25 2024-09-12T02:03:13.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.142+0000 7f4e63a94740 1 -- 172.21.15.16:0/4062621979 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d2614cbed0 con 0x55d2614bd780 2024-09-12T02:03:13.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.142+0000 7f4e427fc640 1 -- 172.21.15.16:0/4062621979 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e54010be0 con 0x55d2614bd780 2024-09-12T02:03:13.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.142+0000 7f4e427fc640 1 -- 172.21.15.16:0/4062621979 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f4e4c01e040 con 0x55d2614bd780 2024-09-12T02:03:13.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.142+0000 7f4e63a94740 10 client.15660 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:03:13.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.142+0000 7f4e63a94740 10 client.15660 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:03:13.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.142+0000 7f4e63a94740 20 client.15660 populate_metadata read hostname 'smithi016' 2024-09-12T02:03:13.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.142+0000 7f4e63a94740 1 --2- 172.21.15.16:0/4062621979 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55d2614cd2e0 0x55d2614cf7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:13.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.142+0000 7f4e63a94740 4 client.15660 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:03:13.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.142+0000 7f4e63a94740 1 -- 172.21.15.16:0/4062621979 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d261421860 con 0x55d2614cd2e0 2024-09-12T02:03:13.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.142+0000 7f4e5affd640 1 --2- 172.21.15.16:0/4062621979 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55d2614cd2e0 0x55d2614cf7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:13.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.143+0000 7f4e5affd640 1 --2- 172.21.15.16:0/4062621979 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55d2614cd2e0 0x55d2614cf7a0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=16560e84fe4e9a02 server_cookie=738e19a91e36a4a0 in_seq=0 out_seq=0 2024-09-12T02:03:13.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.143+0000 7f4e427fc640 10 client.15660 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:03:13.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.144+0000 7f4e427fc640 1 -- 172.21.15.16:0/4062621979 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d261421860 con 0x55d2614cd2e0 2024-09-12T02:03:13.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.144+0000 7f4e427fc640 10 client.15660 handle_command_reply: tid=0 2024-09-12T02:03:13.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.144+0000 7f4e63a94740 1 client.15660 shutdown 2024-09-12T02:03:13.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.144+0000 7f4e63a94740 1 -- 172.21.15.16:0/4062621979 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2614bd780 msgr2=0x55d2614ba8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:13.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.144+0000 7f4e63a94740 1 --2- 172.21.15.16:0/4062621979 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2614bd780 0x55d2614ba8b0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c00fca0 tx=0x7f4e4c0046c0 comp rx=0 tx=0).stop 2024-09-12T02:03:13.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.144+0000 7f4e63a94740 1 -- 172.21.15.16:0/4062621979 shutdown_connections 2024-09-12T02:03:13.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.144+0000 7f4e63a94740 1 --2- 172.21.15.16:0/4062621979 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55d2614cd2e0 0x55d2614cf7a0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:13.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.144+0000 7f4e63a94740 1 --2- 172.21.15.16:0/4062621979 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2614bd780 0x55d2614ba8b0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:13.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.144+0000 7f4e63a94740 1 --2- 172.21.15.16:0/4062621979 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d2614afb40 0x55d2614ba370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:13.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.144+0000 7f4e427fc640 0 client.15660 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:03:13.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.144+0000 7f4e63a94740 1 --2- 172.21.15.16:0/4062621979 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d2614ae5d0 0x55d2614bd0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:13.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.144+0000 7f4e63a94740 1 -- 172.21.15.16:0/4062621979 >> 172.21.15.16:0/4062621979 conn(0x55d261477310 msgr2=0x55d2614b0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:13.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.144+0000 7f4e63a94740 1 -- 172.21.15.16:0/4062621979 shutdown_connections 2024-09-12T02:03:13.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.144+0000 7f4e63a94740 1 -- 172.21.15.16:0/4062621979 wait complete. 2024-09-12T02:03:13.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.144+0000 7f4e63a94740 20 client.15660 trim_cache size 0 max 16384 2024-09-12T02:03:13.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.157+0000 7f4e63a94740 1 Processor -- start 2024-09-12T02:03:13.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.157+0000 7f4e63a94740 1 -- start start 2024-09-12T02:03:13.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.157+0000 7f4e63a94740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d2614ae5d0 0x55d2614c9720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:13.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.157+0000 7f4e63a94740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2614afb40 0x55d2614aa700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:13.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.157+0000 7f4e63a94740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d2614bd780 0x55d2614aac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:13.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.157+0000 7f4e63a94740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4e5c1adad0 con 0x55d2614afb40 2024-09-12T02:03:13.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.157+0000 7f4e63a94740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4e5c1ab3a0 con 0x55d2614ae5d0 2024-09-12T02:03:13.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.157+0000 7f4e63a94740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f4e5c1a9210 con 0x55d2614bd780 2024-09-12T02:03:13.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.158+0000 7f4e5a7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2614afb40 0x55d2614aa700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:13.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.158+0000 7f4e5a7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2614afb40 0x55d2614aa700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49526/0 (socket says 172.21.15.16:49526) 2024-09-12T02:03:13.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.158+0000 7f4e5a7fc640 1 -- 172.21.15.16:0/283688074 learned_addr learned my addr 172.21.15.16:0/283688074 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:03:13.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.158+0000 7f4e5a7fc640 1 -- 172.21.15.16:0/283688074 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d2614bd780 msgr2=0x55d2614aac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:13.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.158+0000 7f4e5a7fc640 1 --2- 172.21.15.16:0/283688074 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d2614bd780 0x55d2614aac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:13.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.158+0000 7f4e5a7fc640 1 -- 172.21.15.16:0/283688074 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d2614ae5d0 msgr2=0x55d2614c9720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:13.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.158+0000 7f4e5a7fc640 1 --2- 172.21.15.16:0/283688074 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d2614ae5d0 0x55d2614c9720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:13.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.158+0000 7f4e5a7fc640 1 -- 172.21.15.16:0/283688074 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2614ab180 con 0x55d2614afb40 2024-09-12T02:03:13.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.158+0000 7f4e5a7fc640 1 --2- 172.21.15.16:0/283688074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2614afb40 0x55d2614aa700 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f4e44031550 tx=0x7f4e4408dd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22ae472934cb33b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:13.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.158+0000 7f4e50ff9640 1 -- 172.21.15.16:0/283688074 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e44049040 con 0x55d2614afb40 2024-09-12T02:03:13.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.158+0000 7f4e50ff9640 1 -- 172.21.15.16:0/283688074 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e440bc3f0 con 0x55d2614afb40 2024-09-12T02:03:13.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.158+0000 7f4e50ff9640 1 -- 172.21.15.16:0/283688074 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e440b9ff0 con 0x55d2614afb40 2024-09-12T02:03:13.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.159+0000 7f4e63a94740 1 -- 172.21.15.16:0/283688074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2614afb40 msgr2=0x55d2614aa700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:13.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.159+0000 7f4e63a94740 1 --2- 172.21.15.16:0/283688074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2614afb40 0x55d2614aa700 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f4e44031550 tx=0x7f4e4408dd00 comp rx=0 tx=0).stop 2024-09-12T02:03:13.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.159+0000 7f4e63a94740 1 -- 172.21.15.16:0/283688074 shutdown_connections 2024-09-12T02:03:13.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.159+0000 7f4e63a94740 1 --2- 172.21.15.16:0/283688074 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d2614bd780 0x55d2614aac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:13.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.159+0000 7f4e63a94740 1 --2- 172.21.15.16:0/283688074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2614afb40 0x55d2614aa700 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:13.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.159+0000 7f4e63a94740 1 --2- 172.21.15.16:0/283688074 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d2614ae5d0 0x55d2614c9720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:13.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.159+0000 7f4e63a94740 1 -- 172.21.15.16:0/283688074 >> 172.21.15.16:0/283688074 conn(0x55d261477310 msgr2=0x55d2614ba330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:13.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.159+0000 7f4e63a94740 1 -- 172.21.15.16:0/283688074 shutdown_connections 2024-09-12T02:03:13.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.159+0000 7f4e63a94740 1 -- 172.21.15.16:0/283688074 wait complete. 2024-09-12T02:03:13.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.159+0000 7f4e63a94740 1 -- start start 2024-09-12T02:03:13.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.159+0000 7f4e63a94740 1 Processor -- start 2024-09-12T02:03:13.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.160+0000 7f4e63a94740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d2614ae5d0 0x55d2614c16f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:13.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.160+0000 7f4e63a94740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2614afb40 0x55d2614c1c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:13.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.160+0000 7f4e63a94740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d2614bd780 0x55d2614c2170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:13.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.160+0000 7f4e63a94740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4e5c1a9860 con 0x55d2614afb40 2024-09-12T02:03:13.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.160+0000 7f4e63a94740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4e5c1aacd0 con 0x55d2614bd780 2024-09-12T02:03:13.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.160+0000 7f4e63a94740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f4e5c1a9f30 con 0x55d2614ae5d0 2024-09-12T02:03:13.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.160+0000 7f4e63a94740 10 client.0 _ll_register_callbacks cb 0x7f4e4c00f450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:03:13.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.160+0000 7f4e5a7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2614afb40 0x55d2614c1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:13.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.160+0000 7f4e5a7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2614afb40 0x55d2614c1c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49536/0 (socket says 172.21.15.16:49536) 2024-09-12T02:03:13.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.160+0000 7f4e5a7fc640 1 -- 172.21.15.16:0/2627615036 learned_addr learned my addr 172.21.15.16:0/2627615036 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:03:13.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.160+0000 7f4e5affd640 1 --2- 172.21.15.16:0/2627615036 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d2614ae5d0 0x55d2614c16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:13.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.160+0000 7f4e5b7fe640 1 --2- 172.21.15.16:0/2627615036 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d2614bd780 0x55d2614c2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:13.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.160+0000 7f4e5a7fc640 1 -- 172.21.15.16:0/2627615036 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d2614ae5d0 msgr2=0x55d2614c16f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:13.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.160+0000 7f4e5a7fc640 1 --2- 172.21.15.16:0/2627615036 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d2614ae5d0 0x55d2614c16f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:13.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.160+0000 7f4e5a7fc640 1 -- 172.21.15.16:0/2627615036 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d2614bd780 msgr2=0x55d2614c2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:13.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.160+0000 7f4e5a7fc640 1 --2- 172.21.15.16:0/2627615036 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d2614bd780 0x55d2614c2170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:13.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.160+0000 7f4e5a7fc640 1 -- 172.21.15.16:0/2627615036 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d261537140 con 0x55d2614afb40 2024-09-12T02:03:13.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.160+0000 7f4e5a7fc640 1 --2- 172.21.15.16:0/2627615036 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2614afb40 0x55d2614c1c30 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f4e440bb9f0 tx=0x7f4e4403b6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5557d2f29ca945 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:13.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.160+0000 7f4e427fc640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:03:13.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.161+0000 7f4e427fc640 1 -- 172.21.15.16:0/2627615036 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e44049070 con 0x55d2614afb40 2024-09-12T02:03:13.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.161+0000 7f4e427fc640 1 -- 172.21.15.16:0/2627615036 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f4e44003800 con 0x55d2614afb40 2024-09-12T02:03:13.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.161+0000 7f4e63a94740 1 -- 172.21.15.16:0/2627615036 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4e440bc3f0 con 0x55d2614afb40 2024-09-12T02:03:13.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.161+0000 7f4e427fc640 1 -- 172.21.15.16:0/2627615036 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4e440c4960 con 0x55d2614afb40 2024-09-12T02:03:13.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.161+0000 7f4e63a94740 10 client.15666 fetch_fsmap learned FSMap version 25 2024-09-12T02:03:13.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.161+0000 7f4e63a94740 1 -- 172.21.15.16:0/2627615036 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d261538420 con 0x55d2614afb40 2024-09-12T02:03:13.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.161+0000 7f4e427fc640 1 -- 172.21.15.16:0/2627615036 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e440bc3f0 con 0x55d2614afb40 2024-09-12T02:03:13.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.161+0000 7f4e427fc640 1 -- 172.21.15.16:0/2627615036 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f4e440d3070 con 0x55d2614afb40 2024-09-12T02:03:13.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.161+0000 7f4e63a94740 10 client.15666 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:03:13.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.161+0000 7f4e63a94740 10 client.15666 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:03:13.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.161+0000 7f4e63a94740 20 client.15666 populate_metadata read hostname 'smithi016' 2024-09-12T02:03:13.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.161+0000 7f4e63a94740 1 --2- 172.21.15.16:0/2627615036 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55d261538af0 0x55d26153af10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:13.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.161+0000 7f4e63a94740 4 client.15666 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T02:03:13.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.161+0000 7f4e63a94740 1 -- 172.21.15.16:0/2627615036 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d26141bd70 con 0x55d261538af0 2024-09-12T02:03:13.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.161+0000 7f4e5affd640 1 --2- 172.21.15.16:0/2627615036 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55d261538af0 0x55d26153af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:13.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.162+0000 7f4e5affd640 1 --2- 172.21.15.16:0/2627615036 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55d261538af0 0x55d26153af10 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b19e0caa382d6a85 server_cookie=77e681479ccab866 in_seq=0 out_seq=0 2024-09-12T02:03:13.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.162+0000 7f4e427fc640 10 client.15666 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:03:13.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.162+0000 7f4e427fc640 1 -- 172.21.15.16:0/2627615036 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d26141bd70 con 0x55d261538af0 2024-09-12T02:03:13.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.162+0000 7f4e427fc640 10 client.15666 handle_command_reply: tid=0 2024-09-12T02:03:13.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.162+0000 7f4e63a94740 1 client.15666 shutdown 2024-09-12T02:03:13.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.162+0000 7f4e63a94740 1 -- 172.21.15.16:0/2627615036 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2614afb40 msgr2=0x55d2614c1c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:13.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.162+0000 7f4e63a94740 1 --2- 172.21.15.16:0/2627615036 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2614afb40 0x55d2614c1c30 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f4e440bb9f0 tx=0x7f4e4403b6b0 comp rx=0 tx=0).stop 2024-09-12T02:03:13.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.162+0000 7f4e63a94740 1 -- 172.21.15.16:0/2627615036 shutdown_connections 2024-09-12T02:03:13.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.162+0000 7f4e63a94740 1 --2- 172.21.15.16:0/2627615036 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55d261538af0 0x55d26153af10 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:13.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.162+0000 7f4e63a94740 1 --2- 172.21.15.16:0/2627615036 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d2614bd780 0x55d2614c2170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:13.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.162+0000 7f4e63a94740 1 --2- 172.21.15.16:0/2627615036 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2614afb40 0x55d2614c1c30 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:13.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.162+0000 7f4e63a94740 1 --2- 172.21.15.16:0/2627615036 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d2614ae5d0 0x55d2614c16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:13.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.162+0000 7f4e63a94740 1 -- 172.21.15.16:0/2627615036 >> 172.21.15.16:0/2627615036 conn(0x55d261477310 msgr2=0x55d2614ba330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:13.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.162+0000 7f4e427fc640 0 client.15666 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:03:13.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.162+0000 7f4e63a94740 1 -- 172.21.15.16:0/2627615036 shutdown_connections 2024-09-12T02:03:13.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.162+0000 7f4e63a94740 1 -- 172.21.15.16:0/2627615036 wait complete. 2024-09-12T02:03:13.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.162+0000 7f4e63a94740 20 client.15666 trim_cache size 0 max 16384 2024-09-12T02:03:13.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.163+0000 7f4e616bb640 1 -- 172.21.15.16:0/1401911983 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f4e480a2c00 msgr2=0x7f4e480a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:13.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.163+0000 7f4e616bb640 1 --2- 172.21.15.16:0/1401911983 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f4e480a2c00 0x7f4e480a50c0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c005fd0 tx=0x7f4e4c005850 comp rx=0 tx=0).stop 2024-09-12T02:03:13.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.163+0000 7f4e616bb640 1 -- 172.21.15.16:0/1401911983 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e5c1036e0 msgr2=0x7f4e5c198a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:13.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.163+0000 7f4e616bb640 1 --2- 172.21.15.16:0/1401911983 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e5c1036e0 0x7f4e5c198a40 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f4e44005940 tx=0x7f4e440042f0 comp rx=0 tx=0).stop 2024-09-12T02:03:13.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.163+0000 7f4e616bb640 1 -- 172.21.15.16:0/1401911983 shutdown_connections 2024-09-12T02:03:13.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.163+0000 7f4e616bb640 1 --2- 172.21.15.16:0/1401911983 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f4e480a2c00 0x7f4e480a50c0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:13.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.163+0000 7f4e616bb640 1 --2- 172.21.15.16:0/1401911983 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e5c1082a0 0x7f4e5c19cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:13.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.163+0000 7f4e616bb640 1 --2- 172.21.15.16:0/1401911983 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e5c1036e0 0x7f4e5c198a40 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:13.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.163+0000 7f4e616bb640 1 --2- 172.21.15.16:0/1401911983 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4e5c102d30 0x7f4e5c198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:13.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.163+0000 7f4e616bb640 1 -- 172.21.15.16:0/1401911983 >> 172.21.15.16:0/1401911983 conn(0x7f4e5c0fc7d0 msgr2=0x7f4e5c105d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:13.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.163+0000 7f4e616bb640 1 -- 172.21.15.16:0/1401911983 shutdown_connections 2024-09-12T02:03:13.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:13.163+0000 7f4e616bb640 1 -- 172.21.15.16:0/1401911983 wait complete. 2024-09-12T02:03:13.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:03:13 smithi184 ceph-mon[35593]: pgmap v374: 97 pgs: 97 active+clean; 772 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 KiB/s wr, 12 op/s 2024-09-12T02:03:13.918 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:03:13 smithi033 ceph-mon[35332]: pgmap v374: 97 pgs: 97 active+clean; 772 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 KiB/s wr, 12 op/s 2024-09-12T02:03:13.928 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:03:13 smithi016 ceph-mon[30947]: pgmap v374: 97 pgs: 97 active+clean; 772 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 KiB/s wr, 12 op/s 2024-09-12T02:03:14.959 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-12T02:03:14.960 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-12T02:03:15.233 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:03:15.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:03:15 smithi016 ceph-mon[30947]: pgmap v375: 97 pgs: 97 active+clean; 833 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 KiB/s wr, 14 op/s 2024-09-12T02:03:15.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.684+0000 7f466c4b0640 1 -- 172.21.15.16:0/350887908 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4664102d40 msgr2=0x7f4664103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:15.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.684+0000 7f466c4b0640 1 --2- 172.21.15.16:0/350887908 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4664102d40 0x7f4664103120 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f4660009900 tx=0x7f466002a480 comp rx=0 tx=0).stop 2024-09-12T02:03:15.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.685+0000 7f466c4b0640 1 -- 172.21.15.16:0/350887908 shutdown_connections 2024-09-12T02:03:15.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.685+0000 7f466c4b0640 1 --2- 172.21.15.16:0/350887908 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f466410c170 0x7f466410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:15.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.685+0000 7f466c4b0640 1 --2- 172.21.15.16:0/350887908 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4664103660 0x7f466410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:15.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.685+0000 7f466c4b0640 1 --2- 172.21.15.16:0/350887908 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4664102d40 0x7f4664103120 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:15.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.685+0000 7f466c4b0640 1 -- 172.21.15.16:0/350887908 >> 172.21.15.16:0/350887908 conn(0x7f46640fc7a0 msgr2=0x7f46640febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:15.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.685+0000 7f466c4b0640 1 -- 172.21.15.16:0/350887908 shutdown_connections 2024-09-12T02:03:15.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.685+0000 7f466c4b0640 1 -- 172.21.15.16:0/350887908 wait complete. 2024-09-12T02:03:15.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.685+0000 7f466c4b0640 1 Processor -- start 2024-09-12T02:03:15.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.685+0000 7f466c4b0640 1 -- start start 2024-09-12T02:03:15.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.686+0000 7f466c4b0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4664102d40 0x7f46641009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:15.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.686+0000 7f466c4b0640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4664103660 0x7f4664100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:15.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.686+0000 7f466c4b0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f466410c170 0x7f4664101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:15.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.686+0000 7f466c4b0640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4664111f70 con 0x7f466410c170 2024-09-12T02:03:15.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.686+0000 7f466c4b0640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4664111df0 con 0x7f4664102d40 2024-09-12T02:03:15.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.686+0000 7f466c4b0640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f46641120f0 con 0x7f4664103660 2024-09-12T02:03:15.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.686+0000 7f466aa4d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f466410c170 0x7f4664101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:15.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.686+0000 7f466aa4d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f466410c170 0x7f4664101420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49556/0 (socket says 172.21.15.16:49556) 2024-09-12T02:03:15.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.686+0000 7f466aa4d640 1 -- 172.21.15.16:0/526239246 learned_addr learned my addr 172.21.15.16:0/526239246 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:03:15.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.686+0000 7f466a24c640 1 --2- 172.21.15.16:0/526239246 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4664102d40 0x7f46641009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:15.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.686+0000 7f4669a4b640 1 --2- 172.21.15.16:0/526239246 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4664103660 0x7f4664100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:15.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.686+0000 7f466aa4d640 1 -- 172.21.15.16:0/526239246 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4664103660 msgr2=0x7f4664100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:15.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.686+0000 7f466aa4d640 1 --2- 172.21.15.16:0/526239246 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4664103660 0x7f4664100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:15.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.686+0000 7f466aa4d640 1 -- 172.21.15.16:0/526239246 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4664102d40 msgr2=0x7f46641009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:15.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.686+0000 7f466aa4d640 1 --2- 172.21.15.16:0/526239246 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4664102d40 0x7f46641009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:15.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.686+0000 7f466aa4d640 1 -- 172.21.15.16:0/526239246 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46641a6910 con 0x7f466410c170 2024-09-12T02:03:15.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.686+0000 7f466aa4d640 1 --2- 172.21.15.16:0/526239246 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f466410c170 0x7f4664101420 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f465c00eaa0 tx=0x7f465c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:15.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.686+0000 7f465b7fe640 1 -- 172.21.15.16:0/526239246 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f465c00cdb0 con 0x7f466410c170 2024-09-12T02:03:15.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.686+0000 7f466c4b0640 1 -- 172.21.15.16:0/526239246 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46641a6c00 con 0x7f466410c170 2024-09-12T02:03:15.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.686+0000 7f465b7fe640 1 -- 172.21.15.16:0/526239246 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f465c005020 con 0x7f466410c170 2024-09-12T02:03:15.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.686+0000 7f465b7fe640 1 -- 172.21.15.16:0/526239246 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f465c0116d0 con 0x7f466410c170 2024-09-12T02:03:15.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.686+0000 7f466c4b0640 1 -- 172.21.15.16:0/526239246 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46641a7140 con 0x7f466410c170 2024-09-12T02:03:15.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.689+0000 7f465b7fe640 1 -- 172.21.15.16:0/526239246 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f465c0054d0 con 0x7f466410c170 2024-09-12T02:03:15.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.689+0000 7f466e889740 1 Processor -- start 2024-09-12T02:03:15.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.689+0000 7f466e889740 1 -- start start 2024-09-12T02:03:15.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.689+0000 7f466e889740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558cd7256b60 0x558cd7256f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:15.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.689+0000 7f466e889740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558cd7257480 0x558cd7261800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:15.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.690+0000 7f466e889740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558cd7261d40 0x558cd7264130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:15.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.690+0000 7f466e889740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x558cd71bc280 con 0x558cd7256b60 2024-09-12T02:03:15.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.690+0000 7f466e889740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x558cd7163b10 con 0x558cd7261d40 2024-09-12T02:03:15.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.690+0000 7f466e889740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x558cd71b8240 con 0x558cd7257480 2024-09-12T02:03:15.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.690+0000 7f465b7fe640 1 --2- 172.21.15.16:0/526239246 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f46500a2b80 0x7f46500a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:15.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.690+0000 7f4669a4b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558cd7256b60 0x558cd7256f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:15.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.690+0000 7f4669a4b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558cd7256b60 0x558cd7256f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49564/0 (socket says 172.21.15.16:49564) 2024-09-12T02:03:15.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.690+0000 7f4669a4b640 1 -- 172.21.15.16:0/312920170 learned_addr learned my addr 172.21.15.16:0/312920170 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:03:15.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.690+0000 7f466a24c640 1 --2- 172.21.15.16:0/312920170 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558cd7257480 0x558cd7261800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:15.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.690+0000 7f466aa4d640 1 --2- 172.21.15.16:0/526239246 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f46500a2b80 0x7f46500a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:15.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.690+0000 7f465b7fe640 1 -- 172.21.15.16:0/526239246 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f465c0cf230 con 0x7f466410c170 2024-09-12T02:03:15.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.690+0000 7f4669a4b640 1 --2- 172.21.15.16:0/312920170 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558cd7261d40 0x558cd7264130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:15.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.690+0000 7f4669a4b640 1 -- 172.21.15.16:0/312920170 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558cd7257480 msgr2=0x558cd7261800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:15.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.690+0000 7f4669a4b640 1 --2- 172.21.15.16:0/312920170 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558cd7257480 0x558cd7261800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:15.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.690+0000 7f4669a4b640 1 -- 172.21.15.16:0/312920170 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558cd7261d40 msgr2=0x558cd7264130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:15.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.690+0000 7f4669a4b640 1 --2- 172.21.15.16:0/312920170 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558cd7261d40 0x558cd7264130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:15.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.690+0000 7f4669a4b640 1 -- 172.21.15.16:0/312920170 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558cd71701f0 con 0x558cd7256b60 2024-09-12T02:03:15.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.690+0000 7f466a24c640 1 --2- 172.21.15.16:0/312920170 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558cd7257480 0x558cd7261800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:03:15.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.690+0000 7f4669a4b640 1 --2- 172.21.15.16:0/312920170 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558cd7256b60 0x558cd7256f40 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f4654005d90 tx=0x7f465400c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8474a6a5fb36036 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:15.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.690+0000 7f466aa4d640 1 --2- 172.21.15.16:0/526239246 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f46500a2b80 0x7f46500a5040 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f465c005480 tx=0x7f465c091010 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:15.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.690+0000 7f4658ff9640 1 -- 172.21.15.16:0/312920170 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4654012110 con 0x558cd7256b60 2024-09-12T02:03:15.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.691+0000 7f4658ff9640 1 -- 172.21.15.16:0/312920170 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4654005500 con 0x558cd7256b60 2024-09-12T02:03:15.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.691+0000 7f4658ff9640 1 -- 172.21.15.16:0/312920170 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f465400e830 con 0x558cd7256b60 2024-09-12T02:03:15.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.691+0000 7f466e889740 1 -- 172.21.15.16:0/312920170 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558cd7256b60 msgr2=0x558cd7256f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:15.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.691+0000 7f466e889740 1 --2- 172.21.15.16:0/312920170 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558cd7256b60 0x558cd7256f40 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f4654005d90 tx=0x7f465400c840 comp rx=0 tx=0).stop 2024-09-12T02:03:15.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.691+0000 7f466e889740 1 -- 172.21.15.16:0/312920170 shutdown_connections 2024-09-12T02:03:15.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.691+0000 7f466e889740 1 --2- 172.21.15.16:0/312920170 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558cd7261d40 0x558cd7264130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:15.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.691+0000 7f466e889740 1 --2- 172.21.15.16:0/312920170 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558cd7257480 0x558cd7261800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:15.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.691+0000 7f466e889740 1 --2- 172.21.15.16:0/312920170 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558cd7256b60 0x558cd7256f40 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:15.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.691+0000 7f466e889740 1 -- 172.21.15.16:0/312920170 >> 172.21.15.16:0/312920170 conn(0x558cd72530f0 msgr2=0x558cd72534d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:15.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.691+0000 7f466e889740 1 -- 172.21.15.16:0/312920170 shutdown_connections 2024-09-12T02:03:15.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.691+0000 7f466e889740 1 -- 172.21.15.16:0/312920170 wait complete. 2024-09-12T02:03:15.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.692+0000 7f466e889740 1 -- start start 2024-09-12T02:03:15.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.692+0000 7f466e889740 1 Processor -- start 2024-09-12T02:03:15.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.692+0000 7f466e889740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558cd7256b60 0x558cd725ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:15.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.692+0000 7f466e889740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558cd7257480 0x558cd725f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:15.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.692+0000 7f466e889740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558cd7261d40 0x558cd725f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:15.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.692+0000 7f466e889740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x558cd7264910 con 0x558cd7256b60 2024-09-12T02:03:15.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.692+0000 7f466e889740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x558cd7176f10 con 0x558cd7257480 2024-09-12T02:03:15.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.692+0000 7f466e889740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x558cd7264a90 con 0x558cd7261d40 2024-09-12T02:03:15.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.692+0000 7f466e889740 10 client.0 _ll_register_callbacks cb 0x558cd713fa40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:03:15.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.692+0000 7f4669a4b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558cd7256b60 0x558cd725ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:15.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.692+0000 7f4669a4b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558cd7256b60 0x558cd725ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49574/0 (socket says 172.21.15.16:49574) 2024-09-12T02:03:15.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.692+0000 7f4669a4b640 1 -- 172.21.15.16:0/206476181 learned_addr learned my addr 172.21.15.16:0/206476181 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:03:15.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.692+0000 7f466a24c640 1 --2- 172.21.15.16:0/206476181 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558cd7257480 0x558cd725f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:15.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.692+0000 7f4669a4b640 1 -- 172.21.15.16:0/206476181 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558cd7261d40 msgr2=0x558cd725f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:15.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.692+0000 7f4669a4b640 1 --2- 172.21.15.16:0/206476181 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558cd7261d40 0x558cd725f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:15.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.692+0000 7f4669a4b640 1 -- 172.21.15.16:0/206476181 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558cd7257480 msgr2=0x558cd725f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:15.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.692+0000 7f4669a4b640 1 --2- 172.21.15.16:0/206476181 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558cd7257480 0x558cd725f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:15.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.692+0000 7f4669a4b640 1 -- 172.21.15.16:0/206476181 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558cd6f2d6a0 con 0x558cd7256b60 2024-09-12T02:03:15.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.692+0000 7f466a24c640 1 --2- 172.21.15.16:0/206476181 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558cd7257480 0x558cd725f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:03:15.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.693+0000 7f4669a4b640 1 --2- 172.21.15.16:0/206476181 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558cd7256b60 0x558cd725ec20 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f465400c350 tx=0x7f4654004f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b0d8b4e203588b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:15.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.693+0000 7f464dffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:03:15.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.693+0000 7f464dffb640 1 -- 172.21.15.16:0/206476181 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4654004020 con 0x558cd7256b60 2024-09-12T02:03:15.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.693+0000 7f464dffb640 1 -- 172.21.15.16:0/206476181 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f46540041c0 con 0x558cd7256b60 2024-09-12T02:03:15.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.693+0000 7f466e889740 1 -- 172.21.15.16:0/206476181 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4654005500 con 0x558cd7256b60 2024-09-12T02:03:15.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.693+0000 7f464dffb640 1 -- 172.21.15.16:0/206476181 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f46540043c0 con 0x558cd7256b60 2024-09-12T02:03:15.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.693+0000 7f466e889740 10 client.15678 fetch_fsmap learned FSMap version 25 2024-09-12T02:03:15.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.693+0000 7f466e889740 1 -- 172.21.15.16:0/206476181 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x558cd7270510 con 0x558cd7256b60 2024-09-12T02:03:15.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.693+0000 7f464dffb640 1 -- 172.21.15.16:0/206476181 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4654005500 con 0x558cd7256b60 2024-09-12T02:03:15.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.693+0000 7f464dffb640 1 -- 172.21.15.16:0/206476181 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f465400e5d0 con 0x558cd7256b60 2024-09-12T02:03:15.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.693+0000 7f466e889740 10 client.15678 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:03:15.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.694+0000 7f466e889740 10 client.15678 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:03:15.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.694+0000 7f466e889740 20 client.15678 populate_metadata read hostname 'smithi016' 2024-09-12T02:03:15.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.694+0000 7f466e889740 1 --2- 172.21.15.16:0/206476181 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x558cd72718c0 0x558cd7273cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:15.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.694+0000 7f466e889740 4 client.15678 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:03:15.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.694+0000 7f466e889740 1 -- 172.21.15.16:0/206476181 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558cd71c97f0 con 0x558cd72718c0 2024-09-12T02:03:15.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.694+0000 7f466a24c640 1 --2- 172.21.15.16:0/206476181 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x558cd72718c0 0x558cd7273cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:15.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.694+0000 7f466a24c640 1 --2- 172.21.15.16:0/206476181 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x558cd72718c0 0x558cd7273cb0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=990888ee1a8bc750 server_cookie=87c03e239d60778d in_seq=0 out_seq=0 2024-09-12T02:03:15.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.694+0000 7f464dffb640 10 client.15678 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:03:15.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.695+0000 7f464dffb640 1 -- 172.21.15.16:0/206476181 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558cd71c97f0 con 0x558cd72718c0 2024-09-12T02:03:15.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.695+0000 7f464dffb640 10 client.15678 handle_command_reply: tid=0 2024-09-12T02:03:15.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.695+0000 7f466e889740 1 client.15678 shutdown 2024-09-12T02:03:15.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.695+0000 7f466e889740 1 -- 172.21.15.16:0/206476181 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558cd7256b60 msgr2=0x558cd725ec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:15.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.695+0000 7f466e889740 1 --2- 172.21.15.16:0/206476181 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558cd7256b60 0x558cd725ec20 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f465400c350 tx=0x7f4654004f50 comp rx=0 tx=0).stop 2024-09-12T02:03:15.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.695+0000 7f466e889740 1 -- 172.21.15.16:0/206476181 shutdown_connections 2024-09-12T02:03:15.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.695+0000 7f466e889740 1 --2- 172.21.15.16:0/206476181 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x558cd72718c0 0x558cd7273cb0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:15.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.695+0000 7f466e889740 1 --2- 172.21.15.16:0/206476181 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558cd7261d40 0x558cd725f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:15.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.695+0000 7f466e889740 1 --2- 172.21.15.16:0/206476181 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558cd7257480 0x558cd725f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:15.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.695+0000 7f466e889740 1 --2- 172.21.15.16:0/206476181 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558cd7256b60 0x558cd725ec20 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:15.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.695+0000 7f464dffb640 0 client.15678 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:03:15.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.695+0000 7f466e889740 1 -- 172.21.15.16:0/206476181 >> 172.21.15.16:0/206476181 conn(0x558cd72530f0 msgr2=0x558cd7258e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:15.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.695+0000 7f466e889740 1 -- 172.21.15.16:0/206476181 shutdown_connections 2024-09-12T02:03:15.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.695+0000 7f466e889740 1 -- 172.21.15.16:0/206476181 wait complete. 2024-09-12T02:03:15.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.695+0000 7f466e889740 20 client.15678 trim_cache size 0 max 16384 2024-09-12T02:03:15.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.713+0000 7f466e889740 1 Processor -- start 2024-09-12T02:03:15.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.713+0000 7f466e889740 1 -- start start 2024-09-12T02:03:15.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.713+0000 7f466e889740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558cd7256b60 0x558cd725f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:15.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.713+0000 7f466e889740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558cd7257480 0x558cd725f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:15.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.713+0000 7f466e889740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558cd7261d40 0x558cd7279340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:15.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.713+0000 7f466e889740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f46641b1ff0 con 0x558cd7257480 2024-09-12T02:03:15.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.713+0000 7f466e889740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f46641af8c0 con 0x558cd7256b60 2024-09-12T02:03:15.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.713+0000 7f466e889740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f46641ad750 con 0x558cd7261d40 2024-09-12T02:03:15.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.713+0000 7f466a24c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558cd7257480 0x558cd725f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:15.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.713+0000 7f466a24c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558cd7257480 0x558cd725f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49578/0 (socket says 172.21.15.16:49578) 2024-09-12T02:03:15.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.713+0000 7f466a24c640 1 -- 172.21.15.16:0/1533414518 learned_addr learned my addr 172.21.15.16:0/1533414518 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:03:15.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.714+0000 7f4669a4b640 1 --2- 172.21.15.16:0/1533414518 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558cd7256b60 0x558cd725f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:15.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.714+0000 7f4669a4b640 1 --2- 172.21.15.16:0/1533414518 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558cd7261d40 0x558cd7279340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:15.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.714+0000 7f466a24c640 1 -- 172.21.15.16:0/1533414518 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558cd7261d40 msgr2=0x558cd7279340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:15.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.714+0000 7f466a24c640 1 --2- 172.21.15.16:0/1533414518 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558cd7261d40 0x558cd7279340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:15.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.714+0000 7f466a24c640 1 -- 172.21.15.16:0/1533414518 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558cd7256b60 msgr2=0x558cd725f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:15.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.714+0000 7f466a24c640 1 --2- 172.21.15.16:0/1533414518 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558cd7256b60 0x558cd725f2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:15.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.714+0000 7f466a24c640 1 -- 172.21.15.16:0/1533414518 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558cd72572b0 con 0x558cd7257480 2024-09-12T02:03:15.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.714+0000 7f4669a4b640 1 --2- 172.21.15.16:0/1533414518 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558cd7256b60 0x558cd725f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:03:15.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.714+0000 7f466a24c640 1 --2- 172.21.15.16:0/1533414518 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558cd7257480 0x558cd725f830 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f4660009900 tx=0x7f4660032cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5f4fe0376cfff58 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:15.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.714+0000 7f464ffff640 1 -- 172.21.15.16:0/1533414518 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4660049070 con 0x558cd7257480 2024-09-12T02:03:15.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.714+0000 7f464ffff640 1 -- 172.21.15.16:0/1533414518 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4660005520 con 0x558cd7257480 2024-09-12T02:03:15.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.714+0000 7f464ffff640 1 -- 172.21.15.16:0/1533414518 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4660035040 con 0x558cd7257480 2024-09-12T02:03:15.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.715+0000 7f466e889740 1 -- 172.21.15.16:0/1533414518 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558cd7257480 msgr2=0x558cd725f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:15.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.715+0000 7f466e889740 1 --2- 172.21.15.16:0/1533414518 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558cd7257480 0x558cd725f830 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f4660009900 tx=0x7f4660032cf0 comp rx=0 tx=0).stop 2024-09-12T02:03:15.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.715+0000 7f466e889740 1 -- 172.21.15.16:0/1533414518 shutdown_connections 2024-09-12T02:03:15.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.715+0000 7f466e889740 1 --2- 172.21.15.16:0/1533414518 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558cd7261d40 0x558cd7279340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:15.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.715+0000 7f466e889740 1 --2- 172.21.15.16:0/1533414518 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558cd7257480 0x558cd725f830 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:15.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.715+0000 7f466e889740 1 --2- 172.21.15.16:0/1533414518 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558cd7256b60 0x558cd725f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:15.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.715+0000 7f466e889740 1 -- 172.21.15.16:0/1533414518 >> 172.21.15.16:0/1533414518 conn(0x558cd72530f0 msgr2=0x558cd725ebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:15.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.715+0000 7f466e889740 1 -- 172.21.15.16:0/1533414518 shutdown_connections 2024-09-12T02:03:15.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.715+0000 7f466e889740 1 -- 172.21.15.16:0/1533414518 wait complete. 2024-09-12T02:03:15.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.715+0000 7f466e889740 1 -- start start 2024-09-12T02:03:15.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.715+0000 7f466e889740 1 Processor -- start 2024-09-12T02:03:15.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.716+0000 7f466e889740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558cd7256b60 0x558cd7265950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:15.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.716+0000 7f466e889740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558cd7257480 0x558cd7265e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:15.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.716+0000 7f466e889740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558cd7261d40 0x558cd72663d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:15.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.716+0000 7f466e889740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f46641adda0 con 0x558cd7256b60 2024-09-12T02:03:15.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.716+0000 7f466e889740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f46641af1f0 con 0x558cd7261d40 2024-09-12T02:03:15.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.716+0000 7f466e889740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f46641ae450 con 0x558cd7257480 2024-09-12T02:03:15.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.716+0000 7f466e889740 10 client.0 _ll_register_callbacks cb 0x7f4660009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:03:15.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.716+0000 7f4669a4b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558cd7256b60 0x558cd7265950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:15.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.716+0000 7f4669a4b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558cd7256b60 0x558cd7265950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49592/0 (socket says 172.21.15.16:49592) 2024-09-12T02:03:15.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.716+0000 7f4669a4b640 1 -- 172.21.15.16:0/3002825631 learned_addr learned my addr 172.21.15.16:0/3002825631 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:03:15.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.716+0000 7f466a24c640 1 --2- 172.21.15.16:0/3002825631 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558cd7257480 0x558cd7265e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:15.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.716+0000 7f4669a4b640 1 --2- 172.21.15.16:0/3002825631 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558cd7261d40 0x558cd72663d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:15.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.716+0000 7f4669a4b640 1 -- 172.21.15.16:0/3002825631 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558cd7257480 msgr2=0x558cd7265e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:15.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.716+0000 7f4669a4b640 1 --2- 172.21.15.16:0/3002825631 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558cd7257480 0x558cd7265e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:15.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.716+0000 7f4669a4b640 1 -- 172.21.15.16:0/3002825631 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558cd7261d40 msgr2=0x558cd72663d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:15.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.716+0000 7f4669a4b640 1 --2- 172.21.15.16:0/3002825631 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558cd7261d40 0x558cd72663d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:15.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.716+0000 7f4669a4b640 1 -- 172.21.15.16:0/3002825631 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558cd72db310 con 0x558cd7256b60 2024-09-12T02:03:15.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.716+0000 7f4669a4b640 1 --2- 172.21.15.16:0/3002825631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558cd7256b60 0x558cd7265950 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f4654005d90 tx=0x7f4654005ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b793a76f9f5f3e39 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:15.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.716+0000 7f464dffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:03:15.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.717+0000 7f464dffb640 1 -- 172.21.15.16:0/3002825631 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4654035070 con 0x558cd7256b60 2024-09-12T02:03:15.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.717+0000 7f464dffb640 1 -- 172.21.15.16:0/3002825631 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f465400cce0 con 0x558cd7256b60 2024-09-12T02:03:15.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.717+0000 7f466e889740 1 -- 172.21.15.16:0/3002825631 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4660005520 con 0x558cd7256b60 2024-09-12T02:03:15.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.717+0000 7f464dffb640 1 -- 172.21.15.16:0/3002825631 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f465400e5d0 con 0x558cd7256b60 2024-09-12T02:03:15.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.717+0000 7f466e889740 10 client.15684 fetch_fsmap learned FSMap version 25 2024-09-12T02:03:15.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.717+0000 7f466e889740 1 -- 172.21.15.16:0/3002825631 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x558cd72dc4c0 con 0x558cd7256b60 2024-09-12T02:03:15.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.717+0000 7f464dffb640 1 -- 172.21.15.16:0/3002825631 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4660005520 con 0x558cd7256b60 2024-09-12T02:03:15.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.717+0000 7f464dffb640 1 -- 172.21.15.16:0/3002825631 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f4654025640 con 0x558cd7256b60 2024-09-12T02:03:15.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.717+0000 7f466e889740 10 client.15684 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:03:15.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.717+0000 7f466e889740 10 client.15684 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:03:15.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.717+0000 7f466e889740 20 client.15684 populate_metadata read hostname 'smithi016' 2024-09-12T02:03:15.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.717+0000 7f466e889740 1 --2- 172.21.15.16:0/3002825631 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x558cd72dc8a0 0x558cd72decc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:15.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.717+0000 7f466e889740 4 client.15684 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-12T02:03:15.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.717+0000 7f466e889740 1 -- 172.21.15.16:0/3002825631 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x558cd71c3d00 con 0x558cd72dc8a0 2024-09-12T02:03:15.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.718+0000 7f466a24c640 1 --2- 172.21.15.16:0/3002825631 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x558cd72dc8a0 0x558cd72decc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:15.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.718+0000 7f466a24c640 1 --2- 172.21.15.16:0/3002825631 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x558cd72dc8a0 0x558cd72decc0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f070892b16c2562 server_cookie=702addd2dbe8d34b in_seq=0 out_seq=0 2024-09-12T02:03:15.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.718+0000 7f464dffb640 10 client.15684 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:03:15.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.719+0000 7f464dffb640 1 -- 172.21.15.16:0/3002825631 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x558cd71c3d00 con 0x558cd72dc8a0 2024-09-12T02:03:15.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.719+0000 7f464dffb640 10 client.15684 handle_command_reply: tid=0 2024-09-12T02:03:15.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.719+0000 7f466e889740 1 client.15684 shutdown 2024-09-12T02:03:15.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.719+0000 7f466e889740 1 -- 172.21.15.16:0/3002825631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558cd7256b60 msgr2=0x558cd7265950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:15.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.719+0000 7f466e889740 1 --2- 172.21.15.16:0/3002825631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558cd7256b60 0x558cd7265950 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f4654005d90 tx=0x7f4654005ce0 comp rx=0 tx=0).stop 2024-09-12T02:03:15.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.719+0000 7f466e889740 1 -- 172.21.15.16:0/3002825631 shutdown_connections 2024-09-12T02:03:15.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.719+0000 7f466e889740 1 --2- 172.21.15.16:0/3002825631 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x558cd72dc8a0 0x558cd72decc0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:15.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.719+0000 7f466e889740 1 --2- 172.21.15.16:0/3002825631 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558cd7261d40 0x558cd72663d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:15.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.719+0000 7f466e889740 1 --2- 172.21.15.16:0/3002825631 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558cd7257480 0x558cd7265e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:15.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.719+0000 7f464dffb640 0 client.15684 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:03:15.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.719+0000 7f466e889740 1 --2- 172.21.15.16:0/3002825631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558cd7256b60 0x558cd7265950 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:15.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.719+0000 7f466e889740 1 -- 172.21.15.16:0/3002825631 >> 172.21.15.16:0/3002825631 conn(0x558cd72530f0 msgr2=0x558cd725f3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:15.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.719+0000 7f466e889740 1 -- 172.21.15.16:0/3002825631 shutdown_connections 2024-09-12T02:03:15.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.719+0000 7f466e889740 1 -- 172.21.15.16:0/3002825631 wait complete. 2024-09-12T02:03:15.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.719+0000 7f466e889740 20 client.15684 trim_cache size 0 max 16384 2024-09-12T02:03:15.729 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T02:03:15.729 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-09-12T02:03:15.729 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "b8d1936a-d601-47a1-9684-0d51663f1f7c", 2024-09-12T02:03:15.729 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-09-12T02:03:15.729 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T02:03:15.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.720+0000 7f466c4b0640 1 -- 172.21.15.16:0/526239246 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f46500a2b80 msgr2=0x7f46500a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:15.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.720+0000 7f466c4b0640 1 --2- 172.21.15.16:0/526239246 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f46500a2b80 0x7f46500a5040 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f465c005480 tx=0x7f465c091010 comp rx=0 tx=0).stop 2024-09-12T02:03:15.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.720+0000 7f466c4b0640 1 -- 172.21.15.16:0/526239246 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f466410c170 msgr2=0x7f4664101420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:15.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.720+0000 7f466c4b0640 1 --2- 172.21.15.16:0/526239246 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f466410c170 0x7f4664101420 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f465c00eaa0 tx=0x7f465c00ef60 comp rx=0 tx=0).stop 2024-09-12T02:03:15.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.720+0000 7f466c4b0640 1 -- 172.21.15.16:0/526239246 shutdown_connections 2024-09-12T02:03:15.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.720+0000 7f466c4b0640 1 --2- 172.21.15.16:0/526239246 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f46500a2b80 0x7f46500a5040 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:15.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.720+0000 7f466c4b0640 1 --2- 172.21.15.16:0/526239246 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f466410c170 0x7f4664101420 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:15.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.720+0000 7f466c4b0640 1 --2- 172.21.15.16:0/526239246 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4664103660 0x7f4664100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:15.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.720+0000 7f466c4b0640 1 --2- 172.21.15.16:0/526239246 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4664102d40 0x7f46641009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:15.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.720+0000 7f466c4b0640 1 -- 172.21.15.16:0/526239246 >> 172.21.15.16:0/526239246 conn(0x7f46640fc7a0 msgr2=0x7f46640fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:15.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.720+0000 7f466c4b0640 1 -- 172.21.15.16:0/526239246 shutdown_connections 2024-09-12T02:03:15.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:15.720+0000 7f466c4b0640 1 -- 172.21.15.16:0/526239246 wait complete. 2024-09-12T02:03:15.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:03:15 smithi184 ceph-mon[35593]: pgmap v375: 97 pgs: 97 active+clean; 833 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 KiB/s wr, 14 op/s 2024-09-12T02:03:15.918 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:03:15 smithi033 ceph-mon[35332]: pgmap v375: 97 pgs: 97 active+clean; 833 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 KiB/s wr, 14 op/s 2024-09-12T02:03:16.524 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.524+0000 7fa2abf49640 1 -- 172.21.15.16:0/1338394292 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2a41025f0 msgr2=0x7fa2a4108e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:16.524 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.524+0000 7fa2abf49640 1 --2- 172.21.15.16:0/1338394292 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2a41025f0 0x7fa2a4108e80 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fa29c009ca0 tx=0x7fa29c02a8c0 comp rx=0 tx=0).stop 2024-09-12T02:03:16.524 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.524+0000 7fa2abf49640 1 -- 172.21.15.16:0/1338394292 shutdown_connections 2024-09-12T02:03:16.524 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.524+0000 7fa2abf49640 1 --2- 172.21.15.16:0/1338394292 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2a41025f0 0x7fa2a4108e80 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:16.524 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.524+0000 7fa2abf49640 1 --2- 172.21.15.16:0/1338394292 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2a4101c30 0x7fa2a41020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:16.524 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.524+0000 7fa2abf49640 1 --2- 172.21.15.16:0/1338394292 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2a4100a40 0x7fa2a4100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:16.524 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.524+0000 7fa2abf49640 1 -- 172.21.15.16:0/1338394292 >> 172.21.15.16:0/1338394292 conn(0x7fa2a40fc1e0 msgr2=0x7fa2a40fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:16.525 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.524+0000 7fa2abf49640 1 -- 172.21.15.16:0/1338394292 shutdown_connections 2024-09-12T02:03:16.525 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.524+0000 7fa2abf49640 1 -- 172.21.15.16:0/1338394292 wait complete. 2024-09-12T02:03:16.526 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.524+0000 7fa2abf49640 1 Processor -- start 2024-09-12T02:03:16.526 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.525+0000 7fa2abf49640 1 -- start start 2024-09-12T02:03:16.527 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.525+0000 7fa2abf49640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2a4100a40 0x7fa2a4197d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:16.527 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.525+0000 7fa2abf49640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2a4101c30 0x7fa2a41982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:16.527 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.525+0000 7fa2abf49640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2a41025f0 0x7fa2a419f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:16.527 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.525+0000 7fa2abf49640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a410c6a0 con 0x7fa2a4101c30 2024-09-12T02:03:16.527 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.525+0000 7fa2abf49640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a410c520 con 0x7fa2a41025f0 2024-09-12T02:03:16.527 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.525+0000 7fa2abf49640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a410c820 con 0x7fa2a4100a40 2024-09-12T02:03:16.527 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.525+0000 7fa2a94e4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2a4101c30 0x7fa2a41982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:16.527 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.525+0000 7fa2a94e4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2a4101c30 0x7fa2a41982a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49604/0 (socket says 172.21.15.16:49604) 2024-09-12T02:03:16.527 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.525+0000 7fa2a94e4640 1 -- 172.21.15.16:0/3644325825 learned_addr learned my addr 172.21.15.16:0/3644325825 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:03:16.527 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.525+0000 7fa2a9ce5640 1 --2- 172.21.15.16:0/3644325825 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2a4100a40 0x7fa2a4197d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:16.527 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.525+0000 7fa2aa4e6640 1 --2- 172.21.15.16:0/3644325825 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2a41025f0 0x7fa2a419f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:16.527 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.525+0000 7fa2a94e4640 1 -- 172.21.15.16:0/3644325825 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2a4100a40 msgr2=0x7fa2a4197d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:16.527 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.525+0000 7fa2a94e4640 1 --2- 172.21.15.16:0/3644325825 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2a4100a40 0x7fa2a4197d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:16.528 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.525+0000 7fa2a94e4640 1 -- 172.21.15.16:0/3644325825 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2a41025f0 msgr2=0x7fa2a419f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:16.528 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.525+0000 7fa2a94e4640 1 --2- 172.21.15.16:0/3644325825 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2a41025f0 0x7fa2a419f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:16.528 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.525+0000 7fa2a94e4640 1 -- 172.21.15.16:0/3644325825 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2a419f980 con 0x7fa2a4101c30 2024-09-12T02:03:16.528 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.525+0000 7fa2a94e4640 1 --2- 172.21.15.16:0/3644325825 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2a4101c30 0x7fa2a41982a0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fa29400d950 tx=0x7fa29400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:16.528 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.526+0000 7fa29affd640 1 -- 172.21.15.16:0/3644325825 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa294017070 con 0x7fa2a4101c30 2024-09-12T02:03:16.528 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.526+0000 7fa29affd640 1 -- 172.21.15.16:0/3644325825 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa294009c30 con 0x7fa2a4101c30 2024-09-12T02:03:16.528 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.526+0000 7fa2abf49640 1 -- 172.21.15.16:0/3644325825 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2a419fc70 con 0x7fa2a4101c30 2024-09-12T02:03:16.528 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.526+0000 7fa29affd640 1 -- 172.21.15.16:0/3644325825 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa29400f6b0 con 0x7fa2a4101c30 2024-09-12T02:03:16.528 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.526+0000 7fa2abf49640 1 -- 172.21.15.16:0/3644325825 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2a41a01c0 con 0x7fa2a4101c30 2024-09-12T02:03:16.530 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.528+0000 7fa29affd640 1 -- 172.21.15.16:0/3644325825 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa2940109d0 con 0x7fa2a4101c30 2024-09-12T02:03:16.530 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.528+0000 7fa2aedbb740 1 Processor -- start 2024-09-12T02:03:16.530 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.529+0000 7fa2aedbb740 1 -- start start 2024-09-12T02:03:16.530 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.529+0000 7fa2aedbb740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d4891f89a0 0x55d4891f8da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:16.531 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.529+0000 7fa2aedbb740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d4891f92e0 0x55d4891fdda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:16.531 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.529+0000 7fa2aedbb740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d4891fe370 0x55d489200830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:16.531 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.529+0000 7fa2aedbb740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55d48915c2f0 con 0x55d4891fe370 2024-09-12T02:03:16.531 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.529+0000 7fa2aedbb740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55d489104270 con 0x55d4891f92e0 2024-09-12T02:03:16.531 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.529+0000 7fa2aedbb740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55d4891582b0 con 0x55d4891f89a0 2024-09-12T02:03:16.531 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.529+0000 7fa2a94e4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d4891fe370 0x55d489200830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:16.531 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.529+0000 7fa2a94e4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d4891fe370 0x55d489200830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49620/0 (socket says 172.21.15.16:49620) 2024-09-12T02:03:16.531 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.529+0000 7fa2a94e4640 1 -- 172.21.15.16:0/341348965 learned_addr learned my addr 172.21.15.16:0/341348965 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:03:16.531 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.529+0000 7fa2a9ce5640 1 --2- 172.21.15.16:0/341348965 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d4891f92e0 0x55d4891fdda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:16.532 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.529+0000 7fa2aa4e6640 1 --2- 172.21.15.16:0/341348965 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d4891f89a0 0x55d4891f8da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:16.532 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.529+0000 7fa29affd640 1 --2- 172.21.15.16:0/3644325825 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fa2900a2c30 0x7fa2900a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:16.532 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.530+0000 7fa2aa4e6640 1 --2- 172.21.15.16:0/3644325825 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fa2900a2c30 0x7fa2900a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:16.532 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.530+0000 7fa2a9ce5640 1 -- 172.21.15.16:0/341348965 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d4891f89a0 msgr2=0x55d4891f8da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:16.532 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.530+0000 7fa2a9ce5640 1 --2- 172.21.15.16:0/341348965 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d4891f89a0 0x55d4891f8da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:16.532 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.530+0000 7fa2a9ce5640 1 -- 172.21.15.16:0/341348965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d4891fe370 msgr2=0x55d489200830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:16.532 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.530+0000 7fa2a9ce5640 1 --2- 172.21.15.16:0/341348965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d4891fe370 0x55d489200830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:16.532 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.530+0000 7fa2a9ce5640 1 -- 172.21.15.16:0/341348965 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d489103340 con 0x55d4891f92e0 2024-09-12T02:03:16.532 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.530+0000 7fa29affd640 1 -- 172.21.15.16:0/3644325825 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fa2940ceb40 con 0x7fa2a4101c30 2024-09-12T02:03:16.532 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.530+0000 7fa2a9ce5640 1 --2- 172.21.15.16:0/341348965 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d4891f92e0 0x55d4891fdda0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa2a0009bf0 tx=0x7fa2a0011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=58d4deff1ba6546d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:16.532 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.530+0000 7fa2aa4e6640 1 --2- 172.21.15.16:0/3644325825 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fa2900a2c30 0x7fa2900a50f0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fa29c006010 tx=0x7fa29c02a8c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:16.532 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.530+0000 7fa28ffff640 1 -- 172.21.15.16:0/341348965 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2a0009da0 con 0x55d4891f92e0 2024-09-12T02:03:16.532 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.530+0000 7fa28ffff640 1 -- 172.21.15.16:0/341348965 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2a0005020 con 0x55d4891f92e0 2024-09-12T02:03:16.533 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.530+0000 7fa28ffff640 1 -- 172.21.15.16:0/341348965 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2a0004b60 con 0x55d4891f92e0 2024-09-12T02:03:16.537 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.531+0000 7fa2aedbb740 1 -- 172.21.15.16:0/341348965 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d4891f92e0 msgr2=0x55d4891fdda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:16.537 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.531+0000 7fa2aedbb740 1 --2- 172.21.15.16:0/341348965 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d4891f92e0 0x55d4891fdda0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa2a0009bf0 tx=0x7fa2a0011810 comp rx=0 tx=0).stop 2024-09-12T02:03:16.537 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.531+0000 7fa2aedbb740 1 -- 172.21.15.16:0/341348965 shutdown_connections 2024-09-12T02:03:16.537 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.531+0000 7fa2aedbb740 1 --2- 172.21.15.16:0/341348965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d4891fe370 0x55d489200830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:16.538 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.531+0000 7fa2aedbb740 1 --2- 172.21.15.16:0/341348965 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d4891f92e0 0x55d4891fdda0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:16.538 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.531+0000 7fa2aedbb740 1 --2- 172.21.15.16:0/341348965 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d4891f89a0 0x55d4891f8da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:16.538 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.531+0000 7fa2aedbb740 1 -- 172.21.15.16:0/341348965 >> 172.21.15.16:0/341348965 conn(0x55d4891f4010 msgr2=0x55d4891f4410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:16.538 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.531+0000 7fa2aedbb740 1 -- 172.21.15.16:0/341348965 shutdown_connections 2024-09-12T02:03:16.538 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.531+0000 7fa2aedbb740 1 -- 172.21.15.16:0/341348965 wait complete. 2024-09-12T02:03:16.538 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.531+0000 7fa2aedbb740 1 -- start start 2024-09-12T02:03:16.538 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.532+0000 7fa2aedbb740 1 Processor -- start 2024-09-12T02:03:16.538 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.532+0000 7fa2aedbb740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d4891f89a0 0x55d489202bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:16.538 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.532+0000 7fa2aedbb740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d4891f92e0 0x55d489203130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:16.538 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.532+0000 7fa2aedbb740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d4891fe370 0x55d48920a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:16.539 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.532+0000 7fa2aedbb740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55d489201340 con 0x55d4891f92e0 2024-09-12T02:03:16.539 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.532+0000 7fa2aedbb740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55d4890e5290 con 0x55d4891f89a0 2024-09-12T02:03:16.539 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.532+0000 7fa2aedbb740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55d4892014c0 con 0x55d4891fe370 2024-09-12T02:03:16.539 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.532+0000 7fa2aedbb740 10 client.0 _ll_register_callbacks cb 0x55d489116140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:03:16.539 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.532+0000 7fa2a94e4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d4891f92e0 0x55d489203130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:16.539 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.532+0000 7fa2a94e4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d4891f92e0 0x55d489203130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49626/0 (socket says 172.21.15.16:49626) 2024-09-12T02:03:16.539 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.532+0000 7fa2a94e4640 1 -- 172.21.15.16:0/217522076 learned_addr learned my addr 172.21.15.16:0/217522076 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:03:16.539 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.532+0000 7fa2aa4e6640 1 --2- 172.21.15.16:0/217522076 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d4891fe370 0x55d48920a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:16.539 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.532+0000 7fa2a9ce5640 1 --2- 172.21.15.16:0/217522076 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d4891f89a0 0x55d489202bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:16.539 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.532+0000 7fa2a94e4640 1 -- 172.21.15.16:0/217522076 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d4891fe370 msgr2=0x55d48920a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:16.539 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.533+0000 7fa2a94e4640 1 --2- 172.21.15.16:0/217522076 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d4891fe370 0x55d48920a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:16.539 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.533+0000 7fa2a94e4640 1 -- 172.21.15.16:0/217522076 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d4891f89a0 msgr2=0x55d489202bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:16.539 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.533+0000 7fa2a94e4640 1 --2- 172.21.15.16:0/217522076 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d4891f89a0 0x55d489202bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:16.540 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.533+0000 7fa2a94e4640 1 -- 172.21.15.16:0/217522076 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d488e6f210 con 0x55d4891f92e0 2024-09-12T02:03:16.540 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.533+0000 7fa2a94e4640 1 --2- 172.21.15.16:0/217522076 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d4891f92e0 0x55d489203130 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fa294010fd0 tx=0x7fa294010d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14ef97e5a7fcaff7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:16.540 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.533+0000 7fa28d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:03:16.540 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.533+0000 7fa28d7fa640 1 -- 172.21.15.16:0/217522076 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa294020040 con 0x55d4891f92e0 2024-09-12T02:03:16.540 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.533+0000 7fa28d7fa640 1 -- 172.21.15.16:0/217522076 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fa29409a940 con 0x55d4891f92e0 2024-09-12T02:03:16.540 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.533+0000 7fa2aedbb740 1 -- 172.21.15.16:0/217522076 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2a0005020 con 0x55d4891f92e0 2024-09-12T02:03:16.540 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.533+0000 7fa28d7fa640 1 -- 172.21.15.16:0/217522076 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa29409ae50 con 0x55d4891f92e0 2024-09-12T02:03:16.540 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.533+0000 7fa2aedbb740 10 client.15696 fetch_fsmap learned FSMap version 25 2024-09-12T02:03:16.540 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.533+0000 7fa2aedbb740 1 -- 172.21.15.16:0/217522076 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d488f3b620 con 0x55d4891f92e0 2024-09-12T02:03:16.540 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.533+0000 7fa28d7fa640 1 -- 172.21.15.16:0/217522076 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2a0005020 con 0x55d4891f92e0 2024-09-12T02:03:16.540 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.533+0000 7fa28d7fa640 1 -- 172.21.15.16:0/217522076 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fa2940b4120 con 0x55d4891f92e0 2024-09-12T02:03:16.540 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.534+0000 7fa2aedbb740 10 client.15696 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:03:16.540 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.534+0000 7fa2aedbb740 10 client.15696 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:03:16.541 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.534+0000 7fa2aedbb740 20 client.15696 populate_metadata read hostname 'smithi016' 2024-09-12T02:03:16.541 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.534+0000 7fa2aedbb740 1 --2- 172.21.15.16:0/217522076 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55d48920e920 0x55d489210d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:16.541 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.534+0000 7fa2aedbb740 4 client.15696 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:03:16.541 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.534+0000 7fa2aedbb740 1 -- 172.21.15.16:0/217522076 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d489169a20 con 0x55d48920e920 2024-09-12T02:03:16.541 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.534+0000 7fa2a9ce5640 1 --2- 172.21.15.16:0/217522076 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55d48920e920 0x55d489210d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:16.541 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.534+0000 7fa28d7fa640 10 client.15696 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:03:16.541 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.534+0000 7fa2a9ce5640 1 --2- 172.21.15.16:0/217522076 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55d48920e920 0x55d489210d10 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5ae65598249840b server_cookie=99c1171640a39943 in_seq=0 out_seq=0 2024-09-12T02:03:16.541 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.535+0000 7fa28d7fa640 1 -- 172.21.15.16:0/217522076 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d489169a20 con 0x55d48920e920 2024-09-12T02:03:16.541 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.535+0000 7fa28d7fa640 10 client.15696 handle_command_reply: tid=0 2024-09-12T02:03:16.541 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.535+0000 7fa2aedbb740 1 client.15696 shutdown 2024-09-12T02:03:16.541 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.535+0000 7fa2aedbb740 1 -- 172.21.15.16:0/217522076 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d4891f92e0 msgr2=0x55d489203130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:16.541 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.535+0000 7fa2aedbb740 1 --2- 172.21.15.16:0/217522076 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d4891f92e0 0x55d489203130 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fa294010fd0 tx=0x7fa294010d70 comp rx=0 tx=0).stop 2024-09-12T02:03:16.541 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.535+0000 7fa2aedbb740 1 -- 172.21.15.16:0/217522076 shutdown_connections 2024-09-12T02:03:16.542 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.535+0000 7fa2aedbb740 1 --2- 172.21.15.16:0/217522076 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55d48920e920 0x55d489210d10 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:16.542 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.535+0000 7fa2aedbb740 1 --2- 172.21.15.16:0/217522076 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d4891fe370 0x55d48920a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:16.542 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.535+0000 7fa2aedbb740 1 --2- 172.21.15.16:0/217522076 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d4891f92e0 0x55d489203130 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:16.542 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.535+0000 7fa2aedbb740 1 --2- 172.21.15.16:0/217522076 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d4891f89a0 0x55d489202bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:16.542 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.535+0000 7fa28d7fa640 0 client.15696 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:03:16.542 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.535+0000 7fa2aedbb740 1 -- 172.21.15.16:0/217522076 >> 172.21.15.16:0/217522076 conn(0x55d4891f4010 msgr2=0x55d4891f5820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:16.542 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.535+0000 7fa2aedbb740 1 -- 172.21.15.16:0/217522076 shutdown_connections 2024-09-12T02:03:16.542 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.535+0000 7fa2aedbb740 1 -- 172.21.15.16:0/217522076 wait complete. 2024-09-12T02:03:16.542 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.536+0000 7fa2aedbb740 20 client.15696 trim_cache size 0 max 16384 2024-09-12T02:03:16.546 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T02:03:16.551 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.550+0000 7fa2aedbb740 1 Processor -- start 2024-09-12T02:03:16.551 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.550+0000 7fa2aedbb740 1 -- start start 2024-09-12T02:03:16.551 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.550+0000 7fa2aedbb740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d4891f89a0 0x55d4891f31c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:16.551 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.550+0000 7fa2aedbb740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d4891f92e0 0x55d48920a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:16.551 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.550+0000 7fa2aedbb740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d4891fe370 0x55d489215610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:16.551 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.550+0000 7fa2aedbb740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a41ab6a0 con 0x55d4891f92e0 2024-09-12T02:03:16.552 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.550+0000 7fa2aedbb740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a41a8f70 con 0x55d4891f89a0 2024-09-12T02:03:16.552 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.550+0000 7fa2aedbb740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a41a6dc0 con 0x55d4891fe370 2024-09-12T02:03:16.552 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.550+0000 7fa2a94e4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d4891f92e0 0x55d48920a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:16.552 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.551+0000 7fa2a9ce5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d4891f89a0 0x55d4891f31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:16.552 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.551+0000 7fa2a94e4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d4891f92e0 0x55d48920a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49638/0 (socket says 172.21.15.16:49638) 2024-09-12T02:03:16.552 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.551+0000 7fa2a94e4640 1 -- 172.21.15.16:0/1213128949 learned_addr learned my addr 172.21.15.16:0/1213128949 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:03:16.552 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.551+0000 7fa2aa4e6640 1 --2- 172.21.15.16:0/1213128949 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d4891fe370 0x55d489215610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:16.552 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.551+0000 7fa2a9ce5640 1 -- 172.21.15.16:0/1213128949 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d4891fe370 msgr2=0x55d489215610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:16.552 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.551+0000 7fa2a9ce5640 1 --2- 172.21.15.16:0/1213128949 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d4891fe370 0x55d489215610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:16.552 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.551+0000 7fa2a9ce5640 1 -- 172.21.15.16:0/1213128949 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d4891f92e0 msgr2=0x55d48920a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:16.552 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.551+0000 7fa2a9ce5640 1 --2- 172.21.15.16:0/1213128949 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d4891f92e0 0x55d48920a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:16.552 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.551+0000 7fa2a9ce5640 1 -- 172.21.15.16:0/1213128949 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d4891fd770 con 0x55d4891f89a0 2024-09-12T02:03:16.552 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.551+0000 7fa2a94e4640 1 --2- 172.21.15.16:0/1213128949 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d4891f92e0 0x55d48920a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:03:16.553 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.551+0000 7fa2a9ce5640 1 --2- 172.21.15.16:0/1213128949 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d4891f89a0 0x55d4891f31c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa2a001b040 tx=0x7fa2a0005040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=97924263a842c3ed server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:16.553 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.551+0000 7fa28f7fe640 1 -- 172.21.15.16:0/1213128949 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2a0030070 con 0x55d4891f89a0 2024-09-12T02:03:16.553 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.551+0000 7fa28f7fe640 1 -- 172.21.15.16:0/1213128949 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2a00054e0 con 0x55d4891f89a0 2024-09-12T02:03:16.553 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.551+0000 7fa28f7fe640 1 -- 172.21.15.16:0/1213128949 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2a0003cc0 con 0x55d4891f89a0 2024-09-12T02:03:16.556 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.552+0000 7fa2aedbb740 1 -- 172.21.15.16:0/1213128949 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d4891f89a0 msgr2=0x55d4891f31c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:16.556 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.552+0000 7fa2aedbb740 1 --2- 172.21.15.16:0/1213128949 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d4891f89a0 0x55d4891f31c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa2a001b040 tx=0x7fa2a0005040 comp rx=0 tx=0).stop 2024-09-12T02:03:16.556 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.552+0000 7fa2aedbb740 1 -- 172.21.15.16:0/1213128949 shutdown_connections 2024-09-12T02:03:16.556 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.552+0000 7fa2aedbb740 1 --2- 172.21.15.16:0/1213128949 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d4891fe370 0x55d489215610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:16.556 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.552+0000 7fa2aedbb740 1 --2- 172.21.15.16:0/1213128949 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d4891f92e0 0x55d48920a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:16.557 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.552+0000 7fa2aedbb740 1 --2- 172.21.15.16:0/1213128949 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d4891f89a0 0x55d4891f31c0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:16.557 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.552+0000 7fa2aedbb740 1 -- 172.21.15.16:0/1213128949 >> 172.21.15.16:0/1213128949 conn(0x55d4891f4010 msgr2=0x55d489201f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:16.557 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.552+0000 7fa2aedbb740 1 -- 172.21.15.16:0/1213128949 shutdown_connections 2024-09-12T02:03:16.557 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.552+0000 7fa2aedbb740 1 -- 172.21.15.16:0/1213128949 wait complete. 2024-09-12T02:03:16.557 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.552+0000 7fa2aedbb740 1 -- start start 2024-09-12T02:03:16.557 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.553+0000 7fa2aedbb740 1 Processor -- start 2024-09-12T02:03:16.557 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.553+0000 7fa2aedbb740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d4891f89a0 0x55d489207db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:16.557 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.553+0000 7fa2aedbb740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d4891f92e0 0x55d4892082f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:16.557 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.553+0000 7fa2aedbb740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d4891fe370 0x55d489209300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:16.557 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.553+0000 7fa2aedbb740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a41a7430 con 0x55d4891fe370 2024-09-12T02:03:16.557 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.553+0000 7fa2aedbb740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a41a88a0 con 0x55d4891f89a0 2024-09-12T02:03:16.557 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.553+0000 7fa2aedbb740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a41a7b00 con 0x55d4891f92e0 2024-09-12T02:03:16.557 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.553+0000 7fa2aedbb740 10 client.0 _ll_register_callbacks cb 0x7fa2940bb040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:03:16.558 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.553+0000 7fa2aa4e6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d4891fe370 0x55d489209300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:16.558 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.553+0000 7fa2aa4e6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d4891fe370 0x55d489209300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49652/0 (socket says 172.21.15.16:49652) 2024-09-12T02:03:16.558 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.553+0000 7fa2aa4e6640 1 -- 172.21.15.16:0/3329454025 learned_addr learned my addr 172.21.15.16:0/3329454025 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:03:16.558 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.553+0000 7fa2a9ce5640 1 --2- 172.21.15.16:0/3329454025 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d4891f89a0 0x55d489207db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:16.558 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.553+0000 7fa2a94e4640 1 --2- 172.21.15.16:0/3329454025 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d4891f92e0 0x55d4892082f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:16.558 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.553+0000 7fa2aa4e6640 1 -- 172.21.15.16:0/3329454025 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d4891f92e0 msgr2=0x55d4892082f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:16.558 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.553+0000 7fa2aa4e6640 1 --2- 172.21.15.16:0/3329454025 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d4891f92e0 0x55d4892082f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:16.558 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.553+0000 7fa2aa4e6640 1 -- 172.21.15.16:0/3329454025 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d4891f89a0 msgr2=0x55d489207db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:16.558 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.553+0000 7fa2aa4e6640 1 --2- 172.21.15.16:0/3329454025 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d4891f89a0 0x55d489207db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:16.558 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.553+0000 7fa2aa4e6640 1 -- 172.21.15.16:0/3329454025 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d489209910 con 0x55d4891fe370 2024-09-12T02:03:16.558 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.553+0000 7fa2a94e4640 1 --2- 172.21.15.16:0/3329454025 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d4891f92e0 0x55d4892082f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:03:16.558 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.554+0000 7fa2aa4e6640 1 --2- 172.21.15.16:0/3329454025 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d4891fe370 0x55d489209300 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fa29c009ca0 tx=0x7fa29c0095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0d1cf19d9a3e45 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:16.558 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.554+0000 7fa28d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:03:16.559 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.554+0000 7fa28d7fa640 1 -- 172.21.15.16:0/3329454025 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa29c055060 con 0x55d4891fe370 2024-09-12T02:03:16.559 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.554+0000 7fa28d7fa640 1 -- 172.21.15.16:0/3329454025 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fa29c0307c0 con 0x55d4891fe370 2024-09-12T02:03:16.559 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.554+0000 7fa2aedbb740 1 -- 172.21.15.16:0/3329454025 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2a00054e0 con 0x55d4891fe370 2024-09-12T02:03:16.559 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.554+0000 7fa28d7fa640 1 -- 172.21.15.16:0/3329454025 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa29c0309c0 con 0x55d4891fe370 2024-09-12T02:03:16.559 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.554+0000 7fa2aedbb740 10 client.15702 fetch_fsmap learned FSMap version 25 2024-09-12T02:03:16.559 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.554+0000 7fa2aedbb740 1 -- 172.21.15.16:0/3329454025 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d489279c00 con 0x55d4891fe370 2024-09-12T02:03:16.559 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.554+0000 7fa28d7fa640 1 -- 172.21.15.16:0/3329454025 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2a00054e0 con 0x55d4891fe370 2024-09-12T02:03:16.559 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.554+0000 7fa28d7fa640 1 -- 172.21.15.16:0/3329454025 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fa29c05a070 con 0x55d4891fe370 2024-09-12T02:03:16.559 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.554+0000 7fa2aedbb740 10 client.15702 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:03:16.559 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.554+0000 7fa2aedbb740 10 client.15702 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:03:16.559 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.554+0000 7fa2aedbb740 20 client.15702 populate_metadata read hostname 'smithi016' 2024-09-12T02:03:16.559 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.554+0000 7fa2aedbb740 1 --2- 172.21.15.16:0/3329454025 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55d48927a5e0 0x55d48927ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:16.559 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.554+0000 7fa2aedbb740 4 client.15702 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:03:16.559 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.554+0000 7fa2aedbb740 1 -- 172.21.15.16:0/3329454025 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d489163d70 con 0x55d48927a5e0 2024-09-12T02:03:16.560 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.555+0000 7fa2a9ce5640 1 --2- 172.21.15.16:0/3329454025 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55d48927a5e0 0x55d48927ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:16.560 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.555+0000 7fa2a9ce5640 1 --2- 172.21.15.16:0/3329454025 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55d48927a5e0 0x55d48927ca00 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=386d204e27143f5f server_cookie=7f5800ced7df493 in_seq=0 out_seq=0 2024-09-12T02:03:16.560 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.555+0000 7fa28d7fa640 10 client.15702 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:03:16.567 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.566+0000 7fa28d7fa640 1 -- 172.21.15.16:0/3329454025 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d489163d70 con 0x55d48927a5e0 2024-09-12T02:03:16.567 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.566+0000 7fa28d7fa640 10 client.15702 handle_command_reply: tid=0 2024-09-12T02:03:16.567 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.566+0000 7fa2aedbb740 1 client.15702 shutdown 2024-09-12T02:03:16.567 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.567+0000 7fa2aedbb740 1 -- 172.21.15.16:0/3329454025 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d4891fe370 msgr2=0x55d489209300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:16.567 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.567+0000 7fa2aedbb740 1 --2- 172.21.15.16:0/3329454025 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d4891fe370 0x55d489209300 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fa29c009ca0 tx=0x7fa29c0095c0 comp rx=0 tx=0).stop 2024-09-12T02:03:16.567 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.567+0000 7fa2aedbb740 1 -- 172.21.15.16:0/3329454025 shutdown_connections 2024-09-12T02:03:16.567 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.567+0000 7fa2aedbb740 1 --2- 172.21.15.16:0/3329454025 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55d48927a5e0 0x55d48927ca00 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:16.567 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.567+0000 7fa2aedbb740 1 --2- 172.21.15.16:0/3329454025 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d4891fe370 0x55d489209300 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:16.568 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.567+0000 7fa28d7fa640 0 client.15702 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:03:16.568 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.567+0000 7fa2aedbb740 1 --2- 172.21.15.16:0/3329454025 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d4891f92e0 0x55d4892082f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:16.568 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.567+0000 7fa2aedbb740 1 --2- 172.21.15.16:0/3329454025 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d4891f89a0 0x55d489207db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:16.568 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.567+0000 7fa2aedbb740 1 -- 172.21.15.16:0/3329454025 >> 172.21.15.16:0/3329454025 conn(0x55d4891f4010 msgr2=0x55d4891f31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:16.568 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.567+0000 7fa2aedbb740 1 -- 172.21.15.16:0/3329454025 shutdown_connections 2024-09-12T02:03:16.568 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.567+0000 7fa2aedbb740 1 -- 172.21.15.16:0/3329454025 wait complete. 2024-09-12T02:03:16.568 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.567+0000 7fa2aedbb740 20 client.15702 trim_cache size 0 max 16384 2024-09-12T02:03:16.568 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-09-12T02:03:16.568 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-09-12T02:03:16.568 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-09-12T02:03:16.569 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-09-12T02:03:16.569 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.568+0000 7fa2abf49640 1 -- 172.21.15.16:0/3644325825 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fa2900a2c30 msgr2=0x7fa2900a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:16.569 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.568+0000 7fa2abf49640 1 --2- 172.21.15.16:0/3644325825 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fa2900a2c30 0x7fa2900a50f0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fa29c006010 tx=0x7fa29c02a8c0 comp rx=0 tx=0).stop 2024-09-12T02:03:16.569 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.568+0000 7fa2abf49640 1 -- 172.21.15.16:0/3644325825 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2a4101c30 msgr2=0x7fa2a41982a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:16.569 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.568+0000 7fa2abf49640 1 --2- 172.21.15.16:0/3644325825 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2a4101c30 0x7fa2a41982a0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fa29400d950 tx=0x7fa29400de10 comp rx=0 tx=0).stop 2024-09-12T02:03:16.570 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.568+0000 7fa2abf49640 1 -- 172.21.15.16:0/3644325825 shutdown_connections 2024-09-12T02:03:16.570 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.568+0000 7fa2abf49640 1 --2- 172.21.15.16:0/3644325825 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fa2900a2c30 0x7fa2900a50f0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:16.570 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.568+0000 7fa2abf49640 1 --2- 172.21.15.16:0/3644325825 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2a41025f0 0x7fa2a419f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:16.570 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.568+0000 7fa2abf49640 1 --2- 172.21.15.16:0/3644325825 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2a4101c30 0x7fa2a41982a0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:16.570 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.568+0000 7fa2abf49640 1 --2- 172.21.15.16:0/3644325825 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2a4100a40 0x7fa2a4197d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:16.570 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.568+0000 7fa2abf49640 1 -- 172.21.15.16:0/3644325825 >> 172.21.15.16:0/3644325825 conn(0x7fa2a40fc1e0 msgr2=0x7fa2a40fe130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:16.570 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.568+0000 7fa2abf49640 1 -- 172.21.15.16:0/3644325825 shutdown_connections 2024-09-12T02:03:16.570 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:16.568+0000 7fa2abf49640 1 -- 172.21.15.16:0/3644325825 wait complete. 2024-09-12T02:03:16.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:03:16 smithi016 ceph-mon[30947]: scrub queued for path: / 2024-09-12T02:03:16.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:03:16 smithi016 ceph-mon[30947]: scrub summary: idle+waiting paths [/] 2024-09-12T02:03:16.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:03:16 smithi016 ceph-mon[30947]: scrub summary: active paths [/] 2024-09-12T02:03:16.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:03:16 smithi016 ceph-mon[30947]: scrub summary: idle+waiting paths [/] 2024-09-12T02:03:16.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:03:16 smithi016 ceph-mon[30947]: scrub summary: idle 2024-09-12T02:03:16.805 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:03:16.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:03:16 smithi184 ceph-mon[35593]: scrub queued for path: / 2024-09-12T02:03:16.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:03:16 smithi184 ceph-mon[35593]: scrub summary: idle+waiting paths [/] 2024-09-12T02:03:16.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:03:16 smithi184 ceph-mon[35593]: scrub summary: active paths [/] 2024-09-12T02:03:16.878 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:03:16 smithi184 ceph-mon[35593]: scrub summary: idle+waiting paths [/] 2024-09-12T02:03:16.878 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:03:16 smithi184 ceph-mon[35593]: scrub summary: idle 2024-09-12T02:03:16.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:03:16 smithi033 ceph-mon[35332]: scrub queued for path: / 2024-09-12T02:03:16.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:03:16 smithi033 ceph-mon[35332]: scrub summary: idle+waiting paths [/] 2024-09-12T02:03:16.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:03:16 smithi033 ceph-mon[35332]: scrub summary: active paths [/] 2024-09-12T02:03:16.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:03:16 smithi033 ceph-mon[35332]: scrub summary: idle+waiting paths [/] 2024-09-12T02:03:16.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:03:16 smithi033 ceph-mon[35332]: scrub summary: idle 2024-09-12T02:03:17.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.232+0000 7f52994c0640 1 -- 172.21.15.16:0/619680579 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5294102790 msgr2=0x7f52941067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:17.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.232+0000 7f52994c0640 1 --2- 172.21.15.16:0/619680579 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5294102790 0x7f52941067e0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f5288009900 tx=0x7f528802a410 comp rx=0 tx=0).stop 2024-09-12T02:03:17.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.232+0000 7f52994c0640 1 -- 172.21.15.16:0/619680579 shutdown_connections 2024-09-12T02:03:17.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.232+0000 7f52994c0640 1 --2- 172.21.15.16:0/619680579 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5294106f10 0x7f529410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:17.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.232+0000 7f52994c0640 1 --2- 172.21.15.16:0/619680579 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5294102790 0x7f52941067e0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:17.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.232+0000 7f52994c0640 1 --2- 172.21.15.16:0/619680579 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5294101de0 0x7f52941021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:17.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.232+0000 7f52994c0640 1 -- 172.21.15.16:0/619680579 >> 172.21.15.16:0/619680579 conn(0x7f52940fd8c0 msgr2=0x7f52940ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:17.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.232+0000 7f52994c0640 1 -- 172.21.15.16:0/619680579 shutdown_connections 2024-09-12T02:03:17.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.232+0000 7f52994c0640 1 -- 172.21.15.16:0/619680579 wait complete. 2024-09-12T02:03:17.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.233+0000 7f52994c0640 1 Processor -- start 2024-09-12T02:03:17.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.233+0000 7f52994c0640 1 -- start start 2024-09-12T02:03:17.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.233+0000 7f52994c0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5294101de0 0x7f5294198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:17.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.233+0000 7f52994c0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5294102790 0x7f52941989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:17.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.233+0000 7f52994c0640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5294106f10 0x7f529419cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:17.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.233+0000 7f52994c0640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5294111f70 con 0x7f5294102790 2024-09-12T02:03:17.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.233+0000 7f52994c0640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5294111df0 con 0x7f5294101de0 2024-09-12T02:03:17.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.233+0000 7f52994c0640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f52941120f0 con 0x7f5294106f10 2024-09-12T02:03:17.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.234+0000 7f52927fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5294102790 0x7f52941989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:17.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.234+0000 7f52927fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5294102790 0x7f52941989d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49672/0 (socket says 172.21.15.16:49672) 2024-09-12T02:03:17.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.234+0000 7f52927fc640 1 -- 172.21.15.16:0/615219223 learned_addr learned my addr 172.21.15.16:0/615219223 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:03:17.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.234+0000 7f5292ffd640 1 --2- 172.21.15.16:0/615219223 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5294101de0 0x7f5294198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:17.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.234+0000 7f52937fe640 1 --2- 172.21.15.16:0/615219223 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5294106f10 0x7f529419cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:17.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.234+0000 7f52927fc640 1 -- 172.21.15.16:0/615219223 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5294106f10 msgr2=0x7f529419cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:17.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.234+0000 7f52927fc640 1 --2- 172.21.15.16:0/615219223 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5294106f10 0x7f529419cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:17.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.234+0000 7f52927fc640 1 -- 172.21.15.16:0/615219223 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5294101de0 msgr2=0x7f5294198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:17.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.234+0000 7f52927fc640 1 --2- 172.21.15.16:0/615219223 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5294101de0 0x7f5294198490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:17.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.234+0000 7f52927fc640 1 -- 172.21.15.16:0/615219223 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f529419d440 con 0x7f5294102790 2024-09-12T02:03:17.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.234+0000 7f52927fc640 1 --2- 172.21.15.16:0/615219223 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5294102790 0x7f52941989d0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f5288006010 tx=0x7f52880042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:17.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.234+0000 7f527bfff640 1 -- 172.21.15.16:0/615219223 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f528803f030 con 0x7f5294102790 2024-09-12T02:03:17.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.234+0000 7f527bfff640 1 -- 172.21.15.16:0/615219223 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5288003ce0 con 0x7f5294102790 2024-09-12T02:03:17.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.234+0000 7f52994c0640 1 -- 172.21.15.16:0/615219223 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f529419d6d0 con 0x7f5294102790 2024-09-12T02:03:17.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.234+0000 7f527bfff640 1 -- 172.21.15.16:0/615219223 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5288035070 con 0x7f5294102790 2024-09-12T02:03:17.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.234+0000 7f52994c0640 1 -- 172.21.15.16:0/615219223 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52941a5010 con 0x7f5294102790 2024-09-12T02:03:17.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.237+0000 7f527bfff640 1 -- 172.21.15.16:0/615219223 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5288042030 con 0x7f5294102790 2024-09-12T02:03:17.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.237+0000 7f529b899740 1 Processor -- start 2024-09-12T02:03:17.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.237+0000 7f529b899740 1 -- start start 2024-09-12T02:03:17.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.237+0000 7f529b899740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b385f44a0 0x560b385f4880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:17.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.237+0000 7f529b899740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560b385f4dc0 0x560b385ff140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:17.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.237+0000 7f529b899740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b385ff680 0x560b38601a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:17.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.237+0000 7f529b899740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x560b3855a2e0 con 0x560b385f44a0 2024-09-12T02:03:17.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.237+0000 7f529b899740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x560b38501ce0 con 0x560b385ff680 2024-09-12T02:03:17.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.238+0000 7f529b899740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x560b385562a0 con 0x560b385f4dc0 2024-09-12T02:03:17.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.238+0000 7f52937fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b385f44a0 0x560b385f4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:17.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.238+0000 7f52937fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b385f44a0 0x560b385f4880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49686/0 (socket says 172.21.15.16:49686) 2024-09-12T02:03:17.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.238+0000 7f52937fe640 1 -- 172.21.15.16:0/1788039967 learned_addr learned my addr 172.21.15.16:0/1788039967 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:03:17.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.238+0000 7f52927fc640 1 --2- 172.21.15.16:0/1788039967 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b385ff680 0x560b38601a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:17.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.238+0000 7f52937fe640 1 -- 172.21.15.16:0/1788039967 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560b385f4dc0 msgr2=0x560b385ff140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:17.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.238+0000 7f52937fe640 1 --2- 172.21.15.16:0/1788039967 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560b385f4dc0 0x560b385ff140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:17.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.238+0000 7f52937fe640 1 -- 172.21.15.16:0/1788039967 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b385ff680 msgr2=0x560b38601a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:17.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.238+0000 7f52937fe640 1 --2- 172.21.15.16:0/1788039967 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b385ff680 0x560b38601a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:17.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.238+0000 7f527bfff640 1 --2- 172.21.15.16:0/615219223 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f527c0a2b80 0x7f527c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:17.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.238+0000 7f52937fe640 1 -- 172.21.15.16:0/1788039967 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b384e6440 con 0x560b385f44a0 2024-09-12T02:03:17.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.238+0000 7f5292ffd640 1 --2- 172.21.15.16:0/615219223 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f527c0a2b80 0x7f527c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:17.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.238+0000 7f527bfff640 1 -- 172.21.15.16:0/615219223 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f52880ef4d0 con 0x7f5294102790 2024-09-12T02:03:17.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.238+0000 7f52937fe640 1 --2- 172.21.15.16:0/1788039967 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b385f44a0 0x560b385f4880 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f528c009f90 tx=0x7f528c010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66429e78a65b2978 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:17.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.238+0000 7f52797fa640 1 -- 172.21.15.16:0/1788039967 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f528c011700 con 0x560b385f44a0 2024-09-12T02:03:17.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.238+0000 7f52797fa640 1 -- 172.21.15.16:0/1788039967 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f528c011d80 con 0x560b385f44a0 2024-09-12T02:03:17.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.238+0000 7f52797fa640 1 -- 172.21.15.16:0/1788039967 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f528c018380 con 0x560b385f44a0 2024-09-12T02:03:17.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.239+0000 7f529b899740 1 -- 172.21.15.16:0/1788039967 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b385f44a0 msgr2=0x560b385f4880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:17.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.239+0000 7f5292ffd640 1 --2- 172.21.15.16:0/615219223 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f527c0a2b80 0x7f527c0a5040 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x560b385f1890 tx=0x7f5284009840 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:17.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.239+0000 7f529b899740 1 --2- 172.21.15.16:0/1788039967 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b385f44a0 0x560b385f4880 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f528c009f90 tx=0x7f528c010860 comp rx=0 tx=0).stop 2024-09-12T02:03:17.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.239+0000 7f529b899740 1 -- 172.21.15.16:0/1788039967 shutdown_connections 2024-09-12T02:03:17.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.239+0000 7f529b899740 1 --2- 172.21.15.16:0/1788039967 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b385ff680 0x560b38601a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:17.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.239+0000 7f529b899740 1 --2- 172.21.15.16:0/1788039967 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560b385f4dc0 0x560b385ff140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:17.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.239+0000 7f529b899740 1 --2- 172.21.15.16:0/1788039967 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b385f44a0 0x560b385f4880 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:17.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.239+0000 7f529b899740 1 -- 172.21.15.16:0/1788039967 >> 172.21.15.16:0/1788039967 conn(0x560b385f0a30 msgr2=0x560b385f0e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:17.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.239+0000 7f529b899740 1 -- 172.21.15.16:0/1788039967 shutdown_connections 2024-09-12T02:03:17.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.239+0000 7f529b899740 1 -- 172.21.15.16:0/1788039967 wait complete. 2024-09-12T02:03:17.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.239+0000 7f529b899740 1 -- start start 2024-09-12T02:03:17.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.240+0000 7f529b899740 1 Processor -- start 2024-09-12T02:03:17.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.240+0000 7f529b899740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b385f44a0 0x560b385fefa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:17.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.240+0000 7f529b899740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560b385f4dc0 0x560b385fc250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:17.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.240+0000 7f529b899740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b385ff680 0x560b385fc790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:17.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.240+0000 7f529b899740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x560b38602250 con 0x560b385f44a0 2024-09-12T02:03:17.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.240+0000 7f529b899740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x560b38515050 con 0x560b385ff680 2024-09-12T02:03:17.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.240+0000 7f529b899740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x560b386023d0 con 0x560b385f4dc0 2024-09-12T02:03:17.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.240+0000 7f529b899740 10 client.0 _ll_register_callbacks cb 0x560b38555160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:03:17.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.240+0000 7f5292ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b385f44a0 0x560b385fefa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:17.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.240+0000 7f5292ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b385f44a0 0x560b385fefa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49702/0 (socket says 172.21.15.16:49702) 2024-09-12T02:03:17.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.240+0000 7f5292ffd640 1 -- 172.21.15.16:0/2305096767 learned_addr learned my addr 172.21.15.16:0/2305096767 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:03:17.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.240+0000 7f52937fe640 1 --2- 172.21.15.16:0/2305096767 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b385ff680 0x560b385fc790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:17.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.240+0000 7f52927fc640 1 --2- 172.21.15.16:0/2305096767 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560b385f4dc0 0x560b385fc250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:17.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.240+0000 7f5292ffd640 1 -- 172.21.15.16:0/2305096767 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560b385f4dc0 msgr2=0x560b385fc250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:17.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.240+0000 7f5292ffd640 1 --2- 172.21.15.16:0/2305096767 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560b385f4dc0 0x560b385fc250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:17.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.240+0000 7f5292ffd640 1 -- 172.21.15.16:0/2305096767 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b385ff680 msgr2=0x560b385fc790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:17.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.240+0000 7f5292ffd640 1 --2- 172.21.15.16:0/2305096767 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b385ff680 0x560b385fc790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:17.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.240+0000 7f5292ffd640 1 -- 172.21.15.16:0/2305096767 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b382cb3c0 con 0x560b385f44a0 2024-09-12T02:03:17.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.240+0000 7f52937fe640 1 --2- 172.21.15.16:0/2305096767 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b385ff680 0x560b385fc790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:03:17.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.241+0000 7f5292ffd640 1 --2- 172.21.15.16:0/2305096767 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b385f44a0 0x560b385fefa0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f528401cad0 tx=0x7f5284005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a39a022e99a521f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:17.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.241+0000 7f5276ff5640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:03:17.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.241+0000 7f5276ff5640 1 -- 172.21.15.16:0/2305096767 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f528401d820 con 0x560b385f44a0 2024-09-12T02:03:17.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.241+0000 7f5276ff5640 1 -- 172.21.15.16:0/2305096767 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f528401d9c0 con 0x560b385f44a0 2024-09-12T02:03:17.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.241+0000 7f529b899740 1 -- 172.21.15.16:0/2305096767 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f528c011d80 con 0x560b385f44a0 2024-09-12T02:03:17.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.241+0000 7f5276ff5640 1 -- 172.21.15.16:0/2305096767 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f528401dbc0 con 0x560b385f44a0 2024-09-12T02:03:17.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.241+0000 7f529b899740 10 client.15714 fetch_fsmap learned FSMap version 25 2024-09-12T02:03:17.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.241+0000 7f529b899740 1 -- 172.21.15.16:0/2305096767 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x560b3860dc30 con 0x560b385f44a0 2024-09-12T02:03:17.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.241+0000 7f5276ff5640 1 -- 172.21.15.16:0/2305096767 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f528c011d80 con 0x560b385f44a0 2024-09-12T02:03:17.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.241+0000 7f5276ff5640 1 -- 172.21.15.16:0/2305096767 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f5284032440 con 0x560b385f44a0 2024-09-12T02:03:17.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.241+0000 7f529b899740 10 client.15714 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:03:17.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.241+0000 7f529b899740 10 client.15714 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:03:17.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.241+0000 7f529b899740 20 client.15714 populate_metadata read hostname 'smithi016' 2024-09-12T02:03:17.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.242+0000 7f529b899740 1 --2- 172.21.15.16:0/2305096767 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x560b3860f0f0 0x560b38611520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:17.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.242+0000 7f529b899740 4 client.15714 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:03:17.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.242+0000 7f529b899740 1 -- 172.21.15.16:0/2305096767 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560b38567850 con 0x560b3860f0f0 2024-09-12T02:03:17.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.242+0000 7f52927fc640 1 --2- 172.21.15.16:0/2305096767 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x560b3860f0f0 0x560b38611520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:17.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.242+0000 7f5276ff5640 10 client.15714 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:03:17.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.242+0000 7f52927fc640 1 --2- 172.21.15.16:0/2305096767 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x560b3860f0f0 0x560b38611520 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d08d9279bae3ea08 server_cookie=10cca93795c0a36d in_seq=0 out_seq=0 2024-09-12T02:03:17.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.243+0000 7f5276ff5640 1 -- 172.21.15.16:0/2305096767 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560b38567850 con 0x560b3860f0f0 2024-09-12T02:03:17.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.243+0000 7f5276ff5640 10 client.15714 handle_command_reply: tid=0 2024-09-12T02:03:17.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.243+0000 7f529b899740 1 client.15714 shutdown 2024-09-12T02:03:17.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.243+0000 7f529b899740 1 -- 172.21.15.16:0/2305096767 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b385f44a0 msgr2=0x560b385fefa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:17.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.243+0000 7f529b899740 1 --2- 172.21.15.16:0/2305096767 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b385f44a0 0x560b385fefa0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f528401cad0 tx=0x7f5284005f90 comp rx=0 tx=0).stop 2024-09-12T02:03:17.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.243+0000 7f529b899740 1 -- 172.21.15.16:0/2305096767 shutdown_connections 2024-09-12T02:03:17.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.243+0000 7f529b899740 1 --2- 172.21.15.16:0/2305096767 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x560b3860f0f0 0x560b38611520 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:17.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.243+0000 7f529b899740 1 --2- 172.21.15.16:0/2305096767 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b385ff680 0x560b385fc790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:17.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.243+0000 7f529b899740 1 --2- 172.21.15.16:0/2305096767 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560b385f4dc0 0x560b385fc250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:17.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.243+0000 7f529b899740 1 --2- 172.21.15.16:0/2305096767 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b385f44a0 0x560b385fefa0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:17.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.243+0000 7f529b899740 1 -- 172.21.15.16:0/2305096767 >> 172.21.15.16:0/2305096767 conn(0x560b385f0a30 msgr2=0x560b385f6710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:17.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.243+0000 7f5276ff5640 0 client.15714 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:03:17.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.243+0000 7f529b899740 1 -- 172.21.15.16:0/2305096767 shutdown_connections 2024-09-12T02:03:17.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.243+0000 7f529b899740 1 -- 172.21.15.16:0/2305096767 wait complete. 2024-09-12T02:03:17.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.243+0000 7f529b899740 20 client.15714 trim_cache size 0 max 16384 2024-09-12T02:03:17.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.260+0000 7f529b899740 1 Processor -- start 2024-09-12T02:03:17.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.260+0000 7f529b899740 1 -- start start 2024-09-12T02:03:17.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.261+0000 7f529b899740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b385f0a30 0x560b385efa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:17.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.261+0000 7f529b899740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b385f44a0 0x560b38616df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:17.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.261+0000 7f529b899740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560b385f4dc0 0x560b38617330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:17.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.261+0000 7f529b899740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f52941afd70 con 0x560b385f0a30 2024-09-12T02:03:17.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.261+0000 7f529b899740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f52941ad640 con 0x560b385f44a0 2024-09-12T02:03:17.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.261+0000 7f529b899740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f52941ab4d0 con 0x560b385f4dc0 2024-09-12T02:03:17.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.261+0000 7f5292ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b385f0a30 0x560b385efa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:17.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.261+0000 7f5292ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b385f0a30 0x560b385efa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49708/0 (socket says 172.21.15.16:49708) 2024-09-12T02:03:17.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.261+0000 7f5292ffd640 1 -- 172.21.15.16:0/3787541001 learned_addr learned my addr 172.21.15.16:0/3787541001 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:03:17.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.261+0000 7f52927fc640 1 --2- 172.21.15.16:0/3787541001 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b385f44a0 0x560b38616df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:17.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.261+0000 7f52937fe640 1 --2- 172.21.15.16:0/3787541001 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560b385f4dc0 0x560b38617330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:17.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.261+0000 7f5292ffd640 1 -- 172.21.15.16:0/3787541001 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560b385f4dc0 msgr2=0x560b38617330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:17.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.261+0000 7f5292ffd640 1 --2- 172.21.15.16:0/3787541001 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560b385f4dc0 0x560b38617330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:17.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.261+0000 7f5292ffd640 1 -- 172.21.15.16:0/3787541001 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b385f44a0 msgr2=0x560b38616df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:17.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.261+0000 7f5292ffd640 1 --2- 172.21.15.16:0/3787541001 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b385f44a0 0x560b38616df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:17.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.261+0000 7f5292ffd640 1 -- 172.21.15.16:0/3787541001 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b385f1ba0 con 0x560b385f0a30 2024-09-12T02:03:17.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.261+0000 7f5292ffd640 1 --2- 172.21.15.16:0/3787541001 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b385f0a30 0x560b385efa80 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f52840092e0 tx=0x7f5284009df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5820440bfc273076 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:17.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.262+0000 7f5278ff9640 1 -- 172.21.15.16:0/3787541001 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5284005c90 con 0x560b385f0a30 2024-09-12T02:03:17.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.262+0000 7f5278ff9640 1 -- 172.21.15.16:0/3787541001 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5284032cd0 con 0x560b385f0a30 2024-09-12T02:03:17.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.262+0000 7f5278ff9640 1 -- 172.21.15.16:0/3787541001 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5284004680 con 0x560b385f0a30 2024-09-12T02:03:17.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.262+0000 7f529b899740 1 -- 172.21.15.16:0/3787541001 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b385f0a30 msgr2=0x560b385efa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:17.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.262+0000 7f529b899740 1 --2- 172.21.15.16:0/3787541001 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b385f0a30 0x560b385efa80 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f52840092e0 tx=0x7f5284009df0 comp rx=0 tx=0).stop 2024-09-12T02:03:17.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.262+0000 7f529b899740 1 -- 172.21.15.16:0/3787541001 shutdown_connections 2024-09-12T02:03:17.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.262+0000 7f529b899740 1 --2- 172.21.15.16:0/3787541001 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560b385f4dc0 0x560b38617330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:17.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.262+0000 7f529b899740 1 --2- 172.21.15.16:0/3787541001 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b385f44a0 0x560b38616df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:17.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.262+0000 7f529b899740 1 --2- 172.21.15.16:0/3787541001 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b385f0a30 0x560b385efa80 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:17.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.262+0000 7f529b899740 1 -- 172.21.15.16:0/3787541001 >> 172.21.15.16:0/3787541001 conn(0x560b385fc8f0 msgr2=0x560b385ef370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:17.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.262+0000 7f529b899740 1 -- 172.21.15.16:0/3787541001 shutdown_connections 2024-09-12T02:03:17.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.262+0000 7f529b899740 1 -- 172.21.15.16:0/3787541001 wait complete. 2024-09-12T02:03:17.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.263+0000 7f529b899740 1 -- start start 2024-09-12T02:03:17.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.263+0000 7f529b899740 1 Processor -- start 2024-09-12T02:03:17.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.263+0000 7f529b899740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560b385f0a30 0x560b38607980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:17.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.263+0000 7f529b899740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b385f44a0 0x560b38607ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:17.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.263+0000 7f529b899740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b385f4dc0 0x560b38604f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:17.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.263+0000 7f529b899740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f52941abb20 con 0x560b385f44a0 2024-09-12T02:03:17.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.263+0000 7f529b899740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f52941acf70 con 0x560b385f4dc0 2024-09-12T02:03:17.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.263+0000 7f529b899740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f52941ac1d0 con 0x560b385f0a30 2024-09-12T02:03:17.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.263+0000 7f529b899740 10 client.0 _ll_register_callbacks cb 0x7f5284009a90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:03:17.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.263+0000 7f52927fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b385f44a0 0x560b38607ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:17.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.263+0000 7f52927fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b385f44a0 0x560b38607ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49720/0 (socket says 172.21.15.16:49720) 2024-09-12T02:03:17.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.263+0000 7f52927fc640 1 -- 172.21.15.16:0/687709748 learned_addr learned my addr 172.21.15.16:0/687709748 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:03:17.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.263+0000 7f5292ffd640 1 --2- 172.21.15.16:0/687709748 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560b385f0a30 0x560b38607980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:17.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.263+0000 7f52937fe640 1 --2- 172.21.15.16:0/687709748 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b385f4dc0 0x560b38604f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:17.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.264+0000 7f52927fc640 1 -- 172.21.15.16:0/687709748 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560b385f0a30 msgr2=0x560b38607980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:17.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.264+0000 7f52927fc640 1 --2- 172.21.15.16:0/687709748 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560b385f0a30 0x560b38607980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:17.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.264+0000 7f52927fc640 1 -- 172.21.15.16:0/687709748 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b385f4dc0 msgr2=0x560b38604f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:17.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.264+0000 7f52927fc640 1 --2- 172.21.15.16:0/687709748 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b385f4dc0 0x560b38604f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:17.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.264+0000 7f52927fc640 1 -- 172.21.15.16:0/687709748 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b386057a0 con 0x560b385f44a0 2024-09-12T02:03:17.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.264+0000 7f52937fe640 1 --2- 172.21.15.16:0/687709748 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b385f4dc0 0x560b38604f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:03:17.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.264+0000 7f5292ffd640 1 --2- 172.21.15.16:0/687709748 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560b385f0a30 0x560b38607980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:03:17.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.264+0000 7f52927fc640 1 --2- 172.21.15.16:0/687709748 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b385f44a0 0x560b38607ec0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f52880ed070 tx=0x7f52880ef6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80641a21d651016c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:17.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.264+0000 7f5276ff5640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:03:17.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.264+0000 7f5276ff5640 1 -- 172.21.15.16:0/687709748 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5288049040 con 0x560b385f44a0 2024-09-12T02:03:17.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.264+0000 7f5276ff5640 1 -- 172.21.15.16:0/687709748 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f52880b85b0 con 0x560b385f44a0 2024-09-12T02:03:17.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.264+0000 7f529b899740 1 -- 172.21.15.16:0/687709748 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5284032cd0 con 0x560b385f44a0 2024-09-12T02:03:17.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.264+0000 7f5276ff5640 1 -- 172.21.15.16:0/687709748 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f52880bba80 con 0x560b385f44a0 2024-09-12T02:03:17.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.264+0000 7f529b899740 10 client.15720 fetch_fsmap learned FSMap version 25 2024-09-12T02:03:17.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.264+0000 7f529b899740 1 -- 172.21.15.16:0/687709748 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x560b38679a60 con 0x560b385f44a0 2024-09-12T02:03:17.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.264+0000 7f5276ff5640 1 -- 172.21.15.16:0/687709748 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5284032cd0 con 0x560b385f44a0 2024-09-12T02:03:17.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.264+0000 7f5276ff5640 1 -- 172.21.15.16:0/687709748 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f52880ef0c0 con 0x560b385f44a0 2024-09-12T02:03:17.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.264+0000 7f529b899740 10 client.15720 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:03:17.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.265+0000 7f529b899740 10 client.15720 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:03:17.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.265+0000 7f529b899740 20 client.15720 populate_metadata read hostname 'smithi016' 2024-09-12T02:03:17.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.265+0000 7f529b899740 1 --2- 172.21.15.16:0/687709748 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x560b385ff680 0x560b3867c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:17.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.265+0000 7f529b899740 4 client.15720 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T02:03:17.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.265+0000 7f529b899740 1 -- 172.21.15.16:0/687709748 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560b38561d60 con 0x560b385ff680 2024-09-12T02:03:17.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.265+0000 7f5292ffd640 1 --2- 172.21.15.16:0/687709748 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x560b385ff680 0x560b3867c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:17.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.265+0000 7f5292ffd640 1 --2- 172.21.15.16:0/687709748 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x560b385ff680 0x560b3867c1b0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=49abc008aeeb2e1f server_cookie=82e6aaa760031b16 in_seq=0 out_seq=0 2024-09-12T02:03:17.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.265+0000 7f5276ff5640 10 client.15720 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:03:17.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.266+0000 7f5276ff5640 1 -- 172.21.15.16:0/687709748 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x560b38561d60 con 0x560b385ff680 2024-09-12T02:03:17.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.266+0000 7f5276ff5640 10 client.15720 handle_command_reply: tid=0 2024-09-12T02:03:17.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.266+0000 7f529b899740 1 client.15720 shutdown 2024-09-12T02:03:17.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.266+0000 7f529b899740 1 -- 172.21.15.16:0/687709748 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b385f44a0 msgr2=0x560b38607ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:17.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.266+0000 7f529b899740 1 --2- 172.21.15.16:0/687709748 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b385f44a0 0x560b38607ec0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f52880ed070 tx=0x7f52880ef6d0 comp rx=0 tx=0).stop 2024-09-12T02:03:17.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.266+0000 7f529b899740 1 -- 172.21.15.16:0/687709748 shutdown_connections 2024-09-12T02:03:17.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.266+0000 7f529b899740 1 --2- 172.21.15.16:0/687709748 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x560b385ff680 0x560b3867c1b0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:17.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.266+0000 7f5276ff5640 0 client.15720 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:03:17.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.266+0000 7f529b899740 1 --2- 172.21.15.16:0/687709748 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b385f4dc0 0x560b38604f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:17.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.266+0000 7f529b899740 1 --2- 172.21.15.16:0/687709748 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b385f44a0 0x560b38607ec0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:17.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.266+0000 7f529b899740 1 --2- 172.21.15.16:0/687709748 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560b385f0a30 0x560b38607980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:17.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.266+0000 7f529b899740 1 -- 172.21.15.16:0/687709748 >> 172.21.15.16:0/687709748 conn(0x560b385fc8f0 msgr2=0x560b385ef270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:17.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.266+0000 7f529b899740 1 -- 172.21.15.16:0/687709748 shutdown_connections 2024-09-12T02:03:17.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.266+0000 7f529b899740 1 -- 172.21.15.16:0/687709748 wait complete. 2024-09-12T02:03:17.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.266+0000 7f529b899740 20 client.15720 trim_cache size 0 max 16384 2024-09-12T02:03:17.272 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T02:03:17.272 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-09-12T02:03:17.272 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {} 2024-09-12T02:03:17.272 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T02:03:17.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.266+0000 7f52994c0640 1 -- 172.21.15.16:0/615219223 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f527c0a2b80 msgr2=0x7f527c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:17.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.266+0000 7f52994c0640 1 --2- 172.21.15.16:0/615219223 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f527c0a2b80 0x7f527c0a5040 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x560b385f1890 tx=0x7f5284009840 comp rx=0 tx=0).stop 2024-09-12T02:03:17.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.266+0000 7f52994c0640 1 -- 172.21.15.16:0/615219223 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5294102790 msgr2=0x7f52941989d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:17.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.266+0000 7f52994c0640 1 --2- 172.21.15.16:0/615219223 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5294102790 0x7f52941989d0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f5288006010 tx=0x7f52880042f0 comp rx=0 tx=0).stop 2024-09-12T02:03:17.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.266+0000 7f52994c0640 1 -- 172.21.15.16:0/615219223 shutdown_connections 2024-09-12T02:03:17.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.266+0000 7f52994c0640 1 --2- 172.21.15.16:0/615219223 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f527c0a2b80 0x7f527c0a5040 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:17.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.266+0000 7f52994c0640 1 --2- 172.21.15.16:0/615219223 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5294106f10 0x7f529419cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:17.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.266+0000 7f52994c0640 1 --2- 172.21.15.16:0/615219223 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5294102790 0x7f52941989d0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:17.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.266+0000 7f52994c0640 1 --2- 172.21.15.16:0/615219223 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5294101de0 0x7f5294198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:17.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.267+0000 7f52994c0640 1 -- 172.21.15.16:0/615219223 >> 172.21.15.16:0/615219223 conn(0x7f52940fd8c0 msgr2=0x7f52940fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:17.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.267+0000 7f52994c0640 1 -- 172.21.15.16:0/615219223 shutdown_connections 2024-09-12T02:03:17.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:17.267+0000 7f52994c0640 1 -- 172.21.15.16:0/615219223 wait complete. 2024-09-12T02:03:17.568 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:03:17 smithi016 ceph-mon[30947]: pgmap v376: 97 pgs: 97 active+clean; 833 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-12T02:03:17.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:03:17 smithi184 ceph-mon[35593]: pgmap v376: 97 pgs: 97 active+clean; 833 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-12T02:03:17.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:03:17 smithi033 ceph-mon[35332]: pgmap v376: 97 pgs: 97 active+clean; 833 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-12T02:03:17.964 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-12T02:03:17.964 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:03:18.261 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:03:18.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:18.713+0000 7f815d06f640 1 -- 172.21.15.16:0/793942730 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f815810c180 msgr2=0x7f815810e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:18.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:18.713+0000 7f815d06f640 1 --2- 172.21.15.16:0/793942730 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f815810c180 0x7f815810e600 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f8150009ca0 tx=0x7f815002a950 comp rx=0 tx=0).stop 2024-09-12T02:03:18.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:18.713+0000 7f815d06f640 1 -- 172.21.15.16:0/793942730 shutdown_connections 2024-09-12T02:03:18.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:18.713+0000 7f815d06f640 1 --2- 172.21.15.16:0/793942730 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f815810c180 0x7f815810e600 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:18.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:18.713+0000 7f815d06f640 1 --2- 172.21.15.16:0/793942730 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f81581036c0 0x7f815810bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:18.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:18.713+0000 7f815d06f640 1 --2- 172.21.15.16:0/793942730 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8158102da0 0x7f8158103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:18.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:18.713+0000 7f815d06f640 1 -- 172.21.15.16:0/793942730 >> 172.21.15.16:0/793942730 conn(0x7f81580fc7a0 msgr2=0x7f81580febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:18.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:18.713+0000 7f815d06f640 1 -- 172.21.15.16:0/793942730 shutdown_connections 2024-09-12T02:03:18.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:18.713+0000 7f815d06f640 1 -- 172.21.15.16:0/793942730 wait complete. 2024-09-12T02:03:18.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:18.714+0000 7f815d06f640 1 Processor -- start 2024-09-12T02:03:18.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:18.714+0000 7f815d06f640 1 -- start start 2024-09-12T02:03:18.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:18.714+0000 7f815d06f640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8158102da0 0x7f8158100a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:18.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:18.714+0000 7f815d06f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81581036c0 0x7f8158100f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:18.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:18.714+0000 7f815d06f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f815810c180 0x7f8158101480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:18.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:18.714+0000 7f815d06f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8158111f40 con 0x7f815810c180 2024-09-12T02:03:18.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:18.714+0000 7f815d06f640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8158111dc0 con 0x7f81581036c0 2024-09-12T02:03:18.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:18.714+0000 7f815d06f640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f81581120c0 con 0x7f8158102da0 2024-09-12T02:03:18.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:18.715+0000 7f815759e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f815810c180 0x7f8158101480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:18.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:18.715+0000 7f815759e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f815810c180 0x7f8158101480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49734/0 (socket says 172.21.15.16:49734) 2024-09-12T02:03:18.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:18.715+0000 7f815759e640 1 -- 172.21.15.16:0/1562465656 learned_addr learned my addr 172.21.15.16:0/1562465656 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:03:18.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:18.715+0000 7f814ffff640 1 --2- 172.21.15.16:0/1562465656 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81581036c0 0x7f8158100f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:18.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:18.715+0000 7f8156d9d640 1 --2- 172.21.15.16:0/1562465656 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8158102da0 0x7f8158100a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:18.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:18.715+0000 7f815759e640 1 -- 172.21.15.16:0/1562465656 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8158102da0 msgr2=0x7f8158100a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:18.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:18.715+0000 7f815759e640 1 --2- 172.21.15.16:0/1562465656 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8158102da0 0x7f8158100a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:18.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:18.715+0000 7f815759e640 1 -- 172.21.15.16:0/1562465656 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81581036c0 msgr2=0x7f8158100f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:18.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:18.715+0000 7f815759e640 1 --2- 172.21.15.16:0/1562465656 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81581036c0 0x7f8158100f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:18.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:18.715+0000 7f815759e640 1 -- 172.21.15.16:0/1562465656 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81581a6880 con 0x7f815810c180 2024-09-12T02:03:18.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:18.715+0000 7f815759e640 1 --2- 172.21.15.16:0/1562465656 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f815810c180 0x7f8158101480 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f8150006010 tx=0x7f815002f710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:18.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:18.715+0000 7f8154d99640 1 -- 172.21.15.16:0/1562465656 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8150034040 con 0x7f815810c180 2024-09-12T02:03:18.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:18.715+0000 7f815d06f640 1 -- 172.21.15.16:0/1562465656 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81581a6b10 con 0x7f815810c180 2024-09-12T02:03:18.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:18.715+0000 7f8154d99640 1 -- 172.21.15.16:0/1562465656 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8150003e00 con 0x7f815810c180 2024-09-12T02:03:18.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:18.715+0000 7f8154d99640 1 -- 172.21.15.16:0/1562465656 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81500456e0 con 0x7f815810c180 2024-09-12T02:03:18.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:18.715+0000 7f815d06f640 1 -- 172.21.15.16:0/1562465656 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81581a6ff0 con 0x7f815810c180 2024-09-12T02:03:18.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:18.716+0000 7f815d06f640 1 -- 172.21.15.16:0/1562465656 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8158103180 con 0x7f815810c180 2024-09-12T02:03:18.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:18.718+0000 7f8154d99640 1 -- 172.21.15.16:0/1562465656 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f8150037020 con 0x7f815810c180 2024-09-12T02:03:18.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:18.718+0000 7f8154d99640 1 --2- 172.21.15.16:0/1562465656 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f81440a2b80 0x7f81440a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:18.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:18.718+0000 7f8154d99640 1 -- 172.21.15.16:0/1562465656 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f81500f0160 con 0x7f815810c180 2024-09-12T02:03:18.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:18.718+0000 7f8156d9d640 1 --2- 172.21.15.16:0/1562465656 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f81440a2b80 0x7f81440a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:18.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:18.719+0000 7f8156d9d640 1 --2- 172.21.15.16:0/1562465656 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f81440a2b80 0x7f81440a5040 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f814800ba40 tx=0x7f8148005cc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:18.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:18.722+0000 7f8154d99640 1 -- 172.21.15.16:0/1562465656 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f81500bd0c0 con 0x7f815810c180 2024-09-12T02:03:18.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:18.933+0000 7f815d06f640 1 -- 172.21.15.16:0/1562465656 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f81581a72e0 con 0x7f815810c180 2024-09-12T02:03:18.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:18.934+0000 7f8154d99640 1 -- 172.21.15.16:0/1562465656 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11911 (secure 0 0 0) 0x7f81500bc9c0 con 0x7f815810c180 2024-09-12T02:03:18.934 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T02:03:18.934 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":25,"btime":"2024-09-12T02:01:37:192021+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3279123279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3279123279},{"type":"v1","addr":"172.21.15.16:6835","nonce":3279123279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4119452382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4119452382},{"type":"v1","addr":"172.21.15.16:6837","nonce":4119452382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/3263823165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3263823165},{"type":"v1","addr":"172.21.15.16:6839","nonce":3263823165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/1510290732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":1510290732},{"type":"v1","addr":"172.21.15.16:6841","nonce":1510290732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/279674603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":279674603},{"type":"v1","addr":"172.21.15.33:6833","nonce":279674603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3555728028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3555728028},{"type":"v1","addr":"172.21.15.33:6835","nonce":3555728028}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T01:59:56.117608+0000","modified":"2024-09-12T02:01:36.294760+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24482,"mds_1":24481,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6833/1563374106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1563374106},{"type":"v1","addr":"172.21.15.184:6833","nonce":1563374106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6837/1892833868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1892833868},{"type":"v1","addr":"172.21.15.33:6837","nonce":1892833868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/1457679686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1457679686},{"type":"v1","addr":"172.21.15.184:6835","nonce":1457679686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.184:6837/1793393087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":1793393087},{"type":"v1","addr":"172.21.15.184:6837","nonce":1793393087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.184:6839/4203347941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":4203347941},{"type":"v1","addr":"172.21.15.184:6839","nonce":4203347941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.33:6839/4136670204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":4136670204},{"type":"v1","addr":"172.21.15.33:6839","nonce":4136670204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24481,24482,24469]},"id":1}]} 2024-09-12T02:03:18.935 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 25 2024-09-12T02:03:18.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:18.937+0000 7f815d06f640 1 -- 172.21.15.16:0/1562465656 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f81440a2b80 msgr2=0x7f81440a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:18.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:18.937+0000 7f815d06f640 1 --2- 172.21.15.16:0/1562465656 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f81440a2b80 0x7f81440a5040 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f814800ba40 tx=0x7f8148005cc0 comp rx=0 tx=0).stop 2024-09-12T02:03:18.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:18.937+0000 7f815d06f640 1 -- 172.21.15.16:0/1562465656 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f815810c180 msgr2=0x7f8158101480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:18.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:18.937+0000 7f815d06f640 1 --2- 172.21.15.16:0/1562465656 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f815810c180 0x7f8158101480 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f8150006010 tx=0x7f815002f710 comp rx=0 tx=0).stop 2024-09-12T02:03:18.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:18.937+0000 7f815d06f640 1 -- 172.21.15.16:0/1562465656 shutdown_connections 2024-09-12T02:03:18.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:18.937+0000 7f815d06f640 1 --2- 172.21.15.16:0/1562465656 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f81440a2b80 0x7f81440a5040 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:18.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:18.937+0000 7f815d06f640 1 --2- 172.21.15.16:0/1562465656 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f815810c180 0x7f8158101480 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:18.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:18.937+0000 7f815d06f640 1 --2- 172.21.15.16:0/1562465656 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81581036c0 0x7f8158100f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:18.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:18.937+0000 7f815d06f640 1 --2- 172.21.15.16:0/1562465656 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8158102da0 0x7f8158100a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:18.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:18.937+0000 7f815d06f640 1 -- 172.21.15.16:0/1562465656 >> 172.21.15.16:0/1562465656 conn(0x7f81580fc7a0 msgr2=0x7f81580fde10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:18.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:18.937+0000 7f815d06f640 1 -- 172.21.15.16:0/1562465656 shutdown_connections 2024-09-12T02:03:18.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:18.937+0000 7f815d06f640 1 -- 172.21.15.16:0/1562465656 wait complete. 2024-09-12T02:03:19.510 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:03:19 smithi016 ceph-mon[30947]: pgmap v377: 97 pgs: 97 active+clean; 833 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-12T02:03:19.511 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:03:19 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/1562465656' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:03:19.542 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-12T02:03:19.801 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:03:19.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:03:19 smithi184 ceph-mon[35593]: pgmap v377: 97 pgs: 97 active+clean; 833 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-12T02:03:19.878 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:03:19 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/1562465656' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:03:19.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:03:19 smithi033 ceph-mon[35332]: pgmap v377: 97 pgs: 97 active+clean; 833 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-12T02:03:19.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:03:19 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/1562465656' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:03:20.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.303+0000 7f08d18af640 1 -- 172.21.15.16:0/3856786353 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08cc106f10 msgr2=0x7f08cc10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:20.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.303+0000 7f08d18af640 1 --2- 172.21.15.16:0/3856786353 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08cc106f10 0x7f08cc10ea90 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f08c4009ca0 tx=0x7f08c402a8c0 comp rx=0 tx=0).stop 2024-09-12T02:03:20.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.303+0000 7f08d18af640 1 -- 172.21.15.16:0/3856786353 shutdown_connections 2024-09-12T02:03:20.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.303+0000 7f08d18af640 1 --2- 172.21.15.16:0/3856786353 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08cc106f10 0x7f08cc10ea90 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:20.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.303+0000 7f08d18af640 1 --2- 172.21.15.16:0/3856786353 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f08cc102790 0x7f08cc1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:20.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.303+0000 7f08d18af640 1 --2- 172.21.15.16:0/3856786353 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08cc101de0 0x7f08cc1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:20.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.303+0000 7f08d18af640 1 -- 172.21.15.16:0/3856786353 >> 172.21.15.16:0/3856786353 conn(0x7f08cc0fd8c0 msgr2=0x7f08cc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:20.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.303+0000 7f08d18af640 1 -- 172.21.15.16:0/3856786353 shutdown_connections 2024-09-12T02:03:20.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.303+0000 7f08d18af640 1 -- 172.21.15.16:0/3856786353 wait complete. 2024-09-12T02:03:20.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.304+0000 7f08d18af640 1 Processor -- start 2024-09-12T02:03:20.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.304+0000 7f08d18af640 1 -- start start 2024-09-12T02:03:20.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.304+0000 7f08d18af640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08cc101de0 0x7f08cc0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:20.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.304+0000 7f08d18af640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08cc102790 0x7f08cc074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:20.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.304+0000 7f08d18af640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f08cc106f10 0x7f08cc074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:20.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.304+0000 7f08d18af640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f08cc111f70 con 0x7f08cc101de0 2024-09-12T02:03:20.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.304+0000 7f08d18af640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f08cc111df0 con 0x7f08cc102790 2024-09-12T02:03:20.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.304+0000 7f08d18af640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f08cc1120f0 con 0x7f08cc106f10 2024-09-12T02:03:20.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.304+0000 7f08caffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08cc101de0 0x7f08cc0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:20.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.304+0000 7f08caffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08cc101de0 0x7f08cc0744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49754/0 (socket says 172.21.15.16:49754) 2024-09-12T02:03:20.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.304+0000 7f08caffd640 1 -- 172.21.15.16:0/675657178 learned_addr learned my addr 172.21.15.16:0/675657178 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:03:20.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.304+0000 7f08ca7fc640 1 --2- 172.21.15.16:0/675657178 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08cc102790 0x7f08cc074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:20.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.305+0000 7f08caffd640 1 -- 172.21.15.16:0/675657178 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f08cc106f10 msgr2=0x7f08cc074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:20.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.305+0000 7f08caffd640 1 --2- 172.21.15.16:0/675657178 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f08cc106f10 0x7f08cc074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:20.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.305+0000 7f08caffd640 1 -- 172.21.15.16:0/675657178 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08cc102790 msgr2=0x7f08cc074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:20.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.305+0000 7f08caffd640 1 --2- 172.21.15.16:0/675657178 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08cc102790 0x7f08cc074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:20.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.305+0000 7f08caffd640 1 -- 172.21.15.16:0/675657178 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08cc1a6880 con 0x7f08cc101de0 2024-09-12T02:03:20.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.305+0000 7f08caffd640 1 --2- 172.21.15.16:0/675657178 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08cc101de0 0x7f08cc0744f0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f08bc00e9f0 tx=0x7f08bc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:20.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.305+0000 7f08d08ad640 1 -- 172.21.15.16:0/675657178 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08bc00cd80 con 0x7f08cc101de0 2024-09-12T02:03:20.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.305+0000 7f08d08ad640 1 -- 172.21.15.16:0/675657178 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08bc010910 con 0x7f08cc101de0 2024-09-12T02:03:20.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.305+0000 7f08d08ad640 1 -- 172.21.15.16:0/675657178 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08bc018990 con 0x7f08cc101de0 2024-09-12T02:03:20.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.305+0000 7f08d18af640 1 -- 172.21.15.16:0/675657178 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08cc1a6b70 con 0x7f08cc101de0 2024-09-12T02:03:20.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.305+0000 7f08d18af640 1 -- 172.21.15.16:0/675657178 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08cc1a7100 con 0x7f08cc101de0 2024-09-12T02:03:20.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.307+0000 7f08d08ad640 1 -- 172.21.15.16:0/675657178 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f08bc010ac0 con 0x7f08cc101de0 2024-09-12T02:03:20.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.308+0000 7f08d3c88740 1 Processor -- start 2024-09-12T02:03:20.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.308+0000 7f08d3c88740 1 -- start start 2024-09-12T02:03:20.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.308+0000 7f08d3c88740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d267a8db40 0x55d267a8c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:20.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.308+0000 7f08d3c88740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d267a8c5d0 0x55d267a8ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:20.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.308+0000 7f08d3c88740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d267a97370 0x55d267a99760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:20.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.308+0000 7f08d3c88740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55d2679f22f0 con 0x55d267a8c5d0 2024-09-12T02:03:20.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.308+0000 7f08d3c88740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55d267999cf0 con 0x55d267a8db40 2024-09-12T02:03:20.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.308+0000 7f08d3c88740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55d2679ee2b0 con 0x55d267a97370 2024-09-12T02:03:20.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.308+0000 7f08caffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d267a8c5d0 0x55d267a8ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:20.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.308+0000 7f08caffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d267a8c5d0 0x55d267a8ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49768/0 (socket says 172.21.15.16:49768) 2024-09-12T02:03:20.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.308+0000 7f08caffd640 1 -- 172.21.15.16:0/2308487005 learned_addr learned my addr 172.21.15.16:0/2308487005 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:03:20.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.308+0000 7f08d08ad640 1 --2- 172.21.15.16:0/675657178 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f08b40a2c00 0x7f08b40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:20.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.308+0000 7f08ca7fc640 1 --2- 172.21.15.16:0/2308487005 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d267a97370 0x55d267a99760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:20.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.308+0000 7f08cb7fe640 1 --2- 172.21.15.16:0/2308487005 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d267a8db40 0x55d267a8c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:20.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.308+0000 7f08cb7fe640 1 --2- 172.21.15.16:0/675657178 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f08b40a2c00 0x7f08b40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:20.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.309+0000 7f08caffd640 1 -- 172.21.15.16:0/2308487005 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d267a97370 msgr2=0x55d267a99760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:20.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.309+0000 7f08caffd640 1 --2- 172.21.15.16:0/2308487005 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d267a97370 0x55d267a99760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:20.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.309+0000 7f08caffd640 1 -- 172.21.15.16:0/2308487005 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d267a8db40 msgr2=0x55d267a8c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:20.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.309+0000 7f08caffd640 1 --2- 172.21.15.16:0/2308487005 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d267a8db40 0x55d267a8c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:20.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.309+0000 7f08d08ad640 1 -- 172.21.15.16:0/675657178 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f08bc0cf350 con 0x7f08cc101de0 2024-09-12T02:03:20.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.309+0000 7f08caffd640 1 -- 172.21.15.16:0/2308487005 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d26797e3a0 con 0x55d267a8c5d0 2024-09-12T02:03:20.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.309+0000 7f08ca7fc640 1 --2- 172.21.15.16:0/2308487005 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d267a97370 0x55d267a99760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:03:20.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.309+0000 7f08caffd640 1 --2- 172.21.15.16:0/2308487005 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d267a8c5d0 0x55d267a8ca10 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f08bc0cce50 tx=0x7f08bc018420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae4bcf5b2d084147 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:20.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.309+0000 7f08b1ffb640 1 -- 172.21.15.16:0/2308487005 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08bc09c2b0 con 0x55d267a8c5d0 2024-09-12T02:03:20.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.309+0000 7f08cb7fe640 1 --2- 172.21.15.16:0/675657178 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f08b40a2c00 0x7f08b40a50c0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f08c4005e30 tx=0x7f08c40095c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:20.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.309+0000 7f08b1ffb640 1 -- 172.21.15.16:0/2308487005 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08bc09bbb0 con 0x55d267a8c5d0 2024-09-12T02:03:20.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.309+0000 7f08b1ffb640 1 -- 172.21.15.16:0/2308487005 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08bc0a4390 con 0x55d267a8c5d0 2024-09-12T02:03:20.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.310+0000 7f08d3c88740 1 -- 172.21.15.16:0/2308487005 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d267a8c5d0 msgr2=0x55d267a8ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:20.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.310+0000 7f08d3c88740 1 --2- 172.21.15.16:0/2308487005 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d267a8c5d0 0x55d267a8ca10 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f08bc0cce50 tx=0x7f08bc018420 comp rx=0 tx=0).stop 2024-09-12T02:03:20.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.310+0000 7f08d3c88740 1 -- 172.21.15.16:0/2308487005 shutdown_connections 2024-09-12T02:03:20.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.310+0000 7f08d3c88740 1 --2- 172.21.15.16:0/2308487005 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d267a97370 0x55d267a99760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:20.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.310+0000 7f08d3c88740 1 --2- 172.21.15.16:0/2308487005 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d267a8c5d0 0x55d267a8ca10 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:20.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.310+0000 7f08d3c88740 1 --2- 172.21.15.16:0/2308487005 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d267a8db40 0x55d267a8c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:20.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.310+0000 7f08d3c88740 1 -- 172.21.15.16:0/2308487005 >> 172.21.15.16:0/2308487005 conn(0x55d267a55310 msgr2=0x55d267a88c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:20.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.310+0000 7f08d3c88740 1 -- 172.21.15.16:0/2308487005 shutdown_connections 2024-09-12T02:03:20.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.310+0000 7f08d3c88740 1 -- 172.21.15.16:0/2308487005 wait complete. 2024-09-12T02:03:20.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.310+0000 7f08d3c88740 1 -- start start 2024-09-12T02:03:20.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.311+0000 7f08d3c88740 1 Processor -- start 2024-09-12T02:03:20.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.311+0000 7f08d3c88740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d267a8c5d0 0x55d267a96d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:20.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.311+0000 7f08d3c88740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d267a8db40 0x55d267a93fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:20.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.311+0000 7f08d3c88740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d267a97370 0x55d267a94530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:20.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.311+0000 7f08d3c88740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55d267a9a010 con 0x55d267a8db40 2024-09-12T02:03:20.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.311+0000 7f08d3c88740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55d2679ad060 con 0x55d267a8c5d0 2024-09-12T02:03:20.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.311+0000 7f08d3c88740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55d267a9a190 con 0x55d267a97370 2024-09-12T02:03:20.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.311+0000 7f08d3c88740 10 client.0 _ll_register_callbacks cb 0x55d2677008c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:03:20.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.311+0000 7f08ca7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d267a8db40 0x55d267a93fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:20.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.311+0000 7f08ca7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d267a8db40 0x55d267a93fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49782/0 (socket says 172.21.15.16:49782) 2024-09-12T02:03:20.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.311+0000 7f08ca7fc640 1 -- 172.21.15.16:0/2555952231 learned_addr learned my addr 172.21.15.16:0/2555952231 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:03:20.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.311+0000 7f08cb7fe640 1 --2- 172.21.15.16:0/2555952231 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d267a97370 0x55d267a94530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:20.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.311+0000 7f08caffd640 1 --2- 172.21.15.16:0/2555952231 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d267a8c5d0 0x55d267a96d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:20.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.311+0000 7f08ca7fc640 1 -- 172.21.15.16:0/2555952231 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d267a97370 msgr2=0x55d267a94530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:20.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.311+0000 7f08ca7fc640 1 --2- 172.21.15.16:0/2555952231 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d267a97370 0x55d267a94530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:20.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.311+0000 7f08ca7fc640 1 -- 172.21.15.16:0/2555952231 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d267a8c5d0 msgr2=0x55d267a96d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:20.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.311+0000 7f08ca7fc640 1 --2- 172.21.15.16:0/2555952231 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d267a8c5d0 0x55d267a96d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:20.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.312+0000 7f08ca7fc640 1 -- 172.21.15.16:0/2555952231 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2677633c0 con 0x55d267a8db40 2024-09-12T02:03:20.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.312+0000 7f08aaffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:03:20.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.312+0000 7f08ca7fc640 1 --2- 172.21.15.16:0/2555952231 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d267a8db40 0x55d267a93fb0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f08c000b9c0 tx=0x7f08c000be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51c4dc8ddfc5c585 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:20.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.312+0000 7f08aaffd640 1 -- 172.21.15.16:0/2555952231 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08c001b440 con 0x55d267a8db40 2024-09-12T02:03:20.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.312+0000 7f08aaffd640 1 -- 172.21.15.16:0/2555952231 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f08c0011440 con 0x55d267a8db40 2024-09-12T02:03:20.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.312+0000 7f08d3c88740 1 -- 172.21.15.16:0/2555952231 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f08bc09bbb0 con 0x55d267a8db40 2024-09-12T02:03:20.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.312+0000 7f08aaffd640 1 -- 172.21.15.16:0/2555952231 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f08c0011640 con 0x55d267a8db40 2024-09-12T02:03:20.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.312+0000 7f08d3c88740 10 client.15738 fetch_fsmap learned FSMap version 25 2024-09-12T02:03:20.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.312+0000 7f08d3c88740 1 -- 172.21.15.16:0/2555952231 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d267aa5b00 con 0x55d267a8db40 2024-09-12T02:03:20.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.312+0000 7f08aaffd640 1 -- 172.21.15.16:0/2555952231 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08bc09bbb0 con 0x55d267a8db40 2024-09-12T02:03:20.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.313+0000 7f08aaffd640 1 -- 172.21.15.16:0/2555952231 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f08c0019020 con 0x55d267a8db40 2024-09-12T02:03:20.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.313+0000 7f08d3c88740 10 client.15738 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:03:20.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.313+0000 7f08d3c88740 10 client.15738 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-12T02:03:20.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.313+0000 7f08d3c88740 20 client.15738 populate_metadata read hostname 'smithi016' 2024-09-12T02:03:20.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.313+0000 7f08d3c88740 1 --2- 172.21.15.16:0/2555952231 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55d267aa7010 0x55d267aa9400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:20.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.313+0000 7f08d3c88740 4 client.15738 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:03:20.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.313+0000 7f08d3c88740 1 -- 172.21.15.16:0/2555952231 --> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d2679ff860 con 0x55d267aa7010 2024-09-12T02:03:20.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.313+0000 7f08caffd640 1 --2- 172.21.15.16:0/2555952231 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55d267aa7010 0x55d267aa9400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:20.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.313+0000 7f08caffd640 1 --2- 172.21.15.16:0/2555952231 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55d267aa7010 0x55d267aa9400 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f9cfe7cb19b9f5b5 server_cookie=87832c5c4ba03d23 in_seq=0 out_seq=0 2024-09-12T02:03:20.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.313+0000 7f08aaffd640 10 client.15738 ms_handle_connect on v2:172.21.15.184:6836/1793393087 2024-09-12T02:03:20.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.314+0000 7f08aaffd640 1 -- 172.21.15.16:0/2555952231 <== mds.2 v2:172.21.15.184:6836/1793393087 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d2679ff860 con 0x55d267aa7010 2024-09-12T02:03:20.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.314+0000 7f08aaffd640 10 client.15738 handle_command_reply: tid=0 2024-09-12T02:03:20.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.314+0000 7f08d3c88740 1 client.15738 shutdown 2024-09-12T02:03:20.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.314+0000 7f08d3c88740 1 -- 172.21.15.16:0/2555952231 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d267a8db40 msgr2=0x55d267a93fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:20.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.314+0000 7f08d3c88740 1 --2- 172.21.15.16:0/2555952231 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d267a8db40 0x55d267a93fb0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f08c000b9c0 tx=0x7f08c000be80 comp rx=0 tx=0).stop 2024-09-12T02:03:20.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.314+0000 7f08d3c88740 1 -- 172.21.15.16:0/2555952231 shutdown_connections 2024-09-12T02:03:20.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.314+0000 7f08d3c88740 1 --2- 172.21.15.16:0/2555952231 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d267a97370 0x55d267a94530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:20.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.314+0000 7f08d3c88740 1 --2- 172.21.15.16:0/2555952231 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55d267aa7010 0x55d267aa9400 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:20.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.314+0000 7f08d3c88740 1 --2- 172.21.15.16:0/2555952231 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d267a8db40 0x55d267a93fb0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:20.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.314+0000 7f08d3c88740 1 --2- 172.21.15.16:0/2555952231 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d267a8c5d0 0x55d267a96d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:20.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.314+0000 7f08d3c88740 1 -- 172.21.15.16:0/2555952231 >> 172.21.15.16:0/2555952231 conn(0x55d267a55310 msgr2=0x55d267a8e990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:20.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.314+0000 7f08aaffd640 0 client.15738 ms_handle_reset on v2:172.21.15.184:6836/1793393087 2024-09-12T02:03:20.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.314+0000 7f08d3c88740 1 -- 172.21.15.16:0/2555952231 shutdown_connections 2024-09-12T02:03:20.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.315+0000 7f08d3c88740 1 -- 172.21.15.16:0/2555952231 wait complete. 2024-09-12T02:03:20.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.315+0000 7f08d3c88740 20 client.15738 trim_cache size 0 max 16384 2024-09-12T02:03:20.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.327+0000 7f08d3c88740 1 Processor -- start 2024-09-12T02:03:20.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.328+0000 7f08d3c88740 1 -- start start 2024-09-12T02:03:20.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.328+0000 7f08d3c88740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d267a8c5d0 0x55d267a88e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:20.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.328+0000 7f08d3c88740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d267a8db40 0x55d267aa3310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:20.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.328+0000 7f08d3c88740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d267a97370 0x55d267a8f180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:20.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.328+0000 7f08d3c88740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f08cc1b1ff0 con 0x55d267a97370 2024-09-12T02:03:20.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.328+0000 7f08d3c88740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f08cc1af8c0 con 0x55d267a8db40 2024-09-12T02:03:20.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.328+0000 7f08d3c88740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f08cc1ad710 con 0x55d267a8c5d0 2024-09-12T02:03:20.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.328+0000 7f08cb7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d267a97370 0x55d267a8f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:20.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.328+0000 7f08cb7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d267a97370 0x55d267a8f180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49786/0 (socket says 172.21.15.16:49786) 2024-09-12T02:03:20.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.328+0000 7f08cb7fe640 1 -- 172.21.15.16:0/1472721023 learned_addr learned my addr 172.21.15.16:0/1472721023 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:03:20.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.328+0000 7f08ca7fc640 1 --2- 172.21.15.16:0/1472721023 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d267a8db40 0x55d267aa3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:20.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.328+0000 7f08cb7fe640 1 -- 172.21.15.16:0/1472721023 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d267a8c5d0 msgr2=0x55d267a88e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:20.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.328+0000 7f08cb7fe640 1 --2- 172.21.15.16:0/1472721023 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d267a8c5d0 0x55d267a88e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:20.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.328+0000 7f08cb7fe640 1 -- 172.21.15.16:0/1472721023 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d267a8db40 msgr2=0x55d267aa3310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:20.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.328+0000 7f08cb7fe640 1 --2- 172.21.15.16:0/1472721023 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d267a8db40 0x55d267aa3310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:20.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.328+0000 7f08cb7fe640 1 -- 172.21.15.16:0/1472721023 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d267a92f10 con 0x55d267a97370 2024-09-12T02:03:20.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.328+0000 7f08cb7fe640 1 --2- 172.21.15.16:0/1472721023 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d267a97370 0x55d267a8f180 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f08c4035000 tx=0x7f08c4041000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da078ef8e45d35c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:20.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.329+0000 7f08b17fa640 1 -- 172.21.15.16:0/1472721023 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08c4031650 con 0x55d267a97370 2024-09-12T02:03:20.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.329+0000 7f08b17fa640 1 -- 172.21.15.16:0/1472721023 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08c404f070 con 0x55d267a97370 2024-09-12T02:03:20.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.329+0000 7f08b17fa640 1 -- 172.21.15.16:0/1472721023 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08c402f8e0 con 0x55d267a97370 2024-09-12T02:03:20.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.329+0000 7f08d3c88740 1 -- 172.21.15.16:0/1472721023 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d267a97370 msgr2=0x55d267a8f180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:20.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.329+0000 7f08d3c88740 1 --2- 172.21.15.16:0/1472721023 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d267a97370 0x55d267a8f180 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f08c4035000 tx=0x7f08c4041000 comp rx=0 tx=0).stop 2024-09-12T02:03:20.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.329+0000 7f08d3c88740 1 -- 172.21.15.16:0/1472721023 shutdown_connections 2024-09-12T02:03:20.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.329+0000 7f08d3c88740 1 --2- 172.21.15.16:0/1472721023 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d267a97370 0x55d267a8f180 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:20.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.329+0000 7f08d3c88740 1 --2- 172.21.15.16:0/1472721023 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d267a8db40 0x55d267aa3310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:20.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.329+0000 7f08d3c88740 1 --2- 172.21.15.16:0/1472721023 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d267a8c5d0 0x55d267a88e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:20.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.329+0000 7f08d3c88740 1 -- 172.21.15.16:0/1472721023 >> 172.21.15.16:0/1472721023 conn(0x55d267a55310 msgr2=0x55d267a88700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:20.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.329+0000 7f08d3c88740 1 -- 172.21.15.16:0/1472721023 shutdown_connections 2024-09-12T02:03:20.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.329+0000 7f08d3c88740 1 -- 172.21.15.16:0/1472721023 wait complete. 2024-09-12T02:03:20.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.330+0000 7f08d3c88740 1 -- start start 2024-09-12T02:03:20.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.330+0000 7f08d3c88740 1 Processor -- start 2024-09-12T02:03:20.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.330+0000 7f08d3c88740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d267a8c5d0 0x55d267a9fac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:20.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.330+0000 7f08d3c88740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d267a8db40 0x55d267a90c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:20.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.330+0000 7f08d3c88740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d267a97370 0x55d267a91180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:20.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.330+0000 7f08d3c88740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f08cc1add80 con 0x55d267a8db40 2024-09-12T02:03:20.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.330+0000 7f08d3c88740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f08cc1af1f0 con 0x55d267a8c5d0 2024-09-12T02:03:20.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.330+0000 7f08d3c88740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f08cc1ae450 con 0x55d267a97370 2024-09-12T02:03:20.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.330+0000 7f08d3c88740 10 client.0 _ll_register_callbacks cb 0x7f08c0004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:03:20.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.330+0000 7f08ca7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d267a8db40 0x55d267a90c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:20.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.330+0000 7f08ca7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d267a8db40 0x55d267a90c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49796/0 (socket says 172.21.15.16:49796) 2024-09-12T02:03:20.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.330+0000 7f08ca7fc640 1 -- 172.21.15.16:0/1092724367 learned_addr learned my addr 172.21.15.16:0/1092724367 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:03:20.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.330+0000 7f08caffd640 1 --2- 172.21.15.16:0/1092724367 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d267a8c5d0 0x55d267a9fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:20.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.331+0000 7f08ca7fc640 1 -- 172.21.15.16:0/1092724367 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d267a97370 msgr2=0x55d267a91180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:20.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.331+0000 7f08ca7fc640 1 --2- 172.21.15.16:0/1092724367 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d267a97370 0x55d267a91180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:20.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.331+0000 7f08ca7fc640 1 -- 172.21.15.16:0/1092724367 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d267a8c5d0 msgr2=0x55d267a9fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:20.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.331+0000 7f08ca7fc640 1 --2- 172.21.15.16:0/1092724367 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d267a8c5d0 0x55d267a9fac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:20.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.331+0000 7f08ca7fc640 1 -- 172.21.15.16:0/1092724367 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d267a918c0 con 0x55d267a8db40 2024-09-12T02:03:20.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.331+0000 7f08caffd640 1 --2- 172.21.15.16:0/1092724367 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d267a8c5d0 0x55d267a9fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:03:20.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.331+0000 7f08ca7fc640 1 --2- 172.21.15.16:0/1092724367 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d267a8db40 0x55d267a90c40 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f08c001bc80 tx=0x7f08c0005bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1667ba0b8e5a10d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:20.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.331+0000 7f08aaffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:03:20.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.331+0000 7f08aaffd640 1 -- 172.21.15.16:0/1092724367 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08c0005850 con 0x55d267a8db40 2024-09-12T02:03:20.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.331+0000 7f08aaffd640 1 -- 172.21.15.16:0/1092724367 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f08c0004260 con 0x55d267a8db40 2024-09-12T02:03:20.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.331+0000 7f08d3c88740 1 -- 172.21.15.16:0/1092724367 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f08c404f070 con 0x55d267a8db40 2024-09-12T02:03:20.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.331+0000 7f08aaffd640 1 -- 172.21.15.16:0/1092724367 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f08c001b440 con 0x55d267a8db40 2024-09-12T02:03:20.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.331+0000 7f08d3c88740 10 client.15744 fetch_fsmap learned FSMap version 25 2024-09-12T02:03:20.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.331+0000 7f08d3c88740 1 -- 172.21.15.16:0/1092724367 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d267b11c30 con 0x55d267a8db40 2024-09-12T02:03:20.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.331+0000 7f08aaffd640 1 -- 172.21.15.16:0/1092724367 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08c404f070 con 0x55d267a8db40 2024-09-12T02:03:20.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.332+0000 7f08aaffd640 1 -- 172.21.15.16:0/1092724367 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f08c0019050 con 0x55d267a8db40 2024-09-12T02:03:20.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.332+0000 7f08d3c88740 10 client.15744 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:03:20.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.332+0000 7f08d3c88740 10 client.15744 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-12T02:03:20.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.332+0000 7f08d3c88740 20 client.15744 populate_metadata read hostname 'smithi016' 2024-09-12T02:03:20.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.332+0000 7f08d3c88740 1 --2- 172.21.15.16:0/1092724367 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55d267b12350 0x55d267b14770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:20.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.332+0000 7f08d3c88740 4 client.15744 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T02:03:20.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.332+0000 7f08d3c88740 1 -- 172.21.15.16:0/1092724367 --> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d2679f9d70 con 0x55d267b12350 2024-09-12T02:03:20.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.332+0000 7f08caffd640 1 --2- 172.21.15.16:0/1092724367 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55d267b12350 0x55d267b14770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:20.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.332+0000 7f08aaffd640 10 client.15744 ms_handle_connect on v2:172.21.15.184:6836/1793393087 2024-09-12T02:03:20.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.332+0000 7f08caffd640 1 --2- 172.21.15.16:0/1092724367 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55d267b12350 0x55d267b14770 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=425529b4c0f77b5 server_cookie=eeaa1cbc09e4494a in_seq=0 out_seq=0 2024-09-12T02:03:20.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.333+0000 7f08aaffd640 1 -- 172.21.15.16:0/1092724367 <== mds.2 v2:172.21.15.184:6836/1793393087 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d2679f9d70 con 0x55d267b12350 2024-09-12T02:03:20.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.333+0000 7f08aaffd640 10 client.15744 handle_command_reply: tid=0 2024-09-12T02:03:20.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.333+0000 7f08d3c88740 1 client.15744 shutdown 2024-09-12T02:03:20.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.333+0000 7f08d3c88740 1 -- 172.21.15.16:0/1092724367 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d267a8db40 msgr2=0x55d267a90c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:20.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.333+0000 7f08d3c88740 1 --2- 172.21.15.16:0/1092724367 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d267a8db40 0x55d267a90c40 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f08c001bc80 tx=0x7f08c0005bb0 comp rx=0 tx=0).stop 2024-09-12T02:03:20.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.333+0000 7f08d3c88740 1 -- 172.21.15.16:0/1092724367 shutdown_connections 2024-09-12T02:03:20.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.333+0000 7f08d3c88740 1 --2- 172.21.15.16:0/1092724367 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d267a97370 0x55d267a91180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:20.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.333+0000 7f08d3c88740 1 --2- 172.21.15.16:0/1092724367 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55d267b12350 0x55d267b14770 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:20.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.333+0000 7f08d3c88740 1 --2- 172.21.15.16:0/1092724367 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d267a8db40 0x55d267a90c40 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:20.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.333+0000 7f08aaffd640 0 client.15744 ms_handle_reset on v2:172.21.15.184:6836/1793393087 2024-09-12T02:03:20.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.333+0000 7f08d3c88740 1 --2- 172.21.15.16:0/1092724367 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d267a8c5d0 0x55d267a9fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:20.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.333+0000 7f08d3c88740 1 -- 172.21.15.16:0/1092724367 >> 172.21.15.16:0/1092724367 conn(0x55d267a55310 msgr2=0x55d267a95810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:20.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.333+0000 7f08d3c88740 1 -- 172.21.15.16:0/1092724367 shutdown_connections 2024-09-12T02:03:20.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.333+0000 7f08d3c88740 1 -- 172.21.15.16:0/1092724367 wait complete. 2024-09-12T02:03:20.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.333+0000 7f08d3c88740 20 client.15744 trim_cache size 0 max 16384 2024-09-12T02:03:20.338 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T02:03:20.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.334+0000 7f08d18af640 1 -- 172.21.15.16:0/675657178 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f08b40a2c00 msgr2=0x7f08b40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:20.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.334+0000 7f08d18af640 1 --2- 172.21.15.16:0/675657178 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f08b40a2c00 0x7f08b40a50c0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f08c4005e30 tx=0x7f08c40095c0 comp rx=0 tx=0).stop 2024-09-12T02:03:20.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.334+0000 7f08d18af640 1 -- 172.21.15.16:0/675657178 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08cc101de0 msgr2=0x7f08cc0744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:20.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.334+0000 7f08d18af640 1 --2- 172.21.15.16:0/675657178 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08cc101de0 0x7f08cc0744f0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f08bc00e9f0 tx=0x7f08bc00eeb0 comp rx=0 tx=0).stop 2024-09-12T02:03:20.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.334+0000 7f08d18af640 1 -- 172.21.15.16:0/675657178 shutdown_connections 2024-09-12T02:03:20.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.334+0000 7f08d18af640 1 --2- 172.21.15.16:0/675657178 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f08b40a2c00 0x7f08b40a50c0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:20.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.334+0000 7f08d18af640 1 --2- 172.21.15.16:0/675657178 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f08cc106f10 0x7f08cc074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:20.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.334+0000 7f08d18af640 1 --2- 172.21.15.16:0/675657178 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08cc102790 0x7f08cc074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:20.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.334+0000 7f08d18af640 1 --2- 172.21.15.16:0/675657178 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08cc101de0 0x7f08cc0744f0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:20.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.334+0000 7f08d18af640 1 -- 172.21.15.16:0/675657178 >> 172.21.15.16:0/675657178 conn(0x7f08cc0fd8c0 msgr2=0x7f08cc0fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:20.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.334+0000 7f08d18af640 1 -- 172.21.15.16:0/675657178 shutdown_connections 2024-09-12T02:03:20.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:20.334+0000 7f08d18af640 1 -- 172.21.15.16:0/675657178 wait complete. 2024-09-12T02:03:21.083 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-12T02:03:21.338 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:03:21.710 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:03:21 smithi016 ceph-mon[30947]: pgmap v378: 97 pgs: 97 active+clean; 903 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-09-12T02:03:21.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.793+0000 7efc2bd2b640 1 -- 172.21.15.16:0/3172019462 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc2410c170 msgr2=0x7efc2410e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:21.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.793+0000 7efc2bd2b640 1 --2- 172.21.15.16:0/3172019462 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc2410c170 0x7efc2410e610 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7efc1c009ca0 tx=0x7efc1c02a950 comp rx=0 tx=0).stop 2024-09-12T02:03:21.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.793+0000 7efc2bd2b640 1 -- 172.21.15.16:0/3172019462 shutdown_connections 2024-09-12T02:03:21.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.793+0000 7efc2bd2b640 1 --2- 172.21.15.16:0/3172019462 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc2410c170 0x7efc2410e610 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:21.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.793+0000 7efc2bd2b640 1 --2- 172.21.15.16:0/3172019462 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc24103660 0x7efc2410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:21.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.793+0000 7efc2bd2b640 1 --2- 172.21.15.16:0/3172019462 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efc24102d40 0x7efc24103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:21.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.793+0000 7efc2bd2b640 1 -- 172.21.15.16:0/3172019462 >> 172.21.15.16:0/3172019462 conn(0x7efc240fc7a0 msgr2=0x7efc240febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:21.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.793+0000 7efc2bd2b640 1 -- 172.21.15.16:0/3172019462 shutdown_connections 2024-09-12T02:03:21.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.793+0000 7efc2bd2b640 1 -- 172.21.15.16:0/3172019462 wait complete. 2024-09-12T02:03:21.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.794+0000 7efc2bd2b640 1 Processor -- start 2024-09-12T02:03:21.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.794+0000 7efc2bd2b640 1 -- start start 2024-09-12T02:03:21.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.794+0000 7efc2bd2b640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efc24102d40 0x7efc241983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:21.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.794+0000 7efc2bd2b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc24103660 0x7efc24198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:21.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.794+0000 7efc2bd2b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc2410c170 0x7efc2419cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:21.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.795+0000 7efc2bd2b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7efc24111fb0 con 0x7efc24103660 2024-09-12T02:03:21.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.795+0000 7efc2bd2b640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7efc24111e30 con 0x7efc2410c170 2024-09-12T02:03:21.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.795+0000 7efc2bd2b640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7efc24112130 con 0x7efc24102d40 2024-09-12T02:03:21.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.795+0000 7efc292c6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc24103660 0x7efc24198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:21.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.795+0000 7efc292c6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc24103660 0x7efc24198910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52048/0 (socket says 172.21.15.16:52048) 2024-09-12T02:03:21.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.795+0000 7efc292c6640 1 -- 172.21.15.16:0/3277408139 learned_addr learned my addr 172.21.15.16:0/3277408139 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:03:21.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.795+0000 7efc2a2c8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc2410c170 0x7efc2419cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:21.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.795+0000 7efc29ac7640 1 --2- 172.21.15.16:0/3277408139 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efc24102d40 0x7efc241983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:21.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.795+0000 7efc292c6640 1 -- 172.21.15.16:0/3277408139 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efc24102d40 msgr2=0x7efc241983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:21.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.795+0000 7efc292c6640 1 --2- 172.21.15.16:0/3277408139 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efc24102d40 0x7efc241983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:21.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.795+0000 7efc292c6640 1 -- 172.21.15.16:0/3277408139 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc2410c170 msgr2=0x7efc2419cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:21.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.795+0000 7efc292c6640 1 --2- 172.21.15.16:0/3277408139 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc2410c170 0x7efc2419cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:21.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.795+0000 7efc292c6640 1 -- 172.21.15.16:0/3277408139 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc2419d380 con 0x7efc24103660 2024-09-12T02:03:21.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.795+0000 7efc292c6640 1 --2- 172.21.15.16:0/3277408139 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc24103660 0x7efc24198910 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7efc1400c970 tx=0x7efc1400cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:21.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.795+0000 7efc1affd640 1 -- 172.21.15.16:0/3277408139 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc14007bb0 con 0x7efc24103660 2024-09-12T02:03:21.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.795+0000 7efc1affd640 1 -- 172.21.15.16:0/3277408139 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc14007d50 con 0x7efc24103660 2024-09-12T02:03:21.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.795+0000 7efc2bd2b640 1 -- 172.21.15.16:0/3277408139 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc2419d670 con 0x7efc24103660 2024-09-12T02:03:21.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.795+0000 7efc1affd640 1 -- 172.21.15.16:0/3277408139 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc14018730 con 0x7efc24103660 2024-09-12T02:03:21.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.795+0000 7efc2bd2b640 1 -- 172.21.15.16:0/3277408139 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc241a4f50 con 0x7efc24103660 2024-09-12T02:03:21.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.798+0000 7efc1affd640 1 -- 172.21.15.16:0/3277408139 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7efc1400f450 con 0x7efc24103660 2024-09-12T02:03:21.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.798+0000 7efc2e104740 1 Processor -- start 2024-09-12T02:03:21.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.798+0000 7efc2e104740 1 -- start start 2024-09-12T02:03:21.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.798+0000 7efc2e104740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576e7383b40 0x5576e7382090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:21.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.798+0000 7efc2e104740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5576e73825d0 0x5576e7382a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:21.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.798+0000 7efc2e104740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5576e738d370 0x5576e738f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:21.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.798+0000 7efc2e104740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5576e72e82f0 con 0x5576e7383b40 2024-09-12T02:03:21.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.798+0000 7efc2e104740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5576e728fcf0 con 0x5576e738d370 2024-09-12T02:03:21.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.798+0000 7efc2e104740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5576e72e42b0 con 0x5576e73825d0 2024-09-12T02:03:21.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.799+0000 7efc2a2c8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576e7383b40 0x5576e7382090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:21.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.799+0000 7efc2a2c8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576e7383b40 0x5576e7382090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52064/0 (socket says 172.21.15.16:52064) 2024-09-12T02:03:21.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.799+0000 7efc2a2c8640 1 -- 172.21.15.16:0/653943171 learned_addr learned my addr 172.21.15.16:0/653943171 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:03:21.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.799+0000 7efc292c6640 1 --2- 172.21.15.16:0/653943171 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5576e738d370 0x5576e738f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:21.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.799+0000 7efc1affd640 1 --2- 172.21.15.16:0/3277408139 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7efc100a2c00 0x7efc100a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:21.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.799+0000 7efc29ac7640 1 --2- 172.21.15.16:0/653943171 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5576e73825d0 0x5576e7382a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:21.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.799+0000 7efc2a2c8640 1 -- 172.21.15.16:0/653943171 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5576e73825d0 msgr2=0x5576e7382a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:21.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.799+0000 7efc2a2c8640 1 --2- 172.21.15.16:0/653943171 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5576e73825d0 0x5576e7382a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:21.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.799+0000 7efc2a2c8640 1 -- 172.21.15.16:0/653943171 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5576e738d370 msgr2=0x5576e738f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:21.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.799+0000 7efc2a2c8640 1 --2- 172.21.15.16:0/653943171 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5576e738d370 0x5576e738f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:21.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.799+0000 7efc2a2c8640 1 -- 172.21.15.16:0/653943171 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576e72743a0 con 0x5576e7383b40 2024-09-12T02:03:21.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.799+0000 7efc2a2c8640 1 --2- 172.21.15.16:0/3277408139 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7efc100a2c00 0x7efc100a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:21.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.799+0000 7efc1affd640 1 -- 172.21.15.16:0/3277408139 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7efc140ceed0 con 0x7efc24103660 2024-09-12T02:03:21.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.799+0000 7efc2a2c8640 1 --2- 172.21.15.16:0/653943171 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576e7383b40 0x5576e7382090 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7efc1c035000 tx=0x7efc1c030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b027a5b93548e4e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:21.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.799+0000 7efc0ffff640 1 -- 172.21.15.16:0/653943171 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc1c034040 con 0x5576e7383b40 2024-09-12T02:03:21.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.800+0000 7efc0ffff640 1 -- 172.21.15.16:0/653943171 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc1c004520 con 0x5576e7383b40 2024-09-12T02:03:21.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.800+0000 7efc0ffff640 1 -- 172.21.15.16:0/653943171 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc1c0468d0 con 0x5576e7383b40 2024-09-12T02:03:21.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.800+0000 7efc2a2c8640 1 --2- 172.21.15.16:0/3277408139 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7efc100a2c00 0x7efc100a50c0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7efc1c04d000 tx=0x7efc1c04a000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:21.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.800+0000 7efc2e104740 1 -- 172.21.15.16:0/653943171 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576e7383b40 msgr2=0x5576e7382090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:21.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.800+0000 7efc2e104740 1 --2- 172.21.15.16:0/653943171 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576e7383b40 0x5576e7382090 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7efc1c035000 tx=0x7efc1c030000 comp rx=0 tx=0).stop 2024-09-12T02:03:21.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.800+0000 7efc2e104740 1 -- 172.21.15.16:0/653943171 shutdown_connections 2024-09-12T02:03:21.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.800+0000 7efc2e104740 1 --2- 172.21.15.16:0/653943171 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5576e738d370 0x5576e738f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:21.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.800+0000 7efc2e104740 1 --2- 172.21.15.16:0/653943171 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5576e73825d0 0x5576e7382a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:21.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.800+0000 7efc2e104740 1 --2- 172.21.15.16:0/653943171 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576e7383b40 0x5576e7382090 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:21.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.800+0000 7efc2e104740 1 -- 172.21.15.16:0/653943171 >> 172.21.15.16:0/653943171 conn(0x5576e734b310 msgr2=0x5576e737ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:21.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.800+0000 7efc2e104740 1 -- 172.21.15.16:0/653943171 shutdown_connections 2024-09-12T02:03:21.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.801+0000 7efc2e104740 1 -- 172.21.15.16:0/653943171 wait complete. 2024-09-12T02:03:21.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.801+0000 7efc2e104740 1 -- start start 2024-09-12T02:03:21.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.801+0000 7efc2e104740 1 Processor -- start 2024-09-12T02:03:21.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.802+0000 7efc2e104740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5576e73825d0 0x5576e738cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:21.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.802+0000 7efc2e104740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5576e7383b40 0x5576e738a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:21.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.802+0000 7efc2e104740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576e738d370 0x5576e738a640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:21.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.802+0000 7efc2e104740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5576e7390070 con 0x5576e738d370 2024-09-12T02:03:21.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.802+0000 7efc2e104740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5576e72a3060 con 0x5576e73825d0 2024-09-12T02:03:21.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.802+0000 7efc2e104740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5576e73901f0 con 0x5576e7383b40 2024-09-12T02:03:21.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.802+0000 7efc2e104740 10 client.0 _ll_register_callbacks cb 0x5576e6ff68c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:03:21.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.802+0000 7efc2a2c8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576e738d370 0x5576e738a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:21.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.802+0000 7efc2a2c8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576e738d370 0x5576e738a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52078/0 (socket says 172.21.15.16:52078) 2024-09-12T02:03:21.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.802+0000 7efc29ac7640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5576e73825d0 0x5576e738cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:21.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.802+0000 7efc2a2c8640 1 -- 172.21.15.16:0/3479457829 learned_addr learned my addr 172.21.15.16:0/3479457829 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:03:21.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.802+0000 7efc292c6640 1 --2- 172.21.15.16:0/3479457829 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5576e7383b40 0x5576e738a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:21.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.802+0000 7efc2a2c8640 1 -- 172.21.15.16:0/3479457829 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5576e7383b40 msgr2=0x5576e738a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:21.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.802+0000 7efc2a2c8640 1 --2- 172.21.15.16:0/3479457829 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5576e7383b40 0x5576e738a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:21.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.802+0000 7efc2a2c8640 1 -- 172.21.15.16:0/3479457829 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5576e73825d0 msgr2=0x5576e738cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:21.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.802+0000 7efc2a2c8640 1 --2- 172.21.15.16:0/3479457829 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5576e73825d0 0x5576e738cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:21.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.802+0000 7efc2a2c8640 1 -- 172.21.15.16:0/3479457829 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5576e70593c0 con 0x5576e738d370 2024-09-12T02:03:21.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.802+0000 7efc29ac7640 1 --2- 172.21.15.16:0/3479457829 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5576e73825d0 0x5576e738cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:03:21.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.802+0000 7efc2a2c8640 1 --2- 172.21.15.16:0/3479457829 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576e738d370 0x5576e738a640 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7efc1c005c60 tx=0x7efc1c046470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea33ece5453f6b90 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:21.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.802+0000 7efc0d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:03:21.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.803+0000 7efc0d7fa640 1 -- 172.21.15.16:0/3479457829 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc1c046890 con 0x5576e738d370 2024-09-12T02:03:21.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.803+0000 7efc0d7fa640 1 -- 172.21.15.16:0/3479457829 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7efc1c004040 con 0x5576e738d370 2024-09-12T02:03:21.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.803+0000 7efc2e104740 1 -- 172.21.15.16:0/3479457829 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efc1c004520 con 0x5576e738d370 2024-09-12T02:03:21.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.803+0000 7efc0d7fa640 1 -- 172.21.15.16:0/3479457829 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7efc1c0098d0 con 0x5576e738d370 2024-09-12T02:03:21.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.803+0000 7efc2e104740 10 client.15756 fetch_fsmap learned FSMap version 25 2024-09-12T02:03:21.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.803+0000 7efc2e104740 1 -- 172.21.15.16:0/3479457829 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5576e7390070 con 0x5576e738d370 2024-09-12T02:03:21.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.803+0000 7efc0d7fa640 1 -- 172.21.15.16:0/3479457829 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc1c004520 con 0x5576e738d370 2024-09-12T02:03:21.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.803+0000 7efc0d7fa640 1 -- 172.21.15.16:0/3479457829 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7efc1c051440 con 0x5576e738d370 2024-09-12T02:03:21.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.803+0000 7efc2e104740 10 client.15756 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:03:21.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.803+0000 7efc2e104740 10 client.15756 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T02:03:21.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.803+0000 7efc2e104740 20 client.15756 populate_metadata read hostname 'smithi016' 2024-09-12T02:03:21.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.803+0000 7efc2e104740 1 --2- 172.21.15.16:0/3479457829 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x5576e739ccc0 0x5576e739f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:21.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.803+0000 7efc2e104740 4 client.15756 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:03:21.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.803+0000 7efc2e104740 1 -- 172.21.15.16:0/3479457829 --> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5576e72f5860 con 0x5576e739ccc0 2024-09-12T02:03:21.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.804+0000 7efc29ac7640 1 --2- 172.21.15.16:0/3479457829 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x5576e739ccc0 0x5576e739f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:21.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.804+0000 7efc29ac7640 1 --2- 172.21.15.16:0/3479457829 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x5576e739ccc0 0x5576e739f0b0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=73706a33e4fec1 server_cookie=bebbdd81c4aa5e6a in_seq=0 out_seq=0 2024-09-12T02:03:21.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.804+0000 7efc0d7fa640 10 client.15756 ms_handle_connect on v2:172.21.15.184:6838/4203347941 2024-09-12T02:03:21.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.805+0000 7efc0d7fa640 1 -- 172.21.15.16:0/3479457829 <== mds.1 v2:172.21.15.184:6838/4203347941 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5576e72f5860 con 0x5576e739ccc0 2024-09-12T02:03:21.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.805+0000 7efc0d7fa640 10 client.15756 handle_command_reply: tid=0 2024-09-12T02:03:21.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.805+0000 7efc2e104740 1 client.15756 shutdown 2024-09-12T02:03:21.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.805+0000 7efc2e104740 1 -- 172.21.15.16:0/3479457829 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576e738d370 msgr2=0x5576e738a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:21.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.805+0000 7efc2e104740 1 --2- 172.21.15.16:0/3479457829 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576e738d370 0x5576e738a640 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7efc1c005c60 tx=0x7efc1c046470 comp rx=0 tx=0).stop 2024-09-12T02:03:21.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.805+0000 7efc2e104740 1 -- 172.21.15.16:0/3479457829 shutdown_connections 2024-09-12T02:03:21.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.805+0000 7efc2e104740 1 --2- 172.21.15.16:0/3479457829 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x5576e739ccc0 0x5576e739f0b0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:21.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.805+0000 7efc2e104740 1 --2- 172.21.15.16:0/3479457829 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576e738d370 0x5576e738a640 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:21.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.805+0000 7efc2e104740 1 --2- 172.21.15.16:0/3479457829 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5576e7383b40 0x5576e738a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:21.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.805+0000 7efc2e104740 1 --2- 172.21.15.16:0/3479457829 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5576e73825d0 0x5576e738cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:21.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.805+0000 7efc0d7fa640 0 client.15756 ms_handle_reset on v2:172.21.15.184:6838/4203347941 2024-09-12T02:03:21.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.805+0000 7efc2e104740 1 -- 172.21.15.16:0/3479457829 >> 172.21.15.16:0/3479457829 conn(0x5576e734b310 msgr2=0x5576e7384960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:21.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.805+0000 7efc2e104740 1 -- 172.21.15.16:0/3479457829 shutdown_connections 2024-09-12T02:03:21.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.805+0000 7efc2e104740 1 -- 172.21.15.16:0/3479457829 wait complete. 2024-09-12T02:03:21.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.805+0000 7efc2e104740 20 client.15756 trim_cache size 0 max 16384 2024-09-12T02:03:21.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.818+0000 7efc2e104740 1 Processor -- start 2024-09-12T02:03:21.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.818+0000 7efc2e104740 1 -- start start 2024-09-12T02:03:21.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.818+0000 7efc2e104740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5576e73825d0 0x5576e7399310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:21.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.818+0000 7efc2e104740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5576e7383b40 0x5576e738bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:21.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.818+0000 7efc2e104740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576e738d370 0x5576e7385120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:21.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.818+0000 7efc2e104740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7efc241afb80 con 0x5576e738d370 2024-09-12T02:03:21.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.818+0000 7efc2e104740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7efc241ad450 con 0x5576e73825d0 2024-09-12T02:03:21.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.818+0000 7efc2e104740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7efc241ab2a0 con 0x5576e7383b40 2024-09-12T02:03:21.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.819+0000 7efc2a2c8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576e738d370 0x5576e7385120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:21.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.819+0000 7efc2a2c8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576e738d370 0x5576e7385120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52090/0 (socket says 172.21.15.16:52090) 2024-09-12T02:03:21.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.819+0000 7efc2a2c8640 1 -- 172.21.15.16:0/1114765717 learned_addr learned my addr 172.21.15.16:0/1114765717 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:03:21.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.819+0000 7efc29ac7640 1 --2- 172.21.15.16:0/1114765717 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5576e73825d0 0x5576e7399310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:21.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.819+0000 7efc292c6640 1 --2- 172.21.15.16:0/1114765717 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5576e7383b40 0x5576e738bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:21.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.819+0000 7efc2a2c8640 1 -- 172.21.15.16:0/1114765717 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5576e7383b40 msgr2=0x5576e738bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:21.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.819+0000 7efc2a2c8640 1 --2- 172.21.15.16:0/1114765717 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5576e7383b40 0x5576e738bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:21.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.819+0000 7efc2a2c8640 1 -- 172.21.15.16:0/1114765717 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5576e73825d0 msgr2=0x5576e7399310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:21.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.819+0000 7efc2a2c8640 1 --2- 172.21.15.16:0/1114765717 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5576e73825d0 0x5576e7399310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:21.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.819+0000 7efc2a2c8640 1 -- 172.21.15.16:0/1114765717 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576e73822d0 con 0x5576e738d370 2024-09-12T02:03:21.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.819+0000 7efc292c6640 1 --2- 172.21.15.16:0/1114765717 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5576e7383b40 0x5576e738bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:03:21.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.819+0000 7efc29ac7640 1 --2- 172.21.15.16:0/1114765717 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5576e73825d0 0x5576e7399310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:03:21.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.819+0000 7efc2a2c8640 1 --2- 172.21.15.16:0/1114765717 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576e738d370 0x5576e7385120 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7efc1c001220 tx=0x7efc1c030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d717aebe4b476bb0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:21.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.819+0000 7efc0f7fe640 1 -- 172.21.15.16:0/1114765717 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc1c051cf0 con 0x5576e738d370 2024-09-12T02:03:21.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.819+0000 7efc0f7fe640 1 -- 172.21.15.16:0/1114765717 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc1c05a070 con 0x5576e738d370 2024-09-12T02:03:21.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.819+0000 7efc0f7fe640 1 -- 172.21.15.16:0/1114765717 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc1c046880 con 0x5576e738d370 2024-09-12T02:03:21.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.820+0000 7efc2e104740 1 -- 172.21.15.16:0/1114765717 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576e738d370 msgr2=0x5576e7385120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:21.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.820+0000 7efc2e104740 1 --2- 172.21.15.16:0/1114765717 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576e738d370 0x5576e7385120 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7efc1c001220 tx=0x7efc1c030000 comp rx=0 tx=0).stop 2024-09-12T02:03:21.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.820+0000 7efc2e104740 1 -- 172.21.15.16:0/1114765717 shutdown_connections 2024-09-12T02:03:21.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.820+0000 7efc2e104740 1 --2- 172.21.15.16:0/1114765717 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576e738d370 0x5576e7385120 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:21.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.820+0000 7efc2e104740 1 --2- 172.21.15.16:0/1114765717 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5576e7383b40 0x5576e738bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:21.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.820+0000 7efc2e104740 1 --2- 172.21.15.16:0/1114765717 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5576e73825d0 0x5576e7399310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:21.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.820+0000 7efc2e104740 1 -- 172.21.15.16:0/1114765717 >> 172.21.15.16:0/1114765717 conn(0x5576e734b310 msgr2=0x5576e738b890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:21.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.820+0000 7efc2e104740 1 -- 172.21.15.16:0/1114765717 shutdown_connections 2024-09-12T02:03:21.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.820+0000 7efc2e104740 1 -- 172.21.15.16:0/1114765717 wait complete. 2024-09-12T02:03:21.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.820+0000 7efc2e104740 1 -- start start 2024-09-12T02:03:21.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.820+0000 7efc2e104740 1 Processor -- start 2024-09-12T02:03:21.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.821+0000 7efc2e104740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5576e73825d0 0x5576e73909b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:21.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.821+0000 7efc2e104740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5576e7383b40 0x5576e7390ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:21.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.821+0000 7efc2e104740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576e738d370 0x5576e74056a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:21.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.821+0000 7efc2e104740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7efc241ab910 con 0x5576e738d370 2024-09-12T02:03:21.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.821+0000 7efc2e104740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7efc241acd80 con 0x5576e73825d0 2024-09-12T02:03:21.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.821+0000 7efc2e104740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7efc241abfe0 con 0x5576e7383b40 2024-09-12T02:03:21.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.821+0000 7efc2e104740 10 client.0 _ll_register_callbacks cb 0x7efc1409b620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:03:21.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.821+0000 7efc2a2c8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576e738d370 0x5576e74056a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:21.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.821+0000 7efc2a2c8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576e738d370 0x5576e74056a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52098/0 (socket says 172.21.15.16:52098) 2024-09-12T02:03:21.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.821+0000 7efc2a2c8640 1 -- 172.21.15.16:0/402197322 learned_addr learned my addr 172.21.15.16:0/402197322 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:03:21.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.821+0000 7efc29ac7640 1 --2- 172.21.15.16:0/402197322 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5576e73825d0 0x5576e73909b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:21.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.821+0000 7efc292c6640 1 --2- 172.21.15.16:0/402197322 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5576e7383b40 0x5576e7390ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:21.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.821+0000 7efc2a2c8640 1 -- 172.21.15.16:0/402197322 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5576e7383b40 msgr2=0x5576e7390ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:21.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.821+0000 7efc2a2c8640 1 --2- 172.21.15.16:0/402197322 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5576e7383b40 0x5576e7390ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:21.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.821+0000 7efc2a2c8640 1 -- 172.21.15.16:0/402197322 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5576e73825d0 msgr2=0x5576e73909b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:21.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.821+0000 7efc2a2c8640 1 --2- 172.21.15.16:0/402197322 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5576e73825d0 0x5576e73909b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:21.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.821+0000 7efc2a2c8640 1 -- 172.21.15.16:0/402197322 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5576e7405c90 con 0x5576e738d370 2024-09-12T02:03:21.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.821+0000 7efc29ac7640 1 --2- 172.21.15.16:0/402197322 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5576e73825d0 0x5576e73909b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:03:21.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.822+0000 7efc2a2c8640 1 --2- 172.21.15.16:0/402197322 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576e738d370 0x5576e74056a0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7efc1c02ae70 tx=0x7efc1c005d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d28a9007c3c522c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:21.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.822+0000 7efc0d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:03:21.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.822+0000 7efc0d7fa640 1 -- 172.21.15.16:0/402197322 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc1c06e070 con 0x5576e738d370 2024-09-12T02:03:21.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.822+0000 7efc0d7fa640 1 -- 172.21.15.16:0/402197322 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7efc1c004040 con 0x5576e738d370 2024-09-12T02:03:21.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.822+0000 7efc2e104740 1 -- 172.21.15.16:0/402197322 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efc1c05a070 con 0x5576e738d370 2024-09-12T02:03:21.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.822+0000 7efc0d7fa640 1 -- 172.21.15.16:0/402197322 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7efc1c07a050 con 0x5576e738d370 2024-09-12T02:03:21.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.822+0000 7efc2e104740 10 client.15762 fetch_fsmap learned FSMap version 25 2024-09-12T02:03:21.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.822+0000 7efc2e104740 1 -- 172.21.15.16:0/402197322 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5576e7406ee0 con 0x5576e738d370 2024-09-12T02:03:21.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.822+0000 7efc0d7fa640 1 -- 172.21.15.16:0/402197322 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc1c05a070 con 0x5576e738d370 2024-09-12T02:03:21.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.822+0000 7efc0d7fa640 1 -- 172.21.15.16:0/402197322 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7efc1c068070 con 0x5576e738d370 2024-09-12T02:03:21.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.822+0000 7efc2e104740 10 client.15762 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:03:21.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.822+0000 7efc2e104740 10 client.15762 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T02:03:21.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.822+0000 7efc2e104740 20 client.15762 populate_metadata read hostname 'smithi016' 2024-09-12T02:03:21.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.822+0000 7efc2e104740 1 --2- 172.21.15.16:0/402197322 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x5576e74075b0 0x5576e74099d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:21.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.822+0000 7efc2e104740 4 client.15762 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T02:03:21.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.822+0000 7efc2e104740 1 -- 172.21.15.16:0/402197322 --> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5576e72efd70 con 0x5576e74075b0 2024-09-12T02:03:21.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.823+0000 7efc29ac7640 1 --2- 172.21.15.16:0/402197322 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x5576e74075b0 0x5576e74099d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:21.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.823+0000 7efc29ac7640 1 --2- 172.21.15.16:0/402197322 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x5576e74075b0 0x5576e74099d0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9810263db83f2f72 server_cookie=10fd66e194e574c3 in_seq=0 out_seq=0 2024-09-12T02:03:21.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.823+0000 7efc0d7fa640 10 client.15762 ms_handle_connect on v2:172.21.15.184:6838/4203347941 2024-09-12T02:03:21.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.823+0000 7efc0d7fa640 1 -- 172.21.15.16:0/402197322 <== mds.1 v2:172.21.15.184:6838/4203347941 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5576e72efd70 con 0x5576e74075b0 2024-09-12T02:03:21.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.823+0000 7efc0d7fa640 10 client.15762 handle_command_reply: tid=0 2024-09-12T02:03:21.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.823+0000 7efc2e104740 1 client.15762 shutdown 2024-09-12T02:03:21.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.824+0000 7efc2e104740 1 -- 172.21.15.16:0/402197322 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576e738d370 msgr2=0x5576e74056a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:21.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.824+0000 7efc2e104740 1 --2- 172.21.15.16:0/402197322 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576e738d370 0x5576e74056a0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7efc1c02ae70 tx=0x7efc1c005d10 comp rx=0 tx=0).stop 2024-09-12T02:03:21.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.824+0000 7efc2e104740 1 -- 172.21.15.16:0/402197322 shutdown_connections 2024-09-12T02:03:21.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.824+0000 7efc2e104740 1 --2- 172.21.15.16:0/402197322 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x5576e74075b0 0x5576e74099d0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:21.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.824+0000 7efc2e104740 1 --2- 172.21.15.16:0/402197322 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576e738d370 0x5576e74056a0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:21.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.824+0000 7efc2e104740 1 --2- 172.21.15.16:0/402197322 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5576e7383b40 0x5576e7390ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:21.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.824+0000 7efc2e104740 1 --2- 172.21.15.16:0/402197322 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5576e73825d0 0x5576e73909b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:21.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.824+0000 7efc0d7fa640 0 client.15762 ms_handle_reset on v2:172.21.15.184:6838/4203347941 2024-09-12T02:03:21.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.824+0000 7efc2e104740 1 -- 172.21.15.16:0/402197322 >> 172.21.15.16:0/402197322 conn(0x5576e734b310 msgr2=0x5576e738c0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:21.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.824+0000 7efc2e104740 1 -- 172.21.15.16:0/402197322 shutdown_connections 2024-09-12T02:03:21.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.824+0000 7efc2e104740 1 -- 172.21.15.16:0/402197322 wait complete. 2024-09-12T02:03:21.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.824+0000 7efc2e104740 20 client.15762 trim_cache size 0 max 16384 2024-09-12T02:03:21.829 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T02:03:21.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.824+0000 7efc2bd2b640 1 -- 172.21.15.16:0/3277408139 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7efc100a2c00 msgr2=0x7efc100a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:21.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.824+0000 7efc2bd2b640 1 --2- 172.21.15.16:0/3277408139 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7efc100a2c00 0x7efc100a50c0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7efc1c04d000 tx=0x7efc1c04a000 comp rx=0 tx=0).stop 2024-09-12T02:03:21.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.824+0000 7efc2bd2b640 1 -- 172.21.15.16:0/3277408139 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc24103660 msgr2=0x7efc24198910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:21.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.824+0000 7efc2bd2b640 1 --2- 172.21.15.16:0/3277408139 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc24103660 0x7efc24198910 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7efc1400c970 tx=0x7efc1400cd70 comp rx=0 tx=0).stop 2024-09-12T02:03:21.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.824+0000 7efc2bd2b640 1 -- 172.21.15.16:0/3277408139 shutdown_connections 2024-09-12T02:03:21.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.824+0000 7efc2bd2b640 1 --2- 172.21.15.16:0/3277408139 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7efc100a2c00 0x7efc100a50c0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:21.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.824+0000 7efc2bd2b640 1 --2- 172.21.15.16:0/3277408139 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc2410c170 0x7efc2419cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:21.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.824+0000 7efc2bd2b640 1 --2- 172.21.15.16:0/3277408139 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc24103660 0x7efc24198910 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:21.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.824+0000 7efc2bd2b640 1 --2- 172.21.15.16:0/3277408139 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efc24102d40 0x7efc241983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:21.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.825+0000 7efc2bd2b640 1 -- 172.21.15.16:0/3277408139 >> 172.21.15.16:0/3277408139 conn(0x7efc240fc7a0 msgr2=0x7efc240fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:21.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.825+0000 7efc2bd2b640 1 -- 172.21.15.16:0/3277408139 shutdown_connections 2024-09-12T02:03:21.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:21.825+0000 7efc2bd2b640 1 -- 172.21.15.16:0/3277408139 wait complete. 2024-09-12T02:03:21.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:03:21 smithi184 ceph-mon[35593]: pgmap v378: 97 pgs: 97 active+clean; 903 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-09-12T02:03:21.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:03:21 smithi033 ceph-mon[35332]: pgmap v378: 97 pgs: 97 active+clean; 903 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-09-12T02:03:22.615 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-12T02:03:22.871 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:03:23.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.271+0000 7fc0f2d1c640 1 -- 172.21.15.16:0/1190824362 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0ec103660 msgr2=0x7fc0ec10bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:23.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.271+0000 7fc0f2d1c640 1 --2- 172.21.15.16:0/1190824362 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0ec103660 0x7fc0ec10bb50 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fc0d8009900 tx=0x7fc0d802a480 comp rx=0 tx=0).stop 2024-09-12T02:03:23.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.272+0000 7fc0f2d1c640 1 -- 172.21.15.16:0/1190824362 shutdown_connections 2024-09-12T02:03:23.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.272+0000 7fc0f2d1c640 1 --2- 172.21.15.16:0/1190824362 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc0ec10c170 0x7fc0ec10e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:23.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.272+0000 7fc0f2d1c640 1 --2- 172.21.15.16:0/1190824362 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0ec103660 0x7fc0ec10bb50 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:23.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.272+0000 7fc0f2d1c640 1 --2- 172.21.15.16:0/1190824362 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0ec102d40 0x7fc0ec103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:23.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.272+0000 7fc0f2d1c640 1 -- 172.21.15.16:0/1190824362 >> 172.21.15.16:0/1190824362 conn(0x7fc0ec0fc7a0 msgr2=0x7fc0ec0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:23.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.272+0000 7fc0f2d1c640 1 -- 172.21.15.16:0/1190824362 shutdown_connections 2024-09-12T02:03:23.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.272+0000 7fc0f2d1c640 1 -- 172.21.15.16:0/1190824362 wait complete. 2024-09-12T02:03:23.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.272+0000 7fc0f2d1c640 1 Processor -- start 2024-09-12T02:03:23.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.272+0000 7fc0f2d1c640 1 -- start start 2024-09-12T02:03:23.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.272+0000 7fc0f2d1c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0ec102d40 0x7fc0ec1009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:23.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.272+0000 7fc0f2d1c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc0ec103660 0x7fc0ec100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:23.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.272+0000 7fc0f2d1c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0ec10c170 0x7fc0ec101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:23.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.272+0000 7fc0f2d1c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc0ec111f70 con 0x7fc0ec10c170 2024-09-12T02:03:23.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.272+0000 7fc0f2d1c640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc0ec111df0 con 0x7fc0ec102d40 2024-09-12T02:03:23.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.272+0000 7fc0f2d1c640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc0ec1120f0 con 0x7fc0ec103660 2024-09-12T02:03:23.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.273+0000 7fc0f0ab8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0ec102d40 0x7fc0ec1009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:23.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.273+0000 7fc0f12b9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0ec10c170 0x7fc0ec101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:23.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.273+0000 7fc0f0ab8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0ec102d40 0x7fc0ec1009a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.16:59434/0 (socket says 172.21.15.16:59434) 2024-09-12T02:03:23.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.273+0000 7fc0f0ab8640 1 -- 172.21.15.16:0/414621011 learned_addr learned my addr 172.21.15.16:0/414621011 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:03:23.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.273+0000 7fc0ebfff640 1 --2- 172.21.15.16:0/414621011 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc0ec103660 0x7fc0ec100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:23.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.273+0000 7fc0f0ab8640 1 -- 172.21.15.16:0/414621011 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc0ec103660 msgr2=0x7fc0ec100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:23.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.273+0000 7fc0f0ab8640 1 --2- 172.21.15.16:0/414621011 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc0ec103660 0x7fc0ec100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:23.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.273+0000 7fc0f0ab8640 1 -- 172.21.15.16:0/414621011 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0ec10c170 msgr2=0x7fc0ec101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:23.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.273+0000 7fc0f0ab8640 1 --2- 172.21.15.16:0/414621011 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0ec10c170 0x7fc0ec101420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:23.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.273+0000 7fc0f0ab8640 1 -- 172.21.15.16:0/414621011 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0ec1a6930 con 0x7fc0ec102d40 2024-09-12T02:03:23.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.273+0000 7fc0ebfff640 1 --2- 172.21.15.16:0/414621011 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc0ec103660 0x7fc0ec100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:03:23.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.273+0000 7fc0f12b9640 1 --2- 172.21.15.16:0/414621011 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0ec10c170 0x7fc0ec101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:03:23.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.273+0000 7fc0f0ab8640 1 --2- 172.21.15.16:0/414621011 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0ec102d40 0x7fc0ec1009a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc0e000e9f0 tx=0x7fc0e000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:23.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.273+0000 7fc0e9ffb640 1 -- 172.21.15.16:0/414621011 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0e000cd80 con 0x7fc0ec102d40 2024-09-12T02:03:23.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.273+0000 7fc0f2d1c640 1 -- 172.21.15.16:0/414621011 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0ec1a6c20 con 0x7fc0ec102d40 2024-09-12T02:03:23.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.273+0000 7fc0e9ffb640 1 -- 172.21.15.16:0/414621011 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc0e0010910 con 0x7fc0ec102d40 2024-09-12T02:03:23.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.273+0000 7fc0e9ffb640 1 -- 172.21.15.16:0/414621011 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0e00188f0 con 0x7fc0ec102d40 2024-09-12T02:03:23.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.274+0000 7fc0f2d1c640 1 -- 172.21.15.16:0/414621011 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0ec1a71b0 con 0x7fc0ec102d40 2024-09-12T02:03:23.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.276+0000 7fc0e9ffb640 1 -- 172.21.15.16:0/414621011 <== mon.2 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fc0e0010ac0 con 0x7fc0ec102d40 2024-09-12T02:03:23.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.276+0000 7fc0f50f5740 1 Processor -- start 2024-09-12T02:03:23.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.276+0000 7fc0f50f5740 1 -- start start 2024-09-12T02:03:23.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.276+0000 7fc0f50f5740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629d8410b40 0x5629d840f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:23.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.276+0000 7fc0f50f5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629d840f5d0 0x5629d840fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:23.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.276+0000 7fc0f50f5740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5629d841a370 0x5629d841c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:23.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.276+0000 7fc0f50f5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5629d83752f0 con 0x5629d840f5d0 2024-09-12T02:03:23.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.276+0000 7fc0f50f5740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5629d831ccf0 con 0x5629d8410b40 2024-09-12T02:03:23.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.276+0000 7fc0f50f5740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5629d83712b0 con 0x5629d841a370 2024-09-12T02:03:23.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.277+0000 7fc0f12b9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629d8410b40 0x5629d840f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:23.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.277+0000 7fc0f0ab8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629d840f5d0 0x5629d840fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:23.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.277+0000 7fc0f12b9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629d8410b40 0x5629d840f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.16:59446/0 (socket says 172.21.15.16:59446) 2024-09-12T02:03:23.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.277+0000 7fc0f12b9640 1 -- 172.21.15.16:0/1581997988 learned_addr learned my addr 172.21.15.16:0/1581997988 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:03:23.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.277+0000 7fc0ebfff640 1 --2- 172.21.15.16:0/1581997988 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5629d841a370 0x5629d841c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:23.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.277+0000 7fc0f12b9640 1 -- 172.21.15.16:0/1581997988 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5629d841a370 msgr2=0x5629d841c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:23.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.277+0000 7fc0f12b9640 1 --2- 172.21.15.16:0/1581997988 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5629d841a370 0x5629d841c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:23.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.277+0000 7fc0f12b9640 1 -- 172.21.15.16:0/1581997988 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629d840f5d0 msgr2=0x5629d840fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:23.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.277+0000 7fc0f12b9640 1 --2- 172.21.15.16:0/1581997988 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629d840f5d0 0x5629d840fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:23.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.277+0000 7fc0f12b9640 1 -- 172.21.15.16:0/1581997988 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629d83013a0 con 0x5629d8410b40 2024-09-12T02:03:23.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.277+0000 7fc0e9ffb640 1 --2- 172.21.15.16:0/414621011 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc0dc0a2b80 0x7fc0dc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:23.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.277+0000 7fc0f0ab8640 1 --2- 172.21.15.16:0/414621011 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc0dc0a2b80 0x7fc0dc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:23.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.277+0000 7fc0f12b9640 1 --2- 172.21.15.16:0/1581997988 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629d8410b40 0x5629d840f090 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc0e4011790 tx=0x7fc0e4011c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=277873fa8c2b61b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:23.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.277+0000 7fc0e9ffb640 1 -- 172.21.15.16:0/414621011 <== mon.2 v2:172.21.15.33:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fc0e00cf400 con 0x7fc0ec102d40 2024-09-12T02:03:23.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.278+0000 7fc0d6ffd640 1 -- 172.21.15.16:0/1581997988 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0e4003c30 con 0x5629d8410b40 2024-09-12T02:03:23.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.278+0000 7fc0d6ffd640 1 -- 172.21.15.16:0/1581997988 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc0e400fe80 con 0x5629d8410b40 2024-09-12T02:03:23.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.278+0000 7fc0d6ffd640 1 -- 172.21.15.16:0/1581997988 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0e40193e0 con 0x5629d8410b40 2024-09-12T02:03:23.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.278+0000 7fc0f0ab8640 1 --2- 172.21.15.16:0/414621011 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc0dc0a2b80 0x7fc0dc0a5040 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fc0e0098160 tx=0x7fc0e00cce70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:23.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.278+0000 7fc0f50f5740 1 -- 172.21.15.16:0/1581997988 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629d8410b40 msgr2=0x5629d840f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:23.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.278+0000 7fc0f50f5740 1 --2- 172.21.15.16:0/1581997988 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629d8410b40 0x5629d840f090 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc0e4011790 tx=0x7fc0e4011c50 comp rx=0 tx=0).stop 2024-09-12T02:03:23.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.278+0000 7fc0f50f5740 1 -- 172.21.15.16:0/1581997988 shutdown_connections 2024-09-12T02:03:23.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.278+0000 7fc0f50f5740 1 --2- 172.21.15.16:0/1581997988 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5629d841a370 0x5629d841c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:23.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.278+0000 7fc0f50f5740 1 --2- 172.21.15.16:0/1581997988 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629d840f5d0 0x5629d840fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:23.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.278+0000 7fc0f50f5740 1 --2- 172.21.15.16:0/1581997988 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629d8410b40 0x5629d840f090 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:23.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.278+0000 7fc0f50f5740 1 -- 172.21.15.16:0/1581997988 >> 172.21.15.16:0/1581997988 conn(0x5629d83d8310 msgr2=0x5629d840bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:23.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.278+0000 7fc0f50f5740 1 -- 172.21.15.16:0/1581997988 shutdown_connections 2024-09-12T02:03:23.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.278+0000 7fc0f50f5740 1 -- 172.21.15.16:0/1581997988 wait complete. 2024-09-12T02:03:23.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.279+0000 7fc0f50f5740 1 -- start start 2024-09-12T02:03:23.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.279+0000 7fc0f50f5740 1 Processor -- start 2024-09-12T02:03:23.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.279+0000 7fc0f50f5740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629d840f5d0 0x5629d8419d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:23.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.279+0000 7fc0f50f5740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5629d8410b40 0x5629d8416fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:23.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.279+0000 7fc0f50f5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629d841a370 0x5629d84175d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:23.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.279+0000 7fc0f50f5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5629d841d070 con 0x5629d841a370 2024-09-12T02:03:23.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.279+0000 7fc0f50f5740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5629d8330060 con 0x5629d840f5d0 2024-09-12T02:03:23.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.279+0000 7fc0f50f5740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5629d841d1f0 con 0x5629d8410b40 2024-09-12T02:03:23.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.279+0000 7fc0f50f5740 10 client.0 _ll_register_callbacks cb 0x5629d80838c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:03:23.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.279+0000 7fc0f12b9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629d841a370 0x5629d84175d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:23.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.279+0000 7fc0f12b9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629d841a370 0x5629d84175d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52146/0 (socket says 172.21.15.16:52146) 2024-09-12T02:03:23.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.279+0000 7fc0f12b9640 1 -- 172.21.15.16:0/128995743 learned_addr learned my addr 172.21.15.16:0/128995743 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:03:23.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.280+0000 7fc0f12b9640 1 --2- 172.21.15.16:0/128995743 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629d840f5d0 0x5629d8419d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:23.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.280+0000 7fc0ebfff640 1 --2- 172.21.15.16:0/128995743 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5629d8410b40 0x5629d8416fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:23.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.280+0000 7fc0f12b9640 1 -- 172.21.15.16:0/128995743 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5629d8410b40 msgr2=0x5629d8416fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:23.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.280+0000 7fc0f12b9640 1 --2- 172.21.15.16:0/128995743 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5629d8410b40 0x5629d8416fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:23.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.280+0000 7fc0f12b9640 1 -- 172.21.15.16:0/128995743 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629d840f5d0 msgr2=0x5629d8419d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:23.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.280+0000 7fc0f12b9640 1 --2- 172.21.15.16:0/128995743 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629d840f5d0 0x5629d8419d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:23.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.280+0000 7fc0f12b9640 1 -- 172.21.15.16:0/128995743 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629d80e63c0 con 0x5629d841a370 2024-09-12T02:03:23.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.280+0000 7fc0f12b9640 1 --2- 172.21.15.16:0/128995743 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629d841a370 0x5629d84175d0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fc0e4011760 tx=0x7fc0e4004520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d5bf988cd47fed7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:23.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.280+0000 7fc0cbfff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:03:23.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.280+0000 7fc0cbfff640 1 -- 172.21.15.16:0/128995743 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0e4011070 con 0x5629d841a370 2024-09-12T02:03:23.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.280+0000 7fc0cbfff640 1 -- 172.21.15.16:0/128995743 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fc0e4011210 con 0x5629d841a370 2024-09-12T02:03:23.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.280+0000 7fc0f50f5740 1 -- 172.21.15.16:0/128995743 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5629d8382860 con 0x5629d841a370 2024-09-12T02:03:23.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.280+0000 7fc0cbfff640 1 -- 172.21.15.16:0/128995743 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc0e4004a40 con 0x5629d841a370 2024-09-12T02:03:23.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.280+0000 7fc0f50f5740 10 client.15774 fetch_fsmap learned FSMap version 25 2024-09-12T02:03:23.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.280+0000 7fc0f50f5740 1 -- 172.21.15.16:0/128995743 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5629d8428b00 con 0x5629d841a370 2024-09-12T02:03:23.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.281+0000 7fc0cbfff640 1 -- 172.21.15.16:0/128995743 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5629d8382860 con 0x5629d841a370 2024-09-12T02:03:23.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.281+0000 7fc0cbfff640 1 -- 172.21.15.16:0/128995743 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fc0e4025040 con 0x5629d841a370 2024-09-12T02:03:23.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.281+0000 7fc0f50f5740 10 client.15774 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:03:23.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.281+0000 7fc0f50f5740 10 client.15774 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:03:23.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.281+0000 7fc0f50f5740 20 client.15774 populate_metadata read hostname 'smithi016' 2024-09-12T02:03:23.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.281+0000 7fc0f50f5740 1 --2- 172.21.15.16:0/128995743 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5629d8429eb0 0x5629d842c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:23.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.281+0000 7fc0f50f5740 4 client.15774 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:03:23.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.281+0000 7fc0f50f5740 1 -- 172.21.15.16:0/128995743 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5629d837cd70 con 0x5629d8429eb0 2024-09-12T02:03:23.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.281+0000 7fc0ebfff640 1 --2- 172.21.15.16:0/128995743 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5629d8429eb0 0x5629d842c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:23.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.282+0000 7fc0ebfff640 1 --2- 172.21.15.16:0/128995743 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5629d8429eb0 0x5629d842c2a0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f4dcd427c895f7 server_cookie=82fb8f0e96ee1797 in_seq=0 out_seq=0 2024-09-12T02:03:23.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.282+0000 7fc0cbfff640 10 client.15774 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:03:23.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.282+0000 7fc0cbfff640 1 -- 172.21.15.16:0/128995743 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5629d837cd70 con 0x5629d8429eb0 2024-09-12T02:03:23.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.282+0000 7fc0cbfff640 10 client.15774 handle_command_reply: tid=0 2024-09-12T02:03:23.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.282+0000 7fc0f50f5740 1 client.15774 shutdown 2024-09-12T02:03:23.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.283+0000 7fc0f50f5740 1 -- 172.21.15.16:0/128995743 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629d841a370 msgr2=0x5629d84175d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:23.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.283+0000 7fc0f50f5740 1 --2- 172.21.15.16:0/128995743 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629d841a370 0x5629d84175d0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fc0e4011760 tx=0x7fc0e4004520 comp rx=0 tx=0).stop 2024-09-12T02:03:23.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.283+0000 7fc0f50f5740 1 -- 172.21.15.16:0/128995743 shutdown_connections 2024-09-12T02:03:23.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.283+0000 7fc0f50f5740 1 --2- 172.21.15.16:0/128995743 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5629d8429eb0 0x5629d842c2a0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:23.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.283+0000 7fc0f50f5740 1 --2- 172.21.15.16:0/128995743 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629d841a370 0x5629d84175d0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:23.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.283+0000 7fc0f50f5740 1 --2- 172.21.15.16:0/128995743 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5629d8410b40 0x5629d8416fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:23.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.283+0000 7fc0f50f5740 1 --2- 172.21.15.16:0/128995743 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629d840f5d0 0x5629d8419d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:23.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.283+0000 7fc0f50f5740 1 -- 172.21.15.16:0/128995743 >> 172.21.15.16:0/128995743 conn(0x5629d83d8310 msgr2=0x5629d8411930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:23.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.283+0000 7fc0cbfff640 0 client.15774 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:03:23.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.283+0000 7fc0f50f5740 1 -- 172.21.15.16:0/128995743 shutdown_connections 2024-09-12T02:03:23.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.283+0000 7fc0f50f5740 1 -- 172.21.15.16:0/128995743 wait complete. 2024-09-12T02:03:23.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.283+0000 7fc0f50f5740 20 client.15774 trim_cache size 0 max 16384 2024-09-12T02:03:23.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.296+0000 7fc0f50f5740 1 Processor -- start 2024-09-12T02:03:23.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.296+0000 7fc0f50f5740 1 -- start start 2024-09-12T02:03:23.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.296+0000 7fc0f50f5740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629d840f5d0 0x5629d8426340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:23.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.296+0000 7fc0f50f5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629d8410b40 0x5629d8412120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:23.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.296+0000 7fc0f50f5740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5629d841a370 0x5629d8412660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:23.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.296+0000 7fc0f50f5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc0ec1b1f10 con 0x5629d8410b40 2024-09-12T02:03:23.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.296+0000 7fc0f50f5740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc0ec1af7e0 con 0x5629d840f5d0 2024-09-12T02:03:23.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.296+0000 7fc0f50f5740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc0ec1ad630 con 0x5629d841a370 2024-09-12T02:03:23.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.296+0000 7fc0ebfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629d8410b40 0x5629d8412120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:23.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.296+0000 7fc0ebfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629d8410b40 0x5629d8412120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52152/0 (socket says 172.21.15.16:52152) 2024-09-12T02:03:23.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.296+0000 7fc0ebfff640 1 -- 172.21.15.16:0/2519591234 learned_addr learned my addr 172.21.15.16:0/2519591234 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:03:23.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.297+0000 7fc0f12b9640 1 --2- 172.21.15.16:0/2519591234 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629d840f5d0 0x5629d8426340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:23.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.297+0000 7fc0f12b9640 1 --2- 172.21.15.16:0/2519591234 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5629d841a370 0x5629d8412660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:23.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.297+0000 7fc0ebfff640 1 -- 172.21.15.16:0/2519591234 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5629d841a370 msgr2=0x5629d8412660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:23.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.297+0000 7fc0ebfff640 1 --2- 172.21.15.16:0/2519591234 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5629d841a370 0x5629d8412660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:23.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.297+0000 7fc0ebfff640 1 -- 172.21.15.16:0/2519591234 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629d840f5d0 msgr2=0x5629d8426340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:23.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.297+0000 7fc0ebfff640 1 --2- 172.21.15.16:0/2519591234 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629d840f5d0 0x5629d8426340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:23.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.297+0000 7fc0ebfff640 1 -- 172.21.15.16:0/2519591234 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629d840a310 con 0x5629d8410b40 2024-09-12T02:03:23.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.297+0000 7fc0ebfff640 1 --2- 172.21.15.16:0/2519591234 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629d8410b40 0x5629d8412120 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fc0d8009900 tx=0x7fc0d8005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3efa33d8bcdbcba0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:23.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.297+0000 7fc0d67fc640 1 -- 172.21.15.16:0/2519591234 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0d8049070 con 0x5629d8410b40 2024-09-12T02:03:23.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.297+0000 7fc0d67fc640 1 -- 172.21.15.16:0/2519591234 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc0d8005520 con 0x5629d8410b40 2024-09-12T02:03:23.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.297+0000 7fc0d67fc640 1 -- 172.21.15.16:0/2519591234 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0d8035040 con 0x5629d8410b40 2024-09-12T02:03:23.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.298+0000 7fc0f50f5740 1 -- 172.21.15.16:0/2519591234 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629d8410b40 msgr2=0x5629d8412120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:23.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.298+0000 7fc0f50f5740 1 --2- 172.21.15.16:0/2519591234 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629d8410b40 0x5629d8412120 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fc0d8009900 tx=0x7fc0d8005b50 comp rx=0 tx=0).stop 2024-09-12T02:03:23.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.298+0000 7fc0f50f5740 1 -- 172.21.15.16:0/2519591234 shutdown_connections 2024-09-12T02:03:23.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.298+0000 7fc0f50f5740 1 --2- 172.21.15.16:0/2519591234 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5629d841a370 0x5629d8412660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:23.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.298+0000 7fc0f50f5740 1 --2- 172.21.15.16:0/2519591234 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629d8410b40 0x5629d8412120 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:23.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.298+0000 7fc0f50f5740 1 --2- 172.21.15.16:0/2519591234 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629d840f5d0 0x5629d8426340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:23.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.298+0000 7fc0f50f5740 1 -- 172.21.15.16:0/2519591234 >> 172.21.15.16:0/2519591234 conn(0x5629d83d8310 msgr2=0x5629d8419060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:23.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.298+0000 7fc0f50f5740 1 -- 172.21.15.16:0/2519591234 shutdown_connections 2024-09-12T02:03:23.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.298+0000 7fc0f50f5740 1 -- 172.21.15.16:0/2519591234 wait complete. 2024-09-12T02:03:23.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.298+0000 7fc0f50f5740 1 -- start start 2024-09-12T02:03:23.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.298+0000 7fc0f50f5740 1 Processor -- start 2024-09-12T02:03:23.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.299+0000 7fc0f50f5740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629d840f5d0 0x5629d841dc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:23.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.299+0000 7fc0f50f5740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5629d8410b40 0x5629d841e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:23.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.299+0000 7fc0f50f5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629d841a370 0x5629d848c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:23.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.299+0000 7fc0f50f5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc0ec1adca0 con 0x5629d841a370 2024-09-12T02:03:23.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.299+0000 7fc0f50f5740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc0ec1af110 con 0x5629d840f5d0 2024-09-12T02:03:23.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.299+0000 7fc0f50f5740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc0ec1ae370 con 0x5629d8410b40 2024-09-12T02:03:23.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.299+0000 7fc0f50f5740 10 client.0 _ll_register_callbacks cb 0x7fc0e00d2df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:03:23.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.299+0000 7fc0f12b9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629d841a370 0x5629d848c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:23.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.299+0000 7fc0f12b9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629d841a370 0x5629d848c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52168/0 (socket says 172.21.15.16:52168) 2024-09-12T02:03:23.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.299+0000 7fc0f12b9640 1 -- 172.21.15.16:0/2479462833 learned_addr learned my addr 172.21.15.16:0/2479462833 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:03:23.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.299+0000 7fc0f12b9640 1 --2- 172.21.15.16:0/2479462833 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629d840f5d0 0x5629d841dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:23.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.299+0000 7fc0ebfff640 1 --2- 172.21.15.16:0/2479462833 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5629d8410b40 0x5629d841e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:23.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.299+0000 7fc0f12b9640 1 -- 172.21.15.16:0/2479462833 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5629d8410b40 msgr2=0x5629d841e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:23.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.299+0000 7fc0f12b9640 1 --2- 172.21.15.16:0/2479462833 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5629d8410b40 0x5629d841e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:23.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.299+0000 7fc0f12b9640 1 -- 172.21.15.16:0/2479462833 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629d840f5d0 msgr2=0x5629d841dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:23.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.299+0000 7fc0f12b9640 1 --2- 172.21.15.16:0/2479462833 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629d840f5d0 0x5629d841dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:23.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.299+0000 7fc0f12b9640 1 -- 172.21.15.16:0/2479462833 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629d848d060 con 0x5629d841a370 2024-09-12T02:03:23.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.299+0000 7fc0ebfff640 1 --2- 172.21.15.16:0/2479462833 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5629d8410b40 0x5629d841e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:03:23.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.299+0000 7fc0f12b9640 1 --2- 172.21.15.16:0/2479462833 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629d841a370 0x5629d848c9c0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x5629d8432690 tx=0x7fc0e4005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be2a246608d1c962 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:23.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.299+0000 7fc0cbfff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:03:23.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.300+0000 7fc0cbfff640 1 -- 172.21.15.16:0/2479462833 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0e400f790 con 0x5629d841a370 2024-09-12T02:03:23.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.300+0000 7fc0cbfff640 1 -- 172.21.15.16:0/2479462833 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fc0e401f070 con 0x5629d841a370 2024-09-12T02:03:23.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.300+0000 7fc0f50f5740 1 -- 172.21.15.16:0/2479462833 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc0d8005520 con 0x5629d841a370 2024-09-12T02:03:23.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.300+0000 7fc0cbfff640 1 -- 172.21.15.16:0/2479462833 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc0e4004360 con 0x5629d841a370 2024-09-12T02:03:23.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.300+0000 7fc0f50f5740 10 client.15780 fetch_fsmap learned FSMap version 25 2024-09-12T02:03:23.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.300+0000 7fc0f50f5740 1 -- 172.21.15.16:0/2479462833 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5629d8494430 con 0x5629d841a370 2024-09-12T02:03:23.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.300+0000 7fc0cbfff640 1 -- 172.21.15.16:0/2479462833 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc0d8005520 con 0x5629d841a370 2024-09-12T02:03:23.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.300+0000 7fc0cbfff640 1 -- 172.21.15.16:0/2479462833 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fc0e4018070 con 0x5629d841a370 2024-09-12T02:03:23.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.300+0000 7fc0f50f5740 10 client.15780 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:03:23.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.300+0000 7fc0f50f5740 10 client.15780 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:03:23.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.300+0000 7fc0f50f5740 20 client.15780 populate_metadata read hostname 'smithi016' 2024-09-12T02:03:23.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.300+0000 7fc0f50f5740 1 --2- 172.21.15.16:0/2479462833 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5629d84949f0 0x5629d8496de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:23.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.300+0000 7fc0f50f5740 4 client.15780 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T02:03:23.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.300+0000 7fc0f50f5740 1 -- 172.21.15.16:0/2479462833 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5629d82f9170 con 0x5629d84949f0 2024-09-12T02:03:23.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.301+0000 7fc0ebfff640 1 --2- 172.21.15.16:0/2479462833 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5629d84949f0 0x5629d8496de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:23.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.301+0000 7fc0ebfff640 1 --2- 172.21.15.16:0/2479462833 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5629d84949f0 0x5629d8496de0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e172267694d45b6 server_cookie=7432fe13aec32b92 in_seq=0 out_seq=0 2024-09-12T02:03:23.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.301+0000 7fc0cbfff640 10 client.15780 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:03:23.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.301+0000 7fc0cbfff640 1 -- 172.21.15.16:0/2479462833 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5629d82f9170 con 0x5629d84949f0 2024-09-12T02:03:23.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.301+0000 7fc0cbfff640 10 client.15780 handle_command_reply: tid=0 2024-09-12T02:03:23.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.301+0000 7fc0f50f5740 1 client.15780 shutdown 2024-09-12T02:03:23.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.301+0000 7fc0f50f5740 1 -- 172.21.15.16:0/2479462833 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629d841a370 msgr2=0x5629d848c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:23.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.301+0000 7fc0f50f5740 1 --2- 172.21.15.16:0/2479462833 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629d841a370 0x5629d848c9c0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x5629d8432690 tx=0x7fc0e4005bd0 comp rx=0 tx=0).stop 2024-09-12T02:03:23.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.301+0000 7fc0f50f5740 1 -- 172.21.15.16:0/2479462833 shutdown_connections 2024-09-12T02:03:23.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.301+0000 7fc0f50f5740 1 --2- 172.21.15.16:0/2479462833 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5629d84949f0 0x5629d8496de0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:23.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.301+0000 7fc0f50f5740 1 --2- 172.21.15.16:0/2479462833 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629d841a370 0x5629d848c9c0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:23.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.301+0000 7fc0f50f5740 1 --2- 172.21.15.16:0/2479462833 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5629d8410b40 0x5629d841e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:23.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.301+0000 7fc0f50f5740 1 --2- 172.21.15.16:0/2479462833 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629d840f5d0 0x5629d841dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:23.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.301+0000 7fc0f50f5740 1 -- 172.21.15.16:0/2479462833 >> 172.21.15.16:0/2479462833 conn(0x5629d83d8310 msgr2=0x5629d8418840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:23.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.301+0000 7fc0cbfff640 0 client.15780 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:03:23.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.302+0000 7fc0f50f5740 1 -- 172.21.15.16:0/2479462833 shutdown_connections 2024-09-12T02:03:23.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.302+0000 7fc0f50f5740 1 -- 172.21.15.16:0/2479462833 wait complete. 2024-09-12T02:03:23.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.302+0000 7fc0f50f5740 20 client.15780 trim_cache size 0 max 16384 2024-09-12T02:03:23.307 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T02:03:23.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.302+0000 7fc0f2d1c640 1 -- 172.21.15.16:0/414621011 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc0dc0a2b80 msgr2=0x7fc0dc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:23.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.302+0000 7fc0f2d1c640 1 --2- 172.21.15.16:0/414621011 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc0dc0a2b80 0x7fc0dc0a5040 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fc0e0098160 tx=0x7fc0e00cce70 comp rx=0 tx=0).stop 2024-09-12T02:03:23.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.302+0000 7fc0f2d1c640 1 -- 172.21.15.16:0/414621011 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0ec102d40 msgr2=0x7fc0ec1009a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:23.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.302+0000 7fc0f2d1c640 1 --2- 172.21.15.16:0/414621011 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0ec102d40 0x7fc0ec1009a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc0e000e9f0 tx=0x7fc0e000eeb0 comp rx=0 tx=0).stop 2024-09-12T02:03:23.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.302+0000 7fc0f2d1c640 1 -- 172.21.15.16:0/414621011 shutdown_connections 2024-09-12T02:03:23.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.302+0000 7fc0f2d1c640 1 --2- 172.21.15.16:0/414621011 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc0dc0a2b80 0x7fc0dc0a5040 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:23.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.302+0000 7fc0f2d1c640 1 --2- 172.21.15.16:0/414621011 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0ec10c170 0x7fc0ec101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:23.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.302+0000 7fc0f2d1c640 1 --2- 172.21.15.16:0/414621011 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc0ec103660 0x7fc0ec100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:23.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.302+0000 7fc0f2d1c640 1 --2- 172.21.15.16:0/414621011 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0ec102d40 0x7fc0ec1009a0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:23.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.302+0000 7fc0f2d1c640 1 -- 172.21.15.16:0/414621011 >> 172.21.15.16:0/414621011 conn(0x7fc0ec0fc7a0 msgr2=0x7fc0ec0fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:23.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.302+0000 7fc0f2d1c640 1 -- 172.21.15.16:0/414621011 shutdown_connections 2024-09-12T02:03:23.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:23.302+0000 7fc0f2d1c640 1 -- 172.21.15.16:0/414621011 wait complete. 2024-09-12T02:03:23.650 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:03:23 smithi016 ceph-mon[30947]: pgmap v379: 97 pgs: 97 active+clean; 903 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-09-12T02:03:23.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:03:23 smithi184 ceph-mon[35593]: pgmap v379: 97 pgs: 97 active+clean; 903 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-09-12T02:03:23.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:03:23 smithi033 ceph-mon[35332]: pgmap v379: 97 pgs: 97 active+clean; 903 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-09-12T02:03:25.074 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-12T02:03:25.074 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-12T02:03:25.344 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:03:25.722 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:03:25 smithi016 ceph-mon[30947]: pgmap v380: 97 pgs: 97 active+clean; 960 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-09-12T02:03:25.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.808+0000 7f29495a7640 1 -- 172.21.15.16:0/3585088483 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2944103650 msgr2=0x7f294410bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:25.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.808+0000 7f29495a7640 1 --2- 172.21.15.16:0/3585088483 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2944103650 0x7f294410bb40 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f292c009900 tx=0x7f292c02a480 comp rx=0 tx=0).stop 2024-09-12T02:03:25.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.808+0000 7f29495a7640 1 -- 172.21.15.16:0/3585088483 shutdown_connections 2024-09-12T02:03:25.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.808+0000 7f29495a7640 1 --2- 172.21.15.16:0/3585088483 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f294410c160 0x7f294410e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:25.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.808+0000 7f29495a7640 1 --2- 172.21.15.16:0/3585088483 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2944103650 0x7f294410bb40 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:25.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.808+0000 7f29495a7640 1 --2- 172.21.15.16:0/3585088483 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2944102d30 0x7f2944103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:25.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.808+0000 7f29495a7640 1 -- 172.21.15.16:0/3585088483 >> 172.21.15.16:0/3585088483 conn(0x7f29440fc7d0 msgr2=0x7f29440febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:25.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.808+0000 7f29495a7640 1 -- 172.21.15.16:0/3585088483 shutdown_connections 2024-09-12T02:03:25.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.808+0000 7f29495a7640 1 -- 172.21.15.16:0/3585088483 wait complete. 2024-09-12T02:03:25.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.809+0000 7f29495a7640 1 Processor -- start 2024-09-12T02:03:25.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.809+0000 7f29495a7640 1 -- start start 2024-09-12T02:03:25.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.809+0000 7f29495a7640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2944102d30 0x7f2944100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:25.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.809+0000 7f29495a7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2944103650 0x7f2944100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:25.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.810+0000 7f29495a7640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f294410c160 0x7f2944101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:25.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.810+0000 7f29495a7640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2944111f60 con 0x7f2944103650 2024-09-12T02:03:25.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.810+0000 7f29495a7640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2944111de0 con 0x7f294410c160 2024-09-12T02:03:25.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.810+0000 7f29495a7640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f29441120e0 con 0x7f2944102d30 2024-09-12T02:03:25.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.810+0000 7f293a7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2944103650 0x7f2944100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:25.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.810+0000 7f2942ffd640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2944102d30 0x7f2944100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:25.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.810+0000 7f293a7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2944103650 0x7f2944100ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52186/0 (socket says 172.21.15.16:52186) 2024-09-12T02:03:25.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.810+0000 7f293a7fc640 1 -- 172.21.15.16:0/2655844437 learned_addr learned my addr 172.21.15.16:0/2655844437 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:03:25.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.810+0000 7f29437fe640 1 --2- 172.21.15.16:0/2655844437 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f294410c160 0x7f2944101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:25.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.810+0000 7f293a7fc640 1 -- 172.21.15.16:0/2655844437 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2944102d30 msgr2=0x7f2944100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:25.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.810+0000 7f293a7fc640 1 --2- 172.21.15.16:0/2655844437 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2944102d30 0x7f2944100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:25.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.810+0000 7f293a7fc640 1 -- 172.21.15.16:0/2655844437 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f294410c160 msgr2=0x7f2944101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:25.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.810+0000 7f293a7fc640 1 --2- 172.21.15.16:0/2655844437 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f294410c160 0x7f2944101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:25.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.810+0000 7f293a7fc640 1 -- 172.21.15.16:0/2655844437 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29441a6920 con 0x7f2944103650 2024-09-12T02:03:25.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.810+0000 7f29437fe640 1 --2- 172.21.15.16:0/2655844437 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f294410c160 0x7f2944101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:03:25.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.810+0000 7f293a7fc640 1 --2- 172.21.15.16:0/2655844437 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2944103650 0x7f2944100ed0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f292c006010 tx=0x7f292c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:25.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.810+0000 7f2940ff9640 1 -- 172.21.15.16:0/2655844437 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f292c03f030 con 0x7f2944103650 2024-09-12T02:03:25.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.810+0000 7f2940ff9640 1 -- 172.21.15.16:0/2655844437 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f292c003ce0 con 0x7f2944103650 2024-09-12T02:03:25.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.810+0000 7f2940ff9640 1 -- 172.21.15.16:0/2655844437 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f292c035070 con 0x7f2944103650 2024-09-12T02:03:25.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.811+0000 7f29495a7640 1 -- 172.21.15.16:0/2655844437 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29441a6bb0 con 0x7f2944103650 2024-09-12T02:03:25.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.811+0000 7f29495a7640 1 -- 172.21.15.16:0/2655844437 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29441a7090 con 0x7f2944103650 2024-09-12T02:03:25.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.813+0000 7f2940ff9640 1 -- 172.21.15.16:0/2655844437 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f292c042030 con 0x7f2944103650 2024-09-12T02:03:25.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.813+0000 7f294b980740 1 Processor -- start 2024-09-12T02:03:25.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.813+0000 7f294b980740 1 -- start start 2024-09-12T02:03:25.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.814+0000 7f294b980740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55627f14fb60 0x55627f14ff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:25.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.814+0000 7f294b980740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55627f150480 0x55627f15a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:25.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.814+0000 7f294b980740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55627f15ad40 0x55627f15d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:25.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.814+0000 7f294b980740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55627f0b5280 con 0x55627f14fb60 2024-09-12T02:03:25.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.814+0000 7f294b980740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55627f05cb10 con 0x55627f150480 2024-09-12T02:03:25.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.814+0000 7f294b980740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55627f0b1240 con 0x55627f15ad40 2024-09-12T02:03:25.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.814+0000 7f29437fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55627f14fb60 0x55627f14ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:25.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.814+0000 7f29437fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55627f14fb60 0x55627f14ff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52188/0 (socket says 172.21.15.16:52188) 2024-09-12T02:03:25.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.814+0000 7f29437fe640 1 -- 172.21.15.16:0/1652274281 learned_addr learned my addr 172.21.15.16:0/1652274281 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:03:25.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.814+0000 7f2940ff9640 1 --2- 172.21.15.16:0/2655844437 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f29300a2b80 0x7f29300a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:25.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.814+0000 7f2942ffd640 1 --2- 172.21.15.16:0/1652274281 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55627f150480 0x55627f15a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:25.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.814+0000 7f293a7fc640 1 --2- 172.21.15.16:0/1652274281 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55627f15ad40 0x55627f15d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:25.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.814+0000 7f29437fe640 1 --2- 172.21.15.16:0/2655844437 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f29300a2b80 0x7f29300a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:25.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.814+0000 7f2940ff9640 1 -- 172.21.15.16:0/2655844437 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f292c0ef620 con 0x7f2944103650 2024-09-12T02:03:25.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.814+0000 7f29437fe640 1 -- 172.21.15.16:0/1652274281 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55627f15ad40 msgr2=0x55627f15d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:25.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.814+0000 7f29437fe640 1 --2- 172.21.15.16:0/1652274281 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55627f15ad40 0x55627f15d130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:25.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.814+0000 7f29437fe640 1 -- 172.21.15.16:0/1652274281 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55627f150480 msgr2=0x55627f15a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:25.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.814+0000 7f29437fe640 1 --2- 172.21.15.16:0/1652274281 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55627f150480 0x55627f15a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:25.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.814+0000 7f29437fe640 1 -- 172.21.15.16:0/1652274281 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55627f0691f0 con 0x55627f14fb60 2024-09-12T02:03:25.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.815+0000 7f29437fe640 1 --2- 172.21.15.16:0/1652274281 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55627f14fb60 0x55627f14ff40 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f293c011510 tx=0x7f293c008f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=382e0340c8f7f0b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:25.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.815+0000 7f29397fa640 1 -- 172.21.15.16:0/1652274281 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f293c020070 con 0x55627f14fb60 2024-09-12T02:03:25.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.815+0000 7f29397fa640 1 -- 172.21.15.16:0/1652274281 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f293c00fc10 con 0x55627f14fb60 2024-09-12T02:03:25.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.815+0000 7f29397fa640 1 -- 172.21.15.16:0/1652274281 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f293c0193b0 con 0x55627f14fb60 2024-09-12T02:03:25.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.816+0000 7f294b980740 1 -- 172.21.15.16:0/1652274281 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55627f14fb60 msgr2=0x55627f14ff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:25.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.816+0000 7f29437fe640 1 --2- 172.21.15.16:0/2655844437 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f29300a2b80 0x7f29300a5040 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f293c01d000 tx=0x7f293c017000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:25.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.816+0000 7f294b980740 1 --2- 172.21.15.16:0/1652274281 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55627f14fb60 0x55627f14ff40 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f293c011510 tx=0x7f293c008f90 comp rx=0 tx=0).stop 2024-09-12T02:03:25.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.816+0000 7f294b980740 1 -- 172.21.15.16:0/1652274281 shutdown_connections 2024-09-12T02:03:25.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.816+0000 7f294b980740 1 --2- 172.21.15.16:0/1652274281 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55627f15ad40 0x55627f15d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:25.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.816+0000 7f294b980740 1 --2- 172.21.15.16:0/1652274281 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55627f150480 0x55627f15a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:25.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.816+0000 7f294b980740 1 --2- 172.21.15.16:0/1652274281 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55627f14fb60 0x55627f14ff40 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:25.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.816+0000 7f294b980740 1 -- 172.21.15.16:0/1652274281 >> 172.21.15.16:0/1652274281 conn(0x55627f14c0f0 msgr2=0x55627f14c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:25.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.816+0000 7f294b980740 1 -- 172.21.15.16:0/1652274281 shutdown_connections 2024-09-12T02:03:25.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.816+0000 7f294b980740 1 -- 172.21.15.16:0/1652274281 wait complete. 2024-09-12T02:03:25.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.816+0000 7f294b980740 1 -- start start 2024-09-12T02:03:25.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.816+0000 7f294b980740 1 Processor -- start 2024-09-12T02:03:25.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.816+0000 7f294b980740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55627f14fb60 0x55627f157bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:25.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.816+0000 7f294b980740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55627f150480 0x55627f1580f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:25.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.816+0000 7f294b980740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55627f15ad40 0x55627f158630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:25.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.816+0000 7f294b980740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55627f15d910 con 0x55627f14fb60 2024-09-12T02:03:25.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.816+0000 7f294b980740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55627f06ff10 con 0x55627f150480 2024-09-12T02:03:25.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.816+0000 7f294b980740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55627f15da90 con 0x55627f15ad40 2024-09-12T02:03:25.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.817+0000 7f294b980740 10 client.0 _ll_register_callbacks cb 0x55627f038a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:03:25.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.817+0000 7f2942ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55627f14fb60 0x55627f157bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:25.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.817+0000 7f2942ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55627f14fb60 0x55627f157bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52192/0 (socket says 172.21.15.16:52192) 2024-09-12T02:03:25.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.817+0000 7f2942ffd640 1 -- 172.21.15.16:0/1966966041 learned_addr learned my addr 172.21.15.16:0/1966966041 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:03:25.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.817+0000 7f293a7fc640 1 --2- 172.21.15.16:0/1966966041 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55627f150480 0x55627f1580f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:25.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.817+0000 7f29437fe640 1 --2- 172.21.15.16:0/1966966041 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55627f15ad40 0x55627f158630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:25.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.817+0000 7f2942ffd640 1 -- 172.21.15.16:0/1966966041 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55627f15ad40 msgr2=0x55627f158630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:25.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.817+0000 7f2942ffd640 1 --2- 172.21.15.16:0/1966966041 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55627f15ad40 0x55627f158630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:25.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.817+0000 7f2942ffd640 1 -- 172.21.15.16:0/1966966041 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55627f150480 msgr2=0x55627f1580f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:25.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.817+0000 7f2942ffd640 1 --2- 172.21.15.16:0/1966966041 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55627f150480 0x55627f1580f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:25.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.817+0000 7f2942ffd640 1 -- 172.21.15.16:0/1966966041 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55627ee266a0 con 0x55627f14fb60 2024-09-12T02:03:25.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.817+0000 7f29437fe640 1 --2- 172.21.15.16:0/1966966041 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55627f15ad40 0x55627f158630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:03:25.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.817+0000 7f2942ffd640 1 --2- 172.21.15.16:0/1966966041 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55627f14fb60 0x55627f157bb0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f293400ba10 tx=0x7f293400be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee7f7f9b2f2afa94 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:25.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.817+0000 7f292a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:03:25.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.817+0000 7f292a7fc640 1 -- 172.21.15.16:0/1966966041 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2934003e20 con 0x55627f14fb60 2024-09-12T02:03:25.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.817+0000 7f292a7fc640 1 -- 172.21.15.16:0/1966966041 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f2934011440 con 0x55627f14fb60 2024-09-12T02:03:25.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.817+0000 7f294b980740 1 -- 172.21.15.16:0/1966966041 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f293c00fc10 con 0x55627f14fb60 2024-09-12T02:03:25.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.818+0000 7f292a7fc640 1 -- 172.21.15.16:0/1966966041 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2934011640 con 0x55627f14fb60 2024-09-12T02:03:25.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.818+0000 7f294b980740 10 client.15792 fetch_fsmap learned FSMap version 25 2024-09-12T02:03:25.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.818+0000 7f294b980740 1 -- 172.21.15.16:0/1966966041 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55627f169560 con 0x55627f14fb60 2024-09-12T02:03:25.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.818+0000 7f292a7fc640 1 -- 172.21.15.16:0/1966966041 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f293c00fc10 con 0x55627f14fb60 2024-09-12T02:03:25.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.818+0000 7f292a7fc640 1 -- 172.21.15.16:0/1966966041 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f293401ba20 con 0x55627f14fb60 2024-09-12T02:03:25.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.818+0000 7f294b980740 10 client.15792 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:03:25.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.818+0000 7f294b980740 10 client.15792 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:03:25.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.818+0000 7f294b980740 20 client.15792 populate_metadata read hostname 'smithi016' 2024-09-12T02:03:25.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.818+0000 7f294b980740 1 --2- 172.21.15.16:0/1966966041 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55627f16a970 0x55627f16cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:25.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.818+0000 7f294b980740 4 client.15792 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:03:25.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.818+0000 7f294b980740 1 -- 172.21.15.16:0/1966966041 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55627f0c27f0 con 0x55627f16a970 2024-09-12T02:03:25.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.818+0000 7f293a7fc640 1 --2- 172.21.15.16:0/1966966041 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55627f16a970 0x55627f16cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:25.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.819+0000 7f293a7fc640 1 --2- 172.21.15.16:0/1966966041 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55627f16a970 0x55627f16cd80 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=633052dde660fa83 server_cookie=3b7725062863c7ad in_seq=0 out_seq=0 2024-09-12T02:03:25.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.819+0000 7f292a7fc640 10 client.15792 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:03:25.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.819+0000 7f292a7fc640 1 -- 172.21.15.16:0/1966966041 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55627f0c27f0 con 0x55627f16a970 2024-09-12T02:03:25.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.819+0000 7f292a7fc640 10 client.15792 handle_command_reply: tid=0 2024-09-12T02:03:25.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.819+0000 7f294b980740 1 client.15792 shutdown 2024-09-12T02:03:25.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.819+0000 7f294b980740 1 -- 172.21.15.16:0/1966966041 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55627f14fb60 msgr2=0x55627f157bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:25.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.819+0000 7f294b980740 1 --2- 172.21.15.16:0/1966966041 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55627f14fb60 0x55627f157bb0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f293400ba10 tx=0x7f293400be10 comp rx=0 tx=0).stop 2024-09-12T02:03:25.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.820+0000 7f294b980740 1 -- 172.21.15.16:0/1966966041 shutdown_connections 2024-09-12T02:03:25.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.820+0000 7f294b980740 1 --2- 172.21.15.16:0/1966966041 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55627f16a970 0x55627f16cd80 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:25.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.820+0000 7f294b980740 1 --2- 172.21.15.16:0/1966966041 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55627f15ad40 0x55627f158630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:25.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.820+0000 7f292a7fc640 0 client.15792 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:03:25.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.820+0000 7f294b980740 1 --2- 172.21.15.16:0/1966966041 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55627f150480 0x55627f1580f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:25.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.820+0000 7f294b980740 1 --2- 172.21.15.16:0/1966966041 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55627f14fb60 0x55627f157bb0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:25.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.820+0000 7f294b980740 1 -- 172.21.15.16:0/1966966041 >> 172.21.15.16:0/1966966041 conn(0x55627f14c0f0 msgr2=0x55627f14b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:25.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.820+0000 7f294b980740 1 -- 172.21.15.16:0/1966966041 shutdown_connections 2024-09-12T02:03:25.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.820+0000 7f294b980740 1 -- 172.21.15.16:0/1966966041 wait complete. 2024-09-12T02:03:25.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.820+0000 7f294b980740 20 client.15792 trim_cache size 0 max 16384 2024-09-12T02:03:25.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.837+0000 7f294b980740 1 Processor -- start 2024-09-12T02:03:25.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.837+0000 7f294b980740 1 -- start start 2024-09-12T02:03:25.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.837+0000 7f294b980740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55627f14fb60 0x55627f152610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:25.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.837+0000 7f294b980740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55627f150480 0x55627f159a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:25.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.837+0000 7f294b980740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55627f15ad40 0x55627f172650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:25.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.837+0000 7f294b980740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f29441b1e70 con 0x55627f150480 2024-09-12T02:03:25.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.837+0000 7f294b980740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f29441af740 con 0x55627f15ad40 2024-09-12T02:03:25.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.837+0000 7f294b980740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f29441ad5b0 con 0x55627f14fb60 2024-09-12T02:03:25.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.838+0000 7f293a7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55627f150480 0x55627f159a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:25.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.838+0000 7f293a7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55627f150480 0x55627f159a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52204/0 (socket says 172.21.15.16:52204) 2024-09-12T02:03:25.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.838+0000 7f293a7fc640 1 -- 172.21.15.16:0/1904135028 learned_addr learned my addr 172.21.15.16:0/1904135028 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:03:25.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.838+0000 7f2942ffd640 1 --2- 172.21.15.16:0/1904135028 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55627f14fb60 0x55627f152610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:25.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.838+0000 7f29437fe640 1 --2- 172.21.15.16:0/1904135028 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55627f15ad40 0x55627f172650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:25.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.838+0000 7f293a7fc640 1 -- 172.21.15.16:0/1904135028 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55627f14fb60 msgr2=0x55627f152610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:25.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.838+0000 7f293a7fc640 1 --2- 172.21.15.16:0/1904135028 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55627f14fb60 0x55627f152610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:25.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.838+0000 7f293a7fc640 1 -- 172.21.15.16:0/1904135028 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55627f15ad40 msgr2=0x55627f172650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:25.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.838+0000 7f293a7fc640 1 --2- 172.21.15.16:0/1904135028 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55627f15ad40 0x55627f172650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:25.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.838+0000 7f293a7fc640 1 -- 172.21.15.16:0/1904135028 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55627f14bf20 con 0x55627f150480 2024-09-12T02:03:25.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.838+0000 7f2942ffd640 1 --2- 172.21.15.16:0/1904135028 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55627f14fb60 0x55627f152610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:03:25.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.838+0000 7f29437fe640 1 --2- 172.21.15.16:0/1904135028 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55627f15ad40 0x55627f172650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:03:25.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.838+0000 7f293a7fc640 1 --2- 172.21.15.16:0/1904135028 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55627f150480 0x55627f159a10 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f292c0ed120 tx=0x7f292c049dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82920e1cfb733cdc server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:25.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.838+0000 7f2938ff9640 1 -- 172.21.15.16:0/1904135028 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f292c049040 con 0x55627f150480 2024-09-12T02:03:25.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.838+0000 7f2938ff9640 1 -- 172.21.15.16:0/1904135028 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f292c0d8070 con 0x55627f150480 2024-09-12T02:03:25.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.838+0000 7f2938ff9640 1 -- 172.21.15.16:0/1904135028 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f292c0c4bb0 con 0x55627f150480 2024-09-12T02:03:25.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.839+0000 7f294b980740 1 -- 172.21.15.16:0/1904135028 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55627f150480 msgr2=0x55627f159a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:25.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.839+0000 7f294b980740 1 --2- 172.21.15.16:0/1904135028 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55627f150480 0x55627f159a10 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f292c0ed120 tx=0x7f292c049dd0 comp rx=0 tx=0).stop 2024-09-12T02:03:25.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.839+0000 7f294b980740 1 -- 172.21.15.16:0/1904135028 shutdown_connections 2024-09-12T02:03:25.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.839+0000 7f294b980740 1 --2- 172.21.15.16:0/1904135028 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55627f15ad40 0x55627f172650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:25.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.839+0000 7f294b980740 1 --2- 172.21.15.16:0/1904135028 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55627f150480 0x55627f159a10 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:25.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.839+0000 7f294b980740 1 --2- 172.21.15.16:0/1904135028 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55627f14fb60 0x55627f152610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:25.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.839+0000 7f294b980740 1 -- 172.21.15.16:0/1904135028 >> 172.21.15.16:0/1904135028 conn(0x55627f14c0f0 msgr2=0x55627f159300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:25.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.839+0000 7f294b980740 1 -- 172.21.15.16:0/1904135028 shutdown_connections 2024-09-12T02:03:25.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.839+0000 7f294b980740 1 -- 172.21.15.16:0/1904135028 wait complete. 2024-09-12T02:03:25.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.839+0000 7f294b980740 1 -- start start 2024-09-12T02:03:25.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.840+0000 7f294b980740 1 Processor -- start 2024-09-12T02:03:25.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.840+0000 7f294b980740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55627f14fb60 0x55627f15e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:25.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.840+0000 7f294b980740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55627f150480 0x55627f15ef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:25.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.840+0000 7f294b980740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55627f15ad40 0x55627f15f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:25.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.840+0000 7f294b980740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f29441adc00 con 0x55627f150480 2024-09-12T02:03:25.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.840+0000 7f294b980740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f29441af070 con 0x55627f15ad40 2024-09-12T02:03:25.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.840+0000 7f294b980740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f29441ae2d0 con 0x55627f14fb60 2024-09-12T02:03:25.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.840+0000 7f294b980740 10 client.0 _ll_register_callbacks cb 0x7f29340042b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:03:25.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.840+0000 7f293a7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55627f150480 0x55627f15ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:25.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.840+0000 7f293a7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55627f150480 0x55627f15ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52216/0 (socket says 172.21.15.16:52216) 2024-09-12T02:03:25.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.840+0000 7f293a7fc640 1 -- 172.21.15.16:0/3861031350 learned_addr learned my addr 172.21.15.16:0/3861031350 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:03:25.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.840+0000 7f2942ffd640 1 --2- 172.21.15.16:0/3861031350 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55627f14fb60 0x55627f15e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:25.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.840+0000 7f29437fe640 1 --2- 172.21.15.16:0/3861031350 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55627f15ad40 0x55627f15f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:25.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.840+0000 7f293a7fc640 1 -- 172.21.15.16:0/3861031350 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55627f14fb60 msgr2=0x55627f15e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:25.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.840+0000 7f293a7fc640 1 --2- 172.21.15.16:0/3861031350 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55627f14fb60 0x55627f15e9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:25.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.840+0000 7f293a7fc640 1 -- 172.21.15.16:0/3861031350 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55627f15ad40 msgr2=0x55627f15f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:25.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.840+0000 7f293a7fc640 1 --2- 172.21.15.16:0/3861031350 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55627f15ad40 0x55627f15f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:25.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.840+0000 7f293a7fc640 1 -- 172.21.15.16:0/3861031350 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55627f1d4620 con 0x55627f150480 2024-09-12T02:03:25.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.840+0000 7f29437fe640 1 --2- 172.21.15.16:0/3861031350 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55627f15ad40 0x55627f15f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:03:25.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.840+0000 7f293a7fc640 1 --2- 172.21.15.16:0/3861031350 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55627f150480 0x55627f15ef10 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f292c0b82d0 tx=0x7f292c0dc0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=617ca77df7bf4f04 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:25.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.840+0000 7f292a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:03:25.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.841+0000 7f292a7fc640 1 -- 172.21.15.16:0/3861031350 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f292c049070 con 0x55627f150480 2024-09-12T02:03:25.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.841+0000 7f292a7fc640 1 -- 172.21.15.16:0/3861031350 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f292c0b85b0 con 0x55627f150480 2024-09-12T02:03:25.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.841+0000 7f294b980740 1 -- 172.21.15.16:0/3861031350 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f292c0d8070 con 0x55627f150480 2024-09-12T02:03:25.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.841+0000 7f292a7fc640 1 -- 172.21.15.16:0/3861031350 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f292c0bc640 con 0x55627f150480 2024-09-12T02:03:25.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.841+0000 7f294b980740 10 client.15798 fetch_fsmap learned FSMap version 25 2024-09-12T02:03:25.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.841+0000 7f294b980740 1 -- 172.21.15.16:0/3861031350 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55627f1d5880 con 0x55627f150480 2024-09-12T02:03:25.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.841+0000 7f292a7fc640 1 -- 172.21.15.16:0/3861031350 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f292c0d8070 con 0x55627f150480 2024-09-12T02:03:25.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.841+0000 7f292a7fc640 1 -- 172.21.15.16:0/3861031350 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f292c0c47b0 con 0x55627f150480 2024-09-12T02:03:25.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.841+0000 7f294b980740 10 client.15798 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:03:25.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.841+0000 7f294b980740 10 client.15798 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:03:25.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.841+0000 7f294b980740 20 client.15798 populate_metadata read hostname 'smithi016' 2024-09-12T02:03:25.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.841+0000 7f294b980740 1 --2- 172.21.15.16:0/3861031350 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55627f1d5e90 0x55627f1d82b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:25.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.841+0000 7f294b980740 4 client.15798 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-12T02:03:25.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.841+0000 7f294b980740 1 -- 172.21.15.16:0/3861031350 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55627f0bcd00 con 0x55627f1d5e90 2024-09-12T02:03:25.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.841+0000 7f2942ffd640 1 --2- 172.21.15.16:0/3861031350 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55627f1d5e90 0x55627f1d82b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:25.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.842+0000 7f2942ffd640 1 --2- 172.21.15.16:0/3861031350 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55627f1d5e90 0x55627f1d82b0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=66460bebe85ae75f server_cookie=ea2a9dbdff0d9541 in_seq=0 out_seq=0 2024-09-12T02:03:25.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.842+0000 7f292a7fc640 10 client.15798 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:03:25.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.843+0000 7f292a7fc640 1 -- 172.21.15.16:0/3861031350 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55627f0bcd00 con 0x55627f1d5e90 2024-09-12T02:03:25.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.843+0000 7f292a7fc640 10 client.15798 handle_command_reply: tid=0 2024-09-12T02:03:25.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.843+0000 7f294b980740 1 client.15798 shutdown 2024-09-12T02:03:25.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.843+0000 7f294b980740 1 -- 172.21.15.16:0/3861031350 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55627f150480 msgr2=0x55627f15ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:25.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.843+0000 7f294b980740 1 --2- 172.21.15.16:0/3861031350 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55627f150480 0x55627f15ef10 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f292c0b82d0 tx=0x7f292c0dc0b0 comp rx=0 tx=0).stop 2024-09-12T02:03:25.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.843+0000 7f294b980740 1 -- 172.21.15.16:0/3861031350 shutdown_connections 2024-09-12T02:03:25.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.843+0000 7f294b980740 1 --2- 172.21.15.16:0/3861031350 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55627f1d5e90 0x55627f1d82b0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:25.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.843+0000 7f294b980740 1 --2- 172.21.15.16:0/3861031350 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55627f15ad40 0x55627f15f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:25.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.843+0000 7f294b980740 1 --2- 172.21.15.16:0/3861031350 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55627f150480 0x55627f15ef10 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:25.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.843+0000 7f294b980740 1 --2- 172.21.15.16:0/3861031350 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55627f14fb60 0x55627f15e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:25.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.843+0000 7f292a7fc640 0 client.15798 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:03:25.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.843+0000 7f294b980740 1 -- 172.21.15.16:0/3861031350 >> 172.21.15.16:0/3861031350 conn(0x55627f14c0f0 msgr2=0x55627f14b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:25.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.843+0000 7f294b980740 1 -- 172.21.15.16:0/3861031350 shutdown_connections 2024-09-12T02:03:25.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.843+0000 7f294b980740 1 -- 172.21.15.16:0/3861031350 wait complete. 2024-09-12T02:03:25.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.843+0000 7f294b980740 20 client.15798 trim_cache size 0 max 16384 2024-09-12T02:03:25.849 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T02:03:25.849 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-09-12T02:03:25.849 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "4cbde434-b6ca-4d36-aa57-d16a8f269be9", 2024-09-12T02:03:25.849 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-09-12T02:03:25.849 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T02:03:25.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.844+0000 7f29495a7640 1 -- 172.21.15.16:0/2655844437 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f29300a2b80 msgr2=0x7f29300a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:25.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.844+0000 7f29495a7640 1 --2- 172.21.15.16:0/2655844437 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f29300a2b80 0x7f29300a5040 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f293c01d000 tx=0x7f293c017000 comp rx=0 tx=0).stop 2024-09-12T02:03:25.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.844+0000 7f29495a7640 1 -- 172.21.15.16:0/2655844437 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2944103650 msgr2=0x7f2944100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:25.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.844+0000 7f29495a7640 1 --2- 172.21.15.16:0/2655844437 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2944103650 0x7f2944100ed0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f292c006010 tx=0x7f292c0042f0 comp rx=0 tx=0).stop 2024-09-12T02:03:25.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.844+0000 7f29495a7640 1 -- 172.21.15.16:0/2655844437 shutdown_connections 2024-09-12T02:03:25.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.844+0000 7f29495a7640 1 --2- 172.21.15.16:0/2655844437 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f29300a2b80 0x7f29300a5040 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:25.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.844+0000 7f29495a7640 1 --2- 172.21.15.16:0/2655844437 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f294410c160 0x7f2944101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:25.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.844+0000 7f29495a7640 1 --2- 172.21.15.16:0/2655844437 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2944103650 0x7f2944100ed0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:25.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.844+0000 7f29495a7640 1 --2- 172.21.15.16:0/2655844437 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2944102d30 0x7f2944100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:25.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.844+0000 7f29495a7640 1 -- 172.21.15.16:0/2655844437 >> 172.21.15.16:0/2655844437 conn(0x7f29440fc7d0 msgr2=0x7f29440fd4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:25.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.844+0000 7f29495a7640 1 -- 172.21.15.16:0/2655844437 shutdown_connections 2024-09-12T02:03:25.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:25.844+0000 7f29495a7640 1 -- 172.21.15.16:0/2655844437 wait complete. 2024-09-12T02:03:25.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:03:25 smithi184 ceph-mon[35593]: pgmap v380: 97 pgs: 97 active+clean; 960 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-09-12T02:03:25.918 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:03:25 smithi033 ceph-mon[35332]: pgmap v380: 97 pgs: 97 active+clean; 960 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-09-12T02:03:26.539 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T02:03:26.794 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:03:26.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:03:26 smithi184 ceph-mon[35593]: scrub queued for path: / 2024-09-12T02:03:26.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:03:26 smithi184 ceph-mon[35593]: scrub summary: idle+waiting paths [/] 2024-09-12T02:03:26.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:03:26 smithi184 ceph-mon[35593]: scrub summary: active paths [/] 2024-09-12T02:03:26.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:03:26 smithi184 ceph-mon[35593]: scrub summary: idle+waiting paths [/] 2024-09-12T02:03:26.878 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:03:26 smithi184 ceph-mon[35593]: scrub summary: idle 2024-09-12T02:03:26.916 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:03:26 smithi016 ceph-mon[30947]: scrub queued for path: / 2024-09-12T02:03:26.916 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:03:26 smithi016 ceph-mon[30947]: scrub summary: idle+waiting paths [/] 2024-09-12T02:03:26.916 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:03:26 smithi016 ceph-mon[30947]: scrub summary: active paths [/] 2024-09-12T02:03:26.916 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:03:26 smithi016 ceph-mon[30947]: scrub summary: idle+waiting paths [/] 2024-09-12T02:03:26.916 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:03:26 smithi016 ceph-mon[30947]: scrub summary: idle 2024-09-12T02:03:26.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:03:26 smithi033 ceph-mon[35332]: scrub queued for path: / 2024-09-12T02:03:26.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:03:26 smithi033 ceph-mon[35332]: scrub summary: idle+waiting paths [/] 2024-09-12T02:03:26.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:03:26 smithi033 ceph-mon[35332]: scrub summary: active paths [/] 2024-09-12T02:03:26.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:03:26 smithi033 ceph-mon[35332]: scrub summary: idle+waiting paths [/] 2024-09-12T02:03:26.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:03:26 smithi033 ceph-mon[35332]: scrub summary: idle 2024-09-12T02:03:27.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.282+0000 7f5c4e632640 1 -- 172.21.15.16:0/460758186 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c48106f10 msgr2=0x7f5c4810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:27.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.282+0000 7f5c4e632640 1 --2- 172.21.15.16:0/460758186 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c48106f10 0x7f5c4810ea90 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f5c40009ca0 tx=0x7f5c4002a8c0 comp rx=0 tx=0).stop 2024-09-12T02:03:27.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.282+0000 7f5c4e632640 1 -- 172.21.15.16:0/460758186 shutdown_connections 2024-09-12T02:03:27.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.282+0000 7f5c4e632640 1 --2- 172.21.15.16:0/460758186 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c48106f10 0x7f5c4810ea90 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:27.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.282+0000 7f5c4e632640 1 --2- 172.21.15.16:0/460758186 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c48102790 0x7f5c481067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:27.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.282+0000 7f5c4e632640 1 --2- 172.21.15.16:0/460758186 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5c48101de0 0x7f5c481021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:27.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.282+0000 7f5c4e632640 1 -- 172.21.15.16:0/460758186 >> 172.21.15.16:0/460758186 conn(0x7f5c480fd8c0 msgr2=0x7f5c480ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:27.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.282+0000 7f5c4e632640 1 -- 172.21.15.16:0/460758186 shutdown_connections 2024-09-12T02:03:27.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.282+0000 7f5c4e632640 1 -- 172.21.15.16:0/460758186 wait complete. 2024-09-12T02:03:27.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.283+0000 7f5c4e632640 1 Processor -- start 2024-09-12T02:03:27.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.283+0000 7f5c4e632640 1 -- start start 2024-09-12T02:03:27.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.283+0000 7f5c4e632640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5c48101de0 0x7f5c480744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:27.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.283+0000 7f5c4e632640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c48102790 0x7f5c48074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:27.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.283+0000 7f5c4e632640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c48106f10 0x7f5c48074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:27.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.283+0000 7f5c4e632640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5c48111f90 con 0x7f5c48106f10 2024-09-12T02:03:27.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.283+0000 7f5c4e632640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5c48111e10 con 0x7f5c48102790 2024-09-12T02:03:27.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.283+0000 7f5c4e632640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5c48112110 con 0x7f5c48101de0 2024-09-12T02:03:27.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.283+0000 7f5c4cbcf640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c48106f10 0x7f5c48074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:27.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.283+0000 7f5c4cbcf640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c48106f10 0x7f5c48074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52236/0 (socket says 172.21.15.16:52236) 2024-09-12T02:03:27.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.283+0000 7f5c4cbcf640 1 -- 172.21.15.16:0/1639947066 learned_addr learned my addr 172.21.15.16:0/1639947066 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:03:27.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.283+0000 7f5c47fff640 1 --2- 172.21.15.16:0/1639947066 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5c48101de0 0x7f5c480744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:27.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.283+0000 7f5c3ffff640 1 --2- 172.21.15.16:0/1639947066 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c48102790 0x7f5c48074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:27.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.283+0000 7f5c4cbcf640 1 -- 172.21.15.16:0/1639947066 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5c48101de0 msgr2=0x7f5c480744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:27.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.283+0000 7f5c4cbcf640 1 --2- 172.21.15.16:0/1639947066 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5c48101de0 0x7f5c480744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:27.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.283+0000 7f5c4cbcf640 1 -- 172.21.15.16:0/1639947066 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c48102790 msgr2=0x7f5c48074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:27.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.283+0000 7f5c4cbcf640 1 --2- 172.21.15.16:0/1639947066 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c48102790 0x7f5c48074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:27.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.283+0000 7f5c4cbcf640 1 -- 172.21.15.16:0/1639947066 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c481a6930 con 0x7f5c48106f10 2024-09-12T02:03:27.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.284+0000 7f5c4cbcf640 1 --2- 172.21.15.16:0/1639947066 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c48106f10 0x7f5c48074f70 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f5c40031a20 tx=0x7f5c40031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:27.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.284+0000 7f5c45ffb640 1 -- 172.21.15.16:0/1639947066 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c40003c30 con 0x7f5c48106f10 2024-09-12T02:03:27.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.284+0000 7f5c45ffb640 1 -- 172.21.15.16:0/1639947066 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c40003dd0 con 0x7f5c48106f10 2024-09-12T02:03:27.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.284+0000 7f5c45ffb640 1 -- 172.21.15.16:0/1639947066 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c4002fcf0 con 0x7f5c48106f10 2024-09-12T02:03:27.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.284+0000 7f5c4e632640 1 -- 172.21.15.16:0/1639947066 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c481a6bc0 con 0x7f5c48106f10 2024-09-12T02:03:27.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.284+0000 7f5c4e632640 1 -- 172.21.15.16:0/1639947066 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c481a7150 con 0x7f5c48106f10 2024-09-12T02:03:27.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.286+0000 7f5c45ffb640 1 -- 172.21.15.16:0/1639947066 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5c40004050 con 0x7f5c48106f10 2024-09-12T02:03:27.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.286+0000 7f5c50a0b740 1 Processor -- start 2024-09-12T02:03:27.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.286+0000 7f5c50a0b740 1 -- start start 2024-09-12T02:03:27.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.287+0000 7f5c50a0b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5653291c54a0 0x5653291c5880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:27.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.287+0000 7f5c50a0b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653291c5dc0 0x5653291d0140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:27.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.287+0000 7f5c50a0b740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5653291d0680 0x5653291d2a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:27.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.287+0000 7f5c50a0b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56532912b2e0 con 0x5653291c5dc0 2024-09-12T02:03:27.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.287+0000 7f5c50a0b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5653290d2ce0 con 0x5653291c54a0 2024-09-12T02:03:27.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.287+0000 7f5c50a0b740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5653291272a0 con 0x5653291d0680 2024-09-12T02:03:27.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.287+0000 7f5c47fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653291c5dc0 0x5653291d0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:27.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.287+0000 7f5c47fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653291c5dc0 0x5653291d0140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52246/0 (socket says 172.21.15.16:52246) 2024-09-12T02:03:27.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.287+0000 7f5c47fff640 1 -- 172.21.15.16:0/2391368845 learned_addr learned my addr 172.21.15.16:0/2391368845 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:03:27.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.287+0000 7f5c3ffff640 1 --2- 172.21.15.16:0/2391368845 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5653291c54a0 0x5653291c5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:27.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.287+0000 7f5c3ffff640 1 --2- 172.21.15.16:0/2391368845 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5653291d0680 0x5653291d2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:27.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.287+0000 7f5c45ffb640 1 --2- 172.21.15.16:0/1639947066 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f5c340a2c00 0x7f5c340a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:27.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.287+0000 7f5c47fff640 1 -- 172.21.15.16:0/2391368845 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5653291d0680 msgr2=0x5653291d2a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:27.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.287+0000 7f5c47fff640 1 --2- 172.21.15.16:0/2391368845 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5653291d0680 0x5653291d2a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:27.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.287+0000 7f5c47fff640 1 -- 172.21.15.16:0/2391368845 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5653291c54a0 msgr2=0x5653291c5880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:27.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.287+0000 7f5c47fff640 1 --2- 172.21.15.16:0/2391368845 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5653291c54a0 0x5653291c5880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:27.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.287+0000 7f5c47fff640 1 -- 172.21.15.16:0/2391368845 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653290b7440 con 0x5653291c5dc0 2024-09-12T02:03:27.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.287+0000 7f5c4cbcf640 1 --2- 172.21.15.16:0/1639947066 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f5c340a2c00 0x7f5c340a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:27.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.287+0000 7f5c45ffb640 1 -- 172.21.15.16:0/1639947066 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f5c400f0370 con 0x7f5c48106f10 2024-09-12T02:03:27.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.287+0000 7f5c47fff640 1 --2- 172.21.15.16:0/2391368845 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653291c5dc0 0x5653291d0140 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f5c3800b820 tx=0x7f5c3800bce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47938f3b32f3e812 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:27.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.288+0000 7f5c3e7fc640 1 -- 172.21.15.16:0/2391368845 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c3800d950 con 0x5653291c5dc0 2024-09-12T02:03:27.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.288+0000 7f5c3e7fc640 1 -- 172.21.15.16:0/2391368845 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c38019070 con 0x5653291c5dc0 2024-09-12T02:03:27.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.288+0000 7f5c4cbcf640 1 --2- 172.21.15.16:0/1639947066 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f5c340a2c00 0x7f5c340a50c0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f5c40009430 tx=0x7f5c40087bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:27.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.288+0000 7f5c3e7fc640 1 -- 172.21.15.16:0/2391368845 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c380123e0 con 0x5653291c5dc0 2024-09-12T02:03:27.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.288+0000 7f5c50a0b740 1 -- 172.21.15.16:0/2391368845 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653291c5dc0 msgr2=0x5653291d0140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:27.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.288+0000 7f5c50a0b740 1 --2- 172.21.15.16:0/2391368845 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653291c5dc0 0x5653291d0140 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f5c3800b820 tx=0x7f5c3800bce0 comp rx=0 tx=0).stop 2024-09-12T02:03:27.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.288+0000 7f5c50a0b740 1 -- 172.21.15.16:0/2391368845 shutdown_connections 2024-09-12T02:03:27.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.288+0000 7f5c50a0b740 1 --2- 172.21.15.16:0/2391368845 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5653291d0680 0x5653291d2a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:27.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.288+0000 7f5c50a0b740 1 --2- 172.21.15.16:0/2391368845 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653291c5dc0 0x5653291d0140 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:27.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.288+0000 7f5c50a0b740 1 --2- 172.21.15.16:0/2391368845 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5653291c54a0 0x5653291c5880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:27.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.288+0000 7f5c50a0b740 1 -- 172.21.15.16:0/2391368845 >> 172.21.15.16:0/2391368845 conn(0x5653291c1a30 msgr2=0x5653291c1e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:27.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.289+0000 7f5c50a0b740 1 -- 172.21.15.16:0/2391368845 shutdown_connections 2024-09-12T02:03:27.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.289+0000 7f5c50a0b740 1 -- 172.21.15.16:0/2391368845 wait complete. 2024-09-12T02:03:27.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.289+0000 7f5c50a0b740 1 -- start start 2024-09-12T02:03:27.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.289+0000 7f5c50a0b740 1 Processor -- start 2024-09-12T02:03:27.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.289+0000 7f5c50a0b740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5653291c54a0 0x5653291cffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:27.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.289+0000 7f5c50a0b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5653291c5dc0 0x5653291cd250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:27.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.289+0000 7f5c50a0b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653291d0680 0x5653291cd790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:27.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.289+0000 7f5c50a0b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5653291d3250 con 0x5653291d0680 2024-09-12T02:03:27.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.289+0000 7f5c50a0b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5653290e6050 con 0x5653291c5dc0 2024-09-12T02:03:27.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.289+0000 7f5c50a0b740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5653291d33d0 con 0x5653291c54a0 2024-09-12T02:03:27.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.289+0000 7f5c50a0b740 10 client.0 _ll_register_callbacks cb 0x565329126160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:03:27.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.289+0000 7f5c3ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653291d0680 0x5653291cd790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:27.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.290+0000 7f5c3ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653291d0680 0x5653291cd790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52260/0 (socket says 172.21.15.16:52260) 2024-09-12T02:03:27.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.290+0000 7f5c47fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5653291c5dc0 0x5653291cd250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:27.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.290+0000 7f5c3ffff640 1 -- 172.21.15.16:0/3272130122 learned_addr learned my addr 172.21.15.16:0/3272130122 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:03:27.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.290+0000 7f5c3ffff640 1 --2- 172.21.15.16:0/3272130122 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5653291c54a0 0x5653291cffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:27.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.290+0000 7f5c3ffff640 1 -- 172.21.15.16:0/3272130122 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5653291c54a0 msgr2=0x5653291cffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:27.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.290+0000 7f5c3ffff640 1 --2- 172.21.15.16:0/3272130122 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5653291c54a0 0x5653291cffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:27.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.290+0000 7f5c3ffff640 1 -- 172.21.15.16:0/3272130122 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5653291c5dc0 msgr2=0x5653291cd250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:27.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.290+0000 7f5c3ffff640 1 --2- 172.21.15.16:0/3272130122 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5653291c5dc0 0x5653291cd250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:27.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.290+0000 7f5c3ffff640 1 -- 172.21.15.16:0/3272130122 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565328e9c3c0 con 0x5653291d0680 2024-09-12T02:03:27.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.290+0000 7f5c47fff640 1 --2- 172.21.15.16:0/3272130122 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5653291c5dc0 0x5653291cd250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:03:27.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.290+0000 7f5c3ffff640 1 --2- 172.21.15.16:0/3272130122 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653291d0680 0x5653291cd790 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f5c30014d30 tx=0x7f5c3000c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c450fc52308a3f90 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:27.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.290+0000 7f5c277fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:03:27.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.290+0000 7f5c277fe640 1 -- 172.21.15.16:0/3272130122 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c300140d0 con 0x5653291d0680 2024-09-12T02:03:27.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.290+0000 7f5c277fe640 1 -- 172.21.15.16:0/3272130122 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f5c30014270 con 0x5653291d0680 2024-09-12T02:03:27.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.290+0000 7f5c50a0b740 1 -- 172.21.15.16:0/3272130122 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5c38019070 con 0x5653291d0680 2024-09-12T02:03:27.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.290+0000 7f5c277fe640 1 -- 172.21.15.16:0/3272130122 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5c30004030 con 0x5653291d0680 2024-09-12T02:03:27.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.290+0000 7f5c50a0b740 10 client.15810 fetch_fsmap learned FSMap version 25 2024-09-12T02:03:27.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.290+0000 7f5c50a0b740 1 -- 172.21.15.16:0/3272130122 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5653291dec00 con 0x5653291d0680 2024-09-12T02:03:27.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.291+0000 7f5c277fe640 1 -- 172.21.15.16:0/3272130122 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c38019070 con 0x5653291d0680 2024-09-12T02:03:27.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.291+0000 7f5c277fe640 1 -- 172.21.15.16:0/3272130122 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f5c30020450 con 0x5653291d0680 2024-09-12T02:03:27.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.291+0000 7f5c50a0b740 10 client.15810 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:03:27.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.291+0000 7f5c50a0b740 10 client.15810 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:03:27.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.291+0000 7f5c50a0b740 20 client.15810 populate_metadata read hostname 'smithi016' 2024-09-12T02:03:27.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.291+0000 7f5c50a0b740 1 --2- 172.21.15.16:0/3272130122 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5653291e00c0 0x5653291e24b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:27.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.291+0000 7f5c50a0b740 4 client.15810 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:03:27.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.291+0000 7f5c50a0b740 1 -- 172.21.15.16:0/3272130122 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565329138850 con 0x5653291e00c0 2024-09-12T02:03:27.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.291+0000 7f5c47fff640 1 --2- 172.21.15.16:0/3272130122 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5653291e00c0 0x5653291e24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:27.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.292+0000 7f5c47fff640 1 --2- 172.21.15.16:0/3272130122 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5653291e00c0 0x5653291e24b0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a5bf63c4d9796af server_cookie=67cf902c38c7e372 in_seq=0 out_seq=0 2024-09-12T02:03:27.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.292+0000 7f5c277fe640 10 client.15810 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:03:27.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.292+0000 7f5c277fe640 1 -- 172.21.15.16:0/3272130122 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x565329138850 con 0x5653291e00c0 2024-09-12T02:03:27.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.292+0000 7f5c277fe640 10 client.15810 handle_command_reply: tid=0 2024-09-12T02:03:27.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.292+0000 7f5c50a0b740 1 client.15810 shutdown 2024-09-12T02:03:27.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.293+0000 7f5c50a0b740 1 -- 172.21.15.16:0/3272130122 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653291d0680 msgr2=0x5653291cd790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:27.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.293+0000 7f5c50a0b740 1 --2- 172.21.15.16:0/3272130122 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653291d0680 0x5653291cd790 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f5c30014d30 tx=0x7f5c3000c7e0 comp rx=0 tx=0).stop 2024-09-12T02:03:27.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.293+0000 7f5c50a0b740 1 -- 172.21.15.16:0/3272130122 shutdown_connections 2024-09-12T02:03:27.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.293+0000 7f5c50a0b740 1 --2- 172.21.15.16:0/3272130122 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5653291e00c0 0x5653291e24b0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:27.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.293+0000 7f5c50a0b740 1 --2- 172.21.15.16:0/3272130122 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653291d0680 0x5653291cd790 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:27.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.293+0000 7f5c277fe640 0 client.15810 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:03:27.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.293+0000 7f5c50a0b740 1 --2- 172.21.15.16:0/3272130122 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5653291c5dc0 0x5653291cd250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:27.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.293+0000 7f5c50a0b740 1 --2- 172.21.15.16:0/3272130122 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5653291c54a0 0x5653291cffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:27.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.293+0000 7f5c50a0b740 1 -- 172.21.15.16:0/3272130122 >> 172.21.15.16:0/3272130122 conn(0x5653291c1a30 msgr2=0x5653291c0b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:27.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.293+0000 7f5c50a0b740 1 -- 172.21.15.16:0/3272130122 shutdown_connections 2024-09-12T02:03:27.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.293+0000 7f5c50a0b740 1 -- 172.21.15.16:0/3272130122 wait complete. 2024-09-12T02:03:27.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.293+0000 7f5c50a0b740 20 client.15810 trim_cache size 0 max 16384 2024-09-12T02:03:27.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.310+0000 7f5c50a0b740 1 Processor -- start 2024-09-12T02:03:27.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.310+0000 7f5c50a0b740 1 -- start start 2024-09-12T02:03:27.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.310+0000 7f5c50a0b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653291c1a30 0x5653291dc4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:27.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.310+0000 7f5c50a0b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5653291c54a0 0x5653291c7600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:27.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.310+0000 7f5c50a0b740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5653291c5dc0 0x5653291e7d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:27.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.310+0000 7f5c50a0b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5c481b1f60 con 0x5653291c1a30 2024-09-12T02:03:27.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.310+0000 7f5c50a0b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5c481af830 con 0x5653291c54a0 2024-09-12T02:03:27.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.310+0000 7f5c50a0b740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5c481ad6a0 con 0x5653291c5dc0 2024-09-12T02:03:27.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.310+0000 7f5c3ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653291c1a30 0x5653291dc4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:27.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.310+0000 7f5c3ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653291c1a30 0x5653291dc4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52264/0 (socket says 172.21.15.16:52264) 2024-09-12T02:03:27.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.310+0000 7f5c3ffff640 1 -- 172.21.15.16:0/2165089215 learned_addr learned my addr 172.21.15.16:0/2165089215 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:03:27.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.310+0000 7f5c47fff640 1 --2- 172.21.15.16:0/2165089215 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5653291c54a0 0x5653291c7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:27.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.311+0000 7f5c3ffff640 1 -- 172.21.15.16:0/2165089215 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5653291c5dc0 msgr2=0x5653291e7d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:27.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.311+0000 7f5c3ffff640 1 --2- 172.21.15.16:0/2165089215 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5653291c5dc0 0x5653291e7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:27.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.311+0000 7f5c3ffff640 1 -- 172.21.15.16:0/2165089215 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5653291c54a0 msgr2=0x5653291c7600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:27.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.311+0000 7f5c3ffff640 1 --2- 172.21.15.16:0/2165089215 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5653291c54a0 0x5653291c7600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:27.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.311+0000 7f5c3ffff640 1 -- 172.21.15.16:0/2165089215 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653291c5bf0 con 0x5653291c1a30 2024-09-12T02:03:27.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.311+0000 7f5c3ffff640 1 --2- 172.21.15.16:0/2165089215 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653291c1a30 0x5653291dc4e0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f5c300059f0 tx=0x7f5c3000ca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef2d747c65c328f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:27.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.311+0000 7f5c3dffb640 1 -- 172.21.15.16:0/2165089215 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c30031070 con 0x5653291c1a30 2024-09-12T02:03:27.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.311+0000 7f5c3dffb640 1 -- 172.21.15.16:0/2165089215 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c300140d0 con 0x5653291c1a30 2024-09-12T02:03:27.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.311+0000 7f5c3dffb640 1 -- 172.21.15.16:0/2165089215 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c30005030 con 0x5653291c1a30 2024-09-12T02:03:27.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.312+0000 7f5c50a0b740 1 -- 172.21.15.16:0/2165089215 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653291c1a30 msgr2=0x5653291dc4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:27.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.312+0000 7f5c50a0b740 1 --2- 172.21.15.16:0/2165089215 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653291c1a30 0x5653291dc4e0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f5c300059f0 tx=0x7f5c3000ca20 comp rx=0 tx=0).stop 2024-09-12T02:03:27.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.312+0000 7f5c50a0b740 1 -- 172.21.15.16:0/2165089215 shutdown_connections 2024-09-12T02:03:27.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.312+0000 7f5c50a0b740 1 --2- 172.21.15.16:0/2165089215 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5653291c5dc0 0x5653291e7d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:27.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.312+0000 7f5c50a0b740 1 --2- 172.21.15.16:0/2165089215 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5653291c54a0 0x5653291c7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:27.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.312+0000 7f5c50a0b740 1 --2- 172.21.15.16:0/2165089215 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653291c1a30 0x5653291dc4e0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:27.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.312+0000 7f5c50a0b740 1 -- 172.21.15.16:0/2165089215 >> 172.21.15.16:0/2165089215 conn(0x5653291cd8f0 msgr2=0x5653291c6ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:27.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.312+0000 7f5c50a0b740 1 -- 172.21.15.16:0/2165089215 shutdown_connections 2024-09-12T02:03:27.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.312+0000 7f5c50a0b740 1 -- 172.21.15.16:0/2165089215 wait complete. 2024-09-12T02:03:27.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.312+0000 7f5c50a0b740 1 -- start start 2024-09-12T02:03:27.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.312+0000 7f5c50a0b740 1 Processor -- start 2024-09-12T02:03:27.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.313+0000 7f5c50a0b740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5653291c1a30 0x5653291d8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:27.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.313+0000 7f5c50a0b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653291c54a0 0x5653291d3e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:27.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.313+0000 7f5c50a0b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5653291c5dc0 0x5653291d4340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:27.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.313+0000 7f5c50a0b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5c481adcf0 con 0x5653291c54a0 2024-09-12T02:03:27.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.313+0000 7f5c50a0b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5c481af160 con 0x5653291c5dc0 2024-09-12T02:03:27.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.313+0000 7f5c50a0b740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5c481ae3c0 con 0x5653291c1a30 2024-09-12T02:03:27.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.313+0000 7f5c50a0b740 10 client.0 _ll_register_callbacks cb 0x7f5c30014990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:03:27.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.313+0000 7f5c47fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653291c54a0 0x5653291d3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:27.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.313+0000 7f5c47fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653291c54a0 0x5653291d3e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52278/0 (socket says 172.21.15.16:52278) 2024-09-12T02:03:27.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.313+0000 7f5c47fff640 1 -- 172.21.15.16:0/1222794428 learned_addr learned my addr 172.21.15.16:0/1222794428 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:03:27.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.313+0000 7f5c3ffff640 1 --2- 172.21.15.16:0/1222794428 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5653291c5dc0 0x5653291d4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:27.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.313+0000 7f5c3ffff640 1 --2- 172.21.15.16:0/1222794428 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5653291c1a30 0x5653291d8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:27.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.313+0000 7f5c47fff640 1 -- 172.21.15.16:0/1222794428 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5653291c1a30 msgr2=0x5653291d8ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:27.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.313+0000 7f5c47fff640 1 --2- 172.21.15.16:0/1222794428 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5653291c1a30 0x5653291d8ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:27.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.313+0000 7f5c47fff640 1 -- 172.21.15.16:0/1222794428 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5653291c5dc0 msgr2=0x5653291d4340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:27.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.313+0000 7f5c47fff640 1 --2- 172.21.15.16:0/1222794428 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5653291c5dc0 0x5653291d4340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:27.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.313+0000 7f5c47fff640 1 -- 172.21.15.16:0/1222794428 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5653291d4880 con 0x5653291c54a0 2024-09-12T02:03:27.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.313+0000 7f5c3ffff640 1 --2- 172.21.15.16:0/1222794428 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5653291c1a30 0x5653291d8ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:03:27.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.313+0000 7f5c47fff640 1 --2- 172.21.15.16:0/1222794428 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653291c54a0 0x5653291d3e00 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x5653291c2890 tx=0x7f5c3800b1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89784bf5cba6ee04 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:27.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.313+0000 7f5c277fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:03:27.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.313+0000 7f5c277fe640 1 -- 172.21.15.16:0/1222794428 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c38014b40 con 0x5653291c54a0 2024-09-12T02:03:27.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.313+0000 7f5c277fe640 1 -- 172.21.15.16:0/1222794428 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f5c380144b0 con 0x5653291c54a0 2024-09-12T02:03:27.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.313+0000 7f5c50a0b740 1 -- 172.21.15.16:0/1222794428 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5c300140d0 con 0x5653291c54a0 2024-09-12T02:03:27.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.314+0000 7f5c277fe640 1 -- 172.21.15.16:0/1222794428 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5c380146b0 con 0x5653291c54a0 2024-09-12T02:03:27.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.314+0000 7f5c50a0b740 10 client.15816 fetch_fsmap learned FSMap version 25 2024-09-12T02:03:27.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.314+0000 7f5c50a0b740 1 -- 172.21.15.16:0/1222794428 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56532924aa70 con 0x5653291c54a0 2024-09-12T02:03:27.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.314+0000 7f5c277fe640 1 -- 172.21.15.16:0/1222794428 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c300140d0 con 0x5653291c54a0 2024-09-12T02:03:27.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.314+0000 7f5c277fe640 1 -- 172.21.15.16:0/1222794428 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f5c38013070 con 0x5653291c54a0 2024-09-12T02:03:27.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.314+0000 7f5c50a0b740 10 client.15816 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:03:27.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.314+0000 7f5c50a0b740 10 client.15816 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:03:27.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.314+0000 7f5c50a0b740 20 client.15816 populate_metadata read hostname 'smithi016' 2024-09-12T02:03:27.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.314+0000 7f5c50a0b740 1 --2- 172.21.15.16:0/1222794428 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5653291d0680 0x56532924cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:27.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.314+0000 7f5c50a0b740 4 client.15816 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T02:03:27.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.314+0000 7f5c50a0b740 1 -- 172.21.15.16:0/1222794428 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "scrub status"}) -- 0x565329132d60 con 0x5653291d0680 2024-09-12T02:03:27.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.314+0000 7f5c3ffff640 1 --2- 172.21.15.16:0/1222794428 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5653291d0680 0x56532924cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:27.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.315+0000 7f5c3ffff640 1 --2- 172.21.15.16:0/1222794428 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5653291d0680 0x56532924cfb0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85766787ab36981c server_cookie=c1ad8bf94ace3821 in_seq=0 out_seq=0 2024-09-12T02:03:27.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.315+0000 7f5c277fe640 10 client.15816 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:03:27.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.315+0000 7f5c277fe640 1 -- 172.21.15.16:0/1222794428 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x565329132d60 con 0x5653291d0680 2024-09-12T02:03:27.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.315+0000 7f5c277fe640 10 client.15816 handle_command_reply: tid=0 2024-09-12T02:03:27.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.315+0000 7f5c50a0b740 1 client.15816 shutdown 2024-09-12T02:03:27.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.315+0000 7f5c50a0b740 1 -- 172.21.15.16:0/1222794428 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653291c54a0 msgr2=0x5653291d3e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:27.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.315+0000 7f5c50a0b740 1 --2- 172.21.15.16:0/1222794428 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653291c54a0 0x5653291d3e00 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x5653291c2890 tx=0x7f5c3800b1c0 comp rx=0 tx=0).stop 2024-09-12T02:03:27.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.315+0000 7f5c50a0b740 1 -- 172.21.15.16:0/1222794428 shutdown_connections 2024-09-12T02:03:27.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.315+0000 7f5c50a0b740 1 --2- 172.21.15.16:0/1222794428 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5653291d0680 0x56532924cfb0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:27.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.315+0000 7f5c50a0b740 1 --2- 172.21.15.16:0/1222794428 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5653291c5dc0 0x5653291d4340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:27.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.315+0000 7f5c50a0b740 1 --2- 172.21.15.16:0/1222794428 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653291c54a0 0x5653291d3e00 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:27.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.315+0000 7f5c50a0b740 1 --2- 172.21.15.16:0/1222794428 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5653291c1a30 0x5653291d8ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:27.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.315+0000 7f5c277fe640 0 client.15816 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:03:27.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.315+0000 7f5c50a0b740 1 -- 172.21.15.16:0/1222794428 >> 172.21.15.16:0/1222794428 conn(0x5653291cd8f0 msgr2=0x5653291c6ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:27.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.315+0000 7f5c50a0b740 1 -- 172.21.15.16:0/1222794428 shutdown_connections 2024-09-12T02:03:27.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.315+0000 7f5c50a0b740 1 -- 172.21.15.16:0/1222794428 wait complete. 2024-09-12T02:03:27.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.315+0000 7f5c50a0b740 20 client.15816 trim_cache size 0 max 16384 2024-09-12T02:03:27.321 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T02:03:27.321 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-09-12T02:03:27.321 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {} 2024-09-12T02:03:27.321 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T02:03:27.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.316+0000 7f5c4e632640 1 -- 172.21.15.16:0/1639947066 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f5c340a2c00 msgr2=0x7f5c340a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:27.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.316+0000 7f5c4e632640 1 --2- 172.21.15.16:0/1639947066 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f5c340a2c00 0x7f5c340a50c0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f5c40009430 tx=0x7f5c40087bc0 comp rx=0 tx=0).stop 2024-09-12T02:03:27.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.316+0000 7f5c4e632640 1 -- 172.21.15.16:0/1639947066 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c48106f10 msgr2=0x7f5c48074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:27.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.316+0000 7f5c4e632640 1 --2- 172.21.15.16:0/1639947066 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c48106f10 0x7f5c48074f70 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f5c40031a20 tx=0x7f5c40031a50 comp rx=0 tx=0).stop 2024-09-12T02:03:27.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.316+0000 7f5c4e632640 1 -- 172.21.15.16:0/1639947066 shutdown_connections 2024-09-12T02:03:27.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.316+0000 7f5c4e632640 1 --2- 172.21.15.16:0/1639947066 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f5c340a2c00 0x7f5c340a50c0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:27.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.316+0000 7f5c4e632640 1 --2- 172.21.15.16:0/1639947066 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c48106f10 0x7f5c48074f70 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:27.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.316+0000 7f5c4e632640 1 --2- 172.21.15.16:0/1639947066 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c48102790 0x7f5c48074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:27.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.316+0000 7f5c4e632640 1 --2- 172.21.15.16:0/1639947066 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5c48101de0 0x7f5c480744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:27.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.316+0000 7f5c4e632640 1 -- 172.21.15.16:0/1639947066 >> 172.21.15.16:0/1639947066 conn(0x7f5c480fd8c0 msgr2=0x7f5c480fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:27.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.316+0000 7f5c4e632640 1 -- 172.21.15.16:0/1639947066 shutdown_connections 2024-09-12T02:03:27.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:27.316+0000 7f5c4e632640 1 -- 172.21.15.16:0/1639947066 wait complete. 2024-09-12T02:03:27.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:03:27 smithi184 ceph-mon[35593]: pgmap v381: 97 pgs: 97 active+clean; 960 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-09-12T02:03:27.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:03:27 smithi033 ceph-mon[35332]: pgmap v381: 97 pgs: 97 active+clean; 960 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-09-12T02:03:27.933 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:03:27 smithi016 ceph-mon[30947]: pgmap v381: 97 pgs: 97 active+clean; 960 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-09-12T02:03:27.973 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-12T02:03:27.974 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:03:28.230 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:03:29.668 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.667+0000 7f2425da6640 1 -- 172.21.15.16:0/3962776421 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2420103710 msgr2=0x7f2420109fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:29.668 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.667+0000 7f2425da6640 1 --2- 172.21.15.16:0/3962776421 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2420103710 0x7f2420109fa0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f2418009ca0 tx=0x7f241802a8c0 comp rx=0 tx=0).stop 2024-09-12T02:03:29.668 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.667+0000 7f2425da6640 1 -- 172.21.15.16:0/3962776421 shutdown_connections 2024-09-12T02:03:29.668 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.667+0000 7f2425da6640 1 --2- 172.21.15.16:0/3962776421 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2420103710 0x7f2420109fa0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:29.668 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.667+0000 7f2425da6640 1 --2- 172.21.15.16:0/3962776421 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2420102d50 0x7f24201031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:29.668 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.667+0000 7f2425da6640 1 --2- 172.21.15.16:0/3962776421 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2420101b60 0x7f2420101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:29.668 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.667+0000 7f2425da6640 1 -- 172.21.15.16:0/3962776421 >> 172.21.15.16:0/3962776421 conn(0x7f24200fd2e0 msgr2=0x7f24200ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:29.668 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.667+0000 7f2425da6640 1 -- 172.21.15.16:0/3962776421 shutdown_connections 2024-09-12T02:03:29.668 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.668+0000 7f2425da6640 1 -- 172.21.15.16:0/3962776421 wait complete. 2024-09-12T02:03:29.668 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.668+0000 7f2425da6640 1 Processor -- start 2024-09-12T02:03:29.671 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.668+0000 7f2425da6640 1 -- start start 2024-09-12T02:03:29.671 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.669+0000 7f2425da6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2420101b60 0x7f2420197d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:29.671 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.669+0000 7f2425da6640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2420102d50 0x7f2420198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:29.671 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.669+0000 7f2425da6640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2420103710 0x7f242019f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:29.671 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.669+0000 7f2425da6640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f242010d7c0 con 0x7f2420101b60 2024-09-12T02:03:29.671 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.669+0000 7f2425da6640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f242010d640 con 0x7f2420103710 2024-09-12T02:03:29.671 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.669+0000 7f2425da6640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f242010d940 con 0x7f2420102d50 2024-09-12T02:03:29.671 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.669+0000 7f241f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2420101b60 0x7f2420197d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:29.671 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.669+0000 7f241f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2420101b60 0x7f2420197d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52300/0 (socket says 172.21.15.16:52300) 2024-09-12T02:03:29.671 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.669+0000 7f241f7fe640 1 -- 172.21.15.16:0/2238764171 learned_addr learned my addr 172.21.15.16:0/2238764171 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:03:29.671 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.669+0000 7f241effd640 1 --2- 172.21.15.16:0/2238764171 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2420102d50 0x7f2420198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:29.671 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.669+0000 7f241ffff640 1 --2- 172.21.15.16:0/2238764171 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2420103710 0x7f242019f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:29.672 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.669+0000 7f241f7fe640 1 -- 172.21.15.16:0/2238764171 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2420102d50 msgr2=0x7f2420198290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:29.672 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.669+0000 7f241f7fe640 1 --2- 172.21.15.16:0/2238764171 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2420102d50 0x7f2420198290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:29.672 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.669+0000 7f241f7fe640 1 -- 172.21.15.16:0/2238764171 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2420103710 msgr2=0x7f242019f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:29.672 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.669+0000 7f241f7fe640 1 --2- 172.21.15.16:0/2238764171 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2420103710 0x7f242019f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:29.672 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.669+0000 7f241f7fe640 1 -- 172.21.15.16:0/2238764171 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f242019f960 con 0x7f2420101b60 2024-09-12T02:03:29.672 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.669+0000 7f241effd640 1 --2- 172.21.15.16:0/2238764171 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2420102d50 0x7f2420198290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:03:29.672 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.669+0000 7f241ffff640 1 --2- 172.21.15.16:0/2238764171 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2420103710 0x7f242019f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:03:29.672 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.669+0000 7f241f7fe640 1 --2- 172.21.15.16:0/2238764171 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2420101b60 0x7f2420197d50 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f241000d770 tx=0x7f241000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:29.672 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.669+0000 7f241cff9640 1 -- 172.21.15.16:0/2238764171 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2410017070 con 0x7f2420101b60 2024-09-12T02:03:29.672 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.670+0000 7f241cff9640 1 -- 172.21.15.16:0/2238764171 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2410009c30 con 0x7f2420101b60 2024-09-12T02:03:29.672 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.670+0000 7f2425da6640 1 -- 172.21.15.16:0/2238764171 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f242019fc50 con 0x7f2420101b60 2024-09-12T02:03:29.672 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.670+0000 7f241cff9640 1 -- 172.21.15.16:0/2238764171 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2410003dd0 con 0x7f2420101b60 2024-09-12T02:03:29.672 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.670+0000 7f2425da6640 1 -- 172.21.15.16:0/2238764171 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24201a01a0 con 0x7f2420101b60 2024-09-12T02:03:29.678 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.672+0000 7f241cff9640 1 -- 172.21.15.16:0/2238764171 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f2410003fb0 con 0x7f2420101b60 2024-09-12T02:03:29.678 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.672+0000 7f2428c1e740 1 Processor -- start 2024-09-12T02:03:29.678 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.673+0000 7f2428c1e740 1 -- start start 2024-09-12T02:03:29.678 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.673+0000 7f2428c1e740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559402dbddb0 0x559402dbe1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:29.678 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.673+0000 7f2428c1e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559402dbe6f0 0x559402dc31b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:29.678 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.673+0000 7f2428c1e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559402dc3780 0x559402dc5c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:29.678 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.673+0000 7f2428c1e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x559402d1d2f0 con 0x559402dbe6f0 2024-09-12T02:03:29.678 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.673+0000 7f2428c1e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x559402cc5270 con 0x559402dc3780 2024-09-12T02:03:29.678 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.673+0000 7f2428c1e740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x559402d192b0 con 0x559402dbddb0 2024-09-12T02:03:29.678 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.673+0000 7f241f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559402dbe6f0 0x559402dc31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:29.678 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.673+0000 7f241cff9640 1 --2- 172.21.15.16:0/2238764171 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f24080a2cb0 0x7f24080a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:29.679 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.673+0000 7f241f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559402dbe6f0 0x559402dc31b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52316/0 (socket says 172.21.15.16:52316) 2024-09-12T02:03:29.679 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.673+0000 7f241f7fe640 1 -- 172.21.15.16:0/3648229536 learned_addr learned my addr 172.21.15.16:0/3648229536 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:03:29.679 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.673+0000 7f241ffff640 1 --2- 172.21.15.16:0/3648229536 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559402dbddb0 0x559402dbe1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:29.679 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.673+0000 7f241effd640 1 --2- 172.21.15.16:0/3648229536 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559402dc3780 0x559402dc5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:29.679 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.673+0000 7f241ffff640 1 --2- 172.21.15.16:0/2238764171 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f24080a2cb0 0x7f24080a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:29.679 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.673+0000 7f241cff9640 1 -- 172.21.15.16:0/2238764171 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f24100ceb10 con 0x7f2420101b60 2024-09-12T02:03:29.679 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.673+0000 7f241f7fe640 1 -- 172.21.15.16:0/3648229536 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559402dbddb0 msgr2=0x559402dbe1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:29.679 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.673+0000 7f241f7fe640 1 --2- 172.21.15.16:0/3648229536 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559402dbddb0 0x559402dbe1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:29.679 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.673+0000 7f241f7fe640 1 -- 172.21.15.16:0/3648229536 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559402dc3780 msgr2=0x559402dc5c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:29.679 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.673+0000 7f241f7fe640 1 --2- 172.21.15.16:0/3648229536 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559402dc3780 0x559402dc5c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:29.679 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.673+0000 7f241f7fe640 1 -- 172.21.15.16:0/3648229536 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559402cc4340 con 0x559402dbe6f0 2024-09-12T02:03:29.679 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.674+0000 7f241f7fe640 1 --2- 172.21.15.16:0/3648229536 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559402dbe6f0 0x559402dc31b0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f241009b300 tx=0x7f2410003870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afa1b0c30233a2b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:29.679 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.674+0000 7f241ffff640 1 --2- 172.21.15.16:0/2238764171 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f24080a2cb0 0x7f24080a5170 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f2418009bf0 tx=0x7f24180095c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:29.679 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.674+0000 7f2405ffb640 1 -- 172.21.15.16:0/3648229536 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2410020070 con 0x559402dbe6f0 2024-09-12T02:03:29.680 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.674+0000 7f2405ffb640 1 -- 172.21.15.16:0/3648229536 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2410095e50 con 0x559402dbe6f0 2024-09-12T02:03:29.680 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.674+0000 7f2405ffb640 1 -- 172.21.15.16:0/3648229536 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f241009f720 con 0x559402dbe6f0 2024-09-12T02:03:29.680 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.675+0000 7f2428c1e740 1 -- 172.21.15.16:0/3648229536 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559402dbe6f0 msgr2=0x559402dc31b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:29.680 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.675+0000 7f2428c1e740 1 --2- 172.21.15.16:0/3648229536 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559402dbe6f0 0x559402dc31b0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f241009b300 tx=0x7f2410003870 comp rx=0 tx=0).stop 2024-09-12T02:03:29.680 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.675+0000 7f2428c1e740 1 -- 172.21.15.16:0/3648229536 shutdown_connections 2024-09-12T02:03:29.680 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.675+0000 7f2428c1e740 1 --2- 172.21.15.16:0/3648229536 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559402dc3780 0x559402dc5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:29.680 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.675+0000 7f2428c1e740 1 --2- 172.21.15.16:0/3648229536 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559402dbe6f0 0x559402dc31b0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:29.680 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.675+0000 7f2428c1e740 1 --2- 172.21.15.16:0/3648229536 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559402dbddb0 0x559402dbe1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:29.680 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.675+0000 7f2428c1e740 1 -- 172.21.15.16:0/3648229536 >> 172.21.15.16:0/3648229536 conn(0x559402db5010 msgr2=0x559402db5410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:29.680 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.675+0000 7f2428c1e740 1 -- 172.21.15.16:0/3648229536 shutdown_connections 2024-09-12T02:03:29.680 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.675+0000 7f2428c1e740 1 -- 172.21.15.16:0/3648229536 wait complete. 2024-09-12T02:03:29.680 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.675+0000 7f2428c1e740 1 -- start start 2024-09-12T02:03:29.680 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.675+0000 7f2428c1e740 1 Processor -- start 2024-09-12T02:03:29.681 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.676+0000 7f2428c1e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559402dbddb0 0x559402dc8050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:29.681 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.676+0000 7f2428c1e740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559402dbe6f0 0x559402dc8590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:29.681 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.676+0000 7f2428c1e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559402dc3780 0x559402dcf5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:29.681 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.676+0000 7f2428c1e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x559402dc66f0 con 0x559402dbddb0 2024-09-12T02:03:29.681 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.676+0000 7f2428c1e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x559402ca6290 con 0x559402dc3780 2024-09-12T02:03:29.681 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.676+0000 7f2428c1e740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x559402dc6870 con 0x559402dbe6f0 2024-09-12T02:03:29.681 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.676+0000 7f2428c1e740 10 client.0 _ll_register_callbacks cb 0x559402cd7140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:03:29.681 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.676+0000 7f241f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559402dbddb0 0x559402dc8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:29.681 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.676+0000 7f241f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559402dbddb0 0x559402dc8050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52332/0 (socket says 172.21.15.16:52332) 2024-09-12T02:03:29.681 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.676+0000 7f241f7fe640 1 -- 172.21.15.16:0/3702048370 learned_addr learned my addr 172.21.15.16:0/3702048370 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:03:29.681 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.676+0000 7f241ffff640 1 --2- 172.21.15.16:0/3702048370 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559402dc3780 0x559402dcf5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:29.681 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.676+0000 7f241effd640 1 --2- 172.21.15.16:0/3702048370 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559402dbe6f0 0x559402dc8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:29.681 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.676+0000 7f241f7fe640 1 -- 172.21.15.16:0/3702048370 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559402dbe6f0 msgr2=0x559402dc8590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:29.682 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.676+0000 7f241f7fe640 1 --2- 172.21.15.16:0/3702048370 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559402dbe6f0 0x559402dc8590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:29.682 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.676+0000 7f241f7fe640 1 -- 172.21.15.16:0/3702048370 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559402dc3780 msgr2=0x559402dcf5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:29.682 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.676+0000 7f241f7fe640 1 --2- 172.21.15.16:0/3702048370 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559402dc3780 0x559402dcf5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:29.682 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.676+0000 7f241f7fe640 1 -- 172.21.15.16:0/3702048370 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559402a30210 con 0x559402dbddb0 2024-09-12T02:03:29.682 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.676+0000 7f241f7fe640 1 --2- 172.21.15.16:0/3702048370 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559402dbddb0 0x559402dc8050 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f24100a3ab0 tx=0x7f2410009f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75fa57e6355a1d64 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:29.682 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.676+0000 7f23feffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:03:29.682 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.676+0000 7f23feffd640 1 -- 172.21.15.16:0/3702048370 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2410020070 con 0x559402dbddb0 2024-09-12T02:03:29.682 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.676+0000 7f23feffd640 1 -- 172.21.15.16:0/3702048370 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f241009b5d0 con 0x559402dbddb0 2024-09-12T02:03:29.684 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.676+0000 7f2428c1e740 1 -- 172.21.15.16:0/3702048370 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2410095e50 con 0x559402dbddb0 2024-09-12T02:03:29.684 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.677+0000 7f23feffd640 1 -- 172.21.15.16:0/3702048370 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f24100994b0 con 0x559402dbddb0 2024-09-12T02:03:29.684 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.677+0000 7f2428c1e740 10 client.15828 fetch_fsmap learned FSMap version 25 2024-09-12T02:03:29.684 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.677+0000 7f2428c1e740 1 -- 172.21.15.16:0/3702048370 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x559402afc620 con 0x559402dbddb0 2024-09-12T02:03:29.684 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.677+0000 7f23feffd640 1 -- 172.21.15.16:0/3702048370 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2410095e50 con 0x559402dbddb0 2024-09-12T02:03:29.684 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.677+0000 7f23feffd640 1 -- 172.21.15.16:0/3702048370 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f24100b4140 con 0x559402dbddb0 2024-09-12T02:03:29.684 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.677+0000 7f2428c1e740 10 client.15828 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:03:29.684 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.677+0000 7f2428c1e740 10 client.15828 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:03:29.684 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.677+0000 7f2428c1e740 20 client.15828 populate_metadata read hostname 'smithi016' 2024-09-12T02:03:29.684 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.677+0000 7f2428c1e740 1 --2- 172.21.15.16:0/3702048370 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x559402dd38d0 0x559402dd5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:29.684 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.677+0000 7f2428c1e740 4 client.15828 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:03:29.685 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.677+0000 7f2428c1e740 1 -- 172.21.15.16:0/3702048370 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559402d2aa20 con 0x559402dd38d0 2024-09-12T02:03:29.685 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.677+0000 7f241effd640 1 --2- 172.21.15.16:0/3702048370 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x559402dd38d0 0x559402dd5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:29.685 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.678+0000 7f241effd640 1 --2- 172.21.15.16:0/3702048370 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x559402dd38d0 0x559402dd5cc0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2924f3a5cd9910 server_cookie=ab1466efe4954e0d in_seq=0 out_seq=0 2024-09-12T02:03:29.685 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.678+0000 7f23feffd640 10 client.15828 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:03:29.686 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.679+0000 7f23feffd640 1 -- 172.21.15.16:0/3702048370 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559402d2aa20 con 0x559402dd38d0 2024-09-12T02:03:29.686 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.679+0000 7f23feffd640 10 client.15828 handle_command_reply: tid=0 2024-09-12T02:03:29.686 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.679+0000 7f2428c1e740 1 client.15828 shutdown 2024-09-12T02:03:29.686 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.679+0000 7f2428c1e740 1 -- 172.21.15.16:0/3702048370 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559402dbddb0 msgr2=0x559402dc8050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:29.686 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.679+0000 7f2428c1e740 1 --2- 172.21.15.16:0/3702048370 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559402dbddb0 0x559402dc8050 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f24100a3ab0 tx=0x7f2410009f50 comp rx=0 tx=0).stop 2024-09-12T02:03:29.686 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.679+0000 7f2428c1e740 1 -- 172.21.15.16:0/3702048370 shutdown_connections 2024-09-12T02:03:29.686 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.679+0000 7f2428c1e740 1 --2- 172.21.15.16:0/3702048370 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x559402dd38d0 0x559402dd5cc0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:29.686 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.679+0000 7f2428c1e740 1 --2- 172.21.15.16:0/3702048370 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559402dc3780 0x559402dcf5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:29.687 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.679+0000 7f2428c1e740 1 --2- 172.21.15.16:0/3702048370 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559402dbe6f0 0x559402dc8590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:29.687 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.679+0000 7f23feffd640 0 client.15828 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:03:29.687 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.679+0000 7f2428c1e740 1 --2- 172.21.15.16:0/3702048370 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559402dbddb0 0x559402dc8050 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:29.687 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.679+0000 7f2428c1e740 1 -- 172.21.15.16:0/3702048370 >> 172.21.15.16:0/3702048370 conn(0x559402db5010 msgr2=0x559402db6820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:29.687 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.679+0000 7f2428c1e740 1 -- 172.21.15.16:0/3702048370 shutdown_connections 2024-09-12T02:03:29.687 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.679+0000 7f2428c1e740 1 -- 172.21.15.16:0/3702048370 wait complete. 2024-09-12T02:03:29.687 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.679+0000 7f2428c1e740 20 client.15828 trim_cache size 0 max 16384 2024-09-12T02:03:29.695 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.694+0000 7f2428c1e740 1 Processor -- start 2024-09-12T02:03:29.695 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.694+0000 7f2428c1e740 1 -- start start 2024-09-12T02:03:29.695 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.694+0000 7f2428c1e740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559402db5010 0x559402dc8510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:29.695 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.694+0000 7f2428c1e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559402dbddb0 0x559402dcf560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:29.696 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.694+0000 7f2428c1e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559402dbe6f0 0x559402db5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:29.696 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.694+0000 7f2428c1e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f24201a9480 con 0x559402dbddb0 2024-09-12T02:03:29.696 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.694+0000 7f2428c1e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f24201a6d50 con 0x559402dbe6f0 2024-09-12T02:03:29.696 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.694+0000 7f2428c1e740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f24201a4ba0 con 0x559402db5010 2024-09-12T02:03:29.696 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.695+0000 7f241effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559402dbddb0 0x559402dcf560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:29.696 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.695+0000 7f241effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559402dbddb0 0x559402dcf560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52348/0 (socket says 172.21.15.16:52348) 2024-09-12T02:03:29.696 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.695+0000 7f241effd640 1 -- 172.21.15.16:0/2327419236 learned_addr learned my addr 172.21.15.16:0/2327419236 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:03:29.696 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.695+0000 7f241f7fe640 1 --2- 172.21.15.16:0/2327419236 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559402db5010 0x559402dc8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:29.696 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.695+0000 7f241ffff640 1 --2- 172.21.15.16:0/2327419236 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559402dbe6f0 0x559402db5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:29.696 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.695+0000 7f241effd640 1 -- 172.21.15.16:0/2327419236 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559402db5010 msgr2=0x559402dc8510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:29.696 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.695+0000 7f241effd640 1 --2- 172.21.15.16:0/2327419236 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559402db5010 0x559402dc8510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:29.696 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.695+0000 7f241effd640 1 -- 172.21.15.16:0/2327419236 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559402dbe6f0 msgr2=0x559402db5ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:29.696 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.695+0000 7f241effd640 1 --2- 172.21.15.16:0/2327419236 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559402dbe6f0 0x559402db5ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:29.697 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.695+0000 7f241effd640 1 -- 172.21.15.16:0/2327419236 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559402db4e30 con 0x559402dbddb0 2024-09-12T02:03:29.697 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.695+0000 7f241effd640 1 --2- 172.21.15.16:0/2327419236 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559402dbddb0 0x559402dcf560 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f241400b370 tx=0x7f24140138f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a6e68d643907eae server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:29.697 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.695+0000 7f24057fa640 1 -- 172.21.15.16:0/2327419236 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2414004030 con 0x559402dbddb0 2024-09-12T02:03:29.697 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.695+0000 7f24057fa640 1 -- 172.21.15.16:0/2327419236 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24140041d0 con 0x559402dbddb0 2024-09-12T02:03:29.697 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.695+0000 7f24057fa640 1 -- 172.21.15.16:0/2327419236 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2414011730 con 0x559402dbddb0 2024-09-12T02:03:29.700 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.696+0000 7f2428c1e740 1 -- 172.21.15.16:0/2327419236 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559402dbddb0 msgr2=0x559402dcf560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:29.700 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.696+0000 7f2428c1e740 1 --2- 172.21.15.16:0/2327419236 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559402dbddb0 0x559402dcf560 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f241400b370 tx=0x7f24140138f0 comp rx=0 tx=0).stop 2024-09-12T02:03:29.700 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.696+0000 7f2428c1e740 1 -- 172.21.15.16:0/2327419236 shutdown_connections 2024-09-12T02:03:29.701 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.696+0000 7f2428c1e740 1 --2- 172.21.15.16:0/2327419236 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559402dbe6f0 0x559402db5ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:29.701 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.696+0000 7f2428c1e740 1 --2- 172.21.15.16:0/2327419236 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559402dbddb0 0x559402dcf560 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:29.701 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.696+0000 7f2428c1e740 1 --2- 172.21.15.16:0/2327419236 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559402db5010 0x559402dc8510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:29.701 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.696+0000 7f2428c1e740 1 -- 172.21.15.16:0/2327419236 >> 172.21.15.16:0/2327419236 conn(0x559402dc32d0 msgr2=0x559402dc7e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:29.701 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.696+0000 7f2428c1e740 1 -- 172.21.15.16:0/2327419236 shutdown_connections 2024-09-12T02:03:29.701 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.696+0000 7f2428c1e740 1 -- 172.21.15.16:0/2327419236 wait complete. 2024-09-12T02:03:29.701 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.696+0000 7f2428c1e740 1 -- start start 2024-09-12T02:03:29.701 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.697+0000 7f2428c1e740 1 Processor -- start 2024-09-12T02:03:29.701 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.697+0000 7f2428c1e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559402db5010 0x559402dcc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:29.701 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.697+0000 7f2428c1e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559402dbddb0 0x559402dcce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:29.701 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.697+0000 7f2428c1e740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559402dbe6f0 0x559402dcddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:29.701 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.697+0000 7f2428c1e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f24201a5210 con 0x559402dbddb0 2024-09-12T02:03:29.701 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.697+0000 7f2428c1e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f24201a6680 con 0x559402db5010 2024-09-12T02:03:29.702 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.697+0000 7f2428c1e740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f24201a58e0 con 0x559402dbe6f0 2024-09-12T02:03:29.702 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.697+0000 7f2428c1e740 10 client.0 _ll_register_callbacks cb 0x7f241400b440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:03:29.702 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.697+0000 7f241effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559402dbddb0 0x559402dcce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:29.702 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.697+0000 7f241effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559402dbddb0 0x559402dcce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52358/0 (socket says 172.21.15.16:52358) 2024-09-12T02:03:29.702 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.697+0000 7f241effd640 1 -- 172.21.15.16:0/2287842747 learned_addr learned my addr 172.21.15.16:0/2287842747 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:03:29.702 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.697+0000 7f241f7fe640 1 --2- 172.21.15.16:0/2287842747 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559402db5010 0x559402dcc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:29.702 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.697+0000 7f241ffff640 1 --2- 172.21.15.16:0/2287842747 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559402dbe6f0 0x559402dcddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:29.702 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.697+0000 7f241effd640 1 -- 172.21.15.16:0/2287842747 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559402dbe6f0 msgr2=0x559402dcddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:29.702 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.697+0000 7f241effd640 1 --2- 172.21.15.16:0/2287842747 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559402dbe6f0 0x559402dcddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:29.702 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.697+0000 7f241effd640 1 -- 172.21.15.16:0/2287842747 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559402db5010 msgr2=0x559402dcc8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:29.702 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.697+0000 7f241effd640 1 --2- 172.21.15.16:0/2287842747 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559402db5010 0x559402dcc8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:29.702 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.697+0000 7f241effd640 1 -- 172.21.15.16:0/2287842747 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559402dce3e0 con 0x559402dbddb0 2024-09-12T02:03:29.702 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.698+0000 7f241effd640 1 --2- 172.21.15.16:0/2287842747 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559402dbddb0 0x559402dcce10 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f241400be10 tx=0x7f2414004050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a5caf958ca6ab98 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:29.702 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.698+0000 7f23feffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:03:29.703 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.698+0000 7f23feffd640 1 -- 172.21.15.16:0/2287842747 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24140035f0 con 0x559402dbddb0 2024-09-12T02:03:29.703 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.698+0000 7f23feffd640 1 -- 172.21.15.16:0/2287842747 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f2414003790 con 0x559402dbddb0 2024-09-12T02:03:29.703 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.698+0000 7f2428c1e740 1 -- 172.21.15.16:0/2287842747 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24140041d0 con 0x559402dbddb0 2024-09-12T02:03:29.703 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.698+0000 7f23feffd640 1 -- 172.21.15.16:0/2287842747 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2414003990 con 0x559402dbddb0 2024-09-12T02:03:29.703 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.698+0000 7f2428c1e740 10 client.15834 fetch_fsmap learned FSMap version 25 2024-09-12T02:03:29.703 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.698+0000 7f2428c1e740 1 -- 172.21.15.16:0/2287842747 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x559402dcf6f0 con 0x559402dbddb0 2024-09-12T02:03:29.703 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.698+0000 7f23feffd640 1 -- 172.21.15.16:0/2287842747 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24140041d0 con 0x559402dbddb0 2024-09-12T02:03:29.703 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.698+0000 7f23feffd640 1 -- 172.21.15.16:0/2287842747 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f2414027410 con 0x559402dbddb0 2024-09-12T02:03:29.703 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.698+0000 7f2428c1e740 10 client.15834 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:03:29.703 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.698+0000 7f2428c1e740 10 client.15834 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:03:29.703 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.698+0000 7f2428c1e740 20 client.15834 populate_metadata read hostname 'smithi016' 2024-09-12T02:03:29.703 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.698+0000 7f2428c1e740 1 --2- 172.21.15.16:0/2287842747 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x559402dc3780 0x559402e409b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:29.703 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.698+0000 7f2428c1e740 4 client.15834 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:03:29.704 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.698+0000 7f2428c1e740 1 -- 172.21.15.16:0/2287842747 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559402d24d70 con 0x559402dc3780 2024-09-12T02:03:29.704 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.699+0000 7f241f7fe640 1 --2- 172.21.15.16:0/2287842747 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x559402dc3780 0x559402e409b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:29.704 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.699+0000 7f23feffd640 10 client.15834 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:03:29.704 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.699+0000 7f241f7fe640 1 --2- 172.21.15.16:0/2287842747 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x559402dc3780 0x559402e409b0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7fbc2ca2373dd9be server_cookie=5328003a01af1aa9 in_seq=0 out_seq=0 2024-09-12T02:03:29.710 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.710+0000 7f23feffd640 1 -- 172.21.15.16:0/2287842747 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559402d24d70 con 0x559402dc3780 2024-09-12T02:03:29.711 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.710+0000 7f23feffd640 10 client.15834 handle_command_reply: tid=0 2024-09-12T02:03:29.711 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.710+0000 7f2428c1e740 1 client.15834 shutdown 2024-09-12T02:03:29.711 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.710+0000 7f2428c1e740 1 -- 172.21.15.16:0/2287842747 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559402dbddb0 msgr2=0x559402dcce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:29.711 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.710+0000 7f2428c1e740 1 --2- 172.21.15.16:0/2287842747 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559402dbddb0 0x559402dcce10 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f241400be10 tx=0x7f2414004050 comp rx=0 tx=0).stop 2024-09-12T02:03:29.711 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.710+0000 7f2428c1e740 1 -- 172.21.15.16:0/2287842747 shutdown_connections 2024-09-12T02:03:29.711 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.710+0000 7f2428c1e740 1 --2- 172.21.15.16:0/2287842747 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x559402dc3780 0x559402e409b0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:29.711 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.710+0000 7f2428c1e740 1 --2- 172.21.15.16:0/2287842747 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559402dbe6f0 0x559402dcddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:29.711 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.710+0000 7f2428c1e740 1 --2- 172.21.15.16:0/2287842747 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559402dbddb0 0x559402dcce10 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:29.711 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.710+0000 7f2428c1e740 1 --2- 172.21.15.16:0/2287842747 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559402db5010 0x559402dcc8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:29.711 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.710+0000 7f23feffd640 0 client.15834 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:03:29.711 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.710+0000 7f2428c1e740 1 -- 172.21.15.16:0/2287842747 >> 172.21.15.16:0/2287842747 conn(0x559402dc32d0 msgr2=0x559402db39a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:29.711 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.711+0000 7f2428c1e740 1 -- 172.21.15.16:0/2287842747 shutdown_connections 2024-09-12T02:03:29.712 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.711+0000 7f2428c1e740 1 -- 172.21.15.16:0/2287842747 wait complete. 2024-09-12T02:03:29.712 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.711+0000 7f2428c1e740 20 client.15834 trim_cache size 0 max 16384 2024-09-12T02:03:29.712 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-09-12T02:03:29.712 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-09-12T02:03:29.712 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-09-12T02:03:29.712 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-09-12T02:03:29.713 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.711+0000 7f2425da6640 1 -- 172.21.15.16:0/2238764171 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f24080a2cb0 msgr2=0x7f24080a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:29.713 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.711+0000 7f2425da6640 1 --2- 172.21.15.16:0/2238764171 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f24080a2cb0 0x7f24080a5170 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f2418009bf0 tx=0x7f24180095c0 comp rx=0 tx=0).stop 2024-09-12T02:03:29.713 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.711+0000 7f2425da6640 1 -- 172.21.15.16:0/2238764171 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2420101b60 msgr2=0x7f2420197d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:29.713 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.711+0000 7f2425da6640 1 --2- 172.21.15.16:0/2238764171 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2420101b60 0x7f2420197d50 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f241000d770 tx=0x7f241000dc30 comp rx=0 tx=0).stop 2024-09-12T02:03:29.713 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.711+0000 7f2425da6640 1 -- 172.21.15.16:0/2238764171 shutdown_connections 2024-09-12T02:03:29.713 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.711+0000 7f2425da6640 1 --2- 172.21.15.16:0/2238764171 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f24080a2cb0 0x7f24080a5170 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:29.713 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.711+0000 7f2425da6640 1 --2- 172.21.15.16:0/2238764171 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2420103710 0x7f242019f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:29.713 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.711+0000 7f2425da6640 1 --2- 172.21.15.16:0/2238764171 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2420102d50 0x7f2420198290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:29.713 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.711+0000 7f2425da6640 1 --2- 172.21.15.16:0/2238764171 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2420101b60 0x7f2420197d50 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:29.714 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.711+0000 7f2425da6640 1 -- 172.21.15.16:0/2238764171 >> 172.21.15.16:0/2238764171 conn(0x7f24200fd2e0 msgr2=0x7f2420106050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:29.714 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.711+0000 7f2425da6640 1 -- 172.21.15.16:0/2238764171 shutdown_connections 2024-09-12T02:03:29.714 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:29.711+0000 7f2425da6640 1 -- 172.21.15.16:0/2238764171 wait complete. 2024-09-12T02:03:29.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:03:29 smithi184 ceph-mon[35593]: pgmap v382: 97 pgs: 97 active+clean; 960 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-12T02:03:29.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:03:29 smithi033 ceph-mon[35332]: pgmap v382: 97 pgs: 97 active+clean; 960 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-12T02:03:29.933 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:03:29 smithi016 ceph-mon[30947]: pgmap v382: 97 pgs: 97 active+clean; 960 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-12T02:03:30.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:30.464+0000 7f8daf4ac640 1 -- 172.21.15.16:0/1276974154 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8da8074590 msgr2=0x7f8da8074970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:30.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:30.464+0000 7f8daf4ac640 1 --2- 172.21.15.16:0/1276974154 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8da8074590 0x7f8da8074970 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f8da4009900 tx=0x7f8da402a480 comp rx=0 tx=0).stop 2024-09-12T02:03:30.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:30.464+0000 7f8daf4ac640 1 -- 172.21.15.16:0/1276974154 shutdown_connections 2024-09-12T02:03:30.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:30.464+0000 7f8daf4ac640 1 --2- 172.21.15.16:0/1276974154 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8da8112b90 0x7f8da8114f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:30.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:30.464+0000 7f8daf4ac640 1 --2- 172.21.15.16:0/1276974154 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8da8074eb0 0x7f8da8112650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:30.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:30.464+0000 7f8daf4ac640 1 --2- 172.21.15.16:0/1276974154 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8da8074590 0x7f8da8074970 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:30.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:30.464+0000 7f8daf4ac640 1 -- 172.21.15.16:0/1276974154 >> 172.21.15.16:0/1276974154 conn(0x7f8da80fd8c0 msgr2=0x7f8da80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:30.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:30.464+0000 7f8daf4ac640 1 -- 172.21.15.16:0/1276974154 shutdown_connections 2024-09-12T02:03:30.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:30.464+0000 7f8daf4ac640 1 -- 172.21.15.16:0/1276974154 wait complete. 2024-09-12T02:03:30.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:30.465+0000 7f8daf4ac640 1 Processor -- start 2024-09-12T02:03:30.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:30.465+0000 7f8daf4ac640 1 -- start start 2024-09-12T02:03:30.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:30.465+0000 7f8daf4ac640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8da8074590 0x7f8da81a4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:30.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:30.465+0000 7f8daf4ac640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8da8074eb0 0x7f8da81a4c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:30.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:30.465+0000 7f8daf4ac640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8da8112b90 0x7f8da819e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:30.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:30.465+0000 7f8daf4ac640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8da8118570 con 0x7f8da8112b90 2024-09-12T02:03:30.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:30.465+0000 7f8daf4ac640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8da81183f0 con 0x7f8da8074eb0 2024-09-12T02:03:30.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:30.466+0000 7f8daf4ac640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8da81186f0 con 0x7f8da8074590 2024-09-12T02:03:30.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:30.466+0000 7f8dada49640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8da8112b90 0x7f8da819e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:30.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:30.466+0000 7f8dada49640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8da8112b90 0x7f8da819e8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52370/0 (socket says 172.21.15.16:52370) 2024-09-12T02:03:30.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:30.466+0000 7f8dada49640 1 -- 172.21.15.16:0/2593703840 learned_addr learned my addr 172.21.15.16:0/2593703840 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:03:30.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:30.466+0000 7f8daca47640 1 --2- 172.21.15.16:0/2593703840 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8da8074eb0 0x7f8da81a4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:30.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:30.466+0000 7f8dad248640 1 --2- 172.21.15.16:0/2593703840 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8da8074590 0x7f8da81a4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:30.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:30.466+0000 7f8dada49640 1 -- 172.21.15.16:0/2593703840 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8da8074590 msgr2=0x7f8da81a4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:30.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:30.466+0000 7f8dada49640 1 --2- 172.21.15.16:0/2593703840 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8da8074590 0x7f8da81a4710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:30.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:30.466+0000 7f8dada49640 1 -- 172.21.15.16:0/2593703840 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8da8074eb0 msgr2=0x7f8da81a4c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:30.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:30.466+0000 7f8dada49640 1 --2- 172.21.15.16:0/2593703840 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8da8074eb0 0x7f8da81a4c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:30.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:30.466+0000 7f8dada49640 1 -- 172.21.15.16:0/2593703840 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8da819f000 con 0x7f8da8112b90 2024-09-12T02:03:30.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:30.466+0000 7f8dad248640 1 --2- 172.21.15.16:0/2593703840 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8da8074590 0x7f8da81a4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:03:30.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:30.466+0000 7f8dada49640 1 --2- 172.21.15.16:0/2593703840 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8da8112b90 0x7f8da819e8d0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f8da000e9d0 tx=0x7f8da000ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:30.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:30.466+0000 7f8d9e7fc640 1 -- 172.21.15.16:0/2593703840 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8da000cd80 con 0x7f8da8112b90 2024-09-12T02:03:30.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:30.466+0000 7f8d9e7fc640 1 -- 172.21.15.16:0/2593703840 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8da0005020 con 0x7f8da8112b90 2024-09-12T02:03:30.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:30.466+0000 7f8daf4ac640 1 -- 172.21.15.16:0/2593703840 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8da819f2f0 con 0x7f8da8112b90 2024-09-12T02:03:30.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:30.466+0000 7f8d9e7fc640 1 -- 172.21.15.16:0/2593703840 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8da00116d0 con 0x7f8da8112b90 2024-09-12T02:03:30.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:30.466+0000 7f8daf4ac640 1 -- 172.21.15.16:0/2593703840 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8da81ab5e0 con 0x7f8da8112b90 2024-09-12T02:03:30.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:30.467+0000 7f8daf4ac640 1 -- 172.21.15.16:0/2593703840 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8da8074970 con 0x7f8da8112b90 2024-09-12T02:03:30.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:30.469+0000 7f8d9e7fc640 1 -- 172.21.15.16:0/2593703840 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f8da00051d0 con 0x7f8da8112b90 2024-09-12T02:03:30.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:30.469+0000 7f8d9e7fc640 1 --2- 172.21.15.16:0/2593703840 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f8d940a2c00 0x7f8d940a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:30.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:30.470+0000 7f8dad248640 1 --2- 172.21.15.16:0/2593703840 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f8d940a2c00 0x7f8d940a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:30.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:30.470+0000 7f8d9e7fc640 1 -- 172.21.15.16:0/2593703840 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f8da00cf4c0 con 0x7f8da8112b90 2024-09-12T02:03:30.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:30.470+0000 7f8dad248640 1 --2- 172.21.15.16:0/2593703840 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f8d940a2c00 0x7f8d940a50c0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f8da4000c00 tx=0x7f8da4032c70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:30.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:30.474+0000 7f8d9e7fc640 1 -- 172.21.15.16:0/2593703840 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8da0016030 con 0x7f8da8112b90 2024-09-12T02:03:30.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:30.687+0000 7f8daf4ac640 1 -- 172.21.15.16:0/2593703840 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8da802d7c0 con 0x7f8da8112b90 2024-09-12T02:03:30.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:30.688+0000 7f8d9e7fc640 1 -- 172.21.15.16:0/2593703840 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11911 (secure 0 0 0) 0x7f8da009c4c0 con 0x7f8da8112b90 2024-09-12T02:03:30.688 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T02:03:30.689 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":25,"btime":"2024-09-12T02:01:37:192021+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3279123279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3279123279},{"type":"v1","addr":"172.21.15.16:6835","nonce":3279123279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4119452382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4119452382},{"type":"v1","addr":"172.21.15.16:6837","nonce":4119452382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/3263823165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3263823165},{"type":"v1","addr":"172.21.15.16:6839","nonce":3263823165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/1510290732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":1510290732},{"type":"v1","addr":"172.21.15.16:6841","nonce":1510290732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/279674603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":279674603},{"type":"v1","addr":"172.21.15.33:6833","nonce":279674603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3555728028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3555728028},{"type":"v1","addr":"172.21.15.33:6835","nonce":3555728028}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T01:59:56.117608+0000","modified":"2024-09-12T02:01:36.294760+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24482,"mds_1":24481,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6833/1563374106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1563374106},{"type":"v1","addr":"172.21.15.184:6833","nonce":1563374106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6837/1892833868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1892833868},{"type":"v1","addr":"172.21.15.33:6837","nonce":1892833868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/1457679686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1457679686},{"type":"v1","addr":"172.21.15.184:6835","nonce":1457679686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.184:6837/1793393087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":1793393087},{"type":"v1","addr":"172.21.15.184:6837","nonce":1793393087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.184:6839/4203347941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":4203347941},{"type":"v1","addr":"172.21.15.184:6839","nonce":4203347941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.33:6839/4136670204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":4136670204},{"type":"v1","addr":"172.21.15.33:6839","nonce":4136670204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24481,24482,24469]},"id":1}]} 2024-09-12T02:03:30.689 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 25 2024-09-12T02:03:30.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:30.690+0000 7f8daf4ac640 1 -- 172.21.15.16:0/2593703840 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f8d940a2c00 msgr2=0x7f8d940a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:30.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:30.690+0000 7f8daf4ac640 1 --2- 172.21.15.16:0/2593703840 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f8d940a2c00 0x7f8d940a50c0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f8da4000c00 tx=0x7f8da4032c70 comp rx=0 tx=0).stop 2024-09-12T02:03:30.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:30.690+0000 7f8daf4ac640 1 -- 172.21.15.16:0/2593703840 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8da8112b90 msgr2=0x7f8da819e8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:30.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:30.690+0000 7f8daf4ac640 1 --2- 172.21.15.16:0/2593703840 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8da8112b90 0x7f8da819e8d0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f8da000e9d0 tx=0x7f8da000ee90 comp rx=0 tx=0).stop 2024-09-12T02:03:30.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:30.690+0000 7f8daf4ac640 1 -- 172.21.15.16:0/2593703840 shutdown_connections 2024-09-12T02:03:30.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:30.690+0000 7f8daf4ac640 1 --2- 172.21.15.16:0/2593703840 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f8d940a2c00 0x7f8d940a50c0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:30.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:30.690+0000 7f8daf4ac640 1 --2- 172.21.15.16:0/2593703840 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8da8112b90 0x7f8da819e8d0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:30.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:30.690+0000 7f8daf4ac640 1 --2- 172.21.15.16:0/2593703840 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8da8074eb0 0x7f8da81a4c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:30.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:30.690+0000 7f8daf4ac640 1 --2- 172.21.15.16:0/2593703840 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8da8074590 0x7f8da81a4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:30.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:30.690+0000 7f8daf4ac640 1 -- 172.21.15.16:0/2593703840 >> 172.21.15.16:0/2593703840 conn(0x7f8da80fd8c0 msgr2=0x7f8da80ff0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:30.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:30.690+0000 7f8daf4ac640 1 -- 172.21.15.16:0/2593703840 shutdown_connections 2024-09-12T02:03:30.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:30.691+0000 7f8daf4ac640 1 -- 172.21.15.16:0/2593703840 wait complete. 2024-09-12T02:03:31.290 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-12T02:03:31.552 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:03:31.674 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:03:31 smithi016 ceph-mon[30947]: pgmap v383: 97 pgs: 97 active+clean; 1016 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 18 op/s 2024-09-12T02:03:31.674 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:03:31 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/2593703840' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:03:31.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:03:31 smithi184 ceph-mon[35593]: pgmap v383: 97 pgs: 97 active+clean; 1016 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 18 op/s 2024-09-12T02:03:31.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:03:31 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/2593703840' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:03:31.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:03:31 smithi033 ceph-mon[35332]: pgmap v383: 97 pgs: 97 active+clean; 1016 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 18 op/s 2024-09-12T02:03:31.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:03:31 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/2593703840' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:03:32.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.024+0000 7f3765eaf640 1 -- 172.21.15.16:0/3473929474 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3760102790 msgr2=0x7f37601067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:32.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.024+0000 7f3765eaf640 1 --2- 172.21.15.16:0/3473929474 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3760102790 0x7f37601067e0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f3754009900 tx=0x7f375402a410 comp rx=0 tx=0).stop 2024-09-12T02:03:32.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.024+0000 7f3765eaf640 1 -- 172.21.15.16:0/3473929474 shutdown_connections 2024-09-12T02:03:32.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.024+0000 7f3765eaf640 1 --2- 172.21.15.16:0/3473929474 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3760106f10 0x7f376010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:32.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.024+0000 7f3765eaf640 1 --2- 172.21.15.16:0/3473929474 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3760102790 0x7f37601067e0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:32.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.024+0000 7f3765eaf640 1 --2- 172.21.15.16:0/3473929474 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3760101de0 0x7f37601021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:32.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.024+0000 7f3765eaf640 1 -- 172.21.15.16:0/3473929474 >> 172.21.15.16:0/3473929474 conn(0x7f37600fd8c0 msgr2=0x7f37600ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:32.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.024+0000 7f3765eaf640 1 -- 172.21.15.16:0/3473929474 shutdown_connections 2024-09-12T02:03:32.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.024+0000 7f3765eaf640 1 -- 172.21.15.16:0/3473929474 wait complete. 2024-09-12T02:03:32.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.025+0000 7f3765eaf640 1 Processor -- start 2024-09-12T02:03:32.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.025+0000 7f3765eaf640 1 -- start start 2024-09-12T02:03:32.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.025+0000 7f3765eaf640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3760101de0 0x7f376019a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:32.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.025+0000 7f3765eaf640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3760102790 0x7f376019ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:32.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.025+0000 7f3765eaf640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3760106f10 0x7f376019efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:32.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.025+0000 7f3765eaf640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3760111f70 con 0x7f3760102790 2024-09-12T02:03:32.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.025+0000 7f3765eaf640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3760111df0 con 0x7f3760106f10 2024-09-12T02:03:32.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.025+0000 7f3765eaf640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f37601120f0 con 0x7f3760101de0 2024-09-12T02:03:32.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.025+0000 7f375effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3760102790 0x7f376019ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:32.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.025+0000 7f375effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3760102790 0x7f376019ac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51684/0 (socket says 172.21.15.16:51684) 2024-09-12T02:03:32.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.025+0000 7f375effd640 1 -- 172.21.15.16:0/419521774 learned_addr learned my addr 172.21.15.16:0/419521774 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:03:32.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.026+0000 7f375ffff640 1 --2- 172.21.15.16:0/419521774 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3760106f10 0x7f376019efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:32.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.026+0000 7f375f7fe640 1 --2- 172.21.15.16:0/419521774 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3760101de0 0x7f376019a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:32.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.026+0000 7f375effd640 1 -- 172.21.15.16:0/419521774 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3760101de0 msgr2=0x7f376019a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:32.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.026+0000 7f375effd640 1 --2- 172.21.15.16:0/419521774 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3760101de0 0x7f376019a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:32.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.026+0000 7f375effd640 1 -- 172.21.15.16:0/419521774 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3760106f10 msgr2=0x7f376019efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:32.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.026+0000 7f375effd640 1 --2- 172.21.15.16:0/419521774 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3760106f10 0x7f376019efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:32.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.026+0000 7f375effd640 1 -- 172.21.15.16:0/419521774 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f376019f680 con 0x7f3760102790 2024-09-12T02:03:32.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.026+0000 7f375effd640 1 --2- 172.21.15.16:0/419521774 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3760102790 0x7f376019ac10 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f3754006010 tx=0x7f3754004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:32.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.026+0000 7f375cff9640 1 -- 172.21.15.16:0/419521774 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f375403f030 con 0x7f3760102790 2024-09-12T02:03:32.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.026+0000 7f375cff9640 1 -- 172.21.15.16:0/419521774 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3754004440 con 0x7f3760102790 2024-09-12T02:03:32.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.026+0000 7f3765eaf640 1 -- 172.21.15.16:0/419521774 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f376019f910 con 0x7f3760102790 2024-09-12T02:03:32.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.026+0000 7f375cff9640 1 -- 172.21.15.16:0/419521774 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3754035070 con 0x7f3760102790 2024-09-12T02:03:32.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.026+0000 7f3765eaf640 1 -- 172.21.15.16:0/419521774 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37601a7250 con 0x7f3760102790 2024-09-12T02:03:32.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.029+0000 7f375cff9640 1 -- 172.21.15.16:0/419521774 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f3754042030 con 0x7f3760102790 2024-09-12T02:03:32.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.029+0000 7f3768288740 1 Processor -- start 2024-09-12T02:03:32.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.029+0000 7f3768288740 1 -- start start 2024-09-12T02:03:32.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.030+0000 7f3768288740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a73bf61b40 0x55a73bf60090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:32.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.030+0000 7f3768288740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a73bf605d0 0x55a73bf60a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:32.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.030+0000 7f3768288740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a73bf6b370 0x55a73bf6d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:32.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.030+0000 7f3768288740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55a73bec62f0 con 0x55a73bf605d0 2024-09-12T02:03:32.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.030+0000 7f3768288740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55a73be6dcf0 con 0x55a73bf6b370 2024-09-12T02:03:32.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.030+0000 7f3768288740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55a73bec22b0 con 0x55a73bf61b40 2024-09-12T02:03:32.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.030+0000 7f375f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a73bf605d0 0x55a73bf60a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:32.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.030+0000 7f375f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a73bf605d0 0x55a73bf60a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51696/0 (socket says 172.21.15.16:51696) 2024-09-12T02:03:32.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.030+0000 7f375f7fe640 1 -- 172.21.15.16:0/2252098927 learned_addr learned my addr 172.21.15.16:0/2252098927 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:03:32.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.030+0000 7f375ffff640 1 --2- 172.21.15.16:0/2252098927 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a73bf61b40 0x55a73bf60090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:32.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.030+0000 7f375cff9640 1 --2- 172.21.15.16:0/419521774 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f37480a2b80 0x7f37480a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:32.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.030+0000 7f375effd640 1 --2- 172.21.15.16:0/2252098927 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a73bf6b370 0x55a73bf6d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:32.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.030+0000 7f375f7fe640 1 -- 172.21.15.16:0/2252098927 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a73bf61b40 msgr2=0x55a73bf60090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:32.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.030+0000 7f375f7fe640 1 --2- 172.21.15.16:0/2252098927 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a73bf61b40 0x55a73bf60090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:32.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.030+0000 7f375f7fe640 1 -- 172.21.15.16:0/2252098927 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a73bf6b370 msgr2=0x55a73bf6d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:32.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.030+0000 7f375f7fe640 1 --2- 172.21.15.16:0/2252098927 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a73bf6b370 0x55a73bf6d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:32.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.030+0000 7f375f7fe640 1 -- 172.21.15.16:0/2252098927 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a73be523a0 con 0x55a73bf605d0 2024-09-12T02:03:32.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.030+0000 7f375effd640 1 --2- 172.21.15.16:0/2252098927 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a73bf6b370 0x55a73bf6d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:03:32.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.030+0000 7f375ffff640 1 --2- 172.21.15.16:0/419521774 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f37480a2b80 0x7f37480a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:32.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.030+0000 7f375cff9640 1 -- 172.21.15.16:0/419521774 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f37540ef440 con 0x7f3760102790 2024-09-12T02:03:32.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.031+0000 7f375f7fe640 1 --2- 172.21.15.16:0/2252098927 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a73bf605d0 0x55a73bf60a10 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f3750009c90 tx=0x7f37500117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b4307c60e5be508 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:32.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.031+0000 7f3745ffb640 1 -- 172.21.15.16:0/2252098927 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3750012450 con 0x55a73bf605d0 2024-09-12T02:03:32.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.031+0000 7f375ffff640 1 --2- 172.21.15.16:0/419521774 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f37480a2b80 0x7f37480a5040 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f3758005fd0 tx=0x7f3758004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:32.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.031+0000 7f3745ffb640 1 -- 172.21.15.16:0/2252098927 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3750012ad0 con 0x55a73bf605d0 2024-09-12T02:03:32.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.031+0000 7f3745ffb640 1 -- 172.21.15.16:0/2252098927 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f375000fc30 con 0x55a73bf605d0 2024-09-12T02:03:32.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.032+0000 7f3768288740 1 -- 172.21.15.16:0/2252098927 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a73bf605d0 msgr2=0x55a73bf60a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:32.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.032+0000 7f3768288740 1 --2- 172.21.15.16:0/2252098927 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a73bf605d0 0x55a73bf60a10 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f3750009c90 tx=0x7f37500117b0 comp rx=0 tx=0).stop 2024-09-12T02:03:32.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.032+0000 7f3768288740 1 -- 172.21.15.16:0/2252098927 shutdown_connections 2024-09-12T02:03:32.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.032+0000 7f3768288740 1 --2- 172.21.15.16:0/2252098927 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a73bf6b370 0x55a73bf6d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:32.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.032+0000 7f3768288740 1 --2- 172.21.15.16:0/2252098927 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a73bf605d0 0x55a73bf60a10 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:32.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.032+0000 7f3768288740 1 --2- 172.21.15.16:0/2252098927 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a73bf61b40 0x55a73bf60090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:32.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.032+0000 7f3768288740 1 -- 172.21.15.16:0/2252098927 >> 172.21.15.16:0/2252098927 conn(0x55a73bf29310 msgr2=0x55a73bf5cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:32.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.032+0000 7f3768288740 1 -- 172.21.15.16:0/2252098927 shutdown_connections 2024-09-12T02:03:32.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.032+0000 7f3768288740 1 -- 172.21.15.16:0/2252098927 wait complete. 2024-09-12T02:03:32.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.032+0000 7f3768288740 1 -- start start 2024-09-12T02:03:32.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.032+0000 7f3768288740 1 Processor -- start 2024-09-12T02:03:32.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.033+0000 7f3768288740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a73bf605d0 0x55a73bf6ad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:32.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.033+0000 7f3768288740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a73bf61b40 0x55a73bf68010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:32.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.033+0000 7f3768288740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a73bf6b370 0x55a73bf68600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:32.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.033+0000 7f3768288740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55a73bf6e070 con 0x55a73bf61b40 2024-09-12T02:03:32.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.033+0000 7f3768288740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55a73be81060 con 0x55a73bf6b370 2024-09-12T02:03:32.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.033+0000 7f3768288740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55a73bf6e1f0 con 0x55a73bf605d0 2024-09-12T02:03:32.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.033+0000 7f3768288740 10 client.0 _ll_register_callbacks cb 0x55a73bbd48c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:03:32.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.033+0000 7f375effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a73bf61b40 0x55a73bf68010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:32.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.033+0000 7f375effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a73bf61b40 0x55a73bf68010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51712/0 (socket says 172.21.15.16:51712) 2024-09-12T02:03:32.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.033+0000 7f375effd640 1 -- 172.21.15.16:0/1706971186 learned_addr learned my addr 172.21.15.16:0/1706971186 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:03:32.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.033+0000 7f375f7fe640 1 --2- 172.21.15.16:0/1706971186 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a73bf605d0 0x55a73bf6ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:32.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.033+0000 7f375ffff640 1 --2- 172.21.15.16:0/1706971186 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a73bf6b370 0x55a73bf68600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:32.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.033+0000 7f375effd640 1 -- 172.21.15.16:0/1706971186 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a73bf605d0 msgr2=0x55a73bf6ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:32.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.033+0000 7f375effd640 1 --2- 172.21.15.16:0/1706971186 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a73bf605d0 0x55a73bf6ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:32.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.033+0000 7f375effd640 1 -- 172.21.15.16:0/1706971186 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a73bf6b370 msgr2=0x55a73bf68600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:32.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.033+0000 7f375effd640 1 --2- 172.21.15.16:0/1706971186 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a73bf6b370 0x55a73bf68600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:32.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.033+0000 7f375effd640 1 -- 172.21.15.16:0/1706971186 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a73bc373c0 con 0x55a73bf61b40 2024-09-12T02:03:32.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.033+0000 7f375effd640 1 --2- 172.21.15.16:0/1706971186 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a73bf61b40 0x55a73bf68010 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f37540c43e0 tx=0x7f37540bbb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dbeca0b9a8ca6c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:32.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.033+0000 7f373effd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:03:32.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.033+0000 7f373effd640 1 -- 172.21.15.16:0/1706971186 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3754049070 con 0x55a73bf61b40 2024-09-12T02:03:32.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.034+0000 7f373effd640 1 -- 172.21.15.16:0/1706971186 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f3754003cd0 con 0x55a73bf61b40 2024-09-12T02:03:32.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.034+0000 7f3768288740 1 -- 172.21.15.16:0/1706971186 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3750012ad0 con 0x55a73bf61b40 2024-09-12T02:03:32.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.034+0000 7f373effd640 1 -- 172.21.15.16:0/1706971186 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f37540be040 con 0x55a73bf61b40 2024-09-12T02:03:32.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.034+0000 7f3768288740 10 client.15852 fetch_fsmap learned FSMap version 25 2024-09-12T02:03:32.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.034+0000 7f3768288740 1 -- 172.21.15.16:0/1706971186 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a73bf79b10 con 0x55a73bf61b40 2024-09-12T02:03:32.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.034+0000 7f373effd640 1 -- 172.21.15.16:0/1706971186 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3750012ad0 con 0x55a73bf61b40 2024-09-12T02:03:32.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.034+0000 7f373effd640 1 -- 172.21.15.16:0/1706971186 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f37540b8310 con 0x55a73bf61b40 2024-09-12T02:03:32.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.034+0000 7f3768288740 10 client.15852 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:03:32.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.034+0000 7f3768288740 10 client.15852 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-12T02:03:32.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.034+0000 7f3768288740 20 client.15852 populate_metadata read hostname 'smithi016' 2024-09-12T02:03:32.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.034+0000 7f3768288740 1 --2- 172.21.15.16:0/1706971186 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55a73bf7af00 0x55a73bf7d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:32.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.034+0000 7f3768288740 4 client.15852 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:03:32.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.034+0000 7f3768288740 1 -- 172.21.15.16:0/1706971186 --> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a73bed3860 con 0x55a73bf7af00 2024-09-12T02:03:32.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.035+0000 7f375f7fe640 1 --2- 172.21.15.16:0/1706971186 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55a73bf7af00 0x55a73bf7d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:32.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.035+0000 7f373effd640 10 client.15852 ms_handle_connect on v2:172.21.15.184:6836/1793393087 2024-09-12T02:03:32.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.035+0000 7f375f7fe640 1 --2- 172.21.15.16:0/1706971186 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55a73bf7af00 0x55a73bf7d310 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c02915b591ff7c90 server_cookie=9f7d6e94c9daa536 in_seq=0 out_seq=0 2024-09-12T02:03:32.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.036+0000 7f373effd640 1 -- 172.21.15.16:0/1706971186 <== mds.2 v2:172.21.15.184:6836/1793393087 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a73bed3860 con 0x55a73bf7af00 2024-09-12T02:03:32.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.036+0000 7f373effd640 10 client.15852 handle_command_reply: tid=0 2024-09-12T02:03:32.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.036+0000 7f3768288740 1 client.15852 shutdown 2024-09-12T02:03:32.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.036+0000 7f3768288740 1 -- 172.21.15.16:0/1706971186 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a73bf61b40 msgr2=0x55a73bf68010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:32.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.036+0000 7f3768288740 1 --2- 172.21.15.16:0/1706971186 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a73bf61b40 0x55a73bf68010 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f37540c43e0 tx=0x7f37540bbb30 comp rx=0 tx=0).stop 2024-09-12T02:03:32.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.036+0000 7f3768288740 1 -- 172.21.15.16:0/1706971186 shutdown_connections 2024-09-12T02:03:32.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.036+0000 7f3768288740 1 --2- 172.21.15.16:0/1706971186 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a73bf6b370 0x55a73bf68600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:32.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.036+0000 7f3768288740 1 --2- 172.21.15.16:0/1706971186 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55a73bf7af00 0x55a73bf7d310 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:32.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.036+0000 7f3768288740 1 --2- 172.21.15.16:0/1706971186 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a73bf61b40 0x55a73bf68010 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:32.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.036+0000 7f373effd640 0 client.15852 ms_handle_reset on v2:172.21.15.184:6836/1793393087 2024-09-12T02:03:32.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.036+0000 7f3768288740 1 --2- 172.21.15.16:0/1706971186 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a73bf605d0 0x55a73bf6ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:32.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.036+0000 7f3768288740 1 -- 172.21.15.16:0/1706971186 >> 172.21.15.16:0/1706971186 conn(0x55a73bf29310 msgr2=0x55a73bf62960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:32.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.036+0000 7f3768288740 1 -- 172.21.15.16:0/1706971186 shutdown_connections 2024-09-12T02:03:32.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.036+0000 7f3768288740 1 -- 172.21.15.16:0/1706971186 wait complete. 2024-09-12T02:03:32.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.037+0000 7f3768288740 20 client.15852 trim_cache size 0 max 16384 2024-09-12T02:03:32.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.049+0000 7f3768288740 1 Processor -- start 2024-09-12T02:03:32.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.049+0000 7f3768288740 1 -- start start 2024-09-12T02:03:32.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.050+0000 7f3768288740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a73bf605d0 0x55a73bf69f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:32.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.050+0000 7f3768288740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a73bf61b40 0x55a73bf63120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:32.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.050+0000 7f3768288740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a73bf6b370 0x55a73bf63660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:32.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.050+0000 7f3768288740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f37601b1e20 con 0x55a73bf6b370 2024-09-12T02:03:32.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.050+0000 7f3768288740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f37601af6f0 con 0x55a73bf605d0 2024-09-12T02:03:32.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.050+0000 7f3768288740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f37601ad540 con 0x55a73bf61b40 2024-09-12T02:03:32.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.050+0000 7f375ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a73bf6b370 0x55a73bf63660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:32.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.050+0000 7f375ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a73bf6b370 0x55a73bf63660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51718/0 (socket says 172.21.15.16:51718) 2024-09-12T02:03:32.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.050+0000 7f375f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a73bf605d0 0x55a73bf69f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:32.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.050+0000 7f375ffff640 1 -- 172.21.15.16:0/3540454805 learned_addr learned my addr 172.21.15.16:0/3540454805 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:03:32.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.050+0000 7f375ffff640 1 -- 172.21.15.16:0/3540454805 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a73bf61b40 msgr2=0x55a73bf63120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:32.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.050+0000 7f375effd640 1 --2- 172.21.15.16:0/3540454805 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a73bf61b40 0x55a73bf63120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:32.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.050+0000 7f375ffff640 1 --2- 172.21.15.16:0/3540454805 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a73bf61b40 0x55a73bf63120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:32.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.050+0000 7f375ffff640 1 -- 172.21.15.16:0/3540454805 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a73bf605d0 msgr2=0x55a73bf69f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:32.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.050+0000 7f375ffff640 1 --2- 172.21.15.16:0/3540454805 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a73bf605d0 0x55a73bf69f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:32.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.050+0000 7f375ffff640 1 -- 172.21.15.16:0/3540454805 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a73bf66560 con 0x55a73bf6b370 2024-09-12T02:03:32.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.050+0000 7f375f7fe640 1 --2- 172.21.15.16:0/3540454805 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a73bf605d0 0x55a73bf69f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:03:32.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.050+0000 7f375effd640 1 --2- 172.21.15.16:0/3540454805 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a73bf61b40 0x55a73bf63120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:03:32.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.050+0000 7f375ffff640 1 --2- 172.21.15.16:0/3540454805 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a73bf6b370 0x55a73bf63660 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f3758010750 tx=0x7f375801a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff753f7f57667187 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:32.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.051+0000 7f37457fa640 1 -- 172.21.15.16:0/3540454805 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f375801e190 con 0x55a73bf6b370 2024-09-12T02:03:32.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.051+0000 7f37457fa640 1 -- 172.21.15.16:0/3540454805 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f375802d070 con 0x55a73bf6b370 2024-09-12T02:03:32.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.051+0000 7f37457fa640 1 -- 172.21.15.16:0/3540454805 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3758028a40 con 0x55a73bf6b370 2024-09-12T02:03:32.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.051+0000 7f3768288740 1 -- 172.21.15.16:0/3540454805 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a73bf6b370 msgr2=0x55a73bf63660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:32.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.051+0000 7f3768288740 1 --2- 172.21.15.16:0/3540454805 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a73bf6b370 0x55a73bf63660 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f3758010750 tx=0x7f375801a000 comp rx=0 tx=0).stop 2024-09-12T02:03:32.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.051+0000 7f3768288740 1 -- 172.21.15.16:0/3540454805 shutdown_connections 2024-09-12T02:03:32.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.051+0000 7f3768288740 1 --2- 172.21.15.16:0/3540454805 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a73bf6b370 0x55a73bf63660 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:32.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.051+0000 7f3768288740 1 --2- 172.21.15.16:0/3540454805 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a73bf61b40 0x55a73bf63120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:32.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.051+0000 7f3768288740 1 --2- 172.21.15.16:0/3540454805 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a73bf605d0 0x55a73bf69f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:32.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.051+0000 7f3768288740 1 -- 172.21.15.16:0/3540454805 >> 172.21.15.16:0/3540454805 conn(0x55a73bf29310 msgr2=0x55a73bf69870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:32.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.051+0000 7f3768288740 1 -- 172.21.15.16:0/3540454805 shutdown_connections 2024-09-12T02:03:32.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.051+0000 7f3768288740 1 -- 172.21.15.16:0/3540454805 wait complete. 2024-09-12T02:03:32.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.052+0000 7f3768288740 1 -- start start 2024-09-12T02:03:32.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.052+0000 7f3768288740 1 Processor -- start 2024-09-12T02:03:32.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.052+0000 7f3768288740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a73bf605d0 0x55a73bf73480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:32.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.052+0000 7f3768288740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a73bf61b40 0x55a73bf739c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:32.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.052+0000 7f3768288740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a73bf6b370 0x55a73bf70450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:32.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.052+0000 7f3768288740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f37601adbb0 con 0x55a73bf6b370 2024-09-12T02:03:32.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.052+0000 7f3768288740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f37601af020 con 0x55a73bf605d0 2024-09-12T02:03:32.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.052+0000 7f3768288740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f37601ae280 con 0x55a73bf61b40 2024-09-12T02:03:32.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.052+0000 7f3768288740 10 client.0 _ll_register_callbacks cb 0x7f37540b7820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:03:32.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.052+0000 7f375ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a73bf6b370 0x55a73bf70450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:32.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.052+0000 7f375f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a73bf605d0 0x55a73bf73480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:32.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.052+0000 7f375ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a73bf6b370 0x55a73bf70450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51732/0 (socket says 172.21.15.16:51732) 2024-09-12T02:03:32.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.052+0000 7f375ffff640 1 -- 172.21.15.16:0/820687879 learned_addr learned my addr 172.21.15.16:0/820687879 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:03:32.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.052+0000 7f375f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a73bf605d0 0x55a73bf73480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.16:50036/0 (socket says 172.21.15.16:50036) 2024-09-12T02:03:32.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.053+0000 7f375effd640 1 --2- 172.21.15.16:0/820687879 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a73bf61b40 0x55a73bf739c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:32.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.053+0000 7f375ffff640 1 -- 172.21.15.16:0/820687879 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a73bf61b40 msgr2=0x55a73bf739c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:32.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.053+0000 7f375ffff640 1 --2- 172.21.15.16:0/820687879 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a73bf61b40 0x55a73bf739c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:32.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.053+0000 7f375ffff640 1 -- 172.21.15.16:0/820687879 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a73bf605d0 msgr2=0x55a73bf73480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:32.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.053+0000 7f375ffff640 1 --2- 172.21.15.16:0/820687879 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a73bf605d0 0x55a73bf73480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:32.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.053+0000 7f375ffff640 1 -- 172.21.15.16:0/820687879 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a73bf65f30 con 0x55a73bf6b370 2024-09-12T02:03:32.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.053+0000 7f375f7fe640 1 --2- 172.21.15.16:0/820687879 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a73bf605d0 0x55a73bf73480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:03:32.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.053+0000 7f375ffff640 1 --2- 172.21.15.16:0/820687879 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a73bf6b370 0x55a73bf70450 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f3758012420 tx=0x7f375801e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cbe35ec44951201 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:32.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.053+0000 7f373effd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:03:32.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.053+0000 7f373effd640 1 -- 172.21.15.16:0/820687879 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3758035070 con 0x55a73bf6b370 2024-09-12T02:03:32.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.053+0000 7f373effd640 1 -- 172.21.15.16:0/820687879 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f375801e820 con 0x55a73bf6b370 2024-09-12T02:03:32.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.053+0000 7f3768288740 1 -- 172.21.15.16:0/820687879 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f375802d070 con 0x55a73bf6b370 2024-09-12T02:03:32.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.053+0000 7f373effd640 1 -- 172.21.15.16:0/820687879 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f3758041050 con 0x55a73bf6b370 2024-09-12T02:03:32.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.053+0000 7f3768288740 10 client.15858 fetch_fsmap learned FSMap version 25 2024-09-12T02:03:32.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.053+0000 7f3768288740 1 -- 172.21.15.16:0/820687879 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a73bfe6010 con 0x55a73bf6b370 2024-09-12T02:03:32.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.054+0000 7f373effd640 1 -- 172.21.15.16:0/820687879 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f375802d070 con 0x55a73bf6b370 2024-09-12T02:03:32.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.054+0000 7f373effd640 1 -- 172.21.15.16:0/820687879 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f375801b070 con 0x55a73bf6b370 2024-09-12T02:03:32.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.054+0000 7f3768288740 10 client.15858 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:03:32.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.054+0000 7f3768288740 10 client.15858 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-12T02:03:32.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.054+0000 7f3768288740 20 client.15858 populate_metadata read hostname 'smithi016' 2024-09-12T02:03:32.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.054+0000 7f3768288740 1 --2- 172.21.15.16:0/820687879 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55a73bfe6700 0x55a73bfe8b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:32.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.054+0000 7f3768288740 4 client.15858 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T02:03:32.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.054+0000 7f3768288740 1 -- 172.21.15.16:0/820687879 --> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a73becdd70 con 0x55a73bfe6700 2024-09-12T02:03:32.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.054+0000 7f375f7fe640 1 --2- 172.21.15.16:0/820687879 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55a73bfe6700 0x55a73bfe8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:32.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.054+0000 7f373effd640 10 client.15858 ms_handle_connect on v2:172.21.15.184:6836/1793393087 2024-09-12T02:03:32.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.054+0000 7f375f7fe640 1 --2- 172.21.15.16:0/820687879 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55a73bfe6700 0x55a73bfe8b20 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=17844931fdb2183a server_cookie=af27a4e56ad9dbb2 in_seq=0 out_seq=0 2024-09-12T02:03:32.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.055+0000 7f373effd640 1 -- 172.21.15.16:0/820687879 <== mds.2 v2:172.21.15.184:6836/1793393087 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a73becdd70 con 0x55a73bfe6700 2024-09-12T02:03:32.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.055+0000 7f373effd640 10 client.15858 handle_command_reply: tid=0 2024-09-12T02:03:32.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.055+0000 7f3768288740 1 client.15858 shutdown 2024-09-12T02:03:32.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.055+0000 7f3768288740 1 -- 172.21.15.16:0/820687879 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a73bf6b370 msgr2=0x55a73bf70450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:32.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.055+0000 7f3768288740 1 --2- 172.21.15.16:0/820687879 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a73bf6b370 0x55a73bf70450 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f3758012420 tx=0x7f375801e3c0 comp rx=0 tx=0).stop 2024-09-12T02:03:32.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.055+0000 7f3768288740 1 -- 172.21.15.16:0/820687879 shutdown_connections 2024-09-12T02:03:32.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.055+0000 7f3768288740 1 --2- 172.21.15.16:0/820687879 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55a73bfe6700 0x55a73bfe8b20 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:32.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.055+0000 7f3768288740 1 --2- 172.21.15.16:0/820687879 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a73bf6b370 0x55a73bf70450 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:32.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.055+0000 7f3768288740 1 --2- 172.21.15.16:0/820687879 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a73bf61b40 0x55a73bf739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:32.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.055+0000 7f373effd640 0 client.15858 ms_handle_reset on v2:172.21.15.184:6836/1793393087 2024-09-12T02:03:32.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.055+0000 7f3768288740 1 --2- 172.21.15.16:0/820687879 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a73bf605d0 0x55a73bf73480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:32.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.055+0000 7f3768288740 1 -- 172.21.15.16:0/820687879 >> 172.21.15.16:0/820687879 conn(0x55a73bf29310 msgr2=0x55a73bf6a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:32.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.055+0000 7f3768288740 1 -- 172.21.15.16:0/820687879 shutdown_connections 2024-09-12T02:03:32.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.055+0000 7f3768288740 1 -- 172.21.15.16:0/820687879 wait complete. 2024-09-12T02:03:32.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.055+0000 7f3768288740 20 client.15858 trim_cache size 0 max 16384 2024-09-12T02:03:32.061 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T02:03:32.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.056+0000 7f3765eaf640 1 -- 172.21.15.16:0/419521774 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f37480a2b80 msgr2=0x7f37480a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:32.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.056+0000 7f3765eaf640 1 --2- 172.21.15.16:0/419521774 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f37480a2b80 0x7f37480a5040 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f3758005fd0 tx=0x7f3758004a40 comp rx=0 tx=0).stop 2024-09-12T02:03:32.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.056+0000 7f3765eaf640 1 -- 172.21.15.16:0/419521774 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3760102790 msgr2=0x7f376019ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:32.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.056+0000 7f3765eaf640 1 --2- 172.21.15.16:0/419521774 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3760102790 0x7f376019ac10 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f3754006010 tx=0x7f3754004040 comp rx=0 tx=0).stop 2024-09-12T02:03:32.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.056+0000 7f3765eaf640 1 -- 172.21.15.16:0/419521774 shutdown_connections 2024-09-12T02:03:32.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.056+0000 7f3765eaf640 1 --2- 172.21.15.16:0/419521774 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f37480a2b80 0x7f37480a5040 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:32.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.056+0000 7f3765eaf640 1 --2- 172.21.15.16:0/419521774 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3760106f10 0x7f376019efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:32.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.056+0000 7f3765eaf640 1 --2- 172.21.15.16:0/419521774 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3760102790 0x7f376019ac10 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:32.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.056+0000 7f3765eaf640 1 --2- 172.21.15.16:0/419521774 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3760101de0 0x7f376019a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:32.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.056+0000 7f3765eaf640 1 -- 172.21.15.16:0/419521774 >> 172.21.15.16:0/419521774 conn(0x7f37600fd8c0 msgr2=0x7f37600fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:32.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.056+0000 7f3765eaf640 1 -- 172.21.15.16:0/419521774 shutdown_connections 2024-09-12T02:03:32.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:32.056+0000 7f3765eaf640 1 -- 172.21.15.16:0/419521774 wait complete. 2024-09-12T02:03:32.746 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-12T02:03:33.002 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:03:33.381 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 13.4 2024-09-12T02:03:33.381 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:03:33.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.475+0000 7f750ac02640 1 -- 172.21.15.16:0/710538472 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75041036e0 msgr2=0x7f7504107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:33.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.475+0000 7f750ac02640 1 --2- 172.21.15.16:0/710538472 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75041036e0 0x7f7504107b90 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f74f4009cd0 tx=0x7f74f402a8e0 comp rx=0 tx=0).stop 2024-09-12T02:03:33.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.475+0000 7f750ac02640 1 -- 172.21.15.16:0/710538472 shutdown_connections 2024-09-12T02:03:33.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.475+0000 7f750ac02640 1 --2- 172.21.15.16:0/710538472 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f75041082a0 0x7f750410ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:33.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.475+0000 7f750ac02640 1 --2- 172.21.15.16:0/710538472 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75041036e0 0x7f7504107b90 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:33.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.475+0000 7f750ac02640 1 --2- 172.21.15.16:0/710538472 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7504102d30 0x7f7504103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:33.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.475+0000 7f750ac02640 1 -- 172.21.15.16:0/710538472 >> 172.21.15.16:0/710538472 conn(0x7f75040fc7d0 msgr2=0x7f75040febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:33.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.475+0000 7f750ac02640 1 -- 172.21.15.16:0/710538472 shutdown_connections 2024-09-12T02:03:33.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.475+0000 7f750ac02640 1 -- 172.21.15.16:0/710538472 wait complete. 2024-09-12T02:03:33.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.476+0000 7f750ac02640 1 Processor -- start 2024-09-12T02:03:33.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.476+0000 7f750ac02640 1 -- start start 2024-09-12T02:03:33.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.476+0000 7f750ac02640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7504102d30 0x7f7504100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:33.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.476+0000 7f750ac02640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f75041036e0 0x7f7504100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:33.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.476+0000 7f750ac02640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75041082a0 0x7f7504101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:33.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.476+0000 7f750ac02640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f75041120a0 con 0x7f75041082a0 2024-09-12T02:03:33.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.476+0000 7f750ac02640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7504111f20 con 0x7f75041036e0 2024-09-12T02:03:33.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.476+0000 7f750ac02640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7504112220 con 0x7f7504102d30 2024-09-12T02:03:33.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.476+0000 7f750919f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75041082a0 0x7f7504101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:33.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.476+0000 7f750919f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75041082a0 0x7f7504101410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51740/0 (socket says 172.21.15.16:51740) 2024-09-12T02:03:33.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.476+0000 7f750919f640 1 -- 172.21.15.16:0/3197715011 learned_addr learned my addr 172.21.15.16:0/3197715011 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:03:33.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.476+0000 7f750899e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7504102d30 0x7f7504100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:33.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.476+0000 7f7503fff640 1 --2- 172.21.15.16:0/3197715011 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f75041036e0 0x7f7504100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:33.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.476+0000 7f750919f640 1 -- 172.21.15.16:0/3197715011 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7504102d30 msgr2=0x7f7504100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:33.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.476+0000 7f750919f640 1 --2- 172.21.15.16:0/3197715011 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7504102d30 0x7f7504100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:33.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.476+0000 7f750919f640 1 -- 172.21.15.16:0/3197715011 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f75041036e0 msgr2=0x7f7504100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:33.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.476+0000 7f750919f640 1 --2- 172.21.15.16:0/3197715011 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f75041036e0 0x7f7504100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:33.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.476+0000 7f750919f640 1 -- 172.21.15.16:0/3197715011 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75041a6a70 con 0x7f75041082a0 2024-09-12T02:03:33.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.476+0000 7f7503fff640 1 --2- 172.21.15.16:0/3197715011 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f75041036e0 0x7f7504100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:03:33.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.477+0000 7f750919f640 1 --2- 172.21.15.16:0/3197715011 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75041082a0 0x7f7504101410 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f74fc00d750 tx=0x7f74fc00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:33.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.477+0000 7f7501ffb640 1 -- 172.21.15.16:0/3197715011 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74fc017070 con 0x7f75041082a0 2024-09-12T02:03:33.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.477+0000 7f7501ffb640 1 -- 172.21.15.16:0/3197715011 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f74fc005500 con 0x7f75041082a0 2024-09-12T02:03:33.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.477+0000 7f750ac02640 1 -- 172.21.15.16:0/3197715011 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75041a6d60 con 0x7f75041082a0 2024-09-12T02:03:33.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.477+0000 7f7501ffb640 1 -- 172.21.15.16:0/3197715011 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74fc004d90 con 0x7f75041082a0 2024-09-12T02:03:33.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.477+0000 7f750ac02640 1 -- 172.21.15.16:0/3197715011 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75041a7270 con 0x7f75041082a0 2024-09-12T02:03:33.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.480+0000 7f750cfdb740 1 Processor -- start 2024-09-12T02:03:33.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.480+0000 7f7501ffb640 1 -- 172.21.15.16:0/3197715011 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f74fc005020 con 0x7f75041082a0 2024-09-12T02:03:33.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.480+0000 7f750cfdb740 1 -- start start 2024-09-12T02:03:33.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.480+0000 7f750cfdb740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56356b835b40 0x56356b834090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:33.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.480+0000 7f750cfdb740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56356b8345d0 0x56356b834a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:33.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.480+0000 7f750cfdb740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56356b83f370 0x56356b841760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:33.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.480+0000 7f750cfdb740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56356b79a2f0 con 0x56356b83f370 2024-09-12T02:03:33.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.480+0000 7f750cfdb740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56356b741cf0 con 0x56356b8345d0 2024-09-12T02:03:33.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.480+0000 7f750cfdb740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x56356b7962b0 con 0x56356b835b40 2024-09-12T02:03:33.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.480+0000 7f7503fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56356b83f370 0x56356b841760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:33.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.480+0000 7f7503fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56356b83f370 0x56356b841760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51756/0 (socket says 172.21.15.16:51756) 2024-09-12T02:03:33.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.480+0000 7f7503fff640 1 -- 172.21.15.16:0/393875537 learned_addr learned my addr 172.21.15.16:0/393875537 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:03:33.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.480+0000 7f750899e640 1 --2- 172.21.15.16:0/393875537 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56356b8345d0 0x56356b834a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:33.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.481+0000 7f7503fff640 1 --2- 172.21.15.16:0/393875537 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56356b835b40 0x56356b834090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:33.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.481+0000 7f7501ffb640 1 --2- 172.21.15.16:0/3197715011 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f74f00a2b80 0x7f74f00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:33.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.481+0000 7f750899e640 1 -- 172.21.15.16:0/393875537 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56356b835b40 msgr2=0x56356b834090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:33.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.481+0000 7f750899e640 1 --2- 172.21.15.16:0/393875537 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56356b835b40 0x56356b834090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:33.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.481+0000 7f750899e640 1 -- 172.21.15.16:0/393875537 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56356b83f370 msgr2=0x56356b841760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:33.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.481+0000 7f750899e640 1 --2- 172.21.15.16:0/393875537 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56356b83f370 0x56356b841760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:33.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.481+0000 7f750899e640 1 -- 172.21.15.16:0/393875537 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56356b7263a0 con 0x56356b8345d0 2024-09-12T02:03:33.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.481+0000 7f750919f640 1 --2- 172.21.15.16:0/3197715011 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f74f00a2b80 0x7f74f00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:33.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.481+0000 7f750899e640 1 --2- 172.21.15.16:0/393875537 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56356b8345d0 0x56356b834a10 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f74ec009c20 tx=0x7f74ec011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4bb6fa8085157ee8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:33.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.481+0000 7f7501ffb640 1 -- 172.21.15.16:0/3197715011 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f74fc0ce840 con 0x7f75041082a0 2024-09-12T02:03:33.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.481+0000 7f74faffd640 1 -- 172.21.15.16:0/393875537 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74ec004050 con 0x56356b8345d0 2024-09-12T02:03:33.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.481+0000 7f74faffd640 1 -- 172.21.15.16:0/393875537 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f74ec005500 con 0x56356b8345d0 2024-09-12T02:03:33.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.481+0000 7f74faffd640 1 -- 172.21.15.16:0/393875537 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74ec004d90 con 0x56356b8345d0 2024-09-12T02:03:33.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.482+0000 7f750cfdb740 1 -- 172.21.15.16:0/393875537 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56356b8345d0 msgr2=0x56356b834a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:33.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.482+0000 7f750919f640 1 --2- 172.21.15.16:0/3197715011 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f74f00a2b80 0x7f74f00a5040 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f74fc096ed0 tx=0x7f74fc06e2b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:33.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.482+0000 7f750cfdb740 1 --2- 172.21.15.16:0/393875537 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56356b8345d0 0x56356b834a10 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f74ec009c20 tx=0x7f74ec011810 comp rx=0 tx=0).stop 2024-09-12T02:03:33.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.482+0000 7f750cfdb740 1 -- 172.21.15.16:0/393875537 shutdown_connections 2024-09-12T02:03:33.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.482+0000 7f750cfdb740 1 --2- 172.21.15.16:0/393875537 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56356b83f370 0x56356b841760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:33.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.482+0000 7f750cfdb740 1 --2- 172.21.15.16:0/393875537 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56356b8345d0 0x56356b834a10 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:33.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.482+0000 7f750cfdb740 1 --2- 172.21.15.16:0/393875537 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56356b835b40 0x56356b834090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:33.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.482+0000 7f750cfdb740 1 -- 172.21.15.16:0/393875537 >> 172.21.15.16:0/393875537 conn(0x56356b7fd310 msgr2=0x56356b830c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:33.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.482+0000 7f750cfdb740 1 -- 172.21.15.16:0/393875537 shutdown_connections 2024-09-12T02:03:33.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.482+0000 7f750cfdb740 1 -- 172.21.15.16:0/393875537 wait complete. 2024-09-12T02:03:33.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.482+0000 7f750cfdb740 1 -- start start 2024-09-12T02:03:33.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.483+0000 7f750cfdb740 1 Processor -- start 2024-09-12T02:03:33.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.483+0000 7f750cfdb740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56356b8345d0 0x56356b83ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:33.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.483+0000 7f750cfdb740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56356b835b40 0x56356b83bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:33.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.483+0000 7f750cfdb740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56356b83f370 0x56356b83c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:33.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.483+0000 7f750cfdb740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56356b842070 con 0x56356b83f370 2024-09-12T02:03:33.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.483+0000 7f750cfdb740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56356b755060 con 0x56356b835b40 2024-09-12T02:03:33.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.483+0000 7f750cfdb740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x56356b8421f0 con 0x56356b8345d0 2024-09-12T02:03:33.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.483+0000 7f750cfdb740 10 client.0 _ll_register_callbacks cb 0x56356b4a88c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:03:33.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.483+0000 7f7503fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56356b83f370 0x56356b83c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:33.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.483+0000 7f7503fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56356b83f370 0x56356b83c470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51764/0 (socket says 172.21.15.16:51764) 2024-09-12T02:03:33.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.483+0000 7f7503fff640 1 -- 172.21.15.16:0/4159288222 learned_addr learned my addr 172.21.15.16:0/4159288222 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:03:33.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.483+0000 7f7503fff640 1 --2- 172.21.15.16:0/4159288222 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56356b8345d0 0x56356b83ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:33.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.483+0000 7f750899e640 1 --2- 172.21.15.16:0/4159288222 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56356b835b40 0x56356b83bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:33.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.484+0000 7f7503fff640 1 -- 172.21.15.16:0/4159288222 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56356b8345d0 msgr2=0x56356b83ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:33.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.484+0000 7f7503fff640 1 --2- 172.21.15.16:0/4159288222 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56356b8345d0 0x56356b83ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:33.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.484+0000 7f7503fff640 1 -- 172.21.15.16:0/4159288222 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56356b835b40 msgr2=0x56356b83bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:33.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.484+0000 7f7503fff640 1 --2- 172.21.15.16:0/4159288222 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56356b835b40 0x56356b83bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:33.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.484+0000 7f7503fff640 1 -- 172.21.15.16:0/4159288222 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56356b50b3c0 con 0x56356b83f370 2024-09-12T02:03:33.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.484+0000 7f7503fff640 1 --2- 172.21.15.16:0/4159288222 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56356b83f370 0x56356b83c470 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f74f402ae00 tx=0x7f74f4005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb6e6538cbb88cfd server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:33.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.484+0000 7f74e3fff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:03:33.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.484+0000 7f74e3fff640 1 -- 172.21.15.16:0/4159288222 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74f4051060 con 0x56356b83f370 2024-09-12T02:03:33.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.484+0000 7f750cfdb740 1 -- 172.21.15.16:0/4159288222 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f74ec005500 con 0x56356b83f370 2024-09-12T02:03:33.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.484+0000 7f74e3fff640 1 -- 172.21.15.16:0/4159288222 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f74f4030870 con 0x56356b83f370 2024-09-12T02:03:33.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.484+0000 7f74e3fff640 1 -- 172.21.15.16:0/4159288222 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f74f4030a70 con 0x56356b83f370 2024-09-12T02:03:33.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.484+0000 7f750cfdb740 10 client.15870 fetch_fsmap learned FSMap version 25 2024-09-12T02:03:33.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.484+0000 7f750cfdb740 1 -- 172.21.15.16:0/4159288222 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56356b84da90 con 0x56356b83f370 2024-09-12T02:03:33.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.484+0000 7f74e3fff640 1 -- 172.21.15.16:0/4159288222 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f74ec005500 con 0x56356b83f370 2024-09-12T02:03:33.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.485+0000 7f74e3fff640 1 -- 172.21.15.16:0/4159288222 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f74f4030070 con 0x56356b83f370 2024-09-12T02:03:33.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.485+0000 7f750cfdb740 10 client.15870 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:03:33.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.485+0000 7f750cfdb740 10 client.15870 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T02:03:33.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.485+0000 7f750cfdb740 20 client.15870 populate_metadata read hostname 'smithi016' 2024-09-12T02:03:33.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.485+0000 7f750cfdb740 1 --2- 172.21.15.16:0/4159288222 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x56356b84ef50 0x56356b851340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:33.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.485+0000 7f750cfdb740 4 client.15870 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:03:33.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.485+0000 7f750cfdb740 1 -- 172.21.15.16:0/4159288222 --> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56356b7a7860 con 0x56356b84ef50 2024-09-12T02:03:33.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.485+0000 7f750899e640 1 --2- 172.21.15.16:0/4159288222 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x56356b84ef50 0x56356b851340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:33.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.485+0000 7f750899e640 1 --2- 172.21.15.16:0/4159288222 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x56356b84ef50 0x56356b851340 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2de9ebbcf9408e07 server_cookie=507bf5c6b2b770 in_seq=0 out_seq=0 2024-09-12T02:03:33.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.485+0000 7f74e3fff640 10 client.15870 ms_handle_connect on v2:172.21.15.184:6838/4203347941 2024-09-12T02:03:33.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.486+0000 7f74e3fff640 1 -- 172.21.15.16:0/4159288222 <== mds.1 v2:172.21.15.184:6838/4203347941 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56356b7a7860 con 0x56356b84ef50 2024-09-12T02:03:33.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.486+0000 7f74e3fff640 10 client.15870 handle_command_reply: tid=0 2024-09-12T02:03:33.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.486+0000 7f750cfdb740 1 client.15870 shutdown 2024-09-12T02:03:33.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.486+0000 7f750cfdb740 1 -- 172.21.15.16:0/4159288222 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56356b83f370 msgr2=0x56356b83c470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:33.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.486+0000 7f750cfdb740 1 --2- 172.21.15.16:0/4159288222 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56356b83f370 0x56356b83c470 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f74f402ae00 tx=0x7f74f4005f10 comp rx=0 tx=0).stop 2024-09-12T02:03:33.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.486+0000 7f750cfdb740 1 -- 172.21.15.16:0/4159288222 shutdown_connections 2024-09-12T02:03:33.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.486+0000 7f750cfdb740 1 --2- 172.21.15.16:0/4159288222 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x56356b84ef50 0x56356b851340 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:33.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.486+0000 7f750cfdb740 1 --2- 172.21.15.16:0/4159288222 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56356b83f370 0x56356b83c470 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:33.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.486+0000 7f750cfdb740 1 --2- 172.21.15.16:0/4159288222 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56356b835b40 0x56356b83bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:33.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.486+0000 7f750cfdb740 1 --2- 172.21.15.16:0/4159288222 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56356b8345d0 0x56356b83ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:33.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.486+0000 7f74e3fff640 0 client.15870 ms_handle_reset on v2:172.21.15.184:6838/4203347941 2024-09-12T02:03:33.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.486+0000 7f750cfdb740 1 -- 172.21.15.16:0/4159288222 >> 172.21.15.16:0/4159288222 conn(0x56356b7fd310 msgr2=0x56356b836960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:33.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.487+0000 7f750cfdb740 1 -- 172.21.15.16:0/4159288222 shutdown_connections 2024-09-12T02:03:33.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.487+0000 7f750cfdb740 1 -- 172.21.15.16:0/4159288222 wait complete. 2024-09-12T02:03:33.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.487+0000 7f750cfdb740 20 client.15870 trim_cache size 0 max 16384 2024-09-12T02:03:33.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.513+0000 7f750cfdb740 1 Processor -- start 2024-09-12T02:03:33.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.513+0000 7f750cfdb740 1 -- start start 2024-09-12T02:03:33.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.513+0000 7f750cfdb740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56356b8345d0 0x56356b83dea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:33.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.513+0000 7f750cfdb740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56356b835b40 0x56356b837120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:33.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.513+0000 7f750cfdb740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56356b83f370 0x56356b837660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:33.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.513+0000 7f750cfdb740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f75041b1f00 con 0x56356b8345d0 2024-09-12T02:03:33.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.513+0000 7f750cfdb740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f75041af7d0 con 0x56356b835b40 2024-09-12T02:03:33.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.513+0000 7f7503fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56356b8345d0 0x56356b83dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:33.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.513+0000 7f750cfdb740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f75041ad6d0 con 0x56356b83f370 2024-09-12T02:03:33.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.513+0000 7f7503fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56356b8345d0 0x56356b83dea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51776/0 (socket says 172.21.15.16:51776) 2024-09-12T02:03:33.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.513+0000 7f7503fff640 1 -- 172.21.15.16:0/3859199128 learned_addr learned my addr 172.21.15.16:0/3859199128 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:03:33.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.514+0000 7f750899e640 1 --2- 172.21.15.16:0/3859199128 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56356b835b40 0x56356b837120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:33.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.514+0000 7f7503fff640 1 -- 172.21.15.16:0/3859199128 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56356b83f370 msgr2=0x56356b837660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:33.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.514+0000 7f7503fff640 1 --2- 172.21.15.16:0/3859199128 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56356b83f370 0x56356b837660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:33.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.514+0000 7f7503fff640 1 -- 172.21.15.16:0/3859199128 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56356b835b40 msgr2=0x56356b837120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:33.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.514+0000 7f7503fff640 1 --2- 172.21.15.16:0/3859199128 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56356b835b40 0x56356b837120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:33.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.514+0000 7f7503fff640 1 -- 172.21.15.16:0/3859199128 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56356b83af00 con 0x56356b8345d0 2024-09-12T02:03:33.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.514+0000 7f7503fff640 1 --2- 172.21.15.16:0/3859199128 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56356b8345d0 0x56356b83dea0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f74f40039f0 tx=0x7f74f4031800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b671820ed2600dab server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:33.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.514+0000 7f74fa7fc640 1 -- 172.21.15.16:0/3859199128 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74f405b020 con 0x56356b8345d0 2024-09-12T02:03:33.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.514+0000 7f74fa7fc640 1 -- 172.21.15.16:0/3859199128 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f74f4036030 con 0x56356b8345d0 2024-09-12T02:03:33.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.514+0000 7f74fa7fc640 1 -- 172.21.15.16:0/3859199128 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74f404c930 con 0x56356b8345d0 2024-09-12T02:03:33.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.515+0000 7f750cfdb740 1 -- 172.21.15.16:0/3859199128 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56356b8345d0 msgr2=0x56356b83dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:33.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.515+0000 7f750cfdb740 1 --2- 172.21.15.16:0/3859199128 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56356b8345d0 0x56356b83dea0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f74f40039f0 tx=0x7f74f4031800 comp rx=0 tx=0).stop 2024-09-12T02:03:33.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.515+0000 7f750cfdb740 1 -- 172.21.15.16:0/3859199128 shutdown_connections 2024-09-12T02:03:33.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.515+0000 7f750cfdb740 1 --2- 172.21.15.16:0/3859199128 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56356b83f370 0x56356b837660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:33.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.515+0000 7f750cfdb740 1 --2- 172.21.15.16:0/3859199128 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56356b835b40 0x56356b837120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:33.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.515+0000 7f750cfdb740 1 --2- 172.21.15.16:0/3859199128 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56356b8345d0 0x56356b83dea0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:33.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.515+0000 7f750cfdb740 1 -- 172.21.15.16:0/3859199128 >> 172.21.15.16:0/3859199128 conn(0x56356b7fd310 msgr2=0x56356b83d790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:33.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.515+0000 7f750cfdb740 1 -- 172.21.15.16:0/3859199128 shutdown_connections 2024-09-12T02:03:33.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.515+0000 7f750cfdb740 1 -- 172.21.15.16:0/3859199128 wait complete. 2024-09-12T02:03:33.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.515+0000 7f750cfdb740 1 -- start start 2024-09-12T02:03:33.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.516+0000 7f750cfdb740 1 Processor -- start 2024-09-12T02:03:33.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.516+0000 7f750cfdb740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56356b8345d0 0x56356b838e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:33.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.516+0000 7f750cfdb740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56356b835b40 0x56356b8393d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:33.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.516+0000 7f750cfdb740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56356b83f370 0x56356b839910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:33.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.516+0000 7f750cfdb740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f75041add20 con 0x56356b8345d0 2024-09-12T02:03:33.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.516+0000 7f750cfdb740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f75041af100 con 0x56356b835b40 2024-09-12T02:03:33.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.516+0000 7f750cfdb740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f75041ae360 con 0x56356b83f370 2024-09-12T02:03:33.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.516+0000 7f750cfdb740 10 client.0 _ll_register_callbacks cb 0x7f74f4031a90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:03:33.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.516+0000 7f7503fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56356b8345d0 0x56356b838e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:33.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.516+0000 7f7503fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56356b8345d0 0x56356b838e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51788/0 (socket says 172.21.15.16:51788) 2024-09-12T02:03:33.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.516+0000 7f750899e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56356b835b40 0x56356b8393d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:33.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.516+0000 7f7503fff640 1 -- 172.21.15.16:0/559237628 learned_addr learned my addr 172.21.15.16:0/559237628 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:03:33.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.516+0000 7f7503fff640 1 -- 172.21.15.16:0/559237628 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56356b83f370 msgr2=0x56356b839910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:33.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.516+0000 7f7503fff640 1 --2- 172.21.15.16:0/559237628 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56356b83f370 0x56356b839910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:33.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.517+0000 7f7503fff640 1 -- 172.21.15.16:0/559237628 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56356b835b40 msgr2=0x56356b8393d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:33.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.517+0000 7f7503fff640 1 --2- 172.21.15.16:0/559237628 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56356b835b40 0x56356b8393d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:33.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.517+0000 7f7503fff640 1 -- 172.21.15.16:0/559237628 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56356b8b8b70 con 0x56356b8345d0 2024-09-12T02:03:33.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.517+0000 7f7503fff640 1 --2- 172.21.15.16:0/559237628 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56356b8345d0 0x56356b838e90 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f74f402ae00 tx=0x7f74f402b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb0b7288ecef0a3f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:33.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.517+0000 7f74e3fff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:03:33.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.517+0000 7f74e3fff640 1 -- 172.21.15.16:0/559237628 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74f4066020 con 0x56356b8345d0 2024-09-12T02:03:33.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.517+0000 7f74e3fff640 1 -- 172.21.15.16:0/559237628 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f74f4030870 con 0x56356b8345d0 2024-09-12T02:03:33.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.517+0000 7f750cfdb740 1 -- 172.21.15.16:0/559237628 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f74f4036030 con 0x56356b8345d0 2024-09-12T02:03:33.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.517+0000 7f74e3fff640 1 -- 172.21.15.16:0/559237628 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f74f405b030 con 0x56356b8345d0 2024-09-12T02:03:33.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.517+0000 7f750cfdb740 10 client.15876 fetch_fsmap learned FSMap version 25 2024-09-12T02:03:33.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.517+0000 7f750cfdb740 1 -- 172.21.15.16:0/559237628 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56356b8b9d70 con 0x56356b8345d0 2024-09-12T02:03:33.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.517+0000 7f74e3fff640 1 -- 172.21.15.16:0/559237628 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f74f4036030 con 0x56356b8345d0 2024-09-12T02:03:33.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.518+0000 7f74e3fff640 1 -- 172.21.15.16:0/559237628 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f74f403bc20 con 0x56356b8345d0 2024-09-12T02:03:33.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.518+0000 7f750cfdb740 10 client.15876 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:03:33.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.518+0000 7f750cfdb740 10 client.15876 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T02:03:33.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.518+0000 7f750cfdb740 20 client.15876 populate_metadata read hostname 'smithi016' 2024-09-12T02:03:33.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.518+0000 7f750cfdb740 1 --2- 172.21.15.16:0/559237628 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x56356b8ba420 0x56356b8bc840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:33.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.518+0000 7f750cfdb740 4 client.15876 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T02:03:33.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.518+0000 7f750cfdb740 1 -- 172.21.15.16:0/559237628 --> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56356b7a1d70 con 0x56356b8ba420 2024-09-12T02:03:33.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.518+0000 7f750899e640 1 --2- 172.21.15.16:0/559237628 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x56356b8ba420 0x56356b8bc840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:33.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.518+0000 7f750899e640 1 --2- 172.21.15.16:0/559237628 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x56356b8ba420 0x56356b8bc840 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=44ad61bb93227fbe server_cookie=34992bfb15c8e0ce in_seq=0 out_seq=0 2024-09-12T02:03:33.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.518+0000 7f74e3fff640 10 client.15876 ms_handle_connect on v2:172.21.15.184:6838/4203347941 2024-09-12T02:03:33.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.519+0000 7f74e3fff640 1 -- 172.21.15.16:0/559237628 <== mds.1 v2:172.21.15.184:6838/4203347941 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56356b7a1d70 con 0x56356b8ba420 2024-09-12T02:03:33.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.519+0000 7f74e3fff640 10 client.15876 handle_command_reply: tid=0 2024-09-12T02:03:33.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.519+0000 7f750cfdb740 1 client.15876 shutdown 2024-09-12T02:03:33.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.519+0000 7f750cfdb740 1 -- 172.21.15.16:0/559237628 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56356b8345d0 msgr2=0x56356b838e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:33.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.519+0000 7f750cfdb740 1 --2- 172.21.15.16:0/559237628 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56356b8345d0 0x56356b838e90 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f74f402ae00 tx=0x7f74f402b000 comp rx=0 tx=0).stop 2024-09-12T02:03:33.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.519+0000 7f750cfdb740 1 -- 172.21.15.16:0/559237628 shutdown_connections 2024-09-12T02:03:33.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.519+0000 7f750cfdb740 1 --2- 172.21.15.16:0/559237628 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x56356b8ba420 0x56356b8bc840 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:33.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.519+0000 7f750cfdb740 1 --2- 172.21.15.16:0/559237628 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56356b83f370 0x56356b839910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:33.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.519+0000 7f750cfdb740 1 --2- 172.21.15.16:0/559237628 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56356b835b40 0x56356b8393d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:33.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.519+0000 7f750cfdb740 1 --2- 172.21.15.16:0/559237628 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56356b8345d0 0x56356b838e90 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:33.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.519+0000 7f750cfdb740 1 -- 172.21.15.16:0/559237628 >> 172.21.15.16:0/559237628 conn(0x56356b7fd310 msgr2=0x56356b83dfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:33.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.519+0000 7f74e3fff640 0 client.15876 ms_handle_reset on v2:172.21.15.184:6838/4203347941 2024-09-12T02:03:33.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.519+0000 7f750cfdb740 1 -- 172.21.15.16:0/559237628 shutdown_connections 2024-09-12T02:03:33.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.519+0000 7f750cfdb740 1 -- 172.21.15.16:0/559237628 wait complete. 2024-09-12T02:03:33.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.519+0000 7f750cfdb740 20 client.15876 trim_cache size 0 max 16384 2024-09-12T02:03:33.524 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T02:03:33.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.520+0000 7f750ac02640 1 -- 172.21.15.16:0/3197715011 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f74f00a2b80 msgr2=0x7f74f00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:33.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.520+0000 7f750ac02640 1 --2- 172.21.15.16:0/3197715011 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f74f00a2b80 0x7f74f00a5040 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f74fc096ed0 tx=0x7f74fc06e2b0 comp rx=0 tx=0).stop 2024-09-12T02:03:33.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.520+0000 7f750ac02640 1 -- 172.21.15.16:0/3197715011 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75041082a0 msgr2=0x7f7504101410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:33.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.520+0000 7f750ac02640 1 --2- 172.21.15.16:0/3197715011 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75041082a0 0x7f7504101410 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f74fc00d750 tx=0x7f74fc00dc10 comp rx=0 tx=0).stop 2024-09-12T02:03:33.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.520+0000 7f750ac02640 1 -- 172.21.15.16:0/3197715011 shutdown_connections 2024-09-12T02:03:33.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.520+0000 7f750ac02640 1 --2- 172.21.15.16:0/3197715011 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f74f00a2b80 0x7f74f00a5040 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:33.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.520+0000 7f750ac02640 1 --2- 172.21.15.16:0/3197715011 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75041082a0 0x7f7504101410 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:33.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.520+0000 7f750ac02640 1 --2- 172.21.15.16:0/3197715011 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f75041036e0 0x7f7504100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:33.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.520+0000 7f750ac02640 1 --2- 172.21.15.16:0/3197715011 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7504102d30 0x7f7504100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:33.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.520+0000 7f750ac02640 1 -- 172.21.15.16:0/3197715011 >> 172.21.15.16:0/3197715011 conn(0x7f75040fc7d0 msgr2=0x7f7504105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:33.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.520+0000 7f750ac02640 1 -- 172.21.15.16:0/3197715011 shutdown_connections 2024-09-12T02:03:33.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:33.520+0000 7f750ac02640 1 -- 172.21.15.16:0/3197715011 wait complete. 2024-09-12T02:03:33.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:03:33 smithi016 ceph-mon[30947]: pgmap v384: 97 pgs: 97 active+clean; 1016 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-12T02:03:33.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:03:33 smithi184 ceph-mon[35593]: pgmap v384: 97 pgs: 97 active+clean; 1016 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-12T02:03:33.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:03:33 smithi033 ceph-mon[35332]: pgmap v384: 97 pgs: 97 active+clean; 1016 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-12T02:03:34.250 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:03:34.284 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-12T02:03:34.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:34.761+0000 7fc41f562640 1 -- 172.21.15.16:0/50396289 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc418101420 msgr2=0x7fc41810d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:34.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:34.761+0000 7fc41f562640 1 --2- 172.21.15.16:0/50396289 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc418101420 0x7fc41810d950 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fc410009cd0 tx=0x7fc41002a8f0 comp rx=0 tx=0).stop 2024-09-12T02:03:34.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:34.761+0000 7fc41f562640 1 -- 172.21.15.16:0/50396289 shutdown_connections 2024-09-12T02:03:34.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:34.761+0000 7fc41f562640 1 --2- 172.21.15.16:0/50396289 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc418101420 0x7fc41810d950 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:34.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:34.761+0000 7fc41f562640 1 --2- 172.21.15.16:0/50396289 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc418100a80 0x7fc418100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:34.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:34.761+0000 7fc41f562640 1 --2- 172.21.15.16:0/50396289 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc418106a80 0x7fc418106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:34.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:34.761+0000 7fc41f562640 1 -- 172.21.15.16:0/50396289 >> 172.21.15.16:0/50396289 conn(0x7fc4180fc7a0 msgr2=0x7fc4180febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:34.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:34.761+0000 7fc41f562640 1 -- 172.21.15.16:0/50396289 shutdown_connections 2024-09-12T02:03:34.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:34.761+0000 7fc41f562640 1 -- 172.21.15.16:0/50396289 wait complete. 2024-09-12T02:03:34.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:34.762+0000 7fc41f562640 1 Processor -- start 2024-09-12T02:03:34.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:34.762+0000 7fc41f562640 1 -- start start 2024-09-12T02:03:34.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:34.762+0000 7fc41f562640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc418100a80 0x7fc4181028e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:34.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:34.762+0000 7fc41f562640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc418101420 0x7fc418102e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:34.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:34.762+0000 7fc41f562640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc418106a80 0x7fc418103360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:34.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:34.762+0000 7fc41f562640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc418110ed0 con 0x7fc418101420 2024-09-12T02:03:34.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:34.762+0000 7fc41f562640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc418110d50 con 0x7fc418100a80 2024-09-12T02:03:34.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:34.762+0000 7fc41f562640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc418111050 con 0x7fc418106a80 2024-09-12T02:03:34.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:34.762+0000 7fc41cafd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc418101420 0x7fc418102e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:34.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:34.762+0000 7fc41cafd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc418101420 0x7fc418102e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51804/0 (socket says 172.21.15.16:51804) 2024-09-12T02:03:34.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:34.762+0000 7fc41cafd640 1 -- 172.21.15.16:0/3599329407 learned_addr learned my addr 172.21.15.16:0/3599329407 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:03:34.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:34.762+0000 7fc41d2fe640 1 --2- 172.21.15.16:0/3599329407 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc418100a80 0x7fc4181028e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:34.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:34.762+0000 7fc41cafd640 1 -- 172.21.15.16:0/3599329407 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc418106a80 msgr2=0x7fc418103360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:34.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:34.762+0000 7fc41cafd640 1 --2- 172.21.15.16:0/3599329407 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc418106a80 0x7fc418103360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:34.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:34.762+0000 7fc41cafd640 1 -- 172.21.15.16:0/3599329407 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc418100a80 msgr2=0x7fc4181028e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:34.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:34.762+0000 7fc41cafd640 1 --2- 172.21.15.16:0/3599329407 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc418100a80 0x7fc4181028e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:34.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:34.762+0000 7fc41cafd640 1 -- 172.21.15.16:0/3599329407 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4181aacb0 con 0x7fc418101420 2024-09-12T02:03:34.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:34.763+0000 7fc41cafd640 1 --2- 172.21.15.16:0/3599329407 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc418101420 0x7fc418102e20 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fc40800c920 tx=0x7fc40800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:34.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:34.763+0000 7fc40e7fc640 1 -- 172.21.15.16:0/3599329407 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc408007c70 con 0x7fc418101420 2024-09-12T02:03:34.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:34.763+0000 7fc40e7fc640 1 -- 172.21.15.16:0/3599329407 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc408007e10 con 0x7fc418101420 2024-09-12T02:03:34.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:34.763+0000 7fc41f562640 1 -- 172.21.15.16:0/3599329407 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4181aaee0 con 0x7fc418101420 2024-09-12T02:03:34.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:34.763+0000 7fc40e7fc640 1 -- 172.21.15.16:0/3599329407 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc408018890 con 0x7fc418101420 2024-09-12T02:03:34.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:34.763+0000 7fc41f562640 1 -- 172.21.15.16:0/3599329407 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4181ab330 con 0x7fc418101420 2024-09-12T02:03:34.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:34.764+0000 7fc41f562640 1 -- 172.21.15.16:0/3599329407 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4181021c0 con 0x7fc418101420 2024-09-12T02:03:34.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:34.765+0000 7fc40e7fc640 1 -- 172.21.15.16:0/3599329407 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fc408018a70 con 0x7fc418101420 2024-09-12T02:03:34.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:34.766+0000 7fc40e7fc640 1 --2- 172.21.15.16:0/3599329407 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc4040a2b80 0x7fc4040a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:34.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:34.766+0000 7fc41d2fe640 1 --2- 172.21.15.16:0/3599329407 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc4040a2b80 0x7fc4040a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:34.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:34.766+0000 7fc40e7fc640 1 -- 172.21.15.16:0/3599329407 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fc4080cf1d0 con 0x7fc418101420 2024-09-12T02:03:34.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:34.766+0000 7fc41d2fe640 1 --2- 172.21.15.16:0/3599329407 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc4040a2b80 0x7fc4040a5040 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fc41400cac0 tx=0x7fc41400c4a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:34.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:34.770+0000 7fc40e7fc640 1 -- 172.21.15.16:0/3599329407 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc408014030 con 0x7fc418101420 2024-09-12T02:03:34.818 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:03:34.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:34.992+0000 7fc41f562640 1 -- 172.21.15.16:0/3599329407 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc4181ab860 con 0x7fc418101420 2024-09-12T02:03:34.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:34.993+0000 7fc40e7fc640 1 -- 172.21.15.16:0/3599329407 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11911 (secure 0 0 0) 0x7fc40800de20 con 0x7fc418101420 2024-09-12T02:03:34.993 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T02:03:34.994 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":25,"btime":"2024-09-12T02:01:37:192021+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3279123279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3279123279},{"type":"v1","addr":"172.21.15.16:6835","nonce":3279123279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4119452382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4119452382},{"type":"v1","addr":"172.21.15.16:6837","nonce":4119452382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/3263823165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3263823165},{"type":"v1","addr":"172.21.15.16:6839","nonce":3263823165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/1510290732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":1510290732},{"type":"v1","addr":"172.21.15.16:6841","nonce":1510290732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/279674603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":279674603},{"type":"v1","addr":"172.21.15.33:6833","nonce":279674603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3555728028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3555728028},{"type":"v1","addr":"172.21.15.33:6835","nonce":3555728028}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T01:59:56.117608+0000","modified":"2024-09-12T02:01:36.294760+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24482,"mds_1":24481,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6833/1563374106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1563374106},{"type":"v1","addr":"172.21.15.184:6833","nonce":1563374106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6837/1892833868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1892833868},{"type":"v1","addr":"172.21.15.33:6837","nonce":1892833868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/1457679686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1457679686},{"type":"v1","addr":"172.21.15.184:6835","nonce":1457679686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.184:6837/1793393087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":1793393087},{"type":"v1","addr":"172.21.15.184:6837","nonce":1793393087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.184:6839/4203347941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":4203347941},{"type":"v1","addr":"172.21.15.184:6839","nonce":4203347941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.33:6839/4136670204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":4136670204},{"type":"v1","addr":"172.21.15.33:6839","nonce":4136670204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24481,24482,24469]},"id":1}]} 2024-09-12T02:03:34.994 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 25 2024-09-12T02:03:34.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:34.995+0000 7fc41f562640 1 -- 172.21.15.16:0/3599329407 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc4040a2b80 msgr2=0x7fc4040a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:34.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:34.995+0000 7fc41f562640 1 --2- 172.21.15.16:0/3599329407 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc4040a2b80 0x7fc4040a5040 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fc41400cac0 tx=0x7fc41400c4a0 comp rx=0 tx=0).stop 2024-09-12T02:03:34.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:34.995+0000 7fc41f562640 1 -- 172.21.15.16:0/3599329407 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc418101420 msgr2=0x7fc418102e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:34.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:34.995+0000 7fc41f562640 1 --2- 172.21.15.16:0/3599329407 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc418101420 0x7fc418102e20 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fc40800c920 tx=0x7fc40800cde0 comp rx=0 tx=0).stop 2024-09-12T02:03:34.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:34.995+0000 7fc41f562640 1 -- 172.21.15.16:0/3599329407 shutdown_connections 2024-09-12T02:03:34.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:34.995+0000 7fc41f562640 1 --2- 172.21.15.16:0/3599329407 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc4040a2b80 0x7fc4040a5040 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:34.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:34.995+0000 7fc41f562640 1 --2- 172.21.15.16:0/3599329407 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc418106a80 0x7fc418103360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:34.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:34.995+0000 7fc41f562640 1 --2- 172.21.15.16:0/3599329407 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc418101420 0x7fc418102e20 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:34.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:34.995+0000 7fc41f562640 1 --2- 172.21.15.16:0/3599329407 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc418100a80 0x7fc4181028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:34.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:34.995+0000 7fc41f562640 1 -- 172.21.15.16:0/3599329407 >> 172.21.15.16:0/3599329407 conn(0x7fc4180fc7a0 msgr2=0x7fc418104a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:34.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:34.995+0000 7fc41f562640 1 -- 172.21.15.16:0/3599329407 shutdown_connections 2024-09-12T02:03:34.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:34.995+0000 7fc41f562640 1 -- 172.21.15.16:0/3599329407 wait complete. 2024-09-12T02:03:35.650 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24482 quiesce db / --timeout 90 --expiration 133.4 --await-for 45.0' 2024-09-12T02:03:35.650 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24482 quiesce db / --timeout 90 --expiration 133.4 --await-for 45.0 2024-09-12T02:03:35.747 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:03:35 smithi016 ceph-mon[30947]: pgmap v385: 97 pgs: 97 active+clean; 1.1 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-09-12T02:03:35.748 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:03:35 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/3599329407' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:03:35.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.833+0000 7efd8de6b640 1 -- 172.21.15.16:0/531698353 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd88106a80 msgr2=0x7efd88106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:35.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.833+0000 7efd8de6b640 1 --2- 172.21.15.16:0/531698353 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd88106a80 0x7efd88106e60 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7efd78009930 tx=0x7efd7802a420 comp rx=0 tx=0).stop 2024-09-12T02:03:35.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.833+0000 7efd8de6b640 1 -- 172.21.15.16:0/531698353 shutdown_connections 2024-09-12T02:03:35.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.833+0000 7efd8de6b640 1 --2- 172.21.15.16:0/531698353 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd88101420 0x7efd8810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:35.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.833+0000 7efd8de6b640 1 --2- 172.21.15.16:0/531698353 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efd88100a80 0x7efd88100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:35.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.833+0000 7efd8de6b640 1 --2- 172.21.15.16:0/531698353 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd88106a80 0x7efd88106e60 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:35.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.833+0000 7efd8de6b640 1 -- 172.21.15.16:0/531698353 >> 172.21.15.16:0/531698353 conn(0x7efd880fc7a0 msgr2=0x7efd880febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:35.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.833+0000 7efd8de6b640 1 -- 172.21.15.16:0/531698353 shutdown_connections 2024-09-12T02:03:35.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.833+0000 7efd8de6b640 1 -- 172.21.15.16:0/531698353 wait complete. 2024-09-12T02:03:35.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.834+0000 7efd8de6b640 1 Processor -- start 2024-09-12T02:03:35.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.834+0000 7efd8de6b640 1 -- start start 2024-09-12T02:03:35.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.834+0000 7efd8de6b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd88100a80 0x7efd88198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:35.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.834+0000 7efd8de6b640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efd88101420 0x7efd88198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:35.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.834+0000 7efd8de6b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd88106a80 0x7efd8819cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:35.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.834+0000 7efd8de6b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7efd88110ea0 con 0x7efd88106a80 2024-09-12T02:03:35.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.834+0000 7efd8de6b640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7efd88110d20 con 0x7efd88100a80 2024-09-12T02:03:35.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.834+0000 7efd8de6b640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7efd88111020 con 0x7efd88101420 2024-09-12T02:03:35.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.835+0000 7efd877fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd88100a80 0x7efd88198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:35.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.835+0000 7efd87fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd88106a80 0x7efd8819cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:35.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.835+0000 7efd877fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd88100a80 0x7efd88198520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.16:50126/0 (socket says 172.21.15.16:50126) 2024-09-12T02:03:35.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.835+0000 7efd877fe640 1 -- 172.21.15.16:0/3759511583 learned_addr learned my addr 172.21.15.16:0/3759511583 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:03:35.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.835+0000 7efd7ffff640 1 --2- 172.21.15.16:0/3759511583 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efd88101420 0x7efd88198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:35.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.835+0000 7efd87fff640 1 -- 172.21.15.16:0/3759511583 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efd88101420 msgr2=0x7efd88198a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:35.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.835+0000 7efd87fff640 1 --2- 172.21.15.16:0/3759511583 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efd88101420 0x7efd88198a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:35.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.835+0000 7efd87fff640 1 -- 172.21.15.16:0/3759511583 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd88100a80 msgr2=0x7efd88198520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:35.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.835+0000 7efd87fff640 1 --2- 172.21.15.16:0/3759511583 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd88100a80 0x7efd88198520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:35.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.835+0000 7efd87fff640 1 -- 172.21.15.16:0/3759511583 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd8819d4d0 con 0x7efd88106a80 2024-09-12T02:03:35.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.835+0000 7efd877fe640 1 --2- 172.21.15.16:0/3759511583 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd88100a80 0x7efd88198520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:03:35.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.835+0000 7efd87fff640 1 --2- 172.21.15.16:0/3759511583 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd88106a80 0x7efd8819cdf0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7efd8000ca30 tx=0x7efd8000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:35.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.835+0000 7efd857fa640 1 -- 172.21.15.16:0/3759511583 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd80004d20 con 0x7efd88106a80 2024-09-12T02:03:35.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.835+0000 7efd857fa640 1 -- 172.21.15.16:0/3759511583 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd80005020 con 0x7efd88106a80 2024-09-12T02:03:35.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.835+0000 7efd8de6b640 1 -- 172.21.15.16:0/3759511583 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd8819d7c0 con 0x7efd88106a80 2024-09-12T02:03:35.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.835+0000 7efd857fa640 1 -- 172.21.15.16:0/3759511583 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd8000f6b0 con 0x7efd88106a80 2024-09-12T02:03:35.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.836+0000 7efd8de6b640 1 -- 172.21.15.16:0/3759511583 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd881a50a0 con 0x7efd88106a80 2024-09-12T02:03:35.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.838+0000 7efd857fa640 1 -- 172.21.15.16:0/3759511583 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7efd80005560 con 0x7efd88106a80 2024-09-12T02:03:35.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.838+0000 7efd90244740 1 Processor -- start 2024-09-12T02:03:35.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.839+0000 7efd90244740 1 -- start start 2024-09-12T02:03:35.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.839+0000 7efd90244740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5584ee36bb40 0x5584ee36a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:35.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.839+0000 7efd90244740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5584ee36a5d0 0x5584ee36aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:35.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.839+0000 7efd90244740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5584ee379780 0x5584ee37bb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:35.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.839+0000 7efd90244740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5584ee2d02f0 con 0x5584ee379780 2024-09-12T02:03:35.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.839+0000 7efd90244740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5584ee277cf0 con 0x5584ee36bb40 2024-09-12T02:03:35.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.839+0000 7efd90244740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5584ee2cc2b0 con 0x5584ee36a5d0 2024-09-12T02:03:35.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.839+0000 7efd7ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5584ee379780 0x5584ee37bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:35.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.839+0000 7efd857fa640 1 --2- 172.21.15.16:0/3759511583 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7efd740a2c00 0x7efd740a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:35.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.839+0000 7efd7ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5584ee379780 0x5584ee37bb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51828/0 (socket says 172.21.15.16:51828) 2024-09-12T02:03:35.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.839+0000 7efd7ffff640 1 -- 172.21.15.16:0/3862311044 learned_addr learned my addr 172.21.15.16:0/3862311044 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:03:35.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.839+0000 7efd7ffff640 1 --2- 172.21.15.16:0/3862311044 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5584ee36bb40 0x5584ee36a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:35.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.839+0000 7efd877fe640 1 --2- 172.21.15.16:0/3862311044 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5584ee36a5d0 0x5584ee36aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:35.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.839+0000 7efd857fa640 1 -- 172.21.15.16:0/3759511583 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7efd800cf0f0 con 0x7efd88106a80 2024-09-12T02:03:35.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.840+0000 7efd87fff640 1 --2- 172.21.15.16:0/3759511583 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7efd740a2c00 0x7efd740a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:35.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.840+0000 7efd7ffff640 1 -- 172.21.15.16:0/3862311044 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5584ee36a5d0 msgr2=0x5584ee36aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:35.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.840+0000 7efd7ffff640 1 --2- 172.21.15.16:0/3862311044 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5584ee36a5d0 0x5584ee36aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:35.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.840+0000 7efd7ffff640 1 -- 172.21.15.16:0/3862311044 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5584ee379780 msgr2=0x5584ee37bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:35.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.840+0000 7efd7ffff640 1 --2- 172.21.15.16:0/3862311044 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5584ee379780 0x5584ee37bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:35.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.840+0000 7efd7ffff640 1 -- 172.21.15.16:0/3862311044 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5584ee25c3a0 con 0x5584ee36bb40 2024-09-12T02:03:35.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.840+0000 7efd7ffff640 1 --2- 172.21.15.16:0/3862311044 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5584ee36bb40 0x5584ee36a090 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x5584ee36b070 tx=0x7efd70009110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5ef6e933ccb66d0a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:35.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.840+0000 7efd87fff640 1 --2- 172.21.15.16:0/3759511583 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7efd740a2c00 0x7efd740a50c0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7efd80097e50 tx=0x7efd80086040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:35.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.840+0000 7efd7dffb640 1 -- 172.21.15.16:0/3862311044 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd70015730 con 0x5584ee36bb40 2024-09-12T02:03:35.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.841+0000 7efd7dffb640 1 -- 172.21.15.16:0/3862311044 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd7001d070 con 0x5584ee36bb40 2024-09-12T02:03:35.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.841+0000 7efd7dffb640 1 -- 172.21.15.16:0/3862311044 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd70018b40 con 0x5584ee36bb40 2024-09-12T02:03:35.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.842+0000 7efd90244740 1 -- 172.21.15.16:0/3862311044 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5584ee36bb40 msgr2=0x5584ee36a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:35.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.842+0000 7efd90244740 1 --2- 172.21.15.16:0/3862311044 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5584ee36bb40 0x5584ee36a090 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x5584ee36b070 tx=0x7efd70009110 comp rx=0 tx=0).stop 2024-09-12T02:03:35.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.842+0000 7efd90244740 1 -- 172.21.15.16:0/3862311044 shutdown_connections 2024-09-12T02:03:35.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.842+0000 7efd90244740 1 --2- 172.21.15.16:0/3862311044 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5584ee379780 0x5584ee37bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:35.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.842+0000 7efd90244740 1 --2- 172.21.15.16:0/3862311044 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5584ee36a5d0 0x5584ee36aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:35.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.842+0000 7efd90244740 1 --2- 172.21.15.16:0/3862311044 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5584ee36bb40 0x5584ee36a090 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:35.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.842+0000 7efd90244740 1 -- 172.21.15.16:0/3862311044 >> 172.21.15.16:0/3862311044 conn(0x5584ee333310 msgr2=0x5584ee366c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:35.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.842+0000 7efd90244740 1 -- 172.21.15.16:0/3862311044 shutdown_connections 2024-09-12T02:03:35.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.842+0000 7efd90244740 1 -- 172.21.15.16:0/3862311044 wait complete. 2024-09-12T02:03:35.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.842+0000 7efd90244740 1 -- start start 2024-09-12T02:03:35.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.842+0000 7efd90244740 1 Processor -- start 2024-09-12T02:03:35.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.843+0000 7efd90244740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5584ee36a5d0 0x5584ee3791f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:35.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.843+0000 7efd90244740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5584ee36bb40 0x5584ee3764a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:35.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.843+0000 7efd90244740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5584ee379780 0x5584ee376be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:35.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.843+0000 7efd90244740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5584ee37c420 con 0x5584ee379780 2024-09-12T02:03:35.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.843+0000 7efd90244740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5584ee28b060 con 0x5584ee36bb40 2024-09-12T02:03:35.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.843+0000 7efd90244740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5584ee37c5a0 con 0x5584ee36a5d0 2024-09-12T02:03:35.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.843+0000 7efd7ffff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5584ee36a5d0 0x5584ee3791f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:35.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.843+0000 7efd7ffff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5584ee36a5d0 0x5584ee3791f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:48962/0 (socket says 172.21.15.16:48962) 2024-09-12T02:03:35.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.843+0000 7efd7ffff640 1 -- 172.21.15.16:0/3188960228 learned_addr learned my addr 172.21.15.16:0/3188960228 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:03:35.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.843+0000 7efd7ffff640 1 --2- 172.21.15.16:0/3188960228 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5584ee379780 0x5584ee376be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:35.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.843+0000 7efd877fe640 1 --2- 172.21.15.16:0/3188960228 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5584ee36bb40 0x5584ee3764a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:35.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.843+0000 7efd90244740 10 client.0 _ll_register_callbacks cb 0x5584edfde8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:03:35.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.843+0000 7efd7ffff640 1 -- 172.21.15.16:0/3188960228 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5584ee36bb40 msgr2=0x5584ee3764a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:35.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.843+0000 7efd7ffff640 1 --2- 172.21.15.16:0/3188960228 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5584ee36bb40 0x5584ee3764a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:35.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.843+0000 7efd7ffff640 1 -- 172.21.15.16:0/3188960228 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5584ee379780 msgr2=0x5584ee376be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:35.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.843+0000 7efd7ffff640 1 --2- 172.21.15.16:0/3188960228 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5584ee379780 0x5584ee376be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:35.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.843+0000 7efd7ffff640 1 -- 172.21.15.16:0/3188960228 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5584ee0413c0 con 0x5584ee36a5d0 2024-09-12T02:03:35.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.844+0000 7efd7ffff640 1 --2- 172.21.15.16:0/3188960228 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5584ee36a5d0 0x5584ee3791f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7efd700187e0 tx=0x7efd700186e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=221d0fcebb7e253 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:35.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.844+0000 7efd66ffd640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-09-12T02:03:35.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.844+0000 7efd66ffd640 1 -- 172.21.15.16:0/3188960228 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd70015730 con 0x5584ee36a5d0 2024-09-12T02:03:35.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.844+0000 7efd90244740 1 -- 172.21.15.16:0/3188960228 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd7001d070 con 0x5584ee36a5d0 2024-09-12T02:03:35.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.844+0000 7efd66ffd640 1 -- 172.21.15.16:0/3188960228 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7efd700158d0 con 0x5584ee36a5d0 2024-09-12T02:03:35.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.844+0000 7efd66ffd640 1 -- 172.21.15.16:0/3188960228 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7efd70015ad0 con 0x5584ee36a5d0 2024-09-12T02:03:35.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.844+0000 7efd90244740 10 client.25426 fetch_fsmap learned FSMap version 25 2024-09-12T02:03:35.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.844+0000 7efd90244740 1 -- 172.21.15.16:0/3188960228 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5584ee387f10 con 0x5584ee36a5d0 2024-09-12T02:03:35.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.845+0000 7efd66ffd640 1 -- 172.21.15.16:0/3188960228 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd7001d070 con 0x5584ee36a5d0 2024-09-12T02:03:35.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.845+0000 7efd66ffd640 1 -- 172.21.15.16:0/3188960228 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7efd700229e0 con 0x5584ee36a5d0 2024-09-12T02:03:35.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.845+0000 7efd90244740 10 client.25426 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:03:35.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.845+0000 7efd90244740 10 client.25426 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:03:35.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.845+0000 7efd90244740 20 client.25426 populate_metadata read hostname 'smithi016' 2024-09-12T02:03:35.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.845+0000 7efd90244740 1 --2- 172.21.15.16:0/3188960228 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5584ee3892c0 0x5584ee38b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:35.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.845+0000 7efd90244740 4 client.25426 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:03:35.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.845+0000 7efd90244740 1 -- 172.21.15.16:0/3188960228 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5584ee2dd860 con 0x5584ee3892c0 2024-09-12T02:03:35.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.845+0000 7efd877fe640 1 --2- 172.21.15.16:0/3188960228 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5584ee3892c0 0x5584ee38b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:35.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.846+0000 7efd66ffd640 10 client.25426 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:03:35.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.846+0000 7efd877fe640 1 --2- 172.21.15.16:0/3188960228 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5584ee3892c0 0x5584ee38b6b0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c57c05b98ce13ea server_cookie=46c9d6823f1dd6c9 in_seq=0 out_seq=0 2024-09-12T02:03:35.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.846+0000 7efd66ffd640 1 -- 172.21.15.16:0/3188960228 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5584ee2dd860 con 0x5584ee3892c0 2024-09-12T02:03:35.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.846+0000 7efd66ffd640 10 client.25426 handle_command_reply: tid=0 2024-09-12T02:03:35.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.847+0000 7efd90244740 1 client.25426 shutdown 2024-09-12T02:03:35.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.847+0000 7efd90244740 1 -- 172.21.15.16:0/3188960228 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5584ee36a5d0 msgr2=0x5584ee3791f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:35.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.847+0000 7efd90244740 1 --2- 172.21.15.16:0/3188960228 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5584ee36a5d0 0x5584ee3791f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7efd700187e0 tx=0x7efd700186e0 comp rx=0 tx=0).stop 2024-09-12T02:03:35.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.847+0000 7efd90244740 1 -- 172.21.15.16:0/3188960228 shutdown_connections 2024-09-12T02:03:35.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.847+0000 7efd90244740 1 --2- 172.21.15.16:0/3188960228 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5584ee3892c0 0x5584ee38b6b0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:35.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.847+0000 7efd66ffd640 0 client.25426 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:03:35.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.847+0000 7efd90244740 1 --2- 172.21.15.16:0/3188960228 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5584ee379780 0x5584ee376be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:35.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.847+0000 7efd90244740 1 --2- 172.21.15.16:0/3188960228 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5584ee36bb40 0x5584ee3764a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:35.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.847+0000 7efd90244740 1 --2- 172.21.15.16:0/3188960228 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5584ee36a5d0 0x5584ee3791f0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:35.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.847+0000 7efd90244740 1 -- 172.21.15.16:0/3188960228 >> 172.21.15.16:0/3188960228 conn(0x5584ee333310 msgr2=0x5584ee36c960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:35.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.847+0000 7efd90244740 1 -- 172.21.15.16:0/3188960228 shutdown_connections 2024-09-12T02:03:35.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.847+0000 7efd90244740 1 -- 172.21.15.16:0/3188960228 wait complete. 2024-09-12T02:03:35.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.847+0000 7efd90244740 20 client.25426 trim_cache size 0 max 16384 2024-09-12T02:03:35.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.861+0000 7efd90244740 1 Processor -- start 2024-09-12T02:03:35.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.861+0000 7efd90244740 1 -- start start 2024-09-12T02:03:35.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.861+0000 7efd90244740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5584ee36a5d0 0x5584ee385840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:35.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.861+0000 7efd90244740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5584ee36bb40 0x5584ee38e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:35.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.861+0000 7efd90244740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5584ee379780 0x5584ee38ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:35.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.861+0000 7efd90244740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7efd881adb80 con 0x5584ee36a5d0 2024-09-12T02:03:35.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.861+0000 7efd90244740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7efd881ab450 con 0x5584ee36bb40 2024-09-12T02:03:35.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.861+0000 7efd90244740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7efd881a92e0 con 0x5584ee379780 2024-09-12T02:03:35.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.861+0000 7efd7ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5584ee36a5d0 0x5584ee385840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:35.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.861+0000 7efd7ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5584ee36a5d0 0x5584ee385840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51850/0 (socket says 172.21.15.16:51850) 2024-09-12T02:03:35.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.861+0000 7efd7ffff640 1 -- 172.21.15.16:0/2696915986 learned_addr learned my addr 172.21.15.16:0/2696915986 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:03:35.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.861+0000 7efd877fe640 1 --2- 172.21.15.16:0/2696915986 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5584ee36bb40 0x5584ee38e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:35.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.862+0000 7efd7ffff640 1 -- 172.21.15.16:0/2696915986 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5584ee379780 msgr2=0x5584ee38ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:35.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.862+0000 7efd7ffff640 1 --2- 172.21.15.16:0/2696915986 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5584ee379780 0x5584ee38ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:35.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.862+0000 7efd7ffff640 1 -- 172.21.15.16:0/2696915986 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5584ee36bb40 msgr2=0x5584ee38e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:35.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.862+0000 7efd7ffff640 1 --2- 172.21.15.16:0/2696915986 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5584ee36bb40 0x5584ee38e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:35.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.862+0000 7efd7ffff640 1 -- 172.21.15.16:0/2696915986 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5584ee367960 con 0x5584ee36a5d0 2024-09-12T02:03:35.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.862+0000 7efd877fe640 1 --2- 172.21.15.16:0/2696915986 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5584ee36bb40 0x5584ee38e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:03:35.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.862+0000 7efd7ffff640 1 --2- 172.21.15.16:0/2696915986 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5584ee36a5d0 0x5584ee385840 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7efd700187b0 tx=0x7efd70009510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fa9c916c8fcb0d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:35.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.862+0000 7efd7d7fa640 1 -- 172.21.15.16:0/2696915986 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd70006d60 con 0x5584ee36a5d0 2024-09-12T02:03:35.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.862+0000 7efd7d7fa640 1 -- 172.21.15.16:0/2696915986 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd7001d070 con 0x5584ee36a5d0 2024-09-12T02:03:35.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.862+0000 7efd7d7fa640 1 -- 172.21.15.16:0/2696915986 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd70018dc0 con 0x5584ee36a5d0 2024-09-12T02:03:35.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.863+0000 7efd90244740 1 -- 172.21.15.16:0/2696915986 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5584ee36a5d0 msgr2=0x5584ee385840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:35.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.863+0000 7efd90244740 1 --2- 172.21.15.16:0/2696915986 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5584ee36a5d0 0x5584ee385840 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7efd700187b0 tx=0x7efd70009510 comp rx=0 tx=0).stop 2024-09-12T02:03:35.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.863+0000 7efd90244740 1 -- 172.21.15.16:0/2696915986 shutdown_connections 2024-09-12T02:03:35.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.863+0000 7efd90244740 1 --2- 172.21.15.16:0/2696915986 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5584ee379780 0x5584ee38ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:35.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.863+0000 7efd90244740 1 --2- 172.21.15.16:0/2696915986 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5584ee36bb40 0x5584ee38e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:35.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.863+0000 7efd90244740 1 --2- 172.21.15.16:0/2696915986 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5584ee36a5d0 0x5584ee385840 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:35.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.863+0000 7efd90244740 1 -- 172.21.15.16:0/2696915986 >> 172.21.15.16:0/2696915986 conn(0x5584ee333310 msgr2=0x5584ee365d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:35.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.863+0000 7efd90244740 1 -- 172.21.15.16:0/2696915986 shutdown_connections 2024-09-12T02:03:35.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.863+0000 7efd90244740 1 -- 172.21.15.16:0/2696915986 wait complete. 2024-09-12T02:03:35.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.864+0000 7efd90244740 1 -- start start 2024-09-12T02:03:35.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.864+0000 7efd90244740 1 Processor -- start 2024-09-12T02:03:35.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.864+0000 7efd90244740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5584ee36a5d0 0x5584ee38ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:35.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.864+0000 7efd90244740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5584ee36bb40 0x5584ee38efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:35.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.864+0000 7efd90244740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5584ee379780 0x5584ee38c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:35.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.864+0000 7efd90244740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7efd881a9930 con 0x5584ee36a5d0 2024-09-12T02:03:35.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.864+0000 7efd90244740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7efd881aad80 con 0x5584ee379780 2024-09-12T02:03:35.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.864+0000 7efd90244740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7efd881a9fe0 con 0x5584ee36bb40 2024-09-12T02:03:35.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.864+0000 7efd7ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5584ee36a5d0 0x5584ee38ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:35.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.864+0000 7efd90244740 10 client.0 _ll_register_callbacks cb 0x7efd70009cd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:03:35.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.864+0000 7efd7ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5584ee36a5d0 0x5584ee38ea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51862/0 (socket says 172.21.15.16:51862) 2024-09-12T02:03:35.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.864+0000 7efd7ffff640 1 -- 172.21.15.16:0/478474781 learned_addr learned my addr 172.21.15.16:0/478474781 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:03:35.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.864+0000 7efd877fe640 1 --2- 172.21.15.16:0/478474781 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5584ee36bb40 0x5584ee38efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:35.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.865+0000 7efd7ffff640 1 --2- 172.21.15.16:0/478474781 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5584ee379780 0x5584ee38c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:35.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.865+0000 7efd7ffff640 1 -- 172.21.15.16:0/478474781 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5584ee36bb40 msgr2=0x5584ee38efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:35.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.865+0000 7efd7ffff640 1 --2- 172.21.15.16:0/478474781 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5584ee36bb40 0x5584ee38efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:35.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.865+0000 7efd7ffff640 1 -- 172.21.15.16:0/478474781 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5584ee379780 msgr2=0x5584ee38c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:35.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.865+0000 7efd7ffff640 1 --2- 172.21.15.16:0/478474781 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5584ee379780 0x5584ee38c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:35.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.865+0000 7efd7ffff640 1 -- 172.21.15.16:0/478474781 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5584ee38c830 con 0x5584ee36a5d0 2024-09-12T02:03:35.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.865+0000 7efd7ffff640 1 --2- 172.21.15.16:0/478474781 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5584ee36a5d0 0x5584ee38ea60 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7efd70000c00 tx=0x7efd70018cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee016f9c888b1f02 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:35.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.865+0000 7efd66ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:03:35.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.865+0000 7efd66ffd640 1 -- 172.21.15.16:0/478474781 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd70018450 con 0x5584ee36a5d0 2024-09-12T02:03:35.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.865+0000 7efd66ffd640 1 -- 172.21.15.16:0/478474781 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7efd70018da0 con 0x5584ee36a5d0 2024-09-12T02:03:35.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.865+0000 7efd90244740 1 -- 172.21.15.16:0/478474781 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd7001d070 con 0x5584ee36a5d0 2024-09-12T02:03:35.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.866+0000 7efd66ffd640 1 -- 172.21.15.16:0/478474781 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7efd70006d60 con 0x5584ee36a5d0 2024-09-12T02:03:35.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.866+0000 7efd90244740 10 client.15900 fetch_fsmap learned FSMap version 25 2024-09-12T02:03:35.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.866+0000 7efd90244740 1 -- 172.21.15.16:0/478474781 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5584ee3f3860 con 0x5584ee36a5d0 2024-09-12T02:03:35.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.866+0000 7efd66ffd640 1 -- 172.21.15.16:0/478474781 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd7001d070 con 0x5584ee36a5d0 2024-09-12T02:03:35.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.866+0000 7efd66ffd640 1 -- 172.21.15.16:0/478474781 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7efd70044030 con 0x5584ee36a5d0 2024-09-12T02:03:35.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.866+0000 7efd90244740 10 client.15900 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:03:35.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.866+0000 7efd90244740 10 client.15900 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:03:35.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.866+0000 7efd90244740 20 client.15900 populate_metadata read hostname 'smithi016' 2024-09-12T02:03:35.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.866+0000 7efd90244740 1 --2- 172.21.15.16:0/478474781 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5584ee3892c0 0x5584ee3f5f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:35.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.866+0000 7efd90244740 4 client.15900 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T02:03:35.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.866+0000 7efd90244740 1 -- 172.21.15.16:0/478474781 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5584ee2d7d70 con 0x5584ee3892c0 2024-09-12T02:03:35.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.867+0000 7efd877fe640 1 --2- 172.21.15.16:0/478474781 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5584ee3892c0 0x5584ee3f5f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:35.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.867+0000 7efd877fe640 1 --2- 172.21.15.16:0/478474781 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5584ee3892c0 0x5584ee3f5f60 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b33d008406e87733 server_cookie=3fd203bd6a9fd20a in_seq=0 out_seq=0 2024-09-12T02:03:35.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.867+0000 7efd66ffd640 10 client.15900 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:03:35.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.867+0000 7efd66ffd640 1 -- 172.21.15.16:0/478474781 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5584ee2d7d70 con 0x5584ee3892c0 2024-09-12T02:03:35.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.867+0000 7efd66ffd640 10 client.15900 handle_command_reply: tid=0 2024-09-12T02:03:35.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.867+0000 7efd90244740 1 client.15900 shutdown 2024-09-12T02:03:35.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.867+0000 7efd90244740 1 -- 172.21.15.16:0/478474781 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5584ee36a5d0 msgr2=0x5584ee38ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:35.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.867+0000 7efd90244740 1 --2- 172.21.15.16:0/478474781 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5584ee36a5d0 0x5584ee38ea60 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7efd70000c00 tx=0x7efd70018cc0 comp rx=0 tx=0).stop 2024-09-12T02:03:35.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.867+0000 7efd90244740 1 -- 172.21.15.16:0/478474781 shutdown_connections 2024-09-12T02:03:35.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.867+0000 7efd90244740 1 --2- 172.21.15.16:0/478474781 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5584ee3892c0 0x5584ee3f5f60 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:35.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.868+0000 7efd66ffd640 0 client.15900 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:03:35.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.868+0000 7efd90244740 1 --2- 172.21.15.16:0/478474781 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5584ee379780 0x5584ee38c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:35.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.868+0000 7efd90244740 1 --2- 172.21.15.16:0/478474781 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5584ee36bb40 0x5584ee38efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:35.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.868+0000 7efd90244740 1 --2- 172.21.15.16:0/478474781 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5584ee36a5d0 0x5584ee38ea60 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:35.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.868+0000 7efd90244740 1 -- 172.21.15.16:0/478474781 >> 172.21.15.16:0/478474781 conn(0x5584ee333310 msgr2=0x5584ee36c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:35.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.868+0000 7efd90244740 1 -- 172.21.15.16:0/478474781 shutdown_connections 2024-09-12T02:03:35.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.868+0000 7efd90244740 1 -- 172.21.15.16:0/478474781 wait complete. 2024-09-12T02:03:35.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.868+0000 7efd90244740 20 client.15900 trim_cache size 0 max 16384 2024-09-12T02:03:35.872 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T02:03:35.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.868+0000 7efd8de6b640 1 -- 172.21.15.16:0/3759511583 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7efd740a2c00 msgr2=0x7efd740a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:35.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.868+0000 7efd8de6b640 1 --2- 172.21.15.16:0/3759511583 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7efd740a2c00 0x7efd740a50c0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7efd80097e50 tx=0x7efd80086040 comp rx=0 tx=0).stop 2024-09-12T02:03:35.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.868+0000 7efd8de6b640 1 -- 172.21.15.16:0/3759511583 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd88106a80 msgr2=0x7efd8819cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:35.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.868+0000 7efd8de6b640 1 --2- 172.21.15.16:0/3759511583 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd88106a80 0x7efd8819cdf0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7efd8000ca30 tx=0x7efd8000ce30 comp rx=0 tx=0).stop 2024-09-12T02:03:35.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.869+0000 7efd8de6b640 1 -- 172.21.15.16:0/3759511583 shutdown_connections 2024-09-12T02:03:35.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.869+0000 7efd8de6b640 1 --2- 172.21.15.16:0/3759511583 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7efd740a2c00 0x7efd740a50c0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:35.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.869+0000 7efd8de6b640 1 --2- 172.21.15.16:0/3759511583 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd88106a80 0x7efd8819cdf0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:35.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.869+0000 7efd8de6b640 1 --2- 172.21.15.16:0/3759511583 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efd88101420 0x7efd88198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:35.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.869+0000 7efd8de6b640 1 --2- 172.21.15.16:0/3759511583 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd88100a80 0x7efd88198520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:35.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.869+0000 7efd8de6b640 1 -- 172.21.15.16:0/3759511583 >> 172.21.15.16:0/3759511583 conn(0x7efd880fc7a0 msgr2=0x7efd8806aad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:35.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.869+0000 7efd8de6b640 1 -- 172.21.15.16:0/3759511583 shutdown_connections 2024-09-12T02:03:35.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:35.869+0000 7efd8de6b640 1 -- 172.21.15.16:0/3759511583 wait complete. 2024-09-12T02:03:35.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:03:35 smithi184 ceph-mon[35593]: pgmap v385: 97 pgs: 97 active+clean; 1.1 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-09-12T02:03:35.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:03:35 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/3599329407' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:03:35.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:03:35 smithi033 ceph-mon[35332]: pgmap v385: 97 pgs: 97 active+clean; 1.1 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-09-12T02:03:35.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:03:35 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/3599329407' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:03:36.677 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:03:37.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.129+0000 7fc22effa640 1 -- 172.21.15.16:0/59637271 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc22810c170 msgr2=0x7fc22810e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:37.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.129+0000 7fc22effa640 1 --2- 172.21.15.16:0/59637271 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc22810c170 0x7fc22810e610 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fc220009ca0 tx=0x7fc22002a950 comp rx=0 tx=0).stop 2024-09-12T02:03:37.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.129+0000 7fc22effa640 1 -- 172.21.15.16:0/59637271 shutdown_connections 2024-09-12T02:03:37.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.129+0000 7fc22effa640 1 --2- 172.21.15.16:0/59637271 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc22810c170 0x7fc22810e610 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:37.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.129+0000 7fc22effa640 1 --2- 172.21.15.16:0/59637271 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc228103660 0x7fc22810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:37.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.129+0000 7fc22effa640 1 --2- 172.21.15.16:0/59637271 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc228102d40 0x7fc228103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:37.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.129+0000 7fc22effa640 1 -- 172.21.15.16:0/59637271 >> 172.21.15.16:0/59637271 conn(0x7fc2280fc7a0 msgr2=0x7fc2280febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:37.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.129+0000 7fc22effa640 1 -- 172.21.15.16:0/59637271 shutdown_connections 2024-09-12T02:03:37.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.129+0000 7fc22effa640 1 -- 172.21.15.16:0/59637271 wait complete. 2024-09-12T02:03:37.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.130+0000 7fc22effa640 1 Processor -- start 2024-09-12T02:03:37.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.130+0000 7fc22effa640 1 -- start start 2024-09-12T02:03:37.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.130+0000 7fc22effa640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc228102d40 0x7fc2281009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:37.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.130+0000 7fc22effa640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc228103660 0x7fc228100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:37.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.130+0000 7fc22effa640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc22810c170 0x7fc228101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:37.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.130+0000 7fc22effa640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc228111fb0 con 0x7fc228103660 2024-09-12T02:03:37.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.130+0000 7fc22effa640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc228111e30 con 0x7fc22810c170 2024-09-12T02:03:37.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.130+0000 7fc22effa640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc228112130 con 0x7fc228102d40 2024-09-12T02:03:37.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.130+0000 7fc227fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc228103660 0x7fc228100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:37.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.130+0000 7fc227fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc228103660 0x7fc228100ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51880/0 (socket says 172.21.15.16:51880) 2024-09-12T02:03:37.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.130+0000 7fc227fff640 1 -- 172.21.15.16:0/1793482958 learned_addr learned my addr 172.21.15.16:0/1793482958 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:03:37.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.130+0000 7fc22cd96640 1 --2- 172.21.15.16:0/1793482958 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc228102d40 0x7fc2281009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:37.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.130+0000 7fc22d597640 1 --2- 172.21.15.16:0/1793482958 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc22810c170 0x7fc228101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:37.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.130+0000 7fc227fff640 1 -- 172.21.15.16:0/1793482958 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc228102d40 msgr2=0x7fc2281009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:37.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.130+0000 7fc227fff640 1 --2- 172.21.15.16:0/1793482958 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc228102d40 0x7fc2281009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:37.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.130+0000 7fc227fff640 1 -- 172.21.15.16:0/1793482958 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc22810c170 msgr2=0x7fc228101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:37.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.130+0000 7fc227fff640 1 --2- 172.21.15.16:0/1793482958 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc22810c170 0x7fc228101420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:37.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.130+0000 7fc227fff640 1 -- 172.21.15.16:0/1793482958 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2281a6930 con 0x7fc228103660 2024-09-12T02:03:37.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.130+0000 7fc22cd96640 1 --2- 172.21.15.16:0/1793482958 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc228102d40 0x7fc2281009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:03:37.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.131+0000 7fc227fff640 1 --2- 172.21.15.16:0/1793482958 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc228103660 0x7fc228100ee0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fc21400b3f0 tx=0x7fc21400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:37.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.131+0000 7fc225ffb640 1 -- 172.21.15.16:0/1793482958 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc214015020 con 0x7fc228103660 2024-09-12T02:03:37.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.131+0000 7fc225ffb640 1 -- 172.21.15.16:0/1793482958 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc214011d10 con 0x7fc228103660 2024-09-12T02:03:37.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.131+0000 7fc22effa640 1 -- 172.21.15.16:0/1793482958 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2281a6c20 con 0x7fc228103660 2024-09-12T02:03:37.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.131+0000 7fc225ffb640 1 -- 172.21.15.16:0/1793482958 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc21400f2e0 con 0x7fc228103660 2024-09-12T02:03:37.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.131+0000 7fc22effa640 1 -- 172.21.15.16:0/1793482958 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2281a71b0 con 0x7fc228103660 2024-09-12T02:03:37.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.133+0000 7fc225ffb640 1 -- 172.21.15.16:0/1793482958 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fc214018020 con 0x7fc228103660 2024-09-12T02:03:37.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.134+0000 7fc2313d3740 1 Processor -- start 2024-09-12T02:03:37.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.134+0000 7fc2313d3740 1 -- start start 2024-09-12T02:03:37.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.134+0000 7fc225ffb640 1 --2- 172.21.15.16:0/1793482958 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc2180a2c00 0x7fc2180a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:37.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.134+0000 7fc2313d3740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fae15e3760 0x55fae15e3b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:37.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.134+0000 7fc2313d3740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fae15e4080 0x55fae15ee400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:37.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.134+0000 7fc2313d3740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fae15ee940 0x55fae15f0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:37.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.134+0000 7fc2313d3740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55fae15492a0 con 0x55fae15ee940 2024-09-12T02:03:37.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.134+0000 7fc2313d3740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55fae14f0b10 con 0x55fae15e3760 2024-09-12T02:03:37.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.134+0000 7fc2313d3740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55fae1545260 con 0x55fae15e4080 2024-09-12T02:03:37.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.134+0000 7fc225ffb640 1 -- 172.21.15.16:0/1793482958 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fc2140ce7c0 con 0x7fc228103660 2024-09-12T02:03:37.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.134+0000 7fc22d597640 1 --2- 172.21.15.16:0/1793482958 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc2180a2c00 0x7fc2180a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:37.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.134+0000 7fc22d597640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fae15ee940 0x55fae15f0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:37.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.134+0000 7fc22d597640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fae15ee940 0x55fae15f0d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51892/0 (socket says 172.21.15.16:51892) 2024-09-12T02:03:37.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.134+0000 7fc22d597640 1 -- 172.21.15.16:0/51956674 learned_addr learned my addr 172.21.15.16:0/51956674 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:03:37.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.134+0000 7fc22cd96640 1 --2- 172.21.15.16:0/51956674 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fae15e3760 0x55fae15e3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:37.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.134+0000 7fc227fff640 1 --2- 172.21.15.16:0/51956674 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fae15e4080 0x55fae15ee400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:37.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.135+0000 7fc22d597640 1 -- 172.21.15.16:0/51956674 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fae15e4080 msgr2=0x55fae15ee400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:37.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.135+0000 7fc22d597640 1 --2- 172.21.15.16:0/51956674 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fae15e4080 0x55fae15ee400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:37.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.135+0000 7fc22d597640 1 -- 172.21.15.16:0/51956674 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fae15e3760 msgr2=0x55fae15e3b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:37.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.135+0000 7fc22d597640 1 --2- 172.21.15.16:0/51956674 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fae15e3760 0x55fae15e3b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:37.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.135+0000 7fc22d597640 1 -- 172.21.15.16:0/51956674 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fae12bab90 con 0x55fae15ee940 2024-09-12T02:03:37.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.135+0000 7fc22cd96640 1 --2- 172.21.15.16:0/51956674 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fae15e3760 0x55fae15e3b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:03:37.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.135+0000 7fc22d597640 1 --2- 172.21.15.16:0/51956674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fae15ee940 0x55fae15f0d30 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fc220009a60 tx=0x7fc22002f4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f732a4005eaa0a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:37.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.135+0000 7fc22d597640 1 --2- 172.21.15.16:0/1793482958 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc2180a2c00 0x7fc2180a50c0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fc22002f540 tx=0x7fc220043000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:37.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.135+0000 7fc212ffd640 1 -- 172.21.15.16:0/51956674 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc22004d070 con 0x55fae15ee940 2024-09-12T02:03:37.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.135+0000 7fc212ffd640 1 -- 172.21.15.16:0/51956674 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc220003e30 con 0x55fae15ee940 2024-09-12T02:03:37.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.135+0000 7fc212ffd640 1 -- 172.21.15.16:0/51956674 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc220034040 con 0x55fae15ee940 2024-09-12T02:03:37.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.136+0000 7fc2313d3740 1 -- 172.21.15.16:0/51956674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fae15ee940 msgr2=0x55fae15f0d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:37.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.136+0000 7fc2313d3740 1 --2- 172.21.15.16:0/51956674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fae15ee940 0x55fae15f0d30 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fc220009a60 tx=0x7fc22002f4d0 comp rx=0 tx=0).stop 2024-09-12T02:03:37.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.136+0000 7fc2313d3740 1 -- 172.21.15.16:0/51956674 shutdown_connections 2024-09-12T02:03:37.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.136+0000 7fc2313d3740 1 --2- 172.21.15.16:0/51956674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fae15ee940 0x55fae15f0d30 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:37.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.136+0000 7fc2313d3740 1 --2- 172.21.15.16:0/51956674 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fae15e4080 0x55fae15ee400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:37.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.136+0000 7fc2313d3740 1 --2- 172.21.15.16:0/51956674 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fae15e3760 0x55fae15e3b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:37.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.136+0000 7fc2313d3740 1 -- 172.21.15.16:0/51956674 >> 172.21.15.16:0/51956674 conn(0x55fae15dfcf0 msgr2=0x55fae15e00d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:37.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.136+0000 7fc2313d3740 1 -- 172.21.15.16:0/51956674 shutdown_connections 2024-09-12T02:03:37.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.136+0000 7fc2313d3740 1 -- 172.21.15.16:0/51956674 wait complete. 2024-09-12T02:03:37.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.136+0000 7fc2313d3740 1 -- start start 2024-09-12T02:03:37.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.136+0000 7fc2313d3740 1 Processor -- start 2024-09-12T02:03:37.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.137+0000 7fc2313d3740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fae15e3760 0x55fae15eb8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:37.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.137+0000 7fc2313d3740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fae15e4080 0x55fae15ebe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:37.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.137+0000 7fc2313d3740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fae15ee940 0x55fae15ec340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:37.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.137+0000 7fc2313d3740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55fae15f1510 con 0x55fae15e4080 2024-09-12T02:03:37.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.137+0000 7fc2313d3740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55fae14d16e0 con 0x55fae15e3760 2024-09-12T02:03:37.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.137+0000 7fc2313d3740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55fae15f1690 con 0x55fae15ee940 2024-09-12T02:03:37.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.137+0000 7fc2313d3740 10 client.0 _ll_register_callbacks cb 0x55fae14f5a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:03:37.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.137+0000 7fc227fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fae15e4080 0x55fae15ebe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:37.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.137+0000 7fc227fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fae15e4080 0x55fae15ebe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51894/0 (socket says 172.21.15.16:51894) 2024-09-12T02:03:37.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.137+0000 7fc227fff640 1 -- 172.21.15.16:0/1591428316 learned_addr learned my addr 172.21.15.16:0/1591428316 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:03:37.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.137+0000 7fc22d597640 1 --2- 172.21.15.16:0/1591428316 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fae15ee940 0x55fae15ec340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:37.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.137+0000 7fc22cd96640 1 --2- 172.21.15.16:0/1591428316 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fae15e3760 0x55fae15eb8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:37.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.137+0000 7fc227fff640 1 -- 172.21.15.16:0/1591428316 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fae15ee940 msgr2=0x55fae15ec340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:37.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.137+0000 7fc227fff640 1 --2- 172.21.15.16:0/1591428316 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fae15ee940 0x55fae15ec340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:37.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.137+0000 7fc227fff640 1 -- 172.21.15.16:0/1591428316 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fae15e3760 msgr2=0x55fae15eb8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:37.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.137+0000 7fc227fff640 1 --2- 172.21.15.16:0/1591428316 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fae15e3760 0x55fae15eb8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:37.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.137+0000 7fc227fff640 1 -- 172.21.15.16:0/1591428316 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fae15fab10 con 0x55fae15e4080 2024-09-12T02:03:37.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.138+0000 7fc227fff640 1 --2- 172.21.15.16:0/1591428316 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fae15e4080 0x55fae15ebe00 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fc214009030 tx=0x7fc21409aae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73d8f22500a0acc5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:37.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.138+0000 7fc207fff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:03:37.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.138+0000 7fc207fff640 1 -- 172.21.15.16:0/1591428316 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2140b4060 con 0x55fae15e4080 2024-09-12T02:03:37.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.138+0000 7fc207fff640 1 -- 172.21.15.16:0/1591428316 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fc2140bb0b0 con 0x55fae15e4080 2024-09-12T02:03:37.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.138+0000 7fc2313d3740 1 -- 172.21.15.16:0/1591428316 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc220003e30 con 0x55fae15e4080 2024-09-12T02:03:37.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.138+0000 7fc207fff640 1 -- 172.21.15.16:0/1591428316 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc214010eb0 con 0x55fae15e4080 2024-09-12T02:03:37.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.138+0000 7fc2313d3740 10 client.15912 fetch_fsmap learned FSMap version 25 2024-09-12T02:03:37.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.138+0000 7fc2313d3740 1 -- 172.21.15.16:0/1591428316 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55fae15fd4a0 con 0x55fae15e4080 2024-09-12T02:03:37.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.138+0000 7fc207fff640 1 -- 172.21.15.16:0/1591428316 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc220003e30 con 0x55fae15e4080 2024-09-12T02:03:37.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.138+0000 7fc207fff640 1 -- 172.21.15.16:0/1591428316 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fc21404d030 con 0x55fae15e4080 2024-09-12T02:03:37.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.138+0000 7fc2313d3740 10 client.15912 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:03:37.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.138+0000 7fc2313d3740 10 client.15912 resolve_mds: validated gid 24482 aka daemon mds.b 2024-09-12T02:03:37.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.138+0000 7fc2313d3740 20 client.15912 populate_metadata read hostname 'smithi016' 2024-09-12T02:03:37.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.138+0000 7fc2313d3740 1 --2- 172.21.15.16:0/1591428316 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55fae15fe8b0 0x55fae1600ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:37.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.138+0000 7fc2313d3740 4 client.15912 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:03:37.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.138+0000 7fc2313d3740 1 -- 172.21.15.16:0/1591428316 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fae15569d0 con 0x55fae15fe8b0 2024-09-12T02:03:37.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.139+0000 7fc22cd96640 1 --2- 172.21.15.16:0/1591428316 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55fae15fe8b0 0x55fae1600ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:37.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.139+0000 7fc22cd96640 1 --2- 172.21.15.16:0/1591428316 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55fae15fe8b0 0x55fae1600ca0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80bc68161fbba7f4 server_cookie=e3fff14ee07aa832 in_seq=0 out_seq=0 2024-09-12T02:03:37.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.139+0000 7fc207fff640 10 client.15912 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:03:37.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.140+0000 7fc207fff640 1 -- 172.21.15.16:0/1591428316 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fae15569d0 con 0x55fae15fe8b0 2024-09-12T02:03:37.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.140+0000 7fc207fff640 10 client.15912 handle_command_reply: tid=0 2024-09-12T02:03:37.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.140+0000 7fc2313d3740 1 client.15912 shutdown 2024-09-12T02:03:37.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.140+0000 7fc2313d3740 1 -- 172.21.15.16:0/1591428316 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fae15e4080 msgr2=0x55fae15ebe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:37.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.140+0000 7fc2313d3740 1 --2- 172.21.15.16:0/1591428316 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fae15e4080 0x55fae15ebe00 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fc214009030 tx=0x7fc21409aae0 comp rx=0 tx=0).stop 2024-09-12T02:03:37.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.140+0000 7fc2313d3740 1 -- 172.21.15.16:0/1591428316 shutdown_connections 2024-09-12T02:03:37.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.140+0000 7fc2313d3740 1 --2- 172.21.15.16:0/1591428316 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55fae15fe8b0 0x55fae1600ca0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:37.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.140+0000 7fc2313d3740 1 --2- 172.21.15.16:0/1591428316 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fae15ee940 0x55fae15ec340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:37.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.140+0000 7fc2313d3740 1 --2- 172.21.15.16:0/1591428316 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fae15e4080 0x55fae15ebe00 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:37.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.140+0000 7fc2313d3740 1 --2- 172.21.15.16:0/1591428316 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fae15e3760 0x55fae15eb8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:37.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.140+0000 7fc207fff640 0 client.15912 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:03:37.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.140+0000 7fc2313d3740 1 -- 172.21.15.16:0/1591428316 >> 172.21.15.16:0/1591428316 conn(0x55fae15dfcf0 msgr2=0x55fae15e5a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:37.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.140+0000 7fc2313d3740 1 -- 172.21.15.16:0/1591428316 shutdown_connections 2024-09-12T02:03:37.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.140+0000 7fc2313d3740 1 -- 172.21.15.16:0/1591428316 wait complete. 2024-09-12T02:03:37.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.140+0000 7fc2313d3740 20 client.15912 trim_cache size 0 max 16384 2024-09-12T02:03:37.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.157+0000 7fc2313d3740 1 Processor -- start 2024-09-12T02:03:37.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.157+0000 7fc2313d3740 1 -- start start 2024-09-12T02:03:37.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.157+0000 7fc2313d3740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fae15e3760 0x55fae15ea800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:37.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.157+0000 7fc2313d3740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fae15e4080 0x55fae1606530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:37.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.157+0000 7fc2313d3740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fae15ee940 0x55fae1606a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:37.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.157+0000 7fc2313d3740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc2281b1eb0 con 0x55fae15ee940 2024-09-12T02:03:37.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.157+0000 7fc2313d3740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc2281af780 con 0x55fae15e3760 2024-09-12T02:03:37.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.157+0000 7fc2313d3740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc2281ad5d0 con 0x55fae15e4080 2024-09-12T02:03:37.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.157+0000 7fc22d597640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fae15ee940 0x55fae1606a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:37.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.158+0000 7fc22d597640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fae15ee940 0x55fae1606a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51898/0 (socket says 172.21.15.16:51898) 2024-09-12T02:03:37.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.158+0000 7fc22d597640 1 -- 172.21.15.16:0/2469220968 learned_addr learned my addr 172.21.15.16:0/2469220968 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:03:37.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.158+0000 7fc22cd96640 1 --2- 172.21.15.16:0/2469220968 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fae15e3760 0x55fae15ea800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:37.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.158+0000 7fc227fff640 1 --2- 172.21.15.16:0/2469220968 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fae15e4080 0x55fae1606530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:37.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.158+0000 7fc22d597640 1 -- 172.21.15.16:0/2469220968 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fae15e4080 msgr2=0x55fae1606530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:37.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.158+0000 7fc22d597640 1 --2- 172.21.15.16:0/2469220968 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fae15e4080 0x55fae1606530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:37.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.158+0000 7fc22d597640 1 -- 172.21.15.16:0/2469220968 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fae15e3760 msgr2=0x55fae15ea800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:37.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.158+0000 7fc22d597640 1 --2- 172.21.15.16:0/2469220968 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fae15e3760 0x55fae15ea800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:37.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.158+0000 7fc22d597640 1 -- 172.21.15.16:0/2469220968 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fae1601010 con 0x55fae15ee940 2024-09-12T02:03:37.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.158+0000 7fc22d597640 1 --2- 172.21.15.16:0/2469220968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fae15ee940 0x55fae1606a70 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fc220005dc0 tx=0x7fc22002fad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e4a833cc2b33be4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:37.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.158+0000 7fc2127fc640 1 -- 172.21.15.16:0/2469220968 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc220056020 con 0x55fae15ee940 2024-09-12T02:03:37.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.158+0000 7fc2127fc640 1 -- 172.21.15.16:0/2469220968 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc220004510 con 0x55fae15ee940 2024-09-12T02:03:37.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.158+0000 7fc2127fc640 1 -- 172.21.15.16:0/2469220968 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc220034070 con 0x55fae15ee940 2024-09-12T02:03:37.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.159+0000 7fc2313d3740 1 -- 172.21.15.16:0/2469220968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fae15ee940 msgr2=0x55fae1606a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:37.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.159+0000 7fc2313d3740 1 --2- 172.21.15.16:0/2469220968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fae15ee940 0x55fae1606a70 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fc220005dc0 tx=0x7fc22002fad0 comp rx=0 tx=0).stop 2024-09-12T02:03:37.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.159+0000 7fc2313d3740 1 -- 172.21.15.16:0/2469220968 shutdown_connections 2024-09-12T02:03:37.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.159+0000 7fc2313d3740 1 --2- 172.21.15.16:0/2469220968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fae15ee940 0x55fae1606a70 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:37.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.159+0000 7fc2313d3740 1 --2- 172.21.15.16:0/2469220968 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fae15e4080 0x55fae1606530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:37.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.159+0000 7fc2313d3740 1 --2- 172.21.15.16:0/2469220968 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fae15e3760 0x55fae15ea800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:37.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.159+0000 7fc2313d3740 1 -- 172.21.15.16:0/2469220968 >> 172.21.15.16:0/2469220968 conn(0x55fae15dfcf0 msgr2=0x55fae15ed940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:37.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.159+0000 7fc2313d3740 1 -- 172.21.15.16:0/2469220968 shutdown_connections 2024-09-12T02:03:37.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.159+0000 7fc2313d3740 1 -- 172.21.15.16:0/2469220968 wait complete. 2024-09-12T02:03:37.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.160+0000 7fc2313d3740 1 -- start start 2024-09-12T02:03:37.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.160+0000 7fc2313d3740 1 Processor -- start 2024-09-12T02:03:37.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.160+0000 7fc2313d3740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fae15e3760 0x55fae15f6df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:37.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.160+0000 7fc2313d3740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fae15e4080 0x55fae15f38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:37.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.160+0000 7fc2313d3740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fae15ee940 0x55fae15f3e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:37.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.160+0000 7fc2313d3740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc2281adc40 con 0x55fae15e3760 2024-09-12T02:03:37.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.160+0000 7fc2313d3740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc2281af0b0 con 0x55fae15ee940 2024-09-12T02:03:37.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.160+0000 7fc2313d3740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc2281ae310 con 0x55fae15e4080 2024-09-12T02:03:37.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.161+0000 7fc2313d3740 10 client.0 _ll_register_callbacks cb 0x7fc21409ac20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:03:37.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.161+0000 7fc22cd96640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fae15e3760 0x55fae15f6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:37.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.161+0000 7fc227fff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fae15e4080 0x55fae15f38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:37.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.161+0000 7fc22cd96640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fae15e3760 0x55fae15f6df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51910/0 (socket says 172.21.15.16:51910) 2024-09-12T02:03:37.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.161+0000 7fc22cd96640 1 -- 172.21.15.16:0/1637009604 learned_addr learned my addr 172.21.15.16:0/1637009604 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:03:37.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.161+0000 7fc22d597640 1 --2- 172.21.15.16:0/1637009604 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fae15ee940 0x55fae15f3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:37.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.161+0000 7fc22cd96640 1 -- 172.21.15.16:0/1637009604 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fae15e4080 msgr2=0x55fae15f38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:37.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.161+0000 7fc22cd96640 1 --2- 172.21.15.16:0/1637009604 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fae15e4080 0x55fae15f38e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:37.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.161+0000 7fc22cd96640 1 -- 172.21.15.16:0/1637009604 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fae15ee940 msgr2=0x55fae15f3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:37.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.161+0000 7fc22cd96640 1 --2- 172.21.15.16:0/1637009604 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fae15ee940 0x55fae15f3e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:37.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.161+0000 7fc22cd96640 1 -- 172.21.15.16:0/1637009604 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fae15f43e0 con 0x55fae15e3760 2024-09-12T02:03:37.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.161+0000 7fc22cd96640 1 --2- 172.21.15.16:0/1637009604 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fae15e3760 0x55fae15f6df0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x55fae16014d0 tx=0x7fc21c010af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1aa0268fa6073297 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:37.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.161+0000 7fc207fff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:03:37.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.161+0000 7fc207fff640 1 -- 172.21.15.16:0/1637009604 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc21c01c8f0 con 0x55fae15e3760 2024-09-12T02:03:37.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.162+0000 7fc207fff640 1 -- 172.21.15.16:0/1637009604 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fc21c01ca90 con 0x55fae15e3760 2024-09-12T02:03:37.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.162+0000 7fc2313d3740 1 -- 172.21.15.16:0/1637009604 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc220004510 con 0x55fae15e3760 2024-09-12T02:03:37.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.162+0000 7fc207fff640 1 -- 172.21.15.16:0/1637009604 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc21c01cc90 con 0x55fae15e3760 2024-09-12T02:03:37.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.162+0000 7fc2313d3740 10 client.15918 fetch_fsmap learned FSMap version 25 2024-09-12T02:03:37.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.162+0000 7fc2313d3740 1 -- 172.21.15.16:0/1637009604 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55fae166a2e0 con 0x55fae15e3760 2024-09-12T02:03:37.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.162+0000 7fc207fff640 1 -- 172.21.15.16:0/1637009604 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc220004510 con 0x55fae15e3760 2024-09-12T02:03:37.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.162+0000 7fc207fff640 1 -- 172.21.15.16:0/1637009604 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fc21c024910 con 0x55fae15e3760 2024-09-12T02:03:37.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.162+0000 7fc2313d3740 10 client.15918 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:03:37.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.162+0000 7fc2313d3740 10 client.15918 resolve_mds: validated gid 24482 aka daemon mds.b 2024-09-12T02:03:37.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.162+0000 7fc2313d3740 20 client.15918 populate_metadata read hostname 'smithi016' 2024-09-12T02:03:37.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.162+0000 7fc2313d3740 1 --2- 172.21.15.16:0/1637009604 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55fae166aaa0 0x55fae166cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:37.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.162+0000 7fc2313d3740 4 client.15918 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 133.4, "await_for": 45.0}] 2024-09-12T02:03:37.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.162+0000 7fc2313d3740 1 -- 172.21.15.16:0/1637009604 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 133.4, "await_for": 45.0}) -- 0x55fae1550d20 con 0x55fae166aaa0 2024-09-12T02:03:37.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.163+0000 7fc227fff640 1 --2- 172.21.15.16:0/1637009604 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55fae166aaa0 0x55fae166cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:37.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.163+0000 7fc207fff640 10 client.15918 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:03:37.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.163+0000 7fc227fff640 1 --2- 172.21.15.16:0/1637009604 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55fae166aaa0 0x55fae166cec0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2be34901fe12617 server_cookie=912e1c9b8ffa37e9 in_seq=0 out_seq=0 2024-09-12T02:03:37.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.179+0000 7fc207fff640 1 -- 172.21.15.16:0/1637009604 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+588 (crc 0 0 0) 0x55fae1550d20 con 0x55fae166aaa0 2024-09-12T02:03:37.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.179+0000 7fc207fff640 10 client.15918 handle_command_reply: tid=0 2024-09-12T02:03:37.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.179+0000 7fc2313d3740 1 client.15918 shutdown 2024-09-12T02:03:37.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.179+0000 7fc2313d3740 1 -- 172.21.15.16:0/1637009604 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fae15e3760 msgr2=0x55fae15f6df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:37.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.179+0000 7fc2313d3740 1 --2- 172.21.15.16:0/1637009604 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fae15e3760 0x55fae15f6df0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x55fae16014d0 tx=0x7fc21c010af0 comp rx=0 tx=0).stop 2024-09-12T02:03:37.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.179+0000 7fc2313d3740 1 -- 172.21.15.16:0/1637009604 shutdown_connections 2024-09-12T02:03:37.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.179+0000 7fc2313d3740 1 --2- 172.21.15.16:0/1637009604 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55fae166aaa0 0x55fae166cec0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:37.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.179+0000 7fc2313d3740 1 --2- 172.21.15.16:0/1637009604 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fae15ee940 0x55fae15f3e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:37.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.179+0000 7fc2313d3740 1 --2- 172.21.15.16:0/1637009604 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fae15e4080 0x55fae15f38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:37.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.179+0000 7fc2313d3740 1 --2- 172.21.15.16:0/1637009604 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fae15e3760 0x55fae15f6df0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:37.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.179+0000 7fc2313d3740 1 -- 172.21.15.16:0/1637009604 >> 172.21.15.16:0/1637009604 conn(0x55fae15dfcf0 msgr2=0x55fae15eb880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:37.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.179+0000 7fc207fff640 0 client.15918 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:03:37.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.179+0000 7fc2313d3740 1 -- 172.21.15.16:0/1637009604 shutdown_connections 2024-09-12T02:03:37.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.179+0000 7fc2313d3740 1 -- 172.21.15.16:0/1637009604 wait complete. 2024-09-12T02:03:37.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.179+0000 7fc2313d3740 20 client.15918 trim_cache size 0 max 16384 2024-09-12T02:03:37.181 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T02:03:37.181 INFO:teuthology.orchestra.run.smithi016.stdout: "epoch": 25, 2024-09-12T02:03:37.181 INFO:teuthology.orchestra.run.smithi016.stdout: "leader": 24482, 2024-09-12T02:03:37.181 INFO:teuthology.orchestra.run.smithi016.stdout: "set_version": 6, 2024-09-12T02:03:37.181 INFO:teuthology.orchestra.run.smithi016.stdout: "sets": { 2024-09-12T02:03:37.181 INFO:teuthology.orchestra.run.smithi016.stdout: "f262487": { 2024-09-12T02:03:37.181 INFO:teuthology.orchestra.run.smithi016.stdout: "version": 6, 2024-09-12T02:03:37.181 INFO:teuthology.orchestra.run.smithi016.stdout: "age_ref": 0.0, 2024-09-12T02:03:37.182 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-12T02:03:37.182 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "QUIESCED", 2024-09-12T02:03:37.182 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-12T02:03:37.182 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:03:37.182 INFO:teuthology.orchestra.run.smithi016.stdout: "timeout": 90.0, 2024-09-12T02:03:37.182 INFO:teuthology.orchestra.run.smithi016.stdout: "expiration": 133.4, 2024-09-12T02:03:37.182 INFO:teuthology.orchestra.run.smithi016.stdout: "members": { 2024-09-12T02:03:37.182 INFO:teuthology.orchestra.run.smithi016.stdout: "file:/": { 2024-09-12T02:03:37.182 INFO:teuthology.orchestra.run.smithi016.stdout: "excluded": false, 2024-09-12T02:03:37.182 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-12T02:03:37.182 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "QUIESCED", 2024-09-12T02:03:37.182 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-12T02:03:37.182 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:03:37.182 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:03:37.183 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:03:37.183 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:03:37.183 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:03:37.183 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T02:03:37.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.180+0000 7fc22effa640 1 -- 172.21.15.16:0/1793482958 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc2180a2c00 msgr2=0x7fc2180a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:37.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.180+0000 7fc22effa640 1 --2- 172.21.15.16:0/1793482958 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc2180a2c00 0x7fc2180a50c0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fc22002f540 tx=0x7fc220043000 comp rx=0 tx=0).stop 2024-09-12T02:03:37.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.180+0000 7fc22effa640 1 -- 172.21.15.16:0/1793482958 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc228103660 msgr2=0x7fc228100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:37.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.180+0000 7fc22effa640 1 --2- 172.21.15.16:0/1793482958 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc228103660 0x7fc228100ee0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fc21400b3f0 tx=0x7fc21400b8b0 comp rx=0 tx=0).stop 2024-09-12T02:03:37.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.180+0000 7fc22effa640 1 -- 172.21.15.16:0/1793482958 shutdown_connections 2024-09-12T02:03:37.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.180+0000 7fc22effa640 1 --2- 172.21.15.16:0/1793482958 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc2180a2c00 0x7fc2180a50c0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:37.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.180+0000 7fc22effa640 1 --2- 172.21.15.16:0/1793482958 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc22810c170 0x7fc228101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:37.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.180+0000 7fc22effa640 1 --2- 172.21.15.16:0/1793482958 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc228103660 0x7fc228100ee0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:37.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.180+0000 7fc22effa640 1 --2- 172.21.15.16:0/1793482958 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc228102d40 0x7fc2281009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:37.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.180+0000 7fc22effa640 1 -- 172.21.15.16:0/1793482958 >> 172.21.15.16:0/1793482958 conn(0x7fc2280fc7a0 msgr2=0x7fc2280fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:37.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.180+0000 7fc22effa640 1 -- 172.21.15.16:0/1793482958 shutdown_connections 2024-09-12T02:03:37.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:37.180+0000 7fc22effa640 1 -- 172.21.15.16:0/1793482958 wait complete. 2024-09-12T02:03:37.638 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-12T02:03:37.639 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-12T02:03:37.767 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:03:37 smithi016 ceph-mon[30947]: pgmap v386: 97 pgs: 97 active+clean; 1.1 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-09-12T02:03:37.797 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'f262487', quiesce took 4.4 seconds. Will release after: 9.0 2024-09-12T02:03:37.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:03:37 smithi184 ceph-mon[35593]: pgmap v386: 97 pgs: 97 active+clean; 1.1 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-09-12T02:03:37.911 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:03:37.918 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:03:37 smithi033 ceph-mon[35332]: pgmap v386: 97 pgs: 97 active+clean; 1.1 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-09-12T02:03:38.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.365+0000 7fd4f5181640 1 -- 172.21.15.16:0/3438525508 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4f0101440 msgr2=0x7fd4f010df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:38.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.365+0000 7fd4f5181640 1 --2- 172.21.15.16:0/3438525508 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4f0101440 0x7fd4f010df30 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fd4e4009900 tx=0x7fd4e402a410 comp rx=0 tx=0).stop 2024-09-12T02:03:38.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.366+0000 7fd4f5181640 1 -- 172.21.15.16:0/3438525508 shutdown_connections 2024-09-12T02:03:38.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.366+0000 7fd4f5181640 1 --2- 172.21.15.16:0/3438525508 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd4f010e500 0x7fd4f0110910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:38.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.366+0000 7fd4f5181640 1 --2- 172.21.15.16:0/3438525508 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4f0101440 0x7fd4f010df30 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:38.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.366+0000 7fd4f5181640 1 --2- 172.21.15.16:0/3438525508 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4f0100b20 0x7fd4f0100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:38.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.366+0000 7fd4f5181640 1 -- 172.21.15.16:0/3438525508 >> 172.21.15.16:0/3438525508 conn(0x7fd4f00fc7d0 msgr2=0x7fd4f00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:38.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.366+0000 7fd4f5181640 1 -- 172.21.15.16:0/3438525508 shutdown_connections 2024-09-12T02:03:38.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.366+0000 7fd4f5181640 1 -- 172.21.15.16:0/3438525508 wait complete. 2024-09-12T02:03:38.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.367+0000 7fd4f5181640 1 Processor -- start 2024-09-12T02:03:38.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.367+0000 7fd4f5181640 1 -- start start 2024-09-12T02:03:38.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.367+0000 7fd4f5181640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4f0100b20 0x7fd4f01a0540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:38.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.367+0000 7fd4f5181640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4f0101440 0x7fd4f01a0a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:38.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.367+0000 7fd4f5181640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd4f010e500 0x7fd4f019a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:38.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.367+0000 7fd4f5181640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd4f01142d0 con 0x7fd4f0100b20 2024-09-12T02:03:38.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.367+0000 7fd4f5181640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd4f0114150 con 0x7fd4f0101440 2024-09-12T02:03:38.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.367+0000 7fd4f5181640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd4f0114450 con 0x7fd4f010e500 2024-09-12T02:03:38.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.367+0000 7fd4eed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4f0100b20 0x7fd4f01a0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:38.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.367+0000 7fd4eed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4f0100b20 0x7fd4f01a0540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51940/0 (socket says 172.21.15.16:51940) 2024-09-12T02:03:38.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.367+0000 7fd4eed9d640 1 -- 172.21.15.16:0/699562162 learned_addr learned my addr 172.21.15.16:0/699562162 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:03:38.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.367+0000 7fd4ee59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4f0101440 0x7fd4f01a0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:38.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.368+0000 7fd4ef59e640 1 --2- 172.21.15.16:0/699562162 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd4f010e500 0x7fd4f019a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:38.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.368+0000 7fd4eed9d640 1 -- 172.21.15.16:0/699562162 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd4f010e500 msgr2=0x7fd4f019a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:38.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.368+0000 7fd4eed9d640 1 --2- 172.21.15.16:0/699562162 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd4f010e500 0x7fd4f019a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:38.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.368+0000 7fd4eed9d640 1 -- 172.21.15.16:0/699562162 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4f0101440 msgr2=0x7fd4f01a0a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:38.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.368+0000 7fd4eed9d640 1 --2- 172.21.15.16:0/699562162 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4f0101440 0x7fd4f01a0a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:38.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.368+0000 7fd4eed9d640 1 -- 172.21.15.16:0/699562162 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4f019aed0 con 0x7fd4f0100b20 2024-09-12T02:03:38.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.368+0000 7fd4ee59c640 1 --2- 172.21.15.16:0/699562162 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4f0101440 0x7fd4f01a0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:03:38.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.368+0000 7fd4eed9d640 1 --2- 172.21.15.16:0/699562162 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4f0100b20 0x7fd4f01a0540 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fd4e000d950 tx=0x7fd4e000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:38.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.368+0000 7fd4d7fff640 1 -- 172.21.15.16:0/699562162 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4e0017070 con 0x7fd4f0100b20 2024-09-12T02:03:38.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.368+0000 7fd4f5181640 1 -- 172.21.15.16:0/699562162 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4f019b1c0 con 0x7fd4f0100b20 2024-09-12T02:03:38.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.368+0000 7fd4d7fff640 1 -- 172.21.15.16:0/699562162 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd4e0009c30 con 0x7fd4f0100b20 2024-09-12T02:03:38.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.368+0000 7fd4d7fff640 1 -- 172.21.15.16:0/699562162 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4e000f6b0 con 0x7fd4f0100b20 2024-09-12T02:03:38.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.368+0000 7fd4f5181640 1 -- 172.21.15.16:0/699562162 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4f01a7530 con 0x7fd4f0100b20 2024-09-12T02:03:38.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.371+0000 7fd4f755a740 1 Processor -- start 2024-09-12T02:03:38.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.371+0000 7fd4f755a740 1 -- start start 2024-09-12T02:03:38.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.371+0000 7fd4f755a740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e72f8fbb60 0x55e72f8fbf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:38.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.371+0000 7fd4f755a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e72f8fc480 0x55e72f906800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:38.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.371+0000 7fd4f755a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e72f906d40 0x55e72f909130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:38.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.371+0000 7fd4f755a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e72f861280 con 0x55e72f8fc480 2024-09-12T02:03:38.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.371+0000 7fd4f755a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55e72f808b10 con 0x55e72f906d40 2024-09-12T02:03:38.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.371+0000 7fd4f755a740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55e72f85d240 con 0x55e72f8fbb60 2024-09-12T02:03:38.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.372+0000 7fd4ef59e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e72f8fbb60 0x55e72f8fbf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:38.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.372+0000 7fd4ee59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e72f906d40 0x55e72f909130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:38.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.372+0000 7fd4ef59e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e72f8fbb60 0x55e72f8fbf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:49074/0 (socket says 172.21.15.16:49074) 2024-09-12T02:03:38.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.372+0000 7fd4ef59e640 1 -- 172.21.15.16:0/3375027848 learned_addr learned my addr 172.21.15.16:0/3375027848 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:03:38.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.372+0000 7fd4d7fff640 1 -- 172.21.15.16:0/699562162 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fd4e00109d0 con 0x7fd4f0100b20 2024-09-12T02:03:38.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.372+0000 7fd4eed9d640 1 --2- 172.21.15.16:0/3375027848 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e72f8fc480 0x55e72f906800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:38.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.372+0000 7fd4ef59e640 1 -- 172.21.15.16:0/3375027848 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e72f906d40 msgr2=0x55e72f909130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:38.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.372+0000 7fd4ef59e640 1 --2- 172.21.15.16:0/3375027848 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e72f906d40 0x55e72f909130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:38.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.372+0000 7fd4ef59e640 1 -- 172.21.15.16:0/3375027848 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e72f8fc480 msgr2=0x55e72f906800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:38.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.372+0000 7fd4ef59e640 1 --2- 172.21.15.16:0/3375027848 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e72f8fc480 0x55e72f906800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:38.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.372+0000 7fd4ef59e640 1 -- 172.21.15.16:0/3375027848 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e72f8151f0 con 0x55e72f8fbb60 2024-09-12T02:03:38.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.372+0000 7fd4ee59c640 1 --2- 172.21.15.16:0/3375027848 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e72f906d40 0x55e72f909130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:03:38.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.372+0000 7fd4eed9d640 1 --2- 172.21.15.16:0/3375027848 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e72f8fc480 0x55e72f906800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:03:38.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.372+0000 7fd4ef59e640 1 --2- 172.21.15.16:0/3375027848 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e72f8fbb60 0x55e72f8fbf40 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd4e800a400 tx=0x7fd4e8009650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e3fc0979dfb7a59c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:38.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.373+0000 7fd4d57fa640 1 -- 172.21.15.16:0/3375027848 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4e8013790 con 0x55e72f8fbb60 2024-09-12T02:03:38.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.373+0000 7fd4d57fa640 1 -- 172.21.15.16:0/3375027848 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd4e8019070 con 0x55e72f8fbb60 2024-09-12T02:03:38.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.373+0000 7fd4d57fa640 1 -- 172.21.15.16:0/3375027848 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4e8014d80 con 0x55e72f8fbb60 2024-09-12T02:03:38.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.373+0000 7fd4d7fff640 1 --2- 172.21.15.16:0/699562162 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fd4d80a2b80 0x7fd4d80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:38.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.373+0000 7fd4eed9d640 1 --2- 172.21.15.16:0/699562162 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fd4d80a2b80 0x7fd4d80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:38.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.373+0000 7fd4d7fff640 1 -- 172.21.15.16:0/699562162 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fd4e00ceb80 con 0x7fd4f0100b20 2024-09-12T02:03:38.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.373+0000 7fd4f755a740 1 -- 172.21.15.16:0/3375027848 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e72f8fbb60 msgr2=0x55e72f8fbf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:38.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.373+0000 7fd4eed9d640 1 --2- 172.21.15.16:0/699562162 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fd4d80a2b80 0x7fd4d80a5040 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fd4e009b4c0 tx=0x7fd4e0000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:38.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.373+0000 7fd4f755a740 1 --2- 172.21.15.16:0/3375027848 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e72f8fbb60 0x55e72f8fbf40 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd4e800a400 tx=0x7fd4e8009650 comp rx=0 tx=0).stop 2024-09-12T02:03:38.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.374+0000 7fd4f755a740 1 -- 172.21.15.16:0/3375027848 shutdown_connections 2024-09-12T02:03:38.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.374+0000 7fd4f755a740 1 --2- 172.21.15.16:0/3375027848 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e72f906d40 0x55e72f909130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:38.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.374+0000 7fd4f755a740 1 --2- 172.21.15.16:0/3375027848 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e72f8fc480 0x55e72f906800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:38.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.374+0000 7fd4f755a740 1 --2- 172.21.15.16:0/3375027848 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e72f8fbb60 0x55e72f8fbf40 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:38.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.374+0000 7fd4f755a740 1 -- 172.21.15.16:0/3375027848 >> 172.21.15.16:0/3375027848 conn(0x55e72f8f80f0 msgr2=0x55e72f8f84d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:38.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.374+0000 7fd4f755a740 1 -- 172.21.15.16:0/3375027848 shutdown_connections 2024-09-12T02:03:38.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.374+0000 7fd4f755a740 1 -- 172.21.15.16:0/3375027848 wait complete. 2024-09-12T02:03:38.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.374+0000 7fd4f755a740 1 -- start start 2024-09-12T02:03:38.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.374+0000 7fd4f755a740 1 Processor -- start 2024-09-12T02:03:38.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.374+0000 7fd4f755a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e72f8fbb60 0x55e72f903bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:38.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.374+0000 7fd4f755a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e72f8fc480 0x55e72f9040f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:38.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.374+0000 7fd4f755a740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e72f906d40 0x55e72f904630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:38.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.374+0000 7fd4f755a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e72f909910 con 0x55e72f8fc480 2024-09-12T02:03:38.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.374+0000 7fd4f755a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55e72f81bf10 con 0x55e72f8fbb60 2024-09-12T02:03:38.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.374+0000 7fd4f755a740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55e72f909a90 con 0x55e72f906d40 2024-09-12T02:03:38.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.374+0000 7fd4f755a740 10 client.0 _ll_register_callbacks cb 0x55e72f7e4a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:03:38.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.375+0000 7fd4ee59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e72f8fc480 0x55e72f9040f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:38.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.375+0000 7fd4ef59e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e72f8fbb60 0x55e72f903bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:38.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.375+0000 7fd4ee59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e72f8fc480 0x55e72f9040f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51948/0 (socket says 172.21.15.16:51948) 2024-09-12T02:03:38.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.375+0000 7fd4ee59c640 1 -- 172.21.15.16:0/2515067995 learned_addr learned my addr 172.21.15.16:0/2515067995 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:03:38.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.375+0000 7fd4ef59e640 1 --2- 172.21.15.16:0/2515067995 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e72f906d40 0x55e72f904630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:38.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.375+0000 7fd4ef59e640 1 -- 172.21.15.16:0/2515067995 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e72f906d40 msgr2=0x55e72f904630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:38.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.375+0000 7fd4ef59e640 1 --2- 172.21.15.16:0/2515067995 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e72f906d40 0x55e72f904630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:38.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.375+0000 7fd4ef59e640 1 -- 172.21.15.16:0/2515067995 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e72f8fc480 msgr2=0x55e72f9040f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:38.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.375+0000 7fd4ef59e640 1 --2- 172.21.15.16:0/2515067995 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e72f8fc480 0x55e72f9040f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:38.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.375+0000 7fd4ef59e640 1 -- 172.21.15.16:0/2515067995 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e72f5d26a0 con 0x55e72f8fbb60 2024-09-12T02:03:38.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.375+0000 7fd4ef59e640 1 --2- 172.21.15.16:0/2515067995 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e72f8fbb60 0x55e72f903bb0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd4e800ac90 tx=0x7fd4e8014960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=93c72c152323e032 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:38.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.375+0000 7fd4d2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-12T02:03:38.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.375+0000 7fd4d2ff5640 1 -- 172.21.15.16:0/2515067995 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4e80139a0 con 0x55e72f8fbb60 2024-09-12T02:03:38.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.375+0000 7fd4f755a740 1 -- 172.21.15.16:0/2515067995 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd4e8019070 con 0x55e72f8fbb60 2024-09-12T02:03:38.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.375+0000 7fd4d2ff5640 1 -- 172.21.15.16:0/2515067995 <== mon.2 v2:172.21.15.33:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fd4e8013b40 con 0x55e72f8fbb60 2024-09-12T02:03:38.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.375+0000 7fd4d2ff5640 1 -- 172.21.15.16:0/2515067995 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd4e8013d40 con 0x55e72f8fbb60 2024-09-12T02:03:38.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.376+0000 7fd4f755a740 10 client.25556 fetch_fsmap learned FSMap version 25 2024-09-12T02:03:38.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.376+0000 7fd4f755a740 1 -- 172.21.15.16:0/2515067995 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e72f915410 con 0x55e72f8fbb60 2024-09-12T02:03:38.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.376+0000 7fd4d2ff5640 1 -- 172.21.15.16:0/2515067995 <== mon.2 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd4e8019070 con 0x55e72f8fbb60 2024-09-12T02:03:38.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.376+0000 7fd4d2ff5640 1 -- 172.21.15.16:0/2515067995 <== mon.2 v2:172.21.15.33:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fd4e8020420 con 0x55e72f8fbb60 2024-09-12T02:03:38.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.376+0000 7fd4f755a740 10 client.25556 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:03:38.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.376+0000 7fd4f755a740 10 client.25556 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:03:38.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.376+0000 7fd4f755a740 20 client.25556 populate_metadata read hostname 'smithi016' 2024-09-12T02:03:38.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.376+0000 7fd4f755a740 1 --2- 172.21.15.16:0/2515067995 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55e72f916820 0x55e72f918c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:38.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.376+0000 7fd4f755a740 4 client.25556 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:03:38.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.376+0000 7fd4f755a740 1 -- 172.21.15.16:0/2515067995 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e72f86e7f0 con 0x55e72f916820 2024-09-12T02:03:38.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.376+0000 7fd4ee59c640 1 --2- 172.21.15.16:0/2515067995 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55e72f916820 0x55e72f918c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:38.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.377+0000 7fd4ee59c640 1 --2- 172.21.15.16:0/2515067995 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55e72f916820 0x55e72f918c10 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4d94d1ff94a71f2 server_cookie=4ead0035c51ec743 in_seq=0 out_seq=0 2024-09-12T02:03:38.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.377+0000 7fd4d2ff5640 10 client.25556 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:03:38.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.377+0000 7fd4d2ff5640 1 -- 172.21.15.16:0/2515067995 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e72f86e7f0 con 0x55e72f916820 2024-09-12T02:03:38.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.377+0000 7fd4d2ff5640 10 client.25556 handle_command_reply: tid=0 2024-09-12T02:03:38.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.377+0000 7fd4f755a740 1 client.25556 shutdown 2024-09-12T02:03:38.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.378+0000 7fd4f755a740 1 -- 172.21.15.16:0/2515067995 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e72f8fbb60 msgr2=0x55e72f903bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:38.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.378+0000 7fd4f755a740 1 --2- 172.21.15.16:0/2515067995 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e72f8fbb60 0x55e72f903bb0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd4e800ac90 tx=0x7fd4e8014960 comp rx=0 tx=0).stop 2024-09-12T02:03:38.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.378+0000 7fd4f755a740 1 -- 172.21.15.16:0/2515067995 shutdown_connections 2024-09-12T02:03:38.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.378+0000 7fd4f755a740 1 --2- 172.21.15.16:0/2515067995 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55e72f916820 0x55e72f918c10 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:38.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.378+0000 7fd4f755a740 1 --2- 172.21.15.16:0/2515067995 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e72f906d40 0x55e72f904630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:38.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.378+0000 7fd4f755a740 1 --2- 172.21.15.16:0/2515067995 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e72f8fc480 0x55e72f9040f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:38.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.378+0000 7fd4d2ff5640 0 client.25556 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:03:38.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.378+0000 7fd4f755a740 1 --2- 172.21.15.16:0/2515067995 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e72f8fbb60 0x55e72f903bb0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:38.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.378+0000 7fd4f755a740 1 -- 172.21.15.16:0/2515067995 >> 172.21.15.16:0/2515067995 conn(0x55e72f8f80f0 msgr2=0x55e72f8fde20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:38.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.378+0000 7fd4f755a740 1 -- 172.21.15.16:0/2515067995 shutdown_connections 2024-09-12T02:03:38.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.378+0000 7fd4f755a740 1 -- 172.21.15.16:0/2515067995 wait complete. 2024-09-12T02:03:38.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.378+0000 7fd4f755a740 20 client.25556 trim_cache size 0 max 16384 2024-09-12T02:03:38.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.396+0000 7fd4f755a740 1 Processor -- start 2024-09-12T02:03:38.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.396+0000 7fd4f755a740 1 -- start start 2024-09-12T02:03:38.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.396+0000 7fd4f755a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e72f8fbb60 0x55e72f905a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:38.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.396+0000 7fd4f755a740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e72f8fc480 0x55e72f91e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:38.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.396+0000 7fd4f755a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e72f906d40 0x55e72f91e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:38.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.396+0000 7fd4f755a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd4f01b1fe0 con 0x55e72f906d40 2024-09-12T02:03:38.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.396+0000 7fd4f755a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd4f01af8b0 con 0x55e72f8fbb60 2024-09-12T02:03:38.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.396+0000 7fd4f755a740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd4f01ad720 con 0x55e72f8fc480 2024-09-12T02:03:38.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.396+0000 7fd4ef59e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e72f906d40 0x55e72f91e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:38.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.396+0000 7fd4ef59e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e72f906d40 0x55e72f91e8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51958/0 (socket says 172.21.15.16:51958) 2024-09-12T02:03:38.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.396+0000 7fd4ef59e640 1 -- 172.21.15.16:0/2466037081 learned_addr learned my addr 172.21.15.16:0/2466037081 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:03:38.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.396+0000 7fd4ee59c640 1 --2- 172.21.15.16:0/2466037081 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e72f8fc480 0x55e72f91e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:38.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.396+0000 7fd4ef59e640 1 --2- 172.21.15.16:0/2466037081 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e72f8fbb60 0x55e72f905a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:38.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.397+0000 7fd4ef59e640 1 -- 172.21.15.16:0/2466037081 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e72f8fc480 msgr2=0x55e72f91e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:38.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.397+0000 7fd4ef59e640 1 --2- 172.21.15.16:0/2466037081 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e72f8fc480 0x55e72f91e3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:38.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.397+0000 7fd4ef59e640 1 -- 172.21.15.16:0/2466037081 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e72f8fbb60 msgr2=0x55e72f905a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:38.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.397+0000 7fd4ef59e640 1 --2- 172.21.15.16:0/2466037081 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e72f8fbb60 0x55e72f905a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:38.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.397+0000 7fd4ef59e640 1 -- 172.21.15.16:0/2466037081 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e72f8fb990 con 0x55e72f906d40 2024-09-12T02:03:38.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.397+0000 7fd4ef59e640 1 --2- 172.21.15.16:0/2466037081 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e72f906d40 0x55e72f91e8e0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fd4e80144f0 tx=0x7fd4e8009650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1675c492c1ccb43 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:38.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.397+0000 7fd4d4ff9640 1 -- 172.21.15.16:0/2466037081 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4e80145e0 con 0x55e72f906d40 2024-09-12T02:03:38.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.397+0000 7fd4d4ff9640 1 -- 172.21.15.16:0/2466037081 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd4e8019040 con 0x55e72f906d40 2024-09-12T02:03:38.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.397+0000 7fd4d4ff9640 1 -- 172.21.15.16:0/2466037081 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4e802d430 con 0x55e72f906d40 2024-09-12T02:03:38.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.398+0000 7fd4f755a740 1 -- 172.21.15.16:0/2466037081 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e72f906d40 msgr2=0x55e72f91e8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:38.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.398+0000 7fd4f755a740 1 --2- 172.21.15.16:0/2466037081 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e72f906d40 0x55e72f91e8e0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fd4e80144f0 tx=0x7fd4e8009650 comp rx=0 tx=0).stop 2024-09-12T02:03:38.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.398+0000 7fd4f755a740 1 -- 172.21.15.16:0/2466037081 shutdown_connections 2024-09-12T02:03:38.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.398+0000 7fd4f755a740 1 --2- 172.21.15.16:0/2466037081 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e72f906d40 0x55e72f91e8e0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:38.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.398+0000 7fd4f755a740 1 --2- 172.21.15.16:0/2466037081 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e72f8fc480 0x55e72f91e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:38.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.398+0000 7fd4f755a740 1 --2- 172.21.15.16:0/2466037081 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e72f8fbb60 0x55e72f905a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:38.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.398+0000 7fd4f755a740 1 -- 172.21.15.16:0/2466037081 >> 172.21.15.16:0/2466037081 conn(0x55e72f8f80f0 msgr2=0x55e72f905300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:38.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.398+0000 7fd4f755a740 1 -- 172.21.15.16:0/2466037081 shutdown_connections 2024-09-12T02:03:38.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.398+0000 7fd4f755a740 1 -- 172.21.15.16:0/2466037081 wait complete. 2024-09-12T02:03:38.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.398+0000 7fd4f755a740 1 -- start start 2024-09-12T02:03:38.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.398+0000 7fd4f755a740 1 Processor -- start 2024-09-12T02:03:38.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.398+0000 7fd4f755a740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e72f8fbb60 0x55e72f90adc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:38.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.398+0000 7fd4f755a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e72f8fc480 0x55e72f90b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:38.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.398+0000 7fd4f755a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e72f906d40 0x55e72f980370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:38.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.398+0000 7fd4f755a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd4f01add70 con 0x55e72f8fc480 2024-09-12T02:03:38.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.399+0000 7fd4f755a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd4f01af1e0 con 0x55e72f906d40 2024-09-12T02:03:38.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.399+0000 7fd4f755a740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd4f01ae440 con 0x55e72f8fbb60 2024-09-12T02:03:38.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.399+0000 7fd4f755a740 10 client.0 _ll_register_callbacks cb 0x7fd4e8009e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:03:38.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.399+0000 7fd4ee59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e72f8fc480 0x55e72f90b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:38.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.399+0000 7fd4ee59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e72f8fc480 0x55e72f90b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51968/0 (socket says 172.21.15.16:51968) 2024-09-12T02:03:38.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.399+0000 7fd4ee59c640 1 -- 172.21.15.16:0/3340259949 learned_addr learned my addr 172.21.15.16:0/3340259949 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:03:38.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.399+0000 7fd4ef59e640 1 --2- 172.21.15.16:0/3340259949 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e72f8fbb60 0x55e72f90adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:38.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.399+0000 7fd4ef59e640 1 --2- 172.21.15.16:0/3340259949 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e72f906d40 0x55e72f980370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:38.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.399+0000 7fd4ee59c640 1 -- 172.21.15.16:0/3340259949 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e72f8fbb60 msgr2=0x55e72f90adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:38.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.399+0000 7fd4ee59c640 1 --2- 172.21.15.16:0/3340259949 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e72f8fbb60 0x55e72f90adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:38.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.399+0000 7fd4ee59c640 1 -- 172.21.15.16:0/3340259949 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e72f906d40 msgr2=0x55e72f980370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:38.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.399+0000 7fd4ee59c640 1 --2- 172.21.15.16:0/3340259949 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e72f906d40 0x55e72f980370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:38.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.399+0000 7fd4ee59c640 1 -- 172.21.15.16:0/3340259949 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e72f8f7f20 con 0x55e72f8fc480 2024-09-12T02:03:38.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.399+0000 7fd4ee59c640 1 --2- 172.21.15.16:0/3340259949 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e72f8fc480 0x55e72f90b300 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fd4e4009850 tx=0x7fd4e40041a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbe6244695199f74 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:38.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.399+0000 7fd4d2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:03:38.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.399+0000 7fd4d2ff5640 1 -- 172.21.15.16:0/3340259949 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4e4035040 con 0x55e72f8fc480 2024-09-12T02:03:38.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.399+0000 7fd4d2ff5640 1 -- 172.21.15.16:0/3340259949 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fd4e4031ab0 con 0x55e72f8fc480 2024-09-12T02:03:38.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.399+0000 7fd4f755a740 1 -- 172.21.15.16:0/3340259949 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd4e8019040 con 0x55e72f8fc480 2024-09-12T02:03:38.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.400+0000 7fd4d2ff5640 1 -- 172.21.15.16:0/3340259949 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd4e4059050 con 0x55e72f8fc480 2024-09-12T02:03:38.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.400+0000 7fd4f755a740 10 client.15933 fetch_fsmap learned FSMap version 25 2024-09-12T02:03:38.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.400+0000 7fd4f755a740 1 -- 172.21.15.16:0/3340259949 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e72f9817e0 con 0x55e72f8fc480 2024-09-12T02:03:38.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.400+0000 7fd4d2ff5640 1 -- 172.21.15.16:0/3340259949 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd4e8019040 con 0x55e72f8fc480 2024-09-12T02:03:38.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.400+0000 7fd4d2ff5640 1 -- 172.21.15.16:0/3340259949 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fd4e4051020 con 0x55e72f8fc480 2024-09-12T02:03:38.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.400+0000 7fd4f755a740 10 client.15933 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:03:38.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.400+0000 7fd4f755a740 10 client.15933 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:03:38.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.400+0000 7fd4f755a740 20 client.15933 populate_metadata read hostname 'smithi016' 2024-09-12T02:03:38.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.400+0000 7fd4f755a740 1 --2- 172.21.15.16:0/3340259949 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55e72f981f90 0x55e72f9843b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:38.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.400+0000 7fd4f755a740 4 client.15933 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-12T02:03:38.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.400+0000 7fd4f755a740 1 -- 172.21.15.16:0/3340259949 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55e72f868d00 con 0x55e72f981f90 2024-09-12T02:03:38.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.400+0000 7fd4ef59e640 1 --2- 172.21.15.16:0/3340259949 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55e72f981f90 0x55e72f9843b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:38.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.401+0000 7fd4ef59e640 1 --2- 172.21.15.16:0/3340259949 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55e72f981f90 0x55e72f9843b0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8bad615998f14b30 server_cookie=918bbfcd958dfabd in_seq=0 out_seq=0 2024-09-12T02:03:38.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.401+0000 7fd4d2ff5640 10 client.15933 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:03:38.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.402+0000 7fd4d2ff5640 1 -- 172.21.15.16:0/3340259949 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55e72f868d00 con 0x55e72f981f90 2024-09-12T02:03:38.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.402+0000 7fd4d2ff5640 10 client.15933 handle_command_reply: tid=0 2024-09-12T02:03:38.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.402+0000 7fd4f755a740 1 client.15933 shutdown 2024-09-12T02:03:38.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.402+0000 7fd4f755a740 1 -- 172.21.15.16:0/3340259949 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e72f8fc480 msgr2=0x55e72f90b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:38.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.402+0000 7fd4f755a740 1 --2- 172.21.15.16:0/3340259949 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e72f8fc480 0x55e72f90b300 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fd4e4009850 tx=0x7fd4e40041a0 comp rx=0 tx=0).stop 2024-09-12T02:03:38.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.402+0000 7fd4f755a740 1 -- 172.21.15.16:0/3340259949 shutdown_connections 2024-09-12T02:03:38.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.402+0000 7fd4f755a740 1 --2- 172.21.15.16:0/3340259949 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55e72f981f90 0x55e72f9843b0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:38.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.402+0000 7fd4f755a740 1 --2- 172.21.15.16:0/3340259949 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e72f906d40 0x55e72f980370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:38.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.402+0000 7fd4f755a740 1 --2- 172.21.15.16:0/3340259949 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e72f8fc480 0x55e72f90b300 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:38.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.402+0000 7fd4d2ff5640 0 client.15933 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:03:38.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.402+0000 7fd4f755a740 1 --2- 172.21.15.16:0/3340259949 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e72f8fbb60 0x55e72f90adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:38.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.402+0000 7fd4f755a740 1 -- 172.21.15.16:0/3340259949 >> 172.21.15.16:0/3340259949 conn(0x55e72f8f80f0 msgr2=0x55e72f904390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:38.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.402+0000 7fd4f755a740 1 -- 172.21.15.16:0/3340259949 shutdown_connections 2024-09-12T02:03:38.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.402+0000 7fd4f755a740 1 -- 172.21.15.16:0/3340259949 wait complete. 2024-09-12T02:03:38.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.402+0000 7fd4f755a740 20 client.15933 trim_cache size 0 max 16384 2024-09-12T02:03:38.408 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T02:03:38.408 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-09-12T02:03:38.408 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "685101fe-bd5a-4995-8286-22d62349bfc3", 2024-09-12T02:03:38.408 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-09-12T02:03:38.408 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T02:03:38.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.403+0000 7fd4f5181640 1 -- 172.21.15.16:0/699562162 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fd4d80a2b80 msgr2=0x7fd4d80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:38.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.403+0000 7fd4f5181640 1 --2- 172.21.15.16:0/699562162 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fd4d80a2b80 0x7fd4d80a5040 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fd4e009b4c0 tx=0x7fd4e0000bc0 comp rx=0 tx=0).stop 2024-09-12T02:03:38.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.403+0000 7fd4f5181640 1 -- 172.21.15.16:0/699562162 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4f0100b20 msgr2=0x7fd4f01a0540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:38.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.403+0000 7fd4f5181640 1 --2- 172.21.15.16:0/699562162 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4f0100b20 0x7fd4f01a0540 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fd4e000d950 tx=0x7fd4e000de10 comp rx=0 tx=0).stop 2024-09-12T02:03:38.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.403+0000 7fd4f5181640 1 -- 172.21.15.16:0/699562162 shutdown_connections 2024-09-12T02:03:38.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.403+0000 7fd4f5181640 1 --2- 172.21.15.16:0/699562162 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fd4d80a2b80 0x7fd4d80a5040 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:38.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.403+0000 7fd4f5181640 1 --2- 172.21.15.16:0/699562162 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd4f010e500 0x7fd4f019a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:38.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.403+0000 7fd4f5181640 1 --2- 172.21.15.16:0/699562162 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4f0101440 0x7fd4f01a0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:38.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.403+0000 7fd4f5181640 1 --2- 172.21.15.16:0/699562162 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4f0100b20 0x7fd4f01a0540 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:38.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.403+0000 7fd4f5181640 1 -- 172.21.15.16:0/699562162 >> 172.21.15.16:0/699562162 conn(0x7fd4f00fc7d0 msgr2=0x7fd4f010f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:38.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.403+0000 7fd4f5181640 1 -- 172.21.15.16:0/699562162 shutdown_connections 2024-09-12T02:03:38.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:38.403+0000 7fd4f5181640 1 -- 172.21.15.16:0/699562162 wait complete. 2024-09-12T02:03:39.081 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T02:03:39.341 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:03:39.780 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:03:39 smithi016 ceph-mon[30947]: pgmap v387: 97 pgs: 97 active+clean; 1.1 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-09-12T02:03:39.780 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:03:39 smithi016 ceph-mon[30947]: scrub queued for path: / 2024-09-12T02:03:39.780 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:03:39 smithi016 ceph-mon[30947]: scrub summary: idle+waiting paths [/] 2024-09-12T02:03:39.780 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:03:39 smithi016 ceph-mon[30947]: scrub summary: active paths [/] 2024-09-12T02:03:39.780 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:03:39 smithi016 ceph-mon[30947]: scrub summary: idle+waiting paths [/] 2024-09-12T02:03:39.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.780+0000 7fca9283c640 1 -- 172.21.15.16:0/232064854 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca8c101420 msgr2=0x7fca8c10d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:39.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.780+0000 7fca9283c640 1 --2- 172.21.15.16:0/232064854 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca8c101420 0x7fca8c10d950 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fca84009cd0 tx=0x7fca8402a8e0 comp rx=0 tx=0).stop 2024-09-12T02:03:39.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.780+0000 7fca9283c640 1 -- 172.21.15.16:0/232064854 shutdown_connections 2024-09-12T02:03:39.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.780+0000 7fca9283c640 1 --2- 172.21.15.16:0/232064854 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca8c101420 0x7fca8c10d950 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:39.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.780+0000 7fca9283c640 1 --2- 172.21.15.16:0/232064854 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca8c100a80 0x7fca8c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:39.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.780+0000 7fca9283c640 1 --2- 172.21.15.16:0/232064854 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fca8c106a80 0x7fca8c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:39.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.780+0000 7fca9283c640 1 -- 172.21.15.16:0/232064854 >> 172.21.15.16:0/232064854 conn(0x7fca8c0fc7a0 msgr2=0x7fca8c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:39.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.780+0000 7fca9283c640 1 -- 172.21.15.16:0/232064854 shutdown_connections 2024-09-12T02:03:39.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.780+0000 7fca9283c640 1 -- 172.21.15.16:0/232064854 wait complete. 2024-09-12T02:03:39.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.780+0000 7fca9283c640 1 Processor -- start 2024-09-12T02:03:39.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.781+0000 7fca9283c640 1 -- start start 2024-09-12T02:03:39.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.781+0000 7fca9283c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca8c100a80 0x7fca8c1992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:39.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.781+0000 7fca9283c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca8c101420 0x7fca8c199830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:39.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.781+0000 7fca9283c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fca8c106a80 0x7fca8c1a4370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:39.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.781+0000 7fca9283c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fca8c110ea0 con 0x7fca8c101420 2024-09-12T02:03:39.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.781+0000 7fca9283c640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fca8c110d20 con 0x7fca8c100a80 2024-09-12T02:03:39.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.781+0000 7fca9283c640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fca8c111020 con 0x7fca8c106a80 2024-09-12T02:03:39.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.781+0000 7fca8b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca8c101420 0x7fca8c199830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:39.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.781+0000 7fca8b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca8c101420 0x7fca8c199830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51988/0 (socket says 172.21.15.16:51988) 2024-09-12T02:03:39.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.781+0000 7fca8b7fe640 1 -- 172.21.15.16:0/1518908889 learned_addr learned my addr 172.21.15.16:0/1518908889 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:03:39.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.781+0000 7fca90dd9640 1 --2- 172.21.15.16:0/1518908889 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fca8c106a80 0x7fca8c1a4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:39.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.781+0000 7fca8bfff640 1 --2- 172.21.15.16:0/1518908889 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca8c100a80 0x7fca8c1992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:39.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.782+0000 7fca8b7fe640 1 -- 172.21.15.16:0/1518908889 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fca8c106a80 msgr2=0x7fca8c1a4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:39.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.782+0000 7fca8b7fe640 1 --2- 172.21.15.16:0/1518908889 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fca8c106a80 0x7fca8c1a4370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:39.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.782+0000 7fca8b7fe640 1 -- 172.21.15.16:0/1518908889 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca8c100a80 msgr2=0x7fca8c1992f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:39.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.782+0000 7fca8b7fe640 1 --2- 172.21.15.16:0/1518908889 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca8c100a80 0x7fca8c1992f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:39.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.782+0000 7fca8b7fe640 1 -- 172.21.15.16:0/1518908889 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca8c1a48b0 con 0x7fca8c101420 2024-09-12T02:03:39.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.782+0000 7fca8bfff640 1 --2- 172.21.15.16:0/1518908889 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca8c100a80 0x7fca8c1992f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:03:39.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.782+0000 7fca8b7fe640 1 --2- 172.21.15.16:0/1518908889 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca8c101420 0x7fca8c199830 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fca8000d950 tx=0x7fca8000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:39.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.782+0000 7fca897fa640 1 -- 172.21.15.16:0/1518908889 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca80017070 con 0x7fca8c101420 2024-09-12T02:03:39.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.782+0000 7fca897fa640 1 -- 172.21.15.16:0/1518908889 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca80009c30 con 0x7fca8c101420 2024-09-12T02:03:39.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.782+0000 7fca9283c640 1 -- 172.21.15.16:0/1518908889 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca8c1a4a80 con 0x7fca8c101420 2024-09-12T02:03:39.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.782+0000 7fca897fa640 1 -- 172.21.15.16:0/1518908889 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca8000f6b0 con 0x7fca8c101420 2024-09-12T02:03:39.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.782+0000 7fca9283c640 1 -- 172.21.15.16:0/1518908889 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca8c1a4f90 con 0x7fca8c101420 2024-09-12T02:03:39.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.784+0000 7fca897fa640 1 -- 172.21.15.16:0/1518908889 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fca800109d0 con 0x7fca8c101420 2024-09-12T02:03:39.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.784+0000 7fca94c15740 1 Processor -- start 2024-09-12T02:03:39.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.785+0000 7fca94c15740 1 -- start start 2024-09-12T02:03:39.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.785+0000 7fca94c15740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c1e00404a0 0x55c1e0040880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:39.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.785+0000 7fca94c15740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c1e0040dc0 0x55c1e004b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:39.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.785+0000 7fca94c15740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c1e004b680 0x55c1e004da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:39.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.785+0000 7fca94c15740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55c1dffa62e0 con 0x55c1e00404a0 2024-09-12T02:03:39.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.785+0000 7fca94c15740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55c1dff4dce0 con 0x55c1e0040dc0 2024-09-12T02:03:39.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.785+0000 7fca94c15740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55c1dffa22a0 con 0x55c1e004b680 2024-09-12T02:03:39.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.785+0000 7fca90dd9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c1e00404a0 0x55c1e0040880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:39.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.785+0000 7fca897fa640 1 --2- 172.21.15.16:0/1518908889 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fca740a2b80 0x7fca740a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:39.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.785+0000 7fca90dd9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c1e00404a0 0x55c1e0040880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51996/0 (socket says 172.21.15.16:51996) 2024-09-12T02:03:39.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.785+0000 7fca90dd9640 1 -- 172.21.15.16:0/1429888295 learned_addr learned my addr 172.21.15.16:0/1429888295 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:03:39.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.785+0000 7fca8bfff640 1 --2- 172.21.15.16:0/1429888295 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c1e0040dc0 0x55c1e004b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:39.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.785+0000 7fca8b7fe640 1 --2- 172.21.15.16:0/1429888295 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c1e004b680 0x55c1e004da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:39.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.785+0000 7fca90dd9640 1 --2- 172.21.15.16:0/1518908889 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fca740a2b80 0x7fca740a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:39.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.785+0000 7fca897fa640 1 -- 172.21.15.16:0/1518908889 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fca800cece0 con 0x7fca8c101420 2024-09-12T02:03:39.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.785+0000 7fca90dd9640 1 -- 172.21.15.16:0/1429888295 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c1e004b680 msgr2=0x55c1e004da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:39.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.785+0000 7fca90dd9640 1 --2- 172.21.15.16:0/1429888295 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c1e004b680 0x55c1e004da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:39.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.785+0000 7fca90dd9640 1 -- 172.21.15.16:0/1429888295 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c1e0040dc0 msgr2=0x55c1e004b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:39.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.785+0000 7fca90dd9640 1 --2- 172.21.15.16:0/1429888295 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c1e0040dc0 0x55c1e004b140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:39.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.785+0000 7fca90dd9640 1 -- 172.21.15.16:0/1429888295 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c1dff32440 con 0x55c1e00404a0 2024-09-12T02:03:39.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.785+0000 7fca8b7fe640 1 --2- 172.21.15.16:0/1429888295 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c1e004b680 0x55c1e004da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:03:39.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.785+0000 7fca8bfff640 1 --2- 172.21.15.16:0/1429888295 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c1e0040dc0 0x55c1e004b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:03:39.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.786+0000 7fca90dd9640 1 --2- 172.21.15.16:0/1429888295 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c1e00404a0 0x55c1e0040880 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fca84009430 tx=0x7fca84031c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=240d36ef2eeed65d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:39.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.786+0000 7fca727fc640 1 -- 172.21.15.16:0/1429888295 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca840315e0 con 0x55c1e00404a0 2024-09-12T02:03:39.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.786+0000 7fca727fc640 1 -- 172.21.15.16:0/1429888295 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca84036040 con 0x55c1e00404a0 2024-09-12T02:03:39.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.786+0000 7fca727fc640 1 -- 172.21.15.16:0/1429888295 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca84030830 con 0x55c1e00404a0 2024-09-12T02:03:39.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.786+0000 7fca94c15740 1 -- 172.21.15.16:0/1429888295 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c1e00404a0 msgr2=0x55c1e0040880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:39.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.786+0000 7fca90dd9640 1 --2- 172.21.15.16:0/1518908889 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fca740a2b80 0x7fca740a5040 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fca8404d000 tx=0x7fca8404b000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:39.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.786+0000 7fca94c15740 1 --2- 172.21.15.16:0/1429888295 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c1e00404a0 0x55c1e0040880 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fca84009430 tx=0x7fca84031c00 comp rx=0 tx=0).stop 2024-09-12T02:03:39.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.787+0000 7fca94c15740 1 -- 172.21.15.16:0/1429888295 shutdown_connections 2024-09-12T02:03:39.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.787+0000 7fca94c15740 1 --2- 172.21.15.16:0/1429888295 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c1e004b680 0x55c1e004da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:39.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.787+0000 7fca94c15740 1 --2- 172.21.15.16:0/1429888295 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c1e0040dc0 0x55c1e004b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:39.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.787+0000 7fca94c15740 1 --2- 172.21.15.16:0/1429888295 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c1e00404a0 0x55c1e0040880 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:39.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.787+0000 7fca94c15740 1 -- 172.21.15.16:0/1429888295 >> 172.21.15.16:0/1429888295 conn(0x55c1e003ca30 msgr2=0x55c1e003ce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:39.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.787+0000 7fca94c15740 1 -- 172.21.15.16:0/1429888295 shutdown_connections 2024-09-12T02:03:39.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.787+0000 7fca94c15740 1 -- 172.21.15.16:0/1429888295 wait complete. 2024-09-12T02:03:39.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.787+0000 7fca94c15740 1 -- start start 2024-09-12T02:03:39.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.787+0000 7fca94c15740 1 Processor -- start 2024-09-12T02:03:39.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.787+0000 7fca94c15740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c1e00404a0 0x55c1e004af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:39.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.788+0000 7fca94c15740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c1e0040dc0 0x55c1e0048200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:39.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.788+0000 7fca94c15740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c1e004b680 0x55c1e0048740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:39.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.788+0000 7fca94c15740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55c1e004e250 con 0x55c1e00404a0 2024-09-12T02:03:39.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.788+0000 7fca94c15740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55c1dff61050 con 0x55c1e0040dc0 2024-09-12T02:03:39.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.788+0000 7fca94c15740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55c1e004e3d0 con 0x55c1e004b680 2024-09-12T02:03:39.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.788+0000 7fca94c15740 10 client.0 _ll_register_callbacks cb 0x55c1dffa1160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:03:39.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.788+0000 7fca8bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c1e00404a0 0x55c1e004af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:39.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.788+0000 7fca8bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c1e00404a0 0x55c1e004af50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52004/0 (socket says 172.21.15.16:52004) 2024-09-12T02:03:39.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.788+0000 7fca8bfff640 1 -- 172.21.15.16:0/3343942285 learned_addr learned my addr 172.21.15.16:0/3343942285 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:03:39.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.788+0000 7fca8b7fe640 1 --2- 172.21.15.16:0/3343942285 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c1e0040dc0 0x55c1e0048200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:39.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.788+0000 7fca90dd9640 1 --2- 172.21.15.16:0/3343942285 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c1e004b680 0x55c1e0048740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:39.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.788+0000 7fca8bfff640 1 -- 172.21.15.16:0/3343942285 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c1e004b680 msgr2=0x55c1e0048740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:39.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.788+0000 7fca8bfff640 1 --2- 172.21.15.16:0/3343942285 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c1e004b680 0x55c1e0048740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:39.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.788+0000 7fca8bfff640 1 -- 172.21.15.16:0/3343942285 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c1e0040dc0 msgr2=0x55c1e0048200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:39.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.788+0000 7fca8bfff640 1 --2- 172.21.15.16:0/3343942285 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c1e0040dc0 0x55c1e0048200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:39.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.788+0000 7fca8bfff640 1 -- 172.21.15.16:0/3343942285 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c1dfd173c0 con 0x55c1e00404a0 2024-09-12T02:03:39.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.788+0000 7fca8b7fe640 1 --2- 172.21.15.16:0/3343942285 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c1e0040dc0 0x55c1e0048200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:03:39.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.788+0000 7fca8bfff640 1 --2- 172.21.15.16:0/3343942285 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c1e00404a0 0x55c1e004af50 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fca7c00d470 tx=0x7fca7c00d930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=444c8a71e8f965f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:39.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.788+0000 7fca6b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:03:39.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.788+0000 7fca6b7fe640 1 -- 172.21.15.16:0/3343942285 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca7c01b060 con 0x55c1e00404a0 2024-09-12T02:03:39.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.789+0000 7fca6b7fe640 1 -- 172.21.15.16:0/3343942285 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fca7c0146f0 con 0x55c1e00404a0 2024-09-12T02:03:39.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.789+0000 7fca94c15740 1 -- 172.21.15.16:0/3343942285 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca84036040 con 0x55c1e00404a0 2024-09-12T02:03:39.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.789+0000 7fca6b7fe640 1 -- 172.21.15.16:0/3343942285 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fca7c0148f0 con 0x55c1e00404a0 2024-09-12T02:03:39.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.789+0000 7fca94c15740 10 client.15945 fetch_fsmap learned FSMap version 25 2024-09-12T02:03:39.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.789+0000 7fca94c15740 1 -- 172.21.15.16:0/3343942285 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c1e005e040 con 0x55c1e00404a0 2024-09-12T02:03:39.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.789+0000 7fca6b7fe640 1 -- 172.21.15.16:0/3343942285 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca84036040 con 0x55c1e00404a0 2024-09-12T02:03:39.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.789+0000 7fca6b7fe640 1 -- 172.21.15.16:0/3343942285 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fca7c020070 con 0x55c1e00404a0 2024-09-12T02:03:39.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.789+0000 7fca94c15740 10 client.15945 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:03:39.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.789+0000 7fca94c15740 10 client.15945 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:03:39.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.789+0000 7fca94c15740 20 client.15945 populate_metadata read hostname 'smithi016' 2024-09-12T02:03:39.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.789+0000 7fca94c15740 1 --2- 172.21.15.16:0/3343942285 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55c1e005f500 0x55c1e0061950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:39.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.789+0000 7fca94c15740 4 client.15945 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:03:39.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.789+0000 7fca94c15740 1 -- 172.21.15.16:0/3343942285 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c1dffb3850 con 0x55c1e005f500 2024-09-12T02:03:39.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.790+0000 7fca8b7fe640 1 --2- 172.21.15.16:0/3343942285 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55c1e005f500 0x55c1e0061950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:39.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.790+0000 7fca8b7fe640 1 --2- 172.21.15.16:0/3343942285 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55c1e005f500 0x55c1e0061950 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb7337f784f597b1 server_cookie=f0b6ae5610082fe8 in_seq=0 out_seq=0 2024-09-12T02:03:39.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.790+0000 7fca6b7fe640 10 client.15945 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:03:39.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.791+0000 7fca6b7fe640 1 -- 172.21.15.16:0/3343942285 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c1dffb3850 con 0x55c1e005f500 2024-09-12T02:03:39.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.791+0000 7fca6b7fe640 10 client.15945 handle_command_reply: tid=0 2024-09-12T02:03:39.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.791+0000 7fca94c15740 1 client.15945 shutdown 2024-09-12T02:03:39.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.791+0000 7fca94c15740 1 -- 172.21.15.16:0/3343942285 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c1e00404a0 msgr2=0x55c1e004af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:39.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.791+0000 7fca94c15740 1 --2- 172.21.15.16:0/3343942285 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c1e00404a0 0x55c1e004af50 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fca7c00d470 tx=0x7fca7c00d930 comp rx=0 tx=0).stop 2024-09-12T02:03:39.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.791+0000 7fca94c15740 1 -- 172.21.15.16:0/3343942285 shutdown_connections 2024-09-12T02:03:39.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.791+0000 7fca94c15740 1 --2- 172.21.15.16:0/3343942285 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55c1e005f500 0x55c1e0061950 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:39.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.791+0000 7fca6b7fe640 0 client.15945 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:03:39.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.791+0000 7fca94c15740 1 --2- 172.21.15.16:0/3343942285 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c1e004b680 0x55c1e0048740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:39.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.791+0000 7fca94c15740 1 --2- 172.21.15.16:0/3343942285 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c1e0040dc0 0x55c1e0048200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:39.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.791+0000 7fca94c15740 1 --2- 172.21.15.16:0/3343942285 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c1e00404a0 0x55c1e004af50 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:39.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.791+0000 7fca94c15740 1 -- 172.21.15.16:0/3343942285 >> 172.21.15.16:0/3343942285 conn(0x55c1e003ca30 msgr2=0x55c1e003bb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:39.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.791+0000 7fca94c15740 1 -- 172.21.15.16:0/3343942285 shutdown_connections 2024-09-12T02:03:39.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.791+0000 7fca94c15740 1 -- 172.21.15.16:0/3343942285 wait complete. 2024-09-12T02:03:39.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.791+0000 7fca94c15740 20 client.15945 trim_cache size 0 max 16384 2024-09-12T02:03:39.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.808+0000 7fca94c15740 1 Processor -- start 2024-09-12T02:03:39.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.808+0000 7fca94c15740 1 -- start start 2024-09-12T02:03:39.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.808+0000 7fca94c15740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c1e003ca30 0x55c1e00481c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:39.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.808+0000 7fca94c15740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c1e00404a0 0x55c1e00437f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:39.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.808+0000 7fca94c15740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c1e0040dc0 0x55c1e0067220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:39.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.808+0000 7fca94c15740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fca8c1adb20 con 0x55c1e0040dc0 2024-09-12T02:03:39.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.808+0000 7fca94c15740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fca8c1ab3f0 con 0x55c1e00404a0 2024-09-12T02:03:39.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.808+0000 7fca94c15740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fca8c1a9240 con 0x55c1e003ca30 2024-09-12T02:03:39.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.809+0000 7fca90dd9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c1e0040dc0 0x55c1e0067220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:39.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.809+0000 7fca90dd9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c1e0040dc0 0x55c1e0067220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52012/0 (socket says 172.21.15.16:52012) 2024-09-12T02:03:39.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.809+0000 7fca90dd9640 1 -- 172.21.15.16:0/2320455121 learned_addr learned my addr 172.21.15.16:0/2320455121 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:03:39.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.809+0000 7fca8b7fe640 1 --2- 172.21.15.16:0/2320455121 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c1e00404a0 0x55c1e00437f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:39.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.809+0000 7fca8bfff640 1 --2- 172.21.15.16:0/2320455121 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c1e003ca30 0x55c1e00481c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:39.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.809+0000 7fca90dd9640 1 -- 172.21.15.16:0/2320455121 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c1e003ca30 msgr2=0x55c1e00481c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:39.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.809+0000 7fca90dd9640 1 --2- 172.21.15.16:0/2320455121 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c1e003ca30 0x55c1e00481c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:39.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.809+0000 7fca90dd9640 1 -- 172.21.15.16:0/2320455121 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c1e00404a0 msgr2=0x55c1e00437f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:39.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.809+0000 7fca90dd9640 1 --2- 172.21.15.16:0/2320455121 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c1e00404a0 0x55c1e00437f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:39.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.809+0000 7fca90dd9640 1 -- 172.21.15.16:0/2320455121 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c1e0040bb0 con 0x55c1e0040dc0 2024-09-12T02:03:39.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.809+0000 7fca90dd9640 1 --2- 172.21.15.16:0/2320455121 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c1e0040dc0 0x55c1e0067220 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fca84031b80 tx=0x7fca84041000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ca01dbe7582dca5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:39.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.809+0000 7fca71ffb640 1 -- 172.21.15.16:0/2320455121 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca84031580 con 0x55c1e0040dc0 2024-09-12T02:03:39.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.809+0000 7fca71ffb640 1 -- 172.21.15.16:0/2320455121 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca84036070 con 0x55c1e0040dc0 2024-09-12T02:03:39.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.809+0000 7fca71ffb640 1 -- 172.21.15.16:0/2320455121 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca84004d80 con 0x55c1e0040dc0 2024-09-12T02:03:39.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.810+0000 7fca94c15740 1 -- 172.21.15.16:0/2320455121 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c1e0040dc0 msgr2=0x55c1e0067220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:39.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.810+0000 7fca94c15740 1 --2- 172.21.15.16:0/2320455121 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c1e0040dc0 0x55c1e0067220 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fca84031b80 tx=0x7fca84041000 comp rx=0 tx=0).stop 2024-09-12T02:03:39.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.810+0000 7fca94c15740 1 -- 172.21.15.16:0/2320455121 shutdown_connections 2024-09-12T02:03:39.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.810+0000 7fca94c15740 1 --2- 172.21.15.16:0/2320455121 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c1e0040dc0 0x55c1e0067220 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:39.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.810+0000 7fca94c15740 1 --2- 172.21.15.16:0/2320455121 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c1e00404a0 0x55c1e00437f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:39.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.810+0000 7fca94c15740 1 --2- 172.21.15.16:0/2320455121 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c1e003ca30 0x55c1e00481c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:39.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.810+0000 7fca94c15740 1 -- 172.21.15.16:0/2320455121 >> 172.21.15.16:0/2320455121 conn(0x55c1e00488a0 msgr2=0x55c1e00430e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:39.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.810+0000 7fca94c15740 1 -- 172.21.15.16:0/2320455121 shutdown_connections 2024-09-12T02:03:39.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.810+0000 7fca94c15740 1 -- 172.21.15.16:0/2320455121 wait complete. 2024-09-12T02:03:39.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.810+0000 7fca94c15740 1 -- start start 2024-09-12T02:03:39.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.811+0000 7fca94c15740 1 Processor -- start 2024-09-12T02:03:39.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.811+0000 7fca94c15740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c1e003ca30 0x55c1e0053a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:39.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.811+0000 7fca94c15740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c1e00404a0 0x55c1e0053fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:39.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.811+0000 7fca94c15740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c1e0040dc0 0x55c1e00544e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:39.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.811+0000 7fca94c15740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fca8c1a98b0 con 0x55c1e0040dc0 2024-09-12T02:03:39.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.811+0000 7fca94c15740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fca8c1aad20 con 0x55c1e00404a0 2024-09-12T02:03:39.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.811+0000 7fca94c15740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fca8c1a9f80 con 0x55c1e003ca30 2024-09-12T02:03:39.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.811+0000 7fca94c15740 10 client.0 _ll_register_callbacks cb 0x7fca7c007500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:03:39.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.811+0000 7fca90dd9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c1e0040dc0 0x55c1e00544e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:39.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.811+0000 7fca90dd9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c1e0040dc0 0x55c1e00544e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52018/0 (socket says 172.21.15.16:52018) 2024-09-12T02:03:39.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.811+0000 7fca90dd9640 1 -- 172.21.15.16:0/1688494656 learned_addr learned my addr 172.21.15.16:0/1688494656 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:03:39.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.811+0000 7fca8bfff640 1 --2- 172.21.15.16:0/1688494656 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c1e003ca30 0x55c1e0053a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:39.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.811+0000 7fca8b7fe640 1 --2- 172.21.15.16:0/1688494656 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c1e00404a0 0x55c1e0053fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:39.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.811+0000 7fca90dd9640 1 -- 172.21.15.16:0/1688494656 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c1e003ca30 msgr2=0x55c1e0053a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:39.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.811+0000 7fca90dd9640 1 --2- 172.21.15.16:0/1688494656 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c1e003ca30 0x55c1e0053a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:39.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.811+0000 7fca90dd9640 1 -- 172.21.15.16:0/1688494656 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c1e00404a0 msgr2=0x55c1e0053fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:39.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.811+0000 7fca90dd9640 1 --2- 172.21.15.16:0/1688494656 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c1e00404a0 0x55c1e0053fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:39.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.811+0000 7fca90dd9640 1 -- 172.21.15.16:0/1688494656 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c1e00c9390 con 0x55c1e0040dc0 2024-09-12T02:03:39.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.812+0000 7fca90dd9640 1 --2- 172.21.15.16:0/1688494656 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c1e0040dc0 0x55c1e00544e0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fca84031bb0 tx=0x7fca84030790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5028e2a997cf9a51 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:39.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.812+0000 7fca6b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:03:39.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.812+0000 7fca6b7fe640 1 -- 172.21.15.16:0/1688494656 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca84064030 con 0x55c1e0040dc0 2024-09-12T02:03:39.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.812+0000 7fca6b7fe640 1 -- 172.21.15.16:0/1688494656 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fca84031580 con 0x55c1e0040dc0 2024-09-12T02:03:39.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.812+0000 7fca94c15740 1 -- 172.21.15.16:0/1688494656 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca84036070 con 0x55c1e0040dc0 2024-09-12T02:03:39.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.812+0000 7fca6b7fe640 1 -- 172.21.15.16:0/1688494656 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fca840305a0 con 0x55c1e0040dc0 2024-09-12T02:03:39.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.812+0000 7fca94c15740 10 client.15951 fetch_fsmap learned FSMap version 25 2024-09-12T02:03:39.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.812+0000 7fca94c15740 1 -- 172.21.15.16:0/1688494656 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c1e00ca660 con 0x55c1e0040dc0 2024-09-12T02:03:39.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.812+0000 7fca6b7fe640 1 -- 172.21.15.16:0/1688494656 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca84036070 con 0x55c1e0040dc0 2024-09-12T02:03:39.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.812+0000 7fca6b7fe640 1 -- 172.21.15.16:0/1688494656 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fca84030070 con 0x55c1e0040dc0 2024-09-12T02:03:39.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.812+0000 7fca94c15740 10 client.15951 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:03:39.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.812+0000 7fca94c15740 10 client.15951 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:03:39.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.812+0000 7fca94c15740 20 client.15951 populate_metadata read hostname 'smithi016' 2024-09-12T02:03:39.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.812+0000 7fca94c15740 1 --2- 172.21.15.16:0/1688494656 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55c1e004b680 0x55c1e00cc8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:39.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.812+0000 7fca94c15740 4 client.15951 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T02:03:39.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.812+0000 7fca94c15740 1 -- 172.21.15.16:0/1688494656 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c1dffadd60 con 0x55c1e004b680 2024-09-12T02:03:39.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.813+0000 7fca8bfff640 1 --2- 172.21.15.16:0/1688494656 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55c1e004b680 0x55c1e00cc8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:39.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.813+0000 7fca8bfff640 1 --2- 172.21.15.16:0/1688494656 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55c1e004b680 0x55c1e00cc8f0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e12df62b9c6573f server_cookie=714a328d37528035 in_seq=0 out_seq=0 2024-09-12T02:03:39.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.813+0000 7fca6b7fe640 10 client.15951 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:03:39.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.813+0000 7fca6b7fe640 1 -- 172.21.15.16:0/1688494656 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55c1dffadd60 con 0x55c1e004b680 2024-09-12T02:03:39.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.813+0000 7fca6b7fe640 10 client.15951 handle_command_reply: tid=0 2024-09-12T02:03:39.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.813+0000 7fca94c15740 1 client.15951 shutdown 2024-09-12T02:03:39.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.813+0000 7fca94c15740 1 -- 172.21.15.16:0/1688494656 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c1e0040dc0 msgr2=0x55c1e00544e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:39.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.813+0000 7fca94c15740 1 --2- 172.21.15.16:0/1688494656 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c1e0040dc0 0x55c1e00544e0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fca84031bb0 tx=0x7fca84030790 comp rx=0 tx=0).stop 2024-09-12T02:03:39.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.814+0000 7fca94c15740 1 -- 172.21.15.16:0/1688494656 shutdown_connections 2024-09-12T02:03:39.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.814+0000 7fca94c15740 1 --2- 172.21.15.16:0/1688494656 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55c1e004b680 0x55c1e00cc8f0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:39.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.814+0000 7fca94c15740 1 --2- 172.21.15.16:0/1688494656 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c1e0040dc0 0x55c1e00544e0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:39.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.814+0000 7fca94c15740 1 --2- 172.21.15.16:0/1688494656 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c1e00404a0 0x55c1e0053fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:39.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.814+0000 7fca94c15740 1 --2- 172.21.15.16:0/1688494656 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c1e003ca30 0x55c1e0053a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:39.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.814+0000 7fca94c15740 1 -- 172.21.15.16:0/1688494656 >> 172.21.15.16:0/1688494656 conn(0x55c1e00488a0 msgr2=0x55c1e003bb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:39.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.814+0000 7fca6b7fe640 0 client.15951 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:03:39.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.814+0000 7fca94c15740 1 -- 172.21.15.16:0/1688494656 shutdown_connections 2024-09-12T02:03:39.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.814+0000 7fca94c15740 1 -- 172.21.15.16:0/1688494656 wait complete. 2024-09-12T02:03:39.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.814+0000 7fca94c15740 20 client.15951 trim_cache size 0 max 16384 2024-09-12T02:03:39.820 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T02:03:39.820 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-12T02:03:39.820 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": { 2024-09-12T02:03:39.820 INFO:teuthology.orchestra.run.smithi016.stdout: "685101fe-bd5a-4995-8286-22d62349bfc3": { 2024-09-12T02:03:39.820 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-12T02:03:39.820 INFO:teuthology.orchestra.run.smithi016.stdout: "tag": "685101fe-bd5a-4995-8286-22d62349bfc3", 2024-09-12T02:03:39.820 INFO:teuthology.orchestra.run.smithi016.stdout: "options": "recursive,force" 2024-09-12T02:03:39.820 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:03:39.820 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:03:39.821 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T02:03:39.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.814+0000 7fca9283c640 1 -- 172.21.15.16:0/1518908889 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fca740a2b80 msgr2=0x7fca740a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:39.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.814+0000 7fca9283c640 1 --2- 172.21.15.16:0/1518908889 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fca740a2b80 0x7fca740a5040 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fca8404d000 tx=0x7fca8404b000 comp rx=0 tx=0).stop 2024-09-12T02:03:39.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.814+0000 7fca9283c640 1 -- 172.21.15.16:0/1518908889 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca8c101420 msgr2=0x7fca8c199830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:39.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.814+0000 7fca9283c640 1 --2- 172.21.15.16:0/1518908889 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca8c101420 0x7fca8c199830 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fca8000d950 tx=0x7fca8000de10 comp rx=0 tx=0).stop 2024-09-12T02:03:39.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.814+0000 7fca9283c640 1 -- 172.21.15.16:0/1518908889 shutdown_connections 2024-09-12T02:03:39.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.814+0000 7fca9283c640 1 --2- 172.21.15.16:0/1518908889 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fca740a2b80 0x7fca740a5040 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:39.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.814+0000 7fca9283c640 1 --2- 172.21.15.16:0/1518908889 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fca8c106a80 0x7fca8c1a4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:39.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.814+0000 7fca9283c640 1 --2- 172.21.15.16:0/1518908889 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca8c101420 0x7fca8c199830 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:39.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.814+0000 7fca9283c640 1 --2- 172.21.15.16:0/1518908889 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca8c100a80 0x7fca8c1992f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:39.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.814+0000 7fca9283c640 1 -- 172.21.15.16:0/1518908889 >> 172.21.15.16:0/1518908889 conn(0x7fca8c0fc7a0 msgr2=0x7fca8c0fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:39.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.815+0000 7fca9283c640 1 -- 172.21.15.16:0/1518908889 shutdown_connections 2024-09-12T02:03:39.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:39.815+0000 7fca9283c640 1 -- 172.21.15.16:0/1518908889 wait complete. 2024-09-12T02:03:39.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:03:39 smithi184 ceph-mon[35593]: pgmap v387: 97 pgs: 97 active+clean; 1.1 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-09-12T02:03:39.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:03:39 smithi184 ceph-mon[35593]: scrub queued for path: / 2024-09-12T02:03:39.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:03:39 smithi184 ceph-mon[35593]: scrub summary: idle+waiting paths [/] 2024-09-12T02:03:39.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:03:39 smithi184 ceph-mon[35593]: scrub summary: active paths [/] 2024-09-12T02:03:39.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:03:39 smithi184 ceph-mon[35593]: scrub summary: idle+waiting paths [/] 2024-09-12T02:03:39.918 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:03:39 smithi033 ceph-mon[35332]: pgmap v387: 97 pgs: 97 active+clean; 1.1 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-09-12T02:03:39.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:03:39 smithi033 ceph-mon[35332]: scrub queued for path: / 2024-09-12T02:03:39.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:03:39 smithi033 ceph-mon[35332]: scrub summary: idle+waiting paths [/] 2024-09-12T02:03:39.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:03:39 smithi033 ceph-mon[35332]: scrub summary: active paths [/] 2024-09-12T02:03:39.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:03:39 smithi033 ceph-mon[35332]: scrub summary: idle+waiting paths [/] 2024-09-12T02:03:40.505 INFO:tasks.cephfs.filesystem:scrub status for tag:685101fe-bd5a-4995-8286-22d62349bfc3 - {'path': '/', 'tag': '685101fe-bd5a-4995-8286-22d62349bfc3', 'options': 'recursive,force'} 2024-09-12T02:03:41.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:03:41 smithi184 ceph-mon[35593]: pgmap v388: 97 pgs: 97 active+clean; 1.1 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-12T02:03:41.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:03:41 smithi184 ceph-mon[35593]: scrub summary: idle 2024-09-12T02:03:41.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:03:41 smithi033 ceph-mon[35332]: pgmap v388: 97 pgs: 97 active+clean; 1.1 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-12T02:03:41.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:03:41 smithi033 ceph-mon[35332]: scrub summary: idle 2024-09-12T02:03:41.933 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:03:41 smithi016 ceph-mon[30947]: pgmap v388: 97 pgs: 97 active+clean; 1.1 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-12T02:03:41.933 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:03:41 smithi016 ceph-mon[30947]: scrub summary: idle 2024-09-12T02:03:42.813 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.813+0000 7eff08eae640 1 -- 172.21.15.16:0/2158837545 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff041025e0 msgr2=0x7eff041049d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:42.813 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.813+0000 7eff08eae640 1 --2- 172.21.15.16:0/2158837545 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff041025e0 0x7eff041049d0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7efef4009900 tx=0x7efef402a410 comp rx=0 tx=0).stop 2024-09-12T02:03:42.813 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.813+0000 7eff08eae640 1 -- 172.21.15.16:0/2158837545 shutdown_connections 2024-09-12T02:03:42.813 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.813+0000 7eff08eae640 1 --2- 172.21.15.16:0/2158837545 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7eff04107840 0x7eff04109cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:42.814 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.813+0000 7eff08eae640 1 --2- 172.21.15.16:0/2158837545 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7eff04104f10 0x7eff04107300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:42.814 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.813+0000 7eff08eae640 1 --2- 172.21.15.16:0/2158837545 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff041025e0 0x7eff041049d0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:42.814 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.813+0000 7eff08eae640 1 -- 172.21.15.16:0/2158837545 >> 172.21.15.16:0/2158837545 conn(0x7eff040fc1e0 msgr2=0x7eff040fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:42.814 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.813+0000 7eff08eae640 1 -- 172.21.15.16:0/2158837545 shutdown_connections 2024-09-12T02:03:42.814 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.813+0000 7eff08eae640 1 -- 172.21.15.16:0/2158837545 wait complete. 2024-09-12T02:03:42.816 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.814+0000 7eff08eae640 1 Processor -- start 2024-09-12T02:03:42.816 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.814+0000 7eff08eae640 1 -- start start 2024-09-12T02:03:42.816 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.814+0000 7eff08eae640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7eff041025e0 0x7eff04101f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:42.816 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.814+0000 7eff08eae640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7eff04104f10 0x7eff04100590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:42.816 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.814+0000 7eff08eae640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff04107840 0x7eff04100ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:42.816 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.814+0000 7eff08eae640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7eff0410d850 con 0x7eff04107840 2024-09-12T02:03:42.817 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.814+0000 7eff08eae640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7eff0410d6d0 con 0x7eff04104f10 2024-09-12T02:03:42.817 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.814+0000 7eff08eae640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7eff0410d9d0 con 0x7eff041025e0 2024-09-12T02:03:42.817 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.814+0000 7eff02d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff04107840 0x7eff04100ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:42.817 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.814+0000 7eff02d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff04107840 0x7eff04100ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50792/0 (socket says 172.21.15.16:50792) 2024-09-12T02:03:42.817 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.814+0000 7eff02d9d640 1 -- 172.21.15.16:0/3054537174 learned_addr learned my addr 172.21.15.16:0/3054537174 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:03:42.817 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.814+0000 7eff0259c640 1 --2- 172.21.15.16:0/3054537174 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7eff041025e0 0x7eff04101f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:42.817 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.814+0000 7eff01d9b640 1 --2- 172.21.15.16:0/3054537174 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7eff04104f10 0x7eff04100590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:42.817 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.815+0000 7eff02d9d640 1 -- 172.21.15.16:0/3054537174 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7eff041025e0 msgr2=0x7eff04101f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:42.817 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.815+0000 7eff02d9d640 1 --2- 172.21.15.16:0/3054537174 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7eff041025e0 0x7eff04101f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:42.817 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.815+0000 7eff02d9d640 1 -- 172.21.15.16:0/3054537174 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7eff04104f10 msgr2=0x7eff04100590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:42.817 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.815+0000 7eff02d9d640 1 --2- 172.21.15.16:0/3054537174 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7eff04104f10 0x7eff04100590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:42.817 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.815+0000 7eff02d9d640 1 -- 172.21.15.16:0/3054537174 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff04101360 con 0x7eff04107840 2024-09-12T02:03:42.818 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.815+0000 7eff01d9b640 1 --2- 172.21.15.16:0/3054537174 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7eff04104f10 0x7eff04100590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:03:42.818 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.815+0000 7eff02d9d640 1 --2- 172.21.15.16:0/3054537174 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff04107840 0x7eff04100ad0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7efefc00e9d0 tx=0x7efefc00ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:42.818 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.815+0000 7efef37fe640 1 -- 172.21.15.16:0/3054537174 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efefc00cd80 con 0x7eff04107840 2024-09-12T02:03:42.818 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.815+0000 7efef37fe640 1 -- 172.21.15.16:0/3054537174 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efefc005020 con 0x7eff04107840 2024-09-12T02:03:42.818 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.815+0000 7efef37fe640 1 -- 172.21.15.16:0/3054537174 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efefc0116d0 con 0x7eff04107840 2024-09-12T02:03:42.818 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.815+0000 7eff08eae640 1 -- 172.21.15.16:0/3054537174 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff041a1f90 con 0x7eff04107840 2024-09-12T02:03:42.818 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.815+0000 7eff08eae640 1 -- 172.21.15.16:0/3054537174 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff041a2450 con 0x7eff04107840 2024-09-12T02:03:42.823 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.818+0000 7efef37fe640 1 -- 172.21.15.16:0/3054537174 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7efefc0051d0 con 0x7eff04107840 2024-09-12T02:03:42.823 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.818+0000 7eff0bc48740 1 Processor -- start 2024-09-12T02:03:42.824 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.818+0000 7eff0bc48740 1 -- start start 2024-09-12T02:03:42.824 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.818+0000 7eff0bc48740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55988112c9a0 0x55988112cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:42.824 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.818+0000 7eff0bc48740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55988112d2e0 0x559881131da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:42.824 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.818+0000 7eff0bc48740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559881132370 0x559881134830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:42.824 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.818+0000 7eff0bc48740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5598810902f0 con 0x55988112c9a0 2024-09-12T02:03:42.824 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.818+0000 7eff0bc48740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x559881038270 con 0x55988112d2e0 2024-09-12T02:03:42.824 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.818+0000 7eff0bc48740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55988108c2b0 con 0x559881132370 2024-09-12T02:03:42.824 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.819+0000 7eff01d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55988112c9a0 0x55988112cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:42.824 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.819+0000 7eff01d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55988112c9a0 0x55988112cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50802/0 (socket says 172.21.15.16:50802) 2024-09-12T02:03:42.824 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.819+0000 7eff01d9b640 1 -- 172.21.15.16:0/2814862749 learned_addr learned my addr 172.21.15.16:0/2814862749 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:03:42.824 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.819+0000 7eff0259c640 1 --2- 172.21.15.16:0/2814862749 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55988112d2e0 0x559881131da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:42.824 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.819+0000 7eff01d9b640 1 --2- 172.21.15.16:0/2814862749 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559881132370 0x559881134830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:42.825 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.819+0000 7efef37fe640 1 --2- 172.21.15.16:0/3054537174 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7efeec0a2cb0 0x7efeec0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:42.825 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.819+0000 7eff01d9b640 1 -- 172.21.15.16:0/2814862749 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559881132370 msgr2=0x559881134830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:42.825 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.819+0000 7eff01d9b640 1 --2- 172.21.15.16:0/2814862749 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559881132370 0x559881134830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:42.825 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.819+0000 7eff01d9b640 1 -- 172.21.15.16:0/2814862749 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55988112d2e0 msgr2=0x559881131da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:42.825 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.819+0000 7eff01d9b640 1 --2- 172.21.15.16:0/2814862749 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55988112d2e0 0x559881131da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:42.825 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.819+0000 7eff01d9b640 1 -- 172.21.15.16:0/2814862749 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559881037340 con 0x55988112c9a0 2024-09-12T02:03:42.825 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.819+0000 7eff02d9d640 1 --2- 172.21.15.16:0/3054537174 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7efeec0a2cb0 0x7efeec0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:42.825 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.819+0000 7eff01d9b640 1 --2- 172.21.15.16:0/2814862749 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55988112c9a0 0x55988112cda0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7efef8014490 tx=0x7efef80096a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af2e3e16cd267888 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:42.825 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.819+0000 7efef37fe640 1 -- 172.21.15.16:0/3054537174 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7efefc0cf510 con 0x7eff04107840 2024-09-12T02:03:42.825 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.819+0000 7efef0ff9640 1 -- 172.21.15.16:0/2814862749 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efef801a030 con 0x55988112c9a0 2024-09-12T02:03:42.825 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.819+0000 7efef0ff9640 1 -- 172.21.15.16:0/2814862749 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efef8003610 con 0x55988112c9a0 2024-09-12T02:03:42.825 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.819+0000 7efef0ff9640 1 -- 172.21.15.16:0/2814862749 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efef80126b0 con 0x55988112c9a0 2024-09-12T02:03:42.825 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.820+0000 7eff0bc48740 1 -- 172.21.15.16:0/2814862749 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55988112c9a0 msgr2=0x55988112cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:42.826 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.820+0000 7eff02d9d640 1 --2- 172.21.15.16:0/3054537174 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7efeec0a2cb0 0x7efeec0a5170 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7efefc009fd0 tx=0x7efefc066e80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:42.826 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.820+0000 7eff0bc48740 1 --2- 172.21.15.16:0/2814862749 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55988112c9a0 0x55988112cda0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7efef8014490 tx=0x7efef80096a0 comp rx=0 tx=0).stop 2024-09-12T02:03:42.826 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.820+0000 7eff0bc48740 1 -- 172.21.15.16:0/2814862749 shutdown_connections 2024-09-12T02:03:42.826 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.820+0000 7eff0bc48740 1 --2- 172.21.15.16:0/2814862749 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559881132370 0x559881134830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:42.826 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.820+0000 7eff0bc48740 1 --2- 172.21.15.16:0/2814862749 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55988112d2e0 0x559881131da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:42.826 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.820+0000 7eff0bc48740 1 --2- 172.21.15.16:0/2814862749 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55988112c9a0 0x55988112cda0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:42.826 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.820+0000 7eff0bc48740 1 -- 172.21.15.16:0/2814862749 >> 172.21.15.16:0/2814862749 conn(0x559881128010 msgr2=0x559881128410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:42.826 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.820+0000 7eff0bc48740 1 -- 172.21.15.16:0/2814862749 shutdown_connections 2024-09-12T02:03:42.826 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.820+0000 7eff0bc48740 1 -- 172.21.15.16:0/2814862749 wait complete. 2024-09-12T02:03:42.826 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.821+0000 7eff0bc48740 1 -- start start 2024-09-12T02:03:42.826 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.821+0000 7eff0bc48740 1 Processor -- start 2024-09-12T02:03:42.826 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.821+0000 7eff0bc48740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55988112c9a0 0x559881136b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:42.826 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.821+0000 7eff0bc48740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55988112d2e0 0x559881137070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:42.827 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.821+0000 7eff0bc48740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559881132370 0x55988113e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:42.827 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.821+0000 7eff0bc48740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x559881135340 con 0x55988112c9a0 2024-09-12T02:03:42.827 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.821+0000 7eff0bc48740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x559881019290 con 0x559881132370 2024-09-12T02:03:42.827 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.821+0000 7eff0bc48740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5598811354c0 con 0x55988112d2e0 2024-09-12T02:03:42.827 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.821+0000 7eff0bc48740 10 client.0 _ll_register_callbacks cb 0x55988104a140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:03:42.827 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.821+0000 7eff01d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55988112c9a0 0x559881136b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:42.827 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.821+0000 7eff01d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55988112c9a0 0x559881136b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50816/0 (socket says 172.21.15.16:50816) 2024-09-12T02:03:42.827 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.821+0000 7eff01d9b640 1 -- 172.21.15.16:0/174554778 learned_addr learned my addr 172.21.15.16:0/174554778 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:03:42.827 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.821+0000 7eff0259c640 1 --2- 172.21.15.16:0/174554778 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55988112d2e0 0x559881137070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:42.827 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.821+0000 7eff01d9b640 1 --2- 172.21.15.16:0/174554778 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559881132370 0x55988113e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:42.827 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.821+0000 7eff01d9b640 1 -- 172.21.15.16:0/174554778 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55988112d2e0 msgr2=0x559881137070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:42.828 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.821+0000 7eff01d9b640 1 --2- 172.21.15.16:0/174554778 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55988112d2e0 0x559881137070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:42.828 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.821+0000 7eff01d9b640 1 -- 172.21.15.16:0/174554778 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559881132370 msgr2=0x55988113e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:42.828 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.821+0000 7eff01d9b640 1 --2- 172.21.15.16:0/174554778 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559881132370 0x55988113e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:42.828 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.822+0000 7eff01d9b640 1 -- 172.21.15.16:0/174554778 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559880da3210 con 0x55988112c9a0 2024-09-12T02:03:42.828 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.822+0000 7eff01d9b640 1 --2- 172.21.15.16:0/174554778 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55988112c9a0 0x559881136b30 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7efef80098f0 tx=0x7efef80093e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc01023efc7444f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:42.828 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.822+0000 7efee9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:03:42.828 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.822+0000 7efee9ffb640 1 -- 172.21.15.16:0/174554778 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efef801a070 con 0x55988112c9a0 2024-09-12T02:03:42.828 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.822+0000 7efee9ffb640 1 -- 172.21.15.16:0/174554778 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7efef8005250 con 0x55988112c9a0 2024-09-12T02:03:42.828 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.822+0000 7eff0bc48740 1 -- 172.21.15.16:0/174554778 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efef8003610 con 0x55988112c9a0 2024-09-12T02:03:42.828 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.822+0000 7efee9ffb640 1 -- 172.21.15.16:0/174554778 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7efef80044e0 con 0x55988112c9a0 2024-09-12T02:03:42.828 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.822+0000 7eff0bc48740 10 client.15963 fetch_fsmap learned FSMap version 25 2024-09-12T02:03:42.828 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.822+0000 7eff0bc48740 1 -- 172.21.15.16:0/174554778 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x559880e6f620 con 0x55988112c9a0 2024-09-12T02:03:42.828 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.822+0000 7efee9ffb640 1 -- 172.21.15.16:0/174554778 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efef8003610 con 0x55988112c9a0 2024-09-12T02:03:42.829 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.822+0000 7efee9ffb640 1 -- 172.21.15.16:0/174554778 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7efef802d060 con 0x55988112c9a0 2024-09-12T02:03:42.829 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.823+0000 7eff0bc48740 10 client.15963 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:03:42.829 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.823+0000 7eff0bc48740 10 client.15963 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:03:42.829 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.823+0000 7eff0bc48740 20 client.15963 populate_metadata read hostname 'smithi016' 2024-09-12T02:03:42.829 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.823+0000 7eff0bc48740 1 --2- 172.21.15.16:0/174554778 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5598811425c0 0x5598811449d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:42.829 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.823+0000 7eff0bc48740 4 client.15963 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:03:42.829 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.823+0000 7eff0bc48740 1 -- 172.21.15.16:0/174554778 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55988109da20 con 0x5598811425c0 2024-09-12T02:03:42.829 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.823+0000 7eff0259c640 1 --2- 172.21.15.16:0/174554778 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5598811425c0 0x5598811449d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:42.829 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.823+0000 7eff0259c640 1 --2- 172.21.15.16:0/174554778 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5598811425c0 0x5598811449d0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1a3388eff22dd73 server_cookie=2c0bfc9e5dbcd1cc in_seq=0 out_seq=0 2024-09-12T02:03:42.829 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.823+0000 7efee9ffb640 10 client.15963 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:03:42.831 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.824+0000 7efee9ffb640 1 -- 172.21.15.16:0/174554778 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55988109da20 con 0x5598811425c0 2024-09-12T02:03:42.831 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.824+0000 7efee9ffb640 10 client.15963 handle_command_reply: tid=0 2024-09-12T02:03:42.831 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.824+0000 7eff0bc48740 1 client.15963 shutdown 2024-09-12T02:03:42.831 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.824+0000 7eff0bc48740 1 -- 172.21.15.16:0/174554778 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55988112c9a0 msgr2=0x559881136b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:42.831 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.824+0000 7eff0bc48740 1 --2- 172.21.15.16:0/174554778 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55988112c9a0 0x559881136b30 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7efef80098f0 tx=0x7efef80093e0 comp rx=0 tx=0).stop 2024-09-12T02:03:42.831 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.824+0000 7eff0bc48740 1 -- 172.21.15.16:0/174554778 shutdown_connections 2024-09-12T02:03:42.831 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.824+0000 7eff0bc48740 1 --2- 172.21.15.16:0/174554778 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5598811425c0 0x5598811449d0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:42.831 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.824+0000 7eff0bc48740 1 --2- 172.21.15.16:0/174554778 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559881132370 0x55988113e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:42.831 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.824+0000 7eff0bc48740 1 --2- 172.21.15.16:0/174554778 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55988112d2e0 0x559881137070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:42.831 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.824+0000 7efee9ffb640 0 client.15963 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:03:42.831 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.824+0000 7eff0bc48740 1 --2- 172.21.15.16:0/174554778 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55988112c9a0 0x559881136b30 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:42.831 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.824+0000 7eff0bc48740 1 -- 172.21.15.16:0/174554778 >> 172.21.15.16:0/174554778 conn(0x559881128010 msgr2=0x559881129820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:42.832 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.824+0000 7eff0bc48740 1 -- 172.21.15.16:0/174554778 shutdown_connections 2024-09-12T02:03:42.832 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.824+0000 7eff0bc48740 1 -- 172.21.15.16:0/174554778 wait complete. 2024-09-12T02:03:42.832 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.825+0000 7eff0bc48740 20 client.15963 trim_cache size 0 max 16384 2024-09-12T02:03:42.844 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.839+0000 7eff0bc48740 1 Processor -- start 2024-09-12T02:03:42.844 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.839+0000 7eff0bc48740 1 -- start start 2024-09-12T02:03:42.844 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.839+0000 7eff0bc48740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55988112c9a0 0x5598811368e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:42.844 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.839+0000 7eff0bc48740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55988112d2e0 0x55988113e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:42.844 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.839+0000 7eff0bc48740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559881132370 0x559881136e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:42.844 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.839+0000 7eff0bc48740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7eff041ada80 con 0x55988112d2e0 2024-09-12T02:03:42.844 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.839+0000 7eff0bc48740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7eff041ab350 con 0x55988112c9a0 2024-09-12T02:03:42.844 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.839+0000 7eff0bc48740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7eff041a91a0 con 0x559881132370 2024-09-12T02:03:42.844 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.840+0000 7eff0259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55988112d2e0 0x55988113e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:42.845 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.840+0000 7eff0259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55988112d2e0 0x55988113e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50824/0 (socket says 172.21.15.16:50824) 2024-09-12T02:03:42.845 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.840+0000 7eff0259c640 1 -- 172.21.15.16:0/3555268862 learned_addr learned my addr 172.21.15.16:0/3555268862 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:03:42.845 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.840+0000 7eff01d9b640 1 --2- 172.21.15.16:0/3555268862 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55988112c9a0 0x5598811368e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:42.845 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.840+0000 7eff0259c640 1 -- 172.21.15.16:0/3555268862 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559881132370 msgr2=0x559881136e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:42.845 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.840+0000 7eff0259c640 1 --2- 172.21.15.16:0/3555268862 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559881132370 0x559881136e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:42.845 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.840+0000 7eff0259c640 1 -- 172.21.15.16:0/3555268862 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55988112c9a0 msgr2=0x5598811368e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:42.845 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.840+0000 7eff0259c640 1 --2- 172.21.15.16:0/3555268862 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55988112c9a0 0x5598811368e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:42.845 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.840+0000 7eff0259c640 1 -- 172.21.15.16:0/3555268862 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559881136670 con 0x55988112d2e0 2024-09-12T02:03:42.845 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.840+0000 7eff0259c640 1 --2- 172.21.15.16:0/3555268862 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55988112d2e0 0x55988113e020 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7efef402a930 tx=0x7efef402a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73904e9c3ff28d86 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:42.845 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.840+0000 7efeebfff640 1 -- 172.21.15.16:0/3555268862 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efef4035040 con 0x55988112d2e0 2024-09-12T02:03:42.845 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.840+0000 7efeebfff640 1 -- 172.21.15.16:0/3555268862 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efef4005530 con 0x55988112d2e0 2024-09-12T02:03:42.845 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.840+0000 7efeebfff640 1 -- 172.21.15.16:0/3555268862 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efef4031910 con 0x55988112d2e0 2024-09-12T02:03:42.845 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.841+0000 7eff0bc48740 1 -- 172.21.15.16:0/3555268862 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55988112d2e0 msgr2=0x55988113e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:42.846 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.841+0000 7eff0bc48740 1 --2- 172.21.15.16:0/3555268862 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55988112d2e0 0x55988113e020 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7efef402a930 tx=0x7efef402a410 comp rx=0 tx=0).stop 2024-09-12T02:03:42.846 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.841+0000 7eff0bc48740 1 -- 172.21.15.16:0/3555268862 shutdown_connections 2024-09-12T02:03:42.846 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.841+0000 7eff0bc48740 1 --2- 172.21.15.16:0/3555268862 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559881132370 0x559881136e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:42.846 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.841+0000 7eff0bc48740 1 --2- 172.21.15.16:0/3555268862 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55988112d2e0 0x55988113e020 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:42.846 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.841+0000 7eff0bc48740 1 --2- 172.21.15.16:0/3555268862 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55988112c9a0 0x5598811368e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:42.846 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.841+0000 7eff0bc48740 1 -- 172.21.15.16:0/3555268862 >> 172.21.15.16:0/3555268862 conn(0x559881128010 msgr2=0x55988112e450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:42.846 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.841+0000 7eff0bc48740 1 -- 172.21.15.16:0/3555268862 shutdown_connections 2024-09-12T02:03:42.846 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.841+0000 7eff0bc48740 1 -- 172.21.15.16:0/3555268862 wait complete. 2024-09-12T02:03:42.846 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.841+0000 7eff0bc48740 1 -- start start 2024-09-12T02:03:42.846 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.841+0000 7eff0bc48740 1 Processor -- start 2024-09-12T02:03:42.846 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.842+0000 7eff0bc48740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55988112c9a0 0x55988113b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:42.846 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.842+0000 7eff0bc48740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55988112d2e0 0x55988113b860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:42.847 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.842+0000 7eff0bc48740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559881132370 0x55988113c870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:42.847 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.842+0000 7eff0bc48740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7eff041a9810 con 0x55988112d2e0 2024-09-12T02:03:42.847 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.842+0000 7eff0bc48740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7eff041aac80 con 0x559881132370 2024-09-12T02:03:42.847 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.842+0000 7eff0bc48740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7eff041a9ee0 con 0x55988112c9a0 2024-09-12T02:03:42.847 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.842+0000 7eff0bc48740 10 client.0 _ll_register_callbacks cb 0x7efef8005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:03:42.847 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.842+0000 7eff0259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55988112d2e0 0x55988113b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:42.847 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.842+0000 7eff0259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55988112d2e0 0x55988113b860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50830/0 (socket says 172.21.15.16:50830) 2024-09-12T02:03:42.847 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.842+0000 7eff0259c640 1 -- 172.21.15.16:0/2879713063 learned_addr learned my addr 172.21.15.16:0/2879713063 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:03:42.847 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.842+0000 7eff01d9b640 1 --2- 172.21.15.16:0/2879713063 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559881132370 0x55988113c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:42.847 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.842+0000 7eff01d9b640 1 --2- 172.21.15.16:0/2879713063 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55988112c9a0 0x55988113b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:42.847 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.842+0000 7eff0259c640 1 -- 172.21.15.16:0/2879713063 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55988112c9a0 msgr2=0x55988113b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:42.847 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.842+0000 7eff0259c640 1 --2- 172.21.15.16:0/2879713063 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55988112c9a0 0x55988113b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:42.847 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.842+0000 7eff0259c640 1 -- 172.21.15.16:0/2879713063 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559881132370 msgr2=0x55988113c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:42.847 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.842+0000 7eff0259c640 1 --2- 172.21.15.16:0/2879713063 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559881132370 0x55988113c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:42.847 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.842+0000 7eff0259c640 1 -- 172.21.15.16:0/2879713063 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55988113ce80 con 0x55988112d2e0 2024-09-12T02:03:42.848 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.842+0000 7eff01d9b640 1 --2- 172.21.15.16:0/2879713063 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55988112c9a0 0x55988113b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:03:42.848 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.842+0000 7eff0259c640 1 --2- 172.21.15.16:0/2879713063 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55988112d2e0 0x55988113b860 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7efef40314e0 tx=0x7efef4004410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48a056c50ac1350a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:42.848 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.842+0000 7efee9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:03:42.848 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.842+0000 7efee9ffb640 1 -- 172.21.15.16:0/2879713063 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efef4035040 con 0x55988112d2e0 2024-09-12T02:03:42.848 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.842+0000 7efee9ffb640 1 -- 172.21.15.16:0/2879713063 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7efef40048c0 con 0x55988112d2e0 2024-09-12T02:03:42.848 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.842+0000 7eff0bc48740 1 -- 172.21.15.16:0/2879713063 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efef4005530 con 0x55988112d2e0 2024-09-12T02:03:42.848 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.843+0000 7efee9ffb640 1 -- 172.21.15.16:0/2879713063 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7efef4004ac0 con 0x55988112d2e0 2024-09-12T02:03:42.848 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.843+0000 7eff0bc48740 10 client.15969 fetch_fsmap learned FSMap version 25 2024-09-12T02:03:42.848 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.843+0000 7eff0bc48740 1 -- 172.21.15.16:0/2879713063 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55988113e190 con 0x55988112d2e0 2024-09-12T02:03:42.848 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.843+0000 7efee9ffb640 1 -- 172.21.15.16:0/2879713063 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efef4005530 con 0x55988112d2e0 2024-09-12T02:03:42.848 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.843+0000 7efee9ffb640 1 -- 172.21.15.16:0/2879713063 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7efef404e820 con 0x55988112d2e0 2024-09-12T02:03:42.848 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.843+0000 7eff0bc48740 10 client.15969 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:03:42.848 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.843+0000 7eff0bc48740 10 client.15969 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:03:42.848 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.843+0000 7eff0bc48740 20 client.15969 populate_metadata read hostname 'smithi016' 2024-09-12T02:03:42.849 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.843+0000 7eff0bc48740 1 --2- 172.21.15.16:0/2879713063 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5598811ad880 0x5598811afca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:42.849 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.843+0000 7eff0bc48740 4 client.15969 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:03:42.849 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.843+0000 7eff0bc48740 1 -- 172.21.15.16:0/2879713063 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559881097d70 con 0x5598811ad880 2024-09-12T02:03:42.849 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.843+0000 7eff01d9b640 1 --2- 172.21.15.16:0/2879713063 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5598811ad880 0x5598811afca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:42.849 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.844+0000 7efee9ffb640 10 client.15969 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:03:42.849 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.844+0000 7eff01d9b640 1 --2- 172.21.15.16:0/2879713063 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5598811ad880 0x5598811afca0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b665ab4cc034244 server_cookie=ab30ff440ecc5272 in_seq=0 out_seq=0 2024-09-12T02:03:42.857 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.856+0000 7efee9ffb640 1 -- 172.21.15.16:0/2879713063 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559881097d70 con 0x5598811ad880 2024-09-12T02:03:42.857 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.856+0000 7efee9ffb640 10 client.15969 handle_command_reply: tid=0 2024-09-12T02:03:42.857 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.856+0000 7eff0bc48740 1 client.15969 shutdown 2024-09-12T02:03:42.857 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.856+0000 7eff0bc48740 1 -- 172.21.15.16:0/2879713063 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55988112d2e0 msgr2=0x55988113b860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:42.857 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.856+0000 7eff0bc48740 1 --2- 172.21.15.16:0/2879713063 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55988112d2e0 0x55988113b860 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7efef40314e0 tx=0x7efef4004410 comp rx=0 tx=0).stop 2024-09-12T02:03:42.857 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.856+0000 7eff0bc48740 1 -- 172.21.15.16:0/2879713063 shutdown_connections 2024-09-12T02:03:42.857 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.856+0000 7eff0bc48740 1 --2- 172.21.15.16:0/2879713063 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5598811ad880 0x5598811afca0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:42.857 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.856+0000 7eff0bc48740 1 --2- 172.21.15.16:0/2879713063 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559881132370 0x55988113c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:42.857 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.856+0000 7eff0bc48740 1 --2- 172.21.15.16:0/2879713063 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55988112d2e0 0x55988113b860 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:42.858 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.856+0000 7eff0bc48740 1 --2- 172.21.15.16:0/2879713063 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55988112c9a0 0x55988113b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:42.858 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.856+0000 7efee9ffb640 0 client.15969 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:03:42.858 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.856+0000 7eff0bc48740 1 -- 172.21.15.16:0/2879713063 >> 172.21.15.16:0/2879713063 conn(0x559881128010 msgr2=0x5598811368e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:42.858 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.857+0000 7eff0bc48740 1 -- 172.21.15.16:0/2879713063 shutdown_connections 2024-09-12T02:03:42.858 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.857+0000 7eff0bc48740 1 -- 172.21.15.16:0/2879713063 wait complete. 2024-09-12T02:03:42.858 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.857+0000 7eff0bc48740 20 client.15969 trim_cache size 0 max 16384 2024-09-12T02:03:42.858 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-09-12T02:03:42.858 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-09-12T02:03:42.858 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-09-12T02:03:42.858 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-09-12T02:03:42.859 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.857+0000 7eff08eae640 1 -- 172.21.15.16:0/3054537174 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7efeec0a2cb0 msgr2=0x7efeec0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:42.859 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.857+0000 7eff08eae640 1 --2- 172.21.15.16:0/3054537174 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7efeec0a2cb0 0x7efeec0a5170 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7efefc009fd0 tx=0x7efefc066e80 comp rx=0 tx=0).stop 2024-09-12T02:03:42.859 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.857+0000 7eff08eae640 1 -- 172.21.15.16:0/3054537174 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff04107840 msgr2=0x7eff04100ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:42.859 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.857+0000 7eff08eae640 1 --2- 172.21.15.16:0/3054537174 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff04107840 0x7eff04100ad0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7efefc00e9d0 tx=0x7efefc00ee90 comp rx=0 tx=0).stop 2024-09-12T02:03:42.859 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.857+0000 7eff08eae640 1 -- 172.21.15.16:0/3054537174 shutdown_connections 2024-09-12T02:03:42.860 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.857+0000 7eff08eae640 1 --2- 172.21.15.16:0/3054537174 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7efeec0a2cb0 0x7efeec0a5170 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:42.860 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.857+0000 7eff08eae640 1 --2- 172.21.15.16:0/3054537174 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff04107840 0x7eff04100ad0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:42.860 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.857+0000 7eff08eae640 1 --2- 172.21.15.16:0/3054537174 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7eff04104f10 0x7eff04100590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:42.860 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.857+0000 7eff08eae640 1 --2- 172.21.15.16:0/3054537174 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7eff041025e0 0x7eff04101f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:42.860 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.857+0000 7eff08eae640 1 -- 172.21.15.16:0/3054537174 >> 172.21.15.16:0/3054537174 conn(0x7eff040fc1e0 msgr2=0x7eff04102ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:42.860 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.857+0000 7eff08eae640 1 -- 172.21.15.16:0/3054537174 shutdown_connections 2024-09-12T02:03:42.860 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:42.857+0000 7eff08eae640 1 -- 172.21.15.16:0/3054537174 wait complete. 2024-09-12T02:03:43.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:03:43 smithi184 ceph-mon[35593]: pgmap v389: 97 pgs: 97 active+clean; 1.1 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-12T02:03:43.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:03:43 smithi033 ceph-mon[35332]: pgmap v389: 97 pgs: 97 active+clean; 1.1 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-12T02:03:43.933 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:03:43 smithi016 ceph-mon[30947]: pgmap v389: 97 pgs: 97 active+clean; 1.1 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-12T02:03:45.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:03:45 smithi184 ceph-mon[35593]: pgmap v390: 97 pgs: 97 active+clean; 1.1 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-09-12T02:03:45.918 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:03:45 smithi033 ceph-mon[35332]: pgmap v390: 97 pgs: 97 active+clean; 1.1 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-09-12T02:03:45.933 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:03:45 smithi016 ceph-mon[30947]: pgmap v390: 97 pgs: 97 active+clean; 1.1 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-09-12T02:03:46.799 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:03:47.071 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:03:47.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:47.556+0000 7f8fb2b35640 1 -- 172.21.15.16:0/1167149384 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8fac10c180 msgr2=0x7f8fac10e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:47.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:47.556+0000 7f8fb2b35640 1 --2- 172.21.15.16:0/1167149384 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8fac10c180 0x7f8fac10e600 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f8fa4009900 tx=0x7f8fa402a480 comp rx=0 tx=0).stop 2024-09-12T02:03:47.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:47.556+0000 7f8fb2b35640 1 -- 172.21.15.16:0/1167149384 shutdown_connections 2024-09-12T02:03:47.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:47.556+0000 7f8fb2b35640 1 --2- 172.21.15.16:0/1167149384 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8fac10c180 0x7f8fac10e600 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:47.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:47.556+0000 7f8fb2b35640 1 --2- 172.21.15.16:0/1167149384 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8fac1036c0 0x7f8fac10bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:47.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:47.556+0000 7f8fb2b35640 1 --2- 172.21.15.16:0/1167149384 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8fac102da0 0x7f8fac103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:47.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:47.556+0000 7f8fb2b35640 1 -- 172.21.15.16:0/1167149384 >> 172.21.15.16:0/1167149384 conn(0x7f8fac0fc7a0 msgr2=0x7f8fac0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:47.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:47.556+0000 7f8fb2b35640 1 -- 172.21.15.16:0/1167149384 shutdown_connections 2024-09-12T02:03:47.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:47.556+0000 7f8fb2b35640 1 -- 172.21.15.16:0/1167149384 wait complete. 2024-09-12T02:03:47.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:47.557+0000 7f8fb2b35640 1 Processor -- start 2024-09-12T02:03:47.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:47.557+0000 7f8fb2b35640 1 -- start start 2024-09-12T02:03:47.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:47.557+0000 7f8fb2b35640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8fac102da0 0x7f8fac1983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:47.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:47.557+0000 7f8fb2b35640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8fac1036c0 0x7f8fac1988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:47.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:47.557+0000 7f8fb2b35640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8fac10c180 0x7f8fac19cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:47.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:47.557+0000 7f8fb2b35640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8fac111f20 con 0x7f8fac102da0 2024-09-12T02:03:47.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:47.557+0000 7f8fb2b35640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8fac111da0 con 0x7f8fac1036c0 2024-09-12T02:03:47.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:47.557+0000 7f8fb2b35640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8fac1120a0 con 0x7f8fac10c180 2024-09-12T02:03:47.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:47.557+0000 7f8fb08d1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8fac102da0 0x7f8fac1983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:47.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:47.557+0000 7f8fb08d1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8fac102da0 0x7f8fac1983a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50844/0 (socket says 172.21.15.16:50844) 2024-09-12T02:03:47.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:47.557+0000 7f8fb08d1640 1 -- 172.21.15.16:0/2271645372 learned_addr learned my addr 172.21.15.16:0/2271645372 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:03:47.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:47.557+0000 7f8fabfff640 1 --2- 172.21.15.16:0/2271645372 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8fac1036c0 0x7f8fac1988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:47.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:47.558+0000 7f8fb10d2640 1 --2- 172.21.15.16:0/2271645372 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8fac10c180 0x7f8fac19cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:47.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:47.558+0000 7f8fb08d1640 1 -- 172.21.15.16:0/2271645372 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8fac10c180 msgr2=0x7f8fac19cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:47.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:47.558+0000 7f8fb08d1640 1 --2- 172.21.15.16:0/2271645372 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8fac10c180 0x7f8fac19cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:47.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:47.558+0000 7f8fb08d1640 1 -- 172.21.15.16:0/2271645372 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8fac1036c0 msgr2=0x7f8fac1988e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:47.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:47.558+0000 7f8fb08d1640 1 --2- 172.21.15.16:0/2271645372 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8fac1036c0 0x7f8fac1988e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:47.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:47.558+0000 7f8fb08d1640 1 -- 172.21.15.16:0/2271645372 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fac19d350 con 0x7f8fac102da0 2024-09-12T02:03:47.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:47.558+0000 7f8fb10d2640 1 --2- 172.21.15.16:0/2271645372 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8fac10c180 0x7f8fac19cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:03:47.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:47.558+0000 7f8fb08d1640 1 --2- 172.21.15.16:0/2271645372 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8fac102da0 0x7f8fac1983a0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f8f9400c920 tx=0x7f8f9400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:47.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:47.558+0000 7f8fa9ffb640 1 -- 172.21.15.16:0/2271645372 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f94007c70 con 0x7f8fac102da0 2024-09-12T02:03:47.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:47.558+0000 7f8fb2b35640 1 -- 172.21.15.16:0/2271645372 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fac19d640 con 0x7f8fac102da0 2024-09-12T02:03:47.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:47.558+0000 7f8fa9ffb640 1 -- 172.21.15.16:0/2271645372 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f94007e10 con 0x7f8fac102da0 2024-09-12T02:03:47.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:47.558+0000 7f8fa9ffb640 1 -- 172.21.15.16:0/2271645372 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f94018890 con 0x7f8fac102da0 2024-09-12T02:03:47.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:47.558+0000 7f8fb2b35640 1 -- 172.21.15.16:0/2271645372 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fac1a4f20 con 0x7f8fac102da0 2024-09-12T02:03:47.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:47.559+0000 7f8fb2b35640 1 -- 172.21.15.16:0/2271645372 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fac103180 con 0x7f8fac102da0 2024-09-12T02:03:47.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:47.561+0000 7f8fa9ffb640 1 -- 172.21.15.16:0/2271645372 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f8f94018a50 con 0x7f8fac102da0 2024-09-12T02:03:47.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:47.561+0000 7f8fa9ffb640 1 --2- 172.21.15.16:0/2271645372 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f8f980a2b80 0x7f8f980a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:47.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:47.561+0000 7f8fa9ffb640 1 -- 172.21.15.16:0/2271645372 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f8f940cf380 con 0x7f8fac102da0 2024-09-12T02:03:47.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:47.561+0000 7f8fabfff640 1 --2- 172.21.15.16:0/2271645372 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f8f980a2b80 0x7f8f980a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:47.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:47.562+0000 7f8fabfff640 1 --2- 172.21.15.16:0/2271645372 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f8f980a2b80 0x7f8f980a5040 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f8fac0fed50 tx=0x7f8f9c00a590 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:47.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:47.565+0000 7f8fa9ffb640 1 -- 172.21.15.16:0/2271645372 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8f94014030 con 0x7f8fac102da0 2024-09-12T02:03:47.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:47.781+0000 7f8fb2b35640 1 -- 172.21.15.16:0/2271645372 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8fac1a5150 con 0x7f8fac102da0 2024-09-12T02:03:47.782 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:03:47 smithi016 ceph-mon[30947]: pgmap v391: 97 pgs: 97 active+clean; 1.1 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 6.8 KiB/s wr, 12 op/s 2024-09-12T02:03:47.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:47.782+0000 7f8fa9ffb640 1 -- 172.21.15.16:0/2271645372 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11911 (secure 0 0 0) 0x7f8f9409c410 con 0x7f8fac102da0 2024-09-12T02:03:47.783 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T02:03:47.783 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":25,"btime":"2024-09-12T02:01:37:192021+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3279123279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3279123279},{"type":"v1","addr":"172.21.15.16:6835","nonce":3279123279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4119452382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4119452382},{"type":"v1","addr":"172.21.15.16:6837","nonce":4119452382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/3263823165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3263823165},{"type":"v1","addr":"172.21.15.16:6839","nonce":3263823165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/1510290732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":1510290732},{"type":"v1","addr":"172.21.15.16:6841","nonce":1510290732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/279674603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":279674603},{"type":"v1","addr":"172.21.15.33:6833","nonce":279674603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3555728028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3555728028},{"type":"v1","addr":"172.21.15.33:6835","nonce":3555728028}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T01:59:56.117608+0000","modified":"2024-09-12T02:01:36.294760+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24482,"mds_1":24481,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6833/1563374106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1563374106},{"type":"v1","addr":"172.21.15.184:6833","nonce":1563374106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6837/1892833868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1892833868},{"type":"v1","addr":"172.21.15.33:6837","nonce":1892833868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/1457679686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1457679686},{"type":"v1","addr":"172.21.15.184:6835","nonce":1457679686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.184:6837/1793393087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":1793393087},{"type":"v1","addr":"172.21.15.184:6837","nonce":1793393087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.184:6839/4203347941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":4203347941},{"type":"v1","addr":"172.21.15.184:6839","nonce":4203347941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.33:6839/4136670204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":4136670204},{"type":"v1","addr":"172.21.15.33:6839","nonce":4136670204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24481,24482,24469]},"id":1}]} 2024-09-12T02:03:47.783 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 25 2024-09-12T02:03:47.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:47.784+0000 7f8fb2b35640 1 -- 172.21.15.16:0/2271645372 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f8f980a2b80 msgr2=0x7f8f980a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:47.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:47.784+0000 7f8fb2b35640 1 --2- 172.21.15.16:0/2271645372 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f8f980a2b80 0x7f8f980a5040 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f8fac0fed50 tx=0x7f8f9c00a590 comp rx=0 tx=0).stop 2024-09-12T02:03:47.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:47.784+0000 7f8fb2b35640 1 -- 172.21.15.16:0/2271645372 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8fac102da0 msgr2=0x7f8fac1983a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:47.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:47.784+0000 7f8fb2b35640 1 --2- 172.21.15.16:0/2271645372 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8fac102da0 0x7f8fac1983a0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f8f9400c920 tx=0x7f8f9400cde0 comp rx=0 tx=0).stop 2024-09-12T02:03:47.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:47.784+0000 7f8fb2b35640 1 -- 172.21.15.16:0/2271645372 shutdown_connections 2024-09-12T02:03:47.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:47.784+0000 7f8fb2b35640 1 --2- 172.21.15.16:0/2271645372 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f8f980a2b80 0x7f8f980a5040 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:47.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:47.784+0000 7f8fb2b35640 1 --2- 172.21.15.16:0/2271645372 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8fac10c180 0x7f8fac19cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:47.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:47.784+0000 7f8fb2b35640 1 --2- 172.21.15.16:0/2271645372 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8fac1036c0 0x7f8fac1988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:47.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:47.784+0000 7f8fb2b35640 1 --2- 172.21.15.16:0/2271645372 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8fac102da0 0x7f8fac1983a0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:47.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:47.784+0000 7f8fb2b35640 1 -- 172.21.15.16:0/2271645372 >> 172.21.15.16:0/2271645372 conn(0x7f8fac0fc7a0 msgr2=0x7f8fac0fde10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:47.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:47.784+0000 7f8fb2b35640 1 -- 172.21.15.16:0/2271645372 shutdown_connections 2024-09-12T02:03:47.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:47.784+0000 7f8fb2b35640 1 -- 172.21.15.16:0/2271645372 wait complete. 2024-09-12T02:03:47.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:03:47 smithi184 ceph-mon[35593]: pgmap v391: 97 pgs: 97 active+clean; 1.1 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 6.8 KiB/s wr, 12 op/s 2024-09-12T02:03:47.918 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:03:47 smithi033 ceph-mon[35332]: pgmap v391: 97 pgs: 97 active+clean; 1.1 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 6.8 KiB/s wr, 12 op/s 2024-09-12T02:03:48.402 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24482 quiesce db --set-id f262487 --release --await' 2024-09-12T02:03:48.402 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24482 quiesce db --set-id f262487 --release --await 2024-09-12T02:03:48.665 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:03:48.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:03:48 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/2271645372' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:03:48.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:03:48 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/2271645372' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:03:48.918 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:03:48 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/2271645372' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:03:49.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.165+0000 7f10ed542640 1 -- 172.21.15.16:0/359866984 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10e8102df0 msgr2=0x7f10e81031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:49.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.165+0000 7f10ed542640 1 --2- 172.21.15.16:0/359866984 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10e8102df0 0x7f10e81031d0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f10d8009900 tx=0x7f10d802a410 comp rx=0 tx=0).stop 2024-09-12T02:03:49.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.165+0000 7f10ed542640 1 -- 172.21.15.16:0/359866984 shutdown_connections 2024-09-12T02:03:49.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.165+0000 7f10ed542640 1 --2- 172.21.15.16:0/359866984 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f10e810c2f0 0x7f10e810e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:49.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.165+0000 7f10ed542640 1 --2- 172.21.15.16:0/359866984 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f10e81037a0 0x7f10e810bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:49.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.165+0000 7f10ed542640 1 --2- 172.21.15.16:0/359866984 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10e8102df0 0x7f10e81031d0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:49.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.165+0000 7f10ed542640 1 -- 172.21.15.16:0/359866984 >> 172.21.15.16:0/359866984 conn(0x7f10e80fc7d0 msgr2=0x7f10e80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:49.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.165+0000 7f10ed542640 1 -- 172.21.15.16:0/359866984 shutdown_connections 2024-09-12T02:03:49.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.165+0000 7f10ed542640 1 -- 172.21.15.16:0/359866984 wait complete. 2024-09-12T02:03:49.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.165+0000 7f10ed542640 1 Processor -- start 2024-09-12T02:03:49.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.166+0000 7f10ed542640 1 -- start start 2024-09-12T02:03:49.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.166+0000 7f10ed542640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f10e8102df0 0x7f10e8100a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:49.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.166+0000 7f10ed542640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f10e81037a0 0x7f10e8100f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:49.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.166+0000 7f10ed542640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10e810c2f0 0x7f10e81014d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:49.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.166+0000 7f10ed542640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f10e8112110 con 0x7f10e810c2f0 2024-09-12T02:03:49.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.166+0000 7f10ed542640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f10e8111f90 con 0x7f10e81037a0 2024-09-12T02:03:49.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.166+0000 7f10ed542640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f10e8112290 con 0x7f10e8102df0 2024-09-12T02:03:49.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.166+0000 7f10e77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10e810c2f0 0x7f10e81014d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:49.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.166+0000 7f10e77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10e810c2f0 0x7f10e81014d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50874/0 (socket says 172.21.15.16:50874) 2024-09-12T02:03:49.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.166+0000 7f10e77fe640 1 -- 172.21.15.16:0/1320661492 learned_addr learned my addr 172.21.15.16:0/1320661492 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:03:49.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.166+0000 7f10e6ffd640 1 --2- 172.21.15.16:0/1320661492 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f10e8102df0 0x7f10e8100a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:49.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.166+0000 7f10e67fc640 1 --2- 172.21.15.16:0/1320661492 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f10e81037a0 0x7f10e8100f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:49.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.166+0000 7f10e77fe640 1 -- 172.21.15.16:0/1320661492 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f10e8102df0 msgr2=0x7f10e8100a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:49.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.166+0000 7f10e77fe640 1 --2- 172.21.15.16:0/1320661492 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f10e8102df0 0x7f10e8100a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:49.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.166+0000 7f10e77fe640 1 -- 172.21.15.16:0/1320661492 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f10e81037a0 msgr2=0x7f10e8100f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:49.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.166+0000 7f10e77fe640 1 --2- 172.21.15.16:0/1320661492 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f10e81037a0 0x7f10e8100f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:49.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.166+0000 7f10e77fe640 1 -- 172.21.15.16:0/1320661492 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10e81a6ad0 con 0x7f10e810c2f0 2024-09-12T02:03:49.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.166+0000 7f10e77fe640 1 --2- 172.21.15.16:0/1320661492 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10e810c2f0 0x7f10e81014d0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f10e000eaa0 tx=0x7f10e000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:49.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.167+0000 7f10cffff640 1 -- 172.21.15.16:0/1320661492 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10e000cdb0 con 0x7f10e810c2f0 2024-09-12T02:03:49.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.167+0000 7f10cffff640 1 -- 172.21.15.16:0/1320661492 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f10e0005020 con 0x7f10e810c2f0 2024-09-12T02:03:49.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.167+0000 7f10ed542640 1 -- 172.21.15.16:0/1320661492 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10e81a6dc0 con 0x7f10e810c2f0 2024-09-12T02:03:49.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.167+0000 7f10cffff640 1 -- 172.21.15.16:0/1320661492 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10e00116b0 con 0x7f10e810c2f0 2024-09-12T02:03:49.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.167+0000 7f10ed542640 1 -- 172.21.15.16:0/1320661492 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10e81a7320 con 0x7f10e810c2f0 2024-09-12T02:03:49.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.169+0000 7f10cffff640 1 -- 172.21.15.16:0/1320661492 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f10e0005560 con 0x7f10e810c2f0 2024-09-12T02:03:49.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.169+0000 7f10ef91b740 1 Processor -- start 2024-09-12T02:03:49.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.170+0000 7f10ef91b740 1 -- start start 2024-09-12T02:03:49.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.170+0000 7f10ef91b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557dcb6bb060 0x557dcb6bb440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:49.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.170+0000 7f10ef91b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557dcb6bb980 0x557dcb6c5d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:49.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.170+0000 7f10ef91b740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557dcb6c6240 0x557dcb6c8630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:49.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.170+0000 7f10ef91b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x557dcb620620 con 0x557dcb6bb980 2024-09-12T02:03:49.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.170+0000 7f10ef91b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x557dcb5c7e50 con 0x557dcb6bb060 2024-09-12T02:03:49.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.170+0000 7f10ef91b740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x557dcb61c5e0 con 0x557dcb6c6240 2024-09-12T02:03:49.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.170+0000 7f10e6ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557dcb6bb980 0x557dcb6c5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:49.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.170+0000 7f10e6ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557dcb6bb980 0x557dcb6c5d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50876/0 (socket says 172.21.15.16:50876) 2024-09-12T02:03:49.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.170+0000 7f10e6ffd640 1 -- 172.21.15.16:0/2377199953 learned_addr learned my addr 172.21.15.16:0/2377199953 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:03:49.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.170+0000 7f10cffff640 1 --2- 172.21.15.16:0/1320661492 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f10d00a2c00 0x7f10d00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:49.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.170+0000 7f10e67fc640 1 --2- 172.21.15.16:0/2377199953 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557dcb6c6240 0x557dcb6c8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:49.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.170+0000 7f10e67fc640 1 --2- 172.21.15.16:0/2377199953 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557dcb6bb060 0x557dcb6bb440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:49.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.170+0000 7f10cffff640 1 -- 172.21.15.16:0/1320661492 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f10e00cf180 con 0x7f10e810c2f0 2024-09-12T02:03:49.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.170+0000 7f10e77fe640 1 --2- 172.21.15.16:0/1320661492 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f10d00a2c00 0x7f10d00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:49.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.170+0000 7f10e6ffd640 1 -- 172.21.15.16:0/2377199953 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557dcb6c6240 msgr2=0x557dcb6c8630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:49.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.170+0000 7f10e6ffd640 1 --2- 172.21.15.16:0/2377199953 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557dcb6c6240 0x557dcb6c8630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:49.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.170+0000 7f10e6ffd640 1 -- 172.21.15.16:0/2377199953 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557dcb6bb060 msgr2=0x557dcb6bb440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:49.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.170+0000 7f10e6ffd640 1 --2- 172.21.15.16:0/2377199953 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557dcb6bb060 0x557dcb6bb440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:49.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.170+0000 7f10e6ffd640 1 -- 172.21.15.16:0/2377199953 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557dcb5b8d40 con 0x557dcb6bb980 2024-09-12T02:03:49.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.171+0000 7f10e6ffd640 1 --2- 172.21.15.16:0/2377199953 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557dcb6bb980 0x557dcb6c5d00 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f10d8009900 tx=0x7f10d8005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d616ecbd9dccac2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:49.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.171+0000 7f10e77fe640 1 --2- 172.21.15.16:0/1320661492 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f10d00a2c00 0x7f10d00a50c0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f10e0005910 tx=0x7f10e0000f40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:49.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.171+0000 7f10cd7fa640 1 -- 172.21.15.16:0/2377199953 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d8040070 con 0x557dcb6bb980 2024-09-12T02:03:49.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.171+0000 7f10cd7fa640 1 -- 172.21.15.16:0/2377199953 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f10d8031970 con 0x557dcb6bb980 2024-09-12T02:03:49.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.171+0000 7f10cd7fa640 1 -- 172.21.15.16:0/2377199953 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d8035040 con 0x557dcb6bb980 2024-09-12T02:03:49.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.172+0000 7f10ef91b740 1 -- 172.21.15.16:0/2377199953 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557dcb6bb980 msgr2=0x557dcb6c5d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:49.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.172+0000 7f10ef91b740 1 --2- 172.21.15.16:0/2377199953 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557dcb6bb980 0x557dcb6c5d00 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f10d8009900 tx=0x7f10d8005e80 comp rx=0 tx=0).stop 2024-09-12T02:03:49.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.172+0000 7f10ef91b740 1 -- 172.21.15.16:0/2377199953 shutdown_connections 2024-09-12T02:03:49.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.172+0000 7f10ef91b740 1 --2- 172.21.15.16:0/2377199953 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557dcb6c6240 0x557dcb6c8630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:49.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.172+0000 7f10ef91b740 1 --2- 172.21.15.16:0/2377199953 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557dcb6bb980 0x557dcb6c5d00 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:49.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.172+0000 7f10ef91b740 1 --2- 172.21.15.16:0/2377199953 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557dcb6bb060 0x557dcb6bb440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:49.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.172+0000 7f10ef91b740 1 -- 172.21.15.16:0/2377199953 >> 172.21.15.16:0/2377199953 conn(0x557dcb6b75f0 msgr2=0x557dcb6b79d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:49.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.172+0000 7f10ef91b740 1 -- 172.21.15.16:0/2377199953 shutdown_connections 2024-09-12T02:03:49.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.172+0000 7f10ef91b740 1 -- 172.21.15.16:0/2377199953 wait complete. 2024-09-12T02:03:49.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.172+0000 7f10ef91b740 1 -- start start 2024-09-12T02:03:49.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.172+0000 7f10ef91b740 1 Processor -- start 2024-09-12T02:03:49.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.172+0000 7f10ef91b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557dcb6bb060 0x557dcb6c3070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:49.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.172+0000 7f10ef91b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557dcb6bb980 0x557dcb6c35b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:49.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.172+0000 7f10ef91b740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557dcb6c6240 0x557dcb6c3af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:49.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.172+0000 7f10ef91b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x557dcb6c8e10 con 0x557dcb6bb980 2024-09-12T02:03:49.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.172+0000 7f10ef91b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x557dcb5a8f10 con 0x557dcb6bb060 2024-09-12T02:03:49.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.172+0000 7f10ef91b740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x557dcb6c8f90 con 0x557dcb6c6240 2024-09-12T02:03:49.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.172+0000 7f10ef91b740 10 client.0 _ll_register_callbacks cb 0x557dcb5ccdb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:03:49.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.173+0000 7f10e6ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557dcb6bb980 0x557dcb6c35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:49.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.173+0000 7f10e6ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557dcb6bb980 0x557dcb6c35b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50882/0 (socket says 172.21.15.16:50882) 2024-09-12T02:03:49.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.173+0000 7f10e6ffd640 1 -- 172.21.15.16:0/304442066 learned_addr learned my addr 172.21.15.16:0/304442066 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:03:49.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.173+0000 7f10e67fc640 1 --2- 172.21.15.16:0/304442066 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557dcb6bb060 0x557dcb6c3070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:49.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.173+0000 7f10e67fc640 1 --2- 172.21.15.16:0/304442066 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557dcb6c6240 0x557dcb6c3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:49.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.173+0000 7f10e6ffd640 1 -- 172.21.15.16:0/304442066 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557dcb6c6240 msgr2=0x557dcb6c3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:49.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.173+0000 7f10e6ffd640 1 --2- 172.21.15.16:0/304442066 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557dcb6c6240 0x557dcb6c3af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:49.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.173+0000 7f10e6ffd640 1 -- 172.21.15.16:0/304442066 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557dcb6bb060 msgr2=0x557dcb6c3070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:49.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.173+0000 7f10e6ffd640 1 --2- 172.21.15.16:0/304442066 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557dcb6bb060 0x557dcb6c3070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:49.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.173+0000 7f10e6ffd640 1 -- 172.21.15.16:0/304442066 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557dcb32f580 con 0x557dcb6bb980 2024-09-12T02:03:49.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.173+0000 7f10e67fc640 1 --2- 172.21.15.16:0/304442066 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557dcb6c6240 0x557dcb6c3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:03:49.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.173+0000 7f10e6ffd640 1 --2- 172.21.15.16:0/304442066 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557dcb6bb980 0x557dcb6c35b0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f10d8005940 tx=0x7f10d80052d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9b297c2714c2793 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:49.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.173+0000 7f10caff5640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:03:49.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.173+0000 7f10caff5640 1 -- 172.21.15.16:0/304442066 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d8040030 con 0x557dcb6bb980 2024-09-12T02:03:49.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.173+0000 7f10caff5640 1 -- 172.21.15.16:0/304442066 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f10d8004260 con 0x557dcb6bb980 2024-09-12T02:03:49.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.173+0000 7f10ef91b740 1 -- 172.21.15.16:0/304442066 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f10d8031970 con 0x557dcb6bb980 2024-09-12T02:03:49.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.173+0000 7f10caff5640 1 -- 172.21.15.16:0/304442066 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f10d8035040 con 0x557dcb6bb980 2024-09-12T02:03:49.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.173+0000 7f10ef91b740 10 client.15987 fetch_fsmap learned FSMap version 25 2024-09-12T02:03:49.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.173+0000 7f10ef91b740 1 -- 172.21.15.16:0/304442066 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x557dcb3f9960 con 0x557dcb6bb980 2024-09-12T02:03:49.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.174+0000 7f10caff5640 1 -- 172.21.15.16:0/304442066 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f10d8031970 con 0x557dcb6bb980 2024-09-12T02:03:49.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.174+0000 7f10caff5640 1 -- 172.21.15.16:0/304442066 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f10d80036d0 con 0x557dcb6bb980 2024-09-12T02:03:49.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.174+0000 7f10ef91b740 10 client.15987 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:03:49.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.174+0000 7f10ef91b740 10 client.15987 resolve_mds: validated gid 24482 aka daemon mds.b 2024-09-12T02:03:49.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.174+0000 7f10ef91b740 20 client.15987 populate_metadata read hostname 'smithi016' 2024-09-12T02:03:49.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.174+0000 7f10ef91b740 1 --2- 172.21.15.16:0/304442066 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x557dcb6d5bc0 0x557dcb6d7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:49.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.174+0000 7f10ef91b740 4 client.15987 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:03:49.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.174+0000 7f10ef91b740 1 -- 172.21.15.16:0/304442066 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557dcb62db90 con 0x557dcb6d5bc0 2024-09-12T02:03:49.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.174+0000 7f10e67fc640 1 --2- 172.21.15.16:0/304442066 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x557dcb6d5bc0 0x557dcb6d7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:49.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.175+0000 7f10caff5640 10 client.15987 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:03:49.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.175+0000 7f10e67fc640 1 --2- 172.21.15.16:0/304442066 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x557dcb6d5bc0 0x557dcb6d7fb0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98ee97864c8408cb server_cookie=425331953e6cdeb3 in_seq=0 out_seq=0 2024-09-12T02:03:49.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.175+0000 7f10caff5640 1 -- 172.21.15.16:0/304442066 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557dcb62db90 con 0x557dcb6d5bc0 2024-09-12T02:03:49.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.175+0000 7f10caff5640 10 client.15987 handle_command_reply: tid=0 2024-09-12T02:03:49.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.175+0000 7f10ef91b740 1 client.15987 shutdown 2024-09-12T02:03:49.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.175+0000 7f10ef91b740 1 -- 172.21.15.16:0/304442066 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557dcb6bb980 msgr2=0x557dcb6c35b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:49.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.175+0000 7f10ef91b740 1 --2- 172.21.15.16:0/304442066 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557dcb6bb980 0x557dcb6c35b0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f10d8005940 tx=0x7f10d80052d0 comp rx=0 tx=0).stop 2024-09-12T02:03:49.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.176+0000 7f10ef91b740 1 -- 172.21.15.16:0/304442066 shutdown_connections 2024-09-12T02:03:49.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.176+0000 7f10ef91b740 1 --2- 172.21.15.16:0/304442066 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x557dcb6d5bc0 0x557dcb6d7fb0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:49.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.176+0000 7f10ef91b740 1 --2- 172.21.15.16:0/304442066 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557dcb6c6240 0x557dcb6c3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:49.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.176+0000 7f10ef91b740 1 --2- 172.21.15.16:0/304442066 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557dcb6bb980 0x557dcb6c35b0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:49.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.176+0000 7f10ef91b740 1 --2- 172.21.15.16:0/304442066 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557dcb6bb060 0x557dcb6c3070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:49.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.176+0000 7f10ef91b740 1 -- 172.21.15.16:0/304442066 >> 172.21.15.16:0/304442066 conn(0x557dcb6b75f0 msgr2=0x557dcb6b6750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:49.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.176+0000 7f10caff5640 0 client.15987 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:03:49.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.176+0000 7f10ef91b740 1 -- 172.21.15.16:0/304442066 shutdown_connections 2024-09-12T02:03:49.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.176+0000 7f10ef91b740 1 -- 172.21.15.16:0/304442066 wait complete. 2024-09-12T02:03:49.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.176+0000 7f10ef91b740 20 client.15987 trim_cache size 0 max 16384 2024-09-12T02:03:49.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.193+0000 7f10ef91b740 1 Processor -- start 2024-09-12T02:03:49.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.193+0000 7f10ef91b740 1 -- start start 2024-09-12T02:03:49.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.193+0000 7f10ef91b740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557dcb6bb060 0x557dcb6dd580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:49.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.193+0000 7f10ef91b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557dcb6bb980 0x557dcb6ddac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:49.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.193+0000 7f10ef91b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557dcb6c6240 0x557dcb6c02d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:49.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.193+0000 7f10ef91b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f10e81b1f30 con 0x557dcb6c6240 2024-09-12T02:03:49.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.193+0000 7f10ef91b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f10e81af800 con 0x557dcb6bb980 2024-09-12T02:03:49.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.193+0000 7f10ef91b740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f10e81ad6e0 con 0x557dcb6bb060 2024-09-12T02:03:49.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.193+0000 7f10e67fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557dcb6c6240 0x557dcb6c02d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:49.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.193+0000 7f10e67fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557dcb6c6240 0x557dcb6c02d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50892/0 (socket says 172.21.15.16:50892) 2024-09-12T02:03:49.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.193+0000 7f10e67fc640 1 -- 172.21.15.16:0/3846523456 learned_addr learned my addr 172.21.15.16:0/3846523456 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:03:49.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.193+0000 7f10e67fc640 1 --2- 172.21.15.16:0/3846523456 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557dcb6bb060 0x557dcb6dd580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:49.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.193+0000 7f10e6ffd640 1 --2- 172.21.15.16:0/3846523456 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557dcb6bb980 0x557dcb6ddac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:49.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.193+0000 7f10e67fc640 1 -- 172.21.15.16:0/3846523456 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557dcb6bb060 msgr2=0x557dcb6dd580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:49.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.193+0000 7f10e67fc640 1 --2- 172.21.15.16:0/3846523456 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557dcb6bb060 0x557dcb6dd580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:49.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.193+0000 7f10e67fc640 1 -- 172.21.15.16:0/3846523456 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557dcb6bb980 msgr2=0x557dcb6ddac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:49.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.193+0000 7f10e67fc640 1 --2- 172.21.15.16:0/3846523456 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557dcb6bb980 0x557dcb6ddac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:49.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.193+0000 7f10e67fc640 1 -- 172.21.15.16:0/3846523456 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557dcb6b7420 con 0x557dcb6c6240 2024-09-12T02:03:49.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.194+0000 7f10e67fc640 1 --2- 172.21.15.16:0/3846523456 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557dcb6c6240 0x557dcb6c02d0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f10dc005fd0 tx=0x7f10dc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f024da5346639f1a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:49.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.194+0000 7f10ccff9640 1 -- 172.21.15.16:0/3846523456 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10dc015e70 con 0x557dcb6c6240 2024-09-12T02:03:49.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.194+0000 7f10ccff9640 1 -- 172.21.15.16:0/3846523456 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f10dc014110 con 0x557dcb6c6240 2024-09-12T02:03:49.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.194+0000 7f10ccff9640 1 -- 172.21.15.16:0/3846523456 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10dc005610 con 0x557dcb6c6240 2024-09-12T02:03:49.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.194+0000 7f10ef91b740 1 -- 172.21.15.16:0/3846523456 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557dcb6c6240 msgr2=0x557dcb6c02d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:49.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.194+0000 7f10ef91b740 1 --2- 172.21.15.16:0/3846523456 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557dcb6c6240 0x557dcb6c02d0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f10dc005fd0 tx=0x7f10dc004730 comp rx=0 tx=0).stop 2024-09-12T02:03:49.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.194+0000 7f10ef91b740 1 -- 172.21.15.16:0/3846523456 shutdown_connections 2024-09-12T02:03:49.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.194+0000 7f10ef91b740 1 --2- 172.21.15.16:0/3846523456 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557dcb6c6240 0x557dcb6c02d0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:49.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.194+0000 7f10ef91b740 1 --2- 172.21.15.16:0/3846523456 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557dcb6bb980 0x557dcb6ddac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:49.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.194+0000 7f10ef91b740 1 --2- 172.21.15.16:0/3846523456 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557dcb6bb060 0x557dcb6dd580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:49.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.194+0000 7f10ef91b740 1 -- 172.21.15.16:0/3846523456 >> 172.21.15.16:0/3846523456 conn(0x557dcb6b75f0 msgr2=0x557dcb6c1940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:49.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.194+0000 7f10ef91b740 1 -- 172.21.15.16:0/3846523456 shutdown_connections 2024-09-12T02:03:49.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.195+0000 7f10ef91b740 1 -- 172.21.15.16:0/3846523456 wait complete. 2024-09-12T02:03:49.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.195+0000 7f10ef91b740 1 -- start start 2024-09-12T02:03:49.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.195+0000 7f10ef91b740 1 Processor -- start 2024-09-12T02:03:49.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.195+0000 7f10ef91b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557dcb6bb060 0x557dcb6ce400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:49.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.195+0000 7f10ef91b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557dcb6bb980 0x557dcb6ce960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:49.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.195+0000 7f10ef91b740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557dcb6c6240 0x557dcb6c9250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:49.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.195+0000 7f10ef91b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f10e81add30 con 0x557dcb6bb980 2024-09-12T02:03:49.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.195+0000 7f10ef91b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f10e81af130 con 0x557dcb6bb060 2024-09-12T02:03:49.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.195+0000 7f10ef91b740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f10e81ae390 con 0x557dcb6c6240 2024-09-12T02:03:49.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.195+0000 7f10ef91b740 10 client.0 _ll_register_callbacks cb 0x7f10d8005d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:03:49.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.195+0000 7f10e6ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557dcb6bb980 0x557dcb6ce960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:49.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.195+0000 7f10e6ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557dcb6bb980 0x557dcb6ce960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50902/0 (socket says 172.21.15.16:50902) 2024-09-12T02:03:49.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.195+0000 7f10e6ffd640 1 -- 172.21.15.16:0/4273437006 learned_addr learned my addr 172.21.15.16:0/4273437006 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:03:49.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.195+0000 7f10e67fc640 1 --2- 172.21.15.16:0/4273437006 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557dcb6bb060 0x557dcb6ce400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:49.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.195+0000 7f10e67fc640 1 --2- 172.21.15.16:0/4273437006 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557dcb6c6240 0x557dcb6c9250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:49.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.196+0000 7f10e6ffd640 1 -- 172.21.15.16:0/4273437006 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557dcb6c6240 msgr2=0x557dcb6c9250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:49.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.196+0000 7f10e6ffd640 1 --2- 172.21.15.16:0/4273437006 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557dcb6c6240 0x557dcb6c9250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:49.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.196+0000 7f10e6ffd640 1 -- 172.21.15.16:0/4273437006 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557dcb6bb060 msgr2=0x557dcb6ce400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:49.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.196+0000 7f10e6ffd640 1 --2- 172.21.15.16:0/4273437006 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557dcb6bb060 0x557dcb6ce400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:49.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.196+0000 7f10e6ffd640 1 -- 172.21.15.16:0/4273437006 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557dcb6c9a80 con 0x557dcb6bb980 2024-09-12T02:03:49.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.196+0000 7f10e67fc640 1 --2- 172.21.15.16:0/4273437006 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557dcb6bb060 0x557dcb6ce400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:03:49.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.196+0000 7f10e6ffd640 1 --2- 172.21.15.16:0/4273437006 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557dcb6bb980 0x557dcb6ce960 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f10d802a450 tx=0x7f10d8009590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb16ca25be0bb4ad server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:49.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.196+0000 7f10caff5640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:03:49.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.196+0000 7f10caff5640 1 -- 172.21.15.16:0/4273437006 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d8061060 con 0x557dcb6bb980 2024-09-12T02:03:49.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.196+0000 7f10caff5640 1 -- 172.21.15.16:0/4273437006 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f10d8031970 con 0x557dcb6bb980 2024-09-12T02:03:49.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.196+0000 7f10ef91b740 1 -- 172.21.15.16:0/4273437006 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f10dc014110 con 0x557dcb6bb980 2024-09-12T02:03:49.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.196+0000 7f10caff5640 1 -- 172.21.15.16:0/4273437006 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f10d8040040 con 0x557dcb6bb980 2024-09-12T02:03:49.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.196+0000 7f10ef91b740 10 client.15993 fetch_fsmap learned FSMap version 25 2024-09-12T02:03:49.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.196+0000 7f10ef91b740 1 -- 172.21.15.16:0/4273437006 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x557dcb741130 con 0x557dcb6bb980 2024-09-12T02:03:49.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.196+0000 7f10caff5640 1 -- 172.21.15.16:0/4273437006 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f10dc014110 con 0x557dcb6bb980 2024-09-12T02:03:49.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.197+0000 7f10caff5640 1 -- 172.21.15.16:0/4273437006 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f10d8066070 con 0x557dcb6bb980 2024-09-12T02:03:49.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.197+0000 7f10ef91b740 10 client.15993 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:03:49.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.197+0000 7f10ef91b740 10 client.15993 resolve_mds: validated gid 24482 aka daemon mds.b 2024-09-12T02:03:49.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.197+0000 7f10ef91b740 20 client.15993 populate_metadata read hostname 'smithi016' 2024-09-12T02:03:49.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.197+0000 7f10ef91b740 1 --2- 172.21.15.16:0/4273437006 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x557dcb741820 0x557dcb743c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:49.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.197+0000 7f10ef91b740 4 client.15993 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "f262487", "release": true, "await": true}] 2024-09-12T02:03:49.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.197+0000 7f10ef91b740 1 -- 172.21.15.16:0/4273437006 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "quiesce db", "set_id": "f262487", "release": true, "await": true}) -- 0x557dcb6280a0 con 0x557dcb741820 2024-09-12T02:03:49.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.197+0000 7f10e67fc640 1 --2- 172.21.15.16:0/4273437006 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x557dcb741820 0x557dcb743c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:49.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.197+0000 7f10e67fc640 1 --2- 172.21.15.16:0/4273437006 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x557dcb741820 0x557dcb743c40 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd041e283ea24048 server_cookie=e3ef14eebad5d461 in_seq=0 out_seq=0 2024-09-12T02:03:49.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.197+0000 7f10caff5640 10 client.15993 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:03:49.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.200+0000 7f10caff5640 1 -- 172.21.15.16:0/4273437006 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+588 (crc 0 0 0) 0x557dcb6280a0 con 0x557dcb741820 2024-09-12T02:03:49.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.200+0000 7f10caff5640 10 client.15993 handle_command_reply: tid=0 2024-09-12T02:03:49.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.200+0000 7f10ef91b740 1 client.15993 shutdown 2024-09-12T02:03:49.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.200+0000 7f10ef91b740 1 -- 172.21.15.16:0/4273437006 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557dcb6bb980 msgr2=0x557dcb6ce960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:49.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.200+0000 7f10ef91b740 1 --2- 172.21.15.16:0/4273437006 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557dcb6bb980 0x557dcb6ce960 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f10d802a450 tx=0x7f10d8009590 comp rx=0 tx=0).stop 2024-09-12T02:03:49.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.200+0000 7f10ef91b740 1 -- 172.21.15.16:0/4273437006 shutdown_connections 2024-09-12T02:03:49.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.200+0000 7f10ef91b740 1 --2- 172.21.15.16:0/4273437006 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x557dcb741820 0x557dcb743c40 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:49.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.200+0000 7f10ef91b740 1 --2- 172.21.15.16:0/4273437006 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557dcb6c6240 0x557dcb6c9250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:49.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.200+0000 7f10ef91b740 1 --2- 172.21.15.16:0/4273437006 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557dcb6bb980 0x557dcb6ce960 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:49.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.200+0000 7f10caff5640 0 client.15993 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:03:49.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.200+0000 7f10ef91b740 1 --2- 172.21.15.16:0/4273437006 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557dcb6bb060 0x557dcb6ce400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:49.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.200+0000 7f10ef91b740 1 -- 172.21.15.16:0/4273437006 >> 172.21.15.16:0/4273437006 conn(0x557dcb6b75f0 msgr2=0x557dcb6bca20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:49.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.200+0000 7f10ef91b740 1 -- 172.21.15.16:0/4273437006 shutdown_connections 2024-09-12T02:03:49.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.200+0000 7f10ef91b740 1 -- 172.21.15.16:0/4273437006 wait complete. 2024-09-12T02:03:49.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.200+0000 7f10ef91b740 20 client.15993 trim_cache size 0 max 16384 2024-09-12T02:03:49.205 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T02:03:49.205 INFO:teuthology.orchestra.run.smithi016.stdout: "epoch": 25, 2024-09-12T02:03:49.205 INFO:teuthology.orchestra.run.smithi016.stdout: "leader": 24482, 2024-09-12T02:03:49.205 INFO:teuthology.orchestra.run.smithi016.stdout: "set_version": 8, 2024-09-12T02:03:49.205 INFO:teuthology.orchestra.run.smithi016.stdout: "sets": { 2024-09-12T02:03:49.205 INFO:teuthology.orchestra.run.smithi016.stdout: "f262487": { 2024-09-12T02:03:49.205 INFO:teuthology.orchestra.run.smithi016.stdout: "version": 8, 2024-09-12T02:03:49.205 INFO:teuthology.orchestra.run.smithi016.stdout: "age_ref": 0.0, 2024-09-12T02:03:49.206 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-12T02:03:49.206 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RELEASED", 2024-09-12T02:03:49.206 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-12T02:03:49.206 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:03:49.206 INFO:teuthology.orchestra.run.smithi016.stdout: "timeout": 90.0, 2024-09-12T02:03:49.206 INFO:teuthology.orchestra.run.smithi016.stdout: "expiration": 133.4, 2024-09-12T02:03:49.206 INFO:teuthology.orchestra.run.smithi016.stdout: "members": { 2024-09-12T02:03:49.206 INFO:teuthology.orchestra.run.smithi016.stdout: "file:/": { 2024-09-12T02:03:49.206 INFO:teuthology.orchestra.run.smithi016.stdout: "excluded": false, 2024-09-12T02:03:49.206 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-12T02:03:49.206 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RELEASED", 2024-09-12T02:03:49.206 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-12T02:03:49.206 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:03:49.207 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:03:49.207 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:03:49.207 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:03:49.207 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:03:49.207 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T02:03:49.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.201+0000 7f10ed542640 1 -- 172.21.15.16:0/1320661492 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f10d00a2c00 msgr2=0x7f10d00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:49.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.201+0000 7f10ed542640 1 --2- 172.21.15.16:0/1320661492 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f10d00a2c00 0x7f10d00a50c0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f10e0005910 tx=0x7f10e0000f40 comp rx=0 tx=0).stop 2024-09-12T02:03:49.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.201+0000 7f10ed542640 1 -- 172.21.15.16:0/1320661492 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10e810c2f0 msgr2=0x7f10e81014d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:49.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.201+0000 7f10ed542640 1 --2- 172.21.15.16:0/1320661492 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10e810c2f0 0x7f10e81014d0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f10e000eaa0 tx=0x7f10e000ef60 comp rx=0 tx=0).stop 2024-09-12T02:03:49.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.201+0000 7f10ed542640 1 -- 172.21.15.16:0/1320661492 shutdown_connections 2024-09-12T02:03:49.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.201+0000 7f10ed542640 1 --2- 172.21.15.16:0/1320661492 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f10d00a2c00 0x7f10d00a50c0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:49.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.201+0000 7f10ed542640 1 --2- 172.21.15.16:0/1320661492 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10e810c2f0 0x7f10e81014d0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:49.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.201+0000 7f10ed542640 1 --2- 172.21.15.16:0/1320661492 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f10e81037a0 0x7f10e8100f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:49.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.201+0000 7f10ed542640 1 --2- 172.21.15.16:0/1320661492 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f10e8102df0 0x7f10e8100a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:49.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.201+0000 7f10ed542640 1 -- 172.21.15.16:0/1320661492 >> 172.21.15.16:0/1320661492 conn(0x7f10e80fc7d0 msgr2=0x7f10e80fd4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:49.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.201+0000 7f10ed542640 1 -- 172.21.15.16:0/1320661492 shutdown_connections 2024-09-12T02:03:49.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:03:49.201+0000 7f10ed542640 1 -- 172.21.15.16:0/1320661492 wait complete. 2024-09-12T02:03:49.873 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:03:49 smithi016 ceph-mon[30947]: pgmap v392: 97 pgs: 97 active+clean; 1.1 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 6.7 KiB/s wr, 10 op/s 2024-09-12T02:03:49.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:03:49 smithi184 ceph-mon[35593]: pgmap v392: 97 pgs: 97 active+clean; 1.1 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 6.7 KiB/s wr, 10 op/s 2024-09-12T02:03:49.902 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'f262487', total seconds elapsed: 16.5 2024-09-12T02:03:49.918 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:03:49 smithi033 ceph-mon[35332]: pgmap v392: 97 pgs: 97 active+clean; 1.1 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 6.7 KiB/s wr, 10 op/s 2024-09-12T02:03:51.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:03:51 smithi184 ceph-mon[35593]: pgmap v393: 97 pgs: 97 active+clean; 1.1 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.7 KiB/s wr, 13 op/s 2024-09-12T02:03:51.918 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:03:51 smithi033 ceph-mon[35332]: pgmap v393: 97 pgs: 97 active+clean; 1.1 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.7 KiB/s wr, 13 op/s 2024-09-12T02:03:51.933 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:03:51 smithi016 ceph-mon[30947]: pgmap v393: 97 pgs: 97 active+clean; 1.1 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.7 KiB/s wr, 13 op/s 2024-09-12T02:03:53.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:03:53 smithi184 ceph-mon[35593]: pgmap v394: 97 pgs: 97 active+clean; 1.1 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.2 KiB/s wr, 8 op/s 2024-09-12T02:03:53.918 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:03:53 smithi033 ceph-mon[35332]: pgmap v394: 97 pgs: 97 active+clean; 1.1 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.2 KiB/s wr, 8 op/s 2024-09-12T02:03:53.934 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:03:53 smithi016 ceph-mon[30947]: pgmap v394: 97 pgs: 97 active+clean; 1.1 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.2 KiB/s wr, 8 op/s 2024-09-12T02:03:55.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:03:55 smithi184 ceph-mon[35593]: pgmap v395: 97 pgs: 97 active+clean; 1.2 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 7.7 KiB/s wr, 11 op/s 2024-09-12T02:03:55.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:03:55 smithi033 ceph-mon[35332]: pgmap v395: 97 pgs: 97 active+clean; 1.2 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 7.7 KiB/s wr, 11 op/s 2024-09-12T02:03:55.933 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:03:55 smithi016 ceph-mon[30947]: pgmap v395: 97 pgs: 97 active+clean; 1.2 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 7.7 KiB/s wr, 11 op/s 2024-09-12T02:03:55.961 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.958+0000 7f1cc65d6640 1 -- 172.21.15.16:0/1863729273 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1cc0107830 msgr2=0x7f1cc0109ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:55.962 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.958+0000 7f1cc65d6640 1 --2- 172.21.15.16:0/1863729273 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1cc0107830 0x7f1cc0109ca0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f1cb8009ca0 tx=0x7f1cb802a8c0 comp rx=0 tx=0).stop 2024-09-12T02:03:55.962 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.958+0000 7f1cc65d6640 1 -- 172.21.15.16:0/1863729273 shutdown_connections 2024-09-12T02:03:55.962 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.958+0000 7f1cc65d6640 1 --2- 172.21.15.16:0/1863729273 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1cc0107830 0x7f1cc0109ca0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:55.962 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.958+0000 7f1cc65d6640 1 --2- 172.21.15.16:0/1863729273 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1cc0104f00 0x7f1cc01072f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:55.962 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.958+0000 7f1cc65d6640 1 --2- 172.21.15.16:0/1863729273 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1cc01025d0 0x7f1cc01049c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:55.962 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.958+0000 7f1cc65d6640 1 -- 172.21.15.16:0/1863729273 >> 172.21.15.16:0/1863729273 conn(0x7f1cc00fc1b0 msgr2=0x7f1cc00fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:55.962 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.958+0000 7f1cc65d6640 1 -- 172.21.15.16:0/1863729273 shutdown_connections 2024-09-12T02:03:55.962 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.958+0000 7f1cc65d6640 1 -- 172.21.15.16:0/1863729273 wait complete. 2024-09-12T02:03:55.962 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.958+0000 7f1cc65d6640 1 Processor -- start 2024-09-12T02:03:55.962 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.959+0000 7f1cc65d6640 1 -- start start 2024-09-12T02:03:55.963 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.959+0000 7f1cc65d6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1cc01025d0 0x7f1cc0197e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:55.963 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.959+0000 7f1cc65d6640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1cc0104f00 0x7f1cc0198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:55.963 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.959+0000 7f1cc65d6640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1cc0107830 0x7f1cc019f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:55.963 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.959+0000 7f1cc65d6640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc010d810 con 0x7f1cc01025d0 2024-09-12T02:03:55.963 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.959+0000 7f1cc65d6640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc010d690 con 0x7f1cc0104f00 2024-09-12T02:03:55.963 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.959+0000 7f1cc65d6640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc010d990 con 0x7f1cc0107830 2024-09-12T02:03:55.963 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.959+0000 7f1cbffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1cc01025d0 0x7f1cc0197e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:55.963 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.959+0000 7f1cbffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1cc01025d0 0x7f1cc0197e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57272/0 (socket says 172.21.15.16:57272) 2024-09-12T02:03:55.963 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.959+0000 7f1cbffff640 1 -- 172.21.15.16:0/992349685 learned_addr learned my addr 172.21.15.16:0/992349685 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:03:55.963 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.959+0000 7f1cc4b73640 1 --2- 172.21.15.16:0/992349685 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1cc0107830 0x7f1cc019f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:55.963 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.959+0000 7f1cbf7fe640 1 --2- 172.21.15.16:0/992349685 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1cc0104f00 0x7f1cc0198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:55.964 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.959+0000 7f1cbffff640 1 -- 172.21.15.16:0/992349685 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1cc0107830 msgr2=0x7f1cc019f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:55.964 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.959+0000 7f1cbffff640 1 --2- 172.21.15.16:0/992349685 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1cc0107830 0x7f1cc019f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:55.964 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.959+0000 7f1cbffff640 1 -- 172.21.15.16:0/992349685 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1cc0104f00 msgr2=0x7f1cc0198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:55.964 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.959+0000 7f1cbffff640 1 --2- 172.21.15.16:0/992349685 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1cc0104f00 0x7f1cc0198380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:55.964 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.959+0000 7f1cbffff640 1 -- 172.21.15.16:0/992349685 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cc019fa60 con 0x7f1cc01025d0 2024-09-12T02:03:55.964 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.959+0000 7f1cc4b73640 1 --2- 172.21.15.16:0/992349685 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1cc0107830 0x7f1cc019f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:03:55.964 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.960+0000 7f1cbffff640 1 --2- 172.21.15.16:0/992349685 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1cc01025d0 0x7f1cc0197e40 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f1cb000e9f0 tx=0x7f1cb000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:55.964 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.960+0000 7f1cbd7fa640 1 -- 172.21.15.16:0/992349685 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb000cd80 con 0x7f1cc01025d0 2024-09-12T02:03:55.964 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.960+0000 7f1cbd7fa640 1 -- 172.21.15.16:0/992349685 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1cb0010910 con 0x7f1cc01025d0 2024-09-12T02:03:55.964 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.960+0000 7f1cc65d6640 1 -- 172.21.15.16:0/992349685 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cc019fd50 con 0x7f1cc01025d0 2024-09-12T02:03:55.965 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.960+0000 7f1cbd7fa640 1 -- 172.21.15.16:0/992349685 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb0018990 con 0x7f1cc01025d0 2024-09-12T02:03:55.965 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.960+0000 7f1cc65d6640 1 -- 172.21.15.16:0/992349685 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cc01a0210 con 0x7f1cc01025d0 2024-09-12T02:03:55.972 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.963+0000 7f1cbd7fa640 1 -- 172.21.15.16:0/992349685 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f1cb0010ac0 con 0x7f1cc01025d0 2024-09-12T02:03:55.972 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.963+0000 7f1cc944e740 1 Processor -- start 2024-09-12T02:03:55.972 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.963+0000 7f1cc944e740 1 -- start start 2024-09-12T02:03:55.973 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.963+0000 7f1cc944e740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bae9a49db0 0x55bae9a4a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:55.973 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.963+0000 7f1cc944e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bae9a4a6f0 0x55bae9a4f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:55.973 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.963+0000 7f1cc944e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bae9a4f780 0x55bae9a51c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:55.973 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.963+0000 7f1cc944e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55bae99a92f0 con 0x55bae9a4a6f0 2024-09-12T02:03:55.973 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.963+0000 7f1cc944e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55bae9951270 con 0x55bae9a4f780 2024-09-12T02:03:55.973 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.963+0000 7f1cc944e740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55bae99a52b0 con 0x55bae9a49db0 2024-09-12T02:03:55.973 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.964+0000 7f1cbffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bae9a4a6f0 0x55bae9a4f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:55.973 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.964+0000 7f1cbf7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bae9a4f780 0x55bae9a51c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:55.973 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.964+0000 7f1cbffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bae9a4a6f0 0x55bae9a4f1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57284/0 (socket says 172.21.15.16:57284) 2024-09-12T02:03:55.973 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.964+0000 7f1cbffff640 1 -- 172.21.15.16:0/1777291252 learned_addr learned my addr 172.21.15.16:0/1777291252 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:03:55.973 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.964+0000 7f1cc4b73640 1 --2- 172.21.15.16:0/1777291252 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bae9a49db0 0x55bae9a4a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:55.973 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.964+0000 7f1cbd7fa640 1 --2- 172.21.15.16:0/992349685 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f1ca80a2cb0 0x7f1ca80a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:55.973 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.964+0000 7f1cbffff640 1 -- 172.21.15.16:0/1777291252 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bae9a49db0 msgr2=0x55bae9a4a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:55.974 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.964+0000 7f1cbffff640 1 --2- 172.21.15.16:0/1777291252 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bae9a49db0 0x55bae9a4a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:55.974 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.964+0000 7f1cbffff640 1 -- 172.21.15.16:0/1777291252 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bae9a4f780 msgr2=0x55bae9a51c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:55.974 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.964+0000 7f1cbffff640 1 --2- 172.21.15.16:0/1777291252 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bae9a4f780 0x55bae9a51c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:55.974 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.964+0000 7f1cbffff640 1 -- 172.21.15.16:0/1777291252 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bae9950340 con 0x55bae9a4a6f0 2024-09-12T02:03:55.974 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.964+0000 7f1cbf7fe640 1 --2- 172.21.15.16:0/1777291252 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bae9a4f780 0x55bae9a51c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:03:55.974 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.964+0000 7f1cc4b73640 1 --2- 172.21.15.16:0/992349685 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f1ca80a2cb0 0x7f1ca80a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:55.974 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.964+0000 7f1cbd7fa640 1 -- 172.21.15.16:0/992349685 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f1cb00cf370 con 0x7f1cc01025d0 2024-09-12T02:03:55.974 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.964+0000 7f1cbffff640 1 --2- 172.21.15.16:0/1777291252 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bae9a4a6f0 0x55bae9a4f1b0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f1cb00785b0 tx=0x7f1cb0066900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38d80a837bd5121e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:55.974 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.965+0000 7f1ca67fc640 1 -- 172.21.15.16:0/1777291252 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb009b7b0 con 0x55bae9a4a6f0 2024-09-12T02:03:55.974 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.965+0000 7f1cc4b73640 1 --2- 172.21.15.16:0/992349685 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f1ca80a2cb0 0x7f1ca80a5170 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f1cb8031ae0 tx=0x7f1cb803e040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:55.974 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.965+0000 7f1ca67fc640 1 -- 172.21.15.16:0/1777291252 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1cb0026050 con 0x55bae9a4a6f0 2024-09-12T02:03:55.974 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.965+0000 7f1ca67fc640 1 -- 172.21.15.16:0/1777291252 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb009b5d0 con 0x55bae9a4a6f0 2024-09-12T02:03:55.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.965+0000 7f1cc944e740 1 -- 172.21.15.16:0/1777291252 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bae9a4a6f0 msgr2=0x55bae9a4f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:55.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.965+0000 7f1cc944e740 1 --2- 172.21.15.16:0/1777291252 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bae9a4a6f0 0x55bae9a4f1b0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f1cb00785b0 tx=0x7f1cb0066900 comp rx=0 tx=0).stop 2024-09-12T02:03:55.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.965+0000 7f1cc944e740 1 -- 172.21.15.16:0/1777291252 shutdown_connections 2024-09-12T02:03:55.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.965+0000 7f1cc944e740 1 --2- 172.21.15.16:0/1777291252 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bae9a4f780 0x55bae9a51c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:55.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.965+0000 7f1cc944e740 1 --2- 172.21.15.16:0/1777291252 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bae9a4a6f0 0x55bae9a4f1b0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:55.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.965+0000 7f1cc944e740 1 --2- 172.21.15.16:0/1777291252 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bae9a49db0 0x55bae9a4a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:55.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.965+0000 7f1cc944e740 1 -- 172.21.15.16:0/1777291252 >> 172.21.15.16:0/1777291252 conn(0x55bae9a41010 msgr2=0x55bae9a41410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:55.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.965+0000 7f1cc944e740 1 -- 172.21.15.16:0/1777291252 shutdown_connections 2024-09-12T02:03:55.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.965+0000 7f1cc944e740 1 -- 172.21.15.16:0/1777291252 wait complete. 2024-09-12T02:03:55.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.966+0000 7f1cc944e740 1 -- start start 2024-09-12T02:03:55.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.966+0000 7f1cc944e740 1 Processor -- start 2024-09-12T02:03:55.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.966+0000 7f1cc944e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bae9a49db0 0x55bae9a53f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:55.976 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.966+0000 7f1cc944e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bae9a4a6f0 0x55bae9a54460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:55.976 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.966+0000 7f1cc944e740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bae9a4f780 0x55bae9a5b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:55.976 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.966+0000 7f1cc944e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55bae9a52690 con 0x55bae9a4a6f0 2024-09-12T02:03:55.976 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.966+0000 7f1cc944e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55bae9932290 con 0x55bae9a49db0 2024-09-12T02:03:55.976 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.966+0000 7f1cc944e740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55bae9a52810 con 0x55bae9a4f780 2024-09-12T02:03:55.976 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.966+0000 7f1cc944e740 10 client.0 _ll_register_callbacks cb 0x55bae9963140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:03:55.976 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.966+0000 7f1cbf7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bae9a4a6f0 0x55bae9a54460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:55.976 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.966+0000 7f1cbf7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bae9a4a6f0 0x55bae9a54460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57298/0 (socket says 172.21.15.16:57298) 2024-09-12T02:03:55.976 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.966+0000 7f1cbf7fe640 1 -- 172.21.15.16:0/1745124119 learned_addr learned my addr 172.21.15.16:0/1745124119 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:03:55.977 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.967+0000 7f1cbffff640 1 --2- 172.21.15.16:0/1745124119 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bae9a49db0 0x55bae9a53f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:55.977 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.967+0000 7f1cc4b73640 1 --2- 172.21.15.16:0/1745124119 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bae9a4f780 0x55bae9a5b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:55.977 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.967+0000 7f1cbf7fe640 1 -- 172.21.15.16:0/1745124119 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bae9a4f780 msgr2=0x55bae9a5b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:55.977 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.967+0000 7f1cbf7fe640 1 --2- 172.21.15.16:0/1745124119 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bae9a4f780 0x55bae9a5b4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:55.977 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.967+0000 7f1cbf7fe640 1 -- 172.21.15.16:0/1745124119 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bae9a49db0 msgr2=0x55bae9a53f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:55.977 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.967+0000 7f1cbf7fe640 1 --2- 172.21.15.16:0/1745124119 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bae9a49db0 0x55bae9a53f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:55.977 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.967+0000 7f1cbf7fe640 1 -- 172.21.15.16:0/1745124119 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bae96bc210 con 0x55bae9a4a6f0 2024-09-12T02:03:55.977 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.967+0000 7f1cbf7fe640 1 --2- 172.21.15.16:0/1745124119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bae9a4a6f0 0x55bae9a54460 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f1cb40136e0 tx=0x7f1cb4013ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=798baa019a5fef80 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:55.977 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.967+0000 7f1c9f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:03:55.977 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.967+0000 7f1c9f7fe640 1 -- 172.21.15.16:0/1745124119 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb4004460 con 0x55bae9a4a6f0 2024-09-12T02:03:55.977 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.967+0000 7f1c9f7fe640 1 -- 172.21.15.16:0/1745124119 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f1cb40119c0 con 0x55bae9a4a6f0 2024-09-12T02:03:55.978 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.967+0000 7f1cc944e740 1 -- 172.21.15.16:0/1745124119 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1cb0026050 con 0x55bae9a4a6f0 2024-09-12T02:03:55.978 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.967+0000 7f1c9f7fe640 1 -- 172.21.15.16:0/1745124119 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1cb4011bc0 con 0x55bae9a4a6f0 2024-09-12T02:03:55.978 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.967+0000 7f1cc944e740 10 client.16005 fetch_fsmap learned FSMap version 25 2024-09-12T02:03:55.978 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.967+0000 7f1cc944e740 1 -- 172.21.15.16:0/1745124119 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55bae9788620 con 0x55bae9a4a6f0 2024-09-12T02:03:55.978 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.968+0000 7f1c9f7fe640 1 -- 172.21.15.16:0/1745124119 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1cb0026050 con 0x55bae9a4a6f0 2024-09-12T02:03:55.978 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.968+0000 7f1c9f7fe640 1 -- 172.21.15.16:0/1745124119 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f1cb401a020 con 0x55bae9a4a6f0 2024-09-12T02:03:55.978 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.968+0000 7f1cc944e740 10 client.16005 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:03:55.978 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.968+0000 7f1cc944e740 10 client.16005 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:03:55.978 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.968+0000 7f1cc944e740 20 client.16005 populate_metadata read hostname 'smithi016' 2024-09-12T02:03:55.978 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.968+0000 7f1cc944e740 1 --2- 172.21.15.16:0/1745124119 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55bae9a5fbf0 0x55bae9a62040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:55.978 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.968+0000 7f1cc944e740 4 client.16005 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:03:55.978 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.968+0000 7f1cc944e740 1 -- 172.21.15.16:0/1745124119 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bae99b6a20 con 0x55bae9a5fbf0 2024-09-12T02:03:55.978 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.968+0000 7f1cbffff640 1 --2- 172.21.15.16:0/1745124119 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55bae9a5fbf0 0x55bae9a62040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:55.979 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.969+0000 7f1cbffff640 1 --2- 172.21.15.16:0/1745124119 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55bae9a5fbf0 0x55bae9a62040 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab4a5a150f9985f6 server_cookie=be6316223cfaddf4 in_seq=0 out_seq=0 2024-09-12T02:03:55.979 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.969+0000 7f1c9f7fe640 10 client.16005 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:03:55.979 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.969+0000 7f1c9f7fe640 1 -- 172.21.15.16:0/1745124119 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bae99b6a20 con 0x55bae9a5fbf0 2024-09-12T02:03:55.979 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.969+0000 7f1c9f7fe640 10 client.16005 handle_command_reply: tid=0 2024-09-12T02:03:55.979 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.969+0000 7f1cc944e740 1 client.16005 shutdown 2024-09-12T02:03:55.979 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.969+0000 7f1cc944e740 1 -- 172.21.15.16:0/1745124119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bae9a4a6f0 msgr2=0x55bae9a54460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:55.979 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.969+0000 7f1cc944e740 1 --2- 172.21.15.16:0/1745124119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bae9a4a6f0 0x55bae9a54460 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f1cb40136e0 tx=0x7f1cb4013ba0 comp rx=0 tx=0).stop 2024-09-12T02:03:55.979 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.970+0000 7f1cc944e740 1 -- 172.21.15.16:0/1745124119 shutdown_connections 2024-09-12T02:03:55.979 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.970+0000 7f1cc944e740 1 --2- 172.21.15.16:0/1745124119 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55bae9a5fbf0 0x55bae9a62040 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:55.979 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.970+0000 7f1cc944e740 1 --2- 172.21.15.16:0/1745124119 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bae9a4f780 0x55bae9a5b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:55.979 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.970+0000 7f1cc944e740 1 --2- 172.21.15.16:0/1745124119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bae9a4a6f0 0x55bae9a54460 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:55.979 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.970+0000 7f1c9f7fe640 0 client.16005 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:03:55.980 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.970+0000 7f1cc944e740 1 --2- 172.21.15.16:0/1745124119 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bae9a49db0 0x55bae9a53f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:55.980 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.970+0000 7f1cc944e740 1 -- 172.21.15.16:0/1745124119 >> 172.21.15.16:0/1745124119 conn(0x55bae9a41010 msgr2=0x55bae9a42820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:55.980 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.970+0000 7f1cc944e740 1 -- 172.21.15.16:0/1745124119 shutdown_connections 2024-09-12T02:03:55.980 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.970+0000 7f1cc944e740 1 -- 172.21.15.16:0/1745124119 wait complete. 2024-09-12T02:03:55.980 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.970+0000 7f1cc944e740 20 client.16005 trim_cache size 0 max 16384 2024-09-12T02:03:55.987 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.984+0000 7f1cc944e740 1 Processor -- start 2024-09-12T02:03:55.987 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.984+0000 7f1cc944e740 1 -- start start 2024-09-12T02:03:55.987 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.984+0000 7f1cc944e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bae9a49db0 0x55bae9a66990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:55.987 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.984+0000 7f1cc944e740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bae9a4a6f0 0x55bae9a5b460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:55.987 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.984+0000 7f1cc944e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bae9a4f780 0x55bae9a66ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:55.988 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.984+0000 7f1cc944e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc01a9600 con 0x55bae9a49db0 2024-09-12T02:03:55.988 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.984+0000 7f1cc944e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc01a6ed0 con 0x55bae9a4f780 2024-09-12T02:03:55.988 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.984+0000 7f1cc944e740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc01a4d40 con 0x55bae9a4a6f0 2024-09-12T02:03:55.988 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.985+0000 7f1cbffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bae9a49db0 0x55bae9a66990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:55.988 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.985+0000 7f1cbffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bae9a49db0 0x55bae9a66990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57306/0 (socket says 172.21.15.16:57306) 2024-09-12T02:03:55.988 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.985+0000 7f1cbffff640 1 -- 172.21.15.16:0/3928750669 learned_addr learned my addr 172.21.15.16:0/3928750669 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:03:55.988 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.985+0000 7f1cc4b73640 1 --2- 172.21.15.16:0/3928750669 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bae9a4f780 0x55bae9a66ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:55.988 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.985+0000 7f1cbf7fe640 1 --2- 172.21.15.16:0/3928750669 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bae9a4a6f0 0x55bae9a5b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:55.988 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.985+0000 7f1cbffff640 1 -- 172.21.15.16:0/3928750669 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bae9a4a6f0 msgr2=0x55bae9a5b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:55.988 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.985+0000 7f1cbffff640 1 --2- 172.21.15.16:0/3928750669 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bae9a4a6f0 0x55bae9a5b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:55.988 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.985+0000 7f1cbffff640 1 -- 172.21.15.16:0/3928750669 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bae9a4f780 msgr2=0x55bae9a66ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:55.988 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.985+0000 7f1cbffff640 1 --2- 172.21.15.16:0/3928750669 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bae9a4f780 0x55bae9a66ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:55.989 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.985+0000 7f1cbffff640 1 -- 172.21.15.16:0/3928750669 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bae9a5e560 con 0x55bae9a49db0 2024-09-12T02:03:55.989 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.985+0000 7f1cbffff640 1 --2- 172.21.15.16:0/3928750669 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bae9a49db0 0x55bae9a66990 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f1cb009bc00 tx=0x7f1cb006d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0e71d7b46e728ca server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:55.989 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.985+0000 7f1ca5ffb640 1 -- 172.21.15.16:0/3928750669 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb006f2f0 con 0x55bae9a49db0 2024-09-12T02:03:55.989 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.985+0000 7f1ca5ffb640 1 -- 172.21.15.16:0/3928750669 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1cb009bc70 con 0x55bae9a49db0 2024-09-12T02:03:55.989 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.985+0000 7f1ca5ffb640 1 -- 172.21.15.16:0/3928750669 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb0099f70 con 0x55bae9a49db0 2024-09-12T02:03:55.989 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.986+0000 7f1cc944e740 1 -- 172.21.15.16:0/3928750669 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bae9a49db0 msgr2=0x55bae9a66990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:55.989 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.986+0000 7f1cc944e740 1 --2- 172.21.15.16:0/3928750669 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bae9a49db0 0x55bae9a66990 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f1cb009bc00 tx=0x7f1cb006d040 comp rx=0 tx=0).stop 2024-09-12T02:03:55.989 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.986+0000 7f1cc944e740 1 -- 172.21.15.16:0/3928750669 shutdown_connections 2024-09-12T02:03:55.989 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.986+0000 7f1cc944e740 1 --2- 172.21.15.16:0/3928750669 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bae9a4f780 0x55bae9a66ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:55.989 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.986+0000 7f1cc944e740 1 --2- 172.21.15.16:0/3928750669 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bae9a4a6f0 0x55bae9a5b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:55.989 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.986+0000 7f1cc944e740 1 --2- 172.21.15.16:0/3928750669 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bae9a49db0 0x55bae9a66990 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:55.989 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.986+0000 7f1cc944e740 1 -- 172.21.15.16:0/3928750669 >> 172.21.15.16:0/3928750669 conn(0x55bae9a41010 msgr2=0x55bae9a401c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:55.989 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.986+0000 7f1cc944e740 1 -- 172.21.15.16:0/3928750669 shutdown_connections 2024-09-12T02:03:55.990 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.986+0000 7f1cc944e740 1 -- 172.21.15.16:0/3928750669 wait complete. 2024-09-12T02:03:55.990 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.986+0000 7f1cc944e740 1 -- start start 2024-09-12T02:03:55.990 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.986+0000 7f1cc944e740 1 Processor -- start 2024-09-12T02:03:55.992 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.987+0000 7f1cc944e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bae9a49db0 0x55bae9a59340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:55.993 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.987+0000 7f1cc944e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bae9a4a6f0 0x55bae9a59880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:55.993 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.987+0000 7f1cc944e740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bae9a4f780 0x55bae9a5a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:55.993 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.987+0000 7f1cc944e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc01a5390 con 0x55bae9a49db0 2024-09-12T02:03:55.993 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.987+0000 7f1cc944e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc01a6800 con 0x55bae9a4a6f0 2024-09-12T02:03:55.993 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.987+0000 7f1cc944e740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc01a5a60 con 0x55bae9a4f780 2024-09-12T02:03:55.993 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.987+0000 7f1cc944e740 10 client.0 _ll_register_callbacks cb 0x7f1cb4005850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:03:55.993 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.987+0000 7f1cbffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bae9a49db0 0x55bae9a59340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:55.993 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.987+0000 7f1cbffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bae9a49db0 0x55bae9a59340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57314/0 (socket says 172.21.15.16:57314) 2024-09-12T02:03:55.993 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.987+0000 7f1cbffff640 1 -- 172.21.15.16:0/1636465858 learned_addr learned my addr 172.21.15.16:0/1636465858 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:03:55.993 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.987+0000 7f1cc4b73640 1 --2- 172.21.15.16:0/1636465858 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bae9a4f780 0x55bae9a5a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:55.993 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.987+0000 7f1cbf7fe640 1 --2- 172.21.15.16:0/1636465858 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bae9a4a6f0 0x55bae9a59880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:55.993 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.987+0000 7f1cbffff640 1 -- 172.21.15.16:0/1636465858 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bae9a4f780 msgr2=0x55bae9a5a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:55.993 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.987+0000 7f1cbffff640 1 --2- 172.21.15.16:0/1636465858 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bae9a4f780 0x55bae9a5a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:55.994 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.987+0000 7f1cbffff640 1 -- 172.21.15.16:0/1636465858 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bae9a4a6f0 msgr2=0x55bae9a59880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:55.994 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.987+0000 7f1cbffff640 1 --2- 172.21.15.16:0/1636465858 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bae9a4a6f0 0x55bae9a59880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:55.994 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.987+0000 7f1cbffff640 1 -- 172.21.15.16:0/1636465858 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bae9a5ade0 con 0x55bae9a49db0 2024-09-12T02:03:55.994 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.987+0000 7f1cbffff640 1 --2- 172.21.15.16:0/1636465858 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bae9a49db0 0x55bae9a59340 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f1cb00a43b0 tx=0x7f1cb006d0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbe25d80a54ad834 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:55.994 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.987+0000 7f1c9f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:03:55.994 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.988+0000 7f1c9f7fe640 1 -- 172.21.15.16:0/1636465858 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb00a48e0 con 0x55bae9a49db0 2024-09-12T02:03:55.994 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.988+0000 7f1c9f7fe640 1 -- 172.21.15.16:0/1636465858 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f1cb0099f70 con 0x55bae9a49db0 2024-09-12T02:03:55.994 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.988+0000 7f1cc944e740 1 -- 172.21.15.16:0/1636465858 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1cb009bc70 con 0x55bae9a49db0 2024-09-12T02:03:55.994 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.988+0000 7f1c9f7fe640 1 -- 172.21.15.16:0/1636465858 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1cb00982e0 con 0x55bae9a49db0 2024-09-12T02:03:55.994 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.988+0000 7f1cc944e740 10 client.16011 fetch_fsmap learned FSMap version 25 2024-09-12T02:03:55.994 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.988+0000 7f1cc944e740 1 -- 172.21.15.16:0/1636465858 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55bae9acb020 con 0x55bae9a49db0 2024-09-12T02:03:55.994 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.988+0000 7f1c9f7fe640 1 -- 172.21.15.16:0/1636465858 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1cb009bc70 con 0x55bae9a49db0 2024-09-12T02:03:55.994 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.988+0000 7f1c9f7fe640 1 -- 172.21.15.16:0/1636465858 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f1cb0018b30 con 0x55bae9a49db0 2024-09-12T02:03:55.995 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.988+0000 7f1cc944e740 10 client.16011 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:03:55.995 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.988+0000 7f1cc944e740 10 client.16011 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:03:55.995 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.988+0000 7f1cc944e740 20 client.16011 populate_metadata read hostname 'smithi016' 2024-09-12T02:03:55.995 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.988+0000 7f1cc944e740 1 --2- 172.21.15.16:0/1636465858 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55bae9acb580 0x55bae9acd970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:55.995 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.988+0000 7f1cc944e740 4 client.16011 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:03:55.995 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.988+0000 7f1cc944e740 1 -- 172.21.15.16:0/1636465858 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bae99b0d70 con 0x55bae9acb580 2024-09-12T02:03:55.995 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.989+0000 7f1cbf7fe640 1 --2- 172.21.15.16:0/1636465858 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55bae9acb580 0x55bae9acd970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:55.995 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.989+0000 7f1cbf7fe640 1 --2- 172.21.15.16:0/1636465858 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55bae9acb580 0x55bae9acd970 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32accd2212ec691b server_cookie=9ff3dbae85ea7d34 in_seq=0 out_seq=0 2024-09-12T02:03:55.995 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:55.989+0000 7f1c9f7fe640 10 client.16011 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:03:56.001 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:56.001+0000 7f1c9f7fe640 1 -- 172.21.15.16:0/1636465858 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55bae99b0d70 con 0x55bae9acb580 2024-09-12T02:03:56.002 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:56.001+0000 7f1c9f7fe640 10 client.16011 handle_command_reply: tid=0 2024-09-12T02:03:56.002 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:56.001+0000 7f1cc944e740 1 client.16011 shutdown 2024-09-12T02:03:56.002 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:56.001+0000 7f1cc944e740 1 -- 172.21.15.16:0/1636465858 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bae9a49db0 msgr2=0x55bae9a59340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:56.002 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:56.001+0000 7f1cc944e740 1 --2- 172.21.15.16:0/1636465858 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bae9a49db0 0x55bae9a59340 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f1cb00a43b0 tx=0x7f1cb006d0b0 comp rx=0 tx=0).stop 2024-09-12T02:03:56.002 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:56.001+0000 7f1cc944e740 1 -- 172.21.15.16:0/1636465858 shutdown_connections 2024-09-12T02:03:56.002 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:56.001+0000 7f1cc944e740 1 --2- 172.21.15.16:0/1636465858 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55bae9acb580 0x55bae9acd970 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:56.002 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:56.001+0000 7f1cc944e740 1 --2- 172.21.15.16:0/1636465858 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bae9a4f780 0x55bae9a5a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:56.002 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:56.001+0000 7f1cc944e740 1 --2- 172.21.15.16:0/1636465858 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bae9a4a6f0 0x55bae9a59880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:56.002 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:56.001+0000 7f1c9f7fe640 0 client.16011 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:03:56.002 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:56.001+0000 7f1cc944e740 1 --2- 172.21.15.16:0/1636465858 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bae9a49db0 0x55bae9a59340 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:56.002 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:56.001+0000 7f1cc944e740 1 -- 172.21.15.16:0/1636465858 >> 172.21.15.16:0/1636465858 conn(0x55bae9a41010 msgr2=0x55bae9a41dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:56.003 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:56.001+0000 7f1cc944e740 1 -- 172.21.15.16:0/1636465858 shutdown_connections 2024-09-12T02:03:56.003 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:56.001+0000 7f1cc944e740 1 -- 172.21.15.16:0/1636465858 wait complete. 2024-09-12T02:03:56.003 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:56.001+0000 7f1cc944e740 20 client.16011 trim_cache size 0 max 16384 2024-09-12T02:03:56.003 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-09-12T02:03:56.003 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-09-12T02:03:56.003 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-09-12T02:03:56.003 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-09-12T02:03:56.004 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:56.001+0000 7f1cc65d6640 1 -- 172.21.15.16:0/992349685 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f1ca80a2cb0 msgr2=0x7f1ca80a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:56.004 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:56.001+0000 7f1cc65d6640 1 --2- 172.21.15.16:0/992349685 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f1ca80a2cb0 0x7f1ca80a5170 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f1cb8031ae0 tx=0x7f1cb803e040 comp rx=0 tx=0).stop 2024-09-12T02:03:56.004 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:56.001+0000 7f1cc65d6640 1 -- 172.21.15.16:0/992349685 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1cc01025d0 msgr2=0x7f1cc0197e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:56.004 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:56.001+0000 7f1cc65d6640 1 --2- 172.21.15.16:0/992349685 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1cc01025d0 0x7f1cc0197e40 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f1cb000e9f0 tx=0x7f1cb000eeb0 comp rx=0 tx=0).stop 2024-09-12T02:03:56.004 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:56.001+0000 7f1cc65d6640 1 -- 172.21.15.16:0/992349685 shutdown_connections 2024-09-12T02:03:56.004 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:56.001+0000 7f1cc65d6640 1 --2- 172.21.15.16:0/992349685 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f1ca80a2cb0 0x7f1ca80a5170 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:56.004 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:56.001+0000 7f1cc65d6640 1 --2- 172.21.15.16:0/992349685 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1cc0107830 0x7f1cc019f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:56.005 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:56.001+0000 7f1cc65d6640 1 --2- 172.21.15.16:0/992349685 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1cc0104f00 0x7f1cc0198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:56.005 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:56.001+0000 7f1cc65d6640 1 --2- 172.21.15.16:0/992349685 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1cc01025d0 0x7f1cc0197e40 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:56.005 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:56.001+0000 7f1cc65d6640 1 -- 172.21.15.16:0/992349685 >> 172.21.15.16:0/992349685 conn(0x7f1cc00fc1b0 msgr2=0x7f1cc0102f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:56.005 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:56.001+0000 7f1cc65d6640 1 -- 172.21.15.16:0/992349685 shutdown_connections 2024-09-12T02:03:56.005 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:03:56.001+0000 7f1cc65d6640 1 -- 172.21.15.16:0/992349685 wait complete. 2024-09-12T02:03:57.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:03:57 smithi184 ceph-mon[35593]: pgmap v396: 97 pgs: 97 active+clean; 1.2 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 6.2 KiB/s wr, 10 op/s 2024-09-12T02:03:57.921 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:03:57 smithi033 ceph-mon[35332]: pgmap v396: 97 pgs: 97 active+clean; 1.2 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 6.2 KiB/s wr, 10 op/s 2024-09-12T02:03:57.933 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:03:57 smithi016 ceph-mon[30947]: pgmap v396: 97 pgs: 97 active+clean; 1.2 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 6.2 KiB/s wr, 10 op/s 2024-09-12T02:03:59.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:03:59 smithi184 ceph-mon[35593]: pgmap v397: 97 pgs: 97 active+clean; 1.2 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 5.9 KiB/s wr, 7 op/s 2024-09-12T02:03:59.918 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:03:59 smithi033 ceph-mon[35332]: pgmap v397: 97 pgs: 97 active+clean; 1.2 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 5.9 KiB/s wr, 7 op/s 2024-09-12T02:03:59.936 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:03:59 smithi016 ceph-mon[30947]: pgmap v397: 97 pgs: 97 active+clean; 1.2 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 5.9 KiB/s wr, 7 op/s 2024-09-12T02:04:01.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:04:01 smithi184 ceph-mon[35593]: pgmap v398: 97 pgs: 97 active+clean; 1.2 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-09-12T02:04:01.920 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:04:01 smithi033 ceph-mon[35332]: pgmap v398: 97 pgs: 97 active+clean; 1.2 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-09-12T02:04:01.933 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:04:01 smithi016 ceph-mon[30947]: pgmap v398: 97 pgs: 97 active+clean; 1.2 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-09-12T02:04:03.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:04:03 smithi184 ceph-mon[35593]: pgmap v399: 97 pgs: 97 active+clean; 1.2 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-09-12T02:04:03.918 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:04:03 smithi033 ceph-mon[35332]: pgmap v399: 97 pgs: 97 active+clean; 1.2 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-09-12T02:04:03.933 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:04:03 smithi016 ceph-mon[30947]: pgmap v399: 97 pgs: 97 active+clean; 1.2 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-09-12T02:04:04.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:04:04 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:04:04.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:04:04 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T02:04:04.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:04:04 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T02:04:04.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:04:04 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T02:04:04.878 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:04:04 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T02:04:04.878 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:04:04 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:04:04.878 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:04:04 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:04:04.878 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:04:04 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T02:04:04.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:04:04 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:04:04.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:04:04 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T02:04:04.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:04:04 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T02:04:04.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:04:04 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T02:04:04.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:04:04 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T02:04:04.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:04:04 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:04:04.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:04:04 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:04:04.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:04:04 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T02:04:04.933 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:04:04 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:04:04.933 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:04:04 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T02:04:04.933 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:04:04 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T02:04:04.933 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:04:04 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T02:04:04.934 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:04:04 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T02:04:04.934 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:04:04 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:04:04.934 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:04:04 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:04:04.934 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:04:04 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T02:04:06.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:04:06 smithi184 ceph-mon[35593]: pgmap v400: 97 pgs: 97 active+clean; 1.3 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-09-12T02:04:06.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:04:06 smithi033 ceph-mon[35332]: pgmap v400: 97 pgs: 97 active+clean; 1.3 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-09-12T02:04:06.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:04:06 smithi016 ceph-mon[30947]: pgmap v400: 97 pgs: 97 active+clean; 1.3 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-09-12T02:04:08.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:04:08 smithi184 ceph-mon[35593]: pgmap v401: 97 pgs: 97 active+clean; 1.3 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-09-12T02:04:08.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:04:08 smithi033 ceph-mon[35332]: pgmap v401: 97 pgs: 97 active+clean; 1.3 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-09-12T02:04:08.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:04:08 smithi016 ceph-mon[30947]: pgmap v401: 97 pgs: 97 active+clean; 1.3 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-09-12T02:04:09.104 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.103+0000 7f976a4ed640 1 -- 172.21.15.16:0/4232215252 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9764101c50 msgr2=0x7f97641020d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:09.104 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.103+0000 7f976a4ed640 1 --2- 172.21.15.16:0/4232215252 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9764101c50 0x7f97641020d0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f9758009900 tx=0x7f975802a410 comp rx=0 tx=0).stop 2024-09-12T02:04:09.104 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.103+0000 7f976a4ed640 1 -- 172.21.15.16:0/4232215252 shutdown_connections 2024-09-12T02:04:09.104 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.103+0000 7f976a4ed640 1 --2- 172.21.15.16:0/4232215252 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9764102610 0x7f9764108ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:09.104 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.103+0000 7f976a4ed640 1 --2- 172.21.15.16:0/4232215252 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9764101c50 0x7f97641020d0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:09.104 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.103+0000 7f976a4ed640 1 --2- 172.21.15.16:0/4232215252 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9764100a60 0x7f9764100e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:09.104 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.103+0000 7f976a4ed640 1 -- 172.21.15.16:0/4232215252 >> 172.21.15.16:0/4232215252 conn(0x7f97640fc1e0 msgr2=0x7f97640fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:04:09.104 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.103+0000 7f976a4ed640 1 -- 172.21.15.16:0/4232215252 shutdown_connections 2024-09-12T02:04:09.104 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.103+0000 7f976a4ed640 1 -- 172.21.15.16:0/4232215252 wait complete. 2024-09-12T02:04:09.105 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.104+0000 7f976a4ed640 1 Processor -- start 2024-09-12T02:04:09.105 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.104+0000 7f976a4ed640 1 -- start start 2024-09-12T02:04:09.106 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.104+0000 7f976a4ed640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9764100a60 0x7f9764197de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:09.107 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.104+0000 7f976a4ed640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9764101c50 0x7f9764198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:09.107 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.104+0000 7f976a4ed640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9764102610 0x7f976419f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:09.107 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.104+0000 7f976a4ed640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f976410c6c0 con 0x7f9764101c50 2024-09-12T02:04:09.107 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.104+0000 7f976a4ed640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f976410c540 con 0x7f9764102610 2024-09-12T02:04:09.107 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.104+0000 7f976a4ed640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f976410c840 con 0x7f9764100a60 2024-09-12T02:04:09.107 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.105+0000 7f97637fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9764101c50 0x7f9764198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:09.107 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.105+0000 7f97637fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9764101c50 0x7f9764198320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49814/0 (socket says 172.21.15.16:49814) 2024-09-12T02:04:09.107 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.105+0000 7f97637fe640 1 -- 172.21.15.16:0/1663301684 learned_addr learned my addr 172.21.15.16:0/1663301684 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:04:09.107 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.105+0000 7f9763fff640 1 --2- 172.21.15.16:0/1663301684 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9764100a60 0x7f9764197de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:09.107 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.105+0000 7f9768a8a640 1 --2- 172.21.15.16:0/1663301684 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9764102610 0x7f976419f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:09.107 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.105+0000 7f97637fe640 1 -- 172.21.15.16:0/1663301684 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9764100a60 msgr2=0x7f9764197de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:09.107 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.105+0000 7f97637fe640 1 --2- 172.21.15.16:0/1663301684 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9764100a60 0x7f9764197de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:09.108 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.105+0000 7f97637fe640 1 -- 172.21.15.16:0/1663301684 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9764102610 msgr2=0x7f976419f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:09.108 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.105+0000 7f97637fe640 1 --2- 172.21.15.16:0/1663301684 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9764102610 0x7f976419f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:09.108 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.105+0000 7f97637fe640 1 -- 172.21.15.16:0/1663301684 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f976419fa00 con 0x7f9764101c50 2024-09-12T02:04:09.108 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.105+0000 7f9763fff640 1 --2- 172.21.15.16:0/1663301684 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9764100a60 0x7f9764197de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:04:09.108 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.105+0000 7f97637fe640 1 --2- 172.21.15.16:0/1663301684 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9764101c50 0x7f9764198320 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f9758009230 tx=0x7f9758004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:04:09.108 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.105+0000 7f97617fa640 1 -- 172.21.15.16:0/1663301684 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f975803f030 con 0x7f9764101c50 2024-09-12T02:04:09.108 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.105+0000 7f97617fa640 1 -- 172.21.15.16:0/1663301684 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9758003ce0 con 0x7f9764101c50 2024-09-12T02:04:09.108 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.105+0000 7f976a4ed640 1 -- 172.21.15.16:0/1663301684 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f976419fc90 con 0x7f9764101c50 2024-09-12T02:04:09.108 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.105+0000 7f97617fa640 1 -- 172.21.15.16:0/1663301684 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9758035070 con 0x7f9764101c50 2024-09-12T02:04:09.108 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.105+0000 7f976a4ed640 1 -- 172.21.15.16:0/1663301684 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97641a0180 con 0x7f9764101c50 2024-09-12T02:04:09.114 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.108+0000 7f97617fa640 1 -- 172.21.15.16:0/1663301684 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f9758042030 con 0x7f9764101c50 2024-09-12T02:04:09.114 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.108+0000 7f976d289740 1 Processor -- start 2024-09-12T02:04:09.114 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.108+0000 7f976d289740 1 -- start start 2024-09-12T02:04:09.114 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.109+0000 7f976d289740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557def17f9a0 0x557def17fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:09.114 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.109+0000 7f976d289740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557def1802e0 0x557def184da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:09.114 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.109+0000 7f976d289740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557def185370 0x557def187830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:09.114 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.109+0000 7f976d289740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x557def0e32f0 con 0x557def185370 2024-09-12T02:04:09.115 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.109+0000 7f976d289740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x557def08b270 con 0x557def1802e0 2024-09-12T02:04:09.115 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.109+0000 7f976d289740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x557def0df2b0 con 0x557def17f9a0 2024-09-12T02:04:09.115 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.109+0000 7f97637fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557def185370 0x557def187830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:09.115 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.109+0000 7f97637fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557def185370 0x557def187830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49816/0 (socket says 172.21.15.16:49816) 2024-09-12T02:04:09.115 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.109+0000 7f97637fe640 1 -- 172.21.15.16:0/104955930 learned_addr learned my addr 172.21.15.16:0/104955930 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:04:09.115 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.109+0000 7f9768a8a640 1 --2- 172.21.15.16:0/104955930 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557def17f9a0 0x557def17fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:09.115 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.109+0000 7f9763fff640 1 --2- 172.21.15.16:0/104955930 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557def1802e0 0x557def184da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:09.115 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.109+0000 7f97617fa640 1 --2- 172.21.15.16:0/1663301684 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f974c0a2c30 0x7f974c0a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:09.115 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.109+0000 7f97637fe640 1 -- 172.21.15.16:0/104955930 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557def17f9a0 msgr2=0x557def17fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:09.115 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.109+0000 7f97637fe640 1 --2- 172.21.15.16:0/104955930 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557def17f9a0 0x557def17fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:09.115 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.109+0000 7f97637fe640 1 -- 172.21.15.16:0/104955930 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557def1802e0 msgr2=0x557def184da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:09.115 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.109+0000 7f97637fe640 1 --2- 172.21.15.16:0/104955930 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557def1802e0 0x557def184da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:09.115 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.109+0000 7f97637fe640 1 -- 172.21.15.16:0/104955930 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557def08a340 con 0x557def185370 2024-09-12T02:04:09.115 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.109+0000 7f9763fff640 1 --2- 172.21.15.16:0/104955930 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557def1802e0 0x557def184da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:04:09.116 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.109+0000 7f9768a8a640 1 --2- 172.21.15.16:0/1663301684 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f974c0a2c30 0x7f974c0a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:09.116 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.109+0000 7f97617fa640 1 -- 172.21.15.16:0/1663301684 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f97580ef700 con 0x7f9764101c50 2024-09-12T02:04:09.116 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.110+0000 7f97637fe640 1 --2- 172.21.15.16:0/104955930 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557def185370 0x557def187830 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f975804b000 tx=0x7f9758042d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d5f097b1f545693 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:04:09.116 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.110+0000 7f9768a8a640 1 --2- 172.21.15.16:0/1663301684 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f974c0a2c30 0x7f974c0a50f0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f975c005fd0 tx=0x7f975c004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:04:09.116 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.110+0000 7f974a7fc640 1 -- 172.21.15.16:0/104955930 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97580bb8c0 con 0x557def185370 2024-09-12T02:04:09.116 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.110+0000 7f974a7fc640 1 -- 172.21.15.16:0/104955930 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9758087040 con 0x557def185370 2024-09-12T02:04:09.116 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.110+0000 7f974a7fc640 1 -- 172.21.15.16:0/104955930 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97580c0310 con 0x557def185370 2024-09-12T02:04:09.116 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.110+0000 7f976d289740 1 -- 172.21.15.16:0/104955930 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557def185370 msgr2=0x557def187830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:09.116 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.111+0000 7f976d289740 1 --2- 172.21.15.16:0/104955930 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557def185370 0x557def187830 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f975804b000 tx=0x7f9758042d70 comp rx=0 tx=0).stop 2024-09-12T02:04:09.116 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.111+0000 7f976d289740 1 -- 172.21.15.16:0/104955930 shutdown_connections 2024-09-12T02:04:09.116 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.111+0000 7f976d289740 1 --2- 172.21.15.16:0/104955930 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557def185370 0x557def187830 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:09.116 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.111+0000 7f976d289740 1 --2- 172.21.15.16:0/104955930 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557def1802e0 0x557def184da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:09.117 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.111+0000 7f976d289740 1 --2- 172.21.15.16:0/104955930 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557def17f9a0 0x557def17fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:09.117 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.111+0000 7f976d289740 1 -- 172.21.15.16:0/104955930 >> 172.21.15.16:0/104955930 conn(0x557def17b010 msgr2=0x557def17b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:04:09.117 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.111+0000 7f976d289740 1 -- 172.21.15.16:0/104955930 shutdown_connections 2024-09-12T02:04:09.117 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.111+0000 7f976d289740 1 -- 172.21.15.16:0/104955930 wait complete. 2024-09-12T02:04:09.117 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.111+0000 7f976d289740 1 -- start start 2024-09-12T02:04:09.117 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.111+0000 7f976d289740 1 Processor -- start 2024-09-12T02:04:09.117 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.112+0000 7f976d289740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557def17f9a0 0x557def189860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:09.117 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.112+0000 7f976d289740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557def1802e0 0x557def189da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:09.117 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.112+0000 7f976d289740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557def185370 0x557def190d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:09.117 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.112+0000 7f976d289740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x557def0e32f0 con 0x557def185370 2024-09-12T02:04:09.117 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.112+0000 7f976d289740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x557def06c290 con 0x557def1802e0 2024-09-12T02:04:09.117 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.112+0000 7f976d289740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x557def188100 con 0x557def17f9a0 2024-09-12T02:04:09.117 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.112+0000 7f976d289740 10 client.0 _ll_register_callbacks cb 0x557def09d140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:04:09.118 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.112+0000 7f9768a8a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557def185370 0x557def190d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:09.118 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.112+0000 7f9768a8a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557def185370 0x557def190d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49824/0 (socket says 172.21.15.16:49824) 2024-09-12T02:04:09.118 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.112+0000 7f9768a8a640 1 -- 172.21.15.16:0/3228701840 learned_addr learned my addr 172.21.15.16:0/3228701840 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:04:09.118 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.112+0000 7f97637fe640 1 --2- 172.21.15.16:0/3228701840 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557def1802e0 0x557def189da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:09.118 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.112+0000 7f9763fff640 1 --2- 172.21.15.16:0/3228701840 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557def17f9a0 0x557def189860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:09.118 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.112+0000 7f9768a8a640 1 -- 172.21.15.16:0/3228701840 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557def17f9a0 msgr2=0x557def189860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:09.118 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.112+0000 7f9768a8a640 1 --2- 172.21.15.16:0/3228701840 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557def17f9a0 0x557def189860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:09.118 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.112+0000 7f9768a8a640 1 -- 172.21.15.16:0/3228701840 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557def1802e0 msgr2=0x557def189da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:09.118 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.112+0000 7f9768a8a640 1 --2- 172.21.15.16:0/3228701840 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557def1802e0 0x557def189da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:09.118 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.112+0000 7f9768a8a640 1 -- 172.21.15.16:0/3228701840 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557deedf6210 con 0x557def185370 2024-09-12T02:04:09.118 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.112+0000 7f9768a8a640 1 --2- 172.21.15.16:0/3228701840 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557def185370 0x557def190d70 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f975c013970 tx=0x7f975c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd7f7663a07044f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:04:09.118 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.112+0000 7f97437fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:04:09.118 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.113+0000 7f97437fe640 1 -- 172.21.15.16:0/3228701840 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f975c01ed70 con 0x557def185370 2024-09-12T02:04:09.119 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.113+0000 7f97437fe640 1 -- 172.21.15.16:0/3228701840 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f975c028470 con 0x557def185370 2024-09-12T02:04:09.119 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.113+0000 7f976d289740 1 -- 172.21.15.16:0/3228701840 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9758087040 con 0x557def185370 2024-09-12T02:04:09.119 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.113+0000 7f97437fe640 1 -- 172.21.15.16:0/3228701840 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f975c004b20 con 0x557def185370 2024-09-12T02:04:09.119 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.113+0000 7f976d289740 10 client.16023 fetch_fsmap learned FSMap version 25 2024-09-12T02:04:09.119 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.113+0000 7f976d289740 1 -- 172.21.15.16:0/3228701840 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x557deeec2620 con 0x557def185370 2024-09-12T02:04:09.119 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.113+0000 7f97437fe640 1 -- 172.21.15.16:0/3228701840 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9758087040 con 0x557def185370 2024-09-12T02:04:09.119 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.113+0000 7f97437fe640 1 -- 172.21.15.16:0/3228701840 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f975c00f040 con 0x557def185370 2024-09-12T02:04:09.119 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.113+0000 7f976d289740 10 client.16023 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:04:09.119 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.113+0000 7f976d289740 10 client.16023 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:04:09.119 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.113+0000 7f976d289740 20 client.16023 populate_metadata read hostname 'smithi016' 2024-09-12T02:04:09.119 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.113+0000 7f976d289740 1 --2- 172.21.15.16:0/3228701840 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x557def195590 0x557def1979f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:09.119 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.113+0000 7f976d289740 4 client.16023 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:04:09.119 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.113+0000 7f976d289740 1 -- 172.21.15.16:0/3228701840 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557def0f0a20 con 0x557def195590 2024-09-12T02:04:09.120 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.114+0000 7f9763fff640 1 --2- 172.21.15.16:0/3228701840 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x557def195590 0x557def1979f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:09.120 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.114+0000 7f9763fff640 1 --2- 172.21.15.16:0/3228701840 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x557def195590 0x557def1979f0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5820aa3630b3313 server_cookie=faa141e7f75a9848 in_seq=0 out_seq=0 2024-09-12T02:04:09.120 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.114+0000 7f97437fe640 10 client.16023 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:04:09.121 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.115+0000 7f97437fe640 1 -- 172.21.15.16:0/3228701840 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557def0f0a20 con 0x557def195590 2024-09-12T02:04:09.121 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.115+0000 7f97437fe640 10 client.16023 handle_command_reply: tid=0 2024-09-12T02:04:09.121 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.115+0000 7f976d289740 1 client.16023 shutdown 2024-09-12T02:04:09.121 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.115+0000 7f976d289740 1 -- 172.21.15.16:0/3228701840 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557def185370 msgr2=0x557def190d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:09.121 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.115+0000 7f976d289740 1 --2- 172.21.15.16:0/3228701840 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557def185370 0x557def190d70 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f975c013970 tx=0x7f975c004ab0 comp rx=0 tx=0).stop 2024-09-12T02:04:09.121 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.115+0000 7f976d289740 1 -- 172.21.15.16:0/3228701840 shutdown_connections 2024-09-12T02:04:09.121 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.115+0000 7f976d289740 1 --2- 172.21.15.16:0/3228701840 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x557def195590 0x557def1979f0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:09.121 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.115+0000 7f976d289740 1 --2- 172.21.15.16:0/3228701840 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557def185370 0x557def190d70 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:09.121 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.115+0000 7f976d289740 1 --2- 172.21.15.16:0/3228701840 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557def1802e0 0x557def189da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:09.122 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.115+0000 7f97437fe640 0 client.16023 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:04:09.122 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.115+0000 7f976d289740 1 --2- 172.21.15.16:0/3228701840 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557def17f9a0 0x557def189860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:09.122 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.115+0000 7f976d289740 1 -- 172.21.15.16:0/3228701840 >> 172.21.15.16:0/3228701840 conn(0x557def17b010 msgr2=0x557def17c820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:04:09.122 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.115+0000 7f976d289740 1 -- 172.21.15.16:0/3228701840 shutdown_connections 2024-09-12T02:04:09.122 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.115+0000 7f976d289740 1 -- 172.21.15.16:0/3228701840 wait complete. 2024-09-12T02:04:09.122 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.115+0000 7f976d289740 20 client.16023 trim_cache size 0 max 16384 2024-09-12T02:04:09.131 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.130+0000 7f976d289740 1 Processor -- start 2024-09-12T02:04:09.131 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.130+0000 7f976d289740 1 -- start start 2024-09-12T02:04:09.131 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.130+0000 7f976d289740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557def17f9a0 0x557def17bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:09.131 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.130+0000 7f976d289740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557def1802e0 0x557def190d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:09.131 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.130+0000 7f976d289740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557def185370 0x557def17c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:09.131 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.130+0000 7f976d289740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f97641ab770 con 0x557def185370 2024-09-12T02:04:09.132 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.130+0000 7f976d289740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f97641a9040 con 0x557def1802e0 2024-09-12T02:04:09.132 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.130+0000 7f976d289740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f97641a6e90 con 0x557def17f9a0 2024-09-12T02:04:09.132 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.130+0000 7f9768a8a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557def185370 0x557def17c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:09.132 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.130+0000 7f9768a8a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557def185370 0x557def17c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49828/0 (socket says 172.21.15.16:49828) 2024-09-12T02:04:09.132 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.130+0000 7f9768a8a640 1 -- 172.21.15.16:0/134164760 learned_addr learned my addr 172.21.15.16:0/134164760 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:04:09.132 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.130+0000 7f97637fe640 1 --2- 172.21.15.16:0/134164760 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557def1802e0 0x557def190d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:09.132 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.130+0000 7f9763fff640 1 --2- 172.21.15.16:0/134164760 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557def17f9a0 0x557def17bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:09.132 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.131+0000 7f9768a8a640 1 -- 172.21.15.16:0/134164760 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557def17f9a0 msgr2=0x557def17bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:09.132 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.131+0000 7f9768a8a640 1 --2- 172.21.15.16:0/134164760 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557def17f9a0 0x557def17bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:09.132 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.131+0000 7f9768a8a640 1 -- 172.21.15.16:0/134164760 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557def1802e0 msgr2=0x557def190d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:09.132 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.131+0000 7f9768a8a640 1 --2- 172.21.15.16:0/134164760 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557def1802e0 0x557def190d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:09.132 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.131+0000 7f9768a8a640 1 -- 172.21.15.16:0/134164760 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557def17bca0 con 0x557def185370 2024-09-12T02:04:09.132 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.131+0000 7f97637fe640 1 --2- 172.21.15.16:0/134164760 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557def1802e0 0x557def190d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:04:09.133 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.131+0000 7f9768a8a640 1 --2- 172.21.15.16:0/134164760 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557def185370 0x557def17c400 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f975c010f10 tx=0x7f975c01e7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c62a7c83062856a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:04:09.133 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.131+0000 7f9749ffb640 1 -- 172.21.15.16:0/134164760 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f975c01f800 con 0x557def185370 2024-09-12T02:04:09.133 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.131+0000 7f9749ffb640 1 -- 172.21.15.16:0/134164760 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f975c038020 con 0x557def185370 2024-09-12T02:04:09.133 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.131+0000 7f9749ffb640 1 -- 172.21.15.16:0/134164760 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f975c028a40 con 0x557def185370 2024-09-12T02:04:09.136 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.132+0000 7f976d289740 1 -- 172.21.15.16:0/134164760 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557def185370 msgr2=0x557def17c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:09.136 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.132+0000 7f976d289740 1 --2- 172.21.15.16:0/134164760 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557def185370 0x557def17c400 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f975c010f10 tx=0x7f975c01e7b0 comp rx=0 tx=0).stop 2024-09-12T02:04:09.136 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.132+0000 7f976d289740 1 -- 172.21.15.16:0/134164760 shutdown_connections 2024-09-12T02:04:09.136 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.132+0000 7f976d289740 1 --2- 172.21.15.16:0/134164760 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557def185370 0x557def17c400 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:09.136 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.132+0000 7f976d289740 1 --2- 172.21.15.16:0/134164760 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557def1802e0 0x557def190d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:09.137 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.132+0000 7f976d289740 1 --2- 172.21.15.16:0/134164760 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557def17f9a0 0x557def17bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:09.137 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.132+0000 7f976d289740 1 -- 172.21.15.16:0/134164760 >> 172.21.15.16:0/134164760 conn(0x557def17b010 msgr2=0x557def1814c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:04:09.137 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.132+0000 7f976d289740 1 -- 172.21.15.16:0/134164760 shutdown_connections 2024-09-12T02:04:09.137 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.132+0000 7f976d289740 1 -- 172.21.15.16:0/134164760 wait complete. 2024-09-12T02:04:09.137 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.132+0000 7f976d289740 1 -- start start 2024-09-12T02:04:09.137 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.132+0000 7f976d289740 1 Processor -- start 2024-09-12T02:04:09.137 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.133+0000 7f976d289740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557def17f9a0 0x557def18e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:09.137 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.133+0000 7f976d289740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557def1802e0 0x557def18ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:09.137 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.133+0000 7f976d289740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557def185370 0x557def18fa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:09.137 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.133+0000 7f976d289740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f97641a7500 con 0x557def1802e0 2024-09-12T02:04:09.137 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.133+0000 7f976d289740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f97641a8970 con 0x557def185370 2024-09-12T02:04:09.137 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.133+0000 7f976d289740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f97641a7bd0 con 0x557def17f9a0 2024-09-12T02:04:09.137 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.133+0000 7f976d289740 10 client.0 _ll_register_callbacks cb 0x7f975c009cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:04:09.138 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.133+0000 7f97637fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557def1802e0 0x557def18ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:09.138 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.133+0000 7f97637fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557def1802e0 0x557def18ea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49842/0 (socket says 172.21.15.16:49842) 2024-09-12T02:04:09.138 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.133+0000 7f97637fe640 1 -- 172.21.15.16:0/1946847273 learned_addr learned my addr 172.21.15.16:0/1946847273 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:04:09.138 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.133+0000 7f9763fff640 1 --2- 172.21.15.16:0/1946847273 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557def17f9a0 0x557def18e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:09.138 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.133+0000 7f9768a8a640 1 --2- 172.21.15.16:0/1946847273 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557def185370 0x557def18fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:09.138 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.133+0000 7f97637fe640 1 -- 172.21.15.16:0/1946847273 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557def17f9a0 msgr2=0x557def18e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:09.138 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.133+0000 7f97637fe640 1 --2- 172.21.15.16:0/1946847273 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557def17f9a0 0x557def18e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:09.138 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.133+0000 7f97637fe640 1 -- 172.21.15.16:0/1946847273 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557def185370 msgr2=0x557def18fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:09.138 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.133+0000 7f97637fe640 1 --2- 172.21.15.16:0/1946847273 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557def185370 0x557def18fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:09.138 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.133+0000 7f97637fe640 1 -- 172.21.15.16:0/1946847273 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557def190050 con 0x557def1802e0 2024-09-12T02:04:09.138 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.133+0000 7f97637fe640 1 --2- 172.21.15.16:0/1946847273 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557def1802e0 0x557def18ea80 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f97580314e0 tx=0x7f975802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d2b459a73228a82 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:04:09.138 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.133+0000 7f97437fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:04:09.138 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.133+0000 7f97437fe640 1 -- 172.21.15.16:0/1946847273 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97580bb8c0 con 0x557def1802e0 2024-09-12T02:04:09.139 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.133+0000 7f97437fe640 1 -- 172.21.15.16:0/1946847273 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f97580b85d0 con 0x557def1802e0 2024-09-12T02:04:09.139 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.133+0000 7f976d289740 1 -- 172.21.15.16:0/1946847273 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f975c038020 con 0x557def1802e0 2024-09-12T02:04:09.139 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.134+0000 7f97437fe640 1 -- 172.21.15.16:0/1946847273 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f97580bbaa0 con 0x557def1802e0 2024-09-12T02:04:09.139 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.134+0000 7f976d289740 10 client.16029 fetch_fsmap learned FSMap version 25 2024-09-12T02:04:09.139 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.134+0000 7f976d289740 1 -- 172.21.15.16:0/1946847273 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x557def200380 con 0x557def1802e0 2024-09-12T02:04:09.139 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.134+0000 7f97437fe640 1 -- 172.21.15.16:0/1946847273 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f975c038020 con 0x557def1802e0 2024-09-12T02:04:09.139 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.134+0000 7f97437fe640 1 -- 172.21.15.16:0/1946847273 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f97580f5150 con 0x557def1802e0 2024-09-12T02:04:09.139 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.134+0000 7f976d289740 10 client.16029 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:04:09.139 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.134+0000 7f976d289740 10 client.16029 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:04:09.139 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.134+0000 7f976d289740 20 client.16029 populate_metadata read hostname 'smithi016' 2024-09-12T02:04:09.139 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.134+0000 7f976d289740 1 --2- 172.21.15.16:0/1946847273 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x557def200bf0 0x557def203010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:09.139 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.134+0000 7f976d289740 4 client.16029 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:04:09.139 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.134+0000 7f976d289740 1 -- 172.21.15.16:0/1946847273 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557def0ead70 con 0x557def200bf0 2024-09-12T02:04:09.140 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.134+0000 7f9763fff640 1 --2- 172.21.15.16:0/1946847273 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x557def200bf0 0x557def203010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:09.140 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.135+0000 7f9763fff640 1 --2- 172.21.15.16:0/1946847273 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x557def200bf0 0x557def203010 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db24f766c34caa0d server_cookie=79238abf3447de5c in_seq=0 out_seq=0 2024-09-12T02:04:09.140 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.135+0000 7f97437fe640 10 client.16029 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:04:09.148 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.147+0000 7f97437fe640 1 -- 172.21.15.16:0/1946847273 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557def0ead70 con 0x557def200bf0 2024-09-12T02:04:09.148 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.147+0000 7f97437fe640 10 client.16029 handle_command_reply: tid=0 2024-09-12T02:04:09.148 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.147+0000 7f976d289740 1 client.16029 shutdown 2024-09-12T02:04:09.148 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.147+0000 7f976d289740 1 -- 172.21.15.16:0/1946847273 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557def1802e0 msgr2=0x557def18ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:09.148 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.147+0000 7f976d289740 1 --2- 172.21.15.16:0/1946847273 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557def1802e0 0x557def18ea80 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f97580314e0 tx=0x7f975802a410 comp rx=0 tx=0).stop 2024-09-12T02:04:09.148 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.147+0000 7f976d289740 1 -- 172.21.15.16:0/1946847273 shutdown_connections 2024-09-12T02:04:09.148 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.147+0000 7f976d289740 1 --2- 172.21.15.16:0/1946847273 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x557def200bf0 0x557def203010 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:09.148 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.147+0000 7f976d289740 1 --2- 172.21.15.16:0/1946847273 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557def185370 0x557def18fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:09.148 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.147+0000 7f976d289740 1 --2- 172.21.15.16:0/1946847273 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557def1802e0 0x557def18ea80 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:09.148 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.147+0000 7f97437fe640 0 client.16029 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:04:09.149 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.147+0000 7f976d289740 1 --2- 172.21.15.16:0/1946847273 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557def17f9a0 0x557def18e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:09.149 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.147+0000 7f976d289740 1 -- 172.21.15.16:0/1946847273 >> 172.21.15.16:0/1946847273 conn(0x557def17b010 msgr2=0x557def1799a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:04:09.149 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.148+0000 7f976d289740 1 -- 172.21.15.16:0/1946847273 shutdown_connections 2024-09-12T02:04:09.149 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.148+0000 7f976d289740 1 -- 172.21.15.16:0/1946847273 wait complete. 2024-09-12T02:04:09.149 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.148+0000 7f976d289740 20 client.16029 trim_cache size 0 max 16384 2024-09-12T02:04:09.149 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-09-12T02:04:09.149 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-09-12T02:04:09.149 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-09-12T02:04:09.149 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-09-12T02:04:09.150 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.148+0000 7f976a4ed640 1 -- 172.21.15.16:0/1663301684 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f974c0a2c30 msgr2=0x7f974c0a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:09.150 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.148+0000 7f976a4ed640 1 --2- 172.21.15.16:0/1663301684 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f974c0a2c30 0x7f974c0a50f0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f975c005fd0 tx=0x7f975c004a40 comp rx=0 tx=0).stop 2024-09-12T02:04:09.150 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.148+0000 7f976a4ed640 1 -- 172.21.15.16:0/1663301684 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9764101c50 msgr2=0x7f9764198320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:09.150 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.148+0000 7f976a4ed640 1 --2- 172.21.15.16:0/1663301684 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9764101c50 0x7f9764198320 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f9758009230 tx=0x7f9758004360 comp rx=0 tx=0).stop 2024-09-12T02:04:09.151 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.148+0000 7f976a4ed640 1 -- 172.21.15.16:0/1663301684 shutdown_connections 2024-09-12T02:04:09.151 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.148+0000 7f976a4ed640 1 --2- 172.21.15.16:0/1663301684 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f974c0a2c30 0x7f974c0a50f0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:09.151 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.148+0000 7f976a4ed640 1 --2- 172.21.15.16:0/1663301684 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9764102610 0x7f976419f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:09.151 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.148+0000 7f976a4ed640 1 --2- 172.21.15.16:0/1663301684 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9764101c50 0x7f9764198320 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:09.151 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.148+0000 7f976a4ed640 1 --2- 172.21.15.16:0/1663301684 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9764100a60 0x7f9764197de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:09.151 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.148+0000 7f976a4ed640 1 -- 172.21.15.16:0/1663301684 >> 172.21.15.16:0/1663301684 conn(0x7f97640fc1e0 msgr2=0x7f97640fdcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:04:09.151 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.148+0000 7f976a4ed640 1 -- 172.21.15.16:0/1663301684 shutdown_connections 2024-09-12T02:04:09.151 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:09.148+0000 7f976a4ed640 1 -- 172.21.15.16:0/1663301684 wait complete. 2024-09-12T02:04:10.506 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T02:04:10.535 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:04:10 smithi016 ceph-mon[30947]: pgmap v402: 97 pgs: 97 active+clean; 1.3 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-09-12T02:04:10.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:04:10 smithi184 ceph-mon[35593]: pgmap v402: 97 pgs: 97 active+clean; 1.3 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-09-12T02:04:10.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:04:10 smithi033 ceph-mon[35332]: pgmap v402: 97 pgs: 97 active+clean; 1.3 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-09-12T02:04:10.778 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:04:11.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.242+0000 7f57ef6f3640 1 -- 172.21.15.16:0/1402535464 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57e8100ca0 msgr2=0x7f57e8106840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:11.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.242+0000 7f57ef6f3640 1 --2- 172.21.15.16:0/1402535464 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57e8100ca0 0x7f57e8106840 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f57d8009930 tx=0x7f57d802a420 comp rx=0 tx=0).stop 2024-09-12T02:04:11.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.242+0000 7f57ef6f3640 1 -- 172.21.15.16:0/1402535464 shutdown_connections 2024-09-12T02:04:11.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.242+0000 7f57ef6f3640 1 --2- 172.21.15.16:0/1402535464 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f57e8106f50 0x7f57e810eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:11.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.242+0000 7f57ef6f3640 1 --2- 172.21.15.16:0/1402535464 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57e8100ca0 0x7f57e8106840 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:11.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.242+0000 7f57ef6f3640 1 --2- 172.21.15.16:0/1402535464 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57e81002f0 0x7f57e81006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:11.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.242+0000 7f57ef6f3640 1 -- 172.21.15.16:0/1402535464 >> 172.21.15.16:0/1402535464 conn(0x7f57e8077210 msgr2=0x7f57e80fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:04:11.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.242+0000 7f57ef6f3640 1 -- 172.21.15.16:0/1402535464 shutdown_connections 2024-09-12T02:04:11.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.242+0000 7f57ef6f3640 1 -- 172.21.15.16:0/1402535464 wait complete. 2024-09-12T02:04:11.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.243+0000 7f57ef6f3640 1 Processor -- start 2024-09-12T02:04:11.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.243+0000 7f57ef6f3640 1 -- start start 2024-09-12T02:04:11.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.243+0000 7f57ef6f3640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f57e81002f0 0x7f57e81984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:11.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.243+0000 7f57ef6f3640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57e8100ca0 0x7f57e81989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:11.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.243+0000 7f57ef6f3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57e8106f50 0x7f57e819cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:11.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.243+0000 7f57ef6f3640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f57e8111f60 con 0x7f57e8106f50 2024-09-12T02:04:11.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.243+0000 7f57ef6f3640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f57e8111de0 con 0x7f57e8100ca0 2024-09-12T02:04:11.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.243+0000 7f57ef6f3640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f57e81120e0 con 0x7f57e81002f0 2024-09-12T02:04:11.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.244+0000 7f57edc90640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57e8106f50 0x7f57e819cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:11.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.244+0000 7f57edc90640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57e8106f50 0x7f57e819cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46604/0 (socket says 172.21.15.16:46604) 2024-09-12T02:04:11.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.244+0000 7f57edc90640 1 -- 172.21.15.16:0/3299321639 learned_addr learned my addr 172.21.15.16:0/3299321639 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:04:11.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.244+0000 7f57ecc8e640 1 --2- 172.21.15.16:0/3299321639 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57e8100ca0 0x7f57e81989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:11.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.244+0000 7f57ed48f640 1 --2- 172.21.15.16:0/3299321639 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f57e81002f0 0x7f57e81984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:11.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.244+0000 7f57edc90640 1 -- 172.21.15.16:0/3299321639 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f57e81002f0 msgr2=0x7f57e81984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:11.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.244+0000 7f57edc90640 1 --2- 172.21.15.16:0/3299321639 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f57e81002f0 0x7f57e81984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:11.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.244+0000 7f57edc90640 1 -- 172.21.15.16:0/3299321639 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57e8100ca0 msgr2=0x7f57e81989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:11.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.244+0000 7f57edc90640 1 --2- 172.21.15.16:0/3299321639 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57e8100ca0 0x7f57e81989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:11.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.244+0000 7f57edc90640 1 -- 172.21.15.16:0/3299321639 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57e819d460 con 0x7f57e8106f50 2024-09-12T02:04:11.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.244+0000 7f57ed48f640 1 --2- 172.21.15.16:0/3299321639 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f57e81002f0 0x7f57e81984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:04:11.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.244+0000 7f57ecc8e640 1 --2- 172.21.15.16:0/3299321639 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57e8100ca0 0x7f57e81989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:04:11.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.244+0000 7f57edc90640 1 --2- 172.21.15.16:0/3299321639 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57e8106f50 0x7f57e819cd80 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f57e000eaa0 tx=0x7f57e000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:04:11.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.244+0000 7f57de7fc640 1 -- 172.21.15.16:0/3299321639 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57e000cdb0 con 0x7f57e8106f50 2024-09-12T02:04:11.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.244+0000 7f57de7fc640 1 -- 172.21.15.16:0/3299321639 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f57e0005020 con 0x7f57e8106f50 2024-09-12T02:04:11.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.244+0000 7f57ef6f3640 1 -- 172.21.15.16:0/3299321639 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57e819d750 con 0x7f57e8106f50 2024-09-12T02:04:11.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.244+0000 7f57de7fc640 1 -- 172.21.15.16:0/3299321639 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57e00116b0 con 0x7f57e8106f50 2024-09-12T02:04:11.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.244+0000 7f57ef6f3640 1 -- 172.21.15.16:0/3299321639 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57e81a5030 con 0x7f57e8106f50 2024-09-12T02:04:11.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.247+0000 7f57de7fc640 1 -- 172.21.15.16:0/3299321639 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f57e0005560 con 0x7f57e8106f50 2024-09-12T02:04:11.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.247+0000 7f57f1acc740 1 Processor -- start 2024-09-12T02:04:11.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.247+0000 7f57f1acc740 1 -- start start 2024-09-12T02:04:11.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.248+0000 7f57f1acc740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563c471b54a0 0x563c471b5880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:11.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.248+0000 7f57f1acc740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c471b5dc0 0x563c471c0140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:11.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.248+0000 7f57f1acc740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c471c0680 0x563c471c2a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:11.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.248+0000 7f57f1acc740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x563c4711b2e0 con 0x563c471b5dc0 2024-09-12T02:04:11.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.248+0000 7f57f1acc740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x563c470c2ce0 con 0x563c471c0680 2024-09-12T02:04:11.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.248+0000 7f57f1acc740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x563c471172a0 con 0x563c471b54a0 2024-09-12T02:04:11.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.248+0000 7f57de7fc640 1 --2- 172.21.15.16:0/3299321639 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f57d40a2b80 0x7f57d40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:11.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.248+0000 7f57ed48f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c471b5dc0 0x563c471c0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:11.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.248+0000 7f57ed48f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c471b5dc0 0x563c471c0140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46614/0 (socket says 172.21.15.16:46614) 2024-09-12T02:04:11.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.248+0000 7f57ed48f640 1 -- 172.21.15.16:0/1727831744 learned_addr learned my addr 172.21.15.16:0/1727831744 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:04:11.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.248+0000 7f57ecc8e640 1 --2- 172.21.15.16:0/1727831744 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563c471b54a0 0x563c471b5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:11.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.248+0000 7f57edc90640 1 --2- 172.21.15.16:0/3299321639 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f57d40a2b80 0x7f57d40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:11.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.248+0000 7f57ecc8e640 1 --2- 172.21.15.16:0/1727831744 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c471c0680 0x563c471c2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:11.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.248+0000 7f57de7fc640 1 -- 172.21.15.16:0/3299321639 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f57e00cf2e0 con 0x7f57e8106f50 2024-09-12T02:04:11.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.248+0000 7f57ed48f640 1 -- 172.21.15.16:0/1727831744 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563c471b54a0 msgr2=0x563c471b5880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:11.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.248+0000 7f57ed48f640 1 --2- 172.21.15.16:0/1727831744 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563c471b54a0 0x563c471b5880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:11.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.248+0000 7f57ed48f640 1 -- 172.21.15.16:0/1727831744 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c471c0680 msgr2=0x563c471c2a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:11.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.248+0000 7f57ed48f640 1 --2- 172.21.15.16:0/1727831744 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c471c0680 0x563c471c2a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:11.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.248+0000 7f57ed48f640 1 -- 172.21.15.16:0/1727831744 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c470a7440 con 0x563c471b5dc0 2024-09-12T02:04:11.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.248+0000 7f57ed48f640 1 --2- 172.21.15.16:0/1727831744 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c471b5dc0 0x563c471c0140 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f57e4005fd0 tx=0x7f57e400e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e892c9525776e866 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:04:11.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.249+0000 7f57edc90640 1 --2- 172.21.15.16:0/3299321639 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f57d40a2b80 0x7f57d40a5040 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f57e000efd0 tx=0x7f57e00054d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:04:11.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.249+0000 7f57d37fe640 1 -- 172.21.15.16:0/1727831744 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57e4016070 con 0x563c471b5dc0 2024-09-12T02:04:11.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.249+0000 7f57d37fe640 1 -- 172.21.15.16:0/1727831744 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f57e4005500 con 0x563c471b5dc0 2024-09-12T02:04:11.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.249+0000 7f57d37fe640 1 -- 172.21.15.16:0/1727831744 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57e4011470 con 0x563c471b5dc0 2024-09-12T02:04:11.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.249+0000 7f57f1acc740 1 -- 172.21.15.16:0/1727831744 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c471b5dc0 msgr2=0x563c471c0140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:11.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.249+0000 7f57f1acc740 1 --2- 172.21.15.16:0/1727831744 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c471b5dc0 0x563c471c0140 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f57e4005fd0 tx=0x7f57e400e6c0 comp rx=0 tx=0).stop 2024-09-12T02:04:11.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.249+0000 7f57f1acc740 1 -- 172.21.15.16:0/1727831744 shutdown_connections 2024-09-12T02:04:11.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.249+0000 7f57f1acc740 1 --2- 172.21.15.16:0/1727831744 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c471c0680 0x563c471c2a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:11.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.249+0000 7f57f1acc740 1 --2- 172.21.15.16:0/1727831744 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c471b5dc0 0x563c471c0140 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:11.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.249+0000 7f57f1acc740 1 --2- 172.21.15.16:0/1727831744 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563c471b54a0 0x563c471b5880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:11.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.249+0000 7f57f1acc740 1 -- 172.21.15.16:0/1727831744 >> 172.21.15.16:0/1727831744 conn(0x563c471b1a30 msgr2=0x563c471b1e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:04:11.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.250+0000 7f57f1acc740 1 -- 172.21.15.16:0/1727831744 shutdown_connections 2024-09-12T02:04:11.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.250+0000 7f57f1acc740 1 -- 172.21.15.16:0/1727831744 wait complete. 2024-09-12T02:04:11.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.250+0000 7f57f1acc740 1 -- start start 2024-09-12T02:04:11.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.250+0000 7f57f1acc740 1 Processor -- start 2024-09-12T02:04:11.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.250+0000 7f57f1acc740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c471b54a0 0x563c471bff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:11.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.250+0000 7f57f1acc740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c471b5dc0 0x563c471bd1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:11.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.250+0000 7f57f1acc740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563c471c0680 0x563c471bd730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:11.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.250+0000 7f57f1acc740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x563c471c3250 con 0x563c471b54a0 2024-09-12T02:04:11.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.250+0000 7f57f1acc740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x563c470d6050 con 0x563c471b5dc0 2024-09-12T02:04:11.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.250+0000 7f57f1acc740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x563c471c33d0 con 0x563c471c0680 2024-09-12T02:04:11.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.250+0000 7f57f1acc740 10 client.0 _ll_register_callbacks cb 0x563c47116160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:04:11.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.250+0000 7f57ecc8e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c471b54a0 0x563c471bff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:11.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.251+0000 7f57ecc8e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c471b54a0 0x563c471bff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46624/0 (socket says 172.21.15.16:46624) 2024-09-12T02:04:11.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.251+0000 7f57ecc8e640 1 -- 172.21.15.16:0/622440268 learned_addr learned my addr 172.21.15.16:0/622440268 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:04:11.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.251+0000 7f57ed48f640 1 --2- 172.21.15.16:0/622440268 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c471b5dc0 0x563c471bd1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:11.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.251+0000 7f57ecc8e640 1 --2- 172.21.15.16:0/622440268 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563c471c0680 0x563c471bd730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:11.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.251+0000 7f57ecc8e640 1 -- 172.21.15.16:0/622440268 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563c471c0680 msgr2=0x563c471bd730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:11.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.251+0000 7f57ecc8e640 1 --2- 172.21.15.16:0/622440268 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563c471c0680 0x563c471bd730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:11.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.251+0000 7f57ecc8e640 1 -- 172.21.15.16:0/622440268 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c471b5dc0 msgr2=0x563c471bd1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:11.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.251+0000 7f57ecc8e640 1 --2- 172.21.15.16:0/622440268 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c471b5dc0 0x563c471bd1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:11.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.251+0000 7f57ecc8e640 1 -- 172.21.15.16:0/622440268 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c46e8c3c0 con 0x563c471b54a0 2024-09-12T02:04:11.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.251+0000 7f57ecc8e640 1 --2- 172.21.15.16:0/622440268 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c471b54a0 0x563c471bff40 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f57d8031550 tx=0x7f57d802a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92e1d3af049c1fcb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:04:11.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.251+0000 7f57d0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:04:11.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.251+0000 7f57d0ff9640 1 -- 172.21.15.16:0/622440268 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57d8035040 con 0x563c471b54a0 2024-09-12T02:04:11.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.251+0000 7f57d0ff9640 1 -- 172.21.15.16:0/622440268 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f57d8004450 con 0x563c471b54a0 2024-09-12T02:04:11.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.251+0000 7f57f1acc740 1 -- 172.21.15.16:0/622440268 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f57e4005500 con 0x563c471b54a0 2024-09-12T02:04:11.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.251+0000 7f57d0ff9640 1 -- 172.21.15.16:0/622440268 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f57d80035f0 con 0x563c471b54a0 2024-09-12T02:04:11.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.251+0000 7f57f1acc740 10 client.16041 fetch_fsmap learned FSMap version 25 2024-09-12T02:04:11.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.251+0000 7f57f1acc740 1 -- 172.21.15.16:0/622440268 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x563c471ceba0 con 0x563c471b54a0 2024-09-12T02:04:11.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.252+0000 7f57d0ff9640 1 -- 172.21.15.16:0/622440268 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f57e4005500 con 0x563c471b54a0 2024-09-12T02:04:11.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.252+0000 7f57d0ff9640 1 -- 172.21.15.16:0/622440268 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f57d803a040 con 0x563c471b54a0 2024-09-12T02:04:11.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.252+0000 7f57f1acc740 10 client.16041 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:04:11.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.252+0000 7f57f1acc740 10 client.16041 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:04:11.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.252+0000 7f57f1acc740 20 client.16041 populate_metadata read hostname 'smithi016' 2024-09-12T02:04:11.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.252+0000 7f57f1acc740 1 --2- 172.21.15.16:0/622440268 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x563c471d0060 0x563c471d2450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:11.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.252+0000 7f57f1acc740 4 client.16041 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:04:11.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.252+0000 7f57f1acc740 1 -- 172.21.15.16:0/622440268 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563c47128850 con 0x563c471d0060 2024-09-12T02:04:11.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.252+0000 7f57ed48f640 1 --2- 172.21.15.16:0/622440268 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x563c471d0060 0x563c471d2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:11.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.253+0000 7f57d0ff9640 10 client.16041 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:04:11.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.253+0000 7f57ed48f640 1 --2- 172.21.15.16:0/622440268 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x563c471d0060 0x563c471d2450 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f1b5fcd1ff04be0 server_cookie=1c9172e8afd8d14f in_seq=0 out_seq=0 2024-09-12T02:04:11.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.253+0000 7f57d0ff9640 1 -- 172.21.15.16:0/622440268 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563c47128850 con 0x563c471d0060 2024-09-12T02:04:11.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.253+0000 7f57d0ff9640 10 client.16041 handle_command_reply: tid=0 2024-09-12T02:04:11.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.253+0000 7f57f1acc740 1 client.16041 shutdown 2024-09-12T02:04:11.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.253+0000 7f57f1acc740 1 -- 172.21.15.16:0/622440268 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c471b54a0 msgr2=0x563c471bff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:11.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.253+0000 7f57f1acc740 1 --2- 172.21.15.16:0/622440268 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c471b54a0 0x563c471bff40 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f57d8031550 tx=0x7f57d802a420 comp rx=0 tx=0).stop 2024-09-12T02:04:11.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.254+0000 7f57f1acc740 1 -- 172.21.15.16:0/622440268 shutdown_connections 2024-09-12T02:04:11.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.254+0000 7f57f1acc740 1 --2- 172.21.15.16:0/622440268 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x563c471d0060 0x563c471d2450 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:11.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.254+0000 7f57f1acc740 1 --2- 172.21.15.16:0/622440268 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563c471c0680 0x563c471bd730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:11.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.254+0000 7f57f1acc740 1 --2- 172.21.15.16:0/622440268 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c471b5dc0 0x563c471bd1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:11.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.254+0000 7f57f1acc740 1 --2- 172.21.15.16:0/622440268 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c471b54a0 0x563c471bff40 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:11.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.254+0000 7f57d0ff9640 0 client.16041 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:04:11.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.254+0000 7f57f1acc740 1 -- 172.21.15.16:0/622440268 >> 172.21.15.16:0/622440268 conn(0x563c471b1a30 msgr2=0x563c471b7710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:04:11.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.254+0000 7f57f1acc740 1 -- 172.21.15.16:0/622440268 shutdown_connections 2024-09-12T02:04:11.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.254+0000 7f57f1acc740 1 -- 172.21.15.16:0/622440268 wait complete. 2024-09-12T02:04:11.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.254+0000 7f57f1acc740 20 client.16041 trim_cache size 0 max 16384 2024-09-12T02:04:11.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.271+0000 7f57f1acc740 1 Processor -- start 2024-09-12T02:04:11.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.271+0000 7f57f1acc740 1 -- start start 2024-09-12T02:04:11.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.272+0000 7f57f1acc740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563c471b54a0 0x563c471cc4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:11.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.272+0000 7f57f1acc740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c471b5dc0 0x563c471d7cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:11.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.272+0000 7f57f1acc740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c471c0680 0x563c471d8230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:11.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.272+0000 7f57f1acc740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f57e81afdd0 con 0x563c471c0680 2024-09-12T02:04:11.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.272+0000 7f57f1acc740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f57e81ad6a0 con 0x563c471b5dc0 2024-09-12T02:04:11.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.272+0000 7f57f1acc740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f57e81ab540 con 0x563c471b54a0 2024-09-12T02:04:11.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.272+0000 7f57ecc8e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c471c0680 0x563c471d8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:11.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.272+0000 7f57ecc8e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c471c0680 0x563c471d8230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46626/0 (socket says 172.21.15.16:46626) 2024-09-12T02:04:11.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.272+0000 7f57ecc8e640 1 -- 172.21.15.16:0/1081651018 learned_addr learned my addr 172.21.15.16:0/1081651018 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:04:11.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.272+0000 7f57ecc8e640 1 --2- 172.21.15.16:0/1081651018 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563c471b54a0 0x563c471cc4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:11.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.272+0000 7f57ed48f640 1 --2- 172.21.15.16:0/1081651018 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c471b5dc0 0x563c471d7cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:11.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.272+0000 7f57ecc8e640 1 -- 172.21.15.16:0/1081651018 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563c471b54a0 msgr2=0x563c471cc4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:11.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.272+0000 7f57ecc8e640 1 --2- 172.21.15.16:0/1081651018 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563c471b54a0 0x563c471cc4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:11.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.272+0000 7f57ecc8e640 1 -- 172.21.15.16:0/1081651018 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c471b5dc0 msgr2=0x563c471d7cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:11.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.272+0000 7f57ecc8e640 1 --2- 172.21.15.16:0/1081651018 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c471b5dc0 0x563c471d7cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:11.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.272+0000 7f57ecc8e640 1 -- 172.21.15.16:0/1081651018 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c471c2de0 con 0x563c471c0680 2024-09-12T02:04:11.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.272+0000 7f57ecc8e640 1 --2- 172.21.15.16:0/1081651018 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c471c0680 0x563c471d8230 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f57d8031520 tx=0x7f57d8005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b42e76625c0565fc server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:04:11.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.273+0000 7f57d2ffd640 1 -- 172.21.15.16:0/1081651018 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57d8035040 con 0x563c471c0680 2024-09-12T02:04:11.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.273+0000 7f57d2ffd640 1 -- 172.21.15.16:0/1081651018 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f57d803dd50 con 0x563c471c0680 2024-09-12T02:04:11.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.273+0000 7f57d2ffd640 1 -- 172.21.15.16:0/1081651018 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57d803d3f0 con 0x563c471c0680 2024-09-12T02:04:11.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.273+0000 7f57f1acc740 1 -- 172.21.15.16:0/1081651018 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c471c0680 msgr2=0x563c471d8230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:11.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.273+0000 7f57f1acc740 1 --2- 172.21.15.16:0/1081651018 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c471c0680 0x563c471d8230 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f57d8031520 tx=0x7f57d8005fd0 comp rx=0 tx=0).stop 2024-09-12T02:04:11.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.273+0000 7f57f1acc740 1 -- 172.21.15.16:0/1081651018 shutdown_connections 2024-09-12T02:04:11.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.273+0000 7f57f1acc740 1 --2- 172.21.15.16:0/1081651018 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c471c0680 0x563c471d8230 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:11.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.273+0000 7f57f1acc740 1 --2- 172.21.15.16:0/1081651018 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c471b5dc0 0x563c471d7cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:11.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.273+0000 7f57f1acc740 1 --2- 172.21.15.16:0/1081651018 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563c471b54a0 0x563c471cc4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:11.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.273+0000 7f57f1acc740 1 -- 172.21.15.16:0/1081651018 >> 172.21.15.16:0/1081651018 conn(0x563c471b1a30 msgr2=0x563c471bbac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:04:11.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.273+0000 7f57f1acc740 1 -- 172.21.15.16:0/1081651018 shutdown_connections 2024-09-12T02:04:11.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.274+0000 7f57f1acc740 1 -- 172.21.15.16:0/1081651018 wait complete. 2024-09-12T02:04:11.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.274+0000 7f57f1acc740 1 -- start start 2024-09-12T02:04:11.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.274+0000 7f57f1acc740 1 Processor -- start 2024-09-12T02:04:11.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.274+0000 7f57f1acc740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c471b54a0 0x563c471c8ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:11.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.274+0000 7f57f1acc740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c471b5dc0 0x563c471c41f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:11.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.274+0000 7f57f1acc740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563c471c0680 0x563c471c4980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:11.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.274+0000 7f57f1acc740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f57e81abbb0 con 0x563c471b5dc0 2024-09-12T02:04:11.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.274+0000 7f57f1acc740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f57e81acfd0 con 0x563c471b54a0 2024-09-12T02:04:11.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.274+0000 7f57f1acc740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f57e81ac230 con 0x563c471c0680 2024-09-12T02:04:11.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.275+0000 7f57f1acc740 10 client.0 _ll_register_callbacks cb 0x7f57d8009580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:04:11.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.275+0000 7f57ed48f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c471b5dc0 0x563c471c41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:11.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.275+0000 7f57ed48f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c471b5dc0 0x563c471c41f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46642/0 (socket says 172.21.15.16:46642) 2024-09-12T02:04:11.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.275+0000 7f57ed48f640 1 -- 172.21.15.16:0/1340879888 learned_addr learned my addr 172.21.15.16:0/1340879888 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:04:11.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.275+0000 7f57ecc8e640 1 --2- 172.21.15.16:0/1340879888 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563c471c0680 0x563c471c4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:11.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.275+0000 7f57ecc8e640 1 --2- 172.21.15.16:0/1340879888 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c471b54a0 0x563c471c8ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:11.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.275+0000 7f57ed48f640 1 -- 172.21.15.16:0/1340879888 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563c471c0680 msgr2=0x563c471c4980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:11.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.275+0000 7f57ed48f640 1 --2- 172.21.15.16:0/1340879888 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563c471c0680 0x563c471c4980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:11.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.275+0000 7f57ed48f640 1 -- 172.21.15.16:0/1340879888 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c471b54a0 msgr2=0x563c471c8ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:11.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.275+0000 7f57ed48f640 1 --2- 172.21.15.16:0/1340879888 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c471b54a0 0x563c471c8ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:11.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.275+0000 7f57ed48f640 1 -- 172.21.15.16:0/1340879888 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c47239cc0 con 0x563c471b5dc0 2024-09-12T02:04:11.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.275+0000 7f57ecc8e640 1 --2- 172.21.15.16:0/1340879888 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563c471c0680 0x563c471c4980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:04:11.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.275+0000 7f57ed48f640 1 --2- 172.21.15.16:0/1340879888 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c471b5dc0 0x563c471c41f0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f57e4005fd0 tx=0x7f57e4004b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4e7fdd281f5e3b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:04:11.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.275+0000 7f57d0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:04:11.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.277+0000 7f57d0ff9640 1 -- 172.21.15.16:0/1340879888 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57e4016040 con 0x563c471b5dc0 2024-09-12T02:04:11.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.277+0000 7f57d0ff9640 1 -- 172.21.15.16:0/1340879888 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f57e4004880 con 0x563c471b5dc0 2024-09-12T02:04:11.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.277+0000 7f57f1acc740 1 -- 172.21.15.16:0/1340879888 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f57d803dd50 con 0x563c471b5dc0 2024-09-12T02:04:11.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.278+0000 7f57d0ff9640 1 -- 172.21.15.16:0/1340879888 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f57e4009ea0 con 0x563c471b5dc0 2024-09-12T02:04:11.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.278+0000 7f57f1acc740 10 client.16047 fetch_fsmap learned FSMap version 25 2024-09-12T02:04:11.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.278+0000 7f57f1acc740 1 -- 172.21.15.16:0/1340879888 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x563c4723aeb0 con 0x563c471b5dc0 2024-09-12T02:04:11.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.278+0000 7f57d0ff9640 1 -- 172.21.15.16:0/1340879888 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f57d803dd50 con 0x563c471b5dc0 2024-09-12T02:04:11.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.278+0000 7f57d0ff9640 1 -- 172.21.15.16:0/1340879888 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f57e400f040 con 0x563c471b5dc0 2024-09-12T02:04:11.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.278+0000 7f57f1acc740 10 client.16047 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:04:11.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.278+0000 7f57f1acc740 10 client.16047 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:04:11.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.278+0000 7f57f1acc740 20 client.16047 populate_metadata read hostname 'smithi016' 2024-09-12T02:04:11.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.278+0000 7f57f1acc740 1 --2- 172.21.15.16:0/1340879888 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x563c4723b4d0 0x563c4723d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:11.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.278+0000 7f57f1acc740 4 client.16047 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T02:04:11.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.278+0000 7f57f1acc740 1 -- 172.21.15.16:0/1340879888 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563c471bdac0 con 0x563c4723b4d0 2024-09-12T02:04:11.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.278+0000 7f57ecc8e640 1 --2- 172.21.15.16:0/1340879888 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x563c4723b4d0 0x563c4723d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:11.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.279+0000 7f57ecc8e640 1 --2- 172.21.15.16:0/1340879888 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x563c4723b4d0 0x563c4723d8f0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e5a21694528c0e9 server_cookie=3affdea17ce3bf00 in_seq=0 out_seq=0 2024-09-12T02:04:11.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.279+0000 7f57d0ff9640 10 client.16047 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:04:11.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.279+0000 7f57d0ff9640 1 -- 172.21.15.16:0/1340879888 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x563c471bdac0 con 0x563c4723b4d0 2024-09-12T02:04:11.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.279+0000 7f57d0ff9640 10 client.16047 handle_command_reply: tid=0 2024-09-12T02:04:11.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.279+0000 7f57f1acc740 1 client.16047 shutdown 2024-09-12T02:04:11.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.279+0000 7f57f1acc740 1 -- 172.21.15.16:0/1340879888 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c471b5dc0 msgr2=0x563c471c41f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:11.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.279+0000 7f57f1acc740 1 --2- 172.21.15.16:0/1340879888 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c471b5dc0 0x563c471c41f0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f57e4005fd0 tx=0x7f57e4004b10 comp rx=0 tx=0).stop 2024-09-12T02:04:11.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.279+0000 7f57f1acc740 1 -- 172.21.15.16:0/1340879888 shutdown_connections 2024-09-12T02:04:11.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.279+0000 7f57f1acc740 1 --2- 172.21.15.16:0/1340879888 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x563c4723b4d0 0x563c4723d8f0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:11.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.279+0000 7f57f1acc740 1 --2- 172.21.15.16:0/1340879888 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563c471c0680 0x563c471c4980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:11.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.279+0000 7f57f1acc740 1 --2- 172.21.15.16:0/1340879888 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c471b5dc0 0x563c471c41f0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:11.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.279+0000 7f57f1acc740 1 --2- 172.21.15.16:0/1340879888 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c471b54a0 0x563c471c8ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:11.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.279+0000 7f57d0ff9640 0 client.16047 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:04:11.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.279+0000 7f57f1acc740 1 -- 172.21.15.16:0/1340879888 >> 172.21.15.16:0/1340879888 conn(0x563c471b1a30 msgr2=0x563c471bb680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:04:11.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.279+0000 7f57f1acc740 1 -- 172.21.15.16:0/1340879888 shutdown_connections 2024-09-12T02:04:11.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.279+0000 7f57f1acc740 1 -- 172.21.15.16:0/1340879888 wait complete. 2024-09-12T02:04:11.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.279+0000 7f57f1acc740 20 client.16047 trim_cache size 0 max 16384 2024-09-12T02:04:11.291 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T02:04:11.291 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-09-12T02:04:11.292 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {} 2024-09-12T02:04:11.292 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T02:04:11.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.280+0000 7f57ef6f3640 1 -- 172.21.15.16:0/3299321639 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f57d40a2b80 msgr2=0x7f57d40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:11.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.280+0000 7f57ef6f3640 1 --2- 172.21.15.16:0/3299321639 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f57d40a2b80 0x7f57d40a5040 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f57e000efd0 tx=0x7f57e00054d0 comp rx=0 tx=0).stop 2024-09-12T02:04:11.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.280+0000 7f57ef6f3640 1 -- 172.21.15.16:0/3299321639 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57e8106f50 msgr2=0x7f57e819cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:11.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.280+0000 7f57ef6f3640 1 --2- 172.21.15.16:0/3299321639 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57e8106f50 0x7f57e819cd80 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f57e000eaa0 tx=0x7f57e000ef60 comp rx=0 tx=0).stop 2024-09-12T02:04:11.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.280+0000 7f57ef6f3640 1 -- 172.21.15.16:0/3299321639 shutdown_connections 2024-09-12T02:04:11.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.280+0000 7f57ef6f3640 1 --2- 172.21.15.16:0/3299321639 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f57d40a2b80 0x7f57d40a5040 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:11.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.280+0000 7f57ef6f3640 1 --2- 172.21.15.16:0/3299321639 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57e8106f50 0x7f57e819cd80 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:11.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.280+0000 7f57ef6f3640 1 --2- 172.21.15.16:0/3299321639 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57e8100ca0 0x7f57e81989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:11.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.280+0000 7f57ef6f3640 1 --2- 172.21.15.16:0/3299321639 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f57e81002f0 0x7f57e81984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:11.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.280+0000 7f57ef6f3640 1 -- 172.21.15.16:0/3299321639 >> 172.21.15.16:0/3299321639 conn(0x7f57e8077210 msgr2=0x7f57e80fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:04:11.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.280+0000 7f57ef6f3640 1 -- 172.21.15.16:0/3299321639 shutdown_connections 2024-09-12T02:04:11.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:11.280+0000 7f57ef6f3640 1 -- 172.21.15.16:0/3299321639 wait complete. 2024-09-12T02:04:12.025 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-12T02:04:12.025 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:04:12.325 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:04:12.582 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:04:12 smithi016 ceph-mon[30947]: pgmap v403: 97 pgs: 97 active+clean; 1.3 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-09-12T02:04:12.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:04:12 smithi184 ceph-mon[35593]: pgmap v403: 97 pgs: 97 active+clean; 1.3 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-09-12T02:04:12.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:04:12 smithi033 ceph-mon[35332]: pgmap v403: 97 pgs: 97 active+clean; 1.3 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-09-12T02:04:12.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:12.783+0000 7fbbb5fdf640 1 -- 172.21.15.16:0/1481608007 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbbb0102d40 msgr2=0x7fbbb0103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:12.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:12.783+0000 7fbbb5fdf640 1 --2- 172.21.15.16:0/1481608007 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbbb0102d40 0x7fbbb0103120 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fbba0009900 tx=0x7fbba002a480 comp rx=0 tx=0).stop 2024-09-12T02:04:12.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:12.783+0000 7fbbb5fdf640 1 -- 172.21.15.16:0/1481608007 shutdown_connections 2024-09-12T02:04:12.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:12.783+0000 7fbbb5fdf640 1 --2- 172.21.15.16:0/1481608007 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbbb010c170 0x7fbbb010e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:12.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:12.783+0000 7fbbb5fdf640 1 --2- 172.21.15.16:0/1481608007 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbbb0103660 0x7fbbb010bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:12.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:12.783+0000 7fbbb5fdf640 1 --2- 172.21.15.16:0/1481608007 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbbb0102d40 0x7fbbb0103120 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:12.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:12.783+0000 7fbbb5fdf640 1 -- 172.21.15.16:0/1481608007 >> 172.21.15.16:0/1481608007 conn(0x7fbbb00fc7a0 msgr2=0x7fbbb00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:04:12.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:12.783+0000 7fbbb5fdf640 1 -- 172.21.15.16:0/1481608007 shutdown_connections 2024-09-12T02:04:12.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:12.783+0000 7fbbb5fdf640 1 -- 172.21.15.16:0/1481608007 wait complete. 2024-09-12T02:04:12.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:12.784+0000 7fbbb5fdf640 1 Processor -- start 2024-09-12T02:04:12.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:12.784+0000 7fbbb5fdf640 1 -- start start 2024-09-12T02:04:12.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:12.784+0000 7fbbb5fdf640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbbb0102d40 0x7fbbb01009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:12.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:12.784+0000 7fbbb5fdf640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbbb0103660 0x7fbbb0100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:12.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:12.785+0000 7fbbb5fdf640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbbb010c170 0x7fbbb0101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:12.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:12.785+0000 7fbbb5fdf640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbbb0111f70 con 0x7fbbb0103660 2024-09-12T02:04:12.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:12.785+0000 7fbbb5fdf640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbbb0111df0 con 0x7fbbb0102d40 2024-09-12T02:04:12.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:12.785+0000 7fbbb5fdf640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fbbb01120f0 con 0x7fbbb010c170 2024-09-12T02:04:12.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:12.785+0000 7fbba6ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbbb0103660 0x7fbbb0100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:12.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:12.785+0000 7fbba6ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbbb0103660 0x7fbbb0100ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46662/0 (socket says 172.21.15.16:46662) 2024-09-12T02:04:12.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:12.785+0000 7fbbaf7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbbb0102d40 0x7fbbb01009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:12.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:12.785+0000 7fbba6ffd640 1 -- 172.21.15.16:0/4117803204 learned_addr learned my addr 172.21.15.16:0/4117803204 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:04:12.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:12.785+0000 7fbbaffff640 1 --2- 172.21.15.16:0/4117803204 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbbb010c170 0x7fbbb0101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:12.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:12.785+0000 7fbba6ffd640 1 -- 172.21.15.16:0/4117803204 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbbb010c170 msgr2=0x7fbbb0101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:12.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:12.785+0000 7fbba6ffd640 1 --2- 172.21.15.16:0/4117803204 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbbb010c170 0x7fbbb0101420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:12.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:12.785+0000 7fbba6ffd640 1 -- 172.21.15.16:0/4117803204 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbbb0102d40 msgr2=0x7fbbb01009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:12.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:12.785+0000 7fbba6ffd640 1 --2- 172.21.15.16:0/4117803204 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbbb0102d40 0x7fbbb01009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:12.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:12.785+0000 7fbba6ffd640 1 -- 172.21.15.16:0/4117803204 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbb01a69a0 con 0x7fbbb0103660 2024-09-12T02:04:12.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:12.785+0000 7fbba6ffd640 1 --2- 172.21.15.16:0/4117803204 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbbb0103660 0x7fbbb0100ee0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fbb9800c970 tx=0x7fbb9800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:04:12.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:12.785+0000 7fbbad7fa640 1 -- 172.21.15.16:0/4117803204 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb98007bb0 con 0x7fbbb0103660 2024-09-12T02:04:12.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:12.785+0000 7fbbad7fa640 1 -- 172.21.15.16:0/4117803204 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb98007d50 con 0x7fbbb0103660 2024-09-12T02:04:12.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:12.785+0000 7fbbb5fdf640 1 -- 172.21.15.16:0/4117803204 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbb01a6c90 con 0x7fbbb0103660 2024-09-12T02:04:12.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:12.785+0000 7fbbad7fa640 1 -- 172.21.15.16:0/4117803204 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb9800f450 con 0x7fbbb0103660 2024-09-12T02:04:12.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:12.785+0000 7fbbb5fdf640 1 -- 172.21.15.16:0/4117803204 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbb01a7180 con 0x7fbbb0103660 2024-09-12T02:04:12.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:12.786+0000 7fbbb5fdf640 1 -- 172.21.15.16:0/4117803204 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbbb0103120 con 0x7fbbb0103660 2024-09-12T02:04:12.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:12.788+0000 7fbbad7fa640 1 -- 172.21.15.16:0/4117803204 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fbb98004f60 con 0x7fbbb0103660 2024-09-12T02:04:12.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:12.788+0000 7fbbad7fa640 1 --2- 172.21.15.16:0/4117803204 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fbb9c0a2b80 0x7fbb9c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:12.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:12.789+0000 7fbbaf7fe640 1 --2- 172.21.15.16:0/4117803204 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fbb9c0a2b80 0x7fbb9c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:12.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:12.789+0000 7fbbad7fa640 1 -- 172.21.15.16:0/4117803204 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fbb980cecb0 con 0x7fbbb0103660 2024-09-12T02:04:12.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:12.789+0000 7fbbaf7fe640 1 --2- 172.21.15.16:0/4117803204 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fbb9c0a2b80 0x7fbb9c0a5040 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fbba0009230 tx=0x7fbba00058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:04:12.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:12.793+0000 7fbbad7fa640 1 -- 172.21.15.16:0/4117803204 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbb98014030 con 0x7fbbb0103660 2024-09-12T02:04:13.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:13.007+0000 7fbbb5fdf640 1 -- 172.21.15.16:0/4117803204 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbbb01071e0 con 0x7fbbb0103660 2024-09-12T02:04:13.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:13.008+0000 7fbbad7fa640 1 -- 172.21.15.16:0/4117803204 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11911 (secure 0 0 0) 0x7fbb9809bcb0 con 0x7fbbb0103660 2024-09-12T02:04:13.008 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T02:04:13.008 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":25,"btime":"2024-09-12T02:01:37:192021+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3279123279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3279123279},{"type":"v1","addr":"172.21.15.16:6835","nonce":3279123279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4119452382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4119452382},{"type":"v1","addr":"172.21.15.16:6837","nonce":4119452382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/3263823165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3263823165},{"type":"v1","addr":"172.21.15.16:6839","nonce":3263823165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/1510290732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":1510290732},{"type":"v1","addr":"172.21.15.16:6841","nonce":1510290732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/279674603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":279674603},{"type":"v1","addr":"172.21.15.33:6833","nonce":279674603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3555728028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3555728028},{"type":"v1","addr":"172.21.15.33:6835","nonce":3555728028}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T01:59:56.117608+0000","modified":"2024-09-12T02:01:36.294760+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24482,"mds_1":24481,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6833/1563374106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1563374106},{"type":"v1","addr":"172.21.15.184:6833","nonce":1563374106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6837/1892833868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1892833868},{"type":"v1","addr":"172.21.15.33:6837","nonce":1892833868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/1457679686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1457679686},{"type":"v1","addr":"172.21.15.184:6835","nonce":1457679686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.184:6837/1793393087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":1793393087},{"type":"v1","addr":"172.21.15.184:6837","nonce":1793393087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.184:6839/4203347941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":4203347941},{"type":"v1","addr":"172.21.15.184:6839","nonce":4203347941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.33:6839/4136670204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":4136670204},{"type":"v1","addr":"172.21.15.33:6839","nonce":4136670204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24481,24482,24469]},"id":1}]} 2024-09-12T02:04:13.008 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 25 2024-09-12T02:04:13.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:13.009+0000 7fbbb5fdf640 1 -- 172.21.15.16:0/4117803204 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fbb9c0a2b80 msgr2=0x7fbb9c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:13.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:13.010+0000 7fbbb5fdf640 1 --2- 172.21.15.16:0/4117803204 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fbb9c0a2b80 0x7fbb9c0a5040 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fbba0009230 tx=0x7fbba00058d0 comp rx=0 tx=0).stop 2024-09-12T02:04:13.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:13.010+0000 7fbbb5fdf640 1 -- 172.21.15.16:0/4117803204 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbbb0103660 msgr2=0x7fbbb0100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:13.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:13.010+0000 7fbbb5fdf640 1 --2- 172.21.15.16:0/4117803204 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbbb0103660 0x7fbbb0100ee0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fbb9800c970 tx=0x7fbb9800ce30 comp rx=0 tx=0).stop 2024-09-12T02:04:13.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:13.010+0000 7fbbb5fdf640 1 -- 172.21.15.16:0/4117803204 shutdown_connections 2024-09-12T02:04:13.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:13.010+0000 7fbbb5fdf640 1 --2- 172.21.15.16:0/4117803204 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fbb9c0a2b80 0x7fbb9c0a5040 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:13.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:13.010+0000 7fbbb5fdf640 1 --2- 172.21.15.16:0/4117803204 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbbb010c170 0x7fbbb0101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:13.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:13.010+0000 7fbbb5fdf640 1 --2- 172.21.15.16:0/4117803204 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbbb0103660 0x7fbbb0100ee0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:13.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:13.010+0000 7fbbb5fdf640 1 --2- 172.21.15.16:0/4117803204 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbbb0102d40 0x7fbbb01009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:13.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:13.010+0000 7fbbb5fdf640 1 -- 172.21.15.16:0/4117803204 >> 172.21.15.16:0/4117803204 conn(0x7fbbb00fc7a0 msgr2=0x7fbbb00fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:04:13.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:13.010+0000 7fbbb5fdf640 1 -- 172.21.15.16:0/4117803204 shutdown_connections 2024-09-12T02:04:13.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:13.010+0000 7fbbb5fdf640 1 -- 172.21.15.16:0/4117803204 wait complete. 2024-09-12T02:04:13.824 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-12T02:04:14.089 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:04:14.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:04:13 smithi184 ceph-mon[35593]: pgmap v404: 97 pgs: 97 active+clean; 1.3 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-09-12T02:04:14.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:04:13 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/4117803204' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:04:14.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:04:13 smithi033 ceph-mon[35332]: pgmap v404: 97 pgs: 97 active+clean; 1.3 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-09-12T02:04:14.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:04:13 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/4117803204' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:04:14.183 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:04:13 smithi016 ceph-mon[30947]: pgmap v404: 97 pgs: 97 active+clean; 1.3 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-09-12T02:04:14.183 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:04:13 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/4117803204' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:04:14.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.550+0000 7febeda10640 1 -- 172.21.15.16:0/1517549799 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febe8101420 msgr2=0x7febe810d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:14.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.550+0000 7febeda10640 1 --2- 172.21.15.16:0/1517549799 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febe8101420 0x7febe810d950 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7febe0009cd0 tx=0x7febe002a8e0 comp rx=0 tx=0).stop 2024-09-12T02:04:14.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.550+0000 7febeda10640 1 -- 172.21.15.16:0/1517549799 shutdown_connections 2024-09-12T02:04:14.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.550+0000 7febeda10640 1 --2- 172.21.15.16:0/1517549799 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febe8101420 0x7febe810d950 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:14.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.550+0000 7febeda10640 1 --2- 172.21.15.16:0/1517549799 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7febe8100a80 0x7febe8100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:14.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.550+0000 7febeda10640 1 --2- 172.21.15.16:0/1517549799 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7febe8106a80 0x7febe8106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:14.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.550+0000 7febeda10640 1 -- 172.21.15.16:0/1517549799 >> 172.21.15.16:0/1517549799 conn(0x7febe80fc7a0 msgr2=0x7febe80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:04:14.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.550+0000 7febeda10640 1 -- 172.21.15.16:0/1517549799 shutdown_connections 2024-09-12T02:04:14.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.550+0000 7febeda10640 1 -- 172.21.15.16:0/1517549799 wait complete. 2024-09-12T02:04:14.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.551+0000 7febeda10640 1 Processor -- start 2024-09-12T02:04:14.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.551+0000 7febeda10640 1 -- start start 2024-09-12T02:04:14.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.551+0000 7febeda10640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7febe8100a80 0x7febe8102650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:14.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.551+0000 7febeda10640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7febe8101420 0x7febe8102b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:14.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.551+0000 7febeda10640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febe8106a80 0x7febe81030d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:14.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.551+0000 7febeda10640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7febe8110e80 con 0x7febe8106a80 2024-09-12T02:04:14.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.551+0000 7febeda10640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7febe8110d00 con 0x7febe8100a80 2024-09-12T02:04:14.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.551+0000 7febeda10640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7febe8111000 con 0x7febe8101420 2024-09-12T02:04:14.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.551+0000 7febe77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febe8106a80 0x7febe81030d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:14.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.551+0000 7febe77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febe8106a80 0x7febe81030d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46666/0 (socket says 172.21.15.16:46666) 2024-09-12T02:04:14.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.551+0000 7febe77fe640 1 -- 172.21.15.16:0/3288153213 learned_addr learned my addr 172.21.15.16:0/3288153213 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:04:14.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.551+0000 7febdffff640 1 --2- 172.21.15.16:0/3288153213 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7febe8101420 0x7febe8102b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:14.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.551+0000 7febe6ffd640 1 --2- 172.21.15.16:0/3288153213 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7febe8100a80 0x7febe8102650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:14.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.552+0000 7febe77fe640 1 -- 172.21.15.16:0/3288153213 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7febe8101420 msgr2=0x7febe8102b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:14.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.552+0000 7febe77fe640 1 --2- 172.21.15.16:0/3288153213 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7febe8101420 0x7febe8102b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:14.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.552+0000 7febe77fe640 1 -- 172.21.15.16:0/3288153213 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7febe8100a80 msgr2=0x7febe8102650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:14.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.552+0000 7febe77fe640 1 --2- 172.21.15.16:0/3288153213 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7febe8100a80 0x7febe8102650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:14.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.552+0000 7febe77fe640 1 -- 172.21.15.16:0/3288153213 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febe81a68d0 con 0x7febe8106a80 2024-09-12T02:04:14.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.552+0000 7febe6ffd640 1 --2- 172.21.15.16:0/3288153213 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7febe8100a80 0x7febe8102650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:04:14.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.552+0000 7febe77fe640 1 --2- 172.21.15.16:0/3288153213 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febe8106a80 0x7febe81030d0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7febe0006010 tx=0x7febe0003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:04:14.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.552+0000 7febeca0e640 1 -- 172.21.15.16:0/3288153213 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febe0004450 con 0x7febe8106a80 2024-09-12T02:04:14.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.552+0000 7febeda10640 1 -- 172.21.15.16:0/3288153213 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febe81a6b60 con 0x7febe8106a80 2024-09-12T02:04:14.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.552+0000 7febeca0e640 1 -- 172.21.15.16:0/3288153213 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7febe0003e10 con 0x7febe8106a80 2024-09-12T02:04:14.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.552+0000 7febeca0e640 1 -- 172.21.15.16:0/3288153213 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febe00466e0 con 0x7febe8106a80 2024-09-12T02:04:14.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.552+0000 7febeda10640 1 -- 172.21.15.16:0/3288153213 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febe81a70a0 con 0x7febe8106a80 2024-09-12T02:04:14.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.554+0000 7febeca0e640 1 -- 172.21.15.16:0/3288153213 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7febe00305a0 con 0x7febe8106a80 2024-09-12T02:04:14.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.555+0000 7febefde9740 1 Processor -- start 2024-09-12T02:04:14.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.555+0000 7febefde9740 1 -- start start 2024-09-12T02:04:14.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.555+0000 7febefde9740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563302542b40 0x563302541090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:14.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.555+0000 7febefde9740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5633025415d0 0x563302541a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:14.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.555+0000 7febefde9740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56330254c370 0x56330254e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:14.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.555+0000 7febefde9740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5633024a72f0 con 0x56330254c370 2024-09-12T02:04:14.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.555+0000 7febefde9740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56330244ecf0 con 0x5633025415d0 2024-09-12T02:04:14.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.555+0000 7febefde9740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5633024a32b0 con 0x563302542b40 2024-09-12T02:04:14.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.555+0000 7febdffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56330254c370 0x56330254e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:14.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.555+0000 7febdffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56330254c370 0x56330254e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46678/0 (socket says 172.21.15.16:46678) 2024-09-12T02:04:14.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.555+0000 7febdffff640 1 -- 172.21.15.16:0/572869355 learned_addr learned my addr 172.21.15.16:0/572869355 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:04:14.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.555+0000 7febdffff640 1 --2- 172.21.15.16:0/572869355 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563302542b40 0x563302541090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:14.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.555+0000 7febe6ffd640 1 --2- 172.21.15.16:0/572869355 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5633025415d0 0x563302541a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:14.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.555+0000 7febeca0e640 1 --2- 172.21.15.16:0/3288153213 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7febd40a2b80 0x7febd40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:14.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.556+0000 7febdffff640 1 -- 172.21.15.16:0/572869355 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563302542b40 msgr2=0x563302541090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:14.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.556+0000 7febdffff640 1 --2- 172.21.15.16:0/572869355 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563302542b40 0x563302541090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:14.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.556+0000 7febdffff640 1 -- 172.21.15.16:0/572869355 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5633025415d0 msgr2=0x563302541a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:14.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.556+0000 7febdffff640 1 --2- 172.21.15.16:0/572869355 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5633025415d0 0x563302541a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:14.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.556+0000 7febdffff640 1 -- 172.21.15.16:0/572869355 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5633024333a0 con 0x56330254c370 2024-09-12T02:04:14.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.556+0000 7febe77fe640 1 --2- 172.21.15.16:0/3288153213 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7febd40a2b80 0x7febd40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:14.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.556+0000 7febdffff640 1 --2- 172.21.15.16:0/572869355 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56330254c370 0x56330254e760 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7febd0011730 tx=0x7febd000a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d283009be34c984 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:04:14.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.556+0000 7febeca0e640 1 -- 172.21.15.16:0/3288153213 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7febe002e030 con 0x7febe8106a80 2024-09-12T02:04:14.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.556+0000 7febddffb640 1 -- 172.21.15.16:0/572869355 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febd0004020 con 0x56330254c370 2024-09-12T02:04:14.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.556+0000 7febddffb640 1 -- 172.21.15.16:0/572869355 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7febd0005500 con 0x56330254c370 2024-09-12T02:04:14.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.556+0000 7febddffb640 1 -- 172.21.15.16:0/572869355 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febd0004bd0 con 0x56330254c370 2024-09-12T02:04:14.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.557+0000 7febefde9740 1 -- 172.21.15.16:0/572869355 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56330254c370 msgr2=0x56330254e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:14.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.557+0000 7febefde9740 1 --2- 172.21.15.16:0/572869355 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56330254c370 0x56330254e760 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7febd0011730 tx=0x7febd000a6a0 comp rx=0 tx=0).stop 2024-09-12T02:04:14.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.557+0000 7febe77fe640 1 --2- 172.21.15.16:0/3288153213 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7febd40a2b80 0x7febd40a5040 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7febe00bccd0 tx=0x7febe0088040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:04:14.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.557+0000 7febefde9740 1 -- 172.21.15.16:0/572869355 shutdown_connections 2024-09-12T02:04:14.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.557+0000 7febefde9740 1 --2- 172.21.15.16:0/572869355 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56330254c370 0x56330254e760 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:14.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.557+0000 7febefde9740 1 --2- 172.21.15.16:0/572869355 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5633025415d0 0x563302541a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:14.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.557+0000 7febefde9740 1 --2- 172.21.15.16:0/572869355 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563302542b40 0x563302541090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:14.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.557+0000 7febefde9740 1 -- 172.21.15.16:0/572869355 >> 172.21.15.16:0/572869355 conn(0x56330250a310 msgr2=0x56330253dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:04:14.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.557+0000 7febefde9740 1 -- 172.21.15.16:0/572869355 shutdown_connections 2024-09-12T02:04:14.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.557+0000 7febefde9740 1 -- 172.21.15.16:0/572869355 wait complete. 2024-09-12T02:04:14.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.557+0000 7febefde9740 1 -- start start 2024-09-12T02:04:14.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.557+0000 7febefde9740 1 Processor -- start 2024-09-12T02:04:14.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.558+0000 7febefde9740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5633025415d0 0x56330254bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:14.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.558+0000 7febefde9740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563302542b40 0x563302548fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:14.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.558+0000 7febefde9740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56330254c370 0x5633025495d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:14.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.558+0000 7febefde9740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56330254f070 con 0x56330254c370 2024-09-12T02:04:14.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.558+0000 7febefde9740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x563302462060 con 0x5633025415d0 2024-09-12T02:04:14.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.558+0000 7febefde9740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x56330254f1f0 con 0x563302542b40 2024-09-12T02:04:14.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.558+0000 7febefde9740 10 client.0 _ll_register_callbacks cb 0x5633021b58c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:04:14.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.558+0000 7febdffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56330254c370 0x5633025495d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:14.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.558+0000 7febdffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56330254c370 0x5633025495d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46682/0 (socket says 172.21.15.16:46682) 2024-09-12T02:04:14.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.558+0000 7febdffff640 1 -- 172.21.15.16:0/1632061747 learned_addr learned my addr 172.21.15.16:0/1632061747 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:04:14.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.558+0000 7febdffff640 1 --2- 172.21.15.16:0/1632061747 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5633025415d0 0x56330254bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:14.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.558+0000 7febe6ffd640 1 --2- 172.21.15.16:0/1632061747 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563302542b40 0x563302548fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:14.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.558+0000 7febdffff640 1 -- 172.21.15.16:0/1632061747 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563302542b40 msgr2=0x563302548fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:14.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.558+0000 7febdffff640 1 --2- 172.21.15.16:0/1632061747 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563302542b40 0x563302548fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:14.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.558+0000 7febdffff640 1 -- 172.21.15.16:0/1632061747 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5633025415d0 msgr2=0x56330254bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:14.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.558+0000 7febdffff640 1 --2- 172.21.15.16:0/1632061747 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5633025415d0 0x56330254bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:14.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.558+0000 7febdffff640 1 -- 172.21.15.16:0/1632061747 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5633022183c0 con 0x56330254c370 2024-09-12T02:04:14.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.559+0000 7febdffff640 1 --2- 172.21.15.16:0/1632061747 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56330254c370 0x5633025495d0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7febd000a3b0 tx=0x7febd0016040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8639278b8bc4260c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:04:14.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.559+0000 7febc6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:04:14.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.559+0000 7febc6ffd640 1 -- 172.21.15.16:0/1632061747 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febd00046c0 con 0x56330254c370 2024-09-12T02:04:14.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.559+0000 7febc6ffd640 1 -- 172.21.15.16:0/1632061747 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7febd0004bc0 con 0x56330254c370 2024-09-12T02:04:14.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.559+0000 7febefde9740 1 -- 172.21.15.16:0/1632061747 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7febd0005500 con 0x56330254c370 2024-09-12T02:04:14.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.559+0000 7febc6ffd640 1 -- 172.21.15.16:0/1632061747 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7febd0004dc0 con 0x56330254c370 2024-09-12T02:04:14.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.559+0000 7febefde9740 10 client.16065 fetch_fsmap learned FSMap version 25 2024-09-12T02:04:14.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.559+0000 7febefde9740 1 -- 172.21.15.16:0/1632061747 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56330255ab00 con 0x56330254c370 2024-09-12T02:04:14.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.559+0000 7febc6ffd640 1 -- 172.21.15.16:0/1632061747 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7febd0005500 con 0x56330254c370 2024-09-12T02:04:14.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.559+0000 7febc6ffd640 1 -- 172.21.15.16:0/1632061747 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7febd002e790 con 0x56330254c370 2024-09-12T02:04:14.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.559+0000 7febefde9740 10 client.16065 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:04:14.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.559+0000 7febefde9740 10 client.16065 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-12T02:04:14.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.559+0000 7febefde9740 20 client.16065 populate_metadata read hostname 'smithi016' 2024-09-12T02:04:14.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.559+0000 7febefde9740 1 --2- 172.21.15.16:0/1632061747 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x56330255be50 0x56330255e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:14.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.559+0000 7febefde9740 4 client.16065 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:04:14.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.559+0000 7febefde9740 1 -- 172.21.15.16:0/1632061747 --> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5633024b4860 con 0x56330255be50 2024-09-12T02:04:14.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.560+0000 7febe6ffd640 1 --2- 172.21.15.16:0/1632061747 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x56330255be50 0x56330255e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:14.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.560+0000 7febe6ffd640 1 --2- 172.21.15.16:0/1632061747 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x56330255be50 0x56330255e240 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=11913fd1e7a53c60 server_cookie=d2ab3ecb06a57043 in_seq=0 out_seq=0 2024-09-12T02:04:14.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.560+0000 7febc6ffd640 10 client.16065 ms_handle_connect on v2:172.21.15.184:6836/1793393087 2024-09-12T02:04:14.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.561+0000 7febc6ffd640 1 -- 172.21.15.16:0/1632061747 <== mds.2 v2:172.21.15.184:6836/1793393087 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5633024b4860 con 0x56330255be50 2024-09-12T02:04:14.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.561+0000 7febc6ffd640 10 client.16065 handle_command_reply: tid=0 2024-09-12T02:04:14.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.561+0000 7febefde9740 1 client.16065 shutdown 2024-09-12T02:04:14.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.561+0000 7febefde9740 1 -- 172.21.15.16:0/1632061747 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56330254c370 msgr2=0x5633025495d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:14.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.561+0000 7febefde9740 1 --2- 172.21.15.16:0/1632061747 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56330254c370 0x5633025495d0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7febd000a3b0 tx=0x7febd0016040 comp rx=0 tx=0).stop 2024-09-12T02:04:14.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.561+0000 7febefde9740 1 -- 172.21.15.16:0/1632061747 shutdown_connections 2024-09-12T02:04:14.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.561+0000 7febefde9740 1 --2- 172.21.15.16:0/1632061747 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x56330255be50 0x56330255e240 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:14.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.561+0000 7febefde9740 1 --2- 172.21.15.16:0/1632061747 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56330254c370 0x5633025495d0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:14.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.561+0000 7febefde9740 1 --2- 172.21.15.16:0/1632061747 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563302542b40 0x563302548fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:14.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.561+0000 7febefde9740 1 --2- 172.21.15.16:0/1632061747 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5633025415d0 0x56330254bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:14.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.561+0000 7febefde9740 1 -- 172.21.15.16:0/1632061747 >> 172.21.15.16:0/1632061747 conn(0x56330250a310 msgr2=0x563302543960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:04:14.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.561+0000 7febc6ffd640 0 client.16065 ms_handle_reset on v2:172.21.15.184:6836/1793393087 2024-09-12T02:04:14.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.562+0000 7febefde9740 1 -- 172.21.15.16:0/1632061747 shutdown_connections 2024-09-12T02:04:14.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.562+0000 7febefde9740 1 -- 172.21.15.16:0/1632061747 wait complete. 2024-09-12T02:04:14.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.562+0000 7febefde9740 20 client.16065 trim_cache size 0 max 16384 2024-09-12T02:04:14.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.574+0000 7febefde9740 1 Processor -- start 2024-09-12T02:04:14.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.575+0000 7febefde9740 1 -- start start 2024-09-12T02:04:14.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.575+0000 7febefde9740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5633025415d0 0x563302558340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:14.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.575+0000 7febefde9740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563302542b40 0x56330253cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:14.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.575+0000 7febefde9740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56330254c370 0x563302563a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:14.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.575+0000 7febefde9740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7febe81b1e30 con 0x563302542b40 2024-09-12T02:04:14.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.575+0000 7febefde9740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7febe81af700 con 0x56330254c370 2024-09-12T02:04:14.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.575+0000 7febefde9740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7febe81ad550 con 0x5633025415d0 2024-09-12T02:04:14.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.575+0000 7febe6ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563302542b40 0x56330253cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:14.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.575+0000 7febe6ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563302542b40 0x56330253cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46686/0 (socket says 172.21.15.16:46686) 2024-09-12T02:04:14.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.575+0000 7febe6ffd640 1 -- 172.21.15.16:0/1873604888 learned_addr learned my addr 172.21.15.16:0/1873604888 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:04:14.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.575+0000 7febdffff640 1 --2- 172.21.15.16:0/1873604888 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5633025415d0 0x563302558340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:14.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.575+0000 7febdffff640 1 --2- 172.21.15.16:0/1873604888 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56330254c370 0x563302563a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:14.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.575+0000 7febe6ffd640 1 -- 172.21.15.16:0/1873604888 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5633025415d0 msgr2=0x563302558340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:14.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.575+0000 7febe6ffd640 1 --2- 172.21.15.16:0/1873604888 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5633025415d0 0x563302558340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:14.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.575+0000 7febe6ffd640 1 -- 172.21.15.16:0/1873604888 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56330254c370 msgr2=0x563302563a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:14.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.575+0000 7febe6ffd640 1 --2- 172.21.15.16:0/1873604888 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56330254c370 0x563302563a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:14.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.575+0000 7febe6ffd640 1 -- 172.21.15.16:0/1873604888 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56330254b950 con 0x563302542b40 2024-09-12T02:04:14.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.575+0000 7febdffff640 1 --2- 172.21.15.16:0/1873604888 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56330254c370 0x563302563a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:04:14.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.575+0000 7febe6ffd640 1 --2- 172.21.15.16:0/1873604888 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563302542b40 0x56330253cbf0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x563302548870 tx=0x7febd8012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63953f74a89ea391 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:04:14.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.576+0000 7febdd7fa640 1 -- 172.21.15.16:0/1873604888 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febd8004020 con 0x563302542b40 2024-09-12T02:04:14.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.576+0000 7febdd7fa640 1 -- 172.21.15.16:0/1873604888 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7febd80041c0 con 0x563302542b40 2024-09-12T02:04:14.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.576+0000 7febdd7fa640 1 -- 172.21.15.16:0/1873604888 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febd801b680 con 0x563302542b40 2024-09-12T02:04:14.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.576+0000 7febefde9740 1 -- 172.21.15.16:0/1873604888 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563302542b40 msgr2=0x56330253cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:14.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.576+0000 7febefde9740 1 --2- 172.21.15.16:0/1873604888 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563302542b40 0x56330253cbf0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x563302548870 tx=0x7febd8012820 comp rx=0 tx=0).stop 2024-09-12T02:04:14.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.576+0000 7febefde9740 1 -- 172.21.15.16:0/1873604888 shutdown_connections 2024-09-12T02:04:14.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.576+0000 7febefde9740 1 --2- 172.21.15.16:0/1873604888 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56330254c370 0x563302563a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:14.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.576+0000 7febefde9740 1 --2- 172.21.15.16:0/1873604888 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563302542b40 0x56330253cbf0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:14.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.576+0000 7febefde9740 1 --2- 172.21.15.16:0/1873604888 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5633025415d0 0x563302558340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:14.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.576+0000 7febefde9740 1 -- 172.21.15.16:0/1873604888 >> 172.21.15.16:0/1873604888 conn(0x56330250a310 msgr2=0x56330253c4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:04:14.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.576+0000 7febefde9740 1 -- 172.21.15.16:0/1873604888 shutdown_connections 2024-09-12T02:04:14.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.576+0000 7febefde9740 1 -- 172.21.15.16:0/1873604888 wait complete. 2024-09-12T02:04:14.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.577+0000 7febefde9740 1 -- start start 2024-09-12T02:04:14.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.577+0000 7febefde9740 1 Processor -- start 2024-09-12T02:04:14.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.577+0000 7febefde9740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5633025415d0 0x563302554bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:14.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.577+0000 7febefde9740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563302542b40 0x563302545cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:14.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.577+0000 7febefde9740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56330254c370 0x563302546210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:14.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.577+0000 7febefde9740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7febe81adbc0 con 0x56330254c370 2024-09-12T02:04:14.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.577+0000 7febefde9740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7febe81af030 con 0x5633025415d0 2024-09-12T02:04:14.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.577+0000 7febefde9740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7febe81ae290 con 0x563302542b40 2024-09-12T02:04:14.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.577+0000 7febefde9740 10 client.0 _ll_register_callbacks cb 0x7febd800c350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:04:14.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.577+0000 7febdffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56330254c370 0x563302546210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:14.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.578+0000 7febdffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56330254c370 0x563302546210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46702/0 (socket says 172.21.15.16:46702) 2024-09-12T02:04:14.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.578+0000 7febdffff640 1 -- 172.21.15.16:0/3761898724 learned_addr learned my addr 172.21.15.16:0/3761898724 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:04:14.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.578+0000 7febdffff640 1 --2- 172.21.15.16:0/3761898724 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5633025415d0 0x563302554bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:14.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.578+0000 7febe6ffd640 1 --2- 172.21.15.16:0/3761898724 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563302542b40 0x563302545cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:14.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.578+0000 7febdffff640 1 -- 172.21.15.16:0/3761898724 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563302542b40 msgr2=0x563302545cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:14.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.578+0000 7febdffff640 1 --2- 172.21.15.16:0/3761898724 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563302542b40 0x563302545cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:14.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.578+0000 7febdffff640 1 -- 172.21.15.16:0/3761898724 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5633025415d0 msgr2=0x563302554bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:14.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.578+0000 7febdffff640 1 --2- 172.21.15.16:0/3761898724 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5633025415d0 0x563302554bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:14.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.578+0000 7febdffff640 1 -- 172.21.15.16:0/3761898724 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563302546a10 con 0x56330254c370 2024-09-12T02:04:14.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.578+0000 7febdffff640 1 --2- 172.21.15.16:0/3761898724 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56330254c370 0x563302546210 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7febd00057c0 tx=0x7febd0013040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d444313fc0ed66b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:04:14.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.578+0000 7febc6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:04:14.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.578+0000 7febc6ffd640 1 -- 172.21.15.16:0/3761898724 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febd000a590 con 0x56330254c370 2024-09-12T02:04:14.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.578+0000 7febc6ffd640 1 -- 172.21.15.16:0/3761898724 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7febd0004bc0 con 0x56330254c370 2024-09-12T02:04:14.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.578+0000 7febefde9740 1 -- 172.21.15.16:0/3761898724 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7febd80041c0 con 0x56330254c370 2024-09-12T02:04:14.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.578+0000 7febc6ffd640 1 -- 172.21.15.16:0/3761898724 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7febd0011260 con 0x56330254c370 2024-09-12T02:04:14.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.578+0000 7febefde9740 10 client.16071 fetch_fsmap learned FSMap version 25 2024-09-12T02:04:14.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.578+0000 7febefde9740 1 -- 172.21.15.16:0/3761898724 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5633025c69a0 con 0x56330254c370 2024-09-12T02:04:14.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.579+0000 7febc6ffd640 1 -- 172.21.15.16:0/3761898724 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7febd80041c0 con 0x56330254c370 2024-09-12T02:04:14.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.579+0000 7febc6ffd640 1 -- 172.21.15.16:0/3761898724 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7febd0045050 con 0x56330254c370 2024-09-12T02:04:14.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.579+0000 7febefde9740 10 client.16071 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:04:14.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.579+0000 7febefde9740 10 client.16071 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-12T02:04:14.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.579+0000 7febefde9740 20 client.16071 populate_metadata read hostname 'smithi016' 2024-09-12T02:04:14.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.579+0000 7febefde9740 1 --2- 172.21.15.16:0/3761898724 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x5633025c7070 0x5633025c9490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:14.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.579+0000 7febefde9740 4 client.16071 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T02:04:14.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.579+0000 7febefde9740 1 -- 172.21.15.16:0/3761898724 --> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5633024aed70 con 0x5633025c7070 2024-09-12T02:04:14.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.579+0000 7febe6ffd640 1 --2- 172.21.15.16:0/3761898724 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x5633025c7070 0x5633025c9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:14.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.580+0000 7febe6ffd640 1 --2- 172.21.15.16:0/3761898724 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x5633025c7070 0x5633025c9490 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b976954d4ab09c55 server_cookie=1ae9629334e85665 in_seq=0 out_seq=0 2024-09-12T02:04:14.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.580+0000 7febc6ffd640 10 client.16071 ms_handle_connect on v2:172.21.15.184:6836/1793393087 2024-09-12T02:04:14.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.580+0000 7febc6ffd640 1 -- 172.21.15.16:0/3761898724 <== mds.2 v2:172.21.15.184:6836/1793393087 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5633024aed70 con 0x5633025c7070 2024-09-12T02:04:14.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.580+0000 7febc6ffd640 10 client.16071 handle_command_reply: tid=0 2024-09-12T02:04:14.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.580+0000 7febefde9740 1 client.16071 shutdown 2024-09-12T02:04:14.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.580+0000 7febefde9740 1 -- 172.21.15.16:0/3761898724 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56330254c370 msgr2=0x563302546210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:14.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.580+0000 7febefde9740 1 --2- 172.21.15.16:0/3761898724 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56330254c370 0x563302546210 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7febd00057c0 tx=0x7febd0013040 comp rx=0 tx=0).stop 2024-09-12T02:04:14.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.580+0000 7febefde9740 1 -- 172.21.15.16:0/3761898724 shutdown_connections 2024-09-12T02:04:14.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.580+0000 7febefde9740 1 --2- 172.21.15.16:0/3761898724 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x5633025c7070 0x5633025c9490 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:14.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.580+0000 7febefde9740 1 --2- 172.21.15.16:0/3761898724 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56330254c370 0x563302546210 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:14.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.580+0000 7febefde9740 1 --2- 172.21.15.16:0/3761898724 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563302542b40 0x563302545cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:14.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.580+0000 7febefde9740 1 --2- 172.21.15.16:0/3761898724 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5633025415d0 0x563302554bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:14.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.580+0000 7febefde9740 1 -- 172.21.15.16:0/3761898724 >> 172.21.15.16:0/3761898724 conn(0x56330250a310 msgr2=0x56330253cd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:04:14.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.580+0000 7febc6ffd640 0 client.16071 ms_handle_reset on v2:172.21.15.184:6836/1793393087 2024-09-12T02:04:14.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.580+0000 7febefde9740 1 -- 172.21.15.16:0/3761898724 shutdown_connections 2024-09-12T02:04:14.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.580+0000 7febefde9740 1 -- 172.21.15.16:0/3761898724 wait complete. 2024-09-12T02:04:14.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.580+0000 7febefde9740 20 client.16071 trim_cache size 0 max 16384 2024-09-12T02:04:14.588 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T02:04:14.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.581+0000 7febeda10640 1 -- 172.21.15.16:0/3288153213 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7febd40a2b80 msgr2=0x7febd40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:14.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.581+0000 7febeda10640 1 --2- 172.21.15.16:0/3288153213 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7febd40a2b80 0x7febd40a5040 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7febe00bccd0 tx=0x7febe0088040 comp rx=0 tx=0).stop 2024-09-12T02:04:14.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.581+0000 7febeda10640 1 -- 172.21.15.16:0/3288153213 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febe8106a80 msgr2=0x7febe81030d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:14.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.581+0000 7febeda10640 1 --2- 172.21.15.16:0/3288153213 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febe8106a80 0x7febe81030d0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7febe0006010 tx=0x7febe0003c20 comp rx=0 tx=0).stop 2024-09-12T02:04:14.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.581+0000 7febeda10640 1 -- 172.21.15.16:0/3288153213 shutdown_connections 2024-09-12T02:04:14.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.581+0000 7febeda10640 1 --2- 172.21.15.16:0/3288153213 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7febd40a2b80 0x7febd40a5040 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:14.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.581+0000 7febeda10640 1 --2- 172.21.15.16:0/3288153213 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febe8106a80 0x7febe81030d0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:14.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.581+0000 7febeda10640 1 --2- 172.21.15.16:0/3288153213 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7febe8101420 0x7febe8102b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:14.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.581+0000 7febeda10640 1 --2- 172.21.15.16:0/3288153213 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7febe8100a80 0x7febe8102650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:14.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.581+0000 7febeda10640 1 -- 172.21.15.16:0/3288153213 >> 172.21.15.16:0/3288153213 conn(0x7febe80fc7a0 msgr2=0x7febe806a830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:04:14.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.581+0000 7febeda10640 1 -- 172.21.15.16:0/3288153213 shutdown_connections 2024-09-12T02:04:14.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:14.581+0000 7febeda10640 1 -- 172.21.15.16:0/3288153213 wait complete. 2024-09-12T02:04:14.788 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 24.7 2024-09-12T02:04:14.789 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:04:15.348 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-12T02:04:15.384 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:04:15.410 INFO:tasks.workunit.client.0.smithi016.stdout:1/600: open fd = 3 2024-09-12T02:04:15.410 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:04:15.410 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:04:15.410 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:04:15.410 INFO:tasks.workunit.client.0.smithi016.stdout:2/600: open fd = 3 2024-09-12T02:04:15.410 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:04:15.410 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:04:15.410 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:04:15.410 INFO:tasks.workunit.client.0.smithi016.stdout:3/600: open fd = 3 2024-09-12T02:04:15.410 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:04:15.410 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:04:15.410 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:04:15.410 INFO:tasks.workunit.client.0.smithi016.stdout:4/600: open fd = 3 2024-09-12T02:04:15.411 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:04:15.411 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:04:15.411 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:04:15.411 INFO:tasks.workunit.client.0.smithi016.stdout:5/600: open fd = 3 2024-09-12T02:04:15.411 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:04:15.411 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:04:15.411 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:04:15.411 INFO:tasks.workunit.client.0.smithi016.stdout:6/600: open fd = 3 2024-09-12T02:04:15.411 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:04:15.411 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:04:15.412 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:04:15.412 INFO:tasks.workunit.client.0.smithi016.stdout:7/600: open fd = 3 2024-09-12T02:04:15.412 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:04:15.412 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:04:15.412 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:04:15.412 INFO:tasks.workunit.client.0.smithi016.stdout:8/600: open fd = 3 2024-09-12T02:04:15.412 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:04:15.412 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:04:15.412 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:04:15.412 INFO:tasks.workunit.client.0.smithi016.stdout:9/600: open fd = 3 2024-09-12T02:04:15.412 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:04:15.412 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:04:15.412 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:04:15.413 INFO:tasks.workunit.client.0.smithi016.stdout:10/600: open fd = 3 2024-09-12T02:04:15.413 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:04:15.413 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:04:15.413 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:04:15.413 INFO:tasks.workunit.client.0.smithi016.stdout:11/600: open fd = 3 2024-09-12T02:04:15.413 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:04:15.413 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:04:15.413 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:04:15.413 INFO:tasks.workunit.client.0.smithi016.stdout:12/600: open fd = 3 2024-09-12T02:04:15.413 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:04:15.413 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:04:15.413 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:04:15.414 INFO:tasks.workunit.client.0.smithi016.stdout:13/600: open fd = 3 2024-09-12T02:04:15.414 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:04:15.414 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:04:15.414 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:04:15.414 INFO:tasks.workunit.client.0.smithi016.stdout:14/600: open fd = 3 2024-09-12T02:04:15.414 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:04:15.414 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:04:15.414 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:04:15.414 INFO:tasks.workunit.client.0.smithi016.stdout:15/600: open fd = 3 2024-09-12T02:04:15.414 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:04:15.415 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:04:15.415 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:04:15.415 INFO:tasks.workunit.client.0.smithi016.stdout:16/600: open fd = 3 2024-09-12T02:04:15.415 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:04:15.415 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:04:15.415 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:04:15.415 INFO:tasks.workunit.client.0.smithi016.stdout:17/600: open fd = 3 2024-09-12T02:04:15.415 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:04:15.415 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:04:15.415 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:04:15.415 INFO:tasks.workunit.client.0.smithi016.stdout:18/600: open fd = 3 2024-09-12T02:04:15.415 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:04:15.415 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:04:15.415 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:04:15.416 INFO:tasks.workunit.client.0.smithi016.stdout:19/600: open fd = 3 2024-09-12T02:04:15.416 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:04:15.416 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:04:15.416 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:04:15.416 INFO:tasks.workunit.client.0.smithi016.stdout:20/600: open fd = 3 2024-09-12T02:04:15.416 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:04:15.416 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:04:15.416 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:04:15.416 INFO:tasks.workunit.client.0.smithi016.stdout:21/600: open fd = 3 2024-09-12T02:04:15.416 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:04:15.416 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:04:15.416 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:04:15.416 INFO:tasks.workunit.client.0.smithi016.stdout:22/600: open fd = 3 2024-09-12T02:04:15.417 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:04:15.417 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:04:15.417 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:04:15.417 INFO:tasks.workunit.client.0.smithi016.stdout:23/600: open fd = 3 2024-09-12T02:04:15.417 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:04:15.417 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:04:15.417 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:04:15.417 INFO:tasks.workunit.client.0.smithi016.stdout:24/600: open fd = 3 2024-09-12T02:04:15.417 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:04:15.417 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:04:15.417 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:04:15.417 INFO:tasks.workunit.client.0.smithi016.stdout:25/600: open fd = 3 2024-09-12T02:04:15.417 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:04:15.418 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:04:15.418 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:04:15.418 INFO:tasks.workunit.client.0.smithi016.stdout:26/600: open fd = 3 2024-09-12T02:04:15.418 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:04:15.418 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:04:15.418 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:04:15.418 INFO:tasks.workunit.client.0.smithi016.stdout:27/600: open fd = 3 2024-09-12T02:04:15.418 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:04:15.418 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:04:15.418 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:04:15.418 INFO:tasks.workunit.client.0.smithi016.stdout:28/600: open fd = 3 2024-09-12T02:04:15.418 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:04:15.419 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:04:15.419 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:04:15.419 INFO:tasks.workunit.client.0.smithi016.stdout:29/600: open fd = 3 2024-09-12T02:04:15.419 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:04:15.419 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:04:15.419 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:04:15.419 INFO:tasks.workunit.client.0.smithi016.stdout:30/600: open fd = 3 2024-09-12T02:04:15.419 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:04:15.419 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:04:15.419 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:04:15.419 INFO:tasks.workunit.client.0.smithi016.stdout:31/600: open fd = 3 2024-09-12T02:04:15.419 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:04:15.419 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:04:15.419 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:04:15.420 INFO:tasks.workunit.client.0.smithi016.stdout:32/600: open fd = 3 2024-09-12T02:04:15.420 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:04:15.420 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:04:15.420 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:04:15.420 INFO:tasks.workunit.client.0.smithi016.stdout:33/600: open fd = 3 2024-09-12T02:04:15.420 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:04:15.420 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:04:15.420 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:04:15.420 INFO:tasks.workunit.client.0.smithi016.stdout:34/600: open fd = 3 2024-09-12T02:04:15.420 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:04:15.420 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:04:15.420 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:04:15.420 INFO:tasks.workunit.client.0.smithi016.stdout:35/600: open fd = 3 2024-09-12T02:04:15.421 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:04:15.421 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:04:15.421 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:04:15.421 INFO:tasks.workunit.client.0.smithi016.stdout:36/600: open fd = 3 2024-09-12T02:04:15.421 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:04:15.421 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:04:15.421 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:04:15.421 INFO:tasks.workunit.client.0.smithi016.stdout:37/600: open fd = 3 2024-09-12T02:04:15.421 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:04:15.421 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:04:15.421 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:04:15.421 INFO:tasks.workunit.client.0.smithi016.stdout:38/600: open fd = 3 2024-09-12T02:04:15.421 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:04:15.422 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:04:15.422 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:04:15.422 INFO:tasks.workunit.client.0.smithi016.stdout:39/600: open fd = 3 2024-09-12T02:04:15.422 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:04:15.422 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:04:15.422 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:04:15.422 INFO:tasks.workunit.client.0.smithi016.stdout:40/600: open fd = 3 2024-09-12T02:04:15.422 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:04:15.422 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:04:15.422 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:04:15.422 INFO:tasks.workunit.client.0.smithi016.stdout:41/600: open fd = 3 2024-09-12T02:04:15.423 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:04:15.423 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:04:15.423 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:04:15.423 INFO:tasks.workunit.client.0.smithi016.stdout:42/600: open fd = 3 2024-09-12T02:04:15.423 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:04:15.423 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:04:15.423 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:04:15.423 INFO:tasks.workunit.client.0.smithi016.stdout:43/600: open fd = 3 2024-09-12T02:04:15.423 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:04:15.423 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:04:15.423 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:04:15.423 INFO:tasks.workunit.client.0.smithi016.stdout:44/600: open fd = 3 2024-09-12T02:04:15.424 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:04:15.424 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:04:15.424 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:04:15.424 INFO:tasks.workunit.client.0.smithi016.stdout:45/600: open fd = 3 2024-09-12T02:04:15.424 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:04:15.424 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:04:15.424 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:04:15.424 INFO:tasks.workunit.client.0.smithi016.stdout:46/600: open fd = 3 2024-09-12T02:04:15.425 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:04:15.425 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:04:15.425 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:04:15.425 INFO:tasks.workunit.client.0.smithi016.stdout:47/600: open fd = 3 2024-09-12T02:04:15.425 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:04:15.425 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:04:15.425 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:04:15.425 INFO:tasks.workunit.client.0.smithi016.stdout:48/600: open fd = 3 2024-09-12T02:04:15.425 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:04:15.425 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:04:15.425 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:04:15.425 INFO:tasks.workunit.client.0.smithi016.stdout:49/600: open fd = 3 2024-09-12T02:04:15.425 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:04:15.425 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:04:15.426 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:04:15.426 INFO:tasks.workunit.client.0.smithi016.stdout:50/600: open fd = 3 2024-09-12T02:04:15.426 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:04:15.426 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:04:15.426 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:04:15.426 INFO:tasks.workunit.client.0.smithi016.stdout:51/600: open fd = 3 2024-09-12T02:04:15.426 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:04:15.426 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:04:15.426 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:04:15.426 INFO:tasks.workunit.client.0.smithi016.stdout:52/600: open fd = 3 2024-09-12T02:04:15.426 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:04:15.426 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:04:15.427 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:04:15.427 INFO:tasks.workunit.client.0.smithi016.stdout:53/600: open fd = 3 2024-09-12T02:04:15.427 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:04:15.427 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:04:15.427 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:04:15.427 INFO:tasks.workunit.client.0.smithi016.stdout:54/600: open fd = 3 2024-09-12T02:04:15.427 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:04:15.427 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:04:15.427 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:04:15.427 INFO:tasks.workunit.client.0.smithi016.stdout:55/600: open fd = 3 2024-09-12T02:04:15.427 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:04:15.427 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:04:15.427 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:04:15.427 INFO:tasks.workunit.client.0.smithi016.stdout:56/600: open fd = 3 2024-09-12T02:04:15.428 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:04:15.428 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:04:15.428 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:04:15.428 INFO:tasks.workunit.client.0.smithi016.stdout:57/600: open fd = 3 2024-09-12T02:04:15.428 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:04:15.428 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:04:15.428 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:04:15.428 INFO:tasks.workunit.client.0.smithi016.stdout:58/600: open fd = 3 2024-09-12T02:04:15.428 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:04:15.428 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:04:15.428 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:04:15.428 INFO:tasks.workunit.client.0.smithi016.stdout:59/600: open fd = 3 2024-09-12T02:04:15.428 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:04:15.428 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:04:15.428 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:04:15.429 INFO:tasks.workunit.client.0.smithi016.stdout:60/600: open fd = 3 2024-09-12T02:04:15.429 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:04:15.429 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:04:15.429 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:04:15.429 INFO:tasks.workunit.client.0.smithi016.stdout:61/600: open fd = 3 2024-09-12T02:04:15.429 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:04:15.429 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:04:15.429 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:04:15.429 INFO:tasks.workunit.client.0.smithi016.stdout:62/600: open fd = 3 2024-09-12T02:04:15.429 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:04:15.429 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:04:15.429 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:04:15.429 INFO:tasks.workunit.client.0.smithi016.stdout:63/600: open fd = 3 2024-09-12T02:04:15.429 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:04:15.430 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:04:15.430 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:04:15.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:15.847+0000 7f8f4cecb640 1 -- 172.21.15.16:0/1724646252 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f48100350 msgr2=0x7f8f48100730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:15.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:15.847+0000 7f8f4cecb640 1 --2- 172.21.15.16:0/1724646252 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f48100350 0x7f8f48100730 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f8f38009930 tx=0x7f8f3802a420 comp rx=0 tx=0).stop 2024-09-12T02:04:15.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:15.847+0000 7f8f4cecb640 1 -- 172.21.15.16:0/1724646252 shutdown_connections 2024-09-12T02:04:15.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:15.847+0000 7f8f4cecb640 1 --2- 172.21.15.16:0/1724646252 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f48106f50 0x7f8f4810eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:15.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:15.847+0000 7f8f4cecb640 1 --2- 172.21.15.16:0/1724646252 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8f48100d00 0x7f8f48106840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:15.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:15.847+0000 7f8f4cecb640 1 --2- 172.21.15.16:0/1724646252 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f48100350 0x7f8f48100730 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:15.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:15.847+0000 7f8f4cecb640 1 -- 172.21.15.16:0/1724646252 >> 172.21.15.16:0/1724646252 conn(0x7f8f48077210 msgr2=0x7f8f480fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:04:15.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:15.848+0000 7f8f4cecb640 1 -- 172.21.15.16:0/1724646252 shutdown_connections 2024-09-12T02:04:15.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:15.848+0000 7f8f4cecb640 1 -- 172.21.15.16:0/1724646252 wait complete. 2024-09-12T02:04:15.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:15.848+0000 7f8f4cecb640 1 Processor -- start 2024-09-12T02:04:15.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:15.849+0000 7f8f4cecb640 1 -- start start 2024-09-12T02:04:15.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:15.849+0000 7f8f4cecb640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f48100350 0x7f8f48198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:15.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:15.849+0000 7f8f4cecb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f48100d00 0x7f8f48198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:15.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:15.849+0000 7f8f4cecb640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8f48106f50 0x7f8f4819cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:15.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:15.849+0000 7f8f4cecb640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8f48112000 con 0x7f8f48100d00 2024-09-12T02:04:15.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:15.849+0000 7f8f4cecb640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8f48111e80 con 0x7f8f48100350 2024-09-12T02:04:15.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:15.849+0000 7f8f4cecb640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8f48112180 con 0x7f8f48106f50 2024-09-12T02:04:15.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:15.849+0000 7f8f4659c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f48100350 0x7f8f48198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:15.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:15.849+0000 7f8f4659c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f48100350 0x7f8f48198520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.16:48146/0 (socket says 172.21.15.16:48146) 2024-09-12T02:04:15.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:15.849+0000 7f8f45d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f48100d00 0x7f8f48198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:15.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:15.849+0000 7f8f4659c640 1 -- 172.21.15.16:0/2846323549 learned_addr learned my addr 172.21.15.16:0/2846323549 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:04:15.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:15.849+0000 7f8f46d9d640 1 --2- 172.21.15.16:0/2846323549 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8f48106f50 0x7f8f4819cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:15.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:15.850+0000 7f8f45d9b640 1 -- 172.21.15.16:0/2846323549 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8f48106f50 msgr2=0x7f8f4819cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:15.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:15.850+0000 7f8f45d9b640 1 --2- 172.21.15.16:0/2846323549 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8f48106f50 0x7f8f4819cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:15.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:15.850+0000 7f8f45d9b640 1 -- 172.21.15.16:0/2846323549 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f48100350 msgr2=0x7f8f48198520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:15.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:15.850+0000 7f8f45d9b640 1 --2- 172.21.15.16:0/2846323549 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f48100350 0x7f8f48198520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:15.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:15.850+0000 7f8f45d9b640 1 -- 172.21.15.16:0/2846323549 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f4819d4d0 con 0x7f8f48100d00 2024-09-12T02:04:15.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:15.850+0000 7f8f45d9b640 1 --2- 172.21.15.16:0/2846323549 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f48100d00 0x7f8f48198a60 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c00c940 tx=0x7f8f3c00ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:04:15.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:15.851+0000 7f8f377fe640 1 -- 172.21.15.16:0/2846323549 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f3c007bb0 con 0x7f8f48100d00 2024-09-12T02:04:15.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:15.851+0000 7f8f4cecb640 1 -- 172.21.15.16:0/2846323549 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f4819d7c0 con 0x7f8f48100d00 2024-09-12T02:04:15.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:15.851+0000 7f8f377fe640 1 -- 172.21.15.16:0/2846323549 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f3c007d50 con 0x7f8f48100d00 2024-09-12T02:04:15.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:15.851+0000 7f8f377fe640 1 -- 172.21.15.16:0/2846323549 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f3c00f450 con 0x7f8f48100d00 2024-09-12T02:04:15.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:15.851+0000 7f8f4cecb640 1 -- 172.21.15.16:0/2846323549 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f481a50a0 con 0x7f8f48100d00 2024-09-12T02:04:15.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:15.852+0000 7f8f4cecb640 1 -- 172.21.15.16:0/2846323549 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f48100730 con 0x7f8f48100d00 2024-09-12T02:04:15.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:15.854+0000 7f8f377fe640 1 -- 172.21.15.16:0/2846323549 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f8f3c004ec0 con 0x7f8f48100d00 2024-09-12T02:04:15.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:15.854+0000 7f8f377fe640 1 --2- 172.21.15.16:0/2846323549 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f8f300a2b80 0x7f8f300a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:15.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:15.854+0000 7f8f4659c640 1 --2- 172.21.15.16:0/2846323549 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f8f300a2b80 0x7f8f300a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:15.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:15.854+0000 7f8f377fe640 1 -- 172.21.15.16:0/2846323549 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f8f3c0ceb20 con 0x7f8f48100d00 2024-09-12T02:04:15.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:15.855+0000 7f8f4659c640 1 --2- 172.21.15.16:0/2846323549 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f8f300a2b80 0x7f8f300a5040 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f8f380096c0 tx=0x7f8f38005df0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:04:15.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:15.859+0000 7f8f377fe640 1 -- 172.21.15.16:0/2846323549 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8f3c014030 con 0x7f8f48100d00 2024-09-12T02:04:15.898 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:04:15.933 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:04:15 smithi016 ceph-mon[30947]: pgmap v405: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-09-12T02:04:16.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:16.077+0000 7f8f4cecb640 1 -- 172.21.15.16:0/2846323549 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8f48109bb0 con 0x7f8f48100d00 2024-09-12T02:04:16.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:16.078+0000 7f8f377fe640 1 -- 172.21.15.16:0/2846323549 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11911 (secure 0 0 0) 0x7f8f3c09b4d0 con 0x7f8f48100d00 2024-09-12T02:04:16.078 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T02:04:16.079 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":25,"btime":"2024-09-12T02:01:37:192021+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3279123279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3279123279},{"type":"v1","addr":"172.21.15.16:6835","nonce":3279123279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4119452382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4119452382},{"type":"v1","addr":"172.21.15.16:6837","nonce":4119452382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/3263823165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3263823165},{"type":"v1","addr":"172.21.15.16:6839","nonce":3263823165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/1510290732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":1510290732},{"type":"v1","addr":"172.21.15.16:6841","nonce":1510290732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/279674603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":279674603},{"type":"v1","addr":"172.21.15.33:6833","nonce":279674603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3555728028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3555728028},{"type":"v1","addr":"172.21.15.33:6835","nonce":3555728028}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T01:59:56.117608+0000","modified":"2024-09-12T02:01:36.294760+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24482,"mds_1":24481,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6833/1563374106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1563374106},{"type":"v1","addr":"172.21.15.184:6833","nonce":1563374106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6837/1892833868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1892833868},{"type":"v1","addr":"172.21.15.33:6837","nonce":1892833868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/1457679686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1457679686},{"type":"v1","addr":"172.21.15.184:6835","nonce":1457679686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.184:6837/1793393087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":1793393087},{"type":"v1","addr":"172.21.15.184:6837","nonce":1793393087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.184:6839/4203347941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":4203347941},{"type":"v1","addr":"172.21.15.184:6839","nonce":4203347941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.33:6839/4136670204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":4136670204},{"type":"v1","addr":"172.21.15.33:6839","nonce":4136670204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24481,24482,24469]},"id":1}]} 2024-09-12T02:04:16.079 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 25 2024-09-12T02:04:16.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:16.080+0000 7f8f4cecb640 1 -- 172.21.15.16:0/2846323549 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f8f300a2b80 msgr2=0x7f8f300a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:16.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:16.080+0000 7f8f4cecb640 1 --2- 172.21.15.16:0/2846323549 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f8f300a2b80 0x7f8f300a5040 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f8f380096c0 tx=0x7f8f38005df0 comp rx=0 tx=0).stop 2024-09-12T02:04:16.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:16.080+0000 7f8f4cecb640 1 -- 172.21.15.16:0/2846323549 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f48100d00 msgr2=0x7f8f48198a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:16.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:16.080+0000 7f8f4cecb640 1 --2- 172.21.15.16:0/2846323549 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f48100d00 0x7f8f48198a60 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c00c940 tx=0x7f8f3c00ce00 comp rx=0 tx=0).stop 2024-09-12T02:04:16.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:16.080+0000 7f8f4cecb640 1 -- 172.21.15.16:0/2846323549 shutdown_connections 2024-09-12T02:04:16.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:16.080+0000 7f8f4cecb640 1 --2- 172.21.15.16:0/2846323549 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f8f300a2b80 0x7f8f300a5040 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:16.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:16.080+0000 7f8f4cecb640 1 --2- 172.21.15.16:0/2846323549 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8f48106f50 0x7f8f4819cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:16.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:16.080+0000 7f8f4cecb640 1 --2- 172.21.15.16:0/2846323549 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f48100d00 0x7f8f48198a60 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:16.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:16.080+0000 7f8f4cecb640 1 --2- 172.21.15.16:0/2846323549 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f48100350 0x7f8f48198520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:16.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:16.080+0000 7f8f4cecb640 1 -- 172.21.15.16:0/2846323549 >> 172.21.15.16:0/2846323549 conn(0x7f8f48077210 msgr2=0x7f8f480fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:04:16.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:16.080+0000 7f8f4cecb640 1 -- 172.21.15.16:0/2846323549 shutdown_connections 2024-09-12T02:04:16.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:16.080+0000 7f8f4cecb640 1 -- 172.21.15.16:0/2846323549 wait complete. 2024-09-12T02:04:16.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:04:15 smithi184 ceph-mon[35593]: pgmap v405: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-09-12T02:04:16.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:04:15 smithi033 ceph-mon[35332]: pgmap v405: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-09-12T02:04:16.762 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24482 quiesce db / --timeout 90 --expiration 144.7 --await-for 45.0' 2024-09-12T02:04:16.762 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24482 quiesce db / --timeout 90 --expiration 144.7 --await-for 45.0 2024-09-12T02:04:17.003 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:04:16 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/2846323549' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:04:17.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.002+0000 7f45d787b640 1 -- 172.21.15.16:0/3331430911 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45d0100aa0 msgr2=0x7f45d010ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:17.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.002+0000 7f45d787b640 1 --2- 172.21.15.16:0/3331430911 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45d0100aa0 0x7f45d010ea20 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f45c8009ca0 tx=0x7f45c802a8c0 comp rx=0 tx=0).stop 2024-09-12T02:04:17.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.002+0000 7f45d787b640 1 -- 172.21.15.16:0/3331430911 shutdown_connections 2024-09-12T02:04:17.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.002+0000 7f45d787b640 1 --2- 172.21.15.16:0/3331430911 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45d0100aa0 0x7f45d010ea20 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:17.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.002+0000 7f45d787b640 1 --2- 172.21.15.16:0/3331430911 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f45d0100100 0x7f45d0100560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:17.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.002+0000 7f45d787b640 1 --2- 172.21.15.16:0/3331430911 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f45d0106b80 0x7f45d0106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:17.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.002+0000 7f45d787b640 1 -- 172.21.15.16:0/3331430911 >> 172.21.15.16:0/3331430911 conn(0x7f45d0077230 msgr2=0x7f45d00fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:04:17.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.003+0000 7f45d787b640 1 -- 172.21.15.16:0/3331430911 shutdown_connections 2024-09-12T02:04:17.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.003+0000 7f45d787b640 1 -- 172.21.15.16:0/3331430911 wait complete. 2024-09-12T02:04:17.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.003+0000 7f45d787b640 1 Processor -- start 2024-09-12T02:04:17.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.003+0000 7f45d787b640 1 -- start start 2024-09-12T02:04:17.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.003+0000 7f45d787b640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f45d0100100 0x7f45d01012e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:17.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.003+0000 7f45d787b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45d0100aa0 0x7f45d0101820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:17.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.003+0000 7f45d787b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f45d0106b80 0x7f45d0101d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:17.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.003+0000 7f45d787b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f45d0112050 con 0x7f45d0100aa0 2024-09-12T02:04:17.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.003+0000 7f45d787b640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f45d0111ed0 con 0x7f45d0106b80 2024-09-12T02:04:17.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.004+0000 7f45d787b640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f45d01121d0 con 0x7f45d0100100 2024-09-12T02:04:17.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.004+0000 7f45d4e16640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45d0100aa0 0x7f45d0101820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:17.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.004+0000 7f45d4e16640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45d0100aa0 0x7f45d0101820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46750/0 (socket says 172.21.15.16:46750) 2024-09-12T02:04:17.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.004+0000 7f45d4e16640 1 -- 172.21.15.16:0/477893674 learned_addr learned my addr 172.21.15.16:0/477893674 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:04:17.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.004+0000 7f45d5e18640 1 --2- 172.21.15.16:0/477893674 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f45d0106b80 0x7f45d0101d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:17.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.004+0000 7f45d5617640 1 --2- 172.21.15.16:0/477893674 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f45d0100100 0x7f45d01012e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:17.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.004+0000 7f45d4e16640 1 -- 172.21.15.16:0/477893674 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f45d0100100 msgr2=0x7f45d01012e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:17.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.004+0000 7f45d4e16640 1 --2- 172.21.15.16:0/477893674 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f45d0100100 0x7f45d01012e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:17.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.004+0000 7f45d4e16640 1 -- 172.21.15.16:0/477893674 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f45d0106b80 msgr2=0x7f45d0101d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:17.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.004+0000 7f45d4e16640 1 --2- 172.21.15.16:0/477893674 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f45d0106b80 0x7f45d0101d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:17.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.004+0000 7f45d4e16640 1 -- 172.21.15.16:0/477893674 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45d01a6950 con 0x7f45d0100aa0 2024-09-12T02:04:17.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.004+0000 7f45d4e16640 1 --2- 172.21.15.16:0/477893674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45d0100aa0 0x7f45d0101820 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f45c000d770 tx=0x7f45c000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:04:17.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.004+0000 7f45c67fc640 1 -- 172.21.15.16:0/477893674 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45c0017070 con 0x7f45d0100aa0 2024-09-12T02:04:17.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.004+0000 7f45d787b640 1 -- 172.21.15.16:0/477893674 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45d01a6c40 con 0x7f45d0100aa0 2024-09-12T02:04:17.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.004+0000 7f45c67fc640 1 -- 172.21.15.16:0/477893674 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45c0005500 con 0x7f45d0100aa0 2024-09-12T02:04:17.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.004+0000 7f45c67fc640 1 -- 172.21.15.16:0/477893674 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45c0004e00 con 0x7f45d0100aa0 2024-09-12T02:04:17.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.004+0000 7f45d787b640 1 -- 172.21.15.16:0/477893674 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45d01a7180 con 0x7f45d0100aa0 2024-09-12T02:04:17.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.007+0000 7f45c67fc640 1 -- 172.21.15.16:0/477893674 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f45c0004fa0 con 0x7f45d0100aa0 2024-09-12T02:04:17.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.007+0000 7f45d9c54740 1 Processor -- start 2024-09-12T02:04:17.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.007+0000 7f45d9c54740 1 -- start start 2024-09-12T02:04:17.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.007+0000 7f45d9c54740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555e03379b40 0x555e03378090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:17.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.007+0000 7f45d9c54740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e033785d0 0x555e03378a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:17.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.007+0000 7f45d9c54740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555e03383370 0x555e03385760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:17.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.007+0000 7f45d9c54740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x555e032de2f0 con 0x555e033785d0 2024-09-12T02:04:17.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.007+0000 7f45d9c54740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x555e03285cf0 con 0x555e03383370 2024-09-12T02:04:17.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.007+0000 7f45d9c54740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x555e032da2b0 con 0x555e03379b40 2024-09-12T02:04:17.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.008+0000 7f45c67fc640 1 --2- 172.21.15.16:0/477893674 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f45bc0a2b80 0x7f45bc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:17.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.008+0000 7f45d5617640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e033785d0 0x555e03378a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:17.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.008+0000 7f45d5617640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e033785d0 0x555e03378a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46758/0 (socket says 172.21.15.16:46758) 2024-09-12T02:04:17.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.008+0000 7f45d4e16640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555e03383370 0x555e03385760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:17.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.008+0000 7f45d5617640 1 -- 172.21.15.16:0/3524618307 learned_addr learned my addr 172.21.15.16:0/3524618307 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:04:17.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.008+0000 7f45d5e18640 1 --2- 172.21.15.16:0/477893674 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f45bc0a2b80 0x7f45bc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:17.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.008+0000 7f45d5e18640 1 --2- 172.21.15.16:0/3524618307 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555e03379b40 0x555e03378090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:17.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.008+0000 7f45c67fc640 1 -- 172.21.15.16:0/477893674 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f45c00cead0 con 0x7f45d0100aa0 2024-09-12T02:04:17.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.008+0000 7f45d5617640 1 -- 172.21.15.16:0/3524618307 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555e03379b40 msgr2=0x555e03378090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:17.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.008+0000 7f45d5617640 1 --2- 172.21.15.16:0/3524618307 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555e03379b40 0x555e03378090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:17.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.008+0000 7f45d5617640 1 -- 172.21.15.16:0/3524618307 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555e03383370 msgr2=0x555e03385760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:17.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.008+0000 7f45d5617640 1 --2- 172.21.15.16:0/3524618307 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555e03383370 0x555e03385760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:17.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.008+0000 7f45d5617640 1 -- 172.21.15.16:0/3524618307 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e0326a3a0 con 0x555e033785d0 2024-09-12T02:04:17.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.008+0000 7f45d4e16640 1 --2- 172.21.15.16:0/3524618307 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555e03383370 0x555e03385760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:04:17.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.008+0000 7f45d5617640 1 --2- 172.21.15.16:0/3524618307 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e033785d0 0x555e03378a10 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f45cc009c00 tx=0x7f45cc0117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48ee93d0aa9bbd2d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:04:17.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.008+0000 7f45d5e18640 1 --2- 172.21.15.16:0/477893674 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f45bc0a2b80 0x7f45bc0a5040 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f45c8009430 tx=0x7f45c80093a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:04:17.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.008+0000 7f45bb7fe640 1 -- 172.21.15.16:0/3524618307 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45cc012450 con 0x555e033785d0 2024-09-12T02:04:17.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.008+0000 7f45bb7fe640 1 -- 172.21.15.16:0/3524618307 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45cc012ad0 con 0x555e033785d0 2024-09-12T02:04:17.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.008+0000 7f45bb7fe640 1 -- 172.21.15.16:0/3524618307 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45cc00fb80 con 0x555e033785d0 2024-09-12T02:04:17.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.009+0000 7f45d9c54740 1 -- 172.21.15.16:0/3524618307 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e033785d0 msgr2=0x555e03378a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:17.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.009+0000 7f45d9c54740 1 --2- 172.21.15.16:0/3524618307 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e033785d0 0x555e03378a10 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f45cc009c00 tx=0x7f45cc0117b0 comp rx=0 tx=0).stop 2024-09-12T02:04:17.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.009+0000 7f45d9c54740 1 -- 172.21.15.16:0/3524618307 shutdown_connections 2024-09-12T02:04:17.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.009+0000 7f45d9c54740 1 --2- 172.21.15.16:0/3524618307 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555e03383370 0x555e03385760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:17.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.009+0000 7f45d9c54740 1 --2- 172.21.15.16:0/3524618307 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e033785d0 0x555e03378a10 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:17.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.009+0000 7f45d9c54740 1 --2- 172.21.15.16:0/3524618307 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555e03379b40 0x555e03378090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:17.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.009+0000 7f45d9c54740 1 -- 172.21.15.16:0/3524618307 >> 172.21.15.16:0/3524618307 conn(0x555e03341310 msgr2=0x555e03374c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:04:17.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.009+0000 7f45d9c54740 1 -- 172.21.15.16:0/3524618307 shutdown_connections 2024-09-12T02:04:17.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.009+0000 7f45d9c54740 1 -- 172.21.15.16:0/3524618307 wait complete. 2024-09-12T02:04:17.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.010+0000 7f45d9c54740 1 -- start start 2024-09-12T02:04:17.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.010+0000 7f45d9c54740 1 Processor -- start 2024-09-12T02:04:17.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.010+0000 7f45d9c54740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555e033785d0 0x555e03382d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:17.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.010+0000 7f45d9c54740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e03379b40 0x555e0337ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:17.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.010+0000 7f45d9c54740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555e03383370 0x555e03380560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:17.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.010+0000 7f45d9c54740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x555e03386070 con 0x555e03379b40 2024-09-12T02:04:17.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.010+0000 7f45d9c54740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x555e03299060 con 0x555e03383370 2024-09-12T02:04:17.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.010+0000 7f45d9c54740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x555e033861f0 con 0x555e033785d0 2024-09-12T02:04:17.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.010+0000 7f45d9c54740 10 client.0 _ll_register_callbacks cb 0x555e02fec8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:04:17.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.011+0000 7f45d4e16640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e03379b40 0x555e0337ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:17.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.011+0000 7f45d4e16640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e03379b40 0x555e0337ffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46772/0 (socket says 172.21.15.16:46772) 2024-09-12T02:04:17.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.011+0000 7f45d4e16640 1 -- 172.21.15.16:0/2498788348 learned_addr learned my addr 172.21.15.16:0/2498788348 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:04:17.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.011+0000 7f45d5e18640 1 --2- 172.21.15.16:0/2498788348 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555e03383370 0x555e03380560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:17.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.011+0000 7f45d5617640 1 --2- 172.21.15.16:0/2498788348 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555e033785d0 0x555e03382d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:17.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.011+0000 7f45d4e16640 1 -- 172.21.15.16:0/2498788348 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555e033785d0 msgr2=0x555e03382d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:17.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.011+0000 7f45d4e16640 1 --2- 172.21.15.16:0/2498788348 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555e033785d0 0x555e03382d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:17.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.011+0000 7f45d4e16640 1 -- 172.21.15.16:0/2498788348 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555e03383370 msgr2=0x555e03380560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:17.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.011+0000 7f45d4e16640 1 --2- 172.21.15.16:0/2498788348 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555e03383370 0x555e03380560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:17.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.011+0000 7f45d4e16640 1 -- 172.21.15.16:0/2498788348 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e0304f3c0 con 0x555e03379b40 2024-09-12T02:04:17.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.011+0000 7f45d4e16640 1 --2- 172.21.15.16:0/2498788348 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e03379b40 0x555e0337ffc0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f45c00cc670 tx=0x7f45c00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1d939bc31593bac server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:04:17.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.011+0000 7f45b8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:04:17.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.011+0000 7f45b8ff9640 1 -- 172.21.15.16:0/2498788348 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45c0020070 con 0x555e03379b40 2024-09-12T02:04:17.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.011+0000 7f45b8ff9640 1 -- 172.21.15.16:0/2498788348 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f45c0009cc0 con 0x555e03379b40 2024-09-12T02:04:17.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.011+0000 7f45d9c54740 1 -- 172.21.15.16:0/2498788348 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45cc012ad0 con 0x555e03379b40 2024-09-12T02:04:17.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.011+0000 7f45b8ff9640 1 -- 172.21.15.16:0/2498788348 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f45c009f780 con 0x555e03379b40 2024-09-12T02:04:17.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.011+0000 7f45d9c54740 10 client.16089 fetch_fsmap learned FSMap version 25 2024-09-12T02:04:17.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.011+0000 7f45d9c54740 1 -- 172.21.15.16:0/2498788348 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555e03391b10 con 0x555e03379b40 2024-09-12T02:04:17.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.011+0000 7f45b8ff9640 1 -- 172.21.15.16:0/2498788348 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45cc012ad0 con 0x555e03379b40 2024-09-12T02:04:17.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.012+0000 7f45b8ff9640 1 -- 172.21.15.16:0/2498788348 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f45c0097a40 con 0x555e03379b40 2024-09-12T02:04:17.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.012+0000 7f45d9c54740 10 client.16089 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:04:17.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.012+0000 7f45d9c54740 10 client.16089 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T02:04:17.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.012+0000 7f45d9c54740 20 client.16089 populate_metadata read hostname 'smithi016' 2024-09-12T02:04:17.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.012+0000 7f45d9c54740 1 --2- 172.21.15.16:0/2498788348 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x555e03393450 0x555e033958d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:17.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.012+0000 7f45d9c54740 4 client.16089 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:04:17.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.012+0000 7f45d9c54740 1 -- 172.21.15.16:0/2498788348 --> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555e032eb860 con 0x555e03393450 2024-09-12T02:04:17.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.012+0000 7f45d5617640 1 --2- 172.21.15.16:0/2498788348 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x555e03393450 0x555e033958d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:17.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.013+0000 7f45d5617640 1 --2- 172.21.15.16:0/2498788348 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x555e03393450 0x555e033958d0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f4d65637f4176177 server_cookie=1845f39c4fe1db4a in_seq=0 out_seq=0 2024-09-12T02:04:17.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.013+0000 7f45b8ff9640 10 client.16089 ms_handle_connect on v2:172.21.15.184:6838/4203347941 2024-09-12T02:04:17.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.013+0000 7f45b8ff9640 1 -- 172.21.15.16:0/2498788348 <== mds.1 v2:172.21.15.184:6838/4203347941 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555e032eb860 con 0x555e03393450 2024-09-12T02:04:17.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.013+0000 7f45b8ff9640 10 client.16089 handle_command_reply: tid=0 2024-09-12T02:04:17.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.013+0000 7f45d9c54740 1 client.16089 shutdown 2024-09-12T02:04:17.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.013+0000 7f45d9c54740 1 -- 172.21.15.16:0/2498788348 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e03379b40 msgr2=0x555e0337ffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:17.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.013+0000 7f45d9c54740 1 --2- 172.21.15.16:0/2498788348 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e03379b40 0x555e0337ffc0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f45c00cc670 tx=0x7f45c00046c0 comp rx=0 tx=0).stop 2024-09-12T02:04:17.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.013+0000 7f45d9c54740 1 -- 172.21.15.16:0/2498788348 shutdown_connections 2024-09-12T02:04:17.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.013+0000 7f45d9c54740 1 --2- 172.21.15.16:0/2498788348 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x555e03393450 0x555e033958d0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:17.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.014+0000 7f45d9c54740 1 --2- 172.21.15.16:0/2498788348 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555e03383370 0x555e03380560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:17.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.014+0000 7f45d9c54740 1 --2- 172.21.15.16:0/2498788348 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e03379b40 0x555e0337ffc0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:17.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.014+0000 7f45b8ff9640 0 client.16089 ms_handle_reset on v2:172.21.15.184:6838/4203347941 2024-09-12T02:04:17.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.014+0000 7f45d9c54740 1 --2- 172.21.15.16:0/2498788348 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555e033785d0 0x555e03382d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:17.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.014+0000 7f45d9c54740 1 -- 172.21.15.16:0/2498788348 >> 172.21.15.16:0/2498788348 conn(0x555e03341310 msgr2=0x555e0337a960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:04:17.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.014+0000 7f45d9c54740 1 -- 172.21.15.16:0/2498788348 shutdown_connections 2024-09-12T02:04:17.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.014+0000 7f45d9c54740 1 -- 172.21.15.16:0/2498788348 wait complete. 2024-09-12T02:04:17.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.014+0000 7f45d9c54740 20 client.16089 trim_cache size 0 max 16384 2024-09-12T02:04:17.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.027+0000 7f45d9c54740 1 Processor -- start 2024-09-12T02:04:17.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.027+0000 7f45d9c54740 1 -- start start 2024-09-12T02:04:17.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.027+0000 7f45d9c54740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555e033785d0 0x555e03385760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:17.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.027+0000 7f45d9c54740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e03379b40 0x555e0338f310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:17.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.027+0000 7f45d9c54740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555e03383370 0x555e0337b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:17.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.027+0000 7f45d9c54740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f45d01b1ed0 con 0x555e03379b40 2024-09-12T02:04:17.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.027+0000 7f45d9c54740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f45d01af7a0 con 0x555e03383370 2024-09-12T02:04:17.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.027+0000 7f45d9c54740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f45d01ad5f0 con 0x555e033785d0 2024-09-12T02:04:17.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.027+0000 7f45d4e16640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e03379b40 0x555e0338f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:17.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.027+0000 7f45d4e16640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e03379b40 0x555e0338f310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46786/0 (socket says 172.21.15.16:46786) 2024-09-12T02:04:17.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.027+0000 7f45d4e16640 1 -- 172.21.15.16:0/1251847251 learned_addr learned my addr 172.21.15.16:0/1251847251 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:04:17.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.027+0000 7f45d5617640 1 --2- 172.21.15.16:0/1251847251 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555e033785d0 0x555e03385760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:17.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.027+0000 7f45d4e16640 1 -- 172.21.15.16:0/1251847251 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555e033785d0 msgr2=0x555e03385760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:17.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.027+0000 7f45d4e16640 1 --2- 172.21.15.16:0/1251847251 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555e033785d0 0x555e03385760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:17.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.027+0000 7f45d4e16640 1 -- 172.21.15.16:0/1251847251 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555e03383370 msgr2=0x555e0337b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:17.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.027+0000 7f45d4e16640 1 --2- 172.21.15.16:0/1251847251 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555e03383370 0x555e0337b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:17.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.027+0000 7f45d4e16640 1 -- 172.21.15.16:0/1251847251 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e03383040 con 0x555e03379b40 2024-09-12T02:04:17.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.027+0000 7f45d5617640 1 --2- 172.21.15.16:0/1251847251 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555e033785d0 0x555e03385760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:04:17.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.028+0000 7f45d4e16640 1 --2- 172.21.15.16:0/1251847251 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e03379b40 0x555e0338f310 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f45c0009610 tx=0x7f45c00048a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cddb5ee7c2fd4910 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:04:17.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.028+0000 7f45baffd640 1 -- 172.21.15.16:0/1251847251 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45c0020070 con 0x555e03379b40 2024-09-12T02:04:17.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.028+0000 7f45baffd640 1 -- 172.21.15.16:0/1251847251 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45c0015070 con 0x555e03379b40 2024-09-12T02:04:17.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.028+0000 7f45baffd640 1 -- 172.21.15.16:0/1251847251 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45c00cc360 con 0x555e03379b40 2024-09-12T02:04:17.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.028+0000 7f45d9c54740 1 -- 172.21.15.16:0/1251847251 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e03379b40 msgr2=0x555e0338f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:17.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.028+0000 7f45d9c54740 1 --2- 172.21.15.16:0/1251847251 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e03379b40 0x555e0338f310 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f45c0009610 tx=0x7f45c00048a0 comp rx=0 tx=0).stop 2024-09-12T02:04:17.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.028+0000 7f45d9c54740 1 -- 172.21.15.16:0/1251847251 shutdown_connections 2024-09-12T02:04:17.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.028+0000 7f45d9c54740 1 --2- 172.21.15.16:0/1251847251 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555e03383370 0x555e0337b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:17.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.028+0000 7f45d9c54740 1 --2- 172.21.15.16:0/1251847251 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e03379b40 0x555e0338f310 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:17.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.028+0000 7f45d9c54740 1 --2- 172.21.15.16:0/1251847251 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555e033785d0 0x555e03385760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:17.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.028+0000 7f45d9c54740 1 -- 172.21.15.16:0/1251847251 >> 172.21.15.16:0/1251847251 conn(0x555e03341310 msgr2=0x555e0336fa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:04:17.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.029+0000 7f45d9c54740 1 -- 172.21.15.16:0/1251847251 shutdown_connections 2024-09-12T02:04:17.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.029+0000 7f45d9c54740 1 -- 172.21.15.16:0/1251847251 wait complete. 2024-09-12T02:04:17.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.029+0000 7f45d9c54740 1 -- start start 2024-09-12T02:04:17.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.029+0000 7f45d9c54740 1 Processor -- start 2024-09-12T02:04:17.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.029+0000 7f45d9c54740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555e033785d0 0x555e0338b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:17.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.029+0000 7f45d9c54740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e03379b40 0x555e0338bc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:17.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.029+0000 7f45d9c54740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555e03383370 0x555e0337c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:17.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.029+0000 7f45d9c54740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f45d01adc60 con 0x555e03379b40 2024-09-12T02:04:17.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.029+0000 7f45d9c54740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f45d01af0d0 con 0x555e03383370 2024-09-12T02:04:17.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.029+0000 7f45d9c54740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f45d01ae330 con 0x555e033785d0 2024-09-12T02:04:17.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.029+0000 7f45d9c54740 10 client.0 _ll_register_callbacks cb 0x7f45c009b250 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:04:17.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.029+0000 7f45d4e16640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e03379b40 0x555e0338bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:17.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.029+0000 7f45d4e16640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e03379b40 0x555e0338bc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46798/0 (socket says 172.21.15.16:46798) 2024-09-12T02:04:17.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.029+0000 7f45d4e16640 1 -- 172.21.15.16:0/1878550863 learned_addr learned my addr 172.21.15.16:0/1878550863 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:04:17.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.030+0000 7f45d5e18640 1 --2- 172.21.15.16:0/1878550863 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555e03383370 0x555e0337c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:17.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.030+0000 7f45d5617640 1 --2- 172.21.15.16:0/1878550863 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555e033785d0 0x555e0338b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:17.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.030+0000 7f45d4e16640 1 -- 172.21.15.16:0/1878550863 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555e033785d0 msgr2=0x555e0338b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:17.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.030+0000 7f45d4e16640 1 --2- 172.21.15.16:0/1878550863 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555e033785d0 0x555e0338b6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:17.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.030+0000 7f45d4e16640 1 -- 172.21.15.16:0/1878550863 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555e03383370 msgr2=0x555e0337c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:17.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.030+0000 7f45d4e16640 1 --2- 172.21.15.16:0/1878550863 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555e03383370 0x555e0337c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:17.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.030+0000 7f45d4e16640 1 -- 172.21.15.16:0/1878550863 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e0337d010 con 0x555e03379b40 2024-09-12T02:04:17.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.030+0000 7f45d4e16640 1 --2- 172.21.15.16:0/1878550863 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e03379b40 0x555e0338bc10 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f45c000d740 tx=0x7f45c0095df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e4cbd7587238d70 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:04:17.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.030+0000 7f45b8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:04:17.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.030+0000 7f45b8ff9640 1 -- 172.21.15.16:0/1878550863 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45c00b8070 con 0x555e03379b40 2024-09-12T02:04:17.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.030+0000 7f45b8ff9640 1 -- 172.21.15.16:0/1878550863 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f45c00cc360 con 0x555e03379b40 2024-09-12T02:04:17.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.030+0000 7f45d9c54740 1 -- 172.21.15.16:0/1878550863 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45c0015070 con 0x555e03379b40 2024-09-12T02:04:17.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.030+0000 7f45b8ff9640 1 -- 172.21.15.16:0/1878550863 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f45c0020070 con 0x555e03379b40 2024-09-12T02:04:17.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.030+0000 7f45d9c54740 10 client.16095 fetch_fsmap learned FSMap version 25 2024-09-12T02:04:17.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.030+0000 7f45d9c54740 1 -- 172.21.15.16:0/1878550863 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555e033fdc20 con 0x555e03379b40 2024-09-12T02:04:17.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.031+0000 7f45b8ff9640 1 -- 172.21.15.16:0/1878550863 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45c00a4040 con 0x555e03379b40 2024-09-12T02:04:17.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.031+0000 7f45b8ff9640 1 -- 172.21.15.16:0/1878550863 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f45c00cca90 con 0x555e03379b40 2024-09-12T02:04:17.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.031+0000 7f45d9c54740 10 client.16095 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:04:17.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.031+0000 7f45d9c54740 10 client.16095 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T02:04:17.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.031+0000 7f45d9c54740 20 client.16095 populate_metadata read hostname 'smithi016' 2024-09-12T02:04:17.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.031+0000 7f45d9c54740 1 --2- 172.21.15.16:0/1878550863 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x555e033fe140 0x555e03400530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:17.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.031+0000 7f45d9c54740 4 client.16095 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T02:04:17.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.031+0000 7f45d9c54740 1 -- 172.21.15.16:0/1878550863 --> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f45c0015070 con 0x555e033fe140 2024-09-12T02:04:17.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.031+0000 7f45d5617640 1 --2- 172.21.15.16:0/1878550863 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x555e033fe140 0x555e03400530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:17.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.031+0000 7f45b8ff9640 10 client.16095 ms_handle_connect on v2:172.21.15.184:6838/4203347941 2024-09-12T02:04:17.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.031+0000 7f45d5617640 1 --2- 172.21.15.16:0/1878550863 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x555e033fe140 0x555e03400530 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ba9701d9b59be39a server_cookie=ea8b478d11e3987a in_seq=0 out_seq=0 2024-09-12T02:04:17.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.032+0000 7f45b8ff9640 1 -- 172.21.15.16:0/1878550863 <== mds.1 v2:172.21.15.184:6838/4203347941 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f45c0015070 con 0x555e033fe140 2024-09-12T02:04:17.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.032+0000 7f45b8ff9640 10 client.16095 handle_command_reply: tid=0 2024-09-12T02:04:17.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.032+0000 7f45d9c54740 1 client.16095 shutdown 2024-09-12T02:04:17.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.032+0000 7f45d9c54740 1 -- 172.21.15.16:0/1878550863 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e03379b40 msgr2=0x555e0338bc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:17.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.032+0000 7f45d9c54740 1 --2- 172.21.15.16:0/1878550863 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e03379b40 0x555e0338bc10 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f45c000d740 tx=0x7f45c0095df0 comp rx=0 tx=0).stop 2024-09-12T02:04:17.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.032+0000 7f45d9c54740 1 -- 172.21.15.16:0/1878550863 shutdown_connections 2024-09-12T02:04:17.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.032+0000 7f45d9c54740 1 --2- 172.21.15.16:0/1878550863 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x555e033fe140 0x555e03400530 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:17.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.032+0000 7f45d9c54740 1 --2- 172.21.15.16:0/1878550863 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555e03383370 0x555e0337c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:17.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.032+0000 7f45d9c54740 1 --2- 172.21.15.16:0/1878550863 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e03379b40 0x555e0338bc10 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:17.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.032+0000 7f45b8ff9640 0 client.16095 ms_handle_reset on v2:172.21.15.184:6838/4203347941 2024-09-12T02:04:17.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.032+0000 7f45d9c54740 1 --2- 172.21.15.16:0/1878550863 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555e033785d0 0x555e0338b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:17.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.032+0000 7f45d9c54740 1 -- 172.21.15.16:0/1878550863 >> 172.21.15.16:0/1878550863 conn(0x555e03341310 msgr2=0x555e03382040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:04:17.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.032+0000 7f45d9c54740 1 -- 172.21.15.16:0/1878550863 shutdown_connections 2024-09-12T02:04:17.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.032+0000 7f45d9c54740 1 -- 172.21.15.16:0/1878550863 wait complete. 2024-09-12T02:04:17.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.032+0000 7f45d9c54740 20 client.16095 trim_cache size 0 max 16384 2024-09-12T02:04:17.054 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T02:04:17.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.033+0000 7f45d787b640 1 -- 172.21.15.16:0/477893674 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f45bc0a2b80 msgr2=0x7f45bc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:17.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.033+0000 7f45d787b640 1 --2- 172.21.15.16:0/477893674 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f45bc0a2b80 0x7f45bc0a5040 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f45c8009430 tx=0x7f45c80093a0 comp rx=0 tx=0).stop 2024-09-12T02:04:17.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.033+0000 7f45d787b640 1 -- 172.21.15.16:0/477893674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45d0100aa0 msgr2=0x7f45d0101820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:17.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.033+0000 7f45d787b640 1 --2- 172.21.15.16:0/477893674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45d0100aa0 0x7f45d0101820 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f45c000d770 tx=0x7f45c000dc30 comp rx=0 tx=0).stop 2024-09-12T02:04:17.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.033+0000 7f45d787b640 1 -- 172.21.15.16:0/477893674 shutdown_connections 2024-09-12T02:04:17.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.033+0000 7f45d787b640 1 --2- 172.21.15.16:0/477893674 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f45bc0a2b80 0x7f45bc0a5040 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:17.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.033+0000 7f45d787b640 1 --2- 172.21.15.16:0/477893674 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f45d0106b80 0x7f45d0101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:17.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.033+0000 7f45d787b640 1 --2- 172.21.15.16:0/477893674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45d0100aa0 0x7f45d0101820 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:17.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.033+0000 7f45d787b640 1 --2- 172.21.15.16:0/477893674 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f45d0100100 0x7f45d01012e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:17.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.033+0000 7f45d787b640 1 -- 172.21.15.16:0/477893674 >> 172.21.15.16:0/477893674 conn(0x7f45d0077230 msgr2=0x7f45d0104b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:04:17.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.033+0000 7f45d787b640 1 -- 172.21.15.16:0/477893674 shutdown_connections 2024-09-12T02:04:17.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:17.033+0000 7f45d787b640 1 -- 172.21.15.16:0/477893674 wait complete. 2024-09-12T02:04:17.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:04:16 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/2846323549' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:04:17.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:04:16 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/2846323549' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:04:17.612 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-12T02:04:17.634 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:04:17.867 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:04:17 smithi016 ceph-mon[30947]: pgmap v406: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-09-12T02:04:18.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.075+0000 7f9b17a84640 1 -- 172.21.15.16:0/296831541 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b10102d30 msgr2=0x7f9b10103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:18.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.075+0000 7f9b17a84640 1 --2- 172.21.15.16:0/296831541 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b10102d30 0x7f9b10103110 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f9b0c009930 tx=0x7f9b0c02a420 comp rx=0 tx=0).stop 2024-09-12T02:04:18.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.075+0000 7f9b17a84640 1 -- 172.21.15.16:0/296831541 shutdown_connections 2024-09-12T02:04:18.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.075+0000 7f9b17a84640 1 --2- 172.21.15.16:0/296831541 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9b101082a0 0x7f9b1010ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:18.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.075+0000 7f9b17a84640 1 --2- 172.21.15.16:0/296831541 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9b101036e0 0x7f9b10107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:18.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.075+0000 7f9b17a84640 1 --2- 172.21.15.16:0/296831541 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b10102d30 0x7f9b10103110 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:18.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.075+0000 7f9b17a84640 1 -- 172.21.15.16:0/296831541 >> 172.21.15.16:0/296831541 conn(0x7f9b100fc7d0 msgr2=0x7f9b100febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:04:18.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.076+0000 7f9b17a84640 1 -- 172.21.15.16:0/296831541 shutdown_connections 2024-09-12T02:04:18.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.076+0000 7f9b17a84640 1 -- 172.21.15.16:0/296831541 wait complete. 2024-09-12T02:04:18.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.076+0000 7f9b17a84640 1 Processor -- start 2024-09-12T02:04:18.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.076+0000 7f9b17a84640 1 -- start start 2024-09-12T02:04:18.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.076+0000 7f9b17a84640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9b10102d30 0x7f9b101a0440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:18.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.076+0000 7f9b17a84640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9b101036e0 0x7f9b101a0980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:18.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.076+0000 7f9b17a84640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b101082a0 0x7f9b1019a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:18.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.076+0000 7f9b17a84640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9b101120a0 con 0x7f9b101082a0 2024-09-12T02:04:18.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.076+0000 7f9b17a84640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9b10111f20 con 0x7f9b101036e0 2024-09-12T02:04:18.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.076+0000 7f9b17a84640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f9b10112220 con 0x7f9b10102d30 2024-09-12T02:04:18.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.077+0000 7f9b16021640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b101082a0 0x7f9b1019a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:18.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.077+0000 7f9b1501f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9b101036e0 0x7f9b101a0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:18.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.077+0000 7f9b16021640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b101082a0 0x7f9b1019a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46826/0 (socket says 172.21.15.16:46826) 2024-09-12T02:04:18.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.077+0000 7f9b16021640 1 -- 172.21.15.16:0/979796591 learned_addr learned my addr 172.21.15.16:0/979796591 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:04:18.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.077+0000 7f9b15820640 1 --2- 172.21.15.16:0/979796591 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9b10102d30 0x7f9b101a0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:18.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.077+0000 7f9b16021640 1 -- 172.21.15.16:0/979796591 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9b10102d30 msgr2=0x7f9b101a0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:18.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.077+0000 7f9b16021640 1 --2- 172.21.15.16:0/979796591 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9b10102d30 0x7f9b101a0440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:18.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.077+0000 7f9b16021640 1 -- 172.21.15.16:0/979796591 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9b101036e0 msgr2=0x7f9b101a0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:18.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.077+0000 7f9b16021640 1 --2- 172.21.15.16:0/979796591 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9b101036e0 0x7f9b101a0980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:18.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.077+0000 7f9b16021640 1 -- 172.21.15.16:0/979796591 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b1019ad30 con 0x7f9b101082a0 2024-09-12T02:04:18.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.077+0000 7f9b15820640 1 --2- 172.21.15.16:0/979796591 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9b10102d30 0x7f9b101a0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:04:18.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.077+0000 7f9b16021640 1 --2- 172.21.15.16:0/979796591 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b101082a0 0x7f9b1019a600 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f9b0800cad0 tx=0x7f9b0800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:04:18.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.078+0000 7f9b06ffd640 1 -- 172.21.15.16:0/979796591 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b0800ae20 con 0x7f9b101082a0 2024-09-12T02:04:18.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.078+0000 7f9b06ffd640 1 -- 172.21.15.16:0/979796591 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b08005020 con 0x7f9b101082a0 2024-09-12T02:04:18.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.078+0000 7f9b17a84640 1 -- 172.21.15.16:0/979796591 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b1019b020 con 0x7f9b101082a0 2024-09-12T02:04:18.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.078+0000 7f9b06ffd640 1 -- 172.21.15.16:0/979796591 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b0800f6f0 con 0x7f9b101082a0 2024-09-12T02:04:18.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.078+0000 7f9b17a84640 1 -- 172.21.15.16:0/979796591 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b101a7310 con 0x7f9b101082a0 2024-09-12T02:04:18.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.080+0000 7f9b06ffd640 1 -- 172.21.15.16:0/979796591 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f9b080051d0 con 0x7f9b101082a0 2024-09-12T02:04:18.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.081+0000 7f9b19e5d740 1 Processor -- start 2024-09-12T02:04:18.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.081+0000 7f9b19e5d740 1 -- start start 2024-09-12T02:04:18.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.081+0000 7f9b19e5d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555f93d98760 0x555f93d98b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:18.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.081+0000 7f9b19e5d740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555f93d99080 0x555f93da3400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:18.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.081+0000 7f9b19e5d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555f93da3940 0x555f93da5d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:18.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.081+0000 7f9b19e5d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x555f93cfe2a0 con 0x555f93da3940 2024-09-12T02:04:18.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.081+0000 7f9b19e5d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x555f93ca5b10 con 0x555f93d98760 2024-09-12T02:04:18.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.081+0000 7f9b19e5d740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x555f93cfa260 con 0x555f93d99080 2024-09-12T02:04:18.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.081+0000 7f9b1501f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555f93da3940 0x555f93da5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:18.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.081+0000 7f9b1501f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555f93da3940 0x555f93da5d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46832/0 (socket says 172.21.15.16:46832) 2024-09-12T02:04:18.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.081+0000 7f9b1501f640 1 -- 172.21.15.16:0/252144869 learned_addr learned my addr 172.21.15.16:0/252144869 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:04:18.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.081+0000 7f9b1501f640 1 --2- 172.21.15.16:0/252144869 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555f93d98760 0x555f93d98b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:18.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.081+0000 7f9b15820640 1 --2- 172.21.15.16:0/252144869 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555f93d99080 0x555f93da3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:18.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.081+0000 7f9b06ffd640 1 --2- 172.21.15.16:0/979796591 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f9afc0a2c00 0x7f9afc0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:18.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.082+0000 7f9b1501f640 1 -- 172.21.15.16:0/252144869 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555f93d99080 msgr2=0x555f93da3400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:18.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.082+0000 7f9b1501f640 1 --2- 172.21.15.16:0/252144869 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555f93d99080 0x555f93da3400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:18.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.082+0000 7f9b1501f640 1 -- 172.21.15.16:0/252144869 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555f93d98760 msgr2=0x555f93d98b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:18.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.082+0000 7f9b16021640 1 --2- 172.21.15.16:0/979796591 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f9afc0a2c00 0x7f9afc0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:18.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.082+0000 7f9b1501f640 1 --2- 172.21.15.16:0/252144869 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555f93d98760 0x555f93d98b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:18.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.082+0000 7f9b1501f640 1 -- 172.21.15.16:0/252144869 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f93a6fb90 con 0x555f93da3940 2024-09-12T02:04:18.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.082+0000 7f9b06ffd640 1 -- 172.21.15.16:0/979796591 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f9b080cf220 con 0x7f9b101082a0 2024-09-12T02:04:18.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.082+0000 7f9b1501f640 1 --2- 172.21.15.16:0/252144869 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555f93da3940 0x555f93da5d30 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f9b1006a370 tx=0x7f9b00004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87d761757b23a359 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:04:18.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.082+0000 7f9afbfff640 1 -- 172.21.15.16:0/252144869 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b00012ca0 con 0x555f93da3940 2024-09-12T02:04:18.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.082+0000 7f9b16021640 1 --2- 172.21.15.16:0/979796591 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f9afc0a2c00 0x7f9afc0a50c0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f9b0809b820 tx=0x7f9b080667a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:04:18.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.082+0000 7f9afbfff640 1 -- 172.21.15.16:0/252144869 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b00012e40 con 0x555f93da3940 2024-09-12T02:04:18.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.082+0000 7f9afbfff640 1 -- 172.21.15.16:0/252144869 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b00012ca0 con 0x555f93da3940 2024-09-12T02:04:18.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.083+0000 7f9b19e5d740 1 -- 172.21.15.16:0/252144869 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555f93da3940 msgr2=0x555f93da5d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:18.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.083+0000 7f9b19e5d740 1 --2- 172.21.15.16:0/252144869 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555f93da3940 0x555f93da5d30 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f9b1006a370 tx=0x7f9b00004730 comp rx=0 tx=0).stop 2024-09-12T02:04:18.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.083+0000 7f9b19e5d740 1 -- 172.21.15.16:0/252144869 shutdown_connections 2024-09-12T02:04:18.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.083+0000 7f9b19e5d740 1 --2- 172.21.15.16:0/252144869 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555f93da3940 0x555f93da5d30 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:18.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.083+0000 7f9b19e5d740 1 --2- 172.21.15.16:0/252144869 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555f93d99080 0x555f93da3400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:18.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.083+0000 7f9b19e5d740 1 --2- 172.21.15.16:0/252144869 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555f93d98760 0x555f93d98b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:18.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.083+0000 7f9b19e5d740 1 -- 172.21.15.16:0/252144869 >> 172.21.15.16:0/252144869 conn(0x555f93d94cf0 msgr2=0x555f93d950d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:04:18.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.083+0000 7f9b19e5d740 1 -- 172.21.15.16:0/252144869 shutdown_connections 2024-09-12T02:04:18.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.083+0000 7f9b19e5d740 1 -- 172.21.15.16:0/252144869 wait complete. 2024-09-12T02:04:18.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.084+0000 7f9b19e5d740 1 -- start start 2024-09-12T02:04:18.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.084+0000 7f9b19e5d740 1 Processor -- start 2024-09-12T02:04:18.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.084+0000 7f9b19e5d740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555f93d98760 0x555f93da0840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:18.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.084+0000 7f9b19e5d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555f93d99080 0x555f93da0d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:18.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.084+0000 7f9b19e5d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555f93da3940 0x555f93da12c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:18.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.084+0000 7f9b19e5d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x555f93da6510 con 0x555f93da3940 2024-09-12T02:04:18.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.084+0000 7f9b19e5d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x555f93c866e0 con 0x555f93d99080 2024-09-12T02:04:18.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.084+0000 7f9b19e5d740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x555f93da6690 con 0x555f93d98760 2024-09-12T02:04:18.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.084+0000 7f9b19e5d740 10 client.0 _ll_register_callbacks cb 0x555f93caaa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:04:18.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.085+0000 7f9b1501f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555f93da3940 0x555f93da12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:18.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.085+0000 7f9b1501f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555f93da3940 0x555f93da12c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46838/0 (socket says 172.21.15.16:46838) 2024-09-12T02:04:18.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.085+0000 7f9b1501f640 1 -- 172.21.15.16:0/3107274650 learned_addr learned my addr 172.21.15.16:0/3107274650 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:04:18.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.085+0000 7f9b1501f640 1 --2- 172.21.15.16:0/3107274650 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555f93d98760 0x555f93da0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:18.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.085+0000 7f9b15820640 1 --2- 172.21.15.16:0/3107274650 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555f93d99080 0x555f93da0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:18.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.085+0000 7f9b1501f640 1 -- 172.21.15.16:0/3107274650 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555f93d98760 msgr2=0x555f93da0840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:18.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.085+0000 7f9b1501f640 1 --2- 172.21.15.16:0/3107274650 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555f93d98760 0x555f93da0840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:18.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.085+0000 7f9b1501f640 1 -- 172.21.15.16:0/3107274650 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555f93d99080 msgr2=0x555f93da0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:18.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.085+0000 7f9b1501f640 1 --2- 172.21.15.16:0/3107274650 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555f93d99080 0x555f93da0d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:18.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.085+0000 7f9b1501f640 1 -- 172.21.15.16:0/3107274650 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f93dafa60 con 0x555f93da3940 2024-09-12T02:04:18.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.085+0000 7f9b15820640 1 --2- 172.21.15.16:0/3107274650 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555f93d99080 0x555f93da0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:04:18.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.085+0000 7f9b1501f640 1 --2- 172.21.15.16:0/3107274650 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555f93da3940 0x555f93da12c0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f9b00005fd0 tx=0x7f9b00017040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=606584314ffef449 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:04:18.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.085+0000 7f9af97fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:04:18.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.086+0000 7f9af97fa640 1 -- 172.21.15.16:0/3107274650 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b00012ca0 con 0x555f93da3940 2024-09-12T02:04:18.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.086+0000 7f9af97fa640 1 -- 172.21.15.16:0/3107274650 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f9b0001b070 con 0x555f93da3940 2024-09-12T02:04:18.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.086+0000 7f9b19e5d740 1 -- 172.21.15.16:0/3107274650 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x555f93d0b9d0 con 0x555f93da3940 2024-09-12T02:04:18.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.086+0000 7f9af97fa640 1 -- 172.21.15.16:0/3107274650 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9b000042f0 con 0x555f93da3940 2024-09-12T02:04:18.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.086+0000 7f9b19e5d740 10 client.16107 fetch_fsmap learned FSMap version 25 2024-09-12T02:04:18.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.086+0000 7f9b19e5d740 1 -- 172.21.15.16:0/3107274650 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555f93db23d0 con 0x555f93da3940 2024-09-12T02:04:18.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.086+0000 7f9af97fa640 1 -- 172.21.15.16:0/3107274650 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x555f93d0b9d0 con 0x555f93da3940 2024-09-12T02:04:18.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.086+0000 7f9af97fa640 1 -- 172.21.15.16:0/3107274650 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f9b0001f470 con 0x555f93da3940 2024-09-12T02:04:18.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.086+0000 7f9b19e5d740 10 client.16107 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:04:18.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.086+0000 7f9b19e5d740 10 client.16107 resolve_mds: validated gid 24482 aka daemon mds.b 2024-09-12T02:04:18.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.086+0000 7f9b19e5d740 20 client.16107 populate_metadata read hostname 'smithi016' 2024-09-12T02:04:18.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.086+0000 7f9b19e5d740 1 --2- 172.21.15.16:0/3107274650 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x555f93db3720 0x555f93db5b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:18.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.087+0000 7f9b19e5d740 4 client.16107 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:04:18.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.087+0000 7f9b19e5d740 1 -- 172.21.15.16:0/3107274650 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555f93d05d20 con 0x555f93db3720 2024-09-12T02:04:18.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.087+0000 7f9b15820640 1 --2- 172.21.15.16:0/3107274650 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x555f93db3720 0x555f93db5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:18.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.087+0000 7f9af97fa640 10 client.16107 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:04:18.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.087+0000 7f9b15820640 1 --2- 172.21.15.16:0/3107274650 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x555f93db3720 0x555f93db5b10 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6fe566783bc0c1b5 server_cookie=6f36f63145e03922 in_seq=0 out_seq=0 2024-09-12T02:04:18.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.088+0000 7f9af97fa640 1 -- 172.21.15.16:0/3107274650 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555f93d05d20 con 0x555f93db3720 2024-09-12T02:04:18.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.088+0000 7f9af97fa640 10 client.16107 handle_command_reply: tid=0 2024-09-12T02:04:18.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.088+0000 7f9b19e5d740 1 client.16107 shutdown 2024-09-12T02:04:18.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.088+0000 7f9b19e5d740 1 -- 172.21.15.16:0/3107274650 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555f93da3940 msgr2=0x555f93da12c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:18.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.088+0000 7f9b19e5d740 1 --2- 172.21.15.16:0/3107274650 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555f93da3940 0x555f93da12c0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f9b00005fd0 tx=0x7f9b00017040 comp rx=0 tx=0).stop 2024-09-12T02:04:18.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.089+0000 7f9b19e5d740 1 -- 172.21.15.16:0/3107274650 shutdown_connections 2024-09-12T02:04:18.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.089+0000 7f9b19e5d740 1 --2- 172.21.15.16:0/3107274650 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x555f93db3720 0x555f93db5b10 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:18.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.089+0000 7f9b19e5d740 1 --2- 172.21.15.16:0/3107274650 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555f93da3940 0x555f93da12c0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:18.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.089+0000 7f9b19e5d740 1 --2- 172.21.15.16:0/3107274650 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555f93d99080 0x555f93da0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:18.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.089+0000 7f9b19e5d740 1 --2- 172.21.15.16:0/3107274650 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555f93d98760 0x555f93da0840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:18.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.089+0000 7f9af97fa640 0 client.16107 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:04:18.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.089+0000 7f9b19e5d740 1 -- 172.21.15.16:0/3107274650 >> 172.21.15.16:0/3107274650 conn(0x555f93d94cf0 msgr2=0x555f93d9a9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:04:18.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.089+0000 7f9b19e5d740 1 -- 172.21.15.16:0/3107274650 shutdown_connections 2024-09-12T02:04:18.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.089+0000 7f9b19e5d740 1 -- 172.21.15.16:0/3107274650 wait complete. 2024-09-12T02:04:18.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.089+0000 7f9b19e5d740 20 client.16107 trim_cache size 0 max 16384 2024-09-12T02:04:18.105 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:04:18.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.106+0000 7f9b19e5d740 1 Processor -- start 2024-09-12T02:04:18.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.106+0000 7f9b19e5d740 1 -- start start 2024-09-12T02:04:18.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.107+0000 7f9b19e5d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555f93d98760 0x555f93d9a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:18.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.107+0000 7f9b19e5d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555f93d99080 0x555f93dbb170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:18.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.107+0000 7f9b19e5d740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555f93da3940 0x555f93dbb6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:18.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.107+0000 7f9b19e5d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9b101b20c0 con 0x555f93d99080 2024-09-12T02:04:18.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.107+0000 7f9b19e5d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9b101af990 con 0x555f93d98760 2024-09-12T02:04:18.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.107+0000 7f9b19e5d740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f9b101ad7a0 con 0x555f93da3940 2024-09-12T02:04:18.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.107+0000 7f9b15820640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555f93d99080 0x555f93dbb170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:18.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.107+0000 7f9b15820640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555f93d99080 0x555f93dbb170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46850/0 (socket says 172.21.15.16:46850) 2024-09-12T02:04:18.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.107+0000 7f9b15820640 1 -- 172.21.15.16:0/2212543979 learned_addr learned my addr 172.21.15.16:0/2212543979 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:04:18.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.107+0000 7f9b1501f640 1 --2- 172.21.15.16:0/2212543979 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555f93d98760 0x555f93d9a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:18.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.107+0000 7f9b1501f640 1 --2- 172.21.15.16:0/2212543979 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555f93da3940 0x555f93dbb6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:18.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.107+0000 7f9b15820640 1 -- 172.21.15.16:0/2212543979 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555f93da3940 msgr2=0x555f93dbb6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:18.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.107+0000 7f9b15820640 1 --2- 172.21.15.16:0/2212543979 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555f93da3940 0x555f93dbb6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:18.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.107+0000 7f9b15820640 1 -- 172.21.15.16:0/2212543979 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555f93d98760 msgr2=0x555f93d9a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:18.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.107+0000 7f9b15820640 1 --2- 172.21.15.16:0/2212543979 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555f93d98760 0x555f93d9a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:18.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.107+0000 7f9b15820640 1 -- 172.21.15.16:0/2212543979 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f93d98eb0 con 0x555f93d99080 2024-09-12T02:04:18.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.107+0000 7f9b15820640 1 --2- 172.21.15.16:0/2212543979 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555f93d99080 0x555f93dbb170 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f9b0c02a940 tx=0x7f9b0c031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9db8d19f2aed988 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:04:18.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.108+0000 7f9afb7fe640 1 -- 172.21.15.16:0/2212543979 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b0c035040 con 0x555f93d99080 2024-09-12T02:04:18.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.108+0000 7f9afb7fe640 1 -- 172.21.15.16:0/2212543979 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b0c049070 con 0x555f93d99080 2024-09-12T02:04:18.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.108+0000 7f9afb7fe640 1 -- 172.21.15.16:0/2212543979 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b0c004920 con 0x555f93d99080 2024-09-12T02:04:18.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.108+0000 7f9b19e5d740 1 -- 172.21.15.16:0/2212543979 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555f93d99080 msgr2=0x555f93dbb170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:18.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.108+0000 7f9b19e5d740 1 --2- 172.21.15.16:0/2212543979 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555f93d99080 0x555f93dbb170 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f9b0c02a940 tx=0x7f9b0c031400 comp rx=0 tx=0).stop 2024-09-12T02:04:18.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.109+0000 7f9b19e5d740 1 -- 172.21.15.16:0/2212543979 shutdown_connections 2024-09-12T02:04:18.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.109+0000 7f9b19e5d740 1 --2- 172.21.15.16:0/2212543979 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555f93da3940 0x555f93dbb6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:18.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.109+0000 7f9b19e5d740 1 --2- 172.21.15.16:0/2212543979 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555f93d99080 0x555f93dbb170 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:18.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.109+0000 7f9b19e5d740 1 --2- 172.21.15.16:0/2212543979 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555f93d98760 0x555f93d9a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:18.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.109+0000 7f9b19e5d740 1 -- 172.21.15.16:0/2212543979 >> 172.21.15.16:0/2212543979 conn(0x555f93d94cf0 msgr2=0x555f93d9a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:04:18.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.109+0000 7f9b19e5d740 1 -- 172.21.15.16:0/2212543979 shutdown_connections 2024-09-12T02:04:18.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.109+0000 7f9b19e5d740 1 -- 172.21.15.16:0/2212543979 wait complete. 2024-09-12T02:04:18.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.109+0000 7f9b19e5d740 1 -- start start 2024-09-12T02:04:18.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.109+0000 7f9b19e5d740 1 Processor -- start 2024-09-12T02:04:18.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.110+0000 7f9b19e5d740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555f93d98760 0x555f93da7810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:18.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.110+0000 7f9b19e5d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555f93d99080 0x555f93da7d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:18.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.110+0000 7f9b19e5d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555f93da3940 0x555f93da8290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:18.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.110+0000 7f9b19e5d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9b101ade50 con 0x555f93da3940 2024-09-12T02:04:18.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.110+0000 7f9b19e5d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9b101af2c0 con 0x555f93d99080 2024-09-12T02:04:18.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.110+0000 7f9b19e5d740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f9b101ae520 con 0x555f93d98760 2024-09-12T02:04:18.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.110+0000 7f9b19e5d740 10 client.0 _ll_register_callbacks cb 0x7f9b00010740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:04:18.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.110+0000 7f9b1501f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555f93da3940 0x555f93da8290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:18.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.110+0000 7f9b1501f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555f93da3940 0x555f93da8290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46860/0 (socket says 172.21.15.16:46860) 2024-09-12T02:04:18.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.110+0000 7f9b1501f640 1 -- 172.21.15.16:0/1694141264 learned_addr learned my addr 172.21.15.16:0/1694141264 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:04:18.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.110+0000 7f9b15820640 1 --2- 172.21.15.16:0/1694141264 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555f93d99080 0x555f93da7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:18.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.110+0000 7f9b1501f640 1 --2- 172.21.15.16:0/1694141264 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555f93d98760 0x555f93da7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:18.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.110+0000 7f9b15820640 1 -- 172.21.15.16:0/1694141264 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555f93d98760 msgr2=0x555f93da7810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:18.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.110+0000 7f9b15820640 1 --2- 172.21.15.16:0/1694141264 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555f93d98760 0x555f93da7810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:18.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.110+0000 7f9b15820640 1 -- 172.21.15.16:0/1694141264 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555f93da3940 msgr2=0x555f93da8290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:18.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.110+0000 7f9b15820640 1 --2- 172.21.15.16:0/1694141264 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555f93da3940 0x555f93da8290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:18.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.110+0000 7f9b15820640 1 -- 172.21.15.16:0/1694141264 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f93e1d2e0 con 0x555f93d99080 2024-09-12T02:04:18.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.111+0000 7f9b15820640 1 --2- 172.21.15.16:0/1694141264 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555f93d99080 0x555f93da7d50 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9b0c009900 tx=0x7f9b0c031510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec81b6cfbe0ecb36 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:04:18.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.111+0000 7f9af97fa640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-12T02:04:18.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.111+0000 7f9af97fa640 1 -- 172.21.15.16:0/1694141264 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b0c035040 con 0x555f93d99080 2024-09-12T02:04:18.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.111+0000 7f9b19e5d740 1 -- 172.21.15.16:0/1694141264 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9b0c049070 con 0x555f93d99080 2024-09-12T02:04:18.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.111+0000 7f9af97fa640 1 -- 172.21.15.16:0/1694141264 <== mon.2 v2:172.21.15.33:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f9b0c004240 con 0x555f93d99080 2024-09-12T02:04:18.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.111+0000 7f9af97fa640 1 -- 172.21.15.16:0/1694141264 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9b0c05c050 con 0x555f93d99080 2024-09-12T02:04:18.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.111+0000 7f9b19e5d740 10 client.25715 fetch_fsmap learned FSMap version 25 2024-09-12T02:04:18.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.111+0000 7f9b19e5d740 1 -- 172.21.15.16:0/1694141264 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555f93e1e5a0 con 0x555f93d99080 2024-09-12T02:04:18.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.111+0000 7f9af97fa640 1 -- 172.21.15.16:0/1694141264 <== mon.2 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b0c049070 con 0x555f93d99080 2024-09-12T02:04:18.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.112+0000 7f9af97fa640 1 -- 172.21.15.16:0/1694141264 <== mon.2 v2:172.21.15.33:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f9b0c03b4c0 con 0x555f93d99080 2024-09-12T02:04:18.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.112+0000 7f9b19e5d740 10 client.25715 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:04:18.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.112+0000 7f9b19e5d740 10 client.25715 resolve_mds: validated gid 24482 aka daemon mds.b 2024-09-12T02:04:18.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.112+0000 7f9b19e5d740 20 client.25715 populate_metadata read hostname 'smithi016' 2024-09-12T02:04:18.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.112+0000 7f9b19e5d740 1 --2- 172.21.15.16:0/1694141264 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x555f93e1edb0 0x555f93e211d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:18.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.112+0000 7f9b19e5d740 4 client.25715 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 144.7, "await_for": 45.0}] 2024-09-12T02:04:18.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.112+0000 7f9b19e5d740 1 -- 172.21.15.16:0/1694141264 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 144.7, "await_for": 45.0}) -- 0x555f93c81ec0 con 0x555f93e1edb0 2024-09-12T02:04:18.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.112+0000 7f9b1501f640 1 --2- 172.21.15.16:0/1694141264 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x555f93e1edb0 0x555f93e211d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:18.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.112+0000 7f9af97fa640 10 client.25715 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:04:18.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.112+0000 7f9b1501f640 1 --2- 172.21.15.16:0/1694141264 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x555f93e1edb0 0x555f93e211d0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=18ab0acca7f86137 server_cookie=636717da7d04ecd in_seq=0 out_seq=0 2024-09-12T02:04:18.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.124+0000 7f9af97fa640 1 -- 172.21.15.16:0/1694141264 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x555f93c81ec0 con 0x555f93e1edb0 2024-09-12T02:04:18.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.124+0000 7f9af97fa640 10 client.25715 handle_command_reply: tid=0 2024-09-12T02:04:18.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.124+0000 7f9b19e5d740 1 client.25715 shutdown 2024-09-12T02:04:18.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.124+0000 7f9b19e5d740 1 -- 172.21.15.16:0/1694141264 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555f93d99080 msgr2=0x555f93da7d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:18.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.124+0000 7f9b19e5d740 1 --2- 172.21.15.16:0/1694141264 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555f93d99080 0x555f93da7d50 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9b0c009900 tx=0x7f9b0c031510 comp rx=0 tx=0).stop 2024-09-12T02:04:18.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.124+0000 7f9b19e5d740 1 -- 172.21.15.16:0/1694141264 shutdown_connections 2024-09-12T02:04:18.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.124+0000 7f9b19e5d740 1 --2- 172.21.15.16:0/1694141264 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x555f93e1edb0 0x555f93e211d0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:18.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.124+0000 7f9b19e5d740 1 --2- 172.21.15.16:0/1694141264 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555f93da3940 0x555f93da8290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:18.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.124+0000 7f9b19e5d740 1 --2- 172.21.15.16:0/1694141264 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555f93d99080 0x555f93da7d50 secure :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9b0c009900 tx=0x7f9b0c031510 comp rx=0 tx=0).stop 2024-09-12T02:04:18.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.124+0000 7f9b19e5d740 1 --2- 172.21.15.16:0/1694141264 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555f93d98760 0x555f93da7810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:18.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.124+0000 7f9af97fa640 0 client.25715 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:04:18.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.124+0000 7f9b19e5d740 1 -- 172.21.15.16:0/1694141264 >> 172.21.15.16:0/1694141264 conn(0x555f93d94cf0 msgr2=0x555f93d93e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:04:18.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.125+0000 7f9b19e5d740 1 -- 172.21.15.16:0/1694141264 shutdown_connections 2024-09-12T02:04:18.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.125+0000 7f9b19e5d740 1 -- 172.21.15.16:0/1694141264 wait complete. 2024-09-12T02:04:18.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.125+0000 7f9b19e5d740 20 client.25715 trim_cache size 0 max 16384 2024-09-12T02:04:18.126 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T02:04:18.126 INFO:teuthology.orchestra.run.smithi016.stdout: "epoch": 25, 2024-09-12T02:04:18.126 INFO:teuthology.orchestra.run.smithi016.stdout: "leader": 24482, 2024-09-12T02:04:18.126 INFO:teuthology.orchestra.run.smithi016.stdout: "set_version": 10, 2024-09-12T02:04:18.127 INFO:teuthology.orchestra.run.smithi016.stdout: "sets": { 2024-09-12T02:04:18.127 INFO:teuthology.orchestra.run.smithi016.stdout: "146e8aed": { 2024-09-12T02:04:18.127 INFO:teuthology.orchestra.run.smithi016.stdout: "version": 10, 2024-09-12T02:04:18.127 INFO:teuthology.orchestra.run.smithi016.stdout: "age_ref": 0.0, 2024-09-12T02:04:18.127 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-12T02:04:18.127 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "QUIESCED", 2024-09-12T02:04:18.127 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-12T02:04:18.127 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:04:18.127 INFO:teuthology.orchestra.run.smithi016.stdout: "timeout": 90.0, 2024-09-12T02:04:18.127 INFO:teuthology.orchestra.run.smithi016.stdout: "expiration": 144.7, 2024-09-12T02:04:18.127 INFO:teuthology.orchestra.run.smithi016.stdout: "members": { 2024-09-12T02:04:18.127 INFO:teuthology.orchestra.run.smithi016.stdout: "file:/": { 2024-09-12T02:04:18.127 INFO:teuthology.orchestra.run.smithi016.stdout: "excluded": false, 2024-09-12T02:04:18.127 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-12T02:04:18.128 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "QUIESCED", 2024-09-12T02:04:18.128 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-12T02:04:18.128 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:04:18.128 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:04:18.128 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:04:18.128 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:04:18.128 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:04:18.128 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T02:04:18.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.125+0000 7f9b17a84640 1 -- 172.21.15.16:0/979796591 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f9afc0a2c00 msgr2=0x7f9afc0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:18.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.125+0000 7f9b17a84640 1 --2- 172.21.15.16:0/979796591 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f9afc0a2c00 0x7f9afc0a50c0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f9b0809b820 tx=0x7f9b080667a0 comp rx=0 tx=0).stop 2024-09-12T02:04:18.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.125+0000 7f9b17a84640 1 -- 172.21.15.16:0/979796591 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b101082a0 msgr2=0x7f9b1019a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:18.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.125+0000 7f9b17a84640 1 --2- 172.21.15.16:0/979796591 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b101082a0 0x7f9b1019a600 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f9b0800cad0 tx=0x7f9b0800cf90 comp rx=0 tx=0).stop 2024-09-12T02:04:18.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.126+0000 7f9b17a84640 1 -- 172.21.15.16:0/979796591 shutdown_connections 2024-09-12T02:04:18.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.126+0000 7f9b17a84640 1 --2- 172.21.15.16:0/979796591 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f9afc0a2c00 0x7f9afc0a50c0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:18.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.126+0000 7f9b17a84640 1 --2- 172.21.15.16:0/979796591 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b101082a0 0x7f9b1019a600 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:18.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.126+0000 7f9b17a84640 1 --2- 172.21.15.16:0/979796591 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9b101036e0 0x7f9b101a0980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:18.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.126+0000 7f9b17a84640 1 --2- 172.21.15.16:0/979796591 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9b10102d30 0x7f9b101a0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:18.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.126+0000 7f9b17a84640 1 -- 172.21.15.16:0/979796591 >> 172.21.15.16:0/979796591 conn(0x7f9b100fc7d0 msgr2=0x7f9b10100cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:04:18.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.126+0000 7f9b17a84640 1 -- 172.21.15.16:0/979796591 shutdown_connections 2024-09-12T02:04:18.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.126+0000 7f9b17a84640 1 -- 172.21.15.16:0/979796591 wait complete. 2024-09-12T02:04:18.130 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:04:17 smithi184 ceph-mon[35593]: pgmap v406: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-09-12T02:04:18.168 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:04:17 smithi033 ceph-mon[35332]: pgmap v406: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-09-12T02:04:18.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.846+0000 7fca091cb640 1 -- 172.21.15.16:0/366586302 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca0410c170 msgr2=0x7fca0410e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:18.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.846+0000 7fca091cb640 1 --2- 172.21.15.16:0/366586302 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca0410c170 0x7fca0410e610 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fc9fc009ca0 tx=0x7fc9fc02a950 comp rx=0 tx=0).stop 2024-09-12T02:04:18.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.846+0000 7fca091cb640 1 -- 172.21.15.16:0/366586302 shutdown_connections 2024-09-12T02:04:18.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.846+0000 7fca091cb640 1 --2- 172.21.15.16:0/366586302 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca0410c170 0x7fca0410e610 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:18.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.846+0000 7fca091cb640 1 --2- 172.21.15.16:0/366586302 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca04103660 0x7fca0410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:18.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.846+0000 7fca091cb640 1 --2- 172.21.15.16:0/366586302 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fca04102d40 0x7fca04103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:18.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.846+0000 7fca091cb640 1 -- 172.21.15.16:0/366586302 >> 172.21.15.16:0/366586302 conn(0x7fca040fc7a0 msgr2=0x7fca040febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:04:18.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.846+0000 7fca091cb640 1 -- 172.21.15.16:0/366586302 shutdown_connections 2024-09-12T02:04:18.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.847+0000 7fca091cb640 1 -- 172.21.15.16:0/366586302 wait complete. 2024-09-12T02:04:18.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.848+0000 7fca091cb640 1 Processor -- start 2024-09-12T02:04:18.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.848+0000 7fca091cb640 1 -- start start 2024-09-12T02:04:18.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.848+0000 7fca091cb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca04102d40 0x7fca041009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:18.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.848+0000 7fca091cb640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca04103660 0x7fca04100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:18.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.848+0000 7fca091cb640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fca0410c170 0x7fca04101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:18.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.848+0000 7fca091cb640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fca04111fb0 con 0x7fca04102d40 2024-09-12T02:04:18.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.848+0000 7fca091cb640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fca04111e30 con 0x7fca04103660 2024-09-12T02:04:18.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.848+0000 7fca091cb640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fca04112130 con 0x7fca0410c170 2024-09-12T02:04:18.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.848+0000 7fca02d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca04102d40 0x7fca041009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:18.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.848+0000 7fca02d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca04102d40 0x7fca041009a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46864/0 (socket says 172.21.15.16:46864) 2024-09-12T02:04:18.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.848+0000 7fca02d9d640 1 -- 172.21.15.16:0/2794657534 learned_addr learned my addr 172.21.15.16:0/2794657534 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:04:18.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.848+0000 7fca0359e640 1 --2- 172.21.15.16:0/2794657534 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fca0410c170 0x7fca04101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:18.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.848+0000 7fca0259c640 1 --2- 172.21.15.16:0/2794657534 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca04103660 0x7fca04100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:18.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.848+0000 7fca02d9d640 1 -- 172.21.15.16:0/2794657534 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fca0410c170 msgr2=0x7fca04101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:18.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.848+0000 7fca02d9d640 1 --2- 172.21.15.16:0/2794657534 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fca0410c170 0x7fca04101420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:18.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.849+0000 7fca02d9d640 1 -- 172.21.15.16:0/2794657534 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca04103660 msgr2=0x7fca04100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:18.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.849+0000 7fca02d9d640 1 --2- 172.21.15.16:0/2794657534 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca04103660 0x7fca04100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:18.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.849+0000 7fca02d9d640 1 -- 172.21.15.16:0/2794657534 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca041a6930 con 0x7fca04102d40 2024-09-12T02:04:18.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.849+0000 7fca0259c640 1 --2- 172.21.15.16:0/2794657534 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca04103660 0x7fca04100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:04:18.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.849+0000 7fca02d9d640 1 --2- 172.21.15.16:0/2794657534 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca04102d40 0x7fca041009a0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fc9f400c890 tx=0x7fc9f400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:04:18.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.849+0000 7fc9ebfff640 1 -- 172.21.15.16:0/2794657534 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9f4004220 con 0x7fca04102d40 2024-09-12T02:04:18.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.849+0000 7fca091cb640 1 -- 172.21.15.16:0/2794657534 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca041a6c20 con 0x7fca04102d40 2024-09-12T02:04:18.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.849+0000 7fc9ebfff640 1 -- 172.21.15.16:0/2794657534 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc9f40043c0 con 0x7fca04102d40 2024-09-12T02:04:18.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.849+0000 7fc9ebfff640 1 -- 172.21.15.16:0/2794657534 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9f4004a00 con 0x7fca04102d40 2024-09-12T02:04:18.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.849+0000 7fca091cb640 1 -- 172.21.15.16:0/2794657534 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca041a7160 con 0x7fca04102d40 2024-09-12T02:04:18.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.851+0000 7fc9ebfff640 1 -- 172.21.15.16:0/2794657534 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fc9f40079b0 con 0x7fca04102d40 2024-09-12T02:04:18.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.851+0000 7fca0b5a4740 1 Processor -- start 2024-09-12T02:04:18.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.851+0000 7fca0b5a4740 1 -- start start 2024-09-12T02:04:18.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.852+0000 7fca0b5a4740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55621f4f4b40 0x55621f4f3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:18.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.852+0000 7fca0b5a4740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55621f4f35d0 0x55621f4f3a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:18.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.852+0000 7fca0b5a4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55621f4fe370 0x55621f500760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:18.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.852+0000 7fca0b5a4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55621f4592f0 con 0x55621f4fe370 2024-09-12T02:04:18.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.852+0000 7fca0b5a4740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55621f400cf0 con 0x55621f4f4b40 2024-09-12T02:04:18.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.852+0000 7fca0b5a4740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55621f4552b0 con 0x55621f4f35d0 2024-09-12T02:04:18.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.852+0000 7fca0259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55621f4fe370 0x55621f500760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:18.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.852+0000 7fc9ebfff640 1 --2- 172.21.15.16:0/2794657534 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc9ec0a2b80 0x7fc9ec0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:18.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.852+0000 7fca0259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55621f4fe370 0x55621f500760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46876/0 (socket says 172.21.15.16:46876) 2024-09-12T02:04:18.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.852+0000 7fca0259c640 1 -- 172.21.15.16:0/2086912536 learned_addr learned my addr 172.21.15.16:0/2086912536 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:04:18.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.852+0000 7fca02d9d640 1 --2- 172.21.15.16:0/2086912536 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55621f4f35d0 0x55621f4f3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:18.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.852+0000 7fca0359e640 1 --2- 172.21.15.16:0/2086912536 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55621f4f4b40 0x55621f4f3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:18.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.852+0000 7fc9ebfff640 1 -- 172.21.15.16:0/2794657534 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fc9f40ced50 con 0x7fca04102d40 2024-09-12T02:04:18.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.852+0000 7fca0359e640 1 --2- 172.21.15.16:0/2794657534 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc9ec0a2b80 0x7fc9ec0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:18.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.852+0000 7fca0259c640 1 -- 172.21.15.16:0/2086912536 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55621f4f35d0 msgr2=0x55621f4f3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:18.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.852+0000 7fca0259c640 1 --2- 172.21.15.16:0/2086912536 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55621f4f35d0 0x55621f4f3a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:18.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.852+0000 7fca0259c640 1 -- 172.21.15.16:0/2086912536 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55621f4f4b40 msgr2=0x55621f4f3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:18.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.852+0000 7fca0259c640 1 --2- 172.21.15.16:0/2086912536 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55621f4f4b40 0x55621f4f3090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:18.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.852+0000 7fca0259c640 1 -- 172.21.15.16:0/2086912536 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55621f3e53a0 con 0x55621f4fe370 2024-09-12T02:04:18.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.853+0000 7fca0259c640 1 --2- 172.21.15.16:0/2086912536 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55621f4fe370 0x55621f500760 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fc9f800a9c0 tx=0x7fc9f800ae80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7ac62974e4bd67b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:04:18.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.853+0000 7fca0359e640 1 --2- 172.21.15.16:0/2794657534 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc9ec0a2b80 0x7fc9ec0a5040 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fc9fc009a60 tx=0x7fc9fc0095c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:04:18.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.853+0000 7fc9e97fa640 1 -- 172.21.15.16:0/2086912536 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9f8015070 con 0x55621f4fe370 2024-09-12T02:04:18.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.853+0000 7fc9e97fa640 1 -- 172.21.15.16:0/2086912536 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc9f800cc60 con 0x55621f4fe370 2024-09-12T02:04:18.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.853+0000 7fc9e97fa640 1 -- 172.21.15.16:0/2086912536 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9f8010710 con 0x55621f4fe370 2024-09-12T02:04:18.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.853+0000 7fca0b5a4740 1 -- 172.21.15.16:0/2086912536 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55621f4fe370 msgr2=0x55621f500760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:18.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.853+0000 7fca0b5a4740 1 --2- 172.21.15.16:0/2086912536 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55621f4fe370 0x55621f500760 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fc9f800a9c0 tx=0x7fc9f800ae80 comp rx=0 tx=0).stop 2024-09-12T02:04:18.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.854+0000 7fca0b5a4740 1 -- 172.21.15.16:0/2086912536 shutdown_connections 2024-09-12T02:04:18.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.854+0000 7fca0b5a4740 1 --2- 172.21.15.16:0/2086912536 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55621f4fe370 0x55621f500760 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:18.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.854+0000 7fca0b5a4740 1 --2- 172.21.15.16:0/2086912536 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55621f4f35d0 0x55621f4f3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:18.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.854+0000 7fca0b5a4740 1 --2- 172.21.15.16:0/2086912536 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55621f4f4b40 0x55621f4f3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:18.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.854+0000 7fca0b5a4740 1 -- 172.21.15.16:0/2086912536 >> 172.21.15.16:0/2086912536 conn(0x55621f4bc310 msgr2=0x55621f4efc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:04:18.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.854+0000 7fca0b5a4740 1 -- 172.21.15.16:0/2086912536 shutdown_connections 2024-09-12T02:04:18.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.854+0000 7fca0b5a4740 1 -- 172.21.15.16:0/2086912536 wait complete. 2024-09-12T02:04:18.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.854+0000 7fca0b5a4740 1 -- start start 2024-09-12T02:04:18.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.854+0000 7fca0b5a4740 1 Processor -- start 2024-09-12T02:04:18.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.854+0000 7fca0b5a4740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55621f4f35d0 0x55621f4fdc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:18.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.854+0000 7fca0b5a4740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55621f4f4b40 0x55621f4faf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:18.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.854+0000 7fca0b5a4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55621f4fe370 0x55621f4fb440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:18.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.854+0000 7fca0b5a4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55621f501070 con 0x55621f4fe370 2024-09-12T02:04:18.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.854+0000 7fca0b5a4740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55621f414060 con 0x55621f4f4b40 2024-09-12T02:04:18.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.854+0000 7fca0b5a4740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55621f5011f0 con 0x55621f4f35d0 2024-09-12T02:04:18.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.854+0000 7fca0b5a4740 10 client.0 _ll_register_callbacks cb 0x55621f1678c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:04:18.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.855+0000 7fca0359e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55621f4fe370 0x55621f4fb440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:18.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.855+0000 7fca0359e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55621f4fe370 0x55621f4fb440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46878/0 (socket says 172.21.15.16:46878) 2024-09-12T02:04:18.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.855+0000 7fca0359e640 1 -- 172.21.15.16:0/4059426820 learned_addr learned my addr 172.21.15.16:0/4059426820 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:04:18.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.855+0000 7fca02d9d640 1 --2- 172.21.15.16:0/4059426820 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55621f4f35d0 0x55621f4fdc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:18.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.855+0000 7fca0259c640 1 --2- 172.21.15.16:0/4059426820 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55621f4f4b40 0x55621f4faf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:18.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.855+0000 7fca0359e640 1 -- 172.21.15.16:0/4059426820 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55621f4f35d0 msgr2=0x55621f4fdc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:18.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.855+0000 7fca0359e640 1 --2- 172.21.15.16:0/4059426820 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55621f4f35d0 0x55621f4fdc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:18.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.855+0000 7fca0359e640 1 -- 172.21.15.16:0/4059426820 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55621f4f4b40 msgr2=0x55621f4faf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:18.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.855+0000 7fca0359e640 1 --2- 172.21.15.16:0/4059426820 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55621f4f4b40 0x55621f4faf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:18.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.855+0000 7fca0359e640 1 -- 172.21.15.16:0/4059426820 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55621f1ca3c0 con 0x55621f4fe370 2024-09-12T02:04:18.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.855+0000 7fca0259c640 1 --2- 172.21.15.16:0/4059426820 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55621f4f4b40 0x55621f4faf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:04:18.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.855+0000 7fca0359e640 1 --2- 172.21.15.16:0/4059426820 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55621f4fe370 0x55621f4fb440 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fc9fc02f8e0 tx=0x7fc9fc02a950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a65ddc2a1ecdd9fb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:04:18.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.855+0000 7fc9e6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:04:18.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.855+0000 7fc9e6ff5640 1 -- 172.21.15.16:0/4059426820 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9fc04c030 con 0x55621f4fe370 2024-09-12T02:04:18.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.855+0000 7fc9e6ff5640 1 -- 172.21.15.16:0/4059426820 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fc9fc004310 con 0x55621f4fe370 2024-09-12T02:04:18.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.855+0000 7fca0b5a4740 1 -- 172.21.15.16:0/4059426820 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc9f800cc60 con 0x55621f4fe370 2024-09-12T02:04:18.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.858+0000 7fc9e6ff5640 1 -- 172.21.15.16:0/4059426820 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc9fc034040 con 0x55621f4fe370 2024-09-12T02:04:18.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.858+0000 7fca0b5a4740 10 client.16125 fetch_fsmap learned FSMap version 25 2024-09-12T02:04:18.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.858+0000 7fca0b5a4740 1 -- 172.21.15.16:0/4059426820 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55621f50ca50 con 0x55621f4fe370 2024-09-12T02:04:18.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.858+0000 7fc9e6ff5640 1 -- 172.21.15.16:0/4059426820 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc9f800cc60 con 0x55621f4fe370 2024-09-12T02:04:18.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.858+0000 7fc9e6ff5640 1 -- 172.21.15.16:0/4059426820 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fc9fc047040 con 0x55621f4fe370 2024-09-12T02:04:18.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.858+0000 7fca0b5a4740 10 client.16125 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:04:18.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.858+0000 7fca0b5a4740 10 client.16125 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:04:18.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.858+0000 7fca0b5a4740 20 client.16125 populate_metadata read hostname 'smithi016' 2024-09-12T02:04:18.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.858+0000 7fca0b5a4740 1 --2- 172.21.15.16:0/4059426820 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55621f50dfb0 0x55621f510450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:18.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.858+0000 7fca0b5a4740 4 client.16125 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:04:18.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.858+0000 7fca0b5a4740 1 -- 172.21.15.16:0/4059426820 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55621f466860 con 0x55621f50dfb0 2024-09-12T02:04:18.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.858+0000 7fca02d9d640 1 --2- 172.21.15.16:0/4059426820 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55621f50dfb0 0x55621f510450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:18.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.859+0000 7fca02d9d640 1 --2- 172.21.15.16:0/4059426820 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55621f50dfb0 0x55621f510450 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52f90758eac0066f server_cookie=fe2a9b1ec5b58acf in_seq=0 out_seq=0 2024-09-12T02:04:18.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.859+0000 7fc9e6ff5640 10 client.16125 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:04:18.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.860+0000 7fc9e6ff5640 1 -- 172.21.15.16:0/4059426820 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55621f466860 con 0x55621f50dfb0 2024-09-12T02:04:18.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.860+0000 7fc9e6ff5640 10 client.16125 handle_command_reply: tid=0 2024-09-12T02:04:18.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.860+0000 7fca0b5a4740 1 client.16125 shutdown 2024-09-12T02:04:18.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.860+0000 7fca0b5a4740 1 -- 172.21.15.16:0/4059426820 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55621f4fe370 msgr2=0x55621f4fb440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:18.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.860+0000 7fca0b5a4740 1 --2- 172.21.15.16:0/4059426820 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55621f4fe370 0x55621f4fb440 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fc9fc02f8e0 tx=0x7fc9fc02a950 comp rx=0 tx=0).stop 2024-09-12T02:04:18.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.860+0000 7fca0b5a4740 1 -- 172.21.15.16:0/4059426820 shutdown_connections 2024-09-12T02:04:18.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.860+0000 7fca0b5a4740 1 --2- 172.21.15.16:0/4059426820 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55621f50dfb0 0x55621f510450 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:18.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.860+0000 7fca0b5a4740 1 --2- 172.21.15.16:0/4059426820 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55621f4fe370 0x55621f4fb440 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:18.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.860+0000 7fca0b5a4740 1 --2- 172.21.15.16:0/4059426820 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55621f4f4b40 0x55621f4faf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:18.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.860+0000 7fc9e6ff5640 0 client.16125 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:04:18.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.860+0000 7fca0b5a4740 1 --2- 172.21.15.16:0/4059426820 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55621f4f35d0 0x55621f4fdc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:18.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.860+0000 7fca0b5a4740 1 -- 172.21.15.16:0/4059426820 >> 172.21.15.16:0/4059426820 conn(0x55621f4bc310 msgr2=0x55621f4f5960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:04:18.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.860+0000 7fca0b5a4740 1 -- 172.21.15.16:0/4059426820 shutdown_connections 2024-09-12T02:04:18.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.860+0000 7fca0b5a4740 1 -- 172.21.15.16:0/4059426820 wait complete. 2024-09-12T02:04:18.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.860+0000 7fca0b5a4740 20 client.16125 trim_cache size 0 max 16384 2024-09-12T02:04:18.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.873+0000 7fca0b5a4740 1 Processor -- start 2024-09-12T02:04:18.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.873+0000 7fca0b5a4740 1 -- start start 2024-09-12T02:04:18.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.873+0000 7fca0b5a4740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55621f4f35d0 0x55621f50a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:18.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.873+0000 7fca0b5a4740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55621f4f4b40 0x55621f4f97f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:18.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.873+0000 7fca0b5a4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55621f4fe370 0x55621f4f9d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:18.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.873+0000 7fca0b5a4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fca041b1f60 con 0x55621f4fe370 2024-09-12T02:04:18.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.873+0000 7fca0b5a4740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fca041af830 con 0x55621f4f35d0 2024-09-12T02:04:18.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.873+0000 7fca0b5a4740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fca041ad680 con 0x55621f4f4b40 2024-09-12T02:04:18.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.873+0000 7fca0359e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55621f4fe370 0x55621f4f9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:18.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.873+0000 7fca0359e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55621f4fe370 0x55621f4f9d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46882/0 (socket says 172.21.15.16:46882) 2024-09-12T02:04:18.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.873+0000 7fca0359e640 1 -- 172.21.15.16:0/351272500 learned_addr learned my addr 172.21.15.16:0/351272500 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:04:18.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.874+0000 7fca02d9d640 1 --2- 172.21.15.16:0/351272500 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55621f4f35d0 0x55621f50a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:18.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.874+0000 7fca0259c640 1 --2- 172.21.15.16:0/351272500 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55621f4f4b40 0x55621f4f97f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:18.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.874+0000 7fca0359e640 1 -- 172.21.15.16:0/351272500 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55621f4f4b40 msgr2=0x55621f4f97f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:18.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.874+0000 7fca0359e640 1 --2- 172.21.15.16:0/351272500 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55621f4f4b40 0x55621f4f97f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:18.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.874+0000 7fca0359e640 1 -- 172.21.15.16:0/351272500 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55621f4f35d0 msgr2=0x55621f50a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:18.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.874+0000 7fca0359e640 1 --2- 172.21.15.16:0/351272500 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55621f4f35d0 0x55621f50a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:18.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.874+0000 7fca0359e640 1 -- 172.21.15.16:0/351272500 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55621f4f01a0 con 0x55621f4fe370 2024-09-12T02:04:18.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.874+0000 7fca0359e640 1 --2- 172.21.15.16:0/351272500 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55621f4fe370 0x55621f4f9d30 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fc9fc030f50 tx=0x7fc9fc005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c54a0dabec7517d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:04:18.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.874+0000 7fc9e8ff9640 1 -- 172.21.15.16:0/351272500 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9fc04c030 con 0x55621f4fe370 2024-09-12T02:04:18.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.874+0000 7fc9e8ff9640 1 -- 172.21.15.16:0/351272500 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc9fc003d40 con 0x55621f4fe370 2024-09-12T02:04:18.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.874+0000 7fc9e8ff9640 1 -- 172.21.15.16:0/351272500 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9fc039420 con 0x55621f4fe370 2024-09-12T02:04:18.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.875+0000 7fca0b5a4740 1 -- 172.21.15.16:0/351272500 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55621f4fe370 msgr2=0x55621f4f9d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:18.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.875+0000 7fca0b5a4740 1 --2- 172.21.15.16:0/351272500 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55621f4fe370 0x55621f4f9d30 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fc9fc030f50 tx=0x7fc9fc005fd0 comp rx=0 tx=0).stop 2024-09-12T02:04:18.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.875+0000 7fca0b5a4740 1 -- 172.21.15.16:0/351272500 shutdown_connections 2024-09-12T02:04:18.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.875+0000 7fca0b5a4740 1 --2- 172.21.15.16:0/351272500 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55621f4fe370 0x55621f4f9d30 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:18.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.875+0000 7fca0b5a4740 1 --2- 172.21.15.16:0/351272500 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55621f4f4b40 0x55621f4f97f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:18.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.875+0000 7fca0b5a4740 1 --2- 172.21.15.16:0/351272500 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55621f4f35d0 0x55621f50a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:18.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.875+0000 7fca0b5a4740 1 -- 172.21.15.16:0/351272500 >> 172.21.15.16:0/351272500 conn(0x55621f4bc310 msgr2=0x55621f4faec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:04:18.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.875+0000 7fca0b5a4740 1 -- 172.21.15.16:0/351272500 shutdown_connections 2024-09-12T02:04:18.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.875+0000 7fca0b5a4740 1 -- 172.21.15.16:0/351272500 wait complete. 2024-09-12T02:04:18.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.875+0000 7fca0b5a4740 1 -- start start 2024-09-12T02:04:18.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.875+0000 7fca0b5a4740 1 Processor -- start 2024-09-12T02:04:18.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.876+0000 7fca0b5a4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55621f4f35d0 0x55621f5069e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:18.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.876+0000 7fca0b5a4740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55621f4f4b40 0x55621f4f7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:18.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.876+0000 7fca0b5a4740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55621f4fe370 0x55621f4f7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:18.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.876+0000 7fca0b5a4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fca041adcf0 con 0x55621f4f35d0 2024-09-12T02:04:18.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.876+0000 7fca0b5a4740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fca041af160 con 0x55621f4f4b40 2024-09-12T02:04:18.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.876+0000 7fca0b5a4740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fca041ae3c0 con 0x55621f4fe370 2024-09-12T02:04:18.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.876+0000 7fca0b5a4740 10 client.0 _ll_register_callbacks cb 0x7fc9fc009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:04:18.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.876+0000 7fca02d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55621f4f35d0 0x55621f5069e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:18.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.876+0000 7fca02d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55621f4f35d0 0x55621f5069e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46892/0 (socket says 172.21.15.16:46892) 2024-09-12T02:04:18.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.876+0000 7fca02d9d640 1 -- 172.21.15.16:0/2066481207 learned_addr learned my addr 172.21.15.16:0/2066481207 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:04:18.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.876+0000 7fca0359e640 1 --2- 172.21.15.16:0/2066481207 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55621f4fe370 0x55621f4f7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:18.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.876+0000 7fca0259c640 1 --2- 172.21.15.16:0/2066481207 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55621f4f4b40 0x55621f4f7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:18.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.876+0000 7fca02d9d640 1 -- 172.21.15.16:0/2066481207 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55621f4fe370 msgr2=0x55621f4f7ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:18.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.876+0000 7fca02d9d640 1 --2- 172.21.15.16:0/2066481207 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55621f4fe370 0x55621f4f7ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:18.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.876+0000 7fca02d9d640 1 -- 172.21.15.16:0/2066481207 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55621f4f4b40 msgr2=0x55621f4f7ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:18.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.876+0000 7fca02d9d640 1 --2- 172.21.15.16:0/2066481207 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55621f4f4b40 0x55621f4f7ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:18.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.876+0000 7fca02d9d640 1 -- 172.21.15.16:0/2066481207 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55621f4ef320 con 0x55621f4f35d0 2024-09-12T02:04:18.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.876+0000 7fca02d9d640 1 --2- 172.21.15.16:0/2066481207 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55621f4f35d0 0x55621f5069e0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fc9f4007590 tx=0x7fc9f406b500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5449621feaa998c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:04:18.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.876+0000 7fc9e6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:04:18.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.877+0000 7fc9e6ff5640 1 -- 172.21.15.16:0/2066481207 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9f401d040 con 0x55621f4f35d0 2024-09-12T02:04:18.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.877+0000 7fc9e6ff5640 1 -- 172.21.15.16:0/2066481207 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fc9f4097cc0 con 0x55621f4f35d0 2024-09-12T02:04:18.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.877+0000 7fca0b5a4740 1 -- 172.21.15.16:0/2066481207 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc9fc003d40 con 0x55621f4f35d0 2024-09-12T02:04:18.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.877+0000 7fc9e6ff5640 1 -- 172.21.15.16:0/2066481207 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc9f4066730 con 0x55621f4f35d0 2024-09-12T02:04:18.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.877+0000 7fca0b5a4740 10 client.16131 fetch_fsmap learned FSMap version 25 2024-09-12T02:04:18.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.877+0000 7fca0b5a4740 1 -- 172.21.15.16:0/2066481207 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55621f578990 con 0x55621f4f35d0 2024-09-12T02:04:18.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.877+0000 7fc9e6ff5640 1 -- 172.21.15.16:0/2066481207 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc9fc003d40 con 0x55621f4f35d0 2024-09-12T02:04:18.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.877+0000 7fc9e6ff5640 1 -- 172.21.15.16:0/2066481207 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fc9f4007ca0 con 0x55621f4f35d0 2024-09-12T02:04:18.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.877+0000 7fca0b5a4740 10 client.16131 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:04:18.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.877+0000 7fca0b5a4740 10 client.16131 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:04:18.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.877+0000 7fca0b5a4740 20 client.16131 populate_metadata read hostname 'smithi016' 2024-09-12T02:04:18.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.877+0000 7fca0b5a4740 1 --2- 172.21.15.16:0/2066481207 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55621f579040 0x55621f57b460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:18.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.877+0000 7fca0b5a4740 4 client.16131 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T02:04:18.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.877+0000 7fca0b5a4740 1 -- 172.21.15.16:0/2066481207 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55621f460d70 con 0x55621f579040 2024-09-12T02:04:18.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.877+0000 7fca0259c640 1 --2- 172.21.15.16:0/2066481207 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55621f579040 0x55621f57b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:18.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.878+0000 7fca0259c640 1 --2- 172.21.15.16:0/2066481207 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55621f579040 0x55621f57b460 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f97e010898f77802 server_cookie=297e346e13501fa8 in_seq=0 out_seq=0 2024-09-12T02:04:18.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.878+0000 7fc9e6ff5640 10 client.16131 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:04:18.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.878+0000 7fc9e6ff5640 1 -- 172.21.15.16:0/2066481207 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55621f460d70 con 0x55621f579040 2024-09-12T02:04:18.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.878+0000 7fc9e6ff5640 10 client.16131 handle_command_reply: tid=0 2024-09-12T02:04:18.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.878+0000 7fca0b5a4740 1 client.16131 shutdown 2024-09-12T02:04:18.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.878+0000 7fca0b5a4740 1 -- 172.21.15.16:0/2066481207 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55621f4f35d0 msgr2=0x55621f5069e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:18.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.878+0000 7fca0b5a4740 1 --2- 172.21.15.16:0/2066481207 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55621f4f35d0 0x55621f5069e0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fc9f4007590 tx=0x7fc9f406b500 comp rx=0 tx=0).stop 2024-09-12T02:04:18.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.878+0000 7fca0b5a4740 1 -- 172.21.15.16:0/2066481207 shutdown_connections 2024-09-12T02:04:18.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.878+0000 7fca0b5a4740 1 --2- 172.21.15.16:0/2066481207 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55621f579040 0x55621f57b460 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:18.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.878+0000 7fca0b5a4740 1 --2- 172.21.15.16:0/2066481207 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55621f4fe370 0x55621f4f7ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:18.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.878+0000 7fca0b5a4740 1 --2- 172.21.15.16:0/2066481207 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55621f4f4b40 0x55621f4f7ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:18.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.878+0000 7fca0b5a4740 1 --2- 172.21.15.16:0/2066481207 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55621f4f35d0 0x55621f5069e0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:18.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.878+0000 7fca0b5a4740 1 -- 172.21.15.16:0/2066481207 >> 172.21.15.16:0/2066481207 conn(0x55621f4bc310 msgr2=0x55621f4faec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:04:18.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.878+0000 7fc9e6ff5640 0 client.16131 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:04:18.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.878+0000 7fca0b5a4740 1 -- 172.21.15.16:0/2066481207 shutdown_connections 2024-09-12T02:04:18.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.878+0000 7fca0b5a4740 1 -- 172.21.15.16:0/2066481207 wait complete. 2024-09-12T02:04:18.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.879+0000 7fca0b5a4740 20 client.16131 trim_cache size 0 max 16384 2024-09-12T02:04:18.898 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T02:04:18.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.879+0000 7fca091cb640 1 -- 172.21.15.16:0/2794657534 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc9ec0a2b80 msgr2=0x7fc9ec0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:18.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.879+0000 7fca091cb640 1 --2- 172.21.15.16:0/2794657534 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc9ec0a2b80 0x7fc9ec0a5040 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fc9fc009a60 tx=0x7fc9fc0095c0 comp rx=0 tx=0).stop 2024-09-12T02:04:18.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.879+0000 7fca091cb640 1 -- 172.21.15.16:0/2794657534 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca04102d40 msgr2=0x7fca041009a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:18.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.879+0000 7fca091cb640 1 --2- 172.21.15.16:0/2794657534 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca04102d40 0x7fca041009a0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fc9f400c890 tx=0x7fc9f400cd50 comp rx=0 tx=0).stop 2024-09-12T02:04:18.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.879+0000 7fca091cb640 1 -- 172.21.15.16:0/2794657534 shutdown_connections 2024-09-12T02:04:18.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.879+0000 7fca091cb640 1 --2- 172.21.15.16:0/2794657534 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc9ec0a2b80 0x7fc9ec0a5040 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:18.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.879+0000 7fca091cb640 1 --2- 172.21.15.16:0/2794657534 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fca0410c170 0x7fca04101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:18.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.879+0000 7fca091cb640 1 --2- 172.21.15.16:0/2794657534 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca04103660 0x7fca04100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:18.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.879+0000 7fca091cb640 1 --2- 172.21.15.16:0/2794657534 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca04102d40 0x7fca041009a0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:18.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.879+0000 7fca091cb640 1 -- 172.21.15.16:0/2794657534 >> 172.21.15.16:0/2794657534 conn(0x7fca040fc7a0 msgr2=0x7fca040fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:04:18.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.879+0000 7fca091cb640 1 -- 172.21.15.16:0/2794657534 shutdown_connections 2024-09-12T02:04:18.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:18.879+0000 7fca091cb640 1 -- 172.21.15.16:0/2794657534 wait complete. 2024-09-12T02:04:20.035 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:04:19 smithi016 ceph-mon[30947]: pgmap v407: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-09-12T02:04:20.072 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '146e8aed', quiesce took 5.3 seconds. Will release after: 19.4 2024-09-12T02:04:20.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:04:19 smithi184 ceph-mon[35593]: pgmap v407: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-09-12T02:04:20.168 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:04:19 smithi033 ceph-mon[35332]: pgmap v407: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-09-12T02:04:20.755 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-12T02:04:20.755 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-12T02:04:21.028 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:04:21.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.498+0000 7fa82c7a6640 1 -- 172.21.15.16:0/2575145290 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa824102d40 msgr2=0x7fa824103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:21.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.498+0000 7fa82c7a6640 1 --2- 172.21.15.16:0/2575145290 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa824102d40 0x7fa824103120 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fa820009900 tx=0x7fa82002a480 comp rx=0 tx=0).stop 2024-09-12T02:04:21.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.498+0000 7fa82c7a6640 1 -- 172.21.15.16:0/2575145290 shutdown_connections 2024-09-12T02:04:21.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.498+0000 7fa82c7a6640 1 --2- 172.21.15.16:0/2575145290 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa82410c170 0x7fa82410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:21.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.498+0000 7fa82c7a6640 1 --2- 172.21.15.16:0/2575145290 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa824103660 0x7fa82410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:21.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.498+0000 7fa82c7a6640 1 --2- 172.21.15.16:0/2575145290 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa824102d40 0x7fa824103120 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:21.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.498+0000 7fa82c7a6640 1 -- 172.21.15.16:0/2575145290 >> 172.21.15.16:0/2575145290 conn(0x7fa8240fc7a0 msgr2=0x7fa8240febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:04:21.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.499+0000 7fa82c7a6640 1 -- 172.21.15.16:0/2575145290 shutdown_connections 2024-09-12T02:04:21.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.499+0000 7fa82c7a6640 1 -- 172.21.15.16:0/2575145290 wait complete. 2024-09-12T02:04:21.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.499+0000 7fa82c7a6640 1 Processor -- start 2024-09-12T02:04:21.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.499+0000 7fa82c7a6640 1 -- start start 2024-09-12T02:04:21.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.499+0000 7fa82c7a6640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa824102d40 0x7fa82419a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:21.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.499+0000 7fa82c7a6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa824103660 0x7fa82419ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:21.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.499+0000 7fa82c7a6640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa82410c170 0x7fa82419ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:21.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.499+0000 7fa82c7a6640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa824111f70 con 0x7fa824103660 2024-09-12T02:04:21.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.500+0000 7fa82c7a6640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa824111df0 con 0x7fa824102d40 2024-09-12T02:04:21.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.500+0000 7fa82c7a6640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa8241120f0 con 0x7fa82410c170 2024-09-12T02:04:21.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.500+0000 7fa829d41640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa824103660 0x7fa82419ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:21.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.500+0000 7fa829d41640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa824103660 0x7fa82419ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42678/0 (socket says 172.21.15.16:42678) 2024-09-12T02:04:21.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.500+0000 7fa829d41640 1 -- 172.21.15.16:0/606999809 learned_addr learned my addr 172.21.15.16:0/606999809 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:04:21.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.500+0000 7fa82a542640 1 --2- 172.21.15.16:0/606999809 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa824102d40 0x7fa82419a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:21.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.500+0000 7fa82ad43640 1 --2- 172.21.15.16:0/606999809 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa82410c170 0x7fa82419ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:21.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.500+0000 7fa829d41640 1 -- 172.21.15.16:0/606999809 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa82410c170 msgr2=0x7fa82419ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:21.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.500+0000 7fa829d41640 1 --2- 172.21.15.16:0/606999809 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa82410c170 0x7fa82419ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:21.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.500+0000 7fa829d41640 1 -- 172.21.15.16:0/606999809 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa824102d40 msgr2=0x7fa82419a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:21.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.500+0000 7fa829d41640 1 --2- 172.21.15.16:0/606999809 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa824102d40 0x7fa82419a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:21.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.500+0000 7fa829d41640 1 -- 172.21.15.16:0/606999809 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa82419f5e0 con 0x7fa824103660 2024-09-12T02:04:21.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.500+0000 7fa82ad43640 1 --2- 172.21.15.16:0/606999809 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa82410c170 0x7fa82419ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:04:21.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.500+0000 7fa829d41640 1 --2- 172.21.15.16:0/606999809 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa824103660 0x7fa82419ab70 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fa81400c970 tx=0x7fa81400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:04:21.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.500+0000 7fa81b7fe640 1 -- 172.21.15.16:0/606999809 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa814007bb0 con 0x7fa824103660 2024-09-12T02:04:21.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.500+0000 7fa82c7a6640 1 -- 172.21.15.16:0/606999809 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa82419f8d0 con 0x7fa824103660 2024-09-12T02:04:21.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.500+0000 7fa81b7fe640 1 -- 172.21.15.16:0/606999809 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa814007d50 con 0x7fa824103660 2024-09-12T02:04:21.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.500+0000 7fa81b7fe640 1 -- 172.21.15.16:0/606999809 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa814018820 con 0x7fa824103660 2024-09-12T02:04:21.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.500+0000 7fa82c7a6640 1 -- 172.21.15.16:0/606999809 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8241a71b0 con 0x7fa824103660 2024-09-12T02:04:21.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.503+0000 7fa81b7fe640 1 -- 172.21.15.16:0/606999809 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa814018a80 con 0x7fa824103660 2024-09-12T02:04:21.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.503+0000 7fa82eb7f740 1 Processor -- start 2024-09-12T02:04:21.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.503+0000 7fa82eb7f740 1 -- start start 2024-09-12T02:04:21.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.503+0000 7fa82eb7f740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f2966c9b60 0x55f2966c9f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:21.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.503+0000 7fa82eb7f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f2966ca480 0x55f2966d4800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:21.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.503+0000 7fa82eb7f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f2966d4d40 0x55f2966d7130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:21.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.504+0000 7fa82eb7f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55f29662f280 con 0x55f2966d4d40 2024-09-12T02:04:21.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.504+0000 7fa82eb7f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55f2965d6b10 con 0x55f2966ca480 2024-09-12T02:04:21.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.504+0000 7fa82eb7f740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55f29662b240 con 0x55f2966c9b60 2024-09-12T02:04:21.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.504+0000 7fa81b7fe640 1 --2- 172.21.15.16:0/606999809 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fa8100a2c00 0x7fa8100a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:21.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.504+0000 7fa829d41640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f2966d4d40 0x55f2966d7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:21.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.504+0000 7fa82a542640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f2966ca480 0x55f2966d4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:21.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.504+0000 7fa82a542640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f2966ca480 0x55f2966d4800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.16:49260/0 (socket says 172.21.15.16:49260) 2024-09-12T02:04:21.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.504+0000 7fa82a542640 1 -- 172.21.15.16:0/3332714348 learned_addr learned my addr 172.21.15.16:0/3332714348 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:04:21.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.504+0000 7fa82ad43640 1 --2- 172.21.15.16:0/606999809 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fa8100a2c00 0x7fa8100a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:21.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.504+0000 7fa82ad43640 1 --2- 172.21.15.16:0/3332714348 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f2966c9b60 0x55f2966c9f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:21.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.504+0000 7fa81b7fe640 1 -- 172.21.15.16:0/606999809 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fa8140cefc0 con 0x7fa824103660 2024-09-12T02:04:21.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.504+0000 7fa82a542640 1 -- 172.21.15.16:0/3332714348 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f2966c9b60 msgr2=0x55f2966c9f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:21.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.504+0000 7fa82a542640 1 --2- 172.21.15.16:0/3332714348 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f2966c9b60 0x55f2966c9f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:21.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.504+0000 7fa82a542640 1 -- 172.21.15.16:0/3332714348 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f2966d4d40 msgr2=0x55f2966d7130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:21.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.504+0000 7fa82a542640 1 --2- 172.21.15.16:0/3332714348 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f2966d4d40 0x55f2966d7130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:21.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.504+0000 7fa82a542640 1 -- 172.21.15.16:0/3332714348 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f2965e31f0 con 0x55f2966ca480 2024-09-12T02:04:21.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.504+0000 7fa829d41640 1 --2- 172.21.15.16:0/3332714348 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f2966d4d40 0x55f2966d7130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:04:21.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.504+0000 7fa82ad43640 1 --2- 172.21.15.16:0/606999809 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fa8100a2c00 0x7fa8100a50c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fa81c005c40 tx=0x7fa81c009d50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:04:21.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.505+0000 7fa82a542640 1 --2- 172.21.15.16:0/3332714348 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f2966ca480 0x55f2966d4800 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa820009900 tx=0x7fa820031630 comp rx=0 tx=0).ready entity=mon.2 client_cookie=72b3fa42c1185e45 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:04:21.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.505+0000 7fa818ff9640 1 -- 172.21.15.16:0/3332714348 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa820040070 con 0x55f2966ca480 2024-09-12T02:04:21.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.505+0000 7fa818ff9640 1 -- 172.21.15.16:0/3332714348 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa820005020 con 0x55f2966ca480 2024-09-12T02:04:21.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.505+0000 7fa818ff9640 1 -- 172.21.15.16:0/3332714348 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa820035040 con 0x55f2966ca480 2024-09-12T02:04:21.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.505+0000 7fa82eb7f740 1 -- 172.21.15.16:0/3332714348 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f2966ca480 msgr2=0x55f2966d4800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:21.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.505+0000 7fa82eb7f740 1 --2- 172.21.15.16:0/3332714348 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f2966ca480 0x55f2966d4800 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa820009900 tx=0x7fa820031630 comp rx=0 tx=0).stop 2024-09-12T02:04:21.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.506+0000 7fa82eb7f740 1 -- 172.21.15.16:0/3332714348 shutdown_connections 2024-09-12T02:04:21.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.506+0000 7fa82eb7f740 1 --2- 172.21.15.16:0/3332714348 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f2966d4d40 0x55f2966d7130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:21.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.506+0000 7fa82eb7f740 1 --2- 172.21.15.16:0/3332714348 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f2966ca480 0x55f2966d4800 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:21.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.506+0000 7fa82eb7f740 1 --2- 172.21.15.16:0/3332714348 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f2966c9b60 0x55f2966c9f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:21.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.506+0000 7fa82eb7f740 1 -- 172.21.15.16:0/3332714348 >> 172.21.15.16:0/3332714348 conn(0x55f2966c60f0 msgr2=0x55f2966c64d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:04:21.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.506+0000 7fa82eb7f740 1 -- 172.21.15.16:0/3332714348 shutdown_connections 2024-09-12T02:04:21.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.506+0000 7fa82eb7f740 1 -- 172.21.15.16:0/3332714348 wait complete. 2024-09-12T02:04:21.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.506+0000 7fa82eb7f740 1 -- start start 2024-09-12T02:04:21.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.506+0000 7fa82eb7f740 1 Processor -- start 2024-09-12T02:04:21.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.506+0000 7fa82eb7f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f2966c9b60 0x55f2966d1b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:21.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.506+0000 7fa82eb7f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f2966ca480 0x55f2966d20b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:21.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.506+0000 7fa82eb7f740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f2966d4d40 0x55f2966d25f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:21.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.506+0000 7fa82eb7f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55f2966d7910 con 0x55f2966c9b60 2024-09-12T02:04:21.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.506+0000 7fa82eb7f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55f2965e9f10 con 0x55f2966ca480 2024-09-12T02:04:21.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.507+0000 7fa82eb7f740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55f2966d7a90 con 0x55f2966d4d40 2024-09-12T02:04:21.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.507+0000 7fa82eb7f740 10 client.0 _ll_register_callbacks cb 0x55f2965b2a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:04:21.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.507+0000 7fa82a542640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f2966c9b60 0x55f2966d1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:21.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.507+0000 7fa82a542640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f2966c9b60 0x55f2966d1b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42702/0 (socket says 172.21.15.16:42702) 2024-09-12T02:04:21.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.507+0000 7fa82a542640 1 -- 172.21.15.16:0/2300503310 learned_addr learned my addr 172.21.15.16:0/2300503310 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:04:21.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.507+0000 7fa829d41640 1 --2- 172.21.15.16:0/2300503310 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f2966ca480 0x55f2966d20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:21.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.507+0000 7fa82ad43640 1 --2- 172.21.15.16:0/2300503310 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f2966d4d40 0x55f2966d25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:21.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.507+0000 7fa82a542640 1 -- 172.21.15.16:0/2300503310 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f2966d4d40 msgr2=0x55f2966d25f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:21.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.507+0000 7fa82a542640 1 --2- 172.21.15.16:0/2300503310 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f2966d4d40 0x55f2966d25f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:21.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.507+0000 7fa82a542640 1 -- 172.21.15.16:0/2300503310 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f2966ca480 msgr2=0x55f2966d20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:21.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.507+0000 7fa82a542640 1 --2- 172.21.15.16:0/2300503310 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f2966ca480 0x55f2966d20b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:21.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.507+0000 7fa82a542640 1 -- 172.21.15.16:0/2300503310 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f2963a06a0 con 0x55f2966c9b60 2024-09-12T02:04:21.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.507+0000 7fa82ad43640 1 --2- 172.21.15.16:0/2300503310 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f2966d4d40 0x55f2966d25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:04:21.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.507+0000 7fa82a542640 1 --2- 172.21.15.16:0/2300503310 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f2966c9b60 0x55f2966d1b70 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fa820009a20 tx=0x7fa820005460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=307938878af862b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:04:21.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.507+0000 7fa80dffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:04:21.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.507+0000 7fa80dffb640 1 -- 172.21.15.16:0/2300503310 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa82004c020 con 0x55f2966c9b60 2024-09-12T02:04:21.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.507+0000 7fa80dffb640 1 -- 172.21.15.16:0/2300503310 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fa820004960 con 0x55f2966c9b60 2024-09-12T02:04:21.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.507+0000 7fa82eb7f740 1 -- 172.21.15.16:0/2300503310 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa820005020 con 0x55f2966c9b60 2024-09-12T02:04:21.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.508+0000 7fa80dffb640 1 -- 172.21.15.16:0/2300503310 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa820040070 con 0x55f2966c9b60 2024-09-12T02:04:21.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.508+0000 7fa82eb7f740 10 client.16143 fetch_fsmap learned FSMap version 25 2024-09-12T02:04:21.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.508+0000 7fa82eb7f740 1 -- 172.21.15.16:0/2300503310 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f2966e3460 con 0x55f2966c9b60 2024-09-12T02:04:21.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.508+0000 7fa80dffb640 1 -- 172.21.15.16:0/2300503310 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa820005020 con 0x55f2966c9b60 2024-09-12T02:04:21.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.508+0000 7fa80dffb640 1 -- 172.21.15.16:0/2300503310 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fa820044400 con 0x55f2966c9b60 2024-09-12T02:04:21.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.508+0000 7fa82eb7f740 10 client.16143 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:04:21.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.508+0000 7fa82eb7f740 10 client.16143 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:04:21.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.508+0000 7fa82eb7f740 20 client.16143 populate_metadata read hostname 'smithi016' 2024-09-12T02:04:21.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.508+0000 7fa82eb7f740 1 --2- 172.21.15.16:0/2300503310 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55f2966e47b0 0x55f2966e6ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:21.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.508+0000 7fa82eb7f740 4 client.16143 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:04:21.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.508+0000 7fa82eb7f740 1 -- 172.21.15.16:0/2300503310 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f29663c7f0 con 0x55f2966e47b0 2024-09-12T02:04:21.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.508+0000 7fa829d41640 1 --2- 172.21.15.16:0/2300503310 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55f2966e47b0 0x55f2966e6ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:21.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.509+0000 7fa80dffb640 10 client.16143 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:04:21.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.509+0000 7fa829d41640 1 --2- 172.21.15.16:0/2300503310 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55f2966e47b0 0x55f2966e6ba0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee7ba035b2bb76b9 server_cookie=f1e837d9c1b87d4a in_seq=0 out_seq=0 2024-09-12T02:04:21.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.509+0000 7fa80dffb640 1 -- 172.21.15.16:0/2300503310 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f29663c7f0 con 0x55f2966e47b0 2024-09-12T02:04:21.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.509+0000 7fa80dffb640 10 client.16143 handle_command_reply: tid=0 2024-09-12T02:04:21.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.509+0000 7fa82eb7f740 1 client.16143 shutdown 2024-09-12T02:04:21.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.510+0000 7fa82eb7f740 1 -- 172.21.15.16:0/2300503310 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f2966c9b60 msgr2=0x55f2966d1b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:21.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.510+0000 7fa82eb7f740 1 --2- 172.21.15.16:0/2300503310 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f2966c9b60 0x55f2966d1b70 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fa820009a20 tx=0x7fa820005460 comp rx=0 tx=0).stop 2024-09-12T02:04:21.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.510+0000 7fa82eb7f740 1 -- 172.21.15.16:0/2300503310 shutdown_connections 2024-09-12T02:04:21.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.510+0000 7fa82eb7f740 1 --2- 172.21.15.16:0/2300503310 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55f2966e47b0 0x55f2966e6ba0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:21.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.510+0000 7fa82eb7f740 1 --2- 172.21.15.16:0/2300503310 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f2966d4d40 0x55f2966d25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:21.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.510+0000 7fa82eb7f740 1 --2- 172.21.15.16:0/2300503310 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f2966ca480 0x55f2966d20b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:21.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.510+0000 7fa80dffb640 0 client.16143 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:04:21.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.510+0000 7fa82eb7f740 1 --2- 172.21.15.16:0/2300503310 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f2966c9b60 0x55f2966d1b70 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:21.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.510+0000 7fa82eb7f740 1 -- 172.21.15.16:0/2300503310 >> 172.21.15.16:0/2300503310 conn(0x55f2966c60f0 msgr2=0x55f2966cbdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:04:21.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.510+0000 7fa82eb7f740 1 -- 172.21.15.16:0/2300503310 shutdown_connections 2024-09-12T02:04:21.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.510+0000 7fa82eb7f740 1 -- 172.21.15.16:0/2300503310 wait complete. 2024-09-12T02:04:21.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.510+0000 7fa82eb7f740 20 client.16143 trim_cache size 0 max 16384 2024-09-12T02:04:21.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.527+0000 7fa82eb7f740 1 Processor -- start 2024-09-12T02:04:21.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.527+0000 7fa82eb7f740 1 -- start start 2024-09-12T02:04:21.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.527+0000 7fa82eb7f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f2966c9b60 0x55f2966d0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:21.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.527+0000 7fa82eb7f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f2966ca480 0x55f2966cc5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:21.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.527+0000 7fa82eb7f740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f2966d4d40 0x55f2966ec140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:21.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.527+0000 7fa82eb7f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa8241b1de0 con 0x55f2966c9b60 2024-09-12T02:04:21.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.527+0000 7fa82eb7f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa8241af6b0 con 0x55f2966ca480 2024-09-12T02:04:21.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.527+0000 7fa82eb7f740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa8241ad550 con 0x55f2966d4d40 2024-09-12T02:04:21.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.528+0000 7fa82a542640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f2966c9b60 0x55f2966d0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:21.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.528+0000 7fa82a542640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f2966c9b60 0x55f2966d0ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42714/0 (socket says 172.21.15.16:42714) 2024-09-12T02:04:21.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.528+0000 7fa82a542640 1 -- 172.21.15.16:0/2544372767 learned_addr learned my addr 172.21.15.16:0/2544372767 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:04:21.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.528+0000 7fa82ad43640 1 --2- 172.21.15.16:0/2544372767 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f2966d4d40 0x55f2966ec140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:21.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.528+0000 7fa82a542640 1 -- 172.21.15.16:0/2544372767 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f2966d4d40 msgr2=0x55f2966ec140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:21.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.528+0000 7fa82a542640 1 --2- 172.21.15.16:0/2544372767 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f2966d4d40 0x55f2966ec140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:21.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.528+0000 7fa829d41640 1 --2- 172.21.15.16:0/2544372767 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f2966ca480 0x55f2966cc5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:21.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.528+0000 7fa82a542640 1 -- 172.21.15.16:0/2544372767 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f2966ca480 msgr2=0x55f2966cc5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:21.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.528+0000 7fa82a542640 1 --2- 172.21.15.16:0/2544372767 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f2966ca480 0x55f2966cc5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:21.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.528+0000 7fa82a542640 1 -- 172.21.15.16:0/2544372767 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f2966ca2b0 con 0x55f2966c9b60 2024-09-12T02:04:21.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.528+0000 7fa82ad43640 1 --2- 172.21.15.16:0/2544372767 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f2966d4d40 0x55f2966ec140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:04:21.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.528+0000 7fa82a542640 1 --2- 172.21.15.16:0/2544372767 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f2966c9b60 0x55f2966d0ad0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fa820000c00 tx=0x7fa820005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4a88483d4895531 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:04:21.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.528+0000 7fa80ffff640 1 -- 172.21.15.16:0/2544372767 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa82004c070 con 0x55f2966c9b60 2024-09-12T02:04:21.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.528+0000 7fa80ffff640 1 -- 172.21.15.16:0/2544372767 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa820044a90 con 0x55f2966c9b60 2024-09-12T02:04:21.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.528+0000 7fa80ffff640 1 -- 172.21.15.16:0/2544372767 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa820044e60 con 0x55f2966c9b60 2024-09-12T02:04:21.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.529+0000 7fa82eb7f740 1 -- 172.21.15.16:0/2544372767 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f2966c9b60 msgr2=0x55f2966d0ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:21.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.529+0000 7fa82eb7f740 1 --2- 172.21.15.16:0/2544372767 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f2966c9b60 0x55f2966d0ad0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fa820000c00 tx=0x7fa820005fd0 comp rx=0 tx=0).stop 2024-09-12T02:04:21.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.529+0000 7fa82eb7f740 1 -- 172.21.15.16:0/2544372767 shutdown_connections 2024-09-12T02:04:21.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.529+0000 7fa82eb7f740 1 --2- 172.21.15.16:0/2544372767 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f2966d4d40 0x55f2966ec140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:21.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.529+0000 7fa82eb7f740 1 --2- 172.21.15.16:0/2544372767 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f2966ca480 0x55f2966cc5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:21.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.529+0000 7fa82eb7f740 1 --2- 172.21.15.16:0/2544372767 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f2966c9b60 0x55f2966d0ad0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:21.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.529+0000 7fa82eb7f740 1 -- 172.21.15.16:0/2544372767 >> 172.21.15.16:0/2544372767 conn(0x55f2966c60f0 msgr2=0x55f2966d3ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:04:21.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.529+0000 7fa82eb7f740 1 -- 172.21.15.16:0/2544372767 shutdown_connections 2024-09-12T02:04:21.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.529+0000 7fa82eb7f740 1 -- 172.21.15.16:0/2544372767 wait complete. 2024-09-12T02:04:21.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.529+0000 7fa82eb7f740 1 -- start start 2024-09-12T02:04:21.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.529+0000 7fa82eb7f740 1 Processor -- start 2024-09-12T02:04:21.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.530+0000 7fa82eb7f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f2966c9b60 0x55f2966dcce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:21.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.530+0000 7fa82eb7f740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f2966ca480 0x55f2966dd220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:21.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.530+0000 7fa82eb7f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f2966d4d40 0x55f2966d9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:21.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.530+0000 7fa82eb7f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa8241adbc0 con 0x55f2966c9b60 2024-09-12T02:04:21.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.530+0000 7fa82eb7f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa8241aefe0 con 0x55f2966d4d40 2024-09-12T02:04:21.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.530+0000 7fa82eb7f740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa8241ae240 con 0x55f2966ca480 2024-09-12T02:04:21.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.530+0000 7fa82eb7f740 10 client.0 _ll_register_callbacks cb 0x7fa81400fdc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:04:21.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.530+0000 7fa82a542640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f2966c9b60 0x55f2966dcce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:21.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.530+0000 7fa82a542640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f2966c9b60 0x55f2966dcce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42722/0 (socket says 172.21.15.16:42722) 2024-09-12T02:04:21.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.530+0000 7fa82a542640 1 -- 172.21.15.16:0/3945464308 learned_addr learned my addr 172.21.15.16:0/3945464308 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:04:21.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.530+0000 7fa829d41640 1 --2- 172.21.15.16:0/3945464308 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f2966ca480 0x55f2966dd220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:21.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.530+0000 7fa82ad43640 1 --2- 172.21.15.16:0/3945464308 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f2966d4d40 0x55f2966d9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:21.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.530+0000 7fa82a542640 1 -- 172.21.15.16:0/3945464308 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f2966ca480 msgr2=0x55f2966dd220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:21.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.530+0000 7fa82a542640 1 --2- 172.21.15.16:0/3945464308 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f2966ca480 0x55f2966dd220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:21.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.530+0000 7fa82a542640 1 -- 172.21.15.16:0/3945464308 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f2966d4d40 msgr2=0x55f2966d9cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:21.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.530+0000 7fa82a542640 1 --2- 172.21.15.16:0/3945464308 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f2966d4d40 0x55f2966d9cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:21.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.530+0000 7fa82a542640 1 -- 172.21.15.16:0/3945464308 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f2966da1f0 con 0x55f2966c9b60 2024-09-12T02:04:21.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.530+0000 7fa82a542640 1 --2- 172.21.15.16:0/3945464308 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f2966c9b60 0x55f2966dcce0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fa820044a10 tx=0x7fa820005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dcb50bdc7180824 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:04:21.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.530+0000 7fa80dffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:04:21.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.530+0000 7fa80dffb640 1 -- 172.21.15.16:0/3945464308 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa82004c040 con 0x55f2966c9b60 2024-09-12T02:04:21.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.531+0000 7fa80dffb640 1 -- 172.21.15.16:0/3945464308 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fa820004b60 con 0x55f2966c9b60 2024-09-12T02:04:21.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.531+0000 7fa82eb7f740 1 -- 172.21.15.16:0/3945464308 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa820044a90 con 0x55f2966c9b60 2024-09-12T02:04:21.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.531+0000 7fa80dffb640 1 -- 172.21.15.16:0/3945464308 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa820035040 con 0x55f2966c9b60 2024-09-12T02:04:21.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.531+0000 7fa82eb7f740 10 client.16149 fetch_fsmap learned FSMap version 25 2024-09-12T02:04:21.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.531+0000 7fa82eb7f740 1 -- 172.21.15.16:0/3945464308 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f29674f6d0 con 0x55f2966c9b60 2024-09-12T02:04:21.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.531+0000 7fa80dffb640 1 -- 172.21.15.16:0/3945464308 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa820044a90 con 0x55f2966c9b60 2024-09-12T02:04:21.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.531+0000 7fa80dffb640 1 -- 172.21.15.16:0/3945464308 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fa82005a020 con 0x55f2966c9b60 2024-09-12T02:04:21.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.531+0000 7fa82eb7f740 10 client.16149 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:04:21.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.531+0000 7fa82eb7f740 10 client.16149 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:04:21.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.531+0000 7fa82eb7f740 20 client.16149 populate_metadata read hostname 'smithi016' 2024-09-12T02:04:21.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.531+0000 7fa82eb7f740 1 --2- 172.21.15.16:0/3945464308 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55f29674fbf0 0x55f296751fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:21.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.531+0000 7fa82eb7f740 4 client.16149 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-12T02:04:21.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.531+0000 7fa82eb7f740 1 -- 172.21.15.16:0/3945464308 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55f296636d00 con 0x55f29674fbf0 2024-09-12T02:04:21.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.531+0000 7fa829d41640 1 --2- 172.21.15.16:0/3945464308 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55f29674fbf0 0x55f296751fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:21.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.532+0000 7fa829d41640 1 --2- 172.21.15.16:0/3945464308 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55f29674fbf0 0x55f296751fe0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40a13c8db099324b server_cookie=8aef51021fa3caa2 in_seq=0 out_seq=0 2024-09-12T02:04:21.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.532+0000 7fa80dffb640 10 client.16149 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:04:21.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.533+0000 7fa80dffb640 1 -- 172.21.15.16:0/3945464308 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55f296636d00 con 0x55f29674fbf0 2024-09-12T02:04:21.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.533+0000 7fa80dffb640 10 client.16149 handle_command_reply: tid=0 2024-09-12T02:04:21.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.533+0000 7fa82eb7f740 1 client.16149 shutdown 2024-09-12T02:04:21.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.533+0000 7fa82eb7f740 1 -- 172.21.15.16:0/3945464308 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f2966c9b60 msgr2=0x55f2966dcce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:21.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.533+0000 7fa82eb7f740 1 --2- 172.21.15.16:0/3945464308 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f2966c9b60 0x55f2966dcce0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fa820044a10 tx=0x7fa820005a20 comp rx=0 tx=0).stop 2024-09-12T02:04:21.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.533+0000 7fa82eb7f740 1 -- 172.21.15.16:0/3945464308 shutdown_connections 2024-09-12T02:04:21.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.533+0000 7fa82eb7f740 1 --2- 172.21.15.16:0/3945464308 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55f29674fbf0 0x55f296751fe0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:21.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.533+0000 7fa82eb7f740 1 --2- 172.21.15.16:0/3945464308 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f2966d4d40 0x55f2966d9cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:21.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.533+0000 7fa82eb7f740 1 --2- 172.21.15.16:0/3945464308 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f2966ca480 0x55f2966dd220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:21.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.533+0000 7fa82eb7f740 1 --2- 172.21.15.16:0/3945464308 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f2966c9b60 0x55f2966dcce0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:21.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.533+0000 7fa82eb7f740 1 -- 172.21.15.16:0/3945464308 >> 172.21.15.16:0/3945464308 conn(0x55f2966c60f0 msgr2=0x55f2966d3100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:04:21.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.533+0000 7fa80dffb640 0 client.16149 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:04:21.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.533+0000 7fa82eb7f740 1 -- 172.21.15.16:0/3945464308 shutdown_connections 2024-09-12T02:04:21.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.533+0000 7fa82eb7f740 1 -- 172.21.15.16:0/3945464308 wait complete. 2024-09-12T02:04:21.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.533+0000 7fa82eb7f740 20 client.16149 trim_cache size 0 max 16384 2024-09-12T02:04:21.541 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T02:04:21.541 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-09-12T02:04:21.541 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "d72e5b90-5676-4cfc-9bf2-eaf44e51a533", 2024-09-12T02:04:21.541 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-09-12T02:04:21.541 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T02:04:21.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.534+0000 7fa82c7a6640 1 -- 172.21.15.16:0/606999809 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fa8100a2c00 msgr2=0x7fa8100a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:21.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.534+0000 7fa82c7a6640 1 --2- 172.21.15.16:0/606999809 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fa8100a2c00 0x7fa8100a50c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fa81c005c40 tx=0x7fa81c009d50 comp rx=0 tx=0).stop 2024-09-12T02:04:21.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.534+0000 7fa82c7a6640 1 -- 172.21.15.16:0/606999809 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa824103660 msgr2=0x7fa82419ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:21.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.534+0000 7fa82c7a6640 1 --2- 172.21.15.16:0/606999809 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa824103660 0x7fa82419ab70 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fa81400c970 tx=0x7fa81400ce30 comp rx=0 tx=0).stop 2024-09-12T02:04:21.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.534+0000 7fa82c7a6640 1 -- 172.21.15.16:0/606999809 shutdown_connections 2024-09-12T02:04:21.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.534+0000 7fa82c7a6640 1 --2- 172.21.15.16:0/606999809 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fa8100a2c00 0x7fa8100a50c0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:21.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.534+0000 7fa82c7a6640 1 --2- 172.21.15.16:0/606999809 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa82410c170 0x7fa82419ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:21.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.534+0000 7fa82c7a6640 1 --2- 172.21.15.16:0/606999809 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa824103660 0x7fa82419ab70 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:21.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.534+0000 7fa82c7a6640 1 --2- 172.21.15.16:0/606999809 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa824102d40 0x7fa82419a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:21.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.534+0000 7fa82c7a6640 1 -- 172.21.15.16:0/606999809 >> 172.21.15.16:0/606999809 conn(0x7fa8240fc7a0 msgr2=0x7fa8240feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:04:21.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.534+0000 7fa82c7a6640 1 -- 172.21.15.16:0/606999809 shutdown_connections 2024-09-12T02:04:21.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:21.534+0000 7fa82c7a6640 1 -- 172.21.15.16:0/606999809 wait complete. 2024-09-12T02:04:22.110 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T02:04:22.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:04:21 smithi184 ceph-mon[35593]: pgmap v408: 97 pgs: 97 active+clean; 1.5 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-09-12T02:04:22.135 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:04:21 smithi016 ceph-mon[30947]: pgmap v408: 97 pgs: 97 active+clean; 1.5 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-09-12T02:04:22.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:04:21 smithi033 ceph-mon[35332]: pgmap v408: 97 pgs: 97 active+clean; 1.5 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-09-12T02:04:22.241 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.241+0000 7f7979c65640 1 -- 172.21.15.16:0/3452687034 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7974104ef0 msgr2=0x7f79741072e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:22.242 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.241+0000 7f7979c65640 1 --2- 172.21.15.16:0/3452687034 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7974104ef0 0x7f79741072e0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f7968009900 tx=0x7f796802a410 comp rx=0 tx=0).stop 2024-09-12T02:04:22.242 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.241+0000 7f7979c65640 1 -- 172.21.15.16:0/3452687034 shutdown_connections 2024-09-12T02:04:22.242 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.241+0000 7f7979c65640 1 --2- 172.21.15.16:0/3452687034 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7974107820 0x7f7974109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:22.242 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.241+0000 7f7979c65640 1 --2- 172.21.15.16:0/3452687034 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7974104ef0 0x7f79741072e0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:22.242 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.241+0000 7f7979c65640 1 --2- 172.21.15.16:0/3452687034 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f79741025c0 0x7f79741049b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:22.242 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.241+0000 7f7979c65640 1 -- 172.21.15.16:0/3452687034 >> 172.21.15.16:0/3452687034 conn(0x7f79740fc1e0 msgr2=0x7f79740fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:04:22.242 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.241+0000 7f7979c65640 1 -- 172.21.15.16:0/3452687034 shutdown_connections 2024-09-12T02:04:22.242 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.241+0000 7f7979c65640 1 -- 172.21.15.16:0/3452687034 wait complete. 2024-09-12T02:04:22.242 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.242+0000 7f7979c65640 1 Processor -- start 2024-09-12T02:04:22.244 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.242+0000 7f7979c65640 1 -- start start 2024-09-12T02:04:22.244 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.242+0000 7f7979c65640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79741025c0 0x7f7974101f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:22.244 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.242+0000 7f7979c65640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7974104ef0 0x7f7974100570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:22.244 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.242+0000 7f7979c65640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7974107820 0x7f7974100ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:22.245 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.242+0000 7f7979c65640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f797410d840 con 0x7f7974104ef0 2024-09-12T02:04:22.245 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.242+0000 7f7979c65640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f797410d6c0 con 0x7f79741025c0 2024-09-12T02:04:22.245 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.242+0000 7f7979c65640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f797410d9c0 con 0x7f7974107820 2024-09-12T02:04:22.245 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.242+0000 7f7972ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7974104ef0 0x7f7974100570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:22.245 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.242+0000 7f7972ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7974104ef0 0x7f7974100570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42744/0 (socket says 172.21.15.16:42744) 2024-09-12T02:04:22.245 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.242+0000 7f7972ffd640 1 -- 172.21.15.16:0/3692256825 learned_addr learned my addr 172.21.15.16:0/3692256825 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:04:22.245 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.242+0000 7f7973fff640 1 --2- 172.21.15.16:0/3692256825 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7974107820 0x7f7974100ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:22.245 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.242+0000 7f79737fe640 1 --2- 172.21.15.16:0/3692256825 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79741025c0 0x7f7974101f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:22.245 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.243+0000 7f7972ffd640 1 -- 172.21.15.16:0/3692256825 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7974107820 msgr2=0x7f7974100ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:22.245 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.243+0000 7f7972ffd640 1 --2- 172.21.15.16:0/3692256825 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7974107820 0x7f7974100ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:22.245 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.243+0000 7f7972ffd640 1 -- 172.21.15.16:0/3692256825 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79741025c0 msgr2=0x7f7974101f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:22.245 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.243+0000 7f7972ffd640 1 --2- 172.21.15.16:0/3692256825 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79741025c0 0x7f7974101f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:22.245 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.243+0000 7f7972ffd640 1 -- 172.21.15.16:0/3692256825 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7974101340 con 0x7f7974104ef0 2024-09-12T02:04:22.246 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.243+0000 7f79737fe640 1 --2- 172.21.15.16:0/3692256825 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79741025c0 0x7f7974101f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:04:22.246 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.243+0000 7f7972ffd640 1 --2- 172.21.15.16:0/3692256825 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7974104ef0 0x7f7974100570 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f7968031440 tx=0x7f7968032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:04:22.246 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.243+0000 7f7970ff9640 1 -- 172.21.15.16:0/3692256825 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f796803f030 con 0x7f7974104ef0 2024-09-12T02:04:22.246 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.243+0000 7f7979c65640 1 -- 172.21.15.16:0/3692256825 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79741a1f00 con 0x7f7974104ef0 2024-09-12T02:04:22.246 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.243+0000 7f7970ff9640 1 -- 172.21.15.16:0/3692256825 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7968003ce0 con 0x7f7974104ef0 2024-09-12T02:04:22.246 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.243+0000 7f7970ff9640 1 -- 172.21.15.16:0/3692256825 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7968035070 con 0x7f7974104ef0 2024-09-12T02:04:22.246 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.243+0000 7f7979c65640 1 -- 172.21.15.16:0/3692256825 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79741a2360 con 0x7f7974104ef0 2024-09-12T02:04:22.252 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.245+0000 7f7970ff9640 1 -- 172.21.15.16:0/3692256825 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f7968042030 con 0x7f7974104ef0 2024-09-12T02:04:22.252 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.246+0000 7f797ca02740 1 Processor -- start 2024-09-12T02:04:22.253 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.246+0000 7f797ca02740 1 -- start start 2024-09-12T02:04:22.253 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.246+0000 7f797ca02740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559071bb09a0 0x559071bb0da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:22.253 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.246+0000 7f797ca02740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559071bb12e0 0x559071bb5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:22.253 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.246+0000 7f797ca02740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559071bb6370 0x559071bb8830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:22.253 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.246+0000 7f797ca02740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x559071b142f0 con 0x559071bb12e0 2024-09-12T02:04:22.253 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.246+0000 7f797ca02740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x559071abc270 con 0x559071bb6370 2024-09-12T02:04:22.253 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.246+0000 7f797ca02740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x559071b102b0 con 0x559071bb09a0 2024-09-12T02:04:22.253 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.246+0000 7f79737fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559071bb12e0 0x559071bb5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:22.253 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.246+0000 7f79737fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559071bb12e0 0x559071bb5da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42748/0 (socket says 172.21.15.16:42748) 2024-09-12T02:04:22.253 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.246+0000 7f79737fe640 1 -- 172.21.15.16:0/880436661 learned_addr learned my addr 172.21.15.16:0/880436661 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:04:22.253 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.246+0000 7f7973fff640 1 --2- 172.21.15.16:0/880436661 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559071bb09a0 0x559071bb0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:22.253 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.246+0000 7f7972ffd640 1 --2- 172.21.15.16:0/880436661 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559071bb6370 0x559071bb8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:22.254 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.246+0000 7f7970ff9640 1 --2- 172.21.15.16:0/3692256825 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f795c0a2c30 0x7f795c0a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:22.254 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.246+0000 7f79737fe640 1 -- 172.21.15.16:0/880436661 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559071bb09a0 msgr2=0x559071bb0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:22.254 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.246+0000 7f79737fe640 1 --2- 172.21.15.16:0/880436661 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559071bb09a0 0x559071bb0da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:22.254 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.246+0000 7f79737fe640 1 -- 172.21.15.16:0/880436661 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559071bb6370 msgr2=0x559071bb8830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:22.254 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.246+0000 7f79737fe640 1 --2- 172.21.15.16:0/880436661 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559071bb6370 0x559071bb8830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:22.254 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.246+0000 7f79737fe640 1 -- 172.21.15.16:0/880436661 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559071abb340 con 0x559071bb12e0 2024-09-12T02:04:22.254 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.246+0000 7f7973fff640 1 --2- 172.21.15.16:0/3692256825 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f795c0a2c30 0x7f795c0a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:22.254 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.246+0000 7f7972ffd640 1 --2- 172.21.15.16:0/880436661 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559071bb6370 0x559071bb8830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:04:22.254 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.247+0000 7f7970ff9640 1 -- 172.21.15.16:0/3692256825 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f79680ef4f0 con 0x7f7974104ef0 2024-09-12T02:04:22.254 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.247+0000 7f79737fe640 1 --2- 172.21.15.16:0/880436661 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559071bb12e0 0x559071bb5da0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f7964009c30 tx=0x7f7964010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65becc838bd0191f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:04:22.254 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.247+0000 7f7959ffb640 1 -- 172.21.15.16:0/880436661 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7964004020 con 0x559071bb12e0 2024-09-12T02:04:22.254 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.247+0000 7f7959ffb640 1 -- 172.21.15.16:0/880436661 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79640041c0 con 0x559071bb12e0 2024-09-12T02:04:22.255 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.247+0000 7f7959ffb640 1 -- 172.21.15.16:0/880436661 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7964012690 con 0x559071bb12e0 2024-09-12T02:04:22.255 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.247+0000 7f7973fff640 1 --2- 172.21.15.16:0/3692256825 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f795c0a2c30 0x7f795c0a50f0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f797406a980 tx=0x7f796c004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:04:22.255 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.248+0000 7f797ca02740 1 -- 172.21.15.16:0/880436661 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559071bb12e0 msgr2=0x559071bb5da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:22.255 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.248+0000 7f797ca02740 1 --2- 172.21.15.16:0/880436661 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559071bb12e0 0x559071bb5da0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f7964009c30 tx=0x7f7964010740 comp rx=0 tx=0).stop 2024-09-12T02:04:22.255 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.248+0000 7f797ca02740 1 -- 172.21.15.16:0/880436661 shutdown_connections 2024-09-12T02:04:22.255 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.248+0000 7f797ca02740 1 --2- 172.21.15.16:0/880436661 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559071bb6370 0x559071bb8830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:22.255 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.248+0000 7f797ca02740 1 --2- 172.21.15.16:0/880436661 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559071bb12e0 0x559071bb5da0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:22.255 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.248+0000 7f797ca02740 1 --2- 172.21.15.16:0/880436661 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559071bb09a0 0x559071bb0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:22.255 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.248+0000 7f797ca02740 1 -- 172.21.15.16:0/880436661 >> 172.21.15.16:0/880436661 conn(0x559071bac010 msgr2=0x559071bac410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:04:22.255 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.248+0000 7f797ca02740 1 -- 172.21.15.16:0/880436661 shutdown_connections 2024-09-12T02:04:22.255 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.248+0000 7f797ca02740 1 -- 172.21.15.16:0/880436661 wait complete. 2024-09-12T02:04:22.255 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.248+0000 7f797ca02740 1 -- start start 2024-09-12T02:04:22.255 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.249+0000 7f797ca02740 1 Processor -- start 2024-09-12T02:04:22.256 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.249+0000 7f797ca02740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559071bb09a0 0x559071bbac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:22.256 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.249+0000 7f797ca02740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559071bb12e0 0x559071bbb1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:22.256 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.249+0000 7f797ca02740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559071bb6370 0x559071bc21e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:22.256 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.249+0000 7f797ca02740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x559071bb9340 con 0x559071bb09a0 2024-09-12T02:04:22.256 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.249+0000 7f797ca02740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x559071a9d290 con 0x559071bb6370 2024-09-12T02:04:22.256 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.249+0000 7f797ca02740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x559071bb94c0 con 0x559071bb12e0 2024-09-12T02:04:22.256 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.249+0000 7f797ca02740 10 client.0 _ll_register_callbacks cb 0x559071ace140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:04:22.256 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.249+0000 7f79737fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559071bb09a0 0x559071bbac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:22.256 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.249+0000 7f79737fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559071bb09a0 0x559071bbac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42750/0 (socket says 172.21.15.16:42750) 2024-09-12T02:04:22.256 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.249+0000 7f79737fe640 1 -- 172.21.15.16:0/229300466 learned_addr learned my addr 172.21.15.16:0/229300466 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:04:22.256 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.249+0000 7f7972ffd640 1 --2- 172.21.15.16:0/229300466 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559071bb12e0 0x559071bbb1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:22.256 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.249+0000 7f7973fff640 1 --2- 172.21.15.16:0/229300466 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559071bb6370 0x559071bc21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:22.257 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.249+0000 7f79737fe640 1 -- 172.21.15.16:0/229300466 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559071bb12e0 msgr2=0x559071bbb1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:22.257 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.249+0000 7f79737fe640 1 --2- 172.21.15.16:0/229300466 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559071bb12e0 0x559071bbb1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:22.257 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.249+0000 7f79737fe640 1 -- 172.21.15.16:0/229300466 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559071bb6370 msgr2=0x559071bc21e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:22.257 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.249+0000 7f79737fe640 1 --2- 172.21.15.16:0/229300466 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559071bb6370 0x559071bc21e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:22.257 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.249+0000 7f79737fe640 1 -- 172.21.15.16:0/229300466 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559071827210 con 0x559071bb09a0 2024-09-12T02:04:22.257 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.249+0000 7f7972ffd640 1 --2- 172.21.15.16:0/229300466 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559071bb12e0 0x559071bbb1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:04:22.257 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.249+0000 7f7973fff640 1 --2- 172.21.15.16:0/229300466 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559071bb6370 0x559071bc21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:04:22.257 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.250+0000 7f79737fe640 1 --2- 172.21.15.16:0/229300466 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559071bb09a0 0x559071bbac60 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f7964009460 tx=0x7f79640045a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a51446a9183231 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:04:22.257 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.250+0000 7f7952ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:04:22.257 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.250+0000 7f7952ffd640 1 -- 172.21.15.16:0/229300466 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79640035f0 con 0x559071bb09a0 2024-09-12T02:04:22.257 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.250+0000 7f7952ffd640 1 -- 172.21.15.16:0/229300466 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f7964003790 con 0x559071bb09a0 2024-09-12T02:04:22.257 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.250+0000 7f797ca02740 1 -- 172.21.15.16:0/229300466 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f79640041c0 con 0x559071bb09a0 2024-09-12T02:04:22.258 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.250+0000 7f7952ffd640 1 -- 172.21.15.16:0/229300466 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f79640039b0 con 0x559071bb09a0 2024-09-12T02:04:22.258 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.250+0000 7f797ca02740 10 client.16161 fetch_fsmap learned FSMap version 25 2024-09-12T02:04:22.258 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.250+0000 7f797ca02740 1 -- 172.21.15.16:0/229300466 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5590718f3620 con 0x559071bb09a0 2024-09-12T02:04:22.258 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.250+0000 7f7952ffd640 1 -- 172.21.15.16:0/229300466 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79640041c0 con 0x559071bb09a0 2024-09-12T02:04:22.258 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.250+0000 7f7952ffd640 1 -- 172.21.15.16:0/229300466 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f796400f040 con 0x559071bb09a0 2024-09-12T02:04:22.258 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.250+0000 7f797ca02740 10 client.16161 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:04:22.258 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.250+0000 7f797ca02740 10 client.16161 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:04:22.258 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.250+0000 7f797ca02740 20 client.16161 populate_metadata read hostname 'smithi016' 2024-09-12T02:04:22.258 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.250+0000 7f797ca02740 1 --2- 172.21.15.16:0/229300466 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x559071bc6640 0x559071bc8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:22.258 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.250+0000 7f797ca02740 4 client.16161 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:04:22.258 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.250+0000 7f797ca02740 1 -- 172.21.15.16:0/229300466 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559071b21a20 con 0x559071bc6640 2024-09-12T02:04:22.258 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.251+0000 7f7972ffd640 1 --2- 172.21.15.16:0/229300466 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x559071bc6640 0x559071bc8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:22.258 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.251+0000 7f7972ffd640 1 --2- 172.21.15.16:0/229300466 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x559071bc6640 0x559071bc8ab0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9429036b509d6318 server_cookie=f334ba29c88519bf in_seq=0 out_seq=0 2024-09-12T02:04:22.259 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.251+0000 7f7952ffd640 10 client.16161 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:04:22.259 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.252+0000 7f7952ffd640 1 -- 172.21.15.16:0/229300466 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559071b21a20 con 0x559071bc6640 2024-09-12T02:04:22.259 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.252+0000 7f7952ffd640 10 client.16161 handle_command_reply: tid=0 2024-09-12T02:04:22.259 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.252+0000 7f797ca02740 1 client.16161 shutdown 2024-09-12T02:04:22.259 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.252+0000 7f797ca02740 1 -- 172.21.15.16:0/229300466 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559071bb09a0 msgr2=0x559071bbac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:22.259 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.252+0000 7f797ca02740 1 --2- 172.21.15.16:0/229300466 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559071bb09a0 0x559071bbac60 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f7964009460 tx=0x7f79640045a0 comp rx=0 tx=0).stop 2024-09-12T02:04:22.259 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.252+0000 7f797ca02740 1 -- 172.21.15.16:0/229300466 shutdown_connections 2024-09-12T02:04:22.259 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.252+0000 7f797ca02740 1 --2- 172.21.15.16:0/229300466 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x559071bc6640 0x559071bc8ab0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:22.259 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.252+0000 7f7952ffd640 0 client.16161 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:04:22.259 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.252+0000 7f797ca02740 1 --2- 172.21.15.16:0/229300466 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559071bb6370 0x559071bc21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:22.259 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.252+0000 7f797ca02740 1 --2- 172.21.15.16:0/229300466 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559071bb12e0 0x559071bbb1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:22.259 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.252+0000 7f797ca02740 1 --2- 172.21.15.16:0/229300466 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559071bb09a0 0x559071bbac60 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:22.259 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.252+0000 7f797ca02740 1 -- 172.21.15.16:0/229300466 >> 172.21.15.16:0/229300466 conn(0x559071bac010 msgr2=0x559071bad820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:04:22.260 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.252+0000 7f797ca02740 1 -- 172.21.15.16:0/229300466 shutdown_connections 2024-09-12T02:04:22.260 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.252+0000 7f797ca02740 1 -- 172.21.15.16:0/229300466 wait complete. 2024-09-12T02:04:22.260 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.252+0000 7f797ca02740 20 client.16161 trim_cache size 0 max 16384 2024-09-12T02:04:22.283 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.281+0000 7f797ca02740 1 Processor -- start 2024-09-12T02:04:22.283 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.281+0000 7f797ca02740 1 -- start start 2024-09-12T02:04:22.283 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.281+0000 7f797ca02740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559071bb09a0 0x559071bb0d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:22.283 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.281+0000 7f797ca02740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559071bb12e0 0x559071bc2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:22.283 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.281+0000 7f797ca02740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559071bb6370 0x559071bab0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:22.283 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.281+0000 7f797ca02740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f79741ad960 con 0x559071bb09a0 2024-09-12T02:04:22.283 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.281+0000 7f797ca02740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f79741ab230 con 0x559071bb6370 2024-09-12T02:04:22.283 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.281+0000 7f797ca02740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f79741a9080 con 0x559071bb12e0 2024-09-12T02:04:22.283 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.281+0000 7f79737fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559071bb09a0 0x559071bb0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:22.284 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.281+0000 7f79737fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559071bb09a0 0x559071bb0d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42752/0 (socket says 172.21.15.16:42752) 2024-09-12T02:04:22.284 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.281+0000 7f79737fe640 1 -- 172.21.15.16:0/2008644634 learned_addr learned my addr 172.21.15.16:0/2008644634 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:04:22.284 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.281+0000 7f7972ffd640 1 --2- 172.21.15.16:0/2008644634 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559071bb12e0 0x559071bc2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:22.284 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.281+0000 7f7973fff640 1 --2- 172.21.15.16:0/2008644634 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559071bb6370 0x559071bab0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:22.284 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.281+0000 7f79737fe640 1 -- 172.21.15.16:0/2008644634 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559071bb12e0 msgr2=0x559071bc2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:22.284 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.281+0000 7f79737fe640 1 --2- 172.21.15.16:0/2008644634 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559071bb12e0 0x559071bc2100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:22.284 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.281+0000 7f79737fe640 1 -- 172.21.15.16:0/2008644634 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559071bb6370 msgr2=0x559071bab0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:22.284 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.281+0000 7f79737fe640 1 --2- 172.21.15.16:0/2008644634 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559071bb6370 0x559071bab0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:22.284 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.281+0000 7f79737fe640 1 -- 172.21.15.16:0/2008644634 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559071babe30 con 0x559071bb09a0 2024-09-12T02:04:22.284 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.282+0000 7f79737fe640 1 --2- 172.21.15.16:0/2008644634 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559071bb09a0 0x559071bb0d80 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f7964010780 tx=0x7f7964003d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=520a72bc0fbcb918 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:04:22.284 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.282+0000 7f79597fa640 1 -- 172.21.15.16:0/2008644634 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7964012450 con 0x559071bb09a0 2024-09-12T02:04:22.284 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.282+0000 7f79597fa640 1 -- 172.21.15.16:0/2008644634 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79640035f0 con 0x559071bb09a0 2024-09-12T02:04:22.285 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.282+0000 7f79597fa640 1 -- 172.21.15.16:0/2008644634 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7964004f70 con 0x559071bb09a0 2024-09-12T02:04:22.285 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.282+0000 7f797ca02740 1 -- 172.21.15.16:0/2008644634 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559071bb09a0 msgr2=0x559071bb0d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:22.285 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.282+0000 7f797ca02740 1 --2- 172.21.15.16:0/2008644634 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559071bb09a0 0x559071bb0d80 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f7964010780 tx=0x7f7964003d20 comp rx=0 tx=0).stop 2024-09-12T02:04:22.285 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.282+0000 7f797ca02740 1 -- 172.21.15.16:0/2008644634 shutdown_connections 2024-09-12T02:04:22.285 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.282+0000 7f797ca02740 1 --2- 172.21.15.16:0/2008644634 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559071bb6370 0x559071bab0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:22.285 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.282+0000 7f797ca02740 1 --2- 172.21.15.16:0/2008644634 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559071bb12e0 0x559071bc2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:22.285 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.282+0000 7f797ca02740 1 --2- 172.21.15.16:0/2008644634 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559071bb09a0 0x559071bb0d80 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:22.285 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.283+0000 7f797ca02740 1 -- 172.21.15.16:0/2008644634 >> 172.21.15.16:0/2008644634 conn(0x559071bac010 msgr2=0x559071baa9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:04:22.285 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.283+0000 7f797ca02740 1 -- 172.21.15.16:0/2008644634 shutdown_connections 2024-09-12T02:04:22.285 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.283+0000 7f797ca02740 1 -- 172.21.15.16:0/2008644634 wait complete. 2024-09-12T02:04:22.285 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.283+0000 7f797ca02740 1 -- start start 2024-09-12T02:04:22.285 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.283+0000 7f797ca02740 1 Processor -- start 2024-09-12T02:04:22.289 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.283+0000 7f797ca02740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559071bb09a0 0x559071bbf560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:22.289 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.283+0000 7f797ca02740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559071bb12e0 0x559071bbfaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:22.289 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.283+0000 7f797ca02740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559071bb6370 0x559071bc0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:22.289 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.283+0000 7f797ca02740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f79741a96f0 con 0x559071bb6370 2024-09-12T02:04:22.289 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.283+0000 7f797ca02740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f79741aab60 con 0x559071bb09a0 2024-09-12T02:04:22.289 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.283+0000 7f797ca02740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f79741a9dc0 con 0x559071bb12e0 2024-09-12T02:04:22.289 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.283+0000 7f797ca02740 10 client.0 _ll_register_callbacks cb 0x7f796c0099c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:04:22.289 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.284+0000 7f7973fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559071bb6370 0x559071bc0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:22.289 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.284+0000 7f79737fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559071bb09a0 0x559071bbf560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:22.289 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.284+0000 7f7973fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559071bb6370 0x559071bc0ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42764/0 (socket says 172.21.15.16:42764) 2024-09-12T02:04:22.289 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.284+0000 7f7973fff640 1 -- 172.21.15.16:0/580722707 learned_addr learned my addr 172.21.15.16:0/580722707 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:04:22.289 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.284+0000 7f7972ffd640 1 --2- 172.21.15.16:0/580722707 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559071bb12e0 0x559071bbfaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:22.290 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.284+0000 7f79737fe640 1 -- 172.21.15.16:0/580722707 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559071bb12e0 msgr2=0x559071bbfaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:22.290 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.284+0000 7f79737fe640 1 --2- 172.21.15.16:0/580722707 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559071bb12e0 0x559071bbfaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:22.290 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.284+0000 7f79737fe640 1 -- 172.21.15.16:0/580722707 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559071bb6370 msgr2=0x559071bc0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:22.290 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.284+0000 7f79737fe640 1 --2- 172.21.15.16:0/580722707 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559071bb6370 0x559071bc0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:22.290 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.284+0000 7f79737fe640 1 -- 172.21.15.16:0/580722707 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559071bc10c0 con 0x559071bb09a0 2024-09-12T02:04:22.290 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.284+0000 7f7973fff640 1 --2- 172.21.15.16:0/580722707 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559071bb6370 0x559071bc0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:04:22.290 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.284+0000 7f79737fe640 1 --2- 172.21.15.16:0/580722707 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559071bb09a0 0x559071bbf560 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7964004ef0 tx=0x7f7964005730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b75651f22ae590d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:04:22.290 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.284+0000 7f7952ffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-12T02:04:22.290 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.284+0000 7f7952ffd640 1 -- 172.21.15.16:0/580722707 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f796403f060 con 0x559071bb09a0 2024-09-12T02:04:22.290 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.284+0000 7f797ca02740 1 -- 172.21.15.16:0/580722707 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f79640035f0 con 0x559071bb09a0 2024-09-12T02:04:22.290 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.284+0000 7f7952ffd640 1 -- 172.21.15.16:0/580722707 <== mon.2 v2:172.21.15.33:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f7964012870 con 0x559071bb09a0 2024-09-12T02:04:22.290 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.285+0000 7f7952ffd640 1 -- 172.21.15.16:0/580722707 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7964003d90 con 0x559071bb09a0 2024-09-12T02:04:22.290 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.285+0000 7f797ca02740 10 client.25751 fetch_fsmap learned FSMap version 25 2024-09-12T02:04:22.291 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.285+0000 7f797ca02740 1 -- 172.21.15.16:0/580722707 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x559071bc23d0 con 0x559071bb09a0 2024-09-12T02:04:22.291 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.285+0000 7f7952ffd640 1 -- 172.21.15.16:0/580722707 <== mon.2 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79640035f0 con 0x559071bb09a0 2024-09-12T02:04:22.291 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.285+0000 7f7952ffd640 1 -- 172.21.15.16:0/580722707 <== mon.2 v2:172.21.15.33:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f7964044070 con 0x559071bb09a0 2024-09-12T02:04:22.291 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.285+0000 7f797ca02740 10 client.25751 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:04:22.291 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.285+0000 7f797ca02740 10 client.25751 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:04:22.291 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.285+0000 7f797ca02740 20 client.25751 populate_metadata read hostname 'smithi016' 2024-09-12T02:04:22.291 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.285+0000 7f797ca02740 1 --2- 172.21.15.16:0/580722707 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x559071bc6640 0x559071c339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:22.291 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.285+0000 7f797ca02740 4 client.25751 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:04:22.291 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.285+0000 7f797ca02740 1 -- 172.21.15.16:0/580722707 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559071b1bd70 con 0x559071bc6640 2024-09-12T02:04:22.291 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.285+0000 7f7972ffd640 1 --2- 172.21.15.16:0/580722707 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x559071bc6640 0x559071c339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:22.291 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.286+0000 7f7952ffd640 10 client.25751 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:04:22.291 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.286+0000 7f7972ffd640 1 --2- 172.21.15.16:0/580722707 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x559071bc6640 0x559071c339f0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff5f51e5987a3692 server_cookie=b482c5b098f93cde in_seq=0 out_seq=0 2024-09-12T02:04:22.300 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.299+0000 7f7952ffd640 1 -- 172.21.15.16:0/580722707 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559071b1bd70 con 0x559071bc6640 2024-09-12T02:04:22.300 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.299+0000 7f7952ffd640 10 client.25751 handle_command_reply: tid=0 2024-09-12T02:04:22.300 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.299+0000 7f797ca02740 1 client.25751 shutdown 2024-09-12T02:04:22.300 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.299+0000 7f797ca02740 1 -- 172.21.15.16:0/580722707 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559071bb09a0 msgr2=0x559071bbf560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:22.300 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.299+0000 7f797ca02740 1 --2- 172.21.15.16:0/580722707 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559071bb09a0 0x559071bbf560 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7964004ef0 tx=0x7f7964005730 comp rx=0 tx=0).stop 2024-09-12T02:04:22.300 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.299+0000 7f797ca02740 1 -- 172.21.15.16:0/580722707 shutdown_connections 2024-09-12T02:04:22.300 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.299+0000 7f797ca02740 1 --2- 172.21.15.16:0/580722707 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x559071bc6640 0x559071c339f0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:22.300 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.299+0000 7f797ca02740 1 --2- 172.21.15.16:0/580722707 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559071bb6370 0x559071bc0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:22.300 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.299+0000 7f797ca02740 1 --2- 172.21.15.16:0/580722707 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559071bb12e0 0x559071bbfaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:22.300 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.299+0000 7f797ca02740 1 --2- 172.21.15.16:0/580722707 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559071bb09a0 0x559071bbf560 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:22.301 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.299+0000 7f7952ffd640 0 client.25751 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:04:22.301 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.299+0000 7f797ca02740 1 -- 172.21.15.16:0/580722707 >> 172.21.15.16:0/580722707 conn(0x559071bac010 msgr2=0x559071bab1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:04:22.301 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.299+0000 7f797ca02740 1 -- 172.21.15.16:0/580722707 shutdown_connections 2024-09-12T02:04:22.301 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.299+0000 7f797ca02740 1 -- 172.21.15.16:0/580722707 wait complete. 2024-09-12T02:04:22.301 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.300+0000 7f797ca02740 20 client.25751 trim_cache size 0 max 16384 2024-09-12T02:04:22.301 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-09-12T02:04:22.301 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-09-12T02:04:22.301 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-09-12T02:04:22.301 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-09-12T02:04:22.302 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.300+0000 7f7979c65640 1 -- 172.21.15.16:0/3692256825 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f795c0a2c30 msgr2=0x7f795c0a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:22.302 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.300+0000 7f7979c65640 1 --2- 172.21.15.16:0/3692256825 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f795c0a2c30 0x7f795c0a50f0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f797406a980 tx=0x7f796c004a40 comp rx=0 tx=0).stop 2024-09-12T02:04:22.302 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.300+0000 7f7979c65640 1 -- 172.21.15.16:0/3692256825 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7974104ef0 msgr2=0x7f7974100570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:22.303 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.300+0000 7f7979c65640 1 --2- 172.21.15.16:0/3692256825 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7974104ef0 0x7f7974100570 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f7968031440 tx=0x7f7968032ce0 comp rx=0 tx=0).stop 2024-09-12T02:04:22.303 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.300+0000 7f7979c65640 1 -- 172.21.15.16:0/3692256825 shutdown_connections 2024-09-12T02:04:22.303 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.300+0000 7f7979c65640 1 --2- 172.21.15.16:0/3692256825 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f795c0a2c30 0x7f795c0a50f0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:22.303 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.300+0000 7f7979c65640 1 --2- 172.21.15.16:0/3692256825 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7974107820 0x7f7974100ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:22.303 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.300+0000 7f7979c65640 1 --2- 172.21.15.16:0/3692256825 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7974104ef0 0x7f7974100570 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:22.303 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.300+0000 7f7979c65640 1 --2- 172.21.15.16:0/3692256825 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79741025c0 0x7f7974101f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:22.303 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.300+0000 7f7979c65640 1 -- 172.21.15.16:0/3692256825 >> 172.21.15.16:0/3692256825 conn(0x7f79740fc1e0 msgr2=0x7f7974102f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:04:22.303 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.300+0000 7f7979c65640 1 -- 172.21.15.16:0/3692256825 shutdown_connections 2024-09-12T02:04:22.303 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:22.300+0000 7f7979c65640 1 -- 172.21.15.16:0/3692256825 wait complete. 2024-09-12T02:04:22.391 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:04:22.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.868+0000 7f83a3842640 1 -- 172.21.15.16:0/4000802526 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f839c103660 msgr2=0x7f839c10bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:22.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.868+0000 7f83a3842640 1 --2- 172.21.15.16:0/4000802526 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f839c103660 0x7f839c10bb50 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f838c009900 tx=0x7f838c02a480 comp rx=0 tx=0).stop 2024-09-12T02:04:22.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.868+0000 7f83a3842640 1 -- 172.21.15.16:0/4000802526 shutdown_connections 2024-09-12T02:04:22.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.868+0000 7f83a3842640 1 --2- 172.21.15.16:0/4000802526 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f839c10c170 0x7f839c10e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:22.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.868+0000 7f83a3842640 1 --2- 172.21.15.16:0/4000802526 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f839c103660 0x7f839c10bb50 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:22.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.868+0000 7f83a3842640 1 --2- 172.21.15.16:0/4000802526 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f839c102d40 0x7f839c103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:22.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.868+0000 7f83a3842640 1 -- 172.21.15.16:0/4000802526 >> 172.21.15.16:0/4000802526 conn(0x7f839c0fc7a0 msgr2=0x7f839c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:04:22.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.868+0000 7f83a3842640 1 -- 172.21.15.16:0/4000802526 shutdown_connections 2024-09-12T02:04:22.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.869+0000 7f83a3842640 1 -- 172.21.15.16:0/4000802526 wait complete. 2024-09-12T02:04:22.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.869+0000 7f83a3842640 1 Processor -- start 2024-09-12T02:04:22.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.869+0000 7f83a3842640 1 -- start start 2024-09-12T02:04:22.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.869+0000 7f83a3842640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f839c102d40 0x7f839c1009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:22.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.869+0000 7f83a3842640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f839c103660 0x7f839c100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:22.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.869+0000 7f83a3842640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f839c10c170 0x7f839c101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:22.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.869+0000 7f83a3842640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f839c111fc0 con 0x7f839c103660 2024-09-12T02:04:22.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.869+0000 7f83a3842640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f839c111e40 con 0x7f839c10c170 2024-09-12T02:04:22.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.869+0000 7f83a3842640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f839c112140 con 0x7f839c102d40 2024-09-12T02:04:22.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.869+0000 7f83a0ddd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f839c103660 0x7f839c100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:22.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.870+0000 7f83a0ddd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f839c103660 0x7f839c100ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42780/0 (socket says 172.21.15.16:42780) 2024-09-12T02:04:22.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.870+0000 7f83a0ddd640 1 -- 172.21.15.16:0/3023003172 learned_addr learned my addr 172.21.15.16:0/3023003172 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:04:22.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.870+0000 7f83a15de640 1 --2- 172.21.15.16:0/3023003172 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f839c102d40 0x7f839c1009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:22.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.870+0000 7f83a1ddf640 1 --2- 172.21.15.16:0/3023003172 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f839c10c170 0x7f839c101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:22.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.870+0000 7f83a0ddd640 1 -- 172.21.15.16:0/3023003172 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f839c102d40 msgr2=0x7f839c1009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:22.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.870+0000 7f83a0ddd640 1 --2- 172.21.15.16:0/3023003172 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f839c102d40 0x7f839c1009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:22.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.870+0000 7f83a0ddd640 1 -- 172.21.15.16:0/3023003172 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f839c10c170 msgr2=0x7f839c101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:22.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.870+0000 7f83a0ddd640 1 --2- 172.21.15.16:0/3023003172 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f839c10c170 0x7f839c101420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:22.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.870+0000 7f83a0ddd640 1 -- 172.21.15.16:0/3023003172 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f839c1a6940 con 0x7f839c103660 2024-09-12T02:04:22.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.870+0000 7f83a1ddf640 1 --2- 172.21.15.16:0/3023003172 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f839c10c170 0x7f839c101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:04:22.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.870+0000 7f83a0ddd640 1 --2- 172.21.15.16:0/3023003172 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f839c103660 0x7f839c100ee0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f838c004770 tx=0x7f838c003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:04:22.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.870+0000 7f83927fc640 1 -- 172.21.15.16:0/3023003172 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f838c03f070 con 0x7f839c103660 2024-09-12T02:04:22.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.870+0000 7f83a3842640 1 -- 172.21.15.16:0/3023003172 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f839c1a6bd0 con 0x7f839c103660 2024-09-12T02:04:22.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.870+0000 7f83927fc640 1 -- 172.21.15.16:0/3023003172 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f838c031ca0 con 0x7f839c103660 2024-09-12T02:04:22.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.870+0000 7f83927fc640 1 -- 172.21.15.16:0/3023003172 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f838c035030 con 0x7f839c103660 2024-09-12T02:04:22.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.870+0000 7f83a3842640 1 -- 172.21.15.16:0/3023003172 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f839c1a70b0 con 0x7f839c103660 2024-09-12T02:04:22.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.873+0000 7f83927fc640 1 -- 172.21.15.16:0/3023003172 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f838c042070 con 0x7f839c103660 2024-09-12T02:04:22.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.873+0000 7f83a5c1b740 1 Processor -- start 2024-09-12T02:04:22.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.873+0000 7f83a5c1b740 1 -- start start 2024-09-12T02:04:22.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.874+0000 7f83927fc640 1 --2- 172.21.15.16:0/3023003172 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f83880a2b80 0x7f83880a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:22.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.874+0000 7f83a5c1b740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5630befdc4a0 0x5630befdc880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:22.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.874+0000 7f83a5c1b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5630befdcdc0 0x5630befe7140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:22.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.874+0000 7f83a5c1b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5630befe7680 0x5630befe9a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:22.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.874+0000 7f83a5c1b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5630bef422e0 con 0x5630befe7680 2024-09-12T02:04:22.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.874+0000 7f83a5c1b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5630beee9ce0 con 0x5630befdcdc0 2024-09-12T02:04:22.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.874+0000 7f83a5c1b740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5630bef3e2a0 con 0x5630befdc4a0 2024-09-12T02:04:22.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.874+0000 7f83a1ddf640 1 --2- 172.21.15.16:0/3023003172 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f83880a2b80 0x7f83880a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:22.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.874+0000 7f83a1ddf640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5630befe7680 0x5630befe9a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:22.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.874+0000 7f83a15de640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5630befdc4a0 0x5630befdc880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:22.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.874+0000 7f83927fc640 1 -- 172.21.15.16:0/3023003172 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f838c0ef6d0 con 0x7f839c103660 2024-09-12T02:04:22.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.874+0000 7f83a15de640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5630befdc4a0 0x5630befdc880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:53310/0 (socket says 172.21.15.16:53310) 2024-09-12T02:04:22.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.874+0000 7f83a0ddd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5630befdcdc0 0x5630befe7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:22.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.874+0000 7f83a15de640 1 -- 172.21.15.16:0/3168965832 learned_addr learned my addr 172.21.15.16:0/3168965832 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:04:22.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.874+0000 7f83a15de640 1 -- 172.21.15.16:0/3168965832 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5630befdcdc0 msgr2=0x5630befe7140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:22.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.874+0000 7f83a15de640 1 --2- 172.21.15.16:0/3168965832 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5630befdcdc0 0x5630befe7140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:22.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.874+0000 7f83a15de640 1 -- 172.21.15.16:0/3168965832 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5630befe7680 msgr2=0x5630befe9a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:22.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.874+0000 7f83a15de640 1 --2- 172.21.15.16:0/3168965832 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5630befe7680 0x5630befe9a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:22.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.874+0000 7f83a15de640 1 -- 172.21.15.16:0/3168965832 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630beece440 con 0x5630befdc4a0 2024-09-12T02:04:22.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.874+0000 7f83a1ddf640 1 --2- 172.21.15.16:0/3023003172 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f83880a2b80 0x7f83880a5040 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f8394009fd0 tx=0x7f8394009d80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:04:22.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.875+0000 7f83a15de640 1 --2- 172.21.15.16:0/3168965832 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5630befdc4a0 0x5630befdc880 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f839800bcd0 tx=0x7f83980107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7c2a908d2c47f903 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:04:22.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.875+0000 7f83877fe640 1 -- 172.21.15.16:0/3168965832 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8398005220 con 0x5630befdc4a0 2024-09-12T02:04:22.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.875+0000 7f83877fe640 1 -- 172.21.15.16:0/3168965832 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8398004c80 con 0x5630befdc4a0 2024-09-12T02:04:22.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.875+0000 7f83877fe640 1 -- 172.21.15.16:0/3168965832 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83980117c0 con 0x5630befdc4a0 2024-09-12T02:04:22.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.875+0000 7f83a5c1b740 1 -- 172.21.15.16:0/3168965832 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5630befdc4a0 msgr2=0x5630befdc880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:22.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.875+0000 7f83a5c1b740 1 --2- 172.21.15.16:0/3168965832 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5630befdc4a0 0x5630befdc880 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f839800bcd0 tx=0x7f83980107a0 comp rx=0 tx=0).stop 2024-09-12T02:04:22.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.875+0000 7f83a5c1b740 1 -- 172.21.15.16:0/3168965832 shutdown_connections 2024-09-12T02:04:22.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.875+0000 7f83a5c1b740 1 --2- 172.21.15.16:0/3168965832 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5630befe7680 0x5630befe9a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:22.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.875+0000 7f83a5c1b740 1 --2- 172.21.15.16:0/3168965832 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5630befdcdc0 0x5630befe7140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:22.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.876+0000 7f83a5c1b740 1 --2- 172.21.15.16:0/3168965832 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5630befdc4a0 0x5630befdc880 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:22.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.876+0000 7f83a5c1b740 1 -- 172.21.15.16:0/3168965832 >> 172.21.15.16:0/3168965832 conn(0x5630befd8a30 msgr2=0x5630befd8e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:04:22.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.876+0000 7f83a5c1b740 1 -- 172.21.15.16:0/3168965832 shutdown_connections 2024-09-12T02:04:22.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.876+0000 7f83a5c1b740 1 -- 172.21.15.16:0/3168965832 wait complete. 2024-09-12T02:04:22.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.876+0000 7f83a5c1b740 1 -- start start 2024-09-12T02:04:22.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.876+0000 7f83a5c1b740 1 Processor -- start 2024-09-12T02:04:22.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.876+0000 7f83a5c1b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5630befdc4a0 0x5630befe6f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:22.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.876+0000 7f83a5c1b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5630befdcdc0 0x5630befe4240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:22.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.876+0000 7f83a5c1b740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5630befe7680 0x5630befe4780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:22.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.876+0000 7f83a5c1b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5630befea250 con 0x5630befdcdc0 2024-09-12T02:04:22.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.876+0000 7f83a5c1b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5630beefd050 con 0x5630befdc4a0 2024-09-12T02:04:22.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.876+0000 7f83a5c1b740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5630befea3d0 con 0x5630befe7680 2024-09-12T02:04:22.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.876+0000 7f83a5c1b740 10 client.0 _ll_register_callbacks cb 0x5630bef3d160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:04:22.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.876+0000 7f83a0ddd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5630befdcdc0 0x5630befe4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:22.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.877+0000 7f83a0ddd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5630befdcdc0 0x5630befe4240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42802/0 (socket says 172.21.15.16:42802) 2024-09-12T02:04:22.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.877+0000 7f83a0ddd640 1 -- 172.21.15.16:0/1054393535 learned_addr learned my addr 172.21.15.16:0/1054393535 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:04:22.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.877+0000 7f83a1ddf640 1 --2- 172.21.15.16:0/1054393535 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5630befe7680 0x5630befe4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:22.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.877+0000 7f83a15de640 1 --2- 172.21.15.16:0/1054393535 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5630befdc4a0 0x5630befe6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:22.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.877+0000 7f83a0ddd640 1 -- 172.21.15.16:0/1054393535 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5630befe7680 msgr2=0x5630befe4780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:22.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.877+0000 7f83a0ddd640 1 --2- 172.21.15.16:0/1054393535 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5630befe7680 0x5630befe4780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:22.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.877+0000 7f83a0ddd640 1 -- 172.21.15.16:0/1054393535 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5630befdc4a0 msgr2=0x5630befe6f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:22.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.877+0000 7f83a0ddd640 1 --2- 172.21.15.16:0/1054393535 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5630befdc4a0 0x5630befe6f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:22.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.877+0000 7f83a0ddd640 1 -- 172.21.15.16:0/1054393535 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630becb33c0 con 0x5630befdcdc0 2024-09-12T02:04:22.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.877+0000 7f83a15de640 1 --2- 172.21.15.16:0/1054393535 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5630befdc4a0 0x5630befe6f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:04:22.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.877+0000 7f83a0ddd640 1 --2- 172.21.15.16:0/1054393535 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5630befdcdc0 0x5630befe4240 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f838c0b8430 tx=0x7f838c0bbbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a43ac2f40ac1ec2e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:04:22.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.877+0000 7f8384ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:04:22.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.877+0000 7f8384ff9640 1 -- 172.21.15.16:0/1054393535 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f838c049070 con 0x5630befdcdc0 2024-09-12T02:04:22.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.877+0000 7f8384ff9640 1 -- 172.21.15.16:0/1054393535 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f838c0ba2d0 con 0x5630befdcdc0 2024-09-12T02:04:22.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.877+0000 7f83a5c1b740 1 -- 172.21.15.16:0/1054393535 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8398004c80 con 0x5630befdcdc0 2024-09-12T02:04:22.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.877+0000 7f8384ff9640 1 -- 172.21.15.16:0/1054393535 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f838c0bc6d0 con 0x5630befdcdc0 2024-09-12T02:04:22.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.877+0000 7f83a5c1b740 10 client.16179 fetch_fsmap learned FSMap version 25 2024-09-12T02:04:22.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.877+0000 7f83a5c1b740 1 -- 172.21.15.16:0/1054393535 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5630beff5bd0 con 0x5630befdcdc0 2024-09-12T02:04:22.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.878+0000 7f8384ff9640 1 -- 172.21.15.16:0/1054393535 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8398004c80 con 0x5630befdcdc0 2024-09-12T02:04:22.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.878+0000 7f8384ff9640 1 -- 172.21.15.16:0/1054393535 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f838c0a6ce0 con 0x5630befdcdc0 2024-09-12T02:04:22.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.878+0000 7f83a5c1b740 10 client.16179 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:04:22.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.878+0000 7f83a5c1b740 10 client.16179 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:04:22.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.878+0000 7f83a5c1b740 20 client.16179 populate_metadata read hostname 'smithi016' 2024-09-12T02:04:22.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.878+0000 7f83a5c1b740 1 --2- 172.21.15.16:0/1054393535 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5630beff6fc0 0x5630beff93f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:22.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.878+0000 7f83a5c1b740 4 client.16179 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:04:22.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.878+0000 7f83a5c1b740 1 -- 172.21.15.16:0/1054393535 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5630bef4f850 con 0x5630beff6fc0 2024-09-12T02:04:22.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.878+0000 7f83a15de640 1 --2- 172.21.15.16:0/1054393535 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5630beff6fc0 0x5630beff93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:22.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.878+0000 7f83a15de640 1 --2- 172.21.15.16:0/1054393535 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5630beff6fc0 0x5630beff93f0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cdf762fca12d363f server_cookie=f80567167554208c in_seq=0 out_seq=0 2024-09-12T02:04:22.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.878+0000 7f8384ff9640 10 client.16179 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:04:22.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.879+0000 7f8384ff9640 1 -- 172.21.15.16:0/1054393535 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5630bef4f850 con 0x5630beff6fc0 2024-09-12T02:04:22.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.879+0000 7f8384ff9640 10 client.16179 handle_command_reply: tid=0 2024-09-12T02:04:22.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.879+0000 7f83a5c1b740 1 client.16179 shutdown 2024-09-12T02:04:22.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.879+0000 7f83a5c1b740 1 -- 172.21.15.16:0/1054393535 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5630befdcdc0 msgr2=0x5630befe4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:22.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.879+0000 7f83a5c1b740 1 --2- 172.21.15.16:0/1054393535 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5630befdcdc0 0x5630befe4240 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f838c0b8430 tx=0x7f838c0bbbd0 comp rx=0 tx=0).stop 2024-09-12T02:04:22.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.879+0000 7f83a5c1b740 1 -- 172.21.15.16:0/1054393535 shutdown_connections 2024-09-12T02:04:22.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.879+0000 7f83a5c1b740 1 --2- 172.21.15.16:0/1054393535 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5630beff6fc0 0x5630beff93f0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:22.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.879+0000 7f83a5c1b740 1 --2- 172.21.15.16:0/1054393535 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5630befe7680 0x5630befe4780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:22.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.879+0000 7f83a5c1b740 1 --2- 172.21.15.16:0/1054393535 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5630befdcdc0 0x5630befe4240 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:22.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.879+0000 7f83a5c1b740 1 --2- 172.21.15.16:0/1054393535 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5630befdc4a0 0x5630befe6f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:22.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.879+0000 7f83a5c1b740 1 -- 172.21.15.16:0/1054393535 >> 172.21.15.16:0/1054393535 conn(0x5630befd8a30 msgr2=0x5630befde710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:04:22.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.879+0000 7f8384ff9640 0 client.16179 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:04:22.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.880+0000 7f83a5c1b740 1 -- 172.21.15.16:0/1054393535 shutdown_connections 2024-09-12T02:04:22.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.880+0000 7f83a5c1b740 1 -- 172.21.15.16:0/1054393535 wait complete. 2024-09-12T02:04:22.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.880+0000 7f83a5c1b740 20 client.16179 trim_cache size 0 max 16384 2024-09-12T02:04:22.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.897+0000 7f83a5c1b740 1 Processor -- start 2024-09-12T02:04:22.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.897+0000 7f83a5c1b740 1 -- start start 2024-09-12T02:04:22.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.897+0000 7f83a5c1b740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5630befdc4a0 0x5630beffecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:22.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.897+0000 7f83a5c1b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5630befdcdc0 0x5630befff200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:22.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.897+0000 7f83a5c1b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5630befe7680 0x5630befe14a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:22.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.897+0000 7f83a5c1b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f839c1b1f70 con 0x5630befe7680 2024-09-12T02:04:22.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.897+0000 7f83a5c1b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f839c1af840 con 0x5630befdcdc0 2024-09-12T02:04:22.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.897+0000 7f83a5c1b740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f839c1ad690 con 0x5630befdc4a0 2024-09-12T02:04:22.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.897+0000 7f83a1ddf640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5630befe7680 0x5630befe14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:22.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.897+0000 7f83a1ddf640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5630befe7680 0x5630befe14a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42814/0 (socket says 172.21.15.16:42814) 2024-09-12T02:04:22.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.897+0000 7f83a1ddf640 1 -- 172.21.15.16:0/430362413 learned_addr learned my addr 172.21.15.16:0/430362413 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:04:22.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.897+0000 7f83a0ddd640 1 --2- 172.21.15.16:0/430362413 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5630befdcdc0 0x5630befff200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:22.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.897+0000 7f83a15de640 1 --2- 172.21.15.16:0/430362413 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5630befdc4a0 0x5630beffecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:22.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.897+0000 7f83a1ddf640 1 -- 172.21.15.16:0/430362413 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5630befdc4a0 msgr2=0x5630beffecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:22.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.897+0000 7f83a1ddf640 1 --2- 172.21.15.16:0/430362413 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5630befdc4a0 0x5630beffecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:22.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.897+0000 7f83a1ddf640 1 -- 172.21.15.16:0/430362413 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5630befdcdc0 msgr2=0x5630befff200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:22.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.897+0000 7f83a1ddf640 1 --2- 172.21.15.16:0/430362413 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5630befdcdc0 0x5630befff200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:22.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.898+0000 7f83a1ddf640 1 -- 172.21.15.16:0/430362413 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630befe7300 con 0x5630befe7680 2024-09-12T02:04:22.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.898+0000 7f83a15de640 1 --2- 172.21.15.16:0/430362413 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5630befdc4a0 0x5630beffecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:04:22.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.898+0000 7f83a1ddf640 1 --2- 172.21.15.16:0/430362413 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5630befe7680 0x5630befe14a0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f83940092f0 tx=0x7f839401f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39d6c9ef216cac5c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:04:22.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.898+0000 7f8386ffd640 1 -- 172.21.15.16:0/430362413 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8394004b20 con 0x5630befe7680 2024-09-12T02:04:22.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.898+0000 7f8386ffd640 1 -- 172.21.15.16:0/430362413 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8394004cc0 con 0x5630befe7680 2024-09-12T02:04:22.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.898+0000 7f8386ffd640 1 -- 172.21.15.16:0/430362413 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83940057d0 con 0x5630befe7680 2024-09-12T02:04:22.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.899+0000 7f83a5c1b740 1 -- 172.21.15.16:0/430362413 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5630befe7680 msgr2=0x5630befe14a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:22.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.899+0000 7f83a5c1b740 1 --2- 172.21.15.16:0/430362413 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5630befe7680 0x5630befe14a0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f83940092f0 tx=0x7f839401f000 comp rx=0 tx=0).stop 2024-09-12T02:04:22.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.899+0000 7f83a5c1b740 1 -- 172.21.15.16:0/430362413 shutdown_connections 2024-09-12T02:04:22.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.899+0000 7f83a5c1b740 1 --2- 172.21.15.16:0/430362413 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5630befe7680 0x5630befe14a0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:22.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.899+0000 7f83a5c1b740 1 --2- 172.21.15.16:0/430362413 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5630befdcdc0 0x5630befff200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:22.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.899+0000 7f83a5c1b740 1 --2- 172.21.15.16:0/430362413 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5630befdc4a0 0x5630beffecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:22.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.899+0000 7f83a5c1b740 1 -- 172.21.15.16:0/430362413 >> 172.21.15.16:0/430362413 conn(0x5630befd8a30 msgr2=0x5630befe62c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:04:22.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.899+0000 7f83a5c1b740 1 -- 172.21.15.16:0/430362413 shutdown_connections 2024-09-12T02:04:22.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.899+0000 7f83a5c1b740 1 -- 172.21.15.16:0/430362413 wait complete. 2024-09-12T02:04:22.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.899+0000 7f83a5c1b740 1 -- start start 2024-09-12T02:04:22.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.899+0000 7f83a5c1b740 1 Processor -- start 2024-09-12T02:04:22.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.900+0000 7f83a5c1b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5630befdc4a0 0x5630befefdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:22.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.900+0000 7f83a5c1b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5630befdcdc0 0x5630befed1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:22.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.900+0000 7f83a5c1b740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5630befe7680 0x5630befed6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:22.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.900+0000 7f83a5c1b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f839c1add00 con 0x5630befdc4a0 2024-09-12T02:04:22.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.900+0000 7f83a5c1b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f839c1af170 con 0x5630befdcdc0 2024-09-12T02:04:22.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.900+0000 7f83a5c1b740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f839c1ae3d0 con 0x5630befe7680 2024-09-12T02:04:22.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.900+0000 7f83a5c1b740 10 client.0 _ll_register_callbacks cb 0x7f838c0b8270 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:04:22.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.900+0000 7f83a15de640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5630befdc4a0 0x5630befefdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:22.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.900+0000 7f83a15de640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5630befdc4a0 0x5630befefdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42816/0 (socket says 172.21.15.16:42816) 2024-09-12T02:04:22.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.900+0000 7f83a15de640 1 -- 172.21.15.16:0/2532836148 learned_addr learned my addr 172.21.15.16:0/2532836148 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:04:22.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.900+0000 7f83a0ddd640 1 --2- 172.21.15.16:0/2532836148 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5630befdcdc0 0x5630befed1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:22.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.900+0000 7f83a1ddf640 1 --2- 172.21.15.16:0/2532836148 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5630befe7680 0x5630befed6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:22.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.900+0000 7f83a15de640 1 -- 172.21.15.16:0/2532836148 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5630befe7680 msgr2=0x5630befed6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:22.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.900+0000 7f83a15de640 1 --2- 172.21.15.16:0/2532836148 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5630befe7680 0x5630befed6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:22.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.900+0000 7f83a15de640 1 -- 172.21.15.16:0/2532836148 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5630befdcdc0 msgr2=0x5630befed1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:22.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.900+0000 7f83a15de640 1 --2- 172.21.15.16:0/2532836148 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5630befdcdc0 0x5630befed1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:22.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.900+0000 7f83a15de640 1 -- 172.21.15.16:0/2532836148 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630befedf20 con 0x5630befdc4a0 2024-09-12T02:04:22.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.900+0000 7f83a15de640 1 --2- 172.21.15.16:0/2532836148 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5630befdc4a0 0x5630befefdc0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f839800bcd0 tx=0x7f839800bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b198c2d081e3b84 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:04:22.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.900+0000 7f8384ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:04:22.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.900+0000 7f8384ff9640 1 -- 172.21.15.16:0/2532836148 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83980277c0 con 0x5630befdc4a0 2024-09-12T02:04:22.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.900+0000 7f8384ff9640 1 -- 172.21.15.16:0/2532836148 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f8398027960 con 0x5630befdc4a0 2024-09-12T02:04:22.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.900+0000 7f83a5c1b740 1 -- 172.21.15.16:0/2532836148 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8394004cc0 con 0x5630befdc4a0 2024-09-12T02:04:22.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.901+0000 7f8384ff9640 1 -- 172.21.15.16:0/2532836148 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f8398027b60 con 0x5630befdc4a0 2024-09-12T02:04:22.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.901+0000 7f83a5c1b740 10 client.16185 fetch_fsmap learned FSMap version 25 2024-09-12T02:04:22.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.901+0000 7f83a5c1b740 1 -- 172.21.15.16:0/2532836148 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5630bf061ce0 con 0x5630befdc4a0 2024-09-12T02:04:22.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.901+0000 7f8384ff9640 1 -- 172.21.15.16:0/2532836148 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8394004cc0 con 0x5630befdc4a0 2024-09-12T02:04:22.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.901+0000 7f8384ff9640 1 -- 172.21.15.16:0/2532836148 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f839802f060 con 0x5630befdc4a0 2024-09-12T02:04:22.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.901+0000 7f83a5c1b740 10 client.16185 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:04:22.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.901+0000 7f83a5c1b740 10 client.16185 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:04:22.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.901+0000 7f83a5c1b740 20 client.16185 populate_metadata read hostname 'smithi016' 2024-09-12T02:04:22.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.901+0000 7f83a5c1b740 1 --2- 172.21.15.16:0/2532836148 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5630bf062270 0x5630bf064660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:22.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.901+0000 7f83a5c1b740 4 client.16185 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T02:04:22.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.901+0000 7f83a5c1b740 1 -- 172.21.15.16:0/2532836148 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5630bef49d60 con 0x5630bf062270 2024-09-12T02:04:22.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.901+0000 7f83a0ddd640 1 --2- 172.21.15.16:0/2532836148 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5630bf062270 0x5630bf064660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:22.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.902+0000 7f8384ff9640 10 client.16185 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:04:22.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.902+0000 7f83a0ddd640 1 --2- 172.21.15.16:0/2532836148 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5630bf062270 0x5630bf064660 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a7d82ff02521030 server_cookie=e472ab70b6e4e32 in_seq=0 out_seq=0 2024-09-12T02:04:22.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.902+0000 7f8384ff9640 1 -- 172.21.15.16:0/2532836148 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5630bef49d60 con 0x5630bf062270 2024-09-12T02:04:22.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.902+0000 7f8384ff9640 10 client.16185 handle_command_reply: tid=0 2024-09-12T02:04:22.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.902+0000 7f83a5c1b740 1 client.16185 shutdown 2024-09-12T02:04:22.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.902+0000 7f83a5c1b740 1 -- 172.21.15.16:0/2532836148 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5630befdc4a0 msgr2=0x5630befefdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:22.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.902+0000 7f83a5c1b740 1 --2- 172.21.15.16:0/2532836148 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5630befdc4a0 0x5630befefdc0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f839800bcd0 tx=0x7f839800bf80 comp rx=0 tx=0).stop 2024-09-12T02:04:22.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.902+0000 7f83a5c1b740 1 -- 172.21.15.16:0/2532836148 shutdown_connections 2024-09-12T02:04:22.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.902+0000 7f83a5c1b740 1 --2- 172.21.15.16:0/2532836148 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5630bf062270 0x5630bf064660 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:22.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.902+0000 7f83a5c1b740 1 --2- 172.21.15.16:0/2532836148 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5630befe7680 0x5630befed6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:22.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.902+0000 7f83a5c1b740 1 --2- 172.21.15.16:0/2532836148 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5630befdcdc0 0x5630befed1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:22.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.902+0000 7f83a5c1b740 1 --2- 172.21.15.16:0/2532836148 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5630befdc4a0 0x5630befefdc0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:22.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.902+0000 7f8384ff9640 0 client.16185 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:04:22.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.902+0000 7f83a5c1b740 1 -- 172.21.15.16:0/2532836148 >> 172.21.15.16:0/2532836148 conn(0x5630befd8a30 msgr2=0x5630befd7350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:04:22.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.902+0000 7f83a5c1b740 1 -- 172.21.15.16:0/2532836148 shutdown_connections 2024-09-12T02:04:22.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.902+0000 7f83a5c1b740 1 -- 172.21.15.16:0/2532836148 wait complete. 2024-09-12T02:04:22.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.902+0000 7f83a5c1b740 20 client.16185 trim_cache size 0 max 16384 2024-09-12T02:04:22.915 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T02:04:22.915 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-12T02:04:22.915 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": { 2024-09-12T02:04:22.915 INFO:teuthology.orchestra.run.smithi016.stdout: "d72e5b90-5676-4cfc-9bf2-eaf44e51a533": { 2024-09-12T02:04:22.915 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-12T02:04:22.916 INFO:teuthology.orchestra.run.smithi016.stdout: "tag": "d72e5b90-5676-4cfc-9bf2-eaf44e51a533", 2024-09-12T02:04:22.916 INFO:teuthology.orchestra.run.smithi016.stdout: "options": "recursive,force" 2024-09-12T02:04:22.916 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:04:22.916 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:04:22.916 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T02:04:22.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.903+0000 7f83a3842640 1 -- 172.21.15.16:0/3023003172 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f83880a2b80 msgr2=0x7f83880a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:22.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.903+0000 7f83a3842640 1 --2- 172.21.15.16:0/3023003172 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f83880a2b80 0x7f83880a5040 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f8394009fd0 tx=0x7f8394009d80 comp rx=0 tx=0).stop 2024-09-12T02:04:22.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.903+0000 7f83a3842640 1 -- 172.21.15.16:0/3023003172 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f839c103660 msgr2=0x7f839c100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:22.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.903+0000 7f83a3842640 1 --2- 172.21.15.16:0/3023003172 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f839c103660 0x7f839c100ee0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f838c004770 tx=0x7f838c003c10 comp rx=0 tx=0).stop 2024-09-12T02:04:22.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.903+0000 7f83a3842640 1 -- 172.21.15.16:0/3023003172 shutdown_connections 2024-09-12T02:04:22.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.903+0000 7f83a3842640 1 --2- 172.21.15.16:0/3023003172 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f83880a2b80 0x7f83880a5040 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:22.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.903+0000 7f83a3842640 1 --2- 172.21.15.16:0/3023003172 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f839c10c170 0x7f839c101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:22.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.903+0000 7f83a3842640 1 --2- 172.21.15.16:0/3023003172 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f839c103660 0x7f839c100ee0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:22.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.903+0000 7f83a3842640 1 --2- 172.21.15.16:0/3023003172 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f839c102d40 0x7f839c1009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:22.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.903+0000 7f83a3842640 1 -- 172.21.15.16:0/3023003172 >> 172.21.15.16:0/3023003172 conn(0x7f839c0fc7a0 msgr2=0x7f839c0fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:04:22.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.903+0000 7f83a3842640 1 -- 172.21.15.16:0/3023003172 shutdown_connections 2024-09-12T02:04:22.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:22.903+0000 7f83a3842640 1 -- 172.21.15.16:0/3023003172 wait complete. 2024-09-12T02:04:23.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:04:22 smithi184 ceph-mon[35593]: scrub queued for path: / 2024-09-12T02:04:23.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:04:22 smithi184 ceph-mon[35593]: scrub summary: idle+waiting paths [/] 2024-09-12T02:04:23.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:04:22 smithi184 ceph-mon[35593]: scrub summary: active paths [/] 2024-09-12T02:04:23.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:04:22 smithi184 ceph-mon[35593]: scrub summary: idle+waiting paths [/] 2024-09-12T02:04:23.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:04:22 smithi033 ceph-mon[35332]: scrub queued for path: / 2024-09-12T02:04:23.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:04:22 smithi033 ceph-mon[35332]: scrub summary: idle+waiting paths [/] 2024-09-12T02:04:23.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:04:22 smithi033 ceph-mon[35332]: scrub summary: active paths [/] 2024-09-12T02:04:23.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:04:22 smithi033 ceph-mon[35332]: scrub summary: idle+waiting paths [/] 2024-09-12T02:04:23.183 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:04:22 smithi016 ceph-mon[30947]: scrub queued for path: / 2024-09-12T02:04:23.183 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:04:22 smithi016 ceph-mon[30947]: scrub summary: idle+waiting paths [/] 2024-09-12T02:04:23.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:04:22 smithi016 ceph-mon[30947]: scrub summary: active paths [/] 2024-09-12T02:04:23.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:04:22 smithi016 ceph-mon[30947]: scrub summary: idle+waiting paths [/] 2024-09-12T02:04:23.771 INFO:tasks.cephfs.filesystem:scrub status for tag:d72e5b90-5676-4cfc-9bf2-eaf44e51a533 - {'path': '/', 'tag': 'd72e5b90-5676-4cfc-9bf2-eaf44e51a533', 'options': 'recursive,force'} 2024-09-12T02:04:24.128 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:04:23 smithi184 ceph-mon[35593]: pgmap v409: 97 pgs: 97 active+clean; 1.5 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 11 op/s 2024-09-12T02:04:24.168 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:04:23 smithi033 ceph-mon[35332]: pgmap v409: 97 pgs: 97 active+clean; 1.5 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 11 op/s 2024-09-12T02:04:24.183 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:04:23 smithi016 ceph-mon[30947]: pgmap v409: 97 pgs: 97 active+clean; 1.5 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 11 op/s 2024-09-12T02:04:26.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:04:25 smithi184 ceph-mon[35593]: pgmap v410: 97 pgs: 97 active+clean; 1.5 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-12T02:04:26.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:04:25 smithi033 ceph-mon[35332]: pgmap v410: 97 pgs: 97 active+clean; 1.5 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-12T02:04:26.183 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:04:25 smithi016 ceph-mon[30947]: pgmap v410: 97 pgs: 97 active+clean; 1.5 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-12T02:04:27.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:04:26 smithi184 ceph-mon[35593]: scrub summary: idle 2024-09-12T02:04:27.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:04:26 smithi033 ceph-mon[35332]: scrub summary: idle 2024-09-12T02:04:27.183 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:04:26 smithi016 ceph-mon[30947]: scrub summary: idle 2024-09-12T02:04:28.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:04:27 smithi033 ceph-mon[35332]: pgmap v411: 97 pgs: 97 active+clean; 1.5 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 7.5 KiB/s wr, 12 op/s 2024-09-12T02:04:28.183 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:04:27 smithi016 ceph-mon[30947]: pgmap v411: 97 pgs: 97 active+clean; 1.5 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 7.5 KiB/s wr, 12 op/s 2024-09-12T02:04:28.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:04:27 smithi184 ceph-mon[35593]: pgmap v411: 97 pgs: 97 active+clean; 1.5 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 7.5 KiB/s wr, 12 op/s 2024-09-12T02:04:30.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:04:29 smithi033 ceph-mon[35332]: pgmap v412: 97 pgs: 97 active+clean; 1.5 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 7.4 KiB/s wr, 11 op/s 2024-09-12T02:04:30.183 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:04:29 smithi016 ceph-mon[30947]: pgmap v412: 97 pgs: 97 active+clean; 1.5 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 7.4 KiB/s wr, 11 op/s 2024-09-12T02:04:30.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:04:29 smithi184 ceph-mon[35593]: pgmap v412: 97 pgs: 97 active+clean; 1.5 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 7.4 KiB/s wr, 11 op/s 2024-09-12T02:04:32.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:04:31 smithi033 ceph-mon[35332]: pgmap v413: 97 pgs: 97 active+clean; 1.5 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.4 KiB/s wr, 13 op/s 2024-09-12T02:04:32.183 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:04:31 smithi016 ceph-mon[30947]: pgmap v413: 97 pgs: 97 active+clean; 1.5 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.4 KiB/s wr, 13 op/s 2024-09-12T02:04:32.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:04:31 smithi184 ceph-mon[35593]: pgmap v413: 97 pgs: 97 active+clean; 1.5 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.4 KiB/s wr, 13 op/s 2024-09-12T02:04:34.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:04:33 smithi033 ceph-mon[35332]: pgmap v414: 97 pgs: 97 active+clean; 1.5 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.3 KiB/s wr, 9 op/s 2024-09-12T02:04:34.183 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:04:33 smithi016 ceph-mon[30947]: pgmap v414: 97 pgs: 97 active+clean; 1.5 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.3 KiB/s wr, 9 op/s 2024-09-12T02:04:34.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:04:33 smithi184 ceph-mon[35593]: pgmap v414: 97 pgs: 97 active+clean; 1.5 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.3 KiB/s wr, 9 op/s 2024-09-12T02:04:35.401 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.400+0000 7f1f49123640 1 -- 172.21.15.16:0/980861740 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f44100a60 msgr2=0x7f1f44100e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:35.401 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.400+0000 7f1f49123640 1 --2- 172.21.15.16:0/980861740 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f44100a60 0x7f1f44100e60 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f1f34009900 tx=0x7f1f3402a410 comp rx=0 tx=0).stop 2024-09-12T02:04:35.401 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.401+0000 7f1f49123640 1 -- 172.21.15.16:0/980861740 shutdown_connections 2024-09-12T02:04:35.401 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.401+0000 7f1f49123640 1 --2- 172.21.15.16:0/980861740 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f44102610 0x7f1f44108ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:35.402 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.401+0000 7f1f49123640 1 --2- 172.21.15.16:0/980861740 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1f44101c50 0x7f1f441020d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:35.402 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.401+0000 7f1f49123640 1 --2- 172.21.15.16:0/980861740 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f44100a60 0x7f1f44100e60 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:35.402 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.401+0000 7f1f49123640 1 -- 172.21.15.16:0/980861740 >> 172.21.15.16:0/980861740 conn(0x7f1f440fc1e0 msgr2=0x7f1f440fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:04:35.402 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.401+0000 7f1f49123640 1 -- 172.21.15.16:0/980861740 shutdown_connections 2024-09-12T02:04:35.402 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.401+0000 7f1f49123640 1 -- 172.21.15.16:0/980861740 wait complete. 2024-09-12T02:04:35.402 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.401+0000 7f1f49123640 1 Processor -- start 2024-09-12T02:04:35.404 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.402+0000 7f1f49123640 1 -- start start 2024-09-12T02:04:35.404 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.402+0000 7f1f49123640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f44100a60 0x7f1f441049f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:35.404 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.402+0000 7f1f49123640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f44101c50 0x7f1f44103030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:35.404 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.402+0000 7f1f49123640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1f44102610 0x7f1f44103570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:35.404 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.402+0000 7f1f49123640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1f4410c6c0 con 0x7f1f44100a60 2024-09-12T02:04:35.404 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.402+0000 7f1f49123640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1f4410c540 con 0x7f1f44101c50 2024-09-12T02:04:35.404 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.402+0000 7f1f49123640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1f4410c840 con 0x7f1f44102610 2024-09-12T02:04:35.404 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.402+0000 7f1f42d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f44100a60 0x7f1f441049f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:35.404 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.402+0000 7f1f42d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f44100a60 0x7f1f441049f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35816/0 (socket says 172.21.15.16:35816) 2024-09-12T02:04:35.405 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.402+0000 7f1f42d9d640 1 -- 172.21.15.16:0/3537607614 learned_addr learned my addr 172.21.15.16:0/3537607614 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:04:35.405 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.402+0000 7f1f4359e640 1 --2- 172.21.15.16:0/3537607614 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1f44102610 0x7f1f44103570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:35.405 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.402+0000 7f1f4259c640 1 --2- 172.21.15.16:0/3537607614 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f44101c50 0x7f1f44103030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:35.405 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.402+0000 7f1f42d9d640 1 -- 172.21.15.16:0/3537607614 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1f44102610 msgr2=0x7f1f44103570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:35.405 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.402+0000 7f1f42d9d640 1 --2- 172.21.15.16:0/3537607614 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1f44102610 0x7f1f44103570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:35.405 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.402+0000 7f1f42d9d640 1 -- 172.21.15.16:0/3537607614 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f44101c50 msgr2=0x7f1f44103030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:35.405 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.402+0000 7f1f42d9d640 1 --2- 172.21.15.16:0/3537607614 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f44101c50 0x7f1f44103030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:35.405 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.402+0000 7f1f42d9d640 1 -- 172.21.15.16:0/3537607614 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f44103e00 con 0x7f1f44100a60 2024-09-12T02:04:35.405 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.403+0000 7f1f42d9d640 1 --2- 172.21.15.16:0/3537607614 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f44100a60 0x7f1f441049f0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f1f34031610 tx=0x7f1f34004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:04:35.405 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.403+0000 7f1f2bfff640 1 -- 172.21.15.16:0/3537607614 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f3403f030 con 0x7f1f44100a60 2024-09-12T02:04:35.405 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.403+0000 7f1f2bfff640 1 -- 172.21.15.16:0/3537607614 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f34003ad0 con 0x7f1f44100a60 2024-09-12T02:04:35.405 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.403+0000 7f1f49123640 1 -- 172.21.15.16:0/3537607614 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f441a1ea0 con 0x7f1f44100a60 2024-09-12T02:04:35.406 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.403+0000 7f1f2bfff640 1 -- 172.21.15.16:0/3537607614 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f34035070 con 0x7f1f44100a60 2024-09-12T02:04:35.406 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.403+0000 7f1f49123640 1 -- 172.21.15.16:0/3537607614 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f441a22b0 con 0x7f1f44100a60 2024-09-12T02:04:35.413 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.405+0000 7f1f2bfff640 1 -- 172.21.15.16:0/3537607614 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f1f34042030 con 0x7f1f44100a60 2024-09-12T02:04:35.413 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.406+0000 7f1f4bebb740 1 Processor -- start 2024-09-12T02:04:35.413 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.406+0000 7f1f4bebb740 1 -- start start 2024-09-12T02:04:35.413 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.406+0000 7f1f4bebb740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5634653639a0 0x563465363da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:35.413 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.406+0000 7f1f4bebb740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5634653642e0 0x563465368da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:35.413 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.406+0000 7f1f4bebb740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563465369370 0x56346536b830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:35.413 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.406+0000 7f1f4bebb740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5634652c72f0 con 0x563465369370 2024-09-12T02:04:35.413 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.406+0000 7f1f4bebb740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56346526f270 con 0x5634653639a0 2024-09-12T02:04:35.413 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.406+0000 7f1f4bebb740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5634652c32b0 con 0x5634653642e0 2024-09-12T02:04:35.413 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.406+0000 7f1f4259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563465369370 0x56346536b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:35.414 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.406+0000 7f1f4359e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5634653639a0 0x563465363da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:35.414 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.406+0000 7f1f4259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563465369370 0x56346536b830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35828/0 (socket says 172.21.15.16:35828) 2024-09-12T02:04:35.414 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.406+0000 7f1f4259c640 1 -- 172.21.15.16:0/540672184 learned_addr learned my addr 172.21.15.16:0/540672184 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:04:35.414 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.406+0000 7f1f4359e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5634653639a0 0x563465363da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.16:46338/0 (socket says 172.21.15.16:46338) 2024-09-12T02:04:35.414 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.406+0000 7f1f2bfff640 1 --2- 172.21.15.16:0/3537607614 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f1f2c0a2c30 0x7f1f2c0a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:35.414 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.406+0000 7f1f42d9d640 1 --2- 172.21.15.16:0/540672184 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5634653642e0 0x563465368da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:35.414 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.407+0000 7f1f4259c640 1 -- 172.21.15.16:0/540672184 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5634653642e0 msgr2=0x563465368da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:35.414 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.407+0000 7f1f4259c640 1 --2- 172.21.15.16:0/540672184 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5634653642e0 0x563465368da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:35.414 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.407+0000 7f1f4259c640 1 -- 172.21.15.16:0/540672184 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5634653639a0 msgr2=0x563465363da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:35.414 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.407+0000 7f1f4259c640 1 --2- 172.21.15.16:0/540672184 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5634653639a0 0x563465363da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:35.414 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.407+0000 7f1f4259c640 1 -- 172.21.15.16:0/540672184 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56346526e340 con 0x563465369370 2024-09-12T02:04:35.414 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.407+0000 7f1f4359e640 1 --2- 172.21.15.16:0/540672184 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5634653639a0 0x563465363da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:04:35.415 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.407+0000 7f1f4359e640 1 --2- 172.21.15.16:0/3537607614 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f1f2c0a2c30 0x7f1f2c0a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:35.415 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.407+0000 7f1f2bfff640 1 -- 172.21.15.16:0/3537607614 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f1f340ef6a0 con 0x7f1f44100a60 2024-09-12T02:04:35.415 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.407+0000 7f1f4259c640 1 --2- 172.21.15.16:0/540672184 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563465369370 0x56346536b830 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f1f3800bd10 tx=0x7f1f380107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20bbc5440c397688 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:04:35.415 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.407+0000 7f1f297fa640 1 -- 172.21.15.16:0/540672184 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f38005220 con 0x563465369370 2024-09-12T02:04:35.415 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.407+0000 7f1f4359e640 1 --2- 172.21.15.16:0/3537607614 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f1f2c0a2c30 0x7f1f2c0a50f0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f1f3c009fd0 tx=0x7f1f3c004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:04:35.415 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.407+0000 7f1f297fa640 1 -- 172.21.15.16:0/540672184 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f38004c80 con 0x563465369370 2024-09-12T02:04:35.415 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.407+0000 7f1f297fa640 1 -- 172.21.15.16:0/540672184 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f380118a0 con 0x563465369370 2024-09-12T02:04:35.415 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.408+0000 7f1f4bebb740 1 -- 172.21.15.16:0/540672184 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563465369370 msgr2=0x56346536b830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:35.415 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.408+0000 7f1f4bebb740 1 --2- 172.21.15.16:0/540672184 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563465369370 0x56346536b830 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f1f3800bd10 tx=0x7f1f380107a0 comp rx=0 tx=0).stop 2024-09-12T02:04:35.415 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.408+0000 7f1f4bebb740 1 -- 172.21.15.16:0/540672184 shutdown_connections 2024-09-12T02:04:35.415 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.408+0000 7f1f4bebb740 1 --2- 172.21.15.16:0/540672184 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563465369370 0x56346536b830 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:35.415 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.408+0000 7f1f4bebb740 1 --2- 172.21.15.16:0/540672184 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5634653642e0 0x563465368da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:35.415 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.408+0000 7f1f4bebb740 1 --2- 172.21.15.16:0/540672184 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5634653639a0 0x563465363da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:35.416 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.408+0000 7f1f4bebb740 1 -- 172.21.15.16:0/540672184 >> 172.21.15.16:0/540672184 conn(0x56346535f010 msgr2=0x56346535f410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:04:35.416 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.408+0000 7f1f4bebb740 1 -- 172.21.15.16:0/540672184 shutdown_connections 2024-09-12T02:04:35.416 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.408+0000 7f1f4bebb740 1 -- 172.21.15.16:0/540672184 wait complete. 2024-09-12T02:04:35.416 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.408+0000 7f1f4bebb740 1 -- start start 2024-09-12T02:04:35.416 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.408+0000 7f1f4bebb740 1 Processor -- start 2024-09-12T02:04:35.416 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.409+0000 7f1f4bebb740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5634653639a0 0x56346536dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:35.416 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.409+0000 7f1f4bebb740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634653642e0 0x56346536e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:35.416 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.409+0000 7f1f4bebb740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563465369370 0x563465375140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:35.416 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.409+0000 7f1f4bebb740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56346536c340 con 0x5634653642e0 2024-09-12T02:04:35.416 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.409+0000 7f1f4bebb740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x563465250290 con 0x5634653639a0 2024-09-12T02:04:35.416 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.409+0000 7f1f4bebb740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x56346536c4c0 con 0x563465369370 2024-09-12T02:04:35.416 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.409+0000 7f1f4bebb740 10 client.0 _ll_register_callbacks cb 0x563465281140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:04:35.417 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.409+0000 7f1f4259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634653642e0 0x56346536e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:35.417 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.409+0000 7f1f4259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634653642e0 0x56346536e100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35842/0 (socket says 172.21.15.16:35842) 2024-09-12T02:04:35.417 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.409+0000 7f1f4259c640 1 -- 172.21.15.16:0/1769044377 learned_addr learned my addr 172.21.15.16:0/1769044377 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:04:35.417 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.409+0000 7f1f4359e640 1 --2- 172.21.15.16:0/1769044377 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563465369370 0x563465375140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:35.417 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.409+0000 7f1f42d9d640 1 --2- 172.21.15.16:0/1769044377 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5634653639a0 0x56346536dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:35.417 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.409+0000 7f1f4259c640 1 -- 172.21.15.16:0/1769044377 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563465369370 msgr2=0x563465375140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:35.417 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.409+0000 7f1f4259c640 1 --2- 172.21.15.16:0/1769044377 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563465369370 0x563465375140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:35.417 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.409+0000 7f1f4259c640 1 -- 172.21.15.16:0/1769044377 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5634653639a0 msgr2=0x56346536dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:35.417 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.409+0000 7f1f4259c640 1 --2- 172.21.15.16:0/1769044377 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5634653639a0 0x56346536dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:35.417 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.409+0000 7f1f4259c640 1 -- 172.21.15.16:0/1769044377 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563464fda210 con 0x5634653642e0 2024-09-12T02:04:35.417 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.409+0000 7f1f42d9d640 1 --2- 172.21.15.16:0/1769044377 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5634653639a0 0x56346536dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:04:35.417 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.409+0000 7f1f4259c640 1 --2- 172.21.15.16:0/1769044377 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634653642e0 0x56346536e100 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f1f38000c00 tx=0x7f1f38004ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b77b3ce519d136e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:04:35.418 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.409+0000 7f1f26ff5640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:04:35.418 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.410+0000 7f1f26ff5640 1 -- 172.21.15.16:0/1769044377 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f3802a060 con 0x5634653642e0 2024-09-12T02:04:35.418 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.410+0000 7f1f26ff5640 1 -- 172.21.15.16:0/1769044377 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f1f38005210 con 0x5634653642e0 2024-09-12T02:04:35.418 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.410+0000 7f1f4bebb740 1 -- 172.21.15.16:0/1769044377 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f38004c80 con 0x5634653642e0 2024-09-12T02:04:35.418 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.410+0000 7f1f26ff5640 1 -- 172.21.15.16:0/1769044377 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1f380046c0 con 0x5634653642e0 2024-09-12T02:04:35.418 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.410+0000 7f1f4bebb740 10 client.16197 fetch_fsmap learned FSMap version 25 2024-09-12T02:04:35.418 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.410+0000 7f1f4bebb740 1 -- 172.21.15.16:0/1769044377 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5634650a6620 con 0x5634653642e0 2024-09-12T02:04:35.418 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.410+0000 7f1f26ff5640 1 -- 172.21.15.16:0/1769044377 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f38004c80 con 0x5634653642e0 2024-09-12T02:04:35.418 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.410+0000 7f1f26ff5640 1 -- 172.21.15.16:0/1769044377 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f1f3802f070 con 0x5634653642e0 2024-09-12T02:04:35.418 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.410+0000 7f1f4bebb740 10 client.16197 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:04:35.418 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.410+0000 7f1f4bebb740 10 client.16197 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:04:35.418 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.410+0000 7f1f4bebb740 20 client.16197 populate_metadata read hostname 'smithi016' 2024-09-12T02:04:35.418 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.410+0000 7f1f4bebb740 1 --2- 172.21.15.16:0/1769044377 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x563465379550 0x56346537b940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:35.418 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.410+0000 7f1f4bebb740 4 client.16197 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:04:35.419 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.410+0000 7f1f4bebb740 1 -- 172.21.15.16:0/1769044377 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5634652d4a20 con 0x563465379550 2024-09-12T02:04:35.419 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.411+0000 7f1f42d9d640 1 --2- 172.21.15.16:0/1769044377 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x563465379550 0x56346537b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:35.419 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.411+0000 7f1f42d9d640 1 --2- 172.21.15.16:0/1769044377 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x563465379550 0x56346537b940 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b519eba89d2a544 server_cookie=68ff4887fbea4342 in_seq=0 out_seq=0 2024-09-12T02:04:35.419 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.411+0000 7f1f26ff5640 10 client.16197 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:04:35.419 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.412+0000 7f1f26ff5640 1 -- 172.21.15.16:0/1769044377 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5634652d4a20 con 0x563465379550 2024-09-12T02:04:35.419 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.412+0000 7f1f26ff5640 10 client.16197 handle_command_reply: tid=0 2024-09-12T02:04:35.419 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.412+0000 7f1f4bebb740 1 client.16197 shutdown 2024-09-12T02:04:35.419 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.412+0000 7f1f4bebb740 1 -- 172.21.15.16:0/1769044377 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634653642e0 msgr2=0x56346536e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:35.419 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.412+0000 7f1f4bebb740 1 --2- 172.21.15.16:0/1769044377 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634653642e0 0x56346536e100 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f1f38000c00 tx=0x7f1f38004ec0 comp rx=0 tx=0).stop 2024-09-12T02:04:35.419 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.412+0000 7f1f4bebb740 1 -- 172.21.15.16:0/1769044377 shutdown_connections 2024-09-12T02:04:35.419 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.412+0000 7f1f4bebb740 1 --2- 172.21.15.16:0/1769044377 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x563465379550 0x56346537b940 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:35.419 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.412+0000 7f1f4bebb740 1 --2- 172.21.15.16:0/1769044377 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563465369370 0x563465375140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:35.419 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.412+0000 7f1f4bebb740 1 --2- 172.21.15.16:0/1769044377 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634653642e0 0x56346536e100 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:35.419 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.412+0000 7f1f4bebb740 1 --2- 172.21.15.16:0/1769044377 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5634653639a0 0x56346536dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:35.420 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.412+0000 7f1f26ff5640 0 client.16197 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:04:35.420 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.412+0000 7f1f4bebb740 1 -- 172.21.15.16:0/1769044377 >> 172.21.15.16:0/1769044377 conn(0x56346535f010 msgr2=0x563465360820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:04:35.420 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.412+0000 7f1f4bebb740 1 -- 172.21.15.16:0/1769044377 shutdown_connections 2024-09-12T02:04:35.420 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.412+0000 7f1f4bebb740 1 -- 172.21.15.16:0/1769044377 wait complete. 2024-09-12T02:04:35.420 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.412+0000 7f1f4bebb740 20 client.16197 trim_cache size 0 max 16384 2024-09-12T02:04:35.431 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.427+0000 7f1f4bebb740 1 Processor -- start 2024-09-12T02:04:35.432 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.427+0000 7f1f4bebb740 1 -- start start 2024-09-12T02:04:35.432 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.427+0000 7f1f4bebb740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5634653639a0 0x56346535e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:35.432 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.427+0000 7f1f4bebb740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5634653642e0 0x563465375100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:35.432 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.427+0000 7f1f4bebb740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563465369370 0x563465360400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:35.432 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.427+0000 7f1f4bebb740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1f441ad930 con 0x563465369370 2024-09-12T02:04:35.432 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.427+0000 7f1f4bebb740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1f441ab200 con 0x5634653642e0 2024-09-12T02:04:35.432 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.427+0000 7f1f4bebb740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1f441a9050 con 0x5634653639a0 2024-09-12T02:04:35.432 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.427+0000 7f1f4359e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563465369370 0x563465360400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:35.432 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.427+0000 7f1f4359e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563465369370 0x563465360400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35854/0 (socket says 172.21.15.16:35854) 2024-09-12T02:04:35.432 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.427+0000 7f1f4359e640 1 -- 172.21.15.16:0/1990212467 learned_addr learned my addr 172.21.15.16:0/1990212467 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:04:35.432 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.427+0000 7f1f4259c640 1 --2- 172.21.15.16:0/1990212467 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5634653642e0 0x563465375100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:35.432 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.427+0000 7f1f42d9d640 1 --2- 172.21.15.16:0/1990212467 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5634653639a0 0x56346535e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:35.433 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.427+0000 7f1f4359e640 1 -- 172.21.15.16:0/1990212467 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5634653639a0 msgr2=0x56346535e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:35.433 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.427+0000 7f1f4359e640 1 --2- 172.21.15.16:0/1990212467 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5634653639a0 0x56346535e1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:35.433 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.427+0000 7f1f4359e640 1 -- 172.21.15.16:0/1990212467 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5634653642e0 msgr2=0x563465375100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:35.433 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.427+0000 7f1f4359e640 1 --2- 172.21.15.16:0/1990212467 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5634653642e0 0x563465375100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:35.433 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.427+0000 7f1f4359e640 1 -- 172.21.15.16:0/1990212467 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56346537bea0 con 0x563465369370 2024-09-12T02:04:35.433 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.427+0000 7f1f4259c640 1 --2- 172.21.15.16:0/1990212467 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5634653642e0 0x563465375100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:04:35.433 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.427+0000 7f1f4359e640 1 --2- 172.21.15.16:0/1990212467 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563465369370 0x563465360400 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f1f3c00f9a0 tx=0x7f1f3c013030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=833383b3c94b3e82 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:04:35.433 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.428+0000 7f1f28ff9640 1 -- 172.21.15.16:0/1990212467 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f3c01d4f0 con 0x563465369370 2024-09-12T02:04:35.433 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.428+0000 7f1f28ff9640 1 -- 172.21.15.16:0/1990212467 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f3c02c070 con 0x563465369370 2024-09-12T02:04:35.433 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.428+0000 7f1f28ff9640 1 -- 172.21.15.16:0/1990212467 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f3c01ba50 con 0x563465369370 2024-09-12T02:04:35.433 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.428+0000 7f1f4bebb740 1 -- 172.21.15.16:0/1990212467 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563465369370 msgr2=0x563465360400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:35.433 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.428+0000 7f1f4bebb740 1 --2- 172.21.15.16:0/1990212467 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563465369370 0x563465360400 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f1f3c00f9a0 tx=0x7f1f3c013030 comp rx=0 tx=0).stop 2024-09-12T02:04:35.434 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.428+0000 7f1f4bebb740 1 -- 172.21.15.16:0/1990212467 shutdown_connections 2024-09-12T02:04:35.434 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.428+0000 7f1f4bebb740 1 --2- 172.21.15.16:0/1990212467 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563465369370 0x563465360400 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:35.434 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.428+0000 7f1f4bebb740 1 --2- 172.21.15.16:0/1990212467 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5634653642e0 0x563465375100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:35.434 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.428+0000 7f1f4bebb740 1 --2- 172.21.15.16:0/1990212467 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5634653639a0 0x56346535e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:35.434 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.428+0000 7f1f4bebb740 1 -- 172.21.15.16:0/1990212467 >> 172.21.15.16:0/1990212467 conn(0x56346535f010 msgr2=0x56346536cf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:04:35.434 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.428+0000 7f1f4bebb740 1 -- 172.21.15.16:0/1990212467 shutdown_connections 2024-09-12T02:04:35.434 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.428+0000 7f1f4bebb740 1 -- 172.21.15.16:0/1990212467 wait complete. 2024-09-12T02:04:35.434 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.429+0000 7f1f4bebb740 1 -- start start 2024-09-12T02:04:35.434 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.429+0000 7f1f4bebb740 1 Processor -- start 2024-09-12T02:04:35.434 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.429+0000 7f1f4bebb740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634653639a0 0x563465372780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:35.434 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.429+0000 7f1f4bebb740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5634653642e0 0x563465372cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:35.434 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.429+0000 7f1f4bebb740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563465369370 0x563465373cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:35.435 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.429+0000 7f1f4bebb740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1f441a96c0 con 0x5634653639a0 2024-09-12T02:04:35.435 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.429+0000 7f1f4bebb740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1f441aab30 con 0x563465369370 2024-09-12T02:04:35.435 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.429+0000 7f1f4bebb740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1f441a9d90 con 0x5634653642e0 2024-09-12T02:04:35.435 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.429+0000 7f1f4bebb740 10 client.0 _ll_register_callbacks cb 0x7f1f3c009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:04:35.435 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.429+0000 7f1f42d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634653639a0 0x563465372780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:35.435 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.429+0000 7f1f42d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634653639a0 0x563465372780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35862/0 (socket says 172.21.15.16:35862) 2024-09-12T02:04:35.435 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.429+0000 7f1f42d9d640 1 -- 172.21.15.16:0/112632062 learned_addr learned my addr 172.21.15.16:0/112632062 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:04:35.435 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.429+0000 7f1f4259c640 1 --2- 172.21.15.16:0/112632062 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5634653642e0 0x563465372cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:35.436 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.429+0000 7f1f4359e640 1 --2- 172.21.15.16:0/112632062 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563465369370 0x563465373cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:35.436 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.429+0000 7f1f42d9d640 1 -- 172.21.15.16:0/112632062 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5634653642e0 msgr2=0x563465372cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:35.436 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.429+0000 7f1f42d9d640 1 --2- 172.21.15.16:0/112632062 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5634653642e0 0x563465372cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:35.436 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.429+0000 7f1f42d9d640 1 -- 172.21.15.16:0/112632062 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563465369370 msgr2=0x563465373cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:35.436 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.429+0000 7f1f42d9d640 1 --2- 172.21.15.16:0/112632062 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563465369370 0x563465373cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:35.436 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.429+0000 7f1f42d9d640 1 -- 172.21.15.16:0/112632062 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634653742e0 con 0x5634653639a0 2024-09-12T02:04:35.436 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.430+0000 7f1f42d9d640 1 --2- 172.21.15.16:0/112632062 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634653639a0 0x563465372780 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f1f340b8410 tx=0x7f1f340bc070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e503c1051f03c34 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:04:35.436 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.430+0000 7f1f26ff5640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:04:35.436 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.430+0000 7f1f26ff5640 1 -- 172.21.15.16:0/112632062 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f34049040 con 0x5634653639a0 2024-09-12T02:04:35.436 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.430+0000 7f1f26ff5640 1 -- 172.21.15.16:0/112632062 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f1f34003c80 con 0x5634653639a0 2024-09-12T02:04:35.436 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.430+0000 7f1f4bebb740 1 -- 172.21.15.16:0/112632062 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f3c02c070 con 0x5634653639a0 2024-09-12T02:04:35.436 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.430+0000 7f1f26ff5640 1 -- 172.21.15.16:0/112632062 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1f340c4b70 con 0x5634653639a0 2024-09-12T02:04:35.437 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.430+0000 7f1f4bebb740 10 client.16203 fetch_fsmap learned FSMap version 25 2024-09-12T02:04:35.437 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.430+0000 7f1f4bebb740 1 -- 172.21.15.16:0/112632062 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5634653e40a0 con 0x5634653639a0 2024-09-12T02:04:35.437 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.430+0000 7f1f26ff5640 1 -- 172.21.15.16:0/112632062 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f3c02c070 con 0x5634653639a0 2024-09-12T02:04:35.437 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.430+0000 7f1f26ff5640 1 -- 172.21.15.16:0/112632062 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f1f340bc210 con 0x5634653639a0 2024-09-12T02:04:35.437 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.430+0000 7f1f4bebb740 10 client.16203 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:04:35.437 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.430+0000 7f1f4bebb740 10 client.16203 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:04:35.437 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.430+0000 7f1f4bebb740 20 client.16203 populate_metadata read hostname 'smithi016' 2024-09-12T02:04:35.437 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.430+0000 7f1f4bebb740 1 --2- 172.21.15.16:0/112632062 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5634653e4950 0x5634653e6d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:35.437 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.430+0000 7f1f4bebb740 4 client.16203 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:04:35.437 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.430+0000 7f1f4bebb740 1 -- 172.21.15.16:0/112632062 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5634652ced70 con 0x5634653e4950 2024-09-12T02:04:35.437 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.431+0000 7f1f4259c640 1 --2- 172.21.15.16:0/112632062 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5634653e4950 0x5634653e6d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:35.437 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.431+0000 7f1f4259c640 1 --2- 172.21.15.16:0/112632062 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5634653e4950 0x5634653e6d40 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=890142f13fa104bd server_cookie=1f792c909ad576da in_seq=0 out_seq=0 2024-09-12T02:04:35.437 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.431+0000 7f1f26ff5640 10 client.16203 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:04:35.442 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.441+0000 7f1f26ff5640 1 -- 172.21.15.16:0/112632062 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5634652ced70 con 0x5634653e4950 2024-09-12T02:04:35.442 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.441+0000 7f1f26ff5640 10 client.16203 handle_command_reply: tid=0 2024-09-12T02:04:35.442 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.441+0000 7f1f4bebb740 1 client.16203 shutdown 2024-09-12T02:04:35.442 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.442+0000 7f1f4bebb740 1 -- 172.21.15.16:0/112632062 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634653639a0 msgr2=0x563465372780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:35.442 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.442+0000 7f1f4bebb740 1 --2- 172.21.15.16:0/112632062 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634653639a0 0x563465372780 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f1f340b8410 tx=0x7f1f340bc070 comp rx=0 tx=0).stop 2024-09-12T02:04:35.442 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.442+0000 7f1f4bebb740 1 -- 172.21.15.16:0/112632062 shutdown_connections 2024-09-12T02:04:35.442 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.442+0000 7f1f4bebb740 1 --2- 172.21.15.16:0/112632062 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5634653e4950 0x5634653e6d40 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:35.442 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.442+0000 7f1f4bebb740 1 --2- 172.21.15.16:0/112632062 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563465369370 0x563465373cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:35.443 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.442+0000 7f1f26ff5640 0 client.16203 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:04:35.443 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.442+0000 7f1f4bebb740 1 --2- 172.21.15.16:0/112632062 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5634653642e0 0x563465372cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:35.443 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.442+0000 7f1f4bebb740 1 --2- 172.21.15.16:0/112632062 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634653639a0 0x563465372780 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:35.443 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.442+0000 7f1f4bebb740 1 -- 172.21.15.16:0/112632062 >> 172.21.15.16:0/112632062 conn(0x56346535f010 msgr2=0x56346536cf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:04:35.443 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.442+0000 7f1f4bebb740 1 -- 172.21.15.16:0/112632062 shutdown_connections 2024-09-12T02:04:35.443 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.442+0000 7f1f4bebb740 1 -- 172.21.15.16:0/112632062 wait complete. 2024-09-12T02:04:35.443 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.442+0000 7f1f4bebb740 20 client.16203 trim_cache size 0 max 16384 2024-09-12T02:04:35.443 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-09-12T02:04:35.444 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-09-12T02:04:35.444 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-09-12T02:04:35.444 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-09-12T02:04:35.444 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.442+0000 7f1f49123640 1 -- 172.21.15.16:0/3537607614 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f1f2c0a2c30 msgr2=0x7f1f2c0a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:35.444 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.442+0000 7f1f49123640 1 --2- 172.21.15.16:0/3537607614 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f1f2c0a2c30 0x7f1f2c0a50f0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f1f3c009fd0 tx=0x7f1f3c004a40 comp rx=0 tx=0).stop 2024-09-12T02:04:35.444 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.442+0000 7f1f49123640 1 -- 172.21.15.16:0/3537607614 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f44100a60 msgr2=0x7f1f441049f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:35.445 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.442+0000 7f1f49123640 1 --2- 172.21.15.16:0/3537607614 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f44100a60 0x7f1f441049f0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f1f34031610 tx=0x7f1f34004270 comp rx=0 tx=0).stop 2024-09-12T02:04:35.445 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.442+0000 7f1f49123640 1 -- 172.21.15.16:0/3537607614 shutdown_connections 2024-09-12T02:04:35.445 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.442+0000 7f1f49123640 1 --2- 172.21.15.16:0/3537607614 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f1f2c0a2c30 0x7f1f2c0a50f0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:35.445 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.442+0000 7f1f49123640 1 --2- 172.21.15.16:0/3537607614 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1f44102610 0x7f1f44103570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:35.445 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.442+0000 7f1f49123640 1 --2- 172.21.15.16:0/3537607614 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f44101c50 0x7f1f44103030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:35.445 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.442+0000 7f1f49123640 1 --2- 172.21.15.16:0/3537607614 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f44100a60 0x7f1f441049f0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:35.445 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.442+0000 7f1f49123640 1 -- 172.21.15.16:0/3537607614 >> 172.21.15.16:0/3537607614 conn(0x7f1f440fc1e0 msgr2=0x7f1f440fdcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:04:35.445 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.443+0000 7f1f49123640 1 -- 172.21.15.16:0/3537607614 shutdown_connections 2024-09-12T02:04:35.445 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:35.443+0000 7f1f49123640 1 -- 172.21.15.16:0/3537607614 wait complete. 2024-09-12T02:04:36.168 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:04:35 smithi033 ceph-mon[35332]: pgmap v415: 97 pgs: 97 active+clean; 1.5 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 2.7 KiB/s wr, 10 op/s 2024-09-12T02:04:36.183 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:04:35 smithi016 ceph-mon[30947]: pgmap v415: 97 pgs: 97 active+clean; 1.5 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 2.7 KiB/s wr, 10 op/s 2024-09-12T02:04:36.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:04:35 smithi184 ceph-mon[35593]: pgmap v415: 97 pgs: 97 active+clean; 1.5 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 2.7 KiB/s wr, 10 op/s 2024-09-12T02:04:38.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:04:37 smithi033 ceph-mon[35332]: pgmap v416: 97 pgs: 97 active+clean; 1.5 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1023 B/s wr, 9 op/s 2024-09-12T02:04:38.183 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:04:37 smithi016 ceph-mon[30947]: pgmap v416: 97 pgs: 97 active+clean; 1.5 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1023 B/s wr, 9 op/s 2024-09-12T02:04:38.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:04:37 smithi184 ceph-mon[35593]: pgmap v416: 97 pgs: 97 active+clean; 1.5 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1023 B/s wr, 9 op/s 2024-09-12T02:04:39.472 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:04:39.744 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:04:40.164 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:04:39 smithi016 ceph-mon[30947]: pgmap v417: 97 pgs: 97 active+clean; 1.5 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 597 B/s wr, 5 op/s 2024-09-12T02:04:40.168 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:04:39 smithi033 ceph-mon[35332]: pgmap v417: 97 pgs: 97 active+clean; 1.5 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 597 B/s wr, 5 op/s 2024-09-12T02:04:40.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:40.163+0000 7f9931760640 1 -- 172.21.15.16:0/4104193567 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f992c102ef0 msgr2=0x7f992c10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:40.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:40.163+0000 7f9931760640 1 --2- 172.21.15.16:0/4104193567 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f992c102ef0 0x7f992c10da40 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f9924009900 tx=0x7f992402a480 comp rx=0 tx=0).stop 2024-09-12T02:04:40.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:40.164+0000 7f9931760640 1 -- 172.21.15.16:0/4104193567 shutdown_connections 2024-09-12T02:04:40.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:40.164+0000 7f9931760640 1 --2- 172.21.15.16:0/4104193567 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f992c102ef0 0x7f992c10da40 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:40.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:40.164+0000 7f9931760640 1 --2- 172.21.15.16:0/4104193567 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f992c102570 0x7f992c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:40.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:40.164+0000 7f9931760640 1 --2- 172.21.15.16:0/4104193567 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f992c100e20 0x7f992c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:40.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:40.164+0000 7f9931760640 1 -- 172.21.15.16:0/4104193567 >> 172.21.15.16:0/4104193567 conn(0x7f992c0fc7d0 msgr2=0x7f992c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:04:40.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:40.164+0000 7f9931760640 1 -- 172.21.15.16:0/4104193567 shutdown_connections 2024-09-12T02:04:40.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:40.164+0000 7f9931760640 1 -- 172.21.15.16:0/4104193567 wait complete. 2024-09-12T02:04:40.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:40.165+0000 7f9931760640 1 Processor -- start 2024-09-12T02:04:40.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:40.165+0000 7f9931760640 1 -- start start 2024-09-12T02:04:40.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:40.165+0000 7f9931760640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f992c100e20 0x7f992c198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:40.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:40.165+0000 7f9931760640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f992c102570 0x7f992c1989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:40.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:40.165+0000 7f9931760640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f992c102ef0 0x7f992c19cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:40.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:40.165+0000 7f9931760640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f992c1120a0 con 0x7f992c100e20 2024-09-12T02:04:40.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:40.165+0000 7f9931760640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f992c111f20 con 0x7f992c102ef0 2024-09-12T02:04:40.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:40.165+0000 7f9931760640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f992c112220 con 0x7f992c102570 2024-09-12T02:04:40.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:40.165+0000 7f992affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f992c100e20 0x7f992c198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:40.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:40.165+0000 7f992a7fc640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f992c102570 0x7f992c1989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:40.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:40.166+0000 7f992affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f992c100e20 0x7f992c198460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35874/0 (socket says 172.21.15.16:35874) 2024-09-12T02:04:40.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:40.166+0000 7f992affd640 1 -- 172.21.15.16:0/1119690261 learned_addr learned my addr 172.21.15.16:0/1119690261 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:04:40.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:40.166+0000 7f992b7fe640 1 --2- 172.21.15.16:0/1119690261 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f992c102ef0 0x7f992c19cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:40.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:40.166+0000 7f992affd640 1 -- 172.21.15.16:0/1119690261 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f992c102570 msgr2=0x7f992c1989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:40.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:40.166+0000 7f992affd640 1 --2- 172.21.15.16:0/1119690261 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f992c102570 0x7f992c1989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:40.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:40.166+0000 7f992affd640 1 -- 172.21.15.16:0/1119690261 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f992c102ef0 msgr2=0x7f992c19cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:40.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:40.166+0000 7f992affd640 1 --2- 172.21.15.16:0/1119690261 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f992c102ef0 0x7f992c19cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:40.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:40.166+0000 7f992affd640 1 -- 172.21.15.16:0/1119690261 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f992c19d410 con 0x7f992c100e20 2024-09-12T02:04:40.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:40.166+0000 7f992affd640 1 --2- 172.21.15.16:0/1119690261 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f992c100e20 0x7f992c198460 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f991c00eab0 tx=0x7f991c00ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:04:40.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:40.166+0000 7f9923fff640 1 -- 172.21.15.16:0/1119690261 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f991c00ce40 con 0x7f992c100e20 2024-09-12T02:04:40.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:40.166+0000 7f9931760640 1 -- 172.21.15.16:0/1119690261 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f992c19d700 con 0x7f992c100e20 2024-09-12T02:04:40.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:40.166+0000 7f9923fff640 1 -- 172.21.15.16:0/1119690261 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f991c011950 con 0x7f992c100e20 2024-09-12T02:04:40.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:40.166+0000 7f9923fff640 1 -- 172.21.15.16:0/1119690261 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f991c010ae0 con 0x7f992c100e20 2024-09-12T02:04:40.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:40.166+0000 7f9931760640 1 -- 172.21.15.16:0/1119690261 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f992c1a4fe0 con 0x7f992c100e20 2024-09-12T02:04:40.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:40.167+0000 7f9931760640 1 -- 172.21.15.16:0/1119690261 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f992c101200 con 0x7f992c100e20 2024-09-12T02:04:40.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:40.168+0000 7f9923fff640 1 -- 172.21.15.16:0/1119690261 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f991c011b00 con 0x7f992c100e20 2024-09-12T02:04:40.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:40.169+0000 7f9923fff640 1 --2- 172.21.15.16:0/1119690261 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f99180a2b80 0x7f99180a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:40.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:40.169+0000 7f992a7fc640 1 --2- 172.21.15.16:0/1119690261 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f99180a2b80 0x7f99180a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:40.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:40.169+0000 7f9923fff640 1 -- 172.21.15.16:0/1119690261 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f991c0cf5b0 con 0x7f992c100e20 2024-09-12T02:04:40.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:40.170+0000 7f992a7fc640 1 --2- 172.21.15.16:0/1119690261 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f99180a2b80 0x7f99180a5040 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f992c0fec10 tx=0x7f9914005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:04:40.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:40.172+0000 7f9923fff640 1 -- 172.21.15.16:0/1119690261 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f991c016030 con 0x7f992c100e20 2024-09-12T02:04:40.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:04:39 smithi184 ceph-mon[35593]: pgmap v417: 97 pgs: 97 active+clean; 1.5 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 597 B/s wr, 5 op/s 2024-09-12T02:04:40.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:40.384+0000 7f9931760640 1 -- 172.21.15.16:0/1119690261 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f992c1997c0 con 0x7f992c100e20 2024-09-12T02:04:40.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:40.385+0000 7f9923fff640 1 -- 172.21.15.16:0/1119690261 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11911 (secure 0 0 0) 0x7f992c1997c0 con 0x7f992c100e20 2024-09-12T02:04:40.385 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T02:04:40.386 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":25,"btime":"2024-09-12T02:01:37:192021+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3279123279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3279123279},{"type":"v1","addr":"172.21.15.16:6835","nonce":3279123279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4119452382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4119452382},{"type":"v1","addr":"172.21.15.16:6837","nonce":4119452382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/3263823165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3263823165},{"type":"v1","addr":"172.21.15.16:6839","nonce":3263823165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/1510290732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":1510290732},{"type":"v1","addr":"172.21.15.16:6841","nonce":1510290732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/279674603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":279674603},{"type":"v1","addr":"172.21.15.33:6833","nonce":279674603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3555728028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3555728028},{"type":"v1","addr":"172.21.15.33:6835","nonce":3555728028}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T01:59:56.117608+0000","modified":"2024-09-12T02:01:36.294760+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24482,"mds_1":24481,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6833/1563374106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1563374106},{"type":"v1","addr":"172.21.15.184:6833","nonce":1563374106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6837/1892833868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1892833868},{"type":"v1","addr":"172.21.15.33:6837","nonce":1892833868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/1457679686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1457679686},{"type":"v1","addr":"172.21.15.184:6835","nonce":1457679686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.184:6837/1793393087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":1793393087},{"type":"v1","addr":"172.21.15.184:6837","nonce":1793393087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.184:6839/4203347941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":4203347941},{"type":"v1","addr":"172.21.15.184:6839","nonce":4203347941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.33:6839/4136670204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":4136670204},{"type":"v1","addr":"172.21.15.33:6839","nonce":4136670204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24481,24482,24469]},"id":1}]} 2024-09-12T02:04:40.386 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 25 2024-09-12T02:04:40.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:40.387+0000 7f9931760640 1 -- 172.21.15.16:0/1119690261 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f99180a2b80 msgr2=0x7f99180a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:40.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:40.387+0000 7f9931760640 1 --2- 172.21.15.16:0/1119690261 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f99180a2b80 0x7f99180a5040 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f992c0fec10 tx=0x7f9914005f90 comp rx=0 tx=0).stop 2024-09-12T02:04:40.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:40.387+0000 7f9931760640 1 -- 172.21.15.16:0/1119690261 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f992c100e20 msgr2=0x7f992c198460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:40.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:40.387+0000 7f9931760640 1 --2- 172.21.15.16:0/1119690261 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f992c100e20 0x7f992c198460 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f991c00eab0 tx=0x7f991c00ef70 comp rx=0 tx=0).stop 2024-09-12T02:04:40.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:40.388+0000 7f9931760640 1 -- 172.21.15.16:0/1119690261 shutdown_connections 2024-09-12T02:04:40.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:40.388+0000 7f9931760640 1 --2- 172.21.15.16:0/1119690261 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f99180a2b80 0x7f99180a5040 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:40.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:40.388+0000 7f9931760640 1 --2- 172.21.15.16:0/1119690261 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f992c102ef0 0x7f992c19cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:40.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:40.388+0000 7f9931760640 1 --2- 172.21.15.16:0/1119690261 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f992c102570 0x7f992c1989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:40.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:40.388+0000 7f9931760640 1 --2- 172.21.15.16:0/1119690261 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f992c100e20 0x7f992c198460 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:40.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:40.388+0000 7f9931760640 1 -- 172.21.15.16:0/1119690261 >> 172.21.15.16:0/1119690261 conn(0x7f992c0fc7d0 msgr2=0x7f992c1068a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:04:40.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:40.388+0000 7f9931760640 1 -- 172.21.15.16:0/1119690261 shutdown_connections 2024-09-12T02:04:40.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:40.388+0000 7f9931760640 1 -- 172.21.15.16:0/1119690261 wait complete. 2024-09-12T02:04:41.053 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:04:40 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/1119690261' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:04:41.085 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24482 quiesce db --set-id 146e8aed --release --await' 2024-09-12T02:04:41.086 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24482 quiesce db --set-id 146e8aed --release --await 2024-09-12T02:04:41.168 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:04:40 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/1119690261' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:04:41.346 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:04:41.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:04:40 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/1119690261' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:04:41.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.819+0000 7f4f871eb640 1 -- 172.21.15.16:0/1648103628 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f80102790 msgr2=0x7f4f801067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:41.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.819+0000 7f4f871eb640 1 --2- 172.21.15.16:0/1648103628 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f80102790 0x7f4f801067e0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f4f68009900 tx=0x7f4f6802a410 comp rx=0 tx=0).stop 2024-09-12T02:04:41.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.820+0000 7f4f871eb640 1 -- 172.21.15.16:0/1648103628 shutdown_connections 2024-09-12T02:04:41.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.820+0000 7f4f871eb640 1 --2- 172.21.15.16:0/1648103628 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4f80106f10 0x7f4f8010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:41.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.820+0000 7f4f871eb640 1 --2- 172.21.15.16:0/1648103628 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f80102790 0x7f4f801067e0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:41.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.820+0000 7f4f871eb640 1 --2- 172.21.15.16:0/1648103628 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4f80101de0 0x7f4f801021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:41.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.820+0000 7f4f871eb640 1 -- 172.21.15.16:0/1648103628 >> 172.21.15.16:0/1648103628 conn(0x7f4f800fd8c0 msgr2=0x7f4f800ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:04:41.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.820+0000 7f4f871eb640 1 -- 172.21.15.16:0/1648103628 shutdown_connections 2024-09-12T02:04:41.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.820+0000 7f4f871eb640 1 -- 172.21.15.16:0/1648103628 wait complete. 2024-09-12T02:04:41.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.820+0000 7f4f871eb640 1 Processor -- start 2024-09-12T02:04:41.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.820+0000 7f4f871eb640 1 -- start start 2024-09-12T02:04:41.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.820+0000 7f4f871eb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f80101de0 0x7f4f8019a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:41.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.821+0000 7f4f871eb640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4f80102790 0x7f4f8019ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:41.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.821+0000 7f4f871eb640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4f80106f10 0x7f4f8019eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:41.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.821+0000 7f4f871eb640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4f80111f70 con 0x7f4f80101de0 2024-09-12T02:04:41.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.821+0000 7f4f871eb640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4f80111df0 con 0x7f4f80102790 2024-09-12T02:04:41.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.821+0000 7f4f871eb640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f4f801120f0 con 0x7f4f80106f10 2024-09-12T02:04:41.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.821+0000 7f4f84f87640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f80101de0 0x7f4f8019a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:41.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.821+0000 7f4f84f87640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f80101de0 0x7f4f8019a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41808/0 (socket says 172.21.15.16:41808) 2024-09-12T02:04:41.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.821+0000 7f4f84f87640 1 -- 172.21.15.16:0/2424858573 learned_addr learned my addr 172.21.15.16:0/2424858573 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:04:41.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.821+0000 7f4f85788640 1 --2- 172.21.15.16:0/2424858573 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4f80106f10 0x7f4f8019eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:41.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.821+0000 7f4f7ffff640 1 --2- 172.21.15.16:0/2424858573 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4f80102790 0x7f4f8019ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:41.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.821+0000 7f4f84f87640 1 -- 172.21.15.16:0/2424858573 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4f80106f10 msgr2=0x7f4f8019eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:41.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.821+0000 7f4f84f87640 1 --2- 172.21.15.16:0/2424858573 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4f80106f10 0x7f4f8019eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:41.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.821+0000 7f4f84f87640 1 -- 172.21.15.16:0/2424858573 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4f80102790 msgr2=0x7f4f8019ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:41.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.821+0000 7f4f84f87640 1 --2- 172.21.15.16:0/2424858573 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4f80102790 0x7f4f8019ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:41.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.821+0000 7f4f84f87640 1 -- 172.21.15.16:0/2424858573 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f8019f5d0 con 0x7f4f80101de0 2024-09-12T02:04:41.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.821+0000 7f4f84f87640 1 --2- 172.21.15.16:0/2424858573 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f80101de0 0x7f4f8019a620 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f4f7000d770 tx=0x7f4f7000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:04:41.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.821+0000 7f4f7dffb640 1 -- 172.21.15.16:0/2424858573 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f70017070 con 0x7f4f80101de0 2024-09-12T02:04:41.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.821+0000 7f4f871eb640 1 -- 172.21.15.16:0/2424858573 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f8019f8c0 con 0x7f4f80101de0 2024-09-12T02:04:41.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.821+0000 7f4f7dffb640 1 -- 172.21.15.16:0/2424858573 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f70005500 con 0x7f4f80101de0 2024-09-12T02:04:41.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.821+0000 7f4f7dffb640 1 -- 172.21.15.16:0/2424858573 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f70004e00 con 0x7f4f80101de0 2024-09-12T02:04:41.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.822+0000 7f4f871eb640 1 -- 172.21.15.16:0/2424858573 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f801a71a0 con 0x7f4f80101de0 2024-09-12T02:04:41.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.824+0000 7f4f7dffb640 1 -- 172.21.15.16:0/2424858573 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f4f70004fa0 con 0x7f4f80101de0 2024-09-12T02:04:41.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.824+0000 7f4f895c4740 1 Processor -- start 2024-09-12T02:04:41.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.824+0000 7f4f895c4740 1 -- start start 2024-09-12T02:04:41.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.824+0000 7f4f895c4740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d5a34c2060 0x55d5a34c2440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:41.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.824+0000 7f4f895c4740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d5a34c2980 0x55d5a34ccd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:41.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.824+0000 7f4f7dffb640 1 --2- 172.21.15.16:0/2424858573 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f4f6c0a2b80 0x7f4f6c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:41.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.825+0000 7f4f895c4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d5a34cd240 0x55d5a34cf630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:41.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.825+0000 7f4f895c4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55d5a3427620 con 0x55d5a34cd240 2024-09-12T02:04:41.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.825+0000 7f4f895c4740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55d5a33cee50 con 0x55d5a34c2060 2024-09-12T02:04:41.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.825+0000 7f4f895c4740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55d5a34235e0 con 0x55d5a34c2980 2024-09-12T02:04:41.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.825+0000 7f4f85788640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d5a34cd240 0x55d5a34cf630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:41.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.825+0000 7f4f7ffff640 1 --2- 172.21.15.16:0/2424858573 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f4f6c0a2b80 0x7f4f6c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:41.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.825+0000 7f4f85788640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d5a34cd240 0x55d5a34cf630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41812/0 (socket says 172.21.15.16:41812) 2024-09-12T02:04:41.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.825+0000 7f4f85788640 1 -- 172.21.15.16:0/3017162776 learned_addr learned my addr 172.21.15.16:0/3017162776 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:04:41.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.825+0000 7f4f85788640 1 --2- 172.21.15.16:0/3017162776 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d5a34c2060 0x55d5a34c2440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:41.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.825+0000 7f4f7dffb640 1 -- 172.21.15.16:0/2424858573 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f4f700ce970 con 0x7f4f80101de0 2024-09-12T02:04:41.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.825+0000 7f4f84f87640 1 --2- 172.21.15.16:0/3017162776 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d5a34c2980 0x55d5a34ccd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:41.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.825+0000 7f4f84f87640 1 -- 172.21.15.16:0/3017162776 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d5a34c2060 msgr2=0x55d5a34c2440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:41.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.825+0000 7f4f84f87640 1 --2- 172.21.15.16:0/3017162776 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d5a34c2060 0x55d5a34c2440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:41.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.825+0000 7f4f84f87640 1 -- 172.21.15.16:0/3017162776 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d5a34cd240 msgr2=0x55d5a34cf630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:41.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.825+0000 7f4f84f87640 1 --2- 172.21.15.16:0/3017162776 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d5a34cd240 0x55d5a34cf630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:41.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.825+0000 7f4f84f87640 1 -- 172.21.15.16:0/3017162776 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5a33bfd40 con 0x55d5a34c2980 2024-09-12T02:04:41.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.825+0000 7f4f85788640 1 --2- 172.21.15.16:0/3017162776 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d5a34cd240 0x55d5a34cf630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:04:41.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.825+0000 7f4f7ffff640 1 --2- 172.21.15.16:0/2424858573 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f4f6c0a2b80 0x7f4f6c0a5040 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f4f68009230 tx=0x7f4f680058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:04:41.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.825+0000 7f4f84f87640 1 --2- 172.21.15.16:0/3017162776 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d5a34c2980 0x55d5a34ccd00 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4f700cc560 tx=0x7f4f70009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=82794b1d4881dec2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:04:41.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.826+0000 7f4f76ffd640 1 -- 172.21.15.16:0/3017162776 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f70020040 con 0x55d5a34c2980 2024-09-12T02:04:41.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.826+0000 7f4f76ffd640 1 -- 172.21.15.16:0/3017162776 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f7009f620 con 0x55d5a34c2980 2024-09-12T02:04:41.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.826+0000 7f4f76ffd640 1 -- 172.21.15.16:0/3017162776 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f700ab0e0 con 0x55d5a34c2980 2024-09-12T02:04:41.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.826+0000 7f4f895c4740 1 -- 172.21.15.16:0/3017162776 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d5a34c2980 msgr2=0x55d5a34ccd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:41.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.826+0000 7f4f895c4740 1 --2- 172.21.15.16:0/3017162776 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d5a34c2980 0x55d5a34ccd00 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4f700cc560 tx=0x7f4f70009c30 comp rx=0 tx=0).stop 2024-09-12T02:04:41.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.826+0000 7f4f895c4740 1 -- 172.21.15.16:0/3017162776 shutdown_connections 2024-09-12T02:04:41.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.826+0000 7f4f895c4740 1 --2- 172.21.15.16:0/3017162776 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d5a34cd240 0x55d5a34cf630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:41.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.826+0000 7f4f895c4740 1 --2- 172.21.15.16:0/3017162776 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d5a34c2980 0x55d5a34ccd00 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:41.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.826+0000 7f4f895c4740 1 --2- 172.21.15.16:0/3017162776 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d5a34c2060 0x55d5a34c2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:41.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.826+0000 7f4f895c4740 1 -- 172.21.15.16:0/3017162776 >> 172.21.15.16:0/3017162776 conn(0x55d5a34be5f0 msgr2=0x55d5a34be9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:04:41.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.826+0000 7f4f895c4740 1 -- 172.21.15.16:0/3017162776 shutdown_connections 2024-09-12T02:04:41.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.827+0000 7f4f895c4740 1 -- 172.21.15.16:0/3017162776 wait complete. 2024-09-12T02:04:41.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.827+0000 7f4f895c4740 1 -- start start 2024-09-12T02:04:41.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.827+0000 7f4f895c4740 1 Processor -- start 2024-09-12T02:04:41.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.827+0000 7f4f895c4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d5a34c2060 0x55d5a34ca180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:41.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.827+0000 7f4f895c4740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d5a34c2980 0x55d5a34ca6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:41.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.827+0000 7f4f895c4740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d5a34cd240 0x55d5a34cac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:41.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.827+0000 7f4f895c4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55d5a34cfe10 con 0x55d5a34c2060 2024-09-12T02:04:41.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.827+0000 7f4f895c4740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55d5a33aff10 con 0x55d5a34cd240 2024-09-12T02:04:41.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.827+0000 7f4f895c4740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55d5a34cff90 con 0x55d5a34c2980 2024-09-12T02:04:41.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.827+0000 7f4f895c4740 10 client.0 _ll_register_callbacks cb 0x55d5a33d3db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:04:41.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.828+0000 7f4f84f87640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d5a34c2060 0x55d5a34ca180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:41.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.828+0000 7f4f84f87640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d5a34c2060 0x55d5a34ca180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41822/0 (socket says 172.21.15.16:41822) 2024-09-12T02:04:41.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.828+0000 7f4f84f87640 1 -- 172.21.15.16:0/3152668666 learned_addr learned my addr 172.21.15.16:0/3152668666 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:04:41.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.828+0000 7f4f85788640 1 --2- 172.21.15.16:0/3152668666 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d5a34cd240 0x55d5a34cac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:41.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.828+0000 7f4f7ffff640 1 --2- 172.21.15.16:0/3152668666 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d5a34c2980 0x55d5a34ca6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:41.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.828+0000 7f4f84f87640 1 -- 172.21.15.16:0/3152668666 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d5a34c2980 msgr2=0x55d5a34ca6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:41.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.828+0000 7f4f84f87640 1 --2- 172.21.15.16:0/3152668666 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d5a34c2980 0x55d5a34ca6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:41.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.828+0000 7f4f84f87640 1 -- 172.21.15.16:0/3152668666 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d5a34cd240 msgr2=0x55d5a34cac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:41.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.828+0000 7f4f84f87640 1 --2- 172.21.15.16:0/3152668666 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d5a34cd240 0x55d5a34cac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:41.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.828+0000 7f4f84f87640 1 -- 172.21.15.16:0/3152668666 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d5a3136580 con 0x55d5a34c2060 2024-09-12T02:04:41.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.828+0000 7f4f85788640 1 --2- 172.21.15.16:0/3152668666 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d5a34cd240 0x55d5a34cac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:04:41.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.828+0000 7f4f84f87640 1 --2- 172.21.15.16:0/3152668666 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d5a34c2060 0x55d5a34ca180 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f4f7000d740 tx=0x7f4f700cec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afa224643d807240 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:04:41.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.828+0000 7f4f5ffff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:04:41.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.828+0000 7f4f5ffff640 1 -- 172.21.15.16:0/3152668666 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f700b9060 con 0x55d5a34c2060 2024-09-12T02:04:41.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.828+0000 7f4f5ffff640 1 -- 172.21.15.16:0/3152668666 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f4f70099620 con 0x55d5a34c2060 2024-09-12T02:04:41.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.828+0000 7f4f895c4740 1 -- 172.21.15.16:0/3152668666 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f7009f620 con 0x55d5a34c2060 2024-09-12T02:04:41.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.829+0000 7f4f5ffff640 1 -- 172.21.15.16:0/3152668666 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4f700a7020 con 0x55d5a34c2060 2024-09-12T02:04:41.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.829+0000 7f4f895c4740 10 client.16221 fetch_fsmap learned FSMap version 25 2024-09-12T02:04:41.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.829+0000 7f4f895c4740 1 -- 172.21.15.16:0/3152668666 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d5a3200960 con 0x55d5a34c2060 2024-09-12T02:04:41.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.829+0000 7f4f5ffff640 1 -- 172.21.15.16:0/3152668666 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f700a6030 con 0x55d5a34c2060 2024-09-12T02:04:41.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.829+0000 7f4f5ffff640 1 -- 172.21.15.16:0/3152668666 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f4f7009f170 con 0x55d5a34c2060 2024-09-12T02:04:41.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.829+0000 7f4f895c4740 10 client.16221 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:04:41.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.829+0000 7f4f895c4740 10 client.16221 resolve_mds: validated gid 24482 aka daemon mds.b 2024-09-12T02:04:41.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.829+0000 7f4f895c4740 20 client.16221 populate_metadata read hostname 'smithi016' 2024-09-12T02:04:41.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.829+0000 7f4f895c4740 1 --2- 172.21.15.16:0/3152668666 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55d5a34dcc90 0x55d5a34df080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:41.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.829+0000 7f4f895c4740 4 client.16221 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:04:41.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.829+0000 7f4f895c4740 1 -- 172.21.15.16:0/3152668666 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d5a3434b90 con 0x55d5a34dcc90 2024-09-12T02:04:41.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.829+0000 7f4f7ffff640 1 --2- 172.21.15.16:0/3152668666 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55d5a34dcc90 0x55d5a34df080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:41.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.830+0000 7f4f7ffff640 1 --2- 172.21.15.16:0/3152668666 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55d5a34dcc90 0x55d5a34df080 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97bcf3018cfa6a1f server_cookie=db731479d2c1cadb in_seq=0 out_seq=0 2024-09-12T02:04:41.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.830+0000 7f4f5ffff640 10 client.16221 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:04:41.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.830+0000 7f4f5ffff640 1 -- 172.21.15.16:0/3152668666 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d5a3434b90 con 0x55d5a34dcc90 2024-09-12T02:04:41.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.830+0000 7f4f5ffff640 10 client.16221 handle_command_reply: tid=0 2024-09-12T02:04:41.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.831+0000 7f4f895c4740 1 client.16221 shutdown 2024-09-12T02:04:41.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.831+0000 7f4f895c4740 1 -- 172.21.15.16:0/3152668666 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d5a34c2060 msgr2=0x55d5a34ca180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:41.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.831+0000 7f4f895c4740 1 --2- 172.21.15.16:0/3152668666 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d5a34c2060 0x55d5a34ca180 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f4f7000d740 tx=0x7f4f700cec70 comp rx=0 tx=0).stop 2024-09-12T02:04:41.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.831+0000 7f4f895c4740 1 -- 172.21.15.16:0/3152668666 shutdown_connections 2024-09-12T02:04:41.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.831+0000 7f4f895c4740 1 --2- 172.21.15.16:0/3152668666 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55d5a34dcc90 0x55d5a34df080 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:41.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.831+0000 7f4f895c4740 1 --2- 172.21.15.16:0/3152668666 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d5a34cd240 0x55d5a34cac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:41.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.831+0000 7f4f895c4740 1 --2- 172.21.15.16:0/3152668666 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d5a34c2980 0x55d5a34ca6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:41.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.831+0000 7f4f895c4740 1 --2- 172.21.15.16:0/3152668666 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d5a34c2060 0x55d5a34ca180 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:41.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.831+0000 7f4f895c4740 1 -- 172.21.15.16:0/3152668666 >> 172.21.15.16:0/3152668666 conn(0x55d5a34be5f0 msgr2=0x55d5a34c4320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:04:41.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.831+0000 7f4f5ffff640 0 client.16221 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:04:41.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.831+0000 7f4f895c4740 1 -- 172.21.15.16:0/3152668666 shutdown_connections 2024-09-12T02:04:41.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.831+0000 7f4f895c4740 1 -- 172.21.15.16:0/3152668666 wait complete. 2024-09-12T02:04:41.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.831+0000 7f4f895c4740 20 client.16221 trim_cache size 0 max 16384 2024-09-12T02:04:41.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.848+0000 7f4f895c4740 1 Processor -- start 2024-09-12T02:04:41.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.848+0000 7f4f895c4740 1 -- start start 2024-09-12T02:04:41.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.848+0000 7f4f895c4740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d5a34c2060 0x55d5a34c90c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:41.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.848+0000 7f4f895c4740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d5a34c2980 0x55d5a34e1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:41.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.848+0000 7f4f895c4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d5a34cd240 0x55d5a34e2200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:41.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.848+0000 7f4f895c4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4f801b1e10 con 0x55d5a34cd240 2024-09-12T02:04:41.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.848+0000 7f4f895c4740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4f801af6e0 con 0x55d5a34c2060 2024-09-12T02:04:41.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.848+0000 7f4f895c4740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f4f801ad530 con 0x55d5a34c2980 2024-09-12T02:04:41.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.848+0000 7f4f85788640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d5a34cd240 0x55d5a34e2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:41.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.848+0000 7f4f85788640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d5a34cd240 0x55d5a34e2200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41834/0 (socket says 172.21.15.16:41834) 2024-09-12T02:04:41.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.848+0000 7f4f85788640 1 -- 172.21.15.16:0/3893464665 learned_addr learned my addr 172.21.15.16:0/3893464665 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:04:41.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.849+0000 7f4f84f87640 1 --2- 172.21.15.16:0/3893464665 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d5a34c2060 0x55d5a34c90c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:41.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.849+0000 7f4f7ffff640 1 --2- 172.21.15.16:0/3893464665 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d5a34c2980 0x55d5a34e1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:41.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.849+0000 7f4f85788640 1 -- 172.21.15.16:0/3893464665 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d5a34c2980 msgr2=0x55d5a34e1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:41.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.849+0000 7f4f85788640 1 --2- 172.21.15.16:0/3893464665 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d5a34c2980 0x55d5a34e1cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:41.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.849+0000 7f4f85788640 1 -- 172.21.15.16:0/3893464665 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d5a34c2060 msgr2=0x55d5a34c90c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:41.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.849+0000 7f4f85788640 1 --2- 172.21.15.16:0/3893464665 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d5a34c2060 0x55d5a34c90c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:41.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.849+0000 7f4f85788640 1 -- 172.21.15.16:0/3893464665 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5a34be090 con 0x55d5a34cd240 2024-09-12T02:04:41.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.849+0000 7f4f84f87640 1 --2- 172.21.15.16:0/3893464665 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d5a34c2060 0x55d5a34c90c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:04:41.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.849+0000 7f4f85788640 1 --2- 172.21.15.16:0/3893464665 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d5a34cd240 0x55d5a34e2200 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f4f78014c90 tx=0x7f4f780079b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=448a01bd6ec9c62e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:04:41.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.849+0000 7f4f767fc640 1 -- 172.21.15.16:0/3893464665 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f780142d0 con 0x55d5a34cd240 2024-09-12T02:04:41.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.849+0000 7f4f767fc640 1 -- 172.21.15.16:0/3893464665 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f78005020 con 0x55d5a34cd240 2024-09-12T02:04:41.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.849+0000 7f4f767fc640 1 -- 172.21.15.16:0/3893464665 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f780126b0 con 0x55d5a34cd240 2024-09-12T02:04:41.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.850+0000 7f4f895c4740 1 -- 172.21.15.16:0/3893464665 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d5a34cd240 msgr2=0x55d5a34e2200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:41.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.850+0000 7f4f895c4740 1 --2- 172.21.15.16:0/3893464665 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d5a34cd240 0x55d5a34e2200 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f4f78014c90 tx=0x7f4f780079b0 comp rx=0 tx=0).stop 2024-09-12T02:04:41.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.850+0000 7f4f895c4740 1 -- 172.21.15.16:0/3893464665 shutdown_connections 2024-09-12T02:04:41.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.850+0000 7f4f895c4740 1 --2- 172.21.15.16:0/3893464665 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d5a34cd240 0x55d5a34e2200 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:41.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.850+0000 7f4f895c4740 1 --2- 172.21.15.16:0/3893464665 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d5a34c2980 0x55d5a34e1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:41.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.850+0000 7f4f895c4740 1 --2- 172.21.15.16:0/3893464665 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d5a34c2060 0x55d5a34c90c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:41.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.850+0000 7f4f895c4740 1 -- 172.21.15.16:0/3893464665 >> 172.21.15.16:0/3893464665 conn(0x55d5a34be5f0 msgr2=0x55d5a34c42d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:04:41.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.850+0000 7f4f895c4740 1 -- 172.21.15.16:0/3893464665 shutdown_connections 2024-09-12T02:04:41.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.850+0000 7f4f895c4740 1 -- 172.21.15.16:0/3893464665 wait complete. 2024-09-12T02:04:41.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.850+0000 7f4f895c4740 1 -- start start 2024-09-12T02:04:41.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.850+0000 7f4f895c4740 1 Processor -- start 2024-09-12T02:04:41.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.851+0000 7f4f895c4740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d5a34c2060 0x55d5a34dfa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:41.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.851+0000 7f4f895c4740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d5a34c2980 0x55d5a34dff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:41.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.851+0000 7f4f895c4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d5a34cd240 0x55d5a34e0490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:41.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.851+0000 7f4f895c4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4f801adba0 con 0x55d5a34cd240 2024-09-12T02:04:41.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.851+0000 7f4f895c4740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4f801af010 con 0x55d5a34c2060 2024-09-12T02:04:41.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.851+0000 7f4f895c4740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f4f801ae270 con 0x55d5a34c2980 2024-09-12T02:04:41.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.851+0000 7f4f895c4740 10 client.0 _ll_register_callbacks cb 0x7f4f700976b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:04:41.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.851+0000 7f4f85788640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d5a34cd240 0x55d5a34e0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:41.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.851+0000 7f4f85788640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d5a34cd240 0x55d5a34e0490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41840/0 (socket says 172.21.15.16:41840) 2024-09-12T02:04:41.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.851+0000 7f4f85788640 1 -- 172.21.15.16:0/3403843711 learned_addr learned my addr 172.21.15.16:0/3403843711 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:04:41.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.851+0000 7f4f7ffff640 1 --2- 172.21.15.16:0/3403843711 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d5a34c2980 0x55d5a34dff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:41.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.851+0000 7f4f84f87640 1 --2- 172.21.15.16:0/3403843711 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d5a34c2060 0x55d5a34dfa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:41.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.851+0000 7f4f85788640 1 -- 172.21.15.16:0/3403843711 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d5a34c2980 msgr2=0x55d5a34dff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:41.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.851+0000 7f4f85788640 1 --2- 172.21.15.16:0/3403843711 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d5a34c2980 0x55d5a34dff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:41.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.851+0000 7f4f85788640 1 -- 172.21.15.16:0/3403843711 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d5a34c2060 msgr2=0x55d5a34dfa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:41.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.851+0000 7f4f85788640 1 --2- 172.21.15.16:0/3403843711 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d5a34c2060 0x55d5a34dfa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:41.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.851+0000 7f4f85788640 1 -- 172.21.15.16:0/3403843711 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d5a35465d0 con 0x55d5a34cd240 2024-09-12T02:04:41.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.851+0000 7f4f84f87640 1 --2- 172.21.15.16:0/3403843711 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d5a34c2060 0x55d5a34dfa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:04:41.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.851+0000 7f4f85788640 1 --2- 172.21.15.16:0/3403843711 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d5a34cd240 0x55d5a34e0490 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f4f78007330 tx=0x7f4f78005870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ccd4c99618c16a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:04:41.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.851+0000 7f4f5ffff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:04:41.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.852+0000 7f4f5ffff640 1 -- 172.21.15.16:0/3403843711 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f780051d0 con 0x55d5a34cd240 2024-09-12T02:04:41.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.852+0000 7f4f5ffff640 1 -- 172.21.15.16:0/3403843711 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f4f78005370 con 0x55d5a34cd240 2024-09-12T02:04:41.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.852+0000 7f4f895c4740 1 -- 172.21.15.16:0/3403843711 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f78005020 con 0x55d5a34cd240 2024-09-12T02:04:41.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.852+0000 7f4f5ffff640 1 -- 172.21.15.16:0/3403843711 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4f78004a40 con 0x55d5a34cd240 2024-09-12T02:04:41.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.852+0000 7f4f895c4740 10 client.16227 fetch_fsmap learned FSMap version 25 2024-09-12T02:04:41.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.852+0000 7f4f895c4740 1 -- 172.21.15.16:0/3403843711 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d5a3547830 con 0x55d5a34cd240 2024-09-12T02:04:41.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.852+0000 7f4f5ffff640 1 -- 172.21.15.16:0/3403843711 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f78005020 con 0x55d5a34cd240 2024-09-12T02:04:41.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.852+0000 7f4f5ffff640 1 -- 172.21.15.16:0/3403843711 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f4f78004020 con 0x55d5a34cd240 2024-09-12T02:04:41.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.852+0000 7f4f895c4740 10 client.16227 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:04:41.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.852+0000 7f4f895c4740 10 client.16227 resolve_mds: validated gid 24482 aka daemon mds.b 2024-09-12T02:04:41.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.852+0000 7f4f895c4740 20 client.16227 populate_metadata read hostname 'smithi016' 2024-09-12T02:04:41.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.852+0000 7f4f895c4740 1 --2- 172.21.15.16:0/3403843711 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55d5a3547ef0 0x55d5a354a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:41.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.852+0000 7f4f895c4740 4 client.16227 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "146e8aed", "release": true, "await": true}] 2024-09-12T02:04:41.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.852+0000 7f4f895c4740 1 -- 172.21.15.16:0/3403843711 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "quiesce db", "set_id": "146e8aed", "release": true, "await": true}) -- 0x55d5a342f0a0 con 0x55d5a3547ef0 2024-09-12T02:04:41.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.853+0000 7f4f84f87640 1 --2- 172.21.15.16:0/3403843711 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55d5a3547ef0 0x55d5a354a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:41.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.853+0000 7f4f84f87640 1 --2- 172.21.15.16:0/3403843711 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55d5a3547ef0 0x55d5a354a310 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9bf45e6da0f80150 server_cookie=42ebe9e310e20fdc in_seq=0 out_seq=0 2024-09-12T02:04:41.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.853+0000 7f4f5ffff640 10 client.16227 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:04:41.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.855+0000 7f4f5ffff640 1 -- 172.21.15.16:0/3403843711 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55d5a342f0a0 con 0x55d5a3547ef0 2024-09-12T02:04:41.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.855+0000 7f4f5ffff640 10 client.16227 handle_command_reply: tid=0 2024-09-12T02:04:41.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.855+0000 7f4f895c4740 1 client.16227 shutdown 2024-09-12T02:04:41.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.855+0000 7f4f895c4740 1 -- 172.21.15.16:0/3403843711 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d5a34cd240 msgr2=0x55d5a34e0490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:41.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.855+0000 7f4f895c4740 1 --2- 172.21.15.16:0/3403843711 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d5a34cd240 0x55d5a34e0490 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f4f78007330 tx=0x7f4f78005870 comp rx=0 tx=0).stop 2024-09-12T02:04:41.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.855+0000 7f4f895c4740 1 -- 172.21.15.16:0/3403843711 shutdown_connections 2024-09-12T02:04:41.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.855+0000 7f4f895c4740 1 --2- 172.21.15.16:0/3403843711 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55d5a3547ef0 0x55d5a354a310 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:41.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.855+0000 7f4f895c4740 1 --2- 172.21.15.16:0/3403843711 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d5a34cd240 0x55d5a34e0490 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:41.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.856+0000 7f4f895c4740 1 --2- 172.21.15.16:0/3403843711 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d5a34c2980 0x55d5a34dff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:41.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.856+0000 7f4f895c4740 1 --2- 172.21.15.16:0/3403843711 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d5a34c2060 0x55d5a34dfa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:41.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.856+0000 7f4f5ffff640 0 client.16227 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:04:41.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.856+0000 7f4f895c4740 1 -- 172.21.15.16:0/3403843711 >> 172.21.15.16:0/3403843711 conn(0x55d5a34be5f0 msgr2=0x55d5a34bcf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:04:41.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.856+0000 7f4f895c4740 1 -- 172.21.15.16:0/3403843711 shutdown_connections 2024-09-12T02:04:41.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.856+0000 7f4f895c4740 1 -- 172.21.15.16:0/3403843711 wait complete. 2024-09-12T02:04:41.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.856+0000 7f4f895c4740 20 client.16227 trim_cache size 0 max 16384 2024-09-12T02:04:41.861 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T02:04:41.861 INFO:teuthology.orchestra.run.smithi016.stdout: "epoch": 25, 2024-09-12T02:04:41.861 INFO:teuthology.orchestra.run.smithi016.stdout: "leader": 24482, 2024-09-12T02:04:41.861 INFO:teuthology.orchestra.run.smithi016.stdout: "set_version": 12, 2024-09-12T02:04:41.861 INFO:teuthology.orchestra.run.smithi016.stdout: "sets": { 2024-09-12T02:04:41.861 INFO:teuthology.orchestra.run.smithi016.stdout: "146e8aed": { 2024-09-12T02:04:41.861 INFO:teuthology.orchestra.run.smithi016.stdout: "version": 12, 2024-09-12T02:04:41.861 INFO:teuthology.orchestra.run.smithi016.stdout: "age_ref": 0.0, 2024-09-12T02:04:41.861 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-12T02:04:41.861 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RELEASED", 2024-09-12T02:04:41.861 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-12T02:04:41.862 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:04:41.862 INFO:teuthology.orchestra.run.smithi016.stdout: "timeout": 90.0, 2024-09-12T02:04:41.862 INFO:teuthology.orchestra.run.smithi016.stdout: "expiration": 144.7, 2024-09-12T02:04:41.862 INFO:teuthology.orchestra.run.smithi016.stdout: "members": { 2024-09-12T02:04:41.862 INFO:teuthology.orchestra.run.smithi016.stdout: "file:/": { 2024-09-12T02:04:41.862 INFO:teuthology.orchestra.run.smithi016.stdout: "excluded": false, 2024-09-12T02:04:41.862 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-12T02:04:41.862 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RELEASED", 2024-09-12T02:04:41.862 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-12T02:04:41.862 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:04:41.862 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:04:41.862 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:04:41.862 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:04:41.863 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:04:41.863 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T02:04:41.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.856+0000 7f4f871eb640 1 -- 172.21.15.16:0/2424858573 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f4f6c0a2b80 msgr2=0x7f4f6c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:41.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.856+0000 7f4f871eb640 1 --2- 172.21.15.16:0/2424858573 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f4f6c0a2b80 0x7f4f6c0a5040 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f4f68009230 tx=0x7f4f680058d0 comp rx=0 tx=0).stop 2024-09-12T02:04:41.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.856+0000 7f4f871eb640 1 -- 172.21.15.16:0/2424858573 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f80101de0 msgr2=0x7f4f8019a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:41.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.856+0000 7f4f871eb640 1 --2- 172.21.15.16:0/2424858573 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f80101de0 0x7f4f8019a620 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f4f7000d770 tx=0x7f4f7000dc30 comp rx=0 tx=0).stop 2024-09-12T02:04:41.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.856+0000 7f4f871eb640 1 -- 172.21.15.16:0/2424858573 shutdown_connections 2024-09-12T02:04:41.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.856+0000 7f4f871eb640 1 --2- 172.21.15.16:0/2424858573 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f4f6c0a2b80 0x7f4f6c0a5040 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:41.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.856+0000 7f4f871eb640 1 --2- 172.21.15.16:0/2424858573 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4f80106f10 0x7f4f8019eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:41.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.856+0000 7f4f871eb640 1 --2- 172.21.15.16:0/2424858573 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4f80102790 0x7f4f8019ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:41.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.856+0000 7f4f871eb640 1 --2- 172.21.15.16:0/2424858573 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f80101de0 0x7f4f8019a620 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:41.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.856+0000 7f4f871eb640 1 -- 172.21.15.16:0/2424858573 >> 172.21.15.16:0/2424858573 conn(0x7f4f800fd8c0 msgr2=0x7f4f800ffbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:04:41.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.856+0000 7f4f871eb640 1 -- 172.21.15.16:0/2424858573 shutdown_connections 2024-09-12T02:04:41.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:41.857+0000 7f4f871eb640 1 -- 172.21.15.16:0/2424858573 wait complete. 2024-09-12T02:04:42.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:04:41 smithi033 ceph-mon[35332]: pgmap v418: 97 pgs: 97 active+clean; 1.5 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-09-12T02:04:42.183 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:04:41 smithi016 ceph-mon[30947]: pgmap v418: 97 pgs: 97 active+clean; 1.5 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-09-12T02:04:42.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:04:41 smithi184 ceph-mon[35593]: pgmap v418: 97 pgs: 97 active+clean; 1.5 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-09-12T02:04:42.695 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '146e8aed', total seconds elapsed: 27.9 2024-09-12T02:04:44.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:04:43 smithi033 ceph-mon[35332]: pgmap v419: 97 pgs: 97 active+clean; 1.5 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 2.0 KiB/s wr, 7 op/s 2024-09-12T02:04:44.183 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:04:43 smithi016 ceph-mon[30947]: pgmap v419: 97 pgs: 97 active+clean; 1.5 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 2.0 KiB/s wr, 7 op/s 2024-09-12T02:04:44.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:04:43 smithi184 ceph-mon[35593]: pgmap v419: 97 pgs: 97 active+clean; 1.5 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 2.0 KiB/s wr, 7 op/s 2024-09-12T02:04:46.168 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:04:45 smithi033 ceph-mon[35332]: pgmap v420: 97 pgs: 97 active+clean; 1.5 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 5.2 KiB/s wr, 9 op/s 2024-09-12T02:04:46.183 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:04:45 smithi016 ceph-mon[30947]: pgmap v420: 97 pgs: 97 active+clean; 1.5 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 5.2 KiB/s wr, 9 op/s 2024-09-12T02:04:46.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:04:45 smithi184 ceph-mon[35593]: pgmap v420: 97 pgs: 97 active+clean; 1.5 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 5.2 KiB/s wr, 9 op/s 2024-09-12T02:04:48.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:04:47 smithi033 ceph-mon[35332]: pgmap v421: 97 pgs: 97 active+clean; 1.5 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 4.7 KiB/s wr, 9 op/s 2024-09-12T02:04:48.183 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:04:47 smithi016 ceph-mon[30947]: pgmap v421: 97 pgs: 97 active+clean; 1.5 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 4.7 KiB/s wr, 9 op/s 2024-09-12T02:04:48.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:04:47 smithi184 ceph-mon[35593]: pgmap v421: 97 pgs: 97 active+clean; 1.5 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 4.7 KiB/s wr, 9 op/s 2024-09-12T02:04:48.543 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.543+0000 7f926eb9f640 1 -- 172.21.15.16:0/2369561563 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9268101490 msgr2=0x7f9268101940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:48.544 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.543+0000 7f926eb9f640 1 --2- 172.21.15.16:0/2369561563 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9268101490 0x7f9268101940 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f9260009ca0 tx=0x7f926002a8c0 comp rx=0 tx=0).stop 2024-09-12T02:04:48.544 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.543+0000 7f926eb9f640 1 -- 172.21.15.16:0/2369561563 shutdown_connections 2024-09-12T02:04:48.544 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.543+0000 7f926eb9f640 1 --2- 172.21.15.16:0/2369561563 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9268101490 0x7f9268101940 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:48.544 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.543+0000 7f926eb9f640 1 --2- 172.21.15.16:0/2369561563 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9268100af0 0x7f9268100f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:48.545 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.543+0000 7f926eb9f640 1 --2- 172.21.15.16:0/2369561563 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f92681001b0 0x7f92681005b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:48.545 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.543+0000 7f926eb9f640 1 -- 172.21.15.16:0/2369561563 >> 172.21.15.16:0/2369561563 conn(0x7f92680fbfa0 msgr2=0x7f92680fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:04:48.545 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.543+0000 7f926eb9f640 1 -- 172.21.15.16:0/2369561563 shutdown_connections 2024-09-12T02:04:48.545 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.543+0000 7f926eb9f640 1 -- 172.21.15.16:0/2369561563 wait complete. 2024-09-12T02:04:48.556 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.544+0000 7f926eb9f640 1 Processor -- start 2024-09-12T02:04:48.556 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.544+0000 7f926eb9f640 1 -- start start 2024-09-12T02:04:48.556 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.544+0000 7f926eb9f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f92681001b0 0x7f9268199e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:48.556 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.544+0000 7f926eb9f640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9268100af0 0x7f926819a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:48.556 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.544+0000 7f926eb9f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9268101490 0x7f92681a1400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:48.556 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.544+0000 7f926eb9f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f926810f790 con 0x7f9268101490 2024-09-12T02:04:48.556 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.545+0000 7f926eb9f640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f926810f610 con 0x7f92681001b0 2024-09-12T02:04:48.557 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.545+0000 7f926eb9f640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f926810f910 con 0x7f9268100af0 2024-09-12T02:04:48.557 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.545+0000 7f926e39e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9268101490 0x7f92681a1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:48.557 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.545+0000 7f926e39e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9268101490 0x7f92681a1400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41864/0 (socket says 172.21.15.16:41864) 2024-09-12T02:04:48.557 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.545+0000 7f926e39e640 1 -- 172.21.15.16:0/4159959644 learned_addr learned my addr 172.21.15.16:0/4159959644 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:04:48.557 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.545+0000 7f926db9d640 1 --2- 172.21.15.16:0/4159959644 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f92681001b0 0x7f9268199e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:48.557 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.545+0000 7f926d39c640 1 --2- 172.21.15.16:0/4159959644 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9268100af0 0x7f926819a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:48.557 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.545+0000 7f926e39e640 1 -- 172.21.15.16:0/4159959644 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9268100af0 msgr2=0x7f926819a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:48.557 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.545+0000 7f926e39e640 1 --2- 172.21.15.16:0/4159959644 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9268100af0 0x7f926819a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:48.557 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.545+0000 7f926e39e640 1 -- 172.21.15.16:0/4159959644 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f92681001b0 msgr2=0x7f9268199e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:48.558 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.545+0000 7f926e39e640 1 --2- 172.21.15.16:0/4159959644 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f92681001b0 0x7f9268199e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:48.558 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.545+0000 7f926e39e640 1 -- 172.21.15.16:0/4159959644 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92681a1a70 con 0x7f9268101490 2024-09-12T02:04:48.558 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.545+0000 7f926e39e640 1 --2- 172.21.15.16:0/4159959644 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9268101490 0x7f92681a1400 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f9260005e30 tx=0x7f9260031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:04:48.558 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.545+0000 7f925affd640 1 -- 172.21.15.16:0/4159959644 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9260003c30 con 0x7f9268101490 2024-09-12T02:04:48.558 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.545+0000 7f926eb9f640 1 -- 172.21.15.16:0/4159959644 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92681a1d00 con 0x7f9268101490 2024-09-12T02:04:48.558 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.545+0000 7f925affd640 1 -- 172.21.15.16:0/4159959644 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9260003dd0 con 0x7f9268101490 2024-09-12T02:04:48.558 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.546+0000 7f926eb9f640 1 -- 172.21.15.16:0/4159959644 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92681a2240 con 0x7f9268101490 2024-09-12T02:04:48.558 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.546+0000 7f925affd640 1 -- 172.21.15.16:0/4159959644 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f926002fd80 con 0x7f9268101490 2024-09-12T02:04:48.558 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.549+0000 7f9272c75740 1 Processor -- start 2024-09-12T02:04:48.558 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.549+0000 7f925affd640 1 -- 172.21.15.16:0/4159959644 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f9260004000 con 0x7f9268101490 2024-09-12T02:04:48.558 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.549+0000 7f9272c75740 1 -- start start 2024-09-12T02:04:48.558 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.549+0000 7f9272c75740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5621dbfb0b70 0x5621dbfb0f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:48.559 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.549+0000 7f9272c75740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5621dbfb14b0 0x5621dbfb5f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:48.559 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.549+0000 7f9272c75740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5621dbfb6540 0x5621dbfb8a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:48.559 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.549+0000 7f9272c75740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5621dbf142f0 con 0x5621dbfb0b70 2024-09-12T02:04:48.559 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.549+0000 7f9272c75740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5621dbebc270 con 0x5621dbfb14b0 2024-09-12T02:04:48.559 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.550+0000 7f9272c75740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5621dbf102b0 con 0x5621dbfb6540 2024-09-12T02:04:48.559 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.550+0000 7f926d39c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5621dbfb0b70 0x5621dbfb0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:48.559 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.550+0000 7f926d39c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5621dbfb0b70 0x5621dbfb0f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41872/0 (socket says 172.21.15.16:41872) 2024-09-12T02:04:48.559 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.550+0000 7f926d39c640 1 -- 172.21.15.16:0/4261507630 learned_addr learned my addr 172.21.15.16:0/4261507630 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:04:48.559 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.550+0000 7f926db9d640 1 --2- 172.21.15.16:0/4261507630 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5621dbfb14b0 0x5621dbfb5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:48.559 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.550+0000 7f926d39c640 1 -- 172.21.15.16:0/4261507630 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5621dbfb6540 msgr2=0x5621dbfb8a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:48.559 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.550+0000 7f926d39c640 1 --2- 172.21.15.16:0/4261507630 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5621dbfb6540 0x5621dbfb8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:48.559 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.550+0000 7f926d39c640 1 -- 172.21.15.16:0/4261507630 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5621dbfb14b0 msgr2=0x5621dbfb5f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:48.559 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.550+0000 7f926d39c640 1 --2- 172.21.15.16:0/4261507630 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5621dbfb14b0 0x5621dbfb5f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:48.560 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.550+0000 7f926d39c640 1 -- 172.21.15.16:0/4261507630 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5621dbebb340 con 0x5621dbfb0b70 2024-09-12T02:04:48.560 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.550+0000 7f925affd640 1 --2- 172.21.15.16:0/4159959644 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f92540a2b80 0x7f92540a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:48.560 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.550+0000 7f926d39c640 1 --2- 172.21.15.16:0/4261507630 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5621dbfb0b70 0x5621dbfb0f70 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f925c00bdb0 tx=0x7f925c0107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f264c1e2205a7c19 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:04:48.560 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.550+0000 7f926db9d640 1 --2- 172.21.15.16:0/4159959644 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f92540a2b80 0x7f92540a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:48.560 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.550+0000 7f925affd640 1 -- 172.21.15.16:0/4159959644 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f92600f03f0 con 0x7f9268101490 2024-09-12T02:04:48.560 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.550+0000 7f9253fff640 1 -- 172.21.15.16:0/4261507630 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f925c00bea0 con 0x5621dbfb0b70 2024-09-12T02:04:48.560 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.551+0000 7f9253fff640 1 -- 172.21.15.16:0/4261507630 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f925c005500 con 0x5621dbfb0b70 2024-09-12T02:04:48.560 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.551+0000 7f9253fff640 1 -- 172.21.15.16:0/4261507630 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f925c004d90 con 0x5621dbfb0b70 2024-09-12T02:04:48.560 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.551+0000 7f9272c75740 1 -- 172.21.15.16:0/4261507630 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5621dbfb0b70 msgr2=0x5621dbfb0f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:48.560 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.551+0000 7f9272c75740 1 --2- 172.21.15.16:0/4261507630 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5621dbfb0b70 0x5621dbfb0f70 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f925c00bdb0 tx=0x7f925c0107a0 comp rx=0 tx=0).stop 2024-09-12T02:04:48.560 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.551+0000 7f926db9d640 1 --2- 172.21.15.16:0/4159959644 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f92540a2b80 0x7f92540a5040 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x5621dbfb1ee0 tx=0x7f926400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:04:48.561 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.551+0000 7f9272c75740 1 -- 172.21.15.16:0/4261507630 shutdown_connections 2024-09-12T02:04:48.561 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.551+0000 7f9272c75740 1 --2- 172.21.15.16:0/4261507630 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5621dbfb6540 0x5621dbfb8a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:48.561 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.551+0000 7f9272c75740 1 --2- 172.21.15.16:0/4261507630 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5621dbfb14b0 0x5621dbfb5f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:48.561 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.551+0000 7f9272c75740 1 --2- 172.21.15.16:0/4261507630 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5621dbfb0b70 0x5621dbfb0f70 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:48.561 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.551+0000 7f9272c75740 1 -- 172.21.15.16:0/4261507630 >> 172.21.15.16:0/4261507630 conn(0x5621dbfac1e0 msgr2=0x5621dbfac5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:04:48.561 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.551+0000 7f9272c75740 1 -- 172.21.15.16:0/4261507630 shutdown_connections 2024-09-12T02:04:48.561 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.551+0000 7f9272c75740 1 -- 172.21.15.16:0/4261507630 wait complete. 2024-09-12T02:04:48.561 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.552+0000 7f9272c75740 1 -- start start 2024-09-12T02:04:48.561 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.552+0000 7f9272c75740 1 Processor -- start 2024-09-12T02:04:48.562 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.552+0000 7f9272c75740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5621dbfb0b70 0x5621dbfbadc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:48.562 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.552+0000 7f9272c75740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5621dbfb14b0 0x5621dbfbb300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:48.562 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.552+0000 7f9272c75740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5621dbfb6540 0x5621dbfc2340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:48.562 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.552+0000 7f9272c75740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5621dbfb9510 con 0x5621dbfb6540 2024-09-12T02:04:48.562 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.552+0000 7f9272c75740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5621dbe9d290 con 0x5621dbfb0b70 2024-09-12T02:04:48.562 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.552+0000 7f9272c75740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5621dbfb9690 con 0x5621dbfb14b0 2024-09-12T02:04:48.562 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.552+0000 7f9272c75740 10 client.0 _ll_register_callbacks cb 0x5621dbece140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:04:48.562 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.553+0000 7f926e39e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5621dbfb6540 0x5621dbfc2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:48.562 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.553+0000 7f926e39e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5621dbfb6540 0x5621dbfc2340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41874/0 (socket says 172.21.15.16:41874) 2024-09-12T02:04:48.562 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.553+0000 7f926e39e640 1 -- 172.21.15.16:0/1675726934 learned_addr learned my addr 172.21.15.16:0/1675726934 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:04:48.562 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.553+0000 7f926db9d640 1 --2- 172.21.15.16:0/1675726934 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5621dbfb0b70 0x5621dbfbadc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:48.563 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.553+0000 7f926d39c640 1 --2- 172.21.15.16:0/1675726934 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5621dbfb14b0 0x5621dbfbb300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:48.563 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.553+0000 7f926e39e640 1 -- 172.21.15.16:0/1675726934 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5621dbfb14b0 msgr2=0x5621dbfbb300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:48.563 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.553+0000 7f926e39e640 1 --2- 172.21.15.16:0/1675726934 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5621dbfb14b0 0x5621dbfbb300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:48.563 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.553+0000 7f926e39e640 1 -- 172.21.15.16:0/1675726934 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5621dbfb0b70 msgr2=0x5621dbfbadc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:48.563 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.553+0000 7f926e39e640 1 --2- 172.21.15.16:0/1675726934 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5621dbfb0b70 0x5621dbfbadc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:48.563 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.553+0000 7f926e39e640 1 -- 172.21.15.16:0/1675726934 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5621dbc27210 con 0x5621dbfb6540 2024-09-12T02:04:48.563 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.553+0000 7f926e39e640 1 --2- 172.21.15.16:0/1675726934 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5621dbfb6540 0x5621dbfc2340 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f92600c5390 tx=0x7f92600bc5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a63a3d41da375a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:04:48.563 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.553+0000 7f92517fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:04:48.563 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.553+0000 7f92517fa640 1 -- 172.21.15.16:0/1675726934 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92600d2030 con 0x5621dbfb6540 2024-09-12T02:04:48.564 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.553+0000 7f92517fa640 1 -- 172.21.15.16:0/1675726934 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f92600bacf0 con 0x5621dbfb6540 2024-09-12T02:04:48.564 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.553+0000 7f9272c75740 1 -- 172.21.15.16:0/1675726934 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f925c005500 con 0x5621dbfb6540 2024-09-12T02:04:48.564 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.553+0000 7f92517fa640 1 -- 172.21.15.16:0/1675726934 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f92600bcc50 con 0x5621dbfb6540 2024-09-12T02:04:48.564 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.553+0000 7f9272c75740 10 client.16239 fetch_fsmap learned FSMap version 25 2024-09-12T02:04:48.564 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.553+0000 7f9272c75740 1 -- 172.21.15.16:0/1675726934 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5621dbcf3620 con 0x5621dbfb6540 2024-09-12T02:04:48.564 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.554+0000 7f92517fa640 1 -- 172.21.15.16:0/1675726934 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f925c005500 con 0x5621dbfb6540 2024-09-12T02:04:48.564 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.554+0000 7f92517fa640 1 -- 172.21.15.16:0/1675726934 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f92600c5430 con 0x5621dbfb6540 2024-09-12T02:04:48.564 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.554+0000 7f9272c75740 10 client.16239 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:04:48.564 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.554+0000 7f9272c75740 10 client.16239 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:04:48.564 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.554+0000 7f9272c75740 20 client.16239 populate_metadata read hostname 'smithi016' 2024-09-12T02:04:48.564 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.554+0000 7f9272c75740 1 --2- 172.21.15.16:0/1675726934 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5621dbfc6b50 0x5621dbfc8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:48.564 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.554+0000 7f9272c75740 4 client.16239 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:04:48.565 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.554+0000 7f9272c75740 1 -- 172.21.15.16:0/1675726934 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5621dbf21a20 con 0x5621dbfc6b50 2024-09-12T02:04:48.565 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.554+0000 7f926db9d640 1 --2- 172.21.15.16:0/1675726934 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5621dbfc6b50 0x5621dbfc8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:48.565 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.554+0000 7f92517fa640 10 client.16239 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:04:48.565 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.554+0000 7f926db9d640 1 --2- 172.21.15.16:0/1675726934 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5621dbfc6b50 0x5621dbfc8fb0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f68c9c5f38b6993 server_cookie=4063162dd0fefa4b in_seq=0 out_seq=0 2024-09-12T02:04:48.565 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.555+0000 7f92517fa640 1 -- 172.21.15.16:0/1675726934 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5621dbf21a20 con 0x5621dbfc6b50 2024-09-12T02:04:48.565 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.555+0000 7f92517fa640 10 client.16239 handle_command_reply: tid=0 2024-09-12T02:04:48.565 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.555+0000 7f9272c75740 1 client.16239 shutdown 2024-09-12T02:04:48.565 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.555+0000 7f9272c75740 1 -- 172.21.15.16:0/1675726934 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5621dbfb6540 msgr2=0x5621dbfc2340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:48.565 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.555+0000 7f9272c75740 1 --2- 172.21.15.16:0/1675726934 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5621dbfb6540 0x5621dbfc2340 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f92600c5390 tx=0x7f92600bc5b0 comp rx=0 tx=0).stop 2024-09-12T02:04:48.565 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.555+0000 7f9272c75740 1 -- 172.21.15.16:0/1675726934 shutdown_connections 2024-09-12T02:04:48.565 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.555+0000 7f9272c75740 1 --2- 172.21.15.16:0/1675726934 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5621dbfc6b50 0x5621dbfc8fb0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:48.565 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.555+0000 7f9272c75740 1 --2- 172.21.15.16:0/1675726934 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5621dbfb6540 0x5621dbfc2340 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:48.565 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.555+0000 7f9272c75740 1 --2- 172.21.15.16:0/1675726934 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5621dbfb14b0 0x5621dbfbb300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:48.566 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.555+0000 7f92517fa640 0 client.16239 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:04:48.566 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.555+0000 7f9272c75740 1 --2- 172.21.15.16:0/1675726934 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5621dbfb0b70 0x5621dbfbadc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:48.566 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.555+0000 7f9272c75740 1 -- 172.21.15.16:0/1675726934 >> 172.21.15.16:0/1675726934 conn(0x5621dbfac1e0 msgr2=0x5621dbfad9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:04:48.566 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.556+0000 7f9272c75740 1 -- 172.21.15.16:0/1675726934 shutdown_connections 2024-09-12T02:04:48.566 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.556+0000 7f9272c75740 1 -- 172.21.15.16:0/1675726934 wait complete. 2024-09-12T02:04:48.566 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.556+0000 7f9272c75740 20 client.16239 trim_cache size 0 max 16384 2024-09-12T02:04:48.575 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.570+0000 7f9272c75740 1 Processor -- start 2024-09-12T02:04:48.576 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.570+0000 7f9272c75740 1 -- start start 2024-09-12T02:04:48.576 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.570+0000 7f9272c75740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5621dbfb0b70 0x5621dbfad7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:48.576 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.571+0000 7f9272c75740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5621dbfb14b0 0x5621dbfadce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:48.576 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.571+0000 7f9272c75740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5621dbfb6540 0x5621dbfc2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:48.576 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.571+0000 7f9272c75740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f92681ad750 con 0x5621dbfb6540 2024-09-12T02:04:48.576 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.571+0000 7f9272c75740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f92681ab020 con 0x5621dbfb0b70 2024-09-12T02:04:48.576 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.571+0000 7f9272c75740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f92681a8e70 con 0x5621dbfb14b0 2024-09-12T02:04:48.576 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.571+0000 7f926e39e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5621dbfb6540 0x5621dbfc2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:48.577 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.571+0000 7f926e39e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5621dbfb6540 0x5621dbfc2300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41886/0 (socket says 172.21.15.16:41886) 2024-09-12T02:04:48.577 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.571+0000 7f926e39e640 1 -- 172.21.15.16:0/2834489352 learned_addr learned my addr 172.21.15.16:0/2834489352 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:04:48.577 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.571+0000 7f926db9d640 1 --2- 172.21.15.16:0/2834489352 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5621dbfb0b70 0x5621dbfad7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:48.577 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.571+0000 7f926d39c640 1 --2- 172.21.15.16:0/2834489352 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5621dbfb14b0 0x5621dbfadce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:48.577 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.571+0000 7f926e39e640 1 -- 172.21.15.16:0/2834489352 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5621dbfb14b0 msgr2=0x5621dbfadce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:48.577 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.571+0000 7f926e39e640 1 --2- 172.21.15.16:0/2834489352 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5621dbfb14b0 0x5621dbfadce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:48.578 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.571+0000 7f926e39e640 1 -- 172.21.15.16:0/2834489352 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5621dbfb0b70 msgr2=0x5621dbfad7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:48.578 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.571+0000 7f926e39e640 1 --2- 172.21.15.16:0/2834489352 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5621dbfb0b70 0x5621dbfad7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:48.578 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.571+0000 7f926e39e640 1 -- 172.21.15.16:0/2834489352 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5621dbfac000 con 0x5621dbfb6540 2024-09-12T02:04:48.578 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.571+0000 7f926e39e640 1 --2- 172.21.15.16:0/2834489352 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5621dbfb6540 0x5621dbfc2300 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f92600b9620 tx=0x7f92600bc6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a020960076300d70 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:04:48.578 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.571+0000 7f92537fe640 1 -- 172.21.15.16:0/2834489352 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92600d2070 con 0x5621dbfb6540 2024-09-12T02:04:48.578 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.571+0000 7f92537fe640 1 -- 172.21.15.16:0/2834489352 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f92600d5e50 con 0x5621dbfb6540 2024-09-12T02:04:48.578 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.571+0000 7f92537fe640 1 -- 172.21.15.16:0/2834489352 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9260004b40 con 0x5621dbfb6540 2024-09-12T02:04:48.578 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.572+0000 7f9272c75740 1 -- 172.21.15.16:0/2834489352 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5621dbfb6540 msgr2=0x5621dbfc2300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:48.578 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.572+0000 7f9272c75740 1 --2- 172.21.15.16:0/2834489352 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5621dbfb6540 0x5621dbfc2300 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f92600b9620 tx=0x7f92600bc6e0 comp rx=0 tx=0).stop 2024-09-12T02:04:48.578 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.572+0000 7f9272c75740 1 -- 172.21.15.16:0/2834489352 shutdown_connections 2024-09-12T02:04:48.578 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.572+0000 7f9272c75740 1 --2- 172.21.15.16:0/2834489352 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5621dbfb6540 0x5621dbfc2300 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:48.578 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.572+0000 7f9272c75740 1 --2- 172.21.15.16:0/2834489352 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5621dbfb14b0 0x5621dbfadce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:48.579 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.572+0000 7f9272c75740 1 --2- 172.21.15.16:0/2834489352 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5621dbfb0b70 0x5621dbfad7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:48.579 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.572+0000 7f9272c75740 1 -- 172.21.15.16:0/2834489352 >> 172.21.15.16:0/2834489352 conn(0x5621dbfac1e0 msgr2=0x5621dbfad090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:04:48.579 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.572+0000 7f9272c75740 1 -- 172.21.15.16:0/2834489352 shutdown_connections 2024-09-12T02:04:48.579 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.572+0000 7f9272c75740 1 -- 172.21.15.16:0/2834489352 wait complete. 2024-09-12T02:04:48.579 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.572+0000 7f9272c75740 1 -- start start 2024-09-12T02:04:48.579 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.572+0000 7f9272c75740 1 Processor -- start 2024-09-12T02:04:48.579 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.573+0000 7f9272c75740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5621dbfb0b70 0x5621dbfc0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:48.579 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.573+0000 7f9272c75740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5621dbfb14b0 0x5621dbfc0740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:48.579 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.573+0000 7f9272c75740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5621dbfb6540 0x5621dbfc1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:48.579 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.573+0000 7f9272c75740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f92681a94e0 con 0x5621dbfb14b0 2024-09-12T02:04:48.579 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.573+0000 7f9272c75740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f92681aa950 con 0x5621dbfb6540 2024-09-12T02:04:48.579 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.573+0000 7f9272c75740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f92681a9bb0 con 0x5621dbfb0b70 2024-09-12T02:04:48.580 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.573+0000 7f9272c75740 10 client.0 _ll_register_callbacks cb 0x7f926002ff20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:04:48.580 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.573+0000 7f926d39c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5621dbfb14b0 0x5621dbfc0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:48.580 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.573+0000 7f926d39c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5621dbfb14b0 0x5621dbfc0740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41888/0 (socket says 172.21.15.16:41888) 2024-09-12T02:04:48.580 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.573+0000 7f926d39c640 1 -- 172.21.15.16:0/3837797449 learned_addr learned my addr 172.21.15.16:0/3837797449 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:04:48.580 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.573+0000 7f926e39e640 1 --2- 172.21.15.16:0/3837797449 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5621dbfb6540 0x5621dbfc1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:48.580 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.573+0000 7f926db9d640 1 --2- 172.21.15.16:0/3837797449 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5621dbfb0b70 0x5621dbfc0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:48.580 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.573+0000 7f926d39c640 1 -- 172.21.15.16:0/3837797449 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5621dbfb0b70 msgr2=0x5621dbfc0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:48.580 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.573+0000 7f926d39c640 1 --2- 172.21.15.16:0/3837797449 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5621dbfb0b70 0x5621dbfc0200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:48.580 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.573+0000 7f926d39c640 1 -- 172.21.15.16:0/3837797449 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5621dbfb6540 msgr2=0x5621dbfc1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:48.580 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.573+0000 7f926d39c640 1 --2- 172.21.15.16:0/3837797449 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5621dbfb6540 0x5621dbfc1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:48.580 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.573+0000 7f926d39c640 1 -- 172.21.15.16:0/3837797449 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5621dbfc1d60 con 0x5621dbfb14b0 2024-09-12T02:04:48.580 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.574+0000 7f926d39c640 1 --2- 172.21.15.16:0/3837797449 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5621dbfb14b0 0x5621dbfc0740 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f925c004620 tx=0x7f925c00bac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71efdbb2acc39636 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:04:48.580 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.574+0000 7f92517fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:04:48.581 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.574+0000 7f92517fa640 1 -- 172.21.15.16:0/3837797449 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f925c02b060 con 0x5621dbfb14b0 2024-09-12T02:04:48.581 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.574+0000 7f92517fa640 1 -- 172.21.15.16:0/3837797449 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f925c0055d0 con 0x5621dbfb14b0 2024-09-12T02:04:48.581 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.574+0000 7f9272c75740 1 -- 172.21.15.16:0/3837797449 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f92600d5e50 con 0x5621dbfb14b0 2024-09-12T02:04:48.581 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.574+0000 7f92517fa640 1 -- 172.21.15.16:0/3837797449 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f925c004680 con 0x5621dbfb14b0 2024-09-12T02:04:48.581 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.574+0000 7f9272c75740 10 client.16245 fetch_fsmap learned FSMap version 25 2024-09-12T02:04:48.581 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.574+0000 7f9272c75740 1 -- 172.21.15.16:0/3837797449 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5621dc032080 con 0x5621dbfb14b0 2024-09-12T02:04:48.581 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.574+0000 7f92517fa640 1 -- 172.21.15.16:0/3837797449 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f92600d5e50 con 0x5621dbfb14b0 2024-09-12T02:04:48.581 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.574+0000 7f92517fa640 1 -- 172.21.15.16:0/3837797449 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f925c030070 con 0x5621dbfb14b0 2024-09-12T02:04:48.581 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.574+0000 7f9272c75740 10 client.16245 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:04:48.581 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.574+0000 7f9272c75740 10 client.16245 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:04:48.581 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.574+0000 7f9272c75740 20 client.16245 populate_metadata read hostname 'smithi016' 2024-09-12T02:04:48.582 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.574+0000 7f9272c75740 1 --2- 172.21.15.16:0/3837797449 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5621dc0326f0 0x5621dc034ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:48.582 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.574+0000 7f9272c75740 4 client.16245 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:04:48.582 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.574+0000 7f9272c75740 1 -- 172.21.15.16:0/3837797449 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5621dbf1bd70 con 0x5621dc0326f0 2024-09-12T02:04:48.582 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.575+0000 7f926db9d640 1 --2- 172.21.15.16:0/3837797449 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5621dc0326f0 0x5621dc034ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:48.582 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.575+0000 7f926db9d640 1 --2- 172.21.15.16:0/3837797449 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5621dc0326f0 0x5621dc034ae0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b56c78d0f330a6b3 server_cookie=8380ed7efccef625 in_seq=0 out_seq=0 2024-09-12T02:04:48.582 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.575+0000 7f92517fa640 10 client.16245 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:04:48.586 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.585+0000 7f92517fa640 1 -- 172.21.15.16:0/3837797449 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5621dbf1bd70 con 0x5621dc0326f0 2024-09-12T02:04:48.586 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.585+0000 7f92517fa640 10 client.16245 handle_command_reply: tid=0 2024-09-12T02:04:48.586 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.585+0000 7f9272c75740 1 client.16245 shutdown 2024-09-12T02:04:48.586 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.585+0000 7f9272c75740 1 -- 172.21.15.16:0/3837797449 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5621dbfb14b0 msgr2=0x5621dbfc0740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:48.586 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.585+0000 7f9272c75740 1 --2- 172.21.15.16:0/3837797449 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5621dbfb14b0 0x5621dbfc0740 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f925c004620 tx=0x7f925c00bac0 comp rx=0 tx=0).stop 2024-09-12T02:04:48.586 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.585+0000 7f9272c75740 1 -- 172.21.15.16:0/3837797449 shutdown_connections 2024-09-12T02:04:48.586 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.585+0000 7f9272c75740 1 --2- 172.21.15.16:0/3837797449 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5621dc0326f0 0x5621dc034ae0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:48.586 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.585+0000 7f9272c75740 1 --2- 172.21.15.16:0/3837797449 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5621dbfb6540 0x5621dbfc1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:48.586 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.585+0000 7f9272c75740 1 --2- 172.21.15.16:0/3837797449 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5621dbfb14b0 0x5621dbfc0740 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:48.586 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.585+0000 7f92517fa640 0 client.16245 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:04:48.587 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.585+0000 7f9272c75740 1 --2- 172.21.15.16:0/3837797449 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5621dbfb0b70 0x5621dbfc0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:48.587 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.585+0000 7f9272c75740 1 -- 172.21.15.16:0/3837797449 >> 172.21.15.16:0/3837797449 conn(0x5621dbfac1e0 msgr2=0x5621dbfad8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:04:48.587 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.585+0000 7f9272c75740 1 -- 172.21.15.16:0/3837797449 shutdown_connections 2024-09-12T02:04:48.587 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.585+0000 7f9272c75740 1 -- 172.21.15.16:0/3837797449 wait complete. 2024-09-12T02:04:48.587 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.585+0000 7f9272c75740 20 client.16245 trim_cache size 0 max 16384 2024-09-12T02:04:48.587 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-09-12T02:04:48.587 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-09-12T02:04:48.587 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-09-12T02:04:48.587 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-09-12T02:04:48.588 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.586+0000 7f926eb9f640 1 -- 172.21.15.16:0/4159959644 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f92540a2b80 msgr2=0x7f92540a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:48.588 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.586+0000 7f926eb9f640 1 --2- 172.21.15.16:0/4159959644 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f92540a2b80 0x7f92540a5040 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x5621dbfb1ee0 tx=0x7f926400b390 comp rx=0 tx=0).stop 2024-09-12T02:04:48.589 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.586+0000 7f926eb9f640 1 -- 172.21.15.16:0/4159959644 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9268101490 msgr2=0x7f92681a1400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:48.589 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.586+0000 7f926eb9f640 1 --2- 172.21.15.16:0/4159959644 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9268101490 0x7f92681a1400 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f9260005e30 tx=0x7f9260031ee0 comp rx=0 tx=0).stop 2024-09-12T02:04:48.589 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.586+0000 7f926eb9f640 1 -- 172.21.15.16:0/4159959644 shutdown_connections 2024-09-12T02:04:48.589 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.586+0000 7f926eb9f640 1 --2- 172.21.15.16:0/4159959644 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f92540a2b80 0x7f92540a5040 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:48.589 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.586+0000 7f926eb9f640 1 --2- 172.21.15.16:0/4159959644 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9268101490 0x7f92681a1400 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:48.589 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.586+0000 7f926eb9f640 1 --2- 172.21.15.16:0/4159959644 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9268100af0 0x7f926819a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:48.589 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.586+0000 7f926eb9f640 1 --2- 172.21.15.16:0/4159959644 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f92681001b0 0x7f9268199e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:48.589 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.586+0000 7f926eb9f640 1 -- 172.21.15.16:0/4159959644 >> 172.21.15.16:0/4159959644 conn(0x7f92680fbfa0 msgr2=0x7f92680fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:04:48.589 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.586+0000 7f926eb9f640 1 -- 172.21.15.16:0/4159959644 shutdown_connections 2024-09-12T02:04:48.589 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:04:48.586+0000 7f926eb9f640 1 -- 172.21.15.16:0/4159959644 wait complete. 2024-09-12T02:04:50.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:04:49 smithi033 ceph-mon[35332]: pgmap v422: 97 pgs: 97 active+clean; 1.5 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 4.7 KiB/s wr, 8 op/s 2024-09-12T02:04:50.183 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:04:49 smithi016 ceph-mon[30947]: pgmap v422: 97 pgs: 97 active+clean; 1.5 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 4.7 KiB/s wr, 8 op/s 2024-09-12T02:04:50.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:04:49 smithi184 ceph-mon[35593]: pgmap v422: 97 pgs: 97 active+clean; 1.5 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 4.7 KiB/s wr, 8 op/s 2024-09-12T02:04:52.168 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:04:51 smithi033 ceph-mon[35332]: pgmap v423: 97 pgs: 97 active+clean; 1.6 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 13 op/s 2024-09-12T02:04:52.179 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:04:51 smithi184 ceph-mon[35593]: pgmap v423: 97 pgs: 97 active+clean; 1.6 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 13 op/s 2024-09-12T02:04:52.183 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:04:51 smithi016 ceph-mon[30947]: pgmap v423: 97 pgs: 97 active+clean; 1.6 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 13 op/s 2024-09-12T02:04:53.772 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T02:04:54.045 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:04:54.161 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:04:53 smithi016 ceph-mon[30947]: pgmap v424: 97 pgs: 97 active+clean; 1.6 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 KiB/s wr, 11 op/s 2024-09-12T02:04:54.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:04:53 smithi184 ceph-mon[35593]: pgmap v424: 97 pgs: 97 active+clean; 1.6 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 KiB/s wr, 11 op/s 2024-09-12T02:04:54.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:04:53 smithi033 ceph-mon[35332]: pgmap v424: 97 pgs: 97 active+clean; 1.6 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 KiB/s wr, 11 op/s 2024-09-12T02:04:54.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.464+0000 7ff1b6450640 1 -- 172.21.15.16:0/3121171963 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1b0100a80 msgr2=0x7ff1b0100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:54.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.464+0000 7ff1b6450640 1 --2- 172.21.15.16:0/3121171963 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1b0100a80 0x7ff1b0100ee0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7ff1a4009930 tx=0x7ff1a402a420 comp rx=0 tx=0).stop 2024-09-12T02:04:54.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.464+0000 7ff1b6450640 1 -- 172.21.15.16:0/3121171963 shutdown_connections 2024-09-12T02:04:54.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.464+0000 7ff1b6450640 1 --2- 172.21.15.16:0/3121171963 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff1b0101420 0x7ff1b010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:54.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.464+0000 7ff1b6450640 1 --2- 172.21.15.16:0/3121171963 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1b0100a80 0x7ff1b0100ee0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:54.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.464+0000 7ff1b6450640 1 --2- 172.21.15.16:0/3121171963 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1b0106a80 0x7ff1b0106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:54.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.464+0000 7ff1b6450640 1 -- 172.21.15.16:0/3121171963 >> 172.21.15.16:0/3121171963 conn(0x7ff1b00fc7a0 msgr2=0x7ff1b00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:04:54.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.464+0000 7ff1b6450640 1 -- 172.21.15.16:0/3121171963 shutdown_connections 2024-09-12T02:04:54.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.464+0000 7ff1b6450640 1 -- 172.21.15.16:0/3121171963 wait complete. 2024-09-12T02:04:54.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.465+0000 7ff1b6450640 1 Processor -- start 2024-09-12T02:04:54.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.465+0000 7ff1b6450640 1 -- start start 2024-09-12T02:04:54.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.465+0000 7ff1b6450640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff1b0100a80 0x7ff1b0198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:54.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.465+0000 7ff1b6450640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1b0101420 0x7ff1b0198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:54.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.465+0000 7ff1b6450640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1b0106a80 0x7ff1b019cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:54.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.465+0000 7ff1b6450640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff1b0110e80 con 0x7ff1b0101420 2024-09-12T02:04:54.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.465+0000 7ff1b6450640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff1b0110d00 con 0x7ff1b0106a80 2024-09-12T02:04:54.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.465+0000 7ff1b6450640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff1b0111000 con 0x7ff1b0100a80 2024-09-12T02:04:54.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.465+0000 7ff1af7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1b0101420 0x7ff1b0198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:54.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.465+0000 7ff1af7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1b0101420 0x7ff1b0198980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57774/0 (socket says 172.21.15.16:57774) 2024-09-12T02:04:54.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.466+0000 7ff1af7fe640 1 -- 172.21.15.16:0/2614579992 learned_addr learned my addr 172.21.15.16:0/2614579992 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:04:54.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.466+0000 7ff1b49ed640 1 --2- 172.21.15.16:0/2614579992 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1b0106a80 0x7ff1b019cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:54.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.466+0000 7ff1affff640 1 --2- 172.21.15.16:0/2614579992 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff1b0100a80 0x7ff1b0198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:54.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.466+0000 7ff1af7fe640 1 -- 172.21.15.16:0/2614579992 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff1b0100a80 msgr2=0x7ff1b0198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:54.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.466+0000 7ff1af7fe640 1 --2- 172.21.15.16:0/2614579992 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff1b0100a80 0x7ff1b0198440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:54.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.466+0000 7ff1af7fe640 1 -- 172.21.15.16:0/2614579992 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1b0106a80 msgr2=0x7ff1b019cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:54.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.466+0000 7ff1af7fe640 1 --2- 172.21.15.16:0/2614579992 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1b0106a80 0x7ff1b019cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:54.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.466+0000 7ff1af7fe640 1 -- 172.21.15.16:0/2614579992 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1b019d3f0 con 0x7ff1b0101420 2024-09-12T02:04:54.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.466+0000 7ff1affff640 1 --2- 172.21.15.16:0/2614579992 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff1b0100a80 0x7ff1b0198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:04:54.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.466+0000 7ff1af7fe640 1 --2- 172.21.15.16:0/2614579992 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1b0101420 0x7ff1b0198980 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7ff1a4009230 tx=0x7ff1a4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:04:54.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.466+0000 7ff1ad7fa640 1 -- 172.21.15.16:0/2614579992 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1a403f030 con 0x7ff1b0101420 2024-09-12T02:04:54.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.466+0000 7ff1ad7fa640 1 -- 172.21.15.16:0/2614579992 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff1a4003ad0 con 0x7ff1b0101420 2024-09-12T02:04:54.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.466+0000 7ff1b6450640 1 -- 172.21.15.16:0/2614579992 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1b019d680 con 0x7ff1b0101420 2024-09-12T02:04:54.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.466+0000 7ff1ad7fa640 1 -- 172.21.15.16:0/2614579992 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1a4035070 con 0x7ff1b0101420 2024-09-12T02:04:54.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.466+0000 7ff1b6450640 1 -- 172.21.15.16:0/2614579992 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1b01a4fc0 con 0x7ff1b0101420 2024-09-12T02:04:54.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.469+0000 7ff1ad7fa640 1 -- 172.21.15.16:0/2614579992 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff1a4042030 con 0x7ff1b0101420 2024-09-12T02:04:54.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.469+0000 7ff1b8829740 1 Processor -- start 2024-09-12T02:04:54.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.469+0000 7ff1b8829740 1 -- start start 2024-09-12T02:04:54.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.470+0000 7ff1b8829740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560128c18380 0x560128c18760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:54.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.470+0000 7ff1b8829740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560128c18ca0 0x560128c27550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:54.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.470+0000 7ff1b8829740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560128c27a90 0x560128c29e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:54.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.470+0000 7ff1b8829740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x560128b7e2e0 con 0x560128c27a90 2024-09-12T02:04:54.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.470+0000 7ff1b8829740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x560128b25ce0 con 0x560128c18380 2024-09-12T02:04:54.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.470+0000 7ff1b8829740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x560128b7a2a0 con 0x560128c18ca0 2024-09-12T02:04:54.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.470+0000 7ff1af7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560128c27a90 0x560128c29e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:54.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.470+0000 7ff1ad7fa640 1 --2- 172.21.15.16:0/2614579992 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7ff1980a2b80 0x7ff1980a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:54.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.470+0000 7ff1af7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560128c27a90 0x560128c29e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57782/0 (socket says 172.21.15.16:57782) 2024-09-12T02:04:54.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.470+0000 7ff1af7fe640 1 -- 172.21.15.16:0/972424574 learned_addr learned my addr 172.21.15.16:0/972424574 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:04:54.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.470+0000 7ff1affff640 1 --2- 172.21.15.16:0/972424574 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560128c18ca0 0x560128c27550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:54.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.470+0000 7ff1b49ed640 1 --2- 172.21.15.16:0/972424574 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560128c18380 0x560128c18760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:54.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.470+0000 7ff1b49ed640 1 --2- 172.21.15.16:0/2614579992 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7ff1980a2b80 0x7ff1980a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:54.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.470+0000 7ff1ad7fa640 1 -- 172.21.15.16:0/2614579992 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7ff1a40ef490 con 0x7ff1b0101420 2024-09-12T02:04:54.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.470+0000 7ff1af7fe640 1 -- 172.21.15.16:0/972424574 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560128c18ca0 msgr2=0x560128c27550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:54.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.470+0000 7ff1af7fe640 1 --2- 172.21.15.16:0/972424574 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560128c18ca0 0x560128c27550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:54.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.470+0000 7ff1af7fe640 1 -- 172.21.15.16:0/972424574 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560128c18380 msgr2=0x560128c18760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:54.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.470+0000 7ff1af7fe640 1 --2- 172.21.15.16:0/972424574 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560128c18380 0x560128c18760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:54.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.470+0000 7ff1af7fe640 1 -- 172.21.15.16:0/972424574 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560128b0a440 con 0x560128c27a90 2024-09-12T02:04:54.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.471+0000 7ff1af7fe640 1 --2- 172.21.15.16:0/972424574 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560128c27a90 0x560128c29e80 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7ff1a40b8760 tx=0x7ff1a4005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32a061f3347c6078 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:04:54.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.471+0000 7ff1b49ed640 1 --2- 172.21.15.16:0/2614579992 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7ff1980a2b80 0x7ff1980a5040 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7ff1a8007fd0 tx=0x7ff1a8007520 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:04:54.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.471+0000 7ff1967fc640 1 -- 172.21.15.16:0/972424574 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1a4049070 con 0x560128c27a90 2024-09-12T02:04:54.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.471+0000 7ff1967fc640 1 -- 172.21.15.16:0/972424574 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff1a40bc490 con 0x560128c27a90 2024-09-12T02:04:54.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.471+0000 7ff1967fc640 1 -- 172.21.15.16:0/972424574 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1a40b8400 con 0x560128c27a90 2024-09-12T02:04:54.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.471+0000 7ff1b8829740 1 -- 172.21.15.16:0/972424574 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560128c27a90 msgr2=0x560128c29e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:54.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.471+0000 7ff1b8829740 1 --2- 172.21.15.16:0/972424574 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560128c27a90 0x560128c29e80 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7ff1a40b8760 tx=0x7ff1a4005e50 comp rx=0 tx=0).stop 2024-09-12T02:04:54.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.471+0000 7ff1b8829740 1 -- 172.21.15.16:0/972424574 shutdown_connections 2024-09-12T02:04:54.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.471+0000 7ff1b8829740 1 --2- 172.21.15.16:0/972424574 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560128c27a90 0x560128c29e80 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:54.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.471+0000 7ff1b8829740 1 --2- 172.21.15.16:0/972424574 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560128c18ca0 0x560128c27550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:54.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.471+0000 7ff1b8829740 1 --2- 172.21.15.16:0/972424574 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560128c18380 0x560128c18760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:54.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.471+0000 7ff1b8829740 1 -- 172.21.15.16:0/972424574 >> 172.21.15.16:0/972424574 conn(0x560128c14a30 msgr2=0x560128c14e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:04:54.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.472+0000 7ff1b8829740 1 -- 172.21.15.16:0/972424574 shutdown_connections 2024-09-12T02:04:54.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.472+0000 7ff1b8829740 1 -- 172.21.15.16:0/972424574 wait complete. 2024-09-12T02:04:54.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.472+0000 7ff1b8829740 1 -- start start 2024-09-12T02:04:54.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.472+0000 7ff1b8829740 1 Processor -- start 2024-09-12T02:04:54.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.472+0000 7ff1b8829740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560128c18380 0x560128c27450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:54.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.472+0000 7ff1b8829740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560128c18ca0 0x560128c24700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:54.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.472+0000 7ff1b8829740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560128c27a90 0x560128c24cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:54.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.472+0000 7ff1b8829740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x560128c2a420 con 0x560128c18ca0 2024-09-12T02:04:54.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.472+0000 7ff1b8829740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x560128b39050 con 0x560128c18380 2024-09-12T02:04:54.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.472+0000 7ff1b8829740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x560128c2a5a0 con 0x560128c27a90 2024-09-12T02:04:54.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.472+0000 7ff1b8829740 10 client.0 _ll_register_callbacks cb 0x560128b79160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:04:54.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.472+0000 7ff1af7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560128c18ca0 0x560128c24700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:54.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.473+0000 7ff1af7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560128c18ca0 0x560128c24700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57796/0 (socket says 172.21.15.16:57796) 2024-09-12T02:04:54.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.473+0000 7ff1af7fe640 1 -- 172.21.15.16:0/2225710656 learned_addr learned my addr 172.21.15.16:0/2225710656 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:04:54.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.473+0000 7ff1affff640 1 --2- 172.21.15.16:0/2225710656 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560128c18380 0x560128c27450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:54.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.473+0000 7ff1b49ed640 1 --2- 172.21.15.16:0/2225710656 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560128c27a90 0x560128c24cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:54.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.473+0000 7ff1af7fe640 1 -- 172.21.15.16:0/2225710656 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560128c27a90 msgr2=0x560128c24cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:54.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.473+0000 7ff1af7fe640 1 --2- 172.21.15.16:0/2225710656 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560128c27a90 0x560128c24cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:54.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.473+0000 7ff1af7fe640 1 -- 172.21.15.16:0/2225710656 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560128c18380 msgr2=0x560128c27450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:54.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.473+0000 7ff1af7fe640 1 --2- 172.21.15.16:0/2225710656 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560128c18380 0x560128c27450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:54.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.473+0000 7ff1af7fe640 1 -- 172.21.15.16:0/2225710656 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5601288ef3c0 con 0x560128c18ca0 2024-09-12T02:04:54.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.473+0000 7ff1affff640 1 --2- 172.21.15.16:0/2225710656 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560128c18380 0x560128c27450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:04:54.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.473+0000 7ff1b49ed640 1 --2- 172.21.15.16:0/2225710656 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560128c27a90 0x560128c24cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:04:54.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.473+0000 7ff1af7fe640 1 --2- 172.21.15.16:0/2225710656 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560128c18ca0 0x560128c24700 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7ff1a403d040 tx=0x7ff1a40bbe50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27841a19058b020f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:04:54.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.473+0000 7ff18f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:04:54.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.473+0000 7ff18f7fe640 1 -- 172.21.15.16:0/2225710656 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1a4049070 con 0x560128c18ca0 2024-09-12T02:04:54.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.473+0000 7ff18f7fe640 1 -- 172.21.15.16:0/2225710656 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7ff1a40b8400 con 0x560128c18ca0 2024-09-12T02:04:54.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.473+0000 7ff1b8829740 1 -- 172.21.15.16:0/2225710656 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff1a40bc490 con 0x560128c18ca0 2024-09-12T02:04:54.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.473+0000 7ff18f7fe640 1 -- 172.21.15.16:0/2225710656 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff1a40b8030 con 0x560128c18ca0 2024-09-12T02:04:54.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.473+0000 7ff1b8829740 10 client.16257 fetch_fsmap learned FSMap version 25 2024-09-12T02:04:54.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.473+0000 7ff1b8829740 1 -- 172.21.15.16:0/2225710656 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x560128c2a420 con 0x560128c18ca0 2024-09-12T02:04:54.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.473+0000 7ff18f7fe640 1 -- 172.21.15.16:0/2225710656 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff1a40bc490 con 0x560128c18ca0 2024-09-12T02:04:54.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.474+0000 7ff18f7fe640 1 -- 172.21.15.16:0/2225710656 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7ff1a4031bd0 con 0x560128c18ca0 2024-09-12T02:04:54.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.474+0000 7ff1b8829740 10 client.16257 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:04:54.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.474+0000 7ff1b8829740 10 client.16257 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:04:54.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.474+0000 7ff1b8829740 20 client.16257 populate_metadata read hostname 'smithi016' 2024-09-12T02:04:54.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.474+0000 7ff1b8829740 1 --2- 172.21.15.16:0/2225710656 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x560128c370a0 0x560128c394c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:54.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.474+0000 7ff1b8829740 4 client.16257 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:04:54.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.474+0000 7ff1b8829740 1 -- 172.21.15.16:0/2225710656 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560128b8b850 con 0x560128c370a0 2024-09-12T02:04:54.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.474+0000 7ff1affff640 1 --2- 172.21.15.16:0/2225710656 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x560128c370a0 0x560128c394c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:54.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.474+0000 7ff1affff640 1 --2- 172.21.15.16:0/2225710656 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x560128c370a0 0x560128c394c0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63f313ccc7670d61 server_cookie=3383e7a763ac650f in_seq=0 out_seq=0 2024-09-12T02:04:54.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.474+0000 7ff18f7fe640 10 client.16257 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:04:54.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.475+0000 7ff18f7fe640 1 -- 172.21.15.16:0/2225710656 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560128b8b850 con 0x560128c370a0 2024-09-12T02:04:54.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.475+0000 7ff18f7fe640 10 client.16257 handle_command_reply: tid=0 2024-09-12T02:04:54.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.475+0000 7ff1b8829740 1 client.16257 shutdown 2024-09-12T02:04:54.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.475+0000 7ff1b8829740 1 -- 172.21.15.16:0/2225710656 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560128c18ca0 msgr2=0x560128c24700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:54.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.475+0000 7ff1b8829740 1 --2- 172.21.15.16:0/2225710656 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560128c18ca0 0x560128c24700 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7ff1a403d040 tx=0x7ff1a40bbe50 comp rx=0 tx=0).stop 2024-09-12T02:04:54.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.475+0000 7ff1b8829740 1 -- 172.21.15.16:0/2225710656 shutdown_connections 2024-09-12T02:04:54.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.475+0000 7ff1b8829740 1 --2- 172.21.15.16:0/2225710656 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x560128c370a0 0x560128c394c0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:54.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.475+0000 7ff1b8829740 1 --2- 172.21.15.16:0/2225710656 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560128c27a90 0x560128c24cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:54.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.475+0000 7ff1b8829740 1 --2- 172.21.15.16:0/2225710656 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560128c18ca0 0x560128c24700 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:54.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.475+0000 7ff1b8829740 1 --2- 172.21.15.16:0/2225710656 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560128c18380 0x560128c27450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:54.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.475+0000 7ff1b8829740 1 -- 172.21.15.16:0/2225710656 >> 172.21.15.16:0/2225710656 conn(0x560128c14a30 msgr2=0x560128c1a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:04:54.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.475+0000 7ff18f7fe640 0 client.16257 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:04:54.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.475+0000 7ff1b8829740 1 -- 172.21.15.16:0/2225710656 shutdown_connections 2024-09-12T02:04:54.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.475+0000 7ff1b8829740 1 -- 172.21.15.16:0/2225710656 wait complete. 2024-09-12T02:04:54.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.476+0000 7ff1b8829740 20 client.16257 trim_cache size 0 max 16384 2024-09-12T02:04:54.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.493+0000 7ff1b8829740 1 Processor -- start 2024-09-12T02:04:54.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.493+0000 7ff1b8829740 1 -- start start 2024-09-12T02:04:54.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.493+0000 7ff1b8829740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560128c18380 0x560128c335f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:54.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.493+0000 7ff1b8829740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560128c18ca0 0x560128c13a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:54.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.493+0000 7ff1b8829740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560128c27a90 0x560128c3ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:54.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.493+0000 7ff1b8829740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff1b01ad9d0 con 0x560128c27a90 2024-09-12T02:04:54.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.493+0000 7ff1b8829740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff1b01ab2a0 con 0x560128c18380 2024-09-12T02:04:54.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.493+0000 7ff1b8829740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff1b01a9130 con 0x560128c18ca0 2024-09-12T02:04:54.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.493+0000 7ff1b49ed640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560128c27a90 0x560128c3ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:54.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.493+0000 7ff1b49ed640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560128c27a90 0x560128c3ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57810/0 (socket says 172.21.15.16:57810) 2024-09-12T02:04:54.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.493+0000 7ff1b49ed640 1 -- 172.21.15.16:0/2256063674 learned_addr learned my addr 172.21.15.16:0/2256063674 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:04:54.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.494+0000 7ff1b49ed640 1 -- 172.21.15.16:0/2256063674 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560128c18ca0 msgr2=0x560128c13a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:54.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.494+0000 7ff1af7fe640 1 --2- 172.21.15.16:0/2256063674 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560128c18ca0 0x560128c13a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:54.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.494+0000 7ff1affff640 1 --2- 172.21.15.16:0/2256063674 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560128c18380 0x560128c335f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:54.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.494+0000 7ff1b49ed640 1 --2- 172.21.15.16:0/2256063674 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560128c18ca0 0x560128c13a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:54.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.494+0000 7ff1b49ed640 1 -- 172.21.15.16:0/2256063674 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560128c18380 msgr2=0x560128c335f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:54.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.494+0000 7ff1b49ed640 1 --2- 172.21.15.16:0/2256063674 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560128c18380 0x560128c335f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:54.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.494+0000 7ff1b49ed640 1 -- 172.21.15.16:0/2256063674 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560128c25060 con 0x560128c27a90 2024-09-12T02:04:54.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.494+0000 7ff1af7fe640 1 --2- 172.21.15.16:0/2256063674 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560128c18ca0 0x560128c13a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:04:54.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.494+0000 7ff1affff640 1 --2- 172.21.15.16:0/2256063674 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560128c18380 0x560128c335f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:04:54.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.494+0000 7ff1b49ed640 1 --2- 172.21.15.16:0/2256063674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560128c27a90 0x560128c3ed90 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7ff1a8012010 tx=0x7ff1a8007940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f526a111a1ab3e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:04:54.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.494+0000 7ff195ffb640 1 -- 172.21.15.16:0/2256063674 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1a8025030 con 0x560128c27a90 2024-09-12T02:04:54.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.494+0000 7ff195ffb640 1 -- 172.21.15.16:0/2256063674 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff1a8007c00 con 0x560128c27a90 2024-09-12T02:04:54.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.494+0000 7ff195ffb640 1 -- 172.21.15.16:0/2256063674 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1a80056f0 con 0x560128c27a90 2024-09-12T02:04:54.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.495+0000 7ff1b8829740 1 -- 172.21.15.16:0/2256063674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560128c27a90 msgr2=0x560128c3ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:54.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.495+0000 7ff1b8829740 1 --2- 172.21.15.16:0/2256063674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560128c27a90 0x560128c3ed90 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7ff1a8012010 tx=0x7ff1a8007940 comp rx=0 tx=0).stop 2024-09-12T02:04:54.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.495+0000 7ff1b8829740 1 -- 172.21.15.16:0/2256063674 shutdown_connections 2024-09-12T02:04:54.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.495+0000 7ff1b8829740 1 --2- 172.21.15.16:0/2256063674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560128c27a90 0x560128c3ed90 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:54.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.495+0000 7ff1b8829740 1 --2- 172.21.15.16:0/2256063674 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560128c18ca0 0x560128c13a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:54.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.495+0000 7ff1b8829740 1 --2- 172.21.15.16:0/2256063674 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560128c18380 0x560128c335f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:54.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.495+0000 7ff1b8829740 1 -- 172.21.15.16:0/2256063674 >> 172.21.15.16:0/2256063674 conn(0x560128c14a30 msgr2=0x560128c1fcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:04:54.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.495+0000 7ff1b8829740 1 -- 172.21.15.16:0/2256063674 shutdown_connections 2024-09-12T02:04:54.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.495+0000 7ff1b8829740 1 -- 172.21.15.16:0/2256063674 wait complete. 2024-09-12T02:04:54.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.495+0000 7ff1b8829740 1 -- start start 2024-09-12T02:04:54.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.495+0000 7ff1b8829740 1 Processor -- start 2024-09-12T02:04:54.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.496+0000 7ff1b8829740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560128c18380 0x560128c2fc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:54.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.496+0000 7ff1b8829740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560128c18ca0 0x560128c2af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:54.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.496+0000 7ff1b8829740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560128c27a90 0x560128c2b480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:54.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.496+0000 7ff1b8829740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff1b01a9780 con 0x560128c18ca0 2024-09-12T02:04:54.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.496+0000 7ff1b8829740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff1b01aabd0 con 0x560128c27a90 2024-09-12T02:04:54.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.496+0000 7ff1b8829740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff1b01a9e30 con 0x560128c18380 2024-09-12T02:04:54.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.496+0000 7ff1b8829740 10 client.0 _ll_register_callbacks cb 0x7ff1a8007e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:04:54.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.496+0000 7ff1af7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560128c18ca0 0x560128c2af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:54.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.496+0000 7ff1af7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560128c18ca0 0x560128c2af40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57818/0 (socket says 172.21.15.16:57818) 2024-09-12T02:04:54.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.496+0000 7ff1af7fe640 1 -- 172.21.15.16:0/1517854107 learned_addr learned my addr 172.21.15.16:0/1517854107 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:04:54.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.496+0000 7ff1affff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560128c18380 0x560128c2fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:54.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.496+0000 7ff1b49ed640 1 --2- 172.21.15.16:0/1517854107 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560128c27a90 0x560128c2b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:54.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.496+0000 7ff1af7fe640 1 -- 172.21.15.16:0/1517854107 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560128c18380 msgr2=0x560128c2fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:54.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.496+0000 7ff1af7fe640 1 --2- 172.21.15.16:0/1517854107 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560128c18380 0x560128c2fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:54.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.496+0000 7ff1af7fe640 1 -- 172.21.15.16:0/1517854107 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560128c27a90 msgr2=0x560128c2b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:54.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.496+0000 7ff1af7fe640 1 --2- 172.21.15.16:0/1517854107 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560128c27a90 0x560128c2b480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:54.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.496+0000 7ff1af7fe640 1 -- 172.21.15.16:0/1517854107 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560128c2b9f0 con 0x560128c18ca0 2024-09-12T02:04:54.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.496+0000 7ff1affff640 1 --2- 172.21.15.16:0/1517854107 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560128c18380 0x560128c2fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:04:54.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.496+0000 7ff1af7fe640 1 --2- 172.21.15.16:0/1517854107 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560128c18ca0 0x560128c2af40 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7ff1a40b8760 tx=0x7ff1a403bd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2193aaa84538c5f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:04:54.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.496+0000 7ff18f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:04:54.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.496+0000 7ff18f7fe640 1 -- 172.21.15.16:0/1517854107 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1a4049070 con 0x560128c18ca0 2024-09-12T02:04:54.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.496+0000 7ff18f7fe640 1 -- 172.21.15.16:0/1517854107 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7ff1a4031bd0 con 0x560128c18ca0 2024-09-12T02:04:54.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.497+0000 7ff1b8829740 1 -- 172.21.15.16:0/1517854107 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff1a8007c00 con 0x560128c18ca0 2024-09-12T02:04:54.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.497+0000 7ff18f7fe640 1 -- 172.21.15.16:0/1517854107 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff1a403be60 con 0x560128c18ca0 2024-09-12T02:04:54.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.497+0000 7ff1b8829740 10 client.16263 fetch_fsmap learned FSMap version 25 2024-09-12T02:04:54.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.497+0000 7ff1b8829740 1 -- 172.21.15.16:0/1517854107 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x560128c2b9f0 con 0x560128c18ca0 2024-09-12T02:04:54.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.497+0000 7ff18f7fe640 1 -- 172.21.15.16:0/1517854107 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff1a8007c00 con 0x560128c18ca0 2024-09-12T02:04:54.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.497+0000 7ff18f7fe640 1 -- 172.21.15.16:0/1517854107 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7ff1a40bb1e0 con 0x560128c18ca0 2024-09-12T02:04:54.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.497+0000 7ff1b8829740 10 client.16263 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:04:54.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.497+0000 7ff1b8829740 10 client.16263 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:04:54.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.497+0000 7ff1b8829740 20 client.16263 populate_metadata read hostname 'smithi016' 2024-09-12T02:04:54.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.497+0000 7ff1b8829740 1 --2- 172.21.15.16:0/1517854107 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x560128ca1e60 0x560128ca4280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:54.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.497+0000 7ff1b8829740 4 client.16263 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T02:04:54.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.497+0000 7ff1b8829740 1 -- 172.21.15.16:0/1517854107 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560128b85d60 con 0x560128ca1e60 2024-09-12T02:04:54.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.497+0000 7ff1affff640 1 --2- 172.21.15.16:0/1517854107 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x560128ca1e60 0x560128ca4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:54.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.498+0000 7ff18f7fe640 10 client.16263 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:04:54.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.498+0000 7ff1affff640 1 --2- 172.21.15.16:0/1517854107 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x560128ca1e60 0x560128ca4280 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fce51858d5b26a45 server_cookie=7ad4edb577325d34 in_seq=0 out_seq=0 2024-09-12T02:04:54.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.498+0000 7ff18f7fe640 1 -- 172.21.15.16:0/1517854107 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x560128b85d60 con 0x560128ca1e60 2024-09-12T02:04:54.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.498+0000 7ff18f7fe640 10 client.16263 handle_command_reply: tid=0 2024-09-12T02:04:54.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.498+0000 7ff1b8829740 1 client.16263 shutdown 2024-09-12T02:04:54.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.498+0000 7ff1b8829740 1 -- 172.21.15.16:0/1517854107 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560128c18ca0 msgr2=0x560128c2af40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:54.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.498+0000 7ff1b8829740 1 --2- 172.21.15.16:0/1517854107 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560128c18ca0 0x560128c2af40 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7ff1a40b8760 tx=0x7ff1a403bd00 comp rx=0 tx=0).stop 2024-09-12T02:04:54.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.498+0000 7ff1b8829740 1 -- 172.21.15.16:0/1517854107 shutdown_connections 2024-09-12T02:04:54.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.498+0000 7ff1b8829740 1 --2- 172.21.15.16:0/1517854107 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x560128ca1e60 0x560128ca4280 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:54.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.498+0000 7ff1b8829740 1 --2- 172.21.15.16:0/1517854107 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560128c27a90 0x560128c2b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:54.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.498+0000 7ff1b8829740 1 --2- 172.21.15.16:0/1517854107 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560128c18ca0 0x560128c2af40 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:54.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.498+0000 7ff1b8829740 1 --2- 172.21.15.16:0/1517854107 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560128c18380 0x560128c2fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:54.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.498+0000 7ff18f7fe640 0 client.16263 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:04:54.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.498+0000 7ff1b8829740 1 -- 172.21.15.16:0/1517854107 >> 172.21.15.16:0/1517854107 conn(0x560128c14a30 msgr2=0x560128c1a6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:04:54.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.498+0000 7ff1b8829740 1 -- 172.21.15.16:0/1517854107 shutdown_connections 2024-09-12T02:04:54.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.498+0000 7ff1b8829740 1 -- 172.21.15.16:0/1517854107 wait complete. 2024-09-12T02:04:54.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.499+0000 7ff1b8829740 20 client.16263 trim_cache size 0 max 16384 2024-09-12T02:04:54.513 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T02:04:54.513 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-09-12T02:04:54.513 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {} 2024-09-12T02:04:54.514 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T02:04:54.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.499+0000 7ff1b6450640 1 -- 172.21.15.16:0/2614579992 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7ff1980a2b80 msgr2=0x7ff1980a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:54.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.499+0000 7ff1b6450640 1 --2- 172.21.15.16:0/2614579992 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7ff1980a2b80 0x7ff1980a5040 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7ff1a8007fd0 tx=0x7ff1a8007520 comp rx=0 tx=0).stop 2024-09-12T02:04:54.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.499+0000 7ff1b6450640 1 -- 172.21.15.16:0/2614579992 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1b0101420 msgr2=0x7ff1b0198980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:54.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.499+0000 7ff1b6450640 1 --2- 172.21.15.16:0/2614579992 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1b0101420 0x7ff1b0198980 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7ff1a4009230 tx=0x7ff1a4004270 comp rx=0 tx=0).stop 2024-09-12T02:04:54.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.499+0000 7ff1b6450640 1 -- 172.21.15.16:0/2614579992 shutdown_connections 2024-09-12T02:04:54.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.499+0000 7ff1b6450640 1 --2- 172.21.15.16:0/2614579992 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7ff1980a2b80 0x7ff1980a5040 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:54.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.499+0000 7ff1b6450640 1 --2- 172.21.15.16:0/2614579992 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1b0106a80 0x7ff1b019cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:54.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.499+0000 7ff1b6450640 1 --2- 172.21.15.16:0/2614579992 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1b0101420 0x7ff1b0198980 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:54.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.499+0000 7ff1b6450640 1 --2- 172.21.15.16:0/2614579992 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff1b0100a80 0x7ff1b0198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:54.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.499+0000 7ff1b6450640 1 -- 172.21.15.16:0/2614579992 >> 172.21.15.16:0/2614579992 conn(0x7ff1b00fc7a0 msgr2=0x7ff1b006aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:04:54.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.499+0000 7ff1b6450640 1 -- 172.21.15.16:0/2614579992 shutdown_connections 2024-09-12T02:04:54.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:54.499+0000 7ff1b6450640 1 -- 172.21.15.16:0/2614579992 wait complete. 2024-09-12T02:04:55.303 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-12T02:04:55.303 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:04:55.565 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:04:56.006 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:04:55 smithi016 ceph-mon[30947]: pgmap v425: 97 pgs: 97 active+clean; 1.6 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 16 KiB/s wr, 14 op/s 2024-09-12T02:04:56.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:56.005+0000 7f66fb5ec640 1 -- 172.21.15.16:0/809521676 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66f41082e0 msgr2=0x7f66f410ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:56.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:56.005+0000 7f66fb5ec640 1 --2- 172.21.15.16:0/809521676 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66f41082e0 0x7f66f410ec50 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f66ec009cd0 tx=0x7f66ec02a8e0 comp rx=0 tx=0).stop 2024-09-12T02:04:56.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:56.005+0000 7f66fb5ec640 1 -- 172.21.15.16:0/809521676 shutdown_connections 2024-09-12T02:04:56.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:56.005+0000 7f66fb5ec640 1 --2- 172.21.15.16:0/809521676 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66f41082e0 0x7f66f410ec50 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:56.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:56.005+0000 7f66fb5ec640 1 --2- 172.21.15.16:0/809521676 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f66f4103760 0x7f66f4107ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:56.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:56.005+0000 7f66fb5ec640 1 --2- 172.21.15.16:0/809521676 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66f4102db0 0x7f66f4103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:56.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:56.005+0000 7f66fb5ec640 1 -- 172.21.15.16:0/809521676 >> 172.21.15.16:0/809521676 conn(0x7f66f40fc7d0 msgr2=0x7f66f40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:04:56.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:56.005+0000 7f66fb5ec640 1 -- 172.21.15.16:0/809521676 shutdown_connections 2024-09-12T02:04:56.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:56.005+0000 7f66fb5ec640 1 -- 172.21.15.16:0/809521676 wait complete. 2024-09-12T02:04:56.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:56.006+0000 7f66fb5ec640 1 Processor -- start 2024-09-12T02:04:56.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:56.006+0000 7f66fb5ec640 1 -- start start 2024-09-12T02:04:56.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:56.006+0000 7f66fb5ec640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66f4102db0 0x7f66f4100a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:56.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:56.006+0000 7f66fb5ec640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66f4103760 0x7f66f4100f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:56.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:56.006+0000 7f66fb5ec640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f66f41082e0 0x7f66f4101490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:56.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:56.006+0000 7f66fb5ec640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f66f41120b0 con 0x7f66f4102db0 2024-09-12T02:04:56.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:56.006+0000 7f66fb5ec640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f66f4111f30 con 0x7f66f4103760 2024-09-12T02:04:56.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:56.006+0000 7f66fb5ec640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f66f4112230 con 0x7f66f41082e0 2024-09-12T02:04:56.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:56.006+0000 7f66f9388640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66f4102db0 0x7f66f4100a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:56.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:56.006+0000 7f66f9388640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66f4102db0 0x7f66f4100a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57830/0 (socket says 172.21.15.16:57830) 2024-09-12T02:04:56.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:56.006+0000 7f66f9388640 1 -- 172.21.15.16:0/1050222181 learned_addr learned my addr 172.21.15.16:0/1050222181 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:04:56.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:56.006+0000 7f66f8b87640 1 --2- 172.21.15.16:0/1050222181 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66f4103760 0x7f66f4100f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:56.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:56.006+0000 7f66f9b89640 1 --2- 172.21.15.16:0/1050222181 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f66f41082e0 0x7f66f4101490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:56.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:56.007+0000 7f66f9388640 1 -- 172.21.15.16:0/1050222181 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f66f41082e0 msgr2=0x7f66f4101490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:56.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:56.007+0000 7f66f9388640 1 --2- 172.21.15.16:0/1050222181 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f66f41082e0 0x7f66f4101490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:56.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:56.007+0000 7f66f9388640 1 -- 172.21.15.16:0/1050222181 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66f4103760 msgr2=0x7f66f4100f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:56.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:56.007+0000 7f66f9388640 1 --2- 172.21.15.16:0/1050222181 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66f4103760 0x7f66f4100f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:56.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:56.007+0000 7f66f9388640 1 -- 172.21.15.16:0/1050222181 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66f41a6a10 con 0x7f66f4102db0 2024-09-12T02:04:56.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:56.007+0000 7f66f9b89640 1 --2- 172.21.15.16:0/1050222181 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f66f41082e0 0x7f66f4101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:04:56.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:56.007+0000 7f66f9388640 1 --2- 172.21.15.16:0/1050222181 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66f4102db0 0x7f66f4100a10 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f66f000d950 tx=0x7f66f000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:04:56.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:56.007+0000 7f66ea7fc640 1 -- 172.21.15.16:0/1050222181 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66f0017070 con 0x7f66f4102db0 2024-09-12T02:04:56.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:56.007+0000 7f66ea7fc640 1 -- 172.21.15.16:0/1050222181 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f66f0009c30 con 0x7f66f4102db0 2024-09-12T02:04:56.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:56.007+0000 7f66ea7fc640 1 -- 172.21.15.16:0/1050222181 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66f000f6b0 con 0x7f66f4102db0 2024-09-12T02:04:56.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:56.007+0000 7f66fb5ec640 1 -- 172.21.15.16:0/1050222181 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66f41a6d00 con 0x7f66f4102db0 2024-09-12T02:04:56.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:56.007+0000 7f66fb5ec640 1 -- 172.21.15.16:0/1050222181 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66f41a7190 con 0x7f66f4102db0 2024-09-12T02:04:56.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:56.008+0000 7f66fb5ec640 1 -- 172.21.15.16:0/1050222181 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66f4103190 con 0x7f66f4102db0 2024-09-12T02:04:56.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:56.009+0000 7f66ea7fc640 1 -- 172.21.15.16:0/1050222181 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f66f00109d0 con 0x7f66f4102db0 2024-09-12T02:04:56.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:56.010+0000 7f66ea7fc640 1 --2- 172.21.15.16:0/1050222181 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f66e00a2c00 0x7f66e00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:56.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:56.010+0000 7f66f8b87640 1 --2- 172.21.15.16:0/1050222181 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f66e00a2c00 0x7f66e00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:56.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:56.010+0000 7f66ea7fc640 1 -- 172.21.15.16:0/1050222181 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f66f00ceb30 con 0x7f66f4102db0 2024-09-12T02:04:56.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:56.010+0000 7f66f8b87640 1 --2- 172.21.15.16:0/1050222181 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f66e00a2c00 0x7f66e00a50c0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f66f41024a0 tx=0x7f66e400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:04:56.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:56.014+0000 7f66ea7fc640 1 -- 172.21.15.16:0/1050222181 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f66f0009de0 con 0x7f66f4102db0 2024-09-12T02:04:56.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:56.224+0000 7f66fb5ec640 1 -- 172.21.15.16:0/1050222181 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f66f410a2c0 con 0x7f66f4102db0 2024-09-12T02:04:56.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:56.225+0000 7f66ea7fc640 1 -- 172.21.15.16:0/1050222181 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11911 (secure 0 0 0) 0x7f66f009bb30 con 0x7f66f4102db0 2024-09-12T02:04:56.226 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T02:04:56.226 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":25,"btime":"2024-09-12T02:01:37:192021+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3279123279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3279123279},{"type":"v1","addr":"172.21.15.16:6835","nonce":3279123279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4119452382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4119452382},{"type":"v1","addr":"172.21.15.16:6837","nonce":4119452382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/3263823165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3263823165},{"type":"v1","addr":"172.21.15.16:6839","nonce":3263823165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/1510290732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":1510290732},{"type":"v1","addr":"172.21.15.16:6841","nonce":1510290732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/279674603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":279674603},{"type":"v1","addr":"172.21.15.33:6833","nonce":279674603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3555728028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3555728028},{"type":"v1","addr":"172.21.15.33:6835","nonce":3555728028}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T01:59:56.117608+0000","modified":"2024-09-12T02:01:36.294760+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24482,"mds_1":24481,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6833/1563374106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1563374106},{"type":"v1","addr":"172.21.15.184:6833","nonce":1563374106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6837/1892833868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1892833868},{"type":"v1","addr":"172.21.15.33:6837","nonce":1892833868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/1457679686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1457679686},{"type":"v1","addr":"172.21.15.184:6835","nonce":1457679686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.184:6837/1793393087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":1793393087},{"type":"v1","addr":"172.21.15.184:6837","nonce":1793393087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.184:6839/4203347941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":4203347941},{"type":"v1","addr":"172.21.15.184:6839","nonce":4203347941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.33:6839/4136670204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":4136670204},{"type":"v1","addr":"172.21.15.33:6839","nonce":4136670204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24481,24482,24469]},"id":1}]} 2024-09-12T02:04:56.227 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 25 2024-09-12T02:04:56.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:56.227+0000 7f66fb5ec640 1 -- 172.21.15.16:0/1050222181 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f66e00a2c00 msgr2=0x7f66e00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:56.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:56.227+0000 7f66fb5ec640 1 --2- 172.21.15.16:0/1050222181 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f66e00a2c00 0x7f66e00a50c0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f66f41024a0 tx=0x7f66e400b390 comp rx=0 tx=0).stop 2024-09-12T02:04:56.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:56.227+0000 7f66fb5ec640 1 -- 172.21.15.16:0/1050222181 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66f4102db0 msgr2=0x7f66f4100a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:56.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:56.227+0000 7f66fb5ec640 1 --2- 172.21.15.16:0/1050222181 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66f4102db0 0x7f66f4100a10 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f66f000d950 tx=0x7f66f000de10 comp rx=0 tx=0).stop 2024-09-12T02:04:56.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:56.227+0000 7f66fb5ec640 1 -- 172.21.15.16:0/1050222181 shutdown_connections 2024-09-12T02:04:56.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:56.227+0000 7f66fb5ec640 1 --2- 172.21.15.16:0/1050222181 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f66e00a2c00 0x7f66e00a50c0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:56.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:56.227+0000 7f66fb5ec640 1 --2- 172.21.15.16:0/1050222181 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f66f41082e0 0x7f66f4101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:56.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:56.227+0000 7f66fb5ec640 1 --2- 172.21.15.16:0/1050222181 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66f4103760 0x7f66f4100f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:56.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:56.227+0000 7f66fb5ec640 1 --2- 172.21.15.16:0/1050222181 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66f4102db0 0x7f66f4100a10 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:56.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:56.227+0000 7f66fb5ec640 1 -- 172.21.15.16:0/1050222181 >> 172.21.15.16:0/1050222181 conn(0x7f66f40fc7d0 msgr2=0x7f66f4105d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:04:56.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:56.227+0000 7f66fb5ec640 1 -- 172.21.15.16:0/1050222181 shutdown_connections 2024-09-12T02:04:56.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:56.227+0000 7f66fb5ec640 1 -- 172.21.15.16:0/1050222181 wait complete. 2024-09-12T02:04:56.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:04:55 smithi184 ceph-mon[35593]: pgmap v425: 97 pgs: 97 active+clean; 1.6 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 16 KiB/s wr, 14 op/s 2024-09-12T02:04:56.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:04:55 smithi033 ceph-mon[35332]: pgmap v425: 97 pgs: 97 active+clean; 1.6 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 16 KiB/s wr, 14 op/s 2024-09-12T02:04:56.937 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-12T02:04:57.183 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:04:56 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/1050222181' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:04:57.198 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:04:57.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:04:56 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/1050222181' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:04:57.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:04:56 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/1050222181' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:04:57.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.631+0000 7fccd6095640 1 -- 172.21.15.16:0/3277729661 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fccd0102d40 msgr2=0x7fccd0103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:57.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.631+0000 7fccd6095640 1 --2- 172.21.15.16:0/3277729661 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fccd0102d40 0x7fccd0103120 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fccc0009900 tx=0x7fccc002a480 comp rx=0 tx=0).stop 2024-09-12T02:04:57.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.631+0000 7fccd6095640 1 -- 172.21.15.16:0/3277729661 shutdown_connections 2024-09-12T02:04:57.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.631+0000 7fccd6095640 1 --2- 172.21.15.16:0/3277729661 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fccd010c170 0x7fccd010e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:57.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.631+0000 7fccd6095640 1 --2- 172.21.15.16:0/3277729661 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fccd0103660 0x7fccd010bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:57.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.631+0000 7fccd6095640 1 --2- 172.21.15.16:0/3277729661 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fccd0102d40 0x7fccd0103120 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:57.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.631+0000 7fccd6095640 1 -- 172.21.15.16:0/3277729661 >> 172.21.15.16:0/3277729661 conn(0x7fccd00fc7a0 msgr2=0x7fccd00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:04:57.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.631+0000 7fccd6095640 1 -- 172.21.15.16:0/3277729661 shutdown_connections 2024-09-12T02:04:57.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.631+0000 7fccd6095640 1 -- 172.21.15.16:0/3277729661 wait complete. 2024-09-12T02:04:57.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.632+0000 7fccd6095640 1 Processor -- start 2024-09-12T02:04:57.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.632+0000 7fccd6095640 1 -- start start 2024-09-12T02:04:57.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.632+0000 7fccd6095640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fccd0102d40 0x7fccd01009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:57.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.633+0000 7fccd6095640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fccd0103660 0x7fccd0100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:57.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.633+0000 7fccd6095640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fccd010c170 0x7fccd0101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:57.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.633+0000 7fccd6095640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fccd0111f70 con 0x7fccd010c170 2024-09-12T02:04:57.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.633+0000 7fccd6095640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fccd0111df0 con 0x7fccd0103660 2024-09-12T02:04:57.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.633+0000 7fccd6095640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fccd01120f0 con 0x7fccd0102d40 2024-09-12T02:04:57.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.633+0000 7fcccffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fccd010c170 0x7fccd0101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:57.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.633+0000 7fcccffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fccd010c170 0x7fccd0101420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57846/0 (socket says 172.21.15.16:57846) 2024-09-12T02:04:57.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.633+0000 7fcccffff640 1 -- 172.21.15.16:0/2698234816 learned_addr learned my addr 172.21.15.16:0/2698234816 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:04:57.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.633+0000 7fccceffd640 1 --2- 172.21.15.16:0/2698234816 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fccd0103660 0x7fccd0100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:57.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.633+0000 7fcccf7fe640 1 --2- 172.21.15.16:0/2698234816 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fccd0102d40 0x7fccd01009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:57.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.633+0000 7fcccffff640 1 -- 172.21.15.16:0/2698234816 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fccd0102d40 msgr2=0x7fccd01009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:57.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.633+0000 7fcccffff640 1 --2- 172.21.15.16:0/2698234816 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fccd0102d40 0x7fccd01009a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:57.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.633+0000 7fcccffff640 1 -- 172.21.15.16:0/2698234816 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fccd0103660 msgr2=0x7fccd0100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:57.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.633+0000 7fcccffff640 1 --2- 172.21.15.16:0/2698234816 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fccd0103660 0x7fccd0100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:57.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.633+0000 7fcccffff640 1 -- 172.21.15.16:0/2698234816 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccd01a69c0 con 0x7fccd010c170 2024-09-12T02:04:57.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.633+0000 7fcccffff640 1 --2- 172.21.15.16:0/2698234816 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fccd010c170 0x7fccd0101420 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fccc800cad0 tx=0x7fccc800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:04:57.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.633+0000 7fccccff9640 1 -- 172.21.15.16:0/2698234816 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccc800ae20 con 0x7fccd010c170 2024-09-12T02:04:57.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.633+0000 7fccd6095640 1 -- 172.21.15.16:0/2698234816 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccd01a6cb0 con 0x7fccd010c170 2024-09-12T02:04:57.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.633+0000 7fccccff9640 1 -- 172.21.15.16:0/2698234816 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fccc8005020 con 0x7fccd010c170 2024-09-12T02:04:57.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.633+0000 7fccccff9640 1 -- 172.21.15.16:0/2698234816 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccc800f6f0 con 0x7fccd010c170 2024-09-12T02:04:57.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.633+0000 7fccd6095640 1 -- 172.21.15.16:0/2698234816 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccd01a71a0 con 0x7fccd010c170 2024-09-12T02:04:57.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.636+0000 7fccccff9640 1 -- 172.21.15.16:0/2698234816 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fccc80051d0 con 0x7fccd010c170 2024-09-12T02:04:57.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.637+0000 7fccd846e740 1 Processor -- start 2024-09-12T02:04:57.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.637+0000 7fccd846e740 1 -- start start 2024-09-12T02:04:57.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.637+0000 7fccd846e740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561686262b40 0x561686261090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:57.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.637+0000 7fccd846e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616862615d0 0x561686261a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:57.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.637+0000 7fccd846e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56168626c370 0x56168626e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:57.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.637+0000 7fccd846e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5616861c72f0 con 0x5616862615d0 2024-09-12T02:04:57.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.637+0000 7fccd846e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56168616ecf0 con 0x56168626c370 2024-09-12T02:04:57.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.637+0000 7fccd846e740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5616861c32b0 con 0x561686262b40 2024-09-12T02:04:57.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.637+0000 7fcccf7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616862615d0 0x561686261a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:57.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.637+0000 7fcccf7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616862615d0 0x561686261a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57854/0 (socket says 172.21.15.16:57854) 2024-09-12T02:04:57.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.637+0000 7fcccf7fe640 1 -- 172.21.15.16:0/3779604569 learned_addr learned my addr 172.21.15.16:0/3779604569 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:04:57.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.637+0000 7fccccff9640 1 --2- 172.21.15.16:0/2698234816 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fccb80a2b80 0x7fccb80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:57.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.637+0000 7fccceffd640 1 --2- 172.21.15.16:0/3779604569 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561686262b40 0x561686261090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:57.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.637+0000 7fccceffd640 1 --2- 172.21.15.16:0/3779604569 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56168626c370 0x56168626e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:57.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.637+0000 7fcccf7fe640 1 -- 172.21.15.16:0/3779604569 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561686262b40 msgr2=0x561686261090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:57.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.637+0000 7fcccf7fe640 1 --2- 172.21.15.16:0/3779604569 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561686262b40 0x561686261090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:57.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.637+0000 7fcccf7fe640 1 -- 172.21.15.16:0/3779604569 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56168626c370 msgr2=0x56168626e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:57.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.637+0000 7fcccf7fe640 1 --2- 172.21.15.16:0/3779604569 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56168626c370 0x56168626e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:57.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.637+0000 7fcccffff640 1 --2- 172.21.15.16:0/2698234816 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fccb80a2b80 0x7fccb80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:57.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.637+0000 7fcccf7fe640 1 -- 172.21.15.16:0/3779604569 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5616861533a0 con 0x5616862615d0 2024-09-12T02:04:57.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.637+0000 7fccccff9640 1 -- 172.21.15.16:0/2698234816 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fccc80cef90 con 0x7fccd010c170 2024-09-12T02:04:57.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.638+0000 7fcccf7fe640 1 --2- 172.21.15.16:0/3779604569 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616862615d0 0x561686261a10 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fccc0006010 tx=0x7fccc0031510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed6c9dc8b626fb21 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:04:57.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.638+0000 7fccb5ffb640 1 -- 172.21.15.16:0/3779604569 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccc003f070 con 0x5616862615d0 2024-09-12T02:04:57.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.638+0000 7fcccffff640 1 --2- 172.21.15.16:0/2698234816 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fccb80a2b80 0x7fccb80a5040 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fccc80a3fd0 tx=0x7fccc801baf0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:04:57.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.638+0000 7fccb5ffb640 1 -- 172.21.15.16:0/3779604569 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fccc00317b0 con 0x5616862615d0 2024-09-12T02:04:57.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.638+0000 7fccb5ffb640 1 -- 172.21.15.16:0/3779604569 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccc0035070 con 0x5616862615d0 2024-09-12T02:04:57.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.639+0000 7fccd846e740 1 -- 172.21.15.16:0/3779604569 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616862615d0 msgr2=0x561686261a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:57.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.639+0000 7fccd846e740 1 --2- 172.21.15.16:0/3779604569 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616862615d0 0x561686261a10 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fccc0006010 tx=0x7fccc0031510 comp rx=0 tx=0).stop 2024-09-12T02:04:57.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.639+0000 7fccd846e740 1 -- 172.21.15.16:0/3779604569 shutdown_connections 2024-09-12T02:04:57.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.639+0000 7fccd846e740 1 --2- 172.21.15.16:0/3779604569 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56168626c370 0x56168626e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:57.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.639+0000 7fccd846e740 1 --2- 172.21.15.16:0/3779604569 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616862615d0 0x561686261a10 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:57.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.639+0000 7fccd846e740 1 --2- 172.21.15.16:0/3779604569 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561686262b40 0x561686261090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:57.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.639+0000 7fccd846e740 1 -- 172.21.15.16:0/3779604569 >> 172.21.15.16:0/3779604569 conn(0x56168622a310 msgr2=0x56168625dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:04:57.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.639+0000 7fccd846e740 1 -- 172.21.15.16:0/3779604569 shutdown_connections 2024-09-12T02:04:57.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.639+0000 7fccd846e740 1 -- 172.21.15.16:0/3779604569 wait complete. 2024-09-12T02:04:57.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.639+0000 7fccd846e740 1 -- start start 2024-09-12T02:04:57.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.639+0000 7fccd846e740 1 Processor -- start 2024-09-12T02:04:57.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.640+0000 7fccd846e740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5616862615d0 0x56168626bd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:57.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.640+0000 7fccd846e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561686262b40 0x561686268fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:57.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.640+0000 7fccd846e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56168626c370 0x561686269530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:57.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.640+0000 7fccd846e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56168626f010 con 0x56168626c370 2024-09-12T02:04:57.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.640+0000 7fccd846e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x561686182060 con 0x561686262b40 2024-09-12T02:04:57.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.640+0000 7fccd846e740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x56168626f190 con 0x5616862615d0 2024-09-12T02:04:57.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.640+0000 7fccd846e740 10 client.0 _ll_register_callbacks cb 0x561685ed58c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:04:57.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.640+0000 7fccceffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56168626c370 0x561686269530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:57.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.640+0000 7fccceffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56168626c370 0x561686269530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57856/0 (socket says 172.21.15.16:57856) 2024-09-12T02:04:57.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.640+0000 7fccceffd640 1 -- 172.21.15.16:0/3999148694 learned_addr learned my addr 172.21.15.16:0/3999148694 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:04:57.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.640+0000 7fccceffd640 1 --2- 172.21.15.16:0/3999148694 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5616862615d0 0x56168626bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:57.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.640+0000 7fcccf7fe640 1 --2- 172.21.15.16:0/3999148694 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561686262b40 0x561686268fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:57.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.640+0000 7fccceffd640 1 -- 172.21.15.16:0/3999148694 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5616862615d0 msgr2=0x56168626bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:57.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.640+0000 7fccceffd640 1 --2- 172.21.15.16:0/3999148694 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5616862615d0 0x56168626bd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:57.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.640+0000 7fccceffd640 1 -- 172.21.15.16:0/3999148694 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561686262b40 msgr2=0x561686268fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:57.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.640+0000 7fccceffd640 1 --2- 172.21.15.16:0/3999148694 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561686262b40 0x561686268fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:57.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.640+0000 7fccceffd640 1 -- 172.21.15.16:0/3999148694 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561685f383c0 con 0x56168626c370 2024-09-12T02:04:57.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.640+0000 7fccceffd640 1 --2- 172.21.15.16:0/3999148694 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56168626c370 0x561686269530 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fccc4004630 tx=0x7fccc40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62b86945672e6a73 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:04:57.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.640+0000 7fccaeffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:04:57.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.641+0000 7fccaeffd640 1 -- 172.21.15.16:0/3999148694 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccc40109e0 con 0x56168626c370 2024-09-12T02:04:57.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.641+0000 7fccaeffd640 1 -- 172.21.15.16:0/3999148694 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fccc4010b80 con 0x56168626c370 2024-09-12T02:04:57.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.641+0000 7fccd846e740 1 -- 172.21.15.16:0/3999148694 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fccc00317b0 con 0x56168626c370 2024-09-12T02:04:57.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.641+0000 7fccaeffd640 1 -- 172.21.15.16:0/3999148694 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fccc4010d80 con 0x56168626c370 2024-09-12T02:04:57.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.641+0000 7fccd846e740 10 client.16281 fetch_fsmap learned FSMap version 25 2024-09-12T02:04:57.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.641+0000 7fccd846e740 1 -- 172.21.15.16:0/3999148694 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56168627ab00 con 0x56168626c370 2024-09-12T02:04:57.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.641+0000 7fccaeffd640 1 -- 172.21.15.16:0/3999148694 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fccc00317b0 con 0x56168626c370 2024-09-12T02:04:57.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.641+0000 7fccaeffd640 1 -- 172.21.15.16:0/3999148694 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fccc402a3b0 con 0x56168626c370 2024-09-12T02:04:57.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.641+0000 7fccd846e740 10 client.16281 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:04:57.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.641+0000 7fccd846e740 10 client.16281 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-12T02:04:57.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.641+0000 7fccd846e740 20 client.16281 populate_metadata read hostname 'smithi016' 2024-09-12T02:04:57.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.641+0000 7fccd846e740 1 --2- 172.21.15.16:0/3999148694 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x56168627bfc0 0x56168627e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:57.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.641+0000 7fccd846e740 4 client.16281 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:04:57.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.641+0000 7fccd846e740 1 -- 172.21.15.16:0/3999148694 --> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5616861d4860 con 0x56168627bfc0 2024-09-12T02:04:57.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.642+0000 7fcccf7fe640 1 --2- 172.21.15.16:0/3999148694 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x56168627bfc0 0x56168627e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:57.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.642+0000 7fcccf7fe640 1 --2- 172.21.15.16:0/3999148694 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x56168627bfc0 0x56168627e3b0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=aa0d2e576688e76a server_cookie=aa31a7601b033d84 in_seq=0 out_seq=0 2024-09-12T02:04:57.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.642+0000 7fccaeffd640 10 client.16281 ms_handle_connect on v2:172.21.15.184:6836/1793393087 2024-09-12T02:04:57.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.643+0000 7fccaeffd640 1 -- 172.21.15.16:0/3999148694 <== mds.2 v2:172.21.15.184:6836/1793393087 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5616861d4860 con 0x56168627bfc0 2024-09-12T02:04:57.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.643+0000 7fccaeffd640 10 client.16281 handle_command_reply: tid=0 2024-09-12T02:04:57.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.643+0000 7fccd846e740 1 client.16281 shutdown 2024-09-12T02:04:57.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.643+0000 7fccd846e740 1 -- 172.21.15.16:0/3999148694 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56168626c370 msgr2=0x561686269530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:57.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.643+0000 7fccd846e740 1 --2- 172.21.15.16:0/3999148694 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56168626c370 0x561686269530 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fccc4004630 tx=0x7fccc40046c0 comp rx=0 tx=0).stop 2024-09-12T02:04:57.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.643+0000 7fccd846e740 1 -- 172.21.15.16:0/3999148694 shutdown_connections 2024-09-12T02:04:57.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.643+0000 7fccd846e740 1 --2- 172.21.15.16:0/3999148694 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x56168627bfc0 0x56168627e3b0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:57.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.643+0000 7fccd846e740 1 --2- 172.21.15.16:0/3999148694 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56168626c370 0x561686269530 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:57.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.643+0000 7fccd846e740 1 --2- 172.21.15.16:0/3999148694 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561686262b40 0x561686268fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:57.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.643+0000 7fccaeffd640 0 client.16281 ms_handle_reset on v2:172.21.15.184:6836/1793393087 2024-09-12T02:04:57.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.643+0000 7fccd846e740 1 --2- 172.21.15.16:0/3999148694 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5616862615d0 0x56168626bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:57.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.643+0000 7fccd846e740 1 -- 172.21.15.16:0/3999148694 >> 172.21.15.16:0/3999148694 conn(0x56168622a310 msgr2=0x561686263930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:04:57.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.643+0000 7fccd846e740 1 -- 172.21.15.16:0/3999148694 shutdown_connections 2024-09-12T02:04:57.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.643+0000 7fccd846e740 1 -- 172.21.15.16:0/3999148694 wait complete. 2024-09-12T02:04:57.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.643+0000 7fccd846e740 20 client.16281 trim_cache size 0 max 16384 2024-09-12T02:04:57.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.656+0000 7fccd846e740 1 Processor -- start 2024-09-12T02:04:57.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.656+0000 7fccd846e740 1 -- start start 2024-09-12T02:04:57.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.656+0000 7fccd846e740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5616862615d0 0x561686278310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:57.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.656+0000 7fccd846e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561686262b40 0x56168626e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:57.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.656+0000 7fccd846e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56168626c370 0x56168626d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:57.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.657+0000 7fccd846e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fccd01b1fb0 con 0x56168626c370 2024-09-12T02:04:57.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.657+0000 7fccd846e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fccd01af880 con 0x561686262b40 2024-09-12T02:04:57.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.657+0000 7fccd846e740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fccd01ad6f0 con 0x5616862615d0 2024-09-12T02:04:57.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.657+0000 7fccceffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56168626c370 0x56168626d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:57.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.657+0000 7fccceffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56168626c370 0x56168626d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57860/0 (socket says 172.21.15.16:57860) 2024-09-12T02:04:57.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.657+0000 7fccceffd640 1 -- 172.21.15.16:0/3035028029 learned_addr learned my addr 172.21.15.16:0/3035028029 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:04:57.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.657+0000 7fcccf7fe640 1 --2- 172.21.15.16:0/3035028029 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561686262b40 0x56168626e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:57.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.657+0000 7fccceffd640 1 --2- 172.21.15.16:0/3035028029 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5616862615d0 0x561686278310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:57.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.657+0000 7fccceffd640 1 -- 172.21.15.16:0/3035028029 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5616862615d0 msgr2=0x561686278310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:57.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.657+0000 7fccceffd640 1 --2- 172.21.15.16:0/3035028029 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5616862615d0 0x561686278310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:57.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.657+0000 7fccceffd640 1 -- 172.21.15.16:0/3035028029 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561686262b40 msgr2=0x56168626e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:57.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.657+0000 7fccceffd640 1 --2- 172.21.15.16:0/3035028029 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561686262b40 0x56168626e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:57.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.657+0000 7fccceffd640 1 -- 172.21.15.16:0/3035028029 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561686267500 con 0x56168626c370 2024-09-12T02:04:57.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.657+0000 7fcccf7fe640 1 --2- 172.21.15.16:0/3035028029 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561686262b40 0x56168626e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:04:57.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.657+0000 7fccceffd640 1 --2- 172.21.15.16:0/3035028029 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56168626c370 0x56168626d760 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fccc400c010 tx=0x7fccc4012710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63a604cbc27333c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:04:57.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.657+0000 7fccb57fa640 1 -- 172.21.15.16:0/3035028029 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccc402a3b0 con 0x56168626c370 2024-09-12T02:04:57.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.658+0000 7fccb57fa640 1 -- 172.21.15.16:0/3035028029 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fccc402a550 con 0x56168626c370 2024-09-12T02:04:57.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.658+0000 7fccb57fa640 1 -- 172.21.15.16:0/3035028029 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccc40120e0 con 0x56168626c370 2024-09-12T02:04:57.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.658+0000 7fccd846e740 1 -- 172.21.15.16:0/3035028029 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56168626c370 msgr2=0x56168626d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:57.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.658+0000 7fccd846e740 1 --2- 172.21.15.16:0/3035028029 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56168626c370 0x56168626d760 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fccc400c010 tx=0x7fccc4012710 comp rx=0 tx=0).stop 2024-09-12T02:04:57.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.658+0000 7fccd846e740 1 -- 172.21.15.16:0/3035028029 shutdown_connections 2024-09-12T02:04:57.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.658+0000 7fccd846e740 1 --2- 172.21.15.16:0/3035028029 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56168626c370 0x56168626d760 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:57.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.658+0000 7fccd846e740 1 --2- 172.21.15.16:0/3035028029 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561686262b40 0x56168626e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:57.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.658+0000 7fccd846e740 1 --2- 172.21.15.16:0/3035028029 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5616862615d0 0x561686278310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:57.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.658+0000 7fccd846e740 1 -- 172.21.15.16:0/3035028029 >> 172.21.15.16:0/3035028029 conn(0x56168622a310 msgr2=0x561686267880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:04:57.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.658+0000 7fccd846e740 1 -- 172.21.15.16:0/3035028029 shutdown_connections 2024-09-12T02:04:57.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.658+0000 7fccd846e740 1 -- 172.21.15.16:0/3035028029 wait complete. 2024-09-12T02:04:57.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.659+0000 7fccd846e740 1 -- start start 2024-09-12T02:04:57.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.659+0000 7fccd846e740 1 Processor -- start 2024-09-12T02:04:57.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.659+0000 7fccd846e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5616862615d0 0x561686274a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:57.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.659+0000 7fccd846e740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561686262b40 0x56168626d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:57.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.659+0000 7fccd846e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56168626c370 0x56168626d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:57.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.659+0000 7fccd846e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fccd01add40 con 0x56168626c370 2024-09-12T02:04:57.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.659+0000 7fccd846e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fccd01af1b0 con 0x5616862615d0 2024-09-12T02:04:57.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.659+0000 7fccd846e740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fccd01ae410 con 0x561686262b40 2024-09-12T02:04:57.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.659+0000 7fccd846e740 10 client.0 _ll_register_callbacks cb 0x7fccc400bb70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:04:57.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.659+0000 7fccceffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56168626c370 0x56168626d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:57.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.659+0000 7fccceffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56168626c370 0x56168626d820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57876/0 (socket says 172.21.15.16:57876) 2024-09-12T02:04:57.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.659+0000 7fccceffd640 1 -- 172.21.15.16:0/2507076412 learned_addr learned my addr 172.21.15.16:0/2507076412 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:04:57.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.659+0000 7fcccf7fe640 1 --2- 172.21.15.16:0/2507076412 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561686262b40 0x56168626d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:57.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.660+0000 7fccceffd640 1 --2- 172.21.15.16:0/2507076412 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5616862615d0 0x561686274a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:57.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.660+0000 7fccceffd640 1 -- 172.21.15.16:0/2507076412 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561686262b40 msgr2=0x56168626d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:57.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.660+0000 7fccceffd640 1 --2- 172.21.15.16:0/2507076412 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561686262b40 0x56168626d2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:57.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.660+0000 7fccceffd640 1 -- 172.21.15.16:0/2507076412 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5616862615d0 msgr2=0x561686274a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:57.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.660+0000 7fccceffd640 1 --2- 172.21.15.16:0/2507076412 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5616862615d0 0x561686274a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:57.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.660+0000 7fccceffd640 1 -- 172.21.15.16:0/2507076412 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56168626de90 con 0x56168626c370 2024-09-12T02:04:57.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.660+0000 7fccceffd640 1 --2- 172.21.15.16:0/2507076412 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56168626c370 0x56168626d820 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fccc4005bf0 tx=0x7fccc4010b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b470d41265258b20 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:04:57.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.660+0000 7fccaeffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:04:57.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.660+0000 7fccaeffd640 1 -- 172.21.15.16:0/2507076412 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccc402a3b0 con 0x56168626c370 2024-09-12T02:04:57.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.660+0000 7fccd846e740 1 -- 172.21.15.16:0/2507076412 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fccc402a550 con 0x56168626c370 2024-09-12T02:04:57.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.660+0000 7fccaeffd640 1 -- 172.21.15.16:0/2507076412 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fccc4004260 con 0x56168626c370 2024-09-12T02:04:57.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.660+0000 7fccaeffd640 1 -- 172.21.15.16:0/2507076412 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fccc4004460 con 0x56168626c370 2024-09-12T02:04:57.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.660+0000 7fccd846e740 10 client.16287 fetch_fsmap learned FSMap version 25 2024-09-12T02:04:57.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.660+0000 7fccd846e740 1 -- 172.21.15.16:0/2507076412 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5616862e6b80 con 0x56168626c370 2024-09-12T02:04:57.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.660+0000 7fccaeffd640 1 -- 172.21.15.16:0/2507076412 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fccc402a550 con 0x56168626c370 2024-09-12T02:04:57.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.661+0000 7fccaeffd640 1 -- 172.21.15.16:0/2507076412 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fccc40046c0 con 0x56168626c370 2024-09-12T02:04:57.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.661+0000 7fccd846e740 10 client.16287 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:04:57.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.661+0000 7fccd846e740 10 client.16287 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-12T02:04:57.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.661+0000 7fccd846e740 20 client.16287 populate_metadata read hostname 'smithi016' 2024-09-12T02:04:57.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.661+0000 7fccd846e740 1 --2- 172.21.15.16:0/2507076412 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x5616862e70f0 0x5616862e9510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:57.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.661+0000 7fccd846e740 4 client.16287 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T02:04:57.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.661+0000 7fccd846e740 1 -- 172.21.15.16:0/2507076412 --> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5616861ced70 con 0x5616862e70f0 2024-09-12T02:04:57.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.661+0000 7fcccf7fe640 1 --2- 172.21.15.16:0/2507076412 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x5616862e70f0 0x5616862e9510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:57.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.661+0000 7fcccf7fe640 1 --2- 172.21.15.16:0/2507076412 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x5616862e70f0 0x5616862e9510 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5e545d65f02f98d7 server_cookie=60aff8b47f23ed19 in_seq=0 out_seq=0 2024-09-12T02:04:57.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.661+0000 7fccaeffd640 10 client.16287 ms_handle_connect on v2:172.21.15.184:6836/1793393087 2024-09-12T02:04:57.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.662+0000 7fccaeffd640 1 -- 172.21.15.16:0/2507076412 <== mds.2 v2:172.21.15.184:6836/1793393087 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5616861ced70 con 0x5616862e70f0 2024-09-12T02:04:57.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.662+0000 7fccaeffd640 10 client.16287 handle_command_reply: tid=0 2024-09-12T02:04:57.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.662+0000 7fccd846e740 1 client.16287 shutdown 2024-09-12T02:04:57.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.662+0000 7fccd846e740 1 -- 172.21.15.16:0/2507076412 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56168626c370 msgr2=0x56168626d820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:57.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.662+0000 7fccd846e740 1 --2- 172.21.15.16:0/2507076412 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56168626c370 0x56168626d820 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fccc4005bf0 tx=0x7fccc4010b80 comp rx=0 tx=0).stop 2024-09-12T02:04:57.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.662+0000 7fccd846e740 1 -- 172.21.15.16:0/2507076412 shutdown_connections 2024-09-12T02:04:57.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.662+0000 7fccd846e740 1 --2- 172.21.15.16:0/2507076412 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x5616862e70f0 0x5616862e9510 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:57.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.662+0000 7fccd846e740 1 --2- 172.21.15.16:0/2507076412 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56168626c370 0x56168626d820 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:57.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.662+0000 7fccaeffd640 0 client.16287 ms_handle_reset on v2:172.21.15.184:6836/1793393087 2024-09-12T02:04:57.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.662+0000 7fccd846e740 1 --2- 172.21.15.16:0/2507076412 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561686262b40 0x56168626d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:57.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.662+0000 7fccd846e740 1 --2- 172.21.15.16:0/2507076412 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5616862615d0 0x561686274a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:57.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.662+0000 7fccd846e740 1 -- 172.21.15.16:0/2507076412 >> 172.21.15.16:0/2507076412 conn(0x56168622a310 msgr2=0x56168626a810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:04:57.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.662+0000 7fccd846e740 1 -- 172.21.15.16:0/2507076412 shutdown_connections 2024-09-12T02:04:57.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.662+0000 7fccd846e740 1 -- 172.21.15.16:0/2507076412 wait complete. 2024-09-12T02:04:57.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.662+0000 7fccd846e740 20 client.16287 trim_cache size 0 max 16384 2024-09-12T02:04:57.703 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T02:04:57.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.662+0000 7fccd6095640 1 -- 172.21.15.16:0/2698234816 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fccb80a2b80 msgr2=0x7fccb80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:57.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.662+0000 7fccd6095640 1 --2- 172.21.15.16:0/2698234816 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fccb80a2b80 0x7fccb80a5040 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fccc80a3fd0 tx=0x7fccc801baf0 comp rx=0 tx=0).stop 2024-09-12T02:04:57.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.662+0000 7fccd6095640 1 -- 172.21.15.16:0/2698234816 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fccd010c170 msgr2=0x7fccd0101420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:57.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.662+0000 7fccd6095640 1 --2- 172.21.15.16:0/2698234816 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fccd010c170 0x7fccd0101420 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fccc800cad0 tx=0x7fccc800cf90 comp rx=0 tx=0).stop 2024-09-12T02:04:57.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.663+0000 7fccd6095640 1 -- 172.21.15.16:0/2698234816 shutdown_connections 2024-09-12T02:04:57.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.663+0000 7fccd6095640 1 --2- 172.21.15.16:0/2698234816 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fccb80a2b80 0x7fccb80a5040 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:57.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.663+0000 7fccd6095640 1 --2- 172.21.15.16:0/2698234816 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fccd010c170 0x7fccd0101420 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:57.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.663+0000 7fccd6095640 1 --2- 172.21.15.16:0/2698234816 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fccd0103660 0x7fccd0100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:57.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.663+0000 7fccd6095640 1 --2- 172.21.15.16:0/2698234816 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fccd0102d40 0x7fccd01009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:57.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.663+0000 7fccd6095640 1 -- 172.21.15.16:0/2698234816 >> 172.21.15.16:0/2698234816 conn(0x7fccd00fc7a0 msgr2=0x7fccd00fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:04:57.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.663+0000 7fccd6095640 1 -- 172.21.15.16:0/2698234816 shutdown_connections 2024-09-12T02:04:57.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:57.663+0000 7fccd6095640 1 -- 172.21.15.16:0/2698234816 wait complete. 2024-09-12T02:04:58.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:04:57 smithi016 ceph-mon[30947]: pgmap v426: 97 pgs: 97 active+clean; 1.6 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-09-12T02:04:58.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:04:57 smithi184 ceph-mon[35593]: pgmap v426: 97 pgs: 97 active+clean; 1.6 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-09-12T02:04:58.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:04:57 smithi033 ceph-mon[35332]: pgmap v426: 97 pgs: 97 active+clean; 1.6 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-09-12T02:04:58.521 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-12T02:04:58.784 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:04:59.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.164+0000 7fe533cc8640 1 -- 172.21.15.16:0/3759169705 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe52c102790 msgr2=0x7fe52c1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:59.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.164+0000 7fe533cc8640 1 --2- 172.21.15.16:0/3759169705 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe52c102790 0x7fe52c1067e0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fe51c009900 tx=0x7fe51c02a410 comp rx=0 tx=0).stop 2024-09-12T02:04:59.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.164+0000 7fe533cc8640 1 -- 172.21.15.16:0/3759169705 shutdown_connections 2024-09-12T02:04:59.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.164+0000 7fe533cc8640 1 --2- 172.21.15.16:0/3759169705 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe52c106f10 0x7fe52c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:59.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.164+0000 7fe533cc8640 1 --2- 172.21.15.16:0/3759169705 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe52c102790 0x7fe52c1067e0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:59.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.164+0000 7fe533cc8640 1 --2- 172.21.15.16:0/3759169705 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe52c101de0 0x7fe52c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:59.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.164+0000 7fe533cc8640 1 -- 172.21.15.16:0/3759169705 >> 172.21.15.16:0/3759169705 conn(0x7fe52c0fd8c0 msgr2=0x7fe52c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:04:59.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.164+0000 7fe533cc8640 1 -- 172.21.15.16:0/3759169705 shutdown_connections 2024-09-12T02:04:59.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.164+0000 7fe533cc8640 1 -- 172.21.15.16:0/3759169705 wait complete. 2024-09-12T02:04:59.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.165+0000 7fe533cc8640 1 Processor -- start 2024-09-12T02:04:59.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.165+0000 7fe533cc8640 1 -- start start 2024-09-12T02:04:59.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.165+0000 7fe533cc8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe52c101de0 0x7fe52c198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:59.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.165+0000 7fe533cc8640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe52c102790 0x7fe52c1989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:59.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.165+0000 7fe533cc8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe52c106f10 0x7fe52c19cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:59.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.165+0000 7fe533cc8640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe52c111fe0 con 0x7fe52c106f10 2024-09-12T02:04:59.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.165+0000 7fe533cc8640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe52c111e60 con 0x7fe52c101de0 2024-09-12T02:04:59.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.165+0000 7fe533cc8640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe52c112160 con 0x7fe52c102790 2024-09-12T02:04:59.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.165+0000 7fe532265640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe52c106f10 0x7fe52c19cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:59.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.165+0000 7fe532265640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe52c106f10 0x7fe52c19cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57892/0 (socket says 172.21.15.16:57892) 2024-09-12T02:04:59.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.165+0000 7fe532265640 1 -- 172.21.15.16:0/1189402078 learned_addr learned my addr 172.21.15.16:0/1189402078 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:04:59.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.165+0000 7fe531263640 1 --2- 172.21.15.16:0/1189402078 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe52c102790 0x7fe52c1989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:59.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.165+0000 7fe531a64640 1 --2- 172.21.15.16:0/1189402078 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe52c101de0 0x7fe52c198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:59.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.166+0000 7fe532265640 1 -- 172.21.15.16:0/1189402078 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe52c102790 msgr2=0x7fe52c1989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:59.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.166+0000 7fe532265640 1 --2- 172.21.15.16:0/1189402078 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe52c102790 0x7fe52c1989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:59.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.166+0000 7fe532265640 1 -- 172.21.15.16:0/1189402078 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe52c101de0 msgr2=0x7fe52c198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:59.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.166+0000 7fe532265640 1 --2- 172.21.15.16:0/1189402078 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe52c101de0 0x7fe52c198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:59.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.166+0000 7fe532265640 1 -- 172.21.15.16:0/1189402078 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe52c19d410 con 0x7fe52c106f10 2024-09-12T02:04:59.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.166+0000 7fe532265640 1 --2- 172.21.15.16:0/1189402078 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe52c106f10 0x7fe52c19cd30 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fe52400b970 tx=0x7fe52400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:04:59.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.166+0000 7fe522ffd640 1 -- 172.21.15.16:0/1189402078 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe524019070 con 0x7fe52c106f10 2024-09-12T02:04:59.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.166+0000 7fe533cc8640 1 -- 172.21.15.16:0/1189402078 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe52c19d700 con 0x7fe52c106f10 2024-09-12T02:04:59.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.166+0000 7fe522ffd640 1 -- 172.21.15.16:0/1189402078 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe52400cb40 con 0x7fe52c106f10 2024-09-12T02:04:59.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.166+0000 7fe533cc8640 1 -- 172.21.15.16:0/1189402078 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe52c1a4fe0 con 0x7fe52c106f10 2024-09-12T02:04:59.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.166+0000 7fe522ffd640 1 -- 172.21.15.16:0/1189402078 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe524007650 con 0x7fe52c106f10 2024-09-12T02:04:59.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.169+0000 7fe522ffd640 1 -- 172.21.15.16:0/1189402078 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fe52401c070 con 0x7fe52c106f10 2024-09-12T02:04:59.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.169+0000 7fe5360a1740 1 Processor -- start 2024-09-12T02:04:59.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.169+0000 7fe5360a1740 1 -- start start 2024-09-12T02:04:59.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.169+0000 7fe5360a1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563875cd2b40 0x563875cd1090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:59.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.169+0000 7fe5360a1740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563875cd15d0 0x563875cd1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:59.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.169+0000 7fe5360a1740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563875cdc370 0x563875cde760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:59.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.169+0000 7fe5360a1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x563875c372f0 con 0x563875cd2b40 2024-09-12T02:04:59.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.169+0000 7fe5360a1740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x563875bdecf0 con 0x563875cd15d0 2024-09-12T02:04:59.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.169+0000 7fe5360a1740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x563875c332b0 con 0x563875cdc370 2024-09-12T02:04:59.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.169+0000 7fe531263640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563875cd2b40 0x563875cd1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:59.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.170+0000 7fe531263640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563875cd2b40 0x563875cd1090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57908/0 (socket says 172.21.15.16:57908) 2024-09-12T02:04:59.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.170+0000 7fe531263640 1 -- 172.21.15.16:0/3841015096 learned_addr learned my addr 172.21.15.16:0/3841015096 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:04:59.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.170+0000 7fe531a64640 1 --2- 172.21.15.16:0/3841015096 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563875cd15d0 0x563875cd1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:59.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.170+0000 7fe522ffd640 1 --2- 172.21.15.16:0/1189402078 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fe5180a2c00 0x7fe5180a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:59.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.170+0000 7fe531263640 1 --2- 172.21.15.16:0/3841015096 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563875cdc370 0x563875cde760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:59.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.170+0000 7fe531263640 1 -- 172.21.15.16:0/3841015096 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563875cdc370 msgr2=0x563875cde760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:59.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.170+0000 7fe531263640 1 --2- 172.21.15.16:0/3841015096 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563875cdc370 0x563875cde760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:59.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.170+0000 7fe531263640 1 -- 172.21.15.16:0/3841015096 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563875cd15d0 msgr2=0x563875cd1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:59.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.170+0000 7fe531263640 1 --2- 172.21.15.16:0/3841015096 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563875cd15d0 0x563875cd1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:59.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.170+0000 7fe531263640 1 -- 172.21.15.16:0/3841015096 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563875bc33a0 con 0x563875cd2b40 2024-09-12T02:04:59.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.170+0000 7fe532265640 1 --2- 172.21.15.16:0/1189402078 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fe5180a2c00 0x7fe5180a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:59.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.170+0000 7fe522ffd640 1 -- 172.21.15.16:0/1189402078 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fe5240ce4b0 con 0x7fe52c106f10 2024-09-12T02:04:59.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.170+0000 7fe531263640 1 --2- 172.21.15.16:0/3841015096 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563875cd2b40 0x563875cd1090 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fe51c005ec0 tx=0x7fe51c031440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0701b0dde43c4d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:04:59.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.170+0000 7fe517fff640 1 -- 172.21.15.16:0/3841015096 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe51c035050 con 0x563875cd2b40 2024-09-12T02:04:59.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.170+0000 7fe517fff640 1 -- 172.21.15.16:0/3841015096 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe51c005500 con 0x563875cd2b40 2024-09-12T02:04:59.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.170+0000 7fe517fff640 1 -- 172.21.15.16:0/3841015096 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe51c003850 con 0x563875cd2b40 2024-09-12T02:04:59.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.171+0000 7fe5360a1740 1 -- 172.21.15.16:0/3841015096 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563875cd2b40 msgr2=0x563875cd1090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:59.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.171+0000 7fe5360a1740 1 --2- 172.21.15.16:0/3841015096 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563875cd2b40 0x563875cd1090 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fe51c005ec0 tx=0x7fe51c031440 comp rx=0 tx=0).stop 2024-09-12T02:04:59.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.171+0000 7fe532265640 1 --2- 172.21.15.16:0/1189402078 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fe5180a2c00 0x7fe5180a50c0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fe524097210 tx=0x7fe524065a30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:04:59.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.171+0000 7fe5360a1740 1 -- 172.21.15.16:0/3841015096 shutdown_connections 2024-09-12T02:04:59.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.171+0000 7fe5360a1740 1 --2- 172.21.15.16:0/3841015096 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563875cdc370 0x563875cde760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:59.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.171+0000 7fe5360a1740 1 --2- 172.21.15.16:0/3841015096 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563875cd15d0 0x563875cd1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:59.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.171+0000 7fe5360a1740 1 --2- 172.21.15.16:0/3841015096 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563875cd2b40 0x563875cd1090 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:59.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.171+0000 7fe5360a1740 1 -- 172.21.15.16:0/3841015096 >> 172.21.15.16:0/3841015096 conn(0x563875c9a310 msgr2=0x563875ccdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:04:59.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.171+0000 7fe5360a1740 1 -- 172.21.15.16:0/3841015096 shutdown_connections 2024-09-12T02:04:59.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.171+0000 7fe5360a1740 1 -- 172.21.15.16:0/3841015096 wait complete. 2024-09-12T02:04:59.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.172+0000 7fe5360a1740 1 -- start start 2024-09-12T02:04:59.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.172+0000 7fe5360a1740 1 Processor -- start 2024-09-12T02:04:59.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.172+0000 7fe5360a1740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563875cd15d0 0x563875cdbcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:59.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.172+0000 7fe5360a1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563875cd2b40 0x563875cd8f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:59.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.172+0000 7fe5360a1740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563875cdc370 0x563875cd94d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:59.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.172+0000 7fe5360a1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x563875cdf010 con 0x563875cd2b40 2024-09-12T02:04:59.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.172+0000 7fe5360a1740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x563875bf2060 con 0x563875cd15d0 2024-09-12T02:04:59.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.172+0000 7fe5360a1740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x563875cdf190 con 0x563875cdc370 2024-09-12T02:04:59.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.172+0000 7fe5360a1740 10 client.0 _ll_register_callbacks cb 0x5638759458c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:04:59.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.172+0000 7fe531a64640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563875cd2b40 0x563875cd8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:59.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.172+0000 7fe531a64640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563875cd2b40 0x563875cd8f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57920/0 (socket says 172.21.15.16:57920) 2024-09-12T02:04:59.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.172+0000 7fe531a64640 1 -- 172.21.15.16:0/3940603824 learned_addr learned my addr 172.21.15.16:0/3940603824 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:04:59.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.173+0000 7fe531263640 1 --2- 172.21.15.16:0/3940603824 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563875cd15d0 0x563875cdbcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:59.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.173+0000 7fe531263640 1 --2- 172.21.15.16:0/3940603824 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563875cdc370 0x563875cd94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:59.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.173+0000 7fe531a64640 1 -- 172.21.15.16:0/3940603824 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563875cdc370 msgr2=0x563875cd94d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:59.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.173+0000 7fe531a64640 1 --2- 172.21.15.16:0/3940603824 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563875cdc370 0x563875cd94d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:59.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.173+0000 7fe531a64640 1 -- 172.21.15.16:0/3940603824 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563875cd15d0 msgr2=0x563875cdbcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:59.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.173+0000 7fe531a64640 1 --2- 172.21.15.16:0/3940603824 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563875cd15d0 0x563875cdbcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:59.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.173+0000 7fe531a64640 1 -- 172.21.15.16:0/3940603824 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638759a83c0 con 0x563875cd2b40 2024-09-12T02:04:59.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.173+0000 7fe531263640 1 --2- 172.21.15.16:0/3940603824 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563875cdc370 0x563875cd94d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:04:59.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.173+0000 7fe531a64640 1 --2- 172.21.15.16:0/3940603824 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563875cd2b40 0x563875cd8f70 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fe528011800 tx=0x7fe528011cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20e5d2641d18a1f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:04:59.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.173+0000 7fe5157fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:04:59.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.173+0000 7fe5157fa640 1 -- 172.21.15.16:0/3940603824 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe52800fdb0 con 0x563875cd2b40 2024-09-12T02:04:59.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.173+0000 7fe5157fa640 1 -- 172.21.15.16:0/3940603824 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fe528005020 con 0x563875cd2b40 2024-09-12T02:04:59.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.173+0000 7fe5360a1740 1 -- 172.21.15.16:0/3940603824 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe51c005500 con 0x563875cd2b40 2024-09-12T02:04:59.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.173+0000 7fe5157fa640 1 -- 172.21.15.16:0/3940603824 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe528005220 con 0x563875cd2b40 2024-09-12T02:04:59.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.173+0000 7fe5360a1740 10 client.16299 fetch_fsmap learned FSMap version 25 2024-09-12T02:04:59.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.173+0000 7fe5360a1740 1 -- 172.21.15.16:0/3940603824 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x563875ceab40 con 0x563875cd2b40 2024-09-12T02:04:59.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.174+0000 7fe5157fa640 1 -- 172.21.15.16:0/3940603824 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe51c005500 con 0x563875cd2b40 2024-09-12T02:04:59.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.174+0000 7fe5157fa640 1 -- 172.21.15.16:0/3940603824 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fe528018030 con 0x563875cd2b40 2024-09-12T02:04:59.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.174+0000 7fe5360a1740 10 client.16299 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:04:59.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.174+0000 7fe5360a1740 10 client.16299 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T02:04:59.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.174+0000 7fe5360a1740 20 client.16299 populate_metadata read hostname 'smithi016' 2024-09-12T02:04:59.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.174+0000 7fe5360a1740 1 --2- 172.21.15.16:0/3940603824 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x563875cec0f0 0x563875cee530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:59.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.174+0000 7fe5360a1740 4 client.16299 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:04:59.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.174+0000 7fe5360a1740 1 -- 172.21.15.16:0/3940603824 --> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563875c44860 con 0x563875cec0f0 2024-09-12T02:04:59.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.174+0000 7fe531263640 1 --2- 172.21.15.16:0/3940603824 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x563875cec0f0 0x563875cee530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:59.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.175+0000 7fe5157fa640 10 client.16299 ms_handle_connect on v2:172.21.15.184:6838/4203347941 2024-09-12T02:04:59.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.175+0000 7fe531263640 1 --2- 172.21.15.16:0/3940603824 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x563875cec0f0 0x563875cee530 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d91d22622b07c72b server_cookie=3b33eedcb05054 in_seq=0 out_seq=0 2024-09-12T02:04:59.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.175+0000 7fe5157fa640 1 -- 172.21.15.16:0/3940603824 <== mds.1 v2:172.21.15.184:6838/4203347941 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563875c44860 con 0x563875cec0f0 2024-09-12T02:04:59.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.175+0000 7fe5157fa640 10 client.16299 handle_command_reply: tid=0 2024-09-12T02:04:59.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.175+0000 7fe5360a1740 1 client.16299 shutdown 2024-09-12T02:04:59.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.176+0000 7fe5360a1740 1 -- 172.21.15.16:0/3940603824 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563875cd2b40 msgr2=0x563875cd8f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:59.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.176+0000 7fe5360a1740 1 --2- 172.21.15.16:0/3940603824 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563875cd2b40 0x563875cd8f70 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fe528011800 tx=0x7fe528011cc0 comp rx=0 tx=0).stop 2024-09-12T02:04:59.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.176+0000 7fe5360a1740 1 -- 172.21.15.16:0/3940603824 shutdown_connections 2024-09-12T02:04:59.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.176+0000 7fe5360a1740 1 --2- 172.21.15.16:0/3940603824 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x563875cec0f0 0x563875cee530 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:59.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.176+0000 7fe5360a1740 1 --2- 172.21.15.16:0/3940603824 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563875cdc370 0x563875cd94d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:59.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.176+0000 7fe5360a1740 1 --2- 172.21.15.16:0/3940603824 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563875cd2b40 0x563875cd8f70 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:59.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.176+0000 7fe5157fa640 0 client.16299 ms_handle_reset on v2:172.21.15.184:6838/4203347941 2024-09-12T02:04:59.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.176+0000 7fe5360a1740 1 --2- 172.21.15.16:0/3940603824 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563875cd15d0 0x563875cdbcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:59.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.176+0000 7fe5360a1740 1 -- 172.21.15.16:0/3940603824 >> 172.21.15.16:0/3940603824 conn(0x563875c9a310 msgr2=0x563875cd3930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:04:59.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.176+0000 7fe5360a1740 1 -- 172.21.15.16:0/3940603824 shutdown_connections 2024-09-12T02:04:59.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.176+0000 7fe5360a1740 1 -- 172.21.15.16:0/3940603824 wait complete. 2024-09-12T02:04:59.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.176+0000 7fe5360a1740 20 client.16299 trim_cache size 0 max 16384 2024-09-12T02:04:59.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.189+0000 7fe5360a1740 1 Processor -- start 2024-09-12T02:04:59.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.189+0000 7fe5360a1740 1 -- start start 2024-09-12T02:04:59.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.189+0000 7fe5360a1740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563875cd15d0 0x563875ce8340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:59.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.189+0000 7fe5360a1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563875cd2b40 0x563875cd4120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:59.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.189+0000 7fe5360a1740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563875cdc370 0x563875cd4660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:59.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.189+0000 7fe5360a1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe510010bb0 con 0x563875cd2b40 2024-09-12T02:04:59.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.189+0000 7fe5360a1740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe51000e480 con 0x563875cdc370 2024-09-12T02:04:59.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.189+0000 7fe5360a1740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe51000c2d0 con 0x563875cd15d0 2024-09-12T02:04:59.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.189+0000 7fe531a64640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563875cd2b40 0x563875cd4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:59.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.189+0000 7fe531a64640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563875cd2b40 0x563875cd4120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57930/0 (socket says 172.21.15.16:57930) 2024-09-12T02:04:59.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.189+0000 7fe531a64640 1 -- 172.21.15.16:0/3113868871 learned_addr learned my addr 172.21.15.16:0/3113868871 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:04:59.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.190+0000 7fe531263640 1 --2- 172.21.15.16:0/3113868871 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563875cd15d0 0x563875ce8340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:59.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.190+0000 7fe531263640 1 --2- 172.21.15.16:0/3113868871 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563875cdc370 0x563875cd4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:59.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.190+0000 7fe531a64640 1 -- 172.21.15.16:0/3113868871 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563875cd15d0 msgr2=0x563875ce8340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:59.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.190+0000 7fe531a64640 1 --2- 172.21.15.16:0/3113868871 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563875cd15d0 0x563875ce8340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:59.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.190+0000 7fe531a64640 1 -- 172.21.15.16:0/3113868871 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563875cdc370 msgr2=0x563875cd4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:59.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.190+0000 7fe531a64640 1 --2- 172.21.15.16:0/3113868871 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563875cdc370 0x563875cd4660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:59.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.190+0000 7fe531a64640 1 -- 172.21.15.16:0/3113868871 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563875cce960 con 0x563875cd2b40 2024-09-12T02:04:59.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.190+0000 7fe531263640 1 --2- 172.21.15.16:0/3113868871 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563875cd15d0 0x563875ce8340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:04:59.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.190+0000 7fe531a64640 1 --2- 172.21.15.16:0/3113868871 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563875cd2b40 0x563875cd4120 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fe528000c00 tx=0x7fe528005a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16ddad8b0be2a449 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:04:59.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.190+0000 7fe5177fe640 1 -- 172.21.15.16:0/3113868871 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe528030020 con 0x563875cd2b40 2024-09-12T02:04:59.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.190+0000 7fe5177fe640 1 -- 172.21.15.16:0/3113868871 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe528019a50 con 0x563875cd2b40 2024-09-12T02:04:59.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.190+0000 7fe5177fe640 1 -- 172.21.15.16:0/3113868871 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe528028040 con 0x563875cd2b40 2024-09-12T02:04:59.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.191+0000 7fe5360a1740 1 -- 172.21.15.16:0/3113868871 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563875cd2b40 msgr2=0x563875cd4120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:59.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.191+0000 7fe5360a1740 1 --2- 172.21.15.16:0/3113868871 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563875cd2b40 0x563875cd4120 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fe528000c00 tx=0x7fe528005a90 comp rx=0 tx=0).stop 2024-09-12T02:04:59.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.191+0000 7fe5360a1740 1 -- 172.21.15.16:0/3113868871 shutdown_connections 2024-09-12T02:04:59.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.191+0000 7fe5360a1740 1 --2- 172.21.15.16:0/3113868871 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563875cdc370 0x563875cd4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:59.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.191+0000 7fe5360a1740 1 --2- 172.21.15.16:0/3113868871 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563875cd2b40 0x563875cd4120 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:59.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.191+0000 7fe5360a1740 1 --2- 172.21.15.16:0/3113868871 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563875cd15d0 0x563875ce8340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:59.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.191+0000 7fe5360a1740 1 -- 172.21.15.16:0/3113868871 >> 172.21.15.16:0/3113868871 conn(0x563875c9a310 msgr2=0x563875cd8060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:04:59.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.191+0000 7fe5360a1740 1 -- 172.21.15.16:0/3113868871 shutdown_connections 2024-09-12T02:04:59.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.191+0000 7fe5360a1740 1 -- 172.21.15.16:0/3113868871 wait complete. 2024-09-12T02:04:59.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.191+0000 7fe5360a1740 1 -- start start 2024-09-12T02:04:59.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.191+0000 7fe5360a1740 1 Processor -- start 2024-09-12T02:04:59.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.192+0000 7fe5360a1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563875cd15d0 0x563875ce4c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:59.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.192+0000 7fe5360a1740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563875cd2b40 0x563875cdff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:59.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.192+0000 7fe5360a1740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563875cdc370 0x563875ce04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:59.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.192+0000 7fe5360a1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe51000c940 con 0x563875cd15d0 2024-09-12T02:04:59.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.192+0000 7fe5360a1740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe51000ddb0 con 0x563875cd2b40 2024-09-12T02:04:59.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.192+0000 7fe5360a1740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe51000d010 con 0x563875cdc370 2024-09-12T02:04:59.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.192+0000 7fe5360a1740 10 client.0 _ll_register_callbacks cb 0x7fe52800c6d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:04:59.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.192+0000 7fe531263640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563875cd15d0 0x563875ce4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:59.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.192+0000 7fe531263640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563875cd15d0 0x563875ce4c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57936/0 (socket says 172.21.15.16:57936) 2024-09-12T02:04:59.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.192+0000 7fe531263640 1 -- 172.21.15.16:0/501181481 learned_addr learned my addr 172.21.15.16:0/501181481 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:04:59.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.192+0000 7fe531a64640 1 --2- 172.21.15.16:0/501181481 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563875cd2b40 0x563875cdff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:59.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.192+0000 7fe531263640 1 --2- 172.21.15.16:0/501181481 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563875cdc370 0x563875ce04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:59.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.192+0000 7fe531263640 1 -- 172.21.15.16:0/501181481 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563875cdc370 msgr2=0x563875ce04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:59.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.192+0000 7fe531263640 1 --2- 172.21.15.16:0/501181481 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563875cdc370 0x563875ce04c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:59.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.192+0000 7fe531263640 1 -- 172.21.15.16:0/501181481 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563875cd2b40 msgr2=0x563875cdff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:59.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.192+0000 7fe531263640 1 --2- 172.21.15.16:0/501181481 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563875cd2b40 0x563875cdff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:59.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.192+0000 7fe531263640 1 -- 172.21.15.16:0/501181481 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563875ce0cf0 con 0x563875cd15d0 2024-09-12T02:04:59.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.192+0000 7fe531263640 1 --2- 172.21.15.16:0/501181481 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563875cd15d0 0x563875ce4c50 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fe51c004de0 tx=0x7fe52800cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be0603d401b56763 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:04:59.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.192+0000 7fe5157fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:04:59.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.192+0000 7fe5157fa640 1 -- 172.21.15.16:0/501181481 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe51c054070 con 0x563875cd15d0 2024-09-12T02:04:59.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.192+0000 7fe5157fa640 1 -- 172.21.15.16:0/501181481 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fe51c004af0 con 0x563875cd15d0 2024-09-12T02:04:59.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.192+0000 7fe5360a1740 1 -- 172.21.15.16:0/501181481 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe528019a50 con 0x563875cd15d0 2024-09-12T02:04:59.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.193+0000 7fe5157fa640 1 -- 172.21.15.16:0/501181481 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe51c035040 con 0x563875cd15d0 2024-09-12T02:04:59.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.193+0000 7fe5360a1740 10 client.16305 fetch_fsmap learned FSMap version 25 2024-09-12T02:04:59.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.193+0000 7fe5360a1740 1 -- 172.21.15.16:0/501181481 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x563875d56e70 con 0x563875cd15d0 2024-09-12T02:04:59.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.193+0000 7fe5157fa640 1 -- 172.21.15.16:0/501181481 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe528019a50 con 0x563875cd15d0 2024-09-12T02:04:59.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.193+0000 7fe5157fa640 1 -- 172.21.15.16:0/501181481 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fe51c0035f0 con 0x563875cd15d0 2024-09-12T02:04:59.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.193+0000 7fe5360a1740 10 client.16305 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:04:59.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.193+0000 7fe5360a1740 10 client.16305 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T02:04:59.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.193+0000 7fe5360a1740 20 client.16305 populate_metadata read hostname 'smithi016' 2024-09-12T02:04:59.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.193+0000 7fe5360a1740 1 --2- 172.21.15.16:0/501181481 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x563875cec0f0 0x563875d591c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:59.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.193+0000 7fe5360a1740 4 client.16305 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T02:04:59.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.193+0000 7fe5360a1740 1 -- 172.21.15.16:0/501181481 --> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563875c3ed70 con 0x563875cec0f0 2024-09-12T02:04:59.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.193+0000 7fe531a64640 1 --2- 172.21.15.16:0/501181481 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x563875cec0f0 0x563875d591c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:59.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.194+0000 7fe531a64640 1 --2- 172.21.15.16:0/501181481 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x563875cec0f0 0x563875d591c0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bb3f176669390463 server_cookie=dd0ec2ecf40712bf in_seq=0 out_seq=0 2024-09-12T02:04:59.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.194+0000 7fe5157fa640 10 client.16305 ms_handle_connect on v2:172.21.15.184:6838/4203347941 2024-09-12T02:04:59.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.194+0000 7fe5157fa640 1 -- 172.21.15.16:0/501181481 <== mds.1 v2:172.21.15.184:6838/4203347941 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563875c3ed70 con 0x563875cec0f0 2024-09-12T02:04:59.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.194+0000 7fe5157fa640 10 client.16305 handle_command_reply: tid=0 2024-09-12T02:04:59.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.194+0000 7fe5360a1740 1 client.16305 shutdown 2024-09-12T02:04:59.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.194+0000 7fe5360a1740 1 -- 172.21.15.16:0/501181481 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563875cd15d0 msgr2=0x563875ce4c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:59.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.194+0000 7fe5360a1740 1 --2- 172.21.15.16:0/501181481 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563875cd15d0 0x563875ce4c50 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fe51c004de0 tx=0x7fe52800cd30 comp rx=0 tx=0).stop 2024-09-12T02:04:59.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.194+0000 7fe5360a1740 1 -- 172.21.15.16:0/501181481 shutdown_connections 2024-09-12T02:04:59.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.194+0000 7fe5360a1740 1 --2- 172.21.15.16:0/501181481 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x563875cec0f0 0x563875d591c0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:59.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.194+0000 7fe5360a1740 1 --2- 172.21.15.16:0/501181481 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563875cdc370 0x563875ce04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:59.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.194+0000 7fe5360a1740 1 --2- 172.21.15.16:0/501181481 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563875cd2b40 0x563875cdff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:59.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.194+0000 7fe5360a1740 1 --2- 172.21.15.16:0/501181481 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563875cd15d0 0x563875ce4c50 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:59.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.194+0000 7fe5360a1740 1 -- 172.21.15.16:0/501181481 >> 172.21.15.16:0/501181481 conn(0x563875c9a310 msgr2=0x563875cda7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:04:59.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.194+0000 7fe5157fa640 0 client.16305 ms_handle_reset on v2:172.21.15.184:6838/4203347941 2024-09-12T02:04:59.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.194+0000 7fe5360a1740 1 -- 172.21.15.16:0/501181481 shutdown_connections 2024-09-12T02:04:59.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.194+0000 7fe5360a1740 1 -- 172.21.15.16:0/501181481 wait complete. 2024-09-12T02:04:59.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.194+0000 7fe5360a1740 20 client.16305 trim_cache size 0 max 16384 2024-09-12T02:04:59.202 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T02:04:59.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.195+0000 7fe533cc8640 1 -- 172.21.15.16:0/1189402078 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fe5180a2c00 msgr2=0x7fe5180a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:59.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.195+0000 7fe533cc8640 1 --2- 172.21.15.16:0/1189402078 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fe5180a2c00 0x7fe5180a50c0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fe524097210 tx=0x7fe524065a30 comp rx=0 tx=0).stop 2024-09-12T02:04:59.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.195+0000 7fe533cc8640 1 -- 172.21.15.16:0/1189402078 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe52c106f10 msgr2=0x7fe52c19cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:59.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.195+0000 7fe533cc8640 1 --2- 172.21.15.16:0/1189402078 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe52c106f10 0x7fe52c19cd30 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fe52400b970 tx=0x7fe52400be30 comp rx=0 tx=0).stop 2024-09-12T02:04:59.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.195+0000 7fe533cc8640 1 -- 172.21.15.16:0/1189402078 shutdown_connections 2024-09-12T02:04:59.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.195+0000 7fe533cc8640 1 --2- 172.21.15.16:0/1189402078 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fe5180a2c00 0x7fe5180a50c0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:59.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.195+0000 7fe533cc8640 1 --2- 172.21.15.16:0/1189402078 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe52c106f10 0x7fe52c19cd30 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:59.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.195+0000 7fe533cc8640 1 --2- 172.21.15.16:0/1189402078 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe52c102790 0x7fe52c1989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:59.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.195+0000 7fe533cc8640 1 --2- 172.21.15.16:0/1189402078 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe52c101de0 0x7fe52c198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:59.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.195+0000 7fe533cc8640 1 -- 172.21.15.16:0/1189402078 >> 172.21.15.16:0/1189402078 conn(0x7fe52c0fd8c0 msgr2=0x7fe52c0fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:04:59.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.195+0000 7fe533cc8640 1 -- 172.21.15.16:0/1189402078 shutdown_connections 2024-09-12T02:04:59.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:04:59.195+0000 7fe533cc8640 1 -- 172.21.15.16:0/1189402078 wait complete. 2024-09-12T02:04:59.982 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-12T02:05:00.183 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:04:59 smithi016 ceph-mon[30947]: pgmap v427: 97 pgs: 97 active+clean; 1.6 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 11 op/s 2024-09-12T02:05:00.240 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:05:00.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:04:59 smithi184 ceph-mon[35593]: pgmap v427: 97 pgs: 97 active+clean; 1.6 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 11 op/s 2024-09-12T02:05:00.418 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:04:59 smithi033 ceph-mon[35332]: pgmap v427: 97 pgs: 97 active+clean; 1.6 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 11 op/s 2024-09-12T02:05:00.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.704+0000 7f6f7615e640 1 -- 172.21.15.16:0/2581590339 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f70101de0 msgr2=0x7f6f701021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:00.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.704+0000 7f6f7615e640 1 --2- 172.21.15.16:0/2581590339 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f70101de0 0x7f6f701021c0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f6f60009900 tx=0x7f6f6002a410 comp rx=0 tx=0).stop 2024-09-12T02:05:00.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.704+0000 7f6f7615e640 1 -- 172.21.15.16:0/2581590339 shutdown_connections 2024-09-12T02:05:00.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.704+0000 7f6f7615e640 1 --2- 172.21.15.16:0/2581590339 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6f70106f10 0x7f6f7010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:00.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.704+0000 7f6f7615e640 1 --2- 172.21.15.16:0/2581590339 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f70102790 0x7f6f701067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:00.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.704+0000 7f6f7615e640 1 --2- 172.21.15.16:0/2581590339 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f70101de0 0x7f6f701021c0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:00.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.704+0000 7f6f7615e640 1 -- 172.21.15.16:0/2581590339 >> 172.21.15.16:0/2581590339 conn(0x7f6f700fd8c0 msgr2=0x7f6f700ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:00.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.704+0000 7f6f7615e640 1 -- 172.21.15.16:0/2581590339 shutdown_connections 2024-09-12T02:05:00.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.704+0000 7f6f7615e640 1 -- 172.21.15.16:0/2581590339 wait complete. 2024-09-12T02:05:00.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.704+0000 7f6f7615e640 1 Processor -- start 2024-09-12T02:05:00.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.705+0000 7f6f7615e640 1 -- start start 2024-09-12T02:05:00.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.705+0000 7f6f7615e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f70101de0 0x7f6f70198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:00.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.705+0000 7f6f7615e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6f70102790 0x7f6f70198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:00.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.705+0000 7f6f7615e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f70106f10 0x7f6f7019cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:00.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.705+0000 7f6f7615e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6f70111f70 con 0x7f6f70106f10 2024-09-12T02:05:00.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.705+0000 7f6f7615e640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6f70111df0 con 0x7f6f70101de0 2024-09-12T02:05:00.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.705+0000 7f6f7615e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f6f701120f0 con 0x7f6f70102790 2024-09-12T02:05:00.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.705+0000 7f6f6ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f70106f10 0x7f6f7019cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:00.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.705+0000 7f6f6ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f70106f10 0x7f6f7019cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57960/0 (socket says 172.21.15.16:57960) 2024-09-12T02:05:00.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.705+0000 7f6f6ffff640 1 -- 172.21.15.16:0/1246043242 learned_addr learned my addr 172.21.15.16:0/1246043242 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:05:00.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.705+0000 7f6f6f7fe640 1 --2- 172.21.15.16:0/1246043242 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f70101de0 0x7f6f70198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:00.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.705+0000 7f6f6effd640 1 --2- 172.21.15.16:0/1246043242 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6f70102790 0x7f6f70198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:00.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.705+0000 7f6f6ffff640 1 -- 172.21.15.16:0/1246043242 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6f70102790 msgr2=0x7f6f70198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:00.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.705+0000 7f6f6ffff640 1 --2- 172.21.15.16:0/1246043242 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6f70102790 0x7f6f70198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:00.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.705+0000 7f6f6ffff640 1 -- 172.21.15.16:0/1246043242 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f70101de0 msgr2=0x7f6f70198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:00.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.705+0000 7f6f6ffff640 1 --2- 172.21.15.16:0/1246043242 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f70101de0 0x7f6f70198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:00.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.705+0000 7f6f6ffff640 1 -- 172.21.15.16:0/1246043242 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f7019d3e0 con 0x7f6f70106f10 2024-09-12T02:05:00.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.705+0000 7f6f6ffff640 1 --2- 172.21.15.16:0/1246043242 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f70106f10 0x7f6f7019cd00 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f6f6800cad0 tx=0x7f6f6800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:00.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.706+0000 7f6f6cff9640 1 -- 172.21.15.16:0/1246043242 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f6800ae20 con 0x7f6f70106f10 2024-09-12T02:05:00.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.706+0000 7f6f6cff9640 1 -- 172.21.15.16:0/1246043242 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f68005020 con 0x7f6f70106f10 2024-09-12T02:05:00.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.706+0000 7f6f7615e640 1 -- 172.21.15.16:0/1246043242 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f7019d6d0 con 0x7f6f70106f10 2024-09-12T02:05:00.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.706+0000 7f6f6cff9640 1 -- 172.21.15.16:0/1246043242 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f6800f6f0 con 0x7f6f70106f10 2024-09-12T02:05:00.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.706+0000 7f6f7615e640 1 -- 172.21.15.16:0/1246043242 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f701a4fb0 con 0x7f6f70106f10 2024-09-12T02:05:00.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.708+0000 7f6f6cff9640 1 -- 172.21.15.16:0/1246043242 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6f680051d0 con 0x7f6f70106f10 2024-09-12T02:05:00.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.708+0000 7f6f78537740 1 Processor -- start 2024-09-12T02:05:00.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.708+0000 7f6f78537740 1 -- start start 2024-09-12T02:05:00.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.708+0000 7f6f78537740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5563f9de9b40 0x5563f9de8090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:00.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.708+0000 7f6f78537740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563f9de85d0 0x5563f9de8a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:00.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.708+0000 7f6f78537740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5563f9df7780 0x5563f9df9b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:00.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.709+0000 7f6f78537740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5563f9d4e2f0 con 0x5563f9de85d0 2024-09-12T02:05:00.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.709+0000 7f6f78537740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5563f9cf5cf0 con 0x5563f9de9b40 2024-09-12T02:05:00.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.709+0000 7f6f78537740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5563f9d4a2b0 con 0x5563f9df7780 2024-09-12T02:05:00.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.709+0000 7f6f6f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563f9de85d0 0x5563f9de8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:00.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.709+0000 7f6f6cff9640 1 --2- 172.21.15.16:0/1246043242 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f6f580a2b80 0x7f6f580a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:00.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.709+0000 7f6f6f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563f9de85d0 0x5563f9de8a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57970/0 (socket says 172.21.15.16:57970) 2024-09-12T02:05:00.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.709+0000 7f6f6f7fe640 1 -- 172.21.15.16:0/2770606361 learned_addr learned my addr 172.21.15.16:0/2770606361 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:05:00.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.709+0000 7f6f6effd640 1 --2- 172.21.15.16:0/2770606361 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5563f9de9b40 0x5563f9de8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:00.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.709+0000 7f6f6effd640 1 --2- 172.21.15.16:0/2770606361 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5563f9df7780 0x5563f9df9b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:00.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.709+0000 7f6f6ffff640 1 --2- 172.21.15.16:0/1246043242 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f6f580a2b80 0x7f6f580a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:00.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.709+0000 7f6f6f7fe640 1 -- 172.21.15.16:0/2770606361 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5563f9df7780 msgr2=0x5563f9df9b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:00.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.709+0000 7f6f6f7fe640 1 --2- 172.21.15.16:0/2770606361 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5563f9df7780 0x5563f9df9b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:00.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.709+0000 7f6f6f7fe640 1 -- 172.21.15.16:0/2770606361 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5563f9de9b40 msgr2=0x5563f9de8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:00.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.709+0000 7f6f6f7fe640 1 --2- 172.21.15.16:0/2770606361 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5563f9de9b40 0x5563f9de8090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:00.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.709+0000 7f6f6f7fe640 1 -- 172.21.15.16:0/2770606361 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563f9cda3a0 con 0x5563f9de85d0 2024-09-12T02:05:00.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.709+0000 7f6f6cff9640 1 -- 172.21.15.16:0/1246043242 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f6f680cf1d0 con 0x7f6f70106f10 2024-09-12T02:05:00.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.709+0000 7f6f6effd640 1 --2- 172.21.15.16:0/2770606361 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5563f9de9b40 0x5563f9de8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:05:00.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.709+0000 7f6f6f7fe640 1 --2- 172.21.15.16:0/2770606361 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563f9de85d0 0x5563f9de8a10 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f6f60009900 tx=0x7f6f60031630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79bd82d72ac0f1c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:00.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.709+0000 7f6f6ffff640 1 --2- 172.21.15.16:0/1246043242 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f6f580a2b80 0x7f6f580a5040 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f6f6800caa0 tx=0x7f6f6801cb40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:00.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.709+0000 7f6f55ffb640 1 -- 172.21.15.16:0/2770606361 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f60040070 con 0x5563f9de85d0 2024-09-12T02:05:00.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.710+0000 7f6f55ffb640 1 -- 172.21.15.16:0/2770606361 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f60005020 con 0x5563f9de85d0 2024-09-12T02:05:00.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.710+0000 7f6f55ffb640 1 -- 172.21.15.16:0/2770606361 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f60035040 con 0x5563f9de85d0 2024-09-12T02:05:00.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.710+0000 7f6f78537740 1 -- 172.21.15.16:0/2770606361 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563f9de85d0 msgr2=0x5563f9de8a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:00.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.710+0000 7f6f78537740 1 --2- 172.21.15.16:0/2770606361 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563f9de85d0 0x5563f9de8a10 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f6f60009900 tx=0x7f6f60031630 comp rx=0 tx=0).stop 2024-09-12T02:05:00.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.710+0000 7f6f78537740 1 -- 172.21.15.16:0/2770606361 shutdown_connections 2024-09-12T02:05:00.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.710+0000 7f6f78537740 1 --2- 172.21.15.16:0/2770606361 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5563f9df7780 0x5563f9df9b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:00.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.710+0000 7f6f78537740 1 --2- 172.21.15.16:0/2770606361 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563f9de85d0 0x5563f9de8a10 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:00.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.710+0000 7f6f78537740 1 --2- 172.21.15.16:0/2770606361 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5563f9de9b40 0x5563f9de8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:00.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.710+0000 7f6f78537740 1 -- 172.21.15.16:0/2770606361 >> 172.21.15.16:0/2770606361 conn(0x5563f9db1310 msgr2=0x5563f9de4c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:00.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.710+0000 7f6f78537740 1 -- 172.21.15.16:0/2770606361 shutdown_connections 2024-09-12T02:05:00.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.710+0000 7f6f78537740 1 -- 172.21.15.16:0/2770606361 wait complete. 2024-09-12T02:05:00.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.711+0000 7f6f78537740 1 -- start start 2024-09-12T02:05:00.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.711+0000 7f6f78537740 1 Processor -- start 2024-09-12T02:05:00.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.711+0000 7f6f78537740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5563f9de85d0 0x5563f9df70f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:00.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.711+0000 7f6f78537740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563f9de9b40 0x5563f9df43a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:00.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.711+0000 7f6f78537740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5563f9df7780 0x5563f9df4900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:00.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.711+0000 7f6f78537740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5563f9dfa420 con 0x5563f9de9b40 2024-09-12T02:05:00.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.711+0000 7f6f78537740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5563f9d09060 con 0x5563f9df7780 2024-09-12T02:05:00.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.711+0000 7f6f78537740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5563f9dfa5a0 con 0x5563f9de85d0 2024-09-12T02:05:00.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.711+0000 7f6f78537740 10 client.0 _ll_register_callbacks cb 0x5563f9a5c8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:05:00.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.711+0000 7f6f6f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563f9de9b40 0x5563f9df43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:00.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.711+0000 7f6f6f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563f9de9b40 0x5563f9df43a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57974/0 (socket says 172.21.15.16:57974) 2024-09-12T02:05:00.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.711+0000 7f6f6f7fe640 1 -- 172.21.15.16:0/3366831094 learned_addr learned my addr 172.21.15.16:0/3366831094 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:05:00.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.711+0000 7f6f6effd640 1 --2- 172.21.15.16:0/3366831094 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5563f9de85d0 0x5563f9df70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:00.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.711+0000 7f6f6effd640 1 --2- 172.21.15.16:0/3366831094 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5563f9df7780 0x5563f9df4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:00.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.712+0000 7f6f6f7fe640 1 -- 172.21.15.16:0/3366831094 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5563f9de85d0 msgr2=0x5563f9df70f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:00.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.712+0000 7f6f6f7fe640 1 --2- 172.21.15.16:0/3366831094 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5563f9de85d0 0x5563f9df70f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:00.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.712+0000 7f6f6f7fe640 1 -- 172.21.15.16:0/3366831094 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5563f9df7780 msgr2=0x5563f9df4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:00.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.712+0000 7f6f6f7fe640 1 --2- 172.21.15.16:0/3366831094 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5563f9df7780 0x5563f9df4900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:00.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.712+0000 7f6f6f7fe640 1 -- 172.21.15.16:0/3366831094 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563f9abf3c0 con 0x5563f9de9b40 2024-09-12T02:05:00.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.712+0000 7f6f6effd640 1 --2- 172.21.15.16:0/3366831094 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5563f9df7780 0x5563f9df4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:05:00.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.712+0000 7f6f6f7fe640 1 --2- 172.21.15.16:0/3366831094 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563f9de9b40 0x5563f9df43a0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f6f6002a930 tx=0x7f6f60005470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82968f854af8dcf0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:00.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.712+0000 7f6f4effd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:05:00.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.712+0000 7f6f4effd640 1 -- 172.21.15.16:0/3366831094 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f6004b070 con 0x5563f9de9b40 2024-09-12T02:05:00.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.712+0000 7f6f78537740 1 -- 172.21.15.16:0/3366831094 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f60005020 con 0x5563f9de9b40 2024-09-12T02:05:00.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.712+0000 7f6f4effd640 1 -- 172.21.15.16:0/3366831094 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f6f60004b60 con 0x5563f9de9b40 2024-09-12T02:05:00.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.712+0000 7f6f4effd640 1 -- 172.21.15.16:0/3366831094 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6f60057050 con 0x5563f9de9b40 2024-09-12T02:05:00.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.712+0000 7f6f78537740 10 client.16317 fetch_fsmap learned FSMap version 25 2024-09-12T02:05:00.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.712+0000 7f6f78537740 1 -- 172.21.15.16:0/3366831094 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5563f9e05e30 con 0x5563f9de9b40 2024-09-12T02:05:00.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.712+0000 7f6f4effd640 1 -- 172.21.15.16:0/3366831094 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f60005020 con 0x5563f9de9b40 2024-09-12T02:05:00.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.713+0000 7f6f4effd640 1 -- 172.21.15.16:0/3366831094 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f6f60044400 con 0x5563f9de9b40 2024-09-12T02:05:00.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.713+0000 7f6f78537740 10 client.16317 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:05:00.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.713+0000 7f6f78537740 10 client.16317 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:05:00.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.713+0000 7f6f78537740 20 client.16317 populate_metadata read hostname 'smithi016' 2024-09-12T02:05:00.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.713+0000 7f6f78537740 1 --2- 172.21.15.16:0/3366831094 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5563f9e07180 0x5563f9e09570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:00.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.713+0000 7f6f78537740 4 client.16317 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:05:00.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.713+0000 7f6f78537740 1 -- 172.21.15.16:0/3366831094 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5563f9d5b860 con 0x5563f9e07180 2024-09-12T02:05:00.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.713+0000 7f6f6effd640 1 --2- 172.21.15.16:0/3366831094 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5563f9e07180 0x5563f9e09570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:00.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.713+0000 7f6f4effd640 10 client.16317 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:05:00.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.713+0000 7f6f6effd640 1 --2- 172.21.15.16:0/3366831094 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5563f9e07180 0x5563f9e09570 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=833fb15d882e6fb3 server_cookie=f98d158554cf5b34 in_seq=0 out_seq=0 2024-09-12T02:05:00.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.714+0000 7f6f4effd640 1 -- 172.21.15.16:0/3366831094 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5563f9d5b860 con 0x5563f9e07180 2024-09-12T02:05:00.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.714+0000 7f6f4effd640 10 client.16317 handle_command_reply: tid=0 2024-09-12T02:05:00.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.714+0000 7f6f78537740 1 client.16317 shutdown 2024-09-12T02:05:00.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.714+0000 7f6f78537740 1 -- 172.21.15.16:0/3366831094 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563f9de9b40 msgr2=0x5563f9df43a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:00.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.714+0000 7f6f78537740 1 --2- 172.21.15.16:0/3366831094 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563f9de9b40 0x5563f9df43a0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f6f6002a930 tx=0x7f6f60005470 comp rx=0 tx=0).stop 2024-09-12T02:05:00.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.714+0000 7f6f78537740 1 -- 172.21.15.16:0/3366831094 shutdown_connections 2024-09-12T02:05:00.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.714+0000 7f6f78537740 1 --2- 172.21.15.16:0/3366831094 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5563f9e07180 0x5563f9e09570 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:00.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.714+0000 7f6f78537740 1 --2- 172.21.15.16:0/3366831094 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5563f9df7780 0x5563f9df4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:00.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.714+0000 7f6f78537740 1 --2- 172.21.15.16:0/3366831094 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563f9de9b40 0x5563f9df43a0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:00.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.714+0000 7f6f4effd640 0 client.16317 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:05:00.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.714+0000 7f6f78537740 1 --2- 172.21.15.16:0/3366831094 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5563f9de85d0 0x5563f9df70f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:00.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.714+0000 7f6f78537740 1 -- 172.21.15.16:0/3366831094 >> 172.21.15.16:0/3366831094 conn(0x5563f9db1310 msgr2=0x5563f9dea930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:00.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.714+0000 7f6f78537740 1 -- 172.21.15.16:0/3366831094 shutdown_connections 2024-09-12T02:05:00.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.715+0000 7f6f78537740 1 -- 172.21.15.16:0/3366831094 wait complete. 2024-09-12T02:05:00.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.715+0000 7f6f78537740 20 client.16317 trim_cache size 0 max 16384 2024-09-12T02:05:00.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.727+0000 7f6f78537740 1 Processor -- start 2024-09-12T02:05:00.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.728+0000 7f6f78537740 1 -- start start 2024-09-12T02:05:00.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.728+0000 7f6f78537740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5563f9de85d0 0x5563f9df4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:00.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.728+0000 7f6f78537740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5563f9de9b40 0x5563f9e036c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:00.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.728+0000 7f6f78537740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563f9df7780 0x5563f9ddfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:00.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.728+0000 7f6f78537740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6f701adb20 con 0x5563f9df7780 2024-09-12T02:05:00.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.728+0000 7f6f78537740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6f701ab3f0 con 0x5563f9de85d0 2024-09-12T02:05:00.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.728+0000 7f6f78537740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f6f701a9260 con 0x5563f9de9b40 2024-09-12T02:05:00.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.728+0000 7f6f6effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563f9df7780 0x5563f9ddfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:00.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.728+0000 7f6f6effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563f9df7780 0x5563f9ddfad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57982/0 (socket says 172.21.15.16:57982) 2024-09-12T02:05:00.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.728+0000 7f6f6effd640 1 -- 172.21.15.16:0/3126948916 learned_addr learned my addr 172.21.15.16:0/3126948916 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:05:00.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.728+0000 7f6f6f7fe640 1 --2- 172.21.15.16:0/3126948916 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5563f9de9b40 0x5563f9e036c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:00.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.728+0000 7f6f6effd640 1 --2- 172.21.15.16:0/3126948916 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5563f9de85d0 0x5563f9df4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:00.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.728+0000 7f6f6f7fe640 1 -- 172.21.15.16:0/3126948916 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5563f9de85d0 msgr2=0x5563f9df4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:00.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.728+0000 7f6f6f7fe640 1 --2- 172.21.15.16:0/3126948916 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5563f9de85d0 0x5563f9df4370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:00.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.728+0000 7f6f6f7fe640 1 -- 172.21.15.16:0/3126948916 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563f9df7780 msgr2=0x5563f9ddfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:00.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.728+0000 7f6f6f7fe640 1 --2- 172.21.15.16:0/3126948916 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563f9df7780 0x5563f9ddfad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:00.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.728+0000 7f6f6f7fe640 1 -- 172.21.15.16:0/3126948916 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563f9df3350 con 0x5563f9de9b40 2024-09-12T02:05:00.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.728+0000 7f6f6effd640 1 --2- 172.21.15.16:0/3126948916 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563f9df7780 0x5563f9ddfad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:05:00.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.729+0000 7f6f6f7fe640 1 --2- 172.21.15.16:0/3126948916 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5563f9de9b40 0x5563f9e036c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f6f60006010 tx=0x7f6f60031630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e3a54d466456e3e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:00.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.729+0000 7f6f557fa640 1 -- 172.21.15.16:0/3126948916 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f6004b040 con 0x5563f9de9b40 2024-09-12T02:05:00.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.729+0000 7f6f557fa640 1 -- 172.21.15.16:0/3126948916 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f60044400 con 0x5563f9de9b40 2024-09-12T02:05:00.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.729+0000 7f6f557fa640 1 -- 172.21.15.16:0/3126948916 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f60035030 con 0x5563f9de9b40 2024-09-12T02:05:00.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.729+0000 7f6f78537740 1 -- 172.21.15.16:0/3126948916 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5563f9de9b40 msgr2=0x5563f9e036c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:00.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.729+0000 7f6f78537740 1 --2- 172.21.15.16:0/3126948916 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5563f9de9b40 0x5563f9e036c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f6f60006010 tx=0x7f6f60031630 comp rx=0 tx=0).stop 2024-09-12T02:05:00.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.730+0000 7f6f78537740 1 -- 172.21.15.16:0/3126948916 shutdown_connections 2024-09-12T02:05:00.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.730+0000 7f6f78537740 1 --2- 172.21.15.16:0/3126948916 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563f9df7780 0x5563f9ddfad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:00.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.730+0000 7f6f78537740 1 --2- 172.21.15.16:0/3126948916 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5563f9de9b40 0x5563f9e036c0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:00.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.730+0000 7f6f78537740 1 --2- 172.21.15.16:0/3126948916 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5563f9de85d0 0x5563f9df4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:00.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.730+0000 7f6f78537740 1 -- 172.21.15.16:0/3126948916 >> 172.21.15.16:0/3126948916 conn(0x5563f9db1310 msgr2=0x5563f9df3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:00.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.730+0000 7f6f78537740 1 -- 172.21.15.16:0/3126948916 shutdown_connections 2024-09-12T02:05:00.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.730+0000 7f6f78537740 1 -- 172.21.15.16:0/3126948916 wait complete. 2024-09-12T02:05:00.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.730+0000 7f6f78537740 1 -- start start 2024-09-12T02:05:00.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.730+0000 7f6f78537740 1 Processor -- start 2024-09-12T02:05:00.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.730+0000 7f6f78537740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5563f9de85d0 0x5563f9dffba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:00.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.730+0000 7f6f78537740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5563f9de9b40 0x5563f9e000e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:00.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.730+0000 7f6f78537740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563f9df7780 0x5563f9df0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:00.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.730+0000 7f6f78537740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6f701a98b0 con 0x5563f9df7780 2024-09-12T02:05:00.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.730+0000 7f6f78537740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6f701aad20 con 0x5563f9de85d0 2024-09-12T02:05:00.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.730+0000 7f6f78537740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f6f701a9f80 con 0x5563f9de9b40 2024-09-12T02:05:00.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.731+0000 7f6f78537740 10 client.0 _ll_register_callbacks cb 0x7f6f60009610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:05:00.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.731+0000 7f6f6effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563f9df7780 0x5563f9df0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:00.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.731+0000 7f6f6effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563f9df7780 0x5563f9df0f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57986/0 (socket says 172.21.15.16:57986) 2024-09-12T02:05:00.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.731+0000 7f6f6effd640 1 -- 172.21.15.16:0/1252230744 learned_addr learned my addr 172.21.15.16:0/1252230744 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:05:00.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.731+0000 7f6f6f7fe640 1 --2- 172.21.15.16:0/1252230744 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5563f9de9b40 0x5563f9e000e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:00.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.731+0000 7f6f6effd640 1 --2- 172.21.15.16:0/1252230744 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5563f9de85d0 0x5563f9dffba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:00.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.731+0000 7f6f6effd640 1 -- 172.21.15.16:0/1252230744 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5563f9de9b40 msgr2=0x5563f9e000e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:00.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.731+0000 7f6f6effd640 1 --2- 172.21.15.16:0/1252230744 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5563f9de9b40 0x5563f9e000e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:00.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.731+0000 7f6f6effd640 1 -- 172.21.15.16:0/1252230744 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5563f9de85d0 msgr2=0x5563f9dffba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:00.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.731+0000 7f6f6effd640 1 --2- 172.21.15.16:0/1252230744 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5563f9de85d0 0x5563f9dffba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:00.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.731+0000 7f6f6effd640 1 -- 172.21.15.16:0/1252230744 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563f9e098e0 con 0x5563f9df7780 2024-09-12T02:05:00.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.731+0000 7f6f6effd640 1 --2- 172.21.15.16:0/1252230744 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563f9df7780 0x5563f9df0f20 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x5563f9de9070 tx=0x7f6f640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=635f27d66ae5727 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:00.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.731+0000 7f6f4effd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:05:00.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.731+0000 7f6f4effd640 1 -- 172.21.15.16:0/1252230744 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f64025070 con 0x5563f9df7780 2024-09-12T02:05:00.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.731+0000 7f6f4effd640 1 -- 172.21.15.16:0/1252230744 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f6f64003d40 con 0x5563f9df7780 2024-09-12T02:05:00.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.731+0000 7f6f78537740 1 -- 172.21.15.16:0/1252230744 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f60044400 con 0x5563f9df7780 2024-09-12T02:05:00.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.732+0000 7f6f4effd640 1 -- 172.21.15.16:0/1252230744 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6f64003f40 con 0x5563f9df7780 2024-09-12T02:05:00.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.732+0000 7f6f78537740 10 client.16323 fetch_fsmap learned FSMap version 25 2024-09-12T02:05:00.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.732+0000 7f6f78537740 1 -- 172.21.15.16:0/1252230744 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5563f9e715e0 con 0x5563f9df7780 2024-09-12T02:05:00.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.732+0000 7f6f4effd640 1 -- 172.21.15.16:0/1252230744 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f60044400 con 0x5563f9df7780 2024-09-12T02:05:00.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.732+0000 7f6f4effd640 1 -- 172.21.15.16:0/1252230744 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f6f64005470 con 0x5563f9df7780 2024-09-12T02:05:00.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.732+0000 7f6f78537740 10 client.16323 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:05:00.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.732+0000 7f6f78537740 10 client.16323 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:05:00.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.732+0000 7f6f78537740 20 client.16323 populate_metadata read hostname 'smithi016' 2024-09-12T02:05:00.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.732+0000 7f6f78537740 1 --2- 172.21.15.16:0/1252230744 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5563f9e71d60 0x5563f9e74180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:00.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.732+0000 7f6f78537740 4 client.16323 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T02:05:00.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.732+0000 7f6f78537740 1 -- 172.21.15.16:0/1252230744 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5563f9d55d70 con 0x5563f9e71d60 2024-09-12T02:05:00.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.732+0000 7f6f6f7fe640 1 --2- 172.21.15.16:0/1252230744 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5563f9e71d60 0x5563f9e74180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:00.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.733+0000 7f6f6f7fe640 1 --2- 172.21.15.16:0/1252230744 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5563f9e71d60 0x5563f9e74180 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=49ce3fd7e04b4ebe server_cookie=a71ba4ab7344566 in_seq=0 out_seq=0 2024-09-12T02:05:00.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.733+0000 7f6f4effd640 10 client.16323 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:05:00.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.733+0000 7f6f4effd640 1 -- 172.21.15.16:0/1252230744 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5563f9d55d70 con 0x5563f9e71d60 2024-09-12T02:05:00.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.733+0000 7f6f4effd640 10 client.16323 handle_command_reply: tid=0 2024-09-12T02:05:00.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.733+0000 7f6f78537740 1 client.16323 shutdown 2024-09-12T02:05:00.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.733+0000 7f6f78537740 1 -- 172.21.15.16:0/1252230744 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563f9df7780 msgr2=0x5563f9df0f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:00.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.733+0000 7f6f78537740 1 --2- 172.21.15.16:0/1252230744 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563f9df7780 0x5563f9df0f20 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x5563f9de9070 tx=0x7f6f640046c0 comp rx=0 tx=0).stop 2024-09-12T02:05:00.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.733+0000 7f6f78537740 1 -- 172.21.15.16:0/1252230744 shutdown_connections 2024-09-12T02:05:00.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.733+0000 7f6f78537740 1 --2- 172.21.15.16:0/1252230744 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5563f9e71d60 0x5563f9e74180 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:00.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.733+0000 7f6f78537740 1 --2- 172.21.15.16:0/1252230744 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563f9df7780 0x5563f9df0f20 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:00.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.733+0000 7f6f78537740 1 --2- 172.21.15.16:0/1252230744 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5563f9de9b40 0x5563f9e000e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:00.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.733+0000 7f6f4effd640 0 client.16323 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:05:00.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.733+0000 7f6f78537740 1 --2- 172.21.15.16:0/1252230744 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5563f9de85d0 0x5563f9dffba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:00.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.733+0000 7f6f78537740 1 -- 172.21.15.16:0/1252230744 >> 172.21.15.16:0/1252230744 conn(0x5563f9db1310 msgr2=0x5563f9de4700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:00.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.733+0000 7f6f78537740 1 -- 172.21.15.16:0/1252230744 shutdown_connections 2024-09-12T02:05:00.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.733+0000 7f6f78537740 1 -- 172.21.15.16:0/1252230744 wait complete. 2024-09-12T02:05:00.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.733+0000 7f6f78537740 20 client.16323 trim_cache size 0 max 16384 2024-09-12T02:05:00.744 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T02:05:00.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.734+0000 7f6f7615e640 1 -- 172.21.15.16:0/1246043242 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f6f580a2b80 msgr2=0x7f6f580a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:00.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.734+0000 7f6f7615e640 1 --2- 172.21.15.16:0/1246043242 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f6f580a2b80 0x7f6f580a5040 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f6f6800caa0 tx=0x7f6f6801cb40 comp rx=0 tx=0).stop 2024-09-12T02:05:00.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.734+0000 7f6f7615e640 1 -- 172.21.15.16:0/1246043242 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f70106f10 msgr2=0x7f6f7019cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:00.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.734+0000 7f6f7615e640 1 --2- 172.21.15.16:0/1246043242 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f70106f10 0x7f6f7019cd00 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f6f6800cad0 tx=0x7f6f6800cf90 comp rx=0 tx=0).stop 2024-09-12T02:05:00.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.734+0000 7f6f7615e640 1 -- 172.21.15.16:0/1246043242 shutdown_connections 2024-09-12T02:05:00.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.734+0000 7f6f7615e640 1 --2- 172.21.15.16:0/1246043242 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f6f580a2b80 0x7f6f580a5040 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:00.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.734+0000 7f6f7615e640 1 --2- 172.21.15.16:0/1246043242 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f70106f10 0x7f6f7019cd00 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:00.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.734+0000 7f6f7615e640 1 --2- 172.21.15.16:0/1246043242 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6f70102790 0x7f6f70198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:00.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.734+0000 7f6f7615e640 1 --2- 172.21.15.16:0/1246043242 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f70101de0 0x7f6f70198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:00.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.734+0000 7f6f7615e640 1 -- 172.21.15.16:0/1246043242 >> 172.21.15.16:0/1246043242 conn(0x7f6f700fd8c0 msgr2=0x7f6f700fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:00.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.734+0000 7f6f7615e640 1 -- 172.21.15.16:0/1246043242 shutdown_connections 2024-09-12T02:05:00.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:00.734+0000 7f6f7615e640 1 -- 172.21.15.16:0/1246043242 wait complete. 2024-09-12T02:05:01.683 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.679+0000 7fc2dfb6d640 1 -- 172.21.15.16:0/673211541 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2d8101c50 msgr2=0x7fc2d81020d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:01.683 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.679+0000 7fc2dfb6d640 1 --2- 172.21.15.16:0/673211541 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2d8101c50 0x7fc2d81020d0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fc2c8009900 tx=0x7fc2c802a410 comp rx=0 tx=0).stop 2024-09-12T02:05:01.683 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.680+0000 7fc2dfb6d640 1 -- 172.21.15.16:0/673211541 shutdown_connections 2024-09-12T02:05:01.683 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.680+0000 7fc2dfb6d640 1 --2- 172.21.15.16:0/673211541 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc2d8102610 0x7fc2d8108ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:01.684 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.680+0000 7fc2dfb6d640 1 --2- 172.21.15.16:0/673211541 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2d8101c50 0x7fc2d81020d0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:01.684 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.680+0000 7fc2dfb6d640 1 --2- 172.21.15.16:0/673211541 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2d8100a60 0x7fc2d8100e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:01.684 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.680+0000 7fc2dfb6d640 1 -- 172.21.15.16:0/673211541 >> 172.21.15.16:0/673211541 conn(0x7fc2d80fc1e0 msgr2=0x7fc2d80fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:01.684 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.680+0000 7fc2dfb6d640 1 -- 172.21.15.16:0/673211541 shutdown_connections 2024-09-12T02:05:01.684 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.680+0000 7fc2dfb6d640 1 -- 172.21.15.16:0/673211541 wait complete. 2024-09-12T02:05:01.684 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.680+0000 7fc2dfb6d640 1 Processor -- start 2024-09-12T02:05:01.684 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.681+0000 7fc2dfb6d640 1 -- start start 2024-09-12T02:05:01.684 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.681+0000 7fc2dfb6d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc2d8100a60 0x7fc2d8197dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:01.684 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.681+0000 7fc2dfb6d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2d8101c50 0x7fc2d8198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:01.684 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.681+0000 7fc2dfb6d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2d8102610 0x7fc2d819f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:01.684 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.681+0000 7fc2dfb6d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc2d810d830 con 0x7fc2d8101c50 2024-09-12T02:05:01.684 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.681+0000 7fc2dfb6d640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc2d810d6b0 con 0x7fc2d8102610 2024-09-12T02:05:01.684 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.681+0000 7fc2dfb6d640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc2d810d9b0 con 0x7fc2d8100a60 2024-09-12T02:05:01.685 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.681+0000 7fc2dd108640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2d8101c50 0x7fc2d8198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:01.685 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.681+0000 7fc2dd108640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2d8101c50 0x7fc2d8198300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44422/0 (socket says 172.21.15.16:44422) 2024-09-12T02:05:01.685 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.681+0000 7fc2dd108640 1 -- 172.21.15.16:0/1164520093 learned_addr learned my addr 172.21.15.16:0/1164520093 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:05:01.685 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.681+0000 7fc2dd909640 1 --2- 172.21.15.16:0/1164520093 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc2d8100a60 0x7fc2d8197dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:01.685 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.681+0000 7fc2de10a640 1 --2- 172.21.15.16:0/1164520093 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2d8102610 0x7fc2d819f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:01.685 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.681+0000 7fc2dd108640 1 -- 172.21.15.16:0/1164520093 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc2d8100a60 msgr2=0x7fc2d8197dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:01.685 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.681+0000 7fc2dd108640 1 --2- 172.21.15.16:0/1164520093 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc2d8100a60 0x7fc2d8197dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:01.685 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.681+0000 7fc2dd108640 1 -- 172.21.15.16:0/1164520093 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2d8102610 msgr2=0x7fc2d819f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:01.685 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.681+0000 7fc2dd108640 1 --2- 172.21.15.16:0/1164520093 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2d8102610 0x7fc2d819f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:01.685 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.681+0000 7fc2dd108640 1 -- 172.21.15.16:0/1164520093 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2d819f9e0 con 0x7fc2d8101c50 2024-09-12T02:05:01.685 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.681+0000 7fc2de10a640 1 --2- 172.21.15.16:0/1164520093 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2d8102610 0x7fc2d819f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:05:01.685 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.681+0000 7fc2dd108640 1 --2- 172.21.15.16:0/1164520093 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2d8101c50 0x7fc2d8198300 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fc2c8009230 tx=0x7fc2c80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:01.686 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.682+0000 7fc2ceffd640 1 -- 172.21.15.16:0/1164520093 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2c803f030 con 0x7fc2d8101c50 2024-09-12T02:05:01.686 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.682+0000 7fc2ceffd640 1 -- 172.21.15.16:0/1164520093 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2c8003ce0 con 0x7fc2d8101c50 2024-09-12T02:05:01.686 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.682+0000 7fc2ceffd640 1 -- 172.21.15.16:0/1164520093 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2c8035070 con 0x7fc2d8101c50 2024-09-12T02:05:01.686 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.682+0000 7fc2dfb6d640 1 -- 172.21.15.16:0/1164520093 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2d819fc70 con 0x7fc2d8101c50 2024-09-12T02:05:01.686 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.682+0000 7fc2dfb6d640 1 -- 172.21.15.16:0/1164520093 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2d81a01b0 con 0x7fc2d8101c50 2024-09-12T02:05:01.689 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.685+0000 7fc2ceffd640 1 -- 172.21.15.16:0/1164520093 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fc2c8042030 con 0x7fc2d8101c50 2024-09-12T02:05:01.689 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.685+0000 7fc2e29eb740 1 Processor -- start 2024-09-12T02:05:01.689 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.685+0000 7fc2e29eb740 1 -- start start 2024-09-12T02:05:01.689 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.686+0000 7fc2e29eb740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564fae16d9a0 0x564fae16dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:01.689 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.686+0000 7fc2e29eb740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564fae16e2e0 0x564fae172da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:01.689 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.686+0000 7fc2e29eb740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564fae173370 0x564fae175830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:01.689 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.686+0000 7fc2e29eb740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x564fae0d12f0 con 0x564fae173370 2024-09-12T02:05:01.689 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.686+0000 7fc2e29eb740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x564fae079270 con 0x564fae16e2e0 2024-09-12T02:05:01.689 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.686+0000 7fc2e29eb740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x564fae0cd2b0 con 0x564fae16d9a0 2024-09-12T02:05:01.689 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.686+0000 7fc2dd108640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564fae173370 0x564fae175830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:01.689 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.686+0000 7fc2dd108640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564fae173370 0x564fae175830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44430/0 (socket says 172.21.15.16:44430) 2024-09-12T02:05:01.690 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.686+0000 7fc2dd108640 1 -- 172.21.15.16:0/2612600802 learned_addr learned my addr 172.21.15.16:0/2612600802 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:05:01.690 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.686+0000 7fc2dd909640 1 --2- 172.21.15.16:0/2612600802 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564fae16e2e0 0x564fae172da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:01.690 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.686+0000 7fc2de10a640 1 --2- 172.21.15.16:0/2612600802 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564fae16d9a0 0x564fae16dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:01.690 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.686+0000 7fc2ceffd640 1 --2- 172.21.15.16:0/1164520093 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc2c40a2cb0 0x7fc2c40a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:01.690 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.686+0000 7fc2dd108640 1 -- 172.21.15.16:0/2612600802 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564fae16d9a0 msgr2=0x564fae16dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:01.690 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.686+0000 7fc2dd108640 1 --2- 172.21.15.16:0/2612600802 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564fae16d9a0 0x564fae16dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:01.690 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.686+0000 7fc2dd108640 1 -- 172.21.15.16:0/2612600802 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564fae16e2e0 msgr2=0x564fae172da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:01.690 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.686+0000 7fc2dd108640 1 --2- 172.21.15.16:0/2612600802 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564fae16e2e0 0x564fae172da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:01.690 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.686+0000 7fc2dd108640 1 -- 172.21.15.16:0/2612600802 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564fae078340 con 0x564fae173370 2024-09-12T02:05:01.690 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.686+0000 7fc2dd909640 1 --2- 172.21.15.16:0/2612600802 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564fae16e2e0 0x564fae172da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:05:01.690 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.686+0000 7fc2de10a640 1 --2- 172.21.15.16:0/1164520093 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc2c40a2cb0 0x7fc2c40a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:01.690 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.686+0000 7fc2ceffd640 1 -- 172.21.15.16:0/1164520093 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fc2c80ef410 con 0x7fc2d8101c50 2024-09-12T02:05:01.690 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.686+0000 7fc2dd108640 1 --2- 172.21.15.16:0/2612600802 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564fae173370 0x564fae175830 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fc2c8005450 tx=0x7fc2c804b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3db5f021f0a31d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:01.691 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.687+0000 7fc2c3fff640 1 -- 172.21.15.16:0/2612600802 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2c8049070 con 0x564fae173370 2024-09-12T02:05:01.691 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.687+0000 7fc2de10a640 1 --2- 172.21.15.16:0/1164520093 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc2c40a2cb0 0x7fc2c40a5170 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fc2d0005fd0 tx=0x7fc2d0004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:01.691 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.687+0000 7fc2c3fff640 1 -- 172.21.15.16:0/2612600802 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2c8049e80 con 0x564fae173370 2024-09-12T02:05:01.691 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.687+0000 7fc2c3fff640 1 -- 172.21.15.16:0/2612600802 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2c800acf0 con 0x564fae173370 2024-09-12T02:05:01.691 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.687+0000 7fc2e29eb740 1 -- 172.21.15.16:0/2612600802 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564fae173370 msgr2=0x564fae175830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:01.691 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.687+0000 7fc2e29eb740 1 --2- 172.21.15.16:0/2612600802 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564fae173370 0x564fae175830 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fc2c8005450 tx=0x7fc2c804b000 comp rx=0 tx=0).stop 2024-09-12T02:05:01.691 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.687+0000 7fc2e29eb740 1 -- 172.21.15.16:0/2612600802 shutdown_connections 2024-09-12T02:05:01.691 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.687+0000 7fc2e29eb740 1 --2- 172.21.15.16:0/2612600802 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564fae173370 0x564fae175830 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:01.691 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.687+0000 7fc2e29eb740 1 --2- 172.21.15.16:0/2612600802 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564fae16e2e0 0x564fae172da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:01.691 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.687+0000 7fc2e29eb740 1 --2- 172.21.15.16:0/2612600802 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564fae16d9a0 0x564fae16dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:01.691 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.687+0000 7fc2e29eb740 1 -- 172.21.15.16:0/2612600802 >> 172.21.15.16:0/2612600802 conn(0x564fae169010 msgr2=0x564fae169410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:01.691 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.688+0000 7fc2e29eb740 1 -- 172.21.15.16:0/2612600802 shutdown_connections 2024-09-12T02:05:01.691 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.688+0000 7fc2e29eb740 1 -- 172.21.15.16:0/2612600802 wait complete. 2024-09-12T02:05:01.691 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.688+0000 7fc2e29eb740 1 -- start start 2024-09-12T02:05:01.692 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.688+0000 7fc2e29eb740 1 Processor -- start 2024-09-12T02:05:01.695 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.688+0000 7fc2e29eb740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564fae16d9a0 0x564fae177860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:01.696 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.688+0000 7fc2e29eb740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564fae16e2e0 0x564fae177da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:01.696 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.688+0000 7fc2e29eb740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564fae173370 0x564fae17ede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:01.696 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.688+0000 7fc2e29eb740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x564fae176100 con 0x564fae173370 2024-09-12T02:05:01.696 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.688+0000 7fc2e29eb740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x564fae05a290 con 0x564fae16d9a0 2024-09-12T02:05:01.696 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.688+0000 7fc2e29eb740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x564fae176280 con 0x564fae16e2e0 2024-09-12T02:05:01.696 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.689+0000 7fc2e29eb740 10 client.0 _ll_register_callbacks cb 0x564fae08b140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:05:01.697 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.689+0000 7fc2de10a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564fae173370 0x564fae17ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:01.697 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.689+0000 7fc2dd909640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564fae16d9a0 0x564fae177860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:01.697 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.689+0000 7fc2de10a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564fae173370 0x564fae17ede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44446/0 (socket says 172.21.15.16:44446) 2024-09-12T02:05:01.697 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.689+0000 7fc2de10a640 1 -- 172.21.15.16:0/1063855345 learned_addr learned my addr 172.21.15.16:0/1063855345 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:05:01.697 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.689+0000 7fc2dd909640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564fae16d9a0 0x564fae177860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.16:47804/0 (socket says 172.21.15.16:47804) 2024-09-12T02:05:01.697 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.689+0000 7fc2dd108640 1 --2- 172.21.15.16:0/1063855345 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564fae16e2e0 0x564fae177da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:01.697 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.689+0000 7fc2de10a640 1 -- 172.21.15.16:0/1063855345 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564fae16e2e0 msgr2=0x564fae177da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:01.697 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.689+0000 7fc2de10a640 1 --2- 172.21.15.16:0/1063855345 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564fae16e2e0 0x564fae177da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:01.697 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.689+0000 7fc2de10a640 1 -- 172.21.15.16:0/1063855345 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564fae16d9a0 msgr2=0x564fae177860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:01.698 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.689+0000 7fc2de10a640 1 --2- 172.21.15.16:0/1063855345 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564fae16d9a0 0x564fae177860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:01.698 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.689+0000 7fc2de10a640 1 -- 172.21.15.16:0/1063855345 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564fadde4210 con 0x564fae173370 2024-09-12T02:05:01.698 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.689+0000 7fc2dd909640 1 --2- 172.21.15.16:0/1063855345 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564fae16d9a0 0x564fae177860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:05:01.698 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.689+0000 7fc2de10a640 1 --2- 172.21.15.16:0/1063855345 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564fae173370 0x564fae17ede0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fc2d0012710 tx=0x7fc2d0004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=889a41ee6c5985f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:01.698 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.689+0000 7fc2c17fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:05:01.698 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.689+0000 7fc2c17fa640 1 -- 172.21.15.16:0/1063855345 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2d001c9e0 con 0x564fae173370 2024-09-12T02:05:01.698 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.689+0000 7fc2c17fa640 1 -- 172.21.15.16:0/1063855345 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fc2d001cb80 con 0x564fae173370 2024-09-12T02:05:01.698 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.689+0000 7fc2e29eb740 1 -- 172.21.15.16:0/1063855345 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc2c8049e80 con 0x564fae173370 2024-09-12T02:05:01.698 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.689+0000 7fc2c17fa640 1 -- 172.21.15.16:0/1063855345 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc2d001cd80 con 0x564fae173370 2024-09-12T02:05:01.698 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.689+0000 7fc2e29eb740 10 client.16335 fetch_fsmap learned FSMap version 25 2024-09-12T02:05:01.698 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.690+0000 7fc2e29eb740 1 -- 172.21.15.16:0/1063855345 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x564fadeb0620 con 0x564fae173370 2024-09-12T02:05:01.698 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.690+0000 7fc2c17fa640 1 -- 172.21.15.16:0/1063855345 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2c8049e80 con 0x564fae173370 2024-09-12T02:05:01.698 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.690+0000 7fc2c17fa640 1 -- 172.21.15.16:0/1063855345 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fc2d0019030 con 0x564fae173370 2024-09-12T02:05:01.699 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.690+0000 7fc2e29eb740 10 client.16335 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:05:01.699 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.690+0000 7fc2e29eb740 10 client.16335 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:05:01.699 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.690+0000 7fc2e29eb740 20 client.16335 populate_metadata read hostname 'smithi016' 2024-09-12T02:05:01.699 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.690+0000 7fc2e29eb740 1 --2- 172.21.15.16:0/1063855345 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x564fae183580 0x564fae185a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:01.699 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.690+0000 7fc2e29eb740 4 client.16335 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:05:01.699 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.690+0000 7fc2e29eb740 1 -- 172.21.15.16:0/1063855345 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564fae0dea20 con 0x564fae183580 2024-09-12T02:05:01.699 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.690+0000 7fc2dd909640 1 --2- 172.21.15.16:0/1063855345 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x564fae183580 0x564fae185a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:01.699 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.691+0000 7fc2dd909640 1 --2- 172.21.15.16:0/1063855345 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x564fae183580 0x564fae185a10 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20d6ec9a7933d77f server_cookie=171295fd2e206d19 in_seq=0 out_seq=0 2024-09-12T02:05:01.699 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.691+0000 7fc2c17fa640 10 client.16335 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:05:01.699 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.691+0000 7fc2c17fa640 1 -- 172.21.15.16:0/1063855345 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564fae0dea20 con 0x564fae183580 2024-09-12T02:05:01.699 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.691+0000 7fc2c17fa640 10 client.16335 handle_command_reply: tid=0 2024-09-12T02:05:01.699 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.691+0000 7fc2e29eb740 1 client.16335 shutdown 2024-09-12T02:05:01.699 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.691+0000 7fc2e29eb740 1 -- 172.21.15.16:0/1063855345 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564fae173370 msgr2=0x564fae17ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:01.700 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.691+0000 7fc2e29eb740 1 --2- 172.21.15.16:0/1063855345 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564fae173370 0x564fae17ede0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fc2d0012710 tx=0x7fc2d0004ab0 comp rx=0 tx=0).stop 2024-09-12T02:05:01.700 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.692+0000 7fc2e29eb740 1 -- 172.21.15.16:0/1063855345 shutdown_connections 2024-09-12T02:05:01.700 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.692+0000 7fc2e29eb740 1 --2- 172.21.15.16:0/1063855345 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x564fae183580 0x564fae185a10 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:01.700 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.692+0000 7fc2e29eb740 1 --2- 172.21.15.16:0/1063855345 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564fae173370 0x564fae17ede0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:01.700 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.692+0000 7fc2e29eb740 1 --2- 172.21.15.16:0/1063855345 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564fae16e2e0 0x564fae177da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:01.700 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.692+0000 7fc2e29eb740 1 --2- 172.21.15.16:0/1063855345 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564fae16d9a0 0x564fae177860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:01.700 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.692+0000 7fc2c17fa640 0 client.16335 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:05:01.700 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.692+0000 7fc2e29eb740 1 -- 172.21.15.16:0/1063855345 >> 172.21.15.16:0/1063855345 conn(0x564fae169010 msgr2=0x564fae16a820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:01.700 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.692+0000 7fc2e29eb740 1 -- 172.21.15.16:0/1063855345 shutdown_connections 2024-09-12T02:05:01.700 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.692+0000 7fc2e29eb740 1 -- 172.21.15.16:0/1063855345 wait complete. 2024-09-12T02:05:01.700 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.692+0000 7fc2e29eb740 20 client.16335 trim_cache size 0 max 16384 2024-09-12T02:05:01.712 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.706+0000 7fc2e29eb740 1 Processor -- start 2024-09-12T02:05:01.712 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.706+0000 7fc2e29eb740 1 -- start start 2024-09-12T02:05:01.712 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.706+0000 7fc2e29eb740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564fae16d9a0 0x564fae169ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:01.712 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.706+0000 7fc2e29eb740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564fae16e2e0 0x564fae17ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:01.712 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.706+0000 7fc2e29eb740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564fae173370 0x564fae16a400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:01.712 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.706+0000 7fc2e29eb740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc2d81ab770 con 0x564fae16e2e0 2024-09-12T02:05:01.712 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.706+0000 7fc2e29eb740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc2d81a9040 con 0x564fae16d9a0 2024-09-12T02:05:01.712 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.706+0000 7fc2e29eb740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc2d81a6e90 con 0x564fae173370 2024-09-12T02:05:01.712 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.706+0000 7fc2dd108640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564fae16e2e0 0x564fae17ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:01.712 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.707+0000 7fc2dd108640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564fae16e2e0 0x564fae17ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44456/0 (socket says 172.21.15.16:44456) 2024-09-12T02:05:01.712 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.707+0000 7fc2dd108640 1 -- 172.21.15.16:0/2467799575 learned_addr learned my addr 172.21.15.16:0/2467799575 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:05:01.712 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.707+0000 7fc2dd909640 1 --2- 172.21.15.16:0/2467799575 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564fae16d9a0 0x564fae169ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:01.713 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.707+0000 7fc2dd108640 1 -- 172.21.15.16:0/2467799575 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564fae173370 msgr2=0x564fae16a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:01.713 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.707+0000 7fc2dd108640 1 --2- 172.21.15.16:0/2467799575 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564fae173370 0x564fae16a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:01.713 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.707+0000 7fc2dd108640 1 -- 172.21.15.16:0/2467799575 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564fae16d9a0 msgr2=0x564fae169ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:01.713 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.707+0000 7fc2dd108640 1 --2- 172.21.15.16:0/2467799575 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564fae16d9a0 0x564fae169ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:01.713 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.707+0000 7fc2dd108640 1 -- 172.21.15.16:0/2467799575 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564fae186090 con 0x564fae16e2e0 2024-09-12T02:05:01.713 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.707+0000 7fc2dd108640 1 --2- 172.21.15.16:0/2467799575 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564fae16e2e0 0x564fae17ed00 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fc2c8031440 tx=0x7fc2c80bb950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98521bcf0bf95e9f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:01.713 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.707+0000 7fc2c37fe640 1 -- 172.21.15.16:0/2467799575 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2c8049050 con 0x564fae16e2e0 2024-09-12T02:05:01.713 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.707+0000 7fc2c37fe640 1 -- 172.21.15.16:0/2467799575 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2c80bc410 con 0x564fae16e2e0 2024-09-12T02:05:01.713 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.707+0000 7fc2c37fe640 1 -- 172.21.15.16:0/2467799575 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2c80b8380 con 0x564fae16e2e0 2024-09-12T02:05:01.713 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.708+0000 7fc2e29eb740 1 -- 172.21.15.16:0/2467799575 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564fae16e2e0 msgr2=0x564fae17ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:01.713 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.708+0000 7fc2e29eb740 1 --2- 172.21.15.16:0/2467799575 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564fae16e2e0 0x564fae17ed00 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fc2c8031440 tx=0x7fc2c80bb950 comp rx=0 tx=0).stop 2024-09-12T02:05:01.713 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.708+0000 7fc2e29eb740 1 -- 172.21.15.16:0/2467799575 shutdown_connections 2024-09-12T02:05:01.713 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.708+0000 7fc2e29eb740 1 --2- 172.21.15.16:0/2467799575 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564fae173370 0x564fae16a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:01.713 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.708+0000 7fc2e29eb740 1 --2- 172.21.15.16:0/2467799575 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564fae16e2e0 0x564fae17ed00 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:01.714 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.708+0000 7fc2e29eb740 1 --2- 172.21.15.16:0/2467799575 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564fae16d9a0 0x564fae169ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:01.714 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.708+0000 7fc2e29eb740 1 -- 172.21.15.16:0/2467799575 >> 172.21.15.16:0/2467799575 conn(0x564fae169010 msgr2=0x564fae177860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:01.714 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.708+0000 7fc2e29eb740 1 -- 172.21.15.16:0/2467799575 shutdown_connections 2024-09-12T02:05:01.714 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.708+0000 7fc2e29eb740 1 -- 172.21.15.16:0/2467799575 wait complete. 2024-09-12T02:05:01.714 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.708+0000 7fc2e29eb740 1 -- start start 2024-09-12T02:05:01.714 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.708+0000 7fc2e29eb740 1 Processor -- start 2024-09-12T02:05:01.714 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.708+0000 7fc2e29eb740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564fae16d9a0 0x564fae17bf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:01.714 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.708+0000 7fc2e29eb740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564fae16e2e0 0x564fae17c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:01.714 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.708+0000 7fc2e29eb740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564fae173370 0x564fae17d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:01.714 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.709+0000 7fc2e29eb740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc2d81a7500 con 0x564fae16d9a0 2024-09-12T02:05:01.714 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.709+0000 7fc2e29eb740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc2d81a8970 con 0x564fae16e2e0 2024-09-12T02:05:01.714 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.709+0000 7fc2e29eb740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc2d81a7bd0 con 0x564fae173370 2024-09-12T02:05:01.714 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.709+0000 7fc2e29eb740 10 client.0 _ll_register_callbacks cb 0x7fc2d0009d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:05:01.715 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.709+0000 7fc2dd909640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564fae16d9a0 0x564fae17bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:01.715 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.709+0000 7fc2dd909640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564fae16d9a0 0x564fae17bf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44458/0 (socket says 172.21.15.16:44458) 2024-09-12T02:05:01.715 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.709+0000 7fc2dd909640 1 -- 172.21.15.16:0/2565121392 learned_addr learned my addr 172.21.15.16:0/2565121392 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:05:01.715 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.709+0000 7fc2dd108640 1 --2- 172.21.15.16:0/2565121392 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564fae16e2e0 0x564fae17c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:01.715 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.709+0000 7fc2dd909640 1 -- 172.21.15.16:0/2565121392 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564fae173370 msgr2=0x564fae17d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:01.715 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.709+0000 7fc2de10a640 1 --2- 172.21.15.16:0/2565121392 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564fae173370 0x564fae17d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:01.715 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.709+0000 7fc2dd909640 1 --2- 172.21.15.16:0/2565121392 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564fae173370 0x564fae17d4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:01.715 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.709+0000 7fc2dd909640 1 -- 172.21.15.16:0/2565121392 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564fae16e2e0 msgr2=0x564fae17c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:01.715 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.709+0000 7fc2dd909640 1 --2- 172.21.15.16:0/2565121392 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564fae16e2e0 0x564fae17c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:01.715 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.709+0000 7fc2dd909640 1 -- 172.21.15.16:0/2565121392 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564fae17dab0 con 0x564fae16d9a0 2024-09-12T02:05:01.715 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.709+0000 7fc2dd909640 1 --2- 172.21.15.16:0/2565121392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564fae16d9a0 0x564fae17bf50 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fc2d4000c00 tx=0x7fc2d4012ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74379d0b0d71c5a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:01.716 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.709+0000 7fc2c17fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:05:01.716 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.709+0000 7fc2c17fa640 1 -- 172.21.15.16:0/2565121392 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2d40107b0 con 0x564fae16d9a0 2024-09-12T02:05:01.716 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.709+0000 7fc2c17fa640 1 -- 172.21.15.16:0/2565121392 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fc2d4010950 con 0x564fae16d9a0 2024-09-12T02:05:01.716 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.709+0000 7fc2e29eb740 1 -- 172.21.15.16:0/2565121392 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc2c80bc410 con 0x564fae16d9a0 2024-09-12T02:05:01.716 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.709+0000 7fc2c17fa640 1 -- 172.21.15.16:0/2565121392 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc2d4010b50 con 0x564fae16d9a0 2024-09-12T02:05:01.716 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.710+0000 7fc2e29eb740 10 client.16341 fetch_fsmap learned FSMap version 25 2024-09-12T02:05:01.716 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.710+0000 7fc2e29eb740 1 -- 172.21.15.16:0/2565121392 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x564fae17ee90 con 0x564fae16d9a0 2024-09-12T02:05:01.716 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.710+0000 7fc2c17fa640 1 -- 172.21.15.16:0/2565121392 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2c80bc410 con 0x564fae16d9a0 2024-09-12T02:05:01.716 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.710+0000 7fc2c17fa640 1 -- 172.21.15.16:0/2565121392 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fc2d402f050 con 0x564fae16d9a0 2024-09-12T02:05:01.716 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.710+0000 7fc2e29eb740 10 client.16341 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:05:01.716 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.710+0000 7fc2e29eb740 10 client.16341 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:05:01.716 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.710+0000 7fc2e29eb740 20 client.16341 populate_metadata read hostname 'smithi016' 2024-09-12T02:05:01.716 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.710+0000 7fc2e29eb740 1 --2- 172.21.15.16:0/2565121392 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x564fae1ee8f0 0x564fae1f0d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:01.717 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.710+0000 7fc2e29eb740 4 client.16341 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:05:01.717 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.710+0000 7fc2e29eb740 1 -- 172.21.15.16:0/2565121392 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564fae0d8d70 con 0x564fae1ee8f0 2024-09-12T02:05:01.717 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.710+0000 7fc2dd108640 1 --2- 172.21.15.16:0/2565121392 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x564fae1ee8f0 0x564fae1f0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:01.717 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.711+0000 7fc2c17fa640 10 client.16341 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:05:01.717 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.711+0000 7fc2dd108640 1 --2- 172.21.15.16:0/2565121392 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x564fae1ee8f0 0x564fae1f0d10 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9fdcc08f2c5b8781 server_cookie=3a9192daec3b43ae in_seq=0 out_seq=0 2024-09-12T02:05:01.725 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.723+0000 7fc2c17fa640 1 -- 172.21.15.16:0/2565121392 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564fae0d8d70 con 0x564fae1ee8f0 2024-09-12T02:05:01.726 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.723+0000 7fc2c17fa640 10 client.16341 handle_command_reply: tid=0 2024-09-12T02:05:01.726 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.723+0000 7fc2e29eb740 1 client.16341 shutdown 2024-09-12T02:05:01.726 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.723+0000 7fc2e29eb740 1 -- 172.21.15.16:0/2565121392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564fae16d9a0 msgr2=0x564fae17bf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:01.726 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.723+0000 7fc2e29eb740 1 --2- 172.21.15.16:0/2565121392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564fae16d9a0 0x564fae17bf50 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fc2d4000c00 tx=0x7fc2d4012ae0 comp rx=0 tx=0).stop 2024-09-12T02:05:01.726 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.724+0000 7fc2e29eb740 1 -- 172.21.15.16:0/2565121392 shutdown_connections 2024-09-12T02:05:01.726 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.724+0000 7fc2e29eb740 1 --2- 172.21.15.16:0/2565121392 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x564fae1ee8f0 0x564fae1f0d10 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:01.726 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.724+0000 7fc2e29eb740 1 --2- 172.21.15.16:0/2565121392 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564fae173370 0x564fae17d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:01.726 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.724+0000 7fc2c17fa640 0 client.16341 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:05:01.727 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.724+0000 7fc2e29eb740 1 --2- 172.21.15.16:0/2565121392 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564fae16e2e0 0x564fae17c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:01.727 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.724+0000 7fc2e29eb740 1 --2- 172.21.15.16:0/2565121392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564fae16d9a0 0x564fae17bf50 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:01.727 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.724+0000 7fc2e29eb740 1 -- 172.21.15.16:0/2565121392 >> 172.21.15.16:0/2565121392 conn(0x564fae169010 msgr2=0x564fae1679a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:01.727 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.724+0000 7fc2e29eb740 1 -- 172.21.15.16:0/2565121392 shutdown_connections 2024-09-12T02:05:01.727 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.724+0000 7fc2e29eb740 1 -- 172.21.15.16:0/2565121392 wait complete. 2024-09-12T02:05:01.728 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.724+0000 7fc2e29eb740 20 client.16341 trim_cache size 0 max 16384 2024-09-12T02:05:01.728 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-09-12T02:05:01.728 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-09-12T02:05:01.728 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-09-12T02:05:01.728 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-09-12T02:05:01.729 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.724+0000 7fc2dfb6d640 1 -- 172.21.15.16:0/1164520093 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc2c40a2cb0 msgr2=0x7fc2c40a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:01.729 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.724+0000 7fc2dfb6d640 1 --2- 172.21.15.16:0/1164520093 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc2c40a2cb0 0x7fc2c40a5170 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fc2d0005fd0 tx=0x7fc2d0004a40 comp rx=0 tx=0).stop 2024-09-12T02:05:01.730 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.724+0000 7fc2dfb6d640 1 -- 172.21.15.16:0/1164520093 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2d8101c50 msgr2=0x7fc2d8198300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:01.730 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.724+0000 7fc2dfb6d640 1 --2- 172.21.15.16:0/1164520093 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2d8101c50 0x7fc2d8198300 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fc2c8009230 tx=0x7fc2c80042f0 comp rx=0 tx=0).stop 2024-09-12T02:05:01.730 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.724+0000 7fc2dfb6d640 1 -- 172.21.15.16:0/1164520093 shutdown_connections 2024-09-12T02:05:01.730 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.724+0000 7fc2dfb6d640 1 --2- 172.21.15.16:0/1164520093 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc2c40a2cb0 0x7fc2c40a5170 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:01.730 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.724+0000 7fc2dfb6d640 1 --2- 172.21.15.16:0/1164520093 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2d8102610 0x7fc2d819f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:01.730 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.725+0000 7fc2dfb6d640 1 --2- 172.21.15.16:0/1164520093 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2d8101c50 0x7fc2d8198300 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:01.730 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.725+0000 7fc2dfb6d640 1 --2- 172.21.15.16:0/1164520093 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc2d8100a60 0x7fc2d8197dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:01.730 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.725+0000 7fc2dfb6d640 1 -- 172.21.15.16:0/1164520093 >> 172.21.15.16:0/1164520093 conn(0x7fc2d80fc1e0 msgr2=0x7fc2d8104f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:01.730 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.725+0000 7fc2dfb6d640 1 -- 172.21.15.16:0/1164520093 shutdown_connections 2024-09-12T02:05:01.730 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:01.725+0000 7fc2dfb6d640 1 -- 172.21.15.16:0/1164520093 wait complete. 2024-09-12T02:05:02.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:05:01 smithi184 ceph-mon[35593]: pgmap v428: 97 pgs: 97 active+clean; 1.7 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-09-12T02:05:02.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:05:01 smithi033 ceph-mon[35332]: pgmap v428: 97 pgs: 97 active+clean; 1.7 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-09-12T02:05:02.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:05:01 smithi016 ceph-mon[30947]: pgmap v428: 97 pgs: 97 active+clean; 1.7 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-09-12T02:05:02.517 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-12T02:05:02.518 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-12T02:05:02.789 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:05:03.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.249+0000 7f156f84b640 1 -- 172.21.15.16:0/3945984783 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f156810c170 msgr2=0x7f156810e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:03.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.249+0000 7f156f84b640 1 --2- 172.21.15.16:0/3945984783 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f156810c170 0x7f156810e610 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f1560009ca0 tx=0x7f156002a950 comp rx=0 tx=0).stop 2024-09-12T02:05:03.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.249+0000 7f156f84b640 1 -- 172.21.15.16:0/3945984783 shutdown_connections 2024-09-12T02:05:03.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.249+0000 7f156f84b640 1 --2- 172.21.15.16:0/3945984783 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f156810c170 0x7f156810e610 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:03.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.249+0000 7f156f84b640 1 --2- 172.21.15.16:0/3945984783 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1568103660 0x7f156810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:03.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.249+0000 7f156f84b640 1 --2- 172.21.15.16:0/3945984783 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1568102d40 0x7f1568103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:03.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.249+0000 7f156f84b640 1 -- 172.21.15.16:0/3945984783 >> 172.21.15.16:0/3945984783 conn(0x7f15680fc7a0 msgr2=0x7f15680febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:03.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.249+0000 7f156f84b640 1 -- 172.21.15.16:0/3945984783 shutdown_connections 2024-09-12T02:05:03.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.249+0000 7f156f84b640 1 -- 172.21.15.16:0/3945984783 wait complete. 2024-09-12T02:05:03.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.250+0000 7f156f84b640 1 Processor -- start 2024-09-12T02:05:03.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.250+0000 7f156f84b640 1 -- start start 2024-09-12T02:05:03.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.250+0000 7f156f84b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1568102d40 0x7f1568196200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:03.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.250+0000 7f156f84b640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1568103660 0x7f1568196740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:03.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.250+0000 7f156f84b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f156810c170 0x7f156819aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:03.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.250+0000 7f156f84b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f156810fd60 con 0x7f1568102d40 2024-09-12T02:05:03.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.250+0000 7f156f84b640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f156810fbe0 con 0x7f156810c170 2024-09-12T02:05:03.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.250+0000 7f156f84b640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f156810fee0 con 0x7f1568103660 2024-09-12T02:05:03.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.250+0000 7f156d5e7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1568102d40 0x7f1568196200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:03.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.250+0000 7f156d5e7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1568102d40 0x7f1568196200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44474/0 (socket says 172.21.15.16:44474) 2024-09-12T02:05:03.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.250+0000 7f156d5e7640 1 -- 172.21.15.16:0/2294795242 learned_addr learned my addr 172.21.15.16:0/2294795242 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:05:03.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.250+0000 7f156dde8640 1 --2- 172.21.15.16:0/2294795242 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f156810c170 0x7f156819aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:03.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.250+0000 7f156cde6640 1 --2- 172.21.15.16:0/2294795242 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1568103660 0x7f1568196740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:03.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.250+0000 7f156d5e7640 1 -- 172.21.15.16:0/2294795242 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1568103660 msgr2=0x7f1568196740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:03.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.250+0000 7f156d5e7640 1 --2- 172.21.15.16:0/2294795242 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1568103660 0x7f1568196740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:03.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.250+0000 7f156d5e7640 1 -- 172.21.15.16:0/2294795242 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f156810c170 msgr2=0x7f156819aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:03.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.250+0000 7f156d5e7640 1 --2- 172.21.15.16:0/2294795242 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f156810c170 0x7f156819aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:03.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.250+0000 7f156d5e7640 1 -- 172.21.15.16:0/2294795242 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f156819b1b0 con 0x7f1568102d40 2024-09-12T02:05:03.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.250+0000 7f156dde8640 1 --2- 172.21.15.16:0/2294795242 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f156810c170 0x7f156819aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:05:03.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.250+0000 7f156cde6640 1 --2- 172.21.15.16:0/2294795242 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1568103660 0x7f1568196740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:05:03.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.251+0000 7f156d5e7640 1 --2- 172.21.15.16:0/2294795242 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1568102d40 0x7f1568196200 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f1564004650 tx=0x7f1564003da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:03.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.251+0000 7f155e7fc640 1 -- 172.21.15.16:0/2294795242 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1564009b60 con 0x7f1568102d40 2024-09-12T02:05:03.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.251+0000 7f155e7fc640 1 -- 172.21.15.16:0/2294795242 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1564009d00 con 0x7f1568102d40 2024-09-12T02:05:03.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.251+0000 7f156f84b640 1 -- 172.21.15.16:0/2294795242 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f156819b4a0 con 0x7f1568102d40 2024-09-12T02:05:03.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.251+0000 7f155e7fc640 1 -- 172.21.15.16:0/2294795242 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1564018420 con 0x7f1568102d40 2024-09-12T02:05:03.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.251+0000 7f156f84b640 1 -- 172.21.15.16:0/2294795242 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15681a2d80 con 0x7f1568102d40 2024-09-12T02:05:03.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.253+0000 7f155e7fc640 1 -- 172.21.15.16:0/2294795242 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f156401f020 con 0x7f1568102d40 2024-09-12T02:05:03.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.253+0000 7f1571c24740 1 Processor -- start 2024-09-12T02:05:03.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.254+0000 7f1571c24740 1 -- start start 2024-09-12T02:05:03.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.254+0000 7f1571c24740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cafa202b60 0x55cafa202f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:03.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.254+0000 7f1571c24740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cafa203480 0x55cafa20d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:03.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.254+0000 7f1571c24740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cafa20dd40 0x55cafa210130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:03.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.254+0000 7f1571c24740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55cafa168280 con 0x55cafa202b60 2024-09-12T02:05:03.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.254+0000 7f1571c24740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55cafa10fb10 con 0x55cafa20dd40 2024-09-12T02:05:03.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.254+0000 7f1571c24740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55cafa164240 con 0x55cafa203480 2024-09-12T02:05:03.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.254+0000 7f156dde8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cafa202b60 0x55cafa202f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:03.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.254+0000 7f156dde8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cafa202b60 0x55cafa202f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44488/0 (socket says 172.21.15.16:44488) 2024-09-12T02:05:03.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.254+0000 7f156dde8640 1 -- 172.21.15.16:0/3258622542 learned_addr learned my addr 172.21.15.16:0/3258622542 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:05:03.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.254+0000 7f156d5e7640 1 --2- 172.21.15.16:0/3258622542 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cafa203480 0x55cafa20d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:03.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.254+0000 7f156cde6640 1 --2- 172.21.15.16:0/3258622542 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cafa20dd40 0x55cafa210130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:03.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.254+0000 7f155e7fc640 1 --2- 172.21.15.16:0/2294795242 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f15540a2c00 0x7f15540a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:03.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.254+0000 7f156dde8640 1 -- 172.21.15.16:0/3258622542 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cafa203480 msgr2=0x55cafa20d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:03.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.254+0000 7f156dde8640 1 --2- 172.21.15.16:0/3258622542 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cafa203480 0x55cafa20d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:03.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.254+0000 7f156dde8640 1 -- 172.21.15.16:0/3258622542 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cafa20dd40 msgr2=0x55cafa210130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:03.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.254+0000 7f156dde8640 1 --2- 172.21.15.16:0/3258622542 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cafa20dd40 0x55cafa210130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:03.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.254+0000 7f156dde8640 1 -- 172.21.15.16:0/3258622542 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cafa11c1f0 con 0x55cafa202b60 2024-09-12T02:05:03.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.254+0000 7f155e7fc640 1 -- 172.21.15.16:0/2294795242 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f15640d2950 con 0x7f1568102d40 2024-09-12T02:05:03.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.255+0000 7f156dde8640 1 --2- 172.21.15.16:0/2294795242 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f15540a2c00 0x7f15540a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:03.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.255+0000 7f156dde8640 1 --2- 172.21.15.16:0/3258622542 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cafa202b60 0x55cafa202f40 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f1560009830 tx=0x7f1560009860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2705e14d49fe3295 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:03.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.255+0000 7f15537fe640 1 -- 172.21.15.16:0/3258622542 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f156004b070 con 0x55cafa202b60 2024-09-12T02:05:03.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.255+0000 7f15537fe640 1 -- 172.21.15.16:0/3258622542 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1560003e40 con 0x55cafa202b60 2024-09-12T02:05:03.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.255+0000 7f15537fe640 1 -- 172.21.15.16:0/3258622542 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1560034040 con 0x55cafa202b60 2024-09-12T02:05:03.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.256+0000 7f1571c24740 1 -- 172.21.15.16:0/3258622542 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cafa202b60 msgr2=0x55cafa202f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:03.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.256+0000 7f156dde8640 1 --2- 172.21.15.16:0/2294795242 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f15540a2c00 0x7f15540a50c0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f1560048000 tx=0x7f1560049000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:03.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.256+0000 7f1571c24740 1 --2- 172.21.15.16:0/3258622542 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cafa202b60 0x55cafa202f40 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f1560009830 tx=0x7f1560009860 comp rx=0 tx=0).stop 2024-09-12T02:05:03.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.256+0000 7f1571c24740 1 -- 172.21.15.16:0/3258622542 shutdown_connections 2024-09-12T02:05:03.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.256+0000 7f1571c24740 1 --2- 172.21.15.16:0/3258622542 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cafa20dd40 0x55cafa210130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:03.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.256+0000 7f1571c24740 1 --2- 172.21.15.16:0/3258622542 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cafa203480 0x55cafa20d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:03.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.256+0000 7f1571c24740 1 --2- 172.21.15.16:0/3258622542 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cafa202b60 0x55cafa202f40 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:03.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.256+0000 7f1571c24740 1 -- 172.21.15.16:0/3258622542 >> 172.21.15.16:0/3258622542 conn(0x55cafa1ff0f0 msgr2=0x55cafa1ff4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:03.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.256+0000 7f1571c24740 1 -- 172.21.15.16:0/3258622542 shutdown_connections 2024-09-12T02:05:03.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.256+0000 7f1571c24740 1 -- 172.21.15.16:0/3258622542 wait complete. 2024-09-12T02:05:03.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.256+0000 7f1571c24740 1 -- start start 2024-09-12T02:05:03.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.256+0000 7f1571c24740 1 Processor -- start 2024-09-12T02:05:03.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.257+0000 7f1571c24740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cafa202b60 0x55cafa20ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:03.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.257+0000 7f1571c24740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cafa203480 0x55cafa20b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:03.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.257+0000 7f1571c24740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cafa20dd40 0x55cafa20b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:03.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.257+0000 7f1571c24740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55cafa210910 con 0x55cafa203480 2024-09-12T02:05:03.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.257+0000 7f1571c24740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55cafa122f10 con 0x55cafa202b60 2024-09-12T02:05:03.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.257+0000 7f1571c24740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55cafa210a90 con 0x55cafa20dd40 2024-09-12T02:05:03.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.257+0000 7f1571c24740 10 client.0 _ll_register_callbacks cb 0x55cafa0eba40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:05:03.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.257+0000 7f156cde6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cafa203480 0x55cafa20b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:03.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.257+0000 7f156cde6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cafa203480 0x55cafa20b1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44494/0 (socket says 172.21.15.16:44494) 2024-09-12T02:05:03.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.257+0000 7f156cde6640 1 -- 172.21.15.16:0/962608808 learned_addr learned my addr 172.21.15.16:0/962608808 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:05:03.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.257+0000 7f156d5e7640 1 --2- 172.21.15.16:0/962608808 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cafa202b60 0x55cafa20ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:03.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.257+0000 7f156dde8640 1 --2- 172.21.15.16:0/962608808 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cafa20dd40 0x55cafa20b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:03.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.257+0000 7f156cde6640 1 -- 172.21.15.16:0/962608808 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cafa20dd40 msgr2=0x55cafa20b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:03.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.257+0000 7f156cde6640 1 --2- 172.21.15.16:0/962608808 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cafa20dd40 0x55cafa20b6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:03.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.257+0000 7f156cde6640 1 -- 172.21.15.16:0/962608808 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cafa202b60 msgr2=0x55cafa20ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:03.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.257+0000 7f156cde6640 1 --2- 172.21.15.16:0/962608808 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cafa202b60 0x55cafa20ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:03.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.257+0000 7f156cde6640 1 -- 172.21.15.16:0/962608808 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55caf9ed96a0 con 0x55cafa203480 2024-09-12T02:05:03.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.257+0000 7f156cde6640 1 --2- 172.21.15.16:0/962608808 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cafa203480 0x55cafa20b1b0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f1558013790 tx=0x7f1558013c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89a2341ec9910509 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:03.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.257+0000 7f1550ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:05:03.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.257+0000 7f1550ff9640 1 -- 172.21.15.16:0/962608808 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1558020060 con 0x55cafa203480 2024-09-12T02:05:03.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.257+0000 7f1550ff9640 1 -- 172.21.15.16:0/962608808 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f1558011dd0 con 0x55cafa203480 2024-09-12T02:05:03.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.257+0000 7f1571c24740 1 -- 172.21.15.16:0/962608808 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55cafa1757f0 con 0x55cafa203480 2024-09-12T02:05:03.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.258+0000 7f1550ff9640 1 -- 172.21.15.16:0/962608808 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f155800f450 con 0x55cafa203480 2024-09-12T02:05:03.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.258+0000 7f1571c24740 10 client.16353 fetch_fsmap learned FSMap version 25 2024-09-12T02:05:03.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.258+0000 7f1571c24740 1 -- 172.21.15.16:0/962608808 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55cafa21c5b0 con 0x55cafa203480 2024-09-12T02:05:03.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.258+0000 7f1550ff9640 1 -- 172.21.15.16:0/962608808 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55cafa1757f0 con 0x55cafa203480 2024-09-12T02:05:03.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.258+0000 7f1550ff9640 1 -- 172.21.15.16:0/962608808 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f1558025070 con 0x55cafa203480 2024-09-12T02:05:03.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.258+0000 7f1571c24740 10 client.16353 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:05:03.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.258+0000 7f1571c24740 10 client.16353 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:05:03.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.258+0000 7f1571c24740 20 client.16353 populate_metadata read hostname 'smithi016' 2024-09-12T02:05:03.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.258+0000 7f1571c24740 1 --2- 172.21.15.16:0/962608808 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55cafa21da70 0x55cafa21fee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:03.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.258+0000 7f1571c24740 4 client.16353 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:05:03.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.258+0000 7f1571c24740 1 -- 172.21.15.16:0/962608808 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cafa16fd00 con 0x55cafa21da70 2024-09-12T02:05:03.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.258+0000 7f156d5e7640 1 --2- 172.21.15.16:0/962608808 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55cafa21da70 0x55cafa21fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:03.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.259+0000 7f156d5e7640 1 --2- 172.21.15.16:0/962608808 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55cafa21da70 0x55cafa21fee0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4f62ac3870dd1f2 server_cookie=db9685d1800054ca in_seq=0 out_seq=0 2024-09-12T02:05:03.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.259+0000 7f1550ff9640 10 client.16353 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:05:03.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.259+0000 7f1550ff9640 1 -- 172.21.15.16:0/962608808 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cafa16fd00 con 0x55cafa21da70 2024-09-12T02:05:03.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.259+0000 7f1550ff9640 10 client.16353 handle_command_reply: tid=0 2024-09-12T02:05:03.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.259+0000 7f1571c24740 1 client.16353 shutdown 2024-09-12T02:05:03.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.260+0000 7f1571c24740 1 -- 172.21.15.16:0/962608808 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cafa203480 msgr2=0x55cafa20b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:03.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.260+0000 7f1571c24740 1 --2- 172.21.15.16:0/962608808 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cafa203480 0x55cafa20b1b0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f1558013790 tx=0x7f1558013c50 comp rx=0 tx=0).stop 2024-09-12T02:05:03.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.260+0000 7f1571c24740 1 -- 172.21.15.16:0/962608808 shutdown_connections 2024-09-12T02:05:03.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.260+0000 7f1571c24740 1 --2- 172.21.15.16:0/962608808 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55cafa21da70 0x55cafa21fee0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:03.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.260+0000 7f1571c24740 1 --2- 172.21.15.16:0/962608808 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cafa20dd40 0x55cafa20b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:03.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.260+0000 7f1571c24740 1 --2- 172.21.15.16:0/962608808 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cafa203480 0x55cafa20b1b0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:03.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.260+0000 7f1550ff9640 0 client.16353 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:05:03.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.260+0000 7f1571c24740 1 --2- 172.21.15.16:0/962608808 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cafa202b60 0x55cafa20ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:03.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.260+0000 7f1571c24740 1 -- 172.21.15.16:0/962608808 >> 172.21.15.16:0/962608808 conn(0x55cafa1ff0f0 msgr2=0x55cafa1fe270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:03.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.260+0000 7f1571c24740 1 -- 172.21.15.16:0/962608808 shutdown_connections 2024-09-12T02:05:03.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.260+0000 7f1571c24740 1 -- 172.21.15.16:0/962608808 wait complete. 2024-09-12T02:05:03.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.260+0000 7f1571c24740 20 client.16353 trim_cache size 0 max 16384 2024-09-12T02:05:03.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.277+0000 7f1571c24740 1 Processor -- start 2024-09-12T02:05:03.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.277+0000 7f1571c24740 1 -- start start 2024-09-12T02:05:03.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.277+0000 7f1571c24740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cafa202b60 0x55cafa20cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:03.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.277+0000 7f1571c24740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cafa203480 0x55cafa2257f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:03.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.277+0000 7f1571c24740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cafa20dd40 0x55cafa225d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:03.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.277+0000 7f1571c24740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f15681adbd0 con 0x55cafa203480 2024-09-12T02:05:03.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.277+0000 7f1571c24740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f15681ab4a0 con 0x55cafa20dd40 2024-09-12T02:05:03.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.277+0000 7f1571c24740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f15681a92f0 con 0x55cafa202b60 2024-09-12T02:05:03.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.278+0000 7f156cde6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cafa203480 0x55cafa2257f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:03.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.278+0000 7f156cde6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cafa203480 0x55cafa2257f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44500/0 (socket says 172.21.15.16:44500) 2024-09-12T02:05:03.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.278+0000 7f156cde6640 1 -- 172.21.15.16:0/2014645396 learned_addr learned my addr 172.21.15.16:0/2014645396 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:05:03.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.278+0000 7f156dde8640 1 --2- 172.21.15.16:0/2014645396 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cafa20dd40 0x55cafa225d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:03.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.278+0000 7f156d5e7640 1 --2- 172.21.15.16:0/2014645396 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cafa202b60 0x55cafa20cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:03.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.278+0000 7f156cde6640 1 -- 172.21.15.16:0/2014645396 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cafa202b60 msgr2=0x55cafa20cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:03.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.278+0000 7f156cde6640 1 --2- 172.21.15.16:0/2014645396 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cafa202b60 0x55cafa20cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:03.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.278+0000 7f156cde6640 1 -- 172.21.15.16:0/2014645396 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cafa20dd40 msgr2=0x55cafa225d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:03.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.278+0000 7f156cde6640 1 --2- 172.21.15.16:0/2014645396 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cafa20dd40 0x55cafa225d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:03.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.278+0000 7f156cde6640 1 -- 172.21.15.16:0/2014645396 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cafa209c40 con 0x55cafa203480 2024-09-12T02:05:03.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.278+0000 7f156cde6640 1 --2- 172.21.15.16:0/2014645396 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cafa203480 0x55cafa2257f0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f1558005eb0 tx=0x7f1558013720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b60c88cc09a2b6e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:03.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.278+0000 7f1552ffd640 1 -- 172.21.15.16:0/2014645396 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f155802b070 con 0x55cafa203480 2024-09-12T02:05:03.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.278+0000 7f1552ffd640 1 -- 172.21.15.16:0/2014645396 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15580117b0 con 0x55cafa203480 2024-09-12T02:05:03.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.278+0000 7f1552ffd640 1 -- 172.21.15.16:0/2014645396 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f155800f5f0 con 0x55cafa203480 2024-09-12T02:05:03.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.279+0000 7f1571c24740 1 -- 172.21.15.16:0/2014645396 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cafa203480 msgr2=0x55cafa2257f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:03.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.279+0000 7f1571c24740 1 --2- 172.21.15.16:0/2014645396 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cafa203480 0x55cafa2257f0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f1558005eb0 tx=0x7f1558013720 comp rx=0 tx=0).stop 2024-09-12T02:05:03.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.279+0000 7f1571c24740 1 -- 172.21.15.16:0/2014645396 shutdown_connections 2024-09-12T02:05:03.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.279+0000 7f1571c24740 1 --2- 172.21.15.16:0/2014645396 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cafa20dd40 0x55cafa225d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:03.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.279+0000 7f1571c24740 1 --2- 172.21.15.16:0/2014645396 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cafa203480 0x55cafa2257f0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:03.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.279+0000 7f1571c24740 1 --2- 172.21.15.16:0/2014645396 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cafa202b60 0x55cafa20cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:03.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.279+0000 7f1571c24740 1 -- 172.21.15.16:0/2014645396 >> 172.21.15.16:0/2014645396 conn(0x55cafa1ff0f0 msgr2=0x55cafa20c3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:03.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.279+0000 7f1571c24740 1 -- 172.21.15.16:0/2014645396 shutdown_connections 2024-09-12T02:05:03.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.279+0000 7f1571c24740 1 -- 172.21.15.16:0/2014645396 wait complete. 2024-09-12T02:05:03.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.279+0000 7f1571c24740 1 -- start start 2024-09-12T02:05:03.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.279+0000 7f1571c24740 1 Processor -- start 2024-09-12T02:05:03.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.280+0000 7f1571c24740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cafa202b60 0x55cafa216140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:03.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.280+0000 7f1571c24740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cafa203480 0x55cafa212cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:03.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.280+0000 7f1571c24740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cafa20dd40 0x55cafa2131f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:03.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.280+0000 7f1571c24740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f15681a9960 con 0x55cafa203480 2024-09-12T02:05:03.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.280+0000 7f1571c24740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f15681aadd0 con 0x55cafa202b60 2024-09-12T02:05:03.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.280+0000 7f1571c24740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f15681aa030 con 0x55cafa20dd40 2024-09-12T02:05:03.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.280+0000 7f1571c24740 10 client.0 _ll_register_callbacks cb 0x7f156806b7f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:05:03.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.280+0000 7f156cde6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cafa203480 0x55cafa212cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:03.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.280+0000 7f156cde6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cafa203480 0x55cafa212cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44504/0 (socket says 172.21.15.16:44504) 2024-09-12T02:05:03.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.280+0000 7f156cde6640 1 -- 172.21.15.16:0/615263391 learned_addr learned my addr 172.21.15.16:0/615263391 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:05:03.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.280+0000 7f156dde8640 1 --2- 172.21.15.16:0/615263391 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cafa20dd40 0x55cafa2131f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:03.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.280+0000 7f156d5e7640 1 --2- 172.21.15.16:0/615263391 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cafa202b60 0x55cafa216140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:03.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.280+0000 7f156cde6640 1 -- 172.21.15.16:0/615263391 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cafa20dd40 msgr2=0x55cafa2131f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:03.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.280+0000 7f156cde6640 1 --2- 172.21.15.16:0/615263391 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cafa20dd40 0x55cafa2131f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:03.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.280+0000 7f156cde6640 1 -- 172.21.15.16:0/615263391 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cafa202b60 msgr2=0x55cafa216140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:03.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.280+0000 7f156cde6640 1 --2- 172.21.15.16:0/615263391 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cafa202b60 0x55cafa216140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:03.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.280+0000 7f156cde6640 1 -- 172.21.15.16:0/615263391 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cafa1fa830 con 0x55cafa203480 2024-09-12T02:05:03.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.281+0000 7f156cde6640 1 --2- 172.21.15.16:0/615263391 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cafa203480 0x55cafa212cb0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f1558011e70 tx=0x7f1558011ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ee53aeb1b6ba70c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:03.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.281+0000 7f1550ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:05:03.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.281+0000 7f1550ff9640 1 -- 172.21.15.16:0/615263391 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1558020070 con 0x55cafa203480 2024-09-12T02:05:03.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.281+0000 7f1571c24740 1 -- 172.21.15.16:0/615263391 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f15580117b0 con 0x55cafa203480 2024-09-12T02:05:03.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.281+0000 7f1550ff9640 1 -- 172.21.15.16:0/615263391 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f15580050b0 con 0x55cafa203480 2024-09-12T02:05:03.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.281+0000 7f1550ff9640 1 -- 172.21.15.16:0/615263391 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f155802b040 con 0x55cafa203480 2024-09-12T02:05:03.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.281+0000 7f1571c24740 10 client.16359 fetch_fsmap learned FSMap version 25 2024-09-12T02:05:03.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.281+0000 7f1571c24740 1 -- 172.21.15.16:0/615263391 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55cafa2032b0 con 0x55cafa203480 2024-09-12T02:05:03.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.281+0000 7f1550ff9640 1 -- 172.21.15.16:0/615263391 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15580117b0 con 0x55cafa203480 2024-09-12T02:05:03.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.281+0000 7f1550ff9640 1 -- 172.21.15.16:0/615263391 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f1558025050 con 0x55cafa203480 2024-09-12T02:05:03.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.281+0000 7f1571c24740 10 client.16359 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:05:03.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.281+0000 7f1571c24740 10 client.16359 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:05:03.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.281+0000 7f1571c24740 20 client.16359 populate_metadata read hostname 'smithi016' 2024-09-12T02:05:03.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.281+0000 7f1571c24740 1 --2- 172.21.15.16:0/615263391 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55cafa2897e0 0x55cafa28bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:03.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.281+0000 7f1571c24740 4 client.16359 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-12T02:05:03.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.281+0000 7f1571c24740 1 -- 172.21.15.16:0/615263391 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55cafa0ec110 con 0x55cafa2897e0 2024-09-12T02:05:03.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.282+0000 7f156d5e7640 1 --2- 172.21.15.16:0/615263391 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55cafa2897e0 0x55cafa28bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:03.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.282+0000 7f1550ff9640 10 client.16359 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:05:03.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.282+0000 7f156d5e7640 1 --2- 172.21.15.16:0/615263391 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55cafa2897e0 0x55cafa28bbd0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e146a89246b88c1 server_cookie=33c2ff2f3ef3caa0 in_seq=0 out_seq=0 2024-09-12T02:05:03.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.283+0000 7f1550ff9640 1 -- 172.21.15.16:0/615263391 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55cafa0ec110 con 0x55cafa2897e0 2024-09-12T02:05:03.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.283+0000 7f1550ff9640 10 client.16359 handle_command_reply: tid=0 2024-09-12T02:05:03.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.283+0000 7f1571c24740 1 client.16359 shutdown 2024-09-12T02:05:03.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.283+0000 7f1571c24740 1 -- 172.21.15.16:0/615263391 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cafa203480 msgr2=0x55cafa212cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:03.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.283+0000 7f1571c24740 1 --2- 172.21.15.16:0/615263391 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cafa203480 0x55cafa212cb0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f1558011e70 tx=0x7f1558011ea0 comp rx=0 tx=0).stop 2024-09-12T02:05:03.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.283+0000 7f1571c24740 1 -- 172.21.15.16:0/615263391 shutdown_connections 2024-09-12T02:05:03.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.283+0000 7f1571c24740 1 --2- 172.21.15.16:0/615263391 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55cafa2897e0 0x55cafa28bbd0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:03.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.283+0000 7f1571c24740 1 --2- 172.21.15.16:0/615263391 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cafa20dd40 0x55cafa2131f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:03.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.283+0000 7f1571c24740 1 --2- 172.21.15.16:0/615263391 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cafa203480 0x55cafa212cb0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:03.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.283+0000 7f1571c24740 1 --2- 172.21.15.16:0/615263391 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cafa202b60 0x55cafa216140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:03.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.283+0000 7f1571c24740 1 -- 172.21.15.16:0/615263391 >> 172.21.15.16:0/615263391 conn(0x55cafa1ff0f0 msgr2=0x55cafa204dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:03.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.283+0000 7f1550ff9640 0 client.16359 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:05:03.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.284+0000 7f1571c24740 1 -- 172.21.15.16:0/615263391 shutdown_connections 2024-09-12T02:05:03.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.284+0000 7f1571c24740 1 -- 172.21.15.16:0/615263391 wait complete. 2024-09-12T02:05:03.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.284+0000 7f1571c24740 20 client.16359 trim_cache size 0 max 16384 2024-09-12T02:05:03.293 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T02:05:03.293 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-09-12T02:05:03.293 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "974ce6cc-afd3-48ee-83af-ef3568489d00", 2024-09-12T02:05:03.293 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-09-12T02:05:03.293 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T02:05:03.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.284+0000 7f156f84b640 1 -- 172.21.15.16:0/2294795242 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f15540a2c00 msgr2=0x7f15540a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:03.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.284+0000 7f156f84b640 1 --2- 172.21.15.16:0/2294795242 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f15540a2c00 0x7f15540a50c0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f1560048000 tx=0x7f1560049000 comp rx=0 tx=0).stop 2024-09-12T02:05:03.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.284+0000 7f156f84b640 1 -- 172.21.15.16:0/2294795242 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1568102d40 msgr2=0x7f1568196200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:03.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.284+0000 7f156f84b640 1 --2- 172.21.15.16:0/2294795242 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1568102d40 0x7f1568196200 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f1564004650 tx=0x7f1564003da0 comp rx=0 tx=0).stop 2024-09-12T02:05:03.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.284+0000 7f156f84b640 1 -- 172.21.15.16:0/2294795242 shutdown_connections 2024-09-12T02:05:03.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.284+0000 7f156f84b640 1 --2- 172.21.15.16:0/2294795242 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f15540a2c00 0x7f15540a50c0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:03.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.284+0000 7f156f84b640 1 --2- 172.21.15.16:0/2294795242 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f156810c170 0x7f156819aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:03.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.284+0000 7f156f84b640 1 --2- 172.21.15.16:0/2294795242 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1568103660 0x7f1568196740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:03.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.284+0000 7f156f84b640 1 --2- 172.21.15.16:0/2294795242 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1568102d40 0x7f1568196200 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:03.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.284+0000 7f156f84b640 1 -- 172.21.15.16:0/2294795242 >> 172.21.15.16:0/2294795242 conn(0x7f15680fc7a0 msgr2=0x7f15680fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:03.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.284+0000 7f156f84b640 1 -- 172.21.15.16:0/2294795242 shutdown_connections 2024-09-12T02:05:03.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:03.285+0000 7f156f84b640 1 -- 172.21.15.16:0/2294795242 wait complete. 2024-09-12T02:05:03.959 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T02:05:04.183 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:05:03 smithi016 ceph-mon[30947]: pgmap v429: 97 pgs: 97 active+clean; 1.7 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T02:05:04.183 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:05:03 smithi016 ceph-mon[30947]: scrub queued for path: / 2024-09-12T02:05:04.183 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:05:03 smithi016 ceph-mon[30947]: scrub summary: idle+waiting paths [/] 2024-09-12T02:05:04.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:05:03 smithi016 ceph-mon[30947]: scrub summary: active paths [/] 2024-09-12T02:05:04.220 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:05:04.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:05:03 smithi184 ceph-mon[35593]: pgmap v429: 97 pgs: 97 active+clean; 1.7 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T02:05:04.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:05:03 smithi184 ceph-mon[35593]: scrub queued for path: / 2024-09-12T02:05:04.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:05:03 smithi184 ceph-mon[35593]: scrub summary: idle+waiting paths [/] 2024-09-12T02:05:04.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:05:03 smithi184 ceph-mon[35593]: scrub summary: active paths [/] 2024-09-12T02:05:04.418 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:05:03 smithi033 ceph-mon[35332]: pgmap v429: 97 pgs: 97 active+clean; 1.7 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T02:05:04.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:05:03 smithi033 ceph-mon[35332]: scrub queued for path: / 2024-09-12T02:05:04.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:05:03 smithi033 ceph-mon[35332]: scrub summary: idle+waiting paths [/] 2024-09-12T02:05:04.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:05:03 smithi033 ceph-mon[35332]: scrub summary: active paths [/] 2024-09-12T02:05:04.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.666+0000 7f4d95bbf640 1 -- 172.21.15.16:0/738195754 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d90102e00 msgr2=0x7f4d901031e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:04.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.666+0000 7f4d95bbf640 1 --2- 172.21.15.16:0/738195754 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d90102e00 0x7f4d901031e0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f4d80009900 tx=0x7f4d8002a480 comp rx=0 tx=0).stop 2024-09-12T02:05:04.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.666+0000 7f4d95bbf640 1 -- 172.21.15.16:0/738195754 shutdown_connections 2024-09-12T02:05:04.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.666+0000 7f4d95bbf640 1 --2- 172.21.15.16:0/738195754 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4d9010c1a0 0x7f4d9010e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:04.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.666+0000 7f4d95bbf640 1 --2- 172.21.15.16:0/738195754 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d90103720 0x7f4d9010bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:04.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.666+0000 7f4d95bbf640 1 --2- 172.21.15.16:0/738195754 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d90102e00 0x7f4d901031e0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:04.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.666+0000 7f4d95bbf640 1 -- 172.21.15.16:0/738195754 >> 172.21.15.16:0/738195754 conn(0x7f4d900fc7a0 msgr2=0x7f4d900febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:04.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.666+0000 7f4d95bbf640 1 -- 172.21.15.16:0/738195754 shutdown_connections 2024-09-12T02:05:04.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.666+0000 7f4d95bbf640 1 -- 172.21.15.16:0/738195754 wait complete. 2024-09-12T02:05:04.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.667+0000 7f4d95bbf640 1 Processor -- start 2024-09-12T02:05:04.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.667+0000 7f4d95bbf640 1 -- start start 2024-09-12T02:05:04.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.667+0000 7f4d95bbf640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4d90102e00 0x7f4d90100a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:04.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.667+0000 7f4d95bbf640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d90103720 0x7f4d90100fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:04.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.667+0000 7f4d95bbf640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d9010c1a0 0x7f4d901014e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:04.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.667+0000 7f4d95bbf640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4d90111f70 con 0x7f4d90103720 2024-09-12T02:05:04.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.667+0000 7f4d95bbf640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4d90111df0 con 0x7f4d9010c1a0 2024-09-12T02:05:04.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.667+0000 7f4d95bbf640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f4d901120f0 con 0x7f4d90102e00 2024-09-12T02:05:04.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.667+0000 7f4d86ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d90103720 0x7f4d90100fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:04.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.667+0000 7f4d86ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d90103720 0x7f4d90100fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44526/0 (socket says 172.21.15.16:44526) 2024-09-12T02:05:04.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.667+0000 7f4d86ffd640 1 -- 172.21.15.16:0/3744626485 learned_addr learned my addr 172.21.15.16:0/3744626485 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:05:04.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.667+0000 7f4d8ffff640 1 --2- 172.21.15.16:0/3744626485 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d9010c1a0 0x7f4d901014e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:04.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.667+0000 7f4d8f7fe640 1 --2- 172.21.15.16:0/3744626485 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4d90102e00 0x7f4d90100a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:04.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.668+0000 7f4d86ffd640 1 -- 172.21.15.16:0/3744626485 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4d90102e00 msgr2=0x7f4d90100a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:04.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.668+0000 7f4d86ffd640 1 --2- 172.21.15.16:0/3744626485 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4d90102e00 0x7f4d90100a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:04.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.668+0000 7f4d86ffd640 1 -- 172.21.15.16:0/3744626485 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d9010c1a0 msgr2=0x7f4d901014e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:04.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.668+0000 7f4d86ffd640 1 --2- 172.21.15.16:0/3744626485 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d9010c1a0 0x7f4d901014e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:04.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.668+0000 7f4d86ffd640 1 -- 172.21.15.16:0/3744626485 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d901a68d0 con 0x7f4d90103720 2024-09-12T02:05:04.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.668+0000 7f4d8ffff640 1 --2- 172.21.15.16:0/3744626485 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d9010c1a0 0x7f4d901014e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:05:04.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.668+0000 7f4d86ffd640 1 --2- 172.21.15.16:0/3744626485 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d90103720 0x7f4d90100fa0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f4d7800e9f0 tx=0x7f4d7800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:04.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.668+0000 7f4d8d7fa640 1 -- 172.21.15.16:0/3744626485 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d7800cd80 con 0x7f4d90103720 2024-09-12T02:05:04.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.668+0000 7f4d8d7fa640 1 -- 172.21.15.16:0/3744626485 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d78010910 con 0x7f4d90103720 2024-09-12T02:05:04.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.668+0000 7f4d95bbf640 1 -- 172.21.15.16:0/3744626485 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d901a6bc0 con 0x7f4d90103720 2024-09-12T02:05:04.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.668+0000 7f4d8d7fa640 1 -- 172.21.15.16:0/3744626485 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d78018990 con 0x7f4d90103720 2024-09-12T02:05:04.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.668+0000 7f4d95bbf640 1 -- 172.21.15.16:0/3744626485 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d901a7100 con 0x7f4d90103720 2024-09-12T02:05:04.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.671+0000 7f4d8d7fa640 1 -- 172.21.15.16:0/3744626485 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f4d78010ac0 con 0x7f4d90103720 2024-09-12T02:05:04.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.671+0000 7f4d97f98740 1 Processor -- start 2024-09-12T02:05:04.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.671+0000 7f4d97f98740 1 -- start start 2024-09-12T02:05:04.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.671+0000 7f4d97f98740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563c706c74a0 0x563c706c7880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:04.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.671+0000 7f4d97f98740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c706c7dc0 0x563c706d2140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:04.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.671+0000 7f4d97f98740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c706d2680 0x563c706d4a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:04.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.671+0000 7f4d97f98740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x563c7062d2e0 con 0x563c706d2680 2024-09-12T02:05:04.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.671+0000 7f4d97f98740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x563c705d4ce0 con 0x563c706c7dc0 2024-09-12T02:05:04.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.671+0000 7f4d97f98740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x563c706292a0 con 0x563c706c74a0 2024-09-12T02:05:04.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.671+0000 7f4d8ffff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563c706c74a0 0x563c706c7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:04.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.671+0000 7f4d86ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c706d2680 0x563c706d4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:04.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.671+0000 7f4d8ffff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563c706c74a0 0x563c706c7880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:39552/0 (socket says 172.21.15.16:39552) 2024-09-12T02:05:04.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.671+0000 7f4d8ffff640 1 -- 172.21.15.16:0/2662291161 learned_addr learned my addr 172.21.15.16:0/2662291161 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:05:04.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.671+0000 7f4d8f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c706c7dc0 0x563c706d2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:04.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.672+0000 7f4d8ffff640 1 -- 172.21.15.16:0/2662291161 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c706c7dc0 msgr2=0x563c706d2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:04.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.672+0000 7f4d8ffff640 1 --2- 172.21.15.16:0/2662291161 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c706c7dc0 0x563c706d2140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:04.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.672+0000 7f4d8ffff640 1 -- 172.21.15.16:0/2662291161 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c706d2680 msgr2=0x563c706d4a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:04.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.672+0000 7f4d8ffff640 1 --2- 172.21.15.16:0/2662291161 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c706d2680 0x563c706d4a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:04.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.672+0000 7f4d8ffff640 1 -- 172.21.15.16:0/2662291161 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c705b9440 con 0x563c706c74a0 2024-09-12T02:05:04.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.672+0000 7f4d8d7fa640 1 --2- 172.21.15.16:0/3744626485 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f4d7c0a2b80 0x7f4d7c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:04.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.672+0000 7f4d8ffff640 1 --2- 172.21.15.16:0/2662291161 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563c706c74a0 0x563c706c7880 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4d88010710 tx=0x7f4d88010bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1f398cbb79a90fa server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:04.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.672+0000 7f4d8f7fe640 1 --2- 172.21.15.16:0/3744626485 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f4d7c0a2b80 0x7f4d7c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:04.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.672+0000 7f4d8d7fa640 1 -- 172.21.15.16:0/3744626485 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f4d780cf290 con 0x7f4d90103720 2024-09-12T02:05:04.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.672+0000 7f4d85ffb640 1 -- 172.21.15.16:0/2662291161 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d8800e8d0 con 0x563c706c74a0 2024-09-12T02:05:04.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.672+0000 7f4d85ffb640 1 -- 172.21.15.16:0/2662291161 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d88012450 con 0x563c706c74a0 2024-09-12T02:05:04.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.672+0000 7f4d85ffb640 1 -- 172.21.15.16:0/2662291161 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d8801c650 con 0x563c706c74a0 2024-09-12T02:05:04.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.673+0000 7f4d8f7fe640 1 --2- 172.21.15.16:0/3744626485 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f4d7c0a2b80 0x7f4d7c0a5040 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f4d80009230 tx=0x7f4d800058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:04.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.673+0000 7f4d97f98740 1 -- 172.21.15.16:0/2662291161 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563c706c74a0 msgr2=0x563c706c7880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:04.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.673+0000 7f4d97f98740 1 --2- 172.21.15.16:0/2662291161 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563c706c74a0 0x563c706c7880 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4d88010710 tx=0x7f4d88010bd0 comp rx=0 tx=0).stop 2024-09-12T02:05:04.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.673+0000 7f4d97f98740 1 -- 172.21.15.16:0/2662291161 shutdown_connections 2024-09-12T02:05:04.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.673+0000 7f4d97f98740 1 --2- 172.21.15.16:0/2662291161 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c706d2680 0x563c706d4a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:04.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.673+0000 7f4d97f98740 1 --2- 172.21.15.16:0/2662291161 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c706c7dc0 0x563c706d2140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:04.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.673+0000 7f4d97f98740 1 --2- 172.21.15.16:0/2662291161 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563c706c74a0 0x563c706c7880 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:04.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.673+0000 7f4d97f98740 1 -- 172.21.15.16:0/2662291161 >> 172.21.15.16:0/2662291161 conn(0x563c706c3a30 msgr2=0x563c706c3e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:04.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.673+0000 7f4d97f98740 1 -- 172.21.15.16:0/2662291161 shutdown_connections 2024-09-12T02:05:04.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.673+0000 7f4d97f98740 1 -- 172.21.15.16:0/2662291161 wait complete. 2024-09-12T02:05:04.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.673+0000 7f4d97f98740 1 -- start start 2024-09-12T02:05:04.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.674+0000 7f4d97f98740 1 Processor -- start 2024-09-12T02:05:04.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.674+0000 7f4d97f98740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c706c74a0 0x563c706d1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:04.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.674+0000 7f4d97f98740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563c706c7dc0 0x563c706cf250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:04.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.674+0000 7f4d97f98740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c706d2680 0x563c706cf790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:04.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.674+0000 7f4d97f98740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x563c706d5250 con 0x563c706c74a0 2024-09-12T02:05:04.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.674+0000 7f4d97f98740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x563c705e8050 con 0x563c706d2680 2024-09-12T02:05:04.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.674+0000 7f4d97f98740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x563c706d53d0 con 0x563c706c7dc0 2024-09-12T02:05:04.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.674+0000 7f4d97f98740 10 client.0 _ll_register_callbacks cb 0x563c70628160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:05:04.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.674+0000 7f4d8f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c706c74a0 0x563c706d1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:04.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.674+0000 7f4d86ffd640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563c706c7dc0 0x563c706cf250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:04.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.674+0000 7f4d8f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c706c74a0 0x563c706d1fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44540/0 (socket says 172.21.15.16:44540) 2024-09-12T02:05:04.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.674+0000 7f4d8f7fe640 1 -- 172.21.15.16:0/895780316 learned_addr learned my addr 172.21.15.16:0/895780316 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:05:04.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.674+0000 7f4d8ffff640 1 --2- 172.21.15.16:0/895780316 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c706d2680 0x563c706cf790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:04.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.674+0000 7f4d8f7fe640 1 -- 172.21.15.16:0/895780316 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563c706c7dc0 msgr2=0x563c706cf250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:04.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.674+0000 7f4d8f7fe640 1 --2- 172.21.15.16:0/895780316 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563c706c7dc0 0x563c706cf250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:04.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.674+0000 7f4d8f7fe640 1 -- 172.21.15.16:0/895780316 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c706d2680 msgr2=0x563c706cf790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:04.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.674+0000 7f4d8f7fe640 1 --2- 172.21.15.16:0/895780316 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c706d2680 0x563c706cf790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:04.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.674+0000 7f4d8f7fe640 1 -- 172.21.15.16:0/895780316 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c7039e3c0 con 0x563c706c74a0 2024-09-12T02:05:04.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.674+0000 7f4d86ffd640 1 --2- 172.21.15.16:0/895780316 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563c706c7dc0 0x563c706cf250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:05:04.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.675+0000 7f4d8f7fe640 1 --2- 172.21.15.16:0/895780316 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c706c74a0 0x563c706d1fa0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f4d80009900 tx=0x7f4d80009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da4224106f964ea3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:04.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.675+0000 7f4d6effd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:05:04.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.675+0000 7f4d6effd640 1 -- 172.21.15.16:0/895780316 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d8004e070 con 0x563c706c74a0 2024-09-12T02:05:04.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.675+0000 7f4d6effd640 1 -- 172.21.15.16:0/895780316 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f4d80003cc0 con 0x563c706c74a0 2024-09-12T02:05:04.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.675+0000 7f4d97f98740 1 -- 172.21.15.16:0/895780316 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d88012450 con 0x563c706c74a0 2024-09-12T02:05:04.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.675+0000 7f4d6effd640 1 -- 172.21.15.16:0/895780316 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4d8005a050 con 0x563c706c74a0 2024-09-12T02:05:04.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.675+0000 7f4d97f98740 10 client.16371 fetch_fsmap learned FSMap version 25 2024-09-12T02:05:04.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.675+0000 7f4d97f98740 1 -- 172.21.15.16:0/895780316 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x563c706e0c20 con 0x563c706c74a0 2024-09-12T02:05:04.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.675+0000 7f4d6effd640 1 -- 172.21.15.16:0/895780316 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d88012450 con 0x563c706c74a0 2024-09-12T02:05:04.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.675+0000 7f4d6effd640 1 -- 172.21.15.16:0/895780316 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f4d8003d4e0 con 0x563c706c74a0 2024-09-12T02:05:04.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.675+0000 7f4d97f98740 10 client.16371 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:05:04.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.675+0000 7f4d97f98740 10 client.16371 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:05:04.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.675+0000 7f4d97f98740 20 client.16371 populate_metadata read hostname 'smithi016' 2024-09-12T02:05:04.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.675+0000 7f4d97f98740 1 --2- 172.21.15.16:0/895780316 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x563c706e1f70 0x563c706e43d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:04.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.675+0000 7f4d97f98740 4 client.16371 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:05:04.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.675+0000 7f4d97f98740 1 -- 172.21.15.16:0/895780316 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563c7063a850 con 0x563c706e1f70 2024-09-12T02:05:04.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.676+0000 7f4d86ffd640 1 --2- 172.21.15.16:0/895780316 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x563c706e1f70 0x563c706e43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:04.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.676+0000 7f4d86ffd640 1 --2- 172.21.15.16:0/895780316 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x563c706e1f70 0x563c706e43d0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3bd5694900918ac6 server_cookie=2686ec2eaddcbbc2 in_seq=0 out_seq=0 2024-09-12T02:05:04.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.676+0000 7f4d6effd640 10 client.16371 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:05:04.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.677+0000 7f4d6effd640 1 -- 172.21.15.16:0/895780316 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563c7063a850 con 0x563c706e1f70 2024-09-12T02:05:04.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.677+0000 7f4d6effd640 10 client.16371 handle_command_reply: tid=0 2024-09-12T02:05:04.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.677+0000 7f4d97f98740 1 client.16371 shutdown 2024-09-12T02:05:04.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.677+0000 7f4d97f98740 1 -- 172.21.15.16:0/895780316 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c706c74a0 msgr2=0x563c706d1fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:04.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.677+0000 7f4d97f98740 1 --2- 172.21.15.16:0/895780316 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c706c74a0 0x563c706d1fa0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f4d80009900 tx=0x7f4d80009260 comp rx=0 tx=0).stop 2024-09-12T02:05:04.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.677+0000 7f4d97f98740 1 -- 172.21.15.16:0/895780316 shutdown_connections 2024-09-12T02:05:04.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.677+0000 7f4d97f98740 1 --2- 172.21.15.16:0/895780316 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x563c706e1f70 0x563c706e43d0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:04.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.677+0000 7f4d97f98740 1 --2- 172.21.15.16:0/895780316 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c706d2680 0x563c706cf790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:04.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.677+0000 7f4d97f98740 1 --2- 172.21.15.16:0/895780316 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563c706c7dc0 0x563c706cf250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:04.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.677+0000 7f4d97f98740 1 --2- 172.21.15.16:0/895780316 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c706c74a0 0x563c706d1fa0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:04.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.677+0000 7f4d6effd640 0 client.16371 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:05:04.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.677+0000 7f4d97f98740 1 -- 172.21.15.16:0/895780316 >> 172.21.15.16:0/895780316 conn(0x563c706c3a30 msgr2=0x563c706c9710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:04.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.677+0000 7f4d97f98740 1 -- 172.21.15.16:0/895780316 shutdown_connections 2024-09-12T02:05:04.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.677+0000 7f4d97f98740 1 -- 172.21.15.16:0/895780316 wait complete. 2024-09-12T02:05:04.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.677+0000 7f4d97f98740 20 client.16371 trim_cache size 0 max 16384 2024-09-12T02:05:04.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.697+0000 7f4d97f98740 1 Processor -- start 2024-09-12T02:05:04.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.697+0000 7f4d97f98740 1 -- start start 2024-09-12T02:05:04.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.697+0000 7f4d97f98740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563c706c74a0 0x563c706c2a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:04.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.697+0000 7f4d97f98740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c706c7dc0 0x563c706e9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:04.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.697+0000 7f4d97f98740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c706d2680 0x563c706ea230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:04.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.697+0000 7f4d97f98740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4d901b1e00 con 0x563c706d2680 2024-09-12T02:05:04.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.697+0000 7f4d97f98740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4d901af6d0 con 0x563c706c7dc0 2024-09-12T02:05:04.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.697+0000 7f4d97f98740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f4d901ad560 con 0x563c706c74a0 2024-09-12T02:05:04.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.697+0000 7f4d8ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c706d2680 0x563c706ea230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:04.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.697+0000 7f4d86ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c706c7dc0 0x563c706e9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:04.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.697+0000 7f4d86ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c706c7dc0 0x563c706e9cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.16:47916/0 (socket says 172.21.15.16:47916) 2024-09-12T02:05:04.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.697+0000 7f4d86ffd640 1 -- 172.21.15.16:0/1618164898 learned_addr learned my addr 172.21.15.16:0/1618164898 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:05:04.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.697+0000 7f4d8ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c706d2680 0x563c706ea230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44542/0 (socket says 172.21.15.16:44542) 2024-09-12T02:05:04.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.697+0000 7f4d8f7fe640 1 --2- 172.21.15.16:0/1618164898 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563c706c74a0 0x563c706c2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:04.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.697+0000 7f4d8ffff640 1 -- 172.21.15.16:0/1618164898 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563c706c74a0 msgr2=0x563c706c2a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:04.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.697+0000 7f4d8ffff640 1 --2- 172.21.15.16:0/1618164898 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563c706c74a0 0x563c706c2a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:04.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.697+0000 7f4d8ffff640 1 -- 172.21.15.16:0/1618164898 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c706c7dc0 msgr2=0x563c706e9cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:04.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.697+0000 7f4d8ffff640 1 --2- 172.21.15.16:0/1618164898 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c706c7dc0 0x563c706e9cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:04.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.697+0000 7f4d8ffff640 1 -- 172.21.15.16:0/1618164898 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c706c7bf0 con 0x563c706d2680 2024-09-12T02:05:04.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.698+0000 7f4d8ffff640 1 --2- 172.21.15.16:0/1618164898 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c706d2680 0x563c706ea230 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f4d8800e6c0 tx=0x7f4d88012e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9a371ca4c7c4fa5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:04.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.698+0000 7f4d857fa640 1 -- 172.21.15.16:0/1618164898 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d88010dc0 con 0x563c706d2680 2024-09-12T02:05:04.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.698+0000 7f4d857fa640 1 -- 172.21.15.16:0/1618164898 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d8802b070 con 0x563c706d2680 2024-09-12T02:05:04.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.698+0000 7f4d857fa640 1 -- 172.21.15.16:0/1618164898 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d8801c410 con 0x563c706d2680 2024-09-12T02:05:04.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.699+0000 7f4d97f98740 1 -- 172.21.15.16:0/1618164898 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c706d2680 msgr2=0x563c706ea230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:04.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.699+0000 7f4d97f98740 1 --2- 172.21.15.16:0/1618164898 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c706d2680 0x563c706ea230 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f4d8800e6c0 tx=0x7f4d88012e90 comp rx=0 tx=0).stop 2024-09-12T02:05:04.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.699+0000 7f4d97f98740 1 -- 172.21.15.16:0/1618164898 shutdown_connections 2024-09-12T02:05:04.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.699+0000 7f4d97f98740 1 --2- 172.21.15.16:0/1618164898 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c706d2680 0x563c706ea230 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:04.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.699+0000 7f4d97f98740 1 --2- 172.21.15.16:0/1618164898 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c706c7dc0 0x563c706e9cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:04.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.699+0000 7f4d97f98740 1 --2- 172.21.15.16:0/1618164898 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563c706c74a0 0x563c706c2a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:04.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.699+0000 7f4d97f98740 1 -- 172.21.15.16:0/1618164898 >> 172.21.15.16:0/1618164898 conn(0x563c706c3a30 msgr2=0x563c706c2350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:04.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.699+0000 7f4d97f98740 1 -- 172.21.15.16:0/1618164898 shutdown_connections 2024-09-12T02:05:04.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.699+0000 7f4d97f98740 1 -- 172.21.15.16:0/1618164898 wait complete. 2024-09-12T02:05:04.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.699+0000 7f4d97f98740 1 -- start start 2024-09-12T02:05:04.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.699+0000 7f4d97f98740 1 Processor -- start 2024-09-12T02:05:04.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.700+0000 7f4d97f98740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c706c74a0 0x563c706dac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:04.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.700+0000 7f4d97f98740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563c706c7dc0 0x563c706d7f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:04.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.700+0000 7f4d97f98740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c706d2680 0x563c706d84a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:04.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.700+0000 7f4d97f98740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4d901adbb0 con 0x563c706d2680 2024-09-12T02:05:04.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.700+0000 7f4d97f98740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4d901af000 con 0x563c706c74a0 2024-09-12T02:05:04.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.700+0000 7f4d97f98740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f4d901ae260 con 0x563c706c7dc0 2024-09-12T02:05:04.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.700+0000 7f4d97f98740 10 client.0 _ll_register_callbacks cb 0x7f4d80031780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:05:04.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.700+0000 7f4d8f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c706c74a0 0x563c706dac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:04.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.700+0000 7f4d86ffd640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563c706c7dc0 0x563c706d7f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:04.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.700+0000 7f4d8f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c706c74a0 0x563c706dac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.16:47926/0 (socket says 172.21.15.16:47926) 2024-09-12T02:05:04.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.700+0000 7f4d8f7fe640 1 -- 172.21.15.16:0/3273548872 learned_addr learned my addr 172.21.15.16:0/3273548872 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:05:04.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.700+0000 7f4d8ffff640 1 --2- 172.21.15.16:0/3273548872 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c706d2680 0x563c706d84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:04.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.700+0000 7f4d86ffd640 1 -- 172.21.15.16:0/3273548872 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c706c74a0 msgr2=0x563c706dac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:04.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.700+0000 7f4d86ffd640 1 --2- 172.21.15.16:0/3273548872 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c706c74a0 0x563c706dac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:04.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.700+0000 7f4d86ffd640 1 -- 172.21.15.16:0/3273548872 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c706d2680 msgr2=0x563c706d84a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:04.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.700+0000 7f4d86ffd640 1 --2- 172.21.15.16:0/3273548872 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c706d2680 0x563c706d84a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:04.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.700+0000 7f4d86ffd640 1 -- 172.21.15.16:0/3273548872 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c706d8b40 con 0x563c706c7dc0 2024-09-12T02:05:04.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.700+0000 7f4d8ffff640 1 --2- 172.21.15.16:0/3273548872 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c706d2680 0x563c706d84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:05:04.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.700+0000 7f4d86ffd640 1 --2- 172.21.15.16:0/3273548872 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563c706c7dc0 0x563c706d7f60 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4d780980a0 tx=0x7f4d78018420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d123457180cfbcd server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:04.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.700+0000 7f4d6effd640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-09-12T02:05:04.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.701+0000 7f4d6effd640 1 -- 172.21.15.16:0/3273548872 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d7809bc40 con 0x563c706c7dc0 2024-09-12T02:05:04.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.701+0000 7f4d6effd640 1 -- 172.21.15.16:0/3273548872 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f4d78099f40 con 0x563c706c7dc0 2024-09-12T02:05:04.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.701+0000 7f4d97f98740 1 -- 172.21.15.16:0/3273548872 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d8802b070 con 0x563c706c7dc0 2024-09-12T02:05:04.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.701+0000 7f4d6effd640 1 -- 172.21.15.16:0/3273548872 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4d7809ff40 con 0x563c706c7dc0 2024-09-12T02:05:04.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.701+0000 7f4d97f98740 10 client.25816 fetch_fsmap learned FSMap version 25 2024-09-12T02:05:04.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.701+0000 7f4d97f98740 1 -- 172.21.15.16:0/3273548872 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x563c7074cb70 con 0x563c706c7dc0 2024-09-12T02:05:04.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.701+0000 7f4d6effd640 1 -- 172.21.15.16:0/3273548872 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d780a4800 con 0x563c706c7dc0 2024-09-12T02:05:04.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.701+0000 7f4d6effd640 1 -- 172.21.15.16:0/3273548872 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f4d78097990 con 0x563c706c7dc0 2024-09-12T02:05:04.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.701+0000 7f4d97f98740 10 client.25816 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:05:04.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.701+0000 7f4d97f98740 10 client.25816 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:05:04.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.701+0000 7f4d97f98740 20 client.25816 populate_metadata read hostname 'smithi016' 2024-09-12T02:05:04.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.701+0000 7f4d97f98740 1 --2- 172.21.15.16:0/3273548872 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x563c7074d320 0x563c7074f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:04.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.701+0000 7f4d97f98740 4 client.25816 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T02:05:04.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.701+0000 7f4d97f98740 1 -- 172.21.15.16:0/3273548872 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f4d8802b070 con 0x563c7074d320 2024-09-12T02:05:04.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.702+0000 7f4d8f7fe640 1 --2- 172.21.15.16:0/3273548872 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x563c7074d320 0x563c7074f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:04.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.702+0000 7f4d6effd640 10 client.25816 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:05:04.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.702+0000 7f4d8f7fe640 1 --2- 172.21.15.16:0/3273548872 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x563c7074d320 0x563c7074f740 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67bf0a21bf69c4d8 server_cookie=121f1853266fd53b in_seq=0 out_seq=0 2024-09-12T02:05:04.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.702+0000 7f4d6effd640 1 -- 172.21.15.16:0/3273548872 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x7f4d8802b070 con 0x563c7074d320 2024-09-12T02:05:04.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.702+0000 7f4d6effd640 10 client.25816 handle_command_reply: tid=0 2024-09-12T02:05:04.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.702+0000 7f4d97f98740 1 client.25816 shutdown 2024-09-12T02:05:04.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.702+0000 7f4d97f98740 1 -- 172.21.15.16:0/3273548872 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563c706c7dc0 msgr2=0x563c706d7f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:04.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.702+0000 7f4d97f98740 1 --2- 172.21.15.16:0/3273548872 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563c706c7dc0 0x563c706d7f60 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4d780980a0 tx=0x7f4d78018420 comp rx=0 tx=0).stop 2024-09-12T02:05:04.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.702+0000 7f4d97f98740 1 -- 172.21.15.16:0/3273548872 shutdown_connections 2024-09-12T02:05:04.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.702+0000 7f4d97f98740 1 --2- 172.21.15.16:0/3273548872 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x563c7074d320 0x563c7074f740 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:04.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.702+0000 7f4d97f98740 1 --2- 172.21.15.16:0/3273548872 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c706d2680 0x563c706d84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:04.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.702+0000 7f4d97f98740 1 --2- 172.21.15.16:0/3273548872 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563c706c7dc0 0x563c706d7f60 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:04.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.702+0000 7f4d97f98740 1 --2- 172.21.15.16:0/3273548872 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c706c74a0 0x563c706dac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:04.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.702+0000 7f4d97f98740 1 -- 172.21.15.16:0/3273548872 >> 172.21.15.16:0/3273548872 conn(0x563c706c3a30 msgr2=0x563c706c2350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:04.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.702+0000 7f4d6effd640 0 client.25816 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:05:04.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.703+0000 7f4d97f98740 1 -- 172.21.15.16:0/3273548872 shutdown_connections 2024-09-12T02:05:04.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.703+0000 7f4d97f98740 1 -- 172.21.15.16:0/3273548872 wait complete. 2024-09-12T02:05:04.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.703+0000 7f4d97f98740 20 client.25816 trim_cache size 0 max 16384 2024-09-12T02:05:04.708 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T02:05:04.708 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-12T02:05:04.708 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": { 2024-09-12T02:05:04.708 INFO:teuthology.orchestra.run.smithi016.stdout: "974ce6cc-afd3-48ee-83af-ef3568489d00": { 2024-09-12T02:05:04.708 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-12T02:05:04.708 INFO:teuthology.orchestra.run.smithi016.stdout: "tag": "974ce6cc-afd3-48ee-83af-ef3568489d00", 2024-09-12T02:05:04.708 INFO:teuthology.orchestra.run.smithi016.stdout: "options": "recursive,force" 2024-09-12T02:05:04.708 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:05:04.708 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:05:04.708 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T02:05:04.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.703+0000 7f4d95bbf640 1 -- 172.21.15.16:0/3744626485 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f4d7c0a2b80 msgr2=0x7f4d7c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:04.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.703+0000 7f4d95bbf640 1 --2- 172.21.15.16:0/3744626485 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f4d7c0a2b80 0x7f4d7c0a5040 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f4d80009230 tx=0x7f4d800058d0 comp rx=0 tx=0).stop 2024-09-12T02:05:04.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.703+0000 7f4d95bbf640 1 -- 172.21.15.16:0/3744626485 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d90103720 msgr2=0x7f4d90100fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:04.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.703+0000 7f4d95bbf640 1 --2- 172.21.15.16:0/3744626485 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d90103720 0x7f4d90100fa0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f4d7800e9f0 tx=0x7f4d7800eeb0 comp rx=0 tx=0).stop 2024-09-12T02:05:04.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.703+0000 7f4d95bbf640 1 -- 172.21.15.16:0/3744626485 shutdown_connections 2024-09-12T02:05:04.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.703+0000 7f4d95bbf640 1 --2- 172.21.15.16:0/3744626485 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f4d7c0a2b80 0x7f4d7c0a5040 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:04.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.703+0000 7f4d95bbf640 1 --2- 172.21.15.16:0/3744626485 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d9010c1a0 0x7f4d901014e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:04.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.703+0000 7f4d95bbf640 1 --2- 172.21.15.16:0/3744626485 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d90103720 0x7f4d90100fa0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:04.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.703+0000 7f4d95bbf640 1 --2- 172.21.15.16:0/3744626485 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4d90102e00 0x7f4d90100a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:04.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.704+0000 7f4d95bbf640 1 -- 172.21.15.16:0/3744626485 >> 172.21.15.16:0/3744626485 conn(0x7f4d900fc7a0 msgr2=0x7f4d900fd400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:04.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.704+0000 7f4d95bbf640 1 -- 172.21.15.16:0/3744626485 shutdown_connections 2024-09-12T02:05:04.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:04.704+0000 7f4d95bbf640 1 -- 172.21.15.16:0/3744626485 wait complete. 2024-09-12T02:05:05.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:05:04 smithi184 ceph-mon[35593]: scrub summary: idle+waiting paths [/] 2024-09-12T02:05:05.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:05:04 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:05:05.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:05:04 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:05:05.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:05:04 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:05:05.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:05:04 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T02:05:05.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:05:04 smithi033 ceph-mon[35332]: scrub summary: idle+waiting paths [/] 2024-09-12T02:05:05.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:05:04 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:05:05.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:05:04 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:05:05.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:05:04 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:05:05.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:05:04 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T02:05:05.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:05:04 smithi016 ceph-mon[30947]: scrub summary: idle+waiting paths [/] 2024-09-12T02:05:05.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:05:04 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:05:05.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:05:04 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:05:05.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:05:04 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:05:05.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:05:04 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T02:05:05.483 INFO:tasks.cephfs.filesystem:scrub status for tag:974ce6cc-afd3-48ee-83af-ef3568489d00 - {'path': '/', 'tag': '974ce6cc-afd3-48ee-83af-ef3568489d00', 'options': 'recursive,force'} 2024-09-12T02:05:06.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:05:05 smithi184 ceph-mon[35593]: pgmap v430: 97 pgs: 97 active+clean; 1.8 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 14 op/s 2024-09-12T02:05:06.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:05:05 smithi033 ceph-mon[35332]: pgmap v430: 97 pgs: 97 active+clean; 1.8 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 14 op/s 2024-09-12T02:05:06.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:05:05 smithi016 ceph-mon[30947]: pgmap v430: 97 pgs: 97 active+clean; 1.8 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 14 op/s 2024-09-12T02:05:07.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:05:06 smithi184 ceph-mon[35593]: scrub summary: idle 2024-09-12T02:05:07.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:05:06 smithi033 ceph-mon[35332]: scrub summary: idle 2024-09-12T02:05:07.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:05:06 smithi016 ceph-mon[30947]: scrub summary: idle 2024-09-12T02:05:08.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:05:07 smithi184 ceph-mon[35593]: pgmap v431: 97 pgs: 97 active+clean; 1.8 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-09-12T02:05:08.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:05:07 smithi033 ceph-mon[35332]: pgmap v431: 97 pgs: 97 active+clean; 1.8 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-09-12T02:05:08.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:05:07 smithi016 ceph-mon[30947]: pgmap v431: 97 pgs: 97 active+clean; 1.8 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-09-12T02:05:10.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:05:09 smithi184 ceph-mon[35593]: pgmap v432: 97 pgs: 97 active+clean; 1.8 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-12T02:05:10.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:05:09 smithi033 ceph-mon[35332]: pgmap v432: 97 pgs: 97 active+clean; 1.8 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-12T02:05:10.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:05:09 smithi016 ceph-mon[30947]: pgmap v432: 97 pgs: 97 active+clean; 1.8 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-12T02:05:12.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:05:11 smithi184 ceph-mon[35593]: pgmap v433: 97 pgs: 97 active+clean; 1.8 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 17 op/s 2024-09-12T02:05:12.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:05:11 smithi033 ceph-mon[35332]: pgmap v433: 97 pgs: 97 active+clean; 1.8 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 17 op/s 2024-09-12T02:05:12.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:05:11 smithi016 ceph-mon[30947]: pgmap v433: 97 pgs: 97 active+clean; 1.8 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 17 op/s 2024-09-12T02:05:14.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:05:13 smithi184 ceph-mon[35593]: pgmap v434: 97 pgs: 97 active+clean; 1.8 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-09-12T02:05:14.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:05:13 smithi033 ceph-mon[35332]: pgmap v434: 97 pgs: 97 active+clean; 1.8 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-09-12T02:05:14.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:05:13 smithi016 ceph-mon[30947]: pgmap v434: 97 pgs: 97 active+clean; 1.8 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-09-12T02:05:14.828 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.825+0000 7facdff18640 1 -- 172.21.15.16:0/1869391309 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facd8103710 msgr2=0x7facd8109fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:14.828 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.825+0000 7facdff18640 1 --2- 172.21.15.16:0/1869391309 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facd8103710 0x7facd8109fa0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7facd0009900 tx=0x7facd002a410 comp rx=0 tx=0).stop 2024-09-12T02:05:14.828 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.825+0000 7facdff18640 1 -- 172.21.15.16:0/1869391309 shutdown_connections 2024-09-12T02:05:14.828 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.825+0000 7facdff18640 1 --2- 172.21.15.16:0/1869391309 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facd8103710 0x7facd8109fa0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:14.828 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.825+0000 7facdff18640 1 --2- 172.21.15.16:0/1869391309 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7facd8102d50 0x7facd81031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:14.828 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.825+0000 7facdff18640 1 --2- 172.21.15.16:0/1869391309 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7facd8101b60 0x7facd8101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:14.828 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.825+0000 7facdff18640 1 -- 172.21.15.16:0/1869391309 >> 172.21.15.16:0/1869391309 conn(0x7facd80fd2e0 msgr2=0x7facd80ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:14.828 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.825+0000 7facdff18640 1 -- 172.21.15.16:0/1869391309 shutdown_connections 2024-09-12T02:05:14.828 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.825+0000 7facdff18640 1 -- 172.21.15.16:0/1869391309 wait complete. 2024-09-12T02:05:14.828 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.826+0000 7facdff18640 1 Processor -- start 2024-09-12T02:05:14.829 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.826+0000 7facdff18640 1 -- start start 2024-09-12T02:05:14.829 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.826+0000 7facdff18640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7facd8101b60 0x7facd8197da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:14.829 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.826+0000 7facdff18640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facd8102d50 0x7facd81982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:14.829 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.826+0000 7facdff18640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7facd8103710 0x7facd819f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:14.829 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.826+0000 7facdff18640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7facd810d7c0 con 0x7facd8102d50 2024-09-12T02:05:14.829 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.826+0000 7facdff18640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7facd810d640 con 0x7facd8103710 2024-09-12T02:05:14.829 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.826+0000 7facdff18640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7facd810d940 con 0x7facd8101b60 2024-09-12T02:05:14.829 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.826+0000 7facdd4b3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facd8102d50 0x7facd81982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:14.829 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.827+0000 7facdd4b3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facd8102d50 0x7facd81982e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38854/0 (socket says 172.21.15.16:38854) 2024-09-12T02:05:14.829 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.827+0000 7facdd4b3640 1 -- 172.21.15.16:0/2170196333 learned_addr learned my addr 172.21.15.16:0/2170196333 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:05:14.829 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.827+0000 7facddcb4640 1 --2- 172.21.15.16:0/2170196333 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7facd8101b60 0x7facd8197da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:14.830 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.827+0000 7facde4b5640 1 --2- 172.21.15.16:0/2170196333 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7facd8103710 0x7facd819f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:14.830 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.827+0000 7facdd4b3640 1 -- 172.21.15.16:0/2170196333 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7facd8101b60 msgr2=0x7facd8197da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:14.830 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.827+0000 7facdd4b3640 1 --2- 172.21.15.16:0/2170196333 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7facd8101b60 0x7facd8197da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:14.830 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.827+0000 7facdd4b3640 1 -- 172.21.15.16:0/2170196333 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7facd8103710 msgr2=0x7facd819f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:14.830 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.827+0000 7facdd4b3640 1 --2- 172.21.15.16:0/2170196333 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7facd8103710 0x7facd819f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:14.830 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.827+0000 7facdd4b3640 1 -- 172.21.15.16:0/2170196333 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facd819f9b0 con 0x7facd8102d50 2024-09-12T02:05:14.830 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.827+0000 7facdd4b3640 1 --2- 172.21.15.16:0/2170196333 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facd8102d50 0x7facd81982e0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7facc800d950 tx=0x7facc800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:14.830 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.827+0000 7facceffd640 1 -- 172.21.15.16:0/2170196333 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facc8017030 con 0x7facd8102d50 2024-09-12T02:05:14.830 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.827+0000 7facceffd640 1 -- 172.21.15.16:0/2170196333 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7facc8004020 con 0x7facd8102d50 2024-09-12T02:05:14.830 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.827+0000 7facdff18640 1 -- 172.21.15.16:0/2170196333 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facd819fca0 con 0x7facd8102d50 2024-09-12T02:05:14.830 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.827+0000 7facceffd640 1 -- 172.21.15.16:0/2170196333 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facc80047a0 con 0x7facd8102d50 2024-09-12T02:05:14.830 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.827+0000 7facdff18640 1 -- 172.21.15.16:0/2170196333 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facd81a01a0 con 0x7facd8102d50 2024-09-12T02:05:14.837 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.830+0000 7facceffd640 1 -- 172.21.15.16:0/2170196333 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7facc80041d0 con 0x7facd8102d50 2024-09-12T02:05:14.837 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.830+0000 7face2cb1740 1 Processor -- start 2024-09-12T02:05:14.838 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.830+0000 7face2cb1740 1 -- start start 2024-09-12T02:05:14.838 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.830+0000 7face2cb1740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56147ec04db0 0x56147ec051b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:14.838 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.830+0000 7face2cb1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56147ec056f0 0x56147ec0a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:14.838 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.830+0000 7face2cb1740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56147ec0a780 0x56147ec0cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:14.838 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.830+0000 7face2cb1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56147eb642f0 con 0x56147ec056f0 2024-09-12T02:05:14.838 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.831+0000 7face2cb1740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56147eb0c270 con 0x56147ec0a780 2024-09-12T02:05:14.838 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.831+0000 7face2cb1740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x56147eb602b0 con 0x56147ec04db0 2024-09-12T02:05:14.838 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.831+0000 7facceffd640 1 --2- 172.21.15.16:0/2170196333 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7facc40a2c30 0x7facc40a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:14.838 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.831+0000 7facddcb4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56147ec056f0 0x56147ec0a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:14.838 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.831+0000 7facddcb4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56147ec056f0 0x56147ec0a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38856/0 (socket says 172.21.15.16:38856) 2024-09-12T02:05:14.838 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.831+0000 7facddcb4640 1 -- 172.21.15.16:0/3823401598 learned_addr learned my addr 172.21.15.16:0/3823401598 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:05:14.839 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.831+0000 7facdd4b3640 1 --2- 172.21.15.16:0/3823401598 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56147ec0a780 0x56147ec0cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:14.839 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.831+0000 7facde4b5640 1 --2- 172.21.15.16:0/3823401598 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56147ec04db0 0x56147ec051b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:14.839 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.831+0000 7facde4b5640 1 --2- 172.21.15.16:0/2170196333 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7facc40a2c30 0x7facc40a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:14.839 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.831+0000 7facceffd640 1 -- 172.21.15.16:0/2170196333 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7facc80ce960 con 0x7facd8102d50 2024-09-12T02:05:14.839 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.831+0000 7facddcb4640 1 -- 172.21.15.16:0/3823401598 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56147ec04db0 msgr2=0x56147ec051b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:14.839 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.831+0000 7facddcb4640 1 --2- 172.21.15.16:0/3823401598 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56147ec04db0 0x56147ec051b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:14.839 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.831+0000 7facddcb4640 1 -- 172.21.15.16:0/3823401598 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56147ec0a780 msgr2=0x56147ec0cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:14.839 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.831+0000 7facddcb4640 1 --2- 172.21.15.16:0/3823401598 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56147ec0a780 0x56147ec0cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:14.839 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.831+0000 7facddcb4640 1 -- 172.21.15.16:0/3823401598 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56147eb0b340 con 0x56147ec056f0 2024-09-12T02:05:14.839 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.831+0000 7facdd4b3640 1 --2- 172.21.15.16:0/3823401598 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56147ec0a780 0x56147ec0cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:05:14.839 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.831+0000 7facddcb4640 1 --2- 172.21.15.16:0/3823401598 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56147ec056f0 0x56147ec0a1b0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7facd40103a0 tx=0x7facd4010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9aa1205ca6255c0a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:14.839 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.831+0000 7facde4b5640 1 --2- 172.21.15.16:0/2170196333 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7facc40a2c30 0x7facc40a50f0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7facd0006010 tx=0x7facd00315b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:14.840 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.832+0000 7facc3fff640 1 -- 172.21.15.16:0/3823401598 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facd4016070 con 0x56147ec056f0 2024-09-12T02:05:14.840 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.832+0000 7facc3fff640 1 -- 172.21.15.16:0/3823401598 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7facd4004d20 con 0x56147ec056f0 2024-09-12T02:05:14.840 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.832+0000 7facc3fff640 1 -- 172.21.15.16:0/3823401598 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facd4005460 con 0x56147ec056f0 2024-09-12T02:05:14.840 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.832+0000 7face2cb1740 1 -- 172.21.15.16:0/3823401598 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56147ec056f0 msgr2=0x56147ec0a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:14.840 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.832+0000 7face2cb1740 1 --2- 172.21.15.16:0/3823401598 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56147ec056f0 0x56147ec0a1b0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7facd40103a0 tx=0x7facd4010860 comp rx=0 tx=0).stop 2024-09-12T02:05:14.840 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.832+0000 7face2cb1740 1 -- 172.21.15.16:0/3823401598 shutdown_connections 2024-09-12T02:05:14.840 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.832+0000 7face2cb1740 1 --2- 172.21.15.16:0/3823401598 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56147ec0a780 0x56147ec0cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:14.840 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.832+0000 7face2cb1740 1 --2- 172.21.15.16:0/3823401598 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56147ec056f0 0x56147ec0a1b0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:14.840 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.832+0000 7face2cb1740 1 --2- 172.21.15.16:0/3823401598 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56147ec04db0 0x56147ec051b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:14.840 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.832+0000 7face2cb1740 1 -- 172.21.15.16:0/3823401598 >> 172.21.15.16:0/3823401598 conn(0x56147ebfc010 msgr2=0x56147ebfc410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:14.840 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.832+0000 7face2cb1740 1 -- 172.21.15.16:0/3823401598 shutdown_connections 2024-09-12T02:05:14.840 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.833+0000 7face2cb1740 1 -- 172.21.15.16:0/3823401598 wait complete. 2024-09-12T02:05:14.840 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.833+0000 7face2cb1740 1 -- start start 2024-09-12T02:05:14.841 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.833+0000 7face2cb1740 1 Processor -- start 2024-09-12T02:05:14.841 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.833+0000 7face2cb1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56147ec04db0 0x56147ec0f000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:14.841 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.833+0000 7face2cb1740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56147ec056f0 0x56147ec0f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:14.841 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.833+0000 7face2cb1740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56147ec0a780 0x56147ec16580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:14.841 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.833+0000 7face2cb1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56147ec0d750 con 0x56147ec04db0 2024-09-12T02:05:14.841 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.833+0000 7face2cb1740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56147eaed290 con 0x56147ec056f0 2024-09-12T02:05:14.841 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.833+0000 7face2cb1740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x56147ec0d8d0 con 0x56147ec0a780 2024-09-12T02:05:14.841 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.833+0000 7face2cb1740 10 client.0 _ll_register_callbacks cb 0x56147eb1e140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:05:14.841 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.834+0000 7facddcb4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56147ec04db0 0x56147ec0f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:14.841 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.834+0000 7facddcb4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56147ec04db0 0x56147ec0f000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38860/0 (socket says 172.21.15.16:38860) 2024-09-12T02:05:14.842 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.834+0000 7facddcb4640 1 -- 172.21.15.16:0/1053397286 learned_addr learned my addr 172.21.15.16:0/1053397286 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:05:14.842 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.834+0000 7facdd4b3640 1 --2- 172.21.15.16:0/1053397286 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56147ec056f0 0x56147ec0f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:14.842 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.834+0000 7facde4b5640 1 --2- 172.21.15.16:0/1053397286 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56147ec0a780 0x56147ec16580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:14.842 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.834+0000 7facddcb4640 1 -- 172.21.15.16:0/1053397286 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56147ec0a780 msgr2=0x56147ec16580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:14.842 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.834+0000 7facddcb4640 1 --2- 172.21.15.16:0/1053397286 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56147ec0a780 0x56147ec16580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:14.842 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.834+0000 7facddcb4640 1 -- 172.21.15.16:0/1053397286 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56147ec056f0 msgr2=0x56147ec0f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:14.842 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.834+0000 7facddcb4640 1 --2- 172.21.15.16:0/1053397286 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56147ec056f0 0x56147ec0f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:14.842 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.834+0000 7facddcb4640 1 -- 172.21.15.16:0/1053397286 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56147e877210 con 0x56147ec04db0 2024-09-12T02:05:14.842 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.834+0000 7facddcb4640 1 --2- 172.21.15.16:0/1053397286 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56147ec04db0 0x56147ec0f000 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7facd4010370 tx=0x7facd4004c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b591ff4d31db3c57 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:14.842 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.834+0000 7facc17fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:05:14.842 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.834+0000 7facc17fa640 1 -- 172.21.15.16:0/1053397286 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facd4029060 con 0x56147ec04db0 2024-09-12T02:05:14.843 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.834+0000 7facc17fa640 1 -- 172.21.15.16:0/1053397286 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7facd4012720 con 0x56147ec04db0 2024-09-12T02:05:14.843 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.834+0000 7face2cb1740 1 -- 172.21.15.16:0/1053397286 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7facd4004d20 con 0x56147ec04db0 2024-09-12T02:05:14.843 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.835+0000 7facc17fa640 1 -- 172.21.15.16:0/1053397286 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7facd4016040 con 0x56147ec04db0 2024-09-12T02:05:14.843 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.835+0000 7face2cb1740 10 client.16389 fetch_fsmap learned FSMap version 25 2024-09-12T02:05:14.843 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.835+0000 7face2cb1740 1 -- 172.21.15.16:0/1053397286 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56147e943620 con 0x56147ec04db0 2024-09-12T02:05:14.843 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.835+0000 7facc17fa640 1 -- 172.21.15.16:0/1053397286 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7facd4004d20 con 0x56147ec04db0 2024-09-12T02:05:14.843 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.835+0000 7facc17fa640 1 -- 172.21.15.16:0/1053397286 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7facd402e070 con 0x56147ec04db0 2024-09-12T02:05:14.843 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.835+0000 7face2cb1740 10 client.16389 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:05:14.843 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.835+0000 7face2cb1740 10 client.16389 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:05:14.843 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.835+0000 7face2cb1740 20 client.16389 populate_metadata read hostname 'smithi016' 2024-09-12T02:05:14.843 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.835+0000 7face2cb1740 1 --2- 172.21.15.16:0/1053397286 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x56147ec1a990 0x56147ec1cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:14.843 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.835+0000 7face2cb1740 4 client.16389 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:05:14.844 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.835+0000 7face2cb1740 1 -- 172.21.15.16:0/1053397286 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56147eb71a20 con 0x56147ec1a990 2024-09-12T02:05:14.844 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.835+0000 7facdd4b3640 1 --2- 172.21.15.16:0/1053397286 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x56147ec1a990 0x56147ec1cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:14.844 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.836+0000 7facdd4b3640 1 --2- 172.21.15.16:0/1053397286 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x56147ec1a990 0x56147ec1cd80 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9def17460007f9c7 server_cookie=c2158bed20d81114 in_seq=0 out_seq=0 2024-09-12T02:05:14.844 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.836+0000 7facc17fa640 10 client.16389 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:05:14.844 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.836+0000 7facc17fa640 1 -- 172.21.15.16:0/1053397286 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56147eb71a20 con 0x56147ec1a990 2024-09-12T02:05:14.844 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.836+0000 7facc17fa640 10 client.16389 handle_command_reply: tid=0 2024-09-12T02:05:14.844 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.837+0000 7face2cb1740 1 client.16389 shutdown 2024-09-12T02:05:14.844 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.837+0000 7face2cb1740 1 -- 172.21.15.16:0/1053397286 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56147ec04db0 msgr2=0x56147ec0f000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:14.844 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.837+0000 7face2cb1740 1 --2- 172.21.15.16:0/1053397286 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56147ec04db0 0x56147ec0f000 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7facd4010370 tx=0x7facd4004c50 comp rx=0 tx=0).stop 2024-09-12T02:05:14.845 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.837+0000 7face2cb1740 1 -- 172.21.15.16:0/1053397286 shutdown_connections 2024-09-12T02:05:14.845 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.837+0000 7face2cb1740 1 --2- 172.21.15.16:0/1053397286 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x56147ec1a990 0x56147ec1cd80 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:14.845 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.837+0000 7face2cb1740 1 --2- 172.21.15.16:0/1053397286 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56147ec0a780 0x56147ec16580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:14.845 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.837+0000 7face2cb1740 1 --2- 172.21.15.16:0/1053397286 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56147ec056f0 0x56147ec0f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:14.845 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.837+0000 7face2cb1740 1 --2- 172.21.15.16:0/1053397286 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56147ec04db0 0x56147ec0f000 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:14.845 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.837+0000 7facc17fa640 0 client.16389 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:05:14.845 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.837+0000 7face2cb1740 1 -- 172.21.15.16:0/1053397286 >> 172.21.15.16:0/1053397286 conn(0x56147ebfc010 msgr2=0x56147ebfd820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:14.845 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.837+0000 7face2cb1740 1 -- 172.21.15.16:0/1053397286 shutdown_connections 2024-09-12T02:05:14.845 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.837+0000 7face2cb1740 1 -- 172.21.15.16:0/1053397286 wait complete. 2024-09-12T02:05:14.845 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.837+0000 7face2cb1740 20 client.16389 trim_cache size 0 max 16384 2024-09-12T02:05:14.857 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.852+0000 7face2cb1740 1 Processor -- start 2024-09-12T02:05:14.857 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.852+0000 7face2cb1740 1 -- start start 2024-09-12T02:05:14.857 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.852+0000 7face2cb1740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56147ec04db0 0x56147ec0f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:14.857 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.852+0000 7face2cb1740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56147ec056f0 0x56147ec16540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:14.857 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.852+0000 7face2cb1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56147ec0a780 0x56147ebfcfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:14.857 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.852+0000 7face2cb1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7facd81a95b0 con 0x56147ec0a780 2024-09-12T02:05:14.858 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.852+0000 7face2cb1740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7facd81a6e80 con 0x56147ec056f0 2024-09-12T02:05:14.858 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.852+0000 7face2cb1740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7facd81a4cd0 con 0x56147ec04db0 2024-09-12T02:05:14.858 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.852+0000 7facde4b5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56147ec0a780 0x56147ebfcfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:14.858 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.852+0000 7facde4b5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56147ec0a780 0x56147ebfcfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38864/0 (socket says 172.21.15.16:38864) 2024-09-12T02:05:14.858 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.852+0000 7facde4b5640 1 -- 172.21.15.16:0/2477705718 learned_addr learned my addr 172.21.15.16:0/2477705718 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:05:14.858 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.852+0000 7facddcb4640 1 --2- 172.21.15.16:0/2477705718 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56147ec04db0 0x56147ec0f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:14.858 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.852+0000 7facde4b5640 1 -- 172.21.15.16:0/2477705718 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56147ec04db0 msgr2=0x56147ec0f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:14.858 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.852+0000 7facde4b5640 1 --2- 172.21.15.16:0/2477705718 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56147ec04db0 0x56147ec0f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:14.858 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.852+0000 7facde4b5640 1 -- 172.21.15.16:0/2477705718 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56147ec056f0 msgr2=0x56147ec16540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:14.858 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.852+0000 7facdd4b3640 1 --2- 172.21.15.16:0/2477705718 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56147ec056f0 0x56147ec16540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:14.858 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.852+0000 7facde4b5640 1 --2- 172.21.15.16:0/2477705718 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56147ec056f0 0x56147ec16540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:14.859 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.852+0000 7facde4b5640 1 -- 172.21.15.16:0/2477705718 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56147ec1d0f0 con 0x56147ec0a780 2024-09-12T02:05:14.859 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.852+0000 7facddcb4640 1 --2- 172.21.15.16:0/2477705718 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56147ec04db0 0x56147ec0f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:05:14.859 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.853+0000 7facde4b5640 1 --2- 172.21.15.16:0/2477705718 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56147ec0a780 0x56147ebfcfa0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7facd0009230 tx=0x7facd00046e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ce6d4c7999c4986 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:14.859 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.853+0000 7facc37fe640 1 -- 172.21.15.16:0/2477705718 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facd0035070 con 0x56147ec0a780 2024-09-12T02:05:14.859 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.853+0000 7facc37fe640 1 -- 172.21.15.16:0/2477705718 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7facd0051070 con 0x56147ec0a780 2024-09-12T02:05:14.859 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.853+0000 7facc37fe640 1 -- 172.21.15.16:0/2477705718 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facd003e880 con 0x56147ec0a780 2024-09-12T02:05:14.859 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.853+0000 7face2cb1740 1 -- 172.21.15.16:0/2477705718 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56147ec0a780 msgr2=0x56147ebfcfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:14.859 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.853+0000 7face2cb1740 1 --2- 172.21.15.16:0/2477705718 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56147ec0a780 0x56147ebfcfa0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7facd0009230 tx=0x7facd00046e0 comp rx=0 tx=0).stop 2024-09-12T02:05:14.859 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.853+0000 7face2cb1740 1 -- 172.21.15.16:0/2477705718 shutdown_connections 2024-09-12T02:05:14.859 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.853+0000 7face2cb1740 1 --2- 172.21.15.16:0/2477705718 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56147ec0a780 0x56147ebfcfa0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:14.860 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.853+0000 7face2cb1740 1 --2- 172.21.15.16:0/2477705718 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56147ec056f0 0x56147ec16540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:14.860 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.853+0000 7face2cb1740 1 --2- 172.21.15.16:0/2477705718 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56147ec04db0 0x56147ec0f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:14.860 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.853+0000 7face2cb1740 1 -- 172.21.15.16:0/2477705718 >> 172.21.15.16:0/2477705718 conn(0x56147ebfc010 msgr2=0x56147ec0edb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:14.860 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.853+0000 7face2cb1740 1 -- 172.21.15.16:0/2477705718 shutdown_connections 2024-09-12T02:05:14.860 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.853+0000 7face2cb1740 1 -- 172.21.15.16:0/2477705718 wait complete. 2024-09-12T02:05:14.860 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.854+0000 7face2cb1740 1 -- start start 2024-09-12T02:05:14.860 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.854+0000 7face2cb1740 1 Processor -- start 2024-09-12T02:05:14.860 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.854+0000 7face2cb1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56147ec04db0 0x56147ec13b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:14.860 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.854+0000 7face2cb1740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56147ec056f0 0x56147ec140b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:14.860 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.854+0000 7face2cb1740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56147ec0a780 0x56147ec15020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:14.860 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.854+0000 7face2cb1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7facd81a5340 con 0x56147ec04db0 2024-09-12T02:05:14.860 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.854+0000 7face2cb1740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7facd81a67b0 con 0x56147ec0a780 2024-09-12T02:05:14.861 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.854+0000 7face2cb1740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7facd81a5a10 con 0x56147ec056f0 2024-09-12T02:05:14.861 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.854+0000 7face2cb1740 10 client.0 _ll_register_callbacks cb 0x7facc8004540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:05:14.861 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.854+0000 7facddcb4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56147ec04db0 0x56147ec13b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:14.861 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.854+0000 7facddcb4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56147ec04db0 0x56147ec13b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38878/0 (socket says 172.21.15.16:38878) 2024-09-12T02:05:14.861 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.854+0000 7facddcb4640 1 -- 172.21.15.16:0/663623054 learned_addr learned my addr 172.21.15.16:0/663623054 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:05:14.861 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.854+0000 7facdd4b3640 1 --2- 172.21.15.16:0/663623054 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56147ec056f0 0x56147ec140b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:14.861 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.855+0000 7facde4b5640 1 --2- 172.21.15.16:0/663623054 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56147ec0a780 0x56147ec15020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:14.861 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.855+0000 7facddcb4640 1 -- 172.21.15.16:0/663623054 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56147ec056f0 msgr2=0x56147ec140b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:14.861 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.855+0000 7facddcb4640 1 --2- 172.21.15.16:0/663623054 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56147ec056f0 0x56147ec140b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:14.861 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.855+0000 7facddcb4640 1 -- 172.21.15.16:0/663623054 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56147ec0a780 msgr2=0x56147ec15020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:14.861 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.855+0000 7facddcb4640 1 --2- 172.21.15.16:0/663623054 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56147ec0a780 0x56147ec15020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:14.862 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.855+0000 7facddcb4640 1 -- 172.21.15.16:0/663623054 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56147ec15630 con 0x56147ec04db0 2024-09-12T02:05:14.862 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.855+0000 7facde4b5640 1 --2- 172.21.15.16:0/663623054 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56147ec0a780 0x56147ec15020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:05:14.862 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.855+0000 7facddcb4640 1 --2- 172.21.15.16:0/663623054 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56147ec04db0 0x56147ec13b70 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7facd4012db0 tx=0x7facd4005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88ffad436f93f815 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:14.862 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.855+0000 7facc17fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:05:14.862 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.855+0000 7facc17fa640 1 -- 172.21.15.16:0/663623054 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facd4034030 con 0x56147ec04db0 2024-09-12T02:05:14.862 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.855+0000 7facc17fa640 1 -- 172.21.15.16:0/663623054 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7facd4002030 con 0x56147ec04db0 2024-09-12T02:05:14.862 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.855+0000 7face2cb1740 1 -- 172.21.15.16:0/663623054 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7facd0051070 con 0x56147ec04db0 2024-09-12T02:05:14.862 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.855+0000 7facc17fa640 1 -- 172.21.15.16:0/663623054 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7facd4009c20 con 0x56147ec04db0 2024-09-12T02:05:14.862 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.855+0000 7face2cb1740 10 client.16395 fetch_fsmap learned FSMap version 25 2024-09-12T02:05:14.862 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.855+0000 7face2cb1740 1 -- 172.21.15.16:0/663623054 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56147ec168b0 con 0x56147ec04db0 2024-09-12T02:05:14.862 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.855+0000 7facc17fa640 1 -- 172.21.15.16:0/663623054 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7facd0051070 con 0x56147ec04db0 2024-09-12T02:05:14.862 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.856+0000 7facc17fa640 1 -- 172.21.15.16:0/663623054 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7facd402e040 con 0x56147ec04db0 2024-09-12T02:05:14.863 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.856+0000 7face2cb1740 10 client.16395 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:05:14.863 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.856+0000 7face2cb1740 10 client.16395 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:05:14.863 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.856+0000 7face2cb1740 20 client.16395 populate_metadata read hostname 'smithi016' 2024-09-12T02:05:14.863 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.856+0000 7face2cb1740 1 --2- 172.21.15.16:0/663623054 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x56147ec85cf0 0x56147ec88110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:14.863 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.856+0000 7face2cb1740 4 client.16395 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:05:14.863 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.856+0000 7face2cb1740 1 -- 172.21.15.16:0/663623054 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56147eb6bd70 con 0x56147ec85cf0 2024-09-12T02:05:14.863 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.856+0000 7facdd4b3640 1 --2- 172.21.15.16:0/663623054 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x56147ec85cf0 0x56147ec88110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:14.863 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.856+0000 7facc17fa640 10 client.16395 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:05:14.863 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.856+0000 7facdd4b3640 1 --2- 172.21.15.16:0/663623054 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x56147ec85cf0 0x56147ec88110 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c016e24b3c7d89c server_cookie=5aeee1fa06b1a423 in_seq=0 out_seq=0 2024-09-12T02:05:14.868 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.867+0000 7facc17fa640 1 -- 172.21.15.16:0/663623054 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56147eb6bd70 con 0x56147ec85cf0 2024-09-12T02:05:14.868 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.867+0000 7facc17fa640 10 client.16395 handle_command_reply: tid=0 2024-09-12T02:05:14.868 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.867+0000 7face2cb1740 1 client.16395 shutdown 2024-09-12T02:05:14.868 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.867+0000 7face2cb1740 1 -- 172.21.15.16:0/663623054 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56147ec04db0 msgr2=0x56147ec13b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:14.868 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.867+0000 7face2cb1740 1 --2- 172.21.15.16:0/663623054 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56147ec04db0 0x56147ec13b70 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7facd4012db0 tx=0x7facd4005bd0 comp rx=0 tx=0).stop 2024-09-12T02:05:14.868 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.868+0000 7face2cb1740 1 -- 172.21.15.16:0/663623054 shutdown_connections 2024-09-12T02:05:14.868 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.868+0000 7face2cb1740 1 --2- 172.21.15.16:0/663623054 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x56147ec85cf0 0x56147ec88110 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:14.868 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.868+0000 7face2cb1740 1 --2- 172.21.15.16:0/663623054 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56147ec0a780 0x56147ec15020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:14.868 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.868+0000 7face2cb1740 1 --2- 172.21.15.16:0/663623054 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56147ec056f0 0x56147ec140b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:14.868 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.868+0000 7face2cb1740 1 --2- 172.21.15.16:0/663623054 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56147ec04db0 0x56147ec13b70 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:14.869 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.868+0000 7facc17fa640 0 client.16395 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:05:14.869 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.868+0000 7face2cb1740 1 -- 172.21.15.16:0/663623054 >> 172.21.15.16:0/663623054 conn(0x56147ebfc010 msgr2=0x56147ebfcfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:14.869 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.868+0000 7face2cb1740 1 -- 172.21.15.16:0/663623054 shutdown_connections 2024-09-12T02:05:14.869 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.868+0000 7face2cb1740 1 -- 172.21.15.16:0/663623054 wait complete. 2024-09-12T02:05:14.869 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.868+0000 7face2cb1740 20 client.16395 trim_cache size 0 max 16384 2024-09-12T02:05:14.869 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-09-12T02:05:14.869 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-09-12T02:05:14.869 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-09-12T02:05:14.869 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-09-12T02:05:14.870 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.868+0000 7facdff18640 1 -- 172.21.15.16:0/2170196333 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7facc40a2c30 msgr2=0x7facc40a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:14.870 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.868+0000 7facdff18640 1 --2- 172.21.15.16:0/2170196333 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7facc40a2c30 0x7facc40a50f0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7facd0006010 tx=0x7facd00315b0 comp rx=0 tx=0).stop 2024-09-12T02:05:14.870 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.868+0000 7facdff18640 1 -- 172.21.15.16:0/2170196333 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facd8102d50 msgr2=0x7facd81982e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:14.870 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.868+0000 7facdff18640 1 --2- 172.21.15.16:0/2170196333 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facd8102d50 0x7facd81982e0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7facc800d950 tx=0x7facc800de10 comp rx=0 tx=0).stop 2024-09-12T02:05:14.871 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.868+0000 7facdff18640 1 -- 172.21.15.16:0/2170196333 shutdown_connections 2024-09-12T02:05:14.871 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.868+0000 7facdff18640 1 --2- 172.21.15.16:0/2170196333 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7facc40a2c30 0x7facc40a50f0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:14.871 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.868+0000 7facdff18640 1 --2- 172.21.15.16:0/2170196333 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7facd8103710 0x7facd819f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:14.871 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.868+0000 7facdff18640 1 --2- 172.21.15.16:0/2170196333 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facd8102d50 0x7facd81982e0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:14.871 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.868+0000 7facdff18640 1 --2- 172.21.15.16:0/2170196333 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7facd8101b60 0x7facd8197da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:14.871 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.868+0000 7facdff18640 1 -- 172.21.15.16:0/2170196333 >> 172.21.15.16:0/2170196333 conn(0x7facd80fd2e0 msgr2=0x7facd81060d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:14.871 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.869+0000 7facdff18640 1 -- 172.21.15.16:0/2170196333 shutdown_connections 2024-09-12T02:05:14.871 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:14.869+0000 7facdff18640 1 -- 172.21.15.16:0/2170196333 wait complete. 2024-09-12T02:05:16.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:05:15 smithi184 ceph-mon[35593]: pgmap v435: 97 pgs: 97 active+clean; 1.9 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-09-12T02:05:16.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:05:15 smithi033 ceph-mon[35332]: pgmap v435: 97 pgs: 97 active+clean; 1.9 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-09-12T02:05:16.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:05:15 smithi016 ceph-mon[30947]: pgmap v435: 97 pgs: 97 active+clean; 1.9 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-09-12T02:05:18.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:05:17 smithi184 ceph-mon[35593]: pgmap v436: 97 pgs: 97 active+clean; 1.9 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-12T02:05:18.418 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:05:17 smithi033 ceph-mon[35332]: pgmap v436: 97 pgs: 97 active+clean; 1.9 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-12T02:05:18.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:05:17 smithi016 ceph-mon[30947]: pgmap v436: 97 pgs: 97 active+clean; 1.9 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-12T02:05:20.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:05:19 smithi184 ceph-mon[35593]: pgmap v437: 97 pgs: 97 active+clean; 1.9 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-12T02:05:20.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:05:19 smithi033 ceph-mon[35332]: pgmap v437: 97 pgs: 97 active+clean; 1.9 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-12T02:05:20.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:05:19 smithi016 ceph-mon[30947]: pgmap v437: 97 pgs: 97 active+clean; 1.9 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-12T02:05:22.221 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:05:21 smithi033 ceph-mon[35332]: pgmap v438: 97 pgs: 97 active+clean; 1.9 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 15 op/s 2024-09-12T02:05:22.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:05:21 smithi184 ceph-mon[35593]: pgmap v438: 97 pgs: 97 active+clean; 1.9 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 15 op/s 2024-09-12T02:05:22.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:05:21 smithi016 ceph-mon[30947]: pgmap v438: 97 pgs: 97 active+clean; 1.9 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 15 op/s 2024-09-12T02:05:24.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:05:23 smithi184 ceph-mon[35593]: pgmap v439: 97 pgs: 97 active+clean; 1.9 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-09-12T02:05:24.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:05:23 smithi033 ceph-mon[35332]: pgmap v439: 97 pgs: 97 active+clean; 1.9 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-09-12T02:05:24.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:05:23 smithi016 ceph-mon[30947]: pgmap v439: 97 pgs: 97 active+clean; 1.9 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-09-12T02:05:26.236 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:05:25 smithi016 ceph-mon[30947]: pgmap v440: 97 pgs: 97 active+clean; 2.0 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-09-12T02:05:26.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:05:25 smithi184 ceph-mon[35593]: pgmap v440: 97 pgs: 97 active+clean; 2.0 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-09-12T02:05:26.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:05:25 smithi033 ceph-mon[35332]: pgmap v440: 97 pgs: 97 active+clean; 2.0 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-09-12T02:05:27.962 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.959+0000 7f72b8781640 1 -- 172.21.15.16:0/4091071399 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72b0102d50 msgr2=0x7f72b01031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:27.962 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.959+0000 7f72b8781640 1 --2- 172.21.15.16:0/4091071399 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72b0102d50 0x7f72b01031d0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f72a0009900 tx=0x7f72a002a410 comp rx=0 tx=0).stop 2024-09-12T02:05:27.962 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.959+0000 7f72b8781640 1 -- 172.21.15.16:0/4091071399 shutdown_connections 2024-09-12T02:05:27.962 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.959+0000 7f72b8781640 1 --2- 172.21.15.16:0/4091071399 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f72b0103710 0x7f72b0109fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:27.962 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.959+0000 7f72b8781640 1 --2- 172.21.15.16:0/4091071399 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72b0102d50 0x7f72b01031d0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:27.962 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.959+0000 7f72b8781640 1 --2- 172.21.15.16:0/4091071399 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f72b0101b60 0x7f72b0101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:27.962 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.959+0000 7f72b8781640 1 -- 172.21.15.16:0/4091071399 >> 172.21.15.16:0/4091071399 conn(0x7f72b00fd2e0 msgr2=0x7f72b00ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:27.962 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.959+0000 7f72b8781640 1 -- 172.21.15.16:0/4091071399 shutdown_connections 2024-09-12T02:05:27.962 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.959+0000 7f72b8781640 1 -- 172.21.15.16:0/4091071399 wait complete. 2024-09-12T02:05:27.962 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.960+0000 7f72b8781640 1 Processor -- start 2024-09-12T02:05:27.963 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.960+0000 7f72b8781640 1 -- start start 2024-09-12T02:05:27.963 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.960+0000 7f72b8781640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72b0101b60 0x7f72b0075fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:27.963 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.960+0000 7f72b8781640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f72b0102d50 0x7f72b00745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:27.963 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.960+0000 7f72b8781640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f72b0103710 0x7f72b0074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:27.963 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.960+0000 7f72b8781640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f72b010d7c0 con 0x7f72b0101b60 2024-09-12T02:05:27.963 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.960+0000 7f72b8781640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f72b010d640 con 0x7f72b0102d50 2024-09-12T02:05:27.963 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.960+0000 7f72b8781640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f72b010d940 con 0x7f72b0103710 2024-09-12T02:05:27.963 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.960+0000 7f72b651d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72b0101b60 0x7f72b0075fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:27.963 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.960+0000 7f72b651d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72b0101b60 0x7f72b0075fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53818/0 (socket says 172.21.15.16:53818) 2024-09-12T02:05:27.963 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.960+0000 7f72b651d640 1 -- 172.21.15.16:0/2213300593 learned_addr learned my addr 172.21.15.16:0/2213300593 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:05:27.963 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.960+0000 7f72b5d1c640 1 --2- 172.21.15.16:0/2213300593 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f72b0102d50 0x7f72b00745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:27.963 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.961+0000 7f72b651d640 1 -- 172.21.15.16:0/2213300593 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f72b0103710 msgr2=0x7f72b0074b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:27.963 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.961+0000 7f72b6d1e640 1 --2- 172.21.15.16:0/2213300593 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f72b0103710 0x7f72b0074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:27.964 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.961+0000 7f72b651d640 1 --2- 172.21.15.16:0/2213300593 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f72b0103710 0x7f72b0074b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:27.964 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.961+0000 7f72b651d640 1 -- 172.21.15.16:0/2213300593 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f72b0102d50 msgr2=0x7f72b00745e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:27.964 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.961+0000 7f72b651d640 1 --2- 172.21.15.16:0/2213300593 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f72b0102d50 0x7f72b00745e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:27.964 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.961+0000 7f72b651d640 1 -- 172.21.15.16:0/2213300593 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72b00753b0 con 0x7f72b0101b60 2024-09-12T02:05:27.964 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.961+0000 7f72b5d1c640 1 --2- 172.21.15.16:0/2213300593 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f72b0102d50 0x7f72b00745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:05:27.964 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.961+0000 7f72b651d640 1 --2- 172.21.15.16:0/2213300593 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72b0101b60 0x7f72b0075fa0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f72ac00c970 tx=0x7f72ac00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:27.964 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.961+0000 7f72a77fe640 1 -- 172.21.15.16:0/2213300593 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72ac007c10 con 0x7f72b0101b60 2024-09-12T02:05:27.964 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.961+0000 7f72a77fe640 1 -- 172.21.15.16:0/2213300593 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72ac007db0 con 0x7f72b0101b60 2024-09-12T02:05:27.964 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.961+0000 7f72b8781640 1 -- 172.21.15.16:0/2213300593 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72b01a1e80 con 0x7f72b0101b60 2024-09-12T02:05:27.964 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.961+0000 7f72a77fe640 1 -- 172.21.15.16:0/2213300593 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72ac00f940 con 0x7f72b0101b60 2024-09-12T02:05:27.964 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.961+0000 7f72b8781640 1 -- 172.21.15.16:0/2213300593 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72b01a22e0 con 0x7f72b0101b60 2024-09-12T02:05:27.970 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.963+0000 7f72a77fe640 1 -- 172.21.15.16:0/2213300593 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f72ac00fb00 con 0x7f72b0101b60 2024-09-12T02:05:27.971 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.964+0000 7f72a77fe640 1 --2- 172.21.15.16:0/2213300593 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f729c0a2cb0 0x7f729c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:27.971 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.964+0000 7f72b5d1c640 1 --2- 172.21.15.16:0/2213300593 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f729c0a2cb0 0x7f729c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:27.971 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.964+0000 7f72a77fe640 1 -- 172.21.15.16:0/2213300593 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f72ac0cf450 con 0x7f72b0101b60 2024-09-12T02:05:27.971 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.964+0000 7f72bb5f4740 1 Processor -- start 2024-09-12T02:05:27.971 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.964+0000 7f72bb5f4740 1 -- start start 2024-09-12T02:05:27.971 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.965+0000 7f72b5d1c640 1 --2- 172.21.15.16:0/2213300593 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f729c0a2cb0 0x7f729c0a5170 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f72a0000c00 tx=0x7f72a00095f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:27.971 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.965+0000 7f72bb5f4740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561fdcc909a0 0x561fdcc90da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:27.971 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.965+0000 7f72bb5f4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561fdcc912e0 0x561fdcc95da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:27.971 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.965+0000 7f72bb5f4740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561fdcc96370 0x561fdcc98830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:27.971 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.965+0000 7f72bb5f4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x561fdcbf42f0 con 0x561fdcc912e0 2024-09-12T02:05:27.971 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.965+0000 7f72bb5f4740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x561fdcb9c270 con 0x561fdcc96370 2024-09-12T02:05:27.971 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.965+0000 7f72bb5f4740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x561fdcbf02b0 con 0x561fdcc909a0 2024-09-12T02:05:27.972 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.965+0000 7f72b5d1c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561fdcc912e0 0x561fdcc95da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:27.972 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.965+0000 7f72b5d1c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561fdcc912e0 0x561fdcc95da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53822/0 (socket says 172.21.15.16:53822) 2024-09-12T02:05:27.972 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.965+0000 7f72b5d1c640 1 -- 172.21.15.16:0/552661346 learned_addr learned my addr 172.21.15.16:0/552661346 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:05:27.972 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.965+0000 7f72b6d1e640 1 --2- 172.21.15.16:0/552661346 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561fdcc96370 0x561fdcc98830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:27.972 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.965+0000 7f72b651d640 1 --2- 172.21.15.16:0/552661346 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561fdcc909a0 0x561fdcc90da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:27.972 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.965+0000 7f72b5d1c640 1 -- 172.21.15.16:0/552661346 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561fdcc909a0 msgr2=0x561fdcc90da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:27.972 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.965+0000 7f72b5d1c640 1 --2- 172.21.15.16:0/552661346 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561fdcc909a0 0x561fdcc90da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:27.972 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.965+0000 7f72b5d1c640 1 -- 172.21.15.16:0/552661346 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561fdcc96370 msgr2=0x561fdcc98830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:27.972 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.965+0000 7f72b5d1c640 1 --2- 172.21.15.16:0/552661346 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561fdcc96370 0x561fdcc98830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:27.972 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.965+0000 7f72b5d1c640 1 -- 172.21.15.16:0/552661346 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561fdcb9b340 con 0x561fdcc912e0 2024-09-12T02:05:27.972 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.965+0000 7f72b651d640 1 --2- 172.21.15.16:0/552661346 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561fdcc909a0 0x561fdcc90da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:05:27.972 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.965+0000 7f72b5d1c640 1 --2- 172.21.15.16:0/552661346 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561fdcc912e0 0x561fdcc95da0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f72a0031610 tx=0x7f72a003c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fd16027d6bbac72 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:27.972 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.965+0000 7f72a4ff9640 1 -- 172.21.15.16:0/552661346 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72a0035050 con 0x561fdcc912e0 2024-09-12T02:05:27.973 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.966+0000 7f72a4ff9640 1 -- 172.21.15.16:0/552661346 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72a004c070 con 0x561fdcc912e0 2024-09-12T02:05:27.973 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.966+0000 7f72a4ff9640 1 -- 172.21.15.16:0/552661346 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72a00049b0 con 0x561fdcc912e0 2024-09-12T02:05:27.973 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.966+0000 7f72bb5f4740 1 -- 172.21.15.16:0/552661346 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561fdcc912e0 msgr2=0x561fdcc95da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:27.973 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.966+0000 7f72bb5f4740 1 --2- 172.21.15.16:0/552661346 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561fdcc912e0 0x561fdcc95da0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f72a0031610 tx=0x7f72a003c000 comp rx=0 tx=0).stop 2024-09-12T02:05:27.973 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.966+0000 7f72bb5f4740 1 -- 172.21.15.16:0/552661346 shutdown_connections 2024-09-12T02:05:27.973 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.966+0000 7f72bb5f4740 1 --2- 172.21.15.16:0/552661346 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561fdcc96370 0x561fdcc98830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:27.973 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.966+0000 7f72bb5f4740 1 --2- 172.21.15.16:0/552661346 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561fdcc912e0 0x561fdcc95da0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:27.973 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.966+0000 7f72bb5f4740 1 --2- 172.21.15.16:0/552661346 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561fdcc909a0 0x561fdcc90da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:27.973 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.966+0000 7f72bb5f4740 1 -- 172.21.15.16:0/552661346 >> 172.21.15.16:0/552661346 conn(0x561fdcc8c010 msgr2=0x561fdcc8c410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:27.973 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.966+0000 7f72bb5f4740 1 -- 172.21.15.16:0/552661346 shutdown_connections 2024-09-12T02:05:27.973 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.966+0000 7f72bb5f4740 1 -- 172.21.15.16:0/552661346 wait complete. 2024-09-12T02:05:27.974 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.967+0000 7f72bb5f4740 1 -- start start 2024-09-12T02:05:27.974 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.967+0000 7f72bb5f4740 1 Processor -- start 2024-09-12T02:05:27.974 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.967+0000 7f72bb5f4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561fdcc909a0 0x561fdcc9ab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:27.974 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.967+0000 7f72bb5f4740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561fdcc912e0 0x561fdcc9b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:27.974 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.967+0000 7f72bb5f4740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561fdcc96370 0x561fdcca20b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:27.974 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.967+0000 7f72bb5f4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x561fdcc992e0 con 0x561fdcc909a0 2024-09-12T02:05:27.974 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.967+0000 7f72bb5f4740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x561fdcb7d290 con 0x561fdcc96370 2024-09-12T02:05:27.974 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.967+0000 7f72bb5f4740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x561fdcc99460 con 0x561fdcc912e0 2024-09-12T02:05:27.974 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.967+0000 7f72bb5f4740 10 client.0 _ll_register_callbacks cb 0x561fdcbae140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:05:27.974 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.967+0000 7f72b651d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561fdcc909a0 0x561fdcc9ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:27.974 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.967+0000 7f72b651d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561fdcc909a0 0x561fdcc9ab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53828/0 (socket says 172.21.15.16:53828) 2024-09-12T02:05:27.974 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.967+0000 7f72b651d640 1 -- 172.21.15.16:0/3406341896 learned_addr learned my addr 172.21.15.16:0/3406341896 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:05:27.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.967+0000 7f72b6d1e640 1 --2- 172.21.15.16:0/3406341896 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561fdcc96370 0x561fdcca20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:27.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.967+0000 7f72b5d1c640 1 --2- 172.21.15.16:0/3406341896 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561fdcc912e0 0x561fdcc9b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:27.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.968+0000 7f72b651d640 1 -- 172.21.15.16:0/3406341896 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561fdcc912e0 msgr2=0x561fdcc9b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:27.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.968+0000 7f72b651d640 1 --2- 172.21.15.16:0/3406341896 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561fdcc912e0 0x561fdcc9b070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:27.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.968+0000 7f72b651d640 1 -- 172.21.15.16:0/3406341896 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561fdcc96370 msgr2=0x561fdcca20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:27.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.968+0000 7f72b651d640 1 --2- 172.21.15.16:0/3406341896 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561fdcc96370 0x561fdcca20b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:27.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.968+0000 7f72b651d640 1 -- 172.21.15.16:0/3406341896 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561fdc907210 con 0x561fdcc909a0 2024-09-12T02:05:27.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.968+0000 7f72b6d1e640 1 --2- 172.21.15.16:0/3406341896 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561fdcc96370 0x561fdcca20b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:05:27.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.968+0000 7f72b651d640 1 --2- 172.21.15.16:0/3406341896 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561fdcc909a0 0x561fdcc9ab30 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f72ac007590 tx=0x7f72ac09be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78fb7e512a57fc67 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:27.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.968+0000 7f7299ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:05:27.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.968+0000 7f7299ffb640 1 -- 172.21.15.16:0/3406341896 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72ac099de0 con 0x561fdcc909a0 2024-09-12T02:05:27.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.968+0000 7f7299ffb640 1 -- 172.21.15.16:0/3406341896 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f72ac09a040 con 0x561fdcc909a0 2024-09-12T02:05:27.976 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.968+0000 7f72bb5f4740 1 -- 172.21.15.16:0/3406341896 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f72a004c070 con 0x561fdcc909a0 2024-09-12T02:05:27.976 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.968+0000 7f7299ffb640 1 -- 172.21.15.16:0/3406341896 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f72ac018b80 con 0x561fdcc909a0 2024-09-12T02:05:27.976 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.968+0000 7f72bb5f4740 10 client.16407 fetch_fsmap learned FSMap version 25 2024-09-12T02:05:27.976 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.968+0000 7f72bb5f4740 1 -- 172.21.15.16:0/3406341896 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x561fdc9d3620 con 0x561fdcc909a0 2024-09-12T02:05:27.976 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.968+0000 7f7299ffb640 1 -- 172.21.15.16:0/3406341896 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72a004c070 con 0x561fdcc909a0 2024-09-12T02:05:27.976 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.968+0000 7f7299ffb640 1 -- 172.21.15.16:0/3406341896 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f72ac07acd0 con 0x561fdcc909a0 2024-09-12T02:05:27.976 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.968+0000 7f72bb5f4740 10 client.16407 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:05:27.976 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.969+0000 7f72bb5f4740 10 client.16407 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:05:27.976 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.969+0000 7f72bb5f4740 20 client.16407 populate_metadata read hostname 'smithi016' 2024-09-12T02:05:27.976 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.969+0000 7f72bb5f4740 1 --2- 172.21.15.16:0/3406341896 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x561fdcca6850 0x561fdcca8c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:27.976 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.969+0000 7f72bb5f4740 4 client.16407 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:05:27.976 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.969+0000 7f72bb5f4740 1 -- 172.21.15.16:0/3406341896 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561fdcc01a20 con 0x561fdcca6850 2024-09-12T02:05:27.976 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.969+0000 7f72b5d1c640 1 --2- 172.21.15.16:0/3406341896 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x561fdcca6850 0x561fdcca8c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:27.977 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.969+0000 7f7299ffb640 10 client.16407 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:05:27.977 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.969+0000 7f72b5d1c640 1 --2- 172.21.15.16:0/3406341896 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x561fdcca6850 0x561fdcca8c60 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d54690aac286483 server_cookie=e42c8464bca16ebc in_seq=0 out_seq=0 2024-09-12T02:05:27.977 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.970+0000 7f7299ffb640 1 -- 172.21.15.16:0/3406341896 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561fdcc01a20 con 0x561fdcca6850 2024-09-12T02:05:27.977 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.970+0000 7f7299ffb640 10 client.16407 handle_command_reply: tid=0 2024-09-12T02:05:27.977 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.970+0000 7f72bb5f4740 1 client.16407 shutdown 2024-09-12T02:05:27.977 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.970+0000 7f72bb5f4740 1 -- 172.21.15.16:0/3406341896 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561fdcc909a0 msgr2=0x561fdcc9ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:27.977 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.970+0000 7f72bb5f4740 1 --2- 172.21.15.16:0/3406341896 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561fdcc909a0 0x561fdcc9ab30 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f72ac007590 tx=0x7f72ac09be00 comp rx=0 tx=0).stop 2024-09-12T02:05:27.977 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.970+0000 7f72bb5f4740 1 -- 172.21.15.16:0/3406341896 shutdown_connections 2024-09-12T02:05:27.977 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.970+0000 7f72bb5f4740 1 --2- 172.21.15.16:0/3406341896 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x561fdcca6850 0x561fdcca8c60 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:27.977 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.970+0000 7f72bb5f4740 1 --2- 172.21.15.16:0/3406341896 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561fdcc96370 0x561fdcca20b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:27.977 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.970+0000 7f72bb5f4740 1 --2- 172.21.15.16:0/3406341896 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561fdcc912e0 0x561fdcc9b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:27.977 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.970+0000 7f7299ffb640 0 client.16407 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:05:27.977 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.970+0000 7f72bb5f4740 1 --2- 172.21.15.16:0/3406341896 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561fdcc909a0 0x561fdcc9ab30 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:27.978 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.970+0000 7f72bb5f4740 1 -- 172.21.15.16:0/3406341896 >> 172.21.15.16:0/3406341896 conn(0x561fdcc8c010 msgr2=0x561fdcc8d820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:27.978 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.970+0000 7f72bb5f4740 1 -- 172.21.15.16:0/3406341896 shutdown_connections 2024-09-12T02:05:27.978 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.970+0000 7f72bb5f4740 1 -- 172.21.15.16:0/3406341896 wait complete. 2024-09-12T02:05:27.978 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.970+0000 7f72bb5f4740 20 client.16407 trim_cache size 0 max 16384 2024-09-12T02:05:27.986 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.985+0000 7f72bb5f4740 1 Processor -- start 2024-09-12T02:05:27.986 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.985+0000 7f72bb5f4740 1 -- start start 2024-09-12T02:05:27.986 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.985+0000 7f72bb5f4740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561fdcc909a0 0x561fdcc9af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:27.987 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.985+0000 7f72bb5f4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561fdcc912e0 0x561fdcca2070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:27.987 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.985+0000 7f72bb5f4740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561fdcc96370 0x561fdccad560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:27.987 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.985+0000 7f72bb5f4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f72b01ad860 con 0x561fdcc912e0 2024-09-12T02:05:27.987 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.985+0000 7f72bb5f4740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f72b01ab130 con 0x561fdcc909a0 2024-09-12T02:05:27.987 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.985+0000 7f72bb5f4740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f72b01a8f80 con 0x561fdcc96370 2024-09-12T02:05:27.987 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.985+0000 7f72b5d1c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561fdcc912e0 0x561fdcca2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:27.987 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.985+0000 7f72b5d1c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561fdcc912e0 0x561fdcca2070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53838/0 (socket says 172.21.15.16:53838) 2024-09-12T02:05:27.987 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.985+0000 7f72b5d1c640 1 -- 172.21.15.16:0/3841156314 learned_addr learned my addr 172.21.15.16:0/3841156314 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:05:27.987 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.986+0000 7f72b651d640 1 --2- 172.21.15.16:0/3841156314 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561fdcc909a0 0x561fdcc9af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:27.987 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.986+0000 7f72b6d1e640 1 --2- 172.21.15.16:0/3841156314 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561fdcc96370 0x561fdccad560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:27.987 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.986+0000 7f72b5d1c640 1 -- 172.21.15.16:0/3841156314 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561fdcc96370 msgr2=0x561fdccad560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:27.987 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.986+0000 7f72b5d1c640 1 --2- 172.21.15.16:0/3841156314 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561fdcc96370 0x561fdccad560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:27.987 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.986+0000 7f72b5d1c640 1 -- 172.21.15.16:0/3841156314 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561fdcc909a0 msgr2=0x561fdcc9af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:27.988 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.986+0000 7f72b5d1c640 1 --2- 172.21.15.16:0/3841156314 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561fdcc909a0 0x561fdcc9af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:27.988 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.986+0000 7f72b5d1c640 1 -- 172.21.15.16:0/3841156314 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561fdcc8cca0 con 0x561fdcc912e0 2024-09-12T02:05:27.988 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.986+0000 7f72b5d1c640 1 --2- 172.21.15.16:0/3841156314 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561fdcc912e0 0x561fdcca2070 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f72a00314e0 tx=0x7f72a003f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e76e66c0e4823651 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:27.988 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.986+0000 7f729bfff640 1 -- 172.21.15.16:0/3841156314 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72a0035040 con 0x561fdcc912e0 2024-09-12T02:05:27.988 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.986+0000 7f729bfff640 1 -- 172.21.15.16:0/3841156314 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72a004c050 con 0x561fdcc912e0 2024-09-12T02:05:27.988 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.986+0000 7f729bfff640 1 -- 172.21.15.16:0/3841156314 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72a005a630 con 0x561fdcc912e0 2024-09-12T02:05:27.991 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.987+0000 7f72bb5f4740 1 -- 172.21.15.16:0/3841156314 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561fdcc912e0 msgr2=0x561fdcca2070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:27.991 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.987+0000 7f72bb5f4740 1 --2- 172.21.15.16:0/3841156314 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561fdcc912e0 0x561fdcca2070 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f72a00314e0 tx=0x7f72a003f000 comp rx=0 tx=0).stop 2024-09-12T02:05:27.992 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.987+0000 7f72bb5f4740 1 -- 172.21.15.16:0/3841156314 shutdown_connections 2024-09-12T02:05:27.992 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.987+0000 7f72bb5f4740 1 --2- 172.21.15.16:0/3841156314 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561fdcc96370 0x561fdccad560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:27.992 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.987+0000 7f72bb5f4740 1 --2- 172.21.15.16:0/3841156314 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561fdcc912e0 0x561fdcca2070 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:27.992 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.987+0000 7f72bb5f4740 1 --2- 172.21.15.16:0/3841156314 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561fdcc909a0 0x561fdcc9af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:27.993 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.987+0000 7f72bb5f4740 1 -- 172.21.15.16:0/3841156314 >> 172.21.15.16:0/3841156314 conn(0x561fdcc8c010 msgr2=0x561fdcc8b1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:27.993 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.987+0000 7f72bb5f4740 1 -- 172.21.15.16:0/3841156314 shutdown_connections 2024-09-12T02:05:27.993 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.987+0000 7f72bb5f4740 1 -- 172.21.15.16:0/3841156314 wait complete. 2024-09-12T02:05:27.993 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.987+0000 7f72bb5f4740 1 -- start start 2024-09-12T02:05:27.993 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.988+0000 7f72bb5f4740 1 Processor -- start 2024-09-12T02:05:27.993 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.988+0000 7f72bb5f4740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561fdcc909a0 0x561fdcc9fd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:27.993 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.988+0000 7f72bb5f4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561fdcc912e0 0x561fdcca0290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:27.993 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.988+0000 7f72bb5f4740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561fdcc96370 0x561fdcca12a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:27.993 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.988+0000 7f72bb5f4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f72b01a95f0 con 0x561fdcc912e0 2024-09-12T02:05:27.993 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.988+0000 7f72bb5f4740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f72b01aaa60 con 0x561fdcc909a0 2024-09-12T02:05:27.993 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.988+0000 7f72bb5f4740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f72b01a9cc0 con 0x561fdcc96370 2024-09-12T02:05:27.993 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.988+0000 7f72bb5f4740 10 client.0 _ll_register_callbacks cb 0x7f72ac09b730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:05:27.993 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.988+0000 7f72b5d1c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561fdcc912e0 0x561fdcca0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:27.994 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.988+0000 7f72b5d1c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561fdcc912e0 0x561fdcca0290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53840/0 (socket says 172.21.15.16:53840) 2024-09-12T02:05:27.994 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.988+0000 7f72b5d1c640 1 -- 172.21.15.16:0/2907295628 learned_addr learned my addr 172.21.15.16:0/2907295628 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:05:27.994 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.988+0000 7f72b6d1e640 1 --2- 172.21.15.16:0/2907295628 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561fdcc96370 0x561fdcca12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:27.994 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.988+0000 7f72b651d640 1 --2- 172.21.15.16:0/2907295628 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561fdcc909a0 0x561fdcc9fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:27.994 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.988+0000 7f72b5d1c640 1 -- 172.21.15.16:0/2907295628 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561fdcc96370 msgr2=0x561fdcca12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:27.994 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.988+0000 7f72b5d1c640 1 --2- 172.21.15.16:0/2907295628 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561fdcc96370 0x561fdcca12a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:27.994 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.988+0000 7f72b5d1c640 1 -- 172.21.15.16:0/2907295628 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561fdcc909a0 msgr2=0x561fdcc9fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:27.994 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.988+0000 7f72b5d1c640 1 --2- 172.21.15.16:0/2907295628 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561fdcc909a0 0x561fdcc9fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:27.994 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.988+0000 7f72b5d1c640 1 -- 172.21.15.16:0/2907295628 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561fdcca18b0 con 0x561fdcc912e0 2024-09-12T02:05:27.994 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.988+0000 7f72b651d640 1 --2- 172.21.15.16:0/2907295628 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561fdcc909a0 0x561fdcc9fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:05:27.994 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.988+0000 7f72b5d1c640 1 --2- 172.21.15.16:0/2907295628 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561fdcc912e0 0x561fdcca0290 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f72a0006010 tx=0x7f72a0005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6318df4b5f759a92 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:27.994 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.988+0000 7f7299ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:05:27.995 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.989+0000 7f7299ffb640 1 -- 172.21.15.16:0/2907295628 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72a0035040 con 0x561fdcc912e0 2024-09-12T02:05:27.995 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.989+0000 7f7299ffb640 1 -- 172.21.15.16:0/2907295628 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f72a00055d0 con 0x561fdcc912e0 2024-09-12T02:05:27.995 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.989+0000 7f72bb5f4740 1 -- 172.21.15.16:0/2907295628 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f72a004c050 con 0x561fdcc912e0 2024-09-12T02:05:27.995 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.989+0000 7f7299ffb640 1 -- 172.21.15.16:0/2907295628 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f72a00316b0 con 0x561fdcc912e0 2024-09-12T02:05:27.995 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.989+0000 7f72bb5f4740 10 client.16413 fetch_fsmap learned FSMap version 25 2024-09-12T02:05:27.995 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.989+0000 7f72bb5f4740 1 -- 172.21.15.16:0/2907295628 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x561fdcd11b10 con 0x561fdcc912e0 2024-09-12T02:05:27.995 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.989+0000 7f7299ffb640 1 -- 172.21.15.16:0/2907295628 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72a004c050 con 0x561fdcc912e0 2024-09-12T02:05:27.995 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.989+0000 7f7299ffb640 1 -- 172.21.15.16:0/2907295628 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f72a005a920 con 0x561fdcc912e0 2024-09-12T02:05:27.995 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.989+0000 7f72bb5f4740 10 client.16413 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:05:27.995 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.989+0000 7f72bb5f4740 10 client.16413 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:05:27.995 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.989+0000 7f72bb5f4740 20 client.16413 populate_metadata read hostname 'smithi016' 2024-09-12T02:05:27.995 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.989+0000 7f72bb5f4740 1 --2- 172.21.15.16:0/2907295628 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x561fdcd12090 0x561fdcd14480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:27.995 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.989+0000 7f72bb5f4740 4 client.16413 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:05:27.996 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.989+0000 7f72bb5f4740 1 -- 172.21.15.16:0/2907295628 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561fdcbfbd70 con 0x561fdcd12090 2024-09-12T02:05:27.996 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.989+0000 7f72b651d640 1 --2- 172.21.15.16:0/2907295628 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x561fdcd12090 0x561fdcd14480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:27.996 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.990+0000 7f72b651d640 1 --2- 172.21.15.16:0/2907295628 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x561fdcd12090 0x561fdcd14480 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c3dbf9962e59f8d server_cookie=66e244eb86ae205a in_seq=0 out_seq=0 2024-09-12T02:05:27.996 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:27.990+0000 7f7299ffb640 10 client.16413 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:05:28.003 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:28.003+0000 7f7299ffb640 1 -- 172.21.15.16:0/2907295628 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561fdcbfbd70 con 0x561fdcd12090 2024-09-12T02:05:28.003 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:28.003+0000 7f7299ffb640 10 client.16413 handle_command_reply: tid=0 2024-09-12T02:05:28.003 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:28.003+0000 7f72bb5f4740 1 client.16413 shutdown 2024-09-12T02:05:28.003 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:28.003+0000 7f72bb5f4740 1 -- 172.21.15.16:0/2907295628 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561fdcc912e0 msgr2=0x561fdcca0290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:28.004 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:28.003+0000 7f72bb5f4740 1 --2- 172.21.15.16:0/2907295628 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561fdcc912e0 0x561fdcca0290 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f72a0006010 tx=0x7f72a0005b50 comp rx=0 tx=0).stop 2024-09-12T02:05:28.004 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:28.003+0000 7f72bb5f4740 1 -- 172.21.15.16:0/2907295628 shutdown_connections 2024-09-12T02:05:28.004 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:28.003+0000 7f72bb5f4740 1 --2- 172.21.15.16:0/2907295628 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x561fdcd12090 0x561fdcd14480 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:28.004 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:28.003+0000 7f72bb5f4740 1 --2- 172.21.15.16:0/2907295628 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561fdcc96370 0x561fdcca12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:28.004 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:28.003+0000 7f7299ffb640 0 client.16413 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:05:28.004 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:28.003+0000 7f72bb5f4740 1 --2- 172.21.15.16:0/2907295628 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561fdcc912e0 0x561fdcca0290 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:28.004 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:28.003+0000 7f72bb5f4740 1 --2- 172.21.15.16:0/2907295628 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561fdcc909a0 0x561fdcc9fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:28.004 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:28.003+0000 7f72bb5f4740 1 -- 172.21.15.16:0/2907295628 >> 172.21.15.16:0/2907295628 conn(0x561fdcc8c010 msgr2=0x561fdcc8a9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:28.004 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:28.003+0000 7f72bb5f4740 1 -- 172.21.15.16:0/2907295628 shutdown_connections 2024-09-12T02:05:28.004 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:28.003+0000 7f72bb5f4740 1 -- 172.21.15.16:0/2907295628 wait complete. 2024-09-12T02:05:28.004 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:28.003+0000 7f72bb5f4740 20 client.16413 trim_cache size 0 max 16384 2024-09-12T02:05:28.005 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-09-12T02:05:28.005 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-09-12T02:05:28.005 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-09-12T02:05:28.005 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-09-12T02:05:28.006 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:28.004+0000 7f72b8781640 1 -- 172.21.15.16:0/2213300593 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f729c0a2cb0 msgr2=0x7f729c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:28.006 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:28.004+0000 7f72b8781640 1 --2- 172.21.15.16:0/2213300593 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f729c0a2cb0 0x7f729c0a5170 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f72a0000c00 tx=0x7f72a00095f0 comp rx=0 tx=0).stop 2024-09-12T02:05:28.006 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:28.004+0000 7f72b8781640 1 -- 172.21.15.16:0/2213300593 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72b0101b60 msgr2=0x7f72b0075fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:28.006 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:28.004+0000 7f72b8781640 1 --2- 172.21.15.16:0/2213300593 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72b0101b60 0x7f72b0075fa0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f72ac00c970 tx=0x7f72ac00ce30 comp rx=0 tx=0).stop 2024-09-12T02:05:28.006 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:28.004+0000 7f72b8781640 1 -- 172.21.15.16:0/2213300593 shutdown_connections 2024-09-12T02:05:28.006 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:28.004+0000 7f72b8781640 1 --2- 172.21.15.16:0/2213300593 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f729c0a2cb0 0x7f729c0a5170 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:28.006 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:28.004+0000 7f72b8781640 1 --2- 172.21.15.16:0/2213300593 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f72b0103710 0x7f72b0074b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:28.006 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:28.004+0000 7f72b8781640 1 --2- 172.21.15.16:0/2213300593 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f72b0102d50 0x7f72b00745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:28.006 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:28.004+0000 7f72b8781640 1 --2- 172.21.15.16:0/2213300593 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72b0101b60 0x7f72b0075fa0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:28.006 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:28.004+0000 7f72b8781640 1 -- 172.21.15.16:0/2213300593 >> 172.21.15.16:0/2213300593 conn(0x7f72b00fd2e0 msgr2=0x7f72b0105ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:28.006 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:28.004+0000 7f72b8781640 1 -- 172.21.15.16:0/2213300593 shutdown_connections 2024-09-12T02:05:28.007 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:28.004+0000 7f72b8781640 1 -- 172.21.15.16:0/2213300593 wait complete. 2024-09-12T02:05:28.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:05:27 smithi184 ceph-mon[35593]: pgmap v441: 97 pgs: 97 active+clean; 2.0 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-09-12T02:05:28.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:05:27 smithi033 ceph-mon[35332]: pgmap v441: 97 pgs: 97 active+clean; 2.0 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-09-12T02:05:28.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:05:27 smithi016 ceph-mon[30947]: pgmap v441: 97 pgs: 97 active+clean; 2.0 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-09-12T02:05:28.568 INFO:tasks.quiescer.fs.[cephfs]:Total duration (40.3) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-12T02:05:28.568 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 20.15 2024-09-12T02:05:28.569 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:05:28.828 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:05:29.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:29.404+0000 7f4558d2a640 1 -- 172.21.15.16:0/211073635 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4554102d40 msgr2=0x7f4554103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:29.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:29.404+0000 7f4558d2a640 1 --2- 172.21.15.16:0/211073635 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4554102d40 0x7f4554103120 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f4544009900 tx=0x7f454402a480 comp rx=0 tx=0).stop 2024-09-12T02:05:29.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:29.404+0000 7f4558d2a640 1 -- 172.21.15.16:0/211073635 shutdown_connections 2024-09-12T02:05:29.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:29.404+0000 7f4558d2a640 1 --2- 172.21.15.16:0/211073635 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f455410c170 0x7f455410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:29.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:29.404+0000 7f4558d2a640 1 --2- 172.21.15.16:0/211073635 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4554103660 0x7f455410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:29.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:29.404+0000 7f4558d2a640 1 --2- 172.21.15.16:0/211073635 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4554102d40 0x7f4554103120 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:29.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:29.404+0000 7f4558d2a640 1 -- 172.21.15.16:0/211073635 >> 172.21.15.16:0/211073635 conn(0x7f45540fc7a0 msgr2=0x7f45540febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:29.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:29.404+0000 7f4558d2a640 1 -- 172.21.15.16:0/211073635 shutdown_connections 2024-09-12T02:05:29.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:29.404+0000 7f4558d2a640 1 -- 172.21.15.16:0/211073635 wait complete. 2024-09-12T02:05:29.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:29.405+0000 7f4558d2a640 1 Processor -- start 2024-09-12T02:05:29.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:29.405+0000 7f4558d2a640 1 -- start start 2024-09-12T02:05:29.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:29.405+0000 7f4558d2a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4554102d40 0x7f45541984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:29.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:29.405+0000 7f4558d2a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4554103660 0x7f4554198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:29.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:29.405+0000 7f4558d2a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f455410c170 0x7f455419cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:29.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:29.405+0000 7f4558d2a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4554111f70 con 0x7f4554102d40 2024-09-12T02:05:29.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:29.405+0000 7f4558d2a640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4554111df0 con 0x7f455410c170 2024-09-12T02:05:29.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:29.405+0000 7f4558d2a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f45541120f0 con 0x7f4554103660 2024-09-12T02:05:29.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:29.405+0000 7f455259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4554102d40 0x7f45541984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:29.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:29.405+0000 7f455259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4554102d40 0x7f45541984f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53862/0 (socket says 172.21.15.16:53862) 2024-09-12T02:05:29.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:29.405+0000 7f455259c640 1 -- 172.21.15.16:0/3060161238 learned_addr learned my addr 172.21.15.16:0/3060161238 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:05:29.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:29.405+0000 7f4552d9d640 1 --2- 172.21.15.16:0/3060161238 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f455410c170 0x7f455419cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:29.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:29.405+0000 7f4551d9b640 1 --2- 172.21.15.16:0/3060161238 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4554103660 0x7f4554198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:29.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:29.405+0000 7f455259c640 1 -- 172.21.15.16:0/3060161238 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4554103660 msgr2=0x7f4554198a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:29.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:29.405+0000 7f455259c640 1 --2- 172.21.15.16:0/3060161238 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4554103660 0x7f4554198a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:29.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:29.405+0000 7f455259c640 1 -- 172.21.15.16:0/3060161238 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f455410c170 msgr2=0x7f455419cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:29.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:29.405+0000 7f455259c640 1 --2- 172.21.15.16:0/3060161238 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f455410c170 0x7f455419cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:29.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:29.405+0000 7f455259c640 1 -- 172.21.15.16:0/3060161238 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f455419d4a0 con 0x7f4554102d40 2024-09-12T02:05:29.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:29.405+0000 7f4552d9d640 1 --2- 172.21.15.16:0/3060161238 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f455410c170 0x7f455419cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:05:29.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:29.406+0000 7f455259c640 1 --2- 172.21.15.16:0/3060161238 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4554102d40 0x7f45541984f0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f4544006010 tx=0x7f45440042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:29.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:29.406+0000 7f45437fe640 1 -- 172.21.15.16:0/3060161238 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f454403f030 con 0x7f4554102d40 2024-09-12T02:05:29.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:29.406+0000 7f4558d2a640 1 -- 172.21.15.16:0/3060161238 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f455419d730 con 0x7f4554102d40 2024-09-12T02:05:29.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:29.406+0000 7f45437fe640 1 -- 172.21.15.16:0/3060161238 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4544003ce0 con 0x7f4554102d40 2024-09-12T02:05:29.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:29.406+0000 7f45437fe640 1 -- 172.21.15.16:0/3060161238 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4544035070 con 0x7f4554102d40 2024-09-12T02:05:29.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:29.406+0000 7f4558d2a640 1 -- 172.21.15.16:0/3060161238 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45541a5070 con 0x7f4554102d40 2024-09-12T02:05:29.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:29.407+0000 7f4558d2a640 1 -- 172.21.15.16:0/3060161238 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4554103120 con 0x7f4554102d40 2024-09-12T02:05:29.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:29.408+0000 7f45437fe640 1 -- 172.21.15.16:0/3060161238 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f4544042030 con 0x7f4554102d40 2024-09-12T02:05:29.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:29.409+0000 7f45437fe640 1 --2- 172.21.15.16:0/3060161238 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f453c0a2b80 0x7f453c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:29.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:29.409+0000 7f4551d9b640 1 --2- 172.21.15.16:0/3060161238 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f453c0a2b80 0x7f453c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:29.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:29.409+0000 7f45437fe640 1 -- 172.21.15.16:0/3060161238 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f45440efa10 con 0x7f4554102d40 2024-09-12T02:05:29.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:29.409+0000 7f4551d9b640 1 --2- 172.21.15.16:0/3060161238 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f453c0a2b80 0x7f453c0a5040 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f4548005fd0 tx=0x7f4548005d00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:29.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:29.412+0000 7f45437fe640 1 -- 172.21.15.16:0/3060161238 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f45440bca30 con 0x7f4554102d40 2024-09-12T02:05:29.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:29.624+0000 7f4558d2a640 1 -- 172.21.15.16:0/3060161238 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f45541071e0 con 0x7f4554102d40 2024-09-12T02:05:29.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:29.624+0000 7f45437fe640 1 -- 172.21.15.16:0/3060161238 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11911 (secure 0 0 0) 0x7f45440bc330 con 0x7f4554102d40 2024-09-12T02:05:29.625 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T02:05:29.625 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":25,"btime":"2024-09-12T02:01:37:192021+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3279123279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3279123279},{"type":"v1","addr":"172.21.15.16:6835","nonce":3279123279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4119452382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4119452382},{"type":"v1","addr":"172.21.15.16:6837","nonce":4119452382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/3263823165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3263823165},{"type":"v1","addr":"172.21.15.16:6839","nonce":3263823165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/1510290732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":1510290732},{"type":"v1","addr":"172.21.15.16:6841","nonce":1510290732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/279674603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":279674603},{"type":"v1","addr":"172.21.15.33:6833","nonce":279674603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3555728028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3555728028},{"type":"v1","addr":"172.21.15.33:6835","nonce":3555728028}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T01:59:56.117608+0000","modified":"2024-09-12T02:01:36.294760+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24482,"mds_1":24481,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6833/1563374106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1563374106},{"type":"v1","addr":"172.21.15.184:6833","nonce":1563374106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6837/1892833868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1892833868},{"type":"v1","addr":"172.21.15.33:6837","nonce":1892833868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/1457679686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1457679686},{"type":"v1","addr":"172.21.15.184:6835","nonce":1457679686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.184:6837/1793393087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":1793393087},{"type":"v1","addr":"172.21.15.184:6837","nonce":1793393087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.184:6839/4203347941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":4203347941},{"type":"v1","addr":"172.21.15.184:6839","nonce":4203347941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.33:6839/4136670204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":4136670204},{"type":"v1","addr":"172.21.15.33:6839","nonce":4136670204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24481,24482,24469]},"id":1}]} 2024-09-12T02:05:29.626 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 25 2024-09-12T02:05:29.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:29.626+0000 7f4558d2a640 1 -- 172.21.15.16:0/3060161238 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f453c0a2b80 msgr2=0x7f453c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:29.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:29.626+0000 7f4558d2a640 1 --2- 172.21.15.16:0/3060161238 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f453c0a2b80 0x7f453c0a5040 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f4548005fd0 tx=0x7f4548005d00 comp rx=0 tx=0).stop 2024-09-12T02:05:29.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:29.626+0000 7f4558d2a640 1 -- 172.21.15.16:0/3060161238 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4554102d40 msgr2=0x7f45541984f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:29.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:29.626+0000 7f4558d2a640 1 --2- 172.21.15.16:0/3060161238 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4554102d40 0x7f45541984f0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f4544006010 tx=0x7f45440042f0 comp rx=0 tx=0).stop 2024-09-12T02:05:29.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:29.627+0000 7f4558d2a640 1 -- 172.21.15.16:0/3060161238 shutdown_connections 2024-09-12T02:05:29.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:29.627+0000 7f4558d2a640 1 --2- 172.21.15.16:0/3060161238 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f453c0a2b80 0x7f453c0a5040 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:29.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:29.627+0000 7f4558d2a640 1 --2- 172.21.15.16:0/3060161238 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f455410c170 0x7f455419cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:29.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:29.627+0000 7f4558d2a640 1 --2- 172.21.15.16:0/3060161238 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4554103660 0x7f4554198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:29.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:29.627+0000 7f4558d2a640 1 --2- 172.21.15.16:0/3060161238 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4554102d40 0x7f45541984f0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:29.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:29.627+0000 7f4558d2a640 1 -- 172.21.15.16:0/3060161238 >> 172.21.15.16:0/3060161238 conn(0x7f45540fc7a0 msgr2=0x7f45540fd4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:29.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:29.627+0000 7f4558d2a640 1 -- 172.21.15.16:0/3060161238 shutdown_connections 2024-09-12T02:05:29.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:29.627+0000 7f4558d2a640 1 -- 172.21.15.16:0/3060161238 wait complete. 2024-09-12T02:05:30.350 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:05:29 smithi016 ceph-mon[30947]: pgmap v442: 97 pgs: 97 active+clean; 2.0 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 11 op/s 2024-09-12T02:05:30.350 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:05:29 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/3060161238' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:05:30.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:05:29 smithi184 ceph-mon[35593]: pgmap v442: 97 pgs: 97 active+clean; 2.0 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 11 op/s 2024-09-12T02:05:30.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:05:29 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/3060161238' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:05:30.383 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24482 quiesce db / --timeout 90 --expiration 140.15 --await-for 45.0' 2024-09-12T02:05:30.383 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24482 quiesce db / --timeout 90 --expiration 140.15 --await-for 45.0 2024-09-12T02:05:30.418 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:05:29 smithi033 ceph-mon[35332]: pgmap v442: 97 pgs: 97 active+clean; 2.0 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 11 op/s 2024-09-12T02:05:30.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:05:29 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/3060161238' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:05:30.651 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:05:31.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.044+0000 7f78c172a640 1 -- 172.21.15.16:0/4256138727 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78bc10c170 msgr2=0x7f78bc10e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:31.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.044+0000 7f78c172a640 1 --2- 172.21.15.16:0/4256138727 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78bc10c170 0x7f78bc10e610 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f78b4009ca0 tx=0x7f78b402a950 comp rx=0 tx=0).stop 2024-09-12T02:05:31.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.045+0000 7f78c172a640 1 -- 172.21.15.16:0/4256138727 shutdown_connections 2024-09-12T02:05:31.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.045+0000 7f78c172a640 1 --2- 172.21.15.16:0/4256138727 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78bc10c170 0x7f78bc10e610 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:31.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.045+0000 7f78c172a640 1 --2- 172.21.15.16:0/4256138727 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78bc103660 0x7f78bc10bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:31.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.045+0000 7f78c172a640 1 --2- 172.21.15.16:0/4256138727 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f78bc102d40 0x7f78bc103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:31.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.045+0000 7f78c172a640 1 -- 172.21.15.16:0/4256138727 >> 172.21.15.16:0/4256138727 conn(0x7f78bc0fc7a0 msgr2=0x7f78bc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:31.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.045+0000 7f78c172a640 1 -- 172.21.15.16:0/4256138727 shutdown_connections 2024-09-12T02:05:31.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.045+0000 7f78c172a640 1 -- 172.21.15.16:0/4256138727 wait complete. 2024-09-12T02:05:31.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.045+0000 7f78c172a640 1 Processor -- start 2024-09-12T02:05:31.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.045+0000 7f78c172a640 1 -- start start 2024-09-12T02:05:31.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.046+0000 7f78c172a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78bc102d40 0x7f78bc1983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:31.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.046+0000 7f78c172a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f78bc103660 0x7f78bc198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:31.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.046+0000 7f78c172a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78bc10c170 0x7f78bc19cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:31.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.046+0000 7f78c172a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f78bc111f70 con 0x7f78bc102d40 2024-09-12T02:05:31.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.046+0000 7f78c172a640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f78bc111df0 con 0x7f78bc10c170 2024-09-12T02:05:31.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.046+0000 7f78c172a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f78bc1120f0 con 0x7f78bc103660 2024-09-12T02:05:31.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.046+0000 7f78baffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78bc102d40 0x7f78bc1983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:31.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.046+0000 7f78baffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78bc102d40 0x7f78bc1983d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53888/0 (socket says 172.21.15.16:53888) 2024-09-12T02:05:31.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.046+0000 7f78baffd640 1 -- 172.21.15.16:0/3838656520 learned_addr learned my addr 172.21.15.16:0/3838656520 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:05:31.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.046+0000 7f78b3fff640 1 --2- 172.21.15.16:0/3838656520 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f78bc103660 0x7f78bc198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:31.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.046+0000 7f78bb7fe640 1 --2- 172.21.15.16:0/3838656520 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78bc10c170 0x7f78bc19cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:31.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.046+0000 7f78baffd640 1 -- 172.21.15.16:0/3838656520 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f78bc103660 msgr2=0x7f78bc198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:31.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.046+0000 7f78baffd640 1 --2- 172.21.15.16:0/3838656520 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f78bc103660 0x7f78bc198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:31.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.046+0000 7f78baffd640 1 -- 172.21.15.16:0/3838656520 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78bc10c170 msgr2=0x7f78bc19cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:31.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.046+0000 7f78baffd640 1 --2- 172.21.15.16:0/3838656520 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78bc10c170 0x7f78bc19cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:31.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.046+0000 7f78baffd640 1 -- 172.21.15.16:0/3838656520 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78bc19d380 con 0x7f78bc102d40 2024-09-12T02:05:31.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.046+0000 7f78b3fff640 1 --2- 172.21.15.16:0/3838656520 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f78bc103660 0x7f78bc198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:05:31.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.046+0000 7f78baffd640 1 --2- 172.21.15.16:0/3838656520 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78bc102d40 0x7f78bc1983d0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f78ac00e9f0 tx=0x7f78ac00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:31.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.047+0000 7f78b8ff9640 1 -- 172.21.15.16:0/3838656520 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78ac00cd80 con 0x7f78bc102d40 2024-09-12T02:05:31.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.047+0000 7f78b8ff9640 1 -- 172.21.15.16:0/3838656520 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f78ac010910 con 0x7f78bc102d40 2024-09-12T02:05:31.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.047+0000 7f78c172a640 1 -- 172.21.15.16:0/3838656520 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78bc19d670 con 0x7f78bc102d40 2024-09-12T02:05:31.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.047+0000 7f78b8ff9640 1 -- 172.21.15.16:0/3838656520 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78ac018990 con 0x7f78bc102d40 2024-09-12T02:05:31.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.047+0000 7f78c172a640 1 -- 172.21.15.16:0/3838656520 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78bc1a4f50 con 0x7f78bc102d40 2024-09-12T02:05:31.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.049+0000 7f78b8ff9640 1 -- 172.21.15.16:0/3838656520 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f78ac010ac0 con 0x7f78bc102d40 2024-09-12T02:05:31.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.049+0000 7f78c3b03740 1 Processor -- start 2024-09-12T02:05:31.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.049+0000 7f78c3b03740 1 -- start start 2024-09-12T02:05:31.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.050+0000 7f78c3b03740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b3df67ec60 0x55b3df67f040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:31.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.050+0000 7f78c3b03740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3df67f580 0x55b3df689900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:31.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.050+0000 7f78c3b03740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b3df689e40 0x55b3df68c230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:31.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.050+0000 7f78c3b03740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55b3df5df160 con 0x55b3df67f580 2024-09-12T02:05:31.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.050+0000 7f78c3b03740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55b3df586720 con 0x55b3df67ec60 2024-09-12T02:05:31.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.050+0000 7f78c3b03740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55b3df5db120 con 0x55b3df689e40 2024-09-12T02:05:31.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.050+0000 7f78baffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3df67f580 0x55b3df689900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:31.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.050+0000 7f78baffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3df67f580 0x55b3df689900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53896/0 (socket says 172.21.15.16:53896) 2024-09-12T02:05:31.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.050+0000 7f78baffd640 1 -- 172.21.15.16:0/3188324390 learned_addr learned my addr 172.21.15.16:0/3188324390 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:05:31.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.050+0000 7f78bb7fe640 1 --2- 172.21.15.16:0/3188324390 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b3df67ec60 0x55b3df67f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:31.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.050+0000 7f78b8ff9640 1 --2- 172.21.15.16:0/3838656520 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f78a80a2b80 0x7f78a80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:31.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.050+0000 7f78b3fff640 1 --2- 172.21.15.16:0/3188324390 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b3df689e40 0x55b3df68c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:31.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.050+0000 7f78bb7fe640 1 --2- 172.21.15.16:0/3838656520 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f78a80a2b80 0x7f78a80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:31.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.050+0000 7f78baffd640 1 -- 172.21.15.16:0/3188324390 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b3df689e40 msgr2=0x55b3df68c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:31.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.050+0000 7f78baffd640 1 --2- 172.21.15.16:0/3188324390 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b3df689e40 0x55b3df68c230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:31.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.050+0000 7f78baffd640 1 -- 172.21.15.16:0/3188324390 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b3df67ec60 msgr2=0x55b3df67f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:31.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.050+0000 7f78baffd640 1 --2- 172.21.15.16:0/3188324390 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b3df67ec60 0x55b3df67f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:31.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.050+0000 7f78baffd640 1 -- 172.21.15.16:0/3188324390 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b3df350ba0 con 0x55b3df67f580 2024-09-12T02:05:31.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.050+0000 7f78b8ff9640 1 -- 172.21.15.16:0/3838656520 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f78ac0cf2b0 con 0x7f78bc102d40 2024-09-12T02:05:31.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.051+0000 7f78baffd640 1 --2- 172.21.15.16:0/3188324390 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3df67f580 0x55b3df689900 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f78ac09b960 tx=0x7f78ac018420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbae6a31c0c5ba48 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:31.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.051+0000 7f78b17fa640 1 -- 172.21.15.16:0/3188324390 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78ac09b5c0 con 0x55b3df67f580 2024-09-12T02:05:31.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.051+0000 7f78bb7fe640 1 --2- 172.21.15.16:0/3838656520 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f78a80a2b80 0x7f78a80a5040 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f78b4004ad0 tx=0x7f78b402a950 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:31.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.051+0000 7f78b17fa640 1 -- 172.21.15.16:0/3188324390 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f78ac09bc60 con 0x55b3df67f580 2024-09-12T02:05:31.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.051+0000 7f78b17fa640 1 -- 172.21.15.16:0/3188324390 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78ac00b2c0 con 0x55b3df67f580 2024-09-12T02:05:31.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.052+0000 7f78c3b03740 1 -- 172.21.15.16:0/3188324390 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3df67f580 msgr2=0x55b3df689900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:31.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.052+0000 7f78c3b03740 1 --2- 172.21.15.16:0/3188324390 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3df67f580 0x55b3df689900 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f78ac09b960 tx=0x7f78ac018420 comp rx=0 tx=0).stop 2024-09-12T02:05:31.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.052+0000 7f78c3b03740 1 -- 172.21.15.16:0/3188324390 shutdown_connections 2024-09-12T02:05:31.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.052+0000 7f78c3b03740 1 --2- 172.21.15.16:0/3188324390 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b3df689e40 0x55b3df68c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:31.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.052+0000 7f78c3b03740 1 --2- 172.21.15.16:0/3188324390 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3df67f580 0x55b3df689900 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:31.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.052+0000 7f78c3b03740 1 --2- 172.21.15.16:0/3188324390 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b3df67ec60 0x55b3df67f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:31.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.052+0000 7f78c3b03740 1 -- 172.21.15.16:0/3188324390 >> 172.21.15.16:0/3188324390 conn(0x55b3df67b1f0 msgr2=0x55b3df67b5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:31.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.052+0000 7f78c3b03740 1 -- 172.21.15.16:0/3188324390 shutdown_connections 2024-09-12T02:05:31.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.052+0000 7f78c3b03740 1 -- 172.21.15.16:0/3188324390 wait complete. 2024-09-12T02:05:31.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.052+0000 7f78c3b03740 1 -- start start 2024-09-12T02:05:31.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.052+0000 7f78c3b03740 1 Processor -- start 2024-09-12T02:05:31.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.053+0000 7f78c3b03740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3df67ec60 0x55b3df686d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:31.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.053+0000 7f78c3b03740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b3df67f580 0x55b3df687260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:31.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.053+0000 7f78c3b03740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b3df689e40 0x55b3df6877a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:31.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.053+0000 7f78c3b03740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55b3df68c9b0 con 0x55b3df67ec60 2024-09-12T02:05:31.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.053+0000 7f78c3b03740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55b3df567900 con 0x55b3df67f580 2024-09-12T02:05:31.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.053+0000 7f78c3b03740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55b3df68cb30 con 0x55b3df689e40 2024-09-12T02:05:31.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.053+0000 7f78baffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3df67ec60 0x55b3df686d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:31.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.053+0000 7f78c3b03740 10 client.0 _ll_register_callbacks cb 0x55b3df565780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:05:31.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.053+0000 7f78baffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3df67ec60 0x55b3df686d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53906/0 (socket says 172.21.15.16:53906) 2024-09-12T02:05:31.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.053+0000 7f78baffd640 1 -- 172.21.15.16:0/1506281137 learned_addr learned my addr 172.21.15.16:0/1506281137 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:05:31.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.053+0000 7f78bb7fe640 1 --2- 172.21.15.16:0/1506281137 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b3df689e40 0x55b3df6877a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:31.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.053+0000 7f78b3fff640 1 --2- 172.21.15.16:0/1506281137 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b3df67f580 0x55b3df687260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:31.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.053+0000 7f78baffd640 1 -- 172.21.15.16:0/1506281137 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b3df689e40 msgr2=0x55b3df6877a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:31.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.053+0000 7f78baffd640 1 --2- 172.21.15.16:0/1506281137 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b3df689e40 0x55b3df6877a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:31.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.053+0000 7f78baffd640 1 -- 172.21.15.16:0/1506281137 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b3df67f580 msgr2=0x55b3df687260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:31.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.053+0000 7f78baffd640 1 --2- 172.21.15.16:0/1506281137 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b3df67f580 0x55b3df687260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:31.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.053+0000 7f78baffd640 1 -- 172.21.15.16:0/1506281137 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b3df5a7c50 con 0x55b3df67ec60 2024-09-12T02:05:31.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.054+0000 7f78baffd640 1 --2- 172.21.15.16:0/1506281137 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3df67ec60 0x55b3df686d20 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f78ac000c00 tx=0x7f78ac0056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9819bbd8a292bf4a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:31.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.054+0000 7f78a27fc640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:05:31.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.054+0000 7f78a27fc640 1 -- 172.21.15.16:0/1506281137 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78ac09b5c0 con 0x55b3df67ec60 2024-09-12T02:05:31.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.054+0000 7f78a27fc640 1 -- 172.21.15.16:0/1506281137 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f78ac010460 con 0x55b3df67ec60 2024-09-12T02:05:31.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.054+0000 7f78c3b03740 1 -- 172.21.15.16:0/1506281137 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f78ac09bc60 con 0x55b3df67ec60 2024-09-12T02:05:31.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.054+0000 7f78a27fc640 1 -- 172.21.15.16:0/1506281137 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f78ac0b4070 con 0x55b3df67ec60 2024-09-12T02:05:31.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.054+0000 7f78c3b03740 10 client.16431 fetch_fsmap learned FSMap version 25 2024-09-12T02:05:31.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.054+0000 7f78c3b03740 1 -- 172.21.15.16:0/1506281137 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b3df698660 con 0x55b3df67ec60 2024-09-12T02:05:31.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.054+0000 7f78a27fc640 1 -- 172.21.15.16:0/1506281137 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f78ac09bc60 con 0x55b3df67ec60 2024-09-12T02:05:31.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.054+0000 7f78a27fc640 1 -- 172.21.15.16:0/1506281137 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f78ac09b0b0 con 0x55b3df67ec60 2024-09-12T02:05:31.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.054+0000 7f78c3b03740 10 client.16431 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:05:31.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.054+0000 7f78c3b03740 10 client.16431 resolve_mds: validated gid 24482 aka daemon mds.b 2024-09-12T02:05:31.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.054+0000 7f78c3b03740 20 client.16431 populate_metadata read hostname 'smithi016' 2024-09-12T02:05:31.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.055+0000 7f78c3b03740 1 --2- 172.21.15.16:0/1506281137 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55b3df6999b0 0x55b3df69bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:31.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.055+0000 7f78c3b03740 4 client.16431 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:05:31.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.055+0000 7f78c3b03740 1 -- 172.21.15.16:0/1506281137 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b3df5ec890 con 0x55b3df6999b0 2024-09-12T02:05:31.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.055+0000 7f78b3fff640 1 --2- 172.21.15.16:0/1506281137 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55b3df6999b0 0x55b3df69bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:31.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.055+0000 7f78b3fff640 1 --2- 172.21.15.16:0/1506281137 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55b3df6999b0 0x55b3df69bda0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=99d8296f95b27214 server_cookie=cffbabd767b2589f in_seq=0 out_seq=0 2024-09-12T02:05:31.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.055+0000 7f78a27fc640 10 client.16431 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:05:31.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.056+0000 7f78a27fc640 1 -- 172.21.15.16:0/1506281137 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b3df5ec890 con 0x55b3df6999b0 2024-09-12T02:05:31.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.056+0000 7f78a27fc640 10 client.16431 handle_command_reply: tid=0 2024-09-12T02:05:31.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.056+0000 7f78c3b03740 1 client.16431 shutdown 2024-09-12T02:05:31.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.056+0000 7f78c3b03740 1 -- 172.21.15.16:0/1506281137 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3df67ec60 msgr2=0x55b3df686d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:31.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.056+0000 7f78c3b03740 1 --2- 172.21.15.16:0/1506281137 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3df67ec60 0x55b3df686d20 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f78ac000c00 tx=0x7f78ac0056d0 comp rx=0 tx=0).stop 2024-09-12T02:05:31.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.056+0000 7f78c3b03740 1 -- 172.21.15.16:0/1506281137 shutdown_connections 2024-09-12T02:05:31.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.056+0000 7f78c3b03740 1 --2- 172.21.15.16:0/1506281137 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55b3df6999b0 0x55b3df69bda0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:31.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.056+0000 7f78c3b03740 1 --2- 172.21.15.16:0/1506281137 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b3df689e40 0x55b3df6877a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:31.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.056+0000 7f78a27fc640 0 client.16431 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:05:31.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.056+0000 7f78c3b03740 1 --2- 172.21.15.16:0/1506281137 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b3df67f580 0x55b3df687260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:31.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.056+0000 7f78c3b03740 1 --2- 172.21.15.16:0/1506281137 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3df67ec60 0x55b3df686d20 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:31.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.056+0000 7f78c3b03740 1 -- 172.21.15.16:0/1506281137 >> 172.21.15.16:0/1506281137 conn(0x55b3df67b1f0 msgr2=0x55b3df680ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:31.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.056+0000 7f78c3b03740 1 -- 172.21.15.16:0/1506281137 shutdown_connections 2024-09-12T02:05:31.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.056+0000 7f78c3b03740 1 -- 172.21.15.16:0/1506281137 wait complete. 2024-09-12T02:05:31.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.057+0000 7f78c3b03740 20 client.16431 trim_cache size 0 max 16384 2024-09-12T02:05:31.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.074+0000 7f78c3b03740 1 Processor -- start 2024-09-12T02:05:31.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.074+0000 7f78c3b03740 1 -- start start 2024-09-12T02:05:31.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.074+0000 7f78c3b03740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b3df67ec60 0x55b3df671d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:31.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.074+0000 7f78c3b03740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b3df67f580 0x55b3df675db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:31.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.074+0000 7f78c3b03740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3df689e40 0x55b3df6a14c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:31.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.074+0000 7f78c3b03740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f78bc1adb10 con 0x55b3df689e40 2024-09-12T02:05:31.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.074+0000 7f78c3b03740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f78bc1ab3e0 con 0x55b3df67ec60 2024-09-12T02:05:31.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.074+0000 7f78c3b03740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f78bc1a9250 con 0x55b3df67f580 2024-09-12T02:05:31.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.074+0000 7f78bb7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3df689e40 0x55b3df6a14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:31.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.074+0000 7f78bb7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3df689e40 0x55b3df6a14c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53908/0 (socket says 172.21.15.16:53908) 2024-09-12T02:05:31.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.074+0000 7f78bb7fe640 1 -- 172.21.15.16:0/3280340206 learned_addr learned my addr 172.21.15.16:0/3280340206 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:05:31.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.074+0000 7f78baffd640 1 --2- 172.21.15.16:0/3280340206 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b3df67ec60 0x55b3df671d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:31.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.074+0000 7f78b3fff640 1 --2- 172.21.15.16:0/3280340206 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b3df67f580 0x55b3df675db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:31.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.075+0000 7f78bb7fe640 1 -- 172.21.15.16:0/3280340206 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b3df67f580 msgr2=0x55b3df675db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:31.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.075+0000 7f78bb7fe640 1 --2- 172.21.15.16:0/3280340206 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b3df67f580 0x55b3df675db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:31.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.075+0000 7f78bb7fe640 1 -- 172.21.15.16:0/3280340206 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b3df67ec60 msgr2=0x55b3df671d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:31.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.075+0000 7f78bb7fe640 1 --2- 172.21.15.16:0/3280340206 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b3df67ec60 0x55b3df671d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:31.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.075+0000 7f78bb7fe640 1 -- 172.21.15.16:0/3280340206 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b3df685cd0 con 0x55b3df689e40 2024-09-12T02:05:31.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.075+0000 7f78b3fff640 1 --2- 172.21.15.16:0/3280340206 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b3df67f580 0x55b3df675db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:05:31.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.075+0000 7f78baffd640 1 --2- 172.21.15.16:0/3280340206 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b3df67ec60 0x55b3df671d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:05:31.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.075+0000 7f78bb7fe640 1 --2- 172.21.15.16:0/3280340206 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3df689e40 0x55b3df6a14c0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f78b4009ca0 tx=0x7f78b4004a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a223f38744052d87 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:31.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.075+0000 7f78b0ff9640 1 -- 172.21.15.16:0/3280340206 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78b4043050 con 0x55b3df689e40 2024-09-12T02:05:31.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.075+0000 7f78b0ff9640 1 -- 172.21.15.16:0/3280340206 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f78b4044040 con 0x55b3df689e40 2024-09-12T02:05:31.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.075+0000 7f78b0ff9640 1 -- 172.21.15.16:0/3280340206 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78b4034040 con 0x55b3df689e40 2024-09-12T02:05:31.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.076+0000 7f78c3b03740 1 -- 172.21.15.16:0/3280340206 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3df689e40 msgr2=0x55b3df6a14c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:31.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.076+0000 7f78c3b03740 1 --2- 172.21.15.16:0/3280340206 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3df689e40 0x55b3df6a14c0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f78b4009ca0 tx=0x7f78b4004a60 comp rx=0 tx=0).stop 2024-09-12T02:05:31.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.076+0000 7f78c3b03740 1 -- 172.21.15.16:0/3280340206 shutdown_connections 2024-09-12T02:05:31.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.076+0000 7f78c3b03740 1 --2- 172.21.15.16:0/3280340206 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3df689e40 0x55b3df6a14c0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:31.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.076+0000 7f78c3b03740 1 --2- 172.21.15.16:0/3280340206 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b3df67f580 0x55b3df675db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:31.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.076+0000 7f78c3b03740 1 --2- 172.21.15.16:0/3280340206 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b3df67ec60 0x55b3df671d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:31.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.076+0000 7f78c3b03740 1 -- 172.21.15.16:0/3280340206 >> 172.21.15.16:0/3280340206 conn(0x55b3df67b1f0 msgr2=0x55b3df6756a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:31.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.076+0000 7f78c3b03740 1 -- 172.21.15.16:0/3280340206 shutdown_connections 2024-09-12T02:05:31.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.076+0000 7f78c3b03740 1 -- 172.21.15.16:0/3280340206 wait complete. 2024-09-12T02:05:31.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.076+0000 7f78c3b03740 1 -- start start 2024-09-12T02:05:31.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.076+0000 7f78c3b03740 1 Processor -- start 2024-09-12T02:05:31.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.077+0000 7f78c3b03740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3df67ec60 0x55b3df692520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:31.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.077+0000 7f78c3b03740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b3df67f580 0x55b3df683a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:31.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.077+0000 7f78c3b03740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b3df689e40 0x55b3df683f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:31.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.077+0000 7f78c3b03740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f78bc1a98a0 con 0x55b3df67ec60 2024-09-12T02:05:31.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.077+0000 7f78c3b03740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f78bc1aad10 con 0x55b3df67f580 2024-09-12T02:05:31.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.077+0000 7f78c3b03740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f78bc1a9f70 con 0x55b3df689e40 2024-09-12T02:05:31.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.077+0000 7f78c3b03740 10 client.0 _ll_register_callbacks cb 0x7f78ac005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:05:31.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.077+0000 7f78baffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3df67ec60 0x55b3df692520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:31.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.077+0000 7f78b3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b3df67f580 0x55b3df683a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:31.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.077+0000 7f78baffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3df67ec60 0x55b3df692520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53910/0 (socket says 172.21.15.16:53910) 2024-09-12T02:05:31.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.077+0000 7f78baffd640 1 -- 172.21.15.16:0/1568349698 learned_addr learned my addr 172.21.15.16:0/1568349698 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:05:31.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.077+0000 7f78bb7fe640 1 --2- 172.21.15.16:0/1568349698 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b3df689e40 0x55b3df683f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:31.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.077+0000 7f78b3fff640 1 -- 172.21.15.16:0/1568349698 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b3df689e40 msgr2=0x55b3df683f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:31.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.077+0000 7f78b3fff640 1 --2- 172.21.15.16:0/1568349698 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b3df689e40 0x55b3df683f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:31.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.077+0000 7f78b3fff640 1 -- 172.21.15.16:0/1568349698 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3df67ec60 msgr2=0x55b3df692520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:31.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.077+0000 7f78b3fff640 1 --2- 172.21.15.16:0/1568349698 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3df67ec60 0x55b3df692520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:31.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.077+0000 7f78b3fff640 1 -- 172.21.15.16:0/1568349698 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b3df6847c0 con 0x55b3df67f580 2024-09-12T02:05:31.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.077+0000 7f78baffd640 1 --2- 172.21.15.16:0/1568349698 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3df67ec60 0x55b3df692520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:05:31.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.077+0000 7f78b3fff640 1 --2- 172.21.15.16:0/1568349698 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b3df67f580 0x55b3df683a50 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x55b3df69c460 tx=0x7f78a4010be0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=79225c3e1924286b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:31.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.077+0000 7f78a27fc640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-12T02:05:31.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.077+0000 7f78a27fc640 1 -- 172.21.15.16:0/1568349698 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78a4004460 con 0x55b3df67f580 2024-09-12T02:05:31.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.078+0000 7f78c3b03740 1 -- 172.21.15.16:0/1568349698 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f78b4044040 con 0x55b3df67f580 2024-09-12T02:05:31.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.078+0000 7f78a27fc640 1 -- 172.21.15.16:0/1568349698 <== mon.2 v2:172.21.15.33:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f78a400f8f0 con 0x55b3df67f580 2024-09-12T02:05:31.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.078+0000 7f78a27fc640 1 -- 172.21.15.16:0/1568349698 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f78a402e050 con 0x55b3df67f580 2024-09-12T02:05:31.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.078+0000 7f78c3b03740 10 client.25982 fetch_fsmap learned FSMap version 25 2024-09-12T02:05:31.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.078+0000 7f78c3b03740 1 -- 172.21.15.16:0/1568349698 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b3df7044f0 con 0x55b3df67f580 2024-09-12T02:05:31.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.078+0000 7f78a27fc640 1 -- 172.21.15.16:0/1568349698 <== mon.2 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f78b4044040 con 0x55b3df67f580 2024-09-12T02:05:31.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.078+0000 7f78a27fc640 1 -- 172.21.15.16:0/1568349698 <== mon.2 v2:172.21.15.33:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f78a401e3f0 con 0x55b3df67f580 2024-09-12T02:05:31.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.078+0000 7f78c3b03740 10 client.25982 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:05:31.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.078+0000 7f78c3b03740 10 client.25982 resolve_mds: validated gid 24482 aka daemon mds.b 2024-09-12T02:05:31.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.078+0000 7f78c3b03740 20 client.25982 populate_metadata read hostname 'smithi016' 2024-09-12T02:05:31.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.078+0000 7f78c3b03740 1 --2- 172.21.15.16:0/1568349698 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55b3df6999b0 0x55b3df706c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:31.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.078+0000 7f78c3b03740 4 client.25982 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.15, "await_for": 45.0}] 2024-09-12T02:05:31.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.078+0000 7f78c3b03740 1 -- 172.21.15.16:0/1568349698 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.15, "await_for": 45.0}) -- 0x55b3df5e6be0 con 0x55b3df6999b0 2024-09-12T02:05:31.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.078+0000 7f78baffd640 1 --2- 172.21.15.16:0/1568349698 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55b3df6999b0 0x55b3df706c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:31.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.079+0000 7f78baffd640 1 --2- 172.21.15.16:0/1568349698 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55b3df6999b0 0x55b3df706c40 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c7c6a5df6650ae84 server_cookie=8b938774d6f86189 in_seq=0 out_seq=0 2024-09-12T02:05:31.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.079+0000 7f78a27fc640 10 client.25982 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:05:31.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.089+0000 7f78a27fc640 1 -- 172.21.15.16:0/1568349698 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55b3df5e6be0 con 0x55b3df6999b0 2024-09-12T02:05:31.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.089+0000 7f78a27fc640 10 client.25982 handle_command_reply: tid=0 2024-09-12T02:05:31.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.089+0000 7f78c3b03740 1 client.25982 shutdown 2024-09-12T02:05:31.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.089+0000 7f78c3b03740 1 -- 172.21.15.16:0/1568349698 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b3df67f580 msgr2=0x55b3df683a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:31.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.089+0000 7f78c3b03740 1 --2- 172.21.15.16:0/1568349698 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b3df67f580 0x55b3df683a50 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x55b3df69c460 tx=0x7f78a4010be0 comp rx=0 tx=0).stop 2024-09-12T02:05:31.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.089+0000 7f78c3b03740 1 -- 172.21.15.16:0/1568349698 shutdown_connections 2024-09-12T02:05:31.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.089+0000 7f78c3b03740 1 --2- 172.21.15.16:0/1568349698 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55b3df6999b0 0x55b3df706c40 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:31.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.089+0000 7f78c3b03740 1 --2- 172.21.15.16:0/1568349698 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b3df689e40 0x55b3df683f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:31.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.089+0000 7f78c3b03740 1 --2- 172.21.15.16:0/1568349698 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b3df67f580 0x55b3df683a50 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:31.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.089+0000 7f78a27fc640 0 client.25982 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:05:31.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.089+0000 7f78c3b03740 1 --2- 172.21.15.16:0/1568349698 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3df67ec60 0x55b3df692520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:31.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.089+0000 7f78c3b03740 1 -- 172.21.15.16:0/1568349698 >> 172.21.15.16:0/1568349698 conn(0x55b3df67b1f0 msgr2=0x55b3df675ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:31.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.089+0000 7f78c3b03740 1 -- 172.21.15.16:0/1568349698 shutdown_connections 2024-09-12T02:05:31.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.089+0000 7f78c3b03740 1 -- 172.21.15.16:0/1568349698 wait complete. 2024-09-12T02:05:31.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.090+0000 7f78c3b03740 20 client.25982 trim_cache size 0 max 16384 2024-09-12T02:05:31.091 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T02:05:31.091 INFO:teuthology.orchestra.run.smithi016.stdout: "epoch": 25, 2024-09-12T02:05:31.091 INFO:teuthology.orchestra.run.smithi016.stdout: "leader": 24482, 2024-09-12T02:05:31.091 INFO:teuthology.orchestra.run.smithi016.stdout: "set_version": 14, 2024-09-12T02:05:31.091 INFO:teuthology.orchestra.run.smithi016.stdout: "sets": { 2024-09-12T02:05:31.091 INFO:teuthology.orchestra.run.smithi016.stdout: "448a5fd6": { 2024-09-12T02:05:31.092 INFO:teuthology.orchestra.run.smithi016.stdout: "version": 14, 2024-09-12T02:05:31.092 INFO:teuthology.orchestra.run.smithi016.stdout: "age_ref": 0.0, 2024-09-12T02:05:31.092 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-12T02:05:31.092 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "QUIESCED", 2024-09-12T02:05:31.092 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-12T02:05:31.092 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:05:31.092 INFO:teuthology.orchestra.run.smithi016.stdout: "timeout": 90.0, 2024-09-12T02:05:31.092 INFO:teuthology.orchestra.run.smithi016.stdout: "expiration": 140.2, 2024-09-12T02:05:31.092 INFO:teuthology.orchestra.run.smithi016.stdout: "members": { 2024-09-12T02:05:31.092 INFO:teuthology.orchestra.run.smithi016.stdout: "file:/": { 2024-09-12T02:05:31.092 INFO:teuthology.orchestra.run.smithi016.stdout: "excluded": false, 2024-09-12T02:05:31.092 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-12T02:05:31.092 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "QUIESCED", 2024-09-12T02:05:31.093 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-12T02:05:31.093 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:05:31.093 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:05:31.093 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:05:31.093 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:05:31.093 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:05:31.093 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T02:05:31.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.090+0000 7f78c172a640 1 -- 172.21.15.16:0/3838656520 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f78a80a2b80 msgr2=0x7f78a80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:31.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.090+0000 7f78c172a640 1 --2- 172.21.15.16:0/3838656520 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f78a80a2b80 0x7f78a80a5040 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f78b4004ad0 tx=0x7f78b402a950 comp rx=0 tx=0).stop 2024-09-12T02:05:31.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.090+0000 7f78c172a640 1 -- 172.21.15.16:0/3838656520 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78bc102d40 msgr2=0x7f78bc1983d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:31.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.090+0000 7f78c172a640 1 --2- 172.21.15.16:0/3838656520 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78bc102d40 0x7f78bc1983d0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f78ac00e9f0 tx=0x7f78ac00eeb0 comp rx=0 tx=0).stop 2024-09-12T02:05:31.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.090+0000 7f78c172a640 1 -- 172.21.15.16:0/3838656520 shutdown_connections 2024-09-12T02:05:31.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.090+0000 7f78c172a640 1 --2- 172.21.15.16:0/3838656520 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f78a80a2b80 0x7f78a80a5040 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:31.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.090+0000 7f78c172a640 1 --2- 172.21.15.16:0/3838656520 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78bc10c170 0x7f78bc19cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:31.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.090+0000 7f78c172a640 1 --2- 172.21.15.16:0/3838656520 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f78bc103660 0x7f78bc198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:31.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.090+0000 7f78c172a640 1 --2- 172.21.15.16:0/3838656520 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78bc102d40 0x7f78bc1983d0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:31.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.090+0000 7f78c172a640 1 -- 172.21.15.16:0/3838656520 >> 172.21.15.16:0/3838656520 conn(0x7f78bc0fc7a0 msgr2=0x7f78bc0fd400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:31.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.090+0000 7f78c172a640 1 -- 172.21.15.16:0/3838656520 shutdown_connections 2024-09-12T02:05:31.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:31.090+0000 7f78c172a640 1 -- 172.21.15.16:0/3838656520 wait complete. 2024-09-12T02:05:31.880 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '448a5fd6', quiesce took 3.3 seconds. Will release after: 16.849999999999998 2024-09-12T02:05:32.183 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:05:31 smithi016 ceph-mon[30947]: pgmap v443: 97 pgs: 97 active+clean; 2.1 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-09-12T02:05:32.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:05:31 smithi184 ceph-mon[35593]: pgmap v443: 97 pgs: 97 active+clean; 2.1 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-09-12T02:05:32.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:05:31 smithi033 ceph-mon[35332]: pgmap v443: 97 pgs: 97 active+clean; 2.1 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-09-12T02:05:34.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:05:33 smithi184 ceph-mon[35593]: pgmap v444: 97 pgs: 97 active+clean; 2.1 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-12T02:05:34.418 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:05:33 smithi033 ceph-mon[35332]: pgmap v444: 97 pgs: 97 active+clean; 2.1 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-12T02:05:34.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:05:33 smithi016 ceph-mon[30947]: pgmap v444: 97 pgs: 97 active+clean; 2.1 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-12T02:05:35.484 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T02:05:35.756 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:05:36.177 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:05:35 smithi016 ceph-mon[30947]: pgmap v445: 97 pgs: 97 active+clean; 2.1 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 KiB/s wr, 14 op/s 2024-09-12T02:05:36.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.261+0000 7f6a10aae640 1 -- 172.21.15.16:0/614463429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a0c10c170 msgr2=0x7f6a0c10e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:36.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.261+0000 7f6a10aae640 1 --2- 172.21.15.16:0/614463429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a0c10c170 0x7f6a0c10e610 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f6a04009ca0 tx=0x7f6a0402a950 comp rx=0 tx=0).stop 2024-09-12T02:05:36.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.261+0000 7f6a10aae640 1 -- 172.21.15.16:0/614463429 shutdown_connections 2024-09-12T02:05:36.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.261+0000 7f6a10aae640 1 --2- 172.21.15.16:0/614463429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a0c10c170 0x7f6a0c10e610 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:36.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.261+0000 7f6a10aae640 1 --2- 172.21.15.16:0/614463429 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6a0c103660 0x7f6a0c10bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:36.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.261+0000 7f6a10aae640 1 --2- 172.21.15.16:0/614463429 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a0c102d40 0x7f6a0c103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:36.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.261+0000 7f6a10aae640 1 -- 172.21.15.16:0/614463429 >> 172.21.15.16:0/614463429 conn(0x7f6a0c0fc7a0 msgr2=0x7f6a0c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:36.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.261+0000 7f6a10aae640 1 -- 172.21.15.16:0/614463429 shutdown_connections 2024-09-12T02:05:36.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.261+0000 7f6a10aae640 1 -- 172.21.15.16:0/614463429 wait complete. 2024-09-12T02:05:36.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.262+0000 7f6a10aae640 1 Processor -- start 2024-09-12T02:05:36.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.262+0000 7f6a10aae640 1 -- start start 2024-09-12T02:05:36.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.262+0000 7f6a10aae640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a0c102d40 0x7f6a0c198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:36.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.263+0000 7f6a10aae640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6a0c103660 0x7f6a0c198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:36.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.263+0000 7f6a10aae640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a0c10c170 0x7f6a0c19cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:36.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.263+0000 7f6a10aae640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6a0c111f30 con 0x7f6a0c10c170 2024-09-12T02:05:36.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.263+0000 7f6a10aae640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6a0c111db0 con 0x7f6a0c102d40 2024-09-12T02:05:36.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.263+0000 7f6a10aae640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f6a0c1120b0 con 0x7f6a0c103660 2024-09-12T02:05:36.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.263+0000 7f6a0ad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a0c10c170 0x7f6a0c19cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:36.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.263+0000 7f6a0ad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a0c10c170 0x7f6a0c19cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54714/0 (socket says 172.21.15.16:54714) 2024-09-12T02:05:36.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.263+0000 7f6a0ad9d640 1 -- 172.21.15.16:0/958125661 learned_addr learned my addr 172.21.15.16:0/958125661 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:05:36.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.263+0000 7f6a09d9b640 1 --2- 172.21.15.16:0/958125661 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6a0c103660 0x7f6a0c198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:36.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.263+0000 7f6a0a59c640 1 --2- 172.21.15.16:0/958125661 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a0c102d40 0x7f6a0c198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:36.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.263+0000 7f6a0ad9d640 1 -- 172.21.15.16:0/958125661 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6a0c103660 msgr2=0x7f6a0c198980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:36.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.263+0000 7f6a0ad9d640 1 --2- 172.21.15.16:0/958125661 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6a0c103660 0x7f6a0c198980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:36.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.263+0000 7f6a0ad9d640 1 -- 172.21.15.16:0/958125661 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a0c102d40 msgr2=0x7f6a0c198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:36.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.263+0000 7f6a0ad9d640 1 --2- 172.21.15.16:0/958125661 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a0c102d40 0x7f6a0c198440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:36.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.263+0000 7f6a0ad9d640 1 -- 172.21.15.16:0/958125661 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a0c19d3f0 con 0x7f6a0c10c170 2024-09-12T02:05:36.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.263+0000 7f6a0ad9d640 1 --2- 172.21.15.16:0/958125661 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a0c10c170 0x7f6a0c19cd10 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f6a04004af0 tx=0x7f6a0402f3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:36.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.263+0000 7f69fb7fe640 1 -- 172.21.15.16:0/958125661 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a04034040 con 0x7f6a0c10c170 2024-09-12T02:05:36.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.264+0000 7f6a10aae640 1 -- 172.21.15.16:0/958125661 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a0c19d680 con 0x7f6a0c10c170 2024-09-12T02:05:36.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.264+0000 7f69fb7fe640 1 -- 172.21.15.16:0/958125661 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a04030920 con 0x7f6a0c10c170 2024-09-12T02:05:36.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.264+0000 7f69fb7fe640 1 -- 172.21.15.16:0/958125661 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a04004670 con 0x7f6a0c10c170 2024-09-12T02:05:36.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.264+0000 7f6a10aae640 1 -- 172.21.15.16:0/958125661 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a0c1a4fc0 con 0x7f6a0c10c170 2024-09-12T02:05:36.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.266+0000 7f69fb7fe640 1 -- 172.21.15.16:0/958125661 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6a04037020 con 0x7f6a0c10c170 2024-09-12T02:05:36.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.266+0000 7f6a12e87740 1 Processor -- start 2024-09-12T02:05:36.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.267+0000 7f6a12e87740 1 -- start start 2024-09-12T02:05:36.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.267+0000 7f6a12e87740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557c05f454a0 0x557c05f45880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:36.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.267+0000 7f6a12e87740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557c05f45dc0 0x557c05f50140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:36.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.267+0000 7f6a12e87740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c05f50680 0x557c05f52a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:36.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.267+0000 7f6a12e87740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x557c05eab2e0 con 0x557c05f50680 2024-09-12T02:05:36.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.267+0000 7f6a12e87740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x557c05e52ce0 con 0x557c05f454a0 2024-09-12T02:05:36.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.267+0000 7f6a12e87740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x557c05ea72a0 con 0x557c05f45dc0 2024-09-12T02:05:36.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.267+0000 7f6a09d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c05f50680 0x557c05f52a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:36.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.267+0000 7f6a09d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c05f50680 0x557c05f52a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54716/0 (socket says 172.21.15.16:54716) 2024-09-12T02:05:36.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.267+0000 7f6a09d9b640 1 -- 172.21.15.16:0/1739735462 learned_addr learned my addr 172.21.15.16:0/1739735462 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:05:36.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.267+0000 7f69fb7fe640 1 --2- 172.21.15.16:0/958125661 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f69f40a2b80 0x7f69f40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:36.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.267+0000 7f6a09d9b640 1 --2- 172.21.15.16:0/1739735462 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557c05f454a0 0x557c05f45880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:36.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.267+0000 7f6a0a59c640 1 --2- 172.21.15.16:0/1739735462 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557c05f45dc0 0x557c05f50140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:36.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.267+0000 7f6a09d9b640 1 -- 172.21.15.16:0/1739735462 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557c05f45dc0 msgr2=0x557c05f50140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:36.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.267+0000 7f6a09d9b640 1 --2- 172.21.15.16:0/1739735462 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557c05f45dc0 0x557c05f50140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:36.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.267+0000 7f6a09d9b640 1 -- 172.21.15.16:0/1739735462 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557c05f454a0 msgr2=0x557c05f45880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:36.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.267+0000 7f6a09d9b640 1 --2- 172.21.15.16:0/1739735462 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557c05f454a0 0x557c05f45880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:36.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.267+0000 7f69fb7fe640 1 -- 172.21.15.16:0/958125661 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f6a040efc40 con 0x7f6a0c10c170 2024-09-12T02:05:36.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.267+0000 7f6a0ad9d640 1 --2- 172.21.15.16:0/958125661 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f69f40a2b80 0x7f69f40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:36.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.267+0000 7f6a09d9b640 1 -- 172.21.15.16:0/1739735462 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c05e37440 con 0x557c05f50680 2024-09-12T02:05:36.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.268+0000 7f6a09d9b640 1 --2- 172.21.15.16:0/1739735462 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c05f50680 0x557c05f52a70 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f6a00011770 tx=0x7f6a00004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=651e44ba37ecbbbb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:36.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.268+0000 7f69f8ff9640 1 -- 172.21.15.16:0/1739735462 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a00004050 con 0x557c05f50680 2024-09-12T02:05:36.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.268+0000 7f6a0ad9d640 1 --2- 172.21.15.16:0/958125661 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f69f40a2b80 0x7f69f40a5040 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f6a0402fde0 tx=0x7f6a0402a950 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:36.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.268+0000 7f69f8ff9640 1 -- 172.21.15.16:0/1739735462 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a0000fca0 con 0x557c05f50680 2024-09-12T02:05:36.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.268+0000 7f69f8ff9640 1 -- 172.21.15.16:0/1739735462 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a00003a30 con 0x557c05f50680 2024-09-12T02:05:36.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.268+0000 7f6a12e87740 1 -- 172.21.15.16:0/1739735462 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c05f50680 msgr2=0x557c05f52a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:36.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.268+0000 7f6a12e87740 1 --2- 172.21.15.16:0/1739735462 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c05f50680 0x557c05f52a70 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f6a00011770 tx=0x7f6a00004730 comp rx=0 tx=0).stop 2024-09-12T02:05:36.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.269+0000 7f6a12e87740 1 -- 172.21.15.16:0/1739735462 shutdown_connections 2024-09-12T02:05:36.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.269+0000 7f6a12e87740 1 --2- 172.21.15.16:0/1739735462 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c05f50680 0x557c05f52a70 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:36.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.269+0000 7f6a12e87740 1 --2- 172.21.15.16:0/1739735462 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557c05f45dc0 0x557c05f50140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:36.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.269+0000 7f6a12e87740 1 --2- 172.21.15.16:0/1739735462 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557c05f454a0 0x557c05f45880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:36.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.269+0000 7f6a12e87740 1 -- 172.21.15.16:0/1739735462 >> 172.21.15.16:0/1739735462 conn(0x557c05f41a30 msgr2=0x557c05f41e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:36.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.269+0000 7f6a12e87740 1 -- 172.21.15.16:0/1739735462 shutdown_connections 2024-09-12T02:05:36.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.269+0000 7f6a12e87740 1 -- 172.21.15.16:0/1739735462 wait complete. 2024-09-12T02:05:36.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.269+0000 7f6a12e87740 1 -- start start 2024-09-12T02:05:36.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.269+0000 7f6a12e87740 1 Processor -- start 2024-09-12T02:05:36.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.269+0000 7f6a12e87740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557c05f454a0 0x557c05f50010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:36.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.269+0000 7f6a12e87740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c05f45dc0 0x557c05f4d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:36.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.269+0000 7f6a12e87740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557c05f50680 0x557c05f4d840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:36.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.269+0000 7f6a12e87740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x557c05f53250 con 0x557c05f45dc0 2024-09-12T02:05:36.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.269+0000 7f6a12e87740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x557c05e66050 con 0x557c05f454a0 2024-09-12T02:05:36.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.269+0000 7f6a12e87740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x557c05f533d0 con 0x557c05f50680 2024-09-12T02:05:36.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.269+0000 7f6a12e87740 10 client.0 _ll_register_callbacks cb 0x557c05ea6160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:05:36.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.270+0000 7f6a0a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c05f45dc0 0x557c05f4d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:36.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.270+0000 7f6a0a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c05f45dc0 0x557c05f4d2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54732/0 (socket says 172.21.15.16:54732) 2024-09-12T02:05:36.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.270+0000 7f6a0a59c640 1 -- 172.21.15.16:0/3838464554 learned_addr learned my addr 172.21.15.16:0/3838464554 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:05:36.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.270+0000 7f6a09d9b640 1 --2- 172.21.15.16:0/3838464554 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557c05f454a0 0x557c05f50010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:36.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.270+0000 7f6a09d9b640 1 --2- 172.21.15.16:0/3838464554 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557c05f50680 0x557c05f4d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:36.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.270+0000 7f6a0a59c640 1 -- 172.21.15.16:0/3838464554 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557c05f50680 msgr2=0x557c05f4d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:36.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.270+0000 7f6a0a59c640 1 --2- 172.21.15.16:0/3838464554 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557c05f50680 0x557c05f4d840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:36.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.270+0000 7f6a0a59c640 1 -- 172.21.15.16:0/3838464554 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557c05f454a0 msgr2=0x557c05f50010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:36.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.270+0000 7f6a0a59c640 1 --2- 172.21.15.16:0/3838464554 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557c05f454a0 0x557c05f50010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:36.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.270+0000 7f6a0a59c640 1 -- 172.21.15.16:0/3838464554 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c05c1c3c0 con 0x557c05f45dc0 2024-09-12T02:05:36.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.270+0000 7f6a09d9b640 1 --2- 172.21.15.16:0/3838464554 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557c05f454a0 0x557c05f50010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:05:36.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.270+0000 7f6a0a59c640 1 --2- 172.21.15.16:0/3838464554 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c05f45dc0 0x557c05f4d2c0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f69fc012740 tx=0x7f69fc012c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbea8cf857f2b747 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:36.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.270+0000 7f69f1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:05:36.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.270+0000 7f69f1ffb640 1 -- 172.21.15.16:0/3838464554 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69fc00f920 con 0x557c05f45dc0 2024-09-12T02:05:36.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.270+0000 7f69f1ffb640 1 -- 172.21.15.16:0/3838464554 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f69fc00fac0 con 0x557c05f45dc0 2024-09-12T02:05:36.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.270+0000 7f6a12e87740 1 -- 172.21.15.16:0/3838464554 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a0000fca0 con 0x557c05f45dc0 2024-09-12T02:05:36.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.270+0000 7f69f1ffb640 1 -- 172.21.15.16:0/3838464554 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f69fc00fcc0 con 0x557c05f45dc0 2024-09-12T02:05:36.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.270+0000 7f6a12e87740 10 client.16449 fetch_fsmap learned FSMap version 25 2024-09-12T02:05:36.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.270+0000 7f6a12e87740 1 -- 172.21.15.16:0/3838464554 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x557c05f5eca0 con 0x557c05f45dc0 2024-09-12T02:05:36.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.271+0000 7f69f1ffb640 1 -- 172.21.15.16:0/3838464554 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a0000fca0 con 0x557c05f45dc0 2024-09-12T02:05:36.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.271+0000 7f69f1ffb640 1 -- 172.21.15.16:0/3838464554 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f69fc01c420 con 0x557c05f45dc0 2024-09-12T02:05:36.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.271+0000 7f6a12e87740 10 client.16449 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:05:36.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.271+0000 7f6a12e87740 10 client.16449 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:05:36.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.271+0000 7f6a12e87740 20 client.16449 populate_metadata read hostname 'smithi016' 2024-09-12T02:05:36.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.271+0000 7f6a12e87740 1 --2- 172.21.15.16:0/3838464554 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x557c05f60150 0x557c05f62590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:36.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.271+0000 7f6a12e87740 4 client.16449 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:05:36.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.271+0000 7f6a12e87740 1 -- 172.21.15.16:0/3838464554 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557c05eb8850 con 0x557c05f60150 2024-09-12T02:05:36.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.271+0000 7f6a09d9b640 1 --2- 172.21.15.16:0/3838464554 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x557c05f60150 0x557c05f62590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:36.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.272+0000 7f6a09d9b640 1 --2- 172.21.15.16:0/3838464554 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x557c05f60150 0x557c05f62590 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6bfe3725cdb5d85b server_cookie=dfad3a1f4cd46188 in_seq=0 out_seq=0 2024-09-12T02:05:36.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.272+0000 7f69f1ffb640 10 client.16449 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:05:36.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.272+0000 7f69f1ffb640 1 -- 172.21.15.16:0/3838464554 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557c05eb8850 con 0x557c05f60150 2024-09-12T02:05:36.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.272+0000 7f69f1ffb640 10 client.16449 handle_command_reply: tid=0 2024-09-12T02:05:36.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.272+0000 7f6a12e87740 1 client.16449 shutdown 2024-09-12T02:05:36.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.272+0000 7f6a12e87740 1 -- 172.21.15.16:0/3838464554 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c05f45dc0 msgr2=0x557c05f4d2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:36.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.273+0000 7f6a12e87740 1 --2- 172.21.15.16:0/3838464554 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c05f45dc0 0x557c05f4d2c0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f69fc012740 tx=0x7f69fc012c00 comp rx=0 tx=0).stop 2024-09-12T02:05:36.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.273+0000 7f6a12e87740 1 -- 172.21.15.16:0/3838464554 shutdown_connections 2024-09-12T02:05:36.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.273+0000 7f6a12e87740 1 --2- 172.21.15.16:0/3838464554 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x557c05f60150 0x557c05f62590 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:36.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.273+0000 7f6a12e87740 1 --2- 172.21.15.16:0/3838464554 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557c05f50680 0x557c05f4d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:36.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.273+0000 7f6a12e87740 1 --2- 172.21.15.16:0/3838464554 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c05f45dc0 0x557c05f4d2c0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:36.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.273+0000 7f69f1ffb640 0 client.16449 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:05:36.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.273+0000 7f6a12e87740 1 --2- 172.21.15.16:0/3838464554 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557c05f454a0 0x557c05f50010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:36.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.273+0000 7f6a12e87740 1 -- 172.21.15.16:0/3838464554 >> 172.21.15.16:0/3838464554 conn(0x557c05f41a30 msgr2=0x557c05f47710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:36.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.273+0000 7f6a12e87740 1 -- 172.21.15.16:0/3838464554 shutdown_connections 2024-09-12T02:05:36.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.273+0000 7f6a12e87740 1 -- 172.21.15.16:0/3838464554 wait complete. 2024-09-12T02:05:36.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.273+0000 7f6a12e87740 20 client.16449 trim_cache size 0 max 16384 2024-09-12T02:05:36.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.290+0000 7f6a12e87740 1 Processor -- start 2024-09-12T02:05:36.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.290+0000 7f6a12e87740 1 -- start start 2024-09-12T02:05:36.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.290+0000 7f6a12e87740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557c05f454a0 0x557c05f5c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:36.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.290+0000 7f6a12e87740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557c05f45dc0 0x557c05f67e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:36.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.290+0000 7f6a12e87740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c05f50680 0x557c05f683a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:36.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.290+0000 7f6a12e87740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6a0c1afc80 con 0x557c05f50680 2024-09-12T02:05:36.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.290+0000 7f6a12e87740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6a0c1ad550 con 0x557c05f454a0 2024-09-12T02:05:36.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.290+0000 7f6a12e87740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f6a0c1ab3e0 con 0x557c05f45dc0 2024-09-12T02:05:36.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.290+0000 7f6a09d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c05f50680 0x557c05f683a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:36.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.290+0000 7f6a09d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c05f50680 0x557c05f683a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54734/0 (socket says 172.21.15.16:54734) 2024-09-12T02:05:36.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.290+0000 7f6a09d9b640 1 -- 172.21.15.16:0/3753212950 learned_addr learned my addr 172.21.15.16:0/3753212950 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:05:36.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.290+0000 7f6a0a59c640 1 --2- 172.21.15.16:0/3753212950 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557c05f45dc0 0x557c05f67e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:36.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.290+0000 7f6a09d9b640 1 --2- 172.21.15.16:0/3753212950 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557c05f454a0 0x557c05f5c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:36.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.291+0000 7f6a09d9b640 1 -- 172.21.15.16:0/3753212950 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557c05f45dc0 msgr2=0x557c05f67e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:36.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.291+0000 7f6a09d9b640 1 --2- 172.21.15.16:0/3753212950 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557c05f45dc0 0x557c05f67e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:36.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.291+0000 7f6a09d9b640 1 -- 172.21.15.16:0/3753212950 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557c05f454a0 msgr2=0x557c05f5c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:36.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.291+0000 7f6a09d9b640 1 --2- 172.21.15.16:0/3753212950 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557c05f454a0 0x557c05f5c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:36.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.291+0000 7f6a09d9b640 1 -- 172.21.15.16:0/3753212950 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c05f45bf0 con 0x557c05f50680 2024-09-12T02:05:36.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.291+0000 7f6a09d9b640 1 --2- 172.21.15.16:0/3753212950 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c05f50680 0x557c05f683a0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f6a0000ff90 tx=0x7f6a00003690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6432bf4eeecd226 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:36.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.291+0000 7f69f3fff640 1 -- 172.21.15.16:0/3753212950 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a0000fa90 con 0x557c05f50680 2024-09-12T02:05:36.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.291+0000 7f69f3fff640 1 -- 172.21.15.16:0/3753212950 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a0000fc30 con 0x557c05f50680 2024-09-12T02:05:36.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.291+0000 7f69f3fff640 1 -- 172.21.15.16:0/3753212950 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a00004de0 con 0x557c05f50680 2024-09-12T02:05:36.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.291+0000 7f6a12e87740 1 -- 172.21.15.16:0/3753212950 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c05f50680 msgr2=0x557c05f683a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:36.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.291+0000 7f6a12e87740 1 --2- 172.21.15.16:0/3753212950 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c05f50680 0x557c05f683a0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f6a0000ff90 tx=0x7f6a00003690 comp rx=0 tx=0).stop 2024-09-12T02:05:36.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.292+0000 7f6a12e87740 1 -- 172.21.15.16:0/3753212950 shutdown_connections 2024-09-12T02:05:36.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.292+0000 7f6a12e87740 1 --2- 172.21.15.16:0/3753212950 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c05f50680 0x557c05f683a0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:36.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.292+0000 7f6a12e87740 1 --2- 172.21.15.16:0/3753212950 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557c05f45dc0 0x557c05f67e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:36.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.292+0000 7f6a12e87740 1 --2- 172.21.15.16:0/3753212950 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557c05f454a0 0x557c05f5c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:36.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.292+0000 7f6a12e87740 1 -- 172.21.15.16:0/3753212950 >> 172.21.15.16:0/3753212950 conn(0x557c05f41a30 msgr2=0x557c05f48090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:36.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.292+0000 7f6a12e87740 1 -- 172.21.15.16:0/3753212950 shutdown_connections 2024-09-12T02:05:36.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.292+0000 7f6a12e87740 1 -- 172.21.15.16:0/3753212950 wait complete. 2024-09-12T02:05:36.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.292+0000 7f6a12e87740 1 -- start start 2024-09-12T02:05:36.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.292+0000 7f6a12e87740 1 Processor -- start 2024-09-12T02:05:36.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.292+0000 7f6a12e87740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557c05f454a0 0x557c05f58cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:36.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.292+0000 7f6a12e87740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c05f45dc0 0x557c05f49fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:36.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.292+0000 7f6a12e87740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557c05f50680 0x557c05f4a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:36.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.292+0000 7f6a12e87740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6a0c1aba30 con 0x557c05f45dc0 2024-09-12T02:05:36.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.292+0000 7f6a12e87740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6a0c1ace80 con 0x557c05f454a0 2024-09-12T02:05:36.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.292+0000 7f6a12e87740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f6a0c1ac0e0 con 0x557c05f50680 2024-09-12T02:05:36.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.292+0000 7f6a12e87740 10 client.0 _ll_register_callbacks cb 0x7f69fc009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:05:36.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.292+0000 7f6a0a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c05f45dc0 0x557c05f49fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:36.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.292+0000 7f6a0a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c05f45dc0 0x557c05f49fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54738/0 (socket says 172.21.15.16:54738) 2024-09-12T02:05:36.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.292+0000 7f6a0a59c640 1 -- 172.21.15.16:0/181473467 learned_addr learned my addr 172.21.15.16:0/181473467 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:05:36.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.293+0000 7f6a09d9b640 1 --2- 172.21.15.16:0/181473467 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557c05f50680 0x557c05f4a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:36.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.293+0000 7f6a09d9b640 1 --2- 172.21.15.16:0/181473467 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557c05f454a0 0x557c05f58cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:36.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.293+0000 7f6a0a59c640 1 -- 172.21.15.16:0/181473467 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557c05f50680 msgr2=0x557c05f4a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:36.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.293+0000 7f6a0a59c640 1 --2- 172.21.15.16:0/181473467 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557c05f50680 0x557c05f4a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:36.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.293+0000 7f6a0a59c640 1 -- 172.21.15.16:0/181473467 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557c05f454a0 msgr2=0x557c05f58cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:36.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.293+0000 7f6a0a59c640 1 --2- 172.21.15.16:0/181473467 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557c05f454a0 0x557c05f58cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:36.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.293+0000 7f6a0a59c640 1 -- 172.21.15.16:0/181473467 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c05f50380 con 0x557c05f45dc0 2024-09-12T02:05:36.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.293+0000 7f6a09d9b640 1 --2- 172.21.15.16:0/181473467 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557c05f50680 0x557c05f4a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:05:36.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.293+0000 7f6a0a59c640 1 --2- 172.21.15.16:0/181473467 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c05f45dc0 0x557c05f49fa0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x557c05f4e900 tx=0x7f69fc009fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4995cf50ee8f3d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:36.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.293+0000 7f69f1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:05:36.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.293+0000 7f69f1ffb640 1 -- 172.21.15.16:0/181473467 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69fc01c630 con 0x557c05f45dc0 2024-09-12T02:05:36.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.293+0000 7f69f1ffb640 1 -- 172.21.15.16:0/181473467 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f69fc01cab0 con 0x557c05f45dc0 2024-09-12T02:05:36.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.293+0000 7f6a12e87740 1 -- 172.21.15.16:0/181473467 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a0000fc30 con 0x557c05f45dc0 2024-09-12T02:05:36.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.293+0000 7f69f1ffb640 1 -- 172.21.15.16:0/181473467 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f69fc01ccb0 con 0x557c05f45dc0 2024-09-12T02:05:36.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.293+0000 7f6a12e87740 10 client.16455 fetch_fsmap learned FSMap version 25 2024-09-12T02:05:36.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.293+0000 7f6a12e87740 1 -- 172.21.15.16:0/181473467 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x557c05fcf0f0 con 0x557c05f45dc0 2024-09-12T02:05:36.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.294+0000 7f69f1ffb640 1 -- 172.21.15.16:0/181473467 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a0000fc30 con 0x557c05f45dc0 2024-09-12T02:05:36.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.294+0000 7f69f1ffb640 1 -- 172.21.15.16:0/181473467 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f69fc0038f0 con 0x557c05f45dc0 2024-09-12T02:05:36.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.294+0000 7f6a12e87740 10 client.16455 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:05:36.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.294+0000 7f6a12e87740 10 client.16455 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:05:36.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.294+0000 7f6a12e87740 20 client.16455 populate_metadata read hostname 'smithi016' 2024-09-12T02:05:36.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.294+0000 7f6a12e87740 1 --2- 172.21.15.16:0/181473467 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x557c05f60150 0x557c05fd18e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:36.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.294+0000 7f6a12e87740 4 client.16455 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T02:05:36.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.294+0000 7f6a12e87740 1 -- 172.21.15.16:0/181473467 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557c05eb2d60 con 0x557c05f60150 2024-09-12T02:05:36.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.294+0000 7f6a09d9b640 1 --2- 172.21.15.16:0/181473467 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x557c05f60150 0x557c05fd18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:36.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.295+0000 7f6a09d9b640 1 --2- 172.21.15.16:0/181473467 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x557c05f60150 0x557c05fd18e0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e8ae14c656fa698 server_cookie=6601323f19107416 in_seq=0 out_seq=0 2024-09-12T02:05:36.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.295+0000 7f69f1ffb640 10 client.16455 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:05:36.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.295+0000 7f69f1ffb640 1 -- 172.21.15.16:0/181473467 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x557c05eb2d60 con 0x557c05f60150 2024-09-12T02:05:36.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.295+0000 7f69f1ffb640 10 client.16455 handle_command_reply: tid=0 2024-09-12T02:05:36.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.295+0000 7f6a12e87740 1 client.16455 shutdown 2024-09-12T02:05:36.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.295+0000 7f6a12e87740 1 -- 172.21.15.16:0/181473467 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c05f45dc0 msgr2=0x557c05f49fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:36.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.295+0000 7f6a12e87740 1 --2- 172.21.15.16:0/181473467 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c05f45dc0 0x557c05f49fa0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x557c05f4e900 tx=0x7f69fc009fd0 comp rx=0 tx=0).stop 2024-09-12T02:05:36.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.295+0000 7f6a12e87740 1 -- 172.21.15.16:0/181473467 shutdown_connections 2024-09-12T02:05:36.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.295+0000 7f6a12e87740 1 --2- 172.21.15.16:0/181473467 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x557c05f60150 0x557c05fd18e0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:36.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.295+0000 7f6a12e87740 1 --2- 172.21.15.16:0/181473467 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557c05f50680 0x557c05f4a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:36.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.295+0000 7f6a12e87740 1 --2- 172.21.15.16:0/181473467 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c05f45dc0 0x557c05f49fa0 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:36.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.295+0000 7f6a12e87740 1 --2- 172.21.15.16:0/181473467 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557c05f454a0 0x557c05f58cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:36.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.295+0000 7f6a12e87740 1 -- 172.21.15.16:0/181473467 >> 172.21.15.16:0/181473467 conn(0x557c05f41a30 msgr2=0x557c05f4ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:36.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.295+0000 7f69f1ffb640 0 client.16455 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:05:36.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.295+0000 7f6a12e87740 1 -- 172.21.15.16:0/181473467 shutdown_connections 2024-09-12T02:05:36.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.295+0000 7f6a12e87740 1 -- 172.21.15.16:0/181473467 wait complete. 2024-09-12T02:05:36.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.295+0000 7f6a12e87740 20 client.16455 trim_cache size 0 max 16384 2024-09-12T02:05:36.302 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T02:05:36.302 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-09-12T02:05:36.302 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {} 2024-09-12T02:05:36.302 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T02:05:36.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.296+0000 7f6a10aae640 1 -- 172.21.15.16:0/958125661 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f69f40a2b80 msgr2=0x7f69f40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:36.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.296+0000 7f6a10aae640 1 --2- 172.21.15.16:0/958125661 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f69f40a2b80 0x7f69f40a5040 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f6a0402fde0 tx=0x7f6a0402a950 comp rx=0 tx=0).stop 2024-09-12T02:05:36.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.296+0000 7f6a10aae640 1 -- 172.21.15.16:0/958125661 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a0c10c170 msgr2=0x7f6a0c19cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:36.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.296+0000 7f6a10aae640 1 --2- 172.21.15.16:0/958125661 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a0c10c170 0x7f6a0c19cd10 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f6a04004af0 tx=0x7f6a0402f3a0 comp rx=0 tx=0).stop 2024-09-12T02:05:36.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.296+0000 7f6a10aae640 1 -- 172.21.15.16:0/958125661 shutdown_connections 2024-09-12T02:05:36.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.296+0000 7f6a10aae640 1 --2- 172.21.15.16:0/958125661 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f69f40a2b80 0x7f69f40a5040 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:36.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.296+0000 7f6a10aae640 1 --2- 172.21.15.16:0/958125661 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a0c10c170 0x7f6a0c19cd10 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:36.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.296+0000 7f6a10aae640 1 --2- 172.21.15.16:0/958125661 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6a0c103660 0x7f6a0c198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:36.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.296+0000 7f6a10aae640 1 --2- 172.21.15.16:0/958125661 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a0c102d40 0x7f6a0c198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:36.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.296+0000 7f6a10aae640 1 -- 172.21.15.16:0/958125661 >> 172.21.15.16:0/958125661 conn(0x7f6a0c0fc7a0 msgr2=0x7f6a0c0fdec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:36.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.296+0000 7f6a10aae640 1 -- 172.21.15.16:0/958125661 shutdown_connections 2024-09-12T02:05:36.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:36.296+0000 7f6a10aae640 1 -- 172.21.15.16:0/958125661 wait complete. 2024-09-12T02:05:36.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:05:35 smithi184 ceph-mon[35593]: pgmap v445: 97 pgs: 97 active+clean; 2.1 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 KiB/s wr, 14 op/s 2024-09-12T02:05:36.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:05:35 smithi033 ceph-mon[35332]: pgmap v445: 97 pgs: 97 active+clean; 2.1 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 KiB/s wr, 14 op/s 2024-09-12T02:05:36.970 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-12T02:05:36.970 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:05:37.229 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:05:37.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:37.710+0000 7f87d4e52640 1 -- 172.21.15.16:0/2287404779 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87d0102d40 msgr2=0x7f87d0103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:37.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:37.710+0000 7f87d4e52640 1 --2- 172.21.15.16:0/2287404779 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87d0102d40 0x7f87d0103120 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f87c0009900 tx=0x7f87c002a480 comp rx=0 tx=0).stop 2024-09-12T02:05:37.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:37.710+0000 7f87d4e52640 1 -- 172.21.15.16:0/2287404779 shutdown_connections 2024-09-12T02:05:37.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:37.710+0000 7f87d4e52640 1 --2- 172.21.15.16:0/2287404779 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f87d010c170 0x7f87d010e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:37.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:37.710+0000 7f87d4e52640 1 --2- 172.21.15.16:0/2287404779 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87d0103660 0x7f87d010bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:37.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:37.710+0000 7f87d4e52640 1 --2- 172.21.15.16:0/2287404779 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87d0102d40 0x7f87d0103120 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:37.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:37.710+0000 7f87d4e52640 1 -- 172.21.15.16:0/2287404779 >> 172.21.15.16:0/2287404779 conn(0x7f87d00fc7a0 msgr2=0x7f87d00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:37.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:37.710+0000 7f87d4e52640 1 -- 172.21.15.16:0/2287404779 shutdown_connections 2024-09-12T02:05:37.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:37.710+0000 7f87d4e52640 1 -- 172.21.15.16:0/2287404779 wait complete. 2024-09-12T02:05:37.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:37.711+0000 7f87d4e52640 1 Processor -- start 2024-09-12T02:05:37.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:37.711+0000 7f87d4e52640 1 -- start start 2024-09-12T02:05:37.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:37.712+0000 7f87d4e52640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87d0102d40 0x7f87d0198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:37.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:37.712+0000 7f87d4e52640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87d0103660 0x7f87d01989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:37.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:37.712+0000 7f87d4e52640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f87d010c170 0x7f87d019cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:37.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:37.712+0000 7f87d4e52640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f87d0111fb0 con 0x7f87d0102d40 2024-09-12T02:05:37.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:37.712+0000 7f87d4e52640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f87d0111e30 con 0x7f87d0103660 2024-09-12T02:05:37.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:37.712+0000 7f87d4e52640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f87d0112130 con 0x7f87d010c170 2024-09-12T02:05:37.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:37.712+0000 7f87ce59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87d0102d40 0x7f87d0198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:37.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:37.712+0000 7f87ce59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87d0102d40 0x7f87d0198470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54760/0 (socket says 172.21.15.16:54760) 2024-09-12T02:05:37.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:37.712+0000 7f87ce59c640 1 -- 172.21.15.16:0/1535655322 learned_addr learned my addr 172.21.15.16:0/1535655322 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:05:37.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:37.712+0000 7f87ced9d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f87d010c170 0x7f87d019cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:37.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:37.712+0000 7f87cdd9b640 1 --2- 172.21.15.16:0/1535655322 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87d0103660 0x7f87d01989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:37.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:37.712+0000 7f87ce59c640 1 -- 172.21.15.16:0/1535655322 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f87d010c170 msgr2=0x7f87d019cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:37.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:37.712+0000 7f87ce59c640 1 --2- 172.21.15.16:0/1535655322 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f87d010c170 0x7f87d019cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:37.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:37.712+0000 7f87ce59c640 1 -- 172.21.15.16:0/1535655322 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87d0103660 msgr2=0x7f87d01989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:37.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:37.712+0000 7f87ce59c640 1 --2- 172.21.15.16:0/1535655322 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87d0103660 0x7f87d01989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:37.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:37.712+0000 7f87ce59c640 1 -- 172.21.15.16:0/1535655322 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87d019d420 con 0x7f87d0102d40 2024-09-12T02:05:37.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:37.712+0000 7f87cdd9b640 1 --2- 172.21.15.16:0/1535655322 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87d0103660 0x7f87d01989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:05:37.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:37.712+0000 7f87ce59c640 1 --2- 172.21.15.16:0/1535655322 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87d0102d40 0x7f87d0198470 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f87c0006010 tx=0x7f87c00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:37.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:37.713+0000 7f87c77fe640 1 -- 172.21.15.16:0/1535655322 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87c003f030 con 0x7f87d0102d40 2024-09-12T02:05:37.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:37.713+0000 7f87c77fe640 1 -- 172.21.15.16:0/1535655322 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87c0003ce0 con 0x7f87d0102d40 2024-09-12T02:05:37.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:37.713+0000 7f87d4e52640 1 -- 172.21.15.16:0/1535655322 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87d019d6b0 con 0x7f87d0102d40 2024-09-12T02:05:37.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:37.713+0000 7f87c77fe640 1 -- 172.21.15.16:0/1535655322 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87c0035070 con 0x7f87d0102d40 2024-09-12T02:05:37.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:37.713+0000 7f87d4e52640 1 -- 172.21.15.16:0/1535655322 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87d01a4ff0 con 0x7f87d0102d40 2024-09-12T02:05:37.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:37.714+0000 7f87d4e52640 1 -- 172.21.15.16:0/1535655322 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87d006b070 con 0x7f87d0102d40 2024-09-12T02:05:37.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:37.715+0000 7f87c77fe640 1 -- 172.21.15.16:0/1535655322 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f87c0042030 con 0x7f87d0102d40 2024-09-12T02:05:37.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:37.715+0000 7f87c77fe640 1 --2- 172.21.15.16:0/1535655322 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f87bc0a2b80 0x7f87bc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:37.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:37.716+0000 7f87cdd9b640 1 --2- 172.21.15.16:0/1535655322 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f87bc0a2b80 0x7f87bc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:37.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:37.716+0000 7f87c77fe640 1 -- 172.21.15.16:0/1535655322 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f87c00ef700 con 0x7f87d0102d40 2024-09-12T02:05:37.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:37.716+0000 7f87cdd9b640 1 --2- 172.21.15.16:0/1535655322 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f87bc0a2b80 0x7f87bc0a5040 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f87d010d4e0 tx=0x7f87b8009670 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:37.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:37.720+0000 7f87c77fe640 1 -- 172.21.15.16:0/1535655322 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f87c00bc720 con 0x7f87d0102d40 2024-09-12T02:05:37.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:37.933+0000 7f87d4e52640 1 -- 172.21.15.16:0/1535655322 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f87d0107210 con 0x7f87d0102d40 2024-09-12T02:05:37.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:37.934+0000 7f87c77fe640 1 -- 172.21.15.16:0/1535655322 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11911 (secure 0 0 0) 0x7f87c00bc020 con 0x7f87d0102d40 2024-09-12T02:05:37.934 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T02:05:37.934 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":25,"btime":"2024-09-12T02:01:37:192021+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3279123279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3279123279},{"type":"v1","addr":"172.21.15.16:6835","nonce":3279123279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4119452382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4119452382},{"type":"v1","addr":"172.21.15.16:6837","nonce":4119452382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/3263823165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3263823165},{"type":"v1","addr":"172.21.15.16:6839","nonce":3263823165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/1510290732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":1510290732},{"type":"v1","addr":"172.21.15.16:6841","nonce":1510290732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/279674603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":279674603},{"type":"v1","addr":"172.21.15.33:6833","nonce":279674603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3555728028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3555728028},{"type":"v1","addr":"172.21.15.33:6835","nonce":3555728028}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T01:59:56.117608+0000","modified":"2024-09-12T02:01:36.294760+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24482,"mds_1":24481,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6833/1563374106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1563374106},{"type":"v1","addr":"172.21.15.184:6833","nonce":1563374106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6837/1892833868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1892833868},{"type":"v1","addr":"172.21.15.33:6837","nonce":1892833868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/1457679686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1457679686},{"type":"v1","addr":"172.21.15.184:6835","nonce":1457679686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.184:6837/1793393087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":1793393087},{"type":"v1","addr":"172.21.15.184:6837","nonce":1793393087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.184:6839/4203347941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":4203347941},{"type":"v1","addr":"172.21.15.184:6839","nonce":4203347941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.33:6839/4136670204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":4136670204},{"type":"v1","addr":"172.21.15.33:6839","nonce":4136670204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24481,24482,24469]},"id":1}]} 2024-09-12T02:05:37.934 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 25 2024-09-12T02:05:37.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:37.936+0000 7f87d4e52640 1 -- 172.21.15.16:0/1535655322 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f87bc0a2b80 msgr2=0x7f87bc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:37.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:37.936+0000 7f87d4e52640 1 --2- 172.21.15.16:0/1535655322 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f87bc0a2b80 0x7f87bc0a5040 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f87d010d4e0 tx=0x7f87b8009670 comp rx=0 tx=0).stop 2024-09-12T02:05:37.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:37.936+0000 7f87d4e52640 1 -- 172.21.15.16:0/1535655322 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87d0102d40 msgr2=0x7f87d0198470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:37.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:37.936+0000 7f87d4e52640 1 --2- 172.21.15.16:0/1535655322 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87d0102d40 0x7f87d0198470 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f87c0006010 tx=0x7f87c00042f0 comp rx=0 tx=0).stop 2024-09-12T02:05:37.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:37.936+0000 7f87d4e52640 1 -- 172.21.15.16:0/1535655322 shutdown_connections 2024-09-12T02:05:37.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:37.936+0000 7f87d4e52640 1 --2- 172.21.15.16:0/1535655322 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f87bc0a2b80 0x7f87bc0a5040 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:37.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:37.936+0000 7f87d4e52640 1 --2- 172.21.15.16:0/1535655322 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f87d010c170 0x7f87d019cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:37.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:37.936+0000 7f87d4e52640 1 --2- 172.21.15.16:0/1535655322 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87d0103660 0x7f87d01989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:37.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:37.936+0000 7f87d4e52640 1 --2- 172.21.15.16:0/1535655322 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87d0102d40 0x7f87d0198470 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:37.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:37.936+0000 7f87d4e52640 1 -- 172.21.15.16:0/1535655322 >> 172.21.15.16:0/1535655322 conn(0x7f87d00fc7a0 msgr2=0x7f87d00fd400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:37.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:37.936+0000 7f87d4e52640 1 -- 172.21.15.16:0/1535655322 shutdown_connections 2024-09-12T02:05:37.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:37.936+0000 7f87d4e52640 1 -- 172.21.15.16:0/1535655322 wait complete. 2024-09-12T02:05:38.183 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:05:38 smithi016 ceph-mon[30947]: pgmap v446: 97 pgs: 97 active+clean; 2.1 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 12 op/s 2024-09-12T02:05:38.183 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:05:38 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/1535655322' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:05:38.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:05:38 smithi184 ceph-mon[35593]: pgmap v446: 97 pgs: 97 active+clean; 2.1 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 12 op/s 2024-09-12T02:05:38.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:05:38 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/1535655322' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:05:38.418 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:05:38 smithi033 ceph-mon[35332]: pgmap v446: 97 pgs: 97 active+clean; 2.1 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 12 op/s 2024-09-12T02:05:38.418 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:05:38 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/1535655322' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:05:38.748 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-12T02:05:39.012 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:05:39.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.459+0000 7f2c958d4640 1 -- 172.21.15.16:0/287772582 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c901027a0 msgr2=0x7f2c901067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:39.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.459+0000 7f2c958d4640 1 --2- 172.21.15.16:0/287772582 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c901027a0 0x7f2c901067f0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f2c84009900 tx=0x7f2c8402a410 comp rx=0 tx=0).stop 2024-09-12T02:05:39.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.459+0000 7f2c958d4640 1 -- 172.21.15.16:0/287772582 shutdown_connections 2024-09-12T02:05:39.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.459+0000 7f2c958d4640 1 --2- 172.21.15.16:0/287772582 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2c90106f20 0x7f2c9010eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:39.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.459+0000 7f2c958d4640 1 --2- 172.21.15.16:0/287772582 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c901027a0 0x7f2c901067f0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:39.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.459+0000 7f2c958d4640 1 --2- 172.21.15.16:0/287772582 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c90101df0 0x7f2c901021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:39.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.459+0000 7f2c958d4640 1 -- 172.21.15.16:0/287772582 >> 172.21.15.16:0/287772582 conn(0x7f2c900fd8b0 msgr2=0x7f2c900ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:39.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.459+0000 7f2c958d4640 1 -- 172.21.15.16:0/287772582 shutdown_connections 2024-09-12T02:05:39.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.459+0000 7f2c958d4640 1 -- 172.21.15.16:0/287772582 wait complete. 2024-09-12T02:05:39.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.460+0000 7f2c958d4640 1 Processor -- start 2024-09-12T02:05:39.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.460+0000 7f2c958d4640 1 -- start start 2024-09-12T02:05:39.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.460+0000 7f2c958d4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c90101df0 0x7f2c90198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:39.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.460+0000 7f2c958d4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c901027a0 0x7f2c901989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:39.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.460+0000 7f2c958d4640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2c90106f20 0x7f2c9019cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:39.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.461+0000 7f2c958d4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2c90111fa0 con 0x7f2c90101df0 2024-09-12T02:05:39.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.461+0000 7f2c958d4640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2c90111e20 con 0x7f2c901027a0 2024-09-12T02:05:39.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.461+0000 7f2c958d4640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f2c90112120 con 0x7f2c90106f20 2024-09-12T02:05:39.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.461+0000 7f2c8effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c90101df0 0x7f2c90198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:39.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.461+0000 7f2c8effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c90101df0 0x7f2c90198470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54780/0 (socket says 172.21.15.16:54780) 2024-09-12T02:05:39.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.461+0000 7f2c8effd640 1 -- 172.21.15.16:0/2067021974 learned_addr learned my addr 172.21.15.16:0/2067021974 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:05:39.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.461+0000 7f2c8e7fc640 1 --2- 172.21.15.16:0/2067021974 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c901027a0 0x7f2c901989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:39.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.461+0000 7f2c8f7fe640 1 --2- 172.21.15.16:0/2067021974 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2c90106f20 0x7f2c9019cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:39.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.461+0000 7f2c8effd640 1 -- 172.21.15.16:0/2067021974 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2c90106f20 msgr2=0x7f2c9019cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:39.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.461+0000 7f2c8effd640 1 --2- 172.21.15.16:0/2067021974 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2c90106f20 0x7f2c9019cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:39.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.461+0000 7f2c8effd640 1 -- 172.21.15.16:0/2067021974 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c901027a0 msgr2=0x7f2c901989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:39.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.461+0000 7f2c8effd640 1 --2- 172.21.15.16:0/2067021974 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c901027a0 0x7f2c901989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:39.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.461+0000 7f2c8effd640 1 -- 172.21.15.16:0/2067021974 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c9019d420 con 0x7f2c90101df0 2024-09-12T02:05:39.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.461+0000 7f2c8e7fc640 1 --2- 172.21.15.16:0/2067021974 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c901027a0 0x7f2c901989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:05:39.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.461+0000 7f2c8effd640 1 --2- 172.21.15.16:0/2067021974 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c90101df0 0x7f2c90198470 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f2c8000d950 tx=0x7f2c8000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:39.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.461+0000 7f2c948d2640 1 -- 172.21.15.16:0/2067021974 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c80017070 con 0x7f2c90101df0 2024-09-12T02:05:39.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.462+0000 7f2c948d2640 1 -- 172.21.15.16:0/2067021974 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c80009c30 con 0x7f2c90101df0 2024-09-12T02:05:39.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.462+0000 7f2c958d4640 1 -- 172.21.15.16:0/2067021974 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c9019d710 con 0x7f2c90101df0 2024-09-12T02:05:39.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.462+0000 7f2c948d2640 1 -- 172.21.15.16:0/2067021974 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c8000f6b0 con 0x7f2c90101df0 2024-09-12T02:05:39.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.462+0000 7f2c958d4640 1 -- 172.21.15.16:0/2067021974 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c901a4ff0 con 0x7f2c90101df0 2024-09-12T02:05:39.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.464+0000 7f2c948d2640 1 -- 172.21.15.16:0/2067021974 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f2c800109d0 con 0x7f2c90101df0 2024-09-12T02:05:39.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.464+0000 7f2c97cad740 1 Processor -- start 2024-09-12T02:05:39.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.464+0000 7f2c97cad740 1 -- start start 2024-09-12T02:05:39.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.465+0000 7f2c97cad740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55972f87ab40 0x55972f879090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:39.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.465+0000 7f2c97cad740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55972f8795d0 0x55972f879a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:39.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.465+0000 7f2c97cad740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55972f884370 0x55972f886760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:39.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.465+0000 7f2c97cad740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55972f7df2f0 con 0x55972f884370 2024-09-12T02:05:39.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.465+0000 7f2c97cad740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55972f786cf0 con 0x55972f87ab40 2024-09-12T02:05:39.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.465+0000 7f2c97cad740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55972f7db2b0 con 0x55972f8795d0 2024-09-12T02:05:39.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.465+0000 7f2c8e7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55972f884370 0x55972f886760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:39.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.465+0000 7f2c8e7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55972f884370 0x55972f886760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54792/0 (socket says 172.21.15.16:54792) 2024-09-12T02:05:39.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.465+0000 7f2c8e7fc640 1 -- 172.21.15.16:0/2242820139 learned_addr learned my addr 172.21.15.16:0/2242820139 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:05:39.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.465+0000 7f2c948d2640 1 --2- 172.21.15.16:0/2067021974 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f2c780a2b80 0x7f2c780a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:39.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.465+0000 7f2c8effd640 1 --2- 172.21.15.16:0/2242820139 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55972f8795d0 0x55972f879a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:39.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.465+0000 7f2c8f7fe640 1 --2- 172.21.15.16:0/2242820139 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55972f87ab40 0x55972f879090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:39.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.465+0000 7f2c8f7fe640 1 --2- 172.21.15.16:0/2067021974 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f2c780a2b80 0x7f2c780a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:39.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.465+0000 7f2c8e7fc640 1 -- 172.21.15.16:0/2242820139 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55972f8795d0 msgr2=0x55972f879a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:39.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.465+0000 7f2c8e7fc640 1 --2- 172.21.15.16:0/2242820139 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55972f8795d0 0x55972f879a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:39.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.465+0000 7f2c8e7fc640 1 -- 172.21.15.16:0/2242820139 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55972f87ab40 msgr2=0x55972f879090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:39.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.465+0000 7f2c8e7fc640 1 --2- 172.21.15.16:0/2242820139 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55972f87ab40 0x55972f879090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:39.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.465+0000 7f2c8e7fc640 1 -- 172.21.15.16:0/2242820139 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55972f76b3a0 con 0x55972f884370 2024-09-12T02:05:39.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.465+0000 7f2c948d2640 1 -- 172.21.15.16:0/2067021974 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f2c800ceb80 con 0x7f2c90101df0 2024-09-12T02:05:39.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.466+0000 7f2c8e7fc640 1 --2- 172.21.15.16:0/2242820139 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55972f884370 0x55972f886760 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f2c8402a930 tx=0x7f2c84031630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac0338ec5f12377f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:39.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.466+0000 7f2c8f7fe640 1 --2- 172.21.15.16:0/2067021974 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f2c780a2b80 0x7f2c780a5040 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f2c88009fd0 tx=0x7f2c88004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:39.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.466+0000 7f2c75ffb640 1 -- 172.21.15.16:0/2242820139 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c84035040 con 0x55972f884370 2024-09-12T02:05:39.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.466+0000 7f2c75ffb640 1 -- 172.21.15.16:0/2242820139 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c84031a20 con 0x55972f884370 2024-09-12T02:05:39.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.466+0000 7f2c75ffb640 1 -- 172.21.15.16:0/2242820139 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c84004a30 con 0x55972f884370 2024-09-12T02:05:39.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.466+0000 7f2c97cad740 1 -- 172.21.15.16:0/2242820139 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55972f884370 msgr2=0x55972f886760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:39.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.466+0000 7f2c97cad740 1 --2- 172.21.15.16:0/2242820139 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55972f884370 0x55972f886760 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f2c8402a930 tx=0x7f2c84031630 comp rx=0 tx=0).stop 2024-09-12T02:05:39.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.467+0000 7f2c97cad740 1 -- 172.21.15.16:0/2242820139 shutdown_connections 2024-09-12T02:05:39.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.467+0000 7f2c97cad740 1 --2- 172.21.15.16:0/2242820139 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55972f884370 0x55972f886760 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:39.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.467+0000 7f2c97cad740 1 --2- 172.21.15.16:0/2242820139 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55972f8795d0 0x55972f879a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:39.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.467+0000 7f2c97cad740 1 --2- 172.21.15.16:0/2242820139 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55972f87ab40 0x55972f879090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:39.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.467+0000 7f2c97cad740 1 -- 172.21.15.16:0/2242820139 >> 172.21.15.16:0/2242820139 conn(0x55972f842310 msgr2=0x55972f875c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:39.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.467+0000 7f2c97cad740 1 -- 172.21.15.16:0/2242820139 shutdown_connections 2024-09-12T02:05:39.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.467+0000 7f2c97cad740 1 -- 172.21.15.16:0/2242820139 wait complete. 2024-09-12T02:05:39.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.467+0000 7f2c97cad740 1 -- start start 2024-09-12T02:05:39.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.467+0000 7f2c97cad740 1 Processor -- start 2024-09-12T02:05:39.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.467+0000 7f2c97cad740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55972f8795d0 0x55972f883c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:39.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.467+0000 7f2c97cad740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55972f87ab40 0x55972f880ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:39.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.468+0000 7f2c97cad740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55972f884370 0x55972f881420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:39.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.468+0000 7f2c97cad740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55972f887010 con 0x55972f884370 2024-09-12T02:05:39.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.468+0000 7f2c97cad740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55972f79a060 con 0x55972f87ab40 2024-09-12T02:05:39.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.468+0000 7f2c97cad740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55972f887190 con 0x55972f8795d0 2024-09-12T02:05:39.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.468+0000 7f2c97cad740 10 client.0 _ll_register_callbacks cb 0x55972f4ed8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:05:39.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.468+0000 7f2c8f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55972f884370 0x55972f881420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:39.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.468+0000 7f2c8f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55972f884370 0x55972f881420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54806/0 (socket says 172.21.15.16:54806) 2024-09-12T02:05:39.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.468+0000 7f2c8f7fe640 1 -- 172.21.15.16:0/999367934 learned_addr learned my addr 172.21.15.16:0/999367934 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:05:39.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.468+0000 7f2c8e7fc640 1 --2- 172.21.15.16:0/999367934 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55972f87ab40 0x55972f880ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:39.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.468+0000 7f2c8effd640 1 --2- 172.21.15.16:0/999367934 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55972f8795d0 0x55972f883c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:39.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.468+0000 7f2c8f7fe640 1 -- 172.21.15.16:0/999367934 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55972f8795d0 msgr2=0x55972f883c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:39.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.468+0000 7f2c8f7fe640 1 --2- 172.21.15.16:0/999367934 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55972f8795d0 0x55972f883c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:39.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.468+0000 7f2c8f7fe640 1 -- 172.21.15.16:0/999367934 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55972f87ab40 msgr2=0x55972f880ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:39.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.468+0000 7f2c8f7fe640 1 --2- 172.21.15.16:0/999367934 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55972f87ab40 0x55972f880ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:39.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.468+0000 7f2c8f7fe640 1 -- 172.21.15.16:0/999367934 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55972f5503c0 con 0x55972f884370 2024-09-12T02:05:39.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.468+0000 7f2c8f7fe640 1 --2- 172.21.15.16:0/999367934 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55972f884370 0x55972f881420 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f2c8800af20 tx=0x7f2c88004b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc2b83a1a8f2c109 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:39.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.468+0000 7f2c6effd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:05:39.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.468+0000 7f2c6effd640 1 -- 172.21.15.16:0/999367934 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c88029060 con 0x55972f884370 2024-09-12T02:05:39.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.468+0000 7f2c6effd640 1 -- 172.21.15.16:0/999367934 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f2c88024600 con 0x55972f884370 2024-09-12T02:05:39.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.468+0000 7f2c97cad740 1 -- 172.21.15.16:0/999367934 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c84031a20 con 0x55972f884370 2024-09-12T02:05:39.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.469+0000 7f2c6effd640 1 -- 172.21.15.16:0/999367934 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2c88019e10 con 0x55972f884370 2024-09-12T02:05:39.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.469+0000 7f2c97cad740 10 client.16473 fetch_fsmap learned FSMap version 25 2024-09-12T02:05:39.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.469+0000 7f2c97cad740 1 -- 172.21.15.16:0/999367934 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55972f892a00 con 0x55972f884370 2024-09-12T02:05:39.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.469+0000 7f2c6effd640 1 -- 172.21.15.16:0/999367934 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c84031a20 con 0x55972f884370 2024-09-12T02:05:39.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.469+0000 7f2c6effd640 1 -- 172.21.15.16:0/999367934 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f2c8802e070 con 0x55972f884370 2024-09-12T02:05:39.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.469+0000 7f2c97cad740 10 client.16473 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:05:39.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.469+0000 7f2c97cad740 10 client.16473 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-12T02:05:39.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.469+0000 7f2c97cad740 20 client.16473 populate_metadata read hostname 'smithi016' 2024-09-12T02:05:39.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.469+0000 7f2c97cad740 1 --2- 172.21.15.16:0/999367934 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55972f893ec0 0x55972f896310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:39.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.469+0000 7f2c97cad740 4 client.16473 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:05:39.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.469+0000 7f2c97cad740 1 -- 172.21.15.16:0/999367934 --> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55972f7ec860 con 0x55972f893ec0 2024-09-12T02:05:39.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.470+0000 7f2c8effd640 1 --2- 172.21.15.16:0/999367934 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55972f893ec0 0x55972f896310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:39.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.470+0000 7f2c8effd640 1 --2- 172.21.15.16:0/999367934 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55972f893ec0 0x55972f896310 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=894d31a330b91a7f server_cookie=24247fcd9ff3c246 in_seq=0 out_seq=0 2024-09-12T02:05:39.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.470+0000 7f2c6effd640 10 client.16473 ms_handle_connect on v2:172.21.15.184:6836/1793393087 2024-09-12T02:05:39.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.471+0000 7f2c6effd640 1 -- 172.21.15.16:0/999367934 <== mds.2 v2:172.21.15.184:6836/1793393087 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55972f7ec860 con 0x55972f893ec0 2024-09-12T02:05:39.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.471+0000 7f2c6effd640 10 client.16473 handle_command_reply: tid=0 2024-09-12T02:05:39.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.471+0000 7f2c97cad740 1 client.16473 shutdown 2024-09-12T02:05:39.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.471+0000 7f2c97cad740 1 -- 172.21.15.16:0/999367934 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55972f884370 msgr2=0x55972f881420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:39.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.471+0000 7f2c97cad740 1 --2- 172.21.15.16:0/999367934 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55972f884370 0x55972f881420 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f2c8800af20 tx=0x7f2c88004b50 comp rx=0 tx=0).stop 2024-09-12T02:05:39.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.471+0000 7f2c97cad740 1 -- 172.21.15.16:0/999367934 shutdown_connections 2024-09-12T02:05:39.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.471+0000 7f2c97cad740 1 --2- 172.21.15.16:0/999367934 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55972f893ec0 0x55972f896310 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:39.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.471+0000 7f2c97cad740 1 --2- 172.21.15.16:0/999367934 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55972f884370 0x55972f881420 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:39.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.471+0000 7f2c97cad740 1 --2- 172.21.15.16:0/999367934 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55972f87ab40 0x55972f880ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:39.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.471+0000 7f2c97cad740 1 --2- 172.21.15.16:0/999367934 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55972f8795d0 0x55972f883c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:39.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.471+0000 7f2c6effd640 0 client.16473 ms_handle_reset on v2:172.21.15.184:6836/1793393087 2024-09-12T02:05:39.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.471+0000 7f2c97cad740 1 -- 172.21.15.16:0/999367934 >> 172.21.15.16:0/999367934 conn(0x55972f842310 msgr2=0x55972f87b960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:39.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.471+0000 7f2c97cad740 1 -- 172.21.15.16:0/999367934 shutdown_connections 2024-09-12T02:05:39.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.471+0000 7f2c97cad740 1 -- 172.21.15.16:0/999367934 wait complete. 2024-09-12T02:05:39.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.472+0000 7f2c97cad740 20 client.16473 trim_cache size 0 max 16384 2024-09-12T02:05:39.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.484+0000 7f2c97cad740 1 Processor -- start 2024-09-12T02:05:39.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.484+0000 7f2c97cad740 1 -- start start 2024-09-12T02:05:39.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.485+0000 7f2c97cad740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55972f8795d0 0x55972f8902b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:39.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.485+0000 7f2c97cad740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55972f87ab40 0x55972f878c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:39.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.485+0000 7f2c97cad740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55972f884370 0x55972f885760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:39.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.485+0000 7f2c97cad740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2c901afd00 con 0x55972f8795d0 2024-09-12T02:05:39.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.485+0000 7f2c97cad740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2c901ad5d0 con 0x55972f884370 2024-09-12T02:05:39.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.485+0000 7f2c97cad740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f2c901ab460 con 0x55972f87ab40 2024-09-12T02:05:39.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.485+0000 7f2c8effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55972f8795d0 0x55972f8902b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:39.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.485+0000 7f2c8effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55972f8795d0 0x55972f8902b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54818/0 (socket says 172.21.15.16:54818) 2024-09-12T02:05:39.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.485+0000 7f2c8effd640 1 -- 172.21.15.16:0/392946970 learned_addr learned my addr 172.21.15.16:0/392946970 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:05:39.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.485+0000 7f2c8effd640 1 -- 172.21.15.16:0/392946970 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55972f87ab40 msgr2=0x55972f878c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:39.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.485+0000 7f2c8effd640 1 --2- 172.21.15.16:0/392946970 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55972f87ab40 0x55972f878c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:39.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.485+0000 7f2c8effd640 1 -- 172.21.15.16:0/392946970 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55972f884370 msgr2=0x55972f885760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:39.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.485+0000 7f2c8f7fe640 1 --2- 172.21.15.16:0/392946970 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55972f884370 0x55972f885760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:39.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.485+0000 7f2c8effd640 1 --2- 172.21.15.16:0/392946970 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55972f884370 0x55972f885760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:39.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.485+0000 7f2c8effd640 1 -- 172.21.15.16:0/392946970 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55972f886af0 con 0x55972f8795d0 2024-09-12T02:05:39.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.485+0000 7f2c8effd640 1 --2- 172.21.15.16:0/392946970 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55972f8795d0 0x55972f8902b0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f2c800cc720 tx=0x7f2c80095f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef3b935b23a3f0d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:39.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.485+0000 7f2c757fa640 1 -- 172.21.15.16:0/392946970 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c80020040 con 0x55972f8795d0 2024-09-12T02:05:39.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.486+0000 7f2c757fa640 1 -- 172.21.15.16:0/392946970 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c80009de0 con 0x55972f8795d0 2024-09-12T02:05:39.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.486+0000 7f2c757fa640 1 -- 172.21.15.16:0/392946970 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c800995d0 con 0x55972f8795d0 2024-09-12T02:05:39.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.486+0000 7f2c97cad740 1 -- 172.21.15.16:0/392946970 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55972f8795d0 msgr2=0x55972f8902b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:39.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.486+0000 7f2c97cad740 1 --2- 172.21.15.16:0/392946970 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55972f8795d0 0x55972f8902b0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f2c800cc720 tx=0x7f2c80095f90 comp rx=0 tx=0).stop 2024-09-12T02:05:39.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.486+0000 7f2c97cad740 1 -- 172.21.15.16:0/392946970 shutdown_connections 2024-09-12T02:05:39.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.486+0000 7f2c97cad740 1 --2- 172.21.15.16:0/392946970 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55972f884370 0x55972f885760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:39.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.486+0000 7f2c97cad740 1 --2- 172.21.15.16:0/392946970 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55972f87ab40 0x55972f878c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:39.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.486+0000 7f2c97cad740 1 --2- 172.21.15.16:0/392946970 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55972f8795d0 0x55972f8902b0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:39.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.486+0000 7f2c97cad740 1 -- 172.21.15.16:0/392946970 >> 172.21.15.16:0/392946970 conn(0x55972f842310 msgr2=0x55972f880ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:39.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.486+0000 7f2c97cad740 1 -- 172.21.15.16:0/392946970 shutdown_connections 2024-09-12T02:05:39.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.486+0000 7f2c97cad740 1 -- 172.21.15.16:0/392946970 wait complete. 2024-09-12T02:05:39.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.486+0000 7f2c97cad740 1 -- start start 2024-09-12T02:05:39.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.487+0000 7f2c97cad740 1 Processor -- start 2024-09-12T02:05:39.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.487+0000 7f2c97cad740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55972f8795d0 0x55972f885960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:39.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.487+0000 7f2c97cad740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55972f87ab40 0x55972f885ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:39.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.487+0000 7f2c97cad740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55972f884370 0x55972f8863e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:39.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.487+0000 7f2c97cad740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2c901abab0 con 0x55972f87ab40 2024-09-12T02:05:39.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.487+0000 7f2c97cad740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2c901acf00 con 0x55972f8795d0 2024-09-12T02:05:39.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.487+0000 7f2c97cad740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f2c901ac160 con 0x55972f884370 2024-09-12T02:05:39.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.487+0000 7f2c97cad740 10 client.0 _ll_register_callbacks cb 0x7f2c88015d00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:05:39.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.487+0000 7f2c8e7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55972f87ab40 0x55972f885ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:39.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.487+0000 7f2c8e7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55972f87ab40 0x55972f885ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54820/0 (socket says 172.21.15.16:54820) 2024-09-12T02:05:39.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.487+0000 7f2c8e7fc640 1 -- 172.21.15.16:0/3447991357 learned_addr learned my addr 172.21.15.16:0/3447991357 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:05:39.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.487+0000 7f2c8effd640 1 --2- 172.21.15.16:0/3447991357 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55972f8795d0 0x55972f885960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:39.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.487+0000 7f2c8f7fe640 1 --2- 172.21.15.16:0/3447991357 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55972f884370 0x55972f8863e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:39.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.487+0000 7f2c8e7fc640 1 -- 172.21.15.16:0/3447991357 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55972f884370 msgr2=0x55972f8863e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:39.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.487+0000 7f2c8e7fc640 1 --2- 172.21.15.16:0/3447991357 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55972f884370 0x55972f8863e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:39.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.487+0000 7f2c8e7fc640 1 -- 172.21.15.16:0/3447991357 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55972f8795d0 msgr2=0x55972f885960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:39.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.487+0000 7f2c8e7fc640 1 --2- 172.21.15.16:0/3447991357 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55972f8795d0 0x55972f885960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:39.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.488+0000 7f2c8e7fc640 1 -- 172.21.15.16:0/3447991357 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55972f88ac10 con 0x55972f87ab40 2024-09-12T02:05:39.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.488+0000 7f2c8effd640 1 --2- 172.21.15.16:0/3447991357 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55972f8795d0 0x55972f885960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:05:39.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.488+0000 7f2c8e7fc640 1 --2- 172.21.15.16:0/3447991357 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55972f87ab40 0x55972f885ea0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f2c84005640 tx=0x7f2c840314e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fadb342eaf14f60a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:39.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.488+0000 7f2c6effd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:05:39.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.488+0000 7f2c6effd640 1 -- 172.21.15.16:0/3447991357 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c84035040 con 0x55972f87ab40 2024-09-12T02:05:39.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.488+0000 7f2c6effd640 1 -- 172.21.15.16:0/3447991357 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f2c84004450 con 0x55972f87ab40 2024-09-12T02:05:39.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.488+0000 7f2c97cad740 1 -- 172.21.15.16:0/3447991357 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c80009de0 con 0x55972f87ab40 2024-09-12T02:05:39.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.488+0000 7f2c6effd640 1 -- 172.21.15.16:0/3447991357 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2c840035f0 con 0x55972f87ab40 2024-09-12T02:05:39.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.488+0000 7f2c97cad740 10 client.16479 fetch_fsmap learned FSMap version 25 2024-09-12T02:05:39.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.488+0000 7f2c97cad740 1 -- 172.21.15.16:0/3447991357 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55972f8feff0 con 0x55972f87ab40 2024-09-12T02:05:39.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.488+0000 7f2c6effd640 1 -- 172.21.15.16:0/3447991357 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c80009de0 con 0x55972f87ab40 2024-09-12T02:05:39.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.488+0000 7f2c6effd640 1 -- 172.21.15.16:0/3447991357 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f2c84058050 con 0x55972f87ab40 2024-09-12T02:05:39.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.489+0000 7f2c97cad740 10 client.16479 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:05:39.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.489+0000 7f2c97cad740 10 client.16479 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-12T02:05:39.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.489+0000 7f2c97cad740 20 client.16479 populate_metadata read hostname 'smithi016' 2024-09-12T02:05:39.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.489+0000 7f2c97cad740 1 --2- 172.21.15.16:0/3447991357 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55972f8ff380 0x55972f901770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:39.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.489+0000 7f2c97cad740 4 client.16479 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T02:05:39.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.489+0000 7f2c97cad740 1 -- 172.21.15.16:0/3447991357 --> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55972f7e6d70 con 0x55972f8ff380 2024-09-12T02:05:39.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.489+0000 7f2c8effd640 1 --2- 172.21.15.16:0/3447991357 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55972f8ff380 0x55972f901770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:39.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.489+0000 7f2c8effd640 1 --2- 172.21.15.16:0/3447991357 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55972f8ff380 0x55972f901770 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4f443b8dd43438f4 server_cookie=50bbd0dff22e5a29 in_seq=0 out_seq=0 2024-09-12T02:05:39.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.489+0000 7f2c6effd640 10 client.16479 ms_handle_connect on v2:172.21.15.184:6836/1793393087 2024-09-12T02:05:39.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.489+0000 7f2c6effd640 1 -- 172.21.15.16:0/3447991357 <== mds.2 v2:172.21.15.184:6836/1793393087 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55972f7e6d70 con 0x55972f8ff380 2024-09-12T02:05:39.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.489+0000 7f2c6effd640 10 client.16479 handle_command_reply: tid=0 2024-09-12T02:05:39.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.489+0000 7f2c97cad740 1 client.16479 shutdown 2024-09-12T02:05:39.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.490+0000 7f2c97cad740 1 -- 172.21.15.16:0/3447991357 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55972f87ab40 msgr2=0x55972f885ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:39.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.490+0000 7f2c97cad740 1 --2- 172.21.15.16:0/3447991357 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55972f87ab40 0x55972f885ea0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f2c84005640 tx=0x7f2c840314e0 comp rx=0 tx=0).stop 2024-09-12T02:05:39.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.490+0000 7f2c97cad740 1 -- 172.21.15.16:0/3447991357 shutdown_connections 2024-09-12T02:05:39.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.490+0000 7f2c97cad740 1 --2- 172.21.15.16:0/3447991357 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55972f884370 0x55972f8863e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:39.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.490+0000 7f2c97cad740 1 --2- 172.21.15.16:0/3447991357 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55972f8ff380 0x55972f901770 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:39.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.490+0000 7f2c97cad740 1 --2- 172.21.15.16:0/3447991357 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55972f87ab40 0x55972f885ea0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:39.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.490+0000 7f2c97cad740 1 --2- 172.21.15.16:0/3447991357 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55972f8795d0 0x55972f885960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:39.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.490+0000 7f2c97cad740 1 -- 172.21.15.16:0/3447991357 >> 172.21.15.16:0/3447991357 conn(0x55972f842310 msgr2=0x55972f882740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:39.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.490+0000 7f2c6effd640 0 client.16479 ms_handle_reset on v2:172.21.15.184:6836/1793393087 2024-09-12T02:05:39.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.490+0000 7f2c97cad740 1 -- 172.21.15.16:0/3447991357 shutdown_connections 2024-09-12T02:05:39.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.490+0000 7f2c97cad740 1 -- 172.21.15.16:0/3447991357 wait complete. 2024-09-12T02:05:39.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.490+0000 7f2c97cad740 20 client.16479 trim_cache size 0 max 16384 2024-09-12T02:05:39.504 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T02:05:39.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.490+0000 7f2c958d4640 1 -- 172.21.15.16:0/2067021974 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f2c780a2b80 msgr2=0x7f2c780a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:39.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.490+0000 7f2c958d4640 1 --2- 172.21.15.16:0/2067021974 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f2c780a2b80 0x7f2c780a5040 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f2c88009fd0 tx=0x7f2c88004a40 comp rx=0 tx=0).stop 2024-09-12T02:05:39.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.490+0000 7f2c958d4640 1 -- 172.21.15.16:0/2067021974 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c90101df0 msgr2=0x7f2c90198470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:39.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.490+0000 7f2c958d4640 1 --2- 172.21.15.16:0/2067021974 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c90101df0 0x7f2c90198470 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f2c8000d950 tx=0x7f2c8000de10 comp rx=0 tx=0).stop 2024-09-12T02:05:39.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.491+0000 7f2c958d4640 1 -- 172.21.15.16:0/2067021974 shutdown_connections 2024-09-12T02:05:39.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.491+0000 7f2c958d4640 1 --2- 172.21.15.16:0/2067021974 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f2c780a2b80 0x7f2c780a5040 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:39.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.491+0000 7f2c958d4640 1 --2- 172.21.15.16:0/2067021974 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2c90106f20 0x7f2c9019cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:39.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.491+0000 7f2c958d4640 1 --2- 172.21.15.16:0/2067021974 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c901027a0 0x7f2c901989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:39.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.491+0000 7f2c958d4640 1 --2- 172.21.15.16:0/2067021974 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c90101df0 0x7f2c90198470 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:39.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.491+0000 7f2c958d4640 1 -- 172.21.15.16:0/2067021974 >> 172.21.15.16:0/2067021974 conn(0x7f2c900fd8b0 msgr2=0x7f2c900feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:39.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.491+0000 7f2c958d4640 1 -- 172.21.15.16:0/2067021974 shutdown_connections 2024-09-12T02:05:39.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:39.491+0000 7f2c958d4640 1 -- 172.21.15.16:0/2067021974 wait complete. 2024-09-12T02:05:40.298 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:05:40 smithi016 ceph-mon[30947]: pgmap v447: 97 pgs: 97 active+clean; 2.1 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 KiB/s wr, 10 op/s 2024-09-12T02:05:40.328 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-12T02:05:40.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:05:40 smithi184 ceph-mon[35593]: pgmap v447: 97 pgs: 97 active+clean; 2.1 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 KiB/s wr, 10 op/s 2024-09-12T02:05:40.418 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:05:40 smithi033 ceph-mon[35332]: pgmap v447: 97 pgs: 97 active+clean; 2.1 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 KiB/s wr, 10 op/s 2024-09-12T02:05:40.587 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:05:41.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.036+0000 7f7eabccc640 1 -- 172.21.15.16:0/2535629642 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ea4102ef0 msgr2=0x7f7ea410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:41.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.036+0000 7f7eabccc640 1 --2- 172.21.15.16:0/2535629642 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ea4102ef0 0x7f7ea410da40 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c009ca0 tx=0x7f7e9c02a950 comp rx=0 tx=0).stop 2024-09-12T02:05:41.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.036+0000 7f7eabccc640 1 -- 172.21.15.16:0/2535629642 shutdown_connections 2024-09-12T02:05:41.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.036+0000 7f7eabccc640 1 --2- 172.21.15.16:0/2535629642 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ea4102ef0 0x7f7ea410da40 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:41.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.037+0000 7f7eabccc640 1 --2- 172.21.15.16:0/2535629642 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7ea4102570 0x7f7ea41029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:41.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.037+0000 7f7eabccc640 1 --2- 172.21.15.16:0/2535629642 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7ea4100e20 0x7f7ea4101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:41.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.037+0000 7f7eabccc640 1 -- 172.21.15.16:0/2535629642 >> 172.21.15.16:0/2535629642 conn(0x7f7ea40fc7d0 msgr2=0x7f7ea40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:41.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.037+0000 7f7eabccc640 1 -- 172.21.15.16:0/2535629642 shutdown_connections 2024-09-12T02:05:41.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.037+0000 7f7eabccc640 1 -- 172.21.15.16:0/2535629642 wait complete. 2024-09-12T02:05:41.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.037+0000 7f7eabccc640 1 Processor -- start 2024-09-12T02:05:41.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.037+0000 7f7eabccc640 1 -- start start 2024-09-12T02:05:41.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.038+0000 7f7eabccc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ea4100e20 0x7f7ea41a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:41.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.038+0000 7f7eabccc640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7ea4102570 0x7f7ea41a4c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:41.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.038+0000 7f7eabccc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7ea4102ef0 0x7f7ea419ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:41.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.038+0000 7f7eabccc640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7ea4110f80 con 0x7f7ea4100e20 2024-09-12T02:05:41.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.038+0000 7f7eabccc640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7ea4110e00 con 0x7f7ea4102ef0 2024-09-12T02:05:41.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.038+0000 7f7eabccc640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7ea4111100 con 0x7f7ea4102570 2024-09-12T02:05:41.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.038+0000 7f7ea9a68640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ea4100e20 0x7f7ea41a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:41.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.038+0000 7f7ea9a68640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ea4100e20 0x7f7ea41a46f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54830/0 (socket says 172.21.15.16:54830) 2024-09-12T02:05:41.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.038+0000 7f7ea9a68640 1 -- 172.21.15.16:0/2361106981 learned_addr learned my addr 172.21.15.16:0/2361106981 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:05:41.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.038+0000 7f7ea9267640 1 --2- 172.21.15.16:0/2361106981 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7ea4102570 0x7f7ea41a4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:41.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.038+0000 7f7eaa269640 1 --2- 172.21.15.16:0/2361106981 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7ea4102ef0 0x7f7ea419ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:41.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.038+0000 7f7ea9a68640 1 -- 172.21.15.16:0/2361106981 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7ea4102570 msgr2=0x7f7ea41a4c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:41.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.038+0000 7f7ea9a68640 1 --2- 172.21.15.16:0/2361106981 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7ea4102570 0x7f7ea41a4c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:41.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.038+0000 7f7ea9a68640 1 -- 172.21.15.16:0/2361106981 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7ea4102ef0 msgr2=0x7f7ea419ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:41.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.038+0000 7f7ea9a68640 1 --2- 172.21.15.16:0/2361106981 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7ea4102ef0 0x7f7ea419ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:41.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.038+0000 7f7ea9a68640 1 -- 172.21.15.16:0/2361106981 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ea419f350 con 0x7f7ea4100e20 2024-09-12T02:05:41.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.038+0000 7f7ea9a68640 1 --2- 172.21.15.16:0/2361106981 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ea4100e20 0x7f7ea41a46f0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f7ea000e9f0 tx=0x7f7ea000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:41.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.039+0000 7f7e9affd640 1 -- 172.21.15.16:0/2361106981 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ea000cd80 con 0x7f7ea4100e20 2024-09-12T02:05:41.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.039+0000 7f7eabccc640 1 -- 172.21.15.16:0/2361106981 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ea419f640 con 0x7f7ea4100e20 2024-09-12T02:05:41.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.039+0000 7f7e9affd640 1 -- 172.21.15.16:0/2361106981 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ea0010910 con 0x7f7ea4100e20 2024-09-12T02:05:41.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.039+0000 7f7e9affd640 1 -- 172.21.15.16:0/2361106981 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ea0018990 con 0x7f7ea4100e20 2024-09-12T02:05:41.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.039+0000 7f7eabccc640 1 -- 172.21.15.16:0/2361106981 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ea41ab5c0 con 0x7f7ea4100e20 2024-09-12T02:05:41.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.041+0000 7f7e9affd640 1 -- 172.21.15.16:0/2361106981 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f7ea0010ac0 con 0x7f7ea4100e20 2024-09-12T02:05:41.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.041+0000 7f7eae0a5740 1 Processor -- start 2024-09-12T02:05:41.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.042+0000 7f7eae0a5740 1 -- start start 2024-09-12T02:05:41.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.042+0000 7f7eae0a5740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55869a2b5b40 0x55869a2b4090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:41.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.042+0000 7f7eae0a5740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55869a2b45d0 0x55869a2b4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:41.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.042+0000 7f7eae0a5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55869a2bf370 0x55869a2c1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:41.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.042+0000 7f7eae0a5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55869a21a2f0 con 0x55869a2bf370 2024-09-12T02:05:41.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.042+0000 7f7eae0a5740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55869a1c1cf0 con 0x55869a2b5b40 2024-09-12T02:05:41.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.042+0000 7f7eae0a5740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55869a2162b0 con 0x55869a2b45d0 2024-09-12T02:05:41.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.042+0000 7f7e9affd640 1 --2- 172.21.15.16:0/2361106981 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f7e900a2c00 0x7f7e900a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:41.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.042+0000 7f7ea9267640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55869a2bf370 0x55869a2c1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:41.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.042+0000 7f7ea9267640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55869a2bf370 0x55869a2c1760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54836/0 (socket says 172.21.15.16:54836) 2024-09-12T02:05:41.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.042+0000 7f7ea9267640 1 -- 172.21.15.16:0/1392883361 learned_addr learned my addr 172.21.15.16:0/1392883361 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:05:41.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.042+0000 7f7ea9a68640 1 --2- 172.21.15.16:0/1392883361 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55869a2b45d0 0x55869a2b4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:41.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.042+0000 7f7eaa269640 1 --2- 172.21.15.16:0/2361106981 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f7e900a2c00 0x7f7e900a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:41.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.042+0000 7f7eaa269640 1 --2- 172.21.15.16:0/1392883361 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55869a2b5b40 0x55869a2b4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:41.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.042+0000 7f7e9affd640 1 -- 172.21.15.16:0/2361106981 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f7ea00cf400 con 0x7f7ea4100e20 2024-09-12T02:05:41.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.043+0000 7f7ea9267640 1 -- 172.21.15.16:0/1392883361 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55869a2b45d0 msgr2=0x55869a2b4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:41.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.043+0000 7f7ea9267640 1 --2- 172.21.15.16:0/1392883361 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55869a2b45d0 0x55869a2b4a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:41.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.043+0000 7f7ea9267640 1 -- 172.21.15.16:0/1392883361 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55869a2b5b40 msgr2=0x55869a2b4090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:41.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.043+0000 7f7ea9267640 1 --2- 172.21.15.16:0/1392883361 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55869a2b5b40 0x55869a2b4090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:41.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.043+0000 7f7ea9267640 1 -- 172.21.15.16:0/1392883361 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55869a1a63a0 con 0x55869a2bf370 2024-09-12T02:05:41.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.043+0000 7f7eaa269640 1 --2- 172.21.15.16:0/2361106981 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f7e900a2c00 0x7f7e900a50c0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c030040 tx=0x7f7e9c005d80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:41.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.043+0000 7f7ea9267640 1 --2- 172.21.15.16:0/1392883361 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55869a2bf370 0x55869a2c1760 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f7e9400bc50 tx=0x7f7e9400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=905e6a9c8bdb465e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:41.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.043+0000 7f7e8ffff640 1 -- 172.21.15.16:0/1392883361 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e94015070 con 0x55869a2bf370 2024-09-12T02:05:41.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.043+0000 7f7e8ffff640 1 -- 172.21.15.16:0/1392883361 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e9401a940 con 0x55869a2bf370 2024-09-12T02:05:41.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.043+0000 7f7e8ffff640 1 -- 172.21.15.16:0/1392883361 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e940119a0 con 0x55869a2bf370 2024-09-12T02:05:41.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.044+0000 7f7eae0a5740 1 -- 172.21.15.16:0/1392883361 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55869a2bf370 msgr2=0x55869a2c1760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:41.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.044+0000 7f7eae0a5740 1 --2- 172.21.15.16:0/1392883361 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55869a2bf370 0x55869a2c1760 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f7e9400bc50 tx=0x7f7e9400f730 comp rx=0 tx=0).stop 2024-09-12T02:05:41.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.044+0000 7f7eae0a5740 1 -- 172.21.15.16:0/1392883361 shutdown_connections 2024-09-12T02:05:41.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.044+0000 7f7eae0a5740 1 --2- 172.21.15.16:0/1392883361 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55869a2bf370 0x55869a2c1760 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:41.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.044+0000 7f7eae0a5740 1 --2- 172.21.15.16:0/1392883361 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55869a2b45d0 0x55869a2b4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:41.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.044+0000 7f7eae0a5740 1 --2- 172.21.15.16:0/1392883361 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55869a2b5b40 0x55869a2b4090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:41.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.044+0000 7f7eae0a5740 1 -- 172.21.15.16:0/1392883361 >> 172.21.15.16:0/1392883361 conn(0x55869a27d310 msgr2=0x55869a2b0c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:41.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.044+0000 7f7eae0a5740 1 -- 172.21.15.16:0/1392883361 shutdown_connections 2024-09-12T02:05:41.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.044+0000 7f7eae0a5740 1 -- 172.21.15.16:0/1392883361 wait complete. 2024-09-12T02:05:41.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.045+0000 7f7eae0a5740 1 -- start start 2024-09-12T02:05:41.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.045+0000 7f7eae0a5740 1 Processor -- start 2024-09-12T02:05:41.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.045+0000 7f7eae0a5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55869a2b45d0 0x55869a2bed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:41.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.045+0000 7f7eae0a5740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55869a2b5b40 0x55869a2bc010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:41.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.045+0000 7f7eae0a5740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55869a2bf370 0x55869a2bc600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:41.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.045+0000 7f7eae0a5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55869a2c2070 con 0x55869a2b45d0 2024-09-12T02:05:41.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.045+0000 7f7eae0a5740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55869a1d5060 con 0x55869a2bf370 2024-09-12T02:05:41.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.045+0000 7f7eae0a5740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55869a2c21f0 con 0x55869a2b5b40 2024-09-12T02:05:41.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.045+0000 7f7eae0a5740 10 client.0 _ll_register_callbacks cb 0x558699f288c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:05:41.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.045+0000 7f7ea9a68640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55869a2b45d0 0x55869a2bed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:41.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.045+0000 7f7ea9a68640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55869a2b45d0 0x55869a2bed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54850/0 (socket says 172.21.15.16:54850) 2024-09-12T02:05:41.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.045+0000 7f7ea9a68640 1 -- 172.21.15.16:0/456238398 learned_addr learned my addr 172.21.15.16:0/456238398 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:05:41.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.045+0000 7f7eaa269640 1 --2- 172.21.15.16:0/456238398 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55869a2bf370 0x55869a2bc600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:41.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.045+0000 7f7ea9267640 1 --2- 172.21.15.16:0/456238398 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55869a2b5b40 0x55869a2bc010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:41.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.046+0000 7f7ea9a68640 1 -- 172.21.15.16:0/456238398 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55869a2b5b40 msgr2=0x55869a2bc010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:41.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.046+0000 7f7ea9a68640 1 --2- 172.21.15.16:0/456238398 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55869a2b5b40 0x55869a2bc010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:41.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.046+0000 7f7ea9a68640 1 -- 172.21.15.16:0/456238398 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55869a2bf370 msgr2=0x55869a2bc600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:41.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.046+0000 7f7ea9a68640 1 --2- 172.21.15.16:0/456238398 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55869a2bf370 0x55869a2bc600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:41.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.046+0000 7f7ea9a68640 1 -- 172.21.15.16:0/456238398 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558699f8b3c0 con 0x55869a2b45d0 2024-09-12T02:05:41.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.046+0000 7f7eaa269640 1 --2- 172.21.15.16:0/456238398 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55869a2bf370 0x55869a2bc600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:05:41.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.046+0000 7f7ea9a68640 1 --2- 172.21.15.16:0/456238398 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55869a2b45d0 0x55869a2bed60 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f7ea0098630 tx=0x7f7ea006aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b5f44de09c8d615 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:41.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.046+0000 7f7e8d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:05:41.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.046+0000 7f7e8d7fa640 1 -- 172.21.15.16:0/456238398 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ea0018d30 con 0x55869a2b45d0 2024-09-12T02:05:41.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.046+0000 7f7e8d7fa640 1 -- 172.21.15.16:0/456238398 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f7ea009b0b0 con 0x55869a2b45d0 2024-09-12T02:05:41.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.046+0000 7f7eae0a5740 1 -- 172.21.15.16:0/456238398 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e9401a940 con 0x55869a2b45d0 2024-09-12T02:05:41.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.046+0000 7f7e8d7fa640 1 -- 172.21.15.16:0/456238398 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7ea00b4e60 con 0x55869a2b45d0 2024-09-12T02:05:41.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.046+0000 7f7eae0a5740 10 client.16491 fetch_fsmap learned FSMap version 25 2024-09-12T02:05:41.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.046+0000 7f7eae0a5740 1 -- 172.21.15.16:0/456238398 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55869a2cdb40 con 0x55869a2b45d0 2024-09-12T02:05:41.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.046+0000 7f7e8d7fa640 1 -- 172.21.15.16:0/456238398 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ea00b1020 con 0x55869a2b45d0 2024-09-12T02:05:41.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.047+0000 7f7e8d7fa640 1 -- 172.21.15.16:0/456238398 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f7ea00979b0 con 0x55869a2b45d0 2024-09-12T02:05:41.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.047+0000 7f7eae0a5740 10 client.16491 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:05:41.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.047+0000 7f7eae0a5740 10 client.16491 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T02:05:41.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.047+0000 7f7eae0a5740 20 client.16491 populate_metadata read hostname 'smithi016' 2024-09-12T02:05:41.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.047+0000 7f7eae0a5740 1 --2- 172.21.15.16:0/456238398 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x55869a2cf000 0x55869a2d1470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:41.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.047+0000 7f7eae0a5740 4 client.16491 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:05:41.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.047+0000 7f7eae0a5740 1 -- 172.21.15.16:0/456238398 --> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55869a227860 con 0x55869a2cf000 2024-09-12T02:05:41.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.047+0000 7f7ea9267640 1 --2- 172.21.15.16:0/456238398 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x55869a2cf000 0x55869a2d1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:41.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.047+0000 7f7ea9267640 1 --2- 172.21.15.16:0/456238398 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x55869a2cf000 0x55869a2d1470 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3521db872f208578 server_cookie=ce841eef4e08e40e in_seq=0 out_seq=0 2024-09-12T02:05:41.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.047+0000 7f7e8d7fa640 10 client.16491 ms_handle_connect on v2:172.21.15.184:6838/4203347941 2024-09-12T02:05:41.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.048+0000 7f7e8d7fa640 1 -- 172.21.15.16:0/456238398 <== mds.1 v2:172.21.15.184:6838/4203347941 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55869a227860 con 0x55869a2cf000 2024-09-12T02:05:41.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.048+0000 7f7e8d7fa640 10 client.16491 handle_command_reply: tid=0 2024-09-12T02:05:41.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.048+0000 7f7eae0a5740 1 client.16491 shutdown 2024-09-12T02:05:41.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.048+0000 7f7eae0a5740 1 -- 172.21.15.16:0/456238398 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55869a2b45d0 msgr2=0x55869a2bed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:41.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.048+0000 7f7eae0a5740 1 --2- 172.21.15.16:0/456238398 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55869a2b45d0 0x55869a2bed60 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f7ea0098630 tx=0x7f7ea006aca0 comp rx=0 tx=0).stop 2024-09-12T02:05:41.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.048+0000 7f7eae0a5740 1 -- 172.21.15.16:0/456238398 shutdown_connections 2024-09-12T02:05:41.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.048+0000 7f7eae0a5740 1 --2- 172.21.15.16:0/456238398 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x55869a2cf000 0x55869a2d1470 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:41.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.048+0000 7f7eae0a5740 1 --2- 172.21.15.16:0/456238398 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55869a2bf370 0x55869a2bc600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:41.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.048+0000 7f7eae0a5740 1 --2- 172.21.15.16:0/456238398 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55869a2b5b40 0x55869a2bc010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:41.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.048+0000 7f7eae0a5740 1 --2- 172.21.15.16:0/456238398 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55869a2b45d0 0x55869a2bed60 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:41.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.048+0000 7f7e8d7fa640 0 client.16491 ms_handle_reset on v2:172.21.15.184:6838/4203347941 2024-09-12T02:05:41.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.048+0000 7f7eae0a5740 1 -- 172.21.15.16:0/456238398 >> 172.21.15.16:0/456238398 conn(0x55869a27d310 msgr2=0x55869a2b6960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:41.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.048+0000 7f7eae0a5740 1 -- 172.21.15.16:0/456238398 shutdown_connections 2024-09-12T02:05:41.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.048+0000 7f7eae0a5740 1 -- 172.21.15.16:0/456238398 wait complete. 2024-09-12T02:05:41.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.048+0000 7f7eae0a5740 20 client.16491 trim_cache size 0 max 16384 2024-09-12T02:05:41.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.062+0000 7f7eae0a5740 1 Processor -- start 2024-09-12T02:05:41.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.062+0000 7f7eae0a5740 1 -- start start 2024-09-12T02:05:41.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.062+0000 7f7eae0a5740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55869a2b45d0 0x55869a2cb340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:41.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.062+0000 7f7eae0a5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55869a2b5b40 0x55869a2c0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:41.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.062+0000 7f7eae0a5740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55869a2bf370 0x55869a2c0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:41.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.062+0000 7f7eae0a5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7ea41b6320 con 0x55869a2b5b40 2024-09-12T02:05:41.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.062+0000 7f7eae0a5740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7ea41b3bf0 con 0x55869a2b45d0 2024-09-12T02:05:41.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.062+0000 7f7eae0a5740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7ea41b1a40 con 0x55869a2bf370 2024-09-12T02:05:41.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.062+0000 7f7ea9267640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55869a2b5b40 0x55869a2c0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:41.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.062+0000 7f7ea9267640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55869a2b5b40 0x55869a2c0760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54864/0 (socket says 172.21.15.16:54864) 2024-09-12T02:05:41.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.062+0000 7f7ea9267640 1 -- 172.21.15.16:0/2544561691 learned_addr learned my addr 172.21.15.16:0/2544561691 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:05:41.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.062+0000 7f7eaa269640 1 --2- 172.21.15.16:0/2544561691 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55869a2bf370 0x55869a2c0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:41.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.062+0000 7f7ea9a68640 1 --2- 172.21.15.16:0/2544561691 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55869a2b45d0 0x55869a2cb340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:41.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.063+0000 7f7ea9267640 1 -- 172.21.15.16:0/2544561691 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55869a2bf370 msgr2=0x55869a2c0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:41.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.063+0000 7f7ea9267640 1 --2- 172.21.15.16:0/2544561691 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55869a2bf370 0x55869a2c0ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:41.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.063+0000 7f7ea9267640 1 -- 172.21.15.16:0/2544561691 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55869a2b45d0 msgr2=0x55869a2cb340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:41.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.063+0000 7f7ea9267640 1 --2- 172.21.15.16:0/2544561691 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55869a2b45d0 0x55869a2cb340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:41.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.063+0000 7f7ea9267640 1 -- 172.21.15.16:0/2544561691 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55869a2b0ed0 con 0x55869a2b5b40 2024-09-12T02:05:41.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.063+0000 7f7ea9267640 1 --2- 172.21.15.16:0/2544561691 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55869a2b5b40 0x55869a2c0760 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f7e94011570 tx=0x7f7e94011440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=763b000d5e16adec server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:41.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.063+0000 7f7e8f7fe640 1 -- 172.21.15.16:0/2544561691 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e94015040 con 0x55869a2b5b40 2024-09-12T02:05:41.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.063+0000 7f7e8f7fe640 1 -- 172.21.15.16:0/2544561691 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e9400be60 con 0x55869a2b5b40 2024-09-12T02:05:41.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.063+0000 7f7e8f7fe640 1 -- 172.21.15.16:0/2544561691 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e94004640 con 0x55869a2b5b40 2024-09-12T02:05:41.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.064+0000 7f7eae0a5740 1 -- 172.21.15.16:0/2544561691 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55869a2b5b40 msgr2=0x55869a2c0760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:41.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.064+0000 7f7eae0a5740 1 --2- 172.21.15.16:0/2544561691 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55869a2b5b40 0x55869a2c0760 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f7e94011570 tx=0x7f7e94011440 comp rx=0 tx=0).stop 2024-09-12T02:05:41.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.064+0000 7f7eae0a5740 1 -- 172.21.15.16:0/2544561691 shutdown_connections 2024-09-12T02:05:41.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.064+0000 7f7eae0a5740 1 --2- 172.21.15.16:0/2544561691 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55869a2bf370 0x55869a2c0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:41.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.064+0000 7f7eae0a5740 1 --2- 172.21.15.16:0/2544561691 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55869a2b5b40 0x55869a2c0760 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:41.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.064+0000 7f7eae0a5740 1 --2- 172.21.15.16:0/2544561691 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55869a2b45d0 0x55869a2cb340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:41.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.064+0000 7f7eae0a5740 1 -- 172.21.15.16:0/2544561691 >> 172.21.15.16:0/2544561691 conn(0x55869a27d310 msgr2=0x55869a2ba8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:41.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.064+0000 7f7eae0a5740 1 -- 172.21.15.16:0/2544561691 shutdown_connections 2024-09-12T02:05:41.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.064+0000 7f7eae0a5740 1 -- 172.21.15.16:0/2544561691 wait complete. 2024-09-12T02:05:41.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.064+0000 7f7eae0a5740 1 -- start start 2024-09-12T02:05:41.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.064+0000 7f7eae0a5740 1 Processor -- start 2024-09-12T02:05:41.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.065+0000 7f7eae0a5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55869a2b45d0 0x55869a2c7c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:41.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.065+0000 7f7eae0a5740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55869a2b5b40 0x55869a2d6650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:41.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.065+0000 7f7eae0a5740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55869a2bf370 0x55869a2d6b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:41.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.065+0000 7f7eae0a5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7ea41b20b0 con 0x55869a2b45d0 2024-09-12T02:05:41.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.065+0000 7f7eae0a5740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7ea41b3520 con 0x55869a2bf370 2024-09-12T02:05:41.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.065+0000 7f7eae0a5740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7ea41b2780 con 0x55869a2b5b40 2024-09-12T02:05:41.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.065+0000 7f7eae0a5740 10 client.0 _ll_register_callbacks cb 0x7f7ea0096680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:05:41.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.065+0000 7f7ea9a68640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55869a2b45d0 0x55869a2c7c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:41.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.065+0000 7f7ea9267640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55869a2b5b40 0x55869a2d6650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:41.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.065+0000 7f7ea9a68640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55869a2b45d0 0x55869a2c7c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54878/0 (socket says 172.21.15.16:54878) 2024-09-12T02:05:41.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.065+0000 7f7ea9a68640 1 -- 172.21.15.16:0/3885951369 learned_addr learned my addr 172.21.15.16:0/3885951369 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:05:41.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.065+0000 7f7eaa269640 1 --2- 172.21.15.16:0/3885951369 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55869a2bf370 0x55869a2d6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:41.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.065+0000 7f7ea9267640 1 -- 172.21.15.16:0/3885951369 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55869a2bf370 msgr2=0x55869a2d6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:41.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.065+0000 7f7ea9267640 1 --2- 172.21.15.16:0/3885951369 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55869a2bf370 0x55869a2d6b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:41.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.065+0000 7f7ea9267640 1 -- 172.21.15.16:0/3885951369 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55869a2b45d0 msgr2=0x55869a2c7c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:41.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.065+0000 7f7ea9267640 1 --2- 172.21.15.16:0/3885951369 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55869a2b45d0 0x55869a2c7c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:41.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.065+0000 7f7ea9267640 1 -- 172.21.15.16:0/3885951369 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55869a2d7370 con 0x55869a2b5b40 2024-09-12T02:05:41.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.065+0000 7f7ea9a68640 1 --2- 172.21.15.16:0/3885951369 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55869a2b45d0 0x55869a2c7c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:05:41.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.065+0000 7f7eaa269640 1 --2- 172.21.15.16:0/3885951369 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55869a2bf370 0x55869a2d6b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:05:41.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.066+0000 7f7ea9267640 1 --2- 172.21.15.16:0/3885951369 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55869a2b5b40 0x55869a2d6650 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x55869a2b5ff0 tx=0x7f7e9400b9d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a01dd2af0b712504 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:41.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.066+0000 7f7e8d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-09-12T02:05:41.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.066+0000 7f7e8d7fa640 1 -- 172.21.15.16:0/3885951369 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e9401a660 con 0x55869a2b5b40 2024-09-12T02:05:41.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.066+0000 7f7eae0a5740 1 -- 172.21.15.16:0/3885951369 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e9400be60 con 0x55869a2b5b40 2024-09-12T02:05:41.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.066+0000 7f7e8d7fa640 1 -- 172.21.15.16:0/3885951369 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f7e9401aaf0 con 0x55869a2b5b40 2024-09-12T02:05:41.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.066+0000 7f7e8d7fa640 1 -- 172.21.15.16:0/3885951369 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7e9401adb0 con 0x55869a2b5b40 2024-09-12T02:05:41.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.066+0000 7f7eae0a5740 10 client.25912 fetch_fsmap learned FSMap version 25 2024-09-12T02:05:41.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.066+0000 7f7eae0a5740 1 -- 172.21.15.16:0/3885951369 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55869a339d10 con 0x55869a2b5b40 2024-09-12T02:05:41.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.066+0000 7f7e8d7fa640 1 -- 172.21.15.16:0/3885951369 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e9400be60 con 0x55869a2b5b40 2024-09-12T02:05:41.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.066+0000 7f7e8d7fa640 1 -- 172.21.15.16:0/3885951369 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f7e940115a0 con 0x55869a2b5b40 2024-09-12T02:05:41.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.066+0000 7f7eae0a5740 10 client.25912 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:05:41.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.066+0000 7f7eae0a5740 10 client.25912 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T02:05:41.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.066+0000 7f7eae0a5740 20 client.25912 populate_metadata read hostname 'smithi016' 2024-09-12T02:05:41.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.066+0000 7f7eae0a5740 1 --2- 172.21.15.16:0/3885951369 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x55869a33a1e0 0x55869a33c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:41.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.066+0000 7f7eae0a5740 4 client.25912 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T02:05:41.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.066+0000 7f7eae0a5740 1 -- 172.21.15.16:0/3885951369 --> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55869a221d70 con 0x55869a33a1e0 2024-09-12T02:05:41.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.067+0000 7f7ea9a68640 1 --2- 172.21.15.16:0/3885951369 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x55869a33a1e0 0x55869a33c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:41.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.067+0000 7f7ea9a68640 1 --2- 172.21.15.16:0/3885951369 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x55869a33a1e0 0x55869a33c5d0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f99dab90d6629d18 server_cookie=bf9d723774f83c48 in_seq=0 out_seq=0 2024-09-12T02:05:41.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.067+0000 7f7e8d7fa640 10 client.25912 ms_handle_connect on v2:172.21.15.184:6838/4203347941 2024-09-12T02:05:41.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.067+0000 7f7e8d7fa640 1 -- 172.21.15.16:0/3885951369 <== mds.1 v2:172.21.15.184:6838/4203347941 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55869a221d70 con 0x55869a33a1e0 2024-09-12T02:05:41.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.067+0000 7f7e8d7fa640 10 client.25912 handle_command_reply: tid=0 2024-09-12T02:05:41.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.067+0000 7f7eae0a5740 1 client.25912 shutdown 2024-09-12T02:05:41.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.067+0000 7f7eae0a5740 1 -- 172.21.15.16:0/3885951369 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55869a2b5b40 msgr2=0x55869a2d6650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:41.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.067+0000 7f7eae0a5740 1 --2- 172.21.15.16:0/3885951369 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55869a2b5b40 0x55869a2d6650 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x55869a2b5ff0 tx=0x7f7e9400b9d0 comp rx=0 tx=0).stop 2024-09-12T02:05:41.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.068+0000 7f7eae0a5740 1 -- 172.21.15.16:0/3885951369 shutdown_connections 2024-09-12T02:05:41.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.068+0000 7f7eae0a5740 1 --2- 172.21.15.16:0/3885951369 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x55869a33a1e0 0x55869a33c5d0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:41.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.068+0000 7f7eae0a5740 1 --2- 172.21.15.16:0/3885951369 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55869a2bf370 0x55869a2d6b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:41.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.068+0000 7f7eae0a5740 1 --2- 172.21.15.16:0/3885951369 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55869a2b5b40 0x55869a2d6650 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:41.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.068+0000 7f7e8d7fa640 0 client.25912 ms_handle_reset on v2:172.21.15.184:6838/4203347941 2024-09-12T02:05:41.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.068+0000 7f7eae0a5740 1 --2- 172.21.15.16:0/3885951369 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55869a2b45d0 0x55869a2c7c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:41.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.068+0000 7f7eae0a5740 1 -- 172.21.15.16:0/3885951369 >> 172.21.15.16:0/3885951369 conn(0x55869a27d310 msgr2=0x55869a2bad00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:41.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.068+0000 7f7eae0a5740 1 -- 172.21.15.16:0/3885951369 shutdown_connections 2024-09-12T02:05:41.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.068+0000 7f7eae0a5740 1 -- 172.21.15.16:0/3885951369 wait complete. 2024-09-12T02:05:41.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.068+0000 7f7eae0a5740 20 client.25912 trim_cache size 0 max 16384 2024-09-12T02:05:41.076 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T02:05:41.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.068+0000 7f7eabccc640 1 -- 172.21.15.16:0/2361106981 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f7e900a2c00 msgr2=0x7f7e900a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:41.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.068+0000 7f7eabccc640 1 --2- 172.21.15.16:0/2361106981 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f7e900a2c00 0x7f7e900a50c0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c030040 tx=0x7f7e9c005d80 comp rx=0 tx=0).stop 2024-09-12T02:05:41.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.068+0000 7f7eabccc640 1 -- 172.21.15.16:0/2361106981 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ea4100e20 msgr2=0x7f7ea41a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:41.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.068+0000 7f7eabccc640 1 --2- 172.21.15.16:0/2361106981 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ea4100e20 0x7f7ea41a46f0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f7ea000e9f0 tx=0x7f7ea000eeb0 comp rx=0 tx=0).stop 2024-09-12T02:05:41.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.069+0000 7f7eabccc640 1 -- 172.21.15.16:0/2361106981 shutdown_connections 2024-09-12T02:05:41.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.069+0000 7f7eabccc640 1 --2- 172.21.15.16:0/2361106981 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f7e900a2c00 0x7f7e900a50c0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:41.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.069+0000 7f7eabccc640 1 --2- 172.21.15.16:0/2361106981 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7ea4102ef0 0x7f7ea419ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:41.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.069+0000 7f7eabccc640 1 --2- 172.21.15.16:0/2361106981 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7ea4102570 0x7f7ea41a4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:41.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.069+0000 7f7eabccc640 1 --2- 172.21.15.16:0/2361106981 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ea4100e20 0x7f7ea41a46f0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:41.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.069+0000 7f7eabccc640 1 -- 172.21.15.16:0/2361106981 >> 172.21.15.16:0/2361106981 conn(0x7f7ea40fc7d0 msgr2=0x7f7ea4069fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:41.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.069+0000 7f7eabccc640 1 -- 172.21.15.16:0/2361106981 shutdown_connections 2024-09-12T02:05:41.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:41.069+0000 7f7eabccc640 1 -- 172.21.15.16:0/2361106981 wait complete. 2024-09-12T02:05:41.104 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.102+0000 7f6565718640 1 -- 172.21.15.16:0/648801526 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6560104f00 msgr2=0x7f65601072f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:41.104 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.102+0000 7f6565718640 1 --2- 172.21.15.16:0/648801526 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6560104f00 0x7f65601072f0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f6548009900 tx=0x7f654802a410 comp rx=0 tx=0).stop 2024-09-12T02:05:41.104 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.102+0000 7f6565718640 1 -- 172.21.15.16:0/648801526 shutdown_connections 2024-09-12T02:05:41.105 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.102+0000 7f6565718640 1 --2- 172.21.15.16:0/648801526 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6560107830 0x7f6560109ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:41.105 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.102+0000 7f6565718640 1 --2- 172.21.15.16:0/648801526 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6560104f00 0x7f65601072f0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:41.105 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.102+0000 7f6565718640 1 --2- 172.21.15.16:0/648801526 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f65601025d0 0x7f65601049c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:41.105 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.102+0000 7f6565718640 1 -- 172.21.15.16:0/648801526 >> 172.21.15.16:0/648801526 conn(0x7f65600fc1b0 msgr2=0x7f65600fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:41.105 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.102+0000 7f6565718640 1 -- 172.21.15.16:0/648801526 shutdown_connections 2024-09-12T02:05:41.105 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.102+0000 7f6565718640 1 -- 172.21.15.16:0/648801526 wait complete. 2024-09-12T02:05:41.105 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.103+0000 7f6565718640 1 Processor -- start 2024-09-12T02:05:41.105 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.103+0000 7f6565718640 1 -- start start 2024-09-12T02:05:41.105 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.103+0000 7f6565718640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65601025d0 0x7f6560197e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:41.105 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.103+0000 7f6565718640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6560104f00 0x7f6560198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:41.105 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.103+0000 7f6565718640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6560107830 0x7f656019f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:41.106 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.103+0000 7f6565718640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f656010d7e0 con 0x7f65601025d0 2024-09-12T02:05:41.106 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.103+0000 7f6565718640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f656010d660 con 0x7f6560107830 2024-09-12T02:05:41.106 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.103+0000 7f6565718640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f656010d960 con 0x7f6560104f00 2024-09-12T02:05:41.106 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.103+0000 7f655effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65601025d0 0x7f6560197e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:41.106 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.103+0000 7f655effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65601025d0 0x7f6560197e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54900/0 (socket says 172.21.15.16:54900) 2024-09-12T02:05:41.106 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.103+0000 7f655effd640 1 -- 172.21.15.16:0/581946678 learned_addr learned my addr 172.21.15.16:0/581946678 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:05:41.106 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.103+0000 7f655f7fe640 1 --2- 172.21.15.16:0/581946678 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6560107830 0x7f656019f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:41.106 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.103+0000 7f6557fff640 1 --2- 172.21.15.16:0/581946678 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6560104f00 0x7f6560198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:41.106 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.103+0000 7f655effd640 1 -- 172.21.15.16:0/581946678 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6560104f00 msgr2=0x7f6560198390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:41.106 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.103+0000 7f655effd640 1 --2- 172.21.15.16:0/581946678 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6560104f00 0x7f6560198390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:41.106 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.103+0000 7f655effd640 1 -- 172.21.15.16:0/581946678 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6560107830 msgr2=0x7f656019f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:41.106 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.103+0000 7f655effd640 1 --2- 172.21.15.16:0/581946678 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6560107830 0x7f656019f400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:41.107 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.103+0000 7f655effd640 1 -- 172.21.15.16:0/581946678 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f656019fa70 con 0x7f65601025d0 2024-09-12T02:05:41.107 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.104+0000 7f655effd640 1 --2- 172.21.15.16:0/581946678 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65601025d0 0x7f6560197e50 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f655000d950 tx=0x7f655000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:41.107 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.104+0000 7f655cff9640 1 -- 172.21.15.16:0/581946678 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6550017030 con 0x7f65601025d0 2024-09-12T02:05:41.107 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.104+0000 7f655cff9640 1 -- 172.21.15.16:0/581946678 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6550004020 con 0x7f65601025d0 2024-09-12T02:05:41.107 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.104+0000 7f655cff9640 1 -- 172.21.15.16:0/581946678 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65500047a0 con 0x7f65601025d0 2024-09-12T02:05:41.107 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.104+0000 7f6565718640 1 -- 172.21.15.16:0/581946678 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f656019fd60 con 0x7f65601025d0 2024-09-12T02:05:41.107 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.104+0000 7f6565718640 1 -- 172.21.15.16:0/581946678 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65601a0210 con 0x7f65601025d0 2024-09-12T02:05:41.113 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.106+0000 7f655cff9640 1 -- 172.21.15.16:0/581946678 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f65500041d0 con 0x7f65601025d0 2024-09-12T02:05:41.113 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.106+0000 7f65684b3740 1 Processor -- start 2024-09-12T02:05:41.113 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.106+0000 7f65684b3740 1 -- start start 2024-09-12T02:05:41.113 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.107+0000 7f65684b3740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556346df8db0 0x556346df91b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:41.113 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.107+0000 7f65684b3740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556346df96f0 0x556346dfe1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:41.113 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.107+0000 7f65684b3740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556346dfe780 0x556346e00c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:41.113 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.107+0000 7f65684b3740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x556346d582f0 con 0x556346df8db0 2024-09-12T02:05:41.113 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.107+0000 7f65684b3740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x556346d00270 con 0x556346dfe780 2024-09-12T02:05:41.113 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.107+0000 7f65684b3740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x556346d542b0 con 0x556346df96f0 2024-09-12T02:05:41.114 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.107+0000 7f655f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556346df8db0 0x556346df91b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:41.114 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.107+0000 7f655f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556346df8db0 0x556346df91b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54912/0 (socket says 172.21.15.16:54912) 2024-09-12T02:05:41.114 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.107+0000 7f655f7fe640 1 -- 172.21.15.16:0/3729913594 learned_addr learned my addr 172.21.15.16:0/3729913594 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:05:41.114 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.107+0000 7f655effd640 1 --2- 172.21.15.16:0/3729913594 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556346df96f0 0x556346dfe1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:41.114 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.107+0000 7f6557fff640 1 --2- 172.21.15.16:0/3729913594 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556346dfe780 0x556346e00c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:41.114 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.107+0000 7f655f7fe640 1 -- 172.21.15.16:0/3729913594 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556346df96f0 msgr2=0x556346dfe1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:41.114 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.107+0000 7f655f7fe640 1 --2- 172.21.15.16:0/3729913594 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556346df96f0 0x556346dfe1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:41.114 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.107+0000 7f655cff9640 1 --2- 172.21.15.16:0/581946678 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f654c0a2cb0 0x7f654c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:41.114 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.107+0000 7f655f7fe640 1 -- 172.21.15.16:0/3729913594 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556346dfe780 msgr2=0x556346e00c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:41.114 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.107+0000 7f655f7fe640 1 --2- 172.21.15.16:0/3729913594 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556346dfe780 0x556346e00c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:41.114 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.107+0000 7f655f7fe640 1 -- 172.21.15.16:0/3729913594 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556346cff340 con 0x556346df8db0 2024-09-12T02:05:41.114 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.107+0000 7f655cff9640 1 -- 172.21.15.16:0/581946678 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f65500ce8c0 con 0x7f65601025d0 2024-09-12T02:05:41.114 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.108+0000 7f655f7fe640 1 --2- 172.21.15.16:0/581946678 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f654c0a2cb0 0x7f654c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:41.114 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.108+0000 7f655f7fe640 1 --2- 172.21.15.16:0/3729913594 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556346df8db0 0x556346df91b0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f6558009f60 tx=0x7f6558009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cca23ce3ac1166a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:41.115 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.108+0000 7f65557fa640 1 -- 172.21.15.16:0/3729913594 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6558003cb0 con 0x556346df8db0 2024-09-12T02:05:41.115 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.108+0000 7f65557fa640 1 -- 172.21.15.16:0/3729913594 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6558015070 con 0x556346df8db0 2024-09-12T02:05:41.115 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.108+0000 7f65557fa640 1 -- 172.21.15.16:0/3729913594 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f655801b400 con 0x556346df8db0 2024-09-12T02:05:41.115 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.109+0000 7f65684b3740 1 -- 172.21.15.16:0/3729913594 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556346df8db0 msgr2=0x556346df91b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:41.115 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.109+0000 7f655f7fe640 1 --2- 172.21.15.16:0/581946678 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f654c0a2cb0 0x7f654c0a5170 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f6558003fd0 tx=0x7f655801bb00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:41.115 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.109+0000 7f65684b3740 1 --2- 172.21.15.16:0/3729913594 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556346df8db0 0x556346df91b0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f6558009f60 tx=0x7f6558009f90 comp rx=0 tx=0).stop 2024-09-12T02:05:41.115 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.109+0000 7f65684b3740 1 -- 172.21.15.16:0/3729913594 shutdown_connections 2024-09-12T02:05:41.115 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.109+0000 7f65684b3740 1 --2- 172.21.15.16:0/3729913594 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556346dfe780 0x556346e00c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:41.115 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.109+0000 7f65684b3740 1 --2- 172.21.15.16:0/3729913594 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556346df96f0 0x556346dfe1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:41.115 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.109+0000 7f65684b3740 1 --2- 172.21.15.16:0/3729913594 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556346df8db0 0x556346df91b0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:41.115 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.109+0000 7f65684b3740 1 -- 172.21.15.16:0/3729913594 >> 172.21.15.16:0/3729913594 conn(0x556346df4420 msgr2=0x556346df4820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:41.116 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.109+0000 7f65684b3740 1 -- 172.21.15.16:0/3729913594 shutdown_connections 2024-09-12T02:05:41.116 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.109+0000 7f65684b3740 1 -- 172.21.15.16:0/3729913594 wait complete. 2024-09-12T02:05:41.116 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.109+0000 7f65684b3740 1 -- start start 2024-09-12T02:05:41.116 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.109+0000 7f65684b3740 1 Processor -- start 2024-09-12T02:05:41.116 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.109+0000 7f65684b3740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556346df8db0 0x556346e02f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:41.116 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.109+0000 7f65684b3740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556346df96f0 0x556346e03480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:41.116 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.109+0000 7f65684b3740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556346dfe780 0x556346e0a450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:41.116 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.109+0000 7f65684b3740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x556346e01750 con 0x556346df8db0 2024-09-12T02:05:41.116 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.109+0000 7f65684b3740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x556346ce1290 con 0x556346dfe780 2024-09-12T02:05:41.116 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.109+0000 7f65684b3740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x556346e018d0 con 0x556346df96f0 2024-09-12T02:05:41.116 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.109+0000 7f65684b3740 10 client.0 _ll_register_callbacks cb 0x556346d12140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:05:41.116 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.109+0000 7f655effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556346df8db0 0x556346e02f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:41.116 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.110+0000 7f655effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556346df8db0 0x556346e02f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54918/0 (socket says 172.21.15.16:54918) 2024-09-12T02:05:41.117 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.110+0000 7f655effd640 1 -- 172.21.15.16:0/3623209474 learned_addr learned my addr 172.21.15.16:0/3623209474 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:05:41.117 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.110+0000 7f655f7fe640 1 --2- 172.21.15.16:0/3623209474 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556346dfe780 0x556346e0a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:41.117 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.110+0000 7f6557fff640 1 --2- 172.21.15.16:0/3623209474 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556346df96f0 0x556346e03480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:41.117 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.110+0000 7f655effd640 1 -- 172.21.15.16:0/3623209474 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556346df96f0 msgr2=0x556346e03480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:41.117 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.110+0000 7f655effd640 1 --2- 172.21.15.16:0/3623209474 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556346df96f0 0x556346e03480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:41.117 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.110+0000 7f655effd640 1 -- 172.21.15.16:0/3623209474 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556346dfe780 msgr2=0x556346e0a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:41.117 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.110+0000 7f655effd640 1 --2- 172.21.15.16:0/3623209474 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556346dfe780 0x556346e0a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:41.117 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.110+0000 7f655effd640 1 -- 172.21.15.16:0/3623209474 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556346a6b210 con 0x556346df8db0 2024-09-12T02:05:41.117 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.110+0000 7f655effd640 1 --2- 172.21.15.16:0/3623209474 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556346df8db0 0x556346e02f40 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f6550009610 tx=0x7f65500051f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb0c52fde00ac47a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:41.117 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.110+0000 7f65467fc640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:05:41.117 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.110+0000 7f65467fc640 1 -- 172.21.15.16:0/3623209474 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f655009ab20 con 0x556346df8db0 2024-09-12T02:05:41.117 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.110+0000 7f65467fc640 1 -- 172.21.15.16:0/3623209474 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f6550097830 con 0x556346df8db0 2024-09-12T02:05:41.118 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.110+0000 7f65684b3740 1 -- 172.21.15.16:0/3623209474 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6558015070 con 0x556346df8db0 2024-09-12T02:05:41.118 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.110+0000 7f65467fc640 1 -- 172.21.15.16:0/3623209474 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f655009b8c0 con 0x556346df8db0 2024-09-12T02:05:41.118 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.110+0000 7f65684b3740 10 client.16509 fetch_fsmap learned FSMap version 25 2024-09-12T02:05:41.118 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.110+0000 7f65684b3740 1 -- 172.21.15.16:0/3623209474 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x556346b37620 con 0x556346df8db0 2024-09-12T02:05:41.118 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.110+0000 7f65467fc640 1 -- 172.21.15.16:0/3623209474 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6558015070 con 0x556346df8db0 2024-09-12T02:05:41.118 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.111+0000 7f65467fc640 1 -- 172.21.15.16:0/3623209474 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f655009f130 con 0x556346df8db0 2024-09-12T02:05:41.118 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.111+0000 7f65684b3740 10 client.16509 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:05:41.118 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.111+0000 7f65684b3740 10 client.16509 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:05:41.118 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.111+0000 7f65684b3740 20 client.16509 populate_metadata read hostname 'smithi016' 2024-09-12T02:05:41.118 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.111+0000 7f65684b3740 1 --2- 172.21.15.16:0/3623209474 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x556346e0eb40 0x556346e10f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:41.118 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.111+0000 7f65684b3740 4 client.16509 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:05:41.118 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.111+0000 7f65684b3740 1 -- 172.21.15.16:0/3623209474 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556346d65a20 con 0x556346e0eb40 2024-09-12T02:05:41.118 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.111+0000 7f6557fff640 1 --2- 172.21.15.16:0/3623209474 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x556346e0eb40 0x556346e10f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:41.119 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.112+0000 7f6557fff640 1 --2- 172.21.15.16:0/3623209474 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x556346e0eb40 0x556346e10f50 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc014c8cc28fb9eb server_cookie=626a1925bd8b2288 in_seq=0 out_seq=0 2024-09-12T02:05:41.119 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.112+0000 7f65467fc640 10 client.16509 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:05:41.119 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.112+0000 7f65467fc640 1 -- 172.21.15.16:0/3623209474 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556346d65a20 con 0x556346e0eb40 2024-09-12T02:05:41.119 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.112+0000 7f65467fc640 10 client.16509 handle_command_reply: tid=0 2024-09-12T02:05:41.119 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.112+0000 7f65684b3740 1 client.16509 shutdown 2024-09-12T02:05:41.119 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.112+0000 7f65684b3740 1 -- 172.21.15.16:0/3623209474 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556346df8db0 msgr2=0x556346e02f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:41.119 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.112+0000 7f65684b3740 1 --2- 172.21.15.16:0/3623209474 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556346df8db0 0x556346e02f40 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f6550009610 tx=0x7f65500051f0 comp rx=0 tx=0).stop 2024-09-12T02:05:41.119 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.112+0000 7f65684b3740 1 -- 172.21.15.16:0/3623209474 shutdown_connections 2024-09-12T02:05:41.119 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.112+0000 7f65684b3740 1 --2- 172.21.15.16:0/3623209474 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x556346e0eb40 0x556346e10f50 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:41.119 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.112+0000 7f65684b3740 1 --2- 172.21.15.16:0/3623209474 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556346dfe780 0x556346e0a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:41.119 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.112+0000 7f65684b3740 1 --2- 172.21.15.16:0/3623209474 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556346df96f0 0x556346e03480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:41.119 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.112+0000 7f65467fc640 0 client.16509 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:05:41.119 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.112+0000 7f65684b3740 1 --2- 172.21.15.16:0/3623209474 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556346df8db0 0x556346e02f40 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:41.120 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.112+0000 7f65684b3740 1 -- 172.21.15.16:0/3623209474 >> 172.21.15.16:0/3623209474 conn(0x556346df4420 msgr2=0x556346def1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:41.120 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.113+0000 7f65684b3740 1 -- 172.21.15.16:0/3623209474 shutdown_connections 2024-09-12T02:05:41.120 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.113+0000 7f65684b3740 1 -- 172.21.15.16:0/3623209474 wait complete. 2024-09-12T02:05:41.120 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.113+0000 7f65684b3740 20 client.16509 trim_cache size 0 max 16384 2024-09-12T02:05:41.128 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.127+0000 7f65684b3740 1 Processor -- start 2024-09-12T02:05:41.128 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.127+0000 7f65684b3740 1 -- start start 2024-09-12T02:05:41.128 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.127+0000 7f65684b3740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556346df8db0 0x556346e03400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:41.128 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.127+0000 7f65684b3740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556346df96f0 0x556346e0a410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:41.128 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.127+0000 7f65684b3740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556346dfe780 0x556346df5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:41.128 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.127+0000 7f65684b3740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f65601a95c0 con 0x556346df96f0 2024-09-12T02:05:41.128 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.127+0000 7f65684b3740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f65601a6e90 con 0x556346df8db0 2024-09-12T02:05:41.128 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.127+0000 7f65684b3740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f65601a4d00 con 0x556346dfe780 2024-09-12T02:05:41.128 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.127+0000 7f6557fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556346df96f0 0x556346e0a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:41.129 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.127+0000 7f6557fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556346df96f0 0x556346e0a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54930/0 (socket says 172.21.15.16:54930) 2024-09-12T02:05:41.129 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.127+0000 7f6557fff640 1 -- 172.21.15.16:0/2060393690 learned_addr learned my addr 172.21.15.16:0/2060393690 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:05:41.129 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.127+0000 7f655effd640 1 --2- 172.21.15.16:0/2060393690 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556346df8db0 0x556346e03400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:41.129 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.127+0000 7f6557fff640 1 -- 172.21.15.16:0/2060393690 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556346dfe780 msgr2=0x556346df5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:41.129 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.127+0000 7f6557fff640 1 --2- 172.21.15.16:0/2060393690 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556346dfe780 0x556346df5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:41.129 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.127+0000 7f6557fff640 1 -- 172.21.15.16:0/2060393690 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556346df8db0 msgr2=0x556346e03400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:41.129 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.127+0000 7f6557fff640 1 --2- 172.21.15.16:0/2060393690 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556346df8db0 0x556346e03400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:41.129 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.127+0000 7f6557fff640 1 -- 172.21.15.16:0/2060393690 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556346e02a80 con 0x556346df96f0 2024-09-12T02:05:41.129 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.128+0000 7f655effd640 1 --2- 172.21.15.16:0/2060393690 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556346df8db0 0x556346e03400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:05:41.129 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.128+0000 7f6557fff640 1 --2- 172.21.15.16:0/2060393690 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556346df96f0 0x556346e0a410 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f6548032fd0 tx=0x7f6548004330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fd036e85e498d0d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:41.129 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.128+0000 7f6554ff9640 1 -- 172.21.15.16:0/2060393690 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f654804a070 con 0x556346df96f0 2024-09-12T02:05:41.129 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.128+0000 7f6554ff9640 1 -- 172.21.15.16:0/2060393690 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6548005520 con 0x556346df96f0 2024-09-12T02:05:41.129 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.128+0000 7f6554ff9640 1 -- 172.21.15.16:0/2060393690 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6548035040 con 0x556346df96f0 2024-09-12T02:05:41.133 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.128+0000 7f65684b3740 1 -- 172.21.15.16:0/2060393690 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556346df96f0 msgr2=0x556346e0a410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:41.133 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.128+0000 7f65684b3740 1 --2- 172.21.15.16:0/2060393690 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556346df96f0 0x556346e0a410 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f6548032fd0 tx=0x7f6548004330 comp rx=0 tx=0).stop 2024-09-12T02:05:41.133 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.129+0000 7f65684b3740 1 -- 172.21.15.16:0/2060393690 shutdown_connections 2024-09-12T02:05:41.133 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.129+0000 7f65684b3740 1 --2- 172.21.15.16:0/2060393690 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556346dfe780 0x556346df5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:41.133 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.129+0000 7f65684b3740 1 --2- 172.21.15.16:0/2060393690 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556346df96f0 0x556346e0a410 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:41.133 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.129+0000 7f65684b3740 1 --2- 172.21.15.16:0/2060393690 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556346df8db0 0x556346e03400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:41.133 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.129+0000 7f65684b3740 1 -- 172.21.15.16:0/2060393690 >> 172.21.15.16:0/2060393690 conn(0x556346df4420 msgr2=0x556346e02cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:41.133 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.129+0000 7f65684b3740 1 -- 172.21.15.16:0/2060393690 shutdown_connections 2024-09-12T02:05:41.134 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.129+0000 7f65684b3740 1 -- 172.21.15.16:0/2060393690 wait complete. 2024-09-12T02:05:41.134 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.129+0000 7f65684b3740 1 -- start start 2024-09-12T02:05:41.134 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.129+0000 7f65684b3740 1 Processor -- start 2024-09-12T02:05:41.134 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.129+0000 7f65684b3740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556346df8db0 0x556346e07800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:41.134 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.129+0000 7f65684b3740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556346df96f0 0x556346e07d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:41.134 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.129+0000 7f65684b3740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556346dfe780 0x556346e08d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:41.134 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.129+0000 7f65684b3740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f65601a5350 con 0x556346dfe780 2024-09-12T02:05:41.134 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.129+0000 7f65684b3740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f65601a67c0 con 0x556346df96f0 2024-09-12T02:05:41.134 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.129+0000 7f65684b3740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f65601a5a20 con 0x556346df8db0 2024-09-12T02:05:41.134 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.129+0000 7f65684b3740 10 client.0 _ll_register_callbacks cb 0x7f65500046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:05:41.134 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.129+0000 7f655f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556346dfe780 0x556346e08d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:41.134 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.129+0000 7f655f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556346dfe780 0x556346e08d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54934/0 (socket says 172.21.15.16:54934) 2024-09-12T02:05:41.134 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.129+0000 7f655f7fe640 1 -- 172.21.15.16:0/3108335443 learned_addr learned my addr 172.21.15.16:0/3108335443 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:05:41.135 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.129+0000 7f655effd640 1 --2- 172.21.15.16:0/3108335443 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556346df8db0 0x556346e07800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:41.135 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.129+0000 7f6557fff640 1 --2- 172.21.15.16:0/3108335443 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556346df96f0 0x556346e07d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:41.135 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.130+0000 7f655f7fe640 1 -- 172.21.15.16:0/3108335443 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556346df8db0 msgr2=0x556346e07800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:41.135 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.130+0000 7f655f7fe640 1 --2- 172.21.15.16:0/3108335443 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556346df8db0 0x556346e07800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:41.135 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.130+0000 7f655f7fe640 1 -- 172.21.15.16:0/3108335443 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556346df96f0 msgr2=0x556346e07d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:41.135 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.130+0000 7f655f7fe640 1 --2- 172.21.15.16:0/3108335443 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556346df96f0 0x556346e07d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:41.135 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.130+0000 7f655f7fe640 1 -- 172.21.15.16:0/3108335443 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556346e09360 con 0x556346dfe780 2024-09-12T02:05:41.135 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.130+0000 7f6557fff640 1 --2- 172.21.15.16:0/3108335443 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556346df96f0 0x556346e07d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:05:41.135 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.130+0000 7f655f7fe640 1 --2- 172.21.15.16:0/3108335443 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556346dfe780 0x556346e08d50 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f6558010cf0 tx=0x7f655800d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df73712caa4c33b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:41.135 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.130+0000 7f65467fc640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:05:41.135 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.130+0000 7f65467fc640 1 -- 172.21.15.16:0/3108335443 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6558034020 con 0x556346dfe780 2024-09-12T02:05:41.135 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.130+0000 7f65467fc640 1 -- 172.21.15.16:0/3108335443 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f6558004260 con 0x556346dfe780 2024-09-12T02:05:41.135 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.130+0000 7f65684b3740 1 -- 172.21.15.16:0/3108335443 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6548005520 con 0x556346dfe780 2024-09-12T02:05:41.135 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.130+0000 7f65467fc640 1 -- 172.21.15.16:0/3108335443 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6558009b50 con 0x556346dfe780 2024-09-12T02:05:41.136 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.130+0000 7f65684b3740 10 client.16515 fetch_fsmap learned FSMap version 25 2024-09-12T02:05:41.136 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.130+0000 7f65684b3740 1 -- 172.21.15.16:0/3108335443 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x556346e0a650 con 0x556346dfe780 2024-09-12T02:05:41.136 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.130+0000 7f65467fc640 1 -- 172.21.15.16:0/3108335443 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6548005520 con 0x556346dfe780 2024-09-12T02:05:41.136 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.130+0000 7f65467fc640 1 -- 172.21.15.16:0/3108335443 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f6558011500 con 0x556346dfe780 2024-09-12T02:05:41.136 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.130+0000 7f65684b3740 10 client.16515 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:05:41.136 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.130+0000 7f65684b3740 10 client.16515 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:05:41.136 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.130+0000 7f65684b3740 20 client.16515 populate_metadata read hostname 'smithi016' 2024-09-12T02:05:41.136 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.130+0000 7f65684b3740 1 --2- 172.21.15.16:0/3108335443 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x556346e79c10 0x556346e7c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:41.136 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.130+0000 7f65684b3740 4 client.16515 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:05:41.136 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.130+0000 7f65684b3740 1 -- 172.21.15.16:0/3108335443 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556346d5fd70 con 0x556346e79c10 2024-09-12T02:05:41.136 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.131+0000 7f655effd640 1 --2- 172.21.15.16:0/3108335443 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x556346e79c10 0x556346e7c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:41.136 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.131+0000 7f65467fc640 10 client.16515 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:05:41.137 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.131+0000 7f655effd640 1 --2- 172.21.15.16:0/3108335443 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x556346e79c10 0x556346e7c030 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8abee8b9cc7c590 server_cookie=3acafe5a5ed69544 in_seq=0 out_seq=0 2024-09-12T02:05:41.144 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.144+0000 7f65467fc640 1 -- 172.21.15.16:0/3108335443 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556346d5fd70 con 0x556346e79c10 2024-09-12T02:05:41.144 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.144+0000 7f65467fc640 10 client.16515 handle_command_reply: tid=0 2024-09-12T02:05:41.145 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.144+0000 7f65684b3740 1 client.16515 shutdown 2024-09-12T02:05:41.145 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.144+0000 7f65684b3740 1 -- 172.21.15.16:0/3108335443 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556346dfe780 msgr2=0x556346e08d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:41.145 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.144+0000 7f65684b3740 1 --2- 172.21.15.16:0/3108335443 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556346dfe780 0x556346e08d50 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f6558010cf0 tx=0x7f655800d000 comp rx=0 tx=0).stop 2024-09-12T02:05:41.145 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.144+0000 7f65684b3740 1 -- 172.21.15.16:0/3108335443 shutdown_connections 2024-09-12T02:05:41.145 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.144+0000 7f65684b3740 1 --2- 172.21.15.16:0/3108335443 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x556346e79c10 0x556346e7c030 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:41.145 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.144+0000 7f65684b3740 1 --2- 172.21.15.16:0/3108335443 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556346dfe780 0x556346e08d50 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:41.145 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.144+0000 7f65684b3740 1 --2- 172.21.15.16:0/3108335443 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556346df96f0 0x556346e07d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:41.145 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.144+0000 7f65467fc640 0 client.16515 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:05:41.145 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.144+0000 7f65684b3740 1 --2- 172.21.15.16:0/3108335443 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556346df8db0 0x556346e07800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:41.145 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.144+0000 7f65684b3740 1 -- 172.21.15.16:0/3108335443 >> 172.21.15.16:0/3108335443 conn(0x556346df4420 msgr2=0x556346dee9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:41.145 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.144+0000 7f65684b3740 1 -- 172.21.15.16:0/3108335443 shutdown_connections 2024-09-12T02:05:41.145 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.144+0000 7f65684b3740 1 -- 172.21.15.16:0/3108335443 wait complete. 2024-09-12T02:05:41.145 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.144+0000 7f65684b3740 20 client.16515 trim_cache size 0 max 16384 2024-09-12T02:05:41.146 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-09-12T02:05:41.146 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-09-12T02:05:41.146 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-09-12T02:05:41.146 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-09-12T02:05:41.147 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.145+0000 7f6565718640 1 -- 172.21.15.16:0/581946678 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f654c0a2cb0 msgr2=0x7f654c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:41.147 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.145+0000 7f6565718640 1 --2- 172.21.15.16:0/581946678 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f654c0a2cb0 0x7f654c0a5170 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f6558003fd0 tx=0x7f655801bb00 comp rx=0 tx=0).stop 2024-09-12T02:05:41.147 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.145+0000 7f6565718640 1 -- 172.21.15.16:0/581946678 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65601025d0 msgr2=0x7f6560197e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:41.147 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.145+0000 7f6565718640 1 --2- 172.21.15.16:0/581946678 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65601025d0 0x7f6560197e50 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f655000d950 tx=0x7f655000de10 comp rx=0 tx=0).stop 2024-09-12T02:05:41.147 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.145+0000 7f6565718640 1 -- 172.21.15.16:0/581946678 shutdown_connections 2024-09-12T02:05:41.147 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.145+0000 7f6565718640 1 --2- 172.21.15.16:0/581946678 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f654c0a2cb0 0x7f654c0a5170 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:41.147 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.145+0000 7f6565718640 1 --2- 172.21.15.16:0/581946678 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6560107830 0x7f656019f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:41.147 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.145+0000 7f6565718640 1 --2- 172.21.15.16:0/581946678 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6560104f00 0x7f6560198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:41.147 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.145+0000 7f6565718640 1 --2- 172.21.15.16:0/581946678 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65601025d0 0x7f6560197e50 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:41.147 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.145+0000 7f6565718640 1 -- 172.21.15.16:0/581946678 >> 172.21.15.16:0/581946678 conn(0x7f65600fc1b0 msgr2=0x7f6560102ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:41.148 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.145+0000 7f6565718640 1 -- 172.21.15.16:0/581946678 shutdown_connections 2024-09-12T02:05:41.148 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:41.145+0000 7f6565718640 1 -- 172.21.15.16:0/581946678 wait complete. 2024-09-12T02:05:41.754 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-12T02:05:42.013 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:05:42.258 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:05:42 smithi016 ceph-mon[30947]: pgmap v448: 97 pgs: 97 active+clean; 2.1 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 12 op/s 2024-09-12T02:05:42.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:05:42 smithi184 ceph-mon[35593]: pgmap v448: 97 pgs: 97 active+clean; 2.1 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 12 op/s 2024-09-12T02:05:42.418 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:05:42 smithi033 ceph-mon[35332]: pgmap v448: 97 pgs: 97 active+clean; 2.1 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 12 op/s 2024-09-12T02:05:42.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.451+0000 7f138ae0d640 1 -- 172.21.15.16:0/2521106353 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f138410e4f0 msgr2=0x7f13841108e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:42.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.451+0000 7f138ae0d640 1 --2- 172.21.15.16:0/2521106353 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f138410e4f0 0x7f13841108e0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f137c009ca0 tx=0x7f137c02a8c0 comp rx=0 tx=0).stop 2024-09-12T02:05:42.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.451+0000 7f138ae0d640 1 -- 172.21.15.16:0/2521106353 shutdown_connections 2024-09-12T02:05:42.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.451+0000 7f138ae0d640 1 --2- 172.21.15.16:0/2521106353 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f138410e4f0 0x7f13841108e0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:42.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.451+0000 7f138ae0d640 1 --2- 172.21.15.16:0/2521106353 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f13841012f0 0x7f138410dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:42.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.451+0000 7f138ae0d640 1 --2- 172.21.15.16:0/2521106353 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1384100940 0x7f1384100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:42.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.451+0000 7f138ae0d640 1 -- 172.21.15.16:0/2521106353 >> 172.21.15.16:0/2521106353 conn(0x7f13840fc790 msgr2=0x7f13840febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:42.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.451+0000 7f138ae0d640 1 -- 172.21.15.16:0/2521106353 shutdown_connections 2024-09-12T02:05:42.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.451+0000 7f138ae0d640 1 -- 172.21.15.16:0/2521106353 wait complete. 2024-09-12T02:05:42.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.451+0000 7f138ae0d640 1 Processor -- start 2024-09-12T02:05:42.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.451+0000 7f138ae0d640 1 -- start start 2024-09-12T02:05:42.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.452+0000 7f138ae0d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1384100940 0x7f13841a0460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:42.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.452+0000 7f138ae0d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13841012f0 0x7f13841a09a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:42.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.452+0000 7f138ae0d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f138410e4f0 0x7f138419a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:42.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.452+0000 7f138ae0d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1384114230 con 0x7f13841012f0 2024-09-12T02:05:42.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.452+0000 7f138ae0d640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f13841140b0 con 0x7f1384100940 2024-09-12T02:05:42.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.452+0000 7f138ae0d640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f13841143b0 con 0x7f138410e4f0 2024-09-12T02:05:42.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.452+0000 7f1383fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13841012f0 0x7f13841a09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:42.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.452+0000 7f1383fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13841012f0 0x7f13841a09a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60316/0 (socket says 172.21.15.16:60316) 2024-09-12T02:05:42.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.452+0000 7f1383fff640 1 -- 172.21.15.16:0/2413962505 learned_addr learned my addr 172.21.15.16:0/2413962505 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:05:42.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.452+0000 7f13893aa640 1 --2- 172.21.15.16:0/2413962505 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f138410e4f0 0x7f138419a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:42.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.452+0000 7f1388ba9640 1 --2- 172.21.15.16:0/2413962505 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1384100940 0x7f13841a0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:42.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.452+0000 7f1383fff640 1 -- 172.21.15.16:0/2413962505 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f138410e4f0 msgr2=0x7f138419a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:42.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.452+0000 7f1383fff640 1 --2- 172.21.15.16:0/2413962505 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f138410e4f0 0x7f138419a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:42.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.452+0000 7f1383fff640 1 -- 172.21.15.16:0/2413962505 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1384100940 msgr2=0x7f13841a0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:42.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.452+0000 7f1383fff640 1 --2- 172.21.15.16:0/2413962505 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1384100940 0x7f13841a0460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:42.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.452+0000 7f1383fff640 1 -- 172.21.15.16:0/2413962505 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f138419ad50 con 0x7f13841012f0 2024-09-12T02:05:42.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.452+0000 7f1388ba9640 1 --2- 172.21.15.16:0/2413962505 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1384100940 0x7f13841a0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:05:42.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.453+0000 7f1383fff640 1 --2- 172.21.15.16:0/2413962505 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13841012f0 0x7f13841a09a0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f137800b4a0 tx=0x7f137800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:42.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.453+0000 7f1381ffb640 1 -- 172.21.15.16:0/2413962505 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1378014070 con 0x7f13841012f0 2024-09-12T02:05:42.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.453+0000 7f1381ffb640 1 -- 172.21.15.16:0/2413962505 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1378010af0 con 0x7f13841012f0 2024-09-12T02:05:42.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.453+0000 7f138ae0d640 1 -- 172.21.15.16:0/2413962505 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f138419b040 con 0x7f13841012f0 2024-09-12T02:05:42.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.453+0000 7f1381ffb640 1 -- 172.21.15.16:0/2413962505 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1378017d10 con 0x7f13841012f0 2024-09-12T02:05:42.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.453+0000 7f138ae0d640 1 -- 172.21.15.16:0/2413962505 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13841a7330 con 0x7f13841012f0 2024-09-12T02:05:42.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.455+0000 7f1381ffb640 1 -- 172.21.15.16:0/2413962505 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f1378017070 con 0x7f13841012f0 2024-09-12T02:05:42.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.455+0000 7f138d1e6740 1 Processor -- start 2024-09-12T02:05:42.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.456+0000 7f138d1e6740 1 -- start start 2024-09-12T02:05:42.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.456+0000 7f138d1e6740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d25f3a8b40 0x55d25f3a7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:42.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.456+0000 7f138d1e6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d25f3a75d0 0x55d25f3a7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:42.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.456+0000 7f138d1e6740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d25f3b2370 0x55d25f3b4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:42.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.456+0000 7f138d1e6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55d25f30d2f0 con 0x55d25f3a75d0 2024-09-12T02:05:42.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.456+0000 7f138d1e6740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55d25f2b4cf0 con 0x55d25f3a8b40 2024-09-12T02:05:42.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.456+0000 7f138d1e6740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55d25f3092b0 con 0x55d25f3b2370 2024-09-12T02:05:42.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.456+0000 7f1381ffb640 1 --2- 172.21.15.16:0/2413962505 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f136c0a2b80 0x7f136c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:42.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.456+0000 7f1388ba9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d25f3a75d0 0x55d25f3a7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:42.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.456+0000 7f1388ba9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d25f3a75d0 0x55d25f3a7a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60330/0 (socket says 172.21.15.16:60330) 2024-09-12T02:05:42.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.456+0000 7f1388ba9640 1 -- 172.21.15.16:0/2162484852 learned_addr learned my addr 172.21.15.16:0/2162484852 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:05:42.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.456+0000 7f13893aa640 1 --2- 172.21.15.16:0/2162484852 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d25f3a8b40 0x55d25f3a7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:42.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.456+0000 7f1381ffb640 1 -- 172.21.15.16:0/2413962505 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f13780ce370 con 0x7f13841012f0 2024-09-12T02:05:42.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.456+0000 7f1383fff640 1 --2- 172.21.15.16:0/2162484852 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d25f3b2370 0x55d25f3b4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:42.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.456+0000 7f13893aa640 1 --2- 172.21.15.16:0/2413962505 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f136c0a2b80 0x7f136c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:42.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.456+0000 7f1388ba9640 1 -- 172.21.15.16:0/2162484852 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d25f3b2370 msgr2=0x55d25f3b4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:42.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.456+0000 7f1388ba9640 1 --2- 172.21.15.16:0/2162484852 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d25f3b2370 0x55d25f3b4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:42.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.456+0000 7f1388ba9640 1 -- 172.21.15.16:0/2162484852 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d25f3a8b40 msgr2=0x55d25f3a7090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:42.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.456+0000 7f1388ba9640 1 --2- 172.21.15.16:0/2162484852 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d25f3a8b40 0x55d25f3a7090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:42.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.456+0000 7f1388ba9640 1 -- 172.21.15.16:0/2162484852 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d25f2993a0 con 0x55d25f3a75d0 2024-09-12T02:05:42.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.457+0000 7f1388ba9640 1 --2- 172.21.15.16:0/2162484852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d25f3a75d0 0x55d25f3a7a10 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f1374007850 tx=0x7f1374007d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30c61a75f8cebd70 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:42.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.457+0000 7f13893aa640 1 --2- 172.21.15.16:0/2413962505 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f136c0a2b80 0x7f136c0a5040 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f137c006010 tx=0x7f137c009380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:42.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.457+0000 7f136affd640 1 -- 172.21.15.16:0/2162484852 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f137400d950 con 0x55d25f3a75d0 2024-09-12T02:05:42.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.457+0000 7f136affd640 1 -- 172.21.15.16:0/2162484852 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1374018070 con 0x55d25f3a75d0 2024-09-12T02:05:42.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.457+0000 7f136affd640 1 -- 172.21.15.16:0/2162484852 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13740123e0 con 0x55d25f3a75d0 2024-09-12T02:05:42.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.458+0000 7f138d1e6740 1 -- 172.21.15.16:0/2162484852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d25f3a75d0 msgr2=0x55d25f3a7a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:42.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.458+0000 7f138d1e6740 1 --2- 172.21.15.16:0/2162484852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d25f3a75d0 0x55d25f3a7a10 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f1374007850 tx=0x7f1374007d10 comp rx=0 tx=0).stop 2024-09-12T02:05:42.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.458+0000 7f138d1e6740 1 -- 172.21.15.16:0/2162484852 shutdown_connections 2024-09-12T02:05:42.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.458+0000 7f138d1e6740 1 --2- 172.21.15.16:0/2162484852 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d25f3b2370 0x55d25f3b4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:42.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.458+0000 7f138d1e6740 1 --2- 172.21.15.16:0/2162484852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d25f3a75d0 0x55d25f3a7a10 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:42.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.458+0000 7f138d1e6740 1 --2- 172.21.15.16:0/2162484852 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d25f3a8b40 0x55d25f3a7090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:42.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.458+0000 7f138d1e6740 1 -- 172.21.15.16:0/2162484852 >> 172.21.15.16:0/2162484852 conn(0x55d25f370310 msgr2=0x55d25f3a3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:42.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.458+0000 7f138d1e6740 1 -- 172.21.15.16:0/2162484852 shutdown_connections 2024-09-12T02:05:42.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.458+0000 7f138d1e6740 1 -- 172.21.15.16:0/2162484852 wait complete. 2024-09-12T02:05:42.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.458+0000 7f138d1e6740 1 -- start start 2024-09-12T02:05:42.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.458+0000 7f138d1e6740 1 Processor -- start 2024-09-12T02:05:42.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.458+0000 7f138d1e6740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d25f3a75d0 0x55d25f3b1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:42.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.458+0000 7f138d1e6740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d25f3a8b40 0x55d25f3aef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:42.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.459+0000 7f138d1e6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d25f3b2370 0x55d25f3af4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:42.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.459+0000 7f138d1e6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55d25f3b5070 con 0x55d25f3b2370 2024-09-12T02:05:42.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.459+0000 7f138d1e6740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55d25f2c8060 con 0x55d25f3a8b40 2024-09-12T02:05:42.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.459+0000 7f138d1e6740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55d25f3b51f0 con 0x55d25f3a75d0 2024-09-12T02:05:42.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.459+0000 7f138d1e6740 10 client.0 _ll_register_callbacks cb 0x55d25f01b8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:05:42.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.459+0000 7f13893aa640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d25f3b2370 0x55d25f3af4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:42.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.459+0000 7f13893aa640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d25f3b2370 0x55d25f3af4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60332/0 (socket says 172.21.15.16:60332) 2024-09-12T02:05:42.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.459+0000 7f13893aa640 1 -- 172.21.15.16:0/55147818 learned_addr learned my addr 172.21.15.16:0/55147818 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:05:42.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.459+0000 7f1388ba9640 1 --2- 172.21.15.16:0/55147818 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d25f3a75d0 0x55d25f3b1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:42.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.459+0000 7f1383fff640 1 --2- 172.21.15.16:0/55147818 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d25f3a8b40 0x55d25f3aef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:42.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.459+0000 7f13893aa640 1 -- 172.21.15.16:0/55147818 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d25f3a75d0 msgr2=0x55d25f3b1ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:42.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.459+0000 7f13893aa640 1 --2- 172.21.15.16:0/55147818 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d25f3a75d0 0x55d25f3b1ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:42.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.459+0000 7f13893aa640 1 -- 172.21.15.16:0/55147818 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d25f3a8b40 msgr2=0x55d25f3aef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:42.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.459+0000 7f13893aa640 1 --2- 172.21.15.16:0/55147818 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d25f3a8b40 0x55d25f3aef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:42.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.459+0000 7f13893aa640 1 -- 172.21.15.16:0/55147818 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d25f07e3c0 con 0x55d25f3b2370 2024-09-12T02:05:42.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.459+0000 7f1383fff640 1 --2- 172.21.15.16:0/55147818 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d25f3a8b40 0x55d25f3aef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:05:42.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.459+0000 7f13893aa640 1 --2- 172.21.15.16:0/55147818 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d25f3b2370 0x55d25f3af4f0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f137c035000 tx=0x7f137c0095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bf83d1bdd098a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:42.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.459+0000 7f1363fff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:05:42.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.459+0000 7f1363fff640 1 -- 172.21.15.16:0/55147818 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f137c051070 con 0x55d25f3b2370 2024-09-12T02:05:42.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.459+0000 7f1363fff640 1 -- 172.21.15.16:0/55147818 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f137c004700 con 0x55d25f3b2370 2024-09-12T02:05:42.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.459+0000 7f138d1e6740 1 -- 172.21.15.16:0/55147818 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1374018070 con 0x55d25f3b2370 2024-09-12T02:05:42.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.460+0000 7f1363fff640 1 -- 172.21.15.16:0/55147818 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f137c02f610 con 0x55d25f3b2370 2024-09-12T02:05:42.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.460+0000 7f138d1e6740 10 client.16527 fetch_fsmap learned FSMap version 25 2024-09-12T02:05:42.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.460+0000 7f138d1e6740 1 -- 172.21.15.16:0/55147818 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d25f3c0a80 con 0x55d25f3b2370 2024-09-12T02:05:42.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.460+0000 7f1363fff640 1 -- 172.21.15.16:0/55147818 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1374018070 con 0x55d25f3b2370 2024-09-12T02:05:42.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.460+0000 7f1363fff640 1 -- 172.21.15.16:0/55147818 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f137c030a30 con 0x55d25f3b2370 2024-09-12T02:05:42.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.460+0000 7f138d1e6740 10 client.16527 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:05:42.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.460+0000 7f138d1e6740 10 client.16527 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:05:42.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.460+0000 7f138d1e6740 20 client.16527 populate_metadata read hostname 'smithi016' 2024-09-12T02:05:42.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.460+0000 7f138d1e6740 1 --2- 172.21.15.16:0/55147818 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55d25f3c1e90 0x55d25f3c4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:42.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.460+0000 7f138d1e6740 4 client.16527 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:05:42.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.460+0000 7f138d1e6740 1 -- 172.21.15.16:0/55147818 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d25f31a860 con 0x55d25f3c1e90 2024-09-12T02:05:42.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.460+0000 7f1388ba9640 1 --2- 172.21.15.16:0/55147818 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55d25f3c1e90 0x55d25f3c4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:42.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.461+0000 7f1388ba9640 1 --2- 172.21.15.16:0/55147818 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55d25f3c1e90 0x55d25f3c4330 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fac5f56149cda60d server_cookie=53bec014516f5fed in_seq=0 out_seq=0 2024-09-12T02:05:42.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.461+0000 7f1363fff640 10 client.16527 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:05:42.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.462+0000 7f1363fff640 1 -- 172.21.15.16:0/55147818 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d25f31a860 con 0x55d25f3c1e90 2024-09-12T02:05:42.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.462+0000 7f1363fff640 10 client.16527 handle_command_reply: tid=0 2024-09-12T02:05:42.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.462+0000 7f138d1e6740 1 client.16527 shutdown 2024-09-12T02:05:42.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.462+0000 7f138d1e6740 1 -- 172.21.15.16:0/55147818 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d25f3b2370 msgr2=0x55d25f3af4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:42.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.462+0000 7f138d1e6740 1 --2- 172.21.15.16:0/55147818 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d25f3b2370 0x55d25f3af4f0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f137c035000 tx=0x7f137c0095c0 comp rx=0 tx=0).stop 2024-09-12T02:05:42.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.462+0000 7f138d1e6740 1 -- 172.21.15.16:0/55147818 shutdown_connections 2024-09-12T02:05:42.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.462+0000 7f138d1e6740 1 --2- 172.21.15.16:0/55147818 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55d25f3c1e90 0x55d25f3c4330 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:42.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.462+0000 7f138d1e6740 1 --2- 172.21.15.16:0/55147818 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d25f3b2370 0x55d25f3af4f0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:42.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.462+0000 7f138d1e6740 1 --2- 172.21.15.16:0/55147818 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d25f3a8b40 0x55d25f3aef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:42.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.462+0000 7f138d1e6740 1 --2- 172.21.15.16:0/55147818 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d25f3a75d0 0x55d25f3b1ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:42.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.462+0000 7f1363fff640 0 client.16527 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:05:42.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.462+0000 7f138d1e6740 1 -- 172.21.15.16:0/55147818 >> 172.21.15.16:0/55147818 conn(0x55d25f370310 msgr2=0x55d25f3a9960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:42.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.462+0000 7f138d1e6740 1 -- 172.21.15.16:0/55147818 shutdown_connections 2024-09-12T02:05:42.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.462+0000 7f138d1e6740 1 -- 172.21.15.16:0/55147818 wait complete. 2024-09-12T02:05:42.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.462+0000 7f138d1e6740 20 client.16527 trim_cache size 0 max 16384 2024-09-12T02:05:42.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.475+0000 7f138d1e6740 1 Processor -- start 2024-09-12T02:05:42.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.475+0000 7f138d1e6740 1 -- start start 2024-09-12T02:05:42.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.476+0000 7f138d1e6740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d25f3a75d0 0x55d25f3a3e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:42.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.476+0000 7f138d1e6740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d25f3a8b40 0x55d25f3be310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:42.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.476+0000 7f138d1e6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d25f3b2370 0x55d25f3ae850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:42.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.476+0000 7f138d1e6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f13841b2060 con 0x55d25f3b2370 2024-09-12T02:05:42.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.476+0000 7f138d1e6740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f13841af930 con 0x55d25f3a75d0 2024-09-12T02:05:42.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.476+0000 7f138d1e6740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f13841ad7c0 con 0x55d25f3a8b40 2024-09-12T02:05:42.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.476+0000 7f13893aa640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d25f3b2370 0x55d25f3ae850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:42.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.476+0000 7f13893aa640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d25f3b2370 0x55d25f3ae850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60338/0 (socket says 172.21.15.16:60338) 2024-09-12T02:05:42.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.476+0000 7f13893aa640 1 -- 172.21.15.16:0/498623870 learned_addr learned my addr 172.21.15.16:0/498623870 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:05:42.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.476+0000 7f1388ba9640 1 --2- 172.21.15.16:0/498623870 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d25f3a75d0 0x55d25f3a3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:42.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.476+0000 7f1383fff640 1 --2- 172.21.15.16:0/498623870 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d25f3a8b40 0x55d25f3be310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:42.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.476+0000 7f13893aa640 1 -- 172.21.15.16:0/498623870 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d25f3a8b40 msgr2=0x55d25f3be310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:42.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.476+0000 7f13893aa640 1 --2- 172.21.15.16:0/498623870 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d25f3a8b40 0x55d25f3be310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:42.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.476+0000 7f13893aa640 1 -- 172.21.15.16:0/498623870 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d25f3a75d0 msgr2=0x55d25f3a3e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:42.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.476+0000 7f13893aa640 1 --2- 172.21.15.16:0/498623870 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d25f3a75d0 0x55d25f3a3e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:42.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.476+0000 7f13893aa640 1 -- 172.21.15.16:0/498623870 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d25f3a72d0 con 0x55d25f3b2370 2024-09-12T02:05:42.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.476+0000 7f13893aa640 1 --2- 172.21.15.16:0/498623870 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d25f3b2370 0x55d25f3ae850 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f137c009bf0 tx=0x7f137c02a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b47e1f18b6618e8f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:42.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.476+0000 7f136a7fc640 1 -- 172.21.15.16:0/498623870 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f137c051040 con 0x55d25f3b2370 2024-09-12T02:05:42.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.477+0000 7f136a7fc640 1 -- 172.21.15.16:0/498623870 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f137c036070 con 0x55d25f3b2370 2024-09-12T02:05:42.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.477+0000 7f136a7fc640 1 -- 172.21.15.16:0/498623870 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f137c02fb20 con 0x55d25f3b2370 2024-09-12T02:05:42.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.477+0000 7f138d1e6740 1 -- 172.21.15.16:0/498623870 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d25f3b2370 msgr2=0x55d25f3ae850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:42.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.477+0000 7f138d1e6740 1 --2- 172.21.15.16:0/498623870 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d25f3b2370 0x55d25f3ae850 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f137c009bf0 tx=0x7f137c02a8c0 comp rx=0 tx=0).stop 2024-09-12T02:05:42.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.477+0000 7f138d1e6740 1 -- 172.21.15.16:0/498623870 shutdown_connections 2024-09-12T02:05:42.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.477+0000 7f138d1e6740 1 --2- 172.21.15.16:0/498623870 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d25f3b2370 0x55d25f3ae850 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:42.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.477+0000 7f138d1e6740 1 --2- 172.21.15.16:0/498623870 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d25f3a8b40 0x55d25f3be310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:42.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.477+0000 7f138d1e6740 1 --2- 172.21.15.16:0/498623870 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d25f3a75d0 0x55d25f3a3e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:42.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.477+0000 7f138d1e6740 1 -- 172.21.15.16:0/498623870 >> 172.21.15.16:0/498623870 conn(0x55d25f370310 msgr2=0x55d25f3a3700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:42.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.477+0000 7f138d1e6740 1 -- 172.21.15.16:0/498623870 shutdown_connections 2024-09-12T02:05:42.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.477+0000 7f138d1e6740 1 -- 172.21.15.16:0/498623870 wait complete. 2024-09-12T02:05:42.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.478+0000 7f138d1e6740 1 -- start start 2024-09-12T02:05:42.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.478+0000 7f138d1e6740 1 Processor -- start 2024-09-12T02:05:42.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.478+0000 7f138d1e6740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d25f3a75d0 0x55d25f3ac090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:42.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.478+0000 7f138d1e6740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d25f3a8b40 0x55d25f3ac5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:42.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.478+0000 7f138d1e6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d25f3b2370 0x55d25f3acb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:42.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.478+0000 7f138d1e6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f13841ade10 con 0x55d25f3b2370 2024-09-12T02:05:42.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.478+0000 7f138d1e6740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f13841af260 con 0x55d25f3a8b40 2024-09-12T02:05:42.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.478+0000 7f138d1e6740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f13841ae4c0 con 0x55d25f3a75d0 2024-09-12T02:05:42.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.478+0000 7f138d1e6740 10 client.0 _ll_register_callbacks cb 0x7f137c031a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:05:42.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.478+0000 7f13893aa640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d25f3b2370 0x55d25f3acb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:42.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.478+0000 7f13893aa640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d25f3b2370 0x55d25f3acb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60354/0 (socket says 172.21.15.16:60354) 2024-09-12T02:05:42.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.478+0000 7f13893aa640 1 -- 172.21.15.16:0/359612797 learned_addr learned my addr 172.21.15.16:0/359612797 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:05:42.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.478+0000 7f1383fff640 1 --2- 172.21.15.16:0/359612797 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d25f3a8b40 0x55d25f3ac5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:42.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.478+0000 7f1388ba9640 1 --2- 172.21.15.16:0/359612797 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d25f3a75d0 0x55d25f3ac090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:42.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.479+0000 7f13893aa640 1 -- 172.21.15.16:0/359612797 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d25f3a75d0 msgr2=0x55d25f3ac090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:42.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.479+0000 7f13893aa640 1 --2- 172.21.15.16:0/359612797 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d25f3a75d0 0x55d25f3ac090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:42.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.479+0000 7f13893aa640 1 -- 172.21.15.16:0/359612797 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d25f3a8b40 msgr2=0x55d25f3ac5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:42.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.479+0000 7f13893aa640 1 --2- 172.21.15.16:0/359612797 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d25f3a8b40 0x55d25f3ac5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:42.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.479+0000 7f13893aa640 1 -- 172.21.15.16:0/359612797 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d25f42bcd0 con 0x55d25f3b2370 2024-09-12T02:05:42.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.479+0000 7f13893aa640 1 --2- 172.21.15.16:0/359612797 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d25f3b2370 0x55d25f3acb10 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f137c059040 tx=0x7f137c030b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=773a41dd468d2512 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:42.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.479+0000 7f1363fff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:05:42.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.479+0000 7f1363fff640 1 -- 172.21.15.16:0/359612797 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f137c051040 con 0x55d25f3b2370 2024-09-12T02:05:42.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.479+0000 7f1363fff640 1 -- 172.21.15.16:0/359612797 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f137c02fa20 con 0x55d25f3b2370 2024-09-12T02:05:42.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.479+0000 7f138d1e6740 1 -- 172.21.15.16:0/359612797 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f137c036070 con 0x55d25f3b2370 2024-09-12T02:05:42.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.479+0000 7f1363fff640 1 -- 172.21.15.16:0/359612797 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f137c02fc20 con 0x55d25f3b2370 2024-09-12T02:05:42.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.479+0000 7f138d1e6740 10 client.16533 fetch_fsmap learned FSMap version 25 2024-09-12T02:05:42.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.479+0000 7f138d1e6740 1 -- 172.21.15.16:0/359612797 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d25f42cfb0 con 0x55d25f3b2370 2024-09-12T02:05:42.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.479+0000 7f1363fff640 1 -- 172.21.15.16:0/359612797 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f137c036070 con 0x55d25f3b2370 2024-09-12T02:05:42.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.480+0000 7f1363fff640 1 -- 172.21.15.16:0/359612797 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f137c04bac0 con 0x55d25f3b2370 2024-09-12T02:05:42.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.480+0000 7f138d1e6740 10 client.16533 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:05:42.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.480+0000 7f138d1e6740 10 client.16533 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:05:42.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.480+0000 7f138d1e6740 20 client.16533 populate_metadata read hostname 'smithi016' 2024-09-12T02:05:42.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.480+0000 7f138d1e6740 1 --2- 172.21.15.16:0/359612797 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55d25f42d610 0x55d25f42fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:42.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.480+0000 7f138d1e6740 4 client.16533 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T02:05:42.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.480+0000 7f138d1e6740 1 -- 172.21.15.16:0/359612797 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d25f314d70 con 0x55d25f42d610 2024-09-12T02:05:42.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.480+0000 7f1388ba9640 1 --2- 172.21.15.16:0/359612797 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55d25f42d610 0x55d25f42fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:42.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.480+0000 7f1363fff640 10 client.16533 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:05:42.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.480+0000 7f1388ba9640 1 --2- 172.21.15.16:0/359612797 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55d25f42d610 0x55d25f42fa30 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d44ea1439b4871a server_cookie=651cacede6c9351c in_seq=0 out_seq=0 2024-09-12T02:05:42.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.481+0000 7f1363fff640 1 -- 172.21.15.16:0/359612797 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d25f314d70 con 0x55d25f42d610 2024-09-12T02:05:42.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.481+0000 7f1363fff640 10 client.16533 handle_command_reply: tid=0 2024-09-12T02:05:42.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.481+0000 7f138d1e6740 1 client.16533 shutdown 2024-09-12T02:05:42.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.481+0000 7f138d1e6740 1 -- 172.21.15.16:0/359612797 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d25f3b2370 msgr2=0x55d25f3acb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:42.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.481+0000 7f138d1e6740 1 --2- 172.21.15.16:0/359612797 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d25f3b2370 0x55d25f3acb10 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f137c059040 tx=0x7f137c030b60 comp rx=0 tx=0).stop 2024-09-12T02:05:42.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.481+0000 7f138d1e6740 1 -- 172.21.15.16:0/359612797 shutdown_connections 2024-09-12T02:05:42.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.481+0000 7f138d1e6740 1 --2- 172.21.15.16:0/359612797 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55d25f42d610 0x55d25f42fa30 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:42.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.481+0000 7f138d1e6740 1 --2- 172.21.15.16:0/359612797 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d25f3b2370 0x55d25f3acb10 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:42.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.481+0000 7f138d1e6740 1 --2- 172.21.15.16:0/359612797 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d25f3a8b40 0x55d25f3ac5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:42.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.481+0000 7f1363fff640 0 client.16533 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:05:42.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.481+0000 7f138d1e6740 1 --2- 172.21.15.16:0/359612797 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d25f3a75d0 0x55d25f3ac090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:42.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.481+0000 7f138d1e6740 1 -- 172.21.15.16:0/359612797 >> 172.21.15.16:0/359612797 conn(0x55d25f370310 msgr2=0x55d25f3a9110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:42.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.481+0000 7f138d1e6740 1 -- 172.21.15.16:0/359612797 shutdown_connections 2024-09-12T02:05:42.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.481+0000 7f138d1e6740 1 -- 172.21.15.16:0/359612797 wait complete. 2024-09-12T02:05:42.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.481+0000 7f138d1e6740 20 client.16533 trim_cache size 0 max 16384 2024-09-12T02:05:42.487 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T02:05:42.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.482+0000 7f138ae0d640 1 -- 172.21.15.16:0/2413962505 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f136c0a2b80 msgr2=0x7f136c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:42.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.482+0000 7f138ae0d640 1 --2- 172.21.15.16:0/2413962505 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f136c0a2b80 0x7f136c0a5040 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f137c006010 tx=0x7f137c009380 comp rx=0 tx=0).stop 2024-09-12T02:05:42.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.482+0000 7f138ae0d640 1 -- 172.21.15.16:0/2413962505 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13841012f0 msgr2=0x7f13841a09a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:42.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.482+0000 7f138ae0d640 1 --2- 172.21.15.16:0/2413962505 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13841012f0 0x7f13841a09a0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f137800b4a0 tx=0x7f137800b960 comp rx=0 tx=0).stop 2024-09-12T02:05:42.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.482+0000 7f138ae0d640 1 -- 172.21.15.16:0/2413962505 shutdown_connections 2024-09-12T02:05:42.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.482+0000 7f138ae0d640 1 --2- 172.21.15.16:0/2413962505 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f136c0a2b80 0x7f136c0a5040 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:42.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.482+0000 7f138ae0d640 1 --2- 172.21.15.16:0/2413962505 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f138410e4f0 0x7f138419a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:42.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.482+0000 7f138ae0d640 1 --2- 172.21.15.16:0/2413962505 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13841012f0 0x7f13841a09a0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:42.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.482+0000 7f138ae0d640 1 --2- 172.21.15.16:0/2413962505 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1384100940 0x7f13841a0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:42.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.482+0000 7f138ae0d640 1 -- 172.21.15.16:0/2413962505 >> 172.21.15.16:0/2413962505 conn(0x7f13840fc790 msgr2=0x7f13840feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:42.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.482+0000 7f138ae0d640 1 -- 172.21.15.16:0/2413962505 shutdown_connections 2024-09-12T02:05:42.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:42.482+0000 7f138ae0d640 1 -- 172.21.15.16:0/2413962505 wait complete. 2024-09-12T02:05:44.184 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-12T02:05:44.184 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-12T02:05:44.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:05:44 smithi184 ceph-mon[35593]: pgmap v449: 97 pgs: 97 active+clean; 2.1 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 3.4 KiB/s wr, 7 op/s 2024-09-12T02:05:44.418 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:05:44 smithi033 ceph-mon[35332]: pgmap v449: 97 pgs: 97 active+clean; 2.1 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 3.4 KiB/s wr, 7 op/s 2024-09-12T02:05:44.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:05:44 smithi016 ceph-mon[30947]: pgmap v449: 97 pgs: 97 active+clean; 2.1 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 3.4 KiB/s wr, 7 op/s 2024-09-12T02:05:44.456 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:05:44.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.894+0000 7f8485094640 1 -- 172.21.15.16:0/3096532493 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8480102d40 msgr2=0x7f8480103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:44.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.894+0000 7f8485094640 1 --2- 172.21.15.16:0/3096532493 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8480102d40 0x7f8480103120 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f8470009900 tx=0x7f847002a480 comp rx=0 tx=0).stop 2024-09-12T02:05:44.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.894+0000 7f8485094640 1 -- 172.21.15.16:0/3096532493 shutdown_connections 2024-09-12T02:05:44.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.894+0000 7f8485094640 1 --2- 172.21.15.16:0/3096532493 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f848010c170 0x7f848010e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:44.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.894+0000 7f8485094640 1 --2- 172.21.15.16:0/3096532493 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8480103660 0x7f848010bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:44.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.894+0000 7f8485094640 1 --2- 172.21.15.16:0/3096532493 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8480102d40 0x7f8480103120 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:44.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.894+0000 7f8485094640 1 -- 172.21.15.16:0/3096532493 >> 172.21.15.16:0/3096532493 conn(0x7f84800fc7a0 msgr2=0x7f84800febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:44.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.894+0000 7f8485094640 1 -- 172.21.15.16:0/3096532493 shutdown_connections 2024-09-12T02:05:44.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.894+0000 7f8485094640 1 -- 172.21.15.16:0/3096532493 wait complete. 2024-09-12T02:05:44.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.895+0000 7f8485094640 1 Processor -- start 2024-09-12T02:05:44.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.895+0000 7f8485094640 1 -- start start 2024-09-12T02:05:44.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.895+0000 7f8485094640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8480102d40 0x7f8480198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:44.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.895+0000 7f8485094640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8480103660 0x7f8480198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:44.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.895+0000 7f8485094640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f848010c170 0x7f848019cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:44.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.895+0000 7f8485094640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8480111f70 con 0x7f8480103660 2024-09-12T02:05:44.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.895+0000 7f8485094640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8480111df0 con 0x7f848010c170 2024-09-12T02:05:44.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.895+0000 7f8485094640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f84801120f0 con 0x7f8480102d40 2024-09-12T02:05:44.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.895+0000 7f847e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8480103660 0x7f8480198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:44.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.895+0000 7f847e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8480103660 0x7f8480198990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60382/0 (socket says 172.21.15.16:60382) 2024-09-12T02:05:44.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.895+0000 7f847e59c640 1 -- 172.21.15.16:0/1560307168 learned_addr learned my addr 172.21.15.16:0/1560307168 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:05:44.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.895+0000 7f847f59e640 1 --2- 172.21.15.16:0/1560307168 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f848010c170 0x7f848019cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:44.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.895+0000 7f847ed9d640 1 --2- 172.21.15.16:0/1560307168 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8480102d40 0x7f8480198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:44.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.896+0000 7f847e59c640 1 -- 172.21.15.16:0/1560307168 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8480102d40 msgr2=0x7f8480198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:44.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.896+0000 7f847e59c640 1 --2- 172.21.15.16:0/1560307168 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8480102d40 0x7f8480198450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:44.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.896+0000 7f847e59c640 1 -- 172.21.15.16:0/1560307168 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f848010c170 msgr2=0x7f848019cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:44.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.896+0000 7f847e59c640 1 --2- 172.21.15.16:0/1560307168 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f848010c170 0x7f848019cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:44.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.896+0000 7f847e59c640 1 -- 172.21.15.16:0/1560307168 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f848019d400 con 0x7f8480103660 2024-09-12T02:05:44.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.896+0000 7f847e59c640 1 --2- 172.21.15.16:0/1560307168 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8480103660 0x7f8480198990 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f847400d740 tx=0x7f847400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:44.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.896+0000 7f8467fff640 1 -- 172.21.15.16:0/1560307168 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8474017070 con 0x7f8480103660 2024-09-12T02:05:44.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.896+0000 7f8485094640 1 -- 172.21.15.16:0/1560307168 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f848019d6f0 con 0x7f8480103660 2024-09-12T02:05:44.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.896+0000 7f8467fff640 1 -- 172.21.15.16:0/1560307168 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f847400de50 con 0x7f8480103660 2024-09-12T02:05:44.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.896+0000 7f8467fff640 1 -- 172.21.15.16:0/1560307168 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f847400f620 con 0x7f8480103660 2024-09-12T02:05:44.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.896+0000 7f8485094640 1 -- 172.21.15.16:0/1560307168 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84801a4fd0 con 0x7f8480103660 2024-09-12T02:05:44.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.898+0000 7f8467fff640 1 -- 172.21.15.16:0/1560307168 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f8474009c30 con 0x7f8480103660 2024-09-12T02:05:44.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.899+0000 7f848746d740 1 Processor -- start 2024-09-12T02:05:44.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.899+0000 7f848746d740 1 -- start start 2024-09-12T02:05:44.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.899+0000 7f848746d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b22fecf70 0x560b22fed350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:44.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.899+0000 7f848746d740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560b22fed890 0x560b22ff7c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:44.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.899+0000 7f848746d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b22ff8150 0x560b22ffa540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:44.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.899+0000 7f848746d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x560b22f4e280 con 0x560b22fecf70 2024-09-12T02:05:44.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.899+0000 7f848746d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x560b22ef5b10 con 0x560b22ff8150 2024-09-12T02:05:44.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.899+0000 7f848746d740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x560b22f4a240 con 0x560b22fed890 2024-09-12T02:05:44.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.899+0000 7f847f59e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b22fecf70 0x560b22fed350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:44.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.899+0000 7f8467fff640 1 --2- 172.21.15.16:0/1560307168 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f84680a2b80 0x7f84680a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:44.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.899+0000 7f847f59e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b22fecf70 0x560b22fed350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60386/0 (socket says 172.21.15.16:60386) 2024-09-12T02:05:44.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.899+0000 7f847f59e640 1 -- 172.21.15.16:0/644389378 learned_addr learned my addr 172.21.15.16:0/644389378 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:05:44.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.899+0000 7f847e59c640 1 --2- 172.21.15.16:0/644389378 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b22ff8150 0x560b22ffa540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:44.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.899+0000 7f847f59e640 1 --2- 172.21.15.16:0/1560307168 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f84680a2b80 0x7f84680a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:44.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.899+0000 7f847ed9d640 1 --2- 172.21.15.16:0/644389378 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560b22fed890 0x560b22ff7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:44.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.900+0000 7f8467fff640 1 -- 172.21.15.16:0/1560307168 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f84740ceaf0 con 0x7f8480103660 2024-09-12T02:05:44.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.900+0000 7f847f59e640 1 -- 172.21.15.16:0/644389378 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560b22fed890 msgr2=0x560b22ff7c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:44.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.900+0000 7f847f59e640 1 --2- 172.21.15.16:0/644389378 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560b22fed890 0x560b22ff7c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:44.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.900+0000 7f847f59e640 1 -- 172.21.15.16:0/644389378 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b22ff8150 msgr2=0x560b22ffa540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:44.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.900+0000 7f847f59e640 1 --2- 172.21.15.16:0/644389378 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b22ff8150 0x560b22ffa540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:44.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.900+0000 7f847f59e640 1 -- 172.21.15.16:0/644389378 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b22f021f0 con 0x560b22fecf70 2024-09-12T02:05:44.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.900+0000 7f847f59e640 1 --2- 172.21.15.16:0/644389378 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b22fecf70 0x560b22fed350 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f84780139d0 tx=0x7f8478013e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0519e85bbb9ebbb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:44.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.900+0000 7f84657fa640 1 -- 172.21.15.16:0/644389378 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8478013050 con 0x560b22fecf70 2024-09-12T02:05:44.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.900+0000 7f84657fa640 1 -- 172.21.15.16:0/644389378 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8478022070 con 0x560b22fecf70 2024-09-12T02:05:44.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.900+0000 7f84657fa640 1 -- 172.21.15.16:0/644389378 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84780126b0 con 0x560b22fecf70 2024-09-12T02:05:44.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.901+0000 7f848746d740 1 -- 172.21.15.16:0/644389378 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b22fecf70 msgr2=0x560b22fed350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:44.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.901+0000 7f847f59e640 1 --2- 172.21.15.16:0/1560307168 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f84680a2b80 0x7f84680a5040 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f8478021000 tx=0x7f847801f000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:44.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.901+0000 7f848746d740 1 --2- 172.21.15.16:0/644389378 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b22fecf70 0x560b22fed350 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f84780139d0 tx=0x7f8478013e90 comp rx=0 tx=0).stop 2024-09-12T02:05:44.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.901+0000 7f848746d740 1 -- 172.21.15.16:0/644389378 shutdown_connections 2024-09-12T02:05:44.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.901+0000 7f848746d740 1 --2- 172.21.15.16:0/644389378 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b22ff8150 0x560b22ffa540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:44.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.901+0000 7f848746d740 1 --2- 172.21.15.16:0/644389378 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560b22fed890 0x560b22ff7c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:44.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.901+0000 7f848746d740 1 --2- 172.21.15.16:0/644389378 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b22fecf70 0x560b22fed350 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:44.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.901+0000 7f848746d740 1 -- 172.21.15.16:0/644389378 >> 172.21.15.16:0/644389378 conn(0x560b22fe9500 msgr2=0x560b22fe98e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:44.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.901+0000 7f848746d740 1 -- 172.21.15.16:0/644389378 shutdown_connections 2024-09-12T02:05:44.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.901+0000 7f848746d740 1 -- 172.21.15.16:0/644389378 wait complete. 2024-09-12T02:05:44.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.901+0000 7f848746d740 1 -- start start 2024-09-12T02:05:44.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.901+0000 7f848746d740 1 Processor -- start 2024-09-12T02:05:44.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.902+0000 7f848746d740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560b22fecf70 0x560b22ff5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:44.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.902+0000 7f848746d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b22fed890 0x560b22ff5590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:44.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.902+0000 7f848746d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b22ff8150 0x560b22ff5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:44.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.902+0000 7f848746d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x560b22ffad20 con 0x560b22fed890 2024-09-12T02:05:44.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.902+0000 7f848746d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x560b22f08f10 con 0x560b22ff8150 2024-09-12T02:05:44.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.902+0000 7f848746d740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x560b22ffaea0 con 0x560b22fecf70 2024-09-12T02:05:44.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.902+0000 7f848746d740 10 client.0 _ll_register_callbacks cb 0x560b22ed1a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:05:44.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.902+0000 7f847ed9d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560b22fecf70 0x560b22ff5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:44.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.902+0000 7f847e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b22fed890 0x560b22ff5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:44.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.902+0000 7f847ed9d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560b22fecf70 0x560b22ff5050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:53218/0 (socket says 172.21.15.16:53218) 2024-09-12T02:05:44.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.902+0000 7f847ed9d640 1 -- 172.21.15.16:0/57324581 learned_addr learned my addr 172.21.15.16:0/57324581 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:05:44.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.902+0000 7f847f59e640 1 --2- 172.21.15.16:0/57324581 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b22ff8150 0x560b22ff5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:44.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.902+0000 7f847e59c640 1 -- 172.21.15.16:0/57324581 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560b22fecf70 msgr2=0x560b22ff5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:44.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.902+0000 7f847e59c640 1 --2- 172.21.15.16:0/57324581 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560b22fecf70 0x560b22ff5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:44.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.902+0000 7f847e59c640 1 -- 172.21.15.16:0/57324581 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b22ff8150 msgr2=0x560b22ff5ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:44.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.902+0000 7f847e59c640 1 --2- 172.21.15.16:0/57324581 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b22ff8150 0x560b22ff5ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:44.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.902+0000 7f847e59c640 1 -- 172.21.15.16:0/57324581 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b22cbf6a0 con 0x560b22fed890 2024-09-12T02:05:44.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.902+0000 7f847e59c640 1 --2- 172.21.15.16:0/57324581 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b22fed890 0x560b22ff5590 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f84740cc6e0 tx=0x7f84740209d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdd2d109bfec617 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:44.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.902+0000 7f8462ff5640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:05:44.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.903+0000 7f8462ff5640 1 -- 172.21.15.16:0/57324581 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8474020040 con 0x560b22fed890 2024-09-12T02:05:44.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.903+0000 7f8462ff5640 1 -- 172.21.15.16:0/57324581 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f8474018d60 con 0x560b22fed890 2024-09-12T02:05:44.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.903+0000 7f848746d740 1 -- 172.21.15.16:0/57324581 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8478022070 con 0x560b22fed890 2024-09-12T02:05:44.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.903+0000 7f8462ff5640 1 -- 172.21.15.16:0/57324581 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f847409b4a0 con 0x560b22fed890 2024-09-12T02:05:44.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.903+0000 7f848746d740 10 client.16545 fetch_fsmap learned FSMap version 25 2024-09-12T02:05:44.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.903+0000 7f848746d740 1 -- 172.21.15.16:0/57324581 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x560b23006950 con 0x560b22fed890 2024-09-12T02:05:44.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.903+0000 7f8462ff5640 1 -- 172.21.15.16:0/57324581 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8478022070 con 0x560b22fed890 2024-09-12T02:05:44.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.903+0000 7f8462ff5640 1 -- 172.21.15.16:0/57324581 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f8474016170 con 0x560b22fed890 2024-09-12T02:05:44.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.903+0000 7f848746d740 10 client.16545 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:05:44.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.903+0000 7f848746d740 10 client.16545 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:05:44.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.903+0000 7f848746d740 20 client.16545 populate_metadata read hostname 'smithi016' 2024-09-12T02:05:44.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.903+0000 7f848746d740 1 --2- 172.21.15.16:0/57324581 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x560b23007db0 0x560b2300a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:44.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.903+0000 7f848746d740 4 client.16545 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:05:44.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.903+0000 7f848746d740 1 -- 172.21.15.16:0/57324581 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560b22f5b7f0 con 0x560b23007db0 2024-09-12T02:05:44.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.904+0000 7f847ed9d640 1 --2- 172.21.15.16:0/57324581 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x560b23007db0 0x560b2300a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:44.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.904+0000 7f847ed9d640 1 --2- 172.21.15.16:0/57324581 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x560b23007db0 0x560b2300a1a0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f29cd55e6288d3d0 server_cookie=657670aa6fe5811b in_seq=0 out_seq=0 2024-09-12T02:05:44.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.904+0000 7f8462ff5640 10 client.16545 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:05:44.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.905+0000 7f8462ff5640 1 -- 172.21.15.16:0/57324581 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560b22f5b7f0 con 0x560b23007db0 2024-09-12T02:05:44.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.905+0000 7f8462ff5640 10 client.16545 handle_command_reply: tid=0 2024-09-12T02:05:44.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.905+0000 7f848746d740 1 client.16545 shutdown 2024-09-12T02:05:44.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.905+0000 7f848746d740 1 -- 172.21.15.16:0/57324581 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b22fed890 msgr2=0x560b22ff5590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:44.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.905+0000 7f848746d740 1 --2- 172.21.15.16:0/57324581 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b22fed890 0x560b22ff5590 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f84740cc6e0 tx=0x7f84740209d0 comp rx=0 tx=0).stop 2024-09-12T02:05:44.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.905+0000 7f848746d740 1 -- 172.21.15.16:0/57324581 shutdown_connections 2024-09-12T02:05:44.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.905+0000 7f848746d740 1 --2- 172.21.15.16:0/57324581 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x560b23007db0 0x560b2300a1a0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:44.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.905+0000 7f848746d740 1 --2- 172.21.15.16:0/57324581 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b22ff8150 0x560b22ff5ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:44.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.905+0000 7f848746d740 1 --2- 172.21.15.16:0/57324581 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b22fed890 0x560b22ff5590 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:44.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.905+0000 7f8462ff5640 0 client.16545 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:05:44.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.905+0000 7f848746d740 1 --2- 172.21.15.16:0/57324581 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560b22fecf70 0x560b22ff5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:44.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.905+0000 7f848746d740 1 -- 172.21.15.16:0/57324581 >> 172.21.15.16:0/57324581 conn(0x560b22fe9500 msgr2=0x560b22fe4270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:44.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.905+0000 7f848746d740 1 -- 172.21.15.16:0/57324581 shutdown_connections 2024-09-12T02:05:44.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.905+0000 7f848746d740 1 -- 172.21.15.16:0/57324581 wait complete. 2024-09-12T02:05:44.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.905+0000 7f848746d740 20 client.16545 trim_cache size 0 max 16384 2024-09-12T02:05:44.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.923+0000 7f848746d740 1 Processor -- start 2024-09-12T02:05:44.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.923+0000 7f848746d740 1 -- start start 2024-09-12T02:05:44.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.923+0000 7f848746d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b22fecf70 0x560b2300fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:44.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.923+0000 7f848746d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b22fed890 0x560b2300ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:44.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.923+0000 7f848746d740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560b22ff8150 0x560b22ff22d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:44.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.923+0000 7f848746d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f84801adb00 con 0x560b22fecf70 2024-09-12T02:05:44.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.923+0000 7f848746d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f84801ab3d0 con 0x560b22fed890 2024-09-12T02:05:44.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.923+0000 7f848746d740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f84801a9240 con 0x560b22ff8150 2024-09-12T02:05:44.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.923+0000 7f847ed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b22fecf70 0x560b2300fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:44.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.923+0000 7f847ed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b22fecf70 0x560b2300fa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60412/0 (socket says 172.21.15.16:60412) 2024-09-12T02:05:44.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.923+0000 7f847ed9d640 1 -- 172.21.15.16:0/1494116866 learned_addr learned my addr 172.21.15.16:0/1494116866 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:05:44.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.923+0000 7f847f59e640 1 --2- 172.21.15.16:0/1494116866 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560b22ff8150 0x560b22ff22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:44.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.923+0000 7f847e59c640 1 --2- 172.21.15.16:0/1494116866 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b22fed890 0x560b2300ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:44.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.923+0000 7f847ed9d640 1 -- 172.21.15.16:0/1494116866 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560b22ff8150 msgr2=0x560b22ff22d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:44.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.923+0000 7f847ed9d640 1 --2- 172.21.15.16:0/1494116866 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560b22ff8150 0x560b22ff22d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:44.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.923+0000 7f847ed9d640 1 -- 172.21.15.16:0/1494116866 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b22fed890 msgr2=0x560b2300ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:44.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.923+0000 7f847ed9d640 1 --2- 172.21.15.16:0/1494116866 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b22fed890 0x560b2300ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:44.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.923+0000 7f847ed9d640 1 -- 172.21.15.16:0/1494116866 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b22fdf680 con 0x560b22fecf70 2024-09-12T02:05:44.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.924+0000 7f847ed9d640 1 --2- 172.21.15.16:0/1494116866 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b22fecf70 0x560b2300fa70 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f8470005940 tx=0x7f847002ac30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a43bc1dc0d81744 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:44.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.924+0000 7f8464ff9640 1 -- 172.21.15.16:0/1494116866 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8470035040 con 0x560b22fecf70 2024-09-12T02:05:44.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.924+0000 7f8464ff9640 1 -- 172.21.15.16:0/1494116866 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8470005500 con 0x560b22fecf70 2024-09-12T02:05:44.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.924+0000 7f8464ff9640 1 -- 172.21.15.16:0/1494116866 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8470004dd0 con 0x560b22fecf70 2024-09-12T02:05:44.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.924+0000 7f848746d740 1 -- 172.21.15.16:0/1494116866 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b22fecf70 msgr2=0x560b2300fa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:44.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.924+0000 7f848746d740 1 --2- 172.21.15.16:0/1494116866 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b22fecf70 0x560b2300fa70 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f8470005940 tx=0x7f847002ac30 comp rx=0 tx=0).stop 2024-09-12T02:05:44.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.924+0000 7f848746d740 1 -- 172.21.15.16:0/1494116866 shutdown_connections 2024-09-12T02:05:44.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.924+0000 7f848746d740 1 --2- 172.21.15.16:0/1494116866 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560b22ff8150 0x560b22ff22d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:44.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.924+0000 7f848746d740 1 --2- 172.21.15.16:0/1494116866 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b22fed890 0x560b2300ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:44.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.924+0000 7f848746d740 1 --2- 172.21.15.16:0/1494116866 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b22fecf70 0x560b2300fa70 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:44.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.924+0000 7f848746d740 1 -- 172.21.15.16:0/1494116866 >> 172.21.15.16:0/1494116866 conn(0x560b22fe9500 msgr2=0x560b22ff6fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:44.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.924+0000 7f848746d740 1 -- 172.21.15.16:0/1494116866 shutdown_connections 2024-09-12T02:05:44.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.924+0000 7f848746d740 1 -- 172.21.15.16:0/1494116866 wait complete. 2024-09-12T02:05:44.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.925+0000 7f848746d740 1 -- start start 2024-09-12T02:05:44.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.925+0000 7f848746d740 1 Processor -- start 2024-09-12T02:05:44.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.925+0000 7f848746d740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560b22fecf70 0x560b23000510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:44.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.925+0000 7f848746d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b22fed890 0x560b22ffd0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:44.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.925+0000 7f848746d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b22ff8150 0x560b22ffd600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:44.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.925+0000 7f848746d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f84801a9890 con 0x560b22fed890 2024-09-12T02:05:44.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.925+0000 7f848746d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f84801aad00 con 0x560b22ff8150 2024-09-12T02:05:44.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.925+0000 7f848746d740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f84801a9f60 con 0x560b22fecf70 2024-09-12T02:05:44.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.925+0000 7f848746d740 10 client.0 _ll_register_callbacks cb 0x7f8474097740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:05:44.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.925+0000 7f847e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b22fed890 0x560b22ffd0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:44.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.925+0000 7f847e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b22fed890 0x560b22ffd0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60428/0 (socket says 172.21.15.16:60428) 2024-09-12T02:05:44.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.925+0000 7f847e59c640 1 -- 172.21.15.16:0/3525579816 learned_addr learned my addr 172.21.15.16:0/3525579816 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:05:44.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.925+0000 7f847f59e640 1 --2- 172.21.15.16:0/3525579816 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b22ff8150 0x560b22ffd600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:44.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.925+0000 7f847ed9d640 1 --2- 172.21.15.16:0/3525579816 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560b22fecf70 0x560b23000510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:44.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.926+0000 7f847e59c640 1 -- 172.21.15.16:0/3525579816 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560b22fecf70 msgr2=0x560b23000510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:44.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.926+0000 7f847e59c640 1 --2- 172.21.15.16:0/3525579816 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560b22fecf70 0x560b23000510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:44.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.926+0000 7f847e59c640 1 -- 172.21.15.16:0/3525579816 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b22ff8150 msgr2=0x560b22ffd600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:44.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.926+0000 7f847e59c640 1 --2- 172.21.15.16:0/3525579816 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b22ff8150 0x560b22ffd600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:44.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.926+0000 7f847e59c640 1 -- 172.21.15.16:0/3525579816 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b22ffdb40 con 0x560b22fed890 2024-09-12T02:05:44.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.926+0000 7f847ed9d640 1 --2- 172.21.15.16:0/3525579816 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560b22fecf70 0x560b23000510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:05:44.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.926+0000 7f847e59c640 1 --2- 172.21.15.16:0/3525579816 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b22fed890 0x560b22ffd0c0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f847409b660 tx=0x7f84740ceef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2ab3c8ca57dd89 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:44.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.926+0000 7f8462ff5640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:05:44.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.926+0000 7f8462ff5640 1 -- 172.21.15.16:0/3525579816 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8474020040 con 0x560b22fed890 2024-09-12T02:05:44.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.926+0000 7f8462ff5640 1 -- 172.21.15.16:0/3525579816 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f847409abc0 con 0x560b22fed890 2024-09-12T02:05:44.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.926+0000 7f848746d740 1 -- 172.21.15.16:0/3525579816 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8470005500 con 0x560b22fed890 2024-09-12T02:05:44.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.926+0000 7f8462ff5640 1 -- 172.21.15.16:0/3525579816 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f847409ae00 con 0x560b22fed890 2024-09-12T02:05:44.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.926+0000 7f848746d740 10 client.16551 fetch_fsmap learned FSMap version 25 2024-09-12T02:05:44.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.926+0000 7f848746d740 1 -- 172.21.15.16:0/3525579816 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x560b23073780 con 0x560b22fed890 2024-09-12T02:05:44.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.926+0000 7f8462ff5640 1 -- 172.21.15.16:0/3525579816 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8470005500 con 0x560b22fed890 2024-09-12T02:05:44.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.926+0000 7f8462ff5640 1 -- 172.21.15.16:0/3525579816 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f847409a360 con 0x560b22fed890 2024-09-12T02:05:44.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.926+0000 7f848746d740 10 client.16551 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:05:44.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.926+0000 7f848746d740 10 client.16551 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:05:44.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.926+0000 7f848746d740 20 client.16551 populate_metadata read hostname 'smithi016' 2024-09-12T02:05:44.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.927+0000 7f848746d740 1 --2- 172.21.15.16:0/3525579816 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x560b23073ea0 0x560b230762c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:44.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.927+0000 7f848746d740 4 client.16551 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-12T02:05:44.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.927+0000 7f848746d740 1 -- 172.21.15.16:0/3525579816 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x560b22f55d00 con 0x560b23073ea0 2024-09-12T02:05:44.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.927+0000 7f847ed9d640 1 --2- 172.21.15.16:0/3525579816 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x560b23073ea0 0x560b230762c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:44.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.927+0000 7f8462ff5640 10 client.16551 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:05:44.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.927+0000 7f847ed9d640 1 --2- 172.21.15.16:0/3525579816 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x560b23073ea0 0x560b230762c0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ebb4047bc058421b server_cookie=cd6f2b794564df30 in_seq=0 out_seq=0 2024-09-12T02:05:44.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.928+0000 7f8462ff5640 1 -- 172.21.15.16:0/3525579816 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x560b22f55d00 con 0x560b23073ea0 2024-09-12T02:05:44.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.928+0000 7f8462ff5640 10 client.16551 handle_command_reply: tid=0 2024-09-12T02:05:44.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.928+0000 7f848746d740 1 client.16551 shutdown 2024-09-12T02:05:44.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.928+0000 7f848746d740 1 -- 172.21.15.16:0/3525579816 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b22fed890 msgr2=0x560b22ffd0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:44.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.928+0000 7f848746d740 1 --2- 172.21.15.16:0/3525579816 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b22fed890 0x560b22ffd0c0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f847409b660 tx=0x7f84740ceef0 comp rx=0 tx=0).stop 2024-09-12T02:05:44.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.928+0000 7f848746d740 1 -- 172.21.15.16:0/3525579816 shutdown_connections 2024-09-12T02:05:44.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.928+0000 7f848746d740 1 --2- 172.21.15.16:0/3525579816 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x560b23073ea0 0x560b230762c0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:44.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.928+0000 7f848746d740 1 --2- 172.21.15.16:0/3525579816 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b22ff8150 0x560b22ffd600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:44.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.928+0000 7f848746d740 1 --2- 172.21.15.16:0/3525579816 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b22fed890 0x560b22ffd0c0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:44.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.928+0000 7f8462ff5640 0 client.16551 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:05:44.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.928+0000 7f848746d740 1 --2- 172.21.15.16:0/3525579816 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560b22fecf70 0x560b23000510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:44.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.928+0000 7f848746d740 1 -- 172.21.15.16:0/3525579816 >> 172.21.15.16:0/3525579816 conn(0x560b22fe9500 msgr2=0x560b22ff66c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:44.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.929+0000 7f848746d740 1 -- 172.21.15.16:0/3525579816 shutdown_connections 2024-09-12T02:05:44.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.929+0000 7f848746d740 1 -- 172.21.15.16:0/3525579816 wait complete. 2024-09-12T02:05:44.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.929+0000 7f848746d740 20 client.16551 trim_cache size 0 max 16384 2024-09-12T02:05:44.935 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T02:05:44.935 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-09-12T02:05:44.935 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "09824400-0346-4d60-8a34-d9e54fa85b3f", 2024-09-12T02:05:44.935 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-09-12T02:05:44.935 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T02:05:44.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.929+0000 7f8485094640 1 -- 172.21.15.16:0/1560307168 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f84680a2b80 msgr2=0x7f84680a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:44.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.929+0000 7f8485094640 1 --2- 172.21.15.16:0/1560307168 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f84680a2b80 0x7f84680a5040 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f8478021000 tx=0x7f847801f000 comp rx=0 tx=0).stop 2024-09-12T02:05:44.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.929+0000 7f8485094640 1 -- 172.21.15.16:0/1560307168 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8480103660 msgr2=0x7f8480198990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:44.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.929+0000 7f8485094640 1 --2- 172.21.15.16:0/1560307168 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8480103660 0x7f8480198990 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f847400d740 tx=0x7f847400dc00 comp rx=0 tx=0).stop 2024-09-12T02:05:44.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.929+0000 7f8485094640 1 -- 172.21.15.16:0/1560307168 shutdown_connections 2024-09-12T02:05:44.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.929+0000 7f8485094640 1 --2- 172.21.15.16:0/1560307168 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f84680a2b80 0x7f84680a5040 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:44.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.929+0000 7f8485094640 1 --2- 172.21.15.16:0/1560307168 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f848010c170 0x7f848019cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:44.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.929+0000 7f8485094640 1 --2- 172.21.15.16:0/1560307168 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8480103660 0x7f8480198990 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:44.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.929+0000 7f8485094640 1 --2- 172.21.15.16:0/1560307168 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8480102d40 0x7f8480198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:44.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.929+0000 7f8485094640 1 -- 172.21.15.16:0/1560307168 >> 172.21.15.16:0/1560307168 conn(0x7f84800fc7a0 msgr2=0x7f84800fd400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:44.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.929+0000 7f8485094640 1 -- 172.21.15.16:0/1560307168 shutdown_connections 2024-09-12T02:05:44.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:44.929+0000 7f8485094640 1 -- 172.21.15.16:0/1560307168 wait complete. 2024-09-12T02:05:45.716 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T02:05:45.972 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:05:46.375 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:05:46 smithi016 ceph-mon[30947]: pgmap v450: 97 pgs: 97 active+clean; 2.1 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 4.7 KiB/s wr, 8 op/s 2024-09-12T02:05:46.375 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:05:46 smithi016 ceph-mon[30947]: scrub queued for path: / 2024-09-12T02:05:46.375 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:05:46 smithi016 ceph-mon[30947]: scrub summary: idle+waiting paths [/] 2024-09-12T02:05:46.375 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:05:46 smithi016 ceph-mon[30947]: scrub summary: active paths [/] 2024-09-12T02:05:46.375 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:05:46 smithi016 ceph-mon[30947]: scrub summary: idle+waiting paths [/] 2024-09-12T02:05:46.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:05:46 smithi184 ceph-mon[35593]: pgmap v450: 97 pgs: 97 active+clean; 2.1 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 4.7 KiB/s wr, 8 op/s 2024-09-12T02:05:46.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:05:46 smithi184 ceph-mon[35593]: scrub queued for path: / 2024-09-12T02:05:46.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:05:46 smithi184 ceph-mon[35593]: scrub summary: idle+waiting paths [/] 2024-09-12T02:05:46.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:05:46 smithi184 ceph-mon[35593]: scrub summary: active paths [/] 2024-09-12T02:05:46.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:05:46 smithi184 ceph-mon[35593]: scrub summary: idle+waiting paths [/] 2024-09-12T02:05:46.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:05:46 smithi033 ceph-mon[35332]: pgmap v450: 97 pgs: 97 active+clean; 2.1 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 4.7 KiB/s wr, 8 op/s 2024-09-12T02:05:46.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:05:46 smithi033 ceph-mon[35332]: scrub queued for path: / 2024-09-12T02:05:46.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:05:46 smithi033 ceph-mon[35332]: scrub summary: idle+waiting paths [/] 2024-09-12T02:05:46.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:05:46 smithi033 ceph-mon[35332]: scrub summary: active paths [/] 2024-09-12T02:05:46.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:05:46 smithi033 ceph-mon[35332]: scrub summary: idle+waiting paths [/] 2024-09-12T02:05:46.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.458+0000 7fe179c9f640 1 -- 172.21.15.16:0/3747273397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe174102d90 msgr2=0x7fe174103170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:46.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.458+0000 7fe179c9f640 1 --2- 172.21.15.16:0/3747273397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe174102d90 0x7fe174103170 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fe164009930 tx=0x7fe16402a420 comp rx=0 tx=0).stop 2024-09-12T02:05:46.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.458+0000 7fe179c9f640 1 -- 172.21.15.16:0/3747273397 shutdown_connections 2024-09-12T02:05:46.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.458+0000 7fe179c9f640 1 --2- 172.21.15.16:0/3747273397 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe174108290 0x7fe17410ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:46.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.458+0000 7fe179c9f640 1 --2- 172.21.15.16:0/3747273397 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe174103740 0x7fe174107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:46.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.458+0000 7fe179c9f640 1 --2- 172.21.15.16:0/3747273397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe174102d90 0x7fe174103170 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:46.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.458+0000 7fe179c9f640 1 -- 172.21.15.16:0/3747273397 >> 172.21.15.16:0/3747273397 conn(0x7fe1740fc7d0 msgr2=0x7fe1740febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:46.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.458+0000 7fe179c9f640 1 -- 172.21.15.16:0/3747273397 shutdown_connections 2024-09-12T02:05:46.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.458+0000 7fe179c9f640 1 -- 172.21.15.16:0/3747273397 wait complete. 2024-09-12T02:05:46.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.459+0000 7fe179c9f640 1 Processor -- start 2024-09-12T02:05:46.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.459+0000 7fe179c9f640 1 -- start start 2024-09-12T02:05:46.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.459+0000 7fe179c9f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe174102d90 0x7fe1741009f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:46.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.459+0000 7fe179c9f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe174103740 0x7fe174100f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:46.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.460+0000 7fe179c9f640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe174108290 0x7fe174101470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:46.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.460+0000 7fe179c9f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe174112090 con 0x7fe174103740 2024-09-12T02:05:46.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.460+0000 7fe179c9f640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe174111f10 con 0x7fe174102d90 2024-09-12T02:05:46.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.460+0000 7fe179c9f640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe174112210 con 0x7fe174108290 2024-09-12T02:05:46.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.460+0000 7fe172ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe174103740 0x7fe174100f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:46.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.460+0000 7fe172ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe174103740 0x7fe174100f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60434/0 (socket says 172.21.15.16:60434) 2024-09-12T02:05:46.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.460+0000 7fe1737fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe174102d90 0x7fe1741009f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:46.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.460+0000 7fe172ffd640 1 -- 172.21.15.16:0/3110026807 learned_addr learned my addr 172.21.15.16:0/3110026807 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:05:46.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.460+0000 7fe173fff640 1 --2- 172.21.15.16:0/3110026807 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe174108290 0x7fe174101470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:46.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.460+0000 7fe172ffd640 1 -- 172.21.15.16:0/3110026807 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe174108290 msgr2=0x7fe174101470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:46.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.460+0000 7fe172ffd640 1 --2- 172.21.15.16:0/3110026807 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe174108290 0x7fe174101470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:46.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.460+0000 7fe172ffd640 1 -- 172.21.15.16:0/3110026807 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe174102d90 msgr2=0x7fe1741009f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:46.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.460+0000 7fe172ffd640 1 --2- 172.21.15.16:0/3110026807 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe174102d90 0x7fe1741009f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:46.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.460+0000 7fe172ffd640 1 -- 172.21.15.16:0/3110026807 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1741a6a60 con 0x7fe174103740 2024-09-12T02:05:46.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.460+0000 7fe173fff640 1 --2- 172.21.15.16:0/3110026807 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe174108290 0x7fe174101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:05:46.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.460+0000 7fe172ffd640 1 --2- 172.21.15.16:0/3110026807 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe174103740 0x7fe174100f30 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fe16800c970 tx=0x7fe16800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:46.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.460+0000 7fe170ff9640 1 -- 172.21.15.16:0/3110026807 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe168007bb0 con 0x7fe174103740 2024-09-12T02:05:46.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.460+0000 7fe179c9f640 1 -- 172.21.15.16:0/3110026807 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1741a6d50 con 0x7fe174103740 2024-09-12T02:05:46.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.460+0000 7fe170ff9640 1 -- 172.21.15.16:0/3110026807 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe168007d50 con 0x7fe174103740 2024-09-12T02:05:46.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.460+0000 7fe170ff9640 1 -- 172.21.15.16:0/3110026807 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe16800f450 con 0x7fe174103740 2024-09-12T02:05:46.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.460+0000 7fe179c9f640 1 -- 172.21.15.16:0/3110026807 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1741a7230 con 0x7fe174103740 2024-09-12T02:05:46.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.463+0000 7fe170ff9640 1 -- 172.21.15.16:0/3110026807 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fe168004f90 con 0x7fe174103740 2024-09-12T02:05:46.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.463+0000 7fe17c078740 1 Processor -- start 2024-09-12T02:05:46.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.463+0000 7fe17c078740 1 -- start start 2024-09-12T02:05:46.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.463+0000 7fe17c078740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560bbb6534a0 0x560bbb653880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:46.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.463+0000 7fe17c078740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560bbb653dc0 0x560bbb65e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:46.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.463+0000 7fe17c078740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560bbb65e680 0x560bbb660a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:46.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.463+0000 7fe17c078740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x560bbb5b92e0 con 0x560bbb653dc0 2024-09-12T02:05:46.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.463+0000 7fe17c078740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x560bbb560ce0 con 0x560bbb65e680 2024-09-12T02:05:46.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.463+0000 7fe17c078740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x560bbb5b52a0 con 0x560bbb6534a0 2024-09-12T02:05:46.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.463+0000 7fe1737fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560bbb653dc0 0x560bbb65e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:46.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.464+0000 7fe1737fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560bbb653dc0 0x560bbb65e140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60450/0 (socket says 172.21.15.16:60450) 2024-09-12T02:05:46.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.464+0000 7fe1737fe640 1 -- 172.21.15.16:0/1198933931 learned_addr learned my addr 172.21.15.16:0/1198933931 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:05:46.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.464+0000 7fe170ff9640 1 --2- 172.21.15.16:0/3110026807 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fe15c0a2b80 0x7fe15c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:46.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.464+0000 7fe172ffd640 1 --2- 172.21.15.16:0/1198933931 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560bbb65e680 0x560bbb660a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:46.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.464+0000 7fe173fff640 1 --2- 172.21.15.16:0/1198933931 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560bbb6534a0 0x560bbb653880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:46.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.464+0000 7fe1737fe640 1 -- 172.21.15.16:0/1198933931 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560bbb6534a0 msgr2=0x560bbb653880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:46.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.464+0000 7fe1737fe640 1 --2- 172.21.15.16:0/1198933931 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560bbb6534a0 0x560bbb653880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:46.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.464+0000 7fe1737fe640 1 -- 172.21.15.16:0/1198933931 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560bbb65e680 msgr2=0x560bbb660a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:46.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.464+0000 7fe1737fe640 1 --2- 172.21.15.16:0/1198933931 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560bbb65e680 0x560bbb660a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:46.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.464+0000 7fe1737fe640 1 -- 172.21.15.16:0/1198933931 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560bbb545440 con 0x560bbb653dc0 2024-09-12T02:05:46.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.464+0000 7fe173fff640 1 --2- 172.21.15.16:0/1198933931 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560bbb6534a0 0x560bbb653880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:05:46.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.464+0000 7fe173fff640 1 --2- 172.21.15.16:0/3110026807 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fe15c0a2b80 0x7fe15c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:46.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.464+0000 7fe170ff9640 1 -- 172.21.15.16:0/3110026807 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fe1680cedb0 con 0x7fe174103740 2024-09-12T02:05:46.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.464+0000 7fe1737fe640 1 --2- 172.21.15.16:0/1198933931 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560bbb653dc0 0x560bbb65e140 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fe164009930 tx=0x7fe164031630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61d890381a6804cc server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:46.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.464+0000 7fe159ffb640 1 -- 172.21.15.16:0/1198933931 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe164040070 con 0x560bbb653dc0 2024-09-12T02:05:46.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.464+0000 7fe159ffb640 1 -- 172.21.15.16:0/1198933931 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe164005020 con 0x560bbb653dc0 2024-09-12T02:05:46.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.464+0000 7fe159ffb640 1 -- 172.21.15.16:0/1198933931 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe164035040 con 0x560bbb653dc0 2024-09-12T02:05:46.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.465+0000 7fe17c078740 1 -- 172.21.15.16:0/1198933931 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560bbb653dc0 msgr2=0x560bbb65e140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:46.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.465+0000 7fe173fff640 1 --2- 172.21.15.16:0/3110026807 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fe15c0a2b80 0x7fe15c0a5040 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fe16c005c40 tx=0x7fe16c004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:46.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.465+0000 7fe17c078740 1 --2- 172.21.15.16:0/1198933931 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560bbb653dc0 0x560bbb65e140 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fe164009930 tx=0x7fe164031630 comp rx=0 tx=0).stop 2024-09-12T02:05:46.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.465+0000 7fe17c078740 1 -- 172.21.15.16:0/1198933931 shutdown_connections 2024-09-12T02:05:46.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.465+0000 7fe17c078740 1 --2- 172.21.15.16:0/1198933931 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560bbb65e680 0x560bbb660a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:46.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.465+0000 7fe17c078740 1 --2- 172.21.15.16:0/1198933931 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560bbb653dc0 0x560bbb65e140 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:46.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.465+0000 7fe17c078740 1 --2- 172.21.15.16:0/1198933931 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560bbb6534a0 0x560bbb653880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:46.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.465+0000 7fe17c078740 1 -- 172.21.15.16:0/1198933931 >> 172.21.15.16:0/1198933931 conn(0x560bbb64fa30 msgr2=0x560bbb64fe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:46.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.465+0000 7fe17c078740 1 -- 172.21.15.16:0/1198933931 shutdown_connections 2024-09-12T02:05:46.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.465+0000 7fe17c078740 1 -- 172.21.15.16:0/1198933931 wait complete. 2024-09-12T02:05:46.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.466+0000 7fe17c078740 1 -- start start 2024-09-12T02:05:46.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.466+0000 7fe17c078740 1 Processor -- start 2024-09-12T02:05:46.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.466+0000 7fe17c078740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560bbb6534a0 0x560bbb65de20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:46.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.466+0000 7fe17c078740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560bbb653dc0 0x560bbb65b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:46.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.466+0000 7fe17c078740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560bbb65e680 0x560bbb65b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:46.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.466+0000 7fe17c078740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x560bbb661250 con 0x560bbb653dc0 2024-09-12T02:05:46.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.466+0000 7fe17c078740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x560bbb574050 con 0x560bbb65e680 2024-09-12T02:05:46.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.466+0000 7fe17c078740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x560bbb6613d0 con 0x560bbb6534a0 2024-09-12T02:05:46.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.466+0000 7fe17c078740 10 client.0 _ll_register_callbacks cb 0x560bbb5b4160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:05:46.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.466+0000 7fe172ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560bbb653dc0 0x560bbb65b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:46.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.466+0000 7fe172ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560bbb653dc0 0x560bbb65b120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60452/0 (socket says 172.21.15.16:60452) 2024-09-12T02:05:46.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.466+0000 7fe172ffd640 1 -- 172.21.15.16:0/4058170522 learned_addr learned my addr 172.21.15.16:0/4058170522 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:05:46.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.466+0000 7fe173fff640 1 --2- 172.21.15.16:0/4058170522 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560bbb65e680 0x560bbb65b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:46.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.466+0000 7fe1737fe640 1 --2- 172.21.15.16:0/4058170522 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560bbb6534a0 0x560bbb65de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:46.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.467+0000 7fe172ffd640 1 -- 172.21.15.16:0/4058170522 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560bbb6534a0 msgr2=0x560bbb65de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:46.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.467+0000 7fe172ffd640 1 --2- 172.21.15.16:0/4058170522 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560bbb6534a0 0x560bbb65de20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:46.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.467+0000 7fe172ffd640 1 -- 172.21.15.16:0/4058170522 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560bbb65e680 msgr2=0x560bbb65b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:46.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.467+0000 7fe172ffd640 1 --2- 172.21.15.16:0/4058170522 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560bbb65e680 0x560bbb65b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:46.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.467+0000 7fe172ffd640 1 -- 172.21.15.16:0/4058170522 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560bbb32a3c0 con 0x560bbb653dc0 2024-09-12T02:05:46.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.467+0000 7fe172ffd640 1 --2- 172.21.15.16:0/4058170522 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560bbb653dc0 0x560bbb65b120 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fe168097fe0 tx=0x7fe16809b210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6aed689ee52b50e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:46.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.467+0000 7fe152ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:05:46.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.467+0000 7fe152ffd640 1 -- 172.21.15.16:0/4058170522 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe16801d040 con 0x560bbb653dc0 2024-09-12T02:05:46.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.467+0000 7fe152ffd640 1 -- 172.21.15.16:0/4058170522 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fe16807a0c0 con 0x560bbb653dc0 2024-09-12T02:05:46.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.467+0000 7fe17c078740 1 -- 172.21.15.16:0/4058170522 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe164005020 con 0x560bbb653dc0 2024-09-12T02:05:46.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.467+0000 7fe152ffd640 1 -- 172.21.15.16:0/4058170522 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe16809bd10 con 0x560bbb653dc0 2024-09-12T02:05:46.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.467+0000 7fe17c078740 10 client.16563 fetch_fsmap learned FSMap version 25 2024-09-12T02:05:46.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.467+0000 7fe17c078740 1 -- 172.21.15.16:0/4058170522 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x560bbb66ca70 con 0x560bbb653dc0 2024-09-12T02:05:46.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.467+0000 7fe152ffd640 1 -- 172.21.15.16:0/4058170522 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe164005020 con 0x560bbb653dc0 2024-09-12T02:05:46.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.468+0000 7fe152ffd640 1 -- 172.21.15.16:0/4058170522 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fe168005280 con 0x560bbb653dc0 2024-09-12T02:05:46.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.468+0000 7fe17c078740 10 client.16563 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:05:46.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.468+0000 7fe17c078740 10 client.16563 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:05:46.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.468+0000 7fe17c078740 20 client.16563 populate_metadata read hostname 'smithi016' 2024-09-12T02:05:46.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.468+0000 7fe17c078740 1 --2- 172.21.15.16:0/4058170522 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x560bbb66de80 0x560bbb670270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:46.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.468+0000 7fe17c078740 4 client.16563 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:05:46.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.468+0000 7fe17c078740 1 -- 172.21.15.16:0/4058170522 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560bbb5c6850 con 0x560bbb66de80 2024-09-12T02:05:46.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.468+0000 7fe1737fe640 1 --2- 172.21.15.16:0/4058170522 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x560bbb66de80 0x560bbb670270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:46.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.468+0000 7fe1737fe640 1 --2- 172.21.15.16:0/4058170522 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x560bbb66de80 0x560bbb670270 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98669cfe1001c34a server_cookie=c6c0b0c10e293726 in_seq=0 out_seq=0 2024-09-12T02:05:46.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.468+0000 7fe152ffd640 10 client.16563 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:05:46.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.469+0000 7fe152ffd640 1 -- 172.21.15.16:0/4058170522 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560bbb5c6850 con 0x560bbb66de80 2024-09-12T02:05:46.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.469+0000 7fe152ffd640 10 client.16563 handle_command_reply: tid=0 2024-09-12T02:05:46.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.469+0000 7fe17c078740 1 client.16563 shutdown 2024-09-12T02:05:46.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.469+0000 7fe17c078740 1 -- 172.21.15.16:0/4058170522 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560bbb653dc0 msgr2=0x560bbb65b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:46.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.469+0000 7fe17c078740 1 --2- 172.21.15.16:0/4058170522 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560bbb653dc0 0x560bbb65b120 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fe168097fe0 tx=0x7fe16809b210 comp rx=0 tx=0).stop 2024-09-12T02:05:46.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.469+0000 7fe17c078740 1 -- 172.21.15.16:0/4058170522 shutdown_connections 2024-09-12T02:05:46.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.469+0000 7fe17c078740 1 --2- 172.21.15.16:0/4058170522 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x560bbb66de80 0x560bbb670270 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:46.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.469+0000 7fe17c078740 1 --2- 172.21.15.16:0/4058170522 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560bbb65e680 0x560bbb65b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:46.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.469+0000 7fe17c078740 1 --2- 172.21.15.16:0/4058170522 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560bbb653dc0 0x560bbb65b120 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:46.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.469+0000 7fe152ffd640 0 client.16563 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:05:46.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.469+0000 7fe17c078740 1 --2- 172.21.15.16:0/4058170522 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560bbb6534a0 0x560bbb65de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:46.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.469+0000 7fe17c078740 1 -- 172.21.15.16:0/4058170522 >> 172.21.15.16:0/4058170522 conn(0x560bbb64fa30 msgr2=0x560bbb64eb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:46.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.469+0000 7fe17c078740 1 -- 172.21.15.16:0/4058170522 shutdown_connections 2024-09-12T02:05:46.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.469+0000 7fe17c078740 1 -- 172.21.15.16:0/4058170522 wait complete. 2024-09-12T02:05:46.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.470+0000 7fe17c078740 20 client.16563 trim_cache size 0 max 16384 2024-09-12T02:05:46.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.487+0000 7fe17c078740 1 Processor -- start 2024-09-12T02:05:46.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.487+0000 7fe17c078740 1 -- start start 2024-09-12T02:05:46.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.487+0000 7fe17c078740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560bbb6534a0 0x560bbb656900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:46.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.487+0000 7fe17c078740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560bbb653dc0 0x560bbb64ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:46.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.487+0000 7fe17c078740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560bbb65e680 0x560bbb675b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:46.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.487+0000 7fe17c078740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe1741b1f90 con 0x560bbb65e680 2024-09-12T02:05:46.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.487+0000 7fe17c078740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe1741af860 con 0x560bbb6534a0 2024-09-12T02:05:46.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.487+0000 7fe17c078740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe1741ad6d0 con 0x560bbb653dc0 2024-09-12T02:05:46.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.487+0000 7fe173fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560bbb65e680 0x560bbb675b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:46.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.487+0000 7fe173fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560bbb65e680 0x560bbb675b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60466/0 (socket says 172.21.15.16:60466) 2024-09-12T02:05:46.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.487+0000 7fe173fff640 1 -- 172.21.15.16:0/1521515836 learned_addr learned my addr 172.21.15.16:0/1521515836 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:05:46.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.487+0000 7fe1737fe640 1 --2- 172.21.15.16:0/1521515836 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560bbb6534a0 0x560bbb656900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:46.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.487+0000 7fe172ffd640 1 --2- 172.21.15.16:0/1521515836 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560bbb653dc0 0x560bbb64ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:46.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.487+0000 7fe173fff640 1 -- 172.21.15.16:0/1521515836 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560bbb653dc0 msgr2=0x560bbb64ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:46.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.487+0000 7fe173fff640 1 --2- 172.21.15.16:0/1521515836 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560bbb653dc0 0x560bbb64ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:46.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.487+0000 7fe173fff640 1 -- 172.21.15.16:0/1521515836 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560bbb6534a0 msgr2=0x560bbb656900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:46.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.487+0000 7fe173fff640 1 --2- 172.21.15.16:0/1521515836 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560bbb6534a0 0x560bbb656900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:46.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.487+0000 7fe173fff640 1 -- 172.21.15.16:0/1521515836 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560bbb65a0d0 con 0x560bbb65e680 2024-09-12T02:05:46.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.487+0000 7fe172ffd640 1 --2- 172.21.15.16:0/1521515836 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560bbb653dc0 0x560bbb64ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:05:46.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.488+0000 7fe173fff640 1 --2- 172.21.15.16:0/1521515836 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560bbb65e680 0x560bbb675b20 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fe16c012770 tx=0x7fe16c013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5929ec69ba19f6b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:46.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.488+0000 7fe1597fa640 1 -- 172.21.15.16:0/1521515836 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe16c016070 con 0x560bbb65e680 2024-09-12T02:05:46.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.488+0000 7fe1597fa640 1 -- 172.21.15.16:0/1521515836 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe16c004d20 con 0x560bbb65e680 2024-09-12T02:05:46.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.488+0000 7fe1597fa640 1 -- 172.21.15.16:0/1521515836 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe16c01e700 con 0x560bbb65e680 2024-09-12T02:05:46.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.488+0000 7fe17c078740 1 -- 172.21.15.16:0/1521515836 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560bbb65e680 msgr2=0x560bbb675b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:46.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.488+0000 7fe17c078740 1 --2- 172.21.15.16:0/1521515836 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560bbb65e680 0x560bbb675b20 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fe16c012770 tx=0x7fe16c013000 comp rx=0 tx=0).stop 2024-09-12T02:05:46.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.488+0000 7fe17c078740 1 -- 172.21.15.16:0/1521515836 shutdown_connections 2024-09-12T02:05:46.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.489+0000 7fe17c078740 1 --2- 172.21.15.16:0/1521515836 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560bbb65e680 0x560bbb675b20 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:46.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.489+0000 7fe17c078740 1 --2- 172.21.15.16:0/1521515836 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560bbb653dc0 0x560bbb64ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:46.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.489+0000 7fe17c078740 1 --2- 172.21.15.16:0/1521515836 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560bbb6534a0 0x560bbb656900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:46.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.489+0000 7fe17c078740 1 -- 172.21.15.16:0/1521515836 >> 172.21.15.16:0/1521515836 conn(0x560bbb64fa30 msgr2=0x560bbb64e370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:46.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.489+0000 7fe17c078740 1 -- 172.21.15.16:0/1521515836 shutdown_connections 2024-09-12T02:05:46.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.489+0000 7fe17c078740 1 -- 172.21.15.16:0/1521515836 wait complete. 2024-09-12T02:05:46.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.489+0000 7fe17c078740 1 -- start start 2024-09-12T02:05:46.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.489+0000 7fe17c078740 1 Processor -- start 2024-09-12T02:05:46.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.489+0000 7fe17c078740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560bbb6534a0 0x560bbb6669b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:46.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.489+0000 7fe17c078740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560bbb653dc0 0x560bbb666ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:46.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.489+0000 7fe17c078740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560bbb65e680 0x560bbb661760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:46.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.489+0000 7fe17c078740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe1741add20 con 0x560bbb65e680 2024-09-12T02:05:46.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.489+0000 7fe17c078740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe1741af190 con 0x560bbb653dc0 2024-09-12T02:05:46.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.489+0000 7fe17c078740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe1741ae3f0 con 0x560bbb6534a0 2024-09-12T02:05:46.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.489+0000 7fe17c078740 10 client.0 _ll_register_callbacks cb 0x7fe16809b3e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:05:46.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.490+0000 7fe173fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560bbb65e680 0x560bbb661760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:46.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.490+0000 7fe173fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560bbb65e680 0x560bbb661760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60476/0 (socket says 172.21.15.16:60476) 2024-09-12T02:05:46.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.490+0000 7fe173fff640 1 -- 172.21.15.16:0/2092539621 learned_addr learned my addr 172.21.15.16:0/2092539621 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:05:46.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.490+0000 7fe172ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560bbb653dc0 0x560bbb666ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:46.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.490+0000 7fe1737fe640 1 --2- 172.21.15.16:0/2092539621 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560bbb6534a0 0x560bbb6669b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:46.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.490+0000 7fe173fff640 1 -- 172.21.15.16:0/2092539621 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560bbb6534a0 msgr2=0x560bbb6669b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:46.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.490+0000 7fe173fff640 1 --2- 172.21.15.16:0/2092539621 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560bbb6534a0 0x560bbb6669b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:46.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.490+0000 7fe173fff640 1 -- 172.21.15.16:0/2092539621 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560bbb653dc0 msgr2=0x560bbb666ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:46.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.490+0000 7fe173fff640 1 --2- 172.21.15.16:0/2092539621 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560bbb653dc0 0x560bbb666ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:46.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.490+0000 7fe173fff640 1 -- 172.21.15.16:0/2092539621 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560bbb661f60 con 0x560bbb65e680 2024-09-12T02:05:46.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.490+0000 7fe172ffd640 1 --2- 172.21.15.16:0/2092539621 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560bbb653dc0 0x560bbb666ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:05:46.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.490+0000 7fe173fff640 1 --2- 172.21.15.16:0/2092539621 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560bbb65e680 0x560bbb661760 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fe16c0097d0 tx=0x7fe16c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a229e712517f8e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:46.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.490+0000 7fe152ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:05:46.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.490+0000 7fe152ffd640 1 -- 172.21.15.16:0/2092539621 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe16c036060 con 0x560bbb65e680 2024-09-12T02:05:46.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.490+0000 7fe152ffd640 1 -- 172.21.15.16:0/2092539621 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fe16c0052e0 con 0x560bbb65e680 2024-09-12T02:05:46.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.490+0000 7fe17c078740 1 -- 172.21.15.16:0/2092539621 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe16c004d20 con 0x560bbb65e680 2024-09-12T02:05:46.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.491+0000 7fe152ffd640 1 -- 172.21.15.16:0/2092539621 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe16c016070 con 0x560bbb65e680 2024-09-12T02:05:46.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.491+0000 7fe17c078740 10 client.16569 fetch_fsmap learned FSMap version 25 2024-09-12T02:05:46.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.491+0000 7fe17c078740 1 -- 172.21.15.16:0/2092539621 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x560bbb6d9630 con 0x560bbb65e680 2024-09-12T02:05:46.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.491+0000 7fe152ffd640 1 -- 172.21.15.16:0/2092539621 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe16c004d20 con 0x560bbb65e680 2024-09-12T02:05:46.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.491+0000 7fe152ffd640 1 -- 172.21.15.16:0/2092539621 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fe16c03b070 con 0x560bbb65e680 2024-09-12T02:05:46.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.491+0000 7fe17c078740 10 client.16569 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:05:46.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.491+0000 7fe17c078740 10 client.16569 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:05:46.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.491+0000 7fe17c078740 20 client.16569 populate_metadata read hostname 'smithi016' 2024-09-12T02:05:46.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.491+0000 7fe17c078740 1 --2- 172.21.15.16:0/2092539621 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x560bbb6d9c40 0x560bbb6dc030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:46.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.491+0000 7fe17c078740 4 client.16569 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T02:05:46.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.491+0000 7fe17c078740 1 -- 172.21.15.16:0/2092539621 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560bbb5c0d60 con 0x560bbb6d9c40 2024-09-12T02:05:46.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.491+0000 7fe1737fe640 1 --2- 172.21.15.16:0/2092539621 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x560bbb6d9c40 0x560bbb6dc030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:46.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.492+0000 7fe1737fe640 1 --2- 172.21.15.16:0/2092539621 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x560bbb6d9c40 0x560bbb6dc030 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eaa15a6e63ee7240 server_cookie=23d2984b1ecf0eb9 in_seq=0 out_seq=0 2024-09-12T02:05:46.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.492+0000 7fe152ffd640 10 client.16569 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:05:46.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.492+0000 7fe152ffd640 1 -- 172.21.15.16:0/2092539621 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x560bbb5c0d60 con 0x560bbb6d9c40 2024-09-12T02:05:46.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.492+0000 7fe152ffd640 10 client.16569 handle_command_reply: tid=0 2024-09-12T02:05:46.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.492+0000 7fe17c078740 1 client.16569 shutdown 2024-09-12T02:05:46.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.492+0000 7fe17c078740 1 -- 172.21.15.16:0/2092539621 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560bbb65e680 msgr2=0x560bbb661760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:46.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.492+0000 7fe17c078740 1 --2- 172.21.15.16:0/2092539621 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560bbb65e680 0x560bbb661760 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fe16c0097d0 tx=0x7fe16c009b50 comp rx=0 tx=0).stop 2024-09-12T02:05:46.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.492+0000 7fe17c078740 1 -- 172.21.15.16:0/2092539621 shutdown_connections 2024-09-12T02:05:46.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.492+0000 7fe17c078740 1 --2- 172.21.15.16:0/2092539621 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x560bbb6d9c40 0x560bbb6dc030 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:46.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.492+0000 7fe17c078740 1 --2- 172.21.15.16:0/2092539621 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560bbb65e680 0x560bbb661760 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:46.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.492+0000 7fe17c078740 1 --2- 172.21.15.16:0/2092539621 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560bbb653dc0 0x560bbb666ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:46.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.492+0000 7fe17c078740 1 --2- 172.21.15.16:0/2092539621 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560bbb6534a0 0x560bbb6669b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:46.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.492+0000 7fe17c078740 1 -- 172.21.15.16:0/2092539621 >> 172.21.15.16:0/2092539621 conn(0x560bbb64fa30 msgr2=0x560bbb656680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:46.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.492+0000 7fe152ffd640 0 client.16569 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:05:46.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.492+0000 7fe17c078740 1 -- 172.21.15.16:0/2092539621 shutdown_connections 2024-09-12T02:05:46.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.492+0000 7fe17c078740 1 -- 172.21.15.16:0/2092539621 wait complete. 2024-09-12T02:05:46.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.492+0000 7fe17c078740 20 client.16569 trim_cache size 0 max 16384 2024-09-12T02:05:46.505 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T02:05:46.505 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-09-12T02:05:46.505 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {} 2024-09-12T02:05:46.505 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T02:05:46.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.493+0000 7fe179c9f640 1 -- 172.21.15.16:0/3110026807 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fe15c0a2b80 msgr2=0x7fe15c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:46.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.493+0000 7fe179c9f640 1 --2- 172.21.15.16:0/3110026807 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fe15c0a2b80 0x7fe15c0a5040 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fe16c005c40 tx=0x7fe16c004a40 comp rx=0 tx=0).stop 2024-09-12T02:05:46.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.493+0000 7fe179c9f640 1 -- 172.21.15.16:0/3110026807 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe174103740 msgr2=0x7fe174100f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:46.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.493+0000 7fe179c9f640 1 --2- 172.21.15.16:0/3110026807 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe174103740 0x7fe174100f30 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fe16800c970 tx=0x7fe16800ce30 comp rx=0 tx=0).stop 2024-09-12T02:05:46.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.493+0000 7fe179c9f640 1 -- 172.21.15.16:0/3110026807 shutdown_connections 2024-09-12T02:05:46.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.493+0000 7fe179c9f640 1 --2- 172.21.15.16:0/3110026807 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fe15c0a2b80 0x7fe15c0a5040 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:46.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.493+0000 7fe179c9f640 1 --2- 172.21.15.16:0/3110026807 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe174108290 0x7fe174101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:46.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.493+0000 7fe179c9f640 1 --2- 172.21.15.16:0/3110026807 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe174103740 0x7fe174100f30 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:46.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.493+0000 7fe179c9f640 1 --2- 172.21.15.16:0/3110026807 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe174102d90 0x7fe1741009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:46.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.493+0000 7fe179c9f640 1 -- 172.21.15.16:0/3110026807 >> 172.21.15.16:0/3110026807 conn(0x7fe1740fc7d0 msgr2=0x7fe174105db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:46.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.493+0000 7fe179c9f640 1 -- 172.21.15.16:0/3110026807 shutdown_connections 2024-09-12T02:05:46.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:46.493+0000 7fe179c9f640 1 -- 172.21.15.16:0/3110026807 wait complete. 2024-09-12T02:05:47.154 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:05:47 smithi016 ceph-mon[30947]: scrub summary: idle 2024-09-12T02:05:47.186 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-12T02:05:47.186 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:05:47.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:05:47 smithi184 ceph-mon[35593]: scrub summary: idle 2024-09-12T02:05:47.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:05:47 smithi033 ceph-mon[35332]: scrub summary: idle 2024-09-12T02:05:47.443 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:05:47.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:47.882+0000 7fa13802e640 1 -- 172.21.15.16:0/2050350369 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa130074590 msgr2=0x7fa130074970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:47.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:47.882+0000 7fa13802e640 1 --2- 172.21.15.16:0/2050350369 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa130074590 0x7fa130074970 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fa12c009900 tx=0x7fa12c02a480 comp rx=0 tx=0).stop 2024-09-12T02:05:47.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:47.882+0000 7fa13802e640 1 -- 172.21.15.16:0/2050350369 shutdown_connections 2024-09-12T02:05:47.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:47.882+0000 7fa13802e640 1 --2- 172.21.15.16:0/2050350369 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa130112b90 0x7fa130114f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:47.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:47.882+0000 7fa13802e640 1 --2- 172.21.15.16:0/2050350369 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa130074eb0 0x7fa130112650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:47.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:47.882+0000 7fa13802e640 1 --2- 172.21.15.16:0/2050350369 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa130074590 0x7fa130074970 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:47.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:47.882+0000 7fa13802e640 1 -- 172.21.15.16:0/2050350369 >> 172.21.15.16:0/2050350369 conn(0x7fa1300fd8c0 msgr2=0x7fa1300ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:47.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:47.882+0000 7fa13802e640 1 -- 172.21.15.16:0/2050350369 shutdown_connections 2024-09-12T02:05:47.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:47.882+0000 7fa13802e640 1 -- 172.21.15.16:0/2050350369 wait complete. 2024-09-12T02:05:47.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:47.883+0000 7fa13802e640 1 Processor -- start 2024-09-12T02:05:47.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:47.883+0000 7fa13802e640 1 -- start start 2024-09-12T02:05:47.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:47.883+0000 7fa13802e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa130074590 0x7fa1301a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:47.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:47.884+0000 7fa13802e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa130074eb0 0x7fa1301a4c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:47.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:47.884+0000 7fa13802e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa130112b90 0x7fa13019e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:47.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:47.884+0000 7fa13802e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa130118570 con 0x7fa130074590 2024-09-12T02:05:47.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:47.884+0000 7fa13802e640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa1301183f0 con 0x7fa130074eb0 2024-09-12T02:05:47.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:47.884+0000 7fa13802e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa1301186f0 con 0x7fa130112b90 2024-09-12T02:05:47.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:47.884+0000 7fa135dca640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa130074590 0x7fa1301a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:47.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:47.884+0000 7fa135dca640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa130074590 0x7fa1301a4720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60490/0 (socket says 172.21.15.16:60490) 2024-09-12T02:05:47.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:47.884+0000 7fa135dca640 1 -- 172.21.15.16:0/1341956188 learned_addr learned my addr 172.21.15.16:0/1341956188 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:05:47.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:47.884+0000 7fa1355c9640 1 --2- 172.21.15.16:0/1341956188 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa130074eb0 0x7fa1301a4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:47.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:47.884+0000 7fa1365cb640 1 --2- 172.21.15.16:0/1341956188 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa130112b90 0x7fa13019e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:47.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:47.884+0000 7fa135dca640 1 -- 172.21.15.16:0/1341956188 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa130112b90 msgr2=0x7fa13019e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:47.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:47.884+0000 7fa135dca640 1 --2- 172.21.15.16:0/1341956188 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa130112b90 0x7fa13019e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:47.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:47.884+0000 7fa135dca640 1 -- 172.21.15.16:0/1341956188 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa130074eb0 msgr2=0x7fa1301a4c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:47.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:47.884+0000 7fa135dca640 1 --2- 172.21.15.16:0/1341956188 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa130074eb0 0x7fa1301a4c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:47.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:47.884+0000 7fa135dca640 1 -- 172.21.15.16:0/1341956188 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa13019f010 con 0x7fa130074590 2024-09-12T02:05:47.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:47.884+0000 7fa135dca640 1 --2- 172.21.15.16:0/1341956188 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa130074590 0x7fa1301a4720 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fa12c006010 tx=0x7fa12c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:47.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:47.884+0000 7fa126ffd640 1 -- 172.21.15.16:0/1341956188 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa12c03f030 con 0x7fa130074590 2024-09-12T02:05:47.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:47.884+0000 7fa126ffd640 1 -- 172.21.15.16:0/1341956188 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa12c003ce0 con 0x7fa130074590 2024-09-12T02:05:47.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:47.884+0000 7fa13802e640 1 -- 172.21.15.16:0/1341956188 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa13019f2a0 con 0x7fa130074590 2024-09-12T02:05:47.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:47.884+0000 7fa126ffd640 1 -- 172.21.15.16:0/1341956188 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa12c035070 con 0x7fa130074590 2024-09-12T02:05:47.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:47.885+0000 7fa13802e640 1 -- 172.21.15.16:0/1341956188 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1301ab5f0 con 0x7fa130074590 2024-09-12T02:05:47.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:47.886+0000 7fa13802e640 1 -- 172.21.15.16:0/1341956188 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa130074970 con 0x7fa130074590 2024-09-12T02:05:47.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:47.887+0000 7fa126ffd640 1 -- 172.21.15.16:0/1341956188 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa12c042030 con 0x7fa130074590 2024-09-12T02:05:47.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:47.888+0000 7fa126ffd640 1 --2- 172.21.15.16:0/1341956188 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fa11c0a2c00 0x7fa11c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:47.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:47.888+0000 7fa1355c9640 1 --2- 172.21.15.16:0/1341956188 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fa11c0a2c00 0x7fa11c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:47.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:47.888+0000 7fa126ffd640 1 -- 172.21.15.16:0/1341956188 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fa12c0ef480 con 0x7fa130074590 2024-09-12T02:05:47.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:47.888+0000 7fa1355c9640 1 --2- 172.21.15.16:0/1341956188 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fa11c0a2c00 0x7fa11c0a50c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fa1301000e0 tx=0x7fa12000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:47.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:47.892+0000 7fa126ffd640 1 -- 172.21.15.16:0/1341956188 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa12c0bbef0 con 0x7fa130074590 2024-09-12T02:05:48.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:48.105+0000 7fa13802e640 1 -- 172.21.15.16:0/1341956188 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa130104fd0 con 0x7fa130074590 2024-09-12T02:05:48.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:48.106+0000 7fa126ffd640 1 -- 172.21.15.16:0/1341956188 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11911 (secure 0 0 0) 0x7fa12c0c01f0 con 0x7fa130074590 2024-09-12T02:05:48.106 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T02:05:48.106 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":25,"btime":"2024-09-12T02:01:37:192021+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3279123279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3279123279},{"type":"v1","addr":"172.21.15.16:6835","nonce":3279123279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4119452382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4119452382},{"type":"v1","addr":"172.21.15.16:6837","nonce":4119452382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/3263823165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3263823165},{"type":"v1","addr":"172.21.15.16:6839","nonce":3263823165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/1510290732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":1510290732},{"type":"v1","addr":"172.21.15.16:6841","nonce":1510290732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/279674603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":279674603},{"type":"v1","addr":"172.21.15.33:6833","nonce":279674603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3555728028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3555728028},{"type":"v1","addr":"172.21.15.33:6835","nonce":3555728028}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T01:59:56.117608+0000","modified":"2024-09-12T02:01:36.294760+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24482,"mds_1":24481,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6833/1563374106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1563374106},{"type":"v1","addr":"172.21.15.184:6833","nonce":1563374106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6837/1892833868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1892833868},{"type":"v1","addr":"172.21.15.33:6837","nonce":1892833868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/1457679686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1457679686},{"type":"v1","addr":"172.21.15.184:6835","nonce":1457679686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.184:6837/1793393087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":1793393087},{"type":"v1","addr":"172.21.15.184:6837","nonce":1793393087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.184:6839/4203347941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":4203347941},{"type":"v1","addr":"172.21.15.184:6839","nonce":4203347941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.33:6839/4136670204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":4136670204},{"type":"v1","addr":"172.21.15.33:6839","nonce":4136670204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24481,24482,24469]},"id":1}]} 2024-09-12T02:05:48.106 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 25 2024-09-12T02:05:48.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:48.108+0000 7fa13802e640 1 -- 172.21.15.16:0/1341956188 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fa11c0a2c00 msgr2=0x7fa11c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:48.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:48.108+0000 7fa13802e640 1 --2- 172.21.15.16:0/1341956188 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fa11c0a2c00 0x7fa11c0a50c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fa1301000e0 tx=0x7fa12000b390 comp rx=0 tx=0).stop 2024-09-12T02:05:48.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:48.108+0000 7fa13802e640 1 -- 172.21.15.16:0/1341956188 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa130074590 msgr2=0x7fa1301a4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:48.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:48.108+0000 7fa13802e640 1 --2- 172.21.15.16:0/1341956188 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa130074590 0x7fa1301a4720 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fa12c006010 tx=0x7fa12c0042f0 comp rx=0 tx=0).stop 2024-09-12T02:05:48.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:48.108+0000 7fa13802e640 1 -- 172.21.15.16:0/1341956188 shutdown_connections 2024-09-12T02:05:48.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:48.108+0000 7fa13802e640 1 --2- 172.21.15.16:0/1341956188 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fa11c0a2c00 0x7fa11c0a50c0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:48.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:48.108+0000 7fa13802e640 1 --2- 172.21.15.16:0/1341956188 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa130112b90 0x7fa13019e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:48.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:48.108+0000 7fa13802e640 1 --2- 172.21.15.16:0/1341956188 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa130074eb0 0x7fa1301a4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:48.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:48.108+0000 7fa13802e640 1 --2- 172.21.15.16:0/1341956188 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa130074590 0x7fa1301a4720 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:48.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:48.108+0000 7fa13802e640 1 -- 172.21.15.16:0/1341956188 >> 172.21.15.16:0/1341956188 conn(0x7fa1300fd8c0 msgr2=0x7fa1300ff490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:48.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:48.108+0000 7fa13802e640 1 -- 172.21.15.16:0/1341956188 shutdown_connections 2024-09-12T02:05:48.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:48.108+0000 7fa13802e640 1 -- 172.21.15.16:0/1341956188 wait complete. 2024-09-12T02:05:48.183 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:05:48 smithi016 ceph-mon[30947]: pgmap v451: 97 pgs: 97 active+clean; 2.1 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.7 KiB/s wr, 9 op/s 2024-09-12T02:05:48.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:05:48 smithi184 ceph-mon[35593]: pgmap v451: 97 pgs: 97 active+clean; 2.1 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.7 KiB/s wr, 9 op/s 2024-09-12T02:05:48.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:05:48 smithi033 ceph-mon[35332]: pgmap v451: 97 pgs: 97 active+clean; 2.1 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.7 KiB/s wr, 9 op/s 2024-09-12T02:05:48.730 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:05:48.875 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-12T02:05:49.005 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:05:49.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:05:49 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/1341956188' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:05:49.391 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:05:49 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/1341956188' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:05:49.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:05:49 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/1341956188' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:05:49.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:49.474+0000 7f793eda5640 1 -- 172.21.15.16:0/3586969719 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7938102d40 msgr2=0x7f7938103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:49.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:49.474+0000 7f793eda5640 1 --2- 172.21.15.16:0/3586969719 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7938102d40 0x7f7938103120 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f7934009900 tx=0x7f793402a480 comp rx=0 tx=0).stop 2024-09-12T02:05:49.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:49.474+0000 7f793eda5640 1 -- 172.21.15.16:0/3586969719 shutdown_connections 2024-09-12T02:05:49.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:49.474+0000 7f793eda5640 1 --2- 172.21.15.16:0/3586969719 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f793810c170 0x7f793810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:49.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:49.474+0000 7f793eda5640 1 --2- 172.21.15.16:0/3586969719 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7938103660 0x7f793810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:49.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:49.474+0000 7f793eda5640 1 --2- 172.21.15.16:0/3586969719 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7938102d40 0x7f7938103120 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:49.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:49.474+0000 7f793eda5640 1 -- 172.21.15.16:0/3586969719 >> 172.21.15.16:0/3586969719 conn(0x7f79380fc7a0 msgr2=0x7f79380febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:49.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:49.474+0000 7f793eda5640 1 -- 172.21.15.16:0/3586969719 shutdown_connections 2024-09-12T02:05:49.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:49.474+0000 7f793eda5640 1 -- 172.21.15.16:0/3586969719 wait complete. 2024-09-12T02:05:49.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:49.474+0000 7f793eda5640 1 Processor -- start 2024-09-12T02:05:49.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:49.475+0000 7f793eda5640 1 -- start start 2024-09-12T02:05:49.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:49.475+0000 7f793eda5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7938102d40 0x7f79381009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:49.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:49.475+0000 7f793eda5640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7938103660 0x7f7938100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:49.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:49.475+0000 7f793eda5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f793810c170 0x7f7938101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:49.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:49.475+0000 7f793eda5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7938111f70 con 0x7f793810c170 2024-09-12T02:05:49.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:49.475+0000 7f793eda5640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7938111df0 con 0x7f7938102d40 2024-09-12T02:05:49.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:49.475+0000 7f793eda5640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f79381120f0 con 0x7f7938103660 2024-09-12T02:05:49.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:49.475+0000 7f793d342640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f793810c170 0x7f7938101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:49.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:49.475+0000 7f793d342640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f793810c170 0x7f7938101420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60504/0 (socket says 172.21.15.16:60504) 2024-09-12T02:05:49.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:49.475+0000 7f793d342640 1 -- 172.21.15.16:0/4165228243 learned_addr learned my addr 172.21.15.16:0/4165228243 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:05:49.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:49.475+0000 7f792ffff640 1 --2- 172.21.15.16:0/4165228243 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7938103660 0x7f7938100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:49.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:49.475+0000 7f793cb41640 1 --2- 172.21.15.16:0/4165228243 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7938102d40 0x7f79381009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:49.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:49.475+0000 7f793d342640 1 -- 172.21.15.16:0/4165228243 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7938103660 msgr2=0x7f7938100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:49.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:49.475+0000 7f793d342640 1 --2- 172.21.15.16:0/4165228243 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7938103660 0x7f7938100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:49.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:49.475+0000 7f793d342640 1 -- 172.21.15.16:0/4165228243 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7938102d40 msgr2=0x7f79381009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:49.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:49.475+0000 7f793d342640 1 --2- 172.21.15.16:0/4165228243 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7938102d40 0x7f79381009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:49.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:49.475+0000 7f793d342640 1 -- 172.21.15.16:0/4165228243 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79381a6910 con 0x7f793810c170 2024-09-12T02:05:49.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:49.476+0000 7f793d342640 1 --2- 172.21.15.16:0/4165228243 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f793810c170 0x7f7938101420 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f793000eaa0 tx=0x7f793000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:49.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:49.476+0000 7f792dffb640 1 -- 172.21.15.16:0/4165228243 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f793000cdb0 con 0x7f793810c170 2024-09-12T02:05:49.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:49.476+0000 7f792dffb640 1 -- 172.21.15.16:0/4165228243 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7930005020 con 0x7f793810c170 2024-09-12T02:05:49.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:49.476+0000 7f793eda5640 1 -- 172.21.15.16:0/4165228243 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79381a6c00 con 0x7f793810c170 2024-09-12T02:05:49.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:49.476+0000 7f792dffb640 1 -- 172.21.15.16:0/4165228243 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79300116b0 con 0x7f793810c170 2024-09-12T02:05:49.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:49.476+0000 7f793eda5640 1 -- 172.21.15.16:0/4165228243 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79381a7190 con 0x7f793810c170 2024-09-12T02:05:49.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:49.477+0000 7f793eda5640 1 -- 172.21.15.16:0/4165228243 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7938103120 con 0x7f793810c170 2024-09-12T02:05:49.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:49.478+0000 7f792dffb640 1 -- 172.21.15.16:0/4165228243 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f7930005560 con 0x7f793810c170 2024-09-12T02:05:49.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:49.479+0000 7f792dffb640 1 --2- 172.21.15.16:0/4165228243 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f79200a2c00 0x7f79200a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:49.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:49.479+0000 7f793cb41640 1 --2- 172.21.15.16:0/4165228243 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f79200a2c00 0x7f79200a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:49.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:49.479+0000 7f792dffb640 1 -- 172.21.15.16:0/4165228243 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f79300cf0e0 con 0x7f793810c170 2024-09-12T02:05:49.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:49.480+0000 7f793cb41640 1 --2- 172.21.15.16:0/4165228243 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f79200a2c00 0x7f79200a50c0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f7934005ab0 tx=0x7f7934005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:49.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:49.483+0000 7f792dffb640 1 -- 172.21.15.16:0/4165228243 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7930016030 con 0x7f793810c170 2024-09-12T02:05:49.566 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:05:49.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:49.718+0000 7f793eda5640 1 -- 172.21.15.16:0/4165228243 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7938101ed0 con 0x7f793810c170 2024-09-12T02:05:49.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:49.719+0000 7f792dffb640 1 -- 172.21.15.16:0/4165228243 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11911 (secure 0 0 0) 0x7f7938101ed0 con 0x7f793810c170 2024-09-12T02:05:49.720 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T02:05:49.720 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":25,"btime":"2024-09-12T02:01:37:192021+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3279123279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3279123279},{"type":"v1","addr":"172.21.15.16:6835","nonce":3279123279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4119452382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4119452382},{"type":"v1","addr":"172.21.15.16:6837","nonce":4119452382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/3263823165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3263823165},{"type":"v1","addr":"172.21.15.16:6839","nonce":3263823165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/1510290732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":1510290732},{"type":"v1","addr":"172.21.15.16:6841","nonce":1510290732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/279674603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":279674603},{"type":"v1","addr":"172.21.15.33:6833","nonce":279674603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3555728028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3555728028},{"type":"v1","addr":"172.21.15.33:6835","nonce":3555728028}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T01:59:56.117608+0000","modified":"2024-09-12T02:01:36.294760+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24482,"mds_1":24481,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6833/1563374106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1563374106},{"type":"v1","addr":"172.21.15.184:6833","nonce":1563374106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6837/1892833868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1892833868},{"type":"v1","addr":"172.21.15.33:6837","nonce":1892833868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/1457679686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1457679686},{"type":"v1","addr":"172.21.15.184:6835","nonce":1457679686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.184:6837/1793393087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":1793393087},{"type":"v1","addr":"172.21.15.184:6837","nonce":1793393087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.184:6839/4203347941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":4203347941},{"type":"v1","addr":"172.21.15.184:6839","nonce":4203347941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.33:6839/4136670204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":4136670204},{"type":"v1","addr":"172.21.15.33:6839","nonce":4136670204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24481,24482,24469]},"id":1}]} 2024-09-12T02:05:49.720 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 25 2024-09-12T02:05:49.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:49.722+0000 7f793eda5640 1 -- 172.21.15.16:0/4165228243 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f79200a2c00 msgr2=0x7f79200a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:49.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:49.722+0000 7f793eda5640 1 --2- 172.21.15.16:0/4165228243 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f79200a2c00 0x7f79200a50c0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f7934005ab0 tx=0x7f7934005a20 comp rx=0 tx=0).stop 2024-09-12T02:05:49.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:49.722+0000 7f793eda5640 1 -- 172.21.15.16:0/4165228243 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f793810c170 msgr2=0x7f7938101420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:49.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:49.722+0000 7f793eda5640 1 --2- 172.21.15.16:0/4165228243 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f793810c170 0x7f7938101420 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f793000eaa0 tx=0x7f793000ef60 comp rx=0 tx=0).stop 2024-09-12T02:05:49.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:49.722+0000 7f793eda5640 1 -- 172.21.15.16:0/4165228243 shutdown_connections 2024-09-12T02:05:49.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:49.722+0000 7f793eda5640 1 --2- 172.21.15.16:0/4165228243 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f79200a2c00 0x7f79200a50c0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:49.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:49.722+0000 7f793eda5640 1 --2- 172.21.15.16:0/4165228243 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f793810c170 0x7f7938101420 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:49.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:49.722+0000 7f793eda5640 1 --2- 172.21.15.16:0/4165228243 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7938103660 0x7f7938100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:49.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:49.722+0000 7f793eda5640 1 --2- 172.21.15.16:0/4165228243 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7938102d40 0x7f79381009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:49.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:49.722+0000 7f793eda5640 1 -- 172.21.15.16:0/4165228243 >> 172.21.15.16:0/4165228243 conn(0x7f79380fc7a0 msgr2=0x7f79380fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:49.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:49.722+0000 7f793eda5640 1 -- 172.21.15.16:0/4165228243 shutdown_connections 2024-09-12T02:05:49.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:49.722+0000 7f793eda5640 1 -- 172.21.15.16:0/4165228243 wait complete. 2024-09-12T02:05:50.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:05:50 smithi184 ceph-mon[35593]: pgmap v452: 97 pgs: 97 active+clean; 2.1 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.7 KiB/s wr, 8 op/s 2024-09-12T02:05:50.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:05:50 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/4165228243' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:05:50.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:05:50 smithi033 ceph-mon[35332]: pgmap v452: 97 pgs: 97 active+clean; 2.1 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.7 KiB/s wr, 8 op/s 2024-09-12T02:05:50.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:05:50 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/4165228243' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:05:50.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:05:50 smithi016 ceph-mon[30947]: pgmap v452: 97 pgs: 97 active+clean; 2.1 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.7 KiB/s wr, 8 op/s 2024-09-12T02:05:50.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:05:50 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/4165228243' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:05:50.476 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24482 quiesce db --set-id 448a5fd6 --release --await' 2024-09-12T02:05:50.476 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24482 quiesce db --set-id 448a5fd6 --release --await 2024-09-12T02:05:50.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.686+0000 7fb9b498f640 1 -- 172.21.15.16:0/2644096342 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9b010c170 msgr2=0x7fb9b010e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:50.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.686+0000 7fb9b498f640 1 --2- 172.21.15.16:0/2644096342 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9b010c170 0x7fb9b010e610 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fb9a8009ca0 tx=0x7fb9a802a950 comp rx=0 tx=0).stop 2024-09-12T02:05:50.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.686+0000 7fb9b498f640 1 -- 172.21.15.16:0/2644096342 shutdown_connections 2024-09-12T02:05:50.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.686+0000 7fb9b498f640 1 --2- 172.21.15.16:0/2644096342 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9b010c170 0x7fb9b010e610 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:50.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.686+0000 7fb9b498f640 1 --2- 172.21.15.16:0/2644096342 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9b0103660 0x7fb9b010bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:50.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.686+0000 7fb9b498f640 1 --2- 172.21.15.16:0/2644096342 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb9b0102d40 0x7fb9b0103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:50.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.686+0000 7fb9b498f640 1 -- 172.21.15.16:0/2644096342 >> 172.21.15.16:0/2644096342 conn(0x7fb9b00fc7a0 msgr2=0x7fb9b00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:50.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.686+0000 7fb9b498f640 1 -- 172.21.15.16:0/2644096342 shutdown_connections 2024-09-12T02:05:50.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.686+0000 7fb9b498f640 1 -- 172.21.15.16:0/2644096342 wait complete. 2024-09-12T02:05:50.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.687+0000 7fb9b498f640 1 Processor -- start 2024-09-12T02:05:50.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.687+0000 7fb9b498f640 1 -- start start 2024-09-12T02:05:50.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.687+0000 7fb9b498f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9b0102d40 0x7fb9b01983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:50.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.687+0000 7fb9b498f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9b0103660 0x7fb9b0198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:50.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.687+0000 7fb9b498f640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb9b010c170 0x7fb9b019cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:50.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.687+0000 7fb9b498f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb9b0111f30 con 0x7fb9b0103660 2024-09-12T02:05:50.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.687+0000 7fb9b498f640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb9b0111db0 con 0x7fb9b0102d40 2024-09-12T02:05:50.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.687+0000 7fb9b498f640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb9b01120b0 con 0x7fb9b010c170 2024-09-12T02:05:50.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.688+0000 7fb9add9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9b0103660 0x7fb9b0198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:50.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.688+0000 7fb9add9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9b0103660 0x7fb9b0198910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60530/0 (socket says 172.21.15.16:60530) 2024-09-12T02:05:50.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.688+0000 7fb9add9b640 1 -- 172.21.15.16:0/4081095380 learned_addr learned my addr 172.21.15.16:0/4081095380 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:05:50.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.688+0000 7fb9ae59c640 1 --2- 172.21.15.16:0/4081095380 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9b0102d40 0x7fb9b01983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:50.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.688+0000 7fb9aed9d640 1 --2- 172.21.15.16:0/4081095380 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb9b010c170 0x7fb9b019cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:50.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.688+0000 7fb9add9b640 1 -- 172.21.15.16:0/4081095380 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb9b010c170 msgr2=0x7fb9b019cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:50.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.688+0000 7fb9add9b640 1 --2- 172.21.15.16:0/4081095380 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb9b010c170 0x7fb9b019cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:50.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.688+0000 7fb9add9b640 1 -- 172.21.15.16:0/4081095380 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9b0102d40 msgr2=0x7fb9b01983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:50.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.688+0000 7fb9add9b640 1 --2- 172.21.15.16:0/4081095380 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9b0102d40 0x7fb9b01983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:50.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.688+0000 7fb9add9b640 1 -- 172.21.15.16:0/4081095380 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9b019d380 con 0x7fb9b0103660 2024-09-12T02:05:50.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.688+0000 7fb9ae59c640 1 --2- 172.21.15.16:0/4081095380 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9b0102d40 0x7fb9b01983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:05:50.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.688+0000 7fb9add9b640 1 --2- 172.21.15.16:0/4081095380 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9b0103660 0x7fb9b0198910 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fb9a400e9f0 tx=0x7fb9a400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:50.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.689+0000 7fb99f7fe640 1 -- 172.21.15.16:0/4081095380 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9a400ccc0 con 0x7fb9b0103660 2024-09-12T02:05:50.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.689+0000 7fb99f7fe640 1 -- 172.21.15.16:0/4081095380 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9a4010910 con 0x7fb9b0103660 2024-09-12T02:05:50.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.689+0000 7fb9b498f640 1 -- 172.21.15.16:0/4081095380 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9b019d670 con 0x7fb9b0103660 2024-09-12T02:05:50.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.689+0000 7fb99f7fe640 1 -- 172.21.15.16:0/4081095380 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9a4018920 con 0x7fb9b0103660 2024-09-12T02:05:50.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.689+0000 7fb9b498f640 1 -- 172.21.15.16:0/4081095380 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9b01a4f50 con 0x7fb9b0103660 2024-09-12T02:05:50.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.691+0000 7fb99f7fe640 1 -- 172.21.15.16:0/4081095380 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fb9a4010430 con 0x7fb9b0103660 2024-09-12T02:05:50.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.692+0000 7fb99f7fe640 1 --2- 172.21.15.16:0/4081095380 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fb9980a2c00 0x7fb9980a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:50.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.692+0000 7fb9ae59c640 1 --2- 172.21.15.16:0/4081095380 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fb9980a2c00 0x7fb9980a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:50.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.692+0000 7fb9b6d68740 1 Processor -- start 2024-09-12T02:05:50.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.692+0000 7fb99f7fe640 1 -- 172.21.15.16:0/4081095380 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fb9a40cf160 con 0x7fb9b0103660 2024-09-12T02:05:50.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.692+0000 7fb9b6d68740 1 -- start start 2024-09-12T02:05:50.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.693+0000 7fb9ae59c640 1 --2- 172.21.15.16:0/4081095380 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fb9980a2c00 0x7fb9980a50c0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fb9a000bfd0 tx=0x7fb9a000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:50.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.693+0000 7fb9b6d68740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556d7dbbd840 0x556d7dbbdc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:50.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.693+0000 7fb9b6d68740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556d7dbbc120 0x556d7dbbc560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:50.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.693+0000 7fb9b6d68740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d7dbbcb80 0x556d7dbd1f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:50.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.693+0000 7fb9b6d68740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x556d7db222f0 con 0x556d7dbbd840 2024-09-12T02:05:50.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.693+0000 7fb9b6d68740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x556d7dac9cf0 con 0x556d7dbbcb80 2024-09-12T02:05:50.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.693+0000 7fb9b6d68740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x556d7db1e2b0 con 0x556d7dbbc120 2024-09-12T02:05:50.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.693+0000 7fb9ae59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556d7dbbd840 0x556d7dbbdc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:50.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.693+0000 7fb9ae59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556d7dbbd840 0x556d7dbbdc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60536/0 (socket says 172.21.15.16:60536) 2024-09-12T02:05:50.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.693+0000 7fb9ae59c640 1 -- 172.21.15.16:0/3761549818 learned_addr learned my addr 172.21.15.16:0/3761549818 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:05:50.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.693+0000 7fb9add9b640 1 --2- 172.21.15.16:0/3761549818 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556d7dbbc120 0x556d7dbbc560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:50.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.693+0000 7fb9ae59c640 1 -- 172.21.15.16:0/3761549818 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556d7dbbc120 msgr2=0x556d7dbbc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:50.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.693+0000 7fb9ae59c640 1 --2- 172.21.15.16:0/3761549818 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556d7dbbc120 0x556d7dbbc560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:50.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.693+0000 7fb9ae59c640 1 -- 172.21.15.16:0/3761549818 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d7dbbcb80 msgr2=0x556d7dbd1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:50.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.693+0000 7fb9aed9d640 1 --2- 172.21.15.16:0/3761549818 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d7dbbcb80 0x556d7dbd1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:50.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.693+0000 7fb9ae59c640 1 --2- 172.21.15.16:0/3761549818 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d7dbbcb80 0x556d7dbd1f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:50.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.693+0000 7fb9ae59c640 1 -- 172.21.15.16:0/3761549818 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d7daae3a0 con 0x556d7dbbd840 2024-09-12T02:05:50.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.693+0000 7fb9add9b640 1 --2- 172.21.15.16:0/3761549818 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556d7dbbc120 0x556d7dbbc560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:05:50.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.693+0000 7fb9aed9d640 1 --2- 172.21.15.16:0/3761549818 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d7dbbcb80 0x556d7dbd1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:05:50.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.693+0000 7fb9ae59c640 1 --2- 172.21.15.16:0/3761549818 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556d7dbbd840 0x556d7dbbdc20 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fb9a0011800 tx=0x7fb9a0013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c72b8ec15ca76393 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:50.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.694+0000 7fb99cff9640 1 -- 172.21.15.16:0/3761549818 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9a001cac0 con 0x556d7dbbd840 2024-09-12T02:05:50.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.694+0000 7fb99cff9640 1 -- 172.21.15.16:0/3761549818 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9a0011e40 con 0x556d7dbbd840 2024-09-12T02:05:50.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.694+0000 7fb99cff9640 1 -- 172.21.15.16:0/3761549818 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9a001d3f0 con 0x556d7dbbd840 2024-09-12T02:05:50.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.695+0000 7fb9b6d68740 1 -- 172.21.15.16:0/3761549818 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556d7dbbd840 msgr2=0x556d7dbbdc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:50.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.695+0000 7fb9b6d68740 1 --2- 172.21.15.16:0/3761549818 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556d7dbbd840 0x556d7dbbdc20 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fb9a0011800 tx=0x7fb9a0013000 comp rx=0 tx=0).stop 2024-09-12T02:05:50.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.695+0000 7fb9b6d68740 1 -- 172.21.15.16:0/3761549818 shutdown_connections 2024-09-12T02:05:50.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.695+0000 7fb9b6d68740 1 --2- 172.21.15.16:0/3761549818 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d7dbbcb80 0x556d7dbd1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:50.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.695+0000 7fb9b6d68740 1 --2- 172.21.15.16:0/3761549818 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556d7dbbc120 0x556d7dbbc560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:50.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.695+0000 7fb9b6d68740 1 --2- 172.21.15.16:0/3761549818 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556d7dbbd840 0x556d7dbbdc20 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:50.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.695+0000 7fb9b6d68740 1 -- 172.21.15.16:0/3761549818 >> 172.21.15.16:0/3761549818 conn(0x556d7db85310 msgr2=0x556d7dbb8c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:50.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.695+0000 7fb9b6d68740 1 -- 172.21.15.16:0/3761549818 shutdown_connections 2024-09-12T02:05:50.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.695+0000 7fb9b6d68740 1 -- 172.21.15.16:0/3761549818 wait complete. 2024-09-12T02:05:50.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.695+0000 7fb9b6d68740 1 -- start start 2024-09-12T02:05:50.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.695+0000 7fb9b6d68740 1 Processor -- start 2024-09-12T02:05:50.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.696+0000 7fb9b6d68740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d7dbbc120 0x556d7dbd1db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:50.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.696+0000 7fb9b6d68740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556d7dbbcb80 0x556d7dbcf060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:50.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.696+0000 7fb9b6d68740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556d7dbbd840 0x556d7dbcf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:50.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.696+0000 7fb9b6d68740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x556d7dbd2930 con 0x556d7dbbd840 2024-09-12T02:05:50.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.696+0000 7fb9b6d68740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x556d7dadd060 con 0x556d7dbbc120 2024-09-12T02:05:50.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.696+0000 7fb9b6d68740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x556d7dbd2ab0 con 0x556d7dbbcb80 2024-09-12T02:05:50.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.696+0000 7fb9b6d68740 10 client.0 _ll_register_callbacks cb 0x556d7d8308c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:05:50.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.696+0000 7fb9ae59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d7dbbc120 0x556d7dbd1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:50.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.696+0000 7fb9aed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556d7dbbd840 0x556d7dbcf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:50.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.696+0000 7fb9ae59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d7dbbc120 0x556d7dbd1db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.16:49506/0 (socket says 172.21.15.16:49506) 2024-09-12T02:05:50.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.696+0000 7fb9ae59c640 1 -- 172.21.15.16:0/1870243635 learned_addr learned my addr 172.21.15.16:0/1870243635 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:05:50.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.696+0000 7fb9add9b640 1 --2- 172.21.15.16:0/1870243635 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556d7dbbcb80 0x556d7dbcf060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:50.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.696+0000 7fb9aed9d640 1 -- 172.21.15.16:0/1870243635 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556d7dbbcb80 msgr2=0x556d7dbcf060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:50.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.696+0000 7fb9aed9d640 1 --2- 172.21.15.16:0/1870243635 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556d7dbbcb80 0x556d7dbcf060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:50.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.696+0000 7fb9aed9d640 1 -- 172.21.15.16:0/1870243635 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d7dbbc120 msgr2=0x556d7dbd1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:50.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.696+0000 7fb9aed9d640 1 --2- 172.21.15.16:0/1870243635 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d7dbbc120 0x556d7dbd1db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:50.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.696+0000 7fb9aed9d640 1 -- 172.21.15.16:0/1870243635 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d7d8933c0 con 0x556d7dbbd840 2024-09-12T02:05:50.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.697+0000 7fb9aed9d640 1 --2- 172.21.15.16:0/1870243635 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556d7dbbd840 0x556d7dbcf5a0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fb9a8005cc0 tx=0x7fb9a802ffe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ada955c459d917a0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:50.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.697+0000 7fb995ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:05:50.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.697+0000 7fb995ffb640 1 -- 172.21.15.16:0/1870243635 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9a8034040 con 0x556d7dbbd840 2024-09-12T02:05:50.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.697+0000 7fb995ffb640 1 -- 172.21.15.16:0/1870243635 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fb9a802f6b0 con 0x556d7dbbd840 2024-09-12T02:05:50.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.697+0000 7fb9b6d68740 1 -- 172.21.15.16:0/1870243635 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556d7db2f860 con 0x556d7dbbd840 2024-09-12T02:05:50.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.697+0000 7fb995ffb640 1 -- 172.21.15.16:0/1870243635 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fb9a8003970 con 0x556d7dbbd840 2024-09-12T02:05:50.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.697+0000 7fb9b6d68740 10 client.16593 fetch_fsmap learned FSMap version 25 2024-09-12T02:05:50.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.697+0000 7fb9b6d68740 1 -- 172.21.15.16:0/1870243635 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x556d7dbde260 con 0x556d7dbbd840 2024-09-12T02:05:50.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.697+0000 7fb995ffb640 1 -- 172.21.15.16:0/1870243635 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x556d7db2f860 con 0x556d7dbbd840 2024-09-12T02:05:50.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.698+0000 7fb995ffb640 1 -- 172.21.15.16:0/1870243635 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fb9a804d050 con 0x556d7dbbd840 2024-09-12T02:05:50.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.698+0000 7fb9b6d68740 10 client.16593 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:05:50.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.698+0000 7fb9b6d68740 10 client.16593 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-12T02:05:50.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.698+0000 7fb9b6d68740 20 client.16593 populate_metadata read hostname 'smithi016' 2024-09-12T02:05:50.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.698+0000 7fb9b6d68740 1 --2- 172.21.15.16:0/1870243635 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x556d7dbdf670 0x556d7dbe1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:50.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.698+0000 7fb9b6d68740 4 client.16593 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:05:50.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.698+0000 7fb9b6d68740 1 -- 172.21.15.16:0/1870243635 --> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556d7db29d70 con 0x556d7dbdf670 2024-09-12T02:05:50.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.698+0000 7fb9ae59c640 1 --2- 172.21.15.16:0/1870243635 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x556d7dbdf670 0x556d7dbe1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:50.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.699+0000 7fb995ffb640 10 client.16593 ms_handle_connect on v2:172.21.15.184:6836/1793393087 2024-09-12T02:05:50.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.699+0000 7fb9ae59c640 1 --2- 172.21.15.16:0/1870243635 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x556d7dbdf670 0x556d7dbe1a80 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9ddc585ee2ec95ee server_cookie=6e1a9dc2726ddae8 in_seq=0 out_seq=0 2024-09-12T02:05:50.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.699+0000 7fb995ffb640 1 -- 172.21.15.16:0/1870243635 <== mds.2 v2:172.21.15.184:6836/1793393087 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556d7db29d70 con 0x556d7dbdf670 2024-09-12T02:05:50.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.699+0000 7fb995ffb640 10 client.16593 handle_command_reply: tid=0 2024-09-12T02:05:50.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.699+0000 7fb9b6d68740 1 client.16593 shutdown 2024-09-12T02:05:50.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.700+0000 7fb9b6d68740 1 -- 172.21.15.16:0/1870243635 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556d7dbbd840 msgr2=0x556d7dbcf5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:50.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.700+0000 7fb9b6d68740 1 --2- 172.21.15.16:0/1870243635 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556d7dbbd840 0x556d7dbcf5a0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fb9a8005cc0 tx=0x7fb9a802ffe0 comp rx=0 tx=0).stop 2024-09-12T02:05:50.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.700+0000 7fb9b6d68740 1 -- 172.21.15.16:0/1870243635 shutdown_connections 2024-09-12T02:05:50.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.700+0000 7fb9b6d68740 1 --2- 172.21.15.16:0/1870243635 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x556d7dbdf670 0x556d7dbe1a80 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:50.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.700+0000 7fb9b6d68740 1 --2- 172.21.15.16:0/1870243635 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556d7dbbd840 0x556d7dbcf5a0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:50.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.700+0000 7fb9b6d68740 1 --2- 172.21.15.16:0/1870243635 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556d7dbbcb80 0x556d7dbcf060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:50.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.700+0000 7fb9b6d68740 1 --2- 172.21.15.16:0/1870243635 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d7dbbc120 0x556d7dbd1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:50.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.700+0000 7fb9b6d68740 1 -- 172.21.15.16:0/1870243635 >> 172.21.15.16:0/1870243635 conn(0x556d7db85310 msgr2=0x556d7dbbe9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:50.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.700+0000 7fb995ffb640 0 client.16593 ms_handle_reset on v2:172.21.15.184:6836/1793393087 2024-09-12T02:05:50.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.700+0000 7fb9b6d68740 1 -- 172.21.15.16:0/1870243635 shutdown_connections 2024-09-12T02:05:50.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.700+0000 7fb9b6d68740 1 -- 172.21.15.16:0/1870243635 wait complete. 2024-09-12T02:05:50.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.700+0000 7fb9b6d68740 20 client.16593 trim_cache size 0 max 16384 2024-09-12T02:05:50.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.714+0000 7fb9b6d68740 1 Processor -- start 2024-09-12T02:05:50.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.714+0000 7fb9b6d68740 1 -- start start 2024-09-12T02:05:50.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.714+0000 7fb9b6d68740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556d7dbbc120 0x556d7dbdbbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:50.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.714+0000 7fb9b6d68740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556d7dbbcb80 0x556d7dbc2f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:50.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.714+0000 7fb9b6d68740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d7dbbd840 0x556d7dbe73a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:50.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.714+0000 7fb9b6d68740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb9b01adb70 con 0x556d7dbbcb80 2024-09-12T02:05:50.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.714+0000 7fb9b6d68740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb9b01ab440 con 0x556d7dbbd840 2024-09-12T02:05:50.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.714+0000 7fb9b6d68740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb9b01a92b0 con 0x556d7dbbc120 2024-09-12T02:05:50.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.714+0000 7fb9add9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556d7dbbcb80 0x556d7dbc2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:50.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.714+0000 7fb9add9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556d7dbbcb80 0x556d7dbc2f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60560/0 (socket says 172.21.15.16:60560) 2024-09-12T02:05:50.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.714+0000 7fb9add9b640 1 -- 172.21.15.16:0/2304905563 learned_addr learned my addr 172.21.15.16:0/2304905563 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:05:50.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.714+0000 7fb9ae59c640 1 --2- 172.21.15.16:0/2304905563 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556d7dbbc120 0x556d7dbdbbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:50.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.715+0000 7fb9aed9d640 1 --2- 172.21.15.16:0/2304905563 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d7dbbd840 0x556d7dbe73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:50.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.715+0000 7fb9add9b640 1 -- 172.21.15.16:0/2304905563 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556d7dbbc120 msgr2=0x556d7dbdbbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:50.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.715+0000 7fb9add9b640 1 --2- 172.21.15.16:0/2304905563 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556d7dbbc120 0x556d7dbdbbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:50.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.715+0000 7fb9add9b640 1 -- 172.21.15.16:0/2304905563 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d7dbbd840 msgr2=0x556d7dbe73a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:50.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.715+0000 7fb9add9b640 1 --2- 172.21.15.16:0/2304905563 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d7dbbd840 0x556d7dbe73a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:50.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.715+0000 7fb9add9b640 1 -- 172.21.15.16:0/2304905563 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d7dbccf40 con 0x556d7dbbcb80 2024-09-12T02:05:50.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.715+0000 7fb9aed9d640 1 --2- 172.21.15.16:0/2304905563 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d7dbbd840 0x556d7dbe73a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:05:50.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.715+0000 7fb9add9b640 1 --2- 172.21.15.16:0/2304905563 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556d7dbbcb80 0x556d7dbc2f30 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fb9a4097e80 tx=0x7fb9a400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b419b73338311551 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:50.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.715+0000 7fb997fff640 1 -- 172.21.15.16:0/2304905563 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9a4010dc0 con 0x556d7dbbcb80 2024-09-12T02:05:50.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.715+0000 7fb997fff640 1 -- 172.21.15.16:0/2304905563 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9a4099d70 con 0x556d7dbbcb80 2024-09-12T02:05:50.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.715+0000 7fb997fff640 1 -- 172.21.15.16:0/2304905563 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9a409b1b0 con 0x556d7dbbcb80 2024-09-12T02:05:50.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.716+0000 7fb9b6d68740 1 -- 172.21.15.16:0/2304905563 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556d7dbbcb80 msgr2=0x556d7dbc2f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:50.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.716+0000 7fb9b6d68740 1 --2- 172.21.15.16:0/2304905563 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556d7dbbcb80 0x556d7dbc2f30 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fb9a4097e80 tx=0x7fb9a400cf60 comp rx=0 tx=0).stop 2024-09-12T02:05:50.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.716+0000 7fb9b6d68740 1 -- 172.21.15.16:0/2304905563 shutdown_connections 2024-09-12T02:05:50.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.716+0000 7fb9b6d68740 1 --2- 172.21.15.16:0/2304905563 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d7dbbd840 0x556d7dbe73a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:50.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.716+0000 7fb9b6d68740 1 --2- 172.21.15.16:0/2304905563 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556d7dbbcb80 0x556d7dbc2f30 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:50.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.716+0000 7fb9b6d68740 1 --2- 172.21.15.16:0/2304905563 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556d7dbbc120 0x556d7dbdbbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:50.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.716+0000 7fb9b6d68740 1 -- 172.21.15.16:0/2304905563 >> 172.21.15.16:0/2304905563 conn(0x556d7db85310 msgr2=0x556d7dbc2820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:50.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.716+0000 7fb9b6d68740 1 -- 172.21.15.16:0/2304905563 shutdown_connections 2024-09-12T02:05:50.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.716+0000 7fb9b6d68740 1 -- 172.21.15.16:0/2304905563 wait complete. 2024-09-12T02:05:50.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.717+0000 7fb9b6d68740 1 -- start start 2024-09-12T02:05:50.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.717+0000 7fb9b6d68740 1 Processor -- start 2024-09-12T02:05:50.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.717+0000 7fb9b6d68740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556d7dbbc120 0x556d7dbd81e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:50.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.717+0000 7fb9b6d68740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556d7dbbcb80 0x556d7dbd5530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:50.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.717+0000 7fb9b6d68740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d7dbbd840 0x556d7dbd5a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:50.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.717+0000 7fb9b6d68740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb9b01a9900 con 0x556d7dbbc120 2024-09-12T02:05:50.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.717+0000 7fb9b6d68740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb9b01aad70 con 0x556d7dbbd840 2024-09-12T02:05:50.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.717+0000 7fb9b6d68740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb9b01a9fd0 con 0x556d7dbbcb80 2024-09-12T02:05:50.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.717+0000 7fb9b6d68740 10 client.0 _ll_register_callbacks cb 0x7fb9a8009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:05:50.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.717+0000 7fb9ae59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556d7dbbc120 0x556d7dbd81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:50.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.717+0000 7fb9ae59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556d7dbbc120 0x556d7dbd81e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60574/0 (socket says 172.21.15.16:60574) 2024-09-12T02:05:50.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.717+0000 7fb9ae59c640 1 -- 172.21.15.16:0/2728382711 learned_addr learned my addr 172.21.15.16:0/2728382711 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:05:50.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.717+0000 7fb9aed9d640 1 --2- 172.21.15.16:0/2728382711 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d7dbbd840 0x556d7dbd5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:50.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.718+0000 7fb9add9b640 1 --2- 172.21.15.16:0/2728382711 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556d7dbbcb80 0x556d7dbd5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:50.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.718+0000 7fb9ae59c640 1 -- 172.21.15.16:0/2728382711 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556d7dbbcb80 msgr2=0x556d7dbd5530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:50.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.718+0000 7fb9ae59c640 1 --2- 172.21.15.16:0/2728382711 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556d7dbbcb80 0x556d7dbd5530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:50.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.718+0000 7fb9ae59c640 1 -- 172.21.15.16:0/2728382711 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d7dbbd840 msgr2=0x556d7dbd5a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:50.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.718+0000 7fb9ae59c640 1 --2- 172.21.15.16:0/2728382711 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d7dbbd840 0x556d7dbd5a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:50.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.718+0000 7fb9ae59c640 1 -- 172.21.15.16:0/2728382711 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d7dbd5fb0 con 0x556d7dbbc120 2024-09-12T02:05:50.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.718+0000 7fb9add9b640 1 --2- 172.21.15.16:0/2728382711 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556d7dbbcb80 0x556d7dbd5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:05:50.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.718+0000 7fb9aed9d640 1 --2- 172.21.15.16:0/2728382711 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d7dbbd840 0x556d7dbd5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:05:50.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.718+0000 7fb9ae59c640 1 --2- 172.21.15.16:0/2728382711 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556d7dbbc120 0x556d7dbd81e0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fb9a0010c30 tx=0x7fb9a001ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eca0ad3c9f1bcdd8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:50.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.718+0000 7fb995ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:05:50.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.718+0000 7fb995ffb640 1 -- 172.21.15.16:0/2728382711 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9a0042060 con 0x556d7dbbc120 2024-09-12T02:05:50.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.718+0000 7fb995ffb640 1 -- 172.21.15.16:0/2728382711 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fb9a001cac0 con 0x556d7dbbc120 2024-09-12T02:05:50.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.718+0000 7fb9b6d68740 1 -- 172.21.15.16:0/2728382711 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb9a4099d70 con 0x556d7dbbc120 2024-09-12T02:05:50.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.718+0000 7fb995ffb640 1 -- 172.21.15.16:0/2728382711 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fb9a001e990 con 0x556d7dbbc120 2024-09-12T02:05:50.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.718+0000 7fb9b6d68740 10 client.16599 fetch_fsmap learned FSMap version 25 2024-09-12T02:05:50.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.719+0000 7fb9b6d68740 1 -- 172.21.15.16:0/2728382711 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x556d7dc4a0d0 con 0x556d7dbbc120 2024-09-12T02:05:50.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.719+0000 7fb995ffb640 1 -- 172.21.15.16:0/2728382711 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9a4099d70 con 0x556d7dbbc120 2024-09-12T02:05:50.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.719+0000 7fb995ffb640 1 -- 172.21.15.16:0/2728382711 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fb9a0047070 con 0x556d7dbbc120 2024-09-12T02:05:50.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.719+0000 7fb9b6d68740 10 client.16599 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:05:50.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.719+0000 7fb9b6d68740 10 client.16599 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-12T02:05:50.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.719+0000 7fb9b6d68740 20 client.16599 populate_metadata read hostname 'smithi016' 2024-09-12T02:05:50.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.719+0000 7fb9b6d68740 1 --2- 172.21.15.16:0/2728382711 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x556d7dc4a4b0 0x556d7dc4c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:50.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.719+0000 7fb9b6d68740 4 client.16599 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T02:05:50.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.719+0000 7fb9b6d68740 1 -- 172.21.15.16:0/2728382711 --> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556d7daa6170 con 0x556d7dc4a4b0 2024-09-12T02:05:50.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.719+0000 7fb9add9b640 1 --2- 172.21.15.16:0/2728382711 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x556d7dc4a4b0 0x556d7dc4c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:50.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.720+0000 7fb9add9b640 1 --2- 172.21.15.16:0/2728382711 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x556d7dc4a4b0 0x556d7dc4c8a0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=41a7d16bdaadd584 server_cookie=f1950d061ac39fac in_seq=0 out_seq=0 2024-09-12T02:05:50.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.720+0000 7fb995ffb640 10 client.16599 ms_handle_connect on v2:172.21.15.184:6836/1793393087 2024-09-12T02:05:50.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.720+0000 7fb995ffb640 1 -- 172.21.15.16:0/2728382711 <== mds.2 v2:172.21.15.184:6836/1793393087 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556d7daa6170 con 0x556d7dc4a4b0 2024-09-12T02:05:50.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.720+0000 7fb995ffb640 10 client.16599 handle_command_reply: tid=0 2024-09-12T02:05:50.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.720+0000 7fb9b6d68740 1 client.16599 shutdown 2024-09-12T02:05:50.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.720+0000 7fb9b6d68740 1 -- 172.21.15.16:0/2728382711 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556d7dbbc120 msgr2=0x556d7dbd81e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:50.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.720+0000 7fb9b6d68740 1 --2- 172.21.15.16:0/2728382711 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556d7dbbc120 0x556d7dbd81e0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fb9a0010c30 tx=0x7fb9a001ec20 comp rx=0 tx=0).stop 2024-09-12T02:05:50.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.720+0000 7fb9b6d68740 1 -- 172.21.15.16:0/2728382711 shutdown_connections 2024-09-12T02:05:50.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.720+0000 7fb9b6d68740 1 --2- 172.21.15.16:0/2728382711 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d7dbbd840 0x556d7dbd5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:50.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.720+0000 7fb9b6d68740 1 --2- 172.21.15.16:0/2728382711 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556d7dbbcb80 0x556d7dbd5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:50.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.720+0000 7fb9b6d68740 1 --2- 172.21.15.16:0/2728382711 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x556d7dc4a4b0 0x556d7dc4c8a0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:50.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.720+0000 7fb9b6d68740 1 --2- 172.21.15.16:0/2728382711 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556d7dbbc120 0x556d7dbd81e0 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:50.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.720+0000 7fb995ffb640 0 client.16599 ms_handle_reset on v2:172.21.15.184:6836/1793393087 2024-09-12T02:05:50.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.720+0000 7fb9b6d68740 1 -- 172.21.15.16:0/2728382711 >> 172.21.15.16:0/2728382711 conn(0x556d7db85310 msgr2=0x556d7dbd08c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:50.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.720+0000 7fb9b6d68740 1 -- 172.21.15.16:0/2728382711 shutdown_connections 2024-09-12T02:05:50.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.720+0000 7fb9b6d68740 1 -- 172.21.15.16:0/2728382711 wait complete. 2024-09-12T02:05:50.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.721+0000 7fb9b6d68740 20 client.16599 trim_cache size 0 max 16384 2024-09-12T02:05:50.729 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T02:05:50.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.721+0000 7fb9b498f640 1 -- 172.21.15.16:0/4081095380 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fb9980a2c00 msgr2=0x7fb9980a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:50.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.721+0000 7fb9b498f640 1 --2- 172.21.15.16:0/4081095380 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fb9980a2c00 0x7fb9980a50c0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fb9a000bfd0 tx=0x7fb9a000b390 comp rx=0 tx=0).stop 2024-09-12T02:05:50.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.721+0000 7fb9b498f640 1 -- 172.21.15.16:0/4081095380 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9b0103660 msgr2=0x7fb9b0198910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:50.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.721+0000 7fb9b498f640 1 --2- 172.21.15.16:0/4081095380 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9b0103660 0x7fb9b0198910 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fb9a400e9f0 tx=0x7fb9a400eeb0 comp rx=0 tx=0).stop 2024-09-12T02:05:50.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.721+0000 7fb9b498f640 1 -- 172.21.15.16:0/4081095380 shutdown_connections 2024-09-12T02:05:50.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.721+0000 7fb9b498f640 1 --2- 172.21.15.16:0/4081095380 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fb9980a2c00 0x7fb9980a50c0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:50.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.721+0000 7fb9b498f640 1 --2- 172.21.15.16:0/4081095380 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb9b010c170 0x7fb9b019cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:50.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.721+0000 7fb9b498f640 1 --2- 172.21.15.16:0/4081095380 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9b0103660 0x7fb9b0198910 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:50.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.721+0000 7fb9b498f640 1 --2- 172.21.15.16:0/4081095380 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9b0102d40 0x7fb9b01983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:50.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.721+0000 7fb9b498f640 1 -- 172.21.15.16:0/4081095380 >> 172.21.15.16:0/4081095380 conn(0x7fb9b00fc7a0 msgr2=0x7fb9b00fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:50.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.721+0000 7fb9b498f640 1 -- 172.21.15.16:0/4081095380 shutdown_connections 2024-09-12T02:05:50.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:50.721+0000 7fb9b498f640 1 -- 172.21.15.16:0/4081095380 wait complete. 2024-09-12T02:05:50.972 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:05:51.445 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-12T02:05:51.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.855+0000 7fa8973e6640 1 -- 172.21.15.16:0/1586908101 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa890102790 msgr2=0x7fa8901067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:51.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.855+0000 7fa8973e6640 1 --2- 172.21.15.16:0/1586908101 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa890102790 0x7fa8901067e0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fa880009900 tx=0x7fa88002a410 comp rx=0 tx=0).stop 2024-09-12T02:05:51.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.855+0000 7fa8973e6640 1 -- 172.21.15.16:0/1586908101 shutdown_connections 2024-09-12T02:05:51.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.855+0000 7fa8973e6640 1 --2- 172.21.15.16:0/1586908101 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa890106f10 0x7fa89010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:51.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.855+0000 7fa8973e6640 1 --2- 172.21.15.16:0/1586908101 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa890102790 0x7fa8901067e0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:51.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.855+0000 7fa8973e6640 1 --2- 172.21.15.16:0/1586908101 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa890101de0 0x7fa8901021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:51.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.855+0000 7fa8973e6640 1 -- 172.21.15.16:0/1586908101 >> 172.21.15.16:0/1586908101 conn(0x7fa8900fd8c0 msgr2=0x7fa8900ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:51.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.855+0000 7fa8973e6640 1 -- 172.21.15.16:0/1586908101 shutdown_connections 2024-09-12T02:05:51.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.855+0000 7fa8973e6640 1 -- 172.21.15.16:0/1586908101 wait complete. 2024-09-12T02:05:51.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.855+0000 7fa8973e6640 1 Processor -- start 2024-09-12T02:05:51.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.856+0000 7fa8973e6640 1 -- start start 2024-09-12T02:05:51.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.856+0000 7fa8973e6640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa890101de0 0x7fa8900744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:51.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.856+0000 7fa8973e6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa890102790 0x7fa890074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:51.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.856+0000 7fa8973e6640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa890106f10 0x7fa890074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:51.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.856+0000 7fa8973e6640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa890111f70 con 0x7fa890102790 2024-09-12T02:05:51.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.856+0000 7fa8973e6640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa890111df0 con 0x7fa890106f10 2024-09-12T02:05:51.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.856+0000 7fa8973e6640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa8901120f0 con 0x7fa890101de0 2024-09-12T02:05:51.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.856+0000 7fa894981640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa890102790 0x7fa890074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:51.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.856+0000 7fa894981640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa890102790 0x7fa890074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54118/0 (socket says 172.21.15.16:54118) 2024-09-12T02:05:51.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.856+0000 7fa894981640 1 -- 172.21.15.16:0/2440232921 learned_addr learned my addr 172.21.15.16:0/2440232921 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:05:51.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.856+0000 7fa895182640 1 --2- 172.21.15.16:0/2440232921 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa890101de0 0x7fa8900744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:51.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.856+0000 7fa894981640 1 -- 172.21.15.16:0/2440232921 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa890101de0 msgr2=0x7fa8900744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:51.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.856+0000 7fa894981640 1 --2- 172.21.15.16:0/2440232921 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa890101de0 0x7fa8900744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:51.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.856+0000 7fa894981640 1 -- 172.21.15.16:0/2440232921 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa890106f10 msgr2=0x7fa890074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:51.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.856+0000 7fa895983640 1 --2- 172.21.15.16:0/2440232921 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa890106f10 0x7fa890074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:51.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.856+0000 7fa894981640 1 --2- 172.21.15.16:0/2440232921 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa890106f10 0x7fa890074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:51.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.856+0000 7fa894981640 1 -- 172.21.15.16:0/2440232921 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8901a6950 con 0x7fa890102790 2024-09-12T02:05:51.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.856+0000 7fa895983640 1 --2- 172.21.15.16:0/2440232921 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa890106f10 0x7fa890074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:05:51.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.857+0000 7fa894981640 1 --2- 172.21.15.16:0/2440232921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa890102790 0x7fa890074a30 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fa880006010 tx=0x7fa8800042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:51.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.857+0000 7fa8867fc640 1 -- 172.21.15.16:0/2440232921 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa88003f030 con 0x7fa890102790 2024-09-12T02:05:51.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.857+0000 7fa8867fc640 1 -- 172.21.15.16:0/2440232921 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa880003ce0 con 0x7fa890102790 2024-09-12T02:05:51.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.857+0000 7fa8973e6640 1 -- 172.21.15.16:0/2440232921 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8901a6be0 con 0x7fa890102790 2024-09-12T02:05:51.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.857+0000 7fa8867fc640 1 -- 172.21.15.16:0/2440232921 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa880035070 con 0x7fa890102790 2024-09-12T02:05:51.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.857+0000 7fa8973e6640 1 -- 172.21.15.16:0/2440232921 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8901a7110 con 0x7fa890102790 2024-09-12T02:05:51.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.859+0000 7fa8867fc640 1 -- 172.21.15.16:0/2440232921 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa880042030 con 0x7fa890102790 2024-09-12T02:05:51.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.860+0000 7fa8997bf740 1 Processor -- start 2024-09-12T02:05:51.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.860+0000 7fa8997bf740 1 -- start start 2024-09-12T02:05:51.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.860+0000 7fa8997bf740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560be3c88060 0x560be3c88440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:51.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.860+0000 7fa8997bf740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560be3c88980 0x560be3c92d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:51.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.860+0000 7fa8997bf740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560be3c93240 0x560be3c95630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:51.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.860+0000 7fa8997bf740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x560be3bed620 con 0x560be3c88060 2024-09-12T02:05:51.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.860+0000 7fa8997bf740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x560be3b94e50 con 0x560be3c93240 2024-09-12T02:05:51.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.860+0000 7fa8997bf740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x560be3be95e0 con 0x560be3c88980 2024-09-12T02:05:51.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.860+0000 7fa8867fc640 1 --2- 172.21.15.16:0/2440232921 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fa87c0a2c00 0x7fa87c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:51.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.861+0000 7fa895182640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560be3c88980 0x560be3c92d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:51.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.861+0000 7fa895983640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560be3c88060 0x560be3c88440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:51.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.861+0000 7fa895182640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560be3c88980 0x560be3c92d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:48726/0 (socket says 172.21.15.16:48726) 2024-09-12T02:05:51.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.861+0000 7fa895182640 1 -- 172.21.15.16:0/3042693455 learned_addr learned my addr 172.21.15.16:0/3042693455 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:05:51.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.861+0000 7fa8867fc640 1 -- 172.21.15.16:0/2440232921 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fa8800ef5b0 con 0x7fa890102790 2024-09-12T02:05:51.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.861+0000 7fa895983640 1 --2- 172.21.15.16:0/2440232921 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fa87c0a2c00 0x7fa87c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:51.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.861+0000 7fa894981640 1 --2- 172.21.15.16:0/3042693455 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560be3c93240 0x560be3c95630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:51.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.861+0000 7fa895983640 1 -- 172.21.15.16:0/3042693455 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560be3c88980 msgr2=0x560be3c92d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:51.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.861+0000 7fa895983640 1 --2- 172.21.15.16:0/3042693455 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560be3c88980 0x560be3c92d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:51.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.861+0000 7fa895983640 1 -- 172.21.15.16:0/3042693455 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560be3c93240 msgr2=0x560be3c95630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:51.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.861+0000 7fa895983640 1 --2- 172.21.15.16:0/3042693455 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560be3c93240 0x560be3c95630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:51.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.861+0000 7fa895983640 1 -- 172.21.15.16:0/3042693455 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560be3b85d40 con 0x560be3c88060 2024-09-12T02:05:51.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.861+0000 7fa894981640 1 --2- 172.21.15.16:0/3042693455 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560be3c93240 0x560be3c95630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:05:51.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.861+0000 7fa895983640 1 --2- 172.21.15.16:0/3042693455 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560be3c88060 0x560be3c88440 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fa888008fd0 tx=0x7fa888014d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bba627da446f2255 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:51.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.862+0000 7fa87b7fe640 1 -- 172.21.15.16:0/3042693455 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa888021070 con 0x560be3c88060 2024-09-12T02:05:51.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.862+0000 7fa87b7fe640 1 -- 172.21.15.16:0/3042693455 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa88801cc90 con 0x560be3c88060 2024-09-12T02:05:51.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.862+0000 7fa87b7fe640 1 -- 172.21.15.16:0/3042693455 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa88800b370 con 0x560be3c88060 2024-09-12T02:05:51.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.862+0000 7fa8997bf740 1 -- 172.21.15.16:0/3042693455 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560be3c88060 msgr2=0x560be3c88440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:51.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.862+0000 7fa8997bf740 1 --2- 172.21.15.16:0/3042693455 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560be3c88060 0x560be3c88440 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fa888008fd0 tx=0x7fa888014d20 comp rx=0 tx=0).stop 2024-09-12T02:05:51.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.862+0000 7fa895983640 1 --2- 172.21.15.16:0/2440232921 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fa87c0a2c00 0x7fa87c0a50c0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fa88801e000 tx=0x7fa88801a000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:51.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.862+0000 7fa8997bf740 1 -- 172.21.15.16:0/3042693455 shutdown_connections 2024-09-12T02:05:51.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.862+0000 7fa8997bf740 1 --2- 172.21.15.16:0/3042693455 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560be3c93240 0x560be3c95630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:51.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.862+0000 7fa8997bf740 1 --2- 172.21.15.16:0/3042693455 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560be3c88980 0x560be3c92d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:51.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.862+0000 7fa8997bf740 1 --2- 172.21.15.16:0/3042693455 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560be3c88060 0x560be3c88440 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:51.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.862+0000 7fa8997bf740 1 -- 172.21.15.16:0/3042693455 >> 172.21.15.16:0/3042693455 conn(0x560be3c845f0 msgr2=0x560be3c849d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:51.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.863+0000 7fa8997bf740 1 -- 172.21.15.16:0/3042693455 shutdown_connections 2024-09-12T02:05:51.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.863+0000 7fa8997bf740 1 -- 172.21.15.16:0/3042693455 wait complete. 2024-09-12T02:05:51.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.863+0000 7fa8997bf740 1 -- start start 2024-09-12T02:05:51.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.863+0000 7fa8997bf740 1 Processor -- start 2024-09-12T02:05:51.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.864+0000 7fa8997bf740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560be3c88060 0x560be3c901b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:51.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.864+0000 7fa8997bf740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560be3c88980 0x560be3c906f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:51.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.864+0000 7fa8997bf740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560be3c93240 0x560be3c90c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:51.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.864+0000 7fa8997bf740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x560be3c95e10 con 0x560be3c88060 2024-09-12T02:05:51.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.864+0000 7fa8997bf740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x560be3b75f10 con 0x560be3c88980 2024-09-12T02:05:51.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.864+0000 7fa8997bf740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x560be3c95f90 con 0x560be3c93240 2024-09-12T02:05:51.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.864+0000 7fa8997bf740 10 client.0 _ll_register_callbacks cb 0x560be3b99db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:05:51.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.864+0000 7fa895182640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560be3c88060 0x560be3c901b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:51.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.864+0000 7fa895182640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560be3c88060 0x560be3c901b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54134/0 (socket says 172.21.15.16:54134) 2024-09-12T02:05:51.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.864+0000 7fa895182640 1 -- 172.21.15.16:0/2398216492 learned_addr learned my addr 172.21.15.16:0/2398216492 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:05:51.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.864+0000 7fa895983640 1 --2- 172.21.15.16:0/2398216492 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560be3c93240 0x560be3c90c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:51.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.864+0000 7fa894981640 1 --2- 172.21.15.16:0/2398216492 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560be3c88980 0x560be3c906f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:51.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.864+0000 7fa895182640 1 -- 172.21.15.16:0/2398216492 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560be3c93240 msgr2=0x560be3c90c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:51.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.864+0000 7fa895182640 1 --2- 172.21.15.16:0/2398216492 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560be3c93240 0x560be3c90c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:51.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.864+0000 7fa895182640 1 -- 172.21.15.16:0/2398216492 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560be3c88980 msgr2=0x560be3c906f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:51.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.864+0000 7fa895182640 1 --2- 172.21.15.16:0/2398216492 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560be3c88980 0x560be3c906f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:51.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.864+0000 7fa895182640 1 -- 172.21.15.16:0/2398216492 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560be38fc580 con 0x560be3c88060 2024-09-12T02:05:51.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.864+0000 7fa894981640 1 --2- 172.21.15.16:0/2398216492 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560be3c88980 0x560be3c906f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:05:51.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.864+0000 7fa895182640 1 --2- 172.21.15.16:0/2398216492 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560be3c88060 0x560be3c901b0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fa88c00bee0 tx=0x7fa88c012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f54cd9e13bbfd85 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:51.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.864+0000 7fa878ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:05:51.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.865+0000 7fa878ff9640 1 -- 172.21.15.16:0/2398216492 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa88c004230 con 0x560be3c88060 2024-09-12T02:05:51.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.865+0000 7fa878ff9640 1 -- 172.21.15.16:0/2398216492 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fa88c0043d0 con 0x560be3c88060 2024-09-12T02:05:51.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.865+0000 7fa8997bf740 1 -- 172.21.15.16:0/2398216492 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa88801cc90 con 0x560be3c88060 2024-09-12T02:05:51.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.865+0000 7fa878ff9640 1 -- 172.21.15.16:0/2398216492 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa88c010760 con 0x560be3c88060 2024-09-12T02:05:51.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.865+0000 7fa8997bf740 10 client.16611 fetch_fsmap learned FSMap version 25 2024-09-12T02:05:51.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.865+0000 7fa8997bf740 1 -- 172.21.15.16:0/2398216492 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x560be39c6960 con 0x560be3c88060 2024-09-12T02:05:51.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.865+0000 7fa878ff9640 1 -- 172.21.15.16:0/2398216492 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa88801cc90 con 0x560be3c88060 2024-09-12T02:05:51.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.865+0000 7fa878ff9640 1 -- 172.21.15.16:0/2398216492 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fa88c003c20 con 0x560be3c88060 2024-09-12T02:05:51.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.865+0000 7fa8997bf740 10 client.16611 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:05:51.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.865+0000 7fa8997bf740 10 client.16611 resolve_mds: validated gid 24482 aka daemon mds.b 2024-09-12T02:05:51.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.865+0000 7fa8997bf740 20 client.16611 populate_metadata read hostname 'smithi016' 2024-09-12T02:05:51.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.865+0000 7fa8997bf740 1 --2- 172.21.15.16:0/2398216492 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x560be3ca2e30 0x560be3ca52a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:51.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.865+0000 7fa8997bf740 4 client.16611 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:05:51.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.865+0000 7fa8997bf740 1 -- 172.21.15.16:0/2398216492 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560be3bfab90 con 0x560be3ca2e30 2024-09-12T02:05:51.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.866+0000 7fa894981640 1 --2- 172.21.15.16:0/2398216492 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x560be3ca2e30 0x560be3ca52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:51.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.866+0000 7fa894981640 1 --2- 172.21.15.16:0/2398216492 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x560be3ca2e30 0x560be3ca52a0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b173a5f3795c5619 server_cookie=153b821a16c740f2 in_seq=0 out_seq=0 2024-09-12T02:05:51.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.866+0000 7fa878ff9640 10 client.16611 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:05:51.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.867+0000 7fa878ff9640 1 -- 172.21.15.16:0/2398216492 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560be3bfab90 con 0x560be3ca2e30 2024-09-12T02:05:51.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.867+0000 7fa878ff9640 10 client.16611 handle_command_reply: tid=0 2024-09-12T02:05:51.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.867+0000 7fa8997bf740 1 client.16611 shutdown 2024-09-12T02:05:51.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.867+0000 7fa8997bf740 1 -- 172.21.15.16:0/2398216492 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560be3c88060 msgr2=0x560be3c901b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:51.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.867+0000 7fa8997bf740 1 --2- 172.21.15.16:0/2398216492 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560be3c88060 0x560be3c901b0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fa88c00bee0 tx=0x7fa88c012820 comp rx=0 tx=0).stop 2024-09-12T02:05:51.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.867+0000 7fa8997bf740 1 -- 172.21.15.16:0/2398216492 shutdown_connections 2024-09-12T02:05:51.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.867+0000 7fa8997bf740 1 --2- 172.21.15.16:0/2398216492 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x560be3ca2e30 0x560be3ca52a0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:51.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.867+0000 7fa8997bf740 1 --2- 172.21.15.16:0/2398216492 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560be3c93240 0x560be3c90c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:51.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.867+0000 7fa8997bf740 1 --2- 172.21.15.16:0/2398216492 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560be3c88980 0x560be3c906f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:51.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.867+0000 7fa8997bf740 1 --2- 172.21.15.16:0/2398216492 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560be3c88060 0x560be3c901b0 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:51.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.867+0000 7fa8997bf740 1 -- 172.21.15.16:0/2398216492 >> 172.21.15.16:0/2398216492 conn(0x560be3c845f0 msgr2=0x560be3c8a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:51.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.867+0000 7fa878ff9640 0 client.16611 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:05:51.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.868+0000 7fa8997bf740 1 -- 172.21.15.16:0/2398216492 shutdown_connections 2024-09-12T02:05:51.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.868+0000 7fa8997bf740 1 -- 172.21.15.16:0/2398216492 wait complete. 2024-09-12T02:05:51.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.868+0000 7fa8997bf740 20 client.16611 trim_cache size 0 max 16384 2024-09-12T02:05:51.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.887+0000 7fa8997bf740 1 Processor -- start 2024-09-12T02:05:51.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.887+0000 7fa8997bf740 1 -- start start 2024-09-12T02:05:51.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.888+0000 7fa8997bf740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560be3c88060 0x560be3c83640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:51.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.888+0000 7fa8997bf740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560be3c88980 0x560be3caab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:51.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.888+0000 7fa8997bf740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560be3c93240 0x560be3cab0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:51.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.888+0000 7fa8997bf740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa8901b1ed0 con 0x560be3c88980 2024-09-12T02:05:51.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.888+0000 7fa8997bf740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa8901af7a0 con 0x560be3c93240 2024-09-12T02:05:51.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.888+0000 7fa894981640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560be3c88980 0x560be3caab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:51.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.888+0000 7fa894981640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560be3c88980 0x560be3caab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54144/0 (socket says 172.21.15.16:54144) 2024-09-12T02:05:51.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.888+0000 7fa894981640 1 -- 172.21.15.16:0/3478690907 learned_addr learned my addr 172.21.15.16:0/3478690907 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:05:51.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.888+0000 7fa8997bf740 1 -- 172.21.15.16:0/3478690907 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa8901ad5f0 con 0x560be3c88060 2024-09-12T02:05:51.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.888+0000 7fa895182640 1 --2- 172.21.15.16:0/3478690907 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560be3c88060 0x560be3c83640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:51.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.888+0000 7fa895983640 1 --2- 172.21.15.16:0/3478690907 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560be3c93240 0x560be3cab0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:51.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.888+0000 7fa894981640 1 -- 172.21.15.16:0/3478690907 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560be3c88060 msgr2=0x560be3c83640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:51.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.888+0000 7fa894981640 1 --2- 172.21.15.16:0/3478690907 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560be3c88060 0x560be3c83640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:51.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.888+0000 7fa894981640 1 -- 172.21.15.16:0/3478690907 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560be3c93240 msgr2=0x560be3cab0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:51.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.888+0000 7fa894981640 1 --2- 172.21.15.16:0/3478690907 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560be3c93240 0x560be3cab0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:51.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.888+0000 7fa894981640 1 -- 172.21.15.16:0/3478690907 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560be3c89570 con 0x560be3c88980 2024-09-12T02:05:51.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.888+0000 7fa895983640 1 --2- 172.21.15.16:0/3478690907 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560be3c93240 0x560be3cab0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:05:51.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.889+0000 7fa894981640 1 --2- 172.21.15.16:0/3478690907 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560be3c88980 0x560be3caab70 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fa880000c00 tx=0x7fa8800dc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa56395cfc4afecd server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:51.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.889+0000 7fa87affd640 1 -- 172.21.15.16:0/3478690907 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa880049070 con 0x560be3c88980 2024-09-12T02:05:51.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.889+0000 7fa87affd640 1 -- 172.21.15.16:0/3478690907 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8800bc5d0 con 0x560be3c88980 2024-09-12T02:05:51.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.889+0000 7fa87affd640 1 -- 172.21.15.16:0/3478690907 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8800b8540 con 0x560be3c88980 2024-09-12T02:05:51.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.889+0000 7fa8997bf740 1 -- 172.21.15.16:0/3478690907 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560be3c88980 msgr2=0x560be3caab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:51.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.889+0000 7fa8997bf740 1 --2- 172.21.15.16:0/3478690907 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560be3c88980 0x560be3caab70 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fa880000c00 tx=0x7fa8800dc040 comp rx=0 tx=0).stop 2024-09-12T02:05:51.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.890+0000 7fa8997bf740 1 -- 172.21.15.16:0/3478690907 shutdown_connections 2024-09-12T02:05:51.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.890+0000 7fa8997bf740 1 --2- 172.21.15.16:0/3478690907 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560be3c93240 0x560be3cab0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:51.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.890+0000 7fa8997bf740 1 --2- 172.21.15.16:0/3478690907 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560be3c88980 0x560be3caab70 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:51.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.890+0000 7fa8997bf740 1 --2- 172.21.15.16:0/3478690907 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560be3c88060 0x560be3c83640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:51.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.890+0000 7fa8997bf740 1 -- 172.21.15.16:0/3478690907 >> 172.21.15.16:0/3478690907 conn(0x560be3c845f0 msgr2=0x560be3c82f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:51.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.890+0000 7fa8997bf740 1 -- 172.21.15.16:0/3478690907 shutdown_connections 2024-09-12T02:05:51.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.890+0000 7fa8997bf740 1 -- 172.21.15.16:0/3478690907 wait complete. 2024-09-12T02:05:51.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.890+0000 7fa8997bf740 1 -- start start 2024-09-12T02:05:51.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.890+0000 7fa8997bf740 1 Processor -- start 2024-09-12T02:05:51.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.890+0000 7fa8997bf740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560be3c88060 0x560be3c97840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:51.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.890+0000 7fa8997bf740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560be3c88980 0x560be3c97d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:51.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.890+0000 7fa8997bf740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560be3c93240 0x560be3d0f390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:51.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.890+0000 7fa8997bf740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa8901adc60 con 0x560be3c93240 2024-09-12T02:05:51.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.890+0000 7fa8997bf740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa8901af0d0 con 0x560be3c88980 2024-09-12T02:05:51.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.890+0000 7fa8997bf740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa8901ae330 con 0x560be3c88060 2024-09-12T02:05:51.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.891+0000 7fa8997bf740 10 client.0 _ll_register_callbacks cb 0x7fa88c00bb20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:05:51.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.891+0000 7fa895983640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560be3c93240 0x560be3d0f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:51.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.891+0000 7fa895983640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560be3c93240 0x560be3d0f390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54148/0 (socket says 172.21.15.16:54148) 2024-09-12T02:05:51.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.891+0000 7fa895983640 1 -- 172.21.15.16:0/620641729 learned_addr learned my addr 172.21.15.16:0/620641729 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:05:51.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.891+0000 7fa894981640 1 --2- 172.21.15.16:0/620641729 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560be3c88980 0x560be3c97d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:51.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.891+0000 7fa895182640 1 --2- 172.21.15.16:0/620641729 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560be3c88060 0x560be3c97840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:51.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.891+0000 7fa895983640 1 -- 172.21.15.16:0/620641729 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560be3c88060 msgr2=0x560be3c97840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:51.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.891+0000 7fa895983640 1 --2- 172.21.15.16:0/620641729 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560be3c88060 0x560be3c97840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:51.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.891+0000 7fa895983640 1 -- 172.21.15.16:0/620641729 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560be3c88980 msgr2=0x560be3c97d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:51.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.891+0000 7fa895983640 1 --2- 172.21.15.16:0/620641729 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560be3c88980 0x560be3c97d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:51.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.891+0000 7fa895983640 1 -- 172.21.15.16:0/620641729 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560be3d0f9d0 con 0x560be3c93240 2024-09-12T02:05:51.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.891+0000 7fa895182640 1 --2- 172.21.15.16:0/620641729 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560be3c88060 0x560be3c97840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:05:51.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.891+0000 7fa895983640 1 --2- 172.21.15.16:0/620641729 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560be3c93240 0x560be3d0f390 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fa888008fd0 tx=0x7fa888011fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f479ee3790290f73 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:51.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.891+0000 7fa878ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:05:51.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.891+0000 7fa878ff9640 1 -- 172.21.15.16:0/620641729 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa888033020 con 0x560be3c93240 2024-09-12T02:05:51.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.891+0000 7fa8997bf740 1 -- 172.21.15.16:0/620641729 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa8800bc5d0 con 0x560be3c93240 2024-09-12T02:05:51.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.891+0000 7fa878ff9640 1 -- 172.21.15.16:0/620641729 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fa888014250 con 0x560be3c93240 2024-09-12T02:05:51.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.892+0000 7fa878ff9640 1 -- 172.21.15.16:0/620641729 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa888021070 con 0x560be3c93240 2024-09-12T02:05:51.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.892+0000 7fa8997bf740 10 client.16617 fetch_fsmap learned FSMap version 25 2024-09-12T02:05:51.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.892+0000 7fa8997bf740 1 -- 172.21.15.16:0/620641729 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x560be3d16d80 con 0x560be3c93240 2024-09-12T02:05:51.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.892+0000 7fa878ff9640 1 -- 172.21.15.16:0/620641729 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8800bc5d0 con 0x560be3c93240 2024-09-12T02:05:51.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.892+0000 7fa878ff9640 1 -- 172.21.15.16:0/620641729 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fa888013760 con 0x560be3c93240 2024-09-12T02:05:51.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.892+0000 7fa8997bf740 10 client.16617 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:05:51.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.892+0000 7fa8997bf740 10 client.16617 resolve_mds: validated gid 24482 aka daemon mds.b 2024-09-12T02:05:51.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.892+0000 7fa8997bf740 20 client.16617 populate_metadata read hostname 'smithi016' 2024-09-12T02:05:51.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.892+0000 7fa8997bf740 1 --2- 172.21.15.16:0/620641729 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x560be3d17300 0x560be3d196f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:51.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.892+0000 7fa8997bf740 4 client.16617 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "448a5fd6", "release": true, "await": true}] 2024-09-12T02:05:51.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.892+0000 7fa8997bf740 1 -- 172.21.15.16:0/620641729 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "quiesce db", "set_id": "448a5fd6", "release": true, "await": true}) -- 0x560be3bf50a0 con 0x560be3d17300 2024-09-12T02:05:51.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.892+0000 7fa895182640 1 --2- 172.21.15.16:0/620641729 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x560be3d17300 0x560be3d196f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:51.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.893+0000 7fa878ff9640 10 client.16617 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:05:51.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.893+0000 7fa895182640 1 --2- 172.21.15.16:0/620641729 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x560be3d17300 0x560be3d196f0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7171200fae1cef96 server_cookie=bbd7afc4ab4d6bd1 in_seq=0 out_seq=0 2024-09-12T02:05:51.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.895+0000 7fa878ff9640 1 -- 172.21.15.16:0/620641729 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x560be3bf50a0 con 0x560be3d17300 2024-09-12T02:05:51.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.895+0000 7fa878ff9640 10 client.16617 handle_command_reply: tid=0 2024-09-12T02:05:51.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.895+0000 7fa8997bf740 1 client.16617 shutdown 2024-09-12T02:05:51.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.896+0000 7fa8997bf740 1 -- 172.21.15.16:0/620641729 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560be3c93240 msgr2=0x560be3d0f390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:51.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.896+0000 7fa8997bf740 1 --2- 172.21.15.16:0/620641729 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560be3c93240 0x560be3d0f390 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fa888008fd0 tx=0x7fa888011fc0 comp rx=0 tx=0).stop 2024-09-12T02:05:51.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.896+0000 7fa8997bf740 1 -- 172.21.15.16:0/620641729 shutdown_connections 2024-09-12T02:05:51.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.896+0000 7fa8997bf740 1 --2- 172.21.15.16:0/620641729 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x560be3d17300 0x560be3d196f0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:51.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.896+0000 7fa8997bf740 1 --2- 172.21.15.16:0/620641729 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560be3c93240 0x560be3d0f390 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:51.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.896+0000 7fa8997bf740 1 --2- 172.21.15.16:0/620641729 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560be3c88980 0x560be3c97d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:51.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.896+0000 7fa8997bf740 1 --2- 172.21.15.16:0/620641729 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560be3c88060 0x560be3c97840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:51.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.896+0000 7fa878ff9640 0 client.16617 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:05:51.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.896+0000 7fa8997bf740 1 -- 172.21.15.16:0/620641729 >> 172.21.15.16:0/620641729 conn(0x560be3c845f0 msgr2=0x560be3c8a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:51.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.896+0000 7fa8997bf740 1 -- 172.21.15.16:0/620641729 shutdown_connections 2024-09-12T02:05:51.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.896+0000 7fa8997bf740 1 -- 172.21.15.16:0/620641729 wait complete. 2024-09-12T02:05:51.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.896+0000 7fa8997bf740 20 client.16617 trim_cache size 0 max 16384 2024-09-12T02:05:51.902 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T02:05:51.902 INFO:teuthology.orchestra.run.smithi016.stdout: "epoch": 25, 2024-09-12T02:05:51.902 INFO:teuthology.orchestra.run.smithi016.stdout: "leader": 24482, 2024-09-12T02:05:51.902 INFO:teuthology.orchestra.run.smithi016.stdout: "set_version": 16, 2024-09-12T02:05:51.902 INFO:teuthology.orchestra.run.smithi016.stdout: "sets": { 2024-09-12T02:05:51.902 INFO:teuthology.orchestra.run.smithi016.stdout: "448a5fd6": { 2024-09-12T02:05:51.902 INFO:teuthology.orchestra.run.smithi016.stdout: "version": 16, 2024-09-12T02:05:51.902 INFO:teuthology.orchestra.run.smithi016.stdout: "age_ref": 0.0, 2024-09-12T02:05:51.902 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-12T02:05:51.902 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RELEASED", 2024-09-12T02:05:51.902 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-12T02:05:51.903 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:05:51.903 INFO:teuthology.orchestra.run.smithi016.stdout: "timeout": 90.0, 2024-09-12T02:05:51.903 INFO:teuthology.orchestra.run.smithi016.stdout: "expiration": 140.2, 2024-09-12T02:05:51.903 INFO:teuthology.orchestra.run.smithi016.stdout: "members": { 2024-09-12T02:05:51.903 INFO:teuthology.orchestra.run.smithi016.stdout: "file:/": { 2024-09-12T02:05:51.903 INFO:teuthology.orchestra.run.smithi016.stdout: "excluded": false, 2024-09-12T02:05:51.903 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-12T02:05:51.903 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RELEASED", 2024-09-12T02:05:51.903 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-12T02:05:51.903 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:05:51.903 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:05:51.903 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:05:51.903 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:05:51.903 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:05:51.904 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T02:05:51.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.897+0000 7fa8973e6640 1 -- 172.21.15.16:0/2440232921 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fa87c0a2c00 msgr2=0x7fa87c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:51.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.897+0000 7fa8973e6640 1 --2- 172.21.15.16:0/2440232921 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fa87c0a2c00 0x7fa87c0a50c0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fa88801e000 tx=0x7fa88801a000 comp rx=0 tx=0).stop 2024-09-12T02:05:51.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.897+0000 7fa8973e6640 1 -- 172.21.15.16:0/2440232921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa890102790 msgr2=0x7fa890074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:51.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.897+0000 7fa8973e6640 1 --2- 172.21.15.16:0/2440232921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa890102790 0x7fa890074a30 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fa880006010 tx=0x7fa8800042f0 comp rx=0 tx=0).stop 2024-09-12T02:05:51.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.897+0000 7fa8973e6640 1 -- 172.21.15.16:0/2440232921 shutdown_connections 2024-09-12T02:05:51.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.897+0000 7fa8973e6640 1 --2- 172.21.15.16:0/2440232921 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fa87c0a2c00 0x7fa87c0a50c0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:51.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.897+0000 7fa8973e6640 1 --2- 172.21.15.16:0/2440232921 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa890106f10 0x7fa890074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:51.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.897+0000 7fa8973e6640 1 --2- 172.21.15.16:0/2440232921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa890102790 0x7fa890074a30 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:51.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.897+0000 7fa8973e6640 1 --2- 172.21.15.16:0/2440232921 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa890101de0 0x7fa8900744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:51.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.897+0000 7fa8973e6640 1 -- 172.21.15.16:0/2440232921 >> 172.21.15.16:0/2440232921 conn(0x7fa8900fd8c0 msgr2=0x7fa8900fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:51.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.897+0000 7fa8973e6640 1 -- 172.21.15.16:0/2440232921 shutdown_connections 2024-09-12T02:05:51.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:51.897+0000 7fa8973e6640 1 -- 172.21.15.16:0/2440232921 wait complete. 2024-09-12T02:05:51.906 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:05:52.285 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:05:52 smithi016 ceph-mon[30947]: pgmap v453: 97 pgs: 97 active+clean; 2.1 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 3.2 KiB/s wr, 11 op/s 2024-09-12T02:05:52.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:05:52 smithi184 ceph-mon[35593]: pgmap v453: 97 pgs: 97 active+clean; 2.1 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 3.2 KiB/s wr, 11 op/s 2024-09-12T02:05:52.418 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:05:52 smithi033 ceph-mon[35332]: pgmap v453: 97 pgs: 97 active+clean; 2.1 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 3.2 KiB/s wr, 11 op/s 2024-09-12T02:05:52.800 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '448a5fd6', total seconds elapsed: 24.2 2024-09-12T02:05:52.800 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 20.15 2024-09-12T02:05:52.800 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:05:52.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:52.993+0000 7f6cdd820640 1 -- 172.21.15.16:0/1621459366 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6cd810c170 msgr2=0x7f6cd810e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:52.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:52.993+0000 7f6cdd820640 1 --2- 172.21.15.16:0/1621459366 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6cd810c170 0x7f6cd810e610 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f6cd0009ca0 tx=0x7f6cd002a950 comp rx=0 tx=0).stop 2024-09-12T02:05:52.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:52.993+0000 7f6cdd820640 1 -- 172.21.15.16:0/1621459366 shutdown_connections 2024-09-12T02:05:52.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:52.993+0000 7f6cdd820640 1 --2- 172.21.15.16:0/1621459366 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6cd810c170 0x7f6cd810e610 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:52.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:52.993+0000 7f6cdd820640 1 --2- 172.21.15.16:0/1621459366 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6cd8103660 0x7f6cd810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:52.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:52.993+0000 7f6cdd820640 1 --2- 172.21.15.16:0/1621459366 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6cd8102d40 0x7f6cd8103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:52.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:52.993+0000 7f6cdd820640 1 -- 172.21.15.16:0/1621459366 >> 172.21.15.16:0/1621459366 conn(0x7f6cd80fc7a0 msgr2=0x7f6cd80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:52.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:52.993+0000 7f6cdd820640 1 -- 172.21.15.16:0/1621459366 shutdown_connections 2024-09-12T02:05:52.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:52.993+0000 7f6cdd820640 1 -- 172.21.15.16:0/1621459366 wait complete. 2024-09-12T02:05:52.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:52.993+0000 7f6cdd820640 1 Processor -- start 2024-09-12T02:05:52.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:52.994+0000 7f6cdd820640 1 -- start start 2024-09-12T02:05:52.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:52.994+0000 7f6cdd820640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6cd8102d40 0x7f6cd8198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:52.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:52.994+0000 7f6cdd820640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6cd8103660 0x7f6cd81989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:52.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:52.994+0000 7f6cdd820640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6cd810c170 0x7f6cd819cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:52.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:52.994+0000 7f6cdd820640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6cd8111f70 con 0x7f6cd8103660 2024-09-12T02:05:52.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:52.994+0000 7f6cdd820640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6cd8111df0 con 0x7f6cd8102d40 2024-09-12T02:05:52.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:52.994+0000 7f6cdd820640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f6cd81120f0 con 0x7f6cd810c170 2024-09-12T02:05:52.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:52.994+0000 7f6cd67fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6cd8103660 0x7f6cd81989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:52.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:52.994+0000 7f6cd67fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6cd8103660 0x7f6cd81989b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54160/0 (socket says 172.21.15.16:54160) 2024-09-12T02:05:52.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:52.994+0000 7f6cd67fc640 1 -- 172.21.15.16:0/2869191051 learned_addr learned my addr 172.21.15.16:0/2869191051 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:05:52.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:52.994+0000 7f6cd6ffd640 1 --2- 172.21.15.16:0/2869191051 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6cd8102d40 0x7f6cd8198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:52.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:52.994+0000 7f6cd77fe640 1 --2- 172.21.15.16:0/2869191051 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6cd810c170 0x7f6cd819cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:52.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:52.995+0000 7f6cd67fc640 1 -- 172.21.15.16:0/2869191051 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6cd810c170 msgr2=0x7f6cd819cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:52.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:52.995+0000 7f6cd67fc640 1 --2- 172.21.15.16:0/2869191051 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6cd810c170 0x7f6cd819cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:52.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:52.995+0000 7f6cd67fc640 1 -- 172.21.15.16:0/2869191051 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6cd8102d40 msgr2=0x7f6cd8198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:52.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:52.995+0000 7f6cd67fc640 1 --2- 172.21.15.16:0/2869191051 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6cd8102d40 0x7f6cd8198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:52.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:52.995+0000 7f6cd67fc640 1 -- 172.21.15.16:0/2869191051 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cd819d420 con 0x7f6cd8103660 2024-09-12T02:05:52.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:52.995+0000 7f6cd67fc640 1 --2- 172.21.15.16:0/2869191051 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6cd8103660 0x7f6cd81989b0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f6cc000d740 tx=0x7f6cc000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:52.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:52.995+0000 7f6cdc81e640 1 -- 172.21.15.16:0/2869191051 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cc0017070 con 0x7f6cd8103660 2024-09-12T02:05:52.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:52.995+0000 7f6cdd820640 1 -- 172.21.15.16:0/2869191051 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cd819d710 con 0x7f6cd8103660 2024-09-12T02:05:52.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:52.995+0000 7f6cdc81e640 1 -- 172.21.15.16:0/2869191051 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6cc000de50 con 0x7f6cd8103660 2024-09-12T02:05:52.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:52.995+0000 7f6cdc81e640 1 -- 172.21.15.16:0/2869191051 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cc000f620 con 0x7f6cd8103660 2024-09-12T02:05:52.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:52.995+0000 7f6cdd820640 1 -- 172.21.15.16:0/2869191051 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cd81008d0 con 0x7f6cd8103660 2024-09-12T02:05:52.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:52.998+0000 7f6cdc81e640 1 -- 172.21.15.16:0/2869191051 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6cc0009c30 con 0x7f6cd8103660 2024-09-12T02:05:52.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:52.998+0000 7f6cdc81e640 1 --2- 172.21.15.16:0/2869191051 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f6cc40a2b80 0x7f6cc40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:52.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:52.999+0000 7f6cdc81e640 1 -- 172.21.15.16:0/2869191051 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f6cc00cea80 con 0x7f6cd8103660 2024-09-12T02:05:52.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:52.999+0000 7f6cdfbf9740 1 Processor -- start 2024-09-12T02:05:52.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:52.999+0000 7f6cdfbf9740 1 -- start start 2024-09-12T02:05:52.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:52.999+0000 7f6cd6ffd640 1 --2- 172.21.15.16:0/2869191051 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f6cc40a2b80 0x7f6cc40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:53.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:52.999+0000 7f6cdfbf9740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5617385b1b40 0x5617385b0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:53.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:52.999+0000 7f6cdfbf9740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5617385b05d0 0x5617385b0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:53.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:52.999+0000 7f6cdfbf9740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5617385bb370 0x5617385bd760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:53.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:52.999+0000 7f6cdfbf9740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5617385162f0 con 0x5617385b1b40 2024-09-12T02:05:53.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:52.999+0000 7f6cdfbf9740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5617384bdcf0 con 0x5617385b05d0 2024-09-12T02:05:53.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:52.999+0000 7f6cdfbf9740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5617385122b0 con 0x5617385bb370 2024-09-12T02:05:53.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:52.999+0000 7f6cd6ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5617385b1b40 0x5617385b0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:53.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:52.999+0000 7f6cd6ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5617385b1b40 0x5617385b0090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54174/0 (socket says 172.21.15.16:54174) 2024-09-12T02:05:53.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:52.999+0000 7f6cd6ffd640 1 -- 172.21.15.16:0/110530591 learned_addr learned my addr 172.21.15.16:0/110530591 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:05:53.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.000+0000 7f6cd67fc640 1 --2- 172.21.15.16:0/110530591 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5617385b05d0 0x5617385b0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:53.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.000+0000 7f6cd77fe640 1 --2- 172.21.15.16:0/110530591 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5617385bb370 0x5617385bd760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:53.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.000+0000 7f6cd6ffd640 1 --2- 172.21.15.16:0/2869191051 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f6cc40a2b80 0x7f6cc40a5040 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f6cc8005fd0 tx=0x7f6cc800bab0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:53.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.000+0000 7f6cd6ffd640 1 -- 172.21.15.16:0/110530591 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5617385bb370 msgr2=0x5617385bd760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:53.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.000+0000 7f6cd6ffd640 1 --2- 172.21.15.16:0/110530591 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5617385bb370 0x5617385bd760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:53.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.000+0000 7f6cd6ffd640 1 -- 172.21.15.16:0/110530591 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5617385b05d0 msgr2=0x5617385b0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:53.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.000+0000 7f6cd6ffd640 1 --2- 172.21.15.16:0/110530591 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5617385b05d0 0x5617385b0a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:53.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.000+0000 7f6cd6ffd640 1 -- 172.21.15.16:0/110530591 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5617384a23a0 con 0x5617385b1b40 2024-09-12T02:05:53.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.000+0000 7f6cd6ffd640 1 --2- 172.21.15.16:0/110530591 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5617385b1b40 0x5617385b0090 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f6cc8003d00 tx=0x7f6cc801b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=399278dc183041f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:53.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.000+0000 7f6ccdffb640 1 -- 172.21.15.16:0/110530591 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cc80130d0 con 0x5617385b1b40 2024-09-12T02:05:53.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.000+0000 7f6ccdffb640 1 -- 172.21.15.16:0/110530591 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6cc8013270 con 0x5617385b1b40 2024-09-12T02:05:53.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.000+0000 7f6ccdffb640 1 -- 172.21.15.16:0/110530591 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cc8011440 con 0x5617385b1b40 2024-09-12T02:05:53.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.001+0000 7f6cdfbf9740 1 -- 172.21.15.16:0/110530591 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5617385b1b40 msgr2=0x5617385b0090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:53.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.001+0000 7f6cdfbf9740 1 --2- 172.21.15.16:0/110530591 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5617385b1b40 0x5617385b0090 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f6cc8003d00 tx=0x7f6cc801b000 comp rx=0 tx=0).stop 2024-09-12T02:05:53.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.001+0000 7f6cdfbf9740 1 -- 172.21.15.16:0/110530591 shutdown_connections 2024-09-12T02:05:53.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.001+0000 7f6cdfbf9740 1 --2- 172.21.15.16:0/110530591 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5617385bb370 0x5617385bd760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:53.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.001+0000 7f6cdfbf9740 1 --2- 172.21.15.16:0/110530591 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5617385b05d0 0x5617385b0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:53.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.001+0000 7f6cdfbf9740 1 --2- 172.21.15.16:0/110530591 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5617385b1b40 0x5617385b0090 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:53.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.001+0000 7f6cdfbf9740 1 -- 172.21.15.16:0/110530591 >> 172.21.15.16:0/110530591 conn(0x561738579310 msgr2=0x5617385acc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:53.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.001+0000 7f6cdfbf9740 1 -- 172.21.15.16:0/110530591 shutdown_connections 2024-09-12T02:05:53.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.001+0000 7f6cdfbf9740 1 -- 172.21.15.16:0/110530591 wait complete. 2024-09-12T02:05:53.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.002+0000 7f6cdfbf9740 1 -- start start 2024-09-12T02:05:53.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.002+0000 7f6cdfbf9740 1 Processor -- start 2024-09-12T02:05:53.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.002+0000 7f6cdfbf9740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5617385b05d0 0x5617385bac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:53.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.002+0000 7f6cdfbf9740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5617385b1b40 0x5617385b7ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:53.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.002+0000 7f6cdfbf9740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5617385bb370 0x5617385b8420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:53.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.002+0000 7f6cdfbf9740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5617385be070 con 0x5617385b1b40 2024-09-12T02:05:53.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.002+0000 7f6cdfbf9740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5617384d1060 con 0x5617385b05d0 2024-09-12T02:05:53.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.002+0000 7f6cdfbf9740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5617385be1f0 con 0x5617385bb370 2024-09-12T02:05:53.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.002+0000 7f6cd67fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5617385b1b40 0x5617385b7ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:53.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.002+0000 7f6cdfbf9740 10 client.0 _ll_register_callbacks cb 0x5617382248c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:05:53.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.002+0000 7f6cd67fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5617385b1b40 0x5617385b7ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54176/0 (socket says 172.21.15.16:54176) 2024-09-12T02:05:53.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.002+0000 7f6cd67fc640 1 -- 172.21.15.16:0/2151978876 learned_addr learned my addr 172.21.15.16:0/2151978876 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:05:53.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.002+0000 7f6cd6ffd640 1 --2- 172.21.15.16:0/2151978876 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5617385b05d0 0x5617385bac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:53.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.002+0000 7f6cd77fe640 1 --2- 172.21.15.16:0/2151978876 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5617385bb370 0x5617385b8420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:53.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.003+0000 7f6cd6ffd640 1 -- 172.21.15.16:0/2151978876 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5617385bb370 msgr2=0x5617385b8420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:53.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.003+0000 7f6cd6ffd640 1 --2- 172.21.15.16:0/2151978876 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5617385bb370 0x5617385b8420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:53.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.003+0000 7f6cd6ffd640 1 -- 172.21.15.16:0/2151978876 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5617385b1b40 msgr2=0x5617385b7ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:53.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.003+0000 7f6cd6ffd640 1 --2- 172.21.15.16:0/2151978876 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5617385b1b40 0x5617385b7ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:53.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.003+0000 7f6cd6ffd640 1 -- 172.21.15.16:0/2151978876 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5617382873c0 con 0x5617385b05d0 2024-09-12T02:05:53.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.003+0000 7f6cd67fc640 1 --2- 172.21.15.16:0/2151978876 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5617385b1b40 0x5617385b7ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:05:53.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.003+0000 7f6cd77fe640 1 --2- 172.21.15.16:0/2151978876 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5617385bb370 0x5617385b8420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:05:53.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.003+0000 7f6cd6ffd640 1 --2- 172.21.15.16:0/2151978876 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5617385b05d0 0x5617385bac30 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6cc8013cc0 tx=0x7f6cc80037b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8323be5932bee2c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:53.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.003+0000 7f6cb6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-12T02:05:53.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.003+0000 7f6cb6ffd640 1 -- 172.21.15.16:0/2151978876 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cc80130d0 con 0x5617385b05d0 2024-09-12T02:05:53.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.003+0000 7f6cdfbf9740 1 -- 172.21.15.16:0/2151978876 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6cc8013270 con 0x5617385b05d0 2024-09-12T02:05:53.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.003+0000 7f6cb6ffd640 1 -- 172.21.15.16:0/2151978876 <== mon.2 v2:172.21.15.33:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f6cc8005080 con 0x5617385b05d0 2024-09-12T02:05:53.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.003+0000 7f6cb6ffd640 1 -- 172.21.15.16:0/2151978876 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6cc80046c0 con 0x5617385b05d0 2024-09-12T02:05:53.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.004+0000 7f6cdfbf9740 10 client.26141 fetch_fsmap learned FSMap version 25 2024-09-12T02:05:53.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.004+0000 7f6cdfbf9740 1 -- 172.21.15.16:0/2151978876 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5617385c9a00 con 0x5617385b05d0 2024-09-12T02:05:53.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.004+0000 7f6cb6ffd640 1 -- 172.21.15.16:0/2151978876 <== mon.2 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6cc8013270 con 0x5617385b05d0 2024-09-12T02:05:53.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.004+0000 7f6cb6ffd640 1 -- 172.21.15.16:0/2151978876 <== mon.2 v2:172.21.15.33:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f6cc8011be0 con 0x5617385b05d0 2024-09-12T02:05:53.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.004+0000 7f6cdfbf9740 10 client.26141 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:05:53.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.004+0000 7f6cdfbf9740 10 client.26141 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T02:05:53.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.004+0000 7f6cdfbf9740 20 client.26141 populate_metadata read hostname 'smithi016' 2024-09-12T02:05:53.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.004+0000 7f6cdfbf9740 1 --2- 172.21.15.16:0/2151978876 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x5617385cadb0 0x5617385cd1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:53.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.004+0000 7f6cdfbf9740 4 client.26141 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:05:53.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.004+0000 7f6cdfbf9740 1 -- 172.21.15.16:0/2151978876 --> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561738523860 con 0x5617385cadb0 2024-09-12T02:05:53.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.004+0000 7f6cd67fc640 1 --2- 172.21.15.16:0/2151978876 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x5617385cadb0 0x5617385cd1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:53.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.005+0000 7f6cd67fc640 1 --2- 172.21.15.16:0/2151978876 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x5617385cadb0 0x5617385cd1a0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a95924ff11b5e99c server_cookie=be8759f1562964ef in_seq=0 out_seq=0 2024-09-12T02:05:53.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.005+0000 7f6cb6ffd640 10 client.26141 ms_handle_connect on v2:172.21.15.184:6838/4203347941 2024-09-12T02:05:53.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.005+0000 7f6cb6ffd640 1 -- 172.21.15.16:0/2151978876 <== mds.1 v2:172.21.15.184:6838/4203347941 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561738523860 con 0x5617385cadb0 2024-09-12T02:05:53.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.005+0000 7f6cb6ffd640 10 client.26141 handle_command_reply: tid=0 2024-09-12T02:05:53.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.006+0000 7f6cdfbf9740 1 client.26141 shutdown 2024-09-12T02:05:53.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.006+0000 7f6cdfbf9740 1 -- 172.21.15.16:0/2151978876 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5617385b05d0 msgr2=0x5617385bac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:53.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.006+0000 7f6cdfbf9740 1 --2- 172.21.15.16:0/2151978876 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5617385b05d0 0x5617385bac30 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6cc8013cc0 tx=0x7f6cc80037b0 comp rx=0 tx=0).stop 2024-09-12T02:05:53.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.006+0000 7f6cdfbf9740 1 -- 172.21.15.16:0/2151978876 shutdown_connections 2024-09-12T02:05:53.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.006+0000 7f6cdfbf9740 1 --2- 172.21.15.16:0/2151978876 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x5617385cadb0 0x5617385cd1a0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:53.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.006+0000 7f6cdfbf9740 1 --2- 172.21.15.16:0/2151978876 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5617385bb370 0x5617385b8420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:53.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.006+0000 7f6cdfbf9740 1 --2- 172.21.15.16:0/2151978876 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5617385b1b40 0x5617385b7ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:53.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.006+0000 7f6cdfbf9740 1 --2- 172.21.15.16:0/2151978876 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5617385b05d0 0x5617385bac30 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:53.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.006+0000 7f6cb6ffd640 0 client.26141 ms_handle_reset on v2:172.21.15.184:6838/4203347941 2024-09-12T02:05:53.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.006+0000 7f6cdfbf9740 1 -- 172.21.15.16:0/2151978876 >> 172.21.15.16:0/2151978876 conn(0x561738579310 msgr2=0x5617385b2960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:53.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.006+0000 7f6cdfbf9740 1 -- 172.21.15.16:0/2151978876 shutdown_connections 2024-09-12T02:05:53.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.006+0000 7f6cdfbf9740 1 -- 172.21.15.16:0/2151978876 wait complete. 2024-09-12T02:05:53.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.006+0000 7f6cdfbf9740 20 client.26141 trim_cache size 0 max 16384 2024-09-12T02:05:53.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.020+0000 7f6cdfbf9740 1 Processor -- start 2024-09-12T02:05:53.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.020+0000 7f6cdfbf9740 1 -- start start 2024-09-12T02:05:53.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.020+0000 7f6cdfbf9740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5617385b05d0 0x5617385c7310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:53.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.020+0000 7f6cdfbf9740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5617385b1b40 0x5617385d27a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:53.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.020+0000 7f6cdfbf9740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5617385bb370 0x5617385d2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:53.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.021+0000 7f6cdfbf9740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6cd81b1f20 con 0x5617385bb370 2024-09-12T02:05:53.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.021+0000 7f6cdfbf9740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6cd81af7f0 con 0x5617385b1b40 2024-09-12T02:05:53.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.021+0000 7f6cdfbf9740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f6cd81ad660 con 0x5617385b05d0 2024-09-12T02:05:53.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.021+0000 7f6cd77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5617385bb370 0x5617385d2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:53.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.021+0000 7f6cd77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5617385bb370 0x5617385d2ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54180/0 (socket says 172.21.15.16:54180) 2024-09-12T02:05:53.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.021+0000 7f6cd77fe640 1 -- 172.21.15.16:0/908930026 learned_addr learned my addr 172.21.15.16:0/908930026 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:05:53.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.021+0000 7f6cd67fc640 1 --2- 172.21.15.16:0/908930026 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5617385b1b40 0x5617385d27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:53.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.021+0000 7f6cd6ffd640 1 --2- 172.21.15.16:0/908930026 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5617385b05d0 0x5617385c7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:53.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.021+0000 7f6cd67fc640 1 -- 172.21.15.16:0/908930026 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5617385b05d0 msgr2=0x5617385c7310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:53.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.021+0000 7f6cd67fc640 1 --2- 172.21.15.16:0/908930026 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5617385b05d0 0x5617385c7310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:53.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.021+0000 7f6cd67fc640 1 -- 172.21.15.16:0/908930026 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5617385bb370 msgr2=0x5617385d2ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:53.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.021+0000 7f6cd67fc640 1 --2- 172.21.15.16:0/908930026 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5617385bb370 0x5617385d2ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:53.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.021+0000 7f6cd67fc640 1 -- 172.21.15.16:0/908930026 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5617385ad1a0 con 0x5617385b1b40 2024-09-12T02:05:53.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.022+0000 7f6cd67fc640 1 --2- 172.21.15.16:0/908930026 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5617385b1b40 0x5617385d27a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6cc0009610 tx=0x7f6cc0097800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f965bf85a86ec0b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:53.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.022+0000 7f6ccd7fa640 1 -- 172.21.15.16:0/908930026 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cc0020040 con 0x5617385b1b40 2024-09-12T02:05:53.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.022+0000 7f6ccd7fa640 1 -- 172.21.15.16:0/908930026 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6cc0099730 con 0x5617385b1b40 2024-09-12T02:05:53.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.022+0000 7f6ccd7fa640 1 -- 172.21.15.16:0/908930026 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cc009a550 con 0x5617385b1b40 2024-09-12T02:05:53.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.023+0000 7f6cdfbf9740 1 -- 172.21.15.16:0/908930026 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5617385b1b40 msgr2=0x5617385d27a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:53.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.023+0000 7f6cdfbf9740 1 --2- 172.21.15.16:0/908930026 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5617385b1b40 0x5617385d27a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6cc0009610 tx=0x7f6cc0097800 comp rx=0 tx=0).stop 2024-09-12T02:05:53.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.023+0000 7f6cdfbf9740 1 -- 172.21.15.16:0/908930026 shutdown_connections 2024-09-12T02:05:53.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.023+0000 7f6cdfbf9740 1 --2- 172.21.15.16:0/908930026 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5617385bb370 0x5617385d2ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:53.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.023+0000 7f6cdfbf9740 1 --2- 172.21.15.16:0/908930026 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5617385b1b40 0x5617385d27a0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:53.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.023+0000 7f6cdfbf9740 1 --2- 172.21.15.16:0/908930026 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5617385b05d0 0x5617385c7310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:53.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.023+0000 7f6cdfbf9740 1 -- 172.21.15.16:0/908930026 >> 172.21.15.16:0/908930026 conn(0x561738579310 msgr2=0x5617385abd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:53.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.023+0000 7f6cdfbf9740 1 -- 172.21.15.16:0/908930026 shutdown_connections 2024-09-12T02:05:53.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.023+0000 7f6cdfbf9740 1 -- 172.21.15.16:0/908930026 wait complete. 2024-09-12T02:05:53.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.023+0000 7f6cdfbf9740 1 -- start start 2024-09-12T02:05:53.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.024+0000 7f6cdfbf9740 1 Processor -- start 2024-09-12T02:05:53.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.024+0000 7f6cdfbf9740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5617385b05d0 0x56173862da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:53.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.024+0000 7f6cdfbf9740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5617385b1b40 0x56173862dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:53.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.024+0000 7f6cdfbf9740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5617385bb370 0x5617385bea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:53.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.024+0000 7f6cdfbf9740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6cd81adcb0 con 0x5617385b1b40 2024-09-12T02:05:53.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.024+0000 7f6cdfbf9740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6cd81af120 con 0x5617385bb370 2024-09-12T02:05:53.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.024+0000 7f6cdfbf9740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f6cd81ae380 con 0x5617385b05d0 2024-09-12T02:05:53.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.024+0000 7f6cdfbf9740 10 client.0 _ll_register_callbacks cb 0x7f6cc8004380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:05:53.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.024+0000 7f6cd67fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5617385b1b40 0x56173862dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:53.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.024+0000 7f6cd67fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5617385b1b40 0x56173862dfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54186/0 (socket says 172.21.15.16:54186) 2024-09-12T02:05:53.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.024+0000 7f6cd6ffd640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5617385b05d0 0x56173862da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:53.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.024+0000 7f6cd67fc640 1 -- 172.21.15.16:0/2943004619 learned_addr learned my addr 172.21.15.16:0/2943004619 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:05:53.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.024+0000 7f6cd77fe640 1 --2- 172.21.15.16:0/2943004619 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5617385bb370 0x5617385bea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:53.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.025+0000 7f6cd6ffd640 1 -- 172.21.15.16:0/2943004619 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5617385bb370 msgr2=0x5617385bea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:53.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.025+0000 7f6cd6ffd640 1 --2- 172.21.15.16:0/2943004619 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5617385bb370 0x5617385bea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:53.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.025+0000 7f6cd6ffd640 1 -- 172.21.15.16:0/2943004619 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5617385b1b40 msgr2=0x56173862dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:53.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.025+0000 7f6cd6ffd640 1 --2- 172.21.15.16:0/2943004619 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5617385b1b40 0x56173862dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:53.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.025+0000 7f6cd6ffd640 1 -- 172.21.15.16:0/2943004619 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5617385ad960 con 0x5617385b05d0 2024-09-12T02:05:53.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.025+0000 7f6cd6ffd640 1 --2- 172.21.15.16:0/2943004619 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5617385b05d0 0x56173862da60 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6cc8004690 tx=0x7f6cc8005c50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=403077e15b5c127a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:53.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.025+0000 7f6cb6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-09-12T02:05:53.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.025+0000 7f6cb6ffd640 1 -- 172.21.15.16:0/2943004619 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cc8004e80 con 0x5617385b05d0 2024-09-12T02:05:53.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.025+0000 7f6cb6ffd640 1 -- 172.21.15.16:0/2943004619 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f6cc8005020 con 0x5617385b05d0 2024-09-12T02:05:53.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.025+0000 7f6cdfbf9740 1 -- 172.21.15.16:0/2943004619 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6cc0099730 con 0x5617385b05d0 2024-09-12T02:05:53.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.026+0000 7f6cb6ffd640 1 -- 172.21.15.16:0/2943004619 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6cc804a050 con 0x5617385b05d0 2024-09-12T02:05:53.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.026+0000 7f6cdfbf9740 10 client.26029 fetch_fsmap learned FSMap version 25 2024-09-12T02:05:53.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.026+0000 7f6cdfbf9740 1 -- 172.21.15.16:0/2943004619 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x561738634fc0 con 0x5617385b05d0 2024-09-12T02:05:53.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.026+0000 7f6cb6ffd640 1 -- 172.21.15.16:0/2943004619 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6cc0099730 con 0x5617385b05d0 2024-09-12T02:05:53.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.026+0000 7f6cb6ffd640 1 -- 172.21.15.16:0/2943004619 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f6cc801ec10 con 0x5617385b05d0 2024-09-12T02:05:53.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.026+0000 7f6cdfbf9740 10 client.26029 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:05:53.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.026+0000 7f6cdfbf9740 10 client.26029 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T02:05:53.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.026+0000 7f6cdfbf9740 20 client.26029 populate_metadata read hostname 'smithi016' 2024-09-12T02:05:53.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.026+0000 7f6cdfbf9740 1 --2- 172.21.15.16:0/2943004619 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x5617386356e0 0x561738637b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:53.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.026+0000 7f6cdfbf9740 4 client.26029 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T02:05:53.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.026+0000 7f6cdfbf9740 1 -- 172.21.15.16:0/2943004619 --> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56173851dd70 con 0x5617386356e0 2024-09-12T02:05:53.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.026+0000 7f6cd67fc640 1 --2- 172.21.15.16:0/2943004619 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x5617386356e0 0x561738637b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:53.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.027+0000 7f6cd67fc640 1 --2- 172.21.15.16:0/2943004619 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x5617386356e0 0x561738637b00 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b27b0ce98f2ffc43 server_cookie=dfa19d57583eb7ac in_seq=0 out_seq=0 2024-09-12T02:05:53.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.027+0000 7f6cb6ffd640 10 client.26029 ms_handle_connect on v2:172.21.15.184:6838/4203347941 2024-09-12T02:05:53.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.027+0000 7f6cb6ffd640 1 -- 172.21.15.16:0/2943004619 <== mds.1 v2:172.21.15.184:6838/4203347941 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56173851dd70 con 0x5617386356e0 2024-09-12T02:05:53.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.027+0000 7f6cb6ffd640 10 client.26029 handle_command_reply: tid=0 2024-09-12T02:05:53.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.027+0000 7f6cdfbf9740 1 client.26029 shutdown 2024-09-12T02:05:53.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.027+0000 7f6cdfbf9740 1 -- 172.21.15.16:0/2943004619 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5617385b05d0 msgr2=0x56173862da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:53.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.027+0000 7f6cdfbf9740 1 --2- 172.21.15.16:0/2943004619 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5617385b05d0 0x56173862da60 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6cc8004690 tx=0x7f6cc8005c50 comp rx=0 tx=0).stop 2024-09-12T02:05:53.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.027+0000 7f6cdfbf9740 1 -- 172.21.15.16:0/2943004619 shutdown_connections 2024-09-12T02:05:53.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.027+0000 7f6cdfbf9740 1 --2- 172.21.15.16:0/2943004619 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x5617386356e0 0x561738637b00 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:53.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.027+0000 7f6cdfbf9740 1 --2- 172.21.15.16:0/2943004619 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5617385bb370 0x5617385bea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:53.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.027+0000 7f6cdfbf9740 1 --2- 172.21.15.16:0/2943004619 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5617385b1b40 0x56173862dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:53.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.027+0000 7f6cb6ffd640 0 client.26029 ms_handle_reset on v2:172.21.15.184:6838/4203347941 2024-09-12T02:05:53.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.027+0000 7f6cdfbf9740 1 --2- 172.21.15.16:0/2943004619 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5617385b05d0 0x56173862da60 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:53.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.027+0000 7f6cdfbf9740 1 -- 172.21.15.16:0/2943004619 >> 172.21.15.16:0/2943004619 conn(0x561738579310 msgr2=0x5617385b2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:53.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.028+0000 7f6cdfbf9740 1 -- 172.21.15.16:0/2943004619 shutdown_connections 2024-09-12T02:05:53.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.028+0000 7f6cdfbf9740 1 -- 172.21.15.16:0/2943004619 wait complete. 2024-09-12T02:05:53.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.028+0000 7f6cdfbf9740 20 client.26029 trim_cache size 0 max 16384 2024-09-12T02:05:53.036 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T02:05:53.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.029+0000 7f6cdd820640 1 -- 172.21.15.16:0/2869191051 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f6cc40a2b80 msgr2=0x7f6cc40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:53.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.029+0000 7f6cdd820640 1 --2- 172.21.15.16:0/2869191051 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f6cc40a2b80 0x7f6cc40a5040 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f6cc8005fd0 tx=0x7f6cc800bab0 comp rx=0 tx=0).stop 2024-09-12T02:05:53.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.029+0000 7f6cdd820640 1 -- 172.21.15.16:0/2869191051 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6cd8103660 msgr2=0x7f6cd81989b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:53.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.029+0000 7f6cdd820640 1 --2- 172.21.15.16:0/2869191051 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6cd8103660 0x7f6cd81989b0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f6cc000d740 tx=0x7f6cc000dc00 comp rx=0 tx=0).stop 2024-09-12T02:05:53.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.029+0000 7f6cdd820640 1 -- 172.21.15.16:0/2869191051 shutdown_connections 2024-09-12T02:05:53.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.029+0000 7f6cdd820640 1 --2- 172.21.15.16:0/2869191051 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f6cc40a2b80 0x7f6cc40a5040 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:53.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.029+0000 7f6cdd820640 1 --2- 172.21.15.16:0/2869191051 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6cd810c170 0x7f6cd819cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:53.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.029+0000 7f6cdd820640 1 --2- 172.21.15.16:0/2869191051 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6cd8103660 0x7f6cd81989b0 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:53.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.029+0000 7f6cdd820640 1 --2- 172.21.15.16:0/2869191051 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6cd8102d40 0x7f6cd8198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:53.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.029+0000 7f6cdd820640 1 -- 172.21.15.16:0/2869191051 >> 172.21.15.16:0/2869191051 conn(0x7f6cd80fc7a0 msgr2=0x7f6cd80fde10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:53.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.030+0000 7f6cdd820640 1 -- 172.21.15.16:0/2869191051 shutdown_connections 2024-09-12T02:05:53.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:53.030+0000 7f6cdd820640 1 -- 172.21.15.16:0/2869191051 wait complete. 2024-09-12T02:05:53.783 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:05:53.818 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-12T02:05:54.235 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.235+0000 7f98330f1640 1 -- 172.21.15.16:0/2983827630 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f982c101b60 msgr2=0x7f982c101f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:54.235 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.235+0000 7f98330f1640 1 --2- 172.21.15.16:0/2983827630 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f982c101b60 0x7f982c101f60 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f9828009900 tx=0x7f982802a410 comp rx=0 tx=0).stop 2024-09-12T02:05:54.235 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.235+0000 7f98330f1640 1 -- 172.21.15.16:0/2983827630 shutdown_connections 2024-09-12T02:05:54.235 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.235+0000 7f98330f1640 1 --2- 172.21.15.16:0/2983827630 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f982c103710 0x7f982c109fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:54.235 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.235+0000 7f98330f1640 1 --2- 172.21.15.16:0/2983827630 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f982c102d50 0x7f982c1031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:54.235 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.235+0000 7f98330f1640 1 --2- 172.21.15.16:0/2983827630 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f982c101b60 0x7f982c101f60 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:54.235 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.235+0000 7f98330f1640 1 -- 172.21.15.16:0/2983827630 >> 172.21.15.16:0/2983827630 conn(0x7f982c0fd2e0 msgr2=0x7f982c0ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:54.235 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.235+0000 7f98330f1640 1 -- 172.21.15.16:0/2983827630 shutdown_connections 2024-09-12T02:05:54.236 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.235+0000 7f98330f1640 1 -- 172.21.15.16:0/2983827630 wait complete. 2024-09-12T02:05:54.237 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.236+0000 7f98330f1640 1 Processor -- start 2024-09-12T02:05:54.238 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.236+0000 7f98330f1640 1 -- start start 2024-09-12T02:05:54.238 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.236+0000 7f98330f1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f982c101b60 0x7f982c197e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:54.238 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.236+0000 7f98330f1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f982c102d50 0x7f982c198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:54.238 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.236+0000 7f98330f1640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f982c103710 0x7f982c19f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:54.238 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.236+0000 7f98330f1640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f982c10d7e0 con 0x7f982c102d50 2024-09-12T02:05:54.238 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.236+0000 7f98330f1640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f982c10d660 con 0x7f982c101b60 2024-09-12T02:05:54.238 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.236+0000 7f98330f1640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f982c10d960 con 0x7f982c103710 2024-09-12T02:05:54.238 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.236+0000 7f9823fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f982c102d50 0x7f982c198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:54.238 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.236+0000 7f9823fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f982c102d50 0x7f982c198340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54202/0 (socket says 172.21.15.16:54202) 2024-09-12T02:05:54.238 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.236+0000 7f9823fff640 1 -- 172.21.15.16:0/3760025924 learned_addr learned my addr 172.21.15.16:0/3760025924 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:05:54.238 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.236+0000 7f9830e8d640 1 --2- 172.21.15.16:0/3760025924 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f982c101b60 0x7f982c197e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:54.238 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.236+0000 7f983168e640 1 --2- 172.21.15.16:0/3760025924 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f982c103710 0x7f982c19f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:54.238 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.236+0000 7f9823fff640 1 -- 172.21.15.16:0/3760025924 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f982c103710 msgr2=0x7f982c19f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:54.239 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.236+0000 7f9823fff640 1 --2- 172.21.15.16:0/3760025924 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f982c103710 0x7f982c19f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:54.239 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.236+0000 7f9823fff640 1 -- 172.21.15.16:0/3760025924 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f982c101b60 msgr2=0x7f982c197e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:54.239 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.236+0000 7f9823fff640 1 --2- 172.21.15.16:0/3760025924 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f982c101b60 0x7f982c197e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:54.239 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.236+0000 7f9823fff640 1 -- 172.21.15.16:0/3760025924 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f982c19fa20 con 0x7f982c102d50 2024-09-12T02:05:54.239 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.237+0000 7f9823fff640 1 --2- 172.21.15.16:0/3760025924 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f982c102d50 0x7f982c198340 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f981c00d950 tx=0x7f981c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:54.239 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.237+0000 7f9821ffb640 1 -- 172.21.15.16:0/3760025924 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f981c017070 con 0x7f982c102d50 2024-09-12T02:05:54.239 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.237+0000 7f9821ffb640 1 -- 172.21.15.16:0/3760025924 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f981c009c30 con 0x7f982c102d50 2024-09-12T02:05:54.239 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.237+0000 7f9821ffb640 1 -- 172.21.15.16:0/3760025924 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f981c00f6b0 con 0x7f982c102d50 2024-09-12T02:05:54.239 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.237+0000 7f98330f1640 1 -- 172.21.15.16:0/3760025924 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f982c19fd10 con 0x7f982c102d50 2024-09-12T02:05:54.239 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.237+0000 7f98330f1640 1 -- 172.21.15.16:0/3760025924 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f982c1a0250 con 0x7f982c102d50 2024-09-12T02:05:54.242 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.240+0000 7f9835e8f740 1 Processor -- start 2024-09-12T02:05:54.242 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.240+0000 7f9835e8f740 1 -- start start 2024-09-12T02:05:54.242 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.240+0000 7f9835e8f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bfbbae69a0 0x55bfbbae6da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:54.242 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.240+0000 7f9835e8f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bfbbae72e0 0x55bfbbaebda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:54.242 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.240+0000 7f9835e8f740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bfbbaec370 0x55bfbbaee830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:54.242 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.240+0000 7f9835e8f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55bfbba4a2f0 con 0x55bfbbae72e0 2024-09-12T02:05:54.242 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.240+0000 7f9835e8f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55bfbb9f2270 con 0x55bfbbae69a0 2024-09-12T02:05:54.242 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.240+0000 7f9835e8f740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55bfbba462b0 con 0x55bfbbaec370 2024-09-12T02:05:54.242 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.240+0000 7f9830e8d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bfbbae72e0 0x55bfbbaebda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:54.242 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.240+0000 7f9830e8d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bfbbae72e0 0x55bfbbaebda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54204/0 (socket says 172.21.15.16:54204) 2024-09-12T02:05:54.242 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.240+0000 7f9830e8d640 1 -- 172.21.15.16:0/4261803917 learned_addr learned my addr 172.21.15.16:0/4261803917 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:05:54.242 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.240+0000 7f9821ffb640 1 -- 172.21.15.16:0/3760025924 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f981c0109d0 con 0x7f982c102d50 2024-09-12T02:05:54.243 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.240+0000 7f983168e640 1 --2- 172.21.15.16:0/4261803917 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bfbbae69a0 0x55bfbbae6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:54.243 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.240+0000 7f9830e8d640 1 -- 172.21.15.16:0/4261803917 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bfbbaec370 msgr2=0x55bfbbaee830 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T02:05:54.243 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.240+0000 7f9830e8d640 1 --2- 172.21.15.16:0/4261803917 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bfbbaec370 0x55bfbbaee830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:54.243 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.240+0000 7f9830e8d640 1 -- 172.21.15.16:0/4261803917 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bfbbae69a0 msgr2=0x55bfbbae6da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:54.243 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.240+0000 7f9830e8d640 1 --2- 172.21.15.16:0/4261803917 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bfbbae69a0 0x55bfbbae6da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:54.243 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.240+0000 7f9830e8d640 1 -- 172.21.15.16:0/4261803917 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bfbb9f1340 con 0x55bfbbae72e0 2024-09-12T02:05:54.243 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.241+0000 7f9830e8d640 1 --2- 172.21.15.16:0/4261803917 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bfbbae72e0 0x55bfbbaebda0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f9828031610 tx=0x7f9828032c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b8ba7348ceee2a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:54.243 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.241+0000 7f9812ffd640 1 -- 172.21.15.16:0/4261803917 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9828040070 con 0x55bfbbae72e0 2024-09-12T02:05:54.243 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.241+0000 7f9812ffd640 1 -- 172.21.15.16:0/4261803917 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9828032e20 con 0x55bfbbae72e0 2024-09-12T02:05:54.243 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.241+0000 7f9812ffd640 1 -- 172.21.15.16:0/4261803917 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9828035040 con 0x55bfbbae72e0 2024-09-12T02:05:54.243 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.241+0000 7f9821ffb640 1 --2- 172.21.15.16:0/3760025924 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f98140a2cb0 0x7f98140a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:54.243 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.241+0000 7f983168e640 1 --2- 172.21.15.16:0/3760025924 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f98140a2cb0 0x7f98140a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:54.244 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.241+0000 7f9821ffb640 1 -- 172.21.15.16:0/3760025924 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f981c0cea80 con 0x7f982c102d50 2024-09-12T02:05:54.244 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.242+0000 7f9835e8f740 1 -- 172.21.15.16:0/4261803917 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bfbbae72e0 msgr2=0x55bfbbaebda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:54.244 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.242+0000 7f983168e640 1 --2- 172.21.15.16:0/3760025924 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f98140a2cb0 0x7f98140a5170 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f9824005fd0 tx=0x7f9824009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:54.244 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.242+0000 7f9835e8f740 1 --2- 172.21.15.16:0/4261803917 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bfbbae72e0 0x55bfbbaebda0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f9828031610 tx=0x7f9828032c30 comp rx=0 tx=0).stop 2024-09-12T02:05:54.244 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.242+0000 7f9835e8f740 1 -- 172.21.15.16:0/4261803917 shutdown_connections 2024-09-12T02:05:54.244 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.242+0000 7f9835e8f740 1 --2- 172.21.15.16:0/4261803917 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bfbbaec370 0x55bfbbaee830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:54.244 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.242+0000 7f9835e8f740 1 --2- 172.21.15.16:0/4261803917 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bfbbae72e0 0x55bfbbaebda0 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:54.244 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.242+0000 7f9835e8f740 1 --2- 172.21.15.16:0/4261803917 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bfbbae69a0 0x55bfbbae6da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:54.244 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.242+0000 7f9835e8f740 1 -- 172.21.15.16:0/4261803917 >> 172.21.15.16:0/4261803917 conn(0x55bfbbae2010 msgr2=0x55bfbbae2410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:54.244 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.242+0000 7f9835e8f740 1 -- 172.21.15.16:0/4261803917 shutdown_connections 2024-09-12T02:05:54.244 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.242+0000 7f9835e8f740 1 -- 172.21.15.16:0/4261803917 wait complete. 2024-09-12T02:05:54.248 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.242+0000 7f9835e8f740 1 -- start start 2024-09-12T02:05:54.248 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.242+0000 7f9835e8f740 1 Processor -- start 2024-09-12T02:05:54.248 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.243+0000 7f9835e8f740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bfbbae69a0 0x55bfbbaf0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:54.248 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.243+0000 7f9835e8f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bfbbae72e0 0x55bfbbaf1130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:54.248 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.243+0000 7f9835e8f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bfbbaec370 0x55bfbbaf8170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:54.248 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.243+0000 7f9835e8f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55bfbbaef2e0 con 0x55bfbbae72e0 2024-09-12T02:05:54.248 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.243+0000 7f9835e8f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55bfbb9d3290 con 0x55bfbbaec370 2024-09-12T02:05:54.249 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.243+0000 7f9835e8f740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55bfbbaef460 con 0x55bfbbae69a0 2024-09-12T02:05:54.249 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.243+0000 7f9835e8f740 10 client.0 _ll_register_callbacks cb 0x55bfbba04140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:05:54.249 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.243+0000 7f9823fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bfbbae72e0 0x55bfbbaf1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:54.249 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.243+0000 7f9823fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bfbbae72e0 0x55bfbbaf1130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54214/0 (socket says 172.21.15.16:54214) 2024-09-12T02:05:54.249 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.243+0000 7f9823fff640 1 -- 172.21.15.16:0/1652651403 learned_addr learned my addr 172.21.15.16:0/1652651403 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:05:54.249 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.243+0000 7f9830e8d640 1 --2- 172.21.15.16:0/1652651403 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bfbbae69a0 0x55bfbbaf0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:54.249 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.243+0000 7f983168e640 1 --2- 172.21.15.16:0/1652651403 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bfbbaec370 0x55bfbbaf8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:54.249 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.243+0000 7f9823fff640 1 -- 172.21.15.16:0/1652651403 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bfbbae69a0 msgr2=0x55bfbbaf0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:54.249 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.243+0000 7f9823fff640 1 --2- 172.21.15.16:0/1652651403 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bfbbae69a0 0x55bfbbaf0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:54.249 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.243+0000 7f9823fff640 1 -- 172.21.15.16:0/1652651403 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bfbbaec370 msgr2=0x55bfbbaf8170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:54.249 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.243+0000 7f9823fff640 1 --2- 172.21.15.16:0/1652651403 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bfbbaec370 0x55bfbbaf8170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:54.249 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.243+0000 7f9823fff640 1 -- 172.21.15.16:0/1652651403 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bfbb75d210 con 0x55bfbbae72e0 2024-09-12T02:05:54.249 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.243+0000 7f9830e8d640 1 --2- 172.21.15.16:0/1652651403 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bfbbae69a0 0x55bfbbaf0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:05:54.250 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.243+0000 7f983168e640 1 --2- 172.21.15.16:0/1652651403 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bfbbaec370 0x55bfbbaf8170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:05:54.250 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.243+0000 7f9823fff640 1 --2- 172.21.15.16:0/1652651403 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bfbbae72e0 0x55bfbbaf1130 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f981c0a3ac0 tx=0x7f981c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53bf6ddb2a8e2ece server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:54.250 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.243+0000 7f980bfff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:05:54.250 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.244+0000 7f980bfff640 1 -- 172.21.15.16:0/1652651403 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f981c020050 con 0x55bfbbae72e0 2024-09-12T02:05:54.250 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.244+0000 7f980bfff640 1 -- 172.21.15.16:0/1652651403 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f981c09a880 con 0x55bfbbae72e0 2024-09-12T02:05:54.250 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.244+0000 7f9835e8f740 1 -- 172.21.15.16:0/1652651403 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9828032e20 con 0x55bfbbae72e0 2024-09-12T02:05:54.250 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.244+0000 7f980bfff640 1 -- 172.21.15.16:0/1652651403 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f981c09f730 con 0x55bfbbae72e0 2024-09-12T02:05:54.250 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.244+0000 7f9835e8f740 10 client.16647 fetch_fsmap learned FSMap version 25 2024-09-12T02:05:54.250 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.244+0000 7f9835e8f740 1 -- 172.21.15.16:0/1652651403 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55bfbb829620 con 0x55bfbbae72e0 2024-09-12T02:05:54.250 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.244+0000 7f980bfff640 1 -- 172.21.15.16:0/1652651403 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9828032e20 con 0x55bfbbae72e0 2024-09-12T02:05:54.250 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.244+0000 7f980bfff640 1 -- 172.21.15.16:0/1652651403 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f981c09a2f0 con 0x55bfbbae72e0 2024-09-12T02:05:54.250 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.244+0000 7f9835e8f740 10 client.16647 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:05:54.250 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.244+0000 7f9835e8f740 10 client.16647 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:05:54.251 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.244+0000 7f9835e8f740 20 client.16647 populate_metadata read hostname 'smithi016' 2024-09-12T02:05:54.251 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.244+0000 7f9835e8f740 1 --2- 172.21.15.16:0/1652651403 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55bfbbafc8b0 0x55bfbbafeca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:54.251 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.244+0000 7f9835e8f740 4 client.16647 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:05:54.251 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.244+0000 7f9835e8f740 1 -- 172.21.15.16:0/1652651403 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bfbba57a20 con 0x55bfbbafc8b0 2024-09-12T02:05:54.251 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.244+0000 7f9830e8d640 1 --2- 172.21.15.16:0/1652651403 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55bfbbafc8b0 0x55bfbbafeca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:54.251 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.245+0000 7f980bfff640 10 client.16647 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:05:54.251 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.245+0000 7f9830e8d640 1 --2- 172.21.15.16:0/1652651403 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55bfbbafc8b0 0x55bfbbafeca0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=35de6836e19eff74 server_cookie=7a8d5d3ac393ea02 in_seq=0 out_seq=0 2024-09-12T02:05:54.251 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.246+0000 7f980bfff640 1 -- 172.21.15.16:0/1652651403 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bfbba57a20 con 0x55bfbbafc8b0 2024-09-12T02:05:54.251 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.246+0000 7f980bfff640 10 client.16647 handle_command_reply: tid=0 2024-09-12T02:05:54.252 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.246+0000 7f9835e8f740 1 client.16647 shutdown 2024-09-12T02:05:54.252 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.246+0000 7f9835e8f740 1 -- 172.21.15.16:0/1652651403 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bfbbae72e0 msgr2=0x55bfbbaf1130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:54.252 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.246+0000 7f9835e8f740 1 --2- 172.21.15.16:0/1652651403 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bfbbae72e0 0x55bfbbaf1130 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f981c0a3ac0 tx=0x7f981c005880 comp rx=0 tx=0).stop 2024-09-12T02:05:54.252 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.246+0000 7f9835e8f740 1 -- 172.21.15.16:0/1652651403 shutdown_connections 2024-09-12T02:05:54.252 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.246+0000 7f9835e8f740 1 --2- 172.21.15.16:0/1652651403 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55bfbbafc8b0 0x55bfbbafeca0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:54.252 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.246+0000 7f9835e8f740 1 --2- 172.21.15.16:0/1652651403 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bfbbaec370 0x55bfbbaf8170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:54.252 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.246+0000 7f980bfff640 0 client.16647 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:05:54.252 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.246+0000 7f9835e8f740 1 --2- 172.21.15.16:0/1652651403 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bfbbae72e0 0x55bfbbaf1130 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:54.252 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.246+0000 7f9835e8f740 1 --2- 172.21.15.16:0/1652651403 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bfbbae69a0 0x55bfbbaf0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:54.252 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.246+0000 7f9835e8f740 1 -- 172.21.15.16:0/1652651403 >> 172.21.15.16:0/1652651403 conn(0x55bfbbae2010 msgr2=0x55bfbbae3820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:54.252 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.246+0000 7f9835e8f740 1 -- 172.21.15.16:0/1652651403 shutdown_connections 2024-09-12T02:05:54.252 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.246+0000 7f9835e8f740 1 -- 172.21.15.16:0/1652651403 wait complete. 2024-09-12T02:05:54.252 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.246+0000 7f9835e8f740 20 client.16647 trim_cache size 0 max 16384 2024-09-12T02:05:54.266 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.261+0000 7f9835e8f740 1 Processor -- start 2024-09-12T02:05:54.266 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.261+0000 7f9835e8f740 1 -- start start 2024-09-12T02:05:54.266 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.261+0000 7f9835e8f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bfbbae69a0 0x55bfbbb03550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:54.266 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.261+0000 7f9835e8f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bfbbae72e0 0x55bfbbaf8130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:54.266 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.261+0000 7f9835e8f740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bfbbaec370 0x55bfbbb03a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:54.266 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.261+0000 7f9835e8f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f982c1ab660 con 0x55bfbbae72e0 2024-09-12T02:05:54.267 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.261+0000 7f9835e8f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f982c1a8f30 con 0x55bfbbae69a0 2024-09-12T02:05:54.267 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.261+0000 7f9835e8f740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f982c1a6d80 con 0x55bfbbaec370 2024-09-12T02:05:54.267 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.261+0000 7f9823fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bfbbae72e0 0x55bfbbaf8130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:54.267 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.261+0000 7f9823fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bfbbae72e0 0x55bfbbaf8130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54218/0 (socket says 172.21.15.16:54218) 2024-09-12T02:05:54.267 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.261+0000 7f9823fff640 1 -- 172.21.15.16:0/4072440349 learned_addr learned my addr 172.21.15.16:0/4072440349 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:05:54.267 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.261+0000 7f983168e640 1 --2- 172.21.15.16:0/4072440349 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bfbbaec370 0x55bfbbb03a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:54.267 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.261+0000 7f9830e8d640 1 --2- 172.21.15.16:0/4072440349 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bfbbae69a0 0x55bfbbb03550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:54.267 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.261+0000 7f9823fff640 1 -- 172.21.15.16:0/4072440349 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bfbbaec370 msgr2=0x55bfbbb03a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:54.267 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.261+0000 7f9823fff640 1 --2- 172.21.15.16:0/4072440349 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bfbbaec370 0x55bfbbb03a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:54.267 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.261+0000 7f9823fff640 1 -- 172.21.15.16:0/4072440349 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bfbbae69a0 msgr2=0x55bfbbb03550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:54.267 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.261+0000 7f9823fff640 1 --2- 172.21.15.16:0/4072440349 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bfbbae69a0 0x55bfbbb03550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:54.268 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.261+0000 7f9823fff640 1 -- 172.21.15.16:0/4072440349 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bfbbae1e30 con 0x55bfbbae72e0 2024-09-12T02:05:54.268 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.262+0000 7f9823fff640 1 --2- 172.21.15.16:0/4072440349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bfbbae72e0 0x55bfbbaf8130 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f981c09b130 tx=0x7f981c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a062ce54ec3de4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:54.268 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.262+0000 7f98127fc640 1 -- 172.21.15.16:0/4072440349 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f981c020030 con 0x55bfbbae72e0 2024-09-12T02:05:54.268 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.262+0000 7f98127fc640 1 -- 172.21.15.16:0/4072440349 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f981c0a3ff0 con 0x55bfbbae72e0 2024-09-12T02:05:54.268 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.262+0000 7f98127fc640 1 -- 172.21.15.16:0/4072440349 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f981c09a880 con 0x55bfbbae72e0 2024-09-12T02:05:54.268 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.262+0000 7f9835e8f740 1 -- 172.21.15.16:0/4072440349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bfbbae72e0 msgr2=0x55bfbbaf8130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:54.268 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.262+0000 7f9835e8f740 1 --2- 172.21.15.16:0/4072440349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bfbbae72e0 0x55bfbbaf8130 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f981c09b130 tx=0x7f981c000bc0 comp rx=0 tx=0).stop 2024-09-12T02:05:54.268 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.263+0000 7f9835e8f740 1 -- 172.21.15.16:0/4072440349 shutdown_connections 2024-09-12T02:05:54.268 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.263+0000 7f9835e8f740 1 --2- 172.21.15.16:0/4072440349 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bfbbaec370 0x55bfbbb03a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:54.268 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.263+0000 7f9835e8f740 1 --2- 172.21.15.16:0/4072440349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bfbbae72e0 0x55bfbbaf8130 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:54.268 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.263+0000 7f9835e8f740 1 --2- 172.21.15.16:0/4072440349 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bfbbae69a0 0x55bfbbb03550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:54.268 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.263+0000 7f9835e8f740 1 -- 172.21.15.16:0/4072440349 >> 172.21.15.16:0/4072440349 conn(0x55bfbbae2010 msgr2=0x55bfbbae11c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:54.269 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.263+0000 7f9835e8f740 1 -- 172.21.15.16:0/4072440349 shutdown_connections 2024-09-12T02:05:54.269 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.263+0000 7f9835e8f740 1 -- 172.21.15.16:0/4072440349 wait complete. 2024-09-12T02:05:54.269 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.263+0000 7f9835e8f740 1 -- start start 2024-09-12T02:05:54.269 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.263+0000 7f9835e8f740 1 Processor -- start 2024-09-12T02:05:54.269 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.263+0000 7f9835e8f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bfbbae69a0 0x55bfbbaf6180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:54.269 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.263+0000 7f9835e8f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bfbbae72e0 0x55bfbbaf66c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:54.269 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.263+0000 7f9835e8f740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bfbbaec370 0x55bfbbaf76d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:54.269 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.263+0000 7f9835e8f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f982c1a73f0 con 0x55bfbbae72e0 2024-09-12T02:05:54.269 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.263+0000 7f9835e8f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f982c1a8860 con 0x55bfbbae69a0 2024-09-12T02:05:54.269 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.263+0000 7f9835e8f740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f982c1a7ac0 con 0x55bfbbaec370 2024-09-12T02:05:54.269 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.263+0000 7f9835e8f740 10 client.0 _ll_register_callbacks cb 0x7f981c09af70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:05:54.269 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.264+0000 7f9823fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bfbbae72e0 0x55bfbbaf66c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:54.270 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.264+0000 7f9823fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bfbbae72e0 0x55bfbbaf66c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54230/0 (socket says 172.21.15.16:54230) 2024-09-12T02:05:54.270 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.264+0000 7f9823fff640 1 -- 172.21.15.16:0/4254929590 learned_addr learned my addr 172.21.15.16:0/4254929590 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:05:54.270 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.264+0000 7f983168e640 1 --2- 172.21.15.16:0/4254929590 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bfbbaec370 0x55bfbbaf76d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:54.270 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.264+0000 7f9830e8d640 1 --2- 172.21.15.16:0/4254929590 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bfbbae69a0 0x55bfbbaf6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:54.270 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.264+0000 7f9823fff640 1 -- 172.21.15.16:0/4254929590 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bfbbaec370 msgr2=0x55bfbbaf76d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:54.270 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.264+0000 7f9823fff640 1 --2- 172.21.15.16:0/4254929590 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bfbbaec370 0x55bfbbaf76d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:54.270 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.264+0000 7f9823fff640 1 -- 172.21.15.16:0/4254929590 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bfbbae69a0 msgr2=0x55bfbbaf6180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:54.270 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.264+0000 7f9823fff640 1 --2- 172.21.15.16:0/4254929590 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bfbbae69a0 0x55bfbbaf6180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:54.270 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.264+0000 7f9823fff640 1 -- 172.21.15.16:0/4254929590 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bfbbaf7ce0 con 0x55bfbbae72e0 2024-09-12T02:05:54.270 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.264+0000 7f983168e640 1 --2- 172.21.15.16:0/4254929590 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bfbbaec370 0x55bfbbaf76d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:05:54.270 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.264+0000 7f9830e8d640 1 --2- 172.21.15.16:0/4254929590 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bfbbae69a0 0x55bfbbaf6180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:05:54.271 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.264+0000 7f9823fff640 1 --2- 172.21.15.16:0/4254929590 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bfbbae72e0 0x55bfbbaf66c0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f981c097890 tx=0x7f981c09a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8db43f2c3270655 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:54.271 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.264+0000 7f980bfff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:05:54.271 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.264+0000 7f980bfff640 1 -- 172.21.15.16:0/4254929590 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f981c020070 con 0x55bfbbae72e0 2024-09-12T02:05:54.271 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.264+0000 7f980bfff640 1 -- 172.21.15.16:0/4254929590 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f981c09a880 con 0x55bfbbae72e0 2024-09-12T02:05:54.271 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.264+0000 7f9835e8f740 1 -- 172.21.15.16:0/4254929590 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f981c0a3ff0 con 0x55bfbbae72e0 2024-09-12T02:05:54.271 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.264+0000 7f980bfff640 1 -- 172.21.15.16:0/4254929590 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f981c0b4340 con 0x55bfbbae72e0 2024-09-12T02:05:54.271 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.264+0000 7f9835e8f740 10 client.16653 fetch_fsmap learned FSMap version 25 2024-09-12T02:05:54.271 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.265+0000 7f9835e8f740 1 -- 172.21.15.16:0/4254929590 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55bfbbb67e00 con 0x55bfbbae72e0 2024-09-12T02:05:54.271 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.265+0000 7f980bfff640 1 -- 172.21.15.16:0/4254929590 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f981c0a3ff0 con 0x55bfbbae72e0 2024-09-12T02:05:54.271 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.265+0000 7f980bfff640 1 -- 172.21.15.16:0/4254929590 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f981c07c030 con 0x55bfbbae72e0 2024-09-12T02:05:54.271 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.265+0000 7f9835e8f740 10 client.16653 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:05:54.271 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.265+0000 7f9835e8f740 10 client.16653 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:05:54.272 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.265+0000 7f9835e8f740 20 client.16653 populate_metadata read hostname 'smithi016' 2024-09-12T02:05:54.272 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.265+0000 7f9835e8f740 1 --2- 172.21.15.16:0/4254929590 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55bfbbafc8b0 0x55bfbbb6a710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:54.272 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.265+0000 7f9835e8f740 4 client.16653 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:05:54.272 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.265+0000 7f9835e8f740 1 -- 172.21.15.16:0/4254929590 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bfbba51d70 con 0x55bfbbafc8b0 2024-09-12T02:05:54.272 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.265+0000 7f9830e8d640 1 --2- 172.21.15.16:0/4254929590 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55bfbbafc8b0 0x55bfbbb6a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:54.272 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.266+0000 7f9830e8d640 1 --2- 172.21.15.16:0/4254929590 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55bfbbafc8b0 0x55bfbbb6a710 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6bc0047de7cf7d31 server_cookie=79808310e2395aca in_seq=0 out_seq=0 2024-09-12T02:05:54.272 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.266+0000 7f980bfff640 10 client.16653 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:05:54.277 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.276+0000 7f980bfff640 1 -- 172.21.15.16:0/4254929590 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f9828054040 con 0x55bfbbafc8b0 2024-09-12T02:05:54.277 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.276+0000 7f980bfff640 10 client.16653 handle_command_reply: tid=0 2024-09-12T02:05:54.277 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.277+0000 7f9835e8f740 1 client.16653 shutdown 2024-09-12T02:05:54.277 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.277+0000 7f9835e8f740 1 -- 172.21.15.16:0/4254929590 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bfbbae72e0 msgr2=0x55bfbbaf66c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:54.278 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.277+0000 7f9835e8f740 1 --2- 172.21.15.16:0/4254929590 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bfbbae72e0 0x55bfbbaf66c0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f981c097890 tx=0x7f981c09a6c0 comp rx=0 tx=0).stop 2024-09-12T02:05:54.278 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.277+0000 7f9835e8f740 1 -- 172.21.15.16:0/4254929590 shutdown_connections 2024-09-12T02:05:54.278 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.277+0000 7f9835e8f740 1 --2- 172.21.15.16:0/4254929590 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55bfbbafc8b0 0x55bfbbb6a710 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:54.278 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.277+0000 7f9835e8f740 1 --2- 172.21.15.16:0/4254929590 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bfbbaec370 0x55bfbbaf76d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:54.278 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.277+0000 7f980bfff640 0 client.16653 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:05:54.278 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.277+0000 7f9835e8f740 1 --2- 172.21.15.16:0/4254929590 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bfbbae72e0 0x55bfbbaf66c0 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:54.278 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.277+0000 7f9835e8f740 1 --2- 172.21.15.16:0/4254929590 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bfbbae69a0 0x55bfbbaf6180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:54.278 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.277+0000 7f9835e8f740 1 -- 172.21.15.16:0/4254929590 >> 172.21.15.16:0/4254929590 conn(0x55bfbbae2010 msgr2=0x55bfbbae09a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:54.278 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.277+0000 7f9835e8f740 1 -- 172.21.15.16:0/4254929590 shutdown_connections 2024-09-12T02:05:54.278 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.277+0000 7f9835e8f740 1 -- 172.21.15.16:0/4254929590 wait complete. 2024-09-12T02:05:54.278 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.277+0000 7f9835e8f740 20 client.16653 trim_cache size 0 max 16384 2024-09-12T02:05:54.279 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-09-12T02:05:54.279 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-09-12T02:05:54.279 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-09-12T02:05:54.279 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-09-12T02:05:54.280 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.278+0000 7f98330f1640 1 -- 172.21.15.16:0/3760025924 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f98140a2cb0 msgr2=0x7f98140a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:54.280 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.278+0000 7f98330f1640 1 --2- 172.21.15.16:0/3760025924 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f98140a2cb0 0x7f98140a5170 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f9824005fd0 tx=0x7f9824009500 comp rx=0 tx=0).stop 2024-09-12T02:05:54.280 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.278+0000 7f98330f1640 1 -- 172.21.15.16:0/3760025924 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f982c102d50 msgr2=0x7f982c198340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:54.280 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.278+0000 7f98330f1640 1 --2- 172.21.15.16:0/3760025924 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f982c102d50 0x7f982c198340 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f981c00d950 tx=0x7f981c00de10 comp rx=0 tx=0).stop 2024-09-12T02:05:54.280 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.278+0000 7f98330f1640 1 -- 172.21.15.16:0/3760025924 shutdown_connections 2024-09-12T02:05:54.280 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.278+0000 7f98330f1640 1 --2- 172.21.15.16:0/3760025924 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f98140a2cb0 0x7f98140a5170 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:54.280 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.278+0000 7f98330f1640 1 --2- 172.21.15.16:0/3760025924 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f982c103710 0x7f982c19f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:54.280 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.278+0000 7f98330f1640 1 --2- 172.21.15.16:0/3760025924 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f982c102d50 0x7f982c198340 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:54.281 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.278+0000 7f98330f1640 1 --2- 172.21.15.16:0/3760025924 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f982c101b60 0x7f982c197e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:54.281 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.278+0000 7f98330f1640 1 -- 172.21.15.16:0/3760025924 >> 172.21.15.16:0/3760025924 conn(0x7f982c0fd2e0 msgr2=0x7f982c106050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:54.281 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.278+0000 7f98330f1640 1 -- 172.21.15.16:0/3760025924 shutdown_connections 2024-09-12T02:05:54.281 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:05:54.278+0000 7f98330f1640 1 -- 172.21.15.16:0/3760025924 wait complete. 2024-09-12T02:05:54.287 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:05:54 smithi016 ceph-mon[30947]: pgmap v454: 97 pgs: 97 active+clean; 2.1 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.5 KiB/s wr, 9 op/s 2024-09-12T02:05:54.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:54.286+0000 7fc1a6e77640 1 -- 172.21.15.16:0/807535969 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1a0103740 msgr2=0x7fc1a0107b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:54.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:54.286+0000 7fc1a6e77640 1 --2- 172.21.15.16:0/807535969 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1a0103740 0x7fc1a0107b80 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fc194009930 tx=0x7fc19402a440 comp rx=0 tx=0).stop 2024-09-12T02:05:54.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:54.286+0000 7fc1a6e77640 1 -- 172.21.15.16:0/807535969 shutdown_connections 2024-09-12T02:05:54.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:54.286+0000 7fc1a6e77640 1 --2- 172.21.15.16:0/807535969 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1a0108290 0x7fc1a010ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:54.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:54.286+0000 7fc1a6e77640 1 --2- 172.21.15.16:0/807535969 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1a0103740 0x7fc1a0107b80 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:54.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:54.286+0000 7fc1a6e77640 1 --2- 172.21.15.16:0/807535969 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc1a0102d90 0x7fc1a0103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:54.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:54.286+0000 7fc1a6e77640 1 -- 172.21.15.16:0/807535969 >> 172.21.15.16:0/807535969 conn(0x7fc1a00fc7d0 msgr2=0x7fc1a00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:54.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:54.286+0000 7fc1a6e77640 1 -- 172.21.15.16:0/807535969 shutdown_connections 2024-09-12T02:05:54.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:54.286+0000 7fc1a6e77640 1 -- 172.21.15.16:0/807535969 wait complete. 2024-09-12T02:05:54.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:54.287+0000 7fc1a6e77640 1 Processor -- start 2024-09-12T02:05:54.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:54.287+0000 7fc1a6e77640 1 -- start start 2024-09-12T02:05:54.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:54.287+0000 7fc1a6e77640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc1a0102d90 0x7fc1a019a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:54.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:54.287+0000 7fc1a6e77640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1a0103740 0x7fc1a019ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:54.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:54.287+0000 7fc1a6e77640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1a0108290 0x7fc1a019efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:54.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:54.287+0000 7fc1a6e77640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc1a0112090 con 0x7fc1a0103740 2024-09-12T02:05:54.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:54.287+0000 7fc1a6e77640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc1a0111f10 con 0x7fc1a0108290 2024-09-12T02:05:54.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:54.287+0000 7fc1a6e77640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc1a0112210 con 0x7fc1a0102d90 2024-09-12T02:05:54.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:54.287+0000 7fc19ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1a0103740 0x7fc1a019ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:54.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:54.288+0000 7fc19ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1a0103740 0x7fc1a019ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54244/0 (socket says 172.21.15.16:54244) 2024-09-12T02:05:54.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:54.288+0000 7fc19ffff640 1 -- 172.21.15.16:0/2614462936 learned_addr learned my addr 172.21.15.16:0/2614462936 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:05:54.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:54.288+0000 7fc1a5414640 1 --2- 172.21.15.16:0/2614462936 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1a0108290 0x7fc1a019efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:54.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:54.288+0000 7fc1a4c13640 1 --2- 172.21.15.16:0/2614462936 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc1a0102d90 0x7fc1a019a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:54.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:54.288+0000 7fc19ffff640 1 -- 172.21.15.16:0/2614462936 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc1a0102d90 msgr2=0x7fc1a019a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:54.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:54.288+0000 7fc19ffff640 1 --2- 172.21.15.16:0/2614462936 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc1a0102d90 0x7fc1a019a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:54.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:54.288+0000 7fc19ffff640 1 -- 172.21.15.16:0/2614462936 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1a0108290 msgr2=0x7fc1a019efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:54.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:54.288+0000 7fc19ffff640 1 --2- 172.21.15.16:0/2614462936 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1a0108290 0x7fc1a019efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:54.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:54.288+0000 7fc19ffff640 1 -- 172.21.15.16:0/2614462936 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1a019f6a0 con 0x7fc1a0103740 2024-09-12T02:05:54.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:54.288+0000 7fc19ffff640 1 --2- 172.21.15.16:0/2614462936 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1a0103740 0x7fc1a019ac30 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fc194009230 tx=0x7fc1940042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:54.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:54.288+0000 7fc19dffb640 1 -- 172.21.15.16:0/2614462936 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc19403f030 con 0x7fc1a0103740 2024-09-12T02:05:54.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:54.289+0000 7fc1a6e77640 1 -- 172.21.15.16:0/2614462936 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1a019f930 con 0x7fc1a0103740 2024-09-12T02:05:54.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:54.289+0000 7fc19dffb640 1 -- 172.21.15.16:0/2614462936 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc194003ce0 con 0x7fc1a0103740 2024-09-12T02:05:54.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:54.289+0000 7fc19dffb640 1 -- 172.21.15.16:0/2614462936 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc194035070 con 0x7fc1a0103740 2024-09-12T02:05:54.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:54.289+0000 7fc1a6e77640 1 -- 172.21.15.16:0/2614462936 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1a01a7270 con 0x7fc1a0103740 2024-09-12T02:05:54.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:54.290+0000 7fc1a6e77640 1 -- 172.21.15.16:0/2614462936 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1a0103170 con 0x7fc1a0103740 2024-09-12T02:05:54.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:54.291+0000 7fc19dffb640 1 -- 172.21.15.16:0/2614462936 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fc194042030 con 0x7fc1a0103740 2024-09-12T02:05:54.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:54.292+0000 7fc19dffb640 1 --2- 172.21.15.16:0/2614462936 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc1880a2c00 0x7fc1880a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:54.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:54.292+0000 7fc1a4c13640 1 --2- 172.21.15.16:0/2614462936 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc1880a2c00 0x7fc1880a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:54.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:54.292+0000 7fc19dffb640 1 -- 172.21.15.16:0/2614462936 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fc1940ef4e0 con 0x7fc1a0103740 2024-09-12T02:05:54.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:54.292+0000 7fc1a4c13640 1 --2- 172.21.15.16:0/2614462936 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc1880a2c00 0x7fc1880a50c0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fc19000abe0 tx=0x7fc19000a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:54.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:54.296+0000 7fc19dffb640 1 -- 172.21.15.16:0/2614462936 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc1940bc440 con 0x7fc1a0103740 2024-09-12T02:05:54.342 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:05:54.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:05:54 smithi184 ceph-mon[35593]: pgmap v454: 97 pgs: 97 active+clean; 2.1 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.5 KiB/s wr, 9 op/s 2024-09-12T02:05:54.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:05:54 smithi033 ceph-mon[35332]: pgmap v454: 97 pgs: 97 active+clean; 2.1 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.5 KiB/s wr, 9 op/s 2024-09-12T02:05:54.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:54.522+0000 7fc1a6e77640 1 -- 172.21.15.16:0/2614462936 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc1a01a7440 con 0x7fc1a0103740 2024-09-12T02:05:54.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:54.522+0000 7fc19dffb640 1 -- 172.21.15.16:0/2614462936 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11911 (secure 0 0 0) 0x7fc1940bbd40 con 0x7fc1a0103740 2024-09-12T02:05:54.523 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T02:05:54.523 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":25,"btime":"2024-09-12T02:01:37:192021+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3279123279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3279123279},{"type":"v1","addr":"172.21.15.16:6835","nonce":3279123279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4119452382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4119452382},{"type":"v1","addr":"172.21.15.16:6837","nonce":4119452382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/3263823165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3263823165},{"type":"v1","addr":"172.21.15.16:6839","nonce":3263823165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/1510290732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":1510290732},{"type":"v1","addr":"172.21.15.16:6841","nonce":1510290732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/279674603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":279674603},{"type":"v1","addr":"172.21.15.33:6833","nonce":279674603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3555728028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3555728028},{"type":"v1","addr":"172.21.15.33:6835","nonce":3555728028}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T01:59:56.117608+0000","modified":"2024-09-12T02:01:36.294760+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24482,"mds_1":24481,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6833/1563374106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1563374106},{"type":"v1","addr":"172.21.15.184:6833","nonce":1563374106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6837/1892833868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1892833868},{"type":"v1","addr":"172.21.15.33:6837","nonce":1892833868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/1457679686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1457679686},{"type":"v1","addr":"172.21.15.184:6835","nonce":1457679686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.184:6837/1793393087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":1793393087},{"type":"v1","addr":"172.21.15.184:6837","nonce":1793393087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.184:6839/4203347941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":4203347941},{"type":"v1","addr":"172.21.15.184:6839","nonce":4203347941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.33:6839/4136670204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":4136670204},{"type":"v1","addr":"172.21.15.33:6839","nonce":4136670204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24481,24482,24469]},"id":1}]} 2024-09-12T02:05:54.523 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 25 2024-09-12T02:05:54.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:54.525+0000 7fc1a6e77640 1 -- 172.21.15.16:0/2614462936 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc1880a2c00 msgr2=0x7fc1880a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:54.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:54.525+0000 7fc1a6e77640 1 --2- 172.21.15.16:0/2614462936 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc1880a2c00 0x7fc1880a50c0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fc19000abe0 tx=0x7fc19000a2d0 comp rx=0 tx=0).stop 2024-09-12T02:05:54.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:54.525+0000 7fc1a6e77640 1 -- 172.21.15.16:0/2614462936 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1a0103740 msgr2=0x7fc1a019ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:54.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:54.525+0000 7fc1a6e77640 1 --2- 172.21.15.16:0/2614462936 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1a0103740 0x7fc1a019ac30 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fc194009230 tx=0x7fc1940042f0 comp rx=0 tx=0).stop 2024-09-12T02:05:54.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:54.525+0000 7fc1a6e77640 1 -- 172.21.15.16:0/2614462936 shutdown_connections 2024-09-12T02:05:54.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:54.525+0000 7fc1a6e77640 1 --2- 172.21.15.16:0/2614462936 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc1880a2c00 0x7fc1880a50c0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:54.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:54.525+0000 7fc1a6e77640 1 --2- 172.21.15.16:0/2614462936 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1a0108290 0x7fc1a019efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:54.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:54.525+0000 7fc1a6e77640 1 --2- 172.21.15.16:0/2614462936 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1a0103740 0x7fc1a019ac30 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:54.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:54.525+0000 7fc1a6e77640 1 --2- 172.21.15.16:0/2614462936 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc1a0102d90 0x7fc1a019a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:54.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:54.525+0000 7fc1a6e77640 1 -- 172.21.15.16:0/2614462936 >> 172.21.15.16:0/2614462936 conn(0x7fc1a00fc7d0 msgr2=0x7fc1a0100920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:54.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:54.525+0000 7fc1a6e77640 1 -- 172.21.15.16:0/2614462936 shutdown_connections 2024-09-12T02:05:54.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:54.525+0000 7fc1a6e77640 1 -- 172.21.15.16:0/2614462936 wait complete. 2024-09-12T02:05:55.286 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24482 quiesce db / --timeout 90 --expiration 140.15 --await-for 45.0' 2024-09-12T02:05:55.286 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24482 quiesce db / --timeout 90 --expiration 140.15 --await-for 45.0 2024-09-12T02:05:55.310 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:05:55 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/2614462936' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:05:55.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:05:55 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/2614462936' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:05:55.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:05:55 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/2614462936' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:05:55.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.532+0000 7f1c2e0ea640 1 -- 172.21.15.16:0/2692825021 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c28108300 msgr2=0x7f1c2810ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:55.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.532+0000 7f1c2e0ea640 1 --2- 172.21.15.16:0/2692825021 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c28108300 0x7f1c2810ec70 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f1c20009930 tx=0x7f1c2002a440 comp rx=0 tx=0).stop 2024-09-12T02:05:55.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.532+0000 7f1c2e0ea640 1 -- 172.21.15.16:0/2692825021 shutdown_connections 2024-09-12T02:05:55.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.532+0000 7f1c2e0ea640 1 --2- 172.21.15.16:0/2692825021 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c28108300 0x7f1c2810ec70 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:55.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.532+0000 7f1c2e0ea640 1 --2- 172.21.15.16:0/2692825021 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1c28103780 0x7f1c28107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:55.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.532+0000 7f1c2e0ea640 1 --2- 172.21.15.16:0/2692825021 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1c28102dd0 0x7f1c281031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:55.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.532+0000 7f1c2e0ea640 1 -- 172.21.15.16:0/2692825021 >> 172.21.15.16:0/2692825021 conn(0x7f1c280fc7d0 msgr2=0x7f1c280febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:55.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.532+0000 7f1c2e0ea640 1 -- 172.21.15.16:0/2692825021 shutdown_connections 2024-09-12T02:05:55.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.532+0000 7f1c2e0ea640 1 -- 172.21.15.16:0/2692825021 wait complete. 2024-09-12T02:05:55.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.532+0000 7f1c2e0ea640 1 Processor -- start 2024-09-12T02:05:55.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.533+0000 7f1c2e0ea640 1 -- start start 2024-09-12T02:05:55.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.533+0000 7f1c2e0ea640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1c28102dd0 0x7f1c28100a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:55.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.533+0000 7f1c2e0ea640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1c28103780 0x7f1c28100f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:55.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.533+0000 7f1c2e0ea640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c28108300 0x7f1c281014b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:55.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.533+0000 7f1c2e0ea640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1c28112140 con 0x7f1c28108300 2024-09-12T02:05:55.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.533+0000 7f1c2e0ea640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1c28111fc0 con 0x7f1c28102dd0 2024-09-12T02:05:55.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.533+0000 7f1c2e0ea640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1c281122c0 con 0x7f1c28103780 2024-09-12T02:05:55.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.533+0000 7f1c27fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c28108300 0x7f1c281014b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:55.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.533+0000 7f1c27fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c28108300 0x7f1c281014b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54272/0 (socket says 172.21.15.16:54272) 2024-09-12T02:05:55.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.533+0000 7f1c27fff640 1 -- 172.21.15.16:0/2209927344 learned_addr learned my addr 172.21.15.16:0/2209927344 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:05:55.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.533+0000 7f1c26ffd640 1 --2- 172.21.15.16:0/2209927344 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1c28103780 0x7f1c28100f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:55.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.533+0000 7f1c277fe640 1 --2- 172.21.15.16:0/2209927344 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1c28102dd0 0x7f1c28100a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:55.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.533+0000 7f1c27fff640 1 -- 172.21.15.16:0/2209927344 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1c28103780 msgr2=0x7f1c28100f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:55.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.533+0000 7f1c27fff640 1 --2- 172.21.15.16:0/2209927344 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1c28103780 0x7f1c28100f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:55.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.533+0000 7f1c27fff640 1 -- 172.21.15.16:0/2209927344 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1c28102dd0 msgr2=0x7f1c28100a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:55.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.533+0000 7f1c27fff640 1 --2- 172.21.15.16:0/2209927344 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1c28102dd0 0x7f1c28100a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:55.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.533+0000 7f1c27fff640 1 -- 172.21.15.16:0/2209927344 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c281a6ac0 con 0x7f1c28108300 2024-09-12T02:05:55.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.533+0000 7f1c277fe640 1 --2- 172.21.15.16:0/2209927344 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1c28102dd0 0x7f1c28100a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:05:55.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.534+0000 7f1c27fff640 1 --2- 172.21.15.16:0/2209927344 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c28108300 0x7f1c281014b0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f1c2002aa30 tx=0x7f1c20003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:55.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.534+0000 7f1c24ff9640 1 -- 172.21.15.16:0/2209927344 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c2003f070 con 0x7f1c28108300 2024-09-12T02:05:55.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.534+0000 7f1c24ff9640 1 -- 172.21.15.16:0/2209927344 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c20031c90 con 0x7f1c28108300 2024-09-12T02:05:55.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.534+0000 7f1c24ff9640 1 -- 172.21.15.16:0/2209927344 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c20035030 con 0x7f1c28108300 2024-09-12T02:05:55.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.534+0000 7f1c2e0ea640 1 -- 172.21.15.16:0/2209927344 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c281a6d50 con 0x7f1c28108300 2024-09-12T02:05:55.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.534+0000 7f1c2e0ea640 1 -- 172.21.15.16:0/2209927344 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c281a7230 con 0x7f1c28108300 2024-09-12T02:05:55.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.536+0000 7f1c24ff9640 1 -- 172.21.15.16:0/2209927344 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f1c20042070 con 0x7f1c28108300 2024-09-12T02:05:55.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.536+0000 7f1c304c3740 1 Processor -- start 2024-09-12T02:05:55.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.536+0000 7f1c304c3740 1 -- start start 2024-09-12T02:05:55.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.537+0000 7f1c304c3740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3387a9b40 0x55c3387a8090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:55.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.537+0000 7f1c304c3740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3387a85d0 0x55c3387a8a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:55.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.537+0000 7f1c304c3740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c3387b3370 0x55c3387b5760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:55.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.537+0000 7f1c304c3740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55c33870e2f0 con 0x55c3387a9b40 2024-09-12T02:05:55.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.537+0000 7f1c304c3740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55c3386b5cf0 con 0x55c3387a85d0 2024-09-12T02:05:55.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.537+0000 7f1c304c3740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55c33870a2b0 con 0x55c3387b3370 2024-09-12T02:05:55.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.537+0000 7f1c26ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3387a9b40 0x55c3387a8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:55.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.537+0000 7f1c24ff9640 1 --2- 172.21.15.16:0/2209927344 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f1c100a2b80 0x7f1c100a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:55.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.537+0000 7f1c26ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3387a9b40 0x55c3387a8090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54280/0 (socket says 172.21.15.16:54280) 2024-09-12T02:05:55.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.537+0000 7f1c26ffd640 1 -- 172.21.15.16:0/1212862789 learned_addr learned my addr 172.21.15.16:0/1212862789 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:05:55.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.537+0000 7f1c277fe640 1 --2- 172.21.15.16:0/1212862789 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3387a85d0 0x55c3387a8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:55.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.537+0000 7f1c26ffd640 1 --2- 172.21.15.16:0/1212862789 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c3387b3370 0x55c3387b5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:55.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.537+0000 7f1c24ff9640 1 -- 172.21.15.16:0/2209927344 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f1c200efae0 con 0x7f1c28108300 2024-09-12T02:05:55.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.537+0000 7f1c27fff640 1 --2- 172.21.15.16:0/2209927344 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f1c100a2b80 0x7f1c100a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:55.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.537+0000 7f1c26ffd640 1 -- 172.21.15.16:0/1212862789 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c3387b3370 msgr2=0x55c3387b5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:55.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.537+0000 7f1c26ffd640 1 --2- 172.21.15.16:0/1212862789 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c3387b3370 0x55c3387b5760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:55.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.537+0000 7f1c26ffd640 1 -- 172.21.15.16:0/1212862789 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3387a85d0 msgr2=0x55c3387a8a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:55.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.537+0000 7f1c26ffd640 1 --2- 172.21.15.16:0/1212862789 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3387a85d0 0x55c3387a8a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:55.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.537+0000 7f1c26ffd640 1 -- 172.21.15.16:0/1212862789 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c33869a3a0 con 0x55c3387a9b40 2024-09-12T02:05:55.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.537+0000 7f1c277fe640 1 --2- 172.21.15.16:0/1212862789 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3387a85d0 0x55c3387a8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:05:55.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.537+0000 7f1c26ffd640 1 --2- 172.21.15.16:0/1212862789 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3387a9b40 0x55c3387a8090 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f1c1c009330 tx=0x7f1c1c009570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fd35222361e7c4e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:55.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.538+0000 7f1c27fff640 1 --2- 172.21.15.16:0/2209927344 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f1c100a2b80 0x7f1c100a5040 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f1c20005ec0 tx=0x7f1c20005d50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:55.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.538+0000 7f1c0dffb640 1 -- 172.21.15.16:0/1212862789 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c1c017610 con 0x55c3387a9b40 2024-09-12T02:05:55.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.538+0000 7f1c0dffb640 1 -- 172.21.15.16:0/1212862789 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c1c01c070 con 0x55c3387a9b40 2024-09-12T02:05:55.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.538+0000 7f1c0dffb640 1 -- 172.21.15.16:0/1212862789 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c1c015880 con 0x55c3387a9b40 2024-09-12T02:05:55.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.538+0000 7f1c304c3740 1 -- 172.21.15.16:0/1212862789 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3387a9b40 msgr2=0x55c3387a8090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:55.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.538+0000 7f1c304c3740 1 --2- 172.21.15.16:0/1212862789 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3387a9b40 0x55c3387a8090 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f1c1c009330 tx=0x7f1c1c009570 comp rx=0 tx=0).stop 2024-09-12T02:05:55.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.539+0000 7f1c304c3740 1 -- 172.21.15.16:0/1212862789 shutdown_connections 2024-09-12T02:05:55.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.539+0000 7f1c304c3740 1 --2- 172.21.15.16:0/1212862789 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c3387b3370 0x55c3387b5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:55.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.539+0000 7f1c304c3740 1 --2- 172.21.15.16:0/1212862789 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3387a85d0 0x55c3387a8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:55.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.539+0000 7f1c304c3740 1 --2- 172.21.15.16:0/1212862789 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3387a9b40 0x55c3387a8090 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:55.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.539+0000 7f1c304c3740 1 -- 172.21.15.16:0/1212862789 >> 172.21.15.16:0/1212862789 conn(0x55c338771310 msgr2=0x55c3387a4c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:55.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.539+0000 7f1c304c3740 1 -- 172.21.15.16:0/1212862789 shutdown_connections 2024-09-12T02:05:55.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.539+0000 7f1c304c3740 1 -- 172.21.15.16:0/1212862789 wait complete. 2024-09-12T02:05:55.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.539+0000 7f1c304c3740 1 -- start start 2024-09-12T02:05:55.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.539+0000 7f1c304c3740 1 Processor -- start 2024-09-12T02:05:55.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.539+0000 7f1c304c3740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3387a85d0 0x55c3387b2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:55.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.539+0000 7f1c304c3740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3387a9b40 0x55c3387aff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:55.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.539+0000 7f1c304c3740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c3387b3370 0x55c3387b04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:55.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.539+0000 7f1c304c3740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55c3387b6010 con 0x55c3387a85d0 2024-09-12T02:05:55.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.539+0000 7f1c304c3740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55c3386c9060 con 0x55c3387a9b40 2024-09-12T02:05:55.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.539+0000 7f1c304c3740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55c3387b6190 con 0x55c3387b3370 2024-09-12T02:05:55.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.539+0000 7f1c304c3740 10 client.0 _ll_register_callbacks cb 0x55c33841c8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:05:55.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.540+0000 7f1c26ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3387a85d0 0x55c3387b2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:55.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.540+0000 7f1c26ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3387a85d0 0x55c3387b2cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54284/0 (socket says 172.21.15.16:54284) 2024-09-12T02:05:55.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.540+0000 7f1c26ffd640 1 -- 172.21.15.16:0/3565389166 learned_addr learned my addr 172.21.15.16:0/3565389166 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:05:55.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.540+0000 7f1c277fe640 1 --2- 172.21.15.16:0/3565389166 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3387a9b40 0x55c3387aff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:55.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.540+0000 7f1c26ffd640 1 --2- 172.21.15.16:0/3565389166 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c3387b3370 0x55c3387b04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:55.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.540+0000 7f1c26ffd640 1 -- 172.21.15.16:0/3565389166 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c3387b3370 msgr2=0x55c3387b04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:55.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.540+0000 7f1c26ffd640 1 --2- 172.21.15.16:0/3565389166 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c3387b3370 0x55c3387b04d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:55.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.540+0000 7f1c26ffd640 1 -- 172.21.15.16:0/3565389166 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3387a9b40 msgr2=0x55c3387aff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:55.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.540+0000 7f1c26ffd640 1 --2- 172.21.15.16:0/3565389166 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3387a9b40 0x55c3387aff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:55.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.540+0000 7f1c26ffd640 1 -- 172.21.15.16:0/3565389166 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c33847f3c0 con 0x55c3387a85d0 2024-09-12T02:05:55.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.540+0000 7f1c277fe640 1 --2- 172.21.15.16:0/3565389166 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3387a9b40 0x55c3387aff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:05:55.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.540+0000 7f1c26ffd640 1 --2- 172.21.15.16:0/3565389166 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3387a85d0 0x55c3387b2cc0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f1c1c015420 tx=0x7f1c1c015520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29f39533c5c329b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:55.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.540+0000 7f1c06ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:05:55.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.540+0000 7f1c06ffd640 1 -- 172.21.15.16:0/3565389166 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c1c0178a0 con 0x55c3387a85d0 2024-09-12T02:05:55.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.540+0000 7f1c06ffd640 1 -- 172.21.15.16:0/3565389166 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f1c1c017a40 con 0x55c3387a85d0 2024-09-12T02:05:55.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.540+0000 7f1c304c3740 1 -- 172.21.15.16:0/3565389166 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c1c01c070 con 0x55c3387a85d0 2024-09-12T02:05:55.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.540+0000 7f1c06ffd640 1 -- 172.21.15.16:0/3565389166 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1c1c015600 con 0x55c3387a85d0 2024-09-12T02:05:55.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.540+0000 7f1c304c3740 10 client.16671 fetch_fsmap learned FSMap version 25 2024-09-12T02:05:55.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.540+0000 7f1c304c3740 1 -- 172.21.15.16:0/3565389166 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c3387c1a50 con 0x55c3387a85d0 2024-09-12T02:05:55.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.541+0000 7f1c06ffd640 1 -- 172.21.15.16:0/3565389166 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c1c01c070 con 0x55c3387a85d0 2024-09-12T02:05:55.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.541+0000 7f1c06ffd640 1 -- 172.21.15.16:0/3565389166 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f1c1c02ca20 con 0x55c3387a85d0 2024-09-12T02:05:55.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.541+0000 7f1c304c3740 10 client.16671 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:05:55.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.541+0000 7f1c304c3740 10 client.16671 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:05:55.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.541+0000 7f1c304c3740 20 client.16671 populate_metadata read hostname 'smithi016' 2024-09-12T02:05:55.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.541+0000 7f1c304c3740 1 --2- 172.21.15.16:0/3565389166 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55c3387c2eb0 0x55c3387c52a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:55.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.541+0000 7f1c304c3740 4 client.16671 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:05:55.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.541+0000 7f1c304c3740 1 -- 172.21.15.16:0/3565389166 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c33871b860 con 0x55c3387c2eb0 2024-09-12T02:05:55.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.541+0000 7f1c277fe640 1 --2- 172.21.15.16:0/3565389166 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55c3387c2eb0 0x55c3387c52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:55.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.542+0000 7f1c277fe640 1 --2- 172.21.15.16:0/3565389166 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55c3387c2eb0 0x55c3387c52a0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1bdca911f881822 server_cookie=b4bf719f367e3d33 in_seq=0 out_seq=0 2024-09-12T02:05:55.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.542+0000 7f1c06ffd640 10 client.16671 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:05:55.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.542+0000 7f1c06ffd640 1 -- 172.21.15.16:0/3565389166 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c33871b860 con 0x55c3387c2eb0 2024-09-12T02:05:55.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.542+0000 7f1c06ffd640 10 client.16671 handle_command_reply: tid=0 2024-09-12T02:05:55.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.542+0000 7f1c304c3740 1 client.16671 shutdown 2024-09-12T02:05:55.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.543+0000 7f1c304c3740 1 -- 172.21.15.16:0/3565389166 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3387a85d0 msgr2=0x55c3387b2cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:55.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.543+0000 7f1c304c3740 1 --2- 172.21.15.16:0/3565389166 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3387a85d0 0x55c3387b2cc0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f1c1c015420 tx=0x7f1c1c015520 comp rx=0 tx=0).stop 2024-09-12T02:05:55.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.543+0000 7f1c304c3740 1 -- 172.21.15.16:0/3565389166 shutdown_connections 2024-09-12T02:05:55.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.543+0000 7f1c304c3740 1 --2- 172.21.15.16:0/3565389166 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55c3387c2eb0 0x55c3387c52a0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:55.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.543+0000 7f1c304c3740 1 --2- 172.21.15.16:0/3565389166 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c3387b3370 0x55c3387b04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:55.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.543+0000 7f1c304c3740 1 --2- 172.21.15.16:0/3565389166 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3387a9b40 0x55c3387aff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:55.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.543+0000 7f1c06ffd640 0 client.16671 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:05:55.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.543+0000 7f1c304c3740 1 --2- 172.21.15.16:0/3565389166 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3387a85d0 0x55c3387b2cc0 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:55.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.543+0000 7f1c304c3740 1 -- 172.21.15.16:0/3565389166 >> 172.21.15.16:0/3565389166 conn(0x55c338771310 msgr2=0x55c3387aa930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:55.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.543+0000 7f1c304c3740 1 -- 172.21.15.16:0/3565389166 shutdown_connections 2024-09-12T02:05:55.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.543+0000 7f1c304c3740 1 -- 172.21.15.16:0/3565389166 wait complete. 2024-09-12T02:05:55.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.543+0000 7f1c304c3740 20 client.16671 trim_cache size 0 max 16384 2024-09-12T02:05:55.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.556+0000 7f1c304c3740 1 Processor -- start 2024-09-12T02:05:55.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.556+0000 7f1c304c3740 1 -- start start 2024-09-12T02:05:55.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.556+0000 7f1c304c3740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3387a85d0 0x55c3387c82c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:55.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.556+0000 7f1c304c3740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c3387a9b40 0x55c3387c8800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:55.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.556+0000 7f1c304c3740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3387b3370 0x55c3387ad1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:55.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.556+0000 7f1c304c3740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1c281b1f40 con 0x55c3387a85d0 2024-09-12T02:05:55.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.556+0000 7f1c304c3740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1c281af810 con 0x55c3387b3370 2024-09-12T02:05:55.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.556+0000 7f1c304c3740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1c281ad6c0 con 0x55c3387a9b40 2024-09-12T02:05:55.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.556+0000 7f1c26ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3387a85d0 0x55c3387c82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:55.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.556+0000 7f1c26ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3387a85d0 0x55c3387c82c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54300/0 (socket says 172.21.15.16:54300) 2024-09-12T02:05:55.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.556+0000 7f1c26ffd640 1 -- 172.21.15.16:0/3746977193 learned_addr learned my addr 172.21.15.16:0/3746977193 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:05:55.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.556+0000 7f1c26ffd640 1 --2- 172.21.15.16:0/3746977193 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3387b3370 0x55c3387ad1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:55.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.556+0000 7f1c277fe640 1 --2- 172.21.15.16:0/3746977193 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c3387a9b40 0x55c3387c8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:55.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.556+0000 7f1c26ffd640 1 -- 172.21.15.16:0/3746977193 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c3387a9b40 msgr2=0x55c3387c8800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:55.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.557+0000 7f1c26ffd640 1 --2- 172.21.15.16:0/3746977193 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c3387a9b40 0x55c3387c8800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:55.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.557+0000 7f1c26ffd640 1 -- 172.21.15.16:0/3746977193 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3387b3370 msgr2=0x55c3387ad1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:55.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.557+0000 7f1c26ffd640 1 --2- 172.21.15.16:0/3746977193 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3387b3370 0x55c3387ad1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:55.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.557+0000 7f1c26ffd640 1 -- 172.21.15.16:0/3746977193 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c3387b0840 con 0x55c3387a85d0 2024-09-12T02:05:55.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.557+0000 7f1c277fe640 1 --2- 172.21.15.16:0/3746977193 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c3387a9b40 0x55c3387c8800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:05:55.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.557+0000 7f1c26ffd640 1 --2- 172.21.15.16:0/3746977193 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3387a85d0 0x55c3387c82c0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f1c1c0154f0 tx=0x7f1c1c015590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e75e1bde897aa947 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:55.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.557+0000 7f1c0d7fa640 1 -- 172.21.15.16:0/3746977193 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c1c0176a0 con 0x55c3387a85d0 2024-09-12T02:05:55.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.557+0000 7f1c0d7fa640 1 -- 172.21.15.16:0/3746977193 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c1c01c040 con 0x55c3387a85d0 2024-09-12T02:05:55.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.557+0000 7f1c0d7fa640 1 -- 172.21.15.16:0/3746977193 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c1c02c6a0 con 0x55c3387a85d0 2024-09-12T02:05:55.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.558+0000 7f1c304c3740 1 -- 172.21.15.16:0/3746977193 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3387a85d0 msgr2=0x55c3387c82c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:55.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.558+0000 7f1c304c3740 1 --2- 172.21.15.16:0/3746977193 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3387a85d0 0x55c3387c82c0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f1c1c0154f0 tx=0x7f1c1c015590 comp rx=0 tx=0).stop 2024-09-12T02:05:55.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.558+0000 7f1c304c3740 1 -- 172.21.15.16:0/3746977193 shutdown_connections 2024-09-12T02:05:55.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.558+0000 7f1c304c3740 1 --2- 172.21.15.16:0/3746977193 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3387b3370 0x55c3387ad1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:55.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.558+0000 7f1c304c3740 1 --2- 172.21.15.16:0/3746977193 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c3387a9b40 0x55c3387c8800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:55.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.558+0000 7f1c304c3740 1 --2- 172.21.15.16:0/3746977193 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3387a85d0 0x55c3387c82c0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:55.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.558+0000 7f1c304c3740 1 -- 172.21.15.16:0/3746977193 >> 172.21.15.16:0/3746977193 conn(0x55c338771310 msgr2=0x55c3387ab310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:55.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.558+0000 7f1c304c3740 1 -- 172.21.15.16:0/3746977193 shutdown_connections 2024-09-12T02:05:55.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.558+0000 7f1c304c3740 1 -- 172.21.15.16:0/3746977193 wait complete. 2024-09-12T02:05:55.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.558+0000 7f1c304c3740 1 -- start start 2024-09-12T02:05:55.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.558+0000 7f1c304c3740 1 Processor -- start 2024-09-12T02:05:55.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.558+0000 7f1c304c3740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3387a85d0 0x55c3387c86c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:55.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.558+0000 7f1c304c3740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3387a9b40 0x55c3387c8c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:55.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.559+0000 7f1c304c3740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c3387b3370 0x55c3387c5c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:55.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.559+0000 7f1c304c3740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1c281add10 con 0x55c3387a85d0 2024-09-12T02:05:55.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.559+0000 7f1c304c3740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1c281af140 con 0x55c3387a9b40 2024-09-12T02:05:55.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.559+0000 7f1c304c3740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1c281ae3a0 con 0x55c3387b3370 2024-09-12T02:05:55.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.559+0000 7f1c304c3740 10 client.0 _ll_register_callbacks cb 0x7f1c1800a630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:05:55.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.559+0000 7f1c26ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3387a85d0 0x55c3387c86c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:55.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.559+0000 7f1c26ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3387a85d0 0x55c3387c86c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54306/0 (socket says 172.21.15.16:54306) 2024-09-12T02:05:55.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.559+0000 7f1c26ffd640 1 -- 172.21.15.16:0/3951641293 learned_addr learned my addr 172.21.15.16:0/3951641293 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:05:55.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.559+0000 7f1c277fe640 1 --2- 172.21.15.16:0/3951641293 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3387a9b40 0x55c3387c8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:55.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.559+0000 7f1c26ffd640 1 --2- 172.21.15.16:0/3951641293 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c3387b3370 0x55c3387c5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:55.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.559+0000 7f1c26ffd640 1 -- 172.21.15.16:0/3951641293 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c3387b3370 msgr2=0x55c3387c5c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:55.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.559+0000 7f1c26ffd640 1 --2- 172.21.15.16:0/3951641293 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c3387b3370 0x55c3387c5c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:55.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.559+0000 7f1c26ffd640 1 -- 172.21.15.16:0/3951641293 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3387a9b40 msgr2=0x55c3387c8c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:55.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.559+0000 7f1c26ffd640 1 --2- 172.21.15.16:0/3951641293 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3387a9b40 0x55c3387c8c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:55.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.559+0000 7f1c26ffd640 1 -- 172.21.15.16:0/3951641293 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c3387c6490 con 0x55c3387a85d0 2024-09-12T02:05:55.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.559+0000 7f1c277fe640 1 --2- 172.21.15.16:0/3951641293 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3387a9b40 0x55c3387c8c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:05:55.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.559+0000 7f1c26ffd640 1 --2- 172.21.15.16:0/3951641293 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3387a85d0 0x55c3387c86c0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f1c1c009150 tx=0x7f1c1c004640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af46ec13f5513ac3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:55.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.559+0000 7f1c06ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:05:55.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.560+0000 7f1c06ffd640 1 -- 172.21.15.16:0/3951641293 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c1c0159b0 con 0x55c3387a85d0 2024-09-12T02:05:55.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.560+0000 7f1c06ffd640 1 -- 172.21.15.16:0/3951641293 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f1c1c015b50 con 0x55c3387a85d0 2024-09-12T02:05:55.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.560+0000 7f1c304c3740 1 -- 172.21.15.16:0/3951641293 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c1c01c040 con 0x55c3387a85d0 2024-09-12T02:05:55.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.560+0000 7f1c06ffd640 1 -- 172.21.15.16:0/3951641293 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1c1c015d50 con 0x55c3387a85d0 2024-09-12T02:05:55.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.560+0000 7f1c304c3740 10 client.16677 fetch_fsmap learned FSMap version 25 2024-09-12T02:05:55.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.560+0000 7f1c304c3740 1 -- 172.21.15.16:0/3951641293 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c33882d410 con 0x55c3387a85d0 2024-09-12T02:05:55.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.560+0000 7f1c06ffd640 1 -- 172.21.15.16:0/3951641293 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c1c01c040 con 0x55c3387a85d0 2024-09-12T02:05:55.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.560+0000 7f1c06ffd640 1 -- 172.21.15.16:0/3951641293 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f1c1c03a680 con 0x55c3387a85d0 2024-09-12T02:05:55.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.560+0000 7f1c304c3740 10 client.16677 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:05:55.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.560+0000 7f1c304c3740 10 client.16677 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:05:55.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.560+0000 7f1c304c3740 20 client.16677 populate_metadata read hostname 'smithi016' 2024-09-12T02:05:55.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.560+0000 7f1c304c3740 1 --2- 172.21.15.16:0/3951641293 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55c33882da30 0x55c33882fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:55.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.560+0000 7f1c304c3740 4 client.16677 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T02:05:55.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.560+0000 7f1c304c3740 1 -- 172.21.15.16:0/3951641293 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c338715d70 con 0x55c33882da30 2024-09-12T02:05:55.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.561+0000 7f1c277fe640 1 --2- 172.21.15.16:0/3951641293 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55c33882da30 0x55c33882fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:55.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.561+0000 7f1c277fe640 1 --2- 172.21.15.16:0/3951641293 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55c33882da30 0x55c33882fe50 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=346efb2372987bd8 server_cookie=6b14a0980225ea05 in_seq=0 out_seq=0 2024-09-12T02:05:55.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.561+0000 7f1c06ffd640 10 client.16677 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:05:55.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.561+0000 7f1c06ffd640 1 -- 172.21.15.16:0/3951641293 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c338715d70 con 0x55c33882da30 2024-09-12T02:05:55.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.561+0000 7f1c06ffd640 10 client.16677 handle_command_reply: tid=0 2024-09-12T02:05:55.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.561+0000 7f1c304c3740 1 client.16677 shutdown 2024-09-12T02:05:55.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.562+0000 7f1c304c3740 1 -- 172.21.15.16:0/3951641293 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3387a85d0 msgr2=0x55c3387c86c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:55.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.562+0000 7f1c304c3740 1 --2- 172.21.15.16:0/3951641293 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3387a85d0 0x55c3387c86c0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f1c1c009150 tx=0x7f1c1c004640 comp rx=0 tx=0).stop 2024-09-12T02:05:55.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.562+0000 7f1c304c3740 1 -- 172.21.15.16:0/3951641293 shutdown_connections 2024-09-12T02:05:55.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.562+0000 7f1c304c3740 1 --2- 172.21.15.16:0/3951641293 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55c33882da30 0x55c33882fe50 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:55.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.562+0000 7f1c304c3740 1 --2- 172.21.15.16:0/3951641293 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c3387b3370 0x55c3387c5c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:55.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.562+0000 7f1c304c3740 1 --2- 172.21.15.16:0/3951641293 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3387a9b40 0x55c3387c8c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:55.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.562+0000 7f1c06ffd640 0 client.16677 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:05:55.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.562+0000 7f1c304c3740 1 --2- 172.21.15.16:0/3951641293 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3387a85d0 0x55c3387c86c0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:55.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.562+0000 7f1c304c3740 1 -- 172.21.15.16:0/3951641293 >> 172.21.15.16:0/3951641293 conn(0x55c338771310 msgr2=0x55c3387a3d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:55.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.562+0000 7f1c304c3740 1 -- 172.21.15.16:0/3951641293 shutdown_connections 2024-09-12T02:05:55.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.562+0000 7f1c304c3740 1 -- 172.21.15.16:0/3951641293 wait complete. 2024-09-12T02:05:55.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.562+0000 7f1c304c3740 20 client.16677 trim_cache size 0 max 16384 2024-09-12T02:05:55.569 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T02:05:55.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.563+0000 7f1c2e0ea640 1 -- 172.21.15.16:0/2209927344 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f1c100a2b80 msgr2=0x7f1c100a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:55.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.563+0000 7f1c2e0ea640 1 --2- 172.21.15.16:0/2209927344 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f1c100a2b80 0x7f1c100a5040 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f1c20005ec0 tx=0x7f1c20005d50 comp rx=0 tx=0).stop 2024-09-12T02:05:55.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.563+0000 7f1c2e0ea640 1 -- 172.21.15.16:0/2209927344 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c28108300 msgr2=0x7f1c281014b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:55.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.563+0000 7f1c2e0ea640 1 --2- 172.21.15.16:0/2209927344 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c28108300 0x7f1c281014b0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f1c2002aa30 tx=0x7f1c20003b90 comp rx=0 tx=0).stop 2024-09-12T02:05:55.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.563+0000 7f1c2e0ea640 1 -- 172.21.15.16:0/2209927344 shutdown_connections 2024-09-12T02:05:55.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.563+0000 7f1c2e0ea640 1 --2- 172.21.15.16:0/2209927344 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f1c100a2b80 0x7f1c100a5040 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:55.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.563+0000 7f1c2e0ea640 1 --2- 172.21.15.16:0/2209927344 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c28108300 0x7f1c281014b0 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:55.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.563+0000 7f1c2e0ea640 1 --2- 172.21.15.16:0/2209927344 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1c28103780 0x7f1c28100f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:55.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.563+0000 7f1c2e0ea640 1 --2- 172.21.15.16:0/2209927344 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1c28102dd0 0x7f1c28100a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:55.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.563+0000 7f1c2e0ea640 1 -- 172.21.15.16:0/2209927344 >> 172.21.15.16:0/2209927344 conn(0x7f1c280fc7d0 msgr2=0x7f1c281063b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:55.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.563+0000 7f1c2e0ea640 1 -- 172.21.15.16:0/2209927344 shutdown_connections 2024-09-12T02:05:55.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:55.563+0000 7f1c2e0ea640 1 -- 172.21.15.16:0/2209927344 wait complete. 2024-09-12T02:05:56.158 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:05:56 smithi016 ceph-mon[30947]: pgmap v455: 97 pgs: 97 active+clean; 2.1 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 5.7 KiB/s wr, 11 op/s 2024-09-12T02:05:56.261 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:05:56.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:05:56 smithi184 ceph-mon[35593]: pgmap v455: 97 pgs: 97 active+clean; 2.1 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 5.7 KiB/s wr, 11 op/s 2024-09-12T02:05:56.418 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:05:56 smithi033 ceph-mon[35332]: pgmap v455: 97 pgs: 97 active+clean; 2.1 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 5.7 KiB/s wr, 11 op/s 2024-09-12T02:05:56.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.728+0000 7f8a157b0640 1 -- 172.21.15.16:0/37151946 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a1010c170 msgr2=0x7f8a1010e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:56.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.728+0000 7f8a157b0640 1 --2- 172.21.15.16:0/37151946 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a1010c170 0x7f8a1010e610 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f8a08009ca0 tx=0x7f8a0802a950 comp rx=0 tx=0).stop 2024-09-12T02:05:56.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.728+0000 7f8a157b0640 1 -- 172.21.15.16:0/37151946 shutdown_connections 2024-09-12T02:05:56.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.728+0000 7f8a157b0640 1 --2- 172.21.15.16:0/37151946 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a1010c170 0x7f8a1010e610 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:56.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.728+0000 7f8a157b0640 1 --2- 172.21.15.16:0/37151946 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a10103660 0x7f8a1010bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:56.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.728+0000 7f8a157b0640 1 --2- 172.21.15.16:0/37151946 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a10102d40 0x7f8a10103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:56.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.728+0000 7f8a157b0640 1 -- 172.21.15.16:0/37151946 >> 172.21.15.16:0/37151946 conn(0x7f8a100fc7a0 msgr2=0x7f8a100febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:56.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.728+0000 7f8a157b0640 1 -- 172.21.15.16:0/37151946 shutdown_connections 2024-09-12T02:05:56.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.728+0000 7f8a157b0640 1 -- 172.21.15.16:0/37151946 wait complete. 2024-09-12T02:05:56.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.729+0000 7f8a157b0640 1 Processor -- start 2024-09-12T02:05:56.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.729+0000 7f8a157b0640 1 -- start start 2024-09-12T02:05:56.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.729+0000 7f8a157b0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a10102d40 0x7f8a101009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:56.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.729+0000 7f8a157b0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a10103660 0x7f8a10100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:56.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.729+0000 7f8a157b0640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a1010c170 0x7f8a10101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:56.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.729+0000 7f8a157b0640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8a10111f50 con 0x7f8a10103660 2024-09-12T02:05:56.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.729+0000 7f8a157b0640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8a10111dd0 con 0x7f8a10102d40 2024-09-12T02:05:56.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.729+0000 7f8a157b0640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8a101120d0 con 0x7f8a1010c170 2024-09-12T02:05:56.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.729+0000 7f8a07fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a10103660 0x7f8a10100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:56.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.730+0000 7f8a07fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a10103660 0x7f8a10100ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54324/0 (socket says 172.21.15.16:54324) 2024-09-12T02:05:56.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.730+0000 7f8a07fff640 1 -- 172.21.15.16:0/2849828008 learned_addr learned my addr 172.21.15.16:0/2849828008 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:05:56.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.730+0000 7f8a0effd640 1 --2- 172.21.15.16:0/2849828008 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a10102d40 0x7f8a101009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:56.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.730+0000 7f8a0f7fe640 1 --2- 172.21.15.16:0/2849828008 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a1010c170 0x7f8a10101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:56.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.730+0000 7f8a07fff640 1 -- 172.21.15.16:0/2849828008 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a1010c170 msgr2=0x7f8a10101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:56.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.730+0000 7f8a07fff640 1 --2- 172.21.15.16:0/2849828008 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a1010c170 0x7f8a10101420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:56.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.730+0000 7f8a07fff640 1 -- 172.21.15.16:0/2849828008 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a10102d40 msgr2=0x7f8a101009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:56.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.730+0000 7f8a07fff640 1 --2- 172.21.15.16:0/2849828008 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a10102d40 0x7f8a101009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:56.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.730+0000 7f8a07fff640 1 -- 172.21.15.16:0/2849828008 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a101a6920 con 0x7f8a10103660 2024-09-12T02:05:56.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.730+0000 7f8a0f7fe640 1 --2- 172.21.15.16:0/2849828008 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a1010c170 0x7f8a10101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:05:56.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.730+0000 7f8a07fff640 1 --2- 172.21.15.16:0/2849828008 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a10103660 0x7f8a10100ee0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f89f800e9f0 tx=0x7f89f800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:56.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.730+0000 7f8a0cff9640 1 -- 172.21.15.16:0/2849828008 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89f800ccc0 con 0x7f8a10103660 2024-09-12T02:05:56.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.730+0000 7f8a157b0640 1 -- 172.21.15.16:0/2849828008 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a101a6c10 con 0x7f8a10103660 2024-09-12T02:05:56.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.730+0000 7f8a0cff9640 1 -- 172.21.15.16:0/2849828008 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89f800ce60 con 0x7f8a10103660 2024-09-12T02:05:56.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.730+0000 7f8a0cff9640 1 -- 172.21.15.16:0/2849828008 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89f8010690 con 0x7f8a10103660 2024-09-12T02:05:56.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.730+0000 7f8a157b0640 1 -- 172.21.15.16:0/2849828008 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a101a7100 con 0x7f8a10103660 2024-09-12T02:05:56.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.733+0000 7f8a0cff9640 1 -- 172.21.15.16:0/2849828008 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f89f80046c0 con 0x7f8a10103660 2024-09-12T02:05:56.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.733+0000 7f8a17b89740 1 Processor -- start 2024-09-12T02:05:56.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.734+0000 7f8a17b89740 1 -- start start 2024-09-12T02:05:56.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.734+0000 7f8a17b89740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556b3eab1850 0x556b3eab1c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:56.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.734+0000 7f8a17b89740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556b3eab2170 0x556b3eabc4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:56.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.734+0000 7f8a17b89740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b3eabca30 0x556b3eabee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:56.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.734+0000 7f8a17b89740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x556b3ea16160 con 0x556b3eab2170 2024-09-12T02:05:56.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.734+0000 7f8a17b89740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x556b3e9bd720 con 0x556b3eabca30 2024-09-12T02:05:56.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.734+0000 7f8a17b89740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x556b3ea12120 con 0x556b3eab1850 2024-09-12T02:05:56.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.734+0000 7f8a0effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556b3eab2170 0x556b3eabc4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:56.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.734+0000 7f8a0effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556b3eab2170 0x556b3eabc4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54332/0 (socket says 172.21.15.16:54332) 2024-09-12T02:05:56.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.734+0000 7f8a0effd640 1 -- 172.21.15.16:0/3762949131 learned_addr learned my addr 172.21.15.16:0/3762949131 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:05:56.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.734+0000 7f8a0f7fe640 1 --2- 172.21.15.16:0/3762949131 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556b3eab1850 0x556b3eab1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:56.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.734+0000 7f8a07fff640 1 --2- 172.21.15.16:0/3762949131 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b3eabca30 0x556b3eabee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:56.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.734+0000 7f8a0effd640 1 -- 172.21.15.16:0/3762949131 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556b3eab1850 msgr2=0x556b3eab1c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:56.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.734+0000 7f8a0effd640 1 --2- 172.21.15.16:0/3762949131 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556b3eab1850 0x556b3eab1c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:56.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.734+0000 7f8a0effd640 1 -- 172.21.15.16:0/3762949131 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b3eabca30 msgr2=0x556b3eabee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:56.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.734+0000 7f8a0effd640 1 --2- 172.21.15.16:0/3762949131 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b3eabca30 0x556b3eabee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:56.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.735+0000 7f8a0effd640 1 -- 172.21.15.16:0/3762949131 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b3e787ba0 con 0x556b3eab2170 2024-09-12T02:05:56.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.735+0000 7f8a07fff640 1 --2- 172.21.15.16:0/3762949131 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b3eabca30 0x556b3eabee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:05:56.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.735+0000 7f8a0cff9640 1 --2- 172.21.15.16:0/2849828008 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f89fc0a2b80 0x7f89fc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:56.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.735+0000 7f8a0f7fe640 1 --2- 172.21.15.16:0/2849828008 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f89fc0a2b80 0x7f89fc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:56.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.735+0000 7f8a0cff9640 1 -- 172.21.15.16:0/2849828008 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f89f80cf3a0 con 0x7f8a10103660 2024-09-12T02:05:56.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.735+0000 7f8a0effd640 1 --2- 172.21.15.16:0/3762949131 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556b3eab2170 0x556b3eabc4f0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f8a0000bd10 tx=0x7f8a000107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84909b46d1444966 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:56.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.735+0000 7f8a057fa640 1 -- 172.21.15.16:0/3762949131 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a00004030 con 0x556b3eab2170 2024-09-12T02:05:56.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.735+0000 7f8a057fa640 1 -- 172.21.15.16:0/3762949131 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a000041d0 con 0x556b3eab2170 2024-09-12T02:05:56.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.735+0000 7f8a057fa640 1 -- 172.21.15.16:0/3762949131 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a000117e0 con 0x556b3eab2170 2024-09-12T02:05:56.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.736+0000 7f8a0f7fe640 1 --2- 172.21.15.16:0/2849828008 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f89fc0a2b80 0x7f89fc0a5040 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f8a08005cc0 tx=0x7f8a08005c50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:56.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.736+0000 7f8a17b89740 1 -- 172.21.15.16:0/3762949131 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556b3eab2170 msgr2=0x556b3eabc4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:56.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.736+0000 7f8a17b89740 1 --2- 172.21.15.16:0/3762949131 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556b3eab2170 0x556b3eabc4f0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f8a0000bd10 tx=0x7f8a000107a0 comp rx=0 tx=0).stop 2024-09-12T02:05:56.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.736+0000 7f8a17b89740 1 -- 172.21.15.16:0/3762949131 shutdown_connections 2024-09-12T02:05:56.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.736+0000 7f8a17b89740 1 --2- 172.21.15.16:0/3762949131 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b3eabca30 0x556b3eabee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:56.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.736+0000 7f8a17b89740 1 --2- 172.21.15.16:0/3762949131 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556b3eab2170 0x556b3eabc4f0 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:56.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.736+0000 7f8a17b89740 1 --2- 172.21.15.16:0/3762949131 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556b3eab1850 0x556b3eab1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:56.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.736+0000 7f8a17b89740 1 -- 172.21.15.16:0/3762949131 >> 172.21.15.16:0/3762949131 conn(0x556b3eaadde0 msgr2=0x556b3eaae1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:56.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.736+0000 7f8a17b89740 1 -- 172.21.15.16:0/3762949131 shutdown_connections 2024-09-12T02:05:56.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.736+0000 7f8a17b89740 1 -- 172.21.15.16:0/3762949131 wait complete. 2024-09-12T02:05:56.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.736+0000 7f8a17b89740 1 -- start start 2024-09-12T02:05:56.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.736+0000 7f8a17b89740 1 Processor -- start 2024-09-12T02:05:56.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.737+0000 7f8a17b89740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556b3eab1850 0x556b3eab98e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:56.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.737+0000 7f8a17b89740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556b3eab2170 0x556b3eab9e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:56.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.737+0000 7f8a17b89740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b3eabca30 0x556b3eaba360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:56.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.737+0000 7f8a17b89740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x556b3eabf600 con 0x556b3eab1850 2024-09-12T02:05:56.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.737+0000 7f8a17b89740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x556b3e99e900 con 0x556b3eabca30 2024-09-12T02:05:56.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.737+0000 7f8a17b89740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x556b3eabf780 con 0x556b3eab2170 2024-09-12T02:05:56.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.737+0000 7f8a17b89740 10 client.0 _ll_register_callbacks cb 0x556b3e99c780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:05:56.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.737+0000 7f8a0effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556b3eab1850 0x556b3eab98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:56.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.737+0000 7f8a07fff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556b3eab2170 0x556b3eab9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:56.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.737+0000 7f8a0effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556b3eab1850 0x556b3eab98e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54344/0 (socket says 172.21.15.16:54344) 2024-09-12T02:05:56.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.737+0000 7f8a0effd640 1 -- 172.21.15.16:0/3099012484 learned_addr learned my addr 172.21.15.16:0/3099012484 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:05:56.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.737+0000 7f8a0f7fe640 1 --2- 172.21.15.16:0/3099012484 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b3eabca30 0x556b3eaba360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:56.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.737+0000 7f8a0effd640 1 -- 172.21.15.16:0/3099012484 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556b3eab2170 msgr2=0x556b3eab9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:56.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.737+0000 7f8a0effd640 1 --2- 172.21.15.16:0/3099012484 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556b3eab2170 0x556b3eab9e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:56.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.737+0000 7f8a0effd640 1 -- 172.21.15.16:0/3099012484 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b3eabca30 msgr2=0x556b3eaba360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:56.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.737+0000 7f8a0effd640 1 --2- 172.21.15.16:0/3099012484 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b3eabca30 0x556b3eaba360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:56.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.737+0000 7f8a0effd640 1 -- 172.21.15.16:0/3099012484 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b3e9dec50 con 0x556b3eab1850 2024-09-12T02:05:56.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.737+0000 7f8a0f7fe640 1 --2- 172.21.15.16:0/3099012484 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b3eabca30 0x556b3eaba360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:05:56.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.737+0000 7f8a0effd640 1 --2- 172.21.15.16:0/3099012484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556b3eab1850 0x556b3eab98e0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f8a00010ca0 tx=0x7f8a000058e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4df911227dc4f20d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:56.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.737+0000 7f89f67fc640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:05:56.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.738+0000 7f89f67fc640 1 -- 172.21.15.16:0/3099012484 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a00029060 con 0x556b3eab1850 2024-09-12T02:05:56.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.738+0000 7f89f67fc640 1 -- 172.21.15.16:0/3099012484 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f8a00004380 con 0x556b3eab1850 2024-09-12T02:05:56.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.738+0000 7f8a17b89740 1 -- 172.21.15.16:0/3099012484 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a000041d0 con 0x556b3eab1850 2024-09-12T02:05:56.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.738+0000 7f89f67fc640 1 -- 172.21.15.16:0/3099012484 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f8a000035f0 con 0x556b3eab1850 2024-09-12T02:05:56.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.738+0000 7f8a17b89740 10 client.16689 fetch_fsmap learned FSMap version 25 2024-09-12T02:05:56.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.738+0000 7f8a17b89740 1 -- 172.21.15.16:0/3099012484 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x556b3eacb220 con 0x556b3eab1850 2024-09-12T02:05:56.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.738+0000 7f89f67fc640 1 -- 172.21.15.16:0/3099012484 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a000041d0 con 0x556b3eab1850 2024-09-12T02:05:56.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.738+0000 7f89f67fc640 1 -- 172.21.15.16:0/3099012484 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f8a0002e070 con 0x556b3eab1850 2024-09-12T02:05:56.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.738+0000 7f8a17b89740 10 client.16689 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:05:56.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.738+0000 7f8a17b89740 10 client.16689 resolve_mds: validated gid 24482 aka daemon mds.b 2024-09-12T02:05:56.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.738+0000 7f8a17b89740 20 client.16689 populate_metadata read hostname 'smithi016' 2024-09-12T02:05:56.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.738+0000 7f8a17b89740 1 --2- 172.21.15.16:0/3099012484 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x556b3eacc5d0 0x556b3eace9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:56.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.738+0000 7f8a17b89740 4 client.16689 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:05:56.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.738+0000 7f8a17b89740 1 -- 172.21.15.16:0/3099012484 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556b3ea23890 con 0x556b3eacc5d0 2024-09-12T02:05:56.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.739+0000 7f8a07fff640 1 --2- 172.21.15.16:0/3099012484 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x556b3eacc5d0 0x556b3eace9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:56.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.739+0000 7f89f67fc640 10 client.16689 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:05:56.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.739+0000 7f8a07fff640 1 --2- 172.21.15.16:0/3099012484 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x556b3eacc5d0 0x556b3eace9c0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=56d1eed805e8f217 server_cookie=efbe917a518b84f7 in_seq=0 out_seq=0 2024-09-12T02:05:56.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.740+0000 7f89f67fc640 1 -- 172.21.15.16:0/3099012484 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556b3ea23890 con 0x556b3eacc5d0 2024-09-12T02:05:56.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.740+0000 7f89f67fc640 10 client.16689 handle_command_reply: tid=0 2024-09-12T02:05:56.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.740+0000 7f8a17b89740 1 client.16689 shutdown 2024-09-12T02:05:56.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.740+0000 7f8a17b89740 1 -- 172.21.15.16:0/3099012484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556b3eab1850 msgr2=0x556b3eab98e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:56.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.740+0000 7f8a17b89740 1 --2- 172.21.15.16:0/3099012484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556b3eab1850 0x556b3eab98e0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f8a00010ca0 tx=0x7f8a000058e0 comp rx=0 tx=0).stop 2024-09-12T02:05:56.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.740+0000 7f8a17b89740 1 -- 172.21.15.16:0/3099012484 shutdown_connections 2024-09-12T02:05:56.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.740+0000 7f8a17b89740 1 --2- 172.21.15.16:0/3099012484 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x556b3eacc5d0 0x556b3eace9c0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:56.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.740+0000 7f8a17b89740 1 --2- 172.21.15.16:0/3099012484 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b3eabca30 0x556b3eaba360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:56.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.740+0000 7f8a17b89740 1 --2- 172.21.15.16:0/3099012484 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556b3eab2170 0x556b3eab9e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:56.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.740+0000 7f8a17b89740 1 --2- 172.21.15.16:0/3099012484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556b3eab1850 0x556b3eab98e0 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:56.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.740+0000 7f89f67fc640 0 client.16689 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:05:56.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.740+0000 7f8a17b89740 1 -- 172.21.15.16:0/3099012484 >> 172.21.15.16:0/3099012484 conn(0x556b3eaadde0 msgr2=0x556b3eab3ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:56.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.740+0000 7f8a17b89740 1 -- 172.21.15.16:0/3099012484 shutdown_connections 2024-09-12T02:05:56.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.740+0000 7f8a17b89740 1 -- 172.21.15.16:0/3099012484 wait complete. 2024-09-12T02:05:56.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.740+0000 7f8a17b89740 20 client.16689 trim_cache size 0 max 16384 2024-09-12T02:05:56.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.757+0000 7f8a17b89740 1 Processor -- start 2024-09-12T02:05:56.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.757+0000 7f8a17b89740 1 -- start start 2024-09-12T02:05:56.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.758+0000 7f8a17b89740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556b3eab1850 0x556b3eab42b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:56.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.758+0000 7f8a17b89740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b3eab2170 0x556b3eaacdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:56.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.758+0000 7f8a17b89740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556b3eabca30 0x556b3eaad2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:56.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.758+0000 7f8a17b89740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8a101b1f40 con 0x556b3eab1850 2024-09-12T02:05:56.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.758+0000 7f8a17b89740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8a101af810 con 0x556b3eab2170 2024-09-12T02:05:56.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.758+0000 7f8a17b89740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8a101ad660 con 0x556b3eabca30 2024-09-12T02:05:56.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.758+0000 7f8a0effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556b3eab1850 0x556b3eab42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:56.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.758+0000 7f8a0effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556b3eab1850 0x556b3eab42b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54346/0 (socket says 172.21.15.16:54346) 2024-09-12T02:05:56.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.758+0000 7f8a0effd640 1 -- 172.21.15.16:0/3037417366 learned_addr learned my addr 172.21.15.16:0/3037417366 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:05:56.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.758+0000 7f8a07fff640 1 --2- 172.21.15.16:0/3037417366 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b3eab2170 0x556b3eaacdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:56.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.758+0000 7f8a0effd640 1 -- 172.21.15.16:0/3037417366 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556b3eabca30 msgr2=0x556b3eaad2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:56.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.758+0000 7f8a0effd640 1 --2- 172.21.15.16:0/3037417366 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556b3eabca30 0x556b3eaad2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:56.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.758+0000 7f8a0effd640 1 -- 172.21.15.16:0/3037417366 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b3eab2170 msgr2=0x556b3eaacdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:56.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.758+0000 7f8a0effd640 1 --2- 172.21.15.16:0/3037417366 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b3eab2170 0x556b3eaacdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:56.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.758+0000 7f8a0effd640 1 -- 172.21.15.16:0/3037417366 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b3eaadc10 con 0x556b3eab1850 2024-09-12T02:05:56.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.758+0000 7f8a0effd640 1 --2- 172.21.15.16:0/3037417366 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556b3eab1850 0x556b3eab42b0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f8a0000c010 tx=0x7f8a00003e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22e559fdf35962de server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:56.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.759+0000 7f8a04ff9640 1 -- 172.21.15.16:0/3037417366 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a00034070 con 0x556b3eab1850 2024-09-12T02:05:56.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.759+0000 7f8a04ff9640 1 -- 172.21.15.16:0/3037417366 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a0000b8e0 con 0x556b3eab1850 2024-09-12T02:05:56.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.759+0000 7f8a04ff9640 1 -- 172.21.15.16:0/3037417366 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a00005080 con 0x556b3eab1850 2024-09-12T02:05:56.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.759+0000 7f8a17b89740 1 -- 172.21.15.16:0/3037417366 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556b3eab1850 msgr2=0x556b3eab42b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:56.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.759+0000 7f8a17b89740 1 --2- 172.21.15.16:0/3037417366 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556b3eab1850 0x556b3eab42b0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f8a0000c010 tx=0x7f8a00003e70 comp rx=0 tx=0).stop 2024-09-12T02:05:56.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.759+0000 7f8a17b89740 1 -- 172.21.15.16:0/3037417366 shutdown_connections 2024-09-12T02:05:56.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.759+0000 7f8a17b89740 1 --2- 172.21.15.16:0/3037417366 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556b3eabca30 0x556b3eaad2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:56.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.759+0000 7f8a17b89740 1 --2- 172.21.15.16:0/3037417366 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b3eab2170 0x556b3eaacdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:56.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.759+0000 7f8a17b89740 1 --2- 172.21.15.16:0/3037417366 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556b3eab1850 0x556b3eab42b0 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:56.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.759+0000 7f8a17b89740 1 -- 172.21.15.16:0/3037417366 >> 172.21.15.16:0/3037417366 conn(0x556b3eaadde0 msgr2=0x556b3eaac6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:56.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.760+0000 7f8a17b89740 1 -- 172.21.15.16:0/3037417366 shutdown_connections 2024-09-12T02:05:56.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.760+0000 7f8a17b89740 1 -- 172.21.15.16:0/3037417366 wait complete. 2024-09-12T02:05:56.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.760+0000 7f8a17b89740 1 -- start start 2024-09-12T02:05:56.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.760+0000 7f8a17b89740 1 Processor -- start 2024-09-12T02:05:56.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.760+0000 7f8a17b89740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b3eab1850 0x556b3eacf9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:56.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.760+0000 7f8a17b89740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556b3eab2170 0x556b3eacfef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:56.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.760+0000 7f8a17b89740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556b3eabca30 0x556b3ead0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:56.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.760+0000 7f8a17b89740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8a101adcd0 con 0x556b3eabca30 2024-09-12T02:05:56.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.760+0000 7f8a17b89740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8a101af140 con 0x556b3eab1850 2024-09-12T02:05:56.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.760+0000 7f8a17b89740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8a101ae3a0 con 0x556b3eab2170 2024-09-12T02:05:56.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.760+0000 7f8a17b89740 10 client.0 _ll_register_callbacks cb 0x7f8a0000bb60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:05:56.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.760+0000 7f8a0f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556b3eabca30 0x556b3ead0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:56.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.760+0000 7f8a0effd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b3eab1850 0x556b3eacf9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:56.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.761+0000 7f8a0f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556b3eabca30 0x556b3ead0430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54362/0 (socket says 172.21.15.16:54362) 2024-09-12T02:05:56.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.761+0000 7f8a0effd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b3eab1850 0x556b3eacf9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.16:40100/0 (socket says 172.21.15.16:40100) 2024-09-12T02:05:56.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.761+0000 7f8a0f7fe640 1 -- 172.21.15.16:0/2343179721 learned_addr learned my addr 172.21.15.16:0/2343179721 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:05:56.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.761+0000 7f8a07fff640 1 --2- 172.21.15.16:0/2343179721 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556b3eab2170 0x556b3eacfef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:56.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.761+0000 7f8a0f7fe640 1 -- 172.21.15.16:0/2343179721 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556b3eab2170 msgr2=0x556b3eacfef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:56.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.761+0000 7f8a0f7fe640 1 --2- 172.21.15.16:0/2343179721 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556b3eab2170 0x556b3eacfef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:56.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.761+0000 7f8a0f7fe640 1 -- 172.21.15.16:0/2343179721 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b3eab1850 msgr2=0x556b3eacf9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:56.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.761+0000 7f8a0f7fe640 1 --2- 172.21.15.16:0/2343179721 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b3eab1850 0x556b3eacf9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:56.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.761+0000 7f8a0f7fe640 1 -- 172.21.15.16:0/2343179721 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b3eaa93f0 con 0x556b3eabca30 2024-09-12T02:05:56.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.761+0000 7f8a0effd640 1 --2- 172.21.15.16:0/2343179721 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b3eab1850 0x556b3eacf9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:05:56.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.761+0000 7f8a0f7fe640 1 --2- 172.21.15.16:0/2343179721 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556b3eabca30 0x556b3ead0430 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f8a0802ff80 tx=0x7f8a0802ffb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d00e783a0bb7d4ab server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:56.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.761+0000 7f89f67fc640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:05:56.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.761+0000 7f89f67fc640 1 -- 172.21.15.16:0/2343179721 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a08034040 con 0x556b3eabca30 2024-09-12T02:05:56.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.761+0000 7f89f67fc640 1 -- 172.21.15.16:0/2343179721 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f8a08053440 con 0x556b3eabca30 2024-09-12T02:05:56.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.761+0000 7f8a17b89740 1 -- 172.21.15.16:0/2343179721 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a0000b8e0 con 0x556b3eabca30 2024-09-12T02:05:56.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.762+0000 7f89f67fc640 1 -- 172.21.15.16:0/2343179721 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f8a08053640 con 0x556b3eabca30 2024-09-12T02:05:56.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.762+0000 7f8a17b89740 10 client.16695 fetch_fsmap learned FSMap version 25 2024-09-12T02:05:56.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.762+0000 7f8a17b89740 1 -- 172.21.15.16:0/2343179721 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x556b3eb37250 con 0x556b3eabca30 2024-09-12T02:05:56.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.762+0000 7f89f67fc640 1 -- 172.21.15.16:0/2343179721 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a0000b8e0 con 0x556b3eabca30 2024-09-12T02:05:56.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.762+0000 7f89f67fc640 1 -- 172.21.15.16:0/2343179721 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f8a08051020 con 0x556b3eabca30 2024-09-12T02:05:56.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.762+0000 7f8a17b89740 10 client.16695 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:05:56.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.762+0000 7f8a17b89740 10 client.16695 resolve_mds: validated gid 24482 aka daemon mds.b 2024-09-12T02:05:56.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.762+0000 7f8a17b89740 20 client.16695 populate_metadata read hostname 'smithi016' 2024-09-12T02:05:56.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.762+0000 7f8a17b89740 1 --2- 172.21.15.16:0/2343179721 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x556b3eb37a40 0x556b3eb39e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:56.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.762+0000 7f8a17b89740 4 client.16695 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.15, "await_for": 45.0}] 2024-09-12T02:05:56.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.762+0000 7f8a17b89740 1 -- 172.21.15.16:0/2343179721 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.15, "await_for": 45.0}) -- 0x556b3ea1dbe0 con 0x556b3eb37a40 2024-09-12T02:05:56.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.762+0000 7f8a0effd640 1 --2- 172.21.15.16:0/2343179721 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x556b3eb37a40 0x556b3eb39e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:56.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.763+0000 7f89f67fc640 10 client.16695 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:05:56.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.763+0000 7f8a0effd640 1 --2- 172.21.15.16:0/2343179721 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x556b3eb37a40 0x556b3eb39e60 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=87124681e55f3dac server_cookie=b9f248ce3b4d831e in_seq=0 out_seq=0 2024-09-12T02:05:56.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.779+0000 7f89f67fc640 1 -- 172.21.15.16:0/2343179721 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x556b3ea1dbe0 con 0x556b3eb37a40 2024-09-12T02:05:56.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.779+0000 7f89f67fc640 10 client.16695 handle_command_reply: tid=0 2024-09-12T02:05:56.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.779+0000 7f8a17b89740 1 client.16695 shutdown 2024-09-12T02:05:56.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.779+0000 7f8a17b89740 1 -- 172.21.15.16:0/2343179721 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556b3eabca30 msgr2=0x556b3ead0430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:56.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.779+0000 7f8a17b89740 1 --2- 172.21.15.16:0/2343179721 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556b3eabca30 0x556b3ead0430 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f8a0802ff80 tx=0x7f8a0802ffb0 comp rx=0 tx=0).stop 2024-09-12T02:05:56.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.779+0000 7f8a17b89740 1 -- 172.21.15.16:0/2343179721 shutdown_connections 2024-09-12T02:05:56.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.779+0000 7f8a17b89740 1 --2- 172.21.15.16:0/2343179721 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x556b3eb37a40 0x556b3eb39e60 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:56.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.779+0000 7f89f67fc640 0 client.16695 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:05:56.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.779+0000 7f8a17b89740 1 --2- 172.21.15.16:0/2343179721 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556b3eabca30 0x556b3ead0430 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:56.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.779+0000 7f8a17b89740 1 --2- 172.21.15.16:0/2343179721 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556b3eab2170 0x556b3eacfef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:56.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.779+0000 7f8a17b89740 1 --2- 172.21.15.16:0/2343179721 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b3eab1850 0x556b3eacf9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:56.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.779+0000 7f8a17b89740 1 -- 172.21.15.16:0/2343179721 >> 172.21.15.16:0/2343179721 conn(0x556b3eaadde0 msgr2=0x556b3eaba0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:56.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.779+0000 7f8a17b89740 1 -- 172.21.15.16:0/2343179721 shutdown_connections 2024-09-12T02:05:56.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.779+0000 7f8a17b89740 1 -- 172.21.15.16:0/2343179721 wait complete. 2024-09-12T02:05:56.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.779+0000 7f8a17b89740 20 client.16695 trim_cache size 0 max 16384 2024-09-12T02:05:56.781 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T02:05:56.781 INFO:teuthology.orchestra.run.smithi016.stdout: "epoch": 25, 2024-09-12T02:05:56.782 INFO:teuthology.orchestra.run.smithi016.stdout: "leader": 24482, 2024-09-12T02:05:56.782 INFO:teuthology.orchestra.run.smithi016.stdout: "set_version": 18, 2024-09-12T02:05:56.782 INFO:teuthology.orchestra.run.smithi016.stdout: "sets": { 2024-09-12T02:05:56.782 INFO:teuthology.orchestra.run.smithi016.stdout: "ee2e48a6": { 2024-09-12T02:05:56.782 INFO:teuthology.orchestra.run.smithi016.stdout: "version": 18, 2024-09-12T02:05:56.782 INFO:teuthology.orchestra.run.smithi016.stdout: "age_ref": 0.0, 2024-09-12T02:05:56.782 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-12T02:05:56.782 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "QUIESCED", 2024-09-12T02:05:56.782 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-12T02:05:56.782 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:05:56.782 INFO:teuthology.orchestra.run.smithi016.stdout: "timeout": 90.0, 2024-09-12T02:05:56.782 INFO:teuthology.orchestra.run.smithi016.stdout: "expiration": 140.2, 2024-09-12T02:05:56.782 INFO:teuthology.orchestra.run.smithi016.stdout: "members": { 2024-09-12T02:05:56.782 INFO:teuthology.orchestra.run.smithi016.stdout: "file:/": { 2024-09-12T02:05:56.783 INFO:teuthology.orchestra.run.smithi016.stdout: "excluded": false, 2024-09-12T02:05:56.783 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-12T02:05:56.783 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "QUIESCED", 2024-09-12T02:05:56.783 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-12T02:05:56.783 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:05:56.783 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:05:56.783 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:05:56.783 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:05:56.783 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:05:56.783 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T02:05:56.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.780+0000 7f8a157b0640 1 -- 172.21.15.16:0/2849828008 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f89fc0a2b80 msgr2=0x7f89fc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:56.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.780+0000 7f8a157b0640 1 --2- 172.21.15.16:0/2849828008 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f89fc0a2b80 0x7f89fc0a5040 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f8a08005cc0 tx=0x7f8a08005c50 comp rx=0 tx=0).stop 2024-09-12T02:05:56.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.780+0000 7f8a157b0640 1 -- 172.21.15.16:0/2849828008 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a10103660 msgr2=0x7f8a10100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:56.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.780+0000 7f8a157b0640 1 --2- 172.21.15.16:0/2849828008 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a10103660 0x7f8a10100ee0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f89f800e9f0 tx=0x7f89f800eeb0 comp rx=0 tx=0).stop 2024-09-12T02:05:56.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.780+0000 7f8a157b0640 1 -- 172.21.15.16:0/2849828008 shutdown_connections 2024-09-12T02:05:56.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.780+0000 7f8a157b0640 1 --2- 172.21.15.16:0/2849828008 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f89fc0a2b80 0x7f89fc0a5040 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:56.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.780+0000 7f8a157b0640 1 --2- 172.21.15.16:0/2849828008 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a1010c170 0x7f8a10101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:56.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.780+0000 7f8a157b0640 1 --2- 172.21.15.16:0/2849828008 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a10103660 0x7f8a10100ee0 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:56.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.780+0000 7f8a157b0640 1 --2- 172.21.15.16:0/2849828008 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a10102d40 0x7f8a101009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:56.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.780+0000 7f8a157b0640 1 -- 172.21.15.16:0/2849828008 >> 172.21.15.16:0/2849828008 conn(0x7f8a100fc7a0 msgr2=0x7f8a100fdec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:56.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.780+0000 7f8a157b0640 1 -- 172.21.15.16:0/2849828008 shutdown_connections 2024-09-12T02:05:56.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:56.780+0000 7f8a157b0640 1 -- 172.21.15.16:0/2849828008 wait complete. 2024-09-12T02:05:57.190 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-12T02:05:57.190 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-12T02:05:57.531 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'ee2e48a6', quiesce took 4.7 seconds. Will release after: 15.45 2024-09-12T02:05:57.584 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:05:58.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.056+0000 7f9462e86640 1 -- 172.21.15.16:0/2585211220 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f945c102570 msgr2=0x7f945c1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:58.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.056+0000 7f9462e86640 1 --2- 172.21.15.16:0/2585211220 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f945c102570 0x7f945c1029b0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f9444009900 tx=0x7f944402a480 comp rx=0 tx=0).stop 2024-09-12T02:05:58.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.056+0000 7f9462e86640 1 -- 172.21.15.16:0/2585211220 shutdown_connections 2024-09-12T02:05:58.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.056+0000 7f9462e86640 1 --2- 172.21.15.16:0/2585211220 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f945c102ef0 0x7f945c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:58.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.056+0000 7f9462e86640 1 --2- 172.21.15.16:0/2585211220 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f945c102570 0x7f945c1029b0 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:58.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.056+0000 7f9462e86640 1 --2- 172.21.15.16:0/2585211220 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f945c100e20 0x7f945c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:58.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.056+0000 7f9462e86640 1 -- 172.21.15.16:0/2585211220 >> 172.21.15.16:0/2585211220 conn(0x7f945c0fc7d0 msgr2=0x7f945c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:58.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.056+0000 7f9462e86640 1 -- 172.21.15.16:0/2585211220 shutdown_connections 2024-09-12T02:05:58.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.056+0000 7f9462e86640 1 -- 172.21.15.16:0/2585211220 wait complete. 2024-09-12T02:05:58.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.057+0000 7f9462e86640 1 Processor -- start 2024-09-12T02:05:58.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.057+0000 7f9462e86640 1 -- start start 2024-09-12T02:05:58.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.057+0000 7f9462e86640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f945c100e20 0x7f945c19a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:58.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.057+0000 7f9462e86640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f945c102570 0x7f945c19ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:58.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.057+0000 7f9462e86640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f945c102ef0 0x7f945c19dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:58.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.057+0000 7f9462e86640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f945c110fa0 con 0x7f945c102570 2024-09-12T02:05:58.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.057+0000 7f9462e86640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f945c110e20 con 0x7f945c102ef0 2024-09-12T02:05:58.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.057+0000 7f9462e86640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f945c111120 con 0x7f945c100e20 2024-09-12T02:05:58.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.057+0000 7f945bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f945c102570 0x7f945c19ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:58.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.057+0000 7f945bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f945c102570 0x7f945c19ace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54378/0 (socket says 172.21.15.16:54378) 2024-09-12T02:05:58.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.057+0000 7f945bfff640 1 -- 172.21.15.16:0/1113806111 learned_addr learned my addr 172.21.15.16:0/1113806111 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:05:58.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.058+0000 7f9461423640 1 --2- 172.21.15.16:0/1113806111 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f945c102ef0 0x7f945c19dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:58.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.058+0000 7f9460c22640 1 --2- 172.21.15.16:0/1113806111 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f945c100e20 0x7f945c19a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:58.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.058+0000 7f945bfff640 1 -- 172.21.15.16:0/1113806111 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f945c100e20 msgr2=0x7f945c19a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:58.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.058+0000 7f945bfff640 1 --2- 172.21.15.16:0/1113806111 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f945c100e20 0x7f945c19a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:58.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.058+0000 7f945bfff640 1 -- 172.21.15.16:0/1113806111 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f945c102ef0 msgr2=0x7f945c19dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:58.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.058+0000 7f945bfff640 1 --2- 172.21.15.16:0/1113806111 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f945c102ef0 0x7f945c19dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:58.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.058+0000 7f945bfff640 1 -- 172.21.15.16:0/1113806111 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f945c19e6f0 con 0x7f945c102570 2024-09-12T02:05:58.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.058+0000 7f9461423640 1 --2- 172.21.15.16:0/1113806111 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f945c102ef0 0x7f945c19dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:05:58.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.058+0000 7f9460c22640 1 --2- 172.21.15.16:0/1113806111 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f945c100e20 0x7f945c19a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:05:58.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.058+0000 7f945bfff640 1 --2- 172.21.15.16:0/1113806111 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f945c102570 0x7f945c19ace0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f9444006010 tx=0x7f9444003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:58.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.058+0000 7f9459ffb640 1 -- 172.21.15.16:0/1113806111 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f944403f070 con 0x7f945c102570 2024-09-12T02:05:58.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.058+0000 7f9459ffb640 1 -- 172.21.15.16:0/1113806111 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9444031c30 con 0x7f945c102570 2024-09-12T02:05:58.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.058+0000 7f9462e86640 1 -- 172.21.15.16:0/1113806111 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f945c19e980 con 0x7f945c102570 2024-09-12T02:05:58.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.058+0000 7f9459ffb640 1 -- 172.21.15.16:0/1113806111 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9444035030 con 0x7f945c102570 2024-09-12T02:05:58.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.059+0000 7f9462e86640 1 -- 172.21.15.16:0/1113806111 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f945c1a7350 con 0x7f945c102570 2024-09-12T02:05:58.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.061+0000 7f9459ffb640 1 -- 172.21.15.16:0/1113806111 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f9444042070 con 0x7f945c102570 2024-09-12T02:05:58.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.061+0000 7f946525f740 1 Processor -- start 2024-09-12T02:05:58.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.061+0000 7f946525f740 1 -- start start 2024-09-12T02:05:58.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.062+0000 7f946525f740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e3b8fe7b60 0x55e3b8fe7f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:58.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.062+0000 7f946525f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e3b8fe8480 0x55e3b8ff2800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:58.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.062+0000 7f946525f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e3b8ff2d40 0x55e3b8ff5130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:58.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.062+0000 7f946525f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e3b8f4d280 con 0x55e3b8fe8480 2024-09-12T02:05:58.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.062+0000 7f946525f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55e3b8ef4b10 con 0x55e3b8ff2d40 2024-09-12T02:05:58.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.062+0000 7f946525f740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55e3b8f49240 con 0x55e3b8fe7b60 2024-09-12T02:05:58.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.062+0000 7f9460c22640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e3b8fe8480 0x55e3b8ff2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:58.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.062+0000 7f9460c22640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e3b8fe8480 0x55e3b8ff2800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54388/0 (socket says 172.21.15.16:54388) 2024-09-12T02:05:58.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.062+0000 7f9460c22640 1 -- 172.21.15.16:0/1309725482 learned_addr learned my addr 172.21.15.16:0/1309725482 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:05:58.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.062+0000 7f945bfff640 1 --2- 172.21.15.16:0/1309725482 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e3b8ff2d40 0x55e3b8ff5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:58.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.062+0000 7f9460c22640 1 -- 172.21.15.16:0/1309725482 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e3b8fe7b60 msgr2=0x55e3b8fe7f40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T02:05:58.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.062+0000 7f9460c22640 1 --2- 172.21.15.16:0/1309725482 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e3b8fe7b60 0x55e3b8fe7f40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:58.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.062+0000 7f9460c22640 1 -- 172.21.15.16:0/1309725482 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e3b8ff2d40 msgr2=0x55e3b8ff5130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:58.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.062+0000 7f9459ffb640 1 --2- 172.21.15.16:0/1113806111 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f94480a2b80 0x7f94480a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:58.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.062+0000 7f9460c22640 1 --2- 172.21.15.16:0/1309725482 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e3b8ff2d40 0x55e3b8ff5130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:58.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.062+0000 7f9460c22640 1 -- 172.21.15.16:0/1309725482 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e3b8f011f0 con 0x55e3b8fe8480 2024-09-12T02:05:58.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.062+0000 7f945bfff640 1 --2- 172.21.15.16:0/1309725482 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e3b8ff2d40 0x55e3b8ff5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:05:58.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.062+0000 7f9460c22640 1 --2- 172.21.15.16:0/1309725482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e3b8fe8480 0x55e3b8ff2800 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f944c00bd40 tx=0x7f944c00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3410aa0b13d580e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:58.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.062+0000 7f9459ffb640 1 -- 172.21.15.16:0/1113806111 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f94440ef8c0 con 0x7f945c102570 2024-09-12T02:05:58.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.063+0000 7f9461423640 1 --2- 172.21.15.16:0/1113806111 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f94480a2b80 0x7f94480a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:58.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.063+0000 7f9452ffd640 1 -- 172.21.15.16:0/1309725482 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f944c015070 con 0x55e3b8fe8480 2024-09-12T02:05:58.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.063+0000 7f9452ffd640 1 -- 172.21.15.16:0/1309725482 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f944c00be30 con 0x55e3b8fe8480 2024-09-12T02:05:58.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.063+0000 7f9452ffd640 1 -- 172.21.15.16:0/1309725482 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f944c011a00 con 0x55e3b8fe8480 2024-09-12T02:05:58.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.064+0000 7f946525f740 1 -- 172.21.15.16:0/1309725482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e3b8fe8480 msgr2=0x55e3b8ff2800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:58.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.064+0000 7f9461423640 1 --2- 172.21.15.16:0/1113806111 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f94480a2b80 0x7f94480a5040 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f9454009ea0 tx=0x7f94540094f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:58.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.064+0000 7f946525f740 1 --2- 172.21.15.16:0/1309725482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e3b8fe8480 0x55e3b8ff2800 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f944c00bd40 tx=0x7f944c00f730 comp rx=0 tx=0).stop 2024-09-12T02:05:58.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.064+0000 7f946525f740 1 -- 172.21.15.16:0/1309725482 shutdown_connections 2024-09-12T02:05:58.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.064+0000 7f946525f740 1 --2- 172.21.15.16:0/1309725482 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e3b8ff2d40 0x55e3b8ff5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:58.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.064+0000 7f946525f740 1 --2- 172.21.15.16:0/1309725482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e3b8fe8480 0x55e3b8ff2800 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:58.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.064+0000 7f946525f740 1 --2- 172.21.15.16:0/1309725482 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e3b8fe7b60 0x55e3b8fe7f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:58.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.064+0000 7f946525f740 1 -- 172.21.15.16:0/1309725482 >> 172.21.15.16:0/1309725482 conn(0x55e3b8fe40f0 msgr2=0x55e3b8fe44d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:58.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.064+0000 7f946525f740 1 -- 172.21.15.16:0/1309725482 shutdown_connections 2024-09-12T02:05:58.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.064+0000 7f946525f740 1 -- 172.21.15.16:0/1309725482 wait complete. 2024-09-12T02:05:58.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.064+0000 7f946525f740 1 -- start start 2024-09-12T02:05:58.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.064+0000 7f946525f740 1 Processor -- start 2024-09-12T02:05:58.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.065+0000 7f946525f740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e3b8fe7b60 0x55e3b8fefb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:58.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.065+0000 7f946525f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e3b8fe8480 0x55e3b8ff00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:58.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.065+0000 7f946525f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e3b8ff2d40 0x55e3b8ff0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:58.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.065+0000 7f946525f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e3b8ff5910 con 0x55e3b8ff2d40 2024-09-12T02:05:58.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.065+0000 7f946525f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55e3b8f07f10 con 0x55e3b8fe8480 2024-09-12T02:05:58.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.065+0000 7f946525f740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55e3b8ff5a90 con 0x55e3b8fe7b60 2024-09-12T02:05:58.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.065+0000 7f946525f740 10 client.0 _ll_register_callbacks cb 0x55e3b8ed0a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:05:58.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.065+0000 7f9461423640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e3b8ff2d40 0x55e3b8ff0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:58.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.065+0000 7f9460c22640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e3b8fe7b60 0x55e3b8fefb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:58.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.065+0000 7f9461423640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e3b8ff2d40 0x55e3b8ff0600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54396/0 (socket says 172.21.15.16:54396) 2024-09-12T02:05:58.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.065+0000 7f9461423640 1 -- 172.21.15.16:0/2272144798 learned_addr learned my addr 172.21.15.16:0/2272144798 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:05:58.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.065+0000 7f945bfff640 1 --2- 172.21.15.16:0/2272144798 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e3b8fe8480 0x55e3b8ff00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:58.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.065+0000 7f9460c22640 1 -- 172.21.15.16:0/2272144798 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e3b8fe8480 msgr2=0x55e3b8ff00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:58.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.065+0000 7f9460c22640 1 --2- 172.21.15.16:0/2272144798 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e3b8fe8480 0x55e3b8ff00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:58.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.065+0000 7f9460c22640 1 -- 172.21.15.16:0/2272144798 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e3b8ff2d40 msgr2=0x55e3b8ff0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:58.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.065+0000 7f9460c22640 1 --2- 172.21.15.16:0/2272144798 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e3b8ff2d40 0x55e3b8ff0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:58.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.065+0000 7f9460c22640 1 -- 172.21.15.16:0/2272144798 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e3b8cbe6a0 con 0x55e3b8fe7b60 2024-09-12T02:05:58.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.065+0000 7f945bfff640 1 --2- 172.21.15.16:0/2272144798 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e3b8fe8480 0x55e3b8ff00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:05:58.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.066+0000 7f9460c22640 1 --2- 172.21.15.16:0/2272144798 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e3b8fe7b60 0x55e3b8fefb80 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f944c00bd40 tx=0x7f944c005270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=69780086e4fa45f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:58.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.066+0000 7f943bfff640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-09-12T02:05:58.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.066+0000 7f943bfff640 1 -- 172.21.15.16:0/2272144798 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f944c028060 con 0x55e3b8fe7b60 2024-09-12T02:05:58.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.066+0000 7f946525f740 1 -- 172.21.15.16:0/2272144798 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f944c00be30 con 0x55e3b8fe7b60 2024-09-12T02:05:58.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.066+0000 7f943bfff640 1 -- 172.21.15.16:0/2272144798 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f944c011780 con 0x55e3b8fe7b60 2024-09-12T02:05:58.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.066+0000 7f943bfff640 1 -- 172.21.15.16:0/2272144798 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f944c015070 con 0x55e3b8fe7b60 2024-09-12T02:05:58.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.066+0000 7f946525f740 10 client.26077 fetch_fsmap learned FSMap version 25 2024-09-12T02:05:58.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.066+0000 7f946525f740 1 -- 172.21.15.16:0/2272144798 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e3b90014b0 con 0x55e3b8fe7b60 2024-09-12T02:05:58.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.066+0000 7f943bfff640 1 -- 172.21.15.16:0/2272144798 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f944c00be30 con 0x55e3b8fe7b60 2024-09-12T02:05:58.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.066+0000 7f943bfff640 1 -- 172.21.15.16:0/2272144798 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f944c02d070 con 0x55e3b8fe7b60 2024-09-12T02:05:58.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.067+0000 7f946525f740 10 client.26077 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:05:58.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.067+0000 7f946525f740 10 client.26077 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:05:58.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.067+0000 7f946525f740 20 client.26077 populate_metadata read hostname 'smithi016' 2024-09-12T02:05:58.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.067+0000 7f946525f740 1 --2- 172.21.15.16:0/2272144798 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55e3b90028c0 0x55e3b9004cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:58.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.067+0000 7f946525f740 4 client.26077 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:05:58.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.067+0000 7f946525f740 1 -- 172.21.15.16:0/2272144798 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e3b8f5a7f0 con 0x55e3b90028c0 2024-09-12T02:05:58.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.067+0000 7f945bfff640 1 --2- 172.21.15.16:0/2272144798 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55e3b90028c0 0x55e3b9004cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:58.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.067+0000 7f943bfff640 10 client.26077 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:05:58.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.067+0000 7f945bfff640 1 --2- 172.21.15.16:0/2272144798 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55e3b90028c0 0x55e3b9004cb0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb52147aa637a9e4 server_cookie=72473286f7716d7d in_seq=0 out_seq=0 2024-09-12T02:05:58.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.068+0000 7f943bfff640 1 -- 172.21.15.16:0/2272144798 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e3b8f5a7f0 con 0x55e3b90028c0 2024-09-12T02:05:58.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.068+0000 7f943bfff640 10 client.26077 handle_command_reply: tid=0 2024-09-12T02:05:58.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.068+0000 7f946525f740 1 client.26077 shutdown 2024-09-12T02:05:58.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.068+0000 7f946525f740 1 -- 172.21.15.16:0/2272144798 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e3b8fe7b60 msgr2=0x55e3b8fefb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:58.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.068+0000 7f946525f740 1 --2- 172.21.15.16:0/2272144798 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e3b8fe7b60 0x55e3b8fefb80 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f944c00bd40 tx=0x7f944c005270 comp rx=0 tx=0).stop 2024-09-12T02:05:58.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.068+0000 7f946525f740 1 -- 172.21.15.16:0/2272144798 shutdown_connections 2024-09-12T02:05:58.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.068+0000 7f946525f740 1 --2- 172.21.15.16:0/2272144798 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55e3b90028c0 0x55e3b9004cb0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:58.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.068+0000 7f946525f740 1 --2- 172.21.15.16:0/2272144798 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e3b8ff2d40 0x55e3b8ff0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:58.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.068+0000 7f946525f740 1 --2- 172.21.15.16:0/2272144798 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e3b8fe8480 0x55e3b8ff00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:58.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.068+0000 7f946525f740 1 --2- 172.21.15.16:0/2272144798 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e3b8fe7b60 0x55e3b8fefb80 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:58.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.068+0000 7f943bfff640 0 client.26077 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:05:58.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.068+0000 7f946525f740 1 -- 172.21.15.16:0/2272144798 >> 172.21.15.16:0/2272144798 conn(0x55e3b8fe40f0 msgr2=0x55e3b8fe9dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:58.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.069+0000 7f946525f740 1 -- 172.21.15.16:0/2272144798 shutdown_connections 2024-09-12T02:05:58.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.069+0000 7f946525f740 1 -- 172.21.15.16:0/2272144798 wait complete. 2024-09-12T02:05:58.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.069+0000 7f946525f740 20 client.26077 trim_cache size 0 max 16384 2024-09-12T02:05:58.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.086+0000 7f946525f740 1 Processor -- start 2024-09-12T02:05:58.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.086+0000 7f946525f740 1 -- start start 2024-09-12T02:05:58.087 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:05:58 smithi016 ceph-mon[30947]: pgmap v456: 97 pgs: 97 active+clean; 2.1 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 4.6 KiB/s wr, 11 op/s 2024-09-12T02:05:58.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.086+0000 7f946525f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e3b8fe7b60 0x55e3b8ff0250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:58.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.086+0000 7f946525f740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e3b8fe8480 0x55e3b8ff0790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:58.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.086+0000 7f946525f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e3b8ff2d40 0x55e3b900a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:58.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.086+0000 7f946525f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f945c1b1fa0 con 0x55e3b8fe7b60 2024-09-12T02:05:58.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.086+0000 7f946525f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f945c1af870 con 0x55e3b8ff2d40 2024-09-12T02:05:58.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.086+0000 7f946525f740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f945c1ad6e0 con 0x55e3b8fe8480 2024-09-12T02:05:58.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.086+0000 7f9460c22640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e3b8fe7b60 0x55e3b8ff0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:58.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.086+0000 7f9460c22640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e3b8fe7b60 0x55e3b8ff0250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54412/0 (socket says 172.21.15.16:54412) 2024-09-12T02:05:58.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.086+0000 7f9460c22640 1 -- 172.21.15.16:0/3506683512 learned_addr learned my addr 172.21.15.16:0/3506683512 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:05:58.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.086+0000 7f945bfff640 1 --2- 172.21.15.16:0/3506683512 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e3b8fe8480 0x55e3b8ff0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:58.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.087+0000 7f9461423640 1 --2- 172.21.15.16:0/3506683512 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e3b8ff2d40 0x55e3b900a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:58.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.087+0000 7f9460c22640 1 -- 172.21.15.16:0/3506683512 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e3b8fe8480 msgr2=0x55e3b8ff0790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:58.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.087+0000 7f9460c22640 1 --2- 172.21.15.16:0/3506683512 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e3b8fe8480 0x55e3b8ff0790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:58.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.087+0000 7f9460c22640 1 -- 172.21.15.16:0/3506683512 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e3b8ff2d40 msgr2=0x55e3b900a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:58.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.087+0000 7f9460c22640 1 --2- 172.21.15.16:0/3506683512 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e3b8ff2d40 0x55e3b900a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:58.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.087+0000 7f9460c22640 1 -- 172.21.15.16:0/3506683512 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e3b8fe3f20 con 0x55e3b8fe7b60 2024-09-12T02:05:58.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.087+0000 7f9460c22640 1 --2- 172.21.15.16:0/3506683512 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e3b8fe7b60 0x55e3b8ff0250 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f944c00f770 tx=0x7f944c011440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=670b2f65027e1423 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:58.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.087+0000 7f94527fc640 1 -- 172.21.15.16:0/3506683512 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f944c033070 con 0x55e3b8fe7b60 2024-09-12T02:05:58.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.087+0000 7f94527fc640 1 -- 172.21.15.16:0/3506683512 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f944c01a460 con 0x55e3b8fe7b60 2024-09-12T02:05:58.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.087+0000 7f94527fc640 1 -- 172.21.15.16:0/3506683512 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f944c028030 con 0x55e3b8fe7b60 2024-09-12T02:05:58.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.088+0000 7f946525f740 1 -- 172.21.15.16:0/3506683512 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e3b8fe7b60 msgr2=0x55e3b8ff0250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:58.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.088+0000 7f946525f740 1 --2- 172.21.15.16:0/3506683512 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e3b8fe7b60 0x55e3b8ff0250 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f944c00f770 tx=0x7f944c011440 comp rx=0 tx=0).stop 2024-09-12T02:05:58.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.088+0000 7f946525f740 1 -- 172.21.15.16:0/3506683512 shutdown_connections 2024-09-12T02:05:58.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.088+0000 7f946525f740 1 --2- 172.21.15.16:0/3506683512 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e3b8ff2d40 0x55e3b900a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:58.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.088+0000 7f946525f740 1 --2- 172.21.15.16:0/3506683512 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e3b8fe8480 0x55e3b8ff0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:58.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.088+0000 7f946525f740 1 --2- 172.21.15.16:0/3506683512 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e3b8fe7b60 0x55e3b8ff0250 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:58.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.088+0000 7f946525f740 1 -- 172.21.15.16:0/3506683512 >> 172.21.15.16:0/3506683512 conn(0x55e3b8fe40f0 msgr2=0x55e3b8fefb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:58.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.088+0000 7f946525f740 1 -- 172.21.15.16:0/3506683512 shutdown_connections 2024-09-12T02:05:58.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.088+0000 7f946525f740 1 -- 172.21.15.16:0/3506683512 wait complete. 2024-09-12T02:05:58.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.088+0000 7f946525f740 1 -- start start 2024-09-12T02:05:58.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.089+0000 7f946525f740 1 Processor -- start 2024-09-12T02:05:58.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.089+0000 7f946525f740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e3b8fe7b60 0x55e3b8ff6ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:58.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.089+0000 7f946525f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e3b8fe8480 0x55e3b8ff7020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:58.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.089+0000 7f946525f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e3b8ff2d40 0x55e3b8ff7560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:58.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.089+0000 7f946525f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f945c1add30 con 0x55e3b8fe8480 2024-09-12T02:05:58.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.089+0000 7f946525f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f945c1af1a0 con 0x55e3b8ff2d40 2024-09-12T02:05:58.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.089+0000 7f946525f740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f945c1ae400 con 0x55e3b8fe7b60 2024-09-12T02:05:58.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.089+0000 7f945bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e3b8fe8480 0x55e3b8ff7020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:58.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.089+0000 7f946525f740 10 client.0 _ll_register_callbacks cb 0x7f944c00bbc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:05:58.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.089+0000 7f945bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e3b8fe8480 0x55e3b8ff7020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54416/0 (socket says 172.21.15.16:54416) 2024-09-12T02:05:58.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.089+0000 7f945bfff640 1 -- 172.21.15.16:0/4028524908 learned_addr learned my addr 172.21.15.16:0/4028524908 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:05:58.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.089+0000 7f9461423640 1 --2- 172.21.15.16:0/4028524908 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e3b8ff2d40 0x55e3b8ff7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:58.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.089+0000 7f9460c22640 1 --2- 172.21.15.16:0/4028524908 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e3b8fe7b60 0x55e3b8ff6ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:58.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.090+0000 7f945bfff640 1 -- 172.21.15.16:0/4028524908 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e3b8fe7b60 msgr2=0x55e3b8ff6ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:58.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.090+0000 7f945bfff640 1 --2- 172.21.15.16:0/4028524908 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e3b8fe7b60 0x55e3b8ff6ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:58.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.090+0000 7f945bfff640 1 -- 172.21.15.16:0/4028524908 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e3b8ff2d40 msgr2=0x55e3b8ff7560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:58.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.090+0000 7f945bfff640 1 --2- 172.21.15.16:0/4028524908 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e3b8ff2d40 0x55e3b8ff7560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:58.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.090+0000 7f945bfff640 1 -- 172.21.15.16:0/4028524908 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e3b906c2a0 con 0x55e3b8fe8480 2024-09-12T02:05:58.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.090+0000 7f9460c22640 1 --2- 172.21.15.16:0/4028524908 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e3b8fe7b60 0x55e3b8ff6ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:05:58.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.090+0000 7f945bfff640 1 --2- 172.21.15.16:0/4028524908 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e3b8fe8480 0x55e3b8ff7020 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f94440ed460 tx=0x7f944403b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96c078531137a72b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:58.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.090+0000 7f943bfff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:05:58.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.090+0000 7f943bfff640 1 -- 172.21.15.16:0/4028524908 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9444049040 con 0x55e3b8fe8480 2024-09-12T02:05:58.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.090+0000 7f943bfff640 1 -- 172.21.15.16:0/4028524908 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f94440ba260 con 0x55e3b8fe8480 2024-09-12T02:05:58.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.090+0000 7f946525f740 1 -- 172.21.15.16:0/4028524908 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f944c01a460 con 0x55e3b8fe8480 2024-09-12T02:05:58.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.090+0000 7f943bfff640 1 -- 172.21.15.16:0/4028524908 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f94440d51a0 con 0x55e3b8fe8480 2024-09-12T02:05:58.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.090+0000 7f946525f740 10 client.16713 fetch_fsmap learned FSMap version 25 2024-09-12T02:05:58.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.090+0000 7f946525f740 1 -- 172.21.15.16:0/4028524908 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e3b906d5f0 con 0x55e3b8fe8480 2024-09-12T02:05:58.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.090+0000 7f943bfff640 1 -- 172.21.15.16:0/4028524908 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94440c0570 con 0x55e3b8fe8480 2024-09-12T02:05:58.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.091+0000 7f943bfff640 1 -- 172.21.15.16:0/4028524908 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f94440dc140 con 0x55e3b8fe8480 2024-09-12T02:05:58.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.091+0000 7f946525f740 10 client.16713 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:05:58.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.091+0000 7f946525f740 10 client.16713 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:05:58.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.091+0000 7f946525f740 20 client.16713 populate_metadata read hostname 'smithi016' 2024-09-12T02:05:58.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.091+0000 7f946525f740 1 --2- 172.21.15.16:0/4028524908 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55e3b906dda0 0x55e3b90701c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:58.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.091+0000 7f946525f740 4 client.16713 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-12T02:05:58.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.091+0000 7f946525f740 1 -- 172.21.15.16:0/4028524908 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7f944c01a460 con 0x55e3b906dda0 2024-09-12T02:05:58.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.091+0000 7f9460c22640 1 --2- 172.21.15.16:0/4028524908 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55e3b906dda0 0x55e3b90701c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:58.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.092+0000 7f9460c22640 1 --2- 172.21.15.16:0/4028524908 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55e3b906dda0 0x55e3b90701c0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd2d79663447fe7e server_cookie=baf21a8b6d93dbc6 in_seq=0 out_seq=0 2024-09-12T02:05:58.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.092+0000 7f943bfff640 10 client.16713 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:05:58.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.093+0000 7f943bfff640 1 -- 172.21.15.16:0/4028524908 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7f944c01a460 con 0x55e3b906dda0 2024-09-12T02:05:58.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.093+0000 7f943bfff640 10 client.16713 handle_command_reply: tid=0 2024-09-12T02:05:58.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.093+0000 7f946525f740 1 client.16713 shutdown 2024-09-12T02:05:58.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.093+0000 7f946525f740 1 -- 172.21.15.16:0/4028524908 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e3b8fe8480 msgr2=0x55e3b8ff7020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:58.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.093+0000 7f946525f740 1 --2- 172.21.15.16:0/4028524908 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e3b8fe8480 0x55e3b8ff7020 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f94440ed460 tx=0x7f944403b870 comp rx=0 tx=0).stop 2024-09-12T02:05:58.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.093+0000 7f946525f740 1 -- 172.21.15.16:0/4028524908 shutdown_connections 2024-09-12T02:05:58.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.093+0000 7f946525f740 1 --2- 172.21.15.16:0/4028524908 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55e3b906dda0 0x55e3b90701c0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:58.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.093+0000 7f946525f740 1 --2- 172.21.15.16:0/4028524908 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e3b8ff2d40 0x55e3b8ff7560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:58.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.093+0000 7f946525f740 1 --2- 172.21.15.16:0/4028524908 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e3b8fe8480 0x55e3b8ff7020 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:58.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.093+0000 7f946525f740 1 --2- 172.21.15.16:0/4028524908 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e3b8fe7b60 0x55e3b8ff6ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:58.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.093+0000 7f946525f740 1 -- 172.21.15.16:0/4028524908 >> 172.21.15.16:0/4028524908 conn(0x55e3b8fe40f0 msgr2=0x55e3b8fe9d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:58.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.093+0000 7f943bfff640 0 client.16713 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:05:58.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.094+0000 7f946525f740 1 -- 172.21.15.16:0/4028524908 shutdown_connections 2024-09-12T02:05:58.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.094+0000 7f946525f740 1 -- 172.21.15.16:0/4028524908 wait complete. 2024-09-12T02:05:58.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.094+0000 7f946525f740 20 client.16713 trim_cache size 0 max 16384 2024-09-12T02:05:58.099 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T02:05:58.099 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-09-12T02:05:58.099 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "0f26a470-aa30-4986-835f-345029a7293c", 2024-09-12T02:05:58.099 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-09-12T02:05:58.099 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T02:05:58.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.094+0000 7f9462e86640 1 -- 172.21.15.16:0/1113806111 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f94480a2b80 msgr2=0x7f94480a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:58.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.094+0000 7f9462e86640 1 --2- 172.21.15.16:0/1113806111 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f94480a2b80 0x7f94480a5040 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f9454009ea0 tx=0x7f94540094f0 comp rx=0 tx=0).stop 2024-09-12T02:05:58.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.094+0000 7f9462e86640 1 -- 172.21.15.16:0/1113806111 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f945c102570 msgr2=0x7f945c19ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:58.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.094+0000 7f9462e86640 1 --2- 172.21.15.16:0/1113806111 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f945c102570 0x7f945c19ace0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f9444006010 tx=0x7f9444003c10 comp rx=0 tx=0).stop 2024-09-12T02:05:58.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.094+0000 7f9462e86640 1 -- 172.21.15.16:0/1113806111 shutdown_connections 2024-09-12T02:05:58.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.094+0000 7f9462e86640 1 --2- 172.21.15.16:0/1113806111 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f94480a2b80 0x7f94480a5040 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:58.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.094+0000 7f9462e86640 1 --2- 172.21.15.16:0/1113806111 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f945c102ef0 0x7f945c19dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:58.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.094+0000 7f9462e86640 1 --2- 172.21.15.16:0/1113806111 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f945c102570 0x7f945c19ace0 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:58.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.094+0000 7f9462e86640 1 --2- 172.21.15.16:0/1113806111 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f945c100e20 0x7f945c19a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:58.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.094+0000 7f9462e86640 1 -- 172.21.15.16:0/1113806111 >> 172.21.15.16:0/1113806111 conn(0x7f945c0fc7d0 msgr2=0x7f945c06a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:58.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.095+0000 7f9462e86640 1 -- 172.21.15.16:0/1113806111 shutdown_connections 2024-09-12T02:05:58.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:58.095+0000 7f9462e86640 1 -- 172.21.15.16:0/1113806111 wait complete. 2024-09-12T02:05:58.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:05:58 smithi184 ceph-mon[35593]: pgmap v456: 97 pgs: 97 active+clean; 2.1 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 4.6 KiB/s wr, 11 op/s 2024-09-12T02:05:58.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:05:58 smithi033 ceph-mon[35332]: pgmap v456: 97 pgs: 97 active+clean; 2.1 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 4.6 KiB/s wr, 11 op/s 2024-09-12T02:05:58.802 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T02:05:59.062 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:05:59.298 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:05:59 smithi016 ceph-mon[30947]: scrub queued for path: / 2024-09-12T02:05:59.298 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:05:59 smithi016 ceph-mon[30947]: scrub summary: idle+waiting paths [/] 2024-09-12T02:05:59.298 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:05:59 smithi016 ceph-mon[30947]: scrub summary: active paths [/] 2024-09-12T02:05:59.298 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:05:59 smithi016 ceph-mon[30947]: scrub summary: idle+waiting paths [/] 2024-09-12T02:05:59.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:05:59 smithi184 ceph-mon[35593]: scrub queued for path: / 2024-09-12T02:05:59.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:05:59 smithi184 ceph-mon[35593]: scrub summary: idle+waiting paths [/] 2024-09-12T02:05:59.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:05:59 smithi184 ceph-mon[35593]: scrub summary: active paths [/] 2024-09-12T02:05:59.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:05:59 smithi184 ceph-mon[35593]: scrub summary: idle+waiting paths [/] 2024-09-12T02:05:59.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:05:59 smithi033 ceph-mon[35332]: scrub queued for path: / 2024-09-12T02:05:59.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:05:59 smithi033 ceph-mon[35332]: scrub summary: idle+waiting paths [/] 2024-09-12T02:05:59.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:05:59 smithi033 ceph-mon[35332]: scrub summary: active paths [/] 2024-09-12T02:05:59.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:05:59 smithi033 ceph-mon[35332]: scrub summary: idle+waiting paths [/] 2024-09-12T02:05:59.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.511+0000 7fbaa6da5640 1 -- 172.21.15.16:0/804560384 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbaa010c2f0 msgr2=0x7fbaa010e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:59.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.511+0000 7fbaa6da5640 1 --2- 172.21.15.16:0/804560384 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbaa010c2f0 0x7fbaa010e790 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fba98009ca0 tx=0x7fba9802a8c0 comp rx=0 tx=0).stop 2024-09-12T02:05:59.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.511+0000 7fbaa6da5640 1 -- 172.21.15.16:0/804560384 shutdown_connections 2024-09-12T02:05:59.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.511+0000 7fbaa6da5640 1 --2- 172.21.15.16:0/804560384 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbaa010c2f0 0x7fbaa010e790 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:59.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.511+0000 7fbaa6da5640 1 --2- 172.21.15.16:0/804560384 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbaa01037a0 0x7fbaa010bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:59.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.511+0000 7fbaa6da5640 1 --2- 172.21.15.16:0/804560384 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbaa0102df0 0x7fbaa01031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:59.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.511+0000 7fbaa6da5640 1 -- 172.21.15.16:0/804560384 >> 172.21.15.16:0/804560384 conn(0x7fbaa00fc7d0 msgr2=0x7fbaa00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:59.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.511+0000 7fbaa6da5640 1 -- 172.21.15.16:0/804560384 shutdown_connections 2024-09-12T02:05:59.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.511+0000 7fbaa6da5640 1 -- 172.21.15.16:0/804560384 wait complete. 2024-09-12T02:05:59.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.511+0000 7fbaa6da5640 1 Processor -- start 2024-09-12T02:05:59.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.512+0000 7fbaa6da5640 1 -- start start 2024-09-12T02:05:59.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.512+0000 7fbaa6da5640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbaa0102df0 0x7fbaa0198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:59.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.512+0000 7fbaa6da5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbaa01037a0 0x7fbaa0198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:59.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.512+0000 7fbaa6da5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbaa010c2f0 0x7fbaa019cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:59.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.512+0000 7fbaa6da5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbaa01120f0 con 0x7fbaa01037a0 2024-09-12T02:05:59.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.512+0000 7fbaa6da5640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbaa0111f70 con 0x7fbaa010c2f0 2024-09-12T02:05:59.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.512+0000 7fbaa6da5640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fbaa0112270 con 0x7fbaa0102df0 2024-09-12T02:05:59.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.512+0000 7fba9ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbaa01037a0 0x7fbaa0198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:59.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.512+0000 7fba9ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbaa01037a0 0x7fbaa0198a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54432/0 (socket says 172.21.15.16:54432) 2024-09-12T02:05:59.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.512+0000 7fba9ffff640 1 -- 172.21.15.16:0/1334932608 learned_addr learned my addr 172.21.15.16:0/1334932608 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:05:59.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.512+0000 7fbaa5342640 1 --2- 172.21.15.16:0/1334932608 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbaa010c2f0 0x7fbaa019cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:59.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.512+0000 7fbaa4b41640 1 --2- 172.21.15.16:0/1334932608 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbaa0102df0 0x7fbaa0198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:59.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.512+0000 7fba9ffff640 1 -- 172.21.15.16:0/1334932608 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbaa0102df0 msgr2=0x7fbaa0198510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:59.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.512+0000 7fba9ffff640 1 --2- 172.21.15.16:0/1334932608 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbaa0102df0 0x7fbaa0198510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:59.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.512+0000 7fba9ffff640 1 -- 172.21.15.16:0/1334932608 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbaa010c2f0 msgr2=0x7fbaa019cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:59.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.512+0000 7fba9ffff640 1 --2- 172.21.15.16:0/1334932608 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbaa010c2f0 0x7fbaa019cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:59.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.512+0000 7fba9ffff640 1 -- 172.21.15.16:0/1334932608 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbaa019d4c0 con 0x7fbaa01037a0 2024-09-12T02:05:59.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.512+0000 7fbaa5342640 1 --2- 172.21.15.16:0/1334932608 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbaa010c2f0 0x7fbaa019cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:05:59.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.512+0000 7fbaa4b41640 1 --2- 172.21.15.16:0/1334932608 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbaa0102df0 0x7fbaa0198510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:05:59.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.513+0000 7fba9ffff640 1 --2- 172.21.15.16:0/1334932608 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbaa01037a0 0x7fbaa0198a50 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fba8c00c8c0 tx=0x7fba8c00cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:59.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.513+0000 7fba9dffb640 1 -- 172.21.15.16:0/1334932608 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba8c007bb0 con 0x7fbaa01037a0 2024-09-12T02:05:59.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.513+0000 7fba9dffb640 1 -- 172.21.15.16:0/1334932608 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba8c007d50 con 0x7fbaa01037a0 2024-09-12T02:05:59.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.513+0000 7fbaa6da5640 1 -- 172.21.15.16:0/1334932608 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbaa019d7b0 con 0x7fbaa01037a0 2024-09-12T02:05:59.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.513+0000 7fba9dffb640 1 -- 172.21.15.16:0/1334932608 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba8c018830 con 0x7fbaa01037a0 2024-09-12T02:05:59.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.513+0000 7fbaa6da5640 1 -- 172.21.15.16:0/1334932608 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbaa01a5090 con 0x7fbaa01037a0 2024-09-12T02:05:59.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.515+0000 7fba9dffb640 1 -- 172.21.15.16:0/1334932608 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fba8c018a10 con 0x7fbaa01037a0 2024-09-12T02:05:59.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.516+0000 7fbaa917e740 1 Processor -- start 2024-09-12T02:05:59.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.516+0000 7fbaa917e740 1 -- start start 2024-09-12T02:05:59.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.516+0000 7fbaa917e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d87cd1e4a0 0x55d87cd1e880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:59.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.516+0000 7fbaa917e740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d87cd1edc0 0x55d87cd29140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:59.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.516+0000 7fbaa917e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d87cd29680 0x55d87cd2ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:59.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.516+0000 7fbaa917e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55d87cc842e0 con 0x55d87cd1e4a0 2024-09-12T02:05:59.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.516+0000 7fbaa917e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55d87cc2bce0 con 0x55d87cd29680 2024-09-12T02:05:59.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.516+0000 7fbaa917e740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55d87cc802a0 con 0x55d87cd1edc0 2024-09-12T02:05:59.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.516+0000 7fba9dffb640 1 --2- 172.21.15.16:0/1334932608 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fba900a2b80 0x7fba900a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:59.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.516+0000 7fbaa5342640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d87cd1e4a0 0x55d87cd1e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:59.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.516+0000 7fbaa5342640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d87cd1e4a0 0x55d87cd1e880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54444/0 (socket says 172.21.15.16:54444) 2024-09-12T02:05:59.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.516+0000 7fbaa5342640 1 -- 172.21.15.16:0/3954835505 learned_addr learned my addr 172.21.15.16:0/3954835505 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:05:59.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.516+0000 7fbaa4b41640 1 --2- 172.21.15.16:0/3954835505 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d87cd1edc0 0x55d87cd29140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:59.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.516+0000 7fba9dffb640 1 -- 172.21.15.16:0/1334932608 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fba8c0cf190 con 0x7fbaa01037a0 2024-09-12T02:05:59.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.516+0000 7fba9ffff640 1 --2- 172.21.15.16:0/3954835505 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d87cd29680 0x55d87cd2ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:59.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.516+0000 7fbaa5342640 1 --2- 172.21.15.16:0/1334932608 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fba900a2b80 0x7fba900a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:59.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.517+0000 7fbaa5342640 1 -- 172.21.15.16:0/3954835505 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d87cd1edc0 msgr2=0x55d87cd29140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:59.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.517+0000 7fbaa5342640 1 --2- 172.21.15.16:0/3954835505 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d87cd1edc0 0x55d87cd29140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:59.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.517+0000 7fbaa5342640 1 -- 172.21.15.16:0/3954835505 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d87cd29680 msgr2=0x55d87cd2ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:59.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.517+0000 7fbaa5342640 1 --2- 172.21.15.16:0/3954835505 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d87cd29680 0x55d87cd2ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:59.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.517+0000 7fbaa5342640 1 -- 172.21.15.16:0/3954835505 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d87cc10440 con 0x55d87cd1e4a0 2024-09-12T02:05:59.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.517+0000 7fbaa5342640 1 --2- 172.21.15.16:0/3954835505 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d87cd1e4a0 0x55d87cd1e880 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fba980314d0 tx=0x7fba98031500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=514ab565ba6cb9d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:59.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.517+0000 7fba8affd640 1 -- 172.21.15.16:0/3954835505 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba9802f610 con 0x55d87cd1e4a0 2024-09-12T02:05:59.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.517+0000 7fba8affd640 1 -- 172.21.15.16:0/3954835505 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba9804d070 con 0x55d87cd1e4a0 2024-09-12T02:05:59.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.517+0000 7fba8affd640 1 -- 172.21.15.16:0/3954835505 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba98030940 con 0x55d87cd1e4a0 2024-09-12T02:05:59.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.518+0000 7fbaa917e740 1 -- 172.21.15.16:0/3954835505 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d87cd1e4a0 msgr2=0x55d87cd1e880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:59.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.518+0000 7fbaa917e740 1 --2- 172.21.15.16:0/3954835505 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d87cd1e4a0 0x55d87cd1e880 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fba980314d0 tx=0x7fba98031500 comp rx=0 tx=0).stop 2024-09-12T02:05:59.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.518+0000 7fbaa5342640 1 --2- 172.21.15.16:0/1334932608 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fba900a2b80 0x7fba900a5040 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fba9804c000 tx=0x7fba9804d000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:59.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.518+0000 7fbaa917e740 1 -- 172.21.15.16:0/3954835505 shutdown_connections 2024-09-12T02:05:59.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.518+0000 7fbaa917e740 1 --2- 172.21.15.16:0/3954835505 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d87cd29680 0x55d87cd2ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:59.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.518+0000 7fbaa917e740 1 --2- 172.21.15.16:0/3954835505 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d87cd1edc0 0x55d87cd29140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:59.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.518+0000 7fbaa917e740 1 --2- 172.21.15.16:0/3954835505 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d87cd1e4a0 0x55d87cd1e880 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:59.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.518+0000 7fbaa917e740 1 -- 172.21.15.16:0/3954835505 >> 172.21.15.16:0/3954835505 conn(0x55d87cd1aa30 msgr2=0x55d87cd1ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:59.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.518+0000 7fbaa917e740 1 -- 172.21.15.16:0/3954835505 shutdown_connections 2024-09-12T02:05:59.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.518+0000 7fbaa917e740 1 -- 172.21.15.16:0/3954835505 wait complete. 2024-09-12T02:05:59.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.518+0000 7fbaa917e740 1 -- start start 2024-09-12T02:05:59.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.518+0000 7fbaa917e740 1 Processor -- start 2024-09-12T02:05:59.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.519+0000 7fbaa917e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d87cd1e4a0 0x55d87cd28f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:59.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.519+0000 7fbaa917e740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d87cd1edc0 0x55d87cd26230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:59.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.519+0000 7fbaa917e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d87cd29680 0x55d87cd26770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:59.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.519+0000 7fbaa917e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55d87cd304e0 con 0x55d87cd29680 2024-09-12T02:05:59.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.519+0000 7fbaa917e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55d87cc3f050 con 0x55d87cd1e4a0 2024-09-12T02:05:59.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.519+0000 7fbaa917e740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55d87cd30660 con 0x55d87cd1edc0 2024-09-12T02:05:59.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.519+0000 7fbaa917e740 10 client.0 _ll_register_callbacks cb 0x55d87cc7f160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:05:59.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.519+0000 7fbaa5342640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d87cd29680 0x55d87cd26770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:59.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.519+0000 7fbaa5342640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d87cd29680 0x55d87cd26770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54458/0 (socket says 172.21.15.16:54458) 2024-09-12T02:05:59.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.519+0000 7fbaa5342640 1 -- 172.21.15.16:0/1395439403 learned_addr learned my addr 172.21.15.16:0/1395439403 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:05:59.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.519+0000 7fba9ffff640 1 --2- 172.21.15.16:0/1395439403 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d87cd1edc0 0x55d87cd26230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:59.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.519+0000 7fbaa4b41640 1 --2- 172.21.15.16:0/1395439403 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d87cd1e4a0 0x55d87cd28f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:59.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.519+0000 7fbaa5342640 1 -- 172.21.15.16:0/1395439403 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d87cd1edc0 msgr2=0x55d87cd26230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:59.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.519+0000 7fbaa5342640 1 --2- 172.21.15.16:0/1395439403 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d87cd1edc0 0x55d87cd26230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:59.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.519+0000 7fbaa5342640 1 -- 172.21.15.16:0/1395439403 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d87cd1e4a0 msgr2=0x55d87cd28f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:59.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.519+0000 7fbaa5342640 1 --2- 172.21.15.16:0/1395439403 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d87cd1e4a0 0x55d87cd28f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:59.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.519+0000 7fbaa5342640 1 -- 172.21.15.16:0/1395439403 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d87c9f53c0 con 0x55d87cd29680 2024-09-12T02:05:59.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.520+0000 7fbaa5342640 1 --2- 172.21.15.16:0/1395439403 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d87cd29680 0x55d87cd26770 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fba98031d30 tx=0x7fba98030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45118dfaf2bbc900 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:59.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.520+0000 7fba7ffff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:05:59.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.520+0000 7fba7ffff640 1 -- 172.21.15.16:0/1395439403 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba98055040 con 0x55d87cd29680 2024-09-12T02:05:59.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.520+0000 7fbaa917e740 1 -- 172.21.15.16:0/1395439403 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fba9804d070 con 0x55d87cd29680 2024-09-12T02:05:59.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.520+0000 7fba7ffff640 1 -- 172.21.15.16:0/1395439403 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fba98004b40 con 0x55d87cd29680 2024-09-12T02:05:59.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.520+0000 7fba7ffff640 1 -- 172.21.15.16:0/1395439403 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fba98030940 con 0x55d87cd29680 2024-09-12T02:05:59.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.520+0000 7fbaa917e740 10 client.16725 fetch_fsmap learned FSMap version 25 2024-09-12T02:05:59.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.520+0000 7fbaa917e740 1 -- 172.21.15.16:0/1395439403 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d87cd304e0 con 0x55d87cd29680 2024-09-12T02:05:59.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.520+0000 7fba7ffff640 1 -- 172.21.15.16:0/1395439403 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba9804d070 con 0x55d87cd29680 2024-09-12T02:05:59.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.520+0000 7fba7ffff640 1 -- 172.21.15.16:0/1395439403 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fba98061030 con 0x55d87cd29680 2024-09-12T02:05:59.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.520+0000 7fbaa917e740 10 client.16725 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:05:59.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.520+0000 7fbaa917e740 10 client.16725 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:05:59.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.520+0000 7fbaa917e740 20 client.16725 populate_metadata read hostname 'smithi016' 2024-09-12T02:05:59.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.520+0000 7fbaa917e740 1 --2- 172.21.15.16:0/1395439403 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55d87cd3cfd0 0x55d87cd3f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:59.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.520+0000 7fbaa917e740 4 client.16725 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:05:59.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.520+0000 7fbaa917e740 1 -- 172.21.15.16:0/1395439403 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d87cc91850 con 0x55d87cd3cfd0 2024-09-12T02:05:59.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.521+0000 7fbaa4b41640 1 --2- 172.21.15.16:0/1395439403 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55d87cd3cfd0 0x55d87cd3f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:59.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.521+0000 7fbaa4b41640 1 --2- 172.21.15.16:0/1395439403 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55d87cd3cfd0 0x55d87cd3f3c0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1e9feac0af95b96 server_cookie=bae7441edeb85ade in_seq=0 out_seq=0 2024-09-12T02:05:59.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.521+0000 7fba7ffff640 10 client.16725 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:05:59.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.522+0000 7fba7ffff640 1 -- 172.21.15.16:0/1395439403 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d87cc91850 con 0x55d87cd3cfd0 2024-09-12T02:05:59.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.522+0000 7fba7ffff640 10 client.16725 handle_command_reply: tid=0 2024-09-12T02:05:59.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.522+0000 7fbaa917e740 1 client.16725 shutdown 2024-09-12T02:05:59.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.522+0000 7fbaa917e740 1 -- 172.21.15.16:0/1395439403 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d87cd29680 msgr2=0x55d87cd26770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:59.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.522+0000 7fbaa917e740 1 --2- 172.21.15.16:0/1395439403 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d87cd29680 0x55d87cd26770 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fba98031d30 tx=0x7fba98030000 comp rx=0 tx=0).stop 2024-09-12T02:05:59.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.522+0000 7fbaa917e740 1 -- 172.21.15.16:0/1395439403 shutdown_connections 2024-09-12T02:05:59.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.522+0000 7fbaa917e740 1 --2- 172.21.15.16:0/1395439403 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55d87cd3cfd0 0x55d87cd3f3c0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:59.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.522+0000 7fbaa917e740 1 --2- 172.21.15.16:0/1395439403 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d87cd29680 0x55d87cd26770 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:59.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.522+0000 7fbaa917e740 1 --2- 172.21.15.16:0/1395439403 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d87cd1edc0 0x55d87cd26230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:59.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.522+0000 7fbaa917e740 1 --2- 172.21.15.16:0/1395439403 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d87cd1e4a0 0x55d87cd28f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:59.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.522+0000 7fbaa917e740 1 -- 172.21.15.16:0/1395439403 >> 172.21.15.16:0/1395439403 conn(0x55d87cd1aa30 msgr2=0x55d87cd20710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:59.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.522+0000 7fba7ffff640 0 client.16725 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:05:59.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.522+0000 7fbaa917e740 1 -- 172.21.15.16:0/1395439403 shutdown_connections 2024-09-12T02:05:59.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.522+0000 7fbaa917e740 1 -- 172.21.15.16:0/1395439403 wait complete. 2024-09-12T02:05:59.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.522+0000 7fbaa917e740 20 client.16725 trim_cache size 0 max 16384 2024-09-12T02:05:59.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.539+0000 7fbaa917e740 1 Processor -- start 2024-09-12T02:05:59.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.539+0000 7fbaa917e740 1 -- start start 2024-09-12T02:05:59.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.540+0000 7fbaa917e740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d87cd1e4a0 0x55d87cd259e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:59.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.540+0000 7fbaa917e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d87cd1edc0 0x55d87cd44c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:59.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.540+0000 7fbaa917e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d87cd29680 0x55d87cd45150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:59.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.540+0000 7fbaa917e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbaa01ad9c0 con 0x55d87cd1edc0 2024-09-12T02:05:59.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.540+0000 7fbaa917e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbaa01ab290 con 0x55d87cd29680 2024-09-12T02:05:59.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.540+0000 7fbaa917e740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fbaa01a9120 con 0x55d87cd1e4a0 2024-09-12T02:05:59.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.540+0000 7fba9ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d87cd1edc0 0x55d87cd44c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:59.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.540+0000 7fba9ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d87cd1edc0 0x55d87cd44c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54462/0 (socket says 172.21.15.16:54462) 2024-09-12T02:05:59.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.540+0000 7fba9ffff640 1 -- 172.21.15.16:0/2044091975 learned_addr learned my addr 172.21.15.16:0/2044091975 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:05:59.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.540+0000 7fbaa4b41640 1 --2- 172.21.15.16:0/2044091975 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d87cd1e4a0 0x55d87cd259e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:59.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.540+0000 7fbaa5342640 1 --2- 172.21.15.16:0/2044091975 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d87cd29680 0x55d87cd45150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:59.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.540+0000 7fba9ffff640 1 -- 172.21.15.16:0/2044091975 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d87cd1e4a0 msgr2=0x55d87cd259e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:59.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.540+0000 7fba9ffff640 1 --2- 172.21.15.16:0/2044091975 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d87cd1e4a0 0x55d87cd259e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:59.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.540+0000 7fba9ffff640 1 -- 172.21.15.16:0/2044091975 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d87cd29680 msgr2=0x55d87cd45150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:59.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.540+0000 7fba9ffff640 1 --2- 172.21.15.16:0/2044091975 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d87cd29680 0x55d87cd45150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:59.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.540+0000 7fba9ffff640 1 -- 172.21.15.16:0/2044091975 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d87cd28ba0 con 0x55d87cd1edc0 2024-09-12T02:05:59.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.540+0000 7fba9ffff640 1 --2- 172.21.15.16:0/2044091975 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d87cd1edc0 0x55d87cd44c10 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fba8c007590 tx=0x7fba8c086040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86d3b9cfd14b3a52 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:59.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.540+0000 7fba8a7fc640 1 -- 172.21.15.16:0/2044091975 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba8c09b9f0 con 0x55d87cd1edc0 2024-09-12T02:05:59.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.540+0000 7fba8a7fc640 1 -- 172.21.15.16:0/2044091975 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba8c09fda0 con 0x55d87cd1edc0 2024-09-12T02:05:59.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.540+0000 7fba8a7fc640 1 -- 172.21.15.16:0/2044091975 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba8c098060 con 0x55d87cd1edc0 2024-09-12T02:05:59.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.541+0000 7fbaa917e740 1 -- 172.21.15.16:0/2044091975 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d87cd1edc0 msgr2=0x55d87cd44c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:59.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.541+0000 7fbaa917e740 1 --2- 172.21.15.16:0/2044091975 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d87cd1edc0 0x55d87cd44c10 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fba8c007590 tx=0x7fba8c086040 comp rx=0 tx=0).stop 2024-09-12T02:05:59.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.541+0000 7fbaa917e740 1 -- 172.21.15.16:0/2044091975 shutdown_connections 2024-09-12T02:05:59.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.541+0000 7fbaa917e740 1 --2- 172.21.15.16:0/2044091975 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d87cd29680 0x55d87cd45150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:59.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.541+0000 7fbaa917e740 1 --2- 172.21.15.16:0/2044091975 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d87cd1edc0 0x55d87cd44c10 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:59.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.541+0000 7fbaa917e740 1 --2- 172.21.15.16:0/2044091975 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d87cd1e4a0 0x55d87cd259e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:59.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.541+0000 7fbaa917e740 1 -- 172.21.15.16:0/2044091975 >> 172.21.15.16:0/2044091975 conn(0x55d87cd1aa30 msgr2=0x55d87cd282b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:59.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.541+0000 7fbaa917e740 1 -- 172.21.15.16:0/2044091975 shutdown_connections 2024-09-12T02:05:59.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.541+0000 7fbaa917e740 1 -- 172.21.15.16:0/2044091975 wait complete. 2024-09-12T02:05:59.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.541+0000 7fbaa917e740 1 -- start start 2024-09-12T02:05:59.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.542+0000 7fbaa917e740 1 Processor -- start 2024-09-12T02:05:59.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.542+0000 7fbaa917e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d87cd1e4a0 0x55d87cd2feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:59.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.542+0000 7fbaa917e740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d87cd1edc0 0x55d87cd2e420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:59.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.542+0000 7fbaa917e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d87cd29680 0x55d87cd2ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:59.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.542+0000 7fbaa917e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbaa01a9770 con 0x55d87cd1e4a0 2024-09-12T02:05:59.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.542+0000 7fbaa917e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbaa01aabc0 con 0x55d87cd29680 2024-09-12T02:05:59.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.542+0000 7fbaa917e740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fbaa01a9e20 con 0x55d87cd1edc0 2024-09-12T02:05:59.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.542+0000 7fbaa917e740 10 client.0 _ll_register_callbacks cb 0x7fba9400b2c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:05:59.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.542+0000 7fbaa4b41640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d87cd1e4a0 0x55d87cd2feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:59.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.542+0000 7fbaa4b41640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d87cd1e4a0 0x55d87cd2feb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54474/0 (socket says 172.21.15.16:54474) 2024-09-12T02:05:59.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.542+0000 7fbaa4b41640 1 -- 172.21.15.16:0/1171723585 learned_addr learned my addr 172.21.15.16:0/1171723585 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:05:59.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.542+0000 7fbaa5342640 1 --2- 172.21.15.16:0/1171723585 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d87cd29680 0x55d87cd2ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:59.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.542+0000 7fba9ffff640 1 --2- 172.21.15.16:0/1171723585 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d87cd1edc0 0x55d87cd2e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:59.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.542+0000 7fbaa4b41640 1 -- 172.21.15.16:0/1171723585 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d87cd1edc0 msgr2=0x55d87cd2e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:59.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.542+0000 7fbaa4b41640 1 --2- 172.21.15.16:0/1171723585 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d87cd1edc0 0x55d87cd2e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:59.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.543+0000 7fbaa4b41640 1 -- 172.21.15.16:0/1171723585 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d87cd29680 msgr2=0x55d87cd2ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:59.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.543+0000 7fbaa4b41640 1 --2- 172.21.15.16:0/1171723585 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d87cd29680 0x55d87cd2ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:59.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.543+0000 7fbaa4b41640 1 -- 172.21.15.16:0/1171723585 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d87cd2f230 con 0x55d87cd1e4a0 2024-09-12T02:05:59.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.543+0000 7fbaa4b41640 1 --2- 172.21.15.16:0/1171723585 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d87cd1e4a0 0x55d87cd2feb0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fba94005fa0 tx=0x7fba9400f8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95526a79d65199fa server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:59.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.543+0000 7fba7ffff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:05:59.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.543+0000 7fba7ffff640 1 -- 172.21.15.16:0/1171723585 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba9401c8f0 con 0x55d87cd1e4a0 2024-09-12T02:05:59.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.543+0000 7fba7ffff640 1 -- 172.21.15.16:0/1171723585 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fba9401ca90 con 0x55d87cd1e4a0 2024-09-12T02:05:59.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.543+0000 7fbaa917e740 1 -- 172.21.15.16:0/1171723585 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fba8c09fda0 con 0x55d87cd1e4a0 2024-09-12T02:05:59.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.543+0000 7fba7ffff640 1 -- 172.21.15.16:0/1171723585 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fba9401cc90 con 0x55d87cd1e4a0 2024-09-12T02:05:59.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.543+0000 7fbaa917e740 10 client.16731 fetch_fsmap learned FSMap version 25 2024-09-12T02:05:59.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.543+0000 7fbaa917e740 1 -- 172.21.15.16:0/1171723585 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d87cd35a20 con 0x55d87cd1e4a0 2024-09-12T02:05:59.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.543+0000 7fba7ffff640 1 -- 172.21.15.16:0/1171723585 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba8c09fda0 con 0x55d87cd1e4a0 2024-09-12T02:05:59.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.543+0000 7fba7ffff640 1 -- 172.21.15.16:0/1171723585 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fba940255c0 con 0x55d87cd1e4a0 2024-09-12T02:05:59.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.543+0000 7fbaa917e740 10 client.16731 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:05:59.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.543+0000 7fbaa917e740 10 client.16731 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:05:59.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.543+0000 7fbaa917e740 20 client.16731 populate_metadata read hostname 'smithi016' 2024-09-12T02:05:59.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.544+0000 7fbaa917e740 1 --2- 172.21.15.16:0/1171723585 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55d87cd361d0 0x55d87cdae3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:59.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.544+0000 7fbaa917e740 4 client.16731 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T02:05:59.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.544+0000 7fbaa917e740 1 -- 172.21.15.16:0/1171723585 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d87cc8bd60 con 0x55d87cd361d0 2024-09-12T02:05:59.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.544+0000 7fba9ffff640 1 --2- 172.21.15.16:0/1171723585 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55d87cd361d0 0x55d87cdae3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:59.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.544+0000 7fba9ffff640 1 --2- 172.21.15.16:0/1171723585 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55d87cd361d0 0x55d87cdae3e0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ac7cb063aa5c2f9 server_cookie=720eebca1afa43a1 in_seq=0 out_seq=0 2024-09-12T02:05:59.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.544+0000 7fba7ffff640 10 client.16731 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:05:59.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.545+0000 7fba7ffff640 1 -- 172.21.15.16:0/1171723585 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55d87cc8bd60 con 0x55d87cd361d0 2024-09-12T02:05:59.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.545+0000 7fba7ffff640 10 client.16731 handle_command_reply: tid=0 2024-09-12T02:05:59.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.545+0000 7fbaa917e740 1 client.16731 shutdown 2024-09-12T02:05:59.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.545+0000 7fbaa917e740 1 -- 172.21.15.16:0/1171723585 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d87cd1e4a0 msgr2=0x55d87cd2feb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:59.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.545+0000 7fbaa917e740 1 --2- 172.21.15.16:0/1171723585 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d87cd1e4a0 0x55d87cd2feb0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fba94005fa0 tx=0x7fba9400f8b0 comp rx=0 tx=0).stop 2024-09-12T02:05:59.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.545+0000 7fbaa917e740 1 -- 172.21.15.16:0/1171723585 shutdown_connections 2024-09-12T02:05:59.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.545+0000 7fbaa917e740 1 --2- 172.21.15.16:0/1171723585 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55d87cd361d0 0x55d87cdae3e0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:59.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.545+0000 7fbaa917e740 1 --2- 172.21.15.16:0/1171723585 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d87cd29680 0x55d87cd2ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:59.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.545+0000 7fbaa917e740 1 --2- 172.21.15.16:0/1171723585 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d87cd1edc0 0x55d87cd2e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:59.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.545+0000 7fbaa917e740 1 --2- 172.21.15.16:0/1171723585 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d87cd1e4a0 0x55d87cd2feb0 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:59.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.545+0000 7fbaa917e740 1 -- 172.21.15.16:0/1171723585 >> 172.21.15.16:0/1171723585 conn(0x55d87cd1aa30 msgr2=0x55d87cd19350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:59.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.545+0000 7fba7ffff640 0 client.16731 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:05:59.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.545+0000 7fbaa917e740 1 -- 172.21.15.16:0/1171723585 shutdown_connections 2024-09-12T02:05:59.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.545+0000 7fbaa917e740 1 -- 172.21.15.16:0/1171723585 wait complete. 2024-09-12T02:05:59.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.545+0000 7fbaa917e740 20 client.16731 trim_cache size 0 max 16384 2024-09-12T02:05:59.552 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T02:05:59.552 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-12T02:05:59.552 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": { 2024-09-12T02:05:59.552 INFO:teuthology.orchestra.run.smithi016.stdout: "0f26a470-aa30-4986-835f-345029a7293c": { 2024-09-12T02:05:59.553 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-12T02:05:59.553 INFO:teuthology.orchestra.run.smithi016.stdout: "tag": "0f26a470-aa30-4986-835f-345029a7293c", 2024-09-12T02:05:59.553 INFO:teuthology.orchestra.run.smithi016.stdout: "options": "recursive,force" 2024-09-12T02:05:59.553 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:05:59.553 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:05:59.553 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T02:05:59.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.546+0000 7fbaa6da5640 1 -- 172.21.15.16:0/1334932608 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fba900a2b80 msgr2=0x7fba900a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:59.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.546+0000 7fbaa6da5640 1 --2- 172.21.15.16:0/1334932608 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fba900a2b80 0x7fba900a5040 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fba9804c000 tx=0x7fba9804d000 comp rx=0 tx=0).stop 2024-09-12T02:05:59.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.546+0000 7fbaa6da5640 1 -- 172.21.15.16:0/1334932608 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbaa01037a0 msgr2=0x7fbaa0198a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:59.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.546+0000 7fbaa6da5640 1 --2- 172.21.15.16:0/1334932608 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbaa01037a0 0x7fbaa0198a50 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fba8c00c8c0 tx=0x7fba8c00cd80 comp rx=0 tx=0).stop 2024-09-12T02:05:59.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.546+0000 7fbaa6da5640 1 -- 172.21.15.16:0/1334932608 shutdown_connections 2024-09-12T02:05:59.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.546+0000 7fbaa6da5640 1 --2- 172.21.15.16:0/1334932608 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fba900a2b80 0x7fba900a5040 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:59.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.546+0000 7fbaa6da5640 1 --2- 172.21.15.16:0/1334932608 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbaa010c2f0 0x7fbaa019cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:59.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.546+0000 7fbaa6da5640 1 --2- 172.21.15.16:0/1334932608 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbaa01037a0 0x7fbaa0198a50 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:59.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.546+0000 7fbaa6da5640 1 --2- 172.21.15.16:0/1334932608 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbaa0102df0 0x7fbaa0198510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:59.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.546+0000 7fbaa6da5640 1 -- 172.21.15.16:0/1334932608 >> 172.21.15.16:0/1334932608 conn(0x7fbaa00fc7d0 msgr2=0x7fbaa00fd3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:59.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.546+0000 7fbaa6da5640 1 -- 172.21.15.16:0/1334932608 shutdown_connections 2024-09-12T02:05:59.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:05:59.546+0000 7fbaa6da5640 1 -- 172.21.15.16:0/1334932608 wait complete. 2024-09-12T02:06:00.274 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:06:00 smithi016 ceph-mon[30947]: pgmap v457: 97 pgs: 97 active+clean; 2.1 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 4.3 KiB/s wr, 8 op/s 2024-09-12T02:06:00.305 INFO:tasks.cephfs.filesystem:scrub status for tag:0f26a470-aa30-4986-835f-345029a7293c - {'path': '/', 'tag': '0f26a470-aa30-4986-835f-345029a7293c', 'options': 'recursive,force'} 2024-09-12T02:06:00.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:06:00 smithi184 ceph-mon[35593]: pgmap v457: 97 pgs: 97 active+clean; 2.1 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 4.3 KiB/s wr, 8 op/s 2024-09-12T02:06:00.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:06:00 smithi033 ceph-mon[35332]: pgmap v457: 97 pgs: 97 active+clean; 2.1 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 4.3 KiB/s wr, 8 op/s 2024-09-12T02:06:02.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:06:02 smithi184 ceph-mon[35593]: pgmap v458: 97 pgs: 97 active+clean; 2.2 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.7 KiB/s wr, 13 op/s 2024-09-12T02:06:02.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:06:02 smithi184 ceph-mon[35593]: scrub summary: idle 2024-09-12T02:06:02.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:06:02 smithi033 ceph-mon[35332]: pgmap v458: 97 pgs: 97 active+clean; 2.2 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.7 KiB/s wr, 13 op/s 2024-09-12T02:06:02.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:06:02 smithi033 ceph-mon[35332]: scrub summary: idle 2024-09-12T02:06:02.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:06:02 smithi016 ceph-mon[30947]: pgmap v458: 97 pgs: 97 active+clean; 2.2 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.7 KiB/s wr, 13 op/s 2024-09-12T02:06:02.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:06:02 smithi016 ceph-mon[30947]: scrub summary: idle 2024-09-12T02:06:04.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:06:04 smithi184 ceph-mon[35593]: pgmap v459: 97 pgs: 97 active+clean; 2.2 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.3 KiB/s wr, 11 op/s 2024-09-12T02:06:04.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:06:04 smithi033 ceph-mon[35332]: pgmap v459: 97 pgs: 97 active+clean; 2.2 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.3 KiB/s wr, 11 op/s 2024-09-12T02:06:04.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:06:04 smithi016 ceph-mon[30947]: pgmap v459: 97 pgs: 97 active+clean; 2.2 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.3 KiB/s wr, 11 op/s 2024-09-12T02:06:05.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:06:05 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:06:05.418 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:06:05 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:06:05.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:06:05 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:06:06.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:06:06 smithi184 ceph-mon[35593]: pgmap v460: 97 pgs: 97 active+clean; 2.2 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.0 KiB/s wr, 12 op/s 2024-09-12T02:06:06.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:06:06 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:06:06.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:06:06 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:06:06.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:06:06 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T02:06:06.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:06:06 smithi033 ceph-mon[35332]: pgmap v460: 97 pgs: 97 active+clean; 2.2 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.0 KiB/s wr, 12 op/s 2024-09-12T02:06:06.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:06:06 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:06:06.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:06:06 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:06:06.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:06:06 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T02:06:06.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:06:06 smithi016 ceph-mon[30947]: pgmap v460: 97 pgs: 97 active+clean; 2.2 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.0 KiB/s wr, 12 op/s 2024-09-12T02:06:06.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:06:06 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:06:06.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:06:06 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:06:06.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:06:06 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T02:06:07.385 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.380+0000 7fcf67e9b640 1 -- 172.21.15.16:0/2547172422 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf60103710 msgr2=0x7fcf60109fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:07.385 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.380+0000 7fcf67e9b640 1 --2- 172.21.15.16:0/2547172422 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf60103710 0x7fcf60109fa0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fcf58009ca0 tx=0x7fcf5802a8c0 comp rx=0 tx=0).stop 2024-09-12T02:06:07.385 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.380+0000 7fcf67e9b640 1 -- 172.21.15.16:0/2547172422 shutdown_connections 2024-09-12T02:06:07.386 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.380+0000 7fcf67e9b640 1 --2- 172.21.15.16:0/2547172422 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf60103710 0x7fcf60109fa0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:07.386 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.380+0000 7fcf67e9b640 1 --2- 172.21.15.16:0/2547172422 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcf60102d50 0x7fcf601031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:07.386 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.380+0000 7fcf67e9b640 1 --2- 172.21.15.16:0/2547172422 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf60101b60 0x7fcf60101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:07.386 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.380+0000 7fcf67e9b640 1 -- 172.21.15.16:0/2547172422 >> 172.21.15.16:0/2547172422 conn(0x7fcf600fd2e0 msgr2=0x7fcf600ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:06:07.386 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.380+0000 7fcf67e9b640 1 -- 172.21.15.16:0/2547172422 shutdown_connections 2024-09-12T02:06:07.386 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.380+0000 7fcf67e9b640 1 -- 172.21.15.16:0/2547172422 wait complete. 2024-09-12T02:06:07.387 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.381+0000 7fcf67e9b640 1 Processor -- start 2024-09-12T02:06:07.387 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.381+0000 7fcf67e9b640 1 -- start start 2024-09-12T02:06:07.387 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.381+0000 7fcf67e9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf60101b60 0x7fcf60197dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:07.387 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.381+0000 7fcf67e9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf60102d50 0x7fcf60198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:07.387 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.381+0000 7fcf67e9b640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcf60103710 0x7fcf6019f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:07.388 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.381+0000 7fcf67e9b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fcf6010d7e0 con 0x7fcf60102d50 2024-09-12T02:06:07.388 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.381+0000 7fcf67e9b640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fcf6010d660 con 0x7fcf60101b60 2024-09-12T02:06:07.388 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.381+0000 7fcf67e9b640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fcf6010d960 con 0x7fcf60103710 2024-09-12T02:06:07.388 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.381+0000 7fcf65436640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf60102d50 0x7fcf60198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:07.388 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.381+0000 7fcf65436640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf60102d50 0x7fcf60198310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36110/0 (socket says 172.21.15.16:36110) 2024-09-12T02:06:07.388 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.381+0000 7fcf65436640 1 -- 172.21.15.16:0/740446309 learned_addr learned my addr 172.21.15.16:0/740446309 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:06:07.388 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.381+0000 7fcf66438640 1 --2- 172.21.15.16:0/740446309 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcf60103710 0x7fcf6019f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:07.388 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.381+0000 7fcf65c37640 1 --2- 172.21.15.16:0/740446309 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf60101b60 0x7fcf60197dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:07.388 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.381+0000 7fcf65436640 1 -- 172.21.15.16:0/740446309 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcf60103710 msgr2=0x7fcf6019f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:07.388 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.381+0000 7fcf65436640 1 --2- 172.21.15.16:0/740446309 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcf60103710 0x7fcf6019f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:07.389 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.381+0000 7fcf65436640 1 -- 172.21.15.16:0/740446309 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf60101b60 msgr2=0x7fcf60197dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:07.389 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.381+0000 7fcf65436640 1 --2- 172.21.15.16:0/740446309 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf60101b60 0x7fcf60197dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:07.389 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.381+0000 7fcf65436640 1 -- 172.21.15.16:0/740446309 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf6019f9f0 con 0x7fcf60102d50 2024-09-12T02:06:07.389 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.382+0000 7fcf65436640 1 --2- 172.21.15.16:0/740446309 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf60102d50 0x7fcf60198310 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fcf5000e9f0 tx=0x7fcf5000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:06:07.389 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.382+0000 7fcf56ffd640 1 -- 172.21.15.16:0/740446309 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf5000cd80 con 0x7fcf60102d50 2024-09-12T02:06:07.389 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.382+0000 7fcf56ffd640 1 -- 172.21.15.16:0/740446309 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf50005500 con 0x7fcf60102d50 2024-09-12T02:06:07.389 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.382+0000 7fcf67e9b640 1 -- 172.21.15.16:0/740446309 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf6019fce0 con 0x7fcf60102d50 2024-09-12T02:06:07.389 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.382+0000 7fcf56ffd640 1 -- 172.21.15.16:0/740446309 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf50010690 con 0x7fcf60102d50 2024-09-12T02:06:07.389 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.382+0000 7fcf67e9b640 1 -- 172.21.15.16:0/740446309 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf601a01d0 con 0x7fcf60102d50 2024-09-12T02:06:07.389 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.385+0000 7fcf6ac34740 1 Processor -- start 2024-09-12T02:06:07.389 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.385+0000 7fcf56ffd640 1 -- 172.21.15.16:0/740446309 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fcf500046c0 con 0x7fcf60102d50 2024-09-12T02:06:07.390 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.385+0000 7fcf6ac34740 1 -- start start 2024-09-12T02:06:07.396 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.386+0000 7fcf6ac34740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e4b77e99a0 0x55e4b77e9da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:07.396 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.386+0000 7fcf6ac34740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e4b77ea2e0 0x55e4b77eeda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:07.396 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.386+0000 7fcf6ac34740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e4b77ef370 0x55e4b77f1830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:07.396 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.386+0000 7fcf6ac34740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e4b774d2f0 con 0x55e4b77ef370 2024-09-12T02:06:07.397 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.386+0000 7fcf6ac34740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55e4b76f5270 con 0x55e4b77ea2e0 2024-09-12T02:06:07.397 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.386+0000 7fcf6ac34740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55e4b77492b0 con 0x55e4b77e99a0 2024-09-12T02:06:07.397 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.386+0000 7fcf65436640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e4b77ef370 0x55e4b77f1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:07.397 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.386+0000 7fcf65436640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e4b77ef370 0x55e4b77f1830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36114/0 (socket says 172.21.15.16:36114) 2024-09-12T02:06:07.397 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.386+0000 7fcf65c37640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e4b77ea2e0 0x55e4b77eeda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:07.397 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.386+0000 7fcf65436640 1 -- 172.21.15.16:0/2607069656 learned_addr learned my addr 172.21.15.16:0/2607069656 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:06:07.398 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.386+0000 7fcf66438640 1 --2- 172.21.15.16:0/2607069656 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e4b77e99a0 0x55e4b77e9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:07.398 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.386+0000 7fcf56ffd640 1 --2- 172.21.15.16:0/740446309 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fcf4c0a2cb0 0x7fcf4c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:07.398 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.386+0000 7fcf65c37640 1 -- 172.21.15.16:0/2607069656 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e4b77e99a0 msgr2=0x55e4b77e9da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:06:07.398 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.386+0000 7fcf65c37640 1 --2- 172.21.15.16:0/2607069656 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e4b77e99a0 0x55e4b77e9da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:07.398 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.386+0000 7fcf65c37640 1 -- 172.21.15.16:0/2607069656 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e4b77ef370 msgr2=0x55e4b77f1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:06:07.398 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.386+0000 7fcf65c37640 1 --2- 172.21.15.16:0/2607069656 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e4b77ef370 0x55e4b77f1830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:07.398 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.386+0000 7fcf65c37640 1 -- 172.21.15.16:0/2607069656 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4b76f4340 con 0x55e4b77ea2e0 2024-09-12T02:06:07.398 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.386+0000 7fcf65436640 1 --2- 172.21.15.16:0/2607069656 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e4b77ef370 0x55e4b77f1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:06:07.398 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.386+0000 7fcf66438640 1 --2- 172.21.15.16:0/740446309 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fcf4c0a2cb0 0x7fcf4c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:07.398 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.386+0000 7fcf56ffd640 1 -- 172.21.15.16:0/740446309 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fcf500cf490 con 0x7fcf60102d50 2024-09-12T02:06:07.399 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.387+0000 7fcf65c37640 1 --2- 172.21.15.16:0/2607069656 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e4b77ea2e0 0x55e4b77eeda0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fcf5c009b40 tx=0x7fcf5c0113e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19fd12bb03bd775d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:06:07.399 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.387+0000 7fcf4bfff640 1 -- 172.21.15.16:0/2607069656 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf5c011e10 con 0x55e4b77ea2e0 2024-09-12T02:06:07.399 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.387+0000 7fcf4bfff640 1 -- 172.21.15.16:0/2607069656 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf5c012930 con 0x55e4b77ea2e0 2024-09-12T02:06:07.399 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.387+0000 7fcf4bfff640 1 -- 172.21.15.16:0/2607069656 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf5c00fa60 con 0x55e4b77ea2e0 2024-09-12T02:06:07.399 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.387+0000 7fcf66438640 1 --2- 172.21.15.16:0/740446309 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fcf4c0a2cb0 0x7fcf4c0a5170 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fcf58004af0 tx=0x7fcf5803e040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:06:07.399 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.388+0000 7fcf6ac34740 1 -- 172.21.15.16:0/2607069656 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e4b77ea2e0 msgr2=0x55e4b77eeda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:07.399 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.388+0000 7fcf6ac34740 1 --2- 172.21.15.16:0/2607069656 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e4b77ea2e0 0x55e4b77eeda0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fcf5c009b40 tx=0x7fcf5c0113e0 comp rx=0 tx=0).stop 2024-09-12T02:06:07.399 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.388+0000 7fcf6ac34740 1 -- 172.21.15.16:0/2607069656 shutdown_connections 2024-09-12T02:06:07.399 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.388+0000 7fcf6ac34740 1 --2- 172.21.15.16:0/2607069656 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e4b77ef370 0x55e4b77f1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:07.399 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.388+0000 7fcf6ac34740 1 --2- 172.21.15.16:0/2607069656 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e4b77ea2e0 0x55e4b77eeda0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:07.399 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.388+0000 7fcf6ac34740 1 --2- 172.21.15.16:0/2607069656 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e4b77e99a0 0x55e4b77e9da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:07.400 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.388+0000 7fcf6ac34740 1 -- 172.21.15.16:0/2607069656 >> 172.21.15.16:0/2607069656 conn(0x55e4b77e5010 msgr2=0x55e4b77e5410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:06:07.400 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.388+0000 7fcf6ac34740 1 -- 172.21.15.16:0/2607069656 shutdown_connections 2024-09-12T02:06:07.400 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.388+0000 7fcf6ac34740 1 -- 172.21.15.16:0/2607069656 wait complete. 2024-09-12T02:06:07.400 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.388+0000 7fcf6ac34740 1 -- start start 2024-09-12T02:06:07.400 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.388+0000 7fcf6ac34740 1 Processor -- start 2024-09-12T02:06:07.400 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.389+0000 7fcf6ac34740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e4b77e99a0 0x55e4b77f3b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:07.400 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.389+0000 7fcf6ac34740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e4b77ea2e0 0x55e4b77f40a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:07.400 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.389+0000 7fcf6ac34740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e4b77ef370 0x55e4b77fb0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:07.400 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.389+0000 7fcf6ac34740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e4b77f2340 con 0x55e4b77ea2e0 2024-09-12T02:06:07.400 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.389+0000 7fcf6ac34740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55e4b76d6290 con 0x55e4b77e99a0 2024-09-12T02:06:07.400 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.389+0000 7fcf6ac34740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55e4b77f24c0 con 0x55e4b77ef370 2024-09-12T02:06:07.401 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.389+0000 7fcf6ac34740 10 client.0 _ll_register_callbacks cb 0x55e4b7707140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:06:07.401 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.389+0000 7fcf65436640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e4b77ea2e0 0x55e4b77f40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:07.401 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.389+0000 7fcf65436640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e4b77ea2e0 0x55e4b77f40a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36128/0 (socket says 172.21.15.16:36128) 2024-09-12T02:06:07.401 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.389+0000 7fcf65436640 1 -- 172.21.15.16:0/4070692266 learned_addr learned my addr 172.21.15.16:0/4070692266 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:06:07.401 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.389+0000 7fcf66438640 1 --2- 172.21.15.16:0/4070692266 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e4b77ef370 0x55e4b77fb0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:07.401 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.389+0000 7fcf65c37640 1 --2- 172.21.15.16:0/4070692266 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e4b77e99a0 0x55e4b77f3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:07.401 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.389+0000 7fcf65436640 1 -- 172.21.15.16:0/4070692266 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e4b77ef370 msgr2=0x55e4b77fb0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:06:07.402 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.389+0000 7fcf65436640 1 --2- 172.21.15.16:0/4070692266 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e4b77ef370 0x55e4b77fb0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:07.402 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.389+0000 7fcf65436640 1 -- 172.21.15.16:0/4070692266 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e4b77e99a0 msgr2=0x55e4b77f3b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:06:07.402 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.389+0000 7fcf65436640 1 --2- 172.21.15.16:0/4070692266 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e4b77e99a0 0x55e4b77f3b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:07.402 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.389+0000 7fcf65436640 1 -- 172.21.15.16:0/4070692266 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e4b7460210 con 0x55e4b77ea2e0 2024-09-12T02:06:07.402 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.389+0000 7fcf65436640 1 --2- 172.21.15.16:0/4070692266 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e4b77ea2e0 0x55e4b77f40a0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fcf500a44d0 tx=0x7fcf500056b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31934a2733861a6c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:06:07.402 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.389+0000 7fcf497fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:06:07.402 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.389+0000 7fcf497fa640 1 -- 172.21.15.16:0/4070692266 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf5009b8d0 con 0x55e4b77ea2e0 2024-09-12T02:06:07.402 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.389+0000 7fcf497fa640 1 -- 172.21.15.16:0/4070692266 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fcf500cf0d0 con 0x55e4b77ea2e0 2024-09-12T02:06:07.402 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.389+0000 7fcf6ac34740 1 -- 172.21.15.16:0/4070692266 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf5c012930 con 0x55e4b77ea2e0 2024-09-12T02:06:07.402 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.390+0000 7fcf497fa640 1 -- 172.21.15.16:0/4070692266 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fcf5009b6f0 con 0x55e4b77ea2e0 2024-09-12T02:06:07.402 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.390+0000 7fcf6ac34740 10 client.16743 fetch_fsmap learned FSMap version 25 2024-09-12T02:06:07.403 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.390+0000 7fcf6ac34740 1 -- 172.21.15.16:0/4070692266 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e4b752c620 con 0x55e4b77ea2e0 2024-09-12T02:06:07.403 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.390+0000 7fcf497fa640 1 -- 172.21.15.16:0/4070692266 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf5c012930 con 0x55e4b77ea2e0 2024-09-12T02:06:07.403 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.390+0000 7fcf497fa640 1 -- 172.21.15.16:0/4070692266 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fcf500b3030 con 0x55e4b77ea2e0 2024-09-12T02:06:07.403 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.390+0000 7fcf6ac34740 10 client.16743 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:06:07.403 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.390+0000 7fcf6ac34740 10 client.16743 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:06:07.403 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.390+0000 7fcf6ac34740 20 client.16743 populate_metadata read hostname 'smithi016' 2024-09-12T02:06:07.403 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.390+0000 7fcf6ac34740 1 --2- 172.21.15.16:0/4070692266 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55e4b77ff880 0x55e4b7801cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:07.403 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.390+0000 7fcf6ac34740 4 client.16743 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:06:07.403 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.390+0000 7fcf6ac34740 1 -- 172.21.15.16:0/4070692266 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e4b775aa20 con 0x55e4b77ff880 2024-09-12T02:06:07.403 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.390+0000 7fcf65c37640 1 --2- 172.21.15.16:0/4070692266 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55e4b77ff880 0x55e4b7801cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:07.403 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.391+0000 7fcf497fa640 10 client.16743 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:06:07.403 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.391+0000 7fcf65c37640 1 --2- 172.21.15.16:0/4070692266 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55e4b77ff880 0x55e4b7801cd0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7c673de6cabc7a3 server_cookie=c70ba7d30042d8a7 in_seq=0 out_seq=0 2024-09-12T02:06:07.404 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.392+0000 7fcf497fa640 1 -- 172.21.15.16:0/4070692266 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e4b775aa20 con 0x55e4b77ff880 2024-09-12T02:06:07.404 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.392+0000 7fcf497fa640 10 client.16743 handle_command_reply: tid=0 2024-09-12T02:06:07.404 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.392+0000 7fcf6ac34740 1 client.16743 shutdown 2024-09-12T02:06:07.404 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.392+0000 7fcf6ac34740 1 -- 172.21.15.16:0/4070692266 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e4b77ea2e0 msgr2=0x55e4b77f40a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:07.404 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.392+0000 7fcf6ac34740 1 --2- 172.21.15.16:0/4070692266 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e4b77ea2e0 0x55e4b77f40a0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fcf500a44d0 tx=0x7fcf500056b0 comp rx=0 tx=0).stop 2024-09-12T02:06:07.404 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.392+0000 7fcf6ac34740 1 -- 172.21.15.16:0/4070692266 shutdown_connections 2024-09-12T02:06:07.404 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.392+0000 7fcf6ac34740 1 --2- 172.21.15.16:0/4070692266 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55e4b77ff880 0x55e4b7801cd0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:07.404 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.392+0000 7fcf6ac34740 1 --2- 172.21.15.16:0/4070692266 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e4b77ef370 0x55e4b77fb0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:07.404 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.392+0000 7fcf6ac34740 1 --2- 172.21.15.16:0/4070692266 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e4b77ea2e0 0x55e4b77f40a0 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:07.404 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.392+0000 7fcf6ac34740 1 --2- 172.21.15.16:0/4070692266 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e4b77e99a0 0x55e4b77f3b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:07.404 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.392+0000 7fcf497fa640 0 client.16743 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:06:07.405 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.392+0000 7fcf6ac34740 1 -- 172.21.15.16:0/4070692266 >> 172.21.15.16:0/4070692266 conn(0x55e4b77e5010 msgr2=0x55e4b77e6820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:06:07.405 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.392+0000 7fcf6ac34740 1 -- 172.21.15.16:0/4070692266 shutdown_connections 2024-09-12T02:06:07.405 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.392+0000 7fcf6ac34740 1 -- 172.21.15.16:0/4070692266 wait complete. 2024-09-12T02:06:07.405 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.392+0000 7fcf6ac34740 20 client.16743 trim_cache size 0 max 16384 2024-09-12T02:06:07.412 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.407+0000 7fcf6ac34740 1 Processor -- start 2024-09-12T02:06:07.412 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.407+0000 7fcf6ac34740 1 -- start start 2024-09-12T02:06:07.412 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.407+0000 7fcf6ac34740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e4b77e99a0 0x55e4b77f2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:07.412 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.407+0000 7fcf6ac34740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e4b77ea2e0 0x55e4b77fb0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:07.412 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.407+0000 7fcf6ac34740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e4b77ef370 0x55e4b77e5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:07.412 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.407+0000 7fcf6ac34740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fcf601ab770 con 0x55e4b77ea2e0 2024-09-12T02:06:07.412 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.407+0000 7fcf6ac34740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fcf601a9040 con 0x55e4b77ef370 2024-09-12T02:06:07.412 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.407+0000 7fcf6ac34740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fcf601a6eb0 con 0x55e4b77e99a0 2024-09-12T02:06:07.412 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.407+0000 7fcf65436640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e4b77ea2e0 0x55e4b77fb0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:07.413 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.407+0000 7fcf65436640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e4b77ea2e0 0x55e4b77fb0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36144/0 (socket says 172.21.15.16:36144) 2024-09-12T02:06:07.413 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.407+0000 7fcf65436640 1 -- 172.21.15.16:0/2449196595 learned_addr learned my addr 172.21.15.16:0/2449196595 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:06:07.413 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.407+0000 7fcf65c37640 1 --2- 172.21.15.16:0/2449196595 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e4b77e99a0 0x55e4b77f2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:07.413 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.407+0000 7fcf65436640 1 -- 172.21.15.16:0/2449196595 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e4b77e99a0 msgr2=0x55e4b77f2fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:06:07.413 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.407+0000 7fcf65436640 1 --2- 172.21.15.16:0/2449196595 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e4b77e99a0 0x55e4b77f2fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:07.413 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.407+0000 7fcf65436640 1 -- 172.21.15.16:0/2449196595 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e4b77ef370 msgr2=0x55e4b77e5ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:06:07.413 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.407+0000 7fcf65436640 1 --2- 172.21.15.16:0/2449196595 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e4b77ef370 0x55e4b77e5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:07.413 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.407+0000 7fcf65436640 1 -- 172.21.15.16:0/2449196595 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4b77e02d0 con 0x55e4b77ea2e0 2024-09-12T02:06:07.413 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.408+0000 7fcf65436640 1 --2- 172.21.15.16:0/2449196595 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e4b77ea2e0 0x55e4b77fb0a0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fcf500981f0 tx=0x7fcf50004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9002df1819002433 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:06:07.413 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.408+0000 7fcf4b7fe640 1 -- 172.21.15.16:0/2449196595 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf5009b8d0 con 0x55e4b77ea2e0 2024-09-12T02:06:07.413 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.408+0000 7fcf4b7fe640 1 -- 172.21.15.16:0/2449196595 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf5009bd90 con 0x55e4b77ea2e0 2024-09-12T02:06:07.414 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.408+0000 7fcf4b7fe640 1 -- 172.21.15.16:0/2449196595 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf500a0140 con 0x55e4b77ea2e0 2024-09-12T02:06:07.414 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.408+0000 7fcf6ac34740 1 -- 172.21.15.16:0/2449196595 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e4b77ea2e0 msgr2=0x55e4b77fb0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:07.414 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.408+0000 7fcf6ac34740 1 --2- 172.21.15.16:0/2449196595 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e4b77ea2e0 0x55e4b77fb0a0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fcf500981f0 tx=0x7fcf50004a40 comp rx=0 tx=0).stop 2024-09-12T02:06:07.414 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.408+0000 7fcf6ac34740 1 -- 172.21.15.16:0/2449196595 shutdown_connections 2024-09-12T02:06:07.414 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.408+0000 7fcf6ac34740 1 --2- 172.21.15.16:0/2449196595 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e4b77ef370 0x55e4b77e5ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:07.414 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.408+0000 7fcf6ac34740 1 --2- 172.21.15.16:0/2449196595 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e4b77ea2e0 0x55e4b77fb0a0 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:07.414 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.408+0000 7fcf6ac34740 1 --2- 172.21.15.16:0/2449196595 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e4b77e99a0 0x55e4b77f2fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:07.414 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.408+0000 7fcf6ac34740 1 -- 172.21.15.16:0/2449196595 >> 172.21.15.16:0/2449196595 conn(0x55e4b77e5010 msgr2=0x55e4b77f28c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:06:07.414 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.408+0000 7fcf6ac34740 1 -- 172.21.15.16:0/2449196595 shutdown_connections 2024-09-12T02:06:07.414 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.409+0000 7fcf6ac34740 1 -- 172.21.15.16:0/2449196595 wait complete. 2024-09-12T02:06:07.414 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.409+0000 7fcf6ac34740 1 -- start start 2024-09-12T02:06:07.415 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.409+0000 7fcf6ac34740 1 Processor -- start 2024-09-12T02:06:07.415 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.409+0000 7fcf6ac34740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e4b77e99a0 0x55e4b77f8590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:07.415 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.409+0000 7fcf6ac34740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e4b77ea2e0 0x55e4b77f8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:07.415 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.409+0000 7fcf6ac34740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e4b77ef370 0x55e4b77f9ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:07.415 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.409+0000 7fcf6ac34740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fcf601a7500 con 0x55e4b77e99a0 2024-09-12T02:06:07.415 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.409+0000 7fcf6ac34740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fcf601a8970 con 0x55e4b77ef370 2024-09-12T02:06:07.415 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.409+0000 7fcf6ac34740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fcf601a7bd0 con 0x55e4b77ea2e0 2024-09-12T02:06:07.415 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.409+0000 7fcf6ac34740 10 client.0 _ll_register_callbacks cb 0x7fcf5000e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:06:07.415 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.409+0000 7fcf65c37640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e4b77e99a0 0x55e4b77f8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:07.415 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.409+0000 7fcf65c37640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e4b77e99a0 0x55e4b77f8590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36152/0 (socket says 172.21.15.16:36152) 2024-09-12T02:06:07.415 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.410+0000 7fcf65c37640 1 -- 172.21.15.16:0/3710637181 learned_addr learned my addr 172.21.15.16:0/3710637181 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:06:07.416 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.410+0000 7fcf65436640 1 --2- 172.21.15.16:0/3710637181 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e4b77ea2e0 0x55e4b77f8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:07.416 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.410+0000 7fcf66438640 1 --2- 172.21.15.16:0/3710637181 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e4b77ef370 0x55e4b77f9ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:07.416 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.410+0000 7fcf65c37640 1 -- 172.21.15.16:0/3710637181 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e4b77ea2e0 msgr2=0x55e4b77f8ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:06:07.416 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.410+0000 7fcf65c37640 1 --2- 172.21.15.16:0/3710637181 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e4b77ea2e0 0x55e4b77f8ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:07.416 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.410+0000 7fcf65c37640 1 -- 172.21.15.16:0/3710637181 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e4b77ef370 msgr2=0x55e4b77f9ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:06:07.416 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.410+0000 7fcf65c37640 1 --2- 172.21.15.16:0/3710637181 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e4b77ef370 0x55e4b77f9ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:07.416 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.410+0000 7fcf65c37640 1 -- 172.21.15.16:0/3710637181 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e4b77fa0f0 con 0x55e4b77e99a0 2024-09-12T02:06:07.416 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.410+0000 7fcf66438640 1 --2- 172.21.15.16:0/3710637181 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e4b77ef370 0x55e4b77f9ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:06:07.416 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.410+0000 7fcf65c37640 1 --2- 172.21.15.16:0/3710637181 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e4b77e99a0 0x55e4b77f8590 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fcf5c011ef0 tx=0x7fcf5c009530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3af6eaf8e59e16c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:06:07.416 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.410+0000 7fcf497fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:06:07.416 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.410+0000 7fcf497fa640 1 -- 172.21.15.16:0/3710637181 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf5c011c10 con 0x55e4b77e99a0 2024-09-12T02:06:07.417 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.410+0000 7fcf497fa640 1 -- 172.21.15.16:0/3710637181 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fcf5c0054c0 con 0x55e4b77e99a0 2024-09-12T02:06:07.417 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.410+0000 7fcf6ac34740 1 -- 172.21.15.16:0/3710637181 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf5009bd90 con 0x55e4b77e99a0 2024-09-12T02:06:07.417 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.410+0000 7fcf497fa640 1 -- 172.21.15.16:0/3710637181 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fcf5c0056c0 con 0x55e4b77e99a0 2024-09-12T02:06:07.417 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.410+0000 7fcf6ac34740 10 client.16749 fetch_fsmap learned FSMap version 25 2024-09-12T02:06:07.417 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.410+0000 7fcf6ac34740 1 -- 172.21.15.16:0/3710637181 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e4b786a5b0 con 0x55e4b77e99a0 2024-09-12T02:06:07.417 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.411+0000 7fcf497fa640 1 -- 172.21.15.16:0/3710637181 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf5009bd90 con 0x55e4b77e99a0 2024-09-12T02:06:07.417 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.411+0000 7fcf497fa640 1 -- 172.21.15.16:0/3710637181 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fcf5c0126c0 con 0x55e4b77e99a0 2024-09-12T02:06:07.417 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.411+0000 7fcf6ac34740 10 client.16749 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:06:07.417 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.411+0000 7fcf6ac34740 10 client.16749 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:06:07.417 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.411+0000 7fcf6ac34740 20 client.16749 populate_metadata read hostname 'smithi016' 2024-09-12T02:06:07.417 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.411+0000 7fcf6ac34740 1 --2- 172.21.15.16:0/3710637181 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55e4b786aca0 0x55e4b786d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:07.418 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.411+0000 7fcf6ac34740 4 client.16749 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:06:07.418 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.411+0000 7fcf6ac34740 1 -- 172.21.15.16:0/3710637181 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e4b7754d70 con 0x55e4b786aca0 2024-09-12T02:06:07.418 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.411+0000 7fcf65436640 1 --2- 172.21.15.16:0/3710637181 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55e4b786aca0 0x55e4b786d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:07.418 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.411+0000 7fcf497fa640 10 client.16749 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:06:07.418 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.411+0000 7fcf65436640 1 --2- 172.21.15.16:0/3710637181 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55e4b786aca0 0x55e4b786d090 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af0e730bff0f2639 server_cookie=df74c36757c1e81e in_seq=0 out_seq=0 2024-09-12T02:06:07.424 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.424+0000 7fcf497fa640 1 -- 172.21.15.16:0/3710637181 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e4b7754d70 con 0x55e4b786aca0 2024-09-12T02:06:07.424 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.424+0000 7fcf497fa640 10 client.16749 handle_command_reply: tid=0 2024-09-12T02:06:07.424 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.424+0000 7fcf6ac34740 1 client.16749 shutdown 2024-09-12T02:06:07.424 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.424+0000 7fcf6ac34740 1 -- 172.21.15.16:0/3710637181 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e4b77e99a0 msgr2=0x55e4b77f8590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:07.425 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.424+0000 7fcf6ac34740 1 --2- 172.21.15.16:0/3710637181 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e4b77e99a0 0x55e4b77f8590 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fcf5c011ef0 tx=0x7fcf5c009530 comp rx=0 tx=0).stop 2024-09-12T02:06:07.425 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.424+0000 7fcf6ac34740 1 -- 172.21.15.16:0/3710637181 shutdown_connections 2024-09-12T02:06:07.425 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.424+0000 7fcf6ac34740 1 --2- 172.21.15.16:0/3710637181 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55e4b786aca0 0x55e4b786d090 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:07.425 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.424+0000 7fcf6ac34740 1 --2- 172.21.15.16:0/3710637181 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e4b77ef370 0x55e4b77f9ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:07.425 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.424+0000 7fcf6ac34740 1 --2- 172.21.15.16:0/3710637181 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e4b77ea2e0 0x55e4b77f8ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:07.425 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.424+0000 7fcf6ac34740 1 --2- 172.21.15.16:0/3710637181 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e4b77e99a0 0x55e4b77f8590 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:07.425 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.424+0000 7fcf497fa640 0 client.16749 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:06:07.425 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.424+0000 7fcf6ac34740 1 -- 172.21.15.16:0/3710637181 >> 172.21.15.16:0/3710637181 conn(0x55e4b77e5010 msgr2=0x55e4b77e39a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:06:07.425 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.424+0000 7fcf6ac34740 1 -- 172.21.15.16:0/3710637181 shutdown_connections 2024-09-12T02:06:07.425 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.424+0000 7fcf6ac34740 1 -- 172.21.15.16:0/3710637181 wait complete. 2024-09-12T02:06:07.425 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.424+0000 7fcf6ac34740 20 client.16749 trim_cache size 0 max 16384 2024-09-12T02:06:07.426 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-09-12T02:06:07.426 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-09-12T02:06:07.426 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-09-12T02:06:07.426 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-09-12T02:06:07.427 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.425+0000 7fcf67e9b640 1 -- 172.21.15.16:0/740446309 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fcf4c0a2cb0 msgr2=0x7fcf4c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:07.427 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.425+0000 7fcf67e9b640 1 --2- 172.21.15.16:0/740446309 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fcf4c0a2cb0 0x7fcf4c0a5170 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fcf58004af0 tx=0x7fcf5803e040 comp rx=0 tx=0).stop 2024-09-12T02:06:07.427 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.425+0000 7fcf67e9b640 1 -- 172.21.15.16:0/740446309 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf60102d50 msgr2=0x7fcf60198310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:07.427 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.425+0000 7fcf67e9b640 1 --2- 172.21.15.16:0/740446309 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf60102d50 0x7fcf60198310 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fcf5000e9f0 tx=0x7fcf5000edf0 comp rx=0 tx=0).stop 2024-09-12T02:06:07.427 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.425+0000 7fcf67e9b640 1 -- 172.21.15.16:0/740446309 shutdown_connections 2024-09-12T02:06:07.427 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.425+0000 7fcf67e9b640 1 --2- 172.21.15.16:0/740446309 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fcf4c0a2cb0 0x7fcf4c0a5170 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:07.428 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.425+0000 7fcf67e9b640 1 --2- 172.21.15.16:0/740446309 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcf60103710 0x7fcf6019f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:07.428 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.425+0000 7fcf67e9b640 1 --2- 172.21.15.16:0/740446309 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf60102d50 0x7fcf60198310 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:07.428 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.425+0000 7fcf67e9b640 1 --2- 172.21.15.16:0/740446309 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf60101b60 0x7fcf60197dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:07.428 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.425+0000 7fcf67e9b640 1 -- 172.21.15.16:0/740446309 >> 172.21.15.16:0/740446309 conn(0x7fcf600fd2e0 msgr2=0x7fcf60105ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:06:07.428 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.425+0000 7fcf67e9b640 1 -- 172.21.15.16:0/740446309 shutdown_connections 2024-09-12T02:06:07.428 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:07.425+0000 7fcf67e9b640 1 -- 172.21.15.16:0/740446309 wait complete. 2024-09-12T02:06:08.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:06:08 smithi184 ceph-mon[35593]: pgmap v461: 97 pgs: 97 active+clean; 2.2 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-09-12T02:06:08.418 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:06:08 smithi033 ceph-mon[35332]: pgmap v461: 97 pgs: 97 active+clean; 2.2 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-09-12T02:06:08.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:06:08 smithi016 ceph-mon[30947]: pgmap v461: 97 pgs: 97 active+clean; 2.2 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-09-12T02:06:10.365 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:06:10 smithi016 ceph-mon[30947]: pgmap v462: 97 pgs: 97 active+clean; 2.2 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 4.5 KiB/s wr, 8 op/s 2024-09-12T02:06:10.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:06:10 smithi184 ceph-mon[35593]: pgmap v462: 97 pgs: 97 active+clean; 2.2 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 4.5 KiB/s wr, 8 op/s 2024-09-12T02:06:10.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:06:10 smithi033 ceph-mon[35332]: pgmap v462: 97 pgs: 97 active+clean; 2.2 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 4.5 KiB/s wr, 8 op/s 2024-09-12T02:06:12.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:06:12 smithi184 ceph-mon[35593]: pgmap v463: 97 pgs: 97 active+clean; 2.2 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 5.7 KiB/s wr, 12 op/s 2024-09-12T02:06:12.418 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:06:12 smithi033 ceph-mon[35332]: pgmap v463: 97 pgs: 97 active+clean; 2.2 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 5.7 KiB/s wr, 12 op/s 2024-09-12T02:06:12.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:06:12 smithi016 ceph-mon[30947]: pgmap v463: 97 pgs: 97 active+clean; 2.2 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 5.7 KiB/s wr, 12 op/s 2024-09-12T02:06:12.982 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:06:13.253 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:06:13.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:13.744+0000 7f133e88c640 1 -- 172.21.15.16:0/3412038073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1338106a80 msgr2=0x7f1338106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:13.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:13.744+0000 7f133e88c640 1 --2- 172.21.15.16:0/3412038073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1338106a80 0x7f1338106e60 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f1328009930 tx=0x7f132802a420 comp rx=0 tx=0).stop 2024-09-12T02:06:13.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:13.744+0000 7f133e88c640 1 -- 172.21.15.16:0/3412038073 shutdown_connections 2024-09-12T02:06:13.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:13.744+0000 7f133e88c640 1 --2- 172.21.15.16:0/3412038073 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1338101420 0x7f133810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:13.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:13.744+0000 7f133e88c640 1 --2- 172.21.15.16:0/3412038073 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1338100a80 0x7f1338100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:13.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:13.744+0000 7f133e88c640 1 --2- 172.21.15.16:0/3412038073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1338106a80 0x7f1338106e60 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:13.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:13.744+0000 7f133e88c640 1 -- 172.21.15.16:0/3412038073 >> 172.21.15.16:0/3412038073 conn(0x7f13380fc7a0 msgr2=0x7f13380febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:06:13.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:13.744+0000 7f133e88c640 1 -- 172.21.15.16:0/3412038073 shutdown_connections 2024-09-12T02:06:13.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:13.744+0000 7f133e88c640 1 -- 172.21.15.16:0/3412038073 wait complete. 2024-09-12T02:06:13.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:13.745+0000 7f133e88c640 1 Processor -- start 2024-09-12T02:06:13.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:13.745+0000 7f133e88c640 1 -- start start 2024-09-12T02:06:13.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:13.745+0000 7f133e88c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1338100a80 0x7f13381984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:13.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:13.745+0000 7f133e88c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1338101420 0x7f1338198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:13.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:13.745+0000 7f133e88c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1338106a80 0x7f133819cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:13.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:13.745+0000 7f133e88c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1338110e80 con 0x7f1338101420 2024-09-12T02:06:13.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:13.745+0000 7f133e88c640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1338110d00 con 0x7f1338106a80 2024-09-12T02:06:13.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:13.745+0000 7f133e88c640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1338111000 con 0x7f1338100a80 2024-09-12T02:06:13.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:13.746+0000 7f13377fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1338101420 0x7f1338198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:13.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:13.746+0000 7f13377fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1338101420 0x7f1338198a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34626/0 (socket says 172.21.15.16:34626) 2024-09-12T02:06:13.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:13.746+0000 7f13377fe640 1 -- 172.21.15.16:0/433035893 learned_addr learned my addr 172.21.15.16:0/433035893 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:06:13.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:13.746+0000 7f133ce29640 1 --2- 172.21.15.16:0/433035893 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1338106a80 0x7f133819cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:13.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:13.746+0000 7f1337fff640 1 --2- 172.21.15.16:0/433035893 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1338100a80 0x7f13381984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:13.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:13.746+0000 7f13377fe640 1 -- 172.21.15.16:0/433035893 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1338100a80 msgr2=0x7f13381984c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:13.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:13.746+0000 7f13377fe640 1 --2- 172.21.15.16:0/433035893 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1338100a80 0x7f13381984c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:13.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:13.746+0000 7f13377fe640 1 -- 172.21.15.16:0/433035893 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1338106a80 msgr2=0x7f133819cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:13.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:13.746+0000 7f13377fe640 1 --2- 172.21.15.16:0/433035893 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1338106a80 0x7f133819cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:13.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:13.746+0000 7f13377fe640 1 -- 172.21.15.16:0/433035893 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f133819d470 con 0x7f1338101420 2024-09-12T02:06:13.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:13.746+0000 7f133ce29640 1 --2- 172.21.15.16:0/433035893 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1338106a80 0x7f133819cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:06:13.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:13.746+0000 7f13377fe640 1 --2- 172.21.15.16:0/433035893 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1338101420 0x7f1338198a00 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f132c00e9f0 tx=0x7f132c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:06:13.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:13.746+0000 7f13357fa640 1 -- 172.21.15.16:0/433035893 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f132c00cd80 con 0x7f1338101420 2024-09-12T02:06:13.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:13.746+0000 7f13357fa640 1 -- 172.21.15.16:0/433035893 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f132c005500 con 0x7f1338101420 2024-09-12T02:06:13.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:13.746+0000 7f133e88c640 1 -- 172.21.15.16:0/433035893 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f133819d760 con 0x7f1338101420 2024-09-12T02:06:13.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:13.746+0000 7f13357fa640 1 -- 172.21.15.16:0/433035893 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f132c010690 con 0x7f1338101420 2024-09-12T02:06:13.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:13.746+0000 7f133e88c640 1 -- 172.21.15.16:0/433035893 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13381a5040 con 0x7f1338101420 2024-09-12T02:06:13.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:13.747+0000 7f133e88c640 1 -- 172.21.15.16:0/433035893 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1338100ee0 con 0x7f1338101420 2024-09-12T02:06:13.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:13.749+0000 7f13357fa640 1 -- 172.21.15.16:0/433035893 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f132c0046c0 con 0x7f1338101420 2024-09-12T02:06:13.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:13.749+0000 7f13357fa640 1 --2- 172.21.15.16:0/433035893 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f13200a2b80 0x7f13200a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:13.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:13.749+0000 7f1337fff640 1 --2- 172.21.15.16:0/433035893 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f13200a2b80 0x7f13200a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:13.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:13.749+0000 7f13357fa640 1 -- 172.21.15.16:0/433035893 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f132c0cf750 con 0x7f1338101420 2024-09-12T02:06:13.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:13.750+0000 7f1337fff640 1 --2- 172.21.15.16:0/433035893 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f13200a2b80 0x7f13200a5040 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f1328000c00 tx=0x7f1328005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:06:13.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:13.752+0000 7f13357fa640 1 -- 172.21.15.16:0/433035893 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f132c0541b0 con 0x7f1338101420 2024-09-12T02:06:13.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:13.964+0000 7f133e88c640 1 -- 172.21.15.16:0/433035893 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1338199820 con 0x7f1338101420 2024-09-12T02:06:13.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:13.964+0000 7f13357fa640 1 -- 172.21.15.16:0/433035893 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11911 (secure 0 0 0) 0x7f1338199820 con 0x7f1338101420 2024-09-12T02:06:13.965 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T02:06:13.965 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":25,"btime":"2024-09-12T02:01:37:192021+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3279123279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3279123279},{"type":"v1","addr":"172.21.15.16:6835","nonce":3279123279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4119452382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4119452382},{"type":"v1","addr":"172.21.15.16:6837","nonce":4119452382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/3263823165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3263823165},{"type":"v1","addr":"172.21.15.16:6839","nonce":3263823165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/1510290732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":1510290732},{"type":"v1","addr":"172.21.15.16:6841","nonce":1510290732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/279674603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":279674603},{"type":"v1","addr":"172.21.15.33:6833","nonce":279674603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3555728028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3555728028},{"type":"v1","addr":"172.21.15.33:6835","nonce":3555728028}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T01:59:56.117608+0000","modified":"2024-09-12T02:01:36.294760+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24482,"mds_1":24481,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6833/1563374106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1563374106},{"type":"v1","addr":"172.21.15.184:6833","nonce":1563374106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6837/1892833868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1892833868},{"type":"v1","addr":"172.21.15.33:6837","nonce":1892833868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/1457679686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1457679686},{"type":"v1","addr":"172.21.15.184:6835","nonce":1457679686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.184:6837/1793393087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":1793393087},{"type":"v1","addr":"172.21.15.184:6837","nonce":1793393087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.184:6839/4203347941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":4203347941},{"type":"v1","addr":"172.21.15.184:6839","nonce":4203347941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.33:6839/4136670204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":4136670204},{"type":"v1","addr":"172.21.15.33:6839","nonce":4136670204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24481,24482,24469]},"id":1}]} 2024-09-12T02:06:13.965 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 25 2024-09-12T02:06:13.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:13.966+0000 7f133e88c640 1 -- 172.21.15.16:0/433035893 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f13200a2b80 msgr2=0x7f13200a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:13.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:13.966+0000 7f133e88c640 1 --2- 172.21.15.16:0/433035893 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f13200a2b80 0x7f13200a5040 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f1328000c00 tx=0x7f1328005a20 comp rx=0 tx=0).stop 2024-09-12T02:06:13.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:13.966+0000 7f133e88c640 1 -- 172.21.15.16:0/433035893 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1338101420 msgr2=0x7f1338198a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:13.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:13.966+0000 7f133e88c640 1 --2- 172.21.15.16:0/433035893 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1338101420 0x7f1338198a00 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f132c00e9f0 tx=0x7f132c00edf0 comp rx=0 tx=0).stop 2024-09-12T02:06:13.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:13.967+0000 7f133e88c640 1 -- 172.21.15.16:0/433035893 shutdown_connections 2024-09-12T02:06:13.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:13.967+0000 7f133e88c640 1 --2- 172.21.15.16:0/433035893 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f13200a2b80 0x7f13200a5040 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:13.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:13.967+0000 7f133e88c640 1 --2- 172.21.15.16:0/433035893 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1338106a80 0x7f133819cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:13.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:13.967+0000 7f133e88c640 1 --2- 172.21.15.16:0/433035893 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1338101420 0x7f1338198a00 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:13.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:13.967+0000 7f133e88c640 1 --2- 172.21.15.16:0/433035893 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1338100a80 0x7f13381984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:13.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:13.967+0000 7f133e88c640 1 -- 172.21.15.16:0/433035893 >> 172.21.15.16:0/433035893 conn(0x7f13380fc7a0 msgr2=0x7f133806aad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:06:13.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:13.967+0000 7f133e88c640 1 -- 172.21.15.16:0/433035893 shutdown_connections 2024-09-12T02:06:13.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:13.967+0000 7f133e88c640 1 -- 172.21.15.16:0/433035893 wait complete. 2024-09-12T02:06:14.273 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:06:14 smithi016 ceph-mon[30947]: pgmap v464: 97 pgs: 97 active+clean; 2.2 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.5 KiB/s wr, 8 op/s 2024-09-12T02:06:14.273 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:06:14 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/433035893' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:06:14.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:06:14 smithi184 ceph-mon[35593]: pgmap v464: 97 pgs: 97 active+clean; 2.2 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.5 KiB/s wr, 8 op/s 2024-09-12T02:06:14.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:06:14 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/433035893' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:06:14.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:06:14 smithi033 ceph-mon[35332]: pgmap v464: 97 pgs: 97 active+clean; 2.2 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.5 KiB/s wr, 8 op/s 2024-09-12T02:06:14.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:06:14 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/433035893' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:06:14.718 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24482 quiesce db --set-id ee2e48a6 --release --await' 2024-09-12T02:06:14.719 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24482 quiesce db --set-id ee2e48a6 --release --await 2024-09-12T02:06:14.984 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:06:15.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.442+0000 7f800ddd3640 1 -- 172.21.15.16:0/2101245385 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8008102d90 msgr2=0x7f8008103170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:15.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.442+0000 7f800ddd3640 1 --2- 172.21.15.16:0/2101245385 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8008102d90 0x7f8008103170 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f7ff8009930 tx=0x7f7ff802a420 comp rx=0 tx=0).stop 2024-09-12T02:06:15.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.442+0000 7f800ddd3640 1 -- 172.21.15.16:0/2101245385 shutdown_connections 2024-09-12T02:06:15.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.442+0000 7f800ddd3640 1 --2- 172.21.15.16:0/2101245385 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8008108290 0x7f800810ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:15.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.442+0000 7f800ddd3640 1 --2- 172.21.15.16:0/2101245385 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8008103740 0x7f8008107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:15.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.442+0000 7f800ddd3640 1 --2- 172.21.15.16:0/2101245385 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8008102d90 0x7f8008103170 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:15.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.442+0000 7f800ddd3640 1 -- 172.21.15.16:0/2101245385 >> 172.21.15.16:0/2101245385 conn(0x7f80080fc7d0 msgr2=0x7f80080febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:06:15.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.442+0000 7f800ddd3640 1 -- 172.21.15.16:0/2101245385 shutdown_connections 2024-09-12T02:06:15.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.442+0000 7f800ddd3640 1 -- 172.21.15.16:0/2101245385 wait complete. 2024-09-12T02:06:15.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.442+0000 7f800ddd3640 1 Processor -- start 2024-09-12T02:06:15.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.443+0000 7f800ddd3640 1 -- start start 2024-09-12T02:06:15.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.443+0000 7f800ddd3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8008102d90 0x7f80081a0590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:15.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.443+0000 7f800ddd3640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8008103740 0x7f80081a0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:15.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.443+0000 7f800ddd3640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8008108290 0x7f800819a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:15.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.443+0000 7f800ddd3640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f80081120d0 con 0x7f8008102d90 2024-09-12T02:06:15.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.443+0000 7f800ddd3640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8008111f50 con 0x7f8008108290 2024-09-12T02:06:15.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.443+0000 7f800ddd3640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8008112250 con 0x7f8008103740 2024-09-12T02:06:15.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.443+0000 7f80077fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8008102d90 0x7f80081a0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:15.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.443+0000 7f80077fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8008102d90 0x7f80081a0590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34646/0 (socket says 172.21.15.16:34646) 2024-09-12T02:06:15.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.443+0000 7f80077fe640 1 -- 172.21.15.16:0/629681311 learned_addr learned my addr 172.21.15.16:0/629681311 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:06:15.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.443+0000 7f8006ffd640 1 --2- 172.21.15.16:0/629681311 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8008103740 0x7f80081a0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:15.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.443+0000 7f8007fff640 1 --2- 172.21.15.16:0/629681311 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8008108290 0x7f800819a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:15.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.443+0000 7f80077fe640 1 -- 172.21.15.16:0/629681311 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8008103740 msgr2=0x7f80081a0ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:15.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.443+0000 7f80077fe640 1 --2- 172.21.15.16:0/629681311 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8008103740 0x7f80081a0ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:15.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.443+0000 7f80077fe640 1 -- 172.21.15.16:0/629681311 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8008108290 msgr2=0x7f800819a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:15.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.443+0000 7f80077fe640 1 --2- 172.21.15.16:0/629681311 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8008108290 0x7f800819a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:15.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.443+0000 7f80077fe640 1 -- 172.21.15.16:0/629681311 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f800819aed0 con 0x7f8008102d90 2024-09-12T02:06:15.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.444+0000 7f80077fe640 1 --2- 172.21.15.16:0/629681311 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8008102d90 0x7f80081a0590 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f7ff8009230 tx=0x7f7ff8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:06:15.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.444+0000 7f8004ff9640 1 -- 172.21.15.16:0/629681311 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ff803f030 con 0x7f8008102d90 2024-09-12T02:06:15.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.444+0000 7f8004ff9640 1 -- 172.21.15.16:0/629681311 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ff8003ad0 con 0x7f8008102d90 2024-09-12T02:06:15.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.444+0000 7f800ddd3640 1 -- 172.21.15.16:0/629681311 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f800819b160 con 0x7f8008102d90 2024-09-12T02:06:15.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.444+0000 7f8004ff9640 1 -- 172.21.15.16:0/629681311 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ff8035070 con 0x7f8008102d90 2024-09-12T02:06:15.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.444+0000 7f800ddd3640 1 -- 172.21.15.16:0/629681311 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80081a7340 con 0x7f8008102d90 2024-09-12T02:06:15.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.446+0000 7f8004ff9640 1 -- 172.21.15.16:0/629681311 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f7ff8042030 con 0x7f8008102d90 2024-09-12T02:06:15.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.446+0000 7f80101ac740 1 Processor -- start 2024-09-12T02:06:15.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.446+0000 7f80101ac740 1 -- start start 2024-09-12T02:06:15.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.447+0000 7f80101ac740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557a0dfa4060 0x557a0dfa4440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:15.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.447+0000 7f80101ac740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557a0dfa4980 0x557a0dfaed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:15.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.447+0000 7f80101ac740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557a0dfaf240 0x557a0dfb1630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:15.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.447+0000 7f80101ac740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x557a0df09620 con 0x557a0dfa4060 2024-09-12T02:06:15.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.447+0000 7f80101ac740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x557a0deb0e50 con 0x557a0dfa4980 2024-09-12T02:06:15.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.447+0000 7f80101ac740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x557a0df055e0 con 0x557a0dfaf240 2024-09-12T02:06:15.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.447+0000 7f8007fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557a0dfa4060 0x557a0dfa4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:15.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.447+0000 7f8007fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557a0dfa4060 0x557a0dfa4440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34652/0 (socket says 172.21.15.16:34652) 2024-09-12T02:06:15.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.447+0000 7f8007fff640 1 -- 172.21.15.16:0/2007520058 learned_addr learned my addr 172.21.15.16:0/2007520058 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:06:15.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.447+0000 7f80077fe640 1 --2- 172.21.15.16:0/2007520058 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557a0dfa4980 0x557a0dfaed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:15.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.447+0000 7f8006ffd640 1 --2- 172.21.15.16:0/2007520058 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557a0dfaf240 0x557a0dfb1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:15.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.447+0000 7f8004ff9640 1 --2- 172.21.15.16:0/629681311 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f7ff00a2b80 0x7f7ff00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:15.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.447+0000 7f8007fff640 1 -- 172.21.15.16:0/2007520058 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557a0dfaf240 msgr2=0x557a0dfb1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:06:15.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.447+0000 7f8007fff640 1 --2- 172.21.15.16:0/2007520058 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557a0dfaf240 0x557a0dfb1630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:15.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.447+0000 7f8007fff640 1 -- 172.21.15.16:0/2007520058 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557a0dfa4980 msgr2=0x557a0dfaed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:06:15.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.447+0000 7f8007fff640 1 --2- 172.21.15.16:0/2007520058 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557a0dfa4980 0x557a0dfaed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:15.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.447+0000 7f8007fff640 1 -- 172.21.15.16:0/2007520058 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a0dea1d40 con 0x557a0dfa4060 2024-09-12T02:06:15.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.447+0000 7f8004ff9640 1 -- 172.21.15.16:0/629681311 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f7ff80ef490 con 0x7f8008102d90 2024-09-12T02:06:15.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.447+0000 7f8007fff640 1 --2- 172.21.15.16:0/629681311 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f7ff00a2b80 0x7f7ff00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:15.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.448+0000 7f8007fff640 1 --2- 172.21.15.16:0/2007520058 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557a0dfa4060 0x557a0dfa4440 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f800000c4a0 tx=0x7f800000c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9613142824d6b10b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:06:15.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.448+0000 7f7fedffb640 1 -- 172.21.15.16:0/2007520058 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8000018640 con 0x557a0dfa4060 2024-09-12T02:06:15.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.448+0000 7f7fedffb640 1 -- 172.21.15.16:0/2007520058 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f800001e070 con 0x557a0dfa4060 2024-09-12T02:06:15.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.448+0000 7f7fedffb640 1 -- 172.21.15.16:0/2007520058 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f800000a970 con 0x557a0dfa4060 2024-09-12T02:06:15.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.448+0000 7f80101ac740 1 -- 172.21.15.16:0/2007520058 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557a0dfa4060 msgr2=0x557a0dfa4440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:15.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.448+0000 7f8007fff640 1 --2- 172.21.15.16:0/629681311 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f7ff00a2b80 0x7f7ff00a5040 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f800001d000 tx=0x7f8000017000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:06:15.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.448+0000 7f80101ac740 1 --2- 172.21.15.16:0/2007520058 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557a0dfa4060 0x557a0dfa4440 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f800000c4a0 tx=0x7f800000c960 comp rx=0 tx=0).stop 2024-09-12T02:06:15.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.449+0000 7f80101ac740 1 -- 172.21.15.16:0/2007520058 shutdown_connections 2024-09-12T02:06:15.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.449+0000 7f80101ac740 1 --2- 172.21.15.16:0/2007520058 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557a0dfaf240 0x557a0dfb1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:15.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.449+0000 7f80101ac740 1 --2- 172.21.15.16:0/2007520058 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557a0dfa4980 0x557a0dfaed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:15.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.449+0000 7f80101ac740 1 --2- 172.21.15.16:0/2007520058 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557a0dfa4060 0x557a0dfa4440 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:15.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.449+0000 7f80101ac740 1 -- 172.21.15.16:0/2007520058 >> 172.21.15.16:0/2007520058 conn(0x557a0dfa05f0 msgr2=0x557a0dfa09d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:06:15.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.449+0000 7f80101ac740 1 -- 172.21.15.16:0/2007520058 shutdown_connections 2024-09-12T02:06:15.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.449+0000 7f80101ac740 1 -- 172.21.15.16:0/2007520058 wait complete. 2024-09-12T02:06:15.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.449+0000 7f80101ac740 1 -- start start 2024-09-12T02:06:15.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.449+0000 7f80101ac740 1 Processor -- start 2024-09-12T02:06:15.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.449+0000 7f80101ac740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557a0dfa4060 0x557a0dfac110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:15.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.449+0000 7f80101ac740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557a0dfa4980 0x557a0dfac650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:15.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.449+0000 7f80101ac740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557a0dfaf240 0x557a0dfacb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:15.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.450+0000 7f80101ac740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x557a0dfb1e10 con 0x557a0dfa4980 2024-09-12T02:06:15.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.450+0000 7f80101ac740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x557a0de91f10 con 0x557a0dfaf240 2024-09-12T02:06:15.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.450+0000 7f80101ac740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x557a0dfb1f90 con 0x557a0dfa4060 2024-09-12T02:06:15.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.450+0000 7f80101ac740 10 client.0 _ll_register_callbacks cb 0x557a0deb5db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:06:15.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.450+0000 7f8006ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557a0dfa4980 0x557a0dfac650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:15.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.450+0000 7f8006ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557a0dfa4980 0x557a0dfac650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34660/0 (socket says 172.21.15.16:34660) 2024-09-12T02:06:15.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.450+0000 7f8006ffd640 1 -- 172.21.15.16:0/2085777487 learned_addr learned my addr 172.21.15.16:0/2085777487 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:06:15.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.450+0000 7f8007fff640 1 --2- 172.21.15.16:0/2085777487 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557a0dfaf240 0x557a0dfacb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:15.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.450+0000 7f80077fe640 1 --2- 172.21.15.16:0/2085777487 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557a0dfa4060 0x557a0dfac110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:15.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.450+0000 7f8006ffd640 1 -- 172.21.15.16:0/2085777487 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557a0dfa4060 msgr2=0x557a0dfac110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:06:15.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.450+0000 7f8006ffd640 1 --2- 172.21.15.16:0/2085777487 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557a0dfa4060 0x557a0dfac110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:15.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.450+0000 7f8006ffd640 1 -- 172.21.15.16:0/2085777487 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557a0dfaf240 msgr2=0x557a0dfacb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:06:15.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.450+0000 7f8006ffd640 1 --2- 172.21.15.16:0/2085777487 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557a0dfaf240 0x557a0dfacb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:15.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.450+0000 7f8006ffd640 1 -- 172.21.15.16:0/2085777487 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a0dc18580 con 0x557a0dfa4980 2024-09-12T02:06:15.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.450+0000 7f80077fe640 1 --2- 172.21.15.16:0/2085777487 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557a0dfa4060 0x557a0dfac110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:06:15.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.450+0000 7f8006ffd640 1 --2- 172.21.15.16:0/2085777487 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557a0dfa4980 0x557a0dfac650 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc00bb00 tx=0x7f7ffc004540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a257d953e1c3760 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:06:15.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.450+0000 7f7fe6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:06:15.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.450+0000 7f7fe6ffd640 1 -- 172.21.15.16:0/2085777487 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ffc003e80 con 0x557a0dfa4980 2024-09-12T02:06:15.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.450+0000 7f7fe6ffd640 1 -- 172.21.15.16:0/2085777487 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f7ffc011440 con 0x557a0dfa4980 2024-09-12T02:06:15.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.450+0000 7f80101ac740 1 -- 172.21.15.16:0/2085777487 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f800001e070 con 0x557a0dfa4980 2024-09-12T02:06:15.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.451+0000 7f7fe6ffd640 1 -- 172.21.15.16:0/2085777487 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7ffc011640 con 0x557a0dfa4980 2024-09-12T02:06:15.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.451+0000 7f80101ac740 10 client.16767 fetch_fsmap learned FSMap version 25 2024-09-12T02:06:15.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.451+0000 7f80101ac740 1 -- 172.21.15.16:0/2085777487 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x557a0dce2960 con 0x557a0dfa4980 2024-09-12T02:06:15.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.451+0000 7f7fe6ffd640 1 -- 172.21.15.16:0/2085777487 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f800001e070 con 0x557a0dfa4980 2024-09-12T02:06:15.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.451+0000 7f7fe6ffd640 1 -- 172.21.15.16:0/2085777487 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f7ffc01baa0 con 0x557a0dfa4980 2024-09-12T02:06:15.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.451+0000 7f80101ac740 10 client.16767 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:06:15.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.451+0000 7f80101ac740 10 client.16767 resolve_mds: validated gid 24482 aka daemon mds.b 2024-09-12T02:06:15.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.451+0000 7f80101ac740 20 client.16767 populate_metadata read hostname 'smithi016' 2024-09-12T02:06:15.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.451+0000 7f80101ac740 1 --2- 172.21.15.16:0/2085777487 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x557a0dfbedc0 0x557a0dfc1260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:15.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.451+0000 7f80101ac740 4 client.16767 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:06:15.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.451+0000 7f80101ac740 1 -- 172.21.15.16:0/2085777487 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557a0df16b90 con 0x557a0dfbedc0 2024-09-12T02:06:15.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.451+0000 7f80077fe640 1 --2- 172.21.15.16:0/2085777487 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x557a0dfbedc0 0x557a0dfc1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:15.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.452+0000 7f80077fe640 1 --2- 172.21.15.16:0/2085777487 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x557a0dfbedc0 0x557a0dfc1260 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f637c8adb7d729e server_cookie=68ae4636f0502485 in_seq=0 out_seq=0 2024-09-12T02:06:15.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.452+0000 7f7fe6ffd640 10 client.16767 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:06:15.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.452+0000 7f7fe6ffd640 1 -- 172.21.15.16:0/2085777487 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557a0df16b90 con 0x557a0dfbedc0 2024-09-12T02:06:15.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.452+0000 7f7fe6ffd640 10 client.16767 handle_command_reply: tid=0 2024-09-12T02:06:15.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.452+0000 7f80101ac740 1 client.16767 shutdown 2024-09-12T02:06:15.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.453+0000 7f80101ac740 1 -- 172.21.15.16:0/2085777487 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557a0dfa4980 msgr2=0x557a0dfac650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:15.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.453+0000 7f80101ac740 1 --2- 172.21.15.16:0/2085777487 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557a0dfa4980 0x557a0dfac650 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc00bb00 tx=0x7f7ffc004540 comp rx=0 tx=0).stop 2024-09-12T02:06:15.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.453+0000 7f80101ac740 1 -- 172.21.15.16:0/2085777487 shutdown_connections 2024-09-12T02:06:15.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.453+0000 7f80101ac740 1 --2- 172.21.15.16:0/2085777487 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x557a0dfbedc0 0x557a0dfc1260 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:15.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.453+0000 7f80101ac740 1 --2- 172.21.15.16:0/2085777487 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557a0dfaf240 0x557a0dfacb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:15.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.453+0000 7f80101ac740 1 --2- 172.21.15.16:0/2085777487 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557a0dfa4980 0x557a0dfac650 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:15.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.453+0000 7f7fe6ffd640 0 client.16767 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:06:15.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.453+0000 7f80101ac740 1 --2- 172.21.15.16:0/2085777487 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557a0dfa4060 0x557a0dfac110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:15.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.453+0000 7f80101ac740 1 -- 172.21.15.16:0/2085777487 >> 172.21.15.16:0/2085777487 conn(0x557a0dfa05f0 msgr2=0x557a0dfa6320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:06:15.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.453+0000 7f80101ac740 1 -- 172.21.15.16:0/2085777487 shutdown_connections 2024-09-12T02:06:15.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.453+0000 7f80101ac740 1 -- 172.21.15.16:0/2085777487 wait complete. 2024-09-12T02:06:15.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.453+0000 7f80101ac740 20 client.16767 trim_cache size 0 max 16384 2024-09-12T02:06:15.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.470+0000 7f80101ac740 1 Processor -- start 2024-09-12T02:06:15.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.470+0000 7f80101ac740 1 -- start start 2024-09-12T02:06:15.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.470+0000 7f80101ac740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557a0dfa4060 0x557a0dfa0c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:15.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.470+0000 7f80101ac740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557a0dfa4980 0x557a0df9f640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:15.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.470+0000 7f80101ac740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557a0dfaf240 0x557a0dfc6b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:15.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.470+0000 7f80101ac740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f80081b1f80 con 0x557a0dfa4060 2024-09-12T02:06:15.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.470+0000 7f80101ac740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f80081af850 con 0x557a0dfaf240 2024-09-12T02:06:15.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.470+0000 7f80101ac740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f80081ad6c0 con 0x557a0dfa4980 2024-09-12T02:06:15.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.470+0000 7f80077fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557a0dfa4060 0x557a0dfa0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:15.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.470+0000 7f80077fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557a0dfa4060 0x557a0dfa0c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34672/0 (socket says 172.21.15.16:34672) 2024-09-12T02:06:15.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.470+0000 7f80077fe640 1 -- 172.21.15.16:0/1675301787 learned_addr learned my addr 172.21.15.16:0/1675301787 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:06:15.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.470+0000 7f8007fff640 1 --2- 172.21.15.16:0/1675301787 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557a0dfaf240 0x557a0dfc6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:15.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.470+0000 7f8006ffd640 1 --2- 172.21.15.16:0/1675301787 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557a0dfa4980 0x557a0df9f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:15.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.470+0000 7f80077fe640 1 -- 172.21.15.16:0/1675301787 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557a0dfa4980 msgr2=0x557a0df9f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:06:15.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.470+0000 7f80077fe640 1 --2- 172.21.15.16:0/1675301787 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557a0dfa4980 0x557a0df9f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:15.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.471+0000 7f80077fe640 1 -- 172.21.15.16:0/1675301787 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557a0dfaf240 msgr2=0x557a0dfc6b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:06:15.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.471+0000 7f80077fe640 1 --2- 172.21.15.16:0/1675301787 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557a0dfaf240 0x557a0dfc6b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:15.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.471+0000 7f80077fe640 1 -- 172.21.15.16:0/1675301787 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a0dfae050 con 0x557a0dfa4060 2024-09-12T02:06:15.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.471+0000 7f8007fff640 1 --2- 172.21.15.16:0/1675301787 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557a0dfaf240 0x557a0dfc6b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:06:15.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.471+0000 7f8006ffd640 1 --2- 172.21.15.16:0/1675301787 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557a0dfa4980 0x557a0df9f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:06:15.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.471+0000 7f80077fe640 1 --2- 172.21.15.16:0/1675301787 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557a0dfa4060 0x557a0dfa0c00 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f7ff80ed030 tx=0x7f7ff80045f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae6bae2c8543de77 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:06:15.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.471+0000 7f7fed7fa640 1 -- 172.21.15.16:0/1675301787 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ff8049040 con 0x557a0dfa4060 2024-09-12T02:06:15.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.471+0000 7f7fed7fa640 1 -- 172.21.15.16:0/1675301787 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ff80bbd90 con 0x557a0dfa4060 2024-09-12T02:06:15.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.471+0000 7f7fed7fa640 1 -- 172.21.15.16:0/1675301787 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ff803b460 con 0x557a0dfa4060 2024-09-12T02:06:15.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.471+0000 7f80101ac740 1 -- 172.21.15.16:0/1675301787 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557a0dfa4060 msgr2=0x557a0dfa0c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:15.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.472+0000 7f80101ac740 1 --2- 172.21.15.16:0/1675301787 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557a0dfa4060 0x557a0dfa0c00 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f7ff80ed030 tx=0x7f7ff80045f0 comp rx=0 tx=0).stop 2024-09-12T02:06:15.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.472+0000 7f80101ac740 1 -- 172.21.15.16:0/1675301787 shutdown_connections 2024-09-12T02:06:15.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.472+0000 7f80101ac740 1 --2- 172.21.15.16:0/1675301787 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557a0dfaf240 0x557a0dfc6b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:15.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.472+0000 7f80101ac740 1 --2- 172.21.15.16:0/1675301787 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557a0dfa4980 0x557a0df9f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:15.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.472+0000 7f80101ac740 1 --2- 172.21.15.16:0/1675301787 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557a0dfa4060 0x557a0dfa0c00 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:15.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.472+0000 7f80101ac740 1 -- 172.21.15.16:0/1675301787 >> 172.21.15.16:0/1675301787 conn(0x557a0dfa05f0 msgr2=0x557a0df9ef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:06:15.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.472+0000 7f80101ac740 1 -- 172.21.15.16:0/1675301787 shutdown_connections 2024-09-12T02:06:15.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.472+0000 7f80101ac740 1 -- 172.21.15.16:0/1675301787 wait complete. 2024-09-12T02:06:15.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.472+0000 7f80101ac740 1 -- start start 2024-09-12T02:06:15.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.472+0000 7f80101ac740 1 Processor -- start 2024-09-12T02:06:15.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.472+0000 7f80101ac740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557a0dfa4060 0x557a0dfb77c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:15.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.472+0000 7f80101ac740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557a0dfa4980 0x557a0dfb3b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:15.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.472+0000 7f80101ac740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557a0dfaf240 0x557a0dfb4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:15.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.472+0000 7f80101ac740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f80081add10 con 0x557a0dfaf240 2024-09-12T02:06:15.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.472+0000 7f80101ac740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f80081af180 con 0x557a0dfa4060 2024-09-12T02:06:15.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.473+0000 7f80101ac740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f80081ae3e0 con 0x557a0dfa4980 2024-09-12T02:06:15.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.473+0000 7f80101ac740 10 client.0 _ll_register_callbacks cb 0x7f7ffc004400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:06:15.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.473+0000 7f80077fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557a0dfa4060 0x557a0dfb77c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:15.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.473+0000 7f8007fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557a0dfaf240 0x557a0dfb4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:15.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.473+0000 7f80077fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557a0dfa4060 0x557a0dfb77c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.16:43794/0 (socket says 172.21.15.16:43794) 2024-09-12T02:06:15.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.473+0000 7f80077fe640 1 -- 172.21.15.16:0/2161266192 learned_addr learned my addr 172.21.15.16:0/2161266192 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:06:15.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.473+0000 7f8006ffd640 1 --2- 172.21.15.16:0/2161266192 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557a0dfa4980 0x557a0dfb3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:15.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.473+0000 7f8007fff640 1 -- 172.21.15.16:0/2161266192 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557a0dfa4980 msgr2=0x557a0dfb3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:06:15.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.473+0000 7f8007fff640 1 --2- 172.21.15.16:0/2161266192 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557a0dfa4980 0x557a0dfb3b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:15.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.473+0000 7f8007fff640 1 -- 172.21.15.16:0/2161266192 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557a0dfa4060 msgr2=0x557a0dfb77c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:06:15.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.473+0000 7f8007fff640 1 --2- 172.21.15.16:0/2161266192 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557a0dfa4060 0x557a0dfb77c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:15.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.473+0000 7f8007fff640 1 -- 172.21.15.16:0/2161266192 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a0dfb46b0 con 0x557a0dfaf240 2024-09-12T02:06:15.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.473+0000 7f8007fff640 1 --2- 172.21.15.16:0/2161266192 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557a0dfaf240 0x557a0dfb4040 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f800000ce60 tx=0x7f8000011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afeeb50d01a8be69 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:06:15.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.473+0000 7f7fe6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:06:15.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.473+0000 7f7fe6ffd640 1 -- 172.21.15.16:0/2161266192 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8000028040 con 0x557a0dfaf240 2024-09-12T02:06:15.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.473+0000 7f80101ac740 1 -- 172.21.15.16:0/2161266192 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ff80bbd90 con 0x557a0dfaf240 2024-09-12T02:06:15.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.473+0000 7f7fe6ffd640 1 -- 172.21.15.16:0/2161266192 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f8000018640 con 0x557a0dfaf240 2024-09-12T02:06:15.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.474+0000 7f7fe6ffd640 1 -- 172.21.15.16:0/2161266192 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f8000009c20 con 0x557a0dfaf240 2024-09-12T02:06:15.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.474+0000 7f80101ac740 10 client.16773 fetch_fsmap learned FSMap version 25 2024-09-12T02:06:15.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.474+0000 7f80101ac740 1 -- 172.21.15.16:0/2161266192 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x557a0e02a970 con 0x557a0dfaf240 2024-09-12T02:06:15.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.474+0000 7f7fe6ffd640 1 -- 172.21.15.16:0/2161266192 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ff80bbd90 con 0x557a0dfaf240 2024-09-12T02:06:15.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.474+0000 7f7fe6ffd640 1 -- 172.21.15.16:0/2161266192 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f800002b070 con 0x557a0dfaf240 2024-09-12T02:06:15.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.474+0000 7f80101ac740 10 client.16773 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:06:15.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.474+0000 7f80101ac740 10 client.16773 resolve_mds: validated gid 24482 aka daemon mds.b 2024-09-12T02:06:15.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.474+0000 7f80101ac740 20 client.16773 populate_metadata read hostname 'smithi016' 2024-09-12T02:06:15.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.474+0000 7f80101ac740 1 --2- 172.21.15.16:0/2161266192 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x557a0e02af50 0x557a0e02d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:15.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.474+0000 7f80101ac740 4 client.16773 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "ee2e48a6", "release": true, "await": true}] 2024-09-12T02:06:15.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.474+0000 7f80101ac740 1 -- 172.21.15.16:0/2161266192 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "quiesce db", "set_id": "ee2e48a6", "release": true, "await": true}) -- 0x557a0df110a0 con 0x557a0e02af50 2024-09-12T02:06:15.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.474+0000 7f80077fe640 1 --2- 172.21.15.16:0/2161266192 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x557a0e02af50 0x557a0e02d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:15.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.475+0000 7f80077fe640 1 --2- 172.21.15.16:0/2161266192 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x557a0e02af50 0x557a0e02d340 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bfc94825701cdd41 server_cookie=a51b93e856ab2f9c in_seq=0 out_seq=0 2024-09-12T02:06:15.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.475+0000 7f7fe6ffd640 10 client.16773 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:06:15.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.477+0000 7f7fe6ffd640 1 -- 172.21.15.16:0/2161266192 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x557a0df110a0 con 0x557a0e02af50 2024-09-12T02:06:15.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.477+0000 7f7fe6ffd640 10 client.16773 handle_command_reply: tid=0 2024-09-12T02:06:15.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.477+0000 7f80101ac740 1 client.16773 shutdown 2024-09-12T02:06:15.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.477+0000 7f80101ac740 1 -- 172.21.15.16:0/2161266192 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557a0dfaf240 msgr2=0x557a0dfb4040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:15.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.477+0000 7f80101ac740 1 --2- 172.21.15.16:0/2161266192 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557a0dfaf240 0x557a0dfb4040 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f800000ce60 tx=0x7f8000011000 comp rx=0 tx=0).stop 2024-09-12T02:06:15.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.477+0000 7f80101ac740 1 -- 172.21.15.16:0/2161266192 shutdown_connections 2024-09-12T02:06:15.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.477+0000 7f80101ac740 1 --2- 172.21.15.16:0/2161266192 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x557a0e02af50 0x557a0e02d340 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:15.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.477+0000 7f80101ac740 1 --2- 172.21.15.16:0/2161266192 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557a0dfaf240 0x557a0dfb4040 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:15.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.477+0000 7f80101ac740 1 --2- 172.21.15.16:0/2161266192 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557a0dfa4980 0x557a0dfb3b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:15.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.477+0000 7f7fe6ffd640 0 client.16773 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:06:15.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.477+0000 7f80101ac740 1 --2- 172.21.15.16:0/2161266192 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557a0dfa4060 0x557a0dfb77c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:15.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.477+0000 7f80101ac740 1 -- 172.21.15.16:0/2161266192 >> 172.21.15.16:0/2161266192 conn(0x557a0dfa05f0 msgr2=0x557a0df9f750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:06:15.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.477+0000 7f80101ac740 1 -- 172.21.15.16:0/2161266192 shutdown_connections 2024-09-12T02:06:15.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.478+0000 7f80101ac740 1 -- 172.21.15.16:0/2161266192 wait complete. 2024-09-12T02:06:15.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.478+0000 7f80101ac740 20 client.16773 trim_cache size 0 max 16384 2024-09-12T02:06:15.481 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T02:06:15.482 INFO:teuthology.orchestra.run.smithi016.stdout: "epoch": 25, 2024-09-12T02:06:15.482 INFO:teuthology.orchestra.run.smithi016.stdout: "leader": 24482, 2024-09-12T02:06:15.482 INFO:teuthology.orchestra.run.smithi016.stdout: "set_version": 20, 2024-09-12T02:06:15.482 INFO:teuthology.orchestra.run.smithi016.stdout: "sets": { 2024-09-12T02:06:15.482 INFO:teuthology.orchestra.run.smithi016.stdout: "ee2e48a6": { 2024-09-12T02:06:15.482 INFO:teuthology.orchestra.run.smithi016.stdout: "version": 20, 2024-09-12T02:06:15.482 INFO:teuthology.orchestra.run.smithi016.stdout: "age_ref": 0.0, 2024-09-12T02:06:15.482 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-12T02:06:15.482 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RELEASED", 2024-09-12T02:06:15.482 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-12T02:06:15.482 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:06:15.482 INFO:teuthology.orchestra.run.smithi016.stdout: "timeout": 90.0, 2024-09-12T02:06:15.482 INFO:teuthology.orchestra.run.smithi016.stdout: "expiration": 140.2, 2024-09-12T02:06:15.482 INFO:teuthology.orchestra.run.smithi016.stdout: "members": { 2024-09-12T02:06:15.483 INFO:teuthology.orchestra.run.smithi016.stdout: "file:/": { 2024-09-12T02:06:15.483 INFO:teuthology.orchestra.run.smithi016.stdout: "excluded": false, 2024-09-12T02:06:15.483 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-12T02:06:15.483 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RELEASED", 2024-09-12T02:06:15.483 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-12T02:06:15.483 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:06:15.483 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:06:15.483 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:06:15.483 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:06:15.483 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:06:15.483 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T02:06:15.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.478+0000 7f800ddd3640 1 -- 172.21.15.16:0/629681311 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f7ff00a2b80 msgr2=0x7f7ff00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:15.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.478+0000 7f800ddd3640 1 --2- 172.21.15.16:0/629681311 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f7ff00a2b80 0x7f7ff00a5040 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f800001d000 tx=0x7f8000017000 comp rx=0 tx=0).stop 2024-09-12T02:06:15.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.478+0000 7f800ddd3640 1 -- 172.21.15.16:0/629681311 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8008102d90 msgr2=0x7f80081a0590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:15.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.478+0000 7f800ddd3640 1 --2- 172.21.15.16:0/629681311 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8008102d90 0x7f80081a0590 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f7ff8009230 tx=0x7f7ff8004270 comp rx=0 tx=0).stop 2024-09-12T02:06:15.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.478+0000 7f800ddd3640 1 -- 172.21.15.16:0/629681311 shutdown_connections 2024-09-12T02:06:15.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.478+0000 7f800ddd3640 1 --2- 172.21.15.16:0/629681311 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f7ff00a2b80 0x7f7ff00a5040 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:15.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.478+0000 7f800ddd3640 1 --2- 172.21.15.16:0/629681311 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8008108290 0x7f800819a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:15.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.478+0000 7f800ddd3640 1 --2- 172.21.15.16:0/629681311 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8008103740 0x7f80081a0ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:15.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.478+0000 7f800ddd3640 1 --2- 172.21.15.16:0/629681311 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8008102d90 0x7f80081a0590 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:15.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.478+0000 7f800ddd3640 1 -- 172.21.15.16:0/629681311 >> 172.21.15.16:0/629681311 conn(0x7f80080fc7d0 msgr2=0x7f80081013f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:06:15.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.478+0000 7f800ddd3640 1 -- 172.21.15.16:0/629681311 shutdown_connections 2024-09-12T02:06:15.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:15.478+0000 7f800ddd3640 1 -- 172.21.15.16:0/629681311 wait complete. 2024-09-12T02:06:16.236 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'ee2e48a6', total seconds elapsed: 23.4 2024-09-12T02:06:16.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:06:16 smithi184 ceph-mon[35593]: pgmap v465: 97 pgs: 97 active+clean; 2.2 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 2.4 KiB/s wr, 9 op/s 2024-09-12T02:06:16.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:06:16 smithi033 ceph-mon[35332]: pgmap v465: 97 pgs: 97 active+clean; 2.2 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 2.4 KiB/s wr, 9 op/s 2024-09-12T02:06:16.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:06:16 smithi016 ceph-mon[30947]: pgmap v465: 97 pgs: 97 active+clean; 2.2 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 2.4 KiB/s wr, 9 op/s 2024-09-12T02:06:18.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:06:18 smithi184 ceph-mon[35593]: pgmap v466: 97 pgs: 97 active+clean; 2.2 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-09-12T02:06:18.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:06:18 smithi033 ceph-mon[35332]: pgmap v466: 97 pgs: 97 active+clean; 2.2 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-09-12T02:06:18.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:06:18 smithi016 ceph-mon[30947]: pgmap v466: 97 pgs: 97 active+clean; 2.2 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-09-12T02:06:20.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:06:20 smithi184 ceph-mon[35593]: pgmap v467: 97 pgs: 97 active+clean; 2.2 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-09-12T02:06:20.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:06:20 smithi033 ceph-mon[35332]: pgmap v467: 97 pgs: 97 active+clean; 2.2 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-09-12T02:06:20.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:06:20 smithi016 ceph-mon[30947]: pgmap v467: 97 pgs: 97 active+clean; 2.2 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-09-12T02:06:20.527 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.525+0000 7f2ebeae4640 1 -- 172.21.15.16:0/1014176430 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2eb81025f0 msgr2=0x7f2eb8108e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:20.527 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.525+0000 7f2ebeae4640 1 --2- 172.21.15.16:0/1014176430 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2eb81025f0 0x7f2eb8108e80 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f2eb0009ca0 tx=0x7f2eb002a8c0 comp rx=0 tx=0).stop 2024-09-12T02:06:20.527 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.525+0000 7f2ebeae4640 1 -- 172.21.15.16:0/1014176430 shutdown_connections 2024-09-12T02:06:20.527 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.525+0000 7f2ebeae4640 1 --2- 172.21.15.16:0/1014176430 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2eb81025f0 0x7f2eb8108e80 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:20.528 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.525+0000 7f2ebeae4640 1 --2- 172.21.15.16:0/1014176430 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2eb8101c30 0x7f2eb81020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:20.528 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.525+0000 7f2ebeae4640 1 --2- 172.21.15.16:0/1014176430 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2eb8100a40 0x7f2eb8100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:20.528 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.525+0000 7f2ebeae4640 1 -- 172.21.15.16:0/1014176430 >> 172.21.15.16:0/1014176430 conn(0x7f2eb80fc1e0 msgr2=0x7f2eb80fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:06:20.528 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.525+0000 7f2ebeae4640 1 -- 172.21.15.16:0/1014176430 shutdown_connections 2024-09-12T02:06:20.528 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.525+0000 7f2ebeae4640 1 -- 172.21.15.16:0/1014176430 wait complete. 2024-09-12T02:06:20.528 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.526+0000 7f2ebeae4640 1 Processor -- start 2024-09-12T02:06:20.528 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.526+0000 7f2ebeae4640 1 -- start start 2024-09-12T02:06:20.528 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.526+0000 7f2ebeae4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2eb8100a40 0x7f2eb81049d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:20.528 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.526+0000 7f2ebeae4640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2eb8101c30 0x7f2eb8103010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:20.528 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.526+0000 7f2ebeae4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2eb81025f0 0x7f2eb8103550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:20.528 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.526+0000 7f2ebeae4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2eb810c6c0 con 0x7f2eb81025f0 2024-09-12T02:06:20.529 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.526+0000 7f2ebeae4640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2eb810c540 con 0x7f2eb8100a40 2024-09-12T02:06:20.529 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.526+0000 7f2ebeae4640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f2eb810c840 con 0x7f2eb8101c30 2024-09-12T02:06:20.529 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.526+0000 7f2ebd081640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2eb81025f0 0x7f2eb8103550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:20.529 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.526+0000 7f2ebd081640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2eb81025f0 0x7f2eb8103550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34690/0 (socket says 172.21.15.16:34690) 2024-09-12T02:06:20.529 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.526+0000 7f2ebd081640 1 -- 172.21.15.16:0/1329564321 learned_addr learned my addr 172.21.15.16:0/1329564321 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:06:20.529 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.526+0000 7f2ebc880640 1 --2- 172.21.15.16:0/1329564321 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2eb8100a40 0x7f2eb81049d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:20.529 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.526+0000 7f2eaffff640 1 --2- 172.21.15.16:0/1329564321 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2eb8101c30 0x7f2eb8103010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:20.530 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.527+0000 7f2ebd081640 1 -- 172.21.15.16:0/1329564321 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2eb8101c30 msgr2=0x7f2eb8103010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:20.530 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.527+0000 7f2ebd081640 1 --2- 172.21.15.16:0/1329564321 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2eb8101c30 0x7f2eb8103010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:20.530 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.527+0000 7f2ebd081640 1 -- 172.21.15.16:0/1329564321 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2eb8100a40 msgr2=0x7f2eb81049d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:20.530 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.527+0000 7f2ebd081640 1 --2- 172.21.15.16:0/1329564321 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2eb8100a40 0x7f2eb81049d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:20.530 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.527+0000 7f2ebd081640 1 -- 172.21.15.16:0/1329564321 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2eb8103de0 con 0x7f2eb81025f0 2024-09-12T02:06:20.530 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.527+0000 7f2ebd081640 1 --2- 172.21.15.16:0/1329564321 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2eb81025f0 0x7f2eb8103550 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f2eb0031ae0 tx=0x7f2eb0031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:06:20.530 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.527+0000 7f2eadffb640 1 -- 172.21.15.16:0/1329564321 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eb0003b70 con 0x7f2eb81025f0 2024-09-12T02:06:20.530 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.527+0000 7f2eadffb640 1 -- 172.21.15.16:0/1329564321 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2eb0003d10 con 0x7f2eb81025f0 2024-09-12T02:06:20.530 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.527+0000 7f2ebeae4640 1 -- 172.21.15.16:0/1329564321 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2eb81a1ec0 con 0x7f2eb81025f0 2024-09-12T02:06:20.530 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.527+0000 7f2eadffb640 1 -- 172.21.15.16:0/1329564321 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eb002fc40 con 0x7f2eb81025f0 2024-09-12T02:06:20.530 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.527+0000 7f2ebeae4640 1 -- 172.21.15.16:0/1329564321 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2eb81a2380 con 0x7f2eb81025f0 2024-09-12T02:06:20.536 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.529+0000 7f2eadffb640 1 -- 172.21.15.16:0/1329564321 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f2eb0004050 con 0x7f2eb81025f0 2024-09-12T02:06:20.537 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.530+0000 7f2ec1884740 1 Processor -- start 2024-09-12T02:06:20.537 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.530+0000 7f2ec1884740 1 -- start start 2024-09-12T02:06:20.537 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.530+0000 7f2ec1884740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a076c6f9a0 0x55a076c6fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:20.537 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.530+0000 7f2ec1884740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a076c702e0 0x55a076c74da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:20.537 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.530+0000 7f2ec1884740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a076c75370 0x55a076c77830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:20.537 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.530+0000 7f2ec1884740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55a076bd32f0 con 0x55a076c702e0 2024-09-12T02:06:20.537 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.530+0000 7f2ec1884740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55a076b7b270 con 0x55a076c75370 2024-09-12T02:06:20.537 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.530+0000 7f2ec1884740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55a076bcf2b0 con 0x55a076c6f9a0 2024-09-12T02:06:20.537 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.530+0000 7f2eadffb640 1 --2- 172.21.15.16:0/1329564321 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f2ea00a2cb0 0x7f2ea00a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:20.537 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.530+0000 7f2ebc880640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a076c702e0 0x55a076c74da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:20.537 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.530+0000 7f2ebc880640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a076c702e0 0x55a076c74da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34700/0 (socket says 172.21.15.16:34700) 2024-09-12T02:06:20.537 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.530+0000 7f2ebc880640 1 -- 172.21.15.16:0/2753803341 learned_addr learned my addr 172.21.15.16:0/2753803341 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:06:20.538 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.530+0000 7f2eaffff640 1 --2- 172.21.15.16:0/2753803341 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a076c75370 0x55a076c77830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:20.538 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.530+0000 7f2ebd081640 1 --2- 172.21.15.16:0/1329564321 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f2ea00a2cb0 0x7f2ea00a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:20.538 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.530+0000 7f2eaffff640 1 --2- 172.21.15.16:0/2753803341 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a076c6f9a0 0x55a076c6fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:20.538 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.531+0000 7f2eadffb640 1 -- 172.21.15.16:0/1329564321 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f2eb00eff00 con 0x7f2eb81025f0 2024-09-12T02:06:20.538 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.531+0000 7f2ebc880640 1 -- 172.21.15.16:0/2753803341 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a076c6f9a0 msgr2=0x55a076c6fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:06:20.538 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.531+0000 7f2ebc880640 1 --2- 172.21.15.16:0/2753803341 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a076c6f9a0 0x55a076c6fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:20.538 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.531+0000 7f2ebc880640 1 -- 172.21.15.16:0/2753803341 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a076c75370 msgr2=0x55a076c77830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:06:20.538 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.531+0000 7f2ebc880640 1 --2- 172.21.15.16:0/2753803341 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a076c75370 0x55a076c77830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:20.538 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.531+0000 7f2ebc880640 1 -- 172.21.15.16:0/2753803341 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a076b7a340 con 0x55a076c702e0 2024-09-12T02:06:20.538 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.531+0000 7f2ebc880640 1 --2- 172.21.15.16:0/2753803341 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a076c702e0 0x55a076c74da0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f2eb4009ca0 tx=0x7f2eb4011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca4eebfe5d96d313 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:06:20.538 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.531+0000 7f2ebd081640 1 --2- 172.21.15.16:0/1329564321 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f2ea00a2cb0 0x7f2ea00a5170 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f2eb004c000 tx=0x7f2eb00b4040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:06:20.538 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.531+0000 7f2e9effd640 1 -- 172.21.15.16:0/2753803341 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eb4009e50 con 0x55a076c702e0 2024-09-12T02:06:20.539 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.531+0000 7f2e9effd640 1 -- 172.21.15.16:0/2753803341 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2eb40035f0 con 0x55a076c702e0 2024-09-12T02:06:20.539 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.531+0000 7f2e9effd640 1 -- 172.21.15.16:0/2753803341 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eb4019650 con 0x55a076c702e0 2024-09-12T02:06:20.539 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.532+0000 7f2ec1884740 1 -- 172.21.15.16:0/2753803341 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a076c702e0 msgr2=0x55a076c74da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:20.539 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.532+0000 7f2ec1884740 1 --2- 172.21.15.16:0/2753803341 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a076c702e0 0x55a076c74da0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f2eb4009ca0 tx=0x7f2eb4011810 comp rx=0 tx=0).stop 2024-09-12T02:06:20.539 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.532+0000 7f2ec1884740 1 -- 172.21.15.16:0/2753803341 shutdown_connections 2024-09-12T02:06:20.539 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.532+0000 7f2ec1884740 1 --2- 172.21.15.16:0/2753803341 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a076c75370 0x55a076c77830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:20.539 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.532+0000 7f2ec1884740 1 --2- 172.21.15.16:0/2753803341 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a076c702e0 0x55a076c74da0 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:20.539 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.532+0000 7f2ec1884740 1 --2- 172.21.15.16:0/2753803341 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a076c6f9a0 0x55a076c6fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:20.539 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.532+0000 7f2ec1884740 1 -- 172.21.15.16:0/2753803341 >> 172.21.15.16:0/2753803341 conn(0x55a076c6b010 msgr2=0x55a076c6b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:06:20.539 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.532+0000 7f2ec1884740 1 -- 172.21.15.16:0/2753803341 shutdown_connections 2024-09-12T02:06:20.539 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.532+0000 7f2ec1884740 1 -- 172.21.15.16:0/2753803341 wait complete. 2024-09-12T02:06:20.539 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.532+0000 7f2ec1884740 1 -- start start 2024-09-12T02:06:20.539 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.532+0000 7f2ec1884740 1 Processor -- start 2024-09-12T02:06:20.540 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.533+0000 7f2ec1884740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a076c6f9a0 0x55a076c79b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:20.540 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.533+0000 7f2ec1884740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a076c702e0 0x55a076c7a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:20.540 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.533+0000 7f2ec1884740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a076c75370 0x55a076c810e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:20.540 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.533+0000 7f2ec1884740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55a076c78340 con 0x55a076c6f9a0 2024-09-12T02:06:20.540 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.533+0000 7f2ec1884740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55a076b5c290 con 0x55a076c702e0 2024-09-12T02:06:20.540 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.533+0000 7f2ec1884740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55a076c784c0 con 0x55a076c75370 2024-09-12T02:06:20.540 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.533+0000 7f2ec1884740 10 client.0 _ll_register_callbacks cb 0x55a076b8d140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:06:20.540 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.533+0000 7f2eaffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a076c6f9a0 0x55a076c79b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:20.540 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.533+0000 7f2eaffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a076c6f9a0 0x55a076c79b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34712/0 (socket says 172.21.15.16:34712) 2024-09-12T02:06:20.540 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.533+0000 7f2eaffff640 1 -- 172.21.15.16:0/3015032957 learned_addr learned my addr 172.21.15.16:0/3015032957 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:06:20.540 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.533+0000 7f2eaffff640 1 --2- 172.21.15.16:0/3015032957 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a076c75370 0x55a076c810e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:20.540 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.533+0000 7f2ebc880640 1 --2- 172.21.15.16:0/3015032957 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a076c702e0 0x55a076c7a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:20.540 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.533+0000 7f2eaffff640 1 -- 172.21.15.16:0/3015032957 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a076c75370 msgr2=0x55a076c810e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:06:20.541 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.533+0000 7f2eaffff640 1 --2- 172.21.15.16:0/3015032957 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a076c75370 0x55a076c810e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:20.541 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.533+0000 7f2eaffff640 1 -- 172.21.15.16:0/3015032957 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a076c702e0 msgr2=0x55a076c7a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:06:20.541 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.533+0000 7f2eaffff640 1 --2- 172.21.15.16:0/3015032957 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a076c702e0 0x55a076c7a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:20.541 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.533+0000 7f2eaffff640 1 -- 172.21.15.16:0/3015032957 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0768e6210 con 0x55a076c6f9a0 2024-09-12T02:06:20.541 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.533+0000 7f2eaffff640 1 --2- 172.21.15.16:0/3015032957 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a076c6f9a0 0x55a076c79b60 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f2ea80038e0 tx=0x7f2ea80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec0a30f9720598e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:06:20.541 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.533+0000 7f2e97fff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:06:20.541 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.534+0000 7f2e97fff640 1 -- 172.21.15.16:0/3015032957 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ea8012c70 con 0x55a076c6f9a0 2024-09-12T02:06:20.541 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.534+0000 7f2e97fff640 1 -- 172.21.15.16:0/3015032957 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f2ea80047a0 con 0x55a076c6f9a0 2024-09-12T02:06:20.541 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.534+0000 7f2ec1884740 1 -- 172.21.15.16:0/3015032957 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2eb40035f0 con 0x55a076c6f9a0 2024-09-12T02:06:20.541 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.534+0000 7f2e97fff640 1 -- 172.21.15.16:0/3015032957 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2ea80049a0 con 0x55a076c6f9a0 2024-09-12T02:06:20.541 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.534+0000 7f2ec1884740 10 client.16785 fetch_fsmap learned FSMap version 25 2024-09-12T02:06:20.541 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.534+0000 7f2ec1884740 1 -- 172.21.15.16:0/3015032957 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a0769b2620 con 0x55a076c6f9a0 2024-09-12T02:06:20.541 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.534+0000 7f2e97fff640 1 -- 172.21.15.16:0/3015032957 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2eb40035f0 con 0x55a076c6f9a0 2024-09-12T02:06:20.542 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.534+0000 7f2e97fff640 1 -- 172.21.15.16:0/3015032957 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f2ea801d070 con 0x55a076c6f9a0 2024-09-12T02:06:20.542 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.534+0000 7f2ec1884740 10 client.16785 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:06:20.542 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.534+0000 7f2ec1884740 10 client.16785 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:06:20.542 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.534+0000 7f2ec1884740 20 client.16785 populate_metadata read hostname 'smithi016' 2024-09-12T02:06:20.542 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.534+0000 7f2ec1884740 1 --2- 172.21.15.16:0/3015032957 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55a076c85810 0x55a076c87ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:20.542 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.534+0000 7f2ec1884740 4 client.16785 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:06:20.542 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.534+0000 7f2ec1884740 1 -- 172.21.15.16:0/3015032957 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a076be0a20 con 0x55a076c85810 2024-09-12T02:06:20.542 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.535+0000 7f2ebc880640 1 --2- 172.21.15.16:0/3015032957 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55a076c85810 0x55a076c87ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:20.542 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.535+0000 7f2ebc880640 1 --2- 172.21.15.16:0/3015032957 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55a076c85810 0x55a076c87ca0 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed3e5be2fd0f7e1b server_cookie=f6388e591d347f6c in_seq=0 out_seq=0 2024-09-12T02:06:20.542 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.535+0000 7f2e97fff640 10 client.16785 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:06:20.542 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.536+0000 7f2e97fff640 1 -- 172.21.15.16:0/3015032957 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a076be0a20 con 0x55a076c85810 2024-09-12T02:06:20.542 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.536+0000 7f2e97fff640 10 client.16785 handle_command_reply: tid=0 2024-09-12T02:06:20.542 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.536+0000 7f2ec1884740 1 client.16785 shutdown 2024-09-12T02:06:20.543 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.536+0000 7f2ec1884740 1 -- 172.21.15.16:0/3015032957 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a076c6f9a0 msgr2=0x55a076c79b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:20.543 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.536+0000 7f2ec1884740 1 --2- 172.21.15.16:0/3015032957 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a076c6f9a0 0x55a076c79b60 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f2ea80038e0 tx=0x7f2ea80046c0 comp rx=0 tx=0).stop 2024-09-12T02:06:20.543 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.536+0000 7f2ec1884740 1 -- 172.21.15.16:0/3015032957 shutdown_connections 2024-09-12T02:06:20.543 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.536+0000 7f2ec1884740 1 --2- 172.21.15.16:0/3015032957 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55a076c85810 0x55a076c87ca0 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:20.543 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.536+0000 7f2ec1884740 1 --2- 172.21.15.16:0/3015032957 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a076c75370 0x55a076c810e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:20.543 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.536+0000 7f2ec1884740 1 --2- 172.21.15.16:0/3015032957 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a076c702e0 0x55a076c7a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:20.543 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.536+0000 7f2e97fff640 0 client.16785 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:06:20.543 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.536+0000 7f2ec1884740 1 --2- 172.21.15.16:0/3015032957 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a076c6f9a0 0x55a076c79b60 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:20.543 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.536+0000 7f2ec1884740 1 -- 172.21.15.16:0/3015032957 >> 172.21.15.16:0/3015032957 conn(0x55a076c6b010 msgr2=0x55a076c6c820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:06:20.543 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.536+0000 7f2ec1884740 1 -- 172.21.15.16:0/3015032957 shutdown_connections 2024-09-12T02:06:20.543 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.536+0000 7f2ec1884740 1 -- 172.21.15.16:0/3015032957 wait complete. 2024-09-12T02:06:20.543 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.536+0000 7f2ec1884740 20 client.16785 trim_cache size 0 max 16384 2024-09-12T02:06:20.556 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.551+0000 7f2ec1884740 1 Processor -- start 2024-09-12T02:06:20.556 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.551+0000 7f2ec1884740 1 -- start start 2024-09-12T02:06:20.556 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.551+0000 7f2ec1884740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a076c6f9a0 0x55a076c6bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:20.556 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.551+0000 7f2ec1884740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a076c702e0 0x55a076c810a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:20.556 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.551+0000 7f2ec1884740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a076c75370 0x55a076c6c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:20.556 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.551+0000 7f2ec1884740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2eb81ad8e0 con 0x55a076c702e0 2024-09-12T02:06:20.557 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.551+0000 7f2ec1884740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2eb81ab1b0 con 0x55a076c75370 2024-09-12T02:06:20.557 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.551+0000 7f2ec1884740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f2eb81a9000 con 0x55a076c6f9a0 2024-09-12T02:06:20.557 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.552+0000 7f2ebc880640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a076c702e0 0x55a076c810a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:20.557 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.552+0000 7f2ebc880640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a076c702e0 0x55a076c810a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34728/0 (socket says 172.21.15.16:34728) 2024-09-12T02:06:20.557 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.552+0000 7f2ebc880640 1 -- 172.21.15.16:0/1220949532 learned_addr learned my addr 172.21.15.16:0/1220949532 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:06:20.557 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.552+0000 7f2eaffff640 1 --2- 172.21.15.16:0/1220949532 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a076c6f9a0 0x55a076c6bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:20.557 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.552+0000 7f2eaffff640 1 --2- 172.21.15.16:0/1220949532 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a076c75370 0x55a076c6c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:20.557 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.552+0000 7f2ebc880640 1 -- 172.21.15.16:0/1220949532 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a076c6f9a0 msgr2=0x55a076c6bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:06:20.557 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.552+0000 7f2ebc880640 1 --2- 172.21.15.16:0/1220949532 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a076c6f9a0 0x55a076c6bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:20.557 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.552+0000 7f2ebc880640 1 -- 172.21.15.16:0/1220949532 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a076c75370 msgr2=0x55a076c6c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:06:20.557 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.552+0000 7f2ebc880640 1 --2- 172.21.15.16:0/1220949532 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a076c75370 0x55a076c6c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:20.557 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.552+0000 7f2ebc880640 1 -- 172.21.15.16:0/1220949532 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a076c750d0 con 0x55a076c702e0 2024-09-12T02:06:20.558 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.552+0000 7f2ebc880640 1 --2- 172.21.15.16:0/1220949532 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a076c702e0 0x55a076c810a0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f2eb4003840 tx=0x7f2eb4003d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48234703678aa21b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:06:20.558 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.552+0000 7f2e9e7fc640 1 -- 172.21.15.16:0/1220949532 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eb4003970 con 0x55a076c702e0 2024-09-12T02:06:20.558 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.552+0000 7f2e9e7fc640 1 -- 172.21.15.16:0/1220949532 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2eb4003b10 con 0x55a076c702e0 2024-09-12T02:06:20.558 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.552+0000 7f2e9e7fc640 1 -- 172.21.15.16:0/1220949532 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eb4025a00 con 0x55a076c702e0 2024-09-12T02:06:20.558 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.553+0000 7f2ec1884740 1 -- 172.21.15.16:0/1220949532 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a076c702e0 msgr2=0x55a076c810a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:20.558 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.553+0000 7f2ec1884740 1 --2- 172.21.15.16:0/1220949532 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a076c702e0 0x55a076c810a0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f2eb4003840 tx=0x7f2eb4003d70 comp rx=0 tx=0).stop 2024-09-12T02:06:20.558 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.553+0000 7f2ec1884740 1 -- 172.21.15.16:0/1220949532 shutdown_connections 2024-09-12T02:06:20.558 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.553+0000 7f2ec1884740 1 --2- 172.21.15.16:0/1220949532 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a076c75370 0x55a076c6c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:20.558 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.553+0000 7f2ec1884740 1 --2- 172.21.15.16:0/1220949532 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a076c702e0 0x55a076c810a0 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:20.558 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.553+0000 7f2ec1884740 1 --2- 172.21.15.16:0/1220949532 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a076c6f9a0 0x55a076c6bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:20.559 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.553+0000 7f2ec1884740 1 -- 172.21.15.16:0/1220949532 >> 172.21.15.16:0/1220949532 conn(0x55a076c6b010 msgr2=0x55a076c79910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:06:20.559 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.553+0000 7f2ec1884740 1 -- 172.21.15.16:0/1220949532 shutdown_connections 2024-09-12T02:06:20.559 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.553+0000 7f2ec1884740 1 -- 172.21.15.16:0/1220949532 wait complete. 2024-09-12T02:06:20.559 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.553+0000 7f2ec1884740 1 -- start start 2024-09-12T02:06:20.559 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.553+0000 7f2ec1884740 1 Processor -- start 2024-09-12T02:06:20.559 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.554+0000 7f2ec1884740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a076c6f9a0 0x55a076c7e950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:20.559 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.554+0000 7f2ec1884740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a076c702e0 0x55a076c7ee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:20.559 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.554+0000 7f2ec1884740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a076c75370 0x55a076c7fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:20.559 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.554+0000 7f2ec1884740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2eb81a9670 con 0x55a076c6f9a0 2024-09-12T02:06:20.559 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.554+0000 7f2ec1884740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2eb81aaae0 con 0x55a076c702e0 2024-09-12T02:06:20.559 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.554+0000 7f2ec1884740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f2eb81a9d40 con 0x55a076c75370 2024-09-12T02:06:20.559 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.554+0000 7f2ec1884740 10 client.0 _ll_register_callbacks cb 0x7f2ea8009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:06:20.559 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.554+0000 7f2eaffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a076c6f9a0 0x55a076c7e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:20.559 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.554+0000 7f2eaffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a076c6f9a0 0x55a076c7e950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34744/0 (socket says 172.21.15.16:34744) 2024-09-12T02:06:20.560 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.554+0000 7f2eaffff640 1 -- 172.21.15.16:0/3720963014 learned_addr learned my addr 172.21.15.16:0/3720963014 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:06:20.560 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.554+0000 7f2ebc880640 1 --2- 172.21.15.16:0/3720963014 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a076c702e0 0x55a076c7ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:20.560 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.554+0000 7f2eaffff640 1 --2- 172.21.15.16:0/3720963014 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a076c75370 0x55a076c7fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:20.560 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.554+0000 7f2eaffff640 1 -- 172.21.15.16:0/3720963014 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a076c75370 msgr2=0x55a076c7fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:06:20.560 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.554+0000 7f2eaffff640 1 --2- 172.21.15.16:0/3720963014 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a076c75370 0x55a076c7fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:20.560 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.554+0000 7f2eaffff640 1 -- 172.21.15.16:0/3720963014 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a076c702e0 msgr2=0x55a076c7ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:06:20.560 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.554+0000 7f2eaffff640 1 --2- 172.21.15.16:0/3720963014 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a076c702e0 0x55a076c7ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:20.560 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.554+0000 7f2eaffff640 1 -- 172.21.15.16:0/3720963014 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a076c80460 con 0x55a076c6f9a0 2024-09-12T02:06:20.560 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.554+0000 7f2eaffff640 1 --2- 172.21.15.16:0/3720963014 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a076c6f9a0 0x55a076c7e950 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f2ea8009730 tx=0x7f2ea80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13584bdbc657d717 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:06:20.560 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.554+0000 7f2e97fff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:06:20.560 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.555+0000 7f2e97fff640 1 -- 172.21.15.16:0/3720963014 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ea802f020 con 0x55a076c6f9a0 2024-09-12T02:06:20.560 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.555+0000 7f2e97fff640 1 -- 172.21.15.16:0/3720963014 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f2ea800eaa0 con 0x55a076c6f9a0 2024-09-12T02:06:20.560 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.555+0000 7f2ec1884740 1 -- 172.21.15.16:0/3720963014 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2eb4003b10 con 0x55a076c6f9a0 2024-09-12T02:06:20.561 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.555+0000 7f2e97fff640 1 -- 172.21.15.16:0/3720963014 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2ea8040050 con 0x55a076c6f9a0 2024-09-12T02:06:20.561 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.555+0000 7f2ec1884740 10 client.16791 fetch_fsmap learned FSMap version 25 2024-09-12T02:06:20.561 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.555+0000 7f2ec1884740 1 -- 172.21.15.16:0/3720963014 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a076cf0710 con 0x55a076c6f9a0 2024-09-12T02:06:20.561 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.555+0000 7f2e97fff640 1 -- 172.21.15.16:0/3720963014 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2eb4003b10 con 0x55a076c6f9a0 2024-09-12T02:06:20.561 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.555+0000 7f2e97fff640 1 -- 172.21.15.16:0/3720963014 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f2ea801d040 con 0x55a076c6f9a0 2024-09-12T02:06:20.561 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.555+0000 7f2ec1884740 10 client.16791 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:06:20.561 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.555+0000 7f2ec1884740 10 client.16791 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:06:20.561 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.555+0000 7f2ec1884740 20 client.16791 populate_metadata read hostname 'smithi016' 2024-09-12T02:06:20.561 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.555+0000 7f2ec1884740 1 --2- 172.21.15.16:0/3720963014 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55a076cf1060 0x55a076cf3480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:20.561 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.555+0000 7f2ec1884740 4 client.16791 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:06:20.561 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.555+0000 7f2ec1884740 1 -- 172.21.15.16:0/3720963014 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a076bdad70 con 0x55a076cf1060 2024-09-12T02:06:20.561 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.555+0000 7f2ebc880640 1 --2- 172.21.15.16:0/3720963014 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55a076cf1060 0x55a076cf3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:20.561 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.556+0000 7f2ebc880640 1 --2- 172.21.15.16:0/3720963014 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55a076cf1060 0x55a076cf3480 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79f6bfc605778695 server_cookie=e5316834889eb9e8 in_seq=0 out_seq=0 2024-09-12T02:06:20.562 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.556+0000 7f2e97fff640 10 client.16791 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:06:20.566 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.566+0000 7f2e97fff640 1 -- 172.21.15.16:0/3720963014 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a076bdad70 con 0x55a076cf1060 2024-09-12T02:06:20.566 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.566+0000 7f2e97fff640 10 client.16791 handle_command_reply: tid=0 2024-09-12T02:06:20.566 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.566+0000 7f2ec1884740 1 client.16791 shutdown 2024-09-12T02:06:20.566 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.566+0000 7f2ec1884740 1 -- 172.21.15.16:0/3720963014 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a076c6f9a0 msgr2=0x55a076c7e950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:20.566 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.566+0000 7f2ec1884740 1 --2- 172.21.15.16:0/3720963014 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a076c6f9a0 0x55a076c7e950 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f2ea8009730 tx=0x7f2ea80046c0 comp rx=0 tx=0).stop 2024-09-12T02:06:20.566 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.566+0000 7f2ec1884740 1 -- 172.21.15.16:0/3720963014 shutdown_connections 2024-09-12T02:06:20.567 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.566+0000 7f2ec1884740 1 --2- 172.21.15.16:0/3720963014 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55a076cf1060 0x55a076cf3480 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:20.567 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.566+0000 7f2ec1884740 1 --2- 172.21.15.16:0/3720963014 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a076c75370 0x55a076c7fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:20.567 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.566+0000 7f2ec1884740 1 --2- 172.21.15.16:0/3720963014 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a076c702e0 0x55a076c7ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:20.567 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.566+0000 7f2e97fff640 0 client.16791 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:06:20.567 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.566+0000 7f2ec1884740 1 --2- 172.21.15.16:0/3720963014 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a076c6f9a0 0x55a076c7e950 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:20.567 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.566+0000 7f2ec1884740 1 -- 172.21.15.16:0/3720963014 >> 172.21.15.16:0/3720963014 conn(0x55a076c6b010 msgr2=0x55a076c699a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:06:20.567 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.566+0000 7f2ec1884740 1 -- 172.21.15.16:0/3720963014 shutdown_connections 2024-09-12T02:06:20.567 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.566+0000 7f2ec1884740 1 -- 172.21.15.16:0/3720963014 wait complete. 2024-09-12T02:06:20.567 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.566+0000 7f2ec1884740 20 client.16791 trim_cache size 0 max 16384 2024-09-12T02:06:20.567 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-09-12T02:06:20.568 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-09-12T02:06:20.568 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-09-12T02:06:20.568 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-09-12T02:06:20.569 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.567+0000 7f2ebeae4640 1 -- 172.21.15.16:0/1329564321 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f2ea00a2cb0 msgr2=0x7f2ea00a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:20.569 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.567+0000 7f2ebeae4640 1 --2- 172.21.15.16:0/1329564321 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f2ea00a2cb0 0x7f2ea00a5170 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f2eb004c000 tx=0x7f2eb00b4040 comp rx=0 tx=0).stop 2024-09-12T02:06:20.569 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.567+0000 7f2ebeae4640 1 -- 172.21.15.16:0/1329564321 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2eb81025f0 msgr2=0x7f2eb8103550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:20.569 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.567+0000 7f2ebeae4640 1 --2- 172.21.15.16:0/1329564321 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2eb81025f0 0x7f2eb8103550 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f2eb0031ae0 tx=0x7f2eb0031ee0 comp rx=0 tx=0).stop 2024-09-12T02:06:20.569 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.567+0000 7f2ebeae4640 1 -- 172.21.15.16:0/1329564321 shutdown_connections 2024-09-12T02:06:20.569 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.567+0000 7f2ebeae4640 1 --2- 172.21.15.16:0/1329564321 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f2ea00a2cb0 0x7f2ea00a5170 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:20.569 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.567+0000 7f2ebeae4640 1 --2- 172.21.15.16:0/1329564321 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2eb81025f0 0x7f2eb8103550 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:20.569 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.567+0000 7f2ebeae4640 1 --2- 172.21.15.16:0/1329564321 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2eb8101c30 0x7f2eb8103010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:20.569 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.567+0000 7f2ebeae4640 1 --2- 172.21.15.16:0/1329564321 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2eb8100a40 0x7f2eb81049d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:20.569 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.567+0000 7f2ebeae4640 1 -- 172.21.15.16:0/1329564321 >> 172.21.15.16:0/1329564321 conn(0x7f2eb80fc1e0 msgr2=0x7f2eb80fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:06:20.569 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.567+0000 7f2ebeae4640 1 -- 172.21.15.16:0/1329564321 shutdown_connections 2024-09-12T02:06:20.569 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:20.567+0000 7f2ebeae4640 1 -- 172.21.15.16:0/1329564321 wait complete. 2024-09-12T02:06:21.521 INFO:tasks.workunit.client.0.smithi016.stdout:64/600: open fd = 3 2024-09-12T02:06:21.521 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:06:21.521 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:06:21.521 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:06:21.521 INFO:tasks.workunit.client.0.smithi016.stdout:65/600: open fd = 3 2024-09-12T02:06:21.521 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:06:21.521 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:06:21.522 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:06:21.522 INFO:tasks.workunit.client.0.smithi016.stdout:66/600: open fd = 3 2024-09-12T02:06:21.522 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:06:21.522 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:06:21.522 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:06:21.522 INFO:tasks.workunit.client.0.smithi016.stdout:67/600: open fd = 3 2024-09-12T02:06:21.522 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:06:21.522 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:06:21.522 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:06:21.522 INFO:tasks.workunit.client.0.smithi016.stdout:68/600: open fd = 3 2024-09-12T02:06:21.522 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:06:21.522 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:06:21.522 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:06:21.523 INFO:tasks.workunit.client.0.smithi016.stdout:69/600: open fd = 3 2024-09-12T02:06:21.523 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:06:21.523 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:06:21.523 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:06:21.523 INFO:tasks.workunit.client.0.smithi016.stdout:70/600: open fd = 3 2024-09-12T02:06:21.523 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:06:21.523 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:06:21.523 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:06:21.523 INFO:tasks.workunit.client.0.smithi016.stdout:71/600: open fd = 3 2024-09-12T02:06:21.523 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:06:21.523 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:06:21.523 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:06:21.523 INFO:tasks.workunit.client.0.smithi016.stdout:72/600: open fd = 3 2024-09-12T02:06:21.523 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:06:21.524 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:06:21.524 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:06:21.524 INFO:tasks.workunit.client.0.smithi016.stdout:73/600: open fd = 3 2024-09-12T02:06:21.524 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:06:21.524 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:06:21.524 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:06:21.524 INFO:tasks.workunit.client.0.smithi016.stdout:74/600: open fd = 3 2024-09-12T02:06:21.524 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:06:21.524 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:06:21.524 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:06:21.524 INFO:tasks.workunit.client.0.smithi016.stdout:75/600: open fd = 3 2024-09-12T02:06:21.524 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:06:21.525 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:06:21.525 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:06:21.525 INFO:tasks.workunit.client.0.smithi016.stdout:76/600: open fd = 3 2024-09-12T02:06:21.525 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:06:21.525 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:06:21.525 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:06:21.525 INFO:tasks.workunit.client.0.smithi016.stdout:77/600: open fd = 3 2024-09-12T02:06:21.525 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:06:21.525 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:06:21.525 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:06:21.525 INFO:tasks.workunit.client.0.smithi016.stdout:78/600: open fd = 3 2024-09-12T02:06:21.525 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:06:21.526 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:06:21.526 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:06:21.526 INFO:tasks.workunit.client.0.smithi016.stdout:79/600: open fd = 3 2024-09-12T02:06:21.526 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:06:21.526 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:06:21.526 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:06:21.526 INFO:tasks.workunit.client.0.smithi016.stdout:80/600: open fd = 3 2024-09-12T02:06:21.526 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:06:21.526 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:06:21.526 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:06:21.526 INFO:tasks.workunit.client.0.smithi016.stdout:81/600: open fd = 3 2024-09-12T02:06:21.526 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:06:21.526 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:06:21.526 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:06:21.527 INFO:tasks.workunit.client.0.smithi016.stdout:82/600: open fd = 3 2024-09-12T02:06:21.527 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:06:21.527 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:06:21.527 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:06:21.527 INFO:tasks.workunit.client.0.smithi016.stdout:83/600: open fd = 3 2024-09-12T02:06:21.527 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:06:21.527 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:06:21.527 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:06:21.527 INFO:tasks.workunit.client.0.smithi016.stdout:84/600: open fd = 3 2024-09-12T02:06:21.527 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:06:21.527 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:06:21.527 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:06:21.527 INFO:tasks.workunit.client.0.smithi016.stdout:85/600: open fd = 3 2024-09-12T02:06:21.528 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:06:21.528 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:06:21.528 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:06:21.528 INFO:tasks.workunit.client.0.smithi016.stdout:86/600: open fd = 3 2024-09-12T02:06:21.528 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:06:21.528 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:06:21.528 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:06:21.528 INFO:tasks.workunit.client.0.smithi016.stdout:87/600: open fd = 3 2024-09-12T02:06:21.528 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:06:21.528 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:06:21.528 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:06:21.528 INFO:tasks.workunit.client.0.smithi016.stdout:88/600: open fd = 3 2024-09-12T02:06:21.528 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:06:21.529 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:06:21.529 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:06:21.529 INFO:tasks.workunit.client.0.smithi016.stdout:89/600: open fd = 3 2024-09-12T02:06:21.529 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:06:21.529 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:06:21.529 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:06:21.529 INFO:tasks.workunit.client.0.smithi016.stdout:90/600: open fd = 3 2024-09-12T02:06:21.529 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:06:21.529 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:06:21.529 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:06:21.529 INFO:tasks.workunit.client.0.smithi016.stdout:91/600: open fd = 3 2024-09-12T02:06:21.529 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:06:21.530 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:06:21.530 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:06:21.530 INFO:tasks.workunit.client.0.smithi016.stdout:92/600: open fd = 3 2024-09-12T02:06:21.530 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:06:21.530 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:06:21.530 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:06:21.530 INFO:tasks.workunit.client.0.smithi016.stdout:93/600: open fd = 3 2024-09-12T02:06:21.530 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:06:21.530 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:06:21.530 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:06:21.530 INFO:tasks.workunit.client.0.smithi016.stdout:94/600: open fd = 3 2024-09-12T02:06:21.530 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:06:21.530 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:06:21.530 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:06:21.531 INFO:tasks.workunit.client.0.smithi016.stdout:95/600: open fd = 3 2024-09-12T02:06:21.531 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:06:21.531 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:06:21.531 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:06:21.531 INFO:tasks.workunit.client.0.smithi016.stdout:96/600: open fd = 3 2024-09-12T02:06:21.531 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:06:21.531 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:06:21.531 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:06:21.531 INFO:tasks.workunit.client.0.smithi016.stdout:97/600: open fd = 3 2024-09-12T02:06:21.531 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:06:21.531 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:06:21.531 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:06:21.531 INFO:tasks.workunit.client.0.smithi016.stdout:98/600: open fd = 3 2024-09-12T02:06:21.532 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:06:21.532 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:06:21.532 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:06:21.532 INFO:tasks.workunit.client.0.smithi016.stdout:99/600: open fd = 3 2024-09-12T02:06:21.532 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:06:21.532 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:06:21.532 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:06:21.532 INFO:tasks.workunit.client.0.smithi016.stdout:100/600: open fd = 3 2024-09-12T02:06:21.532 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:06:21.533 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:06:21.533 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:06:21.533 INFO:tasks.workunit.client.0.smithi016.stdout:101/600: open fd = 3 2024-09-12T02:06:21.533 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:06:21.533 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:06:21.534 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:06:21.534 INFO:tasks.workunit.client.0.smithi016.stdout:102/600: open fd = 3 2024-09-12T02:06:21.534 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:06:21.534 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:06:21.534 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:06:21.534 INFO:tasks.workunit.client.0.smithi016.stdout:103/600: open fd = 3 2024-09-12T02:06:21.534 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:06:21.534 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:06:21.534 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:06:21.534 INFO:tasks.workunit.client.0.smithi016.stdout:104/600: open fd = 3 2024-09-12T02:06:21.535 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:06:21.535 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:06:21.535 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:06:21.535 INFO:tasks.workunit.client.0.smithi016.stdout:105/600: open fd = 3 2024-09-12T02:06:21.535 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:06:21.535 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:06:21.535 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:06:21.535 INFO:tasks.workunit.client.0.smithi016.stdout:106/600: open fd = 3 2024-09-12T02:06:21.535 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:06:21.535 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:06:21.535 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:06:21.535 INFO:tasks.workunit.client.0.smithi016.stdout:107/600: open fd = 3 2024-09-12T02:06:21.535 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:06:21.536 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:06:21.536 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:06:21.536 INFO:tasks.workunit.client.0.smithi016.stdout:108/600: open fd = 3 2024-09-12T02:06:21.536 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:06:21.536 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:06:21.536 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:06:21.536 INFO:tasks.workunit.client.0.smithi016.stdout:109/600: open fd = 3 2024-09-12T02:06:21.536 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:06:21.536 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:06:21.536 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:06:21.536 INFO:tasks.workunit.client.0.smithi016.stdout:110/600: open fd = 3 2024-09-12T02:06:21.536 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:06:21.536 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:06:21.536 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:06:21.537 INFO:tasks.workunit.client.0.smithi016.stdout:111/600: open fd = 3 2024-09-12T02:06:21.537 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:06:21.537 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:06:21.537 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:06:21.537 INFO:tasks.workunit.client.0.smithi016.stdout:112/600: open fd = 3 2024-09-12T02:06:21.537 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:06:21.537 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:06:21.537 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:06:21.537 INFO:tasks.workunit.client.0.smithi016.stdout:113/600: open fd = 3 2024-09-12T02:06:21.537 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:06:21.537 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:06:21.537 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:06:21.537 INFO:tasks.workunit.client.0.smithi016.stdout:114/600: open fd = 3 2024-09-12T02:06:21.538 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:06:21.538 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:06:21.538 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:06:21.538 INFO:tasks.workunit.client.0.smithi016.stdout:115/600: open fd = 3 2024-09-12T02:06:21.538 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:06:21.538 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:06:21.538 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:06:21.538 INFO:tasks.workunit.client.0.smithi016.stdout:116/600: open fd = 3 2024-09-12T02:06:21.538 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:06:21.538 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:06:21.538 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:06:21.538 INFO:tasks.workunit.client.0.smithi016.stdout:117/600: open fd = 3 2024-09-12T02:06:21.538 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:06:21.538 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:06:21.539 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:06:21.539 INFO:tasks.workunit.client.0.smithi016.stdout:118/600: open fd = 3 2024-09-12T02:06:21.539 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:06:21.539 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:06:21.539 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:06:21.539 INFO:tasks.workunit.client.0.smithi016.stdout:119/600: open fd = 3 2024-09-12T02:06:21.539 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:06:21.539 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:06:21.539 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:06:21.539 INFO:tasks.workunit.client.0.smithi016.stdout:120/600: open fd = 3 2024-09-12T02:06:21.539 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:06:21.539 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:06:21.539 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:06:21.539 INFO:tasks.workunit.client.0.smithi016.stdout:121/600: open fd = 3 2024-09-12T02:06:21.540 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:06:21.540 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:06:21.540 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:06:21.540 INFO:tasks.workunit.client.0.smithi016.stdout:122/600: open fd = 3 2024-09-12T02:06:21.540 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:06:21.540 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:06:21.540 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:06:21.540 INFO:tasks.workunit.client.0.smithi016.stdout:123/600: open fd = 3 2024-09-12T02:06:21.540 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:06:21.540 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:06:21.540 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:06:21.540 INFO:tasks.workunit.client.0.smithi016.stdout:124/600: open fd = 3 2024-09-12T02:06:21.540 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:06:21.541 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:06:21.541 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:06:21.541 INFO:tasks.workunit.client.0.smithi016.stdout:125/600: open fd = 3 2024-09-12T02:06:21.541 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:06:21.541 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:06:21.541 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:06:21.541 INFO:tasks.workunit.client.0.smithi016.stdout:126/600: open fd = 3 2024-09-12T02:06:21.541 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:06:22.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:06:22 smithi184 ceph-mon[35593]: pgmap v468: 97 pgs: 97 active+clean; 2.2 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 6.0 KiB/s wr, 10 op/s 2024-09-12T02:06:22.418 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:06:22 smithi033 ceph-mon[35332]: pgmap v468: 97 pgs: 97 active+clean; 2.2 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 6.0 KiB/s wr, 10 op/s 2024-09-12T02:06:22.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:06:22 smithi016 ceph-mon[30947]: pgmap v468: 97 pgs: 97 active+clean; 2.2 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 6.0 KiB/s wr, 10 op/s 2024-09-12T02:06:24.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:06:24 smithi184 ceph-mon[35593]: pgmap v469: 97 pgs: 97 active+clean; 2.2 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 4.8 KiB/s wr, 8 op/s 2024-09-12T02:06:24.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:06:24 smithi033 ceph-mon[35332]: pgmap v469: 97 pgs: 97 active+clean; 2.2 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 4.8 KiB/s wr, 8 op/s 2024-09-12T02:06:24.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:06:24 smithi016 ceph-mon[30947]: pgmap v469: 97 pgs: 97 active+clean; 2.2 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 4.8 KiB/s wr, 8 op/s 2024-09-12T02:06:26.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:06:26 smithi184 ceph-mon[35593]: pgmap v470: 97 pgs: 97 active+clean; 2.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-09-12T02:06:26.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:06:26 smithi033 ceph-mon[35332]: pgmap v470: 97 pgs: 97 active+clean; 2.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-09-12T02:06:26.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:06:26 smithi016 ceph-mon[30947]: pgmap v470: 97 pgs: 97 active+clean; 2.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-09-12T02:06:28.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:06:28 smithi184 ceph-mon[35593]: pgmap v471: 97 pgs: 97 active+clean; 2.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-09-12T02:06:28.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:06:28 smithi033 ceph-mon[35332]: pgmap v471: 97 pgs: 97 active+clean; 2.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-09-12T02:06:28.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:06:28 smithi016 ceph-mon[30947]: pgmap v471: 97 pgs: 97 active+clean; 2.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-09-12T02:06:30.306 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T02:06:30.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:06:30 smithi184 ceph-mon[35593]: pgmap v472: 97 pgs: 97 active+clean; 2.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T02:06:30.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:06:30 smithi033 ceph-mon[35332]: pgmap v472: 97 pgs: 97 active+clean; 2.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T02:06:30.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:06:30 smithi016 ceph-mon[30947]: pgmap v472: 97 pgs: 97 active+clean; 2.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T02:06:30.583 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:06:31.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.053+0000 7f3cdd90b640 1 -- 172.21.15.16:0/1299799705 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3cd8102d30 msgr2=0x7f3cd8103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:31.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.053+0000 7f3cdd90b640 1 --2- 172.21.15.16:0/1299799705 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3cd8102d30 0x7f3cd8103110 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f3cc8009930 tx=0x7f3cc802a420 comp rx=0 tx=0).stop 2024-09-12T02:06:31.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.053+0000 7f3cdd90b640 1 -- 172.21.15.16:0/1299799705 shutdown_connections 2024-09-12T02:06:31.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.053+0000 7f3cdd90b640 1 --2- 172.21.15.16:0/1299799705 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3cd81082a0 0x7f3cd810ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:31.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.053+0000 7f3cdd90b640 1 --2- 172.21.15.16:0/1299799705 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3cd81036e0 0x7f3cd8107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:31.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.053+0000 7f3cdd90b640 1 --2- 172.21.15.16:0/1299799705 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3cd8102d30 0x7f3cd8103110 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:31.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.053+0000 7f3cdd90b640 1 -- 172.21.15.16:0/1299799705 >> 172.21.15.16:0/1299799705 conn(0x7f3cd80fc7d0 msgr2=0x7f3cd80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:06:31.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.053+0000 7f3cdd90b640 1 -- 172.21.15.16:0/1299799705 shutdown_connections 2024-09-12T02:06:31.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.053+0000 7f3cdd90b640 1 -- 172.21.15.16:0/1299799705 wait complete. 2024-09-12T02:06:31.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.053+0000 7f3cdd90b640 1 Processor -- start 2024-09-12T02:06:31.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.054+0000 7f3cdd90b640 1 -- start start 2024-09-12T02:06:31.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.054+0000 7f3cdd90b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3cd8102d30 0x7f3cd8100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:31.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.054+0000 7f3cdd90b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3cd81036e0 0x7f3cd8100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:31.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.054+0000 7f3cdd90b640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3cd81082a0 0x7f3cd8101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:31.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.054+0000 7f3cdd90b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3cd8112100 con 0x7f3cd8102d30 2024-09-12T02:06:31.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.054+0000 7f3cdd90b640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3cd8111f80 con 0x7f3cd81036e0 2024-09-12T02:06:31.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.054+0000 7f3cdd90b640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f3cd8112280 con 0x7f3cd81082a0 2024-09-12T02:06:31.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.054+0000 7f3cd6ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3cd8102d30 0x7f3cd8100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:31.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.054+0000 7f3cd6ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3cd8102d30 0x7f3cd8100990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35468/0 (socket says 172.21.15.16:35468) 2024-09-12T02:06:31.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.054+0000 7f3cd6ffd640 1 -- 172.21.15.16:0/2330204970 learned_addr learned my addr 172.21.15.16:0/2330204970 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:06:31.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.054+0000 7f3cd67fc640 1 --2- 172.21.15.16:0/2330204970 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3cd81036e0 0x7f3cd8100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:31.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.054+0000 7f3cd77fe640 1 --2- 172.21.15.16:0/2330204970 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3cd81082a0 0x7f3cd8101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:31.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.054+0000 7f3cd6ffd640 1 -- 172.21.15.16:0/2330204970 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3cd81082a0 msgr2=0x7f3cd8101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:31.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.054+0000 7f3cd6ffd640 1 --2- 172.21.15.16:0/2330204970 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3cd81082a0 0x7f3cd8101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:31.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.054+0000 7f3cd6ffd640 1 -- 172.21.15.16:0/2330204970 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3cd81036e0 msgr2=0x7f3cd8100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:31.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.054+0000 7f3cd6ffd640 1 --2- 172.21.15.16:0/2330204970 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3cd81036e0 0x7f3cd8100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:31.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.054+0000 7f3cd6ffd640 1 -- 172.21.15.16:0/2330204970 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cd81a6ac0 con 0x7f3cd8102d30 2024-09-12T02:06:31.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.055+0000 7f3cd6ffd640 1 --2- 172.21.15.16:0/2330204970 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3cd8102d30 0x7f3cd8100990 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f3cc8009230 tx=0x7f3cc8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:06:31.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.055+0000 7f3cdc909640 1 -- 172.21.15.16:0/2330204970 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cc803f030 con 0x7f3cd8102d30 2024-09-12T02:06:31.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.055+0000 7f3cdc909640 1 -- 172.21.15.16:0/2330204970 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3cc8003ad0 con 0x7f3cd8102d30 2024-09-12T02:06:31.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.055+0000 7f3cdd90b640 1 -- 172.21.15.16:0/2330204970 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3cd81a6d50 con 0x7f3cd8102d30 2024-09-12T02:06:31.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.055+0000 7f3cdc909640 1 -- 172.21.15.16:0/2330204970 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cc8035070 con 0x7f3cd8102d30 2024-09-12T02:06:31.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.055+0000 7f3cdd90b640 1 -- 172.21.15.16:0/2330204970 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3cd81a71d0 con 0x7f3cd8102d30 2024-09-12T02:06:31.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.057+0000 7f3cdc909640 1 -- 172.21.15.16:0/2330204970 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f3cc8042030 con 0x7f3cd8102d30 2024-09-12T02:06:31.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.057+0000 7f3cdfce4740 1 Processor -- start 2024-09-12T02:06:31.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.057+0000 7f3cdfce4740 1 -- start start 2024-09-12T02:06:31.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.058+0000 7f3cdfce4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5599639af4a0 0x5599639af880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:31.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.058+0000 7f3cdfce4740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599639afdc0 0x5599639ba140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:31.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.058+0000 7f3cdfce4740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5599639ba680 0x5599639bca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:31.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.058+0000 7f3cdfce4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5599639152e0 con 0x5599639af4a0 2024-09-12T02:06:31.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.058+0000 7f3cdfce4740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5599638bcce0 con 0x5599639afdc0 2024-09-12T02:06:31.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.058+0000 7f3cdfce4740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5599639112a0 con 0x5599639ba680 2024-09-12T02:06:31.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.058+0000 7f3cd77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5599639af4a0 0x5599639af880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:31.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.058+0000 7f3cd77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5599639af4a0 0x5599639af880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35480/0 (socket says 172.21.15.16:35480) 2024-09-12T02:06:31.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.058+0000 7f3cd77fe640 1 -- 172.21.15.16:0/1240396819 learned_addr learned my addr 172.21.15.16:0/1240396819 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:06:31.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.058+0000 7f3cd6ffd640 1 --2- 172.21.15.16:0/1240396819 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599639afdc0 0x5599639ba140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:31.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.058+0000 7f3cd67fc640 1 --2- 172.21.15.16:0/1240396819 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5599639ba680 0x5599639bca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:31.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.058+0000 7f3cd77fe640 1 -- 172.21.15.16:0/1240396819 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5599639ba680 msgr2=0x5599639bca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:06:31.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.058+0000 7f3cd77fe640 1 --2- 172.21.15.16:0/1240396819 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5599639ba680 0x5599639bca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:31.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.058+0000 7f3cd77fe640 1 -- 172.21.15.16:0/1240396819 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599639afdc0 msgr2=0x5599639ba140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:06:31.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.058+0000 7f3cd77fe640 1 --2- 172.21.15.16:0/1240396819 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599639afdc0 0x5599639ba140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:31.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.058+0000 7f3cd77fe640 1 -- 172.21.15.16:0/1240396819 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5599638a1440 con 0x5599639af4a0 2024-09-12T02:06:31.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.058+0000 7f3cdc909640 1 --2- 172.21.15.16:0/2330204970 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f3cc40a2b80 0x7f3cc40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:31.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.058+0000 7f3cd6ffd640 1 --2- 172.21.15.16:0/1240396819 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599639afdc0 0x5599639ba140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:06:31.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.058+0000 7f3cd67fc640 1 --2- 172.21.15.16:0/2330204970 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f3cc40a2b80 0x7f3cc40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:31.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.058+0000 7f3cdc909640 1 -- 172.21.15.16:0/2330204970 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f3cc80ef080 con 0x7f3cd8102d30 2024-09-12T02:06:31.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.058+0000 7f3cd77fe640 1 --2- 172.21.15.16:0/1240396819 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5599639af4a0 0x5599639af880 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f3cd000a430 tx=0x7f3cd0009650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a81227115f3fb780 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:06:31.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.059+0000 7f3ccdffb640 1 -- 172.21.15.16:0/1240396819 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cd0013790 con 0x5599639af4a0 2024-09-12T02:06:31.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.059+0000 7f3ccdffb640 1 -- 172.21.15.16:0/1240396819 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3cd0019070 con 0x5599639af4a0 2024-09-12T02:06:31.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.059+0000 7f3ccdffb640 1 -- 172.21.15.16:0/1240396819 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cd0014de0 con 0x5599639af4a0 2024-09-12T02:06:31.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.059+0000 7f3cdfce4740 1 -- 172.21.15.16:0/1240396819 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5599639af4a0 msgr2=0x5599639af880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:31.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.059+0000 7f3cd67fc640 1 --2- 172.21.15.16:0/2330204970 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f3cc40a2b80 0x7f3cc40a5040 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x5599639acdb0 tx=0x7f3cc000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:06:31.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.059+0000 7f3cdfce4740 1 --2- 172.21.15.16:0/1240396819 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5599639af4a0 0x5599639af880 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f3cd000a430 tx=0x7f3cd0009650 comp rx=0 tx=0).stop 2024-09-12T02:06:31.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.059+0000 7f3cdfce4740 1 -- 172.21.15.16:0/1240396819 shutdown_connections 2024-09-12T02:06:31.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.059+0000 7f3cdfce4740 1 --2- 172.21.15.16:0/1240396819 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5599639ba680 0x5599639bca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:31.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.059+0000 7f3cdfce4740 1 --2- 172.21.15.16:0/1240396819 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599639afdc0 0x5599639ba140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:31.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.059+0000 7f3cdfce4740 1 --2- 172.21.15.16:0/1240396819 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5599639af4a0 0x5599639af880 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:31.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.059+0000 7f3cdfce4740 1 -- 172.21.15.16:0/1240396819 >> 172.21.15.16:0/1240396819 conn(0x5599639aba30 msgr2=0x5599639abe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:06:31.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.059+0000 7f3cdfce4740 1 -- 172.21.15.16:0/1240396819 shutdown_connections 2024-09-12T02:06:31.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.060+0000 7f3cdfce4740 1 -- 172.21.15.16:0/1240396819 wait complete. 2024-09-12T02:06:31.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.060+0000 7f3cdfce4740 1 -- start start 2024-09-12T02:06:31.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.060+0000 7f3cdfce4740 1 Processor -- start 2024-09-12T02:06:31.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.060+0000 7f3cdfce4740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599639af4a0 0x5599639b9f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:31.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.060+0000 7f3cdfce4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5599639afdc0 0x5599639b7200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:31.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.060+0000 7f3cdfce4740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5599639ba680 0x5599639b7740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:31.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.060+0000 7f3cdfce4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5599639bd250 con 0x5599639afdc0 2024-09-12T02:06:31.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.060+0000 7f3cdfce4740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5599638d0050 con 0x5599639af4a0 2024-09-12T02:06:31.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.060+0000 7f3cdfce4740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5599639bd3d0 con 0x5599639ba680 2024-09-12T02:06:31.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.060+0000 7f3cdfce4740 10 client.0 _ll_register_callbacks cb 0x559963910160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:06:31.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.060+0000 7f3cd67fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5599639afdc0 0x5599639b7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:31.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.061+0000 7f3cd67fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5599639afdc0 0x5599639b7200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35494/0 (socket says 172.21.15.16:35494) 2024-09-12T02:06:31.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.061+0000 7f3cd67fc640 1 -- 172.21.15.16:0/3824565422 learned_addr learned my addr 172.21.15.16:0/3824565422 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:06:31.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.061+0000 7f3cd6ffd640 1 --2- 172.21.15.16:0/3824565422 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599639af4a0 0x5599639b9f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:31.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.061+0000 7f3cd77fe640 1 --2- 172.21.15.16:0/3824565422 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5599639ba680 0x5599639b7740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:31.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.061+0000 7f3cd67fc640 1 -- 172.21.15.16:0/3824565422 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5599639ba680 msgr2=0x5599639b7740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:06:31.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.061+0000 7f3cd67fc640 1 --2- 172.21.15.16:0/3824565422 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5599639ba680 0x5599639b7740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:31.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.061+0000 7f3cd67fc640 1 -- 172.21.15.16:0/3824565422 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599639af4a0 msgr2=0x5599639b9f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:06:31.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.061+0000 7f3cd67fc640 1 --2- 172.21.15.16:0/3824565422 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599639af4a0 0x5599639b9f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:31.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.061+0000 7f3cd67fc640 1 -- 172.21.15.16:0/3824565422 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599636863c0 con 0x5599639afdc0 2024-09-12T02:06:31.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.061+0000 7f3cd6ffd640 1 --2- 172.21.15.16:0/3824565422 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599639af4a0 0x5599639b9f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:06:31.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.061+0000 7f3cd67fc640 1 --2- 172.21.15.16:0/3824565422 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5599639afdc0 0x5599639b7200 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f3cc0003e20 tx=0x7f3cc00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=111f98cc9e5323d4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:06:31.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.061+0000 7f3cb6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:06:31.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.061+0000 7f3cb6ffd640 1 -- 172.21.15.16:0/3824565422 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cc001b660 con 0x5599639afdc0 2024-09-12T02:06:31.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.061+0000 7f3cb6ffd640 1 -- 172.21.15.16:0/3824565422 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f3cc001b800 con 0x5599639afdc0 2024-09-12T02:06:31.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.061+0000 7f3cdfce4740 1 -- 172.21.15.16:0/3824565422 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3cd0019070 con 0x5599639afdc0 2024-09-12T02:06:31.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.061+0000 7f3cb6ffd640 1 -- 172.21.15.16:0/3824565422 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f3cc001ba40 con 0x5599639afdc0 2024-09-12T02:06:31.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.061+0000 7f3cdfce4740 10 client.16803 fetch_fsmap learned FSMap version 25 2024-09-12T02:06:31.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.061+0000 7f3cdfce4740 1 -- 172.21.15.16:0/3824565422 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5599639c8c30 con 0x5599639afdc0 2024-09-12T02:06:31.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.062+0000 7f3cb6ffd640 1 -- 172.21.15.16:0/3824565422 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3cd0019070 con 0x5599639afdc0 2024-09-12T02:06:31.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.062+0000 7f3cb6ffd640 1 -- 172.21.15.16:0/3824565422 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f3cc0024070 con 0x5599639afdc0 2024-09-12T02:06:31.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.062+0000 7f3cdfce4740 10 client.16803 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:06:31.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.062+0000 7f3cdfce4740 10 client.16803 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:06:31.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.062+0000 7f3cdfce4740 20 client.16803 populate_metadata read hostname 'smithi016' 2024-09-12T02:06:31.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.062+0000 7f3cdfce4740 1 --2- 172.21.15.16:0/3824565422 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5599639ca040 0x5599639cc4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:31.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.062+0000 7f3cdfce4740 4 client.16803 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:06:31.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.062+0000 7f3cdfce4740 1 -- 172.21.15.16:0/3824565422 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559963922850 con 0x5599639ca040 2024-09-12T02:06:31.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.062+0000 7f3cd6ffd640 1 --2- 172.21.15.16:0/3824565422 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5599639ca040 0x5599639cc4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:31.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.063+0000 7f3cd6ffd640 1 --2- 172.21.15.16:0/3824565422 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5599639ca040 0x5599639cc4c0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a1d2199f49fee24 server_cookie=678c4175b40bbeec in_seq=0 out_seq=0 2024-09-12T02:06:31.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.063+0000 7f3cb6ffd640 10 client.16803 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:06:31.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.063+0000 7f3cb6ffd640 1 -- 172.21.15.16:0/3824565422 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559963922850 con 0x5599639ca040 2024-09-12T02:06:31.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.063+0000 7f3cb6ffd640 10 client.16803 handle_command_reply: tid=0 2024-09-12T02:06:31.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.063+0000 7f3cdfce4740 1 client.16803 shutdown 2024-09-12T02:06:31.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.064+0000 7f3cdfce4740 1 -- 172.21.15.16:0/3824565422 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5599639afdc0 msgr2=0x5599639b7200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:31.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.064+0000 7f3cdfce4740 1 --2- 172.21.15.16:0/3824565422 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5599639afdc0 0x5599639b7200 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f3cc0003e20 tx=0x7f3cc00046c0 comp rx=0 tx=0).stop 2024-09-12T02:06:31.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.064+0000 7f3cdfce4740 1 -- 172.21.15.16:0/3824565422 shutdown_connections 2024-09-12T02:06:31.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.064+0000 7f3cdfce4740 1 --2- 172.21.15.16:0/3824565422 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5599639ca040 0x5599639cc4c0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:31.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.064+0000 7f3cb6ffd640 0 client.16803 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:06:31.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.064+0000 7f3cdfce4740 1 --2- 172.21.15.16:0/3824565422 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5599639ba680 0x5599639b7740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:31.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.064+0000 7f3cdfce4740 1 --2- 172.21.15.16:0/3824565422 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5599639afdc0 0x5599639b7200 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:31.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.064+0000 7f3cdfce4740 1 --2- 172.21.15.16:0/3824565422 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599639af4a0 0x5599639b9f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:31.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.064+0000 7f3cdfce4740 1 -- 172.21.15.16:0/3824565422 >> 172.21.15.16:0/3824565422 conn(0x5599639aba30 msgr2=0x5599639aab90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:06:31.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.064+0000 7f3cdfce4740 1 -- 172.21.15.16:0/3824565422 shutdown_connections 2024-09-12T02:06:31.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.064+0000 7f3cdfce4740 1 -- 172.21.15.16:0/3824565422 wait complete. 2024-09-12T02:06:31.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.064+0000 7f3cdfce4740 20 client.16803 trim_cache size 0 max 16384 2024-09-12T02:06:31.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.081+0000 7f3cdfce4740 1 Processor -- start 2024-09-12T02:06:31.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.081+0000 7f3cdfce4740 1 -- start start 2024-09-12T02:06:31.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.081+0000 7f3cdfce4740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599639af4a0 0x5599639c64e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:31.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.081+0000 7f3cdfce4740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5599639afdc0 0x5599639d1d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:31.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.081+0000 7f3cdfce4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5599639ba680 0x5599639d22d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:31.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.081+0000 7f3cdfce4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3cd81b1f70 con 0x5599639ba680 2024-09-12T02:06:31.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.081+0000 7f3cdfce4740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3cd81af840 con 0x5599639af4a0 2024-09-12T02:06:31.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.081+0000 7f3cdfce4740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f3cd81ad710 con 0x5599639afdc0 2024-09-12T02:06:31.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.081+0000 7f3cd77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5599639ba680 0x5599639d22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:31.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.081+0000 7f3cd77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5599639ba680 0x5599639d22d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35506/0 (socket says 172.21.15.16:35506) 2024-09-12T02:06:31.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.081+0000 7f3cd77fe640 1 -- 172.21.15.16:0/3606649953 learned_addr learned my addr 172.21.15.16:0/3606649953 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:06:31.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.082+0000 7f3cd67fc640 1 --2- 172.21.15.16:0/3606649953 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5599639afdc0 0x5599639d1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:31.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.082+0000 7f3cd6ffd640 1 --2- 172.21.15.16:0/3606649953 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599639af4a0 0x5599639c64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:31.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.082+0000 7f3cd77fe640 1 -- 172.21.15.16:0/3606649953 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5599639afdc0 msgr2=0x5599639d1d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:06:31.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.082+0000 7f3cd77fe640 1 --2- 172.21.15.16:0/3606649953 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5599639afdc0 0x5599639d1d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:31.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.082+0000 7f3cd77fe640 1 -- 172.21.15.16:0/3606649953 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599639af4a0 msgr2=0x5599639c64e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:06:31.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.082+0000 7f3cd77fe640 1 --2- 172.21.15.16:0/3606649953 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599639af4a0 0x5599639c64e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:31.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.082+0000 7f3cd77fe640 1 -- 172.21.15.16:0/3606649953 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5599639acae0 con 0x5599639ba680 2024-09-12T02:06:31.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.082+0000 7f3cd6ffd640 1 --2- 172.21.15.16:0/3606649953 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599639af4a0 0x5599639c64e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:06:31.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.082+0000 7f3cd77fe640 1 --2- 172.21.15.16:0/3606649953 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5599639ba680 0x5599639d22d0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f3cd0014040 tx=0x7f3cd0009c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f92a29ad9987a65e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:06:31.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.082+0000 7f3ccd7fa640 1 -- 172.21.15.16:0/3606649953 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cd0014720 con 0x5599639ba680 2024-09-12T02:06:31.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.082+0000 7f3ccd7fa640 1 -- 172.21.15.16:0/3606649953 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3cd0019040 con 0x5599639ba680 2024-09-12T02:06:31.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.082+0000 7f3ccd7fa640 1 -- 172.21.15.16:0/3606649953 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cd002a480 con 0x5599639ba680 2024-09-12T02:06:31.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.083+0000 7f3cdfce4740 1 -- 172.21.15.16:0/3606649953 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5599639ba680 msgr2=0x5599639d22d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:31.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.083+0000 7f3cdfce4740 1 --2- 172.21.15.16:0/3606649953 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5599639ba680 0x5599639d22d0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f3cd0014040 tx=0x7f3cd0009c90 comp rx=0 tx=0).stop 2024-09-12T02:06:31.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.083+0000 7f3cdfce4740 1 -- 172.21.15.16:0/3606649953 shutdown_connections 2024-09-12T02:06:31.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.083+0000 7f3cdfce4740 1 --2- 172.21.15.16:0/3606649953 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5599639ba680 0x5599639d22d0 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:31.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.083+0000 7f3cdfce4740 1 --2- 172.21.15.16:0/3606649953 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5599639afdc0 0x5599639d1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:31.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.083+0000 7f3cdfce4740 1 --2- 172.21.15.16:0/3606649953 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599639af4a0 0x5599639c64e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:31.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.083+0000 7f3cdfce4740 1 -- 172.21.15.16:0/3606649953 >> 172.21.15.16:0/3606649953 conn(0x5599639aba30 msgr2=0x5599639b28e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:06:31.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.083+0000 7f3cdfce4740 1 -- 172.21.15.16:0/3606649953 shutdown_connections 2024-09-12T02:06:31.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.083+0000 7f3cdfce4740 1 -- 172.21.15.16:0/3606649953 wait complete. 2024-09-12T02:06:31.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.083+0000 7f3cdfce4740 1 -- start start 2024-09-12T02:06:31.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.083+0000 7f3cdfce4740 1 Processor -- start 2024-09-12T02:06:31.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.084+0000 7f3cdfce4740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599639af4a0 0x5599639c2830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:31.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.084+0000 7f3cdfce4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5599639afdc0 0x5599639c2d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:31.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.084+0000 7f3cdfce4740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5599639ba680 0x5599639b3a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:31.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.084+0000 7f3cdfce4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3cd81add60 con 0x5599639afdc0 2024-09-12T02:06:31.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.084+0000 7f3cdfce4740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3cd81af170 con 0x5599639af4a0 2024-09-12T02:06:31.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.084+0000 7f3cdfce4740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f3cd81ae3d0 con 0x5599639ba680 2024-09-12T02:06:31.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.084+0000 7f3cdfce4740 10 client.0 _ll_register_callbacks cb 0x7f3cc0003aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:06:31.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.084+0000 7f3cd67fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5599639afdc0 0x5599639c2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:31.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.084+0000 7f3cd67fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5599639afdc0 0x5599639c2d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35522/0 (socket says 172.21.15.16:35522) 2024-09-12T02:06:31.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.084+0000 7f3cd67fc640 1 -- 172.21.15.16:0/435492379 learned_addr learned my addr 172.21.15.16:0/435492379 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:06:31.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.084+0000 7f3cd77fe640 1 --2- 172.21.15.16:0/435492379 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5599639ba680 0x5599639b3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:31.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.084+0000 7f3cd6ffd640 1 --2- 172.21.15.16:0/435492379 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599639af4a0 0x5599639c2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:31.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.084+0000 7f3cd67fc640 1 -- 172.21.15.16:0/435492379 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5599639ba680 msgr2=0x5599639b3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:06:31.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.084+0000 7f3cd67fc640 1 --2- 172.21.15.16:0/435492379 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5599639ba680 0x5599639b3a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:31.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.084+0000 7f3cd67fc640 1 -- 172.21.15.16:0/435492379 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599639af4a0 msgr2=0x5599639c2830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:06:31.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.084+0000 7f3cd67fc640 1 --2- 172.21.15.16:0/435492379 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599639af4a0 0x5599639c2830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:31.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.084+0000 7f3cd67fc640 1 -- 172.21.15.16:0/435492379 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599639b4100 con 0x5599639afdc0 2024-09-12T02:06:31.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.084+0000 7f3cd6ffd640 1 --2- 172.21.15.16:0/435492379 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599639af4a0 0x5599639c2830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:06:31.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.084+0000 7f3cd67fc640 1 --2- 172.21.15.16:0/435492379 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5599639afdc0 0x5599639c2d70 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x5599639b8840 tx=0x7f3cc0005900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5bc653d3049fa3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:06:31.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.084+0000 7f3cb6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:06:31.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.085+0000 7f3cb6ffd640 1 -- 172.21.15.16:0/435492379 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cc00035f0 con 0x5599639afdc0 2024-09-12T02:06:31.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.085+0000 7f3cdfce4740 1 -- 172.21.15.16:0/435492379 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3cd0019040 con 0x5599639afdc0 2024-09-12T02:06:31.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.085+0000 7f3cb6ffd640 1 -- 172.21.15.16:0/435492379 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f3cc001b460 con 0x5599639afdc0 2024-09-12T02:06:31.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.085+0000 7f3cb6ffd640 1 -- 172.21.15.16:0/435492379 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f3cc001b660 con 0x5599639afdc0 2024-09-12T02:06:31.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.085+0000 7f3cdfce4740 10 client.16809 fetch_fsmap learned FSMap version 25 2024-09-12T02:06:31.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.085+0000 7f3cdfce4740 1 -- 172.21.15.16:0/435492379 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x559963a346c0 con 0x5599639afdc0 2024-09-12T02:06:31.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.085+0000 7f3cb6ffd640 1 -- 172.21.15.16:0/435492379 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3cd0019040 con 0x5599639afdc0 2024-09-12T02:06:31.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.085+0000 7f3cb6ffd640 1 -- 172.21.15.16:0/435492379 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f3cc0024050 con 0x5599639afdc0 2024-09-12T02:06:31.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.085+0000 7f3cdfce4740 10 client.16809 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:06:31.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.085+0000 7f3cdfce4740 10 client.16809 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:06:31.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.085+0000 7f3cdfce4740 20 client.16809 populate_metadata read hostname 'smithi016' 2024-09-12T02:06:31.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.085+0000 7f3cdfce4740 1 --2- 172.21.15.16:0/435492379 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5599639ca040 0x559963a36e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:31.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.085+0000 7f3cdfce4740 4 client.16809 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T02:06:31.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.085+0000 7f3cdfce4740 1 -- 172.21.15.16:0/435492379 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55996391cd60 con 0x5599639ca040 2024-09-12T02:06:31.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.086+0000 7f3cd6ffd640 1 --2- 172.21.15.16:0/435492379 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5599639ca040 0x559963a36e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:31.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.086+0000 7f3cd6ffd640 1 --2- 172.21.15.16:0/435492379 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5599639ca040 0x559963a36e50 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad137e6b01560aa7 server_cookie=1ce5e62b7f3d2a5e in_seq=0 out_seq=0 2024-09-12T02:06:31.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.086+0000 7f3cb6ffd640 10 client.16809 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:06:31.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.086+0000 7f3cb6ffd640 1 -- 172.21.15.16:0/435492379 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55996391cd60 con 0x5599639ca040 2024-09-12T02:06:31.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.086+0000 7f3cb6ffd640 10 client.16809 handle_command_reply: tid=0 2024-09-12T02:06:31.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.086+0000 7f3cdfce4740 1 client.16809 shutdown 2024-09-12T02:06:31.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.086+0000 7f3cdfce4740 1 -- 172.21.15.16:0/435492379 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5599639afdc0 msgr2=0x5599639c2d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:31.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.086+0000 7f3cdfce4740 1 --2- 172.21.15.16:0/435492379 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5599639afdc0 0x5599639c2d70 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x5599639b8840 tx=0x7f3cc0005900 comp rx=0 tx=0).stop 2024-09-12T02:06:31.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.086+0000 7f3cdfce4740 1 -- 172.21.15.16:0/435492379 shutdown_connections 2024-09-12T02:06:31.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.086+0000 7f3cdfce4740 1 --2- 172.21.15.16:0/435492379 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5599639ca040 0x559963a36e50 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:31.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.087+0000 7f3cdfce4740 1 --2- 172.21.15.16:0/435492379 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5599639ba680 0x5599639b3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:31.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.087+0000 7f3cdfce4740 1 --2- 172.21.15.16:0/435492379 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5599639afdc0 0x5599639c2d70 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:31.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.087+0000 7f3cdfce4740 1 --2- 172.21.15.16:0/435492379 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599639af4a0 0x5599639c2830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:31.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.087+0000 7f3cb6ffd640 0 client.16809 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:06:31.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.087+0000 7f3cdfce4740 1 -- 172.21.15.16:0/435492379 >> 172.21.15.16:0/435492379 conn(0x5599639aba30 msgr2=0x5599639b5ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:06:31.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.087+0000 7f3cdfce4740 1 -- 172.21.15.16:0/435492379 shutdown_connections 2024-09-12T02:06:31.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.087+0000 7f3cdfce4740 1 -- 172.21.15.16:0/435492379 wait complete. 2024-09-12T02:06:31.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.087+0000 7f3cdfce4740 20 client.16809 trim_cache size 0 max 16384 2024-09-12T02:06:31.093 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T02:06:31.093 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-09-12T02:06:31.093 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {} 2024-09-12T02:06:31.093 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T02:06:31.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.087+0000 7f3cdd90b640 1 -- 172.21.15.16:0/2330204970 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f3cc40a2b80 msgr2=0x7f3cc40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:31.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.087+0000 7f3cdd90b640 1 --2- 172.21.15.16:0/2330204970 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f3cc40a2b80 0x7f3cc40a5040 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x5599639acdb0 tx=0x7f3cc000b390 comp rx=0 tx=0).stop 2024-09-12T02:06:31.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.087+0000 7f3cdd90b640 1 -- 172.21.15.16:0/2330204970 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3cd8102d30 msgr2=0x7f3cd8100990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:31.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.087+0000 7f3cdd90b640 1 --2- 172.21.15.16:0/2330204970 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3cd8102d30 0x7f3cd8100990 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f3cc8009230 tx=0x7f3cc8004270 comp rx=0 tx=0).stop 2024-09-12T02:06:31.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.087+0000 7f3cdd90b640 1 -- 172.21.15.16:0/2330204970 shutdown_connections 2024-09-12T02:06:31.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.087+0000 7f3cdd90b640 1 --2- 172.21.15.16:0/2330204970 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f3cc40a2b80 0x7f3cc40a5040 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:31.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.087+0000 7f3cdd90b640 1 --2- 172.21.15.16:0/2330204970 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3cd81082a0 0x7f3cd8101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:31.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.087+0000 7f3cdd90b640 1 --2- 172.21.15.16:0/2330204970 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3cd81036e0 0x7f3cd8100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:31.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.087+0000 7f3cdd90b640 1 --2- 172.21.15.16:0/2330204970 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3cd8102d30 0x7f3cd8100990 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:31.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.087+0000 7f3cdd90b640 1 -- 172.21.15.16:0/2330204970 >> 172.21.15.16:0/2330204970 conn(0x7f3cd80fc7d0 msgr2=0x7f3cd8105d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:06:31.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.088+0000 7f3cdd90b640 1 -- 172.21.15.16:0/2330204970 shutdown_connections 2024-09-12T02:06:31.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:31.088+0000 7f3cdd90b640 1 -- 172.21.15.16:0/2330204970 wait complete. 2024-09-12T02:06:31.911 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-12T02:06:31.912 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:06:32.173 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:06:32.183 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:06:32 smithi016 ceph-mon[30947]: pgmap v473: 97 pgs: 97 active+clean; 2.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 15 op/s 2024-09-12T02:06:32.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:06:32 smithi184 ceph-mon[35593]: pgmap v473: 97 pgs: 97 active+clean; 2.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 15 op/s 2024-09-12T02:06:32.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:06:32 smithi033 ceph-mon[35332]: pgmap v473: 97 pgs: 97 active+clean; 2.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 15 op/s 2024-09-12T02:06:32.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:32.644+0000 7f76ac03b640 1 -- 172.21.15.16:0/2851233938 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76a41082a0 msgr2=0x7f76a410ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:32.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:32.644+0000 7f76ac03b640 1 --2- 172.21.15.16:0/2851233938 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76a41082a0 0x7f76a410ec40 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f769c009930 tx=0x7f769c02a420 comp rx=0 tx=0).stop 2024-09-12T02:06:32.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:32.644+0000 7f76ac03b640 1 -- 172.21.15.16:0/2851233938 shutdown_connections 2024-09-12T02:06:32.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:32.644+0000 7f76ac03b640 1 --2- 172.21.15.16:0/2851233938 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76a41082a0 0x7f76a410ec40 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:32.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:32.644+0000 7f76ac03b640 1 --2- 172.21.15.16:0/2851233938 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76a41036e0 0x7f76a4107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:32.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:32.644+0000 7f76ac03b640 1 --2- 172.21.15.16:0/2851233938 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f76a4102d30 0x7f76a4103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:32.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:32.644+0000 7f76ac03b640 1 -- 172.21.15.16:0/2851233938 >> 172.21.15.16:0/2851233938 conn(0x7f76a40fc7d0 msgr2=0x7f76a40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:06:32.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:32.644+0000 7f76ac03b640 1 -- 172.21.15.16:0/2851233938 shutdown_connections 2024-09-12T02:06:32.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:32.644+0000 7f76ac03b640 1 -- 172.21.15.16:0/2851233938 wait complete. 2024-09-12T02:06:32.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:32.645+0000 7f76ac03b640 1 Processor -- start 2024-09-12T02:06:32.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:32.645+0000 7f76ac03b640 1 -- start start 2024-09-12T02:06:32.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:32.645+0000 7f76ac03b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76a4102d30 0x7f76a4100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:32.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:32.645+0000 7f76ac03b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76a41036e0 0x7f76a4100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:32.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:32.645+0000 7f76ac03b640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f76a41082a0 0x7f76a4101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:32.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:32.645+0000 7f76ac03b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f76a41120a0 con 0x7f76a4102d30 2024-09-12T02:06:32.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:32.645+0000 7f76ac03b640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f76a4111f20 con 0x7f76a41036e0 2024-09-12T02:06:32.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:32.645+0000 7f76ac03b640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f76a4112220 con 0x7f76a41082a0 2024-09-12T02:06:32.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:32.645+0000 7f76a9dd7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76a4102d30 0x7f76a4100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:32.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:32.645+0000 7f76a9dd7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76a4102d30 0x7f76a4100990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56578/0 (socket says 172.21.15.16:56578) 2024-09-12T02:06:32.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:32.645+0000 7f76a9dd7640 1 -- 172.21.15.16:0/4004397195 learned_addr learned my addr 172.21.15.16:0/4004397195 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:06:32.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:32.645+0000 7f76a95d6640 1 --2- 172.21.15.16:0/4004397195 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76a41036e0 0x7f76a4100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:32.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:32.645+0000 7f76a9dd7640 1 -- 172.21.15.16:0/4004397195 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f76a41082a0 msgr2=0x7f76a4101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:32.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:32.645+0000 7f76aa5d8640 1 --2- 172.21.15.16:0/4004397195 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f76a41082a0 0x7f76a4101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:32.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:32.645+0000 7f76a9dd7640 1 --2- 172.21.15.16:0/4004397195 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f76a41082a0 0x7f76a4101410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:32.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:32.645+0000 7f76a9dd7640 1 -- 172.21.15.16:0/4004397195 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76a41036e0 msgr2=0x7f76a4100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:32.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:32.645+0000 7f76a9dd7640 1 --2- 172.21.15.16:0/4004397195 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76a41036e0 0x7f76a4100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:32.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:32.645+0000 7f76a9dd7640 1 -- 172.21.15.16:0/4004397195 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76a41a6980 con 0x7f76a4102d30 2024-09-12T02:06:32.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:32.645+0000 7f76a95d6640 1 --2- 172.21.15.16:0/4004397195 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76a41036e0 0x7f76a4100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:06:32.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:32.645+0000 7f76a9dd7640 1 --2- 172.21.15.16:0/4004397195 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76a4102d30 0x7f76a4100990 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f76a000cce0 tx=0x7f76a000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:06:32.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:32.646+0000 7f7696ffd640 1 -- 172.21.15.16:0/4004397195 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76a000f650 con 0x7f76a4102d30 2024-09-12T02:06:32.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:32.646+0000 7f7696ffd640 1 -- 172.21.15.16:0/4004397195 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76a000fcd0 con 0x7f76a4102d30 2024-09-12T02:06:32.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:32.646+0000 7f76ac03b640 1 -- 172.21.15.16:0/4004397195 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76a41a6c70 con 0x7f76a4102d30 2024-09-12T02:06:32.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:32.646+0000 7f7696ffd640 1 -- 172.21.15.16:0/4004397195 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76a0018c70 con 0x7f76a4102d30 2024-09-12T02:06:32.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:32.646+0000 7f76ac03b640 1 -- 172.21.15.16:0/4004397195 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76a41a71b0 con 0x7f76a4102d30 2024-09-12T02:06:32.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:32.647+0000 7f76ac03b640 1 -- 172.21.15.16:0/4004397195 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76a4103110 con 0x7f76a4102d30 2024-09-12T02:06:32.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:32.648+0000 7f7696ffd640 1 -- 172.21.15.16:0/4004397195 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f76a000f7f0 con 0x7f76a4102d30 2024-09-12T02:06:32.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:32.649+0000 7f7696ffd640 1 --2- 172.21.15.16:0/4004397195 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f76900a2b80 0x7f76900a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:32.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:32.649+0000 7f76a95d6640 1 --2- 172.21.15.16:0/4004397195 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f76900a2b80 0x7f76900a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:32.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:32.649+0000 7f7696ffd640 1 -- 172.21.15.16:0/4004397195 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f76a00cf850 con 0x7f76a4102d30 2024-09-12T02:06:32.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:32.649+0000 7f76a95d6640 1 --2- 172.21.15.16:0/4004397195 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f76900a2b80 0x7f76900a5040 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f7698005fd0 tx=0x7f76980094d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:06:32.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:32.652+0000 7f7696ffd640 1 -- 172.21.15.16:0/4004397195 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f76a0014030 con 0x7f76a4102d30 2024-09-12T02:06:32.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:32.862+0000 7f76ac03b640 1 -- 172.21.15.16:0/4004397195 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f76a4101ec0 con 0x7f76a4102d30 2024-09-12T02:06:32.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:32.863+0000 7f7696ffd640 1 -- 172.21.15.16:0/4004397195 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11911 (secure 0 0 0) 0x7f76a009c840 con 0x7f76a4102d30 2024-09-12T02:06:32.865 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T02:06:32.865 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":25,"btime":"2024-09-12T02:01:37:192021+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3279123279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3279123279},{"type":"v1","addr":"172.21.15.16:6835","nonce":3279123279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4119452382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4119452382},{"type":"v1","addr":"172.21.15.16:6837","nonce":4119452382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/3263823165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3263823165},{"type":"v1","addr":"172.21.15.16:6839","nonce":3263823165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/1510290732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":1510290732},{"type":"v1","addr":"172.21.15.16:6841","nonce":1510290732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/279674603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":279674603},{"type":"v1","addr":"172.21.15.33:6833","nonce":279674603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3555728028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3555728028},{"type":"v1","addr":"172.21.15.33:6835","nonce":3555728028}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T01:59:56.117608+0000","modified":"2024-09-12T02:01:36.294760+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24482,"mds_1":24481,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6833/1563374106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1563374106},{"type":"v1","addr":"172.21.15.184:6833","nonce":1563374106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6837/1892833868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1892833868},{"type":"v1","addr":"172.21.15.33:6837","nonce":1892833868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/1457679686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1457679686},{"type":"v1","addr":"172.21.15.184:6835","nonce":1457679686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.184:6837/1793393087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":1793393087},{"type":"v1","addr":"172.21.15.184:6837","nonce":1793393087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.184:6839/4203347941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":4203347941},{"type":"v1","addr":"172.21.15.184:6839","nonce":4203347941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.33:6839/4136670204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":4136670204},{"type":"v1","addr":"172.21.15.33:6839","nonce":4136670204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24481,24482,24469]},"id":1}]} 2024-09-12T02:06:32.866 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 25 2024-09-12T02:06:32.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:32.865+0000 7f76ac03b640 1 -- 172.21.15.16:0/4004397195 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f76900a2b80 msgr2=0x7f76900a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:32.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:32.865+0000 7f76ac03b640 1 --2- 172.21.15.16:0/4004397195 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f76900a2b80 0x7f76900a5040 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f7698005fd0 tx=0x7f76980094d0 comp rx=0 tx=0).stop 2024-09-12T02:06:32.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:32.865+0000 7f76ac03b640 1 -- 172.21.15.16:0/4004397195 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76a4102d30 msgr2=0x7f76a4100990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:32.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:32.865+0000 7f76ac03b640 1 --2- 172.21.15.16:0/4004397195 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76a4102d30 0x7f76a4100990 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f76a000cce0 tx=0x7f76a000a620 comp rx=0 tx=0).stop 2024-09-12T02:06:32.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:32.865+0000 7f76ac03b640 1 -- 172.21.15.16:0/4004397195 shutdown_connections 2024-09-12T02:06:32.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:32.865+0000 7f76ac03b640 1 --2- 172.21.15.16:0/4004397195 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f76900a2b80 0x7f76900a5040 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:32.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:32.865+0000 7f76ac03b640 1 --2- 172.21.15.16:0/4004397195 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f76a41082a0 0x7f76a4101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:32.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:32.865+0000 7f76ac03b640 1 --2- 172.21.15.16:0/4004397195 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76a41036e0 0x7f76a4100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:32.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:32.865+0000 7f76ac03b640 1 --2- 172.21.15.16:0/4004397195 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76a4102d30 0x7f76a4100990 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:32.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:32.865+0000 7f76ac03b640 1 -- 172.21.15.16:0/4004397195 >> 172.21.15.16:0/4004397195 conn(0x7f76a40fc7d0 msgr2=0x7f76a40fe3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:06:32.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:32.866+0000 7f76ac03b640 1 -- 172.21.15.16:0/4004397195 shutdown_connections 2024-09-12T02:06:32.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:32.866+0000 7f76ac03b640 1 -- 172.21.15.16:0/4004397195 wait complete. 2024-09-12T02:06:33.183 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:06:33 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/4004397195' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:06:33.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:06:33 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/4004397195' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:06:33.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:06:33 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/4004397195' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:06:33.615 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-12T02:06:33.665 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.663+0000 7fb487586640 1 -- 172.21.15.16:0/1307376744 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb480101c30 msgr2=0x7fb4801020b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:33.665 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.663+0000 7fb487586640 1 --2- 172.21.15.16:0/1307376744 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb480101c30 0x7fb4801020b0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fb470009900 tx=0x7fb47002a410 comp rx=0 tx=0).stop 2024-09-12T02:06:33.665 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.663+0000 7fb487586640 1 -- 172.21.15.16:0/1307376744 shutdown_connections 2024-09-12T02:06:33.665 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.663+0000 7fb487586640 1 --2- 172.21.15.16:0/1307376744 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb4801025f0 0x7fb480108e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:33.666 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.663+0000 7fb487586640 1 --2- 172.21.15.16:0/1307376744 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb480101c30 0x7fb4801020b0 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:33.666 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.663+0000 7fb487586640 1 --2- 172.21.15.16:0/1307376744 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb480100a40 0x7fb480100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:33.666 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.663+0000 7fb487586640 1 -- 172.21.15.16:0/1307376744 >> 172.21.15.16:0/1307376744 conn(0x7fb4800fc1e0 msgr2=0x7fb4800fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:06:33.666 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.663+0000 7fb487586640 1 -- 172.21.15.16:0/1307376744 shutdown_connections 2024-09-12T02:06:33.666 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.663+0000 7fb487586640 1 -- 172.21.15.16:0/1307376744 wait complete. 2024-09-12T02:06:33.666 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.664+0000 7fb487586640 1 Processor -- start 2024-09-12T02:06:33.666 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.664+0000 7fb487586640 1 -- start start 2024-09-12T02:06:33.666 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.664+0000 7fb487586640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb480100a40 0x7fb480197dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:33.666 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.664+0000 7fb487586640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb480101c30 0x7fb480198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:33.666 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.664+0000 7fb487586640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb4801025f0 0x7fb48019f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:33.666 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.664+0000 7fb487586640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb48010d810 con 0x7fb480101c30 2024-09-12T02:06:33.666 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.664+0000 7fb487586640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb48010d690 con 0x7fb480100a40 2024-09-12T02:06:33.666 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.664+0000 7fb487586640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb48010d990 con 0x7fb4801025f0 2024-09-12T02:06:33.666 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.664+0000 7fb484b21640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb480101c30 0x7fb480198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:33.667 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.664+0000 7fb484b21640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb480101c30 0x7fb480198310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56610/0 (socket says 172.21.15.16:56610) 2024-09-12T02:06:33.667 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.664+0000 7fb484b21640 1 -- 172.21.15.16:0/1782678600 learned_addr learned my addr 172.21.15.16:0/1782678600 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:06:33.667 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.664+0000 7fb485b23640 1 --2- 172.21.15.16:0/1782678600 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb4801025f0 0x7fb48019f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:33.667 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.664+0000 7fb485322640 1 --2- 172.21.15.16:0/1782678600 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb480100a40 0x7fb480197dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:33.667 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.665+0000 7fb484b21640 1 -- 172.21.15.16:0/1782678600 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb4801025f0 msgr2=0x7fb48019f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:33.667 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.665+0000 7fb484b21640 1 --2- 172.21.15.16:0/1782678600 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb4801025f0 0x7fb48019f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:33.667 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.665+0000 7fb484b21640 1 -- 172.21.15.16:0/1782678600 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb480100a40 msgr2=0x7fb480197dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:33.667 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.665+0000 7fb484b21640 1 --2- 172.21.15.16:0/1782678600 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb480100a40 0x7fb480197dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:33.667 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.665+0000 7fb484b21640 1 -- 172.21.15.16:0/1782678600 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb48019f9f0 con 0x7fb480101c30 2024-09-12T02:06:33.667 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.665+0000 7fb485322640 1 --2- 172.21.15.16:0/1782678600 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb480100a40 0x7fb480197dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:06:33.667 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.665+0000 7fb484b21640 1 --2- 172.21.15.16:0/1782678600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb480101c30 0x7fb480198310 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fb470009230 tx=0x7fb470031b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:06:33.667 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.665+0000 7fb4767fc640 1 -- 172.21.15.16:0/1782678600 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb47003f030 con 0x7fb480101c30 2024-09-12T02:06:33.667 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.665+0000 7fb4767fc640 1 -- 172.21.15.16:0/1782678600 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb470003ad0 con 0x7fb480101c30 2024-09-12T02:06:33.667 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.665+0000 7fb487586640 1 -- 172.21.15.16:0/1782678600 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb48019fc80 con 0x7fb480101c30 2024-09-12T02:06:33.667 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.665+0000 7fb4767fc640 1 -- 172.21.15.16:0/1782678600 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb470035070 con 0x7fb480101c30 2024-09-12T02:06:33.668 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.665+0000 7fb487586640 1 -- 172.21.15.16:0/1782678600 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4801a01c0 con 0x7fb480101c30 2024-09-12T02:06:33.673 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.667+0000 7fb4767fc640 1 -- 172.21.15.16:0/1782678600 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fb470042030 con 0x7fb480101c30 2024-09-12T02:06:33.673 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.668+0000 7fb48a3f8740 1 Processor -- start 2024-09-12T02:06:33.673 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.668+0000 7fb48a3f8740 1 -- start start 2024-09-12T02:06:33.673 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.668+0000 7fb48a3f8740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5566f8cb8db0 0x5566f8cb91b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:33.673 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.668+0000 7fb48a3f8740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5566f8cb96f0 0x5566f8cbe1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:33.673 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.668+0000 7fb48a3f8740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5566f8cbe780 0x5566f8cc0c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:33.674 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.668+0000 7fb48a3f8740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5566f8c182f0 con 0x5566f8cb8db0 2024-09-12T02:06:33.674 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.668+0000 7fb48a3f8740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5566f8bc0270 con 0x5566f8cb96f0 2024-09-12T02:06:33.674 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.668+0000 7fb48a3f8740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5566f8c142b0 con 0x5566f8cbe780 2024-09-12T02:06:33.674 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.668+0000 7fb485b23640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5566f8cb8db0 0x5566f8cb91b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:33.674 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.668+0000 7fb485b23640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5566f8cb8db0 0x5566f8cb91b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56618/0 (socket says 172.21.15.16:56618) 2024-09-12T02:06:33.674 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.668+0000 7fb485b23640 1 -- 172.21.15.16:0/1656544837 learned_addr learned my addr 172.21.15.16:0/1656544837 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:06:33.674 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.668+0000 7fb4767fc640 1 --2- 172.21.15.16:0/1782678600 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fb46c0a2cb0 0x7fb46c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:33.674 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.668+0000 7fb484b21640 1 --2- 172.21.15.16:0/1656544837 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5566f8cbe780 0x5566f8cc0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:33.674 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.668+0000 7fb485322640 1 --2- 172.21.15.16:0/1656544837 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5566f8cb96f0 0x5566f8cbe1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:33.674 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.669+0000 7fb485b23640 1 --2- 172.21.15.16:0/1782678600 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fb46c0a2cb0 0x7fb46c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:33.674 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.669+0000 7fb4767fc640 1 -- 172.21.15.16:0/1782678600 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fb4700ef4e0 con 0x7fb480101c30 2024-09-12T02:06:33.674 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.669+0000 7fb485b23640 1 -- 172.21.15.16:0/1656544837 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5566f8cbe780 msgr2=0x5566f8cc0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:06:33.674 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.669+0000 7fb485b23640 1 --2- 172.21.15.16:0/1656544837 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5566f8cbe780 0x5566f8cc0c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:33.675 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.669+0000 7fb485b23640 1 -- 172.21.15.16:0/1656544837 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5566f8cb96f0 msgr2=0x5566f8cbe1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:06:33.675 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.669+0000 7fb485b23640 1 --2- 172.21.15.16:0/1656544837 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5566f8cb96f0 0x5566f8cbe1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:33.675 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.669+0000 7fb485b23640 1 -- 172.21.15.16:0/1656544837 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566f8bbf340 con 0x5566f8cb8db0 2024-09-12T02:06:33.675 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.669+0000 7fb484b21640 1 --2- 172.21.15.16:0/1656544837 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5566f8cbe780 0x5566f8cc0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:06:33.675 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.669+0000 7fb485b23640 1 --2- 172.21.15.16:0/1656544837 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5566f8cb8db0 0x5566f8cb91b0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fb478004140 tx=0x7fb478003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c804cda4c6a1fcb5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:06:33.675 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.669+0000 7fb46b7fe640 1 -- 172.21.15.16:0/1656544837 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb478012e60 con 0x5566f8cb8db0 2024-09-12T02:06:33.675 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.669+0000 7fb46b7fe640 1 -- 172.21.15.16:0/1656544837 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb478023070 con 0x5566f8cb8db0 2024-09-12T02:06:33.675 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.669+0000 7fb46b7fe640 1 -- 172.21.15.16:0/1656544837 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb478003ad0 con 0x5566f8cb8db0 2024-09-12T02:06:33.675 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.669+0000 7fb485b23640 1 --2- 172.21.15.16:0/1782678600 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fb46c0a2cb0 0x7fb46c0a5170 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fb478022000 tx=0x7fb478020000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:06:33.675 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.670+0000 7fb48a3f8740 1 -- 172.21.15.16:0/1656544837 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5566f8cb8db0 msgr2=0x5566f8cb91b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:33.675 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.670+0000 7fb48a3f8740 1 --2- 172.21.15.16:0/1656544837 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5566f8cb8db0 0x5566f8cb91b0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fb478004140 tx=0x7fb478003f90 comp rx=0 tx=0).stop 2024-09-12T02:06:33.675 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.670+0000 7fb48a3f8740 1 -- 172.21.15.16:0/1656544837 shutdown_connections 2024-09-12T02:06:33.675 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.670+0000 7fb48a3f8740 1 --2- 172.21.15.16:0/1656544837 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5566f8cbe780 0x5566f8cc0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:33.675 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.670+0000 7fb48a3f8740 1 --2- 172.21.15.16:0/1656544837 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5566f8cb96f0 0x5566f8cbe1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:33.676 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.670+0000 7fb48a3f8740 1 --2- 172.21.15.16:0/1656544837 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5566f8cb8db0 0x5566f8cb91b0 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:33.676 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.670+0000 7fb48a3f8740 1 -- 172.21.15.16:0/1656544837 >> 172.21.15.16:0/1656544837 conn(0x5566f8cb0010 msgr2=0x5566f8cb0410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:06:33.676 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.670+0000 7fb48a3f8740 1 -- 172.21.15.16:0/1656544837 shutdown_connections 2024-09-12T02:06:33.676 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.670+0000 7fb48a3f8740 1 -- 172.21.15.16:0/1656544837 wait complete. 2024-09-12T02:06:33.676 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.670+0000 7fb48a3f8740 1 -- start start 2024-09-12T02:06:33.676 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.670+0000 7fb48a3f8740 1 Processor -- start 2024-09-12T02:06:33.676 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.671+0000 7fb48a3f8740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5566f8cb8db0 0x5566f8cc2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:33.676 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.671+0000 7fb48a3f8740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5566f8cb96f0 0x5566f8cc3510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:33.676 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.671+0000 7fb48a3f8740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5566f8cbe780 0x5566f8cca550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:33.676 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.671+0000 7fb48a3f8740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5566f8cc1750 con 0x5566f8cb8db0 2024-09-12T02:06:33.676 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.671+0000 7fb48a3f8740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5566f8ba1290 con 0x5566f8cbe780 2024-09-12T02:06:33.676 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.671+0000 7fb48a3f8740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5566f8cc18d0 con 0x5566f8cb96f0 2024-09-12T02:06:33.676 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.671+0000 7fb48a3f8740 10 client.0 _ll_register_callbacks cb 0x5566f8bd2140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:06:33.676 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.671+0000 7fb485322640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5566f8cb8db0 0x5566f8cc2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:33.677 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.671+0000 7fb485322640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5566f8cb8db0 0x5566f8cc2fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56626/0 (socket says 172.21.15.16:56626) 2024-09-12T02:06:33.677 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.671+0000 7fb485322640 1 -- 172.21.15.16:0/3775860095 learned_addr learned my addr 172.21.15.16:0/3775860095 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:06:33.677 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.671+0000 7fb484b21640 1 --2- 172.21.15.16:0/3775860095 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5566f8cb96f0 0x5566f8cc3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:33.677 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.671+0000 7fb485b23640 1 --2- 172.21.15.16:0/3775860095 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5566f8cbe780 0x5566f8cca550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:33.677 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.671+0000 7fb485322640 1 -- 172.21.15.16:0/3775860095 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5566f8cb96f0 msgr2=0x5566f8cc3510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:06:33.677 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.671+0000 7fb485322640 1 --2- 172.21.15.16:0/3775860095 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5566f8cb96f0 0x5566f8cc3510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:33.677 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.671+0000 7fb485322640 1 -- 172.21.15.16:0/3775860095 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5566f8cbe780 msgr2=0x5566f8cca550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:06:33.677 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.671+0000 7fb485322640 1 --2- 172.21.15.16:0/3775860095 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5566f8cbe780 0x5566f8cca550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:33.677 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.671+0000 7fb485322640 1 -- 172.21.15.16:0/3775860095 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566f892b210 con 0x5566f8cb8db0 2024-09-12T02:06:33.677 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.671+0000 7fb484b21640 1 --2- 172.21.15.16:0/3775860095 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5566f8cb96f0 0x5566f8cc3510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:06:33.677 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.671+0000 7fb485b23640 1 --2- 172.21.15.16:0/3775860095 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5566f8cbe780 0x5566f8cca550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:06:33.677 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.672+0000 7fb485322640 1 --2- 172.21.15.16:0/3775860095 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5566f8cb8db0 0x5566f8cc2fd0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fb47c009c40 tx=0x7fb47c0039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92c5169b478c7d8b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:06:33.677 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.672+0000 7fb468ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:06:33.677 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.672+0000 7fb468ff9640 1 -- 172.21.15.16:0/3775860095 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb47c005220 con 0x5566f8cb8db0 2024-09-12T02:06:33.678 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.672+0000 7fb468ff9640 1 -- 172.21.15.16:0/3775860095 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fb47c0053c0 con 0x5566f8cb8db0 2024-09-12T02:06:33.678 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.672+0000 7fb48a3f8740 1 -- 172.21.15.16:0/3775860095 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb478023070 con 0x5566f8cb8db0 2024-09-12T02:06:33.678 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.672+0000 7fb468ff9640 1 -- 172.21.15.16:0/3775860095 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fb47c0055c0 con 0x5566f8cb8db0 2024-09-12T02:06:33.678 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.672+0000 7fb48a3f8740 10 client.16827 fetch_fsmap learned FSMap version 25 2024-09-12T02:06:33.678 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.672+0000 7fb48a3f8740 1 -- 172.21.15.16:0/3775860095 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5566f89f7620 con 0x5566f8cb8db0 2024-09-12T02:06:33.678 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.672+0000 7fb468ff9640 1 -- 172.21.15.16:0/3775860095 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb478023070 con 0x5566f8cb8db0 2024-09-12T02:06:33.678 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.672+0000 7fb468ff9640 1 -- 172.21.15.16:0/3775860095 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fb47c01a030 con 0x5566f8cb8db0 2024-09-12T02:06:33.678 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.672+0000 7fb48a3f8740 10 client.16827 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:06:33.678 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.672+0000 7fb48a3f8740 10 client.16827 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:06:33.678 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.672+0000 7fb48a3f8740 20 client.16827 populate_metadata read hostname 'smithi016' 2024-09-12T02:06:33.678 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.673+0000 7fb48a3f8740 1 --2- 172.21.15.16:0/3775860095 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5566f8cceca0 0x5566f8cd1130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:33.678 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.673+0000 7fb48a3f8740 4 client.16827 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:06:33.678 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.673+0000 7fb48a3f8740 1 -- 172.21.15.16:0/3775860095 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5566f8c25a20 con 0x5566f8cceca0 2024-09-12T02:06:33.678 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.673+0000 7fb484b21640 1 --2- 172.21.15.16:0/3775860095 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5566f8cceca0 0x5566f8cd1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:33.678 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.673+0000 7fb484b21640 1 --2- 172.21.15.16:0/3775860095 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5566f8cceca0 0x5566f8cd1130 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f82b247850260507 server_cookie=1219b20ee18b798e in_seq=0 out_seq=0 2024-09-12T02:06:33.679 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.673+0000 7fb468ff9640 10 client.16827 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:06:33.680 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.674+0000 7fb468ff9640 1 -- 172.21.15.16:0/3775860095 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5566f8c25a20 con 0x5566f8cceca0 2024-09-12T02:06:33.680 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.674+0000 7fb468ff9640 10 client.16827 handle_command_reply: tid=0 2024-09-12T02:06:33.680 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.674+0000 7fb48a3f8740 1 client.16827 shutdown 2024-09-12T02:06:33.680 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.674+0000 7fb48a3f8740 1 -- 172.21.15.16:0/3775860095 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5566f8cb8db0 msgr2=0x5566f8cc2fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:33.680 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.674+0000 7fb48a3f8740 1 --2- 172.21.15.16:0/3775860095 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5566f8cb8db0 0x5566f8cc2fd0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fb47c009c40 tx=0x7fb47c0039c0 comp rx=0 tx=0).stop 2024-09-12T02:06:33.680 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.674+0000 7fb48a3f8740 1 -- 172.21.15.16:0/3775860095 shutdown_connections 2024-09-12T02:06:33.680 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.674+0000 7fb48a3f8740 1 --2- 172.21.15.16:0/3775860095 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5566f8cceca0 0x5566f8cd1130 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:33.680 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.674+0000 7fb48a3f8740 1 --2- 172.21.15.16:0/3775860095 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5566f8cbe780 0x5566f8cca550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:33.680 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.674+0000 7fb48a3f8740 1 --2- 172.21.15.16:0/3775860095 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5566f8cb96f0 0x5566f8cc3510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:33.680 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.674+0000 7fb48a3f8740 1 --2- 172.21.15.16:0/3775860095 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5566f8cb8db0 0x5566f8cc2fd0 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:33.680 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.674+0000 7fb48a3f8740 1 -- 172.21.15.16:0/3775860095 >> 172.21.15.16:0/3775860095 conn(0x5566f8cb0010 msgr2=0x5566f8cb1820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:06:33.680 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.674+0000 7fb468ff9640 0 client.16827 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:06:33.681 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.674+0000 7fb48a3f8740 1 -- 172.21.15.16:0/3775860095 shutdown_connections 2024-09-12T02:06:33.681 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.675+0000 7fb48a3f8740 1 -- 172.21.15.16:0/3775860095 wait complete. 2024-09-12T02:06:33.681 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.675+0000 7fb48a3f8740 20 client.16827 trim_cache size 0 max 16384 2024-09-12T02:06:33.694 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.689+0000 7fb48a3f8740 1 Processor -- start 2024-09-12T02:06:33.694 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.689+0000 7fb48a3f8740 1 -- start start 2024-09-12T02:06:33.694 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.689+0000 7fb48a3f8740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5566f8cb8db0 0x5566f8cc35a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:33.694 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.689+0000 7fb48a3f8740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5566f8cb96f0 0x5566f8cca510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:33.694 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.689+0000 7fb48a3f8740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5566f8cbe780 0x5566f8caf0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:33.694 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.689+0000 7fb48a3f8740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb4801a9460 con 0x5566f8cbe780 2024-09-12T02:06:33.694 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.689+0000 7fb48a3f8740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb4801a6d30 con 0x5566f8cb96f0 2024-09-12T02:06:33.694 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.689+0000 7fb48a3f8740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb4801a4b80 con 0x5566f8cb8db0 2024-09-12T02:06:33.694 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.690+0000 7fb485b23640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5566f8cbe780 0x5566f8caf0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:33.695 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.690+0000 7fb485b23640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5566f8cbe780 0x5566f8caf0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56642/0 (socket says 172.21.15.16:56642) 2024-09-12T02:06:33.695 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.690+0000 7fb485b23640 1 -- 172.21.15.16:0/2816991613 learned_addr learned my addr 172.21.15.16:0/2816991613 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:06:33.695 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.690+0000 7fb484b21640 1 --2- 172.21.15.16:0/2816991613 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5566f8cb96f0 0x5566f8cca510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:33.695 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.690+0000 7fb485322640 1 --2- 172.21.15.16:0/2816991613 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5566f8cb8db0 0x5566f8cc35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:33.695 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.690+0000 7fb484b21640 1 -- 172.21.15.16:0/2816991613 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5566f8cb8db0 msgr2=0x5566f8cc35a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:06:33.695 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.690+0000 7fb484b21640 1 --2- 172.21.15.16:0/2816991613 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5566f8cb8db0 0x5566f8cc35a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:33.695 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.690+0000 7fb484b21640 1 -- 172.21.15.16:0/2816991613 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5566f8cbe780 msgr2=0x5566f8caf0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:06:33.695 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.690+0000 7fb484b21640 1 --2- 172.21.15.16:0/2816991613 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5566f8cbe780 0x5566f8caf0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:33.695 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.690+0000 7fb484b21640 1 -- 172.21.15.16:0/2816991613 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566f8cb1ea0 con 0x5566f8cb96f0 2024-09-12T02:06:33.695 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.690+0000 7fb485b23640 1 --2- 172.21.15.16:0/2816991613 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5566f8cbe780 0x5566f8caf0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:06:33.695 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.690+0000 7fb484b21640 1 --2- 172.21.15.16:0/2816991613 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5566f8cb96f0 0x5566f8cca510 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb4700ed0d0 tx=0x7fb470086f00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7812889b3144d2dc server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:06:33.695 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.690+0000 7fb46affd640 1 -- 172.21.15.16:0/2816991613 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb470049040 con 0x5566f8cb96f0 2024-09-12T02:06:33.695 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.690+0000 7fb46affd640 1 -- 172.21.15.16:0/2816991613 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4700c0190 con 0x5566f8cb96f0 2024-09-12T02:06:33.695 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.690+0000 7fb46affd640 1 -- 172.21.15.16:0/2816991613 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb470003c80 con 0x5566f8cb96f0 2024-09-12T02:06:33.695 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.691+0000 7fb48a3f8740 1 -- 172.21.15.16:0/2816991613 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5566f8cb96f0 msgr2=0x5566f8cca510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:33.696 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.691+0000 7fb48a3f8740 1 --2- 172.21.15.16:0/2816991613 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5566f8cb96f0 0x5566f8cca510 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb4700ed0d0 tx=0x7fb470086f00 comp rx=0 tx=0).stop 2024-09-12T02:06:33.696 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.691+0000 7fb48a3f8740 1 -- 172.21.15.16:0/2816991613 shutdown_connections 2024-09-12T02:06:33.696 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.691+0000 7fb48a3f8740 1 --2- 172.21.15.16:0/2816991613 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5566f8cbe780 0x5566f8caf0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:33.696 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.691+0000 7fb48a3f8740 1 --2- 172.21.15.16:0/2816991613 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5566f8cb96f0 0x5566f8cca510 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:33.696 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.691+0000 7fb48a3f8740 1 --2- 172.21.15.16:0/2816991613 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5566f8cb8db0 0x5566f8cc35a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:33.696 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.691+0000 7fb48a3f8740 1 -- 172.21.15.16:0/2816991613 >> 172.21.15.16:0/2816991613 conn(0x5566f8cb0010 msgr2=0x5566f8cae9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:06:33.696 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.691+0000 7fb48a3f8740 1 -- 172.21.15.16:0/2816991613 shutdown_connections 2024-09-12T02:06:33.696 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.691+0000 7fb48a3f8740 1 -- 172.21.15.16:0/2816991613 wait complete. 2024-09-12T02:06:33.696 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.691+0000 7fb48a3f8740 1 -- start start 2024-09-12T02:06:33.696 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.692+0000 7fb48a3f8740 1 Processor -- start 2024-09-12T02:06:33.696 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.692+0000 7fb48a3f8740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5566f8cb8db0 0x5566f8cc7e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:33.696 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.692+0000 7fb48a3f8740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5566f8cb96f0 0x5566f8cc8350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:33.696 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.692+0000 7fb48a3f8740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5566f8cbe780 0x5566f8cc9360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:33.696 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.692+0000 7fb48a3f8740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb4801a51f0 con 0x5566f8cb96f0 2024-09-12T02:06:33.697 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.692+0000 7fb48a3f8740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb4801a6660 con 0x5566f8cb8db0 2024-09-12T02:06:33.697 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.692+0000 7fb48a3f8740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb4801a58c0 con 0x5566f8cbe780 2024-09-12T02:06:33.697 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.692+0000 7fb48a3f8740 10 client.0 _ll_register_callbacks cb 0x7fb47c004120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:06:33.697 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.692+0000 7fb484b21640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5566f8cb96f0 0x5566f8cc8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:33.697 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.692+0000 7fb484b21640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5566f8cb96f0 0x5566f8cc8350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56658/0 (socket says 172.21.15.16:56658) 2024-09-12T02:06:33.697 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.692+0000 7fb484b21640 1 -- 172.21.15.16:0/1729837016 learned_addr learned my addr 172.21.15.16:0/1729837016 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:06:33.697 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.692+0000 7fb485b23640 1 --2- 172.21.15.16:0/1729837016 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5566f8cbe780 0x5566f8cc9360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:33.697 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.692+0000 7fb485322640 1 --2- 172.21.15.16:0/1729837016 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5566f8cb8db0 0x5566f8cc7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:33.697 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.692+0000 7fb484b21640 1 -- 172.21.15.16:0/1729837016 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5566f8cbe780 msgr2=0x5566f8cc9360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:06:33.697 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.692+0000 7fb484b21640 1 --2- 172.21.15.16:0/1729837016 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5566f8cbe780 0x5566f8cc9360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:33.697 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.692+0000 7fb484b21640 1 -- 172.21.15.16:0/1729837016 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5566f8cb8db0 msgr2=0x5566f8cc7e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:06:33.697 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.692+0000 7fb484b21640 1 --2- 172.21.15.16:0/1729837016 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5566f8cb8db0 0x5566f8cc7e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:33.698 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.692+0000 7fb484b21640 1 -- 172.21.15.16:0/1729837016 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566f8cc9970 con 0x5566f8cb96f0 2024-09-12T02:06:33.698 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.693+0000 7fb484b21640 1 --2- 172.21.15.16:0/1729837016 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5566f8cb96f0 0x5566f8cc8350 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fb4700bbd70 tx=0x7fb4700bc570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ccfc369450eef35 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:06:33.698 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.693+0000 7fb468ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:06:33.698 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.693+0000 7fb468ff9640 1 -- 172.21.15.16:0/1729837016 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb470049070 con 0x5566f8cb96f0 2024-09-12T02:06:33.698 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.693+0000 7fb468ff9640 1 -- 172.21.15.16:0/1729837016 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fb4700b84c0 con 0x5566f8cb96f0 2024-09-12T02:06:33.698 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.693+0000 7fb48a3f8740 1 -- 172.21.15.16:0/1729837016 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4700c0190 con 0x5566f8cb96f0 2024-09-12T02:06:33.698 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.693+0000 7fb468ff9640 1 -- 172.21.15.16:0/1729837016 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fb470003c80 con 0x5566f8cb96f0 2024-09-12T02:06:33.698 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.693+0000 7fb48a3f8740 10 client.16833 fetch_fsmap learned FSMap version 25 2024-09-12T02:06:33.698 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.693+0000 7fb48a3f8740 1 -- 172.21.15.16:0/1729837016 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5566f8d39c90 con 0x5566f8cb96f0 2024-09-12T02:06:33.698 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.693+0000 7fb468ff9640 1 -- 172.21.15.16:0/1729837016 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4700c0190 con 0x5566f8cb96f0 2024-09-12T02:06:33.698 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.693+0000 7fb468ff9640 1 -- 172.21.15.16:0/1729837016 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fb470031bd0 con 0x5566f8cb96f0 2024-09-12T02:06:33.698 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.693+0000 7fb48a3f8740 10 client.16833 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:06:33.698 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.693+0000 7fb48a3f8740 10 client.16833 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:06:33.698 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.693+0000 7fb48a3f8740 20 client.16833 populate_metadata read hostname 'smithi016' 2024-09-12T02:06:33.699 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.693+0000 7fb48a3f8740 1 --2- 172.21.15.16:0/1729837016 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5566f8d3a600 0x5566f8d3ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:33.699 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.693+0000 7fb48a3f8740 4 client.16833 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:06:33.699 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.693+0000 7fb48a3f8740 1 -- 172.21.15.16:0/1729837016 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5566f8c1fd70 con 0x5566f8d3a600 2024-09-12T02:06:33.699 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.694+0000 7fb485322640 1 --2- 172.21.15.16:0/1729837016 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5566f8d3a600 0x5566f8d3ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:33.699 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.694+0000 7fb485322640 1 --2- 172.21.15.16:0/1729837016 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5566f8d3a600 0x5566f8d3ca20 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c36731e34a5eed8a server_cookie=c22c17d6b0f4ff2a in_seq=0 out_seq=0 2024-09-12T02:06:33.699 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.694+0000 7fb468ff9640 10 client.16833 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:06:33.708 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.707+0000 7fb468ff9640 1 -- 172.21.15.16:0/1729837016 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5566f8c1fd70 con 0x5566f8d3a600 2024-09-12T02:06:33.708 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.707+0000 7fb468ff9640 10 client.16833 handle_command_reply: tid=0 2024-09-12T02:06:33.708 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.707+0000 7fb48a3f8740 1 client.16833 shutdown 2024-09-12T02:06:33.708 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.707+0000 7fb48a3f8740 1 -- 172.21.15.16:0/1729837016 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5566f8cb96f0 msgr2=0x5566f8cc8350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:33.708 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.707+0000 7fb48a3f8740 1 --2- 172.21.15.16:0/1729837016 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5566f8cb96f0 0x5566f8cc8350 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fb4700bbd70 tx=0x7fb4700bc570 comp rx=0 tx=0).stop 2024-09-12T02:06:33.708 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.707+0000 7fb48a3f8740 1 -- 172.21.15.16:0/1729837016 shutdown_connections 2024-09-12T02:06:33.708 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.707+0000 7fb48a3f8740 1 --2- 172.21.15.16:0/1729837016 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5566f8d3a600 0x5566f8d3ca20 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:33.708 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.707+0000 7fb48a3f8740 1 --2- 172.21.15.16:0/1729837016 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5566f8cbe780 0x5566f8cc9360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:33.708 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.707+0000 7fb48a3f8740 1 --2- 172.21.15.16:0/1729837016 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5566f8cb96f0 0x5566f8cc8350 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:33.708 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.707+0000 7fb468ff9640 0 client.16833 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:06:33.708 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.707+0000 7fb48a3f8740 1 --2- 172.21.15.16:0/1729837016 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5566f8cb8db0 0x5566f8cc7e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:33.709 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.707+0000 7fb48a3f8740 1 -- 172.21.15.16:0/1729837016 >> 172.21.15.16:0/1729837016 conn(0x5566f8cb0010 msgr2=0x5566f8cb15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:06:33.709 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.708+0000 7fb48a3f8740 1 -- 172.21.15.16:0/1729837016 shutdown_connections 2024-09-12T02:06:33.709 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.708+0000 7fb48a3f8740 1 -- 172.21.15.16:0/1729837016 wait complete. 2024-09-12T02:06:33.709 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.708+0000 7fb48a3f8740 20 client.16833 trim_cache size 0 max 16384 2024-09-12T02:06:33.709 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-09-12T02:06:33.709 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-09-12T02:06:33.709 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-09-12T02:06:33.709 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-09-12T02:06:33.710 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.708+0000 7fb487586640 1 -- 172.21.15.16:0/1782678600 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fb46c0a2cb0 msgr2=0x7fb46c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:33.710 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.708+0000 7fb487586640 1 --2- 172.21.15.16:0/1782678600 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fb46c0a2cb0 0x7fb46c0a5170 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fb478022000 tx=0x7fb478020000 comp rx=0 tx=0).stop 2024-09-12T02:06:33.710 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.708+0000 7fb487586640 1 -- 172.21.15.16:0/1782678600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb480101c30 msgr2=0x7fb480198310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:33.710 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.708+0000 7fb487586640 1 --2- 172.21.15.16:0/1782678600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb480101c30 0x7fb480198310 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fb470009230 tx=0x7fb470031b60 comp rx=0 tx=0).stop 2024-09-12T02:06:33.710 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.708+0000 7fb487586640 1 -- 172.21.15.16:0/1782678600 shutdown_connections 2024-09-12T02:06:33.711 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.708+0000 7fb487586640 1 --2- 172.21.15.16:0/1782678600 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fb46c0a2cb0 0x7fb46c0a5170 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:33.711 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.708+0000 7fb487586640 1 --2- 172.21.15.16:0/1782678600 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb4801025f0 0x7fb48019f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:33.711 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.708+0000 7fb487586640 1 --2- 172.21.15.16:0/1782678600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb480101c30 0x7fb480198310 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:33.711 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.708+0000 7fb487586640 1 --2- 172.21.15.16:0/1782678600 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb480100a40 0x7fb480197dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:33.711 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.708+0000 7fb487586640 1 -- 172.21.15.16:0/1782678600 >> 172.21.15.16:0/1782678600 conn(0x7fb4800fc1e0 msgr2=0x7fb480104f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:06:33.711 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.708+0000 7fb487586640 1 -- 172.21.15.16:0/1782678600 shutdown_connections 2024-09-12T02:06:33.711 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:33.708+0000 7fb487586640 1 -- 172.21.15.16:0/1782678600 wait complete. 2024-09-12T02:06:33.878 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:06:34.294 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:06:34 smithi016 ceph-mon[30947]: pgmap v474: 97 pgs: 97 active+clean; 2.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-09-12T02:06:34.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.294+0000 7ff8227ca640 1 -- 172.21.15.16:0/1417340320 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff81c102790 msgr2=0x7ff81c1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:34.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.294+0000 7ff8227ca640 1 --2- 172.21.15.16:0/1417340320 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff81c102790 0x7ff81c1067e0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7ff810009900 tx=0x7ff81002a410 comp rx=0 tx=0).stop 2024-09-12T02:06:34.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.294+0000 7ff8227ca640 1 -- 172.21.15.16:0/1417340320 shutdown_connections 2024-09-12T02:06:34.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.294+0000 7ff8227ca640 1 --2- 172.21.15.16:0/1417340320 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff81c106f10 0x7ff81c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:34.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.294+0000 7ff8227ca640 1 --2- 172.21.15.16:0/1417340320 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff81c102790 0x7ff81c1067e0 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:34.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.294+0000 7ff8227ca640 1 --2- 172.21.15.16:0/1417340320 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff81c101de0 0x7ff81c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:34.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.294+0000 7ff8227ca640 1 -- 172.21.15.16:0/1417340320 >> 172.21.15.16:0/1417340320 conn(0x7ff81c0fd8c0 msgr2=0x7ff81c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:06:34.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.294+0000 7ff8227ca640 1 -- 172.21.15.16:0/1417340320 shutdown_connections 2024-09-12T02:06:34.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.294+0000 7ff8227ca640 1 -- 172.21.15.16:0/1417340320 wait complete. 2024-09-12T02:06:34.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.295+0000 7ff8227ca640 1 Processor -- start 2024-09-12T02:06:34.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.295+0000 7ff8227ca640 1 -- start start 2024-09-12T02:06:34.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.295+0000 7ff8227ca640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff81c101de0 0x7ff81c198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:34.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.295+0000 7ff8227ca640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff81c102790 0x7ff81c198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:34.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.295+0000 7ff8227ca640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff81c106f10 0x7ff81c19cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:34.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.295+0000 7ff8227ca640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff81c111f90 con 0x7ff81c102790 2024-09-12T02:06:34.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.295+0000 7ff8227ca640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff81c111e10 con 0x7ff81c106f10 2024-09-12T02:06:34.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.295+0000 7ff8227ca640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff81c112110 con 0x7ff81c101de0 2024-09-12T02:06:34.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.295+0000 7ff81b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff81c102790 0x7ff81c198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:34.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.295+0000 7ff81b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff81c102790 0x7ff81c198a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56670/0 (socket says 172.21.15.16:56670) 2024-09-12T02:06:34.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.295+0000 7ff81b7fe640 1 -- 172.21.15.16:0/3355822181 learned_addr learned my addr 172.21.15.16:0/3355822181 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:06:34.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.295+0000 7ff820d67640 1 --2- 172.21.15.16:0/3355822181 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff81c106f10 0x7ff81c19cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:34.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.295+0000 7ff81bfff640 1 --2- 172.21.15.16:0/3355822181 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff81c101de0 0x7ff81c198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:34.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.296+0000 7ff81b7fe640 1 -- 172.21.15.16:0/3355822181 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff81c101de0 msgr2=0x7ff81c198510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:34.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.296+0000 7ff81b7fe640 1 --2- 172.21.15.16:0/3355822181 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff81c101de0 0x7ff81c198510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:34.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.296+0000 7ff81b7fe640 1 -- 172.21.15.16:0/3355822181 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff81c106f10 msgr2=0x7ff81c19cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:34.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.296+0000 7ff81b7fe640 1 --2- 172.21.15.16:0/3355822181 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff81c106f10 0x7ff81c19cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:34.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.296+0000 7ff81b7fe640 1 -- 172.21.15.16:0/3355822181 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff81c19d4c0 con 0x7ff81c102790 2024-09-12T02:06:34.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.296+0000 7ff81b7fe640 1 --2- 172.21.15.16:0/3355822181 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff81c102790 0x7ff81c198a50 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7ff810005ab0 tx=0x7ff8100042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:06:34.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.296+0000 7ff8197fa640 1 -- 172.21.15.16:0/3355822181 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff81003f030 con 0x7ff81c102790 2024-09-12T02:06:34.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.296+0000 7ff8197fa640 1 -- 172.21.15.16:0/3355822181 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff810003ce0 con 0x7ff81c102790 2024-09-12T02:06:34.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.296+0000 7ff8227ca640 1 -- 172.21.15.16:0/3355822181 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff81c19d750 con 0x7ff81c102790 2024-09-12T02:06:34.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.296+0000 7ff8197fa640 1 -- 172.21.15.16:0/3355822181 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff810035070 con 0x7ff81c102790 2024-09-12T02:06:34.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.296+0000 7ff8227ca640 1 -- 172.21.15.16:0/3355822181 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff81c1a5090 con 0x7ff81c102790 2024-09-12T02:06:34.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.299+0000 7ff8197fa640 1 -- 172.21.15.16:0/3355822181 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff810042030 con 0x7ff81c102790 2024-09-12T02:06:34.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.299+0000 7ff824ba3740 1 Processor -- start 2024-09-12T02:06:34.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.299+0000 7ff824ba3740 1 -- start start 2024-09-12T02:06:34.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.299+0000 7ff824ba3740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555a1a76ab40 0x555a1a769090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:34.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.299+0000 7ff824ba3740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555a1a7695d0 0x555a1a769a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:34.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.299+0000 7ff824ba3740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555a1a774370 0x555a1a776760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:34.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.299+0000 7ff824ba3740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x555a1a6cf2f0 con 0x555a1a774370 2024-09-12T02:06:34.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.299+0000 7ff824ba3740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x555a1a676cf0 con 0x555a1a76ab40 2024-09-12T02:06:34.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.299+0000 7ff824ba3740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x555a1a6cb2b0 con 0x555a1a7695d0 2024-09-12T02:06:34.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.299+0000 7ff81b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555a1a774370 0x555a1a776760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:34.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.299+0000 7ff81b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555a1a774370 0x555a1a776760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56678/0 (socket says 172.21.15.16:56678) 2024-09-12T02:06:34.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.299+0000 7ff81bfff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555a1a7695d0 0x555a1a769a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:34.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.299+0000 7ff8197fa640 1 --2- 172.21.15.16:0/3355822181 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7ff8040a2b80 0x7ff8040a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:34.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.299+0000 7ff81b7fe640 1 -- 172.21.15.16:0/317082303 learned_addr learned my addr 172.21.15.16:0/317082303 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:06:34.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.300+0000 7ff820d67640 1 --2- 172.21.15.16:0/317082303 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555a1a76ab40 0x555a1a769090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:34.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.300+0000 7ff820d67640 1 --2- 172.21.15.16:0/3355822181 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7ff8040a2b80 0x7ff8040a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:34.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.300+0000 7ff8197fa640 1 -- 172.21.15.16:0/3355822181 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7ff8100ef870 con 0x7ff81c102790 2024-09-12T02:06:34.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.300+0000 7ff81bfff640 1 -- 172.21.15.16:0/317082303 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555a1a76ab40 msgr2=0x555a1a769090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:06:34.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.300+0000 7ff81bfff640 1 --2- 172.21.15.16:0/317082303 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555a1a76ab40 0x555a1a769090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:34.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.300+0000 7ff81bfff640 1 -- 172.21.15.16:0/317082303 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555a1a774370 msgr2=0x555a1a776760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:06:34.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.300+0000 7ff81bfff640 1 --2- 172.21.15.16:0/317082303 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555a1a774370 0x555a1a776760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:34.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.300+0000 7ff81bfff640 1 -- 172.21.15.16:0/317082303 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a1a65b3a0 con 0x555a1a7695d0 2024-09-12T02:06:34.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.300+0000 7ff81b7fe640 1 --2- 172.21.15.16:0/317082303 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555a1a774370 0x555a1a776760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:06:34.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.300+0000 7ff81bfff640 1 --2- 172.21.15.16:0/317082303 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555a1a7695d0 0x555a1a769a10 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff80c0099a0 tx=0x7ff80c009e60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb6d3fc4f53a0192 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:06:34.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.300+0000 7ff820d67640 1 --2- 172.21.15.16:0/3355822181 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7ff8040a2b80 0x7ff8040a5040 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7ff81400a780 tx=0x7ff81400a4d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:06:34.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.300+0000 7ff8027fc640 1 -- 172.21.15.16:0/317082303 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff80c017070 con 0x555a1a7695d0 2024-09-12T02:06:34.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.300+0000 7ff8027fc640 1 -- 172.21.15.16:0/317082303 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff80c013470 con 0x555a1a7695d0 2024-09-12T02:06:34.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.300+0000 7ff8027fc640 1 -- 172.21.15.16:0/317082303 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff80c012610 con 0x555a1a7695d0 2024-09-12T02:06:34.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.301+0000 7ff824ba3740 1 -- 172.21.15.16:0/317082303 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555a1a7695d0 msgr2=0x555a1a769a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:34.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.301+0000 7ff824ba3740 1 --2- 172.21.15.16:0/317082303 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555a1a7695d0 0x555a1a769a10 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff80c0099a0 tx=0x7ff80c009e60 comp rx=0 tx=0).stop 2024-09-12T02:06:34.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.301+0000 7ff824ba3740 1 -- 172.21.15.16:0/317082303 shutdown_connections 2024-09-12T02:06:34.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.301+0000 7ff824ba3740 1 --2- 172.21.15.16:0/317082303 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555a1a774370 0x555a1a776760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:34.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.301+0000 7ff824ba3740 1 --2- 172.21.15.16:0/317082303 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555a1a7695d0 0x555a1a769a10 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:34.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.301+0000 7ff824ba3740 1 --2- 172.21.15.16:0/317082303 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555a1a76ab40 0x555a1a769090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:34.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.301+0000 7ff824ba3740 1 -- 172.21.15.16:0/317082303 >> 172.21.15.16:0/317082303 conn(0x555a1a732310 msgr2=0x555a1a765c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:06:34.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.301+0000 7ff824ba3740 1 -- 172.21.15.16:0/317082303 shutdown_connections 2024-09-12T02:06:34.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.301+0000 7ff824ba3740 1 -- 172.21.15.16:0/317082303 wait complete. 2024-09-12T02:06:34.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.302+0000 7ff824ba3740 1 -- start start 2024-09-12T02:06:34.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.302+0000 7ff824ba3740 1 Processor -- start 2024-09-12T02:06:34.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.302+0000 7ff824ba3740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555a1a7695d0 0x555a1a773cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:34.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.302+0000 7ff824ba3740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555a1a76ab40 0x555a1a770f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:34.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.302+0000 7ff824ba3740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555a1a774370 0x555a1a7714d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:34.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.302+0000 7ff824ba3740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x555a1a777070 con 0x555a1a7695d0 2024-09-12T02:06:34.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.302+0000 7ff824ba3740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x555a1a68a060 con 0x555a1a76ab40 2024-09-12T02:06:34.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.302+0000 7ff824ba3740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x555a1a7771f0 con 0x555a1a774370 2024-09-12T02:06:34.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.302+0000 7ff824ba3740 10 client.0 _ll_register_callbacks cb 0x555a1a3dd8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:06:34.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.302+0000 7ff81bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555a1a7695d0 0x555a1a773cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:34.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.302+0000 7ff81bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555a1a7695d0 0x555a1a773cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56686/0 (socket says 172.21.15.16:56686) 2024-09-12T02:06:34.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.302+0000 7ff81bfff640 1 -- 172.21.15.16:0/2452820691 learned_addr learned my addr 172.21.15.16:0/2452820691 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:06:34.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.302+0000 7ff81b7fe640 1 --2- 172.21.15.16:0/2452820691 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555a1a76ab40 0x555a1a770f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:34.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.303+0000 7ff820d67640 1 --2- 172.21.15.16:0/2452820691 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555a1a774370 0x555a1a7714d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:34.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.303+0000 7ff81bfff640 1 -- 172.21.15.16:0/2452820691 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555a1a774370 msgr2=0x555a1a7714d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:06:34.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.303+0000 7ff81bfff640 1 --2- 172.21.15.16:0/2452820691 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555a1a774370 0x555a1a7714d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:34.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.303+0000 7ff81bfff640 1 -- 172.21.15.16:0/2452820691 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555a1a76ab40 msgr2=0x555a1a770f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:06:34.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.303+0000 7ff81bfff640 1 --2- 172.21.15.16:0/2452820691 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555a1a76ab40 0x555a1a770f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:34.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.303+0000 7ff81bfff640 1 -- 172.21.15.16:0/2452820691 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a1a4403c0 con 0x555a1a7695d0 2024-09-12T02:06:34.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.303+0000 7ff81bfff640 1 --2- 172.21.15.16:0/2452820691 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555a1a7695d0 0x555a1a773cc0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7ff80c004810 tx=0x7ff80c006e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=249a941c2e79ccf0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:06:34.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.303+0000 7ff7fb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:06:34.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.303+0000 7ff7fb7fe640 1 -- 172.21.15.16:0/2452820691 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff80c017050 con 0x555a1a7695d0 2024-09-12T02:06:34.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.303+0000 7ff7fb7fe640 1 -- 172.21.15.16:0/2452820691 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7ff80c01c450 con 0x555a1a7695d0 2024-09-12T02:06:34.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.303+0000 7ff824ba3740 1 -- 172.21.15.16:0/2452820691 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff80c013470 con 0x555a1a7695d0 2024-09-12T02:06:34.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.303+0000 7ff7fb7fe640 1 -- 172.21.15.16:0/2452820691 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff80c01c650 con 0x555a1a7695d0 2024-09-12T02:06:34.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.303+0000 7ff824ba3740 10 client.16845 fetch_fsmap learned FSMap version 25 2024-09-12T02:06:34.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.303+0000 7ff824ba3740 1 -- 172.21.15.16:0/2452820691 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555a1a782a80 con 0x555a1a7695d0 2024-09-12T02:06:34.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.304+0000 7ff7fb7fe640 1 -- 172.21.15.16:0/2452820691 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff80c013470 con 0x555a1a7695d0 2024-09-12T02:06:34.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.304+0000 7ff7fb7fe640 1 -- 172.21.15.16:0/2452820691 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7ff80c026740 con 0x555a1a7695d0 2024-09-12T02:06:34.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.304+0000 7ff824ba3740 10 client.16845 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:06:34.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.304+0000 7ff824ba3740 10 client.16845 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-12T02:06:34.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.304+0000 7ff824ba3740 20 client.16845 populate_metadata read hostname 'smithi016' 2024-09-12T02:06:34.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.304+0000 7ff824ba3740 1 --2- 172.21.15.16:0/2452820691 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x555a1a783e80 0x555a1a786270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:34.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.304+0000 7ff824ba3740 4 client.16845 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:06:34.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.304+0000 7ff824ba3740 1 -- 172.21.15.16:0/2452820691 --> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555a1a6dc860 con 0x555a1a783e80 2024-09-12T02:06:34.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.304+0000 7ff81b7fe640 1 --2- 172.21.15.16:0/2452820691 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x555a1a783e80 0x555a1a786270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:34.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.305+0000 7ff81b7fe640 1 --2- 172.21.15.16:0/2452820691 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x555a1a783e80 0x555a1a786270 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=30a6c5066707fc95 server_cookie=c5debbd5ac83e7ab in_seq=0 out_seq=0 2024-09-12T02:06:34.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.305+0000 7ff7fb7fe640 10 client.16845 ms_handle_connect on v2:172.21.15.184:6836/1793393087 2024-09-12T02:06:34.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.305+0000 7ff7fb7fe640 1 -- 172.21.15.16:0/2452820691 <== mds.2 v2:172.21.15.184:6836/1793393087 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555a1a6dc860 con 0x555a1a783e80 2024-09-12T02:06:34.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.305+0000 7ff7fb7fe640 10 client.16845 handle_command_reply: tid=0 2024-09-12T02:06:34.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.305+0000 7ff824ba3740 1 client.16845 shutdown 2024-09-12T02:06:34.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.305+0000 7ff824ba3740 1 -- 172.21.15.16:0/2452820691 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555a1a7695d0 msgr2=0x555a1a773cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:34.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.305+0000 7ff824ba3740 1 --2- 172.21.15.16:0/2452820691 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555a1a7695d0 0x555a1a773cc0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7ff80c004810 tx=0x7ff80c006e40 comp rx=0 tx=0).stop 2024-09-12T02:06:34.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.305+0000 7ff824ba3740 1 -- 172.21.15.16:0/2452820691 shutdown_connections 2024-09-12T02:06:34.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.305+0000 7ff824ba3740 1 --2- 172.21.15.16:0/2452820691 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555a1a774370 0x555a1a7714d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:34.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.305+0000 7ff824ba3740 1 --2- 172.21.15.16:0/2452820691 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555a1a76ab40 0x555a1a770f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:34.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.305+0000 7ff824ba3740 1 --2- 172.21.15.16:0/2452820691 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x555a1a783e80 0x555a1a786270 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:34.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.305+0000 7ff824ba3740 1 --2- 172.21.15.16:0/2452820691 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555a1a7695d0 0x555a1a773cc0 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:34.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.305+0000 7ff7fb7fe640 0 client.16845 ms_handle_reset on v2:172.21.15.184:6836/1793393087 2024-09-12T02:06:34.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.305+0000 7ff824ba3740 1 -- 172.21.15.16:0/2452820691 >> 172.21.15.16:0/2452820691 conn(0x555a1a732310 msgr2=0x555a1a76b960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:06:34.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.306+0000 7ff824ba3740 1 -- 172.21.15.16:0/2452820691 shutdown_connections 2024-09-12T02:06:34.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.306+0000 7ff824ba3740 1 -- 172.21.15.16:0/2452820691 wait complete. 2024-09-12T02:06:34.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.306+0000 7ff824ba3740 20 client.16845 trim_cache size 0 max 16384 2024-09-12T02:06:34.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.318+0000 7ff824ba3740 1 Processor -- start 2024-09-12T02:06:34.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.319+0000 7ff824ba3740 1 -- start start 2024-09-12T02:06:34.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.319+0000 7ff824ba3740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555a1a7695d0 0x555a1a780310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:34.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.319+0000 7ff824ba3740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555a1a76ab40 0x555a1a76ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:34.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.319+0000 7ff824ba3740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555a1a774370 0x555a1a77c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:34.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.319+0000 7ff824ba3740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff81c1afd40 con 0x555a1a774370 2024-09-12T02:06:34.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.319+0000 7ff824ba3740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff81c1ad610 con 0x555a1a76ab40 2024-09-12T02:06:34.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.319+0000 7ff824ba3740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff81c1ab4a0 con 0x555a1a7695d0 2024-09-12T02:06:34.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.319+0000 7ff820d67640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555a1a774370 0x555a1a77c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:34.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.319+0000 7ff820d67640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555a1a774370 0x555a1a77c860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56702/0 (socket says 172.21.15.16:56702) 2024-09-12T02:06:34.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.319+0000 7ff820d67640 1 -- 172.21.15.16:0/2144237502 learned_addr learned my addr 172.21.15.16:0/2144237502 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:06:34.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.319+0000 7ff81bfff640 1 --2- 172.21.15.16:0/2144237502 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555a1a7695d0 0x555a1a780310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:34.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.319+0000 7ff81b7fe640 1 --2- 172.21.15.16:0/2144237502 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555a1a76ab40 0x555a1a76ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:34.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.319+0000 7ff820d67640 1 -- 172.21.15.16:0/2144237502 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555a1a7695d0 msgr2=0x555a1a780310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:06:34.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.319+0000 7ff820d67640 1 --2- 172.21.15.16:0/2144237502 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555a1a7695d0 0x555a1a780310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:34.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.319+0000 7ff820d67640 1 -- 172.21.15.16:0/2144237502 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555a1a76ab40 msgr2=0x555a1a76ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:06:34.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.319+0000 7ff820d67640 1 --2- 172.21.15.16:0/2144237502 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555a1a76ab40 0x555a1a76ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:34.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.319+0000 7ff820d67640 1 -- 172.21.15.16:0/2144237502 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a1a7771e0 con 0x555a1a774370 2024-09-12T02:06:34.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.319+0000 7ff81b7fe640 1 --2- 172.21.15.16:0/2144237502 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555a1a76ab40 0x555a1a76ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:06:34.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.320+0000 7ff820d67640 1 --2- 172.21.15.16:0/2144237502 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555a1a774370 0x555a1a77c860 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7ff8140145a0 tx=0x7ff81400a910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db9024d63a87db36 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:06:34.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.320+0000 7ff801ffb640 1 -- 172.21.15.16:0/2144237502 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff814004d20 con 0x555a1a774370 2024-09-12T02:06:34.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.320+0000 7ff801ffb640 1 -- 172.21.15.16:0/2144237502 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff8140148d0 con 0x555a1a774370 2024-09-12T02:06:34.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.320+0000 7ff801ffb640 1 -- 172.21.15.16:0/2144237502 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff814004480 con 0x555a1a774370 2024-09-12T02:06:34.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.321+0000 7ff824ba3740 1 -- 172.21.15.16:0/2144237502 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555a1a774370 msgr2=0x555a1a77c860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:34.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.321+0000 7ff824ba3740 1 --2- 172.21.15.16:0/2144237502 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555a1a774370 0x555a1a77c860 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7ff8140145a0 tx=0x7ff81400a910 comp rx=0 tx=0).stop 2024-09-12T02:06:34.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.321+0000 7ff824ba3740 1 -- 172.21.15.16:0/2144237502 shutdown_connections 2024-09-12T02:06:34.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.321+0000 7ff824ba3740 1 --2- 172.21.15.16:0/2144237502 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555a1a774370 0x555a1a77c860 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:34.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.321+0000 7ff824ba3740 1 --2- 172.21.15.16:0/2144237502 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555a1a76ab40 0x555a1a76ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:34.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.321+0000 7ff824ba3740 1 --2- 172.21.15.16:0/2144237502 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555a1a7695d0 0x555a1a780310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:34.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.321+0000 7ff824ba3740 1 -- 172.21.15.16:0/2144237502 >> 172.21.15.16:0/2144237502 conn(0x555a1a732310 msgr2=0x555a1a76f860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:06:34.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.321+0000 7ff824ba3740 1 -- 172.21.15.16:0/2144237502 shutdown_connections 2024-09-12T02:06:34.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.321+0000 7ff824ba3740 1 -- 172.21.15.16:0/2144237502 wait complete. 2024-09-12T02:06:34.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.321+0000 7ff824ba3740 1 -- start start 2024-09-12T02:06:34.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.321+0000 7ff824ba3740 1 Processor -- start 2024-09-12T02:06:34.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.322+0000 7ff824ba3740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555a1a7695d0 0x555a1a76d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:34.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.322+0000 7ff824ba3740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555a1a76ab40 0x555a1a76d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:34.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.322+0000 7ff824ba3740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555a1a774370 0x555a1a76dc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:34.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.322+0000 7ff824ba3740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff81c1abaf0 con 0x555a1a76ab40 2024-09-12T02:06:34.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.322+0000 7ff824ba3740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff81c1acf40 con 0x555a1a774370 2024-09-12T02:06:34.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.322+0000 7ff824ba3740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff81c1ac1a0 con 0x555a1a7695d0 2024-09-12T02:06:34.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.322+0000 7ff824ba3740 10 client.0 _ll_register_callbacks cb 0x7ff8100b7d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:06:34.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.322+0000 7ff81b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555a1a76ab40 0x555a1a76d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:34.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.322+0000 7ff81b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555a1a76ab40 0x555a1a76d740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56710/0 (socket says 172.21.15.16:56710) 2024-09-12T02:06:34.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.322+0000 7ff81b7fe640 1 -- 172.21.15.16:0/53657284 learned_addr learned my addr 172.21.15.16:0/53657284 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:06:34.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.322+0000 7ff81bfff640 1 --2- 172.21.15.16:0/53657284 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555a1a7695d0 0x555a1a76d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:34.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.322+0000 7ff820d67640 1 --2- 172.21.15.16:0/53657284 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555a1a774370 0x555a1a76dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:34.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.322+0000 7ff81b7fe640 1 -- 172.21.15.16:0/53657284 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555a1a7695d0 msgr2=0x555a1a76d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:06:34.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.322+0000 7ff81b7fe640 1 --2- 172.21.15.16:0/53657284 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555a1a7695d0 0x555a1a76d200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:34.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.322+0000 7ff81b7fe640 1 -- 172.21.15.16:0/53657284 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555a1a774370 msgr2=0x555a1a76dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:06:34.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.322+0000 7ff81b7fe640 1 --2- 172.21.15.16:0/53657284 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555a1a774370 0x555a1a76dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:34.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.322+0000 7ff81b7fe640 1 -- 172.21.15.16:0/53657284 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a1a76c120 con 0x555a1a76ab40 2024-09-12T02:06:34.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.322+0000 7ff820d67640 1 --2- 172.21.15.16:0/53657284 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555a1a774370 0x555a1a76dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:06:34.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.322+0000 7ff81b7fe640 1 --2- 172.21.15.16:0/53657284 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555a1a76ab40 0x555a1a76d740 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7ff810000c00 tx=0x7ff81003b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48215b786b746aeb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:06:34.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.322+0000 7ff7fb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:06:34.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.323+0000 7ff7fb7fe640 1 -- 172.21.15.16:0/53657284 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff810049040 con 0x555a1a76ab40 2024-09-12T02:06:34.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.323+0000 7ff7fb7fe640 1 -- 172.21.15.16:0/53657284 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7ff81003bd00 con 0x555a1a76ab40 2024-09-12T02:06:34.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.323+0000 7ff824ba3740 1 -- 172.21.15.16:0/53657284 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff8140148d0 con 0x555a1a76ab40 2024-09-12T02:06:34.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.323+0000 7ff7fb7fe640 1 -- 172.21.15.16:0/53657284 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff8100bc870 con 0x555a1a76ab40 2024-09-12T02:06:34.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.323+0000 7ff824ba3740 10 client.16851 fetch_fsmap learned FSMap version 25 2024-09-12T02:06:34.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.323+0000 7ff824ba3740 1 -- 172.21.15.16:0/53657284 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555a1a7f2ac0 con 0x555a1a76ab40 2024-09-12T02:06:34.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.323+0000 7ff7fb7fe640 1 -- 172.21.15.16:0/53657284 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff8100f1e50 con 0x555a1a76ab40 2024-09-12T02:06:34.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.323+0000 7ff7fb7fe640 1 -- 172.21.15.16:0/53657284 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7ff8100b6ce0 con 0x555a1a76ab40 2024-09-12T02:06:34.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.323+0000 7ff824ba3740 10 client.16851 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:06:34.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.323+0000 7ff824ba3740 10 client.16851 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-12T02:06:34.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.323+0000 7ff824ba3740 20 client.16851 populate_metadata read hostname 'smithi016' 2024-09-12T02:06:34.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.323+0000 7ff824ba3740 1 --2- 172.21.15.16:0/53657284 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x555a1a7f31a0 0x555a1a7f55c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:34.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.323+0000 7ff824ba3740 4 client.16851 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T02:06:34.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.323+0000 7ff824ba3740 1 -- 172.21.15.16:0/53657284 --> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7ff8140148d0 con 0x555a1a7f31a0 2024-09-12T02:06:34.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.324+0000 7ff81bfff640 1 --2- 172.21.15.16:0/53657284 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x555a1a7f31a0 0x555a1a7f55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:34.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.324+0000 7ff81bfff640 1 --2- 172.21.15.16:0/53657284 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x555a1a7f31a0 0x555a1a7f55c0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d1e6aa1b35bc6499 server_cookie=2eb886bd6e4a9e80 in_seq=0 out_seq=0 2024-09-12T02:06:34.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.324+0000 7ff7fb7fe640 10 client.16851 ms_handle_connect on v2:172.21.15.184:6836/1793393087 2024-09-12T02:06:34.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.324+0000 7ff7fb7fe640 1 -- 172.21.15.16:0/53657284 <== mds.2 v2:172.21.15.184:6836/1793393087 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7ff8140148d0 con 0x555a1a7f31a0 2024-09-12T02:06:34.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.324+0000 7ff7fb7fe640 10 client.16851 handle_command_reply: tid=0 2024-09-12T02:06:34.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.324+0000 7ff824ba3740 1 client.16851 shutdown 2024-09-12T02:06:34.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.325+0000 7ff824ba3740 1 -- 172.21.15.16:0/53657284 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555a1a76ab40 msgr2=0x555a1a76d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:34.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.325+0000 7ff824ba3740 1 --2- 172.21.15.16:0/53657284 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555a1a76ab40 0x555a1a76d740 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7ff810000c00 tx=0x7ff81003b790 comp rx=0 tx=0).stop 2024-09-12T02:06:34.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.325+0000 7ff824ba3740 1 -- 172.21.15.16:0/53657284 shutdown_connections 2024-09-12T02:06:34.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.325+0000 7ff824ba3740 1 --2- 172.21.15.16:0/53657284 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555a1a774370 0x555a1a76dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:34.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.325+0000 7ff824ba3740 1 --2- 172.21.15.16:0/53657284 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x555a1a7f31a0 0x555a1a7f55c0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:34.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.325+0000 7ff824ba3740 1 --2- 172.21.15.16:0/53657284 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555a1a76ab40 0x555a1a76d740 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:34.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.325+0000 7ff824ba3740 1 --2- 172.21.15.16:0/53657284 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555a1a7695d0 0x555a1a76d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:34.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.325+0000 7ff7fb7fe640 0 client.16851 ms_handle_reset on v2:172.21.15.184:6836/1793393087 2024-09-12T02:06:34.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.325+0000 7ff824ba3740 1 -- 172.21.15.16:0/53657284 >> 172.21.15.16:0/53657284 conn(0x555a1a732310 msgr2=0x555a1a770080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:06:34.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.325+0000 7ff824ba3740 1 -- 172.21.15.16:0/53657284 shutdown_connections 2024-09-12T02:06:34.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.325+0000 7ff824ba3740 1 -- 172.21.15.16:0/53657284 wait complete. 2024-09-12T02:06:34.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.325+0000 7ff824ba3740 20 client.16851 trim_cache size 0 max 16384 2024-09-12T02:06:34.340 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T02:06:34.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.326+0000 7ff8227ca640 1 -- 172.21.15.16:0/3355822181 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7ff8040a2b80 msgr2=0x7ff8040a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:34.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.326+0000 7ff8227ca640 1 --2- 172.21.15.16:0/3355822181 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7ff8040a2b80 0x7ff8040a5040 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7ff81400a780 tx=0x7ff81400a4d0 comp rx=0 tx=0).stop 2024-09-12T02:06:34.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.326+0000 7ff8227ca640 1 -- 172.21.15.16:0/3355822181 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff81c102790 msgr2=0x7ff81c198a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:34.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.326+0000 7ff8227ca640 1 --2- 172.21.15.16:0/3355822181 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff81c102790 0x7ff81c198a50 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7ff810005ab0 tx=0x7ff8100042f0 comp rx=0 tx=0).stop 2024-09-12T02:06:34.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.326+0000 7ff8227ca640 1 -- 172.21.15.16:0/3355822181 shutdown_connections 2024-09-12T02:06:34.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.326+0000 7ff8227ca640 1 --2- 172.21.15.16:0/3355822181 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7ff8040a2b80 0x7ff8040a5040 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:34.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.326+0000 7ff8227ca640 1 --2- 172.21.15.16:0/3355822181 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff81c106f10 0x7ff81c19cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:34.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.326+0000 7ff8227ca640 1 --2- 172.21.15.16:0/3355822181 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff81c102790 0x7ff81c198a50 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:34.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.326+0000 7ff8227ca640 1 --2- 172.21.15.16:0/3355822181 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff81c101de0 0x7ff81c198510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:34.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.326+0000 7ff8227ca640 1 -- 172.21.15.16:0/3355822181 >> 172.21.15.16:0/3355822181 conn(0x7ff81c0fd8c0 msgr2=0x7ff81c0fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:06:34.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.326+0000 7ff8227ca640 1 -- 172.21.15.16:0/3355822181 shutdown_connections 2024-09-12T02:06:34.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:34.326+0000 7ff8227ca640 1 -- 172.21.15.16:0/3355822181 wait complete. 2024-09-12T02:06:34.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:06:34 smithi184 ceph-mon[35593]: pgmap v474: 97 pgs: 97 active+clean; 2.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-09-12T02:06:34.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:06:34 smithi033 ceph-mon[35332]: pgmap v474: 97 pgs: 97 active+clean; 2.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-09-12T02:06:34.987 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-12T02:06:35.248 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:06:35.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.736+0000 7f77d487e640 1 -- 172.21.15.16:0/2354958997 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77d0101420 msgr2=0x7f77d010d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:35.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.736+0000 7f77d487e640 1 --2- 172.21.15.16:0/2354958997 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77d0101420 0x7f77d010d950 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f77c8009cd0 tx=0x7f77c802a8e0 comp rx=0 tx=0).stop 2024-09-12T02:06:35.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.736+0000 7f77d487e640 1 -- 172.21.15.16:0/2354958997 shutdown_connections 2024-09-12T02:06:35.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.736+0000 7f77d487e640 1 --2- 172.21.15.16:0/2354958997 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77d0101420 0x7f77d010d950 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:35.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.736+0000 7f77d487e640 1 --2- 172.21.15.16:0/2354958997 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f77d0100a80 0x7f77d0100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:35.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.736+0000 7f77d487e640 1 --2- 172.21.15.16:0/2354958997 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77d0106a80 0x7f77d0106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:35.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.736+0000 7f77d487e640 1 -- 172.21.15.16:0/2354958997 >> 172.21.15.16:0/2354958997 conn(0x7f77d00fc7a0 msgr2=0x7f77d00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:06:35.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.736+0000 7f77d487e640 1 -- 172.21.15.16:0/2354958997 shutdown_connections 2024-09-12T02:06:35.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.736+0000 7f77d487e640 1 -- 172.21.15.16:0/2354958997 wait complete. 2024-09-12T02:06:35.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.737+0000 7f77d487e640 1 Processor -- start 2024-09-12T02:06:35.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.737+0000 7f77d487e640 1 -- start start 2024-09-12T02:06:35.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.737+0000 7f77d487e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77d0100a80 0x7f77d0199230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:35.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.737+0000 7f77d487e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77d0101420 0x7f77d0199770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:35.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.737+0000 7f77d487e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f77d0106a80 0x7f77d01a42b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:35.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.737+0000 7f77d487e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f77d0110e80 con 0x7f77d0101420 2024-09-12T02:06:35.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.737+0000 7f77d487e640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f77d0110d00 con 0x7f77d0100a80 2024-09-12T02:06:35.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.737+0000 7f77d487e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f77d0111000 con 0x7f77d0106a80 2024-09-12T02:06:35.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.738+0000 7f77cdd9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77d0101420 0x7f77d0199770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:35.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.738+0000 7f77cdd9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77d0101420 0x7f77d0199770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56724/0 (socket says 172.21.15.16:56724) 2024-09-12T02:06:35.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.738+0000 7f77cdd9b640 1 -- 172.21.15.16:0/2374693390 learned_addr learned my addr 172.21.15.16:0/2374693390 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:06:35.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.738+0000 7f77ce59c640 1 --2- 172.21.15.16:0/2374693390 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77d0100a80 0x7f77d0199230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:35.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.738+0000 7f77ced9d640 1 --2- 172.21.15.16:0/2374693390 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f77d0106a80 0x7f77d01a42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:35.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.738+0000 7f77cdd9b640 1 -- 172.21.15.16:0/2374693390 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f77d0106a80 msgr2=0x7f77d01a42b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:35.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.738+0000 7f77cdd9b640 1 --2- 172.21.15.16:0/2374693390 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f77d0106a80 0x7f77d01a42b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:35.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.738+0000 7f77cdd9b640 1 -- 172.21.15.16:0/2374693390 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77d0100a80 msgr2=0x7f77d0199230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:35.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.738+0000 7f77cdd9b640 1 --2- 172.21.15.16:0/2374693390 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77d0100a80 0x7f77d0199230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:35.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.738+0000 7f77cdd9b640 1 -- 172.21.15.16:0/2374693390 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77d01a47f0 con 0x7f77d0101420 2024-09-12T02:06:35.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.738+0000 7f77cdd9b640 1 --2- 172.21.15.16:0/2374693390 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77d0101420 0x7f77d0199770 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f77c400c920 tx=0x7f77c400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:06:35.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.738+0000 7f77bf7fe640 1 -- 172.21.15.16:0/2374693390 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77c4007c70 con 0x7f77d0101420 2024-09-12T02:06:35.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.738+0000 7f77d487e640 1 -- 172.21.15.16:0/2374693390 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77d01a49c0 con 0x7f77d0101420 2024-09-12T02:06:35.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.738+0000 7f77bf7fe640 1 -- 172.21.15.16:0/2374693390 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77c4007e10 con 0x7f77d0101420 2024-09-12T02:06:35.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.738+0000 7f77bf7fe640 1 -- 172.21.15.16:0/2374693390 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77c4018890 con 0x7f77d0101420 2024-09-12T02:06:35.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.738+0000 7f77d487e640 1 -- 172.21.15.16:0/2374693390 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77d01a4f00 con 0x7f77d0101420 2024-09-12T02:06:35.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.741+0000 7f77bf7fe640 1 -- 172.21.15.16:0/2374693390 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f77c4018a70 con 0x7f77d0101420 2024-09-12T02:06:35.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.741+0000 7f77d6c57740 1 Processor -- start 2024-09-12T02:06:35.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.741+0000 7f77d6c57740 1 -- start start 2024-09-12T02:06:35.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.741+0000 7f77d6c57740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5652b73d6b40 0x5652b73d5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:35.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.741+0000 7f77d6c57740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5652b73d55d0 0x5652b73d5a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:35.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.741+0000 7f77d6c57740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5652b73e4780 0x5652b73e6b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:35.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.742+0000 7f77d6c57740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5652b733b2f0 con 0x5652b73e4780 2024-09-12T02:06:35.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.742+0000 7f77d6c57740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5652b72e2cf0 con 0x5652b73d55d0 2024-09-12T02:06:35.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.742+0000 7f77d6c57740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5652b73372b0 con 0x5652b73d6b40 2024-09-12T02:06:35.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.742+0000 7f77cdd9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5652b73e4780 0x5652b73e6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:35.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.742+0000 7f77cdd9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5652b73e4780 0x5652b73e6b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56728/0 (socket says 172.21.15.16:56728) 2024-09-12T02:06:35.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.742+0000 7f77cdd9b640 1 -- 172.21.15.16:0/1736702586 learned_addr learned my addr 172.21.15.16:0/1736702586 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:06:35.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.742+0000 7f77ced9d640 1 --2- 172.21.15.16:0/1736702586 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5652b73d6b40 0x5652b73d5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:35.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.742+0000 7f77ce59c640 1 --2- 172.21.15.16:0/1736702586 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5652b73d55d0 0x5652b73d5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:35.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.742+0000 7f77cdd9b640 1 -- 172.21.15.16:0/1736702586 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5652b73d6b40 msgr2=0x5652b73d5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:06:35.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.742+0000 7f77cdd9b640 1 --2- 172.21.15.16:0/1736702586 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5652b73d6b40 0x5652b73d5090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:35.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.742+0000 7f77bf7fe640 1 --2- 172.21.15.16:0/2374693390 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f77b80a2b80 0x7f77b80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:35.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.742+0000 7f77cdd9b640 1 -- 172.21.15.16:0/1736702586 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5652b73d55d0 msgr2=0x5652b73d5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:06:35.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.742+0000 7f77cdd9b640 1 --2- 172.21.15.16:0/1736702586 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5652b73d55d0 0x5652b73d5a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:35.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.742+0000 7f77cdd9b640 1 -- 172.21.15.16:0/1736702586 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5652b72c73a0 con 0x5652b73e4780 2024-09-12T02:06:35.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.742+0000 7f77ced9d640 1 --2- 172.21.15.16:0/2374693390 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f77b80a2b80 0x7f77b80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:35.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.742+0000 7f77bf7fe640 1 -- 172.21.15.16:0/2374693390 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f77c40cf1a0 con 0x7f77d0101420 2024-09-12T02:06:35.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.742+0000 7f77cdd9b640 1 --2- 172.21.15.16:0/1736702586 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5652b73e4780 0x5652b73e6b70 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f77c40bd3c0 tx=0x7f77c40c6630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=918720d936e0ea8b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:06:35.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.743+0000 7f77bcff9640 1 -- 172.21.15.16:0/1736702586 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77c40bbe60 con 0x5652b73e4780 2024-09-12T02:06:35.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.743+0000 7f77bcff9640 1 -- 172.21.15.16:0/1736702586 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77c40bd3f0 con 0x5652b73e4780 2024-09-12T02:06:35.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.743+0000 7f77bcff9640 1 -- 172.21.15.16:0/1736702586 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77c40d2db0 con 0x5652b73e4780 2024-09-12T02:06:35.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.743+0000 7f77d6c57740 1 -- 172.21.15.16:0/1736702586 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5652b73e4780 msgr2=0x5652b73e6b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:35.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.743+0000 7f77ced9d640 1 --2- 172.21.15.16:0/2374693390 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f77b80a2b80 0x7f77b80a5040 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f77c8009430 tx=0x7f77c8005c50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:06:35.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.743+0000 7f77d6c57740 1 --2- 172.21.15.16:0/1736702586 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5652b73e4780 0x5652b73e6b70 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f77c40bd3c0 tx=0x7f77c40c6630 comp rx=0 tx=0).stop 2024-09-12T02:06:35.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.743+0000 7f77d6c57740 1 -- 172.21.15.16:0/1736702586 shutdown_connections 2024-09-12T02:06:35.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.743+0000 7f77d6c57740 1 --2- 172.21.15.16:0/1736702586 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5652b73e4780 0x5652b73e6b70 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:35.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.743+0000 7f77d6c57740 1 --2- 172.21.15.16:0/1736702586 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5652b73d55d0 0x5652b73d5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:35.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.743+0000 7f77d6c57740 1 --2- 172.21.15.16:0/1736702586 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5652b73d6b40 0x5652b73d5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:35.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.743+0000 7f77d6c57740 1 -- 172.21.15.16:0/1736702586 >> 172.21.15.16:0/1736702586 conn(0x5652b739e310 msgr2=0x5652b73d1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:06:35.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.744+0000 7f77d6c57740 1 -- 172.21.15.16:0/1736702586 shutdown_connections 2024-09-12T02:06:35.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.744+0000 7f77d6c57740 1 -- 172.21.15.16:0/1736702586 wait complete. 2024-09-12T02:06:35.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.744+0000 7f77d6c57740 1 -- start start 2024-09-12T02:06:35.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.744+0000 7f77d6c57740 1 Processor -- start 2024-09-12T02:06:35.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.744+0000 7f77d6c57740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5652b73d55d0 0x5652b73e41e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:35.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.744+0000 7f77d6c57740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5652b73d6b40 0x5652b73e1490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:35.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.744+0000 7f77d6c57740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5652b73e4780 0x5652b73e1c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:35.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.744+0000 7f77d6c57740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5652b73e7420 con 0x5652b73e4780 2024-09-12T02:06:35.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.744+0000 7f77d6c57740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5652b72f6060 con 0x5652b73d6b40 2024-09-12T02:06:35.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.744+0000 7f77d6c57740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5652b73e75a0 con 0x5652b73d55d0 2024-09-12T02:06:35.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.744+0000 7f77d6c57740 10 client.0 _ll_register_callbacks cb 0x5652b70498c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:06:35.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.744+0000 7f77ced9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5652b73e4780 0x5652b73e1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:35.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.744+0000 7f77ced9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5652b73e4780 0x5652b73e1c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56742/0 (socket says 172.21.15.16:56742) 2024-09-12T02:06:35.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.744+0000 7f77ced9d640 1 -- 172.21.15.16:0/1685047611 learned_addr learned my addr 172.21.15.16:0/1685047611 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:06:35.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.744+0000 7f77ce59c640 1 --2- 172.21.15.16:0/1685047611 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5652b73d55d0 0x5652b73e41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:35.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.744+0000 7f77cdd9b640 1 --2- 172.21.15.16:0/1685047611 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5652b73d6b40 0x5652b73e1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:35.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.745+0000 7f77ced9d640 1 -- 172.21.15.16:0/1685047611 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5652b73d55d0 msgr2=0x5652b73e41e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:06:35.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.745+0000 7f77ced9d640 1 --2- 172.21.15.16:0/1685047611 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5652b73d55d0 0x5652b73e41e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:35.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.745+0000 7f77ced9d640 1 -- 172.21.15.16:0/1685047611 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5652b73d6b40 msgr2=0x5652b73e1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:06:35.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.745+0000 7f77ced9d640 1 --2- 172.21.15.16:0/1685047611 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5652b73d6b40 0x5652b73e1490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:35.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.745+0000 7f77ced9d640 1 -- 172.21.15.16:0/1685047611 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5652b70ac3c0 con 0x5652b73e4780 2024-09-12T02:06:35.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.745+0000 7f77ced9d640 1 --2- 172.21.15.16:0/1685047611 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5652b73e4780 0x5652b73e1c20 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f77c8009cd0 tx=0x7f77c802a8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11b9da74ba8358a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:06:35.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.745+0000 7f77b5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:06:35.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.745+0000 7f77b5ffb640 1 -- 172.21.15.16:0/1685047611 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77c8056060 con 0x5652b73e4780 2024-09-12T02:06:35.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.745+0000 7f77b5ffb640 1 -- 172.21.15.16:0/1685047611 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f77c8041030 con 0x5652b73e4780 2024-09-12T02:06:35.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.745+0000 7f77d6c57740 1 -- 172.21.15.16:0/1685047611 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5652b7348860 con 0x5652b73e4780 2024-09-12T02:06:35.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.745+0000 7f77b5ffb640 1 -- 172.21.15.16:0/1685047611 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f77c8003fd0 con 0x5652b73e4780 2024-09-12T02:06:35.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.745+0000 7f77d6c57740 10 client.16863 fetch_fsmap learned FSMap version 25 2024-09-12T02:06:35.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.745+0000 7f77d6c57740 1 -- 172.21.15.16:0/1685047611 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5652b73f2f70 con 0x5652b73e4780 2024-09-12T02:06:35.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.745+0000 7f77b5ffb640 1 -- 172.21.15.16:0/1685047611 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5652b7348860 con 0x5652b73e4780 2024-09-12T02:06:35.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.746+0000 7f77b5ffb640 1 -- 172.21.15.16:0/1685047611 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f77c805b070 con 0x5652b73e4780 2024-09-12T02:06:35.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.746+0000 7f77d6c57740 10 client.16863 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:06:35.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.746+0000 7f77d6c57740 10 client.16863 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T02:06:35.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.746+0000 7f77d6c57740 20 client.16863 populate_metadata read hostname 'smithi016' 2024-09-12T02:06:35.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.746+0000 7f77d6c57740 1 --2- 172.21.15.16:0/1685047611 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x5652b73f4430 0x5652b73f6860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:35.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.746+0000 7f77d6c57740 4 client.16863 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:06:35.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.746+0000 7f77d6c57740 1 -- 172.21.15.16:0/1685047611 --> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5652b7342d70 con 0x5652b73f4430 2024-09-12T02:06:35.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.746+0000 7f77ce59c640 1 --2- 172.21.15.16:0/1685047611 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x5652b73f4430 0x5652b73f6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:35.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.746+0000 7f77ce59c640 1 --2- 172.21.15.16:0/1685047611 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x5652b73f4430 0x5652b73f6860 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8f86f9ba0ed0bc88 server_cookie=7bd22676d46fcaa7 in_seq=0 out_seq=0 2024-09-12T02:06:35.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.746+0000 7f77b5ffb640 10 client.16863 ms_handle_connect on v2:172.21.15.184:6838/4203347941 2024-09-12T02:06:35.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.747+0000 7f77b5ffb640 1 -- 172.21.15.16:0/1685047611 <== mds.1 v2:172.21.15.184:6838/4203347941 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5652b7342d70 con 0x5652b73f4430 2024-09-12T02:06:35.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.747+0000 7f77b5ffb640 10 client.16863 handle_command_reply: tid=0 2024-09-12T02:06:35.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.747+0000 7f77d6c57740 1 client.16863 shutdown 2024-09-12T02:06:35.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.747+0000 7f77d6c57740 1 -- 172.21.15.16:0/1685047611 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5652b73e4780 msgr2=0x5652b73e1c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:35.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.747+0000 7f77d6c57740 1 --2- 172.21.15.16:0/1685047611 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5652b73e4780 0x5652b73e1c20 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f77c8009cd0 tx=0x7f77c802a8e0 comp rx=0 tx=0).stop 2024-09-12T02:06:35.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.747+0000 7f77d6c57740 1 -- 172.21.15.16:0/1685047611 shutdown_connections 2024-09-12T02:06:35.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.747+0000 7f77d6c57740 1 --2- 172.21.15.16:0/1685047611 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x5652b73f4430 0x5652b73f6860 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:35.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.747+0000 7f77d6c57740 1 --2- 172.21.15.16:0/1685047611 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5652b73e4780 0x5652b73e1c20 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:35.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.747+0000 7f77b5ffb640 0 client.16863 ms_handle_reset on v2:172.21.15.184:6838/4203347941 2024-09-12T02:06:35.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.747+0000 7f77d6c57740 1 --2- 172.21.15.16:0/1685047611 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5652b73d6b40 0x5652b73e1490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:35.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.747+0000 7f77d6c57740 1 --2- 172.21.15.16:0/1685047611 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5652b73d55d0 0x5652b73e41e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:35.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.747+0000 7f77d6c57740 1 -- 172.21.15.16:0/1685047611 >> 172.21.15.16:0/1685047611 conn(0x5652b739e310 msgr2=0x5652b73d7990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:06:35.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.748+0000 7f77d6c57740 1 -- 172.21.15.16:0/1685047611 shutdown_connections 2024-09-12T02:06:35.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.748+0000 7f77d6c57740 1 -- 172.21.15.16:0/1685047611 wait complete. 2024-09-12T02:06:35.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.748+0000 7f77d6c57740 20 client.16863 trim_cache size 0 max 16384 2024-09-12T02:06:35.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.761+0000 7f77d6c57740 1 Processor -- start 2024-09-12T02:06:35.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.761+0000 7f77d6c57740 1 -- start start 2024-09-12T02:06:35.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.761+0000 7f77d6c57740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5652b73d55d0 0x5652b73d1e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:35.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.761+0000 7f77d6c57740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5652b73d6b40 0x5652b73fc130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:35.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.761+0000 7f77d6c57740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5652b73e4780 0x5652b73fc670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:35.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.761+0000 7f77d6c57740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f77d01adaf0 con 0x5652b73d6b40 2024-09-12T02:06:35.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.761+0000 7f77d6c57740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f77d01ab3c0 con 0x5652b73d55d0 2024-09-12T02:06:35.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.761+0000 7f77d6c57740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f77d01a9230 con 0x5652b73e4780 2024-09-12T02:06:35.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.761+0000 7f77cdd9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5652b73d6b40 0x5652b73fc130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:35.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.761+0000 7f77cdd9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5652b73d6b40 0x5652b73fc130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56750/0 (socket says 172.21.15.16:56750) 2024-09-12T02:06:35.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.761+0000 7f77cdd9b640 1 -- 172.21.15.16:0/3151588578 learned_addr learned my addr 172.21.15.16:0/3151588578 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:06:35.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.761+0000 7f77ced9d640 1 --2- 172.21.15.16:0/3151588578 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5652b73e4780 0x5652b73fc670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:35.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.761+0000 7f77ce59c640 1 --2- 172.21.15.16:0/3151588578 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5652b73d55d0 0x5652b73d1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:35.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.761+0000 7f77cdd9b640 1 -- 172.21.15.16:0/3151588578 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5652b73e4780 msgr2=0x5652b73fc670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:06:35.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.761+0000 7f77cdd9b640 1 --2- 172.21.15.16:0/3151588578 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5652b73e4780 0x5652b73fc670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:35.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.761+0000 7f77cdd9b640 1 -- 172.21.15.16:0/3151588578 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5652b73d55d0 msgr2=0x5652b73d1e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:06:35.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.761+0000 7f77cdd9b640 1 --2- 172.21.15.16:0/3151588578 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5652b73d55d0 0x5652b73d1e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:35.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.761+0000 7f77cdd9b640 1 -- 172.21.15.16:0/3151588578 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5652b73df9e0 con 0x5652b73d6b40 2024-09-12T02:06:35.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.761+0000 7f77ced9d640 1 --2- 172.21.15.16:0/3151588578 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5652b73e4780 0x5652b73fc670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:06:35.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.761+0000 7f77cdd9b640 1 --2- 172.21.15.16:0/3151588578 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5652b73d6b40 0x5652b73fc130 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f77c409b7a0 tx=0x7f77c400cf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c695cb810e73327a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:06:35.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.762+0000 7f77b7fff640 1 -- 172.21.15.16:0/3151588578 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77c40bbe60 con 0x5652b73d6b40 2024-09-12T02:06:35.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.762+0000 7f77b7fff640 1 -- 172.21.15.16:0/3151588578 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77c40d2b10 con 0x5652b73d6b40 2024-09-12T02:06:35.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.762+0000 7f77b7fff640 1 -- 172.21.15.16:0/3151588578 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77c409baa0 con 0x5652b73d6b40 2024-09-12T02:06:35.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.762+0000 7f77d6c57740 1 -- 172.21.15.16:0/3151588578 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5652b73d6b40 msgr2=0x5652b73fc130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:35.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.762+0000 7f77d6c57740 1 --2- 172.21.15.16:0/3151588578 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5652b73d6b40 0x5652b73fc130 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f77c409b7a0 tx=0x7f77c400cf10 comp rx=0 tx=0).stop 2024-09-12T02:06:35.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.762+0000 7f77d6c57740 1 -- 172.21.15.16:0/3151588578 shutdown_connections 2024-09-12T02:06:35.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.762+0000 7f77d6c57740 1 --2- 172.21.15.16:0/3151588578 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5652b73e4780 0x5652b73fc670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:35.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.762+0000 7f77d6c57740 1 --2- 172.21.15.16:0/3151588578 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5652b73d6b40 0x5652b73fc130 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:35.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.762+0000 7f77d6c57740 1 --2- 172.21.15.16:0/3151588578 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5652b73d55d0 0x5652b73d1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:35.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.762+0000 7f77d6c57740 1 -- 172.21.15.16:0/3151588578 >> 172.21.15.16:0/3151588578 conn(0x5652b739e310 msgr2=0x5652b73d1700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:06:35.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.762+0000 7f77d6c57740 1 -- 172.21.15.16:0/3151588578 shutdown_connections 2024-09-12T02:06:35.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.762+0000 7f77d6c57740 1 -- 172.21.15.16:0/3151588578 wait complete. 2024-09-12T02:06:35.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.763+0000 7f77d6c57740 1 -- start start 2024-09-12T02:06:35.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.763+0000 7f77d6c57740 1 Processor -- start 2024-09-12T02:06:35.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.763+0000 7f77d6c57740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5652b73d55d0 0x5652b73e8610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:35.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.763+0000 7f77d6c57740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5652b73d6b40 0x5652b73e8b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:35.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.763+0000 7f77d6c57740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5652b73e4780 0x5652b73e9090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:35.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.763+0000 7f77d6c57740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f77d01a9880 con 0x5652b73d55d0 2024-09-12T02:06:35.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.763+0000 7f77d6c57740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f77d01aacf0 con 0x5652b73d6b40 2024-09-12T02:06:35.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.763+0000 7f77d6c57740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f77d01a9f50 con 0x5652b73e4780 2024-09-12T02:06:35.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.763+0000 7f77d6c57740 10 client.0 _ll_register_callbacks cb 0x7f77c8005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:06:35.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.763+0000 7f77ce59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5652b73d55d0 0x5652b73e8610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:35.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.763+0000 7f77ce59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5652b73d55d0 0x5652b73e8610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56754/0 (socket says 172.21.15.16:56754) 2024-09-12T02:06:35.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.763+0000 7f77ce59c640 1 -- 172.21.15.16:0/1207970467 learned_addr learned my addr 172.21.15.16:0/1207970467 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:06:35.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.763+0000 7f77cdd9b640 1 --2- 172.21.15.16:0/1207970467 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5652b73d6b40 0x5652b73e8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:35.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.763+0000 7f77ced9d640 1 --2- 172.21.15.16:0/1207970467 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5652b73e4780 0x5652b73e9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:35.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.763+0000 7f77ce59c640 1 -- 172.21.15.16:0/1207970467 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5652b73e4780 msgr2=0x5652b73e9090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:06:35.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.763+0000 7f77ce59c640 1 --2- 172.21.15.16:0/1207970467 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5652b73e4780 0x5652b73e9090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:35.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.763+0000 7f77ce59c640 1 -- 172.21.15.16:0/1207970467 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5652b73d6b40 msgr2=0x5652b73e8b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:06:35.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.763+0000 7f77ce59c640 1 --2- 172.21.15.16:0/1207970467 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5652b73d6b40 0x5652b73e8b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:35.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.763+0000 7f77ce59c640 1 -- 172.21.15.16:0/1207970467 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5652b745e150 con 0x5652b73d55d0 2024-09-12T02:06:35.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.764+0000 7f77ced9d640 1 --2- 172.21.15.16:0/1207970467 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5652b73e4780 0x5652b73e9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:06:35.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.764+0000 7f77ce59c640 1 --2- 172.21.15.16:0/1207970467 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5652b73d55d0 0x5652b73e8610 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f77c000d980 tx=0x7f77c000de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45fcb15109c902cd server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:06:35.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.764+0000 7f77b5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:06:35.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.764+0000 7f77b5ffb640 1 -- 172.21.15.16:0/1207970467 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77c001d380 con 0x5652b73d55d0 2024-09-12T02:06:35.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.764+0000 7f77d6c57740 1 -- 172.21.15.16:0/1207970467 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f77c40d2b10 con 0x5652b73d55d0 2024-09-12T02:06:35.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.764+0000 7f77b5ffb640 1 -- 172.21.15.16:0/1207970467 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f77c001d520 con 0x5652b73d55d0 2024-09-12T02:06:35.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.764+0000 7f77b5ffb640 1 -- 172.21.15.16:0/1207970467 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f77c001d720 con 0x5652b73d55d0 2024-09-12T02:06:35.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.764+0000 7f77d6c57740 10 client.16869 fetch_fsmap learned FSMap version 25 2024-09-12T02:06:35.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.764+0000 7f77d6c57740 1 -- 172.21.15.16:0/1207970467 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5652b745f490 con 0x5652b73d55d0 2024-09-12T02:06:35.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.764+0000 7f77b5ffb640 1 -- 172.21.15.16:0/1207970467 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77c40d2b10 con 0x5652b73d55d0 2024-09-12T02:06:35.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.764+0000 7f77b5ffb640 1 -- 172.21.15.16:0/1207970467 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f77c0011c00 con 0x5652b73d55d0 2024-09-12T02:06:35.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.765+0000 7f77d6c57740 10 client.16869 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:06:35.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.765+0000 7f77d6c57740 10 client.16869 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T02:06:35.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.765+0000 7f77d6c57740 20 client.16869 populate_metadata read hostname 'smithi016' 2024-09-12T02:06:35.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.765+0000 7f77d6c57740 1 --2- 172.21.15.16:0/1207970467 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x5652b73f4430 0x5652b7461c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:35.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.765+0000 7f77d6c57740 4 client.16869 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T02:06:35.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.765+0000 7f77d6c57740 1 -- 172.21.15.16:0/1207970467 --> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5652b72bf170 con 0x5652b73f4430 2024-09-12T02:06:35.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.765+0000 7f77cdd9b640 1 --2- 172.21.15.16:0/1207970467 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x5652b73f4430 0x5652b7461c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:35.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.765+0000 7f77cdd9b640 1 --2- 172.21.15.16:0/1207970467 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x5652b73f4430 0x5652b7461c20 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=137df72697e27a46 server_cookie=bf71a0957c07bd47 in_seq=0 out_seq=0 2024-09-12T02:06:35.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.765+0000 7f77b5ffb640 10 client.16869 ms_handle_connect on v2:172.21.15.184:6838/4203347941 2024-09-12T02:06:35.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.766+0000 7f77b5ffb640 1 -- 172.21.15.16:0/1207970467 <== mds.1 v2:172.21.15.184:6838/4203347941 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5652b72bf170 con 0x5652b73f4430 2024-09-12T02:06:35.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.766+0000 7f77b5ffb640 10 client.16869 handle_command_reply: tid=0 2024-09-12T02:06:35.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.766+0000 7f77d6c57740 1 client.16869 shutdown 2024-09-12T02:06:35.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.766+0000 7f77d6c57740 1 -- 172.21.15.16:0/1207970467 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5652b73d55d0 msgr2=0x5652b73e8610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:35.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.766+0000 7f77d6c57740 1 --2- 172.21.15.16:0/1207970467 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5652b73d55d0 0x5652b73e8610 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f77c000d980 tx=0x7f77c000de40 comp rx=0 tx=0).stop 2024-09-12T02:06:35.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.766+0000 7f77d6c57740 1 -- 172.21.15.16:0/1207970467 shutdown_connections 2024-09-12T02:06:35.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.766+0000 7f77d6c57740 1 --2- 172.21.15.16:0/1207970467 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x5652b73f4430 0x5652b7461c20 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:35.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.766+0000 7f77d6c57740 1 --2- 172.21.15.16:0/1207970467 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5652b73e4780 0x5652b73e9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:35.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.766+0000 7f77d6c57740 1 --2- 172.21.15.16:0/1207970467 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5652b73d6b40 0x5652b73e8b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:35.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.766+0000 7f77d6c57740 1 --2- 172.21.15.16:0/1207970467 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5652b73d55d0 0x5652b73e8610 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:35.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.766+0000 7f77d6c57740 1 -- 172.21.15.16:0/1207970467 >> 172.21.15.16:0/1207970467 conn(0x5652b739e310 msgr2=0x5652b73e2cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:06:35.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.766+0000 7f77b5ffb640 0 client.16869 ms_handle_reset on v2:172.21.15.184:6838/4203347941 2024-09-12T02:06:35.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.766+0000 7f77d6c57740 1 -- 172.21.15.16:0/1207970467 shutdown_connections 2024-09-12T02:06:35.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.766+0000 7f77d6c57740 1 -- 172.21.15.16:0/1207970467 wait complete. 2024-09-12T02:06:35.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.766+0000 7f77d6c57740 20 client.16869 trim_cache size 0 max 16384 2024-09-12T02:06:35.782 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T02:06:35.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.766+0000 7f77d487e640 1 -- 172.21.15.16:0/2374693390 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f77b80a2b80 msgr2=0x7f77b80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:35.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.766+0000 7f77d487e640 1 --2- 172.21.15.16:0/2374693390 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f77b80a2b80 0x7f77b80a5040 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f77c8009430 tx=0x7f77c8005c50 comp rx=0 tx=0).stop 2024-09-12T02:06:35.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.766+0000 7f77d487e640 1 -- 172.21.15.16:0/2374693390 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77d0101420 msgr2=0x7f77d0199770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:35.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.766+0000 7f77d487e640 1 --2- 172.21.15.16:0/2374693390 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77d0101420 0x7f77d0199770 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f77c400c920 tx=0x7f77c400cde0 comp rx=0 tx=0).stop 2024-09-12T02:06:35.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.767+0000 7f77d487e640 1 -- 172.21.15.16:0/2374693390 shutdown_connections 2024-09-12T02:06:35.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.767+0000 7f77d487e640 1 --2- 172.21.15.16:0/2374693390 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f77b80a2b80 0x7f77b80a5040 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:35.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.767+0000 7f77d487e640 1 --2- 172.21.15.16:0/2374693390 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f77d0106a80 0x7f77d01a42b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:35.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.767+0000 7f77d487e640 1 --2- 172.21.15.16:0/2374693390 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77d0101420 0x7f77d0199770 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:35.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.767+0000 7f77d487e640 1 --2- 172.21.15.16:0/2374693390 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77d0100a80 0x7f77d0199230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:35.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.767+0000 7f77d487e640 1 -- 172.21.15.16:0/2374693390 >> 172.21.15.16:0/2374693390 conn(0x7f77d00fc7a0 msgr2=0x7f77d00fce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:06:35.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.767+0000 7f77d487e640 1 -- 172.21.15.16:0/2374693390 shutdown_connections 2024-09-12T02:06:35.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:35.767+0000 7f77d487e640 1 -- 172.21.15.16:0/2374693390 wait complete. 2024-09-12T02:06:36.373 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:06:36 smithi016 ceph-mon[30947]: pgmap v475: 97 pgs: 97 active+clean; 2.4 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-09-12T02:06:36.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:06:36 smithi184 ceph-mon[35593]: pgmap v475: 97 pgs: 97 active+clean; 2.4 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-09-12T02:06:36.401 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-12T02:06:36.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:06:36 smithi033 ceph-mon[35332]: pgmap v475: 97 pgs: 97 active+clean; 2.4 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-09-12T02:06:36.658 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:06:37.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.103+0000 7f4aeb0b5640 1 -- 172.21.15.16:0/1086843561 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ae41082e0 msgr2=0x7f4ae410ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:37.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.103+0000 7f4aeb0b5640 1 --2- 172.21.15.16:0/1086843561 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ae41082e0 0x7f4ae410ec50 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f4ad4009cd0 tx=0x7f4ad402a8e0 comp rx=0 tx=0).stop 2024-09-12T02:06:37.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.103+0000 7f4aeb0b5640 1 -- 172.21.15.16:0/1086843561 shutdown_connections 2024-09-12T02:06:37.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.103+0000 7f4aeb0b5640 1 --2- 172.21.15.16:0/1086843561 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ae41082e0 0x7f4ae410ec50 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:37.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.103+0000 7f4aeb0b5640 1 --2- 172.21.15.16:0/1086843561 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ae4103760 0x7f4ae4107ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:37.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.103+0000 7f4aeb0b5640 1 --2- 172.21.15.16:0/1086843561 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4ae4102db0 0x7f4ae4103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:37.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.103+0000 7f4aeb0b5640 1 -- 172.21.15.16:0/1086843561 >> 172.21.15.16:0/1086843561 conn(0x7f4ae40fc7d0 msgr2=0x7f4ae40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:06:37.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.103+0000 7f4aeb0b5640 1 -- 172.21.15.16:0/1086843561 shutdown_connections 2024-09-12T02:06:37.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.103+0000 7f4aeb0b5640 1 -- 172.21.15.16:0/1086843561 wait complete. 2024-09-12T02:06:37.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.104+0000 7f4aeb0b5640 1 Processor -- start 2024-09-12T02:06:37.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.104+0000 7f4aeb0b5640 1 -- start start 2024-09-12T02:06:37.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.104+0000 7f4aeb0b5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ae4102db0 0x7f4ae41985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:37.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.104+0000 7f4aeb0b5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ae4103760 0x7f4ae4198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:37.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.104+0000 7f4aeb0b5640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4ae41082e0 0x7f4ae419ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:37.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.104+0000 7f4aeb0b5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4ae41120b0 con 0x7f4ae4103760 2024-09-12T02:06:37.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.104+0000 7f4aeb0b5640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4ae4111f30 con 0x7f4ae4102db0 2024-09-12T02:06:37.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.104+0000 7f4aeb0b5640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f4ae4112230 con 0x7f4ae41082e0 2024-09-12T02:06:37.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.104+0000 7f4ae3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ae4103760 0x7f4ae4198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:37.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.104+0000 7f4ae3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ae4103760 0x7f4ae4198af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56772/0 (socket says 172.21.15.16:56772) 2024-09-12T02:06:37.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.104+0000 7f4ae3fff640 1 -- 172.21.15.16:0/611075813 learned_addr learned my addr 172.21.15.16:0/611075813 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:06:37.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.104+0000 7f4ae9652640 1 --2- 172.21.15.16:0/611075813 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4ae41082e0 0x7f4ae419ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:37.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.104+0000 7f4ae8e51640 1 --2- 172.21.15.16:0/611075813 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ae4102db0 0x7f4ae41985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:37.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.104+0000 7f4ae3fff640 1 -- 172.21.15.16:0/611075813 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4ae41082e0 msgr2=0x7f4ae419ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:37.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.104+0000 7f4ae3fff640 1 --2- 172.21.15.16:0/611075813 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4ae41082e0 0x7f4ae419ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:37.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.104+0000 7f4ae3fff640 1 -- 172.21.15.16:0/611075813 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ae4102db0 msgr2=0x7f4ae41985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:37.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.104+0000 7f4ae3fff640 1 --2- 172.21.15.16:0/611075813 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ae4102db0 0x7f4ae41985b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:37.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.104+0000 7f4ae3fff640 1 -- 172.21.15.16:0/611075813 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ae419d560 con 0x7f4ae4103760 2024-09-12T02:06:37.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.104+0000 7f4ae9652640 1 --2- 172.21.15.16:0/611075813 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4ae41082e0 0x7f4ae419ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:06:37.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.105+0000 7f4ae3fff640 1 --2- 172.21.15.16:0/611075813 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ae4103760 0x7f4ae4198af0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f4acc00e9f0 tx=0x7f4acc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:06:37.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.105+0000 7f4ae1ffb640 1 -- 172.21.15.16:0/611075813 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4acc00ccc0 con 0x7f4ae4103760 2024-09-12T02:06:37.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.105+0000 7f4ae1ffb640 1 -- 172.21.15.16:0/611075813 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4acc010910 con 0x7f4ae4103760 2024-09-12T02:06:37.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.105+0000 7f4aeb0b5640 1 -- 172.21.15.16:0/611075813 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ae419d850 con 0x7f4ae4103760 2024-09-12T02:06:37.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.105+0000 7f4ae1ffb640 1 -- 172.21.15.16:0/611075813 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4acc018920 con 0x7f4ae4103760 2024-09-12T02:06:37.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.105+0000 7f4aeb0b5640 1 -- 172.21.15.16:0/611075813 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ae41a5130 con 0x7f4ae4103760 2024-09-12T02:06:37.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.107+0000 7f4ae1ffb640 1 -- 172.21.15.16:0/611075813 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f4acc010ac0 con 0x7f4ae4103760 2024-09-12T02:06:37.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.107+0000 7f4aed48e740 1 Processor -- start 2024-09-12T02:06:37.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.108+0000 7f4aed48e740 1 -- start start 2024-09-12T02:06:37.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.108+0000 7f4aed48e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fc34080b40 0x55fc3407f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:37.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.108+0000 7f4aed48e740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fc3407f5d0 0x55fc3407fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:37.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.108+0000 7f4aed48e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fc3408a370 0x55fc3408c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:37.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.108+0000 7f4aed48e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55fc33fe52f0 con 0x55fc34080b40 2024-09-12T02:06:37.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.108+0000 7f4aed48e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55fc33f8ccf0 con 0x55fc3408a370 2024-09-12T02:06:37.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.108+0000 7f4aed48e740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55fc33fe12b0 con 0x55fc3407f5d0 2024-09-12T02:06:37.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.108+0000 7f4ae1ffb640 1 --2- 172.21.15.16:0/611075813 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f4ad00a2c00 0x7f4ad00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:37.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.108+0000 7f4ae9652640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fc34080b40 0x55fc3407f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:37.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.108+0000 7f4ae9652640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fc34080b40 0x55fc3407f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56776/0 (socket says 172.21.15.16:56776) 2024-09-12T02:06:37.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.108+0000 7f4ae9652640 1 -- 172.21.15.16:0/703839570 learned_addr learned my addr 172.21.15.16:0/703839570 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:06:37.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.108+0000 7f4ae9652640 1 --2- 172.21.15.16:0/611075813 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f4ad00a2c00 0x7f4ad00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:37.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.108+0000 7f4ae8e51640 1 --2- 172.21.15.16:0/703839570 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fc3407f5d0 0x55fc3407fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:37.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.108+0000 7f4ae1ffb640 1 -- 172.21.15.16:0/611075813 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f4acc0cf370 con 0x7f4ae4103760 2024-09-12T02:06:37.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.108+0000 7f4ae3fff640 1 --2- 172.21.15.16:0/703839570 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fc3408a370 0x55fc3408c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:37.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.108+0000 7f4ae9652640 1 -- 172.21.15.16:0/703839570 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fc3407f5d0 msgr2=0x55fc3407fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:06:37.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.108+0000 7f4ae9652640 1 --2- 172.21.15.16:0/703839570 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fc3407f5d0 0x55fc3407fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:37.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.108+0000 7f4ae9652640 1 -- 172.21.15.16:0/703839570 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fc3408a370 msgr2=0x55fc3408c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:06:37.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.108+0000 7f4ae9652640 1 --2- 172.21.15.16:0/703839570 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fc3408a370 0x55fc3408c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:37.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.108+0000 7f4ae9652640 1 -- 172.21.15.16:0/703839570 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc33f713a0 con 0x55fc34080b40 2024-09-12T02:06:37.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.109+0000 7f4ae9652640 1 --2- 172.21.15.16:0/703839570 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fc34080b40 0x55fc3407f090 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f4ad4009430 tx=0x7f4ad40317f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20e321cdbbf557f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:06:37.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.109+0000 7f4ae9652640 1 --2- 172.21.15.16:0/611075813 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f4ad00a2c00 0x7f4ad00a50c0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f4ad403b000 tx=0x7f4ad4048000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:06:37.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.109+0000 7f4adaffd640 1 -- 172.21.15.16:0/703839570 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ad4003970 con 0x55fc34080b40 2024-09-12T02:06:37.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.111+0000 7f4adaffd640 1 -- 172.21.15.16:0/703839570 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ad4036040 con 0x55fc34080b40 2024-09-12T02:06:37.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.111+0000 7f4adaffd640 1 -- 172.21.15.16:0/703839570 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ad40305a0 con 0x55fc34080b40 2024-09-12T02:06:37.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.112+0000 7f4aed48e740 1 -- 172.21.15.16:0/703839570 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fc34080b40 msgr2=0x55fc3407f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:37.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.112+0000 7f4aed48e740 1 --2- 172.21.15.16:0/703839570 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fc34080b40 0x55fc3407f090 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f4ad4009430 tx=0x7f4ad40317f0 comp rx=0 tx=0).stop 2024-09-12T02:06:37.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.112+0000 7f4aed48e740 1 -- 172.21.15.16:0/703839570 shutdown_connections 2024-09-12T02:06:37.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.112+0000 7f4aed48e740 1 --2- 172.21.15.16:0/703839570 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fc3408a370 0x55fc3408c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:37.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.112+0000 7f4aed48e740 1 --2- 172.21.15.16:0/703839570 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fc3407f5d0 0x55fc3407fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:37.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.112+0000 7f4aed48e740 1 --2- 172.21.15.16:0/703839570 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fc34080b40 0x55fc3407f090 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:37.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.112+0000 7f4aed48e740 1 -- 172.21.15.16:0/703839570 >> 172.21.15.16:0/703839570 conn(0x55fc34048310 msgr2=0x55fc3407bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:06:37.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.112+0000 7f4aed48e740 1 -- 172.21.15.16:0/703839570 shutdown_connections 2024-09-12T02:06:37.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.112+0000 7f4aed48e740 1 -- 172.21.15.16:0/703839570 wait complete. 2024-09-12T02:06:37.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.112+0000 7f4aed48e740 1 -- start start 2024-09-12T02:06:37.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.113+0000 7f4aed48e740 1 Processor -- start 2024-09-12T02:06:37.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.113+0000 7f4aed48e740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fc3407f5d0 0x55fc34089de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:37.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.113+0000 7f4aed48e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fc34080b40 0x55fc34087090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:37.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.113+0000 7f4aed48e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fc3408a370 0x55fc340877d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:37.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.113+0000 7f4aed48e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55fc3408d010 con 0x55fc34080b40 2024-09-12T02:06:37.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.113+0000 7f4aed48e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55fc33fa0060 con 0x55fc3408a370 2024-09-12T02:06:37.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.113+0000 7f4aed48e740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55fc3408d190 con 0x55fc3407f5d0 2024-09-12T02:06:37.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.113+0000 7f4aed48e740 10 client.0 _ll_register_callbacks cb 0x55fc33cf38c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:06:37.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.113+0000 7f4ae3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fc34080b40 0x55fc34087090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:37.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.113+0000 7f4ae3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fc34080b40 0x55fc34087090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56780/0 (socket says 172.21.15.16:56780) 2024-09-12T02:06:37.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.113+0000 7f4ae3fff640 1 -- 172.21.15.16:0/3333437942 learned_addr learned my addr 172.21.15.16:0/3333437942 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:06:37.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.113+0000 7f4ae9652640 1 --2- 172.21.15.16:0/3333437942 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fc3408a370 0x55fc340877d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:37.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.113+0000 7f4ae8e51640 1 --2- 172.21.15.16:0/3333437942 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fc3407f5d0 0x55fc34089de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:37.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.114+0000 7f4ae3fff640 1 -- 172.21.15.16:0/3333437942 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fc3407f5d0 msgr2=0x55fc34089de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:06:37.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.114+0000 7f4ae3fff640 1 --2- 172.21.15.16:0/3333437942 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fc3407f5d0 0x55fc34089de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:37.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.114+0000 7f4ae3fff640 1 -- 172.21.15.16:0/3333437942 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fc3408a370 msgr2=0x55fc340877d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:06:37.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.114+0000 7f4ae3fff640 1 --2- 172.21.15.16:0/3333437942 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fc3408a370 0x55fc340877d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:37.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.114+0000 7f4ae3fff640 1 -- 172.21.15.16:0/3333437942 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc33d563c0 con 0x55fc34080b40 2024-09-12T02:06:37.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.114+0000 7f4ae3fff640 1 --2- 172.21.15.16:0/3333437942 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fc34080b40 0x55fc34087090 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f4acc098030 tx=0x7f4acc09c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2db4d24ebde3a85f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:06:37.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.114+0000 7f4ac3fff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:06:37.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.114+0000 7f4ac3fff640 1 -- 172.21.15.16:0/3333437942 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4acc09b220 con 0x55fc34080b40 2024-09-12T02:06:37.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.114+0000 7f4ac3fff640 1 -- 172.21.15.16:0/3333437942 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f4acc010430 con 0x55fc34080b40 2024-09-12T02:06:37.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.114+0000 7f4aed48e740 1 -- 172.21.15.16:0/3333437942 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ad4036040 con 0x55fc34080b40 2024-09-12T02:06:37.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.114+0000 7f4ac3fff640 1 -- 172.21.15.16:0/3333437942 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4acc0a4840 con 0x55fc34080b40 2024-09-12T02:06:37.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.114+0000 7f4aed48e740 10 client.16881 fetch_fsmap learned FSMap version 25 2024-09-12T02:06:37.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.114+0000 7f4aed48e740 1 -- 172.21.15.16:0/3333437942 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55fc34098bf0 con 0x55fc34080b40 2024-09-12T02:06:37.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.114+0000 7f4ac3fff640 1 -- 172.21.15.16:0/3333437942 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ad4036040 con 0x55fc34080b40 2024-09-12T02:06:37.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.114+0000 7f4ac3fff640 1 -- 172.21.15.16:0/3333437942 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f4acc09aa90 con 0x55fc34080b40 2024-09-12T02:06:37.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.114+0000 7f4aed48e740 10 client.16881 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:06:37.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.114+0000 7f4aed48e740 10 client.16881 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:06:37.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.115+0000 7f4aed48e740 20 client.16881 populate_metadata read hostname 'smithi016' 2024-09-12T02:06:37.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.115+0000 7f4aed48e740 1 --2- 172.21.15.16:0/3333437942 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55fc3409a0b0 0x55fc3409c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:37.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.115+0000 7f4aed48e740 4 client.16881 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:06:37.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.115+0000 7f4aed48e740 1 -- 172.21.15.16:0/3333437942 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fc33ff2860 con 0x55fc3409a0b0 2024-09-12T02:06:37.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.115+0000 7f4ae8e51640 1 --2- 172.21.15.16:0/3333437942 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55fc3409a0b0 0x55fc3409c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:37.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.115+0000 7f4ae8e51640 1 --2- 172.21.15.16:0/3333437942 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55fc3409a0b0 0x55fc3409c4a0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5cfa93c91a103ab8 server_cookie=2807e6329d150e75 in_seq=0 out_seq=0 2024-09-12T02:06:37.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.115+0000 7f4ac3fff640 10 client.16881 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:06:37.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.116+0000 7f4ac3fff640 1 -- 172.21.15.16:0/3333437942 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fc33ff2860 con 0x55fc3409a0b0 2024-09-12T02:06:37.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.116+0000 7f4ac3fff640 10 client.16881 handle_command_reply: tid=0 2024-09-12T02:06:37.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.116+0000 7f4aed48e740 1 client.16881 shutdown 2024-09-12T02:06:37.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.116+0000 7f4aed48e740 1 -- 172.21.15.16:0/3333437942 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fc34080b40 msgr2=0x55fc34087090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:37.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.116+0000 7f4aed48e740 1 --2- 172.21.15.16:0/3333437942 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fc34080b40 0x55fc34087090 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f4acc098030 tx=0x7f4acc09c2d0 comp rx=0 tx=0).stop 2024-09-12T02:06:37.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.116+0000 7f4aed48e740 1 -- 172.21.15.16:0/3333437942 shutdown_connections 2024-09-12T02:06:37.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.116+0000 7f4aed48e740 1 --2- 172.21.15.16:0/3333437942 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55fc3409a0b0 0x55fc3409c4a0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:37.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.116+0000 7f4aed48e740 1 --2- 172.21.15.16:0/3333437942 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fc3408a370 0x55fc340877d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:37.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.116+0000 7f4aed48e740 1 --2- 172.21.15.16:0/3333437942 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fc34080b40 0x55fc34087090 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:37.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.116+0000 7f4ac3fff640 0 client.16881 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:06:37.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.116+0000 7f4aed48e740 1 --2- 172.21.15.16:0/3333437942 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fc3407f5d0 0x55fc34089de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:37.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.116+0000 7f4aed48e740 1 -- 172.21.15.16:0/3333437942 >> 172.21.15.16:0/3333437942 conn(0x55fc34048310 msgr2=0x55fc34081960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:06:37.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.116+0000 7f4aed48e740 1 -- 172.21.15.16:0/3333437942 shutdown_connections 2024-09-12T02:06:37.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.116+0000 7f4aed48e740 1 -- 172.21.15.16:0/3333437942 wait complete. 2024-09-12T02:06:37.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.116+0000 7f4aed48e740 20 client.16881 trim_cache size 0 max 16384 2024-09-12T02:06:37.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.130+0000 7f4aed48e740 1 Processor -- start 2024-09-12T02:06:37.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.130+0000 7f4aed48e740 1 -- start start 2024-09-12T02:06:37.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.130+0000 7f4aed48e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fc3407f5d0 0x55fc340962b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:37.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.130+0000 7f4aed48e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fc34080b40 0x55fc34085f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:37.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.130+0000 7f4aed48e740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fc3408a370 0x55fc340a1d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:37.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.130+0000 7f4aed48e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4ae41afd20 con 0x55fc34080b40 2024-09-12T02:06:37.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.130+0000 7f4aed48e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4ae41ad5f0 con 0x55fc3407f5d0 2024-09-12T02:06:37.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.130+0000 7f4aed48e740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f4ae41ab440 con 0x55fc3408a370 2024-09-12T02:06:37.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.130+0000 7f4ae3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fc34080b40 0x55fc34085f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:37.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.130+0000 7f4ae3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fc34080b40 0x55fc34085f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56788/0 (socket says 172.21.15.16:56788) 2024-09-12T02:06:37.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.130+0000 7f4ae3fff640 1 -- 172.21.15.16:0/1993802017 learned_addr learned my addr 172.21.15.16:0/1993802017 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:06:37.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.130+0000 7f4ae3fff640 1 -- 172.21.15.16:0/1993802017 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fc3408a370 msgr2=0x55fc340a1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:06:37.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.130+0000 7f4ae3fff640 1 --2- 172.21.15.16:0/1993802017 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fc3408a370 0x55fc340a1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:37.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.130+0000 7f4ae8e51640 1 --2- 172.21.15.16:0/1993802017 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fc3407f5d0 0x55fc340962b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:37.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.130+0000 7f4ae3fff640 1 -- 172.21.15.16:0/1993802017 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fc3407f5d0 msgr2=0x55fc340962b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:06:37.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.130+0000 7f4ae3fff640 1 --2- 172.21.15.16:0/1993802017 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fc3407f5d0 0x55fc340962b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:37.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.130+0000 7f4ae3fff640 1 -- 172.21.15.16:0/1993802017 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc3408d060 con 0x55fc34080b40 2024-09-12T02:06:37.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.131+0000 7f4ae3fff640 1 --2- 172.21.15.16:0/1993802017 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fc34080b40 0x55fc34085f40 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f4acc09bb60 tx=0x7f4acc00cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da3b01afc6eefd37 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:06:37.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.131+0000 7f4ada7fc640 1 -- 172.21.15.16:0/1993802017 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4acc09b020 con 0x55fc34080b40 2024-09-12T02:06:37.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.131+0000 7f4ada7fc640 1 -- 172.21.15.16:0/1993802017 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4acc09b1c0 con 0x55fc34080b40 2024-09-12T02:06:37.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.131+0000 7f4ada7fc640 1 -- 172.21.15.16:0/1993802017 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4acc018d20 con 0x55fc34080b40 2024-09-12T02:06:37.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.131+0000 7f4aed48e740 1 -- 172.21.15.16:0/1993802017 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fc34080b40 msgr2=0x55fc34085f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:37.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.131+0000 7f4aed48e740 1 --2- 172.21.15.16:0/1993802017 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fc34080b40 0x55fc34085f40 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f4acc09bb60 tx=0x7f4acc00cf60 comp rx=0 tx=0).stop 2024-09-12T02:06:37.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.131+0000 7f4aed48e740 1 -- 172.21.15.16:0/1993802017 shutdown_connections 2024-09-12T02:06:37.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.131+0000 7f4aed48e740 1 --2- 172.21.15.16:0/1993802017 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fc3408a370 0x55fc340a1d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:37.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.131+0000 7f4aed48e740 1 --2- 172.21.15.16:0/1993802017 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fc34080b40 0x55fc34085f40 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:37.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.131+0000 7f4aed48e740 1 --2- 172.21.15.16:0/1993802017 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fc3407f5d0 0x55fc340962b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:37.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.131+0000 7f4aed48e740 1 -- 172.21.15.16:0/1993802017 >> 172.21.15.16:0/1993802017 conn(0x55fc34048310 msgr2=0x55fc34085830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:06:37.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.131+0000 7f4aed48e740 1 -- 172.21.15.16:0/1993802017 shutdown_connections 2024-09-12T02:06:37.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.131+0000 7f4aed48e740 1 -- 172.21.15.16:0/1993802017 wait complete. 2024-09-12T02:06:37.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.132+0000 7f4aed48e740 1 -- start start 2024-09-12T02:06:37.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.132+0000 7f4aed48e740 1 Processor -- start 2024-09-12T02:06:37.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.132+0000 7f4aed48e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fc3407f5d0 0x55fc3408eb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:37.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.132+0000 7f4aed48e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fc34080b40 0x55fc3408f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:37.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.132+0000 7f4aed48e740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fc3408a370 0x55fc340fdd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:37.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.132+0000 7f4aed48e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4ae41abab0 con 0x55fc3407f5d0 2024-09-12T02:06:37.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.132+0000 7f4aed48e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4ae41acf20 con 0x55fc34080b40 2024-09-12T02:06:37.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.132+0000 7f4aed48e740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f4ae41ac180 con 0x55fc3408a370 2024-09-12T02:06:37.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.132+0000 7f4aed48e740 10 client.0 _ll_register_callbacks cb 0x7f4acc097e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:06:37.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.132+0000 7f4ae8e51640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fc3407f5d0 0x55fc3408eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:37.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.132+0000 7f4ae8e51640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fc3407f5d0 0x55fc3408eb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56794/0 (socket says 172.21.15.16:56794) 2024-09-12T02:06:37.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.132+0000 7f4ae8e51640 1 -- 172.21.15.16:0/245483768 learned_addr learned my addr 172.21.15.16:0/245483768 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:06:37.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.132+0000 7f4ae3fff640 1 --2- 172.21.15.16:0/245483768 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fc34080b40 0x55fc3408f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:37.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.133+0000 7f4ae8e51640 1 -- 172.21.15.16:0/245483768 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fc3408a370 msgr2=0x55fc340fdd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:06:37.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.133+0000 7f4ae8e51640 1 --2- 172.21.15.16:0/245483768 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fc3408a370 0x55fc340fdd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:37.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.133+0000 7f4ae8e51640 1 -- 172.21.15.16:0/245483768 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fc34080b40 msgr2=0x55fc3408f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:06:37.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.133+0000 7f4ae8e51640 1 --2- 172.21.15.16:0/245483768 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fc34080b40 0x55fc3408f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:37.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.133+0000 7f4ae8e51640 1 -- 172.21.15.16:0/245483768 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc340fe3f0 con 0x55fc3407f5d0 2024-09-12T02:06:37.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.133+0000 7f4ae8e51640 1 --2- 172.21.15.16:0/245483768 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fc3407f5d0 0x55fc3408eb20 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x55fc3409ccb0 tx=0x7f4adc012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92bf8cf6f0838678 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:06:37.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.133+0000 7f4ac3fff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:06:37.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.133+0000 7f4ac3fff640 1 -- 172.21.15.16:0/245483768 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4adc004440 con 0x55fc3407f5d0 2024-09-12T02:06:37.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.133+0000 7f4ac3fff640 1 -- 172.21.15.16:0/245483768 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f4adc0035f0 con 0x55fc3407f5d0 2024-09-12T02:06:37.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.133+0000 7f4aed48e740 1 -- 172.21.15.16:0/245483768 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4acc09b1c0 con 0x55fc3407f5d0 2024-09-12T02:06:37.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.133+0000 7f4ac3fff640 1 -- 172.21.15.16:0/245483768 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4adc0037f0 con 0x55fc3407f5d0 2024-09-12T02:06:37.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.133+0000 7f4aed48e740 10 client.16887 fetch_fsmap learned FSMap version 25 2024-09-12T02:06:37.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.133+0000 7f4aed48e740 1 -- 172.21.15.16:0/245483768 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55fc341057d0 con 0x55fc3407f5d0 2024-09-12T02:06:37.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.133+0000 7f4ac3fff640 1 -- 172.21.15.16:0/245483768 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4acc09b1c0 con 0x55fc3407f5d0 2024-09-12T02:06:37.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.133+0000 7f4ac3fff640 1 -- 172.21.15.16:0/245483768 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f4adc027b10 con 0x55fc3407f5d0 2024-09-12T02:06:37.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.134+0000 7f4aed48e740 10 client.16887 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:06:37.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.134+0000 7f4aed48e740 10 client.16887 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:06:37.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.134+0000 7f4aed48e740 20 client.16887 populate_metadata read hostname 'smithi016' 2024-09-12T02:06:37.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.134+0000 7f4aed48e740 1 --2- 172.21.15.16:0/245483768 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55fc34105f70 0x55fc34108390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:37.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.134+0000 7f4aed48e740 4 client.16887 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T02:06:37.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.134+0000 7f4aed48e740 1 -- 172.21.15.16:0/245483768 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fc33fecd70 con 0x55fc34105f70 2024-09-12T02:06:37.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.134+0000 7f4ae3fff640 1 --2- 172.21.15.16:0/245483768 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55fc34105f70 0x55fc34108390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:37.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.134+0000 7f4ac3fff640 10 client.16887 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:06:37.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.134+0000 7f4ae3fff640 1 --2- 172.21.15.16:0/245483768 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55fc34105f70 0x55fc34108390 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de7276bcdfdc7e4c server_cookie=ba6c99a276055ce3 in_seq=0 out_seq=0 2024-09-12T02:06:37.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.135+0000 7f4ac3fff640 1 -- 172.21.15.16:0/245483768 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fc33fecd70 con 0x55fc34105f70 2024-09-12T02:06:37.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.135+0000 7f4ac3fff640 10 client.16887 handle_command_reply: tid=0 2024-09-12T02:06:37.151 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T02:06:37.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.135+0000 7f4aed48e740 1 client.16887 shutdown 2024-09-12T02:06:37.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.135+0000 7f4aed48e740 1 -- 172.21.15.16:0/245483768 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fc3407f5d0 msgr2=0x55fc3408eb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:37.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.135+0000 7f4aed48e740 1 --2- 172.21.15.16:0/245483768 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fc3407f5d0 0x55fc3408eb20 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x55fc3409ccb0 tx=0x7f4adc012820 comp rx=0 tx=0).stop 2024-09-12T02:06:37.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.135+0000 7f4aed48e740 1 -- 172.21.15.16:0/245483768 shutdown_connections 2024-09-12T02:06:37.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.135+0000 7f4aed48e740 1 --2- 172.21.15.16:0/245483768 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55fc34105f70 0x55fc34108390 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:37.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.135+0000 7f4aed48e740 1 --2- 172.21.15.16:0/245483768 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fc3408a370 0x55fc340fdd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:37.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.135+0000 7f4aed48e740 1 --2- 172.21.15.16:0/245483768 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fc34080b40 0x55fc3408f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:37.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.135+0000 7f4ac3fff640 0 client.16887 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:06:37.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.135+0000 7f4aed48e740 1 --2- 172.21.15.16:0/245483768 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fc3407f5d0 0x55fc3408eb20 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:37.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.135+0000 7f4aed48e740 1 -- 172.21.15.16:0/245483768 >> 172.21.15.16:0/245483768 conn(0x55fc34048310 msgr2=0x55fc34081930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:06:37.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.135+0000 7f4aed48e740 1 -- 172.21.15.16:0/245483768 shutdown_connections 2024-09-12T02:06:37.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.135+0000 7f4aed48e740 1 -- 172.21.15.16:0/245483768 wait complete. 2024-09-12T02:06:37.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.135+0000 7f4aed48e740 20 client.16887 trim_cache size 0 max 16384 2024-09-12T02:06:37.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.136+0000 7f4aeb0b5640 1 -- 172.21.15.16:0/611075813 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f4ad00a2c00 msgr2=0x7f4ad00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:37.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.136+0000 7f4aeb0b5640 1 --2- 172.21.15.16:0/611075813 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f4ad00a2c00 0x7f4ad00a50c0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f4ad403b000 tx=0x7f4ad4048000 comp rx=0 tx=0).stop 2024-09-12T02:06:37.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.136+0000 7f4aeb0b5640 1 -- 172.21.15.16:0/611075813 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ae4103760 msgr2=0x7f4ae4198af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:37.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.136+0000 7f4aeb0b5640 1 --2- 172.21.15.16:0/611075813 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ae4103760 0x7f4ae4198af0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f4acc00e9f0 tx=0x7f4acc00eeb0 comp rx=0 tx=0).stop 2024-09-12T02:06:37.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.136+0000 7f4aeb0b5640 1 -- 172.21.15.16:0/611075813 shutdown_connections 2024-09-12T02:06:37.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.136+0000 7f4aeb0b5640 1 --2- 172.21.15.16:0/611075813 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f4ad00a2c00 0x7f4ad00a50c0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:37.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.136+0000 7f4aeb0b5640 1 --2- 172.21.15.16:0/611075813 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4ae41082e0 0x7f4ae419ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:37.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.136+0000 7f4aeb0b5640 1 --2- 172.21.15.16:0/611075813 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ae4103760 0x7f4ae4198af0 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:37.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.136+0000 7f4aeb0b5640 1 --2- 172.21.15.16:0/611075813 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ae4102db0 0x7f4ae41985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:37.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.136+0000 7f4aeb0b5640 1 -- 172.21.15.16:0/611075813 >> 172.21.15.16:0/611075813 conn(0x7f4ae40fc7d0 msgr2=0x7f4ae4106390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:06:37.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.136+0000 7f4aeb0b5640 1 -- 172.21.15.16:0/611075813 shutdown_connections 2024-09-12T02:06:37.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:37.136+0000 7f4aeb0b5640 1 -- 172.21.15.16:0/611075813 wait complete. 2024-09-12T02:06:38.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:06:38 smithi184 ceph-mon[35593]: pgmap v476: 97 pgs: 97 active+clean; 2.4 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-09-12T02:06:38.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:06:38 smithi033 ceph-mon[35332]: pgmap v476: 97 pgs: 97 active+clean; 2.4 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-09-12T02:06:38.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:06:38 smithi016 ceph-mon[30947]: pgmap v476: 97 pgs: 97 active+clean; 2.4 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-09-12T02:06:38.743 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-12T02:06:38.744 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-12T02:06:39.007 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:06:39.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.461+0000 7f5e2e338640 1 -- 172.21.15.16:0/3036153688 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e2810c170 msgr2=0x7f5e2810e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:39.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.461+0000 7f5e2e338640 1 --2- 172.21.15.16:0/3036153688 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e2810c170 0x7f5e2810e610 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f5e20009ca0 tx=0x7f5e2002a950 comp rx=0 tx=0).stop 2024-09-12T02:06:39.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.461+0000 7f5e2e338640 1 -- 172.21.15.16:0/3036153688 shutdown_connections 2024-09-12T02:06:39.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.461+0000 7f5e2e338640 1 --2- 172.21.15.16:0/3036153688 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e2810c170 0x7f5e2810e610 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:39.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.461+0000 7f5e2e338640 1 --2- 172.21.15.16:0/3036153688 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5e28103660 0x7f5e2810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:39.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.461+0000 7f5e2e338640 1 --2- 172.21.15.16:0/3036153688 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5e28102d40 0x7f5e28103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:39.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.461+0000 7f5e2e338640 1 -- 172.21.15.16:0/3036153688 >> 172.21.15.16:0/3036153688 conn(0x7f5e280fc7a0 msgr2=0x7f5e280febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:06:39.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.461+0000 7f5e2e338640 1 -- 172.21.15.16:0/3036153688 shutdown_connections 2024-09-12T02:06:39.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.461+0000 7f5e2e338640 1 -- 172.21.15.16:0/3036153688 wait complete. 2024-09-12T02:06:39.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.462+0000 7f5e2e338640 1 Processor -- start 2024-09-12T02:06:39.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.462+0000 7f5e2e338640 1 -- start start 2024-09-12T02:06:39.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.462+0000 7f5e2e338640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5e28102d40 0x7f5e281009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:39.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.462+0000 7f5e2e338640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e28103660 0x7f5e28100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:39.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.462+0000 7f5e2e338640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5e2810c170 0x7f5e28101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:39.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.462+0000 7f5e2e338640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5e28111fb0 con 0x7f5e28103660 2024-09-12T02:06:39.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.462+0000 7f5e2e338640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5e28111e30 con 0x7f5e28102d40 2024-09-12T02:06:39.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.462+0000 7f5e2e338640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5e28112130 con 0x7f5e2810c170 2024-09-12T02:06:39.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.463+0000 7f5e277fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e28103660 0x7f5e28100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:39.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.463+0000 7f5e277fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e28103660 0x7f5e28100ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56824/0 (socket says 172.21.15.16:56824) 2024-09-12T02:06:39.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.463+0000 7f5e277fe640 1 -- 172.21.15.16:0/737059011 learned_addr learned my addr 172.21.15.16:0/737059011 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:06:39.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.463+0000 7f5e2c8d5640 1 --2- 172.21.15.16:0/737059011 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5e2810c170 0x7f5e28101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:39.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.463+0000 7f5e27fff640 1 --2- 172.21.15.16:0/737059011 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5e28102d40 0x7f5e281009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:39.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.463+0000 7f5e277fe640 1 -- 172.21.15.16:0/737059011 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5e2810c170 msgr2=0x7f5e28101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:39.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.463+0000 7f5e277fe640 1 --2- 172.21.15.16:0/737059011 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5e2810c170 0x7f5e28101420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:39.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.463+0000 7f5e277fe640 1 -- 172.21.15.16:0/737059011 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5e28102d40 msgr2=0x7f5e281009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:39.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.463+0000 7f5e277fe640 1 --2- 172.21.15.16:0/737059011 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5e28102d40 0x7f5e281009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:39.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.463+0000 7f5e277fe640 1 -- 172.21.15.16:0/737059011 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e281a6890 con 0x7f5e28103660 2024-09-12T02:06:39.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.463+0000 7f5e277fe640 1 --2- 172.21.15.16:0/737059011 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e28103660 0x7f5e28100ee0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c00b3f0 tx=0x7f5e1c00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:06:39.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.463+0000 7f5e257fa640 1 -- 172.21.15.16:0/737059011 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e1c015020 con 0x7f5e28103660 2024-09-12T02:06:39.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.463+0000 7f5e257fa640 1 -- 172.21.15.16:0/737059011 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e1c011d10 con 0x7f5e28103660 2024-09-12T02:06:39.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.463+0000 7f5e2e338640 1 -- 172.21.15.16:0/737059011 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e281a6b80 con 0x7f5e28103660 2024-09-12T02:06:39.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.463+0000 7f5e257fa640 1 -- 172.21.15.16:0/737059011 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e1c00f2e0 con 0x7f5e28103660 2024-09-12T02:06:39.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.463+0000 7f5e2e338640 1 -- 172.21.15.16:0/737059011 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e281a7070 con 0x7f5e28103660 2024-09-12T02:06:39.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.466+0000 7f5e257fa640 1 -- 172.21.15.16:0/737059011 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5e1c018020 con 0x7f5e28103660 2024-09-12T02:06:39.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.466+0000 7f5e30711740 1 Processor -- start 2024-09-12T02:06:39.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.466+0000 7f5e30711740 1 -- start start 2024-09-12T02:06:39.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.467+0000 7f5e30711740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d0b7873b60 0x55d0b7873f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:39.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.467+0000 7f5e30711740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d0b7874480 0x55d0b787e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:39.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.467+0000 7f5e30711740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d0b787ed40 0x55d0b7881130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:39.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.467+0000 7f5e30711740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55d0b77d9280 con 0x55d0b7874480 2024-09-12T02:06:39.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.467+0000 7f5e30711740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55d0b7780b10 con 0x55d0b7873b60 2024-09-12T02:06:39.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.467+0000 7f5e30711740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55d0b77d5240 con 0x55d0b787ed40 2024-09-12T02:06:39.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.467+0000 7f5e27fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d0b7874480 0x55d0b787e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:39.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.467+0000 7f5e27fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d0b7874480 0x55d0b787e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56836/0 (socket says 172.21.15.16:56836) 2024-09-12T02:06:39.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.467+0000 7f5e27fff640 1 -- 172.21.15.16:0/2362492732 learned_addr learned my addr 172.21.15.16:0/2362492732 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:06:39.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.467+0000 7f5e2c8d5640 1 --2- 172.21.15.16:0/2362492732 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d0b7873b60 0x55d0b7873f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:39.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.467+0000 7f5e257fa640 1 --2- 172.21.15.16:0/737059011 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f5e100a2b80 0x7f5e100a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:39.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.467+0000 7f5e277fe640 1 --2- 172.21.15.16:0/2362492732 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d0b787ed40 0x55d0b7881130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:39.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.467+0000 7f5e2c8d5640 1 --2- 172.21.15.16:0/737059011 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f5e100a2b80 0x7f5e100a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:39.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.467+0000 7f5e27fff640 1 -- 172.21.15.16:0/2362492732 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d0b787ed40 msgr2=0x55d0b7881130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:06:39.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.467+0000 7f5e27fff640 1 --2- 172.21.15.16:0/2362492732 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d0b787ed40 0x55d0b7881130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:39.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.467+0000 7f5e27fff640 1 -- 172.21.15.16:0/2362492732 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d0b7873b60 msgr2=0x55d0b7873f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:06:39.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.467+0000 7f5e27fff640 1 --2- 172.21.15.16:0/2362492732 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d0b7873b60 0x55d0b7873f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:39.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.467+0000 7f5e27fff640 1 -- 172.21.15.16:0/2362492732 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0b778d1f0 con 0x55d0b7874480 2024-09-12T02:06:39.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.467+0000 7f5e257fa640 1 -- 172.21.15.16:0/737059011 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f5e1c0ce850 con 0x7f5e28103660 2024-09-12T02:06:39.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.467+0000 7f5e27fff640 1 --2- 172.21.15.16:0/2362492732 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d0b7874480 0x55d0b787e800 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f5e1800cc60 tx=0x7f5e1800e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdcf94d817306b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:06:39.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.468+0000 7f5e0e7fc640 1 -- 172.21.15.16:0/2362492732 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e18014070 con 0x55d0b7874480 2024-09-12T02:06:39.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.468+0000 7f5e2c8d5640 1 --2- 172.21.15.16:0/737059011 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f5e100a2b80 0x7f5e100a5040 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f5e20030040 tx=0x7f5e2002a950 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:06:39.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.468+0000 7f5e0e7fc640 1 -- 172.21.15.16:0/2362492732 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e180047a0 con 0x55d0b7874480 2024-09-12T02:06:39.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.468+0000 7f5e0e7fc640 1 -- 172.21.15.16:0/2362492732 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e18010790 con 0x55d0b7874480 2024-09-12T02:06:39.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.468+0000 7f5e30711740 1 -- 172.21.15.16:0/2362492732 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d0b7874480 msgr2=0x55d0b787e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:39.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.468+0000 7f5e30711740 1 --2- 172.21.15.16:0/2362492732 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d0b7874480 0x55d0b787e800 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f5e1800cc60 tx=0x7f5e1800e720 comp rx=0 tx=0).stop 2024-09-12T02:06:39.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.468+0000 7f5e30711740 1 -- 172.21.15.16:0/2362492732 shutdown_connections 2024-09-12T02:06:39.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.468+0000 7f5e30711740 1 --2- 172.21.15.16:0/2362492732 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d0b787ed40 0x55d0b7881130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:39.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.468+0000 7f5e30711740 1 --2- 172.21.15.16:0/2362492732 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d0b7874480 0x55d0b787e800 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:39.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.468+0000 7f5e30711740 1 --2- 172.21.15.16:0/2362492732 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d0b7873b60 0x55d0b7873f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:39.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.468+0000 7f5e30711740 1 -- 172.21.15.16:0/2362492732 >> 172.21.15.16:0/2362492732 conn(0x55d0b78700f0 msgr2=0x55d0b78704d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:06:39.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.469+0000 7f5e30711740 1 -- 172.21.15.16:0/2362492732 shutdown_connections 2024-09-12T02:06:39.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.469+0000 7f5e30711740 1 -- 172.21.15.16:0/2362492732 wait complete. 2024-09-12T02:06:39.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.469+0000 7f5e30711740 1 -- start start 2024-09-12T02:06:39.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.469+0000 7f5e30711740 1 Processor -- start 2024-09-12T02:06:39.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.469+0000 7f5e30711740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d0b7873b60 0x55d0b787bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:39.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.469+0000 7f5e30711740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d0b7874480 0x55d0b787c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:39.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.469+0000 7f5e30711740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d0b787ed40 0x55d0b787c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:39.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.469+0000 7f5e30711740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55d0b7881910 con 0x55d0b7874480 2024-09-12T02:06:39.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.469+0000 7f5e30711740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55d0b7793f10 con 0x55d0b7873b60 2024-09-12T02:06:39.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.469+0000 7f5e30711740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55d0b7881a90 con 0x55d0b787ed40 2024-09-12T02:06:39.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.469+0000 7f5e30711740 10 client.0 _ll_register_callbacks cb 0x55d0b775ca40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:06:39.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.470+0000 7f5e27fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d0b7873b60 0x55d0b787bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:39.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.470+0000 7f5e277fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d0b7874480 0x55d0b787c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:39.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.470+0000 7f5e27fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d0b7873b60 0x55d0b787bbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.16:47108/0 (socket says 172.21.15.16:47108) 2024-09-12T02:06:39.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.470+0000 7f5e27fff640 1 -- 172.21.15.16:0/3510332064 learned_addr learned my addr 172.21.15.16:0/3510332064 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:06:39.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.470+0000 7f5e2c8d5640 1 --2- 172.21.15.16:0/3510332064 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d0b787ed40 0x55d0b787c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:39.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.470+0000 7f5e277fe640 1 -- 172.21.15.16:0/3510332064 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d0b787ed40 msgr2=0x55d0b787c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:06:39.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.470+0000 7f5e277fe640 1 --2- 172.21.15.16:0/3510332064 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d0b787ed40 0x55d0b787c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:39.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.470+0000 7f5e277fe640 1 -- 172.21.15.16:0/3510332064 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d0b7873b60 msgr2=0x55d0b787bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:06:39.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.470+0000 7f5e277fe640 1 --2- 172.21.15.16:0/3510332064 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d0b7873b60 0x55d0b787bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:39.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.470+0000 7f5e277fe640 1 -- 172.21.15.16:0/3510332064 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0b754a6a0 con 0x55d0b7874480 2024-09-12T02:06:39.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.470+0000 7f5e27fff640 1 --2- 172.21.15.16:0/3510332064 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d0b7873b60 0x55d0b787bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:06:39.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.470+0000 7f5e277fe640 1 --2- 172.21.15.16:0/3510332064 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d0b7874480 0x55d0b787c110 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x55d0b7871470 tx=0x7f5e1c09ab10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c9e2046303acd7b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:06:39.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.470+0000 7f5e077fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:06:39.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.470+0000 7f5e077fe640 1 -- 172.21.15.16:0/3510332064 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e1c09b0f0 con 0x55d0b7874480 2024-09-12T02:06:39.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.470+0000 7f5e077fe640 1 -- 172.21.15.16:0/3510332064 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f5e1c0bb0f0 con 0x55d0b7874480 2024-09-12T02:06:39.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.470+0000 7f5e30711740 1 -- 172.21.15.16:0/3510332064 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5e180047a0 con 0x55d0b7874480 2024-09-12T02:06:39.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.470+0000 7f5e077fe640 1 -- 172.21.15.16:0/3510332064 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5e1c09f4a0 con 0x55d0b7874480 2024-09-12T02:06:39.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.470+0000 7f5e30711740 10 client.16899 fetch_fsmap learned FSMap version 25 2024-09-12T02:06:39.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.471+0000 7f5e30711740 1 -- 172.21.15.16:0/3510332064 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d0b788d4f0 con 0x55d0b7874480 2024-09-12T02:06:39.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.471+0000 7f5e077fe640 1 -- 172.21.15.16:0/3510332064 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e180047a0 con 0x55d0b7874480 2024-09-12T02:06:39.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.471+0000 7f5e077fe640 1 -- 172.21.15.16:0/3510332064 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f5e1c0b5020 con 0x55d0b7874480 2024-09-12T02:06:39.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.471+0000 7f5e30711740 10 client.16899 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:06:39.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.471+0000 7f5e30711740 10 client.16899 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:06:39.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.471+0000 7f5e30711740 20 client.16899 populate_metadata read hostname 'smithi016' 2024-09-12T02:06:39.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.471+0000 7f5e30711740 1 --2- 172.21.15.16:0/3510332064 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55d0b788e9b0 0x55d0b7890dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:39.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.471+0000 7f5e30711740 4 client.16899 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:06:39.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.471+0000 7f5e30711740 1 -- 172.21.15.16:0/3510332064 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d0b77e67f0 con 0x55d0b788e9b0 2024-09-12T02:06:39.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.471+0000 7f5e27fff640 1 --2- 172.21.15.16:0/3510332064 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55d0b788e9b0 0x55d0b7890dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:39.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.472+0000 7f5e27fff640 1 --2- 172.21.15.16:0/3510332064 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55d0b788e9b0 0x55d0b7890dc0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d35c2a14c230525d server_cookie=102c92617bca86ed in_seq=0 out_seq=0 2024-09-12T02:06:39.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.472+0000 7f5e077fe640 10 client.16899 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:06:39.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.472+0000 7f5e077fe640 1 -- 172.21.15.16:0/3510332064 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d0b77e67f0 con 0x55d0b788e9b0 2024-09-12T02:06:39.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.472+0000 7f5e077fe640 10 client.16899 handle_command_reply: tid=0 2024-09-12T02:06:39.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.472+0000 7f5e30711740 1 client.16899 shutdown 2024-09-12T02:06:39.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.473+0000 7f5e30711740 1 -- 172.21.15.16:0/3510332064 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d0b7874480 msgr2=0x55d0b787c110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:39.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.473+0000 7f5e30711740 1 --2- 172.21.15.16:0/3510332064 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d0b7874480 0x55d0b787c110 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x55d0b7871470 tx=0x7f5e1c09ab10 comp rx=0 tx=0).stop 2024-09-12T02:06:39.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.473+0000 7f5e30711740 1 -- 172.21.15.16:0/3510332064 shutdown_connections 2024-09-12T02:06:39.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.473+0000 7f5e30711740 1 --2- 172.21.15.16:0/3510332064 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55d0b788e9b0 0x55d0b7890dc0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:39.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.473+0000 7f5e30711740 1 --2- 172.21.15.16:0/3510332064 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d0b787ed40 0x55d0b787c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:39.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.473+0000 7f5e30711740 1 --2- 172.21.15.16:0/3510332064 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d0b7874480 0x55d0b787c110 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:39.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.473+0000 7f5e30711740 1 --2- 172.21.15.16:0/3510332064 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d0b7873b60 0x55d0b787bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:39.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.473+0000 7f5e077fe640 0 client.16899 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:06:39.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.473+0000 7f5e30711740 1 -- 172.21.15.16:0/3510332064 >> 172.21.15.16:0/3510332064 conn(0x55d0b78700f0 msgr2=0x55d0b7875e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:06:39.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.473+0000 7f5e30711740 1 -- 172.21.15.16:0/3510332064 shutdown_connections 2024-09-12T02:06:39.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.473+0000 7f5e30711740 1 -- 172.21.15.16:0/3510332064 wait complete. 2024-09-12T02:06:39.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.473+0000 7f5e30711740 20 client.16899 trim_cache size 0 max 16384 2024-09-12T02:06:39.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.490+0000 7f5e30711740 1 Processor -- start 2024-09-12T02:06:39.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.490+0000 7f5e30711740 1 -- start start 2024-09-12T02:06:39.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.490+0000 7f5e30711740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d0b7873b60 0x55d0b7875d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:39.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.490+0000 7f5e30711740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d0b7874480 0x55d0b7896660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:39.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.490+0000 7f5e30711740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d0b787ed40 0x55d0b7896ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:39.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.490+0000 7f5e30711740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5e281b1e50 con 0x55d0b787ed40 2024-09-12T02:06:39.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.490+0000 7f5e30711740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5e281af720 con 0x55d0b7873b60 2024-09-12T02:06:39.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.490+0000 7f5e30711740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5e281ad570 con 0x55d0b7874480 2024-09-12T02:06:39.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.491+0000 7f5e2c8d5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d0b787ed40 0x55d0b7896ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:39.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.491+0000 7f5e2c8d5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d0b787ed40 0x55d0b7896ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56850/0 (socket says 172.21.15.16:56850) 2024-09-12T02:06:39.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.491+0000 7f5e2c8d5640 1 -- 172.21.15.16:0/2459146744 learned_addr learned my addr 172.21.15.16:0/2459146744 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:06:39.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.491+0000 7f5e27fff640 1 --2- 172.21.15.16:0/2459146744 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d0b7873b60 0x55d0b7875d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:39.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.491+0000 7f5e277fe640 1 --2- 172.21.15.16:0/2459146744 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d0b7874480 0x55d0b7896660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:39.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.491+0000 7f5e2c8d5640 1 -- 172.21.15.16:0/2459146744 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d0b7874480 msgr2=0x55d0b7896660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:06:39.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.491+0000 7f5e2c8d5640 1 --2- 172.21.15.16:0/2459146744 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d0b7874480 0x55d0b7896660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:39.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.491+0000 7f5e2c8d5640 1 -- 172.21.15.16:0/2459146744 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d0b7873b60 msgr2=0x55d0b7875d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:06:39.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.491+0000 7f5e2c8d5640 1 --2- 172.21.15.16:0/2459146744 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d0b7873b60 0x55d0b7875d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:39.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.491+0000 7f5e2c8d5640 1 -- 172.21.15.16:0/2459146744 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0b786ff20 con 0x55d0b787ed40 2024-09-12T02:06:39.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.491+0000 7f5e27fff640 1 --2- 172.21.15.16:0/2459146744 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d0b7873b60 0x55d0b7875d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:06:39.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.491+0000 7f5e2c8d5640 1 --2- 172.21.15.16:0/2459146744 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d0b787ed40 0x55d0b7896ba0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f5e20035000 tx=0x7f5e20005fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30fba15e6c50b65c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:06:39.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.491+0000 7f5e0dffb640 1 -- 172.21.15.16:0/2459146744 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e20043050 con 0x55d0b787ed40 2024-09-12T02:06:39.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.491+0000 7f5e0dffb640 1 -- 172.21.15.16:0/2459146744 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e20044040 con 0x55d0b787ed40 2024-09-12T02:06:39.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.491+0000 7f5e0dffb640 1 -- 172.21.15.16:0/2459146744 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e20034040 con 0x55d0b787ed40 2024-09-12T02:06:39.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.492+0000 7f5e30711740 1 -- 172.21.15.16:0/2459146744 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d0b787ed40 msgr2=0x55d0b7896ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:39.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.492+0000 7f5e30711740 1 --2- 172.21.15.16:0/2459146744 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d0b787ed40 0x55d0b7896ba0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f5e20035000 tx=0x7f5e20005fa0 comp rx=0 tx=0).stop 2024-09-12T02:06:39.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.492+0000 7f5e30711740 1 -- 172.21.15.16:0/2459146744 shutdown_connections 2024-09-12T02:06:39.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.492+0000 7f5e30711740 1 --2- 172.21.15.16:0/2459146744 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d0b787ed40 0x55d0b7896ba0 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:39.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.492+0000 7f5e30711740 1 --2- 172.21.15.16:0/2459146744 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d0b7874480 0x55d0b7896660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:39.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.492+0000 7f5e30711740 1 --2- 172.21.15.16:0/2459146744 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d0b7873b60 0x55d0b7875d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:39.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.492+0000 7f5e30711740 1 -- 172.21.15.16:0/2459146744 >> 172.21.15.16:0/2459146744 conn(0x55d0b78700f0 msgr2=0x55d0b7875600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:06:39.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.492+0000 7f5e30711740 1 -- 172.21.15.16:0/2459146744 shutdown_connections 2024-09-12T02:06:39.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.492+0000 7f5e30711740 1 -- 172.21.15.16:0/2459146744 wait complete. 2024-09-12T02:06:39.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.492+0000 7f5e30711740 1 -- start start 2024-09-12T02:06:39.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.493+0000 7f5e30711740 1 Processor -- start 2024-09-12T02:06:39.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.493+0000 7f5e30711740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d0b7873b60 0x55d0b7886d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:39.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.493+0000 7f5e30711740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d0b7874480 0x55d0b7887250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:39.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.493+0000 7f5e30711740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d0b787ed40 0x55d0b7883ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:39.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.493+0000 7f5e30711740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5e281adbe0 con 0x55d0b7874480 2024-09-12T02:06:39.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.493+0000 7f5e30711740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5e281af050 con 0x55d0b7873b60 2024-09-12T02:06:39.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.493+0000 7f5e30711740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5e281ae2b0 con 0x55d0b787ed40 2024-09-12T02:06:39.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.493+0000 7f5e30711740 10 client.0 _ll_register_callbacks cb 0x7f5e1c00b340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:06:39.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.493+0000 7f5e277fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d0b7874480 0x55d0b7887250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:39.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.493+0000 7f5e277fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d0b7874480 0x55d0b7887250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56864/0 (socket says 172.21.15.16:56864) 2024-09-12T02:06:39.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.493+0000 7f5e277fe640 1 -- 172.21.15.16:0/3346146597 learned_addr learned my addr 172.21.15.16:0/3346146597 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:06:39.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.493+0000 7f5e27fff640 1 --2- 172.21.15.16:0/3346146597 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d0b7873b60 0x55d0b7886d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:39.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.493+0000 7f5e2c8d5640 1 --2- 172.21.15.16:0/3346146597 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d0b787ed40 0x55d0b7883ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:39.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.493+0000 7f5e277fe640 1 -- 172.21.15.16:0/3346146597 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d0b787ed40 msgr2=0x55d0b7883ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:06:39.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.493+0000 7f5e277fe640 1 --2- 172.21.15.16:0/3346146597 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d0b787ed40 0x55d0b7883ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:39.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.493+0000 7f5e277fe640 1 -- 172.21.15.16:0/3346146597 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d0b7873b60 msgr2=0x55d0b7886d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:06:39.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.493+0000 7f5e277fe640 1 --2- 172.21.15.16:0/3346146597 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d0b7873b60 0x55d0b7886d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:39.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.493+0000 7f5e277fe640 1 -- 172.21.15.16:0/3346146597 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0b7884220 con 0x55d0b7874480 2024-09-12T02:06:39.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.493+0000 7f5e27fff640 1 --2- 172.21.15.16:0/3346146597 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d0b7873b60 0x55d0b7886d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:06:39.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.494+0000 7f5e277fe640 1 --2- 172.21.15.16:0/3346146597 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d0b7874480 0x55d0b7887250 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c097ac0 tx=0x7f5e1c09a9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c70182654c2cbcd0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:06:39.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.494+0000 7f5e077fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:06:39.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.494+0000 7f5e077fe640 1 -- 172.21.15.16:0/3346146597 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e1c09aec0 con 0x55d0b7874480 2024-09-12T02:06:39.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.494+0000 7f5e077fe640 1 -- 172.21.15.16:0/3346146597 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f5e1c011830 con 0x55d0b7874480 2024-09-12T02:06:39.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.494+0000 7f5e30711740 1 -- 172.21.15.16:0/3346146597 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5e20044040 con 0x55d0b7874480 2024-09-12T02:06:39.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.494+0000 7f5e077fe640 1 -- 172.21.15.16:0/3346146597 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5e1c097390 con 0x55d0b7874480 2024-09-12T02:06:39.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.494+0000 7f5e30711740 10 client.16905 fetch_fsmap learned FSMap version 25 2024-09-12T02:06:39.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.494+0000 7f5e30711740 1 -- 172.21.15.16:0/3346146597 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d0b78f9d90 con 0x55d0b7874480 2024-09-12T02:06:39.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.494+0000 7f5e077fe640 1 -- 172.21.15.16:0/3346146597 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e20044040 con 0x55d0b7874480 2024-09-12T02:06:39.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.494+0000 7f5e077fe640 1 -- 172.21.15.16:0/3346146597 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f5e1c0b5070 con 0x55d0b7874480 2024-09-12T02:06:39.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.494+0000 7f5e30711740 10 client.16905 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:06:39.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.494+0000 7f5e30711740 10 client.16905 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:06:39.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.494+0000 7f5e30711740 20 client.16905 populate_metadata read hostname 'smithi016' 2024-09-12T02:06:39.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.494+0000 7f5e30711740 1 --2- 172.21.15.16:0/3346146597 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55d0b78fa510 0x55d0b78fc930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:39.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.494+0000 7f5e30711740 4 client.16905 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-12T02:06:39.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.494+0000 7f5e30711740 1 -- 172.21.15.16:0/3346146597 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55d0b77e0d00 con 0x55d0b78fa510 2024-09-12T02:06:39.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.495+0000 7f5e27fff640 1 --2- 172.21.15.16:0/3346146597 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55d0b78fa510 0x55d0b78fc930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:39.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.495+0000 7f5e077fe640 10 client.16905 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:06:39.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.495+0000 7f5e27fff640 1 --2- 172.21.15.16:0/3346146597 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55d0b78fa510 0x55d0b78fc930 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db392cc5a076452 server_cookie=5868906990211c47 in_seq=0 out_seq=0 2024-09-12T02:06:39.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.496+0000 7f5e077fe640 1 -- 172.21.15.16:0/3346146597 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55d0b77e0d00 con 0x55d0b78fa510 2024-09-12T02:06:39.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.496+0000 7f5e077fe640 10 client.16905 handle_command_reply: tid=0 2024-09-12T02:06:39.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.496+0000 7f5e30711740 1 client.16905 shutdown 2024-09-12T02:06:39.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.496+0000 7f5e30711740 1 -- 172.21.15.16:0/3346146597 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d0b7874480 msgr2=0x55d0b7887250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:39.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.496+0000 7f5e30711740 1 --2- 172.21.15.16:0/3346146597 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d0b7874480 0x55d0b7887250 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c097ac0 tx=0x7f5e1c09a9a0 comp rx=0 tx=0).stop 2024-09-12T02:06:39.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.496+0000 7f5e30711740 1 -- 172.21.15.16:0/3346146597 shutdown_connections 2024-09-12T02:06:39.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.496+0000 7f5e30711740 1 --2- 172.21.15.16:0/3346146597 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55d0b78fa510 0x55d0b78fc930 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:39.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.496+0000 7f5e30711740 1 --2- 172.21.15.16:0/3346146597 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d0b787ed40 0x55d0b7883ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:39.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.496+0000 7f5e30711740 1 --2- 172.21.15.16:0/3346146597 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d0b7874480 0x55d0b7887250 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:39.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.496+0000 7f5e30711740 1 --2- 172.21.15.16:0/3346146597 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d0b7873b60 0x55d0b7886d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:39.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.496+0000 7f5e30711740 1 -- 172.21.15.16:0/3346146597 >> 172.21.15.16:0/3346146597 conn(0x55d0b78700f0 msgr2=0x55d0b786f270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:06:39.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.496+0000 7f5e077fe640 0 client.16905 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:06:39.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.496+0000 7f5e30711740 1 -- 172.21.15.16:0/3346146597 shutdown_connections 2024-09-12T02:06:39.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.497+0000 7f5e30711740 1 -- 172.21.15.16:0/3346146597 wait complete. 2024-09-12T02:06:39.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.497+0000 7f5e30711740 20 client.16905 trim_cache size 0 max 16384 2024-09-12T02:06:39.502 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T02:06:39.502 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-09-12T02:06:39.502 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "0524a479-e19c-4268-885e-25b18cf2eaf7", 2024-09-12T02:06:39.502 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-09-12T02:06:39.502 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T02:06:39.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.497+0000 7f5e2e338640 1 -- 172.21.15.16:0/737059011 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f5e100a2b80 msgr2=0x7f5e100a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:39.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.497+0000 7f5e2e338640 1 --2- 172.21.15.16:0/737059011 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f5e100a2b80 0x7f5e100a5040 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f5e20030040 tx=0x7f5e2002a950 comp rx=0 tx=0).stop 2024-09-12T02:06:39.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.497+0000 7f5e2e338640 1 -- 172.21.15.16:0/737059011 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e28103660 msgr2=0x7f5e28100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:39.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.497+0000 7f5e2e338640 1 --2- 172.21.15.16:0/737059011 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e28103660 0x7f5e28100ee0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c00b3f0 tx=0x7f5e1c00b8b0 comp rx=0 tx=0).stop 2024-09-12T02:06:39.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.497+0000 7f5e2e338640 1 -- 172.21.15.16:0/737059011 shutdown_connections 2024-09-12T02:06:39.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.497+0000 7f5e2e338640 1 --2- 172.21.15.16:0/737059011 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f5e100a2b80 0x7f5e100a5040 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:39.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.497+0000 7f5e2e338640 1 --2- 172.21.15.16:0/737059011 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5e2810c170 0x7f5e28101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:39.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.497+0000 7f5e2e338640 1 --2- 172.21.15.16:0/737059011 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e28103660 0x7f5e28100ee0 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:39.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.497+0000 7f5e2e338640 1 --2- 172.21.15.16:0/737059011 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5e28102d40 0x7f5e281009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:39.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.497+0000 7f5e2e338640 1 -- 172.21.15.16:0/737059011 >> 172.21.15.16:0/737059011 conn(0x7f5e280fc7a0 msgr2=0x7f5e280fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:06:39.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.497+0000 7f5e2e338640 1 -- 172.21.15.16:0/737059011 shutdown_connections 2024-09-12T02:06:39.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:39.497+0000 7f5e2e338640 1 -- 172.21.15.16:0/737059011 wait complete. 2024-09-12T02:06:40.186 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T02:06:40.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:06:40 smithi184 ceph-mon[35593]: pgmap v477: 97 pgs: 97 active+clean; 2.4 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-09-12T02:06:40.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:06:40 smithi033 ceph-mon[35332]: pgmap v477: 97 pgs: 97 active+clean; 2.4 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-09-12T02:06:40.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:06:40 smithi016 ceph-mon[30947]: pgmap v477: 97 pgs: 97 active+clean; 2.4 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-09-12T02:06:40.445 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:06:40.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.900+0000 7f2274d70640 1 -- 172.21.15.16:0/3425835765 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2270103660 msgr2=0x7f227010bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:40.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.900+0000 7f2274d70640 1 --2- 172.21.15.16:0/3425835765 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2270103660 0x7f227010bb50 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f2264009900 tx=0x7f226402a480 comp rx=0 tx=0).stop 2024-09-12T02:06:40.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.900+0000 7f2274d70640 1 -- 172.21.15.16:0/3425835765 shutdown_connections 2024-09-12T02:06:40.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.900+0000 7f2274d70640 1 --2- 172.21.15.16:0/3425835765 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f227010c170 0x7f227010e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:40.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.900+0000 7f2274d70640 1 --2- 172.21.15.16:0/3425835765 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2270103660 0x7f227010bb50 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:40.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.900+0000 7f2274d70640 1 --2- 172.21.15.16:0/3425835765 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2270102d40 0x7f2270103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:40.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.900+0000 7f2274d70640 1 -- 172.21.15.16:0/3425835765 >> 172.21.15.16:0/3425835765 conn(0x7f22700fc7a0 msgr2=0x7f22700febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:06:40.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.900+0000 7f2274d70640 1 -- 172.21.15.16:0/3425835765 shutdown_connections 2024-09-12T02:06:40.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.900+0000 7f2274d70640 1 -- 172.21.15.16:0/3425835765 wait complete. 2024-09-12T02:06:40.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.901+0000 7f2274d70640 1 Processor -- start 2024-09-12T02:06:40.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.901+0000 7f2274d70640 1 -- start start 2024-09-12T02:06:40.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.901+0000 7f2274d70640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2270102d40 0x7f22701009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:40.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.901+0000 7f2274d70640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2270103660 0x7f2270100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:40.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.901+0000 7f2274d70640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f227010c170 0x7f2270101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:40.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.901+0000 7f2274d70640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2270111f70 con 0x7f227010c170 2024-09-12T02:06:40.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.901+0000 7f2274d70640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2270111df0 con 0x7f2270103660 2024-09-12T02:06:40.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.901+0000 7f2274d70640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f22701120f0 con 0x7f2270102d40 2024-09-12T02:06:40.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.901+0000 7f226ed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f227010c170 0x7f2270101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:40.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.901+0000 7f226ed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f227010c170 0x7f2270101420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56878/0 (socket says 172.21.15.16:56878) 2024-09-12T02:06:40.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.901+0000 7f226ed9d640 1 -- 172.21.15.16:0/3052902763 learned_addr learned my addr 172.21.15.16:0/3052902763 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:06:40.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.901+0000 7f226dd9b640 1 --2- 172.21.15.16:0/3052902763 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2270103660 0x7f2270100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:40.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.901+0000 7f226e59c640 1 --2- 172.21.15.16:0/3052902763 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2270102d40 0x7f22701009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:40.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.902+0000 7f226ed9d640 1 -- 172.21.15.16:0/3052902763 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2270102d40 msgr2=0x7f22701009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:40.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.902+0000 7f226ed9d640 1 --2- 172.21.15.16:0/3052902763 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2270102d40 0x7f22701009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:40.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.902+0000 7f226ed9d640 1 -- 172.21.15.16:0/3052902763 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2270103660 msgr2=0x7f2270100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:40.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.902+0000 7f226ed9d640 1 --2- 172.21.15.16:0/3052902763 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2270103660 0x7f2270100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:40.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.902+0000 7f226ed9d640 1 -- 172.21.15.16:0/3052902763 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22701a68d0 con 0x7f227010c170 2024-09-12T02:06:40.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.902+0000 7f226ed9d640 1 --2- 172.21.15.16:0/3052902763 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f227010c170 0x7f2270101420 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f226800eaa0 tx=0x7f226800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:06:40.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.902+0000 7f225f7fe640 1 -- 172.21.15.16:0/3052902763 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f226800cdb0 con 0x7f227010c170 2024-09-12T02:06:40.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.902+0000 7f225f7fe640 1 -- 172.21.15.16:0/3052902763 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2268005020 con 0x7f227010c170 2024-09-12T02:06:40.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.902+0000 7f2274d70640 1 -- 172.21.15.16:0/3052902763 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22701a6bc0 con 0x7f227010c170 2024-09-12T02:06:40.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.902+0000 7f225f7fe640 1 -- 172.21.15.16:0/3052902763 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22680116b0 con 0x7f227010c170 2024-09-12T02:06:40.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.902+0000 7f2274d70640 1 -- 172.21.15.16:0/3052902763 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22701a70d0 con 0x7f227010c170 2024-09-12T02:06:40.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.905+0000 7f225f7fe640 1 -- 172.21.15.16:0/3052902763 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f2268005560 con 0x7f227010c170 2024-09-12T02:06:40.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.905+0000 7f2277149740 1 Processor -- start 2024-09-12T02:06:40.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.905+0000 7f2277149740 1 -- start start 2024-09-12T02:06:40.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.905+0000 7f2277149740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bd143194a0 0x55bd14319880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:40.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.905+0000 7f2277149740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bd14319dc0 0x55bd14324140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:40.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.905+0000 7f2277149740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bd14324680 0x55bd14326a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:40.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.905+0000 7f2277149740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55bd1427f2e0 con 0x55bd143194a0 2024-09-12T02:06:40.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.905+0000 7f2277149740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55bd14226ce0 con 0x55bd14319dc0 2024-09-12T02:06:40.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.905+0000 7f2277149740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55bd1427b2a0 con 0x55bd14324680 2024-09-12T02:06:40.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.905+0000 7f226dd9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bd143194a0 0x55bd14319880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:40.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.905+0000 7f226dd9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bd143194a0 0x55bd14319880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56886/0 (socket says 172.21.15.16:56886) 2024-09-12T02:06:40.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.905+0000 7f226dd9b640 1 -- 172.21.15.16:0/1901626393 learned_addr learned my addr 172.21.15.16:0/1901626393 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:06:40.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.905+0000 7f225f7fe640 1 --2- 172.21.15.16:0/3052902763 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f22580a2c00 0x7f22580a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:40.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.906+0000 7f226e59c640 1 --2- 172.21.15.16:0/1901626393 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bd14319dc0 0x55bd14324140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:40.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.906+0000 7f226dd9b640 1 --2- 172.21.15.16:0/1901626393 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bd14324680 0x55bd14326a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:40.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.906+0000 7f225f7fe640 1 -- 172.21.15.16:0/3052902763 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f22680cf130 con 0x7f227010c170 2024-09-12T02:06:40.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.906+0000 7f226ed9d640 1 --2- 172.21.15.16:0/3052902763 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f22580a2c00 0x7f22580a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:40.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.906+0000 7f226dd9b640 1 -- 172.21.15.16:0/1901626393 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bd14324680 msgr2=0x55bd14326a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:06:40.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.906+0000 7f226dd9b640 1 --2- 172.21.15.16:0/1901626393 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bd14324680 0x55bd14326a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:40.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.906+0000 7f226dd9b640 1 -- 172.21.15.16:0/1901626393 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bd14319dc0 msgr2=0x55bd14324140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:06:40.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.906+0000 7f226dd9b640 1 --2- 172.21.15.16:0/1901626393 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bd14319dc0 0x55bd14324140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:40.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.906+0000 7f226dd9b640 1 -- 172.21.15.16:0/1901626393 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd1420b440 con 0x55bd143194a0 2024-09-12T02:06:40.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.906+0000 7f226dd9b640 1 --2- 172.21.15.16:0/1901626393 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bd143194a0 0x55bd14319880 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f2264005c40 tx=0x7f2264000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb8762457c6fccbb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:06:40.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.906+0000 7f226ed9d640 1 --2- 172.21.15.16:0/3052902763 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f22580a2c00 0x7f22580a50c0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f22680cccd0 tx=0x7f2268009270 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:06:40.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.906+0000 7f225cff9640 1 -- 172.21.15.16:0/1901626393 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2264035040 con 0x55bd143194a0 2024-09-12T02:06:40.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.906+0000 7f225cff9640 1 -- 172.21.15.16:0/1901626393 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2264042070 con 0x55bd143194a0 2024-09-12T02:06:40.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.906+0000 7f225cff9640 1 -- 172.21.15.16:0/1901626393 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2264004a00 con 0x55bd143194a0 2024-09-12T02:06:40.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.907+0000 7f2277149740 1 -- 172.21.15.16:0/1901626393 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bd143194a0 msgr2=0x55bd14319880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:40.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.907+0000 7f2277149740 1 --2- 172.21.15.16:0/1901626393 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bd143194a0 0x55bd14319880 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f2264005c40 tx=0x7f2264000bc0 comp rx=0 tx=0).stop 2024-09-12T02:06:40.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.907+0000 7f2277149740 1 -- 172.21.15.16:0/1901626393 shutdown_connections 2024-09-12T02:06:40.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.907+0000 7f2277149740 1 --2- 172.21.15.16:0/1901626393 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bd14324680 0x55bd14326a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:40.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.907+0000 7f2277149740 1 --2- 172.21.15.16:0/1901626393 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bd14319dc0 0x55bd14324140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:40.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.907+0000 7f2277149740 1 --2- 172.21.15.16:0/1901626393 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bd143194a0 0x55bd14319880 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:40.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.907+0000 7f2277149740 1 -- 172.21.15.16:0/1901626393 >> 172.21.15.16:0/1901626393 conn(0x55bd14315a30 msgr2=0x55bd14315e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:06:40.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.907+0000 7f2277149740 1 -- 172.21.15.16:0/1901626393 shutdown_connections 2024-09-12T02:06:40.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.907+0000 7f2277149740 1 -- 172.21.15.16:0/1901626393 wait complete. 2024-09-12T02:06:40.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.907+0000 7f2277149740 1 -- start start 2024-09-12T02:06:40.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.908+0000 7f2277149740 1 Processor -- start 2024-09-12T02:06:40.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.908+0000 7f2277149740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bd143194a0 0x55bd14324020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:40.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.908+0000 7f2277149740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bd14319dc0 0x55bd143212d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:40.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.908+0000 7f2277149740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bd14324680 0x55bd14321870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:40.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.908+0000 7f2277149740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55bd14327250 con 0x55bd14319dc0 2024-09-12T02:06:40.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.908+0000 7f2277149740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55bd1423a050 con 0x55bd143194a0 2024-09-12T02:06:40.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.908+0000 7f2277149740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55bd143273d0 con 0x55bd14324680 2024-09-12T02:06:40.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.908+0000 7f2277149740 10 client.0 _ll_register_callbacks cb 0x55bd1427a160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:06:40.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.908+0000 7f226e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bd14319dc0 0x55bd143212d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:40.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.908+0000 7f226e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bd14319dc0 0x55bd143212d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56890/0 (socket says 172.21.15.16:56890) 2024-09-12T02:06:40.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.908+0000 7f226e59c640 1 -- 172.21.15.16:0/2336691190 learned_addr learned my addr 172.21.15.16:0/2336691190 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:06:40.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.908+0000 7f226dd9b640 1 --2- 172.21.15.16:0/2336691190 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bd143194a0 0x55bd14324020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:40.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.908+0000 7f226dd9b640 1 --2- 172.21.15.16:0/2336691190 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bd14324680 0x55bd14321870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:40.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.908+0000 7f226e59c640 1 -- 172.21.15.16:0/2336691190 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bd14324680 msgr2=0x55bd14321870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:06:40.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.908+0000 7f226e59c640 1 --2- 172.21.15.16:0/2336691190 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bd14324680 0x55bd14321870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:40.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.908+0000 7f226e59c640 1 -- 172.21.15.16:0/2336691190 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bd143194a0 msgr2=0x55bd14324020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:06:40.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.908+0000 7f226e59c640 1 --2- 172.21.15.16:0/2336691190 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bd143194a0 0x55bd14324020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:40.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.908+0000 7f226e59c640 1 -- 172.21.15.16:0/2336691190 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd13ff03c0 con 0x55bd14319dc0 2024-09-12T02:06:40.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.909+0000 7f226e59c640 1 --2- 172.21.15.16:0/2336691190 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bd14319dc0 0x55bd143212d0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f226000bee0 tx=0x7f2260012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb73d57ff5c00b10 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:06:40.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.909+0000 7f2255ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:06:40.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.909+0000 7f2255ffb640 1 -- 172.21.15.16:0/2336691190 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2260004230 con 0x55bd14319dc0 2024-09-12T02:06:40.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.909+0000 7f2255ffb640 1 -- 172.21.15.16:0/2336691190 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f22600043d0 con 0x55bd14319dc0 2024-09-12T02:06:40.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.909+0000 7f2277149740 1 -- 172.21.15.16:0/2336691190 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2264042070 con 0x55bd14319dc0 2024-09-12T02:06:40.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.909+0000 7f2255ffb640 1 -- 172.21.15.16:0/2336691190 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f22600106a0 con 0x55bd14319dc0 2024-09-12T02:06:40.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.909+0000 7f2277149740 10 client.16917 fetch_fsmap learned FSMap version 25 2024-09-12T02:06:40.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.909+0000 7f2277149740 1 -- 172.21.15.16:0/2336691190 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55bd14332cf0 con 0x55bd14319dc0 2024-09-12T02:06:40.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.909+0000 7f2255ffb640 1 -- 172.21.15.16:0/2336691190 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2264042070 con 0x55bd14319dc0 2024-09-12T02:06:40.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.910+0000 7f2255ffb640 1 -- 172.21.15.16:0/2336691190 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f2260003bf0 con 0x55bd14319dc0 2024-09-12T02:06:40.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.910+0000 7f2277149740 10 client.16917 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:06:40.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.910+0000 7f2277149740 10 client.16917 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:06:40.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.910+0000 7f2277149740 20 client.16917 populate_metadata read hostname 'smithi016' 2024-09-12T02:06:40.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.910+0000 7f2277149740 1 --2- 172.21.15.16:0/2336691190 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55bd14334200 0x55bd143365f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:40.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.910+0000 7f2277149740 4 client.16917 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:06:40.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.910+0000 7f2277149740 1 -- 172.21.15.16:0/2336691190 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bd1428c850 con 0x55bd14334200 2024-09-12T02:06:40.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.910+0000 7f226dd9b640 1 --2- 172.21.15.16:0/2336691190 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55bd14334200 0x55bd143365f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:40.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.911+0000 7f2255ffb640 10 client.16917 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:06:40.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.911+0000 7f226dd9b640 1 --2- 172.21.15.16:0/2336691190 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55bd14334200 0x55bd143365f0 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44dbe5c86e2a24e server_cookie=c1506e4b09b39976 in_seq=0 out_seq=0 2024-09-12T02:06:40.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.911+0000 7f2255ffb640 1 -- 172.21.15.16:0/2336691190 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bd1428c850 con 0x55bd14334200 2024-09-12T02:06:40.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.911+0000 7f2255ffb640 10 client.16917 handle_command_reply: tid=0 2024-09-12T02:06:40.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.911+0000 7f2277149740 1 client.16917 shutdown 2024-09-12T02:06:40.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.911+0000 7f2277149740 1 -- 172.21.15.16:0/2336691190 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bd14319dc0 msgr2=0x55bd143212d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:40.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.911+0000 7f2277149740 1 --2- 172.21.15.16:0/2336691190 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bd14319dc0 0x55bd143212d0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f226000bee0 tx=0x7f2260012820 comp rx=0 tx=0).stop 2024-09-12T02:06:40.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.911+0000 7f2277149740 1 -- 172.21.15.16:0/2336691190 shutdown_connections 2024-09-12T02:06:40.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.911+0000 7f2277149740 1 --2- 172.21.15.16:0/2336691190 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55bd14334200 0x55bd143365f0 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:40.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.911+0000 7f2277149740 1 --2- 172.21.15.16:0/2336691190 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bd14324680 0x55bd14321870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:40.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.911+0000 7f2277149740 1 --2- 172.21.15.16:0/2336691190 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bd14319dc0 0x55bd143212d0 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:40.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.911+0000 7f2277149740 1 --2- 172.21.15.16:0/2336691190 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bd143194a0 0x55bd14324020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:40.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.911+0000 7f2277149740 1 -- 172.21.15.16:0/2336691190 >> 172.21.15.16:0/2336691190 conn(0x55bd14315a30 msgr2=0x55bd1431b760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:06:40.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.911+0000 7f2255ffb640 0 client.16917 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:06:40.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.912+0000 7f2277149740 1 -- 172.21.15.16:0/2336691190 shutdown_connections 2024-09-12T02:06:40.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.912+0000 7f2277149740 1 -- 172.21.15.16:0/2336691190 wait complete. 2024-09-12T02:06:40.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.912+0000 7f2277149740 20 client.16917 trim_cache size 0 max 16384 2024-09-12T02:06:40.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.929+0000 7f2277149740 1 Processor -- start 2024-09-12T02:06:40.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.929+0000 7f2277149740 1 -- start start 2024-09-12T02:06:40.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.929+0000 7f2277149740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bd143194a0 0x55bd143212a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:40.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.929+0000 7f2277149740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bd14319dc0 0x55bd143304e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:40.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.929+0000 7f2277149740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bd14324680 0x55bd1433bf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:40.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.929+0000 7f2277149740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f22701b1dc0 con 0x55bd143194a0 2024-09-12T02:06:40.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.929+0000 7f2277149740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f22701af690 con 0x55bd14319dc0 2024-09-12T02:06:40.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.929+0000 7f2277149740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f22701ad520 con 0x55bd14324680 2024-09-12T02:06:40.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.929+0000 7f226dd9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bd143194a0 0x55bd143212a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:40.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.929+0000 7f226dd9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bd143194a0 0x55bd143212a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56902/0 (socket says 172.21.15.16:56902) 2024-09-12T02:06:40.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.929+0000 7f226dd9b640 1 -- 172.21.15.16:0/2530996439 learned_addr learned my addr 172.21.15.16:0/2530996439 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:06:40.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.929+0000 7f226e59c640 1 --2- 172.21.15.16:0/2530996439 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bd14319dc0 0x55bd143304e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:40.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.929+0000 7f226dd9b640 1 -- 172.21.15.16:0/2530996439 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bd14324680 msgr2=0x55bd1433bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:06:40.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.929+0000 7f226dd9b640 1 --2- 172.21.15.16:0/2530996439 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bd14324680 0x55bd1433bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:40.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.929+0000 7f226dd9b640 1 -- 172.21.15.16:0/2530996439 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bd14319dc0 msgr2=0x55bd143304e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:06:40.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.929+0000 7f226dd9b640 1 --2- 172.21.15.16:0/2530996439 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bd14319dc0 0x55bd143304e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:40.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.929+0000 7f226dd9b640 1 -- 172.21.15.16:0/2530996439 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd14319bf0 con 0x55bd143194a0 2024-09-12T02:06:40.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.929+0000 7f226e59c640 1 --2- 172.21.15.16:0/2530996439 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bd14319dc0 0x55bd143304e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:06:40.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.929+0000 7f226dd9b640 1 --2- 172.21.15.16:0/2530996439 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bd143194a0 0x55bd143212a0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f2264009900 tx=0x7f2264005500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3f72744fe8d7554 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:06:40.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.930+0000 7f2257fff640 1 -- 172.21.15.16:0/2530996439 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2264035040 con 0x55bd143194a0 2024-09-12T02:06:40.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.930+0000 7f2257fff640 1 -- 172.21.15.16:0/2530996439 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2264042050 con 0x55bd143194a0 2024-09-12T02:06:40.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.930+0000 7f2257fff640 1 -- 172.21.15.16:0/2530996439 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22640037d0 con 0x55bd143194a0 2024-09-12T02:06:40.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.930+0000 7f2277149740 1 -- 172.21.15.16:0/2530996439 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bd143194a0 msgr2=0x55bd143212a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:40.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.930+0000 7f2277149740 1 --2- 172.21.15.16:0/2530996439 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bd143194a0 0x55bd143212a0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f2264009900 tx=0x7f2264005500 comp rx=0 tx=0).stop 2024-09-12T02:06:40.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.930+0000 7f2277149740 1 -- 172.21.15.16:0/2530996439 shutdown_connections 2024-09-12T02:06:40.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.930+0000 7f2277149740 1 --2- 172.21.15.16:0/2530996439 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bd14324680 0x55bd1433bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:40.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.930+0000 7f2277149740 1 --2- 172.21.15.16:0/2530996439 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bd14319dc0 0x55bd143304e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:40.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.930+0000 7f2277149740 1 --2- 172.21.15.16:0/2530996439 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bd143194a0 0x55bd143212a0 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:40.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.930+0000 7f2277149740 1 -- 172.21.15.16:0/2530996439 >> 172.21.15.16:0/2530996439 conn(0x55bd14315a30 msgr2=0x55bd14320b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:06:40.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.930+0000 7f2277149740 1 -- 172.21.15.16:0/2530996439 shutdown_connections 2024-09-12T02:06:40.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.930+0000 7f2277149740 1 -- 172.21.15.16:0/2530996439 wait complete. 2024-09-12T02:06:40.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.931+0000 7f2277149740 1 -- start start 2024-09-12T02:06:40.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.931+0000 7f2277149740 1 Processor -- start 2024-09-12T02:06:40.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.931+0000 7f2277149740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bd143194a0 0x55bd1432cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:40.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.931+0000 7f2277149740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bd14319dc0 0x55bd1432a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:40.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.931+0000 7f2277149740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bd14324680 0x55bd1432a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:40.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.931+0000 7f2277149740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f22701adb70 con 0x55bd14324680 2024-09-12T02:06:40.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.931+0000 7f2277149740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f22701aefc0 con 0x55bd143194a0 2024-09-12T02:06:40.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.931+0000 7f2277149740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f22701ae220 con 0x55bd14319dc0 2024-09-12T02:06:40.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.931+0000 7f2277149740 10 client.0 _ll_register_callbacks cb 0x7f226000b730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:06:40.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.931+0000 7f226dd9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bd14324680 0x55bd1432a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:40.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.931+0000 7f226dd9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bd14324680 0x55bd1432a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56906/0 (socket says 172.21.15.16:56906) 2024-09-12T02:06:40.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.931+0000 7f226dd9b640 1 -- 172.21.15.16:0/1375535003 learned_addr learned my addr 172.21.15.16:0/1375535003 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:06:40.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.931+0000 7f226e59c640 1 --2- 172.21.15.16:0/1375535003 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bd14319dc0 0x55bd1432a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:40.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.931+0000 7f226dd9b640 1 --2- 172.21.15.16:0/1375535003 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bd143194a0 0x55bd1432cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:40.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.931+0000 7f226dd9b640 1 -- 172.21.15.16:0/1375535003 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bd14319dc0 msgr2=0x55bd1432a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:06:40.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.931+0000 7f226dd9b640 1 --2- 172.21.15.16:0/1375535003 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bd14319dc0 0x55bd1432a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:40.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.931+0000 7f226dd9b640 1 -- 172.21.15.16:0/1375535003 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bd143194a0 msgr2=0x55bd1432cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:06:40.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.931+0000 7f226dd9b640 1 --2- 172.21.15.16:0/1375535003 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bd143194a0 0x55bd1432cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:40.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.931+0000 7f226dd9b640 1 -- 172.21.15.16:0/1375535003 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd1432ae40 con 0x55bd14324680 2024-09-12T02:06:40.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.932+0000 7f226dd9b640 1 --2- 172.21.15.16:0/1375535003 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bd14324680 0x55bd1432a610 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f2264005c40 tx=0x7f22640035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97b6445c1fad3dc1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:06:40.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.932+0000 7f2255ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:06:40.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.932+0000 7f2255ffb640 1 -- 172.21.15.16:0/1375535003 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2264035070 con 0x55bd14324680 2024-09-12T02:06:40.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.932+0000 7f2255ffb640 1 -- 172.21.15.16:0/1375535003 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f2264004250 con 0x55bd14324680 2024-09-12T02:06:40.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.932+0000 7f2277149740 1 -- 172.21.15.16:0/1375535003 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2264042050 con 0x55bd14324680 2024-09-12T02:06:40.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.932+0000 7f2255ffb640 1 -- 172.21.15.16:0/1375535003 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2264031a70 con 0x55bd14324680 2024-09-12T02:06:40.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.932+0000 7f2277149740 10 client.16923 fetch_fsmap learned FSMap version 25 2024-09-12T02:06:40.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.932+0000 7f2277149740 1 -- 172.21.15.16:0/1375535003 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55bd1439eee0 con 0x55bd14324680 2024-09-12T02:06:40.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.932+0000 7f2255ffb640 1 -- 172.21.15.16:0/1375535003 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2264042050 con 0x55bd14324680 2024-09-12T02:06:40.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.932+0000 7f2255ffb640 1 -- 172.21.15.16:0/1375535003 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f22640537b0 con 0x55bd14324680 2024-09-12T02:06:40.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.932+0000 7f2277149740 10 client.16923 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:06:40.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.932+0000 7f2277149740 10 client.16923 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:06:40.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.932+0000 7f2277149740 20 client.16923 populate_metadata read hostname 'smithi016' 2024-09-12T02:06:40.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.932+0000 7f2277149740 1 --2- 172.21.15.16:0/1375535003 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55bd1439f0b0 0x55bd143a14a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:40.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.932+0000 7f2277149740 4 client.16923 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T02:06:40.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.932+0000 7f2277149740 1 -- 172.21.15.16:0/1375535003 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55bd14286d60 con 0x55bd1439f0b0 2024-09-12T02:06:40.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.933+0000 7f226e59c640 1 --2- 172.21.15.16:0/1375535003 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55bd1439f0b0 0x55bd143a14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:40.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.933+0000 7f226e59c640 1 --2- 172.21.15.16:0/1375535003 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55bd1439f0b0 0x55bd143a14a0 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=68824d36632ee8b3 server_cookie=148cfa9614fb8817 in_seq=0 out_seq=0 2024-09-12T02:06:40.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.933+0000 7f2255ffb640 10 client.16923 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:06:40.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.933+0000 7f2255ffb640 1 -- 172.21.15.16:0/1375535003 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55bd14286d60 con 0x55bd1439f0b0 2024-09-12T02:06:40.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.933+0000 7f2255ffb640 10 client.16923 handle_command_reply: tid=0 2024-09-12T02:06:40.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.933+0000 7f2277149740 1 client.16923 shutdown 2024-09-12T02:06:40.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.933+0000 7f2277149740 1 -- 172.21.15.16:0/1375535003 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bd14324680 msgr2=0x55bd1432a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:40.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.933+0000 7f2277149740 1 --2- 172.21.15.16:0/1375535003 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bd14324680 0x55bd1432a610 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f2264005c40 tx=0x7f22640035f0 comp rx=0 tx=0).stop 2024-09-12T02:06:40.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.933+0000 7f2277149740 1 -- 172.21.15.16:0/1375535003 shutdown_connections 2024-09-12T02:06:40.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.933+0000 7f2277149740 1 --2- 172.21.15.16:0/1375535003 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55bd1439f0b0 0x55bd143a14a0 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:40.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.933+0000 7f2277149740 1 --2- 172.21.15.16:0/1375535003 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bd14324680 0x55bd1432a610 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:40.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.933+0000 7f2277149740 1 --2- 172.21.15.16:0/1375535003 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bd14319dc0 0x55bd1432a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:40.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.933+0000 7f2255ffb640 0 client.16923 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:06:40.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.933+0000 7f2277149740 1 --2- 172.21.15.16:0/1375535003 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bd143194a0 0x55bd1432cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:40.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.933+0000 7f2277149740 1 -- 172.21.15.16:0/1375535003 >> 172.21.15.16:0/1375535003 conn(0x55bd14315a30 msgr2=0x55bd14322b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:06:40.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.934+0000 7f2277149740 1 -- 172.21.15.16:0/1375535003 shutdown_connections 2024-09-12T02:06:40.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.934+0000 7f2277149740 1 -- 172.21.15.16:0/1375535003 wait complete. 2024-09-12T02:06:40.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.934+0000 7f2277149740 20 client.16923 trim_cache size 0 max 16384 2024-09-12T02:06:40.965 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T02:06:40.965 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-12T02:06:40.965 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": { 2024-09-12T02:06:40.965 INFO:teuthology.orchestra.run.smithi016.stdout: "0524a479-e19c-4268-885e-25b18cf2eaf7": { 2024-09-12T02:06:40.966 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-12T02:06:40.966 INFO:teuthology.orchestra.run.smithi016.stdout: "tag": "0524a479-e19c-4268-885e-25b18cf2eaf7", 2024-09-12T02:06:40.966 INFO:teuthology.orchestra.run.smithi016.stdout: "options": "recursive,force" 2024-09-12T02:06:40.966 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:06:40.966 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:06:40.967 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T02:06:40.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.934+0000 7f2274d70640 1 -- 172.21.15.16:0/3052902763 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f22580a2c00 msgr2=0x7f22580a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:40.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.934+0000 7f2274d70640 1 --2- 172.21.15.16:0/3052902763 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f22580a2c00 0x7f22580a50c0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f22680cccd0 tx=0x7f2268009270 comp rx=0 tx=0).stop 2024-09-12T02:06:40.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.934+0000 7f2274d70640 1 -- 172.21.15.16:0/3052902763 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f227010c170 msgr2=0x7f2270101420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:40.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.934+0000 7f2274d70640 1 --2- 172.21.15.16:0/3052902763 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f227010c170 0x7f2270101420 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f226800eaa0 tx=0x7f226800ef60 comp rx=0 tx=0).stop 2024-09-12T02:06:40.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.934+0000 7f2274d70640 1 -- 172.21.15.16:0/3052902763 shutdown_connections 2024-09-12T02:06:40.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.934+0000 7f2274d70640 1 --2- 172.21.15.16:0/3052902763 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f22580a2c00 0x7f22580a50c0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:40.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.934+0000 7f2274d70640 1 --2- 172.21.15.16:0/3052902763 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f227010c170 0x7f2270101420 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:40.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.934+0000 7f2274d70640 1 --2- 172.21.15.16:0/3052902763 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2270103660 0x7f2270100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:40.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.934+0000 7f2274d70640 1 --2- 172.21.15.16:0/3052902763 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2270102d40 0x7f22701009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:40.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.934+0000 7f2274d70640 1 -- 172.21.15.16:0/3052902763 >> 172.21.15.16:0/3052902763 conn(0x7f22700fc7a0 msgr2=0x7f22700fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:06:40.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.934+0000 7f2274d70640 1 -- 172.21.15.16:0/3052902763 shutdown_connections 2024-09-12T02:06:40.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:06:40.934+0000 7f2274d70640 1 -- 172.21.15.16:0/3052902763 wait complete. 2024-09-12T02:06:41.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:06:41 smithi184 ceph-mon[35593]: scrub queued for path: / 2024-09-12T02:06:41.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:06:41 smithi184 ceph-mon[35593]: scrub summary: idle+waiting paths [/] 2024-09-12T02:06:41.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:06:41 smithi184 ceph-mon[35593]: scrub summary: active paths [/] 2024-09-12T02:06:41.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:06:41 smithi184 ceph-mon[35593]: scrub summary: idle+waiting paths [/] 2024-09-12T02:06:41.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:06:41 smithi033 ceph-mon[35332]: scrub queued for path: / 2024-09-12T02:06:41.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:06:41 smithi033 ceph-mon[35332]: scrub summary: idle+waiting paths [/] 2024-09-12T02:06:41.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:06:41 smithi033 ceph-mon[35332]: scrub summary: active paths [/] 2024-09-12T02:06:41.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:06:41 smithi033 ceph-mon[35332]: scrub summary: idle+waiting paths [/] 2024-09-12T02:06:41.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:06:41 smithi016 ceph-mon[30947]: scrub queued for path: / 2024-09-12T02:06:41.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:06:41 smithi016 ceph-mon[30947]: scrub summary: idle+waiting paths [/] 2024-09-12T02:06:41.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:06:41 smithi016 ceph-mon[30947]: scrub summary: active paths [/] 2024-09-12T02:06:41.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:06:41 smithi016 ceph-mon[30947]: scrub summary: idle+waiting paths [/] 2024-09-12T02:06:41.603 INFO:tasks.cephfs.filesystem:scrub status for tag:0524a479-e19c-4268-885e-25b18cf2eaf7 - {'path': '/', 'tag': '0524a479-e19c-4268-885e-25b18cf2eaf7', 'options': 'recursive,force'} 2024-09-12T02:06:42.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:06:42 smithi184 ceph-mon[35593]: pgmap v478: 97 pgs: 97 active+clean; 2.5 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-09-12T02:06:42.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:06:42 smithi184 ceph-mon[35593]: scrub summary: idle 2024-09-12T02:06:42.418 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:06:42 smithi033 ceph-mon[35332]: pgmap v478: 97 pgs: 97 active+clean; 2.5 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-09-12T02:06:42.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:06:42 smithi033 ceph-mon[35332]: scrub summary: idle 2024-09-12T02:06:42.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:06:42 smithi016 ceph-mon[30947]: pgmap v478: 97 pgs: 97 active+clean; 2.5 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-09-12T02:06:42.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:06:42 smithi016 ceph-mon[30947]: scrub summary: idle 2024-09-12T02:06:44.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:06:44 smithi184 ceph-mon[35593]: pgmap v479: 97 pgs: 97 active+clean; 2.5 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-12T02:06:44.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:06:44 smithi033 ceph-mon[35332]: pgmap v479: 97 pgs: 97 active+clean; 2.5 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-12T02:06:44.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:06:44 smithi016 ceph-mon[30947]: pgmap v479: 97 pgs: 97 active+clean; 2.5 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-12T02:06:46.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:06:46 smithi184 ceph-mon[35593]: pgmap v480: 97 pgs: 97 active+clean; 2.5 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-09-12T02:06:46.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:06:46 smithi033 ceph-mon[35332]: pgmap v480: 97 pgs: 97 active+clean; 2.5 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-09-12T02:06:46.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:06:46 smithi016 ceph-mon[30947]: pgmap v480: 97 pgs: 97 active+clean; 2.5 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-09-12T02:06:46.809 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.809+0000 7f8d48731640 1 -- 172.21.15.16:0/3115459280 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d40102610 msgr2=0x7f8d40108ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:46.809 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.809+0000 7f8d48731640 1 --2- 172.21.15.16:0/3115459280 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d40102610 0x7f8d40108ea0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f8d38009ca0 tx=0x7f8d3802a8c0 comp rx=0 tx=0).stop 2024-09-12T02:06:46.809 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.809+0000 7f8d48731640 1 -- 172.21.15.16:0/3115459280 shutdown_connections 2024-09-12T02:06:46.809 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.809+0000 7f8d48731640 1 --2- 172.21.15.16:0/3115459280 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d40102610 0x7f8d40108ea0 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:46.810 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.809+0000 7f8d48731640 1 --2- 172.21.15.16:0/3115459280 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8d40101c50 0x7f8d401020d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:46.810 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.809+0000 7f8d48731640 1 --2- 172.21.15.16:0/3115459280 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8d40100a60 0x7f8d40100e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:46.810 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.809+0000 7f8d48731640 1 -- 172.21.15.16:0/3115459280 >> 172.21.15.16:0/3115459280 conn(0x7f8d400fc1e0 msgr2=0x7f8d400fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:06:46.810 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.809+0000 7f8d48731640 1 -- 172.21.15.16:0/3115459280 shutdown_connections 2024-09-12T02:06:46.810 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.809+0000 7f8d48731640 1 -- 172.21.15.16:0/3115459280 wait complete. 2024-09-12T02:06:46.812 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.810+0000 7f8d48731640 1 Processor -- start 2024-09-12T02:06:46.812 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.810+0000 7f8d48731640 1 -- start start 2024-09-12T02:06:46.812 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.810+0000 7f8d48731640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d40100a60 0x7f8d40197db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:46.812 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.810+0000 7f8d48731640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8d40101c50 0x7f8d401982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:46.813 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.810+0000 7f8d48731640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8d40102610 0x7f8d4019f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:46.813 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.810+0000 7f8d48731640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8d4010c6e0 con 0x7f8d40100a60 2024-09-12T02:06:46.813 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.810+0000 7f8d48731640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8d4010c560 con 0x7f8d40101c50 2024-09-12T02:06:46.813 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.810+0000 7f8d48731640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8d4010c860 con 0x7f8d40102610 2024-09-12T02:06:46.813 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.810+0000 7f8d464cd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d40100a60 0x7f8d40197db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:46.813 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.810+0000 7f8d464cd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d40100a60 0x7f8d40197db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46582/0 (socket says 172.21.15.16:46582) 2024-09-12T02:06:46.813 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.810+0000 7f8d464cd640 1 -- 172.21.15.16:0/3732858504 learned_addr learned my addr 172.21.15.16:0/3732858504 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:06:46.813 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.810+0000 7f8d46cce640 1 --2- 172.21.15.16:0/3732858504 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8d40102610 0x7f8d4019f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:46.813 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.810+0000 7f8d45ccc640 1 --2- 172.21.15.16:0/3732858504 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8d40101c50 0x7f8d401982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:46.813 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.810+0000 7f8d464cd640 1 -- 172.21.15.16:0/3732858504 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8d40102610 msgr2=0x7f8d4019f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:46.813 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.810+0000 7f8d464cd640 1 --2- 172.21.15.16:0/3732858504 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8d40102610 0x7f8d4019f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:46.813 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.810+0000 7f8d464cd640 1 -- 172.21.15.16:0/3732858504 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8d40101c50 msgr2=0x7f8d401982f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:46.813 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.810+0000 7f8d464cd640 1 --2- 172.21.15.16:0/3732858504 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8d40101c50 0x7f8d401982f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:46.814 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.810+0000 7f8d464cd640 1 -- 172.21.15.16:0/3732858504 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d4019f9c0 con 0x7f8d40100a60 2024-09-12T02:06:46.814 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.810+0000 7f8d46cce640 1 --2- 172.21.15.16:0/3732858504 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8d40102610 0x7f8d4019f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:06:46.814 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.810+0000 7f8d45ccc640 1 --2- 172.21.15.16:0/3732858504 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8d40101c50 0x7f8d401982f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:06:46.814 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.811+0000 7f8d464cd640 1 --2- 172.21.15.16:0/3732858504 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d40100a60 0x7f8d40197db0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f8d3c00d770 tx=0x7f8d3c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:06:46.814 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.811+0000 7f8d377fe640 1 -- 172.21.15.16:0/3732858504 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d3c017070 con 0x7f8d40100a60 2024-09-12T02:06:46.814 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.811+0000 7f8d377fe640 1 -- 172.21.15.16:0/3732858504 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d3c009c30 con 0x7f8d40100a60 2024-09-12T02:06:46.814 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.811+0000 7f8d48731640 1 -- 172.21.15.16:0/3732858504 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d4019fcb0 con 0x7f8d40100a60 2024-09-12T02:06:46.814 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.811+0000 7f8d377fe640 1 -- 172.21.15.16:0/3732858504 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d3c003dd0 con 0x7f8d40100a60 2024-09-12T02:06:46.814 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.811+0000 7f8d48731640 1 -- 172.21.15.16:0/3732858504 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d401a01a0 con 0x7f8d40100a60 2024-09-12T02:06:46.820 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.813+0000 7f8d377fe640 1 -- 172.21.15.16:0/3732858504 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f8d3c003fb0 con 0x7f8d40100a60 2024-09-12T02:06:46.820 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.813+0000 7f8d4b4d0740 1 Processor -- start 2024-09-12T02:06:46.821 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.814+0000 7f8d4b4d0740 1 -- start start 2024-09-12T02:06:46.821 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.814+0000 7f8d4b4d0740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564feb9b59a0 0x564feb9b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:46.821 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.814+0000 7f8d4b4d0740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564feb9b62e0 0x564feb9bada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:46.821 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.814+0000 7f8d4b4d0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564feb9bb370 0x564feb9bd830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:46.821 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.814+0000 7f8d4b4d0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x564feb9192f0 con 0x564feb9bb370 2024-09-12T02:06:46.821 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.814+0000 7f8d4b4d0740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x564feb8c1270 con 0x564feb9b62e0 2024-09-12T02:06:46.821 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.814+0000 7f8d4b4d0740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x564feb9152b0 con 0x564feb9b59a0 2024-09-12T02:06:46.821 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.814+0000 7f8d45ccc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564feb9bb370 0x564feb9bd830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:46.821 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.814+0000 7f8d45ccc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564feb9bb370 0x564feb9bd830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46590/0 (socket says 172.21.15.16:46590) 2024-09-12T02:06:46.821 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.814+0000 7f8d45ccc640 1 -- 172.21.15.16:0/1304511019 learned_addr learned my addr 172.21.15.16:0/1304511019 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:06:46.821 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.814+0000 7f8d377fe640 1 --2- 172.21.15.16:0/3732858504 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f8d2c0a2cb0 0x7f8d2c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:46.821 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.814+0000 7f8d46cce640 1 --2- 172.21.15.16:0/1304511019 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564feb9b59a0 0x564feb9b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:46.822 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.814+0000 7f8d464cd640 1 --2- 172.21.15.16:0/1304511019 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564feb9b62e0 0x564feb9bada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:46.822 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.814+0000 7f8d46cce640 1 --2- 172.21.15.16:0/3732858504 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f8d2c0a2cb0 0x7f8d2c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:46.822 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.815+0000 7f8d377fe640 1 -- 172.21.15.16:0/3732858504 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f8d3c0ceaa0 con 0x7f8d40100a60 2024-09-12T02:06:46.822 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.815+0000 7f8d45ccc640 1 -- 172.21.15.16:0/1304511019 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564feb9b59a0 msgr2=0x564feb9b5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:06:46.822 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.815+0000 7f8d45ccc640 1 --2- 172.21.15.16:0/1304511019 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564feb9b59a0 0x564feb9b5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:46.822 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.815+0000 7f8d45ccc640 1 -- 172.21.15.16:0/1304511019 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564feb9b62e0 msgr2=0x564feb9bada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:06:46.822 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.815+0000 7f8d45ccc640 1 --2- 172.21.15.16:0/1304511019 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564feb9b62e0 0x564feb9bada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:46.822 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.815+0000 7f8d45ccc640 1 -- 172.21.15.16:0/1304511019 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564feb8c0340 con 0x564feb9bb370 2024-09-12T02:06:46.822 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.815+0000 7f8d45ccc640 1 --2- 172.21.15.16:0/1304511019 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564feb9bb370 0x564feb9bd830 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f8d30009c30 tx=0x7f8d30010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e1b82b447d78cce server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:06:46.822 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.815+0000 7f8d46cce640 1 --2- 172.21.15.16:0/3732858504 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f8d2c0a2cb0 0x7f8d2c0a5170 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f8d3802f540 tx=0x7f8d380095c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:06:46.822 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.815+0000 7f8d34ff9640 1 -- 172.21.15.16:0/1304511019 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d30004020 con 0x564feb9bb370 2024-09-12T02:06:46.822 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.815+0000 7f8d34ff9640 1 -- 172.21.15.16:0/1304511019 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d300041c0 con 0x564feb9bb370 2024-09-12T02:06:46.822 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.815+0000 7f8d34ff9640 1 -- 172.21.15.16:0/1304511019 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d30012670 con 0x564feb9bb370 2024-09-12T02:06:46.823 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.816+0000 7f8d4b4d0740 1 -- 172.21.15.16:0/1304511019 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564feb9bb370 msgr2=0x564feb9bd830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:46.823 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.816+0000 7f8d4b4d0740 1 --2- 172.21.15.16:0/1304511019 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564feb9bb370 0x564feb9bd830 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f8d30009c30 tx=0x7f8d30010740 comp rx=0 tx=0).stop 2024-09-12T02:06:46.823 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.816+0000 7f8d4b4d0740 1 -- 172.21.15.16:0/1304511019 shutdown_connections 2024-09-12T02:06:46.823 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.816+0000 7f8d4b4d0740 1 --2- 172.21.15.16:0/1304511019 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564feb9bb370 0x564feb9bd830 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:46.823 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.816+0000 7f8d4b4d0740 1 --2- 172.21.15.16:0/1304511019 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564feb9b62e0 0x564feb9bada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:46.823 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.816+0000 7f8d4b4d0740 1 --2- 172.21.15.16:0/1304511019 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564feb9b59a0 0x564feb9b5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:46.823 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.816+0000 7f8d4b4d0740 1 -- 172.21.15.16:0/1304511019 >> 172.21.15.16:0/1304511019 conn(0x564feb9b1010 msgr2=0x564feb9b1410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:06:46.823 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.816+0000 7f8d4b4d0740 1 -- 172.21.15.16:0/1304511019 shutdown_connections 2024-09-12T02:06:46.823 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.816+0000 7f8d4b4d0740 1 -- 172.21.15.16:0/1304511019 wait complete. 2024-09-12T02:06:46.823 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.816+0000 7f8d4b4d0740 1 -- start start 2024-09-12T02:06:46.823 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.816+0000 7f8d4b4d0740 1 Processor -- start 2024-09-12T02:06:46.823 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.817+0000 7f8d4b4d0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564feb9b59a0 0x564feb9bfc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:46.823 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.817+0000 7f8d4b4d0740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564feb9b62e0 0x564feb9c0190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:46.824 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.817+0000 7f8d4b4d0740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564feb9bb370 0x564feb9c71d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:46.824 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.817+0000 7f8d4b4d0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x564feb9be340 con 0x564feb9b59a0 2024-09-12T02:06:46.824 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.817+0000 7f8d4b4d0740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x564feb8a2290 con 0x564feb9bb370 2024-09-12T02:06:46.824 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.817+0000 7f8d4b4d0740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x564feb9be4c0 con 0x564feb9b62e0 2024-09-12T02:06:46.824 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.817+0000 7f8d4b4d0740 10 client.0 _ll_register_callbacks cb 0x564feb8d3140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:06:46.824 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.817+0000 7f8d464cd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564feb9b59a0 0x564feb9bfc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:46.824 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.817+0000 7f8d464cd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564feb9b59a0 0x564feb9bfc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46594/0 (socket says 172.21.15.16:46594) 2024-09-12T02:06:46.824 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.817+0000 7f8d464cd640 1 -- 172.21.15.16:0/659826093 learned_addr learned my addr 172.21.15.16:0/659826093 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:06:46.824 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.817+0000 7f8d45ccc640 1 --2- 172.21.15.16:0/659826093 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564feb9b62e0 0x564feb9c0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:46.824 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.817+0000 7f8d46cce640 1 --2- 172.21.15.16:0/659826093 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564feb9bb370 0x564feb9c71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:46.824 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.817+0000 7f8d464cd640 1 -- 172.21.15.16:0/659826093 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564feb9b62e0 msgr2=0x564feb9c0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:06:46.824 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.817+0000 7f8d464cd640 1 --2- 172.21.15.16:0/659826093 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564feb9b62e0 0x564feb9c0190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:46.824 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.817+0000 7f8d464cd640 1 -- 172.21.15.16:0/659826093 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564feb9bb370 msgr2=0x564feb9c71d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:06:46.825 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.817+0000 7f8d464cd640 1 --2- 172.21.15.16:0/659826093 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564feb9bb370 0x564feb9c71d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:46.825 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.817+0000 7f8d464cd640 1 -- 172.21.15.16:0/659826093 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564feb62c210 con 0x564feb9b59a0 2024-09-12T02:06:46.825 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.817+0000 7f8d46cce640 1 --2- 172.21.15.16:0/659826093 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564feb9bb370 0x564feb9c71d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:06:46.825 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.817+0000 7f8d464cd640 1 --2- 172.21.15.16:0/659826093 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564feb9b59a0 0x564feb9bfc50 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f8d3c0a3a40 tx=0x7f8d3c004590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60e795da04a18aa9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:06:46.825 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.817+0000 7f8d29ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:06:46.825 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.818+0000 7f8d29ffb640 1 -- 172.21.15.16:0/659826093 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d3c020040 con 0x564feb9b59a0 2024-09-12T02:06:46.825 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.818+0000 7f8d29ffb640 1 -- 172.21.15.16:0/659826093 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f8d3c016bb0 con 0x564feb9b59a0 2024-09-12T02:06:46.825 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.818+0000 7f8d4b4d0740 1 -- 172.21.15.16:0/659826093 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d300041c0 con 0x564feb9b59a0 2024-09-12T02:06:46.825 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.818+0000 7f8d29ffb640 1 -- 172.21.15.16:0/659826093 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f8d3c099440 con 0x564feb9b59a0 2024-09-12T02:06:46.825 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.818+0000 7f8d4b4d0740 10 client.16935 fetch_fsmap learned FSMap version 25 2024-09-12T02:06:46.825 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.818+0000 7f8d4b4d0740 1 -- 172.21.15.16:0/659826093 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x564feb6f8620 con 0x564feb9b59a0 2024-09-12T02:06:46.825 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.818+0000 7f8d29ffb640 1 -- 172.21.15.16:0/659826093 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d300041c0 con 0x564feb9b59a0 2024-09-12T02:06:46.826 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.818+0000 7f8d29ffb640 1 -- 172.21.15.16:0/659826093 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f8d3c0b4150 con 0x564feb9b59a0 2024-09-12T02:06:46.826 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.818+0000 7f8d4b4d0740 10 client.16935 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:06:46.826 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.818+0000 7f8d4b4d0740 10 client.16935 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:06:46.826 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.818+0000 7f8d4b4d0740 20 client.16935 populate_metadata read hostname 'smithi016' 2024-09-12T02:06:46.826 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.818+0000 7f8d4b4d0740 1 --2- 172.21.15.16:0/659826093 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x564feb9cb820 0x564feb9cdc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:46.826 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.818+0000 7f8d4b4d0740 4 client.16935 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:06:46.826 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.818+0000 7f8d4b4d0740 1 -- 172.21.15.16:0/659826093 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564feb926a20 con 0x564feb9cb820 2024-09-12T02:06:46.826 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.819+0000 7f8d45ccc640 1 --2- 172.21.15.16:0/659826093 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x564feb9cb820 0x564feb9cdc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:46.826 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.819+0000 7f8d29ffb640 10 client.16935 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:06:46.826 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.819+0000 7f8d45ccc640 1 --2- 172.21.15.16:0/659826093 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x564feb9cb820 0x564feb9cdc10 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8daf964b4eac7134 server_cookie=c4fa40cb7f8237dd in_seq=0 out_seq=0 2024-09-12T02:06:46.826 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.820+0000 7f8d29ffb640 1 -- 172.21.15.16:0/659826093 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564feb926a20 con 0x564feb9cb820 2024-09-12T02:06:46.826 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.820+0000 7f8d29ffb640 10 client.16935 handle_command_reply: tid=0 2024-09-12T02:06:46.826 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.820+0000 7f8d4b4d0740 1 client.16935 shutdown 2024-09-12T02:06:46.826 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.820+0000 7f8d4b4d0740 1 -- 172.21.15.16:0/659826093 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564feb9b59a0 msgr2=0x564feb9bfc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:46.827 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.820+0000 7f8d4b4d0740 1 --2- 172.21.15.16:0/659826093 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564feb9b59a0 0x564feb9bfc50 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f8d3c0a3a40 tx=0x7f8d3c004590 comp rx=0 tx=0).stop 2024-09-12T02:06:46.827 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.820+0000 7f8d4b4d0740 1 -- 172.21.15.16:0/659826093 shutdown_connections 2024-09-12T02:06:46.827 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.820+0000 7f8d4b4d0740 1 --2- 172.21.15.16:0/659826093 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x564feb9cb820 0x564feb9cdc10 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:46.827 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.820+0000 7f8d4b4d0740 1 --2- 172.21.15.16:0/659826093 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564feb9bb370 0x564feb9c71d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:46.827 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.820+0000 7f8d29ffb640 0 client.16935 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:06:46.827 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.820+0000 7f8d4b4d0740 1 --2- 172.21.15.16:0/659826093 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564feb9b62e0 0x564feb9c0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:46.827 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.820+0000 7f8d4b4d0740 1 --2- 172.21.15.16:0/659826093 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564feb9b59a0 0x564feb9bfc50 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:46.827 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.820+0000 7f8d4b4d0740 1 -- 172.21.15.16:0/659826093 >> 172.21.15.16:0/659826093 conn(0x564feb9b1010 msgr2=0x564feb9b2820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:06:46.827 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.820+0000 7f8d4b4d0740 1 -- 172.21.15.16:0/659826093 shutdown_connections 2024-09-12T02:06:46.827 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.820+0000 7f8d4b4d0740 1 -- 172.21.15.16:0/659826093 wait complete. 2024-09-12T02:06:46.827 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.820+0000 7f8d4b4d0740 20 client.16935 trim_cache size 0 max 16384 2024-09-12T02:06:46.839 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.835+0000 7f8d4b4d0740 1 Processor -- start 2024-09-12T02:06:46.839 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.835+0000 7f8d4b4d0740 1 -- start start 2024-09-12T02:06:46.839 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.835+0000 7f8d4b4d0740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564feb9b1010 0x564feb9b1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:46.839 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.835+0000 7f8d4b4d0740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564feb9b59a0 0x564feb9c7190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:46.839 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.835+0000 7f8d4b4d0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564feb9b62e0 0x564feb9b2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:46.840 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.835+0000 7f8d4b4d0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8d401ab6e0 con 0x564feb9b62e0 2024-09-12T02:06:46.840 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.835+0000 7f8d4b4d0740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8d401a8fb0 con 0x564feb9b59a0 2024-09-12T02:06:46.840 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.835+0000 7f8d4b4d0740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8d401a6e00 con 0x564feb9b1010 2024-09-12T02:06:46.840 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.835+0000 7f8d46cce640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564feb9b62e0 0x564feb9b2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:46.840 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.835+0000 7f8d46cce640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564feb9b62e0 0x564feb9b2400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46610/0 (socket says 172.21.15.16:46610) 2024-09-12T02:06:46.840 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.835+0000 7f8d46cce640 1 -- 172.21.15.16:0/3947825785 learned_addr learned my addr 172.21.15.16:0/3947825785 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:06:46.840 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.835+0000 7f8d45ccc640 1 --2- 172.21.15.16:0/3947825785 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564feb9b59a0 0x564feb9c7190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:46.840 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.835+0000 7f8d464cd640 1 --2- 172.21.15.16:0/3947825785 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564feb9b1010 0x564feb9b1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:46.840 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.835+0000 7f8d46cce640 1 -- 172.21.15.16:0/3947825785 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564feb9b1010 msgr2=0x564feb9b1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:06:46.840 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.835+0000 7f8d46cce640 1 --2- 172.21.15.16:0/3947825785 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564feb9b1010 0x564feb9b1ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:46.840 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.835+0000 7f8d46cce640 1 -- 172.21.15.16:0/3947825785 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564feb9b59a0 msgr2=0x564feb9c7190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:06:46.840 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.835+0000 7f8d46cce640 1 --2- 172.21.15.16:0/3947825785 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564feb9b59a0 0x564feb9c7190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:46.841 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.835+0000 7f8d46cce640 1 -- 172.21.15.16:0/3947825785 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564feb9bf790 con 0x564feb9b62e0 2024-09-12T02:06:46.841 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.835+0000 7f8d45ccc640 1 --2- 172.21.15.16:0/3947825785 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564feb9b59a0 0x564feb9c7190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:06:46.841 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.835+0000 7f8d46cce640 1 --2- 172.21.15.16:0/3947825785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564feb9b62e0 0x564feb9b2400 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f8d38031cb0 tx=0x7f8d38040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=238af59361d5dd3f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:06:46.841 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.836+0000 7f8d2bfff640 1 -- 172.21.15.16:0/3947825785 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d38003c30 con 0x564feb9b62e0 2024-09-12T02:06:46.841 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.836+0000 7f8d2bfff640 1 -- 172.21.15.16:0/3947825785 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d38036040 con 0x564feb9b62e0 2024-09-12T02:06:46.841 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.836+0000 7f8d2bfff640 1 -- 172.21.15.16:0/3947825785 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d3802f980 con 0x564feb9b62e0 2024-09-12T02:06:46.841 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.836+0000 7f8d4b4d0740 1 -- 172.21.15.16:0/3947825785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564feb9b62e0 msgr2=0x564feb9b2400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:46.841 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.836+0000 7f8d4b4d0740 1 --2- 172.21.15.16:0/3947825785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564feb9b62e0 0x564feb9b2400 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f8d38031cb0 tx=0x7f8d38040000 comp rx=0 tx=0).stop 2024-09-12T02:06:46.841 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.836+0000 7f8d4b4d0740 1 -- 172.21.15.16:0/3947825785 shutdown_connections 2024-09-12T02:06:46.841 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.836+0000 7f8d4b4d0740 1 --2- 172.21.15.16:0/3947825785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564feb9b62e0 0x564feb9b2400 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:46.841 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.836+0000 7f8d4b4d0740 1 --2- 172.21.15.16:0/3947825785 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564feb9b59a0 0x564feb9c7190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:46.841 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.836+0000 7f8d4b4d0740 1 --2- 172.21.15.16:0/3947825785 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564feb9b1010 0x564feb9b1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:46.841 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.836+0000 7f8d4b4d0740 1 -- 172.21.15.16:0/3947825785 >> 172.21.15.16:0/3947825785 conn(0x564feb9baf00 msgr2=0x564feb9bfa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:06:46.842 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.836+0000 7f8d4b4d0740 1 -- 172.21.15.16:0/3947825785 shutdown_connections 2024-09-12T02:06:46.842 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.836+0000 7f8d4b4d0740 1 -- 172.21.15.16:0/3947825785 wait complete. 2024-09-12T02:06:46.842 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.837+0000 7f8d4b4d0740 1 -- start start 2024-09-12T02:06:46.842 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.837+0000 7f8d4b4d0740 1 Processor -- start 2024-09-12T02:06:46.842 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.837+0000 7f8d4b4d0740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564feb9b1010 0x564feb9c41c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:46.842 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.837+0000 7f8d4b4d0740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564feb9b59a0 0x564feb9c4700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:46.842 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.837+0000 7f8d4b4d0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564feb9b62e0 0x564feb9c5710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:46.842 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.837+0000 7f8d4b4d0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8d401a7470 con 0x564feb9b62e0 2024-09-12T02:06:46.842 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.837+0000 7f8d4b4d0740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8d401a88e0 con 0x564feb9b1010 2024-09-12T02:06:46.842 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.837+0000 7f8d4b4d0740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8d401a7b40 con 0x564feb9b59a0 2024-09-12T02:06:46.842 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.837+0000 7f8d4b4d0740 10 client.0 _ll_register_callbacks cb 0x7f8d3c0046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:06:46.842 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.837+0000 7f8d46cce640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564feb9b62e0 0x564feb9c5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:46.842 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.837+0000 7f8d46cce640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564feb9b62e0 0x564feb9c5710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46622/0 (socket says 172.21.15.16:46622) 2024-09-12T02:06:46.843 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.837+0000 7f8d46cce640 1 -- 172.21.15.16:0/2625006038 learned_addr learned my addr 172.21.15.16:0/2625006038 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:06:46.843 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.837+0000 7f8d45ccc640 1 --2- 172.21.15.16:0/2625006038 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564feb9b59a0 0x564feb9c4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:46.843 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.837+0000 7f8d464cd640 1 --2- 172.21.15.16:0/2625006038 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564feb9b1010 0x564feb9c41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:46.843 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.837+0000 7f8d46cce640 1 -- 172.21.15.16:0/2625006038 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564feb9b59a0 msgr2=0x564feb9c4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:06:46.843 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.837+0000 7f8d46cce640 1 --2- 172.21.15.16:0/2625006038 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564feb9b59a0 0x564feb9c4700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:46.843 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.837+0000 7f8d46cce640 1 -- 172.21.15.16:0/2625006038 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564feb9b1010 msgr2=0x564feb9c41c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:06:46.843 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.837+0000 7f8d46cce640 1 --2- 172.21.15.16:0/2625006038 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564feb9b1010 0x564feb9c41c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:46.843 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.837+0000 7f8d46cce640 1 -- 172.21.15.16:0/2625006038 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564feb9c5d20 con 0x564feb9b62e0 2024-09-12T02:06:46.843 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.837+0000 7f8d45ccc640 1 --2- 172.21.15.16:0/2625006038 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564feb9b59a0 0x564feb9c4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:06:46.843 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.837+0000 7f8d46cce640 1 --2- 172.21.15.16:0/2625006038 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564feb9b62e0 0x564feb9c5710 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f8d38009ca0 tx=0x7f8d38031ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6406d336680f5e74 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:06:46.843 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.837+0000 7f8d29ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:06:46.843 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.838+0000 7f8d29ffb640 1 -- 172.21.15.16:0/2625006038 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d38061060 con 0x564feb9b62e0 2024-09-12T02:06:46.843 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.838+0000 7f8d29ffb640 1 -- 172.21.15.16:0/2625006038 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f8d3802f680 con 0x564feb9b62e0 2024-09-12T02:06:46.844 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.838+0000 7f8d4b4d0740 1 -- 172.21.15.16:0/2625006038 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d38036040 con 0x564feb9b62e0 2024-09-12T02:06:46.844 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.838+0000 7f8d29ffb640 1 -- 172.21.15.16:0/2625006038 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f8d3802f880 con 0x564feb9b62e0 2024-09-12T02:06:46.844 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.838+0000 7f8d4b4d0740 10 client.16941 fetch_fsmap learned FSMap version 25 2024-09-12T02:06:46.844 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.838+0000 7f8d4b4d0740 1 -- 172.21.15.16:0/2625006038 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x564feb9c6fa0 con 0x564feb9b62e0 2024-09-12T02:06:46.844 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.838+0000 7f8d29ffb640 1 -- 172.21.15.16:0/2625006038 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d38036040 con 0x564feb9b62e0 2024-09-12T02:06:46.844 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.838+0000 7f8d29ffb640 1 -- 172.21.15.16:0/2625006038 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f8d38066070 con 0x564feb9b62e0 2024-09-12T02:06:46.844 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.838+0000 7f8d4b4d0740 10 client.16941 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:06:46.844 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.838+0000 7f8d4b4d0740 10 client.16941 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:06:46.844 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.838+0000 7f8d4b4d0740 20 client.16941 populate_metadata read hostname 'smithi016' 2024-09-12T02:06:46.844 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.838+0000 7f8d4b4d0740 1 --2- 172.21.15.16:0/2625006038 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x564feb9bb370 0x564feba38840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:46.844 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.838+0000 7f8d4b4d0740 4 client.16941 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:06:46.844 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.838+0000 7f8d4b4d0740 1 -- 172.21.15.16:0/2625006038 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564feb920d70 con 0x564feb9bb370 2024-09-12T02:06:46.844 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.839+0000 7f8d464cd640 1 --2- 172.21.15.16:0/2625006038 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x564feb9bb370 0x564feba38840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:46.845 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.839+0000 7f8d29ffb640 10 client.16941 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:06:46.845 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.839+0000 7f8d464cd640 1 --2- 172.21.15.16:0/2625006038 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x564feb9bb370 0x564feba38840 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=56d66c3f53612887 server_cookie=f58edf19977a57e3 in_seq=0 out_seq=0 2024-09-12T02:06:46.851 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.851+0000 7f8d29ffb640 1 -- 172.21.15.16:0/2625006038 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564feb920d70 con 0x564feb9bb370 2024-09-12T02:06:46.851 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.851+0000 7f8d29ffb640 10 client.16941 handle_command_reply: tid=0 2024-09-12T02:06:46.852 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.851+0000 7f8d4b4d0740 1 client.16941 shutdown 2024-09-12T02:06:46.852 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.851+0000 7f8d4b4d0740 1 -- 172.21.15.16:0/2625006038 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564feb9b62e0 msgr2=0x564feb9c5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:46.852 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.851+0000 7f8d4b4d0740 1 --2- 172.21.15.16:0/2625006038 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564feb9b62e0 0x564feb9c5710 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f8d38009ca0 tx=0x7f8d38031ce0 comp rx=0 tx=0).stop 2024-09-12T02:06:46.852 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.851+0000 7f8d4b4d0740 1 -- 172.21.15.16:0/2625006038 shutdown_connections 2024-09-12T02:06:46.852 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.851+0000 7f8d4b4d0740 1 --2- 172.21.15.16:0/2625006038 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x564feb9bb370 0x564feba38840 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:46.852 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.851+0000 7f8d4b4d0740 1 --2- 172.21.15.16:0/2625006038 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564feb9b62e0 0x564feb9c5710 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:46.852 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.851+0000 7f8d4b4d0740 1 --2- 172.21.15.16:0/2625006038 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564feb9b59a0 0x564feb9c4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:46.852 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.851+0000 7f8d29ffb640 0 client.16941 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:06:46.852 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.851+0000 7f8d4b4d0740 1 --2- 172.21.15.16:0/2625006038 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564feb9b1010 0x564feb9c41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:46.852 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.851+0000 7f8d4b4d0740 1 -- 172.21.15.16:0/2625006038 >> 172.21.15.16:0/2625006038 conn(0x564feb9baf00 msgr2=0x564feb9bfa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:06:46.852 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.851+0000 7f8d4b4d0740 1 -- 172.21.15.16:0/2625006038 shutdown_connections 2024-09-12T02:06:46.852 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.851+0000 7f8d4b4d0740 1 -- 172.21.15.16:0/2625006038 wait complete. 2024-09-12T02:06:46.852 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.851+0000 7f8d4b4d0740 20 client.16941 trim_cache size 0 max 16384 2024-09-12T02:06:46.853 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-09-12T02:06:46.853 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-09-12T02:06:46.853 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-09-12T02:06:46.853 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-09-12T02:06:46.854 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.852+0000 7f8d48731640 1 -- 172.21.15.16:0/3732858504 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f8d2c0a2cb0 msgr2=0x7f8d2c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:46.854 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.852+0000 7f8d48731640 1 --2- 172.21.15.16:0/3732858504 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f8d2c0a2cb0 0x7f8d2c0a5170 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f8d3802f540 tx=0x7f8d380095c0 comp rx=0 tx=0).stop 2024-09-12T02:06:46.854 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.852+0000 7f8d48731640 1 -- 172.21.15.16:0/3732858504 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d40100a60 msgr2=0x7f8d40197db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:46.854 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.852+0000 7f8d48731640 1 --2- 172.21.15.16:0/3732858504 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d40100a60 0x7f8d40197db0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f8d3c00d770 tx=0x7f8d3c00dc30 comp rx=0 tx=0).stop 2024-09-12T02:06:46.854 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.852+0000 7f8d48731640 1 -- 172.21.15.16:0/3732858504 shutdown_connections 2024-09-12T02:06:46.854 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.852+0000 7f8d48731640 1 --2- 172.21.15.16:0/3732858504 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f8d2c0a2cb0 0x7f8d2c0a5170 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:46.854 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.852+0000 7f8d48731640 1 --2- 172.21.15.16:0/3732858504 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8d40102610 0x7f8d4019f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:46.854 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.852+0000 7f8d48731640 1 --2- 172.21.15.16:0/3732858504 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8d40101c50 0x7f8d401982f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:46.854 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.852+0000 7f8d48731640 1 --2- 172.21.15.16:0/3732858504 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d40100a60 0x7f8d40197db0 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:46.854 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.852+0000 7f8d48731640 1 -- 172.21.15.16:0/3732858504 >> 172.21.15.16:0/3732858504 conn(0x7f8d400fc1e0 msgr2=0x7f8d4006a300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:06:46.854 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.852+0000 7f8d48731640 1 -- 172.21.15.16:0/3732858504 shutdown_connections 2024-09-12T02:06:46.855 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:46.852+0000 7f8d48731640 1 -- 172.21.15.16:0/3732858504 wait complete. 2024-09-12T02:06:48.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:06:48 smithi033 ceph-mon[35332]: pgmap v481: 97 pgs: 97 active+clean; 2.5 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-09-12T02:06:48.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:06:48 smithi016 ceph-mon[30947]: pgmap v481: 97 pgs: 97 active+clean; 2.5 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-09-12T02:06:48.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:06:48 smithi184 ceph-mon[35593]: pgmap v481: 97 pgs: 97 active+clean; 2.5 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-09-12T02:06:50.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:06:50 smithi033 ceph-mon[35332]: pgmap v482: 97 pgs: 97 active+clean; 2.5 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T02:06:50.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:06:50 smithi016 ceph-mon[30947]: pgmap v482: 97 pgs: 97 active+clean; 2.5 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T02:06:50.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:06:50 smithi184 ceph-mon[35593]: pgmap v482: 97 pgs: 97 active+clean; 2.5 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T02:06:52.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:06:52 smithi033 ceph-mon[35332]: pgmap v483: 97 pgs: 97 active+clean; 2.6 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-09-12T02:06:52.429 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:06:52 smithi184 ceph-mon[35593]: pgmap v483: 97 pgs: 97 active+clean; 2.6 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-09-12T02:06:52.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:06:52 smithi016 ceph-mon[30947]: pgmap v483: 97 pgs: 97 active+clean; 2.6 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-09-12T02:06:54.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:06:54 smithi033 ceph-mon[35332]: pgmap v484: 97 pgs: 97 active+clean; 2.6 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-09-12T02:06:54.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:06:54 smithi016 ceph-mon[30947]: pgmap v484: 97 pgs: 97 active+clean; 2.6 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-09-12T02:06:54.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:06:54 smithi184 ceph-mon[35593]: pgmap v484: 97 pgs: 97 active+clean; 2.6 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-09-12T02:06:56.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:06:56 smithi033 ceph-mon[35332]: pgmap v485: 97 pgs: 97 active+clean; 2.7 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-09-12T02:06:56.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:06:56 smithi016 ceph-mon[30947]: pgmap v485: 97 pgs: 97 active+clean; 2.7 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-09-12T02:06:56.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:06:56 smithi184 ceph-mon[35593]: pgmap v485: 97 pgs: 97 active+clean; 2.7 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-09-12T02:06:58.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:06:58 smithi033 ceph-mon[35332]: pgmap v486: 97 pgs: 97 active+clean; 2.7 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-09-12T02:06:58.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:06:58 smithi016 ceph-mon[30947]: pgmap v486: 97 pgs: 97 active+clean; 2.7 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-09-12T02:06:58.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:06:58 smithi184 ceph-mon[35593]: pgmap v486: 97 pgs: 97 active+clean; 2.7 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-09-12T02:06:59.954 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.953+0000 7f972f711640 1 -- 172.21.15.16:0/61594901 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9728100d10 msgr2=0x7f9728101170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:59.955 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.953+0000 7f972f711640 1 --2- 172.21.15.16:0/61594901 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9728100d10 0x7f9728101170 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f9718009900 tx=0x7f971802a410 comp rx=0 tx=0).stop 2024-09-12T02:06:59.955 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.953+0000 7f972f711640 1 -- 172.21.15.16:0/61594901 shutdown_connections 2024-09-12T02:06:59.955 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.953+0000 7f972f711640 1 --2- 172.21.15.16:0/61594901 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97281016b0 0x7f9728101b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:59.955 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.953+0000 7f972f711640 1 --2- 172.21.15.16:0/61594901 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9728100d10 0x7f9728101170 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:59.955 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.953+0000 7f972f711640 1 --2- 172.21.15.16:0/61594901 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f97281003d0 0x7f97281007d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:59.955 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.953+0000 7f972f711640 1 -- 172.21.15.16:0/61594901 >> 172.21.15.16:0/61594901 conn(0x7f97280fc1e0 msgr2=0x7f97280fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:06:59.955 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.953+0000 7f972f711640 1 -- 172.21.15.16:0/61594901 shutdown_connections 2024-09-12T02:06:59.955 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.953+0000 7f972f711640 1 -- 172.21.15.16:0/61594901 wait complete. 2024-09-12T02:06:59.955 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.954+0000 7f972f711640 1 Processor -- start 2024-09-12T02:06:59.955 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.954+0000 7f972f711640 1 -- start start 2024-09-12T02:06:59.957 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.954+0000 7f972f711640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97281003d0 0x7f972819a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:59.957 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.954+0000 7f972f711640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9728100d10 0x7f972819a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:59.957 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.954+0000 7f972f711640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97281016b0 0x7f97281a15d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:59.957 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.954+0000 7f972f711640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f972810fa00 con 0x7f97281003d0 2024-09-12T02:06:59.957 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.954+0000 7f972f711640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f972810f880 con 0x7f97281016b0 2024-09-12T02:06:59.957 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.954+0000 7f972f711640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f972810fb80 con 0x7f9728100d10 2024-09-12T02:06:59.958 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.954+0000 7f972d4ad640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97281003d0 0x7f972819a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:59.958 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.954+0000 7f972d4ad640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97281003d0 0x7f972819a020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42916/0 (socket says 172.21.15.16:42916) 2024-09-12T02:06:59.958 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.954+0000 7f972d4ad640 1 -- 172.21.15.16:0/558497916 learned_addr learned my addr 172.21.15.16:0/558497916 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:06:59.958 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.954+0000 7f972dcae640 1 --2- 172.21.15.16:0/558497916 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97281016b0 0x7f97281a15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:59.958 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.954+0000 7f972ccac640 1 --2- 172.21.15.16:0/558497916 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9728100d10 0x7f972819a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:59.958 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.954+0000 7f972d4ad640 1 -- 172.21.15.16:0/558497916 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9728100d10 msgr2=0x7f972819a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:59.958 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.954+0000 7f972d4ad640 1 --2- 172.21.15.16:0/558497916 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9728100d10 0x7f972819a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:59.958 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.954+0000 7f972d4ad640 1 -- 172.21.15.16:0/558497916 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97281016b0 msgr2=0x7f97281a15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:59.958 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.954+0000 7f972d4ad640 1 --2- 172.21.15.16:0/558497916 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97281016b0 0x7f97281a15d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:59.958 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.954+0000 7f972d4ad640 1 -- 172.21.15.16:0/558497916 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97281a1c40 con 0x7f97281003d0 2024-09-12T02:06:59.958 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.954+0000 7f972ccac640 1 --2- 172.21.15.16:0/558497916 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9728100d10 0x7f972819a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:06:59.958 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.955+0000 7f972d4ad640 1 --2- 172.21.15.16:0/558497916 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97281003d0 0x7f972819a020 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f972400d950 tx=0x7f972400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:06:59.958 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.955+0000 7f971e7fc640 1 -- 172.21.15.16:0/558497916 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9724017070 con 0x7f97281003d0 2024-09-12T02:06:59.959 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.955+0000 7f971e7fc640 1 -- 172.21.15.16:0/558497916 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9724009c30 con 0x7f97281003d0 2024-09-12T02:06:59.959 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.955+0000 7f972f711640 1 -- 172.21.15.16:0/558497916 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97281a1f30 con 0x7f97281003d0 2024-09-12T02:06:59.959 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.955+0000 7f971e7fc640 1 -- 172.21.15.16:0/558497916 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f972400f6b0 con 0x7f97281003d0 2024-09-12T02:06:59.959 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.955+0000 7f972f711640 1 -- 172.21.15.16:0/558497916 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97281a2390 con 0x7f97281003d0 2024-09-12T02:06:59.964 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.958+0000 7f971e7fc640 1 -- 172.21.15.16:0/558497916 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f97240109d0 con 0x7f97281003d0 2024-09-12T02:06:59.965 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.958+0000 7f97324aa740 1 Processor -- start 2024-09-12T02:06:59.965 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.958+0000 7f97324aa740 1 -- start start 2024-09-12T02:06:59.965 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.958+0000 7f97324aa740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560078dd19a0 0x560078dd1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:59.965 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.958+0000 7f97324aa740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560078dd22e0 0x560078dd6da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:59.965 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.958+0000 7f97324aa740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560078dd7370 0x560078dd9830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:59.965 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.958+0000 7f97324aa740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x560078d352f0 con 0x560078dd19a0 2024-09-12T02:06:59.965 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.958+0000 7f97324aa740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x560078cdd270 con 0x560078dd22e0 2024-09-12T02:06:59.965 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.958+0000 7f97324aa740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x560078d312b0 con 0x560078dd7370 2024-09-12T02:06:59.965 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.958+0000 7f972dcae640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560078dd19a0 0x560078dd1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:59.965 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.958+0000 7f972dcae640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560078dd19a0 0x560078dd1da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42932/0 (socket says 172.21.15.16:42932) 2024-09-12T02:06:59.966 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.958+0000 7f972dcae640 1 -- 172.21.15.16:0/3051940172 learned_addr learned my addr 172.21.15.16:0/3051940172 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:06:59.966 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.958+0000 7f972d4ad640 1 --2- 172.21.15.16:0/3051940172 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560078dd22e0 0x560078dd6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:59.966 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.959+0000 7f971e7fc640 1 --2- 172.21.15.16:0/558497916 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f97140a2c30 0x7f97140a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:59.966 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.959+0000 7f972ccac640 1 --2- 172.21.15.16:0/3051940172 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560078dd7370 0x560078dd9830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:59.966 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.959+0000 7f972dcae640 1 -- 172.21.15.16:0/3051940172 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560078dd7370 msgr2=0x560078dd9830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:06:59.966 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.959+0000 7f972dcae640 1 --2- 172.21.15.16:0/3051940172 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560078dd7370 0x560078dd9830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:59.966 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.959+0000 7f972dcae640 1 -- 172.21.15.16:0/3051940172 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560078dd22e0 msgr2=0x560078dd6da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:06:59.966 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.959+0000 7f972dcae640 1 --2- 172.21.15.16:0/3051940172 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560078dd22e0 0x560078dd6da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:59.966 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.959+0000 7f972dcae640 1 -- 172.21.15.16:0/3051940172 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560078cdc340 con 0x560078dd19a0 2024-09-12T02:06:59.966 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.959+0000 7f971e7fc640 1 -- 172.21.15.16:0/558497916 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f97240cec30 con 0x7f97281003d0 2024-09-12T02:06:59.966 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.959+0000 7f972dcae640 1 --2- 172.21.15.16:0/558497916 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f97140a2c30 0x7f97140a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:59.966 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.959+0000 7f972dcae640 1 --2- 172.21.15.16:0/3051940172 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560078dd19a0 0x560078dd1da0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f972000afd0 tx=0x7f972000f850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43ee6b5aa3dba3f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:06:59.966 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.959+0000 7f97137fe640 1 -- 172.21.15.16:0/3051940172 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9720024430 con 0x560078dd19a0 2024-09-12T02:06:59.967 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.959+0000 7f97137fe640 1 -- 172.21.15.16:0/3051940172 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9720012110 con 0x560078dd19a0 2024-09-12T02:06:59.967 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.959+0000 7f97137fe640 1 -- 172.21.15.16:0/3051940172 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97200057c0 con 0x560078dd19a0 2024-09-12T02:06:59.967 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.960+0000 7f97324aa740 1 -- 172.21.15.16:0/3051940172 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560078dd19a0 msgr2=0x560078dd1da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:59.967 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.960+0000 7f97324aa740 1 --2- 172.21.15.16:0/3051940172 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560078dd19a0 0x560078dd1da0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f972000afd0 tx=0x7f972000f850 comp rx=0 tx=0).stop 2024-09-12T02:06:59.967 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.960+0000 7f972dcae640 1 --2- 172.21.15.16:0/558497916 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f97140a2c30 0x7f97140a50f0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f9720020000 tx=0x7f972001c030 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:06:59.967 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.960+0000 7f97324aa740 1 -- 172.21.15.16:0/3051940172 shutdown_connections 2024-09-12T02:06:59.967 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.960+0000 7f97324aa740 1 --2- 172.21.15.16:0/3051940172 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560078dd7370 0x560078dd9830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:59.967 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.960+0000 7f97324aa740 1 --2- 172.21.15.16:0/3051940172 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560078dd22e0 0x560078dd6da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:59.967 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.960+0000 7f97324aa740 1 --2- 172.21.15.16:0/3051940172 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560078dd19a0 0x560078dd1da0 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:59.967 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.960+0000 7f97324aa740 1 -- 172.21.15.16:0/3051940172 >> 172.21.15.16:0/3051940172 conn(0x560078dcd010 msgr2=0x560078dcd410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:06:59.967 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.960+0000 7f97324aa740 1 -- 172.21.15.16:0/3051940172 shutdown_connections 2024-09-12T02:06:59.967 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.960+0000 7f97324aa740 1 -- 172.21.15.16:0/3051940172 wait complete. 2024-09-12T02:06:59.967 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.960+0000 7f97324aa740 1 -- start start 2024-09-12T02:06:59.967 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.961+0000 7f97324aa740 1 Processor -- start 2024-09-12T02:06:59.968 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.961+0000 7f97324aa740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560078dd19a0 0x560078ddbb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:59.968 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.961+0000 7f97324aa740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560078dd22e0 0x560078ddc080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:59.968 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.961+0000 7f97324aa740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560078dd7370 0x560078de30c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:59.968 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.961+0000 7f97324aa740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x560078dda340 con 0x560078dd22e0 2024-09-12T02:06:59.968 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.961+0000 7f97324aa740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x560078cbe290 con 0x560078dd7370 2024-09-12T02:06:59.968 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.961+0000 7f97324aa740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x560078dda4c0 con 0x560078dd19a0 2024-09-12T02:06:59.968 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.961+0000 7f97324aa740 10 client.0 _ll_register_callbacks cb 0x560078cef140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:06:59.968 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.961+0000 7f972ccac640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560078dd22e0 0x560078ddc080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:59.968 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.961+0000 7f972ccac640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560078dd22e0 0x560078ddc080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42936/0 (socket says 172.21.15.16:42936) 2024-09-12T02:06:59.968 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.961+0000 7f972ccac640 1 -- 172.21.15.16:0/1540194954 learned_addr learned my addr 172.21.15.16:0/1540194954 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:06:59.968 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.961+0000 7f972d4ad640 1 --2- 172.21.15.16:0/1540194954 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560078dd19a0 0x560078ddbb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:59.968 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.961+0000 7f972dcae640 1 --2- 172.21.15.16:0/1540194954 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560078dd7370 0x560078de30c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:59.969 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.962+0000 7f972ccac640 1 -- 172.21.15.16:0/1540194954 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560078dd19a0 msgr2=0x560078ddbb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:06:59.969 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.962+0000 7f972ccac640 1 --2- 172.21.15.16:0/1540194954 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560078dd19a0 0x560078ddbb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:59.969 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.962+0000 7f972ccac640 1 -- 172.21.15.16:0/1540194954 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560078dd7370 msgr2=0x560078de30c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:06:59.969 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.962+0000 7f972ccac640 1 --2- 172.21.15.16:0/1540194954 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560078dd7370 0x560078de30c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:59.969 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.962+0000 7f972ccac640 1 -- 172.21.15.16:0/1540194954 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560078a48210 con 0x560078dd22e0 2024-09-12T02:06:59.969 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.962+0000 7f972d4ad640 1 --2- 172.21.15.16:0/1540194954 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560078dd19a0 0x560078ddbb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:06:59.969 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.962+0000 7f972ccac640 1 --2- 172.21.15.16:0/1540194954 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560078dd22e0 0x560078ddc080 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f972819b530 tx=0x7f9718005250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=442ea1479206d73 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:06:59.969 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.962+0000 7f9710ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:06:59.969 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.962+0000 7f9710ff9640 1 -- 172.21.15.16:0/1540194954 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9718035030 con 0x560078dd22e0 2024-09-12T02:06:59.969 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.962+0000 7f9710ff9640 1 -- 172.21.15.16:0/1540194954 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f97180318b0 con 0x560078dd22e0 2024-09-12T02:06:59.969 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.962+0000 7f97324aa740 1 -- 172.21.15.16:0/1540194954 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9720012110 con 0x560078dd22e0 2024-09-12T02:06:59.969 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.962+0000 7f9710ff9640 1 -- 172.21.15.16:0/1540194954 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9718005650 con 0x560078dd22e0 2024-09-12T02:06:59.969 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.962+0000 7f97324aa740 10 client.16953 fetch_fsmap learned FSMap version 25 2024-09-12T02:06:59.970 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.962+0000 7f97324aa740 1 -- 172.21.15.16:0/1540194954 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x560078b14620 con 0x560078dd22e0 2024-09-12T02:06:59.970 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.962+0000 7f9710ff9640 1 -- 172.21.15.16:0/1540194954 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9720012110 con 0x560078dd22e0 2024-09-12T02:06:59.970 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.962+0000 7f9710ff9640 1 -- 172.21.15.16:0/1540194954 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f97180035f0 con 0x560078dd22e0 2024-09-12T02:06:59.970 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.963+0000 7f97324aa740 10 client.16953 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:06:59.970 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.963+0000 7f97324aa740 10 client.16953 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:06:59.970 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.963+0000 7f97324aa740 20 client.16953 populate_metadata read hostname 'smithi016' 2024-09-12T02:06:59.970 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.963+0000 7f97324aa740 1 --2- 172.21.15.16:0/1540194954 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x560078de77b0 0x560078de9bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:59.970 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.963+0000 7f97324aa740 4 client.16953 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:06:59.970 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.963+0000 7f97324aa740 1 -- 172.21.15.16:0/1540194954 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560078d42a20 con 0x560078de77b0 2024-09-12T02:06:59.970 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.963+0000 7f972d4ad640 1 --2- 172.21.15.16:0/1540194954 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x560078de77b0 0x560078de9bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:59.970 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.963+0000 7f9710ff9640 10 client.16953 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:06:59.970 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.963+0000 7f972d4ad640 1 --2- 172.21.15.16:0/1540194954 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x560078de77b0 0x560078de9bc0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b9c55833af4c98d server_cookie=bcb87a757400553c in_seq=0 out_seq=0 2024-09-12T02:06:59.971 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.964+0000 7f9710ff9640 1 -- 172.21.15.16:0/1540194954 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560078d42a20 con 0x560078de77b0 2024-09-12T02:06:59.972 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.964+0000 7f9710ff9640 10 client.16953 handle_command_reply: tid=0 2024-09-12T02:06:59.972 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.964+0000 7f97324aa740 1 client.16953 shutdown 2024-09-12T02:06:59.972 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.964+0000 7f97324aa740 1 -- 172.21.15.16:0/1540194954 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560078dd22e0 msgr2=0x560078ddc080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:59.972 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.964+0000 7f97324aa740 1 --2- 172.21.15.16:0/1540194954 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560078dd22e0 0x560078ddc080 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f972819b530 tx=0x7f9718005250 comp rx=0 tx=0).stop 2024-09-12T02:06:59.972 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.965+0000 7f97324aa740 1 -- 172.21.15.16:0/1540194954 shutdown_connections 2024-09-12T02:06:59.972 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.965+0000 7f97324aa740 1 --2- 172.21.15.16:0/1540194954 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x560078de77b0 0x560078de9bc0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:59.972 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.965+0000 7f97324aa740 1 --2- 172.21.15.16:0/1540194954 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560078dd7370 0x560078de30c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:59.972 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.965+0000 7f97324aa740 1 --2- 172.21.15.16:0/1540194954 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560078dd22e0 0x560078ddc080 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:59.972 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.965+0000 7f9710ff9640 0 client.16953 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:06:59.972 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.965+0000 7f97324aa740 1 --2- 172.21.15.16:0/1540194954 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560078dd19a0 0x560078ddbb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:59.972 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.965+0000 7f97324aa740 1 -- 172.21.15.16:0/1540194954 >> 172.21.15.16:0/1540194954 conn(0x560078dcd010 msgr2=0x560078dce820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:06:59.973 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.965+0000 7f97324aa740 1 -- 172.21.15.16:0/1540194954 shutdown_connections 2024-09-12T02:06:59.973 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.965+0000 7f97324aa740 1 -- 172.21.15.16:0/1540194954 wait complete. 2024-09-12T02:06:59.973 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.965+0000 7f97324aa740 20 client.16953 trim_cache size 0 max 16384 2024-09-12T02:06:59.982 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.980+0000 7f97324aa740 1 Processor -- start 2024-09-12T02:06:59.982 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.980+0000 7f97324aa740 1 -- start start 2024-09-12T02:06:59.982 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.980+0000 7f97324aa740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560078dd19a0 0x560078dceb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:59.983 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.980+0000 7f97324aa740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560078dd22e0 0x560078de3080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:59.983 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.980+0000 7f97324aa740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560078dd7370 0x560078dee510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:59.983 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.980+0000 7f97324aa740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f97281ad910 con 0x560078dd7370 2024-09-12T02:06:59.983 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.980+0000 7f97324aa740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f97281ab1e0 con 0x560078dd19a0 2024-09-12T02:06:59.983 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.980+0000 7f97324aa740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f97281a9030 con 0x560078dd22e0 2024-09-12T02:06:59.983 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.980+0000 7f972dcae640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560078dd7370 0x560078dee510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:59.983 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.980+0000 7f972dcae640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560078dd7370 0x560078dee510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42938/0 (socket says 172.21.15.16:42938) 2024-09-12T02:06:59.983 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.980+0000 7f972dcae640 1 -- 172.21.15.16:0/271129581 learned_addr learned my addr 172.21.15.16:0/271129581 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:06:59.983 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.980+0000 7f972d4ad640 1 --2- 172.21.15.16:0/271129581 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560078dd19a0 0x560078dceb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:59.983 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.980+0000 7f972ccac640 1 --2- 172.21.15.16:0/271129581 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560078dd22e0 0x560078de3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:59.983 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.980+0000 7f972dcae640 1 -- 172.21.15.16:0/271129581 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560078dd22e0 msgr2=0x560078de3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:06:59.983 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.980+0000 7f972dcae640 1 --2- 172.21.15.16:0/271129581 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560078dd22e0 0x560078de3080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:59.983 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.980+0000 7f972dcae640 1 -- 172.21.15.16:0/271129581 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560078dd19a0 msgr2=0x560078dceb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:06:59.984 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.980+0000 7f972dcae640 1 --2- 172.21.15.16:0/271129581 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560078dd19a0 0x560078dceb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:59.984 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.980+0000 7f972dcae640 1 -- 172.21.15.16:0/271129581 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560078dcce30 con 0x560078dd7370 2024-09-12T02:06:59.984 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.980+0000 7f972dcae640 1 --2- 172.21.15.16:0/271129581 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560078dd7370 0x560078dee510 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f9720012510 tx=0x7f9720012540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cabb700304834a4f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:06:59.984 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.981+0000 7f9712ffd640 1 -- 172.21.15.16:0/271129581 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9720005a60 con 0x560078dd7370 2024-09-12T02:06:59.984 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.981+0000 7f9712ffd640 1 -- 172.21.15.16:0/271129581 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9720012ca0 con 0x560078dd7370 2024-09-12T02:06:59.984 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.981+0000 7f9712ffd640 1 -- 172.21.15.16:0/271129581 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9720004a00 con 0x560078dd7370 2024-09-12T02:06:59.984 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.981+0000 7f97324aa740 1 -- 172.21.15.16:0/271129581 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560078dd7370 msgr2=0x560078dee510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:59.984 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.981+0000 7f97324aa740 1 --2- 172.21.15.16:0/271129581 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560078dd7370 0x560078dee510 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f9720012510 tx=0x7f9720012540 comp rx=0 tx=0).stop 2024-09-12T02:06:59.984 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.981+0000 7f97324aa740 1 -- 172.21.15.16:0/271129581 shutdown_connections 2024-09-12T02:06:59.984 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.981+0000 7f97324aa740 1 --2- 172.21.15.16:0/271129581 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560078dd7370 0x560078dee510 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:59.984 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.981+0000 7f97324aa740 1 --2- 172.21.15.16:0/271129581 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560078dd22e0 0x560078de3080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:59.984 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.981+0000 7f97324aa740 1 --2- 172.21.15.16:0/271129581 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560078dd19a0 0x560078dceb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:59.985 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.981+0000 7f97324aa740 1 -- 172.21.15.16:0/271129581 >> 172.21.15.16:0/271129581 conn(0x560078dcd010 msgr2=0x560078dcd3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:06:59.985 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.981+0000 7f97324aa740 1 -- 172.21.15.16:0/271129581 shutdown_connections 2024-09-12T02:06:59.985 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.981+0000 7f97324aa740 1 -- 172.21.15.16:0/271129581 wait complete. 2024-09-12T02:06:59.985 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.982+0000 7f97324aa740 1 -- start start 2024-09-12T02:06:59.985 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.982+0000 7f97324aa740 1 Processor -- start 2024-09-12T02:06:59.987 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.982+0000 7f97324aa740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560078dd19a0 0x560078de29c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:59.988 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.982+0000 7f97324aa740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560078dd22e0 0x560078de2f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:59.988 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.982+0000 7f97324aa740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560078dd7370 0x560078de0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:59.988 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.982+0000 7f97324aa740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f97281a96a0 con 0x560078dd7370 2024-09-12T02:06:59.988 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.982+0000 7f97324aa740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f97281aab10 con 0x560078dd22e0 2024-09-12T02:06:59.988 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.982+0000 7f97324aa740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f97281a9d70 con 0x560078dd19a0 2024-09-12T02:06:59.988 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.982+0000 7f97324aa740 10 client.0 _ll_register_callbacks cb 0x7f9718009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:06:59.988 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.982+0000 7f972dcae640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560078dd7370 0x560078de0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:59.988 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.982+0000 7f972ccac640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560078dd22e0 0x560078de2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:59.988 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.982+0000 7f972dcae640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560078dd7370 0x560078de0fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42954/0 (socket says 172.21.15.16:42954) 2024-09-12T02:06:59.988 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.982+0000 7f972dcae640 1 -- 172.21.15.16:0/399974801 learned_addr learned my addr 172.21.15.16:0/399974801 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:06:59.988 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.982+0000 7f972d4ad640 1 --2- 172.21.15.16:0/399974801 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560078dd19a0 0x560078de29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:59.989 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.983+0000 7f972dcae640 1 -- 172.21.15.16:0/399974801 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560078dd19a0 msgr2=0x560078de29c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:06:59.989 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.983+0000 7f972dcae640 1 --2- 172.21.15.16:0/399974801 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560078dd19a0 0x560078de29c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:59.989 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.983+0000 7f972dcae640 1 -- 172.21.15.16:0/399974801 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560078dd22e0 msgr2=0x560078de2f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:06:59.989 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.983+0000 7f972dcae640 1 --2- 172.21.15.16:0/399974801 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560078dd22e0 0x560078de2f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:59.989 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.983+0000 7f972dcae640 1 -- 172.21.15.16:0/399974801 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560078de1800 con 0x560078dd7370 2024-09-12T02:06:59.989 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.983+0000 7f972dcae640 1 --2- 172.21.15.16:0/399974801 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560078dd7370 0x560078de0fd0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f9720005970 tx=0x7f9720012320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec6a32082e5052f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:06:59.989 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.983+0000 7f9710ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:06:59.989 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.983+0000 7f9710ff9640 1 -- 172.21.15.16:0/399974801 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9720005a60 con 0x560078dd7370 2024-09-12T02:06:59.989 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.983+0000 7f9710ff9640 1 -- 172.21.15.16:0/399974801 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f9720005240 con 0x560078dd7370 2024-09-12T02:06:59.989 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.983+0000 7f97324aa740 1 -- 172.21.15.16:0/399974801 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9720012ca0 con 0x560078dd7370 2024-09-12T02:06:59.989 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.983+0000 7f9710ff9640 1 -- 172.21.15.16:0/399974801 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9720004a00 con 0x560078dd7370 2024-09-12T02:06:59.989 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.983+0000 7f97324aa740 10 client.16959 fetch_fsmap learned FSMap version 25 2024-09-12T02:06:59.989 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.983+0000 7f97324aa740 1 -- 172.21.15.16:0/399974801 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x560078e5b7b0 con 0x560078dd7370 2024-09-12T02:06:59.990 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.983+0000 7f9710ff9640 1 -- 172.21.15.16:0/399974801 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9720012ca0 con 0x560078dd7370 2024-09-12T02:06:59.990 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.983+0000 7f9710ff9640 1 -- 172.21.15.16:0/399974801 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f9720041020 con 0x560078dd7370 2024-09-12T02:06:59.990 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.983+0000 7f97324aa740 10 client.16959 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:06:59.990 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.983+0000 7f97324aa740 10 client.16959 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:06:59.990 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.984+0000 7f97324aa740 20 client.16959 populate_metadata read hostname 'smithi016' 2024-09-12T02:06:59.990 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.984+0000 7f97324aa740 1 --2- 172.21.15.16:0/399974801 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x560078e5bf30 0x560078e5e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:59.990 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.984+0000 7f97324aa740 4 client.16959 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:06:59.990 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.984+0000 7f97324aa740 1 -- 172.21.15.16:0/399974801 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560078d3cd70 con 0x560078e5bf30 2024-09-12T02:06:59.990 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.984+0000 7f972d4ad640 1 --2- 172.21.15.16:0/399974801 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x560078e5bf30 0x560078e5e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:59.990 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.984+0000 7f9710ff9640 10 client.16959 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:06:59.990 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.984+0000 7f972d4ad640 1 --2- 172.21.15.16:0/399974801 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x560078e5bf30 0x560078e5e350 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6860281b608bc87 server_cookie=161d1d6fca7b2ab6 in_seq=0 out_seq=0 2024-09-12T02:06:59.999 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.997+0000 7f9710ff9640 1 -- 172.21.15.16:0/399974801 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560078d3cd70 con 0x560078e5bf30 2024-09-12T02:06:59.999 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.997+0000 7f9710ff9640 10 client.16959 handle_command_reply: tid=0 2024-09-12T02:06:59.999 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.997+0000 7f97324aa740 1 client.16959 shutdown 2024-09-12T02:06:59.999 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.997+0000 7f97324aa740 1 -- 172.21.15.16:0/399974801 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560078dd7370 msgr2=0x560078de0fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:59.999 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.997+0000 7f97324aa740 1 --2- 172.21.15.16:0/399974801 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560078dd7370 0x560078de0fd0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f9720005970 tx=0x7f9720012320 comp rx=0 tx=0).stop 2024-09-12T02:06:59.999 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.998+0000 7f97324aa740 1 -- 172.21.15.16:0/399974801 shutdown_connections 2024-09-12T02:06:59.999 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.998+0000 7f97324aa740 1 --2- 172.21.15.16:0/399974801 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x560078e5bf30 0x560078e5e350 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:59.999 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.998+0000 7f97324aa740 1 --2- 172.21.15.16:0/399974801 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560078dd7370 0x560078de0fd0 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:00.000 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.998+0000 7f97324aa740 1 --2- 172.21.15.16:0/399974801 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560078dd22e0 0x560078de2f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:00.000 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.998+0000 7f97324aa740 1 --2- 172.21.15.16:0/399974801 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560078dd19a0 0x560078de29c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:00.000 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.998+0000 7f9710ff9640 0 client.16959 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:07:00.000 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.998+0000 7f97324aa740 1 -- 172.21.15.16:0/399974801 >> 172.21.15.16:0/399974801 conn(0x560078dcd010 msgr2=0x560078dcdc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:07:00.000 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.998+0000 7f97324aa740 1 -- 172.21.15.16:0/399974801 shutdown_connections 2024-09-12T02:07:00.000 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.998+0000 7f97324aa740 1 -- 172.21.15.16:0/399974801 wait complete. 2024-09-12T02:07:00.000 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.998+0000 7f97324aa740 20 client.16959 trim_cache size 0 max 16384 2024-09-12T02:07:00.000 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-09-12T02:07:00.000 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-09-12T02:07:00.000 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-09-12T02:07:00.001 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-09-12T02:07:00.001 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.998+0000 7f972f711640 1 -- 172.21.15.16:0/558497916 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f97140a2c30 msgr2=0x7f97140a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:00.001 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.998+0000 7f972f711640 1 --2- 172.21.15.16:0/558497916 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f97140a2c30 0x7f97140a50f0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f9720020000 tx=0x7f972001c030 comp rx=0 tx=0).stop 2024-09-12T02:07:00.001 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.998+0000 7f972f711640 1 -- 172.21.15.16:0/558497916 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97281003d0 msgr2=0x7f972819a020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:00.002 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.998+0000 7f972f711640 1 --2- 172.21.15.16:0/558497916 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97281003d0 0x7f972819a020 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f972400d950 tx=0x7f972400de10 comp rx=0 tx=0).stop 2024-09-12T02:07:00.002 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.998+0000 7f972f711640 1 -- 172.21.15.16:0/558497916 shutdown_connections 2024-09-12T02:07:00.002 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.998+0000 7f972f711640 1 --2- 172.21.15.16:0/558497916 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f97140a2c30 0x7f97140a50f0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:00.002 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.998+0000 7f972f711640 1 --2- 172.21.15.16:0/558497916 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97281016b0 0x7f97281a15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:00.002 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.998+0000 7f972f711640 1 --2- 172.21.15.16:0/558497916 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9728100d10 0x7f972819a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:00.002 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.998+0000 7f972f711640 1 --2- 172.21.15.16:0/558497916 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97281003d0 0x7f972819a020 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:00.002 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.998+0000 7f972f711640 1 -- 172.21.15.16:0/558497916 >> 172.21.15.16:0/558497916 conn(0x7f97280fc1e0 msgr2=0x7f97280fdc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:07:00.002 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.999+0000 7f972f711640 1 -- 172.21.15.16:0/558497916 shutdown_connections 2024-09-12T02:07:00.002 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:06:59.999+0000 7f972f711640 1 -- 172.21.15.16:0/558497916 wait complete. 2024-09-12T02:07:00.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:07:00 smithi033 ceph-mon[35332]: pgmap v487: 97 pgs: 97 active+clean; 2.7 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 11 op/s 2024-09-12T02:07:00.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:07:00 smithi016 ceph-mon[30947]: pgmap v487: 97 pgs: 97 active+clean; 2.7 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 11 op/s 2024-09-12T02:07:00.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:07:00 smithi184 ceph-mon[35593]: pgmap v487: 97 pgs: 97 active+clean; 2.7 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 11 op/s 2024-09-12T02:07:02.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:07:02 smithi033 ceph-mon[35332]: pgmap v488: 97 pgs: 97 active+clean; 2.7 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-09-12T02:07:02.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:07:02 smithi016 ceph-mon[30947]: pgmap v488: 97 pgs: 97 active+clean; 2.7 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-09-12T02:07:02.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:07:02 smithi184 ceph-mon[35593]: pgmap v488: 97 pgs: 97 active+clean; 2.7 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-09-12T02:07:04.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:07:04 smithi033 ceph-mon[35332]: pgmap v489: 97 pgs: 97 active+clean; 2.7 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-09-12T02:07:04.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:07:04 smithi016 ceph-mon[30947]: pgmap v489: 97 pgs: 97 active+clean; 2.7 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-09-12T02:07:04.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:07:04 smithi184 ceph-mon[35593]: pgmap v489: 97 pgs: 97 active+clean; 2.7 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-09-12T02:07:06.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:07:06 smithi033 ceph-mon[35332]: pgmap v490: 97 pgs: 97 active+clean; 2.8 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-09-12T02:07:06.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:07:06 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:07:06.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:07:06 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:07:06.420 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:07:06 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:07:06.420 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:07:06 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T02:07:06.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:07:06 smithi016 ceph-mon[30947]: pgmap v490: 97 pgs: 97 active+clean; 2.8 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-09-12T02:07:06.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:07:06 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:07:06.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:07:06 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:07:06.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:07:06 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:07:06.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:07:06 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T02:07:06.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:07:06 smithi184 ceph-mon[35593]: pgmap v490: 97 pgs: 97 active+clean; 2.8 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-09-12T02:07:06.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:07:06 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:07:06.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:07:06 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:07:06.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:07:06 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:07:06.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:07:06 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T02:07:08.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:07:08 smithi033 ceph-mon[35332]: pgmap v491: 97 pgs: 97 active+clean; 2.8 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-09-12T02:07:08.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:07:08 smithi016 ceph-mon[30947]: pgmap v491: 97 pgs: 97 active+clean; 2.8 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-09-12T02:07:08.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:07:08 smithi184 ceph-mon[35593]: pgmap v491: 97 pgs: 97 active+clean; 2.8 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-09-12T02:07:10.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:07:10 smithi033 ceph-mon[35332]: pgmap v492: 97 pgs: 97 active+clean; 2.8 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-09-12T02:07:10.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:07:10 smithi016 ceph-mon[30947]: pgmap v492: 97 pgs: 97 active+clean; 2.8 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-09-12T02:07:10.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:07:10 smithi184 ceph-mon[35593]: pgmap v492: 97 pgs: 97 active+clean; 2.8 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-09-12T02:07:11.605 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T02:07:11.877 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:07:12.249 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:07:12 smithi016 ceph-mon[30947]: pgmap v493: 97 pgs: 97 active+clean; 2.8 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-09-12T02:07:12.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.337+0000 7f5de87e9640 1 -- 172.21.15.16:0/977069393 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5de01082e0 msgr2=0x7f5de010ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:12.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.337+0000 7f5de87e9640 1 --2- 172.21.15.16:0/977069393 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5de01082e0 0x7f5de010ec50 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f5dd8009cd0 tx=0x7f5dd802a8e0 comp rx=0 tx=0).stop 2024-09-12T02:07:12.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.337+0000 7f5de87e9640 1 -- 172.21.15.16:0/977069393 shutdown_connections 2024-09-12T02:07:12.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.337+0000 7f5de87e9640 1 --2- 172.21.15.16:0/977069393 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5de01082e0 0x7f5de010ec50 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:12.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.337+0000 7f5de87e9640 1 --2- 172.21.15.16:0/977069393 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5de0103760 0x7f5de0107ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:12.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.337+0000 7f5de87e9640 1 --2- 172.21.15.16:0/977069393 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5de0102db0 0x7f5de0103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:12.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.337+0000 7f5de87e9640 1 -- 172.21.15.16:0/977069393 >> 172.21.15.16:0/977069393 conn(0x7f5de00fc7d0 msgr2=0x7f5de00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:07:12.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.337+0000 7f5de87e9640 1 -- 172.21.15.16:0/977069393 shutdown_connections 2024-09-12T02:07:12.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.337+0000 7f5de87e9640 1 -- 172.21.15.16:0/977069393 wait complete. 2024-09-12T02:07:12.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.337+0000 7f5de87e9640 1 Processor -- start 2024-09-12T02:07:12.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.338+0000 7f5de87e9640 1 -- start start 2024-09-12T02:07:12.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.338+0000 7f5de87e9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5de0102db0 0x7f5de0198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:12.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.338+0000 7f5de87e9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5de0103760 0x7f5de0198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:12.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.338+0000 7f5de87e9640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5de01082e0 0x7f5de019ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:12.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.338+0000 7f5de87e9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5de0112100 con 0x7f5de0102db0 2024-09-12T02:07:12.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.338+0000 7f5de87e9640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5de0111f80 con 0x7f5de0103760 2024-09-12T02:07:12.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.338+0000 7f5de87e9640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5de0112280 con 0x7f5de01082e0 2024-09-12T02:07:12.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.338+0000 7f5de6585640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5de0102db0 0x7f5de0198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:12.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.338+0000 7f5de6585640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5de0102db0 0x7f5de0198570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49302/0 (socket says 172.21.15.16:49302) 2024-09-12T02:07:12.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.338+0000 7f5de6585640 1 -- 172.21.15.16:0/1298709824 learned_addr learned my addr 172.21.15.16:0/1298709824 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:07:12.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.338+0000 7f5de5d84640 1 --2- 172.21.15.16:0/1298709824 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5de0103760 0x7f5de0198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:12.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.338+0000 7f5de6d86640 1 --2- 172.21.15.16:0/1298709824 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5de01082e0 0x7f5de019ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:12.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.338+0000 7f5de6585640 1 -- 172.21.15.16:0/1298709824 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5de01082e0 msgr2=0x7f5de019ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:12.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.338+0000 7f5de6585640 1 --2- 172.21.15.16:0/1298709824 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5de01082e0 0x7f5de019ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:12.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.338+0000 7f5de6585640 1 -- 172.21.15.16:0/1298709824 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5de0103760 msgr2=0x7f5de0198ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:12.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.338+0000 7f5de6585640 1 --2- 172.21.15.16:0/1298709824 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5de0103760 0x7f5de0198ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:12.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.338+0000 7f5de6585640 1 -- 172.21.15.16:0/1298709824 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5de019d520 con 0x7f5de0102db0 2024-09-12T02:07:12.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.339+0000 7f5de6585640 1 --2- 172.21.15.16:0/1298709824 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5de0102db0 0x7f5de0198570 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f5ddc00e9f0 tx=0x7f5ddc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:07:12.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.339+0000 7f5dd77fe640 1 -- 172.21.15.16:0/1298709824 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ddc00cd80 con 0x7f5de0102db0 2024-09-12T02:07:12.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.339+0000 7f5de87e9640 1 -- 172.21.15.16:0/1298709824 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5de019d810 con 0x7f5de0102db0 2024-09-12T02:07:12.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.339+0000 7f5dd77fe640 1 -- 172.21.15.16:0/1298709824 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ddc005500 con 0x7f5de0102db0 2024-09-12T02:07:12.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.339+0000 7f5dd77fe640 1 -- 172.21.15.16:0/1298709824 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ddc010690 con 0x7f5de0102db0 2024-09-12T02:07:12.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.339+0000 7f5de87e9640 1 -- 172.21.15.16:0/1298709824 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5de01a50f0 con 0x7f5de0102db0 2024-09-12T02:07:12.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.341+0000 7f5dd77fe640 1 -- 172.21.15.16:0/1298709824 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5ddc0046c0 con 0x7f5de0102db0 2024-09-12T02:07:12.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.342+0000 7f5dd77fe640 1 --2- 172.21.15.16:0/1298709824 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f5dcc0a2b80 0x7f5dcc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:12.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.342+0000 7f5dd77fe640 1 -- 172.21.15.16:0/1298709824 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f5ddc0cf380 con 0x7f5de0102db0 2024-09-12T02:07:12.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.342+0000 7f5de5d84640 1 --2- 172.21.15.16:0/1298709824 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f5dcc0a2b80 0x7f5dcc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:12.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.342+0000 7f5de5d84640 1 --2- 172.21.15.16:0/1298709824 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f5dcc0a2b80 0x7f5dcc0a5040 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f5de00fedf0 tx=0x7f5dd000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:07:12.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.343+0000 7f5deabc2740 1 Processor -- start 2024-09-12T02:07:12.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.343+0000 7f5deabc2740 1 -- start start 2024-09-12T02:07:12.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.343+0000 7f5deabc2740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56330cf724a0 0x56330cf72880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:12.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.343+0000 7f5deabc2740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56330cf72dc0 0x56330cf7d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:12.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.343+0000 7f5deabc2740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56330cf7d680 0x56330cf7fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:12.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.343+0000 7f5deabc2740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56330ced82e0 con 0x56330cf7d680 2024-09-12T02:07:12.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.343+0000 7f5deabc2740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56330ce7fce0 con 0x56330cf72dc0 2024-09-12T02:07:12.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.343+0000 7f5deabc2740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x56330ced42a0 con 0x56330cf724a0 2024-09-12T02:07:12.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.343+0000 7f5de6d86640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56330cf7d680 0x56330cf7fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:12.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.343+0000 7f5de6d86640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56330cf7d680 0x56330cf7fa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49304/0 (socket says 172.21.15.16:49304) 2024-09-12T02:07:12.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.343+0000 7f5de6d86640 1 -- 172.21.15.16:0/2621695154 learned_addr learned my addr 172.21.15.16:0/2621695154 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:07:12.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.344+0000 7f5de5d84640 1 --2- 172.21.15.16:0/2621695154 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56330cf72dc0 0x56330cf7d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:12.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.344+0000 7f5de6585640 1 --2- 172.21.15.16:0/2621695154 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56330cf724a0 0x56330cf72880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:12.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.344+0000 7f5de6d86640 1 -- 172.21.15.16:0/2621695154 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56330cf724a0 msgr2=0x56330cf72880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:12.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.344+0000 7f5de6d86640 1 --2- 172.21.15.16:0/2621695154 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56330cf724a0 0x56330cf72880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:12.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.344+0000 7f5de6d86640 1 -- 172.21.15.16:0/2621695154 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56330cf72dc0 msgr2=0x56330cf7d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:12.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.344+0000 7f5de6d86640 1 --2- 172.21.15.16:0/2621695154 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56330cf72dc0 0x56330cf7d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:12.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.344+0000 7f5de6d86640 1 -- 172.21.15.16:0/2621695154 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56330ce64440 con 0x56330cf7d680 2024-09-12T02:07:12.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.344+0000 7f5de5d84640 1 --2- 172.21.15.16:0/2621695154 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56330cf72dc0 0x56330cf7d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:07:12.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.344+0000 7f5de6d86640 1 --2- 172.21.15.16:0/2621695154 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56330cf7d680 0x56330cf7fa70 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f5dd8009cd0 tx=0x7f5dd80093e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=967866481b26ceb3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:07:12.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.344+0000 7f5dd4ff9640 1 -- 172.21.15.16:0/2621695154 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd8048070 con 0x56330cf7d680 2024-09-12T02:07:12.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.344+0000 7f5dd4ff9640 1 -- 172.21.15.16:0/2621695154 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5dd8036070 con 0x56330cf7d680 2024-09-12T02:07:12.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.344+0000 7f5dd4ff9640 1 -- 172.21.15.16:0/2621695154 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd8004330 con 0x56330cf7d680 2024-09-12T02:07:12.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.345+0000 7f5deabc2740 1 -- 172.21.15.16:0/2621695154 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56330cf7d680 msgr2=0x56330cf7fa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:12.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.345+0000 7f5deabc2740 1 --2- 172.21.15.16:0/2621695154 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56330cf7d680 0x56330cf7fa70 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f5dd8009cd0 tx=0x7f5dd80093e0 comp rx=0 tx=0).stop 2024-09-12T02:07:12.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.345+0000 7f5deabc2740 1 -- 172.21.15.16:0/2621695154 shutdown_connections 2024-09-12T02:07:12.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.345+0000 7f5deabc2740 1 --2- 172.21.15.16:0/2621695154 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56330cf7d680 0x56330cf7fa70 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:12.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.345+0000 7f5deabc2740 1 --2- 172.21.15.16:0/2621695154 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56330cf72dc0 0x56330cf7d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:12.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.345+0000 7f5deabc2740 1 --2- 172.21.15.16:0/2621695154 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56330cf724a0 0x56330cf72880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:12.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.345+0000 7f5deabc2740 1 -- 172.21.15.16:0/2621695154 >> 172.21.15.16:0/2621695154 conn(0x56330cf6ea30 msgr2=0x56330cf6ee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:07:12.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.345+0000 7f5deabc2740 1 -- 172.21.15.16:0/2621695154 shutdown_connections 2024-09-12T02:07:12.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.345+0000 7f5deabc2740 1 -- 172.21.15.16:0/2621695154 wait complete. 2024-09-12T02:07:12.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.346+0000 7f5deabc2740 1 -- start start 2024-09-12T02:07:12.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.346+0000 7f5deabc2740 1 Processor -- start 2024-09-12T02:07:12.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.346+0000 7f5deabc2740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56330cf724a0 0x56330cf7cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:12.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.346+0000 7f5deabc2740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56330cf72dc0 0x56330cf7a250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:12.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.346+0000 7f5deabc2740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56330cf7d680 0x56330cf7a790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:12.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.346+0000 7f5deabc2740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56330cf80250 con 0x56330cf7d680 2024-09-12T02:07:12.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.346+0000 7f5deabc2740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56330ce93050 con 0x56330cf724a0 2024-09-12T02:07:12.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.346+0000 7f5deabc2740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x56330cf803d0 con 0x56330cf72dc0 2024-09-12T02:07:12.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.346+0000 7f5deabc2740 10 client.0 _ll_register_callbacks cb 0x56330ced3160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:07:12.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.347+0000 7f5de6d86640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56330cf7d680 0x56330cf7a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:12.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.347+0000 7f5de6d86640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56330cf7d680 0x56330cf7a790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49314/0 (socket says 172.21.15.16:49314) 2024-09-12T02:07:12.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.347+0000 7f5de6d86640 1 -- 172.21.15.16:0/2199273679 learned_addr learned my addr 172.21.15.16:0/2199273679 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:07:12.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.347+0000 7f5de5d84640 1 --2- 172.21.15.16:0/2199273679 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56330cf72dc0 0x56330cf7a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:12.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.347+0000 7f5de6585640 1 --2- 172.21.15.16:0/2199273679 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56330cf724a0 0x56330cf7cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:12.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.347+0000 7f5de6d86640 1 -- 172.21.15.16:0/2199273679 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56330cf72dc0 msgr2=0x56330cf7a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:12.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.347+0000 7f5de6d86640 1 --2- 172.21.15.16:0/2199273679 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56330cf72dc0 0x56330cf7a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:12.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.347+0000 7f5de6d86640 1 -- 172.21.15.16:0/2199273679 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56330cf724a0 msgr2=0x56330cf7cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:12.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.347+0000 7f5de6d86640 1 --2- 172.21.15.16:0/2199273679 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56330cf724a0 0x56330cf7cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:12.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.347+0000 7f5de6d86640 1 -- 172.21.15.16:0/2199273679 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56330cc493c0 con 0x56330cf7d680 2024-09-12T02:07:12.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.347+0000 7f5de6d86640 1 --2- 172.21.15.16:0/2199273679 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56330cf7d680 0x56330cf7a790 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f5dd8009ca0 tx=0x7f5dd8031b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbdca4a0360a516 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:07:12.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.347+0000 7f5dc9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:07:12.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.347+0000 7f5dc9ffb640 1 -- 172.21.15.16:0/2199273679 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd8052070 con 0x56330cf7d680 2024-09-12T02:07:12.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.347+0000 7f5dc9ffb640 1 -- 172.21.15.16:0/2199273679 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f5dd8003b70 con 0x56330cf7d680 2024-09-12T02:07:12.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.347+0000 7f5deabc2740 1 -- 172.21.15.16:0/2199273679 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5dd8036070 con 0x56330cf7d680 2024-09-12T02:07:12.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.348+0000 7f5dc9ffb640 1 -- 172.21.15.16:0/2199273679 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5dd805e050 con 0x56330cf7d680 2024-09-12T02:07:12.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.348+0000 7f5deabc2740 10 client.16971 fetch_fsmap learned FSMap version 25 2024-09-12T02:07:12.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.348+0000 7f5deabc2740 1 -- 172.21.15.16:0/2199273679 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56330cf80250 con 0x56330cf7d680 2024-09-12T02:07:12.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.348+0000 7f5dc9ffb640 1 -- 172.21.15.16:0/2199273679 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5dd8036070 con 0x56330cf7d680 2024-09-12T02:07:12.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.348+0000 7f5dc9ffb640 1 -- 172.21.15.16:0/2199273679 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f5dd8042030 con 0x56330cf7d680 2024-09-12T02:07:12.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.348+0000 7f5deabc2740 10 client.16971 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:07:12.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.348+0000 7f5deabc2740 10 client.16971 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:07:12.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.348+0000 7f5deabc2740 20 client.16971 populate_metadata read hostname 'smithi016' 2024-09-12T02:07:12.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.348+0000 7f5deabc2740 1 --2- 172.21.15.16:0/2199273679 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x56330cf8cd50 0x56330cf8f140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:12.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.348+0000 7f5deabc2740 4 client.16971 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:07:12.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.348+0000 7f5deabc2740 1 -- 172.21.15.16:0/2199273679 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56330cee5850 con 0x56330cf8cd50 2024-09-12T02:07:12.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.348+0000 7f5de6585640 1 --2- 172.21.15.16:0/2199273679 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x56330cf8cd50 0x56330cf8f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:12.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.349+0000 7f5dc9ffb640 10 client.16971 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:07:12.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.349+0000 7f5de6585640 1 --2- 172.21.15.16:0/2199273679 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x56330cf8cd50 0x56330cf8f140 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ceb0eefec974f737 server_cookie=5a969beac153b673 in_seq=0 out_seq=0 2024-09-12T02:07:12.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.349+0000 7f5dc9ffb640 1 -- 172.21.15.16:0/2199273679 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56330cee5850 con 0x56330cf8cd50 2024-09-12T02:07:12.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.349+0000 7f5dc9ffb640 10 client.16971 handle_command_reply: tid=0 2024-09-12T02:07:12.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.349+0000 7f5deabc2740 1 client.16971 shutdown 2024-09-12T02:07:12.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.350+0000 7f5deabc2740 1 -- 172.21.15.16:0/2199273679 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56330cf7d680 msgr2=0x56330cf7a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:12.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.350+0000 7f5deabc2740 1 --2- 172.21.15.16:0/2199273679 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56330cf7d680 0x56330cf7a790 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f5dd8009ca0 tx=0x7f5dd8031b90 comp rx=0 tx=0).stop 2024-09-12T02:07:12.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.350+0000 7f5deabc2740 1 -- 172.21.15.16:0/2199273679 shutdown_connections 2024-09-12T02:07:12.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.350+0000 7f5deabc2740 1 --2- 172.21.15.16:0/2199273679 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x56330cf8cd50 0x56330cf8f140 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:12.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.350+0000 7f5deabc2740 1 --2- 172.21.15.16:0/2199273679 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56330cf7d680 0x56330cf7a790 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:12.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.350+0000 7f5deabc2740 1 --2- 172.21.15.16:0/2199273679 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56330cf72dc0 0x56330cf7a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:12.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.350+0000 7f5deabc2740 1 --2- 172.21.15.16:0/2199273679 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56330cf724a0 0x56330cf7cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:12.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.350+0000 7f5deabc2740 1 -- 172.21.15.16:0/2199273679 >> 172.21.15.16:0/2199273679 conn(0x56330cf6ea30 msgr2=0x56330cf74710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:07:12.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.350+0000 7f5dc9ffb640 0 client.16971 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:07:12.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.350+0000 7f5deabc2740 1 -- 172.21.15.16:0/2199273679 shutdown_connections 2024-09-12T02:07:12.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.350+0000 7f5deabc2740 1 -- 172.21.15.16:0/2199273679 wait complete. 2024-09-12T02:07:12.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.350+0000 7f5deabc2740 20 client.16971 trim_cache size 0 max 16384 2024-09-12T02:07:12.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.367+0000 7f5deabc2740 1 Processor -- start 2024-09-12T02:07:12.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.367+0000 7f5deabc2740 1 -- start start 2024-09-12T02:07:12.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.368+0000 7f5deabc2740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56330cf724a0 0x56330cf894b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:12.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.368+0000 7f5deabc2740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56330cf72dc0 0x56330cf948b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:12.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.368+0000 7f5deabc2740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56330cf7d680 0x56330cf94df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:12.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.368+0000 7f5deabc2740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5de01afc50 con 0x56330cf72dc0 2024-09-12T02:07:12.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.368+0000 7f5deabc2740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5de01ad520 con 0x56330cf7d680 2024-09-12T02:07:12.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.368+0000 7f5deabc2740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5de01ab370 con 0x56330cf724a0 2024-09-12T02:07:12.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.368+0000 7f5de5d84640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56330cf72dc0 0x56330cf948b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:12.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.368+0000 7f5de5d84640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56330cf72dc0 0x56330cf948b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49320/0 (socket says 172.21.15.16:49320) 2024-09-12T02:07:12.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.368+0000 7f5de5d84640 1 -- 172.21.15.16:0/2753544668 learned_addr learned my addr 172.21.15.16:0/2753544668 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:07:12.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.368+0000 7f5de6585640 1 --2- 172.21.15.16:0/2753544668 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56330cf724a0 0x56330cf894b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:12.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.368+0000 7f5de6d86640 1 --2- 172.21.15.16:0/2753544668 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56330cf7d680 0x56330cf94df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:12.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.368+0000 7f5de5d84640 1 -- 172.21.15.16:0/2753544668 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56330cf724a0 msgr2=0x56330cf894b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:12.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.368+0000 7f5de5d84640 1 --2- 172.21.15.16:0/2753544668 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56330cf724a0 0x56330cf894b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:12.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.368+0000 7f5de5d84640 1 -- 172.21.15.16:0/2753544668 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56330cf7d680 msgr2=0x56330cf94df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:12.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.368+0000 7f5de5d84640 1 --2- 172.21.15.16:0/2753544668 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56330cf7d680 0x56330cf94df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:12.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.368+0000 7f5de5d84640 1 -- 172.21.15.16:0/2753544668 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56330cf722d0 con 0x56330cf72dc0 2024-09-12T02:07:12.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.368+0000 7f5de5d84640 1 --2- 172.21.15.16:0/2753544668 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56330cf72dc0 0x56330cf948b0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f5de006a2f0 tx=0x7f5dd0013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7d8a88302df8f3b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:07:12.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.369+0000 7f5dcbfff640 1 -- 172.21.15.16:0/2753544668 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd0005020 con 0x56330cf72dc0 2024-09-12T02:07:12.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.369+0000 7f5dcbfff640 1 -- 172.21.15.16:0/2753544668 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5dd00056a0 con 0x56330cf72dc0 2024-09-12T02:07:12.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.369+0000 7f5dcbfff640 1 -- 172.21.15.16:0/2753544668 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd00296d0 con 0x56330cf72dc0 2024-09-12T02:07:12.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.369+0000 7f5deabc2740 1 -- 172.21.15.16:0/2753544668 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56330cf72dc0 msgr2=0x56330cf948b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:12.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.369+0000 7f5deabc2740 1 --2- 172.21.15.16:0/2753544668 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56330cf72dc0 0x56330cf948b0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f5de006a2f0 tx=0x7f5dd0013000 comp rx=0 tx=0).stop 2024-09-12T02:07:12.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.369+0000 7f5deabc2740 1 -- 172.21.15.16:0/2753544668 shutdown_connections 2024-09-12T02:07:12.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.369+0000 7f5deabc2740 1 --2- 172.21.15.16:0/2753544668 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56330cf7d680 0x56330cf94df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:12.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.370+0000 7f5deabc2740 1 --2- 172.21.15.16:0/2753544668 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56330cf72dc0 0x56330cf948b0 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:12.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.370+0000 7f5deabc2740 1 --2- 172.21.15.16:0/2753544668 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56330cf724a0 0x56330cf894b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:12.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.370+0000 7f5deabc2740 1 -- 172.21.15.16:0/2753544668 >> 172.21.15.16:0/2753544668 conn(0x56330cf6ea30 msgr2=0x56330cf79340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:07:12.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.370+0000 7f5deabc2740 1 -- 172.21.15.16:0/2753544668 shutdown_connections 2024-09-12T02:07:12.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.370+0000 7f5deabc2740 1 -- 172.21.15.16:0/2753544668 wait complete. 2024-09-12T02:07:12.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.370+0000 7f5deabc2740 1 -- start start 2024-09-12T02:07:12.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.370+0000 7f5deabc2740 1 Processor -- start 2024-09-12T02:07:12.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.370+0000 7f5deabc2740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56330cf724a0 0x56330cf85e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:12.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.370+0000 7f5deabc2740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56330cf7d680 0x56330cf810a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:12.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.370+0000 7f5deabc2740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56330cf815e0 0x56330cf81a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:12.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.370+0000 7f5deabc2740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5de01ab9e0 con 0x56330cf7d680 2024-09-12T02:07:12.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.370+0000 7f5deabc2740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5de01ace50 con 0x56330cf724a0 2024-09-12T02:07:12.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.370+0000 7f5deabc2740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5de01ac0b0 con 0x56330cf815e0 2024-09-12T02:07:12.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.370+0000 7f5deabc2740 10 client.0 _ll_register_callbacks cb 0x7f5dd8046070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:07:12.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.371+0000 7f5de5d84640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56330cf7d680 0x56330cf810a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:12.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.371+0000 7f5de5d84640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56330cf7d680 0x56330cf810a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49332/0 (socket says 172.21.15.16:49332) 2024-09-12T02:07:12.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.371+0000 7f5de5d84640 1 -- 172.21.15.16:0/2046274132 learned_addr learned my addr 172.21.15.16:0/2046274132 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:07:12.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.371+0000 7f5de6585640 1 --2- 172.21.15.16:0/2046274132 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56330cf724a0 0x56330cf85e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:12.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.371+0000 7f5de6d86640 1 --2- 172.21.15.16:0/2046274132 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56330cf815e0 0x56330cf81a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:12.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.371+0000 7f5de5d84640 1 -- 172.21.15.16:0/2046274132 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56330cf815e0 msgr2=0x56330cf81a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:12.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.371+0000 7f5de5d84640 1 --2- 172.21.15.16:0/2046274132 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56330cf815e0 0x56330cf81a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:12.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.371+0000 7f5de5d84640 1 -- 172.21.15.16:0/2046274132 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56330cf724a0 msgr2=0x56330cf85e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:12.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.371+0000 7f5de5d84640 1 --2- 172.21.15.16:0/2046274132 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56330cf724a0 0x56330cf85e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:12.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.371+0000 7f5de5d84640 1 -- 172.21.15.16:0/2046274132 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56330cf6f6c0 con 0x56330cf7d680 2024-09-12T02:07:12.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.371+0000 7f5de6585640 1 --2- 172.21.15.16:0/2046274132 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56330cf724a0 0x56330cf85e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:07:12.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.371+0000 7f5de5d84640 1 --2- 172.21.15.16:0/2046274132 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56330cf7d680 0x56330cf810a0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f5dd000b940 tx=0x7f5dd0010ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c3f92201227f20f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:07:12.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.371+0000 7f5dc9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:07:12.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.371+0000 7f5dc9ffb640 1 -- 172.21.15.16:0/2046274132 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd0005340 con 0x56330cf7d680 2024-09-12T02:07:12.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.371+0000 7f5dc9ffb640 1 -- 172.21.15.16:0/2046274132 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f5dd0025040 con 0x56330cf7d680 2024-09-12T02:07:12.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.371+0000 7f5deabc2740 1 -- 172.21.15.16:0/2046274132 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5dd00056a0 con 0x56330cf7d680 2024-09-12T02:07:12.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.372+0000 7f5dc9ffb640 1 -- 172.21.15.16:0/2046274132 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5dd00054e0 con 0x56330cf7d680 2024-09-12T02:07:12.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.372+0000 7f5deabc2740 10 client.16977 fetch_fsmap learned FSMap version 25 2024-09-12T02:07:12.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.372+0000 7f5deabc2740 1 -- 172.21.15.16:0/2046274132 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56330cff7b40 con 0x56330cf7d680 2024-09-12T02:07:12.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.372+0000 7f5dc9ffb640 1 -- 172.21.15.16:0/2046274132 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5dd00056a0 con 0x56330cf7d680 2024-09-12T02:07:12.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.372+0000 7f5dc9ffb640 1 -- 172.21.15.16:0/2046274132 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f5dd0034420 con 0x56330cf7d680 2024-09-12T02:07:12.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.372+0000 7f5deabc2740 10 client.16977 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:07:12.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.372+0000 7f5deabc2740 10 client.16977 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:07:12.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.372+0000 7f5deabc2740 20 client.16977 populate_metadata read hostname 'smithi016' 2024-09-12T02:07:12.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.372+0000 7f5deabc2740 1 --2- 172.21.15.16:0/2046274132 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x56330cff8230 0x56330cffa650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:12.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.372+0000 7f5deabc2740 4 client.16977 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T02:07:12.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.372+0000 7f5deabc2740 1 -- 172.21.15.16:0/2046274132 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56330cedfd60 con 0x56330cff8230 2024-09-12T02:07:12.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.372+0000 7f5de6585640 1 --2- 172.21.15.16:0/2046274132 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x56330cff8230 0x56330cffa650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:12.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.373+0000 7f5dc9ffb640 10 client.16977 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:07:12.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.373+0000 7f5de6585640 1 --2- 172.21.15.16:0/2046274132 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x56330cff8230 0x56330cffa650 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e49482a991ce3e19 server_cookie=954b5a793716b84e in_seq=0 out_seq=0 2024-09-12T02:07:12.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.373+0000 7f5dc9ffb640 1 -- 172.21.15.16:0/2046274132 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56330cedfd60 con 0x56330cff8230 2024-09-12T02:07:12.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.373+0000 7f5dc9ffb640 10 client.16977 handle_command_reply: tid=0 2024-09-12T02:07:12.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.373+0000 7f5deabc2740 1 client.16977 shutdown 2024-09-12T02:07:12.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.373+0000 7f5deabc2740 1 -- 172.21.15.16:0/2046274132 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56330cf7d680 msgr2=0x56330cf810a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:12.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.373+0000 7f5deabc2740 1 --2- 172.21.15.16:0/2046274132 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56330cf7d680 0x56330cf810a0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f5dd000b940 tx=0x7f5dd0010ee0 comp rx=0 tx=0).stop 2024-09-12T02:07:12.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.373+0000 7f5deabc2740 1 -- 172.21.15.16:0/2046274132 shutdown_connections 2024-09-12T02:07:12.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.373+0000 7f5deabc2740 1 --2- 172.21.15.16:0/2046274132 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x56330cff8230 0x56330cffa650 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:12.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.373+0000 7f5deabc2740 1 --2- 172.21.15.16:0/2046274132 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56330cf815e0 0x56330cf81a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:12.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.373+0000 7f5dc9ffb640 0 client.16977 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:07:12.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.373+0000 7f5deabc2740 1 --2- 172.21.15.16:0/2046274132 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56330cf7d680 0x56330cf810a0 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:12.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.373+0000 7f5deabc2740 1 --2- 172.21.15.16:0/2046274132 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56330cf724a0 0x56330cf85e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:12.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.373+0000 7f5deabc2740 1 -- 172.21.15.16:0/2046274132 >> 172.21.15.16:0/2046274132 conn(0x56330cf6ea30 msgr2=0x56330cf75030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:07:12.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.374+0000 7f5deabc2740 1 -- 172.21.15.16:0/2046274132 shutdown_connections 2024-09-12T02:07:12.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.374+0000 7f5deabc2740 1 -- 172.21.15.16:0/2046274132 wait complete. 2024-09-12T02:07:12.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.374+0000 7f5deabc2740 20 client.16977 trim_cache size 0 max 16384 2024-09-12T02:07:12.386 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T02:07:12.386 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-09-12T02:07:12.387 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {} 2024-09-12T02:07:12.387 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T02:07:12.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.374+0000 7f5de87e9640 1 -- 172.21.15.16:0/1298709824 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f5dcc0a2b80 msgr2=0x7f5dcc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:12.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.374+0000 7f5de87e9640 1 --2- 172.21.15.16:0/1298709824 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f5dcc0a2b80 0x7f5dcc0a5040 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f5de00fedf0 tx=0x7f5dd000b390 comp rx=0 tx=0).stop 2024-09-12T02:07:12.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.374+0000 7f5de87e9640 1 -- 172.21.15.16:0/1298709824 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5de0102db0 msgr2=0x7f5de0198570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:12.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.374+0000 7f5de87e9640 1 --2- 172.21.15.16:0/1298709824 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5de0102db0 0x7f5de0198570 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f5ddc00e9f0 tx=0x7f5ddc00edf0 comp rx=0 tx=0).stop 2024-09-12T02:07:12.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.374+0000 7f5de87e9640 1 -- 172.21.15.16:0/1298709824 shutdown_connections 2024-09-12T02:07:12.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.374+0000 7f5de87e9640 1 --2- 172.21.15.16:0/1298709824 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f5dcc0a2b80 0x7f5dcc0a5040 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:12.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.374+0000 7f5de87e9640 1 --2- 172.21.15.16:0/1298709824 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5de01082e0 0x7f5de019ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:12.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.374+0000 7f5de87e9640 1 --2- 172.21.15.16:0/1298709824 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5de0103760 0x7f5de0198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:12.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.374+0000 7f5de87e9640 1 --2- 172.21.15.16:0/1298709824 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5de0102db0 0x7f5de0198570 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:12.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.374+0000 7f5de87e9640 1 -- 172.21.15.16:0/1298709824 >> 172.21.15.16:0/1298709824 conn(0x7f5de00fc7d0 msgr2=0x7f5de0105cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:07:12.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.375+0000 7f5de87e9640 1 -- 172.21.15.16:0/1298709824 shutdown_connections 2024-09-12T02:07:12.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:12.375+0000 7f5de87e9640 1 -- 172.21.15.16:0/1298709824 wait complete. 2024-09-12T02:07:12.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:07:12 smithi033 ceph-mon[35332]: pgmap v493: 97 pgs: 97 active+clean; 2.8 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-09-12T02:07:12.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:07:12 smithi184 ceph-mon[35593]: pgmap v493: 97 pgs: 97 active+clean; 2.8 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-09-12T02:07:13.092 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.092+0000 7effda447640 1 -- 172.21.15.16:0/473055720 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7effd4100a40 msgr2=0x7effd4100e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:13.093 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.092+0000 7effda447640 1 --2- 172.21.15.16:0/473055720 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7effd4100a40 0x7effd4100e40 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7effc4009900 tx=0x7effc402a410 comp rx=0 tx=0).stop 2024-09-12T02:07:13.093 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.092+0000 7effda447640 1 -- 172.21.15.16:0/473055720 shutdown_connections 2024-09-12T02:07:13.093 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.092+0000 7effda447640 1 --2- 172.21.15.16:0/473055720 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7effd41025f0 0x7effd4108e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:13.093 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.092+0000 7effda447640 1 --2- 172.21.15.16:0/473055720 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7effd4101c30 0x7effd41020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:13.093 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.092+0000 7effda447640 1 --2- 172.21.15.16:0/473055720 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7effd4100a40 0x7effd4100e40 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:13.093 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.092+0000 7effda447640 1 -- 172.21.15.16:0/473055720 >> 172.21.15.16:0/473055720 conn(0x7effd40fc1e0 msgr2=0x7effd40fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:07:13.093 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.092+0000 7effda447640 1 -- 172.21.15.16:0/473055720 shutdown_connections 2024-09-12T02:07:13.093 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.092+0000 7effda447640 1 -- 172.21.15.16:0/473055720 wait complete. 2024-09-12T02:07:13.093 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.093+0000 7effda447640 1 Processor -- start 2024-09-12T02:07:13.093 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.093+0000 7effda447640 1 -- start start 2024-09-12T02:07:13.095 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.093+0000 7effda447640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7effd4100a40 0x7effd4197e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:13.095 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.093+0000 7effda447640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7effd4101c30 0x7effd4198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:13.095 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.093+0000 7effda447640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7effd41025f0 0x7effd419f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:13.095 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.093+0000 7effda447640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7effd410d830 con 0x7effd41025f0 2024-09-12T02:07:13.095 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.093+0000 7effda447640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7effd410d6b0 con 0x7effd4101c30 2024-09-12T02:07:13.095 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.093+0000 7effda447640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7effd410d9b0 con 0x7effd4100a40 2024-09-12T02:07:13.096 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.093+0000 7effd89e4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7effd41025f0 0x7effd419f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:13.096 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.093+0000 7effd89e4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7effd41025f0 0x7effd419f400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49346/0 (socket says 172.21.15.16:49346) 2024-09-12T02:07:13.096 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.093+0000 7effd89e4640 1 -- 172.21.15.16:0/318533929 learned_addr learned my addr 172.21.15.16:0/318533929 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:07:13.096 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.093+0000 7effd37fe640 1 --2- 172.21.15.16:0/318533929 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7effd4101c30 0x7effd4198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:13.096 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.093+0000 7effd3fff640 1 --2- 172.21.15.16:0/318533929 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7effd4100a40 0x7effd4197e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:13.096 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.093+0000 7effd89e4640 1 -- 172.21.15.16:0/318533929 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7effd4100a40 msgr2=0x7effd4197e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:13.096 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.093+0000 7effd89e4640 1 --2- 172.21.15.16:0/318533929 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7effd4100a40 0x7effd4197e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:13.096 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.093+0000 7effd89e4640 1 -- 172.21.15.16:0/318533929 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7effd4101c30 msgr2=0x7effd4198390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:13.096 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.093+0000 7effd89e4640 1 --2- 172.21.15.16:0/318533929 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7effd4101c30 0x7effd4198390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:13.096 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.093+0000 7effd89e4640 1 -- 172.21.15.16:0/318533929 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effd419fa70 con 0x7effd41025f0 2024-09-12T02:07:13.096 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.094+0000 7effd89e4640 1 --2- 172.21.15.16:0/318533929 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7effd41025f0 0x7effd419f400 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7effcc00cad0 tx=0x7effcc00ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:07:13.096 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.094+0000 7effd17fa640 1 -- 172.21.15.16:0/318533929 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effcc00ad60 con 0x7effd41025f0 2024-09-12T02:07:13.097 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.094+0000 7effda447640 1 -- 172.21.15.16:0/318533929 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effd419fd60 con 0x7effd41025f0 2024-09-12T02:07:13.097 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.094+0000 7effd17fa640 1 -- 172.21.15.16:0/318533929 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7effcc005020 con 0x7effd41025f0 2024-09-12T02:07:13.097 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.094+0000 7effd17fa640 1 -- 172.21.15.16:0/318533929 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effcc00f6b0 con 0x7effd41025f0 2024-09-12T02:07:13.097 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.094+0000 7effda447640 1 -- 172.21.15.16:0/318533929 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effd41a0280 con 0x7effd41025f0 2024-09-12T02:07:13.107 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.096+0000 7effdd1e0740 1 Processor -- start 2024-09-12T02:07:13.107 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.097+0000 7effd17fa640 1 -- 172.21.15.16:0/318533929 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7effcc005560 con 0x7effd41025f0 2024-09-12T02:07:13.107 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.097+0000 7effdd1e0740 1 -- start start 2024-09-12T02:07:13.107 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.097+0000 7effdd1e0740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5644b772e9a0 0x5644b772eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:13.107 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.097+0000 7effdd1e0740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5644b772f2e0 0x5644b7733da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:13.107 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.098+0000 7effdd1e0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5644b7734370 0x5644b7736830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:13.108 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.098+0000 7effdd1e0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5644b76922f0 con 0x5644b7734370 2024-09-12T02:07:13.108 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.098+0000 7effdd1e0740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5644b763a270 con 0x5644b772e9a0 2024-09-12T02:07:13.108 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.098+0000 7effdd1e0740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5644b768e2b0 con 0x5644b772f2e0 2024-09-12T02:07:13.108 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.098+0000 7effd37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5644b7734370 0x5644b7736830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:13.108 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.098+0000 7effd37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5644b7734370 0x5644b7736830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49358/0 (socket says 172.21.15.16:49358) 2024-09-12T02:07:13.108 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.098+0000 7effd37fe640 1 -- 172.21.15.16:0/586318122 learned_addr learned my addr 172.21.15.16:0/586318122 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:07:13.108 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.098+0000 7effd3fff640 1 --2- 172.21.15.16:0/586318122 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5644b772f2e0 0x5644b7733da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:13.108 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.098+0000 7effd37fe640 1 --2- 172.21.15.16:0/586318122 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5644b772e9a0 0x5644b772eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:13.108 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.098+0000 7effd17fa640 1 --2- 172.21.15.16:0/318533929 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7effbc0a2c30 0x7effbc0a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:13.108 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.098+0000 7effd37fe640 1 -- 172.21.15.16:0/586318122 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5644b772f2e0 msgr2=0x5644b7733da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:13.108 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.098+0000 7effd37fe640 1 --2- 172.21.15.16:0/586318122 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5644b772f2e0 0x5644b7733da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:13.108 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.098+0000 7effd37fe640 1 -- 172.21.15.16:0/586318122 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5644b772e9a0 msgr2=0x5644b772eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:13.109 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.098+0000 7effd37fe640 1 --2- 172.21.15.16:0/586318122 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5644b772e9a0 0x5644b772eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:13.109 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.098+0000 7effd37fe640 1 -- 172.21.15.16:0/586318122 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644b7639340 con 0x5644b7734370 2024-09-12T02:07:13.109 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.098+0000 7effd3fff640 1 --2- 172.21.15.16:0/586318122 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5644b772f2e0 0x5644b7733da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:07:13.109 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.098+0000 7effd89e4640 1 --2- 172.21.15.16:0/318533929 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7effbc0a2c30 0x7effbc0a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:13.109 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.098+0000 7effd17fa640 1 -- 172.21.15.16:0/318533929 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7effcc0cefb0 con 0x7effd41025f0 2024-09-12T02:07:13.109 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.098+0000 7effd37fe640 1 --2- 172.21.15.16:0/586318122 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5644b7734370 0x5644b7736830 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7effc8006aa0 tx=0x7effc8009510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cd607795bb1b4da server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:07:13.109 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.099+0000 7effba7fc640 1 -- 172.21.15.16:0/586318122 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effc8015760 con 0x5644b7734370 2024-09-12T02:07:13.109 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.099+0000 7effba7fc640 1 -- 172.21.15.16:0/586318122 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7effc801d070 con 0x5644b7734370 2024-09-12T02:07:13.109 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.099+0000 7effba7fc640 1 -- 172.21.15.16:0/586318122 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effc8018b20 con 0x5644b7734370 2024-09-12T02:07:13.109 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.099+0000 7effdd1e0740 1 -- 172.21.15.16:0/586318122 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5644b7734370 msgr2=0x5644b7736830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:13.109 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.099+0000 7effdd1e0740 1 --2- 172.21.15.16:0/586318122 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5644b7734370 0x5644b7736830 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7effc8006aa0 tx=0x7effc8009510 comp rx=0 tx=0).stop 2024-09-12T02:07:13.109 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.099+0000 7effd89e4640 1 --2- 172.21.15.16:0/318533929 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7effbc0a2c30 0x7effbc0a50f0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7effcc00a620 tx=0x7effcc090d90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:07:13.109 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.099+0000 7effdd1e0740 1 -- 172.21.15.16:0/586318122 shutdown_connections 2024-09-12T02:07:13.109 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.099+0000 7effdd1e0740 1 --2- 172.21.15.16:0/586318122 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5644b7734370 0x5644b7736830 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:13.110 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.099+0000 7effdd1e0740 1 --2- 172.21.15.16:0/586318122 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5644b772f2e0 0x5644b7733da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:13.110 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.099+0000 7effdd1e0740 1 --2- 172.21.15.16:0/586318122 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5644b772e9a0 0x5644b772eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:13.110 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.099+0000 7effdd1e0740 1 -- 172.21.15.16:0/586318122 >> 172.21.15.16:0/586318122 conn(0x5644b772a010 msgr2=0x5644b772a410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:07:13.110 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.100+0000 7effdd1e0740 1 -- 172.21.15.16:0/586318122 shutdown_connections 2024-09-12T02:07:13.110 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.100+0000 7effdd1e0740 1 -- 172.21.15.16:0/586318122 wait complete. 2024-09-12T02:07:13.110 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.100+0000 7effdd1e0740 1 -- start start 2024-09-12T02:07:13.110 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.100+0000 7effdd1e0740 1 Processor -- start 2024-09-12T02:07:13.110 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.100+0000 7effdd1e0740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5644b772e9a0 0x5644b7738b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:13.110 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.100+0000 7effdd1e0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5644b772f2e0 0x5644b77390d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:13.110 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.100+0000 7effdd1e0740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5644b7734370 0x5644b77400a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:13.110 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.100+0000 7effdd1e0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5644b77372e0 con 0x5644b772f2e0 2024-09-12T02:07:13.110 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.100+0000 7effdd1e0740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5644b761b290 con 0x5644b7734370 2024-09-12T02:07:13.110 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.100+0000 7effdd1e0740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5644b7737460 con 0x5644b772e9a0 2024-09-12T02:07:13.111 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.100+0000 7effdd1e0740 10 client.0 _ll_register_callbacks cb 0x5644b764c140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:07:13.111 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.100+0000 7effd3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5644b772f2e0 0x5644b77390d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:13.111 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.100+0000 7effd3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5644b772f2e0 0x5644b77390d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49368/0 (socket says 172.21.15.16:49368) 2024-09-12T02:07:13.111 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.100+0000 7effd3fff640 1 -- 172.21.15.16:0/2466421419 learned_addr learned my addr 172.21.15.16:0/2466421419 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:07:13.111 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.101+0000 7effd37fe640 1 --2- 172.21.15.16:0/2466421419 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5644b7734370 0x5644b77400a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:13.111 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.101+0000 7effd37fe640 1 --2- 172.21.15.16:0/2466421419 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5644b772e9a0 0x5644b7738b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:13.111 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.101+0000 7effd3fff640 1 -- 172.21.15.16:0/2466421419 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5644b772e9a0 msgr2=0x5644b7738b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:13.111 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.101+0000 7effd3fff640 1 --2- 172.21.15.16:0/2466421419 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5644b772e9a0 0x5644b7738b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:13.111 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.101+0000 7effd3fff640 1 -- 172.21.15.16:0/2466421419 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5644b7734370 msgr2=0x5644b77400a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:13.111 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.101+0000 7effd3fff640 1 --2- 172.21.15.16:0/2466421419 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5644b7734370 0x5644b77400a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:13.111 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.101+0000 7effd3fff640 1 -- 172.21.15.16:0/2466421419 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644b73a5210 con 0x5644b772f2e0 2024-09-12T02:07:13.111 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.101+0000 7effd37fe640 1 --2- 172.21.15.16:0/2466421419 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5644b772e9a0 0x5644b7738b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:07:13.111 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.101+0000 7effd3fff640 1 --2- 172.21.15.16:0/2466421419 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5644b772f2e0 0x5644b77390d0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7effc40314e0 tx=0x7effc4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fe6a669dfd613ab server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:07:13.112 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.101+0000 7effb37fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:07:13.112 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.101+0000 7effb37fe640 1 -- 172.21.15.16:0/2466421419 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effc4035040 con 0x5644b772f2e0 2024-09-12T02:07:13.112 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.101+0000 7effb37fe640 1 -- 172.21.15.16:0/2466421419 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7effc40035f0 con 0x5644b772f2e0 2024-09-12T02:07:13.112 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.101+0000 7effdd1e0740 1 -- 172.21.15.16:0/2466421419 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7effc801d070 con 0x5644b772f2e0 2024-09-12T02:07:13.112 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.101+0000 7effb37fe640 1 -- 172.21.15.16:0/2466421419 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7effc40319e0 con 0x5644b772f2e0 2024-09-12T02:07:13.112 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.101+0000 7effdd1e0740 10 client.16989 fetch_fsmap learned FSMap version 25 2024-09-12T02:07:13.112 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.101+0000 7effdd1e0740 1 -- 172.21.15.16:0/2466421419 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5644b7471620 con 0x5644b772f2e0 2024-09-12T02:07:13.112 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.102+0000 7effb37fe640 1 -- 172.21.15.16:0/2466421419 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7effc801d070 con 0x5644b772f2e0 2024-09-12T02:07:13.112 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.102+0000 7effb37fe640 1 -- 172.21.15.16:0/2466421419 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7effc403da90 con 0x5644b772f2e0 2024-09-12T02:07:13.112 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.102+0000 7effdd1e0740 10 client.16989 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:07:13.112 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.102+0000 7effdd1e0740 10 client.16989 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:07:13.112 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.102+0000 7effdd1e0740 20 client.16989 populate_metadata read hostname 'smithi016' 2024-09-12T02:07:13.112 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.102+0000 7effdd1e0740 1 --2- 172.21.15.16:0/2466421419 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5644b77447b0 0x5644b7746bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:13.112 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.102+0000 7effdd1e0740 4 client.16989 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:07:13.113 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.102+0000 7effdd1e0740 1 -- 172.21.15.16:0/2466421419 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5644b769fa20 con 0x5644b77447b0 2024-09-12T02:07:13.113 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.102+0000 7effd37fe640 1 --2- 172.21.15.16:0/2466421419 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5644b77447b0 0x5644b7746bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:13.113 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.103+0000 7effd37fe640 1 --2- 172.21.15.16:0/2466421419 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5644b77447b0 0x5644b7746bc0 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14c01843cbf22698 server_cookie=edd56f21f7944bec in_seq=0 out_seq=0 2024-09-12T02:07:13.113 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.103+0000 7effb37fe640 10 client.16989 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:07:13.113 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.103+0000 7effb37fe640 1 -- 172.21.15.16:0/2466421419 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5644b769fa20 con 0x5644b77447b0 2024-09-12T02:07:13.113 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.103+0000 7effb37fe640 10 client.16989 handle_command_reply: tid=0 2024-09-12T02:07:13.113 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.103+0000 7effdd1e0740 1 client.16989 shutdown 2024-09-12T02:07:13.113 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.103+0000 7effdd1e0740 1 -- 172.21.15.16:0/2466421419 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5644b772f2e0 msgr2=0x5644b77390d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:13.113 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.103+0000 7effdd1e0740 1 --2- 172.21.15.16:0/2466421419 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5644b772f2e0 0x5644b77390d0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7effc40314e0 tx=0x7effc4004270 comp rx=0 tx=0).stop 2024-09-12T02:07:13.113 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.104+0000 7effdd1e0740 1 -- 172.21.15.16:0/2466421419 shutdown_connections 2024-09-12T02:07:13.113 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.104+0000 7effdd1e0740 1 --2- 172.21.15.16:0/2466421419 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5644b77447b0 0x5644b7746bc0 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:13.113 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.104+0000 7effdd1e0740 1 --2- 172.21.15.16:0/2466421419 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5644b7734370 0x5644b77400a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:13.113 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.104+0000 7effdd1e0740 1 --2- 172.21.15.16:0/2466421419 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5644b772f2e0 0x5644b77390d0 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:13.114 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.104+0000 7effdd1e0740 1 --2- 172.21.15.16:0/2466421419 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5644b772e9a0 0x5644b7738b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:13.114 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.104+0000 7effb37fe640 0 client.16989 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:07:13.114 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.104+0000 7effdd1e0740 1 -- 172.21.15.16:0/2466421419 >> 172.21.15.16:0/2466421419 conn(0x5644b772a010 msgr2=0x5644b772b820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:07:13.114 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.104+0000 7effdd1e0740 1 -- 172.21.15.16:0/2466421419 shutdown_connections 2024-09-12T02:07:13.114 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.104+0000 7effdd1e0740 1 -- 172.21.15.16:0/2466421419 wait complete. 2024-09-12T02:07:13.114 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.104+0000 7effdd1e0740 20 client.16989 trim_cache size 0 max 16384 2024-09-12T02:07:13.138 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.133+0000 7effdd1e0740 1 Processor -- start 2024-09-12T02:07:13.138 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.133+0000 7effdd1e0740 1 -- start start 2024-09-12T02:07:13.138 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.133+0000 7effdd1e0740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5644b772a010 0x5644b774b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:13.138 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.133+0000 7effdd1e0740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5644b772e9a0 0x5644b774ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:13.139 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.133+0000 7effdd1e0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5644b772f2e0 0x5644b774bf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:13.139 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.133+0000 7effdd1e0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7effd41ab810 con 0x5644b772f2e0 2024-09-12T02:07:13.139 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.133+0000 7effdd1e0740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7effd41a90e0 con 0x5644b772a010 2024-09-12T02:07:13.139 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.133+0000 7effdd1e0740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7effd41a6f30 con 0x5644b772e9a0 2024-09-12T02:07:13.139 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.133+0000 7effd37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5644b772f2e0 0x5644b774bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:13.139 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.133+0000 7effd37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5644b772f2e0 0x5644b774bf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49380/0 (socket says 172.21.15.16:49380) 2024-09-12T02:07:13.139 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.133+0000 7effd37fe640 1 -- 172.21.15.16:0/3517892217 learned_addr learned my addr 172.21.15.16:0/3517892217 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:07:13.139 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.133+0000 7effd37fe640 1 --2- 172.21.15.16:0/3517892217 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5644b772a010 0x5644b774b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:13.139 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.133+0000 7effd3fff640 1 --2- 172.21.15.16:0/3517892217 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5644b772e9a0 0x5644b774ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:13.139 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.133+0000 7effd37fe640 1 -- 172.21.15.16:0/3517892217 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5644b772e9a0 msgr2=0x5644b774ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:13.139 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.133+0000 7effd37fe640 1 --2- 172.21.15.16:0/3517892217 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5644b772e9a0 0x5644b774ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:13.139 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.133+0000 7effd37fe640 1 -- 172.21.15.16:0/3517892217 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5644b772a010 msgr2=0x5644b774b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:13.140 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.133+0000 7effd37fe640 1 --2- 172.21.15.16:0/3517892217 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5644b772a010 0x5644b774b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:13.140 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.133+0000 7effd37fe640 1 -- 172.21.15.16:0/3517892217 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644b77252d0 con 0x5644b772f2e0 2024-09-12T02:07:13.140 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.133+0000 7effd3fff640 1 --2- 172.21.15.16:0/3517892217 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5644b772e9a0 0x5644b774ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:07:13.140 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.134+0000 7effd37fe640 1 --2- 172.21.15.16:0/3517892217 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5644b772f2e0 0x5644b774bf40 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7effc8009bc0 tx=0x7effc80099b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99e2c11d8460546b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:07:13.140 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.134+0000 7effb9ffb640 1 -- 172.21.15.16:0/3517892217 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effc8018770 con 0x5644b772f2e0 2024-09-12T02:07:13.140 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.134+0000 7effb9ffb640 1 -- 172.21.15.16:0/3517892217 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7effc801d030 con 0x5644b772f2e0 2024-09-12T02:07:13.140 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.134+0000 7effb9ffb640 1 -- 172.21.15.16:0/3517892217 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effc8015e80 con 0x5644b772f2e0 2024-09-12T02:07:13.140 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.135+0000 7effdd1e0740 1 -- 172.21.15.16:0/3517892217 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5644b772f2e0 msgr2=0x5644b774bf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:13.140 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.135+0000 7effdd1e0740 1 --2- 172.21.15.16:0/3517892217 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5644b772f2e0 0x5644b774bf40 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7effc8009bc0 tx=0x7effc80099b0 comp rx=0 tx=0).stop 2024-09-12T02:07:13.140 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.135+0000 7effdd1e0740 1 -- 172.21.15.16:0/3517892217 shutdown_connections 2024-09-12T02:07:13.140 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.135+0000 7effdd1e0740 1 --2- 172.21.15.16:0/3517892217 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5644b772f2e0 0x5644b774bf40 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:13.140 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.135+0000 7effdd1e0740 1 --2- 172.21.15.16:0/3517892217 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5644b772e9a0 0x5644b774ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:13.140 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.135+0000 7effdd1e0740 1 --2- 172.21.15.16:0/3517892217 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5644b772a010 0x5644b774b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:13.141 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.135+0000 7effdd1e0740 1 -- 172.21.15.16:0/3517892217 >> 172.21.15.16:0/3517892217 conn(0x5644b7733ec0 msgr2=0x5644b77291c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:07:13.141 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.135+0000 7effdd1e0740 1 -- 172.21.15.16:0/3517892217 shutdown_connections 2024-09-12T02:07:13.141 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.135+0000 7effdd1e0740 1 -- 172.21.15.16:0/3517892217 wait complete. 2024-09-12T02:07:13.141 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.135+0000 7effdd1e0740 1 -- start start 2024-09-12T02:07:13.141 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.135+0000 7effdd1e0740 1 Processor -- start 2024-09-12T02:07:13.141 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.136+0000 7effdd1e0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5644b772a010 0x5644b77ab060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:13.141 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.136+0000 7effdd1e0740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5644b772e9a0 0x5644b77ab5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:13.141 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.136+0000 7effdd1e0740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5644b772f2e0 0x5644b77b25e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:13.141 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.136+0000 7effdd1e0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7effd41a75a0 con 0x5644b772a010 2024-09-12T02:07:13.141 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.136+0000 7effdd1e0740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7effd41a8a10 con 0x5644b772e9a0 2024-09-12T02:07:13.141 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.136+0000 7effdd1e0740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7effd41a7c70 con 0x5644b772f2e0 2024-09-12T02:07:13.141 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.136+0000 7effdd1e0740 10 client.0 _ll_register_callbacks cb 0x7effc4009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:07:13.141 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.136+0000 7effd37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5644b772a010 0x5644b77ab060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:13.142 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.136+0000 7effd37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5644b772a010 0x5644b77ab060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49388/0 (socket says 172.21.15.16:49388) 2024-09-12T02:07:13.142 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.136+0000 7effd37fe640 1 -- 172.21.15.16:0/3783922470 learned_addr learned my addr 172.21.15.16:0/3783922470 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:07:13.142 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.136+0000 7effd3fff640 1 --2- 172.21.15.16:0/3783922470 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5644b772e9a0 0x5644b77ab5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:13.142 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.136+0000 7effd37fe640 1 --2- 172.21.15.16:0/3783922470 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5644b772f2e0 0x5644b77b25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:13.142 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.136+0000 7effd37fe640 1 -- 172.21.15.16:0/3783922470 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5644b772f2e0 msgr2=0x5644b77b25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:13.142 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.136+0000 7effd37fe640 1 --2- 172.21.15.16:0/3783922470 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5644b772f2e0 0x5644b77b25e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:13.142 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.136+0000 7effd37fe640 1 -- 172.21.15.16:0/3783922470 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5644b772e9a0 msgr2=0x5644b77ab5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:13.142 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.136+0000 7effd37fe640 1 --2- 172.21.15.16:0/3783922470 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5644b772e9a0 0x5644b77ab5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:13.142 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.136+0000 7effd37fe640 1 -- 172.21.15.16:0/3783922470 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644b77b2bf0 con 0x5644b772a010 2024-09-12T02:07:13.142 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.137+0000 7effd37fe640 1 --2- 172.21.15.16:0/3783922470 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5644b772a010 0x5644b77ab060 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7effc8009e90 tx=0x7effc8009e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1614d3ae8a57d43e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:07:13.142 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.137+0000 7effb37fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:07:13.142 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.137+0000 7effb37fe640 1 -- 172.21.15.16:0/3783922470 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effc8015dc0 con 0x5644b772a010 2024-09-12T02:07:13.142 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.137+0000 7effdd1e0740 1 -- 172.21.15.16:0/3783922470 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7effc801d030 con 0x5644b772a010 2024-09-12T02:07:13.142 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.137+0000 7effb37fe640 1 -- 172.21.15.16:0/3783922470 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7effc8018700 con 0x5644b772a010 2024-09-12T02:07:13.143 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.137+0000 7effb37fe640 1 -- 172.21.15.16:0/3783922470 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7effc80059c0 con 0x5644b772a010 2024-09-12T02:07:13.143 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.137+0000 7effdd1e0740 10 client.16995 fetch_fsmap learned FSMap version 25 2024-09-12T02:07:13.143 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.137+0000 7effdd1e0740 1 -- 172.21.15.16:0/3783922470 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5644b77b3f00 con 0x5644b772a010 2024-09-12T02:07:13.143 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.137+0000 7effb37fe640 1 -- 172.21.15.16:0/3783922470 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7effc801d030 con 0x5644b772a010 2024-09-12T02:07:13.143 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.137+0000 7effb37fe640 1 -- 172.21.15.16:0/3783922470 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7effc8030020 con 0x5644b772a010 2024-09-12T02:07:13.143 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.137+0000 7effdd1e0740 10 client.16995 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:07:13.143 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.137+0000 7effdd1e0740 10 client.16995 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:07:13.143 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.138+0000 7effdd1e0740 20 client.16995 populate_metadata read hostname 'smithi016' 2024-09-12T02:07:13.143 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.138+0000 7effdd1e0740 1 --2- 172.21.15.16:0/3783922470 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5644b7734370 0x5644b77b6590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:13.143 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.138+0000 7effdd1e0740 4 client.16995 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:07:13.143 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.138+0000 7effdd1e0740 1 -- 172.21.15.16:0/3783922470 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5644b7699d70 con 0x5644b7734370 2024-09-12T02:07:13.143 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.138+0000 7effd3fff640 1 --2- 172.21.15.16:0/3783922470 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5644b7734370 0x5644b77b6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:13.143 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.138+0000 7effb37fe640 10 client.16995 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:07:13.143 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.138+0000 7effd3fff640 1 --2- 172.21.15.16:0/3783922470 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5644b7734370 0x5644b77b6590 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b2f5dc3a0ac61b86 server_cookie=398b9e28144aede3 in_seq=0 out_seq=0 2024-09-12T02:07:13.152 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.152+0000 7effb37fe640 1 -- 172.21.15.16:0/3783922470 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5644b7699d70 con 0x5644b7734370 2024-09-12T02:07:13.153 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.152+0000 7effb37fe640 10 client.16995 handle_command_reply: tid=0 2024-09-12T02:07:13.153 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.152+0000 7effdd1e0740 1 client.16995 shutdown 2024-09-12T02:07:13.153 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.152+0000 7effdd1e0740 1 -- 172.21.15.16:0/3783922470 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5644b772a010 msgr2=0x5644b77ab060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:13.153 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.152+0000 7effdd1e0740 1 --2- 172.21.15.16:0/3783922470 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5644b772a010 0x5644b77ab060 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7effc8009e90 tx=0x7effc8009e00 comp rx=0 tx=0).stop 2024-09-12T02:07:13.153 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.152+0000 7effdd1e0740 1 -- 172.21.15.16:0/3783922470 shutdown_connections 2024-09-12T02:07:13.153 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.152+0000 7effdd1e0740 1 --2- 172.21.15.16:0/3783922470 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5644b7734370 0x5644b77b6590 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:13.153 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.152+0000 7effdd1e0740 1 --2- 172.21.15.16:0/3783922470 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5644b772f2e0 0x5644b77b25e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:13.153 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.152+0000 7effdd1e0740 1 --2- 172.21.15.16:0/3783922470 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5644b772e9a0 0x5644b77ab5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:13.153 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.152+0000 7effb37fe640 0 client.16995 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:07:13.153 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.152+0000 7effdd1e0740 1 --2- 172.21.15.16:0/3783922470 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5644b772a010 0x5644b77ab060 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:13.153 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.152+0000 7effdd1e0740 1 -- 172.21.15.16:0/3783922470 >> 172.21.15.16:0/3783922470 conn(0x5644b7733ec0 msgr2=0x5644b77289a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:07:13.154 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.152+0000 7effdd1e0740 1 -- 172.21.15.16:0/3783922470 shutdown_connections 2024-09-12T02:07:13.154 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.152+0000 7effdd1e0740 1 -- 172.21.15.16:0/3783922470 wait complete. 2024-09-12T02:07:13.154 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.152+0000 7effdd1e0740 20 client.16995 trim_cache size 0 max 16384 2024-09-12T02:07:13.154 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-09-12T02:07:13.154 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-09-12T02:07:13.154 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-09-12T02:07:13.154 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-09-12T02:07:13.155 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.153+0000 7effda447640 1 -- 172.21.15.16:0/318533929 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7effbc0a2c30 msgr2=0x7effbc0a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:13.155 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.153+0000 7effda447640 1 --2- 172.21.15.16:0/318533929 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7effbc0a2c30 0x7effbc0a50f0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7effcc00a620 tx=0x7effcc090d90 comp rx=0 tx=0).stop 2024-09-12T02:07:13.155 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.153+0000 7effda447640 1 -- 172.21.15.16:0/318533929 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7effd41025f0 msgr2=0x7effd419f400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:13.155 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.153+0000 7effda447640 1 --2- 172.21.15.16:0/318533929 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7effd41025f0 0x7effd419f400 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7effcc00cad0 tx=0x7effcc00ced0 comp rx=0 tx=0).stop 2024-09-12T02:07:13.156 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.153+0000 7effda447640 1 -- 172.21.15.16:0/318533929 shutdown_connections 2024-09-12T02:07:13.156 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.153+0000 7effda447640 1 --2- 172.21.15.16:0/318533929 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7effbc0a2c30 0x7effbc0a50f0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:13.156 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.153+0000 7effda447640 1 --2- 172.21.15.16:0/318533929 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7effd41025f0 0x7effd419f400 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:13.156 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.153+0000 7effda447640 1 --2- 172.21.15.16:0/318533929 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7effd4101c30 0x7effd4198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:13.156 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.153+0000 7effda447640 1 --2- 172.21.15.16:0/318533929 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7effd4100a40 0x7effd4197e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:13.156 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.153+0000 7effda447640 1 -- 172.21.15.16:0/318533929 >> 172.21.15.16:0/318533929 conn(0x7effd40fc1e0 msgr2=0x7effd4104f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:07:13.156 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.153+0000 7effda447640 1 -- 172.21.15.16:0/318533929 shutdown_connections 2024-09-12T02:07:13.156 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:13.153+0000 7effda447640 1 -- 172.21.15.16:0/318533929 wait complete. 2024-09-12T02:07:13.163 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-12T02:07:13.163 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:07:13.426 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:07:13.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:13.865+0000 7f905bbd7640 1 -- 172.21.15.16:0/1395640807 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9054101df0 msgr2=0x7f90541021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:13.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:13.865+0000 7f905bbd7640 1 --2- 172.21.15.16:0/1395640807 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9054101df0 0x7f90541021d0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f904c009900 tx=0x7f904c02a410 comp rx=0 tx=0).stop 2024-09-12T02:07:13.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:13.865+0000 7f905bbd7640 1 -- 172.21.15.16:0/1395640807 shutdown_connections 2024-09-12T02:07:13.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:13.865+0000 7f905bbd7640 1 --2- 172.21.15.16:0/1395640807 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9054106f20 0x7f905410eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:13.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:13.865+0000 7f905bbd7640 1 --2- 172.21.15.16:0/1395640807 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f90541027a0 0x7f90541067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:13.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:13.865+0000 7f905bbd7640 1 --2- 172.21.15.16:0/1395640807 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9054101df0 0x7f90541021d0 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:13.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:13.865+0000 7f905bbd7640 1 -- 172.21.15.16:0/1395640807 >> 172.21.15.16:0/1395640807 conn(0x7f90540fd8f0 msgr2=0x7f90540ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:07:13.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:13.865+0000 7f905bbd7640 1 -- 172.21.15.16:0/1395640807 shutdown_connections 2024-09-12T02:07:13.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:13.865+0000 7f905bbd7640 1 -- 172.21.15.16:0/1395640807 wait complete. 2024-09-12T02:07:13.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:13.866+0000 7f905bbd7640 1 Processor -- start 2024-09-12T02:07:13.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:13.866+0000 7f905bbd7640 1 -- start start 2024-09-12T02:07:13.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:13.866+0000 7f905bbd7640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9054101df0 0x7f905419a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:13.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:13.866+0000 7f905bbd7640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f90541027a0 0x7f905419abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:13.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:13.866+0000 7f905bbd7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9054106f20 0x7f905419ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:13.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:13.866+0000 7f905bbd7640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9054111f40 con 0x7f9054106f20 2024-09-12T02:07:13.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:13.866+0000 7f905bbd7640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9054111dc0 con 0x7f90541027a0 2024-09-12T02:07:13.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:13.866+0000 7f905bbd7640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f90541120c0 con 0x7f9054101df0 2024-09-12T02:07:13.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:13.866+0000 7f9059172640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f90541027a0 0x7f905419abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:13.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:13.866+0000 7f905a174640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9054106f20 0x7f905419ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:13.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:13.866+0000 7f9059172640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f90541027a0 0x7f905419abc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.16:38834/0 (socket says 172.21.15.16:38834) 2024-09-12T02:07:13.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:13.866+0000 7f9059172640 1 -- 172.21.15.16:0/2379710935 learned_addr learned my addr 172.21.15.16:0/2379710935 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:07:13.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:13.866+0000 7f9059973640 1 --2- 172.21.15.16:0/2379710935 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9054101df0 0x7f905419a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:13.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:13.867+0000 7f905a174640 1 -- 172.21.15.16:0/2379710935 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9054101df0 msgr2=0x7f905419a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:13.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:13.867+0000 7f905a174640 1 --2- 172.21.15.16:0/2379710935 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9054101df0 0x7f905419a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:13.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:13.867+0000 7f905a174640 1 -- 172.21.15.16:0/2379710935 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f90541027a0 msgr2=0x7f905419abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:13.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:13.867+0000 7f905a174640 1 --2- 172.21.15.16:0/2379710935 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f90541027a0 0x7f905419abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:13.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:13.867+0000 7f905a174640 1 -- 172.21.15.16:0/2379710935 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f905419f630 con 0x7f9054106f20 2024-09-12T02:07:13.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:13.867+0000 7f905a174640 1 --2- 172.21.15.16:0/2379710935 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9054106f20 0x7f905419ef50 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f904400ca30 tx=0x7f904400cef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:07:13.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:13.867+0000 7f9052ffd640 1 -- 172.21.15.16:0/2379710935 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9044007d90 con 0x7f9054106f20 2024-09-12T02:07:13.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:13.867+0000 7f905bbd7640 1 -- 172.21.15.16:0/2379710935 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f905419f920 con 0x7f9054106f20 2024-09-12T02:07:13.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:13.867+0000 7f9052ffd640 1 -- 172.21.15.16:0/2379710935 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9044005020 con 0x7f9054106f20 2024-09-12T02:07:13.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:13.867+0000 7f9052ffd640 1 -- 172.21.15.16:0/2379710935 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f904400f780 con 0x7f9054106f20 2024-09-12T02:07:13.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:13.867+0000 7f905bbd7640 1 -- 172.21.15.16:0/2379710935 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90541a7200 con 0x7f9054106f20 2024-09-12T02:07:13.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:13.868+0000 7f905bbd7640 1 -- 172.21.15.16:0/2379710935 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90541021d0 con 0x7f9054106f20 2024-09-12T02:07:13.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:13.870+0000 7f9052ffd640 1 -- 172.21.15.16:0/2379710935 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f90440051d0 con 0x7f9054106f20 2024-09-12T02:07:13.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:13.870+0000 7f9052ffd640 1 --2- 172.21.15.16:0/2379710935 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f903c0a2b80 0x7f903c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:13.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:13.870+0000 7f9052ffd640 1 -- 172.21.15.16:0/2379710935 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f90440cf100 con 0x7f9054106f20 2024-09-12T02:07:13.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:13.870+0000 7f9059973640 1 --2- 172.21.15.16:0/2379710935 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f903c0a2b80 0x7f903c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:13.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:13.871+0000 7f9059973640 1 --2- 172.21.15.16:0/2379710935 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f903c0a2b80 0x7f903c0a5040 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f904c009230 tx=0x7f904c0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:07:13.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:13.875+0000 7f9052ffd640 1 -- 172.21.15.16:0/2379710935 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9044014070 con 0x7f9054106f20 2024-09-12T02:07:14.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:14.088+0000 7f905bbd7640 1 -- 172.21.15.16:0/2379710935 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f90541a7430 con 0x7f9054106f20 2024-09-12T02:07:14.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:14.089+0000 7f9052ffd640 1 -- 172.21.15.16:0/2379710935 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11911 (secure 0 0 0) 0x7f904409bab0 con 0x7f9054106f20 2024-09-12T02:07:14.089 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T02:07:14.089 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":25,"btime":"2024-09-12T02:01:37:192021+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3279123279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3279123279},{"type":"v1","addr":"172.21.15.16:6835","nonce":3279123279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4119452382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4119452382},{"type":"v1","addr":"172.21.15.16:6837","nonce":4119452382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/3263823165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3263823165},{"type":"v1","addr":"172.21.15.16:6839","nonce":3263823165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/1510290732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":1510290732},{"type":"v1","addr":"172.21.15.16:6841","nonce":1510290732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/279674603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":279674603},{"type":"v1","addr":"172.21.15.33:6833","nonce":279674603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3555728028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3555728028},{"type":"v1","addr":"172.21.15.33:6835","nonce":3555728028}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T01:59:56.117608+0000","modified":"2024-09-12T02:01:36.294760+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24482,"mds_1":24481,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6833/1563374106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1563374106},{"type":"v1","addr":"172.21.15.184:6833","nonce":1563374106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6837/1892833868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1892833868},{"type":"v1","addr":"172.21.15.33:6837","nonce":1892833868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/1457679686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1457679686},{"type":"v1","addr":"172.21.15.184:6835","nonce":1457679686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.184:6837/1793393087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":1793393087},{"type":"v1","addr":"172.21.15.184:6837","nonce":1793393087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.184:6839/4203347941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":4203347941},{"type":"v1","addr":"172.21.15.184:6839","nonce":4203347941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.33:6839/4136670204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":4136670204},{"type":"v1","addr":"172.21.15.33:6839","nonce":4136670204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24481,24482,24469]},"id":1}]} 2024-09-12T02:07:14.090 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 25 2024-09-12T02:07:14.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:14.091+0000 7f905bbd7640 1 -- 172.21.15.16:0/2379710935 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f903c0a2b80 msgr2=0x7f903c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:14.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:14.091+0000 7f905bbd7640 1 --2- 172.21.15.16:0/2379710935 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f903c0a2b80 0x7f903c0a5040 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f904c009230 tx=0x7f904c0058d0 comp rx=0 tx=0).stop 2024-09-12T02:07:14.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:14.091+0000 7f905bbd7640 1 -- 172.21.15.16:0/2379710935 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9054106f20 msgr2=0x7f905419ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:14.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:14.091+0000 7f905bbd7640 1 --2- 172.21.15.16:0/2379710935 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9054106f20 0x7f905419ef50 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f904400ca30 tx=0x7f904400cef0 comp rx=0 tx=0).stop 2024-09-12T02:07:14.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:14.091+0000 7f905bbd7640 1 -- 172.21.15.16:0/2379710935 shutdown_connections 2024-09-12T02:07:14.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:14.091+0000 7f905bbd7640 1 --2- 172.21.15.16:0/2379710935 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f903c0a2b80 0x7f903c0a5040 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:14.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:14.091+0000 7f905bbd7640 1 --2- 172.21.15.16:0/2379710935 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9054106f20 0x7f905419ef50 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:14.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:14.091+0000 7f905bbd7640 1 --2- 172.21.15.16:0/2379710935 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f90541027a0 0x7f905419abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:14.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:14.091+0000 7f905bbd7640 1 --2- 172.21.15.16:0/2379710935 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9054101df0 0x7f905419a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:14.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:14.091+0000 7f905bbd7640 1 -- 172.21.15.16:0/2379710935 >> 172.21.15.16:0/2379710935 conn(0x7f90540fd8f0 msgr2=0x7f90540fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:07:14.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:14.092+0000 7f905bbd7640 1 -- 172.21.15.16:0/2379710935 shutdown_connections 2024-09-12T02:07:14.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:14.092+0000 7f905bbd7640 1 -- 172.21.15.16:0/2379710935 wait complete. 2024-09-12T02:07:14.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:07:14 smithi033 ceph-mon[35332]: pgmap v494: 97 pgs: 97 active+clean; 2.8 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-09-12T02:07:14.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:07:14 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/2379710935' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:07:14.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:07:14 smithi016 ceph-mon[30947]: pgmap v494: 97 pgs: 97 active+clean; 2.8 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-09-12T02:07:14.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:07:14 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/2379710935' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:07:14.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:07:14 smithi184 ceph-mon[35593]: pgmap v494: 97 pgs: 97 active+clean; 2.8 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-09-12T02:07:14.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:07:14 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/2379710935' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:07:14.708 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-12T02:07:14.964 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:07:15.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.560+0000 7f6f12045640 1 -- 172.21.15.16:0/3246041078 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f0c10c170 msgr2=0x7f6f0c10e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:15.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.560+0000 7f6f12045640 1 --2- 172.21.15.16:0/3246041078 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f0c10c170 0x7f6f0c10e610 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f6f04009ca0 tx=0x7f6f0402a950 comp rx=0 tx=0).stop 2024-09-12T02:07:15.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.560+0000 7f6f12045640 1 -- 172.21.15.16:0/3246041078 shutdown_connections 2024-09-12T02:07:15.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.560+0000 7f6f12045640 1 --2- 172.21.15.16:0/3246041078 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f0c10c170 0x7f6f0c10e610 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:15.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.560+0000 7f6f12045640 1 --2- 172.21.15.16:0/3246041078 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f0c103660 0x7f6f0c10bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:15.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.560+0000 7f6f12045640 1 --2- 172.21.15.16:0/3246041078 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6f0c102d40 0x7f6f0c103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:15.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.560+0000 7f6f12045640 1 -- 172.21.15.16:0/3246041078 >> 172.21.15.16:0/3246041078 conn(0x7f6f0c0fc7a0 msgr2=0x7f6f0c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:07:15.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.560+0000 7f6f12045640 1 -- 172.21.15.16:0/3246041078 shutdown_connections 2024-09-12T02:07:15.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.561+0000 7f6f12045640 1 -- 172.21.15.16:0/3246041078 wait complete. 2024-09-12T02:07:15.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.561+0000 7f6f12045640 1 Processor -- start 2024-09-12T02:07:15.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.561+0000 7f6f12045640 1 -- start start 2024-09-12T02:07:15.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.562+0000 7f6f12045640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6f0c102d40 0x7f6f0c1984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:15.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.562+0000 7f6f12045640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f0c103660 0x7f6f0c1989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:15.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.562+0000 7f6f12045640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f0c10c170 0x7f6f0c19cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:15.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.562+0000 7f6f12045640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6f0c111fb0 con 0x7f6f0c103660 2024-09-12T02:07:15.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.562+0000 7f6f12045640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6f0c111e30 con 0x7f6f0c10c170 2024-09-12T02:07:15.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.562+0000 7f6f12045640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f6f0c112130 con 0x7f6f0c102d40 2024-09-12T02:07:15.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.562+0000 7f6f0affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f0c103660 0x7f6f0c1989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:15.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.562+0000 7f6f0affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f0c103660 0x7f6f0c1989f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49424/0 (socket says 172.21.15.16:49424) 2024-09-12T02:07:15.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.562+0000 7f6f0affd640 1 -- 172.21.15.16:0/3340755292 learned_addr learned my addr 172.21.15.16:0/3340755292 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:07:15.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.562+0000 7f6f0bfff640 1 --2- 172.21.15.16:0/3340755292 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f0c10c170 0x7f6f0c19cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:15.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.562+0000 7f6f0b7fe640 1 --2- 172.21.15.16:0/3340755292 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6f0c102d40 0x7f6f0c1984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:15.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.562+0000 7f6f0affd640 1 -- 172.21.15.16:0/3340755292 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6f0c102d40 msgr2=0x7f6f0c1984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:15.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.562+0000 7f6f0affd640 1 --2- 172.21.15.16:0/3340755292 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6f0c102d40 0x7f6f0c1984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:15.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.562+0000 7f6f0affd640 1 -- 172.21.15.16:0/3340755292 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f0c10c170 msgr2=0x7f6f0c19cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:15.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.562+0000 7f6f0affd640 1 --2- 172.21.15.16:0/3340755292 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f0c10c170 0x7f6f0c19cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:15.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.562+0000 7f6f0affd640 1 -- 172.21.15.16:0/3340755292 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f0c19d460 con 0x7f6f0c103660 2024-09-12T02:07:15.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.563+0000 7f6f0affd640 1 --2- 172.21.15.16:0/3340755292 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f0c103660 0x7f6f0c1989f0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f6f0000c940 tx=0x7f6f0000cd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:07:15.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.563+0000 7f6f08ff9640 1 -- 172.21.15.16:0/3340755292 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f00004220 con 0x7f6f0c103660 2024-09-12T02:07:15.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.563+0000 7f6f08ff9640 1 -- 172.21.15.16:0/3340755292 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f000043c0 con 0x7f6f0c103660 2024-09-12T02:07:15.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.563+0000 7f6f12045640 1 -- 172.21.15.16:0/3340755292 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f0c19d750 con 0x7f6f0c103660 2024-09-12T02:07:15.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.563+0000 7f6f08ff9640 1 -- 172.21.15.16:0/3340755292 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f00005650 con 0x7f6f0c103660 2024-09-12T02:07:15.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.563+0000 7f6f12045640 1 -- 172.21.15.16:0/3340755292 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f0c1a5030 con 0x7f6f0c103660 2024-09-12T02:07:15.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.565+0000 7f6f08ff9640 1 -- 172.21.15.16:0/3340755292 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6f000079b0 con 0x7f6f0c103660 2024-09-12T02:07:15.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.565+0000 7f6f1441e740 1 Processor -- start 2024-09-12T02:07:15.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.565+0000 7f6f1441e740 1 -- start start 2024-09-12T02:07:15.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.566+0000 7f6f1441e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563292113b40 0x563292112090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:15.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.566+0000 7f6f1441e740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5632921125d0 0x563292112a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:15.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.566+0000 7f6f1441e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56329211d370 0x56329211f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:15.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.566+0000 7f6f1441e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5632920782f0 con 0x563292113b40 2024-09-12T02:07:15.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.566+0000 7f6f1441e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56329201fcf0 con 0x56329211d370 2024-09-12T02:07:15.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.566+0000 7f6f1441e740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5632920742b0 con 0x5632921125d0 2024-09-12T02:07:15.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.566+0000 7f6f08ff9640 1 --2- 172.21.15.16:0/3340755292 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f6ef40a2c00 0x7f6ef40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:15.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.566+0000 7f6f0bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563292113b40 0x563292112090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:15.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.566+0000 7f6f0bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563292113b40 0x563292112090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49434/0 (socket says 172.21.15.16:49434) 2024-09-12T02:07:15.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.566+0000 7f6f0bfff640 1 -- 172.21.15.16:0/673253072 learned_addr learned my addr 172.21.15.16:0/673253072 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:07:15.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.566+0000 7f6f0bfff640 1 --2- 172.21.15.16:0/3340755292 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f6ef40a2c00 0x7f6ef40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:15.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.566+0000 7f6f08ff9640 1 -- 172.21.15.16:0/3340755292 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f6f000cea90 con 0x7f6f0c103660 2024-09-12T02:07:15.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.566+0000 7f6f0affd640 1 --2- 172.21.15.16:0/673253072 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56329211d370 0x56329211f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:15.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.566+0000 7f6f0b7fe640 1 --2- 172.21.15.16:0/673253072 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5632921125d0 0x563292112a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:15.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.566+0000 7f6f0bfff640 1 -- 172.21.15.16:0/673253072 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5632921125d0 msgr2=0x563292112a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:15.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.566+0000 7f6f0bfff640 1 --2- 172.21.15.16:0/673253072 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5632921125d0 0x563292112a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:15.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.566+0000 7f6f0bfff640 1 -- 172.21.15.16:0/673253072 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56329211d370 msgr2=0x56329211f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:15.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.566+0000 7f6f0bfff640 1 --2- 172.21.15.16:0/673253072 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56329211d370 0x56329211f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:15.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.566+0000 7f6f0bfff640 1 -- 172.21.15.16:0/673253072 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5632920043a0 con 0x563292113b40 2024-09-12T02:07:15.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.566+0000 7f6f0affd640 1 --2- 172.21.15.16:0/673253072 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56329211d370 0x56329211f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:07:15.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.567+0000 7f6f0bfff640 1 --2- 172.21.15.16:0/673253072 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563292113b40 0x563292112090 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f6f04009a60 tx=0x7f6f0402f780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44e25de33f7b99df server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:07:15.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.567+0000 7f6f0bfff640 1 --2- 172.21.15.16:0/3340755292 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f6ef40a2c00 0x7f6ef40a50c0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f6f04030040 tx=0x7f6f04043000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:07:15.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.567+0000 7f6ef1ffb640 1 -- 172.21.15.16:0/673253072 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f04034040 con 0x563292113b40 2024-09-12T02:07:15.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.567+0000 7f6ef1ffb640 1 -- 172.21.15.16:0/673253072 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f0404e070 con 0x563292113b40 2024-09-12T02:07:15.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.567+0000 7f6ef1ffb640 1 -- 172.21.15.16:0/673253072 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f04005050 con 0x563292113b40 2024-09-12T02:07:15.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.568+0000 7f6f1441e740 1 -- 172.21.15.16:0/673253072 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563292113b40 msgr2=0x563292112090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:15.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.568+0000 7f6f1441e740 1 --2- 172.21.15.16:0/673253072 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563292113b40 0x563292112090 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f6f04009a60 tx=0x7f6f0402f780 comp rx=0 tx=0).stop 2024-09-12T02:07:15.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.568+0000 7f6f1441e740 1 -- 172.21.15.16:0/673253072 shutdown_connections 2024-09-12T02:07:15.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.568+0000 7f6f1441e740 1 --2- 172.21.15.16:0/673253072 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56329211d370 0x56329211f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:15.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.568+0000 7f6f1441e740 1 --2- 172.21.15.16:0/673253072 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5632921125d0 0x563292112a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:15.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.568+0000 7f6f1441e740 1 --2- 172.21.15.16:0/673253072 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563292113b40 0x563292112090 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:15.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.568+0000 7f6f1441e740 1 -- 172.21.15.16:0/673253072 >> 172.21.15.16:0/673253072 conn(0x5632920db310 msgr2=0x56329210ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:07:15.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.568+0000 7f6f1441e740 1 -- 172.21.15.16:0/673253072 shutdown_connections 2024-09-12T02:07:15.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.568+0000 7f6f1441e740 1 -- 172.21.15.16:0/673253072 wait complete. 2024-09-12T02:07:15.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.568+0000 7f6f1441e740 1 -- start start 2024-09-12T02:07:15.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.568+0000 7f6f1441e740 1 Processor -- start 2024-09-12T02:07:15.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.569+0000 7f6f1441e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5632921125d0 0x56329211cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:15.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.569+0000 7f6f1441e740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563292113b40 0x56329211a050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:15.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.569+0000 7f6f1441e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56329211d370 0x56329211a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:15.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.569+0000 7f6f1441e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x563292120010 con 0x56329211d370 2024-09-12T02:07:15.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.569+0000 7f6f1441e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x563292033060 con 0x5632921125d0 2024-09-12T02:07:15.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.569+0000 7f6f1441e740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x563292120190 con 0x563292113b40 2024-09-12T02:07:15.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.569+0000 7f6f1441e740 10 client.0 _ll_register_callbacks cb 0x563291d868c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:07:15.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.569+0000 7f6f0bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56329211d370 0x56329211a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:15.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.569+0000 7f6f0bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56329211d370 0x56329211a7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49448/0 (socket says 172.21.15.16:49448) 2024-09-12T02:07:15.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.569+0000 7f6f0bfff640 1 -- 172.21.15.16:0/1881507618 learned_addr learned my addr 172.21.15.16:0/1881507618 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:07:15.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.569+0000 7f6f0affd640 1 --2- 172.21.15.16:0/1881507618 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563292113b40 0x56329211a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:15.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.569+0000 7f6f0b7fe640 1 --2- 172.21.15.16:0/1881507618 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5632921125d0 0x56329211cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:15.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.569+0000 7f6f0bfff640 1 -- 172.21.15.16:0/1881507618 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563292113b40 msgr2=0x56329211a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:15.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.569+0000 7f6f0bfff640 1 --2- 172.21.15.16:0/1881507618 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563292113b40 0x56329211a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:15.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.569+0000 7f6f0bfff640 1 -- 172.21.15.16:0/1881507618 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5632921125d0 msgr2=0x56329211cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:15.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.569+0000 7f6f0bfff640 1 --2- 172.21.15.16:0/1881507618 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5632921125d0 0x56329211cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:15.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.569+0000 7f6f0bfff640 1 -- 172.21.15.16:0/1881507618 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563291de93c0 con 0x56329211d370 2024-09-12T02:07:15.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.569+0000 7f6f0b7fe640 1 --2- 172.21.15.16:0/1881507618 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5632921125d0 0x56329211cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:07:15.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.569+0000 7f6f0bfff640 1 --2- 172.21.15.16:0/1881507618 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56329211d370 0x56329211a7e0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f6f0402a990 tx=0x7f6f0402f490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8828fa38540e9cd server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:07:15.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.569+0000 7f6eeaffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:07:15.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.569+0000 7f6eeaffd640 1 -- 172.21.15.16:0/1881507618 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f04056070 con 0x56329211d370 2024-09-12T02:07:15.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.570+0000 7f6eeaffd640 1 -- 172.21.15.16:0/1881507618 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f6f04004d40 con 0x56329211d370 2024-09-12T02:07:15.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.570+0000 7f6f1441e740 1 -- 172.21.15.16:0/1881507618 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f0404e070 con 0x56329211d370 2024-09-12T02:07:15.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.570+0000 7f6eeaffd640 1 -- 172.21.15.16:0/1881507618 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6f04004040 con 0x56329211d370 2024-09-12T02:07:15.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.570+0000 7f6f1441e740 10 client.17013 fetch_fsmap learned FSMap version 25 2024-09-12T02:07:15.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.570+0000 7f6f1441e740 1 -- 172.21.15.16:0/1881507618 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x563292120010 con 0x56329211d370 2024-09-12T02:07:15.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.570+0000 7f6eeaffd640 1 -- 172.21.15.16:0/1881507618 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f0404e070 con 0x56329211d370 2024-09-12T02:07:15.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.570+0000 7f6eeaffd640 1 -- 172.21.15.16:0/1881507618 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f6f040304d0 con 0x56329211d370 2024-09-12T02:07:15.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.570+0000 7f6f1441e740 10 client.17013 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:07:15.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.570+0000 7f6f1441e740 10 client.17013 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-12T02:07:15.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.570+0000 7f6f1441e740 20 client.17013 populate_metadata read hostname 'smithi016' 2024-09-12T02:07:15.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.570+0000 7f6f1441e740 1 --2- 172.21.15.16:0/1881507618 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x56329212cc70 0x56329212f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:15.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.570+0000 7f6f1441e740 4 client.17013 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:07:15.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.570+0000 7f6f1441e740 1 -- 172.21.15.16:0/1881507618 --> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563292085860 con 0x56329212cc70 2024-09-12T02:07:15.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.570+0000 7f6f0b7fe640 1 --2- 172.21.15.16:0/1881507618 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x56329212cc70 0x56329212f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:15.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.571+0000 7f6f0b7fe640 1 --2- 172.21.15.16:0/1881507618 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x56329212cc70 0x56329212f060 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=69e9b3aab47fe3ab server_cookie=ba6d256bbdb5e1a3 in_seq=0 out_seq=0 2024-09-12T02:07:15.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.571+0000 7f6eeaffd640 10 client.17013 ms_handle_connect on v2:172.21.15.184:6836/1793393087 2024-09-12T02:07:15.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.572+0000 7f6eeaffd640 1 -- 172.21.15.16:0/1881507618 <== mds.2 v2:172.21.15.184:6836/1793393087 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563292085860 con 0x56329212cc70 2024-09-12T02:07:15.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.572+0000 7f6eeaffd640 10 client.17013 handle_command_reply: tid=0 2024-09-12T02:07:15.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.572+0000 7f6f1441e740 1 client.17013 shutdown 2024-09-12T02:07:15.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.572+0000 7f6f1441e740 1 -- 172.21.15.16:0/1881507618 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56329211d370 msgr2=0x56329211a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:15.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.572+0000 7f6f1441e740 1 --2- 172.21.15.16:0/1881507618 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56329211d370 0x56329211a7e0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f6f0402a990 tx=0x7f6f0402f490 comp rx=0 tx=0).stop 2024-09-12T02:07:15.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.572+0000 7f6f1441e740 1 -- 172.21.15.16:0/1881507618 shutdown_connections 2024-09-12T02:07:15.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.572+0000 7f6f1441e740 1 --2- 172.21.15.16:0/1881507618 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x56329212cc70 0x56329212f060 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:15.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.572+0000 7f6f1441e740 1 --2- 172.21.15.16:0/1881507618 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56329211d370 0x56329211a7e0 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:15.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.572+0000 7f6f1441e740 1 --2- 172.21.15.16:0/1881507618 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563292113b40 0x56329211a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:15.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.572+0000 7f6f1441e740 1 --2- 172.21.15.16:0/1881507618 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5632921125d0 0x56329211cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:15.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.572+0000 7f6f1441e740 1 -- 172.21.15.16:0/1881507618 >> 172.21.15.16:0/1881507618 conn(0x5632920db310 msgr2=0x563292114960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:07:15.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.572+0000 7f6eeaffd640 0 client.17013 ms_handle_reset on v2:172.21.15.184:6836/1793393087 2024-09-12T02:07:15.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.572+0000 7f6f1441e740 1 -- 172.21.15.16:0/1881507618 shutdown_connections 2024-09-12T02:07:15.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.572+0000 7f6f1441e740 1 -- 172.21.15.16:0/1881507618 wait complete. 2024-09-12T02:07:15.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.572+0000 7f6f1441e740 20 client.17013 trim_cache size 0 max 16384 2024-09-12T02:07:15.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.586+0000 7f6f1441e740 1 Processor -- start 2024-09-12T02:07:15.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.586+0000 7f6f1441e740 1 -- start start 2024-09-12T02:07:15.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.586+0000 7f6f1441e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632921125d0 0x563292115120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:15.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.586+0000 7f6f1441e740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563292113b40 0x563292115660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:15.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.586+0000 7f6f1441e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56329211d370 0x5632921347a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:15.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.586+0000 7f6f1441e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6f0c1afc90 con 0x5632921125d0 2024-09-12T02:07:15.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.586+0000 7f6f1441e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6f0c1ad560 con 0x56329211d370 2024-09-12T02:07:15.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.586+0000 7f6f1441e740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f6f0c1ab3b0 con 0x563292113b40 2024-09-12T02:07:15.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.586+0000 7f6f0b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632921125d0 0x563292115120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:15.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.586+0000 7f6f0b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632921125d0 0x563292115120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49450/0 (socket says 172.21.15.16:49450) 2024-09-12T02:07:15.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.586+0000 7f6f0b7fe640 1 -- 172.21.15.16:0/3006667927 learned_addr learned my addr 172.21.15.16:0/3006667927 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:07:15.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.586+0000 7f6f0affd640 1 --2- 172.21.15.16:0/3006667927 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563292113b40 0x563292115660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:15.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.586+0000 7f6f0bfff640 1 --2- 172.21.15.16:0/3006667927 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56329211d370 0x5632921347a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:15.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.586+0000 7f6f0b7fe640 1 -- 172.21.15.16:0/3006667927 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563292113b40 msgr2=0x563292115660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:15.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.586+0000 7f6f0b7fe640 1 --2- 172.21.15.16:0/3006667927 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563292113b40 0x563292115660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:15.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.586+0000 7f6f0b7fe640 1 -- 172.21.15.16:0/3006667927 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56329211d370 msgr2=0x5632921347a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:15.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.586+0000 7f6f0b7fe640 1 --2- 172.21.15.16:0/3006667927 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56329211d370 0x5632921347a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:15.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.586+0000 7f6f0b7fe640 1 -- 172.21.15.16:0/3006667927 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56329210f960 con 0x5632921125d0 2024-09-12T02:07:15.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.587+0000 7f6f0b7fe640 1 --2- 172.21.15.16:0/3006667927 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632921125d0 0x563292115120 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f6efc009e50 tx=0x7f6efc008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cd8e3bb733c7e98 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:07:15.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.587+0000 7f6ef17fa640 1 -- 172.21.15.16:0/3006667927 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6efc026420 con 0x5632921125d0 2024-09-12T02:07:15.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.587+0000 7f6ef17fa640 1 -- 172.21.15.16:0/3006667927 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6efc021030 con 0x5632921125d0 2024-09-12T02:07:15.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.587+0000 7f6ef17fa640 1 -- 172.21.15.16:0/3006667927 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6efc025550 con 0x5632921125d0 2024-09-12T02:07:15.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.587+0000 7f6f1441e740 1 -- 172.21.15.16:0/3006667927 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632921125d0 msgr2=0x563292115120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:15.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.588+0000 7f6f1441e740 1 --2- 172.21.15.16:0/3006667927 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632921125d0 0x563292115120 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f6efc009e50 tx=0x7f6efc008040 comp rx=0 tx=0).stop 2024-09-12T02:07:15.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.588+0000 7f6f1441e740 1 -- 172.21.15.16:0/3006667927 shutdown_connections 2024-09-12T02:07:15.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.588+0000 7f6f1441e740 1 --2- 172.21.15.16:0/3006667927 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56329211d370 0x5632921347a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:15.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.588+0000 7f6f1441e740 1 --2- 172.21.15.16:0/3006667927 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563292113b40 0x563292115660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:15.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.588+0000 7f6f1441e740 1 --2- 172.21.15.16:0/3006667927 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632921125d0 0x563292115120 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:15.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.588+0000 7f6f1441e740 1 -- 172.21.15.16:0/3006667927 >> 172.21.15.16:0/3006667927 conn(0x5632920db310 msgr2=0x5632921191a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:07:15.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.588+0000 7f6f1441e740 1 -- 172.21.15.16:0/3006667927 shutdown_connections 2024-09-12T02:07:15.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.588+0000 7f6f1441e740 1 -- 172.21.15.16:0/3006667927 wait complete. 2024-09-12T02:07:15.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.588+0000 7f6f1441e740 1 -- start start 2024-09-12T02:07:15.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.588+0000 7f6f1441e740 1 Processor -- start 2024-09-12T02:07:15.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.588+0000 7f6f1441e740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5632921125d0 0x563292125600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:15.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.588+0000 7f6f1441e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563292113b40 0x563292125b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:15.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.588+0000 7f6f1441e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56329211d370 0x5632921168e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:15.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.588+0000 7f6f1441e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6f0c1aba20 con 0x563292113b40 2024-09-12T02:07:15.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.588+0000 7f6f1441e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6f0c1ace90 con 0x56329211d370 2024-09-12T02:07:15.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.588+0000 7f6f1441e740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f6f0c1ac0f0 con 0x5632921125d0 2024-09-12T02:07:15.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.589+0000 7f6f1441e740 10 client.0 _ll_register_callbacks cb 0x7f6efc0093c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:07:15.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.589+0000 7f6f0affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563292113b40 0x563292125b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:15.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.589+0000 7f6f0affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563292113b40 0x563292125b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49458/0 (socket says 172.21.15.16:49458) 2024-09-12T02:07:15.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.589+0000 7f6f0affd640 1 -- 172.21.15.16:0/46011513 learned_addr learned my addr 172.21.15.16:0/46011513 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:07:15.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.589+0000 7f6f0b7fe640 1 --2- 172.21.15.16:0/46011513 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5632921125d0 0x563292125600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:15.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.589+0000 7f6f0bfff640 1 --2- 172.21.15.16:0/46011513 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56329211d370 0x5632921168e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:15.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.589+0000 7f6f0affd640 1 -- 172.21.15.16:0/46011513 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5632921125d0 msgr2=0x563292125600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:15.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.589+0000 7f6f0affd640 1 --2- 172.21.15.16:0/46011513 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5632921125d0 0x563292125600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:15.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.589+0000 7f6f0affd640 1 -- 172.21.15.16:0/46011513 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56329211d370 msgr2=0x5632921168e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:15.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.589+0000 7f6f0affd640 1 --2- 172.21.15.16:0/46011513 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56329211d370 0x5632921168e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:15.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.589+0000 7f6f0affd640 1 -- 172.21.15.16:0/46011513 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5632921170c0 con 0x563292113b40 2024-09-12T02:07:15.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.589+0000 7f6f0affd640 1 --2- 172.21.15.16:0/46011513 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563292113b40 0x563292125b40 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f6f00007590 tx=0x7f6f00005310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3903376420472e55 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:07:15.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.589+0000 7f6eeaffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:07:15.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.589+0000 7f6eeaffd640 1 -- 172.21.15.16:0/46011513 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f000b9060 con 0x563292113b40 2024-09-12T02:07:15.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.589+0000 7f6eeaffd640 1 -- 172.21.15.16:0/46011513 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f6f00007ca0 con 0x563292113b40 2024-09-12T02:07:15.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.589+0000 7f6f1441e740 1 -- 172.21.15.16:0/46011513 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6efc021030 con 0x563292113b40 2024-09-12T02:07:15.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.589+0000 7f6eeaffd640 1 -- 172.21.15.16:0/46011513 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6f0001d040 con 0x563292113b40 2024-09-12T02:07:15.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.590+0000 7f6f1441e740 10 client.17019 fetch_fsmap learned FSMap version 25 2024-09-12T02:07:15.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.590+0000 7f6f1441e740 1 -- 172.21.15.16:0/46011513 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56329219b660 con 0x563292113b40 2024-09-12T02:07:15.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.590+0000 7f6eeaffd640 1 -- 172.21.15.16:0/46011513 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6efc021030 con 0x563292113b40 2024-09-12T02:07:15.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.590+0000 7f6eeaffd640 1 -- 172.21.15.16:0/46011513 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f6f0007f070 con 0x563292113b40 2024-09-12T02:07:15.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.590+0000 7f6f1441e740 10 client.17019 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:07:15.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.590+0000 7f6f1441e740 10 client.17019 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-12T02:07:15.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.590+0000 7f6f1441e740 20 client.17019 populate_metadata read hostname 'smithi016' 2024-09-12T02:07:15.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.590+0000 7f6f1441e740 1 --2- 172.21.15.16:0/46011513 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x56329219bde0 0x56329219e200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:15.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.590+0000 7f6f1441e740 4 client.17019 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T02:07:15.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.590+0000 7f6f1441e740 1 -- 172.21.15.16:0/46011513 --> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56329207fd70 con 0x56329219bde0 2024-09-12T02:07:15.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.590+0000 7f6f0b7fe640 1 --2- 172.21.15.16:0/46011513 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x56329219bde0 0x56329219e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:15.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.591+0000 7f6f0b7fe640 1 --2- 172.21.15.16:0/46011513 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x56329219bde0 0x56329219e200 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b1aafe725ba4e808 server_cookie=ab995bca144fa399 in_seq=0 out_seq=0 2024-09-12T02:07:15.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.591+0000 7f6eeaffd640 10 client.17019 ms_handle_connect on v2:172.21.15.184:6836/1793393087 2024-09-12T02:07:15.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.591+0000 7f6eeaffd640 1 -- 172.21.15.16:0/46011513 <== mds.2 v2:172.21.15.184:6836/1793393087 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56329207fd70 con 0x56329219bde0 2024-09-12T02:07:15.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.591+0000 7f6eeaffd640 10 client.17019 handle_command_reply: tid=0 2024-09-12T02:07:15.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.591+0000 7f6f1441e740 1 client.17019 shutdown 2024-09-12T02:07:15.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.591+0000 7f6f1441e740 1 -- 172.21.15.16:0/46011513 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563292113b40 msgr2=0x563292125b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:15.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.591+0000 7f6f1441e740 1 --2- 172.21.15.16:0/46011513 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563292113b40 0x563292125b40 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f6f00007590 tx=0x7f6f00005310 comp rx=0 tx=0).stop 2024-09-12T02:07:15.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.591+0000 7f6f1441e740 1 -- 172.21.15.16:0/46011513 shutdown_connections 2024-09-12T02:07:15.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.591+0000 7f6f1441e740 1 --2- 172.21.15.16:0/46011513 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56329211d370 0x5632921168e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:15.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.591+0000 7f6f1441e740 1 --2- 172.21.15.16:0/46011513 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x56329219bde0 0x56329219e200 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:15.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.591+0000 7f6f1441e740 1 --2- 172.21.15.16:0/46011513 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563292113b40 0x563292125b40 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:15.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.591+0000 7f6f1441e740 1 --2- 172.21.15.16:0/46011513 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5632921125d0 0x563292125600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:15.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.591+0000 7f6f1441e740 1 -- 172.21.15.16:0/46011513 >> 172.21.15.16:0/46011513 conn(0x5632920db310 msgr2=0x56329211b8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:07:15.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.591+0000 7f6eeaffd640 0 client.17019 ms_handle_reset on v2:172.21.15.184:6836/1793393087 2024-09-12T02:07:15.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.591+0000 7f6f1441e740 1 -- 172.21.15.16:0/46011513 shutdown_connections 2024-09-12T02:07:15.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.591+0000 7f6f1441e740 1 -- 172.21.15.16:0/46011513 wait complete. 2024-09-12T02:07:15.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.591+0000 7f6f1441e740 20 client.17019 trim_cache size 0 max 16384 2024-09-12T02:07:15.598 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T02:07:15.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.592+0000 7f6f12045640 1 -- 172.21.15.16:0/3340755292 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f6ef40a2c00 msgr2=0x7f6ef40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:15.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.592+0000 7f6f12045640 1 --2- 172.21.15.16:0/3340755292 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f6ef40a2c00 0x7f6ef40a50c0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f6f04030040 tx=0x7f6f04043000 comp rx=0 tx=0).stop 2024-09-12T02:07:15.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.592+0000 7f6f12045640 1 -- 172.21.15.16:0/3340755292 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f0c103660 msgr2=0x7f6f0c1989f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:15.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.592+0000 7f6f12045640 1 --2- 172.21.15.16:0/3340755292 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f0c103660 0x7f6f0c1989f0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f6f0000c940 tx=0x7f6f0000cd40 comp rx=0 tx=0).stop 2024-09-12T02:07:15.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.592+0000 7f6f12045640 1 -- 172.21.15.16:0/3340755292 shutdown_connections 2024-09-12T02:07:15.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.592+0000 7f6f12045640 1 --2- 172.21.15.16:0/3340755292 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f6ef40a2c00 0x7f6ef40a50c0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:15.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.592+0000 7f6f12045640 1 --2- 172.21.15.16:0/3340755292 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f0c10c170 0x7f6f0c19cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:15.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.592+0000 7f6f12045640 1 --2- 172.21.15.16:0/3340755292 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f0c103660 0x7f6f0c1989f0 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:15.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.592+0000 7f6f12045640 1 --2- 172.21.15.16:0/3340755292 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6f0c102d40 0x7f6f0c1984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:15.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.592+0000 7f6f12045640 1 -- 172.21.15.16:0/3340755292 >> 172.21.15.16:0/3340755292 conn(0x7f6f0c0fc7a0 msgr2=0x7f6f0c0fd400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:07:15.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.592+0000 7f6f12045640 1 -- 172.21.15.16:0/3340755292 shutdown_connections 2024-09-12T02:07:15.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:15.592+0000 7f6f12045640 1 -- 172.21.15.16:0/3340755292 wait complete. 2024-09-12T02:07:16.378 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:07:16 smithi016 ceph-mon[30947]: pgmap v495: 97 pgs: 97 active+clean; 2.9 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 21 KiB/s wr, 15 op/s 2024-09-12T02:07:16.407 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-12T02:07:16.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:07:16 smithi184 ceph-mon[35593]: pgmap v495: 97 pgs: 97 active+clean; 2.9 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 21 KiB/s wr, 15 op/s 2024-09-12T02:07:16.665 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:07:16.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:07:16 smithi033 ceph-mon[35332]: pgmap v495: 97 pgs: 97 active+clean; 2.9 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 21 KiB/s wr, 15 op/s 2024-09-12T02:07:17.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.242+0000 7f2ede15a640 1 -- 172.21.15.16:0/3978818173 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ed8102d40 msgr2=0x7f2ed8103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:17.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.242+0000 7f2ede15a640 1 --2- 172.21.15.16:0/3978818173 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ed8102d40 0x7f2ed8103120 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f2ec8009900 tx=0x7f2ec802a480 comp rx=0 tx=0).stop 2024-09-12T02:07:17.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.242+0000 7f2ede15a640 1 -- 172.21.15.16:0/3978818173 shutdown_connections 2024-09-12T02:07:17.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.242+0000 7f2ede15a640 1 --2- 172.21.15.16:0/3978818173 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2ed810c170 0x7f2ed810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:17.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.242+0000 7f2ede15a640 1 --2- 172.21.15.16:0/3978818173 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ed8103660 0x7f2ed810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:17.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.242+0000 7f2ede15a640 1 --2- 172.21.15.16:0/3978818173 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ed8102d40 0x7f2ed8103120 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:17.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.242+0000 7f2ede15a640 1 -- 172.21.15.16:0/3978818173 >> 172.21.15.16:0/3978818173 conn(0x7f2ed80fc7a0 msgr2=0x7f2ed80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:07:17.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.242+0000 7f2ede15a640 1 -- 172.21.15.16:0/3978818173 shutdown_connections 2024-09-12T02:07:17.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.242+0000 7f2ede15a640 1 -- 172.21.15.16:0/3978818173 wait complete. 2024-09-12T02:07:17.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.243+0000 7f2ede15a640 1 Processor -- start 2024-09-12T02:07:17.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.243+0000 7f2ede15a640 1 -- start start 2024-09-12T02:07:17.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.243+0000 7f2ede15a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ed8102d40 0x7f2ed81009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:17.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.243+0000 7f2ede15a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ed8103660 0x7f2ed8100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:17.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.243+0000 7f2ede15a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2ed810c170 0x7f2ed8101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:17.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.243+0000 7f2ede15a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2ed8111f70 con 0x7f2ed8103660 2024-09-12T02:07:17.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.243+0000 7f2ede15a640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2ed8111df0 con 0x7f2ed8102d40 2024-09-12T02:07:17.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.243+0000 7f2ede15a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f2ed81120f0 con 0x7f2ed810c170 2024-09-12T02:07:17.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.244+0000 7f2ed6ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ed8103660 0x7f2ed8100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:17.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.244+0000 7f2ed6ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ed8103660 0x7f2ed8100ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49478/0 (socket says 172.21.15.16:49478) 2024-09-12T02:07:17.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.244+0000 7f2ed6ffd640 1 -- 172.21.15.16:0/3753435066 learned_addr learned my addr 172.21.15.16:0/3753435066 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:07:17.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.244+0000 7f2ed77fe640 1 --2- 172.21.15.16:0/3753435066 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ed8102d40 0x7f2ed81009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:17.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.244+0000 7f2ed7fff640 1 --2- 172.21.15.16:0/3753435066 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2ed810c170 0x7f2ed8101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:17.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.244+0000 7f2ed6ffd640 1 -- 172.21.15.16:0/3753435066 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2ed810c170 msgr2=0x7f2ed8101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:17.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.244+0000 7f2ed6ffd640 1 --2- 172.21.15.16:0/3753435066 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2ed810c170 0x7f2ed8101420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:17.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.244+0000 7f2ed6ffd640 1 -- 172.21.15.16:0/3753435066 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ed8102d40 msgr2=0x7f2ed81009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:17.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.244+0000 7f2ed6ffd640 1 --2- 172.21.15.16:0/3753435066 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ed8102d40 0x7f2ed81009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:17.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.244+0000 7f2ed6ffd640 1 -- 172.21.15.16:0/3753435066 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ed81a6910 con 0x7f2ed8103660 2024-09-12T02:07:17.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.244+0000 7f2ed7fff640 1 --2- 172.21.15.16:0/3753435066 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2ed810c170 0x7f2ed8101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:07:17.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.244+0000 7f2ed6ffd640 1 --2- 172.21.15.16:0/3753435066 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ed8103660 0x7f2ed8100ee0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc00d950 tx=0x7f2ecc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:07:17.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.244+0000 7f2ed4ff9640 1 -- 172.21.15.16:0/3753435066 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ecc017030 con 0x7f2ed8103660 2024-09-12T02:07:17.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.244+0000 7f2ed4ff9640 1 -- 172.21.15.16:0/3753435066 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ecc004020 con 0x7f2ed8103660 2024-09-12T02:07:17.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.244+0000 7f2ede15a640 1 -- 172.21.15.16:0/3753435066 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ed81a6c00 con 0x7f2ed8103660 2024-09-12T02:07:17.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.244+0000 7f2ed4ff9640 1 -- 172.21.15.16:0/3753435066 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ecc0047a0 con 0x7f2ed8103660 2024-09-12T02:07:17.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.244+0000 7f2ede15a640 1 -- 172.21.15.16:0/3753435066 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ed81a7140 con 0x7f2ed8103660 2024-09-12T02:07:17.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.247+0000 7f2ed4ff9640 1 -- 172.21.15.16:0/3753435066 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f2ecc0041d0 con 0x7f2ed8103660 2024-09-12T02:07:17.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.247+0000 7f2ee0533740 1 Processor -- start 2024-09-12T02:07:17.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.247+0000 7f2ee0533740 1 -- start start 2024-09-12T02:07:17.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.247+0000 7f2ed4ff9640 1 --2- 172.21.15.16:0/3753435066 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f2ec00a2b80 0x7f2ec00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:17.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.247+0000 7f2ee0533740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563317c8eb40 0x563317c8d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:17.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.247+0000 7f2ee0533740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563317c8d5d0 0x563317c8da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:17.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.247+0000 7f2ee0533740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563317c98370 0x563317c9a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:17.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.247+0000 7f2ee0533740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x563317bf32f0 con 0x563317c98370 2024-09-12T02:07:17.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.247+0000 7f2ee0533740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x563317b9acf0 con 0x563317c8d5d0 2024-09-12T02:07:17.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.247+0000 7f2ee0533740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x563317bef2b0 con 0x563317c8eb40 2024-09-12T02:07:17.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.247+0000 7f2ed7fff640 1 --2- 172.21.15.16:0/3753435066 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f2ec00a2b80 0x7f2ec00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:17.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.248+0000 7f2ed4ff9640 1 -- 172.21.15.16:0/3753435066 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f2ecc0ce930 con 0x7f2ed8103660 2024-09-12T02:07:17.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.248+0000 7f2ed7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563317c98370 0x563317c9a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:17.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.248+0000 7f2ed7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563317c98370 0x563317c9a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49488/0 (socket says 172.21.15.16:49488) 2024-09-12T02:07:17.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.248+0000 7f2ed7fff640 1 -- 172.21.15.16:0/3851957028 learned_addr learned my addr 172.21.15.16:0/3851957028 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:07:17.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.248+0000 7f2ed6ffd640 1 --2- 172.21.15.16:0/3851957028 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563317c8d5d0 0x563317c8da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:17.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.248+0000 7f2ed77fe640 1 --2- 172.21.15.16:0/3851957028 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563317c8eb40 0x563317c8d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:17.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.248+0000 7f2ed7fff640 1 -- 172.21.15.16:0/3851957028 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563317c8eb40 msgr2=0x563317c8d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:17.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.248+0000 7f2ed7fff640 1 --2- 172.21.15.16:0/3851957028 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563317c8eb40 0x563317c8d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:17.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.248+0000 7f2ed7fff640 1 -- 172.21.15.16:0/3851957028 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563317c8d5d0 msgr2=0x563317c8da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:17.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.248+0000 7f2ed7fff640 1 --2- 172.21.15.16:0/3851957028 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563317c8d5d0 0x563317c8da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:17.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.248+0000 7f2ed7fff640 1 -- 172.21.15.16:0/3851957028 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563317b7f3a0 con 0x563317c98370 2024-09-12T02:07:17.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.248+0000 7f2ed7fff640 1 --2- 172.21.15.16:0/3753435066 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f2ec00a2b80 0x7f2ec00a5040 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f2ed0005c40 tx=0x7f2ed0009d70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:07:17.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.248+0000 7f2ed7fff640 1 --2- 172.21.15.16:0/3851957028 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563317c98370 0x563317c9a760 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f2ed0011ce0 tx=0x7f2ed001b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce76494235b15758 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:07:17.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.248+0000 7f2ebdffb640 1 -- 172.21.15.16:0/3851957028 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed0022070 con 0x563317c98370 2024-09-12T02:07:17.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.248+0000 7f2ebdffb640 1 -- 172.21.15.16:0/3851957028 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ed000fc00 con 0x563317c98370 2024-09-12T02:07:17.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.248+0000 7f2ebdffb640 1 -- 172.21.15.16:0/3851957028 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed0005360 con 0x563317c98370 2024-09-12T02:07:17.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.249+0000 7f2ee0533740 1 -- 172.21.15.16:0/3851957028 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563317c98370 msgr2=0x563317c9a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:17.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.249+0000 7f2ee0533740 1 --2- 172.21.15.16:0/3851957028 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563317c98370 0x563317c9a760 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f2ed0011ce0 tx=0x7f2ed001b000 comp rx=0 tx=0).stop 2024-09-12T02:07:17.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.249+0000 7f2ee0533740 1 -- 172.21.15.16:0/3851957028 shutdown_connections 2024-09-12T02:07:17.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.249+0000 7f2ee0533740 1 --2- 172.21.15.16:0/3851957028 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563317c98370 0x563317c9a760 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:17.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.249+0000 7f2ee0533740 1 --2- 172.21.15.16:0/3851957028 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563317c8d5d0 0x563317c8da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:17.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.249+0000 7f2ee0533740 1 --2- 172.21.15.16:0/3851957028 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563317c8eb40 0x563317c8d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:17.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.249+0000 7f2ee0533740 1 -- 172.21.15.16:0/3851957028 >> 172.21.15.16:0/3851957028 conn(0x563317c56310 msgr2=0x563317c89c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:07:17.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.249+0000 7f2ee0533740 1 -- 172.21.15.16:0/3851957028 shutdown_connections 2024-09-12T02:07:17.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.249+0000 7f2ee0533740 1 -- 172.21.15.16:0/3851957028 wait complete. 2024-09-12T02:07:17.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.249+0000 7f2ee0533740 1 -- start start 2024-09-12T02:07:17.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.250+0000 7f2ee0533740 1 Processor -- start 2024-09-12T02:07:17.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.250+0000 7f2ee0533740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563317c8d5d0 0x563317c97d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:17.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.250+0000 7f2ee0533740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563317c8eb40 0x563317c94ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:17.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.250+0000 7f2ee0533740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563317c98370 0x563317c955e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:17.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.250+0000 7f2ee0533740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x563317c9b010 con 0x563317c8eb40 2024-09-12T02:07:17.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.250+0000 7f2ee0533740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x563317bae060 con 0x563317c98370 2024-09-12T02:07:17.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.250+0000 7f2ee0533740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x563317c9b190 con 0x563317c8d5d0 2024-09-12T02:07:17.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.250+0000 7f2ee0533740 10 client.0 _ll_register_callbacks cb 0x5633179018c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:07:17.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.250+0000 7f2ed6ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563317c8eb40 0x563317c94ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:17.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.250+0000 7f2ed6ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563317c8eb40 0x563317c94ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49494/0 (socket says 172.21.15.16:49494) 2024-09-12T02:07:17.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.250+0000 7f2ed6ffd640 1 -- 172.21.15.16:0/1957869618 learned_addr learned my addr 172.21.15.16:0/1957869618 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:07:17.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.250+0000 7f2ed77fe640 1 --2- 172.21.15.16:0/1957869618 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563317c8d5d0 0x563317c97d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:17.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.250+0000 7f2ed7fff640 1 --2- 172.21.15.16:0/1957869618 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563317c98370 0x563317c955e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:17.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.250+0000 7f2ed6ffd640 1 -- 172.21.15.16:0/1957869618 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563317c8d5d0 msgr2=0x563317c97d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:17.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.250+0000 7f2ed6ffd640 1 --2- 172.21.15.16:0/1957869618 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563317c8d5d0 0x563317c97d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:17.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.250+0000 7f2ed6ffd640 1 -- 172.21.15.16:0/1957869618 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563317c98370 msgr2=0x563317c955e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:17.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.250+0000 7f2ed6ffd640 1 --2- 172.21.15.16:0/1957869618 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563317c98370 0x563317c955e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:17.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.250+0000 7f2ed6ffd640 1 -- 172.21.15.16:0/1957869618 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5633179643c0 con 0x563317c8eb40 2024-09-12T02:07:17.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.250+0000 7f2ed77fe640 1 --2- 172.21.15.16:0/1957869618 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563317c8d5d0 0x563317c97d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:07:17.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.251+0000 7f2ed6ffd640 1 --2- 172.21.15.16:0/1957869618 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563317c8eb40 0x563317c94ff0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc09af30 tx=0x7f2ecc09ae90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1c3a306befd1cae server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:07:17.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.251+0000 7f2eb6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:07:17.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.251+0000 7f2eb6ffd640 1 -- 172.21.15.16:0/1957869618 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ecc0a3ea0 con 0x563317c8eb40 2024-09-12T02:07:17.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.251+0000 7f2eb6ffd640 1 -- 172.21.15.16:0/1957869618 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f2ecc0978a0 con 0x563317c8eb40 2024-09-12T02:07:17.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.251+0000 7f2ee0533740 1 -- 172.21.15.16:0/1957869618 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ed000fc00 con 0x563317c8eb40 2024-09-12T02:07:17.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.251+0000 7f2eb6ffd640 1 -- 172.21.15.16:0/1957869618 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2ecc0b4140 con 0x563317c8eb40 2024-09-12T02:07:17.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.251+0000 7f2ee0533740 10 client.17031 fetch_fsmap learned FSMap version 25 2024-09-12T02:07:17.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.251+0000 7f2ee0533740 1 -- 172.21.15.16:0/1957869618 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x563317ca6ad0 con 0x563317c8eb40 2024-09-12T02:07:17.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.251+0000 7f2eb6ffd640 1 -- 172.21.15.16:0/1957869618 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ed000fc00 con 0x563317c8eb40 2024-09-12T02:07:17.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.251+0000 7f2eb6ffd640 1 -- 172.21.15.16:0/1957869618 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f2ecc0beb30 con 0x563317c8eb40 2024-09-12T02:07:17.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.251+0000 7f2ee0533740 10 client.17031 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:07:17.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.251+0000 7f2ee0533740 10 client.17031 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T02:07:17.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.251+0000 7f2ee0533740 20 client.17031 populate_metadata read hostname 'smithi016' 2024-09-12T02:07:17.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.251+0000 7f2ee0533740 1 --2- 172.21.15.16:0/1957869618 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x563317ca7e70 0x563317caa2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:17.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.251+0000 7f2ee0533740 4 client.17031 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:07:17.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.251+0000 7f2ee0533740 1 -- 172.21.15.16:0/1957869618 --> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563317c00860 con 0x563317ca7e70 2024-09-12T02:07:17.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.252+0000 7f2ed77fe640 1 --2- 172.21.15.16:0/1957869618 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x563317ca7e70 0x563317caa2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:17.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.252+0000 7f2ed77fe640 1 --2- 172.21.15.16:0/1957869618 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x563317ca7e70 0x563317caa2c0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=dcdcaed64545c08b server_cookie=cd9ab3841897d38b in_seq=0 out_seq=0 2024-09-12T02:07:17.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.252+0000 7f2eb6ffd640 10 client.17031 ms_handle_connect on v2:172.21.15.184:6838/4203347941 2024-09-12T02:07:17.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.253+0000 7f2eb6ffd640 1 -- 172.21.15.16:0/1957869618 <== mds.1 v2:172.21.15.184:6838/4203347941 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563317c00860 con 0x563317ca7e70 2024-09-12T02:07:17.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.253+0000 7f2eb6ffd640 10 client.17031 handle_command_reply: tid=0 2024-09-12T02:07:17.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.253+0000 7f2ee0533740 1 client.17031 shutdown 2024-09-12T02:07:17.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.253+0000 7f2ee0533740 1 -- 172.21.15.16:0/1957869618 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563317c8eb40 msgr2=0x563317c94ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:17.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.253+0000 7f2ee0533740 1 --2- 172.21.15.16:0/1957869618 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563317c8eb40 0x563317c94ff0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc09af30 tx=0x7f2ecc09ae90 comp rx=0 tx=0).stop 2024-09-12T02:07:17.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.253+0000 7f2ee0533740 1 -- 172.21.15.16:0/1957869618 shutdown_connections 2024-09-12T02:07:17.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.253+0000 7f2ee0533740 1 --2- 172.21.15.16:0/1957869618 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x563317ca7e70 0x563317caa2c0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:17.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.253+0000 7f2ee0533740 1 --2- 172.21.15.16:0/1957869618 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563317c98370 0x563317c955e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:17.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.253+0000 7f2ee0533740 1 --2- 172.21.15.16:0/1957869618 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563317c8eb40 0x563317c94ff0 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:17.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.253+0000 7f2ee0533740 1 --2- 172.21.15.16:0/1957869618 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563317c8d5d0 0x563317c97d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:17.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.253+0000 7f2eb6ffd640 0 client.17031 ms_handle_reset on v2:172.21.15.184:6838/4203347941 2024-09-12T02:07:17.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.253+0000 7f2ee0533740 1 -- 172.21.15.16:0/1957869618 >> 172.21.15.16:0/1957869618 conn(0x563317c56310 msgr2=0x563317c8f960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:07:17.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.253+0000 7f2ee0533740 1 -- 172.21.15.16:0/1957869618 shutdown_connections 2024-09-12T02:07:17.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.253+0000 7f2ee0533740 1 -- 172.21.15.16:0/1957869618 wait complete. 2024-09-12T02:07:17.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.253+0000 7f2ee0533740 20 client.17031 trim_cache size 0 max 16384 2024-09-12T02:07:17.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.267+0000 7f2ee0533740 1 Processor -- start 2024-09-12T02:07:17.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.267+0000 7f2ee0533740 1 -- start start 2024-09-12T02:07:17.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.267+0000 7f2ee0533740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563317c8d5d0 0x563317ca4300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:17.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.267+0000 7f2ee0533740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563317c8eb40 0x563317c948b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:17.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.267+0000 7f2ee0533740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563317c98370 0x563317c96f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:17.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.267+0000 7f2ee0533740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2ed81b1ff0 con 0x563317c8eb40 2024-09-12T02:07:17.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.267+0000 7f2ee0533740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2ed81af8c0 con 0x563317c8d5d0 2024-09-12T02:07:17.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.267+0000 7f2ee0533740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f2ed81ad710 con 0x563317c98370 2024-09-12T02:07:17.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.267+0000 7f2ed6ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563317c8eb40 0x563317c948b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:17.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.267+0000 7f2ed6ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563317c8eb40 0x563317c948b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49496/0 (socket says 172.21.15.16:49496) 2024-09-12T02:07:17.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.267+0000 7f2ed6ffd640 1 -- 172.21.15.16:0/457624651 learned_addr learned my addr 172.21.15.16:0/457624651 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:07:17.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.267+0000 7f2ed7fff640 1 --2- 172.21.15.16:0/457624651 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563317c98370 0x563317c96f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:17.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.267+0000 7f2ed77fe640 1 --2- 172.21.15.16:0/457624651 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563317c8d5d0 0x563317ca4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:17.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.267+0000 7f2ed6ffd640 1 -- 172.21.15.16:0/457624651 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563317c98370 msgr2=0x563317c96f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:17.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.267+0000 7f2ed6ffd640 1 --2- 172.21.15.16:0/457624651 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563317c98370 0x563317c96f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:17.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.267+0000 7f2ed6ffd640 1 -- 172.21.15.16:0/457624651 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563317c8d5d0 msgr2=0x563317ca4300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:17.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.267+0000 7f2ed6ffd640 1 --2- 172.21.15.16:0/457624651 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563317c8d5d0 0x563317ca4300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:17.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.267+0000 7f2ed6ffd640 1 -- 172.21.15.16:0/457624651 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563317c93fa0 con 0x563317c8eb40 2024-09-12T02:07:17.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.267+0000 7f2ed7fff640 1 --2- 172.21.15.16:0/457624651 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563317c98370 0x563317c96f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:07:17.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.267+0000 7f2ed77fe640 1 --2- 172.21.15.16:0/457624651 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563317c8d5d0 0x563317ca4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:07:17.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.268+0000 7f2ed6ffd640 1 --2- 172.21.15.16:0/457624651 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563317c8eb40 0x563317c948b0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc09b1c0 tx=0x7f2ecc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f224e5e060501ee9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:07:17.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.268+0000 7f2ebd7fa640 1 -- 172.21.15.16:0/457624651 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ecc0a3ea0 con 0x563317c8eb40 2024-09-12T02:07:17.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.268+0000 7f2ebd7fa640 1 -- 172.21.15.16:0/457624651 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ecc0978a0 con 0x563317c8eb40 2024-09-12T02:07:17.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.268+0000 7f2ebd7fa640 1 -- 172.21.15.16:0/457624651 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ecc09ab90 con 0x563317c8eb40 2024-09-12T02:07:17.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.268+0000 7f2ee0533740 1 -- 172.21.15.16:0/457624651 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563317c8eb40 msgr2=0x563317c948b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:17.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.268+0000 7f2ee0533740 1 --2- 172.21.15.16:0/457624651 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563317c8eb40 0x563317c948b0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc09b1c0 tx=0x7f2ecc0046c0 comp rx=0 tx=0).stop 2024-09-12T02:07:17.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.268+0000 7f2ee0533740 1 -- 172.21.15.16:0/457624651 shutdown_connections 2024-09-12T02:07:17.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.268+0000 7f2ee0533740 1 --2- 172.21.15.16:0/457624651 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563317c98370 0x563317c96f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:17.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.268+0000 7f2ee0533740 1 --2- 172.21.15.16:0/457624651 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563317c8eb40 0x563317c948b0 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:17.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.268+0000 7f2ee0533740 1 --2- 172.21.15.16:0/457624651 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563317c8d5d0 0x563317ca4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:17.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.268+0000 7f2ee0533740 1 -- 172.21.15.16:0/457624651 >> 172.21.15.16:0/457624651 conn(0x563317c56310 msgr2=0x563317c96850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:07:17.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.268+0000 7f2ee0533740 1 -- 172.21.15.16:0/457624651 shutdown_connections 2024-09-12T02:07:17.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.268+0000 7f2ee0533740 1 -- 172.21.15.16:0/457624651 wait complete. 2024-09-12T02:07:17.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.269+0000 7f2ee0533740 1 -- start start 2024-09-12T02:07:17.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.269+0000 7f2ee0533740 1 Processor -- start 2024-09-12T02:07:17.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.269+0000 7f2ee0533740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563317c8d5d0 0x563317c9c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:17.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.269+0000 7f2ee0533740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563317c8eb40 0x563317c9c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:17.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.269+0000 7f2ee0533740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563317c98370 0x563317c9cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:17.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.269+0000 7f2ee0533740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2ed81add80 con 0x563317c8eb40 2024-09-12T02:07:17.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.269+0000 7f2ee0533740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2ed81af1f0 con 0x563317c8d5d0 2024-09-12T02:07:17.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.269+0000 7f2ee0533740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f2ed81ae450 con 0x563317c98370 2024-09-12T02:07:17.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.269+0000 7f2ee0533740 10 client.0 _ll_register_callbacks cb 0x7f2ecc09b000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:07:17.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.269+0000 7f2ed6ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563317c8eb40 0x563317c9c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:17.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.269+0000 7f2ed6ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563317c8eb40 0x563317c9c7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49500/0 (socket says 172.21.15.16:49500) 2024-09-12T02:07:17.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.269+0000 7f2ed6ffd640 1 -- 172.21.15.16:0/183004322 learned_addr learned my addr 172.21.15.16:0/183004322 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:07:17.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.270+0000 7f2ed77fe640 1 --2- 172.21.15.16:0/183004322 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563317c8d5d0 0x563317c9c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:17.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.270+0000 7f2ed7fff640 1 --2- 172.21.15.16:0/183004322 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563317c98370 0x563317c9cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:17.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.270+0000 7f2ed6ffd640 1 -- 172.21.15.16:0/183004322 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563317c98370 msgr2=0x563317c9cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:17.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.270+0000 7f2ed6ffd640 1 --2- 172.21.15.16:0/183004322 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563317c98370 0x563317c9cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:17.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.270+0000 7f2ed6ffd640 1 -- 172.21.15.16:0/183004322 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563317c8d5d0 msgr2=0x563317c9c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:17.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.270+0000 7f2ed6ffd640 1 --2- 172.21.15.16:0/183004322 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563317c8d5d0 0x563317c9c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:17.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.270+0000 7f2ed6ffd640 1 -- 172.21.15.16:0/183004322 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563317d11c10 con 0x563317c8eb40 2024-09-12T02:07:17.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.270+0000 7f2ed6ffd640 1 --2- 172.21.15.16:0/183004322 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563317c8eb40 0x563317c9c7a0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc0a3970 tx=0x7f2ecc0be690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c5d5353349e91db server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:07:17.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.270+0000 7f2eb6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:07:17.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.270+0000 7f2eb6ffd640 1 -- 172.21.15.16:0/183004322 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ecc09ab90 con 0x563317c8eb40 2024-09-12T02:07:17.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.270+0000 7f2eb6ffd640 1 -- 172.21.15.16:0/183004322 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f2ecc09f320 con 0x563317c8eb40 2024-09-12T02:07:17.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.270+0000 7f2ee0533740 1 -- 172.21.15.16:0/183004322 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ecc0978a0 con 0x563317c8eb40 2024-09-12T02:07:17.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.270+0000 7f2eb6ffd640 1 -- 172.21.15.16:0/183004322 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2ecc0be0d0 con 0x563317c8eb40 2024-09-12T02:07:17.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.270+0000 7f2ee0533740 10 client.17037 fetch_fsmap learned FSMap version 25 2024-09-12T02:07:17.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.270+0000 7f2ee0533740 1 -- 172.21.15.16:0/183004322 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x563317d12ed0 con 0x563317c8eb40 2024-09-12T02:07:17.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.270+0000 7f2eb6ffd640 1 -- 172.21.15.16:0/183004322 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ecc0978a0 con 0x563317c8eb40 2024-09-12T02:07:17.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.271+0000 7f2eb6ffd640 1 -- 172.21.15.16:0/183004322 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f2ecc09a210 con 0x563317c8eb40 2024-09-12T02:07:17.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.271+0000 7f2ee0533740 10 client.17037 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:07:17.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.271+0000 7f2ee0533740 10 client.17037 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T02:07:17.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.271+0000 7f2ee0533740 20 client.17037 populate_metadata read hostname 'smithi016' 2024-09-12T02:07:17.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.271+0000 7f2ee0533740 1 --2- 172.21.15.16:0/183004322 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x563317d13300 0x563317d156f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:17.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.271+0000 7f2ee0533740 4 client.17037 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T02:07:17.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.271+0000 7f2ee0533740 1 -- 172.21.15.16:0/183004322 --> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563317bfad70 con 0x563317d13300 2024-09-12T02:07:17.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.271+0000 7f2ed77fe640 1 --2- 172.21.15.16:0/183004322 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x563317d13300 0x563317d156f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:17.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.271+0000 7f2eb6ffd640 10 client.17037 ms_handle_connect on v2:172.21.15.184:6838/4203347941 2024-09-12T02:07:17.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.271+0000 7f2ed77fe640 1 --2- 172.21.15.16:0/183004322 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x563317d13300 0x563317d156f0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=84d0a091267af0b1 server_cookie=8b0c4a957bba69e3 in_seq=0 out_seq=0 2024-09-12T02:07:17.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.271+0000 7f2eb6ffd640 1 -- 172.21.15.16:0/183004322 <== mds.1 v2:172.21.15.184:6838/4203347941 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563317bfad70 con 0x563317d13300 2024-09-12T02:07:17.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.272+0000 7f2eb6ffd640 10 client.17037 handle_command_reply: tid=0 2024-09-12T02:07:17.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.272+0000 7f2ee0533740 1 client.17037 shutdown 2024-09-12T02:07:17.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.272+0000 7f2ee0533740 1 -- 172.21.15.16:0/183004322 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563317c8eb40 msgr2=0x563317c9c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:17.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.272+0000 7f2ee0533740 1 --2- 172.21.15.16:0/183004322 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563317c8eb40 0x563317c9c7a0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc0a3970 tx=0x7f2ecc0be690 comp rx=0 tx=0).stop 2024-09-12T02:07:17.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.272+0000 7f2ee0533740 1 -- 172.21.15.16:0/183004322 shutdown_connections 2024-09-12T02:07:17.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.272+0000 7f2ee0533740 1 --2- 172.21.15.16:0/183004322 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x563317d13300 0x563317d156f0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:17.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.272+0000 7f2ee0533740 1 --2- 172.21.15.16:0/183004322 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563317c98370 0x563317c9cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:17.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.272+0000 7f2ee0533740 1 --2- 172.21.15.16:0/183004322 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563317c8eb40 0x563317c9c7a0 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:17.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.272+0000 7f2eb6ffd640 0 client.17037 ms_handle_reset on v2:172.21.15.184:6838/4203347941 2024-09-12T02:07:17.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.272+0000 7f2ee0533740 1 --2- 172.21.15.16:0/183004322 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563317c8d5d0 0x563317c9c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:17.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.272+0000 7f2ee0533740 1 -- 172.21.15.16:0/183004322 >> 172.21.15.16:0/183004322 conn(0x563317c56310 msgr2=0x563317c96850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:07:17.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.272+0000 7f2ee0533740 1 -- 172.21.15.16:0/183004322 shutdown_connections 2024-09-12T02:07:17.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.272+0000 7f2ee0533740 1 -- 172.21.15.16:0/183004322 wait complete. 2024-09-12T02:07:17.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.272+0000 7f2ee0533740 20 client.17037 trim_cache size 0 max 16384 2024-09-12T02:07:17.287 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T02:07:17.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.272+0000 7f2ede15a640 1 -- 172.21.15.16:0/3753435066 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f2ec00a2b80 msgr2=0x7f2ec00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:17.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.272+0000 7f2ede15a640 1 --2- 172.21.15.16:0/3753435066 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f2ec00a2b80 0x7f2ec00a5040 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f2ed0005c40 tx=0x7f2ed0009d70 comp rx=0 tx=0).stop 2024-09-12T02:07:17.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.272+0000 7f2ede15a640 1 -- 172.21.15.16:0/3753435066 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ed8103660 msgr2=0x7f2ed8100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:17.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.272+0000 7f2ede15a640 1 --2- 172.21.15.16:0/3753435066 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ed8103660 0x7f2ed8100ee0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc00d950 tx=0x7f2ecc00de10 comp rx=0 tx=0).stop 2024-09-12T02:07:17.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.273+0000 7f2ede15a640 1 -- 172.21.15.16:0/3753435066 shutdown_connections 2024-09-12T02:07:17.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.273+0000 7f2ede15a640 1 --2- 172.21.15.16:0/3753435066 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f2ec00a2b80 0x7f2ec00a5040 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:17.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.273+0000 7f2ede15a640 1 --2- 172.21.15.16:0/3753435066 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2ed810c170 0x7f2ed8101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:17.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.273+0000 7f2ede15a640 1 --2- 172.21.15.16:0/3753435066 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ed8103660 0x7f2ed8100ee0 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:17.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.273+0000 7f2ede15a640 1 --2- 172.21.15.16:0/3753435066 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ed8102d40 0x7f2ed81009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:17.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.273+0000 7f2ede15a640 1 -- 172.21.15.16:0/3753435066 >> 172.21.15.16:0/3753435066 conn(0x7f2ed80fc7a0 msgr2=0x7f2ed80fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:07:17.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.273+0000 7f2ede15a640 1 -- 172.21.15.16:0/3753435066 shutdown_connections 2024-09-12T02:07:17.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:17.273+0000 7f2ede15a640 1 -- 172.21.15.16:0/3753435066 wait complete. 2024-09-12T02:07:18.054 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-12T02:07:18.319 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:07:18.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:07:18 smithi016 ceph-mon[30947]: pgmap v496: 97 pgs: 97 active+clean; 2.9 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-09-12T02:07:18.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:07:18 smithi184 ceph-mon[35593]: pgmap v496: 97 pgs: 97 active+clean; 2.9 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-09-12T02:07:18.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:07:18 smithi033 ceph-mon[35332]: pgmap v496: 97 pgs: 97 active+clean; 2.9 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-09-12T02:07:18.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.783+0000 7f86904b8640 1 -- 172.21.15.16:0/1387333939 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8688106a80 msgr2=0x7f8688106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:18.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.783+0000 7f86904b8640 1 --2- 172.21.15.16:0/1387333939 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8688106a80 0x7f8688106e60 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f8684009930 tx=0x7f868402a420 comp rx=0 tx=0).stop 2024-09-12T02:07:18.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.783+0000 7f86904b8640 1 -- 172.21.15.16:0/1387333939 shutdown_connections 2024-09-12T02:07:18.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.783+0000 7f86904b8640 1 --2- 172.21.15.16:0/1387333939 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8688101420 0x7f868810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:18.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.783+0000 7f86904b8640 1 --2- 172.21.15.16:0/1387333939 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8688100a80 0x7f8688100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:18.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.783+0000 7f86904b8640 1 --2- 172.21.15.16:0/1387333939 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8688106a80 0x7f8688106e60 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:18.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.783+0000 7f86904b8640 1 -- 172.21.15.16:0/1387333939 >> 172.21.15.16:0/1387333939 conn(0x7f86880fc7a0 msgr2=0x7f86880febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:07:18.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.783+0000 7f86904b8640 1 -- 172.21.15.16:0/1387333939 shutdown_connections 2024-09-12T02:07:18.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.783+0000 7f86904b8640 1 -- 172.21.15.16:0/1387333939 wait complete. 2024-09-12T02:07:18.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.784+0000 7f86904b8640 1 Processor -- start 2024-09-12T02:07:18.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.784+0000 7f86904b8640 1 -- start start 2024-09-12T02:07:18.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.784+0000 7f86904b8640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8688100a80 0x7f868819a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:18.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.784+0000 7f86904b8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8688101420 0x7f868819aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:18.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.784+0000 7f86904b8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8688106a80 0x7f868819ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:18.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.784+0000 7f86904b8640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8688111f60 con 0x7f8688101420 2024-09-12T02:07:18.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.784+0000 7f86904b8640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8688111de0 con 0x7f8688106a80 2024-09-12T02:07:18.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.784+0000 7f86904b8640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f86881120e0 con 0x7f8688100a80 2024-09-12T02:07:18.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.784+0000 7f868da53640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8688101420 0x7f868819aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:18.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.784+0000 7f868da53640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8688101420 0x7f868819aae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49528/0 (socket says 172.21.15.16:49528) 2024-09-12T02:07:18.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.784+0000 7f868da53640 1 -- 172.21.15.16:0/2271316863 learned_addr learned my addr 172.21.15.16:0/2271316863 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:07:18.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.784+0000 7f868e254640 1 --2- 172.21.15.16:0/2271316863 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8688100a80 0x7f868819a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:18.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.784+0000 7f868ea55640 1 --2- 172.21.15.16:0/2271316863 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8688106a80 0x7f868819ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:18.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.784+0000 7f868da53640 1 -- 172.21.15.16:0/2271316863 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8688100a80 msgr2=0x7f868819a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:18.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.784+0000 7f868da53640 1 --2- 172.21.15.16:0/2271316863 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8688100a80 0x7f868819a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:18.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.784+0000 7f868da53640 1 -- 172.21.15.16:0/2271316863 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8688106a80 msgr2=0x7f868819ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:18.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.784+0000 7f868da53640 1 --2- 172.21.15.16:0/2271316863 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8688106a80 0x7f868819ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:18.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.784+0000 7f868da53640 1 -- 172.21.15.16:0/2271316863 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f868819f550 con 0x7f8688101420 2024-09-12T02:07:18.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.785+0000 7f868da53640 1 --2- 172.21.15.16:0/2271316863 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8688101420 0x7f868819aae0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f867800c920 tx=0x7f867800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:07:18.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.785+0000 7f867f7fe640 1 -- 172.21.15.16:0/2271316863 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8678007c70 con 0x7f8688101420 2024-09-12T02:07:18.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.785+0000 7f867f7fe640 1 -- 172.21.15.16:0/2271316863 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8678007e10 con 0x7f8688101420 2024-09-12T02:07:18.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.785+0000 7f86904b8640 1 -- 172.21.15.16:0/2271316863 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f868819f840 con 0x7f8688101420 2024-09-12T02:07:18.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.785+0000 7f867f7fe640 1 -- 172.21.15.16:0/2271316863 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8678018890 con 0x7f8688101420 2024-09-12T02:07:18.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.785+0000 7f86904b8640 1 -- 172.21.15.16:0/2271316863 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86881a7120 con 0x7f8688101420 2024-09-12T02:07:18.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.787+0000 7f867f7fe640 1 -- 172.21.15.16:0/2271316863 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f8678018a70 con 0x7f8688101420 2024-09-12T02:07:18.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.787+0000 7f8692891740 1 Processor -- start 2024-09-12T02:07:18.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.788+0000 7f8692891740 1 -- start start 2024-09-12T02:07:18.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.788+0000 7f8692891740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55891e2c8b40 0x55891e2c7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:18.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.788+0000 7f8692891740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55891e2c75d0 0x55891e2c7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:18.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.788+0000 7f8692891740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55891e2d2370 0x55891e2d4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:18.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.788+0000 7f8692891740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55891e22d2f0 con 0x55891e2c8b40 2024-09-12T02:07:18.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.788+0000 7f8692891740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55891e1d4cf0 con 0x55891e2d2370 2024-09-12T02:07:18.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.788+0000 7f8692891740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55891e2292b0 con 0x55891e2c75d0 2024-09-12T02:07:18.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.788+0000 7f868ea55640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55891e2c8b40 0x55891e2c7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:18.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.788+0000 7f868ea55640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55891e2c8b40 0x55891e2c7090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49532/0 (socket says 172.21.15.16:49532) 2024-09-12T02:07:18.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.788+0000 7f868ea55640 1 -- 172.21.15.16:0/3417572752 learned_addr learned my addr 172.21.15.16:0/3417572752 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:07:18.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.788+0000 7f867f7fe640 1 --2- 172.21.15.16:0/2271316863 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f86740a2b80 0x7f86740a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:18.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.788+0000 7f868e254640 1 --2- 172.21.15.16:0/3417572752 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55891e2c75d0 0x55891e2c7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:18.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.788+0000 7f868da53640 1 --2- 172.21.15.16:0/3417572752 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55891e2d2370 0x55891e2d4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:18.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.789+0000 7f868ea55640 1 -- 172.21.15.16:0/3417572752 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55891e2c75d0 msgr2=0x55891e2c7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:18.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.789+0000 7f868ea55640 1 --2- 172.21.15.16:0/3417572752 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55891e2c75d0 0x55891e2c7a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:18.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.789+0000 7f868ea55640 1 -- 172.21.15.16:0/3417572752 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55891e2d2370 msgr2=0x55891e2d4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:18.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.789+0000 7f868ea55640 1 --2- 172.21.15.16:0/3417572752 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55891e2d2370 0x55891e2d4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:18.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.789+0000 7f868ea55640 1 -- 172.21.15.16:0/3417572752 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55891e1b93a0 con 0x55891e2c8b40 2024-09-12T02:07:18.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.789+0000 7f868e254640 1 --2- 172.21.15.16:0/3417572752 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55891e2c75d0 0x55891e2c7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:07:18.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.789+0000 7f867f7fe640 1 -- 172.21.15.16:0/2271316863 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f86780cf190 con 0x7f8688101420 2024-09-12T02:07:18.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.789+0000 7f868ea55640 1 --2- 172.21.15.16:0/2271316863 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f86740a2b80 0x7f86740a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:18.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.789+0000 7f868ea55640 1 --2- 172.21.15.16:0/3417572752 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55891e2c8b40 0x55891e2c7090 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f8680009fc0 tx=0x7f8680010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54380db8073a7c2c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:07:18.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.789+0000 7f867cff9640 1 -- 172.21.15.16:0/3417572752 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f868001c900 con 0x55891e2c8b40 2024-09-12T02:07:18.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.789+0000 7f867cff9640 1 -- 172.21.15.16:0/3417572752 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86800113e0 con 0x55891e2c8b40 2024-09-12T02:07:18.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.789+0000 7f867cff9640 1 -- 172.21.15.16:0/3417572752 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8680013620 con 0x55891e2c8b40 2024-09-12T02:07:18.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.789+0000 7f868ea55640 1 --2- 172.21.15.16:0/2271316863 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f86740a2b80 0x7f86740a5040 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f8680020000 tx=0x7f868001d000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:07:18.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.790+0000 7f8692891740 1 -- 172.21.15.16:0/3417572752 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55891e2c8b40 msgr2=0x55891e2c7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:18.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.790+0000 7f8692891740 1 --2- 172.21.15.16:0/3417572752 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55891e2c8b40 0x55891e2c7090 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f8680009fc0 tx=0x7f8680010800 comp rx=0 tx=0).stop 2024-09-12T02:07:18.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.790+0000 7f8692891740 1 -- 172.21.15.16:0/3417572752 shutdown_connections 2024-09-12T02:07:18.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.790+0000 7f8692891740 1 --2- 172.21.15.16:0/3417572752 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55891e2d2370 0x55891e2d4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:18.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.790+0000 7f8692891740 1 --2- 172.21.15.16:0/3417572752 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55891e2c75d0 0x55891e2c7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:18.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.790+0000 7f8692891740 1 --2- 172.21.15.16:0/3417572752 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55891e2c8b40 0x55891e2c7090 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:18.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.790+0000 7f8692891740 1 -- 172.21.15.16:0/3417572752 >> 172.21.15.16:0/3417572752 conn(0x55891e290310 msgr2=0x55891e2c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:07:18.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.790+0000 7f8692891740 1 -- 172.21.15.16:0/3417572752 shutdown_connections 2024-09-12T02:07:18.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.790+0000 7f8692891740 1 -- 172.21.15.16:0/3417572752 wait complete. 2024-09-12T02:07:18.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.790+0000 7f8692891740 1 -- start start 2024-09-12T02:07:18.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.790+0000 7f8692891740 1 Processor -- start 2024-09-12T02:07:18.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.791+0000 7f8692891740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55891e2c75d0 0x55891e2d1d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:18.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.791+0000 7f8692891740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55891e2c8b40 0x55891e2cefe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:18.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.791+0000 7f8692891740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55891e2d2370 0x55891e2cf5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:18.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.791+0000 7f8692891740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55891e2d5070 con 0x55891e2d2370 2024-09-12T02:07:18.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.791+0000 7f8692891740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55891e1e8060 con 0x55891e2c8b40 2024-09-12T02:07:18.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.791+0000 7f8692891740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55891e2d51f0 con 0x55891e2c75d0 2024-09-12T02:07:18.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.791+0000 7f8692891740 10 client.0 _ll_register_callbacks cb 0x55891df3b8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:07:18.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.791+0000 7f868ea55640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55891e2d2370 0x55891e2cf5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:18.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.791+0000 7f868ea55640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55891e2d2370 0x55891e2cf5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49546/0 (socket says 172.21.15.16:49546) 2024-09-12T02:07:18.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.791+0000 7f868ea55640 1 -- 172.21.15.16:0/2385186758 learned_addr learned my addr 172.21.15.16:0/2385186758 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:07:18.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.791+0000 7f868e254640 1 --2- 172.21.15.16:0/2385186758 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55891e2c75d0 0x55891e2d1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:18.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.791+0000 7f868da53640 1 --2- 172.21.15.16:0/2385186758 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55891e2c8b40 0x55891e2cefe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:18.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.791+0000 7f868ea55640 1 -- 172.21.15.16:0/2385186758 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55891e2c75d0 msgr2=0x55891e2d1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:18.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.791+0000 7f868ea55640 1 --2- 172.21.15.16:0/2385186758 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55891e2c75d0 0x55891e2d1d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:18.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.791+0000 7f868ea55640 1 -- 172.21.15.16:0/2385186758 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55891e2c8b40 msgr2=0x55891e2cefe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:18.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.791+0000 7f868ea55640 1 --2- 172.21.15.16:0/2385186758 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55891e2c8b40 0x55891e2cefe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:18.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.791+0000 7f868ea55640 1 -- 172.21.15.16:0/2385186758 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55891df9e3c0 con 0x55891e2d2370 2024-09-12T02:07:18.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.791+0000 7f868ea55640 1 --2- 172.21.15.16:0/2385186758 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55891e2d2370 0x55891e2cf5d0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f8680011fd0 tx=0x7f868001b600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=447e2e405e287f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:07:18.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.791+0000 7f8671ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:07:18.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.791+0000 7f8671ffb640 1 -- 172.21.15.16:0/2385186758 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8680033060 con 0x55891e2d2370 2024-09-12T02:07:18.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.791+0000 7f8671ffb640 1 -- 172.21.15.16:0/2385186758 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f86800117a0 con 0x55891e2d2370 2024-09-12T02:07:18.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.791+0000 7f8692891740 1 -- 172.21.15.16:0/2385186758 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86800113e0 con 0x55891e2d2370 2024-09-12T02:07:18.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.792+0000 7f8671ffb640 1 -- 172.21.15.16:0/2385186758 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f86800119a0 con 0x55891e2d2370 2024-09-12T02:07:18.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.792+0000 7f8692891740 10 client.17049 fetch_fsmap learned FSMap version 25 2024-09-12T02:07:18.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.792+0000 7f8692891740 1 -- 172.21.15.16:0/2385186758 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55891e2e0ad0 con 0x55891e2d2370 2024-09-12T02:07:18.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.792+0000 7f8671ffb640 1 -- 172.21.15.16:0/2385186758 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86800113e0 con 0x55891e2d2370 2024-09-12T02:07:18.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.792+0000 7f8671ffb640 1 -- 172.21.15.16:0/2385186758 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f8680038070 con 0x55891e2d2370 2024-09-12T02:07:18.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.792+0000 7f8692891740 10 client.17049 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:07:18.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.792+0000 7f8692891740 10 client.17049 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:07:18.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.792+0000 7f8692891740 20 client.17049 populate_metadata read hostname 'smithi016' 2024-09-12T02:07:18.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.792+0000 7f8692891740 1 --2- 172.21.15.16:0/2385186758 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55891e2e1f30 0x55891e2e4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:18.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.792+0000 7f8692891740 4 client.17049 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:07:18.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.792+0000 7f8692891740 1 -- 172.21.15.16:0/2385186758 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55891e23a860 con 0x55891e2e1f30 2024-09-12T02:07:18.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.792+0000 7f868e254640 1 --2- 172.21.15.16:0/2385186758 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55891e2e1f30 0x55891e2e4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:18.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.793+0000 7f8671ffb640 10 client.17049 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:07:18.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.793+0000 7f868e254640 1 --2- 172.21.15.16:0/2385186758 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55891e2e1f30 0x55891e2e4320 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8b08ad47a164daf server_cookie=b066e6ac25cb73da in_seq=0 out_seq=0 2024-09-12T02:07:18.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.793+0000 7f8671ffb640 1 -- 172.21.15.16:0/2385186758 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55891e23a860 con 0x55891e2e1f30 2024-09-12T02:07:18.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.793+0000 7f8671ffb640 10 client.17049 handle_command_reply: tid=0 2024-09-12T02:07:18.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.794+0000 7f8692891740 1 client.17049 shutdown 2024-09-12T02:07:18.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.794+0000 7f8692891740 1 -- 172.21.15.16:0/2385186758 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55891e2d2370 msgr2=0x55891e2cf5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:18.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.794+0000 7f8692891740 1 --2- 172.21.15.16:0/2385186758 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55891e2d2370 0x55891e2cf5d0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f8680011fd0 tx=0x7f868001b600 comp rx=0 tx=0).stop 2024-09-12T02:07:18.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.794+0000 7f8692891740 1 -- 172.21.15.16:0/2385186758 shutdown_connections 2024-09-12T02:07:18.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.794+0000 7f8692891740 1 --2- 172.21.15.16:0/2385186758 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55891e2e1f30 0x55891e2e4320 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:18.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.794+0000 7f8692891740 1 --2- 172.21.15.16:0/2385186758 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55891e2d2370 0x55891e2cf5d0 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:18.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.794+0000 7f8692891740 1 --2- 172.21.15.16:0/2385186758 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55891e2c8b40 0x55891e2cefe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:18.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.794+0000 7f8692891740 1 --2- 172.21.15.16:0/2385186758 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55891e2c75d0 0x55891e2d1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:18.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.794+0000 7f8671ffb640 0 client.17049 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:07:18.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.794+0000 7f8692891740 1 -- 172.21.15.16:0/2385186758 >> 172.21.15.16:0/2385186758 conn(0x55891e290310 msgr2=0x55891e2c9960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:07:18.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.794+0000 7f8692891740 1 -- 172.21.15.16:0/2385186758 shutdown_connections 2024-09-12T02:07:18.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.794+0000 7f8692891740 1 -- 172.21.15.16:0/2385186758 wait complete. 2024-09-12T02:07:18.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.794+0000 7f8692891740 20 client.17049 trim_cache size 0 max 16384 2024-09-12T02:07:18.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.807+0000 7f8692891740 1 Processor -- start 2024-09-12T02:07:18.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.807+0000 7f8692891740 1 -- start start 2024-09-12T02:07:18.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.807+0000 7f8692891740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55891e2c75d0 0x55891e2de310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:18.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.807+0000 7f8692891740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55891e2c8b40 0x55891e2d0f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:18.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.807+0000 7f8692891740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55891e2d2370 0x55891e2ca120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:18.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.807+0000 7f8692891740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f86881b1ec0 con 0x55891e2c8b40 2024-09-12T02:07:18.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.807+0000 7f8692891740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f86881af790 con 0x55891e2c75d0 2024-09-12T02:07:18.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.807+0000 7f8692891740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f86881ad5e0 con 0x55891e2d2370 2024-09-12T02:07:18.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.807+0000 7f868da53640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55891e2c8b40 0x55891e2d0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:18.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.807+0000 7f868da53640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55891e2c8b40 0x55891e2d0f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49550/0 (socket says 172.21.15.16:49550) 2024-09-12T02:07:18.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.807+0000 7f868da53640 1 -- 172.21.15.16:0/992691940 learned_addr learned my addr 172.21.15.16:0/992691940 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:07:18.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.807+0000 7f868ea55640 1 --2- 172.21.15.16:0/992691940 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55891e2d2370 0x55891e2ca120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:18.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.807+0000 7f868e254640 1 --2- 172.21.15.16:0/992691940 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55891e2c75d0 0x55891e2de310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:18.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.808+0000 7f868da53640 1 -- 172.21.15.16:0/992691940 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55891e2d2370 msgr2=0x55891e2ca120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:18.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.808+0000 7f868da53640 1 --2- 172.21.15.16:0/992691940 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55891e2d2370 0x55891e2ca120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:18.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.808+0000 7f868da53640 1 -- 172.21.15.16:0/992691940 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55891e2c75d0 msgr2=0x55891e2de310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:18.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.808+0000 7f868da53640 1 --2- 172.21.15.16:0/992691940 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55891e2c75d0 0x55891e2de310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:18.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.808+0000 7f868da53640 1 -- 172.21.15.16:0/992691940 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55891e2c8780 con 0x55891e2c8b40 2024-09-12T02:07:18.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.808+0000 7f868e254640 1 --2- 172.21.15.16:0/992691940 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55891e2c75d0 0x55891e2de310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:07:18.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.808+0000 7f868ea55640 1 --2- 172.21.15.16:0/992691940 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55891e2d2370 0x55891e2ca120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:07:18.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.808+0000 7f868da53640 1 --2- 172.21.15.16:0/992691940 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55891e2c8b40 0x55891e2d0f50 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f8678007590 tx=0x7f867809b9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8531fc61149aeb35 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:07:18.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.808+0000 7f8673fff640 1 -- 172.21.15.16:0/992691940 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86780a4660 con 0x55891e2c8b40 2024-09-12T02:07:18.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.808+0000 7f8673fff640 1 -- 172.21.15.16:0/992691940 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86780ccda0 con 0x55891e2c8b40 2024-09-12T02:07:18.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.808+0000 7f8673fff640 1 -- 172.21.15.16:0/992691940 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86780a41d0 con 0x55891e2c8b40 2024-09-12T02:07:18.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.809+0000 7f8692891740 1 -- 172.21.15.16:0/992691940 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55891e2c8b40 msgr2=0x55891e2d0f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:18.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.809+0000 7f8692891740 1 --2- 172.21.15.16:0/992691940 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55891e2c8b40 0x55891e2d0f50 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f8678007590 tx=0x7f867809b9f0 comp rx=0 tx=0).stop 2024-09-12T02:07:18.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.809+0000 7f8692891740 1 -- 172.21.15.16:0/992691940 shutdown_connections 2024-09-12T02:07:18.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.809+0000 7f8692891740 1 --2- 172.21.15.16:0/992691940 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55891e2d2370 0x55891e2ca120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:18.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.809+0000 7f8692891740 1 --2- 172.21.15.16:0/992691940 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55891e2c8b40 0x55891e2d0f50 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:18.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.809+0000 7f8692891740 1 --2- 172.21.15.16:0/992691940 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55891e2c75d0 0x55891e2de310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:18.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.809+0000 7f8692891740 1 -- 172.21.15.16:0/992691940 >> 172.21.15.16:0/992691940 conn(0x55891e290310 msgr2=0x55891e2d0840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:07:18.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.809+0000 7f8692891740 1 -- 172.21.15.16:0/992691940 shutdown_connections 2024-09-12T02:07:18.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.809+0000 7f8692891740 1 -- 172.21.15.16:0/992691940 wait complete. 2024-09-12T02:07:18.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.809+0000 7f8692891740 1 -- start start 2024-09-12T02:07:18.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.809+0000 7f8692891740 1 Processor -- start 2024-09-12T02:07:18.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.809+0000 7f8692891740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55891e2c75d0 0x55891e2d5a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:18.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.809+0000 7f8692891740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55891e2c8b40 0x55891e2d5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:18.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.809+0000 7f8692891740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55891e2d2370 0x55891e34aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:18.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.809+0000 7f8692891740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f86881adc50 con 0x55891e2c8b40 2024-09-12T02:07:18.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.809+0000 7f8692891740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f86881af0c0 con 0x55891e2c75d0 2024-09-12T02:07:18.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.809+0000 7f8692891740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f86881ae320 con 0x55891e2d2370 2024-09-12T02:07:18.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.809+0000 7f8692891740 10 client.0 _ll_register_callbacks cb 0x7f868001f020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:07:18.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.809+0000 7f868da53640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55891e2c8b40 0x55891e2d5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:18.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.810+0000 7f868da53640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55891e2c8b40 0x55891e2d5f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49564/0 (socket says 172.21.15.16:49564) 2024-09-12T02:07:18.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.810+0000 7f868da53640 1 -- 172.21.15.16:0/3608248654 learned_addr learned my addr 172.21.15.16:0/3608248654 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:07:18.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.810+0000 7f868ea55640 1 --2- 172.21.15.16:0/3608248654 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55891e2d2370 0x55891e34aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:18.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.810+0000 7f868e254640 1 --2- 172.21.15.16:0/3608248654 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55891e2c75d0 0x55891e2d5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:18.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.810+0000 7f868da53640 1 -- 172.21.15.16:0/3608248654 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55891e2d2370 msgr2=0x55891e34aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:18.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.810+0000 7f868da53640 1 --2- 172.21.15.16:0/3608248654 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55891e2d2370 0x55891e34aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:18.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.810+0000 7f868da53640 1 -- 172.21.15.16:0/3608248654 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55891e2c75d0 msgr2=0x55891e2d5a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:18.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.810+0000 7f868da53640 1 --2- 172.21.15.16:0/3608248654 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55891e2c75d0 0x55891e2d5a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:18.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.810+0000 7f868da53640 1 -- 172.21.15.16:0/3608248654 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55891e34af40 con 0x55891e2c8b40 2024-09-12T02:07:18.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.810+0000 7f868e254640 1 --2- 172.21.15.16:0/3608248654 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55891e2c75d0 0x55891e2d5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:07:18.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.810+0000 7f868da53640 1 --2- 172.21.15.16:0/3608248654 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55891e2c8b40 0x55891e2d5f40 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f867809c050 tx=0x7f8678097d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e57fd4b440cf5593 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:07:18.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.810+0000 7f8671ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:07:18.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.810+0000 7f8671ffb640 1 -- 172.21.15.16:0/3608248654 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86780962d0 con 0x55891e2c8b40 2024-09-12T02:07:18.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.810+0000 7f8671ffb640 1 -- 172.21.15.16:0/3608248654 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f86780a41d0 con 0x55891e2c8b40 2024-09-12T02:07:18.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.810+0000 7f8692891740 1 -- 172.21.15.16:0/3608248654 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86780ccda0 con 0x55891e2c8b40 2024-09-12T02:07:18.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.810+0000 7f8671ffb640 1 -- 172.21.15.16:0/3608248654 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f8678099db0 con 0x55891e2c8b40 2024-09-12T02:07:18.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.810+0000 7f8692891740 10 client.17055 fetch_fsmap learned FSMap version 25 2024-09-12T02:07:18.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.810+0000 7f8692891740 1 -- 172.21.15.16:0/3608248654 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55891e34c150 con 0x55891e2c8b40 2024-09-12T02:07:18.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.811+0000 7f8671ffb640 1 -- 172.21.15.16:0/3608248654 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86780ccda0 con 0x55891e2c8b40 2024-09-12T02:07:18.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.811+0000 7f8671ffb640 1 -- 172.21.15.16:0/3608248654 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f86780977d0 con 0x55891e2c8b40 2024-09-12T02:07:18.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.811+0000 7f8692891740 10 client.17055 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:07:18.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.811+0000 7f8692891740 10 client.17055 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:07:18.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.811+0000 7f8692891740 20 client.17055 populate_metadata read hostname 'smithi016' 2024-09-12T02:07:18.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.811+0000 7f8692891740 1 --2- 172.21.15.16:0/3608248654 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55891e34c890 0x55891e34ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:18.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.811+0000 7f8692891740 4 client.17055 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T02:07:18.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.811+0000 7f8692891740 1 -- 172.21.15.16:0/3608248654 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55891e234d70 con 0x55891e34c890 2024-09-12T02:07:18.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.811+0000 7f868e254640 1 --2- 172.21.15.16:0/3608248654 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55891e34c890 0x55891e34ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:18.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.812+0000 7f868e254640 1 --2- 172.21.15.16:0/3608248654 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55891e34c890 0x55891e34ecb0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=25ad84d2af379a server_cookie=5752198540cc1142 in_seq=0 out_seq=0 2024-09-12T02:07:18.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.812+0000 7f8671ffb640 10 client.17055 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:07:18.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.812+0000 7f8671ffb640 1 -- 172.21.15.16:0/3608248654 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55891e234d70 con 0x55891e34c890 2024-09-12T02:07:18.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.812+0000 7f8671ffb640 10 client.17055 handle_command_reply: tid=0 2024-09-12T02:07:18.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.812+0000 7f8692891740 1 client.17055 shutdown 2024-09-12T02:07:18.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.812+0000 7f8692891740 1 -- 172.21.15.16:0/3608248654 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55891e2c8b40 msgr2=0x55891e2d5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:18.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.812+0000 7f8692891740 1 --2- 172.21.15.16:0/3608248654 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55891e2c8b40 0x55891e2d5f40 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f867809c050 tx=0x7f8678097d60 comp rx=0 tx=0).stop 2024-09-12T02:07:18.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.812+0000 7f8692891740 1 -- 172.21.15.16:0/3608248654 shutdown_connections 2024-09-12T02:07:18.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.812+0000 7f8692891740 1 --2- 172.21.15.16:0/3608248654 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55891e34c890 0x55891e34ecb0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:18.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.812+0000 7f8692891740 1 --2- 172.21.15.16:0/3608248654 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55891e2d2370 0x55891e34aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:18.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.812+0000 7f8692891740 1 --2- 172.21.15.16:0/3608248654 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55891e2c8b40 0x55891e2d5f40 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:18.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.812+0000 7f8671ffb640 0 client.17055 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:07:18.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.812+0000 7f8692891740 1 --2- 172.21.15.16:0/3608248654 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55891e2c75d0 0x55891e2d5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:18.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.812+0000 7f8692891740 1 -- 172.21.15.16:0/3608248654 >> 172.21.15.16:0/3608248654 conn(0x55891e290310 msgr2=0x55891e2d1060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:07:18.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.812+0000 7f8692891740 1 -- 172.21.15.16:0/3608248654 shutdown_connections 2024-09-12T02:07:18.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.812+0000 7f8692891740 1 -- 172.21.15.16:0/3608248654 wait complete. 2024-09-12T02:07:18.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.812+0000 7f8692891740 20 client.17055 trim_cache size 0 max 16384 2024-09-12T02:07:18.828 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T02:07:18.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.813+0000 7f86904b8640 1 -- 172.21.15.16:0/2271316863 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f86740a2b80 msgr2=0x7f86740a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:18.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.813+0000 7f86904b8640 1 --2- 172.21.15.16:0/2271316863 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f86740a2b80 0x7f86740a5040 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f8680020000 tx=0x7f868001d000 comp rx=0 tx=0).stop 2024-09-12T02:07:18.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.813+0000 7f86904b8640 1 -- 172.21.15.16:0/2271316863 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8688101420 msgr2=0x7f868819aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:18.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.813+0000 7f86904b8640 1 --2- 172.21.15.16:0/2271316863 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8688101420 0x7f868819aae0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f867800c920 tx=0x7f867800cde0 comp rx=0 tx=0).stop 2024-09-12T02:07:18.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.813+0000 7f86904b8640 1 -- 172.21.15.16:0/2271316863 shutdown_connections 2024-09-12T02:07:18.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.813+0000 7f86904b8640 1 --2- 172.21.15.16:0/2271316863 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f86740a2b80 0x7f86740a5040 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:18.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.813+0000 7f86904b8640 1 --2- 172.21.15.16:0/2271316863 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8688106a80 0x7f868819ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:18.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.813+0000 7f86904b8640 1 --2- 172.21.15.16:0/2271316863 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8688101420 0x7f868819aae0 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:18.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.813+0000 7f86904b8640 1 --2- 172.21.15.16:0/2271316863 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8688100a80 0x7f868819a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:18.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.813+0000 7f86904b8640 1 -- 172.21.15.16:0/2271316863 >> 172.21.15.16:0/2271316863 conn(0x7f86880fc7a0 msgr2=0x7f868810c0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:07:18.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.813+0000 7f86904b8640 1 -- 172.21.15.16:0/2271316863 shutdown_connections 2024-09-12T02:07:18.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:18.813+0000 7f86904b8640 1 -- 172.21.15.16:0/2271316863 wait complete. 2024-09-12T02:07:20.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:07:20 smithi016 ceph-mon[30947]: pgmap v497: 97 pgs: 97 active+clean; 2.9 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-09-12T02:07:20.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:07:20 smithi184 ceph-mon[35593]: pgmap v497: 97 pgs: 97 active+clean; 2.9 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-09-12T02:07:20.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:07:20 smithi033 ceph-mon[35332]: pgmap v497: 97 pgs: 97 active+clean; 2.9 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-09-12T02:07:20.675 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-12T02:07:20.676 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-12T02:07:20.947 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:07:21.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.377+0000 7f4d3aa2a640 1 -- 172.21.15.16:0/446578107 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d341055b0 msgr2=0x7f4d3410d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:21.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.377+0000 7f4d3aa2a640 1 --2- 172.21.15.16:0/446578107 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d341055b0 0x7f4d3410d130 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f4d2c009ca0 tx=0x7f4d2c02a8c0 comp rx=0 tx=0).stop 2024-09-12T02:07:21.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.377+0000 7f4d3aa2a640 1 -- 172.21.15.16:0/446578107 shutdown_connections 2024-09-12T02:07:21.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.377+0000 7f4d3aa2a640 1 --2- 172.21.15.16:0/446578107 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d341055b0 0x7f4d3410d130 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:21.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.377+0000 7f4d3aa2a640 1 --2- 172.21.15.16:0/446578107 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4d34100e30 0x7f4d34104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:21.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.377+0000 7f4d3aa2a640 1 --2- 172.21.15.16:0/446578107 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d34100480 0x7f4d34100860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:21.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.378+0000 7f4d3aa2a640 1 -- 172.21.15.16:0/446578107 >> 172.21.15.16:0/446578107 conn(0x7f4d34077210 msgr2=0x7f4d340fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:07:21.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.378+0000 7f4d3aa2a640 1 -- 172.21.15.16:0/446578107 shutdown_connections 2024-09-12T02:07:21.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.378+0000 7f4d3aa2a640 1 -- 172.21.15.16:0/446578107 wait complete. 2024-09-12T02:07:21.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.378+0000 7f4d3aa2a640 1 Processor -- start 2024-09-12T02:07:21.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.378+0000 7f4d3aa2a640 1 -- start start 2024-09-12T02:07:21.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.379+0000 7f4d3aa2a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4d34100480 0x7f4d341071d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:21.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.379+0000 7f4d3aa2a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d34100e30 0x7f4d34107710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:21.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.379+0000 7f4d3aa2a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d341055b0 0x7f4d34107c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:21.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.379+0000 7f4d3aa2a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4d3410fdb0 con 0x7f4d34100e30 2024-09-12T02:07:21.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.379+0000 7f4d3aa2a640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4d3410fc30 con 0x7f4d341055b0 2024-09-12T02:07:21.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.379+0000 7f4d3aa2a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f4d3410ff30 con 0x7f4d34100480 2024-09-12T02:07:21.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.379+0000 7f4d337fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d34100e30 0x7f4d34107710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:21.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.379+0000 7f4d337fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d34100e30 0x7f4d34107710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56726/0 (socket says 172.21.15.16:56726) 2024-09-12T02:07:21.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.379+0000 7f4d337fe640 1 -- 172.21.15.16:0/3440513265 learned_addr learned my addr 172.21.15.16:0/3440513265 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:07:21.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.379+0000 7f4d38fc7640 1 --2- 172.21.15.16:0/3440513265 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d341055b0 0x7f4d34107c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:21.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.379+0000 7f4d33fff640 1 --2- 172.21.15.16:0/3440513265 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4d34100480 0x7f4d341071d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:21.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.379+0000 7f4d337fe640 1 -- 172.21.15.16:0/3440513265 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4d34100480 msgr2=0x7f4d341071d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:21.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.379+0000 7f4d337fe640 1 --2- 172.21.15.16:0/3440513265 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4d34100480 0x7f4d341071d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:21.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.379+0000 7f4d337fe640 1 -- 172.21.15.16:0/3440513265 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d341055b0 msgr2=0x7f4d34107c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:21.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.379+0000 7f4d337fe640 1 --2- 172.21.15.16:0/3440513265 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d341055b0 0x7f4d34107c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:21.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.379+0000 7f4d337fe640 1 -- 172.21.15.16:0/3440513265 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d341a6a00 con 0x7f4d34100e30 2024-09-12T02:07:21.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.379+0000 7f4d337fe640 1 --2- 172.21.15.16:0/3440513265 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d34100e30 0x7f4d34107710 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f4d2800b4a0 tx=0x7f4d2800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:07:21.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.379+0000 7f4d317fa640 1 -- 172.21.15.16:0/3440513265 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d28014070 con 0x7f4d34100e30 2024-09-12T02:07:21.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.379+0000 7f4d317fa640 1 -- 172.21.15.16:0/3440513265 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d28010af0 con 0x7f4d34100e30 2024-09-12T02:07:21.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.379+0000 7f4d317fa640 1 -- 172.21.15.16:0/3440513265 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d28017d10 con 0x7f4d34100e30 2024-09-12T02:07:21.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.379+0000 7f4d3aa2a640 1 -- 172.21.15.16:0/3440513265 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d341a6cf0 con 0x7f4d34100e30 2024-09-12T02:07:21.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.380+0000 7f4d3aa2a640 1 -- 172.21.15.16:0/3440513265 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d341a71d0 con 0x7f4d34100e30 2024-09-12T02:07:21.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.382+0000 7f4d317fa640 1 -- 172.21.15.16:0/3440513265 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f4d28017070 con 0x7f4d34100e30 2024-09-12T02:07:21.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.382+0000 7f4d3ce03740 1 Processor -- start 2024-09-12T02:07:21.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.383+0000 7f4d3ce03740 1 -- start start 2024-09-12T02:07:21.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.383+0000 7f4d3ce03740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561660a7ab60 0x561660a7af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:21.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.383+0000 7f4d3ce03740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561660a7b480 0x561660a85800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:21.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.383+0000 7f4d3ce03740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561660a85d40 0x561660a88130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:21.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.383+0000 7f4d3ce03740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5616609e0280 con 0x561660a7ab60 2024-09-12T02:07:21.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.383+0000 7f4d3ce03740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x561660987b10 con 0x561660a85d40 2024-09-12T02:07:21.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.383+0000 7f4d3ce03740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5616609dc240 con 0x561660a7b480 2024-09-12T02:07:21.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.383+0000 7f4d317fa640 1 --2- 172.21.15.16:0/3440513265 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f4d1c0a2c00 0x7f4d1c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:21.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.383+0000 7f4d38fc7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561660a7ab60 0x561660a7af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:21.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.383+0000 7f4d38fc7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561660a7ab60 0x561660a7af40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56730/0 (socket says 172.21.15.16:56730) 2024-09-12T02:07:21.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.383+0000 7f4d38fc7640 1 -- 172.21.15.16:0/1381408470 learned_addr learned my addr 172.21.15.16:0/1381408470 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:07:21.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.383+0000 7f4d38fc7640 1 --2- 172.21.15.16:0/3440513265 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f4d1c0a2c00 0x7f4d1c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:21.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.383+0000 7f4d317fa640 1 -- 172.21.15.16:0/3440513265 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f4d280ce330 con 0x7f4d34100e30 2024-09-12T02:07:21.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.383+0000 7f4d33fff640 1 --2- 172.21.15.16:0/1381408470 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561660a7b480 0x561660a85800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:21.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.383+0000 7f4d337fe640 1 --2- 172.21.15.16:0/1381408470 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561660a85d40 0x561660a88130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:21.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.384+0000 7f4d337fe640 1 -- 172.21.15.16:0/1381408470 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561660a7b480 msgr2=0x561660a85800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:21.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.384+0000 7f4d337fe640 1 --2- 172.21.15.16:0/1381408470 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561660a7b480 0x561660a85800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:21.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.384+0000 7f4d337fe640 1 -- 172.21.15.16:0/1381408470 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561660a7ab60 msgr2=0x561660a7af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:21.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.384+0000 7f4d337fe640 1 --2- 172.21.15.16:0/1381408470 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561660a7ab60 0x561660a7af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:21.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.384+0000 7f4d337fe640 1 -- 172.21.15.16:0/1381408470 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5616609941f0 con 0x561660a85d40 2024-09-12T02:07:21.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.384+0000 7f4d38fc7640 1 --2- 172.21.15.16:0/1381408470 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561660a7ab60 0x561660a7af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:07:21.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.384+0000 7f4d337fe640 1 --2- 172.21.15.16:0/1381408470 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561660a85d40 0x561660a88130 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4d28097560 tx=0x7f4d280046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e92c64549e2fff3d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:07:21.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.384+0000 7f4d38fc7640 1 --2- 172.21.15.16:0/3440513265 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f4d1c0a2c00 0x7f4d1c0a50c0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f4d2c031f80 tx=0x7f4d2c02a8c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:07:21.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.384+0000 7f4d1a7fc640 1 -- 172.21.15.16:0/1381408470 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d2801e070 con 0x561660a85d40 2024-09-12T02:07:21.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.384+0000 7f4d1a7fc640 1 -- 172.21.15.16:0/1381408470 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d2809ef40 con 0x561660a85d40 2024-09-12T02:07:21.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.384+0000 7f4d1a7fc640 1 -- 172.21.15.16:0/1381408470 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d28010e00 con 0x561660a85d40 2024-09-12T02:07:21.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.385+0000 7f4d3ce03740 1 -- 172.21.15.16:0/1381408470 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561660a85d40 msgr2=0x561660a88130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:21.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.385+0000 7f4d3ce03740 1 --2- 172.21.15.16:0/1381408470 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561660a85d40 0x561660a88130 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4d28097560 tx=0x7f4d280046c0 comp rx=0 tx=0).stop 2024-09-12T02:07:21.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.385+0000 7f4d3ce03740 1 -- 172.21.15.16:0/1381408470 shutdown_connections 2024-09-12T02:07:21.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.385+0000 7f4d3ce03740 1 --2- 172.21.15.16:0/1381408470 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561660a85d40 0x561660a88130 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:21.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.385+0000 7f4d3ce03740 1 --2- 172.21.15.16:0/1381408470 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561660a7b480 0x561660a85800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:21.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.385+0000 7f4d3ce03740 1 --2- 172.21.15.16:0/1381408470 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561660a7ab60 0x561660a7af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:21.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.385+0000 7f4d3ce03740 1 -- 172.21.15.16:0/1381408470 >> 172.21.15.16:0/1381408470 conn(0x561660a770f0 msgr2=0x561660a774d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:07:21.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.385+0000 7f4d3ce03740 1 -- 172.21.15.16:0/1381408470 shutdown_connections 2024-09-12T02:07:21.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.385+0000 7f4d3ce03740 1 -- 172.21.15.16:0/1381408470 wait complete. 2024-09-12T02:07:21.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.385+0000 7f4d3ce03740 1 -- start start 2024-09-12T02:07:21.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.385+0000 7f4d3ce03740 1 Processor -- start 2024-09-12T02:07:21.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.386+0000 7f4d3ce03740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561660a7ab60 0x561660a82c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:21.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.386+0000 7f4d3ce03740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561660a7b480 0x561660a83190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:21.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.386+0000 7f4d3ce03740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561660a85d40 0x561660a836d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:21.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.386+0000 7f4d3ce03740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x561660a88910 con 0x561660a7ab60 2024-09-12T02:07:21.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.386+0000 7f4d3ce03740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56166099af10 con 0x561660a85d40 2024-09-12T02:07:21.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.386+0000 7f4d3ce03740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x561660a88a90 con 0x561660a7b480 2024-09-12T02:07:21.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.386+0000 7f4d3ce03740 10 client.0 _ll_register_callbacks cb 0x561660963a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:07:21.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.386+0000 7f4d33fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561660a7ab60 0x561660a82c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:21.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.386+0000 7f4d33fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561660a7ab60 0x561660a82c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56734/0 (socket says 172.21.15.16:56734) 2024-09-12T02:07:21.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.386+0000 7f4d33fff640 1 -- 172.21.15.16:0/3320197092 learned_addr learned my addr 172.21.15.16:0/3320197092 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:07:21.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.386+0000 7f4d38fc7640 1 --2- 172.21.15.16:0/3320197092 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561660a85d40 0x561660a836d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:21.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.386+0000 7f4d337fe640 1 --2- 172.21.15.16:0/3320197092 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561660a7b480 0x561660a83190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:21.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.386+0000 7f4d33fff640 1 -- 172.21.15.16:0/3320197092 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561660a7b480 msgr2=0x561660a83190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:21.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.386+0000 7f4d33fff640 1 --2- 172.21.15.16:0/3320197092 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561660a7b480 0x561660a83190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:21.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.386+0000 7f4d33fff640 1 -- 172.21.15.16:0/3320197092 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561660a85d40 msgr2=0x561660a836d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:21.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.386+0000 7f4d33fff640 1 --2- 172.21.15.16:0/3320197092 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561660a85d40 0x561660a836d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:21.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.386+0000 7f4d33fff640 1 -- 172.21.15.16:0/3320197092 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5616607516a0 con 0x561660a7ab60 2024-09-12T02:07:21.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.386+0000 7f4d38fc7640 1 --2- 172.21.15.16:0/3320197092 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561660a85d40 0x561660a836d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:07:21.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.386+0000 7f4d33fff640 1 --2- 172.21.15.16:0/3320197092 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561660a7ab60 0x561660a82c50 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f4d2400fab0 tx=0x7f4d2400ff70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cce629f502175fd server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:07:21.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.386+0000 7f4d137fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:07:21.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.387+0000 7f4d137fe640 1 -- 172.21.15.16:0/3320197092 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d2401d060 con 0x561660a7ab60 2024-09-12T02:07:21.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.387+0000 7f4d137fe640 1 -- 172.21.15.16:0/3320197092 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f4d2400dcd0 con 0x561660a7ab60 2024-09-12T02:07:21.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.387+0000 7f4d3ce03740 1 -- 172.21.15.16:0/3320197092 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d2809ef40 con 0x561660a7ab60 2024-09-12T02:07:21.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.387+0000 7f4d137fe640 1 -- 172.21.15.16:0/3320197092 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4d24004020 con 0x561660a7ab60 2024-09-12T02:07:21.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.387+0000 7f4d3ce03740 10 client.17067 fetch_fsmap learned FSMap version 25 2024-09-12T02:07:21.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.387+0000 7f4d3ce03740 1 -- 172.21.15.16:0/3320197092 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x561660a945a0 con 0x561660a7ab60 2024-09-12T02:07:21.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.387+0000 7f4d137fe640 1 -- 172.21.15.16:0/3320197092 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d2809ef40 con 0x561660a7ab60 2024-09-12T02:07:21.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.387+0000 7f4d137fe640 1 -- 172.21.15.16:0/3320197092 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f4d2400b040 con 0x561660a7ab60 2024-09-12T02:07:21.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.387+0000 7f4d3ce03740 10 client.17067 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:07:21.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.387+0000 7f4d3ce03740 10 client.17067 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:07:21.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.387+0000 7f4d3ce03740 20 client.17067 populate_metadata read hostname 'smithi016' 2024-09-12T02:07:21.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.387+0000 7f4d3ce03740 1 --2- 172.21.15.16:0/3320197092 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x561660a95a60 0x561660a97e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:21.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.387+0000 7f4d3ce03740 4 client.17067 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:07:21.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.387+0000 7f4d3ce03740 1 -- 172.21.15.16:0/3320197092 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5616609ed7f0 con 0x561660a95a60 2024-09-12T02:07:21.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.388+0000 7f4d337fe640 1 --2- 172.21.15.16:0/3320197092 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x561660a95a60 0x561660a97e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:21.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.388+0000 7f4d337fe640 1 --2- 172.21.15.16:0/3320197092 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x561660a95a60 0x561660a97e50 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7dbb7249c3acb09b server_cookie=ff960e44aab14740 in_seq=0 out_seq=0 2024-09-12T02:07:21.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.388+0000 7f4d137fe640 10 client.17067 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:07:21.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.389+0000 7f4d137fe640 1 -- 172.21.15.16:0/3320197092 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5616609ed7f0 con 0x561660a95a60 2024-09-12T02:07:21.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.389+0000 7f4d137fe640 10 client.17067 handle_command_reply: tid=0 2024-09-12T02:07:21.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.389+0000 7f4d3ce03740 1 client.17067 shutdown 2024-09-12T02:07:21.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.389+0000 7f4d3ce03740 1 -- 172.21.15.16:0/3320197092 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561660a7ab60 msgr2=0x561660a82c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:21.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.389+0000 7f4d3ce03740 1 --2- 172.21.15.16:0/3320197092 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561660a7ab60 0x561660a82c50 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f4d2400fab0 tx=0x7f4d2400ff70 comp rx=0 tx=0).stop 2024-09-12T02:07:21.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.389+0000 7f4d3ce03740 1 -- 172.21.15.16:0/3320197092 shutdown_connections 2024-09-12T02:07:21.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.389+0000 7f4d3ce03740 1 --2- 172.21.15.16:0/3320197092 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x561660a95a60 0x561660a97e50 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:21.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.389+0000 7f4d3ce03740 1 --2- 172.21.15.16:0/3320197092 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561660a85d40 0x561660a836d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:21.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.389+0000 7f4d3ce03740 1 --2- 172.21.15.16:0/3320197092 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561660a7b480 0x561660a83190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:21.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.389+0000 7f4d3ce03740 1 --2- 172.21.15.16:0/3320197092 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561660a7ab60 0x561660a82c50 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:21.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.389+0000 7f4d3ce03740 1 -- 172.21.15.16:0/3320197092 >> 172.21.15.16:0/3320197092 conn(0x561660a770f0 msgr2=0x561660a7ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:07:21.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.389+0000 7f4d137fe640 0 client.17067 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:07:21.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.389+0000 7f4d3ce03740 1 -- 172.21.15.16:0/3320197092 shutdown_connections 2024-09-12T02:07:21.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.389+0000 7f4d3ce03740 1 -- 172.21.15.16:0/3320197092 wait complete. 2024-09-12T02:07:21.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.389+0000 7f4d3ce03740 20 client.17067 trim_cache size 0 max 16384 2024-09-12T02:07:21.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.407+0000 7f4d3ce03740 1 Processor -- start 2024-09-12T02:07:21.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.407+0000 7f4d3ce03740 1 -- start start 2024-09-12T02:07:21.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.407+0000 7f4d3ce03740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561660a7ab60 0x561660a84ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:21.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.407+0000 7f4d3ce03740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561660a7b480 0x561660a9d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:21.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.407+0000 7f4d3ce03740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561660a85d40 0x561660a9dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:21.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.407+0000 7f4d3ce03740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4d341b1fd0 con 0x561660a7b480 2024-09-12T02:07:21.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.407+0000 7f4d3ce03740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4d341af8a0 con 0x561660a7ab60 2024-09-12T02:07:21.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.407+0000 7f4d3ce03740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f4d341ad710 con 0x561660a85d40 2024-09-12T02:07:21.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.407+0000 7f4d337fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561660a7b480 0x561660a9d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:21.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.407+0000 7f4d337fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561660a7b480 0x561660a9d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56736/0 (socket says 172.21.15.16:56736) 2024-09-12T02:07:21.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.407+0000 7f4d337fe640 1 -- 172.21.15.16:0/300677432 learned_addr learned my addr 172.21.15.16:0/300677432 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:07:21.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.407+0000 7f4d33fff640 1 --2- 172.21.15.16:0/300677432 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561660a7ab60 0x561660a84ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:21.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.407+0000 7f4d38fc7640 1 --2- 172.21.15.16:0/300677432 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561660a85d40 0x561660a9dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:21.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.407+0000 7f4d337fe640 1 -- 172.21.15.16:0/300677432 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561660a85d40 msgr2=0x561660a9dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:21.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.407+0000 7f4d337fe640 1 --2- 172.21.15.16:0/300677432 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561660a85d40 0x561660a9dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:21.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.407+0000 7f4d337fe640 1 -- 172.21.15.16:0/300677432 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561660a7ab60 msgr2=0x561660a84ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:21.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.407+0000 7f4d337fe640 1 --2- 172.21.15.16:0/300677432 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561660a7ab60 0x561660a84ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:21.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.407+0000 7f4d337fe640 1 -- 172.21.15.16:0/300677432 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561660a81c00 con 0x561660a7b480 2024-09-12T02:07:21.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.407+0000 7f4d38fc7640 1 --2- 172.21.15.16:0/300677432 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561660a85d40 0x561660a9dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:07:21.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.407+0000 7f4d33fff640 1 --2- 172.21.15.16:0/300677432 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561660a7ab60 0x561660a84ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:07:21.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.407+0000 7f4d337fe640 1 --2- 172.21.15.16:0/300677432 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561660a7b480 0x561660a9d700 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f4d2800b470 tx=0x7f4d28017730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dc984593afc7bc4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:07:21.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.408+0000 7f4d19ffb640 1 -- 172.21.15.16:0/300677432 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d280b8070 con 0x561660a7b480 2024-09-12T02:07:21.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.408+0000 7f4d19ffb640 1 -- 172.21.15.16:0/300677432 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d28010cc0 con 0x561660a7b480 2024-09-12T02:07:21.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.408+0000 7f4d19ffb640 1 -- 172.21.15.16:0/300677432 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d2801e040 con 0x561660a7b480 2024-09-12T02:07:21.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.408+0000 7f4d3ce03740 1 -- 172.21.15.16:0/300677432 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561660a7b480 msgr2=0x561660a9d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:21.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.408+0000 7f4d3ce03740 1 --2- 172.21.15.16:0/300677432 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561660a7b480 0x561660a9d700 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f4d2800b470 tx=0x7f4d28017730 comp rx=0 tx=0).stop 2024-09-12T02:07:21.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.408+0000 7f4d3ce03740 1 -- 172.21.15.16:0/300677432 shutdown_connections 2024-09-12T02:07:21.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.408+0000 7f4d3ce03740 1 --2- 172.21.15.16:0/300677432 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561660a85d40 0x561660a9dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:21.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.408+0000 7f4d3ce03740 1 --2- 172.21.15.16:0/300677432 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561660a7b480 0x561660a9d700 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:21.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.408+0000 7f4d3ce03740 1 --2- 172.21.15.16:0/300677432 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561660a7ab60 0x561660a84ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:21.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.408+0000 7f4d3ce03740 1 -- 172.21.15.16:0/300677432 >> 172.21.15.16:0/300677432 conn(0x561660a770f0 msgr2=0x561660a843a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:07:21.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.408+0000 7f4d3ce03740 1 -- 172.21.15.16:0/300677432 shutdown_connections 2024-09-12T02:07:21.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.408+0000 7f4d3ce03740 1 -- 172.21.15.16:0/300677432 wait complete. 2024-09-12T02:07:21.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.409+0000 7f4d3ce03740 1 -- start start 2024-09-12T02:07:21.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.409+0000 7f4d3ce03740 1 Processor -- start 2024-09-12T02:07:21.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.409+0000 7f4d3ce03740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561660a7ab60 0x561660a89d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:21.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.409+0000 7f4d3ce03740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561660a7b480 0x561660a8a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:21.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.409+0000 7f4d3ce03740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561660a85d40 0x561660aff6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:21.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.409+0000 7f4d3ce03740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4d341add60 con 0x561660a7b480 2024-09-12T02:07:21.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.409+0000 7f4d3ce03740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4d341af1d0 con 0x561660a7ab60 2024-09-12T02:07:21.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.409+0000 7f4d3ce03740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f4d341ae430 con 0x561660a85d40 2024-09-12T02:07:21.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.409+0000 7f4d3ce03740 10 client.0 _ll_register_callbacks cb 0x7f4d2400b830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:07:21.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.409+0000 7f4d337fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561660a7b480 0x561660a8a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:21.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.409+0000 7f4d337fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561660a7b480 0x561660a8a2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56740/0 (socket says 172.21.15.16:56740) 2024-09-12T02:07:21.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.409+0000 7f4d337fe640 1 -- 172.21.15.16:0/2801737493 learned_addr learned my addr 172.21.15.16:0/2801737493 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:07:21.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.409+0000 7f4d33fff640 1 --2- 172.21.15.16:0/2801737493 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561660a7ab60 0x561660a89d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:21.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.410+0000 7f4d38fc7640 1 --2- 172.21.15.16:0/2801737493 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561660a85d40 0x561660aff6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:21.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.410+0000 7f4d337fe640 1 -- 172.21.15.16:0/2801737493 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561660a85d40 msgr2=0x561660aff6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:21.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.410+0000 7f4d337fe640 1 --2- 172.21.15.16:0/2801737493 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561660a85d40 0x561660aff6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:21.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.410+0000 7f4d337fe640 1 -- 172.21.15.16:0/2801737493 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561660a7ab60 msgr2=0x561660a89d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:21.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.410+0000 7f4d337fe640 1 --2- 172.21.15.16:0/2801737493 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561660a7ab60 0x561660a89d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:21.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.410+0000 7f4d337fe640 1 -- 172.21.15.16:0/2801737493 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561660a8ab30 con 0x561660a7b480 2024-09-12T02:07:21.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.410+0000 7f4d38fc7640 1 --2- 172.21.15.16:0/2801737493 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561660a85d40 0x561660aff6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:07:21.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.410+0000 7f4d337fe640 1 --2- 172.21.15.16:0/2801737493 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561660a7b480 0x561660a8a2a0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f4d28098e40 tx=0x7f4d2809ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75bdcc8428866b2c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:07:21.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.410+0000 7f4d137fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:07:21.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.410+0000 7f4d137fe640 1 -- 172.21.15.16:0/2801737493 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d2801e050 con 0x561660a7b480 2024-09-12T02:07:21.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.410+0000 7f4d137fe640 1 -- 172.21.15.16:0/2801737493 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f4d28099fe0 con 0x561660a7b480 2024-09-12T02:07:21.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.410+0000 7f4d3ce03740 1 -- 172.21.15.16:0/2801737493 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d28010cc0 con 0x561660a7b480 2024-09-12T02:07:21.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.410+0000 7f4d137fe640 1 -- 172.21.15.16:0/2801737493 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4d280b8070 con 0x561660a7b480 2024-09-12T02:07:21.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.410+0000 7f4d3ce03740 10 client.17073 fetch_fsmap learned FSMap version 25 2024-09-12T02:07:21.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.410+0000 7f4d3ce03740 1 -- 172.21.15.16:0/2801737493 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x561660b00c90 con 0x561660a7b480 2024-09-12T02:07:21.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.410+0000 7f4d137fe640 1 -- 172.21.15.16:0/2801737493 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d28010cc0 con 0x561660a7b480 2024-09-12T02:07:21.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.411+0000 7f4d137fe640 1 -- 172.21.15.16:0/2801737493 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f4d280b9380 con 0x561660a7b480 2024-09-12T02:07:21.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.411+0000 7f4d3ce03740 10 client.17073 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:07:21.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.411+0000 7f4d3ce03740 10 client.17073 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:07:21.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.411+0000 7f4d3ce03740 20 client.17073 populate_metadata read hostname 'smithi016' 2024-09-12T02:07:21.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.411+0000 7f4d3ce03740 1 --2- 172.21.15.16:0/2801737493 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x561660b00f10 0x561660b03300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:21.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.411+0000 7f4d3ce03740 4 client.17073 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-12T02:07:21.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.411+0000 7f4d3ce03740 1 -- 172.21.15.16:0/2801737493 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5616609e7d00 con 0x561660b00f10 2024-09-12T02:07:21.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.411+0000 7f4d33fff640 1 --2- 172.21.15.16:0/2801737493 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x561660b00f10 0x561660b03300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:21.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.411+0000 7f4d33fff640 1 --2- 172.21.15.16:0/2801737493 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x561660b00f10 0x561660b03300 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=25d9233c0dcc8d77 server_cookie=602cf7884fabd035 in_seq=0 out_seq=0 2024-09-12T02:07:21.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.411+0000 7f4d137fe640 10 client.17073 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:07:21.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.413+0000 7f4d137fe640 1 -- 172.21.15.16:0/2801737493 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5616609e7d00 con 0x561660b00f10 2024-09-12T02:07:21.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.413+0000 7f4d137fe640 10 client.17073 handle_command_reply: tid=0 2024-09-12T02:07:21.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.413+0000 7f4d3ce03740 1 client.17073 shutdown 2024-09-12T02:07:21.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.413+0000 7f4d3ce03740 1 -- 172.21.15.16:0/2801737493 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561660a7b480 msgr2=0x561660a8a2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:21.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.413+0000 7f4d3ce03740 1 --2- 172.21.15.16:0/2801737493 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561660a7b480 0x561660a8a2a0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f4d28098e40 tx=0x7f4d2809ad10 comp rx=0 tx=0).stop 2024-09-12T02:07:21.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.413+0000 7f4d3ce03740 1 -- 172.21.15.16:0/2801737493 shutdown_connections 2024-09-12T02:07:21.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.413+0000 7f4d3ce03740 1 --2- 172.21.15.16:0/2801737493 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x561660b00f10 0x561660b03300 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:21.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.413+0000 7f4d3ce03740 1 --2- 172.21.15.16:0/2801737493 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561660a85d40 0x561660aff6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:21.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.413+0000 7f4d3ce03740 1 --2- 172.21.15.16:0/2801737493 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561660a7b480 0x561660a8a2a0 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:21.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.413+0000 7f4d137fe640 0 client.17073 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:07:21.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.413+0000 7f4d3ce03740 1 --2- 172.21.15.16:0/2801737493 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561660a7ab60 0x561660a89d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:21.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.413+0000 7f4d3ce03740 1 -- 172.21.15.16:0/2801737493 >> 172.21.15.16:0/2801737493 conn(0x561660a770f0 msgr2=0x561660a7cdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:07:21.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.413+0000 7f4d3ce03740 1 -- 172.21.15.16:0/2801737493 shutdown_connections 2024-09-12T02:07:21.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.413+0000 7f4d3ce03740 1 -- 172.21.15.16:0/2801737493 wait complete. 2024-09-12T02:07:21.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.413+0000 7f4d3ce03740 20 client.17073 trim_cache size 0 max 16384 2024-09-12T02:07:21.431 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T02:07:21.431 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-09-12T02:07:21.431 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "f01d3f23-7779-4f53-b011-418060ac2d07", 2024-09-12T02:07:21.431 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-09-12T02:07:21.431 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T02:07:21.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.414+0000 7f4d3aa2a640 1 -- 172.21.15.16:0/3440513265 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f4d1c0a2c00 msgr2=0x7f4d1c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:21.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.414+0000 7f4d3aa2a640 1 --2- 172.21.15.16:0/3440513265 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f4d1c0a2c00 0x7f4d1c0a50c0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f4d2c031f80 tx=0x7f4d2c02a8c0 comp rx=0 tx=0).stop 2024-09-12T02:07:21.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.414+0000 7f4d3aa2a640 1 -- 172.21.15.16:0/3440513265 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d34100e30 msgr2=0x7f4d34107710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:21.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.414+0000 7f4d3aa2a640 1 --2- 172.21.15.16:0/3440513265 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d34100e30 0x7f4d34107710 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f4d2800b4a0 tx=0x7f4d2800b960 comp rx=0 tx=0).stop 2024-09-12T02:07:21.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.414+0000 7f4d3aa2a640 1 -- 172.21.15.16:0/3440513265 shutdown_connections 2024-09-12T02:07:21.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.414+0000 7f4d3aa2a640 1 --2- 172.21.15.16:0/3440513265 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f4d1c0a2c00 0x7f4d1c0a50c0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:21.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.414+0000 7f4d3aa2a640 1 --2- 172.21.15.16:0/3440513265 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d341055b0 0x7f4d34107c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:21.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.414+0000 7f4d3aa2a640 1 --2- 172.21.15.16:0/3440513265 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d34100e30 0x7f4d34107710 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:21.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.414+0000 7f4d3aa2a640 1 --2- 172.21.15.16:0/3440513265 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4d34100480 0x7f4d341071d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:21.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.414+0000 7f4d3aa2a640 1 -- 172.21.15.16:0/3440513265 >> 172.21.15.16:0/3440513265 conn(0x7f4d34077210 msgr2=0x7f4d34103ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:07:21.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.414+0000 7f4d3aa2a640 1 -- 172.21.15.16:0/3440513265 shutdown_connections 2024-09-12T02:07:21.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:21.414+0000 7f4d3aa2a640 1 -- 172.21.15.16:0/3440513265 wait complete. 2024-09-12T02:07:22.252 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T02:07:22.429 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:07:22 smithi184 ceph-mon[35593]: pgmap v498: 97 pgs: 97 active+clean; 3.0 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-09-12T02:07:22.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:07:22 smithi016 ceph-mon[30947]: pgmap v498: 97 pgs: 97 active+clean; 3.0 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-09-12T02:07:22.471 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:07:22 smithi033 ceph-mon[35332]: pgmap v498: 97 pgs: 97 active+clean; 3.0 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-09-12T02:07:22.509 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:07:22.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:22.975+0000 7fbb9b1fc640 1 -- 172.21.15.16:0/1933907488 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb94102d40 msgr2=0x7fbb94103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:22.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:22.975+0000 7fbb9b1fc640 1 --2- 172.21.15.16:0/1933907488 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb94102d40 0x7fbb94103120 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fbb90009900 tx=0x7fbb9002a480 comp rx=0 tx=0).stop 2024-09-12T02:07:22.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:22.975+0000 7fbb9b1fc640 1 -- 172.21.15.16:0/1933907488 shutdown_connections 2024-09-12T02:07:22.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:22.975+0000 7fbb9b1fc640 1 --2- 172.21.15.16:0/1933907488 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbb9410c170 0x7fbb9410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:22.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:22.975+0000 7fbb9b1fc640 1 --2- 172.21.15.16:0/1933907488 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbb94103660 0x7fbb9410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:22.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:22.975+0000 7fbb9b1fc640 1 --2- 172.21.15.16:0/1933907488 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb94102d40 0x7fbb94103120 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:22.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:22.975+0000 7fbb9b1fc640 1 -- 172.21.15.16:0/1933907488 >> 172.21.15.16:0/1933907488 conn(0x7fbb940fc7a0 msgr2=0x7fbb940febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:07:22.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:22.975+0000 7fbb9b1fc640 1 -- 172.21.15.16:0/1933907488 shutdown_connections 2024-09-12T02:07:22.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:22.975+0000 7fbb9b1fc640 1 -- 172.21.15.16:0/1933907488 wait complete. 2024-09-12T02:07:22.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:22.976+0000 7fbb9b1fc640 1 Processor -- start 2024-09-12T02:07:22.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:22.976+0000 7fbb9b1fc640 1 -- start start 2024-09-12T02:07:22.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:22.976+0000 7fbb9b1fc640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbb94102d40 0x7fbb941983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:22.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:22.976+0000 7fbb9b1fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbb94103660 0x7fbb94198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:22.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:22.976+0000 7fbb9b1fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb9410c170 0x7fbb9419ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:22.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:22.976+0000 7fbb9b1fc640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbb94111f70 con 0x7fbb9410c170 2024-09-12T02:07:22.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:22.976+0000 7fbb9b1fc640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbb94111df0 con 0x7fbb94103660 2024-09-12T02:07:22.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:22.976+0000 7fbb9b1fc640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fbb941120f0 con 0x7fbb94102d40 2024-09-12T02:07:22.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:22.976+0000 7fbb99799640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb9410c170 0x7fbb9419ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:22.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:22.976+0000 7fbb99799640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb9410c170 0x7fbb9419ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56774/0 (socket says 172.21.15.16:56774) 2024-09-12T02:07:22.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:22.976+0000 7fbb99799640 1 -- 172.21.15.16:0/2168043942 learned_addr learned my addr 172.21.15.16:0/2168043942 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:07:22.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:22.976+0000 7fbb98f98640 1 --2- 172.21.15.16:0/2168043942 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbb94102d40 0x7fbb941983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:22.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:22.976+0000 7fbb8bfff640 1 --2- 172.21.15.16:0/2168043942 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbb94103660 0x7fbb94198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:22.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:22.976+0000 7fbb99799640 1 -- 172.21.15.16:0/2168043942 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbb94102d40 msgr2=0x7fbb941983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:22.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:22.976+0000 7fbb99799640 1 --2- 172.21.15.16:0/2168043942 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbb94102d40 0x7fbb941983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:22.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:22.976+0000 7fbb99799640 1 -- 172.21.15.16:0/2168043942 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbb94103660 msgr2=0x7fbb94198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:22.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:22.976+0000 7fbb99799640 1 --2- 172.21.15.16:0/2168043942 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbb94103660 0x7fbb94198930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:22.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:22.976+0000 7fbb99799640 1 -- 172.21.15.16:0/2168043942 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb9419d3a0 con 0x7fbb9410c170 2024-09-12T02:07:22.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:22.977+0000 7fbb99799640 1 --2- 172.21.15.16:0/2168043942 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb9410c170 0x7fbb9419ccc0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7fbb8c00eaa0 tx=0x7fbb8c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:07:22.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:22.977+0000 7fbb89ffb640 1 -- 172.21.15.16:0/2168043942 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb8c00cdb0 con 0x7fbb9410c170 2024-09-12T02:07:22.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:22.977+0000 7fbb89ffb640 1 -- 172.21.15.16:0/2168043942 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb8c005020 con 0x7fbb9410c170 2024-09-12T02:07:22.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:22.977+0000 7fbb9b1fc640 1 -- 172.21.15.16:0/2168043942 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb9419d690 con 0x7fbb9410c170 2024-09-12T02:07:22.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:22.977+0000 7fbb89ffb640 1 -- 172.21.15.16:0/2168043942 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb8c0116b0 con 0x7fbb9410c170 2024-09-12T02:07:22.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:22.977+0000 7fbb9b1fc640 1 -- 172.21.15.16:0/2168043942 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb941a4f70 con 0x7fbb9410c170 2024-09-12T02:07:22.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:22.979+0000 7fbb89ffb640 1 -- 172.21.15.16:0/2168043942 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fbb8c005560 con 0x7fbb9410c170 2024-09-12T02:07:22.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:22.980+0000 7fbb9d5d5740 1 Processor -- start 2024-09-12T02:07:22.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:22.980+0000 7fbb9d5d5740 1 -- start start 2024-09-12T02:07:22.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:22.980+0000 7fbb9d5d5740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5569511a34a0 0x5569511a3880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:22.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:22.980+0000 7fbb9d5d5740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5569511a3dc0 0x5569511ae140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:22.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:22.980+0000 7fbb9d5d5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5569511ae680 0x5569511b0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:22.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:22.980+0000 7fbb9d5d5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5569511092e0 con 0x5569511ae680 2024-09-12T02:07:22.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:22.980+0000 7fbb9d5d5740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5569510b0ce0 con 0x5569511a3dc0 2024-09-12T02:07:22.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:22.980+0000 7fbb9d5d5740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5569511052a0 con 0x5569511a34a0 2024-09-12T02:07:22.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:22.980+0000 7fbb89ffb640 1 --2- 172.21.15.16:0/2168043942 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fbb7c0a2c00 0x7fbb7c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:22.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:22.980+0000 7fbb8bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5569511ae680 0x5569511b0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:22.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:22.980+0000 7fbb98f98640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5569511a3dc0 0x5569511ae140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:22.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:22.980+0000 7fbb8bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5569511ae680 0x5569511b0a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56790/0 (socket says 172.21.15.16:56790) 2024-09-12T02:07:22.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:22.980+0000 7fbb8bfff640 1 -- 172.21.15.16:0/1618321367 learned_addr learned my addr 172.21.15.16:0/1618321367 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:07:22.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:22.980+0000 7fbb98f98640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5569511a3dc0 0x5569511ae140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.16:42628/0 (socket says 172.21.15.16:42628) 2024-09-12T02:07:22.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:22.980+0000 7fbb99799640 1 --2- 172.21.15.16:0/2168043942 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fbb7c0a2c00 0x7fbb7c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:22.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:22.980+0000 7fbb8bfff640 1 --2- 172.21.15.16:0/1618321367 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5569511a34a0 0x5569511a3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:22.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:22.980+0000 7fbb89ffb640 1 -- 172.21.15.16:0/2168043942 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fbb8c0cf230 con 0x7fbb9410c170 2024-09-12T02:07:22.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:22.981+0000 7fbb98f98640 1 -- 172.21.15.16:0/1618321367 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5569511a34a0 msgr2=0x5569511a3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:22.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:22.981+0000 7fbb98f98640 1 --2- 172.21.15.16:0/1618321367 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5569511a34a0 0x5569511a3880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:22.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:22.981+0000 7fbb98f98640 1 -- 172.21.15.16:0/1618321367 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5569511ae680 msgr2=0x5569511b0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:22.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:22.981+0000 7fbb98f98640 1 --2- 172.21.15.16:0/1618321367 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5569511ae680 0x5569511b0a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:22.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:22.981+0000 7fbb98f98640 1 -- 172.21.15.16:0/1618321367 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556951095440 con 0x5569511a3dc0 2024-09-12T02:07:22.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:22.981+0000 7fbb99799640 1 --2- 172.21.15.16:0/2168043942 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fbb7c0a2c00 0x7fbb7c0a50c0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fbb8c09ba20 tx=0x7fbb8c06aa10 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:07:22.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:22.981+0000 7fbb98f98640 1 --2- 172.21.15.16:0/1618321367 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5569511a3dc0 0x5569511ae140 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fbb90009230 tx=0x7fbb90032ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ccdba9869f423700 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:07:22.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:22.981+0000 7fbb7affd640 1 -- 172.21.15.16:0/1618321367 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb9003f070 con 0x5569511a3dc0 2024-09-12T02:07:22.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:22.981+0000 7fbb7affd640 1 -- 172.21.15.16:0/1618321367 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb900317b0 con 0x5569511a3dc0 2024-09-12T02:07:22.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:22.981+0000 7fbb7affd640 1 -- 172.21.15.16:0/1618321367 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb90035070 con 0x5569511a3dc0 2024-09-12T02:07:22.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:22.982+0000 7fbb9d5d5740 1 -- 172.21.15.16:0/1618321367 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5569511a3dc0 msgr2=0x5569511ae140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:22.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:22.982+0000 7fbb9d5d5740 1 --2- 172.21.15.16:0/1618321367 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5569511a3dc0 0x5569511ae140 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fbb90009230 tx=0x7fbb90032ce0 comp rx=0 tx=0).stop 2024-09-12T02:07:22.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:22.982+0000 7fbb9d5d5740 1 -- 172.21.15.16:0/1618321367 shutdown_connections 2024-09-12T02:07:22.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:22.982+0000 7fbb9d5d5740 1 --2- 172.21.15.16:0/1618321367 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5569511ae680 0x5569511b0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:22.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:22.982+0000 7fbb9d5d5740 1 --2- 172.21.15.16:0/1618321367 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5569511a3dc0 0x5569511ae140 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:22.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:22.982+0000 7fbb9d5d5740 1 --2- 172.21.15.16:0/1618321367 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5569511a34a0 0x5569511a3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:22.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:22.982+0000 7fbb9d5d5740 1 -- 172.21.15.16:0/1618321367 >> 172.21.15.16:0/1618321367 conn(0x55695119fa30 msgr2=0x55695119fe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:07:22.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:22.982+0000 7fbb9d5d5740 1 -- 172.21.15.16:0/1618321367 shutdown_connections 2024-09-12T02:07:22.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:22.982+0000 7fbb9d5d5740 1 -- 172.21.15.16:0/1618321367 wait complete. 2024-09-12T02:07:22.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:22.982+0000 7fbb9d5d5740 1 -- start start 2024-09-12T02:07:22.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:22.982+0000 7fbb9d5d5740 1 Processor -- start 2024-09-12T02:07:22.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:22.983+0000 7fbb9d5d5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5569511a34a0 0x5569511adef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:22.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:22.983+0000 7fbb9d5d5740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5569511a3dc0 0x5569511ab1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:22.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:22.983+0000 7fbb9d5d5740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5569511ae680 0x5569511ab6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:22.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:22.983+0000 7fbb9d5d5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5569511b1250 con 0x5569511a34a0 2024-09-12T02:07:22.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:22.983+0000 7fbb9d5d5740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5569510c4050 con 0x5569511a3dc0 2024-09-12T02:07:22.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:22.983+0000 7fbb9d5d5740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5569511b13d0 con 0x5569511ae680 2024-09-12T02:07:22.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:22.983+0000 7fbb9d5d5740 10 client.0 _ll_register_callbacks cb 0x556951104160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:07:22.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:22.983+0000 7fbb8bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5569511a34a0 0x5569511adef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:22.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:22.983+0000 7fbb8bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5569511a34a0 0x5569511adef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56792/0 (socket says 172.21.15.16:56792) 2024-09-12T02:07:22.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:22.983+0000 7fbb8bfff640 1 -- 172.21.15.16:0/3040688602 learned_addr learned my addr 172.21.15.16:0/3040688602 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:07:22.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:22.983+0000 7fbb98f98640 1 --2- 172.21.15.16:0/3040688602 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5569511a3dc0 0x5569511ab1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:22.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:22.983+0000 7fbb8bfff640 1 -- 172.21.15.16:0/3040688602 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5569511ae680 msgr2=0x5569511ab6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:22.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:22.983+0000 7fbb8bfff640 1 --2- 172.21.15.16:0/3040688602 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5569511ae680 0x5569511ab6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:22.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:22.983+0000 7fbb8bfff640 1 -- 172.21.15.16:0/3040688602 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5569511a3dc0 msgr2=0x5569511ab1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:22.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:22.983+0000 7fbb8bfff640 1 --2- 172.21.15.16:0/3040688602 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5569511a3dc0 0x5569511ab1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:22.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:22.983+0000 7fbb8bfff640 1 -- 172.21.15.16:0/3040688602 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556950e7a3c0 con 0x5569511a34a0 2024-09-12T02:07:22.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:22.983+0000 7fbb8bfff640 1 --2- 172.21.15.16:0/3040688602 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5569511a34a0 0x5569511adef0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7fbb84013f00 tx=0x7fbb84010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5f18e8373c3e874 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:07:22.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:22.983+0000 7fbb73fff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:07:23.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:22.984+0000 7fbb73fff640 1 -- 172.21.15.16:0/3040688602 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb84011440 con 0x5569511a34a0 2024-09-12T02:07:23.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:22.984+0000 7fbb73fff640 1 -- 172.21.15.16:0/3040688602 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fbb840115e0 con 0x5569511a34a0 2024-09-12T02:07:23.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:22.984+0000 7fbb9d5d5740 1 -- 172.21.15.16:0/3040688602 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb900317b0 con 0x5569511a34a0 2024-09-12T02:07:23.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:22.984+0000 7fbb73fff640 1 -- 172.21.15.16:0/3040688602 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fbb84026050 con 0x5569511a34a0 2024-09-12T02:07:23.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:22.984+0000 7fbb9d5d5740 10 client.17085 fetch_fsmap learned FSMap version 25 2024-09-12T02:07:23.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:22.984+0000 7fbb9d5d5740 1 -- 172.21.15.16:0/3040688602 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5569511bcbf0 con 0x5569511a34a0 2024-09-12T02:07:23.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:22.984+0000 7fbb73fff640 1 -- 172.21.15.16:0/3040688602 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb900317b0 con 0x5569511a34a0 2024-09-12T02:07:23.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:22.984+0000 7fbb73fff640 1 -- 172.21.15.16:0/3040688602 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fbb8401d070 con 0x5569511a34a0 2024-09-12T02:07:23.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:22.984+0000 7fbb9d5d5740 10 client.17085 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:07:23.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:22.984+0000 7fbb9d5d5740 10 client.17085 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:07:23.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:22.984+0000 7fbb9d5d5740 20 client.17085 populate_metadata read hostname 'smithi016' 2024-09-12T02:07:23.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:22.984+0000 7fbb9d5d5740 1 --2- 172.21.15.16:0/3040688602 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5569511be050 0x5569511c04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:23.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:22.984+0000 7fbb9d5d5740 4 client.17085 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:07:23.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:22.984+0000 7fbb9d5d5740 1 -- 172.21.15.16:0/3040688602 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556951116850 con 0x5569511be050 2024-09-12T02:07:23.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:22.985+0000 7fbb98f98640 1 --2- 172.21.15.16:0/3040688602 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5569511be050 0x5569511c04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:23.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:22.985+0000 7fbb73fff640 10 client.17085 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:07:23.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:22.985+0000 7fbb98f98640 1 --2- 172.21.15.16:0/3040688602 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5569511be050 0x5569511c04d0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d99fa7111aa7af47 server_cookie=a91c1e3345978e65 in_seq=0 out_seq=0 2024-09-12T02:07:23.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:22.986+0000 7fbb73fff640 1 -- 172.21.15.16:0/3040688602 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556951116850 con 0x5569511be050 2024-09-12T02:07:23.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:22.986+0000 7fbb73fff640 10 client.17085 handle_command_reply: tid=0 2024-09-12T02:07:23.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:22.986+0000 7fbb9d5d5740 1 client.17085 shutdown 2024-09-12T02:07:23.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:22.986+0000 7fbb9d5d5740 1 -- 172.21.15.16:0/3040688602 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5569511a34a0 msgr2=0x5569511adef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:23.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:22.986+0000 7fbb9d5d5740 1 --2- 172.21.15.16:0/3040688602 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5569511a34a0 0x5569511adef0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7fbb84013f00 tx=0x7fbb84010800 comp rx=0 tx=0).stop 2024-09-12T02:07:23.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:22.986+0000 7fbb9d5d5740 1 -- 172.21.15.16:0/3040688602 shutdown_connections 2024-09-12T02:07:23.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:22.986+0000 7fbb9d5d5740 1 --2- 172.21.15.16:0/3040688602 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5569511be050 0x5569511c04d0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:23.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:22.986+0000 7fbb9d5d5740 1 --2- 172.21.15.16:0/3040688602 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5569511ae680 0x5569511ab6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:23.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:22.986+0000 7fbb9d5d5740 1 --2- 172.21.15.16:0/3040688602 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5569511a3dc0 0x5569511ab1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:23.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:22.986+0000 7fbb73fff640 0 client.17085 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:07:23.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:22.986+0000 7fbb9d5d5740 1 --2- 172.21.15.16:0/3040688602 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5569511a34a0 0x5569511adef0 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:23.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:22.986+0000 7fbb9d5d5740 1 -- 172.21.15.16:0/3040688602 >> 172.21.15.16:0/3040688602 conn(0x55695119fa30 msgr2=0x5569511a5710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:07:23.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:22.986+0000 7fbb9d5d5740 1 -- 172.21.15.16:0/3040688602 shutdown_connections 2024-09-12T02:07:23.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:22.986+0000 7fbb9d5d5740 1 -- 172.21.15.16:0/3040688602 wait complete. 2024-09-12T02:07:23.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:22.986+0000 7fbb9d5d5740 20 client.17085 trim_cache size 0 max 16384 2024-09-12T02:07:23.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:23.003+0000 7fbb9d5d5740 1 Processor -- start 2024-09-12T02:07:23.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:23.003+0000 7fbb9d5d5740 1 -- start start 2024-09-12T02:07:23.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:23.003+0000 7fbb9d5d5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5569511a34a0 0x5569511a9a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:23.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:23.003+0000 7fbb9d5d5740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5569511a3dc0 0x5569511b6a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:23.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:23.003+0000 7fbb9d5d5740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5569511ae680 0x5569511b6f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:23.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:23.004+0000 7fbb9d5d5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbb941afc10 con 0x5569511a34a0 2024-09-12T02:07:23.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:23.004+0000 7fbb9d5d5740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbb941ad4e0 con 0x5569511ae680 2024-09-12T02:07:23.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:23.004+0000 7fbb9d5d5740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fbb941ab330 con 0x5569511a3dc0 2024-09-12T02:07:23.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:23.004+0000 7fbb8bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5569511a34a0 0x5569511a9a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:23.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:23.004+0000 7fbb8bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5569511a34a0 0x5569511a9a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56810/0 (socket says 172.21.15.16:56810) 2024-09-12T02:07:23.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:23.004+0000 7fbb8bfff640 1 -- 172.21.15.16:0/639444418 learned_addr learned my addr 172.21.15.16:0/639444418 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:07:23.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:23.004+0000 7fbb98f98640 1 --2- 172.21.15.16:0/639444418 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5569511a3dc0 0x5569511b6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:23.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:23.004+0000 7fbb8bfff640 1 --2- 172.21.15.16:0/639444418 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5569511ae680 0x5569511b6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:23.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:23.004+0000 7fbb8bfff640 1 -- 172.21.15.16:0/639444418 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5569511a3dc0 msgr2=0x5569511b6a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:23.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:23.004+0000 7fbb8bfff640 1 --2- 172.21.15.16:0/639444418 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5569511a3dc0 0x5569511b6a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:23.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:23.004+0000 7fbb8bfff640 1 -- 172.21.15.16:0/639444418 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5569511ae680 msgr2=0x5569511b6f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:23.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:23.004+0000 7fbb8bfff640 1 --2- 172.21.15.16:0/639444418 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5569511ae680 0x5569511b6f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:23.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:23.004+0000 7fbb8bfff640 1 -- 172.21.15.16:0/639444418 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5569511aa150 con 0x5569511a34a0 2024-09-12T02:07:23.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:23.004+0000 7fbb8bfff640 1 --2- 172.21.15.16:0/639444418 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5569511a34a0 0x5569511a9a70 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7fbb84010d80 tx=0x7fbb84005b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c22e3cfdcaa9ed2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:07:23.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:23.004+0000 7fbb7a7fc640 1 -- 172.21.15.16:0/639444418 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb8401eca0 con 0x5569511a34a0 2024-09-12T02:07:23.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:23.004+0000 7fbb7a7fc640 1 -- 172.21.15.16:0/639444418 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb84004520 con 0x5569511a34a0 2024-09-12T02:07:23.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:23.004+0000 7fbb7a7fc640 1 -- 172.21.15.16:0/639444418 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb84003bb0 con 0x5569511a34a0 2024-09-12T02:07:23.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:23.005+0000 7fbb9d5d5740 1 -- 172.21.15.16:0/639444418 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5569511a34a0 msgr2=0x5569511a9a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:23.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:23.005+0000 7fbb9d5d5740 1 --2- 172.21.15.16:0/639444418 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5569511a34a0 0x5569511a9a70 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7fbb84010d80 tx=0x7fbb84005b10 comp rx=0 tx=0).stop 2024-09-12T02:07:23.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:23.005+0000 7fbb9d5d5740 1 -- 172.21.15.16:0/639444418 shutdown_connections 2024-09-12T02:07:23.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:23.005+0000 7fbb9d5d5740 1 --2- 172.21.15.16:0/639444418 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5569511ae680 0x5569511b6f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:23.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:23.005+0000 7fbb9d5d5740 1 --2- 172.21.15.16:0/639444418 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5569511a3dc0 0x5569511b6a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:23.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:23.005+0000 7fbb9d5d5740 1 --2- 172.21.15.16:0/639444418 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5569511a34a0 0x5569511a9a70 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:23.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:23.005+0000 7fbb9d5d5740 1 -- 172.21.15.16:0/639444418 >> 172.21.15.16:0/639444418 conn(0x55695119fa30 msgr2=0x55695119eb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:07:23.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:23.005+0000 7fbb9d5d5740 1 -- 172.21.15.16:0/639444418 shutdown_connections 2024-09-12T02:07:23.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:23.005+0000 7fbb9d5d5740 1 -- 172.21.15.16:0/639444418 wait complete. 2024-09-12T02:07:23.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:23.005+0000 7fbb9d5d5740 1 -- start start 2024-09-12T02:07:23.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:23.006+0000 7fbb9d5d5740 1 Processor -- start 2024-09-12T02:07:23.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:23.006+0000 7fbb9d5d5740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5569511a34a0 0x5569512204c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:23.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:23.006+0000 7fbb9d5d5740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5569511a3dc0 0x556951220a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:23.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:23.006+0000 7fbb9d5d5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5569511ae680 0x556951224ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:23.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:23.006+0000 7fbb9d5d5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbb941ab9a0 con 0x5569511ae680 2024-09-12T02:07:23.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:23.006+0000 7fbb9d5d5740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbb941ace10 con 0x5569511a34a0 2024-09-12T02:07:23.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:23.006+0000 7fbb9d5d5740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fbb941ac070 con 0x5569511a3dc0 2024-09-12T02:07:23.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:23.006+0000 7fbb9d5d5740 10 client.0 _ll_register_callbacks cb 0x7fbb8400bdb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:07:23.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:23.006+0000 7fbb8bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5569511ae680 0x556951224ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:23.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:23.006+0000 7fbb8bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5569511ae680 0x556951224ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56818/0 (socket says 172.21.15.16:56818) 2024-09-12T02:07:23.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:23.006+0000 7fbb8bfff640 1 -- 172.21.15.16:0/3272174823 learned_addr learned my addr 172.21.15.16:0/3272174823 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:07:23.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:23.006+0000 7fbb98f98640 1 --2- 172.21.15.16:0/3272174823 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5569511a3dc0 0x556951220a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:23.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:23.006+0000 7fbb8bfff640 1 --2- 172.21.15.16:0/3272174823 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5569511a34a0 0x5569512204c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:23.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:23.006+0000 7fbb8bfff640 1 -- 172.21.15.16:0/3272174823 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5569511a3dc0 msgr2=0x556951220a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:23.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:23.007+0000 7fbb8bfff640 1 --2- 172.21.15.16:0/3272174823 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5569511a3dc0 0x556951220a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:23.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:23.007+0000 7fbb8bfff640 1 -- 172.21.15.16:0/3272174823 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5569511a34a0 msgr2=0x5569512204c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:23.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:23.007+0000 7fbb8bfff640 1 --2- 172.21.15.16:0/3272174823 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5569511a34a0 0x5569512204c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:23.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:23.007+0000 7fbb8bfff640 1 -- 172.21.15.16:0/3272174823 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556951225170 con 0x5569511ae680 2024-09-12T02:07:23.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:23.007+0000 7fbb98f98640 1 --2- 172.21.15.16:0/3272174823 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5569511a3dc0 0x556951220a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:07:23.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:23.007+0000 7fbb8bfff640 1 --2- 172.21.15.16:0/3272174823 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5569511ae680 0x556951224ad0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7fbb840138e0 tx=0x7fbb84003790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5503455dbf7284f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:07:23.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:23.007+0000 7fbb73fff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:07:23.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:23.007+0000 7fbb73fff640 1 -- 172.21.15.16:0/3272174823 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb8401ecb0 con 0x5569511ae680 2024-09-12T02:07:23.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:23.007+0000 7fbb73fff640 1 -- 172.21.15.16:0/3272174823 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fbb8401e3e0 con 0x5569511ae680 2024-09-12T02:07:23.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:23.007+0000 7fbb9d5d5740 1 -- 172.21.15.16:0/3272174823 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb84004520 con 0x5569511ae680 2024-09-12T02:07:23.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:23.007+0000 7fbb73fff640 1 -- 172.21.15.16:0/3272174823 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fbb8401e5e0 con 0x5569511ae680 2024-09-12T02:07:23.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:23.007+0000 7fbb9d5d5740 10 client.17091 fetch_fsmap learned FSMap version 25 2024-09-12T02:07:23.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:23.007+0000 7fbb9d5d5740 1 -- 172.21.15.16:0/3272174823 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55695122d710 con 0x5569511ae680 2024-09-12T02:07:23.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:23.007+0000 7fbb73fff640 1 -- 172.21.15.16:0/3272174823 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb84004520 con 0x5569511ae680 2024-09-12T02:07:23.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:23.008+0000 7fbb73fff640 1 -- 172.21.15.16:0/3272174823 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fbb84038020 con 0x5569511ae680 2024-09-12T02:07:23.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:23.008+0000 7fbb9d5d5740 10 client.17091 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:07:23.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:23.008+0000 7fbb9d5d5740 10 client.17091 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:07:23.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:23.008+0000 7fbb9d5d5740 20 client.17091 populate_metadata read hostname 'smithi016' 2024-09-12T02:07:23.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:23.008+0000 7fbb9d5d5740 1 --2- 172.21.15.16:0/3272174823 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55695122dd10 0x556951230130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:23.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:23.008+0000 7fbb9d5d5740 4 client.17091 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T02:07:23.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:23.008+0000 7fbb9d5d5740 1 -- 172.21.15.16:0/3272174823 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556951110d60 con 0x55695122dd10 2024-09-12T02:07:23.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:23.008+0000 7fbb98f98640 1 --2- 172.21.15.16:0/3272174823 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55695122dd10 0x556951230130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:23.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:23.008+0000 7fbb73fff640 10 client.17091 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:07:23.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:23.008+0000 7fbb98f98640 1 --2- 172.21.15.16:0/3272174823 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55695122dd10 0x556951230130 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b611e9304b840178 server_cookie=b63b51d38807a9c0 in_seq=0 out_seq=0 2024-09-12T02:07:23.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:23.009+0000 7fbb73fff640 1 -- 172.21.15.16:0/3272174823 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x556951110d60 con 0x55695122dd10 2024-09-12T02:07:23.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:23.009+0000 7fbb73fff640 10 client.17091 handle_command_reply: tid=0 2024-09-12T02:07:23.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:23.009+0000 7fbb9d5d5740 1 client.17091 shutdown 2024-09-12T02:07:23.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:23.009+0000 7fbb9d5d5740 1 -- 172.21.15.16:0/3272174823 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5569511ae680 msgr2=0x556951224ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:23.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:23.009+0000 7fbb9d5d5740 1 --2- 172.21.15.16:0/3272174823 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5569511ae680 0x556951224ad0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7fbb840138e0 tx=0x7fbb84003790 comp rx=0 tx=0).stop 2024-09-12T02:07:23.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:23.009+0000 7fbb9d5d5740 1 -- 172.21.15.16:0/3272174823 shutdown_connections 2024-09-12T02:07:23.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:23.009+0000 7fbb9d5d5740 1 --2- 172.21.15.16:0/3272174823 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55695122dd10 0x556951230130 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:23.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:23.009+0000 7fbb9d5d5740 1 --2- 172.21.15.16:0/3272174823 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5569511ae680 0x556951224ad0 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:23.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:23.009+0000 7fbb73fff640 0 client.17091 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:07:23.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:23.009+0000 7fbb9d5d5740 1 --2- 172.21.15.16:0/3272174823 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5569511a3dc0 0x556951220a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:23.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:23.009+0000 7fbb9d5d5740 1 --2- 172.21.15.16:0/3272174823 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5569511a34a0 0x5569512204c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:23.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:23.009+0000 7fbb9d5d5740 1 -- 172.21.15.16:0/3272174823 >> 172.21.15.16:0/3272174823 conn(0x55695119fa30 msgr2=0x55695119eb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:07:23.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:23.009+0000 7fbb9d5d5740 1 -- 172.21.15.16:0/3272174823 shutdown_connections 2024-09-12T02:07:23.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:23.009+0000 7fbb9d5d5740 1 -- 172.21.15.16:0/3272174823 wait complete. 2024-09-12T02:07:23.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:23.009+0000 7fbb9d5d5740 20 client.17091 trim_cache size 0 max 16384 2024-09-12T02:07:23.015 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T02:07:23.015 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-12T02:07:23.015 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": { 2024-09-12T02:07:23.015 INFO:teuthology.orchestra.run.smithi016.stdout: "f01d3f23-7779-4f53-b011-418060ac2d07": { 2024-09-12T02:07:23.015 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-12T02:07:23.015 INFO:teuthology.orchestra.run.smithi016.stdout: "tag": "f01d3f23-7779-4f53-b011-418060ac2d07", 2024-09-12T02:07:23.015 INFO:teuthology.orchestra.run.smithi016.stdout: "options": "recursive,force" 2024-09-12T02:07:23.015 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:07:23.015 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:07:23.015 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T02:07:23.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:23.010+0000 7fbb9b1fc640 1 -- 172.21.15.16:0/2168043942 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fbb7c0a2c00 msgr2=0x7fbb7c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:23.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:23.010+0000 7fbb9b1fc640 1 --2- 172.21.15.16:0/2168043942 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fbb7c0a2c00 0x7fbb7c0a50c0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fbb8c09ba20 tx=0x7fbb8c06aa10 comp rx=0 tx=0).stop 2024-09-12T02:07:23.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:23.010+0000 7fbb9b1fc640 1 -- 172.21.15.16:0/2168043942 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb9410c170 msgr2=0x7fbb9419ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:23.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:23.010+0000 7fbb9b1fc640 1 --2- 172.21.15.16:0/2168043942 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb9410c170 0x7fbb9419ccc0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7fbb8c00eaa0 tx=0x7fbb8c00ef60 comp rx=0 tx=0).stop 2024-09-12T02:07:23.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:23.010+0000 7fbb9b1fc640 1 -- 172.21.15.16:0/2168043942 shutdown_connections 2024-09-12T02:07:23.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:23.010+0000 7fbb9b1fc640 1 --2- 172.21.15.16:0/2168043942 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fbb7c0a2c00 0x7fbb7c0a50c0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:23.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:23.010+0000 7fbb9b1fc640 1 --2- 172.21.15.16:0/2168043942 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb9410c170 0x7fbb9419ccc0 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:23.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:23.010+0000 7fbb9b1fc640 1 --2- 172.21.15.16:0/2168043942 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbb94103660 0x7fbb94198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:23.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:23.010+0000 7fbb9b1fc640 1 --2- 172.21.15.16:0/2168043942 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbb94102d40 0x7fbb941983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:23.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:23.010+0000 7fbb9b1fc640 1 -- 172.21.15.16:0/2168043942 >> 172.21.15.16:0/2168043942 conn(0x7fbb940fc7a0 msgr2=0x7fbb940fd400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:07:23.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:23.010+0000 7fbb9b1fc640 1 -- 172.21.15.16:0/2168043942 shutdown_connections 2024-09-12T02:07:23.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:23.010+0000 7fbb9b1fc640 1 -- 172.21.15.16:0/2168043942 wait complete. 2024-09-12T02:07:23.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:07:23 smithi016 ceph-mon[30947]: scrub queued for path: / 2024-09-12T02:07:23.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:07:23 smithi016 ceph-mon[30947]: scrub summary: idle+waiting paths [/] 2024-09-12T02:07:23.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:07:23 smithi016 ceph-mon[30947]: scrub summary: active paths [/] 2024-09-12T02:07:23.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:07:23 smithi016 ceph-mon[30947]: scrub summary: idle+waiting paths [/] 2024-09-12T02:07:23.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:07:23 smithi184 ceph-mon[35593]: scrub queued for path: / 2024-09-12T02:07:23.628 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:07:23 smithi184 ceph-mon[35593]: scrub summary: idle+waiting paths [/] 2024-09-12T02:07:23.628 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:07:23 smithi184 ceph-mon[35593]: scrub summary: active paths [/] 2024-09-12T02:07:23.628 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:07:23 smithi184 ceph-mon[35593]: scrub summary: idle+waiting paths [/] 2024-09-12T02:07:23.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:07:23 smithi033 ceph-mon[35332]: scrub queued for path: / 2024-09-12T02:07:23.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:07:23 smithi033 ceph-mon[35332]: scrub summary: idle+waiting paths [/] 2024-09-12T02:07:23.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:07:23 smithi033 ceph-mon[35332]: scrub summary: active paths [/] 2024-09-12T02:07:23.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:07:23 smithi033 ceph-mon[35332]: scrub summary: idle+waiting paths [/] 2024-09-12T02:07:23.733 INFO:tasks.cephfs.filesystem:scrub status for tag:f01d3f23-7779-4f53-b011-418060ac2d07 - {'path': '/', 'tag': 'f01d3f23-7779-4f53-b011-418060ac2d07', 'options': 'recursive,force'} 2024-09-12T02:07:24.005 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:07:24.005 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:07:24.006 INFO:tasks.workunit.client.0.smithi016.stdout:127/600: open fd = 3 2024-09-12T02:07:24.006 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:07:24.006 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:07:24.006 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:07:24.006 INFO:tasks.workunit.client.0.smithi016.stdout:128/600: open fd = 3 2024-09-12T02:07:24.006 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:07:24.006 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:07:24.007 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:07:24.007 INFO:tasks.workunit.client.0.smithi016.stdout:129/600: open fd = 3 2024-09-12T02:07:24.007 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:07:24.007 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:07:24.007 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:07:24.007 INFO:tasks.workunit.client.0.smithi016.stdout:130/600: open fd = 3 2024-09-12T02:07:24.007 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:07:24.007 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:07:24.008 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:07:24.008 INFO:tasks.workunit.client.0.smithi016.stdout:131/600: open fd = 3 2024-09-12T02:07:24.008 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:07:24.008 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:07:24.008 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:07:24.008 INFO:tasks.workunit.client.0.smithi016.stdout:132/600: open fd = 3 2024-09-12T02:07:24.008 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:07:24.009 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:07:24.009 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:07:24.009 INFO:tasks.workunit.client.0.smithi016.stdout:133/600: open fd = 3 2024-09-12T02:07:24.009 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:07:24.010 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:07:24.010 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:07:24.010 INFO:tasks.workunit.client.0.smithi016.stdout:134/600: open fd = 3 2024-09-12T02:07:24.010 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:07:24.010 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:07:24.010 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:07:24.010 INFO:tasks.workunit.client.0.smithi016.stdout:135/600: open fd = 3 2024-09-12T02:07:24.010 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:07:24.010 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:07:24.010 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:07:24.010 INFO:tasks.workunit.client.0.smithi016.stdout:136/600: open fd = 3 2024-09-12T02:07:24.010 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:07:24.011 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:07:24.011 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:07:24.011 INFO:tasks.workunit.client.0.smithi016.stdout:137/600: open fd = 3 2024-09-12T02:07:24.011 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:07:24.011 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:07:24.011 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:07:24.011 INFO:tasks.workunit.client.0.smithi016.stdout:138/600: open fd = 3 2024-09-12T02:07:24.011 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:07:24.011 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:07:24.011 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:07:24.012 INFO:tasks.workunit.client.0.smithi016.stdout:139/600: open fd = 3 2024-09-12T02:07:24.012 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:07:24.012 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:07:24.012 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:07:24.012 INFO:tasks.workunit.client.0.smithi016.stdout:140/600: open fd = 3 2024-09-12T02:07:24.012 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:07:24.012 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:07:24.013 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:07:24.013 INFO:tasks.workunit.client.0.smithi016.stdout:141/600: open fd = 3 2024-09-12T02:07:24.013 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:07:24.013 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:07:24.013 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:07:24.013 INFO:tasks.workunit.client.0.smithi016.stdout:142/600: open fd = 3 2024-09-12T02:07:24.013 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:07:24.014 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:07:24.014 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:07:24.014 INFO:tasks.workunit.client.0.smithi016.stdout:143/600: open fd = 3 2024-09-12T02:07:24.014 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:07:24.014 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:07:24.014 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:07:24.014 INFO:tasks.workunit.client.0.smithi016.stdout:144/600: open fd = 3 2024-09-12T02:07:24.014 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:07:24.014 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:07:24.014 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:07:24.014 INFO:tasks.workunit.client.0.smithi016.stdout:145/600: open fd = 3 2024-09-12T02:07:24.014 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:07:24.015 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:07:24.015 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:07:24.015 INFO:tasks.workunit.client.0.smithi016.stdout:146/600: open fd = 3 2024-09-12T02:07:24.015 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:07:24.015 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:07:24.015 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:07:24.015 INFO:tasks.workunit.client.0.smithi016.stdout:147/600: open fd = 3 2024-09-12T02:07:24.015 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:07:24.015 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:07:24.015 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:07:24.015 INFO:tasks.workunit.client.0.smithi016.stdout:148/600: open fd = 3 2024-09-12T02:07:24.015 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:07:24.016 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:07:24.016 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:07:24.016 INFO:tasks.workunit.client.0.smithi016.stdout:149/600: open fd = 3 2024-09-12T02:07:24.016 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:07:24.016 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:07:24.016 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:07:24.016 INFO:tasks.workunit.client.0.smithi016.stdout:150/600: open fd = 3 2024-09-12T02:07:24.016 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:07:24.016 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:07:24.016 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:07:24.017 INFO:tasks.workunit.client.0.smithi016.stdout:151/600: open fd = 3 2024-09-12T02:07:24.017 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:07:24.017 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:07:24.017 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:07:24.017 INFO:tasks.workunit.client.0.smithi016.stdout:152/600: open fd = 3 2024-09-12T02:07:24.017 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:07:24.017 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:07:24.017 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:07:24.018 INFO:tasks.workunit.client.0.smithi016.stdout:153/600: open fd = 3 2024-09-12T02:07:24.018 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:07:24.018 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:07:24.018 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:07:24.018 INFO:tasks.workunit.client.0.smithi016.stdout:154/600: open fd = 3 2024-09-12T02:07:24.018 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:07:24.018 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:07:24.019 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:07:24.019 INFO:tasks.workunit.client.0.smithi016.stdout:155/600: open fd = 3 2024-09-12T02:07:24.019 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:07:24.019 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:07:24.019 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:07:24.019 INFO:tasks.workunit.client.0.smithi016.stdout:156/600: open fd = 3 2024-09-12T02:07:24.019 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:07:24.019 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:07:24.019 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:07:24.019 INFO:tasks.workunit.client.0.smithi016.stdout:157/600: open fd = 3 2024-09-12T02:07:24.020 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:07:24.020 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:07:24.020 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:07:24.020 INFO:tasks.workunit.client.0.smithi016.stdout:158/600: open fd = 3 2024-09-12T02:07:24.020 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:07:24.020 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:07:24.020 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:07:24.020 INFO:tasks.workunit.client.0.smithi016.stdout:159/600: open fd = 3 2024-09-12T02:07:24.021 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:07:24.021 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:07:24.021 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:07:24.021 INFO:tasks.workunit.client.0.smithi016.stdout:160/600: open fd = 3 2024-09-12T02:07:24.021 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:07:24.021 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:07:24.021 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:07:24.022 INFO:tasks.workunit.client.0.smithi016.stdout:161/600: open fd = 3 2024-09-12T02:07:24.022 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:07:24.022 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:07:24.022 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:07:24.022 INFO:tasks.workunit.client.0.smithi016.stdout:162/600: open fd = 3 2024-09-12T02:07:24.022 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:07:24.022 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:07:24.023 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:07:24.023 INFO:tasks.workunit.client.0.smithi016.stdout:163/600: open fd = 3 2024-09-12T02:07:24.023 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:07:24.023 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:07:24.023 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:07:24.023 INFO:tasks.workunit.client.0.smithi016.stdout:164/600: open fd = 3 2024-09-12T02:07:24.023 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:07:24.024 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:07:24.024 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:07:24.024 INFO:tasks.workunit.client.0.smithi016.stdout:165/600: open fd = 3 2024-09-12T02:07:24.024 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:07:24.024 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:07:24.024 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:07:24.024 INFO:tasks.workunit.client.0.smithi016.stdout:166/600: open fd = 3 2024-09-12T02:07:24.024 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:07:24.025 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:07:24.025 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:07:24.025 INFO:tasks.workunit.client.0.smithi016.stdout:167/600: open fd = 3 2024-09-12T02:07:24.025 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:07:24.025 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:07:24.025 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:07:24.025 INFO:tasks.workunit.client.0.smithi016.stdout:168/600: open fd = 3 2024-09-12T02:07:24.026 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:07:24.026 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:07:24.026 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:07:24.026 INFO:tasks.workunit.client.0.smithi016.stdout:169/600: open fd = 3 2024-09-12T02:07:24.026 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:07:24.026 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:07:24.026 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:07:24.027 INFO:tasks.workunit.client.0.smithi016.stdout:170/600: open fd = 3 2024-09-12T02:07:24.027 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:07:24.027 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:07:24.027 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:07:24.027 INFO:tasks.workunit.client.0.smithi016.stdout:171/600: open fd = 3 2024-09-12T02:07:24.027 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:07:24.027 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:07:24.027 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:07:24.028 INFO:tasks.workunit.client.0.smithi016.stdout:172/600: open fd = 3 2024-09-12T02:07:24.028 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:07:24.028 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:07:24.028 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:07:24.028 INFO:tasks.workunit.client.0.smithi016.stdout:173/600: open fd = 3 2024-09-12T02:07:24.028 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:07:24.028 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:07:24.029 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:07:24.029 INFO:tasks.workunit.client.0.smithi016.stdout:174/600: open fd = 3 2024-09-12T02:07:24.029 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:07:24.030 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:07:24.030 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:07:24.030 INFO:tasks.workunit.client.0.smithi016.stdout:175/600: open fd = 3 2024-09-12T02:07:24.030 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:07:24.030 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:07:24.030 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:07:24.030 INFO:tasks.workunit.client.0.smithi016.stdout:176/600: open fd = 3 2024-09-12T02:07:24.030 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:07:24.030 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:07:24.030 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:07:24.030 INFO:tasks.workunit.client.0.smithi016.stdout:177/600: open fd = 3 2024-09-12T02:07:24.030 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:07:24.030 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:07:24.031 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:07:24.031 INFO:tasks.workunit.client.0.smithi016.stdout:178/600: open fd = 3 2024-09-12T02:07:24.031 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:07:24.031 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:07:24.031 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:07:24.031 INFO:tasks.workunit.client.0.smithi016.stdout:179/600: open fd = 3 2024-09-12T02:07:24.031 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:07:24.032 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:07:24.032 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:07:24.032 INFO:tasks.workunit.client.0.smithi016.stdout:180/600: open fd = 3 2024-09-12T02:07:24.032 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:07:24.032 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:07:24.032 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:07:24.032 INFO:tasks.workunit.client.0.smithi016.stdout:181/600: open fd = 3 2024-09-12T02:07:24.032 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:07:24.032 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:07:24.032 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:07:24.032 INFO:tasks.workunit.client.0.smithi016.stdout:182/600: open fd = 3 2024-09-12T02:07:24.033 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:07:24.033 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:07:24.033 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:07:24.033 INFO:tasks.workunit.client.0.smithi016.stdout:183/600: open fd = 3 2024-09-12T02:07:24.033 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:07:24.033 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:07:24.033 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:07:24.033 INFO:tasks.workunit.client.0.smithi016.stdout:184/600: open fd = 3 2024-09-12T02:07:24.033 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:07:24.033 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:07:24.033 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:07:24.033 INFO:tasks.workunit.client.0.smithi016.stdout:185/600: open fd = 3 2024-09-12T02:07:24.033 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:07:24.033 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:07:24.034 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:07:24.034 INFO:tasks.workunit.client.0.smithi016.stdout:186/600: open fd = 3 2024-09-12T02:07:24.034 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:07:24.034 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:07:24.034 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:07:24.034 INFO:tasks.workunit.client.0.smithi016.stdout:187/600: open fd = 3 2024-09-12T02:07:24.034 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:07:24.034 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:07:24.034 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:07:24.034 INFO:tasks.workunit.client.0.smithi016.stdout:188/600: open fd = 3 2024-09-12T02:07:24.034 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:07:24.034 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:07:24.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:07:24 smithi016 ceph-mon[30947]: pgmap v499: 97 pgs: 97 active+clean; 3.0 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-12T02:07:24.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:07:24 smithi184 ceph-mon[35593]: pgmap v499: 97 pgs: 97 active+clean; 3.0 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-12T02:07:24.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:07:24 smithi033 ceph-mon[35332]: pgmap v499: 97 pgs: 97 active+clean; 3.0 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-12T02:07:26.261 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.260+0000 7fddca4c9640 1 -- 172.21.15.16:0/827546147 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fddc4107840 msgr2=0x7fddc4109cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:26.261 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.260+0000 7fddca4c9640 1 --2- 172.21.15.16:0/827546147 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fddc4107840 0x7fddc4109cb0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7fddbc009ca0 tx=0x7fddbc02a8c0 comp rx=0 tx=0).stop 2024-09-12T02:07:26.261 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.260+0000 7fddca4c9640 1 -- 172.21.15.16:0/827546147 shutdown_connections 2024-09-12T02:07:26.261 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.260+0000 7fddca4c9640 1 --2- 172.21.15.16:0/827546147 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fddc4107840 0x7fddc4109cb0 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:26.261 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.260+0000 7fddca4c9640 1 --2- 172.21.15.16:0/827546147 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fddc4104f10 0x7fddc4107300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:26.261 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.260+0000 7fddca4c9640 1 --2- 172.21.15.16:0/827546147 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fddc41025e0 0x7fddc41049d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:26.261 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.260+0000 7fddca4c9640 1 -- 172.21.15.16:0/827546147 >> 172.21.15.16:0/827546147 conn(0x7fddc40fc1e0 msgr2=0x7fddc40fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:07:26.261 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.260+0000 7fddca4c9640 1 -- 172.21.15.16:0/827546147 shutdown_connections 2024-09-12T02:07:26.261 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.260+0000 7fddca4c9640 1 -- 172.21.15.16:0/827546147 wait complete. 2024-09-12T02:07:26.261 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.261+0000 7fddca4c9640 1 Processor -- start 2024-09-12T02:07:26.261 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.261+0000 7fddca4c9640 1 -- start start 2024-09-12T02:07:26.263 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.261+0000 7fddca4c9640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fddc41025e0 0x7fddc4197e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:26.263 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.261+0000 7fddca4c9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fddc4104f10 0x7fddc4198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:26.263 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.261+0000 7fddca4c9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fddc4107840 0x7fddc419f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:26.264 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.261+0000 7fddca4c9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fddc410d820 con 0x7fddc4107840 2024-09-12T02:07:26.264 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.261+0000 7fddca4c9640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fddc410d6a0 con 0x7fddc4104f10 2024-09-12T02:07:26.264 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.261+0000 7fddca4c9640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fddc410d9a0 con 0x7fddc41025e0 2024-09-12T02:07:26.264 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.261+0000 7fddc8a66640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fddc4107840 0x7fddc419f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:26.264 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.261+0000 7fddc8a66640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fddc4107840 0x7fddc419f3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56834/0 (socket says 172.21.15.16:56834) 2024-09-12T02:07:26.264 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.261+0000 7fddc8a66640 1 -- 172.21.15.16:0/1077094903 learned_addr learned my addr 172.21.15.16:0/1077094903 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:07:26.264 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.261+0000 7fddc37fe640 1 --2- 172.21.15.16:0/1077094903 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fddc4104f10 0x7fddc4198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:26.264 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.261+0000 7fddc3fff640 1 --2- 172.21.15.16:0/1077094903 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fddc41025e0 0x7fddc4197e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:26.264 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.262+0000 7fddc8a66640 1 -- 172.21.15.16:0/1077094903 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fddc41025e0 msgr2=0x7fddc4197e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:26.264 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.262+0000 7fddc8a66640 1 --2- 172.21.15.16:0/1077094903 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fddc41025e0 0x7fddc4197e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:26.264 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.262+0000 7fddc8a66640 1 -- 172.21.15.16:0/1077094903 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fddc4104f10 msgr2=0x7fddc4198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:26.264 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.262+0000 7fddc8a66640 1 --2- 172.21.15.16:0/1077094903 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fddc4104f10 0x7fddc4198360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:26.264 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.262+0000 7fddc8a66640 1 -- 172.21.15.16:0/1077094903 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddc419fa40 con 0x7fddc4107840 2024-09-12T02:07:26.265 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.262+0000 7fddc37fe640 1 --2- 172.21.15.16:0/1077094903 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fddc4104f10 0x7fddc4198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:07:26.265 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.262+0000 7fddc8a66640 1 --2- 172.21.15.16:0/1077094903 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fddc4107840 0x7fddc419f3d0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7fddbc006010 tx=0x7fddbc004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:07:26.265 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.262+0000 7fddc17fa640 1 -- 172.21.15.16:0/1077094903 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddbc003b70 con 0x7fddc4107840 2024-09-12T02:07:26.265 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.262+0000 7fddc17fa640 1 -- 172.21.15.16:0/1077094903 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fddbc003d10 con 0x7fddc4107840 2024-09-12T02:07:26.265 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.262+0000 7fddc17fa640 1 -- 172.21.15.16:0/1077094903 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddbc02fc40 con 0x7fddc4107840 2024-09-12T02:07:26.265 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.262+0000 7fddca4c9640 1 -- 172.21.15.16:0/1077094903 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddc419fcd0 con 0x7fddc4107840 2024-09-12T02:07:26.265 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.262+0000 7fddca4c9640 1 -- 172.21.15.16:0/1077094903 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddc41a0210 con 0x7fddc4107840 2024-09-12T02:07:26.270 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.264+0000 7fddc17fa640 1 -- 172.21.15.16:0/1077094903 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fddbc004050 con 0x7fddc4107840 2024-09-12T02:07:26.270 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.265+0000 7fddcd262740 1 Processor -- start 2024-09-12T02:07:26.270 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.265+0000 7fddcd262740 1 -- start start 2024-09-12T02:07:26.270 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.265+0000 7fddcd262740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562d2d2459a0 0x562d2d245da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:26.270 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.265+0000 7fddcd262740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562d2d2462e0 0x562d2d24ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:26.270 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.265+0000 7fddcd262740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562d2d24b370 0x562d2d24d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:26.270 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.265+0000 7fddcd262740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x562d2d1a92f0 con 0x562d2d2459a0 2024-09-12T02:07:26.271 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.265+0000 7fddcd262740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x562d2d151270 con 0x562d2d24b370 2024-09-12T02:07:26.271 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.265+0000 7fddcd262740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x562d2d1a52b0 con 0x562d2d2462e0 2024-09-12T02:07:26.271 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.265+0000 7fddc37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562d2d2459a0 0x562d2d245da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:26.271 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.265+0000 7fddc37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562d2d2459a0 0x562d2d245da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56838/0 (socket says 172.21.15.16:56838) 2024-09-12T02:07:26.271 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.265+0000 7fddc37fe640 1 -- 172.21.15.16:0/1051087815 learned_addr learned my addr 172.21.15.16:0/1051087815 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:07:26.271 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.265+0000 7fddc17fa640 1 --2- 172.21.15.16:0/1077094903 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fddac0a2cb0 0x7fddac0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:26.271 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.265+0000 7fddc37fe640 1 -- 172.21.15.16:0/1051087815 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562d2d2462e0 msgr2=0x562d2d24ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:26.271 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.265+0000 7fddc3fff640 1 --2- 172.21.15.16:0/1051087815 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562d2d2462e0 0x562d2d24ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:26.271 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.265+0000 7fddc37fe640 1 --2- 172.21.15.16:0/1051087815 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562d2d2462e0 0x562d2d24ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:26.271 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.265+0000 7fddc37fe640 1 -- 172.21.15.16:0/1051087815 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562d2d24b370 msgr2=0x562d2d24d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:26.271 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.265+0000 7fddc37fe640 1 --2- 172.21.15.16:0/1051087815 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562d2d24b370 0x562d2d24d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:26.271 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.265+0000 7fddc37fe640 1 -- 172.21.15.16:0/1051087815 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d2d150340 con 0x562d2d2459a0 2024-09-12T02:07:26.272 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.266+0000 7fddc8a66640 1 --2- 172.21.15.16:0/1077094903 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fddac0a2cb0 0x7fddac0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:26.272 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.266+0000 7fddc17fa640 1 -- 172.21.15.16:0/1077094903 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fddbc0f0320 con 0x7fddc4107840 2024-09-12T02:07:26.272 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.266+0000 7fddc37fe640 1 --2- 172.21.15.16:0/1051087815 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562d2d2459a0 0x562d2d245da0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fddb8009d60 tx=0x7fddb80137d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49f037eacc0e7bb9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:07:26.272 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.266+0000 7fddaa7fc640 1 -- 172.21.15.16:0/1051087815 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddb8009e50 con 0x562d2d2459a0 2024-09-12T02:07:26.272 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.266+0000 7fddaa7fc640 1 -- 172.21.15.16:0/1051087815 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fddb8017030 con 0x562d2d2459a0 2024-09-12T02:07:26.272 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.266+0000 7fddaa7fc640 1 -- 172.21.15.16:0/1051087815 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddb8011710 con 0x562d2d2459a0 2024-09-12T02:07:26.272 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.267+0000 7fddcd262740 1 -- 172.21.15.16:0/1051087815 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562d2d2459a0 msgr2=0x562d2d245da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:26.272 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.267+0000 7fddc8a66640 1 --2- 172.21.15.16:0/1077094903 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fddac0a2cb0 0x7fddac0a5170 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fddbc0c5360 tx=0x7fddbc0877f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:07:26.272 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.267+0000 7fddcd262740 1 --2- 172.21.15.16:0/1051087815 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562d2d2459a0 0x562d2d245da0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fddb8009d60 tx=0x7fddb80137d0 comp rx=0 tx=0).stop 2024-09-12T02:07:26.272 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.267+0000 7fddcd262740 1 -- 172.21.15.16:0/1051087815 shutdown_connections 2024-09-12T02:07:26.272 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.267+0000 7fddcd262740 1 --2- 172.21.15.16:0/1051087815 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562d2d24b370 0x562d2d24d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:26.272 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.267+0000 7fddcd262740 1 --2- 172.21.15.16:0/1051087815 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562d2d2462e0 0x562d2d24ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:26.273 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.267+0000 7fddcd262740 1 --2- 172.21.15.16:0/1051087815 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562d2d2459a0 0x562d2d245da0 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:26.273 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.267+0000 7fddcd262740 1 -- 172.21.15.16:0/1051087815 >> 172.21.15.16:0/1051087815 conn(0x562d2d241010 msgr2=0x562d2d241410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:07:26.273 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.267+0000 7fddcd262740 1 -- 172.21.15.16:0/1051087815 shutdown_connections 2024-09-12T02:07:26.273 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.267+0000 7fddcd262740 1 -- 172.21.15.16:0/1051087815 wait complete. 2024-09-12T02:07:26.273 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.267+0000 7fddcd262740 1 -- start start 2024-09-12T02:07:26.273 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.267+0000 7fddcd262740 1 Processor -- start 2024-09-12T02:07:26.273 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.268+0000 7fddcd262740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562d2d2459a0 0x562d2d24fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:26.273 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.268+0000 7fddcd262740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562d2d2462e0 0x562d2d250130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:26.273 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.268+0000 7fddcd262740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562d2d24b370 0x562d2d257170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:26.273 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.268+0000 7fddcd262740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x562d2d24e340 con 0x562d2d24b370 2024-09-12T02:07:26.273 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.268+0000 7fddcd262740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x562d2d132290 con 0x562d2d2462e0 2024-09-12T02:07:26.273 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.268+0000 7fddcd262740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x562d2d24e4c0 con 0x562d2d2459a0 2024-09-12T02:07:26.273 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.268+0000 7fddcd262740 10 client.0 _ll_register_callbacks cb 0x562d2d163140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:07:26.274 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.268+0000 7fddc37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562d2d24b370 0x562d2d257170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:26.274 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.268+0000 7fddc37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562d2d24b370 0x562d2d257170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56844/0 (socket says 172.21.15.16:56844) 2024-09-12T02:07:26.274 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.268+0000 7fddc37fe640 1 -- 172.21.15.16:0/879628046 learned_addr learned my addr 172.21.15.16:0/879628046 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:07:26.274 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.268+0000 7fddc37fe640 1 --2- 172.21.15.16:0/879628046 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562d2d2459a0 0x562d2d24fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:26.274 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.268+0000 7fddc3fff640 1 --2- 172.21.15.16:0/879628046 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562d2d2462e0 0x562d2d250130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:26.274 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.268+0000 7fddc37fe640 1 -- 172.21.15.16:0/879628046 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562d2d2459a0 msgr2=0x562d2d24fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:26.274 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.268+0000 7fddc37fe640 1 --2- 172.21.15.16:0/879628046 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562d2d2459a0 0x562d2d24fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:26.274 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.268+0000 7fddc37fe640 1 -- 172.21.15.16:0/879628046 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562d2d2462e0 msgr2=0x562d2d250130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:26.275 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.268+0000 7fddc37fe640 1 --2- 172.21.15.16:0/879628046 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562d2d2462e0 0x562d2d250130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:26.275 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.268+0000 7fddc37fe640 1 -- 172.21.15.16:0/879628046 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d2cebc210 con 0x562d2d24b370 2024-09-12T02:07:26.275 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.268+0000 7fddc37fe640 1 --2- 172.21.15.16:0/879628046 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562d2d24b370 0x562d2d257170 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fddb8009fd0 tx=0x7fddb8015040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3703a341c398b2cf server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:07:26.275 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.268+0000 7fdda37fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:07:26.275 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.269+0000 7fdda37fe640 1 -- 172.21.15.16:0/879628046 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddb80130d0 con 0x562d2d24b370 2024-09-12T02:07:26.275 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.269+0000 7fdda37fe640 1 -- 172.21.15.16:0/879628046 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fddb8011510 con 0x562d2d24b370 2024-09-12T02:07:26.275 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.269+0000 7fddcd262740 1 -- 172.21.15.16:0/879628046 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fddb8017030 con 0x562d2d24b370 2024-09-12T02:07:26.275 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.269+0000 7fdda37fe640 1 -- 172.21.15.16:0/879628046 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fddb8033050 con 0x562d2d24b370 2024-09-12T02:07:26.275 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.269+0000 7fddcd262740 10 client.17103 fetch_fsmap learned FSMap version 25 2024-09-12T02:07:26.275 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.269+0000 7fddcd262740 1 -- 172.21.15.16:0/879628046 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x562d2cf88620 con 0x562d2d24b370 2024-09-12T02:07:26.275 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.269+0000 7fdda37fe640 1 -- 172.21.15.16:0/879628046 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fddb8017030 con 0x562d2d24b370 2024-09-12T02:07:26.275 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.269+0000 7fdda37fe640 1 -- 172.21.15.16:0/879628046 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fddb801c5e0 con 0x562d2d24b370 2024-09-12T02:07:26.276 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.269+0000 7fddcd262740 10 client.17103 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:07:26.276 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.269+0000 7fddcd262740 10 client.17103 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:07:26.276 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.269+0000 7fddcd262740 20 client.17103 populate_metadata read hostname 'smithi016' 2024-09-12T02:07:26.276 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.269+0000 7fddcd262740 1 --2- 172.21.15.16:0/879628046 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x562d2d25b5d0 0x562d2d25da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:26.276 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.269+0000 7fddcd262740 4 client.17103 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:07:26.276 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.269+0000 7fddcd262740 1 -- 172.21.15.16:0/879628046 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562d2d1b6a20 con 0x562d2d25b5d0 2024-09-12T02:07:26.276 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.270+0000 7fddc3fff640 1 --2- 172.21.15.16:0/879628046 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x562d2d25b5d0 0x562d2d25da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:26.276 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.270+0000 7fdda37fe640 10 client.17103 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:07:26.276 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.270+0000 7fddc3fff640 1 --2- 172.21.15.16:0/879628046 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x562d2d25b5d0 0x562d2d25da30 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d806991e52464c65 server_cookie=faf66759a220c194 in_seq=0 out_seq=0 2024-09-12T02:07:26.277 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.271+0000 7fdda37fe640 1 -- 172.21.15.16:0/879628046 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562d2d1b6a20 con 0x562d2d25b5d0 2024-09-12T02:07:26.277 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.271+0000 7fdda37fe640 10 client.17103 handle_command_reply: tid=0 2024-09-12T02:07:26.277 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.271+0000 7fddcd262740 1 client.17103 shutdown 2024-09-12T02:07:26.278 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.271+0000 7fddcd262740 1 -- 172.21.15.16:0/879628046 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562d2d24b370 msgr2=0x562d2d257170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:26.278 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.271+0000 7fddcd262740 1 --2- 172.21.15.16:0/879628046 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562d2d24b370 0x562d2d257170 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fddb8009fd0 tx=0x7fddb8015040 comp rx=0 tx=0).stop 2024-09-12T02:07:26.278 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.271+0000 7fddcd262740 1 -- 172.21.15.16:0/879628046 shutdown_connections 2024-09-12T02:07:26.278 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.271+0000 7fddcd262740 1 --2- 172.21.15.16:0/879628046 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x562d2d25b5d0 0x562d2d25da30 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:26.278 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.271+0000 7fddcd262740 1 --2- 172.21.15.16:0/879628046 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562d2d24b370 0x562d2d257170 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:26.278 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.271+0000 7fddcd262740 1 --2- 172.21.15.16:0/879628046 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562d2d2462e0 0x562d2d250130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:26.278 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.271+0000 7fdda37fe640 0 client.17103 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:07:26.278 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.271+0000 7fddcd262740 1 --2- 172.21.15.16:0/879628046 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562d2d2459a0 0x562d2d24fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:26.278 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.271+0000 7fddcd262740 1 -- 172.21.15.16:0/879628046 >> 172.21.15.16:0/879628046 conn(0x562d2d241010 msgr2=0x562d2d242820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:07:26.278 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.271+0000 7fddcd262740 1 -- 172.21.15.16:0/879628046 shutdown_connections 2024-09-12T02:07:26.278 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.271+0000 7fddcd262740 1 -- 172.21.15.16:0/879628046 wait complete. 2024-09-12T02:07:26.278 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.271+0000 7fddcd262740 20 client.17103 trim_cache size 0 max 16384 2024-09-12T02:07:26.286 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.286+0000 7fddcd262740 1 Processor -- start 2024-09-12T02:07:26.287 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.286+0000 7fddcd262740 1 -- start start 2024-09-12T02:07:26.287 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.286+0000 7fddcd262740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562d2d2459a0 0x562d2d2400b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:26.287 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.286+0000 7fddcd262740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562d2d2462e0 0x562d2d2570e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:26.287 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.286+0000 7fddcd262740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562d2d24b370 0x562d2d2622f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:26.287 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.286+0000 7fddcd262740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fddc41ab720 con 0x562d2d2459a0 2024-09-12T02:07:26.287 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.286+0000 7fddcd262740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fddc41a8ff0 con 0x562d2d2462e0 2024-09-12T02:07:26.287 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.286+0000 7fddcd262740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fddc41a6e60 con 0x562d2d24b370 2024-09-12T02:07:26.287 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.286+0000 7fddc37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562d2d2459a0 0x562d2d2400b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:26.287 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.286+0000 7fddc37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562d2d2459a0 0x562d2d2400b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56854/0 (socket says 172.21.15.16:56854) 2024-09-12T02:07:26.287 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.286+0000 7fddc37fe640 1 -- 172.21.15.16:0/3384403258 learned_addr learned my addr 172.21.15.16:0/3384403258 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:07:26.287 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.286+0000 7fddc3fff640 1 --2- 172.21.15.16:0/3384403258 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562d2d2462e0 0x562d2d2570e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:26.287 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.286+0000 7fddc37fe640 1 -- 172.21.15.16:0/3384403258 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562d2d24b370 msgr2=0x562d2d2622f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:26.288 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.286+0000 7fddc37fe640 1 --2- 172.21.15.16:0/3384403258 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562d2d24b370 0x562d2d2622f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:26.288 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.286+0000 7fddc37fe640 1 -- 172.21.15.16:0/3384403258 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562d2d2462e0 msgr2=0x562d2d2570e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:26.288 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.286+0000 7fddc37fe640 1 --2- 172.21.15.16:0/3384403258 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562d2d2462e0 0x562d2d2570e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:26.288 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.286+0000 7fddc37fe640 1 -- 172.21.15.16:0/3384403258 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d2d2467c0 con 0x562d2d2459a0 2024-09-12T02:07:26.288 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.287+0000 7fddc37fe640 1 --2- 172.21.15.16:0/3384403258 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562d2d2459a0 0x562d2d2400b0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fddb8000c00 tx=0x7fddb800fa80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c77c583c4b1b9274 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:07:26.288 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.287+0000 7fdda9ffb640 1 -- 172.21.15.16:0/3384403258 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddb8030020 con 0x562d2d2459a0 2024-09-12T02:07:26.288 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.287+0000 7fdda9ffb640 1 -- 172.21.15.16:0/3384403258 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fddb8021070 con 0x562d2d2459a0 2024-09-12T02:07:26.288 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.287+0000 7fdda9ffb640 1 -- 172.21.15.16:0/3384403258 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddb8005040 con 0x562d2d2459a0 2024-09-12T02:07:26.292 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.287+0000 7fddcd262740 1 -- 172.21.15.16:0/3384403258 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562d2d2459a0 msgr2=0x562d2d2400b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:26.292 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.287+0000 7fddcd262740 1 --2- 172.21.15.16:0/3384403258 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562d2d2459a0 0x562d2d2400b0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fddb8000c00 tx=0x7fddb800fa80 comp rx=0 tx=0).stop 2024-09-12T02:07:26.292 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.287+0000 7fddcd262740 1 -- 172.21.15.16:0/3384403258 shutdown_connections 2024-09-12T02:07:26.292 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.287+0000 7fddcd262740 1 --2- 172.21.15.16:0/3384403258 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562d2d24b370 0x562d2d2622f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:26.292 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.287+0000 7fddcd262740 1 --2- 172.21.15.16:0/3384403258 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562d2d2462e0 0x562d2d2570e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:26.292 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.287+0000 7fddcd262740 1 --2- 172.21.15.16:0/3384403258 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562d2d2459a0 0x562d2d2400b0 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:26.292 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.287+0000 7fddcd262740 1 -- 172.21.15.16:0/3384403258 >> 172.21.15.16:0/3384403258 conn(0x562d2d241010 msgr2=0x562d2d23f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:07:26.292 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.287+0000 7fddcd262740 1 -- 172.21.15.16:0/3384403258 shutdown_connections 2024-09-12T02:07:26.292 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.287+0000 7fddcd262740 1 -- 172.21.15.16:0/3384403258 wait complete. 2024-09-12T02:07:26.292 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.288+0000 7fddcd262740 1 -- start start 2024-09-12T02:07:26.292 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.288+0000 7fddcd262740 1 Processor -- start 2024-09-12T02:07:26.293 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.288+0000 7fddcd262740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562d2d2459a0 0x562d2d254690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:26.293 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.288+0000 7fddcd262740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562d2d2462e0 0x562d2d254bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:26.293 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.288+0000 7fddcd262740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562d2d24b370 0x562d2d255b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:26.293 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.288+0000 7fddcd262740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fddc41a74b0 con 0x562d2d2459a0 2024-09-12T02:07:26.293 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.288+0000 7fddcd262740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fddc41a8920 con 0x562d2d24b370 2024-09-12T02:07:26.293 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.288+0000 7fddcd262740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fddc41a7b80 con 0x562d2d2462e0 2024-09-12T02:07:26.293 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.288+0000 7fddcd262740 10 client.0 _ll_register_callbacks cb 0x7fddb8009c30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:07:26.293 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.288+0000 7fddc37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562d2d2459a0 0x562d2d254690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:26.293 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.288+0000 7fddc37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562d2d2459a0 0x562d2d254690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56860/0 (socket says 172.21.15.16:56860) 2024-09-12T02:07:26.294 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.288+0000 7fddc37fe640 1 -- 172.21.15.16:0/695858453 learned_addr learned my addr 172.21.15.16:0/695858453 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:07:26.294 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.289+0000 7fddc37fe640 1 --2- 172.21.15.16:0/695858453 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562d2d24b370 0x562d2d255b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:26.294 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.289+0000 7fddc3fff640 1 --2- 172.21.15.16:0/695858453 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562d2d2462e0 0x562d2d254bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:26.294 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.289+0000 7fddc37fe640 1 -- 172.21.15.16:0/695858453 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562d2d2462e0 msgr2=0x562d2d254bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:26.294 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.289+0000 7fddc37fe640 1 --2- 172.21.15.16:0/695858453 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562d2d2462e0 0x562d2d254bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:26.294 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.289+0000 7fddc37fe640 1 -- 172.21.15.16:0/695858453 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562d2d24b370 msgr2=0x562d2d255b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:26.294 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.289+0000 7fddc37fe640 1 --2- 172.21.15.16:0/695858453 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562d2d24b370 0x562d2d255b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:26.294 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.289+0000 7fddc37fe640 1 -- 172.21.15.16:0/695858453 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d2d256180 con 0x562d2d2459a0 2024-09-12T02:07:26.294 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.289+0000 7fddc3fff640 1 --2- 172.21.15.16:0/695858453 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562d2d2462e0 0x562d2d254bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:07:26.294 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.289+0000 7fddc37fe640 1 --2- 172.21.15.16:0/695858453 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562d2d2459a0 0x562d2d254690 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fddb8009d60 tx=0x7fddb8013330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15802ee14f0b4964 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:07:26.294 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.289+0000 7fdda37fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:07:26.294 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.289+0000 7fdda37fe640 1 -- 172.21.15.16:0/695858453 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddb8030070 con 0x562d2d2459a0 2024-09-12T02:07:26.294 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.289+0000 7fdda37fe640 1 -- 172.21.15.16:0/695858453 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fddb803c070 con 0x562d2d2459a0 2024-09-12T02:07:26.294 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.289+0000 7fddcd262740 1 -- 172.21.15.16:0/695858453 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fddb8021070 con 0x562d2d2459a0 2024-09-12T02:07:26.295 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.289+0000 7fdda37fe640 1 -- 172.21.15.16:0/695858453 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fddb801cbd0 con 0x562d2d2459a0 2024-09-12T02:07:26.295 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.289+0000 7fddcd262740 10 client.17109 fetch_fsmap learned FSMap version 25 2024-09-12T02:07:26.295 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.289+0000 7fddcd262740 1 -- 172.21.15.16:0/695858453 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x562d2d2c62d0 con 0x562d2d2459a0 2024-09-12T02:07:26.295 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.289+0000 7fdda37fe640 1 -- 172.21.15.16:0/695858453 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fddb8021070 con 0x562d2d2459a0 2024-09-12T02:07:26.295 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.290+0000 7fdda37fe640 1 -- 172.21.15.16:0/695858453 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fddb803f030 con 0x562d2d2459a0 2024-09-12T02:07:26.295 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.290+0000 7fddcd262740 10 client.17109 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:07:26.295 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.290+0000 7fddcd262740 10 client.17109 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:07:26.295 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.290+0000 7fddcd262740 20 client.17109 populate_metadata read hostname 'smithi016' 2024-09-12T02:07:26.295 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.290+0000 7fddcd262740 1 --2- 172.21.15.16:0/695858453 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x562d2d2c6af0 0x562d2d2c8f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:26.295 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.290+0000 7fddcd262740 4 client.17109 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:07:26.295 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.290+0000 7fddcd262740 1 -- 172.21.15.16:0/695858453 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562d2d1b0d70 con 0x562d2d2c6af0 2024-09-12T02:07:26.295 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.290+0000 7fddc3fff640 1 --2- 172.21.15.16:0/695858453 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x562d2d2c6af0 0x562d2d2c8f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:26.295 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.290+0000 7fddc3fff640 1 --2- 172.21.15.16:0/695858453 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x562d2d2c6af0 0x562d2d2c8f10 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b5e1776ef34fab6 server_cookie=7b3736a55eee42f9 in_seq=0 out_seq=0 2024-09-12T02:07:26.296 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.290+0000 7fdda37fe640 10 client.17109 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:07:26.304 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.303+0000 7fdda37fe640 1 -- 172.21.15.16:0/695858453 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562d2d1b0d70 con 0x562d2d2c6af0 2024-09-12T02:07:26.304 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.303+0000 7fdda37fe640 10 client.17109 handle_command_reply: tid=0 2024-09-12T02:07:26.304 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.303+0000 7fddcd262740 1 client.17109 shutdown 2024-09-12T02:07:26.304 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.303+0000 7fddcd262740 1 -- 172.21.15.16:0/695858453 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562d2d2459a0 msgr2=0x562d2d254690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:26.304 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.303+0000 7fddcd262740 1 --2- 172.21.15.16:0/695858453 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562d2d2459a0 0x562d2d254690 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fddb8009d60 tx=0x7fddb8013330 comp rx=0 tx=0).stop 2024-09-12T02:07:26.304 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.303+0000 7fddcd262740 1 -- 172.21.15.16:0/695858453 shutdown_connections 2024-09-12T02:07:26.304 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.303+0000 7fddcd262740 1 --2- 172.21.15.16:0/695858453 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x562d2d2c6af0 0x562d2d2c8f10 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:26.304 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.303+0000 7fddcd262740 1 --2- 172.21.15.16:0/695858453 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562d2d24b370 0x562d2d255b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:26.304 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.303+0000 7fddcd262740 1 --2- 172.21.15.16:0/695858453 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562d2d2462e0 0x562d2d254bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:26.304 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.303+0000 7fddcd262740 1 --2- 172.21.15.16:0/695858453 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562d2d2459a0 0x562d2d254690 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:26.304 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.303+0000 7fdda37fe640 0 client.17109 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:07:26.305 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.303+0000 7fddcd262740 1 -- 172.21.15.16:0/695858453 >> 172.21.15.16:0/695858453 conn(0x562d2d241010 msgr2=0x562d2d2401c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:07:26.305 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.303+0000 7fddcd262740 1 -- 172.21.15.16:0/695858453 shutdown_connections 2024-09-12T02:07:26.305 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.304+0000 7fddcd262740 1 -- 172.21.15.16:0/695858453 wait complete. 2024-09-12T02:07:26.305 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.304+0000 7fddcd262740 20 client.17109 trim_cache size 0 max 16384 2024-09-12T02:07:26.305 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-09-12T02:07:26.305 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-09-12T02:07:26.305 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-09-12T02:07:26.305 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-09-12T02:07:26.306 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.304+0000 7fddca4c9640 1 -- 172.21.15.16:0/1077094903 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fddac0a2cb0 msgr2=0x7fddac0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:26.306 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.304+0000 7fddca4c9640 1 --2- 172.21.15.16:0/1077094903 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fddac0a2cb0 0x7fddac0a5170 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fddbc0c5360 tx=0x7fddbc0877f0 comp rx=0 tx=0).stop 2024-09-12T02:07:26.306 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.304+0000 7fddca4c9640 1 -- 172.21.15.16:0/1077094903 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fddc4107840 msgr2=0x7fddc419f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:26.306 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.304+0000 7fddca4c9640 1 --2- 172.21.15.16:0/1077094903 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fddc4107840 0x7fddc419f3d0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7fddbc006010 tx=0x7fddbc004ab0 comp rx=0 tx=0).stop 2024-09-12T02:07:26.306 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.304+0000 7fddca4c9640 1 -- 172.21.15.16:0/1077094903 shutdown_connections 2024-09-12T02:07:26.306 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.304+0000 7fddca4c9640 1 --2- 172.21.15.16:0/1077094903 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fddac0a2cb0 0x7fddac0a5170 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:26.307 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.304+0000 7fddca4c9640 1 --2- 172.21.15.16:0/1077094903 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fddc4107840 0x7fddc419f3d0 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:26.307 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.304+0000 7fddca4c9640 1 --2- 172.21.15.16:0/1077094903 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fddc4104f10 0x7fddc4198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:26.307 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.304+0000 7fddca4c9640 1 --2- 172.21.15.16:0/1077094903 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fddc41025e0 0x7fddc4197e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:26.307 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.304+0000 7fddca4c9640 1 -- 172.21.15.16:0/1077094903 >> 172.21.15.16:0/1077094903 conn(0x7fddc40fc1e0 msgr2=0x7fddc4102e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:07:26.307 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.304+0000 7fddca4c9640 1 -- 172.21.15.16:0/1077094903 shutdown_connections 2024-09-12T02:07:26.307 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:26.304+0000 7fddca4c9640 1 -- 172.21.15.16:0/1077094903 wait complete. 2024-09-12T02:07:26.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:07:26 smithi184 ceph-mon[35593]: pgmap v500: 97 pgs: 97 active+clean; 3.0 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-09-12T02:07:26.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:07:26 smithi033 ceph-mon[35332]: pgmap v500: 97 pgs: 97 active+clean; 3.0 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-09-12T02:07:26.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:07:26 smithi016 ceph-mon[30947]: pgmap v500: 97 pgs: 97 active+clean; 3.0 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-09-12T02:07:27.527 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:07:27 smithi016 ceph-mon[30947]: scrub summary: idle 2024-09-12T02:07:27.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:07:27 smithi184 ceph-mon[35593]: scrub summary: idle 2024-09-12T02:07:27.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:07:27 smithi033 ceph-mon[35332]: scrub summary: idle 2024-09-12T02:07:28.536 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:07:28 smithi016 ceph-mon[30947]: pgmap v501: 97 pgs: 97 active+clean; 3.0 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-09-12T02:07:28.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:07:28 smithi184 ceph-mon[35593]: pgmap v501: 97 pgs: 97 active+clean; 3.0 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-09-12T02:07:28.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:07:28 smithi033 ceph-mon[35332]: pgmap v501: 97 pgs: 97 active+clean; 3.0 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-09-12T02:07:30.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:07:30 smithi184 ceph-mon[35593]: pgmap v502: 97 pgs: 97 active+clean; 3.0 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T02:07:30.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:07:30 smithi033 ceph-mon[35332]: pgmap v502: 97 pgs: 97 active+clean; 3.0 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T02:07:30.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:07:30 smithi016 ceph-mon[30947]: pgmap v502: 97 pgs: 97 active+clean; 3.0 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T02:07:31.080 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 20.1 2024-09-12T02:07:31.080 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:07:31.356 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:07:31.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:31.800+0000 7fae284b7640 1 -- 172.21.15.16:0/3052465365 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fae201036e0 msgr2=0x7fae20107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:31.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:31.800+0000 7fae284b7640 1 --2- 172.21.15.16:0/3052465365 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fae201036e0 0x7fae20107b90 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fae14009930 tx=0x7fae1402a420 comp rx=0 tx=0).stop 2024-09-12T02:07:31.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:31.800+0000 7fae284b7640 1 -- 172.21.15.16:0/3052465365 shutdown_connections 2024-09-12T02:07:31.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:31.800+0000 7fae284b7640 1 --2- 172.21.15.16:0/3052465365 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fae201082a0 0x7fae2010ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:31.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:31.800+0000 7fae284b7640 1 --2- 172.21.15.16:0/3052465365 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fae201036e0 0x7fae20107b90 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:31.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:31.800+0000 7fae284b7640 1 --2- 172.21.15.16:0/3052465365 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae20102d30 0x7fae20103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:31.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:31.800+0000 7fae284b7640 1 -- 172.21.15.16:0/3052465365 >> 172.21.15.16:0/3052465365 conn(0x7fae200fc7d0 msgr2=0x7fae200febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:07:31.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:31.800+0000 7fae284b7640 1 -- 172.21.15.16:0/3052465365 shutdown_connections 2024-09-12T02:07:31.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:31.800+0000 7fae284b7640 1 -- 172.21.15.16:0/3052465365 wait complete. 2024-09-12T02:07:31.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:31.801+0000 7fae284b7640 1 Processor -- start 2024-09-12T02:07:31.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:31.801+0000 7fae284b7640 1 -- start start 2024-09-12T02:07:31.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:31.801+0000 7fae284b7640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fae20102d30 0x7fae20100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:31.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:31.801+0000 7fae284b7640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fae201036e0 0x7fae20100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:31.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:31.801+0000 7fae284b7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae201082a0 0x7fae20101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:31.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:31.801+0000 7fae284b7640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fae201120a0 con 0x7fae201082a0 2024-09-12T02:07:31.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:31.801+0000 7fae284b7640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fae20111f20 con 0x7fae201036e0 2024-09-12T02:07:31.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:31.801+0000 7fae284b7640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fae20112220 con 0x7fae20102d30 2024-09-12T02:07:31.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:31.801+0000 7fae26a54640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae201082a0 0x7fae20101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:31.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:31.801+0000 7fae26a54640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae201082a0 0x7fae20101410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53944/0 (socket says 172.21.15.16:53944) 2024-09-12T02:07:31.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:31.801+0000 7fae26a54640 1 -- 172.21.15.16:0/1213676420 learned_addr learned my addr 172.21.15.16:0/1213676420 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:07:31.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:31.801+0000 7fae25a52640 1 --2- 172.21.15.16:0/1213676420 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fae201036e0 0x7fae20100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:31.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:31.801+0000 7fae26253640 1 --2- 172.21.15.16:0/1213676420 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fae20102d30 0x7fae20100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:31.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:31.802+0000 7fae26a54640 1 -- 172.21.15.16:0/1213676420 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fae20102d30 msgr2=0x7fae20100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:31.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:31.802+0000 7fae26a54640 1 --2- 172.21.15.16:0/1213676420 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fae20102d30 0x7fae20100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:31.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:31.802+0000 7fae26a54640 1 -- 172.21.15.16:0/1213676420 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fae201036e0 msgr2=0x7fae20100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:31.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:31.802+0000 7fae26a54640 1 --2- 172.21.15.16:0/1213676420 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fae201036e0 0x7fae20100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:31.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:31.802+0000 7fae26a54640 1 -- 172.21.15.16:0/1213676420 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae201a6ae0 con 0x7fae201082a0 2024-09-12T02:07:31.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:31.802+0000 7fae26253640 1 --2- 172.21.15.16:0/1213676420 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fae20102d30 0x7fae20100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:07:31.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:31.802+0000 7fae26a54640 1 --2- 172.21.15.16:0/1213676420 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae201082a0 0x7fae20101410 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fae1800eb10 tx=0x7fae1800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:07:31.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:31.802+0000 7fae1f7fe640 1 -- 172.21.15.16:0/1213676420 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae18004b20 con 0x7fae201082a0 2024-09-12T02:07:31.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:31.802+0000 7fae1f7fe640 1 -- 172.21.15.16:0/1213676420 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae18004cc0 con 0x7fae201082a0 2024-09-12T02:07:31.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:31.802+0000 7fae284b7640 1 -- 172.21.15.16:0/1213676420 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae201a6dd0 con 0x7fae201082a0 2024-09-12T02:07:31.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:31.802+0000 7fae1f7fe640 1 -- 172.21.15.16:0/1213676420 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae18010760 con 0x7fae201082a0 2024-09-12T02:07:31.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:31.802+0000 7fae284b7640 1 -- 172.21.15.16:0/1213676420 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae201a7290 con 0x7fae201082a0 2024-09-12T02:07:31.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:31.803+0000 7fae284b7640 1 -- 172.21.15.16:0/1213676420 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae20103110 con 0x7fae201082a0 2024-09-12T02:07:31.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:31.804+0000 7fae1f7fe640 1 -- 172.21.15.16:0/1213676420 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fae18005020 con 0x7fae201082a0 2024-09-12T02:07:31.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:31.805+0000 7fae1f7fe640 1 --2- 172.21.15.16:0/1213676420 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fae080a2b80 0x7fae080a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:31.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:31.805+0000 7fae1f7fe640 1 -- 172.21.15.16:0/1213676420 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fae180cf5d0 con 0x7fae201082a0 2024-09-12T02:07:31.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:31.805+0000 7fae26253640 1 --2- 172.21.15.16:0/1213676420 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fae080a2b80 0x7fae080a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:31.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:31.806+0000 7fae26253640 1 --2- 172.21.15.16:0/1213676420 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fae080a2b80 0x7fae080a5040 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fae10009c20 tx=0x7fae10005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:07:31.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:31.809+0000 7fae1f7fe640 1 -- 172.21.15.16:0/1213676420 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fae1809c530 con 0x7fae201082a0 2024-09-12T02:07:32.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:32.020+0000 7fae284b7640 1 -- 172.21.15.16:0/1213676420 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fae2010a230 con 0x7fae201082a0 2024-09-12T02:07:32.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:32.021+0000 7fae1f7fe640 1 -- 172.21.15.16:0/1213676420 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11911 (secure 0 0 0) 0x7fae1809be30 con 0x7fae201082a0 2024-09-12T02:07:32.155 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T02:07:32.156 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":25,"btime":"2024-09-12T02:01:37:192021+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3279123279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3279123279},{"type":"v1","addr":"172.21.15.16:6835","nonce":3279123279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4119452382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4119452382},{"type":"v1","addr":"172.21.15.16:6837","nonce":4119452382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/3263823165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3263823165},{"type":"v1","addr":"172.21.15.16:6839","nonce":3263823165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/1510290732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":1510290732},{"type":"v1","addr":"172.21.15.16:6841","nonce":1510290732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/279674603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":279674603},{"type":"v1","addr":"172.21.15.33:6833","nonce":279674603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3555728028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3555728028},{"type":"v1","addr":"172.21.15.33:6835","nonce":3555728028}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T01:59:56.117608+0000","modified":"2024-09-12T02:01:36.294760+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24482,"mds_1":24481,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6833/1563374106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1563374106},{"type":"v1","addr":"172.21.15.184:6833","nonce":1563374106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6837/1892833868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1892833868},{"type":"v1","addr":"172.21.15.33:6837","nonce":1892833868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/1457679686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1457679686},{"type":"v1","addr":"172.21.15.184:6835","nonce":1457679686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.184:6837/1793393087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":1793393087},{"type":"v1","addr":"172.21.15.184:6837","nonce":1793393087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.184:6839/4203347941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":4203347941},{"type":"v1","addr":"172.21.15.184:6839","nonce":4203347941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.33:6839/4136670204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":4136670204},{"type":"v1","addr":"172.21.15.33:6839","nonce":4136670204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24481,24482,24469]},"id":1}]} 2024-09-12T02:07:32.156 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 25 2024-09-12T02:07:32.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:32.023+0000 7fae284b7640 1 -- 172.21.15.16:0/1213676420 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fae080a2b80 msgr2=0x7fae080a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:32.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:32.023+0000 7fae284b7640 1 --2- 172.21.15.16:0/1213676420 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fae080a2b80 0x7fae080a5040 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fae10009c20 tx=0x7fae10005f90 comp rx=0 tx=0).stop 2024-09-12T02:07:32.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:32.023+0000 7fae284b7640 1 -- 172.21.15.16:0/1213676420 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae201082a0 msgr2=0x7fae20101410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:32.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:32.023+0000 7fae284b7640 1 --2- 172.21.15.16:0/1213676420 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae201082a0 0x7fae20101410 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fae1800eb10 tx=0x7fae1800c2d0 comp rx=0 tx=0).stop 2024-09-12T02:07:32.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:32.024+0000 7fae284b7640 1 -- 172.21.15.16:0/1213676420 shutdown_connections 2024-09-12T02:07:32.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:32.024+0000 7fae284b7640 1 --2- 172.21.15.16:0/1213676420 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fae080a2b80 0x7fae080a5040 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:32.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:32.024+0000 7fae284b7640 1 --2- 172.21.15.16:0/1213676420 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae201082a0 0x7fae20101410 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:32.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:32.024+0000 7fae284b7640 1 --2- 172.21.15.16:0/1213676420 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fae201036e0 0x7fae20100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:32.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:32.024+0000 7fae284b7640 1 --2- 172.21.15.16:0/1213676420 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fae20102d30 0x7fae20100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:32.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:32.024+0000 7fae284b7640 1 -- 172.21.15.16:0/1213676420 >> 172.21.15.16:0/1213676420 conn(0x7fae200fc7d0 msgr2=0x7fae20103e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:07:32.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:32.024+0000 7fae284b7640 1 -- 172.21.15.16:0/1213676420 shutdown_connections 2024-09-12T02:07:32.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:32.024+0000 7fae284b7640 1 -- 172.21.15.16:0/1213676420 wait complete. 2024-09-12T02:07:32.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:07:32 smithi016 ceph-mon[30947]: pgmap v503: 97 pgs: 97 active+clean; 3.1 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 17 op/s 2024-09-12T02:07:32.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:07:32 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/1213676420' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:07:32.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:07:32 smithi184 ceph-mon[35593]: pgmap v503: 97 pgs: 97 active+clean; 3.1 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 17 op/s 2024-09-12T02:07:32.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:07:32 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/1213676420' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:07:32.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:07:32 smithi033 ceph-mon[35332]: pgmap v503: 97 pgs: 97 active+clean; 3.1 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 17 op/s 2024-09-12T02:07:32.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:07:32 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/1213676420' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:07:32.741 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24482 quiesce db / --timeout 90 --expiration 140.1 --await-for 45.0' 2024-09-12T02:07:32.742 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24482 quiesce db / --timeout 90 --expiration 140.1 --await-for 45.0 2024-09-12T02:07:33.001 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:07:33.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.500+0000 7f7418d8f640 1 -- 172.21.15.16:0/4010571710 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7414102790 msgr2=0x7f74141067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:33.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.500+0000 7f7418d8f640 1 --2- 172.21.15.16:0/4010571710 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7414102790 0x7f74141067e0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f73fc009900 tx=0x7f73fc02a410 comp rx=0 tx=0).stop 2024-09-12T02:07:33.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.500+0000 7f7418d8f640 1 -- 172.21.15.16:0/4010571710 shutdown_connections 2024-09-12T02:07:33.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.500+0000 7f7418d8f640 1 --2- 172.21.15.16:0/4010571710 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7414106f10 0x7f741410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:33.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.500+0000 7f7418d8f640 1 --2- 172.21.15.16:0/4010571710 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7414102790 0x7f74141067e0 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:33.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.500+0000 7f7418d8f640 1 --2- 172.21.15.16:0/4010571710 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7414101de0 0x7f74141021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:33.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.500+0000 7f7418d8f640 1 -- 172.21.15.16:0/4010571710 >> 172.21.15.16:0/4010571710 conn(0x7f74140fd8c0 msgr2=0x7f74140ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:07:33.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.500+0000 7f7418d8f640 1 -- 172.21.15.16:0/4010571710 shutdown_connections 2024-09-12T02:07:33.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.500+0000 7f7418d8f640 1 -- 172.21.15.16:0/4010571710 wait complete. 2024-09-12T02:07:33.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.501+0000 7f7418d8f640 1 Processor -- start 2024-09-12T02:07:33.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.501+0000 7f7418d8f640 1 -- start start 2024-09-12T02:07:33.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.501+0000 7f7418d8f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7414101de0 0x7f74140744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:33.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.501+0000 7f7418d8f640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7414102790 0x7f7414074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:33.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.501+0000 7f7418d8f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7414106f10 0x7f7414074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:33.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.501+0000 7f7418d8f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7414111f90 con 0x7f7414101de0 2024-09-12T02:07:33.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.501+0000 7f7418d8f640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7414111e10 con 0x7f7414106f10 2024-09-12T02:07:33.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.501+0000 7f7418d8f640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7414112110 con 0x7f7414102790 2024-09-12T02:07:33.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.501+0000 7f741259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7414101de0 0x7f74140744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:33.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.501+0000 7f741259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7414101de0 0x7f74140744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53976/0 (socket says 172.21.15.16:53976) 2024-09-12T02:07:33.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.501+0000 7f741259c640 1 -- 172.21.15.16:0/3985351024 learned_addr learned my addr 172.21.15.16:0/3985351024 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:07:33.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.501+0000 7f740bfff640 1 --2- 172.21.15.16:0/3985351024 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7414102790 0x7f7414074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:33.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.501+0000 7f7412d9d640 1 --2- 172.21.15.16:0/3985351024 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7414106f10 0x7f7414074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:33.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.502+0000 7f741259c640 1 -- 172.21.15.16:0/3985351024 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7414102790 msgr2=0x7f7414074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:33.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.502+0000 7f741259c640 1 --2- 172.21.15.16:0/3985351024 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7414102790 0x7f7414074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:33.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.502+0000 7f741259c640 1 -- 172.21.15.16:0/3985351024 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7414106f10 msgr2=0x7f7414074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:33.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.502+0000 7f741259c640 1 --2- 172.21.15.16:0/3985351024 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7414106f10 0x7f7414074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:33.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.502+0000 7f741259c640 1 -- 172.21.15.16:0/3985351024 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74141a68d0 con 0x7f7414101de0 2024-09-12T02:07:33.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.502+0000 7f741259c640 1 --2- 172.21.15.16:0/3985351024 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7414101de0 0x7f74140744f0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f740400b4a0 tx=0x7f740400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:07:33.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.502+0000 7f740b7fe640 1 -- 172.21.15.16:0/3985351024 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7404014070 con 0x7f7414101de0 2024-09-12T02:07:33.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.502+0000 7f7418d8f640 1 -- 172.21.15.16:0/3985351024 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74141a6bc0 con 0x7f7414101de0 2024-09-12T02:07:33.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.502+0000 7f740b7fe640 1 -- 172.21.15.16:0/3985351024 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7404010af0 con 0x7f7414101de0 2024-09-12T02:07:33.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.502+0000 7f740b7fe640 1 -- 172.21.15.16:0/3985351024 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7404017d10 con 0x7f7414101de0 2024-09-12T02:07:33.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.502+0000 7f7418d8f640 1 -- 172.21.15.16:0/3985351024 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74141a7120 con 0x7f7414101de0 2024-09-12T02:07:33.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.505+0000 7f740b7fe640 1 -- 172.21.15.16:0/3985351024 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f7404017070 con 0x7f7414101de0 2024-09-12T02:07:33.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.505+0000 7f741b168740 1 Processor -- start 2024-09-12T02:07:33.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.505+0000 7f741b168740 1 -- start start 2024-09-12T02:07:33.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.505+0000 7f741b168740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558798e59760 0x558798e59b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:33.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.506+0000 7f741b168740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558798e5a080 0x558798e64400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:33.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.506+0000 7f741b168740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558798e64940 0x558798e66d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:33.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.506+0000 7f741b168740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x558798dbf2a0 con 0x558798e59760 2024-09-12T02:07:33.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.506+0000 7f741b168740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x558798d66b10 con 0x558798e5a080 2024-09-12T02:07:33.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.506+0000 7f741b168740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x558798dbb260 con 0x558798e64940 2024-09-12T02:07:33.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.506+0000 7f740b7fe640 1 --2- 172.21.15.16:0/3985351024 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f74000a2b80 0x7f74000a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:33.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.506+0000 7f7412d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558798e59760 0x558798e59b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:33.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.506+0000 7f740bfff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558798e64940 0x558798e66d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:33.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.506+0000 7f7412d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558798e59760 0x558798e59b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53978/0 (socket says 172.21.15.16:53978) 2024-09-12T02:07:33.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.506+0000 7f7412d9d640 1 -- 172.21.15.16:0/3979413564 learned_addr learned my addr 172.21.15.16:0/3979413564 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:07:33.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.506+0000 7f741259c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558798e5a080 0x558798e64400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:33.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.506+0000 7f7412d9d640 1 --2- 172.21.15.16:0/3985351024 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f74000a2b80 0x7f74000a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:33.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.506+0000 7f740b7fe640 1 -- 172.21.15.16:0/3985351024 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f74040ce260 con 0x7f7414101de0 2024-09-12T02:07:33.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.506+0000 7f7412d9d640 1 -- 172.21.15.16:0/3979413564 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558798e64940 msgr2=0x558798e66d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:33.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.506+0000 7f7412d9d640 1 --2- 172.21.15.16:0/3979413564 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558798e64940 0x558798e66d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:33.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.506+0000 7f7412d9d640 1 -- 172.21.15.16:0/3979413564 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558798e5a080 msgr2=0x558798e64400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:33.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.506+0000 7f7412d9d640 1 --2- 172.21.15.16:0/3979413564 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558798e5a080 0x558798e64400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:33.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.506+0000 7f7412d9d640 1 -- 172.21.15.16:0/3979413564 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558798b30b90 con 0x558798e59760 2024-09-12T02:07:33.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.506+0000 7f740bfff640 1 --2- 172.21.15.16:0/3979413564 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558798e64940 0x558798e66d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:07:33.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.506+0000 7f741259c640 1 --2- 172.21.15.16:0/3979413564 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558798e5a080 0x558798e64400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:07:33.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.507+0000 7f7412d9d640 1 --2- 172.21.15.16:0/3979413564 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558798e59760 0x558798e59b40 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f740c014f00 tx=0x7f740c0117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b72722c1f833b2d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:07:33.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.507+0000 7f7412d9d640 1 --2- 172.21.15.16:0/3985351024 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f74000a2b80 0x7f74000a5040 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f740c007fd0 tx=0x7f740c01c000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:07:33.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.507+0000 7f7408ff9640 1 -- 172.21.15.16:0/3979413564 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f740c022070 con 0x558798e59760 2024-09-12T02:07:33.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.507+0000 7f7408ff9640 1 -- 172.21.15.16:0/3979413564 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f740c00ce50 con 0x558798e59760 2024-09-12T02:07:33.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.507+0000 7f7408ff9640 1 -- 172.21.15.16:0/3979413564 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f740c01e670 con 0x558798e59760 2024-09-12T02:07:33.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.508+0000 7f741b168740 1 -- 172.21.15.16:0/3979413564 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558798e59760 msgr2=0x558798e59b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:33.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.508+0000 7f741b168740 1 --2- 172.21.15.16:0/3979413564 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558798e59760 0x558798e59b40 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f740c014f00 tx=0x7f740c0117b0 comp rx=0 tx=0).stop 2024-09-12T02:07:33.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.508+0000 7f741b168740 1 -- 172.21.15.16:0/3979413564 shutdown_connections 2024-09-12T02:07:33.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.508+0000 7f741b168740 1 --2- 172.21.15.16:0/3979413564 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558798e64940 0x558798e66d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:33.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.508+0000 7f741b168740 1 --2- 172.21.15.16:0/3979413564 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558798e5a080 0x558798e64400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:33.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.508+0000 7f741b168740 1 --2- 172.21.15.16:0/3979413564 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558798e59760 0x558798e59b40 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:33.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.508+0000 7f741b168740 1 -- 172.21.15.16:0/3979413564 >> 172.21.15.16:0/3979413564 conn(0x558798e55cf0 msgr2=0x558798e560d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:07:33.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.508+0000 7f741b168740 1 -- 172.21.15.16:0/3979413564 shutdown_connections 2024-09-12T02:07:33.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.508+0000 7f741b168740 1 -- 172.21.15.16:0/3979413564 wait complete. 2024-09-12T02:07:33.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.508+0000 7f741b168740 1 -- start start 2024-09-12T02:07:33.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.508+0000 7f741b168740 1 Processor -- start 2024-09-12T02:07:33.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.509+0000 7f741b168740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558798e59760 0x558798e618e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:33.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.509+0000 7f741b168740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558798e5a080 0x558798e61e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:33.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.509+0000 7f741b168740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558798e64940 0x558798e62360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:33.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.509+0000 7f741b168740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x558798e67510 con 0x558798e59760 2024-09-12T02:07:33.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.509+0000 7f741b168740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x558798d476e0 con 0x558798e64940 2024-09-12T02:07:33.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.509+0000 7f741b168740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x558798e67690 con 0x558798e5a080 2024-09-12T02:07:33.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.509+0000 7f741b168740 10 client.0 _ll_register_callbacks cb 0x558798d6ba80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:07:33.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.509+0000 7f741259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558798e59760 0x558798e618e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:33.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.509+0000 7f741259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558798e59760 0x558798e618e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53990/0 (socket says 172.21.15.16:53990) 2024-09-12T02:07:33.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.509+0000 7f741259c640 1 -- 172.21.15.16:0/180002778 learned_addr learned my addr 172.21.15.16:0/180002778 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:07:33.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.509+0000 7f7412d9d640 1 --2- 172.21.15.16:0/180002778 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558798e64940 0x558798e62360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:33.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.509+0000 7f740bfff640 1 --2- 172.21.15.16:0/180002778 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558798e5a080 0x558798e61e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:33.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.509+0000 7f741259c640 1 -- 172.21.15.16:0/180002778 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558798e5a080 msgr2=0x558798e61e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:33.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.509+0000 7f741259c640 1 --2- 172.21.15.16:0/180002778 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558798e5a080 0x558798e61e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:33.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.509+0000 7f741259c640 1 -- 172.21.15.16:0/180002778 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558798e64940 msgr2=0x558798e62360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:33.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.509+0000 7f741259c640 1 --2- 172.21.15.16:0/180002778 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558798e64940 0x558798e62360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:33.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.509+0000 7f741259c640 1 -- 172.21.15.16:0/180002778 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558798e70ae0 con 0x558798e59760 2024-09-12T02:07:33.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.509+0000 7f7412d9d640 1 --2- 172.21.15.16:0/180002778 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558798e64940 0x558798e62360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:07:33.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.509+0000 7f741259c640 1 --2- 172.21.15.16:0/180002778 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558798e59760 0x558798e618e0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f7404009420 tx=0x7f7404065890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=358a14b5a283972e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:07:33.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.509+0000 7f73f9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:07:33.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.509+0000 7f73f9ffb640 1 -- 172.21.15.16:0/180002778 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f740401e040 con 0x558798e59760 2024-09-12T02:07:33.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.509+0000 7f73f9ffb640 1 -- 172.21.15.16:0/180002778 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f7404017360 con 0x558798e59760 2024-09-12T02:07:33.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.509+0000 7f741b168740 1 -- 172.21.15.16:0/180002778 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f740c00ce50 con 0x558798e59760 2024-09-12T02:07:33.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.510+0000 7f73f9ffb640 1 -- 172.21.15.16:0/180002778 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f74040a37d0 con 0x558798e59760 2024-09-12T02:07:33.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.510+0000 7f741b168740 10 client.17127 fetch_fsmap learned FSMap version 25 2024-09-12T02:07:33.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.510+0000 7f741b168740 1 -- 172.21.15.16:0/180002778 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x558798e73490 con 0x558798e59760 2024-09-12T02:07:33.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.510+0000 7f73f9ffb640 1 -- 172.21.15.16:0/180002778 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f740c00ce50 con 0x558798e59760 2024-09-12T02:07:33.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.510+0000 7f73f9ffb640 1 -- 172.21.15.16:0/180002778 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f74040a3340 con 0x558798e59760 2024-09-12T02:07:33.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.510+0000 7f741b168740 10 client.17127 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:07:33.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.510+0000 7f741b168740 10 client.17127 resolve_mds: validated gid 24482 aka daemon mds.b 2024-09-12T02:07:33.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.510+0000 7f741b168740 20 client.17127 populate_metadata read hostname 'smithi016' 2024-09-12T02:07:33.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.510+0000 7f741b168740 1 --2- 172.21.15.16:0/180002778 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x558798e748a0 0x558798e76c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:33.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.510+0000 7f741b168740 4 client.17127 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:07:33.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.510+0000 7f741b168740 1 -- 172.21.15.16:0/180002778 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558798dcc9d0 con 0x558798e748a0 2024-09-12T02:07:33.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.510+0000 7f740bfff640 1 --2- 172.21.15.16:0/180002778 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x558798e748a0 0x558798e76c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:33.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.511+0000 7f740bfff640 1 --2- 172.21.15.16:0/180002778 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x558798e748a0 0x558798e76c90 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e2da6837568c34d server_cookie=39b86cb88ef29556 in_seq=0 out_seq=0 2024-09-12T02:07:33.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.511+0000 7f73f9ffb640 10 client.17127 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:07:33.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.511+0000 7f73f9ffb640 1 -- 172.21.15.16:0/180002778 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558798dcc9d0 con 0x558798e748a0 2024-09-12T02:07:33.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.511+0000 7f73f9ffb640 10 client.17127 handle_command_reply: tid=0 2024-09-12T02:07:33.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.511+0000 7f741b168740 1 client.17127 shutdown 2024-09-12T02:07:33.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.512+0000 7f741b168740 1 -- 172.21.15.16:0/180002778 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558798e59760 msgr2=0x558798e618e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:33.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.512+0000 7f741b168740 1 --2- 172.21.15.16:0/180002778 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558798e59760 0x558798e618e0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f7404009420 tx=0x7f7404065890 comp rx=0 tx=0).stop 2024-09-12T02:07:33.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.512+0000 7f741b168740 1 -- 172.21.15.16:0/180002778 shutdown_connections 2024-09-12T02:07:33.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.512+0000 7f741b168740 1 --2- 172.21.15.16:0/180002778 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x558798e748a0 0x558798e76c90 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:33.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.512+0000 7f741b168740 1 --2- 172.21.15.16:0/180002778 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558798e64940 0x558798e62360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:33.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.512+0000 7f741b168740 1 --2- 172.21.15.16:0/180002778 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558798e5a080 0x558798e61e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:33.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.512+0000 7f741b168740 1 --2- 172.21.15.16:0/180002778 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558798e59760 0x558798e618e0 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:33.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.512+0000 7f741b168740 1 -- 172.21.15.16:0/180002778 >> 172.21.15.16:0/180002778 conn(0x558798e55cf0 msgr2=0x558798e54e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:07:33.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.512+0000 7f73f9ffb640 0 client.17127 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:07:33.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.512+0000 7f741b168740 1 -- 172.21.15.16:0/180002778 shutdown_connections 2024-09-12T02:07:33.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.512+0000 7f741b168740 1 -- 172.21.15.16:0/180002778 wait complete. 2024-09-12T02:07:33.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.512+0000 7f741b168740 20 client.17127 trim_cache size 0 max 16384 2024-09-12T02:07:33.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.529+0000 7f741b168740 1 Processor -- start 2024-09-12T02:07:33.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.529+0000 7f741b168740 1 -- start start 2024-09-12T02:07:33.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.529+0000 7f741b168740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558798e59760 0x558798e5c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:33.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.529+0000 7f741b168740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558798e5a080 0x558798e54d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:33.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.529+0000 7f741b168740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558798e64940 0x558798e7c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:33.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.529+0000 7f741b168740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f74141b1db0 con 0x558798e64940 2024-09-12T02:07:33.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.529+0000 7f741b168740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f74141af680 con 0x558798e59760 2024-09-12T02:07:33.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.529+0000 7f741b168740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f74141ad510 con 0x558798e5a080 2024-09-12T02:07:33.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.529+0000 7f7412d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558798e64940 0x558798e7c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:33.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.529+0000 7f7412d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558798e64940 0x558798e7c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53998/0 (socket says 172.21.15.16:53998) 2024-09-12T02:07:33.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.529+0000 7f7412d9d640 1 -- 172.21.15.16:0/1144735714 learned_addr learned my addr 172.21.15.16:0/1144735714 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:07:33.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.529+0000 7f741259c640 1 --2- 172.21.15.16:0/1144735714 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558798e59760 0x558798e5c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:33.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.529+0000 7f740bfff640 1 --2- 172.21.15.16:0/1144735714 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558798e5a080 0x558798e54d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:33.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.530+0000 7f7412d9d640 1 -- 172.21.15.16:0/1144735714 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558798e5a080 msgr2=0x558798e54d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:33.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.530+0000 7f7412d9d640 1 --2- 172.21.15.16:0/1144735714 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558798e5a080 0x558798e54d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:33.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.530+0000 7f7412d9d640 1 -- 172.21.15.16:0/1144735714 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558798e59760 msgr2=0x558798e5c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:33.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.530+0000 7f7412d9d640 1 --2- 172.21.15.16:0/1144735714 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558798e59760 0x558798e5c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:33.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.530+0000 7f7412d9d640 1 -- 172.21.15.16:0/1144735714 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558798e51260 con 0x558798e64940 2024-09-12T02:07:33.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.530+0000 7f7412d9d640 1 --2- 172.21.15.16:0/1144735714 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558798e64940 0x558798e7c490 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f740c00fd60 tx=0x7f740c00fcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23cb1db4c8ea9bb0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:07:33.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.530+0000 7f73fbfff640 1 -- 172.21.15.16:0/1144735714 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f740c034070 con 0x558798e64940 2024-09-12T02:07:33.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.530+0000 7f73fbfff640 1 -- 172.21.15.16:0/1144735714 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f740c00f930 con 0x558798e64940 2024-09-12T02:07:33.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.530+0000 7f73fbfff640 1 -- 172.21.15.16:0/1144735714 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f740c022040 con 0x558798e64940 2024-09-12T02:07:33.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.531+0000 7f741b168740 1 -- 172.21.15.16:0/1144735714 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558798e64940 msgr2=0x558798e7c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:33.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.531+0000 7f741b168740 1 --2- 172.21.15.16:0/1144735714 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558798e64940 0x558798e7c490 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f740c00fd60 tx=0x7f740c00fcf0 comp rx=0 tx=0).stop 2024-09-12T02:07:33.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.531+0000 7f741b168740 1 -- 172.21.15.16:0/1144735714 shutdown_connections 2024-09-12T02:07:33.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.531+0000 7f741b168740 1 --2- 172.21.15.16:0/1144735714 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558798e64940 0x558798e7c490 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:33.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.531+0000 7f741b168740 1 --2- 172.21.15.16:0/1144735714 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558798e5a080 0x558798e54d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:33.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.531+0000 7f741b168740 1 --2- 172.21.15.16:0/1144735714 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558798e59760 0x558798e5c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:33.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.531+0000 7f741b168740 1 -- 172.21.15.16:0/1144735714 >> 172.21.15.16:0/1144735714 conn(0x558798e55cf0 msgr2=0x558798e54650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:07:33.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.531+0000 7f741b168740 1 -- 172.21.15.16:0/1144735714 shutdown_connections 2024-09-12T02:07:33.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.531+0000 7f741b168740 1 -- 172.21.15.16:0/1144735714 wait complete. 2024-09-12T02:07:33.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.531+0000 7f741b168740 1 -- start start 2024-09-12T02:07:33.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.531+0000 7f741b168740 1 Processor -- start 2024-09-12T02:07:33.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.531+0000 7f741b168740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558798e59760 0x558798e68820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:33.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.531+0000 7f741b168740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558798e5a080 0x558798e68d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:33.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.531+0000 7f741b168740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558798e64940 0x558798e692a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:33.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.531+0000 7f741b168740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f74141adb60 con 0x558798e5a080 2024-09-12T02:07:33.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.531+0000 7f741b168740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f74141aefb0 con 0x558798e59760 2024-09-12T02:07:33.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.531+0000 7f741b168740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f74141ae210 con 0x558798e64940 2024-09-12T02:07:33.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.532+0000 7f741b168740 10 client.0 _ll_register_callbacks cb 0x7f740409a750 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:07:33.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.532+0000 7f740bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558798e5a080 0x558798e68d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:33.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.532+0000 7f740bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558798e5a080 0x558798e68d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54000/0 (socket says 172.21.15.16:54000) 2024-09-12T02:07:33.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.532+0000 7f740bfff640 1 -- 172.21.15.16:0/1060686774 learned_addr learned my addr 172.21.15.16:0/1060686774 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:07:33.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.532+0000 7f741259c640 1 --2- 172.21.15.16:0/1060686774 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558798e59760 0x558798e68820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:33.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.532+0000 7f7412d9d640 1 --2- 172.21.15.16:0/1060686774 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558798e64940 0x558798e692a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:33.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.532+0000 7f740bfff640 1 -- 172.21.15.16:0/1060686774 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558798e64940 msgr2=0x558798e692a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:33.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.532+0000 7f740bfff640 1 --2- 172.21.15.16:0/1060686774 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558798e64940 0x558798e692a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:33.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.532+0000 7f740bfff640 1 -- 172.21.15.16:0/1060686774 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558798e59760 msgr2=0x558798e68820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:33.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.532+0000 7f740bfff640 1 --2- 172.21.15.16:0/1060686774 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558798e59760 0x558798e68820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:33.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.532+0000 7f740bfff640 1 -- 172.21.15.16:0/1060686774 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558798ede660 con 0x558798e5a080 2024-09-12T02:07:33.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.532+0000 7f740bfff640 1 --2- 172.21.15.16:0/1060686774 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558798e5a080 0x558798e68d60 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f73fc031860 tx=0x7f73fc031990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45b6f37ac57b5f04 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:07:33.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.532+0000 7f73f9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:07:33.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.532+0000 7f73f9ffb640 1 -- 172.21.15.16:0/1060686774 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73fc04a070 con 0x558798e5a080 2024-09-12T02:07:33.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.532+0000 7f73f9ffb640 1 -- 172.21.15.16:0/1060686774 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f73fc004450 con 0x558798e5a080 2024-09-12T02:07:33.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.532+0000 7f741b168740 1 -- 172.21.15.16:0/1060686774 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f740c00f930 con 0x558798e5a080 2024-09-12T02:07:33.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.533+0000 7f73f9ffb640 1 -- 172.21.15.16:0/1060686774 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f73fc056050 con 0x558798e5a080 2024-09-12T02:07:33.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.533+0000 7f741b168740 10 client.17133 fetch_fsmap learned FSMap version 25 2024-09-12T02:07:33.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.533+0000 7f741b168740 1 -- 172.21.15.16:0/1060686774 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x558798edf990 con 0x558798e5a080 2024-09-12T02:07:33.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.533+0000 7f73f9ffb640 1 -- 172.21.15.16:0/1060686774 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f740c00f930 con 0x558798e5a080 2024-09-12T02:07:33.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.533+0000 7f73f9ffb640 1 -- 172.21.15.16:0/1060686774 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f73fc0035f0 con 0x558798e5a080 2024-09-12T02:07:33.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.533+0000 7f741b168740 10 client.17133 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:07:33.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.533+0000 7f741b168740 10 client.17133 resolve_mds: validated gid 24482 aka daemon mds.b 2024-09-12T02:07:33.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.533+0000 7f741b168740 20 client.17133 populate_metadata read hostname 'smithi016' 2024-09-12T02:07:33.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.533+0000 7f741b168740 1 --2- 172.21.15.16:0/1060686774 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x558798ee0160 0x558798ee2580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:33.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.533+0000 7f741b168740 4 client.17133 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.1, "await_for": 45.0}] 2024-09-12T02:07:33.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.533+0000 7f741b168740 1 -- 172.21.15.16:0/1060686774 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.1, "await_for": 45.0}) -- 0x558798dc6d20 con 0x558798ee0160 2024-09-12T02:07:33.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.533+0000 7f741259c640 1 --2- 172.21.15.16:0/1060686774 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x558798ee0160 0x558798ee2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:33.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.534+0000 7f741259c640 1 --2- 172.21.15.16:0/1060686774 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x558798ee0160 0x558798ee2580 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50fff7d665d1c130 server_cookie=8a9870619194ba64 in_seq=0 out_seq=0 2024-09-12T02:07:33.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.534+0000 7f73f9ffb640 10 client.17133 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:07:33.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.544+0000 7f73f9ffb640 1 -- 172.21.15.16:0/1060686774 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x558798dc6d20 con 0x558798ee0160 2024-09-12T02:07:33.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.544+0000 7f73f9ffb640 10 client.17133 handle_command_reply: tid=0 2024-09-12T02:07:33.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.544+0000 7f741b168740 1 client.17133 shutdown 2024-09-12T02:07:33.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.544+0000 7f741b168740 1 -- 172.21.15.16:0/1060686774 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558798e5a080 msgr2=0x558798e68d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:33.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.544+0000 7f741b168740 1 --2- 172.21.15.16:0/1060686774 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558798e5a080 0x558798e68d60 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f73fc031860 tx=0x7f73fc031990 comp rx=0 tx=0).stop 2024-09-12T02:07:33.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.544+0000 7f741b168740 1 -- 172.21.15.16:0/1060686774 shutdown_connections 2024-09-12T02:07:33.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.544+0000 7f741b168740 1 --2- 172.21.15.16:0/1060686774 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x558798ee0160 0x558798ee2580 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:33.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.544+0000 7f73f9ffb640 0 client.17133 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:07:33.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.544+0000 7f741b168740 1 --2- 172.21.15.16:0/1060686774 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558798e64940 0x558798e692a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:33.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.544+0000 7f741b168740 1 --2- 172.21.15.16:0/1060686774 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558798e5a080 0x558798e68d60 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:33.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.544+0000 7f741b168740 1 --2- 172.21.15.16:0/1060686774 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558798e59760 0x558798e68820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:33.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.544+0000 7f741b168740 1 -- 172.21.15.16:0/1060686774 >> 172.21.15.16:0/1060686774 conn(0x558798e55cf0 msgr2=0x558798e54e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:07:33.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.544+0000 7f741b168740 1 -- 172.21.15.16:0/1060686774 shutdown_connections 2024-09-12T02:07:33.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.544+0000 7f741b168740 1 -- 172.21.15.16:0/1060686774 wait complete. 2024-09-12T02:07:33.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.544+0000 7f741b168740 20 client.17133 trim_cache size 0 max 16384 2024-09-12T02:07:33.547 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T02:07:33.547 INFO:teuthology.orchestra.run.smithi016.stdout: "epoch": 25, 2024-09-12T02:07:33.547 INFO:teuthology.orchestra.run.smithi016.stdout: "leader": 24482, 2024-09-12T02:07:33.547 INFO:teuthology.orchestra.run.smithi016.stdout: "set_version": 22, 2024-09-12T02:07:33.547 INFO:teuthology.orchestra.run.smithi016.stdout: "sets": { 2024-09-12T02:07:33.547 INFO:teuthology.orchestra.run.smithi016.stdout: "a93721c1": { 2024-09-12T02:07:33.547 INFO:teuthology.orchestra.run.smithi016.stdout: "version": 22, 2024-09-12T02:07:33.547 INFO:teuthology.orchestra.run.smithi016.stdout: "age_ref": 0.0, 2024-09-12T02:07:33.547 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-12T02:07:33.547 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "QUIESCED", 2024-09-12T02:07:33.547 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-12T02:07:33.547 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:07:33.548 INFO:teuthology.orchestra.run.smithi016.stdout: "timeout": 90.0, 2024-09-12T02:07:33.548 INFO:teuthology.orchestra.run.smithi016.stdout: "expiration": 140.1, 2024-09-12T02:07:33.548 INFO:teuthology.orchestra.run.smithi016.stdout: "members": { 2024-09-12T02:07:33.548 INFO:teuthology.orchestra.run.smithi016.stdout: "file:/": { 2024-09-12T02:07:33.548 INFO:teuthology.orchestra.run.smithi016.stdout: "excluded": false, 2024-09-12T02:07:33.548 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-12T02:07:33.548 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "QUIESCED", 2024-09-12T02:07:33.548 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-12T02:07:33.548 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:07:33.548 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:07:33.548 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:07:33.548 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:07:33.548 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:07:33.549 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T02:07:33.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.545+0000 7f7418d8f640 1 -- 172.21.15.16:0/3985351024 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f74000a2b80 msgr2=0x7f74000a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:33.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.545+0000 7f7418d8f640 1 --2- 172.21.15.16:0/3985351024 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f74000a2b80 0x7f74000a5040 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f740c007fd0 tx=0x7f740c01c000 comp rx=0 tx=0).stop 2024-09-12T02:07:33.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.545+0000 7f7418d8f640 1 -- 172.21.15.16:0/3985351024 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7414101de0 msgr2=0x7f74140744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:33.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.545+0000 7f7418d8f640 1 --2- 172.21.15.16:0/3985351024 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7414101de0 0x7f74140744f0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f740400b4a0 tx=0x7f740400b960 comp rx=0 tx=0).stop 2024-09-12T02:07:33.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.545+0000 7f7418d8f640 1 -- 172.21.15.16:0/3985351024 shutdown_connections 2024-09-12T02:07:33.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.545+0000 7f7418d8f640 1 --2- 172.21.15.16:0/3985351024 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f74000a2b80 0x7f74000a5040 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:33.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.545+0000 7f7418d8f640 1 --2- 172.21.15.16:0/3985351024 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7414106f10 0x7f7414074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:33.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.545+0000 7f7418d8f640 1 --2- 172.21.15.16:0/3985351024 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7414102790 0x7f7414074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:33.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.545+0000 7f7418d8f640 1 --2- 172.21.15.16:0/3985351024 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7414101de0 0x7f74140744f0 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:33.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.545+0000 7f7418d8f640 1 -- 172.21.15.16:0/3985351024 >> 172.21.15.16:0/3985351024 conn(0x7f74140fd8c0 msgr2=0x7f74140fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:07:33.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.545+0000 7f7418d8f640 1 -- 172.21.15.16:0/3985351024 shutdown_connections 2024-09-12T02:07:33.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:33.545+0000 7f7418d8f640 1 -- 172.21.15.16:0/3985351024 wait complete. 2024-09-12T02:07:34.414 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:07:34 smithi016 ceph-mon[30947]: pgmap v504: 97 pgs: 97 active+clean; 3.1 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-09-12T02:07:34.448 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'a93721c1', quiesce took 3.4 seconds. Will release after: 16.700000000000003 2024-09-12T02:07:34.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:07:34 smithi184 ceph-mon[35593]: pgmap v504: 97 pgs: 97 active+clean; 3.1 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-09-12T02:07:34.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:07:34 smithi033 ceph-mon[35332]: pgmap v504: 97 pgs: 97 active+clean; 3.1 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-09-12T02:07:36.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:07:36 smithi184 ceph-mon[35593]: pgmap v505: 97 pgs: 97 active+clean; 3.1 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-09-12T02:07:36.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:07:36 smithi033 ceph-mon[35332]: pgmap v505: 97 pgs: 97 active+clean; 3.1 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-09-12T02:07:36.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:07:36 smithi016 ceph-mon[30947]: pgmap v505: 97 pgs: 97 active+clean; 3.1 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-09-12T02:07:38.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:07:38 smithi016 ceph-mon[30947]: pgmap v506: 97 pgs: 97 active+clean; 3.1 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-09-12T02:07:38.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:07:38 smithi184 ceph-mon[35593]: pgmap v506: 97 pgs: 97 active+clean; 3.1 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-09-12T02:07:38.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:07:38 smithi033 ceph-mon[35332]: pgmap v506: 97 pgs: 97 active+clean; 3.1 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-09-12T02:07:39.408 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.406+0000 7f72eefcf640 1 -- 172.21.15.16:0/1236317660 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72e8102660 msgr2=0x7f72e8104a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:39.408 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.406+0000 7f72eefcf640 1 --2- 172.21.15.16:0/1236317660 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72e8102660 0x7f72e8104a50 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f72d8009900 tx=0x7f72d802a410 comp rx=0 tx=0).stop 2024-09-12T02:07:39.408 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.406+0000 7f72eefcf640 1 -- 172.21.15.16:0/1236317660 shutdown_connections 2024-09-12T02:07:39.408 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.406+0000 7f72eefcf640 1 --2- 172.21.15.16:0/1236317660 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f72e81078c0 0x7f72e8109cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:39.408 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.406+0000 7f72eefcf640 1 --2- 172.21.15.16:0/1236317660 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f72e8104f90 0x7f72e8107380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:39.409 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.406+0000 7f72eefcf640 1 --2- 172.21.15.16:0/1236317660 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72e8102660 0x7f72e8104a50 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:39.409 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.406+0000 7f72eefcf640 1 -- 172.21.15.16:0/1236317660 >> 172.21.15.16:0/1236317660 conn(0x7f72e80fc1e0 msgr2=0x7f72e80fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:07:39.409 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.406+0000 7f72eefcf640 1 -- 172.21.15.16:0/1236317660 shutdown_connections 2024-09-12T02:07:39.409 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.406+0000 7f72eefcf640 1 -- 172.21.15.16:0/1236317660 wait complete. 2024-09-12T02:07:39.409 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.407+0000 7f72eefcf640 1 Processor -- start 2024-09-12T02:07:39.409 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.407+0000 7f72eefcf640 1 -- start start 2024-09-12T02:07:39.409 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.407+0000 7f72eefcf640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72e8102660 0x7f72e8197e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:39.410 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.407+0000 7f72eefcf640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f72e8104f90 0x7f72e81983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:39.410 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.407+0000 7f72eefcf640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f72e81078c0 0x7f72e819f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:39.410 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.407+0000 7f72eefcf640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f72e810d800 con 0x7f72e8102660 2024-09-12T02:07:39.410 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.407+0000 7f72eefcf640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f72e810d680 con 0x7f72e8104f90 2024-09-12T02:07:39.410 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.407+0000 7f72eefcf640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f72e810d980 con 0x7f72e81078c0 2024-09-12T02:07:39.410 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.407+0000 7f72ecd6b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72e8102660 0x7f72e8197e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:39.410 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.407+0000 7f72ecd6b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72e8102660 0x7f72e8197e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54020/0 (socket says 172.21.15.16:54020) 2024-09-12T02:07:39.411 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.407+0000 7f72ecd6b640 1 -- 172.21.15.16:0/2140698366 learned_addr learned my addr 172.21.15.16:0/2140698366 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:07:39.411 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.407+0000 7f72ed56c640 1 --2- 172.21.15.16:0/2140698366 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f72e81078c0 0x7f72e819f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:39.411 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.407+0000 7f72e7fff640 1 --2- 172.21.15.16:0/2140698366 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f72e8104f90 0x7f72e81983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:39.411 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.407+0000 7f72ecd6b640 1 -- 172.21.15.16:0/2140698366 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f72e81078c0 msgr2=0x7f72e819f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:39.411 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.408+0000 7f72ecd6b640 1 --2- 172.21.15.16:0/2140698366 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f72e81078c0 0x7f72e819f420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:39.411 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.408+0000 7f72ecd6b640 1 -- 172.21.15.16:0/2140698366 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f72e8104f90 msgr2=0x7f72e81983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:39.411 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.408+0000 7f72ecd6b640 1 --2- 172.21.15.16:0/2140698366 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f72e8104f90 0x7f72e81983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:39.412 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.408+0000 7f72ecd6b640 1 -- 172.21.15.16:0/2140698366 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72e819fa90 con 0x7f72e8102660 2024-09-12T02:07:39.412 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.408+0000 7f72ecd6b640 1 --2- 172.21.15.16:0/2140698366 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72e8102660 0x7f72e8197e70 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f72d8009230 tx=0x7f72d8031b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:07:39.412 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.408+0000 7f72e5ffb640 1 -- 172.21.15.16:0/2140698366 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72d803f030 con 0x7f72e8102660 2024-09-12T02:07:39.412 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.408+0000 7f72e5ffb640 1 -- 172.21.15.16:0/2140698366 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72d8003ad0 con 0x7f72e8102660 2024-09-12T02:07:39.412 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.408+0000 7f72eefcf640 1 -- 172.21.15.16:0/2140698366 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72e819fd20 con 0x7f72e8102660 2024-09-12T02:07:39.412 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.408+0000 7f72e5ffb640 1 -- 172.21.15.16:0/2140698366 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72d8035070 con 0x7f72e8102660 2024-09-12T02:07:39.412 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.408+0000 7f72eefcf640 1 -- 172.21.15.16:0/2140698366 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72e81a01b0 con 0x7f72e8102660 2024-09-12T02:07:39.419 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.411+0000 7f72e5ffb640 1 -- 172.21.15.16:0/2140698366 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f72d8042030 con 0x7f72e8102660 2024-09-12T02:07:39.420 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.411+0000 7f72f1e47740 1 Processor -- start 2024-09-12T02:07:39.420 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.411+0000 7f72f1e47740 1 -- start start 2024-09-12T02:07:39.420 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.411+0000 7f72f1e47740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5634de1cc9a0 0x5634de1ccda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:39.420 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.411+0000 7f72f1e47740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634de1cd2e0 0x5634de1d1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:39.420 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.411+0000 7f72f1e47740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5634de1d2370 0x5634de1d4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:39.420 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.411+0000 7f72f1e47740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5634de1302f0 con 0x5634de1cd2e0 2024-09-12T02:07:39.420 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.411+0000 7f72f1e47740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5634de0d8270 con 0x5634de1cc9a0 2024-09-12T02:07:39.420 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.411+0000 7f72f1e47740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5634de12c2b0 con 0x5634de1d2370 2024-09-12T02:07:39.420 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.412+0000 7f72ecd6b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634de1cd2e0 0x5634de1d1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:39.420 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.412+0000 7f72ecd6b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634de1cd2e0 0x5634de1d1da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54030/0 (socket says 172.21.15.16:54030) 2024-09-12T02:07:39.420 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.412+0000 7f72ecd6b640 1 -- 172.21.15.16:0/1966901684 learned_addr learned my addr 172.21.15.16:0/1966901684 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:07:39.420 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.412+0000 7f72ed56c640 1 --2- 172.21.15.16:0/1966901684 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5634de1cc9a0 0x5634de1ccda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:39.421 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.412+0000 7f72e7fff640 1 --2- 172.21.15.16:0/1966901684 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5634de1d2370 0x5634de1d4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:39.421 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.412+0000 7f72e5ffb640 1 --2- 172.21.15.16:0/2140698366 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f72d00a2cb0 0x7f72d00a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:39.421 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.412+0000 7f72ecd6b640 1 -- 172.21.15.16:0/1966901684 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5634de1d2370 msgr2=0x5634de1d4830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:39.421 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.412+0000 7f72ecd6b640 1 --2- 172.21.15.16:0/1966901684 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5634de1d2370 0x5634de1d4830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:39.421 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.412+0000 7f72ecd6b640 1 -- 172.21.15.16:0/1966901684 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5634de1cc9a0 msgr2=0x5634de1ccda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:39.421 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.412+0000 7f72ecd6b640 1 --2- 172.21.15.16:0/1966901684 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5634de1cc9a0 0x5634de1ccda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:39.421 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.412+0000 7f72ecd6b640 1 -- 172.21.15.16:0/1966901684 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634de0d7340 con 0x5634de1cd2e0 2024-09-12T02:07:39.421 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.412+0000 7f72ed56c640 1 --2- 172.21.15.16:0/1966901684 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5634de1cc9a0 0x5634de1ccda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:07:39.421 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.412+0000 7f72ed56c640 1 --2- 172.21.15.16:0/2140698366 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f72d00a2cb0 0x7f72d00a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:39.421 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.412+0000 7f72e5ffb640 1 -- 172.21.15.16:0/2140698366 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f72d80ef4a0 con 0x7f72e8102660 2024-09-12T02:07:39.421 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.413+0000 7f72ecd6b640 1 --2- 172.21.15.16:0/1966901684 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634de1cd2e0 0x5634de1d1da0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f72d80a34c0 tx=0x7f72d8086a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=845cefb7d18a84db server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:07:39.421 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.413+0000 7f72ed56c640 1 --2- 172.21.15.16:0/2140698366 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f72d00a2cb0 0x7f72d00a5170 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f72e806a980 tx=0x7f72e0004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:07:39.421 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.413+0000 7f72ceffd640 1 -- 172.21.15.16:0/1966901684 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72d80a5040 con 0x5634de1cd2e0 2024-09-12T02:07:39.422 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.413+0000 7f72ceffd640 1 -- 172.21.15.16:0/1966901684 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72d80a34f0 con 0x5634de1cd2e0 2024-09-12T02:07:39.422 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.413+0000 7f72ceffd640 1 -- 172.21.15.16:0/1966901684 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72d808ab40 con 0x5634de1cd2e0 2024-09-12T02:07:39.422 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.413+0000 7f72f1e47740 1 -- 172.21.15.16:0/1966901684 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634de1cd2e0 msgr2=0x5634de1d1da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:39.422 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.413+0000 7f72f1e47740 1 --2- 172.21.15.16:0/1966901684 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634de1cd2e0 0x5634de1d1da0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f72d80a34c0 tx=0x7f72d8086a30 comp rx=0 tx=0).stop 2024-09-12T02:07:39.422 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.414+0000 7f72f1e47740 1 -- 172.21.15.16:0/1966901684 shutdown_connections 2024-09-12T02:07:39.422 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.414+0000 7f72f1e47740 1 --2- 172.21.15.16:0/1966901684 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5634de1d2370 0x5634de1d4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:39.422 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.414+0000 7f72f1e47740 1 --2- 172.21.15.16:0/1966901684 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634de1cd2e0 0x5634de1d1da0 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:39.422 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.414+0000 7f72f1e47740 1 --2- 172.21.15.16:0/1966901684 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5634de1cc9a0 0x5634de1ccda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:39.422 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.414+0000 7f72f1e47740 1 -- 172.21.15.16:0/1966901684 >> 172.21.15.16:0/1966901684 conn(0x5634de1c8010 msgr2=0x5634de1c8410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:07:39.422 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.414+0000 7f72f1e47740 1 -- 172.21.15.16:0/1966901684 shutdown_connections 2024-09-12T02:07:39.422 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.414+0000 7f72f1e47740 1 -- 172.21.15.16:0/1966901684 wait complete. 2024-09-12T02:07:39.422 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.414+0000 7f72f1e47740 1 -- start start 2024-09-12T02:07:39.422 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.414+0000 7f72f1e47740 1 Processor -- start 2024-09-12T02:07:39.423 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.414+0000 7f72f1e47740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5634de1cc9a0 0x5634de1d6890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:39.423 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.414+0000 7f72f1e47740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5634de1cd2e0 0x5634de1d6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:39.423 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.414+0000 7f72f1e47740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634de1d2370 0x5634de1ddda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:39.423 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.414+0000 7f72f1e47740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5634de1d5100 con 0x5634de1d2370 2024-09-12T02:07:39.423 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.414+0000 7f72f1e47740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5634de0b9290 con 0x5634de1cc9a0 2024-09-12T02:07:39.423 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.414+0000 7f72f1e47740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5634de1d5280 con 0x5634de1cd2e0 2024-09-12T02:07:39.423 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.415+0000 7f72f1e47740 10 client.0 _ll_register_callbacks cb 0x5634de0ea140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:07:39.423 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.415+0000 7f72ed56c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634de1d2370 0x5634de1ddda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:39.423 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.415+0000 7f72ed56c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634de1d2370 0x5634de1ddda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54044/0 (socket says 172.21.15.16:54044) 2024-09-12T02:07:39.423 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.415+0000 7f72ed56c640 1 -- 172.21.15.16:0/2120172219 learned_addr learned my addr 172.21.15.16:0/2120172219 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:07:39.423 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.415+0000 7f72ecd6b640 1 --2- 172.21.15.16:0/2120172219 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5634de1cc9a0 0x5634de1d6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:39.423 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.415+0000 7f72e7fff640 1 --2- 172.21.15.16:0/2120172219 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5634de1cd2e0 0x5634de1d6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:39.423 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.415+0000 7f72ed56c640 1 -- 172.21.15.16:0/2120172219 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5634de1cd2e0 msgr2=0x5634de1d6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:39.424 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.415+0000 7f72ed56c640 1 --2- 172.21.15.16:0/2120172219 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5634de1cd2e0 0x5634de1d6dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:39.424 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.415+0000 7f72ed56c640 1 -- 172.21.15.16:0/2120172219 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5634de1cc9a0 msgr2=0x5634de1d6890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:39.424 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.415+0000 7f72ed56c640 1 --2- 172.21.15.16:0/2120172219 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5634de1cc9a0 0x5634de1d6890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:39.424 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.415+0000 7f72ed56c640 1 -- 172.21.15.16:0/2120172219 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634dde43210 con 0x5634de1d2370 2024-09-12T02:07:39.424 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.415+0000 7f72e7fff640 1 --2- 172.21.15.16:0/2120172219 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5634de1cd2e0 0x5634de1d6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:07:39.424 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.415+0000 7f72ed56c640 1 --2- 172.21.15.16:0/2120172219 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634de1d2370 0x5634de1ddda0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f72e0004010 tx=0x7f72e0004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b682a1579a5eaf3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:07:39.424 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.415+0000 7f72c7fff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:07:39.424 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.415+0000 7f72c7fff640 1 -- 172.21.15.16:0/2120172219 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72e0010bb0 con 0x5634de1d2370 2024-09-12T02:07:39.424 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.415+0000 7f72c7fff640 1 -- 172.21.15.16:0/2120172219 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f72e0010d50 con 0x5634de1d2370 2024-09-12T02:07:39.424 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.415+0000 7f72f1e47740 1 -- 172.21.15.16:0/2120172219 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5634de13da20 con 0x5634de1d2370 2024-09-12T02:07:39.424 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.416+0000 7f72c7fff640 1 -- 172.21.15.16:0/2120172219 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f72e0029440 con 0x5634de1d2370 2024-09-12T02:07:39.424 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.416+0000 7f72f1e47740 10 client.17145 fetch_fsmap learned FSMap version 25 2024-09-12T02:07:39.424 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.416+0000 7f72f1e47740 1 -- 172.21.15.16:0/2120172219 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5634ddf0f620 con 0x5634de1d2370 2024-09-12T02:07:39.425 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.416+0000 7f72c7fff640 1 -- 172.21.15.16:0/2120172219 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5634de13da20 con 0x5634de1d2370 2024-09-12T02:07:39.425 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.416+0000 7f72c7fff640 1 -- 172.21.15.16:0/2120172219 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f72e0027020 con 0x5634de1d2370 2024-09-12T02:07:39.425 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.416+0000 7f72f1e47740 10 client.17145 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:07:39.425 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.416+0000 7f72f1e47740 10 client.17145 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:07:39.425 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.416+0000 7f72f1e47740 20 client.17145 populate_metadata read hostname 'smithi016' 2024-09-12T02:07:39.425 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.416+0000 7f72f1e47740 1 --2- 172.21.15.16:0/2120172219 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5634de1e2510 0x5634de1e4920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:39.425 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.416+0000 7f72f1e47740 4 client.17145 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:07:39.425 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.416+0000 7f72f1e47740 1 -- 172.21.15.16:0/2120172219 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5634de137d70 con 0x5634de1e2510 2024-09-12T02:07:39.425 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.417+0000 7f72ecd6b640 1 --2- 172.21.15.16:0/2120172219 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5634de1e2510 0x5634de1e4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:39.425 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.417+0000 7f72ecd6b640 1 --2- 172.21.15.16:0/2120172219 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5634de1e2510 0x5634de1e4920 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=76c5b833026b19d1 server_cookie=87a756fdd7e53477 in_seq=0 out_seq=0 2024-09-12T02:07:39.425 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.417+0000 7f72c7fff640 10 client.17145 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:07:39.425 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.418+0000 7f72c7fff640 1 -- 172.21.15.16:0/2120172219 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5634de137d70 con 0x5634de1e2510 2024-09-12T02:07:39.425 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.418+0000 7f72c7fff640 10 client.17145 handle_command_reply: tid=0 2024-09-12T02:07:39.425 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.418+0000 7f72f1e47740 1 client.17145 shutdown 2024-09-12T02:07:39.426 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.418+0000 7f72f1e47740 1 -- 172.21.15.16:0/2120172219 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634de1d2370 msgr2=0x5634de1ddda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:39.426 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.418+0000 7f72f1e47740 1 --2- 172.21.15.16:0/2120172219 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634de1d2370 0x5634de1ddda0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f72e0004010 tx=0x7f72e0004ab0 comp rx=0 tx=0).stop 2024-09-12T02:07:39.426 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.418+0000 7f72f1e47740 1 -- 172.21.15.16:0/2120172219 shutdown_connections 2024-09-12T02:07:39.426 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.418+0000 7f72f1e47740 1 --2- 172.21.15.16:0/2120172219 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5634de1e2510 0x5634de1e4920 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:39.426 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.418+0000 7f72f1e47740 1 --2- 172.21.15.16:0/2120172219 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634de1d2370 0x5634de1ddda0 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:39.426 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.418+0000 7f72f1e47740 1 --2- 172.21.15.16:0/2120172219 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5634de1cd2e0 0x5634de1d6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:39.426 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.418+0000 7f72c7fff640 0 client.17145 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:07:39.426 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.418+0000 7f72f1e47740 1 --2- 172.21.15.16:0/2120172219 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5634de1cc9a0 0x5634de1d6890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:39.427 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.418+0000 7f72f1e47740 1 -- 172.21.15.16:0/2120172219 >> 172.21.15.16:0/2120172219 conn(0x5634de1c8010 msgr2=0x5634de1c9820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:07:39.427 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.418+0000 7f72f1e47740 1 -- 172.21.15.16:0/2120172219 shutdown_connections 2024-09-12T02:07:39.427 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.418+0000 7f72f1e47740 1 -- 172.21.15.16:0/2120172219 wait complete. 2024-09-12T02:07:39.427 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.418+0000 7f72f1e47740 20 client.17145 trim_cache size 0 max 16384 2024-09-12T02:07:39.434 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.433+0000 7f72f1e47740 1 Processor -- start 2024-09-12T02:07:39.435 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.433+0000 7f72f1e47740 1 -- start start 2024-09-12T02:07:39.435 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.433+0000 7f72f1e47740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5634de1c8010 0x5634de1c8ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:39.435 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.433+0000 7f72f1e47740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5634de1cc9a0 0x5634de1ddd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:39.435 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.433+0000 7f72f1e47740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634de1cd2e0 0x5634de1c9400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:39.435 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.433+0000 7f72f1e47740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f72e81ab800 con 0x5634de1cd2e0 2024-09-12T02:07:39.435 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.433+0000 7f72f1e47740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f72e81a90d0 con 0x5634de1c8010 2024-09-12T02:07:39.435 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.433+0000 7f72f1e47740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f72e81a6f20 con 0x5634de1cc9a0 2024-09-12T02:07:39.435 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.433+0000 7f72ed56c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634de1cd2e0 0x5634de1c9400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:39.435 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.433+0000 7f72ed56c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634de1cd2e0 0x5634de1c9400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54046/0 (socket says 172.21.15.16:54046) 2024-09-12T02:07:39.435 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.433+0000 7f72ed56c640 1 -- 172.21.15.16:0/670817447 learned_addr learned my addr 172.21.15.16:0/670817447 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:07:39.435 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.433+0000 7f72ecd6b640 1 --2- 172.21.15.16:0/670817447 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5634de1c8010 0x5634de1c8ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:39.435 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.433+0000 7f72e7fff640 1 --2- 172.21.15.16:0/670817447 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5634de1cc9a0 0x5634de1ddd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:39.436 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.433+0000 7f72ed56c640 1 -- 172.21.15.16:0/670817447 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5634de1cc9a0 msgr2=0x5634de1ddd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:39.436 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.433+0000 7f72ed56c640 1 --2- 172.21.15.16:0/670817447 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5634de1cc9a0 0x5634de1ddd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:39.436 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.433+0000 7f72ed56c640 1 -- 172.21.15.16:0/670817447 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5634de1c8010 msgr2=0x5634de1c8ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:39.436 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.433+0000 7f72ed56c640 1 --2- 172.21.15.16:0/670817447 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5634de1c8010 0x5634de1c8ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:39.436 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.433+0000 7f72ed56c640 1 -- 172.21.15.16:0/670817447 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634de1c7e30 con 0x5634de1cd2e0 2024-09-12T02:07:39.436 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.434+0000 7f72ed56c640 1 --2- 172.21.15.16:0/670817447 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634de1cd2e0 0x5634de1c9400 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f72e0004910 tx=0x7f72e0005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cc1023413ebdfd3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:07:39.436 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.434+0000 7f72ce7fc640 1 -- 172.21.15.16:0/670817447 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72e001d8c0 con 0x5634de1cd2e0 2024-09-12T02:07:39.436 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.434+0000 7f72ce7fc640 1 -- 172.21.15.16:0/670817447 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72e0010800 con 0x5634de1cd2e0 2024-09-12T02:07:39.436 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.434+0000 7f72ce7fc640 1 -- 172.21.15.16:0/670817447 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72e0029be0 con 0x5634de1cd2e0 2024-09-12T02:07:39.436 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.434+0000 7f72f1e47740 1 -- 172.21.15.16:0/670817447 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634de1cd2e0 msgr2=0x5634de1c9400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:39.436 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.434+0000 7f72f1e47740 1 --2- 172.21.15.16:0/670817447 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634de1cd2e0 0x5634de1c9400 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f72e0004910 tx=0x7f72e0005f10 comp rx=0 tx=0).stop 2024-09-12T02:07:39.436 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.434+0000 7f72f1e47740 1 -- 172.21.15.16:0/670817447 shutdown_connections 2024-09-12T02:07:39.436 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.434+0000 7f72f1e47740 1 --2- 172.21.15.16:0/670817447 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634de1cd2e0 0x5634de1c9400 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:39.437 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.434+0000 7f72f1e47740 1 --2- 172.21.15.16:0/670817447 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5634de1cc9a0 0x5634de1ddd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:39.437 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.434+0000 7f72f1e47740 1 --2- 172.21.15.16:0/670817447 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5634de1c8010 0x5634de1c8ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:39.437 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.434+0000 7f72f1e47740 1 -- 172.21.15.16:0/670817447 >> 172.21.15.16:0/670817447 conn(0x5634de1d1ec0 msgr2=0x5634de1d6890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:07:39.437 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.434+0000 7f72f1e47740 1 -- 172.21.15.16:0/670817447 shutdown_connections 2024-09-12T02:07:39.437 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.434+0000 7f72f1e47740 1 -- 172.21.15.16:0/670817447 wait complete. 2024-09-12T02:07:39.437 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.435+0000 7f72f1e47740 1 -- start start 2024-09-12T02:07:39.437 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.435+0000 7f72f1e47740 1 Processor -- start 2024-09-12T02:07:39.440 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.435+0000 7f72f1e47740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634de1c8010 0x5634de1db640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:39.440 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.435+0000 7f72f1e47740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5634de1cc9a0 0x5634de1dbb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:39.441 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.435+0000 7f72f1e47740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5634de1cd2e0 0x5634de1dcb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:39.441 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.435+0000 7f72f1e47740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f72e81a7590 con 0x5634de1c8010 2024-09-12T02:07:39.441 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.435+0000 7f72f1e47740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f72e81a8a00 con 0x5634de1cc9a0 2024-09-12T02:07:39.441 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.435+0000 7f72f1e47740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f72e81a7c60 con 0x5634de1cd2e0 2024-09-12T02:07:39.441 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.435+0000 7f72f1e47740 10 client.0 _ll_register_callbacks cb 0x7f72e0009e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:07:39.441 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.435+0000 7f72ecd6b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634de1c8010 0x5634de1db640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:39.441 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.435+0000 7f72ecd6b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634de1c8010 0x5634de1db640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54052/0 (socket says 172.21.15.16:54052) 2024-09-12T02:07:39.441 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.435+0000 7f72ecd6b640 1 -- 172.21.15.16:0/3571853683 learned_addr learned my addr 172.21.15.16:0/3571853683 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:07:39.441 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.435+0000 7f72e7fff640 1 --2- 172.21.15.16:0/3571853683 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5634de1cc9a0 0x5634de1dbb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:39.441 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.436+0000 7f72ed56c640 1 --2- 172.21.15.16:0/3571853683 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5634de1cd2e0 0x5634de1dcb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:39.441 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.436+0000 7f72ecd6b640 1 -- 172.21.15.16:0/3571853683 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5634de1cd2e0 msgr2=0x5634de1dcb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:39.442 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.436+0000 7f72ecd6b640 1 --2- 172.21.15.16:0/3571853683 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5634de1cd2e0 0x5634de1dcb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:39.442 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.436+0000 7f72ecd6b640 1 -- 172.21.15.16:0/3571853683 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5634de1cc9a0 msgr2=0x5634de1dbb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:39.442 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.436+0000 7f72ecd6b640 1 --2- 172.21.15.16:0/3571853683 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5634de1cc9a0 0x5634de1dbb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:39.442 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.436+0000 7f72ecd6b640 1 -- 172.21.15.16:0/3571853683 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634de1dd150 con 0x5634de1c8010 2024-09-12T02:07:39.442 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.436+0000 7f72e7fff640 1 --2- 172.21.15.16:0/3571853683 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5634de1cc9a0 0x5634de1dbb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:07:39.442 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.436+0000 7f72ed56c640 1 --2- 172.21.15.16:0/3571853683 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5634de1cd2e0 0x5634de1dcb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:07:39.442 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.436+0000 7f72ecd6b640 1 --2- 172.21.15.16:0/3571853683 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634de1c8010 0x5634de1db640 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f72d80096c0 tx=0x7f72d8009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8708077724266373 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:07:39.442 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.436+0000 7f72c7fff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:07:39.442 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.436+0000 7f72c7fff640 1 -- 172.21.15.16:0/3571853683 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72d8049040 con 0x5634de1c8010 2024-09-12T02:07:39.442 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.436+0000 7f72c7fff640 1 -- 172.21.15.16:0/3571853683 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f72d80b9e50 con 0x5634de1c8010 2024-09-12T02:07:39.442 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.436+0000 7f72f1e47740 1 -- 172.21.15.16:0/3571853683 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f72e0010800 con 0x5634de1c8010 2024-09-12T02:07:39.442 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.436+0000 7f72c7fff640 1 -- 172.21.15.16:0/3571853683 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f72d80a5040 con 0x5634de1c8010 2024-09-12T02:07:39.442 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.436+0000 7f72f1e47740 10 client.17151 fetch_fsmap learned FSMap version 25 2024-09-12T02:07:39.443 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.436+0000 7f72f1e47740 1 -- 172.21.15.16:0/3571853683 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5634de24d3f0 con 0x5634de1c8010 2024-09-12T02:07:39.443 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.436+0000 7f72c7fff640 1 -- 172.21.15.16:0/3571853683 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72e0010800 con 0x5634de1c8010 2024-09-12T02:07:39.443 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.437+0000 7f72c7fff640 1 -- 172.21.15.16:0/3571853683 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f72d80ed020 con 0x5634de1c8010 2024-09-12T02:07:39.443 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.437+0000 7f72f1e47740 10 client.17151 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:07:39.443 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.437+0000 7f72f1e47740 10 client.17151 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:07:39.443 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.437+0000 7f72f1e47740 20 client.17151 populate_metadata read hostname 'smithi016' 2024-09-12T02:07:39.443 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.437+0000 7f72f1e47740 1 --2- 172.21.15.16:0/3571853683 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5634de1d2370 0x5634de24fca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:39.443 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.437+0000 7f72f1e47740 4 client.17151 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:07:39.443 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.437+0000 7f72f1e47740 1 -- 172.21.15.16:0/3571853683 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5634de0b4900 con 0x5634de1d2370 2024-09-12T02:07:39.443 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.437+0000 7f72e7fff640 1 --2- 172.21.15.16:0/3571853683 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5634de1d2370 0x5634de24fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:39.443 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.437+0000 7f72e7fff640 1 --2- 172.21.15.16:0/3571853683 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5634de1d2370 0x5634de24fca0 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=55e01adc8087f7ec server_cookie=75a27c42268f71fc in_seq=0 out_seq=0 2024-09-12T02:07:39.443 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.437+0000 7f72c7fff640 10 client.17151 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:07:39.451 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.450+0000 7f72c7fff640 1 -- 172.21.15.16:0/3571853683 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5634de0b4900 con 0x5634de1d2370 2024-09-12T02:07:39.451 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.450+0000 7f72c7fff640 10 client.17151 handle_command_reply: tid=0 2024-09-12T02:07:39.451 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.450+0000 7f72f1e47740 1 client.17151 shutdown 2024-09-12T02:07:39.451 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.451+0000 7f72f1e47740 1 -- 172.21.15.16:0/3571853683 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634de1c8010 msgr2=0x5634de1db640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:39.451 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.451+0000 7f72f1e47740 1 --2- 172.21.15.16:0/3571853683 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634de1c8010 0x5634de1db640 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f72d80096c0 tx=0x7f72d8009260 comp rx=0 tx=0).stop 2024-09-12T02:07:39.451 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.451+0000 7f72f1e47740 1 -- 172.21.15.16:0/3571853683 shutdown_connections 2024-09-12T02:07:39.451 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.451+0000 7f72f1e47740 1 --2- 172.21.15.16:0/3571853683 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5634de1d2370 0x5634de24fca0 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:39.451 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.451+0000 7f72f1e47740 1 --2- 172.21.15.16:0/3571853683 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5634de1cd2e0 0x5634de1dcb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:39.451 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.451+0000 7f72c7fff640 0 client.17151 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:07:39.452 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.451+0000 7f72f1e47740 1 --2- 172.21.15.16:0/3571853683 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5634de1cc9a0 0x5634de1dbb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:39.452 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.451+0000 7f72f1e47740 1 --2- 172.21.15.16:0/3571853683 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634de1c8010 0x5634de1db640 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:39.452 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.451+0000 7f72f1e47740 1 -- 172.21.15.16:0/3571853683 >> 172.21.15.16:0/3571853683 conn(0x5634de1d1ec0 msgr2=0x5634de1d6890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:07:39.452 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.451+0000 7f72f1e47740 1 -- 172.21.15.16:0/3571853683 shutdown_connections 2024-09-12T02:07:39.452 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.451+0000 7f72f1e47740 1 -- 172.21.15.16:0/3571853683 wait complete. 2024-09-12T02:07:39.452 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.451+0000 7f72f1e47740 20 client.17151 trim_cache size 0 max 16384 2024-09-12T02:07:39.452 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-09-12T02:07:39.452 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-09-12T02:07:39.452 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-09-12T02:07:39.453 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-09-12T02:07:39.453 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.452+0000 7f72eefcf640 1 -- 172.21.15.16:0/2140698366 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f72d00a2cb0 msgr2=0x7f72d00a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:39.453 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.452+0000 7f72eefcf640 1 --2- 172.21.15.16:0/2140698366 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f72d00a2cb0 0x7f72d00a5170 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f72e806a980 tx=0x7f72e0004a40 comp rx=0 tx=0).stop 2024-09-12T02:07:39.453 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.452+0000 7f72eefcf640 1 -- 172.21.15.16:0/2140698366 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72e8102660 msgr2=0x7f72e8197e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:39.454 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.452+0000 7f72eefcf640 1 --2- 172.21.15.16:0/2140698366 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72e8102660 0x7f72e8197e70 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f72d8009230 tx=0x7f72d8031b60 comp rx=0 tx=0).stop 2024-09-12T02:07:39.454 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.452+0000 7f72eefcf640 1 -- 172.21.15.16:0/2140698366 shutdown_connections 2024-09-12T02:07:39.454 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.452+0000 7f72eefcf640 1 --2- 172.21.15.16:0/2140698366 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f72d00a2cb0 0x7f72d00a5170 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:39.454 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.452+0000 7f72eefcf640 1 --2- 172.21.15.16:0/2140698366 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f72e81078c0 0x7f72e819f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:39.454 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.452+0000 7f72eefcf640 1 --2- 172.21.15.16:0/2140698366 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f72e8104f90 0x7f72e81983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:39.454 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.452+0000 7f72eefcf640 1 --2- 172.21.15.16:0/2140698366 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72e8102660 0x7f72e8197e70 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:39.454 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.452+0000 7f72eefcf640 1 -- 172.21.15.16:0/2140698366 >> 172.21.15.16:0/2140698366 conn(0x7f72e80fc1e0 msgr2=0x7f72e8102fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:07:39.454 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.452+0000 7f72eefcf640 1 -- 172.21.15.16:0/2140698366 shutdown_connections 2024-09-12T02:07:39.454 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:39.452+0000 7f72eefcf640 1 -- 172.21.15.16:0/2140698366 wait complete. 2024-09-12T02:07:40.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:07:40 smithi184 ceph-mon[35593]: pgmap v507: 97 pgs: 97 active+clean; 3.1 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 11 op/s 2024-09-12T02:07:40.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:07:40 smithi033 ceph-mon[35332]: pgmap v507: 97 pgs: 97 active+clean; 3.1 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 11 op/s 2024-09-12T02:07:40.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:07:40 smithi016 ceph-mon[30947]: pgmap v507: 97 pgs: 97 active+clean; 3.1 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 11 op/s 2024-09-12T02:07:42.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:07:42 smithi184 ceph-mon[35593]: pgmap v508: 97 pgs: 97 active+clean; 3.1 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-09-12T02:07:42.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:07:42 smithi033 ceph-mon[35332]: pgmap v508: 97 pgs: 97 active+clean; 3.1 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-09-12T02:07:42.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:07:42 smithi016 ceph-mon[30947]: pgmap v508: 97 pgs: 97 active+clean; 3.1 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-09-12T02:07:44.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:07:44 smithi184 ceph-mon[35593]: pgmap v509: 97 pgs: 97 active+clean; 3.1 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-09-12T02:07:44.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:07:44 smithi033 ceph-mon[35332]: pgmap v509: 97 pgs: 97 active+clean; 3.1 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-09-12T02:07:44.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:07:44 smithi016 ceph-mon[30947]: pgmap v509: 97 pgs: 97 active+clean; 3.1 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-09-12T02:07:46.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:07:46 smithi184 ceph-mon[35593]: pgmap v510: 97 pgs: 97 active+clean; 3.2 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.8 KiB/s wr, 11 op/s 2024-09-12T02:07:46.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:07:46 smithi033 ceph-mon[35332]: pgmap v510: 97 pgs: 97 active+clean; 3.2 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.8 KiB/s wr, 11 op/s 2024-09-12T02:07:46.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:07:46 smithi016 ceph-mon[30947]: pgmap v510: 97 pgs: 97 active+clean; 3.2 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.8 KiB/s wr, 11 op/s 2024-09-12T02:07:48.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:07:48 smithi184 ceph-mon[35593]: pgmap v511: 97 pgs: 97 active+clean; 3.2 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 3.0 KiB/s wr, 9 op/s 2024-09-12T02:07:48.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:07:48 smithi033 ceph-mon[35332]: pgmap v511: 97 pgs: 97 active+clean; 3.2 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 3.0 KiB/s wr, 9 op/s 2024-09-12T02:07:48.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:07:48 smithi016 ceph-mon[30947]: pgmap v511: 97 pgs: 97 active+clean; 3.2 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 3.0 KiB/s wr, 9 op/s 2024-09-12T02:07:50.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:07:50 smithi184 ceph-mon[35593]: pgmap v512: 97 pgs: 97 active+clean; 3.2 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 2.9 KiB/s wr, 8 op/s 2024-09-12T02:07:50.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:07:50 smithi033 ceph-mon[35332]: pgmap v512: 97 pgs: 97 active+clean; 3.2 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 2.9 KiB/s wr, 8 op/s 2024-09-12T02:07:50.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:07:50 smithi016 ceph-mon[30947]: pgmap v512: 97 pgs: 97 active+clean; 3.2 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 2.9 KiB/s wr, 8 op/s 2024-09-12T02:07:51.149 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:07:51.422 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:07:51.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:51.926+0000 7f68236a1640 1 -- 172.21.15.16:0/3356131798 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f681c100b20 msgr2=0x7f681c100f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:51.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:51.926+0000 7f68236a1640 1 --2- 172.21.15.16:0/3356131798 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f681c100b20 0x7f681c100f00 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f6818009900 tx=0x7f681802a410 comp rx=0 tx=0).stop 2024-09-12T02:07:51.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:51.926+0000 7f68236a1640 1 -- 172.21.15.16:0/3356131798 shutdown_connections 2024-09-12T02:07:51.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:51.926+0000 7f68236a1640 1 --2- 172.21.15.16:0/3356131798 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f681c10e500 0x7f681c110910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:51.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:51.926+0000 7f68236a1640 1 --2- 172.21.15.16:0/3356131798 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f681c101440 0x7f681c10df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:51.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:51.926+0000 7f68236a1640 1 --2- 172.21.15.16:0/3356131798 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f681c100b20 0x7f681c100f00 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:51.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:51.926+0000 7f68236a1640 1 -- 172.21.15.16:0/3356131798 >> 172.21.15.16:0/3356131798 conn(0x7f681c0fc7d0 msgr2=0x7f681c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:07:51.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:51.926+0000 7f68236a1640 1 -- 172.21.15.16:0/3356131798 shutdown_connections 2024-09-12T02:07:51.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:51.926+0000 7f68236a1640 1 -- 172.21.15.16:0/3356131798 wait complete. 2024-09-12T02:07:51.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:51.926+0000 7f68236a1640 1 Processor -- start 2024-09-12T02:07:51.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:51.927+0000 7f68236a1640 1 -- start start 2024-09-12T02:07:51.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:51.927+0000 7f68236a1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f681c100b20 0x7f681c1a03f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:51.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:51.927+0000 7f68236a1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f681c101440 0x7f681c1a0930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:51.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:51.927+0000 7f68236a1640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f681c10e500 0x7f681c19a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:51.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:51.927+0000 7f68236a1640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f681c114280 con 0x7f681c100b20 2024-09-12T02:07:51.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:51.927+0000 7f68236a1640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f681c114100 con 0x7f681c101440 2024-09-12T02:07:51.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:51.927+0000 7f68236a1640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f681c114400 con 0x7f681c10e500 2024-09-12T02:07:51.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:51.927+0000 7f682143d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f681c100b20 0x7f681c1a03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:51.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:51.927+0000 7f682143d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f681c100b20 0x7f681c1a03f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35502/0 (socket says 172.21.15.16:35502) 2024-09-12T02:07:51.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:51.927+0000 7f682143d640 1 -- 172.21.15.16:0/2625741878 learned_addr learned my addr 172.21.15.16:0/2625741878 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:07:51.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:51.927+0000 7f6820c3c640 1 --2- 172.21.15.16:0/2625741878 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f681c101440 0x7f681c1a0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:51.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:51.927+0000 7f6821c3e640 1 --2- 172.21.15.16:0/2625741878 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f681c10e500 0x7f681c19a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:51.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:51.927+0000 7f682143d640 1 -- 172.21.15.16:0/2625741878 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f681c10e500 msgr2=0x7f681c19a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:51.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:51.927+0000 7f682143d640 1 --2- 172.21.15.16:0/2625741878 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f681c10e500 0x7f681c19a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:51.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:51.927+0000 7f682143d640 1 -- 172.21.15.16:0/2625741878 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f681c101440 msgr2=0x7f681c1a0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:51.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:51.927+0000 7f682143d640 1 --2- 172.21.15.16:0/2625741878 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f681c101440 0x7f681c1a0930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:51.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:51.927+0000 7f682143d640 1 -- 172.21.15.16:0/2625741878 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f681c19ace0 con 0x7f681c100b20 2024-09-12T02:07:51.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:51.928+0000 7f682143d640 1 --2- 172.21.15.16:0/2625741878 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f681c100b20 0x7f681c1a03f0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f6818006010 tx=0x7f68180042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:07:51.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:51.928+0000 7f68127fc640 1 -- 172.21.15.16:0/2625741878 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f681803f030 con 0x7f681c100b20 2024-09-12T02:07:51.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:51.928+0000 7f68127fc640 1 -- 172.21.15.16:0/2625741878 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6818003ce0 con 0x7f681c100b20 2024-09-12T02:07:51.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:51.928+0000 7f68127fc640 1 -- 172.21.15.16:0/2625741878 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6818035070 con 0x7f681c100b20 2024-09-12T02:07:51.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:51.928+0000 7f68236a1640 1 -- 172.21.15.16:0/2625741878 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f681c19af70 con 0x7f681c100b20 2024-09-12T02:07:51.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:51.928+0000 7f68236a1640 1 -- 172.21.15.16:0/2625741878 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f681c1a72c0 con 0x7f681c100b20 2024-09-12T02:07:51.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:51.929+0000 7f68236a1640 1 -- 172.21.15.16:0/2625741878 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f681c06a850 con 0x7f681c100b20 2024-09-12T02:07:51.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:51.931+0000 7f68127fc640 1 -- 172.21.15.16:0/2625741878 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6818042030 con 0x7f681c100b20 2024-09-12T02:07:51.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:51.932+0000 7f68127fc640 1 --2- 172.21.15.16:0/2625741878 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f68080a2c00 0x7f68080a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:51.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:51.932+0000 7f6820c3c640 1 --2- 172.21.15.16:0/2625741878 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f68080a2c00 0x7f68080a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:51.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:51.932+0000 7f68127fc640 1 -- 172.21.15.16:0/2625741878 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f68180ef6b0 con 0x7f681c100b20 2024-09-12T02:07:51.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:51.932+0000 7f6820c3c640 1 --2- 172.21.15.16:0/2625741878 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f68080a2c00 0x7f68080a50c0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f680c0095f0 tx=0x7f680c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:07:51.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:51.936+0000 7f68127fc640 1 -- 172.21.15.16:0/2625741878 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f68180bc080 con 0x7f681c100b20 2024-09-12T02:07:52.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:52.148+0000 7f68236a1640 1 -- 172.21.15.16:0/2625741878 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f681c109600 con 0x7f681c100b20 2024-09-12T02:07:52.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:52.149+0000 7f68127fc640 1 -- 172.21.15.16:0/2625741878 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11911 (secure 0 0 0) 0x7f68180c0380 con 0x7f681c100b20 2024-09-12T02:07:52.149 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T02:07:52.149 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":25,"btime":"2024-09-12T02:01:37:192021+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3279123279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3279123279},{"type":"v1","addr":"172.21.15.16:6835","nonce":3279123279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4119452382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4119452382},{"type":"v1","addr":"172.21.15.16:6837","nonce":4119452382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/3263823165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3263823165},{"type":"v1","addr":"172.21.15.16:6839","nonce":3263823165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/1510290732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":1510290732},{"type":"v1","addr":"172.21.15.16:6841","nonce":1510290732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/279674603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":279674603},{"type":"v1","addr":"172.21.15.33:6833","nonce":279674603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3555728028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3555728028},{"type":"v1","addr":"172.21.15.33:6835","nonce":3555728028}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T01:59:56.117608+0000","modified":"2024-09-12T02:01:36.294760+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24482,"mds_1":24481,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6833/1563374106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1563374106},{"type":"v1","addr":"172.21.15.184:6833","nonce":1563374106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6837/1892833868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1892833868},{"type":"v1","addr":"172.21.15.33:6837","nonce":1892833868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/1457679686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1457679686},{"type":"v1","addr":"172.21.15.184:6835","nonce":1457679686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.184:6837/1793393087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":1793393087},{"type":"v1","addr":"172.21.15.184:6837","nonce":1793393087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.184:6839/4203347941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":4203347941},{"type":"v1","addr":"172.21.15.184:6839","nonce":4203347941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.33:6839/4136670204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":4136670204},{"type":"v1","addr":"172.21.15.33:6839","nonce":4136670204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24481,24482,24469]},"id":1}]} 2024-09-12T02:07:52.149 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 25 2024-09-12T02:07:52.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:52.151+0000 7f68236a1640 1 -- 172.21.15.16:0/2625741878 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f68080a2c00 msgr2=0x7f68080a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:52.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:52.151+0000 7f68236a1640 1 --2- 172.21.15.16:0/2625741878 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f68080a2c00 0x7f68080a50c0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f680c0095f0 tx=0x7f680c005f90 comp rx=0 tx=0).stop 2024-09-12T02:07:52.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:52.151+0000 7f68236a1640 1 -- 172.21.15.16:0/2625741878 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f681c100b20 msgr2=0x7f681c1a03f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:52.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:52.151+0000 7f68236a1640 1 --2- 172.21.15.16:0/2625741878 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f681c100b20 0x7f681c1a03f0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f6818006010 tx=0x7f68180042f0 comp rx=0 tx=0).stop 2024-09-12T02:07:52.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:52.151+0000 7f68236a1640 1 -- 172.21.15.16:0/2625741878 shutdown_connections 2024-09-12T02:07:52.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:52.151+0000 7f68236a1640 1 --2- 172.21.15.16:0/2625741878 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f68080a2c00 0x7f68080a50c0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:52.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:52.151+0000 7f68236a1640 1 --2- 172.21.15.16:0/2625741878 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f681c10e500 0x7f681c19a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:52.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:52.151+0000 7f68236a1640 1 --2- 172.21.15.16:0/2625741878 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f681c101440 0x7f681c1a0930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:52.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:52.151+0000 7f68236a1640 1 --2- 172.21.15.16:0/2625741878 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f681c100b20 0x7f681c1a03f0 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:52.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:52.151+0000 7f68236a1640 1 -- 172.21.15.16:0/2625741878 >> 172.21.15.16:0/2625741878 conn(0x7f681c0fc7d0 msgr2=0x7f681c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:07:52.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:52.151+0000 7f68236a1640 1 -- 172.21.15.16:0/2625741878 shutdown_connections 2024-09-12T02:07:52.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:52.151+0000 7f68236a1640 1 -- 172.21.15.16:0/2625741878 wait complete. 2024-09-12T02:07:52.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:07:52 smithi016 ceph-mon[30947]: pgmap v513: 97 pgs: 97 active+clean; 3.2 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.3 KiB/s wr, 10 op/s 2024-09-12T02:07:52.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:07:52 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/2625741878' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:07:52.544 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.544+0000 7fe719827640 1 -- 172.21.15.16:0/3099958631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe714100d10 msgr2=0x7fe714101170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:52.544 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.544+0000 7fe719827640 1 --2- 172.21.15.16:0/3099958631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe714100d10 0x7fe714101170 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7fe708009900 tx=0x7fe70802a410 comp rx=0 tx=0).stop 2024-09-12T02:07:52.544 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.544+0000 7fe719827640 1 -- 172.21.15.16:0/3099958631 shutdown_connections 2024-09-12T02:07:52.544 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.544+0000 7fe719827640 1 --2- 172.21.15.16:0/3099958631 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7141016b0 0x7fe714101b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:52.544 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.544+0000 7fe719827640 1 --2- 172.21.15.16:0/3099958631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe714100d10 0x7fe714101170 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:52.545 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.544+0000 7fe719827640 1 --2- 172.21.15.16:0/3099958631 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe7141003d0 0x7fe7141007d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:52.545 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.544+0000 7fe719827640 1 -- 172.21.15.16:0/3099958631 >> 172.21.15.16:0/3099958631 conn(0x7fe7140fc1e0 msgr2=0x7fe7140fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:07:52.545 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.544+0000 7fe719827640 1 -- 172.21.15.16:0/3099958631 shutdown_connections 2024-09-12T02:07:52.545 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.544+0000 7fe719827640 1 -- 172.21.15.16:0/3099958631 wait complete. 2024-09-12T02:07:52.545 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.544+0000 7fe719827640 1 Processor -- start 2024-09-12T02:07:52.547 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.545+0000 7fe719827640 1 -- start start 2024-09-12T02:07:52.547 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.545+0000 7fe719827640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7141003d0 0x7fe71419a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:52.547 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.545+0000 7fe719827640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe714100d10 0x7fe71419a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:52.547 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.545+0000 7fe719827640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe7141016b0 0x7fe7141a1600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:52.547 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.545+0000 7fe719827640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe71410f9d0 con 0x7fe7141003d0 2024-09-12T02:07:52.547 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.545+0000 7fe719827640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe71410f850 con 0x7fe714100d10 2024-09-12T02:07:52.547 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.545+0000 7fe719827640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe71410fb50 con 0x7fe7141016b0 2024-09-12T02:07:52.547 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.545+0000 7fe712ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7141003d0 0x7fe71419a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:52.547 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.545+0000 7fe712ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7141003d0 0x7fe71419a050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35520/0 (socket says 172.21.15.16:35520) 2024-09-12T02:07:52.547 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.545+0000 7fe712ffd640 1 -- 172.21.15.16:0/4162621149 learned_addr learned my addr 172.21.15.16:0/4162621149 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:07:52.548 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.545+0000 7fe7137fe640 1 --2- 172.21.15.16:0/4162621149 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe7141016b0 0x7fe7141a1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:52.548 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.545+0000 7fe7127fc640 1 --2- 172.21.15.16:0/4162621149 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe714100d10 0x7fe71419a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:52.548 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.545+0000 7fe712ffd640 1 -- 172.21.15.16:0/4162621149 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe7141016b0 msgr2=0x7fe7141a1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:52.548 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.545+0000 7fe712ffd640 1 --2- 172.21.15.16:0/4162621149 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe7141016b0 0x7fe7141a1600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:52.548 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.545+0000 7fe712ffd640 1 -- 172.21.15.16:0/4162621149 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe714100d10 msgr2=0x7fe71419a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:52.548 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.545+0000 7fe712ffd640 1 --2- 172.21.15.16:0/4162621149 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe714100d10 0x7fe71419a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:52.548 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.545+0000 7fe712ffd640 1 -- 172.21.15.16:0/4162621149 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7141a1c70 con 0x7fe7141003d0 2024-09-12T02:07:52.548 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.545+0000 7fe712ffd640 1 --2- 172.21.15.16:0/4162621149 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7141003d0 0x7fe71419a050 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7fe70400c970 tx=0x7fe70400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:07:52.548 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.546+0000 7fe718825640 1 -- 172.21.15.16:0/4162621149 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe704007bb0 con 0x7fe7141003d0 2024-09-12T02:07:52.548 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.546+0000 7fe718825640 1 -- 172.21.15.16:0/4162621149 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe704007d50 con 0x7fe7141003d0 2024-09-12T02:07:52.548 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.546+0000 7fe718825640 1 -- 172.21.15.16:0/4162621149 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe70400f450 con 0x7fe7141003d0 2024-09-12T02:07:52.548 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.546+0000 7fe719827640 1 -- 172.21.15.16:0/4162621149 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7141a1f60 con 0x7fe7141003d0 2024-09-12T02:07:52.548 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.546+0000 7fe719827640 1 -- 172.21.15.16:0/4162621149 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7141a2370 con 0x7fe7141003d0 2024-09-12T02:07:52.555 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.548+0000 7fe71c5c6740 1 Processor -- start 2024-09-12T02:07:52.555 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.548+0000 7fe71c5c6740 1 -- start start 2024-09-12T02:07:52.555 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.548+0000 7fe718825640 1 -- 172.21.15.16:0/4162621149 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fe704005020 con 0x7fe7141003d0 2024-09-12T02:07:52.556 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.548+0000 7fe71c5c6740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5623345f99a0 0x5623345f9da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:52.556 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.548+0000 7fe71c5c6740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5623345fa2e0 0x5623345feda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:52.556 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.548+0000 7fe71c5c6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5623345ff370 0x562334601830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:52.556 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.548+0000 7fe71c5c6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56233455d2f0 con 0x5623345ff370 2024-09-12T02:07:52.556 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.548+0000 7fe71c5c6740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x562334505270 con 0x5623345f99a0 2024-09-12T02:07:52.556 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.548+0000 7fe71c5c6740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5623345592b0 con 0x5623345fa2e0 2024-09-12T02:07:52.556 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.548+0000 7fe7127fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5623345ff370 0x562334601830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:52.556 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.549+0000 7fe7127fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5623345ff370 0x562334601830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35530/0 (socket says 172.21.15.16:35530) 2024-09-12T02:07:52.556 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.549+0000 7fe7127fc640 1 -- 172.21.15.16:0/2108735174 learned_addr learned my addr 172.21.15.16:0/2108735174 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:07:52.557 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.549+0000 7fe7137fe640 1 --2- 172.21.15.16:0/2108735174 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5623345f99a0 0x5623345f9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:52.557 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.549+0000 7fe712ffd640 1 --2- 172.21.15.16:0/2108735174 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5623345fa2e0 0x5623345feda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:52.557 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.549+0000 7fe7127fc640 1 -- 172.21.15.16:0/2108735174 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5623345fa2e0 msgr2=0x5623345feda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:52.557 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.549+0000 7fe7127fc640 1 --2- 172.21.15.16:0/2108735174 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5623345fa2e0 0x5623345feda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:52.557 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.549+0000 7fe7127fc640 1 -- 172.21.15.16:0/2108735174 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5623345f99a0 msgr2=0x5623345f9da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:52.557 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.549+0000 7fe7127fc640 1 --2- 172.21.15.16:0/2108735174 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5623345f99a0 0x5623345f9da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:52.557 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.549+0000 7fe7127fc640 1 -- 172.21.15.16:0/2108735174 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562334504340 con 0x5623345ff370 2024-09-12T02:07:52.557 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.549+0000 7fe7137fe640 1 --2- 172.21.15.16:0/2108735174 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5623345f99a0 0x5623345f9da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:07:52.557 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.549+0000 7fe718825640 1 --2- 172.21.15.16:0/4162621149 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fe6fc0a2cb0 0x7fe6fc0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:52.557 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.549+0000 7fe7137fe640 1 --2- 172.21.15.16:0/4162621149 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fe6fc0a2cb0 0x7fe6fc0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:52.557 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.549+0000 7fe7127fc640 1 --2- 172.21.15.16:0/2108735174 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5623345ff370 0x562334601830 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7fe708031610 tx=0x7fe708005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1faadd8196bdefb7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:07:52.557 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.549+0000 7fe718825640 1 -- 172.21.15.16:0/4162621149 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fe7040cec60 con 0x7fe7141003d0 2024-09-12T02:07:52.557 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.549+0000 7fe6f9ffb640 1 -- 172.21.15.16:0/2108735174 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe708040070 con 0x5623345ff370 2024-09-12T02:07:52.558 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.549+0000 7fe6f9ffb640 1 -- 172.21.15.16:0/2108735174 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe708005500 con 0x5623345ff370 2024-09-12T02:07:52.558 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.549+0000 7fe6f9ffb640 1 -- 172.21.15.16:0/2108735174 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe708035040 con 0x5623345ff370 2024-09-12T02:07:52.558 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.550+0000 7fe7137fe640 1 --2- 172.21.15.16:0/4162621149 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fe6fc0a2cb0 0x7fe6fc0a5170 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fe70c00afd0 tx=0x7fe70c00a4d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:07:52.558 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.550+0000 7fe71c5c6740 1 -- 172.21.15.16:0/2108735174 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5623345ff370 msgr2=0x562334601830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:52.558 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.550+0000 7fe71c5c6740 1 --2- 172.21.15.16:0/2108735174 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5623345ff370 0x562334601830 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7fe708031610 tx=0x7fe708005c00 comp rx=0 tx=0).stop 2024-09-12T02:07:52.558 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.550+0000 7fe71c5c6740 1 -- 172.21.15.16:0/2108735174 shutdown_connections 2024-09-12T02:07:52.558 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.550+0000 7fe71c5c6740 1 --2- 172.21.15.16:0/2108735174 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5623345ff370 0x562334601830 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:52.558 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.550+0000 7fe71c5c6740 1 --2- 172.21.15.16:0/2108735174 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5623345fa2e0 0x5623345feda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:52.558 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.550+0000 7fe71c5c6740 1 --2- 172.21.15.16:0/2108735174 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5623345f99a0 0x5623345f9da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:52.559 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.550+0000 7fe71c5c6740 1 -- 172.21.15.16:0/2108735174 >> 172.21.15.16:0/2108735174 conn(0x5623345f5010 msgr2=0x5623345f5410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:07:52.559 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.550+0000 7fe71c5c6740 1 -- 172.21.15.16:0/2108735174 shutdown_connections 2024-09-12T02:07:52.559 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.550+0000 7fe71c5c6740 1 -- 172.21.15.16:0/2108735174 wait complete. 2024-09-12T02:07:52.559 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.551+0000 7fe71c5c6740 1 -- start start 2024-09-12T02:07:52.559 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.551+0000 7fe71c5c6740 1 Processor -- start 2024-09-12T02:07:52.559 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.551+0000 7fe71c5c6740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5623345f99a0 0x562334603b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:52.559 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.551+0000 7fe71c5c6740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5623345fa2e0 0x562334604050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:52.559 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.551+0000 7fe71c5c6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5623345ff370 0x56233460b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:52.559 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.551+0000 7fe71c5c6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5623346022e0 con 0x5623345ff370 2024-09-12T02:07:52.559 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.551+0000 7fe71c5c6740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5623344e6290 con 0x5623345f99a0 2024-09-12T02:07:52.559 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.551+0000 7fe71c5c6740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x562334602460 con 0x5623345fa2e0 2024-09-12T02:07:52.559 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.551+0000 7fe71c5c6740 10 client.0 _ll_register_callbacks cb 0x562334517140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:07:52.559 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.551+0000 7fe7137fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5623345ff370 0x56233460b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:52.560 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.551+0000 7fe7137fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5623345ff370 0x56233460b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35546/0 (socket says 172.21.15.16:35546) 2024-09-12T02:07:52.560 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.551+0000 7fe7137fe640 1 -- 172.21.15.16:0/3775657654 learned_addr learned my addr 172.21.15.16:0/3775657654 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:07:52.560 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.551+0000 7fe712ffd640 1 --2- 172.21.15.16:0/3775657654 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5623345f99a0 0x562334603b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:52.560 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.551+0000 7fe7127fc640 1 --2- 172.21.15.16:0/3775657654 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5623345fa2e0 0x562334604050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:52.560 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.552+0000 7fe7137fe640 1 -- 172.21.15.16:0/3775657654 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5623345fa2e0 msgr2=0x562334604050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:52.560 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.552+0000 7fe7137fe640 1 --2- 172.21.15.16:0/3775657654 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5623345fa2e0 0x562334604050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:52.560 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.552+0000 7fe7137fe640 1 -- 172.21.15.16:0/3775657654 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5623345f99a0 msgr2=0x562334603b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:52.560 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.552+0000 7fe7137fe640 1 --2- 172.21.15.16:0/3775657654 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5623345f99a0 0x562334603b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:52.560 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.552+0000 7fe7137fe640 1 -- 172.21.15.16:0/3775657654 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562334270210 con 0x5623345ff370 2024-09-12T02:07:52.561 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.552+0000 7fe7137fe640 1 --2- 172.21.15.16:0/3775657654 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5623345ff370 0x56233460b090 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7fe70c00fd20 tx=0x7fe70c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d27f181de21b523 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:07:52.561 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.552+0000 7fe6f2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:07:52.561 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.552+0000 7fe6f2ffd640 1 -- 172.21.15.16:0/3775657654 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe70c00fe10 con 0x5623345ff370 2024-09-12T02:07:52.561 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.552+0000 7fe6f2ffd640 1 -- 172.21.15.16:0/3775657654 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fe70c016030 con 0x5623345ff370 2024-09-12T02:07:52.561 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.552+0000 7fe71c5c6740 1 -- 172.21.15.16:0/3775657654 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe708005500 con 0x5623345ff370 2024-09-12T02:07:52.561 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.552+0000 7fe6f2ffd640 1 -- 172.21.15.16:0/3775657654 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe70c003970 con 0x5623345ff370 2024-09-12T02:07:52.561 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.552+0000 7fe71c5c6740 10 client.17169 fetch_fsmap learned FSMap version 25 2024-09-12T02:07:52.561 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.552+0000 7fe71c5c6740 1 -- 172.21.15.16:0/3775657654 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56233433c620 con 0x5623345ff370 2024-09-12T02:07:52.561 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.552+0000 7fe6f2ffd640 1 -- 172.21.15.16:0/3775657654 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe708005500 con 0x5623345ff370 2024-09-12T02:07:52.561 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.553+0000 7fe6f2ffd640 1 -- 172.21.15.16:0/3775657654 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fe70c004230 con 0x5623345ff370 2024-09-12T02:07:52.562 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.553+0000 7fe71c5c6740 10 client.17169 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:07:52.562 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.553+0000 7fe71c5c6740 10 client.17169 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:07:52.562 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.553+0000 7fe71c5c6740 20 client.17169 populate_metadata read hostname 'smithi016' 2024-09-12T02:07:52.562 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.553+0000 7fe71c5c6740 1 --2- 172.21.15.16:0/3775657654 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x56233460f850 0x562334611c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:52.562 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.553+0000 7fe71c5c6740 4 client.17169 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:07:52.562 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.553+0000 7fe71c5c6740 1 -- 172.21.15.16:0/3775657654 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56233456aa20 con 0x56233460f850 2024-09-12T02:07:52.562 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.553+0000 7fe712ffd640 1 --2- 172.21.15.16:0/3775657654 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x56233460f850 0x562334611c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:52.562 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.554+0000 7fe6f2ffd640 10 client.17169 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:07:52.562 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.554+0000 7fe712ffd640 1 --2- 172.21.15.16:0/3775657654 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x56233460f850 0x562334611c40 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32b8db9ec1d2904a server_cookie=13866b836e1f4e in_seq=0 out_seq=0 2024-09-12T02:07:52.562 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.554+0000 7fe6f2ffd640 1 -- 172.21.15.16:0/3775657654 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56233456aa20 con 0x56233460f850 2024-09-12T02:07:52.563 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.554+0000 7fe6f2ffd640 10 client.17169 handle_command_reply: tid=0 2024-09-12T02:07:52.563 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.554+0000 7fe71c5c6740 1 client.17169 shutdown 2024-09-12T02:07:52.563 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.554+0000 7fe71c5c6740 1 -- 172.21.15.16:0/3775657654 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5623345ff370 msgr2=0x56233460b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:52.563 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.554+0000 7fe71c5c6740 1 --2- 172.21.15.16:0/3775657654 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5623345ff370 0x56233460b090 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7fe70c00fd20 tx=0x7fe70c004ab0 comp rx=0 tx=0).stop 2024-09-12T02:07:52.563 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.554+0000 7fe71c5c6740 1 -- 172.21.15.16:0/3775657654 shutdown_connections 2024-09-12T02:07:52.563 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.554+0000 7fe71c5c6740 1 --2- 172.21.15.16:0/3775657654 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x56233460f850 0x562334611c40 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:52.563 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.555+0000 7fe71c5c6740 1 --2- 172.21.15.16:0/3775657654 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5623345ff370 0x56233460b090 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:52.563 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.555+0000 7fe6f2ffd640 0 client.17169 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:07:52.563 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.555+0000 7fe71c5c6740 1 --2- 172.21.15.16:0/3775657654 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5623345fa2e0 0x562334604050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:52.563 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.555+0000 7fe71c5c6740 1 --2- 172.21.15.16:0/3775657654 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5623345f99a0 0x562334603b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:52.564 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.555+0000 7fe71c5c6740 1 -- 172.21.15.16:0/3775657654 >> 172.21.15.16:0/3775657654 conn(0x5623345f5010 msgr2=0x5623345f6820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:07:52.564 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.555+0000 7fe71c5c6740 1 -- 172.21.15.16:0/3775657654 shutdown_connections 2024-09-12T02:07:52.564 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.555+0000 7fe71c5c6740 1 -- 172.21.15.16:0/3775657654 wait complete. 2024-09-12T02:07:52.564 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.555+0000 7fe71c5c6740 20 client.17169 trim_cache size 0 max 16384 2024-09-12T02:07:52.570 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.569+0000 7fe71c5c6740 1 Processor -- start 2024-09-12T02:07:52.571 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.569+0000 7fe71c5c6740 1 -- start start 2024-09-12T02:07:52.571 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.569+0000 7fe71c5c6740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5623345f99a0 0x5623345f5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:52.571 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.569+0000 7fe71c5c6740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5623345fa2e0 0x56233460b050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:52.571 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.569+0000 7fe71c5c6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5623345ff370 0x5623345f6400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:52.571 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.569+0000 7fe71c5c6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe7141ad920 con 0x5623345ff370 2024-09-12T02:07:52.571 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.569+0000 7fe71c5c6740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe7141ab1f0 con 0x5623345f99a0 2024-09-12T02:07:52.571 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.569+0000 7fe71c5c6740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe7141a9040 con 0x5623345fa2e0 2024-09-12T02:07:52.571 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.570+0000 7fe7137fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5623345ff370 0x5623345f6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:52.571 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.570+0000 7fe7137fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5623345ff370 0x5623345f6400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35550/0 (socket says 172.21.15.16:35550) 2024-09-12T02:07:52.571 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.570+0000 7fe7137fe640 1 -- 172.21.15.16:0/126976698 learned_addr learned my addr 172.21.15.16:0/126976698 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:07:52.571 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.570+0000 7fe712ffd640 1 --2- 172.21.15.16:0/126976698 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5623345f99a0 0x5623345f5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:52.571 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.570+0000 7fe7127fc640 1 --2- 172.21.15.16:0/126976698 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5623345fa2e0 0x56233460b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:52.571 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.570+0000 7fe7137fe640 1 -- 172.21.15.16:0/126976698 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5623345fa2e0 msgr2=0x56233460b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:52.572 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.570+0000 7fe7137fe640 1 --2- 172.21.15.16:0/126976698 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5623345fa2e0 0x56233460b050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:52.572 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.570+0000 7fe7137fe640 1 -- 172.21.15.16:0/126976698 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5623345f99a0 msgr2=0x5623345f5ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:52.572 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.570+0000 7fe7137fe640 1 --2- 172.21.15.16:0/126976698 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5623345f99a0 0x5623345f5ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:52.572 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.570+0000 7fe7137fe640 1 -- 172.21.15.16:0/126976698 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623346025b0 con 0x5623345ff370 2024-09-12T02:07:52.572 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.570+0000 7fe7137fe640 1 --2- 172.21.15.16:0/126976698 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5623345ff370 0x5623345f6400 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fe70c0046a0 tx=0x7fe70c004910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=404f1b7b11ef2b1e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:07:52.572 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.570+0000 7fe6f97fa640 1 -- 172.21.15.16:0/126976698 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe70c03e020 con 0x5623345ff370 2024-09-12T02:07:52.572 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.570+0000 7fe6f97fa640 1 -- 172.21.15.16:0/126976698 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe70c00fe10 con 0x5623345ff370 2024-09-12T02:07:52.572 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.570+0000 7fe6f97fa640 1 -- 172.21.15.16:0/126976698 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe70c036040 con 0x5623345ff370 2024-09-12T02:07:52.576 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.571+0000 7fe71c5c6740 1 -- 172.21.15.16:0/126976698 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5623345ff370 msgr2=0x5623345f6400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:52.576 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.571+0000 7fe71c5c6740 1 --2- 172.21.15.16:0/126976698 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5623345ff370 0x5623345f6400 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fe70c0046a0 tx=0x7fe70c004910 comp rx=0 tx=0).stop 2024-09-12T02:07:52.576 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.571+0000 7fe71c5c6740 1 -- 172.21.15.16:0/126976698 shutdown_connections 2024-09-12T02:07:52.576 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.571+0000 7fe71c5c6740 1 --2- 172.21.15.16:0/126976698 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5623345ff370 0x5623345f6400 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:52.576 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.571+0000 7fe71c5c6740 1 --2- 172.21.15.16:0/126976698 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5623345fa2e0 0x56233460b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:52.576 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.571+0000 7fe71c5c6740 1 --2- 172.21.15.16:0/126976698 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5623345f99a0 0x5623345f5ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:52.576 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.571+0000 7fe71c5c6740 1 -- 172.21.15.16:0/126976698 >> 172.21.15.16:0/126976698 conn(0x5623345f5010 msgr2=0x562334603b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:07:52.576 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.571+0000 7fe71c5c6740 1 -- 172.21.15.16:0/126976698 shutdown_connections 2024-09-12T02:07:52.576 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.571+0000 7fe71c5c6740 1 -- 172.21.15.16:0/126976698 wait complete. 2024-09-12T02:07:52.576 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.571+0000 7fe71c5c6740 1 -- start start 2024-09-12T02:07:52.577 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.572+0000 7fe71c5c6740 1 Processor -- start 2024-09-12T02:07:52.577 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.572+0000 7fe71c5c6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5623345f99a0 0x562334608970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:52.577 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.572+0000 7fe71c5c6740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5623345fa2e0 0x562334608eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:52.577 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.572+0000 7fe71c5c6740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5623345ff370 0x562334609e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:52.577 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.572+0000 7fe71c5c6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe7141a96b0 con 0x5623345f99a0 2024-09-12T02:07:52.577 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.572+0000 7fe71c5c6740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe7141aab20 con 0x5623345fa2e0 2024-09-12T02:07:52.577 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.572+0000 7fe71c5c6740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe7141a9d80 con 0x5623345ff370 2024-09-12T02:07:52.577 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.572+0000 7fe71c5c6740 10 client.0 _ll_register_callbacks cb 0x7fe70c00adf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:07:52.577 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.572+0000 7fe712ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5623345f99a0 0x562334608970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:52.577 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.572+0000 7fe712ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5623345f99a0 0x562334608970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35562/0 (socket says 172.21.15.16:35562) 2024-09-12T02:07:52.577 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.572+0000 7fe712ffd640 1 -- 172.21.15.16:0/2727131365 learned_addr learned my addr 172.21.15.16:0/2727131365 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:07:52.577 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.572+0000 7fe7127fc640 1 --2- 172.21.15.16:0/2727131365 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5623345fa2e0 0x562334608eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:52.577 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.572+0000 7fe7137fe640 1 --2- 172.21.15.16:0/2727131365 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5623345ff370 0x562334609e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:52.577 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.572+0000 7fe712ffd640 1 -- 172.21.15.16:0/2727131365 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5623345ff370 msgr2=0x562334609e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:52.578 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.572+0000 7fe712ffd640 1 --2- 172.21.15.16:0/2727131365 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5623345ff370 0x562334609e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:52.578 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.572+0000 7fe712ffd640 1 -- 172.21.15.16:0/2727131365 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5623345fa2e0 msgr2=0x562334608eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:52.578 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.572+0000 7fe712ffd640 1 --2- 172.21.15.16:0/2727131365 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5623345fa2e0 0x562334608eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:52.578 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.572+0000 7fe712ffd640 1 -- 172.21.15.16:0/2727131365 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56233460a460 con 0x5623345f99a0 2024-09-12T02:07:52.578 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.573+0000 7fe712ffd640 1 --2- 172.21.15.16:0/2727131365 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5623345f99a0 0x562334608970 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fe7040058c0 tx=0x7fe704004a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2086c1485c1235b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:07:52.578 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.573+0000 7fe6f2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:07:52.578 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.573+0000 7fe6f2ffd640 1 -- 172.21.15.16:0/2727131365 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe70401d040 con 0x5623345f99a0 2024-09-12T02:07:52.578 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.573+0000 7fe6f2ffd640 1 -- 172.21.15.16:0/2727131365 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fe704099910 con 0x5623345f99a0 2024-09-12T02:07:52.578 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.573+0000 7fe71c5c6740 1 -- 172.21.15.16:0/2727131365 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe70c00fe10 con 0x5623345f99a0 2024-09-12T02:07:52.578 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.573+0000 7fe6f2ffd640 1 -- 172.21.15.16:0/2727131365 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe70409b610 con 0x5623345f99a0 2024-09-12T02:07:52.578 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.573+0000 7fe71c5c6740 10 client.17175 fetch_fsmap learned FSMap version 25 2024-09-12T02:07:52.578 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.573+0000 7fe71c5c6740 1 -- 172.21.15.16:0/2727131365 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56233467a780 con 0x5623345f99a0 2024-09-12T02:07:52.578 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.573+0000 7fe6f2ffd640 1 -- 172.21.15.16:0/2727131365 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe70c00fe10 con 0x5623345f99a0 2024-09-12T02:07:52.579 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.573+0000 7fe6f2ffd640 1 -- 172.21.15.16:0/2727131365 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fe704084ba0 con 0x5623345f99a0 2024-09-12T02:07:52.579 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.573+0000 7fe71c5c6740 10 client.17175 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:07:52.579 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.573+0000 7fe71c5c6740 10 client.17175 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:07:52.579 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.573+0000 7fe71c5c6740 20 client.17175 populate_metadata read hostname 'smithi016' 2024-09-12T02:07:52.579 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.573+0000 7fe71c5c6740 1 --2- 172.21.15.16:0/2727131365 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x56233467b0b0 0x56233467d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:52.579 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.573+0000 7fe71c5c6740 4 client.17175 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:07:52.579 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.573+0000 7fe71c5c6740 1 -- 172.21.15.16:0/2727131365 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562334564d70 con 0x56233467b0b0 2024-09-12T02:07:52.580 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.574+0000 7fe7127fc640 1 --2- 172.21.15.16:0/2727131365 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x56233467b0b0 0x56233467d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:52.580 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.574+0000 7fe6f2ffd640 10 client.17175 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:07:52.580 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.574+0000 7fe7127fc640 1 --2- 172.21.15.16:0/2727131365 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x56233467b0b0 0x56233467d4d0 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90d50161dd49ec6f server_cookie=8620425d474b1b01 in_seq=0 out_seq=0 2024-09-12T02:07:52.585 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.585+0000 7fe6f2ffd640 1 -- 172.21.15.16:0/2727131365 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562334564d70 con 0x56233467b0b0 2024-09-12T02:07:52.585 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.585+0000 7fe6f2ffd640 10 client.17175 handle_command_reply: tid=0 2024-09-12T02:07:52.585 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.585+0000 7fe71c5c6740 1 client.17175 shutdown 2024-09-12T02:07:52.586 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.585+0000 7fe71c5c6740 1 -- 172.21.15.16:0/2727131365 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5623345f99a0 msgr2=0x562334608970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:52.586 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.585+0000 7fe71c5c6740 1 --2- 172.21.15.16:0/2727131365 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5623345f99a0 0x562334608970 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fe7040058c0 tx=0x7fe704004a20 comp rx=0 tx=0).stop 2024-09-12T02:07:52.586 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.585+0000 7fe71c5c6740 1 -- 172.21.15.16:0/2727131365 shutdown_connections 2024-09-12T02:07:52.586 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.585+0000 7fe71c5c6740 1 --2- 172.21.15.16:0/2727131365 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x56233467b0b0 0x56233467d4d0 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:52.586 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.585+0000 7fe6f2ffd640 0 client.17175 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:07:52.586 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.585+0000 7fe71c5c6740 1 --2- 172.21.15.16:0/2727131365 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5623345ff370 0x562334609e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:52.586 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.585+0000 7fe71c5c6740 1 --2- 172.21.15.16:0/2727131365 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5623345fa2e0 0x562334608eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:52.586 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.585+0000 7fe71c5c6740 1 --2- 172.21.15.16:0/2727131365 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5623345f99a0 0x562334608970 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:52.586 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.585+0000 7fe71c5c6740 1 -- 172.21.15.16:0/2727131365 >> 172.21.15.16:0/2727131365 conn(0x5623345f5010 msgr2=0x5623345f39a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:07:52.586 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.585+0000 7fe71c5c6740 1 -- 172.21.15.16:0/2727131365 shutdown_connections 2024-09-12T02:07:52.586 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.585+0000 7fe71c5c6740 1 -- 172.21.15.16:0/2727131365 wait complete. 2024-09-12T02:07:52.586 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.585+0000 7fe71c5c6740 20 client.17175 trim_cache size 0 max 16384 2024-09-12T02:07:52.587 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-09-12T02:07:52.587 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-09-12T02:07:52.587 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-09-12T02:07:52.587 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-09-12T02:07:52.588 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.586+0000 7fe719827640 1 -- 172.21.15.16:0/4162621149 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fe6fc0a2cb0 msgr2=0x7fe6fc0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:52.588 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.586+0000 7fe719827640 1 --2- 172.21.15.16:0/4162621149 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fe6fc0a2cb0 0x7fe6fc0a5170 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fe70c00afd0 tx=0x7fe70c00a4d0 comp rx=0 tx=0).stop 2024-09-12T02:07:52.588 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.586+0000 7fe719827640 1 -- 172.21.15.16:0/4162621149 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7141003d0 msgr2=0x7fe71419a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:52.588 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.586+0000 7fe719827640 1 --2- 172.21.15.16:0/4162621149 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7141003d0 0x7fe71419a050 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7fe70400c970 tx=0x7fe70400ce30 comp rx=0 tx=0).stop 2024-09-12T02:07:52.588 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.586+0000 7fe719827640 1 -- 172.21.15.16:0/4162621149 shutdown_connections 2024-09-12T02:07:52.588 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.586+0000 7fe719827640 1 --2- 172.21.15.16:0/4162621149 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fe6fc0a2cb0 0x7fe6fc0a5170 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:52.588 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.586+0000 7fe719827640 1 --2- 172.21.15.16:0/4162621149 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe7141016b0 0x7fe7141a1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:52.589 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.586+0000 7fe719827640 1 --2- 172.21.15.16:0/4162621149 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe714100d10 0x7fe71419a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:52.589 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.586+0000 7fe719827640 1 --2- 172.21.15.16:0/4162621149 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7141003d0 0x7fe71419a050 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:52.589 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.586+0000 7fe719827640 1 -- 172.21.15.16:0/4162621149 >> 172.21.15.16:0/4162621149 conn(0x7fe7140fc1e0 msgr2=0x7fe7140fdc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:07:52.589 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.586+0000 7fe719827640 1 -- 172.21.15.16:0/4162621149 shutdown_connections 2024-09-12T02:07:52.589 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:07:52.586+0000 7fe719827640 1 -- 172.21.15.16:0/4162621149 wait complete. 2024-09-12T02:07:52.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:07:52 smithi184 ceph-mon[35593]: pgmap v513: 97 pgs: 97 active+clean; 3.2 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.3 KiB/s wr, 10 op/s 2024-09-12T02:07:52.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:07:52 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/2625741878' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:07:52.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:07:52 smithi033 ceph-mon[35332]: pgmap v513: 97 pgs: 97 active+clean; 3.2 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.3 KiB/s wr, 10 op/s 2024-09-12T02:07:52.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:07:52 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/2625741878' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:07:52.918 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24482 quiesce db --set-id a93721c1 --release --await' 2024-09-12T02:07:52.918 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24482 quiesce db --set-id a93721c1 --release --await 2024-09-12T02:07:53.184 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:07:53.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.691+0000 7f3b0f497640 1 -- 172.21.15.16:0/3376108488 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b08106f50 msgr2=0x7f3b0810eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:53.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.691+0000 7f3b0f497640 1 --2- 172.21.15.16:0/3376108488 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b08106f50 0x7f3b0810eaf0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f3b00009cd0 tx=0x7f3b0002a8e0 comp rx=0 tx=0).stop 2024-09-12T02:07:53.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.691+0000 7f3b0f497640 1 -- 172.21.15.16:0/3376108488 shutdown_connections 2024-09-12T02:07:53.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.691+0000 7f3b0f497640 1 --2- 172.21.15.16:0/3376108488 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b08106f50 0x7f3b0810eaf0 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:53.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.691+0000 7f3b0f497640 1 --2- 172.21.15.16:0/3376108488 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3b08100d00 0x7f3b08106840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:53.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.691+0000 7f3b0f497640 1 --2- 172.21.15.16:0/3376108488 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3b08100350 0x7f3b08100730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:53.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.691+0000 7f3b0f497640 1 -- 172.21.15.16:0/3376108488 >> 172.21.15.16:0/3376108488 conn(0x7f3b08077210 msgr2=0x7f3b080fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:07:53.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.691+0000 7f3b0f497640 1 -- 172.21.15.16:0/3376108488 shutdown_connections 2024-09-12T02:07:53.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.691+0000 7f3b0f497640 1 -- 172.21.15.16:0/3376108488 wait complete. 2024-09-12T02:07:53.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.692+0000 7f3b0f497640 1 Processor -- start 2024-09-12T02:07:53.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.692+0000 7f3b0f497640 1 -- start start 2024-09-12T02:07:53.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.692+0000 7f3b0f497640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b08100350 0x7f3b081984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:53.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.692+0000 7f3b0f497640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3b08100d00 0x7f3b08198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:53.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.692+0000 7f3b0f497640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3b08106f50 0x7f3b0819cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:53.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.692+0000 7f3b0f497640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3b08112000 con 0x7f3b08100350 2024-09-12T02:07:53.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.692+0000 7f3b0f497640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3b08111e80 con 0x7f3b08106f50 2024-09-12T02:07:53.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.692+0000 7f3b0f497640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f3b08112180 con 0x7f3b08100d00 2024-09-12T02:07:53.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.692+0000 7f3b0d233640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b08100350 0x7f3b081984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:53.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.692+0000 7f3b0d233640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b08100350 0x7f3b081984d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35580/0 (socket says 172.21.15.16:35580) 2024-09-12T02:07:53.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.692+0000 7f3b0d233640 1 -- 172.21.15.16:0/3283433628 learned_addr learned my addr 172.21.15.16:0/3283433628 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:07:53.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.692+0000 7f3b0ca32640 1 --2- 172.21.15.16:0/3283433628 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3b08100d00 0x7f3b08198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:53.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.692+0000 7f3b0da34640 1 --2- 172.21.15.16:0/3283433628 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3b08106f50 0x7f3b0819cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:53.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.692+0000 7f3b0d233640 1 -- 172.21.15.16:0/3283433628 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3b08100d00 msgr2=0x7f3b08198a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:53.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.692+0000 7f3b0d233640 1 --2- 172.21.15.16:0/3283433628 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3b08100d00 0x7f3b08198a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:53.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.692+0000 7f3b0d233640 1 -- 172.21.15.16:0/3283433628 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3b08106f50 msgr2=0x7f3b0819cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:53.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.692+0000 7f3b0d233640 1 --2- 172.21.15.16:0/3283433628 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3b08106f50 0x7f3b0819cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:53.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.692+0000 7f3b0d233640 1 -- 172.21.15.16:0/3283433628 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b0819d480 con 0x7f3b08100350 2024-09-12T02:07:53.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.692+0000 7f3b0da34640 1 --2- 172.21.15.16:0/3283433628 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3b08106f50 0x7f3b0819cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:07:53.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.693+0000 7f3b0d233640 1 --2- 172.21.15.16:0/3283433628 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b08100350 0x7f3b081984d0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f3b0400d950 tx=0x7f3b0400dd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:07:53.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.693+0000 7f3afe7fc640 1 -- 172.21.15.16:0/3283433628 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b04017030 con 0x7f3b08100350 2024-09-12T02:07:53.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.693+0000 7f3afe7fc640 1 -- 172.21.15.16:0/3283433628 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b04004020 con 0x7f3b08100350 2024-09-12T02:07:53.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.693+0000 7f3b0f497640 1 -- 172.21.15.16:0/3283433628 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b0819d770 con 0x7f3b08100350 2024-09-12T02:07:53.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.693+0000 7f3afe7fc640 1 -- 172.21.15.16:0/3283433628 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b04005650 con 0x7f3b08100350 2024-09-12T02:07:53.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.693+0000 7f3b0f497640 1 -- 172.21.15.16:0/3283433628 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b081a5050 con 0x7f3b08100350 2024-09-12T02:07:53.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.695+0000 7f3afe7fc640 1 -- 172.21.15.16:0/3283433628 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f3b040041d0 con 0x7f3b08100350 2024-09-12T02:07:53.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.696+0000 7f3b11870740 1 Processor -- start 2024-09-12T02:07:53.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.696+0000 7f3b11870740 1 -- start start 2024-09-12T02:07:53.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.696+0000 7f3b11870740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555dc99ccf60 0x555dc99cd340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:53.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.696+0000 7f3b11870740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555dc99cd880 0x555dc99dc110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:53.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.696+0000 7f3b11870740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555dc99dc650 0x555dc99dea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:53.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.696+0000 7f3b11870740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x555dc9932620 con 0x555dc99dc650 2024-09-12T02:07:53.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.696+0000 7f3b11870740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x555dc98d9e50 con 0x555dc99ccf60 2024-09-12T02:07:53.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.696+0000 7f3b11870740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x555dc992e5e0 con 0x555dc99cd880 2024-09-12T02:07:53.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.696+0000 7f3afe7fc640 1 --2- 172.21.15.16:0/3283433628 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f3af40a2c00 0x7f3af40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:53.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.696+0000 7f3b0ca32640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555dc99dc650 0x555dc99dea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:53.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.696+0000 7f3b0ca32640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555dc99dc650 0x555dc99dea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35596/0 (socket says 172.21.15.16:35596) 2024-09-12T02:07:53.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.696+0000 7f3b0ca32640 1 -- 172.21.15.16:0/746614012 learned_addr learned my addr 172.21.15.16:0/746614012 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:07:53.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.696+0000 7f3b0da34640 1 --2- 172.21.15.16:0/3283433628 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f3af40a2c00 0x7f3af40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:53.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.696+0000 7f3b0d233640 1 --2- 172.21.15.16:0/746614012 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555dc99cd880 0x555dc99dc110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:53.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.696+0000 7f3b0da34640 1 --2- 172.21.15.16:0/746614012 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555dc99ccf60 0x555dc99cd340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:53.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.697+0000 7f3afe7fc640 1 -- 172.21.15.16:0/3283433628 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f3b040ce8f0 con 0x7f3b08100350 2024-09-12T02:07:53.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.697+0000 7f3b0ca32640 1 -- 172.21.15.16:0/746614012 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555dc99cd880 msgr2=0x555dc99dc110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:53.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.697+0000 7f3b0ca32640 1 --2- 172.21.15.16:0/746614012 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555dc99cd880 0x555dc99dc110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:53.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.697+0000 7f3b0ca32640 1 -- 172.21.15.16:0/746614012 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555dc99ccf60 msgr2=0x555dc99cd340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:53.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.697+0000 7f3b0ca32640 1 --2- 172.21.15.16:0/746614012 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555dc99ccf60 0x555dc99cd340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:53.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.697+0000 7f3b0ca32640 1 -- 172.21.15.16:0/746614012 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555dc98cad40 con 0x555dc99dc650 2024-09-12T02:07:53.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.697+0000 7f3b0ca32640 1 --2- 172.21.15.16:0/746614012 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555dc99dc650 0x555dc99dea40 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f3af8009a00 tx=0x7f3af8009ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e77fb868c64c874d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:07:53.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.697+0000 7f3b0da34640 1 --2- 172.21.15.16:0/3283433628 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f3af40a2c00 0x7f3af40a50c0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f3b00033040 tx=0x7f3b00005da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:07:53.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.697+0000 7f3af37fe640 1 -- 172.21.15.16:0/746614012 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3af8017070 con 0x555dc99dc650 2024-09-12T02:07:53.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.697+0000 7f3af37fe640 1 -- 172.21.15.16:0/746614012 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3af8013470 con 0x555dc99dc650 2024-09-12T02:07:53.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.697+0000 7f3af37fe640 1 -- 172.21.15.16:0/746614012 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3af8012630 con 0x555dc99dc650 2024-09-12T02:07:53.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.698+0000 7f3b11870740 1 -- 172.21.15.16:0/746614012 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555dc99dc650 msgr2=0x555dc99dea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:53.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.698+0000 7f3b11870740 1 --2- 172.21.15.16:0/746614012 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555dc99dc650 0x555dc99dea40 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f3af8009a00 tx=0x7f3af8009ec0 comp rx=0 tx=0).stop 2024-09-12T02:07:53.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.698+0000 7f3b11870740 1 -- 172.21.15.16:0/746614012 shutdown_connections 2024-09-12T02:07:53.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.698+0000 7f3b11870740 1 --2- 172.21.15.16:0/746614012 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555dc99dc650 0x555dc99dea40 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:53.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.698+0000 7f3b11870740 1 --2- 172.21.15.16:0/746614012 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555dc99cd880 0x555dc99dc110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:53.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.698+0000 7f3b11870740 1 --2- 172.21.15.16:0/746614012 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555dc99ccf60 0x555dc99cd340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:53.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.698+0000 7f3b11870740 1 -- 172.21.15.16:0/746614012 >> 172.21.15.16:0/746614012 conn(0x555dc99c95f0 msgr2=0x555dc99c99d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:07:53.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.698+0000 7f3b11870740 1 -- 172.21.15.16:0/746614012 shutdown_connections 2024-09-12T02:07:53.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.698+0000 7f3b11870740 1 -- 172.21.15.16:0/746614012 wait complete. 2024-09-12T02:07:53.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.698+0000 7f3b11870740 1 -- start start 2024-09-12T02:07:53.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.698+0000 7f3b11870740 1 Processor -- start 2024-09-12T02:07:53.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.699+0000 7f3b11870740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555dc99ccf60 0x555dc99d95a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:53.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.699+0000 7f3b11870740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555dc99cd880 0x555dc99d9ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:53.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.699+0000 7f3b11870740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555dc99dc650 0x555dc99da020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:53.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.699+0000 7f3b11870740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x555dc99df280 con 0x555dc99dc650 2024-09-12T02:07:53.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.699+0000 7f3b11870740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x555dc98baf10 con 0x555dc99ccf60 2024-09-12T02:07:53.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.699+0000 7f3b11870740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x555dc99df400 con 0x555dc99cd880 2024-09-12T02:07:53.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.699+0000 7f3b11870740 10 client.0 _ll_register_callbacks cb 0x555dc98dedb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:07:53.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.699+0000 7f3b0da34640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555dc99dc650 0x555dc99da020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:53.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.699+0000 7f3b0d233640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555dc99ccf60 0x555dc99d95a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:53.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.699+0000 7f3b0da34640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555dc99dc650 0x555dc99da020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35606/0 (socket says 172.21.15.16:35606) 2024-09-12T02:07:53.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.699+0000 7f3b0da34640 1 -- 172.21.15.16:0/1200345597 learned_addr learned my addr 172.21.15.16:0/1200345597 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:07:53.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.699+0000 7f3b0ca32640 1 --2- 172.21.15.16:0/1200345597 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555dc99cd880 0x555dc99d9ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:53.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.699+0000 7f3b0da34640 1 -- 172.21.15.16:0/1200345597 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555dc99cd880 msgr2=0x555dc99d9ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:53.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.699+0000 7f3b0da34640 1 --2- 172.21.15.16:0/1200345597 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555dc99cd880 0x555dc99d9ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:53.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.699+0000 7f3b0da34640 1 -- 172.21.15.16:0/1200345597 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555dc99ccf60 msgr2=0x555dc99d95a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:53.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.699+0000 7f3b0da34640 1 --2- 172.21.15.16:0/1200345597 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555dc99ccf60 0x555dc99d95a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:53.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.699+0000 7f3b0da34640 1 -- 172.21.15.16:0/1200345597 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555dc9641580 con 0x555dc99dc650 2024-09-12T02:07:53.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.699+0000 7f3b0da34640 1 --2- 172.21.15.16:0/1200345597 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555dc99dc650 0x555dc99da020 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f3b00003bf0 tx=0x7f3b0002a8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd4a5537811b7c05 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:07:53.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.699+0000 7f3af0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:07:53.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.700+0000 7f3af0ff9640 1 -- 172.21.15.16:0/1200345597 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b0004f030 con 0x555dc99dc650 2024-09-12T02:07:53.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.700+0000 7f3b11870740 1 -- 172.21.15.16:0/1200345597 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3af8013470 con 0x555dc99dc650 2024-09-12T02:07:53.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.700+0000 7f3af0ff9640 1 -- 172.21.15.16:0/1200345597 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f3b00004390 con 0x555dc99dc650 2024-09-12T02:07:53.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.700+0000 7f3af0ff9640 1 -- 172.21.15.16:0/1200345597 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f3b00004590 con 0x555dc99dc650 2024-09-12T02:07:53.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.700+0000 7f3b11870740 10 client.17187 fetch_fsmap learned FSMap version 25 2024-09-12T02:07:53.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.700+0000 7f3b11870740 1 -- 172.21.15.16:0/1200345597 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555dc970b960 con 0x555dc99dc650 2024-09-12T02:07:53.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.701+0000 7f3af0ff9640 1 -- 172.21.15.16:0/1200345597 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3af8013470 con 0x555dc99dc650 2024-09-12T02:07:53.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.701+0000 7f3af0ff9640 1 -- 172.21.15.16:0/1200345597 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f3b0005d080 con 0x555dc99dc650 2024-09-12T02:07:53.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.701+0000 7f3b11870740 10 client.17187 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:07:53.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.701+0000 7f3b11870740 10 client.17187 resolve_mds: validated gid 24482 aka daemon mds.b 2024-09-12T02:07:53.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.701+0000 7f3b11870740 20 client.17187 populate_metadata read hostname 'smithi016' 2024-09-12T02:07:53.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.701+0000 7f3b11870740 1 --2- 172.21.15.16:0/1200345597 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x555dc99ec360 0x555dc99ee800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:53.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.701+0000 7f3b11870740 4 client.17187 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:07:53.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.701+0000 7f3b11870740 1 -- 172.21.15.16:0/1200345597 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555dc993fb90 con 0x555dc99ec360 2024-09-12T02:07:53.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.701+0000 7f3b0d233640 1 --2- 172.21.15.16:0/1200345597 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x555dc99ec360 0x555dc99ee800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:53.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.702+0000 7f3b0d233640 1 --2- 172.21.15.16:0/1200345597 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x555dc99ec360 0x555dc99ee800 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=223ab268ee43b21d server_cookie=e64542cc0137714e in_seq=0 out_seq=0 2024-09-12T02:07:53.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.702+0000 7f3af0ff9640 10 client.17187 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:07:53.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.702+0000 7f3af0ff9640 1 -- 172.21.15.16:0/1200345597 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555dc993fb90 con 0x555dc99ec360 2024-09-12T02:07:53.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.702+0000 7f3af0ff9640 10 client.17187 handle_command_reply: tid=0 2024-09-12T02:07:53.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.702+0000 7f3b11870740 1 client.17187 shutdown 2024-09-12T02:07:53.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.703+0000 7f3b11870740 1 -- 172.21.15.16:0/1200345597 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555dc99dc650 msgr2=0x555dc99da020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:53.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.703+0000 7f3b11870740 1 --2- 172.21.15.16:0/1200345597 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555dc99dc650 0x555dc99da020 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f3b00003bf0 tx=0x7f3b0002a8e0 comp rx=0 tx=0).stop 2024-09-12T02:07:53.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.703+0000 7f3b11870740 1 -- 172.21.15.16:0/1200345597 shutdown_connections 2024-09-12T02:07:53.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.703+0000 7f3b11870740 1 --2- 172.21.15.16:0/1200345597 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x555dc99ec360 0x555dc99ee800 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:53.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.703+0000 7f3b11870740 1 --2- 172.21.15.16:0/1200345597 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555dc99dc650 0x555dc99da020 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:53.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.703+0000 7f3b11870740 1 --2- 172.21.15.16:0/1200345597 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555dc99cd880 0x555dc99d9ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:53.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.703+0000 7f3b11870740 1 --2- 172.21.15.16:0/1200345597 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555dc99ccf60 0x555dc99d95a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:53.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.703+0000 7f3af0ff9640 0 client.17187 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:07:53.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.703+0000 7f3b11870740 1 -- 172.21.15.16:0/1200345597 >> 172.21.15.16:0/1200345597 conn(0x555dc99c95f0 msgr2=0x555dc99cf350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:07:53.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.703+0000 7f3b11870740 1 -- 172.21.15.16:0/1200345597 shutdown_connections 2024-09-12T02:07:53.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.703+0000 7f3b11870740 1 -- 172.21.15.16:0/1200345597 wait complete. 2024-09-12T02:07:53.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.703+0000 7f3b11870740 20 client.17187 trim_cache size 0 max 16384 2024-09-12T02:07:53.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.721+0000 7f3b11870740 1 Processor -- start 2024-09-12T02:07:53.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.721+0000 7f3b11870740 1 -- start start 2024-09-12T02:07:53.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.722+0000 7f3b11870740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555dc99ccf60 0x555dc99f40d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:53.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.722+0000 7f3b11870740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555dc99cd880 0x555dc99f4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:53.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.722+0000 7f3b11870740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555dc99dc650 0x555dc99d6800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:53.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.722+0000 7f3b11870740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3b081adb00 con 0x555dc99ccf60 2024-09-12T02:07:53.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.722+0000 7f3b11870740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3b081ab3d0 con 0x555dc99cd880 2024-09-12T02:07:53.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.722+0000 7f3b11870740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f3b081a9240 con 0x555dc99dc650 2024-09-12T02:07:53.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.722+0000 7f3b0d233640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555dc99ccf60 0x555dc99f40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:53.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.722+0000 7f3b0d233640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555dc99ccf60 0x555dc99f40d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35612/0 (socket says 172.21.15.16:35612) 2024-09-12T02:07:53.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.722+0000 7f3b0d233640 1 -- 172.21.15.16:0/897911130 learned_addr learned my addr 172.21.15.16:0/897911130 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:07:53.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.722+0000 7f3b0d233640 1 -- 172.21.15.16:0/897911130 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555dc99dc650 msgr2=0x555dc99d6800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:53.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.722+0000 7f3b0d233640 1 --2- 172.21.15.16:0/897911130 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555dc99dc650 0x555dc99d6800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:53.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.722+0000 7f3b0d233640 1 -- 172.21.15.16:0/897911130 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555dc99cd880 msgr2=0x555dc99f4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:53.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.722+0000 7f3b0ca32640 1 --2- 172.21.15.16:0/897911130 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555dc99cd880 0x555dc99f4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:53.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.722+0000 7f3b0d233640 1 --2- 172.21.15.16:0/897911130 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555dc99cd880 0x555dc99f4610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:53.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.722+0000 7f3b0d233640 1 -- 172.21.15.16:0/897911130 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555dc99c7bb0 con 0x555dc99ccf60 2024-09-12T02:07:53.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.722+0000 7f3b0d233640 1 --2- 172.21.15.16:0/897911130 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555dc99ccf60 0x555dc99f40d0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f3b04095c90 tx=0x7f3b0406b0a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79c0e5b1674dfef1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:07:53.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.722+0000 7f3af2ffd640 1 -- 172.21.15.16:0/897911130 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b040662d0 con 0x555dc99ccf60 2024-09-12T02:07:53.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.722+0000 7f3af2ffd640 1 -- 172.21.15.16:0/897911130 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b0409b1f0 con 0x555dc99ccf60 2024-09-12T02:07:53.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.722+0000 7f3af2ffd640 1 -- 172.21.15.16:0/897911130 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b0409f120 con 0x555dc99ccf60 2024-09-12T02:07:53.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.723+0000 7f3b11870740 1 -- 172.21.15.16:0/897911130 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555dc99ccf60 msgr2=0x555dc99f40d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:53.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.723+0000 7f3b11870740 1 --2- 172.21.15.16:0/897911130 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555dc99ccf60 0x555dc99f40d0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f3b04095c90 tx=0x7f3b0406b0a0 comp rx=0 tx=0).stop 2024-09-12T02:07:53.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.723+0000 7f3b11870740 1 -- 172.21.15.16:0/897911130 shutdown_connections 2024-09-12T02:07:53.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.723+0000 7f3b11870740 1 --2- 172.21.15.16:0/897911130 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555dc99dc650 0x555dc99d6800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:53.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.723+0000 7f3b11870740 1 --2- 172.21.15.16:0/897911130 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555dc99cd880 0x555dc99f4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:53.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.723+0000 7f3b11870740 1 --2- 172.21.15.16:0/897911130 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555dc99ccf60 0x555dc99f40d0 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:53.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.723+0000 7f3b11870740 1 -- 172.21.15.16:0/897911130 >> 172.21.15.16:0/897911130 conn(0x555dc99c95f0 msgr2=0x555dc99cf350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:07:53.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.723+0000 7f3b11870740 1 -- 172.21.15.16:0/897911130 shutdown_connections 2024-09-12T02:07:53.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.723+0000 7f3b11870740 1 -- 172.21.15.16:0/897911130 wait complete. 2024-09-12T02:07:53.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.723+0000 7f3b11870740 1 -- start start 2024-09-12T02:07:53.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.724+0000 7f3b11870740 1 Processor -- start 2024-09-12T02:07:53.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.724+0000 7f3b11870740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555dc99ccf60 0x555dc99e0b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:53.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.724+0000 7f3b11870740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555dc99cd880 0x555dc99e10a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:53.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.724+0000 7f3b11870740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555dc99dc650 0x555dc9a560a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:53.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.724+0000 7f3b11870740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3b081a9890 con 0x555dc99ccf60 2024-09-12T02:07:53.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.724+0000 7f3b11870740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3b081aad00 con 0x555dc99dc650 2024-09-12T02:07:53.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.724+0000 7f3b11870740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f3b081a9f60 con 0x555dc99cd880 2024-09-12T02:07:53.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.724+0000 7f3b11870740 10 client.0 _ll_register_callbacks cb 0x7f3b00031d80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:07:53.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.724+0000 7f3b0d233640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555dc99ccf60 0x555dc99e0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:53.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.724+0000 7f3b0d233640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555dc99ccf60 0x555dc99e0b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35618/0 (socket says 172.21.15.16:35618) 2024-09-12T02:07:53.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.724+0000 7f3b0d233640 1 -- 172.21.15.16:0/4097677452 learned_addr learned my addr 172.21.15.16:0/4097677452 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:07:53.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.724+0000 7f3b0da34640 1 --2- 172.21.15.16:0/4097677452 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555dc99dc650 0x555dc9a560a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:53.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.724+0000 7f3b0ca32640 1 --2- 172.21.15.16:0/4097677452 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555dc99cd880 0x555dc99e10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:53.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.725+0000 7f3b0d233640 1 -- 172.21.15.16:0/4097677452 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555dc99cd880 msgr2=0x555dc99e10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:53.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.725+0000 7f3b0d233640 1 --2- 172.21.15.16:0/4097677452 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555dc99cd880 0x555dc99e10a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:53.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.725+0000 7f3b0d233640 1 -- 172.21.15.16:0/4097677452 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555dc99dc650 msgr2=0x555dc9a560a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:53.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.725+0000 7f3b0d233640 1 --2- 172.21.15.16:0/4097677452 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555dc99dc650 0x555dc9a560a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:53.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.725+0000 7f3b0d233640 1 -- 172.21.15.16:0/4097677452 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555dc9a56660 con 0x555dc99ccf60 2024-09-12T02:07:53.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.725+0000 7f3b0d233640 1 --2- 172.21.15.16:0/4097677452 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555dc99ccf60 0x555dc99e0b60 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f3b0409b180 tx=0x7f3b04000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2f7404ac162807 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:07:53.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.725+0000 7f3af0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:07:53.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.725+0000 7f3af0ff9640 1 -- 172.21.15.16:0/4097677452 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b04097860 con 0x555dc99ccf60 2024-09-12T02:07:53.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.725+0000 7f3af0ff9640 1 -- 172.21.15.16:0/4097677452 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f3b040a39d0 con 0x555dc99ccf60 2024-09-12T02:07:53.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.725+0000 7f3b11870740 1 -- 172.21.15.16:0/4097677452 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b0409b1f0 con 0x555dc99ccf60 2024-09-12T02:07:53.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.725+0000 7f3af0ff9640 1 -- 172.21.15.16:0/4097677452 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f3b04097a00 con 0x555dc99ccf60 2024-09-12T02:07:53.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.725+0000 7f3b11870740 10 client.17193 fetch_fsmap learned FSMap version 25 2024-09-12T02:07:53.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.725+0000 7f3b11870740 1 -- 172.21.15.16:0/4097677452 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555dc9a57960 con 0x555dc99ccf60 2024-09-12T02:07:53.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.725+0000 7f3af0ff9640 1 -- 172.21.15.16:0/4097677452 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b0409b1f0 con 0x555dc99ccf60 2024-09-12T02:07:53.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.725+0000 7f3af0ff9640 1 -- 172.21.15.16:0/4097677452 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f3b040841e0 con 0x555dc99ccf60 2024-09-12T02:07:53.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.725+0000 7f3b11870740 10 client.17193 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:07:53.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.725+0000 7f3b11870740 10 client.17193 resolve_mds: validated gid 24482 aka daemon mds.b 2024-09-12T02:07:53.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.725+0000 7f3b11870740 20 client.17193 populate_metadata read hostname 'smithi016' 2024-09-12T02:07:53.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.725+0000 7f3b11870740 1 --2- 172.21.15.16:0/4097677452 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x555dc9a57e20 0x555dc9a5a240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:53.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.725+0000 7f3b11870740 4 client.17193 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "a93721c1", "release": true, "await": true}] 2024-09-12T02:07:53.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.726+0000 7f3b11870740 1 -- 172.21.15.16:0/4097677452 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "quiesce db", "set_id": "a93721c1", "release": true, "await": true}) -- 0x555dc99c4950 con 0x555dc9a57e20 2024-09-12T02:07:53.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.726+0000 7f3b0ca32640 1 --2- 172.21.15.16:0/4097677452 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x555dc9a57e20 0x555dc9a5a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:53.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.726+0000 7f3b0ca32640 1 --2- 172.21.15.16:0/4097677452 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x555dc9a57e20 0x555dc9a5a240 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac1a17ef19ccf5cf server_cookie=3b5b45ab5d09a642 in_seq=0 out_seq=0 2024-09-12T02:07:53.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.726+0000 7f3af0ff9640 10 client.17193 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:07:53.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.728+0000 7f3af0ff9640 1 -- 172.21.15.16:0/4097677452 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x555dc99c4950 con 0x555dc9a57e20 2024-09-12T02:07:53.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.728+0000 7f3af0ff9640 10 client.17193 handle_command_reply: tid=0 2024-09-12T02:07:53.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.728+0000 7f3b11870740 1 client.17193 shutdown 2024-09-12T02:07:53.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.728+0000 7f3b11870740 1 -- 172.21.15.16:0/4097677452 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555dc99ccf60 msgr2=0x555dc99e0b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:53.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.728+0000 7f3b11870740 1 --2- 172.21.15.16:0/4097677452 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555dc99ccf60 0x555dc99e0b60 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f3b0409b180 tx=0x7f3b04000bc0 comp rx=0 tx=0).stop 2024-09-12T02:07:53.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.728+0000 7f3b11870740 1 -- 172.21.15.16:0/4097677452 shutdown_connections 2024-09-12T02:07:53.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.728+0000 7f3b11870740 1 --2- 172.21.15.16:0/4097677452 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x555dc9a57e20 0x555dc9a5a240 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:53.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.728+0000 7f3b11870740 1 --2- 172.21.15.16:0/4097677452 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555dc99dc650 0x555dc9a560a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:53.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.728+0000 7f3b11870740 1 --2- 172.21.15.16:0/4097677452 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555dc99cd880 0x555dc99e10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:53.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.728+0000 7f3af0ff9640 0 client.17193 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:07:53.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.728+0000 7f3b11870740 1 --2- 172.21.15.16:0/4097677452 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555dc99ccf60 0x555dc99e0b60 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:53.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.728+0000 7f3b11870740 1 -- 172.21.15.16:0/4097677452 >> 172.21.15.16:0/4097677452 conn(0x555dc99c95f0 msgr2=0x555dc99cf350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:07:53.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.729+0000 7f3b11870740 1 -- 172.21.15.16:0/4097677452 shutdown_connections 2024-09-12T02:07:53.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.729+0000 7f3b11870740 1 -- 172.21.15.16:0/4097677452 wait complete. 2024-09-12T02:07:53.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.729+0000 7f3b11870740 20 client.17193 trim_cache size 0 max 16384 2024-09-12T02:07:53.733 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T02:07:53.733 INFO:teuthology.orchestra.run.smithi016.stdout: "epoch": 25, 2024-09-12T02:07:53.733 INFO:teuthology.orchestra.run.smithi016.stdout: "leader": 24482, 2024-09-12T02:07:53.733 INFO:teuthology.orchestra.run.smithi016.stdout: "set_version": 24, 2024-09-12T02:07:53.733 INFO:teuthology.orchestra.run.smithi016.stdout: "sets": { 2024-09-12T02:07:53.733 INFO:teuthology.orchestra.run.smithi016.stdout: "a93721c1": { 2024-09-12T02:07:53.733 INFO:teuthology.orchestra.run.smithi016.stdout: "version": 24, 2024-09-12T02:07:53.733 INFO:teuthology.orchestra.run.smithi016.stdout: "age_ref": 0.0, 2024-09-12T02:07:53.733 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-12T02:07:53.733 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RELEASED", 2024-09-12T02:07:53.733 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-12T02:07:53.733 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:07:53.734 INFO:teuthology.orchestra.run.smithi016.stdout: "timeout": 90.0, 2024-09-12T02:07:53.734 INFO:teuthology.orchestra.run.smithi016.stdout: "expiration": 140.1, 2024-09-12T02:07:53.734 INFO:teuthology.orchestra.run.smithi016.stdout: "members": { 2024-09-12T02:07:53.734 INFO:teuthology.orchestra.run.smithi016.stdout: "file:/": { 2024-09-12T02:07:53.734 INFO:teuthology.orchestra.run.smithi016.stdout: "excluded": false, 2024-09-12T02:07:53.734 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-12T02:07:53.734 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RELEASED", 2024-09-12T02:07:53.734 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-12T02:07:53.734 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:07:53.735 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:07:53.735 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:07:53.735 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:07:53.735 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:07:53.735 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T02:07:53.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.729+0000 7f3b0f497640 1 -- 172.21.15.16:0/3283433628 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f3af40a2c00 msgr2=0x7f3af40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:53.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.729+0000 7f3b0f497640 1 --2- 172.21.15.16:0/3283433628 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f3af40a2c00 0x7f3af40a50c0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f3b00033040 tx=0x7f3b00005da0 comp rx=0 tx=0).stop 2024-09-12T02:07:53.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.729+0000 7f3b0f497640 1 -- 172.21.15.16:0/3283433628 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b08100350 msgr2=0x7f3b081984d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:53.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.729+0000 7f3b0f497640 1 --2- 172.21.15.16:0/3283433628 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b08100350 0x7f3b081984d0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f3b0400d950 tx=0x7f3b0400dd50 comp rx=0 tx=0).stop 2024-09-12T02:07:53.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.729+0000 7f3b0f497640 1 -- 172.21.15.16:0/3283433628 shutdown_connections 2024-09-12T02:07:53.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.729+0000 7f3b0f497640 1 --2- 172.21.15.16:0/3283433628 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f3af40a2c00 0x7f3af40a50c0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:53.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.729+0000 7f3b0f497640 1 --2- 172.21.15.16:0/3283433628 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3b08106f50 0x7f3b0819cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:53.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.729+0000 7f3b0f497640 1 --2- 172.21.15.16:0/3283433628 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3b08100d00 0x7f3b08198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:53.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.729+0000 7f3b0f497640 1 --2- 172.21.15.16:0/3283433628 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b08100350 0x7f3b081984d0 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:53.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.729+0000 7f3b0f497640 1 -- 172.21.15.16:0/3283433628 >> 172.21.15.16:0/3283433628 conn(0x7f3b08077210 msgr2=0x7f3b080fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:07:53.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.729+0000 7f3b0f497640 1 -- 172.21.15.16:0/3283433628 shutdown_connections 2024-09-12T02:07:53.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:53.729+0000 7f3b0f497640 1 -- 172.21.15.16:0/3283433628 wait complete. 2024-09-12T02:07:53.737 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T02:07:54.388 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:07:54 smithi016 ceph-mon[30947]: pgmap v514: 97 pgs: 97 active+clean; 3.2 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-09-12T02:07:54.415 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'a93721c1', total seconds elapsed: 23.3 2024-09-12T02:07:54.444 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:07:54.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:07:54 smithi184 ceph-mon[35593]: pgmap v514: 97 pgs: 97 active+clean; 3.2 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-09-12T02:07:54.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:07:54 smithi033 ceph-mon[35332]: pgmap v514: 97 pgs: 97 active+clean; 3.2 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-09-12T02:07:54.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.918+0000 7f5b38098640 1 -- 172.21.15.16:0/3772669354 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b30106a80 msgr2=0x7f5b30106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:54.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.918+0000 7f5b38098640 1 --2- 172.21.15.16:0/3772669354 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b30106a80 0x7f5b30106e60 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f5b20009cd0 tx=0x7f5b2002a8e0 comp rx=0 tx=0).stop 2024-09-12T02:07:54.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.918+0000 7f5b38098640 1 -- 172.21.15.16:0/3772669354 shutdown_connections 2024-09-12T02:07:54.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.918+0000 7f5b38098640 1 --2- 172.21.15.16:0/3772669354 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5b30101420 0x7f5b3010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:54.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.918+0000 7f5b38098640 1 --2- 172.21.15.16:0/3772669354 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b30100a80 0x7f5b30100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:54.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.918+0000 7f5b38098640 1 --2- 172.21.15.16:0/3772669354 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b30106a80 0x7f5b30106e60 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:54.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.918+0000 7f5b38098640 1 -- 172.21.15.16:0/3772669354 >> 172.21.15.16:0/3772669354 conn(0x7f5b300fc7a0 msgr2=0x7f5b300febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:07:54.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.918+0000 7f5b38098640 1 -- 172.21.15.16:0/3772669354 shutdown_connections 2024-09-12T02:07:54.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.918+0000 7f5b38098640 1 -- 172.21.15.16:0/3772669354 wait complete. 2024-09-12T02:07:54.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.918+0000 7f5b38098640 1 Processor -- start 2024-09-12T02:07:54.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.919+0000 7f5b38098640 1 -- start start 2024-09-12T02:07:54.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.919+0000 7f5b38098640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b30100a80 0x7f5b30198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:54.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.919+0000 7f5b38098640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5b30101420 0x7f5b30198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:54.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.919+0000 7f5b38098640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b30106a80 0x7f5b3019cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:54.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.919+0000 7f5b38098640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5b30110e40 con 0x7f5b30100a80 2024-09-12T02:07:54.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.919+0000 7f5b38098640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5b30110cc0 con 0x7f5b30106a80 2024-09-12T02:07:54.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.919+0000 7f5b38098640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5b30110fc0 con 0x7f5b30101420 2024-09-12T02:07:54.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.919+0000 7f5b35e34640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b30100a80 0x7f5b30198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:54.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.919+0000 7f5b35e34640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b30100a80 0x7f5b30198410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35638/0 (socket says 172.21.15.16:35638) 2024-09-12T02:07:54.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.919+0000 7f5b35e34640 1 -- 172.21.15.16:0/1068763744 learned_addr learned my addr 172.21.15.16:0/1068763744 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:07:54.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.919+0000 7f5b35633640 1 --2- 172.21.15.16:0/1068763744 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5b30101420 0x7f5b30198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:54.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.919+0000 7f5b36635640 1 --2- 172.21.15.16:0/1068763744 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b30106a80 0x7f5b3019cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:54.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.919+0000 7f5b35e34640 1 -- 172.21.15.16:0/1068763744 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5b30101420 msgr2=0x7f5b30198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:54.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.919+0000 7f5b35e34640 1 --2- 172.21.15.16:0/1068763744 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5b30101420 0x7f5b30198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:54.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.919+0000 7f5b35e34640 1 -- 172.21.15.16:0/1068763744 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b30106a80 msgr2=0x7f5b3019cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:54.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.919+0000 7f5b35e34640 1 --2- 172.21.15.16:0/1068763744 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b30106a80 0x7f5b3019cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:54.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.919+0000 7f5b35e34640 1 -- 172.21.15.16:0/1068763744 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b3019d3c0 con 0x7f5b30100a80 2024-09-12T02:07:54.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.920+0000 7f5b35e34640 1 --2- 172.21.15.16:0/1068763744 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b30100a80 0x7f5b30198410 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f5b2002ae00 tx=0x7f5b20003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:07:54.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.920+0000 7f5b2effd640 1 -- 172.21.15.16:0/1068763744 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b2003d790 con 0x7f5b30100a80 2024-09-12T02:07:54.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.920+0000 7f5b2effd640 1 -- 172.21.15.16:0/1068763744 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b2003de10 con 0x7f5b30100a80 2024-09-12T02:07:54.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.920+0000 7f5b2effd640 1 -- 172.21.15.16:0/1068763744 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b20030800 con 0x7f5b30100a80 2024-09-12T02:07:54.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.920+0000 7f5b38098640 1 -- 172.21.15.16:0/1068763744 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b3019d650 con 0x7f5b30100a80 2024-09-12T02:07:54.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.920+0000 7f5b38098640 1 -- 172.21.15.16:0/1068763744 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b3019dae0 con 0x7f5b30100a80 2024-09-12T02:07:54.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.922+0000 7f5b2effd640 1 -- 172.21.15.16:0/1068763744 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5b2004f020 con 0x7f5b30100a80 2024-09-12T02:07:54.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.923+0000 7f5b3a471740 1 Processor -- start 2024-09-12T02:07:54.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.923+0000 7f5b3a471740 1 -- start start 2024-09-12T02:07:54.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.923+0000 7f5b3a471740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cbe97fe4a0 0x55cbe97fe880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:54.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.923+0000 7f5b3a471740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cbe97fedc0 0x55cbe9809140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:54.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.923+0000 7f5b3a471740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cbe9809680 0x55cbe980ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:54.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.923+0000 7f5b3a471740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55cbe97642e0 con 0x55cbe97fe4a0 2024-09-12T02:07:54.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.923+0000 7f5b3a471740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55cbe970bce0 con 0x55cbe97fedc0 2024-09-12T02:07:54.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.923+0000 7f5b3a471740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55cbe97602a0 con 0x55cbe9809680 2024-09-12T02:07:54.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.923+0000 7f5b36635640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cbe97fe4a0 0x55cbe97fe880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:54.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.923+0000 7f5b36635640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cbe97fe4a0 0x55cbe97fe880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35650/0 (socket says 172.21.15.16:35650) 2024-09-12T02:07:54.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.923+0000 7f5b36635640 1 -- 172.21.15.16:0/2552430592 learned_addr learned my addr 172.21.15.16:0/2552430592 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:07:54.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.924+0000 7f5b35e34640 1 --2- 172.21.15.16:0/2552430592 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cbe97fedc0 0x55cbe9809140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:54.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.924+0000 7f5b2effd640 1 --2- 172.21.15.16:0/1068763744 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f5b180a2b80 0x7f5b180a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:54.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.924+0000 7f5b35633640 1 --2- 172.21.15.16:0/2552430592 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cbe9809680 0x55cbe980ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:54.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.924+0000 7f5b36635640 1 -- 172.21.15.16:0/2552430592 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cbe9809680 msgr2=0x55cbe980ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:54.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.924+0000 7f5b36635640 1 --2- 172.21.15.16:0/2552430592 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cbe9809680 0x55cbe980ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:54.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.924+0000 7f5b36635640 1 -- 172.21.15.16:0/2552430592 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cbe97fedc0 msgr2=0x55cbe9809140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:54.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.924+0000 7f5b36635640 1 --2- 172.21.15.16:0/2552430592 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cbe97fedc0 0x55cbe9809140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:54.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.924+0000 7f5b36635640 1 -- 172.21.15.16:0/2552430592 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cbe96f0440 con 0x55cbe97fe4a0 2024-09-12T02:07:54.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.924+0000 7f5b35633640 1 --2- 172.21.15.16:0/2552430592 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cbe9809680 0x55cbe980ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:07:54.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.924+0000 7f5b2effd640 1 -- 172.21.15.16:0/1068763744 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f5b2002e020 con 0x7f5b30100a80 2024-09-12T02:07:54.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.924+0000 7f5b36635640 1 --2- 172.21.15.16:0/1068763744 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f5b180a2b80 0x7f5b180a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:54.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.924+0000 7f5b36635640 1 --2- 172.21.15.16:0/2552430592 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cbe97fe4a0 0x55cbe97fe880 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f5b2800b7a0 tx=0x7f5b2800bc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93aa0b2032db3be8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:07:54.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.924+0000 7f5b17fff640 1 -- 172.21.15.16:0/2552430592 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b28024070 con 0x55cbe97fe4a0 2024-09-12T02:07:54.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.924+0000 7f5b17fff640 1 -- 172.21.15.16:0/2552430592 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b2801e960 con 0x55cbe97fe4a0 2024-09-12T02:07:54.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.924+0000 7f5b17fff640 1 -- 172.21.15.16:0/2552430592 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b280118c0 con 0x55cbe97fe4a0 2024-09-12T02:07:54.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.924+0000 7f5b36635640 1 --2- 172.21.15.16:0/1068763744 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f5b180a2b80 0x7f5b180a5040 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f5b28021000 tx=0x7f5b2801e000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:07:54.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.925+0000 7f5b3a471740 1 -- 172.21.15.16:0/2552430592 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cbe97fe4a0 msgr2=0x55cbe97fe880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:54.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.925+0000 7f5b3a471740 1 --2- 172.21.15.16:0/2552430592 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cbe97fe4a0 0x55cbe97fe880 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f5b2800b7a0 tx=0x7f5b2800bc60 comp rx=0 tx=0).stop 2024-09-12T02:07:54.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.925+0000 7f5b3a471740 1 -- 172.21.15.16:0/2552430592 shutdown_connections 2024-09-12T02:07:54.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.925+0000 7f5b3a471740 1 --2- 172.21.15.16:0/2552430592 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cbe9809680 0x55cbe980ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:54.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.925+0000 7f5b3a471740 1 --2- 172.21.15.16:0/2552430592 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cbe97fedc0 0x55cbe9809140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:54.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.925+0000 7f5b3a471740 1 --2- 172.21.15.16:0/2552430592 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cbe97fe4a0 0x55cbe97fe880 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:54.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.925+0000 7f5b3a471740 1 -- 172.21.15.16:0/2552430592 >> 172.21.15.16:0/2552430592 conn(0x55cbe97faa30 msgr2=0x55cbe97fae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:07:54.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.925+0000 7f5b3a471740 1 -- 172.21.15.16:0/2552430592 shutdown_connections 2024-09-12T02:07:54.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.925+0000 7f5b3a471740 1 -- 172.21.15.16:0/2552430592 wait complete. 2024-09-12T02:07:54.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.926+0000 7f5b3a471740 1 -- start start 2024-09-12T02:07:54.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.926+0000 7f5b3a471740 1 Processor -- start 2024-09-12T02:07:54.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.926+0000 7f5b3a471740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cbe97fe4a0 0x55cbe9808f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:54.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.926+0000 7f5b3a471740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cbe97fedc0 0x55cbe9806220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:54.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.926+0000 7f5b3a471740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cbe9809680 0x55cbe9806760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:54.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.926+0000 7f5b3a471740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55cbe980c250 con 0x55cbe9809680 2024-09-12T02:07:54.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.926+0000 7f5b3a471740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55cbe971f050 con 0x55cbe97fedc0 2024-09-12T02:07:54.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.926+0000 7f5b3a471740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55cbe980c3d0 con 0x55cbe97fe4a0 2024-09-12T02:07:54.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.926+0000 7f5b3a471740 10 client.0 _ll_register_callbacks cb 0x55cbe975f160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:07:54.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.926+0000 7f5b36635640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cbe9809680 0x55cbe9806760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:54.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.926+0000 7f5b36635640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cbe9809680 0x55cbe9806760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35662/0 (socket says 172.21.15.16:35662) 2024-09-12T02:07:54.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.926+0000 7f5b36635640 1 -- 172.21.15.16:0/676430443 learned_addr learned my addr 172.21.15.16:0/676430443 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:07:54.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.926+0000 7f5b35633640 1 --2- 172.21.15.16:0/676430443 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cbe97fedc0 0x55cbe9806220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:54.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.926+0000 7f5b35e34640 1 --2- 172.21.15.16:0/676430443 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cbe97fe4a0 0x55cbe9808f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:54.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.927+0000 7f5b36635640 1 -- 172.21.15.16:0/676430443 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cbe97fe4a0 msgr2=0x55cbe9808f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:54.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.927+0000 7f5b36635640 1 --2- 172.21.15.16:0/676430443 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cbe97fe4a0 0x55cbe9808f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:54.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.927+0000 7f5b36635640 1 -- 172.21.15.16:0/676430443 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cbe97fedc0 msgr2=0x55cbe9806220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:54.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.927+0000 7f5b36635640 1 --2- 172.21.15.16:0/676430443 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cbe97fedc0 0x55cbe9806220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:54.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.927+0000 7f5b36635640 1 -- 172.21.15.16:0/676430443 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cbe94d53c0 con 0x55cbe9809680 2024-09-12T02:07:54.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.927+0000 7f5b36635640 1 --2- 172.21.15.16:0/676430443 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cbe9809680 0x55cbe9806760 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f5b28016010 tx=0x7f5b28011440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d8276eb0a91a4f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:07:54.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.927+0000 7f5b157fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:07:54.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.927+0000 7f5b157fa640 1 -- 172.21.15.16:0/676430443 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b28024040 con 0x55cbe9809680 2024-09-12T02:07:54.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.927+0000 7f5b3a471740 1 -- 172.21.15.16:0/676430443 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b2801e960 con 0x55cbe9809680 2024-09-12T02:07:54.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.927+0000 7f5b157fa640 1 -- 172.21.15.16:0/676430443 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f5b28013250 con 0x55cbe9809680 2024-09-12T02:07:54.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.927+0000 7f5b157fa640 1 -- 172.21.15.16:0/676430443 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5b28038050 con 0x55cbe9809680 2024-09-12T02:07:54.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.927+0000 7f5b3a471740 10 client.17205 fetch_fsmap learned FSMap version 25 2024-09-12T02:07:54.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.927+0000 7f5b3a471740 1 -- 172.21.15.16:0/676430443 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55cbe9817bf0 con 0x55cbe9809680 2024-09-12T02:07:54.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.927+0000 7f5b157fa640 1 -- 172.21.15.16:0/676430443 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b2801e960 con 0x55cbe9809680 2024-09-12T02:07:54.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.928+0000 7f5b157fa640 1 -- 172.21.15.16:0/676430443 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f5b28028440 con 0x55cbe9809680 2024-09-12T02:07:54.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.928+0000 7f5b3a471740 10 client.17205 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:07:54.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.928+0000 7f5b3a471740 10 client.17205 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:07:54.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.928+0000 7f5b3a471740 20 client.17205 populate_metadata read hostname 'smithi016' 2024-09-12T02:07:54.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.928+0000 7f5b3a471740 1 --2- 172.21.15.16:0/676430443 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55cbe9818fa0 0x55cbe981b390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:54.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.928+0000 7f5b3a471740 4 client.17205 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:07:54.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.928+0000 7f5b3a471740 1 -- 172.21.15.16:0/676430443 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cbe9771850 con 0x55cbe9818fa0 2024-09-12T02:07:54.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.928+0000 7f5b35e34640 1 --2- 172.21.15.16:0/676430443 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55cbe9818fa0 0x55cbe981b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:54.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.928+0000 7f5b35e34640 1 --2- 172.21.15.16:0/676430443 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55cbe9818fa0 0x55cbe981b390 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a78b41c6b100923 server_cookie=8e57161ab57e50b5 in_seq=0 out_seq=0 2024-09-12T02:07:54.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.928+0000 7f5b157fa640 10 client.17205 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:07:54.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.929+0000 7f5b157fa640 1 -- 172.21.15.16:0/676430443 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cbe9771850 con 0x55cbe9818fa0 2024-09-12T02:07:54.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.929+0000 7f5b157fa640 10 client.17205 handle_command_reply: tid=0 2024-09-12T02:07:54.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.929+0000 7f5b3a471740 1 client.17205 shutdown 2024-09-12T02:07:54.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.929+0000 7f5b3a471740 1 -- 172.21.15.16:0/676430443 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cbe9809680 msgr2=0x55cbe9806760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:54.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.929+0000 7f5b3a471740 1 --2- 172.21.15.16:0/676430443 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cbe9809680 0x55cbe9806760 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f5b28016010 tx=0x7f5b28011440 comp rx=0 tx=0).stop 2024-09-12T02:07:54.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.929+0000 7f5b3a471740 1 -- 172.21.15.16:0/676430443 shutdown_connections 2024-09-12T02:07:54.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.929+0000 7f5b3a471740 1 --2- 172.21.15.16:0/676430443 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55cbe9818fa0 0x55cbe981b390 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:54.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.929+0000 7f5b3a471740 1 --2- 172.21.15.16:0/676430443 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cbe9809680 0x55cbe9806760 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:54.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.929+0000 7f5b3a471740 1 --2- 172.21.15.16:0/676430443 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cbe97fedc0 0x55cbe9806220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:54.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.929+0000 7f5b157fa640 0 client.17205 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:07:54.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.929+0000 7f5b3a471740 1 --2- 172.21.15.16:0/676430443 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cbe97fe4a0 0x55cbe9808f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:54.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.929+0000 7f5b3a471740 1 -- 172.21.15.16:0/676430443 >> 172.21.15.16:0/676430443 conn(0x55cbe97faa30 msgr2=0x55cbe9800760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:07:54.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.929+0000 7f5b3a471740 1 -- 172.21.15.16:0/676430443 shutdown_connections 2024-09-12T02:07:54.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.929+0000 7f5b3a471740 1 -- 172.21.15.16:0/676430443 wait complete. 2024-09-12T02:07:54.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.929+0000 7f5b3a471740 20 client.17205 trim_cache size 0 max 16384 2024-09-12T02:07:54.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.946+0000 7f5b3a471740 1 Processor -- start 2024-09-12T02:07:54.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.947+0000 7f5b3a471740 1 -- start start 2024-09-12T02:07:54.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.947+0000 7f5b3a471740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cbe97fe4a0 0x55cbe98154e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:54.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.947+0000 7f5b3a471740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cbe97fedc0 0x55cbe9820a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:54.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.947+0000 7f5b3a471740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cbe9809680 0x55cbe9820fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:54.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.947+0000 7f5b3a471740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5b301adab0 con 0x55cbe9809680 2024-09-12T02:07:54.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.947+0000 7f5b3a471740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5b301ab380 con 0x55cbe97fedc0 2024-09-12T02:07:54.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.947+0000 7f5b3a471740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5b301a91d0 con 0x55cbe97fe4a0 2024-09-12T02:07:54.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.947+0000 7f5b36635640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cbe9809680 0x55cbe9820fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:54.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.947+0000 7f5b36635640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cbe9809680 0x55cbe9820fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35664/0 (socket says 172.21.15.16:35664) 2024-09-12T02:07:54.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.947+0000 7f5b36635640 1 -- 172.21.15.16:0/2726210813 learned_addr learned my addr 172.21.15.16:0/2726210813 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:07:54.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.947+0000 7f5b35633640 1 --2- 172.21.15.16:0/2726210813 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cbe97fedc0 0x55cbe9820a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:54.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.947+0000 7f5b35e34640 1 --2- 172.21.15.16:0/2726210813 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cbe97fe4a0 0x55cbe98154e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:54.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.947+0000 7f5b36635640 1 -- 172.21.15.16:0/2726210813 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cbe97fe4a0 msgr2=0x55cbe98154e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:54.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.947+0000 7f5b36635640 1 --2- 172.21.15.16:0/2726210813 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cbe97fe4a0 0x55cbe98154e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:54.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.947+0000 7f5b36635640 1 -- 172.21.15.16:0/2726210813 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cbe97fedc0 msgr2=0x55cbe9820a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:54.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.947+0000 7f5b36635640 1 --2- 172.21.15.16:0/2726210813 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cbe97fedc0 0x55cbe9820a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:54.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.947+0000 7f5b36635640 1 -- 172.21.15.16:0/2726210813 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cbe97fa4d0 con 0x55cbe9809680 2024-09-12T02:07:54.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.947+0000 7f5b35e34640 1 --2- 172.21.15.16:0/2726210813 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cbe97fe4a0 0x55cbe98154e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:07:54.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.947+0000 7f5b36635640 1 --2- 172.21.15.16:0/2726210813 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cbe9809680 0x55cbe9820fc0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f5b28000ea0 tx=0x7f5b2800bf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bf0873f28bf025b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:07:54.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.948+0000 7f5b177fe640 1 -- 172.21.15.16:0/2726210813 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b28024020 con 0x55cbe9809680 2024-09-12T02:07:54.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.948+0000 7f5b177fe640 1 -- 172.21.15.16:0/2726210813 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b2802c070 con 0x55cbe9809680 2024-09-12T02:07:54.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.948+0000 7f5b177fe640 1 -- 172.21.15.16:0/2726210813 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b28005070 con 0x55cbe9809680 2024-09-12T02:07:54.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.948+0000 7f5b3a471740 1 -- 172.21.15.16:0/2726210813 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cbe9809680 msgr2=0x55cbe9820fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:54.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.948+0000 7f5b3a471740 1 --2- 172.21.15.16:0/2726210813 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cbe9809680 0x55cbe9820fc0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f5b28000ea0 tx=0x7f5b2800bf30 comp rx=0 tx=0).stop 2024-09-12T02:07:54.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.948+0000 7f5b3a471740 1 -- 172.21.15.16:0/2726210813 shutdown_connections 2024-09-12T02:07:54.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.948+0000 7f5b3a471740 1 --2- 172.21.15.16:0/2726210813 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cbe9809680 0x55cbe9820fc0 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:54.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.948+0000 7f5b3a471740 1 --2- 172.21.15.16:0/2726210813 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cbe97fedc0 0x55cbe9820a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:54.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.948+0000 7f5b3a471740 1 --2- 172.21.15.16:0/2726210813 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cbe97fe4a0 0x55cbe98154e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:54.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.948+0000 7f5b3a471740 1 -- 172.21.15.16:0/2726210813 >> 172.21.15.16:0/2726210813 conn(0x55cbe97faa30 msgr2=0x55cbe9805310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:07:54.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.948+0000 7f5b3a471740 1 -- 172.21.15.16:0/2726210813 shutdown_connections 2024-09-12T02:07:54.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.948+0000 7f5b3a471740 1 -- 172.21.15.16:0/2726210813 wait complete. 2024-09-12T02:07:54.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.948+0000 7f5b3a471740 1 -- start start 2024-09-12T02:07:54.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.949+0000 7f5b3a471740 1 Processor -- start 2024-09-12T02:07:54.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.949+0000 7f5b3a471740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cbe97fe4a0 0x55cbe980ec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:54.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.949+0000 7f5b3a471740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cbe97fedc0 0x55cbe980f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:54.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.949+0000 7f5b3a471740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cbe9809680 0x55cbe987ba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:54.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.949+0000 7f5b3a471740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5b301a9840 con 0x55cbe97fedc0 2024-09-12T02:07:54.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.949+0000 7f5b3a471740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5b301aacb0 con 0x55cbe97fe4a0 2024-09-12T02:07:54.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.949+0000 7f5b3a471740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5b301a9f10 con 0x55cbe9809680 2024-09-12T02:07:54.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.949+0000 7f5b3a471740 10 client.0 _ll_register_callbacks cb 0x7f5b28022070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:07:54.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.949+0000 7f5b35633640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cbe97fedc0 0x55cbe980f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:54.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.949+0000 7f5b35633640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cbe97fedc0 0x55cbe980f1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35676/0 (socket says 172.21.15.16:35676) 2024-09-12T02:07:54.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.949+0000 7f5b35633640 1 -- 172.21.15.16:0/466514708 learned_addr learned my addr 172.21.15.16:0/466514708 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:07:54.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.949+0000 7f5b35e34640 1 --2- 172.21.15.16:0/466514708 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cbe97fe4a0 0x55cbe980ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:54.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.949+0000 7f5b36635640 1 --2- 172.21.15.16:0/466514708 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cbe9809680 0x55cbe987ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:54.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.949+0000 7f5b35633640 1 -- 172.21.15.16:0/466514708 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cbe9809680 msgr2=0x55cbe987ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:54.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.949+0000 7f5b35633640 1 --2- 172.21.15.16:0/466514708 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cbe9809680 0x55cbe987ba90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:54.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.949+0000 7f5b35633640 1 -- 172.21.15.16:0/466514708 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cbe97fe4a0 msgr2=0x55cbe980ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:54.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.949+0000 7f5b35633640 1 --2- 172.21.15.16:0/466514708 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cbe97fe4a0 0x55cbe980ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:54.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.950+0000 7f5b35633640 1 -- 172.21.15.16:0/466514708 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cbe987c100 con 0x55cbe97fedc0 2024-09-12T02:07:54.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.950+0000 7f5b35633640 1 --2- 172.21.15.16:0/466514708 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cbe97fedc0 0x55cbe980f1a0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x55cbe9805ab0 tx=0x7f5b2400bc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7839316d04d4d412 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:07:54.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.950+0000 7f5b157fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:07:54.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.950+0000 7f5b157fa640 1 -- 172.21.15.16:0/466514708 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b24005020 con 0x55cbe97fedc0 2024-09-12T02:07:54.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.950+0000 7f5b157fa640 1 -- 172.21.15.16:0/466514708 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f5b240051c0 con 0x55cbe97fedc0 2024-09-12T02:07:54.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.950+0000 7f5b3a471740 1 -- 172.21.15.16:0/466514708 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b2802c070 con 0x55cbe97fedc0 2024-09-12T02:07:54.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.950+0000 7f5b157fa640 1 -- 172.21.15.16:0/466514708 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5b240053e0 con 0x55cbe97fedc0 2024-09-12T02:07:54.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.950+0000 7f5b3a471740 10 client.17211 fetch_fsmap learned FSMap version 25 2024-09-12T02:07:54.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.950+0000 7f5b3a471740 1 -- 172.21.15.16:0/466514708 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55cbe9883470 con 0x55cbe97fedc0 2024-09-12T02:07:54.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.950+0000 7f5b157fa640 1 -- 172.21.15.16:0/466514708 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b2802c070 con 0x55cbe97fedc0 2024-09-12T02:07:54.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.950+0000 7f5b157fa640 1 -- 172.21.15.16:0/466514708 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f5b2401c6b0 con 0x55cbe97fedc0 2024-09-12T02:07:54.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.950+0000 7f5b3a471740 10 client.17211 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:07:54.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.950+0000 7f5b3a471740 10 client.17211 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:07:54.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.950+0000 7f5b3a471740 20 client.17211 populate_metadata read hostname 'smithi016' 2024-09-12T02:07:54.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.951+0000 7f5b3a471740 1 --2- 172.21.15.16:0/466514708 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55cbe9883c70 0x55cbe9886090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:54.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.951+0000 7f5b3a471740 4 client.17211 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T02:07:54.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.951+0000 7f5b3a471740 1 -- 172.21.15.16:0/466514708 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55cbe976bd60 con 0x55cbe9883c70 2024-09-12T02:07:54.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.951+0000 7f5b35e34640 1 --2- 172.21.15.16:0/466514708 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55cbe9883c70 0x55cbe9886090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:54.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.951+0000 7f5b157fa640 10 client.17211 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:07:54.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.951+0000 7f5b35e34640 1 --2- 172.21.15.16:0/466514708 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55cbe9883c70 0x55cbe9886090 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1dcb44c920f55ef6 server_cookie=e5ad78b628edc456 in_seq=0 out_seq=0 2024-09-12T02:07:54.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.951+0000 7f5b157fa640 1 -- 172.21.15.16:0/466514708 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55cbe976bd60 con 0x55cbe9883c70 2024-09-12T02:07:54.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.951+0000 7f5b157fa640 10 client.17211 handle_command_reply: tid=0 2024-09-12T02:07:54.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.951+0000 7f5b3a471740 1 client.17211 shutdown 2024-09-12T02:07:54.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.951+0000 7f5b3a471740 1 -- 172.21.15.16:0/466514708 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cbe97fedc0 msgr2=0x55cbe980f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:54.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.951+0000 7f5b3a471740 1 --2- 172.21.15.16:0/466514708 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cbe97fedc0 0x55cbe980f1a0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x55cbe9805ab0 tx=0x7f5b2400bc50 comp rx=0 tx=0).stop 2024-09-12T02:07:54.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.952+0000 7f5b3a471740 1 -- 172.21.15.16:0/466514708 shutdown_connections 2024-09-12T02:07:54.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.952+0000 7f5b3a471740 1 --2- 172.21.15.16:0/466514708 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55cbe9883c70 0x55cbe9886090 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:54.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.952+0000 7f5b3a471740 1 --2- 172.21.15.16:0/466514708 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cbe9809680 0x55cbe987ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:54.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.952+0000 7f5b3a471740 1 --2- 172.21.15.16:0/466514708 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cbe97fedc0 0x55cbe980f1a0 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:54.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.952+0000 7f5b157fa640 0 client.17211 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:07:54.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.952+0000 7f5b3a471740 1 --2- 172.21.15.16:0/466514708 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cbe97fe4a0 0x55cbe980ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:54.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.952+0000 7f5b3a471740 1 -- 172.21.15.16:0/466514708 >> 172.21.15.16:0/466514708 conn(0x55cbe97faa30 msgr2=0x55cbe9807a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:07:54.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.952+0000 7f5b3a471740 1 -- 172.21.15.16:0/466514708 shutdown_connections 2024-09-12T02:07:54.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.952+0000 7f5b3a471740 1 -- 172.21.15.16:0/466514708 wait complete. 2024-09-12T02:07:54.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.952+0000 7f5b3a471740 20 client.17211 trim_cache size 0 max 16384 2024-09-12T02:07:54.961 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T02:07:54.961 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-09-12T02:07:54.962 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {} 2024-09-12T02:07:54.962 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T02:07:54.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.952+0000 7f5b38098640 1 -- 172.21.15.16:0/1068763744 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f5b180a2b80 msgr2=0x7f5b180a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:54.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.952+0000 7f5b38098640 1 --2- 172.21.15.16:0/1068763744 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f5b180a2b80 0x7f5b180a5040 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f5b28021000 tx=0x7f5b2801e000 comp rx=0 tx=0).stop 2024-09-12T02:07:54.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.952+0000 7f5b38098640 1 -- 172.21.15.16:0/1068763744 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b30100a80 msgr2=0x7f5b30198410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:54.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.952+0000 7f5b38098640 1 --2- 172.21.15.16:0/1068763744 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b30100a80 0x7f5b30198410 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f5b2002ae00 tx=0x7f5b20003970 comp rx=0 tx=0).stop 2024-09-12T02:07:54.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.952+0000 7f5b38098640 1 -- 172.21.15.16:0/1068763744 shutdown_connections 2024-09-12T02:07:54.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.952+0000 7f5b38098640 1 --2- 172.21.15.16:0/1068763744 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f5b180a2b80 0x7f5b180a5040 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:54.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.952+0000 7f5b38098640 1 --2- 172.21.15.16:0/1068763744 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b30106a80 0x7f5b3019cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:54.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.952+0000 7f5b38098640 1 --2- 172.21.15.16:0/1068763744 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5b30101420 0x7f5b30198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:54.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.952+0000 7f5b38098640 1 --2- 172.21.15.16:0/1068763744 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b30100a80 0x7f5b30198410 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:54.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.952+0000 7f5b38098640 1 -- 172.21.15.16:0/1068763744 >> 172.21.15.16:0/1068763744 conn(0x7f5b300fc7a0 msgr2=0x7f5b3006a9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:07:54.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.953+0000 7f5b38098640 1 -- 172.21.15.16:0/1068763744 shutdown_connections 2024-09-12T02:07:54.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:54.953+0000 7f5b38098640 1 -- 172.21.15.16:0/1068763744 wait complete. 2024-09-12T02:07:55.592 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-12T02:07:55.592 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:07:55.899 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:07:56.366 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:07:56 smithi016 ceph-mon[30947]: pgmap v515: 97 pgs: 97 active+clean; 3.2 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 4.2 KiB/s wr, 9 op/s 2024-09-12T02:07:56.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:56.448+0000 7f0175314640 1 -- 172.21.15.16:0/822430906 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0170106f50 msgr2=0x7f017010eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:56.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:56.448+0000 7f0175314640 1 --2- 172.21.15.16:0/822430906 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0170106f50 0x7f017010eaf0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f0168009cd0 tx=0x7f016802a8e0 comp rx=0 tx=0).stop 2024-09-12T02:07:56.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:56.448+0000 7f0175314640 1 -- 172.21.15.16:0/822430906 shutdown_connections 2024-09-12T02:07:56.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:56.448+0000 7f0175314640 1 --2- 172.21.15.16:0/822430906 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0170106f50 0x7f017010eaf0 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:56.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:56.448+0000 7f0175314640 1 --2- 172.21.15.16:0/822430906 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0170100ca0 0x7f0170106840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:56.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:56.448+0000 7f0175314640 1 --2- 172.21.15.16:0/822430906 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f01701002f0 0x7f01701006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:56.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:56.448+0000 7f0175314640 1 -- 172.21.15.16:0/822430906 >> 172.21.15.16:0/822430906 conn(0x7f0170077210 msgr2=0x7f01700fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:07:56.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:56.448+0000 7f0175314640 1 -- 172.21.15.16:0/822430906 shutdown_connections 2024-09-12T02:07:56.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:56.448+0000 7f0175314640 1 -- 172.21.15.16:0/822430906 wait complete. 2024-09-12T02:07:56.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:56.449+0000 7f0175314640 1 Processor -- start 2024-09-12T02:07:56.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:56.449+0000 7f0175314640 1 -- start start 2024-09-12T02:07:56.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:56.449+0000 7f0175314640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01701002f0 0x7f01701985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:56.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:56.449+0000 7f0175314640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0170100ca0 0x7f0170198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:56.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:56.449+0000 7f0175314640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0170106f50 0x7f017019ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:56.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:56.449+0000 7f0175314640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0170112080 con 0x7f01701002f0 2024-09-12T02:07:56.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:56.449+0000 7f0175314640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0170111f00 con 0x7f0170106f50 2024-09-12T02:07:56.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:56.449+0000 7f0175314640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f0170112200 con 0x7f0170100ca0 2024-09-12T02:07:56.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:56.449+0000 7f016effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01701002f0 0x7f01701985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:56.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:56.449+0000 7f016effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01701002f0 0x7f01701985c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35692/0 (socket says 172.21.15.16:35692) 2024-09-12T02:07:56.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:56.449+0000 7f016effd640 1 -- 172.21.15.16:0/2052302376 learned_addr learned my addr 172.21.15.16:0/2052302376 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:07:56.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:56.450+0000 7f016f7fe640 1 --2- 172.21.15.16:0/2052302376 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0170106f50 0x7f017019ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:56.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:56.450+0000 7f016e7fc640 1 --2- 172.21.15.16:0/2052302376 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0170100ca0 0x7f0170198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:56.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:56.450+0000 7f016effd640 1 -- 172.21.15.16:0/2052302376 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0170100ca0 msgr2=0x7f0170198b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:56.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:56.450+0000 7f016effd640 1 --2- 172.21.15.16:0/2052302376 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0170100ca0 0x7f0170198b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:56.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:56.450+0000 7f016effd640 1 -- 172.21.15.16:0/2052302376 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0170106f50 msgr2=0x7f017019ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:56.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:56.450+0000 7f016effd640 1 --2- 172.21.15.16:0/2052302376 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0170106f50 0x7f017019ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:56.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:56.450+0000 7f016effd640 1 -- 172.21.15.16:0/2052302376 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f017019d570 con 0x7f01701002f0 2024-09-12T02:07:56.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:56.450+0000 7f016f7fe640 1 --2- 172.21.15.16:0/2052302376 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0170106f50 0x7f017019ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:07:56.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:56.450+0000 7f016effd640 1 --2- 172.21.15.16:0/2052302376 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01701002f0 0x7f01701985c0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f016000b4a0 tx=0x7f016000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:07:56.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:56.450+0000 7f0157fff640 1 -- 172.21.15.16:0/2052302376 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0160014070 con 0x7f01701002f0 2024-09-12T02:07:56.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:56.450+0000 7f0175314640 1 -- 172.21.15.16:0/2052302376 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f017019d860 con 0x7f01701002f0 2024-09-12T02:07:56.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:56.450+0000 7f0157fff640 1 -- 172.21.15.16:0/2052302376 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0160010af0 con 0x7f01701002f0 2024-09-12T02:07:56.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:56.450+0000 7f0157fff640 1 -- 172.21.15.16:0/2052302376 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0160017d10 con 0x7f01701002f0 2024-09-12T02:07:56.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:56.450+0000 7f0175314640 1 -- 172.21.15.16:0/2052302376 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01701a5140 con 0x7f01701002f0 2024-09-12T02:07:56.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:56.451+0000 7f0175314640 1 -- 172.21.15.16:0/2052302376 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0170100750 con 0x7f01701002f0 2024-09-12T02:07:56.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:56.452+0000 7f0157fff640 1 -- 172.21.15.16:0/2052302376 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f0160017070 con 0x7f01701002f0 2024-09-12T02:07:56.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:56.453+0000 7f0157fff640 1 --2- 172.21.15.16:0/2052302376 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f01580a2b80 0x7f01580a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:56.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:56.453+0000 7f0157fff640 1 -- 172.21.15.16:0/2052302376 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f01600ce490 con 0x7f01701002f0 2024-09-12T02:07:56.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:56.453+0000 7f016e7fc640 1 --2- 172.21.15.16:0/2052302376 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f01580a2b80 0x7f01580a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:56.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:56.454+0000 7f016e7fc640 1 --2- 172.21.15.16:0/2052302376 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f01580a2b80 0x7f01580a5040 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f01700ff880 tx=0x7f0164008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:07:56.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:56.457+0000 7f0157fff640 1 -- 172.21.15.16:0/2052302376 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f016009b490 con 0x7f01701002f0 2024-09-12T02:07:56.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:07:56 smithi184 ceph-mon[35593]: pgmap v515: 97 pgs: 97 active+clean; 3.2 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 4.2 KiB/s wr, 9 op/s 2024-09-12T02:07:56.668 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:07:56 smithi033 ceph-mon[35332]: pgmap v515: 97 pgs: 97 active+clean; 3.2 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 4.2 KiB/s wr, 9 op/s 2024-09-12T02:07:56.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:56.671+0000 7f0175314640 1 -- 172.21.15.16:0/2052302376 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f017010a1a0 con 0x7f01701002f0 2024-09-12T02:07:56.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:56.672+0000 7f0157fff640 1 -- 172.21.15.16:0/2052302376 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11911 (secure 0 0 0) 0x7f016009ad90 con 0x7f01701002f0 2024-09-12T02:07:56.672 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T02:07:56.673 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":25,"btime":"2024-09-12T02:01:37:192021+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3279123279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3279123279},{"type":"v1","addr":"172.21.15.16:6835","nonce":3279123279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4119452382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4119452382},{"type":"v1","addr":"172.21.15.16:6837","nonce":4119452382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/3263823165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3263823165},{"type":"v1","addr":"172.21.15.16:6839","nonce":3263823165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/1510290732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":1510290732},{"type":"v1","addr":"172.21.15.16:6841","nonce":1510290732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/279674603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":279674603},{"type":"v1","addr":"172.21.15.33:6833","nonce":279674603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3555728028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3555728028},{"type":"v1","addr":"172.21.15.33:6835","nonce":3555728028}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T01:59:56.117608+0000","modified":"2024-09-12T02:01:36.294760+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24482,"mds_1":24481,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6833/1563374106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1563374106},{"type":"v1","addr":"172.21.15.184:6833","nonce":1563374106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6837/1892833868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1892833868},{"type":"v1","addr":"172.21.15.33:6837","nonce":1892833868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/1457679686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1457679686},{"type":"v1","addr":"172.21.15.184:6835","nonce":1457679686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.184:6837/1793393087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":1793393087},{"type":"v1","addr":"172.21.15.184:6837","nonce":1793393087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.184:6839/4203347941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":4203347941},{"type":"v1","addr":"172.21.15.184:6839","nonce":4203347941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.33:6839/4136670204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":4136670204},{"type":"v1","addr":"172.21.15.33:6839","nonce":4136670204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24481,24482,24469]},"id":1}]} 2024-09-12T02:07:56.673 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 25 2024-09-12T02:07:56.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:56.674+0000 7f0175314640 1 -- 172.21.15.16:0/2052302376 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f01580a2b80 msgr2=0x7f01580a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:56.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:56.674+0000 7f0175314640 1 --2- 172.21.15.16:0/2052302376 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f01580a2b80 0x7f01580a5040 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f01700ff880 tx=0x7f0164008040 comp rx=0 tx=0).stop 2024-09-12T02:07:56.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:56.675+0000 7f0175314640 1 -- 172.21.15.16:0/2052302376 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01701002f0 msgr2=0x7f01701985c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:56.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:56.675+0000 7f0175314640 1 --2- 172.21.15.16:0/2052302376 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01701002f0 0x7f01701985c0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f016000b4a0 tx=0x7f016000b960 comp rx=0 tx=0).stop 2024-09-12T02:07:56.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:56.675+0000 7f0175314640 1 -- 172.21.15.16:0/2052302376 shutdown_connections 2024-09-12T02:07:56.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:56.675+0000 7f0175314640 1 --2- 172.21.15.16:0/2052302376 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f01580a2b80 0x7f01580a5040 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:56.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:56.675+0000 7f0175314640 1 --2- 172.21.15.16:0/2052302376 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0170106f50 0x7f017019ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:56.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:56.675+0000 7f0175314640 1 --2- 172.21.15.16:0/2052302376 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0170100ca0 0x7f0170198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:56.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:56.675+0000 7f0175314640 1 --2- 172.21.15.16:0/2052302376 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01701002f0 0x7f01701985c0 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:56.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:56.675+0000 7f0175314640 1 -- 172.21.15.16:0/2052302376 >> 172.21.15.16:0/2052302376 conn(0x7f0170077210 msgr2=0x7f01701051f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:07:56.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:56.675+0000 7f0175314640 1 -- 172.21.15.16:0/2052302376 shutdown_connections 2024-09-12T02:07:56.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:56.675+0000 7f0175314640 1 -- 172.21.15.16:0/2052302376 wait complete. 2024-09-12T02:07:57.304 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-12T02:07:57.566 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:07:57.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:07:57 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/2052302376' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:07:57.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:07:57 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/2052302376' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:07:57.671 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:07:57 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/2052302376' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:07:58.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.008+0000 7f61e3160640 1 -- 172.21.15.16:0/1844235442 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61dc102570 msgr2=0x7f61dc1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:58.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.008+0000 7f61e3160640 1 --2- 172.21.15.16:0/1844235442 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61dc102570 0x7f61dc1029b0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f61cc009900 tx=0x7f61cc02a480 comp rx=0 tx=0).stop 2024-09-12T02:07:58.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.008+0000 7f61e3160640 1 -- 172.21.15.16:0/1844235442 shutdown_connections 2024-09-12T02:07:58.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.008+0000 7f61e3160640 1 --2- 172.21.15.16:0/1844235442 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f61dc102ef0 0x7f61dc10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:58.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.008+0000 7f61e3160640 1 --2- 172.21.15.16:0/1844235442 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61dc102570 0x7f61dc1029b0 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:58.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.008+0000 7f61e3160640 1 --2- 172.21.15.16:0/1844235442 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f61dc100e20 0x7f61dc101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:58.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.008+0000 7f61e3160640 1 -- 172.21.15.16:0/1844235442 >> 172.21.15.16:0/1844235442 conn(0x7f61dc0fc7d0 msgr2=0x7f61dc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:07:58.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.008+0000 7f61e3160640 1 -- 172.21.15.16:0/1844235442 shutdown_connections 2024-09-12T02:07:58.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.008+0000 7f61e3160640 1 -- 172.21.15.16:0/1844235442 wait complete. 2024-09-12T02:07:58.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.009+0000 7f61e3160640 1 Processor -- start 2024-09-12T02:07:58.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.009+0000 7f61e3160640 1 -- start start 2024-09-12T02:07:58.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.009+0000 7f61e3160640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f61dc100e20 0x7f61dc198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:58.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.009+0000 7f61e3160640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61dc102570 0x7f61dc198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:58.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.009+0000 7f61e3160640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f61dc102ef0 0x7f61dc19cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:58.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.009+0000 7f61e3160640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f61dc110f80 con 0x7f61dc102570 2024-09-12T02:07:58.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.009+0000 7f61e3160640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f61dc110e00 con 0x7f61dc100e20 2024-09-12T02:07:58.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.009+0000 7f61e3160640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f61dc111100 con 0x7f61dc102ef0 2024-09-12T02:07:58.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.009+0000 7f61d3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61dc102570 0x7f61dc198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:58.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.009+0000 7f61d3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61dc102570 0x7f61dc198a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35706/0 (socket says 172.21.15.16:35706) 2024-09-12T02:07:58.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.009+0000 7f61d3fff640 1 -- 172.21.15.16:0/3817012606 learned_addr learned my addr 172.21.15.16:0/3817012606 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:07:58.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.009+0000 7f61e0efc640 1 --2- 172.21.15.16:0/3817012606 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f61dc100e20 0x7f61dc198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:58.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.009+0000 7f61e16fd640 1 --2- 172.21.15.16:0/3817012606 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f61dc102ef0 0x7f61dc19cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:58.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.009+0000 7f61d3fff640 1 -- 172.21.15.16:0/3817012606 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f61dc102ef0 msgr2=0x7f61dc19cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:58.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.009+0000 7f61d3fff640 1 --2- 172.21.15.16:0/3817012606 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f61dc102ef0 0x7f61dc19cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:58.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.009+0000 7f61d3fff640 1 -- 172.21.15.16:0/3817012606 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f61dc100e20 msgr2=0x7f61dc198500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:58.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.009+0000 7f61d3fff640 1 --2- 172.21.15.16:0/3817012606 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f61dc100e20 0x7f61dc198500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:58.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.009+0000 7f61d3fff640 1 -- 172.21.15.16:0/3817012606 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61dc19d4b0 con 0x7f61dc102570 2024-09-12T02:07:58.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.010+0000 7f61d3fff640 1 --2- 172.21.15.16:0/3817012606 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61dc102570 0x7f61dc198a40 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f61cc006010 tx=0x7f61cc003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:07:58.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.010+0000 7f61d1ffb640 1 -- 172.21.15.16:0/3817012606 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61cc03f070 con 0x7f61dc102570 2024-09-12T02:07:58.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.010+0000 7f61d1ffb640 1 -- 172.21.15.16:0/3817012606 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61cc031c30 con 0x7f61dc102570 2024-09-12T02:07:58.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.010+0000 7f61e3160640 1 -- 172.21.15.16:0/3817012606 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61dc19d740 con 0x7f61dc102570 2024-09-12T02:07:58.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.010+0000 7f61d1ffb640 1 -- 172.21.15.16:0/3817012606 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61cc035030 con 0x7f61dc102570 2024-09-12T02:07:58.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.010+0000 7f61e3160640 1 -- 172.21.15.16:0/3817012606 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61dc1a5080 con 0x7f61dc102570 2024-09-12T02:07:58.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.012+0000 7f61d1ffb640 1 -- 172.21.15.16:0/3817012606 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f61cc042070 con 0x7f61dc102570 2024-09-12T02:07:58.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.012+0000 7f61e5539740 1 Processor -- start 2024-09-12T02:07:58.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.012+0000 7f61e5539740 1 -- start start 2024-09-12T02:07:58.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.013+0000 7f61e5539740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fca76d4b40 0x55fca76d3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:58.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.013+0000 7f61e5539740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fca76d35d0 0x55fca76d3a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:58.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.013+0000 7f61e5539740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fca76de370 0x55fca76e0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:58.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.013+0000 7f61e5539740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55fca76392f0 con 0x55fca76d4b40 2024-09-12T02:07:58.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.013+0000 7f61e5539740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55fca75e0cf0 con 0x55fca76d35d0 2024-09-12T02:07:58.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.013+0000 7f61e5539740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55fca76352b0 con 0x55fca76de370 2024-09-12T02:07:58.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.013+0000 7f61e16fd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fca76d4b40 0x55fca76d3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:58.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.013+0000 7f61e16fd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fca76d4b40 0x55fca76d3090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35710/0 (socket says 172.21.15.16:35710) 2024-09-12T02:07:58.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.013+0000 7f61e16fd640 1 -- 172.21.15.16:0/3296051314 learned_addr learned my addr 172.21.15.16:0/3296051314 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:07:58.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.013+0000 7f61d3fff640 1 --2- 172.21.15.16:0/3296051314 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fca76de370 0x55fca76e0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:58.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.013+0000 7f61e0efc640 1 --2- 172.21.15.16:0/3296051314 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fca76d35d0 0x55fca76d3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:58.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.013+0000 7f61e16fd640 1 -- 172.21.15.16:0/3296051314 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fca76de370 msgr2=0x55fca76e0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:58.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.013+0000 7f61e16fd640 1 --2- 172.21.15.16:0/3296051314 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fca76de370 0x55fca76e0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:58.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.013+0000 7f61e16fd640 1 -- 172.21.15.16:0/3296051314 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fca76d35d0 msgr2=0x55fca76d3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:58.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.013+0000 7f61e16fd640 1 --2- 172.21.15.16:0/3296051314 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fca76d35d0 0x55fca76d3a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:58.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.013+0000 7f61e16fd640 1 -- 172.21.15.16:0/3296051314 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fca75c53a0 con 0x55fca76d4b40 2024-09-12T02:07:58.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.013+0000 7f61d3fff640 1 --2- 172.21.15.16:0/3296051314 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fca76de370 0x55fca76e0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:07:58.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.013+0000 7f61e0efc640 1 --2- 172.21.15.16:0/3296051314 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fca76d35d0 0x55fca76d3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:07:58.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.013+0000 7f61d1ffb640 1 --2- 172.21.15.16:0/3817012606 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f61c40a2c00 0x7f61c40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:58.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.013+0000 7f61e16fd640 1 --2- 172.21.15.16:0/3296051314 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fca76d4b40 0x55fca76d3090 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f61d4007dd0 tx=0x7f61d400e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f7cf3308a92ce4e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:07:58.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.013+0000 7f61e0efc640 1 --2- 172.21.15.16:0/3817012606 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f61c40a2c00 0x7f61c40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:58.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.013+0000 7f61d1ffb640 1 -- 172.21.15.16:0/3817012606 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f61cc0ef820 con 0x7f61dc102570 2024-09-12T02:07:58.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.014+0000 7f61c2ffd640 1 -- 172.21.15.16:0/3296051314 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d4014070 con 0x55fca76d4b40 2024-09-12T02:07:58.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.014+0000 7f61c2ffd640 1 -- 172.21.15.16:0/3296051314 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61d4004d20 con 0x55fca76d4b40 2024-09-12T02:07:58.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.014+0000 7f61c2ffd640 1 -- 172.21.15.16:0/3296051314 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d40108f0 con 0x55fca76d4b40 2024-09-12T02:07:58.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.014+0000 7f61e5539740 1 -- 172.21.15.16:0/3296051314 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fca76d4b40 msgr2=0x55fca76d3090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:58.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.014+0000 7f61e0efc640 1 --2- 172.21.15.16:0/3817012606 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f61c40a2c00 0x7f61c40a50c0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x55fca76d0ac0 tx=0x7f61d800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:07:58.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.014+0000 7f61e5539740 1 --2- 172.21.15.16:0/3296051314 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fca76d4b40 0x55fca76d3090 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f61d4007dd0 tx=0x7f61d400e720 comp rx=0 tx=0).stop 2024-09-12T02:07:58.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.014+0000 7f61e5539740 1 -- 172.21.15.16:0/3296051314 shutdown_connections 2024-09-12T02:07:58.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.014+0000 7f61e5539740 1 --2- 172.21.15.16:0/3296051314 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fca76de370 0x55fca76e0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:58.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.014+0000 7f61e5539740 1 --2- 172.21.15.16:0/3296051314 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fca76d35d0 0x55fca76d3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:58.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.014+0000 7f61e5539740 1 --2- 172.21.15.16:0/3296051314 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fca76d4b40 0x55fca76d3090 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:58.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.014+0000 7f61e5539740 1 -- 172.21.15.16:0/3296051314 >> 172.21.15.16:0/3296051314 conn(0x55fca769c310 msgr2=0x55fca76cfc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:07:58.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.014+0000 7f61e5539740 1 -- 172.21.15.16:0/3296051314 shutdown_connections 2024-09-12T02:07:58.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.014+0000 7f61e5539740 1 -- 172.21.15.16:0/3296051314 wait complete. 2024-09-12T02:07:58.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.015+0000 7f61e5539740 1 -- start start 2024-09-12T02:07:58.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.015+0000 7f61e5539740 1 Processor -- start 2024-09-12T02:07:58.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.015+0000 7f61e5539740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fca76d35d0 0x55fca76ddda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:58.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.015+0000 7f61e5539740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fca76d4b40 0x55fca76db050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:58.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.015+0000 7f61e5539740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fca76de370 0x55fca76db7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:58.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.015+0000 7f61e5539740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55fca76e1070 con 0x55fca76d35d0 2024-09-12T02:07:58.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.016+0000 7f61e5539740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55fca75f4060 con 0x55fca76de370 2024-09-12T02:07:58.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.016+0000 7f61e5539740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55fca76e11f0 con 0x55fca76d4b40 2024-09-12T02:07:58.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.016+0000 7f61e5539740 10 client.0 _ll_register_callbacks cb 0x55fca73478c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:07:58.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.016+0000 7f61e0efc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fca76d35d0 0x55fca76ddda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:58.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.016+0000 7f61e0efc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fca76d35d0 0x55fca76ddda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35712/0 (socket says 172.21.15.16:35712) 2024-09-12T02:07:58.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.016+0000 7f61e0efc640 1 -- 172.21.15.16:0/3859232938 learned_addr learned my addr 172.21.15.16:0/3859232938 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:07:58.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.016+0000 7f61d3fff640 1 --2- 172.21.15.16:0/3859232938 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fca76d4b40 0x55fca76db050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:58.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.016+0000 7f61e16fd640 1 --2- 172.21.15.16:0/3859232938 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fca76de370 0x55fca76db7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:58.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.016+0000 7f61e0efc640 1 -- 172.21.15.16:0/3859232938 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fca76d4b40 msgr2=0x55fca76db050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:58.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.016+0000 7f61e0efc640 1 --2- 172.21.15.16:0/3859232938 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fca76d4b40 0x55fca76db050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:58.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.016+0000 7f61e0efc640 1 -- 172.21.15.16:0/3859232938 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fca76de370 msgr2=0x55fca76db7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:58.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.016+0000 7f61e0efc640 1 --2- 172.21.15.16:0/3859232938 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fca76de370 0x55fca76db7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:58.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.016+0000 7f61e0efc640 1 -- 172.21.15.16:0/3859232938 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fca73aa3c0 con 0x55fca76d35d0 2024-09-12T02:07:58.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.016+0000 7f61e0efc640 1 --2- 172.21.15.16:0/3859232938 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fca76d35d0 0x55fca76ddda0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f61d800b5a0 tx=0x7f61d80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18f80510a249852e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:07:58.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.016+0000 7f61bbfff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:07:58.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.016+0000 7f61bbfff640 1 -- 172.21.15.16:0/3859232938 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d801b660 con 0x55fca76d35d0 2024-09-12T02:07:58.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.016+0000 7f61bbfff640 1 -- 172.21.15.16:0/3859232938 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f61d801b800 con 0x55fca76d35d0 2024-09-12T02:07:58.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.017+0000 7f61e5539740 1 -- 172.21.15.16:0/3859232938 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61d4004d20 con 0x55fca76d35d0 2024-09-12T02:07:58.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.017+0000 7f61bbfff640 1 -- 172.21.15.16:0/3859232938 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f61d801ba00 con 0x55fca76d35d0 2024-09-12T02:07:58.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.017+0000 7f61e5539740 10 client.17229 fetch_fsmap learned FSMap version 25 2024-09-12T02:07:58.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.017+0000 7f61e5539740 1 -- 172.21.15.16:0/3859232938 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55fca76ecbb0 con 0x55fca76d35d0 2024-09-12T02:07:58.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.017+0000 7f61bbfff640 1 -- 172.21.15.16:0/3859232938 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61d4004d20 con 0x55fca76d35d0 2024-09-12T02:07:58.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.017+0000 7f61bbfff640 1 -- 172.21.15.16:0/3859232938 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f61d8024070 con 0x55fca76d35d0 2024-09-12T02:07:58.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.017+0000 7f61e5539740 10 client.17229 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:07:58.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.017+0000 7f61e5539740 10 client.17229 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-12T02:07:58.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.017+0000 7f61e5539740 20 client.17229 populate_metadata read hostname 'smithi016' 2024-09-12T02:07:58.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.017+0000 7f61e5539740 1 --2- 172.21.15.16:0/3859232938 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55fca76edfc0 0x55fca76f0420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:58.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.017+0000 7f61e5539740 4 client.17229 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:07:58.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.017+0000 7f61e5539740 1 -- 172.21.15.16:0/3859232938 --> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fca7646860 con 0x55fca76edfc0 2024-09-12T02:07:58.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.017+0000 7f61d3fff640 1 --2- 172.21.15.16:0/3859232938 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55fca76edfc0 0x55fca76f0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:58.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.018+0000 7f61bbfff640 10 client.17229 ms_handle_connect on v2:172.21.15.184:6836/1793393087 2024-09-12T02:07:58.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.018+0000 7f61d3fff640 1 --2- 172.21.15.16:0/3859232938 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55fca76edfc0 0x55fca76f0420 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e2573cc791556279 server_cookie=de18e712f174f8cc in_seq=0 out_seq=0 2024-09-12T02:07:58.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.019+0000 7f61bbfff640 1 -- 172.21.15.16:0/3859232938 <== mds.2 v2:172.21.15.184:6836/1793393087 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fca7646860 con 0x55fca76edfc0 2024-09-12T02:07:58.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.019+0000 7f61bbfff640 10 client.17229 handle_command_reply: tid=0 2024-09-12T02:07:58.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.019+0000 7f61e5539740 1 client.17229 shutdown 2024-09-12T02:07:58.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.019+0000 7f61e5539740 1 -- 172.21.15.16:0/3859232938 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fca76d35d0 msgr2=0x55fca76ddda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:58.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.019+0000 7f61e5539740 1 --2- 172.21.15.16:0/3859232938 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fca76d35d0 0x55fca76ddda0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f61d800b5a0 tx=0x7f61d80046c0 comp rx=0 tx=0).stop 2024-09-12T02:07:58.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.019+0000 7f61e5539740 1 -- 172.21.15.16:0/3859232938 shutdown_connections 2024-09-12T02:07:58.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.019+0000 7f61e5539740 1 --2- 172.21.15.16:0/3859232938 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fca76de370 0x55fca76db7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:58.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.019+0000 7f61e5539740 1 --2- 172.21.15.16:0/3859232938 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fca76d4b40 0x55fca76db050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:58.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.019+0000 7f61e5539740 1 --2- 172.21.15.16:0/3859232938 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55fca76edfc0 0x55fca76f0420 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:58.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.019+0000 7f61e5539740 1 --2- 172.21.15.16:0/3859232938 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fca76d35d0 0x55fca76ddda0 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:58.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.019+0000 7f61e5539740 1 -- 172.21.15.16:0/3859232938 >> 172.21.15.16:0/3859232938 conn(0x55fca769c310 msgr2=0x55fca76d5930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:07:58.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.019+0000 7f61bbfff640 0 client.17229 ms_handle_reset on v2:172.21.15.184:6836/1793393087 2024-09-12T02:07:58.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.019+0000 7f61e5539740 1 -- 172.21.15.16:0/3859232938 shutdown_connections 2024-09-12T02:07:58.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.019+0000 7f61e5539740 1 -- 172.21.15.16:0/3859232938 wait complete. 2024-09-12T02:07:58.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.019+0000 7f61e5539740 20 client.17229 trim_cache size 0 max 16384 2024-09-12T02:07:58.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.032+0000 7f61e5539740 1 Processor -- start 2024-09-12T02:07:58.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.032+0000 7f61e5539740 1 -- start start 2024-09-12T02:07:58.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.032+0000 7f61e5539740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fca76d35d0 0x55fca76d6120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:58.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.032+0000 7f61e5539740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fca76d4b40 0x55fca76d6660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:58.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.032+0000 7f61e5539740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fca76de370 0x55fca76f5d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:58.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.032+0000 7f61e5539740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f61dc1afc90 con 0x55fca76de370 2024-09-12T02:07:58.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.032+0000 7f61e5539740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f61dc1ad560 con 0x55fca76d4b40 2024-09-12T02:07:58.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.032+0000 7f61e5539740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f61dc1ab3d0 con 0x55fca76d35d0 2024-09-12T02:07:58.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.032+0000 7f61e16fd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fca76de370 0x55fca76f5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:58.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.032+0000 7f61e16fd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fca76de370 0x55fca76f5d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35718/0 (socket says 172.21.15.16:35718) 2024-09-12T02:07:58.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.032+0000 7f61e16fd640 1 -- 172.21.15.16:0/593475854 learned_addr learned my addr 172.21.15.16:0/593475854 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:07:58.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.033+0000 7f61e0efc640 1 --2- 172.21.15.16:0/593475854 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fca76d35d0 0x55fca76d6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:58.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.033+0000 7f61d3fff640 1 --2- 172.21.15.16:0/593475854 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fca76d4b40 0x55fca76d6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:58.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.033+0000 7f61e16fd640 1 -- 172.21.15.16:0/593475854 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fca76d35d0 msgr2=0x55fca76d6120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:58.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.033+0000 7f61e16fd640 1 --2- 172.21.15.16:0/593475854 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fca76d35d0 0x55fca76d6120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:58.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.033+0000 7f61e16fd640 1 -- 172.21.15.16:0/593475854 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fca76d4b40 msgr2=0x55fca76d6660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:58.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.033+0000 7f61e16fd640 1 --2- 172.21.15.16:0/593475854 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fca76d4b40 0x55fca76d6660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:58.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.033+0000 7f61e16fd640 1 -- 172.21.15.16:0/593475854 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fca76f0aa0 con 0x55fca76de370 2024-09-12T02:07:58.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.033+0000 7f61e0efc640 1 --2- 172.21.15.16:0/593475854 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fca76d35d0 0x55fca76d6120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:07:58.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.033+0000 7f61d3fff640 1 --2- 172.21.15.16:0/593475854 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fca76d4b40 0x55fca76d6660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:07:58.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.033+0000 7f61e16fd640 1 --2- 172.21.15.16:0/593475854 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fca76de370 0x55fca76f5d40 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f61d4007dd0 tx=0x7f61d4007b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd08a2fc4fe9c5b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:07:58.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.033+0000 7f61c27fc640 1 -- 172.21.15.16:0/593475854 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d4014070 con 0x55fca76de370 2024-09-12T02:07:58.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.033+0000 7f61c27fc640 1 -- 172.21.15.16:0/593475854 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61d4004c60 con 0x55fca76de370 2024-09-12T02:07:58.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.033+0000 7f61c27fc640 1 -- 172.21.15.16:0/593475854 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d4005350 con 0x55fca76de370 2024-09-12T02:07:58.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.034+0000 7f61e5539740 1 -- 172.21.15.16:0/593475854 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fca76de370 msgr2=0x55fca76f5d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:58.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.034+0000 7f61e5539740 1 --2- 172.21.15.16:0/593475854 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fca76de370 0x55fca76f5d40 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f61d4007dd0 tx=0x7f61d4007b20 comp rx=0 tx=0).stop 2024-09-12T02:07:58.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.034+0000 7f61e5539740 1 -- 172.21.15.16:0/593475854 shutdown_connections 2024-09-12T02:07:58.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.034+0000 7f61e5539740 1 --2- 172.21.15.16:0/593475854 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fca76de370 0x55fca76f5d40 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:58.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.034+0000 7f61e5539740 1 --2- 172.21.15.16:0/593475854 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fca76d4b40 0x55fca76d6660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:58.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.034+0000 7f61e5539740 1 --2- 172.21.15.16:0/593475854 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fca76d35d0 0x55fca76d6120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:58.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.034+0000 7f61e5539740 1 -- 172.21.15.16:0/593475854 >> 172.21.15.16:0/593475854 conn(0x55fca769c310 msgr2=0x55fca76ced00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:07:58.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.034+0000 7f61e5539740 1 -- 172.21.15.16:0/593475854 shutdown_connections 2024-09-12T02:07:58.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.034+0000 7f61e5539740 1 -- 172.21.15.16:0/593475854 wait complete. 2024-09-12T02:07:58.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.034+0000 7f61e5539740 1 -- start start 2024-09-12T02:07:58.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.034+0000 7f61e5539740 1 Processor -- start 2024-09-12T02:07:58.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.035+0000 7f61e5539740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fca76d35d0 0x55fca76e2430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:58.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.035+0000 7f61e5539740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fca76d4b40 0x55fca76e2970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:58.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.035+0000 7f61e5539740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fca76de370 0x55fca76e2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:58.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.035+0000 7f61e5539740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f61dc1aba20 con 0x55fca76d4b40 2024-09-12T02:07:58.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.035+0000 7f61e5539740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f61dc1ace90 con 0x55fca76d35d0 2024-09-12T02:07:58.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.035+0000 7f61e5539740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f61dc1ac0f0 con 0x55fca76de370 2024-09-12T02:07:58.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.035+0000 7f61e5539740 10 client.0 _ll_register_callbacks cb 0x7f61d80043e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:07:58.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.035+0000 7f61d3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fca76d4b40 0x55fca76e2970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:58.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.035+0000 7f61d3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fca76d4b40 0x55fca76e2970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35722/0 (socket says 172.21.15.16:35722) 2024-09-12T02:07:58.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.035+0000 7f61d3fff640 1 -- 172.21.15.16:0/3753085361 learned_addr learned my addr 172.21.15.16:0/3753085361 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:07:58.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.035+0000 7f61e0efc640 1 --2- 172.21.15.16:0/3753085361 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fca76d35d0 0x55fca76e2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:58.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.035+0000 7f61e16fd640 1 --2- 172.21.15.16:0/3753085361 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fca76de370 0x55fca76e2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:58.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.035+0000 7f61d3fff640 1 -- 172.21.15.16:0/3753085361 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fca76de370 msgr2=0x55fca76e2eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:58.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.035+0000 7f61d3fff640 1 --2- 172.21.15.16:0/3753085361 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fca76de370 0x55fca76e2eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:58.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.035+0000 7f61d3fff640 1 -- 172.21.15.16:0/3753085361 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fca76d35d0 msgr2=0x55fca76e2430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:58.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.035+0000 7f61d3fff640 1 --2- 172.21.15.16:0/3753085361 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fca76d35d0 0x55fca76e2430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:58.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.035+0000 7f61d3fff640 1 -- 172.21.15.16:0/3753085361 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fca775c370 con 0x55fca76d4b40 2024-09-12T02:07:58.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.035+0000 7f61e0efc640 1 --2- 172.21.15.16:0/3753085361 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fca76d35d0 0x55fca76e2430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:07:58.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.035+0000 7f61d3fff640 1 --2- 172.21.15.16:0/3753085361 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fca76d4b40 0x55fca76e2970 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f61cc0b84e0 tx=0x7f61cc03b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a84b46652acac203 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:07:58.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.035+0000 7f61bbfff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:07:58.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.036+0000 7f61bbfff640 1 -- 172.21.15.16:0/3753085361 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61cc049040 con 0x55fca76d4b40 2024-09-12T02:07:58.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.036+0000 7f61bbfff640 1 -- 172.21.15.16:0/3753085361 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f61cc0ed430 con 0x55fca76d4b40 2024-09-12T02:07:58.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.036+0000 7f61e5539740 1 -- 172.21.15.16:0/3753085361 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61d4004c60 con 0x55fca76d4b40 2024-09-12T02:07:58.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.036+0000 7f61bbfff640 1 -- 172.21.15.16:0/3753085361 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f61cc0bc080 con 0x55fca76d4b40 2024-09-12T02:07:58.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.036+0000 7f61e5539740 10 client.17235 fetch_fsmap learned FSMap version 25 2024-09-12T02:07:58.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.036+0000 7f61e5539740 1 -- 172.21.15.16:0/3753085361 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55fca775d6b0 con 0x55fca76d4b40 2024-09-12T02:07:58.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.036+0000 7f61bbfff640 1 -- 172.21.15.16:0/3753085361 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61d4004c60 con 0x55fca76d4b40 2024-09-12T02:07:58.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.036+0000 7f61bbfff640 1 -- 172.21.15.16:0/3753085361 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f61cc0c0080 con 0x55fca76d4b40 2024-09-12T02:07:58.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.036+0000 7f61e5539740 10 client.17235 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:07:58.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.036+0000 7f61e5539740 10 client.17235 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-12T02:07:58.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.036+0000 7f61e5539740 20 client.17235 populate_metadata read hostname 'smithi016' 2024-09-12T02:07:58.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.036+0000 7f61e5539740 1 --2- 172.21.15.16:0/3753085361 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55fca775dde0 0x55fca7760200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:58.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.036+0000 7f61e5539740 4 client.17235 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T02:07:58.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.036+0000 7f61e5539740 1 -- 172.21.15.16:0/3753085361 --> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fca7640d70 con 0x55fca775dde0 2024-09-12T02:07:58.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.037+0000 7f61e0efc640 1 --2- 172.21.15.16:0/3753085361 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55fca775dde0 0x55fca7760200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:58.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.037+0000 7f61e0efc640 1 --2- 172.21.15.16:0/3753085361 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55fca775dde0 0x55fca7760200 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=83f574113f5a92d0 server_cookie=1859642ec81b5d37 in_seq=0 out_seq=0 2024-09-12T02:07:58.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.037+0000 7f61bbfff640 10 client.17235 ms_handle_connect on v2:172.21.15.184:6836/1793393087 2024-09-12T02:07:58.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.037+0000 7f61bbfff640 1 -- 172.21.15.16:0/3753085361 <== mds.2 v2:172.21.15.184:6836/1793393087 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fca7640d70 con 0x55fca775dde0 2024-09-12T02:07:58.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.037+0000 7f61bbfff640 10 client.17235 handle_command_reply: tid=0 2024-09-12T02:07:58.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.037+0000 7f61e5539740 1 client.17235 shutdown 2024-09-12T02:07:58.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.037+0000 7f61e5539740 1 -- 172.21.15.16:0/3753085361 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fca76d4b40 msgr2=0x55fca76e2970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:58.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.037+0000 7f61e5539740 1 --2- 172.21.15.16:0/3753085361 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fca76d4b40 0x55fca76e2970 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f61cc0b84e0 tx=0x7f61cc03b870 comp rx=0 tx=0).stop 2024-09-12T02:07:58.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.037+0000 7f61e5539740 1 -- 172.21.15.16:0/3753085361 shutdown_connections 2024-09-12T02:07:58.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.037+0000 7f61e5539740 1 --2- 172.21.15.16:0/3753085361 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fca76de370 0x55fca76e2eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:58.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.037+0000 7f61e5539740 1 --2- 172.21.15.16:0/3753085361 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55fca775dde0 0x55fca7760200 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:58.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.037+0000 7f61e5539740 1 --2- 172.21.15.16:0/3753085361 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fca76d4b40 0x55fca76e2970 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:58.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.037+0000 7f61e5539740 1 --2- 172.21.15.16:0/3753085361 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fca76d35d0 0x55fca76e2430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:58.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.037+0000 7f61e5539740 1 -- 172.21.15.16:0/3753085361 >> 172.21.15.16:0/3753085361 conn(0x55fca769c310 msgr2=0x55fca76e9300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:07:58.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.037+0000 7f61bbfff640 0 client.17235 ms_handle_reset on v2:172.21.15.184:6836/1793393087 2024-09-12T02:07:58.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.038+0000 7f61e5539740 1 -- 172.21.15.16:0/3753085361 shutdown_connections 2024-09-12T02:07:58.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.038+0000 7f61e5539740 1 -- 172.21.15.16:0/3753085361 wait complete. 2024-09-12T02:07:58.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.038+0000 7f61e5539740 20 client.17235 trim_cache size 0 max 16384 2024-09-12T02:07:58.043 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T02:07:58.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.038+0000 7f61e3160640 1 -- 172.21.15.16:0/3817012606 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f61c40a2c00 msgr2=0x7f61c40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:58.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.038+0000 7f61e3160640 1 --2- 172.21.15.16:0/3817012606 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f61c40a2c00 0x7f61c40a50c0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x55fca76d0ac0 tx=0x7f61d800b390 comp rx=0 tx=0).stop 2024-09-12T02:07:58.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.038+0000 7f61e3160640 1 -- 172.21.15.16:0/3817012606 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61dc102570 msgr2=0x7f61dc198a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:58.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.038+0000 7f61e3160640 1 --2- 172.21.15.16:0/3817012606 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61dc102570 0x7f61dc198a40 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f61cc006010 tx=0x7f61cc003c10 comp rx=0 tx=0).stop 2024-09-12T02:07:58.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.038+0000 7f61e3160640 1 -- 172.21.15.16:0/3817012606 shutdown_connections 2024-09-12T02:07:58.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.038+0000 7f61e3160640 1 --2- 172.21.15.16:0/3817012606 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f61c40a2c00 0x7f61c40a50c0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:58.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.038+0000 7f61e3160640 1 --2- 172.21.15.16:0/3817012606 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f61dc102ef0 0x7f61dc19cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:58.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.038+0000 7f61e3160640 1 --2- 172.21.15.16:0/3817012606 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61dc102570 0x7f61dc198a40 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:58.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.038+0000 7f61e3160640 1 --2- 172.21.15.16:0/3817012606 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f61dc100e20 0x7f61dc198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:58.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.038+0000 7f61e3160640 1 -- 172.21.15.16:0/3817012606 >> 172.21.15.16:0/3817012606 conn(0x7f61dc0fc7d0 msgr2=0x7f61dc06a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:07:58.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.038+0000 7f61e3160640 1 -- 172.21.15.16:0/3817012606 shutdown_connections 2024-09-12T02:07:58.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:58.038+0000 7f61e3160640 1 -- 172.21.15.16:0/3817012606 wait complete. 2024-09-12T02:07:58.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:07:58 smithi016 ceph-mon[30947]: pgmap v516: 97 pgs: 97 active+clean; 3.2 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 2.8 KiB/s wr, 8 op/s 2024-09-12T02:07:58.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:07:58 smithi184 ceph-mon[35593]: pgmap v516: 97 pgs: 97 active+clean; 3.2 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 2.8 KiB/s wr, 8 op/s 2024-09-12T02:07:58.944 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:07:58 smithi033 ceph-mon[35332]: pgmap v516: 97 pgs: 97 active+clean; 3.2 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 2.8 KiB/s wr, 8 op/s 2024-09-12T02:07:58.945 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-12T02:07:59.204 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:07:59.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.673+0000 7f8bfab5a640 1 -- 172.21.15.16:0/2874638568 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8bf410c180 msgr2=0x7f8bf410e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:59.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.673+0000 7f8bfab5a640 1 --2- 172.21.15.16:0/2874638568 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8bf410c180 0x7f8bf410e610 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f8bec009ca0 tx=0x7f8bec02a950 comp rx=0 tx=0).stop 2024-09-12T02:07:59.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.673+0000 7f8bfab5a640 1 -- 172.21.15.16:0/2874638568 shutdown_connections 2024-09-12T02:07:59.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.673+0000 7f8bfab5a640 1 --2- 172.21.15.16:0/2874638568 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8bf410c180 0x7f8bf410e610 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:59.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.674+0000 7f8bfab5a640 1 --2- 172.21.15.16:0/2874638568 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8bf4103700 0x7f8bf410bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:59.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.674+0000 7f8bfab5a640 1 --2- 172.21.15.16:0/2874638568 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8bf4102de0 0x7f8bf41031c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:59.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.674+0000 7f8bfab5a640 1 -- 172.21.15.16:0/2874638568 >> 172.21.15.16:0/2874638568 conn(0x7f8bf40fc7a0 msgr2=0x7f8bf40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:07:59.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.674+0000 7f8bfab5a640 1 -- 172.21.15.16:0/2874638568 shutdown_connections 2024-09-12T02:07:59.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.674+0000 7f8bfab5a640 1 -- 172.21.15.16:0/2874638568 wait complete. 2024-09-12T02:07:59.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.674+0000 7f8bfab5a640 1 Processor -- start 2024-09-12T02:07:59.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.674+0000 7f8bfab5a640 1 -- start start 2024-09-12T02:07:59.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.674+0000 7f8bfab5a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8bf4102de0 0x7f8bf4198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:59.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.674+0000 7f8bfab5a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8bf4103700 0x7f8bf41988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:59.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.675+0000 7f8bfab5a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8bf410c180 0x7f8bf419cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:59.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.675+0000 7f8bfab5a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8bf4111f30 con 0x7f8bf4103700 2024-09-12T02:07:59.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.675+0000 7f8bfab5a640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8bf4111db0 con 0x7f8bf410c180 2024-09-12T02:07:59.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.675+0000 7f8bfab5a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8bf41120b0 con 0x7f8bf4102de0 2024-09-12T02:07:59.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.675+0000 7f8bf3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8bf4103700 0x7f8bf41988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:59.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.675+0000 7f8bf3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8bf4103700 0x7f8bf41988b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35738/0 (socket says 172.21.15.16:35738) 2024-09-12T02:07:59.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.675+0000 7f8bf3fff640 1 -- 172.21.15.16:0/3180961744 learned_addr learned my addr 172.21.15.16:0/3180961744 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:07:59.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.675+0000 7f8bf90f7640 1 --2- 172.21.15.16:0/3180961744 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8bf410c180 0x7f8bf419cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:59.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.675+0000 7f8bf88f6640 1 --2- 172.21.15.16:0/3180961744 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8bf4102de0 0x7f8bf4198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:59.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.675+0000 7f8bf3fff640 1 -- 172.21.15.16:0/3180961744 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8bf4102de0 msgr2=0x7f8bf4198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:59.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.675+0000 7f8bf3fff640 1 --2- 172.21.15.16:0/3180961744 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8bf4102de0 0x7f8bf4198370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:59.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.675+0000 7f8bf3fff640 1 -- 172.21.15.16:0/3180961744 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8bf410c180 msgr2=0x7f8bf419cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:59.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.675+0000 7f8bf3fff640 1 --2- 172.21.15.16:0/3180961744 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8bf410c180 0x7f8bf419cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:59.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.675+0000 7f8bf3fff640 1 -- 172.21.15.16:0/3180961744 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bf419d320 con 0x7f8bf4103700 2024-09-12T02:07:59.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.675+0000 7f8bf90f7640 1 --2- 172.21.15.16:0/3180961744 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8bf410c180 0x7f8bf419cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:07:59.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.675+0000 7f8bf3fff640 1 --2- 172.21.15.16:0/3180961744 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8bf4103700 0x7f8bf41988b0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f8be800d950 tx=0x7f8be800dd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:07:59.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.675+0000 7f8bf1ffb640 1 -- 172.21.15.16:0/3180961744 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8be8017070 con 0x7f8bf4103700 2024-09-12T02:07:59.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.675+0000 7f8bf1ffb640 1 -- 172.21.15.16:0/3180961744 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8be800de80 con 0x7f8bf4103700 2024-09-12T02:07:59.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.675+0000 7f8bfab5a640 1 -- 172.21.15.16:0/3180961744 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bf419d610 con 0x7f8bf4103700 2024-09-12T02:07:59.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.676+0000 7f8bf1ffb640 1 -- 172.21.15.16:0/3180961744 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8be800f620 con 0x7f8bf4103700 2024-09-12T02:07:59.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.676+0000 7f8bfab5a640 1 -- 172.21.15.16:0/3180961744 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bf41a4ef0 con 0x7f8bf4103700 2024-09-12T02:07:59.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.678+0000 7f8bf1ffb640 1 -- 172.21.15.16:0/3180961744 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f8be8009c30 con 0x7f8bf4103700 2024-09-12T02:07:59.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.678+0000 7f8bfcf33740 1 Processor -- start 2024-09-12T02:07:59.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.678+0000 7f8bfcf33740 1 -- start start 2024-09-12T02:07:59.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.679+0000 7f8bfcf33740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561c95bceb40 0x561c95bcd090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:59.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.679+0000 7f8bfcf33740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561c95bcd5d0 0x561c95bcda10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:59.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.679+0000 7f8bfcf33740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561c95bd8370 0x561c95bda760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:59.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.679+0000 7f8bf1ffb640 1 --2- 172.21.15.16:0/3180961744 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f8bdc0a2c00 0x7f8bdc0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:59.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.679+0000 7f8bfcf33740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x561c95b332f0 con 0x561c95bceb40 2024-09-12T02:07:59.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.679+0000 7f8bfcf33740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x561c95adacf0 con 0x561c95bcd5d0 2024-09-12T02:07:59.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.679+0000 7f8bfcf33740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x561c95b2f2b0 con 0x561c95bd8370 2024-09-12T02:07:59.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.679+0000 7f8bf90f7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561c95bceb40 0x561c95bcd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:59.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.679+0000 7f8bf90f7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561c95bceb40 0x561c95bcd090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35750/0 (socket says 172.21.15.16:35750) 2024-09-12T02:07:59.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.679+0000 7f8bf88f6640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561c95bcd5d0 0x561c95bcda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:59.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.679+0000 7f8bf90f7640 1 -- 172.21.15.16:0/4201222713 learned_addr learned my addr 172.21.15.16:0/4201222713 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:07:59.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.679+0000 7f8bf1ffb640 1 -- 172.21.15.16:0/3180961744 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f8be80ceae0 con 0x7f8bf4103700 2024-09-12T02:07:59.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.679+0000 7f8bf3fff640 1 --2- 172.21.15.16:0/3180961744 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f8bdc0a2c00 0x7f8bdc0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:59.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.679+0000 7f8bf90f7640 1 --2- 172.21.15.16:0/4201222713 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561c95bd8370 0x561c95bda760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:59.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.679+0000 7f8bf90f7640 1 -- 172.21.15.16:0/4201222713 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561c95bd8370 msgr2=0x561c95bda760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:59.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.679+0000 7f8bf90f7640 1 --2- 172.21.15.16:0/4201222713 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561c95bd8370 0x561c95bda760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:59.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.679+0000 7f8bf90f7640 1 -- 172.21.15.16:0/4201222713 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561c95bcd5d0 msgr2=0x561c95bcda10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:59.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.679+0000 7f8bf90f7640 1 --2- 172.21.15.16:0/4201222713 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561c95bcd5d0 0x561c95bcda10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:59.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.679+0000 7f8bf90f7640 1 -- 172.21.15.16:0/4201222713 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c95abf3a0 con 0x561c95bceb40 2024-09-12T02:07:59.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.679+0000 7f8bf88f6640 1 --2- 172.21.15.16:0/4201222713 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561c95bcd5d0 0x561c95bcda10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:07:59.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.679+0000 7f8bf90f7640 1 --2- 172.21.15.16:0/4201222713 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561c95bceb40 0x561c95bcd090 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f8bec030040 tx=0x7f8bec02fd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=946a343ebf71bafe server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:07:59.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.679+0000 7f8bf3fff640 1 --2- 172.21.15.16:0/3180961744 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f8bdc0a2c00 0x7f8bdc0a50c0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f8be80cc680 tx=0x7f8be8000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:07:59.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.680+0000 7f8bdaffd640 1 -- 172.21.15.16:0/4201222713 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bec034040 con 0x561c95bceb40 2024-09-12T02:07:59.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.680+0000 7f8bdaffd640 1 -- 172.21.15.16:0/4201222713 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8bec044040 con 0x561c95bceb40 2024-09-12T02:07:59.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.680+0000 7f8bdaffd640 1 -- 172.21.15.16:0/4201222713 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bec030440 con 0x561c95bceb40 2024-09-12T02:07:59.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.680+0000 7f8bfcf33740 1 -- 172.21.15.16:0/4201222713 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561c95bceb40 msgr2=0x561c95bcd090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:59.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.680+0000 7f8bfcf33740 1 --2- 172.21.15.16:0/4201222713 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561c95bceb40 0x561c95bcd090 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f8bec030040 tx=0x7f8bec02fd30 comp rx=0 tx=0).stop 2024-09-12T02:07:59.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.680+0000 7f8bfcf33740 1 -- 172.21.15.16:0/4201222713 shutdown_connections 2024-09-12T02:07:59.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.680+0000 7f8bfcf33740 1 --2- 172.21.15.16:0/4201222713 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561c95bd8370 0x561c95bda760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:59.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.680+0000 7f8bfcf33740 1 --2- 172.21.15.16:0/4201222713 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561c95bcd5d0 0x561c95bcda10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:59.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.680+0000 7f8bfcf33740 1 --2- 172.21.15.16:0/4201222713 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561c95bceb40 0x561c95bcd090 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:59.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.680+0000 7f8bfcf33740 1 -- 172.21.15.16:0/4201222713 >> 172.21.15.16:0/4201222713 conn(0x561c95b96310 msgr2=0x561c95bc9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:07:59.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.680+0000 7f8bfcf33740 1 -- 172.21.15.16:0/4201222713 shutdown_connections 2024-09-12T02:07:59.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.680+0000 7f8bfcf33740 1 -- 172.21.15.16:0/4201222713 wait complete. 2024-09-12T02:07:59.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.681+0000 7f8bfcf33740 1 -- start start 2024-09-12T02:07:59.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.681+0000 7f8bfcf33740 1 Processor -- start 2024-09-12T02:07:59.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.681+0000 7f8bfcf33740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561c95bcd5d0 0x561c95bd7dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:59.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.681+0000 7f8bfcf33740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561c95bceb40 0x561c95bd5080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:59.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.681+0000 7f8bfcf33740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561c95bd8370 0x561c95bd5810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:59.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.681+0000 7f8bfcf33740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x561c95bdb010 con 0x561c95bd8370 2024-09-12T02:07:59.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.681+0000 7f8bfcf33740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x561c95aee060 con 0x561c95bcd5d0 2024-09-12T02:07:59.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.681+0000 7f8bfcf33740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x561c95bdb190 con 0x561c95bceb40 2024-09-12T02:07:59.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.681+0000 7f8bfcf33740 10 client.0 _ll_register_callbacks cb 0x561c958418c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:07:59.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.681+0000 7f8bf90f7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561c95bd8370 0x561c95bd5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:59.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.681+0000 7f8bf90f7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561c95bd8370 0x561c95bd5810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35754/0 (socket says 172.21.15.16:35754) 2024-09-12T02:07:59.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.681+0000 7f8bf90f7640 1 -- 172.21.15.16:0/816514855 learned_addr learned my addr 172.21.15.16:0/816514855 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:07:59.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.682+0000 7f8bf90f7640 1 --2- 172.21.15.16:0/816514855 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561c95bcd5d0 0x561c95bd7dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:59.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.682+0000 7f8bf88f6640 1 --2- 172.21.15.16:0/816514855 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561c95bceb40 0x561c95bd5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:59.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.682+0000 7f8bf90f7640 1 -- 172.21.15.16:0/816514855 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561c95bceb40 msgr2=0x561c95bd5080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:59.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.682+0000 7f8bf90f7640 1 --2- 172.21.15.16:0/816514855 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561c95bceb40 0x561c95bd5080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:59.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.682+0000 7f8bf90f7640 1 -- 172.21.15.16:0/816514855 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561c95bcd5d0 msgr2=0x561c95bd7dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:59.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.682+0000 7f8bf90f7640 1 --2- 172.21.15.16:0/816514855 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561c95bcd5d0 0x561c95bd7dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:59.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.682+0000 7f8bf90f7640 1 -- 172.21.15.16:0/816514855 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c958a43c0 con 0x561c95bd8370 2024-09-12T02:07:59.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.682+0000 7f8bf90f7640 1 --2- 172.21.15.16:0/816514855 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561c95bd8370 0x561c95bd5810 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f8bec02f8e0 tx=0x7f8bec009820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e23d9f4a6e56a397 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:07:59.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.682+0000 7f8bd3fff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:07:59.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.682+0000 7f8bd3fff640 1 -- 172.21.15.16:0/816514855 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bec034050 con 0x561c95bd8370 2024-09-12T02:07:59.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.682+0000 7f8bd3fff640 1 -- 172.21.15.16:0/816514855 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f8bec003970 con 0x561c95bd8370 2024-09-12T02:07:59.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.682+0000 7f8bfcf33740 1 -- 172.21.15.16:0/816514855 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8bec044040 con 0x561c95bd8370 2024-09-12T02:07:59.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.682+0000 7f8bd3fff640 1 -- 172.21.15.16:0/816514855 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f8bec05e050 con 0x561c95bd8370 2024-09-12T02:07:59.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.682+0000 7f8bfcf33740 10 client.17247 fetch_fsmap learned FSMap version 25 2024-09-12T02:07:59.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.682+0000 7f8bfcf33740 1 -- 172.21.15.16:0/816514855 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x561c95be6b60 con 0x561c95bd8370 2024-09-12T02:07:59.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.683+0000 7f8bd3fff640 1 -- 172.21.15.16:0/816514855 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8bec044040 con 0x561c95bd8370 2024-09-12T02:07:59.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.683+0000 7f8bd3fff640 1 -- 172.21.15.16:0/816514855 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f8bec056020 con 0x561c95bd8370 2024-09-12T02:07:59.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.683+0000 7f8bfcf33740 10 client.17247 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:07:59.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.683+0000 7f8bfcf33740 10 client.17247 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T02:07:59.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.683+0000 7f8bfcf33740 20 client.17247 populate_metadata read hostname 'smithi016' 2024-09-12T02:07:59.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.683+0000 7f8bfcf33740 1 --2- 172.21.15.16:0/816514855 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x561c95be7eb0 0x561c95bea2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:59.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.683+0000 7f8bfcf33740 4 client.17247 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:07:59.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.683+0000 7f8bfcf33740 1 -- 172.21.15.16:0/816514855 --> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561c95b40860 con 0x561c95be7eb0 2024-09-12T02:07:59.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.683+0000 7f8bf88f6640 1 --2- 172.21.15.16:0/816514855 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x561c95be7eb0 0x561c95bea2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:59.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.684+0000 7f8bf88f6640 1 --2- 172.21.15.16:0/816514855 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x561c95be7eb0 0x561c95bea2a0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5f559eac5ef450c server_cookie=fbf9c6f3a7063322 in_seq=0 out_seq=0 2024-09-12T02:07:59.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.684+0000 7f8bd3fff640 10 client.17247 ms_handle_connect on v2:172.21.15.184:6838/4203347941 2024-09-12T02:07:59.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.684+0000 7f8bd3fff640 1 -- 172.21.15.16:0/816514855 <== mds.1 v2:172.21.15.184:6838/4203347941 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561c95b40860 con 0x561c95be7eb0 2024-09-12T02:07:59.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.684+0000 7f8bd3fff640 10 client.17247 handle_command_reply: tid=0 2024-09-12T02:07:59.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.684+0000 7f8bfcf33740 1 client.17247 shutdown 2024-09-12T02:07:59.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.685+0000 7f8bfcf33740 1 -- 172.21.15.16:0/816514855 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561c95bd8370 msgr2=0x561c95bd5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:59.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.685+0000 7f8bfcf33740 1 --2- 172.21.15.16:0/816514855 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561c95bd8370 0x561c95bd5810 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f8bec02f8e0 tx=0x7f8bec009820 comp rx=0 tx=0).stop 2024-09-12T02:07:59.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.685+0000 7f8bfcf33740 1 -- 172.21.15.16:0/816514855 shutdown_connections 2024-09-12T02:07:59.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.685+0000 7f8bfcf33740 1 --2- 172.21.15.16:0/816514855 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x561c95be7eb0 0x561c95bea2a0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:59.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.685+0000 7f8bd3fff640 0 client.17247 ms_handle_reset on v2:172.21.15.184:6838/4203347941 2024-09-12T02:07:59.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.685+0000 7f8bfcf33740 1 --2- 172.21.15.16:0/816514855 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561c95bd8370 0x561c95bd5810 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:59.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.685+0000 7f8bfcf33740 1 --2- 172.21.15.16:0/816514855 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561c95bceb40 0x561c95bd5080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:59.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.685+0000 7f8bfcf33740 1 --2- 172.21.15.16:0/816514855 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561c95bcd5d0 0x561c95bd7dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:59.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.685+0000 7f8bfcf33740 1 -- 172.21.15.16:0/816514855 >> 172.21.15.16:0/816514855 conn(0x561c95b96310 msgr2=0x561c95bcf930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:07:59.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.685+0000 7f8bfcf33740 1 -- 172.21.15.16:0/816514855 shutdown_connections 2024-09-12T02:07:59.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.685+0000 7f8bfcf33740 1 -- 172.21.15.16:0/816514855 wait complete. 2024-09-12T02:07:59.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.685+0000 7f8bfcf33740 20 client.17247 trim_cache size 0 max 16384 2024-09-12T02:07:59.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.698+0000 7f8bfcf33740 1 Processor -- start 2024-09-12T02:07:59.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.698+0000 7f8bfcf33740 1 -- start start 2024-09-12T02:07:59.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.698+0000 7f8bfcf33740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561c95bcd5d0 0x561c95bd0120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:59.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.698+0000 7f8bfcf33740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561c95bceb40 0x561c95bd0660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:59.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.698+0000 7f8bfcf33740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561c95bd8370 0x561c95bed2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:59.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.698+0000 7f8bfcf33740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8bf41afcd0 con 0x561c95bceb40 2024-09-12T02:07:59.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.698+0000 7f8bfcf33740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8bf41ad5a0 con 0x561c95bd8370 2024-09-12T02:07:59.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.698+0000 7f8bfcf33740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8bf41ab3f0 con 0x561c95bcd5d0 2024-09-12T02:07:59.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.698+0000 7f8bf88f6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561c95bceb40 0x561c95bd0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:59.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.698+0000 7f8bf88f6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561c95bceb40 0x561c95bd0660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35768/0 (socket says 172.21.15.16:35768) 2024-09-12T02:07:59.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.698+0000 7f8bf88f6640 1 -- 172.21.15.16:0/2767801355 learned_addr learned my addr 172.21.15.16:0/2767801355 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:07:59.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.699+0000 7f8bf90f7640 1 --2- 172.21.15.16:0/2767801355 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561c95bcd5d0 0x561c95bd0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:59.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.699+0000 7f8bf88f6640 1 -- 172.21.15.16:0/2767801355 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561c95bcd5d0 msgr2=0x561c95bd0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:59.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.699+0000 7f8bf88f6640 1 --2- 172.21.15.16:0/2767801355 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561c95bcd5d0 0x561c95bd0120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:59.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.699+0000 7f8bf88f6640 1 -- 172.21.15.16:0/2767801355 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561c95bd8370 msgr2=0x561c95bed2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:59.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.699+0000 7f8bf88f6640 1 --2- 172.21.15.16:0/2767801355 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561c95bd8370 0x561c95bed2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:59.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.699+0000 7f8bf88f6640 1 -- 172.21.15.16:0/2767801355 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c95bc9510 con 0x561c95bceb40 2024-09-12T02:07:59.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.699+0000 7f8bf90f7640 1 --2- 172.21.15.16:0/2767801355 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561c95bcd5d0 0x561c95bd0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:07:59.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.699+0000 7f8bf88f6640 1 --2- 172.21.15.16:0/2767801355 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561c95bceb40 0x561c95bd0660 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f8be4005e10 tx=0x7f8be4010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67105aefa94489b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:07:59.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.699+0000 7f8bda7fc640 1 -- 172.21.15.16:0/2767801355 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8be401d670 con 0x561c95bceb40 2024-09-12T02:07:59.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.699+0000 7f8bda7fc640 1 -- 172.21.15.16:0/2767801355 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8be400ae90 con 0x561c95bceb40 2024-09-12T02:07:59.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.699+0000 7f8bda7fc640 1 -- 172.21.15.16:0/2767801355 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8be400fcd0 con 0x561c95bceb40 2024-09-12T02:07:59.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.700+0000 7f8bfcf33740 1 -- 172.21.15.16:0/2767801355 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561c95bceb40 msgr2=0x561c95bd0660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:59.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.700+0000 7f8bfcf33740 1 --2- 172.21.15.16:0/2767801355 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561c95bceb40 0x561c95bd0660 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f8be4005e10 tx=0x7f8be4010800 comp rx=0 tx=0).stop 2024-09-12T02:07:59.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.700+0000 7f8bfcf33740 1 -- 172.21.15.16:0/2767801355 shutdown_connections 2024-09-12T02:07:59.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.700+0000 7f8bfcf33740 1 --2- 172.21.15.16:0/2767801355 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561c95bd8370 0x561c95bed2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:59.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.700+0000 7f8bfcf33740 1 --2- 172.21.15.16:0/2767801355 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561c95bceb40 0x561c95bd0660 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:59.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.700+0000 7f8bfcf33740 1 --2- 172.21.15.16:0/2767801355 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561c95bcd5d0 0x561c95bd0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:59.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.700+0000 7f8bfcf33740 1 -- 172.21.15.16:0/2767801355 >> 172.21.15.16:0/2767801355 conn(0x561c95b96310 msgr2=0x561c95bd4170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:07:59.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.700+0000 7f8bfcf33740 1 -- 172.21.15.16:0/2767801355 shutdown_connections 2024-09-12T02:07:59.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.700+0000 7f8bfcf33740 1 -- 172.21.15.16:0/2767801355 wait complete. 2024-09-12T02:07:59.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.700+0000 7f8bfcf33740 1 -- start start 2024-09-12T02:07:59.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.700+0000 7f8bfcf33740 1 Processor -- start 2024-09-12T02:07:59.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.700+0000 7f8bfcf33740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561c95bcd5d0 0x561c95beda90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:59.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.700+0000 7f8bfcf33740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561c95bceb40 0x561c95beae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:59.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.700+0000 7f8bfcf33740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561c95bd8370 0x561c95beb390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:59.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.700+0000 7f8bfcf33740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8bf41aba60 con 0x561c95bd8370 2024-09-12T02:07:59.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.701+0000 7f8bfcf33740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8bf41aced0 con 0x561c95bceb40 2024-09-12T02:07:59.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.701+0000 7f8bfcf33740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8bf41ac130 con 0x561c95bcd5d0 2024-09-12T02:07:59.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.701+0000 7f8bfcf33740 10 client.0 _ll_register_callbacks cb 0x7f8be400a630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:07:59.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.701+0000 7f8bf90f7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561c95bd8370 0x561c95beb390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:59.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.701+0000 7f8bf90f7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561c95bd8370 0x561c95beb390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35780/0 (socket says 172.21.15.16:35780) 2024-09-12T02:07:59.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.701+0000 7f8bf90f7640 1 -- 172.21.15.16:0/654909504 learned_addr learned my addr 172.21.15.16:0/654909504 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:07:59.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.701+0000 7f8bf88f6640 1 --2- 172.21.15.16:0/654909504 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561c95bceb40 0x561c95beae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:59.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.701+0000 7f8bf90f7640 1 --2- 172.21.15.16:0/654909504 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561c95bcd5d0 0x561c95beda90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:59.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.701+0000 7f8bf90f7640 1 -- 172.21.15.16:0/654909504 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561c95bcd5d0 msgr2=0x561c95beda90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:59.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.701+0000 7f8bf90f7640 1 --2- 172.21.15.16:0/654909504 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561c95bcd5d0 0x561c95beda90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:59.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.701+0000 7f8bf90f7640 1 -- 172.21.15.16:0/654909504 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561c95bceb40 msgr2=0x561c95beae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:59.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.701+0000 7f8bf90f7640 1 --2- 172.21.15.16:0/654909504 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561c95bceb40 0x561c95beae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:59.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.701+0000 7f8bf90f7640 1 -- 172.21.15.16:0/654909504 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c95bebbc0 con 0x561c95bd8370 2024-09-12T02:07:59.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.701+0000 7f8bf88f6640 1 --2- 172.21.15.16:0/654909504 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561c95bceb40 0x561c95beae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:07:59.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.701+0000 7f8bf90f7640 1 --2- 172.21.15.16:0/654909504 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561c95bd8370 0x561c95beb390 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f8bec009bf0 tx=0x7f8bec005d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=747197504945ae75 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:07:59.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.701+0000 7f8bd3fff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:07:59.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.701+0000 7f8bd3fff640 1 -- 172.21.15.16:0/654909504 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bec034070 con 0x561c95bd8370 2024-09-12T02:07:59.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.701+0000 7f8bfcf33740 1 -- 172.21.15.16:0/654909504 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8be400ae90 con 0x561c95bd8370 2024-09-12T02:07:59.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.701+0000 7f8bd3fff640 1 -- 172.21.15.16:0/654909504 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f8bec053040 con 0x561c95bd8370 2024-09-12T02:07:59.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.702+0000 7f8bd3fff640 1 -- 172.21.15.16:0/654909504 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f8bec004620 con 0x561c95bd8370 2024-09-12T02:07:59.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.702+0000 7f8bfcf33740 10 client.17253 fetch_fsmap learned FSMap version 25 2024-09-12T02:07:59.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.702+0000 7f8bfcf33740 1 -- 172.21.15.16:0/654909504 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x561c95c56c00 con 0x561c95bd8370 2024-09-12T02:07:59.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.702+0000 7f8bd3fff640 1 -- 172.21.15.16:0/654909504 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8be400ae90 con 0x561c95bd8370 2024-09-12T02:07:59.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.702+0000 7f8bd3fff640 1 -- 172.21.15.16:0/654909504 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f8bec056040 con 0x561c95bd8370 2024-09-12T02:07:59.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.702+0000 7f8bfcf33740 10 client.17253 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:07:59.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.702+0000 7f8bfcf33740 10 client.17253 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T02:07:59.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.702+0000 7f8bfcf33740 20 client.17253 populate_metadata read hostname 'smithi016' 2024-09-12T02:07:59.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.702+0000 7f8bfcf33740 1 --2- 172.21.15.16:0/654909504 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x561c95be7eb0 0x561c95c59270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:59.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.702+0000 7f8bfcf33740 4 client.17253 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T02:07:59.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.702+0000 7f8bfcf33740 1 -- 172.21.15.16:0/654909504 --> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561c95b3ad70 con 0x561c95be7eb0 2024-09-12T02:07:59.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.702+0000 7f8bf88f6640 1 --2- 172.21.15.16:0/654909504 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x561c95be7eb0 0x561c95c59270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:59.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.703+0000 7f8bf88f6640 1 --2- 172.21.15.16:0/654909504 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x561c95be7eb0 0x561c95c59270 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e27639db75cb84bb server_cookie=b328b4121dec082a in_seq=0 out_seq=0 2024-09-12T02:07:59.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.703+0000 7f8bd3fff640 10 client.17253 ms_handle_connect on v2:172.21.15.184:6838/4203347941 2024-09-12T02:07:59.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.703+0000 7f8bd3fff640 1 -- 172.21.15.16:0/654909504 <== mds.1 v2:172.21.15.184:6838/4203347941 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561c95b3ad70 con 0x561c95be7eb0 2024-09-12T02:07:59.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.703+0000 7f8bd3fff640 10 client.17253 handle_command_reply: tid=0 2024-09-12T02:07:59.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.703+0000 7f8bfcf33740 1 client.17253 shutdown 2024-09-12T02:07:59.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.703+0000 7f8bfcf33740 1 -- 172.21.15.16:0/654909504 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561c95bd8370 msgr2=0x561c95beb390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:59.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.703+0000 7f8bfcf33740 1 --2- 172.21.15.16:0/654909504 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561c95bd8370 0x561c95beb390 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f8bec009bf0 tx=0x7f8bec005d80 comp rx=0 tx=0).stop 2024-09-12T02:07:59.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.703+0000 7f8bfcf33740 1 -- 172.21.15.16:0/654909504 shutdown_connections 2024-09-12T02:07:59.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.703+0000 7f8bfcf33740 1 --2- 172.21.15.16:0/654909504 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x561c95be7eb0 0x561c95c59270 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:59.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.703+0000 7f8bfcf33740 1 --2- 172.21.15.16:0/654909504 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561c95bd8370 0x561c95beb390 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:59.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.703+0000 7f8bfcf33740 1 --2- 172.21.15.16:0/654909504 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561c95bceb40 0x561c95beae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:59.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.703+0000 7f8bd3fff640 0 client.17253 ms_handle_reset on v2:172.21.15.184:6838/4203347941 2024-09-12T02:07:59.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.703+0000 7f8bfcf33740 1 --2- 172.21.15.16:0/654909504 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561c95bcd5d0 0x561c95beda90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:59.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.703+0000 7f8bfcf33740 1 -- 172.21.15.16:0/654909504 >> 172.21.15.16:0/654909504 conn(0x561c95b96310 msgr2=0x561c95bc84e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:07:59.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.703+0000 7f8bfcf33740 1 -- 172.21.15.16:0/654909504 shutdown_connections 2024-09-12T02:07:59.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.703+0000 7f8bfcf33740 1 -- 172.21.15.16:0/654909504 wait complete. 2024-09-12T02:07:59.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.704+0000 7f8bfcf33740 20 client.17253 trim_cache size 0 max 16384 2024-09-12T02:07:59.710 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T02:07:59.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.704+0000 7f8bfab5a640 1 -- 172.21.15.16:0/3180961744 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f8bdc0a2c00 msgr2=0x7f8bdc0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:59.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.704+0000 7f8bfab5a640 1 --2- 172.21.15.16:0/3180961744 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f8bdc0a2c00 0x7f8bdc0a50c0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f8be80cc680 tx=0x7f8be8000bc0 comp rx=0 tx=0).stop 2024-09-12T02:07:59.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.704+0000 7f8bfab5a640 1 -- 172.21.15.16:0/3180961744 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8bf4103700 msgr2=0x7f8bf41988b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:59.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.704+0000 7f8bfab5a640 1 --2- 172.21.15.16:0/3180961744 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8bf4103700 0x7f8bf41988b0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f8be800d950 tx=0x7f8be800dd50 comp rx=0 tx=0).stop 2024-09-12T02:07:59.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.704+0000 7f8bfab5a640 1 -- 172.21.15.16:0/3180961744 shutdown_connections 2024-09-12T02:07:59.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.704+0000 7f8bfab5a640 1 --2- 172.21.15.16:0/3180961744 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f8bdc0a2c00 0x7f8bdc0a50c0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:59.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.704+0000 7f8bfab5a640 1 --2- 172.21.15.16:0/3180961744 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8bf410c180 0x7f8bf419cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:59.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.704+0000 7f8bfab5a640 1 --2- 172.21.15.16:0/3180961744 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8bf4103700 0x7f8bf41988b0 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:59.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.704+0000 7f8bfab5a640 1 --2- 172.21.15.16:0/3180961744 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8bf4102de0 0x7f8bf4198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:59.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.704+0000 7f8bfab5a640 1 -- 172.21.15.16:0/3180961744 >> 172.21.15.16:0/3180961744 conn(0x7f8bf40fc7a0 msgr2=0x7f8bf40fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:07:59.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.704+0000 7f8bfab5a640 1 -- 172.21.15.16:0/3180961744 shutdown_connections 2024-09-12T02:07:59.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:07:59.704+0000 7f8bfab5a640 1 -- 172.21.15.16:0/3180961744 wait complete. 2024-09-12T02:08:00.380 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:08:00 smithi016 ceph-mon[30947]: pgmap v517: 97 pgs: 97 active+clean; 3.2 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 2.8 KiB/s wr, 7 op/s 2024-09-12T02:08:00.409 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-12T02:08:00.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:08:00 smithi184 ceph-mon[35593]: pgmap v517: 97 pgs: 97 active+clean; 3.2 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 2.8 KiB/s wr, 7 op/s 2024-09-12T02:08:00.666 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:08:00.668 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:08:00 smithi033 ceph-mon[35332]: pgmap v517: 97 pgs: 97 active+clean; 3.2 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 2.8 KiB/s wr, 7 op/s 2024-09-12T02:08:01.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.174+0000 7f37c181c640 1 -- 172.21.15.16:0/746770799 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37bc1036e0 msgr2=0x7f37bc107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:01.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.174+0000 7f37c181c640 1 --2- 172.21.15.16:0/746770799 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37bc1036e0 0x7f37bc107b90 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f37a4009930 tx=0x7f37a402a420 comp rx=0 tx=0).stop 2024-09-12T02:08:01.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.175+0000 7f37c181c640 1 -- 172.21.15.16:0/746770799 shutdown_connections 2024-09-12T02:08:01.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.175+0000 7f37c181c640 1 --2- 172.21.15.16:0/746770799 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f37bc1082a0 0x7f37bc10ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:01.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.175+0000 7f37c181c640 1 --2- 172.21.15.16:0/746770799 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37bc1036e0 0x7f37bc107b90 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:01.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.175+0000 7f37c181c640 1 --2- 172.21.15.16:0/746770799 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f37bc102d30 0x7f37bc103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:01.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.175+0000 7f37c181c640 1 -- 172.21.15.16:0/746770799 >> 172.21.15.16:0/746770799 conn(0x7f37bc0fc7d0 msgr2=0x7f37bc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:01.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.175+0000 7f37c181c640 1 -- 172.21.15.16:0/746770799 shutdown_connections 2024-09-12T02:08:01.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.175+0000 7f37c181c640 1 -- 172.21.15.16:0/746770799 wait complete. 2024-09-12T02:08:01.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.175+0000 7f37c181c640 1 Processor -- start 2024-09-12T02:08:01.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.175+0000 7f37c181c640 1 -- start start 2024-09-12T02:08:01.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.176+0000 7f37c181c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f37bc102d30 0x7f37bc198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:01.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.176+0000 7f37c181c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f37bc1036e0 0x7f37bc198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:01.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.176+0000 7f37c181c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37bc1082a0 0x7f37bc19ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:01.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.176+0000 7f37c181c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f37bc112060 con 0x7f37bc1082a0 2024-09-12T02:08:01.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.176+0000 7f37c181c640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f37bc111ee0 con 0x7f37bc1036e0 2024-09-12T02:08:01.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.176+0000 7f37c181c640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f37bc1121e0 con 0x7f37bc102d30 2024-09-12T02:08:01.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.176+0000 7f37bb7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37bc1082a0 0x7f37bc19ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:01.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.176+0000 7f37bb7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37bc1082a0 0x7f37bc19ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35790/0 (socket says 172.21.15.16:35790) 2024-09-12T02:08:01.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.176+0000 7f37bb7fe640 1 -- 172.21.15.16:0/1517295823 learned_addr learned my addr 172.21.15.16:0/1517295823 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:08:01.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.176+0000 7f37baffd640 1 --2- 172.21.15.16:0/1517295823 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f37bc102d30 0x7f37bc198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:01.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.176+0000 7f37ba7fc640 1 --2- 172.21.15.16:0/1517295823 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f37bc1036e0 0x7f37bc198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:01.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.176+0000 7f37bb7fe640 1 -- 172.21.15.16:0/1517295823 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f37bc102d30 msgr2=0x7f37bc198530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:01.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.176+0000 7f37bb7fe640 1 --2- 172.21.15.16:0/1517295823 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f37bc102d30 0x7f37bc198530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:01.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.176+0000 7f37bb7fe640 1 -- 172.21.15.16:0/1517295823 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f37bc1036e0 msgr2=0x7f37bc198a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:01.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.176+0000 7f37bb7fe640 1 --2- 172.21.15.16:0/1517295823 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f37bc1036e0 0x7f37bc198a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:01.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.176+0000 7f37bb7fe640 1 -- 172.21.15.16:0/1517295823 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37bc19d4e0 con 0x7f37bc1082a0 2024-09-12T02:08:01.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.176+0000 7f37ba7fc640 1 --2- 172.21.15.16:0/1517295823 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f37bc1036e0 0x7f37bc198a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:08:01.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.176+0000 7f37bb7fe640 1 --2- 172.21.15.16:0/1517295823 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37bc1082a0 0x7f37bc19ce00 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f37b4009fd0 tx=0x7f37b400def0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:01.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.176+0000 7f37c081a640 1 -- 172.21.15.16:0/1517295823 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37b4017070 con 0x7f37bc1082a0 2024-09-12T02:08:01.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.176+0000 7f37c181c640 1 -- 172.21.15.16:0/1517295823 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37bc19d7d0 con 0x7f37bc1082a0 2024-09-12T02:08:01.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.177+0000 7f37c081a640 1 -- 172.21.15.16:0/1517295823 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37b4003e50 con 0x7f37bc1082a0 2024-09-12T02:08:01.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.177+0000 7f37c081a640 1 -- 172.21.15.16:0/1517295823 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37b4010460 con 0x7f37bc1082a0 2024-09-12T02:08:01.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.177+0000 7f37c181c640 1 -- 172.21.15.16:0/1517295823 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37bc1a50b0 con 0x7f37bc1082a0 2024-09-12T02:08:01.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.179+0000 7f37c081a640 1 -- 172.21.15.16:0/1517295823 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f37b4003970 con 0x7f37bc1082a0 2024-09-12T02:08:01.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.179+0000 7f37c3bf5740 1 Processor -- start 2024-09-12T02:08:01.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.179+0000 7f37c3bf5740 1 -- start start 2024-09-12T02:08:01.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.179+0000 7f37c3bf5740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a8065bb40 0x559a8065a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:01.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.179+0000 7f37c3bf5740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559a8065a5d0 0x559a8065aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:01.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.179+0000 7f37c3bf5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a80665370 0x559a80667760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:01.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.179+0000 7f37c3bf5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x559a805c02f0 con 0x559a80665370 2024-09-12T02:08:01.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.179+0000 7f37c3bf5740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x559a80567cf0 con 0x559a8065bb40 2024-09-12T02:08:01.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.179+0000 7f37c3bf5740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x559a805bc2b0 con 0x559a8065a5d0 2024-09-12T02:08:01.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.180+0000 7f37c081a640 1 --2- 172.21.15.16:0/1517295823 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f37a80a2c00 0x7f37a80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:01.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.180+0000 7f37ba7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a80665370 0x559a80667760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:01.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.180+0000 7f37ba7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a80665370 0x559a80667760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35800/0 (socket says 172.21.15.16:35800) 2024-09-12T02:08:01.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.180+0000 7f37ba7fc640 1 -- 172.21.15.16:0/1454545139 learned_addr learned my addr 172.21.15.16:0/1454545139 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:08:01.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.180+0000 7f37ba7fc640 1 --2- 172.21.15.16:0/1454545139 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a8065bb40 0x559a8065a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:01.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.180+0000 7f37bb7fe640 1 --2- 172.21.15.16:0/1517295823 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f37a80a2c00 0x7f37a80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:01.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.180+0000 7f37baffd640 1 --2- 172.21.15.16:0/1454545139 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559a8065a5d0 0x559a8065aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:01.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.180+0000 7f37ba7fc640 1 -- 172.21.15.16:0/1454545139 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559a8065a5d0 msgr2=0x559a8065aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:01.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.180+0000 7f37ba7fc640 1 --2- 172.21.15.16:0/1454545139 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559a8065a5d0 0x559a8065aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:01.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.180+0000 7f37c081a640 1 -- 172.21.15.16:0/1517295823 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f37b40cebf0 con 0x7f37bc1082a0 2024-09-12T02:08:01.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.180+0000 7f37ba7fc640 1 -- 172.21.15.16:0/1454545139 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a8065bb40 msgr2=0x559a8065a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:01.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.180+0000 7f37ba7fc640 1 --2- 172.21.15.16:0/1454545139 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a8065bb40 0x559a8065a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:01.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.180+0000 7f37ba7fc640 1 -- 172.21.15.16:0/1454545139 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a8054c3a0 con 0x559a80665370 2024-09-12T02:08:01.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.180+0000 7f37ba7fc640 1 --2- 172.21.15.16:0/1454545139 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a80665370 0x559a80667760 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f37a40057c0 tx=0x7f37a4000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2ebd082c3e0c81f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:01.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.180+0000 7f37bb7fe640 1 --2- 172.21.15.16:0/1517295823 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f37a80a2c00 0x7f37a80a50c0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f37b4097950 tx=0x7f37b4097150 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:01.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.180+0000 7f37b1ffb640 1 -- 172.21.15.16:0/1454545139 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37a4035040 con 0x559a80665370 2024-09-12T02:08:01.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.180+0000 7f37b1ffb640 1 -- 172.21.15.16:0/1454545139 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37a40035f0 con 0x559a80665370 2024-09-12T02:08:01.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.180+0000 7f37b1ffb640 1 -- 172.21.15.16:0/1454545139 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37a403c670 con 0x559a80665370 2024-09-12T02:08:01.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.181+0000 7f37c3bf5740 1 -- 172.21.15.16:0/1454545139 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a80665370 msgr2=0x559a80667760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:01.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.181+0000 7f37c3bf5740 1 --2- 172.21.15.16:0/1454545139 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a80665370 0x559a80667760 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f37a40057c0 tx=0x7f37a4000bc0 comp rx=0 tx=0).stop 2024-09-12T02:08:01.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.181+0000 7f37c3bf5740 1 -- 172.21.15.16:0/1454545139 shutdown_connections 2024-09-12T02:08:01.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.181+0000 7f37c3bf5740 1 --2- 172.21.15.16:0/1454545139 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a80665370 0x559a80667760 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:01.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.181+0000 7f37c3bf5740 1 --2- 172.21.15.16:0/1454545139 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559a8065a5d0 0x559a8065aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:01.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.181+0000 7f37c3bf5740 1 --2- 172.21.15.16:0/1454545139 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a8065bb40 0x559a8065a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:01.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.181+0000 7f37c3bf5740 1 -- 172.21.15.16:0/1454545139 >> 172.21.15.16:0/1454545139 conn(0x559a80623310 msgr2=0x559a80656c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:01.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.181+0000 7f37c3bf5740 1 -- 172.21.15.16:0/1454545139 shutdown_connections 2024-09-12T02:08:01.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.181+0000 7f37c3bf5740 1 -- 172.21.15.16:0/1454545139 wait complete. 2024-09-12T02:08:01.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.181+0000 7f37c3bf5740 1 -- start start 2024-09-12T02:08:01.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.182+0000 7f37c3bf5740 1 Processor -- start 2024-09-12T02:08:01.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.182+0000 7f37c3bf5740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559a8065a5d0 0x559a80664d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:01.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.182+0000 7f37c3bf5740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a8065bb40 0x559a80662020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:01.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.182+0000 7f37c3bf5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a80665370 0x559a80662610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:01.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.182+0000 7f37c3bf5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x559a80668010 con 0x559a80665370 2024-09-12T02:08:01.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.182+0000 7f37c3bf5740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x559a8057b060 con 0x559a8065bb40 2024-09-12T02:08:01.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.182+0000 7f37c3bf5740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x559a80668190 con 0x559a8065a5d0 2024-09-12T02:08:01.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.182+0000 7f37c3bf5740 10 client.0 _ll_register_callbacks cb 0x559a802ce8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:08:01.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.182+0000 7f37ba7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a80665370 0x559a80662610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:01.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.182+0000 7f37ba7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a80665370 0x559a80662610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35806/0 (socket says 172.21.15.16:35806) 2024-09-12T02:08:01.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.182+0000 7f37ba7fc640 1 -- 172.21.15.16:0/3579767244 learned_addr learned my addr 172.21.15.16:0/3579767244 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:08:01.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.182+0000 7f37ba7fc640 1 --2- 172.21.15.16:0/3579767244 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559a8065a5d0 0x559a80664d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:01.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.182+0000 7f37baffd640 1 --2- 172.21.15.16:0/3579767244 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a8065bb40 0x559a80662020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:01.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.183+0000 7f37ba7fc640 1 -- 172.21.15.16:0/3579767244 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559a8065a5d0 msgr2=0x559a80664d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:01.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.183+0000 7f37ba7fc640 1 --2- 172.21.15.16:0/3579767244 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559a8065a5d0 0x559a80664d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:01.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.183+0000 7f37ba7fc640 1 -- 172.21.15.16:0/3579767244 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a8065bb40 msgr2=0x559a80662020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:01.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.183+0000 7f37ba7fc640 1 --2- 172.21.15.16:0/3579767244 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a8065bb40 0x559a80662020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:01.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.183+0000 7f37ba7fc640 1 -- 172.21.15.16:0/3579767244 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a803313c0 con 0x559a80665370 2024-09-12T02:08:01.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.183+0000 7f37baffd640 1 --2- 172.21.15.16:0/3579767244 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a8065bb40 0x559a80662020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:08:01.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.183+0000 7f37ba7fc640 1 --2- 172.21.15.16:0/3579767244 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a80665370 0x559a80662610 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f37a40319d0 tx=0x7f37a4003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a8ae59f9fa851aa server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:01.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.183+0000 7f379affd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:08:01.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.183+0000 7f379affd640 1 -- 172.21.15.16:0/3579767244 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37a4035030 con 0x559a80665370 2024-09-12T02:08:01.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.183+0000 7f37c3bf5740 1 -- 172.21.15.16:0/3579767244 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f37a40035f0 con 0x559a80665370 2024-09-12T02:08:01.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.183+0000 7f379affd640 1 -- 172.21.15.16:0/3579767244 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f37a40037a0 con 0x559a80665370 2024-09-12T02:08:01.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.183+0000 7f379affd640 1 -- 172.21.15.16:0/3579767244 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f37a40039a0 con 0x559a80665370 2024-09-12T02:08:01.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.183+0000 7f37c3bf5740 10 client.17265 fetch_fsmap learned FSMap version 25 2024-09-12T02:08:01.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.183+0000 7f37c3bf5740 1 -- 172.21.15.16:0/3579767244 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x559a80677ec0 con 0x559a80665370 2024-09-12T02:08:01.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.183+0000 7f379affd640 1 -- 172.21.15.16:0/3579767244 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37a40035f0 con 0x559a80665370 2024-09-12T02:08:01.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.183+0000 7f379affd640 1 -- 172.21.15.16:0/3579767244 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f37a4056020 con 0x559a80665370 2024-09-12T02:08:01.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.184+0000 7f37c3bf5740 10 client.17265 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:08:01.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.184+0000 7f37c3bf5740 10 client.17265 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:08:01.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.184+0000 7f37c3bf5740 20 client.17265 populate_metadata read hostname 'smithi016' 2024-09-12T02:08:01.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.184+0000 7f37c3bf5740 1 --2- 172.21.15.16:0/3579767244 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x559a80679210 0x559a8067b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:01.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.184+0000 7f37c3bf5740 4 client.17265 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:08:01.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.184+0000 7f37c3bf5740 1 -- 172.21.15.16:0/3579767244 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559a805cd860 con 0x559a80679210 2024-09-12T02:08:01.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.184+0000 7f37baffd640 1 --2- 172.21.15.16:0/3579767244 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x559a80679210 0x559a8067b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:01.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.184+0000 7f379affd640 10 client.17265 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:08:01.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.184+0000 7f37baffd640 1 --2- 172.21.15.16:0/3579767244 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x559a80679210 0x559a8067b600 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea08590a6bfddebb server_cookie=8bd7fa5ff19229a in_seq=0 out_seq=0 2024-09-12T02:08:01.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.185+0000 7f379affd640 1 -- 172.21.15.16:0/3579767244 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559a805cd860 con 0x559a80679210 2024-09-12T02:08:01.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.185+0000 7f379affd640 10 client.17265 handle_command_reply: tid=0 2024-09-12T02:08:01.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.185+0000 7f37c3bf5740 1 client.17265 shutdown 2024-09-12T02:08:01.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.185+0000 7f37c3bf5740 1 -- 172.21.15.16:0/3579767244 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a80665370 msgr2=0x559a80662610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:01.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.185+0000 7f37c3bf5740 1 --2- 172.21.15.16:0/3579767244 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a80665370 0x559a80662610 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f37a40319d0 tx=0x7f37a4003f10 comp rx=0 tx=0).stop 2024-09-12T02:08:01.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.185+0000 7f37c3bf5740 1 -- 172.21.15.16:0/3579767244 shutdown_connections 2024-09-12T02:08:01.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.185+0000 7f37c3bf5740 1 --2- 172.21.15.16:0/3579767244 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x559a80679210 0x559a8067b600 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:01.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.185+0000 7f37c3bf5740 1 --2- 172.21.15.16:0/3579767244 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a80665370 0x559a80662610 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:01.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.185+0000 7f37c3bf5740 1 --2- 172.21.15.16:0/3579767244 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a8065bb40 0x559a80662020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:01.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.185+0000 7f37c3bf5740 1 --2- 172.21.15.16:0/3579767244 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559a8065a5d0 0x559a80664d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:01.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.185+0000 7f379affd640 0 client.17265 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:08:01.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.185+0000 7f37c3bf5740 1 -- 172.21.15.16:0/3579767244 >> 172.21.15.16:0/3579767244 conn(0x559a80623310 msgr2=0x559a8065c960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:01.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.185+0000 7f37c3bf5740 1 -- 172.21.15.16:0/3579767244 shutdown_connections 2024-09-12T02:08:01.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.185+0000 7f37c3bf5740 1 -- 172.21.15.16:0/3579767244 wait complete. 2024-09-12T02:08:01.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.185+0000 7f37c3bf5740 20 client.17265 trim_cache size 0 max 16384 2024-09-12T02:08:01.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.198+0000 7f37c3bf5740 1 Processor -- start 2024-09-12T02:08:01.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.198+0000 7f37c3bf5740 1 -- start start 2024-09-12T02:08:01.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.198+0000 7f37c3bf5740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a8065a5d0 0x559a8065d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:01.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.198+0000 7f37c3bf5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a8065bb40 0x559a8065d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:01.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.198+0000 7f37c3bf5740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559a80665370 0x559a8067e620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:01.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.199+0000 7f37c3bf5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f37bc1ada70 con 0x559a8065bb40 2024-09-12T02:08:01.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.199+0000 7f37c3bf5740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f37bc1ab340 con 0x559a8065a5d0 2024-09-12T02:08:01.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.199+0000 7f37c3bf5740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f37bc1a91d0 con 0x559a80665370 2024-09-12T02:08:01.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.199+0000 7f37baffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a8065bb40 0x559a8065d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:01.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.199+0000 7f37baffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a8065bb40 0x559a8065d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55312/0 (socket says 172.21.15.16:55312) 2024-09-12T02:08:01.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.199+0000 7f37baffd640 1 -- 172.21.15.16:0/445327163 learned_addr learned my addr 172.21.15.16:0/445327163 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:08:01.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.199+0000 7f37ba7fc640 1 --2- 172.21.15.16:0/445327163 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a8065a5d0 0x559a8065d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:01.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.199+0000 7f37baffd640 1 -- 172.21.15.16:0/445327163 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559a80665370 msgr2=0x559a8067e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:01.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.199+0000 7f37baffd640 1 --2- 172.21.15.16:0/445327163 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559a80665370 0x559a8067e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:01.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.199+0000 7f37baffd640 1 -- 172.21.15.16:0/445327163 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a8065a5d0 msgr2=0x559a8065d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:01.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.199+0000 7f37baffd640 1 --2- 172.21.15.16:0/445327163 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a8065a5d0 0x559a8065d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:01.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.199+0000 7f37baffd640 1 -- 172.21.15.16:0/445327163 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a806650e0 con 0x559a8065bb40 2024-09-12T02:08:01.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.199+0000 7f37baffd640 1 --2- 172.21.15.16:0/445327163 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a8065bb40 0x559a8065d660 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f37ac00adf0 tx=0x7f37ac0108c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bfb9060119ae4d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:01.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.199+0000 7f37b17fa640 1 -- 172.21.15.16:0/445327163 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37ac005220 con 0x559a8065bb40 2024-09-12T02:08:01.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.199+0000 7f37b17fa640 1 -- 172.21.15.16:0/445327163 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37ac004c80 con 0x559a8065bb40 2024-09-12T02:08:01.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.199+0000 7f37b17fa640 1 -- 172.21.15.16:0/445327163 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37ac01ea60 con 0x559a8065bb40 2024-09-12T02:08:01.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.200+0000 7f37c3bf5740 1 -- 172.21.15.16:0/445327163 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a8065bb40 msgr2=0x559a8065d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:01.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.200+0000 7f37c3bf5740 1 --2- 172.21.15.16:0/445327163 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a8065bb40 0x559a8065d660 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f37ac00adf0 tx=0x7f37ac0108c0 comp rx=0 tx=0).stop 2024-09-12T02:08:01.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.200+0000 7f37c3bf5740 1 -- 172.21.15.16:0/445327163 shutdown_connections 2024-09-12T02:08:01.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.200+0000 7f37c3bf5740 1 --2- 172.21.15.16:0/445327163 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559a80665370 0x559a8067e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:01.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.200+0000 7f37c3bf5740 1 --2- 172.21.15.16:0/445327163 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a8065bb40 0x559a8065d660 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:01.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.200+0000 7f37c3bf5740 1 --2- 172.21.15.16:0/445327163 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a8065a5d0 0x559a8065d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:01.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.200+0000 7f37c3bf5740 1 -- 172.21.15.16:0/445327163 >> 172.21.15.16:0/445327163 conn(0x559a80623310 msgr2=0x559a806640a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:01.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.200+0000 7f37c3bf5740 1 -- 172.21.15.16:0/445327163 shutdown_connections 2024-09-12T02:08:01.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.200+0000 7f37c3bf5740 1 -- 172.21.15.16:0/445327163 wait complete. 2024-09-12T02:08:01.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.200+0000 7f37c3bf5740 1 -- start start 2024-09-12T02:08:01.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.201+0000 7f37c3bf5740 1 Processor -- start 2024-09-12T02:08:01.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.201+0000 7f37c3bf5740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a8065a5d0 0x559a8067ec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:01.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.201+0000 7f37c3bf5740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559a8065bb40 0x559a8067bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:01.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.201+0000 7f37c3bf5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a80665370 0x559a8067c500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:01.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.201+0000 7f37c3bf5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f37bc1a9820 con 0x559a80665370 2024-09-12T02:08:01.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.201+0000 7f37c3bf5740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f37bc1aac70 con 0x559a8065a5d0 2024-09-12T02:08:01.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.201+0000 7f37c3bf5740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f37bc1a9ed0 con 0x559a8065bb40 2024-09-12T02:08:01.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.201+0000 7f37c3bf5740 10 client.0 _ll_register_callbacks cb 0x7f37a4032cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:08:01.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.201+0000 7f37ba7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a80665370 0x559a8067c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:01.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.201+0000 7f37ba7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a80665370 0x559a8067c500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55318/0 (socket says 172.21.15.16:55318) 2024-09-12T02:08:01.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.201+0000 7f37ba7fc640 1 -- 172.21.15.16:0/104016401 learned_addr learned my addr 172.21.15.16:0/104016401 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:08:01.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.201+0000 7f37ba7fc640 1 --2- 172.21.15.16:0/104016401 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a8065a5d0 0x559a8067ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:01.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.201+0000 7f37baffd640 1 --2- 172.21.15.16:0/104016401 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559a8065bb40 0x559a8067bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:01.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.201+0000 7f37ba7fc640 1 -- 172.21.15.16:0/104016401 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559a8065bb40 msgr2=0x559a8067bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:01.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.201+0000 7f37ba7fc640 1 --2- 172.21.15.16:0/104016401 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559a8065bb40 0x559a8067bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:01.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.201+0000 7f37ba7fc640 1 -- 172.21.15.16:0/104016401 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a8065a5d0 msgr2=0x559a8067ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:01.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.201+0000 7f37ba7fc640 1 --2- 172.21.15.16:0/104016401 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a8065a5d0 0x559a8067ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:01.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.201+0000 7f37ba7fc640 1 -- 172.21.15.16:0/104016401 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a8067cbf0 con 0x559a80665370 2024-09-12T02:08:01.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.202+0000 7f37ba7fc640 1 --2- 172.21.15.16:0/104016401 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a80665370 0x559a8067c500 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f37a40057c0 tx=0x7f37a402a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50b7a93150ac9634 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:01.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.202+0000 7f379affd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:08:01.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.202+0000 7f379affd640 1 -- 172.21.15.16:0/104016401 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37a4035040 con 0x559a80665370 2024-09-12T02:08:01.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.202+0000 7f379affd640 1 -- 172.21.15.16:0/104016401 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f37a4058d40 con 0x559a80665370 2024-09-12T02:08:01.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.202+0000 7f37c3bf5740 1 -- 172.21.15.16:0/104016401 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f37ac004c80 con 0x559a80665370 2024-09-12T02:08:01.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.202+0000 7f379affd640 1 -- 172.21.15.16:0/104016401 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f37a406d050 con 0x559a80665370 2024-09-12T02:08:01.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.202+0000 7f37c3bf5740 10 client.17271 fetch_fsmap learned FSMap version 25 2024-09-12T02:08:01.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.202+0000 7f37c3bf5740 1 -- 172.21.15.16:0/104016401 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x559a806e3a30 con 0x559a80665370 2024-09-12T02:08:01.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.202+0000 7f379affd640 1 -- 172.21.15.16:0/104016401 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37ac004c80 con 0x559a80665370 2024-09-12T02:08:01.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.202+0000 7f379affd640 1 -- 172.21.15.16:0/104016401 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f37a4056040 con 0x559a80665370 2024-09-12T02:08:01.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.202+0000 7f37c3bf5740 10 client.17271 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:08:01.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.202+0000 7f37c3bf5740 10 client.17271 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:08:01.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.202+0000 7f37c3bf5740 20 client.17271 populate_metadata read hostname 'smithi016' 2024-09-12T02:08:01.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.202+0000 7f37c3bf5740 1 --2- 172.21.15.16:0/104016401 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x559a806e4040 0x559a806e6430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:01.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.202+0000 7f37c3bf5740 4 client.17271 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T02:08:01.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.202+0000 7f37c3bf5740 1 -- 172.21.15.16:0/104016401 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559a805c7d70 con 0x559a806e4040 2024-09-12T02:08:01.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.203+0000 7f37baffd640 1 --2- 172.21.15.16:0/104016401 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x559a806e4040 0x559a806e6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:01.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.203+0000 7f379affd640 10 client.17271 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:08:01.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.203+0000 7f37baffd640 1 --2- 172.21.15.16:0/104016401 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x559a806e4040 0x559a806e6430 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a669bf8237aa2a9 server_cookie=31053c97d9860185 in_seq=0 out_seq=0 2024-09-12T02:08:01.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.203+0000 7f379affd640 1 -- 172.21.15.16:0/104016401 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559a805c7d70 con 0x559a806e4040 2024-09-12T02:08:01.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.203+0000 7f379affd640 10 client.17271 handle_command_reply: tid=0 2024-09-12T02:08:01.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.203+0000 7f37c3bf5740 1 client.17271 shutdown 2024-09-12T02:08:01.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.203+0000 7f37c3bf5740 1 -- 172.21.15.16:0/104016401 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a80665370 msgr2=0x559a8067c500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:01.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.204+0000 7f37c3bf5740 1 --2- 172.21.15.16:0/104016401 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a80665370 0x559a8067c500 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f37a40057c0 tx=0x7f37a402a420 comp rx=0 tx=0).stop 2024-09-12T02:08:01.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.204+0000 7f37c3bf5740 1 -- 172.21.15.16:0/104016401 shutdown_connections 2024-09-12T02:08:01.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.204+0000 7f37c3bf5740 1 --2- 172.21.15.16:0/104016401 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x559a806e4040 0x559a806e6430 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:01.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.204+0000 7f37c3bf5740 1 --2- 172.21.15.16:0/104016401 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a80665370 0x559a8067c500 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:01.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.204+0000 7f37c3bf5740 1 --2- 172.21.15.16:0/104016401 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559a8065bb40 0x559a8067bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:01.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.204+0000 7f37c3bf5740 1 --2- 172.21.15.16:0/104016401 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a8065a5d0 0x559a8067ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:01.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.204+0000 7f37c3bf5740 1 -- 172.21.15.16:0/104016401 >> 172.21.15.16:0/104016401 conn(0x559a80623310 msgr2=0x559a80670420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:01.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.204+0000 7f379affd640 0 client.17271 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:08:01.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.204+0000 7f37c3bf5740 1 -- 172.21.15.16:0/104016401 shutdown_connections 2024-09-12T02:08:01.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.204+0000 7f37c3bf5740 1 -- 172.21.15.16:0/104016401 wait complete. 2024-09-12T02:08:01.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.204+0000 7f37c3bf5740 20 client.17271 trim_cache size 0 max 16384 2024-09-12T02:08:01.220 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T02:08:01.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.204+0000 7f37c181c640 1 -- 172.21.15.16:0/1517295823 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f37a80a2c00 msgr2=0x7f37a80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:01.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.204+0000 7f37c181c640 1 --2- 172.21.15.16:0/1517295823 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f37a80a2c00 0x7f37a80a50c0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f37b4097950 tx=0x7f37b4097150 comp rx=0 tx=0).stop 2024-09-12T02:08:01.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.204+0000 7f37c181c640 1 -- 172.21.15.16:0/1517295823 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37bc1082a0 msgr2=0x7f37bc19ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:01.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.204+0000 7f37c181c640 1 --2- 172.21.15.16:0/1517295823 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37bc1082a0 0x7f37bc19ce00 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f37b4009fd0 tx=0x7f37b400def0 comp rx=0 tx=0).stop 2024-09-12T02:08:01.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.204+0000 7f37c181c640 1 -- 172.21.15.16:0/1517295823 shutdown_connections 2024-09-12T02:08:01.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.204+0000 7f37c181c640 1 --2- 172.21.15.16:0/1517295823 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f37a80a2c00 0x7f37a80a50c0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:01.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.204+0000 7f37c181c640 1 --2- 172.21.15.16:0/1517295823 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37bc1082a0 0x7f37bc19ce00 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:01.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.204+0000 7f37c181c640 1 --2- 172.21.15.16:0/1517295823 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f37bc1036e0 0x7f37bc198a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:01.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.204+0000 7f37c181c640 1 --2- 172.21.15.16:0/1517295823 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f37bc102d30 0x7f37bc198530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:01.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.204+0000 7f37c181c640 1 -- 172.21.15.16:0/1517295823 >> 172.21.15.16:0/1517295823 conn(0x7f37bc0fc7d0 msgr2=0x7f37bc105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:01.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.205+0000 7f37c181c640 1 -- 172.21.15.16:0/1517295823 shutdown_connections 2024-09-12T02:08:01.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:01.205+0000 7f37c181c640 1 -- 172.21.15.16:0/1517295823 wait complete. 2024-09-12T02:08:02.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:08:02 smithi184 ceph-mon[35593]: pgmap v518: 97 pgs: 97 active+clean; 3.2 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 8.5 KiB/s wr, 11 op/s 2024-09-12T02:08:02.634 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:08:02 smithi016 ceph-mon[30947]: pgmap v518: 97 pgs: 97 active+clean; 3.2 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 8.5 KiB/s wr, 11 op/s 2024-09-12T02:08:02.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:08:02 smithi033 ceph-mon[35332]: pgmap v518: 97 pgs: 97 active+clean; 3.2 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 8.5 KiB/s wr, 11 op/s 2024-09-12T02:08:02.984 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-12T02:08:02.984 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-12T02:08:03.261 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:08:03.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.703+0000 7f1b65f23640 1 -- 172.21.15.16:0/408010179 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b601036c0 msgr2=0x7f1b6010bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:03.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.703+0000 7f1b65f23640 1 --2- 172.21.15.16:0/408010179 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b601036c0 0x7f1b6010bbb0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f1b48009900 tx=0x7f1b4802a480 comp rx=0 tx=0).stop 2024-09-12T02:08:03.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.703+0000 7f1b65f23640 1 -- 172.21.15.16:0/408010179 shutdown_connections 2024-09-12T02:08:03.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.703+0000 7f1b65f23640 1 --2- 172.21.15.16:0/408010179 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b6010c180 0x7f1b6010e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:03.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.703+0000 7f1b65f23640 1 --2- 172.21.15.16:0/408010179 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b601036c0 0x7f1b6010bbb0 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:03.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.703+0000 7f1b65f23640 1 --2- 172.21.15.16:0/408010179 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1b60102da0 0x7f1b60103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:03.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.703+0000 7f1b65f23640 1 -- 172.21.15.16:0/408010179 >> 172.21.15.16:0/408010179 conn(0x7f1b600fc7a0 msgr2=0x7f1b600febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:03.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.703+0000 7f1b65f23640 1 -- 172.21.15.16:0/408010179 shutdown_connections 2024-09-12T02:08:03.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.703+0000 7f1b65f23640 1 -- 172.21.15.16:0/408010179 wait complete. 2024-09-12T02:08:03.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.704+0000 7f1b65f23640 1 Processor -- start 2024-09-12T02:08:03.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.704+0000 7f1b65f23640 1 -- start start 2024-09-12T02:08:03.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.704+0000 7f1b65f23640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1b60102da0 0x7f1b601984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:03.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.704+0000 7f1b65f23640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b601036c0 0x7f1b601989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:03.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.704+0000 7f1b65f23640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b6010c180 0x7f1b6019cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:03.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.704+0000 7f1b65f23640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1b60111fa0 con 0x7f1b6010c180 2024-09-12T02:08:03.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.704+0000 7f1b65f23640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1b60111e20 con 0x7f1b601036c0 2024-09-12T02:08:03.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.704+0000 7f1b65f23640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1b60112120 con 0x7f1b60102da0 2024-09-12T02:08:03.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.704+0000 7f1b57fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b601036c0 0x7f1b601989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:03.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.704+0000 7f1b57fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b601036c0 0x7f1b601989f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.16:39328/0 (socket says 172.21.15.16:39328) 2024-09-12T02:08:03.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.704+0000 7f1b57fff640 1 -- 172.21.15.16:0/2686367907 learned_addr learned my addr 172.21.15.16:0/2686367907 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:08:03.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.704+0000 7f1b5ffff640 1 --2- 172.21.15.16:0/2686367907 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b6010c180 0x7f1b6019cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:03.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.704+0000 7f1b5f7fe640 1 --2- 172.21.15.16:0/2686367907 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1b60102da0 0x7f1b601984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:03.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.705+0000 7f1b5ffff640 1 -- 172.21.15.16:0/2686367907 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1b60102da0 msgr2=0x7f1b601984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:03.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.705+0000 7f1b5ffff640 1 --2- 172.21.15.16:0/2686367907 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1b60102da0 0x7f1b601984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:03.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.705+0000 7f1b5ffff640 1 -- 172.21.15.16:0/2686367907 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b601036c0 msgr2=0x7f1b601989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:03.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.705+0000 7f1b5ffff640 1 --2- 172.21.15.16:0/2686367907 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b601036c0 0x7f1b601989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:03.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.705+0000 7f1b5ffff640 1 -- 172.21.15.16:0/2686367907 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b6019d460 con 0x7f1b6010c180 2024-09-12T02:08:03.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.705+0000 7f1b57fff640 1 --2- 172.21.15.16:0/2686367907 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b601036c0 0x7f1b601989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:08:03.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.705+0000 7f1b5ffff640 1 --2- 172.21.15.16:0/2686367907 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b6010c180 0x7f1b6019cd80 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f1b5800e940 tx=0x7f1b5800ed40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:03.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.705+0000 7f1b5d7fa640 1 -- 172.21.15.16:0/2686367907 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b58004d20 con 0x7f1b6010c180 2024-09-12T02:08:03.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.705+0000 7f1b5d7fa640 1 -- 172.21.15.16:0/2686367907 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b58005020 con 0x7f1b6010c180 2024-09-12T02:08:03.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.705+0000 7f1b65f23640 1 -- 172.21.15.16:0/2686367907 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b6019d750 con 0x7f1b6010c180 2024-09-12T02:08:03.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.705+0000 7f1b5d7fa640 1 -- 172.21.15.16:0/2686367907 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b580116b0 con 0x7f1b6010c180 2024-09-12T02:08:03.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.705+0000 7f1b65f23640 1 -- 172.21.15.16:0/2686367907 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b601a5030 con 0x7f1b6010c180 2024-09-12T02:08:03.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.707+0000 7f1b5d7fa640 1 -- 172.21.15.16:0/2686367907 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f1b580054d0 con 0x7f1b6010c180 2024-09-12T02:08:03.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.708+0000 7f1b682fc740 1 Processor -- start 2024-09-12T02:08:03.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.708+0000 7f1b682fc740 1 -- start start 2024-09-12T02:08:03.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.708+0000 7f1b682fc740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f54336b60 0x560f54336f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:03.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.708+0000 7f1b682fc740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560f54337480 0x560f54341800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:03.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.708+0000 7f1b682fc740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560f54341d40 0x560f54344130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:03.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.708+0000 7f1b682fc740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x560f5429c280 con 0x560f54336b60 2024-09-12T02:08:03.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.708+0000 7f1b682fc740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x560f54243b10 con 0x560f54337480 2024-09-12T02:08:03.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.708+0000 7f1b682fc740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x560f54298240 con 0x560f54341d40 2024-09-12T02:08:03.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.709+0000 7f1b57fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f54336b60 0x560f54336f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:03.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.709+0000 7f1b57fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f54336b60 0x560f54336f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55358/0 (socket says 172.21.15.16:55358) 2024-09-12T02:08:03.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.709+0000 7f1b57fff640 1 -- 172.21.15.16:0/1964371978 learned_addr learned my addr 172.21.15.16:0/1964371978 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:08:03.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.709+0000 7f1b5d7fa640 1 --2- 172.21.15.16:0/2686367907 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f1b4c0a2b80 0x7f1b4c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:03.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.709+0000 7f1b5f7fe640 1 --2- 172.21.15.16:0/1964371978 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560f54337480 0x560f54341800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:03.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.709+0000 7f1b57fff640 1 --2- 172.21.15.16:0/1964371978 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560f54341d40 0x560f54344130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:03.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.709+0000 7f1b5ffff640 1 --2- 172.21.15.16:0/2686367907 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f1b4c0a2b80 0x7f1b4c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:03.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.709+0000 7f1b57fff640 1 -- 172.21.15.16:0/1964371978 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560f54341d40 msgr2=0x560f54344130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:03.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.709+0000 7f1b57fff640 1 --2- 172.21.15.16:0/1964371978 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560f54341d40 0x560f54344130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:03.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.709+0000 7f1b57fff640 1 -- 172.21.15.16:0/1964371978 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560f54337480 msgr2=0x560f54341800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:03.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.709+0000 7f1b5d7fa640 1 -- 172.21.15.16:0/2686367907 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f1b580cf0e0 con 0x7f1b6010c180 2024-09-12T02:08:03.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.709+0000 7f1b57fff640 1 --2- 172.21.15.16:0/1964371978 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560f54337480 0x560f54341800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:03.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.709+0000 7f1b57fff640 1 -- 172.21.15.16:0/1964371978 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f542501f0 con 0x560f54336b60 2024-09-12T02:08:03.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.709+0000 7f1b5f7fe640 1 --2- 172.21.15.16:0/1964371978 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560f54337480 0x560f54341800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:08:03.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.709+0000 7f1b57fff640 1 --2- 172.21.15.16:0/1964371978 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f54336b60 0x560f54336f40 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f1b480096c0 tx=0x7f1b48009650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d55a53f5e5169272 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:03.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.709+0000 7f1b55ffb640 1 -- 172.21.15.16:0/1964371978 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b48044020 con 0x560f54336b60 2024-09-12T02:08:03.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.709+0000 7f1b55ffb640 1 -- 172.21.15.16:0/1964371978 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b48005520 con 0x560f54336b60 2024-09-12T02:08:03.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.709+0000 7f1b55ffb640 1 -- 172.21.15.16:0/1964371978 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b48035040 con 0x560f54336b60 2024-09-12T02:08:03.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.710+0000 7f1b682fc740 1 -- 172.21.15.16:0/1964371978 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f54336b60 msgr2=0x560f54336f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:03.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.710+0000 7f1b5ffff640 1 --2- 172.21.15.16:0/2686367907 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f1b4c0a2b80 0x7f1b4c0a5040 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f1b5800c640 tx=0x7f1b580665c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:03.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.710+0000 7f1b682fc740 1 --2- 172.21.15.16:0/1964371978 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f54336b60 0x560f54336f40 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f1b480096c0 tx=0x7f1b48009650 comp rx=0 tx=0).stop 2024-09-12T02:08:03.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.710+0000 7f1b682fc740 1 -- 172.21.15.16:0/1964371978 shutdown_connections 2024-09-12T02:08:03.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.710+0000 7f1b682fc740 1 --2- 172.21.15.16:0/1964371978 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560f54341d40 0x560f54344130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:03.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.710+0000 7f1b682fc740 1 --2- 172.21.15.16:0/1964371978 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560f54337480 0x560f54341800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:03.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.710+0000 7f1b682fc740 1 --2- 172.21.15.16:0/1964371978 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f54336b60 0x560f54336f40 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:03.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.710+0000 7f1b682fc740 1 -- 172.21.15.16:0/1964371978 >> 172.21.15.16:0/1964371978 conn(0x560f543330f0 msgr2=0x560f543334d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:03.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.710+0000 7f1b682fc740 1 -- 172.21.15.16:0/1964371978 shutdown_connections 2024-09-12T02:08:03.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.710+0000 7f1b682fc740 1 -- 172.21.15.16:0/1964371978 wait complete. 2024-09-12T02:08:03.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.711+0000 7f1b682fc740 1 -- start start 2024-09-12T02:08:03.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.711+0000 7f1b682fc740 1 Processor -- start 2024-09-12T02:08:03.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.711+0000 7f1b682fc740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560f54336b60 0x560f5433ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:03.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.711+0000 7f1b682fc740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560f54337480 0x560f5433f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:03.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.711+0000 7f1b682fc740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f54341d40 0x560f5433f640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:03.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.711+0000 7f1b682fc740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x560f54344910 con 0x560f54341d40 2024-09-12T02:08:03.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.711+0000 7f1b682fc740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x560f54256f10 con 0x560f54336b60 2024-09-12T02:08:03.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.711+0000 7f1b682fc740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x560f54344a90 con 0x560f54337480 2024-09-12T02:08:03.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.711+0000 7f1b682fc740 10 client.0 _ll_register_callbacks cb 0x560f5421fa40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:08:03.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.711+0000 7f1b57fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f54341d40 0x560f5433f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:03.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.711+0000 7f1b57fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f54341d40 0x560f5433f640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55360/0 (socket says 172.21.15.16:55360) 2024-09-12T02:08:03.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.711+0000 7f1b57fff640 1 -- 172.21.15.16:0/3799135252 learned_addr learned my addr 172.21.15.16:0/3799135252 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:08:03.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.711+0000 7f1b57fff640 1 --2- 172.21.15.16:0/3799135252 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560f54336b60 0x560f5433ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:03.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.711+0000 7f1b5f7fe640 1 --2- 172.21.15.16:0/3799135252 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560f54337480 0x560f5433f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:03.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.712+0000 7f1b57fff640 1 -- 172.21.15.16:0/3799135252 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560f54337480 msgr2=0x560f5433f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:03.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.712+0000 7f1b57fff640 1 --2- 172.21.15.16:0/3799135252 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560f54337480 0x560f5433f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:03.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.712+0000 7f1b57fff640 1 -- 172.21.15.16:0/3799135252 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560f54336b60 msgr2=0x560f5433ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:03.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.712+0000 7f1b57fff640 1 --2- 172.21.15.16:0/3799135252 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560f54336b60 0x560f5433ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:03.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.712+0000 7f1b57fff640 1 -- 172.21.15.16:0/3799135252 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f5400d6a0 con 0x560f54341d40 2024-09-12T02:08:03.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.712+0000 7f1b57fff640 1 --2- 172.21.15.16:0/3799135252 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f54341d40 0x560f5433f640 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f1b48004b10 tx=0x7f1b48004a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=863e185c946cb57b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:03.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.712+0000 7f1b3effd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:08:03.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.712+0000 7f1b3effd640 1 -- 172.21.15.16:0/3799135252 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b4804d020 con 0x560f54341d40 2024-09-12T02:08:03.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.712+0000 7f1b3effd640 1 -- 172.21.15.16:0/3799135252 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f1b48005250 con 0x560f54341d40 2024-09-12T02:08:03.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.712+0000 7f1b682fc740 1 -- 172.21.15.16:0/3799135252 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b48005520 con 0x560f54341d40 2024-09-12T02:08:03.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.712+0000 7f1b3effd640 1 -- 172.21.15.16:0/3799135252 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1b48035070 con 0x560f54341d40 2024-09-12T02:08:03.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.712+0000 7f1b682fc740 10 client.17283 fetch_fsmap learned FSMap version 25 2024-09-12T02:08:03.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.712+0000 7f1b682fc740 1 -- 172.21.15.16:0/3799135252 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x560f543504c0 con 0x560f54341d40 2024-09-12T02:08:03.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.712+0000 7f1b3effd640 1 -- 172.21.15.16:0/3799135252 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b48005520 con 0x560f54341d40 2024-09-12T02:08:03.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.713+0000 7f1b3effd640 1 -- 172.21.15.16:0/3799135252 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f1b48057020 con 0x560f54341d40 2024-09-12T02:08:03.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.713+0000 7f1b682fc740 10 client.17283 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:08:03.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.713+0000 7f1b682fc740 10 client.17283 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:08:03.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.713+0000 7f1b682fc740 20 client.17283 populate_metadata read hostname 'smithi016' 2024-09-12T02:08:03.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.713+0000 7f1b682fc740 1 --2- 172.21.15.16:0/3799135252 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x560f54351810 0x560f54353c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:03.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.713+0000 7f1b682fc740 4 client.17283 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:08:03.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.713+0000 7f1b682fc740 1 -- 172.21.15.16:0/3799135252 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560f542a97f0 con 0x560f54351810 2024-09-12T02:08:03.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.713+0000 7f1b5f7fe640 1 --2- 172.21.15.16:0/3799135252 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x560f54351810 0x560f54353c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:03.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.713+0000 7f1b3effd640 10 client.17283 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:08:03.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.713+0000 7f1b5f7fe640 1 --2- 172.21.15.16:0/3799135252 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x560f54351810 0x560f54353c00 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e78317eb0125ea8a server_cookie=68906ecfa275e9fb in_seq=0 out_seq=0 2024-09-12T02:08:03.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.714+0000 7f1b3effd640 1 -- 172.21.15.16:0/3799135252 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560f542a97f0 con 0x560f54351810 2024-09-12T02:08:03.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.714+0000 7f1b3effd640 10 client.17283 handle_command_reply: tid=0 2024-09-12T02:08:03.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.714+0000 7f1b682fc740 1 client.17283 shutdown 2024-09-12T02:08:03.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.714+0000 7f1b682fc740 1 -- 172.21.15.16:0/3799135252 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f54341d40 msgr2=0x560f5433f640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:03.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.714+0000 7f1b682fc740 1 --2- 172.21.15.16:0/3799135252 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f54341d40 0x560f5433f640 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f1b48004b10 tx=0x7f1b48004a10 comp rx=0 tx=0).stop 2024-09-12T02:08:03.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.714+0000 7f1b682fc740 1 -- 172.21.15.16:0/3799135252 shutdown_connections 2024-09-12T02:08:03.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.714+0000 7f1b682fc740 1 --2- 172.21.15.16:0/3799135252 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x560f54351810 0x560f54353c00 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:03.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.714+0000 7f1b682fc740 1 --2- 172.21.15.16:0/3799135252 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f54341d40 0x560f5433f640 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:03.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.714+0000 7f1b682fc740 1 --2- 172.21.15.16:0/3799135252 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560f54337480 0x560f5433f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:03.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.714+0000 7f1b682fc740 1 --2- 172.21.15.16:0/3799135252 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560f54336b60 0x560f5433ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:03.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.714+0000 7f1b682fc740 1 -- 172.21.15.16:0/3799135252 >> 172.21.15.16:0/3799135252 conn(0x560f543330f0 msgr2=0x560f54332270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:03.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.714+0000 7f1b3effd640 0 client.17283 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:08:03.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.715+0000 7f1b682fc740 1 -- 172.21.15.16:0/3799135252 shutdown_connections 2024-09-12T02:08:03.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.715+0000 7f1b682fc740 1 -- 172.21.15.16:0/3799135252 wait complete. 2024-09-12T02:08:03.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.715+0000 7f1b682fc740 20 client.17283 trim_cache size 0 max 16384 2024-09-12T02:08:03.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.732+0000 7f1b682fc740 1 Processor -- start 2024-09-12T02:08:03.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.732+0000 7f1b682fc740 1 -- start start 2024-09-12T02:08:03.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.732+0000 7f1b682fc740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560f54336b60 0x560f5434a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:03.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.732+0000 7f1b682fc740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560f54337480 0x560f5434a600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:03.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.732+0000 7f1b682fc740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f54341d40 0x560f543385b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:03.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.732+0000 7f1b682fc740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1b601afc40 con 0x560f54341d40 2024-09-12T02:08:03.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.732+0000 7f1b682fc740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1b601ad510 con 0x560f54336b60 2024-09-12T02:08:03.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.732+0000 7f1b682fc740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1b601ab360 con 0x560f54337480 2024-09-12T02:08:03.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.732+0000 7f1b57fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f54341d40 0x560f543385b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:03.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.732+0000 7f1b57fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f54341d40 0x560f543385b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55366/0 (socket says 172.21.15.16:55366) 2024-09-12T02:08:03.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.732+0000 7f1b57fff640 1 -- 172.21.15.16:0/4044039817 learned_addr learned my addr 172.21.15.16:0/4044039817 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:08:03.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.733+0000 7f1b5f7fe640 1 --2- 172.21.15.16:0/4044039817 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560f54337480 0x560f5434a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:03.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.733+0000 7f1b57fff640 1 --2- 172.21.15.16:0/4044039817 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560f54336b60 0x560f5434a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:03.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.733+0000 7f1b57fff640 1 -- 172.21.15.16:0/4044039817 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560f54337480 msgr2=0x560f5434a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:03.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.733+0000 7f1b57fff640 1 --2- 172.21.15.16:0/4044039817 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560f54337480 0x560f5434a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:03.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.733+0000 7f1b57fff640 1 -- 172.21.15.16:0/4044039817 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560f54336b60 msgr2=0x560f5434a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:03.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.733+0000 7f1b57fff640 1 --2- 172.21.15.16:0/4044039817 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560f54336b60 0x560f5434a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:03.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.733+0000 7f1b57fff640 1 -- 172.21.15.16:0/4044039817 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f5432d680 con 0x560f54341d40 2024-09-12T02:08:03.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.733+0000 7f1b5f7fe640 1 --2- 172.21.15.16:0/4044039817 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560f54337480 0x560f5434a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:08:03.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.733+0000 7f1b57fff640 1 --2- 172.21.15.16:0/4044039817 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f54341d40 0x560f543385b0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f1b48003c70 tx=0x7f1b480035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30e990b0a88ed9fd server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:03.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.733+0000 7f1b557fa640 1 -- 172.21.15.16:0/4044039817 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b4804d070 con 0x560f54341d40 2024-09-12T02:08:03.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.733+0000 7f1b557fa640 1 -- 172.21.15.16:0/4044039817 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b48009520 con 0x560f54341d40 2024-09-12T02:08:03.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.733+0000 7f1b557fa640 1 -- 172.21.15.16:0/4044039817 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b48044070 con 0x560f54341d40 2024-09-12T02:08:03.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.734+0000 7f1b682fc740 1 -- 172.21.15.16:0/4044039817 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f54341d40 msgr2=0x560f543385b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:03.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.734+0000 7f1b682fc740 1 --2- 172.21.15.16:0/4044039817 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f54341d40 0x560f543385b0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f1b48003c70 tx=0x7f1b480035f0 comp rx=0 tx=0).stop 2024-09-12T02:08:03.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.734+0000 7f1b682fc740 1 -- 172.21.15.16:0/4044039817 shutdown_connections 2024-09-12T02:08:03.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.734+0000 7f1b682fc740 1 --2- 172.21.15.16:0/4044039817 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f54341d40 0x560f543385b0 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:03.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.734+0000 7f1b682fc740 1 --2- 172.21.15.16:0/4044039817 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560f54337480 0x560f5434a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:03.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.734+0000 7f1b682fc740 1 --2- 172.21.15.16:0/4044039817 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560f54336b60 0x560f5434a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:03.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.734+0000 7f1b682fc740 1 -- 172.21.15.16:0/4044039817 >> 172.21.15.16:0/4044039817 conn(0x560f543330f0 msgr2=0x560f5433dcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:03.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.734+0000 7f1b682fc740 1 -- 172.21.15.16:0/4044039817 shutdown_connections 2024-09-12T02:08:03.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.734+0000 7f1b682fc740 1 -- 172.21.15.16:0/4044039817 wait complete. 2024-09-12T02:08:03.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.734+0000 7f1b682fc740 1 -- start start 2024-09-12T02:08:03.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.734+0000 7f1b682fc740 1 Processor -- start 2024-09-12T02:08:03.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.735+0000 7f1b682fc740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560f54336b60 0x560f5433ae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:03.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.735+0000 7f1b682fc740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560f54337480 0x560f5433b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:03.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.735+0000 7f1b682fc740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f54341d40 0x560f5433b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:03.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.735+0000 7f1b682fc740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1b601ab9d0 con 0x560f54341d40 2024-09-12T02:08:03.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.735+0000 7f1b682fc740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1b601ace40 con 0x560f54337480 2024-09-12T02:08:03.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.735+0000 7f1b682fc740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1b601ac0a0 con 0x560f54336b60 2024-09-12T02:08:03.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.735+0000 7f1b682fc740 10 client.0 _ll_register_callbacks cb 0x7f1b500093c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:08:03.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.735+0000 7f1b57fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f54341d40 0x560f5433b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:03.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.735+0000 7f1b57fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f54341d40 0x560f5433b8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55368/0 (socket says 172.21.15.16:55368) 2024-09-12T02:08:03.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.735+0000 7f1b57fff640 1 -- 172.21.15.16:0/1871332121 learned_addr learned my addr 172.21.15.16:0/1871332121 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:08:03.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.735+0000 7f1b5f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560f54337480 0x560f5433b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:03.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.735+0000 7f1b57fff640 1 --2- 172.21.15.16:0/1871332121 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560f54336b60 0x560f5433ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:03.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.735+0000 7f1b57fff640 1 -- 172.21.15.16:0/1871332121 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560f54336b60 msgr2=0x560f5433ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:03.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.735+0000 7f1b57fff640 1 --2- 172.21.15.16:0/1871332121 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560f54336b60 0x560f5433ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:03.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.735+0000 7f1b57fff640 1 -- 172.21.15.16:0/1871332121 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560f54337480 msgr2=0x560f5433b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:03.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.735+0000 7f1b57fff640 1 --2- 172.21.15.16:0/1871332121 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560f54337480 0x560f5433b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:03.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.735+0000 7f1b57fff640 1 -- 172.21.15.16:0/1871332121 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f5433c100 con 0x560f54341d40 2024-09-12T02:08:03.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.735+0000 7f1b57fff640 1 --2- 172.21.15.16:0/1871332121 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f54341d40 0x560f5433b8d0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f1b48003b00 tx=0x7f1b48005de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ecfc07f57909d44 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:03.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.735+0000 7f1b3effd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:08:03.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.735+0000 7f1b3effd640 1 -- 172.21.15.16:0/1871332121 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b4804d070 con 0x560f54341d40 2024-09-12T02:08:03.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.735+0000 7f1b3effd640 1 -- 172.21.15.16:0/1871332121 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f1b48005250 con 0x560f54341d40 2024-09-12T02:08:03.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.735+0000 7f1b682fc740 1 -- 172.21.15.16:0/1871332121 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b48009520 con 0x560f54341d40 2024-09-12T02:08:03.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.736+0000 7f1b3effd640 1 -- 172.21.15.16:0/1871332121 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1b48044040 con 0x560f54341d40 2024-09-12T02:08:03.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.736+0000 7f1b682fc740 10 client.17289 fetch_fsmap learned FSMap version 25 2024-09-12T02:08:03.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.736+0000 7f1b682fc740 1 -- 172.21.15.16:0/1871332121 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x560f543c07f0 con 0x560f54341d40 2024-09-12T02:08:03.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.736+0000 7f1b3effd640 1 -- 172.21.15.16:0/1871332121 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b48009520 con 0x560f54341d40 2024-09-12T02:08:03.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.736+0000 7f1b3effd640 1 -- 172.21.15.16:0/1871332121 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f1b4803e790 con 0x560f54341d40 2024-09-12T02:08:03.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.736+0000 7f1b682fc740 10 client.17289 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:08:03.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.736+0000 7f1b682fc740 10 client.17289 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:08:03.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.736+0000 7f1b682fc740 20 client.17289 populate_metadata read hostname 'smithi016' 2024-09-12T02:08:03.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.736+0000 7f1b682fc740 1 --2- 172.21.15.16:0/1871332121 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x560f543c0c60 0x560f543c3050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:03.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.736+0000 7f1b682fc740 4 client.17289 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-12T02:08:03.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.736+0000 7f1b682fc740 1 -- 172.21.15.16:0/1871332121 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x560f542a3d00 con 0x560f543c0c60 2024-09-12T02:08:03.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.736+0000 7f1b5f7fe640 1 --2- 172.21.15.16:0/1871332121 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x560f543c0c60 0x560f543c3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:03.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.737+0000 7f1b5f7fe640 1 --2- 172.21.15.16:0/1871332121 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x560f543c0c60 0x560f543c3050 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=28c47d56d4f02b9e server_cookie=e77bcb6742e40bf in_seq=0 out_seq=0 2024-09-12T02:08:03.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.737+0000 7f1b3effd640 10 client.17289 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:08:03.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.738+0000 7f1b3effd640 1 -- 172.21.15.16:0/1871332121 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x560f542a3d00 con 0x560f543c0c60 2024-09-12T02:08:03.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.738+0000 7f1b3effd640 10 client.17289 handle_command_reply: tid=0 2024-09-12T02:08:03.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.738+0000 7f1b682fc740 1 client.17289 shutdown 2024-09-12T02:08:03.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.738+0000 7f1b682fc740 1 -- 172.21.15.16:0/1871332121 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f54341d40 msgr2=0x560f5433b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:03.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.738+0000 7f1b682fc740 1 --2- 172.21.15.16:0/1871332121 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f54341d40 0x560f5433b8d0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f1b48003b00 tx=0x7f1b48005de0 comp rx=0 tx=0).stop 2024-09-12T02:08:03.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.738+0000 7f1b682fc740 1 -- 172.21.15.16:0/1871332121 shutdown_connections 2024-09-12T02:08:03.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.738+0000 7f1b682fc740 1 --2- 172.21.15.16:0/1871332121 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x560f543c0c60 0x560f543c3050 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:03.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.738+0000 7f1b682fc740 1 --2- 172.21.15.16:0/1871332121 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f54341d40 0x560f5433b8d0 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:03.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.738+0000 7f1b682fc740 1 --2- 172.21.15.16:0/1871332121 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560f54337480 0x560f5433b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:03.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.738+0000 7f1b682fc740 1 --2- 172.21.15.16:0/1871332121 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560f54336b60 0x560f5433ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:03.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.738+0000 7f1b682fc740 1 -- 172.21.15.16:0/1871332121 >> 172.21.15.16:0/1871332121 conn(0x560f543330f0 msgr2=0x560f54331a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:03.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.738+0000 7f1b3effd640 0 client.17289 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:08:03.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.738+0000 7f1b682fc740 1 -- 172.21.15.16:0/1871332121 shutdown_connections 2024-09-12T02:08:03.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.738+0000 7f1b682fc740 1 -- 172.21.15.16:0/1871332121 wait complete. 2024-09-12T02:08:03.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.738+0000 7f1b682fc740 20 client.17289 trim_cache size 0 max 16384 2024-09-12T02:08:03.746 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T02:08:03.746 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-09-12T02:08:03.746 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "0636b809-e471-4db7-afa8-75f0219b5288", 2024-09-12T02:08:03.746 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-09-12T02:08:03.746 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T02:08:03.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.739+0000 7f1b65f23640 1 -- 172.21.15.16:0/2686367907 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f1b4c0a2b80 msgr2=0x7f1b4c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:03.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.739+0000 7f1b65f23640 1 --2- 172.21.15.16:0/2686367907 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f1b4c0a2b80 0x7f1b4c0a5040 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f1b5800c640 tx=0x7f1b580665c0 comp rx=0 tx=0).stop 2024-09-12T02:08:03.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.739+0000 7f1b65f23640 1 -- 172.21.15.16:0/2686367907 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b6010c180 msgr2=0x7f1b6019cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:03.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.739+0000 7f1b65f23640 1 --2- 172.21.15.16:0/2686367907 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b6010c180 0x7f1b6019cd80 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f1b5800e940 tx=0x7f1b5800ed40 comp rx=0 tx=0).stop 2024-09-12T02:08:03.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.739+0000 7f1b65f23640 1 -- 172.21.15.16:0/2686367907 shutdown_connections 2024-09-12T02:08:03.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.739+0000 7f1b65f23640 1 --2- 172.21.15.16:0/2686367907 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f1b4c0a2b80 0x7f1b4c0a5040 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:03.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.739+0000 7f1b65f23640 1 --2- 172.21.15.16:0/2686367907 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b6010c180 0x7f1b6019cd80 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:03.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.739+0000 7f1b65f23640 1 --2- 172.21.15.16:0/2686367907 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b601036c0 0x7f1b601989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:03.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.739+0000 7f1b65f23640 1 --2- 172.21.15.16:0/2686367907 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1b60102da0 0x7f1b601984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:03.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.739+0000 7f1b65f23640 1 -- 172.21.15.16:0/2686367907 >> 172.21.15.16:0/2686367907 conn(0x7f1b600fc7a0 msgr2=0x7f1b600fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:03.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.739+0000 7f1b65f23640 1 -- 172.21.15.16:0/2686367907 shutdown_connections 2024-09-12T02:08:03.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:03.739+0000 7f1b65f23640 1 -- 172.21.15.16:0/2686367907 wait complete. 2024-09-12T02:08:04.434 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T02:08:04.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:08:04 smithi184 ceph-mon[35593]: pgmap v519: 97 pgs: 97 active+clean; 3.2 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 8.2 KiB/s wr, 10 op/s 2024-09-12T02:08:04.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:08:04 smithi033 ceph-mon[35332]: pgmap v519: 97 pgs: 97 active+clean; 3.2 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 8.2 KiB/s wr, 10 op/s 2024-09-12T02:08:04.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:08:04 smithi016 ceph-mon[30947]: pgmap v519: 97 pgs: 97 active+clean; 3.2 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 8.2 KiB/s wr, 10 op/s 2024-09-12T02:08:04.689 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:08:05.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.098+0000 7f88b9b70640 1 -- 172.21.15.16:0/2424849960 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88b4101de0 msgr2=0x7f88b41021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:05.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.098+0000 7f88b9b70640 1 --2- 172.21.15.16:0/2424849960 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88b4101de0 0x7f88b41021c0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f88a4009ca0 tx=0x7f88a402a8c0 comp rx=0 tx=0).stop 2024-09-12T02:08:05.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.099+0000 7f88b9b70640 1 -- 172.21.15.16:0/2424849960 shutdown_connections 2024-09-12T02:08:05.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.099+0000 7f88b9b70640 1 --2- 172.21.15.16:0/2424849960 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f88b4106f10 0x7f88b410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:05.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.099+0000 7f88b9b70640 1 --2- 172.21.15.16:0/2424849960 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f88b4102790 0x7f88b41067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:05.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.099+0000 7f88b9b70640 1 --2- 172.21.15.16:0/2424849960 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88b4101de0 0x7f88b41021c0 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:05.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.099+0000 7f88b9b70640 1 -- 172.21.15.16:0/2424849960 >> 172.21.15.16:0/2424849960 conn(0x7f88b40fd8c0 msgr2=0x7f88b40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:05.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.099+0000 7f88b9b70640 1 -- 172.21.15.16:0/2424849960 shutdown_connections 2024-09-12T02:08:05.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.099+0000 7f88b9b70640 1 -- 172.21.15.16:0/2424849960 wait complete. 2024-09-12T02:08:05.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.099+0000 7f88b9b70640 1 Processor -- start 2024-09-12T02:08:05.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.099+0000 7f88b9b70640 1 -- start start 2024-09-12T02:08:05.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.100+0000 7f88b9b70640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88b4101de0 0x7f88b40744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:05.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.100+0000 7f88b9b70640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f88b4102790 0x7f88b4074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:05.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.100+0000 7f88b9b70640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f88b4106f10 0x7f88b4074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:05.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.100+0000 7f88b9b70640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f88b4111f30 con 0x7f88b4101de0 2024-09-12T02:08:05.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.100+0000 7f88b9b70640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f88b4111db0 con 0x7f88b4102790 2024-09-12T02:08:05.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.100+0000 7f88b9b70640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f88b41120b0 con 0x7f88b4106f10 2024-09-12T02:08:05.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.100+0000 7f88b37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88b4101de0 0x7f88b40744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:05.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.100+0000 7f88b37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88b4101de0 0x7f88b40744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55390/0 (socket says 172.21.15.16:55390) 2024-09-12T02:08:05.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.100+0000 7f88b37fe640 1 -- 172.21.15.16:0/4270818617 learned_addr learned my addr 172.21.15.16:0/4270818617 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:08:05.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.100+0000 7f88b2ffd640 1 --2- 172.21.15.16:0/4270818617 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f88b4102790 0x7f88b4074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:05.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.100+0000 7f88b3fff640 1 --2- 172.21.15.16:0/4270818617 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f88b4106f10 0x7f88b4074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:05.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.100+0000 7f88b37fe640 1 -- 172.21.15.16:0/4270818617 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f88b4106f10 msgr2=0x7f88b4074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:05.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.100+0000 7f88b37fe640 1 --2- 172.21.15.16:0/4270818617 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f88b4106f10 0x7f88b4074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:05.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.100+0000 7f88b37fe640 1 -- 172.21.15.16:0/4270818617 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f88b4102790 msgr2=0x7f88b4074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:05.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.100+0000 7f88b37fe640 1 --2- 172.21.15.16:0/4270818617 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f88b4102790 0x7f88b4074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:05.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.100+0000 7f88b37fe640 1 -- 172.21.15.16:0/4270818617 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88b41a6980 con 0x7f88b4101de0 2024-09-12T02:08:05.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.100+0000 7f88b37fe640 1 --2- 172.21.15.16:0/4270818617 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88b4101de0 0x7f88b40744f0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f88a402ade0 tx=0x7f88a40319f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:05.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.101+0000 7f88b0ff9640 1 -- 172.21.15.16:0/4270818617 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88a402fa20 con 0x7f88b4101de0 2024-09-12T02:08:05.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.101+0000 7f88b0ff9640 1 -- 172.21.15.16:0/4270818617 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88a402fbc0 con 0x7f88b4101de0 2024-09-12T02:08:05.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.101+0000 7f88b9b70640 1 -- 172.21.15.16:0/4270818617 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88b41a6c10 con 0x7f88b4101de0 2024-09-12T02:08:05.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.101+0000 7f88b0ff9640 1 -- 172.21.15.16:0/4270818617 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88a40466e0 con 0x7f88b4101de0 2024-09-12T02:08:05.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.101+0000 7f88b9b70640 1 -- 172.21.15.16:0/4270818617 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88b41a71a0 con 0x7f88b4101de0 2024-09-12T02:08:05.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.103+0000 7f88b0ff9640 1 -- 172.21.15.16:0/4270818617 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f88a4046940 con 0x7f88b4101de0 2024-09-12T02:08:05.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.103+0000 7f88bbf49740 1 Processor -- start 2024-09-12T02:08:05.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.103+0000 7f88bbf49740 1 -- start start 2024-09-12T02:08:05.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.103+0000 7f88bbf49740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c863ebe4a0 0x55c863ebe880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:05.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.103+0000 7f88bbf49740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c863ebedc0 0x55c863ec9140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:05.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.103+0000 7f88bbf49740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c863ec9680 0x55c863ecba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:05.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.103+0000 7f88bbf49740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55c863e242e0 con 0x55c863ebe4a0 2024-09-12T02:08:05.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.103+0000 7f88bbf49740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55c863dcbce0 con 0x55c863ebedc0 2024-09-12T02:08:05.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.103+0000 7f88bbf49740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55c863e202a0 con 0x55c863ec9680 2024-09-12T02:08:05.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.104+0000 7f88b3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c863ebe4a0 0x55c863ebe880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:05.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.104+0000 7f88b3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c863ebe4a0 0x55c863ebe880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55394/0 (socket says 172.21.15.16:55394) 2024-09-12T02:08:05.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.104+0000 7f88b3fff640 1 -- 172.21.15.16:0/4292665584 learned_addr learned my addr 172.21.15.16:0/4292665584 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:08:05.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.104+0000 7f88b0ff9640 1 --2- 172.21.15.16:0/4270818617 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f889c0a2c00 0x7f889c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:05.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.104+0000 7f88b37fe640 1 --2- 172.21.15.16:0/4292665584 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c863ebedc0 0x55c863ec9140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:05.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.104+0000 7f88b2ffd640 1 --2- 172.21.15.16:0/4292665584 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c863ec9680 0x55c863ecba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:05.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.104+0000 7f88b3fff640 1 -- 172.21.15.16:0/4292665584 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c863ec9680 msgr2=0x55c863ecba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:05.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.104+0000 7f88b3fff640 1 --2- 172.21.15.16:0/4292665584 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c863ec9680 0x55c863ecba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:05.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.104+0000 7f88b3fff640 1 -- 172.21.15.16:0/4292665584 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c863ebedc0 msgr2=0x55c863ec9140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:05.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.104+0000 7f88b3fff640 1 --2- 172.21.15.16:0/4292665584 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c863ebedc0 0x55c863ec9140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:05.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.104+0000 7f88b3fff640 1 -- 172.21.15.16:0/4292665584 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c863db0440 con 0x55c863ebe4a0 2024-09-12T02:08:05.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.104+0000 7f88b0ff9640 1 -- 172.21.15.16:0/4270818617 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f88a40f0310 con 0x7f88b4101de0 2024-09-12T02:08:05.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.104+0000 7f88b3fff640 1 --2- 172.21.15.16:0/4270818617 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f889c0a2c00 0x7f889c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:05.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.104+0000 7f88b3fff640 1 --2- 172.21.15.16:0/4292665584 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c863ebe4a0 0x55c863ebe880 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f88ac009da0 tx=0x7f88ac011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bbbd8cee4406000 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:05.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.104+0000 7f8899ffb640 1 -- 172.21.15.16:0/4292665584 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88ac017050 con 0x55c863ebe4a0 2024-09-12T02:08:05.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.104+0000 7f8899ffb640 1 -- 172.21.15.16:0/4292665584 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88ac026920 con 0x55c863ebe4a0 2024-09-12T02:08:05.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.105+0000 7f8899ffb640 1 -- 172.21.15.16:0/4292665584 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88ac00f820 con 0x55c863ebe4a0 2024-09-12T02:08:05.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.105+0000 7f88bbf49740 1 -- 172.21.15.16:0/4292665584 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c863ebe4a0 msgr2=0x55c863ebe880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:05.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.105+0000 7f88bbf49740 1 --2- 172.21.15.16:0/4292665584 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c863ebe4a0 0x55c863ebe880 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f88ac009da0 tx=0x7f88ac011810 comp rx=0 tx=0).stop 2024-09-12T02:08:05.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.105+0000 7f88b3fff640 1 --2- 172.21.15.16:0/4270818617 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f889c0a2c00 0x7f889c0a50c0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f88ac023000 tx=0x7f88ac024000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:05.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.105+0000 7f88bbf49740 1 -- 172.21.15.16:0/4292665584 shutdown_connections 2024-09-12T02:08:05.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.105+0000 7f88bbf49740 1 --2- 172.21.15.16:0/4292665584 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c863ec9680 0x55c863ecba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:05.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.105+0000 7f88bbf49740 1 --2- 172.21.15.16:0/4292665584 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c863ebedc0 0x55c863ec9140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:05.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.105+0000 7f88bbf49740 1 --2- 172.21.15.16:0/4292665584 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c863ebe4a0 0x55c863ebe880 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:05.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.105+0000 7f88bbf49740 1 -- 172.21.15.16:0/4292665584 >> 172.21.15.16:0/4292665584 conn(0x55c863ebaa30 msgr2=0x55c863ebae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:05.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.105+0000 7f88bbf49740 1 -- 172.21.15.16:0/4292665584 shutdown_connections 2024-09-12T02:08:05.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.105+0000 7f88bbf49740 1 -- 172.21.15.16:0/4292665584 wait complete. 2024-09-12T02:08:05.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.106+0000 7f88bbf49740 1 -- start start 2024-09-12T02:08:05.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.106+0000 7f88bbf49740 1 Processor -- start 2024-09-12T02:08:05.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.106+0000 7f88bbf49740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c863ebe4a0 0x55c863ec8f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:05.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.106+0000 7f88bbf49740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c863ebedc0 0x55c863ec6200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:05.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.106+0000 7f88bbf49740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c863ec9680 0x55c863ec6740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:05.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.106+0000 7f88bbf49740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55c863ecc250 con 0x55c863ebe4a0 2024-09-12T02:08:05.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.106+0000 7f88bbf49740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55c863ddf050 con 0x55c863ebedc0 2024-09-12T02:08:05.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.106+0000 7f88bbf49740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55c863ecc3d0 con 0x55c863ec9680 2024-09-12T02:08:05.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.106+0000 7f88bbf49740 10 client.0 _ll_register_callbacks cb 0x55c863e1f160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:08:05.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.106+0000 7f88b37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c863ebe4a0 0x55c863ec8f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:05.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.106+0000 7f88b37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c863ebe4a0 0x55c863ec8f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55402/0 (socket says 172.21.15.16:55402) 2024-09-12T02:08:05.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.106+0000 7f88b37fe640 1 -- 172.21.15.16:0/777801248 learned_addr learned my addr 172.21.15.16:0/777801248 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:08:05.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.106+0000 7f88b3fff640 1 --2- 172.21.15.16:0/777801248 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c863ec9680 0x55c863ec6740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:05.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.107+0000 7f88b2ffd640 1 --2- 172.21.15.16:0/777801248 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c863ebedc0 0x55c863ec6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:05.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.107+0000 7f88b37fe640 1 -- 172.21.15.16:0/777801248 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c863ec9680 msgr2=0x55c863ec6740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:05.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.107+0000 7f88b37fe640 1 --2- 172.21.15.16:0/777801248 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c863ec9680 0x55c863ec6740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:05.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.107+0000 7f88b37fe640 1 -- 172.21.15.16:0/777801248 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c863ebedc0 msgr2=0x55c863ec6200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:05.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.107+0000 7f88b37fe640 1 --2- 172.21.15.16:0/777801248 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c863ebedc0 0x55c863ec6200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:05.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.107+0000 7f88b37fe640 1 -- 172.21.15.16:0/777801248 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c863b953c0 con 0x55c863ebe4a0 2024-09-12T02:08:05.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.107+0000 7f88b2ffd640 1 --2- 172.21.15.16:0/777801248 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c863ebedc0 0x55c863ec6200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:08:05.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.107+0000 7f88b37fe640 1 --2- 172.21.15.16:0/777801248 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c863ebe4a0 0x55c863ec8f50 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f88a40bcba0 tx=0x7f88a40bc810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42b5ec9fe0ad4fa1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:05.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.107+0000 7f8892ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:08:05.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.107+0000 7f8892ffd640 1 -- 172.21.15.16:0/777801248 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88a40bcc10 con 0x55c863ebe4a0 2024-09-12T02:08:05.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.107+0000 7f8892ffd640 1 -- 172.21.15.16:0/777801248 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f88a402fd70 con 0x55c863ebe4a0 2024-09-12T02:08:05.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.107+0000 7f88bbf49740 1 -- 172.21.15.16:0/777801248 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f88ac026920 con 0x55c863ebe4a0 2024-09-12T02:08:05.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.107+0000 7f8892ffd640 1 -- 172.21.15.16:0/777801248 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f88a40c0fc0 con 0x55c863ebe4a0 2024-09-12T02:08:05.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.107+0000 7f88bbf49740 10 client.17301 fetch_fsmap learned FSMap version 25 2024-09-12T02:08:05.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.107+0000 7f88bbf49740 1 -- 172.21.15.16:0/777801248 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c863ecc250 con 0x55c863ebe4a0 2024-09-12T02:08:05.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.107+0000 7f8892ffd640 1 -- 172.21.15.16:0/777801248 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88ac026920 con 0x55c863ebe4a0 2024-09-12T02:08:05.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.108+0000 7f8892ffd640 1 -- 172.21.15.16:0/777801248 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f88a404e030 con 0x55c863ebe4a0 2024-09-12T02:08:05.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.108+0000 7f88bbf49740 10 client.17301 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:08:05.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.108+0000 7f88bbf49740 10 client.17301 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:08:05.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.108+0000 7f88bbf49740 20 client.17301 populate_metadata read hostname 'smithi016' 2024-09-12T02:08:05.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.108+0000 7f88bbf49740 1 --2- 172.21.15.16:0/777801248 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55c863ed8fa0 0x55c863edb3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:05.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.108+0000 7f88bbf49740 4 client.17301 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:08:05.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.108+0000 7f88bbf49740 1 -- 172.21.15.16:0/777801248 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c863e31850 con 0x55c863ed8fa0 2024-09-12T02:08:05.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.108+0000 7f88b2ffd640 1 --2- 172.21.15.16:0/777801248 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55c863ed8fa0 0x55c863edb3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:05.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.108+0000 7f8892ffd640 10 client.17301 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:08:05.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.108+0000 7f88b2ffd640 1 --2- 172.21.15.16:0/777801248 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55c863ed8fa0 0x55c863edb3f0 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dde2faa749e39f02 server_cookie=c8097b5b2ce096fb in_seq=0 out_seq=0 2024-09-12T02:08:05.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.109+0000 7f8892ffd640 1 -- 172.21.15.16:0/777801248 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c863e31850 con 0x55c863ed8fa0 2024-09-12T02:08:05.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.109+0000 7f8892ffd640 10 client.17301 handle_command_reply: tid=0 2024-09-12T02:08:05.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.109+0000 7f88bbf49740 1 client.17301 shutdown 2024-09-12T02:08:05.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.109+0000 7f88bbf49740 1 -- 172.21.15.16:0/777801248 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c863ebe4a0 msgr2=0x55c863ec8f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:05.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.109+0000 7f88bbf49740 1 --2- 172.21.15.16:0/777801248 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c863ebe4a0 0x55c863ec8f50 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f88a40bcba0 tx=0x7f88a40bc810 comp rx=0 tx=0).stop 2024-09-12T02:08:05.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.109+0000 7f88bbf49740 1 -- 172.21.15.16:0/777801248 shutdown_connections 2024-09-12T02:08:05.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.109+0000 7f88bbf49740 1 --2- 172.21.15.16:0/777801248 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55c863ed8fa0 0x55c863edb3f0 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:05.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.109+0000 7f88bbf49740 1 --2- 172.21.15.16:0/777801248 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c863ec9680 0x55c863ec6740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:05.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.109+0000 7f88bbf49740 1 --2- 172.21.15.16:0/777801248 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c863ebedc0 0x55c863ec6200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:05.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.109+0000 7f88bbf49740 1 --2- 172.21.15.16:0/777801248 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c863ebe4a0 0x55c863ec8f50 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:05.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.109+0000 7f8892ffd640 0 client.17301 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:08:05.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.109+0000 7f88bbf49740 1 -- 172.21.15.16:0/777801248 >> 172.21.15.16:0/777801248 conn(0x55c863ebaa30 msgr2=0x55c863eb9b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:05.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.110+0000 7f88bbf49740 1 -- 172.21.15.16:0/777801248 shutdown_connections 2024-09-12T02:08:05.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.110+0000 7f88bbf49740 1 -- 172.21.15.16:0/777801248 wait complete. 2024-09-12T02:08:05.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.110+0000 7f88bbf49740 20 client.17301 trim_cache size 0 max 16384 2024-09-12T02:08:05.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.131+0000 7f88bbf49740 1 Processor -- start 2024-09-12T02:08:05.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.132+0000 7f88bbf49740 1 -- start start 2024-09-12T02:08:05.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.132+0000 7f88bbf49740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c863ebaa30 0x55c863ed54e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:05.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.132+0000 7f88bbf49740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c863ebe4a0 0x55c863ec17f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:05.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.132+0000 7f88bbf49740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c863ebedc0 0x55c863ee0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:05.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.132+0000 7f88bbf49740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f88b41b1da0 con 0x55c863ebe4a0 2024-09-12T02:08:05.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.132+0000 7f88bbf49740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f88b41af670 con 0x55c863ebedc0 2024-09-12T02:08:05.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.132+0000 7f88bbf49740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f88b41ad4c0 con 0x55c863ebaa30 2024-09-12T02:08:05.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.132+0000 7f88b2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c863ebe4a0 0x55c863ec17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:05.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.132+0000 7f88b2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c863ebe4a0 0x55c863ec17f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55416/0 (socket says 172.21.15.16:55416) 2024-09-12T02:08:05.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.132+0000 7f88b2ffd640 1 -- 172.21.15.16:0/91666891 learned_addr learned my addr 172.21.15.16:0/91666891 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:08:05.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.132+0000 7f88b37fe640 1 --2- 172.21.15.16:0/91666891 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c863ebaa30 0x55c863ed54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:05.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.132+0000 7f88b3fff640 1 --2- 172.21.15.16:0/91666891 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c863ebedc0 0x55c863ee0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:05.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.132+0000 7f88b2ffd640 1 -- 172.21.15.16:0/91666891 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c863ebaa30 msgr2=0x55c863ed54e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:05.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.132+0000 7f88b2ffd640 1 --2- 172.21.15.16:0/91666891 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c863ebaa30 0x55c863ed54e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:05.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.132+0000 7f88b2ffd640 1 -- 172.21.15.16:0/91666891 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c863ebedc0 msgr2=0x55c863ee0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:05.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.132+0000 7f88b2ffd640 1 --2- 172.21.15.16:0/91666891 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c863ebedc0 0x55c863ee0cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:05.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.132+0000 7f88b2ffd640 1 -- 172.21.15.16:0/91666891 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c863ebebf0 con 0x55c863ebe4a0 2024-09-12T02:08:05.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.132+0000 7f88b3fff640 1 --2- 172.21.15.16:0/91666891 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c863ebedc0 0x55c863ee0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:08:05.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.133+0000 7f88b2ffd640 1 --2- 172.21.15.16:0/91666891 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c863ebe4a0 0x55c863ec17f0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f88b41087b0 tx=0x7f88a8013830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d16b44786d0edba2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:05.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.133+0000 7f88997fa640 1 -- 172.21.15.16:0/91666891 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88a8004020 con 0x55c863ebe4a0 2024-09-12T02:08:05.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.133+0000 7f88997fa640 1 -- 172.21.15.16:0/91666891 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88a80041c0 con 0x55c863ebe4a0 2024-09-12T02:08:05.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.133+0000 7f88997fa640 1 -- 172.21.15.16:0/91666891 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88a801c6c0 con 0x55c863ebe4a0 2024-09-12T02:08:05.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.133+0000 7f88bbf49740 1 -- 172.21.15.16:0/91666891 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c863ebe4a0 msgr2=0x55c863ec17f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:05.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.133+0000 7f88bbf49740 1 --2- 172.21.15.16:0/91666891 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c863ebe4a0 0x55c863ec17f0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f88b41087b0 tx=0x7f88a8013830 comp rx=0 tx=0).stop 2024-09-12T02:08:05.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.134+0000 7f88bbf49740 1 -- 172.21.15.16:0/91666891 shutdown_connections 2024-09-12T02:08:05.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.134+0000 7f88bbf49740 1 --2- 172.21.15.16:0/91666891 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c863ebedc0 0x55c863ee0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:05.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.134+0000 7f88bbf49740 1 --2- 172.21.15.16:0/91666891 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c863ebe4a0 0x55c863ec17f0 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:05.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.134+0000 7f88bbf49740 1 --2- 172.21.15.16:0/91666891 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c863ebaa30 0x55c863ed54e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:05.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.134+0000 7f88bbf49740 1 -- 172.21.15.16:0/91666891 >> 172.21.15.16:0/91666891 conn(0x55c863ec68a0 msgr2=0x55c863ec10e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:05.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.134+0000 7f88bbf49740 1 -- 172.21.15.16:0/91666891 shutdown_connections 2024-09-12T02:08:05.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.134+0000 7f88bbf49740 1 -- 172.21.15.16:0/91666891 wait complete. 2024-09-12T02:08:05.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.134+0000 7f88bbf49740 1 -- start start 2024-09-12T02:08:05.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.134+0000 7f88bbf49740 1 Processor -- start 2024-09-12T02:08:05.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.135+0000 7f88bbf49740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c863ebaa30 0x55c863ed1f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:05.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.135+0000 7f88bbf49740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c863ebe4a0 0x55c863ecd290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:05.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.135+0000 7f88bbf49740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c863ebedc0 0x55c863ecda20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:05.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.135+0000 7f88bbf49740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f88b41adb30 con 0x55c863ebedc0 2024-09-12T02:08:05.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.135+0000 7f88bbf49740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f88b41aefa0 con 0x55c863ebe4a0 2024-09-12T02:08:05.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.135+0000 7f88bbf49740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f88b41ae200 con 0x55c863ebaa30 2024-09-12T02:08:05.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.135+0000 7f88bbf49740 10 client.0 _ll_register_callbacks cb 0x7f88a40b8f80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:08:05.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.135+0000 7f88b3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c863ebedc0 0x55c863ecda20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:05.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.135+0000 7f88b37fe640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c863ebaa30 0x55c863ed1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:05.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.135+0000 7f88b3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c863ebedc0 0x55c863ecda20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55424/0 (socket says 172.21.15.16:55424) 2024-09-12T02:08:05.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.135+0000 7f88b3fff640 1 -- 172.21.15.16:0/2608544026 learned_addr learned my addr 172.21.15.16:0/2608544026 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:08:05.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.135+0000 7f88b2ffd640 1 --2- 172.21.15.16:0/2608544026 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c863ebe4a0 0x55c863ecd290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:05.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.135+0000 7f88b3fff640 1 -- 172.21.15.16:0/2608544026 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c863ebaa30 msgr2=0x55c863ed1f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:05.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.135+0000 7f88b3fff640 1 --2- 172.21.15.16:0/2608544026 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c863ebaa30 0x55c863ed1f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:05.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.135+0000 7f88b3fff640 1 -- 172.21.15.16:0/2608544026 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c863ebe4a0 msgr2=0x55c863ecd290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:05.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.135+0000 7f88b3fff640 1 --2- 172.21.15.16:0/2608544026 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c863ebe4a0 0x55c863ecd290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:05.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.135+0000 7f88b3fff640 1 -- 172.21.15.16:0/2608544026 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c863f42ca0 con 0x55c863ebedc0 2024-09-12T02:08:05.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.135+0000 7f88b37fe640 1 --2- 172.21.15.16:0/2608544026 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c863ebaa30 0x55c863ed1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:08:05.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.136+0000 7f88b3fff640 1 --2- 172.21.15.16:0/2608544026 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c863ebedc0 0x55c863ecda20 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f88ac00f510 tx=0x7f88ac00fd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22e1ff4e3af95da0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:05.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.136+0000 7f8892ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:08:05.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.136+0000 7f8892ffd640 1 -- 172.21.15.16:0/2608544026 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88ac026850 con 0x55c863ebedc0 2024-09-12T02:08:05.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.136+0000 7f8892ffd640 1 -- 172.21.15.16:0/2608544026 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f88ac0057f0 con 0x55c863ebedc0 2024-09-12T02:08:05.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.136+0000 7f88bbf49740 1 -- 172.21.15.16:0/2608544026 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f88a80041c0 con 0x55c863ebedc0 2024-09-12T02:08:05.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.136+0000 7f8892ffd640 1 -- 172.21.15.16:0/2608544026 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f88ac017020 con 0x55c863ebedc0 2024-09-12T02:08:05.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.136+0000 7f88bbf49740 10 client.17307 fetch_fsmap learned FSMap version 25 2024-09-12T02:08:05.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.136+0000 7f88bbf49740 1 -- 172.21.15.16:0/2608544026 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c863f43e30 con 0x55c863ebedc0 2024-09-12T02:08:05.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.136+0000 7f8892ffd640 1 -- 172.21.15.16:0/2608544026 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88a80041c0 con 0x55c863ebedc0 2024-09-12T02:08:05.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.136+0000 7f8892ffd640 1 -- 172.21.15.16:0/2608544026 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f88ac031070 con 0x55c863ebedc0 2024-09-12T02:08:05.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.137+0000 7f88bbf49740 10 client.17307 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:08:05.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.137+0000 7f88bbf49740 10 client.17307 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:08:05.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.137+0000 7f88bbf49740 20 client.17307 populate_metadata read hostname 'smithi016' 2024-09-12T02:08:05.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.137+0000 7f88bbf49740 1 --2- 172.21.15.16:0/2608544026 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55c863ec9680 0x55c863f46470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:05.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.137+0000 7f88bbf49740 4 client.17307 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T02:08:05.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.137+0000 7f88bbf49740 1 -- 172.21.15.16:0/2608544026 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c863e2bd60 con 0x55c863ec9680 2024-09-12T02:08:05.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.137+0000 7f88b37fe640 1 --2- 172.21.15.16:0/2608544026 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55c863ec9680 0x55c863f46470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:05.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.137+0000 7f8892ffd640 10 client.17307 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:08:05.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.137+0000 7f88b37fe640 1 --2- 172.21.15.16:0/2608544026 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55c863ec9680 0x55c863f46470 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d33b6d19f80540ad server_cookie=a3f502b0c535cde3 in_seq=0 out_seq=0 2024-09-12T02:08:05.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.137+0000 7f8892ffd640 1 -- 172.21.15.16:0/2608544026 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55c863e2bd60 con 0x55c863ec9680 2024-09-12T02:08:05.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.137+0000 7f8892ffd640 10 client.17307 handle_command_reply: tid=0 2024-09-12T02:08:05.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.137+0000 7f88bbf49740 1 client.17307 shutdown 2024-09-12T02:08:05.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.138+0000 7f88bbf49740 1 -- 172.21.15.16:0/2608544026 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c863ebedc0 msgr2=0x55c863ecda20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:05.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.138+0000 7f88bbf49740 1 --2- 172.21.15.16:0/2608544026 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c863ebedc0 0x55c863ecda20 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f88ac00f510 tx=0x7f88ac00fd30 comp rx=0 tx=0).stop 2024-09-12T02:08:05.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.138+0000 7f88bbf49740 1 -- 172.21.15.16:0/2608544026 shutdown_connections 2024-09-12T02:08:05.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.138+0000 7f88bbf49740 1 --2- 172.21.15.16:0/2608544026 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55c863ec9680 0x55c863f46470 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:05.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.138+0000 7f88bbf49740 1 --2- 172.21.15.16:0/2608544026 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c863ebedc0 0x55c863ecda20 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:05.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.138+0000 7f88bbf49740 1 --2- 172.21.15.16:0/2608544026 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c863ebe4a0 0x55c863ecd290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:05.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.138+0000 7f88bbf49740 1 --2- 172.21.15.16:0/2608544026 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c863ebaa30 0x55c863ed1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:05.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.138+0000 7f88bbf49740 1 -- 172.21.15.16:0/2608544026 >> 172.21.15.16:0/2608544026 conn(0x55c863ec68a0 msgr2=0x55c863eb9b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:05.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.138+0000 7f8892ffd640 0 client.17307 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:08:05.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.138+0000 7f88bbf49740 1 -- 172.21.15.16:0/2608544026 shutdown_connections 2024-09-12T02:08:05.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.138+0000 7f88bbf49740 1 -- 172.21.15.16:0/2608544026 wait complete. 2024-09-12T02:08:05.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.138+0000 7f88bbf49740 20 client.17307 trim_cache size 0 max 16384 2024-09-12T02:08:05.148 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T02:08:05.148 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-12T02:08:05.148 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": { 2024-09-12T02:08:05.148 INFO:teuthology.orchestra.run.smithi016.stdout: "0636b809-e471-4db7-afa8-75f0219b5288": { 2024-09-12T02:08:05.148 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-12T02:08:05.149 INFO:teuthology.orchestra.run.smithi016.stdout: "tag": "0636b809-e471-4db7-afa8-75f0219b5288", 2024-09-12T02:08:05.149 INFO:teuthology.orchestra.run.smithi016.stdout: "options": "recursive,force" 2024-09-12T02:08:05.149 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:08:05.149 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:08:05.149 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T02:08:05.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.138+0000 7f88b9b70640 1 -- 172.21.15.16:0/4270818617 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f889c0a2c00 msgr2=0x7f889c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:05.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.138+0000 7f88b9b70640 1 --2- 172.21.15.16:0/4270818617 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f889c0a2c00 0x7f889c0a50c0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f88ac023000 tx=0x7f88ac024000 comp rx=0 tx=0).stop 2024-09-12T02:08:05.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.138+0000 7f88b9b70640 1 -- 172.21.15.16:0/4270818617 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88b4101de0 msgr2=0x7f88b40744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:05.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.138+0000 7f88b9b70640 1 --2- 172.21.15.16:0/4270818617 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88b4101de0 0x7f88b40744f0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f88a402ade0 tx=0x7f88a40319f0 comp rx=0 tx=0).stop 2024-09-12T02:08:05.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.139+0000 7f88b9b70640 1 -- 172.21.15.16:0/4270818617 shutdown_connections 2024-09-12T02:08:05.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.139+0000 7f88b9b70640 1 --2- 172.21.15.16:0/4270818617 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f889c0a2c00 0x7f889c0a50c0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:05.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.139+0000 7f88b9b70640 1 --2- 172.21.15.16:0/4270818617 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f88b4106f10 0x7f88b4074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:05.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.139+0000 7f88b9b70640 1 --2- 172.21.15.16:0/4270818617 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f88b4102790 0x7f88b4074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:05.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.139+0000 7f88b9b70640 1 --2- 172.21.15.16:0/4270818617 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88b4101de0 0x7f88b40744f0 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:05.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.139+0000 7f88b9b70640 1 -- 172.21.15.16:0/4270818617 >> 172.21.15.16:0/4270818617 conn(0x7f88b40fd8c0 msgr2=0x7f88b40fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:05.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.139+0000 7f88b9b70640 1 -- 172.21.15.16:0/4270818617 shutdown_connections 2024-09-12T02:08:05.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:05.139+0000 7f88b9b70640 1 -- 172.21.15.16:0/4270818617 wait complete. 2024-09-12T02:08:05.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:08:05 smithi016 ceph-mon[30947]: scrub queued for path: / 2024-09-12T02:08:05.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:08:05 smithi016 ceph-mon[30947]: scrub summary: idle+waiting paths [/] 2024-09-12T02:08:05.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:08:05 smithi016 ceph-mon[30947]: scrub summary: active paths [/] 2024-09-12T02:08:05.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:08:05 smithi016 ceph-mon[30947]: scrub summary: idle+waiting paths [/] 2024-09-12T02:08:05.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:08:05 smithi184 ceph-mon[35593]: scrub queued for path: / 2024-09-12T02:08:05.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:08:05 smithi184 ceph-mon[35593]: scrub summary: idle+waiting paths [/] 2024-09-12T02:08:05.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:08:05 smithi184 ceph-mon[35593]: scrub summary: active paths [/] 2024-09-12T02:08:05.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:08:05 smithi184 ceph-mon[35593]: scrub summary: idle+waiting paths [/] 2024-09-12T02:08:05.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:08:05 smithi033 ceph-mon[35332]: scrub queued for path: / 2024-09-12T02:08:05.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:08:05 smithi033 ceph-mon[35332]: scrub summary: idle+waiting paths [/] 2024-09-12T02:08:05.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:08:05 smithi033 ceph-mon[35332]: scrub summary: active paths [/] 2024-09-12T02:08:05.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:08:05 smithi033 ceph-mon[35332]: scrub summary: idle+waiting paths [/] 2024-09-12T02:08:05.681 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.678+0000 7facc412a640 1 -- 172.21.15.16:0/908112672 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facbc101c50 msgr2=0x7facbc1020d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:05.681 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.678+0000 7facc412a640 1 --2- 172.21.15.16:0/908112672 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facbc101c50 0x7facbc1020d0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7facac009900 tx=0x7facac02a410 comp rx=0 tx=0).stop 2024-09-12T02:08:05.681 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.678+0000 7facc412a640 1 -- 172.21.15.16:0/908112672 shutdown_connections 2024-09-12T02:08:05.682 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.678+0000 7facc412a640 1 --2- 172.21.15.16:0/908112672 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7facbc102610 0x7facbc108ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:05.682 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.678+0000 7facc412a640 1 --2- 172.21.15.16:0/908112672 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facbc101c50 0x7facbc1020d0 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:05.682 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.678+0000 7facc412a640 1 --2- 172.21.15.16:0/908112672 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7facbc100a60 0x7facbc100e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:05.682 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.678+0000 7facc412a640 1 -- 172.21.15.16:0/908112672 >> 172.21.15.16:0/908112672 conn(0x7facbc0fc1e0 msgr2=0x7facbc0fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:05.682 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.678+0000 7facc412a640 1 -- 172.21.15.16:0/908112672 shutdown_connections 2024-09-12T02:08:05.682 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.678+0000 7facc412a640 1 -- 172.21.15.16:0/908112672 wait complete. 2024-09-12T02:08:05.682 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.678+0000 7facc412a640 1 Processor -- start 2024-09-12T02:08:05.682 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.678+0000 7facc412a640 1 -- start start 2024-09-12T02:08:05.682 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.679+0000 7facc412a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7facbc100a60 0x7facbc06ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:05.683 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.679+0000 7facc412a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facbc101c50 0x7facbc19cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:05.683 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.679+0000 7facc412a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7facbc102610 0x7facbc19f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:05.683 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.679+0000 7facc412a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7facbc10c6c0 con 0x7facbc101c50 2024-09-12T02:08:05.683 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.679+0000 7facc412a640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7facbc10c540 con 0x7facbc100a60 2024-09-12T02:08:05.683 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.679+0000 7facc412a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7facbc10c840 con 0x7facbc102610 2024-09-12T02:08:05.683 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.679+0000 7facc16c5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facbc101c50 0x7facbc19cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:05.683 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.679+0000 7facc16c5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facbc101c50 0x7facbc19cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55452/0 (socket says 172.21.15.16:55452) 2024-09-12T02:08:05.683 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.679+0000 7facc16c5640 1 -- 172.21.15.16:0/1775357546 learned_addr learned my addr 172.21.15.16:0/1775357546 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:08:05.683 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.679+0000 7facc1ec6640 1 --2- 172.21.15.16:0/1775357546 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7facbc100a60 0x7facbc06ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:05.683 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.679+0000 7facc26c7640 1 --2- 172.21.15.16:0/1775357546 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7facbc102610 0x7facbc19f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:05.683 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.679+0000 7facc16c5640 1 -- 172.21.15.16:0/1775357546 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7facbc102610 msgr2=0x7facbc19f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:05.683 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.679+0000 7facc16c5640 1 --2- 172.21.15.16:0/1775357546 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7facbc102610 0x7facbc19f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:05.683 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.679+0000 7facc16c5640 1 -- 172.21.15.16:0/1775357546 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7facbc100a60 msgr2=0x7facbc06ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:05.683 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.679+0000 7facc16c5640 1 --2- 172.21.15.16:0/1775357546 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7facbc100a60 0x7facbc06ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:05.684 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.679+0000 7facc16c5640 1 -- 172.21.15.16:0/1775357546 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facbc19fa30 con 0x7facbc101c50 2024-09-12T02:08:05.684 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.679+0000 7facc26c7640 1 --2- 172.21.15.16:0/1775357546 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7facbc102610 0x7facbc19f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:08:05.684 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.679+0000 7facc16c5640 1 --2- 172.21.15.16:0/1775357546 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facbc101c50 0x7facbc19cb00 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7facac009230 tx=0x7facac031b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:05.684 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.679+0000 7facb2ffd640 1 -- 172.21.15.16:0/1775357546 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facac03f030 con 0x7facbc101c50 2024-09-12T02:08:05.684 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.679+0000 7facb2ffd640 1 -- 172.21.15.16:0/1775357546 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7facac003ad0 con 0x7facbc101c50 2024-09-12T02:08:05.684 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.679+0000 7facb2ffd640 1 -- 172.21.15.16:0/1775357546 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facac035070 con 0x7facbc101c50 2024-09-12T02:08:05.684 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.680+0000 7facc412a640 1 -- 172.21.15.16:0/1775357546 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facbc19fc60 con 0x7facbc101c50 2024-09-12T02:08:05.684 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.680+0000 7facc412a640 1 -- 172.21.15.16:0/1775357546 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facbc1a0100 con 0x7facbc101c50 2024-09-12T02:08:05.690 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.682+0000 7facb2ffd640 1 -- 172.21.15.16:0/1775357546 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7facac042030 con 0x7facbc101c50 2024-09-12T02:08:05.691 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.682+0000 7facc6ec7740 1 Processor -- start 2024-09-12T02:08:05.691 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.682+0000 7facc6ec7740 1 -- start start 2024-09-12T02:08:05.691 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.682+0000 7facc6ec7740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558dfa8f69a0 0x558dfa8f6da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:05.691 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.682+0000 7facc6ec7740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558dfa8f72e0 0x558dfa8fbda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:05.691 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.682+0000 7facc6ec7740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558dfa8fc370 0x558dfa8fe830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:05.691 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.682+0000 7facc6ec7740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x558dfa85a2f0 con 0x558dfa8f72e0 2024-09-12T02:08:05.691 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.683+0000 7facc6ec7740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x558dfa802270 con 0x558dfa8fc370 2024-09-12T02:08:05.691 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.683+0000 7facc6ec7740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x558dfa8562b0 con 0x558dfa8f69a0 2024-09-12T02:08:05.691 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.683+0000 7facc1ec6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558dfa8f72e0 0x558dfa8fbda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:05.692 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.683+0000 7facc1ec6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558dfa8f72e0 0x558dfa8fbda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55466/0 (socket says 172.21.15.16:55466) 2024-09-12T02:08:05.692 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.683+0000 7facc1ec6640 1 -- 172.21.15.16:0/2508696056 learned_addr learned my addr 172.21.15.16:0/2508696056 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:08:05.692 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.683+0000 7facb2ffd640 1 --2- 172.21.15.16:0/1775357546 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7faca80a2cb0 0x7faca80a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:05.692 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.683+0000 7facc16c5640 1 --2- 172.21.15.16:0/2508696056 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558dfa8fc370 0x558dfa8fe830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:05.692 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.683+0000 7facc26c7640 1 --2- 172.21.15.16:0/2508696056 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558dfa8f69a0 0x558dfa8f6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:05.692 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.683+0000 7facc1ec6640 1 -- 172.21.15.16:0/2508696056 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558dfa8f69a0 msgr2=0x558dfa8f6da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:05.692 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.683+0000 7facc1ec6640 1 --2- 172.21.15.16:0/2508696056 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558dfa8f69a0 0x558dfa8f6da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:05.692 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.683+0000 7facc1ec6640 1 -- 172.21.15.16:0/2508696056 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558dfa8fc370 msgr2=0x558dfa8fe830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:05.692 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.683+0000 7facc1ec6640 1 --2- 172.21.15.16:0/2508696056 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558dfa8fc370 0x558dfa8fe830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:05.692 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.683+0000 7facc1ec6640 1 -- 172.21.15.16:0/2508696056 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558dfa801340 con 0x558dfa8f72e0 2024-09-12T02:08:05.692 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.683+0000 7facc26c7640 1 --2- 172.21.15.16:0/2508696056 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558dfa8f69a0 0x558dfa8f6da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:08:05.692 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.683+0000 7facc16c5640 1 --2- 172.21.15.16:0/2508696056 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558dfa8fc370 0x558dfa8fe830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:08:05.692 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.683+0000 7facc26c7640 1 --2- 172.21.15.16:0/1775357546 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7faca80a2cb0 0x7faca80a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:05.693 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.683+0000 7facb2ffd640 1 -- 172.21.15.16:0/1775357546 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7facac0ef4c0 con 0x7facbc101c50 2024-09-12T02:08:05.693 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.683+0000 7facc1ec6640 1 --2- 172.21.15.16:0/2508696056 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558dfa8f72e0 0x558dfa8fbda0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7facb8009ca0 tx=0x7facb8011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=397def7f0ff5f5fb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:05.693 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.683+0000 7faca7fff640 1 -- 172.21.15.16:0/2508696056 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facb8004220 con 0x558dfa8f72e0 2024-09-12T02:08:05.693 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.683+0000 7faca7fff640 1 -- 172.21.15.16:0/2508696056 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7facb80043c0 con 0x558dfa8f72e0 2024-09-12T02:08:05.693 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.683+0000 7faca7fff640 1 -- 172.21.15.16:0/2508696056 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facb8019970 con 0x558dfa8f72e0 2024-09-12T02:08:05.693 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.684+0000 7facc6ec7740 1 -- 172.21.15.16:0/2508696056 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558dfa8f72e0 msgr2=0x558dfa8fbda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:05.693 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.684+0000 7facc26c7640 1 --2- 172.21.15.16:0/1775357546 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7faca80a2cb0 0x7faca80a5170 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7facb4009270 tx=0x7facb4004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:05.693 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.684+0000 7facc6ec7740 1 --2- 172.21.15.16:0/2508696056 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558dfa8f72e0 0x558dfa8fbda0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7facb8009ca0 tx=0x7facb8011810 comp rx=0 tx=0).stop 2024-09-12T02:08:05.693 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.684+0000 7facc6ec7740 1 -- 172.21.15.16:0/2508696056 shutdown_connections 2024-09-12T02:08:05.693 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.684+0000 7facc6ec7740 1 --2- 172.21.15.16:0/2508696056 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558dfa8fc370 0x558dfa8fe830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:05.693 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.684+0000 7facc6ec7740 1 --2- 172.21.15.16:0/2508696056 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558dfa8f72e0 0x558dfa8fbda0 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:05.693 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.684+0000 7facc6ec7740 1 --2- 172.21.15.16:0/2508696056 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558dfa8f69a0 0x558dfa8f6da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:05.693 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.684+0000 7facc6ec7740 1 -- 172.21.15.16:0/2508696056 >> 172.21.15.16:0/2508696056 conn(0x558dfa8f2010 msgr2=0x558dfa8f2410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:05.693 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.684+0000 7facc6ec7740 1 -- 172.21.15.16:0/2508696056 shutdown_connections 2024-09-12T02:08:05.694 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.684+0000 7facc6ec7740 1 -- 172.21.15.16:0/2508696056 wait complete. 2024-09-12T02:08:05.694 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.685+0000 7facc6ec7740 1 -- start start 2024-09-12T02:08:05.694 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.685+0000 7facc6ec7740 1 Processor -- start 2024-09-12T02:08:05.694 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.685+0000 7facc6ec7740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558dfa8f69a0 0x558dfa900ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:05.694 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.685+0000 7facc6ec7740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558dfa8f72e0 0x558dfa900ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:05.694 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.685+0000 7facc6ec7740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558dfa8fc370 0x558dfa908030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:05.694 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.685+0000 7facc6ec7740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x558dfa8ff340 con 0x558dfa8fc370 2024-09-12T02:08:05.694 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.685+0000 7facc6ec7740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x558dfa7e3290 con 0x558dfa8f72e0 2024-09-12T02:08:05.694 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.685+0000 7facc6ec7740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x558dfa8ff4c0 con 0x558dfa8f69a0 2024-09-12T02:08:05.694 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.685+0000 7facc6ec7740 10 client.0 _ll_register_callbacks cb 0x558dfa814140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:08:05.694 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.685+0000 7facc26c7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558dfa8fc370 0x558dfa908030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:05.694 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.685+0000 7facc26c7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558dfa8fc370 0x558dfa908030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55482/0 (socket says 172.21.15.16:55482) 2024-09-12T02:08:05.694 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.685+0000 7facc26c7640 1 -- 172.21.15.16:0/846219937 learned_addr learned my addr 172.21.15.16:0/846219937 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:08:05.695 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.685+0000 7facc16c5640 1 --2- 172.21.15.16:0/846219937 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558dfa8f72e0 0x558dfa900ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:05.695 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.685+0000 7facc1ec6640 1 --2- 172.21.15.16:0/846219937 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558dfa8f69a0 0x558dfa900ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:05.695 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.685+0000 7facc26c7640 1 -- 172.21.15.16:0/846219937 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558dfa8f69a0 msgr2=0x558dfa900ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:05.695 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.685+0000 7facc26c7640 1 --2- 172.21.15.16:0/846219937 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558dfa8f69a0 0x558dfa900ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:05.695 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.686+0000 7facc26c7640 1 -- 172.21.15.16:0/846219937 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558dfa8f72e0 msgr2=0x558dfa900ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:05.695 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.686+0000 7facc26c7640 1 --2- 172.21.15.16:0/846219937 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558dfa8f72e0 0x558dfa900ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:05.695 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.686+0000 7facc26c7640 1 -- 172.21.15.16:0/846219937 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558dfa56d210 con 0x558dfa8fc370 2024-09-12T02:08:05.695 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.686+0000 7facc1ec6640 1 --2- 172.21.15.16:0/846219937 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558dfa8f69a0 0x558dfa900ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:08:05.695 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.686+0000 7facc26c7640 1 --2- 172.21.15.16:0/846219937 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558dfa8fc370 0x558dfa908030 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7facb4012780 tx=0x7facb4004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac11c445d4954bed server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:05.695 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.686+0000 7faca57fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:08:05.695 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.686+0000 7faca57fa640 1 -- 172.21.15.16:0/846219937 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facb4022070 con 0x558dfa8fc370 2024-09-12T02:08:05.695 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.686+0000 7facc6ec7740 1 -- 172.21.15.16:0/846219937 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7facb80043c0 con 0x558dfa8fc370 2024-09-12T02:08:05.695 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.686+0000 7faca57fa640 1 -- 172.21.15.16:0/846219937 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7facb4011b60 con 0x558dfa8fc370 2024-09-12T02:08:05.695 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.686+0000 7faca57fa640 1 -- 172.21.15.16:0/846219937 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7facb402e050 con 0x558dfa8fc370 2024-09-12T02:08:05.696 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.686+0000 7facc6ec7740 10 client.17319 fetch_fsmap learned FSMap version 25 2024-09-12T02:08:05.696 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.686+0000 7facc6ec7740 1 -- 172.21.15.16:0/846219937 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x558dfa639620 con 0x558dfa8fc370 2024-09-12T02:08:05.696 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.686+0000 7faca57fa640 1 -- 172.21.15.16:0/846219937 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7facb80043c0 con 0x558dfa8fc370 2024-09-12T02:08:05.696 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.686+0000 7faca57fa640 1 -- 172.21.15.16:0/846219937 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7facb401a070 con 0x558dfa8fc370 2024-09-12T02:08:05.696 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.687+0000 7facc6ec7740 10 client.17319 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:08:05.696 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.687+0000 7facc6ec7740 10 client.17319 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:08:05.696 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.687+0000 7facc6ec7740 20 client.17319 populate_metadata read hostname 'smithi016' 2024-09-12T02:08:05.696 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.687+0000 7facc6ec7740 1 --2- 172.21.15.16:0/846219937 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x558dfa90c6f0 0x558dfa90eb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:05.696 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.687+0000 7facc6ec7740 4 client.17319 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:08:05.696 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.687+0000 7facc6ec7740 1 -- 172.21.15.16:0/846219937 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558dfa867a20 con 0x558dfa90c6f0 2024-09-12T02:08:05.696 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.687+0000 7facc1ec6640 1 --2- 172.21.15.16:0/846219937 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x558dfa90c6f0 0x558dfa90eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:05.696 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.687+0000 7faca57fa640 10 client.17319 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:08:05.696 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.687+0000 7facc1ec6640 1 --2- 172.21.15.16:0/846219937 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x558dfa90c6f0 0x558dfa90eb80 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b89af6c9a9177ef1 server_cookie=ce7781f0560d054b in_seq=0 out_seq=0 2024-09-12T02:08:05.697 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.688+0000 7faca57fa640 1 -- 172.21.15.16:0/846219937 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558dfa867a20 con 0x558dfa90c6f0 2024-09-12T02:08:05.697 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.688+0000 7faca57fa640 10 client.17319 handle_command_reply: tid=0 2024-09-12T02:08:05.697 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.688+0000 7facc6ec7740 1 client.17319 shutdown 2024-09-12T02:08:05.697 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.688+0000 7facc6ec7740 1 -- 172.21.15.16:0/846219937 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558dfa8fc370 msgr2=0x558dfa908030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:05.697 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.688+0000 7facc6ec7740 1 --2- 172.21.15.16:0/846219937 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558dfa8fc370 0x558dfa908030 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7facb4012780 tx=0x7facb4004ab0 comp rx=0 tx=0).stop 2024-09-12T02:08:05.697 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.688+0000 7facc6ec7740 1 -- 172.21.15.16:0/846219937 shutdown_connections 2024-09-12T02:08:05.697 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.688+0000 7facc6ec7740 1 --2- 172.21.15.16:0/846219937 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x558dfa90c6f0 0x558dfa90eb80 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:05.697 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.688+0000 7facc6ec7740 1 --2- 172.21.15.16:0/846219937 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558dfa8fc370 0x558dfa908030 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:05.697 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.688+0000 7facc6ec7740 1 --2- 172.21.15.16:0/846219937 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558dfa8f72e0 0x558dfa900ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:05.697 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.688+0000 7facc6ec7740 1 --2- 172.21.15.16:0/846219937 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558dfa8f69a0 0x558dfa900ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:05.698 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.688+0000 7faca57fa640 0 client.17319 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:08:05.698 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.688+0000 7facc6ec7740 1 -- 172.21.15.16:0/846219937 >> 172.21.15.16:0/846219937 conn(0x558dfa8f2010 msgr2=0x558dfa8f3820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:05.698 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.688+0000 7facc6ec7740 1 -- 172.21.15.16:0/846219937 shutdown_connections 2024-09-12T02:08:05.698 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.688+0000 7facc6ec7740 1 -- 172.21.15.16:0/846219937 wait complete. 2024-09-12T02:08:05.698 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.688+0000 7facc6ec7740 20 client.17319 trim_cache size 0 max 16384 2024-09-12T02:08:05.708 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.703+0000 7facc6ec7740 1 Processor -- start 2024-09-12T02:08:05.708 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.703+0000 7facc6ec7740 1 -- start start 2024-09-12T02:08:05.708 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.703+0000 7facc6ec7740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558dfa8f69a0 0x558dfa8f10b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:05.708 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.703+0000 7facc6ec7740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558dfa8f72e0 0x558dfa907f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:05.708 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.703+0000 7facc6ec7740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558dfa8fc370 0x558dfa913480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:05.708 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.703+0000 7facc6ec7740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7facbc1ab6e0 con 0x558dfa8f69a0 2024-09-12T02:08:05.708 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.703+0000 7facc6ec7740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7facbc1a8fb0 con 0x558dfa8f72e0 2024-09-12T02:08:05.708 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.703+0000 7facc6ec7740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7facbc1a6e00 con 0x558dfa8fc370 2024-09-12T02:08:05.708 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.703+0000 7facc1ec6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558dfa8f69a0 0x558dfa8f10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:05.709 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.703+0000 7facc1ec6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558dfa8f69a0 0x558dfa8f10b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55496/0 (socket says 172.21.15.16:55496) 2024-09-12T02:08:05.709 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.703+0000 7facc1ec6640 1 -- 172.21.15.16:0/3507989863 learned_addr learned my addr 172.21.15.16:0/3507989863 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:08:05.709 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.703+0000 7facc26c7640 1 --2- 172.21.15.16:0/3507989863 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558dfa8fc370 0x558dfa913480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:05.709 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.703+0000 7facc1ec6640 1 -- 172.21.15.16:0/3507989863 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558dfa8fc370 msgr2=0x558dfa913480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:05.709 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.703+0000 7facc16c5640 1 --2- 172.21.15.16:0/3507989863 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558dfa8f72e0 0x558dfa907f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:05.709 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.703+0000 7facc1ec6640 1 --2- 172.21.15.16:0/3507989863 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558dfa8fc370 0x558dfa913480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:05.709 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.703+0000 7facc1ec6640 1 -- 172.21.15.16:0/3507989863 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558dfa8f72e0 msgr2=0x558dfa907f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:05.709 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.703+0000 7facc1ec6640 1 --2- 172.21.15.16:0/3507989863 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558dfa8f72e0 0x558dfa907f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:05.709 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.703+0000 7facc1ec6640 1 -- 172.21.15.16:0/3507989863 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558dfa8f77c0 con 0x558dfa8f69a0 2024-09-12T02:08:05.709 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.703+0000 7facc26c7640 1 --2- 172.21.15.16:0/3507989863 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558dfa8fc370 0x558dfa913480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:08:05.709 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.703+0000 7facc16c5640 1 --2- 172.21.15.16:0/3507989863 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558dfa8f72e0 0x558dfa907f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:08:05.709 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.703+0000 7facc1ec6640 1 --2- 172.21.15.16:0/3507989863 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558dfa8f69a0 0x558dfa8f10b0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7facb80040a0 tx=0x7facb8009660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a87c05b42ac77c0b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:05.709 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.703+0000 7faca77fe640 1 -- 172.21.15.16:0/3507989863 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facb8005230 con 0x558dfa8f69a0 2024-09-12T02:08:05.709 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.703+0000 7faca77fe640 1 -- 172.21.15.16:0/3507989863 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7facb8003ad0 con 0x558dfa8f69a0 2024-09-12T02:08:05.710 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.703+0000 7faca77fe640 1 -- 172.21.15.16:0/3507989863 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facb80197b0 con 0x558dfa8f69a0 2024-09-12T02:08:05.710 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.704+0000 7facc6ec7740 1 -- 172.21.15.16:0/3507989863 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558dfa8f69a0 msgr2=0x558dfa8f10b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:05.710 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.704+0000 7facc6ec7740 1 --2- 172.21.15.16:0/3507989863 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558dfa8f69a0 0x558dfa8f10b0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7facb80040a0 tx=0x7facb8009660 comp rx=0 tx=0).stop 2024-09-12T02:08:05.710 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.704+0000 7facc6ec7740 1 -- 172.21.15.16:0/3507989863 shutdown_connections 2024-09-12T02:08:05.710 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.704+0000 7facc6ec7740 1 --2- 172.21.15.16:0/3507989863 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558dfa8fc370 0x558dfa913480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:05.710 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.704+0000 7facc6ec7740 1 --2- 172.21.15.16:0/3507989863 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558dfa8f72e0 0x558dfa907f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:05.710 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.704+0000 7facc6ec7740 1 --2- 172.21.15.16:0/3507989863 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558dfa8f69a0 0x558dfa8f10b0 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:05.710 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.704+0000 7facc6ec7740 1 -- 172.21.15.16:0/3507989863 >> 172.21.15.16:0/3507989863 conn(0x558dfa8f2010 msgr2=0x558dfa8f09a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:05.710 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.704+0000 7facc6ec7740 1 -- 172.21.15.16:0/3507989863 shutdown_connections 2024-09-12T02:08:05.710 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.704+0000 7facc6ec7740 1 -- 172.21.15.16:0/3507989863 wait complete. 2024-09-12T02:08:05.710 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.704+0000 7facc6ec7740 1 -- start start 2024-09-12T02:08:05.710 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.704+0000 7facc6ec7740 1 Processor -- start 2024-09-12T02:08:05.710 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.705+0000 7facc6ec7740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558dfa8f69a0 0x558dfa907890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:05.711 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.705+0000 7facc6ec7740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558dfa8f72e0 0x558dfa907dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:05.711 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.705+0000 7facc6ec7740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558dfa8fc370 0x558dfa905ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:05.711 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.705+0000 7facc6ec7740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7facbc1a7470 con 0x558dfa8f69a0 2024-09-12T02:08:05.711 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.705+0000 7facc6ec7740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7facbc1a88e0 con 0x558dfa8f72e0 2024-09-12T02:08:05.711 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.705+0000 7facc6ec7740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7facbc1a7b40 con 0x558dfa8fc370 2024-09-12T02:08:05.711 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.705+0000 7facc6ec7740 10 client.0 _ll_register_callbacks cb 0x7facb4009b90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:08:05.711 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.705+0000 7facc1ec6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558dfa8f69a0 0x558dfa907890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:05.711 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.705+0000 7facc1ec6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558dfa8f69a0 0x558dfa907890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55508/0 (socket says 172.21.15.16:55508) 2024-09-12T02:08:05.711 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.705+0000 7facc1ec6640 1 -- 172.21.15.16:0/906688018 learned_addr learned my addr 172.21.15.16:0/906688018 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:08:05.711 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.705+0000 7facc16c5640 1 --2- 172.21.15.16:0/906688018 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558dfa8f72e0 0x558dfa907dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:05.711 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.705+0000 7facc1ec6640 1 -- 172.21.15.16:0/906688018 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558dfa8fc370 msgr2=0x558dfa905ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:05.711 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.705+0000 7facc1ec6640 1 --2- 172.21.15.16:0/906688018 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558dfa8fc370 0x558dfa905ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:05.711 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.705+0000 7facc1ec6640 1 -- 172.21.15.16:0/906688018 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558dfa8f72e0 msgr2=0x558dfa907dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:05.712 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.705+0000 7facc1ec6640 1 --2- 172.21.15.16:0/906688018 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558dfa8f72e0 0x558dfa907dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:05.712 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.705+0000 7facc1ec6640 1 -- 172.21.15.16:0/906688018 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558dfa906630 con 0x558dfa8f69a0 2024-09-12T02:08:05.712 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.705+0000 7facc16c5640 1 --2- 172.21.15.16:0/906688018 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558dfa8f72e0 0x558dfa907dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:08:05.712 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.705+0000 7facc1ec6640 1 --2- 172.21.15.16:0/906688018 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558dfa8f69a0 0x558dfa907890 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7facb8009bc0 tx=0x7facb8009f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24f74eca3ccecd9b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:05.712 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.705+0000 7faca57fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:08:05.712 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.705+0000 7faca57fa640 1 -- 172.21.15.16:0/906688018 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facb8037020 con 0x558dfa8f69a0 2024-09-12T02:08:05.712 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.705+0000 7faca57fa640 1 -- 172.21.15.16:0/906688018 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7facb8003800 con 0x558dfa8f69a0 2024-09-12T02:08:05.712 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.706+0000 7facc6ec7740 1 -- 172.21.15.16:0/906688018 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7facb8003ad0 con 0x558dfa8f69a0 2024-09-12T02:08:05.712 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.706+0000 7faca57fa640 1 -- 172.21.15.16:0/906688018 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7facb8005250 con 0x558dfa8f69a0 2024-09-12T02:08:05.713 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.706+0000 7facc6ec7740 10 client.17325 fetch_fsmap learned FSMap version 25 2024-09-12T02:08:05.713 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.706+0000 7facc6ec7740 1 -- 172.21.15.16:0/906688018 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x558dfa977e60 con 0x558dfa8f69a0 2024-09-12T02:08:05.713 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.706+0000 7faca57fa640 1 -- 172.21.15.16:0/906688018 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7facb8003ad0 con 0x558dfa8f69a0 2024-09-12T02:08:05.713 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.706+0000 7faca57fa640 1 -- 172.21.15.16:0/906688018 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7facb8033420 con 0x558dfa8f69a0 2024-09-12T02:08:05.713 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.706+0000 7facc6ec7740 10 client.17325 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:08:05.713 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.706+0000 7facc6ec7740 10 client.17325 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:08:05.713 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.706+0000 7facc6ec7740 20 client.17325 populate_metadata read hostname 'smithi016' 2024-09-12T02:08:05.713 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.706+0000 7facc6ec7740 1 --2- 172.21.15.16:0/906688018 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x558dfa978750 0x558dfa97ab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:05.713 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.706+0000 7facc6ec7740 4 client.17325 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:08:05.713 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.706+0000 7facc6ec7740 1 -- 172.21.15.16:0/906688018 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558dfa861d70 con 0x558dfa978750 2024-09-12T02:08:05.713 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.706+0000 7facc16c5640 1 --2- 172.21.15.16:0/906688018 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x558dfa978750 0x558dfa97ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:05.714 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.707+0000 7faca57fa640 10 client.17325 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:08:05.714 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.707+0000 7facc16c5640 1 --2- 172.21.15.16:0/906688018 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x558dfa978750 0x558dfa97ab70 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d552c6d585b02585 server_cookie=211e823c2f76fe9d in_seq=0 out_seq=0 2024-09-12T02:08:05.721 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.719+0000 7faca57fa640 1 -- 172.21.15.16:0/906688018 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558dfa861d70 con 0x558dfa978750 2024-09-12T02:08:05.721 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.719+0000 7faca57fa640 10 client.17325 handle_command_reply: tid=0 2024-09-12T02:08:05.721 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.719+0000 7facc6ec7740 1 client.17325 shutdown 2024-09-12T02:08:05.721 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.720+0000 7facc6ec7740 1 -- 172.21.15.16:0/906688018 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558dfa8f69a0 msgr2=0x558dfa907890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:05.721 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.720+0000 7facc6ec7740 1 --2- 172.21.15.16:0/906688018 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558dfa8f69a0 0x558dfa907890 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7facb8009bc0 tx=0x7facb8009f70 comp rx=0 tx=0).stop 2024-09-12T02:08:05.721 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.720+0000 7facc6ec7740 1 -- 172.21.15.16:0/906688018 shutdown_connections 2024-09-12T02:08:05.722 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.720+0000 7facc6ec7740 1 --2- 172.21.15.16:0/906688018 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x558dfa978750 0x558dfa97ab70 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:05.722 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.720+0000 7facc6ec7740 1 --2- 172.21.15.16:0/906688018 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558dfa8fc370 0x558dfa905ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:05.722 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.720+0000 7facc6ec7740 1 --2- 172.21.15.16:0/906688018 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558dfa8f72e0 0x558dfa907dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:05.722 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.720+0000 7facc6ec7740 1 --2- 172.21.15.16:0/906688018 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558dfa8f69a0 0x558dfa907890 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:05.722 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.720+0000 7faca57fa640 0 client.17325 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:08:05.722 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.720+0000 7facc6ec7740 1 -- 172.21.15.16:0/906688018 >> 172.21.15.16:0/906688018 conn(0x558dfa8f2010 msgr2=0x558dfa8f11c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:05.722 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.720+0000 7facc6ec7740 1 -- 172.21.15.16:0/906688018 shutdown_connections 2024-09-12T02:08:05.722 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.720+0000 7facc6ec7740 1 -- 172.21.15.16:0/906688018 wait complete. 2024-09-12T02:08:05.722 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.720+0000 7facc6ec7740 20 client.17325 trim_cache size 0 max 16384 2024-09-12T02:08:05.723 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-09-12T02:08:05.723 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-09-12T02:08:05.723 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-09-12T02:08:05.723 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-09-12T02:08:05.724 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.720+0000 7facc412a640 1 -- 172.21.15.16:0/1775357546 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7faca80a2cb0 msgr2=0x7faca80a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:05.724 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.720+0000 7facc412a640 1 --2- 172.21.15.16:0/1775357546 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7faca80a2cb0 0x7faca80a5170 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7facb4009270 tx=0x7facb4004a40 comp rx=0 tx=0).stop 2024-09-12T02:08:05.724 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.720+0000 7facc412a640 1 -- 172.21.15.16:0/1775357546 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facbc101c50 msgr2=0x7facbc19cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:05.724 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.720+0000 7facc412a640 1 --2- 172.21.15.16:0/1775357546 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facbc101c50 0x7facbc19cb00 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7facac009230 tx=0x7facac031b60 comp rx=0 tx=0).stop 2024-09-12T02:08:05.724 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.720+0000 7facc412a640 1 -- 172.21.15.16:0/1775357546 shutdown_connections 2024-09-12T02:08:05.724 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.720+0000 7facc412a640 1 --2- 172.21.15.16:0/1775357546 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7faca80a2cb0 0x7faca80a5170 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:05.725 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.720+0000 7facc412a640 1 --2- 172.21.15.16:0/1775357546 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7facbc102610 0x7facbc19f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:05.725 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.720+0000 7facc412a640 1 --2- 172.21.15.16:0/1775357546 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facbc101c50 0x7facbc19cb00 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:05.725 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.720+0000 7facc412a640 1 --2- 172.21.15.16:0/1775357546 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7facbc100a60 0x7facbc06ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:05.725 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.720+0000 7facc412a640 1 -- 172.21.15.16:0/1775357546 >> 172.21.15.16:0/1775357546 conn(0x7facbc0fc1e0 msgr2=0x7facbc104f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:05.725 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.721+0000 7facc412a640 1 -- 172.21.15.16:0/1775357546 shutdown_connections 2024-09-12T02:08:05.725 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:05.721+0000 7facc412a640 1 -- 172.21.15.16:0/1775357546 wait complete. 2024-09-12T02:08:05.848 INFO:tasks.cephfs.filesystem:scrub status for tag:0636b809-e471-4db7-afa8-75f0219b5288 - {'path': '/', 'tag': '0636b809-e471-4db7-afa8-75f0219b5288', 'options': 'recursive,force'} 2024-09-12T02:08:06.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:08:06 smithi184 ceph-mon[35593]: pgmap v520: 97 pgs: 97 active+clean; 3.3 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-12T02:08:06.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:08:06 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:08:06.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:08:06 smithi033 ceph-mon[35332]: pgmap v520: 97 pgs: 97 active+clean; 3.3 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-12T02:08:06.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:08:06 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:08:06.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:08:06 smithi016 ceph-mon[30947]: pgmap v520: 97 pgs: 97 active+clean; 3.3 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-12T02:08:06.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:08:06 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:08:07.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:08:07 smithi184 ceph-mon[35593]: scrub summary: idle 2024-09-12T02:08:07.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:08:07 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:08:07.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:08:07 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:08:07.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:08:07 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T02:08:07.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:08:07 smithi033 ceph-mon[35332]: scrub summary: idle 2024-09-12T02:08:07.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:08:07 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:08:07.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:08:07 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:08:07.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:08:07 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T02:08:07.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:08:07 smithi016 ceph-mon[30947]: scrub summary: idle 2024-09-12T02:08:07.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:08:07 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:08:07.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:08:07 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:08:07.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:08:07 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T02:08:08.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:08:08 smithi184 ceph-mon[35593]: pgmap v521: 97 pgs: 97 active+clean; 3.3 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-12T02:08:08.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:08:08 smithi033 ceph-mon[35332]: pgmap v521: 97 pgs: 97 active+clean; 3.3 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-12T02:08:08.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:08:08 smithi016 ceph-mon[30947]: pgmap v521: 97 pgs: 97 active+clean; 3.3 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-12T02:08:10.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:08:10 smithi184 ceph-mon[35593]: pgmap v522: 97 pgs: 97 active+clean; 3.3 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T02:08:10.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:08:10 smithi033 ceph-mon[35332]: pgmap v522: 97 pgs: 97 active+clean; 3.3 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T02:08:10.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:08:10 smithi016 ceph-mon[30947]: pgmap v522: 97 pgs: 97 active+clean; 3.3 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T02:08:12.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:08:12 smithi184 ceph-mon[35593]: pgmap v523: 97 pgs: 97 active+clean; 3.3 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 17 op/s 2024-09-12T02:08:12.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:08:12 smithi033 ceph-mon[35332]: pgmap v523: 97 pgs: 97 active+clean; 3.3 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 17 op/s 2024-09-12T02:08:12.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:08:12 smithi016 ceph-mon[30947]: pgmap v523: 97 pgs: 97 active+clean; 3.3 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 17 op/s 2024-09-12T02:08:14.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:08:14 smithi184 ceph-mon[35593]: pgmap v524: 97 pgs: 97 active+clean; 3.3 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-09-12T02:08:14.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:08:14 smithi033 ceph-mon[35332]: pgmap v524: 97 pgs: 97 active+clean; 3.3 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-09-12T02:08:14.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:08:14 smithi016 ceph-mon[30947]: pgmap v524: 97 pgs: 97 active+clean; 3.3 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-09-12T02:08:16.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:08:16 smithi184 ceph-mon[35593]: pgmap v525: 97 pgs: 97 active+clean; 3.4 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-09-12T02:08:16.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:08:16 smithi033 ceph-mon[35332]: pgmap v525: 97 pgs: 97 active+clean; 3.4 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-09-12T02:08:16.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:08:16 smithi016 ceph-mon[30947]: pgmap v525: 97 pgs: 97 active+clean; 3.4 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-09-12T02:08:18.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:08:18 smithi184 ceph-mon[35593]: pgmap v526: 97 pgs: 97 active+clean; 3.4 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-09-12T02:08:18.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:08:18 smithi033 ceph-mon[35332]: pgmap v526: 97 pgs: 97 active+clean; 3.4 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-09-12T02:08:18.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:08:18 smithi016 ceph-mon[30947]: pgmap v526: 97 pgs: 97 active+clean; 3.4 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-09-12T02:08:18.841 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.823+0000 7f8c72aae640 1 -- 172.21.15.16:0/3078469686 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c6c104f10 msgr2=0x7f8c6c107300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:18.842 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.823+0000 7f8c72aae640 1 --2- 172.21.15.16:0/3078469686 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c6c104f10 0x7f8c6c107300 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f8c58009900 tx=0x7f8c5802a410 comp rx=0 tx=0).stop 2024-09-12T02:08:18.842 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.823+0000 7f8c72aae640 1 -- 172.21.15.16:0/3078469686 shutdown_connections 2024-09-12T02:08:18.842 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.823+0000 7f8c72aae640 1 --2- 172.21.15.16:0/3078469686 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8c6c107840 0x7f8c6c109cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:18.842 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.823+0000 7f8c72aae640 1 --2- 172.21.15.16:0/3078469686 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c6c104f10 0x7f8c6c107300 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:18.843 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.823+0000 7f8c72aae640 1 --2- 172.21.15.16:0/3078469686 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c6c1025e0 0x7f8c6c1049d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:18.843 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.823+0000 7f8c72aae640 1 -- 172.21.15.16:0/3078469686 >> 172.21.15.16:0/3078469686 conn(0x7f8c6c0fc1e0 msgr2=0x7f8c6c0fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:18.843 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.823+0000 7f8c72aae640 1 -- 172.21.15.16:0/3078469686 shutdown_connections 2024-09-12T02:08:18.843 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.823+0000 7f8c72aae640 1 -- 172.21.15.16:0/3078469686 wait complete. 2024-09-12T02:08:18.843 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.823+0000 7f8c72aae640 1 Processor -- start 2024-09-12T02:08:18.844 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.823+0000 7f8c72aae640 1 -- start start 2024-09-12T02:08:18.844 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.824+0000 7f8c72aae640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c6c1025e0 0x7f8c6c197e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:18.844 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.824+0000 7f8c72aae640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c6c104f10 0x7f8c6c198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:18.844 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.824+0000 7f8c72aae640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8c6c107840 0x7f8c6c19f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:18.844 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.824+0000 7f8c72aae640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8c6c10d820 con 0x7f8c6c1025e0 2024-09-12T02:08:18.844 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.824+0000 7f8c72aae640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8c6c10d6a0 con 0x7f8c6c104f10 2024-09-12T02:08:18.844 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.824+0000 7f8c72aae640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8c6c10d9a0 con 0x7f8c6c107840 2024-09-12T02:08:18.845 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.824+0000 7f8c7084a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c6c1025e0 0x7f8c6c197e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:18.845 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.824+0000 7f8c7084a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c6c1025e0 0x7f8c6c197e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55166/0 (socket says 172.21.15.16:55166) 2024-09-12T02:08:18.845 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.824+0000 7f8c7084a640 1 -- 172.21.15.16:0/4107771565 learned_addr learned my addr 172.21.15.16:0/4107771565 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:08:18.845 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.824+0000 7f8c6bfff640 1 --2- 172.21.15.16:0/4107771565 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c6c104f10 0x7f8c6c198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:18.845 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.824+0000 7f8c7104b640 1 --2- 172.21.15.16:0/4107771565 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8c6c107840 0x7f8c6c19f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:18.845 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.824+0000 7f8c7084a640 1 -- 172.21.15.16:0/4107771565 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8c6c107840 msgr2=0x7f8c6c19f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:18.845 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.824+0000 7f8c7084a640 1 --2- 172.21.15.16:0/4107771565 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8c6c107840 0x7f8c6c19f400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:18.845 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.824+0000 7f8c7084a640 1 -- 172.21.15.16:0/4107771565 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c6c104f10 msgr2=0x7f8c6c198390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:18.845 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.824+0000 7f8c7084a640 1 --2- 172.21.15.16:0/4107771565 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c6c104f10 0x7f8c6c198390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:18.845 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.824+0000 7f8c7084a640 1 -- 172.21.15.16:0/4107771565 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c6c19fa70 con 0x7f8c6c1025e0 2024-09-12T02:08:18.845 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.824+0000 7f8c7104b640 1 --2- 172.21.15.16:0/4107771565 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8c6c107840 0x7f8c6c19f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:08:18.845 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.824+0000 7f8c7084a640 1 --2- 172.21.15.16:0/4107771565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c6c1025e0 0x7f8c6c197e50 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f8c6000d950 tx=0x7f8c6000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:18.845 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.824+0000 7f8c69ffb640 1 -- 172.21.15.16:0/4107771565 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c60017070 con 0x7f8c6c1025e0 2024-09-12T02:08:18.846 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.824+0000 7f8c69ffb640 1 -- 172.21.15.16:0/4107771565 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c60009c30 con 0x7f8c6c1025e0 2024-09-12T02:08:18.846 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.825+0000 7f8c72aae640 1 -- 172.21.15.16:0/4107771565 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c6c19fd60 con 0x7f8c6c1025e0 2024-09-12T02:08:18.846 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.825+0000 7f8c69ffb640 1 -- 172.21.15.16:0/4107771565 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c6000f6b0 con 0x7f8c6c1025e0 2024-09-12T02:08:18.846 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.825+0000 7f8c72aae640 1 -- 172.21.15.16:0/4107771565 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c6c1a0210 con 0x7f8c6c1025e0 2024-09-12T02:08:18.846 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.827+0000 7f8c69ffb640 1 -- 172.21.15.16:0/4107771565 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f8c600109d0 con 0x7f8c6c1025e0 2024-09-12T02:08:18.846 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.827+0000 7f8c75849740 1 Processor -- start 2024-09-12T02:08:18.846 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.828+0000 7f8c75849740 1 -- start start 2024-09-12T02:08:18.846 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.828+0000 7f8c75849740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5647f7a8cdb0 0x5647f7a8d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:18.846 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.828+0000 7f8c75849740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5647f7a8d6f0 0x5647f7a921b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:18.846 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.828+0000 7f8c75849740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5647f7a92780 0x5647f7a94c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:18.846 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.828+0000 7f8c75849740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5647f79ec2f0 con 0x5647f7a92780 2024-09-12T02:08:18.846 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.828+0000 7f8c75849740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5647f7994270 con 0x5647f7a8d6f0 2024-09-12T02:08:18.846 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.828+0000 7f8c75849740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5647f79e82b0 con 0x5647f7a8cdb0 2024-09-12T02:08:18.847 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.828+0000 7f8c69ffb640 1 --2- 172.21.15.16:0/4107771565 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f8c5c0a2cb0 0x7f8c5c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:18.847 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.828+0000 7f8c6bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5647f7a92780 0x5647f7a94c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:18.847 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.828+0000 7f8c6bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5647f7a92780 0x5647f7a94c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55170/0 (socket says 172.21.15.16:55170) 2024-09-12T02:08:18.847 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.828+0000 7f8c6bfff640 1 -- 172.21.15.16:0/2342198663 learned_addr learned my addr 172.21.15.16:0/2342198663 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:08:18.847 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.828+0000 7f8c69ffb640 1 -- 172.21.15.16:0/4107771565 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f8c600ceb30 con 0x7f8c6c1025e0 2024-09-12T02:08:18.847 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.828+0000 7f8c7104b640 1 --2- 172.21.15.16:0/4107771565 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f8c5c0a2cb0 0x7f8c5c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:18.847 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.828+0000 7f8c7084a640 1 --2- 172.21.15.16:0/2342198663 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5647f7a8d6f0 0x5647f7a921b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:18.847 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.828+0000 7f8c7104b640 1 --2- 172.21.15.16:0/2342198663 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5647f7a8cdb0 0x5647f7a8d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:18.847 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.828+0000 7f8c6bfff640 1 -- 172.21.15.16:0/2342198663 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5647f7a8cdb0 msgr2=0x5647f7a8d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:18.847 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.828+0000 7f8c6bfff640 1 --2- 172.21.15.16:0/2342198663 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5647f7a8cdb0 0x5647f7a8d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:18.847 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.828+0000 7f8c6bfff640 1 -- 172.21.15.16:0/2342198663 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5647f7a8d6f0 msgr2=0x5647f7a921b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:18.847 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.828+0000 7f8c6bfff640 1 --2- 172.21.15.16:0/2342198663 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5647f7a8d6f0 0x5647f7a921b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:18.847 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.828+0000 7f8c6bfff640 1 -- 172.21.15.16:0/2342198663 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5647f7993340 con 0x5647f7a92780 2024-09-12T02:08:18.848 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.829+0000 7f8c6bfff640 1 --2- 172.21.15.16:0/2342198663 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5647f7a92780 0x5647f7a94c40 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f8c58031610 tx=0x7f8c58005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c74d66fe3819ace server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:18.848 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.829+0000 7f8c7104b640 1 --2- 172.21.15.16:0/4107771565 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f8c5c0a2cb0 0x7f8c5c0a5170 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f8c64009fd0 tx=0x7f8c64009530 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:18.848 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.829+0000 7f8c56ffd640 1 -- 172.21.15.16:0/2342198663 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c58040070 con 0x5647f7a92780 2024-09-12T02:08:18.848 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.829+0000 7f8c56ffd640 1 -- 172.21.15.16:0/2342198663 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c580318b0 con 0x5647f7a92780 2024-09-12T02:08:18.848 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.829+0000 7f8c56ffd640 1 -- 172.21.15.16:0/2342198663 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c58035040 con 0x5647f7a92780 2024-09-12T02:08:18.848 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.830+0000 7f8c75849740 1 -- 172.21.15.16:0/2342198663 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5647f7a92780 msgr2=0x5647f7a94c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:18.848 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.830+0000 7f8c75849740 1 --2- 172.21.15.16:0/2342198663 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5647f7a92780 0x5647f7a94c40 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f8c58031610 tx=0x7f8c58005c00 comp rx=0 tx=0).stop 2024-09-12T02:08:18.848 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.830+0000 7f8c75849740 1 -- 172.21.15.16:0/2342198663 shutdown_connections 2024-09-12T02:08:18.848 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.830+0000 7f8c75849740 1 --2- 172.21.15.16:0/2342198663 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5647f7a92780 0x5647f7a94c40 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:18.848 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.830+0000 7f8c75849740 1 --2- 172.21.15.16:0/2342198663 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5647f7a8d6f0 0x5647f7a921b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:18.848 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.830+0000 7f8c75849740 1 --2- 172.21.15.16:0/2342198663 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5647f7a8cdb0 0x5647f7a8d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:18.848 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.830+0000 7f8c75849740 1 -- 172.21.15.16:0/2342198663 >> 172.21.15.16:0/2342198663 conn(0x5647f7a84010 msgr2=0x5647f7a84410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:18.849 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.830+0000 7f8c75849740 1 -- 172.21.15.16:0/2342198663 shutdown_connections 2024-09-12T02:08:18.849 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.830+0000 7f8c75849740 1 -- 172.21.15.16:0/2342198663 wait complete. 2024-09-12T02:08:18.849 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.830+0000 7f8c75849740 1 -- start start 2024-09-12T02:08:18.849 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.830+0000 7f8c75849740 1 Processor -- start 2024-09-12T02:08:18.849 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.831+0000 7f8c75849740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5647f7a8cdb0 0x5647f7a96f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:18.849 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.831+0000 7f8c75849740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5647f7a8d6f0 0x5647f7a97450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:18.849 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.831+0000 7f8c75849740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5647f7a92780 0x5647f7a9e490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:18.849 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.831+0000 7f8c75849740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5647f7a956f0 con 0x5647f7a92780 2024-09-12T02:08:18.849 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.831+0000 7f8c75849740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5647f7975290 con 0x5647f7a8cdb0 2024-09-12T02:08:18.849 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.831+0000 7f8c75849740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5647f7a95870 con 0x5647f7a8d6f0 2024-09-12T02:08:18.849 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.831+0000 7f8c75849740 10 client.0 _ll_register_callbacks cb 0x5647f79a6140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:08:18.849 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.831+0000 7f8c7104b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5647f7a92780 0x5647f7a9e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:18.849 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.831+0000 7f8c7104b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5647f7a92780 0x5647f7a9e490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55172/0 (socket says 172.21.15.16:55172) 2024-09-12T02:08:18.849 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.831+0000 7f8c7104b640 1 -- 172.21.15.16:0/2290959653 learned_addr learned my addr 172.21.15.16:0/2290959653 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:08:18.850 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.831+0000 7f8c7084a640 1 --2- 172.21.15.16:0/2290959653 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5647f7a8cdb0 0x5647f7a96f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:18.850 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.831+0000 7f8c6bfff640 1 --2- 172.21.15.16:0/2290959653 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5647f7a8d6f0 0x5647f7a97450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:18.850 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.831+0000 7f8c7104b640 1 -- 172.21.15.16:0/2290959653 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5647f7a8d6f0 msgr2=0x5647f7a97450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:18.850 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.831+0000 7f8c7104b640 1 --2- 172.21.15.16:0/2290959653 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5647f7a8d6f0 0x5647f7a97450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:18.850 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.831+0000 7f8c7104b640 1 -- 172.21.15.16:0/2290959653 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5647f7a8cdb0 msgr2=0x5647f7a96f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:18.850 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.831+0000 7f8c7104b640 1 --2- 172.21.15.16:0/2290959653 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5647f7a8cdb0 0x5647f7a96f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:18.850 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.831+0000 7f8c7104b640 1 -- 172.21.15.16:0/2290959653 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5647f76ff210 con 0x5647f7a92780 2024-09-12T02:08:18.850 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.831+0000 7f8c6bfff640 1 --2- 172.21.15.16:0/2290959653 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5647f7a8d6f0 0x5647f7a97450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:08:18.850 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.831+0000 7f8c7104b640 1 --2- 172.21.15.16:0/2290959653 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5647f7a92780 0x5647f7a9e490 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f8c6400f9a0 tx=0x7f8c64004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88193b8f40c7bb7b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:18.850 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.831+0000 7f8c4bfff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:08:18.850 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.831+0000 7f8c4bfff640 1 -- 172.21.15.16:0/2290959653 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c64024030 con 0x5647f7a92780 2024-09-12T02:08:18.850 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.831+0000 7f8c4bfff640 1 -- 172.21.15.16:0/2290959653 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f8c64012b80 con 0x5647f7a92780 2024-09-12T02:08:18.850 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.831+0000 7f8c75849740 1 -- 172.21.15.16:0/2290959653 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c580318b0 con 0x5647f7a92780 2024-09-12T02:08:18.851 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.832+0000 7f8c4bfff640 1 -- 172.21.15.16:0/2290959653 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f8c64012d80 con 0x5647f7a92780 2024-09-12T02:08:18.851 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.832+0000 7f8c75849740 10 client.17337 fetch_fsmap learned FSMap version 25 2024-09-12T02:08:18.851 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.832+0000 7f8c75849740 1 -- 172.21.15.16:0/2290959653 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5647f77cb620 con 0x5647f7a92780 2024-09-12T02:08:18.851 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.832+0000 7f8c4bfff640 1 -- 172.21.15.16:0/2290959653 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c580318b0 con 0x5647f7a92780 2024-09-12T02:08:18.851 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.832+0000 7f8c4bfff640 1 -- 172.21.15.16:0/2290959653 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f8c6401d050 con 0x5647f7a92780 2024-09-12T02:08:18.851 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.832+0000 7f8c75849740 10 client.17337 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:08:18.851 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.832+0000 7f8c75849740 10 client.17337 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:08:18.851 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.832+0000 7f8c75849740 20 client.17337 populate_metadata read hostname 'smithi016' 2024-09-12T02:08:18.851 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.832+0000 7f8c75849740 1 --2- 172.21.15.16:0/2290959653 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5647f7aa2ca0 0x5647f7aa50f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:18.851 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.832+0000 7f8c75849740 4 client.17337 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:08:18.851 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.832+0000 7f8c75849740 1 -- 172.21.15.16:0/2290959653 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5647f79f9a20 con 0x5647f7aa2ca0 2024-09-12T02:08:18.851 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.832+0000 7f8c7084a640 1 --2- 172.21.15.16:0/2290959653 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5647f7aa2ca0 0x5647f7aa50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:18.851 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.833+0000 7f8c4bfff640 10 client.17337 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:08:18.852 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.833+0000 7f8c7084a640 1 --2- 172.21.15.16:0/2290959653 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5647f7aa2ca0 0x5647f7aa50f0 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=541bb54bfdc9a8d7 server_cookie=24fec9b912e0a3b9 in_seq=0 out_seq=0 2024-09-12T02:08:18.852 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.833+0000 7f8c4bfff640 1 -- 172.21.15.16:0/2290959653 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5647f79f9a20 con 0x5647f7aa2ca0 2024-09-12T02:08:18.852 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.833+0000 7f8c4bfff640 10 client.17337 handle_command_reply: tid=0 2024-09-12T02:08:18.852 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.834+0000 7f8c75849740 1 client.17337 shutdown 2024-09-12T02:08:18.852 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.834+0000 7f8c75849740 1 -- 172.21.15.16:0/2290959653 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5647f7a92780 msgr2=0x5647f7a9e490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:18.852 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.834+0000 7f8c75849740 1 --2- 172.21.15.16:0/2290959653 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5647f7a92780 0x5647f7a9e490 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f8c6400f9a0 tx=0x7f8c64004ab0 comp rx=0 tx=0).stop 2024-09-12T02:08:18.852 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.834+0000 7f8c75849740 1 -- 172.21.15.16:0/2290959653 shutdown_connections 2024-09-12T02:08:18.852 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.834+0000 7f8c75849740 1 --2- 172.21.15.16:0/2290959653 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5647f7aa2ca0 0x5647f7aa50f0 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:18.852 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.834+0000 7f8c75849740 1 --2- 172.21.15.16:0/2290959653 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5647f7a92780 0x5647f7a9e490 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:18.852 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.834+0000 7f8c75849740 1 --2- 172.21.15.16:0/2290959653 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5647f7a8d6f0 0x5647f7a97450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:18.852 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.834+0000 7f8c75849740 1 --2- 172.21.15.16:0/2290959653 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5647f7a8cdb0 0x5647f7a96f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:18.852 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.834+0000 7f8c75849740 1 -- 172.21.15.16:0/2290959653 >> 172.21.15.16:0/2290959653 conn(0x5647f7a84010 msgr2=0x5647f7a85820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:18.853 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.834+0000 7f8c4bfff640 0 client.17337 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:08:18.853 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.834+0000 7f8c75849740 1 -- 172.21.15.16:0/2290959653 shutdown_connections 2024-09-12T02:08:18.853 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.834+0000 7f8c75849740 1 -- 172.21.15.16:0/2290959653 wait complete. 2024-09-12T02:08:18.853 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.834+0000 7f8c75849740 20 client.17337 trim_cache size 0 max 16384 2024-09-12T02:08:18.857 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.849+0000 7f8c75849740 1 Processor -- start 2024-09-12T02:08:18.857 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.849+0000 7f8c75849740 1 -- start start 2024-09-12T02:08:18.857 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.849+0000 7f8c75849740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5647f7a8cdb0 0x5647f7a84ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:18.857 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.849+0000 7f8c75849740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5647f7a8d6f0 0x5647f7a9e450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:18.857 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.849+0000 7f8c75849740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5647f7a92780 0x5647f7a85400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:18.857 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.849+0000 7f8c75849740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8c6c1a94e0 con 0x5647f7a8cdb0 2024-09-12T02:08:18.857 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.849+0000 7f8c75849740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8c6c1a6db0 con 0x5647f7a92780 2024-09-12T02:08:18.858 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.849+0000 7f8c75849740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8c6c1a4c00 con 0x5647f7a8d6f0 2024-09-12T02:08:18.858 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.849+0000 7f8c7084a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5647f7a8cdb0 0x5647f7a84ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:18.858 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.849+0000 7f8c7084a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5647f7a8cdb0 0x5647f7a84ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55174/0 (socket says 172.21.15.16:55174) 2024-09-12T02:08:18.858 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.849+0000 7f8c7084a640 1 -- 172.21.15.16:0/360371700 learned_addr learned my addr 172.21.15.16:0/360371700 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:08:18.858 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.849+0000 7f8c7084a640 1 -- 172.21.15.16:0/360371700 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5647f7a8d6f0 msgr2=0x5647f7a9e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:18.858 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.849+0000 7f8c7084a640 1 --2- 172.21.15.16:0/360371700 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5647f7a8d6f0 0x5647f7a9e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:18.858 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.849+0000 7f8c7084a640 1 -- 172.21.15.16:0/360371700 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5647f7a92780 msgr2=0x5647f7a85400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:18.858 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.849+0000 7f8c7084a640 1 --2- 172.21.15.16:0/360371700 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5647f7a92780 0x5647f7a85400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:18.858 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.849+0000 7f8c7084a640 1 -- 172.21.15.16:0/360371700 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5647f7a83e30 con 0x5647f7a8cdb0 2024-09-12T02:08:18.858 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.850+0000 7f8c7084a640 1 --2- 172.21.15.16:0/360371700 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5647f7a8cdb0 0x5647f7a84ec0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f8c60097890 tx=0x7f8c600cc5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a848a1a34fb7dbec server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:18.858 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.850+0000 7f8c567fc640 1 -- 172.21.15.16:0/360371700 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c60020040 con 0x5647f7a8cdb0 2024-09-12T02:08:18.858 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.850+0000 7f8c567fc640 1 -- 172.21.15.16:0/360371700 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c6009f7e0 con 0x5647f7a8cdb0 2024-09-12T02:08:18.858 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.850+0000 7f8c567fc640 1 -- 172.21.15.16:0/360371700 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c600a3c10 con 0x5647f7a8cdb0 2024-09-12T02:08:18.859 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.850+0000 7f8c75849740 1 -- 172.21.15.16:0/360371700 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5647f7a8cdb0 msgr2=0x5647f7a84ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:18.859 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.850+0000 7f8c75849740 1 --2- 172.21.15.16:0/360371700 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5647f7a8cdb0 0x5647f7a84ec0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f8c60097890 tx=0x7f8c600cc5a0 comp rx=0 tx=0).stop 2024-09-12T02:08:18.859 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.850+0000 7f8c75849740 1 -- 172.21.15.16:0/360371700 shutdown_connections 2024-09-12T02:08:18.859 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.850+0000 7f8c75849740 1 --2- 172.21.15.16:0/360371700 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5647f7a92780 0x5647f7a85400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:18.859 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.850+0000 7f8c75849740 1 --2- 172.21.15.16:0/360371700 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5647f7a8d6f0 0x5647f7a9e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:18.859 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.850+0000 7f8c75849740 1 --2- 172.21.15.16:0/360371700 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5647f7a8cdb0 0x5647f7a84ec0 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:18.859 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.850+0000 7f8c75849740 1 -- 172.21.15.16:0/360371700 >> 172.21.15.16:0/360371700 conn(0x5647f7a84010 msgr2=0x5647f7a96f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:18.859 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.850+0000 7f8c75849740 1 -- 172.21.15.16:0/360371700 shutdown_connections 2024-09-12T02:08:18.859 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.850+0000 7f8c75849740 1 -- 172.21.15.16:0/360371700 wait complete. 2024-09-12T02:08:18.859 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.851+0000 7f8c75849740 1 -- start start 2024-09-12T02:08:18.859 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.851+0000 7f8c75849740 1 Processor -- start 2024-09-12T02:08:18.859 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.851+0000 7f8c75849740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5647f7a8cdb0 0x5647f7a9ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:18.859 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.851+0000 7f8c75849740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5647f7a8d6f0 0x5647f7a9bf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:18.860 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.851+0000 7f8c75849740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5647f7a92780 0x5647f7a9cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:18.860 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.851+0000 7f8c75849740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8c6c1a5270 con 0x5647f7a8d6f0 2024-09-12T02:08:18.860 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.851+0000 7f8c75849740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8c6c1a66e0 con 0x5647f7a92780 2024-09-12T02:08:18.860 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.851+0000 7f8c75849740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8c6c1a5940 con 0x5647f7a8cdb0 2024-09-12T02:08:18.860 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.851+0000 7f8c75849740 10 client.0 _ll_register_callbacks cb 0x7f8c64009af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:08:18.860 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.851+0000 7f8c6bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5647f7a8d6f0 0x5647f7a9bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:18.860 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.851+0000 7f8c6bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5647f7a8d6f0 0x5647f7a9bf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55182/0 (socket says 172.21.15.16:55182) 2024-09-12T02:08:18.860 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.851+0000 7f8c6bfff640 1 -- 172.21.15.16:0/2575336704 learned_addr learned my addr 172.21.15.16:0/2575336704 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:08:18.860 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.851+0000 7f8c7104b640 1 --2- 172.21.15.16:0/2575336704 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5647f7a92780 0x5647f7a9cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:18.860 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.852+0000 7f8c7084a640 1 --2- 172.21.15.16:0/2575336704 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5647f7a8cdb0 0x5647f7a9ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:18.860 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.852+0000 7f8c6bfff640 1 -- 172.21.15.16:0/2575336704 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5647f7a8cdb0 msgr2=0x5647f7a9ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:18.860 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.852+0000 7f8c6bfff640 1 --2- 172.21.15.16:0/2575336704 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5647f7a8cdb0 0x5647f7a9ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:18.861 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.852+0000 7f8c6bfff640 1 -- 172.21.15.16:0/2575336704 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5647f7a92780 msgr2=0x5647f7a9cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:18.861 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.852+0000 7f8c6bfff640 1 --2- 172.21.15.16:0/2575336704 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5647f7a92780 0x5647f7a9cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:18.861 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.852+0000 7f8c6bfff640 1 -- 172.21.15.16:0/2575336704 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5647f7a9d5a0 con 0x5647f7a8d6f0 2024-09-12T02:08:18.861 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.852+0000 7f8c6bfff640 1 --2- 172.21.15.16:0/2575336704 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5647f7a8d6f0 0x5647f7a9bf80 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f8c58005ab0 tx=0x7f8c580318c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d8e022e362cda19 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:18.861 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.852+0000 7f8c4bfff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:08:18.861 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.852+0000 7f8c4bfff640 1 -- 172.21.15.16:0/2575336704 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c58040070 con 0x5647f7a8d6f0 2024-09-12T02:08:18.861 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.852+0000 7f8c4bfff640 1 -- 172.21.15.16:0/2575336704 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f8c58004450 con 0x5647f7a8d6f0 2024-09-12T02:08:18.861 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.852+0000 7f8c75849740 1 -- 172.21.15.16:0/2575336704 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c6009f7e0 con 0x5647f7a8d6f0 2024-09-12T02:08:18.861 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.852+0000 7f8c4bfff640 1 -- 172.21.15.16:0/2575336704 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f8c58035040 con 0x5647f7a8d6f0 2024-09-12T02:08:18.861 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.852+0000 7f8c75849740 10 client.17343 fetch_fsmap learned FSMap version 25 2024-09-12T02:08:18.861 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.852+0000 7f8c75849740 1 -- 172.21.15.16:0/2575336704 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5647f7b0d9d0 con 0x5647f7a8d6f0 2024-09-12T02:08:18.861 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.852+0000 7f8c4bfff640 1 -- 172.21.15.16:0/2575336704 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c6009f7e0 con 0x5647f7a8d6f0 2024-09-12T02:08:18.861 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.853+0000 7f8c4bfff640 1 -- 172.21.15.16:0/2575336704 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f8c58004b00 con 0x5647f7a8d6f0 2024-09-12T02:08:18.862 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.853+0000 7f8c75849740 10 client.17343 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:08:18.862 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.853+0000 7f8c75849740 10 client.17343 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:08:18.862 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.853+0000 7f8c75849740 20 client.17343 populate_metadata read hostname 'smithi016' 2024-09-12T02:08:18.862 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.853+0000 7f8c75849740 1 --2- 172.21.15.16:0/2575336704 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5647f7b0df20 0x5647f7b10310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:18.862 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.853+0000 7f8c75849740 4 client.17343 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:08:18.862 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.853+0000 7f8c75849740 1 -- 172.21.15.16:0/2575336704 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5647f79f3d70 con 0x5647f7b0df20 2024-09-12T02:08:18.862 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.853+0000 7f8c7084a640 1 --2- 172.21.15.16:0/2575336704 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5647f7b0df20 0x5647f7b10310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:18.862 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.853+0000 7f8c4bfff640 10 client.17343 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:08:18.862 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.853+0000 7f8c7084a640 1 --2- 172.21.15.16:0/2575336704 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5647f7b0df20 0x5647f7b10310 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1fb62ee264a7d727 server_cookie=5b6c242c43c69ebd in_seq=0 out_seq=0 2024-09-12T02:08:18.867 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.866+0000 7f8c4bfff640 1 -- 172.21.15.16:0/2575336704 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5647f79f3d70 con 0x5647f7b0df20 2024-09-12T02:08:18.867 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.866+0000 7f8c4bfff640 10 client.17343 handle_command_reply: tid=0 2024-09-12T02:08:18.867 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.866+0000 7f8c75849740 1 client.17343 shutdown 2024-09-12T02:08:18.867 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.866+0000 7f8c75849740 1 -- 172.21.15.16:0/2575336704 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5647f7a8d6f0 msgr2=0x5647f7a9bf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:18.867 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.866+0000 7f8c75849740 1 --2- 172.21.15.16:0/2575336704 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5647f7a8d6f0 0x5647f7a9bf80 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f8c58005ab0 tx=0x7f8c580318c0 comp rx=0 tx=0).stop 2024-09-12T02:08:18.867 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.866+0000 7f8c75849740 1 -- 172.21.15.16:0/2575336704 shutdown_connections 2024-09-12T02:08:18.867 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.866+0000 7f8c75849740 1 --2- 172.21.15.16:0/2575336704 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5647f7b0df20 0x5647f7b10310 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:18.867 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.866+0000 7f8c75849740 1 --2- 172.21.15.16:0/2575336704 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5647f7a92780 0x5647f7a9cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:18.867 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.866+0000 7f8c75849740 1 --2- 172.21.15.16:0/2575336704 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5647f7a8d6f0 0x5647f7a9bf80 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:18.868 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.866+0000 7f8c75849740 1 --2- 172.21.15.16:0/2575336704 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5647f7a8cdb0 0x5647f7a9ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:18.868 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.866+0000 7f8c4bfff640 0 client.17343 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:08:18.868 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.866+0000 7f8c75849740 1 -- 172.21.15.16:0/2575336704 >> 172.21.15.16:0/2575336704 conn(0x5647f7a84010 msgr2=0x5647f7a829a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:18.868 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.866+0000 7f8c75849740 1 -- 172.21.15.16:0/2575336704 shutdown_connections 2024-09-12T02:08:18.868 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.866+0000 7f8c75849740 1 -- 172.21.15.16:0/2575336704 wait complete. 2024-09-12T02:08:18.868 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.867+0000 7f8c75849740 20 client.17343 trim_cache size 0 max 16384 2024-09-12T02:08:18.868 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-09-12T02:08:18.868 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-09-12T02:08:18.868 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-09-12T02:08:18.869 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-09-12T02:08:18.869 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.867+0000 7f8c72aae640 1 -- 172.21.15.16:0/4107771565 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f8c5c0a2cb0 msgr2=0x7f8c5c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:18.870 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.867+0000 7f8c72aae640 1 --2- 172.21.15.16:0/4107771565 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f8c5c0a2cb0 0x7f8c5c0a5170 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f8c64009fd0 tx=0x7f8c64009530 comp rx=0 tx=0).stop 2024-09-12T02:08:18.870 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.867+0000 7f8c72aae640 1 -- 172.21.15.16:0/4107771565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c6c1025e0 msgr2=0x7f8c6c197e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:18.870 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.867+0000 7f8c72aae640 1 --2- 172.21.15.16:0/4107771565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c6c1025e0 0x7f8c6c197e50 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f8c6000d950 tx=0x7f8c6000de10 comp rx=0 tx=0).stop 2024-09-12T02:08:18.870 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.867+0000 7f8c72aae640 1 -- 172.21.15.16:0/4107771565 shutdown_connections 2024-09-12T02:08:18.870 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.867+0000 7f8c72aae640 1 --2- 172.21.15.16:0/4107771565 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f8c5c0a2cb0 0x7f8c5c0a5170 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:18.870 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.867+0000 7f8c72aae640 1 --2- 172.21.15.16:0/4107771565 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8c6c107840 0x7f8c6c19f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:18.870 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.867+0000 7f8c72aae640 1 --2- 172.21.15.16:0/4107771565 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c6c104f10 0x7f8c6c198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:18.870 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.867+0000 7f8c72aae640 1 --2- 172.21.15.16:0/4107771565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c6c1025e0 0x7f8c6c197e50 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:18.870 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.867+0000 7f8c72aae640 1 -- 172.21.15.16:0/4107771565 >> 172.21.15.16:0/4107771565 conn(0x7f8c6c0fc1e0 msgr2=0x7f8c6c102fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:18.870 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.867+0000 7f8c72aae640 1 -- 172.21.15.16:0/4107771565 shutdown_connections 2024-09-12T02:08:18.870 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:18.867+0000 7f8c72aae640 1 -- 172.21.15.16:0/4107771565 wait complete. 2024-09-12T02:08:20.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:08:20 smithi184 ceph-mon[35593]: pgmap v527: 97 pgs: 97 active+clean; 3.4 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 11 op/s 2024-09-12T02:08:20.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:08:20 smithi033 ceph-mon[35332]: pgmap v527: 97 pgs: 97 active+clean; 3.4 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 11 op/s 2024-09-12T02:08:20.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:08:20 smithi016 ceph-mon[30947]: pgmap v527: 97 pgs: 97 active+clean; 3.4 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 11 op/s 2024-09-12T02:08:22.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:08:22 smithi184 ceph-mon[35593]: pgmap v528: 97 pgs: 97 active+clean; 3.5 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 21 KiB/s wr, 17 op/s 2024-09-12T02:08:22.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:08:22 smithi033 ceph-mon[35332]: pgmap v528: 97 pgs: 97 active+clean; 3.5 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 21 KiB/s wr, 17 op/s 2024-09-12T02:08:22.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:08:22 smithi016 ceph-mon[30947]: pgmap v528: 97 pgs: 97 active+clean; 3.5 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 21 KiB/s wr, 17 op/s 2024-09-12T02:08:24.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:08:24 smithi184 ceph-mon[35593]: pgmap v529: 97 pgs: 97 active+clean; 3.5 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-09-12T02:08:24.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:08:24 smithi033 ceph-mon[35332]: pgmap v529: 97 pgs: 97 active+clean; 3.5 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-09-12T02:08:24.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:08:24 smithi016 ceph-mon[30947]: pgmap v529: 97 pgs: 97 active+clean; 3.5 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-09-12T02:08:26.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:08:26 smithi184 ceph-mon[35593]: pgmap v530: 97 pgs: 97 active+clean; 3.5 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-09-12T02:08:26.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:08:26 smithi033 ceph-mon[35332]: pgmap v530: 97 pgs: 97 active+clean; 3.5 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-09-12T02:08:26.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:08:26 smithi016 ceph-mon[30947]: pgmap v530: 97 pgs: 97 active+clean; 3.5 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-09-12T02:08:28.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:08:28 smithi184 ceph-mon[35593]: pgmap v531: 97 pgs: 97 active+clean; 3.5 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-12T02:08:28.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:08:28 smithi033 ceph-mon[35332]: pgmap v531: 97 pgs: 97 active+clean; 3.5 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-12T02:08:28.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:08:28 smithi016 ceph-mon[30947]: pgmap v531: 97 pgs: 97 active+clean; 3.5 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-12T02:08:30.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:08:30 smithi184 ceph-mon[35593]: pgmap v532: 97 pgs: 97 active+clean; 3.5 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-09-12T02:08:30.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:08:30 smithi033 ceph-mon[35332]: pgmap v532: 97 pgs: 97 active+clean; 3.5 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-09-12T02:08:30.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:08:30 smithi016 ceph-mon[30947]: pgmap v532: 97 pgs: 97 active+clean; 3.5 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-09-12T02:08:31.744 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 16.6 2024-09-12T02:08:31.745 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:08:31.961 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.960+0000 7f8424778640 1 -- 172.21.15.16:0/610657392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f841c102030 msgr2=0x7f841c102430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:31.961 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.960+0000 7f8424778640 1 --2- 172.21.15.16:0/610657392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f841c102030 0x7f841c102430 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f8418009900 tx=0x7f841802a410 comp rx=0 tx=0).stop 2024-09-12T02:08:31.962 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.960+0000 7f8424778640 1 -- 172.21.15.16:0/610657392 shutdown_connections 2024-09-12T02:08:31.962 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.960+0000 7f8424778640 1 --2- 172.21.15.16:0/610657392 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f841c101690 0x7f841c108f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:31.962 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.960+0000 7f8424778640 1 --2- 172.21.15.16:0/610657392 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f841c100cd0 0x7f841c101150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:31.962 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.960+0000 7f8424778640 1 --2- 172.21.15.16:0/610657392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f841c102030 0x7f841c102430 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:31.962 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.960+0000 7f8424778640 1 -- 172.21.15.16:0/610657392 >> 172.21.15.16:0/610657392 conn(0x7f841c0fb970 msgr2=0x7f841c0fdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:31.962 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.960+0000 7f8424778640 1 -- 172.21.15.16:0/610657392 shutdown_connections 2024-09-12T02:08:31.962 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.960+0000 7f8424778640 1 -- 172.21.15.16:0/610657392 wait complete. 2024-09-12T02:08:31.962 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.961+0000 7f8424778640 1 Processor -- start 2024-09-12T02:08:31.962 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.961+0000 7f8424778640 1 -- start start 2024-09-12T02:08:31.962 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.961+0000 7f8424778640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f841c100cd0 0x7f841c197dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:31.962 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.961+0000 7f8424778640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f841c101690 0x7f841c198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:31.962 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.961+0000 7f8424778640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f841c102030 0x7f841c19f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:31.962 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.961+0000 7f8424778640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f841c10c730 con 0x7f841c102030 2024-09-12T02:08:31.963 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.961+0000 7f8424778640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f841c10c5b0 con 0x7f841c101690 2024-09-12T02:08:31.963 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.961+0000 7f8424778640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f841c10c8b0 con 0x7f841c100cd0 2024-09-12T02:08:31.963 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.961+0000 7f8422d15640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f841c102030 0x7f841c19f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:31.963 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.961+0000 7f8422d15640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f841c102030 0x7f841c19f330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55298/0 (socket says 172.21.15.16:55298) 2024-09-12T02:08:31.963 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.961+0000 7f8422d15640 1 -- 172.21.15.16:0/1834947897 learned_addr learned my addr 172.21.15.16:0/1834947897 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:08:31.963 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.961+0000 7f8422514640 1 --2- 172.21.15.16:0/1834947897 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f841c100cd0 0x7f841c197dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:31.963 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.961+0000 7f8421d13640 1 --2- 172.21.15.16:0/1834947897 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f841c101690 0x7f841c198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:31.963 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.961+0000 7f8422d15640 1 -- 172.21.15.16:0/1834947897 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f841c100cd0 msgr2=0x7f841c197dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:31.963 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.961+0000 7f8422d15640 1 --2- 172.21.15.16:0/1834947897 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f841c100cd0 0x7f841c197dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:31.963 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.961+0000 7f8422d15640 1 -- 172.21.15.16:0/1834947897 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f841c101690 msgr2=0x7f841c198300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:31.965 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.961+0000 7f8422d15640 1 --2- 172.21.15.16:0/1834947897 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f841c101690 0x7f841c198300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:31.965 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.961+0000 7f8422d15640 1 -- 172.21.15.16:0/1834947897 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f841c19f9d0 con 0x7f841c102030 2024-09-12T02:08:31.965 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.962+0000 7f8422d15640 1 --2- 172.21.15.16:0/1834947897 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f841c102030 0x7f841c19f330 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f841400eb10 tx=0x7f841400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:31.965 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.962+0000 7f84137fe640 1 -- 172.21.15.16:0/1834947897 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8414004b20 con 0x7f841c102030 2024-09-12T02:08:31.965 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.962+0000 7f84137fe640 1 -- 172.21.15.16:0/1834947897 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8414004cc0 con 0x7f841c102030 2024-09-12T02:08:31.965 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.962+0000 7f8424778640 1 -- 172.21.15.16:0/1834947897 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f841c19fcc0 con 0x7f841c102030 2024-09-12T02:08:31.965 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.962+0000 7f84137fe640 1 -- 172.21.15.16:0/1834947897 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8414010760 con 0x7f841c102030 2024-09-12T02:08:31.965 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.962+0000 7f8424778640 1 -- 172.21.15.16:0/1834947897 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f841c1a0190 con 0x7f841c102030 2024-09-12T02:08:31.966 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.964+0000 7f84137fe640 1 -- 172.21.15.16:0/1834947897 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f8414005020 con 0x7f841c102030 2024-09-12T02:08:31.966 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.965+0000 7f84275ee740 1 Processor -- start 2024-09-12T02:08:31.966 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.965+0000 7f84275ee740 1 -- start start 2024-09-12T02:08:31.974 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.965+0000 7f84275ee740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c75e2edb0 0x555c75e2f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:31.974 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.965+0000 7f84275ee740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555c75e2f6f0 0x555c75e341b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:31.974 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.965+0000 7f84275ee740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555c75e34780 0x555c75e36c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:31.974 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.965+0000 7f84275ee740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x555c75d8e2f0 con 0x555c75e2edb0 2024-09-12T02:08:31.974 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.965+0000 7f84275ee740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x555c75d36270 con 0x555c75e2f6f0 2024-09-12T02:08:31.974 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.965+0000 7f84275ee740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x555c75d8a2b0 con 0x555c75e34780 2024-09-12T02:08:31.974 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.965+0000 7f84137fe640 1 --2- 172.21.15.16:0/1834947897 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f84080a2c30 0x7f84080a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:31.974 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.965+0000 7f8421d13640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c75e2edb0 0x555c75e2f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:31.974 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.965+0000 7f8422514640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555c75e2f6f0 0x555c75e341b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:31.974 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.965+0000 7f8421d13640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c75e2edb0 0x555c75e2f1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55306/0 (socket says 172.21.15.16:55306) 2024-09-12T02:08:31.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.965+0000 7f8421d13640 1 -- 172.21.15.16:0/1160760433 learned_addr learned my addr 172.21.15.16:0/1160760433 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:08:31.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.965+0000 7f8422d15640 1 --2- 172.21.15.16:0/1834947897 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f84080a2c30 0x7f84080a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:31.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.966+0000 7f84137fe640 1 -- 172.21.15.16:0/1834947897 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f84140cf820 con 0x7f841c102030 2024-09-12T02:08:31.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.966+0000 7f8421d13640 1 --2- 172.21.15.16:0/1160760433 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555c75e34780 0x555c75e36c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:31.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.966+0000 7f8422514640 1 -- 172.21.15.16:0/1160760433 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555c75e34780 msgr2=0x555c75e36c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:31.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.966+0000 7f8422514640 1 --2- 172.21.15.16:0/1160760433 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555c75e34780 0x555c75e36c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:31.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.966+0000 7f8422514640 1 -- 172.21.15.16:0/1160760433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c75e2edb0 msgr2=0x555c75e2f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:31.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.966+0000 7f8422514640 1 --2- 172.21.15.16:0/1160760433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c75e2edb0 0x555c75e2f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:31.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.966+0000 7f8422514640 1 -- 172.21.15.16:0/1160760433 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c75d35340 con 0x555c75e2f6f0 2024-09-12T02:08:31.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.966+0000 7f8421d13640 1 --2- 172.21.15.16:0/1160760433 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555c75e34780 0x555c75e36c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:08:31.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.966+0000 7f8422d15640 1 --2- 172.21.15.16:0/1834947897 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f84080a2c30 0x7f84080a50f0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f84140049d0 tx=0x7f841401bb30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:31.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.966+0000 7f8422514640 1 --2- 172.21.15.16:0/1160760433 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555c75e2f6f0 0x555c75e341b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f8418009230 tx=0x7f8418004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f6c62c6cbe126f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:31.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.966+0000 7f8410ff9640 1 -- 172.21.15.16:0/1160760433 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8418035030 con 0x555c75e2f6f0 2024-09-12T02:08:31.976 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.966+0000 7f8410ff9640 1 -- 172.21.15.16:0/1160760433 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8418005520 con 0x555c75e2f6f0 2024-09-12T02:08:31.976 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.966+0000 7f8410ff9640 1 -- 172.21.15.16:0/1160760433 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84180316b0 con 0x555c75e2f6f0 2024-09-12T02:08:31.976 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.967+0000 7f84275ee740 1 -- 172.21.15.16:0/1160760433 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555c75e2f6f0 msgr2=0x555c75e341b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:31.976 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.967+0000 7f84275ee740 1 --2- 172.21.15.16:0/1160760433 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555c75e2f6f0 0x555c75e341b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f8418009230 tx=0x7f8418004040 comp rx=0 tx=0).stop 2024-09-12T02:08:31.976 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.967+0000 7f84275ee740 1 -- 172.21.15.16:0/1160760433 shutdown_connections 2024-09-12T02:08:31.976 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.967+0000 7f84275ee740 1 --2- 172.21.15.16:0/1160760433 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555c75e34780 0x555c75e36c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:31.976 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.967+0000 7f84275ee740 1 --2- 172.21.15.16:0/1160760433 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555c75e2f6f0 0x555c75e341b0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:31.976 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.967+0000 7f84275ee740 1 --2- 172.21.15.16:0/1160760433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c75e2edb0 0x555c75e2f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:31.976 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.967+0000 7f84275ee740 1 -- 172.21.15.16:0/1160760433 >> 172.21.15.16:0/1160760433 conn(0x555c75e26010 msgr2=0x555c75e26410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:31.976 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.967+0000 7f84275ee740 1 -- 172.21.15.16:0/1160760433 shutdown_connections 2024-09-12T02:08:31.976 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.967+0000 7f84275ee740 1 -- 172.21.15.16:0/1160760433 wait complete. 2024-09-12T02:08:31.976 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.967+0000 7f84275ee740 1 -- start start 2024-09-12T02:08:31.976 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.967+0000 7f84275ee740 1 Processor -- start 2024-09-12T02:08:31.977 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.968+0000 7f84275ee740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c75e2edb0 0x555c75e38fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:31.977 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.968+0000 7f84275ee740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555c75e2f6f0 0x555c75e394e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:31.977 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.968+0000 7f84275ee740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555c75e34780 0x555c75e40520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:31.977 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.968+0000 7f84275ee740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x555c75e376f0 con 0x555c75e2edb0 2024-09-12T02:08:31.977 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.968+0000 7f84275ee740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x555c75d17290 con 0x555c75e34780 2024-09-12T02:08:31.977 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.968+0000 7f84275ee740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x555c75e37870 con 0x555c75e2f6f0 2024-09-12T02:08:31.977 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.968+0000 7f84275ee740 10 client.0 _ll_register_callbacks cb 0x555c75d48140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:08:31.977 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.968+0000 7f8421d13640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c75e2edb0 0x555c75e38fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:31.977 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.968+0000 7f8421d13640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c75e2edb0 0x555c75e38fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55314/0 (socket says 172.21.15.16:55314) 2024-09-12T02:08:31.977 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.968+0000 7f8421d13640 1 -- 172.21.15.16:0/3557419920 learned_addr learned my addr 172.21.15.16:0/3557419920 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:08:31.977 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.968+0000 7f8422514640 1 --2- 172.21.15.16:0/3557419920 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555c75e2f6f0 0x555c75e394e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:31.977 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.968+0000 7f8421d13640 1 --2- 172.21.15.16:0/3557419920 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555c75e34780 0x555c75e40520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:31.977 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.968+0000 7f8422514640 1 -- 172.21.15.16:0/3557419920 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555c75e34780 msgr2=0x555c75e40520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:31.978 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.968+0000 7f8422514640 1 --2- 172.21.15.16:0/3557419920 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555c75e34780 0x555c75e40520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:31.978 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.968+0000 7f8422514640 1 -- 172.21.15.16:0/3557419920 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c75e2edb0 msgr2=0x555c75e38fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:31.978 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.968+0000 7f8422514640 1 --2- 172.21.15.16:0/3557419920 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c75e2edb0 0x555c75e38fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:31.978 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.968+0000 7f8422514640 1 -- 172.21.15.16:0/3557419920 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c75aa1210 con 0x555c75e2f6f0 2024-09-12T02:08:31.978 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.968+0000 7f8421d13640 1 --2- 172.21.15.16:0/3557419920 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c75e2edb0 0x555c75e38fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:08:31.978 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.969+0000 7f8422514640 1 --2- 172.21.15.16:0/3557419920 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555c75e2f6f0 0x555c75e394e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f841802aa00 tx=0x7f84180048f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c175e96f428c9e04 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:31.978 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.969+0000 7f8405ffb640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-09-12T02:08:31.978 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.969+0000 7f8405ffb640 1 -- 172.21.15.16:0/3557419920 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8418051060 con 0x555c75e2f6f0 2024-09-12T02:08:31.978 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.969+0000 7f84275ee740 1 -- 172.21.15.16:0/3557419920 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8418005520 con 0x555c75e2f6f0 2024-09-12T02:08:31.978 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.969+0000 7f8405ffb640 1 -- 172.21.15.16:0/3557419920 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f8418005010 con 0x555c75e2f6f0 2024-09-12T02:08:31.978 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.969+0000 7f8405ffb640 1 -- 172.21.15.16:0/3557419920 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f8418035050 con 0x555c75e2f6f0 2024-09-12T02:08:31.978 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.969+0000 7f84275ee740 10 client.26554 fetch_fsmap learned FSMap version 25 2024-09-12T02:08:31.978 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.969+0000 7f84275ee740 1 -- 172.21.15.16:0/3557419920 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555c75b6d620 con 0x555c75e2f6f0 2024-09-12T02:08:31.979 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.969+0000 7f8405ffb640 1 -- 172.21.15.16:0/3557419920 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8418005520 con 0x555c75e2f6f0 2024-09-12T02:08:31.979 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.969+0000 7f8405ffb640 1 -- 172.21.15.16:0/3557419920 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f8418056070 con 0x555c75e2f6f0 2024-09-12T02:08:31.979 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.969+0000 7f84275ee740 10 client.26554 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:08:31.979 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.969+0000 7f84275ee740 10 client.26554 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:08:31.979 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.969+0000 7f84275ee740 20 client.26554 populate_metadata read hostname 'smithi016' 2024-09-12T02:08:31.979 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.969+0000 7f84275ee740 1 --2- 172.21.15.16:0/3557419920 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x555c75e44790 0x555c75e46b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:31.979 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.969+0000 7f84275ee740 4 client.26554 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:08:31.979 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.969+0000 7f84275ee740 1 -- 172.21.15.16:0/3557419920 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555c75d9ba20 con 0x555c75e44790 2024-09-12T02:08:31.979 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.970+0000 7f8421d13640 1 --2- 172.21.15.16:0/3557419920 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x555c75e44790 0x555c75e46b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:31.979 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.970+0000 7f8405ffb640 10 client.26554 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:08:31.979 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.970+0000 7f8421d13640 1 --2- 172.21.15.16:0/3557419920 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x555c75e44790 0x555c75e46b80 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89a84d856a5411d6 server_cookie=394bb49c4d4f08cf in_seq=0 out_seq=0 2024-09-12T02:08:31.979 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.971+0000 7f8405ffb640 1 -- 172.21.15.16:0/3557419920 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555c75d9ba20 con 0x555c75e44790 2024-09-12T02:08:31.979 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.971+0000 7f8405ffb640 10 client.26554 handle_command_reply: tid=0 2024-09-12T02:08:31.980 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.971+0000 7f84275ee740 1 client.26554 shutdown 2024-09-12T02:08:31.980 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.971+0000 7f84275ee740 1 -- 172.21.15.16:0/3557419920 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555c75e2f6f0 msgr2=0x555c75e394e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:31.980 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.971+0000 7f84275ee740 1 --2- 172.21.15.16:0/3557419920 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555c75e2f6f0 0x555c75e394e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f841802aa00 tx=0x7f84180048f0 comp rx=0 tx=0).stop 2024-09-12T02:08:31.980 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.971+0000 7f84275ee740 1 -- 172.21.15.16:0/3557419920 shutdown_connections 2024-09-12T02:08:31.980 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.971+0000 7f84275ee740 1 --2- 172.21.15.16:0/3557419920 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x555c75e44790 0x555c75e46b80 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:31.980 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.971+0000 7f84275ee740 1 --2- 172.21.15.16:0/3557419920 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555c75e34780 0x555c75e40520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:31.980 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.971+0000 7f84275ee740 1 --2- 172.21.15.16:0/3557419920 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555c75e2f6f0 0x555c75e394e0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:31.980 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.971+0000 7f84275ee740 1 --2- 172.21.15.16:0/3557419920 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c75e2edb0 0x555c75e38fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:31.980 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.971+0000 7f8405ffb640 0 client.26554 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:08:31.980 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.971+0000 7f84275ee740 1 -- 172.21.15.16:0/3557419920 >> 172.21.15.16:0/3557419920 conn(0x555c75e26010 msgr2=0x555c75e27820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:31.980 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.971+0000 7f84275ee740 1 -- 172.21.15.16:0/3557419920 shutdown_connections 2024-09-12T02:08:31.980 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.971+0000 7f84275ee740 1 -- 172.21.15.16:0/3557419920 wait complete. 2024-09-12T02:08:31.980 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.971+0000 7f84275ee740 20 client.26554 trim_cache size 0 max 16384 2024-09-12T02:08:31.988 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.987+0000 7f84275ee740 1 Processor -- start 2024-09-12T02:08:31.988 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.987+0000 7f84275ee740 1 -- start start 2024-09-12T02:08:31.988 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.987+0000 7f84275ee740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555c75e2edb0 0x555c75e38fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:31.988 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.987+0000 7f84275ee740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555c75e2f6f0 0x555c75e394e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:31.988 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.987+0000 7f84275ee740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c75e34780 0x555c75e404e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:31.988 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.987+0000 7f84275ee740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f841c1a9420 con 0x555c75e34780 2024-09-12T02:08:31.988 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.987+0000 7f84275ee740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f841c1a6cf0 con 0x555c75e2f6f0 2024-09-12T02:08:31.988 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.987+0000 7f84275ee740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f841c1a4b90 con 0x555c75e2edb0 2024-09-12T02:08:32.004 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.988+0000 7f8421d13640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c75e34780 0x555c75e404e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:32.004 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.988+0000 7f8421d13640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c75e34780 0x555c75e404e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55320/0 (socket says 172.21.15.16:55320) 2024-09-12T02:08:32.004 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.988+0000 7f8421d13640 1 -- 172.21.15.16:0/3270554858 learned_addr learned my addr 172.21.15.16:0/3270554858 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:08:32.004 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.988+0000 7f8422514640 1 --2- 172.21.15.16:0/3270554858 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555c75e2f6f0 0x555c75e394e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:32.005 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.988+0000 7f8421d13640 1 -- 172.21.15.16:0/3270554858 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555c75e2edb0 msgr2=0x555c75e38fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:32.005 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.988+0000 7f8421d13640 1 --2- 172.21.15.16:0/3270554858 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555c75e2edb0 0x555c75e38fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:32.005 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.988+0000 7f8421d13640 1 -- 172.21.15.16:0/3270554858 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555c75e2f6f0 msgr2=0x555c75e394e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:32.005 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.988+0000 7f8421d13640 1 --2- 172.21.15.16:0/3270554858 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555c75e2f6f0 0x555c75e394e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:32.005 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.988+0000 7f8421d13640 1 -- 172.21.15.16:0/3270554858 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c75e212d0 con 0x555c75e34780 2024-09-12T02:08:32.005 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.988+0000 7f8421d13640 1 --2- 172.21.15.16:0/3270554858 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c75e34780 0x555c75e404e0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f840c0095e0 tx=0x7f840c0097e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3eb724ded8cf8917 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:32.005 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.988+0000 7f8407fff640 1 -- 172.21.15.16:0/3270554858 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f840c0151e0 con 0x555c75e34780 2024-09-12T02:08:32.005 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.988+0000 7f8407fff640 1 -- 172.21.15.16:0/3270554858 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f840c012d80 con 0x555c75e34780 2024-09-12T02:08:32.005 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.988+0000 7f8407fff640 1 -- 172.21.15.16:0/3270554858 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f840c004990 con 0x555c75e34780 2024-09-12T02:08:32.008 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.989+0000 7f84275ee740 1 -- 172.21.15.16:0/3270554858 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c75e34780 msgr2=0x555c75e404e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:32.008 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.989+0000 7f84275ee740 1 --2- 172.21.15.16:0/3270554858 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c75e34780 0x555c75e404e0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f840c0095e0 tx=0x7f840c0097e0 comp rx=0 tx=0).stop 2024-09-12T02:08:32.008 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.989+0000 7f84275ee740 1 -- 172.21.15.16:0/3270554858 shutdown_connections 2024-09-12T02:08:32.008 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.989+0000 7f84275ee740 1 --2- 172.21.15.16:0/3270554858 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c75e34780 0x555c75e404e0 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:32.008 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.989+0000 7f84275ee740 1 --2- 172.21.15.16:0/3270554858 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555c75e2f6f0 0x555c75e394e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:32.008 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.989+0000 7f84275ee740 1 --2- 172.21.15.16:0/3270554858 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555c75e2edb0 0x555c75e38fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:32.008 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.989+0000 7f84275ee740 1 -- 172.21.15.16:0/3270554858 >> 172.21.15.16:0/3270554858 conn(0x555c75e26010 msgr2=0x555c75e30930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:32.008 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.989+0000 7f84275ee740 1 -- 172.21.15.16:0/3270554858 shutdown_connections 2024-09-12T02:08:32.008 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.989+0000 7f84275ee740 1 -- 172.21.15.16:0/3270554858 wait complete. 2024-09-12T02:08:32.009 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.989+0000 7f84275ee740 1 -- start start 2024-09-12T02:08:32.009 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.989+0000 7f84275ee740 1 Processor -- start 2024-09-12T02:08:32.009 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.990+0000 7f84275ee740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555c75e2edb0 0x555c75e3d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:32.009 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.990+0000 7f84275ee740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555c75e2f6f0 0x555c75e3d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:32.009 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.990+0000 7f84275ee740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c75e34780 0x555c75ead300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:32.009 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.990+0000 7f84275ee740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f841c1a51b0 con 0x555c75e34780 2024-09-12T02:08:32.009 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.990+0000 7f84275ee740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f841c1a6620 con 0x555c75e2f6f0 2024-09-12T02:08:32.009 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.990+0000 7f84275ee740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f841c1a5880 con 0x555c75e2edb0 2024-09-12T02:08:32.009 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.990+0000 7f84275ee740 10 client.0 _ll_register_callbacks cb 0x7f840c0096f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:08:32.009 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.990+0000 7f8421d13640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c75e34780 0x555c75ead300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:32.009 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.990+0000 7f8422514640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555c75e2f6f0 0x555c75e3d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:32.009 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.990+0000 7f8421d13640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c75e34780 0x555c75ead300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55328/0 (socket says 172.21.15.16:55328) 2024-09-12T02:08:32.009 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.990+0000 7f8421d13640 1 -- 172.21.15.16:0/3654994002 learned_addr learned my addr 172.21.15.16:0/3654994002 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:08:32.010 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.990+0000 7f8421d13640 1 --2- 172.21.15.16:0/3654994002 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555c75e2edb0 0x555c75e3d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:32.010 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.990+0000 7f8422514640 1 -- 172.21.15.16:0/3654994002 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555c75e2edb0 msgr2=0x555c75e3d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:32.010 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.990+0000 7f8422514640 1 --2- 172.21.15.16:0/3654994002 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555c75e2edb0 0x555c75e3d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:32.010 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.990+0000 7f8422514640 1 -- 172.21.15.16:0/3654994002 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c75e34780 msgr2=0x555c75ead300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:32.010 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.990+0000 7f8422514640 1 --2- 172.21.15.16:0/3654994002 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c75e34780 0x555c75ead300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:32.010 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.990+0000 7f8422514640 1 -- 172.21.15.16:0/3654994002 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c75ead840 con 0x555c75e2f6f0 2024-09-12T02:08:32.010 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.990+0000 7f8422514640 1 --2- 172.21.15.16:0/3654994002 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555c75e2f6f0 0x555c75e3d900 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f841802a930 tx=0x7f84180052c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc3f2b77fd846b18 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:32.010 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.990+0000 7f8405ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-12T02:08:32.010 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.991+0000 7f8405ffb640 1 -- 172.21.15.16:0/3654994002 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f841805c030 con 0x555c75e2f6f0 2024-09-12T02:08:32.010 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.991+0000 7f84275ee740 1 -- 172.21.15.16:0/3654994002 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f840c012d80 con 0x555c75e2f6f0 2024-09-12T02:08:32.010 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.991+0000 7f8405ffb640 1 -- 172.21.15.16:0/3654994002 <== mon.2 v2:172.21.15.33:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f8418049050 con 0x555c75e2f6f0 2024-09-12T02:08:32.010 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.991+0000 7f8405ffb640 1 -- 172.21.15.16:0/3654994002 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f8418005640 con 0x555c75e2f6f0 2024-09-12T02:08:32.010 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.991+0000 7f84275ee740 10 client.26759 fetch_fsmap learned FSMap version 25 2024-09-12T02:08:32.011 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.991+0000 7f84275ee740 1 -- 172.21.15.16:0/3654994002 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555c75eae9a0 con 0x555c75e2f6f0 2024-09-12T02:08:32.011 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.991+0000 7f8405ffb640 1 -- 172.21.15.16:0/3654994002 <== mon.2 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f840c012d80 con 0x555c75e2f6f0 2024-09-12T02:08:32.012 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.991+0000 7f8405ffb640 1 -- 172.21.15.16:0/3654994002 <== mon.2 v2:172.21.15.33:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f8418056040 con 0x555c75e2f6f0 2024-09-12T02:08:32.013 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.991+0000 7f84275ee740 10 client.26759 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:08:32.013 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.991+0000 7f84275ee740 10 client.26759 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:08:32.013 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.991+0000 7f84275ee740 20 client.26759 populate_metadata read hostname 'smithi016' 2024-09-12T02:08:32.013 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.991+0000 7f84275ee740 1 --2- 172.21.15.16:0/3654994002 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x555c75eaf3a0 0x555c75eb17c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:32.013 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.991+0000 7f84275ee740 4 client.26759 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:08:32.013 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.991+0000 7f84275ee740 1 -- 172.21.15.16:0/3654994002 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555c75d95d70 con 0x555c75eaf3a0 2024-09-12T02:08:32.013 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.992+0000 7f8421d13640 1 --2- 172.21.15.16:0/3654994002 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x555c75eaf3a0 0x555c75eb17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:32.013 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.992+0000 7f8405ffb640 10 client.26759 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:08:32.013 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:31.992+0000 7f8421d13640 1 --2- 172.21.15.16:0/3654994002 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x555c75eaf3a0 0x555c75eb17c0 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80129e2d0c550eec server_cookie=c6676a8cc56e0e1d in_seq=0 out_seq=0 2024-09-12T02:08:32.013 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:32.004+0000 7f8405ffb640 1 -- 172.21.15.16:0/3654994002 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555c75d95d70 con 0x555c75eaf3a0 2024-09-12T02:08:32.013 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:32.004+0000 7f8405ffb640 10 client.26759 handle_command_reply: tid=0 2024-09-12T02:08:32.014 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:32.004+0000 7f84275ee740 1 client.26759 shutdown 2024-09-12T02:08:32.014 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:32.005+0000 7f84275ee740 1 -- 172.21.15.16:0/3654994002 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555c75e2f6f0 msgr2=0x555c75e3d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:32.014 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:32.005+0000 7f84275ee740 1 --2- 172.21.15.16:0/3654994002 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555c75e2f6f0 0x555c75e3d900 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f841802a930 tx=0x7f84180052c0 comp rx=0 tx=0).stop 2024-09-12T02:08:32.014 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:32.005+0000 7f84275ee740 1 -- 172.21.15.16:0/3654994002 shutdown_connections 2024-09-12T02:08:32.014 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:32.005+0000 7f84275ee740 1 --2- 172.21.15.16:0/3654994002 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x555c75eaf3a0 0x555c75eb17c0 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:32.014 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:32.005+0000 7f84275ee740 1 --2- 172.21.15.16:0/3654994002 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c75e34780 0x555c75ead300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:32.014 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:32.005+0000 7f8405ffb640 0 client.26759 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:08:32.014 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:32.005+0000 7f84275ee740 1 --2- 172.21.15.16:0/3654994002 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555c75e2f6f0 0x555c75e3d900 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:32.014 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:32.005+0000 7f84275ee740 1 --2- 172.21.15.16:0/3654994002 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555c75e2edb0 0x555c75e3d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:32.014 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:32.005+0000 7f84275ee740 1 -- 172.21.15.16:0/3654994002 >> 172.21.15.16:0/3654994002 conn(0x555c75e26010 msgr2=0x555c75e38fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:32.014 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:32.005+0000 7f84275ee740 1 -- 172.21.15.16:0/3654994002 shutdown_connections 2024-09-12T02:08:32.014 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:32.005+0000 7f84275ee740 1 -- 172.21.15.16:0/3654994002 wait complete. 2024-09-12T02:08:32.015 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:32.005+0000 7f84275ee740 20 client.26759 trim_cache size 0 max 16384 2024-09-12T02:08:32.015 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-09-12T02:08:32.015 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-09-12T02:08:32.015 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-09-12T02:08:32.015 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-09-12T02:08:32.016 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:32.006+0000 7f8424778640 1 -- 172.21.15.16:0/1834947897 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f84080a2c30 msgr2=0x7f84080a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:32.016 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:32.006+0000 7f8424778640 1 --2- 172.21.15.16:0/1834947897 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f84080a2c30 0x7f84080a50f0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f84140049d0 tx=0x7f841401bb30 comp rx=0 tx=0).stop 2024-09-12T02:08:32.016 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:32.006+0000 7f8424778640 1 -- 172.21.15.16:0/1834947897 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f841c102030 msgr2=0x7f841c19f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:32.016 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:32.006+0000 7f8424778640 1 --2- 172.21.15.16:0/1834947897 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f841c102030 0x7f841c19f330 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f841400eb10 tx=0x7f841400c2d0 comp rx=0 tx=0).stop 2024-09-12T02:08:32.016 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:32.006+0000 7f8424778640 1 -- 172.21.15.16:0/1834947897 shutdown_connections 2024-09-12T02:08:32.016 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:32.006+0000 7f8424778640 1 --2- 172.21.15.16:0/1834947897 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f84080a2c30 0x7f84080a50f0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:32.016 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:32.006+0000 7f8424778640 1 --2- 172.21.15.16:0/1834947897 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f841c102030 0x7f841c19f330 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:32.017 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:32.006+0000 7f8424778640 1 --2- 172.21.15.16:0/1834947897 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f841c101690 0x7f841c198300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:32.017 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:32.006+0000 7f8424778640 1 --2- 172.21.15.16:0/1834947897 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f841c100cd0 0x7f841c197dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:32.017 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:32.006+0000 7f8424778640 1 -- 172.21.15.16:0/1834947897 >> 172.21.15.16:0/1834947897 conn(0x7f841c0fb970 msgr2=0x7f841c1004a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:32.017 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:32.006+0000 7f8424778640 1 -- 172.21.15.16:0/1834947897 shutdown_connections 2024-09-12T02:08:32.017 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:32.006+0000 7f8424778640 1 -- 172.21.15.16:0/1834947897 wait complete. 2024-09-12T02:08:32.033 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:08:32.436 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:08:32 smithi016 ceph-mon[30947]: pgmap v533: 97 pgs: 97 active+clean; 3.6 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-09-12T02:08:32.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:32.523+0000 7f1e64618640 1 -- 172.21.15.16:0/4262069570 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e5c10c170 msgr2=0x7f1e5c10e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:32.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:32.523+0000 7f1e64618640 1 --2- 172.21.15.16:0/4262069570 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e5c10c170 0x7f1e5c10e610 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f1e54009ca0 tx=0x7f1e5402a950 comp rx=0 tx=0).stop 2024-09-12T02:08:32.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:32.523+0000 7f1e64618640 1 -- 172.21.15.16:0/4262069570 shutdown_connections 2024-09-12T02:08:32.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:32.523+0000 7f1e64618640 1 --2- 172.21.15.16:0/4262069570 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e5c10c170 0x7f1e5c10e610 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:32.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:32.523+0000 7f1e64618640 1 --2- 172.21.15.16:0/4262069570 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1e5c103660 0x7f1e5c10bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:32.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:32.523+0000 7f1e64618640 1 --2- 172.21.15.16:0/4262069570 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e5c102d40 0x7f1e5c103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:32.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:32.523+0000 7f1e64618640 1 -- 172.21.15.16:0/4262069570 >> 172.21.15.16:0/4262069570 conn(0x7f1e5c0fc7a0 msgr2=0x7f1e5c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:32.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:32.523+0000 7f1e64618640 1 -- 172.21.15.16:0/4262069570 shutdown_connections 2024-09-12T02:08:32.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:32.523+0000 7f1e64618640 1 -- 172.21.15.16:0/4262069570 wait complete. 2024-09-12T02:08:32.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:32.524+0000 7f1e64618640 1 Processor -- start 2024-09-12T02:08:32.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:32.524+0000 7f1e64618640 1 -- start start 2024-09-12T02:08:32.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:32.524+0000 7f1e64618640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e5c102d40 0x7f1e5c1984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:32.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:32.524+0000 7f1e64618640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1e5c103660 0x7f1e5c198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:32.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:32.524+0000 7f1e64618640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e5c10c170 0x7f1e5c19cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:32.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:32.524+0000 7f1e64618640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1e5c111fb0 con 0x7f1e5c10c170 2024-09-12T02:08:32.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:32.524+0000 7f1e64618640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1e5c111e30 con 0x7f1e5c102d40 2024-09-12T02:08:32.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:32.524+0000 7f1e64618640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1e5c112130 con 0x7f1e5c103660 2024-09-12T02:08:32.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:32.524+0000 7f1e62bb5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e5c10c170 0x7f1e5c19cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:32.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:32.524+0000 7f1e62bb5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e5c10c170 0x7f1e5c19cd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55348/0 (socket says 172.21.15.16:55348) 2024-09-12T02:08:32.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:32.524+0000 7f1e62bb5640 1 -- 172.21.15.16:0/1363814115 learned_addr learned my addr 172.21.15.16:0/1363814115 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:08:32.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:32.525+0000 7f1e623b4640 1 --2- 172.21.15.16:0/1363814115 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e5c102d40 0x7f1e5c1984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:32.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:32.525+0000 7f1e61bb3640 1 --2- 172.21.15.16:0/1363814115 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1e5c103660 0x7f1e5c198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:32.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:32.525+0000 7f1e62bb5640 1 -- 172.21.15.16:0/1363814115 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1e5c103660 msgr2=0x7f1e5c198a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:32.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:32.525+0000 7f1e62bb5640 1 --2- 172.21.15.16:0/1363814115 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1e5c103660 0x7f1e5c198a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:32.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:32.525+0000 7f1e62bb5640 1 -- 172.21.15.16:0/1363814115 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e5c102d40 msgr2=0x7f1e5c1984c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:32.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:32.525+0000 7f1e62bb5640 1 --2- 172.21.15.16:0/1363814115 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e5c102d40 0x7f1e5c1984c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:32.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:32.525+0000 7f1e62bb5640 1 -- 172.21.15.16:0/1363814115 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e5c19d470 con 0x7f1e5c10c170 2024-09-12T02:08:32.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:32.525+0000 7f1e62bb5640 1 --2- 172.21.15.16:0/1363814115 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e5c10c170 0x7f1e5c19cd90 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f1e5402ae70 tx=0x7f1e5402f5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:32.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:32.525+0000 7f1e537fe640 1 -- 172.21.15.16:0/1363814115 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e54034040 con 0x7f1e5c10c170 2024-09-12T02:08:32.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:32.525+0000 7f1e64618640 1 -- 172.21.15.16:0/1363814115 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e5c19d700 con 0x7f1e5c10c170 2024-09-12T02:08:32.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:32.525+0000 7f1e537fe640 1 -- 172.21.15.16:0/1363814115 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e54030920 con 0x7f1e5c10c170 2024-09-12T02:08:32.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:32.525+0000 7f1e537fe640 1 -- 172.21.15.16:0/1363814115 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e54045480 con 0x7f1e5c10c170 2024-09-12T02:08:32.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:32.525+0000 7f1e64618640 1 -- 172.21.15.16:0/1363814115 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e5c1a5040 con 0x7f1e5c10c170 2024-09-12T02:08:32.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:32.526+0000 7f1e64618640 1 -- 172.21.15.16:0/1363814115 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e5c06b070 con 0x7f1e5c10c170 2024-09-12T02:08:32.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:32.527+0000 7f1e537fe640 1 -- 172.21.15.16:0/1363814115 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f1e54037020 con 0x7f1e5c10c170 2024-09-12T02:08:32.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:32.528+0000 7f1e537fe640 1 --2- 172.21.15.16:0/1363814115 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f1e480a2b80 0x7f1e480a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:32.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:32.528+0000 7f1e537fe640 1 -- 172.21.15.16:0/1363814115 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f1e540f01e0 con 0x7f1e5c10c170 2024-09-12T02:08:32.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:32.528+0000 7f1e623b4640 1 --2- 172.21.15.16:0/1363814115 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f1e480a2b80 0x7f1e480a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:32.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:32.529+0000 7f1e623b4640 1 --2- 172.21.15.16:0/1363814115 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f1e480a2b80 0x7f1e480a5040 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f1e58009890 tx=0x7f1e58008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:32.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:32.531+0000 7f1e537fe640 1 -- 172.21.15.16:0/1363814115 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1e540811b0 con 0x7f1e5c10c170 2024-09-12T02:08:32.628 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:08:32 smithi184 ceph-mon[35593]: pgmap v533: 97 pgs: 97 active+clean; 3.6 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-09-12T02:08:32.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:08:32 smithi033 ceph-mon[35332]: pgmap v533: 97 pgs: 97 active+clean; 3.6 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-09-12T02:08:32.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:32.740+0000 7f1e64618640 1 -- 172.21.15.16:0/1363814115 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1e5c103120 con 0x7f1e5c10c170 2024-09-12T02:08:32.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:32.741+0000 7f1e537fe640 1 -- 172.21.15.16:0/1363814115 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11911 (secure 0 0 0) 0x7f1e540bd140 con 0x7f1e5c10c170 2024-09-12T02:08:32.741 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T02:08:32.742 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":25,"btime":"2024-09-12T02:01:37:192021+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3279123279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3279123279},{"type":"v1","addr":"172.21.15.16:6835","nonce":3279123279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4119452382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4119452382},{"type":"v1","addr":"172.21.15.16:6837","nonce":4119452382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/3263823165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3263823165},{"type":"v1","addr":"172.21.15.16:6839","nonce":3263823165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/1510290732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":1510290732},{"type":"v1","addr":"172.21.15.16:6841","nonce":1510290732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/279674603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":279674603},{"type":"v1","addr":"172.21.15.33:6833","nonce":279674603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3555728028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3555728028},{"type":"v1","addr":"172.21.15.33:6835","nonce":3555728028}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T01:59:56.117608+0000","modified":"2024-09-12T02:01:36.294760+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24482,"mds_1":24481,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6833/1563374106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1563374106},{"type":"v1","addr":"172.21.15.184:6833","nonce":1563374106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6837/1892833868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1892833868},{"type":"v1","addr":"172.21.15.33:6837","nonce":1892833868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/1457679686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1457679686},{"type":"v1","addr":"172.21.15.184:6835","nonce":1457679686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.184:6837/1793393087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":1793393087},{"type":"v1","addr":"172.21.15.184:6837","nonce":1793393087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.184:6839/4203347941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":4203347941},{"type":"v1","addr":"172.21.15.184:6839","nonce":4203347941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.33:6839/4136670204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":4136670204},{"type":"v1","addr":"172.21.15.33:6839","nonce":4136670204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24481,24482,24469]},"id":1}]} 2024-09-12T02:08:32.742 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 25 2024-09-12T02:08:32.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:32.743+0000 7f1e64618640 1 -- 172.21.15.16:0/1363814115 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f1e480a2b80 msgr2=0x7f1e480a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:32.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:32.743+0000 7f1e64618640 1 --2- 172.21.15.16:0/1363814115 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f1e480a2b80 0x7f1e480a5040 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f1e58009890 tx=0x7f1e58008040 comp rx=0 tx=0).stop 2024-09-12T02:08:32.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:32.743+0000 7f1e64618640 1 -- 172.21.15.16:0/1363814115 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e5c10c170 msgr2=0x7f1e5c19cd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:32.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:32.743+0000 7f1e64618640 1 --2- 172.21.15.16:0/1363814115 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e5c10c170 0x7f1e5c19cd90 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f1e5402ae70 tx=0x7f1e5402f5b0 comp rx=0 tx=0).stop 2024-09-12T02:08:32.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:32.743+0000 7f1e64618640 1 -- 172.21.15.16:0/1363814115 shutdown_connections 2024-09-12T02:08:32.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:32.743+0000 7f1e64618640 1 --2- 172.21.15.16:0/1363814115 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f1e480a2b80 0x7f1e480a5040 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:32.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:32.743+0000 7f1e64618640 1 --2- 172.21.15.16:0/1363814115 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e5c10c170 0x7f1e5c19cd90 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:32.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:32.743+0000 7f1e64618640 1 --2- 172.21.15.16:0/1363814115 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1e5c103660 0x7f1e5c198a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:32.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:32.743+0000 7f1e64618640 1 --2- 172.21.15.16:0/1363814115 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e5c102d40 0x7f1e5c1984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:32.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:32.743+0000 7f1e64618640 1 -- 172.21.15.16:0/1363814115 >> 172.21.15.16:0/1363814115 conn(0x7f1e5c0fc7a0 msgr2=0x7f1e5c0fdec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:32.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:32.744+0000 7f1e64618640 1 -- 172.21.15.16:0/1363814115 shutdown_connections 2024-09-12T02:08:32.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:32.744+0000 7f1e64618640 1 -- 172.21.15.16:0/1363814115 wait complete. 2024-09-12T02:08:33.603 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:08:33 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/1363814115' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:08:33.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:08:33 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/1363814115' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:08:33.639 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24482 quiesce db / --timeout 90 --expiration 136.6 --await-for 45.0' 2024-09-12T02:08:33.639 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24482 quiesce db / --timeout 90 --expiration 136.6 --await-for 45.0 2024-09-12T02:08:33.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:08:33 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/1363814115' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:08:33.907 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:08:34.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.376+0000 7f358f738640 1 -- 172.21.15.16:0/2373626208 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3588106f10 msgr2=0x7f358810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:34.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.376+0000 7f358f738640 1 --2- 172.21.15.16:0/2373626208 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3588106f10 0x7f358810ea90 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f3580009900 tx=0x7f358002a3f0 comp rx=0 tx=0).stop 2024-09-12T02:08:34.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.376+0000 7f358f738640 1 -- 172.21.15.16:0/2373626208 shutdown_connections 2024-09-12T02:08:34.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.376+0000 7f358f738640 1 --2- 172.21.15.16:0/2373626208 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3588106f10 0x7f358810ea90 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:34.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.376+0000 7f358f738640 1 --2- 172.21.15.16:0/2373626208 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3588102790 0x7f35881067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:34.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.376+0000 7f358f738640 1 --2- 172.21.15.16:0/2373626208 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3588101de0 0x7f35881021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:34.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.376+0000 7f358f738640 1 -- 172.21.15.16:0/2373626208 >> 172.21.15.16:0/2373626208 conn(0x7f35880fd8c0 msgr2=0x7f35880ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:34.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.376+0000 7f358f738640 1 -- 172.21.15.16:0/2373626208 shutdown_connections 2024-09-12T02:08:34.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.376+0000 7f358f738640 1 -- 172.21.15.16:0/2373626208 wait complete. 2024-09-12T02:08:34.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.376+0000 7f358f738640 1 Processor -- start 2024-09-12T02:08:34.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.376+0000 7f358f738640 1 -- start start 2024-09-12T02:08:34.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.377+0000 7f358f738640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3588101de0 0x7f3588198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:34.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.377+0000 7f358f738640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3588102790 0x7f3588198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:34.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.377+0000 7f358f738640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3588106f10 0x7f358819cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:34.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.377+0000 7f358f738640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3588111f70 con 0x7f3588106f10 2024-09-12T02:08:34.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.377+0000 7f358f738640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3588111df0 con 0x7f3588101de0 2024-09-12T02:08:34.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.377+0000 7f358f738640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f35881120f0 con 0x7f3588102790 2024-09-12T02:08:34.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.377+0000 7f358dcd5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3588106f10 0x7f358819cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:34.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.377+0000 7f358ccd3640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3588102790 0x7f3588198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:34.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.377+0000 7f358dcd5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3588106f10 0x7f358819cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55352/0 (socket says 172.21.15.16:55352) 2024-09-12T02:08:34.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.377+0000 7f358dcd5640 1 -- 172.21.15.16:0/901403023 learned_addr learned my addr 172.21.15.16:0/901403023 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:08:34.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.377+0000 7f358d4d4640 1 --2- 172.21.15.16:0/901403023 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3588101de0 0x7f3588198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:34.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.377+0000 7f358dcd5640 1 -- 172.21.15.16:0/901403023 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3588102790 msgr2=0x7f3588198860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:34.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.377+0000 7f358dcd5640 1 --2- 172.21.15.16:0/901403023 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3588102790 0x7f3588198860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:34.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.377+0000 7f358dcd5640 1 -- 172.21.15.16:0/901403023 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3588101de0 msgr2=0x7f3588198320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:34.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.377+0000 7f358dcd5640 1 --2- 172.21.15.16:0/901403023 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3588101de0 0x7f3588198320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:34.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.377+0000 7f358dcd5640 1 -- 172.21.15.16:0/901403023 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f358819d2d0 con 0x7f3588106f10 2024-09-12T02:08:34.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.377+0000 7f358d4d4640 1 --2- 172.21.15.16:0/901403023 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3588101de0 0x7f3588198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:08:34.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.378+0000 7f358dcd5640 1 --2- 172.21.15.16:0/901403023 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3588106f10 0x7f358819cbf0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f3580005ab0 tx=0x7f35800042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:34.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.378+0000 7f35867fc640 1 -- 172.21.15.16:0/901403023 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f358003f030 con 0x7f3588106f10 2024-09-12T02:08:34.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.378+0000 7f35867fc640 1 -- 172.21.15.16:0/901403023 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3580003ce0 con 0x7f3588106f10 2024-09-12T02:08:34.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.378+0000 7f35867fc640 1 -- 172.21.15.16:0/901403023 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3580035070 con 0x7f3588106f10 2024-09-12T02:08:34.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.378+0000 7f358f738640 1 -- 172.21.15.16:0/901403023 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f358819d560 con 0x7f3588106f10 2024-09-12T02:08:34.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.378+0000 7f358f738640 1 -- 172.21.15.16:0/901403023 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f358819d9f0 con 0x7f3588106f10 2024-09-12T02:08:34.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.380+0000 7f35867fc640 1 -- 172.21.15.16:0/901403023 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f3580042030 con 0x7f3588106f10 2024-09-12T02:08:34.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.380+0000 7f3591b11740 1 Processor -- start 2024-09-12T02:08:34.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.380+0000 7f3591b11740 1 -- start start 2024-09-12T02:08:34.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.381+0000 7f3591b11740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a9798c760 0x562a9798cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:34.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.381+0000 7f3591b11740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562a9798d080 0x562a97997400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:34.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.381+0000 7f3591b11740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562a97997940 0x562a97999d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:34.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.381+0000 7f3591b11740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x562a978f22a0 con 0x562a9798c760 2024-09-12T02:08:34.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.381+0000 7f3591b11740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x562a97899b10 con 0x562a9798d080 2024-09-12T02:08:34.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.381+0000 7f3591b11740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x562a978ee260 con 0x562a97997940 2024-09-12T02:08:34.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.381+0000 7f358ccd3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a9798c760 0x562a9798cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:34.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.381+0000 7f35867fc640 1 --2- 172.21.15.16:0/901403023 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f35700a2c00 0x7f35700a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:34.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.381+0000 7f358ccd3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a9798c760 0x562a9798cb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55354/0 (socket says 172.21.15.16:55354) 2024-09-12T02:08:34.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.381+0000 7f358ccd3640 1 -- 172.21.15.16:0/3197990376 learned_addr learned my addr 172.21.15.16:0/3197990376 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:08:34.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.381+0000 7f358d4d4640 1 --2- 172.21.15.16:0/3197990376 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562a9798d080 0x562a97997400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:34.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.381+0000 7f358dcd5640 1 --2- 172.21.15.16:0/901403023 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f35700a2c00 0x7f35700a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:34.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.381+0000 7f358ccd3640 1 --2- 172.21.15.16:0/3197990376 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562a97997940 0x562a97999d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:34.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.381+0000 7f35867fc640 1 -- 172.21.15.16:0/901403023 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f35800ef3f0 con 0x7f3588106f10 2024-09-12T02:08:34.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.381+0000 7f358ccd3640 1 -- 172.21.15.16:0/3197990376 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562a97997940 msgr2=0x562a97999d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:34.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.381+0000 7f358ccd3640 1 --2- 172.21.15.16:0/3197990376 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562a97997940 0x562a97999d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:34.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.381+0000 7f358ccd3640 1 -- 172.21.15.16:0/3197990376 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562a9798d080 msgr2=0x562a97997400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:34.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.381+0000 7f358ccd3640 1 --2- 172.21.15.16:0/3197990376 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562a9798d080 0x562a97997400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:34.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.381+0000 7f358ccd3640 1 -- 172.21.15.16:0/3197990376 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a97663b90 con 0x562a9798c760 2024-09-12T02:08:34.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.382+0000 7f358ccd3640 1 --2- 172.21.15.16:0/3197990376 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a9798c760 0x562a9798cb40 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f357c00af60 tx=0x7f357c00a780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d397c58ed3fab1de server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:34.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.382+0000 7f356f7fe640 1 -- 172.21.15.16:0/3197990376 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f357c019030 con 0x562a9798c760 2024-09-12T02:08:34.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.382+0000 7f356f7fe640 1 -- 172.21.15.16:0/3197990376 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f357c0110d0 con 0x562a9798c760 2024-09-12T02:08:34.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.382+0000 7f356f7fe640 1 -- 172.21.15.16:0/3197990376 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f357c00f8e0 con 0x562a9798c760 2024-09-12T02:08:34.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.382+0000 7f3591b11740 1 -- 172.21.15.16:0/3197990376 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a9798c760 msgr2=0x562a9798cb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:34.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.382+0000 7f358dcd5640 1 --2- 172.21.15.16:0/901403023 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f35700a2c00 0x7f35700a50c0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f3580009a20 tx=0x7f35800058b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:34.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.382+0000 7f3591b11740 1 --2- 172.21.15.16:0/3197990376 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a9798c760 0x562a9798cb40 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f357c00af60 tx=0x7f357c00a780 comp rx=0 tx=0).stop 2024-09-12T02:08:34.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.383+0000 7f3591b11740 1 -- 172.21.15.16:0/3197990376 shutdown_connections 2024-09-12T02:08:34.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.383+0000 7f3591b11740 1 --2- 172.21.15.16:0/3197990376 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562a97997940 0x562a97999d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:34.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.383+0000 7f3591b11740 1 --2- 172.21.15.16:0/3197990376 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562a9798d080 0x562a97997400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:34.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.383+0000 7f3591b11740 1 --2- 172.21.15.16:0/3197990376 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a9798c760 0x562a9798cb40 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:34.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.383+0000 7f3591b11740 1 -- 172.21.15.16:0/3197990376 >> 172.21.15.16:0/3197990376 conn(0x562a97988cf0 msgr2=0x562a979890d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:34.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.383+0000 7f3591b11740 1 -- 172.21.15.16:0/3197990376 shutdown_connections 2024-09-12T02:08:34.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.383+0000 7f3591b11740 1 -- 172.21.15.16:0/3197990376 wait complete. 2024-09-12T02:08:34.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.383+0000 7f3591b11740 1 -- start start 2024-09-12T02:08:34.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.383+0000 7f3591b11740 1 Processor -- start 2024-09-12T02:08:34.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.383+0000 7f3591b11740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a9798c760 0x562a97994810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:34.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.383+0000 7f3591b11740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562a9798d080 0x562a97994d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:34.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.384+0000 7f3591b11740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562a97997940 0x562a97995290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:34.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.384+0000 7f3591b11740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x562a9799a510 con 0x562a9798c760 2024-09-12T02:08:34.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.384+0000 7f3591b11740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x562a9787a6e0 con 0x562a9798d080 2024-09-12T02:08:34.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.384+0000 7f3591b11740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x562a9799a690 con 0x562a97997940 2024-09-12T02:08:34.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.384+0000 7f358ccd3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a9798c760 0x562a97994810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:34.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.384+0000 7f3591b11740 10 client.0 _ll_register_callbacks cb 0x562a9789ea80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:08:34.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.384+0000 7f358ccd3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a9798c760 0x562a97994810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55370/0 (socket says 172.21.15.16:55370) 2024-09-12T02:08:34.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.384+0000 7f358ccd3640 1 -- 172.21.15.16:0/835914200 learned_addr learned my addr 172.21.15.16:0/835914200 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:08:34.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.384+0000 7f358d4d4640 1 --2- 172.21.15.16:0/835914200 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562a9798d080 0x562a97994d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:34.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.384+0000 7f358ccd3640 1 -- 172.21.15.16:0/835914200 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562a97997940 msgr2=0x562a97995290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:34.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.384+0000 7f358ccd3640 1 --2- 172.21.15.16:0/835914200 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562a97997940 0x562a97995290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:34.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.384+0000 7f358ccd3640 1 -- 172.21.15.16:0/835914200 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562a9798d080 msgr2=0x562a97994d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:34.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.384+0000 7f358ccd3640 1 --2- 172.21.15.16:0/835914200 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562a9798d080 0x562a97994d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:34.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.384+0000 7f358ccd3640 1 -- 172.21.15.16:0/835914200 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a979a3a00 con 0x562a9798c760 2024-09-12T02:08:34.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.384+0000 7f358d4d4640 1 --2- 172.21.15.16:0/835914200 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562a9798d080 0x562a97994d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:08:34.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.384+0000 7f358ccd3640 1 --2- 172.21.15.16:0/835914200 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a9798c760 0x562a97994810 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f357c00a660 tx=0x7f357c011280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9eeb9e788285ea server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:34.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.384+0000 7f356cff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:08:34.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.384+0000 7f356cff9640 1 -- 172.21.15.16:0/835914200 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f357c019030 con 0x562a9798c760 2024-09-12T02:08:34.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.384+0000 7f356cff9640 1 -- 172.21.15.16:0/835914200 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f357c005480 con 0x562a9798c760 2024-09-12T02:08:34.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.384+0000 7f3591b11740 1 -- 172.21.15.16:0/835914200 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f357c0110d0 con 0x562a9798c760 2024-09-12T02:08:34.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.385+0000 7f356cff9640 1 -- 172.21.15.16:0/835914200 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f357c005d60 con 0x562a9798c760 2024-09-12T02:08:34.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.385+0000 7f3591b11740 10 client.17379 fetch_fsmap learned FSMap version 25 2024-09-12T02:08:34.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.385+0000 7f3591b11740 1 -- 172.21.15.16:0/835914200 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x562a979a6370 con 0x562a9798c760 2024-09-12T02:08:34.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.385+0000 7f356cff9640 1 -- 172.21.15.16:0/835914200 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f357c0110d0 con 0x562a9798c760 2024-09-12T02:08:34.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.385+0000 7f356cff9640 1 -- 172.21.15.16:0/835914200 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f357c025070 con 0x562a9798c760 2024-09-12T02:08:34.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.385+0000 7f3591b11740 10 client.17379 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:08:34.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.385+0000 7f3591b11740 10 client.17379 resolve_mds: validated gid 24482 aka daemon mds.b 2024-09-12T02:08:34.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.385+0000 7f3591b11740 20 client.17379 populate_metadata read hostname 'smithi016' 2024-09-12T02:08:34.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.385+0000 7f3591b11740 1 --2- 172.21.15.16:0/835914200 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x562a979a76c0 0x562a979a9ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:34.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.385+0000 7f3591b11740 4 client.17379 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:08:34.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.385+0000 7f3591b11740 1 -- 172.21.15.16:0/835914200 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562a978ff9d0 con 0x562a979a76c0 2024-09-12T02:08:34.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.385+0000 7f358d4d4640 1 --2- 172.21.15.16:0/835914200 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x562a979a76c0 0x562a979a9ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:34.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.386+0000 7f358d4d4640 1 --2- 172.21.15.16:0/835914200 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x562a979a76c0 0x562a979a9ab0 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1195f30550b463c8 server_cookie=633fa1063fcc4bc8 in_seq=0 out_seq=0 2024-09-12T02:08:34.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.386+0000 7f356cff9640 10 client.17379 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:08:34.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.387+0000 7f356cff9640 1 -- 172.21.15.16:0/835914200 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562a978ff9d0 con 0x562a979a76c0 2024-09-12T02:08:34.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.387+0000 7f356cff9640 10 client.17379 handle_command_reply: tid=0 2024-09-12T02:08:34.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.387+0000 7f3591b11740 1 client.17379 shutdown 2024-09-12T02:08:34.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.387+0000 7f3591b11740 1 -- 172.21.15.16:0/835914200 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a9798c760 msgr2=0x562a97994810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:34.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.387+0000 7f3591b11740 1 --2- 172.21.15.16:0/835914200 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a9798c760 0x562a97994810 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f357c00a660 tx=0x7f357c011280 comp rx=0 tx=0).stop 2024-09-12T02:08:34.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.387+0000 7f3591b11740 1 -- 172.21.15.16:0/835914200 shutdown_connections 2024-09-12T02:08:34.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.387+0000 7f3591b11740 1 --2- 172.21.15.16:0/835914200 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x562a979a76c0 0x562a979a9ab0 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:34.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.387+0000 7f3591b11740 1 --2- 172.21.15.16:0/835914200 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562a97997940 0x562a97995290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:34.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.387+0000 7f3591b11740 1 --2- 172.21.15.16:0/835914200 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562a9798d080 0x562a97994d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:34.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.387+0000 7f3591b11740 1 --2- 172.21.15.16:0/835914200 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a9798c760 0x562a97994810 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:34.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.387+0000 7f3591b11740 1 -- 172.21.15.16:0/835914200 >> 172.21.15.16:0/835914200 conn(0x562a97988cf0 msgr2=0x562a9798ea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:34.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.387+0000 7f356cff9640 0 client.17379 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:08:34.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.387+0000 7f3591b11740 1 -- 172.21.15.16:0/835914200 shutdown_connections 2024-09-12T02:08:34.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.387+0000 7f3591b11740 1 -- 172.21.15.16:0/835914200 wait complete. 2024-09-12T02:08:34.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.387+0000 7f3591b11740 20 client.17379 trim_cache size 0 max 16384 2024-09-12T02:08:34.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.404+0000 7f3591b11740 1 Processor -- start 2024-09-12T02:08:34.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.404+0000 7f3591b11740 1 -- start start 2024-09-12T02:08:34.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.405+0000 7f3591b11740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a9798c760 0x562a9798f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:34.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.405+0000 7f3591b11740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562a9798d080 0x562a979af1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:34.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.405+0000 7f3591b11740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562a97997940 0x562a979af730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:34.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.405+0000 7f3591b11740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f35881afc40 con 0x562a9798c760 2024-09-12T02:08:34.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.405+0000 7f3591b11740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f35881ad510 con 0x562a97997940 2024-09-12T02:08:34.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.405+0000 7f3591b11740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f35881ab360 con 0x562a9798d080 2024-09-12T02:08:34.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.405+0000 7f358ccd3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a9798c760 0x562a9798f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:34.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.405+0000 7f358ccd3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a9798c760 0x562a9798f1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55376/0 (socket says 172.21.15.16:55376) 2024-09-12T02:08:34.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.405+0000 7f358ccd3640 1 -- 172.21.15.16:0/1851883013 learned_addr learned my addr 172.21.15.16:0/1851883013 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:08:34.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.405+0000 7f358d4d4640 1 --2- 172.21.15.16:0/1851883013 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562a9798d080 0x562a979af1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:34.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.405+0000 7f358ccd3640 1 -- 172.21.15.16:0/1851883013 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562a9798d080 msgr2=0x562a979af1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:34.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.405+0000 7f358ccd3640 1 --2- 172.21.15.16:0/1851883013 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562a9798d080 0x562a979af1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:34.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.405+0000 7f358ccd3640 1 -- 172.21.15.16:0/1851883013 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562a97997940 msgr2=0x562a979af730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:34.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.405+0000 7f358ccd3640 1 --2- 172.21.15.16:0/1851883013 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562a97997940 0x562a979af730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:34.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.405+0000 7f358ccd3640 1 -- 172.21.15.16:0/1851883013 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a97988b20 con 0x562a9798c760 2024-09-12T02:08:34.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.405+0000 7f358d4d4640 1 --2- 172.21.15.16:0/1851883013 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562a9798d080 0x562a979af1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:08:34.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.405+0000 7f358ccd3640 1 --2- 172.21.15.16:0/1851883013 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a9798c760 0x562a9798f1c0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f357c013510 tx=0x7f357c005b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d4aeac34dcc53e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:34.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.406+0000 7f356effd640 1 -- 172.21.15.16:0/1851883013 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f357c036070 con 0x562a9798c760 2024-09-12T02:08:34.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.406+0000 7f356effd640 1 -- 172.21.15.16:0/1851883013 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f357c00acc0 con 0x562a9798c760 2024-09-12T02:08:34.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.406+0000 7f356effd640 1 -- 172.21.15.16:0/1851883013 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f357c019070 con 0x562a9798c760 2024-09-12T02:08:34.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.406+0000 7f3591b11740 1 -- 172.21.15.16:0/1851883013 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a9798c760 msgr2=0x562a9798f1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:34.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.406+0000 7f3591b11740 1 --2- 172.21.15.16:0/1851883013 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a9798c760 0x562a9798f1c0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f357c013510 tx=0x7f357c005b70 comp rx=0 tx=0).stop 2024-09-12T02:08:34.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.406+0000 7f3591b11740 1 -- 172.21.15.16:0/1851883013 shutdown_connections 2024-09-12T02:08:34.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.406+0000 7f3591b11740 1 --2- 172.21.15.16:0/1851883013 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562a97997940 0x562a979af730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:34.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.406+0000 7f3591b11740 1 --2- 172.21.15.16:0/1851883013 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562a9798d080 0x562a979af1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:34.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.406+0000 7f3591b11740 1 --2- 172.21.15.16:0/1851883013 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a9798c760 0x562a9798f1c0 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:34.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.406+0000 7f3591b11740 1 -- 172.21.15.16:0/1851883013 >> 172.21.15.16:0/1851883013 conn(0x562a97988cf0 msgr2=0x562a97996890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:34.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.406+0000 7f3591b11740 1 -- 172.21.15.16:0/1851883013 shutdown_connections 2024-09-12T02:08:34.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.406+0000 7f3591b11740 1 -- 172.21.15.16:0/1851883013 wait complete. 2024-09-12T02:08:34.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.407+0000 7f3591b11740 1 -- start start 2024-09-12T02:08:34.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.407+0000 7f3591b11740 1 Processor -- start 2024-09-12T02:08:34.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.407+0000 7f3591b11740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562a9798c760 0x562a9799b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:34.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.407+0000 7f3591b11740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562a9798d080 0x562a9799bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:34.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.407+0000 7f3591b11740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a97997940 0x562a9799c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:34.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.407+0000 7f3591b11740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f35881ab9d0 con 0x562a97997940 2024-09-12T02:08:34.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.407+0000 7f3591b11740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f35881ace40 con 0x562a9798c760 2024-09-12T02:08:34.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.407+0000 7f3591b11740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f35881ac0a0 con 0x562a9798d080 2024-09-12T02:08:34.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.407+0000 7f3591b11740 10 client.0 _ll_register_callbacks cb 0x7f357c011af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:08:34.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.407+0000 7f358ccd3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a97997940 0x562a9799c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:34.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.407+0000 7f358ccd3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a97997940 0x562a9799c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55388/0 (socket says 172.21.15.16:55388) 2024-09-12T02:08:34.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.407+0000 7f358ccd3640 1 -- 172.21.15.16:0/1895164456 learned_addr learned my addr 172.21.15.16:0/1895164456 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:08:34.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.408+0000 7f358ccd3640 1 --2- 172.21.15.16:0/1895164456 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562a9798c760 0x562a9799b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:34.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.408+0000 7f358d4d4640 1 --2- 172.21.15.16:0/1895164456 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562a9798d080 0x562a9799bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:34.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.408+0000 7f358ccd3640 1 -- 172.21.15.16:0/1895164456 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562a9798d080 msgr2=0x562a9799bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:34.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.408+0000 7f358ccd3640 1 --2- 172.21.15.16:0/1895164456 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562a9798d080 0x562a9799bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:34.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.408+0000 7f358ccd3640 1 -- 172.21.15.16:0/1895164456 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562a9798c760 msgr2=0x562a9799b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:34.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.408+0000 7f358ccd3640 1 --2- 172.21.15.16:0/1895164456 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562a9798c760 0x562a9799b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:34.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.408+0000 7f358ccd3640 1 -- 172.21.15.16:0/1895164456 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a97984260 con 0x562a97997940 2024-09-12T02:08:34.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.408+0000 7f358ccd3640 1 --2- 172.21.15.16:0/1895164456 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a97997940 0x562a9799c280 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f357c034040 tx=0x7f357c020040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=558fb12d4334de04 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:34.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.408+0000 7f356cff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:08:34.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.408+0000 7f356cff9640 1 -- 172.21.15.16:0/1895164456 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f357c036040 con 0x562a97997940 2024-09-12T02:08:34.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.408+0000 7f356cff9640 1 -- 172.21.15.16:0/1895164456 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f357c004660 con 0x562a97997940 2024-09-12T02:08:34.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.408+0000 7f3591b11740 1 -- 172.21.15.16:0/1895164456 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f357c00acc0 con 0x562a97997940 2024-09-12T02:08:34.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.408+0000 7f356cff9640 1 -- 172.21.15.16:0/1895164456 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f357c019040 con 0x562a97997940 2024-09-12T02:08:34.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.408+0000 7f3591b11740 10 client.17385 fetch_fsmap learned FSMap version 25 2024-09-12T02:08:34.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.408+0000 7f3591b11740 1 -- 172.21.15.16:0/1895164456 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x562a97989da0 con 0x562a97997940 2024-09-12T02:08:34.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.409+0000 7f356cff9640 1 -- 172.21.15.16:0/1895164456 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f357c00acc0 con 0x562a97997940 2024-09-12T02:08:34.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.409+0000 7f356cff9640 1 -- 172.21.15.16:0/1895164456 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f357c025040 con 0x562a97997940 2024-09-12T02:08:34.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.409+0000 7f3591b11740 10 client.17385 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:08:34.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.409+0000 7f3591b11740 10 client.17385 resolve_mds: validated gid 24482 aka daemon mds.b 2024-09-12T02:08:34.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.409+0000 7f3591b11740 20 client.17385 populate_metadata read hostname 'smithi016' 2024-09-12T02:08:34.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.409+0000 7f3591b11740 1 --2- 172.21.15.16:0/1895164456 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x562a97a128e0 0x562a97a14cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:34.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.409+0000 7f3591b11740 4 client.17385 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 136.6, "await_for": 45.0}] 2024-09-12T02:08:34.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.409+0000 7f3591b11740 1 -- 172.21.15.16:0/1895164456 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 136.6, "await_for": 45.0}) -- 0x562a978f9d20 con 0x562a97a128e0 2024-09-12T02:08:34.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.409+0000 7f358d4d4640 1 --2- 172.21.15.16:0/1895164456 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x562a97a128e0 0x562a97a14cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:34.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.409+0000 7f358d4d4640 1 --2- 172.21.15.16:0/1895164456 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x562a97a128e0 0x562a97a14cd0 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6df6ba780abf07ba server_cookie=3932d9f80e3da7a8 in_seq=0 out_seq=0 2024-09-12T02:08:34.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.409+0000 7f356cff9640 10 client.17385 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:08:34.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.420+0000 7f356cff9640 1 -- 172.21.15.16:0/1895164456 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x562a978f9d20 con 0x562a97a128e0 2024-09-12T02:08:34.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.420+0000 7f356cff9640 10 client.17385 handle_command_reply: tid=0 2024-09-12T02:08:34.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.420+0000 7f3591b11740 1 client.17385 shutdown 2024-09-12T02:08:34.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.420+0000 7f3591b11740 1 -- 172.21.15.16:0/1895164456 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a97997940 msgr2=0x562a9799c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:34.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.420+0000 7f3591b11740 1 --2- 172.21.15.16:0/1895164456 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a97997940 0x562a9799c280 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f357c034040 tx=0x7f357c020040 comp rx=0 tx=0).stop 2024-09-12T02:08:34.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.420+0000 7f3591b11740 1 -- 172.21.15.16:0/1895164456 shutdown_connections 2024-09-12T02:08:34.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.420+0000 7f3591b11740 1 --2- 172.21.15.16:0/1895164456 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x562a97a128e0 0x562a97a14cd0 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:34.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.420+0000 7f3591b11740 1 --2- 172.21.15.16:0/1895164456 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a97997940 0x562a9799c280 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:34.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.420+0000 7f3591b11740 1 --2- 172.21.15.16:0/1895164456 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562a9798d080 0x562a9799bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:34.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.420+0000 7f3591b11740 1 --2- 172.21.15.16:0/1895164456 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562a9798c760 0x562a9799b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:34.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.420+0000 7f3591b11740 1 -- 172.21.15.16:0/1895164456 >> 172.21.15.16:0/1895164456 conn(0x562a97988cf0 msgr2=0x562a979947d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:34.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.420+0000 7f356cff9640 0 client.17385 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:08:34.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.421+0000 7f3591b11740 1 -- 172.21.15.16:0/1895164456 shutdown_connections 2024-09-12T02:08:34.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.421+0000 7f3591b11740 1 -- 172.21.15.16:0/1895164456 wait complete. 2024-09-12T02:08:34.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.421+0000 7f3591b11740 20 client.17385 trim_cache size 0 max 16384 2024-09-12T02:08:34.422 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T02:08:34.422 INFO:teuthology.orchestra.run.smithi016.stdout: "epoch": 25, 2024-09-12T02:08:34.422 INFO:teuthology.orchestra.run.smithi016.stdout: "leader": 24482, 2024-09-12T02:08:34.423 INFO:teuthology.orchestra.run.smithi016.stdout: "set_version": 26, 2024-09-12T02:08:34.423 INFO:teuthology.orchestra.run.smithi016.stdout: "sets": { 2024-09-12T02:08:34.423 INFO:teuthology.orchestra.run.smithi016.stdout: "e05b9d01": { 2024-09-12T02:08:34.423 INFO:teuthology.orchestra.run.smithi016.stdout: "version": 26, 2024-09-12T02:08:34.423 INFO:teuthology.orchestra.run.smithi016.stdout: "age_ref": 0.0, 2024-09-12T02:08:34.423 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-12T02:08:34.423 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "QUIESCED", 2024-09-12T02:08:34.423 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-12T02:08:34.423 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:08:34.423 INFO:teuthology.orchestra.run.smithi016.stdout: "timeout": 90.0, 2024-09-12T02:08:34.424 INFO:teuthology.orchestra.run.smithi016.stdout: "expiration": 136.6, 2024-09-12T02:08:34.424 INFO:teuthology.orchestra.run.smithi016.stdout: "members": { 2024-09-12T02:08:34.424 INFO:teuthology.orchestra.run.smithi016.stdout: "file:/": { 2024-09-12T02:08:34.424 INFO:teuthology.orchestra.run.smithi016.stdout: "excluded": false, 2024-09-12T02:08:34.424 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-12T02:08:34.424 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "QUIESCED", 2024-09-12T02:08:34.424 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-12T02:08:34.424 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:08:34.424 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:08:34.425 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:08:34.425 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:08:34.425 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:08:34.425 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T02:08:34.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.421+0000 7f358f738640 1 -- 172.21.15.16:0/901403023 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f35700a2c00 msgr2=0x7f35700a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:34.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.421+0000 7f358f738640 1 --2- 172.21.15.16:0/901403023 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f35700a2c00 0x7f35700a50c0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f3580009a20 tx=0x7f35800058b0 comp rx=0 tx=0).stop 2024-09-12T02:08:34.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.421+0000 7f358f738640 1 -- 172.21.15.16:0/901403023 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3588106f10 msgr2=0x7f358819cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:34.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.421+0000 7f358f738640 1 --2- 172.21.15.16:0/901403023 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3588106f10 0x7f358819cbf0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f3580005ab0 tx=0x7f35800042f0 comp rx=0 tx=0).stop 2024-09-12T02:08:34.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.421+0000 7f358f738640 1 -- 172.21.15.16:0/901403023 shutdown_connections 2024-09-12T02:08:34.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.421+0000 7f358f738640 1 --2- 172.21.15.16:0/901403023 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f35700a2c00 0x7f35700a50c0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:34.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.421+0000 7f358f738640 1 --2- 172.21.15.16:0/901403023 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3588106f10 0x7f358819cbf0 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:34.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.421+0000 7f358f738640 1 --2- 172.21.15.16:0/901403023 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3588102790 0x7f3588198860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:34.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.421+0000 7f358f738640 1 --2- 172.21.15.16:0/901403023 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3588101de0 0x7f3588198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:34.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.421+0000 7f358f738640 1 -- 172.21.15.16:0/901403023 >> 172.21.15.16:0/901403023 conn(0x7f35880fd8c0 msgr2=0x7f35880febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:34.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.422+0000 7f358f738640 1 -- 172.21.15.16:0/901403023 shutdown_connections 2024-09-12T02:08:34.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:34.422+0000 7f358f738640 1 -- 172.21.15.16:0/901403023 wait complete. 2024-09-12T02:08:34.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:08:34 smithi184 ceph-mon[35593]: pgmap v534: 97 pgs: 97 active+clean; 3.6 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T02:08:34.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:08:34 smithi033 ceph-mon[35332]: pgmap v534: 97 pgs: 97 active+clean; 3.6 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T02:08:34.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:08:34 smithi016 ceph-mon[30947]: pgmap v534: 97 pgs: 97 active+clean; 3.6 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T02:08:35.165 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'e05b9d01', quiesce took 3.4 seconds. Will release after: 13.200000000000001 2024-09-12T02:08:35.848 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T02:08:36.118 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:08:36.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:08:36 smithi016 ceph-mon[30947]: pgmap v535: 97 pgs: 97 active+clean; 3.7 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 14 op/s 2024-09-12T02:08:36.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.599+0000 7fd1d0b8f640 1 -- 172.21.15.16:0/2769229688 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1cc101de0 msgr2=0x7fd1cc1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:36.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.599+0000 7fd1d0b8f640 1 --2- 172.21.15.16:0/2769229688 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1cc101de0 0x7fd1cc1021c0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7fd1bc009900 tx=0x7fd1bc02a410 comp rx=0 tx=0).stop 2024-09-12T02:08:36.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.599+0000 7fd1d0b8f640 1 -- 172.21.15.16:0/2769229688 shutdown_connections 2024-09-12T02:08:36.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.599+0000 7fd1d0b8f640 1 --2- 172.21.15.16:0/2769229688 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd1cc106f10 0x7fd1cc10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:36.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.599+0000 7fd1d0b8f640 1 --2- 172.21.15.16:0/2769229688 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd1cc102790 0x7fd1cc1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:36.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.599+0000 7fd1d0b8f640 1 --2- 172.21.15.16:0/2769229688 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1cc101de0 0x7fd1cc1021c0 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:36.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.599+0000 7fd1d0b8f640 1 -- 172.21.15.16:0/2769229688 >> 172.21.15.16:0/2769229688 conn(0x7fd1cc0fd8c0 msgr2=0x7fd1cc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:36.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.599+0000 7fd1d0b8f640 1 -- 172.21.15.16:0/2769229688 shutdown_connections 2024-09-12T02:08:36.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.599+0000 7fd1d0b8f640 1 -- 172.21.15.16:0/2769229688 wait complete. 2024-09-12T02:08:36.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.600+0000 7fd1d0b8f640 1 Processor -- start 2024-09-12T02:08:36.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.600+0000 7fd1d0b8f640 1 -- start start 2024-09-12T02:08:36.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.600+0000 7fd1d0b8f640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd1cc101de0 0x7fd1cc0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:36.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.600+0000 7fd1d0b8f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1cc102790 0x7fd1cc074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:36.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.600+0000 7fd1d0b8f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd1cc106f10 0x7fd1cc074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:36.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.600+0000 7fd1d0b8f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd1cc112010 con 0x7fd1cc102790 2024-09-12T02:08:36.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.600+0000 7fd1d0b8f640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd1cc111e90 con 0x7fd1cc106f10 2024-09-12T02:08:36.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.600+0000 7fd1d0b8f640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd1cc112190 con 0x7fd1cc101de0 2024-09-12T02:08:36.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.600+0000 7fd1c3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1cc102790 0x7fd1cc074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:36.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.600+0000 7fd1c3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1cc102790 0x7fd1cc074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55410/0 (socket says 172.21.15.16:55410) 2024-09-12T02:08:36.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.600+0000 7fd1c3fff640 1 -- 172.21.15.16:0/2934483527 learned_addr learned my addr 172.21.15.16:0/2934483527 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:08:36.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.600+0000 7fd1ca59c640 1 --2- 172.21.15.16:0/2934483527 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd1cc101de0 0x7fd1cc0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:36.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.601+0000 7fd1cad9d640 1 --2- 172.21.15.16:0/2934483527 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd1cc106f10 0x7fd1cc074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:36.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.601+0000 7fd1c3fff640 1 -- 172.21.15.16:0/2934483527 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd1cc101de0 msgr2=0x7fd1cc0744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:36.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.601+0000 7fd1c3fff640 1 --2- 172.21.15.16:0/2934483527 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd1cc101de0 0x7fd1cc0744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:36.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.601+0000 7fd1c3fff640 1 -- 172.21.15.16:0/2934483527 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd1cc106f10 msgr2=0x7fd1cc074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:36.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.601+0000 7fd1c3fff640 1 --2- 172.21.15.16:0/2934483527 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd1cc106f10 0x7fd1cc074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:36.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.601+0000 7fd1c3fff640 1 -- 172.21.15.16:0/2934483527 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1cc1a6a70 con 0x7fd1cc102790 2024-09-12T02:08:36.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.601+0000 7fd1cad9d640 1 --2- 172.21.15.16:0/2934483527 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd1cc106f10 0x7fd1cc074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:08:36.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.601+0000 7fd1c3fff640 1 --2- 172.21.15.16:0/2934483527 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1cc102790 0x7fd1cc074a30 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7fd1b400b4a0 tx=0x7fd1b400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:36.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.601+0000 7fd1c37fe640 1 -- 172.21.15.16:0/2934483527 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1b4014070 con 0x7fd1cc102790 2024-09-12T02:08:36.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.601+0000 7fd1c37fe640 1 -- 172.21.15.16:0/2934483527 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1b4010af0 con 0x7fd1cc102790 2024-09-12T02:08:36.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.601+0000 7fd1d0b8f640 1 -- 172.21.15.16:0/2934483527 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1cc1a6d60 con 0x7fd1cc102790 2024-09-12T02:08:36.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.601+0000 7fd1c37fe640 1 -- 172.21.15.16:0/2934483527 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1b4017d10 con 0x7fd1cc102790 2024-09-12T02:08:36.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.601+0000 7fd1d0b8f640 1 -- 172.21.15.16:0/2934483527 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1cc1a7260 con 0x7fd1cc102790 2024-09-12T02:08:36.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.603+0000 7fd1c37fe640 1 -- 172.21.15.16:0/2934483527 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fd1b4017070 con 0x7fd1cc102790 2024-09-12T02:08:36.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.604+0000 7fd1d2f68740 1 Processor -- start 2024-09-12T02:08:36.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.604+0000 7fd1d2f68740 1 -- start start 2024-09-12T02:08:36.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.604+0000 7fd1d2f68740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c88c124a0 0x556c88c12880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:36.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.604+0000 7fd1d2f68740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556c88c12dc0 0x556c88c1d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:36.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.604+0000 7fd1d2f68740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556c88c1d680 0x556c88c1fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:36.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.604+0000 7fd1d2f68740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x556c88b782e0 con 0x556c88c124a0 2024-09-12T02:08:36.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.604+0000 7fd1d2f68740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x556c88b1fce0 con 0x556c88c12dc0 2024-09-12T02:08:36.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.604+0000 7fd1d2f68740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x556c88b742a0 con 0x556c88c1d680 2024-09-12T02:08:36.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.604+0000 7fd1cad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c88c124a0 0x556c88c12880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:36.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.604+0000 7fd1cad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c88c124a0 0x556c88c12880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55418/0 (socket says 172.21.15.16:55418) 2024-09-12T02:08:36.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.604+0000 7fd1cad9d640 1 -- 172.21.15.16:0/3258682623 learned_addr learned my addr 172.21.15.16:0/3258682623 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:08:36.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.604+0000 7fd1c37fe640 1 --2- 172.21.15.16:0/2934483527 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fd1b80a2c00 0x7fd1b80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:36.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.604+0000 7fd1ca59c640 1 --2- 172.21.15.16:0/3258682623 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556c88c12dc0 0x556c88c1d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:36.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.605+0000 7fd1c3fff640 1 --2- 172.21.15.16:0/3258682623 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556c88c1d680 0x556c88c1fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:36.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.605+0000 7fd1cad9d640 1 --2- 172.21.15.16:0/2934483527 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fd1b80a2c00 0x7fd1b80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:36.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.605+0000 7fd1c37fe640 1 -- 172.21.15.16:0/2934483527 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fd1b40ce310 con 0x7fd1cc102790 2024-09-12T02:08:36.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.605+0000 7fd1cad9d640 1 -- 172.21.15.16:0/3258682623 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556c88c1d680 msgr2=0x556c88c1fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:36.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.605+0000 7fd1cad9d640 1 --2- 172.21.15.16:0/3258682623 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556c88c1d680 0x556c88c1fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:36.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.605+0000 7fd1cad9d640 1 -- 172.21.15.16:0/3258682623 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556c88c12dc0 msgr2=0x556c88c1d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:36.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.605+0000 7fd1cad9d640 1 --2- 172.21.15.16:0/3258682623 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556c88c12dc0 0x556c88c1d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:36.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.605+0000 7fd1cad9d640 1 -- 172.21.15.16:0/3258682623 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c88b04440 con 0x556c88c124a0 2024-09-12T02:08:36.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.605+0000 7fd1ca59c640 1 --2- 172.21.15.16:0/3258682623 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556c88c12dc0 0x556c88c1d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:08:36.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.605+0000 7fd1cad9d640 1 --2- 172.21.15.16:0/3258682623 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c88c124a0 0x556c88c12880 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7fd1c400cbf0 tx=0x7fd1c4007680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5f911596de38e67 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:36.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.605+0000 7fd1c0ff9640 1 -- 172.21.15.16:0/3258682623 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1c4015070 con 0x556c88c124a0 2024-09-12T02:08:36.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.605+0000 7fd1c0ff9640 1 -- 172.21.15.16:0/3258682623 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1c4016040 con 0x556c88c124a0 2024-09-12T02:08:36.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.605+0000 7fd1c0ff9640 1 -- 172.21.15.16:0/3258682623 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1c401e020 con 0x556c88c124a0 2024-09-12T02:08:36.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.606+0000 7fd1d2f68740 1 -- 172.21.15.16:0/3258682623 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c88c124a0 msgr2=0x556c88c12880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:36.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.606+0000 7fd1cad9d640 1 --2- 172.21.15.16:0/2934483527 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fd1b80a2c00 0x7fd1b80a50c0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fd1c401f000 tx=0x7fd1c401b000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:36.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.606+0000 7fd1d2f68740 1 --2- 172.21.15.16:0/3258682623 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c88c124a0 0x556c88c12880 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7fd1c400cbf0 tx=0x7fd1c4007680 comp rx=0 tx=0).stop 2024-09-12T02:08:36.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.606+0000 7fd1d2f68740 1 -- 172.21.15.16:0/3258682623 shutdown_connections 2024-09-12T02:08:36.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.606+0000 7fd1d2f68740 1 --2- 172.21.15.16:0/3258682623 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556c88c1d680 0x556c88c1fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:36.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.606+0000 7fd1d2f68740 1 --2- 172.21.15.16:0/3258682623 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556c88c12dc0 0x556c88c1d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:36.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.606+0000 7fd1d2f68740 1 --2- 172.21.15.16:0/3258682623 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c88c124a0 0x556c88c12880 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:36.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.606+0000 7fd1d2f68740 1 -- 172.21.15.16:0/3258682623 >> 172.21.15.16:0/3258682623 conn(0x556c88c0ea30 msgr2=0x556c88c0ee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:36.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.606+0000 7fd1d2f68740 1 -- 172.21.15.16:0/3258682623 shutdown_connections 2024-09-12T02:08:36.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.606+0000 7fd1d2f68740 1 -- 172.21.15.16:0/3258682623 wait complete. 2024-09-12T02:08:36.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.606+0000 7fd1d2f68740 1 -- start start 2024-09-12T02:08:36.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.606+0000 7fd1d2f68740 1 Processor -- start 2024-09-12T02:08:36.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.607+0000 7fd1d2f68740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556c88c124a0 0x556c88c1cf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:36.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.607+0000 7fd1d2f68740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556c88c12dc0 0x556c88c1a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:36.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.607+0000 7fd1d2f68740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c88c1d680 0x556c88c1a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:36.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.607+0000 7fd1d2f68740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x556c88c20250 con 0x556c88c1d680 2024-09-12T02:08:36.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.607+0000 7fd1d2f68740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x556c88b33050 con 0x556c88c124a0 2024-09-12T02:08:36.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.607+0000 7fd1d2f68740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x556c88c203d0 con 0x556c88c12dc0 2024-09-12T02:08:36.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.607+0000 7fd1d2f68740 10 client.0 _ll_register_callbacks cb 0x556c88b73160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:08:36.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.607+0000 7fd1cad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c88c1d680 0x556c88c1a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:36.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.607+0000 7fd1cad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c88c1d680 0x556c88c1a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55432/0 (socket says 172.21.15.16:55432) 2024-09-12T02:08:36.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.607+0000 7fd1cad9d640 1 -- 172.21.15.16:0/2182820552 learned_addr learned my addr 172.21.15.16:0/2182820552 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:08:36.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.607+0000 7fd1c3fff640 1 --2- 172.21.15.16:0/2182820552 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556c88c12dc0 0x556c88c1a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:36.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.607+0000 7fd1ca59c640 1 --2- 172.21.15.16:0/2182820552 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556c88c124a0 0x556c88c1cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:36.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.607+0000 7fd1cad9d640 1 -- 172.21.15.16:0/2182820552 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556c88c12dc0 msgr2=0x556c88c1a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:36.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.607+0000 7fd1cad9d640 1 --2- 172.21.15.16:0/2182820552 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556c88c12dc0 0x556c88c1a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:36.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.607+0000 7fd1cad9d640 1 -- 172.21.15.16:0/2182820552 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556c88c124a0 msgr2=0x556c88c1cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:36.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.607+0000 7fd1cad9d640 1 --2- 172.21.15.16:0/2182820552 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556c88c124a0 0x556c88c1cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:36.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.607+0000 7fd1cad9d640 1 -- 172.21.15.16:0/2182820552 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c888e93c0 con 0x556c88c1d680 2024-09-12T02:08:36.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.607+0000 7fd1cad9d640 1 --2- 172.21.15.16:0/2182820552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c88c1d680 0x556c88c1a730 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7fd1c4009530 tx=0x7fd1c400a510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81f544c11316f3a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:36.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.607+0000 7fd1b1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:08:36.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.607+0000 7fd1b1ffb640 1 -- 172.21.15.16:0/2182820552 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1c4031060 con 0x556c88c1d680 2024-09-12T02:08:36.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.608+0000 7fd1b1ffb640 1 -- 172.21.15.16:0/2182820552 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fd1c400a730 con 0x556c88c1d680 2024-09-12T02:08:36.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.608+0000 7fd1d2f68740 1 -- 172.21.15.16:0/2182820552 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd1c4016040 con 0x556c88c1d680 2024-09-12T02:08:36.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.608+0000 7fd1b1ffb640 1 -- 172.21.15.16:0/2182820552 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd1c4015070 con 0x556c88c1d680 2024-09-12T02:08:36.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.608+0000 7fd1d2f68740 10 client.17397 fetch_fsmap learned FSMap version 25 2024-09-12T02:08:36.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.608+0000 7fd1d2f68740 1 -- 172.21.15.16:0/2182820552 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x556c88c2bba0 con 0x556c88c1d680 2024-09-12T02:08:36.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.608+0000 7fd1b1ffb640 1 -- 172.21.15.16:0/2182820552 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1c4016040 con 0x556c88c1d680 2024-09-12T02:08:36.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.608+0000 7fd1b1ffb640 1 -- 172.21.15.16:0/2182820552 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fd1c40235e0 con 0x556c88c1d680 2024-09-12T02:08:36.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.608+0000 7fd1d2f68740 10 client.17397 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:08:36.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.608+0000 7fd1d2f68740 10 client.17397 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:08:36.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.608+0000 7fd1d2f68740 20 client.17397 populate_metadata read hostname 'smithi016' 2024-09-12T02:08:36.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.608+0000 7fd1d2f68740 1 --2- 172.21.15.16:0/2182820552 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x556c88c2cf50 0x556c88c2f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:36.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.608+0000 7fd1d2f68740 4 client.17397 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:08:36.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.608+0000 7fd1d2f68740 1 -- 172.21.15.16:0/2182820552 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556c88b85850 con 0x556c88c2cf50 2024-09-12T02:08:36.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.609+0000 7fd1ca59c640 1 --2- 172.21.15.16:0/2182820552 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x556c88c2cf50 0x556c88c2f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:36.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.609+0000 7fd1ca59c640 1 --2- 172.21.15.16:0/2182820552 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x556c88c2cf50 0x556c88c2f340 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1e91a1d15c21d3f server_cookie=c2c54776c09ba407 in_seq=0 out_seq=0 2024-09-12T02:08:36.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.609+0000 7fd1b1ffb640 10 client.17397 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:08:36.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.610+0000 7fd1b1ffb640 1 -- 172.21.15.16:0/2182820552 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556c88b85850 con 0x556c88c2cf50 2024-09-12T02:08:36.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.610+0000 7fd1b1ffb640 10 client.17397 handle_command_reply: tid=0 2024-09-12T02:08:36.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.610+0000 7fd1d2f68740 1 client.17397 shutdown 2024-09-12T02:08:36.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.610+0000 7fd1d2f68740 1 -- 172.21.15.16:0/2182820552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c88c1d680 msgr2=0x556c88c1a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:36.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.610+0000 7fd1d2f68740 1 --2- 172.21.15.16:0/2182820552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c88c1d680 0x556c88c1a730 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7fd1c4009530 tx=0x7fd1c400a510 comp rx=0 tx=0).stop 2024-09-12T02:08:36.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.610+0000 7fd1d2f68740 1 -- 172.21.15.16:0/2182820552 shutdown_connections 2024-09-12T02:08:36.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.610+0000 7fd1d2f68740 1 --2- 172.21.15.16:0/2182820552 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x556c88c2cf50 0x556c88c2f340 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:36.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.610+0000 7fd1d2f68740 1 --2- 172.21.15.16:0/2182820552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c88c1d680 0x556c88c1a730 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:36.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.610+0000 7fd1d2f68740 1 --2- 172.21.15.16:0/2182820552 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556c88c12dc0 0x556c88c1a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:36.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.610+0000 7fd1d2f68740 1 --2- 172.21.15.16:0/2182820552 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556c88c124a0 0x556c88c1cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:36.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.610+0000 7fd1b1ffb640 0 client.17397 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:08:36.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.610+0000 7fd1d2f68740 1 -- 172.21.15.16:0/2182820552 >> 172.21.15.16:0/2182820552 conn(0x556c88c0ea30 msgr2=0x556c88c0db90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:36.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.610+0000 7fd1d2f68740 1 -- 172.21.15.16:0/2182820552 shutdown_connections 2024-09-12T02:08:36.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.610+0000 7fd1d2f68740 1 -- 172.21.15.16:0/2182820552 wait complete. 2024-09-12T02:08:36.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.610+0000 7fd1d2f68740 20 client.17397 trim_cache size 0 max 16384 2024-09-12T02:08:36.635 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:08:36 smithi184 ceph-mon[35593]: pgmap v535: 97 pgs: 97 active+clean; 3.7 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 14 op/s 2024-09-12T02:08:36.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.627+0000 7fd1d2f68740 1 Processor -- start 2024-09-12T02:08:36.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.627+0000 7fd1d2f68740 1 -- start start 2024-09-12T02:08:36.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.628+0000 7fd1d2f68740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556c88c124a0 0x556c88c294b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:36.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.628+0000 7fd1d2f68740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c88c12dc0 0x556c88c191d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:36.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.628+0000 7fd1d2f68740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556c88c1d680 0x556c88c34940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:36.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.628+0000 7fd1d2f68740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd1cc1b2000 con 0x556c88c12dc0 2024-09-12T02:08:36.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.628+0000 7fd1d2f68740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd1cc1af8d0 con 0x556c88c124a0 2024-09-12T02:08:36.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.628+0000 7fd1d2f68740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd1cc1ad720 con 0x556c88c1d680 2024-09-12T02:08:36.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.628+0000 7fd1c3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c88c12dc0 0x556c88c191d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:36.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.628+0000 7fd1c3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c88c12dc0 0x556c88c191d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55448/0 (socket says 172.21.15.16:55448) 2024-09-12T02:08:36.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.628+0000 7fd1c3fff640 1 -- 172.21.15.16:0/2097425029 learned_addr learned my addr 172.21.15.16:0/2097425029 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:08:36.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.628+0000 7fd1cad9d640 1 --2- 172.21.15.16:0/2097425029 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556c88c1d680 0x556c88c34940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:36.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.628+0000 7fd1ca59c640 1 --2- 172.21.15.16:0/2097425029 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556c88c124a0 0x556c88c294b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:36.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.628+0000 7fd1c3fff640 1 -- 172.21.15.16:0/2097425029 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556c88c1d680 msgr2=0x556c88c34940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:36.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.628+0000 7fd1c3fff640 1 --2- 172.21.15.16:0/2097425029 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556c88c1d680 0x556c88c34940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:36.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.628+0000 7fd1c3fff640 1 -- 172.21.15.16:0/2097425029 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556c88c124a0 msgr2=0x556c88c294b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:36.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.628+0000 7fd1c3fff640 1 --2- 172.21.15.16:0/2097425029 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556c88c124a0 0x556c88c294b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:36.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.628+0000 7fd1c3fff640 1 -- 172.21.15.16:0/2097425029 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c88c12bf0 con 0x556c88c12dc0 2024-09-12T02:08:36.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.628+0000 7fd1c3fff640 1 --2- 172.21.15.16:0/2097425029 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c88c12dc0 0x556c88c191d0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7fd1b400be60 tx=0x7fd1b4013310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81afa0a0dbf8e076 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:36.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.628+0000 7fd1b3fff640 1 -- 172.21.15.16:0/2097425029 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1b401e040 con 0x556c88c12dc0 2024-09-12T02:08:36.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.629+0000 7fd1b3fff640 1 -- 172.21.15.16:0/2097425029 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1b409ef20 con 0x556c88c12dc0 2024-09-12T02:08:36.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.629+0000 7fd1b3fff640 1 -- 172.21.15.16:0/2097425029 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1b40971e0 con 0x556c88c12dc0 2024-09-12T02:08:36.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.629+0000 7fd1d2f68740 1 -- 172.21.15.16:0/2097425029 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c88c12dc0 msgr2=0x556c88c191d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:36.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.629+0000 7fd1d2f68740 1 --2- 172.21.15.16:0/2097425029 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c88c12dc0 0x556c88c191d0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7fd1b400be60 tx=0x7fd1b4013310 comp rx=0 tx=0).stop 2024-09-12T02:08:36.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.629+0000 7fd1d2f68740 1 -- 172.21.15.16:0/2097425029 shutdown_connections 2024-09-12T02:08:36.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.629+0000 7fd1d2f68740 1 --2- 172.21.15.16:0/2097425029 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556c88c1d680 0x556c88c34940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:36.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.629+0000 7fd1d2f68740 1 --2- 172.21.15.16:0/2097425029 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c88c12dc0 0x556c88c191d0 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:36.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.629+0000 7fd1d2f68740 1 --2- 172.21.15.16:0/2097425029 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556c88c124a0 0x556c88c294b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:36.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.629+0000 7fd1d2f68740 1 -- 172.21.15.16:0/2097425029 >> 172.21.15.16:0/2097425029 conn(0x556c88c0ea30 msgr2=0x556c88c18ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:36.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.629+0000 7fd1d2f68740 1 -- 172.21.15.16:0/2097425029 shutdown_connections 2024-09-12T02:08:36.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.629+0000 7fd1d2f68740 1 -- 172.21.15.16:0/2097425029 wait complete. 2024-09-12T02:08:36.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.630+0000 7fd1d2f68740 1 -- start start 2024-09-12T02:08:36.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.630+0000 7fd1d2f68740 1 Processor -- start 2024-09-12T02:08:36.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.630+0000 7fd1d2f68740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c88c124a0 0x556c88c25e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:36.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.630+0000 7fd1d2f68740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556c88c12dc0 0x556c88c21180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:36.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.630+0000 7fd1d2f68740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556c88c1d680 0x556c88c21770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:36.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.630+0000 7fd1d2f68740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd1cc1add90 con 0x556c88c124a0 2024-09-12T02:08:36.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.630+0000 7fd1d2f68740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd1cc1af200 con 0x556c88c12dc0 2024-09-12T02:08:36.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.630+0000 7fd1d2f68740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd1cc1ae460 con 0x556c88c1d680 2024-09-12T02:08:36.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.630+0000 7fd1d2f68740 10 client.0 _ll_register_callbacks cb 0x7fd1c400cf50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:08:36.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.630+0000 7fd1ca59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c88c124a0 0x556c88c25e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:36.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.630+0000 7fd1ca59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c88c124a0 0x556c88c25e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55464/0 (socket says 172.21.15.16:55464) 2024-09-12T02:08:36.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.630+0000 7fd1ca59c640 1 -- 172.21.15.16:0/3281858938 learned_addr learned my addr 172.21.15.16:0/3281858938 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:08:36.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.630+0000 7fd1cad9d640 1 --2- 172.21.15.16:0/3281858938 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556c88c1d680 0x556c88c21770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:36.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.630+0000 7fd1c3fff640 1 --2- 172.21.15.16:0/3281858938 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556c88c12dc0 0x556c88c21180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:36.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.630+0000 7fd1ca59c640 1 -- 172.21.15.16:0/3281858938 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556c88c1d680 msgr2=0x556c88c21770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:36.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.630+0000 7fd1ca59c640 1 --2- 172.21.15.16:0/3281858938 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556c88c1d680 0x556c88c21770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:36.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.630+0000 7fd1ca59c640 1 -- 172.21.15.16:0/3281858938 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556c88c12dc0 msgr2=0x556c88c21180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:36.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.630+0000 7fd1ca59c640 1 --2- 172.21.15.16:0/3281858938 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556c88c12dc0 0x556c88c21180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:36.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.630+0000 7fd1ca59c640 1 -- 172.21.15.16:0/3281858938 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c88c1d2b0 con 0x556c88c124a0 2024-09-12T02:08:36.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.630+0000 7fd1c3fff640 1 --2- 172.21.15.16:0/3281858938 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556c88c12dc0 0x556c88c21180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:08:36.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.631+0000 7fd1ca59c640 1 --2- 172.21.15.16:0/3281858938 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c88c124a0 0x556c88c25e50 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7fd1bc02a930 tx=0x7fd1bc000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=286c7ef5b455003a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:36.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.631+0000 7fd1b1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:08:36.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.631+0000 7fd1b1ffb640 1 -- 172.21.15.16:0/3281858938 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1bc049070 con 0x556c88c124a0 2024-09-12T02:08:36.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.631+0000 7fd1d2f68740 1 -- 172.21.15.16:0/3281858938 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd1b409ef20 con 0x556c88c124a0 2024-09-12T02:08:36.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.631+0000 7fd1b1ffb640 1 -- 172.21.15.16:0/3281858938 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fd1bc004450 con 0x556c88c124a0 2024-09-12T02:08:36.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.631+0000 7fd1b1ffb640 1 -- 172.21.15.16:0/3281858938 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd1bc035040 con 0x556c88c124a0 2024-09-12T02:08:36.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.631+0000 7fd1d2f68740 10 client.17403 fetch_fsmap learned FSMap version 25 2024-09-12T02:08:36.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.631+0000 7fd1d2f68740 1 -- 172.21.15.16:0/3281858938 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x556c88c97880 con 0x556c88c124a0 2024-09-12T02:08:36.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.631+0000 7fd1b1ffb640 1 -- 172.21.15.16:0/3281858938 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1b409ef20 con 0x556c88c124a0 2024-09-12T02:08:36.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.631+0000 7fd1b1ffb640 1 -- 172.21.15.16:0/3281858938 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fd1bc004c50 con 0x556c88c124a0 2024-09-12T02:08:36.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.631+0000 7fd1d2f68740 10 client.17403 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:08:36.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.631+0000 7fd1d2f68740 10 client.17403 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:08:36.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.631+0000 7fd1d2f68740 20 client.17403 populate_metadata read hostname 'smithi016' 2024-09-12T02:08:36.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.631+0000 7fd1d2f68740 1 --2- 172.21.15.16:0/3281858938 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x556c88c97fd0 0x556c88c9a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:36.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.631+0000 7fd1d2f68740 4 client.17403 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T02:08:36.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.631+0000 7fd1d2f68740 1 -- 172.21.15.16:0/3281858938 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556c88b7fd60 con 0x556c88c97fd0 2024-09-12T02:08:36.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.632+0000 7fd1c3fff640 1 --2- 172.21.15.16:0/3281858938 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x556c88c97fd0 0x556c88c9a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:36.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.632+0000 7fd1c3fff640 1 --2- 172.21.15.16:0/3281858938 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x556c88c97fd0 0x556c88c9a3f0 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8985f7714e8ec0f server_cookie=a1b484579de9bada in_seq=0 out_seq=0 2024-09-12T02:08:36.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.632+0000 7fd1b1ffb640 10 client.17403 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:08:36.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.632+0000 7fd1b1ffb640 1 -- 172.21.15.16:0/3281858938 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x556c88b7fd60 con 0x556c88c97fd0 2024-09-12T02:08:36.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.632+0000 7fd1b1ffb640 10 client.17403 handle_command_reply: tid=0 2024-09-12T02:08:36.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.632+0000 7fd1d2f68740 1 client.17403 shutdown 2024-09-12T02:08:36.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.632+0000 7fd1d2f68740 1 -- 172.21.15.16:0/3281858938 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c88c124a0 msgr2=0x556c88c25e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:36.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.632+0000 7fd1d2f68740 1 --2- 172.21.15.16:0/3281858938 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c88c124a0 0x556c88c25e50 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7fd1bc02a930 tx=0x7fd1bc000bc0 comp rx=0 tx=0).stop 2024-09-12T02:08:36.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.632+0000 7fd1d2f68740 1 -- 172.21.15.16:0/3281858938 shutdown_connections 2024-09-12T02:08:36.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.632+0000 7fd1d2f68740 1 --2- 172.21.15.16:0/3281858938 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x556c88c97fd0 0x556c88c9a3f0 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:36.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.632+0000 7fd1d2f68740 1 --2- 172.21.15.16:0/3281858938 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556c88c1d680 0x556c88c21770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:36.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.632+0000 7fd1d2f68740 1 --2- 172.21.15.16:0/3281858938 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556c88c12dc0 0x556c88c21180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:36.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.632+0000 7fd1d2f68740 1 --2- 172.21.15.16:0/3281858938 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c88c124a0 0x556c88c25e50 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:36.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.632+0000 7fd1b1ffb640 0 client.17403 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:08:36.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.632+0000 7fd1d2f68740 1 -- 172.21.15.16:0/3281858938 >> 172.21.15.16:0/3281858938 conn(0x556c88c0ea30 msgr2=0x556c88c192e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:36.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.633+0000 7fd1d2f68740 1 -- 172.21.15.16:0/3281858938 shutdown_connections 2024-09-12T02:08:36.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.633+0000 7fd1d2f68740 1 -- 172.21.15.16:0/3281858938 wait complete. 2024-09-12T02:08:36.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.633+0000 7fd1d2f68740 20 client.17403 trim_cache size 0 max 16384 2024-09-12T02:08:36.646 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T02:08:36.646 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-09-12T02:08:36.646 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {} 2024-09-12T02:08:36.646 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T02:08:36.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.633+0000 7fd1d0b8f640 1 -- 172.21.15.16:0/2934483527 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fd1b80a2c00 msgr2=0x7fd1b80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:36.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.633+0000 7fd1d0b8f640 1 --2- 172.21.15.16:0/2934483527 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fd1b80a2c00 0x7fd1b80a50c0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fd1c401f000 tx=0x7fd1c401b000 comp rx=0 tx=0).stop 2024-09-12T02:08:36.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.633+0000 7fd1d0b8f640 1 -- 172.21.15.16:0/2934483527 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1cc102790 msgr2=0x7fd1cc074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:36.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.633+0000 7fd1d0b8f640 1 --2- 172.21.15.16:0/2934483527 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1cc102790 0x7fd1cc074a30 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7fd1b400b4a0 tx=0x7fd1b400b960 comp rx=0 tx=0).stop 2024-09-12T02:08:36.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.633+0000 7fd1d0b8f640 1 -- 172.21.15.16:0/2934483527 shutdown_connections 2024-09-12T02:08:36.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.633+0000 7fd1d0b8f640 1 --2- 172.21.15.16:0/2934483527 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fd1b80a2c00 0x7fd1b80a50c0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:36.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.633+0000 7fd1d0b8f640 1 --2- 172.21.15.16:0/2934483527 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd1cc106f10 0x7fd1cc074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:36.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.633+0000 7fd1d0b8f640 1 --2- 172.21.15.16:0/2934483527 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1cc102790 0x7fd1cc074a30 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:36.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.633+0000 7fd1d0b8f640 1 --2- 172.21.15.16:0/2934483527 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd1cc101de0 0x7fd1cc0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:36.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.633+0000 7fd1d0b8f640 1 -- 172.21.15.16:0/2934483527 >> 172.21.15.16:0/2934483527 conn(0x7fd1cc0fd8c0 msgr2=0x7fd1cc0fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:36.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.633+0000 7fd1d0b8f640 1 -- 172.21.15.16:0/2934483527 shutdown_connections 2024-09-12T02:08:36.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:36.633+0000 7fd1d0b8f640 1 -- 172.21.15.16:0/2934483527 wait complete. 2024-09-12T02:08:36.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:08:36 smithi033 ceph-mon[35332]: pgmap v535: 97 pgs: 97 active+clean; 3.7 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 14 op/s 2024-09-12T02:08:37.453 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-12T02:08:37.453 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:08:37.712 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:08:38.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:38.172+0000 7f8553437640 1 -- 172.21.15.16:0/908874725 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f854c1036e0 msgr2=0x7f854c107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:38.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:38.172+0000 7f8553437640 1 --2- 172.21.15.16:0/908874725 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f854c1036e0 0x7f854c107b90 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f8540009930 tx=0x7f854002a420 comp rx=0 tx=0).stop 2024-09-12T02:08:38.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:38.172+0000 7f8553437640 1 -- 172.21.15.16:0/908874725 shutdown_connections 2024-09-12T02:08:38.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:38.172+0000 7f8553437640 1 --2- 172.21.15.16:0/908874725 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f854c1082a0 0x7f854c10ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:38.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:38.172+0000 7f8553437640 1 --2- 172.21.15.16:0/908874725 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f854c1036e0 0x7f854c107b90 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:38.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:38.172+0000 7f8553437640 1 --2- 172.21.15.16:0/908874725 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f854c102d30 0x7f854c103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:38.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:38.172+0000 7f8553437640 1 -- 172.21.15.16:0/908874725 >> 172.21.15.16:0/908874725 conn(0x7f854c0fc7d0 msgr2=0x7f854c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:38.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:38.172+0000 7f8553437640 1 -- 172.21.15.16:0/908874725 shutdown_connections 2024-09-12T02:08:38.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:38.172+0000 7f8553437640 1 -- 172.21.15.16:0/908874725 wait complete. 2024-09-12T02:08:38.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:38.173+0000 7f8553437640 1 Processor -- start 2024-09-12T02:08:38.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:38.173+0000 7f8553437640 1 -- start start 2024-09-12T02:08:38.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:38.173+0000 7f8553437640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f854c102d30 0x7f854c198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:38.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:38.173+0000 7f8553437640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f854c1036e0 0x7f854c198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:38.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:38.173+0000 7f8553437640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f854c1082a0 0x7f854c19ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:38.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:38.173+0000 7f8553437640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f854c1120a0 con 0x7f854c102d30 2024-09-12T02:08:38.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:38.173+0000 7f8553437640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f854c111f20 con 0x7f854c1036e0 2024-09-12T02:08:38.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:38.173+0000 7f8553437640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f854c112220 con 0x7f854c1082a0 2024-09-12T02:08:38.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:38.173+0000 7f85511d3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f854c102d30 0x7f854c198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:38.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:38.173+0000 7f85511d3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f854c102d30 0x7f854c198550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55480/0 (socket says 172.21.15.16:55480) 2024-09-12T02:08:38.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:38.173+0000 7f85511d3640 1 -- 172.21.15.16:0/3951526062 learned_addr learned my addr 172.21.15.16:0/3951526062 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:08:38.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:38.173+0000 7f85509d2640 1 --2- 172.21.15.16:0/3951526062 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f854c1036e0 0x7f854c198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:38.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:38.174+0000 7f85511d3640 1 -- 172.21.15.16:0/3951526062 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f854c1082a0 msgr2=0x7f854c19ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:38.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:38.174+0000 7f85519d4640 1 --2- 172.21.15.16:0/3951526062 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f854c1082a0 0x7f854c19ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:38.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:38.174+0000 7f85511d3640 1 --2- 172.21.15.16:0/3951526062 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f854c1082a0 0x7f854c19ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:38.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:38.174+0000 7f85511d3640 1 -- 172.21.15.16:0/3951526062 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f854c1036e0 msgr2=0x7f854c198a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:38.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:38.174+0000 7f85511d3640 1 --2- 172.21.15.16:0/3951526062 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f854c1036e0 0x7f854c198a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:38.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:38.174+0000 7f85511d3640 1 -- 172.21.15.16:0/3951526062 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f854c19d500 con 0x7f854c102d30 2024-09-12T02:08:38.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:38.174+0000 7f85509d2640 1 --2- 172.21.15.16:0/3951526062 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f854c1036e0 0x7f854c198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:08:38.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:38.174+0000 7f85511d3640 1 --2- 172.21.15.16:0/3951526062 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f854c102d30 0x7f854c198550 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f853c00cce0 tx=0x7f853c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:38.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:38.174+0000 7f854a7fc640 1 -- 172.21.15.16:0/3951526062 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f853c00f650 con 0x7f854c102d30 2024-09-12T02:08:38.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:38.174+0000 7f854a7fc640 1 -- 172.21.15.16:0/3951526062 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f853c00fcd0 con 0x7f854c102d30 2024-09-12T02:08:38.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:38.174+0000 7f854a7fc640 1 -- 172.21.15.16:0/3951526062 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f853c018c70 con 0x7f854c102d30 2024-09-12T02:08:38.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:38.174+0000 7f8553437640 1 -- 172.21.15.16:0/3951526062 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f854c19d7f0 con 0x7f854c102d30 2024-09-12T02:08:38.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:38.174+0000 7f8553437640 1 -- 172.21.15.16:0/3951526062 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f854c1a50d0 con 0x7f854c102d30 2024-09-12T02:08:38.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:38.175+0000 7f8553437640 1 -- 172.21.15.16:0/3951526062 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f854c103110 con 0x7f854c102d30 2024-09-12T02:08:38.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:38.176+0000 7f854a7fc640 1 -- 172.21.15.16:0/3951526062 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f853c00f7f0 con 0x7f854c102d30 2024-09-12T02:08:38.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:38.177+0000 7f854a7fc640 1 --2- 172.21.15.16:0/3951526062 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f85340a2c00 0x7f85340a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:38.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:38.177+0000 7f85509d2640 1 --2- 172.21.15.16:0/3951526062 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f85340a2c00 0x7f85340a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:38.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:38.177+0000 7f854a7fc640 1 -- 172.21.15.16:0/3951526062 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f853c0cf750 con 0x7f854c102d30 2024-09-12T02:08:38.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:38.178+0000 7f85509d2640 1 --2- 172.21.15.16:0/3951526062 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f85340a2c00 0x7f85340a50c0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f854002a940 tx=0x7f8540032d50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:38.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:38.180+0000 7f854a7fc640 1 -- 172.21.15.16:0/3951526062 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f853c014030 con 0x7f854c102d30 2024-09-12T02:08:38.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:38.393+0000 7f8553437640 1 -- 172.21.15.16:0/3951526062 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f854c1998b0 con 0x7f854c102d30 2024-09-12T02:08:38.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:08:38 smithi016 ceph-mon[30947]: pgmap v536: 97 pgs: 97 active+clean; 3.7 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-12T02:08:38.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:38.393+0000 7f854a7fc640 1 -- 172.21.15.16:0/3951526062 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11911 (secure 0 0 0) 0x7f853c09c860 con 0x7f854c102d30 2024-09-12T02:08:38.394 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T02:08:38.394 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":25,"btime":"2024-09-12T02:01:37:192021+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3279123279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3279123279},{"type":"v1","addr":"172.21.15.16:6835","nonce":3279123279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4119452382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4119452382},{"type":"v1","addr":"172.21.15.16:6837","nonce":4119452382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/3263823165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3263823165},{"type":"v1","addr":"172.21.15.16:6839","nonce":3263823165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/1510290732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":1510290732},{"type":"v1","addr":"172.21.15.16:6841","nonce":1510290732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/279674603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":279674603},{"type":"v1","addr":"172.21.15.33:6833","nonce":279674603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3555728028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3555728028},{"type":"v1","addr":"172.21.15.33:6835","nonce":3555728028}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T01:59:56.117608+0000","modified":"2024-09-12T02:01:36.294760+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24482,"mds_1":24481,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6833/1563374106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1563374106},{"type":"v1","addr":"172.21.15.184:6833","nonce":1563374106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6837/1892833868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1892833868},{"type":"v1","addr":"172.21.15.33:6837","nonce":1892833868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/1457679686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1457679686},{"type":"v1","addr":"172.21.15.184:6835","nonce":1457679686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.184:6837/1793393087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":1793393087},{"type":"v1","addr":"172.21.15.184:6837","nonce":1793393087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.184:6839/4203347941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":4203347941},{"type":"v1","addr":"172.21.15.184:6839","nonce":4203347941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.33:6839/4136670204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":4136670204},{"type":"v1","addr":"172.21.15.33:6839","nonce":4136670204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24481,24482,24469]},"id":1}]} 2024-09-12T02:08:38.394 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 25 2024-09-12T02:08:38.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:38.395+0000 7f8553437640 1 -- 172.21.15.16:0/3951526062 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f85340a2c00 msgr2=0x7f85340a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:38.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:38.395+0000 7f8553437640 1 --2- 172.21.15.16:0/3951526062 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f85340a2c00 0x7f85340a50c0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f854002a940 tx=0x7f8540032d50 comp rx=0 tx=0).stop 2024-09-12T02:08:38.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:38.395+0000 7f8553437640 1 -- 172.21.15.16:0/3951526062 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f854c102d30 msgr2=0x7f854c198550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:38.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:38.395+0000 7f8553437640 1 --2- 172.21.15.16:0/3951526062 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f854c102d30 0x7f854c198550 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f853c00cce0 tx=0x7f853c00a620 comp rx=0 tx=0).stop 2024-09-12T02:08:38.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:38.396+0000 7f8553437640 1 -- 172.21.15.16:0/3951526062 shutdown_connections 2024-09-12T02:08:38.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:38.396+0000 7f8553437640 1 --2- 172.21.15.16:0/3951526062 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f85340a2c00 0x7f85340a50c0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:38.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:38.396+0000 7f8553437640 1 --2- 172.21.15.16:0/3951526062 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f854c1082a0 0x7f854c19ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:38.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:38.396+0000 7f8553437640 1 --2- 172.21.15.16:0/3951526062 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f854c1036e0 0x7f854c198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:38.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:38.396+0000 7f8553437640 1 --2- 172.21.15.16:0/3951526062 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f854c102d30 0x7f854c198550 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:38.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:38.396+0000 7f8553437640 1 -- 172.21.15.16:0/3951526062 >> 172.21.15.16:0/3951526062 conn(0x7f854c0fc7d0 msgr2=0x7f854c105d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:38.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:38.396+0000 7f8553437640 1 -- 172.21.15.16:0/3951526062 shutdown_connections 2024-09-12T02:08:38.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:38.396+0000 7f8553437640 1 -- 172.21.15.16:0/3951526062 wait complete. 2024-09-12T02:08:38.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:08:38 smithi184 ceph-mon[35593]: pgmap v536: 97 pgs: 97 active+clean; 3.7 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-12T02:08:38.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:08:38 smithi033 ceph-mon[35332]: pgmap v536: 97 pgs: 97 active+clean; 3.7 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-12T02:08:39.129 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-12T02:08:39.391 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:08:39.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:08:39 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/3951526062' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:08:39.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:08:39 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/3951526062' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:08:39.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:08:39 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/3951526062' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:08:39.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.858+0000 7f1272d8c640 1 -- 172.21.15.16:0/1052183875 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f126c102790 msgr2=0x7f126c1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:39.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.858+0000 7f1272d8c640 1 --2- 172.21.15.16:0/1052183875 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f126c102790 0x7f126c1067e0 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f125c009900 tx=0x7f125c02a410 comp rx=0 tx=0).stop 2024-09-12T02:08:39.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.858+0000 7f1272d8c640 1 -- 172.21.15.16:0/1052183875 shutdown_connections 2024-09-12T02:08:39.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.858+0000 7f1272d8c640 1 --2- 172.21.15.16:0/1052183875 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f126c106f10 0x7f126c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:39.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.858+0000 7f1272d8c640 1 --2- 172.21.15.16:0/1052183875 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f126c102790 0x7f126c1067e0 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:39.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.858+0000 7f1272d8c640 1 --2- 172.21.15.16:0/1052183875 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f126c101de0 0x7f126c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:39.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.858+0000 7f1272d8c640 1 -- 172.21.15.16:0/1052183875 >> 172.21.15.16:0/1052183875 conn(0x7f126c0fd8c0 msgr2=0x7f126c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:39.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.858+0000 7f1272d8c640 1 -- 172.21.15.16:0/1052183875 shutdown_connections 2024-09-12T02:08:39.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.858+0000 7f1272d8c640 1 -- 172.21.15.16:0/1052183875 wait complete. 2024-09-12T02:08:39.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.859+0000 7f1272d8c640 1 Processor -- start 2024-09-12T02:08:39.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.859+0000 7f1272d8c640 1 -- start start 2024-09-12T02:08:39.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.859+0000 7f1272d8c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f126c101de0 0x7f126c198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:39.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.859+0000 7f1272d8c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f126c102790 0x7f126c198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:39.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.859+0000 7f1272d8c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f126c106f10 0x7f126c19cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:39.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.859+0000 7f1272d8c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f126c111f30 con 0x7f126c106f10 2024-09-12T02:08:39.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.859+0000 7f1272d8c640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f126c111db0 con 0x7f126c101de0 2024-09-12T02:08:39.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.859+0000 7f1272d8c640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f126c1120b0 con 0x7f126c102790 2024-09-12T02:08:39.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.859+0000 7f1271329640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f126c106f10 0x7f126c19cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:39.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.859+0000 7f1271329640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f126c106f10 0x7f126c19cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55494/0 (socket says 172.21.15.16:55494) 2024-09-12T02:08:39.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.859+0000 7f1271329640 1 -- 172.21.15.16:0/1215765705 learned_addr learned my addr 172.21.15.16:0/1215765705 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:08:39.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.860+0000 7f1270b28640 1 --2- 172.21.15.16:0/1215765705 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f126c101de0 0x7f126c198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:39.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.860+0000 7f1271329640 1 -- 172.21.15.16:0/1215765705 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f126c102790 msgr2=0x7f126c198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:39.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.860+0000 7f126bfff640 1 --2- 172.21.15.16:0/1215765705 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f126c102790 0x7f126c198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:39.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.860+0000 7f1271329640 1 --2- 172.21.15.16:0/1215765705 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f126c102790 0x7f126c198950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:39.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.860+0000 7f1271329640 1 -- 172.21.15.16:0/1215765705 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f126c101de0 msgr2=0x7f126c198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:39.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.860+0000 7f1271329640 1 --2- 172.21.15.16:0/1215765705 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f126c101de0 0x7f126c198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:39.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.860+0000 7f1271329640 1 -- 172.21.15.16:0/1215765705 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f126c19d3c0 con 0x7f126c106f10 2024-09-12T02:08:39.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.860+0000 7f1270b28640 1 --2- 172.21.15.16:0/1215765705 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f126c101de0 0x7f126c198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:08:39.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.860+0000 7f1271329640 1 --2- 172.21.15.16:0/1215765705 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f126c106f10 0x7f126c19cce0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f126400cad0 tx=0x7f126400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:39.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.860+0000 7f1269ffb640 1 -- 172.21.15.16:0/1215765705 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f126400ae20 con 0x7f126c106f10 2024-09-12T02:08:39.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.860+0000 7f1269ffb640 1 -- 172.21.15.16:0/1215765705 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1264005020 con 0x7f126c106f10 2024-09-12T02:08:39.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.860+0000 7f1272d8c640 1 -- 172.21.15.16:0/1215765705 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f126c19d6b0 con 0x7f126c106f10 2024-09-12T02:08:39.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.860+0000 7f1269ffb640 1 -- 172.21.15.16:0/1215765705 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f126400f6f0 con 0x7f126c106f10 2024-09-12T02:08:39.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.860+0000 7f1272d8c640 1 -- 172.21.15.16:0/1215765705 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f126c1a4f90 con 0x7f126c106f10 2024-09-12T02:08:39.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.862+0000 7f1269ffb640 1 -- 172.21.15.16:0/1215765705 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f12640051d0 con 0x7f126c106f10 2024-09-12T02:08:39.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.863+0000 7f1275165740 1 Processor -- start 2024-09-12T02:08:39.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.863+0000 7f1275165740 1 -- start start 2024-09-12T02:08:39.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.863+0000 7f1275165740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c4dc522b40 0x55c4dc521090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:39.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.863+0000 7f1275165740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4dc5215d0 0x55c4dc521a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:39.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.863+0000 7f1275165740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c4dc52c370 0x55c4dc52e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:39.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.863+0000 7f1275165740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55c4dc4872f0 con 0x55c4dc5215d0 2024-09-12T02:08:39.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.863+0000 7f1275165740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55c4dc42ecf0 con 0x55c4dc52c370 2024-09-12T02:08:39.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.863+0000 7f1275165740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55c4dc4832b0 con 0x55c4dc522b40 2024-09-12T02:08:39.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.863+0000 7f1269ffb640 1 --2- 172.21.15.16:0/1215765705 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f12580a2c00 0x7f12580a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:39.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.863+0000 7f1270b28640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4dc5215d0 0x55c4dc521a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:39.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.863+0000 7f1270b28640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4dc5215d0 0x55c4dc521a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55502/0 (socket says 172.21.15.16:55502) 2024-09-12T02:08:39.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.863+0000 7f1270b28640 1 -- 172.21.15.16:0/769661680 learned_addr learned my addr 172.21.15.16:0/769661680 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:08:39.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.863+0000 7f1271329640 1 --2- 172.21.15.16:0/1215765705 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f12580a2c00 0x7f12580a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:39.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.863+0000 7f1269ffb640 1 -- 172.21.15.16:0/1215765705 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f12640cf090 con 0x7f126c106f10 2024-09-12T02:08:39.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.863+0000 7f126bfff640 1 --2- 172.21.15.16:0/769661680 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c4dc522b40 0x55c4dc521090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:39.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.863+0000 7f126bfff640 1 --2- 172.21.15.16:0/769661680 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c4dc52c370 0x55c4dc52e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:39.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.864+0000 7f1270b28640 1 -- 172.21.15.16:0/769661680 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c4dc522b40 msgr2=0x55c4dc521090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:39.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.864+0000 7f1270b28640 1 --2- 172.21.15.16:0/769661680 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c4dc522b40 0x55c4dc521090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:39.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.864+0000 7f1270b28640 1 -- 172.21.15.16:0/769661680 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c4dc52c370 msgr2=0x55c4dc52e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:39.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.864+0000 7f1270b28640 1 --2- 172.21.15.16:0/769661680 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c4dc52c370 0x55c4dc52e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:39.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.864+0000 7f1270b28640 1 -- 172.21.15.16:0/769661680 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c4dc4133a0 con 0x55c4dc5215d0 2024-09-12T02:08:39.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.864+0000 7f126bfff640 1 --2- 172.21.15.16:0/769661680 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c4dc522b40 0x55c4dc521090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:08:39.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.864+0000 7f1270b28640 1 --2- 172.21.15.16:0/769661680 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4dc5215d0 0x55c4dc521a10 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f125400bc20 tx=0x7f125400d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cc7f531a447e2d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:39.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.864+0000 7f1271329640 1 --2- 172.21.15.16:0/1215765705 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f12580a2c00 0x7f12580a50c0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f12640a4030 tx=0x7f1264090dd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:39.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.864+0000 7f1262ffd640 1 -- 172.21.15.16:0/769661680 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1254017620 con 0x55c4dc5215d0 2024-09-12T02:08:39.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.864+0000 7f1262ffd640 1 -- 172.21.15.16:0/769661680 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1254017ca0 con 0x55c4dc5215d0 2024-09-12T02:08:39.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.864+0000 7f1262ffd640 1 -- 172.21.15.16:0/769661680 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f125400fcf0 con 0x55c4dc5215d0 2024-09-12T02:08:39.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.865+0000 7f1275165740 1 -- 172.21.15.16:0/769661680 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4dc5215d0 msgr2=0x55c4dc521a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:39.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.865+0000 7f1275165740 1 --2- 172.21.15.16:0/769661680 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4dc5215d0 0x55c4dc521a10 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f125400bc20 tx=0x7f125400d710 comp rx=0 tx=0).stop 2024-09-12T02:08:39.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.865+0000 7f1275165740 1 -- 172.21.15.16:0/769661680 shutdown_connections 2024-09-12T02:08:39.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.865+0000 7f1275165740 1 --2- 172.21.15.16:0/769661680 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c4dc52c370 0x55c4dc52e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:39.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.865+0000 7f1275165740 1 --2- 172.21.15.16:0/769661680 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4dc5215d0 0x55c4dc521a10 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:39.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.865+0000 7f1275165740 1 --2- 172.21.15.16:0/769661680 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c4dc522b40 0x55c4dc521090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:39.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.865+0000 7f1275165740 1 -- 172.21.15.16:0/769661680 >> 172.21.15.16:0/769661680 conn(0x55c4dc4ea310 msgr2=0x55c4dc51dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:39.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.865+0000 7f1275165740 1 -- 172.21.15.16:0/769661680 shutdown_connections 2024-09-12T02:08:39.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.865+0000 7f1275165740 1 -- 172.21.15.16:0/769661680 wait complete. 2024-09-12T02:08:39.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.865+0000 7f1275165740 1 -- start start 2024-09-12T02:08:39.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.865+0000 7f1275165740 1 Processor -- start 2024-09-12T02:08:39.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.866+0000 7f1275165740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c4dc5215d0 0x55c4dc52bd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:39.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.866+0000 7f1275165740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c4dc522b40 0x55c4dc529030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:39.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.866+0000 7f1275165740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4dc52c370 0x55c4dc529640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:39.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.866+0000 7f1275165740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55c4dc52f070 con 0x55c4dc52c370 2024-09-12T02:08:39.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.866+0000 7f1275165740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55c4dc442060 con 0x55c4dc522b40 2024-09-12T02:08:39.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.866+0000 7f1275165740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55c4dc52f1f0 con 0x55c4dc5215d0 2024-09-12T02:08:39.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.866+0000 7f1275165740 10 client.0 _ll_register_callbacks cb 0x55c4dc1958c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:08:39.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.866+0000 7f126bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4dc52c370 0x55c4dc529640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:39.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.866+0000 7f126bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4dc52c370 0x55c4dc529640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55504/0 (socket says 172.21.15.16:55504) 2024-09-12T02:08:39.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.866+0000 7f126bfff640 1 -- 172.21.15.16:0/1715851441 learned_addr learned my addr 172.21.15.16:0/1715851441 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:08:39.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.866+0000 7f1270b28640 1 --2- 172.21.15.16:0/1715851441 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c4dc522b40 0x55c4dc529030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:39.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.866+0000 7f126bfff640 1 --2- 172.21.15.16:0/1715851441 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c4dc5215d0 0x55c4dc52bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:39.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.866+0000 7f126bfff640 1 -- 172.21.15.16:0/1715851441 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c4dc5215d0 msgr2=0x55c4dc52bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:39.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.866+0000 7f126bfff640 1 --2- 172.21.15.16:0/1715851441 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c4dc5215d0 0x55c4dc52bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:39.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.866+0000 7f126bfff640 1 -- 172.21.15.16:0/1715851441 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c4dc522b40 msgr2=0x55c4dc529030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:39.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.866+0000 7f126bfff640 1 --2- 172.21.15.16:0/1715851441 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c4dc522b40 0x55c4dc529030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:39.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.866+0000 7f126bfff640 1 -- 172.21.15.16:0/1715851441 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c4dc1f83c0 con 0x55c4dc52c370 2024-09-12T02:08:39.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.866+0000 7f126bfff640 1 --2- 172.21.15.16:0/1715851441 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4dc52c370 0x55c4dc529640 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f125c032fd0 tx=0x7f125c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7d47b7b7e9f41aa server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:39.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.866+0000 7f124bfff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:08:39.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.866+0000 7f124bfff640 1 -- 172.21.15.16:0/1715851441 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f125c035040 con 0x55c4dc52c370 2024-09-12T02:08:39.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.867+0000 7f124bfff640 1 -- 172.21.15.16:0/1715851441 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f125c0042d0 con 0x55c4dc52c370 2024-09-12T02:08:39.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.867+0000 7f1275165740 1 -- 172.21.15.16:0/1715851441 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1254017ca0 con 0x55c4dc52c370 2024-09-12T02:08:39.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.867+0000 7f124bfff640 1 -- 172.21.15.16:0/1715851441 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f125c050050 con 0x55c4dc52c370 2024-09-12T02:08:39.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.867+0000 7f1275165740 10 client.17421 fetch_fsmap learned FSMap version 25 2024-09-12T02:08:39.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.867+0000 7f1275165740 1 -- 172.21.15.16:0/1715851441 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c4dc53aab0 con 0x55c4dc52c370 2024-09-12T02:08:39.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.867+0000 7f124bfff640 1 -- 172.21.15.16:0/1715851441 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1254017ca0 con 0x55c4dc52c370 2024-09-12T02:08:39.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.867+0000 7f124bfff640 1 -- 172.21.15.16:0/1715851441 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f125c004be0 con 0x55c4dc52c370 2024-09-12T02:08:39.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.867+0000 7f1275165740 10 client.17421 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:08:39.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.867+0000 7f1275165740 10 client.17421 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-12T02:08:39.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.867+0000 7f1275165740 20 client.17421 populate_metadata read hostname 'smithi016' 2024-09-12T02:08:39.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.867+0000 7f1275165740 1 --2- 172.21.15.16:0/1715851441 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55c4dc53bec0 0x55c4dc53e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:39.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.867+0000 7f1275165740 4 client.17421 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:08:39.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.867+0000 7f1275165740 1 -- 172.21.15.16:0/1715851441 --> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c4dc494860 con 0x55c4dc53bec0 2024-09-12T02:08:39.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.868+0000 7f1270b28640 1 --2- 172.21.15.16:0/1715851441 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55c4dc53bec0 0x55c4dc53e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:39.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.868+0000 7f1270b28640 1 --2- 172.21.15.16:0/1715851441 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55c4dc53bec0 0x55c4dc53e2b0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3e9c43c70fdb71b1 server_cookie=f370520e5878ef89 in_seq=0 out_seq=0 2024-09-12T02:08:39.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.868+0000 7f124bfff640 10 client.17421 ms_handle_connect on v2:172.21.15.184:6836/1793393087 2024-09-12T02:08:39.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.869+0000 7f124bfff640 1 -- 172.21.15.16:0/1715851441 <== mds.2 v2:172.21.15.184:6836/1793393087 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c4dc494860 con 0x55c4dc53bec0 2024-09-12T02:08:39.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.869+0000 7f124bfff640 10 client.17421 handle_command_reply: tid=0 2024-09-12T02:08:39.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.869+0000 7f1275165740 1 client.17421 shutdown 2024-09-12T02:08:39.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.869+0000 7f1275165740 1 -- 172.21.15.16:0/1715851441 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4dc52c370 msgr2=0x55c4dc529640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:39.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.869+0000 7f1275165740 1 --2- 172.21.15.16:0/1715851441 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4dc52c370 0x55c4dc529640 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f125c032fd0 tx=0x7f125c009260 comp rx=0 tx=0).stop 2024-09-12T02:08:39.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.869+0000 7f1275165740 1 -- 172.21.15.16:0/1715851441 shutdown_connections 2024-09-12T02:08:39.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.869+0000 7f1275165740 1 --2- 172.21.15.16:0/1715851441 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55c4dc53bec0 0x55c4dc53e2b0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:39.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.869+0000 7f1275165740 1 --2- 172.21.15.16:0/1715851441 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4dc52c370 0x55c4dc529640 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:39.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.869+0000 7f1275165740 1 --2- 172.21.15.16:0/1715851441 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c4dc522b40 0x55c4dc529030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:39.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.869+0000 7f124bfff640 0 client.17421 ms_handle_reset on v2:172.21.15.184:6836/1793393087 2024-09-12T02:08:39.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.869+0000 7f1275165740 1 --2- 172.21.15.16:0/1715851441 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c4dc5215d0 0x55c4dc52bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:39.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.869+0000 7f1275165740 1 -- 172.21.15.16:0/1715851441 >> 172.21.15.16:0/1715851441 conn(0x55c4dc4ea310 msgr2=0x55c4dc523960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:39.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.869+0000 7f1275165740 1 -- 172.21.15.16:0/1715851441 shutdown_connections 2024-09-12T02:08:39.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.869+0000 7f1275165740 1 -- 172.21.15.16:0/1715851441 wait complete. 2024-09-12T02:08:39.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.869+0000 7f1275165740 20 client.17421 trim_cache size 0 max 16384 2024-09-12T02:08:39.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.882+0000 7f1275165740 1 Processor -- start 2024-09-12T02:08:39.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.882+0000 7f1275165740 1 -- start start 2024-09-12T02:08:39.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.882+0000 7f1275165740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c4dc5215d0 0x55c4dc52afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:39.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.882+0000 7f1275165740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c4dc522b40 0x55c4dc524120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:39.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.882+0000 7f1275165740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4dc52c370 0x55c4dc524660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:39.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.882+0000 7f1275165740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f126c1afcc0 con 0x55c4dc52c370 2024-09-12T02:08:39.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.883+0000 7f1275165740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f126c1ad590 con 0x55c4dc5215d0 2024-09-12T02:08:39.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.883+0000 7f1275165740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f126c1ab3e0 con 0x55c4dc522b40 2024-09-12T02:08:39.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.883+0000 7f126bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4dc52c370 0x55c4dc524660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:39.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.883+0000 7f126bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4dc52c370 0x55c4dc524660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55520/0 (socket says 172.21.15.16:55520) 2024-09-12T02:08:39.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.883+0000 7f126bfff640 1 -- 172.21.15.16:0/920902733 learned_addr learned my addr 172.21.15.16:0/920902733 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:08:39.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.883+0000 7f126bfff640 1 --2- 172.21.15.16:0/920902733 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c4dc5215d0 0x55c4dc52afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:39.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.883+0000 7f1270b28640 1 --2- 172.21.15.16:0/920902733 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c4dc522b40 0x55c4dc524120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:39.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.883+0000 7f126bfff640 1 -- 172.21.15.16:0/920902733 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c4dc522b40 msgr2=0x55c4dc524120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:39.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.883+0000 7f126bfff640 1 --2- 172.21.15.16:0/920902733 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c4dc522b40 0x55c4dc524120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:39.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.883+0000 7f126bfff640 1 -- 172.21.15.16:0/920902733 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c4dc5215d0 msgr2=0x55c4dc52afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:39.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.883+0000 7f126bfff640 1 --2- 172.21.15.16:0/920902733 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c4dc5215d0 0x55c4dc52afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:39.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.883+0000 7f126bfff640 1 -- 172.21.15.16:0/920902733 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c4dc53e910 con 0x55c4dc52c370 2024-09-12T02:08:39.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.883+0000 7f126bfff640 1 --2- 172.21.15.16:0/920902733 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4dc52c370 0x55c4dc524660 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x55c4dc521e60 tx=0x7f125c02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6df017e923a4dd5d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:39.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.883+0000 7f12627fc640 1 -- 172.21.15.16:0/920902733 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f125c035040 con 0x55c4dc52c370 2024-09-12T02:08:39.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.883+0000 7f12627fc640 1 -- 172.21.15.16:0/920902733 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f125c003b40 con 0x55c4dc52c370 2024-09-12T02:08:39.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.883+0000 7f12627fc640 1 -- 172.21.15.16:0/920902733 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f125c0053e0 con 0x55c4dc52c370 2024-09-12T02:08:39.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.884+0000 7f1275165740 1 -- 172.21.15.16:0/920902733 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4dc52c370 msgr2=0x55c4dc524660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:39.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.884+0000 7f1275165740 1 --2- 172.21.15.16:0/920902733 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4dc52c370 0x55c4dc524660 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x55c4dc521e60 tx=0x7f125c02a410 comp rx=0 tx=0).stop 2024-09-12T02:08:39.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.884+0000 7f1275165740 1 -- 172.21.15.16:0/920902733 shutdown_connections 2024-09-12T02:08:39.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.884+0000 7f1275165740 1 --2- 172.21.15.16:0/920902733 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4dc52c370 0x55c4dc524660 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:39.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.884+0000 7f1275165740 1 --2- 172.21.15.16:0/920902733 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c4dc522b40 0x55c4dc524120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:39.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.884+0000 7f1275165740 1 --2- 172.21.15.16:0/920902733 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c4dc5215d0 0x55c4dc52afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:39.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.884+0000 7f1275165740 1 -- 172.21.15.16:0/920902733 >> 172.21.15.16:0/920902733 conn(0x55c4dc4ea310 msgr2=0x55c4dc52a890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:39.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.884+0000 7f1275165740 1 -- 172.21.15.16:0/920902733 shutdown_connections 2024-09-12T02:08:39.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.884+0000 7f1275165740 1 -- 172.21.15.16:0/920902733 wait complete. 2024-09-12T02:08:39.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.884+0000 7f1275165740 1 -- start start 2024-09-12T02:08:39.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.884+0000 7f1275165740 1 Processor -- start 2024-09-12T02:08:39.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.885+0000 7f1275165740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c4dc5215d0 0x55c4dc534880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:39.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.885+0000 7f1275165740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4dc522b40 0x55c4dc525a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:39.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.885+0000 7f1275165740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c4dc52c370 0x55c4dc525f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:39.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.885+0000 7f1275165740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f126c1aba50 con 0x55c4dc522b40 2024-09-12T02:08:39.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.885+0000 7f1275165740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f126c1acec0 con 0x55c4dc52c370 2024-09-12T02:08:39.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.885+0000 7f1275165740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f126c1ac120 con 0x55c4dc5215d0 2024-09-12T02:08:39.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.885+0000 7f1275165740 10 client.0 _ll_register_callbacks cb 0x7f125c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:08:39.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.885+0000 7f1270b28640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4dc522b40 0x55c4dc525a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:39.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.885+0000 7f1270b28640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4dc522b40 0x55c4dc525a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55536/0 (socket says 172.21.15.16:55536) 2024-09-12T02:08:39.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.885+0000 7f1270b28640 1 -- 172.21.15.16:0/3270931859 learned_addr learned my addr 172.21.15.16:0/3270931859 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:08:39.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.885+0000 7f126bfff640 1 --2- 172.21.15.16:0/3270931859 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c4dc5215d0 0x55c4dc534880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:39.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.885+0000 7f126bfff640 1 --2- 172.21.15.16:0/3270931859 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c4dc52c370 0x55c4dc525f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:39.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.885+0000 7f1270b28640 1 -- 172.21.15.16:0/3270931859 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c4dc5215d0 msgr2=0x55c4dc534880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:39.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.885+0000 7f1270b28640 1 --2- 172.21.15.16:0/3270931859 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c4dc5215d0 0x55c4dc534880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:39.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.885+0000 7f1270b28640 1 -- 172.21.15.16:0/3270931859 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c4dc52c370 msgr2=0x55c4dc525f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:39.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.885+0000 7f1270b28640 1 --2- 172.21.15.16:0/3270931859 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c4dc52c370 0x55c4dc525f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:39.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.885+0000 7f1270b28640 1 -- 172.21.15.16:0/3270931859 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c4dc5264a0 con 0x55c4dc522b40 2024-09-12T02:08:39.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.885+0000 7f126bfff640 1 --2- 172.21.15.16:0/3270931859 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c4dc52c370 0x55c4dc525f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:08:39.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.886+0000 7f1270b28640 1 --2- 172.21.15.16:0/3270931859 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4dc522b40 0x55c4dc525a20 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f125400bc20 tx=0x7f125400f420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e363aca5bc4fb74 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:39.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.886+0000 7f124bfff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:08:39.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.886+0000 7f124bfff640 1 -- 172.21.15.16:0/3270931859 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1254005040 con 0x55c4dc522b40 2024-09-12T02:08:39.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.886+0000 7f124bfff640 1 -- 172.21.15.16:0/3270931859 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f12540051e0 con 0x55c4dc522b40 2024-09-12T02:08:39.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.886+0000 7f1275165740 1 -- 172.21.15.16:0/3270931859 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f125c003b40 con 0x55c4dc522b40 2024-09-12T02:08:39.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.886+0000 7f124bfff640 1 -- 172.21.15.16:0/3270931859 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f12540053e0 con 0x55c4dc522b40 2024-09-12T02:08:39.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.886+0000 7f1275165740 10 client.17427 fetch_fsmap learned FSMap version 25 2024-09-12T02:08:39.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.886+0000 7f1275165740 1 -- 172.21.15.16:0/3270931859 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c4dc5a67c0 con 0x55c4dc522b40 2024-09-12T02:08:39.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.886+0000 7f124bfff640 1 -- 172.21.15.16:0/3270931859 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f125c003b40 con 0x55c4dc522b40 2024-09-12T02:08:39.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.886+0000 7f124bfff640 1 -- 172.21.15.16:0/3270931859 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f1254004780 con 0x55c4dc522b40 2024-09-12T02:08:39.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.886+0000 7f1275165740 10 client.17427 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:08:39.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.886+0000 7f1275165740 10 client.17427 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-12T02:08:39.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.886+0000 7f1275165740 20 client.17427 populate_metadata read hostname 'smithi016' 2024-09-12T02:08:39.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.886+0000 7f1275165740 1 --2- 172.21.15.16:0/3270931859 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55c4dc5a6cc0 0x55c4dc5a90e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:39.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.886+0000 7f1275165740 4 client.17427 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T02:08:39.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.886+0000 7f1275165740 1 -- 172.21.15.16:0/3270931859 --> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c4dc48ed70 con 0x55c4dc5a6cc0 2024-09-12T02:08:39.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.887+0000 7f126bfff640 1 --2- 172.21.15.16:0/3270931859 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55c4dc5a6cc0 0x55c4dc5a90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:39.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.887+0000 7f126bfff640 1 --2- 172.21.15.16:0/3270931859 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55c4dc5a6cc0 0x55c4dc5a90e0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4188a35f5d1a2047 server_cookie=b59893d011c0f196 in_seq=0 out_seq=0 2024-09-12T02:08:39.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.887+0000 7f124bfff640 10 client.17427 ms_handle_connect on v2:172.21.15.184:6836/1793393087 2024-09-12T02:08:39.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.887+0000 7f124bfff640 1 -- 172.21.15.16:0/3270931859 <== mds.2 v2:172.21.15.184:6836/1793393087 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c4dc48ed70 con 0x55c4dc5a6cc0 2024-09-12T02:08:39.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.887+0000 7f124bfff640 10 client.17427 handle_command_reply: tid=0 2024-09-12T02:08:39.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.887+0000 7f1275165740 1 client.17427 shutdown 2024-09-12T02:08:39.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.888+0000 7f1275165740 1 -- 172.21.15.16:0/3270931859 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4dc522b40 msgr2=0x55c4dc525a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:39.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.888+0000 7f1275165740 1 --2- 172.21.15.16:0/3270931859 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4dc522b40 0x55c4dc525a20 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f125400bc20 tx=0x7f125400f420 comp rx=0 tx=0).stop 2024-09-12T02:08:39.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.888+0000 7f1275165740 1 -- 172.21.15.16:0/3270931859 shutdown_connections 2024-09-12T02:08:39.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.888+0000 7f1275165740 1 --2- 172.21.15.16:0/3270931859 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c4dc52c370 0x55c4dc525f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:39.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.888+0000 7f1275165740 1 --2- 172.21.15.16:0/3270931859 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55c4dc5a6cc0 0x55c4dc5a90e0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:39.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.888+0000 7f1275165740 1 --2- 172.21.15.16:0/3270931859 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4dc522b40 0x55c4dc525a20 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:39.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.888+0000 7f1275165740 1 --2- 172.21.15.16:0/3270931859 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c4dc5215d0 0x55c4dc534880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:39.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.888+0000 7f124bfff640 0 client.17427 ms_handle_reset on v2:172.21.15.184:6836/1793393087 2024-09-12T02:08:39.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.888+0000 7f1275165740 1 -- 172.21.15.16:0/3270931859 >> 172.21.15.16:0/3270931859 conn(0x55c4dc4ea310 msgr2=0x55c4dc52b0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:39.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.888+0000 7f1275165740 1 -- 172.21.15.16:0/3270931859 shutdown_connections 2024-09-12T02:08:39.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.888+0000 7f1275165740 1 -- 172.21.15.16:0/3270931859 wait complete. 2024-09-12T02:08:39.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.888+0000 7f1275165740 20 client.17427 trim_cache size 0 max 16384 2024-09-12T02:08:39.904 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T02:08:39.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.888+0000 7f1272d8c640 1 -- 172.21.15.16:0/1215765705 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f12580a2c00 msgr2=0x7f12580a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:39.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.888+0000 7f1272d8c640 1 --2- 172.21.15.16:0/1215765705 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f12580a2c00 0x7f12580a50c0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f12640a4030 tx=0x7f1264090dd0 comp rx=0 tx=0).stop 2024-09-12T02:08:39.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.888+0000 7f1272d8c640 1 -- 172.21.15.16:0/1215765705 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f126c106f10 msgr2=0x7f126c19cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:39.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.888+0000 7f1272d8c640 1 --2- 172.21.15.16:0/1215765705 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f126c106f10 0x7f126c19cce0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f126400cad0 tx=0x7f126400cf90 comp rx=0 tx=0).stop 2024-09-12T02:08:39.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.888+0000 7f1272d8c640 1 -- 172.21.15.16:0/1215765705 shutdown_connections 2024-09-12T02:08:39.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.888+0000 7f1272d8c640 1 --2- 172.21.15.16:0/1215765705 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f12580a2c00 0x7f12580a50c0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:39.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.888+0000 7f1272d8c640 1 --2- 172.21.15.16:0/1215765705 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f126c106f10 0x7f126c19cce0 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:39.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.888+0000 7f1272d8c640 1 --2- 172.21.15.16:0/1215765705 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f126c102790 0x7f126c198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:39.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.889+0000 7f1272d8c640 1 --2- 172.21.15.16:0/1215765705 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f126c101de0 0x7f126c198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:39.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.889+0000 7f1272d8c640 1 -- 172.21.15.16:0/1215765705 >> 172.21.15.16:0/1215765705 conn(0x7f126c0fd8c0 msgr2=0x7f126c0fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:39.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.889+0000 7f1272d8c640 1 -- 172.21.15.16:0/1215765705 shutdown_connections 2024-09-12T02:08:39.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:39.889+0000 7f1272d8c640 1 -- 172.21.15.16:0/1215765705 wait complete. 2024-09-12T02:08:40.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:08:40 smithi016 ceph-mon[30947]: pgmap v537: 97 pgs: 97 active+clean; 3.7 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-09-12T02:08:40.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:08:40 smithi184 ceph-mon[35593]: pgmap v537: 97 pgs: 97 active+clean; 3.7 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-09-12T02:08:40.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:08:40 smithi033 ceph-mon[35332]: pgmap v537: 97 pgs: 97 active+clean; 3.7 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-09-12T02:08:40.679 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-12T02:08:40.939 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:08:41.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.451+0000 7fa0992d7640 1 -- 172.21.15.16:0/4198347555 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0941036e0 msgr2=0x7fa094107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:41.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.451+0000 7fa0992d7640 1 --2- 172.21.15.16:0/4198347555 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0941036e0 0x7fa094107b90 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7fa088009930 tx=0x7fa08802a420 comp rx=0 tx=0).stop 2024-09-12T02:08:41.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.451+0000 7fa0992d7640 1 -- 172.21.15.16:0/4198347555 shutdown_connections 2024-09-12T02:08:41.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.451+0000 7fa0992d7640 1 --2- 172.21.15.16:0/4198347555 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa0941082a0 0x7fa09410ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:41.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.451+0000 7fa0992d7640 1 --2- 172.21.15.16:0/4198347555 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0941036e0 0x7fa094107b90 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:41.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.451+0000 7fa0992d7640 1 --2- 172.21.15.16:0/4198347555 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa094102d30 0x7fa094103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:41.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.451+0000 7fa0992d7640 1 -- 172.21.15.16:0/4198347555 >> 172.21.15.16:0/4198347555 conn(0x7fa0940fc7d0 msgr2=0x7fa0940febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:41.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.451+0000 7fa0992d7640 1 -- 172.21.15.16:0/4198347555 shutdown_connections 2024-09-12T02:08:41.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.451+0000 7fa0992d7640 1 -- 172.21.15.16:0/4198347555 wait complete. 2024-09-12T02:08:41.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.452+0000 7fa0992d7640 1 Processor -- start 2024-09-12T02:08:41.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.452+0000 7fa0992d7640 1 -- start start 2024-09-12T02:08:41.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.452+0000 7fa0992d7640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa094102d30 0x7fa0941a03f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:41.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.452+0000 7fa0992d7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0941036e0 0x7fa0941a0930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:41.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.452+0000 7fa0992d7640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa0941082a0 0x7fa09419a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:41.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.452+0000 7fa0992d7640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa0941120c0 con 0x7fa0941036e0 2024-09-12T02:08:41.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.452+0000 7fa0992d7640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa094111f40 con 0x7fa094102d30 2024-09-12T02:08:41.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.452+0000 7fa0992d7640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa094112240 con 0x7fa0941082a0 2024-09-12T02:08:41.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.453+0000 7fa0927fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0941036e0 0x7fa0941a0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:41.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.453+0000 7fa0927fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0941036e0 0x7fa0941a0930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48436/0 (socket says 172.21.15.16:48436) 2024-09-12T02:08:41.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.453+0000 7fa0927fc640 1 -- 172.21.15.16:0/2061764035 learned_addr learned my addr 172.21.15.16:0/2061764035 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:08:41.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.453+0000 7fa0937fe640 1 --2- 172.21.15.16:0/2061764035 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa0941082a0 0x7fa09419a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:41.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.453+0000 7fa092ffd640 1 --2- 172.21.15.16:0/2061764035 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa094102d30 0x7fa0941a03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:41.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.453+0000 7fa0927fc640 1 -- 172.21.15.16:0/2061764035 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa0941082a0 msgr2=0x7fa09419a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:41.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.453+0000 7fa0927fc640 1 --2- 172.21.15.16:0/2061764035 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa0941082a0 0x7fa09419a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:41.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.453+0000 7fa0927fc640 1 -- 172.21.15.16:0/2061764035 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa094102d30 msgr2=0x7fa0941a03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:41.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.453+0000 7fa0927fc640 1 --2- 172.21.15.16:0/2061764035 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa094102d30 0x7fa0941a03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:41.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.453+0000 7fa0927fc640 1 -- 172.21.15.16:0/2061764035 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa09419ace0 con 0x7fa0941036e0 2024-09-12T02:08:41.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.453+0000 7fa0927fc640 1 --2- 172.21.15.16:0/2061764035 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0941036e0 0x7fa0941a0930 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7fa088005940 tx=0x7fa0880042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:41.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.453+0000 7fa07bfff640 1 -- 172.21.15.16:0/2061764035 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa08803f030 con 0x7fa0941036e0 2024-09-12T02:08:41.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.453+0000 7fa07bfff640 1 -- 172.21.15.16:0/2061764035 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa088003ce0 con 0x7fa0941036e0 2024-09-12T02:08:41.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.453+0000 7fa0992d7640 1 -- 172.21.15.16:0/2061764035 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa09419af70 con 0x7fa0941036e0 2024-09-12T02:08:41.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.453+0000 7fa07bfff640 1 -- 172.21.15.16:0/2061764035 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa088035070 con 0x7fa0941036e0 2024-09-12T02:08:41.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.453+0000 7fa0992d7640 1 -- 172.21.15.16:0/2061764035 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0941a72c0 con 0x7fa0941036e0 2024-09-12T02:08:41.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.456+0000 7fa07bfff640 1 -- 172.21.15.16:0/2061764035 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa088042030 con 0x7fa0941036e0 2024-09-12T02:08:41.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.456+0000 7fa09b6b0740 1 Processor -- start 2024-09-12T02:08:41.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.457+0000 7fa09b6b0740 1 -- start start 2024-09-12T02:08:41.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.457+0000 7fa09b6b0740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5561ce5c3b40 0x5561ce5c2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:41.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.457+0000 7fa09b6b0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5561ce5c25d0 0x5561ce5c2a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:41.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.457+0000 7fa09b6b0740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5561ce5cd370 0x5561ce5cf760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:41.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.457+0000 7fa09b6b0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5561ce5282f0 con 0x5561ce5c25d0 2024-09-12T02:08:41.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.457+0000 7fa09b6b0740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5561ce4cfcf0 con 0x5561ce5cd370 2024-09-12T02:08:41.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.457+0000 7fa09b6b0740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5561ce5242b0 con 0x5561ce5c3b40 2024-09-12T02:08:41.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.457+0000 7fa092ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5561ce5c25d0 0x5561ce5c2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:41.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.457+0000 7fa092ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5561ce5c25d0 0x5561ce5c2a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48438/0 (socket says 172.21.15.16:48438) 2024-09-12T02:08:41.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.457+0000 7fa092ffd640 1 -- 172.21.15.16:0/1490986733 learned_addr learned my addr 172.21.15.16:0/1490986733 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:08:41.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.457+0000 7fa0937fe640 1 --2- 172.21.15.16:0/1490986733 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5561ce5c3b40 0x5561ce5c2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:41.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.457+0000 7fa07bfff640 1 --2- 172.21.15.16:0/2061764035 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fa07c0a2b80 0x7fa07c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:41.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.457+0000 7fa0927fc640 1 --2- 172.21.15.16:0/1490986733 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5561ce5cd370 0x5561ce5cf760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:41.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.457+0000 7fa092ffd640 1 -- 172.21.15.16:0/1490986733 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5561ce5c3b40 msgr2=0x5561ce5c2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:41.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.457+0000 7fa092ffd640 1 --2- 172.21.15.16:0/1490986733 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5561ce5c3b40 0x5561ce5c2090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:41.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.457+0000 7fa092ffd640 1 -- 172.21.15.16:0/1490986733 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5561ce5cd370 msgr2=0x5561ce5cf760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:41.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.457+0000 7fa092ffd640 1 --2- 172.21.15.16:0/1490986733 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5561ce5cd370 0x5561ce5cf760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:41.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.457+0000 7fa0937fe640 1 --2- 172.21.15.16:0/2061764035 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fa07c0a2b80 0x7fa07c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:41.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.457+0000 7fa092ffd640 1 -- 172.21.15.16:0/1490986733 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561ce4b43a0 con 0x5561ce5c25d0 2024-09-12T02:08:41.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.458+0000 7fa0927fc640 1 --2- 172.21.15.16:0/1490986733 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5561ce5cd370 0x5561ce5cf760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:08:41.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.458+0000 7fa07bfff640 1 -- 172.21.15.16:0/2061764035 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fa0880ef440 con 0x7fa0941036e0 2024-09-12T02:08:41.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.458+0000 7fa092ffd640 1 --2- 172.21.15.16:0/1490986733 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5561ce5c25d0 0x5561ce5c2a10 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7fa08400a4b0 tx=0x7fa0840096d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0b88cca6566c270 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:41.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.458+0000 7fa0797fa640 1 -- 172.21.15.16:0/1490986733 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa084018020 con 0x5561ce5c25d0 2024-09-12T02:08:41.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.458+0000 7fa0797fa640 1 -- 172.21.15.16:0/1490986733 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa084014cf0 con 0x5561ce5c25d0 2024-09-12T02:08:41.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.458+0000 7fa0797fa640 1 -- 172.21.15.16:0/1490986733 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa084013d50 con 0x5561ce5c25d0 2024-09-12T02:08:41.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.458+0000 7fa0937fe640 1 --2- 172.21.15.16:0/2061764035 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fa07c0a2b80 0x7fa07c0a5040 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fa08c00ac40 tx=0x7fa08c0046c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:41.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.459+0000 7fa09b6b0740 1 -- 172.21.15.16:0/1490986733 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5561ce5c25d0 msgr2=0x5561ce5c2a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:41.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.459+0000 7fa09b6b0740 1 --2- 172.21.15.16:0/1490986733 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5561ce5c25d0 0x5561ce5c2a10 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7fa08400a4b0 tx=0x7fa0840096d0 comp rx=0 tx=0).stop 2024-09-12T02:08:41.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.459+0000 7fa09b6b0740 1 -- 172.21.15.16:0/1490986733 shutdown_connections 2024-09-12T02:08:41.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.459+0000 7fa09b6b0740 1 --2- 172.21.15.16:0/1490986733 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5561ce5cd370 0x5561ce5cf760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:41.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.459+0000 7fa09b6b0740 1 --2- 172.21.15.16:0/1490986733 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5561ce5c25d0 0x5561ce5c2a10 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:41.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.459+0000 7fa09b6b0740 1 --2- 172.21.15.16:0/1490986733 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5561ce5c3b40 0x5561ce5c2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:41.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.459+0000 7fa09b6b0740 1 -- 172.21.15.16:0/1490986733 >> 172.21.15.16:0/1490986733 conn(0x5561ce58b310 msgr2=0x5561ce5bec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:41.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.459+0000 7fa09b6b0740 1 -- 172.21.15.16:0/1490986733 shutdown_connections 2024-09-12T02:08:41.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.459+0000 7fa09b6b0740 1 -- 172.21.15.16:0/1490986733 wait complete. 2024-09-12T02:08:41.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.459+0000 7fa09b6b0740 1 -- start start 2024-09-12T02:08:41.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.459+0000 7fa09b6b0740 1 Processor -- start 2024-09-12T02:08:41.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.459+0000 7fa09b6b0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5561ce5c25d0 0x5561ce5cccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:41.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.459+0000 7fa09b6b0740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5561ce5c3b40 0x5561ce5c9f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:41.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.460+0000 7fa09b6b0740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5561ce5cd370 0x5561ce5ca4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:41.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.460+0000 7fa09b6b0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5561ce5d0070 con 0x5561ce5c25d0 2024-09-12T02:08:41.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.460+0000 7fa09b6b0740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5561ce4e3060 con 0x5561ce5c3b40 2024-09-12T02:08:41.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.460+0000 7fa09b6b0740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5561ce5d01f0 con 0x5561ce5cd370 2024-09-12T02:08:41.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.460+0000 7fa09b6b0740 10 client.0 _ll_register_callbacks cb 0x5561ce2368c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:08:41.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.460+0000 7fa092ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5561ce5c25d0 0x5561ce5cccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:41.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.460+0000 7fa092ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5561ce5c25d0 0x5561ce5cccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48454/0 (socket says 172.21.15.16:48454) 2024-09-12T02:08:41.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.460+0000 7fa092ffd640 1 -- 172.21.15.16:0/2350999330 learned_addr learned my addr 172.21.15.16:0/2350999330 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:08:41.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.460+0000 7fa0927fc640 1 --2- 172.21.15.16:0/2350999330 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5561ce5c3b40 0x5561ce5c9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:41.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.460+0000 7fa0937fe640 1 --2- 172.21.15.16:0/2350999330 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5561ce5cd370 0x5561ce5ca4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:41.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.460+0000 7fa092ffd640 1 -- 172.21.15.16:0/2350999330 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5561ce5cd370 msgr2=0x5561ce5ca4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:41.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.460+0000 7fa092ffd640 1 --2- 172.21.15.16:0/2350999330 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5561ce5cd370 0x5561ce5ca4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:41.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.460+0000 7fa092ffd640 1 -- 172.21.15.16:0/2350999330 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5561ce5c3b40 msgr2=0x5561ce5c9f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:41.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.460+0000 7fa092ffd640 1 --2- 172.21.15.16:0/2350999330 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5561ce5c3b40 0x5561ce5c9f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:41.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.460+0000 7fa092ffd640 1 -- 172.21.15.16:0/2350999330 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561ce2993c0 con 0x5561ce5c25d0 2024-09-12T02:08:41.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.460+0000 7fa0937fe640 1 --2- 172.21.15.16:0/2350999330 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5561ce5cd370 0x5561ce5ca4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:08:41.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.460+0000 7fa092ffd640 1 --2- 172.21.15.16:0/2350999330 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5561ce5c25d0 0x5561ce5cccc0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7fa0840138d0 tx=0x7fa084009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=301a6717ea63c994 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:41.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.460+0000 7fa076ff5640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:08:41.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.460+0000 7fa076ff5640 1 -- 172.21.15.16:0/2350999330 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa08402c060 con 0x5561ce5c25d0 2024-09-12T02:08:41.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.460+0000 7fa076ff5640 1 -- 172.21.15.16:0/2350999330 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fa084014810 con 0x5561ce5c25d0 2024-09-12T02:08:41.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.460+0000 7fa09b6b0740 1 -- 172.21.15.16:0/2350999330 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa084014cf0 con 0x5561ce5c25d0 2024-09-12T02:08:41.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.461+0000 7fa076ff5640 1 -- 172.21.15.16:0/2350999330 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa084014a10 con 0x5561ce5c25d0 2024-09-12T02:08:41.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.461+0000 7fa09b6b0740 10 client.17439 fetch_fsmap learned FSMap version 25 2024-09-12T02:08:41.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.461+0000 7fa09b6b0740 1 -- 172.21.15.16:0/2350999330 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5561ce5dba20 con 0x5561ce5c25d0 2024-09-12T02:08:41.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.461+0000 7fa076ff5640 1 -- 172.21.15.16:0/2350999330 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa084014cf0 con 0x5561ce5c25d0 2024-09-12T02:08:41.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.461+0000 7fa076ff5640 1 -- 172.21.15.16:0/2350999330 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fa084031070 con 0x5561ce5c25d0 2024-09-12T02:08:41.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.461+0000 7fa09b6b0740 10 client.17439 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:08:41.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.461+0000 7fa09b6b0740 10 client.17439 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T02:08:41.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.461+0000 7fa09b6b0740 20 client.17439 populate_metadata read hostname 'smithi016' 2024-09-12T02:08:41.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.461+0000 7fa09b6b0740 1 --2- 172.21.15.16:0/2350999330 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x5561ce5dce20 0x5561ce5df210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:41.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.461+0000 7fa09b6b0740 4 client.17439 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:08:41.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.461+0000 7fa09b6b0740 1 -- 172.21.15.16:0/2350999330 --> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5561ce535860 con 0x5561ce5dce20 2024-09-12T02:08:41.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.461+0000 7fa0927fc640 1 --2- 172.21.15.16:0/2350999330 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x5561ce5dce20 0x5561ce5df210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:41.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.462+0000 7fa0927fc640 1 --2- 172.21.15.16:0/2350999330 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x5561ce5dce20 0x5561ce5df210 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f4ebb56c69fcc43a server_cookie=b9990bd1f74cac45 in_seq=0 out_seq=0 2024-09-12T02:08:41.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.462+0000 7fa076ff5640 10 client.17439 ms_handle_connect on v2:172.21.15.184:6838/4203347941 2024-09-12T02:08:41.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.462+0000 7fa076ff5640 1 -- 172.21.15.16:0/2350999330 <== mds.1 v2:172.21.15.184:6838/4203347941 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5561ce535860 con 0x5561ce5dce20 2024-09-12T02:08:41.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.462+0000 7fa076ff5640 10 client.17439 handle_command_reply: tid=0 2024-09-12T02:08:41.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.463+0000 7fa09b6b0740 1 client.17439 shutdown 2024-09-12T02:08:41.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.463+0000 7fa09b6b0740 1 -- 172.21.15.16:0/2350999330 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5561ce5c25d0 msgr2=0x5561ce5cccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:41.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.463+0000 7fa09b6b0740 1 --2- 172.21.15.16:0/2350999330 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5561ce5c25d0 0x5561ce5cccc0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7fa0840138d0 tx=0x7fa084009b80 comp rx=0 tx=0).stop 2024-09-12T02:08:41.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.463+0000 7fa09b6b0740 1 -- 172.21.15.16:0/2350999330 shutdown_connections 2024-09-12T02:08:41.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.463+0000 7fa09b6b0740 1 --2- 172.21.15.16:0/2350999330 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x5561ce5dce20 0x5561ce5df210 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:41.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.463+0000 7fa09b6b0740 1 --2- 172.21.15.16:0/2350999330 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5561ce5cd370 0x5561ce5ca4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:41.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.463+0000 7fa09b6b0740 1 --2- 172.21.15.16:0/2350999330 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5561ce5c3b40 0x5561ce5c9f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:41.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.463+0000 7fa09b6b0740 1 --2- 172.21.15.16:0/2350999330 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5561ce5c25d0 0x5561ce5cccc0 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:41.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.463+0000 7fa09b6b0740 1 -- 172.21.15.16:0/2350999330 >> 172.21.15.16:0/2350999330 conn(0x5561ce58b310 msgr2=0x5561ce5c4930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:41.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.463+0000 7fa076ff5640 0 client.17439 ms_handle_reset on v2:172.21.15.184:6838/4203347941 2024-09-12T02:08:41.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.463+0000 7fa09b6b0740 1 -- 172.21.15.16:0/2350999330 shutdown_connections 2024-09-12T02:08:41.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.463+0000 7fa09b6b0740 1 -- 172.21.15.16:0/2350999330 wait complete. 2024-09-12T02:08:41.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.463+0000 7fa09b6b0740 20 client.17439 trim_cache size 0 max 16384 2024-09-12T02:08:41.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.476+0000 7fa09b6b0740 1 Processor -- start 2024-09-12T02:08:41.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.476+0000 7fa09b6b0740 1 -- start start 2024-09-12T02:08:41.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.476+0000 7fa09b6b0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5561ce5c25d0 0x5561ce5bee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:41.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.476+0000 7fa09b6b0740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5561ce5c3b40 0x5561ce5c9870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:41.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.476+0000 7fa09b6b0740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5561ce5cd370 0x5561ce5c5120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:41.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.476+0000 7fa09b6b0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa0941b1f10 con 0x5561ce5c25d0 2024-09-12T02:08:41.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.476+0000 7fa09b6b0740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa0941af7e0 con 0x5561ce5c3b40 2024-09-12T02:08:41.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.476+0000 7fa09b6b0740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa0941ad650 con 0x5561ce5cd370 2024-09-12T02:08:41.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.476+0000 7fa092ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5561ce5c25d0 0x5561ce5bee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:41.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.476+0000 7fa092ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5561ce5c25d0 0x5561ce5bee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48470/0 (socket says 172.21.15.16:48470) 2024-09-12T02:08:41.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.476+0000 7fa092ffd640 1 -- 172.21.15.16:0/2057592745 learned_addr learned my addr 172.21.15.16:0/2057592745 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:08:41.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.476+0000 7fa0927fc640 1 --2- 172.21.15.16:0/2057592745 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5561ce5c3b40 0x5561ce5c9870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:41.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.476+0000 7fa0937fe640 1 --2- 172.21.15.16:0/2057592745 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5561ce5cd370 0x5561ce5c5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:41.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.477+0000 7fa092ffd640 1 -- 172.21.15.16:0/2057592745 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5561ce5cd370 msgr2=0x5561ce5c5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:41.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.477+0000 7fa092ffd640 1 --2- 172.21.15.16:0/2057592745 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5561ce5cd370 0x5561ce5c5120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:41.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.477+0000 7fa092ffd640 1 -- 172.21.15.16:0/2057592745 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5561ce5c3b40 msgr2=0x5561ce5c9870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:41.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.477+0000 7fa092ffd640 1 --2- 172.21.15.16:0/2057592745 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5561ce5c3b40 0x5561ce5c9870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:41.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.477+0000 7fa092ffd640 1 -- 172.21.15.16:0/2057592745 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561ce5ca840 con 0x5561ce5c25d0 2024-09-12T02:08:41.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.477+0000 7fa092ffd640 1 --2- 172.21.15.16:0/2057592745 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5561ce5c25d0 0x5561ce5bee10 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fa084004b90 tx=0x7fa0840049c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18a2f8a28459f1f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:41.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.477+0000 7fa078ff9640 1 -- 172.21.15.16:0/2057592745 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa084018040 con 0x5561ce5c25d0 2024-09-12T02:08:41.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.477+0000 7fa078ff9640 1 -- 172.21.15.16:0/2057592745 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa084013900 con 0x5561ce5c25d0 2024-09-12T02:08:41.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.477+0000 7fa078ff9640 1 -- 172.21.15.16:0/2057592745 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa084014610 con 0x5561ce5c25d0 2024-09-12T02:08:41.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.478+0000 7fa09b6b0740 1 -- 172.21.15.16:0/2057592745 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5561ce5c25d0 msgr2=0x5561ce5bee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:41.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.478+0000 7fa09b6b0740 1 --2- 172.21.15.16:0/2057592745 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5561ce5c25d0 0x5561ce5bee10 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fa084004b90 tx=0x7fa0840049c0 comp rx=0 tx=0).stop 2024-09-12T02:08:41.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.478+0000 7fa09b6b0740 1 -- 172.21.15.16:0/2057592745 shutdown_connections 2024-09-12T02:08:41.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.478+0000 7fa09b6b0740 1 --2- 172.21.15.16:0/2057592745 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5561ce5cd370 0x5561ce5c5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:41.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.478+0000 7fa09b6b0740 1 --2- 172.21.15.16:0/2057592745 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5561ce5c3b40 0x5561ce5c9870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:41.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.478+0000 7fa09b6b0740 1 --2- 172.21.15.16:0/2057592745 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5561ce5c25d0 0x5561ce5bee10 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:41.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.478+0000 7fa09b6b0740 1 -- 172.21.15.16:0/2057592745 >> 172.21.15.16:0/2057592745 conn(0x5561ce58b310 msgr2=0x5561ce5be700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:41.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.478+0000 7fa09b6b0740 1 -- 172.21.15.16:0/2057592745 shutdown_connections 2024-09-12T02:08:41.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.478+0000 7fa09b6b0740 1 -- 172.21.15.16:0/2057592745 wait complete. 2024-09-12T02:08:41.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.478+0000 7fa09b6b0740 1 -- start start 2024-09-12T02:08:41.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.478+0000 7fa09b6b0740 1 Processor -- start 2024-09-12T02:08:41.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.479+0000 7fa09b6b0740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5561ce5c25d0 0x5561ce5c7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:41.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.479+0000 7fa09b6b0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5561ce5c3b40 0x5561ce5d54b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:41.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.479+0000 7fa09b6b0740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5561ce5cd370 0x5561ce5d5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:41.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.479+0000 7fa09b6b0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa0941adca0 con 0x5561ce5c3b40 2024-09-12T02:08:41.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.479+0000 7fa09b6b0740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa0941af110 con 0x5561ce5cd370 2024-09-12T02:08:41.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.479+0000 7fa09b6b0740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa0941ae370 con 0x5561ce5c25d0 2024-09-12T02:08:41.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.479+0000 7fa09b6b0740 10 client.0 _ll_register_callbacks cb 0x7fa084009f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:08:41.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.479+0000 7fa0927fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5561ce5c3b40 0x5561ce5d54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:41.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.479+0000 7fa0927fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5561ce5c3b40 0x5561ce5d54b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48480/0 (socket says 172.21.15.16:48480) 2024-09-12T02:08:41.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.479+0000 7fa0927fc640 1 -- 172.21.15.16:0/3646014945 learned_addr learned my addr 172.21.15.16:0/3646014945 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:08:41.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.479+0000 7fa0937fe640 1 --2- 172.21.15.16:0/3646014945 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5561ce5cd370 0x5561ce5d5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:41.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.479+0000 7fa092ffd640 1 --2- 172.21.15.16:0/3646014945 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5561ce5c25d0 0x5561ce5c7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:41.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.479+0000 7fa0927fc640 1 -- 172.21.15.16:0/3646014945 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5561ce5c25d0 msgr2=0x5561ce5c7bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:41.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.479+0000 7fa0927fc640 1 --2- 172.21.15.16:0/3646014945 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5561ce5c25d0 0x5561ce5c7bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:41.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.479+0000 7fa0927fc640 1 -- 172.21.15.16:0/3646014945 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5561ce5cd370 msgr2=0x5561ce5d5aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:41.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.479+0000 7fa0927fc640 1 --2- 172.21.15.16:0/3646014945 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5561ce5cd370 0x5561ce5d5aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:41.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.479+0000 7fa0927fc640 1 -- 172.21.15.16:0/3646014945 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561ce5c6200 con 0x5561ce5c3b40 2024-09-12T02:08:41.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.479+0000 7fa0937fe640 1 --2- 172.21.15.16:0/3646014945 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5561ce5cd370 0x5561ce5d5aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:08:41.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.479+0000 7fa0927fc640 1 --2- 172.21.15.16:0/3646014945 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5561ce5c3b40 0x5561ce5d54b0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7fa088031550 tx=0x7fa088000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d961aa8bd083dcb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:41.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.479+0000 7fa076ff5640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:08:41.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.479+0000 7fa076ff5640 1 -- 172.21.15.16:0/3646014945 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa088049040 con 0x5561ce5c3b40 2024-09-12T02:08:41.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.479+0000 7fa076ff5640 1 -- 172.21.15.16:0/3646014945 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fa0880ed060 con 0x5561ce5c3b40 2024-09-12T02:08:41.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.479+0000 7fa09b6b0740 1 -- 172.21.15.16:0/3646014945 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa084013900 con 0x5561ce5c3b40 2024-09-12T02:08:41.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.480+0000 7fa076ff5640 1 -- 172.21.15.16:0/3646014945 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa0880c00f0 con 0x5561ce5c3b40 2024-09-12T02:08:41.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.480+0000 7fa09b6b0740 10 client.17445 fetch_fsmap learned FSMap version 25 2024-09-12T02:08:41.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.480+0000 7fa09b6b0740 1 -- 172.21.15.16:0/3646014945 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5561ce646bd0 con 0x5561ce5c3b40 2024-09-12T02:08:41.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.480+0000 7fa076ff5640 1 -- 172.21.15.16:0/3646014945 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa084013900 con 0x5561ce5c3b40 2024-09-12T02:08:41.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.480+0000 7fa076ff5640 1 -- 172.21.15.16:0/3646014945 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fa0880ef0b0 con 0x5561ce5c3b40 2024-09-12T02:08:41.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.480+0000 7fa09b6b0740 10 client.17445 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:08:41.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.480+0000 7fa09b6b0740 10 client.17445 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T02:08:41.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.480+0000 7fa09b6b0740 20 client.17445 populate_metadata read hostname 'smithi016' 2024-09-12T02:08:41.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.480+0000 7fa09b6b0740 1 --2- 172.21.15.16:0/3646014945 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x5561ce647350 0x5561ce649770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:41.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.480+0000 7fa09b6b0740 4 client.17445 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T02:08:41.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.480+0000 7fa09b6b0740 1 -- 172.21.15.16:0/3646014945 --> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5561ce52fd70 con 0x5561ce647350 2024-09-12T02:08:41.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.480+0000 7fa092ffd640 1 --2- 172.21.15.16:0/3646014945 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x5561ce647350 0x5561ce649770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:41.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.481+0000 7fa092ffd640 1 --2- 172.21.15.16:0/3646014945 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x5561ce647350 0x5561ce649770 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b1b7232e95f3ba6c server_cookie=28e5ecb57a97fa37 in_seq=0 out_seq=0 2024-09-12T02:08:41.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.481+0000 7fa076ff5640 10 client.17445 ms_handle_connect on v2:172.21.15.184:6838/4203347941 2024-09-12T02:08:41.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.481+0000 7fa076ff5640 1 -- 172.21.15.16:0/3646014945 <== mds.1 v2:172.21.15.184:6838/4203347941 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5561ce52fd70 con 0x5561ce647350 2024-09-12T02:08:41.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.481+0000 7fa076ff5640 10 client.17445 handle_command_reply: tid=0 2024-09-12T02:08:41.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.481+0000 7fa09b6b0740 1 client.17445 shutdown 2024-09-12T02:08:41.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.481+0000 7fa09b6b0740 1 -- 172.21.15.16:0/3646014945 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5561ce5c3b40 msgr2=0x5561ce5d54b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:41.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.481+0000 7fa09b6b0740 1 --2- 172.21.15.16:0/3646014945 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5561ce5c3b40 0x5561ce5d54b0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7fa088031550 tx=0x7fa088000bc0 comp rx=0 tx=0).stop 2024-09-12T02:08:41.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.481+0000 7fa09b6b0740 1 -- 172.21.15.16:0/3646014945 shutdown_connections 2024-09-12T02:08:41.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.481+0000 7fa09b6b0740 1 --2- 172.21.15.16:0/3646014945 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x5561ce647350 0x5561ce649770 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:41.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.481+0000 7fa09b6b0740 1 --2- 172.21.15.16:0/3646014945 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5561ce5cd370 0x5561ce5d5aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:41.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.481+0000 7fa09b6b0740 1 --2- 172.21.15.16:0/3646014945 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5561ce5c3b40 0x5561ce5d54b0 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:41.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.481+0000 7fa09b6b0740 1 --2- 172.21.15.16:0/3646014945 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5561ce5c25d0 0x5561ce5c7bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:41.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.481+0000 7fa09b6b0740 1 -- 172.21.15.16:0/3646014945 >> 172.21.15.16:0/3646014945 conn(0x5561ce58b310 msgr2=0x5561ce5cb7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:41.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.481+0000 7fa076ff5640 0 client.17445 ms_handle_reset on v2:172.21.15.184:6838/4203347941 2024-09-12T02:08:41.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.481+0000 7fa09b6b0740 1 -- 172.21.15.16:0/3646014945 shutdown_connections 2024-09-12T02:08:41.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.481+0000 7fa09b6b0740 1 -- 172.21.15.16:0/3646014945 wait complete. 2024-09-12T02:08:41.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.481+0000 7fa09b6b0740 20 client.17445 trim_cache size 0 max 16384 2024-09-12T02:08:41.490 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T02:08:41.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.482+0000 7fa0992d7640 1 -- 172.21.15.16:0/2061764035 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fa07c0a2b80 msgr2=0x7fa07c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:41.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.482+0000 7fa0992d7640 1 --2- 172.21.15.16:0/2061764035 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fa07c0a2b80 0x7fa07c0a5040 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fa08c00ac40 tx=0x7fa08c0046c0 comp rx=0 tx=0).stop 2024-09-12T02:08:41.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.482+0000 7fa0992d7640 1 -- 172.21.15.16:0/2061764035 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0941036e0 msgr2=0x7fa0941a0930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:41.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.482+0000 7fa0992d7640 1 --2- 172.21.15.16:0/2061764035 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0941036e0 0x7fa0941a0930 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7fa088005940 tx=0x7fa0880042f0 comp rx=0 tx=0).stop 2024-09-12T02:08:41.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.482+0000 7fa0992d7640 1 -- 172.21.15.16:0/2061764035 shutdown_connections 2024-09-12T02:08:41.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.482+0000 7fa0992d7640 1 --2- 172.21.15.16:0/2061764035 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fa07c0a2b80 0x7fa07c0a5040 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:41.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.482+0000 7fa0992d7640 1 --2- 172.21.15.16:0/2061764035 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa0941082a0 0x7fa09419a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:41.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.482+0000 7fa0992d7640 1 --2- 172.21.15.16:0/2061764035 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0941036e0 0x7fa0941a0930 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:41.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.482+0000 7fa0992d7640 1 --2- 172.21.15.16:0/2061764035 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa094102d30 0x7fa0941a03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:41.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.482+0000 7fa0992d7640 1 -- 172.21.15.16:0/2061764035 >> 172.21.15.16:0/2061764035 conn(0x7fa0940fc7d0 msgr2=0x7fa0941013c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:41.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.482+0000 7fa0992d7640 1 -- 172.21.15.16:0/2061764035 shutdown_connections 2024-09-12T02:08:41.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:41.482+0000 7fa0992d7640 1 -- 172.21.15.16:0/2061764035 wait complete. 2024-09-12T02:08:42.127 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-12T02:08:42.386 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:08:42.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:08:42 smithi016 ceph-mon[30947]: pgmap v538: 97 pgs: 97 active+clean; 3.7 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-09-12T02:08:42.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:08:42 smithi184 ceph-mon[35593]: pgmap v538: 97 pgs: 97 active+clean; 3.7 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-09-12T02:08:42.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:08:42 smithi033 ceph-mon[35332]: pgmap v538: 97 pgs: 97 active+clean; 3.7 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-09-12T02:08:42.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.852+0000 7f5e624d6640 1 -- 172.21.15.16:0/157523905 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e5c10c170 msgr2=0x7f5e5c10e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:42.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.852+0000 7f5e624d6640 1 --2- 172.21.15.16:0/157523905 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e5c10c170 0x7f5e5c10e610 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f5e54009ca0 tx=0x7f5e5402a950 comp rx=0 tx=0).stop 2024-09-12T02:08:42.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.852+0000 7f5e624d6640 1 -- 172.21.15.16:0/157523905 shutdown_connections 2024-09-12T02:08:42.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.852+0000 7f5e624d6640 1 --2- 172.21.15.16:0/157523905 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e5c10c170 0x7f5e5c10e610 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:42.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.852+0000 7f5e624d6640 1 --2- 172.21.15.16:0/157523905 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5e5c103660 0x7f5e5c10bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:42.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.852+0000 7f5e624d6640 1 --2- 172.21.15.16:0/157523905 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5e5c102d40 0x7f5e5c103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:42.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.852+0000 7f5e624d6640 1 -- 172.21.15.16:0/157523905 >> 172.21.15.16:0/157523905 conn(0x7f5e5c0fc7a0 msgr2=0x7f5e5c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:42.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.852+0000 7f5e624d6640 1 -- 172.21.15.16:0/157523905 shutdown_connections 2024-09-12T02:08:42.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.852+0000 7f5e624d6640 1 -- 172.21.15.16:0/157523905 wait complete. 2024-09-12T02:08:42.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.852+0000 7f5e624d6640 1 Processor -- start 2024-09-12T02:08:42.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.852+0000 7f5e624d6640 1 -- start start 2024-09-12T02:08:42.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.853+0000 7f5e624d6640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5e5c102d40 0x7f5e5c19a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:42.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.853+0000 7f5e624d6640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5e5c103660 0x7f5e5c19ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:42.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.853+0000 7f5e624d6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e5c10c170 0x7f5e5c19ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:42.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.853+0000 7f5e624d6640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5e5c111f50 con 0x7f5e5c10c170 2024-09-12T02:08:42.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.853+0000 7f5e624d6640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5e5c111dd0 con 0x7f5e5c103660 2024-09-12T02:08:42.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.853+0000 7f5e624d6640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5e5c1120d0 con 0x7f5e5c102d40 2024-09-12T02:08:42.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.853+0000 7f5e60a73640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e5c10c170 0x7f5e5c19ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:42.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.853+0000 7f5e5bfff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5e5c102d40 0x7f5e5c19a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:42.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.853+0000 7f5e60a73640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e5c10c170 0x7f5e5c19ee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48498/0 (socket says 172.21.15.16:48498) 2024-09-12T02:08:42.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.853+0000 7f5e60a73640 1 -- 172.21.15.16:0/1030221267 learned_addr learned my addr 172.21.15.16:0/1030221267 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:08:42.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.853+0000 7f5e5b7fe640 1 --2- 172.21.15.16:0/1030221267 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5e5c103660 0x7f5e5c19ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:42.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.853+0000 7f5e60a73640 1 -- 172.21.15.16:0/1030221267 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5e5c102d40 msgr2=0x7f5e5c19a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:42.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.853+0000 7f5e60a73640 1 --2- 172.21.15.16:0/1030221267 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5e5c102d40 0x7f5e5c19a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:42.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.853+0000 7f5e60a73640 1 -- 172.21.15.16:0/1030221267 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5e5c103660 msgr2=0x7f5e5c19ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:42.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.853+0000 7f5e60a73640 1 --2- 172.21.15.16:0/1030221267 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5e5c103660 0x7f5e5c19ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:42.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.853+0000 7f5e60a73640 1 -- 172.21.15.16:0/1030221267 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e5c19f570 con 0x7f5e5c10c170 2024-09-12T02:08:42.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.854+0000 7f5e60a73640 1 --2- 172.21.15.16:0/1030221267 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e5c10c170 0x7f5e5c19ee90 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f5e54004af0 tx=0x7f5e5402f3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:42.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.854+0000 7f5e597fa640 1 -- 172.21.15.16:0/1030221267 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e54034040 con 0x7f5e5c10c170 2024-09-12T02:08:42.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.854+0000 7f5e624d6640 1 -- 172.21.15.16:0/1030221267 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e5c19f800 con 0x7f5e5c10c170 2024-09-12T02:08:42.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.854+0000 7f5e597fa640 1 -- 172.21.15.16:0/1030221267 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e54030920 con 0x7f5e5c10c170 2024-09-12T02:08:42.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.854+0000 7f5e597fa640 1 -- 172.21.15.16:0/1030221267 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e54004670 con 0x7f5e5c10c170 2024-09-12T02:08:42.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.854+0000 7f5e624d6640 1 -- 172.21.15.16:0/1030221267 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e5c1a7140 con 0x7f5e5c10c170 2024-09-12T02:08:42.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.856+0000 7f5e597fa640 1 -- 172.21.15.16:0/1030221267 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5e54037020 con 0x7f5e5c10c170 2024-09-12T02:08:42.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.856+0000 7f5e648af740 1 Processor -- start 2024-09-12T02:08:42.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.856+0000 7f5e648af740 1 -- start start 2024-09-12T02:08:42.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.857+0000 7f5e648af740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556e4c670b40 0x556e4c66f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:42.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.857+0000 7f5e648af740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556e4c66f5d0 0x556e4c66fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:42.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.857+0000 7f5e648af740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e4c67a370 0x556e4c67c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:42.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.857+0000 7f5e648af740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x556e4c5d52f0 con 0x556e4c67a370 2024-09-12T02:08:42.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.857+0000 7f5e648af740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x556e4c57ccf0 con 0x556e4c66f5d0 2024-09-12T02:08:42.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.857+0000 7f5e648af740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x556e4c5d12b0 con 0x556e4c670b40 2024-09-12T02:08:42.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.857+0000 7f5e597fa640 1 --2- 172.21.15.16:0/1030221267 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f5e440a2b80 0x7f5e440a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:42.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.857+0000 7f5e5b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e4c67a370 0x556e4c67c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:42.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.857+0000 7f5e5b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e4c67a370 0x556e4c67c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48508/0 (socket says 172.21.15.16:48508) 2024-09-12T02:08:42.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.857+0000 7f5e5b7fe640 1 -- 172.21.15.16:0/4159422813 learned_addr learned my addr 172.21.15.16:0/4159422813 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:08:42.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.857+0000 7f5e597fa640 1 -- 172.21.15.16:0/1030221267 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f5e540efd10 con 0x7f5e5c10c170 2024-09-12T02:08:42.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.857+0000 7f5e60a73640 1 --2- 172.21.15.16:0/1030221267 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f5e440a2b80 0x7f5e440a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:42.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.857+0000 7f5e5b7fe640 1 --2- 172.21.15.16:0/4159422813 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556e4c670b40 0x556e4c66f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:42.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.857+0000 7f5e5bfff640 1 --2- 172.21.15.16:0/4159422813 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556e4c66f5d0 0x556e4c66fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:42.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.857+0000 7f5e5b7fe640 1 -- 172.21.15.16:0/4159422813 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556e4c670b40 msgr2=0x556e4c66f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:42.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.857+0000 7f5e5b7fe640 1 --2- 172.21.15.16:0/4159422813 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556e4c670b40 0x556e4c66f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:42.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.857+0000 7f5e5b7fe640 1 -- 172.21.15.16:0/4159422813 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556e4c66f5d0 msgr2=0x556e4c66fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:42.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.857+0000 7f5e5b7fe640 1 --2- 172.21.15.16:0/4159422813 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556e4c66f5d0 0x556e4c66fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:42.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.857+0000 7f5e5b7fe640 1 -- 172.21.15.16:0/4159422813 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e4c5613a0 con 0x556e4c67a370 2024-09-12T02:08:42.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.857+0000 7f5e60a73640 1 --2- 172.21.15.16:0/1030221267 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f5e440a2b80 0x7f5e440a5040 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f5e540effb0 tx=0x7f5e5403c040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:42.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.858+0000 7f5e5b7fe640 1 --2- 172.21.15.16:0/4159422813 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e4c67a370 0x556e4c67c760 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f5e50011780 tx=0x7f5e50005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88c3e525c6e4c52b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:42.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.858+0000 7f5e427fc640 1 -- 172.21.15.16:0/4159422813 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e5000f7c0 con 0x556e4c67a370 2024-09-12T02:08:42.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.858+0000 7f5e427fc640 1 -- 172.21.15.16:0/4159422813 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e5000fe40 con 0x556e4c67a370 2024-09-12T02:08:42.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.858+0000 7f5e427fc640 1 -- 172.21.15.16:0/4159422813 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e5001fda0 con 0x556e4c67a370 2024-09-12T02:08:42.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.858+0000 7f5e648af740 1 -- 172.21.15.16:0/4159422813 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e4c67a370 msgr2=0x556e4c67c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:42.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.858+0000 7f5e648af740 1 --2- 172.21.15.16:0/4159422813 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e4c67a370 0x556e4c67c760 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f5e50011780 tx=0x7f5e50005850 comp rx=0 tx=0).stop 2024-09-12T02:08:42.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.859+0000 7f5e648af740 1 -- 172.21.15.16:0/4159422813 shutdown_connections 2024-09-12T02:08:42.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.859+0000 7f5e648af740 1 --2- 172.21.15.16:0/4159422813 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e4c67a370 0x556e4c67c760 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:42.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.859+0000 7f5e648af740 1 --2- 172.21.15.16:0/4159422813 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556e4c66f5d0 0x556e4c66fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:42.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.859+0000 7f5e648af740 1 --2- 172.21.15.16:0/4159422813 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556e4c670b40 0x556e4c66f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:42.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.859+0000 7f5e648af740 1 -- 172.21.15.16:0/4159422813 >> 172.21.15.16:0/4159422813 conn(0x556e4c638310 msgr2=0x556e4c66bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:42.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.859+0000 7f5e648af740 1 -- 172.21.15.16:0/4159422813 shutdown_connections 2024-09-12T02:08:42.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.859+0000 7f5e648af740 1 -- 172.21.15.16:0/4159422813 wait complete. 2024-09-12T02:08:42.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.859+0000 7f5e648af740 1 -- start start 2024-09-12T02:08:42.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.859+0000 7f5e648af740 1 Processor -- start 2024-09-12T02:08:42.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.860+0000 7f5e648af740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e4c66f5d0 0x556e4c679e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:42.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.860+0000 7f5e648af740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556e4c670b40 0x556e4c6770b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:42.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.860+0000 7f5e648af740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556e4c67a370 0x556e4c677840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:42.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.860+0000 7f5e648af740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x556e4c67d070 con 0x556e4c66f5d0 2024-09-12T02:08:42.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.860+0000 7f5e648af740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x556e4c590060 con 0x556e4c670b40 2024-09-12T02:08:42.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.860+0000 7f5e648af740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x556e4c67d1f0 con 0x556e4c67a370 2024-09-12T02:08:42.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.860+0000 7f5e648af740 10 client.0 _ll_register_callbacks cb 0x556e4c2e38c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:08:42.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.860+0000 7f5e5b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e4c66f5d0 0x556e4c679e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:42.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.860+0000 7f5e5b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e4c66f5d0 0x556e4c679e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48516/0 (socket says 172.21.15.16:48516) 2024-09-12T02:08:42.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.860+0000 7f5e5b7fe640 1 -- 172.21.15.16:0/3837312640 learned_addr learned my addr 172.21.15.16:0/3837312640 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:08:42.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.860+0000 7f5e5bfff640 1 --2- 172.21.15.16:0/3837312640 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556e4c670b40 0x556e4c6770b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:42.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.860+0000 7f5e5b7fe640 1 --2- 172.21.15.16:0/3837312640 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556e4c67a370 0x556e4c677840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:42.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.860+0000 7f5e5b7fe640 1 -- 172.21.15.16:0/3837312640 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556e4c67a370 msgr2=0x556e4c677840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:42.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.860+0000 7f5e5b7fe640 1 --2- 172.21.15.16:0/3837312640 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556e4c67a370 0x556e4c677840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:42.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.860+0000 7f5e5b7fe640 1 -- 172.21.15.16:0/3837312640 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556e4c670b40 msgr2=0x556e4c6770b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:42.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.860+0000 7f5e5b7fe640 1 --2- 172.21.15.16:0/3837312640 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556e4c670b40 0x556e4c6770b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:42.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.860+0000 7f5e5b7fe640 1 -- 172.21.15.16:0/3837312640 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e4c3463c0 con 0x556e4c66f5d0 2024-09-12T02:08:42.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.860+0000 7f5e5b7fe640 1 --2- 172.21.15.16:0/3837312640 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e4c66f5d0 0x556e4c679e00 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f5e5001f940 tx=0x7f5e50005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23181f19fce3f2d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:42.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.860+0000 7f5e3b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:08:42.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.860+0000 7f5e3b7fe640 1 -- 172.21.15.16:0/3837312640 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e5001fb20 con 0x556e4c66f5d0 2024-09-12T02:08:42.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.860+0000 7f5e3b7fe640 1 -- 172.21.15.16:0/3837312640 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f5e5001fcc0 con 0x556e4c66f5d0 2024-09-12T02:08:42.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.860+0000 7f5e648af740 1 -- 172.21.15.16:0/3837312640 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5e5000fe40 con 0x556e4c66f5d0 2024-09-12T02:08:42.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.861+0000 7f5e3b7fe640 1 -- 172.21.15.16:0/3837312640 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5e50035050 con 0x556e4c66f5d0 2024-09-12T02:08:42.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.861+0000 7f5e648af740 10 client.17457 fetch_fsmap learned FSMap version 25 2024-09-12T02:08:42.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.861+0000 7f5e648af740 1 -- 172.21.15.16:0/3837312640 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x556e4c688bc0 con 0x556e4c66f5d0 2024-09-12T02:08:42.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.861+0000 7f5e3b7fe640 1 -- 172.21.15.16:0/3837312640 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e5000fe40 con 0x556e4c66f5d0 2024-09-12T02:08:42.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.861+0000 7f5e3b7fe640 1 -- 172.21.15.16:0/3837312640 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f5e5002c070 con 0x556e4c66f5d0 2024-09-12T02:08:42.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.861+0000 7f5e648af740 10 client.17457 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:08:42.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.861+0000 7f5e648af740 10 client.17457 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:08:42.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.861+0000 7f5e648af740 20 client.17457 populate_metadata read hostname 'smithi016' 2024-09-12T02:08:42.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.861+0000 7f5e648af740 1 --2- 172.21.15.16:0/3837312640 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x556e4c689f10 0x556e4c68c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:42.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.861+0000 7f5e648af740 4 client.17457 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:08:42.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.861+0000 7f5e648af740 1 -- 172.21.15.16:0/3837312640 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556e4c5e2860 con 0x556e4c689f10 2024-09-12T02:08:42.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.862+0000 7f5e5bfff640 1 --2- 172.21.15.16:0/3837312640 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x556e4c689f10 0x556e4c68c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:42.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.862+0000 7f5e5bfff640 1 --2- 172.21.15.16:0/3837312640 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x556e4c689f10 0x556e4c68c300 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5bba197cbed7b9d2 server_cookie=42899944a535a626 in_seq=0 out_seq=0 2024-09-12T02:08:42.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.862+0000 7f5e3b7fe640 10 client.17457 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:08:42.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.863+0000 7f5e3b7fe640 1 -- 172.21.15.16:0/3837312640 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556e4c5e2860 con 0x556e4c689f10 2024-09-12T02:08:42.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.863+0000 7f5e3b7fe640 10 client.17457 handle_command_reply: tid=0 2024-09-12T02:08:42.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.863+0000 7f5e648af740 1 client.17457 shutdown 2024-09-12T02:08:42.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.863+0000 7f5e648af740 1 -- 172.21.15.16:0/3837312640 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e4c66f5d0 msgr2=0x556e4c679e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:42.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.863+0000 7f5e648af740 1 --2- 172.21.15.16:0/3837312640 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e4c66f5d0 0x556e4c679e00 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f5e5001f940 tx=0x7f5e50005f90 comp rx=0 tx=0).stop 2024-09-12T02:08:42.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.863+0000 7f5e648af740 1 -- 172.21.15.16:0/3837312640 shutdown_connections 2024-09-12T02:08:42.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.863+0000 7f5e648af740 1 --2- 172.21.15.16:0/3837312640 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x556e4c689f10 0x556e4c68c300 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:42.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.863+0000 7f5e648af740 1 --2- 172.21.15.16:0/3837312640 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556e4c67a370 0x556e4c677840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:42.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.863+0000 7f5e648af740 1 --2- 172.21.15.16:0/3837312640 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556e4c670b40 0x556e4c6770b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:42.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.863+0000 7f5e648af740 1 --2- 172.21.15.16:0/3837312640 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e4c66f5d0 0x556e4c679e00 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:42.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.863+0000 7f5e648af740 1 -- 172.21.15.16:0/3837312640 >> 172.21.15.16:0/3837312640 conn(0x556e4c638310 msgr2=0x556e4c671960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:42.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.863+0000 7f5e3b7fe640 0 client.17457 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:08:42.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.863+0000 7f5e648af740 1 -- 172.21.15.16:0/3837312640 shutdown_connections 2024-09-12T02:08:42.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.863+0000 7f5e648af740 1 -- 172.21.15.16:0/3837312640 wait complete. 2024-09-12T02:08:42.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.863+0000 7f5e648af740 20 client.17457 trim_cache size 0 max 16384 2024-09-12T02:08:42.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.876+0000 7f5e648af740 1 Processor -- start 2024-09-12T02:08:42.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.876+0000 7f5e648af740 1 -- start start 2024-09-12T02:08:42.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.876+0000 7f5e648af740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556e4c66f5d0 0x556e4c676090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:42.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.877+0000 7f5e648af740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556e4c670b40 0x556e4c691a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:42.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.877+0000 7f5e648af740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e4c67a370 0x556e4c691f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:42.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.877+0000 7f5e648af740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5e5c1b1ee0 con 0x556e4c67a370 2024-09-12T02:08:42.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.877+0000 7f5e648af740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5e5c1af7b0 con 0x556e4c670b40 2024-09-12T02:08:42.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.877+0000 7f5e648af740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5e5c1ad600 con 0x556e4c66f5d0 2024-09-12T02:08:42.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.877+0000 7f5e5b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e4c67a370 0x556e4c691f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:42.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.877+0000 7f5e5b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e4c67a370 0x556e4c691f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48522/0 (socket says 172.21.15.16:48522) 2024-09-12T02:08:42.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.877+0000 7f5e5b7fe640 1 -- 172.21.15.16:0/1224553550 learned_addr learned my addr 172.21.15.16:0/1224553550 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:08:42.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.877+0000 7f5e5bfff640 1 --2- 172.21.15.16:0/1224553550 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556e4c670b40 0x556e4c691a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:42.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.877+0000 7f5e5b7fe640 1 -- 172.21.15.16:0/1224553550 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556e4c66f5d0 msgr2=0x556e4c676090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:42.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.877+0000 7f5e5b7fe640 1 --2- 172.21.15.16:0/1224553550 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556e4c66f5d0 0x556e4c676090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:42.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.877+0000 7f5e5b7fe640 1 -- 172.21.15.16:0/1224553550 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556e4c670b40 msgr2=0x556e4c691a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:42.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.877+0000 7f5e5b7fe640 1 --2- 172.21.15.16:0/1224553550 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556e4c670b40 0x556e4c691a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:42.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.877+0000 7f5e5b7fe640 1 -- 172.21.15.16:0/1224553550 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e4c675600 con 0x556e4c67a370 2024-09-12T02:08:42.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.877+0000 7f5e5b7fe640 1 --2- 172.21.15.16:0/1224553550 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e4c67a370 0x556e4c691f60 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f5e5001f8d0 tx=0x7f5e5002d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49d72c1d73c72b5e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:42.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.877+0000 7f5e41ffb640 1 -- 172.21.15.16:0/1224553550 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e50009e10 con 0x556e4c67a370 2024-09-12T02:08:42.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.877+0000 7f5e41ffb640 1 -- 172.21.15.16:0/1224553550 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e50022a30 con 0x556e4c67a370 2024-09-12T02:08:42.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.877+0000 7f5e41ffb640 1 -- 172.21.15.16:0/1224553550 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e5001fcc0 con 0x556e4c67a370 2024-09-12T02:08:42.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.878+0000 7f5e648af740 1 -- 172.21.15.16:0/1224553550 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e4c67a370 msgr2=0x556e4c691f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:42.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.878+0000 7f5e648af740 1 --2- 172.21.15.16:0/1224553550 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e4c67a370 0x556e4c691f60 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f5e5001f8d0 tx=0x7f5e5002d430 comp rx=0 tx=0).stop 2024-09-12T02:08:42.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.878+0000 7f5e648af740 1 -- 172.21.15.16:0/1224553550 shutdown_connections 2024-09-12T02:08:42.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.878+0000 7f5e648af740 1 --2- 172.21.15.16:0/1224553550 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e4c67a370 0x556e4c691f60 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:42.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.878+0000 7f5e648af740 1 --2- 172.21.15.16:0/1224553550 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556e4c670b40 0x556e4c691a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:42.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.878+0000 7f5e648af740 1 --2- 172.21.15.16:0/1224553550 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556e4c66f5d0 0x556e4c676090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:42.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.878+0000 7f5e648af740 1 -- 172.21.15.16:0/1224553550 >> 172.21.15.16:0/1224553550 conn(0x556e4c638310 msgr2=0x556e4c675980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:42.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.878+0000 7f5e648af740 1 -- 172.21.15.16:0/1224553550 shutdown_connections 2024-09-12T02:08:42.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.878+0000 7f5e648af740 1 -- 172.21.15.16:0/1224553550 wait complete. 2024-09-12T02:08:42.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.878+0000 7f5e648af740 1 -- start start 2024-09-12T02:08:42.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.879+0000 7f5e648af740 1 Processor -- start 2024-09-12T02:08:42.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.879+0000 7f5e648af740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556e4c66f5d0 0x556e4c6829f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:42.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.879+0000 7f5e648af740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556e4c670b40 0x556e4c673d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:42.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.879+0000 7f5e648af740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e4c67a370 0x556e4c674280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:42.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.879+0000 7f5e648af740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5e5c1adc70 con 0x556e4c67a370 2024-09-12T02:08:42.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.879+0000 7f5e648af740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5e5c1af0e0 con 0x556e4c670b40 2024-09-12T02:08:42.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.879+0000 7f5e648af740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5e5c1ae340 con 0x556e4c66f5d0 2024-09-12T02:08:42.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.879+0000 7f5e648af740 10 client.0 _ll_register_callbacks cb 0x7f5e4c0097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:08:42.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.879+0000 7f5e5b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e4c67a370 0x556e4c674280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:42.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.879+0000 7f5e5b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e4c67a370 0x556e4c674280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48536/0 (socket says 172.21.15.16:48536) 2024-09-12T02:08:42.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.879+0000 7f5e5b7fe640 1 -- 172.21.15.16:0/293708138 learned_addr learned my addr 172.21.15.16:0/293708138 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:08:42.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.879+0000 7f5e5bfff640 1 --2- 172.21.15.16:0/293708138 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556e4c670b40 0x556e4c673d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:42.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.879+0000 7f5e5b7fe640 1 --2- 172.21.15.16:0/293708138 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556e4c66f5d0 0x556e4c6829f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:42.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.879+0000 7f5e5b7fe640 1 -- 172.21.15.16:0/293708138 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556e4c66f5d0 msgr2=0x556e4c6829f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:42.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.879+0000 7f5e5b7fe640 1 --2- 172.21.15.16:0/293708138 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556e4c66f5d0 0x556e4c6829f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:42.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.879+0000 7f5e5b7fe640 1 -- 172.21.15.16:0/293708138 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556e4c670b40 msgr2=0x556e4c673d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:42.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.879+0000 7f5e5b7fe640 1 --2- 172.21.15.16:0/293708138 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556e4c670b40 0x556e4c673d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:42.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.879+0000 7f5e5b7fe640 1 -- 172.21.15.16:0/293708138 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e4c6748f0 con 0x556e4c67a370 2024-09-12T02:08:42.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.880+0000 7f5e5b7fe640 1 --2- 172.21.15.16:0/293708138 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e4c67a370 0x556e4c674280 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f5e50009d20 tx=0x7f5e500095e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d79b0218a5018c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:42.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.880+0000 7f5e3b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:08:42.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.880+0000 7f5e3b7fe640 1 -- 172.21.15.16:0/293708138 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e50022590 con 0x556e4c67a370 2024-09-12T02:08:42.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.880+0000 7f5e3b7fe640 1 -- 172.21.15.16:0/293708138 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f5e50005040 con 0x556e4c67a370 2024-09-12T02:08:42.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.880+0000 7f5e648af740 1 -- 172.21.15.16:0/293708138 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5e50022a30 con 0x556e4c67a370 2024-09-12T02:08:42.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.880+0000 7f5e3b7fe640 1 -- 172.21.15.16:0/293708138 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5e50005240 con 0x556e4c67a370 2024-09-12T02:08:42.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.880+0000 7f5e648af740 10 client.17463 fetch_fsmap learned FSMap version 25 2024-09-12T02:08:42.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.880+0000 7f5e648af740 1 -- 172.21.15.16:0/293708138 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x556e4c6f48a0 con 0x556e4c67a370 2024-09-12T02:08:42.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.880+0000 7f5e3b7fe640 1 -- 172.21.15.16:0/293708138 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e50022a30 con 0x556e4c67a370 2024-09-12T02:08:42.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.880+0000 7f5e3b7fe640 1 -- 172.21.15.16:0/293708138 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f5e5002c070 con 0x556e4c67a370 2024-09-12T02:08:42.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.880+0000 7f5e648af740 10 client.17463 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:08:42.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.880+0000 7f5e648af740 10 client.17463 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:08:42.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.880+0000 7f5e648af740 20 client.17463 populate_metadata read hostname 'smithi016' 2024-09-12T02:08:42.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.880+0000 7f5e648af740 1 --2- 172.21.15.16:0/293708138 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x556e4c6f4eb0 0x556e4c6f72a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:42.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.880+0000 7f5e648af740 4 client.17463 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T02:08:42.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.880+0000 7f5e648af740 1 -- 172.21.15.16:0/293708138 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556e4c5dcd70 con 0x556e4c6f4eb0 2024-09-12T02:08:42.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.881+0000 7f5e5bfff640 1 --2- 172.21.15.16:0/293708138 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x556e4c6f4eb0 0x556e4c6f72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:42.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.881+0000 7f5e3b7fe640 10 client.17463 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:08:42.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.881+0000 7f5e5bfff640 1 --2- 172.21.15.16:0/293708138 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x556e4c6f4eb0 0x556e4c6f72a0 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83f4b8945f33ab6 server_cookie=8d4a7845f314f2b5 in_seq=0 out_seq=0 2024-09-12T02:08:42.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.881+0000 7f5e3b7fe640 1 -- 172.21.15.16:0/293708138 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556e4c5dcd70 con 0x556e4c6f4eb0 2024-09-12T02:08:42.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.881+0000 7f5e3b7fe640 10 client.17463 handle_command_reply: tid=0 2024-09-12T02:08:42.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.881+0000 7f5e648af740 1 client.17463 shutdown 2024-09-12T02:08:42.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.882+0000 7f5e648af740 1 -- 172.21.15.16:0/293708138 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e4c67a370 msgr2=0x556e4c674280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:42.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.882+0000 7f5e648af740 1 --2- 172.21.15.16:0/293708138 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e4c67a370 0x556e4c674280 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f5e50009d20 tx=0x7f5e500095e0 comp rx=0 tx=0).stop 2024-09-12T02:08:42.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.882+0000 7f5e648af740 1 -- 172.21.15.16:0/293708138 shutdown_connections 2024-09-12T02:08:42.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.882+0000 7f5e648af740 1 --2- 172.21.15.16:0/293708138 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x556e4c6f4eb0 0x556e4c6f72a0 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:42.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.882+0000 7f5e3b7fe640 0 client.17463 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:08:42.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.882+0000 7f5e648af740 1 --2- 172.21.15.16:0/293708138 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e4c67a370 0x556e4c674280 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:42.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.882+0000 7f5e648af740 1 --2- 172.21.15.16:0/293708138 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556e4c670b40 0x556e4c673d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:42.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.882+0000 7f5e648af740 1 --2- 172.21.15.16:0/293708138 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556e4c66f5d0 0x556e4c6829f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:42.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.882+0000 7f5e648af740 1 -- 172.21.15.16:0/293708138 >> 172.21.15.16:0/293708138 conn(0x556e4c638310 msgr2=0x556e4c685300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:42.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.882+0000 7f5e648af740 1 -- 172.21.15.16:0/293708138 shutdown_connections 2024-09-12T02:08:42.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.882+0000 7f5e648af740 1 -- 172.21.15.16:0/293708138 wait complete. 2024-09-12T02:08:42.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.882+0000 7f5e648af740 20 client.17463 trim_cache size 0 max 16384 2024-09-12T02:08:42.888 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T02:08:42.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.882+0000 7f5e624d6640 1 -- 172.21.15.16:0/1030221267 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f5e440a2b80 msgr2=0x7f5e440a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:42.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.882+0000 7f5e624d6640 1 --2- 172.21.15.16:0/1030221267 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f5e440a2b80 0x7f5e440a5040 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f5e540effb0 tx=0x7f5e5403c040 comp rx=0 tx=0).stop 2024-09-12T02:08:42.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.882+0000 7f5e624d6640 1 -- 172.21.15.16:0/1030221267 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e5c10c170 msgr2=0x7f5e5c19ee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:42.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.882+0000 7f5e624d6640 1 --2- 172.21.15.16:0/1030221267 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e5c10c170 0x7f5e5c19ee90 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f5e54004af0 tx=0x7f5e5402f3a0 comp rx=0 tx=0).stop 2024-09-12T02:08:42.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.883+0000 7f5e624d6640 1 -- 172.21.15.16:0/1030221267 shutdown_connections 2024-09-12T02:08:42.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.883+0000 7f5e624d6640 1 --2- 172.21.15.16:0/1030221267 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f5e440a2b80 0x7f5e440a5040 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:42.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.883+0000 7f5e624d6640 1 --2- 172.21.15.16:0/1030221267 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e5c10c170 0x7f5e5c19ee90 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:42.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.883+0000 7f5e624d6640 1 --2- 172.21.15.16:0/1030221267 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5e5c103660 0x7f5e5c19ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:42.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.883+0000 7f5e624d6640 1 --2- 172.21.15.16:0/1030221267 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5e5c102d40 0x7f5e5c19a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:42.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.883+0000 7f5e624d6640 1 -- 172.21.15.16:0/1030221267 >> 172.21.15.16:0/1030221267 conn(0x7f5e5c0fc7a0 msgr2=0x7f5e5c0fcb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:42.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.883+0000 7f5e624d6640 1 -- 172.21.15.16:0/1030221267 shutdown_connections 2024-09-12T02:08:42.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:42.883+0000 7f5e624d6640 1 -- 172.21.15.16:0/1030221267 wait complete. 2024-09-12T02:08:44.627 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-12T02:08:44.627 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-12T02:08:44.628 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:08:44 smithi184 ceph-mon[35593]: pgmap v539: 97 pgs: 97 active+clean; 3.7 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 8.0 KiB/s wr, 10 op/s 2024-09-12T02:08:44.656 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:08:44 smithi016 ceph-mon[30947]: pgmap v539: 97 pgs: 97 active+clean; 3.7 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 8.0 KiB/s wr, 10 op/s 2024-09-12T02:08:44.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:08:44 smithi033 ceph-mon[35332]: pgmap v539: 97 pgs: 97 active+clean; 3.7 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 8.0 KiB/s wr, 10 op/s 2024-09-12T02:08:44.899 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:08:45.100 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.100+0000 7fc844e65640 1 -- 172.21.15.16:0/2268040994 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8401025e0 msgr2=0x7fc8401049d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:45.100 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.100+0000 7fc844e65640 1 --2- 172.21.15.16:0/2268040994 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8401025e0 0x7fc8401049d0 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7fc830009ca0 tx=0x7fc83002a8c0 comp rx=0 tx=0).stop 2024-09-12T02:08:45.100 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.100+0000 7fc844e65640 1 -- 172.21.15.16:0/2268040994 shutdown_connections 2024-09-12T02:08:45.101 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.100+0000 7fc844e65640 1 --2- 172.21.15.16:0/2268040994 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc840107840 0x7fc840109cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:45.101 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.100+0000 7fc844e65640 1 --2- 172.21.15.16:0/2268040994 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc840104f10 0x7fc840107300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:45.101 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.100+0000 7fc844e65640 1 --2- 172.21.15.16:0/2268040994 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8401025e0 0x7fc8401049d0 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:45.101 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.100+0000 7fc844e65640 1 -- 172.21.15.16:0/2268040994 >> 172.21.15.16:0/2268040994 conn(0x7fc8400fc1c0 msgr2=0x7fc8400fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:45.101 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.100+0000 7fc844e65640 1 -- 172.21.15.16:0/2268040994 shutdown_connections 2024-09-12T02:08:45.101 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.100+0000 7fc844e65640 1 -- 172.21.15.16:0/2268040994 wait complete. 2024-09-12T02:08:45.103 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.101+0000 7fc844e65640 1 Processor -- start 2024-09-12T02:08:45.103 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.101+0000 7fc844e65640 1 -- start start 2024-09-12T02:08:45.103 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.101+0000 7fc844e65640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8401025e0 0x7fc840197e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:45.103 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.101+0000 7fc844e65640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc840104f10 0x7fc840198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:45.103 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.101+0000 7fc844e65640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc840107840 0x7fc84019f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:45.104 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.101+0000 7fc844e65640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc84010d890 con 0x7fc8401025e0 2024-09-12T02:08:45.104 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.101+0000 7fc844e65640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc84010d710 con 0x7fc840107840 2024-09-12T02:08:45.104 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.101+0000 7fc844e65640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc84010da10 con 0x7fc840104f10 2024-09-12T02:08:45.104 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.101+0000 7fc83e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8401025e0 0x7fc840197e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:45.104 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.101+0000 7fc83e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8401025e0 0x7fc840197e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48560/0 (socket says 172.21.15.16:48560) 2024-09-12T02:08:45.104 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.101+0000 7fc83e59c640 1 -- 172.21.15.16:0/1385976918 learned_addr learned my addr 172.21.15.16:0/1385976918 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:08:45.104 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.101+0000 7fc83ed9d640 1 --2- 172.21.15.16:0/1385976918 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc840107840 0x7fc84019f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:45.104 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.101+0000 7fc83dd9b640 1 --2- 172.21.15.16:0/1385976918 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc840104f10 0x7fc840198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:45.104 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.102+0000 7fc83e59c640 1 -- 172.21.15.16:0/1385976918 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc840104f10 msgr2=0x7fc840198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:45.104 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.102+0000 7fc83e59c640 1 --2- 172.21.15.16:0/1385976918 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc840104f10 0x7fc840198360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:45.104 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.102+0000 7fc83e59c640 1 -- 172.21.15.16:0/1385976918 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc840107840 msgr2=0x7fc84019f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:45.104 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.102+0000 7fc83e59c640 1 --2- 172.21.15.16:0/1385976918 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc840107840 0x7fc84019f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:45.105 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.102+0000 7fc83e59c640 1 -- 172.21.15.16:0/1385976918 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc84019fa40 con 0x7fc8401025e0 2024-09-12T02:08:45.105 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.102+0000 7fc83e59c640 1 --2- 172.21.15.16:0/1385976918 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8401025e0 0x7fc840197e20 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7fc830000f80 tx=0x7fc8300319f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:45.105 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.102+0000 7fc8377fe640 1 -- 172.21.15.16:0/1385976918 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc830003c70 con 0x7fc8401025e0 2024-09-12T02:08:45.105 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.102+0000 7fc8377fe640 1 -- 172.21.15.16:0/1385976918 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc830003e10 con 0x7fc8401025e0 2024-09-12T02:08:45.105 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.102+0000 7fc844e65640 1 -- 172.21.15.16:0/1385976918 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc84019fcd0 con 0x7fc8401025e0 2024-09-12T02:08:45.105 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.102+0000 7fc8377fe640 1 -- 172.21.15.16:0/1385976918 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc83002fdc0 con 0x7fc8401025e0 2024-09-12T02:08:45.105 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.102+0000 7fc844e65640 1 -- 172.21.15.16:0/1385976918 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8401a01c0 con 0x7fc8401025e0 2024-09-12T02:08:45.112 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.104+0000 7fc8377fe640 1 -- 172.21.15.16:0/1385976918 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fc830004040 con 0x7fc8401025e0 2024-09-12T02:08:45.112 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.104+0000 7fc847c02740 1 Processor -- start 2024-09-12T02:08:45.112 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.105+0000 7fc847c02740 1 -- start start 2024-09-12T02:08:45.112 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.105+0000 7fc847c02740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b8c132db0 0x558b8c1331b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:45.113 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.105+0000 7fc847c02740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558b8c1336f0 0x558b8c1381b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:45.113 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.105+0000 7fc847c02740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b8c138780 0x558b8c13ac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:45.113 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.105+0000 7fc847c02740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x558b8c0922f0 con 0x558b8c132db0 2024-09-12T02:08:45.113 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.105+0000 7fc847c02740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x558b8c03a270 con 0x558b8c138780 2024-09-12T02:08:45.113 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.105+0000 7fc847c02740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x558b8c08e2b0 con 0x558b8c1336f0 2024-09-12T02:08:45.113 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.105+0000 7fc83ed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b8c132db0 0x558b8c1331b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:45.113 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.105+0000 7fc8377fe640 1 --2- 172.21.15.16:0/1385976918 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc82c0a2cb0 0x7fc82c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:45.113 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.105+0000 7fc83ed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b8c132db0 0x558b8c1331b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48564/0 (socket says 172.21.15.16:48564) 2024-09-12T02:08:45.113 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.105+0000 7fc83ed9d640 1 -- 172.21.15.16:0/3073982661 learned_addr learned my addr 172.21.15.16:0/3073982661 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:08:45.113 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.105+0000 7fc83e59c640 1 --2- 172.21.15.16:0/3073982661 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558b8c1336f0 0x558b8c1381b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:45.114 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.105+0000 7fc83dd9b640 1 --2- 172.21.15.16:0/3073982661 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b8c138780 0x558b8c13ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:45.114 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.105+0000 7fc83ed9d640 1 --2- 172.21.15.16:0/1385976918 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc82c0a2cb0 0x7fc82c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:45.114 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.105+0000 7fc8377fe640 1 -- 172.21.15.16:0/1385976918 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fc8300f03f0 con 0x7fc8401025e0 2024-09-12T02:08:45.114 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.105+0000 7fc83ed9d640 1 -- 172.21.15.16:0/3073982661 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558b8c1336f0 msgr2=0x558b8c1381b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:45.114 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.105+0000 7fc83ed9d640 1 --2- 172.21.15.16:0/3073982661 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558b8c1336f0 0x558b8c1381b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:45.114 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.105+0000 7fc83ed9d640 1 -- 172.21.15.16:0/3073982661 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b8c138780 msgr2=0x558b8c13ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:45.114 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.105+0000 7fc83ed9d640 1 --2- 172.21.15.16:0/3073982661 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b8c138780 0x558b8c13ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:45.114 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.105+0000 7fc83ed9d640 1 -- 172.21.15.16:0/3073982661 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b8c039340 con 0x558b8c132db0 2024-09-12T02:08:45.114 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.106+0000 7fc83ed9d640 1 --2- 172.21.15.16:0/3073982661 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b8c132db0 0x558b8c1331b0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7fc8380137f0 tx=0x7fc838013cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11fb8ca8ebb177f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:45.114 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.106+0000 7fc834ff9640 1 -- 172.21.15.16:0/3073982661 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc838021030 con 0x558b8c132db0 2024-09-12T02:08:45.114 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.106+0000 7fc834ff9640 1 -- 172.21.15.16:0/3073982661 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc83800ad00 con 0x558b8c132db0 2024-09-12T02:08:45.114 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.106+0000 7fc834ff9640 1 -- 172.21.15.16:0/3073982661 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc83801d660 con 0x558b8c132db0 2024-09-12T02:08:45.114 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.106+0000 7fc847c02740 1 -- 172.21.15.16:0/3073982661 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b8c132db0 msgr2=0x558b8c1331b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:45.115 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.106+0000 7fc83ed9d640 1 --2- 172.21.15.16:0/1385976918 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc82c0a2cb0 0x7fc82c0a5170 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fc838021000 tx=0x7fc838020000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:45.115 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.106+0000 7fc847c02740 1 --2- 172.21.15.16:0/3073982661 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b8c132db0 0x558b8c1331b0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7fc8380137f0 tx=0x7fc838013cb0 comp rx=0 tx=0).stop 2024-09-12T02:08:45.115 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.106+0000 7fc847c02740 1 -- 172.21.15.16:0/3073982661 shutdown_connections 2024-09-12T02:08:45.115 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.106+0000 7fc847c02740 1 --2- 172.21.15.16:0/3073982661 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b8c138780 0x558b8c13ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:45.115 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.106+0000 7fc847c02740 1 --2- 172.21.15.16:0/3073982661 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558b8c1336f0 0x558b8c1381b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:45.115 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.106+0000 7fc847c02740 1 --2- 172.21.15.16:0/3073982661 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b8c132db0 0x558b8c1331b0 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:45.115 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.106+0000 7fc847c02740 1 -- 172.21.15.16:0/3073982661 >> 172.21.15.16:0/3073982661 conn(0x558b8c12a010 msgr2=0x558b8c12a410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:45.115 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.107+0000 7fc847c02740 1 -- 172.21.15.16:0/3073982661 shutdown_connections 2024-09-12T02:08:45.115 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.107+0000 7fc847c02740 1 -- 172.21.15.16:0/3073982661 wait complete. 2024-09-12T02:08:45.115 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.107+0000 7fc847c02740 1 -- start start 2024-09-12T02:08:45.115 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.107+0000 7fc847c02740 1 Processor -- start 2024-09-12T02:08:45.115 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.107+0000 7fc847c02740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b8c132db0 0x558b8c13d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:45.115 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.107+0000 7fc847c02740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558b8c1336f0 0x558b8c13d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:45.116 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.107+0000 7fc847c02740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b8c138780 0x558b8c144550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:45.116 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.107+0000 7fc847c02740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x558b8c13b750 con 0x558b8c132db0 2024-09-12T02:08:45.116 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.107+0000 7fc847c02740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x558b8c01b290 con 0x558b8c138780 2024-09-12T02:08:45.116 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.107+0000 7fc847c02740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x558b8c13b8d0 con 0x558b8c1336f0 2024-09-12T02:08:45.116 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.107+0000 7fc847c02740 10 client.0 _ll_register_callbacks cb 0x558b8c04c140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:08:45.116 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.107+0000 7fc83e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b8c132db0 0x558b8c13d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:45.116 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.107+0000 7fc83e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b8c132db0 0x558b8c13d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48570/0 (socket says 172.21.15.16:48570) 2024-09-12T02:08:45.116 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.107+0000 7fc83e59c640 1 -- 172.21.15.16:0/3549298208 learned_addr learned my addr 172.21.15.16:0/3549298208 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:08:45.116 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.107+0000 7fc83ed9d640 1 --2- 172.21.15.16:0/3549298208 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b8c138780 0x558b8c144550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:45.116 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.107+0000 7fc83dd9b640 1 --2- 172.21.15.16:0/3549298208 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558b8c1336f0 0x558b8c13d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:45.116 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.108+0000 7fc83e59c640 1 -- 172.21.15.16:0/3549298208 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558b8c1336f0 msgr2=0x558b8c13d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:45.116 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.108+0000 7fc83e59c640 1 --2- 172.21.15.16:0/3549298208 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558b8c1336f0 0x558b8c13d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:45.117 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.108+0000 7fc83e59c640 1 -- 172.21.15.16:0/3549298208 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b8c138780 msgr2=0x558b8c144550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:45.117 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.108+0000 7fc83e59c640 1 --2- 172.21.15.16:0/3549298208 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b8c138780 0x558b8c144550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:45.117 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.108+0000 7fc83e59c640 1 -- 172.21.15.16:0/3549298208 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b8bda5210 con 0x558b8c132db0 2024-09-12T02:08:45.117 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.108+0000 7fc83dd9b640 1 --2- 172.21.15.16:0/3549298208 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558b8c1336f0 0x558b8c13d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:08:45.117 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.108+0000 7fc83ed9d640 1 --2- 172.21.15.16:0/3549298208 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b8c138780 0x558b8c144550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:08:45.117 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.108+0000 7fc83e59c640 1 --2- 172.21.15.16:0/3549298208 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b8c132db0 0x558b8c13d040 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7fc8300b91e0 tx=0x7fc8300b90e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1919f5568a0c89e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:45.117 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.108+0000 7fc825ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:08:45.117 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.108+0000 7fc825ffb640 1 -- 172.21.15.16:0/3549298208 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc83008ef20 con 0x558b8c132db0 2024-09-12T02:08:45.117 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.108+0000 7fc825ffb640 1 -- 172.21.15.16:0/3549298208 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fc8300b93f0 con 0x558b8c132db0 2024-09-12T02:08:45.117 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.108+0000 7fc847c02740 1 -- 172.21.15.16:0/3549298208 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc83800ad00 con 0x558b8c132db0 2024-09-12T02:08:45.117 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.108+0000 7fc825ffb640 1 -- 172.21.15.16:0/3549298208 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc8300bd480 con 0x558b8c132db0 2024-09-12T02:08:45.117 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.108+0000 7fc847c02740 10 client.17475 fetch_fsmap learned FSMap version 25 2024-09-12T02:08:45.117 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.108+0000 7fc847c02740 1 -- 172.21.15.16:0/3549298208 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x558b8be71620 con 0x558b8c132db0 2024-09-12T02:08:45.118 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.108+0000 7fc825ffb640 1 -- 172.21.15.16:0/3549298208 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc83800ad00 con 0x558b8c132db0 2024-09-12T02:08:45.118 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.108+0000 7fc825ffb640 1 -- 172.21.15.16:0/3549298208 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fc8300d5050 con 0x558b8c132db0 2024-09-12T02:08:45.118 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.108+0000 7fc847c02740 10 client.17475 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:08:45.118 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.108+0000 7fc847c02740 10 client.17475 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:08:45.118 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.108+0000 7fc847c02740 20 client.17475 populate_metadata read hostname 'smithi016' 2024-09-12T02:08:45.118 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.108+0000 7fc847c02740 1 --2- 172.21.15.16:0/3549298208 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x558b8c148c60 0x558b8c14b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:45.118 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.108+0000 7fc847c02740 4 client.17475 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:08:45.118 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.108+0000 7fc847c02740 1 -- 172.21.15.16:0/3549298208 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558b8c09fa20 con 0x558b8c148c60 2024-09-12T02:08:45.118 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.109+0000 7fc83dd9b640 1 --2- 172.21.15.16:0/3549298208 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x558b8c148c60 0x558b8c14b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:45.118 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.109+0000 7fc83dd9b640 1 --2- 172.21.15.16:0/3549298208 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x558b8c148c60 0x558b8c14b090 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=508b2b227f4e78b0 server_cookie=843e71c4ae46c37e in_seq=0 out_seq=0 2024-09-12T02:08:45.118 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.109+0000 7fc825ffb640 10 client.17475 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:08:45.118 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.110+0000 7fc825ffb640 1 -- 172.21.15.16:0/3549298208 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558b8c09fa20 con 0x558b8c148c60 2024-09-12T02:08:45.118 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.110+0000 7fc825ffb640 10 client.17475 handle_command_reply: tid=0 2024-09-12T02:08:45.119 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.110+0000 7fc847c02740 1 client.17475 shutdown 2024-09-12T02:08:45.119 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.110+0000 7fc847c02740 1 -- 172.21.15.16:0/3549298208 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b8c132db0 msgr2=0x558b8c13d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:45.119 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.110+0000 7fc847c02740 1 --2- 172.21.15.16:0/3549298208 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b8c132db0 0x558b8c13d040 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7fc8300b91e0 tx=0x7fc8300b90e0 comp rx=0 tx=0).stop 2024-09-12T02:08:45.119 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.110+0000 7fc847c02740 1 -- 172.21.15.16:0/3549298208 shutdown_connections 2024-09-12T02:08:45.119 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.110+0000 7fc847c02740 1 --2- 172.21.15.16:0/3549298208 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x558b8c148c60 0x558b8c14b090 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:45.119 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.110+0000 7fc847c02740 1 --2- 172.21.15.16:0/3549298208 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b8c138780 0x558b8c144550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:45.119 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.110+0000 7fc847c02740 1 --2- 172.21.15.16:0/3549298208 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558b8c1336f0 0x558b8c13d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:45.119 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.110+0000 7fc825ffb640 0 client.17475 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:08:45.119 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.110+0000 7fc847c02740 1 --2- 172.21.15.16:0/3549298208 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b8c132db0 0x558b8c13d040 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:45.119 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.110+0000 7fc847c02740 1 -- 172.21.15.16:0/3549298208 >> 172.21.15.16:0/3549298208 conn(0x558b8c12a010 msgr2=0x558b8c12b820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:45.119 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.110+0000 7fc847c02740 1 -- 172.21.15.16:0/3549298208 shutdown_connections 2024-09-12T02:08:45.119 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.110+0000 7fc847c02740 1 -- 172.21.15.16:0/3549298208 wait complete. 2024-09-12T02:08:45.119 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.110+0000 7fc847c02740 20 client.17475 trim_cache size 0 max 16384 2024-09-12T02:08:45.129 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.124+0000 7fc847c02740 1 Processor -- start 2024-09-12T02:08:45.129 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.124+0000 7fc847c02740 1 -- start start 2024-09-12T02:08:45.129 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.125+0000 7fc847c02740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558b8c132db0 0x558b8c13d610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:45.129 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.125+0000 7fc847c02740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b8c1336f0 0x558b8c144510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:45.129 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.125+0000 7fc847c02740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b8c138780 0x558b8c1290b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:45.129 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.125+0000 7fc847c02740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc8401a9450 con 0x558b8c1336f0 2024-09-12T02:08:45.129 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.125+0000 7fc847c02740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc8401a6d20 con 0x558b8c138780 2024-09-12T02:08:45.130 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.125+0000 7fc847c02740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc8401a4b90 con 0x558b8c132db0 2024-09-12T02:08:45.130 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.125+0000 7fc83dd9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b8c1336f0 0x558b8c144510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:45.130 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.125+0000 7fc83dd9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b8c1336f0 0x558b8c144510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48572/0 (socket says 172.21.15.16:48572) 2024-09-12T02:08:45.130 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.125+0000 7fc83dd9b640 1 -- 172.21.15.16:0/4197965885 learned_addr learned my addr 172.21.15.16:0/4197965885 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:08:45.130 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.125+0000 7fc83e59c640 1 --2- 172.21.15.16:0/4197965885 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558b8c132db0 0x558b8c13d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:45.130 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.125+0000 7fc83dd9b640 1 -- 172.21.15.16:0/4197965885 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558b8c132db0 msgr2=0x558b8c13d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:45.130 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.125+0000 7fc83dd9b640 1 --2- 172.21.15.16:0/4197965885 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558b8c132db0 0x558b8c13d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:45.130 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.125+0000 7fc83dd9b640 1 -- 172.21.15.16:0/4197965885 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b8c138780 msgr2=0x558b8c1290b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:45.130 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.125+0000 7fc83dd9b640 1 --2- 172.21.15.16:0/4197965885 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b8c138780 0x558b8c1290b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:45.130 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.125+0000 7fc83dd9b640 1 -- 172.21.15.16:0/4197965885 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b8c13cb80 con 0x558b8c1336f0 2024-09-12T02:08:45.130 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.125+0000 7fc83e59c640 1 --2- 172.21.15.16:0/4197965885 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558b8c132db0 0x558b8c13d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:08:45.130 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.125+0000 7fc83dd9b640 1 --2- 172.21.15.16:0/4197965885 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b8c1336f0 0x558b8c144510 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7fc828005f00 tx=0x7fc82800b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fed36f39db32e1f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:45.131 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.125+0000 7fc827fff640 1 -- 172.21.15.16:0/4197965885 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc828003e40 con 0x558b8c1336f0 2024-09-12T02:08:45.131 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.125+0000 7fc827fff640 1 -- 172.21.15.16:0/4197965885 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc828026920 con 0x558b8c1336f0 2024-09-12T02:08:45.131 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.125+0000 7fc827fff640 1 -- 172.21.15.16:0/4197965885 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc828011aa0 con 0x558b8c1336f0 2024-09-12T02:08:45.131 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.126+0000 7fc847c02740 1 -- 172.21.15.16:0/4197965885 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b8c1336f0 msgr2=0x558b8c144510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:45.131 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.126+0000 7fc847c02740 1 --2- 172.21.15.16:0/4197965885 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b8c1336f0 0x558b8c144510 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7fc828005f00 tx=0x7fc82800b390 comp rx=0 tx=0).stop 2024-09-12T02:08:45.131 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.126+0000 7fc847c02740 1 -- 172.21.15.16:0/4197965885 shutdown_connections 2024-09-12T02:08:45.131 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.126+0000 7fc847c02740 1 --2- 172.21.15.16:0/4197965885 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b8c138780 0x558b8c1290b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:45.131 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.126+0000 7fc847c02740 1 --2- 172.21.15.16:0/4197965885 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b8c1336f0 0x558b8c144510 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:45.131 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.126+0000 7fc847c02740 1 --2- 172.21.15.16:0/4197965885 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558b8c132db0 0x558b8c13d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:45.131 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.126+0000 7fc847c02740 1 -- 172.21.15.16:0/4197965885 >> 172.21.15.16:0/4197965885 conn(0x558b8c12a010 msgr2=0x558b8c1289a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:45.131 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.126+0000 7fc847c02740 1 -- 172.21.15.16:0/4197965885 shutdown_connections 2024-09-12T02:08:45.131 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.126+0000 7fc847c02740 1 -- 172.21.15.16:0/4197965885 wait complete. 2024-09-12T02:08:45.131 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.126+0000 7fc847c02740 1 -- start start 2024-09-12T02:08:45.131 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.126+0000 7fc847c02740 1 Processor -- start 2024-09-12T02:08:45.132 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.127+0000 7fc847c02740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b8c132db0 0x558b8c142400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:45.132 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.127+0000 7fc847c02740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558b8c1336f0 0x558b8c142940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:45.132 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.127+0000 7fc847c02740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b8c138780 0x558b8c143950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:45.132 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.127+0000 7fc847c02740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc8401a51e0 con 0x558b8c138780 2024-09-12T02:08:45.132 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.127+0000 7fc847c02740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc8401a6650 con 0x558b8c132db0 2024-09-12T02:08:45.132 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.127+0000 7fc847c02740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc8401a58b0 con 0x558b8c1336f0 2024-09-12T02:08:45.132 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.127+0000 7fc847c02740 10 client.0 _ll_register_callbacks cb 0x7fc83002ff60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:08:45.132 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.127+0000 7fc83ed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b8c138780 0x558b8c143950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:45.132 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.127+0000 7fc83ed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b8c138780 0x558b8c143950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48584/0 (socket says 172.21.15.16:48584) 2024-09-12T02:08:45.132 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.127+0000 7fc83ed9d640 1 -- 172.21.15.16:0/2216029058 learned_addr learned my addr 172.21.15.16:0/2216029058 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:08:45.132 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.127+0000 7fc83dd9b640 1 --2- 172.21.15.16:0/2216029058 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558b8c1336f0 0x558b8c142940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:45.132 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.127+0000 7fc83e59c640 1 --2- 172.21.15.16:0/2216029058 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b8c132db0 0x558b8c142400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:45.133 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.127+0000 7fc83ed9d640 1 -- 172.21.15.16:0/2216029058 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558b8c1336f0 msgr2=0x558b8c142940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:45.133 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.127+0000 7fc83ed9d640 1 --2- 172.21.15.16:0/2216029058 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558b8c1336f0 0x558b8c142940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:45.133 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.127+0000 7fc83ed9d640 1 -- 172.21.15.16:0/2216029058 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b8c132db0 msgr2=0x558b8c142400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:45.133 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.127+0000 7fc83ed9d640 1 --2- 172.21.15.16:0/2216029058 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b8c132db0 0x558b8c142400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:45.133 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.127+0000 7fc83ed9d640 1 -- 172.21.15.16:0/2216029058 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b8c132750 con 0x558b8c138780 2024-09-12T02:08:45.133 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.127+0000 7fc83ed9d640 1 --2- 172.21.15.16:0/2216029058 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b8c138780 0x558b8c143950 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7fc838005790 tx=0x7fc83800a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc3738b169fcaa2f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:45.133 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.127+0000 7fc825ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:08:45.133 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.127+0000 7fc825ffb640 1 -- 172.21.15.16:0/2216029058 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc83801d610 con 0x558b8c138780 2024-09-12T02:08:45.133 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.127+0000 7fc825ffb640 1 -- 172.21.15.16:0/2216029058 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fc838013070 con 0x558b8c138780 2024-09-12T02:08:45.133 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.127+0000 7fc847c02740 1 -- 172.21.15.16:0/2216029058 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc828026920 con 0x558b8c138780 2024-09-12T02:08:45.133 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.128+0000 7fc825ffb640 1 -- 172.21.15.16:0/2216029058 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc838013270 con 0x558b8c138780 2024-09-12T02:08:45.133 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.128+0000 7fc847c02740 10 client.17481 fetch_fsmap learned FSMap version 25 2024-09-12T02:08:45.133 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.128+0000 7fc847c02740 1 -- 172.21.15.16:0/2216029058 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x558b8c1b8560 con 0x558b8c138780 2024-09-12T02:08:45.134 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.128+0000 7fc825ffb640 1 -- 172.21.15.16:0/2216029058 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc828026920 con 0x558b8c138780 2024-09-12T02:08:45.134 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.128+0000 7fc825ffb640 1 -- 172.21.15.16:0/2216029058 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fc8380037e0 con 0x558b8c138780 2024-09-12T02:08:45.134 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.128+0000 7fc847c02740 10 client.17481 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:08:45.134 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.128+0000 7fc847c02740 10 client.17481 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:08:45.134 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.128+0000 7fc847c02740 20 client.17481 populate_metadata read hostname 'smithi016' 2024-09-12T02:08:45.134 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.128+0000 7fc847c02740 1 --2- 172.21.15.16:0/2216029058 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x558b8c1b8c60 0x558b8c1bb080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:45.134 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.128+0000 7fc847c02740 4 client.17481 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:08:45.134 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.128+0000 7fc847c02740 1 -- 172.21.15.16:0/2216029058 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558b8c099d70 con 0x558b8c1b8c60 2024-09-12T02:08:45.134 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.128+0000 7fc83e59c640 1 --2- 172.21.15.16:0/2216029058 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x558b8c1b8c60 0x558b8c1bb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:45.134 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.129+0000 7fc825ffb640 10 client.17481 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:08:45.134 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.129+0000 7fc83e59c640 1 --2- 172.21.15.16:0/2216029058 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x558b8c1b8c60 0x558b8c1bb080 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4bffe7bfeb91920e server_cookie=47456bb72031179a in_seq=0 out_seq=0 2024-09-12T02:08:45.144 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.143+0000 7fc825ffb640 1 -- 172.21.15.16:0/2216029058 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558b8c099d70 con 0x558b8c1b8c60 2024-09-12T02:08:45.144 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.143+0000 7fc825ffb640 10 client.17481 handle_command_reply: tid=0 2024-09-12T02:08:45.144 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.143+0000 7fc847c02740 1 client.17481 shutdown 2024-09-12T02:08:45.144 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.143+0000 7fc847c02740 1 -- 172.21.15.16:0/2216029058 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b8c138780 msgr2=0x558b8c143950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:45.144 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.143+0000 7fc847c02740 1 --2- 172.21.15.16:0/2216029058 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b8c138780 0x558b8c143950 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7fc838005790 tx=0x7fc83800a6d0 comp rx=0 tx=0).stop 2024-09-12T02:08:45.144 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.143+0000 7fc847c02740 1 -- 172.21.15.16:0/2216029058 shutdown_connections 2024-09-12T02:08:45.144 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.143+0000 7fc847c02740 1 --2- 172.21.15.16:0/2216029058 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x558b8c1b8c60 0x558b8c1bb080 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:45.144 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.143+0000 7fc847c02740 1 --2- 172.21.15.16:0/2216029058 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b8c138780 0x558b8c143950 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:45.144 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.143+0000 7fc847c02740 1 --2- 172.21.15.16:0/2216029058 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558b8c1336f0 0x558b8c142940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:45.144 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.143+0000 7fc825ffb640 0 client.17481 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:08:45.145 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.143+0000 7fc847c02740 1 --2- 172.21.15.16:0/2216029058 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b8c132db0 0x558b8c142400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:45.145 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.143+0000 7fc847c02740 1 -- 172.21.15.16:0/2216029058 >> 172.21.15.16:0/2216029058 conn(0x558b8c12a010 msgr2=0x558b8c1291c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:45.145 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.143+0000 7fc847c02740 1 -- 172.21.15.16:0/2216029058 shutdown_connections 2024-09-12T02:08:45.145 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.143+0000 7fc847c02740 1 -- 172.21.15.16:0/2216029058 wait complete. 2024-09-12T02:08:45.145 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.143+0000 7fc847c02740 20 client.17481 trim_cache size 0 max 16384 2024-09-12T02:08:45.145 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-09-12T02:08:45.145 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-09-12T02:08:45.145 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-09-12T02:08:45.145 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-09-12T02:08:45.146 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.144+0000 7fc844e65640 1 -- 172.21.15.16:0/1385976918 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc82c0a2cb0 msgr2=0x7fc82c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:45.146 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.144+0000 7fc844e65640 1 --2- 172.21.15.16:0/1385976918 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc82c0a2cb0 0x7fc82c0a5170 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fc838021000 tx=0x7fc838020000 comp rx=0 tx=0).stop 2024-09-12T02:08:45.146 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.144+0000 7fc844e65640 1 -- 172.21.15.16:0/1385976918 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8401025e0 msgr2=0x7fc840197e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:45.146 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.144+0000 7fc844e65640 1 --2- 172.21.15.16:0/1385976918 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8401025e0 0x7fc840197e20 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7fc830000f80 tx=0x7fc8300319f0 comp rx=0 tx=0).stop 2024-09-12T02:08:45.146 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.144+0000 7fc844e65640 1 -- 172.21.15.16:0/1385976918 shutdown_connections 2024-09-12T02:08:45.146 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.144+0000 7fc844e65640 1 --2- 172.21.15.16:0/1385976918 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc82c0a2cb0 0x7fc82c0a5170 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:45.146 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.144+0000 7fc844e65640 1 --2- 172.21.15.16:0/1385976918 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc840107840 0x7fc84019f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:45.147 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.144+0000 7fc844e65640 1 --2- 172.21.15.16:0/1385976918 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc840104f10 0x7fc840198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:45.147 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.144+0000 7fc844e65640 1 --2- 172.21.15.16:0/1385976918 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8401025e0 0x7fc840197e20 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:45.147 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.144+0000 7fc844e65640 1 -- 172.21.15.16:0/1385976918 >> 172.21.15.16:0/1385976918 conn(0x7fc8400fc1c0 msgr2=0x7fc840102e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:45.147 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.144+0000 7fc844e65640 1 -- 172.21.15.16:0/1385976918 shutdown_connections 2024-09-12T02:08:45.147 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:45.144+0000 7fc844e65640 1 -- 172.21.15.16:0/1385976918 wait complete. 2024-09-12T02:08:45.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.404+0000 7fb8f97d0640 1 -- 172.21.15.16:0/2865682891 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8f4102d30 msgr2=0x7fb8f4103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:45.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.404+0000 7fb8f97d0640 1 --2- 172.21.15.16:0/2865682891 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8f4102d30 0x7fb8f4103110 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7fb8e4009930 tx=0x7fb8e402a420 comp rx=0 tx=0).stop 2024-09-12T02:08:45.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.404+0000 7fb8f97d0640 1 -- 172.21.15.16:0/2865682891 shutdown_connections 2024-09-12T02:08:45.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.404+0000 7fb8f97d0640 1 --2- 172.21.15.16:0/2865682891 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb8f41082a0 0x7fb8f410ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:45.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.404+0000 7fb8f97d0640 1 --2- 172.21.15.16:0/2865682891 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb8f41036e0 0x7fb8f4107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:45.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.404+0000 7fb8f97d0640 1 --2- 172.21.15.16:0/2865682891 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8f4102d30 0x7fb8f4103110 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:45.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.404+0000 7fb8f97d0640 1 -- 172.21.15.16:0/2865682891 >> 172.21.15.16:0/2865682891 conn(0x7fb8f40fc7d0 msgr2=0x7fb8f40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:45.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.404+0000 7fb8f97d0640 1 -- 172.21.15.16:0/2865682891 shutdown_connections 2024-09-12T02:08:45.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.404+0000 7fb8f97d0640 1 -- 172.21.15.16:0/2865682891 wait complete. 2024-09-12T02:08:45.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.404+0000 7fb8f97d0640 1 Processor -- start 2024-09-12T02:08:45.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.405+0000 7fb8f97d0640 1 -- start start 2024-09-12T02:08:45.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.405+0000 7fb8f97d0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb8f4102d30 0x7fb8f4100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:45.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.405+0000 7fb8f97d0640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb8f41036e0 0x7fb8f4100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:45.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.405+0000 7fb8f97d0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8f41082a0 0x7fb8f4101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:45.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.405+0000 7fb8f97d0640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb8f41120a0 con 0x7fb8f41082a0 2024-09-12T02:08:45.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.405+0000 7fb8f97d0640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb8f4111f20 con 0x7fb8f4102d30 2024-09-12T02:08:45.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.405+0000 7fb8f97d0640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb8f4112220 con 0x7fb8f41036e0 2024-09-12T02:08:45.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.405+0000 7fb8f37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8f41082a0 0x7fb8f4101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:45.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.405+0000 7fb8f37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8f41082a0 0x7fb8f4101410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48604/0 (socket says 172.21.15.16:48604) 2024-09-12T02:08:45.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.405+0000 7fb8f37fe640 1 -- 172.21.15.16:0/1827918201 learned_addr learned my addr 172.21.15.16:0/1827918201 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:08:45.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.405+0000 7fb8f2ffd640 1 --2- 172.21.15.16:0/1827918201 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb8f4102d30 0x7fb8f4100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:45.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.405+0000 7fb8f27fc640 1 --2- 172.21.15.16:0/1827918201 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb8f41036e0 0x7fb8f4100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:45.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.405+0000 7fb8f37fe640 1 -- 172.21.15.16:0/1827918201 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb8f41036e0 msgr2=0x7fb8f4100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:45.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.405+0000 7fb8f37fe640 1 --2- 172.21.15.16:0/1827918201 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb8f41036e0 0x7fb8f4100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:45.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.405+0000 7fb8f37fe640 1 -- 172.21.15.16:0/1827918201 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb8f4102d30 msgr2=0x7fb8f4100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:45.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.405+0000 7fb8f37fe640 1 --2- 172.21.15.16:0/1827918201 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb8f4102d30 0x7fb8f4100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:45.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.405+0000 7fb8f37fe640 1 -- 172.21.15.16:0/1827918201 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8f41a6a40 con 0x7fb8f41082a0 2024-09-12T02:08:45.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.406+0000 7fb8f37fe640 1 --2- 172.21.15.16:0/1827918201 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8f41082a0 0x7fb8f4101410 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7fb8ec00eaa0 tx=0x7fb8ec00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:45.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.406+0000 7fb8dbfff640 1 -- 172.21.15.16:0/1827918201 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8ec00cdb0 con 0x7fb8f41082a0 2024-09-12T02:08:45.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.406+0000 7fb8dbfff640 1 -- 172.21.15.16:0/1827918201 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8ec005020 con 0x7fb8f41082a0 2024-09-12T02:08:45.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.406+0000 7fb8dbfff640 1 -- 172.21.15.16:0/1827918201 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8ec0116b0 con 0x7fb8f41082a0 2024-09-12T02:08:45.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.406+0000 7fb8f97d0640 1 -- 172.21.15.16:0/1827918201 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8f41a6d30 con 0x7fb8f41082a0 2024-09-12T02:08:45.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.406+0000 7fb8f97d0640 1 -- 172.21.15.16:0/1827918201 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8f41a7220 con 0x7fb8f41082a0 2024-09-12T02:08:45.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.408+0000 7fb8dbfff640 1 -- 172.21.15.16:0/1827918201 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fb8ec005560 con 0x7fb8f41082a0 2024-09-12T02:08:45.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.409+0000 7fb8fbba9740 1 Processor -- start 2024-09-12T02:08:45.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.409+0000 7fb8fbba9740 1 -- start start 2024-09-12T02:08:45.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.409+0000 7fb8fbba9740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620bff3eb60 0x5620bff3ef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:45.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.409+0000 7fb8fbba9740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620bff3f480 0x5620bff49800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:45.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.409+0000 7fb8fbba9740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5620bff49d40 0x5620bff4c130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:45.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.409+0000 7fb8fbba9740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5620bfea4280 con 0x5620bff3eb60 2024-09-12T02:08:45.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.409+0000 7fb8fbba9740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5620bfe4bb10 con 0x5620bff3f480 2024-09-12T02:08:45.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.409+0000 7fb8fbba9740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5620bfea0240 con 0x5620bff49d40 2024-09-12T02:08:45.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.409+0000 7fb8f27fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620bff3eb60 0x5620bff3ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:45.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.409+0000 7fb8f27fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620bff3eb60 0x5620bff3ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48620/0 (socket says 172.21.15.16:48620) 2024-09-12T02:08:45.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.409+0000 7fb8f27fc640 1 -- 172.21.15.16:0/2912714665 learned_addr learned my addr 172.21.15.16:0/2912714665 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:08:45.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.409+0000 7fb8dbfff640 1 --2- 172.21.15.16:0/1827918201 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fb8dc0a2b80 0x7fb8dc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:45.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.409+0000 7fb8f2ffd640 1 --2- 172.21.15.16:0/2912714665 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620bff3f480 0x5620bff49800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:45.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.409+0000 7fb8f37fe640 1 --2- 172.21.15.16:0/1827918201 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fb8dc0a2b80 0x7fb8dc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:45.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.410+0000 7fb8f27fc640 1 -- 172.21.15.16:0/2912714665 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5620bff49d40 msgr2=0x5620bff4c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:45.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.410+0000 7fb8f27fc640 1 --2- 172.21.15.16:0/2912714665 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5620bff49d40 0x5620bff4c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:45.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.410+0000 7fb8f27fc640 1 -- 172.21.15.16:0/2912714665 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620bff3f480 msgr2=0x5620bff49800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:45.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.410+0000 7fb8f27fc640 1 --2- 172.21.15.16:0/2912714665 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620bff3f480 0x5620bff49800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:45.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.410+0000 7fb8f27fc640 1 -- 172.21.15.16:0/2912714665 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5620bfe581f0 con 0x5620bff3eb60 2024-09-12T02:08:45.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.410+0000 7fb8dbfff640 1 -- 172.21.15.16:0/1827918201 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fb8ec0cf1a0 con 0x7fb8f41082a0 2024-09-12T02:08:45.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.410+0000 7fb8f27fc640 1 --2- 172.21.15.16:0/2912714665 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620bff3eb60 0x5620bff3ef40 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7fb8e80058a0 tx=0x7fb8e80127c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dea417c0539c22e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:45.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.410+0000 7fb8f37fe640 1 --2- 172.21.15.16:0/1827918201 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fb8dc0a2b80 0x7fb8dc0a5040 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fb8ec00efd0 tx=0x7fb8ec0c6590 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:45.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.410+0000 7fb8d97fa640 1 -- 172.21.15.16:0/2912714665 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8e80120d0 con 0x5620bff3eb60 2024-09-12T02:08:45.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.410+0000 7fb8d97fa640 1 -- 172.21.15.16:0/2912714665 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8e8016070 con 0x5620bff3eb60 2024-09-12T02:08:45.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.410+0000 7fb8d97fa640 1 -- 172.21.15.16:0/2912714665 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8e8004a00 con 0x5620bff3eb60 2024-09-12T02:08:45.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.411+0000 7fb8fbba9740 1 -- 172.21.15.16:0/2912714665 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620bff3eb60 msgr2=0x5620bff3ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:45.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.411+0000 7fb8fbba9740 1 --2- 172.21.15.16:0/2912714665 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620bff3eb60 0x5620bff3ef40 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7fb8e80058a0 tx=0x7fb8e80127c0 comp rx=0 tx=0).stop 2024-09-12T02:08:45.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.411+0000 7fb8fbba9740 1 -- 172.21.15.16:0/2912714665 shutdown_connections 2024-09-12T02:08:45.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.411+0000 7fb8fbba9740 1 --2- 172.21.15.16:0/2912714665 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5620bff49d40 0x5620bff4c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:45.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.411+0000 7fb8fbba9740 1 --2- 172.21.15.16:0/2912714665 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620bff3f480 0x5620bff49800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:45.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.411+0000 7fb8fbba9740 1 --2- 172.21.15.16:0/2912714665 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620bff3eb60 0x5620bff3ef40 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:45.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.411+0000 7fb8fbba9740 1 -- 172.21.15.16:0/2912714665 >> 172.21.15.16:0/2912714665 conn(0x5620bff3b0f0 msgr2=0x5620bff3b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:45.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.411+0000 7fb8fbba9740 1 -- 172.21.15.16:0/2912714665 shutdown_connections 2024-09-12T02:08:45.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.411+0000 7fb8fbba9740 1 -- 172.21.15.16:0/2912714665 wait complete. 2024-09-12T02:08:45.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.411+0000 7fb8fbba9740 1 -- start start 2024-09-12T02:08:45.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.411+0000 7fb8fbba9740 1 Processor -- start 2024-09-12T02:08:45.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.412+0000 7fb8fbba9740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5620bff3eb60 0x5620bff46ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:45.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.412+0000 7fb8fbba9740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620bff3f480 0x5620bff470e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:45.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.412+0000 7fb8fbba9740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620bff49d40 0x5620bff47620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:45.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.412+0000 7fb8fbba9740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5620bff4c910 con 0x5620bff3f480 2024-09-12T02:08:45.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.412+0000 7fb8fbba9740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5620bfe5ef10 con 0x5620bff49d40 2024-09-12T02:08:45.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.412+0000 7fb8fbba9740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5620bff4ca90 con 0x5620bff3eb60 2024-09-12T02:08:45.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.412+0000 7fb8fbba9740 10 client.0 _ll_register_callbacks cb 0x5620bfe27a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:08:45.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.412+0000 7fb8f2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620bff3f480 0x5620bff470e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:45.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.412+0000 7fb8f2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620bff3f480 0x5620bff470e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48632/0 (socket says 172.21.15.16:48632) 2024-09-12T02:08:45.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.412+0000 7fb8f2ffd640 1 -- 172.21.15.16:0/3383239031 learned_addr learned my addr 172.21.15.16:0/3383239031 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:08:45.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.412+0000 7fb8f27fc640 1 --2- 172.21.15.16:0/3383239031 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620bff49d40 0x5620bff47620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:45.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.412+0000 7fb8f27fc640 1 --2- 172.21.15.16:0/3383239031 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5620bff3eb60 0x5620bff46ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:45.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.412+0000 7fb8f2ffd640 1 -- 172.21.15.16:0/3383239031 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5620bff3eb60 msgr2=0x5620bff46ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:45.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.412+0000 7fb8f2ffd640 1 --2- 172.21.15.16:0/3383239031 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5620bff3eb60 0x5620bff46ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:45.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.412+0000 7fb8f2ffd640 1 -- 172.21.15.16:0/3383239031 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620bff49d40 msgr2=0x5620bff47620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:45.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.412+0000 7fb8f2ffd640 1 --2- 172.21.15.16:0/3383239031 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620bff49d40 0x5620bff47620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:45.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.412+0000 7fb8f2ffd640 1 -- 172.21.15.16:0/3383239031 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5620bfc156a0 con 0x5620bff3f480 2024-09-12T02:08:45.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.412+0000 7fb8f27fc640 1 --2- 172.21.15.16:0/3383239031 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620bff49d40 0x5620bff47620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:08:45.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.412+0000 7fb8f2ffd640 1 --2- 172.21.15.16:0/3383239031 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620bff3f480 0x5620bff470e0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7fb8e402a940 tx=0x7fb8e4031850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcf10bfb76d1f832 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:45.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.412+0000 7fb8d6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:08:45.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.412+0000 7fb8d6ff5640 1 -- 172.21.15.16:0/3383239031 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8e4035040 con 0x5620bff3f480 2024-09-12T02:08:45.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.412+0000 7fb8d6ff5640 1 -- 172.21.15.16:0/3383239031 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fb8e4004240 con 0x5620bff3f480 2024-09-12T02:08:45.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.412+0000 7fb8fbba9740 1 -- 172.21.15.16:0/3383239031 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8e8016070 con 0x5620bff3f480 2024-09-12T02:08:45.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.413+0000 7fb8d6ff5640 1 -- 172.21.15.16:0/3383239031 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fb8e4004440 con 0x5620bff3f480 2024-09-12T02:08:45.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.413+0000 7fb8fbba9740 10 client.17493 fetch_fsmap learned FSMap version 25 2024-09-12T02:08:45.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.413+0000 7fb8fbba9740 1 -- 172.21.15.16:0/3383239031 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5620bff584a0 con 0x5620bff3f480 2024-09-12T02:08:45.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.413+0000 7fb8d6ff5640 1 -- 172.21.15.16:0/3383239031 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8e8016070 con 0x5620bff3f480 2024-09-12T02:08:45.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.413+0000 7fb8d6ff5640 1 -- 172.21.15.16:0/3383239031 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fb8e403c6e0 con 0x5620bff3f480 2024-09-12T02:08:45.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.413+0000 7fb8fbba9740 10 client.17493 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:08:45.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.413+0000 7fb8fbba9740 10 client.17493 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:08:45.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.413+0000 7fb8fbba9740 20 client.17493 populate_metadata read hostname 'smithi016' 2024-09-12T02:08:45.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.413+0000 7fb8fbba9740 1 --2- 172.21.15.16:0/3383239031 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5620bff59900 0x5620bff5bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:45.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.413+0000 7fb8fbba9740 4 client.17493 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:08:45.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.413+0000 7fb8fbba9740 1 -- 172.21.15.16:0/3383239031 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5620bfeb17f0 con 0x5620bff59900 2024-09-12T02:08:45.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.413+0000 7fb8f27fc640 1 --2- 172.21.15.16:0/3383239031 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5620bff59900 0x5620bff5bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:45.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.414+0000 7fb8f27fc640 1 --2- 172.21.15.16:0/3383239031 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5620bff59900 0x5620bff5bd40 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=162b46f48dd1f891 server_cookie=f9a382e5d5435ab0 in_seq=0 out_seq=0 2024-09-12T02:08:45.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.414+0000 7fb8d6ff5640 10 client.17493 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:08:45.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.414+0000 7fb8d6ff5640 1 -- 172.21.15.16:0/3383239031 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5620bfeb17f0 con 0x5620bff59900 2024-09-12T02:08:45.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.414+0000 7fb8d6ff5640 10 client.17493 handle_command_reply: tid=0 2024-09-12T02:08:45.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.414+0000 7fb8fbba9740 1 client.17493 shutdown 2024-09-12T02:08:45.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.414+0000 7fb8fbba9740 1 -- 172.21.15.16:0/3383239031 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620bff3f480 msgr2=0x5620bff470e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:45.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.414+0000 7fb8fbba9740 1 --2- 172.21.15.16:0/3383239031 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620bff3f480 0x5620bff470e0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7fb8e402a940 tx=0x7fb8e4031850 comp rx=0 tx=0).stop 2024-09-12T02:08:45.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.414+0000 7fb8fbba9740 1 -- 172.21.15.16:0/3383239031 shutdown_connections 2024-09-12T02:08:45.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.414+0000 7fb8fbba9740 1 --2- 172.21.15.16:0/3383239031 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5620bff59900 0x5620bff5bd40 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:45.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.414+0000 7fb8fbba9740 1 --2- 172.21.15.16:0/3383239031 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620bff49d40 0x5620bff47620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:45.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.414+0000 7fb8fbba9740 1 --2- 172.21.15.16:0/3383239031 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620bff3f480 0x5620bff470e0 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:45.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.414+0000 7fb8fbba9740 1 --2- 172.21.15.16:0/3383239031 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5620bff3eb60 0x5620bff46ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:45.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.414+0000 7fb8fbba9740 1 -- 172.21.15.16:0/3383239031 >> 172.21.15.16:0/3383239031 conn(0x5620bff3b0f0 msgr2=0x5620bff40e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:45.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.414+0000 7fb8d6ff5640 0 client.17493 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:08:45.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.415+0000 7fb8fbba9740 1 -- 172.21.15.16:0/3383239031 shutdown_connections 2024-09-12T02:08:45.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.415+0000 7fb8fbba9740 1 -- 172.21.15.16:0/3383239031 wait complete. 2024-09-12T02:08:45.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.415+0000 7fb8fbba9740 20 client.17493 trim_cache size 0 max 16384 2024-09-12T02:08:45.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.432+0000 7fb8fbba9740 1 Processor -- start 2024-09-12T02:08:45.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.432+0000 7fb8fbba9740 1 -- start start 2024-09-12T02:08:45.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.432+0000 7fb8fbba9740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620bff3eb60 0x5620bff47470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:45.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.432+0000 7fb8fbba9740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5620bff3f480 0x5620bff45a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:45.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.432+0000 7fb8fbba9740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620bff49d40 0x5620bff61610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:45.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.432+0000 7fb8fbba9740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb8f41b1f70 con 0x5620bff3eb60 2024-09-12T02:08:45.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.432+0000 7fb8fbba9740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb8f41af840 con 0x5620bff49d40 2024-09-12T02:08:45.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.432+0000 7fb8fbba9740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb8f41ad690 con 0x5620bff3f480 2024-09-12T02:08:45.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.432+0000 7fb8f27fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620bff3eb60 0x5620bff47470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:45.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.432+0000 7fb8f27fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620bff3eb60 0x5620bff47470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48634/0 (socket says 172.21.15.16:48634) 2024-09-12T02:08:45.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.432+0000 7fb8f27fc640 1 -- 172.21.15.16:0/2556482079 learned_addr learned my addr 172.21.15.16:0/2556482079 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:08:45.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.433+0000 7fb8f2ffd640 1 --2- 172.21.15.16:0/2556482079 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5620bff3f480 0x5620bff45a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:45.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.433+0000 7fb8f27fc640 1 -- 172.21.15.16:0/2556482079 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5620bff3f480 msgr2=0x5620bff45a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:45.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.433+0000 7fb8f27fc640 1 --2- 172.21.15.16:0/2556482079 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5620bff3f480 0x5620bff45a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:45.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.433+0000 7fb8f27fc640 1 -- 172.21.15.16:0/2556482079 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620bff49d40 msgr2=0x5620bff61610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:45.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.433+0000 7fb8f27fc640 1 --2- 172.21.15.16:0/2556482079 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620bff49d40 0x5620bff61610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:45.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.433+0000 7fb8f27fc640 1 -- 172.21.15.16:0/2556482079 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5620bff49470 con 0x5620bff3eb60 2024-09-12T02:08:45.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.433+0000 7fb8f27fc640 1 --2- 172.21.15.16:0/2556482079 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620bff3eb60 0x5620bff47470 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7fb8e80052f0 tx=0x7fb8e8005540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6d4b8d21414a19d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:45.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.433+0000 7fb8d8ff9640 1 -- 172.21.15.16:0/2556482079 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8e80120b0 con 0x5620bff3eb60 2024-09-12T02:08:45.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.433+0000 7fb8d8ff9640 1 -- 172.21.15.16:0/2556482079 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8e8016070 con 0x5620bff3eb60 2024-09-12T02:08:45.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.433+0000 7fb8d8ff9640 1 -- 172.21.15.16:0/2556482079 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8e8010790 con 0x5620bff3eb60 2024-09-12T02:08:45.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.434+0000 7fb8fbba9740 1 -- 172.21.15.16:0/2556482079 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620bff3eb60 msgr2=0x5620bff47470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:45.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.434+0000 7fb8fbba9740 1 --2- 172.21.15.16:0/2556482079 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620bff3eb60 0x5620bff47470 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7fb8e80052f0 tx=0x7fb8e8005540 comp rx=0 tx=0).stop 2024-09-12T02:08:45.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.434+0000 7fb8fbba9740 1 -- 172.21.15.16:0/2556482079 shutdown_connections 2024-09-12T02:08:45.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.434+0000 7fb8fbba9740 1 --2- 172.21.15.16:0/2556482079 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620bff49d40 0x5620bff61610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:45.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.434+0000 7fb8fbba9740 1 --2- 172.21.15.16:0/2556482079 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5620bff3f480 0x5620bff45a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:45.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.434+0000 7fb8fbba9740 1 --2- 172.21.15.16:0/2556482079 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620bff3eb60 0x5620bff47470 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:45.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.434+0000 7fb8fbba9740 1 -- 172.21.15.16:0/2556482079 >> 172.21.15.16:0/2556482079 conn(0x5620bff3b0f0 msgr2=0x5620bff45360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:45.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.434+0000 7fb8fbba9740 1 -- 172.21.15.16:0/2556482079 shutdown_connections 2024-09-12T02:08:45.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.434+0000 7fb8fbba9740 1 -- 172.21.15.16:0/2556482079 wait complete. 2024-09-12T02:08:45.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.434+0000 7fb8fbba9740 1 -- start start 2024-09-12T02:08:45.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.434+0000 7fb8fbba9740 1 Processor -- start 2024-09-12T02:08:45.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.435+0000 7fb8fbba9740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620bff3eb60 0x5620bff43b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:45.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.435+0000 7fb8fbba9740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620bff3f480 0x5620bff440d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:45.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.435+0000 7fb8fbba9740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5620bff49d40 0x5620bff44610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:45.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.435+0000 7fb8fbba9740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb8f41add00 con 0x5620bff3eb60 2024-09-12T02:08:45.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.435+0000 7fb8fbba9740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb8f41af170 con 0x5620bff3f480 2024-09-12T02:08:45.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.435+0000 7fb8fbba9740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb8f41ae3d0 con 0x5620bff49d40 2024-09-12T02:08:45.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.435+0000 7fb8fbba9740 10 client.0 _ll_register_callbacks cb 0x7fb8e4005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:08:45.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.435+0000 7fb8f27fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620bff3eb60 0x5620bff43b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:45.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.435+0000 7fb8f27fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620bff3eb60 0x5620bff43b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48638/0 (socket says 172.21.15.16:48638) 2024-09-12T02:08:45.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.435+0000 7fb8f27fc640 1 -- 172.21.15.16:0/880788021 learned_addr learned my addr 172.21.15.16:0/880788021 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:08:45.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.435+0000 7fb8f2ffd640 1 --2- 172.21.15.16:0/880788021 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620bff3f480 0x5620bff440d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:45.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.435+0000 7fb8f27fc640 1 --2- 172.21.15.16:0/880788021 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5620bff49d40 0x5620bff44610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:45.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.435+0000 7fb8f27fc640 1 -- 172.21.15.16:0/880788021 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5620bff49d40 msgr2=0x5620bff44610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:45.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.435+0000 7fb8f27fc640 1 --2- 172.21.15.16:0/880788021 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5620bff49d40 0x5620bff44610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:45.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.435+0000 7fb8f27fc640 1 -- 172.21.15.16:0/880788021 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620bff3f480 msgr2=0x5620bff440d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:45.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.435+0000 7fb8f27fc640 1 --2- 172.21.15.16:0/880788021 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620bff3f480 0x5620bff440d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:45.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.435+0000 7fb8f27fc640 1 -- 172.21.15.16:0/880788021 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5620bffc35e0 con 0x5620bff3eb60 2024-09-12T02:08:45.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.435+0000 7fb8f27fc640 1 --2- 172.21.15.16:0/880788021 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620bff3eb60 0x5620bff43b90 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7fb8e800b4e0 tx=0x7fb8e80054a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=952fd04a976ab99b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:45.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.435+0000 7fb8d6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:08:45.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.435+0000 7fb8d6ff5640 1 -- 172.21.15.16:0/880788021 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8e80121d0 con 0x5620bff3eb60 2024-09-12T02:08:45.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.435+0000 7fb8d6ff5640 1 -- 172.21.15.16:0/880788021 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fb8e8003a00 con 0x5620bff3eb60 2024-09-12T02:08:45.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.435+0000 7fb8fbba9740 1 -- 172.21.15.16:0/880788021 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8e8016070 con 0x5620bff3eb60 2024-09-12T02:08:45.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.436+0000 7fb8d6ff5640 1 -- 172.21.15.16:0/880788021 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fb8e8026360 con 0x5620bff3eb60 2024-09-12T02:08:45.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.436+0000 7fb8fbba9740 10 client.17499 fetch_fsmap learned FSMap version 25 2024-09-12T02:08:45.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.436+0000 7fb8fbba9740 1 -- 172.21.15.16:0/880788021 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5620bffc4880 con 0x5620bff3eb60 2024-09-12T02:08:45.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.436+0000 7fb8d6ff5640 1 -- 172.21.15.16:0/880788021 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8e8016070 con 0x5620bff3eb60 2024-09-12T02:08:45.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.436+0000 7fb8d6ff5640 1 -- 172.21.15.16:0/880788021 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fb8e8036880 con 0x5620bff3eb60 2024-09-12T02:08:45.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.436+0000 7fb8fbba9740 10 client.17499 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:08:45.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.436+0000 7fb8fbba9740 10 client.17499 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:08:45.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.436+0000 7fb8fbba9740 20 client.17499 populate_metadata read hostname 'smithi016' 2024-09-12T02:08:45.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.436+0000 7fb8fbba9740 1 --2- 172.21.15.16:0/880788021 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5620bffc4ca0 0x5620bffc7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:45.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.436+0000 7fb8fbba9740 4 client.17499 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-12T02:08:45.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.436+0000 7fb8fbba9740 1 -- 172.21.15.16:0/880788021 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5620bfeabd00 con 0x5620bffc4ca0 2024-09-12T02:08:45.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.436+0000 7fb8f2ffd640 1 --2- 172.21.15.16:0/880788021 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5620bffc4ca0 0x5620bffc7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:45.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.437+0000 7fb8f2ffd640 1 --2- 172.21.15.16:0/880788021 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5620bffc4ca0 0x5620bffc7090 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=43e4cb78344284ed server_cookie=5a898eaf0fc82f0d in_seq=0 out_seq=0 2024-09-12T02:08:45.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.437+0000 7fb8d6ff5640 10 client.17499 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:08:45.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.438+0000 7fb8d6ff5640 1 -- 172.21.15.16:0/880788021 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5620bfeabd00 con 0x5620bffc4ca0 2024-09-12T02:08:45.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.438+0000 7fb8d6ff5640 10 client.17499 handle_command_reply: tid=0 2024-09-12T02:08:45.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.438+0000 7fb8fbba9740 1 client.17499 shutdown 2024-09-12T02:08:45.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.438+0000 7fb8fbba9740 1 -- 172.21.15.16:0/880788021 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620bff3eb60 msgr2=0x5620bff43b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:45.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.438+0000 7fb8fbba9740 1 --2- 172.21.15.16:0/880788021 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620bff3eb60 0x5620bff43b90 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7fb8e800b4e0 tx=0x7fb8e80054a0 comp rx=0 tx=0).stop 2024-09-12T02:08:45.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.438+0000 7fb8fbba9740 1 -- 172.21.15.16:0/880788021 shutdown_connections 2024-09-12T02:08:45.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.438+0000 7fb8fbba9740 1 --2- 172.21.15.16:0/880788021 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5620bffc4ca0 0x5620bffc7090 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:45.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.438+0000 7fb8d6ff5640 0 client.17499 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:08:45.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.438+0000 7fb8fbba9740 1 --2- 172.21.15.16:0/880788021 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5620bff49d40 0x5620bff44610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:45.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.438+0000 7fb8fbba9740 1 --2- 172.21.15.16:0/880788021 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620bff3f480 0x5620bff440d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:45.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.438+0000 7fb8fbba9740 1 --2- 172.21.15.16:0/880788021 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620bff3eb60 0x5620bff43b90 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:45.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.438+0000 7fb8fbba9740 1 -- 172.21.15.16:0/880788021 >> 172.21.15.16:0/880788021 conn(0x5620bff3b0f0 msgr2=0x5620bff3a270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:45.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.438+0000 7fb8fbba9740 1 -- 172.21.15.16:0/880788021 shutdown_connections 2024-09-12T02:08:45.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.438+0000 7fb8fbba9740 1 -- 172.21.15.16:0/880788021 wait complete. 2024-09-12T02:08:45.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.438+0000 7fb8fbba9740 20 client.17499 trim_cache size 0 max 16384 2024-09-12T02:08:45.443 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T02:08:45.444 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-09-12T02:08:45.444 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "97e3e6bf-29a1-403b-bcfc-3e2898916b27", 2024-09-12T02:08:45.444 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-09-12T02:08:45.444 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T02:08:45.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.439+0000 7fb8f97d0640 1 -- 172.21.15.16:0/1827918201 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fb8dc0a2b80 msgr2=0x7fb8dc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:45.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.439+0000 7fb8f97d0640 1 --2- 172.21.15.16:0/1827918201 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fb8dc0a2b80 0x7fb8dc0a5040 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fb8ec00efd0 tx=0x7fb8ec0c6590 comp rx=0 tx=0).stop 2024-09-12T02:08:45.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.439+0000 7fb8f97d0640 1 -- 172.21.15.16:0/1827918201 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8f41082a0 msgr2=0x7fb8f4101410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:45.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.439+0000 7fb8f97d0640 1 --2- 172.21.15.16:0/1827918201 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8f41082a0 0x7fb8f4101410 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7fb8ec00eaa0 tx=0x7fb8ec00ef60 comp rx=0 tx=0).stop 2024-09-12T02:08:45.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.439+0000 7fb8f97d0640 1 -- 172.21.15.16:0/1827918201 shutdown_connections 2024-09-12T02:08:45.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.439+0000 7fb8f97d0640 1 --2- 172.21.15.16:0/1827918201 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fb8dc0a2b80 0x7fb8dc0a5040 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:45.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.439+0000 7fb8f97d0640 1 --2- 172.21.15.16:0/1827918201 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8f41082a0 0x7fb8f4101410 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:45.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.439+0000 7fb8f97d0640 1 --2- 172.21.15.16:0/1827918201 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb8f41036e0 0x7fb8f4100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:45.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.439+0000 7fb8f97d0640 1 --2- 172.21.15.16:0/1827918201 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb8f4102d30 0x7fb8f4100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:45.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.439+0000 7fb8f97d0640 1 -- 172.21.15.16:0/1827918201 >> 172.21.15.16:0/1827918201 conn(0x7fb8f40fc7d0 msgr2=0x7fb8f4105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:45.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.439+0000 7fb8f97d0640 1 -- 172.21.15.16:0/1827918201 shutdown_connections 2024-09-12T02:08:45.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:45.439+0000 7fb8f97d0640 1 -- 172.21.15.16:0/1827918201 wait complete. 2024-09-12T02:08:46.166 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T02:08:46.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:08:46 smithi016 ceph-mon[30947]: pgmap v540: 97 pgs: 97 active+clean; 3.7 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.7 KiB/s wr, 11 op/s 2024-09-12T02:08:46.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:08:46 smithi016 ceph-mon[30947]: scrub queued for path: / 2024-09-12T02:08:46.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:08:46 smithi016 ceph-mon[30947]: scrub summary: idle+waiting paths [/] 2024-09-12T02:08:46.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:08:46 smithi016 ceph-mon[30947]: scrub summary: active paths [/] 2024-09-12T02:08:46.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:08:46 smithi016 ceph-mon[30947]: scrub summary: idle+waiting paths [/] 2024-09-12T02:08:46.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:08:46 smithi016 ceph-mon[30947]: scrub summary: idle 2024-09-12T02:08:46.471 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:08:46.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:08:46 smithi184 ceph-mon[35593]: pgmap v540: 97 pgs: 97 active+clean; 3.7 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.7 KiB/s wr, 11 op/s 2024-09-12T02:08:46.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:08:46 smithi184 ceph-mon[35593]: scrub queued for path: / 2024-09-12T02:08:46.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:08:46 smithi184 ceph-mon[35593]: scrub summary: idle+waiting paths [/] 2024-09-12T02:08:46.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:08:46 smithi184 ceph-mon[35593]: scrub summary: active paths [/] 2024-09-12T02:08:46.628 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:08:46 smithi184 ceph-mon[35593]: scrub summary: idle+waiting paths [/] 2024-09-12T02:08:46.628 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:08:46 smithi184 ceph-mon[35593]: scrub summary: idle 2024-09-12T02:08:46.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:08:46 smithi033 ceph-mon[35332]: pgmap v540: 97 pgs: 97 active+clean; 3.7 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.7 KiB/s wr, 11 op/s 2024-09-12T02:08:46.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:08:46 smithi033 ceph-mon[35332]: scrub queued for path: / 2024-09-12T02:08:46.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:08:46 smithi033 ceph-mon[35332]: scrub summary: idle+waiting paths [/] 2024-09-12T02:08:46.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:08:46 smithi033 ceph-mon[35332]: scrub summary: active paths [/] 2024-09-12T02:08:46.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:08:46 smithi033 ceph-mon[35332]: scrub summary: idle+waiting paths [/] 2024-09-12T02:08:46.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:08:46 smithi033 ceph-mon[35332]: scrub summary: idle 2024-09-12T02:08:46.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.930+0000 7ff6a7903640 1 -- 172.21.15.16:0/2459582066 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6a0102570 msgr2=0x7ff6a01029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:46.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.930+0000 7ff6a7903640 1 --2- 172.21.15.16:0/2459582066 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6a0102570 0x7ff6a01029b0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7ff690009900 tx=0x7ff69002a480 comp rx=0 tx=0).stop 2024-09-12T02:08:46.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.930+0000 7ff6a7903640 1 -- 172.21.15.16:0/2459582066 shutdown_connections 2024-09-12T02:08:46.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.930+0000 7ff6a7903640 1 --2- 172.21.15.16:0/2459582066 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff6a0102ef0 0x7ff6a010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:46.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.930+0000 7ff6a7903640 1 --2- 172.21.15.16:0/2459582066 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6a0102570 0x7ff6a01029b0 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:46.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.930+0000 7ff6a7903640 1 --2- 172.21.15.16:0/2459582066 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6a0100e20 0x7ff6a0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:46.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.930+0000 7ff6a7903640 1 -- 172.21.15.16:0/2459582066 >> 172.21.15.16:0/2459582066 conn(0x7ff6a00fc7d0 msgr2=0x7ff6a00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:46.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.930+0000 7ff6a7903640 1 -- 172.21.15.16:0/2459582066 shutdown_connections 2024-09-12T02:08:46.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.930+0000 7ff6a7903640 1 -- 172.21.15.16:0/2459582066 wait complete. 2024-09-12T02:08:46.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.931+0000 7ff6a7903640 1 Processor -- start 2024-09-12T02:08:46.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.931+0000 7ff6a7903640 1 -- start start 2024-09-12T02:08:46.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.931+0000 7ff6a7903640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6a0100e20 0x7ff6a01984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:46.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.931+0000 7ff6a7903640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff6a0102570 0x7ff6a0198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:46.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.931+0000 7ff6a7903640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6a0102ef0 0x7ff6a019cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:46.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.931+0000 7ff6a7903640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff6a0110f80 con 0x7ff6a0100e20 2024-09-12T02:08:46.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.931+0000 7ff6a7903640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff6a0110e00 con 0x7ff6a0102ef0 2024-09-12T02:08:46.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.931+0000 7ff6a7903640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff6a0111100 con 0x7ff6a0102570 2024-09-12T02:08:46.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.931+0000 7ff6a569f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6a0100e20 0x7ff6a01984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:46.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.931+0000 7ff6a569f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6a0100e20 0x7ff6a01984e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48652/0 (socket says 172.21.15.16:48652) 2024-09-12T02:08:46.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.931+0000 7ff6a569f640 1 -- 172.21.15.16:0/382766454 learned_addr learned my addr 172.21.15.16:0/382766454 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:08:46.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.931+0000 7ff6a5ea0640 1 --2- 172.21.15.16:0/382766454 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6a0102ef0 0x7ff6a019cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:46.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.932+0000 7ff6a4e9e640 1 --2- 172.21.15.16:0/382766454 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff6a0102570 0x7ff6a0198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:46.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.932+0000 7ff6a569f640 1 -- 172.21.15.16:0/382766454 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff6a0102570 msgr2=0x7ff6a0198a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:46.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.932+0000 7ff6a569f640 1 --2- 172.21.15.16:0/382766454 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff6a0102570 0x7ff6a0198a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:46.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.932+0000 7ff6a569f640 1 -- 172.21.15.16:0/382766454 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6a0102ef0 msgr2=0x7ff6a019cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:46.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.932+0000 7ff6a569f640 1 --2- 172.21.15.16:0/382766454 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6a0102ef0 0x7ff6a019cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:46.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.932+0000 7ff6a569f640 1 -- 172.21.15.16:0/382766454 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6a019d490 con 0x7ff6a0100e20 2024-09-12T02:08:46.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.932+0000 7ff6a569f640 1 --2- 172.21.15.16:0/382766454 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6a0100e20 0x7ff6a01984e0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7ff69c00d950 tx=0x7ff69c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:46.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.932+0000 7ff6967fc640 1 -- 172.21.15.16:0/382766454 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff69c017070 con 0x7ff6a0100e20 2024-09-12T02:08:46.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.932+0000 7ff6967fc640 1 -- 172.21.15.16:0/382766454 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff69c009c30 con 0x7ff6a0100e20 2024-09-12T02:08:46.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.932+0000 7ff6967fc640 1 -- 172.21.15.16:0/382766454 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff69c00f6b0 con 0x7ff6a0100e20 2024-09-12T02:08:46.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.932+0000 7ff6a7903640 1 -- 172.21.15.16:0/382766454 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6a019d780 con 0x7ff6a0100e20 2024-09-12T02:08:46.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.932+0000 7ff6a7903640 1 -- 172.21.15.16:0/382766454 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6a00fcbb0 con 0x7ff6a0100e20 2024-09-12T02:08:46.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.935+0000 7ff6967fc640 1 -- 172.21.15.16:0/382766454 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff69c0109d0 con 0x7ff6a0100e20 2024-09-12T02:08:46.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.935+0000 7ff6a9cdc740 1 Processor -- start 2024-09-12T02:08:46.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.935+0000 7ff6a9cdc740 1 -- start start 2024-09-12T02:08:46.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.935+0000 7ff6a9cdc740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563666eb64a0 0x563666eb6880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:46.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.935+0000 7ff6a9cdc740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563666eb6dc0 0x563666ec1140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:46.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.935+0000 7ff6a9cdc740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563666ec1680 0x563666ec3a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:46.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.935+0000 7ff6a9cdc740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x563666e1c2e0 con 0x563666eb6dc0 2024-09-12T02:08:46.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.935+0000 7ff6a9cdc740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x563666dc3ce0 con 0x563666ec1680 2024-09-12T02:08:46.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.935+0000 7ff6a9cdc740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x563666e182a0 con 0x563666eb64a0 2024-09-12T02:08:46.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.935+0000 7ff6a569f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563666eb6dc0 0x563666ec1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:46.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.935+0000 7ff6967fc640 1 --2- 172.21.15.16:0/382766454 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7ff68c0a2c00 0x7ff68c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:46.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.935+0000 7ff6a569f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563666eb6dc0 0x563666ec1140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48658/0 (socket says 172.21.15.16:48658) 2024-09-12T02:08:46.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.935+0000 7ff6a569f640 1 -- 172.21.15.16:0/3825359887 learned_addr learned my addr 172.21.15.16:0/3825359887 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:08:46.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.936+0000 7ff6a4e9e640 1 --2- 172.21.15.16:0/3825359887 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563666ec1680 0x563666ec3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:46.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.936+0000 7ff6a5ea0640 1 --2- 172.21.15.16:0/382766454 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7ff68c0a2c00 0x7ff68c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:46.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.936+0000 7ff6a5ea0640 1 --2- 172.21.15.16:0/3825359887 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563666eb64a0 0x563666eb6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:46.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.936+0000 7ff6a569f640 1 -- 172.21.15.16:0/3825359887 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563666eb64a0 msgr2=0x563666eb6880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:46.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.936+0000 7ff6967fc640 1 -- 172.21.15.16:0/382766454 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7ff69c0ceb90 con 0x7ff6a0100e20 2024-09-12T02:08:46.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.936+0000 7ff6a569f640 1 --2- 172.21.15.16:0/3825359887 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563666eb64a0 0x563666eb6880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:46.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.936+0000 7ff6a569f640 1 -- 172.21.15.16:0/3825359887 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563666ec1680 msgr2=0x563666ec3a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:46.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.936+0000 7ff6a569f640 1 --2- 172.21.15.16:0/3825359887 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563666ec1680 0x563666ec3a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:46.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.936+0000 7ff6a569f640 1 -- 172.21.15.16:0/3825359887 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563666da8440 con 0x563666eb6dc0 2024-09-12T02:08:46.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.936+0000 7ff6a5ea0640 1 --2- 172.21.15.16:0/3825359887 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563666eb64a0 0x563666eb6880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:08:46.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.936+0000 7ff6a4e9e640 1 --2- 172.21.15.16:0/3825359887 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563666ec1680 0x563666ec3a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:08:46.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.936+0000 7ff6a569f640 1 --2- 172.21.15.16:0/3825359887 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563666eb6dc0 0x563666ec1140 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7ff69c0cc670 tx=0x7ff69c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1109d957f7112e91 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:46.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.936+0000 7ff6a5ea0640 1 --2- 172.21.15.16:0/382766454 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7ff68c0a2c00 0x7ff68c0a50c0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7ff69800a780 tx=0x7ff698005f60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:46.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.936+0000 7ff68b7fe640 1 -- 172.21.15.16:0/3825359887 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff69c09bb90 con 0x563666eb6dc0 2024-09-12T02:08:46.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.936+0000 7ff68b7fe640 1 -- 172.21.15.16:0/3825359887 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff69c009de0 con 0x563666eb6dc0 2024-09-12T02:08:46.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.936+0000 7ff68b7fe640 1 -- 172.21.15.16:0/3825359887 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff69c0591b0 con 0x563666eb6dc0 2024-09-12T02:08:46.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.937+0000 7ff6a9cdc740 1 -- 172.21.15.16:0/3825359887 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563666eb6dc0 msgr2=0x563666ec1140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:46.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.937+0000 7ff6a9cdc740 1 --2- 172.21.15.16:0/3825359887 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563666eb6dc0 0x563666ec1140 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7ff69c0cc670 tx=0x7ff69c005880 comp rx=0 tx=0).stop 2024-09-12T02:08:46.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.937+0000 7ff6a9cdc740 1 -- 172.21.15.16:0/3825359887 shutdown_connections 2024-09-12T02:08:46.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.937+0000 7ff6a9cdc740 1 --2- 172.21.15.16:0/3825359887 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563666ec1680 0x563666ec3a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:46.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.937+0000 7ff6a9cdc740 1 --2- 172.21.15.16:0/3825359887 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563666eb6dc0 0x563666ec1140 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:46.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.937+0000 7ff6a9cdc740 1 --2- 172.21.15.16:0/3825359887 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563666eb64a0 0x563666eb6880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:46.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.937+0000 7ff6a9cdc740 1 -- 172.21.15.16:0/3825359887 >> 172.21.15.16:0/3825359887 conn(0x563666eb2a30 msgr2=0x563666eb2e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:46.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.937+0000 7ff6a9cdc740 1 -- 172.21.15.16:0/3825359887 shutdown_connections 2024-09-12T02:08:46.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.937+0000 7ff6a9cdc740 1 -- 172.21.15.16:0/3825359887 wait complete. 2024-09-12T02:08:46.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.938+0000 7ff6a9cdc740 1 -- start start 2024-09-12T02:08:46.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.938+0000 7ff6a9cdc740 1 Processor -- start 2024-09-12T02:08:46.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.938+0000 7ff6a9cdc740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563666eb64a0 0x563666ec0f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:46.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.938+0000 7ff6a9cdc740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563666eb6dc0 0x563666ebe210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:46.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.938+0000 7ff6a9cdc740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563666ec1680 0x563666ebe750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:46.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.938+0000 7ff6a9cdc740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x563666ec41f0 con 0x563666eb64a0 2024-09-12T02:08:46.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.938+0000 7ff6a9cdc740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x563666dd7050 con 0x563666eb6dc0 2024-09-12T02:08:46.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.938+0000 7ff6a9cdc740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x563666ec4370 con 0x563666ec1680 2024-09-12T02:08:46.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.938+0000 7ff6a569f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563666eb64a0 0x563666ec0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:46.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.938+0000 7ff6a9cdc740 10 client.0 _ll_register_callbacks cb 0x563666e17160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:08:46.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.938+0000 7ff6a569f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563666eb64a0 0x563666ec0f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48674/0 (socket says 172.21.15.16:48674) 2024-09-12T02:08:46.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.938+0000 7ff6a569f640 1 -- 172.21.15.16:0/1945984431 learned_addr learned my addr 172.21.15.16:0/1945984431 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:08:46.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.939+0000 7ff6a4e9e640 1 --2- 172.21.15.16:0/1945984431 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563666eb6dc0 0x563666ebe210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:46.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.939+0000 7ff6a5ea0640 1 --2- 172.21.15.16:0/1945984431 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563666ec1680 0x563666ebe750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:46.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.939+0000 7ff6a569f640 1 -- 172.21.15.16:0/1945984431 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563666ec1680 msgr2=0x563666ebe750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:46.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.939+0000 7ff6a569f640 1 --2- 172.21.15.16:0/1945984431 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563666ec1680 0x563666ebe750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:46.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.939+0000 7ff6a569f640 1 -- 172.21.15.16:0/1945984431 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563666eb6dc0 msgr2=0x563666ebe210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:46.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.939+0000 7ff6a569f640 1 --2- 172.21.15.16:0/1945984431 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563666eb6dc0 0x563666ebe210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:46.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.939+0000 7ff6a569f640 1 -- 172.21.15.16:0/1945984431 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563666b8d3c0 con 0x563666eb64a0 2024-09-12T02:08:46.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.939+0000 7ff6a4e9e640 1 --2- 172.21.15.16:0/1945984431 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563666eb6dc0 0x563666ebe210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:08:46.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.939+0000 7ff6a5ea0640 1 --2- 172.21.15.16:0/1945984431 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563666ec1680 0x563666ebe750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:08:46.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.939+0000 7ff6a569f640 1 --2- 172.21.15.16:0/1945984431 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563666eb64a0 0x563666ec0f60 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7ff69c00bfa0 tx=0x7ff69c0a4190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3255d426789c844 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:46.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.939+0000 7ff688ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:08:46.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.939+0000 7ff688ff9640 1 -- 172.21.15.16:0/1945984431 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff69c020040 con 0x563666eb64a0 2024-09-12T02:08:46.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.939+0000 7ff688ff9640 1 -- 172.21.15.16:0/1945984431 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7ff69c0ccb50 con 0x563666eb64a0 2024-09-12T02:08:46.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.939+0000 7ff6a9cdc740 1 -- 172.21.15.16:0/1945984431 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff69c009de0 con 0x563666eb64a0 2024-09-12T02:08:46.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.939+0000 7ff688ff9640 1 -- 172.21.15.16:0/1945984431 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff69c0ccd70 con 0x563666eb64a0 2024-09-12T02:08:46.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.939+0000 7ff6a9cdc740 10 client.17511 fetch_fsmap learned FSMap version 25 2024-09-12T02:08:46.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.939+0000 7ff6a9cdc740 1 -- 172.21.15.16:0/1945984431 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x563666ec41f0 con 0x563666eb64a0 2024-09-12T02:08:46.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.940+0000 7ff688ff9640 1 -- 172.21.15.16:0/1945984431 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff69c009de0 con 0x563666eb64a0 2024-09-12T02:08:46.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.940+0000 7ff688ff9640 1 -- 172.21.15.16:0/1945984431 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7ff69c0b4200 con 0x563666eb64a0 2024-09-12T02:08:46.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.940+0000 7ff6a9cdc740 10 client.17511 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:08:46.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.940+0000 7ff6a9cdc740 10 client.17511 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:08:46.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.940+0000 7ff6a9cdc740 20 client.17511 populate_metadata read hostname 'smithi016' 2024-09-12T02:08:46.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.940+0000 7ff6a9cdc740 1 --2- 172.21.15.16:0/1945984431 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x563666ed0ce0 0x563666ed30d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:46.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.940+0000 7ff6a9cdc740 4 client.17511 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:08:46.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.940+0000 7ff6a9cdc740 1 -- 172.21.15.16:0/1945984431 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563666e29850 con 0x563666ed0ce0 2024-09-12T02:08:46.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.940+0000 7ff6a4e9e640 1 --2- 172.21.15.16:0/1945984431 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x563666ed0ce0 0x563666ed30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:46.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.941+0000 7ff6a4e9e640 1 --2- 172.21.15.16:0/1945984431 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x563666ed0ce0 0x563666ed30d0 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=819b5ba976d24a56 server_cookie=f68f5fa2a3659b9 in_seq=0 out_seq=0 2024-09-12T02:08:46.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.941+0000 7ff688ff9640 10 client.17511 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:08:46.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.941+0000 7ff688ff9640 1 -- 172.21.15.16:0/1945984431 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563666e29850 con 0x563666ed0ce0 2024-09-12T02:08:46.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.941+0000 7ff688ff9640 10 client.17511 handle_command_reply: tid=0 2024-09-12T02:08:46.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.941+0000 7ff6a9cdc740 1 client.17511 shutdown 2024-09-12T02:08:46.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.942+0000 7ff6a9cdc740 1 -- 172.21.15.16:0/1945984431 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563666eb64a0 msgr2=0x563666ec0f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:46.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.942+0000 7ff6a9cdc740 1 --2- 172.21.15.16:0/1945984431 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563666eb64a0 0x563666ec0f60 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7ff69c00bfa0 tx=0x7ff69c0a4190 comp rx=0 tx=0).stop 2024-09-12T02:08:46.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.942+0000 7ff6a9cdc740 1 -- 172.21.15.16:0/1945984431 shutdown_connections 2024-09-12T02:08:46.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.942+0000 7ff6a9cdc740 1 --2- 172.21.15.16:0/1945984431 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x563666ed0ce0 0x563666ed30d0 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:46.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.942+0000 7ff6a9cdc740 1 --2- 172.21.15.16:0/1945984431 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563666ec1680 0x563666ebe750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:46.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.942+0000 7ff6a9cdc740 1 --2- 172.21.15.16:0/1945984431 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563666eb6dc0 0x563666ebe210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:46.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.942+0000 7ff6a9cdc740 1 --2- 172.21.15.16:0/1945984431 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563666eb64a0 0x563666ec0f60 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:46.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.942+0000 7ff6a9cdc740 1 -- 172.21.15.16:0/1945984431 >> 172.21.15.16:0/1945984431 conn(0x563666eb2a30 msgr2=0x563666eb8710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:46.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.942+0000 7ff688ff9640 0 client.17511 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:08:46.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.942+0000 7ff6a9cdc740 1 -- 172.21.15.16:0/1945984431 shutdown_connections 2024-09-12T02:08:46.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.942+0000 7ff6a9cdc740 1 -- 172.21.15.16:0/1945984431 wait complete. 2024-09-12T02:08:46.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.942+0000 7ff6a9cdc740 20 client.17511 trim_cache size 0 max 16384 2024-09-12T02:08:46.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.959+0000 7ff6a9cdc740 1 Processor -- start 2024-09-12T02:08:46.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.959+0000 7ff6a9cdc740 1 -- start start 2024-09-12T02:08:46.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.959+0000 7ff6a9cdc740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563666eb64a0 0x563666ecd450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:46.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.959+0000 7ff6a9cdc740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563666eb6dc0 0x563666ed87f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:46.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.959+0000 7ff6a9cdc740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563666ec1680 0x563666ed8d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:46.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.959+0000 7ff6a9cdc740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff6a01b1f70 con 0x563666eb6dc0 2024-09-12T02:08:46.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.959+0000 7ff6a9cdc740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff6a01af840 con 0x563666eb64a0 2024-09-12T02:08:46.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.959+0000 7ff6a9cdc740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff6a01ad6b0 con 0x563666ec1680 2024-09-12T02:08:46.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.959+0000 7ff6a4e9e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563666eb6dc0 0x563666ed87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:46.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.960+0000 7ff6a4e9e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563666eb6dc0 0x563666ed87f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48688/0 (socket says 172.21.15.16:48688) 2024-09-12T02:08:46.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.960+0000 7ff6a4e9e640 1 -- 172.21.15.16:0/2993160344 learned_addr learned my addr 172.21.15.16:0/2993160344 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:08:46.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.960+0000 7ff6a569f640 1 --2- 172.21.15.16:0/2993160344 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563666eb64a0 0x563666ecd450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:46.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.960+0000 7ff6a5ea0640 1 --2- 172.21.15.16:0/2993160344 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563666ec1680 0x563666ed8d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:46.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.960+0000 7ff6a4e9e640 1 -- 172.21.15.16:0/2993160344 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563666ec1680 msgr2=0x563666ed8d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:46.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.960+0000 7ff6a4e9e640 1 --2- 172.21.15.16:0/2993160344 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563666ec1680 0x563666ed8d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:46.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.960+0000 7ff6a4e9e640 1 -- 172.21.15.16:0/2993160344 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563666eb64a0 msgr2=0x563666ecd450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:46.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.960+0000 7ff6a4e9e640 1 --2- 172.21.15.16:0/2993160344 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563666eb64a0 0x563666ecd450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:46.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.960+0000 7ff6a4e9e640 1 -- 172.21.15.16:0/2993160344 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563666eb7a20 con 0x563666eb6dc0 2024-09-12T02:08:46.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.960+0000 7ff6a5ea0640 1 --2- 172.21.15.16:0/2993160344 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563666ec1680 0x563666ed8d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:08:46.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.960+0000 7ff6a4e9e640 1 --2- 172.21.15.16:0/2993160344 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563666eb6dc0 0x563666ed87f0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7ff690009900 tx=0x7ff6900058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ddc70a00f5b2097 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:46.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.960+0000 7ff68affd640 1 -- 172.21.15.16:0/2993160344 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff690035040 con 0x563666eb6dc0 2024-09-12T02:08:46.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.960+0000 7ff68affd640 1 -- 172.21.15.16:0/2993160344 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff690049070 con 0x563666eb6dc0 2024-09-12T02:08:46.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.960+0000 7ff68affd640 1 -- 172.21.15.16:0/2993160344 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff690031830 con 0x563666eb6dc0 2024-09-12T02:08:46.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.961+0000 7ff6a9cdc740 1 -- 172.21.15.16:0/2993160344 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563666eb6dc0 msgr2=0x563666ed87f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:46.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.961+0000 7ff6a9cdc740 1 --2- 172.21.15.16:0/2993160344 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563666eb6dc0 0x563666ed87f0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7ff690009900 tx=0x7ff6900058b0 comp rx=0 tx=0).stop 2024-09-12T02:08:46.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.961+0000 7ff6a9cdc740 1 -- 172.21.15.16:0/2993160344 shutdown_connections 2024-09-12T02:08:46.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.961+0000 7ff6a9cdc740 1 --2- 172.21.15.16:0/2993160344 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563666ec1680 0x563666ed8d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:46.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.961+0000 7ff6a9cdc740 1 --2- 172.21.15.16:0/2993160344 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563666eb6dc0 0x563666ed87f0 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:46.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.961+0000 7ff6a9cdc740 1 --2- 172.21.15.16:0/2993160344 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563666eb64a0 0x563666ecd450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:46.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.961+0000 7ff6a9cdc740 1 -- 172.21.15.16:0/2993160344 >> 172.21.15.16:0/2993160344 conn(0x563666eb2a30 msgr2=0x563666ebd1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:46.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.961+0000 7ff6a9cdc740 1 -- 172.21.15.16:0/2993160344 shutdown_connections 2024-09-12T02:08:46.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.961+0000 7ff6a9cdc740 1 -- 172.21.15.16:0/2993160344 wait complete. 2024-09-12T02:08:46.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.961+0000 7ff6a9cdc740 1 -- start start 2024-09-12T02:08:46.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.961+0000 7ff6a9cdc740 1 Processor -- start 2024-09-12T02:08:46.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.962+0000 7ff6a9cdc740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563666eb64a0 0x563666ec9930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:46.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.962+0000 7ff6a9cdc740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563666eb6dc0 0x563666ec9e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:46.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.962+0000 7ff6a9cdc740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563666ec1680 0x563666ec6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:46.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.962+0000 7ff6a9cdc740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff6a01add00 con 0x563666ec1680 2024-09-12T02:08:46.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.962+0000 7ff6a9cdc740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff6a01af170 con 0x563666eb6dc0 2024-09-12T02:08:46.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.962+0000 7ff6a9cdc740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff6a01ae3d0 con 0x563666eb64a0 2024-09-12T02:08:46.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.962+0000 7ff6a9cdc740 10 client.0 _ll_register_callbacks cb 0x7ff690009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:08:46.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.962+0000 7ff6a5ea0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563666ec1680 0x563666ec6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:46.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.962+0000 7ff6a5ea0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563666ec1680 0x563666ec6ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48696/0 (socket says 172.21.15.16:48696) 2024-09-12T02:08:46.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.962+0000 7ff6a5ea0640 1 -- 172.21.15.16:0/3336036975 learned_addr learned my addr 172.21.15.16:0/3336036975 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:08:46.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.962+0000 7ff6a4e9e640 1 --2- 172.21.15.16:0/3336036975 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563666eb6dc0 0x563666ec9e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:46.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.962+0000 7ff6a569f640 1 --2- 172.21.15.16:0/3336036975 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563666eb64a0 0x563666ec9930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:46.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.962+0000 7ff6a5ea0640 1 -- 172.21.15.16:0/3336036975 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563666eb64a0 msgr2=0x563666ec9930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:46.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.962+0000 7ff6a5ea0640 1 --2- 172.21.15.16:0/3336036975 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563666eb64a0 0x563666ec9930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:46.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.962+0000 7ff6a5ea0640 1 -- 172.21.15.16:0/3336036975 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563666eb6dc0 msgr2=0x563666ec9e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:46.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.962+0000 7ff6a5ea0640 1 --2- 172.21.15.16:0/3336036975 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563666eb6dc0 0x563666ec9e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:46.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.962+0000 7ff6a5ea0640 1 -- 172.21.15.16:0/3336036975 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563666ec7700 con 0x563666ec1680 2024-09-12T02:08:46.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.962+0000 7ff6a5ea0640 1 --2- 172.21.15.16:0/3336036975 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563666ec1680 0x563666ec6ed0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7ff69800aca0 tx=0x7ff698004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=306a657e70b1c707 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:46.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.963+0000 7ff688ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:08:46.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.963+0000 7ff688ff9640 1 -- 172.21.15.16:0/3336036975 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff698012b20 con 0x563666ec1680 2024-09-12T02:08:46.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.963+0000 7ff6a9cdc740 1 -- 172.21.15.16:0/3336036975 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff690049070 con 0x563666ec1680 2024-09-12T02:08:46.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.963+0000 7ff688ff9640 1 -- 172.21.15.16:0/3336036975 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7ff698012cc0 con 0x563666ec1680 2024-09-12T02:08:46.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.963+0000 7ff688ff9640 1 -- 172.21.15.16:0/3336036975 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff698020450 con 0x563666ec1680 2024-09-12T02:08:46.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.963+0000 7ff6a9cdc740 10 client.17517 fetch_fsmap learned FSMap version 25 2024-09-12T02:08:46.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.963+0000 7ff6a9cdc740 1 -- 172.21.15.16:0/3336036975 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x563666f3b3f0 con 0x563666ec1680 2024-09-12T02:08:46.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.963+0000 7ff688ff9640 1 -- 172.21.15.16:0/3336036975 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff690049070 con 0x563666ec1680 2024-09-12T02:08:46.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.963+0000 7ff688ff9640 1 -- 172.21.15.16:0/3336036975 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7ff6980295c0 con 0x563666ec1680 2024-09-12T02:08:46.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.963+0000 7ff6a9cdc740 10 client.17517 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:08:46.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.963+0000 7ff6a9cdc740 10 client.17517 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:08:46.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.963+0000 7ff6a9cdc740 20 client.17517 populate_metadata read hostname 'smithi016' 2024-09-12T02:08:46.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.963+0000 7ff6a9cdc740 1 --2- 172.21.15.16:0/3336036975 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x563666f3bb70 0x563666f3df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:46.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.963+0000 7ff6a9cdc740 4 client.17517 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T02:08:46.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.963+0000 7ff6a9cdc740 1 -- 172.21.15.16:0/3336036975 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563666e23d60 con 0x563666f3bb70 2024-09-12T02:08:46.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.964+0000 7ff6a569f640 1 --2- 172.21.15.16:0/3336036975 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x563666f3bb70 0x563666f3df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:46.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.964+0000 7ff6a569f640 1 --2- 172.21.15.16:0/3336036975 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x563666f3bb70 0x563666f3df90 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5d243eae78c4ebf server_cookie=66f324ae2e0f5624 in_seq=0 out_seq=0 2024-09-12T02:08:46.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.964+0000 7ff688ff9640 10 client.17517 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:08:46.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.964+0000 7ff688ff9640 1 -- 172.21.15.16:0/3336036975 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x563666e23d60 con 0x563666f3bb70 2024-09-12T02:08:46.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.964+0000 7ff688ff9640 10 client.17517 handle_command_reply: tid=0 2024-09-12T02:08:46.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.964+0000 7ff6a9cdc740 1 client.17517 shutdown 2024-09-12T02:08:46.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.964+0000 7ff6a9cdc740 1 -- 172.21.15.16:0/3336036975 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563666ec1680 msgr2=0x563666ec6ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:46.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.964+0000 7ff6a9cdc740 1 --2- 172.21.15.16:0/3336036975 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563666ec1680 0x563666ec6ed0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7ff69800aca0 tx=0x7ff698004a40 comp rx=0 tx=0).stop 2024-09-12T02:08:46.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.964+0000 7ff6a9cdc740 1 -- 172.21.15.16:0/3336036975 shutdown_connections 2024-09-12T02:08:46.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.964+0000 7ff6a9cdc740 1 --2- 172.21.15.16:0/3336036975 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x563666f3bb70 0x563666f3df90 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:46.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.964+0000 7ff6a9cdc740 1 --2- 172.21.15.16:0/3336036975 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563666ec1680 0x563666ec6ed0 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:46.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.964+0000 7ff6a9cdc740 1 --2- 172.21.15.16:0/3336036975 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563666eb6dc0 0x563666ec9e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:46.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.964+0000 7ff6a9cdc740 1 --2- 172.21.15.16:0/3336036975 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563666eb64a0 0x563666ec9930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:46.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.964+0000 7ff688ff9640 0 client.17517 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:08:46.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.964+0000 7ff6a9cdc740 1 -- 172.21.15.16:0/3336036975 >> 172.21.15.16:0/3336036975 conn(0x563666eb2a30 msgr2=0x563666eb1270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:46.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.965+0000 7ff6a9cdc740 1 -- 172.21.15.16:0/3336036975 shutdown_connections 2024-09-12T02:08:46.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.965+0000 7ff6a9cdc740 1 -- 172.21.15.16:0/3336036975 wait complete. 2024-09-12T02:08:46.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.965+0000 7ff6a9cdc740 20 client.17517 trim_cache size 0 max 16384 2024-09-12T02:08:46.970 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T02:08:46.970 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-09-12T02:08:46.970 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {} 2024-09-12T02:08:46.970 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T02:08:46.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.965+0000 7ff6a7903640 1 -- 172.21.15.16:0/382766454 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7ff68c0a2c00 msgr2=0x7ff68c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:46.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.965+0000 7ff6a7903640 1 --2- 172.21.15.16:0/382766454 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7ff68c0a2c00 0x7ff68c0a50c0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7ff69800a780 tx=0x7ff698005f60 comp rx=0 tx=0).stop 2024-09-12T02:08:46.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.965+0000 7ff6a7903640 1 -- 172.21.15.16:0/382766454 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6a0100e20 msgr2=0x7ff6a01984e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:46.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.965+0000 7ff6a7903640 1 --2- 172.21.15.16:0/382766454 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6a0100e20 0x7ff6a01984e0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7ff69c00d950 tx=0x7ff69c00de10 comp rx=0 tx=0).stop 2024-09-12T02:08:46.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.965+0000 7ff6a7903640 1 -- 172.21.15.16:0/382766454 shutdown_connections 2024-09-12T02:08:46.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.965+0000 7ff6a7903640 1 --2- 172.21.15.16:0/382766454 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7ff68c0a2c00 0x7ff68c0a50c0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:46.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.965+0000 7ff6a7903640 1 --2- 172.21.15.16:0/382766454 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6a0102ef0 0x7ff6a019cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:46.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.965+0000 7ff6a7903640 1 --2- 172.21.15.16:0/382766454 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff6a0102570 0x7ff6a0198a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:46.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.965+0000 7ff6a7903640 1 --2- 172.21.15.16:0/382766454 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6a0100e20 0x7ff6a01984e0 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:46.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.965+0000 7ff6a7903640 1 -- 172.21.15.16:0/382766454 >> 172.21.15.16:0/382766454 conn(0x7ff6a00fc7d0 msgr2=0x7ff6a006a210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:46.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.965+0000 7ff6a7903640 1 -- 172.21.15.16:0/382766454 shutdown_connections 2024-09-12T02:08:46.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:46.965+0000 7ff6a7903640 1 -- 172.21.15.16:0/382766454 wait complete. 2024-09-12T02:08:47.771 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-12T02:08:47.771 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:08:48.031 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:08:48.365 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:08:48.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:08:48 smithi016 ceph-mon[30947]: pgmap v541: 97 pgs: 97 active+clean; 3.7 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.9 KiB/s wr, 9 op/s 2024-09-12T02:08:48.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:48.530+0000 7f00b5ed3640 1 -- 172.21.15.16:0/896081173 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00b0106f20 msgr2=0x7f00b010eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:48.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:48.530+0000 7f00b5ed3640 1 --2- 172.21.15.16:0/896081173 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00b0106f20 0x7f00b010eaa0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f00a8009ca0 tx=0x7f00a802a8c0 comp rx=0 tx=0).stop 2024-09-12T02:08:48.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:48.530+0000 7f00b5ed3640 1 -- 172.21.15.16:0/896081173 shutdown_connections 2024-09-12T02:08:48.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:48.530+0000 7f00b5ed3640 1 --2- 172.21.15.16:0/896081173 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00b0106f20 0x7f00b010eaa0 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:48.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:48.530+0000 7f00b5ed3640 1 --2- 172.21.15.16:0/896081173 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00b01027a0 0x7f00b01067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:48.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:48.530+0000 7f00b5ed3640 1 --2- 172.21.15.16:0/896081173 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f00b0101df0 0x7f00b01021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:48.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:48.530+0000 7f00b5ed3640 1 -- 172.21.15.16:0/896081173 >> 172.21.15.16:0/896081173 conn(0x7f00b00fd8f0 msgr2=0x7f00b00ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:48.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:48.530+0000 7f00b5ed3640 1 -- 172.21.15.16:0/896081173 shutdown_connections 2024-09-12T02:08:48.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:48.530+0000 7f00b5ed3640 1 -- 172.21.15.16:0/896081173 wait complete. 2024-09-12T02:08:48.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:48.531+0000 7f00b5ed3640 1 Processor -- start 2024-09-12T02:08:48.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:48.531+0000 7f00b5ed3640 1 -- start start 2024-09-12T02:08:48.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:48.531+0000 7f00b5ed3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00b0101df0 0x7f00b019a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:48.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:48.531+0000 7f00b5ed3640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f00b01027a0 0x7f00b019ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:48.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:48.531+0000 7f00b5ed3640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00b0106f20 0x7f00b019ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:48.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:48.531+0000 7f00b5ed3640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f00b0111ff0 con 0x7f00b0101df0 2024-09-12T02:08:48.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:48.531+0000 7f00b5ed3640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f00b0111e70 con 0x7f00b0106f20 2024-09-12T02:08:48.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:48.531+0000 7f00b5ed3640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f00b0112170 con 0x7f00b01027a0 2024-09-12T02:08:48.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:48.531+0000 7f00af7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00b0101df0 0x7f00b019a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:48.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:48.531+0000 7f00af7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00b0101df0 0x7f00b019a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48716/0 (socket says 172.21.15.16:48716) 2024-09-12T02:08:48.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:48.531+0000 7f00af7fe640 1 -- 172.21.15.16:0/2665802327 learned_addr learned my addr 172.21.15.16:0/2665802327 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:08:48.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:48.531+0000 7f00aeffd640 1 --2- 172.21.15.16:0/2665802327 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f00b01027a0 0x7f00b019ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:48.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:48.531+0000 7f00affff640 1 --2- 172.21.15.16:0/2665802327 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00b0106f20 0x7f00b019ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:48.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:48.532+0000 7f00af7fe640 1 -- 172.21.15.16:0/2665802327 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f00b01027a0 msgr2=0x7f00b019ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:48.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:48.532+0000 7f00af7fe640 1 --2- 172.21.15.16:0/2665802327 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f00b01027a0 0x7f00b019ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:48.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:48.532+0000 7f00af7fe640 1 -- 172.21.15.16:0/2665802327 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00b0106f20 msgr2=0x7f00b019ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:48.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:48.532+0000 7f00af7fe640 1 --2- 172.21.15.16:0/2665802327 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00b0106f20 0x7f00b019ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:48.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:48.532+0000 7f00af7fe640 1 -- 172.21.15.16:0/2665802327 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00b019f5f0 con 0x7f00b0101df0 2024-09-12T02:08:48.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:48.532+0000 7f00af7fe640 1 --2- 172.21.15.16:0/2665802327 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00b0101df0 0x7f00b019a640 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f00a000b4a0 tx=0x7f00a000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:48.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:48.532+0000 7f00acff9640 1 -- 172.21.15.16:0/2665802327 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00a0014070 con 0x7f00b0101df0 2024-09-12T02:08:48.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:48.532+0000 7f00b5ed3640 1 -- 172.21.15.16:0/2665802327 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00b019f8e0 con 0x7f00b0101df0 2024-09-12T02:08:48.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:48.532+0000 7f00acff9640 1 -- 172.21.15.16:0/2665802327 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00a0010af0 con 0x7f00b0101df0 2024-09-12T02:08:48.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:48.532+0000 7f00acff9640 1 -- 172.21.15.16:0/2665802327 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00a0017d10 con 0x7f00b0101df0 2024-09-12T02:08:48.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:48.532+0000 7f00b5ed3640 1 -- 172.21.15.16:0/2665802327 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00b01a71c0 con 0x7f00b0101df0 2024-09-12T02:08:48.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:48.533+0000 7f00b5ed3640 1 -- 172.21.15.16:0/2665802327 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00b01021d0 con 0x7f00b0101df0 2024-09-12T02:08:48.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:48.534+0000 7f00acff9640 1 -- 172.21.15.16:0/2665802327 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f00a0017070 con 0x7f00b0101df0 2024-09-12T02:08:48.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:48.535+0000 7f00acff9640 1 --2- 172.21.15.16:0/2665802327 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f009c0a2b80 0x7f009c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:48.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:48.535+0000 7f00aeffd640 1 --2- 172.21.15.16:0/2665802327 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f009c0a2b80 0x7f009c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:48.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:48.535+0000 7f00acff9640 1 -- 172.21.15.16:0/2665802327 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f00a00ce3e0 con 0x7f00b0101df0 2024-09-12T02:08:48.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:48.536+0000 7f00aeffd640 1 --2- 172.21.15.16:0/2665802327 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f009c0a2b80 0x7f009c0a5040 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f00b00ff720 tx=0x7f009800a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:48.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:48.544+0000 7f00acff9640 1 -- 172.21.15.16:0/2665802327 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f00a009b3e0 con 0x7f00b0101df0 2024-09-12T02:08:48.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:08:48 smithi184 ceph-mon[35593]: pgmap v541: 97 pgs: 97 active+clean; 3.7 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.9 KiB/s wr, 9 op/s 2024-09-12T02:08:48.641 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:08:48.668 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:08:48 smithi033 ceph-mon[35332]: pgmap v541: 97 pgs: 97 active+clean; 3.7 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.9 KiB/s wr, 9 op/s 2024-09-12T02:08:48.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:48.773+0000 7f00b5ed3640 1 -- 172.21.15.16:0/2665802327 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f00b019b9a0 con 0x7f00b0101df0 2024-09-12T02:08:48.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:48.774+0000 7f00acff9640 1 -- 172.21.15.16:0/2665802327 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11911 (secure 0 0 0) 0x7f00a009ace0 con 0x7f00b0101df0 2024-09-12T02:08:48.774 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T02:08:48.775 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":25,"btime":"2024-09-12T02:01:37:192021+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3279123279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3279123279},{"type":"v1","addr":"172.21.15.16:6835","nonce":3279123279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4119452382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4119452382},{"type":"v1","addr":"172.21.15.16:6837","nonce":4119452382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/3263823165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3263823165},{"type":"v1","addr":"172.21.15.16:6839","nonce":3263823165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/1510290732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":1510290732},{"type":"v1","addr":"172.21.15.16:6841","nonce":1510290732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/279674603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":279674603},{"type":"v1","addr":"172.21.15.33:6833","nonce":279674603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3555728028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3555728028},{"type":"v1","addr":"172.21.15.33:6835","nonce":3555728028}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T01:59:56.117608+0000","modified":"2024-09-12T02:01:36.294760+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24482,"mds_1":24481,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6833/1563374106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1563374106},{"type":"v1","addr":"172.21.15.184:6833","nonce":1563374106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6837/1892833868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1892833868},{"type":"v1","addr":"172.21.15.33:6837","nonce":1892833868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/1457679686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1457679686},{"type":"v1","addr":"172.21.15.184:6835","nonce":1457679686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.184:6837/1793393087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":1793393087},{"type":"v1","addr":"172.21.15.184:6837","nonce":1793393087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.184:6839/4203347941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":4203347941},{"type":"v1","addr":"172.21.15.184:6839","nonce":4203347941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.33:6839/4136670204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":4136670204},{"type":"v1","addr":"172.21.15.33:6839","nonce":4136670204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24481,24482,24469]},"id":1}]} 2024-09-12T02:08:48.775 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 25 2024-09-12T02:08:48.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:48.776+0000 7f00b5ed3640 1 -- 172.21.15.16:0/2665802327 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f009c0a2b80 msgr2=0x7f009c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:48.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:48.776+0000 7f00b5ed3640 1 --2- 172.21.15.16:0/2665802327 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f009c0a2b80 0x7f009c0a5040 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f00b00ff720 tx=0x7f009800a2d0 comp rx=0 tx=0).stop 2024-09-12T02:08:48.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:48.776+0000 7f00b5ed3640 1 -- 172.21.15.16:0/2665802327 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00b0101df0 msgr2=0x7f00b019a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:48.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:48.776+0000 7f00b5ed3640 1 --2- 172.21.15.16:0/2665802327 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00b0101df0 0x7f00b019a640 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f00a000b4a0 tx=0x7f00a000b960 comp rx=0 tx=0).stop 2024-09-12T02:08:48.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:48.776+0000 7f00b5ed3640 1 -- 172.21.15.16:0/2665802327 shutdown_connections 2024-09-12T02:08:48.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:48.776+0000 7f00b5ed3640 1 --2- 172.21.15.16:0/2665802327 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f009c0a2b80 0x7f009c0a5040 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:48.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:48.776+0000 7f00b5ed3640 1 --2- 172.21.15.16:0/2665802327 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00b0106f20 0x7f00b019ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:48.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:48.776+0000 7f00b5ed3640 1 --2- 172.21.15.16:0/2665802327 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f00b01027a0 0x7f00b019ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:48.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:48.776+0000 7f00b5ed3640 1 --2- 172.21.15.16:0/2665802327 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00b0101df0 0x7f00b019a640 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:48.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:48.776+0000 7f00b5ed3640 1 -- 172.21.15.16:0/2665802327 >> 172.21.15.16:0/2665802327 conn(0x7f00b00fd8f0 msgr2=0x7f00b00feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:48.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:48.777+0000 7f00b5ed3640 1 -- 172.21.15.16:0/2665802327 shutdown_connections 2024-09-12T02:08:48.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:48.777+0000 7f00b5ed3640 1 -- 172.21.15.16:0/2665802327 wait complete. 2024-09-12T02:08:49.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:08:49 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/2665802327' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:08:49.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:08:49 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/2665802327' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:08:49.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:08:49 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/2665802327' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:08:49.670 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-12T02:08:49.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:49.897+0000 7fc87cff7640 1 -- 172.21.15.16:0/1451484844 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc878102ef0 msgr2=0x7fc87810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:49.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:49.897+0000 7fc87cff7640 1 --2- 172.21.15.16:0/1451484844 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc878102ef0 0x7fc87810da40 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7fc870009ca0 tx=0x7fc87002a950 comp rx=0 tx=0).stop 2024-09-12T02:08:49.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:49.897+0000 7fc87cff7640 1 -- 172.21.15.16:0/1451484844 shutdown_connections 2024-09-12T02:08:49.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:49.898+0000 7fc87cff7640 1 --2- 172.21.15.16:0/1451484844 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc878102ef0 0x7fc87810da40 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:49.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:49.898+0000 7fc87cff7640 1 --2- 172.21.15.16:0/1451484844 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc878102570 0x7fc8781029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:49.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:49.898+0000 7fc87cff7640 1 --2- 172.21.15.16:0/1451484844 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc878100e20 0x7fc878101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:49.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:49.898+0000 7fc87cff7640 1 -- 172.21.15.16:0/1451484844 >> 172.21.15.16:0/1451484844 conn(0x7fc8780fc7d0 msgr2=0x7fc8780febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:49.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:49.898+0000 7fc87cff7640 1 -- 172.21.15.16:0/1451484844 shutdown_connections 2024-09-12T02:08:49.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:49.898+0000 7fc87cff7640 1 -- 172.21.15.16:0/1451484844 wait complete. 2024-09-12T02:08:49.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:49.898+0000 7fc87cff7640 1 Processor -- start 2024-09-12T02:08:49.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:49.898+0000 7fc87cff7640 1 -- start start 2024-09-12T02:08:49.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:49.899+0000 7fc87cff7640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc878100e20 0x7fc8781992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:49.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:49.899+0000 7fc87cff7640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc878102570 0x7fc878199810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:49.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:49.899+0000 7fc87cff7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc878102ef0 0x7fc8781a4350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:49.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:49.899+0000 7fc87cff7640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc878110f80 con 0x7fc878102ef0 2024-09-12T02:08:49.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:49.899+0000 7fc87cff7640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc878110e00 con 0x7fc878102570 2024-09-12T02:08:49.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:49.899+0000 7fc87cff7640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc878111100 con 0x7fc878100e20 2024-09-12T02:08:49.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:49.899+0000 7fc876d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc878102ef0 0x7fc8781a4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:49.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:49.899+0000 7fc876d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc878102ef0 0x7fc8781a4350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48728/0 (socket says 172.21.15.16:48728) 2024-09-12T02:08:49.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:49.899+0000 7fc876d9d640 1 -- 172.21.15.16:0/1884948864 learned_addr learned my addr 172.21.15.16:0/1884948864 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:08:49.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:49.899+0000 7fc87659c640 1 --2- 172.21.15.16:0/1884948864 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc878100e20 0x7fc8781992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:49.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:49.899+0000 7fc875d9b640 1 --2- 172.21.15.16:0/1884948864 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc878102570 0x7fc878199810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:49.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:49.899+0000 7fc876d9d640 1 -- 172.21.15.16:0/1884948864 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc878100e20 msgr2=0x7fc8781992d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:49.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:49.899+0000 7fc876d9d640 1 --2- 172.21.15.16:0/1884948864 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc878100e20 0x7fc8781992d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:49.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:49.899+0000 7fc876d9d640 1 -- 172.21.15.16:0/1884948864 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc878102570 msgr2=0x7fc878199810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:49.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:49.899+0000 7fc876d9d640 1 --2- 172.21.15.16:0/1884948864 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc878102570 0x7fc878199810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:49.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:49.899+0000 7fc876d9d640 1 -- 172.21.15.16:0/1884948864 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8781a4890 con 0x7fc878102ef0 2024-09-12T02:08:49.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:49.899+0000 7fc875d9b640 1 --2- 172.21.15.16:0/1884948864 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc878102570 0x7fc878199810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:08:49.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:49.899+0000 7fc876d9d640 1 --2- 172.21.15.16:0/1884948864 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc878102ef0 0x7fc8781a4350 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7fc87002ae70 tx=0x7fc87002f5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:49.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:49.900+0000 7fc86f7fe640 1 -- 172.21.15.16:0/1884948864 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc870034040 con 0x7fc878102ef0 2024-09-12T02:08:49.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:49.900+0000 7fc87cff7640 1 -- 172.21.15.16:0/1884948864 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8781a4a60 con 0x7fc878102ef0 2024-09-12T02:08:49.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:49.900+0000 7fc86f7fe640 1 -- 172.21.15.16:0/1884948864 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc870030920 con 0x7fc878102ef0 2024-09-12T02:08:49.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:49.900+0000 7fc86f7fe640 1 -- 172.21.15.16:0/1884948864 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc870045480 con 0x7fc878102ef0 2024-09-12T02:08:49.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:49.900+0000 7fc87cff7640 1 -- 172.21.15.16:0/1884948864 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8781a4eb0 con 0x7fc878102ef0 2024-09-12T02:08:49.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:49.901+0000 7fc87cff7640 1 -- 172.21.15.16:0/1884948864 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc878101200 con 0x7fc878102ef0 2024-09-12T02:08:49.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:49.902+0000 7fc86f7fe640 1 -- 172.21.15.16:0/1884948864 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fc870037020 con 0x7fc878102ef0 2024-09-12T02:08:49.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:49.903+0000 7fc86f7fe640 1 --2- 172.21.15.16:0/1884948864 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc8640a2b80 0x7fc8640a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:49.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:49.903+0000 7fc87659c640 1 --2- 172.21.15.16:0/1884948864 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc8640a2b80 0x7fc8640a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:49.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:49.903+0000 7fc86f7fe640 1 -- 172.21.15.16:0/1884948864 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fc8700f0000 con 0x7fc878102ef0 2024-09-12T02:08:49.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:49.903+0000 7fc87659c640 1 --2- 172.21.15.16:0/1884948864 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc8640a2b80 0x7fc8640a5040 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fc86800bbe0 tx=0x7fc86800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:49.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:49.907+0000 7fc86f7fe640 1 -- 172.21.15.16:0/1884948864 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc8700811b0 con 0x7fc878102ef0 2024-09-12T02:08:49.974 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:08:50.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:50.135+0000 7fc87cff7640 1 -- 172.21.15.16:0/1884948864 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc8781a5110 con 0x7fc878102ef0 2024-09-12T02:08:50.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:50.136+0000 7fc86f7fe640 1 -- 172.21.15.16:0/1884948864 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11911 (secure 0 0 0) 0x7fc8700d0e20 con 0x7fc878102ef0 2024-09-12T02:08:50.136 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T02:08:50.136 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":25,"btime":"2024-09-12T02:01:37:192021+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3279123279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3279123279},{"type":"v1","addr":"172.21.15.16:6835","nonce":3279123279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4119452382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4119452382},{"type":"v1","addr":"172.21.15.16:6837","nonce":4119452382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/3263823165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3263823165},{"type":"v1","addr":"172.21.15.16:6839","nonce":3263823165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/1510290732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":1510290732},{"type":"v1","addr":"172.21.15.16:6841","nonce":1510290732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/279674603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":279674603},{"type":"v1","addr":"172.21.15.33:6833","nonce":279674603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3555728028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3555728028},{"type":"v1","addr":"172.21.15.33:6835","nonce":3555728028}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T01:59:56.117608+0000","modified":"2024-09-12T02:01:36.294760+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24482,"mds_1":24481,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6833/1563374106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1563374106},{"type":"v1","addr":"172.21.15.184:6833","nonce":1563374106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6837/1892833868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1892833868},{"type":"v1","addr":"172.21.15.33:6837","nonce":1892833868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/1457679686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1457679686},{"type":"v1","addr":"172.21.15.184:6835","nonce":1457679686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.184:6837/1793393087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":1793393087},{"type":"v1","addr":"172.21.15.184:6837","nonce":1793393087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.184:6839/4203347941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":4203347941},{"type":"v1","addr":"172.21.15.184:6839","nonce":4203347941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.33:6839/4136670204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":4136670204},{"type":"v1","addr":"172.21.15.33:6839","nonce":4136670204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24481,24482,24469]},"id":1}]} 2024-09-12T02:08:50.137 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 25 2024-09-12T02:08:50.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:50.138+0000 7fc87cff7640 1 -- 172.21.15.16:0/1884948864 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc8640a2b80 msgr2=0x7fc8640a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:50.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:50.138+0000 7fc87cff7640 1 --2- 172.21.15.16:0/1884948864 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc8640a2b80 0x7fc8640a5040 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fc86800bbe0 tx=0x7fc86800b390 comp rx=0 tx=0).stop 2024-09-12T02:08:50.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:50.138+0000 7fc87cff7640 1 -- 172.21.15.16:0/1884948864 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc878102ef0 msgr2=0x7fc8781a4350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:50.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:50.138+0000 7fc87cff7640 1 --2- 172.21.15.16:0/1884948864 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc878102ef0 0x7fc8781a4350 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7fc87002ae70 tx=0x7fc87002f5d0 comp rx=0 tx=0).stop 2024-09-12T02:08:50.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:50.138+0000 7fc87cff7640 1 -- 172.21.15.16:0/1884948864 shutdown_connections 2024-09-12T02:08:50.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:50.138+0000 7fc87cff7640 1 --2- 172.21.15.16:0/1884948864 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc8640a2b80 0x7fc8640a5040 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:50.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:50.138+0000 7fc87cff7640 1 --2- 172.21.15.16:0/1884948864 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc878102ef0 0x7fc8781a4350 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:50.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:50.138+0000 7fc87cff7640 1 --2- 172.21.15.16:0/1884948864 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc878102570 0x7fc878199810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:50.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:50.138+0000 7fc87cff7640 1 --2- 172.21.15.16:0/1884948864 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc878100e20 0x7fc8781992d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:50.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:50.138+0000 7fc87cff7640 1 -- 172.21.15.16:0/1884948864 >> 172.21.15.16:0/1884948864 conn(0x7fc8780fc7d0 msgr2=0x7fc878106840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:50.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:50.138+0000 7fc87cff7640 1 -- 172.21.15.16:0/1884948864 shutdown_connections 2024-09-12T02:08:50.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:50.138+0000 7fc87cff7640 1 -- 172.21.15.16:0/1884948864 wait complete. 2024-09-12T02:08:50.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:08:50 smithi016 ceph-mon[30947]: pgmap v542: 97 pgs: 97 active+clean; 3.7 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.6 KiB/s wr, 7 op/s 2024-09-12T02:08:50.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:08:50 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/1884948864' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:08:50.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:08:50 smithi184 ceph-mon[35593]: pgmap v542: 97 pgs: 97 active+clean; 3.7 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.6 KiB/s wr, 7 op/s 2024-09-12T02:08:50.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:08:50 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/1884948864' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:08:50.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:08:50 smithi033 ceph-mon[35332]: pgmap v542: 97 pgs: 97 active+clean; 3.7 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.6 KiB/s wr, 7 op/s 2024-09-12T02:08:50.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:08:50 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/1884948864' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:08:50.876 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24482 quiesce db --set-id e05b9d01 --release --await' 2024-09-12T02:08:50.876 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24482 quiesce db --set-id e05b9d01 --release --await 2024-09-12T02:08:51.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.065+0000 7f5ef66e1640 1 -- 172.21.15.16:0/877384052 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ef01036e0 msgr2=0x7f5ef0107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:51.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.065+0000 7f5ef66e1640 1 --2- 172.21.15.16:0/877384052 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ef01036e0 0x7f5ef0107b90 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f5ee4009930 tx=0x7f5ee402a420 comp rx=0 tx=0).stop 2024-09-12T02:08:51.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.065+0000 7f5ef66e1640 1 -- 172.21.15.16:0/877384052 shutdown_connections 2024-09-12T02:08:51.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.065+0000 7f5ef66e1640 1 --2- 172.21.15.16:0/877384052 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5ef01082a0 0x7f5ef010ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:51.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.065+0000 7f5ef66e1640 1 --2- 172.21.15.16:0/877384052 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ef01036e0 0x7f5ef0107b90 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:51.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.065+0000 7f5ef66e1640 1 --2- 172.21.15.16:0/877384052 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ef0102d30 0x7f5ef0103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:51.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.065+0000 7f5ef66e1640 1 -- 172.21.15.16:0/877384052 >> 172.21.15.16:0/877384052 conn(0x7f5ef00fc7d0 msgr2=0x7f5ef00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:51.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.065+0000 7f5ef66e1640 1 -- 172.21.15.16:0/877384052 shutdown_connections 2024-09-12T02:08:51.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.065+0000 7f5ef66e1640 1 -- 172.21.15.16:0/877384052 wait complete. 2024-09-12T02:08:51.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.066+0000 7f5ef66e1640 1 Processor -- start 2024-09-12T02:08:51.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.066+0000 7f5ef66e1640 1 -- start start 2024-09-12T02:08:51.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.066+0000 7f5ef66e1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ef0102d30 0x7f5ef0198600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:51.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.066+0000 7f5ef66e1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ef01036e0 0x7f5ef0198b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:51.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.066+0000 7f5ef66e1640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5ef01082a0 0x7f5ef019ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:51.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.067+0000 7f5ef66e1640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5ef01120a0 con 0x7f5ef01036e0 2024-09-12T02:08:51.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.067+0000 7f5ef66e1640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5ef0111f20 con 0x7f5ef0102d30 2024-09-12T02:08:51.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.067+0000 7f5ef66e1640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5ef0112220 con 0x7f5ef01082a0 2024-09-12T02:08:51.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.067+0000 7f5eef7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ef01036e0 0x7f5ef0198b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:51.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.067+0000 7f5eeffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ef0102d30 0x7f5ef0198600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:51.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.067+0000 7f5eef7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ef01036e0 0x7f5ef0198b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48758/0 (socket says 172.21.15.16:48758) 2024-09-12T02:08:51.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.067+0000 7f5eeffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ef0102d30 0x7f5ef0198600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.16:41076/0 (socket says 172.21.15.16:41076) 2024-09-12T02:08:51.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.067+0000 7f5eef7fe640 1 -- 172.21.15.16:0/4222304943 learned_addr learned my addr 172.21.15.16:0/4222304943 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:08:51.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.067+0000 7f5ef4c7e640 1 --2- 172.21.15.16:0/4222304943 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5ef01082a0 0x7f5ef019ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:51.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.067+0000 7f5eeffff640 1 -- 172.21.15.16:0/4222304943 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5ef01082a0 msgr2=0x7f5ef019ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:51.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.067+0000 7f5eeffff640 1 --2- 172.21.15.16:0/4222304943 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5ef01082a0 0x7f5ef019ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:51.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.067+0000 7f5eeffff640 1 -- 172.21.15.16:0/4222304943 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ef01036e0 msgr2=0x7f5ef0198b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:51.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.067+0000 7f5eeffff640 1 --2- 172.21.15.16:0/4222304943 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ef01036e0 0x7f5ef0198b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:51.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.067+0000 7f5eeffff640 1 -- 172.21.15.16:0/4222304943 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ef019d5b0 con 0x7f5ef0102d30 2024-09-12T02:08:51.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.067+0000 7f5eeffff640 1 --2- 172.21.15.16:0/4222304943 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ef0102d30 0x7f5ef0198600 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f5ee000d6d0 tx=0x7f5ee000db90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:51.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.068+0000 7f5eed7fa640 1 -- 172.21.15.16:0/4222304943 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ee0017070 con 0x7f5ef0102d30 2024-09-12T02:08:51.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.068+0000 7f5eed7fa640 1 -- 172.21.15.16:0/4222304943 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ee0009820 con 0x7f5ef0102d30 2024-09-12T02:08:51.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.068+0000 7f5ef66e1640 1 -- 172.21.15.16:0/4222304943 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ef019d8a0 con 0x7f5ef0102d30 2024-09-12T02:08:51.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.068+0000 7f5eed7fa640 1 -- 172.21.15.16:0/4222304943 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ee0004d00 con 0x7f5ef0102d30 2024-09-12T02:08:51.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.068+0000 7f5ef66e1640 1 -- 172.21.15.16:0/4222304943 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ef01a5180 con 0x7f5ef0102d30 2024-09-12T02:08:51.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.070+0000 7f5ef8aba740 1 Processor -- start 2024-09-12T02:08:51.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.070+0000 7f5ef8aba740 1 -- start start 2024-09-12T02:08:51.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.070+0000 7f5eed7fa640 1 -- 172.21.15.16:0/4222304943 <== mon.2 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5ee00099d0 con 0x7f5ef0102d30 2024-09-12T02:08:51.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.071+0000 7f5ef8aba740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fdd7cdbb40 0x55fdd7cda090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:51.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.071+0000 7f5ef8aba740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fdd7cda5d0 0x55fdd7cdaa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:51.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.071+0000 7f5ef4c7e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fdd7cdbb40 0x55fdd7cda090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:51.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.071+0000 7f5ef4c7e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fdd7cdbb40 0x55fdd7cda090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48760/0 (socket says 172.21.15.16:48760) 2024-09-12T02:08:51.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.071+0000 7f5ef8aba740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fdd7ce9780 0x55fdd7cebb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:51.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.071+0000 7f5ef8aba740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55fdd7c402f0 con 0x55fdd7cdbb40 2024-09-12T02:08:51.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.071+0000 7f5ef8aba740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55fdd7be7cf0 con 0x55fdd7ce9780 2024-09-12T02:08:51.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.071+0000 7f5ef8aba740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55fdd7c3c2b0 con 0x55fdd7cda5d0 2024-09-12T02:08:51.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.071+0000 7f5ef4c7e640 1 -- 172.21.15.16:0/3972371639 learned_addr learned my addr 172.21.15.16:0/3972371639 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:08:51.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.071+0000 7f5eeffff640 1 --2- 172.21.15.16:0/3972371639 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fdd7cda5d0 0x55fdd7cdaa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:51.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.071+0000 7f5eef7fe640 1 --2- 172.21.15.16:0/3972371639 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fdd7ce9780 0x55fdd7cebb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:51.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.071+0000 7f5ef4c7e640 1 -- 172.21.15.16:0/3972371639 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fdd7cda5d0 msgr2=0x55fdd7cdaa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:51.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.071+0000 7f5ef4c7e640 1 --2- 172.21.15.16:0/3972371639 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fdd7cda5d0 0x55fdd7cdaa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:51.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.071+0000 7f5ef4c7e640 1 -- 172.21.15.16:0/3972371639 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fdd7ce9780 msgr2=0x55fdd7cebb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:51.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.071+0000 7f5ef4c7e640 1 --2- 172.21.15.16:0/3972371639 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fdd7ce9780 0x55fdd7cebb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:51.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.071+0000 7f5ef4c7e640 1 -- 172.21.15.16:0/3972371639 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fdd7bcc3a0 con 0x55fdd7cdbb40 2024-09-12T02:08:51.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.072+0000 7f5eef7fe640 1 --2- 172.21.15.16:0/3972371639 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fdd7ce9780 0x55fdd7cebb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:08:51.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.072+0000 7f5ef4c7e640 1 --2- 172.21.15.16:0/3972371639 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fdd7cdbb40 0x55fdd7cda090 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f5ee800ae10 tx=0x7f5ee800f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=745a59dcde1bd07a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:51.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.072+0000 7f5ed67fc640 1 -- 172.21.15.16:0/3972371639 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ee8003b70 con 0x55fdd7cdbb40 2024-09-12T02:08:51.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.072+0000 7f5ed67fc640 1 -- 172.21.15.16:0/3972371639 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ee8003d10 con 0x55fdd7cdbb40 2024-09-12T02:08:51.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.072+0000 7f5ed67fc640 1 -- 172.21.15.16:0/3972371639 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ee8010a30 con 0x55fdd7cdbb40 2024-09-12T02:08:51.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.072+0000 7f5eed7fa640 1 --2- 172.21.15.16:0/4222304943 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f5ed80a2b80 0x7f5ed80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:51.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.072+0000 7f5eeffff640 1 --2- 172.21.15.16:0/4222304943 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f5ed80a2b80 0x7f5ed80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:51.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.072+0000 7f5eed7fa640 1 -- 172.21.15.16:0/4222304943 <== mon.2 v2:172.21.15.33:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f5ee00ceac0 con 0x7f5ef0102d30 2024-09-12T02:08:51.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.073+0000 7f5ef8aba740 1 -- 172.21.15.16:0/3972371639 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fdd7cdbb40 msgr2=0x55fdd7cda090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:51.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.073+0000 7f5ef8aba740 1 --2- 172.21.15.16:0/3972371639 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fdd7cdbb40 0x55fdd7cda090 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f5ee800ae10 tx=0x7f5ee800f790 comp rx=0 tx=0).stop 2024-09-12T02:08:51.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.073+0000 7f5ef8aba740 1 -- 172.21.15.16:0/3972371639 shutdown_connections 2024-09-12T02:08:51.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.073+0000 7f5ef8aba740 1 --2- 172.21.15.16:0/3972371639 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fdd7ce9780 0x55fdd7cebb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:51.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.073+0000 7f5ef8aba740 1 --2- 172.21.15.16:0/3972371639 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fdd7cda5d0 0x55fdd7cdaa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:51.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.073+0000 7f5ef8aba740 1 --2- 172.21.15.16:0/3972371639 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fdd7cdbb40 0x55fdd7cda090 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:51.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.073+0000 7f5ef8aba740 1 -- 172.21.15.16:0/3972371639 >> 172.21.15.16:0/3972371639 conn(0x55fdd7ca3310 msgr2=0x55fdd7cd6c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:51.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.073+0000 7f5ef8aba740 1 -- 172.21.15.16:0/3972371639 shutdown_connections 2024-09-12T02:08:51.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.073+0000 7f5eeffff640 1 --2- 172.21.15.16:0/4222304943 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f5ed80a2b80 0x7f5ed80a5040 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f5ee00a3b00 tx=0x7f5ee0097040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:51.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.073+0000 7f5ef8aba740 1 -- 172.21.15.16:0/3972371639 wait complete. 2024-09-12T02:08:51.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.073+0000 7f5ef8aba740 1 -- start start 2024-09-12T02:08:51.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.074+0000 7f5ef8aba740 1 Processor -- start 2024-09-12T02:08:51.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.074+0000 7f5ef8aba740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fdd7cda5d0 0x55fdd7ce90e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:51.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.074+0000 7f5ef8aba740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fdd7cdbb40 0x55fdd7ce6390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:51.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.074+0000 7f5ef8aba740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fdd7ce9780 0x55fdd7ce68f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:51.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.074+0000 7f5ef8aba740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55fdd7cec480 con 0x55fdd7ce9780 2024-09-12T02:08:51.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.074+0000 7f5ef8aba740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55fdd7bfb060 con 0x55fdd7cda5d0 2024-09-12T02:08:51.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.074+0000 7f5ef8aba740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55fdd7cec600 con 0x55fdd7cdbb40 2024-09-12T02:08:51.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.074+0000 7f5ef8aba740 10 client.0 _ll_register_callbacks cb 0x55fdd794e8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:08:51.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.074+0000 7f5ef4c7e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fdd7ce9780 0x55fdd7ce68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:51.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.074+0000 7f5ef4c7e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fdd7ce9780 0x55fdd7ce68f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48764/0 (socket says 172.21.15.16:48764) 2024-09-12T02:08:51.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.074+0000 7f5ef4c7e640 1 -- 172.21.15.16:0/1880701689 learned_addr learned my addr 172.21.15.16:0/1880701689 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:08:51.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.075+0000 7f5ef4c7e640 1 --2- 172.21.15.16:0/1880701689 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fdd7cda5d0 0x55fdd7ce90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:51.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.075+0000 7f5eef7fe640 1 --2- 172.21.15.16:0/1880701689 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fdd7cdbb40 0x55fdd7ce6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:51.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.075+0000 7f5ef4c7e640 1 -- 172.21.15.16:0/1880701689 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fdd7cdbb40 msgr2=0x55fdd7ce6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:51.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.075+0000 7f5ef4c7e640 1 --2- 172.21.15.16:0/1880701689 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fdd7cdbb40 0x55fdd7ce6390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:51.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.075+0000 7f5ef4c7e640 1 -- 172.21.15.16:0/1880701689 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fdd7cda5d0 msgr2=0x55fdd7ce90e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:51.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.075+0000 7f5ef4c7e640 1 --2- 172.21.15.16:0/1880701689 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fdd7cda5d0 0x55fdd7ce90e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:51.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.075+0000 7f5ef4c7e640 1 -- 172.21.15.16:0/1880701689 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fdd79b13c0 con 0x55fdd7ce9780 2024-09-12T02:08:51.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.075+0000 7f5ef4c7e640 1 --2- 172.21.15.16:0/1880701689 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fdd7ce9780 0x55fdd7ce68f0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f5ee8003c80 tx=0x7f5ee8004950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68563155c42ab67b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:51.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.075+0000 7f5ecf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:08:51.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.075+0000 7f5ecf7fe640 1 -- 172.21.15.16:0/1880701689 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ee8004410 con 0x55fdd7ce9780 2024-09-12T02:08:51.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.075+0000 7f5ecf7fe640 1 -- 172.21.15.16:0/1880701689 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f5ee8004a40 con 0x55fdd7ce9780 2024-09-12T02:08:51.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.075+0000 7f5ef8aba740 1 -- 172.21.15.16:0/1880701689 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ee8003d10 con 0x55fdd7ce9780 2024-09-12T02:08:51.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.075+0000 7f5ecf7fe640 1 -- 172.21.15.16:0/1880701689 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5ee80045b0 con 0x55fdd7ce9780 2024-09-12T02:08:51.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.075+0000 7f5ef8aba740 10 client.17541 fetch_fsmap learned FSMap version 25 2024-09-12T02:08:51.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.075+0000 7f5ef8aba740 1 -- 172.21.15.16:0/1880701689 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55fdd7cf7e90 con 0x55fdd7ce9780 2024-09-12T02:08:51.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.076+0000 7f5ecf7fe640 1 -- 172.21.15.16:0/1880701689 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ee8003d10 con 0x55fdd7ce9780 2024-09-12T02:08:51.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.076+0000 7f5ecf7fe640 1 -- 172.21.15.16:0/1880701689 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f5ee8005690 con 0x55fdd7ce9780 2024-09-12T02:08:51.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.076+0000 7f5ef8aba740 10 client.17541 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:08:51.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.076+0000 7f5ef8aba740 10 client.17541 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-12T02:08:51.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.076+0000 7f5ef8aba740 20 client.17541 populate_metadata read hostname 'smithi016' 2024-09-12T02:08:51.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.076+0000 7f5ef8aba740 1 --2- 172.21.15.16:0/1880701689 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55fdd7cf9290 0x55fdd7cfb680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:51.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.076+0000 7f5ef8aba740 4 client.17541 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:08:51.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.076+0000 7f5ef8aba740 1 -- 172.21.15.16:0/1880701689 --> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fdd7c4d860 con 0x55fdd7cf9290 2024-09-12T02:08:51.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.076+0000 7f5eef7fe640 1 --2- 172.21.15.16:0/1880701689 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55fdd7cf9290 0x55fdd7cfb680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:51.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.077+0000 7f5eef7fe640 1 --2- 172.21.15.16:0/1880701689 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55fdd7cf9290 0x55fdd7cfb680 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=275149acf6337ea3 server_cookie=e76ac2e9069d51dd in_seq=0 out_seq=0 2024-09-12T02:08:51.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.077+0000 7f5ecf7fe640 10 client.17541 ms_handle_connect on v2:172.21.15.184:6836/1793393087 2024-09-12T02:08:51.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.077+0000 7f5ecf7fe640 1 -- 172.21.15.16:0/1880701689 <== mds.2 v2:172.21.15.184:6836/1793393087 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fdd7c4d860 con 0x55fdd7cf9290 2024-09-12T02:08:51.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.077+0000 7f5ecf7fe640 10 client.17541 handle_command_reply: tid=0 2024-09-12T02:08:51.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.077+0000 7f5ef8aba740 1 client.17541 shutdown 2024-09-12T02:08:51.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.078+0000 7f5ef8aba740 1 -- 172.21.15.16:0/1880701689 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fdd7ce9780 msgr2=0x55fdd7ce68f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:51.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.078+0000 7f5ef8aba740 1 --2- 172.21.15.16:0/1880701689 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fdd7ce9780 0x55fdd7ce68f0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f5ee8003c80 tx=0x7f5ee8004950 comp rx=0 tx=0).stop 2024-09-12T02:08:51.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.078+0000 7f5ef8aba740 1 -- 172.21.15.16:0/1880701689 shutdown_connections 2024-09-12T02:08:51.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.078+0000 7f5ef8aba740 1 --2- 172.21.15.16:0/1880701689 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55fdd7cf9290 0x55fdd7cfb680 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:51.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.078+0000 7f5ef8aba740 1 --2- 172.21.15.16:0/1880701689 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fdd7ce9780 0x55fdd7ce68f0 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:51.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.078+0000 7f5ef8aba740 1 --2- 172.21.15.16:0/1880701689 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fdd7cdbb40 0x55fdd7ce6390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:51.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.078+0000 7f5ef8aba740 1 --2- 172.21.15.16:0/1880701689 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fdd7cda5d0 0x55fdd7ce90e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:51.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.078+0000 7f5ef8aba740 1 -- 172.21.15.16:0/1880701689 >> 172.21.15.16:0/1880701689 conn(0x55fdd7ca3310 msgr2=0x55fdd7cdc960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:51.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.078+0000 7f5ecf7fe640 0 client.17541 ms_handle_reset on v2:172.21.15.184:6836/1793393087 2024-09-12T02:08:51.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.078+0000 7f5ef8aba740 1 -- 172.21.15.16:0/1880701689 shutdown_connections 2024-09-12T02:08:51.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.078+0000 7f5ef8aba740 1 -- 172.21.15.16:0/1880701689 wait complete. 2024-09-12T02:08:51.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.078+0000 7f5ef8aba740 20 client.17541 trim_cache size 0 max 16384 2024-09-12T02:08:51.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.091+0000 7f5ef8aba740 1 Processor -- start 2024-09-12T02:08:51.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.091+0000 7f5ef8aba740 1 -- start start 2024-09-12T02:08:51.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.092+0000 7f5ef8aba740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fdd7cda5d0 0x55fdd7cfe2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:51.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.092+0000 7f5ef8aba740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fdd7cdbb40 0x55fdd7cfe800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:51.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.092+0000 7f5ef8aba740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fdd7ce9780 0x55fdd7ce35f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:51.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.092+0000 7f5ef8aba740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5ef01adbb0 con 0x55fdd7ce9780 2024-09-12T02:08:51.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.092+0000 7f5ef8aba740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5ef01ab480 con 0x55fdd7cda5d0 2024-09-12T02:08:51.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.092+0000 7f5ef8aba740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5ef01a9310 con 0x55fdd7cdbb40 2024-09-12T02:08:51.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.092+0000 7f5ef4c7e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fdd7ce9780 0x55fdd7ce35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:51.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.092+0000 7f5ef4c7e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fdd7ce9780 0x55fdd7ce35f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48776/0 (socket says 172.21.15.16:48776) 2024-09-12T02:08:51.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.092+0000 7f5ef4c7e640 1 -- 172.21.15.16:0/2867982339 learned_addr learned my addr 172.21.15.16:0/2867982339 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:08:51.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.092+0000 7f5ef4c7e640 1 --2- 172.21.15.16:0/2867982339 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fdd7cda5d0 0x55fdd7cfe2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:51.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.092+0000 7f5eef7fe640 1 --2- 172.21.15.16:0/2867982339 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fdd7cdbb40 0x55fdd7cfe800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:51.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.092+0000 7f5ef4c7e640 1 -- 172.21.15.16:0/2867982339 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fdd7cdbb40 msgr2=0x55fdd7cfe800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:51.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.092+0000 7f5ef4c7e640 1 --2- 172.21.15.16:0/2867982339 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fdd7cdbb40 0x55fdd7cfe800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:51.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.092+0000 7f5ef4c7e640 1 -- 172.21.15.16:0/2867982339 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fdd7cda5d0 msgr2=0x55fdd7cfe2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:51.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.092+0000 7f5ef4c7e640 1 --2- 172.21.15.16:0/2867982339 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fdd7cda5d0 0x55fdd7cfe2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:51.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.092+0000 7f5ef4c7e640 1 -- 172.21.15.16:0/2867982339 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fdd7cdd120 con 0x55fdd7ce9780 2024-09-12T02:08:51.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.092+0000 7f5ef4c7e640 1 --2- 172.21.15.16:0/2867982339 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fdd7ce9780 0x55fdd7ce35f0 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f5ee800fdd0 tx=0x7f5ee800a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f06355705174136 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:51.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.092+0000 7f5ed5ffb640 1 -- 172.21.15.16:0/2867982339 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ee8004a00 con 0x55fdd7ce9780 2024-09-12T02:08:51.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.093+0000 7f5ed5ffb640 1 -- 172.21.15.16:0/2867982339 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ee8037040 con 0x55fdd7ce9780 2024-09-12T02:08:51.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.093+0000 7f5ed5ffb640 1 -- 172.21.15.16:0/2867982339 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ee8027b40 con 0x55fdd7ce9780 2024-09-12T02:08:51.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.093+0000 7f5ef8aba740 1 -- 172.21.15.16:0/2867982339 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fdd7ce9780 msgr2=0x55fdd7ce35f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:51.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.093+0000 7f5ef8aba740 1 --2- 172.21.15.16:0/2867982339 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fdd7ce9780 0x55fdd7ce35f0 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f5ee800fdd0 tx=0x7f5ee800a5c0 comp rx=0 tx=0).stop 2024-09-12T02:08:51.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.093+0000 7f5ef8aba740 1 -- 172.21.15.16:0/2867982339 shutdown_connections 2024-09-12T02:08:51.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.093+0000 7f5ef8aba740 1 --2- 172.21.15.16:0/2867982339 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fdd7ce9780 0x55fdd7ce35f0 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:51.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.093+0000 7f5ef8aba740 1 --2- 172.21.15.16:0/2867982339 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fdd7cdbb40 0x55fdd7cfe800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:51.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.093+0000 7f5ef8aba740 1 --2- 172.21.15.16:0/2867982339 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fdd7cda5d0 0x55fdd7cfe2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:51.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.093+0000 7f5ef8aba740 1 -- 172.21.15.16:0/2867982339 >> 172.21.15.16:0/2867982339 conn(0x55fdd7ca3310 msgr2=0x55fdd7ce5480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:51.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.093+0000 7f5ef8aba740 1 -- 172.21.15.16:0/2867982339 shutdown_connections 2024-09-12T02:08:51.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.094+0000 7f5ef8aba740 1 -- 172.21.15.16:0/2867982339 wait complete. 2024-09-12T02:08:51.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.094+0000 7f5ef8aba740 1 -- start start 2024-09-12T02:08:51.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.094+0000 7f5ef8aba740 1 Processor -- start 2024-09-12T02:08:51.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.094+0000 7f5ef8aba740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fdd7cda5d0 0x55fdd7cfb7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:51.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.094+0000 7f5ef8aba740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fdd7cdbb40 0x55fdd7cfbcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:51.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.094+0000 7f5ef8aba740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fdd7ce9780 0x55fdd7d61bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:51.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.094+0000 7f5ef8aba740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5ef01a9960 con 0x55fdd7cdbb40 2024-09-12T02:08:51.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.094+0000 7f5ef8aba740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5ef01aadb0 con 0x55fdd7ce9780 2024-09-12T02:08:51.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.094+0000 7f5ef8aba740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5ef01aa010 con 0x55fdd7cda5d0 2024-09-12T02:08:51.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.094+0000 7f5ef8aba740 10 client.0 _ll_register_callbacks cb 0x7f5ee800ac60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:08:51.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.095+0000 7f5eef7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fdd7cdbb40 0x55fdd7cfbcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:51.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.095+0000 7f5eef7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fdd7cdbb40 0x55fdd7cfbcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48792/0 (socket says 172.21.15.16:48792) 2024-09-12T02:08:51.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.095+0000 7f5eef7fe640 1 -- 172.21.15.16:0/2716389506 learned_addr learned my addr 172.21.15.16:0/2716389506 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:08:51.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.095+0000 7f5ef4c7e640 1 --2- 172.21.15.16:0/2716389506 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fdd7cda5d0 0x55fdd7cfb7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:51.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.095+0000 7f5ef4c7e640 1 --2- 172.21.15.16:0/2716389506 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fdd7ce9780 0x55fdd7d61bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:51.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.095+0000 7f5eef7fe640 1 -- 172.21.15.16:0/2716389506 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fdd7cda5d0 msgr2=0x55fdd7cfb7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:51.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.095+0000 7f5eef7fe640 1 --2- 172.21.15.16:0/2716389506 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fdd7cda5d0 0x55fdd7cfb7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:51.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.095+0000 7f5eef7fe640 1 -- 172.21.15.16:0/2716389506 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fdd7ce9780 msgr2=0x55fdd7d61bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:51.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.095+0000 7f5eef7fe640 1 --2- 172.21.15.16:0/2716389506 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fdd7ce9780 0x55fdd7d61bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:51.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.095+0000 7f5eef7fe640 1 -- 172.21.15.16:0/2716389506 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fdd7d62180 con 0x55fdd7cdbb40 2024-09-12T02:08:51.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.095+0000 7f5eef7fe640 1 --2- 172.21.15.16:0/2716389506 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fdd7cdbb40 0x55fdd7cfbcf0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f5ee4005940 tx=0x7f5ee4031a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e0b1e5a3fc02aaa server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:51.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.095+0000 7f5ecf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:08:51.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.095+0000 7f5ecf7fe640 1 -- 172.21.15.16:0/2716389506 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ee4035040 con 0x55fdd7cdbb40 2024-09-12T02:08:51.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.095+0000 7f5ecf7fe640 1 -- 172.21.15.16:0/2716389506 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f5ee4004450 con 0x55fdd7cdbb40 2024-09-12T02:08:51.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.095+0000 7f5ef8aba740 1 -- 172.21.15.16:0/2716389506 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ee8037040 con 0x55fdd7cdbb40 2024-09-12T02:08:51.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.096+0000 7f5ecf7fe640 1 -- 172.21.15.16:0/2716389506 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5ee4005020 con 0x55fdd7cdbb40 2024-09-12T02:08:51.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.096+0000 7f5ef8aba740 10 client.17547 fetch_fsmap learned FSMap version 25 2024-09-12T02:08:51.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.096+0000 7f5ef8aba740 1 -- 172.21.15.16:0/2716389506 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55fdd7d634d0 con 0x55fdd7cdbb40 2024-09-12T02:08:51.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.096+0000 7f5ecf7fe640 1 -- 172.21.15.16:0/2716389506 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ee8037040 con 0x55fdd7cdbb40 2024-09-12T02:08:51.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.096+0000 7f5ecf7fe640 1 -- 172.21.15.16:0/2716389506 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f5ee40035f0 con 0x55fdd7cdbb40 2024-09-12T02:08:51.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.096+0000 7f5ef8aba740 10 client.17547 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:08:51.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.096+0000 7f5ef8aba740 10 client.17547 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-12T02:08:51.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.096+0000 7f5ef8aba740 20 client.17547 populate_metadata read hostname 'smithi016' 2024-09-12T02:08:51.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.096+0000 7f5ef8aba740 1 --2- 172.21.15.16:0/2716389506 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55fdd7d63d00 0x55fdd7d66120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:51.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.096+0000 7f5ef8aba740 4 client.17547 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T02:08:51.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.096+0000 7f5ef8aba740 1 -- 172.21.15.16:0/2716389506 --> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fdd7c47d70 con 0x55fdd7d63d00 2024-09-12T02:08:51.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.096+0000 7f5ef4c7e640 1 --2- 172.21.15.16:0/2716389506 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55fdd7d63d00 0x55fdd7d66120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:51.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.097+0000 7f5ef4c7e640 1 --2- 172.21.15.16:0/2716389506 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55fdd7d63d00 0x55fdd7d66120 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=23ae3d9179dfe875 server_cookie=1bd11f808d923ad4 in_seq=0 out_seq=0 2024-09-12T02:08:51.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.097+0000 7f5ecf7fe640 10 client.17547 ms_handle_connect on v2:172.21.15.184:6836/1793393087 2024-09-12T02:08:51.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.097+0000 7f5ecf7fe640 1 -- 172.21.15.16:0/2716389506 <== mds.2 v2:172.21.15.184:6836/1793393087 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fdd7c47d70 con 0x55fdd7d63d00 2024-09-12T02:08:51.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.097+0000 7f5ecf7fe640 10 client.17547 handle_command_reply: tid=0 2024-09-12T02:08:51.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.097+0000 7f5ef8aba740 1 client.17547 shutdown 2024-09-12T02:08:51.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.097+0000 7f5ef8aba740 1 -- 172.21.15.16:0/2716389506 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fdd7cdbb40 msgr2=0x55fdd7cfbcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:51.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.097+0000 7f5ef8aba740 1 --2- 172.21.15.16:0/2716389506 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fdd7cdbb40 0x55fdd7cfbcf0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f5ee4005940 tx=0x7f5ee4031a60 comp rx=0 tx=0).stop 2024-09-12T02:08:51.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.097+0000 7f5ef8aba740 1 -- 172.21.15.16:0/2716389506 shutdown_connections 2024-09-12T02:08:51.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.097+0000 7f5ef8aba740 1 --2- 172.21.15.16:0/2716389506 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fdd7ce9780 0x55fdd7d61bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:51.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.097+0000 7f5ef8aba740 1 --2- 172.21.15.16:0/2716389506 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55fdd7d63d00 0x55fdd7d66120 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:51.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.097+0000 7f5ef8aba740 1 --2- 172.21.15.16:0/2716389506 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fdd7cdbb40 0x55fdd7cfbcf0 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:51.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.097+0000 7f5ef8aba740 1 --2- 172.21.15.16:0/2716389506 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fdd7cda5d0 0x55fdd7cfb7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:51.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.097+0000 7f5ef8aba740 1 -- 172.21.15.16:0/2716389506 >> 172.21.15.16:0/2716389506 conn(0x55fdd7ca3310 msgr2=0x55fdd7cd54e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:51.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.097+0000 7f5ecf7fe640 0 client.17547 ms_handle_reset on v2:172.21.15.184:6836/1793393087 2024-09-12T02:08:51.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.098+0000 7f5ef8aba740 1 -- 172.21.15.16:0/2716389506 shutdown_connections 2024-09-12T02:08:51.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.098+0000 7f5ef8aba740 1 -- 172.21.15.16:0/2716389506 wait complete. 2024-09-12T02:08:51.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.098+0000 7f5ef8aba740 20 client.17547 trim_cache size 0 max 16384 2024-09-12T02:08:51.103 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T02:08:51.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.098+0000 7f5ef66e1640 1 -- 172.21.15.16:0/4222304943 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f5ed80a2b80 msgr2=0x7f5ed80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:51.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.098+0000 7f5ef66e1640 1 --2- 172.21.15.16:0/4222304943 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f5ed80a2b80 0x7f5ed80a5040 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f5ee00a3b00 tx=0x7f5ee0097040 comp rx=0 tx=0).stop 2024-09-12T02:08:51.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.098+0000 7f5ef66e1640 1 -- 172.21.15.16:0/4222304943 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ef0102d30 msgr2=0x7f5ef0198600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:51.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.098+0000 7f5ef66e1640 1 --2- 172.21.15.16:0/4222304943 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ef0102d30 0x7f5ef0198600 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f5ee000d6d0 tx=0x7f5ee000db90 comp rx=0 tx=0).stop 2024-09-12T02:08:51.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.098+0000 7f5ef66e1640 1 -- 172.21.15.16:0/4222304943 shutdown_connections 2024-09-12T02:08:51.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.098+0000 7f5ef66e1640 1 --2- 172.21.15.16:0/4222304943 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f5ed80a2b80 0x7f5ed80a5040 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:51.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.098+0000 7f5ef66e1640 1 --2- 172.21.15.16:0/4222304943 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5ef01082a0 0x7f5ef019ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:51.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.098+0000 7f5ef66e1640 1 --2- 172.21.15.16:0/4222304943 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ef01036e0 0x7f5ef0198b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:51.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.098+0000 7f5ef66e1640 1 --2- 172.21.15.16:0/4222304943 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ef0102d30 0x7f5ef0198600 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:51.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.098+0000 7f5ef66e1640 1 -- 172.21.15.16:0/4222304943 >> 172.21.15.16:0/4222304943 conn(0x7f5ef00fc7d0 msgr2=0x7f5ef0105d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:51.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.099+0000 7f5ef66e1640 1 -- 172.21.15.16:0/4222304943 shutdown_connections 2024-09-12T02:08:51.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:51.099+0000 7f5ef66e1640 1 -- 172.21.15.16:0/4222304943 wait complete. 2024-09-12T02:08:51.896 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-12T02:08:51.910 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:08:52.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.370+0000 7f6b6a4be640 1 -- 172.21.15.16:0/893330897 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b64103740 msgr2=0x7f6b64107b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:52.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.370+0000 7f6b6a4be640 1 --2- 172.21.15.16:0/893330897 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b64103740 0x7f6b64107b80 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f6b58009930 tx=0x7f6b5802a420 comp rx=0 tx=0).stop 2024-09-12T02:08:52.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.370+0000 7f6b6a4be640 1 -- 172.21.15.16:0/893330897 shutdown_connections 2024-09-12T02:08:52.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.370+0000 7f6b6a4be640 1 --2- 172.21.15.16:0/893330897 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6b64108290 0x7f6b6410ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:52.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.370+0000 7f6b6a4be640 1 --2- 172.21.15.16:0/893330897 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b64103740 0x7f6b64107b80 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:52.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.370+0000 7f6b6a4be640 1 --2- 172.21.15.16:0/893330897 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b64102d90 0x7f6b64103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:52.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.370+0000 7f6b6a4be640 1 -- 172.21.15.16:0/893330897 >> 172.21.15.16:0/893330897 conn(0x7f6b640fc7d0 msgr2=0x7f6b640febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:52.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.370+0000 7f6b6a4be640 1 -- 172.21.15.16:0/893330897 shutdown_connections 2024-09-12T02:08:52.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.370+0000 7f6b6a4be640 1 -- 172.21.15.16:0/893330897 wait complete. 2024-09-12T02:08:52.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.371+0000 7f6b6a4be640 1 Processor -- start 2024-09-12T02:08:52.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.371+0000 7f6b6a4be640 1 -- start start 2024-09-12T02:08:52.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.371+0000 7f6b6a4be640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b64102d90 0x7f6b641009f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:52.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.371+0000 7f6b6a4be640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b64103740 0x7f6b64100f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:52.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.371+0000 7f6b6a4be640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6b64108290 0x7f6b64101470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:52.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.371+0000 7f6b6a4be640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6b64112090 con 0x7f6b64102d90 2024-09-12T02:08:52.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.371+0000 7f6b6a4be640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6b64111f10 con 0x7f6b64103740 2024-09-12T02:08:52.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.371+0000 7f6b6a4be640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f6b64112210 con 0x7f6b64108290 2024-09-12T02:08:52.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.372+0000 7f6b63fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b64102d90 0x7f6b641009f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:52.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.372+0000 7f6b63fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b64102d90 0x7f6b641009f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33560/0 (socket says 172.21.15.16:33560) 2024-09-12T02:08:52.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.372+0000 7f6b63fff640 1 -- 172.21.15.16:0/1976507905 learned_addr learned my addr 172.21.15.16:0/1976507905 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:08:52.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.372+0000 7f6b637fe640 1 --2- 172.21.15.16:0/1976507905 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b64103740 0x7f6b64100f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:52.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.372+0000 7f6b68a5b640 1 --2- 172.21.15.16:0/1976507905 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6b64108290 0x7f6b64101470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:52.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.372+0000 7f6b63fff640 1 -- 172.21.15.16:0/1976507905 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6b64108290 msgr2=0x7f6b64101470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:52.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.372+0000 7f6b63fff640 1 --2- 172.21.15.16:0/1976507905 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6b64108290 0x7f6b64101470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:52.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.372+0000 7f6b63fff640 1 -- 172.21.15.16:0/1976507905 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b64103740 msgr2=0x7f6b64100f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:52.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.372+0000 7f6b63fff640 1 --2- 172.21.15.16:0/1976507905 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b64103740 0x7f6b64100f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:52.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.372+0000 7f6b63fff640 1 -- 172.21.15.16:0/1976507905 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b641a6a50 con 0x7f6b64102d90 2024-09-12T02:08:52.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.372+0000 7f6b63fff640 1 --2- 172.21.15.16:0/1976507905 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b64102d90 0x7f6b641009f0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f6b5400e9f0 tx=0x7f6b5400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:52.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.372+0000 7f6b617fa640 1 -- 172.21.15.16:0/1976507905 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b5400cd80 con 0x7f6b64102d90 2024-09-12T02:08:52.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.372+0000 7f6b6a4be640 1 -- 172.21.15.16:0/1976507905 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b641a6d40 con 0x7f6b64102d90 2024-09-12T02:08:52.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.372+0000 7f6b617fa640 1 -- 172.21.15.16:0/1976507905 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b54010910 con 0x7f6b64102d90 2024-09-12T02:08:52.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.372+0000 7f6b617fa640 1 -- 172.21.15.16:0/1976507905 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b540188f0 con 0x7f6b64102d90 2024-09-12T02:08:52.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.373+0000 7f6b6a4be640 1 -- 172.21.15.16:0/1976507905 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b641a7240 con 0x7f6b64102d90 2024-09-12T02:08:52.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.376+0000 7f6b617fa640 1 -- 172.21.15.16:0/1976507905 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6b54018bd0 con 0x7f6b64102d90 2024-09-12T02:08:52.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.376+0000 7f6b6c897740 1 Processor -- start 2024-09-12T02:08:52.376 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:08:52 smithi016 ceph-mon[30947]: pgmap v543: 97 pgs: 97 active+clean; 3.7 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 3.2 KiB/s wr, 11 op/s 2024-09-12T02:08:52.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.376+0000 7f6b6c897740 1 -- start start 2024-09-12T02:08:52.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.376+0000 7f6b6c897740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee307fd870 0x55ee307fdc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:52.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.377+0000 7f6b6c897740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ee307fe190 0x55ee30808510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:52.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.377+0000 7f6b6c897740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ee30808a50 0x55ee3080ae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:52.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.377+0000 7f6b6c897740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55ee3075a620 con 0x55ee307fd870 2024-09-12T02:08:52.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.377+0000 7f6b6c897740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55ee30701e50 con 0x55ee307fe190 2024-09-12T02:08:52.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.377+0000 7f6b6c897740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55ee307565e0 con 0x55ee30808a50 2024-09-12T02:08:52.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.377+0000 7f6b68a5b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee307fd870 0x55ee307fdc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:52.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.377+0000 7f6b68a5b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee307fd870 0x55ee307fdc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33572/0 (socket says 172.21.15.16:33572) 2024-09-12T02:08:52.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.377+0000 7f6b68a5b640 1 -- 172.21.15.16:0/3361698760 learned_addr learned my addr 172.21.15.16:0/3361698760 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:08:52.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.377+0000 7f6b63fff640 1 --2- 172.21.15.16:0/3361698760 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ee307fe190 0x55ee30808510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:52.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.377+0000 7f6b617fa640 1 --2- 172.21.15.16:0/1976507905 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f6b4c0a2c00 0x7f6b4c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:52.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.377+0000 7f6b637fe640 1 --2- 172.21.15.16:0/3361698760 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ee30808a50 0x55ee3080ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:52.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.377+0000 7f6b68a5b640 1 -- 172.21.15.16:0/3361698760 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ee30808a50 msgr2=0x55ee3080ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:52.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.377+0000 7f6b68a5b640 1 --2- 172.21.15.16:0/3361698760 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ee30808a50 0x55ee3080ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:52.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.377+0000 7f6b68a5b640 1 -- 172.21.15.16:0/3361698760 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ee307fe190 msgr2=0x55ee30808510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:52.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.377+0000 7f6b68a5b640 1 --2- 172.21.15.16:0/3361698760 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ee307fe190 0x55ee30808510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:52.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.377+0000 7f6b68a5b640 1 -- 172.21.15.16:0/3361698760 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee306f2d40 con 0x55ee307fd870 2024-09-12T02:08:52.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.377+0000 7f6b617fa640 1 -- 172.21.15.16:0/1976507905 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f6b540cf2c0 con 0x7f6b64102d90 2024-09-12T02:08:52.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.377+0000 7f6b68a5b640 1 --2- 172.21.15.16:0/1976507905 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f6b4c0a2c00 0x7f6b4c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:52.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.378+0000 7f6b68a5b640 1 --2- 172.21.15.16:0/3361698760 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee307fd870 0x55ee307fdc50 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c011590 tx=0x7f6b5c011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df36815b69f6ac66 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:52.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.378+0000 7f6b4a7fc640 1 -- 172.21.15.16:0/3361698760 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b5c017050 con 0x55ee307fd870 2024-09-12T02:08:52.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.378+0000 7f6b4a7fc640 1 -- 172.21.15.16:0/3361698760 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b5c01bb90 con 0x55ee307fd870 2024-09-12T02:08:52.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.378+0000 7f6b4a7fc640 1 -- 172.21.15.16:0/3361698760 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b5c00e3b0 con 0x55ee307fd870 2024-09-12T02:08:52.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.379+0000 7f6b6c897740 1 -- 172.21.15.16:0/3361698760 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee307fd870 msgr2=0x55ee307fdc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:52.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.379+0000 7f6b6c897740 1 --2- 172.21.15.16:0/3361698760 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee307fd870 0x55ee307fdc50 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c011590 tx=0x7f6b5c011a50 comp rx=0 tx=0).stop 2024-09-12T02:08:52.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.379+0000 7f6b68a5b640 1 --2- 172.21.15.16:0/1976507905 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f6b4c0a2c00 0x7f6b4c0a50c0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c022000 tx=0x7f6b5c023000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:52.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.379+0000 7f6b6c897740 1 -- 172.21.15.16:0/3361698760 shutdown_connections 2024-09-12T02:08:52.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.379+0000 7f6b6c897740 1 --2- 172.21.15.16:0/3361698760 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ee30808a50 0x55ee3080ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:52.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.379+0000 7f6b6c897740 1 --2- 172.21.15.16:0/3361698760 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ee307fe190 0x55ee30808510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:52.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.379+0000 7f6b6c897740 1 --2- 172.21.15.16:0/3361698760 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee307fd870 0x55ee307fdc50 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:52.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.379+0000 7f6b6c897740 1 -- 172.21.15.16:0/3361698760 >> 172.21.15.16:0/3361698760 conn(0x55ee307f9e00 msgr2=0x55ee307fa1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:52.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.379+0000 7f6b6c897740 1 -- 172.21.15.16:0/3361698760 shutdown_connections 2024-09-12T02:08:52.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.379+0000 7f6b6c897740 1 -- 172.21.15.16:0/3361698760 wait complete. 2024-09-12T02:08:52.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.380+0000 7f6b6c897740 1 -- start start 2024-09-12T02:08:52.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.380+0000 7f6b6c897740 1 Processor -- start 2024-09-12T02:08:52.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.380+0000 7f6b6c897740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ee307fd870 0x55ee308058e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:52.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.380+0000 7f6b6c897740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee307fe190 0x55ee30805e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:52.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.380+0000 7f6b6c897740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ee30808a50 0x55ee30806360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:52.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.380+0000 7f6b6c897740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55ee3080b620 con 0x55ee307fe190 2024-09-12T02:08:52.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.380+0000 7f6b6c897740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55ee306e2f10 con 0x55ee30808a50 2024-09-12T02:08:52.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.380+0000 7f6b6c897740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55ee3080b7a0 con 0x55ee307fd870 2024-09-12T02:08:52.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.380+0000 7f6b6c897740 10 client.0 _ll_register_callbacks cb 0x55ee30706db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:08:52.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.380+0000 7f6b637fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee307fe190 0x55ee30805e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:52.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.380+0000 7f6b637fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee307fe190 0x55ee30805e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33578/0 (socket says 172.21.15.16:33578) 2024-09-12T02:08:52.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.380+0000 7f6b637fe640 1 -- 172.21.15.16:0/3018562526 learned_addr learned my addr 172.21.15.16:0/3018562526 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:08:52.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.380+0000 7f6b68a5b640 1 --2- 172.21.15.16:0/3018562526 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ee30808a50 0x55ee30806360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:52.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.381+0000 7f6b63fff640 1 --2- 172.21.15.16:0/3018562526 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ee307fd870 0x55ee308058e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:52.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.381+0000 7f6b637fe640 1 -- 172.21.15.16:0/3018562526 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ee307fd870 msgr2=0x55ee308058e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:52.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.381+0000 7f6b637fe640 1 --2- 172.21.15.16:0/3018562526 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ee307fd870 0x55ee308058e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:52.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.381+0000 7f6b637fe640 1 -- 172.21.15.16:0/3018562526 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ee30808a50 msgr2=0x55ee30806360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:52.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.381+0000 7f6b637fe640 1 --2- 172.21.15.16:0/3018562526 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ee30808a50 0x55ee30806360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:52.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.381+0000 7f6b637fe640 1 -- 172.21.15.16:0/3018562526 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee30469580 con 0x55ee307fe190 2024-09-12T02:08:52.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.381+0000 7f6b637fe640 1 --2- 172.21.15.16:0/3018562526 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee307fe190 0x55ee30805e20 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f6b58009900 tx=0x7f6b58009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9234c7bfc57b5f04 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:52.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.381+0000 7f6b437fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:08:52.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.381+0000 7f6b437fe640 1 -- 172.21.15.16:0/3018562526 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b58035040 con 0x55ee307fe190 2024-09-12T02:08:52.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.381+0000 7f6b437fe640 1 -- 172.21.15.16:0/3018562526 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f6b58004040 con 0x55ee307fe190 2024-09-12T02:08:52.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.381+0000 7f6b6c897740 1 -- 172.21.15.16:0/3018562526 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b5c01bb90 con 0x55ee307fe190 2024-09-12T02:08:52.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.382+0000 7f6b437fe640 1 -- 172.21.15.16:0/3018562526 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6b58005d50 con 0x55ee307fe190 2024-09-12T02:08:52.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.382+0000 7f6b6c897740 10 client.17559 fetch_fsmap learned FSMap version 25 2024-09-12T02:08:52.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.382+0000 7f6b6c897740 1 -- 172.21.15.16:0/3018562526 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ee30533960 con 0x55ee307fe190 2024-09-12T02:08:52.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.382+0000 7f6b437fe640 1 -- 172.21.15.16:0/3018562526 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b5c01bb90 con 0x55ee307fe190 2024-09-12T02:08:52.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.382+0000 7f6b437fe640 1 -- 172.21.15.16:0/3018562526 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f6b58045480 con 0x55ee307fe190 2024-09-12T02:08:52.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.382+0000 7f6b6c897740 10 client.17559 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:08:52.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.382+0000 7f6b6c897740 10 client.17559 resolve_mds: validated gid 24482 aka daemon mds.b 2024-09-12T02:08:52.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.382+0000 7f6b6c897740 20 client.17559 populate_metadata read hostname 'smithi016' 2024-09-12T02:08:52.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.382+0000 7f6b6c897740 1 --2- 172.21.15.16:0/3018562526 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55ee30818550 0x55ee3081a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:52.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.382+0000 7f6b6c897740 4 client.17559 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:08:52.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.382+0000 7f6b6c897740 1 -- 172.21.15.16:0/3018562526 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ee30767b90 con 0x55ee30818550 2024-09-12T02:08:52.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.383+0000 7f6b63fff640 1 --2- 172.21.15.16:0/3018562526 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55ee30818550 0x55ee3081a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:52.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.383+0000 7f6b63fff640 1 --2- 172.21.15.16:0/3018562526 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55ee30818550 0x55ee3081a9b0 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=995678be92cacb65 server_cookie=18b645c46a64134d in_seq=0 out_seq=0 2024-09-12T02:08:52.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.383+0000 7f6b437fe640 10 client.17559 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:08:52.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.384+0000 7f6b437fe640 1 -- 172.21.15.16:0/3018562526 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ee30767b90 con 0x55ee30818550 2024-09-12T02:08:52.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.384+0000 7f6b437fe640 10 client.17559 handle_command_reply: tid=0 2024-09-12T02:08:52.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.384+0000 7f6b6c897740 1 client.17559 shutdown 2024-09-12T02:08:52.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.384+0000 7f6b6c897740 1 -- 172.21.15.16:0/3018562526 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee307fe190 msgr2=0x55ee30805e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:52.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.384+0000 7f6b6c897740 1 --2- 172.21.15.16:0/3018562526 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee307fe190 0x55ee30805e20 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f6b58009900 tx=0x7f6b58009680 comp rx=0 tx=0).stop 2024-09-12T02:08:52.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.384+0000 7f6b6c897740 1 -- 172.21.15.16:0/3018562526 shutdown_connections 2024-09-12T02:08:52.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.384+0000 7f6b6c897740 1 --2- 172.21.15.16:0/3018562526 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55ee30818550 0x55ee3081a9b0 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:52.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.384+0000 7f6b6c897740 1 --2- 172.21.15.16:0/3018562526 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ee30808a50 0x55ee30806360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:52.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.384+0000 7f6b6c897740 1 --2- 172.21.15.16:0/3018562526 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee307fe190 0x55ee30805e20 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:52.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.384+0000 7f6b6c897740 1 --2- 172.21.15.16:0/3018562526 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ee307fd870 0x55ee308058e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:52.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.384+0000 7f6b6c897740 1 -- 172.21.15.16:0/3018562526 >> 172.21.15.16:0/3018562526 conn(0x55ee307f9e00 msgr2=0x55ee307f0750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:52.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.384+0000 7f6b437fe640 0 client.17559 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:08:52.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.384+0000 7f6b6c897740 1 -- 172.21.15.16:0/3018562526 shutdown_connections 2024-09-12T02:08:52.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.384+0000 7f6b6c897740 1 -- 172.21.15.16:0/3018562526 wait complete. 2024-09-12T02:08:52.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.385+0000 7f6b6c897740 20 client.17559 trim_cache size 0 max 16384 2024-09-12T02:08:52.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.403+0000 7f6b6c897740 1 Processor -- start 2024-09-12T02:08:52.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.403+0000 7f6b6c897740 1 -- start start 2024-09-12T02:08:52.404 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:08:52.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.404+0000 7f6b6c897740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ee307fd870 0x55ee308002d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:52.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.404+0000 7f6b6c897740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee30808a50 0x55ee308041b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:52.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.404+0000 7f6b6c897740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ee307f0640 0x55ee30820280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:52.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.404+0000 7f6b6c897740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6b641adc20 con 0x55ee30808a50 2024-09-12T02:08:52.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.404+0000 7f6b6c897740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6b641ab4f0 con 0x55ee307f0640 2024-09-12T02:08:52.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.404+0000 7f6b6c897740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f6b641a9380 con 0x55ee307fd870 2024-09-12T02:08:52.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.404+0000 7f6b637fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee30808a50 0x55ee308041b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:52.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.404+0000 7f6b637fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee30808a50 0x55ee308041b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33586/0 (socket says 172.21.15.16:33586) 2024-09-12T02:08:52.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.404+0000 7f6b637fe640 1 -- 172.21.15.16:0/1101385789 learned_addr learned my addr 172.21.15.16:0/1101385789 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:08:52.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.404+0000 7f6b63fff640 1 --2- 172.21.15.16:0/1101385789 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ee307fd870 0x55ee308002d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:52.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.404+0000 7f6b68a5b640 1 --2- 172.21.15.16:0/1101385789 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ee307f0640 0x55ee30820280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:52.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.404+0000 7f6b637fe640 1 -- 172.21.15.16:0/1101385789 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ee307fd870 msgr2=0x55ee308002d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:52.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.404+0000 7f6b637fe640 1 --2- 172.21.15.16:0/1101385789 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ee307fd870 0x55ee308002d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:52.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.404+0000 7f6b637fe640 1 -- 172.21.15.16:0/1101385789 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ee307f0640 msgr2=0x55ee30820280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:52.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.404+0000 7f6b637fe640 1 --2- 172.21.15.16:0/1101385789 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ee307f0640 0x55ee30820280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:52.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.404+0000 7f6b637fe640 1 -- 172.21.15.16:0/1101385789 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee307f9c30 con 0x55ee30808a50 2024-09-12T02:08:52.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.405+0000 7f6b637fe640 1 --2- 172.21.15.16:0/1101385789 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee30808a50 0x55ee308041b0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f6b58045a40 tx=0x7f6b58000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbf0ebc7dc4ef1e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:52.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.405+0000 7f6b49ffb640 1 -- 172.21.15.16:0/1101385789 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b58035070 con 0x55ee30808a50 2024-09-12T02:08:52.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.405+0000 7f6b49ffb640 1 -- 172.21.15.16:0/1101385789 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b58045cc0 con 0x55ee30808a50 2024-09-12T02:08:52.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.405+0000 7f6b49ffb640 1 -- 172.21.15.16:0/1101385789 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b580049d0 con 0x55ee30808a50 2024-09-12T02:08:52.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.406+0000 7f6b6c897740 1 -- 172.21.15.16:0/1101385789 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee30808a50 msgr2=0x55ee308041b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:52.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.406+0000 7f6b6c897740 1 --2- 172.21.15.16:0/1101385789 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee30808a50 0x55ee308041b0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f6b58045a40 tx=0x7f6b58000bc0 comp rx=0 tx=0).stop 2024-09-12T02:08:52.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.406+0000 7f6b6c897740 1 -- 172.21.15.16:0/1101385789 shutdown_connections 2024-09-12T02:08:52.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.406+0000 7f6b6c897740 1 --2- 172.21.15.16:0/1101385789 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ee307f0640 0x55ee30820280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:52.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.406+0000 7f6b6c897740 1 --2- 172.21.15.16:0/1101385789 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee30808a50 0x55ee308041b0 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:52.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.406+0000 7f6b6c897740 1 --2- 172.21.15.16:0/1101385789 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ee307fd870 0x55ee308002d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:52.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.406+0000 7f6b6c897740 1 -- 172.21.15.16:0/1101385789 >> 172.21.15.16:0/1101385789 conn(0x55ee307f9e00 msgr2=0x55ee307eff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:52.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.406+0000 7f6b6c897740 1 -- 172.21.15.16:0/1101385789 shutdown_connections 2024-09-12T02:08:52.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.406+0000 7f6b6c897740 1 -- 172.21.15.16:0/1101385789 wait complete. 2024-09-12T02:08:52.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.406+0000 7f6b6c897740 1 -- start start 2024-09-12T02:08:52.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.407+0000 7f6b6c897740 1 Processor -- start 2024-09-12T02:08:52.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.407+0000 7f6b6c897740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ee307f0640 0x55ee30811250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:52.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.407+0000 7f6b6c897740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ee307fd870 0x55ee3080d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:52.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.407+0000 7f6b6c897740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee30808a50 0x55ee3080db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:52.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.407+0000 7f6b6c897740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6b641a99d0 con 0x55ee30808a50 2024-09-12T02:08:52.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.407+0000 7f6b6c897740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6b641aae20 con 0x55ee307fd870 2024-09-12T02:08:52.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.407+0000 7f6b6c897740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f6b641aa080 con 0x55ee307f0640 2024-09-12T02:08:52.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.407+0000 7f6b6c897740 10 client.0 _ll_register_callbacks cb 0x7f6b58009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:08:52.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.407+0000 7f6b68a5b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee30808a50 0x55ee3080db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:52.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.407+0000 7f6b68a5b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee30808a50 0x55ee3080db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33588/0 (socket says 172.21.15.16:33588) 2024-09-12T02:08:52.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.407+0000 7f6b68a5b640 1 -- 172.21.15.16:0/2438482068 learned_addr learned my addr 172.21.15.16:0/2438482068 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:08:52.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.407+0000 7f6b637fe640 1 --2- 172.21.15.16:0/2438482068 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ee307fd870 0x55ee3080d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:52.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.407+0000 7f6b63fff640 1 --2- 172.21.15.16:0/2438482068 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ee307f0640 0x55ee30811250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:52.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.407+0000 7f6b68a5b640 1 -- 172.21.15.16:0/2438482068 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ee307f0640 msgr2=0x55ee30811250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:52.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.407+0000 7f6b68a5b640 1 --2- 172.21.15.16:0/2438482068 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ee307f0640 0x55ee30811250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:52.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.407+0000 7f6b68a5b640 1 -- 172.21.15.16:0/2438482068 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ee307fd870 msgr2=0x55ee3080d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:52.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.407+0000 7f6b68a5b640 1 --2- 172.21.15.16:0/2438482068 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ee307fd870 0x55ee3080d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:52.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.407+0000 7f6b68a5b640 1 -- 172.21.15.16:0/2438482068 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee307faa90 con 0x55ee30808a50 2024-09-12T02:08:52.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.407+0000 7f6b637fe640 1 --2- 172.21.15.16:0/2438482068 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ee307fd870 0x55ee3080d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:08:52.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.408+0000 7f6b68a5b640 1 --2- 172.21.15.16:0/2438482068 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee30808a50 0x55ee3080db80 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c005fd0 tx=0x7f6b5c025a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be3ec68e1dc32107 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:52.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.408+0000 7f6b437fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:08:52.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.408+0000 7f6b437fe640 1 -- 172.21.15.16:0/2438482068 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b5c035020 con 0x55ee30808a50 2024-09-12T02:08:52.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.408+0000 7f6b437fe640 1 -- 172.21.15.16:0/2438482068 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f6b5c004bc0 con 0x55ee30808a50 2024-09-12T02:08:52.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.408+0000 7f6b6c897740 1 -- 172.21.15.16:0/2438482068 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b58045cc0 con 0x55ee30808a50 2024-09-12T02:08:52.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.408+0000 7f6b437fe640 1 -- 172.21.15.16:0/2438482068 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6b5c017020 con 0x55ee30808a50 2024-09-12T02:08:52.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.408+0000 7f6b6c897740 10 client.17565 fetch_fsmap learned FSMap version 25 2024-09-12T02:08:52.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.408+0000 7f6b6c897740 1 -- 172.21.15.16:0/2438482068 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ee308841e0 con 0x55ee30808a50 2024-09-12T02:08:52.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.408+0000 7f6b437fe640 1 -- 172.21.15.16:0/2438482068 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b58045cc0 con 0x55ee30808a50 2024-09-12T02:08:52.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.409+0000 7f6b437fe640 1 -- 172.21.15.16:0/2438482068 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f6b5c00e6e0 con 0x55ee30808a50 2024-09-12T02:08:52.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.409+0000 7f6b6c897740 10 client.17565 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:08:52.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.409+0000 7f6b6c897740 10 client.17565 resolve_mds: validated gid 24482 aka daemon mds.b 2024-09-12T02:08:52.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.409+0000 7f6b6c897740 20 client.17565 populate_metadata read hostname 'smithi016' 2024-09-12T02:08:52.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.409+0000 7f6b6c897740 1 --2- 172.21.15.16:0/2438482068 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55ee308846d0 0x55ee30886ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:52.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.409+0000 7f6b6c897740 4 client.17565 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "e05b9d01", "release": true, "await": true}] 2024-09-12T02:08:52.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.409+0000 7f6b6c897740 1 -- 172.21.15.16:0/2438482068 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "quiesce db", "set_id": "e05b9d01", "release": true, "await": true}) -- 0x55ee30801500 con 0x55ee308846d0 2024-09-12T02:08:52.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.409+0000 7f6b63fff640 1 --2- 172.21.15.16:0/2438482068 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55ee308846d0 0x55ee30886ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:52.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.409+0000 7f6b63fff640 1 --2- 172.21.15.16:0/2438482068 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55ee308846d0 0x55ee30886ac0 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=361ca4aecc8de264 server_cookie=944eaf4aa2742eb4 in_seq=0 out_seq=0 2024-09-12T02:08:52.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.409+0000 7f6b437fe640 10 client.17565 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:08:52.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.412+0000 7f6b437fe640 1 -- 172.21.15.16:0/2438482068 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55ee30801500 con 0x55ee308846d0 2024-09-12T02:08:52.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.412+0000 7f6b437fe640 10 client.17565 handle_command_reply: tid=0 2024-09-12T02:08:52.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.412+0000 7f6b6c897740 1 client.17565 shutdown 2024-09-12T02:08:52.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.412+0000 7f6b6c897740 1 -- 172.21.15.16:0/2438482068 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee30808a50 msgr2=0x55ee3080db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:52.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.412+0000 7f6b6c897740 1 --2- 172.21.15.16:0/2438482068 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee30808a50 0x55ee3080db80 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c005fd0 tx=0x7f6b5c025a20 comp rx=0 tx=0).stop 2024-09-12T02:08:52.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.412+0000 7f6b6c897740 1 -- 172.21.15.16:0/2438482068 shutdown_connections 2024-09-12T02:08:52.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.412+0000 7f6b6c897740 1 --2- 172.21.15.16:0/2438482068 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55ee308846d0 0x55ee30886ac0 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:52.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.412+0000 7f6b6c897740 1 --2- 172.21.15.16:0/2438482068 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee30808a50 0x55ee3080db80 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:52.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.412+0000 7f6b6c897740 1 --2- 172.21.15.16:0/2438482068 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ee307fd870 0x55ee3080d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:52.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.412+0000 7f6b437fe640 0 client.17565 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:08:52.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.412+0000 7f6b6c897740 1 --2- 172.21.15.16:0/2438482068 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ee307f0640 0x55ee30811250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:52.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.412+0000 7f6b6c897740 1 -- 172.21.15.16:0/2438482068 >> 172.21.15.16:0/2438482068 conn(0x55ee307f9e00 msgr2=0x55ee30805880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:52.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.412+0000 7f6b6c897740 1 -- 172.21.15.16:0/2438482068 shutdown_connections 2024-09-12T02:08:52.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.412+0000 7f6b6c897740 1 -- 172.21.15.16:0/2438482068 wait complete. 2024-09-12T02:08:52.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.412+0000 7f6b6c897740 20 client.17565 trim_cache size 0 max 16384 2024-09-12T02:08:52.418 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T02:08:52.418 INFO:teuthology.orchestra.run.smithi016.stdout: "epoch": 25, 2024-09-12T02:08:52.418 INFO:teuthology.orchestra.run.smithi016.stdout: "leader": 24482, 2024-09-12T02:08:52.418 INFO:teuthology.orchestra.run.smithi016.stdout: "set_version": 28, 2024-09-12T02:08:52.418 INFO:teuthology.orchestra.run.smithi016.stdout: "sets": { 2024-09-12T02:08:52.418 INFO:teuthology.orchestra.run.smithi016.stdout: "e05b9d01": { 2024-09-12T02:08:52.418 INFO:teuthology.orchestra.run.smithi016.stdout: "version": 28, 2024-09-12T02:08:52.419 INFO:teuthology.orchestra.run.smithi016.stdout: "age_ref": 0.0, 2024-09-12T02:08:52.419 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-12T02:08:52.419 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RELEASED", 2024-09-12T02:08:52.419 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-12T02:08:52.419 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:08:52.419 INFO:teuthology.orchestra.run.smithi016.stdout: "timeout": 90.0, 2024-09-12T02:08:52.419 INFO:teuthology.orchestra.run.smithi016.stdout: "expiration": 136.6, 2024-09-12T02:08:52.419 INFO:teuthology.orchestra.run.smithi016.stdout: "members": { 2024-09-12T02:08:52.419 INFO:teuthology.orchestra.run.smithi016.stdout: "file:/": { 2024-09-12T02:08:52.419 INFO:teuthology.orchestra.run.smithi016.stdout: "excluded": false, 2024-09-12T02:08:52.419 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-12T02:08:52.419 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RELEASED", 2024-09-12T02:08:52.419 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-12T02:08:52.419 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:08:52.420 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:08:52.420 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:08:52.420 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:08:52.420 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:08:52.420 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T02:08:52.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.413+0000 7f6b6a4be640 1 -- 172.21.15.16:0/1976507905 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f6b4c0a2c00 msgr2=0x7f6b4c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:52.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.413+0000 7f6b6a4be640 1 --2- 172.21.15.16:0/1976507905 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f6b4c0a2c00 0x7f6b4c0a50c0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c022000 tx=0x7f6b5c023000 comp rx=0 tx=0).stop 2024-09-12T02:08:52.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.413+0000 7f6b6a4be640 1 -- 172.21.15.16:0/1976507905 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b64102d90 msgr2=0x7f6b641009f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:52.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.413+0000 7f6b6a4be640 1 --2- 172.21.15.16:0/1976507905 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b64102d90 0x7f6b641009f0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f6b5400e9f0 tx=0x7f6b5400eeb0 comp rx=0 tx=0).stop 2024-09-12T02:08:52.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.413+0000 7f6b6a4be640 1 -- 172.21.15.16:0/1976507905 shutdown_connections 2024-09-12T02:08:52.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.413+0000 7f6b6a4be640 1 --2- 172.21.15.16:0/1976507905 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f6b4c0a2c00 0x7f6b4c0a50c0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:52.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.413+0000 7f6b6a4be640 1 --2- 172.21.15.16:0/1976507905 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6b64108290 0x7f6b64101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:52.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.413+0000 7f6b6a4be640 1 --2- 172.21.15.16:0/1976507905 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b64103740 0x7f6b64100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:52.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.413+0000 7f6b6a4be640 1 --2- 172.21.15.16:0/1976507905 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b64102d90 0x7f6b641009f0 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:52.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.413+0000 7f6b6a4be640 1 -- 172.21.15.16:0/1976507905 >> 172.21.15.16:0/1976507905 conn(0x7f6b640fc7d0 msgr2=0x7f6b64106370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:52.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.413+0000 7f6b6a4be640 1 -- 172.21.15.16:0/1976507905 shutdown_connections 2024-09-12T02:08:52.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:52.413+0000 7f6b6a4be640 1 -- 172.21.15.16:0/1976507905 wait complete. 2024-09-12T02:08:52.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:08:52 smithi184 ceph-mon[35593]: pgmap v543: 97 pgs: 97 active+clean; 3.7 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 3.2 KiB/s wr, 11 op/s 2024-09-12T02:08:52.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:08:52 smithi033 ceph-mon[35332]: pgmap v543: 97 pgs: 97 active+clean; 3.7 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 3.2 KiB/s wr, 11 op/s 2024-09-12T02:08:53.089 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'e05b9d01', total seconds elapsed: 21.3 2024-09-12T02:08:53.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.624+0000 7f96c4f03640 1 -- 172.21.15.16:0/3781416119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96c0101420 msgr2=0x7f96c010d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:53.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.624+0000 7f96c4f03640 1 --2- 172.21.15.16:0/3781416119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96c0101420 0x7f96c010d950 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f96b8009cd0 tx=0x7f96b802a8e0 comp rx=0 tx=0).stop 2024-09-12T02:08:53.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.624+0000 7f96c4f03640 1 -- 172.21.15.16:0/3781416119 shutdown_connections 2024-09-12T02:08:53.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.624+0000 7f96c4f03640 1 --2- 172.21.15.16:0/3781416119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96c0101420 0x7f96c010d950 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:53.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.624+0000 7f96c4f03640 1 --2- 172.21.15.16:0/3781416119 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f96c0100a80 0x7f96c0100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:53.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.624+0000 7f96c4f03640 1 --2- 172.21.15.16:0/3781416119 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96c0106a80 0x7f96c0106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:53.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.624+0000 7f96c4f03640 1 -- 172.21.15.16:0/3781416119 >> 172.21.15.16:0/3781416119 conn(0x7f96c00fc7a0 msgr2=0x7f96c00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:53.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.625+0000 7f96c4f03640 1 -- 172.21.15.16:0/3781416119 shutdown_connections 2024-09-12T02:08:53.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.625+0000 7f96c4f03640 1 -- 172.21.15.16:0/3781416119 wait complete. 2024-09-12T02:08:53.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.625+0000 7f96c4f03640 1 Processor -- start 2024-09-12T02:08:53.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.626+0000 7f96c4f03640 1 -- start start 2024-09-12T02:08:53.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.626+0000 7f96c4f03640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96c0100a80 0x7f96c01992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:53.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.626+0000 7f96c4f03640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f96c0101420 0x7f96c01997f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:53.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.626+0000 7f96c4f03640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96c0106a80 0x7f96c01a4330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:53.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.626+0000 7f96c4f03640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f96c0110e80 con 0x7f96c0106a80 2024-09-12T02:08:53.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.626+0000 7f96c4f03640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f96c0110d00 con 0x7f96c0100a80 2024-09-12T02:08:53.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.626+0000 7f96c4f03640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f96c0111000 con 0x7f96c0101420 2024-09-12T02:08:53.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.626+0000 7f96bed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96c0106a80 0x7f96c01a4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:53.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.626+0000 7f96bed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96c0106a80 0x7f96c01a4330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33604/0 (socket says 172.21.15.16:33604) 2024-09-12T02:08:53.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.626+0000 7f96bed9d640 1 -- 172.21.15.16:0/3615656256 learned_addr learned my addr 172.21.15.16:0/3615656256 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:08:53.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.626+0000 7f96be59c640 1 --2- 172.21.15.16:0/3615656256 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96c0100a80 0x7f96c01992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:53.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.626+0000 7f96bdd9b640 1 --2- 172.21.15.16:0/3615656256 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f96c0101420 0x7f96c01997f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:53.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.626+0000 7f96bed9d640 1 -- 172.21.15.16:0/3615656256 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f96c0101420 msgr2=0x7f96c01997f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:53.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.626+0000 7f96bed9d640 1 --2- 172.21.15.16:0/3615656256 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f96c0101420 0x7f96c01997f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:53.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.626+0000 7f96bed9d640 1 -- 172.21.15.16:0/3615656256 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96c0100a80 msgr2=0x7f96c01992b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:53.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.626+0000 7f96bed9d640 1 --2- 172.21.15.16:0/3615656256 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96c0100a80 0x7f96c01992b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:53.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.626+0000 7f96bed9d640 1 -- 172.21.15.16:0/3615656256 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96c01a4870 con 0x7f96c0106a80 2024-09-12T02:08:53.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.627+0000 7f96bed9d640 1 --2- 172.21.15.16:0/3615656256 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96c0106a80 0x7f96c01a4330 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f96b8005cc0 tx=0x7f96b8003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:53.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.627+0000 7f96af7fe640 1 -- 172.21.15.16:0/3615656256 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96b8004040 con 0x7f96c0106a80 2024-09-12T02:08:53.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.627+0000 7f96af7fe640 1 -- 172.21.15.16:0/3615656256 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96b80046c0 con 0x7f96c0106a80 2024-09-12T02:08:53.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.627+0000 7f96c4f03640 1 -- 172.21.15.16:0/3615656256 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96c01a4a40 con 0x7f96c0106a80 2024-09-12T02:08:53.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.627+0000 7f96af7fe640 1 -- 172.21.15.16:0/3615656256 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96b8046480 con 0x7f96c0106a80 2024-09-12T02:08:53.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.627+0000 7f96c4f03640 1 -- 172.21.15.16:0/3615656256 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96c01a4f20 con 0x7f96c0106a80 2024-09-12T02:08:53.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.629+0000 7f96af7fe640 1 -- 172.21.15.16:0/3615656256 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f96b80305a0 con 0x7f96c0106a80 2024-09-12T02:08:53.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.630+0000 7f96c72dc740 1 Processor -- start 2024-09-12T02:08:53.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.630+0000 7f96c72dc740 1 -- start start 2024-09-12T02:08:53.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.630+0000 7f96af7fe640 1 --2- 172.21.15.16:0/3615656256 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f96a80a2c00 0x7f96a80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:53.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.630+0000 7f96c72dc740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad77a0cf50 0x55ad77a0b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:53.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.630+0000 7f96c72dc740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad77a0b9e0 0x55ad77a0be20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:53.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.630+0000 7f96c72dc740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ad77a16780 0x55ad77a18b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:53.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.630+0000 7f96c72dc740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55ad7796d2f0 con 0x55ad77a0cf50 2024-09-12T02:08:53.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.630+0000 7f96c72dc740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55ad77914cf0 con 0x55ad77a0b9e0 2024-09-12T02:08:53.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.630+0000 7f96c72dc740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55ad779692b0 con 0x55ad77a16780 2024-09-12T02:08:53.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.630+0000 7f96af7fe640 1 -- 172.21.15.16:0/3615656256 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f96b802e030 con 0x7f96c0106a80 2024-09-12T02:08:53.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.630+0000 7f96be59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad77a0cf50 0x55ad77a0b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:53.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.630+0000 7f96bdd9b640 1 --2- 172.21.15.16:0/3615656256 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f96a80a2c00 0x7f96a80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:53.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.631+0000 7f96be59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad77a0cf50 0x55ad77a0b4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33610/0 (socket says 172.21.15.16:33610) 2024-09-12T02:08:53.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.631+0000 7f96be59c640 1 -- 172.21.15.16:0/415770321 learned_addr learned my addr 172.21.15.16:0/415770321 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:08:53.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.631+0000 7f96bdd9b640 1 --2- 172.21.15.16:0/415770321 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad77a0b9e0 0x55ad77a0be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:53.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.631+0000 7f96bed9d640 1 --2- 172.21.15.16:0/415770321 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ad77a16780 0x55ad77a18b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:53.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.631+0000 7f96be59c640 1 -- 172.21.15.16:0/415770321 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ad77a16780 msgr2=0x55ad77a18b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:53.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.631+0000 7f96be59c640 1 --2- 172.21.15.16:0/415770321 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ad77a16780 0x55ad77a18b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:53.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.631+0000 7f96be59c640 1 -- 172.21.15.16:0/415770321 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad77a0b9e0 msgr2=0x55ad77a0be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:53.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.631+0000 7f96be59c640 1 --2- 172.21.15.16:0/415770321 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad77a0b9e0 0x55ad77a0be20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:53.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.631+0000 7f96be59c640 1 -- 172.21.15.16:0/415770321 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad778f93a0 con 0x55ad77a0cf50 2024-09-12T02:08:53.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.631+0000 7f96bdd9b640 1 --2- 172.21.15.16:0/3615656256 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f96a80a2c00 0x7f96a80a50c0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f96c006b2e0 tx=0x7f96b400aaf0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:53.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.631+0000 7f96be59c640 1 --2- 172.21.15.16:0/415770321 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad77a0cf50 0x55ad77a0b4a0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f96b000bc90 tx=0x7f96b00107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87c3240119d74217 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:53.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.631+0000 7f96acff9640 1 -- 172.21.15.16:0/415770321 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96b001c480 con 0x55ad77a0cf50 2024-09-12T02:08:53.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.631+0000 7f96acff9640 1 -- 172.21.15.16:0/415770321 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96b001cb00 con 0x55ad77a0cf50 2024-09-12T02:08:53.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.631+0000 7f96acff9640 1 -- 172.21.15.16:0/415770321 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96b0011b40 con 0x55ad77a0cf50 2024-09-12T02:08:53.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.632+0000 7f96c72dc740 1 -- 172.21.15.16:0/415770321 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad77a0cf50 msgr2=0x55ad77a0b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:53.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.632+0000 7f96c72dc740 1 --2- 172.21.15.16:0/415770321 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad77a0cf50 0x55ad77a0b4a0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f96b000bc90 tx=0x7f96b00107a0 comp rx=0 tx=0).stop 2024-09-12T02:08:53.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.632+0000 7f96c72dc740 1 -- 172.21.15.16:0/415770321 shutdown_connections 2024-09-12T02:08:53.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.632+0000 7f96c72dc740 1 --2- 172.21.15.16:0/415770321 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ad77a16780 0x55ad77a18b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:53.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.632+0000 7f96c72dc740 1 --2- 172.21.15.16:0/415770321 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad77a0b9e0 0x55ad77a0be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:53.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.632+0000 7f96c72dc740 1 --2- 172.21.15.16:0/415770321 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad77a0cf50 0x55ad77a0b4a0 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:53.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.632+0000 7f96c72dc740 1 -- 172.21.15.16:0/415770321 >> 172.21.15.16:0/415770321 conn(0x55ad779d0310 msgr2=0x55ad77a08070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:53.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.632+0000 7f96c72dc740 1 -- 172.21.15.16:0/415770321 shutdown_connections 2024-09-12T02:08:53.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.632+0000 7f96c72dc740 1 -- 172.21.15.16:0/415770321 wait complete. 2024-09-12T02:08:53.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.633+0000 7f96c72dc740 1 -- start start 2024-09-12T02:08:53.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.633+0000 7f96c72dc740 1 Processor -- start 2024-09-12T02:08:53.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.633+0000 7f96c72dc740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad77a0b9e0 0x55ad77a160f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:53.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.633+0000 7f96c72dc740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ad77a0cf50 0x55ad77a133a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:53.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.633+0000 7f96c72dc740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad77a16780 0x55ad77a13900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:53.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.633+0000 7f96c72dc740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55ad77a19480 con 0x55ad77a16780 2024-09-12T02:08:53.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.633+0000 7f96c72dc740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55ad77928060 con 0x55ad77a0b9e0 2024-09-12T02:08:53.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.633+0000 7f96c72dc740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55ad77a19600 con 0x55ad77a0cf50 2024-09-12T02:08:53.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.633+0000 7f96c72dc740 10 client.0 _ll_register_callbacks cb 0x55ad7767b8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:08:53.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.633+0000 7f96bed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad77a16780 0x55ad77a13900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:53.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.633+0000 7f96bed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad77a16780 0x55ad77a13900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33612/0 (socket says 172.21.15.16:33612) 2024-09-12T02:08:53.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.633+0000 7f96bed9d640 1 -- 172.21.15.16:0/3918525135 learned_addr learned my addr 172.21.15.16:0/3918525135 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:08:53.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.633+0000 7f96be59c640 1 --2- 172.21.15.16:0/3918525135 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad77a0b9e0 0x55ad77a160f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:53.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.633+0000 7f96bdd9b640 1 --2- 172.21.15.16:0/3918525135 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ad77a0cf50 0x55ad77a133a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:53.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.634+0000 7f96bed9d640 1 -- 172.21.15.16:0/3918525135 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ad77a0cf50 msgr2=0x55ad77a133a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:53.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.634+0000 7f96bed9d640 1 --2- 172.21.15.16:0/3918525135 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ad77a0cf50 0x55ad77a133a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:53.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.634+0000 7f96bed9d640 1 -- 172.21.15.16:0/3918525135 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad77a0b9e0 msgr2=0x55ad77a160f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:53.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.634+0000 7f96bed9d640 1 --2- 172.21.15.16:0/3918525135 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad77a0b9e0 0x55ad77a160f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:53.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.634+0000 7f96bed9d640 1 -- 172.21.15.16:0/3918525135 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad776de3c0 con 0x55ad77a16780 2024-09-12T02:08:53.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.634+0000 7f96be59c640 1 --2- 172.21.15.16:0/3918525135 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad77a0b9e0 0x55ad77a160f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:08:53.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.634+0000 7f96bed9d640 1 --2- 172.21.15.16:0/3918525135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad77a16780 0x55ad77a13900 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f96b80c56d0 tx=0x7f96b80bce50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e751ba232fe37687 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:53.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.634+0000 7f96a5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:08:53.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.634+0000 7f96a5ffb640 1 -- 172.21.15.16:0/3918525135 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96b8031e60 con 0x55ad77a16780 2024-09-12T02:08:53.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.634+0000 7f96a5ffb640 1 -- 172.21.15.16:0/3918525135 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f96b80c5970 con 0x55ad77a16780 2024-09-12T02:08:53.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.634+0000 7f96c72dc740 1 -- 172.21.15.16:0/3918525135 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96b001cb00 con 0x55ad77a16780 2024-09-12T02:08:53.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.634+0000 7f96a5ffb640 1 -- 172.21.15.16:0/3918525135 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f96b80098b0 con 0x55ad77a16780 2024-09-12T02:08:53.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.634+0000 7f96c72dc740 10 client.17577 fetch_fsmap learned FSMap version 25 2024-09-12T02:08:53.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.634+0000 7f96c72dc740 1 -- 172.21.15.16:0/3918525135 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ad77a24ef0 con 0x55ad77a16780 2024-09-12T02:08:53.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.634+0000 7f96a5ffb640 1 -- 172.21.15.16:0/3918525135 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96b001cb00 con 0x55ad77a16780 2024-09-12T02:08:53.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.635+0000 7f96a5ffb640 1 -- 172.21.15.16:0/3918525135 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f96b80db020 con 0x55ad77a16780 2024-09-12T02:08:53.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.635+0000 7f96c72dc740 10 client.17577 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:08:53.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.635+0000 7f96c72dc740 10 client.17577 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T02:08:53.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.635+0000 7f96c72dc740 20 client.17577 populate_metadata read hostname 'smithi016' 2024-09-12T02:08:53.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.635+0000 7f96c72dc740 1 --2- 172.21.15.16:0/3918525135 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x55ad77a26290 0x55ad77a286e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:53.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.635+0000 7f96c72dc740 4 client.17577 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:08:53.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.635+0000 7f96c72dc740 1 -- 172.21.15.16:0/3918525135 --> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ad7797a860 con 0x55ad77a26290 2024-09-12T02:08:53.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.635+0000 7f96be59c640 1 --2- 172.21.15.16:0/3918525135 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x55ad77a26290 0x55ad77a286e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:53.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.635+0000 7f96a5ffb640 10 client.17577 ms_handle_connect on v2:172.21.15.184:6838/4203347941 2024-09-12T02:08:53.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.635+0000 7f96be59c640 1 --2- 172.21.15.16:0/3918525135 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x55ad77a26290 0x55ad77a286e0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a7126edd3a11db72 server_cookie=ab4ce628de043fc1 in_seq=0 out_seq=0 2024-09-12T02:08:53.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.636+0000 7f96a5ffb640 1 -- 172.21.15.16:0/3918525135 <== mds.1 v2:172.21.15.184:6838/4203347941 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ad7797a860 con 0x55ad77a26290 2024-09-12T02:08:53.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.636+0000 7f96a5ffb640 10 client.17577 handle_command_reply: tid=0 2024-09-12T02:08:53.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.636+0000 7f96c72dc740 1 client.17577 shutdown 2024-09-12T02:08:53.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.636+0000 7f96c72dc740 1 -- 172.21.15.16:0/3918525135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad77a16780 msgr2=0x55ad77a13900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:53.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.636+0000 7f96c72dc740 1 --2- 172.21.15.16:0/3918525135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad77a16780 0x55ad77a13900 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f96b80c56d0 tx=0x7f96b80bce50 comp rx=0 tx=0).stop 2024-09-12T02:08:53.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.636+0000 7f96c72dc740 1 -- 172.21.15.16:0/3918525135 shutdown_connections 2024-09-12T02:08:53.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.636+0000 7f96c72dc740 1 --2- 172.21.15.16:0/3918525135 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x55ad77a26290 0x55ad77a286e0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:53.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.636+0000 7f96c72dc740 1 --2- 172.21.15.16:0/3918525135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad77a16780 0x55ad77a13900 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:53.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.636+0000 7f96c72dc740 1 --2- 172.21.15.16:0/3918525135 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ad77a0cf50 0x55ad77a133a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:53.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.636+0000 7f96c72dc740 1 --2- 172.21.15.16:0/3918525135 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad77a0b9e0 0x55ad77a160f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:53.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.636+0000 7f96c72dc740 1 -- 172.21.15.16:0/3918525135 >> 172.21.15.16:0/3918525135 conn(0x55ad779d0310 msgr2=0x55ad77a0dd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:53.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.636+0000 7f96a5ffb640 0 client.17577 ms_handle_reset on v2:172.21.15.184:6838/4203347941 2024-09-12T02:08:53.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.636+0000 7f96c72dc740 1 -- 172.21.15.16:0/3918525135 shutdown_connections 2024-09-12T02:08:53.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.636+0000 7f96c72dc740 1 -- 172.21.15.16:0/3918525135 wait complete. 2024-09-12T02:08:53.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.637+0000 7f96c72dc740 20 client.17577 trim_cache size 0 max 16384 2024-09-12T02:08:53.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.650+0000 7f96c72dc740 1 Processor -- start 2024-09-12T02:08:53.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.650+0000 7f96c72dc740 1 -- start start 2024-09-12T02:08:53.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.650+0000 7f96c72dc740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad77a0b9e0 0x55ad77a22780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:53.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.650+0000 7f96c72dc740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad77a0cf50 0x55ad77a12c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:53.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.650+0000 7f96c72dc740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ad77a16780 0x55ad77a0e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:53.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.650+0000 7f96c72dc740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f96c01ad980 con 0x55ad77a0cf50 2024-09-12T02:08:53.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.650+0000 7f96c72dc740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f96c01ab250 con 0x55ad77a0b9e0 2024-09-12T02:08:53.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.650+0000 7f96c72dc740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f96c01a90c0 con 0x55ad77a16780 2024-09-12T02:08:53.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.650+0000 7f96bdd9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad77a0cf50 0x55ad77a12c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:53.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.650+0000 7f96bdd9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad77a0cf50 0x55ad77a12c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33624/0 (socket says 172.21.15.16:33624) 2024-09-12T02:08:53.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.650+0000 7f96bdd9b640 1 -- 172.21.15.16:0/4019603676 learned_addr learned my addr 172.21.15.16:0/4019603676 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:08:53.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.650+0000 7f96bed9d640 1 --2- 172.21.15.16:0/4019603676 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ad77a16780 0x55ad77a0e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:53.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.650+0000 7f96be59c640 1 --2- 172.21.15.16:0/4019603676 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad77a0b9e0 0x55ad77a22780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:53.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.650+0000 7f96bdd9b640 1 -- 172.21.15.16:0/4019603676 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ad77a16780 msgr2=0x55ad77a0e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:53.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.650+0000 7f96bdd9b640 1 --2- 172.21.15.16:0/4019603676 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ad77a16780 0x55ad77a0e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:53.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.650+0000 7f96bdd9b640 1 -- 172.21.15.16:0/4019603676 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad77a0b9e0 msgr2=0x55ad77a22780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:53.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.650+0000 7f96bdd9b640 1 --2- 172.21.15.16:0/4019603676 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad77a0b9e0 0x55ad77a22780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:53.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.650+0000 7f96bdd9b640 1 -- 172.21.15.16:0/4019603676 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad77a118f0 con 0x55ad77a0cf50 2024-09-12T02:08:53.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.650+0000 7f96bed9d640 1 --2- 172.21.15.16:0/4019603676 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ad77a16780 0x55ad77a0e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:08:53.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.651+0000 7f96bdd9b640 1 --2- 172.21.15.16:0/4019603676 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad77a0cf50 0x55ad77a12c60 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f96b400ae70 tx=0x7f96b401f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a41708fd384f8ae server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:53.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.651+0000 7f96a7fff640 1 -- 172.21.15.16:0/4019603676 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96b4011e40 con 0x55ad77a0cf50 2024-09-12T02:08:53.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.651+0000 7f96a7fff640 1 -- 172.21.15.16:0/4019603676 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96b4012e30 con 0x55ad77a0cf50 2024-09-12T02:08:53.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.651+0000 7f96a7fff640 1 -- 172.21.15.16:0/4019603676 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96b4005660 con 0x55ad77a0cf50 2024-09-12T02:08:53.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.651+0000 7f96c72dc740 1 -- 172.21.15.16:0/4019603676 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad77a0cf50 msgr2=0x55ad77a12c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:53.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.651+0000 7f96c72dc740 1 --2- 172.21.15.16:0/4019603676 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad77a0cf50 0x55ad77a12c60 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f96b400ae70 tx=0x7f96b401f000 comp rx=0 tx=0).stop 2024-09-12T02:08:53.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.651+0000 7f96c72dc740 1 -- 172.21.15.16:0/4019603676 shutdown_connections 2024-09-12T02:08:53.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.651+0000 7f96c72dc740 1 --2- 172.21.15.16:0/4019603676 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ad77a16780 0x55ad77a0e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:53.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.651+0000 7f96c72dc740 1 --2- 172.21.15.16:0/4019603676 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad77a0cf50 0x55ad77a12c60 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:53.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.651+0000 7f96c72dc740 1 --2- 172.21.15.16:0/4019603676 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad77a0b9e0 0x55ad77a22780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:53.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.651+0000 7f96c72dc740 1 -- 172.21.15.16:0/4019603676 >> 172.21.15.16:0/4019603676 conn(0x55ad779d0310 msgr2=0x55ad77a11c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:53.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.651+0000 7f96c72dc740 1 -- 172.21.15.16:0/4019603676 shutdown_connections 2024-09-12T02:08:53.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.651+0000 7f96c72dc740 1 -- 172.21.15.16:0/4019603676 wait complete. 2024-09-12T02:08:53.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.652+0000 7f96c72dc740 1 -- start start 2024-09-12T02:08:53.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.652+0000 7f96c72dc740 1 Processor -- start 2024-09-12T02:08:53.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.652+0000 7f96c72dc740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad77a0b9e0 0x55ad77a1ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:53.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.652+0000 7f96c72dc740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad77a0cf50 0x55ad77a1f010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:53.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.652+0000 7f96c72dc740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ad77a16780 0x55ad77a0fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:53.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.652+0000 7f96c72dc740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f96c01a9710 con 0x55ad77a0cf50 2024-09-12T02:08:53.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.652+0000 7f96c72dc740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f96c01aab80 con 0x55ad77a0b9e0 2024-09-12T02:08:53.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.652+0000 7f96c72dc740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f96c01a9de0 con 0x55ad77a16780 2024-09-12T02:08:53.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.652+0000 7f96c72dc740 10 client.0 _ll_register_callbacks cb 0x7f96b8009c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:08:53.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.652+0000 7f96bdd9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad77a0cf50 0x55ad77a1f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:53.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.652+0000 7f96bdd9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad77a0cf50 0x55ad77a1f010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33636/0 (socket says 172.21.15.16:33636) 2024-09-12T02:08:53.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.652+0000 7f96bdd9b640 1 -- 172.21.15.16:0/1359477595 learned_addr learned my addr 172.21.15.16:0/1359477595 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:08:53.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.652+0000 7f96be59c640 1 --2- 172.21.15.16:0/1359477595 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad77a0b9e0 0x55ad77a1ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:53.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.653+0000 7f96bed9d640 1 --2- 172.21.15.16:0/1359477595 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ad77a16780 0x55ad77a0fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:53.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.653+0000 7f96bdd9b640 1 -- 172.21.15.16:0/1359477595 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ad77a16780 msgr2=0x55ad77a0fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:53.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.653+0000 7f96bdd9b640 1 --2- 172.21.15.16:0/1359477595 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ad77a16780 0x55ad77a0fbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:53.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.653+0000 7f96bdd9b640 1 -- 172.21.15.16:0/1359477595 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad77a0b9e0 msgr2=0x55ad77a1ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:53.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.653+0000 7f96bdd9b640 1 --2- 172.21.15.16:0/1359477595 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad77a0b9e0 0x55ad77a1ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:53.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.653+0000 7f96bdd9b640 1 -- 172.21.15.16:0/1359477595 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad77a103b0 con 0x55ad77a0cf50 2024-09-12T02:08:53.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.653+0000 7f96bdd9b640 1 --2- 172.21.15.16:0/1359477595 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad77a0cf50 0x55ad77a1f010 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f96b400ae40 tx=0x7f96b4005280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=922da8af9e89834 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:53.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.653+0000 7f96a5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:08:53.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.653+0000 7f96a5ffb640 1 -- 172.21.15.16:0/1359477595 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96b402f070 con 0x55ad77a0cf50 2024-09-12T02:08:53.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.653+0000 7f96a5ffb640 1 -- 172.21.15.16:0/1359477595 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f96b4011e40 con 0x55ad77a0cf50 2024-09-12T02:08:53.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.653+0000 7f96c72dc740 1 -- 172.21.15.16:0/1359477595 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96b4012e30 con 0x55ad77a0cf50 2024-09-12T02:08:53.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.653+0000 7f96a5ffb640 1 -- 172.21.15.16:0/1359477595 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f96b4012b60 con 0x55ad77a0cf50 2024-09-12T02:08:53.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.653+0000 7f96c72dc740 10 client.17583 fetch_fsmap learned FSMap version 25 2024-09-12T02:08:53.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.653+0000 7f96c72dc740 1 -- 172.21.15.16:0/1359477595 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ad77a90a40 con 0x55ad77a0cf50 2024-09-12T02:08:53.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.653+0000 7f96a5ffb640 1 -- 172.21.15.16:0/1359477595 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96b4012e30 con 0x55ad77a0cf50 2024-09-12T02:08:53.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.654+0000 7f96a5ffb640 1 -- 172.21.15.16:0/1359477595 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f96b4004cf0 con 0x55ad77a0cf50 2024-09-12T02:08:53.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.654+0000 7f96c72dc740 10 client.17583 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:08:53.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.654+0000 7f96c72dc740 10 client.17583 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T02:08:53.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.654+0000 7f96c72dc740 20 client.17583 populate_metadata read hostname 'smithi016' 2024-09-12T02:08:53.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.654+0000 7f96c72dc740 1 --2- 172.21.15.16:0/1359477595 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x55ad77a91140 0x55ad77a93560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:53.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.654+0000 7f96c72dc740 4 client.17583 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T02:08:53.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.654+0000 7f96c72dc740 1 -- 172.21.15.16:0/1359477595 --> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ad77974d70 con 0x55ad77a91140 2024-09-12T02:08:53.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.654+0000 7f96be59c640 1 --2- 172.21.15.16:0/1359477595 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x55ad77a91140 0x55ad77a93560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:53.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.654+0000 7f96be59c640 1 --2- 172.21.15.16:0/1359477595 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x55ad77a91140 0x55ad77a93560 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=92ad347287ea2298 server_cookie=57f36ed2fe16ecc in_seq=0 out_seq=0 2024-09-12T02:08:53.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.654+0000 7f96a5ffb640 10 client.17583 ms_handle_connect on v2:172.21.15.184:6838/4203347941 2024-09-12T02:08:53.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.654+0000 7f96a5ffb640 1 -- 172.21.15.16:0/1359477595 <== mds.1 v2:172.21.15.184:6838/4203347941 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ad77974d70 con 0x55ad77a91140 2024-09-12T02:08:53.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.654+0000 7f96a5ffb640 10 client.17583 handle_command_reply: tid=0 2024-09-12T02:08:53.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.654+0000 7f96c72dc740 1 client.17583 shutdown 2024-09-12T02:08:53.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.655+0000 7f96c72dc740 1 -- 172.21.15.16:0/1359477595 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad77a0cf50 msgr2=0x55ad77a1f010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:53.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.655+0000 7f96c72dc740 1 --2- 172.21.15.16:0/1359477595 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad77a0cf50 0x55ad77a1f010 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f96b400ae40 tx=0x7f96b4005280 comp rx=0 tx=0).stop 2024-09-12T02:08:53.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.655+0000 7f96c72dc740 1 -- 172.21.15.16:0/1359477595 shutdown_connections 2024-09-12T02:08:53.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.655+0000 7f96c72dc740 1 --2- 172.21.15.16:0/1359477595 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x55ad77a91140 0x55ad77a93560 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:53.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.655+0000 7f96c72dc740 1 --2- 172.21.15.16:0/1359477595 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ad77a16780 0x55ad77a0fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:53.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.655+0000 7f96c72dc740 1 --2- 172.21.15.16:0/1359477595 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad77a0cf50 0x55ad77a1f010 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:53.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.655+0000 7f96c72dc740 1 --2- 172.21.15.16:0/1359477595 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad77a0b9e0 0x55ad77a1ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:53.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.655+0000 7f96c72dc740 1 -- 172.21.15.16:0/1359477595 >> 172.21.15.16:0/1359477595 conn(0x55ad779d0310 msgr2=0x55ad77a14c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:53.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.655+0000 7f96a5ffb640 0 client.17583 ms_handle_reset on v2:172.21.15.184:6838/4203347941 2024-09-12T02:08:53.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.655+0000 7f96c72dc740 1 -- 172.21.15.16:0/1359477595 shutdown_connections 2024-09-12T02:08:53.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.655+0000 7f96c72dc740 1 -- 172.21.15.16:0/1359477595 wait complete. 2024-09-12T02:08:53.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.655+0000 7f96c72dc740 20 client.17583 trim_cache size 0 max 16384 2024-09-12T02:08:53.667 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T02:08:53.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.655+0000 7f96c4f03640 1 -- 172.21.15.16:0/3615656256 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f96a80a2c00 msgr2=0x7f96a80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:53.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.655+0000 7f96c4f03640 1 --2- 172.21.15.16:0/3615656256 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f96a80a2c00 0x7f96a80a50c0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f96c006b2e0 tx=0x7f96b400aaf0 comp rx=0 tx=0).stop 2024-09-12T02:08:53.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.655+0000 7f96c4f03640 1 -- 172.21.15.16:0/3615656256 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96c0106a80 msgr2=0x7f96c01a4330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:53.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.655+0000 7f96c4f03640 1 --2- 172.21.15.16:0/3615656256 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96c0106a80 0x7f96c01a4330 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f96b8005cc0 tx=0x7f96b8003970 comp rx=0 tx=0).stop 2024-09-12T02:08:53.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.656+0000 7f96c4f03640 1 -- 172.21.15.16:0/3615656256 shutdown_connections 2024-09-12T02:08:53.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.656+0000 7f96c4f03640 1 --2- 172.21.15.16:0/3615656256 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f96a80a2c00 0x7f96a80a50c0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:53.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.656+0000 7f96c4f03640 1 --2- 172.21.15.16:0/3615656256 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96c0106a80 0x7f96c01a4330 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:53.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.656+0000 7f96c4f03640 1 --2- 172.21.15.16:0/3615656256 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f96c0101420 0x7f96c01997f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:53.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.656+0000 7f96c4f03640 1 --2- 172.21.15.16:0/3615656256 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96c0100a80 0x7f96c01992b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:53.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.656+0000 7f96c4f03640 1 -- 172.21.15.16:0/3615656256 >> 172.21.15.16:0/3615656256 conn(0x7f96c00fc7a0 msgr2=0x7f96c00fce70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:53.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.656+0000 7f96c4f03640 1 -- 172.21.15.16:0/3615656256 shutdown_connections 2024-09-12T02:08:53.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:53.656+0000 7f96c4f03640 1 -- 172.21.15.16:0/3615656256 wait complete. 2024-09-12T02:08:54.372 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-12T02:08:54.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:08:54 smithi184 ceph-mon[35593]: pgmap v544: 97 pgs: 97 active+clean; 3.7 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 3.0 KiB/s wr, 9 op/s 2024-09-12T02:08:54.629 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:08:54.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:08:54 smithi033 ceph-mon[35332]: pgmap v544: 97 pgs: 97 active+clean; 3.7 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 3.0 KiB/s wr, 9 op/s 2024-09-12T02:08:54.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:08:54 smithi016 ceph-mon[30947]: pgmap v544: 97 pgs: 97 active+clean; 3.7 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 3.0 KiB/s wr, 9 op/s 2024-09-12T02:08:55.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.078+0000 7f9a9e095640 1 -- 172.21.15.16:0/3340975673 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a981002f0 msgr2=0x7f9a981006d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:55.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.078+0000 7f9a9e095640 1 --2- 172.21.15.16:0/3340975673 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a981002f0 0x7f9a981006d0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f9a88009930 tx=0x7f9a8802a420 comp rx=0 tx=0).stop 2024-09-12T02:08:55.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.078+0000 7f9a9e095640 1 -- 172.21.15.16:0/3340975673 shutdown_connections 2024-09-12T02:08:55.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.078+0000 7f9a9e095640 1 --2- 172.21.15.16:0/3340975673 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a98106f50 0x7f9a9810eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:55.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.079+0000 7f9a9e095640 1 --2- 172.21.15.16:0/3340975673 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9a98100ca0 0x7f9a98106840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:55.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.079+0000 7f9a9e095640 1 --2- 172.21.15.16:0/3340975673 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a981002f0 0x7f9a981006d0 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:55.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.079+0000 7f9a9e095640 1 -- 172.21.15.16:0/3340975673 >> 172.21.15.16:0/3340975673 conn(0x7f9a98077210 msgr2=0x7f9a980fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:55.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.079+0000 7f9a9e095640 1 -- 172.21.15.16:0/3340975673 shutdown_connections 2024-09-12T02:08:55.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.079+0000 7f9a9e095640 1 -- 172.21.15.16:0/3340975673 wait complete. 2024-09-12T02:08:55.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.079+0000 7f9a9e095640 1 Processor -- start 2024-09-12T02:08:55.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.080+0000 7f9a9e095640 1 -- start start 2024-09-12T02:08:55.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.080+0000 7f9a9e095640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9a981002f0 0x7f9a98198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:55.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.080+0000 7f9a9e095640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a98100ca0 0x7f9a98198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:55.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.080+0000 7f9a9e095640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a98106f50 0x7f9a9819cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:55.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.080+0000 7f9a9e095640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9a98111fa0 con 0x7f9a98100ca0 2024-09-12T02:08:55.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.080+0000 7f9a9e095640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9a98111e20 con 0x7f9a98106f50 2024-09-12T02:08:55.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.080+0000 7f9a9e095640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f9a98112120 con 0x7f9a981002f0 2024-09-12T02:08:55.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.080+0000 7f9a977fe640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9a981002f0 0x7f9a98198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:55.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.080+0000 7f9a977fe640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9a981002f0 0x7f9a98198500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:50512/0 (socket says 172.21.15.16:50512) 2024-09-12T02:08:55.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.080+0000 7f9a977fe640 1 -- 172.21.15.16:0/395557954 learned_addr learned my addr 172.21.15.16:0/395557954 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:08:55.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.080+0000 7f9a96ffd640 1 --2- 172.21.15.16:0/395557954 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a98100ca0 0x7f9a98198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:55.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.080+0000 7f9a97fff640 1 --2- 172.21.15.16:0/395557954 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a98106f50 0x7f9a9819cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:55.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.081+0000 7f9a977fe640 1 -- 172.21.15.16:0/395557954 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a98106f50 msgr2=0x7f9a9819cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:55.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.081+0000 7f9a977fe640 1 --2- 172.21.15.16:0/395557954 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a98106f50 0x7f9a9819cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:55.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.081+0000 7f9a977fe640 1 -- 172.21.15.16:0/395557954 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a98100ca0 msgr2=0x7f9a98198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:55.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.081+0000 7f9a977fe640 1 --2- 172.21.15.16:0/395557954 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a98100ca0 0x7f9a98198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:55.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.081+0000 7f9a977fe640 1 -- 172.21.15.16:0/395557954 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a9819d4b0 con 0x7f9a981002f0 2024-09-12T02:08:55.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.081+0000 7f9a96ffd640 1 --2- 172.21.15.16:0/395557954 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a98100ca0 0x7f9a98198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:08:55.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.081+0000 7f9a977fe640 1 --2- 172.21.15.16:0/395557954 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9a981002f0 0x7f9a98198500 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9a88009230 tx=0x7f9a88004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:55.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.081+0000 7f9a94ff9640 1 -- 172.21.15.16:0/395557954 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a8803f030 con 0x7f9a981002f0 2024-09-12T02:08:55.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.081+0000 7f9a94ff9640 1 -- 172.21.15.16:0/395557954 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a88003ad0 con 0x7f9a981002f0 2024-09-12T02:08:55.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.081+0000 7f9a9e095640 1 -- 172.21.15.16:0/395557954 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a9819d740 con 0x7f9a981002f0 2024-09-12T02:08:55.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.081+0000 7f9a94ff9640 1 -- 172.21.15.16:0/395557954 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a88035070 con 0x7f9a981002f0 2024-09-12T02:08:55.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.081+0000 7f9a9e095640 1 -- 172.21.15.16:0/395557954 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a981a5080 con 0x7f9a981002f0 2024-09-12T02:08:55.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.084+0000 7f9aa046e740 1 Processor -- start 2024-09-12T02:08:55.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.084+0000 7f9aa046e740 1 -- start start 2024-09-12T02:08:55.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.084+0000 7f9aa046e740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555bb7c92b40 0x555bb7c91090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:55.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.084+0000 7f9aa046e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555bb7c915d0 0x555bb7c91a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:55.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.084+0000 7f9a94ff9640 1 -- 172.21.15.16:0/395557954 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f9a88042030 con 0x7f9a981002f0 2024-09-12T02:08:55.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.084+0000 7f9aa046e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555bb7c9c370 0x555bb7c9e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:55.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.084+0000 7f9aa046e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x555bb7bf72f0 con 0x555bb7c9c370 2024-09-12T02:08:55.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.084+0000 7f9aa046e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x555bb7b9ecf0 con 0x555bb7c915d0 2024-09-12T02:08:55.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.084+0000 7f9aa046e740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x555bb7bf32b0 con 0x555bb7c92b40 2024-09-12T02:08:55.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.084+0000 7f9a96ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555bb7c9c370 0x555bb7c9e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:55.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.084+0000 7f9a96ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555bb7c9c370 0x555bb7c9e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33652/0 (socket says 172.21.15.16:33652) 2024-09-12T02:08:55.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.084+0000 7f9a96ffd640 1 -- 172.21.15.16:0/2406727327 learned_addr learned my addr 172.21.15.16:0/2406727327 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:08:55.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.084+0000 7f9a97fff640 1 --2- 172.21.15.16:0/2406727327 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555bb7c92b40 0x555bb7c91090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:55.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.084+0000 7f9a977fe640 1 --2- 172.21.15.16:0/2406727327 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555bb7c915d0 0x555bb7c91a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:55.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.085+0000 7f9a96ffd640 1 -- 172.21.15.16:0/2406727327 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555bb7c92b40 msgr2=0x555bb7c91090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:55.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.085+0000 7f9a96ffd640 1 --2- 172.21.15.16:0/2406727327 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555bb7c92b40 0x555bb7c91090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:55.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.085+0000 7f9a96ffd640 1 -- 172.21.15.16:0/2406727327 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555bb7c915d0 msgr2=0x555bb7c91a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:55.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.085+0000 7f9a96ffd640 1 --2- 172.21.15.16:0/2406727327 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555bb7c915d0 0x555bb7c91a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:55.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.085+0000 7f9a96ffd640 1 -- 172.21.15.16:0/2406727327 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555bb7b833a0 con 0x555bb7c9c370 2024-09-12T02:08:55.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.085+0000 7f9a96ffd640 1 --2- 172.21.15.16:0/2406727327 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555bb7c9c370 0x555bb7c9e760 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f9a8c010480 tx=0x7f9a8c010940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4dd92ea8a3d2d96 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:55.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.085+0000 7f9a94ff9640 1 --2- 172.21.15.16:0/395557954 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f9a800a2b80 0x7f9a800a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:55.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.085+0000 7f9a7dffb640 1 -- 172.21.15.16:0/2406727327 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a8c011760 con 0x555bb7c9c370 2024-09-12T02:08:55.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.085+0000 7f9a97fff640 1 --2- 172.21.15.16:0/395557954 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f9a800a2b80 0x7f9a800a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:55.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.085+0000 7f9a94ff9640 1 -- 172.21.15.16:0/395557954 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f9a880ef080 con 0x7f9a981002f0 2024-09-12T02:08:55.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.085+0000 7f9a7dffb640 1 -- 172.21.15.16:0/2406727327 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a8c011de0 con 0x555bb7c9c370 2024-09-12T02:08:55.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.085+0000 7f9a7dffb640 1 -- 172.21.15.16:0/2406727327 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a8c00e430 con 0x555bb7c9c370 2024-09-12T02:08:55.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.086+0000 7f9aa046e740 1 -- 172.21.15.16:0/2406727327 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555bb7c9c370 msgr2=0x555bb7c9e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:55.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.086+0000 7f9a97fff640 1 --2- 172.21.15.16:0/395557954 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f9a800a2b80 0x7f9a800a5040 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f9a90005fd0 tx=0x7f9a900096a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:55.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.086+0000 7f9aa046e740 1 --2- 172.21.15.16:0/2406727327 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555bb7c9c370 0x555bb7c9e760 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f9a8c010480 tx=0x7f9a8c010940 comp rx=0 tx=0).stop 2024-09-12T02:08:55.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.086+0000 7f9aa046e740 1 -- 172.21.15.16:0/2406727327 shutdown_connections 2024-09-12T02:08:55.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.086+0000 7f9aa046e740 1 --2- 172.21.15.16:0/2406727327 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555bb7c9c370 0x555bb7c9e760 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:55.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.086+0000 7f9aa046e740 1 --2- 172.21.15.16:0/2406727327 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555bb7c915d0 0x555bb7c91a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:55.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.086+0000 7f9aa046e740 1 --2- 172.21.15.16:0/2406727327 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555bb7c92b40 0x555bb7c91090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:55.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.086+0000 7f9aa046e740 1 -- 172.21.15.16:0/2406727327 >> 172.21.15.16:0/2406727327 conn(0x555bb7c5a310 msgr2=0x555bb7c8dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:55.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.086+0000 7f9aa046e740 1 -- 172.21.15.16:0/2406727327 shutdown_connections 2024-09-12T02:08:55.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.086+0000 7f9aa046e740 1 -- 172.21.15.16:0/2406727327 wait complete. 2024-09-12T02:08:55.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.086+0000 7f9aa046e740 1 -- start start 2024-09-12T02:08:55.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.086+0000 7f9aa046e740 1 Processor -- start 2024-09-12T02:08:55.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.087+0000 7f9aa046e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555bb7c915d0 0x555bb7c9bd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:55.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.087+0000 7f9aa046e740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555bb7c92b40 0x555bb7c99000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:55.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.087+0000 7f9aa046e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555bb7c9c370 0x555bb7c995d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:55.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.087+0000 7f9aa046e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x555bb7c9f070 con 0x555bb7c915d0 2024-09-12T02:08:55.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.087+0000 7f9aa046e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x555bb7bb2060 con 0x555bb7c9c370 2024-09-12T02:08:55.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.087+0000 7f9aa046e740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x555bb7c9f1f0 con 0x555bb7c92b40 2024-09-12T02:08:55.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.087+0000 7f9aa046e740 10 client.0 _ll_register_callbacks cb 0x555bb79058c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:08:55.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.087+0000 7f9a977fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555bb7c915d0 0x555bb7c9bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:55.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.087+0000 7f9a977fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555bb7c915d0 0x555bb7c9bd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33664/0 (socket says 172.21.15.16:33664) 2024-09-12T02:08:55.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.087+0000 7f9a977fe640 1 -- 172.21.15.16:0/185993829 learned_addr learned my addr 172.21.15.16:0/185993829 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:08:55.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.087+0000 7f9a96ffd640 1 --2- 172.21.15.16:0/185993829 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555bb7c92b40 0x555bb7c99000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:55.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.087+0000 7f9a97fff640 1 --2- 172.21.15.16:0/185993829 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555bb7c9c370 0x555bb7c995d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:55.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.087+0000 7f9a977fe640 1 -- 172.21.15.16:0/185993829 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555bb7c92b40 msgr2=0x555bb7c99000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:55.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.087+0000 7f9a977fe640 1 --2- 172.21.15.16:0/185993829 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555bb7c92b40 0x555bb7c99000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:55.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.087+0000 7f9a977fe640 1 -- 172.21.15.16:0/185993829 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555bb7c9c370 msgr2=0x555bb7c995d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:55.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.087+0000 7f9a977fe640 1 --2- 172.21.15.16:0/185993829 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555bb7c9c370 0x555bb7c995d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:55.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.087+0000 7f9a977fe640 1 -- 172.21.15.16:0/185993829 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555bb79683c0 con 0x555bb7c915d0 2024-09-12T02:08:55.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.087+0000 7f9a977fe640 1 --2- 172.21.15.16:0/185993829 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555bb7c915d0 0x555bb7c9bd50 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f9a880c40c0 tx=0x7f9a88005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98c4fc36f6037525 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:55.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.087+0000 7f9a76ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:08:55.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.087+0000 7f9a76ffd640 1 -- 172.21.15.16:0/185993829 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a88049070 con 0x555bb7c915d0 2024-09-12T02:08:55.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.088+0000 7f9a76ffd640 1 -- 172.21.15.16:0/185993829 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f9a880b80a0 con 0x555bb7c915d0 2024-09-12T02:08:55.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.088+0000 7f9aa046e740 1 -- 172.21.15.16:0/185993829 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9a8c011de0 con 0x555bb7c915d0 2024-09-12T02:08:55.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.088+0000 7f9a76ffd640 1 -- 172.21.15.16:0/185993829 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9a880bfd30 con 0x555bb7c915d0 2024-09-12T02:08:55.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.088+0000 7f9aa046e740 10 client.17595 fetch_fsmap learned FSMap version 25 2024-09-12T02:08:55.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.088+0000 7f9aa046e740 1 -- 172.21.15.16:0/185993829 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555bb7caaaf0 con 0x555bb7c915d0 2024-09-12T02:08:55.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.088+0000 7f9a76ffd640 1 -- 172.21.15.16:0/185993829 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a8c011de0 con 0x555bb7c915d0 2024-09-12T02:08:55.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.088+0000 7f9a76ffd640 1 -- 172.21.15.16:0/185993829 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f9a8804c060 con 0x555bb7c915d0 2024-09-12T02:08:55.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.088+0000 7f9aa046e740 10 client.17595 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:08:55.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.088+0000 7f9aa046e740 10 client.17595 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:08:55.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.088+0000 7f9aa046e740 20 client.17595 populate_metadata read hostname 'smithi016' 2024-09-12T02:08:55.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.088+0000 7f9aa046e740 1 --2- 172.21.15.16:0/185993829 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x555bb7cabe40 0x555bb7cae230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:55.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.088+0000 7f9aa046e740 4 client.17595 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:08:55.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.088+0000 7f9aa046e740 1 -- 172.21.15.16:0/185993829 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555bb7c04860 con 0x555bb7cabe40 2024-09-12T02:08:55.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.089+0000 7f9a96ffd640 1 --2- 172.21.15.16:0/185993829 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x555bb7cabe40 0x555bb7cae230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:55.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.089+0000 7f9a96ffd640 1 --2- 172.21.15.16:0/185993829 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x555bb7cabe40 0x555bb7cae230 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5542bc0dcbaddeb server_cookie=d54cd556796459c9 in_seq=0 out_seq=0 2024-09-12T02:08:55.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.089+0000 7f9a76ffd640 10 client.17595 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:08:55.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.090+0000 7f9a76ffd640 1 -- 172.21.15.16:0/185993829 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555bb7c04860 con 0x555bb7cabe40 2024-09-12T02:08:55.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.090+0000 7f9a76ffd640 10 client.17595 handle_command_reply: tid=0 2024-09-12T02:08:55.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.090+0000 7f9aa046e740 1 client.17595 shutdown 2024-09-12T02:08:55.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.090+0000 7f9aa046e740 1 -- 172.21.15.16:0/185993829 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555bb7c915d0 msgr2=0x555bb7c9bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:55.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.090+0000 7f9aa046e740 1 --2- 172.21.15.16:0/185993829 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555bb7c915d0 0x555bb7c9bd50 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f9a880c40c0 tx=0x7f9a88005c00 comp rx=0 tx=0).stop 2024-09-12T02:08:55.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.090+0000 7f9aa046e740 1 -- 172.21.15.16:0/185993829 shutdown_connections 2024-09-12T02:08:55.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.090+0000 7f9aa046e740 1 --2- 172.21.15.16:0/185993829 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x555bb7cabe40 0x555bb7cae230 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:55.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.090+0000 7f9aa046e740 1 --2- 172.21.15.16:0/185993829 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555bb7c9c370 0x555bb7c995d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:55.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.090+0000 7f9aa046e740 1 --2- 172.21.15.16:0/185993829 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555bb7c92b40 0x555bb7c99000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:55.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.090+0000 7f9aa046e740 1 --2- 172.21.15.16:0/185993829 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555bb7c915d0 0x555bb7c9bd50 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:55.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.090+0000 7f9a76ffd640 0 client.17595 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:08:55.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.090+0000 7f9aa046e740 1 -- 172.21.15.16:0/185993829 >> 172.21.15.16:0/185993829 conn(0x555bb7c5a310 msgr2=0x555bb7c93960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:55.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.090+0000 7f9aa046e740 1 -- 172.21.15.16:0/185993829 shutdown_connections 2024-09-12T02:08:55.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.090+0000 7f9aa046e740 1 -- 172.21.15.16:0/185993829 wait complete. 2024-09-12T02:08:55.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.090+0000 7f9aa046e740 20 client.17595 trim_cache size 0 max 16384 2024-09-12T02:08:55.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.103+0000 7f9aa046e740 1 Processor -- start 2024-09-12T02:08:55.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.103+0000 7f9aa046e740 1 -- start start 2024-09-12T02:08:55.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.104+0000 7f9aa046e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555bb7c915d0 0x555bb7ca8310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:55.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.104+0000 7f9aa046e740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555bb7c92b40 0x555bb7c9af70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:55.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.104+0000 7f9aa046e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555bb7c9c370 0x555bb7c9b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:55.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.104+0000 7f9aa046e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9a981afd90 con 0x555bb7c915d0 2024-09-12T02:08:55.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.104+0000 7f9aa046e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9a981ad660 con 0x555bb7c9c370 2024-09-12T02:08:55.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.104+0000 7f9aa046e740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f9a981ab4d0 con 0x555bb7c92b40 2024-09-12T02:08:55.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.104+0000 7f9a977fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555bb7c915d0 0x555bb7ca8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:55.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.104+0000 7f9a977fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555bb7c915d0 0x555bb7ca8310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33670/0 (socket says 172.21.15.16:33670) 2024-09-12T02:08:55.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.104+0000 7f9a977fe640 1 -- 172.21.15.16:0/3385770559 learned_addr learned my addr 172.21.15.16:0/3385770559 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:08:55.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.104+0000 7f9a97fff640 1 --2- 172.21.15.16:0/3385770559 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555bb7c9c370 0x555bb7c9b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:55.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.104+0000 7f9a977fe640 1 -- 172.21.15.16:0/3385770559 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555bb7c92b40 msgr2=0x555bb7c9af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:55.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.104+0000 7f9a977fe640 1 --2- 172.21.15.16:0/3385770559 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555bb7c92b40 0x555bb7c9af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:55.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.104+0000 7f9a977fe640 1 -- 172.21.15.16:0/3385770559 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555bb7c9c370 msgr2=0x555bb7c9b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:55.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.104+0000 7f9a977fe640 1 --2- 172.21.15.16:0/3385770559 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555bb7c9c370 0x555bb7c9b4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:55.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.104+0000 7f9a977fe640 1 -- 172.21.15.16:0/3385770559 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555bb7c9c0c0 con 0x555bb7c915d0 2024-09-12T02:08:55.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.104+0000 7f9a977fe640 1 --2- 172.21.15.16:0/3385770559 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555bb7c915d0 0x555bb7ca8310 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f9a880bbf10 tx=0x7f9a880bbcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=981243ad4deb3cf server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:55.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.104+0000 7f9a7d7fa640 1 -- 172.21.15.16:0/3385770559 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a88049070 con 0x555bb7c915d0 2024-09-12T02:08:55.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.104+0000 7f9a7d7fa640 1 -- 172.21.15.16:0/3385770559 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a880b80a0 con 0x555bb7c915d0 2024-09-12T02:08:55.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.104+0000 7f9a7d7fa640 1 -- 172.21.15.16:0/3385770559 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a880c43c0 con 0x555bb7c915d0 2024-09-12T02:08:55.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.105+0000 7f9aa046e740 1 -- 172.21.15.16:0/3385770559 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555bb7c915d0 msgr2=0x555bb7ca8310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:55.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.105+0000 7f9aa046e740 1 --2- 172.21.15.16:0/3385770559 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555bb7c915d0 0x555bb7ca8310 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f9a880bbf10 tx=0x7f9a880bbcb0 comp rx=0 tx=0).stop 2024-09-12T02:08:55.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.105+0000 7f9aa046e740 1 -- 172.21.15.16:0/3385770559 shutdown_connections 2024-09-12T02:08:55.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.105+0000 7f9aa046e740 1 --2- 172.21.15.16:0/3385770559 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555bb7c9c370 0x555bb7c9b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:55.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.105+0000 7f9aa046e740 1 --2- 172.21.15.16:0/3385770559 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555bb7c92b40 0x555bb7c9af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:55.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.105+0000 7f9aa046e740 1 --2- 172.21.15.16:0/3385770559 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555bb7c915d0 0x555bb7ca8310 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:55.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.105+0000 7f9aa046e740 1 -- 172.21.15.16:0/3385770559 >> 172.21.15.16:0/3385770559 conn(0x555bb7c5a310 msgr2=0x555bb7c9a860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:55.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.105+0000 7f9aa046e740 1 -- 172.21.15.16:0/3385770559 shutdown_connections 2024-09-12T02:08:55.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.105+0000 7f9aa046e740 1 -- 172.21.15.16:0/3385770559 wait complete. 2024-09-12T02:08:55.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.105+0000 7f9aa046e740 1 -- start start 2024-09-12T02:08:55.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.106+0000 7f9aa046e740 1 Processor -- start 2024-09-12T02:08:55.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.106+0000 7f9aa046e740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555bb7c915d0 0x555bb7ca0990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:55.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.106+0000 7f9aa046e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555bb7c92b40 0x555bb7ca0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:55.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.106+0000 7f9aa046e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555bb7c9c370 0x555bb7d19ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:55.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.106+0000 7f9aa046e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9a981abb20 con 0x555bb7c92b40 2024-09-12T02:08:55.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.106+0000 7f9aa046e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9a981acf90 con 0x555bb7c9c370 2024-09-12T02:08:55.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.106+0000 7f9aa046e740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f9a981ac1f0 con 0x555bb7c915d0 2024-09-12T02:08:55.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.106+0000 7f9aa046e740 10 client.0 _ll_register_callbacks cb 0x7f9a88005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:08:55.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.106+0000 7f9a96ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555bb7c92b40 0x555bb7ca0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:55.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.106+0000 7f9a96ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555bb7c92b40 0x555bb7ca0ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33686/0 (socket says 172.21.15.16:33686) 2024-09-12T02:08:55.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.106+0000 7f9a96ffd640 1 -- 172.21.15.16:0/3662769799 learned_addr learned my addr 172.21.15.16:0/3662769799 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:08:55.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.106+0000 7f9a97fff640 1 --2- 172.21.15.16:0/3662769799 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555bb7c9c370 0x555bb7d19ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:55.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.106+0000 7f9a977fe640 1 --2- 172.21.15.16:0/3662769799 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555bb7c915d0 0x555bb7ca0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:55.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.106+0000 7f9a96ffd640 1 -- 172.21.15.16:0/3662769799 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555bb7c915d0 msgr2=0x555bb7ca0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:55.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.106+0000 7f9a96ffd640 1 --2- 172.21.15.16:0/3662769799 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555bb7c915d0 0x555bb7ca0990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:55.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.106+0000 7f9a96ffd640 1 -- 172.21.15.16:0/3662769799 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555bb7c9c370 msgr2=0x555bb7d19ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:55.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.106+0000 7f9a96ffd640 1 --2- 172.21.15.16:0/3662769799 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555bb7c9c370 0x555bb7d19ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:55.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.106+0000 7f9a96ffd640 1 -- 172.21.15.16:0/3662769799 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555bb7d1a4b0 con 0x555bb7c92b40 2024-09-12T02:08:55.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.107+0000 7f9a96ffd640 1 --2- 172.21.15.16:0/3662769799 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555bb7c92b40 0x555bb7ca0ed0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f9a8c00bdf0 tx=0x7f9a8c01cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fd5ca9d5a7beece server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:55.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.107+0000 7f9a76ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:08:55.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.107+0000 7f9a76ffd640 1 -- 172.21.15.16:0/3662769799 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a8c01c680 con 0x555bb7c92b40 2024-09-12T02:08:55.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.107+0000 7f9a76ffd640 1 -- 172.21.15.16:0/3662769799 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f9a8c0050b0 con 0x555bb7c92b40 2024-09-12T02:08:55.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.107+0000 7f9aa046e740 1 -- 172.21.15.16:0/3662769799 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9a880b80a0 con 0x555bb7c92b40 2024-09-12T02:08:55.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.107+0000 7f9a76ffd640 1 -- 172.21.15.16:0/3662769799 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9a8c0052b0 con 0x555bb7c92b40 2024-09-12T02:08:55.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.107+0000 7f9aa046e740 10 client.17601 fetch_fsmap learned FSMap version 25 2024-09-12T02:08:55.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.107+0000 7f9aa046e740 1 -- 172.21.15.16:0/3662769799 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555bb7d1b800 con 0x555bb7c92b40 2024-09-12T02:08:55.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.107+0000 7f9a76ffd640 1 -- 172.21.15.16:0/3662769799 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a880b80a0 con 0x555bb7c92b40 2024-09-12T02:08:55.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.107+0000 7f9a76ffd640 1 -- 172.21.15.16:0/3662769799 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f9a8c029450 con 0x555bb7c92b40 2024-09-12T02:08:55.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.107+0000 7f9aa046e740 10 client.17601 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:08:55.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.107+0000 7f9aa046e740 10 client.17601 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:08:55.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.107+0000 7f9aa046e740 20 client.17601 populate_metadata read hostname 'smithi016' 2024-09-12T02:08:55.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.107+0000 7f9aa046e740 1 --2- 172.21.15.16:0/3662769799 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x555bb7d1bd50 0x555bb7d1e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:55.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.108+0000 7f9aa046e740 4 client.17601 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T02:08:55.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.108+0000 7f9aa046e740 1 -- 172.21.15.16:0/3662769799 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555bb7bfed70 con 0x555bb7d1bd50 2024-09-12T02:08:55.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.108+0000 7f9a977fe640 1 --2- 172.21.15.16:0/3662769799 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x555bb7d1bd50 0x555bb7d1e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:55.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.108+0000 7f9a977fe640 1 --2- 172.21.15.16:0/3662769799 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x555bb7d1bd50 0x555bb7d1e170 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=35354e50287c4d00 server_cookie=a30e887212087b66 in_seq=0 out_seq=0 2024-09-12T02:08:55.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.108+0000 7f9a76ffd640 10 client.17601 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:08:55.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.108+0000 7f9a76ffd640 1 -- 172.21.15.16:0/3662769799 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555bb7bfed70 con 0x555bb7d1bd50 2024-09-12T02:08:55.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.108+0000 7f9a76ffd640 10 client.17601 handle_command_reply: tid=0 2024-09-12T02:08:55.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.108+0000 7f9aa046e740 1 client.17601 shutdown 2024-09-12T02:08:55.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.108+0000 7f9aa046e740 1 -- 172.21.15.16:0/3662769799 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555bb7c92b40 msgr2=0x555bb7ca0ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:55.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.109+0000 7f9aa046e740 1 --2- 172.21.15.16:0/3662769799 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555bb7c92b40 0x555bb7ca0ed0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f9a8c00bdf0 tx=0x7f9a8c01cc80 comp rx=0 tx=0).stop 2024-09-12T02:08:55.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.109+0000 7f9aa046e740 1 -- 172.21.15.16:0/3662769799 shutdown_connections 2024-09-12T02:08:55.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.109+0000 7f9aa046e740 1 --2- 172.21.15.16:0/3662769799 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x555bb7d1bd50 0x555bb7d1e170 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:55.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.109+0000 7f9aa046e740 1 --2- 172.21.15.16:0/3662769799 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555bb7c9c370 0x555bb7d19ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:55.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.109+0000 7f9aa046e740 1 --2- 172.21.15.16:0/3662769799 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555bb7c92b40 0x555bb7ca0ed0 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:55.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.109+0000 7f9aa046e740 1 --2- 172.21.15.16:0/3662769799 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555bb7c915d0 0x555bb7ca0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:55.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.109+0000 7f9aa046e740 1 -- 172.21.15.16:0/3662769799 >> 172.21.15.16:0/3662769799 conn(0x555bb7c5a310 msgr2=0x555bb7c9b080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:55.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.109+0000 7f9a76ffd640 0 client.17601 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:08:55.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.109+0000 7f9aa046e740 1 -- 172.21.15.16:0/3662769799 shutdown_connections 2024-09-12T02:08:55.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.109+0000 7f9aa046e740 1 -- 172.21.15.16:0/3662769799 wait complete. 2024-09-12T02:08:55.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.109+0000 7f9aa046e740 20 client.17601 trim_cache size 0 max 16384 2024-09-12T02:08:55.115 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T02:08:55.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.109+0000 7f9a9e095640 1 -- 172.21.15.16:0/395557954 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f9a800a2b80 msgr2=0x7f9a800a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:55.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.109+0000 7f9a9e095640 1 --2- 172.21.15.16:0/395557954 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f9a800a2b80 0x7f9a800a5040 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f9a90005fd0 tx=0x7f9a900096a0 comp rx=0 tx=0).stop 2024-09-12T02:08:55.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.109+0000 7f9a9e095640 1 -- 172.21.15.16:0/395557954 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9a981002f0 msgr2=0x7f9a98198500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:55.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.109+0000 7f9a9e095640 1 --2- 172.21.15.16:0/395557954 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9a981002f0 0x7f9a98198500 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9a88009230 tx=0x7f9a88004270 comp rx=0 tx=0).stop 2024-09-12T02:08:55.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.109+0000 7f9a9e095640 1 -- 172.21.15.16:0/395557954 shutdown_connections 2024-09-12T02:08:55.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.109+0000 7f9a9e095640 1 --2- 172.21.15.16:0/395557954 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f9a800a2b80 0x7f9a800a5040 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:55.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.109+0000 7f9a9e095640 1 --2- 172.21.15.16:0/395557954 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a98106f50 0x7f9a9819cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:55.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.109+0000 7f9a9e095640 1 --2- 172.21.15.16:0/395557954 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a98100ca0 0x7f9a98198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:55.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.109+0000 7f9a9e095640 1 --2- 172.21.15.16:0/395557954 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9a981002f0 0x7f9a98198500 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:55.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.109+0000 7f9a9e095640 1 -- 172.21.15.16:0/395557954 >> 172.21.15.16:0/395557954 conn(0x7f9a98077210 msgr2=0x7f9a98105240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:55.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.110+0000 7f9a9e095640 1 -- 172.21.15.16:0/395557954 shutdown_connections 2024-09-12T02:08:55.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:55.110+0000 7f9a9e095640 1 -- 172.21.15.16:0/395557954 wait complete. 2024-09-12T02:08:56.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:08:56 smithi184 ceph-mon[35593]: pgmap v545: 97 pgs: 97 active+clean; 3.7 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 5.4 KiB/s wr, 11 op/s 2024-09-12T02:08:56.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:08:56 smithi033 ceph-mon[35332]: pgmap v545: 97 pgs: 97 active+clean; 3.7 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 5.4 KiB/s wr, 11 op/s 2024-09-12T02:08:56.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:08:56 smithi016 ceph-mon[30947]: pgmap v545: 97 pgs: 97 active+clean; 3.7 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 5.4 KiB/s wr, 11 op/s 2024-09-12T02:08:56.812 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-12T02:08:56.812 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-12T02:08:57.083 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:08:57.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.577+0000 7facfbdfe640 1 -- 172.21.15.16:0/3284651801 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facf4101df0 msgr2=0x7facf41021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:57.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.577+0000 7facfbdfe640 1 --2- 172.21.15.16:0/3284651801 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facf4101df0 0x7facf41021d0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7facf0009900 tx=0x7facf002a3f0 comp rx=0 tx=0).stop 2024-09-12T02:08:57.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.577+0000 7facfbdfe640 1 -- 172.21.15.16:0/3284651801 shutdown_connections 2024-09-12T02:08:57.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.577+0000 7facfbdfe640 1 --2- 172.21.15.16:0/3284651801 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7facf4106f20 0x7facf410eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:57.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.577+0000 7facfbdfe640 1 --2- 172.21.15.16:0/3284651801 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7facf41027a0 0x7facf41067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:57.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.577+0000 7facfbdfe640 1 --2- 172.21.15.16:0/3284651801 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facf4101df0 0x7facf41021d0 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:57.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.577+0000 7facfbdfe640 1 -- 172.21.15.16:0/3284651801 >> 172.21.15.16:0/3284651801 conn(0x7facf40fd8f0 msgr2=0x7facf40ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:57.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.577+0000 7facfbdfe640 1 -- 172.21.15.16:0/3284651801 shutdown_connections 2024-09-12T02:08:57.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.577+0000 7facfbdfe640 1 -- 172.21.15.16:0/3284651801 wait complete. 2024-09-12T02:08:57.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.577+0000 7facfbdfe640 1 Processor -- start 2024-09-12T02:08:57.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.577+0000 7facfbdfe640 1 -- start start 2024-09-12T02:08:57.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.578+0000 7facfbdfe640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7facf4101df0 0x7facf40744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:57.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.578+0000 7facfbdfe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7facf41027a0 0x7facf4074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:57.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.578+0000 7facfbdfe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facf4106f20 0x7facf4074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:57.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.578+0000 7facfbdfe640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7facf4111fa0 con 0x7facf4106f20 2024-09-12T02:08:57.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.578+0000 7facfbdfe640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7facf4111e20 con 0x7facf41027a0 2024-09-12T02:08:57.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.578+0000 7facfbdfe640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7facf4112120 con 0x7facf4101df0 2024-09-12T02:08:57.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.578+0000 7facfa39b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facf4106f20 0x7facf4074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:57.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.578+0000 7facfa39b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facf4106f20 0x7facf4074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33706/0 (socket says 172.21.15.16:33706) 2024-09-12T02:08:57.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.578+0000 7facfa39b640 1 -- 172.21.15.16:0/3755105508 learned_addr learned my addr 172.21.15.16:0/3755105508 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:08:57.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.578+0000 7facf9b9a640 1 --2- 172.21.15.16:0/3755105508 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7facf4101df0 0x7facf40744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:57.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.578+0000 7facf9399640 1 --2- 172.21.15.16:0/3755105508 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7facf41027a0 0x7facf4074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:57.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.578+0000 7facfa39b640 1 -- 172.21.15.16:0/3755105508 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7facf4101df0 msgr2=0x7facf40744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:57.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.578+0000 7facfa39b640 1 --2- 172.21.15.16:0/3755105508 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7facf4101df0 0x7facf40744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:57.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.578+0000 7facfa39b640 1 -- 172.21.15.16:0/3755105508 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7facf41027a0 msgr2=0x7facf4074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:57.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.578+0000 7facfa39b640 1 --2- 172.21.15.16:0/3755105508 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7facf41027a0 0x7facf4074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:57.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.578+0000 7facfa39b640 1 -- 172.21.15.16:0/3755105508 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facf41a6920 con 0x7facf4106f20 2024-09-12T02:08:57.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.578+0000 7facfa39b640 1 --2- 172.21.15.16:0/3755105508 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facf4106f20 0x7facf4074f70 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7facec00b970 tx=0x7facec00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:57.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.579+0000 7faceaffd640 1 -- 172.21.15.16:0/3755105508 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facec019070 con 0x7facf4106f20 2024-09-12T02:08:57.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.579+0000 7faceaffd640 1 -- 172.21.15.16:0/3755105508 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7facec00cb40 con 0x7facf4106f20 2024-09-12T02:08:57.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.579+0000 7facfbdfe640 1 -- 172.21.15.16:0/3755105508 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facf41a6c10 con 0x7facf4106f20 2024-09-12T02:08:57.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.579+0000 7faceaffd640 1 -- 172.21.15.16:0/3755105508 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facec007650 con 0x7facf4106f20 2024-09-12T02:08:57.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.579+0000 7facfbdfe640 1 -- 172.21.15.16:0/3755105508 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facf41a7150 con 0x7facf4106f20 2024-09-12T02:08:57.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.581+0000 7faceaffd640 1 -- 172.21.15.16:0/3755105508 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7facec01c070 con 0x7facf4106f20 2024-09-12T02:08:57.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.581+0000 7facfe1d7740 1 Processor -- start 2024-09-12T02:08:57.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.582+0000 7facfe1d7740 1 -- start start 2024-09-12T02:08:57.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.582+0000 7facfe1d7740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bdef1b2b60 0x55bdef1b2f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:57.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.582+0000 7facfe1d7740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bdef1b3480 0x55bdef1bd800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:57.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.582+0000 7facfe1d7740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bdef1bdd40 0x55bdef1c0130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:57.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.582+0000 7facfe1d7740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55bdef118280 con 0x55bdef1bdd40 2024-09-12T02:08:57.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.582+0000 7facfe1d7740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55bdef0bfb10 con 0x55bdef1b2b60 2024-09-12T02:08:57.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.582+0000 7facfe1d7740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55bdef114240 con 0x55bdef1b3480 2024-09-12T02:08:57.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.582+0000 7faceaffd640 1 --2- 172.21.15.16:0/3755105508 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7face00a2c00 0x7face00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:57.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.582+0000 7facf9399640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bdef1bdd40 0x55bdef1c0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:57.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.582+0000 7facf9399640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bdef1bdd40 0x55bdef1c0130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33716/0 (socket says 172.21.15.16:33716) 2024-09-12T02:08:57.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.582+0000 7facf9399640 1 -- 172.21.15.16:0/2086991108 learned_addr learned my addr 172.21.15.16:0/2086991108 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:08:57.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.582+0000 7facf9b9a640 1 --2- 172.21.15.16:0/2086991108 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bdef1b3480 0x55bdef1bd800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:57.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.582+0000 7facf9399640 1 --2- 172.21.15.16:0/2086991108 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bdef1b2b60 0x55bdef1b2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:57.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.582+0000 7facfa39b640 1 --2- 172.21.15.16:0/3755105508 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7face00a2c00 0x7face00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:57.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.582+0000 7faceaffd640 1 -- 172.21.15.16:0/3755105508 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7facec0ce4f0 con 0x7facf4106f20 2024-09-12T02:08:57.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.583+0000 7facf9b9a640 1 -- 172.21.15.16:0/2086991108 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bdef1b2b60 msgr2=0x55bdef1b2f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:57.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.583+0000 7facf9b9a640 1 --2- 172.21.15.16:0/2086991108 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bdef1b2b60 0x55bdef1b2f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:57.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.583+0000 7facf9b9a640 1 -- 172.21.15.16:0/2086991108 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bdef1bdd40 msgr2=0x55bdef1c0130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:57.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.583+0000 7facf9b9a640 1 --2- 172.21.15.16:0/2086991108 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bdef1bdd40 0x55bdef1c0130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:57.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.583+0000 7facf9b9a640 1 -- 172.21.15.16:0/2086991108 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bdef0cc1f0 con 0x55bdef1b3480 2024-09-12T02:08:57.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.583+0000 7facfa39b640 1 --2- 172.21.15.16:0/3755105508 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7face00a2c00 0x7face00a50c0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7facec00cfd0 tx=0x7facec090380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:57.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.583+0000 7facf9b9a640 1 --2- 172.21.15.16:0/2086991108 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bdef1b3480 0x55bdef1bd800 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7facf0005940 tx=0x7facf00316e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2692e007cf194388 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:57.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.583+0000 7facdffff640 1 -- 172.21.15.16:0/2086991108 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facf0040070 con 0x55bdef1b3480 2024-09-12T02:08:57.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.583+0000 7facdffff640 1 -- 172.21.15.16:0/2086991108 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7facf0005020 con 0x55bdef1b3480 2024-09-12T02:08:57.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.583+0000 7facdffff640 1 -- 172.21.15.16:0/2086991108 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facf0035040 con 0x55bdef1b3480 2024-09-12T02:08:57.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.584+0000 7facfe1d7740 1 -- 172.21.15.16:0/2086991108 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bdef1b3480 msgr2=0x55bdef1bd800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:57.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.584+0000 7facfe1d7740 1 --2- 172.21.15.16:0/2086991108 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bdef1b3480 0x55bdef1bd800 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7facf0005940 tx=0x7facf00316e0 comp rx=0 tx=0).stop 2024-09-12T02:08:57.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.584+0000 7facfe1d7740 1 -- 172.21.15.16:0/2086991108 shutdown_connections 2024-09-12T02:08:57.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.584+0000 7facfe1d7740 1 --2- 172.21.15.16:0/2086991108 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bdef1bdd40 0x55bdef1c0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:57.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.584+0000 7facfe1d7740 1 --2- 172.21.15.16:0/2086991108 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bdef1b3480 0x55bdef1bd800 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:57.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.584+0000 7facfe1d7740 1 --2- 172.21.15.16:0/2086991108 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bdef1b2b60 0x55bdef1b2f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:57.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.584+0000 7facfe1d7740 1 -- 172.21.15.16:0/2086991108 >> 172.21.15.16:0/2086991108 conn(0x55bdef1af0f0 msgr2=0x55bdef1af4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:57.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.584+0000 7facfe1d7740 1 -- 172.21.15.16:0/2086991108 shutdown_connections 2024-09-12T02:08:57.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.584+0000 7facfe1d7740 1 -- 172.21.15.16:0/2086991108 wait complete. 2024-09-12T02:08:57.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.585+0000 7facfe1d7740 1 -- start start 2024-09-12T02:08:57.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.585+0000 7facfe1d7740 1 Processor -- start 2024-09-12T02:08:57.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.585+0000 7facfe1d7740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bdef1b2b60 0x55bdef1bd800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:57.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.585+0000 7facfe1d7740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bdef1b3480 0x55bdef1bab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:57.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.585+0000 7facfe1d7740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bdef1bdd40 0x55bdef1bb2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:57.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.585+0000 7facfe1d7740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55bdef1c0910 con 0x55bdef1b3480 2024-09-12T02:08:57.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.585+0000 7facfe1d7740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55bdef0d2f10 con 0x55bdef1bdd40 2024-09-12T02:08:57.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.585+0000 7facfe1d7740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55bdef1c0a90 con 0x55bdef1b2b60 2024-09-12T02:08:57.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.585+0000 7facfe1d7740 10 client.0 _ll_register_callbacks cb 0x55bdef09ba40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:08:57.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.585+0000 7facf9b9a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bdef1b3480 0x55bdef1bab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:57.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.585+0000 7facf9b9a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bdef1b3480 0x55bdef1bab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33718/0 (socket says 172.21.15.16:33718) 2024-09-12T02:08:57.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.585+0000 7facf9b9a640 1 -- 172.21.15.16:0/96373612 learned_addr learned my addr 172.21.15.16:0/96373612 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:08:57.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.586+0000 7facf9399640 1 --2- 172.21.15.16:0/96373612 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bdef1bdd40 0x55bdef1bb2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:57.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.586+0000 7facf9399640 1 --2- 172.21.15.16:0/96373612 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bdef1b2b60 0x55bdef1bd800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:57.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.586+0000 7facf9b9a640 1 -- 172.21.15.16:0/96373612 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bdef1b2b60 msgr2=0x55bdef1bd800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:57.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.586+0000 7facf9b9a640 1 --2- 172.21.15.16:0/96373612 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bdef1b2b60 0x55bdef1bd800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:57.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.586+0000 7facf9b9a640 1 -- 172.21.15.16:0/96373612 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bdef1bdd40 msgr2=0x55bdef1bb2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:57.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.586+0000 7facf9b9a640 1 --2- 172.21.15.16:0/96373612 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bdef1bdd40 0x55bdef1bb2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:57.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.586+0000 7facf9b9a640 1 -- 172.21.15.16:0/96373612 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bdeee896a0 con 0x55bdef1b3480 2024-09-12T02:08:57.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.586+0000 7facf9b9a640 1 --2- 172.21.15.16:0/96373612 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bdef1b3480 0x55bdef1bab50 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7facf002ab20 tx=0x7facf002ab50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c2b757381abc516 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:57.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.586+0000 7facdd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:08:57.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.586+0000 7facdd7fa640 1 -- 172.21.15.16:0/96373612 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facf0040030 con 0x55bdef1b3480 2024-09-12T02:08:57.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.586+0000 7facfe1d7740 1 -- 172.21.15.16:0/96373612 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7facf0005020 con 0x55bdef1b3480 2024-09-12T02:08:57.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.586+0000 7facdd7fa640 1 -- 172.21.15.16:0/96373612 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7facf00053e0 con 0x55bdef1b3480 2024-09-12T02:08:57.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.586+0000 7facdd7fa640 1 -- 172.21.15.16:0/96373612 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7facf0035070 con 0x55bdef1b3480 2024-09-12T02:08:57.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.586+0000 7facfe1d7740 10 client.17613 fetch_fsmap learned FSMap version 25 2024-09-12T02:08:57.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.586+0000 7facfe1d7740 1 -- 172.21.15.16:0/96373612 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55bdef1cc440 con 0x55bdef1b3480 2024-09-12T02:08:57.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.587+0000 7facdd7fa640 1 -- 172.21.15.16:0/96373612 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7facf0005020 con 0x55bdef1b3480 2024-09-12T02:08:57.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.587+0000 7facdd7fa640 1 -- 172.21.15.16:0/96373612 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7facf003c900 con 0x55bdef1b3480 2024-09-12T02:08:57.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.587+0000 7facfe1d7740 10 client.17613 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:08:57.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.587+0000 7facfe1d7740 10 client.17613 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:08:57.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.587+0000 7facfe1d7740 20 client.17613 populate_metadata read hostname 'smithi016' 2024-09-12T02:08:57.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.587+0000 7facfe1d7740 1 --2- 172.21.15.16:0/96373612 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55bdef1cd790 0x55bdef1cfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:57.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.587+0000 7facfe1d7740 4 client.17613 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:08:57.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.587+0000 7facfe1d7740 1 -- 172.21.15.16:0/96373612 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bdef1257f0 con 0x55bdef1cd790 2024-09-12T02:08:57.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.587+0000 7facf9399640 1 --2- 172.21.15.16:0/96373612 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55bdef1cd790 0x55bdef1cfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:57.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.588+0000 7facf9399640 1 --2- 172.21.15.16:0/96373612 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55bdef1cd790 0x55bdef1cfb80 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d36d74ada05bec03 server_cookie=428fd55457bef127 in_seq=0 out_seq=0 2024-09-12T02:08:57.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.588+0000 7facdd7fa640 10 client.17613 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:08:57.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.589+0000 7facdd7fa640 1 -- 172.21.15.16:0/96373612 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bdef1257f0 con 0x55bdef1cd790 2024-09-12T02:08:57.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.589+0000 7facdd7fa640 10 client.17613 handle_command_reply: tid=0 2024-09-12T02:08:57.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.589+0000 7facfe1d7740 1 client.17613 shutdown 2024-09-12T02:08:57.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.589+0000 7facfe1d7740 1 -- 172.21.15.16:0/96373612 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bdef1b3480 msgr2=0x55bdef1bab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:57.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.589+0000 7facfe1d7740 1 --2- 172.21.15.16:0/96373612 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bdef1b3480 0x55bdef1bab50 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7facf002ab20 tx=0x7facf002ab50 comp rx=0 tx=0).stop 2024-09-12T02:08:57.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.589+0000 7facfe1d7740 1 -- 172.21.15.16:0/96373612 shutdown_connections 2024-09-12T02:08:57.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.589+0000 7facfe1d7740 1 --2- 172.21.15.16:0/96373612 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55bdef1cd790 0x55bdef1cfb80 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:57.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.589+0000 7facfe1d7740 1 --2- 172.21.15.16:0/96373612 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bdef1bdd40 0x55bdef1bb2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:57.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.589+0000 7facfe1d7740 1 --2- 172.21.15.16:0/96373612 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bdef1b3480 0x55bdef1bab50 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:57.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.589+0000 7facfe1d7740 1 --2- 172.21.15.16:0/96373612 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bdef1b2b60 0x55bdef1bd800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:57.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.589+0000 7facdd7fa640 0 client.17613 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:08:57.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.589+0000 7facfe1d7740 1 -- 172.21.15.16:0/96373612 >> 172.21.15.16:0/96373612 conn(0x55bdef1af0f0 msgr2=0x55bdef1ae270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:57.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.589+0000 7facfe1d7740 1 -- 172.21.15.16:0/96373612 shutdown_connections 2024-09-12T02:08:57.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.589+0000 7facfe1d7740 1 -- 172.21.15.16:0/96373612 wait complete. 2024-09-12T02:08:57.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.589+0000 7facfe1d7740 20 client.17613 trim_cache size 0 max 16384 2024-09-12T02:08:57.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.607+0000 7facfe1d7740 1 Processor -- start 2024-09-12T02:08:57.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.607+0000 7facfe1d7740 1 -- start start 2024-09-12T02:08:57.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.607+0000 7facfe1d7740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bdef1b2b60 0x55bdef1b55c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:57.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.607+0000 7facfe1d7740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bdef1b3480 0x55bdef1d5120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:57.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.607+0000 7facfe1d7740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bdef1bdd40 0x55bdef1d5660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:57.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.607+0000 7facfe1d7740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7facf41b1dd0 con 0x55bdef1bdd40 2024-09-12T02:08:57.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.607+0000 7facfe1d7740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7facf41af6a0 con 0x55bdef1b2b60 2024-09-12T02:08:57.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.607+0000 7facfe1d7740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7facf41ad5a0 con 0x55bdef1b3480 2024-09-12T02:08:57.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.607+0000 7facf9399640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bdef1bdd40 0x55bdef1d5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:57.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.607+0000 7facf9399640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bdef1bdd40 0x55bdef1d5660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33724/0 (socket says 172.21.15.16:33724) 2024-09-12T02:08:57.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.607+0000 7facf9399640 1 -- 172.21.15.16:0/1219522916 learned_addr learned my addr 172.21.15.16:0/1219522916 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:08:57.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.607+0000 7facf9b9a640 1 --2- 172.21.15.16:0/1219522916 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bdef1b3480 0x55bdef1d5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:57.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.607+0000 7facf9399640 1 --2- 172.21.15.16:0/1219522916 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bdef1b2b60 0x55bdef1b55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:57.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.608+0000 7facf9399640 1 -- 172.21.15.16:0/1219522916 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bdef1b3480 msgr2=0x55bdef1d5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:57.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.608+0000 7facf9399640 1 --2- 172.21.15.16:0/1219522916 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bdef1b3480 0x55bdef1d5120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:57.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.608+0000 7facf9399640 1 -- 172.21.15.16:0/1219522916 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bdef1b2b60 msgr2=0x55bdef1b55c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:57.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.608+0000 7facf9399640 1 --2- 172.21.15.16:0/1219522916 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bdef1b2b60 0x55bdef1b55c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:57.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.608+0000 7facf9399640 1 -- 172.21.15.16:0/1219522916 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bdef1b32b0 con 0x55bdef1bdd40 2024-09-12T02:08:57.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.608+0000 7facf9399640 1 --2- 172.21.15.16:0/1219522916 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bdef1bdd40 0x55bdef1d5660 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x55bdef1b0470 tx=0x7face400bb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=535d707ae992d0aa server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:57.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.608+0000 7facdf7fe640 1 -- 172.21.15.16:0/1219522916 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7face4021630 con 0x55bdef1bdd40 2024-09-12T02:08:57.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.608+0000 7facdf7fe640 1 -- 172.21.15.16:0/1219522916 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7face4021cb0 con 0x55bdef1bdd40 2024-09-12T02:08:57.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.608+0000 7facdf7fe640 1 -- 172.21.15.16:0/1219522916 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7face4011dd0 con 0x55bdef1bdd40 2024-09-12T02:08:57.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.609+0000 7facfe1d7740 1 -- 172.21.15.16:0/1219522916 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bdef1bdd40 msgr2=0x55bdef1d5660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:57.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.609+0000 7facfe1d7740 1 --2- 172.21.15.16:0/1219522916 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bdef1bdd40 0x55bdef1d5660 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x55bdef1b0470 tx=0x7face400bb80 comp rx=0 tx=0).stop 2024-09-12T02:08:57.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.609+0000 7facfe1d7740 1 -- 172.21.15.16:0/1219522916 shutdown_connections 2024-09-12T02:08:57.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.609+0000 7facfe1d7740 1 --2- 172.21.15.16:0/1219522916 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bdef1bdd40 0x55bdef1d5660 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:57.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.609+0000 7facfe1d7740 1 --2- 172.21.15.16:0/1219522916 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bdef1b3480 0x55bdef1d5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:57.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.609+0000 7facfe1d7740 1 --2- 172.21.15.16:0/1219522916 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bdef1b2b60 0x55bdef1b55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:57.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.609+0000 7facfe1d7740 1 -- 172.21.15.16:0/1219522916 >> 172.21.15.16:0/1219522916 conn(0x55bdef1af0f0 msgr2=0x55bdef1bcac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:57.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.609+0000 7facfe1d7740 1 -- 172.21.15.16:0/1219522916 shutdown_connections 2024-09-12T02:08:57.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.609+0000 7facfe1d7740 1 -- 172.21.15.16:0/1219522916 wait complete. 2024-09-12T02:08:57.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.609+0000 7facfe1d7740 1 -- start start 2024-09-12T02:08:57.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.609+0000 7facfe1d7740 1 Processor -- start 2024-09-12T02:08:57.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.610+0000 7facfe1d7740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bdef1b2b60 0x55bdef1c5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:57.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.610+0000 7facfe1d7740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bdef1b3480 0x55bdef1c6220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:57.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.610+0000 7facfe1d7740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bdef1bdd40 0x55bdef1c2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:57.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.610+0000 7facfe1d7740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7facf41adbf0 con 0x55bdef1bdd40 2024-09-12T02:08:57.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.610+0000 7facfe1d7740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7facf41aefd0 con 0x55bdef1b3480 2024-09-12T02:08:57.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.610+0000 7facfe1d7740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7facf41ae230 con 0x55bdef1b2b60 2024-09-12T02:08:57.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.610+0000 7facfe1d7740 10 client.0 _ll_register_callbacks cb 0x7face400ba20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:08:57.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.610+0000 7facf9399640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bdef1bdd40 0x55bdef1c2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:57.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.610+0000 7facf9399640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bdef1bdd40 0x55bdef1c2cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33728/0 (socket says 172.21.15.16:33728) 2024-09-12T02:08:57.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.610+0000 7facf9399640 1 -- 172.21.15.16:0/3439389361 learned_addr learned my addr 172.21.15.16:0/3439389361 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:08:57.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.610+0000 7facf9399640 1 --2- 172.21.15.16:0/3439389361 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bdef1b2b60 0x55bdef1c5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:57.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.610+0000 7facf9b9a640 1 --2- 172.21.15.16:0/3439389361 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bdef1b3480 0x55bdef1c6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:57.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.610+0000 7facf9399640 1 -- 172.21.15.16:0/3439389361 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bdef1b2b60 msgr2=0x55bdef1c5ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:57.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.610+0000 7facf9399640 1 --2- 172.21.15.16:0/3439389361 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bdef1b2b60 0x55bdef1c5ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:57.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.610+0000 7facf9399640 1 -- 172.21.15.16:0/3439389361 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bdef1b3480 msgr2=0x55bdef1c6220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:57.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.610+0000 7facf9399640 1 --2- 172.21.15.16:0/3439389361 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bdef1b3480 0x55bdef1c6220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:57.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.610+0000 7facf9399640 1 -- 172.21.15.16:0/3439389361 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bdef1bdb70 con 0x55bdef1bdd40 2024-09-12T02:08:57.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.610+0000 7facf9399640 1 --2- 172.21.15.16:0/3439389361 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bdef1bdd40 0x55bdef1c2cb0 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7face4011ab0 tx=0x7face40119b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13f93b214f4d9253 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:57.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.610+0000 7facdd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:08:57.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.611+0000 7facdd7fa640 1 -- 172.21.15.16:0/3439389361 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7face4011b50 con 0x55bdef1bdd40 2024-09-12T02:08:57.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.611+0000 7facdd7fa640 1 -- 172.21.15.16:0/3439389361 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7face4011cf0 con 0x55bdef1bdd40 2024-09-12T02:08:57.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.611+0000 7facfe1d7740 1 -- 172.21.15.16:0/3439389361 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7face4021cb0 con 0x55bdef1bdd40 2024-09-12T02:08:57.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.611+0000 7facdd7fa640 1 -- 172.21.15.16:0/3439389361 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7face4005030 con 0x55bdef1bdd40 2024-09-12T02:08:57.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.611+0000 7facfe1d7740 10 client.17619 fetch_fsmap learned FSMap version 25 2024-09-12T02:08:57.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.611+0000 7facfe1d7740 1 -- 172.21.15.16:0/3439389361 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55bdef2388a0 con 0x55bdef1bdd40 2024-09-12T02:08:57.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.611+0000 7facdd7fa640 1 -- 172.21.15.16:0/3439389361 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7face4021cb0 con 0x55bdef1bdd40 2024-09-12T02:08:57.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.611+0000 7facdd7fa640 1 -- 172.21.15.16:0/3439389361 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7face4004730 con 0x55bdef1bdd40 2024-09-12T02:08:57.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.611+0000 7facfe1d7740 10 client.17619 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:08:57.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.611+0000 7facfe1d7740 10 client.17619 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:08:57.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.611+0000 7facfe1d7740 20 client.17619 populate_metadata read hostname 'smithi016' 2024-09-12T02:08:57.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.611+0000 7facfe1d7740 1 --2- 172.21.15.16:0/3439389361 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55bdef238f60 0x55bdef23b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:57.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.611+0000 7facfe1d7740 4 client.17619 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-12T02:08:57.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.611+0000 7facfe1d7740 1 -- 172.21.15.16:0/3439389361 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55bdef11fd00 con 0x55bdef238f60 2024-09-12T02:08:57.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.612+0000 7facf9b9a640 1 --2- 172.21.15.16:0/3439389361 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55bdef238f60 0x55bdef23b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:57.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.612+0000 7facf9b9a640 1 --2- 172.21.15.16:0/3439389361 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55bdef238f60 0x55bdef23b380 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46d8cce56b863bd server_cookie=af771379a7658ecb in_seq=0 out_seq=0 2024-09-12T02:08:57.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.612+0000 7facdd7fa640 10 client.17619 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:08:57.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.613+0000 7facdd7fa640 1 -- 172.21.15.16:0/3439389361 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55bdef11fd00 con 0x55bdef238f60 2024-09-12T02:08:57.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.613+0000 7facdd7fa640 10 client.17619 handle_command_reply: tid=0 2024-09-12T02:08:57.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.613+0000 7facfe1d7740 1 client.17619 shutdown 2024-09-12T02:08:57.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.613+0000 7facfe1d7740 1 -- 172.21.15.16:0/3439389361 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bdef1bdd40 msgr2=0x55bdef1c2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:57.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.613+0000 7facfe1d7740 1 --2- 172.21.15.16:0/3439389361 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bdef1bdd40 0x55bdef1c2cb0 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7face4011ab0 tx=0x7face40119b0 comp rx=0 tx=0).stop 2024-09-12T02:08:57.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.613+0000 7facfe1d7740 1 -- 172.21.15.16:0/3439389361 shutdown_connections 2024-09-12T02:08:57.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.613+0000 7facfe1d7740 1 --2- 172.21.15.16:0/3439389361 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55bdef238f60 0x55bdef23b380 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:57.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.613+0000 7facfe1d7740 1 --2- 172.21.15.16:0/3439389361 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bdef1bdd40 0x55bdef1c2cb0 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:57.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.613+0000 7facfe1d7740 1 --2- 172.21.15.16:0/3439389361 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bdef1b3480 0x55bdef1c6220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:57.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.613+0000 7facfe1d7740 1 --2- 172.21.15.16:0/3439389361 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bdef1b2b60 0x55bdef1c5ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:57.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.613+0000 7facfe1d7740 1 -- 172.21.15.16:0/3439389361 >> 172.21.15.16:0/3439389361 conn(0x55bdef1af0f0 msgr2=0x55bdef1b92f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:57.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.613+0000 7facdd7fa640 0 client.17619 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:08:57.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.614+0000 7facfe1d7740 1 -- 172.21.15.16:0/3439389361 shutdown_connections 2024-09-12T02:08:57.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.614+0000 7facfe1d7740 1 -- 172.21.15.16:0/3439389361 wait complete. 2024-09-12T02:08:57.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.614+0000 7facfe1d7740 20 client.17619 trim_cache size 0 max 16384 2024-09-12T02:08:57.626 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T02:08:57.626 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-09-12T02:08:57.626 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "0258bee0-6dc2-4035-bba2-e5c7b7137e41", 2024-09-12T02:08:57.626 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-09-12T02:08:57.626 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T02:08:57.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.614+0000 7facfbdfe640 1 -- 172.21.15.16:0/3755105508 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7face00a2c00 msgr2=0x7face00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:57.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.614+0000 7facfbdfe640 1 --2- 172.21.15.16:0/3755105508 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7face00a2c00 0x7face00a50c0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7facec00cfd0 tx=0x7facec090380 comp rx=0 tx=0).stop 2024-09-12T02:08:57.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.614+0000 7facfbdfe640 1 -- 172.21.15.16:0/3755105508 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facf4106f20 msgr2=0x7facf4074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:57.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.614+0000 7facfbdfe640 1 --2- 172.21.15.16:0/3755105508 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facf4106f20 0x7facf4074f70 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7facec00b970 tx=0x7facec00be30 comp rx=0 tx=0).stop 2024-09-12T02:08:57.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.614+0000 7facfbdfe640 1 -- 172.21.15.16:0/3755105508 shutdown_connections 2024-09-12T02:08:57.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.614+0000 7facfbdfe640 1 --2- 172.21.15.16:0/3755105508 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7face00a2c00 0x7face00a50c0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:57.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.614+0000 7facfbdfe640 1 --2- 172.21.15.16:0/3755105508 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facf4106f20 0x7facf4074f70 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:57.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.614+0000 7facfbdfe640 1 --2- 172.21.15.16:0/3755105508 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7facf41027a0 0x7facf4074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:57.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.614+0000 7facfbdfe640 1 --2- 172.21.15.16:0/3755105508 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7facf4101df0 0x7facf40744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:57.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.614+0000 7facfbdfe640 1 -- 172.21.15.16:0/3755105508 >> 172.21.15.16:0/3755105508 conn(0x7facf40fd8f0 msgr2=0x7facf40febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:57.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.614+0000 7facfbdfe640 1 -- 172.21.15.16:0/3755105508 shutdown_connections 2024-09-12T02:08:57.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:57.615+0000 7facfbdfe640 1 -- 172.21.15.16:0/3755105508 wait complete. 2024-09-12T02:08:58.237 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.235+0000 7fa6e3f28640 1 -- 172.21.15.16:0/469506665 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6dc101690 msgr2=0x7fa6dc101b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:58.237 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.235+0000 7fa6e3f28640 1 --2- 172.21.15.16:0/469506665 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6dc101690 0x7fa6dc101b40 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7fa6d4009ca0 tx=0x7fa6d402a8c0 comp rx=0 tx=0).stop 2024-09-12T02:08:58.237 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.235+0000 7fa6e3f28640 1 -- 172.21.15.16:0/469506665 shutdown_connections 2024-09-12T02:08:58.237 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.235+0000 7fa6e3f28640 1 --2- 172.21.15.16:0/469506665 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6dc101690 0x7fa6dc101b40 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:58.237 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.235+0000 7fa6e3f28640 1 --2- 172.21.15.16:0/469506665 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa6dc100cf0 0x7fa6dc101150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:58.238 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.235+0000 7fa6e3f28640 1 --2- 172.21.15.16:0/469506665 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa6dc1003b0 0x7fa6dc1007b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:58.238 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.235+0000 7fa6e3f28640 1 -- 172.21.15.16:0/469506665 >> 172.21.15.16:0/469506665 conn(0x7fa6dc0fc1e0 msgr2=0x7fa6dc0fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:58.238 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.235+0000 7fa6e3f28640 1 -- 172.21.15.16:0/469506665 shutdown_connections 2024-09-12T02:08:58.238 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.235+0000 7fa6e3f28640 1 -- 172.21.15.16:0/469506665 wait complete. 2024-09-12T02:08:58.238 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.236+0000 7fa6e3f28640 1 Processor -- start 2024-09-12T02:08:58.238 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.236+0000 7fa6e3f28640 1 -- start start 2024-09-12T02:08:58.238 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.236+0000 7fa6e3f28640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6dc1003b0 0x7fa6dc199f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:58.238 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.236+0000 7fa6e3f28640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa6dc100cf0 0x7fa6dc19a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:58.238 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.236+0000 7fa6e3f28640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa6dc101690 0x7fa6dc1a14d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:58.238 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.236+0000 7fa6e3f28640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa6dc10f9b0 con 0x7fa6dc1003b0 2024-09-12T02:08:58.238 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.236+0000 7fa6e3f28640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa6dc10f830 con 0x7fa6dc100cf0 2024-09-12T02:08:58.238 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.236+0000 7fa6e3f28640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa6dc10fb30 con 0x7fa6dc101690 2024-09-12T02:08:58.239 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.236+0000 7fa6e1cc4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6dc1003b0 0x7fa6dc199f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:58.239 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.236+0000 7fa6e1cc4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6dc1003b0 0x7fa6dc199f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33746/0 (socket says 172.21.15.16:33746) 2024-09-12T02:08:58.239 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.236+0000 7fa6e1cc4640 1 -- 172.21.15.16:0/695757455 learned_addr learned my addr 172.21.15.16:0/695757455 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:08:58.239 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.236+0000 7fa6e24c5640 1 --2- 172.21.15.16:0/695757455 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa6dc101690 0x7fa6dc1a14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:58.239 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.236+0000 7fa6e14c3640 1 --2- 172.21.15.16:0/695757455 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa6dc100cf0 0x7fa6dc19a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:58.239 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.236+0000 7fa6e1cc4640 1 -- 172.21.15.16:0/695757455 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa6dc101690 msgr2=0x7fa6dc1a14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:58.239 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.236+0000 7fa6e1cc4640 1 --2- 172.21.15.16:0/695757455 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa6dc101690 0x7fa6dc1a14d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:58.239 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.236+0000 7fa6e1cc4640 1 -- 172.21.15.16:0/695757455 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa6dc100cf0 msgr2=0x7fa6dc19a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:58.239 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.236+0000 7fa6e1cc4640 1 --2- 172.21.15.16:0/695757455 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa6dc100cf0 0x7fa6dc19a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:58.239 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.236+0000 7fa6e1cc4640 1 -- 172.21.15.16:0/695757455 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6dc1a1b40 con 0x7fa6dc1003b0 2024-09-12T02:08:58.239 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.236+0000 7fa6e24c5640 1 --2- 172.21.15.16:0/695757455 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa6dc101690 0x7fa6dc1a14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:08:58.239 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.237+0000 7fa6e1cc4640 1 --2- 172.21.15.16:0/695757455 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6dc1003b0 0x7fa6dc199f20 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7fa6d800d770 tx=0x7fa6d800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:58.239 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.237+0000 7fa6d2ffd640 1 -- 172.21.15.16:0/695757455 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6d8017070 con 0x7fa6dc1003b0 2024-09-12T02:08:58.240 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.237+0000 7fa6d2ffd640 1 -- 172.21.15.16:0/695757455 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6d8005500 con 0x7fa6dc1003b0 2024-09-12T02:08:58.240 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.237+0000 7fa6e3f28640 1 -- 172.21.15.16:0/695757455 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6dc1a1e30 con 0x7fa6dc1003b0 2024-09-12T02:08:58.240 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.237+0000 7fa6d2ffd640 1 -- 172.21.15.16:0/695757455 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6d8004e00 con 0x7fa6dc1003b0 2024-09-12T02:08:58.240 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.237+0000 7fa6e3f28640 1 -- 172.21.15.16:0/695757455 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6dc1a2240 con 0x7fa6dc1003b0 2024-09-12T02:08:58.242 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.239+0000 7fa6d2ffd640 1 -- 172.21.15.16:0/695757455 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa6d8004fa0 con 0x7fa6dc1003b0 2024-09-12T02:08:58.242 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.240+0000 7fa6e6cb7740 1 Processor -- start 2024-09-12T02:08:58.242 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.240+0000 7fa6e6cb7740 1 -- start start 2024-09-12T02:08:58.242 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.240+0000 7fa6e6cb7740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5603e9ad49a0 0x5603e9ad4da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:58.242 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.240+0000 7fa6e6cb7740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5603e9ad52e0 0x5603e9ad9da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:58.242 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.240+0000 7fa6e6cb7740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5603e9ada370 0x5603e9adc830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:58.242 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.240+0000 7fa6e6cb7740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5603e9a382f0 con 0x5603e9ad52e0 2024-09-12T02:08:58.242 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.240+0000 7fa6e6cb7740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5603e99e0270 con 0x5603e9ada370 2024-09-12T02:08:58.242 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.240+0000 7fa6e6cb7740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5603e9a342b0 con 0x5603e9ad49a0 2024-09-12T02:08:58.242 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.241+0000 7fa6d2ffd640 1 --2- 172.21.15.16:0/695757455 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fa6c80a2cb0 0x7fa6c80a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:58.243 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.241+0000 7fa6e1cc4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5603e9ad52e0 0x5603e9ad9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:58.243 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.241+0000 7fa6e14c3640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5603e9ada370 0x5603e9adc830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:58.243 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.241+0000 7fa6e1cc4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5603e9ad52e0 0x5603e9ad9da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33750/0 (socket says 172.21.15.16:33750) 2024-09-12T02:08:58.243 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.241+0000 7fa6e1cc4640 1 -- 172.21.15.16:0/476710745 learned_addr learned my addr 172.21.15.16:0/476710745 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:08:58.243 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.241+0000 7fa6e24c5640 1 --2- 172.21.15.16:0/476710745 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5603e9ad49a0 0x5603e9ad4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:58.243 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.241+0000 7fa6e24c5640 1 --2- 172.21.15.16:0/695757455 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fa6c80a2cb0 0x7fa6c80a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:58.243 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.241+0000 7fa6d2ffd640 1 -- 172.21.15.16:0/695757455 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fa6d80cea30 con 0x7fa6dc1003b0 2024-09-12T02:08:58.243 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.241+0000 7fa6e14c3640 1 -- 172.21.15.16:0/476710745 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5603e9ad49a0 msgr2=0x5603e9ad4da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:58.243 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.241+0000 7fa6e14c3640 1 --2- 172.21.15.16:0/476710745 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5603e9ad49a0 0x5603e9ad4da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:58.243 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.241+0000 7fa6e14c3640 1 -- 172.21.15.16:0/476710745 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5603e9ad52e0 msgr2=0x5603e9ad9da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:58.243 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.241+0000 7fa6e14c3640 1 --2- 172.21.15.16:0/476710745 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5603e9ad52e0 0x5603e9ad9da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:58.243 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.241+0000 7fa6e14c3640 1 -- 172.21.15.16:0/476710745 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603e99df340 con 0x5603e9ada370 2024-09-12T02:08:58.243 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.241+0000 7fa6e24c5640 1 --2- 172.21.15.16:0/695757455 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fa6c80a2cb0 0x7fa6c80a5170 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fa6d4005cc0 tx=0x7fa6d40095c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:58.244 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.241+0000 7fa6e14c3640 1 --2- 172.21.15.16:0/476710745 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5603e9ada370 0x5603e9adc830 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc00cb70 tx=0x7fa6cc00e720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e68087866686cc3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:58.244 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.242+0000 7fa6c7fff640 1 -- 172.21.15.16:0/476710745 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6cc004220 con 0x5603e9ada370 2024-09-12T02:08:58.244 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.242+0000 7fa6c7fff640 1 -- 172.21.15.16:0/476710745 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6cc0043c0 con 0x5603e9ada370 2024-09-12T02:08:58.244 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.242+0000 7fa6c7fff640 1 -- 172.21.15.16:0/476710745 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6cc004de0 con 0x5603e9ada370 2024-09-12T02:08:58.248 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.242+0000 7fa6e6cb7740 1 -- 172.21.15.16:0/476710745 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5603e9ada370 msgr2=0x5603e9adc830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:58.248 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.242+0000 7fa6e6cb7740 1 --2- 172.21.15.16:0/476710745 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5603e9ada370 0x5603e9adc830 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc00cb70 tx=0x7fa6cc00e720 comp rx=0 tx=0).stop 2024-09-12T02:08:58.248 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.242+0000 7fa6e6cb7740 1 -- 172.21.15.16:0/476710745 shutdown_connections 2024-09-12T02:08:58.248 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.242+0000 7fa6e6cb7740 1 --2- 172.21.15.16:0/476710745 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5603e9ada370 0x5603e9adc830 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:58.248 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.242+0000 7fa6e6cb7740 1 --2- 172.21.15.16:0/476710745 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5603e9ad52e0 0x5603e9ad9da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:58.248 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.242+0000 7fa6e6cb7740 1 --2- 172.21.15.16:0/476710745 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5603e9ad49a0 0x5603e9ad4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:58.248 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.242+0000 7fa6e6cb7740 1 -- 172.21.15.16:0/476710745 >> 172.21.15.16:0/476710745 conn(0x5603e9ad0010 msgr2=0x5603e9ad0410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:58.249 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.242+0000 7fa6e6cb7740 1 -- 172.21.15.16:0/476710745 shutdown_connections 2024-09-12T02:08:58.249 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.242+0000 7fa6e6cb7740 1 -- 172.21.15.16:0/476710745 wait complete. 2024-09-12T02:08:58.249 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.243+0000 7fa6e6cb7740 1 -- start start 2024-09-12T02:08:58.249 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.243+0000 7fa6e6cb7740 1 Processor -- start 2024-09-12T02:08:58.249 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.243+0000 7fa6e6cb7740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5603e9ad49a0 0x5603e9adeb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:58.249 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.243+0000 7fa6e6cb7740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5603e9ad52e0 0x5603e9adf070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:58.249 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.243+0000 7fa6e6cb7740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5603e9ada370 0x5603e9ae60b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:58.249 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.243+0000 7fa6e6cb7740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5603e9add340 con 0x5603e9ad49a0 2024-09-12T02:08:58.249 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.243+0000 7fa6e6cb7740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5603e99c1290 con 0x5603e9ada370 2024-09-12T02:08:58.249 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.243+0000 7fa6e6cb7740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5603e9add4c0 con 0x5603e9ad52e0 2024-09-12T02:08:58.249 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.243+0000 7fa6e6cb7740 10 client.0 _ll_register_callbacks cb 0x5603e99f2140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:08:58.249 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.243+0000 7fa6e1cc4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5603e9ad49a0 0x5603e9adeb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:58.249 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.243+0000 7fa6e1cc4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5603e9ad49a0 0x5603e9adeb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33760/0 (socket says 172.21.15.16:33760) 2024-09-12T02:08:58.250 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.243+0000 7fa6e1cc4640 1 -- 172.21.15.16:0/2821694741 learned_addr learned my addr 172.21.15.16:0/2821694741 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:08:58.250 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.243+0000 7fa6e24c5640 1 --2- 172.21.15.16:0/2821694741 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5603e9ada370 0x5603e9ae60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:58.250 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.244+0000 7fa6e14c3640 1 --2- 172.21.15.16:0/2821694741 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5603e9ad52e0 0x5603e9adf070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:58.250 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.244+0000 7fa6e1cc4640 1 -- 172.21.15.16:0/2821694741 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5603e9ad52e0 msgr2=0x5603e9adf070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:58.250 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.244+0000 7fa6e1cc4640 1 --2- 172.21.15.16:0/2821694741 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5603e9ad52e0 0x5603e9adf070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:58.250 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.244+0000 7fa6e1cc4640 1 -- 172.21.15.16:0/2821694741 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5603e9ada370 msgr2=0x5603e9ae60b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:58.250 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.244+0000 7fa6e1cc4640 1 --2- 172.21.15.16:0/2821694741 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5603e9ada370 0x5603e9ae60b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:58.250 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.244+0000 7fa6e1cc4640 1 -- 172.21.15.16:0/2821694741 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5603e974b210 con 0x5603e9ad49a0 2024-09-12T02:08:58.250 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.244+0000 7fa6e14c3640 1 --2- 172.21.15.16:0/2821694741 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5603e9ad52e0 0x5603e9adf070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:08:58.250 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.244+0000 7fa6e1cc4640 1 --2- 172.21.15.16:0/2821694741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5603e9ad49a0 0x5603e9adeb30 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7fa6d809b2a0 tx=0x7fa6d80048a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f04879300afa607b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:58.250 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.244+0000 7fa6c57fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:08:58.250 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.244+0000 7fa6c57fa640 1 -- 172.21.15.16:0/2821694741 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6d8020040 con 0x5603e9ad49a0 2024-09-12T02:08:58.250 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.244+0000 7fa6c57fa640 1 -- 172.21.15.16:0/2821694741 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fa6d8009cc0 con 0x5603e9ad49a0 2024-09-12T02:08:58.251 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.244+0000 7fa6e6cb7740 1 -- 172.21.15.16:0/2821694741 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa6cc0043c0 con 0x5603e9ad49a0 2024-09-12T02:08:58.251 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.244+0000 7fa6c57fa640 1 -- 172.21.15.16:0/2821694741 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa6d809b310 con 0x5603e9ad49a0 2024-09-12T02:08:58.251 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.244+0000 7fa6e6cb7740 10 client.17631 fetch_fsmap learned FSMap version 25 2024-09-12T02:08:58.251 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.244+0000 7fa6e6cb7740 1 -- 172.21.15.16:0/2821694741 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5603e9817620 con 0x5603e9ad49a0 2024-09-12T02:08:58.251 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.244+0000 7fa6c57fa640 1 -- 172.21.15.16:0/2821694741 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6cc0043c0 con 0x5603e9ad49a0 2024-09-12T02:08:58.251 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.245+0000 7fa6c57fa640 1 -- 172.21.15.16:0/2821694741 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fa6d80b40f0 con 0x5603e9ad49a0 2024-09-12T02:08:58.251 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.245+0000 7fa6e6cb7740 10 client.17631 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:08:58.251 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.245+0000 7fa6e6cb7740 10 client.17631 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:08:58.251 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.245+0000 7fa6e6cb7740 20 client.17631 populate_metadata read hostname 'smithi016' 2024-09-12T02:08:58.251 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.245+0000 7fa6e6cb7740 1 --2- 172.21.15.16:0/2821694741 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5603e9aea7a0 0x5603e9aecb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:58.251 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.245+0000 7fa6e6cb7740 4 client.17631 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:08:58.251 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.245+0000 7fa6e6cb7740 1 -- 172.21.15.16:0/2821694741 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5603e9a45a20 con 0x5603e9aea7a0 2024-09-12T02:08:58.251 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.245+0000 7fa6e14c3640 1 --2- 172.21.15.16:0/2821694741 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5603e9aea7a0 0x5603e9aecb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:58.252 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.245+0000 7fa6e14c3640 1 --2- 172.21.15.16:0/2821694741 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5603e9aea7a0 0x5603e9aecb90 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15422d1b823673e8 server_cookie=80536f2bca03b43f in_seq=0 out_seq=0 2024-09-12T02:08:58.252 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.245+0000 7fa6c57fa640 10 client.17631 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:08:58.252 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.246+0000 7fa6c57fa640 1 -- 172.21.15.16:0/2821694741 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5603e9a45a20 con 0x5603e9aea7a0 2024-09-12T02:08:58.252 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.246+0000 7fa6c57fa640 10 client.17631 handle_command_reply: tid=0 2024-09-12T02:08:58.252 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.246+0000 7fa6e6cb7740 1 client.17631 shutdown 2024-09-12T02:08:58.252 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.246+0000 7fa6e6cb7740 1 -- 172.21.15.16:0/2821694741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5603e9ad49a0 msgr2=0x5603e9adeb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:58.252 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.246+0000 7fa6e6cb7740 1 --2- 172.21.15.16:0/2821694741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5603e9ad49a0 0x5603e9adeb30 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7fa6d809b2a0 tx=0x7fa6d80048a0 comp rx=0 tx=0).stop 2024-09-12T02:08:58.252 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.246+0000 7fa6e6cb7740 1 -- 172.21.15.16:0/2821694741 shutdown_connections 2024-09-12T02:08:58.252 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.246+0000 7fa6e6cb7740 1 --2- 172.21.15.16:0/2821694741 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5603e9aea7a0 0x5603e9aecb90 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:58.252 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.246+0000 7fa6e6cb7740 1 --2- 172.21.15.16:0/2821694741 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5603e9ada370 0x5603e9ae60b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:58.252 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.246+0000 7fa6e6cb7740 1 --2- 172.21.15.16:0/2821694741 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5603e9ad52e0 0x5603e9adf070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:58.252 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.246+0000 7fa6e6cb7740 1 --2- 172.21.15.16:0/2821694741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5603e9ad49a0 0x5603e9adeb30 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:58.252 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.246+0000 7fa6c57fa640 0 client.17631 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:08:58.253 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.246+0000 7fa6e6cb7740 1 -- 172.21.15.16:0/2821694741 >> 172.21.15.16:0/2821694741 conn(0x5603e9ad0010 msgr2=0x5603e9ad1820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:58.253 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.246+0000 7fa6e6cb7740 1 -- 172.21.15.16:0/2821694741 shutdown_connections 2024-09-12T02:08:58.253 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.246+0000 7fa6e6cb7740 1 -- 172.21.15.16:0/2821694741 wait complete. 2024-09-12T02:08:58.253 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.247+0000 7fa6e6cb7740 20 client.17631 trim_cache size 0 max 16384 2024-09-12T02:08:58.262 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.261+0000 7fa6e6cb7740 1 Processor -- start 2024-09-12T02:08:58.262 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.261+0000 7fa6e6cb7740 1 -- start start 2024-09-12T02:08:58.262 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.261+0000 7fa6e6cb7740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5603e9ad49a0 0x5603e9acf1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:58.262 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.261+0000 7fa6e6cb7740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5603e9ad52e0 0x5603e9ae6070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:58.262 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.261+0000 7fa6e6cb7740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5603e9ada370 0x5603e9af14e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:58.262 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.261+0000 7fa6e6cb7740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa6dc1ad820 con 0x5603e9ad49a0 2024-09-12T02:08:58.262 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.261+0000 7fa6e6cb7740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa6dc1ab0f0 con 0x5603e9ad52e0 2024-09-12T02:08:58.262 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.261+0000 7fa6e6cb7740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa6dc1a8f40 con 0x5603e9ada370 2024-09-12T02:08:58.262 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.261+0000 7fa6e1cc4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5603e9ad49a0 0x5603e9acf1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:58.262 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.261+0000 7fa6e1cc4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5603e9ad49a0 0x5603e9acf1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33764/0 (socket says 172.21.15.16:33764) 2024-09-12T02:08:58.263 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.261+0000 7fa6e1cc4640 1 -- 172.21.15.16:0/544699950 learned_addr learned my addr 172.21.15.16:0/544699950 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:08:58.263 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.261+0000 7fa6e1cc4640 1 -- 172.21.15.16:0/544699950 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5603e9ada370 msgr2=0x5603e9af14e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:58.263 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.261+0000 7fa6e1cc4640 1 --2- 172.21.15.16:0/544699950 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5603e9ada370 0x5603e9af14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:58.263 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.261+0000 7fa6e1cc4640 1 -- 172.21.15.16:0/544699950 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5603e9ad52e0 msgr2=0x5603e9ae6070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:58.263 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.261+0000 7fa6e1cc4640 1 --2- 172.21.15.16:0/544699950 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5603e9ad52e0 0x5603e9ae6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:58.263 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.261+0000 7fa6e1cc4640 1 -- 172.21.15.16:0/544699950 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603e9ad57c0 con 0x5603e9ad49a0 2024-09-12T02:08:58.263 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.262+0000 7fa6e1cc4640 1 --2- 172.21.15.16:0/544699950 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5603e9ad49a0 0x5603e9acf1c0 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7fa6d8005760 tx=0x7fa6d8005ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd254a6b41df54c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:58.263 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.262+0000 7fa6c77fe640 1 -- 172.21.15.16:0/544699950 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6d80b8020 con 0x5603e9ad49a0 2024-09-12T02:08:58.263 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.262+0000 7fa6c77fe640 1 -- 172.21.15.16:0/544699950 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6d809ba10 con 0x5603e9ad49a0 2024-09-12T02:08:58.263 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.262+0000 7fa6c77fe640 1 -- 172.21.15.16:0/544699950 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6d8020040 con 0x5603e9ad49a0 2024-09-12T02:08:58.266 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.262+0000 7fa6e6cb7740 1 -- 172.21.15.16:0/544699950 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5603e9ad49a0 msgr2=0x5603e9acf1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:58.266 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.262+0000 7fa6e6cb7740 1 --2- 172.21.15.16:0/544699950 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5603e9ad49a0 0x5603e9acf1c0 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7fa6d8005760 tx=0x7fa6d8005ab0 comp rx=0 tx=0).stop 2024-09-12T02:08:58.267 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.262+0000 7fa6e6cb7740 1 -- 172.21.15.16:0/544699950 shutdown_connections 2024-09-12T02:08:58.267 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.262+0000 7fa6e6cb7740 1 --2- 172.21.15.16:0/544699950 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5603e9ada370 0x5603e9af14e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:58.267 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.262+0000 7fa6e6cb7740 1 --2- 172.21.15.16:0/544699950 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5603e9ad52e0 0x5603e9ae6070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:58.267 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.262+0000 7fa6e6cb7740 1 --2- 172.21.15.16:0/544699950 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5603e9ad49a0 0x5603e9acf1c0 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:58.267 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.262+0000 7fa6e6cb7740 1 -- 172.21.15.16:0/544699950 >> 172.21.15.16:0/544699950 conn(0x5603e9ad0010 msgr2=0x5603e9adef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:58.267 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.263+0000 7fa6e6cb7740 1 -- 172.21.15.16:0/544699950 shutdown_connections 2024-09-12T02:08:58.267 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.263+0000 7fa6e6cb7740 1 -- 172.21.15.16:0/544699950 wait complete. 2024-09-12T02:08:58.267 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.263+0000 7fa6e6cb7740 1 -- start start 2024-09-12T02:08:58.267 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.263+0000 7fa6e6cb7740 1 Processor -- start 2024-09-12T02:08:58.267 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.263+0000 7fa6e6cb7740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5603e9ad49a0 0x5603e9ae42c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:58.267 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.263+0000 7fa6e6cb7740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5603e9ad52e0 0x5603e9ae4800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:58.268 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.263+0000 7fa6e6cb7740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5603e9ada370 0x5603e9ae5680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:58.268 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.263+0000 7fa6e6cb7740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa6dc1a95b0 con 0x5603e9ada370 2024-09-12T02:08:58.268 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.263+0000 7fa6e6cb7740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa6dc1aaa20 con 0x5603e9ad49a0 2024-09-12T02:08:58.268 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.263+0000 7fa6e6cb7740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa6dc1a9c80 con 0x5603e9ad52e0 2024-09-12T02:08:58.268 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.263+0000 7fa6e6cb7740 10 client.0 _ll_register_callbacks cb 0x7fa6d80b9f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:08:58.268 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.263+0000 7fa6e24c5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5603e9ada370 0x5603e9ae5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:58.268 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.263+0000 7fa6e24c5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5603e9ada370 0x5603e9ae5680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33780/0 (socket says 172.21.15.16:33780) 2024-09-12T02:08:58.268 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.263+0000 7fa6e24c5640 1 -- 172.21.15.16:0/3620091699 learned_addr learned my addr 172.21.15.16:0/3620091699 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:08:58.268 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.264+0000 7fa6e14c3640 1 --2- 172.21.15.16:0/3620091699 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5603e9ad52e0 0x5603e9ae4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:58.268 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.264+0000 7fa6e1cc4640 1 --2- 172.21.15.16:0/3620091699 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5603e9ad49a0 0x5603e9ae42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:58.268 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.264+0000 7fa6e24c5640 1 -- 172.21.15.16:0/3620091699 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5603e9ad52e0 msgr2=0x5603e9ae4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:58.268 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.264+0000 7fa6e24c5640 1 --2- 172.21.15.16:0/3620091699 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5603e9ad52e0 0x5603e9ae4800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:58.268 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.264+0000 7fa6e24c5640 1 -- 172.21.15.16:0/3620091699 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5603e9ad49a0 msgr2=0x5603e9ae42c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:58.269 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.264+0000 7fa6e24c5640 1 --2- 172.21.15.16:0/3620091699 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5603e9ad49a0 0x5603e9ae42c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:58.269 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.264+0000 7fa6e24c5640 1 -- 172.21.15.16:0/3620091699 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5603e9ae5d70 con 0x5603e9ada370 2024-09-12T02:08:58.269 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.264+0000 7fa6e14c3640 1 --2- 172.21.15.16:0/3620091699 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5603e9ad52e0 0x5603e9ae4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:08:58.269 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.264+0000 7fa6e24c5640 1 --2- 172.21.15.16:0/3620091699 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5603e9ada370 0x5603e9ae5680 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7fa6d4035000 tx=0x7fa6d4005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7f6dc55201e8d2c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:58.269 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.264+0000 7fa6c57fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:08:58.269 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.264+0000 7fa6c57fa640 1 -- 172.21.15.16:0/3620091699 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6d4004440 con 0x5603e9ada370 2024-09-12T02:08:58.269 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.264+0000 7fa6c57fa640 1 -- 172.21.15.16:0/3620091699 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fa6d4046070 con 0x5603e9ada370 2024-09-12T02:08:58.269 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.264+0000 7fa6e6cb7740 1 -- 172.21.15.16:0/3620091699 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa6d809ba10 con 0x5603e9ada370 2024-09-12T02:08:58.269 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.264+0000 7fa6c57fa640 1 -- 172.21.15.16:0/3620091699 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa6d40045e0 con 0x5603e9ada370 2024-09-12T02:08:58.269 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.264+0000 7fa6e6cb7740 10 client.17637 fetch_fsmap learned FSMap version 25 2024-09-12T02:08:58.269 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.264+0000 7fa6e6cb7740 1 -- 172.21.15.16:0/3620091699 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5603e9b55f00 con 0x5603e9ada370 2024-09-12T02:08:58.269 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.265+0000 7fa6c57fa640 1 -- 172.21.15.16:0/3620091699 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6d809ba10 con 0x5603e9ada370 2024-09-12T02:08:58.270 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.265+0000 7fa6c57fa640 1 -- 172.21.15.16:0/3620091699 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fa6d402fb80 con 0x5603e9ada370 2024-09-12T02:08:58.270 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.265+0000 7fa6e6cb7740 10 client.17637 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:08:58.270 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.265+0000 7fa6e6cb7740 10 client.17637 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:08:58.270 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.265+0000 7fa6e6cb7740 20 client.17637 populate_metadata read hostname 'smithi016' 2024-09-12T02:08:58.270 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.265+0000 7fa6e6cb7740 1 --2- 172.21.15.16:0/3620091699 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5603e9b56910 0x5603e9b58d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:58.270 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.265+0000 7fa6e6cb7740 4 client.17637 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:08:58.270 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.265+0000 7fa6e6cb7740 1 -- 172.21.15.16:0/3620091699 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5603e9a3fd70 con 0x5603e9b56910 2024-09-12T02:08:58.270 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.265+0000 7fa6e1cc4640 1 --2- 172.21.15.16:0/3620091699 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5603e9b56910 0x5603e9b58d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:58.270 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.265+0000 7fa6e1cc4640 1 --2- 172.21.15.16:0/3620091699 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5603e9b56910 0x5603e9b58d30 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=773ee50fa7672930 server_cookie=d0fc8ac724075961 in_seq=0 out_seq=0 2024-09-12T02:08:58.270 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.265+0000 7fa6c57fa640 10 client.17637 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:08:58.277 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.277+0000 7fa6c57fa640 1 -- 172.21.15.16:0/3620091699 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5603e9a3fd70 con 0x5603e9b56910 2024-09-12T02:08:58.277 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.277+0000 7fa6c57fa640 10 client.17637 handle_command_reply: tid=0 2024-09-12T02:08:58.277 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.277+0000 7fa6e6cb7740 1 client.17637 shutdown 2024-09-12T02:08:58.277 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.277+0000 7fa6e6cb7740 1 -- 172.21.15.16:0/3620091699 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5603e9ada370 msgr2=0x5603e9ae5680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:58.277 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.277+0000 7fa6e6cb7740 1 --2- 172.21.15.16:0/3620091699 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5603e9ada370 0x5603e9ae5680 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7fa6d4035000 tx=0x7fa6d4005ef0 comp rx=0 tx=0).stop 2024-09-12T02:08:58.277 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.277+0000 7fa6e6cb7740 1 -- 172.21.15.16:0/3620091699 shutdown_connections 2024-09-12T02:08:58.278 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.277+0000 7fa6e6cb7740 1 --2- 172.21.15.16:0/3620091699 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5603e9b56910 0x5603e9b58d30 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:58.278 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.277+0000 7fa6e6cb7740 1 --2- 172.21.15.16:0/3620091699 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5603e9ada370 0x5603e9ae5680 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:58.278 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.277+0000 7fa6e6cb7740 1 --2- 172.21.15.16:0/3620091699 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5603e9ad52e0 0x5603e9ae4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:58.278 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.277+0000 7fa6e6cb7740 1 --2- 172.21.15.16:0/3620091699 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5603e9ad49a0 0x5603e9ae42c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:58.278 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.277+0000 7fa6c57fa640 0 client.17637 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:08:58.278 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.277+0000 7fa6e6cb7740 1 -- 172.21.15.16:0/3620091699 >> 172.21.15.16:0/3620091699 conn(0x5603e9ad0010 msgr2=0x5603e9acf1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:58.278 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.277+0000 7fa6e6cb7740 1 -- 172.21.15.16:0/3620091699 shutdown_connections 2024-09-12T02:08:58.278 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.277+0000 7fa6e6cb7740 1 -- 172.21.15.16:0/3620091699 wait complete. 2024-09-12T02:08:58.278 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.277+0000 7fa6e6cb7740 20 client.17637 trim_cache size 0 max 16384 2024-09-12T02:08:58.278 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-09-12T02:08:58.279 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-09-12T02:08:58.279 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-09-12T02:08:58.279 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-09-12T02:08:58.279 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.278+0000 7fa6e3f28640 1 -- 172.21.15.16:0/695757455 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fa6c80a2cb0 msgr2=0x7fa6c80a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:58.279 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.278+0000 7fa6e3f28640 1 --2- 172.21.15.16:0/695757455 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fa6c80a2cb0 0x7fa6c80a5170 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fa6d4005cc0 tx=0x7fa6d40095c0 comp rx=0 tx=0).stop 2024-09-12T02:08:58.280 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.278+0000 7fa6e3f28640 1 -- 172.21.15.16:0/695757455 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6dc1003b0 msgr2=0x7fa6dc199f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:58.280 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.278+0000 7fa6e3f28640 1 --2- 172.21.15.16:0/695757455 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6dc1003b0 0x7fa6dc199f20 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7fa6d800d770 tx=0x7fa6d800dc30 comp rx=0 tx=0).stop 2024-09-12T02:08:58.280 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.278+0000 7fa6e3f28640 1 -- 172.21.15.16:0/695757455 shutdown_connections 2024-09-12T02:08:58.280 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.278+0000 7fa6e3f28640 1 --2- 172.21.15.16:0/695757455 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fa6c80a2cb0 0x7fa6c80a5170 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:58.280 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.278+0000 7fa6e3f28640 1 --2- 172.21.15.16:0/695757455 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa6dc101690 0x7fa6dc1a14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:58.280 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.278+0000 7fa6e3f28640 1 --2- 172.21.15.16:0/695757455 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa6dc100cf0 0x7fa6dc19a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:58.280 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.278+0000 7fa6e3f28640 1 --2- 172.21.15.16:0/695757455 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6dc1003b0 0x7fa6dc199f20 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:58.280 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.278+0000 7fa6e3f28640 1 -- 172.21.15.16:0/695757455 >> 172.21.15.16:0/695757455 conn(0x7fa6dc0fc1e0 msgr2=0x7fa6dc0fdc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:58.280 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.278+0000 7fa6e3f28640 1 -- 172.21.15.16:0/695757455 shutdown_connections 2024-09-12T02:08:58.280 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:08:58.278+0000 7fa6e3f28640 1 -- 172.21.15.16:0/695757455 wait complete. 2024-09-12T02:08:58.356 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T02:08:58.616 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:08:58.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:08:58 smithi184 ceph-mon[35593]: pgmap v546: 97 pgs: 97 active+clean; 3.7 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 4.6 KiB/s wr, 11 op/s 2024-09-12T02:08:58.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:08:58 smithi184 ceph-mon[35593]: scrub queued for path: / 2024-09-12T02:08:58.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:08:58 smithi184 ceph-mon[35593]: scrub summary: idle+waiting paths [/] 2024-09-12T02:08:58.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:08:58 smithi184 ceph-mon[35593]: scrub summary: active paths [/] 2024-09-12T02:08:58.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:08:58 smithi184 ceph-mon[35593]: scrub summary: idle+waiting paths [/] 2024-09-12T02:08:58.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:08:58 smithi033 ceph-mon[35332]: pgmap v546: 97 pgs: 97 active+clean; 3.7 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 4.6 KiB/s wr, 11 op/s 2024-09-12T02:08:58.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:08:58 smithi033 ceph-mon[35332]: scrub queued for path: / 2024-09-12T02:08:58.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:08:58 smithi033 ceph-mon[35332]: scrub summary: idle+waiting paths [/] 2024-09-12T02:08:58.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:08:58 smithi033 ceph-mon[35332]: scrub summary: active paths [/] 2024-09-12T02:08:58.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:08:58 smithi033 ceph-mon[35332]: scrub summary: idle+waiting paths [/] 2024-09-12T02:08:58.678 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:08:58 smithi016 ceph-mon[30947]: pgmap v546: 97 pgs: 97 active+clean; 3.7 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 4.6 KiB/s wr, 11 op/s 2024-09-12T02:08:58.678 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:08:58 smithi016 ceph-mon[30947]: scrub queued for path: / 2024-09-12T02:08:58.678 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:08:58 smithi016 ceph-mon[30947]: scrub summary: idle+waiting paths [/] 2024-09-12T02:08:58.678 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:08:58 smithi016 ceph-mon[30947]: scrub summary: active paths [/] 2024-09-12T02:08:58.678 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:08:58 smithi016 ceph-mon[30947]: scrub summary: idle+waiting paths [/] 2024-09-12T02:08:59.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.124+0000 7f76079f5640 1 -- 172.21.15.16:0/1259171372 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76001008b0 msgr2=0x7f7600100c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:59.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.124+0000 7f76079f5640 1 --2- 172.21.15.16:0/1259171372 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76001008b0 0x7f7600100c90 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f75fc009900 tx=0x7f75fc02a410 comp rx=0 tx=0).stop 2024-09-12T02:08:59.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.124+0000 7f76079f5640 1 -- 172.21.15.16:0/1259171372 shutdown_connections 2024-09-12T02:08:59.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.124+0000 7f76079f5640 1 --2- 172.21.15.16:0/1259171372 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f760010e460 0x7f7600110850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:59.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.124+0000 7f76079f5640 1 --2- 172.21.15.16:0/1259171372 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7600101260 0x7f760010df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:59.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.124+0000 7f76079f5640 1 --2- 172.21.15.16:0/1259171372 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76001008b0 0x7f7600100c90 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:59.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.124+0000 7f76079f5640 1 -- 172.21.15.16:0/1259171372 >> 172.21.15.16:0/1259171372 conn(0x7f76000fc780 msgr2=0x7f76000feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:59.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.124+0000 7f76079f5640 1 -- 172.21.15.16:0/1259171372 shutdown_connections 2024-09-12T02:08:59.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.124+0000 7f76079f5640 1 -- 172.21.15.16:0/1259171372 wait complete. 2024-09-12T02:08:59.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.125+0000 7f76079f5640 1 Processor -- start 2024-09-12T02:08:59.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.125+0000 7f76079f5640 1 -- start start 2024-09-12T02:08:59.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.125+0000 7f76079f5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76001008b0 0x7f760019a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:59.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.125+0000 7f76079f5640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7600101260 0x7f760019ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:59.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.125+0000 7f76079f5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f760010e460 0x7f760019ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:59.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.125+0000 7f76079f5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f76001141e0 con 0x7f760010e460 2024-09-12T02:08:59.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.125+0000 7f76079f5640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7600114060 con 0x7f76001008b0 2024-09-12T02:08:59.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.125+0000 7f76079f5640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7600114360 con 0x7f7600101260 2024-09-12T02:08:59.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.125+0000 7f7605f92640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f760010e460 0x7f760019ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:59.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.125+0000 7f7605f92640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f760010e460 0x7f760019ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33790/0 (socket says 172.21.15.16:33790) 2024-09-12T02:08:59.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.125+0000 7f7605f92640 1 -- 172.21.15.16:0/3613718531 learned_addr learned my addr 172.21.15.16:0/3613718531 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:08:59.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.125+0000 7f7605791640 1 --2- 172.21.15.16:0/3613718531 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76001008b0 0x7f760019a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:59.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.126+0000 7f7604f90640 1 --2- 172.21.15.16:0/3613718531 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7600101260 0x7f760019ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:59.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.126+0000 7f7605f92640 1 -- 172.21.15.16:0/3613718531 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7600101260 msgr2=0x7f760019ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:59.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.126+0000 7f7605f92640 1 --2- 172.21.15.16:0/3613718531 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7600101260 0x7f760019ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:59.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.126+0000 7f7605f92640 1 -- 172.21.15.16:0/3613718531 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76001008b0 msgr2=0x7f760019a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:59.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.126+0000 7f7605f92640 1 --2- 172.21.15.16:0/3613718531 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76001008b0 0x7f760019a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:59.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.126+0000 7f7605f92640 1 -- 172.21.15.16:0/3613718531 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f760019f670 con 0x7f760010e460 2024-09-12T02:08:59.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.126+0000 7f7604f90640 1 --2- 172.21.15.16:0/3613718531 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7600101260 0x7f760019ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:08:59.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.126+0000 7f7605f92640 1 --2- 172.21.15.16:0/3613718531 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f760010e460 0x7f760019ef90 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f75f800cce0 tx=0x7f75f800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:59.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.126+0000 7f75f67fc640 1 -- 172.21.15.16:0/3613718531 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75f800f450 con 0x7f760010e460 2024-09-12T02:08:59.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.126+0000 7f76079f5640 1 -- 172.21.15.16:0/3613718531 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f760019f960 con 0x7f760010e460 2024-09-12T02:08:59.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.126+0000 7f75f67fc640 1 -- 172.21.15.16:0/3613718531 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75f800fad0 con 0x7f760010e460 2024-09-12T02:08:59.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.126+0000 7f75f67fc640 1 -- 172.21.15.16:0/3613718531 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75f8018bd0 con 0x7f760010e460 2024-09-12T02:08:59.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.126+0000 7f76079f5640 1 -- 172.21.15.16:0/3613718531 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76001a7240 con 0x7f760010e460 2024-09-12T02:08:59.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.128+0000 7f75f67fc640 1 -- 172.21.15.16:0/3613718531 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f75f800fc80 con 0x7f760010e460 2024-09-12T02:08:59.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.129+0000 7f7609dce740 1 Processor -- start 2024-09-12T02:08:59.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.129+0000 7f7609dce740 1 -- start start 2024-09-12T02:08:59.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.129+0000 7f7609dce740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563fab6464a0 0x563fab646880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:59.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.129+0000 7f7609dce740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563fab646dc0 0x563fab651140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:59.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.129+0000 7f7609dce740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563fab651680 0x563fab653a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:59.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.129+0000 7f7609dce740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x563fab5ac2e0 con 0x563fab6464a0 2024-09-12T02:08:59.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.129+0000 7f7609dce740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x563fab553ce0 con 0x563fab651680 2024-09-12T02:08:59.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.129+0000 7f7609dce740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x563fab5a82a0 con 0x563fab646dc0 2024-09-12T02:08:59.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.129+0000 7f7604f90640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563fab6464a0 0x563fab646880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:59.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.129+0000 7f7604f90640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563fab6464a0 0x563fab646880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33806/0 (socket says 172.21.15.16:33806) 2024-09-12T02:08:59.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.129+0000 7f7604f90640 1 -- 172.21.15.16:0/2444403407 learned_addr learned my addr 172.21.15.16:0/2444403407 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:08:59.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.129+0000 7f75f67fc640 1 --2- 172.21.15.16:0/3613718531 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f75ec0a2b80 0x7f75ec0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:59.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.130+0000 7f7605791640 1 --2- 172.21.15.16:0/2444403407 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563fab646dc0 0x563fab651140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:59.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.130+0000 7f7604f90640 1 --2- 172.21.15.16:0/2444403407 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563fab651680 0x563fab653a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:59.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.130+0000 7f7605f92640 1 --2- 172.21.15.16:0/3613718531 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f75ec0a2b80 0x7f75ec0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:59.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.130+0000 7f7604f90640 1 -- 172.21.15.16:0/2444403407 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563fab646dc0 msgr2=0x563fab651140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:59.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.130+0000 7f7604f90640 1 --2- 172.21.15.16:0/2444403407 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563fab646dc0 0x563fab651140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:59.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.130+0000 7f7604f90640 1 -- 172.21.15.16:0/2444403407 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563fab651680 msgr2=0x563fab653a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:59.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.130+0000 7f75f67fc640 1 -- 172.21.15.16:0/3613718531 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f75f80cf780 con 0x7f760010e460 2024-09-12T02:08:59.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.130+0000 7f7604f90640 1 --2- 172.21.15.16:0/2444403407 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563fab651680 0x563fab653a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:59.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.130+0000 7f7604f90640 1 -- 172.21.15.16:0/2444403407 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563fab538440 con 0x563fab6464a0 2024-09-12T02:08:59.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.130+0000 7f7604f90640 1 --2- 172.21.15.16:0/2444403407 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563fab6464a0 0x563fab646880 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f75f000af90 tx=0x7f75f000a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=372b9802c16435c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:59.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.130+0000 7f75eb7fe640 1 -- 172.21.15.16:0/2444403407 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75f001b070 con 0x563fab6464a0 2024-09-12T02:08:59.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.130+0000 7f75eb7fe640 1 -- 172.21.15.16:0/2444403407 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75f00168d0 con 0x563fab6464a0 2024-09-12T02:08:59.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.130+0000 7f75eb7fe640 1 -- 172.21.15.16:0/2444403407 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75f000f040 con 0x563fab6464a0 2024-09-12T02:08:59.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.130+0000 7f7605f92640 1 --2- 172.21.15.16:0/3613718531 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f75ec0a2b80 0x7f75ec0a5040 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f75f80cd320 tx=0x7f75f8066c50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:59.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.131+0000 7f7609dce740 1 -- 172.21.15.16:0/2444403407 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563fab6464a0 msgr2=0x563fab646880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:59.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.131+0000 7f7609dce740 1 --2- 172.21.15.16:0/2444403407 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563fab6464a0 0x563fab646880 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f75f000af90 tx=0x7f75f000a410 comp rx=0 tx=0).stop 2024-09-12T02:08:59.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.131+0000 7f7609dce740 1 -- 172.21.15.16:0/2444403407 shutdown_connections 2024-09-12T02:08:59.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.131+0000 7f7609dce740 1 --2- 172.21.15.16:0/2444403407 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563fab651680 0x563fab653a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:59.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.131+0000 7f7609dce740 1 --2- 172.21.15.16:0/2444403407 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563fab646dc0 0x563fab651140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:59.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.131+0000 7f7609dce740 1 --2- 172.21.15.16:0/2444403407 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563fab6464a0 0x563fab646880 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:59.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.131+0000 7f7609dce740 1 -- 172.21.15.16:0/2444403407 >> 172.21.15.16:0/2444403407 conn(0x563fab642a30 msgr2=0x563fab642e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:59.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.131+0000 7f7609dce740 1 -- 172.21.15.16:0/2444403407 shutdown_connections 2024-09-12T02:08:59.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.131+0000 7f7609dce740 1 -- 172.21.15.16:0/2444403407 wait complete. 2024-09-12T02:08:59.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.131+0000 7f7609dce740 1 -- start start 2024-09-12T02:08:59.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.131+0000 7f7609dce740 1 Processor -- start 2024-09-12T02:08:59.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.132+0000 7f7609dce740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563fab6464a0 0x563fab651000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:59.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.132+0000 7f7609dce740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563fab646dc0 0x563fab64e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:59.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.132+0000 7f7609dce740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563fab651680 0x563fab64e830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:59.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.132+0000 7f7609dce740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x563fab654250 con 0x563fab646dc0 2024-09-12T02:08:59.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.132+0000 7f7609dce740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x563fab567050 con 0x563fab651680 2024-09-12T02:08:59.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.132+0000 7f7609dce740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x563fab6543d0 con 0x563fab6464a0 2024-09-12T02:08:59.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.132+0000 7f7609dce740 10 client.0 _ll_register_callbacks cb 0x563fab5a7160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:08:59.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.132+0000 7f7605791640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563fab646dc0 0x563fab64e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:59.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.132+0000 7f7605791640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563fab646dc0 0x563fab64e2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33816/0 (socket says 172.21.15.16:33816) 2024-09-12T02:08:59.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.132+0000 7f7605791640 1 -- 172.21.15.16:0/875839825 learned_addr learned my addr 172.21.15.16:0/875839825 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:08:59.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.132+0000 7f7604f90640 1 --2- 172.21.15.16:0/875839825 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563fab6464a0 0x563fab651000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:59.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.132+0000 7f7604f90640 1 --2- 172.21.15.16:0/875839825 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563fab651680 0x563fab64e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:59.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.132+0000 7f7605791640 1 -- 172.21.15.16:0/875839825 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563fab6464a0 msgr2=0x563fab651000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:59.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.132+0000 7f7605791640 1 --2- 172.21.15.16:0/875839825 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563fab6464a0 0x563fab651000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:59.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.132+0000 7f7605791640 1 -- 172.21.15.16:0/875839825 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563fab651680 msgr2=0x563fab64e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:59.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.132+0000 7f7605791640 1 --2- 172.21.15.16:0/875839825 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563fab651680 0x563fab64e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:59.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.132+0000 7f7605791640 1 -- 172.21.15.16:0/875839825 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563fab31d3c0 con 0x563fab646dc0 2024-09-12T02:08:59.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.132+0000 7f7605791640 1 --2- 172.21.15.16:0/875839825 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563fab646dc0 0x563fab64e2b0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f75fc031550 tx=0x7f75fc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2f8898bea3e31d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:59.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.132+0000 7f75e8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:08:59.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.133+0000 7f75e8ff9640 1 -- 172.21.15.16:0/875839825 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75fc035070 con 0x563fab646dc0 2024-09-12T02:08:59.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.133+0000 7f75e8ff9640 1 -- 172.21.15.16:0/875839825 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f75fc0037f0 con 0x563fab646dc0 2024-09-12T02:08:59.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.133+0000 7f7609dce740 1 -- 172.21.15.16:0/875839825 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f75f00168d0 con 0x563fab646dc0 2024-09-12T02:08:59.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.133+0000 7f75e8ff9640 1 -- 172.21.15.16:0/875839825 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f75fc005d50 con 0x563fab646dc0 2024-09-12T02:08:59.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.133+0000 7f7609dce740 10 client.17649 fetch_fsmap learned FSMap version 25 2024-09-12T02:08:59.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.133+0000 7f7609dce740 1 -- 172.21.15.16:0/875839825 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x563fab65fc40 con 0x563fab646dc0 2024-09-12T02:08:59.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.133+0000 7f75e8ff9640 1 -- 172.21.15.16:0/875839825 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75f00168d0 con 0x563fab646dc0 2024-09-12T02:08:59.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.133+0000 7f75e8ff9640 1 -- 172.21.15.16:0/875839825 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f75fc03b430 con 0x563fab646dc0 2024-09-12T02:08:59.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.133+0000 7f7609dce740 10 client.17649 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:08:59.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.133+0000 7f7609dce740 10 client.17649 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:08:59.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.133+0000 7f7609dce740 20 client.17649 populate_metadata read hostname 'smithi016' 2024-09-12T02:08:59.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.133+0000 7f7609dce740 1 --2- 172.21.15.16:0/875839825 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x563fab6610a0 0x563fab663490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:59.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.133+0000 7f7609dce740 4 client.17649 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:08:59.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.133+0000 7f7609dce740 1 -- 172.21.15.16:0/875839825 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563fab5b9850 con 0x563fab6610a0 2024-09-12T02:08:59.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.134+0000 7f7604f90640 1 --2- 172.21.15.16:0/875839825 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x563fab6610a0 0x563fab663490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:59.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.134+0000 7f7604f90640 1 --2- 172.21.15.16:0/875839825 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x563fab6610a0 0x563fab663490 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ea0fa63faab570a server_cookie=691002470fccb189 in_seq=0 out_seq=0 2024-09-12T02:08:59.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.134+0000 7f75e8ff9640 10 client.17649 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:08:59.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.135+0000 7f75e8ff9640 1 -- 172.21.15.16:0/875839825 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563fab5b9850 con 0x563fab6610a0 2024-09-12T02:08:59.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.135+0000 7f75e8ff9640 10 client.17649 handle_command_reply: tid=0 2024-09-12T02:08:59.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.135+0000 7f7609dce740 1 client.17649 shutdown 2024-09-12T02:08:59.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.135+0000 7f7609dce740 1 -- 172.21.15.16:0/875839825 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563fab646dc0 msgr2=0x563fab64e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:59.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.135+0000 7f7609dce740 1 --2- 172.21.15.16:0/875839825 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563fab646dc0 0x563fab64e2b0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f75fc031550 tx=0x7f75fc0042f0 comp rx=0 tx=0).stop 2024-09-12T02:08:59.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.135+0000 7f7609dce740 1 -- 172.21.15.16:0/875839825 shutdown_connections 2024-09-12T02:08:59.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.135+0000 7f7609dce740 1 --2- 172.21.15.16:0/875839825 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x563fab6610a0 0x563fab663490 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:59.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.135+0000 7f7609dce740 1 --2- 172.21.15.16:0/875839825 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563fab651680 0x563fab64e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:59.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.135+0000 7f7609dce740 1 --2- 172.21.15.16:0/875839825 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563fab646dc0 0x563fab64e2b0 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:59.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.135+0000 7f75e8ff9640 0 client.17649 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:08:59.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.135+0000 7f7609dce740 1 --2- 172.21.15.16:0/875839825 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563fab6464a0 0x563fab651000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:59.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.135+0000 7f7609dce740 1 -- 172.21.15.16:0/875839825 >> 172.21.15.16:0/875839825 conn(0x563fab642a30 msgr2=0x563fab648710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:59.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.135+0000 7f7609dce740 1 -- 172.21.15.16:0/875839825 shutdown_connections 2024-09-12T02:08:59.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.135+0000 7f7609dce740 1 -- 172.21.15.16:0/875839825 wait complete. 2024-09-12T02:08:59.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.135+0000 7f7609dce740 20 client.17649 trim_cache size 0 max 16384 2024-09-12T02:08:59.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.153+0000 7f7609dce740 1 Processor -- start 2024-09-12T02:08:59.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.153+0000 7f7609dce740 1 -- start start 2024-09-12T02:08:59.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.153+0000 7f7609dce740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563fab6464a0 0x563fab65d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:59.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.153+0000 7f7609dce740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563fab646dc0 0x563fab650220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:59.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.153+0000 7f7609dce740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563fab651680 0x563fab668d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:59.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.153+0000 7f7609dce740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f76001b1e90 con 0x563fab6464a0 2024-09-12T02:08:59.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.153+0000 7f7609dce740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f76001af760 con 0x563fab646dc0 2024-09-12T02:08:59.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.153+0000 7f7609dce740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f76001ad5b0 con 0x563fab651680 2024-09-12T02:08:59.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.153+0000 7f7604f90640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563fab6464a0 0x563fab65d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:59.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.153+0000 7f7604f90640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563fab6464a0 0x563fab65d4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33828/0 (socket says 172.21.15.16:33828) 2024-09-12T02:08:59.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.153+0000 7f7604f90640 1 -- 172.21.15.16:0/724770005 learned_addr learned my addr 172.21.15.16:0/724770005 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:08:59.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.154+0000 7f7605791640 1 --2- 172.21.15.16:0/724770005 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563fab646dc0 0x563fab650220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:59.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.154+0000 7f7604f90640 1 -- 172.21.15.16:0/724770005 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563fab651680 msgr2=0x563fab668d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:59.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.154+0000 7f7604f90640 1 --2- 172.21.15.16:0/724770005 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563fab651680 0x563fab668d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:59.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.154+0000 7f7604f90640 1 -- 172.21.15.16:0/724770005 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563fab646dc0 msgr2=0x563fab650220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:59.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.154+0000 7f7604f90640 1 --2- 172.21.15.16:0/724770005 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563fab646dc0 0x563fab650220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:59.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.154+0000 7f7604f90640 1 -- 172.21.15.16:0/724770005 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563fab6462d0 con 0x563fab6464a0 2024-09-12T02:08:59.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.154+0000 7f7605791640 1 --2- 172.21.15.16:0/724770005 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563fab646dc0 0x563fab650220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:08:59.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.154+0000 7f7604f90640 1 --2- 172.21.15.16:0/724770005 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563fab6464a0 0x563fab65d4b0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f75f0007fd0 tx=0x7f75f000a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfdad61bb0d5ae34 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:59.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.154+0000 7f75eaffd640 1 -- 172.21.15.16:0/724770005 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75f001b040 con 0x563fab6464a0 2024-09-12T02:08:59.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.154+0000 7f75eaffd640 1 -- 172.21.15.16:0/724770005 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75f002f030 con 0x563fab6464a0 2024-09-12T02:08:59.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.154+0000 7f75eaffd640 1 -- 172.21.15.16:0/724770005 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75f00249e0 con 0x563fab6464a0 2024-09-12T02:08:59.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.155+0000 7f7609dce740 1 -- 172.21.15.16:0/724770005 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563fab6464a0 msgr2=0x563fab65d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:59.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.155+0000 7f7609dce740 1 --2- 172.21.15.16:0/724770005 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563fab6464a0 0x563fab65d4b0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f75f0007fd0 tx=0x7f75f000a410 comp rx=0 tx=0).stop 2024-09-12T02:08:59.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.155+0000 7f7609dce740 1 -- 172.21.15.16:0/724770005 shutdown_connections 2024-09-12T02:08:59.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.155+0000 7f7609dce740 1 --2- 172.21.15.16:0/724770005 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563fab651680 0x563fab668d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:59.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.155+0000 7f7609dce740 1 --2- 172.21.15.16:0/724770005 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563fab646dc0 0x563fab650220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:59.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.155+0000 7f7609dce740 1 --2- 172.21.15.16:0/724770005 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563fab6464a0 0x563fab65d4b0 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:59.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.155+0000 7f7609dce740 1 -- 172.21.15.16:0/724770005 >> 172.21.15.16:0/724770005 conn(0x563fab642a30 msgr2=0x563fab64fb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:59.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.155+0000 7f7609dce740 1 -- 172.21.15.16:0/724770005 shutdown_connections 2024-09-12T02:08:59.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.155+0000 7f7609dce740 1 -- 172.21.15.16:0/724770005 wait complete. 2024-09-12T02:08:59.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.155+0000 7f7609dce740 1 -- start start 2024-09-12T02:08:59.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.155+0000 7f7609dce740 1 Processor -- start 2024-09-12T02:08:59.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.156+0000 7f7609dce740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563fab6464a0 0x563fab655f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:59.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.156+0000 7f7609dce740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563fab646dc0 0x563fab6564d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:59.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.156+0000 7f7609dce740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563fab651680 0x563fab6c3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:59.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.156+0000 7f7609dce740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f76001adc20 con 0x563fab6464a0 2024-09-12T02:08:59.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.156+0000 7f7609dce740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f76001af090 con 0x563fab646dc0 2024-09-12T02:08:59.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.156+0000 7f7609dce740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f76001ae2f0 con 0x563fab651680 2024-09-12T02:08:59.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.156+0000 7f7609dce740 10 client.0 _ll_register_callbacks cb 0x7f75fc009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:08:59.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.156+0000 7f7604f90640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563fab6464a0 0x563fab655f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:59.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.156+0000 7f7604f90640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563fab6464a0 0x563fab655f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33830/0 (socket says 172.21.15.16:33830) 2024-09-12T02:08:59.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.156+0000 7f7604f90640 1 -- 172.21.15.16:0/1265442591 learned_addr learned my addr 172.21.15.16:0/1265442591 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:08:59.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.156+0000 7f7605791640 1 --2- 172.21.15.16:0/1265442591 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563fab646dc0 0x563fab6564d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:59.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.156+0000 7f7604f90640 1 -- 172.21.15.16:0/1265442591 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563fab651680 msgr2=0x563fab6c3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:59.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.156+0000 7f7604f90640 1 --2- 172.21.15.16:0/1265442591 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563fab651680 0x563fab6c3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:59.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.156+0000 7f7604f90640 1 -- 172.21.15.16:0/1265442591 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563fab646dc0 msgr2=0x563fab6564d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:59.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.156+0000 7f7604f90640 1 --2- 172.21.15.16:0/1265442591 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563fab646dc0 0x563fab6564d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:59.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.156+0000 7f7604f90640 1 -- 172.21.15.16:0/1265442591 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563fab6c44e0 con 0x563fab6464a0 2024-09-12T02:08:59.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.156+0000 7f7605791640 1 --2- 172.21.15.16:0/1265442591 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563fab646dc0 0x563fab6564d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:08:59.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.157+0000 7f7604f90640 1 --2- 172.21.15.16:0/1265442591 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563fab6464a0 0x563fab655f90 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f75f000aca0 tx=0x7f75f0024660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6243499cd6208d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:59.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.157+0000 7f75e8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:08:59.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.157+0000 7f75e8ff9640 1 -- 172.21.15.16:0/1265442591 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75f0038070 con 0x563fab6464a0 2024-09-12T02:08:59.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.157+0000 7f75e8ff9640 1 -- 172.21.15.16:0/1265442591 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f75f0024a50 con 0x563fab6464a0 2024-09-12T02:08:59.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.157+0000 7f7609dce740 1 -- 172.21.15.16:0/1265442591 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f75f002f030 con 0x563fab6464a0 2024-09-12T02:08:59.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.157+0000 7f75e8ff9640 1 -- 172.21.15.16:0/1265442591 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f75f001b050 con 0x563fab6464a0 2024-09-12T02:08:59.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.157+0000 7f7609dce740 10 client.17655 fetch_fsmap learned FSMap version 25 2024-09-12T02:08:59.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.157+0000 7f7609dce740 1 -- 172.21.15.16:0/1265442591 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x563fab6cc7b0 con 0x563fab6464a0 2024-09-12T02:08:59.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.157+0000 7f75e8ff9640 1 -- 172.21.15.16:0/1265442591 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75f002f030 con 0x563fab6464a0 2024-09-12T02:08:59.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.157+0000 7f75e8ff9640 1 -- 172.21.15.16:0/1265442591 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f75f0015030 con 0x563fab6464a0 2024-09-12T02:08:59.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.158+0000 7f7609dce740 10 client.17655 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:08:59.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.158+0000 7f7609dce740 10 client.17655 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:08:59.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.158+0000 7f7609dce740 20 client.17655 populate_metadata read hostname 'smithi016' 2024-09-12T02:08:59.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.158+0000 7f7609dce740 1 --2- 172.21.15.16:0/1265442591 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x563fab6ccd30 0x563fab6cf120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:59.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.158+0000 7f7609dce740 4 client.17655 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T02:08:59.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.158+0000 7f7609dce740 1 -- 172.21.15.16:0/1265442591 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563fab5b3d60 con 0x563fab6ccd30 2024-09-12T02:08:59.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.158+0000 7f7605791640 1 --2- 172.21.15.16:0/1265442591 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x563fab6ccd30 0x563fab6cf120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:59.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.158+0000 7f75e8ff9640 10 client.17655 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:08:59.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.158+0000 7f7605791640 1 --2- 172.21.15.16:0/1265442591 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x563fab6ccd30 0x563fab6cf120 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb3ecf21c42a5b6 server_cookie=520896c59ba40055 in_seq=0 out_seq=0 2024-09-12T02:08:59.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.158+0000 7f75e8ff9640 1 -- 172.21.15.16:0/1265442591 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x563fab5b3d60 con 0x563fab6ccd30 2024-09-12T02:08:59.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.158+0000 7f75e8ff9640 10 client.17655 handle_command_reply: tid=0 2024-09-12T02:08:59.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.158+0000 7f7609dce740 1 client.17655 shutdown 2024-09-12T02:08:59.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.158+0000 7f7609dce740 1 -- 172.21.15.16:0/1265442591 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563fab6464a0 msgr2=0x563fab655f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:59.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.158+0000 7f7609dce740 1 --2- 172.21.15.16:0/1265442591 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563fab6464a0 0x563fab655f90 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f75f000aca0 tx=0x7f75f0024660 comp rx=0 tx=0).stop 2024-09-12T02:08:59.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.159+0000 7f7609dce740 1 -- 172.21.15.16:0/1265442591 shutdown_connections 2024-09-12T02:08:59.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.159+0000 7f7609dce740 1 --2- 172.21.15.16:0/1265442591 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x563fab6ccd30 0x563fab6cf120 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:59.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.159+0000 7f7609dce740 1 --2- 172.21.15.16:0/1265442591 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563fab651680 0x563fab6c3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:59.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.159+0000 7f7609dce740 1 --2- 172.21.15.16:0/1265442591 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563fab646dc0 0x563fab6564d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:59.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.159+0000 7f7609dce740 1 --2- 172.21.15.16:0/1265442591 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563fab6464a0 0x563fab655f90 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:59.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.159+0000 7f7609dce740 1 -- 172.21.15.16:0/1265442591 >> 172.21.15.16:0/1265442591 conn(0x563fab642a30 msgr2=0x563fab64fb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:59.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.159+0000 7f75e8ff9640 0 client.17655 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:08:59.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.159+0000 7f7609dce740 1 -- 172.21.15.16:0/1265442591 shutdown_connections 2024-09-12T02:08:59.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.159+0000 7f7609dce740 1 -- 172.21.15.16:0/1265442591 wait complete. 2024-09-12T02:08:59.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.159+0000 7f7609dce740 20 client.17655 trim_cache size 0 max 16384 2024-09-12T02:08:59.164 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T02:08:59.165 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-12T02:08:59.165 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": { 2024-09-12T02:08:59.165 INFO:teuthology.orchestra.run.smithi016.stdout: "0258bee0-6dc2-4035-bba2-e5c7b7137e41": { 2024-09-12T02:08:59.165 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-12T02:08:59.165 INFO:teuthology.orchestra.run.smithi016.stdout: "tag": "0258bee0-6dc2-4035-bba2-e5c7b7137e41", 2024-09-12T02:08:59.165 INFO:teuthology.orchestra.run.smithi016.stdout: "options": "recursive,force" 2024-09-12T02:08:59.165 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:08:59.165 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:08:59.165 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T02:08:59.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.159+0000 7f76079f5640 1 -- 172.21.15.16:0/3613718531 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f75ec0a2b80 msgr2=0x7f75ec0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:59.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.159+0000 7f76079f5640 1 --2- 172.21.15.16:0/3613718531 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f75ec0a2b80 0x7f75ec0a5040 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f75f80cd320 tx=0x7f75f8066c50 comp rx=0 tx=0).stop 2024-09-12T02:08:59.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.159+0000 7f76079f5640 1 -- 172.21.15.16:0/3613718531 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f760010e460 msgr2=0x7f760019ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:59.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.159+0000 7f76079f5640 1 --2- 172.21.15.16:0/3613718531 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f760010e460 0x7f760019ef90 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f75f800cce0 tx=0x7f75f800a620 comp rx=0 tx=0).stop 2024-09-12T02:08:59.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.159+0000 7f76079f5640 1 -- 172.21.15.16:0/3613718531 shutdown_connections 2024-09-12T02:08:59.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.159+0000 7f76079f5640 1 --2- 172.21.15.16:0/3613718531 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f75ec0a2b80 0x7f75ec0a5040 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:59.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.159+0000 7f76079f5640 1 --2- 172.21.15.16:0/3613718531 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f760010e460 0x7f760019ef90 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:59.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.159+0000 7f76079f5640 1 --2- 172.21.15.16:0/3613718531 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7600101260 0x7f760019ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:59.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.159+0000 7f76079f5640 1 --2- 172.21.15.16:0/3613718531 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76001008b0 0x7f760019a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:59.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.159+0000 7f76079f5640 1 -- 172.21.15.16:0/3613718531 >> 172.21.15.16:0/3613718531 conn(0x7f76000fc780 msgr2=0x7f760010f850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:59.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.160+0000 7f76079f5640 1 -- 172.21.15.16:0/3613718531 shutdown_connections 2024-09-12T02:08:59.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:08:59.160+0000 7f76079f5640 1 -- 172.21.15.16:0/3613718531 wait complete. 2024-09-12T02:08:59.805 INFO:tasks.cephfs.filesystem:scrub status for tag:0258bee0-6dc2-4035-bba2-e5c7b7137e41 - {'path': '/', 'tag': '0258bee0-6dc2-4035-bba2-e5c7b7137e41', 'options': 'recursive,force'} 2024-09-12T02:09:00.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:09:00 smithi184 ceph-mon[35593]: pgmap v547: 97 pgs: 97 active+clean; 3.7 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 4.4 KiB/s wr, 9 op/s 2024-09-12T02:09:00.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:09:00 smithi033 ceph-mon[35332]: pgmap v547: 97 pgs: 97 active+clean; 3.7 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 4.4 KiB/s wr, 9 op/s 2024-09-12T02:09:00.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:09:00 smithi016 ceph-mon[30947]: pgmap v547: 97 pgs: 97 active+clean; 3.7 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 4.4 KiB/s wr, 9 op/s 2024-09-12T02:09:01.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:09:01 smithi184 ceph-mon[35593]: scrub summary: idle 2024-09-12T02:09:01.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:09:01 smithi033 ceph-mon[35332]: scrub summary: idle 2024-09-12T02:09:01.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:09:01 smithi016 ceph-mon[30947]: scrub summary: idle 2024-09-12T02:09:02.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:09:02 smithi184 ceph-mon[35593]: pgmap v548: 97 pgs: 97 active+clean; 3.8 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-09-12T02:09:02.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:09:02 smithi033 ceph-mon[35332]: pgmap v548: 97 pgs: 97 active+clean; 3.8 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-09-12T02:09:02.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:09:02 smithi016 ceph-mon[30947]: pgmap v548: 97 pgs: 97 active+clean; 3.8 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-09-12T02:09:03.501 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:09:03.501 INFO:tasks.workunit.client.0.smithi016.stdout:189/600: open fd = 3 2024-09-12T02:09:03.501 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:09:03.501 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:09:03.501 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:09:03.502 INFO:tasks.workunit.client.0.smithi016.stdout:190/600: open fd = 3 2024-09-12T02:09:03.502 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:09:03.502 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:09:03.502 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:09:03.502 INFO:tasks.workunit.client.0.smithi016.stdout:191/600: open fd = 3 2024-09-12T02:09:03.502 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:09:03.502 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:09:03.502 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:09:03.502 INFO:tasks.workunit.client.0.smithi016.stdout:192/600: open fd = 3 2024-09-12T02:09:03.502 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:09:03.502 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:09:03.502 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:09:03.502 INFO:tasks.workunit.client.0.smithi016.stdout:193/600: open fd = 3 2024-09-12T02:09:03.503 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:09:03.503 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:09:03.503 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:09:03.503 INFO:tasks.workunit.client.0.smithi016.stdout:194/600: open fd = 3 2024-09-12T02:09:03.503 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:09:03.503 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:09:03.503 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:09:03.503 INFO:tasks.workunit.client.0.smithi016.stdout:195/600: open fd = 3 2024-09-12T02:09:03.503 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:09:03.503 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:09:03.503 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:09:03.503 INFO:tasks.workunit.client.0.smithi016.stdout:196/600: open fd = 3 2024-09-12T02:09:03.503 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:09:03.503 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:09:03.503 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:09:03.504 INFO:tasks.workunit.client.0.smithi016.stdout:197/600: open fd = 3 2024-09-12T02:09:03.504 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:09:03.504 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:09:03.504 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:09:03.504 INFO:tasks.workunit.client.0.smithi016.stdout:198/600: open fd = 3 2024-09-12T02:09:03.504 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:09:03.504 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:09:03.504 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:09:03.504 INFO:tasks.workunit.client.0.smithi016.stdout:199/600: open fd = 3 2024-09-12T02:09:03.504 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:09:03.504 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:09:03.504 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:09:03.504 INFO:tasks.workunit.client.0.smithi016.stdout:200/600: open fd = 3 2024-09-12T02:09:03.504 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:09:03.505 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:09:03.505 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:09:03.505 INFO:tasks.workunit.client.0.smithi016.stdout:201/600: open fd = 3 2024-09-12T02:09:03.505 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:09:03.505 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:09:03.505 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:09:03.505 INFO:tasks.workunit.client.0.smithi016.stdout:202/600: open fd = 3 2024-09-12T02:09:03.505 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:09:03.505 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:09:03.505 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:09:03.505 INFO:tasks.workunit.client.0.smithi016.stdout:203/600: open fd = 3 2024-09-12T02:09:03.505 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:09:03.505 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:09:03.505 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:09:03.506 INFO:tasks.workunit.client.0.smithi016.stdout:204/600: open fd = 3 2024-09-12T02:09:03.506 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:09:03.506 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:09:03.506 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:09:03.506 INFO:tasks.workunit.client.0.smithi016.stdout:205/600: open fd = 3 2024-09-12T02:09:03.506 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:09:03.506 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:09:03.506 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:09:03.506 INFO:tasks.workunit.client.0.smithi016.stdout:206/600: open fd = 3 2024-09-12T02:09:03.507 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:09:03.507 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:09:03.507 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:09:03.507 INFO:tasks.workunit.client.0.smithi016.stdout:207/600: open fd = 3 2024-09-12T02:09:03.507 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:09:03.507 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:09:03.507 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:09:03.507 INFO:tasks.workunit.client.0.smithi016.stdout:208/600: open fd = 3 2024-09-12T02:09:03.507 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:09:03.507 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:09:03.507 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:09:03.507 INFO:tasks.workunit.client.0.smithi016.stdout:209/600: open fd = 3 2024-09-12T02:09:03.507 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:09:03.508 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:09:03.508 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:09:03.508 INFO:tasks.workunit.client.0.smithi016.stdout:210/600: open fd = 3 2024-09-12T02:09:03.508 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:09:03.508 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:09:03.508 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:09:03.508 INFO:tasks.workunit.client.0.smithi016.stdout:211/600: open fd = 3 2024-09-12T02:09:03.508 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:09:03.508 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:09:03.508 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:09:03.508 INFO:tasks.workunit.client.0.smithi016.stdout:212/600: open fd = 3 2024-09-12T02:09:03.508 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:09:03.508 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:09:03.508 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:09:03.509 INFO:tasks.workunit.client.0.smithi016.stdout:213/600: open fd = 3 2024-09-12T02:09:03.509 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:09:03.509 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:09:03.509 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:09:03.509 INFO:tasks.workunit.client.0.smithi016.stdout:214/600: open fd = 3 2024-09-12T02:09:03.509 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:09:03.509 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:09:03.509 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:09:03.509 INFO:tasks.workunit.client.0.smithi016.stdout:215/600: open fd = 3 2024-09-12T02:09:03.509 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:09:03.509 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:09:03.509 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:09:03.509 INFO:tasks.workunit.client.0.smithi016.stdout:216/600: open fd = 3 2024-09-12T02:09:03.510 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:09:03.510 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:09:03.510 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:09:03.510 INFO:tasks.workunit.client.0.smithi016.stdout:217/600: open fd = 3 2024-09-12T02:09:03.510 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:09:03.510 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:09:03.510 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:09:03.510 INFO:tasks.workunit.client.0.smithi016.stdout:218/600: open fd = 3 2024-09-12T02:09:03.510 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:09:03.510 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:09:03.511 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:09:03.511 INFO:tasks.workunit.client.0.smithi016.stdout:219/600: open fd = 3 2024-09-12T02:09:03.511 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:09:03.511 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:09:03.511 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:09:03.511 INFO:tasks.workunit.client.0.smithi016.stdout:220/600: open fd = 3 2024-09-12T02:09:03.511 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:09:03.511 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:09:03.511 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:09:03.511 INFO:tasks.workunit.client.0.smithi016.stdout:221/600: open fd = 3 2024-09-12T02:09:03.511 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:09:03.511 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:09:03.511 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:09:03.512 INFO:tasks.workunit.client.0.smithi016.stdout:222/600: open fd = 3 2024-09-12T02:09:03.512 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:09:03.512 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:09:03.512 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:09:03.512 INFO:tasks.workunit.client.0.smithi016.stdout:223/600: open fd = 3 2024-09-12T02:09:03.512 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:09:03.512 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:09:03.512 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:09:03.512 INFO:tasks.workunit.client.0.smithi016.stdout:224/600: open fd = 3 2024-09-12T02:09:03.513 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:09:03.513 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:09:03.513 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:09:03.513 INFO:tasks.workunit.client.0.smithi016.stdout:225/600: open fd = 3 2024-09-12T02:09:03.513 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:09:03.513 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:09:03.513 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:09:03.513 INFO:tasks.workunit.client.0.smithi016.stdout:226/600: open fd = 3 2024-09-12T02:09:03.513 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:09:03.513 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:09:03.513 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:09:03.513 INFO:tasks.workunit.client.0.smithi016.stdout:227/600: open fd = 3 2024-09-12T02:09:03.513 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:09:03.514 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:09:03.514 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:09:03.514 INFO:tasks.workunit.client.0.smithi016.stdout:228/600: open fd = 3 2024-09-12T02:09:03.514 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:09:03.514 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:09:03.514 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:09:03.514 INFO:tasks.workunit.client.0.smithi016.stdout:229/600: open fd = 3 2024-09-12T02:09:03.514 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:09:03.514 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:09:03.514 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:09:03.514 INFO:tasks.workunit.client.0.smithi016.stdout:230/600: open fd = 3 2024-09-12T02:09:03.514 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:09:03.514 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:09:03.515 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:09:03.515 INFO:tasks.workunit.client.0.smithi016.stdout:231/600: open fd = 3 2024-09-12T02:09:03.515 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:09:03.515 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:09:03.515 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:09:03.515 INFO:tasks.workunit.client.0.smithi016.stdout:232/600: open fd = 3 2024-09-12T02:09:03.515 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:09:03.515 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:09:03.515 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:09:03.515 INFO:tasks.workunit.client.0.smithi016.stdout:233/600: open fd = 3 2024-09-12T02:09:03.515 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:09:03.515 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:09:03.515 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:09:03.516 INFO:tasks.workunit.client.0.smithi016.stdout:234/600: open fd = 3 2024-09-12T02:09:03.516 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:09:03.516 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:09:03.516 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:09:03.516 INFO:tasks.workunit.client.0.smithi016.stdout:235/600: open fd = 3 2024-09-12T02:09:03.516 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:09:03.516 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:09:03.516 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:09:03.516 INFO:tasks.workunit.client.0.smithi016.stdout:236/600: open fd = 3 2024-09-12T02:09:03.516 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:09:03.516 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:09:03.516 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:09:03.516 INFO:tasks.workunit.client.0.smithi016.stdout:237/600: open fd = 3 2024-09-12T02:09:03.516 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:09:03.517 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:09:03.517 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:09:03.517 INFO:tasks.workunit.client.0.smithi016.stdout:238/600: open fd = 3 2024-09-12T02:09:03.517 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:09:03.517 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:09:03.517 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:09:03.517 INFO:tasks.workunit.client.0.smithi016.stdout:239/600: open fd = 3 2024-09-12T02:09:03.517 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:09:03.517 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:09:03.517 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:09:03.517 INFO:tasks.workunit.client.0.smithi016.stdout:240/600: open fd = 3 2024-09-12T02:09:03.517 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:09:03.517 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:09:03.518 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:09:03.518 INFO:tasks.workunit.client.0.smithi016.stdout:241/600: open fd = 3 2024-09-12T02:09:03.518 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:09:03.518 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:09:03.518 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:09:03.518 INFO:tasks.workunit.client.0.smithi016.stdout:242/600: open fd = 3 2024-09-12T02:09:03.518 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:09:03.518 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:09:03.518 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:09:03.518 INFO:tasks.workunit.client.0.smithi016.stdout:243/600: open fd = 3 2024-09-12T02:09:03.518 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:09:03.518 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:09:03.518 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:09:03.518 INFO:tasks.workunit.client.0.smithi016.stdout:244/600: open fd = 3 2024-09-12T02:09:03.519 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:09:03.519 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:09:03.519 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:09:03.519 INFO:tasks.workunit.client.0.smithi016.stdout:245/600: open fd = 3 2024-09-12T02:09:03.519 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:09:03.519 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:09:03.519 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:09:03.519 INFO:tasks.workunit.client.0.smithi016.stdout:246/600: open fd = 3 2024-09-12T02:09:03.519 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:09:03.519 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:09:03.519 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:09:03.519 INFO:tasks.workunit.client.0.smithi016.stdout:247/600: open fd = 3 2024-09-12T02:09:03.519 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:09:03.519 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:09:03.520 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:09:03.520 INFO:tasks.workunit.client.0.smithi016.stdout:248/600: open fd = 3 2024-09-12T02:09:03.520 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:09:03.520 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:09:03.520 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:09:03.520 INFO:tasks.workunit.client.0.smithi016.stdout:249/600: open fd = 3 2024-09-12T02:09:03.520 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:09:03.520 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:09:03.520 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:09:03.520 INFO:tasks.workunit.client.0.smithi016.stdout:250/600: open fd = 3 2024-09-12T02:09:03.520 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:09:03.520 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:09:04.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:09:04 smithi184 ceph-mon[35593]: pgmap v549: 97 pgs: 97 active+clean; 3.8 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.7 KiB/s wr, 12 op/s 2024-09-12T02:09:04.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:09:04 smithi033 ceph-mon[35332]: pgmap v549: 97 pgs: 97 active+clean; 3.8 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.7 KiB/s wr, 12 op/s 2024-09-12T02:09:04.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:09:04 smithi016 ceph-mon[30947]: pgmap v549: 97 pgs: 97 active+clean; 3.8 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.7 KiB/s wr, 12 op/s 2024-09-12T02:09:06.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:09:06 smithi184 ceph-mon[35593]: pgmap v550: 97 pgs: 97 active+clean; 3.8 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 KiB/s wr, 15 op/s 2024-09-12T02:09:06.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:09:06 smithi033 ceph-mon[35332]: pgmap v550: 97 pgs: 97 active+clean; 3.8 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 KiB/s wr, 15 op/s 2024-09-12T02:09:06.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:09:06 smithi016 ceph-mon[30947]: pgmap v550: 97 pgs: 97 active+clean; 3.8 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 KiB/s wr, 15 op/s 2024-09-12T02:09:07.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:09:07 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:09:07.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:09:07 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:09:07.628 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:09:07 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:09:07.628 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:09:07 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:09:07.628 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:09:07 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:09:07.628 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:09:07 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:09:07.628 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:09:07 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:09:07.628 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:09:07 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:09:07.628 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:09:07 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:09:07.628 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:09:07 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:09:07.628 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:09:07 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:09:07.628 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:09:07 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:09:07.628 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:09:07 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:09:07.628 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:09:07 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:09:07.628 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:09:07 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:09:07.629 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:09:07 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T02:09:07.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:09:07 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:09:07.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:09:07 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:09:07.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:09:07 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:09:07.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:09:07 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:09:07.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:09:07 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:09:07.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:09:07 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:09:07.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:09:07 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:09:07.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:09:07 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:09:07.670 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:09:07 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:09:07.670 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:09:07 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:09:07.670 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:09:07 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:09:07.670 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:09:07 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:09:07.670 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:09:07 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:09:07.670 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:09:07 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:09:07.670 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:09:07 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:09:07.670 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:09:07 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T02:09:07.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:09:07 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:09:07.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:09:07 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:09:07.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:09:07 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:09:07.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:09:07 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:09:07.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:09:07 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:09:07.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:09:07 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:09:07.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:09:07 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:09:07.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:09:07 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:09:07.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:09:07 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:09:07.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:09:07 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:09:07.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:09:07 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:09:07.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:09:07 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:09:07.685 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:09:07 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:09:07.685 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:09:07 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:09:07.685 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:09:07 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:09:07.685 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:09:07 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T02:09:08.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:09:08 smithi016 ceph-mon[30947]: pgmap v551: 97 pgs: 97 active+clean; 3.8 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-09-12T02:09:08.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:09:08 smithi016 ceph-mon[30947]: Adjusting osd_memory_target on smithi016 to 177.0M 2024-09-12T02:09:08.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:09:08 smithi016 ceph-mon[30947]: Adjusting osd_memory_target on smithi184 to 176.0M 2024-09-12T02:09:08.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:09:08 smithi016 ceph-mon[30947]: Adjusting osd_memory_target on smithi033 to 177.0M 2024-09-12T02:09:08.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:09:08 smithi016 ceph-mon[30947]: Unable to set osd_memory_target on smithi016 to 185688473: error parsing value: Value '185688473' is below minimum 939524096 2024-09-12T02:09:08.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:09:08 smithi016 ceph-mon[30947]: Unable to set osd_memory_target on smithi184 to 184595353: error parsing value: Value '184595353' is below minimum 939524096 2024-09-12T02:09:08.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:09:08 smithi016 ceph-mon[30947]: Unable to set osd_memory_target on smithi033 to 185700659: error parsing value: Value '185700659' is below minimum 939524096 2024-09-12T02:09:08.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:09:08 smithi184 ceph-mon[35593]: pgmap v551: 97 pgs: 97 active+clean; 3.8 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-09-12T02:09:08.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:09:08 smithi184 ceph-mon[35593]: Adjusting osd_memory_target on smithi016 to 177.0M 2024-09-12T02:09:08.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:09:08 smithi184 ceph-mon[35593]: Adjusting osd_memory_target on smithi184 to 176.0M 2024-09-12T02:09:08.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:09:08 smithi184 ceph-mon[35593]: Adjusting osd_memory_target on smithi033 to 177.0M 2024-09-12T02:09:08.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:09:08 smithi184 ceph-mon[35593]: Unable to set osd_memory_target on smithi016 to 185688473: error parsing value: Value '185688473' is below minimum 939524096 2024-09-12T02:09:08.628 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:09:08 smithi184 ceph-mon[35593]: Unable to set osd_memory_target on smithi184 to 184595353: error parsing value: Value '184595353' is below minimum 939524096 2024-09-12T02:09:08.628 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:09:08 smithi184 ceph-mon[35593]: Unable to set osd_memory_target on smithi033 to 185700659: error parsing value: Value '185700659' is below minimum 939524096 2024-09-12T02:09:08.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:09:08 smithi033 ceph-mon[35332]: pgmap v551: 97 pgs: 97 active+clean; 3.8 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-09-12T02:09:08.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:09:08 smithi033 ceph-mon[35332]: Adjusting osd_memory_target on smithi016 to 177.0M 2024-09-12T02:09:08.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:09:08 smithi033 ceph-mon[35332]: Adjusting osd_memory_target on smithi184 to 176.0M 2024-09-12T02:09:08.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:09:08 smithi033 ceph-mon[35332]: Adjusting osd_memory_target on smithi033 to 177.0M 2024-09-12T02:09:08.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:09:08 smithi033 ceph-mon[35332]: Unable to set osd_memory_target on smithi016 to 185688473: error parsing value: Value '185688473' is below minimum 939524096 2024-09-12T02:09:08.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:09:08 smithi033 ceph-mon[35332]: Unable to set osd_memory_target on smithi184 to 184595353: error parsing value: Value '184595353' is below minimum 939524096 2024-09-12T02:09:08.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:09:08 smithi033 ceph-mon[35332]: Unable to set osd_memory_target on smithi033 to 185700659: error parsing value: Value '185700659' is below minimum 939524096 2024-09-12T02:09:10.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:09:10 smithi033 ceph-mon[35332]: pgmap v552: 97 pgs: 97 active+clean; 3.8 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-09-12T02:09:10.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:09:10 smithi016 ceph-mon[30947]: pgmap v552: 97 pgs: 97 active+clean; 3.8 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-09-12T02:09:10.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:09:10 smithi184 ceph-mon[35593]: pgmap v552: 97 pgs: 97 active+clean; 3.8 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-09-12T02:09:11.378 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.377+0000 7fc254aeb640 1 -- 172.21.15.16:0/951209210 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc250101b60 msgr2=0x7fc250101f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:11.378 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.377+0000 7fc254aeb640 1 --2- 172.21.15.16:0/951209210 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc250101b60 0x7fc250101f60 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7fc240009900 tx=0x7fc24002a410 comp rx=0 tx=0).stop 2024-09-12T02:09:11.379 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.377+0000 7fc254aeb640 1 -- 172.21.15.16:0/951209210 shutdown_connections 2024-09-12T02:09:11.379 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.377+0000 7fc254aeb640 1 --2- 172.21.15.16:0/951209210 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc250103710 0x7fc250109fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:11.379 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.377+0000 7fc254aeb640 1 --2- 172.21.15.16:0/951209210 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc250102d50 0x7fc2501031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:11.379 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.377+0000 7fc254aeb640 1 --2- 172.21.15.16:0/951209210 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc250101b60 0x7fc250101f60 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:11.379 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.377+0000 7fc254aeb640 1 -- 172.21.15.16:0/951209210 >> 172.21.15.16:0/951209210 conn(0x7fc2500fd2e0 msgr2=0x7fc2500ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:11.379 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.377+0000 7fc254aeb640 1 -- 172.21.15.16:0/951209210 shutdown_connections 2024-09-12T02:09:11.379 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.377+0000 7fc254aeb640 1 -- 172.21.15.16:0/951209210 wait complete. 2024-09-12T02:09:11.381 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.378+0000 7fc254aeb640 1 Processor -- start 2024-09-12T02:09:11.381 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.378+0000 7fc254aeb640 1 -- start start 2024-09-12T02:09:11.381 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.378+0000 7fc254aeb640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc250101b60 0x7fc250197dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:11.381 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.378+0000 7fc254aeb640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc250102d50 0x7fc250198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:11.381 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.378+0000 7fc254aeb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc250103710 0x7fc25019f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:11.381 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.378+0000 7fc254aeb640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc25010d7e0 con 0x7fc250103710 2024-09-12T02:09:11.382 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.378+0000 7fc254aeb640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc25010d660 con 0x7fc250101b60 2024-09-12T02:09:11.382 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.378+0000 7fc254aeb640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc25010d960 con 0x7fc250102d50 2024-09-12T02:09:11.382 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.378+0000 7fc24ed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc250103710 0x7fc25019f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:11.382 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.378+0000 7fc24ed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc250103710 0x7fc25019f380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41748/0 (socket says 172.21.15.16:41748) 2024-09-12T02:09:11.382 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.378+0000 7fc24ed9d640 1 -- 172.21.15.16:0/2760105930 learned_addr learned my addr 172.21.15.16:0/2760105930 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:09:11.382 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.378+0000 7fc24e59c640 1 --2- 172.21.15.16:0/2760105930 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc250101b60 0x7fc250197dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:11.382 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.378+0000 7fc24dd9b640 1 --2- 172.21.15.16:0/2760105930 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc250102d50 0x7fc250198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:11.382 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.379+0000 7fc24ed9d640 1 -- 172.21.15.16:0/2760105930 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc250102d50 msgr2=0x7fc250198310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:11.382 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.379+0000 7fc24ed9d640 1 --2- 172.21.15.16:0/2760105930 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc250102d50 0x7fc250198310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:11.382 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.379+0000 7fc24ed9d640 1 -- 172.21.15.16:0/2760105930 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc250101b60 msgr2=0x7fc250197dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:11.382 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.379+0000 7fc24ed9d640 1 --2- 172.21.15.16:0/2760105930 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc250101b60 0x7fc250197dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:11.382 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.379+0000 7fc24ed9d640 1 -- 172.21.15.16:0/2760105930 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc25019f9f0 con 0x7fc250103710 2024-09-12T02:09:11.382 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.379+0000 7fc24ed9d640 1 --2- 172.21.15.16:0/2760105930 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc250103710 0x7fc25019f380 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7fc24800eb10 tx=0x7fc24800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:11.383 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.379+0000 7fc23f7fe640 1 -- 172.21.15.16:0/2760105930 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc248004b20 con 0x7fc250103710 2024-09-12T02:09:11.383 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.379+0000 7fc254aeb640 1 -- 172.21.15.16:0/2760105930 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc25019fce0 con 0x7fc250103710 2024-09-12T02:09:11.383 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.379+0000 7fc23f7fe640 1 -- 172.21.15.16:0/2760105930 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc248004cc0 con 0x7fc250103710 2024-09-12T02:09:11.383 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.379+0000 7fc23f7fe640 1 -- 172.21.15.16:0/2760105930 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc248010760 con 0x7fc250103710 2024-09-12T02:09:11.383 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.379+0000 7fc254aeb640 1 -- 172.21.15.16:0/2760105930 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2501a01d0 con 0x7fc250103710 2024-09-12T02:09:11.390 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.382+0000 7fc23f7fe640 1 -- 172.21.15.16:0/2760105930 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fc248010900 con 0x7fc250103710 2024-09-12T02:09:11.390 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.382+0000 7fc25788a740 1 Processor -- start 2024-09-12T02:09:11.390 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.382+0000 7fc25788a740 1 -- start start 2024-09-12T02:09:11.390 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.382+0000 7fc25788a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b6789edb0 0x560b6789f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:11.390 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.382+0000 7fc25788a740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560b6789f6f0 0x560b678a41b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:11.390 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.382+0000 7fc25788a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b678a4780 0x560b678a6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:11.391 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.382+0000 7fc25788a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x560b677fe2f0 con 0x560b678a4780 2024-09-12T02:09:11.391 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.382+0000 7fc25788a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x560b677a6270 con 0x560b6789edb0 2024-09-12T02:09:11.391 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.382+0000 7fc25788a740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x560b677fa2b0 con 0x560b6789f6f0 2024-09-12T02:09:11.391 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.383+0000 7fc24dd9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b6789edb0 0x560b6789f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:11.391 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.383+0000 7fc24dd9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b6789edb0 0x560b6789f1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.16:33120/0 (socket says 172.21.15.16:33120) 2024-09-12T02:09:11.391 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.383+0000 7fc24e59c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560b6789f6f0 0x560b678a41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:11.391 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.383+0000 7fc23f7fe640 1 --2- 172.21.15.16:0/2760105930 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc2380a2c30 0x7fc2380a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:11.391 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.383+0000 7fc24dd9b640 1 -- 172.21.15.16:0/120465403 learned_addr learned my addr 172.21.15.16:0/120465403 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:09:11.391 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.383+0000 7fc24dd9b640 1 --2- 172.21.15.16:0/120465403 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b678a4780 0x560b678a6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:11.391 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.383+0000 7fc24ed9d640 1 --2- 172.21.15.16:0/2760105930 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc2380a2c30 0x7fc2380a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:11.391 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.383+0000 7fc24e59c640 1 -- 172.21.15.16:0/120465403 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b6789edb0 msgr2=0x560b6789f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:09:11.391 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.383+0000 7fc24e59c640 1 --2- 172.21.15.16:0/120465403 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b6789edb0 0x560b6789f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:11.391 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.383+0000 7fc23f7fe640 1 -- 172.21.15.16:0/2760105930 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fc2480cf670 con 0x7fc250103710 2024-09-12T02:09:11.391 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.383+0000 7fc24e59c640 1 -- 172.21.15.16:0/120465403 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b678a4780 msgr2=0x560b678a6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:09:11.392 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.383+0000 7fc24e59c640 1 --2- 172.21.15.16:0/120465403 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b678a4780 0x560b678a6c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:11.392 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.383+0000 7fc24e59c640 1 -- 172.21.15.16:0/120465403 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b677a5340 con 0x560b6789f6f0 2024-09-12T02:09:11.392 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.383+0000 7fc24e59c640 1 --2- 172.21.15.16:0/120465403 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560b6789f6f0 0x560b678a41b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fc240009230 tx=0x7fc240005bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cf8eb51a05f00ce8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:11.392 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.383+0000 7fc24ed9d640 1 --2- 172.21.15.16:0/2760105930 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc2380a2c30 0x7fc2380a50f0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fc248098940 tx=0x7fc248091450 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:11.392 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.384+0000 7fc23cff9640 1 -- 172.21.15.16:0/120465403 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc240040070 con 0x560b6789f6f0 2024-09-12T02:09:11.392 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.384+0000 7fc23cff9640 1 -- 172.21.15.16:0/120465403 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc240004030 con 0x560b6789f6f0 2024-09-12T02:09:11.392 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.384+0000 7fc23cff9640 1 -- 172.21.15.16:0/120465403 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc240035040 con 0x560b6789f6f0 2024-09-12T02:09:11.392 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.384+0000 7fc25788a740 1 -- 172.21.15.16:0/120465403 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560b6789f6f0 msgr2=0x560b678a41b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:11.392 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.384+0000 7fc25788a740 1 --2- 172.21.15.16:0/120465403 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560b6789f6f0 0x560b678a41b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fc240009230 tx=0x7fc240005bf0 comp rx=0 tx=0).stop 2024-09-12T02:09:11.392 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.384+0000 7fc25788a740 1 -- 172.21.15.16:0/120465403 shutdown_connections 2024-09-12T02:09:11.392 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.384+0000 7fc25788a740 1 --2- 172.21.15.16:0/120465403 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b678a4780 0x560b678a6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:11.392 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.384+0000 7fc25788a740 1 --2- 172.21.15.16:0/120465403 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560b6789f6f0 0x560b678a41b0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:11.392 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.384+0000 7fc25788a740 1 --2- 172.21.15.16:0/120465403 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b6789edb0 0x560b6789f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:11.393 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.384+0000 7fc25788a740 1 -- 172.21.15.16:0/120465403 >> 172.21.15.16:0/120465403 conn(0x560b67896010 msgr2=0x560b67896410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:11.393 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.385+0000 7fc25788a740 1 -- 172.21.15.16:0/120465403 shutdown_connections 2024-09-12T02:09:11.393 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.385+0000 7fc25788a740 1 -- 172.21.15.16:0/120465403 wait complete. 2024-09-12T02:09:11.393 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.385+0000 7fc25788a740 1 -- start start 2024-09-12T02:09:11.393 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.385+0000 7fc25788a740 1 Processor -- start 2024-09-12T02:09:11.393 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.385+0000 7fc25788a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b6789edb0 0x560b678a8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:11.393 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.385+0000 7fc25788a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b6789f6f0 0x560b678a94e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:11.393 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.385+0000 7fc25788a740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560b678a4780 0x560b678b0520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:11.393 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.385+0000 7fc25788a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x560b678a76f0 con 0x560b6789f6f0 2024-09-12T02:09:11.393 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.385+0000 7fc25788a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x560b67787290 con 0x560b6789edb0 2024-09-12T02:09:11.394 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.385+0000 7fc25788a740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x560b678a7870 con 0x560b678a4780 2024-09-12T02:09:11.394 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.385+0000 7fc25788a740 10 client.0 _ll_register_callbacks cb 0x560b677b8140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:09:11.394 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.385+0000 7fc24e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b6789f6f0 0x560b678a94e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:11.394 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.385+0000 7fc24e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b6789f6f0 0x560b678a94e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41770/0 (socket says 172.21.15.16:41770) 2024-09-12T02:09:11.394 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.385+0000 7fc24e59c640 1 -- 172.21.15.16:0/243820117 learned_addr learned my addr 172.21.15.16:0/243820117 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:09:11.394 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.386+0000 7fc24dd9b640 1 --2- 172.21.15.16:0/243820117 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560b678a4780 0x560b678b0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:11.394 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.386+0000 7fc24dd9b640 1 --2- 172.21.15.16:0/243820117 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b6789edb0 0x560b678a8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:11.394 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.386+0000 7fc24e59c640 1 -- 172.21.15.16:0/243820117 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560b678a4780 msgr2=0x560b678b0520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:09:11.394 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.386+0000 7fc24e59c640 1 --2- 172.21.15.16:0/243820117 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560b678a4780 0x560b678b0520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:11.394 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.386+0000 7fc24e59c640 1 -- 172.21.15.16:0/243820117 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b6789edb0 msgr2=0x560b678a8fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:09:11.394 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.386+0000 7fc24e59c640 1 --2- 172.21.15.16:0/243820117 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b6789edb0 0x560b678a8fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:11.394 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.386+0000 7fc24e59c640 1 -- 172.21.15.16:0/243820117 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b67511210 con 0x560b6789f6f0 2024-09-12T02:09:11.394 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.386+0000 7fc24dd9b640 1 --2- 172.21.15.16:0/243820117 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b6789edb0 0x560b678a8fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:09:11.395 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.386+0000 7fc24e59c640 1 --2- 172.21.15.16:0/243820117 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b6789f6f0 0x560b678a94e0 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7fc24002a930 tx=0x7fc2400095a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f9812f493aa52e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:11.395 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.386+0000 7fc235ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:09:11.395 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.386+0000 7fc235ffb640 1 -- 172.21.15.16:0/243820117 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc24004c020 con 0x560b6789f6f0 2024-09-12T02:09:11.395 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.386+0000 7fc235ffb640 1 -- 172.21.15.16:0/243820117 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fc2400043e0 con 0x560b6789f6f0 2024-09-12T02:09:11.395 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.386+0000 7fc25788a740 1 -- 172.21.15.16:0/243820117 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc240004030 con 0x560b6789f6f0 2024-09-12T02:09:11.395 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.386+0000 7fc235ffb640 1 -- 172.21.15.16:0/243820117 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc240040070 con 0x560b6789f6f0 2024-09-12T02:09:11.395 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.386+0000 7fc25788a740 10 client.17667 fetch_fsmap learned FSMap version 25 2024-09-12T02:09:11.395 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.386+0000 7fc25788a740 1 -- 172.21.15.16:0/243820117 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x560b675dd620 con 0x560b6789f6f0 2024-09-12T02:09:11.395 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.387+0000 7fc235ffb640 1 -- 172.21.15.16:0/243820117 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc240004030 con 0x560b6789f6f0 2024-09-12T02:09:11.395 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.387+0000 7fc235ffb640 1 -- 172.21.15.16:0/243820117 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fc240004e20 con 0x560b6789f6f0 2024-09-12T02:09:11.395 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.387+0000 7fc25788a740 10 client.17667 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:09:11.395 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.387+0000 7fc25788a740 10 client.17667 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:09:11.395 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.387+0000 7fc25788a740 20 client.17667 populate_metadata read hostname 'smithi016' 2024-09-12T02:09:11.396 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.387+0000 7fc25788a740 1 --2- 172.21.15.16:0/243820117 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x560b678b4820 0x560b678b6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:11.396 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.387+0000 7fc25788a740 4 client.17667 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:09:11.396 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.387+0000 7fc25788a740 1 -- 172.21.15.16:0/243820117 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560b6780ba20 con 0x560b678b4820 2024-09-12T02:09:11.396 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.387+0000 7fc24dd9b640 1 --2- 172.21.15.16:0/243820117 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x560b678b4820 0x560b678b6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:11.396 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.388+0000 7fc235ffb640 10 client.17667 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:09:11.396 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.388+0000 7fc24dd9b640 1 --2- 172.21.15.16:0/243820117 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x560b678b4820 0x560b678b6c10 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c6be5fdade33d4b server_cookie=52091acbdf597db7 in_seq=0 out_seq=0 2024-09-12T02:09:11.396 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.388+0000 7fc235ffb640 1 -- 172.21.15.16:0/243820117 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560b6780ba20 con 0x560b678b4820 2024-09-12T02:09:11.396 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.388+0000 7fc235ffb640 10 client.17667 handle_command_reply: tid=0 2024-09-12T02:09:11.396 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.388+0000 7fc25788a740 1 client.17667 shutdown 2024-09-12T02:09:11.396 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.389+0000 7fc25788a740 1 -- 172.21.15.16:0/243820117 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b6789f6f0 msgr2=0x560b678a94e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:11.396 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.389+0000 7fc25788a740 1 --2- 172.21.15.16:0/243820117 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b6789f6f0 0x560b678a94e0 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7fc24002a930 tx=0x7fc2400095a0 comp rx=0 tx=0).stop 2024-09-12T02:09:11.396 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.389+0000 7fc25788a740 1 -- 172.21.15.16:0/243820117 shutdown_connections 2024-09-12T02:09:11.397 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.389+0000 7fc25788a740 1 --2- 172.21.15.16:0/243820117 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x560b678b4820 0x560b678b6c10 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:11.397 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.389+0000 7fc25788a740 1 --2- 172.21.15.16:0/243820117 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560b678a4780 0x560b678b0520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:11.397 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.389+0000 7fc235ffb640 0 client.17667 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:09:11.397 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.389+0000 7fc25788a740 1 --2- 172.21.15.16:0/243820117 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b6789f6f0 0x560b678a94e0 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:11.397 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.389+0000 7fc25788a740 1 --2- 172.21.15.16:0/243820117 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b6789edb0 0x560b678a8fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:11.397 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.389+0000 7fc25788a740 1 -- 172.21.15.16:0/243820117 >> 172.21.15.16:0/243820117 conn(0x560b67896010 msgr2=0x560b67897820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:11.397 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.389+0000 7fc25788a740 1 -- 172.21.15.16:0/243820117 shutdown_connections 2024-09-12T02:09:11.397 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.389+0000 7fc25788a740 1 -- 172.21.15.16:0/243820117 wait complete. 2024-09-12T02:09:11.397 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.389+0000 7fc25788a740 20 client.17667 trim_cache size 0 max 16384 2024-09-12T02:09:11.409 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.404+0000 7fc25788a740 1 Processor -- start 2024-09-12T02:09:11.410 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.404+0000 7fc25788a740 1 -- start start 2024-09-12T02:09:11.410 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.404+0000 7fc25788a740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560b6789edb0 0x560b67896620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:11.410 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.404+0000 7fc25788a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b6789f6f0 0x560b678b04e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:11.410 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.404+0000 7fc25788a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b678a4780 0x560b678950b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:11.410 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.404+0000 7fc25788a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc2501a94d0 con 0x560b6789f6f0 2024-09-12T02:09:11.410 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.404+0000 7fc25788a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc2501a6da0 con 0x560b678a4780 2024-09-12T02:09:11.410 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.404+0000 7fc25788a740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc2501a4c40 con 0x560b6789edb0 2024-09-12T02:09:11.410 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.404+0000 7fc24e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b6789f6f0 0x560b678b04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:11.410 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.404+0000 7fc24e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b6789f6f0 0x560b678b04e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41786/0 (socket says 172.21.15.16:41786) 2024-09-12T02:09:11.410 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.404+0000 7fc24e59c640 1 -- 172.21.15.16:0/2808501407 learned_addr learned my addr 172.21.15.16:0/2808501407 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:09:11.410 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.404+0000 7fc24dd9b640 1 --2- 172.21.15.16:0/2808501407 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560b6789edb0 0x560b67896620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:11.410 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.404+0000 7fc24e59c640 1 -- 172.21.15.16:0/2808501407 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560b6789edb0 msgr2=0x560b67896620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:09:11.411 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.404+0000 7fc24e59c640 1 --2- 172.21.15.16:0/2808501407 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560b6789edb0 0x560b67896620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:11.411 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.404+0000 7fc24e59c640 1 -- 172.21.15.16:0/2808501407 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b678a4780 msgr2=0x560b678950b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:09:11.411 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.404+0000 7fc24e59c640 1 --2- 172.21.15.16:0/2808501407 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b678a4780 0x560b678950b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:11.411 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.404+0000 7fc24e59c640 1 -- 172.21.15.16:0/2808501407 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b678912d0 con 0x560b6789f6f0 2024-09-12T02:09:11.411 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.404+0000 7fc24dd9b640 1 --2- 172.21.15.16:0/2808501407 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560b6789edb0 0x560b67896620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:09:11.411 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.405+0000 7fc24e59c640 1 --2- 172.21.15.16:0/2808501407 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b6789f6f0 0x560b678b04e0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7fc2400037b0 tx=0x7fc24002ab10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d2e64f8ed1663a0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:11.411 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.405+0000 7fc237fff640 1 -- 172.21.15.16:0/2808501407 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc24004c070 con 0x560b6789f6f0 2024-09-12T02:09:11.411 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.405+0000 7fc237fff640 1 -- 172.21.15.16:0/2808501407 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2400318b0 con 0x560b6789f6f0 2024-09-12T02:09:11.411 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.405+0000 7fc237fff640 1 -- 172.21.15.16:0/2808501407 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc240035040 con 0x560b6789f6f0 2024-09-12T02:09:11.411 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.405+0000 7fc25788a740 1 -- 172.21.15.16:0/2808501407 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b6789f6f0 msgr2=0x560b678b04e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:11.412 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.405+0000 7fc25788a740 1 --2- 172.21.15.16:0/2808501407 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b6789f6f0 0x560b678b04e0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7fc2400037b0 tx=0x7fc24002ab10 comp rx=0 tx=0).stop 2024-09-12T02:09:11.412 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.405+0000 7fc25788a740 1 -- 172.21.15.16:0/2808501407 shutdown_connections 2024-09-12T02:09:11.412 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.405+0000 7fc25788a740 1 --2- 172.21.15.16:0/2808501407 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b678a4780 0x560b678950b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:11.412 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.405+0000 7fc25788a740 1 --2- 172.21.15.16:0/2808501407 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b6789f6f0 0x560b678b04e0 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:11.412 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.405+0000 7fc25788a740 1 --2- 172.21.15.16:0/2808501407 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560b6789edb0 0x560b67896620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:11.412 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.405+0000 7fc25788a740 1 -- 172.21.15.16:0/2808501407 >> 172.21.15.16:0/2808501407 conn(0x560b67896010 msgr2=0x560b678949a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:11.412 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.405+0000 7fc25788a740 1 -- 172.21.15.16:0/2808501407 shutdown_connections 2024-09-12T02:09:11.412 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.405+0000 7fc25788a740 1 -- 172.21.15.16:0/2808501407 wait complete. 2024-09-12T02:09:11.412 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.406+0000 7fc25788a740 1 -- start start 2024-09-12T02:09:11.412 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.406+0000 7fc25788a740 1 Processor -- start 2024-09-12T02:09:11.412 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.406+0000 7fc25788a740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560b6789edb0 0x560b678ae1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:11.412 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.406+0000 7fc25788a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b6789f6f0 0x560b678ae6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:11.412 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.406+0000 7fc25788a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b678a4780 0x560b678af6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:11.413 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.406+0000 7fc25788a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc2501a5260 con 0x560b6789f6f0 2024-09-12T02:09:11.413 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.406+0000 7fc25788a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc2501a66d0 con 0x560b678a4780 2024-09-12T02:09:11.413 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.406+0000 7fc25788a740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc2501a5930 con 0x560b6789edb0 2024-09-12T02:09:11.413 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.406+0000 7fc25788a740 10 client.0 _ll_register_callbacks cb 0x7fc24400be90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:09:11.413 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.406+0000 7fc24e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b6789f6f0 0x560b678ae6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:11.413 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.406+0000 7fc24e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b6789f6f0 0x560b678ae6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41792/0 (socket says 172.21.15.16:41792) 2024-09-12T02:09:11.413 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.406+0000 7fc24e59c640 1 -- 172.21.15.16:0/2597984064 learned_addr learned my addr 172.21.15.16:0/2597984064 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:09:11.413 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.406+0000 7fc24dd9b640 1 --2- 172.21.15.16:0/2597984064 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b678a4780 0x560b678af6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:11.413 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.406+0000 7fc24dd9b640 1 --2- 172.21.15.16:0/2597984064 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560b6789edb0 0x560b678ae1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:11.413 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.406+0000 7fc24e59c640 1 -- 172.21.15.16:0/2597984064 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560b6789edb0 msgr2=0x560b678ae1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:09:11.413 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.406+0000 7fc24e59c640 1 --2- 172.21.15.16:0/2597984064 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560b6789edb0 0x560b678ae1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:11.413 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.406+0000 7fc24e59c640 1 -- 172.21.15.16:0/2597984064 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b678a4780 msgr2=0x560b678af6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:09:11.413 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.406+0000 7fc24e59c640 1 --2- 172.21.15.16:0/2597984064 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b678a4780 0x560b678af6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:11.414 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.406+0000 7fc24e59c640 1 -- 172.21.15.16:0/2597984064 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b678afcb0 con 0x560b6789f6f0 2024-09-12T02:09:11.414 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.406+0000 7fc24dd9b640 1 --2- 172.21.15.16:0/2597984064 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560b6789edb0 0x560b678ae1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:09:11.414 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.407+0000 7fc24e59c640 1 --2- 172.21.15.16:0/2597984064 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b6789f6f0 0x560b678ae6e0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7fc240033000 tx=0x7fc240000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=898ae9b257bffc14 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:11.414 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.407+0000 7fc235ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:09:11.414 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.407+0000 7fc235ffb640 1 -- 172.21.15.16:0/2597984064 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc240061060 con 0x560b6789f6f0 2024-09-12T02:09:11.414 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.407+0000 7fc235ffb640 1 -- 172.21.15.16:0/2597984064 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fc240053530 con 0x560b6789f6f0 2024-09-12T02:09:11.414 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.407+0000 7fc25788a740 1 -- 172.21.15.16:0/2597984064 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc2400318b0 con 0x560b6789f6f0 2024-09-12T02:09:11.414 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.407+0000 7fc235ffb640 1 -- 172.21.15.16:0/2597984064 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc240035070 con 0x560b6789f6f0 2024-09-12T02:09:11.414 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.407+0000 7fc25788a740 10 client.17673 fetch_fsmap learned FSMap version 25 2024-09-12T02:09:11.414 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.407+0000 7fc25788a740 1 -- 172.21.15.16:0/2597984064 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x560b6791f8f0 con 0x560b6789f6f0 2024-09-12T02:09:11.414 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.407+0000 7fc235ffb640 1 -- 172.21.15.16:0/2597984064 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2400318b0 con 0x560b6789f6f0 2024-09-12T02:09:11.414 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.407+0000 7fc235ffb640 1 -- 172.21.15.16:0/2597984064 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fc240066070 con 0x560b6789f6f0 2024-09-12T02:09:11.414 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.407+0000 7fc25788a740 10 client.17673 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:09:11.414 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.407+0000 7fc25788a740 10 client.17673 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:09:11.415 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.407+0000 7fc25788a740 20 client.17673 populate_metadata read hostname 'smithi016' 2024-09-12T02:09:11.415 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.407+0000 7fc25788a740 1 --2- 172.21.15.16:0/2597984064 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x560b678b4820 0x560b679220f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:11.415 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.407+0000 7fc25788a740 4 client.17673 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:09:11.415 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.407+0000 7fc25788a740 1 -- 172.21.15.16:0/2597984064 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560b67805d70 con 0x560b678b4820 2024-09-12T02:09:11.415 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.408+0000 7fc24dd9b640 1 --2- 172.21.15.16:0/2597984064 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x560b678b4820 0x560b679220f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:11.415 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.408+0000 7fc235ffb640 10 client.17673 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:09:11.415 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.408+0000 7fc24dd9b640 1 --2- 172.21.15.16:0/2597984064 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x560b678b4820 0x560b679220f0 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee5d432d205938a5 server_cookie=8ef8ac801429bb32 in_seq=0 out_seq=0 2024-09-12T02:09:11.423 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.422+0000 7fc235ffb640 1 -- 172.21.15.16:0/2597984064 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560b67805d70 con 0x560b678b4820 2024-09-12T02:09:11.423 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.422+0000 7fc235ffb640 10 client.17673 handle_command_reply: tid=0 2024-09-12T02:09:11.423 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.422+0000 7fc25788a740 1 client.17673 shutdown 2024-09-12T02:09:11.424 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.422+0000 7fc25788a740 1 -- 172.21.15.16:0/2597984064 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b6789f6f0 msgr2=0x560b678ae6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:11.424 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.422+0000 7fc25788a740 1 --2- 172.21.15.16:0/2597984064 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b6789f6f0 0x560b678ae6e0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7fc240033000 tx=0x7fc240000bc0 comp rx=0 tx=0).stop 2024-09-12T02:09:11.424 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.422+0000 7fc25788a740 1 -- 172.21.15.16:0/2597984064 shutdown_connections 2024-09-12T02:09:11.424 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.422+0000 7fc25788a740 1 --2- 172.21.15.16:0/2597984064 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x560b678b4820 0x560b679220f0 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:11.424 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.422+0000 7fc25788a740 1 --2- 172.21.15.16:0/2597984064 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b678a4780 0x560b678af6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:11.424 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.422+0000 7fc25788a740 1 --2- 172.21.15.16:0/2597984064 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b6789f6f0 0x560b678ae6e0 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:11.424 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.422+0000 7fc235ffb640 0 client.17673 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:09:11.424 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.422+0000 7fc25788a740 1 --2- 172.21.15.16:0/2597984064 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560b6789edb0 0x560b678ae1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:11.424 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.422+0000 7fc25788a740 1 -- 172.21.15.16:0/2597984064 >> 172.21.15.16:0/2597984064 conn(0x560b67896010 msgr2=0x560b678951c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:11.424 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.422+0000 7fc25788a740 1 -- 172.21.15.16:0/2597984064 shutdown_connections 2024-09-12T02:09:11.424 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.422+0000 7fc25788a740 1 -- 172.21.15.16:0/2597984064 wait complete. 2024-09-12T02:09:11.424 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.422+0000 7fc25788a740 20 client.17673 trim_cache size 0 max 16384 2024-09-12T02:09:11.425 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-09-12T02:09:11.425 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-09-12T02:09:11.425 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-09-12T02:09:11.425 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-09-12T02:09:11.426 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.423+0000 7fc254aeb640 1 -- 172.21.15.16:0/2760105930 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc2380a2c30 msgr2=0x7fc2380a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:11.426 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.423+0000 7fc254aeb640 1 --2- 172.21.15.16:0/2760105930 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc2380a2c30 0x7fc2380a50f0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fc248098940 tx=0x7fc248091450 comp rx=0 tx=0).stop 2024-09-12T02:09:11.426 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.423+0000 7fc254aeb640 1 -- 172.21.15.16:0/2760105930 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc250103710 msgr2=0x7fc25019f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:11.426 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.423+0000 7fc254aeb640 1 --2- 172.21.15.16:0/2760105930 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc250103710 0x7fc25019f380 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7fc24800eb10 tx=0x7fc24800c2d0 comp rx=0 tx=0).stop 2024-09-12T02:09:11.426 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.423+0000 7fc254aeb640 1 -- 172.21.15.16:0/2760105930 shutdown_connections 2024-09-12T02:09:11.426 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.423+0000 7fc254aeb640 1 --2- 172.21.15.16:0/2760105930 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc2380a2c30 0x7fc2380a50f0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:11.426 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.423+0000 7fc254aeb640 1 --2- 172.21.15.16:0/2760105930 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc250103710 0x7fc25019f380 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:11.426 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.423+0000 7fc254aeb640 1 --2- 172.21.15.16:0/2760105930 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc250102d50 0x7fc250198310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:11.426 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.423+0000 7fc254aeb640 1 --2- 172.21.15.16:0/2760105930 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc250101b60 0x7fc250197dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:11.426 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.423+0000 7fc254aeb640 1 -- 172.21.15.16:0/2760105930 >> 172.21.15.16:0/2760105930 conn(0x7fc2500fd2e0 msgr2=0x7fc2501060d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:11.427 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.423+0000 7fc254aeb640 1 -- 172.21.15.16:0/2760105930 shutdown_connections 2024-09-12T02:09:11.427 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:11.423+0000 7fc254aeb640 1 -- 172.21.15.16:0/2760105930 wait complete. 2024-09-12T02:09:12.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:09:12 smithi033 ceph-mon[35332]: pgmap v553: 97 pgs: 97 active+clean; 3.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-09-12T02:09:12.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:09:12 smithi016 ceph-mon[30947]: pgmap v553: 97 pgs: 97 active+clean; 3.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-09-12T02:09:12.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:09:12 smithi184 ceph-mon[35593]: pgmap v553: 97 pgs: 97 active+clean; 3.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-09-12T02:09:14.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:09:14 smithi033 ceph-mon[35332]: pgmap v554: 97 pgs: 97 active+clean; 3.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T02:09:14.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:09:14 smithi016 ceph-mon[30947]: pgmap v554: 97 pgs: 97 active+clean; 3.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T02:09:14.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:09:14 smithi184 ceph-mon[35593]: pgmap v554: 97 pgs: 97 active+clean; 3.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T02:09:15.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:09:15 smithi184 ceph-mon[35593]: pgmap v555: 97 pgs: 97 active+clean; 3.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 13 op/s 2024-09-12T02:09:15.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:09:15 smithi033 ceph-mon[35332]: pgmap v555: 97 pgs: 97 active+clean; 3.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 13 op/s 2024-09-12T02:09:15.933 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:09:15 smithi016 ceph-mon[30947]: pgmap v555: 97 pgs: 97 active+clean; 3.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 13 op/s 2024-09-12T02:09:17.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:09:17 smithi184 ceph-mon[35593]: pgmap v556: 97 pgs: 97 active+clean; 3.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-09-12T02:09:17.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:09:17 smithi033 ceph-mon[35332]: pgmap v556: 97 pgs: 97 active+clean; 3.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-09-12T02:09:17.933 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:09:17 smithi016 ceph-mon[30947]: pgmap v556: 97 pgs: 97 active+clean; 3.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-09-12T02:09:19.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:09:19 smithi184 ceph-mon[35593]: pgmap v557: 97 pgs: 97 active+clean; 3.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-09-12T02:09:19.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:09:19 smithi033 ceph-mon[35332]: pgmap v557: 97 pgs: 97 active+clean; 3.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-09-12T02:09:19.933 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:09:19 smithi016 ceph-mon[30947]: pgmap v557: 97 pgs: 97 active+clean; 3.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-09-12T02:09:21.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:09:21 smithi184 ceph-mon[35593]: pgmap v558: 97 pgs: 97 active+clean; 4.0 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-09-12T02:09:21.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:09:21 smithi033 ceph-mon[35332]: pgmap v558: 97 pgs: 97 active+clean; 4.0 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-09-12T02:09:21.933 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:09:21 smithi016 ceph-mon[30947]: pgmap v558: 97 pgs: 97 active+clean; 4.0 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-09-12T02:09:23.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:09:23 smithi184 ceph-mon[35593]: pgmap v559: 97 pgs: 97 active+clean; 4.0 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-09-12T02:09:23.919 INFO:tasks.quiescer.fs.[cephfs]:Total duration (38.8) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-12T02:09:23.919 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 19.4 2024-09-12T02:09:23.919 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:09:23.920 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:09:23 smithi033 ceph-mon[35332]: pgmap v559: 97 pgs: 97 active+clean; 4.0 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-09-12T02:09:23.933 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:09:23 smithi016 ceph-mon[30947]: pgmap v559: 97 pgs: 97 active+clean; 4.0 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-09-12T02:09:24.191 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:09:24.516 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.516+0000 7f5f90dd8640 1 -- 172.21.15.16:0/2522368317 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f8c100a40 msgr2=0x7f5f8c100e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:24.517 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.516+0000 7f5f90dd8640 1 --2- 172.21.15.16:0/2522368317 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f8c100a40 0x7f5f8c100e40 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c009900 tx=0x7f5f7c02a410 comp rx=0 tx=0).stop 2024-09-12T02:09:24.517 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.516+0000 7f5f90dd8640 1 -- 172.21.15.16:0/2522368317 shutdown_connections 2024-09-12T02:09:24.517 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.516+0000 7f5f90dd8640 1 --2- 172.21.15.16:0/2522368317 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5f8c1025f0 0x7f5f8c108e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:24.517 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.516+0000 7f5f90dd8640 1 --2- 172.21.15.16:0/2522368317 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f8c101c30 0x7f5f8c1020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:24.517 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.516+0000 7f5f90dd8640 1 --2- 172.21.15.16:0/2522368317 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f8c100a40 0x7f5f8c100e40 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:24.517 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.516+0000 7f5f90dd8640 1 -- 172.21.15.16:0/2522368317 >> 172.21.15.16:0/2522368317 conn(0x7f5f8c0fc1e0 msgr2=0x7f5f8c0fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:24.517 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.516+0000 7f5f90dd8640 1 -- 172.21.15.16:0/2522368317 shutdown_connections 2024-09-12T02:09:24.517 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.516+0000 7f5f90dd8640 1 -- 172.21.15.16:0/2522368317 wait complete. 2024-09-12T02:09:24.519 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.517+0000 7f5f90dd8640 1 Processor -- start 2024-09-12T02:09:24.520 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.517+0000 7f5f90dd8640 1 -- start start 2024-09-12T02:09:24.520 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.517+0000 7f5f90dd8640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5f8c100a40 0x7f5f8c10b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:24.520 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.517+0000 7f5f90dd8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f8c101c30 0x7f5f8c109a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:24.520 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.517+0000 7f5f90dd8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f8c1025f0 0x7f5f8c109f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:24.520 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.517+0000 7f5f90dd8640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5f8c10d810 con 0x7f5f8c1025f0 2024-09-12T02:09:24.520 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.517+0000 7f5f90dd8640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5f8c10d690 con 0x7f5f8c101c30 2024-09-12T02:09:24.520 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.517+0000 7f5f90dd8640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5f8c10d990 con 0x7f5f8c100a40 2024-09-12T02:09:24.520 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.517+0000 7f5f8ad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f8c1025f0 0x7f5f8c109f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:24.520 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.518+0000 7f5f8a59c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5f8c100a40 0x7f5f8c10b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:24.520 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.518+0000 7f5f8ad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f8c1025f0 0x7f5f8c109f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48162/0 (socket says 172.21.15.16:48162) 2024-09-12T02:09:24.521 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.518+0000 7f5f8ad9d640 1 -- 172.21.15.16:0/703723466 learned_addr learned my addr 172.21.15.16:0/703723466 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:09:24.521 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.518+0000 7f5f89d9b640 1 --2- 172.21.15.16:0/703723466 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f8c101c30 0x7f5f8c109a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:24.521 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.518+0000 7f5f8ad9d640 1 -- 172.21.15.16:0/703723466 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5f8c100a40 msgr2=0x7f5f8c10b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:24.521 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.518+0000 7f5f8ad9d640 1 --2- 172.21.15.16:0/703723466 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5f8c100a40 0x7f5f8c10b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:24.521 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.518+0000 7f5f8ad9d640 1 -- 172.21.15.16:0/703723466 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f8c101c30 msgr2=0x7f5f8c109a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:24.521 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.518+0000 7f5f8ad9d640 1 --2- 172.21.15.16:0/703723466 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f8c101c30 0x7f5f8c109a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:24.521 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.518+0000 7f5f8ad9d640 1 -- 172.21.15.16:0/703723466 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f8c10a7d0 con 0x7f5f8c1025f0 2024-09-12T02:09:24.521 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.518+0000 7f5f8ad9d640 1 --2- 172.21.15.16:0/703723466 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f8c1025f0 0x7f5f8c109f40 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f5f8400cad0 tx=0x7f5f8400ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:24.521 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.518+0000 7f5f7b7fe640 1 -- 172.21.15.16:0/703723466 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f8400ad60 con 0x7f5f8c1025f0 2024-09-12T02:09:24.521 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.518+0000 7f5f7b7fe640 1 -- 172.21.15.16:0/703723466 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f84005020 con 0x7f5f8c1025f0 2024-09-12T02:09:24.521 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.518+0000 7f5f90dd8640 1 -- 172.21.15.16:0/703723466 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f8c1a1ed0 con 0x7f5f8c1025f0 2024-09-12T02:09:24.521 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.518+0000 7f5f7b7fe640 1 -- 172.21.15.16:0/703723466 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f8400f6b0 con 0x7f5f8c1025f0 2024-09-12T02:09:24.521 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.518+0000 7f5f90dd8640 1 -- 172.21.15.16:0/703723466 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f8c1a2330 con 0x7f5f8c1025f0 2024-09-12T02:09:24.528 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.521+0000 7f5f7b7fe640 1 -- 172.21.15.16:0/703723466 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5f84005560 con 0x7f5f8c1025f0 2024-09-12T02:09:24.529 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.521+0000 7f5f93c4c740 1 Processor -- start 2024-09-12T02:09:24.529 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.521+0000 7f5f93c4c740 1 -- start start 2024-09-12T02:09:24.529 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.521+0000 7f5f93c4c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613db7209a0 0x5613db720da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:24.529 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.521+0000 7f5f93c4c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5613db7212e0 0x5613db725da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:24.529 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.521+0000 7f5f93c4c740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5613db726370 0x5613db728830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:24.529 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.522+0000 7f5f93c4c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5613db6842f0 con 0x5613db7212e0 2024-09-12T02:09:24.529 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.522+0000 7f5f93c4c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5613db62c270 con 0x5613db7209a0 2024-09-12T02:09:24.529 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.522+0000 7f5f93c4c740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5613db6802b0 con 0x5613db726370 2024-09-12T02:09:24.529 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.522+0000 7f5f8a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5613db7212e0 0x5613db725da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:24.529 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.522+0000 7f5f7b7fe640 1 --2- 172.21.15.16:0/703723466 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f5f740a2cb0 0x7f5f740a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:24.529 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.522+0000 7f5f8a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5613db7212e0 0x5613db725da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48172/0 (socket says 172.21.15.16:48172) 2024-09-12T02:09:24.530 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.522+0000 7f5f8a59c640 1 -- 172.21.15.16:0/2770556477 learned_addr learned my addr 172.21.15.16:0/2770556477 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:09:24.530 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.522+0000 7f5f89d9b640 1 --2- 172.21.15.16:0/2770556477 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613db7209a0 0x5613db720da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:24.530 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.522+0000 7f5f8ad9d640 1 --2- 172.21.15.16:0/703723466 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f5f740a2cb0 0x7f5f740a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:24.530 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.522+0000 7f5f89d9b640 1 --2- 172.21.15.16:0/2770556477 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5613db726370 0x5613db728830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:24.530 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.522+0000 7f5f7b7fe640 1 -- 172.21.15.16:0/703723466 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f5f840cef90 con 0x7f5f8c1025f0 2024-09-12T02:09:24.530 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.522+0000 7f5f8a59c640 1 -- 172.21.15.16:0/2770556477 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5613db726370 msgr2=0x5613db728830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:09:24.530 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.522+0000 7f5f8a59c640 1 --2- 172.21.15.16:0/2770556477 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5613db726370 0x5613db728830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:24.530 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.522+0000 7f5f8a59c640 1 -- 172.21.15.16:0/2770556477 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613db7209a0 msgr2=0x5613db720da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:09:24.530 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.522+0000 7f5f8a59c640 1 --2- 172.21.15.16:0/2770556477 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613db7209a0 0x5613db720da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:24.530 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.522+0000 7f5f8a59c640 1 -- 172.21.15.16:0/2770556477 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5613db62b340 con 0x5613db7212e0 2024-09-12T02:09:24.530 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.522+0000 7f5f89d9b640 1 --2- 172.21.15.16:0/2770556477 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613db7209a0 0x5613db720da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:09:24.530 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.522+0000 7f5f8a59c640 1 --2- 172.21.15.16:0/2770556477 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5613db7212e0 0x5613db725da0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c0314e0 tx=0x7f5f7c005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c6838372ae61c57 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:24.530 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.522+0000 7f5f8ad9d640 1 --2- 172.21.15.16:0/703723466 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f5f740a2cb0 0x7f5f740a5170 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f5f8400a620 tx=0x7f5f8401baf0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:24.531 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.522+0000 7f5f78ff9640 1 -- 172.21.15.16:0/2770556477 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f7c040070 con 0x5613db7212e0 2024-09-12T02:09:24.531 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.522+0000 7f5f78ff9640 1 -- 172.21.15.16:0/2770556477 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f7c005500 con 0x5613db7212e0 2024-09-12T02:09:24.531 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.522+0000 7f5f78ff9640 1 -- 172.21.15.16:0/2770556477 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f7c035040 con 0x5613db7212e0 2024-09-12T02:09:24.531 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.523+0000 7f5f93c4c740 1 -- 172.21.15.16:0/2770556477 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5613db7212e0 msgr2=0x5613db725da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:24.531 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.523+0000 7f5f93c4c740 1 --2- 172.21.15.16:0/2770556477 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5613db7212e0 0x5613db725da0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c0314e0 tx=0x7f5f7c005c00 comp rx=0 tx=0).stop 2024-09-12T02:09:24.531 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.523+0000 7f5f93c4c740 1 -- 172.21.15.16:0/2770556477 shutdown_connections 2024-09-12T02:09:24.531 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.523+0000 7f5f93c4c740 1 --2- 172.21.15.16:0/2770556477 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5613db726370 0x5613db728830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:24.531 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.523+0000 7f5f93c4c740 1 --2- 172.21.15.16:0/2770556477 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5613db7212e0 0x5613db725da0 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:24.531 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.523+0000 7f5f93c4c740 1 --2- 172.21.15.16:0/2770556477 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613db7209a0 0x5613db720da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:24.531 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.523+0000 7f5f93c4c740 1 -- 172.21.15.16:0/2770556477 >> 172.21.15.16:0/2770556477 conn(0x5613db71c010 msgr2=0x5613db71c410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:24.531 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.523+0000 7f5f93c4c740 1 -- 172.21.15.16:0/2770556477 shutdown_connections 2024-09-12T02:09:24.531 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.523+0000 7f5f93c4c740 1 -- 172.21.15.16:0/2770556477 wait complete. 2024-09-12T02:09:24.531 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.524+0000 7f5f93c4c740 1 -- start start 2024-09-12T02:09:24.532 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.524+0000 7f5f93c4c740 1 Processor -- start 2024-09-12T02:09:24.532 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.524+0000 7f5f93c4c740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5613db7209a0 0x5613db72ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:24.532 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.524+0000 7f5f93c4c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5613db7212e0 0x5613db72b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:24.532 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.524+0000 7f5f93c4c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613db726370 0x5613db732080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:24.532 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.524+0000 7f5f93c4c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5613db7292e0 con 0x5613db7212e0 2024-09-12T02:09:24.532 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.524+0000 7f5f93c4c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5613db60d290 con 0x5613db726370 2024-09-12T02:09:24.532 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.524+0000 7f5f93c4c740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5613db729460 con 0x5613db7209a0 2024-09-12T02:09:24.532 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.524+0000 7f5f93c4c740 10 client.0 _ll_register_callbacks cb 0x5613db63e140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:09:24.532 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.524+0000 7f5f8a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5613db7212e0 0x5613db72b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:24.532 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.524+0000 7f5f8a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5613db7212e0 0x5613db72b040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48186/0 (socket says 172.21.15.16:48186) 2024-09-12T02:09:24.532 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.524+0000 7f5f8a59c640 1 -- 172.21.15.16:0/2424597804 learned_addr learned my addr 172.21.15.16:0/2424597804 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:09:24.532 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.524+0000 7f5f89d9b640 1 --2- 172.21.15.16:0/2424597804 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5613db7209a0 0x5613db72ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:24.532 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.524+0000 7f5f8a59c640 1 -- 172.21.15.16:0/2424597804 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5613db7209a0 msgr2=0x5613db72ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:09:24.533 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.524+0000 7f5f8a59c640 1 --2- 172.21.15.16:0/2424597804 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5613db7209a0 0x5613db72ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:24.533 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.524+0000 7f5f8a59c640 1 -- 172.21.15.16:0/2424597804 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613db726370 msgr2=0x5613db732080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:09:24.533 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.524+0000 7f5f8a59c640 1 --2- 172.21.15.16:0/2424597804 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613db726370 0x5613db732080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:24.533 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.524+0000 7f5f8a59c640 1 -- 172.21.15.16:0/2424597804 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5613db397210 con 0x5613db7212e0 2024-09-12T02:09:24.533 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.524+0000 7f5f89d9b640 1 --2- 172.21.15.16:0/2424597804 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5613db7209a0 0x5613db72ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:09:24.533 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.525+0000 7f5f8a59c640 1 --2- 172.21.15.16:0/2424597804 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5613db7212e0 0x5613db72b040 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c031440 tx=0x7f5f7c0052d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64627e636b26e0d4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:24.533 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.525+0000 7f5f71ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:09:24.533 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.525+0000 7f5f71ffb640 1 -- 172.21.15.16:0/2424597804 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f7c04c020 con 0x5613db7212e0 2024-09-12T02:09:24.533 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.525+0000 7f5f71ffb640 1 -- 172.21.15.16:0/2424597804 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f5f7c004050 con 0x5613db7212e0 2024-09-12T02:09:24.533 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.525+0000 7f5f93c4c740 1 -- 172.21.15.16:0/2424597804 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f7c005500 con 0x5613db7212e0 2024-09-12T02:09:24.533 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.525+0000 7f5f71ffb640 1 -- 172.21.15.16:0/2424597804 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5f7c040070 con 0x5613db7212e0 2024-09-12T02:09:24.534 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.525+0000 7f5f93c4c740 10 client.17685 fetch_fsmap learned FSMap version 25 2024-09-12T02:09:24.534 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.525+0000 7f5f93c4c740 1 -- 172.21.15.16:0/2424597804 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5613db463620 con 0x5613db7212e0 2024-09-12T02:09:24.534 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.525+0000 7f5f71ffb640 1 -- 172.21.15.16:0/2424597804 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f7c005500 con 0x5613db7212e0 2024-09-12T02:09:24.534 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.525+0000 7f5f71ffb640 1 -- 172.21.15.16:0/2424597804 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f5f7c003c40 con 0x5613db7212e0 2024-09-12T02:09:24.534 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.525+0000 7f5f93c4c740 10 client.17685 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:09:24.534 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.525+0000 7f5f93c4c740 10 client.17685 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:09:24.534 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.525+0000 7f5f93c4c740 20 client.17685 populate_metadata read hostname 'smithi016' 2024-09-12T02:09:24.534 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.525+0000 7f5f93c4c740 1 --2- 172.21.15.16:0/2424597804 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5613db736380 0x5613db738770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:24.534 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.525+0000 7f5f93c4c740 4 client.17685 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:09:24.534 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.525+0000 7f5f93c4c740 1 -- 172.21.15.16:0/2424597804 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5613db691a20 con 0x5613db736380 2024-09-12T02:09:24.534 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.526+0000 7f5f89d9b640 1 --2- 172.21.15.16:0/2424597804 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5613db736380 0x5613db738770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:24.534 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.526+0000 7f5f89d9b640 1 --2- 172.21.15.16:0/2424597804 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5613db736380 0x5613db738770 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e537204bf779f739 server_cookie=d2babf6b79f2c038 in_seq=0 out_seq=0 2024-09-12T02:09:24.534 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.526+0000 7f5f71ffb640 10 client.17685 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:09:24.535 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.527+0000 7f5f71ffb640 1 -- 172.21.15.16:0/2424597804 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5613db691a20 con 0x5613db736380 2024-09-12T02:09:24.535 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.527+0000 7f5f71ffb640 10 client.17685 handle_command_reply: tid=0 2024-09-12T02:09:24.535 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.527+0000 7f5f93c4c740 1 client.17685 shutdown 2024-09-12T02:09:24.535 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.527+0000 7f5f93c4c740 1 -- 172.21.15.16:0/2424597804 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5613db7212e0 msgr2=0x5613db72b040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:24.535 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.527+0000 7f5f93c4c740 1 --2- 172.21.15.16:0/2424597804 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5613db7212e0 0x5613db72b040 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c031440 tx=0x7f5f7c0052d0 comp rx=0 tx=0).stop 2024-09-12T02:09:24.535 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.527+0000 7f5f93c4c740 1 -- 172.21.15.16:0/2424597804 shutdown_connections 2024-09-12T02:09:24.535 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.527+0000 7f5f93c4c740 1 --2- 172.21.15.16:0/2424597804 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5613db736380 0x5613db738770 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:24.535 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.527+0000 7f5f93c4c740 1 --2- 172.21.15.16:0/2424597804 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613db726370 0x5613db732080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:24.535 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.527+0000 7f5f93c4c740 1 --2- 172.21.15.16:0/2424597804 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5613db7212e0 0x5613db72b040 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:24.535 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.527+0000 7f5f71ffb640 0 client.17685 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:09:24.535 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.527+0000 7f5f93c4c740 1 --2- 172.21.15.16:0/2424597804 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5613db7209a0 0x5613db72ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:24.535 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.527+0000 7f5f93c4c740 1 -- 172.21.15.16:0/2424597804 >> 172.21.15.16:0/2424597804 conn(0x5613db71c010 msgr2=0x5613db71d820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:24.536 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.527+0000 7f5f93c4c740 1 -- 172.21.15.16:0/2424597804 shutdown_connections 2024-09-12T02:09:24.536 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.527+0000 7f5f93c4c740 1 -- 172.21.15.16:0/2424597804 wait complete. 2024-09-12T02:09:24.536 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.527+0000 7f5f93c4c740 20 client.17685 trim_cache size 0 max 16384 2024-09-12T02:09:24.542 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.541+0000 7f5f93c4c740 1 Processor -- start 2024-09-12T02:09:24.543 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.541+0000 7f5f93c4c740 1 -- start start 2024-09-12T02:09:24.543 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.541+0000 7f5f93c4c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613db7209a0 0x5613db71b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:24.543 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.541+0000 7f5f93c4c740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5613db7212e0 0x5613db732040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:24.543 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.541+0000 7f5f93c4c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5613db726370 0x5613db71d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:24.543 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.541+0000 7f5f93c4c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5f8c1ad820 con 0x5613db726370 2024-09-12T02:09:24.543 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.542+0000 7f5f93c4c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5f8c1ab0f0 con 0x5613db7209a0 2024-09-12T02:09:24.543 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.542+0000 7f5f93c4c740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5f8c1a8f40 con 0x5613db7212e0 2024-09-12T02:09:24.543 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.542+0000 7f5f89d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5613db726370 0x5613db71d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:24.543 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.542+0000 7f5f89d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5613db726370 0x5613db71d400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48200/0 (socket says 172.21.15.16:48200) 2024-09-12T02:09:24.543 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.542+0000 7f5f89d9b640 1 -- 172.21.15.16:0/1791760598 learned_addr learned my addr 172.21.15.16:0/1791760598 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:09:24.543 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.542+0000 7f5f89d9b640 1 --2- 172.21.15.16:0/1791760598 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613db7209a0 0x5613db71b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:24.543 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.542+0000 7f5f89d9b640 1 -- 172.21.15.16:0/1791760598 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5613db7212e0 msgr2=0x5613db732040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:09:24.544 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.542+0000 7f5f8a59c640 1 --2- 172.21.15.16:0/1791760598 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5613db7212e0 0x5613db732040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:24.544 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.542+0000 7f5f89d9b640 1 --2- 172.21.15.16:0/1791760598 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5613db7212e0 0x5613db732040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:24.544 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.542+0000 7f5f89d9b640 1 -- 172.21.15.16:0/1791760598 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613db7209a0 msgr2=0x5613db71b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:09:24.544 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.542+0000 7f5f89d9b640 1 --2- 172.21.15.16:0/1791760598 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613db7209a0 0x5613db71b0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:24.544 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.542+0000 7f5f89d9b640 1 -- 172.21.15.16:0/1791760598 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5613db7172d0 con 0x5613db726370 2024-09-12T02:09:24.544 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.542+0000 7f5f89d9b640 1 --2- 172.21.15.16:0/1791760598 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5613db726370 0x5613db71d400 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f5f8000ae10 tx=0x7f5f80000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66d70d5dc602675 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:24.544 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.542+0000 7f5f73fff640 1 -- 172.21.15.16:0/1791760598 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f80012ba0 con 0x5613db726370 2024-09-12T02:09:24.544 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.542+0000 7f5f73fff640 1 -- 172.21.15.16:0/1791760598 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f80012d40 con 0x5613db726370 2024-09-12T02:09:24.544 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.542+0000 7f5f73fff640 1 -- 172.21.15.16:0/1791760598 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f80004d10 con 0x5613db726370 2024-09-12T02:09:24.547 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.543+0000 7f5f93c4c740 1 -- 172.21.15.16:0/1791760598 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5613db726370 msgr2=0x5613db71d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:24.547 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.543+0000 7f5f93c4c740 1 --2- 172.21.15.16:0/1791760598 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5613db726370 0x5613db71d400 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f5f8000ae10 tx=0x7f5f80000bc0 comp rx=0 tx=0).stop 2024-09-12T02:09:24.547 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.543+0000 7f5f93c4c740 1 -- 172.21.15.16:0/1791760598 shutdown_connections 2024-09-12T02:09:24.548 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.543+0000 7f5f93c4c740 1 --2- 172.21.15.16:0/1791760598 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5613db726370 0x5613db71d400 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:24.548 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.543+0000 7f5f93c4c740 1 --2- 172.21.15.16:0/1791760598 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5613db7212e0 0x5613db732040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:24.548 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.543+0000 7f5f93c4c740 1 --2- 172.21.15.16:0/1791760598 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613db7209a0 0x5613db71b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:24.548 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.543+0000 7f5f93c4c740 1 -- 172.21.15.16:0/1791760598 >> 172.21.15.16:0/1791760598 conn(0x5613db71c010 msgr2=0x5613db71a9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:24.548 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.543+0000 7f5f93c4c740 1 -- 172.21.15.16:0/1791760598 shutdown_connections 2024-09-12T02:09:24.548 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.543+0000 7f5f93c4c740 1 -- 172.21.15.16:0/1791760598 wait complete. 2024-09-12T02:09:24.548 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.543+0000 7f5f93c4c740 1 -- start start 2024-09-12T02:09:24.548 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.543+0000 7f5f93c4c740 1 Processor -- start 2024-09-12T02:09:24.548 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.544+0000 7f5f93c4c740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5613db7209a0 0x5613db72fde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:24.548 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.544+0000 7f5f93c4c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613db7212e0 0x5613db730320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:24.548 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.544+0000 7f5f93c4c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5613db726370 0x5613db731330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:24.548 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.544+0000 7f5f93c4c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5f8c1a95b0 con 0x5613db726370 2024-09-12T02:09:24.549 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.544+0000 7f5f93c4c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5f8c1aaa20 con 0x5613db7212e0 2024-09-12T02:09:24.549 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.544+0000 7f5f93c4c740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5f8c1a9c80 con 0x5613db7209a0 2024-09-12T02:09:24.549 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.544+0000 7f5f93c4c740 10 client.0 _ll_register_callbacks cb 0x7f5f8000a680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:09:24.549 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.544+0000 7f5f89d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5613db726370 0x5613db731330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:24.549 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.544+0000 7f5f89d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5613db726370 0x5613db731330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48214/0 (socket says 172.21.15.16:48214) 2024-09-12T02:09:24.549 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.544+0000 7f5f89d9b640 1 -- 172.21.15.16:0/2941367330 learned_addr learned my addr 172.21.15.16:0/2941367330 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:09:24.549 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.544+0000 7f5f8a59c640 1 --2- 172.21.15.16:0/2941367330 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613db7212e0 0x5613db730320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:24.549 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.544+0000 7f5f89d9b640 1 --2- 172.21.15.16:0/2941367330 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5613db7209a0 0x5613db72fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:24.549 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.544+0000 7f5f89d9b640 1 -- 172.21.15.16:0/2941367330 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5613db7209a0 msgr2=0x5613db72fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:09:24.549 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.544+0000 7f5f89d9b640 1 --2- 172.21.15.16:0/2941367330 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5613db7209a0 0x5613db72fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:24.549 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.544+0000 7f5f89d9b640 1 -- 172.21.15.16:0/2941367330 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613db7212e0 msgr2=0x5613db730320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:09:24.549 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.544+0000 7f5f89d9b640 1 --2- 172.21.15.16:0/2941367330 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613db7212e0 0x5613db730320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:24.549 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.544+0000 7f5f89d9b640 1 -- 172.21.15.16:0/2941367330 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5613db731940 con 0x5613db726370 2024-09-12T02:09:24.550 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.544+0000 7f5f89d9b640 1 --2- 172.21.15.16:0/2941367330 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5613db726370 0x5613db731330 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x5613db72f3b0 tx=0x7f5f8000abb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5158826d4e551631 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:24.550 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.544+0000 7f5f71ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:09:24.550 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.544+0000 7f5f71ffb640 1 -- 172.21.15.16:0/2941367330 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f80004a90 con 0x5613db726370 2024-09-12T02:09:24.550 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.544+0000 7f5f71ffb640 1 -- 172.21.15.16:0/2941367330 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f5f80004c30 con 0x5613db726370 2024-09-12T02:09:24.550 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.544+0000 7f5f93c4c740 1 -- 172.21.15.16:0/2941367330 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f80012d40 con 0x5613db726370 2024-09-12T02:09:24.550 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.545+0000 7f5f71ffb640 1 -- 172.21.15.16:0/2941367330 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5f80005000 con 0x5613db726370 2024-09-12T02:09:24.550 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.545+0000 7f5f93c4c740 10 client.17691 fetch_fsmap learned FSMap version 25 2024-09-12T02:09:24.550 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.545+0000 7f5f93c4c740 1 -- 172.21.15.16:0/2941367330 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5613db7a1630 con 0x5613db726370 2024-09-12T02:09:24.550 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.545+0000 7f5f71ffb640 1 -- 172.21.15.16:0/2941367330 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f80012d40 con 0x5613db726370 2024-09-12T02:09:24.550 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.545+0000 7f5f71ffb640 1 -- 172.21.15.16:0/2941367330 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f5f8000f920 con 0x5613db726370 2024-09-12T02:09:24.550 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.545+0000 7f5f93c4c740 10 client.17691 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:09:24.551 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.545+0000 7f5f93c4c740 10 client.17691 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:09:24.551 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.545+0000 7f5f93c4c740 20 client.17691 populate_metadata read hostname 'smithi016' 2024-09-12T02:09:24.551 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.545+0000 7f5f93c4c740 1 --2- 172.21.15.16:0/2941367330 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5613db7a1ed0 0x5613db7a42f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:24.551 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.545+0000 7f5f93c4c740 4 client.17691 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:09:24.551 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.545+0000 7f5f93c4c740 1 -- 172.21.15.16:0/2941367330 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5613db68bd70 con 0x5613db7a1ed0 2024-09-12T02:09:24.551 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.545+0000 7f5f8a59c640 1 --2- 172.21.15.16:0/2941367330 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5613db7a1ed0 0x5613db7a42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:24.551 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.546+0000 7f5f71ffb640 10 client.17691 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:09:24.551 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.546+0000 7f5f8a59c640 1 --2- 172.21.15.16:0/2941367330 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5613db7a1ed0 0x5613db7a42f0 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe1c47266c991b6b server_cookie=8191cf7b91547ca6 in_seq=0 out_seq=0 2024-09-12T02:09:24.560 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.560+0000 7f5f71ffb640 1 -- 172.21.15.16:0/2941367330 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5613db68bd70 con 0x5613db7a1ed0 2024-09-12T02:09:24.561 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.560+0000 7f5f71ffb640 10 client.17691 handle_command_reply: tid=0 2024-09-12T02:09:24.561 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.560+0000 7f5f93c4c740 1 client.17691 shutdown 2024-09-12T02:09:24.561 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.560+0000 7f5f93c4c740 1 -- 172.21.15.16:0/2941367330 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5613db726370 msgr2=0x5613db731330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:24.561 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.560+0000 7f5f93c4c740 1 --2- 172.21.15.16:0/2941367330 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5613db726370 0x5613db731330 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x5613db72f3b0 tx=0x7f5f8000abb0 comp rx=0 tx=0).stop 2024-09-12T02:09:24.561 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.560+0000 7f5f93c4c740 1 -- 172.21.15.16:0/2941367330 shutdown_connections 2024-09-12T02:09:24.561 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.560+0000 7f5f93c4c740 1 --2- 172.21.15.16:0/2941367330 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5613db7a1ed0 0x5613db7a42f0 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:24.561 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.560+0000 7f5f93c4c740 1 --2- 172.21.15.16:0/2941367330 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5613db726370 0x5613db731330 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:24.561 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.560+0000 7f5f93c4c740 1 --2- 172.21.15.16:0/2941367330 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613db7212e0 0x5613db730320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:24.562 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.560+0000 7f5f93c4c740 1 --2- 172.21.15.16:0/2941367330 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5613db7209a0 0x5613db72fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:24.562 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.560+0000 7f5f71ffb640 0 client.17691 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:09:24.562 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.560+0000 7f5f93c4c740 1 -- 172.21.15.16:0/2941367330 >> 172.21.15.16:0/2941367330 conn(0x5613db71c010 msgr2=0x5613db71b1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:24.562 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.560+0000 7f5f93c4c740 1 -- 172.21.15.16:0/2941367330 shutdown_connections 2024-09-12T02:09:24.562 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.560+0000 7f5f93c4c740 1 -- 172.21.15.16:0/2941367330 wait complete. 2024-09-12T02:09:24.562 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.560+0000 7f5f93c4c740 20 client.17691 trim_cache size 0 max 16384 2024-09-12T02:09:24.562 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-09-12T02:09:24.562 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-09-12T02:09:24.562 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-09-12T02:09:24.563 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-09-12T02:09:24.564 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.561+0000 7f5f90dd8640 1 -- 172.21.15.16:0/703723466 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f5f740a2cb0 msgr2=0x7f5f740a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:24.564 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.561+0000 7f5f90dd8640 1 --2- 172.21.15.16:0/703723466 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f5f740a2cb0 0x7f5f740a5170 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f5f8400a620 tx=0x7f5f8401baf0 comp rx=0 tx=0).stop 2024-09-12T02:09:24.564 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.561+0000 7f5f90dd8640 1 -- 172.21.15.16:0/703723466 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f8c1025f0 msgr2=0x7f5f8c109f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:24.564 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.561+0000 7f5f90dd8640 1 --2- 172.21.15.16:0/703723466 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f8c1025f0 0x7f5f8c109f40 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f5f8400cad0 tx=0x7f5f8400ced0 comp rx=0 tx=0).stop 2024-09-12T02:09:24.564 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.561+0000 7f5f90dd8640 1 -- 172.21.15.16:0/703723466 shutdown_connections 2024-09-12T02:09:24.565 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.561+0000 7f5f90dd8640 1 --2- 172.21.15.16:0/703723466 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f5f740a2cb0 0x7f5f740a5170 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:24.565 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.561+0000 7f5f90dd8640 1 --2- 172.21.15.16:0/703723466 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f8c1025f0 0x7f5f8c109f40 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:24.565 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.561+0000 7f5f90dd8640 1 --2- 172.21.15.16:0/703723466 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f8c101c30 0x7f5f8c109a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:24.565 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.561+0000 7f5f90dd8640 1 --2- 172.21.15.16:0/703723466 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5f8c100a40 0x7f5f8c10b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:24.565 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.561+0000 7f5f90dd8640 1 -- 172.21.15.16:0/703723466 >> 172.21.15.16:0/703723466 conn(0x7f5f8c0fc1e0 msgr2=0x7f5f8c104f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:24.565 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.561+0000 7f5f90dd8640 1 -- 172.21.15.16:0/703723466 shutdown_connections 2024-09-12T02:09:24.565 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:24.561+0000 7f5f90dd8640 1 -- 172.21.15.16:0/703723466 wait complete. 2024-09-12T02:09:24.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:24.648+0000 7f7124d15640 1 -- 172.21.15.16:0/4196120311 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7120108290 msgr2=0x7f712010ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:24.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:24.648+0000 7f7124d15640 1 --2- 172.21.15.16:0/4196120311 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7120108290 0x7f712010ec30 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f7118009cd0 tx=0x7f711802a8e0 comp rx=0 tx=0).stop 2024-09-12T02:09:24.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:24.648+0000 7f7124d15640 1 -- 172.21.15.16:0/4196120311 shutdown_connections 2024-09-12T02:09:24.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:24.648+0000 7f7124d15640 1 --2- 172.21.15.16:0/4196120311 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7120108290 0x7f712010ec30 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:24.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:24.648+0000 7f7124d15640 1 --2- 172.21.15.16:0/4196120311 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7120103740 0x7f7120107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:24.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:24.648+0000 7f7124d15640 1 --2- 172.21.15.16:0/4196120311 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7120102d90 0x7f7120103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:24.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:24.648+0000 7f7124d15640 1 -- 172.21.15.16:0/4196120311 >> 172.21.15.16:0/4196120311 conn(0x7f71200fc7d0 msgr2=0x7f71200febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:24.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:24.648+0000 7f7124d15640 1 -- 172.21.15.16:0/4196120311 shutdown_connections 2024-09-12T02:09:24.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:24.648+0000 7f7124d15640 1 -- 172.21.15.16:0/4196120311 wait complete. 2024-09-12T02:09:24.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:24.649+0000 7f7124d15640 1 Processor -- start 2024-09-12T02:09:24.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:24.649+0000 7f7124d15640 1 -- start start 2024-09-12T02:09:24.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:24.649+0000 7f7124d15640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7120102d90 0x7f71201009f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:24.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:24.649+0000 7f7124d15640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7120103740 0x7f7120100f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:24.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:24.649+0000 7f7124d15640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7120108290 0x7f7120101470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:24.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:24.649+0000 7f7124d15640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7120112090 con 0x7f7120108290 2024-09-12T02:09:24.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:24.649+0000 7f7124d15640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7120111f10 con 0x7f7120103740 2024-09-12T02:09:24.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:24.649+0000 7f7124d15640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7120112210 con 0x7f7120102d90 2024-09-12T02:09:24.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:24.649+0000 7f711ed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7120108290 0x7f7120101470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:24.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:24.649+0000 7f711ed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7120108290 0x7f7120101470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48232/0 (socket says 172.21.15.16:48232) 2024-09-12T02:09:24.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:24.649+0000 7f711ed9d640 1 -- 172.21.15.16:0/586350597 learned_addr learned my addr 172.21.15.16:0/586350597 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:09:24.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:24.650+0000 7f711e59c640 1 --2- 172.21.15.16:0/586350597 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7120102d90 0x7f71201009f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:24.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:24.650+0000 7f711dd9b640 1 --2- 172.21.15.16:0/586350597 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7120103740 0x7f7120100f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:24.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:24.650+0000 7f711ed9d640 1 -- 172.21.15.16:0/586350597 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7120102d90 msgr2=0x7f71201009f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:24.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:24.650+0000 7f711ed9d640 1 --2- 172.21.15.16:0/586350597 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7120102d90 0x7f71201009f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:24.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:24.650+0000 7f711ed9d640 1 -- 172.21.15.16:0/586350597 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7120103740 msgr2=0x7f7120100f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:24.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:24.650+0000 7f711ed9d640 1 --2- 172.21.15.16:0/586350597 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7120103740 0x7f7120100f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:24.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:24.650+0000 7f711ed9d640 1 -- 172.21.15.16:0/586350597 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71201a6a10 con 0x7f7120108290 2024-09-12T02:09:24.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:24.650+0000 7f711e59c640 1 --2- 172.21.15.16:0/586350597 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7120102d90 0x7f71201009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:09:24.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:24.650+0000 7f711ed9d640 1 --2- 172.21.15.16:0/586350597 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7120108290 0x7f7120101470 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f7118005cc0 tx=0x7f7118003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:24.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:24.650+0000 7f710f7fe640 1 -- 172.21.15.16:0/586350597 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7118004040 con 0x7f7120108290 2024-09-12T02:09:24.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:24.650+0000 7f710f7fe640 1 -- 172.21.15.16:0/586350597 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71180046c0 con 0x7f7120108290 2024-09-12T02:09:24.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:24.650+0000 7f7124d15640 1 -- 172.21.15.16:0/586350597 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71201a6ca0 con 0x7f7120108290 2024-09-12T02:09:24.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:24.650+0000 7f710f7fe640 1 -- 172.21.15.16:0/586350597 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7118046480 con 0x7f7120108290 2024-09-12T02:09:24.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:24.650+0000 7f7124d15640 1 -- 172.21.15.16:0/586350597 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71201a7230 con 0x7f7120108290 2024-09-12T02:09:24.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:24.651+0000 7f7124d15640 1 -- 172.21.15.16:0/586350597 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7120103170 con 0x7f7120108290 2024-09-12T02:09:24.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:24.652+0000 7f710f7fe640 1 -- 172.21.15.16:0/586350597 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f71180305a0 con 0x7f7120108290 2024-09-12T02:09:24.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:24.653+0000 7f710f7fe640 1 --2- 172.21.15.16:0/586350597 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f71080a2c00 0x7f71080a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:24.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:24.653+0000 7f711e59c640 1 --2- 172.21.15.16:0/586350597 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f71080a2c00 0x7f71080a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:24.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:24.653+0000 7f710f7fe640 1 -- 172.21.15.16:0/586350597 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f711802e030 con 0x7f7120108290 2024-09-12T02:09:24.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:24.653+0000 7f711e59c640 1 --2- 172.21.15.16:0/586350597 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f71080a2c00 0x7f71080a50c0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f71100058c0 tx=0x7f7110005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:24.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:24.656+0000 7f710f7fe640 1 -- 172.21.15.16:0/586350597 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f711804b060 con 0x7f7120108290 2024-09-12T02:09:24.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:24.870+0000 7f7124d15640 1 -- 172.21.15.16:0/586350597 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7120101f20 con 0x7f7120108290 2024-09-12T02:09:24.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:24.870+0000 7f710f7fe640 1 -- 172.21.15.16:0/586350597 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11911 (secure 0 0 0) 0x7f7120101f20 con 0x7f7120108290 2024-09-12T02:09:24.871 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T02:09:24.871 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":25,"btime":"2024-09-12T02:01:37:192021+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3279123279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3279123279},{"type":"v1","addr":"172.21.15.16:6835","nonce":3279123279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4119452382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4119452382},{"type":"v1","addr":"172.21.15.16:6837","nonce":4119452382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/3263823165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3263823165},{"type":"v1","addr":"172.21.15.16:6839","nonce":3263823165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/1510290732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":1510290732},{"type":"v1","addr":"172.21.15.16:6841","nonce":1510290732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/279674603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":279674603},{"type":"v1","addr":"172.21.15.33:6833","nonce":279674603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3555728028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3555728028},{"type":"v1","addr":"172.21.15.33:6835","nonce":3555728028}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T01:59:56.117608+0000","modified":"2024-09-12T02:01:36.294760+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24482,"mds_1":24481,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6833/1563374106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1563374106},{"type":"v1","addr":"172.21.15.184:6833","nonce":1563374106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6837/1892833868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1892833868},{"type":"v1","addr":"172.21.15.33:6837","nonce":1892833868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/1457679686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1457679686},{"type":"v1","addr":"172.21.15.184:6835","nonce":1457679686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.184:6837/1793393087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":1793393087},{"type":"v1","addr":"172.21.15.184:6837","nonce":1793393087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.184:6839/4203347941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":4203347941},{"type":"v1","addr":"172.21.15.184:6839","nonce":4203347941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.33:6839/4136670204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":4136670204},{"type":"v1","addr":"172.21.15.33:6839","nonce":4136670204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24481,24482,24469]},"id":1}]} 2024-09-12T02:09:24.871 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 25 2024-09-12T02:09:24.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:24.873+0000 7f7124d15640 1 -- 172.21.15.16:0/586350597 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f71080a2c00 msgr2=0x7f71080a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:24.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:24.873+0000 7f7124d15640 1 --2- 172.21.15.16:0/586350597 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f71080a2c00 0x7f71080a50c0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f71100058c0 tx=0x7f7110005850 comp rx=0 tx=0).stop 2024-09-12T02:09:24.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:24.873+0000 7f7124d15640 1 -- 172.21.15.16:0/586350597 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7120108290 msgr2=0x7f7120101470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:24.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:24.873+0000 7f7124d15640 1 --2- 172.21.15.16:0/586350597 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7120108290 0x7f7120101470 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f7118005cc0 tx=0x7f7118003970 comp rx=0 tx=0).stop 2024-09-12T02:09:24.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:24.873+0000 7f7124d15640 1 -- 172.21.15.16:0/586350597 shutdown_connections 2024-09-12T02:09:24.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:24.873+0000 7f7124d15640 1 --2- 172.21.15.16:0/586350597 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f71080a2c00 0x7f71080a50c0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:24.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:24.873+0000 7f7124d15640 1 --2- 172.21.15.16:0/586350597 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7120108290 0x7f7120101470 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:24.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:24.873+0000 7f7124d15640 1 --2- 172.21.15.16:0/586350597 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7120103740 0x7f7120100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:24.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:24.873+0000 7f7124d15640 1 --2- 172.21.15.16:0/586350597 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7120102d90 0x7f71201009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:24.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:24.874+0000 7f7124d15640 1 -- 172.21.15.16:0/586350597 >> 172.21.15.16:0/586350597 conn(0x7f71200fc7d0 msgr2=0x7f7120105d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:24.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:24.874+0000 7f7124d15640 1 -- 172.21.15.16:0/586350597 shutdown_connections 2024-09-12T02:09:24.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:24.874+0000 7f7124d15640 1 -- 172.21.15.16:0/586350597 wait complete. 2024-09-12T02:09:25.549 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24482 quiesce db / --timeout 90 --expiration 139.4 --await-for 45.0' 2024-09-12T02:09:25.549 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24482 quiesce db / --timeout 90 --expiration 139.4 --await-for 45.0 2024-09-12T02:09:25.850 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:09:25.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:09:25 smithi184 ceph-mon[35593]: pgmap v560: 97 pgs: 97 active+clean; 4.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-09-12T02:09:25.878 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:09:25 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/586350597' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:09:25.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:09:25 smithi033 ceph-mon[35332]: pgmap v560: 97 pgs: 97 active+clean; 4.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-09-12T02:09:25.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:09:25 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/586350597' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:09:25.933 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:09:25 smithi016 ceph-mon[30947]: pgmap v560: 97 pgs: 97 active+clean; 4.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-09-12T02:09:25.933 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:09:25 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/586350597' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:09:26.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.333+0000 7f723e454640 1 -- 172.21.15.16:0/1278402766 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7238102d30 msgr2=0x7f7238103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:26.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.333+0000 7f723e454640 1 --2- 172.21.15.16:0/1278402766 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7238102d30 0x7f7238103110 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f7228009930 tx=0x7f722802a420 comp rx=0 tx=0).stop 2024-09-12T02:09:26.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.333+0000 7f723e454640 1 -- 172.21.15.16:0/1278402766 shutdown_connections 2024-09-12T02:09:26.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.333+0000 7f723e454640 1 --2- 172.21.15.16:0/1278402766 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f72381082a0 0x7f723810ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:26.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.333+0000 7f723e454640 1 --2- 172.21.15.16:0/1278402766 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f72381036e0 0x7f7238107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:26.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.333+0000 7f723e454640 1 --2- 172.21.15.16:0/1278402766 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7238102d30 0x7f7238103110 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:26.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.333+0000 7f723e454640 1 -- 172.21.15.16:0/1278402766 >> 172.21.15.16:0/1278402766 conn(0x7f72380fc7d0 msgr2=0x7f72380febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:26.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.333+0000 7f723e454640 1 -- 172.21.15.16:0/1278402766 shutdown_connections 2024-09-12T02:09:26.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.333+0000 7f723e454640 1 -- 172.21.15.16:0/1278402766 wait complete. 2024-09-12T02:09:26.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.333+0000 7f723e454640 1 Processor -- start 2024-09-12T02:09:26.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.334+0000 7f723e454640 1 -- start start 2024-09-12T02:09:26.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.334+0000 7f723e454640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7238102d30 0x7f72381985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:26.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.334+0000 7f723e454640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f72381036e0 0x7f7238198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:26.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.334+0000 7f723e454640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f72381082a0 0x7f723819ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:26.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.334+0000 7f723e454640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f72381120a0 con 0x7f7238102d30 2024-09-12T02:09:26.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.334+0000 7f723e454640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7238111f20 con 0x7f72381036e0 2024-09-12T02:09:26.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.334+0000 7f723e454640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7238112220 con 0x7f72381082a0 2024-09-12T02:09:26.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.334+0000 7f7237fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7238102d30 0x7f72381985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:26.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.334+0000 7f7237fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7238102d30 0x7f72381985c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48244/0 (socket says 172.21.15.16:48244) 2024-09-12T02:09:26.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.334+0000 7f7237fff640 1 -- 172.21.15.16:0/783444430 learned_addr learned my addr 172.21.15.16:0/783444430 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:09:26.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.334+0000 7f72377fe640 1 --2- 172.21.15.16:0/783444430 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f72381036e0 0x7f7238198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:26.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.334+0000 7f723c9f1640 1 --2- 172.21.15.16:0/783444430 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f72381082a0 0x7f723819ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:26.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.334+0000 7f7237fff640 1 -- 172.21.15.16:0/783444430 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f72381082a0 msgr2=0x7f723819ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:26.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.334+0000 7f7237fff640 1 --2- 172.21.15.16:0/783444430 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f72381082a0 0x7f723819ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:26.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.334+0000 7f7237fff640 1 -- 172.21.15.16:0/783444430 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f72381036e0 msgr2=0x7f7238198b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:26.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.334+0000 7f7237fff640 1 --2- 172.21.15.16:0/783444430 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f72381036e0 0x7f7238198b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:26.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.334+0000 7f7237fff640 1 -- 172.21.15.16:0/783444430 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f723819d570 con 0x7f7238102d30 2024-09-12T02:09:26.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.335+0000 7f7237fff640 1 --2- 172.21.15.16:0/783444430 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7238102d30 0x7f72381985c0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f7228009230 tx=0x7f7228004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:26.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.335+0000 7f72357fa640 1 -- 172.21.15.16:0/783444430 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f722803f030 con 0x7f7238102d30 2024-09-12T02:09:26.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.335+0000 7f72357fa640 1 -- 172.21.15.16:0/783444430 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7228003ad0 con 0x7f7238102d30 2024-09-12T02:09:26.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.335+0000 7f723e454640 1 -- 172.21.15.16:0/783444430 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f723819d800 con 0x7f7238102d30 2024-09-12T02:09:26.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.335+0000 7f72357fa640 1 -- 172.21.15.16:0/783444430 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7228035070 con 0x7f7238102d30 2024-09-12T02:09:26.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.335+0000 7f723e454640 1 -- 172.21.15.16:0/783444430 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72381a5140 con 0x7f7238102d30 2024-09-12T02:09:26.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.337+0000 7f72357fa640 1 -- 172.21.15.16:0/783444430 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f7228042030 con 0x7f7238102d30 2024-09-12T02:09:26.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.337+0000 7f724082d740 1 Processor -- start 2024-09-12T02:09:26.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.338+0000 7f724082d740 1 -- start start 2024-09-12T02:09:26.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.338+0000 7f724082d740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56472cf80620 0x56472cf80a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:26.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.338+0000 7f724082d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56472cf80f40 0x56472cf8f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:26.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.338+0000 7f724082d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56472cf8fd50 0x56472cf92140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:26.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.338+0000 7f724082d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56472cee62a0 con 0x56472cf80f40 2024-09-12T02:09:26.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.338+0000 7f724082d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56472ce8db10 con 0x56472cf8fd50 2024-09-12T02:09:26.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.338+0000 7f724082d740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x56472cee2260 con 0x56472cf80620 2024-09-12T02:09:26.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.338+0000 7f7237fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56472cf80f40 0x56472cf8f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:26.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.338+0000 7f7237fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56472cf80f40 0x56472cf8f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48260/0 (socket says 172.21.15.16:48260) 2024-09-12T02:09:26.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.338+0000 7f7237fff640 1 -- 172.21.15.16:0/4256201857 learned_addr learned my addr 172.21.15.16:0/4256201857 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:09:26.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.338+0000 7f72357fa640 1 --2- 172.21.15.16:0/783444430 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f72200a2c00 0x7f72200a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:26.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.338+0000 7f72377fe640 1 --2- 172.21.15.16:0/4256201857 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56472cf8fd50 0x56472cf92140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:26.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.338+0000 7f723c9f1640 1 --2- 172.21.15.16:0/4256201857 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56472cf80620 0x56472cf80a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:26.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.338+0000 7f723c9f1640 1 --2- 172.21.15.16:0/783444430 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f72200a2c00 0x7f72200a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:26.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.339+0000 7f7237fff640 1 -- 172.21.15.16:0/4256201857 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56472cf80620 msgr2=0x56472cf80a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:09:26.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.339+0000 7f7237fff640 1 --2- 172.21.15.16:0/4256201857 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56472cf80620 0x56472cf80a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:26.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.339+0000 7f72357fa640 1 -- 172.21.15.16:0/783444430 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f72280ef430 con 0x7f7238102d30 2024-09-12T02:09:26.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.339+0000 7f7237fff640 1 -- 172.21.15.16:0/4256201857 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56472cf8fd50 msgr2=0x56472cf92140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:09:26.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.339+0000 7f7237fff640 1 --2- 172.21.15.16:0/4256201857 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56472cf8fd50 0x56472cf92140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:26.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.339+0000 7f7237fff640 1 -- 172.21.15.16:0/4256201857 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56472cc57b90 con 0x56472cf80f40 2024-09-12T02:09:26.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.339+0000 7f72377fe640 1 --2- 172.21.15.16:0/4256201857 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56472cf8fd50 0x56472cf92140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:09:26.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.339+0000 7f723c9f1640 1 --2- 172.21.15.16:0/783444430 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f72200a2c00 0x7f72200a50c0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f7230005c40 tx=0x7f7230005bd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:26.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.339+0000 7f7237fff640 1 --2- 172.21.15.16:0/4256201857 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56472cf80f40 0x56472cf8f810 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f72280bba30 tx=0x7f7228005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f83ea3fb5008e9f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:26.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.339+0000 7f721e7fc640 1 -- 172.21.15.16:0/4256201857 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7228049070 con 0x56472cf80f40 2024-09-12T02:09:26.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.339+0000 7f721e7fc640 1 -- 172.21.15.16:0/4256201857 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72280bbd30 con 0x56472cf80f40 2024-09-12T02:09:26.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.339+0000 7f721e7fc640 1 -- 172.21.15.16:0/4256201857 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f722803b460 con 0x56472cf80f40 2024-09-12T02:09:26.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.340+0000 7f724082d740 1 -- 172.21.15.16:0/4256201857 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56472cf80f40 msgr2=0x56472cf8f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:26.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.340+0000 7f724082d740 1 --2- 172.21.15.16:0/4256201857 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56472cf80f40 0x56472cf8f810 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f72280bba30 tx=0x7f7228005e50 comp rx=0 tx=0).stop 2024-09-12T02:09:26.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.340+0000 7f724082d740 1 -- 172.21.15.16:0/4256201857 shutdown_connections 2024-09-12T02:09:26.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.340+0000 7f724082d740 1 --2- 172.21.15.16:0/4256201857 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56472cf8fd50 0x56472cf92140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:26.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.340+0000 7f724082d740 1 --2- 172.21.15.16:0/4256201857 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56472cf80f40 0x56472cf8f810 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:26.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.340+0000 7f724082d740 1 --2- 172.21.15.16:0/4256201857 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56472cf80620 0x56472cf80a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:26.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.340+0000 7f724082d740 1 -- 172.21.15.16:0/4256201857 >> 172.21.15.16:0/4256201857 conn(0x56472cf7ccf0 msgr2=0x56472cf7d0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:26.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.340+0000 7f724082d740 1 -- 172.21.15.16:0/4256201857 shutdown_connections 2024-09-12T02:09:26.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.340+0000 7f724082d740 1 -- 172.21.15.16:0/4256201857 wait complete. 2024-09-12T02:09:26.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.340+0000 7f724082d740 1 -- start start 2024-09-12T02:09:26.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.341+0000 7f724082d740 1 Processor -- start 2024-09-12T02:09:26.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.341+0000 7f724082d740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56472cf80620 0x56472cf8cc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:26.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.341+0000 7f724082d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56472cf80f40 0x56472cf8d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:26.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.341+0000 7f724082d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56472cf8fd50 0x56472cf8d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:26.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.341+0000 7f724082d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56472cee62a0 con 0x56472cf8fd50 2024-09-12T02:09:26.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.341+0000 7f724082d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56472ce6e6e0 con 0x56472cf80f40 2024-09-12T02:09:26.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.341+0000 7f724082d740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x56472cf926e0 con 0x56472cf80620 2024-09-12T02:09:26.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.341+0000 7f724082d740 10 client.0 _ll_register_callbacks cb 0x56472ce92a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:09:26.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.341+0000 7f723c9f1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56472cf8fd50 0x56472cf8d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:26.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.341+0000 7f723c9f1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56472cf8fd50 0x56472cf8d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48264/0 (socket says 172.21.15.16:48264) 2024-09-12T02:09:26.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.341+0000 7f723c9f1640 1 -- 172.21.15.16:0/2472835820 learned_addr learned my addr 172.21.15.16:0/2472835820 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:09:26.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.341+0000 7f7237fff640 1 --2- 172.21.15.16:0/2472835820 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56472cf80620 0x56472cf8cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:26.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.341+0000 7f72377fe640 1 --2- 172.21.15.16:0/2472835820 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56472cf80f40 0x56472cf8d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:26.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.341+0000 7f723c9f1640 1 -- 172.21.15.16:0/2472835820 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56472cf80620 msgr2=0x56472cf8cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:09:26.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.341+0000 7f723c9f1640 1 --2- 172.21.15.16:0/2472835820 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56472cf80620 0x56472cf8cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:26.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.341+0000 7f723c9f1640 1 -- 172.21.15.16:0/2472835820 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56472cf80f40 msgr2=0x56472cf8d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:09:26.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.341+0000 7f723c9f1640 1 --2- 172.21.15.16:0/2472835820 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56472cf80f40 0x56472cf8d1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:26.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.341+0000 7f723c9f1640 1 -- 172.21.15.16:0/2472835820 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56472cf9bae0 con 0x56472cf8fd50 2024-09-12T02:09:26.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.342+0000 7f723c9f1640 1 --2- 172.21.15.16:0/2472835820 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56472cf8fd50 0x56472cf8d700 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f723000daf0 tx=0x7f7230004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f54aadd61e8e8f9a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:26.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.342+0000 7f72177fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:09:26.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.342+0000 7f72177fe640 1 -- 172.21.15.16:0/2472835820 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7230010ca0 con 0x56472cf8fd50 2024-09-12T02:09:26.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.342+0000 7f72177fe640 1 -- 172.21.15.16:0/2472835820 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f7230020440 con 0x56472cf8fd50 2024-09-12T02:09:26.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.342+0000 7f724082d740 1 -- 172.21.15.16:0/2472835820 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f72280bbd30 con 0x56472cf8fd50 2024-09-12T02:09:26.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.342+0000 7f72177fe640 1 -- 172.21.15.16:0/2472835820 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7230010e40 con 0x56472cf8fd50 2024-09-12T02:09:26.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.342+0000 7f724082d740 10 client.17709 fetch_fsmap learned FSMap version 25 2024-09-12T02:09:26.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.342+0000 7f724082d740 1 -- 172.21.15.16:0/2472835820 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56472cf9e4d0 con 0x56472cf8fd50 2024-09-12T02:09:26.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.342+0000 7f72177fe640 1 -- 172.21.15.16:0/2472835820 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72280bbd30 con 0x56472cf8fd50 2024-09-12T02:09:26.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.343+0000 7f72177fe640 1 -- 172.21.15.16:0/2472835820 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f723002a7a0 con 0x56472cf8fd50 2024-09-12T02:09:26.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.343+0000 7f724082d740 10 client.17709 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:09:26.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.343+0000 7f724082d740 10 client.17709 resolve_mds: validated gid 24482 aka daemon mds.b 2024-09-12T02:09:26.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.343+0000 7f724082d740 20 client.17709 populate_metadata read hostname 'smithi016' 2024-09-12T02:09:26.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.343+0000 7f724082d740 1 --2- 172.21.15.16:0/2472835820 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x56472cf9f8e0 0x56472cfa1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:26.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.343+0000 7f724082d740 4 client.17709 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:09:26.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.343+0000 7f724082d740 1 -- 172.21.15.16:0/2472835820 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56472cef39d0 con 0x56472cf9f8e0 2024-09-12T02:09:26.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.343+0000 7f7237fff640 1 --2- 172.21.15.16:0/2472835820 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x56472cf9f8e0 0x56472cfa1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:26.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.343+0000 7f7237fff640 1 --2- 172.21.15.16:0/2472835820 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x56472cf9f8e0 0x56472cfa1d40 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32f9f9e5a723646d server_cookie=7fb5f149851268a3 in_seq=0 out_seq=0 2024-09-12T02:09:26.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.343+0000 7f72177fe640 10 client.17709 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:09:26.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.344+0000 7f72177fe640 1 -- 172.21.15.16:0/2472835820 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56472cef39d0 con 0x56472cf9f8e0 2024-09-12T02:09:26.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.344+0000 7f72177fe640 10 client.17709 handle_command_reply: tid=0 2024-09-12T02:09:26.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.344+0000 7f724082d740 1 client.17709 shutdown 2024-09-12T02:09:26.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.344+0000 7f724082d740 1 -- 172.21.15.16:0/2472835820 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56472cf8fd50 msgr2=0x56472cf8d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:26.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.344+0000 7f724082d740 1 --2- 172.21.15.16:0/2472835820 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56472cf8fd50 0x56472cf8d700 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f723000daf0 tx=0x7f7230004a40 comp rx=0 tx=0).stop 2024-09-12T02:09:26.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.344+0000 7f724082d740 1 -- 172.21.15.16:0/2472835820 shutdown_connections 2024-09-12T02:09:26.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.344+0000 7f724082d740 1 --2- 172.21.15.16:0/2472835820 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x56472cf9f8e0 0x56472cfa1d40 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:26.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.344+0000 7f724082d740 1 --2- 172.21.15.16:0/2472835820 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56472cf8fd50 0x56472cf8d700 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:26.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.344+0000 7f72177fe640 0 client.17709 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:09:26.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.344+0000 7f724082d740 1 --2- 172.21.15.16:0/2472835820 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56472cf80f40 0x56472cf8d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:26.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.344+0000 7f724082d740 1 --2- 172.21.15.16:0/2472835820 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56472cf80620 0x56472cf8cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:26.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.344+0000 7f724082d740 1 -- 172.21.15.16:0/2472835820 >> 172.21.15.16:0/2472835820 conn(0x56472cf7ccf0 msgr2=0x56472cf82a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:26.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.345+0000 7f724082d740 1 -- 172.21.15.16:0/2472835820 shutdown_connections 2024-09-12T02:09:26.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.345+0000 7f724082d740 1 -- 172.21.15.16:0/2472835820 wait complete. 2024-09-12T02:09:26.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.345+0000 7f724082d740 20 client.17709 trim_cache size 0 max 16384 2024-09-12T02:09:26.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.361+0000 7f724082d740 1 Processor -- start 2024-09-12T02:09:26.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.361+0000 7f724082d740 1 -- start start 2024-09-12T02:09:26.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.362+0000 7f724082d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56472cf80620 0x56472cf8eb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:26.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.362+0000 7f724082d740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56472cf80f40 0x56472cf8b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:26.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.362+0000 7f724082d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56472cf8fd50 0x56472cfa7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:26.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.362+0000 7f724082d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f72381adbb0 con 0x56472cf80620 2024-09-12T02:09:26.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.362+0000 7f724082d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f72381ab480 con 0x56472cf8fd50 2024-09-12T02:09:26.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.362+0000 7f724082d740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f72381a92d0 con 0x56472cf80f40 2024-09-12T02:09:26.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.362+0000 7f7237fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56472cf80620 0x56472cf8eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:26.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.362+0000 7f7237fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56472cf80620 0x56472cf8eb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48276/0 (socket says 172.21.15.16:48276) 2024-09-12T02:09:26.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.362+0000 7f7237fff640 1 -- 172.21.15.16:0/3014525005 learned_addr learned my addr 172.21.15.16:0/3014525005 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:09:26.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.362+0000 7f72377fe640 1 --2- 172.21.15.16:0/3014525005 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56472cf80f40 0x56472cf8b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:26.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.362+0000 7f723c9f1640 1 --2- 172.21.15.16:0/3014525005 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56472cf8fd50 0x56472cfa7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:26.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.362+0000 7f7237fff640 1 -- 172.21.15.16:0/3014525005 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56472cf80f40 msgr2=0x56472cf8b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:09:26.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.362+0000 7f7237fff640 1 --2- 172.21.15.16:0/3014525005 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56472cf80f40 0x56472cf8b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:26.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.362+0000 7f7237fff640 1 -- 172.21.15.16:0/3014525005 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56472cf8fd50 msgr2=0x56472cfa7650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:09:26.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.362+0000 7f7237fff640 1 --2- 172.21.15.16:0/3014525005 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56472cf8fd50 0x56472cfa7650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:26.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.362+0000 7f7237fff640 1 -- 172.21.15.16:0/3014525005 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56472cf78260 con 0x56472cf80620 2024-09-12T02:09:26.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.362+0000 7f7237fff640 1 --2- 172.21.15.16:0/3014525005 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56472cf80620 0x56472cf8eb50 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f72280bbcc0 tx=0x7f72280045f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29e77ca691b3e9e9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:26.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.363+0000 7f721dffb640 1 -- 172.21.15.16:0/3014525005 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7228049040 con 0x56472cf80620 2024-09-12T02:09:26.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.363+0000 7f721dffb640 1 -- 172.21.15.16:0/3014525005 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f722803b460 con 0x56472cf80620 2024-09-12T02:09:26.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.363+0000 7f721dffb640 1 -- 172.21.15.16:0/3014525005 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72280bc1f0 con 0x56472cf80620 2024-09-12T02:09:26.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.363+0000 7f724082d740 1 -- 172.21.15.16:0/3014525005 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56472cf80620 msgr2=0x56472cf8eb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:26.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.363+0000 7f724082d740 1 --2- 172.21.15.16:0/3014525005 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56472cf80620 0x56472cf8eb50 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f72280bbcc0 tx=0x7f72280045f0 comp rx=0 tx=0).stop 2024-09-12T02:09:26.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.363+0000 7f724082d740 1 -- 172.21.15.16:0/3014525005 shutdown_connections 2024-09-12T02:09:26.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.363+0000 7f724082d740 1 --2- 172.21.15.16:0/3014525005 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56472cf8fd50 0x56472cfa7650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:26.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.363+0000 7f724082d740 1 --2- 172.21.15.16:0/3014525005 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56472cf80f40 0x56472cf8b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:26.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.363+0000 7f724082d740 1 --2- 172.21.15.16:0/3014525005 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56472cf80620 0x56472cf8eb50 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:26.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.363+0000 7f724082d740 1 -- 172.21.15.16:0/3014525005 >> 172.21.15.16:0/3014525005 conn(0x56472cf7ccf0 msgr2=0x56472cf7be70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:26.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.363+0000 7f724082d740 1 -- 172.21.15.16:0/3014525005 shutdown_connections 2024-09-12T02:09:26.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.363+0000 7f724082d740 1 -- 172.21.15.16:0/3014525005 wait complete. 2024-09-12T02:09:26.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.364+0000 7f724082d740 1 -- start start 2024-09-12T02:09:26.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.364+0000 7f724082d740 1 Processor -- start 2024-09-12T02:09:26.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.364+0000 7f724082d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56472cf80620 0x56472cf97fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:26.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.364+0000 7f724082d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56472cf80f40 0x56472cf94940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:26.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.364+0000 7f724082d740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56472cf8fd50 0x56472cf94e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:26.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.364+0000 7f724082d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f72381a9940 con 0x56472cf80620 2024-09-12T02:09:26.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.364+0000 7f724082d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f72381aadb0 con 0x56472cf80f40 2024-09-12T02:09:26.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.364+0000 7f724082d740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f72381aa010 con 0x56472cf8fd50 2024-09-12T02:09:26.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.364+0000 7f724082d740 10 client.0 _ll_register_callbacks cb 0x7f72300127e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:09:26.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.364+0000 7f7237fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56472cf80620 0x56472cf97fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:26.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.364+0000 7f7237fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56472cf80620 0x56472cf97fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48290/0 (socket says 172.21.15.16:48290) 2024-09-12T02:09:26.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.364+0000 7f7237fff640 1 -- 172.21.15.16:0/1635561575 learned_addr learned my addr 172.21.15.16:0/1635561575 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:09:26.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.364+0000 7f723c9f1640 1 --2- 172.21.15.16:0/1635561575 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56472cf8fd50 0x56472cf94e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:26.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.365+0000 7f72377fe640 1 --2- 172.21.15.16:0/1635561575 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56472cf80f40 0x56472cf94940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:26.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.365+0000 7f7237fff640 1 -- 172.21.15.16:0/1635561575 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56472cf8fd50 msgr2=0x56472cf94e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:09:26.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.365+0000 7f7237fff640 1 --2- 172.21.15.16:0/1635561575 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56472cf8fd50 0x56472cf94e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:26.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.365+0000 7f7237fff640 1 -- 172.21.15.16:0/1635561575 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56472cf80f40 msgr2=0x56472cf94940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:09:26.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.365+0000 7f7237fff640 1 --2- 172.21.15.16:0/1635561575 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56472cf80f40 0x56472cf94940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:26.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.365+0000 7f7237fff640 1 -- 172.21.15.16:0/1635561575 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56472cf95520 con 0x56472cf80620 2024-09-12T02:09:26.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.365+0000 7f72377fe640 1 --2- 172.21.15.16:0/1635561575 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56472cf80f40 0x56472cf94940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:09:26.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.365+0000 7f7237fff640 1 --2- 172.21.15.16:0/1635561575 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56472cf80620 0x56472cf97fd0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f72280c02a0 tx=0x7f7228005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e419e7933100e59a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:26.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.365+0000 7f72177fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:09:26.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.365+0000 7f72177fe640 1 -- 172.21.15.16:0/1635561575 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7228049050 con 0x56472cf80620 2024-09-12T02:09:26.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.365+0000 7f72177fe640 1 -- 172.21.15.16:0/1635561575 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f72280b8440 con 0x56472cf80620 2024-09-12T02:09:26.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.365+0000 7f724082d740 1 -- 172.21.15.16:0/1635561575 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f722803b460 con 0x56472cf80620 2024-09-12T02:09:26.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.365+0000 7f72177fe640 1 -- 172.21.15.16:0/1635561575 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f72280bbd30 con 0x56472cf80620 2024-09-12T02:09:26.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.365+0000 7f724082d740 10 client.17715 fetch_fsmap learned FSMap version 25 2024-09-12T02:09:26.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.365+0000 7f724082d740 1 -- 172.21.15.16:0/1635561575 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56472d00b4a0 con 0x56472cf80620 2024-09-12T02:09:26.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.365+0000 7f72177fe640 1 -- 172.21.15.16:0/1635561575 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72280f3040 con 0x56472cf80620 2024-09-12T02:09:26.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.366+0000 7f72177fe640 1 -- 172.21.15.16:0/1635561575 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f72280d2070 con 0x56472cf80620 2024-09-12T02:09:26.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.366+0000 7f724082d740 10 client.17715 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:09:26.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.366+0000 7f724082d740 10 client.17715 resolve_mds: validated gid 24482 aka daemon mds.b 2024-09-12T02:09:26.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.366+0000 7f724082d740 20 client.17715 populate_metadata read hostname 'smithi016' 2024-09-12T02:09:26.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.366+0000 7f724082d740 1 --2- 172.21.15.16:0/1635561575 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x56472cf9f8e0 0x56472d00d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:26.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.366+0000 7f724082d740 4 client.17715 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 139.4, "await_for": 45.0}] 2024-09-12T02:09:26.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.366+0000 7f724082d740 1 -- 172.21.15.16:0/1635561575 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 139.4, "await_for": 45.0}) -- 0x7f722803b460 con 0x56472cf9f8e0 2024-09-12T02:09:26.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.366+0000 7f72377fe640 1 --2- 172.21.15.16:0/1635561575 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x56472cf9f8e0 0x56472d00d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:26.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.367+0000 7f72177fe640 10 client.17715 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:09:26.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.367+0000 7f72377fe640 1 --2- 172.21.15.16:0/1635561575 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x56472cf9f8e0 0x56472d00d810 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e579209950dc2db server_cookie=d8827661584e504 in_seq=0 out_seq=0 2024-09-12T02:09:26.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.377+0000 7f72177fe640 1 -- 172.21.15.16:0/1635561575 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x7f722803b460 con 0x56472cf9f8e0 2024-09-12T02:09:26.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.377+0000 7f72177fe640 10 client.17715 handle_command_reply: tid=0 2024-09-12T02:09:26.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.377+0000 7f724082d740 1 client.17715 shutdown 2024-09-12T02:09:26.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.377+0000 7f724082d740 1 -- 172.21.15.16:0/1635561575 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56472cf80620 msgr2=0x56472cf97fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:26.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.377+0000 7f724082d740 1 --2- 172.21.15.16:0/1635561575 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56472cf80620 0x56472cf97fd0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f72280c02a0 tx=0x7f7228005c00 comp rx=0 tx=0).stop 2024-09-12T02:09:26.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.377+0000 7f724082d740 1 -- 172.21.15.16:0/1635561575 shutdown_connections 2024-09-12T02:09:26.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.377+0000 7f724082d740 1 --2- 172.21.15.16:0/1635561575 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x56472cf9f8e0 0x56472d00d810 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:26.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.377+0000 7f724082d740 1 --2- 172.21.15.16:0/1635561575 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56472cf8fd50 0x56472cf94e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:26.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.377+0000 7f724082d740 1 --2- 172.21.15.16:0/1635561575 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56472cf80f40 0x56472cf94940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:26.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.377+0000 7f724082d740 1 --2- 172.21.15.16:0/1635561575 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56472cf80620 0x56472cf97fd0 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:26.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.377+0000 7f724082d740 1 -- 172.21.15.16:0/1635561575 >> 172.21.15.16:0/1635561575 conn(0x56472cf7ccf0 msgr2=0x56472cf7b650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:26.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.377+0000 7f72177fe640 0 client.17715 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:09:26.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.378+0000 7f724082d740 1 -- 172.21.15.16:0/1635561575 shutdown_connections 2024-09-12T02:09:26.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.378+0000 7f724082d740 1 -- 172.21.15.16:0/1635561575 wait complete. 2024-09-12T02:09:26.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.378+0000 7f724082d740 20 client.17715 trim_cache size 0 max 16384 2024-09-12T02:09:26.379 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T02:09:26.379 INFO:teuthology.orchestra.run.smithi016.stdout: "epoch": 25, 2024-09-12T02:09:26.379 INFO:teuthology.orchestra.run.smithi016.stdout: "leader": 24482, 2024-09-12T02:09:26.379 INFO:teuthology.orchestra.run.smithi016.stdout: "set_version": 30, 2024-09-12T02:09:26.379 INFO:teuthology.orchestra.run.smithi016.stdout: "sets": { 2024-09-12T02:09:26.379 INFO:teuthology.orchestra.run.smithi016.stdout: "d676e596": { 2024-09-12T02:09:26.379 INFO:teuthology.orchestra.run.smithi016.stdout: "version": 30, 2024-09-12T02:09:26.379 INFO:teuthology.orchestra.run.smithi016.stdout: "age_ref": 0.0, 2024-09-12T02:09:26.379 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-12T02:09:26.380 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "QUIESCED", 2024-09-12T02:09:26.380 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-12T02:09:26.380 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:09:26.380 INFO:teuthology.orchestra.run.smithi016.stdout: "timeout": 90.0, 2024-09-12T02:09:26.380 INFO:teuthology.orchestra.run.smithi016.stdout: "expiration": 139.4, 2024-09-12T02:09:26.380 INFO:teuthology.orchestra.run.smithi016.stdout: "members": { 2024-09-12T02:09:26.380 INFO:teuthology.orchestra.run.smithi016.stdout: "file:/": { 2024-09-12T02:09:26.380 INFO:teuthology.orchestra.run.smithi016.stdout: "excluded": false, 2024-09-12T02:09:26.380 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-12T02:09:26.380 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "QUIESCED", 2024-09-12T02:09:26.380 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-12T02:09:26.380 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:09:26.380 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:09:26.381 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:09:26.381 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:09:26.381 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:09:26.381 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T02:09:26.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.378+0000 7f723e454640 1 -- 172.21.15.16:0/783444430 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f72200a2c00 msgr2=0x7f72200a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:26.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.378+0000 7f723e454640 1 --2- 172.21.15.16:0/783444430 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f72200a2c00 0x7f72200a50c0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f7230005c40 tx=0x7f7230005bd0 comp rx=0 tx=0).stop 2024-09-12T02:09:26.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.378+0000 7f723e454640 1 -- 172.21.15.16:0/783444430 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7238102d30 msgr2=0x7f72381985c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:26.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.378+0000 7f723e454640 1 --2- 172.21.15.16:0/783444430 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7238102d30 0x7f72381985c0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f7228009230 tx=0x7f7228004270 comp rx=0 tx=0).stop 2024-09-12T02:09:26.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.378+0000 7f723e454640 1 -- 172.21.15.16:0/783444430 shutdown_connections 2024-09-12T02:09:26.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.378+0000 7f723e454640 1 --2- 172.21.15.16:0/783444430 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f72200a2c00 0x7f72200a50c0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:26.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.378+0000 7f723e454640 1 --2- 172.21.15.16:0/783444430 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f72381082a0 0x7f723819ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:26.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.378+0000 7f723e454640 1 --2- 172.21.15.16:0/783444430 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f72381036e0 0x7f7238198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:26.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.378+0000 7f723e454640 1 --2- 172.21.15.16:0/783444430 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7238102d30 0x7f72381985c0 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:26.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.378+0000 7f723e454640 1 -- 172.21.15.16:0/783444430 >> 172.21.15.16:0/783444430 conn(0x7f72380fc7d0 msgr2=0x7f7238105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:26.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.378+0000 7f723e454640 1 -- 172.21.15.16:0/783444430 shutdown_connections 2024-09-12T02:09:26.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:26.378+0000 7f723e454640 1 -- 172.21.15.16:0/783444430 wait complete. 2024-09-12T02:09:27.164 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'd676e596', quiesce took 3.2 seconds. Will release after: 16.2 2024-09-12T02:09:27.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:09:27 smithi184 ceph-mon[35593]: pgmap v561: 97 pgs: 97 active+clean; 4.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-12T02:09:27.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:09:27 smithi033 ceph-mon[35332]: pgmap v561: 97 pgs: 97 active+clean; 4.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-12T02:09:27.933 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:09:27 smithi016 ceph-mon[30947]: pgmap v561: 97 pgs: 97 active+clean; 4.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-12T02:09:29.806 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T02:09:29.836 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:09:29 smithi016 ceph-mon[30947]: pgmap v562: 97 pgs: 97 active+clean; 4.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T02:09:29.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:09:29 smithi184 ceph-mon[35593]: pgmap v562: 97 pgs: 97 active+clean; 4.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T02:09:29.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:09:29 smithi033 ceph-mon[35332]: pgmap v562: 97 pgs: 97 active+clean; 4.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T02:09:30.080 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:09:30.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.565+0000 7f89d951d640 1 -- 172.21.15.16:0/433220899 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89d4106f10 msgr2=0x7f89d410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:30.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.565+0000 7f89d951d640 1 --2- 172.21.15.16:0/433220899 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89d4106f10 0x7f89d410ea90 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f89cc009ca0 tx=0x7f89cc02a8c0 comp rx=0 tx=0).stop 2024-09-12T02:09:30.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.565+0000 7f89d951d640 1 -- 172.21.15.16:0/433220899 shutdown_connections 2024-09-12T02:09:30.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.565+0000 7f89d951d640 1 --2- 172.21.15.16:0/433220899 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89d4106f10 0x7f89d410ea90 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:30.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.565+0000 7f89d951d640 1 --2- 172.21.15.16:0/433220899 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f89d4102790 0x7f89d41067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:30.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.565+0000 7f89d951d640 1 --2- 172.21.15.16:0/433220899 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f89d4101de0 0x7f89d41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:30.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.565+0000 7f89d951d640 1 -- 172.21.15.16:0/433220899 >> 172.21.15.16:0/433220899 conn(0x7f89d40fd8c0 msgr2=0x7f89d40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:30.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.565+0000 7f89d951d640 1 -- 172.21.15.16:0/433220899 shutdown_connections 2024-09-12T02:09:30.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.565+0000 7f89d951d640 1 -- 172.21.15.16:0/433220899 wait complete. 2024-09-12T02:09:30.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.566+0000 7f89d951d640 1 Processor -- start 2024-09-12T02:09:30.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.566+0000 7f89d951d640 1 -- start start 2024-09-12T02:09:30.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.566+0000 7f89d951d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f89d4101de0 0x7f89d40744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:30.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.566+0000 7f89d951d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89d4102790 0x7f89d4074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:30.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.566+0000 7f89d951d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f89d4106f10 0x7f89d4074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:30.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.566+0000 7f89d951d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f89d4111fc0 con 0x7f89d4102790 2024-09-12T02:09:30.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.566+0000 7f89d951d640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f89d4111e40 con 0x7f89d4101de0 2024-09-12T02:09:30.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.566+0000 7f89d951d640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f89d4112140 con 0x7f89d4106f10 2024-09-12T02:09:30.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.566+0000 7f89d27fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89d4102790 0x7f89d4074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:30.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.566+0000 7f89d27fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89d4102790 0x7f89d4074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48304/0 (socket says 172.21.15.16:48304) 2024-09-12T02:09:30.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.566+0000 7f89d27fc640 1 -- 172.21.15.16:0/757724494 learned_addr learned my addr 172.21.15.16:0/757724494 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:09:30.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.566+0000 7f89d2ffd640 1 --2- 172.21.15.16:0/757724494 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f89d4101de0 0x7f89d40744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:30.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.567+0000 7f89d37fe640 1 --2- 172.21.15.16:0/757724494 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f89d4106f10 0x7f89d4074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:30.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.567+0000 7f89d27fc640 1 -- 172.21.15.16:0/757724494 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f89d4106f10 msgr2=0x7f89d4074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:30.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.567+0000 7f89d27fc640 1 --2- 172.21.15.16:0/757724494 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f89d4106f10 0x7f89d4074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:30.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.567+0000 7f89d27fc640 1 -- 172.21.15.16:0/757724494 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f89d4101de0 msgr2=0x7f89d40744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:30.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.567+0000 7f89d27fc640 1 --2- 172.21.15.16:0/757724494 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f89d4101de0 0x7f89d40744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:30.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.567+0000 7f89d27fc640 1 -- 172.21.15.16:0/757724494 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89d41a68c0 con 0x7f89d4102790 2024-09-12T02:09:30.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.567+0000 7f89d27fc640 1 --2- 172.21.15.16:0/757724494 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89d4102790 0x7f89d4074a30 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f89c800d770 tx=0x7f89c800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:30.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.567+0000 7f89bbfff640 1 -- 172.21.15.16:0/757724494 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89c8017070 con 0x7f89d4102790 2024-09-12T02:09:30.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.567+0000 7f89bbfff640 1 -- 172.21.15.16:0/757724494 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89c8005500 con 0x7f89d4102790 2024-09-12T02:09:30.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.567+0000 7f89d951d640 1 -- 172.21.15.16:0/757724494 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89d41a6bb0 con 0x7f89d4102790 2024-09-12T02:09:30.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.567+0000 7f89bbfff640 1 -- 172.21.15.16:0/757724494 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89c8004e00 con 0x7f89d4102790 2024-09-12T02:09:30.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.567+0000 7f89d951d640 1 -- 172.21.15.16:0/757724494 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89d41a70a0 con 0x7f89d4102790 2024-09-12T02:09:30.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.570+0000 7f89bbfff640 1 -- 172.21.15.16:0/757724494 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f89c8004fa0 con 0x7f89d4102790 2024-09-12T02:09:30.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.570+0000 7f89db8f6740 1 Processor -- start 2024-09-12T02:09:30.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.570+0000 7f89db8f6740 1 -- start start 2024-09-12T02:09:30.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.570+0000 7f89db8f6740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558932aeb4a0 0x558932aeb880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:30.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.570+0000 7f89db8f6740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558932aebdc0 0x558932af6140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:30.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.570+0000 7f89db8f6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558932af6680 0x558932af8a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:30.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.570+0000 7f89db8f6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x558932a512e0 con 0x558932af6680 2024-09-12T02:09:30.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.570+0000 7f89db8f6740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5589329f8ce0 con 0x558932aeb4a0 2024-09-12T02:09:30.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.570+0000 7f89db8f6740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x558932a4d2a0 con 0x558932aebdc0 2024-09-12T02:09:30.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.570+0000 7f89d27fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558932af6680 0x558932af8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:30.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.570+0000 7f89d27fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558932af6680 0x558932af8a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48306/0 (socket says 172.21.15.16:48306) 2024-09-12T02:09:30.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.570+0000 7f89d27fc640 1 -- 172.21.15.16:0/2054862056 learned_addr learned my addr 172.21.15.16:0/2054862056 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:09:30.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.570+0000 7f89d2ffd640 1 --2- 172.21.15.16:0/2054862056 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558932aebdc0 0x558932af6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:30.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.570+0000 7f89d37fe640 1 --2- 172.21.15.16:0/2054862056 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558932aeb4a0 0x558932aeb880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:30.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.570+0000 7f89bbfff640 1 --2- 172.21.15.16:0/757724494 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f89bc0a2b80 0x7f89bc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:30.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.571+0000 7f89d27fc640 1 -- 172.21.15.16:0/2054862056 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558932aebdc0 msgr2=0x558932af6140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:09:30.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.571+0000 7f89d27fc640 1 --2- 172.21.15.16:0/2054862056 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558932aebdc0 0x558932af6140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:30.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.571+0000 7f89d27fc640 1 -- 172.21.15.16:0/2054862056 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558932aeb4a0 msgr2=0x558932aeb880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:09:30.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.571+0000 7f89d27fc640 1 --2- 172.21.15.16:0/2054862056 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558932aeb4a0 0x558932aeb880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:30.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.571+0000 7f89d27fc640 1 -- 172.21.15.16:0/2054862056 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589329dd440 con 0x558932af6680 2024-09-12T02:09:30.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.571+0000 7f89d37fe640 1 --2- 172.21.15.16:0/757724494 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f89bc0a2b80 0x7f89bc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:30.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.571+0000 7f89bbfff640 1 -- 172.21.15.16:0/757724494 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f89c80ce960 con 0x7f89d4102790 2024-09-12T02:09:30.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.571+0000 7f89d27fc640 1 --2- 172.21.15.16:0/2054862056 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558932af6680 0x558932af8a70 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f89c806bfe0 tx=0x7f89c806b110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de679b874e8cc04a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:30.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.571+0000 7f89b97fa640 1 -- 172.21.15.16:0/2054862056 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89c8020070 con 0x558932af6680 2024-09-12T02:09:30.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.571+0000 7f89b97fa640 1 -- 172.21.15.16:0/2054862056 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89c806c750 con 0x558932af6680 2024-09-12T02:09:30.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.571+0000 7f89b97fa640 1 -- 172.21.15.16:0/2054862056 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89c80056b0 con 0x558932af6680 2024-09-12T02:09:30.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.572+0000 7f89db8f6740 1 -- 172.21.15.16:0/2054862056 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558932af6680 msgr2=0x558932af8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:30.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.572+0000 7f89db8f6740 1 --2- 172.21.15.16:0/2054862056 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558932af6680 0x558932af8a70 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f89c806bfe0 tx=0x7f89c806b110 comp rx=0 tx=0).stop 2024-09-12T02:09:30.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.572+0000 7f89d37fe640 1 --2- 172.21.15.16:0/757724494 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f89bc0a2b80 0x7f89bc0a5040 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f89cc005fe0 tx=0x7f89cc0095c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:30.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.572+0000 7f89db8f6740 1 -- 172.21.15.16:0/2054862056 shutdown_connections 2024-09-12T02:09:30.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.572+0000 7f89db8f6740 1 --2- 172.21.15.16:0/2054862056 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558932af6680 0x558932af8a70 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:30.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.572+0000 7f89db8f6740 1 --2- 172.21.15.16:0/2054862056 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558932aebdc0 0x558932af6140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:30.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.572+0000 7f89db8f6740 1 --2- 172.21.15.16:0/2054862056 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558932aeb4a0 0x558932aeb880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:30.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.572+0000 7f89db8f6740 1 -- 172.21.15.16:0/2054862056 >> 172.21.15.16:0/2054862056 conn(0x558932ae7a30 msgr2=0x558932ae7e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:30.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.572+0000 7f89db8f6740 1 -- 172.21.15.16:0/2054862056 shutdown_connections 2024-09-12T02:09:30.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.572+0000 7f89db8f6740 1 -- 172.21.15.16:0/2054862056 wait complete. 2024-09-12T02:09:30.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.572+0000 7f89db8f6740 1 -- start start 2024-09-12T02:09:30.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.572+0000 7f89db8f6740 1 Processor -- start 2024-09-12T02:09:30.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.573+0000 7f89db8f6740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558932aeb4a0 0x558932af5f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:30.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.573+0000 7f89db8f6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558932aebdc0 0x558932af3210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:30.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.573+0000 7f89db8f6740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558932af6680 0x558932af3750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:30.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.573+0000 7f89db8f6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x558932af9250 con 0x558932aebdc0 2024-09-12T02:09:30.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.573+0000 7f89db8f6740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x558932a0c050 con 0x558932af6680 2024-09-12T02:09:30.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.573+0000 7f89db8f6740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x558932af93d0 con 0x558932aeb4a0 2024-09-12T02:09:30.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.573+0000 7f89db8f6740 10 client.0 _ll_register_callbacks cb 0x558932a4c160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:09:30.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.573+0000 7f89d27fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558932aebdc0 0x558932af3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:30.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.573+0000 7f89d27fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558932aebdc0 0x558932af3210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48322/0 (socket says 172.21.15.16:48322) 2024-09-12T02:09:30.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.573+0000 7f89d27fc640 1 -- 172.21.15.16:0/669486407 learned_addr learned my addr 172.21.15.16:0/669486407 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:09:30.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.573+0000 7f89d2ffd640 1 --2- 172.21.15.16:0/669486407 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558932aeb4a0 0x558932af5f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:30.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.573+0000 7f89d37fe640 1 --2- 172.21.15.16:0/669486407 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558932af6680 0x558932af3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:30.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.573+0000 7f89d27fc640 1 -- 172.21.15.16:0/669486407 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558932aeb4a0 msgr2=0x558932af5f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:09:30.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.573+0000 7f89d27fc640 1 --2- 172.21.15.16:0/669486407 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558932aeb4a0 0x558932af5f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:30.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.573+0000 7f89d27fc640 1 -- 172.21.15.16:0/669486407 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558932af6680 msgr2=0x558932af3750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:09:30.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.573+0000 7f89d27fc640 1 --2- 172.21.15.16:0/669486407 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558932af6680 0x558932af3750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:30.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.573+0000 7f89d27fc640 1 -- 172.21.15.16:0/669486407 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5589327c23c0 con 0x558932aebdc0 2024-09-12T02:09:30.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.573+0000 7f89d37fe640 1 --2- 172.21.15.16:0/669486407 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558932af6680 0x558932af3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:09:30.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.573+0000 7f89d27fc640 1 --2- 172.21.15.16:0/669486407 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558932aebdc0 0x558932af3210 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f89c809af60 tx=0x7f89c809b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f64bdc7403d6b07 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:30.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.573+0000 7f89b6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:09:30.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.573+0000 7f89b6ff5640 1 -- 172.21.15.16:0/669486407 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89c8020040 con 0x558932aebdc0 2024-09-12T02:09:30.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.574+0000 7f89b6ff5640 1 -- 172.21.15.16:0/669486407 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f89c8099300 con 0x558932aebdc0 2024-09-12T02:09:30.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.574+0000 7f89db8f6740 1 -- 172.21.15.16:0/669486407 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x558932a5e850 con 0x558932aebdc0 2024-09-12T02:09:30.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.574+0000 7f89b6ff5640 1 -- 172.21.15.16:0/669486407 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f89c80056b0 con 0x558932aebdc0 2024-09-12T02:09:30.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.574+0000 7f89db8f6740 10 client.17727 fetch_fsmap learned FSMap version 25 2024-09-12T02:09:30.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.574+0000 7f89db8f6740 1 -- 172.21.15.16:0/669486407 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x558932b04bc0 con 0x558932aebdc0 2024-09-12T02:09:30.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.574+0000 7f89b6ff5640 1 -- 172.21.15.16:0/669486407 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x558932a5e850 con 0x558932aebdc0 2024-09-12T02:09:30.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.574+0000 7f89b6ff5640 1 -- 172.21.15.16:0/669486407 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f89c80978d0 con 0x558932aebdc0 2024-09-12T02:09:30.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.574+0000 7f89db8f6740 10 client.17727 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:09:30.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.574+0000 7f89db8f6740 10 client.17727 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:09:30.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.574+0000 7f89db8f6740 20 client.17727 populate_metadata read hostname 'smithi016' 2024-09-12T02:09:30.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.574+0000 7f89db8f6740 1 --2- 172.21.15.16:0/669486407 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x558932b05f10 0x558932b08300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:30.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.574+0000 7f89db8f6740 4 client.17727 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:09:30.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.574+0000 7f89db8f6740 1 -- 172.21.15.16:0/669486407 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558932a58d60 con 0x558932b05f10 2024-09-12T02:09:30.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.574+0000 7f89d2ffd640 1 --2- 172.21.15.16:0/669486407 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x558932b05f10 0x558932b08300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:30.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.575+0000 7f89b6ff5640 10 client.17727 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:09:30.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.575+0000 7f89d2ffd640 1 --2- 172.21.15.16:0/669486407 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x558932b05f10 0x558932b08300 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e21fd48726f83e1b server_cookie=d7614c570c340390 in_seq=0 out_seq=0 2024-09-12T02:09:30.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.575+0000 7f89b6ff5640 1 -- 172.21.15.16:0/669486407 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558932a58d60 con 0x558932b05f10 2024-09-12T02:09:30.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.575+0000 7f89b6ff5640 10 client.17727 handle_command_reply: tid=0 2024-09-12T02:09:30.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.576+0000 7f89db8f6740 1 client.17727 shutdown 2024-09-12T02:09:30.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.576+0000 7f89db8f6740 1 -- 172.21.15.16:0/669486407 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558932aebdc0 msgr2=0x558932af3210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:30.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.576+0000 7f89db8f6740 1 --2- 172.21.15.16:0/669486407 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558932aebdc0 0x558932af3210 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f89c809af60 tx=0x7f89c809b260 comp rx=0 tx=0).stop 2024-09-12T02:09:30.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.576+0000 7f89db8f6740 1 -- 172.21.15.16:0/669486407 shutdown_connections 2024-09-12T02:09:30.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.576+0000 7f89db8f6740 1 --2- 172.21.15.16:0/669486407 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x558932b05f10 0x558932b08300 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:30.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.576+0000 7f89db8f6740 1 --2- 172.21.15.16:0/669486407 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558932af6680 0x558932af3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:30.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.576+0000 7f89db8f6740 1 --2- 172.21.15.16:0/669486407 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558932aebdc0 0x558932af3210 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:30.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.576+0000 7f89db8f6740 1 --2- 172.21.15.16:0/669486407 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558932aeb4a0 0x558932af5f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:30.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.576+0000 7f89db8f6740 1 -- 172.21.15.16:0/669486407 >> 172.21.15.16:0/669486407 conn(0x558932ae7a30 msgr2=0x558932aed710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:30.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.576+0000 7f89b6ff5640 0 client.17727 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:09:30.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.576+0000 7f89db8f6740 1 -- 172.21.15.16:0/669486407 shutdown_connections 2024-09-12T02:09:30.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.576+0000 7f89db8f6740 1 -- 172.21.15.16:0/669486407 wait complete. 2024-09-12T02:09:30.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.576+0000 7f89db8f6740 20 client.17727 trim_cache size 0 max 16384 2024-09-12T02:09:30.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.593+0000 7f89db8f6740 1 Processor -- start 2024-09-12T02:09:30.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.593+0000 7f89db8f6740 1 -- start start 2024-09-12T02:09:30.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.593+0000 7f89db8f6740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558932ae7a30 0x558932af3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:30.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.593+0000 7f89db8f6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558932aeb4a0 0x558932b0af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:30.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.593+0000 7f89db8f6740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558932aebdc0 0x558932b0b480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:30.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.593+0000 7f89db8f6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f89d41b1e80 con 0x558932aeb4a0 2024-09-12T02:09:30.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.593+0000 7f89db8f6740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f89d41af750 con 0x558932ae7a30 2024-09-12T02:09:30.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.593+0000 7f89db8f6740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f89d41ad5c0 con 0x558932aebdc0 2024-09-12T02:09:30.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.593+0000 7f89d27fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558932aeb4a0 0x558932b0af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:30.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.593+0000 7f89d27fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558932aeb4a0 0x558932b0af40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48324/0 (socket says 172.21.15.16:48324) 2024-09-12T02:09:30.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.593+0000 7f89d27fc640 1 -- 172.21.15.16:0/2890052839 learned_addr learned my addr 172.21.15.16:0/2890052839 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:09:30.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.593+0000 7f89d27fc640 1 -- 172.21.15.16:0/2890052839 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558932aebdc0 msgr2=0x558932b0b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:09:30.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.593+0000 7f89d27fc640 1 --2- 172.21.15.16:0/2890052839 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558932aebdc0 0x558932b0b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:30.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.594+0000 7f89d27fc640 1 -- 172.21.15.16:0/2890052839 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558932ae7a30 msgr2=0x558932af3fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:09:30.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.594+0000 7f89d2ffd640 1 --2- 172.21.15.16:0/2890052839 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558932ae7a30 0x558932af3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:30.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.594+0000 7f89d27fc640 1 --2- 172.21.15.16:0/2890052839 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558932ae7a30 0x558932af3fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:30.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.594+0000 7f89d27fc640 1 -- 172.21.15.16:0/2890052839 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558932aebbb0 con 0x558932aeb4a0 2024-09-12T02:09:30.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.594+0000 7f89d2ffd640 1 --2- 172.21.15.16:0/2890052839 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558932ae7a30 0x558932af3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:09:30.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.594+0000 7f89d27fc640 1 --2- 172.21.15.16:0/2890052839 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558932aeb4a0 0x558932b0af40 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f89c809b1f0 tx=0x7f89c80cc550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=527ee5dbc84f60d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:30.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.594+0000 7f89b8ff9640 1 -- 172.21.15.16:0/2890052839 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89c8020070 con 0x558932aeb4a0 2024-09-12T02:09:30.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.594+0000 7f89b8ff9640 1 -- 172.21.15.16:0/2890052839 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89c80b8070 con 0x558932aeb4a0 2024-09-12T02:09:30.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.594+0000 7f89b8ff9640 1 -- 172.21.15.16:0/2890052839 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89c809ada0 con 0x558932aeb4a0 2024-09-12T02:09:30.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.594+0000 7f89db8f6740 1 -- 172.21.15.16:0/2890052839 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558932aeb4a0 msgr2=0x558932b0af40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:30.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.594+0000 7f89db8f6740 1 --2- 172.21.15.16:0/2890052839 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558932aeb4a0 0x558932b0af40 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f89c809b1f0 tx=0x7f89c80cc550 comp rx=0 tx=0).stop 2024-09-12T02:09:30.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.594+0000 7f89db8f6740 1 -- 172.21.15.16:0/2890052839 shutdown_connections 2024-09-12T02:09:30.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.594+0000 7f89db8f6740 1 --2- 172.21.15.16:0/2890052839 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558932aebdc0 0x558932b0b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:30.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.594+0000 7f89db8f6740 1 --2- 172.21.15.16:0/2890052839 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558932aeb4a0 0x558932b0af40 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:30.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.594+0000 7f89db8f6740 1 --2- 172.21.15.16:0/2890052839 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558932ae7a30 0x558932af3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:30.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.595+0000 7f89db8f6740 1 -- 172.21.15.16:0/2890052839 >> 172.21.15.16:0/2890052839 conn(0x558932af38b0 msgr2=0x558932af1ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:30.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.595+0000 7f89db8f6740 1 -- 172.21.15.16:0/2890052839 shutdown_connections 2024-09-12T02:09:30.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.595+0000 7f89db8f6740 1 -- 172.21.15.16:0/2890052839 wait complete. 2024-09-12T02:09:30.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.595+0000 7f89db8f6740 1 -- start start 2024-09-12T02:09:30.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.595+0000 7f89db8f6740 1 Processor -- start 2024-09-12T02:09:30.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.595+0000 7f89db8f6740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558932ae7a30 0x558932b0b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:30.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.595+0000 7f89db8f6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558932aeb4a0 0x558932b08b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:30.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.595+0000 7f89db8f6740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558932aebdc0 0x558932b092f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:30.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.595+0000 7f89db8f6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f89d41adc10 con 0x558932aeb4a0 2024-09-12T02:09:30.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.595+0000 7f89db8f6740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f89d41af080 con 0x558932ae7a30 2024-09-12T02:09:30.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.595+0000 7f89db8f6740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f89d41ae2e0 con 0x558932aebdc0 2024-09-12T02:09:30.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.595+0000 7f89db8f6740 10 client.0 _ll_register_callbacks cb 0x7f89c4004120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:09:30.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.595+0000 7f89d27fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558932aeb4a0 0x558932b08b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:30.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.595+0000 7f89d27fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558932aeb4a0 0x558932b08b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48330/0 (socket says 172.21.15.16:48330) 2024-09-12T02:09:30.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.595+0000 7f89d27fc640 1 -- 172.21.15.16:0/133261767 learned_addr learned my addr 172.21.15.16:0/133261767 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:09:30.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.595+0000 7f89d2ffd640 1 --2- 172.21.15.16:0/133261767 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558932ae7a30 0x558932b0b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:30.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.596+0000 7f89d37fe640 1 --2- 172.21.15.16:0/133261767 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558932aebdc0 0x558932b092f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:30.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.596+0000 7f89d27fc640 1 -- 172.21.15.16:0/133261767 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558932aebdc0 msgr2=0x558932b092f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:09:30.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.596+0000 7f89d27fc640 1 --2- 172.21.15.16:0/133261767 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558932aebdc0 0x558932b092f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:30.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.596+0000 7f89d27fc640 1 -- 172.21.15.16:0/133261767 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558932ae7a30 msgr2=0x558932b0b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:09:30.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.596+0000 7f89d27fc640 1 --2- 172.21.15.16:0/133261767 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558932ae7a30 0x558932b0b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:30.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.596+0000 7f89d27fc640 1 -- 172.21.15.16:0/133261767 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558932b6f850 con 0x558932aeb4a0 2024-09-12T02:09:30.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.596+0000 7f89d27fc640 1 --2- 172.21.15.16:0/133261767 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558932aeb4a0 0x558932b08b60 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f89c80cede0 tx=0x7f89c80b9d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d164ad2b0bdb49c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:30.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.596+0000 7f89b6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:09:30.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.596+0000 7f89b6ff5640 1 -- 172.21.15.16:0/133261767 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89c8020030 con 0x558932aeb4a0 2024-09-12T02:09:30.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.596+0000 7f89b6ff5640 1 -- 172.21.15.16:0/133261767 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f89c8005290 con 0x558932aeb4a0 2024-09-12T02:09:30.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.596+0000 7f89db8f6740 1 -- 172.21.15.16:0/133261767 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f89c80b8070 con 0x558932aeb4a0 2024-09-12T02:09:30.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.596+0000 7f89b6ff5640 1 -- 172.21.15.16:0/133261767 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f89c80046c0 con 0x558932aeb4a0 2024-09-12T02:09:30.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.596+0000 7f89db8f6740 10 client.17733 fetch_fsmap learned FSMap version 25 2024-09-12T02:09:30.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.596+0000 7f89db8f6740 1 -- 172.21.15.16:0/133261767 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x558932b6f850 con 0x558932aeb4a0 2024-09-12T02:09:30.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.596+0000 7f89b6ff5640 1 -- 172.21.15.16:0/133261767 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89c80b8070 con 0x558932aeb4a0 2024-09-12T02:09:30.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.597+0000 7f89b6ff5640 1 -- 172.21.15.16:0/133261767 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f89c80bb110 con 0x558932aeb4a0 2024-09-12T02:09:30.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.597+0000 7f89db8f6740 10 client.17733 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:09:30.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.597+0000 7f89db8f6740 10 client.17733 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:09:30.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.597+0000 7f89db8f6740 20 client.17733 populate_metadata read hostname 'smithi016' 2024-09-12T02:09:30.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.597+0000 7f89db8f6740 1 --2- 172.21.15.16:0/133261767 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x558932af6680 0x558932b72cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:30.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.597+0000 7f89db8f6740 4 client.17733 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T02:09:30.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.597+0000 7f89db8f6740 1 -- 172.21.15.16:0/133261767 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5589329d5210 con 0x558932af6680 2024-09-12T02:09:30.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.597+0000 7f89d2ffd640 1 --2- 172.21.15.16:0/133261767 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x558932af6680 0x558932b72cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:30.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.597+0000 7f89d2ffd640 1 --2- 172.21.15.16:0/133261767 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x558932af6680 0x558932b72cb0 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4eabc53e23971537 server_cookie=ad59e34b9b4e1554 in_seq=0 out_seq=0 2024-09-12T02:09:30.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.597+0000 7f89b6ff5640 10 client.17733 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:09:30.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.598+0000 7f89b6ff5640 1 -- 172.21.15.16:0/133261767 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5589329d5210 con 0x558932af6680 2024-09-12T02:09:30.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.598+0000 7f89b6ff5640 10 client.17733 handle_command_reply: tid=0 2024-09-12T02:09:30.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.598+0000 7f89db8f6740 1 client.17733 shutdown 2024-09-12T02:09:30.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.598+0000 7f89db8f6740 1 -- 172.21.15.16:0/133261767 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558932aeb4a0 msgr2=0x558932b08b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:30.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.598+0000 7f89db8f6740 1 --2- 172.21.15.16:0/133261767 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558932aeb4a0 0x558932b08b60 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f89c80cede0 tx=0x7f89c80b9d10 comp rx=0 tx=0).stop 2024-09-12T02:09:30.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.598+0000 7f89db8f6740 1 -- 172.21.15.16:0/133261767 shutdown_connections 2024-09-12T02:09:30.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.598+0000 7f89db8f6740 1 --2- 172.21.15.16:0/133261767 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x558932af6680 0x558932b72cb0 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:30.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.598+0000 7f89db8f6740 1 --2- 172.21.15.16:0/133261767 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558932aebdc0 0x558932b092f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:30.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.598+0000 7f89db8f6740 1 --2- 172.21.15.16:0/133261767 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558932aeb4a0 0x558932b08b60 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:30.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.598+0000 7f89db8f6740 1 --2- 172.21.15.16:0/133261767 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558932ae7a30 0x558932b0b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:30.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.598+0000 7f89db8f6740 1 -- 172.21.15.16:0/133261767 >> 172.21.15.16:0/133261767 conn(0x558932af38b0 msgr2=0x558932ae6b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:30.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.598+0000 7f89b6ff5640 0 client.17733 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:09:30.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.598+0000 7f89db8f6740 1 -- 172.21.15.16:0/133261767 shutdown_connections 2024-09-12T02:09:30.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.598+0000 7f89db8f6740 1 -- 172.21.15.16:0/133261767 wait complete. 2024-09-12T02:09:30.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.598+0000 7f89db8f6740 20 client.17733 trim_cache size 0 max 16384 2024-09-12T02:09:30.655 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T02:09:30.655 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-09-12T02:09:30.655 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {} 2024-09-12T02:09:30.655 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T02:09:30.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.598+0000 7f89d951d640 1 -- 172.21.15.16:0/757724494 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f89bc0a2b80 msgr2=0x7f89bc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:30.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.598+0000 7f89d951d640 1 --2- 172.21.15.16:0/757724494 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f89bc0a2b80 0x7f89bc0a5040 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f89cc005fe0 tx=0x7f89cc0095c0 comp rx=0 tx=0).stop 2024-09-12T02:09:30.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.598+0000 7f89d951d640 1 -- 172.21.15.16:0/757724494 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89d4102790 msgr2=0x7f89d4074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:30.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.598+0000 7f89d951d640 1 --2- 172.21.15.16:0/757724494 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89d4102790 0x7f89d4074a30 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f89c800d770 tx=0x7f89c800dc30 comp rx=0 tx=0).stop 2024-09-12T02:09:30.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.598+0000 7f89d951d640 1 -- 172.21.15.16:0/757724494 shutdown_connections 2024-09-12T02:09:30.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.598+0000 7f89d951d640 1 --2- 172.21.15.16:0/757724494 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f89bc0a2b80 0x7f89bc0a5040 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:30.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.598+0000 7f89d951d640 1 --2- 172.21.15.16:0/757724494 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f89d4106f10 0x7f89d4074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:30.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.598+0000 7f89d951d640 1 --2- 172.21.15.16:0/757724494 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89d4102790 0x7f89d4074a30 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:30.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.598+0000 7f89d951d640 1 --2- 172.21.15.16:0/757724494 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f89d4101de0 0x7f89d40744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:30.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.598+0000 7f89d951d640 1 -- 172.21.15.16:0/757724494 >> 172.21.15.16:0/757724494 conn(0x7f89d40fd8c0 msgr2=0x7f89d40febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:30.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.599+0000 7f89d951d640 1 -- 172.21.15.16:0/757724494 shutdown_connections 2024-09-12T02:09:30.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:30.599+0000 7f89d951d640 1 -- 172.21.15.16:0/757724494 wait complete. 2024-09-12T02:09:31.339 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-12T02:09:31.339 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:09:31.644 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:09:31.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:09:31 smithi016 ceph-mon[30947]: pgmap v563: 97 pgs: 97 active+clean; 4.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 16 KiB/s wr, 15 op/s 2024-09-12T02:09:31.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:09:31 smithi184 ceph-mon[35593]: pgmap v563: 97 pgs: 97 active+clean; 4.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 16 KiB/s wr, 15 op/s 2024-09-12T02:09:31.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:09:31 smithi033 ceph-mon[35332]: pgmap v563: 97 pgs: 97 active+clean; 4.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 16 KiB/s wr, 15 op/s 2024-09-12T02:09:32.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:32.171+0000 7f6759c9e640 1 -- 172.21.15.16:0/608230718 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6754106f50 msgr2=0x7f675410eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:32.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:32.171+0000 7f6759c9e640 1 --2- 172.21.15.16:0/608230718 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6754106f50 0x7f675410eaf0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f674c009cd0 tx=0x7f674c02a8e0 comp rx=0 tx=0).stop 2024-09-12T02:09:32.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:32.171+0000 7f6759c9e640 1 -- 172.21.15.16:0/608230718 shutdown_connections 2024-09-12T02:09:32.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:32.171+0000 7f6759c9e640 1 --2- 172.21.15.16:0/608230718 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6754106f50 0x7f675410eaf0 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:32.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:32.171+0000 7f6759c9e640 1 --2- 172.21.15.16:0/608230718 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6754100ca0 0x7f6754106840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:32.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:32.171+0000 7f6759c9e640 1 --2- 172.21.15.16:0/608230718 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f67541002f0 0x7f67541006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:32.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:32.171+0000 7f6759c9e640 1 -- 172.21.15.16:0/608230718 >> 172.21.15.16:0/608230718 conn(0x7f6754077210 msgr2=0x7f67540fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:32.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:32.171+0000 7f6759c9e640 1 -- 172.21.15.16:0/608230718 shutdown_connections 2024-09-12T02:09:32.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:32.171+0000 7f6759c9e640 1 -- 172.21.15.16:0/608230718 wait complete. 2024-09-12T02:09:32.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:32.172+0000 7f6759c9e640 1 Processor -- start 2024-09-12T02:09:32.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:32.172+0000 7f6759c9e640 1 -- start start 2024-09-12T02:09:32.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:32.172+0000 7f6759c9e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f67541002f0 0x7f6754101270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:32.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:32.172+0000 7f6759c9e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6754100ca0 0x7f67541017b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:32.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:32.172+0000 7f6759c9e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6754106f50 0x7f6754101cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:32.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:32.172+0000 7f6759c9e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6754111fa0 con 0x7f6754106f50 2024-09-12T02:09:32.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:32.172+0000 7f6759c9e640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6754111e20 con 0x7f67541002f0 2024-09-12T02:09:32.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:32.172+0000 7f6759c9e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f6754112120 con 0x7f6754100ca0 2024-09-12T02:09:32.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:32.173+0000 7f6753fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6754106f50 0x7f6754101cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:32.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:32.173+0000 7f67537fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f67541002f0 0x7f6754101270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:32.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:32.173+0000 7f6753fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6754106f50 0x7f6754101cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39646/0 (socket says 172.21.15.16:39646) 2024-09-12T02:09:32.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:32.173+0000 7f6753fff640 1 -- 172.21.15.16:0/2431157120 learned_addr learned my addr 172.21.15.16:0/2431157120 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:09:32.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:32.173+0000 7f674bfff640 1 --2- 172.21.15.16:0/2431157120 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6754100ca0 0x7f67541017b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:32.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:32.173+0000 7f6753fff640 1 -- 172.21.15.16:0/2431157120 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6754100ca0 msgr2=0x7f67541017b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:32.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:32.173+0000 7f6753fff640 1 --2- 172.21.15.16:0/2431157120 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6754100ca0 0x7f67541017b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:32.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:32.173+0000 7f6753fff640 1 -- 172.21.15.16:0/2431157120 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f67541002f0 msgr2=0x7f6754101270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:32.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:32.173+0000 7f6753fff640 1 --2- 172.21.15.16:0/2431157120 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f67541002f0 0x7f6754101270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:32.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:32.173+0000 7f6753fff640 1 -- 172.21.15.16:0/2431157120 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67541a6960 con 0x7f6754106f50 2024-09-12T02:09:32.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:32.173+0000 7f6753fff640 1 --2- 172.21.15.16:0/2431157120 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6754106f50 0x7f6754101cf0 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f674c005cc0 tx=0x7f674c003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:32.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:32.173+0000 7f67517fa640 1 -- 172.21.15.16:0/2431157120 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f674c004040 con 0x7f6754106f50 2024-09-12T02:09:32.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:32.173+0000 7f67517fa640 1 -- 172.21.15.16:0/2431157120 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f674c0046c0 con 0x7f6754106f50 2024-09-12T02:09:32.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:32.173+0000 7f6759c9e640 1 -- 172.21.15.16:0/2431157120 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67541a6bf0 con 0x7f6754106f50 2024-09-12T02:09:32.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:32.173+0000 7f67517fa640 1 -- 172.21.15.16:0/2431157120 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f674c046480 con 0x7f6754106f50 2024-09-12T02:09:32.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:32.173+0000 7f6759c9e640 1 -- 172.21.15.16:0/2431157120 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67541a7180 con 0x7f6754106f50 2024-09-12T02:09:32.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:32.174+0000 7f6759c9e640 1 -- 172.21.15.16:0/2431157120 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67541006d0 con 0x7f6754106f50 2024-09-12T02:09:32.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:32.176+0000 7f67517fa640 1 -- 172.21.15.16:0/2431157120 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f674c0305a0 con 0x7f6754106f50 2024-09-12T02:09:32.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:32.176+0000 7f67517fa640 1 --2- 172.21.15.16:0/2431157120 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f67400a2c00 0x7f67400a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:32.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:32.176+0000 7f67537fe640 1 --2- 172.21.15.16:0/2431157120 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f67400a2c00 0x7f67400a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:32.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:32.176+0000 7f67517fa640 1 -- 172.21.15.16:0/2431157120 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f674c02e030 con 0x7f6754106f50 2024-09-12T02:09:32.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:32.177+0000 7f67537fe640 1 --2- 172.21.15.16:0/2431157120 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f67400a2c00 0x7f67400a50c0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f674400bbe0 tx=0x7f674400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:32.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:32.181+0000 7f67517fa640 1 -- 172.21.15.16:0/2431157120 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f674c04b060 con 0x7f6754106f50 2024-09-12T02:09:32.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:32.392+0000 7f6759c9e640 1 -- 172.21.15.16:0/2431157120 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f675410a120 con 0x7f6754106f50 2024-09-12T02:09:32.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:32.393+0000 7f67517fa640 1 -- 172.21.15.16:0/2431157120 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11911 (secure 0 0 0) 0x7f674c0bd740 con 0x7f6754106f50 2024-09-12T02:09:32.394 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T02:09:32.394 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":25,"btime":"2024-09-12T02:01:37:192021+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3279123279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3279123279},{"type":"v1","addr":"172.21.15.16:6835","nonce":3279123279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4119452382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4119452382},{"type":"v1","addr":"172.21.15.16:6837","nonce":4119452382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/3263823165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3263823165},{"type":"v1","addr":"172.21.15.16:6839","nonce":3263823165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/1510290732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":1510290732},{"type":"v1","addr":"172.21.15.16:6841","nonce":1510290732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/279674603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":279674603},{"type":"v1","addr":"172.21.15.33:6833","nonce":279674603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3555728028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3555728028},{"type":"v1","addr":"172.21.15.33:6835","nonce":3555728028}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T01:59:56.117608+0000","modified":"2024-09-12T02:01:36.294760+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24482,"mds_1":24481,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6833/1563374106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1563374106},{"type":"v1","addr":"172.21.15.184:6833","nonce":1563374106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6837/1892833868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1892833868},{"type":"v1","addr":"172.21.15.33:6837","nonce":1892833868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/1457679686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1457679686},{"type":"v1","addr":"172.21.15.184:6835","nonce":1457679686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.184:6837/1793393087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":1793393087},{"type":"v1","addr":"172.21.15.184:6837","nonce":1793393087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.184:6839/4203347941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":4203347941},{"type":"v1","addr":"172.21.15.184:6839","nonce":4203347941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.33:6839/4136670204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":4136670204},{"type":"v1","addr":"172.21.15.33:6839","nonce":4136670204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24481,24482,24469]},"id":1}]} 2024-09-12T02:09:32.394 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 25 2024-09-12T02:09:32.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:32.395+0000 7f6759c9e640 1 -- 172.21.15.16:0/2431157120 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f67400a2c00 msgr2=0x7f67400a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:32.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:32.395+0000 7f6759c9e640 1 --2- 172.21.15.16:0/2431157120 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f67400a2c00 0x7f67400a50c0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f674400bbe0 tx=0x7f674400b390 comp rx=0 tx=0).stop 2024-09-12T02:09:32.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:32.395+0000 7f6759c9e640 1 -- 172.21.15.16:0/2431157120 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6754106f50 msgr2=0x7f6754101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:32.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:32.395+0000 7f6759c9e640 1 --2- 172.21.15.16:0/2431157120 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6754106f50 0x7f6754101cf0 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f674c005cc0 tx=0x7f674c003970 comp rx=0 tx=0).stop 2024-09-12T02:09:32.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:32.396+0000 7f6759c9e640 1 -- 172.21.15.16:0/2431157120 shutdown_connections 2024-09-12T02:09:32.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:32.396+0000 7f6759c9e640 1 --2- 172.21.15.16:0/2431157120 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f67400a2c00 0x7f67400a50c0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:32.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:32.396+0000 7f6759c9e640 1 --2- 172.21.15.16:0/2431157120 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6754106f50 0x7f6754101cf0 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:32.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:32.396+0000 7f6759c9e640 1 --2- 172.21.15.16:0/2431157120 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6754100ca0 0x7f67541017b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:32.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:32.396+0000 7f6759c9e640 1 --2- 172.21.15.16:0/2431157120 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f67541002f0 0x7f6754101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:32.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:32.396+0000 7f6759c9e640 1 -- 172.21.15.16:0/2431157120 >> 172.21.15.16:0/2431157120 conn(0x7f6754077210 msgr2=0x7f6754105210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:32.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:32.396+0000 7f6759c9e640 1 -- 172.21.15.16:0/2431157120 shutdown_connections 2024-09-12T02:09:32.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:32.396+0000 7f6759c9e640 1 -- 172.21.15.16:0/2431157120 wait complete. 2024-09-12T02:09:32.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:09:32 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/2431157120' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:09:32.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:09:32 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/2431157120' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:09:32.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:09:32 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/2431157120' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:09:33.129 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-12T02:09:33.393 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:09:33.797 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:09:33 smithi016 ceph-mon[30947]: pgmap v564: 97 pgs: 97 active+clean; 4.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-09-12T02:09:33.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:09:33 smithi184 ceph-mon[35593]: pgmap v564: 97 pgs: 97 active+clean; 4.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-09-12T02:09:33.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.881+0000 7f09eecba640 1 -- 172.21.15.16:0/1136664073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09e8106f10 msgr2=0x7f09e810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:33.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.881+0000 7f09eecba640 1 --2- 172.21.15.16:0/1136664073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09e8106f10 0x7f09e810ea90 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f09e0009ca0 tx=0x7f09e002a8c0 comp rx=0 tx=0).stop 2024-09-12T02:09:33.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.881+0000 7f09eecba640 1 -- 172.21.15.16:0/1136664073 shutdown_connections 2024-09-12T02:09:33.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.881+0000 7f09eecba640 1 --2- 172.21.15.16:0/1136664073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09e8106f10 0x7f09e810ea90 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:33.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.881+0000 7f09eecba640 1 --2- 172.21.15.16:0/1136664073 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f09e8102790 0x7f09e81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:33.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.881+0000 7f09eecba640 1 --2- 172.21.15.16:0/1136664073 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f09e8101de0 0x7f09e81021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:33.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.881+0000 7f09eecba640 1 -- 172.21.15.16:0/1136664073 >> 172.21.15.16:0/1136664073 conn(0x7f09e80fd8c0 msgr2=0x7f09e80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:33.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.881+0000 7f09eecba640 1 -- 172.21.15.16:0/1136664073 shutdown_connections 2024-09-12T02:09:33.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.881+0000 7f09eecba640 1 -- 172.21.15.16:0/1136664073 wait complete. 2024-09-12T02:09:33.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.882+0000 7f09eecba640 1 Processor -- start 2024-09-12T02:09:33.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.882+0000 7f09eecba640 1 -- start start 2024-09-12T02:09:33.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.882+0000 7f09eecba640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09e8101de0 0x7f09e819a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:33.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.882+0000 7f09eecba640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f09e8102790 0x7f09e819ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:33.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.882+0000 7f09eecba640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f09e8106f10 0x7f09e819ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:33.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.882+0000 7f09eecba640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f09e8111f70 con 0x7f09e8101de0 2024-09-12T02:09:33.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.882+0000 7f09eecba640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f09e8111df0 con 0x7f09e8102790 2024-09-12T02:09:33.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.882+0000 7f09eecba640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f09e81120f0 con 0x7f09e8106f10 2024-09-12T02:09:33.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.882+0000 7f09eca56640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09e8101de0 0x7f09e819a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:33.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.882+0000 7f09eca56640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09e8101de0 0x7f09e819a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39666/0 (socket says 172.21.15.16:39666) 2024-09-12T02:09:33.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.882+0000 7f09eca56640 1 -- 172.21.15.16:0/3902936669 learned_addr learned my addr 172.21.15.16:0/3902936669 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:09:33.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.882+0000 7f09ed257640 1 --2- 172.21.15.16:0/3902936669 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f09e8106f10 0x7f09e819ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:33.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.883+0000 7f09e7fff640 1 --2- 172.21.15.16:0/3902936669 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f09e8102790 0x7f09e819ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:33.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.883+0000 7f09eca56640 1 -- 172.21.15.16:0/3902936669 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f09e8106f10 msgr2=0x7f09e819ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:33.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.883+0000 7f09eca56640 1 --2- 172.21.15.16:0/3902936669 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f09e8106f10 0x7f09e819ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:33.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.883+0000 7f09eca56640 1 -- 172.21.15.16:0/3902936669 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f09e8102790 msgr2=0x7f09e819ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:33.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.883+0000 7f09eca56640 1 --2- 172.21.15.16:0/3902936669 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f09e8102790 0x7f09e819ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:33.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.883+0000 7f09eca56640 1 -- 172.21.15.16:0/3902936669 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09e819f5e0 con 0x7f09e8101de0 2024-09-12T02:09:33.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.883+0000 7f09e7fff640 1 --2- 172.21.15.16:0/3902936669 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f09e8102790 0x7f09e819ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:09:33.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.883+0000 7f09eca56640 1 --2- 172.21.15.16:0/3902936669 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09e8101de0 0x7f09e819a630 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f09dc00c890 tx=0x7f09dc00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:33.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.883+0000 7f09e5ffb640 1 -- 172.21.15.16:0/3902936669 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09dc004220 con 0x7f09e8101de0 2024-09-12T02:09:33.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.883+0000 7f09eecba640 1 -- 172.21.15.16:0/3902936669 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09e819f8d0 con 0x7f09e8101de0 2024-09-12T02:09:33.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.883+0000 7f09e5ffb640 1 -- 172.21.15.16:0/3902936669 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09dc0043c0 con 0x7f09e8101de0 2024-09-12T02:09:33.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.883+0000 7f09e5ffb640 1 -- 172.21.15.16:0/3902936669 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09dc0049e0 con 0x7f09e8101de0 2024-09-12T02:09:33.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.883+0000 7f09eecba640 1 -- 172.21.15.16:0/3902936669 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09e81a71b0 con 0x7f09e8101de0 2024-09-12T02:09:33.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.885+0000 7f09e5ffb640 1 -- 172.21.15.16:0/3902936669 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f09dc0079b0 con 0x7f09e8101de0 2024-09-12T02:09:33.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.886+0000 7f09f1093740 1 Processor -- start 2024-09-12T02:09:33.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.886+0000 7f09f1093740 1 -- start start 2024-09-12T02:09:33.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.886+0000 7f09f1093740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55babcf74b40 0x55babcf73090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:33.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.886+0000 7f09f1093740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55babcf735d0 0x55babcf73a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:33.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.886+0000 7f09f1093740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55babcf7e370 0x55babcf80760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:33.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.886+0000 7f09f1093740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55babced92f0 con 0x55babcf7e370 2024-09-12T02:09:33.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.886+0000 7f09f1093740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55babce80cf0 con 0x55babcf735d0 2024-09-12T02:09:33.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.886+0000 7f09f1093740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55babced52b0 con 0x55babcf74b40 2024-09-12T02:09:33.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.886+0000 7f09e5ffb640 1 --2- 172.21.15.16:0/3902936669 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f09d80a2c00 0x7f09d80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:33.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.886+0000 7f09e7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55babcf7e370 0x55babcf80760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:33.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.886+0000 7f09ed257640 1 --2- 172.21.15.16:0/3902936669 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f09d80a2c00 0x7f09d80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:33.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.886+0000 7f09eca56640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55babcf735d0 0x55babcf73a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:33.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.886+0000 7f09e7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55babcf7e370 0x55babcf80760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39668/0 (socket says 172.21.15.16:39668) 2024-09-12T02:09:33.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.886+0000 7f09e7fff640 1 -- 172.21.15.16:0/3574844161 learned_addr learned my addr 172.21.15.16:0/3574844161 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:09:33.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.887+0000 7f09ed257640 1 --2- 172.21.15.16:0/3574844161 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55babcf74b40 0x55babcf73090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:33.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.887+0000 7f09e5ffb640 1 -- 172.21.15.16:0/3902936669 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f09dc0cf0f0 con 0x7f09e8101de0 2024-09-12T02:09:33.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.887+0000 7f09e7fff640 1 -- 172.21.15.16:0/3574844161 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55babcf74b40 msgr2=0x55babcf73090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:09:33.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.887+0000 7f09e7fff640 1 --2- 172.21.15.16:0/3574844161 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55babcf74b40 0x55babcf73090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:33.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.887+0000 7f09e7fff640 1 -- 172.21.15.16:0/3574844161 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55babcf735d0 msgr2=0x55babcf73a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:09:33.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.887+0000 7f09e7fff640 1 --2- 172.21.15.16:0/3574844161 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55babcf735d0 0x55babcf73a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:33.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.887+0000 7f09e7fff640 1 -- 172.21.15.16:0/3574844161 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55babce653a0 con 0x55babcf7e370 2024-09-12T02:09:33.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.887+0000 7f09ed257640 1 --2- 172.21.15.16:0/3902936669 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f09d80a2c00 0x7f09d80a50c0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f09e002f540 tx=0x7f09e002f4b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:33.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.887+0000 7f09e7fff640 1 --2- 172.21.15.16:0/3574844161 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55babcf7e370 0x55babcf80760 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f09d400bac0 tx=0x7f09d400bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac16e68b769ac489 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:33.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.887+0000 7f09d2ffd640 1 -- 172.21.15.16:0/3574844161 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09d4017420 con 0x55babcf7e370 2024-09-12T02:09:33.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.887+0000 7f09d2ffd640 1 -- 172.21.15.16:0/3574844161 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09d4017aa0 con 0x55babcf7e370 2024-09-12T02:09:33.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.887+0000 7f09d2ffd640 1 -- 172.21.15.16:0/3574844161 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09d400fa90 con 0x55babcf7e370 2024-09-12T02:09:33.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.888+0000 7f09f1093740 1 -- 172.21.15.16:0/3574844161 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55babcf7e370 msgr2=0x55babcf80760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:33.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.888+0000 7f09f1093740 1 --2- 172.21.15.16:0/3574844161 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55babcf7e370 0x55babcf80760 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f09d400bac0 tx=0x7f09d400bf80 comp rx=0 tx=0).stop 2024-09-12T02:09:33.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.888+0000 7f09f1093740 1 -- 172.21.15.16:0/3574844161 shutdown_connections 2024-09-12T02:09:33.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.888+0000 7f09f1093740 1 --2- 172.21.15.16:0/3574844161 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55babcf7e370 0x55babcf80760 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:33.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.888+0000 7f09f1093740 1 --2- 172.21.15.16:0/3574844161 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55babcf735d0 0x55babcf73a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:33.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.888+0000 7f09f1093740 1 --2- 172.21.15.16:0/3574844161 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55babcf74b40 0x55babcf73090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:33.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.888+0000 7f09f1093740 1 -- 172.21.15.16:0/3574844161 >> 172.21.15.16:0/3574844161 conn(0x55babcf3c310 msgr2=0x55babcf6fc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:33.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.888+0000 7f09f1093740 1 -- 172.21.15.16:0/3574844161 shutdown_connections 2024-09-12T02:09:33.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.888+0000 7f09f1093740 1 -- 172.21.15.16:0/3574844161 wait complete. 2024-09-12T02:09:33.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.888+0000 7f09f1093740 1 -- start start 2024-09-12T02:09:33.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.888+0000 7f09f1093740 1 Processor -- start 2024-09-12T02:09:33.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.889+0000 7f09f1093740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55babcf735d0 0x55babcf7dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:33.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.889+0000 7f09f1093740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55babcf74b40 0x55babcf7af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:33.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.889+0000 7f09f1093740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55babcf7e370 0x55babcf7b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:33.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.889+0000 7f09f1093740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55babcf81070 con 0x55babcf7e370 2024-09-12T02:09:33.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.889+0000 7f09f1093740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55babce94060 con 0x55babcf735d0 2024-09-12T02:09:33.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.889+0000 7f09f1093740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55babcf811f0 con 0x55babcf74b40 2024-09-12T02:09:33.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.889+0000 7f09f1093740 10 client.0 _ll_register_callbacks cb 0x55babcbe78c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:09:33.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.889+0000 7f09ed257640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55babcf7e370 0x55babcf7b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:33.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.889+0000 7f09ed257640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55babcf7e370 0x55babcf7b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39676/0 (socket says 172.21.15.16:39676) 2024-09-12T02:09:33.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.889+0000 7f09ed257640 1 -- 172.21.15.16:0/3774803153 learned_addr learned my addr 172.21.15.16:0/3774803153 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:09:33.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.889+0000 7f09eca56640 1 --2- 172.21.15.16:0/3774803153 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55babcf735d0 0x55babcf7dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:33.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.889+0000 7f09e7fff640 1 --2- 172.21.15.16:0/3774803153 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55babcf74b40 0x55babcf7af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:33.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.889+0000 7f09ed257640 1 -- 172.21.15.16:0/3774803153 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55babcf74b40 msgr2=0x55babcf7af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:09:33.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.889+0000 7f09ed257640 1 --2- 172.21.15.16:0/3774803153 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55babcf74b40 0x55babcf7af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:33.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.889+0000 7f09ed257640 1 -- 172.21.15.16:0/3774803153 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55babcf735d0 msgr2=0x55babcf7dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:09:33.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.889+0000 7f09ed257640 1 --2- 172.21.15.16:0/3774803153 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55babcf735d0 0x55babcf7dcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:33.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.889+0000 7f09ed257640 1 -- 172.21.15.16:0/3774803153 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55babcc4a3c0 con 0x55babcf7e370 2024-09-12T02:09:33.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.890+0000 7f09ed257640 1 --2- 172.21.15.16:0/3774803153 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55babcf7e370 0x55babcf7b4e0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f09e0035000 tx=0x7f09e0047000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4bb269902dc39cd server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:33.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.890+0000 7f09c7fff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:09:33.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.890+0000 7f09c7fff640 1 -- 172.21.15.16:0/3774803153 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09e004f030 con 0x55babcf7e370 2024-09-12T02:09:33.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.890+0000 7f09c7fff640 1 -- 172.21.15.16:0/3774803153 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f09e0046040 con 0x55babcf7e370 2024-09-12T02:09:33.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.890+0000 7f09f1093740 1 -- 172.21.15.16:0/3774803153 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f09d4017aa0 con 0x55babcf7e370 2024-09-12T02:09:33.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.890+0000 7f09c7fff640 1 -- 172.21.15.16:0/3774803153 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f09e0004240 con 0x55babcf7e370 2024-09-12T02:09:33.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.890+0000 7f09f1093740 10 client.17751 fetch_fsmap learned FSMap version 25 2024-09-12T02:09:33.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.890+0000 7f09f1093740 1 -- 172.21.15.16:0/3774803153 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55babcf8ca80 con 0x55babcf7e370 2024-09-12T02:09:33.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.890+0000 7f09c7fff640 1 -- 172.21.15.16:0/3774803153 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09d4017aa0 con 0x55babcf7e370 2024-09-12T02:09:33.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.891+0000 7f09c7fff640 1 -- 172.21.15.16:0/3774803153 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f09e004a070 con 0x55babcf7e370 2024-09-12T02:09:33.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.891+0000 7f09f1093740 10 client.17751 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:09:33.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.891+0000 7f09f1093740 10 client.17751 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-12T02:09:33.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.891+0000 7f09f1093740 20 client.17751 populate_metadata read hostname 'smithi016' 2024-09-12T02:09:33.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.891+0000 7f09f1093740 1 --2- 172.21.15.16:0/3774803153 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55babcf8de20 0x55babcf902c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:33.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.891+0000 7f09f1093740 4 client.17751 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:09:33.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.891+0000 7f09f1093740 1 -- 172.21.15.16:0/3774803153 --> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55babcee6860 con 0x55babcf8de20 2024-09-12T02:09:33.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.891+0000 7f09eca56640 1 --2- 172.21.15.16:0/3774803153 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55babcf8de20 0x55babcf902c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:33.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.891+0000 7f09eca56640 1 --2- 172.21.15.16:0/3774803153 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55babcf8de20 0x55babcf902c0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a3c621ef3aa5d144 server_cookie=f83a1d267dbaf162 in_seq=0 out_seq=0 2024-09-12T02:09:33.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.891+0000 7f09c7fff640 10 client.17751 ms_handle_connect on v2:172.21.15.184:6836/1793393087 2024-09-12T02:09:33.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.892+0000 7f09c7fff640 1 -- 172.21.15.16:0/3774803153 <== mds.2 v2:172.21.15.184:6836/1793393087 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55babcee6860 con 0x55babcf8de20 2024-09-12T02:09:33.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.892+0000 7f09c7fff640 10 client.17751 handle_command_reply: tid=0 2024-09-12T02:09:33.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.892+0000 7f09f1093740 1 client.17751 shutdown 2024-09-12T02:09:33.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.892+0000 7f09f1093740 1 -- 172.21.15.16:0/3774803153 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55babcf7e370 msgr2=0x55babcf7b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:33.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.892+0000 7f09f1093740 1 --2- 172.21.15.16:0/3774803153 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55babcf7e370 0x55babcf7b4e0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f09e0035000 tx=0x7f09e0047000 comp rx=0 tx=0).stop 2024-09-12T02:09:33.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.892+0000 7f09f1093740 1 -- 172.21.15.16:0/3774803153 shutdown_connections 2024-09-12T02:09:33.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.892+0000 7f09f1093740 1 --2- 172.21.15.16:0/3774803153 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55babcf8de20 0x55babcf902c0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:33.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.892+0000 7f09f1093740 1 --2- 172.21.15.16:0/3774803153 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55babcf7e370 0x55babcf7b4e0 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:33.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.892+0000 7f09f1093740 1 --2- 172.21.15.16:0/3774803153 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55babcf74b40 0x55babcf7af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:33.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.892+0000 7f09c7fff640 0 client.17751 ms_handle_reset on v2:172.21.15.184:6836/1793393087 2024-09-12T02:09:33.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.892+0000 7f09f1093740 1 --2- 172.21.15.16:0/3774803153 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55babcf735d0 0x55babcf7dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:33.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.892+0000 7f09f1093740 1 -- 172.21.15.16:0/3774803153 >> 172.21.15.16:0/3774803153 conn(0x55babcf3c310 msgr2=0x55babcf75960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:33.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.892+0000 7f09f1093740 1 -- 172.21.15.16:0/3774803153 shutdown_connections 2024-09-12T02:09:33.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.892+0000 7f09f1093740 1 -- 172.21.15.16:0/3774803153 wait complete. 2024-09-12T02:09:33.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.892+0000 7f09f1093740 20 client.17751 trim_cache size 0 max 16384 2024-09-12T02:09:33.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.905+0000 7f09f1093740 1 Processor -- start 2024-09-12T02:09:33.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.905+0000 7f09f1093740 1 -- start start 2024-09-12T02:09:33.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.906+0000 7f09f1093740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55babcf735d0 0x55babcf6ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:33.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.906+0000 7f09f1093740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55babcf74b40 0x55babcf7f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:33.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.906+0000 7f09f1093740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55babcf7e370 0x55babcf7fca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:33.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.906+0000 7f09f1093740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f09e81b1f10 con 0x55babcf735d0 2024-09-12T02:09:33.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.906+0000 7f09f1093740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f09e81af7e0 con 0x55babcf74b40 2024-09-12T02:09:33.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.906+0000 7f09f1093740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f09e81ad630 con 0x55babcf7e370 2024-09-12T02:09:33.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.906+0000 7f09eca56640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55babcf735d0 0x55babcf6ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:33.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.906+0000 7f09eca56640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55babcf735d0 0x55babcf6ebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39678/0 (socket says 172.21.15.16:39678) 2024-09-12T02:09:33.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.906+0000 7f09eca56640 1 -- 172.21.15.16:0/716687807 learned_addr learned my addr 172.21.15.16:0/716687807 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:09:33.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.906+0000 7f09e7fff640 1 --2- 172.21.15.16:0/716687807 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55babcf74b40 0x55babcf7f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:33.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.906+0000 7f09ed257640 1 --2- 172.21.15.16:0/716687807 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55babcf7e370 0x55babcf7fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:33.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.906+0000 7f09eca56640 1 -- 172.21.15.16:0/716687807 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55babcf7e370 msgr2=0x55babcf7fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:09:33.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.906+0000 7f09eca56640 1 --2- 172.21.15.16:0/716687807 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55babcf7e370 0x55babcf7fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:33.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.906+0000 7f09eca56640 1 -- 172.21.15.16:0/716687807 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55babcf74b40 msgr2=0x55babcf7f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:09:33.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.906+0000 7f09eca56640 1 --2- 172.21.15.16:0/716687807 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55babcf74b40 0x55babcf7f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:33.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.906+0000 7f09eca56640 1 -- 172.21.15.16:0/716687807 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55babcf6fed0 con 0x55babcf735d0 2024-09-12T02:09:33.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.906+0000 7f09e7fff640 1 --2- 172.21.15.16:0/716687807 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55babcf74b40 0x55babcf7f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:09:33.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.906+0000 7f09ed257640 1 --2- 172.21.15.16:0/716687807 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55babcf7e370 0x55babcf7fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:09:33.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.906+0000 7f09eca56640 1 --2- 172.21.15.16:0/716687807 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55babcf735d0 0x55babcf6ebf0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f09dc09b650 tx=0x7f09dc005300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28e01102f97a2118 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:33.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.906+0000 7f09d27fc640 1 -- 172.21.15.16:0/716687807 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09dc01d040 con 0x55babcf735d0 2024-09-12T02:09:33.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.907+0000 7f09d27fc640 1 -- 172.21.15.16:0/716687807 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09dc09fd00 con 0x55babcf735d0 2024-09-12T02:09:33.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.907+0000 7f09d27fc640 1 -- 172.21.15.16:0/716687807 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09dc098150 con 0x55babcf735d0 2024-09-12T02:09:33.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.907+0000 7f09f1093740 1 -- 172.21.15.16:0/716687807 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55babcf735d0 msgr2=0x55babcf6ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:33.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.907+0000 7f09f1093740 1 --2- 172.21.15.16:0/716687807 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55babcf735d0 0x55babcf6ebf0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f09dc09b650 tx=0x7f09dc005300 comp rx=0 tx=0).stop 2024-09-12T02:09:33.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.907+0000 7f09f1093740 1 -- 172.21.15.16:0/716687807 shutdown_connections 2024-09-12T02:09:33.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.907+0000 7f09f1093740 1 --2- 172.21.15.16:0/716687807 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55babcf7e370 0x55babcf7fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:33.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.907+0000 7f09f1093740 1 --2- 172.21.15.16:0/716687807 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55babcf74b40 0x55babcf7f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:33.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.907+0000 7f09f1093740 1 --2- 172.21.15.16:0/716687807 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55babcf735d0 0x55babcf6ebf0 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:33.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.907+0000 7f09f1093740 1 -- 172.21.15.16:0/716687807 >> 172.21.15.16:0/716687807 conn(0x55babcf3c310 msgr2=0x55babcf6e4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:33.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.907+0000 7f09f1093740 1 -- 172.21.15.16:0/716687807 shutdown_connections 2024-09-12T02:09:33.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.907+0000 7f09f1093740 1 -- 172.21.15.16:0/716687807 wait complete. 2024-09-12T02:09:33.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.907+0000 7f09f1093740 1 -- start start 2024-09-12T02:09:33.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.908+0000 7f09f1093740 1 Processor -- start 2024-09-12T02:09:33.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.908+0000 7f09f1093740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55babcf735d0 0x55babcf86cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:33.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.908+0000 7f09f1093740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55babcf74b40 0x55babcf95570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:33.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.908+0000 7f09f1093740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55babcf7e370 0x55babcf95b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:33.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.908+0000 7f09f1093740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f09e81adca0 con 0x55babcf7e370 2024-09-12T02:09:33.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.908+0000 7f09f1093740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f09e81af110 con 0x55babcf74b40 2024-09-12T02:09:33.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.908+0000 7f09f1093740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f09e81ae370 con 0x55babcf735d0 2024-09-12T02:09:33.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.908+0000 7f09f1093740 10 client.0 _ll_register_callbacks cb 0x7f09e0031d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:09:33.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.908+0000 7f09ed257640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55babcf7e370 0x55babcf95b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:33.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.908+0000 7f09ed257640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55babcf7e370 0x55babcf95b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39682/0 (socket says 172.21.15.16:39682) 2024-09-12T02:09:33.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.908+0000 7f09ed257640 1 -- 172.21.15.16:0/3011878372 learned_addr learned my addr 172.21.15.16:0/3011878372 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:09:33.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.908+0000 7f09eca56640 1 --2- 172.21.15.16:0/3011878372 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55babcf735d0 0x55babcf86cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:33.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.908+0000 7f09e7fff640 1 --2- 172.21.15.16:0/3011878372 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55babcf74b40 0x55babcf95570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:33.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.908+0000 7f09ed257640 1 -- 172.21.15.16:0/3011878372 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55babcf735d0 msgr2=0x55babcf86cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:09:33.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.908+0000 7f09ed257640 1 --2- 172.21.15.16:0/3011878372 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55babcf735d0 0x55babcf86cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:33.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.908+0000 7f09ed257640 1 -- 172.21.15.16:0/3011878372 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55babcf74b40 msgr2=0x55babcf95570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:09:33.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.909+0000 7f09ed257640 1 --2- 172.21.15.16:0/3011878372 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55babcf74b40 0x55babcf95570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:33.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.909+0000 7f09ed257640 1 -- 172.21.15.16:0/3011878372 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55babcf96370 con 0x55babcf7e370 2024-09-12T02:09:33.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.909+0000 7f09e7fff640 1 --2- 172.21.15.16:0/3011878372 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55babcf74b40 0x55babcf95570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:09:33.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.909+0000 7f09ed257640 1 --2- 172.21.15.16:0/3011878372 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55babcf7e370 0x55babcf95b40 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f09e0031b50 tx=0x7f09e0031450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=292df7a5ed043586 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:33.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.909+0000 7f09c7fff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:09:33.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.909+0000 7f09c7fff640 1 -- 172.21.15.16:0/3011878372 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09e004f070 con 0x55babcf7e370 2024-09-12T02:09:33.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.909+0000 7f09c7fff640 1 -- 172.21.15.16:0/3011878372 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f09e00045f0 con 0x55babcf7e370 2024-09-12T02:09:33.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.909+0000 7f09f1093740 1 -- 172.21.15.16:0/3011878372 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f09dc09fd00 con 0x55babcf7e370 2024-09-12T02:09:33.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.909+0000 7f09c7fff640 1 -- 172.21.15.16:0/3011878372 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f09e00047f0 con 0x55babcf7e370 2024-09-12T02:09:33.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.909+0000 7f09f1093740 10 client.17757 fetch_fsmap learned FSMap version 25 2024-09-12T02:09:33.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.909+0000 7f09f1093740 1 -- 172.21.15.16:0/3011878372 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55babcff8bc0 con 0x55babcf7e370 2024-09-12T02:09:33.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.909+0000 7f09c7fff640 1 -- 172.21.15.16:0/3011878372 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09dc09fd00 con 0x55babcf7e370 2024-09-12T02:09:33.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.909+0000 7f09c7fff640 1 -- 172.21.15.16:0/3011878372 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f09e0030910 con 0x55babcf7e370 2024-09-12T02:09:33.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.909+0000 7f09f1093740 10 client.17757 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:09:33.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.909+0000 7f09f1093740 10 client.17757 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-12T02:09:33.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.910+0000 7f09f1093740 20 client.17757 populate_metadata read hostname 'smithi016' 2024-09-12T02:09:33.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.910+0000 7f09f1093740 1 --2- 172.21.15.16:0/3011878372 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55babcff9290 0x55babcffb680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:33.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.910+0000 7f09f1093740 4 client.17757 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T02:09:33.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.910+0000 7f09f1093740 1 -- 172.21.15.16:0/3011878372 --> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55babcee0d70 con 0x55babcff9290 2024-09-12T02:09:33.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.910+0000 7f09eca56640 1 --2- 172.21.15.16:0/3011878372 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55babcff9290 0x55babcffb680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:33.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.910+0000 7f09c7fff640 10 client.17757 ms_handle_connect on v2:172.21.15.184:6836/1793393087 2024-09-12T02:09:33.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.910+0000 7f09eca56640 1 --2- 172.21.15.16:0/3011878372 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55babcff9290 0x55babcffb680 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f46dde73e09b7664 server_cookie=8ae01272ce384697 in_seq=0 out_seq=0 2024-09-12T02:09:33.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.910+0000 7f09c7fff640 1 -- 172.21.15.16:0/3011878372 <== mds.2 v2:172.21.15.184:6836/1793393087 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55babcee0d70 con 0x55babcff9290 2024-09-12T02:09:33.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.910+0000 7f09c7fff640 10 client.17757 handle_command_reply: tid=0 2024-09-12T02:09:33.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.910+0000 7f09f1093740 1 client.17757 shutdown 2024-09-12T02:09:33.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.910+0000 7f09f1093740 1 -- 172.21.15.16:0/3011878372 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55babcf7e370 msgr2=0x55babcf95b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:33.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.910+0000 7f09f1093740 1 --2- 172.21.15.16:0/3011878372 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55babcf7e370 0x55babcf95b40 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f09e0031b50 tx=0x7f09e0031450 comp rx=0 tx=0).stop 2024-09-12T02:09:33.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.911+0000 7f09f1093740 1 -- 172.21.15.16:0/3011878372 shutdown_connections 2024-09-12T02:09:33.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.911+0000 7f09f1093740 1 --2- 172.21.15.16:0/3011878372 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55babcff9290 0x55babcffb680 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:33.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.911+0000 7f09f1093740 1 --2- 172.21.15.16:0/3011878372 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55babcf7e370 0x55babcf95b40 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:33.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.911+0000 7f09f1093740 1 --2- 172.21.15.16:0/3011878372 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55babcf74b40 0x55babcf95570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:33.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.911+0000 7f09f1093740 1 --2- 172.21.15.16:0/3011878372 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55babcf735d0 0x55babcf86cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:33.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.911+0000 7f09f1093740 1 -- 172.21.15.16:0/3011878372 >> 172.21.15.16:0/3011878372 conn(0x55babcf3c310 msgr2=0x55babcf7d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:33.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.911+0000 7f09c7fff640 0 client.17757 ms_handle_reset on v2:172.21.15.184:6836/1793393087 2024-09-12T02:09:33.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.911+0000 7f09f1093740 1 -- 172.21.15.16:0/3011878372 shutdown_connections 2024-09-12T02:09:33.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.911+0000 7f09f1093740 1 -- 172.21.15.16:0/3011878372 wait complete. 2024-09-12T02:09:33.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.911+0000 7f09f1093740 20 client.17757 trim_cache size 0 max 16384 2024-09-12T02:09:33.919 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T02:09:33.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.911+0000 7f09eecba640 1 -- 172.21.15.16:0/3902936669 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f09d80a2c00 msgr2=0x7f09d80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:33.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.911+0000 7f09eecba640 1 --2- 172.21.15.16:0/3902936669 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f09d80a2c00 0x7f09d80a50c0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f09e002f540 tx=0x7f09e002f4b0 comp rx=0 tx=0).stop 2024-09-12T02:09:33.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.911+0000 7f09eecba640 1 -- 172.21.15.16:0/3902936669 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09e8101de0 msgr2=0x7f09e819a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:33.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.911+0000 7f09eecba640 1 --2- 172.21.15.16:0/3902936669 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09e8101de0 0x7f09e819a630 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f09dc00c890 tx=0x7f09dc00cd50 comp rx=0 tx=0).stop 2024-09-12T02:09:33.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.912+0000 7f09eecba640 1 -- 172.21.15.16:0/3902936669 shutdown_connections 2024-09-12T02:09:33.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.912+0000 7f09eecba640 1 --2- 172.21.15.16:0/3902936669 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f09d80a2c00 0x7f09d80a50c0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:33.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.912+0000 7f09eecba640 1 --2- 172.21.15.16:0/3902936669 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f09e8106f10 0x7f09e819ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:33.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.912+0000 7f09eecba640 1 --2- 172.21.15.16:0/3902936669 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f09e8102790 0x7f09e819ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:33.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.912+0000 7f09eecba640 1 --2- 172.21.15.16:0/3902936669 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09e8101de0 0x7f09e819a630 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:33.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.912+0000 7f09eecba640 1 -- 172.21.15.16:0/3902936669 >> 172.21.15.16:0/3902936669 conn(0x7f09e80fd8c0 msgr2=0x7f09e80feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:33.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.912+0000 7f09eecba640 1 -- 172.21.15.16:0/3902936669 shutdown_connections 2024-09-12T02:09:33.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:33.912+0000 7f09eecba640 1 -- 172.21.15.16:0/3902936669 wait complete. 2024-09-12T02:09:33.921 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:09:33 smithi033 ceph-mon[35332]: pgmap v564: 97 pgs: 97 active+clean; 4.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-09-12T02:09:34.650 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-12T02:09:34.909 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:09:35.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.365+0000 7f17c1afd640 1 -- 172.21.15.16:0/3006072770 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17bc102790 msgr2=0x7f17bc1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:35.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.365+0000 7f17c1afd640 1 --2- 172.21.15.16:0/3006072770 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17bc102790 0x7f17bc1067e0 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f17a4009900 tx=0x7f17a402a3f0 comp rx=0 tx=0).stop 2024-09-12T02:09:35.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.366+0000 7f17c1afd640 1 -- 172.21.15.16:0/3006072770 shutdown_connections 2024-09-12T02:09:35.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.366+0000 7f17c1afd640 1 --2- 172.21.15.16:0/3006072770 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f17bc106f10 0x7f17bc10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:35.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.366+0000 7f17c1afd640 1 --2- 172.21.15.16:0/3006072770 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17bc102790 0x7f17bc1067e0 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:35.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.366+0000 7f17c1afd640 1 --2- 172.21.15.16:0/3006072770 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f17bc101de0 0x7f17bc1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:35.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.366+0000 7f17c1afd640 1 -- 172.21.15.16:0/3006072770 >> 172.21.15.16:0/3006072770 conn(0x7f17bc0fd8c0 msgr2=0x7f17bc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:35.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.366+0000 7f17c1afd640 1 -- 172.21.15.16:0/3006072770 shutdown_connections 2024-09-12T02:09:35.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.366+0000 7f17c1afd640 1 -- 172.21.15.16:0/3006072770 wait complete. 2024-09-12T02:09:35.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.366+0000 7f17c1afd640 1 Processor -- start 2024-09-12T02:09:35.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.366+0000 7f17c1afd640 1 -- start start 2024-09-12T02:09:35.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.367+0000 7f17c1afd640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f17bc101de0 0x7f17bc19a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:35.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.367+0000 7f17c1afd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f17bc102790 0x7f17bc19abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:35.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.367+0000 7f17c1afd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17bc106f10 0x7f17bc19ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:35.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.367+0000 7f17c1afd640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f17bc111f90 con 0x7f17bc106f10 2024-09-12T02:09:35.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.367+0000 7f17c1afd640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f17bc111e10 con 0x7f17bc102790 2024-09-12T02:09:35.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.367+0000 7f17c1afd640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f17bc112110 con 0x7f17bc101de0 2024-09-12T02:09:35.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.367+0000 7f17bbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17bc106f10 0x7f17bc19ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:35.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.367+0000 7f17bbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17bc106f10 0x7f17bc19ef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39700/0 (socket says 172.21.15.16:39700) 2024-09-12T02:09:35.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.367+0000 7f17bbfff640 1 -- 172.21.15.16:0/303522341 learned_addr learned my addr 172.21.15.16:0/303522341 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:09:35.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.367+0000 7f17bb7fe640 1 --2- 172.21.15.16:0/303522341 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f17bc101de0 0x7f17bc19a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:35.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.367+0000 7f17b2ffd640 1 --2- 172.21.15.16:0/303522341 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f17bc102790 0x7f17bc19abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:35.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.367+0000 7f17bbfff640 1 -- 172.21.15.16:0/303522341 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f17bc101de0 msgr2=0x7f17bc19a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:35.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.367+0000 7f17bbfff640 1 --2- 172.21.15.16:0/303522341 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f17bc101de0 0x7f17bc19a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:35.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.367+0000 7f17bbfff640 1 -- 172.21.15.16:0/303522341 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f17bc102790 msgr2=0x7f17bc19abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:35.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.367+0000 7f17bbfff640 1 --2- 172.21.15.16:0/303522341 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f17bc102790 0x7f17bc19abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:35.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.367+0000 7f17bbfff640 1 -- 172.21.15.16:0/303522341 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17bc19f660 con 0x7f17bc106f10 2024-09-12T02:09:35.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.367+0000 7f17bbfff640 1 --2- 172.21.15.16:0/303522341 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17bc106f10 0x7f17bc19ef80 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f17b400b970 tx=0x7f17b400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:35.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.367+0000 7f17b97fa640 1 -- 172.21.15.16:0/303522341 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17b4019070 con 0x7f17bc106f10 2024-09-12T02:09:35.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.367+0000 7f17b97fa640 1 -- 172.21.15.16:0/303522341 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17b400cb40 con 0x7f17bc106f10 2024-09-12T02:09:35.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.367+0000 7f17c1afd640 1 -- 172.21.15.16:0/303522341 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17bc19f950 con 0x7f17bc106f10 2024-09-12T02:09:35.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.368+0000 7f17b97fa640 1 -- 172.21.15.16:0/303522341 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17b4007650 con 0x7f17bc106f10 2024-09-12T02:09:35.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.368+0000 7f17c1afd640 1 -- 172.21.15.16:0/303522341 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17bc1a7230 con 0x7f17bc106f10 2024-09-12T02:09:35.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.370+0000 7f17b97fa640 1 -- 172.21.15.16:0/303522341 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f17b401c070 con 0x7f17bc106f10 2024-09-12T02:09:35.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.370+0000 7f17c3ed6740 1 Processor -- start 2024-09-12T02:09:35.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.370+0000 7f17c3ed6740 1 -- start start 2024-09-12T02:09:35.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.371+0000 7f17c3ed6740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f01115ab40 0x55f011159090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:35.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.371+0000 7f17c3ed6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f0111595d0 0x55f011159a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:35.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.371+0000 7f17c3ed6740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f011164370 0x55f011166760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:35.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.371+0000 7f17c3ed6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55f0110bf2f0 con 0x55f0111595d0 2024-09-12T02:09:35.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.371+0000 7f17c3ed6740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55f011066cf0 con 0x55f011164370 2024-09-12T02:09:35.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.371+0000 7f17c3ed6740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55f0110bb2b0 con 0x55f01115ab40 2024-09-12T02:09:35.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.371+0000 7f17bb7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f0111595d0 0x55f011159a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:35.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.371+0000 7f17bb7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f0111595d0 0x55f011159a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39708/0 (socket says 172.21.15.16:39708) 2024-09-12T02:09:35.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.371+0000 7f17bb7fe640 1 -- 172.21.15.16:0/1397056341 learned_addr learned my addr 172.21.15.16:0/1397056341 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:09:35.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.371+0000 7f17b2ffd640 1 --2- 172.21.15.16:0/1397056341 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f01115ab40 0x55f011159090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:35.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.371+0000 7f17b2ffd640 1 --2- 172.21.15.16:0/1397056341 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f011164370 0x55f011166760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:35.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.371+0000 7f17b97fa640 1 --2- 172.21.15.16:0/303522341 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f17a80a2b80 0x7f17a80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:35.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.371+0000 7f17bb7fe640 1 -- 172.21.15.16:0/1397056341 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f01115ab40 msgr2=0x55f011159090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:09:35.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.371+0000 7f17bb7fe640 1 --2- 172.21.15.16:0/1397056341 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f01115ab40 0x55f011159090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:35.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.371+0000 7f17bb7fe640 1 -- 172.21.15.16:0/1397056341 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f011164370 msgr2=0x55f011166760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:09:35.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.371+0000 7f17bb7fe640 1 --2- 172.21.15.16:0/1397056341 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f011164370 0x55f011166760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:35.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.371+0000 7f17bb7fe640 1 -- 172.21.15.16:0/1397056341 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f01104b3a0 con 0x55f0111595d0 2024-09-12T02:09:35.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.371+0000 7f17b2ffd640 1 --2- 172.21.15.16:0/1397056341 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f011164370 0x55f011166760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:09:35.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.371+0000 7f17bbfff640 1 --2- 172.21.15.16:0/303522341 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f17a80a2b80 0x7f17a80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:35.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.371+0000 7f17bb7fe640 1 --2- 172.21.15.16:0/1397056341 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f0111595d0 0x55f011159a10 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f17ac00cc60 tx=0x7f17ac00e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52a9d460ccc6626e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:35.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.371+0000 7f17b97fa640 1 -- 172.21.15.16:0/303522341 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f17b40ce650 con 0x7f17bc106f10 2024-09-12T02:09:35.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.372+0000 7f17b1ffb640 1 -- 172.21.15.16:0/1397056341 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17ac014070 con 0x55f0111595d0 2024-09-12T02:09:35.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.372+0000 7f17b1ffb640 1 -- 172.21.15.16:0/1397056341 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17ac0047a0 con 0x55f0111595d0 2024-09-12T02:09:35.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.372+0000 7f17b1ffb640 1 -- 172.21.15.16:0/1397056341 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17ac010720 con 0x55f0111595d0 2024-09-12T02:09:35.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.372+0000 7f17c3ed6740 1 -- 172.21.15.16:0/1397056341 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f0111595d0 msgr2=0x55f011159a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:35.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.372+0000 7f17bbfff640 1 --2- 172.21.15.16:0/303522341 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f17a80a2b80 0x7f17a80a5040 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f17b409ae40 tx=0x7f17b406df70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:35.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.372+0000 7f17c3ed6740 1 --2- 172.21.15.16:0/1397056341 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f0111595d0 0x55f011159a10 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f17ac00cc60 tx=0x7f17ac00e720 comp rx=0 tx=0).stop 2024-09-12T02:09:35.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.372+0000 7f17c3ed6740 1 -- 172.21.15.16:0/1397056341 shutdown_connections 2024-09-12T02:09:35.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.372+0000 7f17c3ed6740 1 --2- 172.21.15.16:0/1397056341 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f011164370 0x55f011166760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:35.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.372+0000 7f17c3ed6740 1 --2- 172.21.15.16:0/1397056341 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f0111595d0 0x55f011159a10 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:35.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.372+0000 7f17c3ed6740 1 --2- 172.21.15.16:0/1397056341 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f01115ab40 0x55f011159090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:35.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.372+0000 7f17c3ed6740 1 -- 172.21.15.16:0/1397056341 >> 172.21.15.16:0/1397056341 conn(0x55f011122310 msgr2=0x55f011155c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:35.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.373+0000 7f17c3ed6740 1 -- 172.21.15.16:0/1397056341 shutdown_connections 2024-09-12T02:09:35.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.373+0000 7f17c3ed6740 1 -- 172.21.15.16:0/1397056341 wait complete. 2024-09-12T02:09:35.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.373+0000 7f17c3ed6740 1 -- start start 2024-09-12T02:09:35.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.373+0000 7f17c3ed6740 1 Processor -- start 2024-09-12T02:09:35.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.373+0000 7f17c3ed6740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f0111595d0 0x55f011163da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:35.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.373+0000 7f17c3ed6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f01115ab40 0x55f011161050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:35.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.373+0000 7f17c3ed6740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f011164370 0x55f0111617e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:35.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.373+0000 7f17c3ed6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55f011167070 con 0x55f01115ab40 2024-09-12T02:09:35.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.373+0000 7f17c3ed6740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55f01107a060 con 0x55f011164370 2024-09-12T02:09:35.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.373+0000 7f17c3ed6740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55f0111671f0 con 0x55f0111595d0 2024-09-12T02:09:35.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.373+0000 7f17c3ed6740 10 client.0 _ll_register_callbacks cb 0x55f010dcd8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:09:35.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.374+0000 7f17bb7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f01115ab40 0x55f011161050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:35.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.374+0000 7f17bb7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f01115ab40 0x55f011161050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39718/0 (socket says 172.21.15.16:39718) 2024-09-12T02:09:35.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.374+0000 7f17bb7fe640 1 -- 172.21.15.16:0/1954884192 learned_addr learned my addr 172.21.15.16:0/1954884192 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:09:35.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.374+0000 7f17b2ffd640 1 --2- 172.21.15.16:0/1954884192 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f0111595d0 0x55f011163da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:35.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.374+0000 7f17b2ffd640 1 --2- 172.21.15.16:0/1954884192 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f011164370 0x55f0111617e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:35.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.374+0000 7f17bb7fe640 1 -- 172.21.15.16:0/1954884192 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f0111595d0 msgr2=0x55f011163da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:09:35.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.374+0000 7f17bb7fe640 1 --2- 172.21.15.16:0/1954884192 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f0111595d0 0x55f011163da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:35.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.374+0000 7f17bb7fe640 1 -- 172.21.15.16:0/1954884192 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f011164370 msgr2=0x55f0111617e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:09:35.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.374+0000 7f17bb7fe640 1 --2- 172.21.15.16:0/1954884192 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f011164370 0x55f0111617e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:35.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.374+0000 7f17bb7fe640 1 -- 172.21.15.16:0/1954884192 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f010e303c0 con 0x55f01115ab40 2024-09-12T02:09:35.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.374+0000 7f17b2ffd640 1 --2- 172.21.15.16:0/1954884192 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f011164370 0x55f0111617e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:09:35.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.374+0000 7f17bb7fe640 1 --2- 172.21.15.16:0/1954884192 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f01115ab40 0x55f011161050 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f17ac00cc30 tx=0x7f17ac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13f5cf0d8e021887 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:35.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.374+0000 7f179affd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:09:35.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.374+0000 7f179affd640 1 -- 172.21.15.16:0/1954884192 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17ac014070 con 0x55f01115ab40 2024-09-12T02:09:35.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.374+0000 7f179affd640 1 -- 172.21.15.16:0/1954884192 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f17ac005210 con 0x55f01115ab40 2024-09-12T02:09:35.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.374+0000 7f17c3ed6740 1 -- 172.21.15.16:0/1954884192 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17ac0047a0 con 0x55f01115ab40 2024-09-12T02:09:35.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.375+0000 7f179affd640 1 -- 172.21.15.16:0/1954884192 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f17ac005410 con 0x55f01115ab40 2024-09-12T02:09:35.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.375+0000 7f17c3ed6740 10 client.17769 fetch_fsmap learned FSMap version 25 2024-09-12T02:09:35.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.375+0000 7f17c3ed6740 1 -- 172.21.15.16:0/1954884192 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f011172b50 con 0x55f01115ab40 2024-09-12T02:09:35.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.375+0000 7f179affd640 1 -- 172.21.15.16:0/1954884192 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17ac0047a0 con 0x55f01115ab40 2024-09-12T02:09:35.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.375+0000 7f179affd640 1 -- 172.21.15.16:0/1954884192 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f17ac023540 con 0x55f01115ab40 2024-09-12T02:09:35.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.375+0000 7f17c3ed6740 10 client.17769 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:09:35.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.375+0000 7f17c3ed6740 10 client.17769 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T02:09:35.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.375+0000 7f17c3ed6740 20 client.17769 populate_metadata read hostname 'smithi016' 2024-09-12T02:09:35.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.375+0000 7f17c3ed6740 1 --2- 172.21.15.16:0/1954884192 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x55f011173ea0 0x55f011176290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:35.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.375+0000 7f17c3ed6740 4 client.17769 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:09:35.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.375+0000 7f17c3ed6740 1 -- 172.21.15.16:0/1954884192 --> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f0110cc860 con 0x55f011173ea0 2024-09-12T02:09:35.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.375+0000 7f17b2ffd640 1 --2- 172.21.15.16:0/1954884192 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x55f011173ea0 0x55f011176290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:35.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.376+0000 7f17b2ffd640 1 --2- 172.21.15.16:0/1954884192 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x55f011173ea0 0x55f011176290 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b70047e88ae38b9c server_cookie=39aaedcea31a694a in_seq=0 out_seq=0 2024-09-12T02:09:35.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.376+0000 7f179affd640 10 client.17769 ms_handle_connect on v2:172.21.15.184:6838/4203347941 2024-09-12T02:09:35.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.377+0000 7f179affd640 1 -- 172.21.15.16:0/1954884192 <== mds.1 v2:172.21.15.184:6838/4203347941 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f0110cc860 con 0x55f011173ea0 2024-09-12T02:09:35.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.377+0000 7f179affd640 10 client.17769 handle_command_reply: tid=0 2024-09-12T02:09:35.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.377+0000 7f17c3ed6740 1 client.17769 shutdown 2024-09-12T02:09:35.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.377+0000 7f17c3ed6740 1 -- 172.21.15.16:0/1954884192 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f01115ab40 msgr2=0x55f011161050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:35.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.377+0000 7f17c3ed6740 1 --2- 172.21.15.16:0/1954884192 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f01115ab40 0x55f011161050 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f17ac00cc30 tx=0x7f17ac0046c0 comp rx=0 tx=0).stop 2024-09-12T02:09:35.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.377+0000 7f17c3ed6740 1 -- 172.21.15.16:0/1954884192 shutdown_connections 2024-09-12T02:09:35.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.377+0000 7f17c3ed6740 1 --2- 172.21.15.16:0/1954884192 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x55f011173ea0 0x55f011176290 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:35.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.377+0000 7f179affd640 0 client.17769 ms_handle_reset on v2:172.21.15.184:6838/4203347941 2024-09-12T02:09:35.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.377+0000 7f17c3ed6740 1 --2- 172.21.15.16:0/1954884192 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f011164370 0x55f0111617e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:35.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.377+0000 7f17c3ed6740 1 --2- 172.21.15.16:0/1954884192 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f01115ab40 0x55f011161050 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:35.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.377+0000 7f17c3ed6740 1 --2- 172.21.15.16:0/1954884192 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f0111595d0 0x55f011163da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:35.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.377+0000 7f17c3ed6740 1 -- 172.21.15.16:0/1954884192 >> 172.21.15.16:0/1954884192 conn(0x55f011122310 msgr2=0x55f01115b960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:35.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.377+0000 7f17c3ed6740 1 -- 172.21.15.16:0/1954884192 shutdown_connections 2024-09-12T02:09:35.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.377+0000 7f17c3ed6740 1 -- 172.21.15.16:0/1954884192 wait complete. 2024-09-12T02:09:35.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.377+0000 7f17c3ed6740 20 client.17769 trim_cache size 0 max 16384 2024-09-12T02:09:35.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.390+0000 7f17c3ed6740 1 Processor -- start 2024-09-12T02:09:35.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.390+0000 7f17c3ed6740 1 -- start start 2024-09-12T02:09:35.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.390+0000 7f17c3ed6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f0111595d0 0x55f01115c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:35.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.391+0000 7f17c3ed6740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f01115ab40 0x55f01115c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:35.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.391+0000 7f17c3ed6740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f011164370 0x55f01117ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:35.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.391+0000 7f17c3ed6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f17bc1b1fe0 con 0x55f0111595d0 2024-09-12T02:09:35.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.391+0000 7f17c3ed6740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f17bc1af8b0 con 0x55f011164370 2024-09-12T02:09:35.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.391+0000 7f17c3ed6740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f17bc1ad700 con 0x55f01115ab40 2024-09-12T02:09:35.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.391+0000 7f17b2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f0111595d0 0x55f01115c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:35.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.391+0000 7f17b2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f0111595d0 0x55f01115c120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39734/0 (socket says 172.21.15.16:39734) 2024-09-12T02:09:35.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.391+0000 7f17b2ffd640 1 -- 172.21.15.16:0/487679570 learned_addr learned my addr 172.21.15.16:0/487679570 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:09:35.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.391+0000 7f17bb7fe640 1 --2- 172.21.15.16:0/487679570 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f01115ab40 0x55f01115c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:35.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.391+0000 7f17b2ffd640 1 --2- 172.21.15.16:0/487679570 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f011164370 0x55f01117ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:35.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.391+0000 7f17b2ffd640 1 -- 172.21.15.16:0/487679570 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f01115ab40 msgr2=0x55f01115c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:09:35.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.391+0000 7f17b2ffd640 1 --2- 172.21.15.16:0/487679570 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f01115ab40 0x55f01115c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:35.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.391+0000 7f17b2ffd640 1 -- 172.21.15.16:0/487679570 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f011164370 msgr2=0x55f01117ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:09:35.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.391+0000 7f17b2ffd640 1 --2- 172.21.15.16:0/487679570 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f011164370 0x55f01117ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:35.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.391+0000 7f17b2ffd640 1 -- 172.21.15.16:0/487679570 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f011156960 con 0x55f0111595d0 2024-09-12T02:09:35.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.391+0000 7f17bb7fe640 1 --2- 172.21.15.16:0/487679570 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f01115ab40 0x55f01115c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:09:35.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.391+0000 7f17b2ffd640 1 --2- 172.21.15.16:0/487679570 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f0111595d0 0x55f01115c120 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f17a402aac0 tx=0x7f17a4009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b14724a6a66df1e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:35.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.391+0000 7f17b17fa640 1 -- 172.21.15.16:0/487679570 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17a4035040 con 0x55f0111595d0 2024-09-12T02:09:35.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.391+0000 7f17b17fa640 1 -- 172.21.15.16:0/487679570 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17a4005500 con 0x55f0111595d0 2024-09-12T02:09:35.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.391+0000 7f17b17fa640 1 -- 172.21.15.16:0/487679570 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17a40035f0 con 0x55f0111595d0 2024-09-12T02:09:35.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.392+0000 7f17c3ed6740 1 -- 172.21.15.16:0/487679570 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f0111595d0 msgr2=0x55f01115c120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:35.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.392+0000 7f17c3ed6740 1 --2- 172.21.15.16:0/487679570 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f0111595d0 0x55f01115c120 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f17a402aac0 tx=0x7f17a4009260 comp rx=0 tx=0).stop 2024-09-12T02:09:35.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.392+0000 7f17c3ed6740 1 -- 172.21.15.16:0/487679570 shutdown_connections 2024-09-12T02:09:35.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.392+0000 7f17c3ed6740 1 --2- 172.21.15.16:0/487679570 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f011164370 0x55f01117ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:35.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.392+0000 7f17c3ed6740 1 --2- 172.21.15.16:0/487679570 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f01115ab40 0x55f01115c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:35.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.392+0000 7f17c3ed6740 1 --2- 172.21.15.16:0/487679570 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f0111595d0 0x55f01115c120 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:35.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.392+0000 7f17c3ed6740 1 -- 172.21.15.16:0/487679570 >> 172.21.15.16:0/487679570 conn(0x55f011122310 msgr2=0x55f0111630d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:35.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.392+0000 7f17c3ed6740 1 -- 172.21.15.16:0/487679570 shutdown_connections 2024-09-12T02:09:35.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.392+0000 7f17c3ed6740 1 -- 172.21.15.16:0/487679570 wait complete. 2024-09-12T02:09:35.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.392+0000 7f17c3ed6740 1 -- start start 2024-09-12T02:09:35.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.393+0000 7f17c3ed6740 1 Processor -- start 2024-09-12T02:09:35.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.393+0000 7f17c3ed6740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f0111595d0 0x55f01116ca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:35.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.393+0000 7f17c3ed6740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f01115ab40 0x55f01115dc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:35.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.393+0000 7f17c3ed6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f011164370 0x55f01115e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:35.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.393+0000 7f17c3ed6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f17bc1add70 con 0x55f011164370 2024-09-12T02:09:35.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.393+0000 7f17c3ed6740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f17bc1af1e0 con 0x55f01115ab40 2024-09-12T02:09:35.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.393+0000 7f17c3ed6740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f17bc1ae440 con 0x55f0111595d0 2024-09-12T02:09:35.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.393+0000 7f17c3ed6740 10 client.0 _ll_register_callbacks cb 0x7f17a4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:09:35.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.393+0000 7f17b2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f011164370 0x55f01115e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:35.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.393+0000 7f17b2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f011164370 0x55f01115e150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39736/0 (socket says 172.21.15.16:39736) 2024-09-12T02:09:35.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.393+0000 7f17b2ffd640 1 -- 172.21.15.16:0/2376350908 learned_addr learned my addr 172.21.15.16:0/2376350908 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:09:35.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.393+0000 7f17b2ffd640 1 --2- 172.21.15.16:0/2376350908 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f0111595d0 0x55f01116ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:35.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.393+0000 7f17bb7fe640 1 --2- 172.21.15.16:0/2376350908 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f01115ab40 0x55f01115dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:35.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.393+0000 7f17b2ffd640 1 -- 172.21.15.16:0/2376350908 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f0111595d0 msgr2=0x55f01116ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:09:35.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.393+0000 7f17b2ffd640 1 --2- 172.21.15.16:0/2376350908 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f0111595d0 0x55f01116ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:35.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.393+0000 7f17b2ffd640 1 -- 172.21.15.16:0/2376350908 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f01115ab40 msgr2=0x55f01115dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:09:35.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.393+0000 7f17b2ffd640 1 --2- 172.21.15.16:0/2376350908 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f01115ab40 0x55f01115dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:35.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.393+0000 7f17b2ffd640 1 -- 172.21.15.16:0/2376350908 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f01115e7f0 con 0x55f011164370 2024-09-12T02:09:35.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.394+0000 7f17b2ffd640 1 --2- 172.21.15.16:0/2376350908 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f011164370 0x55f01115e150 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f17a4005910 tx=0x7f17a4004b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad2a3a48aa0ab145 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:35.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.394+0000 7f179affd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:09:35.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.394+0000 7f179affd640 1 -- 172.21.15.16:0/2376350908 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17a4057070 con 0x55f011164370 2024-09-12T02:09:35.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.394+0000 7f179affd640 1 -- 172.21.15.16:0/2376350908 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f17a40042d0 con 0x55f011164370 2024-09-12T02:09:35.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.394+0000 7f17c3ed6740 1 -- 172.21.15.16:0/2376350908 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17a4005500 con 0x55f011164370 2024-09-12T02:09:35.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.394+0000 7f179affd640 1 -- 172.21.15.16:0/2376350908 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f17a4035030 con 0x55f011164370 2024-09-12T02:09:35.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.394+0000 7f17c3ed6740 10 client.17775 fetch_fsmap learned FSMap version 25 2024-09-12T02:09:35.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.394+0000 7f17c3ed6740 1 -- 172.21.15.16:0/2376350908 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f0111de8d0 con 0x55f011164370 2024-09-12T02:09:35.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.394+0000 7f179affd640 1 -- 172.21.15.16:0/2376350908 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17a4005500 con 0x55f011164370 2024-09-12T02:09:35.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.394+0000 7f179affd640 1 -- 172.21.15.16:0/2376350908 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f17a403a040 con 0x55f011164370 2024-09-12T02:09:35.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.394+0000 7f17c3ed6740 10 client.17775 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:09:35.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.394+0000 7f17c3ed6740 10 client.17775 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T02:09:35.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.394+0000 7f17c3ed6740 20 client.17775 populate_metadata read hostname 'smithi016' 2024-09-12T02:09:35.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.394+0000 7f17c3ed6740 1 --2- 172.21.15.16:0/2376350908 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x55f011173ea0 0x55f0111e0f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:35.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.394+0000 7f17c3ed6740 4 client.17775 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T02:09:35.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.394+0000 7f17c3ed6740 1 -- 172.21.15.16:0/2376350908 --> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f0110c6d70 con 0x55f011173ea0 2024-09-12T02:09:35.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.395+0000 7f17bb7fe640 1 --2- 172.21.15.16:0/2376350908 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x55f011173ea0 0x55f0111e0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:35.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.395+0000 7f17bb7fe640 1 --2- 172.21.15.16:0/2376350908 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x55f011173ea0 0x55f0111e0f60 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=be267dcca983147f server_cookie=64ee47e89bae727b in_seq=0 out_seq=0 2024-09-12T02:09:35.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.395+0000 7f179affd640 10 client.17775 ms_handle_connect on v2:172.21.15.184:6838/4203347941 2024-09-12T02:09:35.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.395+0000 7f179affd640 1 -- 172.21.15.16:0/2376350908 <== mds.1 v2:172.21.15.184:6838/4203347941 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f0110c6d70 con 0x55f011173ea0 2024-09-12T02:09:35.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.395+0000 7f179affd640 10 client.17775 handle_command_reply: tid=0 2024-09-12T02:09:35.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.395+0000 7f17c3ed6740 1 client.17775 shutdown 2024-09-12T02:09:35.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.396+0000 7f17c3ed6740 1 -- 172.21.15.16:0/2376350908 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f011164370 msgr2=0x55f01115e150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:35.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.396+0000 7f17c3ed6740 1 --2- 172.21.15.16:0/2376350908 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f011164370 0x55f01115e150 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f17a4005910 tx=0x7f17a4004b30 comp rx=0 tx=0).stop 2024-09-12T02:09:35.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.396+0000 7f17c3ed6740 1 -- 172.21.15.16:0/2376350908 shutdown_connections 2024-09-12T02:09:35.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.396+0000 7f17c3ed6740 1 --2- 172.21.15.16:0/2376350908 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x55f011173ea0 0x55f0111e0f60 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:35.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.396+0000 7f17c3ed6740 1 --2- 172.21.15.16:0/2376350908 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f011164370 0x55f01115e150 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:35.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.396+0000 7f17c3ed6740 1 --2- 172.21.15.16:0/2376350908 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f01115ab40 0x55f01115dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:35.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.396+0000 7f17c3ed6740 1 --2- 172.21.15.16:0/2376350908 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f0111595d0 0x55f01116ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:35.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.396+0000 7f17c3ed6740 1 -- 172.21.15.16:0/2376350908 >> 172.21.15.16:0/2376350908 conn(0x55f011122310 msgr2=0x55f01115f7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:35.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.396+0000 7f179affd640 0 client.17775 ms_handle_reset on v2:172.21.15.184:6838/4203347941 2024-09-12T02:09:35.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.396+0000 7f17c3ed6740 1 -- 172.21.15.16:0/2376350908 shutdown_connections 2024-09-12T02:09:35.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.396+0000 7f17c3ed6740 1 -- 172.21.15.16:0/2376350908 wait complete. 2024-09-12T02:09:35.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.396+0000 7f17c3ed6740 20 client.17775 trim_cache size 0 max 16384 2024-09-12T02:09:35.402 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T02:09:35.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.396+0000 7f17c1afd640 1 -- 172.21.15.16:0/303522341 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f17a80a2b80 msgr2=0x7f17a80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:35.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.396+0000 7f17c1afd640 1 --2- 172.21.15.16:0/303522341 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f17a80a2b80 0x7f17a80a5040 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f17b409ae40 tx=0x7f17b406df70 comp rx=0 tx=0).stop 2024-09-12T02:09:35.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.396+0000 7f17c1afd640 1 -- 172.21.15.16:0/303522341 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17bc106f10 msgr2=0x7f17bc19ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:35.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.396+0000 7f17c1afd640 1 --2- 172.21.15.16:0/303522341 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17bc106f10 0x7f17bc19ef80 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f17b400b970 tx=0x7f17b400be30 comp rx=0 tx=0).stop 2024-09-12T02:09:35.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.396+0000 7f17c1afd640 1 -- 172.21.15.16:0/303522341 shutdown_connections 2024-09-12T02:09:35.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.396+0000 7f17c1afd640 1 --2- 172.21.15.16:0/303522341 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f17a80a2b80 0x7f17a80a5040 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:35.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.396+0000 7f17c1afd640 1 --2- 172.21.15.16:0/303522341 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17bc106f10 0x7f17bc19ef80 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:35.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.396+0000 7f17c1afd640 1 --2- 172.21.15.16:0/303522341 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f17bc102790 0x7f17bc19abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:35.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.396+0000 7f17c1afd640 1 --2- 172.21.15.16:0/303522341 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f17bc101de0 0x7f17bc19a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:35.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.396+0000 7f17c1afd640 1 -- 172.21.15.16:0/303522341 >> 172.21.15.16:0/303522341 conn(0x7f17bc0fd8c0 msgr2=0x7f17bc0ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:35.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.397+0000 7f17c1afd640 1 -- 172.21.15.16:0/303522341 shutdown_connections 2024-09-12T02:09:35.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:35.397+0000 7f17c1afd640 1 -- 172.21.15.16:0/303522341 wait complete. 2024-09-12T02:09:35.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:09:35 smithi184 ceph-mon[35593]: pgmap v565: 97 pgs: 97 active+clean; 4.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 12 op/s 2024-09-12T02:09:35.921 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:09:35 smithi033 ceph-mon[35332]: pgmap v565: 97 pgs: 97 active+clean; 4.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 12 op/s 2024-09-12T02:09:35.934 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:09:35 smithi016 ceph-mon[30947]: pgmap v565: 97 pgs: 97 active+clean; 4.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 12 op/s 2024-09-12T02:09:36.083 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-12T02:09:36.364 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:09:36.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.816+0000 7f6113c66640 1 -- 172.21.15.16:0/2887910496 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f610c1008d0 msgr2=0x7f610c100cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:36.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.816+0000 7f6113c66640 1 --2- 172.21.15.16:0/2887910496 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f610c1008d0 0x7f610c100cb0 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f6108009ca0 tx=0x7f610802a8c0 comp rx=0 tx=0).stop 2024-09-12T02:09:36.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.816+0000 7f6113c66640 1 -- 172.21.15.16:0/2887910496 shutdown_connections 2024-09-12T02:09:36.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.816+0000 7f6113c66640 1 --2- 172.21.15.16:0/2887910496 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f610c10e480 0x7f610c110870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:36.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.816+0000 7f6113c66640 1 --2- 172.21.15.16:0/2887910496 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f610c101280 0x7f610c10df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:36.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.816+0000 7f6113c66640 1 --2- 172.21.15.16:0/2887910496 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f610c1008d0 0x7f610c100cb0 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:36.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.816+0000 7f6113c66640 1 -- 172.21.15.16:0/2887910496 >> 172.21.15.16:0/2887910496 conn(0x7f610c0fc7a0 msgr2=0x7f610c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:36.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.816+0000 7f6113c66640 1 -- 172.21.15.16:0/2887910496 shutdown_connections 2024-09-12T02:09:36.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.816+0000 7f6113c66640 1 -- 172.21.15.16:0/2887910496 wait complete. 2024-09-12T02:09:36.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.817+0000 7f6113c66640 1 Processor -- start 2024-09-12T02:09:36.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.817+0000 7f6113c66640 1 -- start start 2024-09-12T02:09:36.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.817+0000 7f6113c66640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f610c1008d0 0x7f610c19a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:36.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.817+0000 7f6113c66640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f610c101280 0x7f610c19abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:36.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.817+0000 7f6113c66640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f610c10e480 0x7f610c19ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:36.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.817+0000 7f6113c66640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f610c1141b0 con 0x7f610c1008d0 2024-09-12T02:09:36.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.817+0000 7f6113c66640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f610c114030 con 0x7f610c101280 2024-09-12T02:09:36.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.817+0000 7f6113c66640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f610c114330 con 0x7f610c10e480 2024-09-12T02:09:36.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.817+0000 7f6111a02640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f610c1008d0 0x7f610c19a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:36.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.817+0000 7f6111a02640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f610c1008d0 0x7f610c19a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39750/0 (socket says 172.21.15.16:39750) 2024-09-12T02:09:36.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.817+0000 7f6111a02640 1 -- 172.21.15.16:0/2624275930 learned_addr learned my addr 172.21.15.16:0/2624275930 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:09:36.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.818+0000 7f6111201640 1 --2- 172.21.15.16:0/2624275930 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f610c101280 0x7f610c19abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:36.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.818+0000 7f6112203640 1 --2- 172.21.15.16:0/2624275930 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f610c10e480 0x7f610c19ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:36.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.818+0000 7f6111a02640 1 -- 172.21.15.16:0/2624275930 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f610c10e480 msgr2=0x7f610c19ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:36.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.818+0000 7f6111a02640 1 --2- 172.21.15.16:0/2624275930 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f610c10e480 0x7f610c19ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:36.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.818+0000 7f6111a02640 1 -- 172.21.15.16:0/2624275930 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f610c101280 msgr2=0x7f610c19abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:36.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.818+0000 7f6111a02640 1 --2- 172.21.15.16:0/2624275930 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f610c101280 0x7f610c19abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:36.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.818+0000 7f6111a02640 1 -- 172.21.15.16:0/2624275930 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f610c19f640 con 0x7f610c1008d0 2024-09-12T02:09:36.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.818+0000 7f6111201640 1 --2- 172.21.15.16:0/2624275930 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f610c101280 0x7f610c19abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:09:36.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.818+0000 7f6112203640 1 --2- 172.21.15.16:0/2624275930 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f610c10e480 0x7f610c19ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:09:36.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.818+0000 7f6111a02640 1 --2- 172.21.15.16:0/2624275930 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f610c1008d0 0x7f610c19a690 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f610802ade0 tx=0x7f61080319f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:36.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.818+0000 7f60feffd640 1 -- 172.21.15.16:0/2624275930 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f610802fa20 con 0x7f610c1008d0 2024-09-12T02:09:36.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.818+0000 7f60feffd640 1 -- 172.21.15.16:0/2624275930 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f610802fbc0 con 0x7f610c1008d0 2024-09-12T02:09:36.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.818+0000 7f6113c66640 1 -- 172.21.15.16:0/2624275930 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f610c19f8d0 con 0x7f610c1008d0 2024-09-12T02:09:36.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.818+0000 7f60feffd640 1 -- 172.21.15.16:0/2624275930 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61080466e0 con 0x7f610c1008d0 2024-09-12T02:09:36.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.818+0000 7f6113c66640 1 -- 172.21.15.16:0/2624275930 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f610c1a7210 con 0x7f610c1008d0 2024-09-12T02:09:36.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.821+0000 7f60feffd640 1 -- 172.21.15.16:0/2624275930 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f61080305c0 con 0x7f610c1008d0 2024-09-12T02:09:36.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.821+0000 7f611603f740 1 Processor -- start 2024-09-12T02:09:36.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.821+0000 7f611603f740 1 -- start start 2024-09-12T02:09:36.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.821+0000 7f611603f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c498912b40 0x55c498911090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:36.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.821+0000 7f611603f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c4989115d0 0x55c498911a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:36.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.821+0000 7f611603f740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c49891c370 0x55c49891e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:36.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.821+0000 7f611603f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55c4988772f0 con 0x55c498912b40 2024-09-12T02:09:36.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.821+0000 7f611603f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55c49881ecf0 con 0x55c4989115d0 2024-09-12T02:09:36.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.821+0000 7f611603f740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55c4988732b0 con 0x55c49891c370 2024-09-12T02:09:36.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.822+0000 7f60feffd640 1 --2- 172.21.15.16:0/2624275930 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f60f80a2c00 0x7f60f80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:36.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.822+0000 7f6112203640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c498912b40 0x55c498911090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:36.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.822+0000 7f6112203640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c498912b40 0x55c498911090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39754/0 (socket says 172.21.15.16:39754) 2024-09-12T02:09:36.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.822+0000 7f6112203640 1 -- 172.21.15.16:0/141678715 learned_addr learned my addr 172.21.15.16:0/141678715 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:09:36.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.822+0000 7f6111a02640 1 --2- 172.21.15.16:0/141678715 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c4989115d0 0x55c498911a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:36.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.822+0000 7f6111201640 1 --2- 172.21.15.16:0/141678715 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c49891c370 0x55c49891e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:36.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.822+0000 7f6112203640 1 --2- 172.21.15.16:0/2624275930 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f60f80a2c00 0x7f60f80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:36.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.822+0000 7f60feffd640 1 -- 172.21.15.16:0/2624275930 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f61080f05a0 con 0x7f610c1008d0 2024-09-12T02:09:36.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.822+0000 7f6112203640 1 -- 172.21.15.16:0/141678715 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c49891c370 msgr2=0x55c49891e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:09:36.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.822+0000 7f6112203640 1 --2- 172.21.15.16:0/141678715 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c49891c370 0x55c49891e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:36.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.822+0000 7f6112203640 1 -- 172.21.15.16:0/141678715 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c4989115d0 msgr2=0x55c498911a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:09:36.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.822+0000 7f6112203640 1 --2- 172.21.15.16:0/141678715 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c4989115d0 0x55c498911a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:36.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.822+0000 7f6112203640 1 -- 172.21.15.16:0/141678715 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c4988033a0 con 0x55c498912b40 2024-09-12T02:09:36.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.822+0000 7f6112203640 1 --2- 172.21.15.16:0/141678715 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c498912b40 0x55c498911090 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f61040147d0 tx=0x7f6104014c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb1a59cab193f4db server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:36.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.822+0000 7f60f7fff640 1 -- 172.21.15.16:0/141678715 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6104009cf0 con 0x55c498912b40 2024-09-12T02:09:36.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.823+0000 7f60f7fff640 1 -- 172.21.15.16:0/141678715 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6104012e50 con 0x55c498912b40 2024-09-12T02:09:36.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.823+0000 7f60f7fff640 1 -- 172.21.15.16:0/141678715 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6104003e60 con 0x55c498912b40 2024-09-12T02:09:36.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.823+0000 7f611603f740 1 -- 172.21.15.16:0/141678715 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c498912b40 msgr2=0x55c498911090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:36.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.823+0000 7f6112203640 1 --2- 172.21.15.16:0/2624275930 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f60f80a2c00 0x7f60f80a50c0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f6104021000 tx=0x7f6104020000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:36.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.823+0000 7f611603f740 1 --2- 172.21.15.16:0/141678715 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c498912b40 0x55c498911090 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f61040147d0 tx=0x7f6104014c90 comp rx=0 tx=0).stop 2024-09-12T02:09:36.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.823+0000 7f611603f740 1 -- 172.21.15.16:0/141678715 shutdown_connections 2024-09-12T02:09:36.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.823+0000 7f611603f740 1 --2- 172.21.15.16:0/141678715 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c49891c370 0x55c49891e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:36.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.823+0000 7f611603f740 1 --2- 172.21.15.16:0/141678715 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c4989115d0 0x55c498911a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:36.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.823+0000 7f611603f740 1 --2- 172.21.15.16:0/141678715 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c498912b40 0x55c498911090 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:36.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.823+0000 7f611603f740 1 -- 172.21.15.16:0/141678715 >> 172.21.15.16:0/141678715 conn(0x55c4988da310 msgr2=0x55c49890dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:36.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.824+0000 7f611603f740 1 -- 172.21.15.16:0/141678715 shutdown_connections 2024-09-12T02:09:36.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.824+0000 7f611603f740 1 -- 172.21.15.16:0/141678715 wait complete. 2024-09-12T02:09:36.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.824+0000 7f611603f740 1 -- start start 2024-09-12T02:09:36.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.824+0000 7f611603f740 1 Processor -- start 2024-09-12T02:09:36.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.824+0000 7f611603f740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c4989115d0 0x55c49891bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:36.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.824+0000 7f611603f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c498912b40 0x55c498918fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:36.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.824+0000 7f611603f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c49891c370 0x55c4989195d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:36.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.824+0000 7f611603f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55c49891f070 con 0x55c498912b40 2024-09-12T02:09:36.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.824+0000 7f611603f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55c498832060 con 0x55c49891c370 2024-09-12T02:09:36.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.824+0000 7f611603f740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55c49891f1f0 con 0x55c4989115d0 2024-09-12T02:09:36.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.824+0000 7f611603f740 10 client.0 _ll_register_callbacks cb 0x55c4985858c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:09:36.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.824+0000 7f6111201640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c498912b40 0x55c498918fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:36.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.824+0000 7f6111201640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c498912b40 0x55c498918fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39764/0 (socket says 172.21.15.16:39764) 2024-09-12T02:09:36.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.825+0000 7f6111201640 1 -- 172.21.15.16:0/1110847722 learned_addr learned my addr 172.21.15.16:0/1110847722 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:09:36.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.825+0000 7f6111a02640 1 --2- 172.21.15.16:0/1110847722 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c4989115d0 0x55c49891bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:36.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.825+0000 7f6112203640 1 --2- 172.21.15.16:0/1110847722 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c49891c370 0x55c4989195d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:36.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.825+0000 7f6111201640 1 -- 172.21.15.16:0/1110847722 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c4989115d0 msgr2=0x55c49891bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:09:36.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.825+0000 7f6111201640 1 --2- 172.21.15.16:0/1110847722 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c4989115d0 0x55c49891bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:36.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.825+0000 7f6111201640 1 -- 172.21.15.16:0/1110847722 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c49891c370 msgr2=0x55c4989195d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:09:36.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.825+0000 7f6111201640 1 --2- 172.21.15.16:0/1110847722 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c49891c370 0x55c4989195d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:36.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.825+0000 7f6111201640 1 -- 172.21.15.16:0/1110847722 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c4985e83c0 con 0x55c498912b40 2024-09-12T02:09:36.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.825+0000 7f6111201640 1 --2- 172.21.15.16:0/1110847722 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c498912b40 0x55c498918fe0 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f6100013780 tx=0x7f6100013c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db48f7e139f37d1b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:36.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.825+0000 7f60f57fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:09:36.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.825+0000 7f60f57fa640 1 -- 172.21.15.16:0/1110847722 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6100011dd0 con 0x55c498912b40 2024-09-12T02:09:36.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.825+0000 7f60f57fa640 1 -- 172.21.15.16:0/1110847722 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f6100005020 con 0x55c498912b40 2024-09-12T02:09:36.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.825+0000 7f611603f740 1 -- 172.21.15.16:0/1110847722 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6104012e50 con 0x55c498912b40 2024-09-12T02:09:36.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.825+0000 7f60f57fa640 1 -- 172.21.15.16:0/1110847722 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6100005260 con 0x55c498912b40 2024-09-12T02:09:36.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.825+0000 7f611603f740 10 client.17787 fetch_fsmap learned FSMap version 25 2024-09-12T02:09:36.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.825+0000 7f611603f740 1 -- 172.21.15.16:0/1110847722 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c49892ab30 con 0x55c498912b40 2024-09-12T02:09:36.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.826+0000 7f60f57fa640 1 -- 172.21.15.16:0/1110847722 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6104012e50 con 0x55c498912b40 2024-09-12T02:09:36.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.826+0000 7f60f57fa640 1 -- 172.21.15.16:0/1110847722 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f610001a030 con 0x55c498912b40 2024-09-12T02:09:36.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.826+0000 7f611603f740 10 client.17787 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:09:36.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.826+0000 7f611603f740 10 client.17787 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:09:36.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.826+0000 7f611603f740 20 client.17787 populate_metadata read hostname 'smithi016' 2024-09-12T02:09:36.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.826+0000 7f611603f740 1 --2- 172.21.15.16:0/1110847722 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55c49892c090 0x55c49892e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:36.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.826+0000 7f611603f740 4 client.17787 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:09:36.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.826+0000 7f611603f740 1 -- 172.21.15.16:0/1110847722 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c498884860 con 0x55c49892c090 2024-09-12T02:09:36.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.826+0000 7f6111a02640 1 --2- 172.21.15.16:0/1110847722 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55c49892c090 0x55c49892e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:36.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.827+0000 7f6111a02640 1 --2- 172.21.15.16:0/1110847722 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55c49892c090 0x55c49892e4d0 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=190523135c6f7974 server_cookie=a6bbc374c74dd564 in_seq=0 out_seq=0 2024-09-12T02:09:36.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.827+0000 7f60f57fa640 10 client.17787 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:09:36.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.827+0000 7f60f57fa640 1 -- 172.21.15.16:0/1110847722 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c498884860 con 0x55c49892c090 2024-09-12T02:09:36.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.827+0000 7f60f57fa640 10 client.17787 handle_command_reply: tid=0 2024-09-12T02:09:36.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.827+0000 7f611603f740 1 client.17787 shutdown 2024-09-12T02:09:36.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.828+0000 7f611603f740 1 -- 172.21.15.16:0/1110847722 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c498912b40 msgr2=0x55c498918fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:36.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.828+0000 7f611603f740 1 --2- 172.21.15.16:0/1110847722 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c498912b40 0x55c498918fe0 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f6100013780 tx=0x7f6100013c40 comp rx=0 tx=0).stop 2024-09-12T02:09:36.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.828+0000 7f611603f740 1 -- 172.21.15.16:0/1110847722 shutdown_connections 2024-09-12T02:09:36.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.828+0000 7f611603f740 1 --2- 172.21.15.16:0/1110847722 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55c49892c090 0x55c49892e4d0 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:36.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.828+0000 7f611603f740 1 --2- 172.21.15.16:0/1110847722 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c49891c370 0x55c4989195d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:36.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.828+0000 7f611603f740 1 --2- 172.21.15.16:0/1110847722 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c498912b40 0x55c498918fe0 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:36.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.828+0000 7f611603f740 1 --2- 172.21.15.16:0/1110847722 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c4989115d0 0x55c49891bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:36.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.828+0000 7f60f57fa640 0 client.17787 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:09:36.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.828+0000 7f611603f740 1 -- 172.21.15.16:0/1110847722 >> 172.21.15.16:0/1110847722 conn(0x55c4988da310 msgr2=0x55c498913960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:36.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.828+0000 7f611603f740 1 -- 172.21.15.16:0/1110847722 shutdown_connections 2024-09-12T02:09:36.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.828+0000 7f611603f740 1 -- 172.21.15.16:0/1110847722 wait complete. 2024-09-12T02:09:36.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.828+0000 7f611603f740 20 client.17787 trim_cache size 0 max 16384 2024-09-12T02:09:36.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.841+0000 7f611603f740 1 Processor -- start 2024-09-12T02:09:36.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.841+0000 7f611603f740 1 -- start start 2024-09-12T02:09:36.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.841+0000 7f611603f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c4989115d0 0x55c498928310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:36.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.841+0000 7f611603f740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c498912b40 0x55c498914120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:36.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.841+0000 7f611603f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c49891c370 0x55c498914660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:36.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.841+0000 7f611603f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f610c1b1e20 con 0x55c49891c370 2024-09-12T02:09:36.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.841+0000 7f611603f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f610c1af6f0 con 0x55c4989115d0 2024-09-12T02:09:36.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.841+0000 7f611603f740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f610c1ad540 con 0x55c498912b40 2024-09-12T02:09:36.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.841+0000 7f6112203640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c49891c370 0x55c498914660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:36.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.841+0000 7f6112203640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c49891c370 0x55c498914660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39768/0 (socket says 172.21.15.16:39768) 2024-09-12T02:09:36.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.841+0000 7f6112203640 1 -- 172.21.15.16:0/195379790 learned_addr learned my addr 172.21.15.16:0/195379790 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:09:36.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.841+0000 7f6111201640 1 --2- 172.21.15.16:0/195379790 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c498912b40 0x55c498914120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:36.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.841+0000 7f6111a02640 1 --2- 172.21.15.16:0/195379790 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c4989115d0 0x55c498928310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:36.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.842+0000 7f6112203640 1 -- 172.21.15.16:0/195379790 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c498912b40 msgr2=0x55c498914120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:09:36.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.842+0000 7f6112203640 1 --2- 172.21.15.16:0/195379790 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c498912b40 0x55c498914120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:36.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.842+0000 7f6112203640 1 -- 172.21.15.16:0/195379790 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c4989115d0 msgr2=0x55c498928310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:09:36.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.842+0000 7f6112203640 1 --2- 172.21.15.16:0/195379790 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c4989115d0 0x55c498928310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:36.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.842+0000 7f6112203640 1 -- 172.21.15.16:0/195379790 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c49891a100 con 0x55c49891c370 2024-09-12T02:09:36.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.842+0000 7f6111201640 1 --2- 172.21.15.16:0/195379790 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c498912b40 0x55c498914120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:09:36.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.842+0000 7f6111a02640 1 --2- 172.21.15.16:0/195379790 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c4989115d0 0x55c498928310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:09:36.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.842+0000 7f6112203640 1 --2- 172.21.15.16:0/195379790 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c49891c370 0x55c498914660 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f6104005f10 tx=0x7f6104028000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86cc08b4df71377c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:36.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.842+0000 7f60f77fe640 1 -- 172.21.15.16:0/195379790 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6104035030 con 0x55c49891c370 2024-09-12T02:09:36.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.842+0000 7f60f77fe640 1 -- 172.21.15.16:0/195379790 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f610402b030 con 0x55c49891c370 2024-09-12T02:09:36.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.842+0000 7f60f77fe640 1 -- 172.21.15.16:0/195379790 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6104005040 con 0x55c49891c370 2024-09-12T02:09:36.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.842+0000 7f611603f740 1 -- 172.21.15.16:0/195379790 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c49891c370 msgr2=0x55c498914660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:36.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.842+0000 7f611603f740 1 --2- 172.21.15.16:0/195379790 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c49891c370 0x55c498914660 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f6104005f10 tx=0x7f6104028000 comp rx=0 tx=0).stop 2024-09-12T02:09:36.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.843+0000 7f611603f740 1 -- 172.21.15.16:0/195379790 shutdown_connections 2024-09-12T02:09:36.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.843+0000 7f611603f740 1 --2- 172.21.15.16:0/195379790 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c49891c370 0x55c498914660 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:36.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.843+0000 7f611603f740 1 --2- 172.21.15.16:0/195379790 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c498912b40 0x55c498914120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:36.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.843+0000 7f611603f740 1 --2- 172.21.15.16:0/195379790 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c4989115d0 0x55c498928310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:36.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.843+0000 7f611603f740 1 -- 172.21.15.16:0/195379790 >> 172.21.15.16:0/195379790 conn(0x55c4988da310 msgr2=0x55c4989180d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:36.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.843+0000 7f611603f740 1 -- 172.21.15.16:0/195379790 shutdown_connections 2024-09-12T02:09:36.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.843+0000 7f611603f740 1 -- 172.21.15.16:0/195379790 wait complete. 2024-09-12T02:09:36.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.843+0000 7f611603f740 1 -- start start 2024-09-12T02:09:36.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.843+0000 7f611603f740 1 Processor -- start 2024-09-12T02:09:36.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.843+0000 7f611603f740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c4989115d0 0x55c498924c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:36.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.843+0000 7f611603f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c498912b40 0x55c49891ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:36.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.843+0000 7f611603f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c49891c370 0x55c4989204d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:36.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.843+0000 7f611603f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f610c1adbb0 con 0x55c498912b40 2024-09-12T02:09:36.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.843+0000 7f611603f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f610c1af020 con 0x55c49891c370 2024-09-12T02:09:36.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.843+0000 7f611603f740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f610c1ae280 con 0x55c4989115d0 2024-09-12T02:09:36.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.844+0000 7f611603f740 10 client.0 _ll_register_callbacks cb 0x7f6100009860 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:09:36.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.844+0000 7f6111201640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c498912b40 0x55c49891ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:36.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.844+0000 7f6111201640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c498912b40 0x55c49891ff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39782/0 (socket says 172.21.15.16:39782) 2024-09-12T02:09:36.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.844+0000 7f6111201640 1 -- 172.21.15.16:0/2948280373 learned_addr learned my addr 172.21.15.16:0/2948280373 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:09:36.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.844+0000 7f6111a02640 1 --2- 172.21.15.16:0/2948280373 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c4989115d0 0x55c498924c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:36.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.844+0000 7f6112203640 1 --2- 172.21.15.16:0/2948280373 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c49891c370 0x55c4989204d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:36.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.844+0000 7f6111201640 1 -- 172.21.15.16:0/2948280373 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c4989115d0 msgr2=0x55c498924c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:09:36.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.844+0000 7f6111201640 1 --2- 172.21.15.16:0/2948280373 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c4989115d0 0x55c498924c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:36.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.844+0000 7f6111201640 1 -- 172.21.15.16:0/2948280373 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c49891c370 msgr2=0x55c4989204d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:09:36.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.844+0000 7f6111201640 1 --2- 172.21.15.16:0/2948280373 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c49891c370 0x55c4989204d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:36.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.844+0000 7f6111201640 1 -- 172.21.15.16:0/2948280373 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c498920d00 con 0x55c498912b40 2024-09-12T02:09:36.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.844+0000 7f6111201640 1 --2- 172.21.15.16:0/2948280373 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c498912b40 0x55c49891ff70 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f6100000c00 tx=0x7f610001b620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=282af96b9a0ec813 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:36.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.844+0000 7f60f57fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:09:36.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.844+0000 7f60f57fa640 1 -- 172.21.15.16:0/2948280373 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f610002a040 con 0x55c498912b40 2024-09-12T02:09:36.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.844+0000 7f60f57fa640 1 -- 172.21.15.16:0/2948280373 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f6100005600 con 0x55c498912b40 2024-09-12T02:09:36.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.844+0000 7f611603f740 1 -- 172.21.15.16:0/2948280373 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f610402b030 con 0x55c498912b40 2024-09-12T02:09:36.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.845+0000 7f60f57fa640 1 -- 172.21.15.16:0/2948280373 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6100011bd0 con 0x55c498912b40 2024-09-12T02:09:36.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.845+0000 7f611603f740 10 client.17793 fetch_fsmap learned FSMap version 25 2024-09-12T02:09:36.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.845+0000 7f611603f740 1 -- 172.21.15.16:0/2948280373 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c498996d70 con 0x55c498912b40 2024-09-12T02:09:36.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.845+0000 7f60f57fa640 1 -- 172.21.15.16:0/2948280373 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f610402b030 con 0x55c498912b40 2024-09-12T02:09:36.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.845+0000 7f60f57fa640 1 -- 172.21.15.16:0/2948280373 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f6100037050 con 0x55c498912b40 2024-09-12T02:09:36.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.845+0000 7f611603f740 10 client.17793 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:09:36.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.845+0000 7f611603f740 10 client.17793 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:09:36.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.845+0000 7f611603f740 20 client.17793 populate_metadata read hostname 'smithi016' 2024-09-12T02:09:36.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.845+0000 7f611603f740 1 --2- 172.21.15.16:0/2948280373 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55c498997460 0x55c498999880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:36.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.845+0000 7f611603f740 4 client.17793 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T02:09:36.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.845+0000 7f611603f740 1 -- 172.21.15.16:0/2948280373 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c49887ed70 con 0x55c498997460 2024-09-12T02:09:36.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.846+0000 7f6111a02640 1 --2- 172.21.15.16:0/2948280373 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55c498997460 0x55c498999880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:36.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.846+0000 7f60f57fa640 10 client.17793 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:09:36.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.846+0000 7f6111a02640 1 --2- 172.21.15.16:0/2948280373 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55c498997460 0x55c498999880 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab34ba88f8b4c368 server_cookie=259d3b5754d15607 in_seq=0 out_seq=0 2024-09-12T02:09:36.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.846+0000 7f60f57fa640 1 -- 172.21.15.16:0/2948280373 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c49887ed70 con 0x55c498997460 2024-09-12T02:09:36.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.846+0000 7f60f57fa640 10 client.17793 handle_command_reply: tid=0 2024-09-12T02:09:36.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.846+0000 7f611603f740 1 client.17793 shutdown 2024-09-12T02:09:36.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.846+0000 7f611603f740 1 -- 172.21.15.16:0/2948280373 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c498912b40 msgr2=0x55c49891ff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:36.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.846+0000 7f611603f740 1 --2- 172.21.15.16:0/2948280373 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c498912b40 0x55c49891ff70 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f6100000c00 tx=0x7f610001b620 comp rx=0 tx=0).stop 2024-09-12T02:09:36.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.846+0000 7f611603f740 1 -- 172.21.15.16:0/2948280373 shutdown_connections 2024-09-12T02:09:36.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.846+0000 7f611603f740 1 --2- 172.21.15.16:0/2948280373 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55c498997460 0x55c498999880 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:36.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.846+0000 7f611603f740 1 --2- 172.21.15.16:0/2948280373 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c49891c370 0x55c4989204d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:36.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.846+0000 7f611603f740 1 --2- 172.21.15.16:0/2948280373 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c498912b40 0x55c49891ff70 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:36.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.846+0000 7f611603f740 1 --2- 172.21.15.16:0/2948280373 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c4989115d0 0x55c498924c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:36.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.846+0000 7f60f57fa640 0 client.17793 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:09:36.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.846+0000 7f611603f740 1 -- 172.21.15.16:0/2948280373 >> 172.21.15.16:0/2948280373 conn(0x55c4988da310 msgr2=0x55c49890c4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:36.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.847+0000 7f611603f740 1 -- 172.21.15.16:0/2948280373 shutdown_connections 2024-09-12T02:09:36.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.847+0000 7f611603f740 1 -- 172.21.15.16:0/2948280373 wait complete. 2024-09-12T02:09:36.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.847+0000 7f611603f740 20 client.17793 trim_cache size 0 max 16384 2024-09-12T02:09:36.853 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T02:09:36.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.847+0000 7f6113c66640 1 -- 172.21.15.16:0/2624275930 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f60f80a2c00 msgr2=0x7f60f80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:36.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.847+0000 7f6113c66640 1 --2- 172.21.15.16:0/2624275930 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f60f80a2c00 0x7f60f80a50c0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f6104021000 tx=0x7f6104020000 comp rx=0 tx=0).stop 2024-09-12T02:09:36.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.847+0000 7f6113c66640 1 -- 172.21.15.16:0/2624275930 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f610c1008d0 msgr2=0x7f610c19a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:36.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.847+0000 7f6113c66640 1 --2- 172.21.15.16:0/2624275930 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f610c1008d0 0x7f610c19a690 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f610802ade0 tx=0x7f61080319f0 comp rx=0 tx=0).stop 2024-09-12T02:09:36.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.847+0000 7f6113c66640 1 -- 172.21.15.16:0/2624275930 shutdown_connections 2024-09-12T02:09:36.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.847+0000 7f6113c66640 1 --2- 172.21.15.16:0/2624275930 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f60f80a2c00 0x7f60f80a50c0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:36.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.847+0000 7f6113c66640 1 --2- 172.21.15.16:0/2624275930 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f610c10e480 0x7f610c19ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:36.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.847+0000 7f6113c66640 1 --2- 172.21.15.16:0/2624275930 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f610c101280 0x7f610c19abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:36.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.847+0000 7f6113c66640 1 --2- 172.21.15.16:0/2624275930 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f610c1008d0 0x7f610c19a690 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:36.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.847+0000 7f6113c66640 1 -- 172.21.15.16:0/2624275930 >> 172.21.15.16:0/2624275930 conn(0x7f610c0fc7a0 msgr2=0x7f610c0feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:36.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.847+0000 7f6113c66640 1 -- 172.21.15.16:0/2624275930 shutdown_connections 2024-09-12T02:09:36.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:36.848+0000 7f6113c66640 1 -- 172.21.15.16:0/2624275930 wait complete. 2024-09-12T02:09:37.594 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:09:37 smithi016 ceph-mon[30947]: pgmap v566: 97 pgs: 97 active+clean; 4.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 5.2 KiB/s wr, 10 op/s 2024-09-12T02:09:37.659 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.658+0000 7fbaaddfd640 1 -- 172.21.15.16:0/3708639777 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbaa8102610 msgr2=0x7fbaa8108ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:37.659 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.658+0000 7fbaaddfd640 1 --2- 172.21.15.16:0/3708639777 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbaa8102610 0x7fbaa8108ea0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7fbaa0009ca0 tx=0x7fbaa002a8c0 comp rx=0 tx=0).stop 2024-09-12T02:09:37.659 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.658+0000 7fbaaddfd640 1 -- 172.21.15.16:0/3708639777 shutdown_connections 2024-09-12T02:09:37.659 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.658+0000 7fbaaddfd640 1 --2- 172.21.15.16:0/3708639777 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbaa8102610 0x7fbaa8108ea0 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:37.659 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.658+0000 7fbaaddfd640 1 --2- 172.21.15.16:0/3708639777 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbaa8101c50 0x7fbaa81020d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:37.659 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.658+0000 7fbaaddfd640 1 --2- 172.21.15.16:0/3708639777 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbaa8100a60 0x7fbaa8100e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:37.659 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.658+0000 7fbaaddfd640 1 -- 172.21.15.16:0/3708639777 >> 172.21.15.16:0/3708639777 conn(0x7fbaa80fc1e0 msgr2=0x7fbaa80fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:37.659 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.658+0000 7fbaaddfd640 1 -- 172.21.15.16:0/3708639777 shutdown_connections 2024-09-12T02:09:37.659 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.658+0000 7fbaaddfd640 1 -- 172.21.15.16:0/3708639777 wait complete. 2024-09-12T02:09:37.659 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.659+0000 7fbaaddfd640 1 Processor -- start 2024-09-12T02:09:37.659 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.659+0000 7fbaaddfd640 1 -- start start 2024-09-12T02:09:37.661 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.659+0000 7fbaaddfd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbaa8100a60 0x7fbaa8197da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:37.661 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.659+0000 7fbaaddfd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbaa8101c50 0x7fbaa81982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:37.661 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.659+0000 7fbaaddfd640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbaa8102610 0x7fbaa819f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:37.661 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.659+0000 7fbaaddfd640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbaa810c710 con 0x7fbaa8100a60 2024-09-12T02:09:37.661 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.659+0000 7fbaaddfd640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbaa810c590 con 0x7fbaa8101c50 2024-09-12T02:09:37.662 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.659+0000 7fbaaddfd640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fbaa810c890 con 0x7fbaa8102610 2024-09-12T02:09:37.662 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.659+0000 7fbaa77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbaa8100a60 0x7fbaa8197da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:37.662 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.659+0000 7fbaa77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbaa8100a60 0x7fbaa8197da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39810/0 (socket says 172.21.15.16:39810) 2024-09-12T02:09:37.662 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.659+0000 7fbaa77fe640 1 -- 172.21.15.16:0/2778291165 learned_addr learned my addr 172.21.15.16:0/2778291165 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:09:37.662 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.659+0000 7fba9effd640 1 --2- 172.21.15.16:0/2778291165 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbaa8101c50 0x7fbaa81982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:37.662 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.660+0000 7fbaa7fff640 1 --2- 172.21.15.16:0/2778291165 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbaa8102610 0x7fbaa819f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:37.662 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.660+0000 7fbaa77fe640 1 -- 172.21.15.16:0/2778291165 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbaa8102610 msgr2=0x7fbaa819f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:37.662 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.660+0000 7fbaa77fe640 1 --2- 172.21.15.16:0/2778291165 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbaa8102610 0x7fbaa819f310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:37.662 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.660+0000 7fbaa77fe640 1 -- 172.21.15.16:0/2778291165 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbaa8101c50 msgr2=0x7fbaa81982e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:37.662 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.660+0000 7fbaa77fe640 1 --2- 172.21.15.16:0/2778291165 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbaa8101c50 0x7fbaa81982e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:37.662 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.660+0000 7fbaa77fe640 1 -- 172.21.15.16:0/2778291165 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbaa819f9b0 con 0x7fbaa8100a60 2024-09-12T02:09:37.662 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.660+0000 7fbaa77fe640 1 --2- 172.21.15.16:0/2778291165 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbaa8100a60 0x7fbaa8197da0 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7fba9800d770 tx=0x7fba9800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:37.662 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.660+0000 7fbaa57fa640 1 -- 172.21.15.16:0/2778291165 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba98017070 con 0x7fbaa8100a60 2024-09-12T02:09:37.663 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.660+0000 7fbaa57fa640 1 -- 172.21.15.16:0/2778291165 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba98009c30 con 0x7fbaa8100a60 2024-09-12T02:09:37.663 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.660+0000 7fbaaddfd640 1 -- 172.21.15.16:0/2778291165 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbaa819fca0 con 0x7fbaa8100a60 2024-09-12T02:09:37.663 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.660+0000 7fbaa57fa640 1 -- 172.21.15.16:0/2778291165 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba98003dd0 con 0x7fbaa8100a60 2024-09-12T02:09:37.663 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.660+0000 7fbaaddfd640 1 -- 172.21.15.16:0/2778291165 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbaa81a01a0 con 0x7fbaa8100a60 2024-09-12T02:09:37.670 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.662+0000 7fbaa57fa640 1 -- 172.21.15.16:0/2778291165 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fba98003fb0 con 0x7fbaa8100a60 2024-09-12T02:09:37.670 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.662+0000 7fbab0c7d740 1 Processor -- start 2024-09-12T02:09:37.670 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.663+0000 7fbab0c7d740 1 -- start start 2024-09-12T02:09:37.670 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.663+0000 7fbab0c7d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560413811db0 0x5604138121b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:37.670 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.663+0000 7fbab0c7d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5604138126f0 0x5604138171b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:37.670 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.663+0000 7fbab0c7d740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560413817780 0x560413819c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:37.670 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.663+0000 7fbab0c7d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5604137712f0 con 0x560413811db0 2024-09-12T02:09:37.670 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.663+0000 7fbab0c7d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x560413719270 con 0x5604138126f0 2024-09-12T02:09:37.670 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.663+0000 7fbab0c7d740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x56041376d2b0 con 0x560413817780 2024-09-12T02:09:37.671 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.663+0000 7fbaa7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560413811db0 0x5604138121b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:37.671 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.663+0000 7fbaa7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560413811db0 0x5604138121b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39826/0 (socket says 172.21.15.16:39826) 2024-09-12T02:09:37.671 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.663+0000 7fbaa7fff640 1 -- 172.21.15.16:0/1790987183 learned_addr learned my addr 172.21.15.16:0/1790987183 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:09:37.671 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.663+0000 7fba9effd640 1 --2- 172.21.15.16:0/1790987183 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560413817780 0x560413819c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:37.671 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.663+0000 7fbaa77fe640 1 --2- 172.21.15.16:0/1790987183 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5604138126f0 0x5604138171b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:37.671 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.663+0000 7fbaa7fff640 1 -- 172.21.15.16:0/1790987183 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560413817780 msgr2=0x560413819c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:09:37.671 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.663+0000 7fbaa7fff640 1 --2- 172.21.15.16:0/1790987183 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560413817780 0x560413819c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:37.671 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.663+0000 7fbaa7fff640 1 -- 172.21.15.16:0/1790987183 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5604138126f0 msgr2=0x5604138171b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:09:37.671 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.663+0000 7fbaa7fff640 1 --2- 172.21.15.16:0/1790987183 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5604138126f0 0x5604138171b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:37.671 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.663+0000 7fbaa7fff640 1 -- 172.21.15.16:0/1790987183 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560413718340 con 0x560413811db0 2024-09-12T02:09:37.671 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.663+0000 7fbaa77fe640 1 --2- 172.21.15.16:0/1790987183 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5604138126f0 0x5604138171b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:09:37.671 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.663+0000 7fbaa57fa640 1 --2- 172.21.15.16:0/2778291165 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fba940a2c30 0x7fba940a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:37.671 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.663+0000 7fbaa7fff640 1 --2- 172.21.15.16:0/1790987183 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560413811db0 0x5604138121b0 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7fbaa00097b0 tx=0x7fbaa0005fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61e425b40b3faeb0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:37.672 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.664+0000 7fbaa77fe640 1 --2- 172.21.15.16:0/2778291165 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fba940a2c30 0x7fba940a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:37.672 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.664+0000 7fbaa57fa640 1 -- 172.21.15.16:0/2778291165 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fba980ce9c0 con 0x7fbaa8100a60 2024-09-12T02:09:37.672 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.664+0000 7fba9dffb640 1 -- 172.21.15.16:0/1790987183 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaa0031650 con 0x560413811db0 2024-09-12T02:09:37.672 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.664+0000 7fba9dffb640 1 -- 172.21.15.16:0/1790987183 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbaa0036070 con 0x560413811db0 2024-09-12T02:09:37.672 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.664+0000 7fba9dffb640 1 -- 172.21.15.16:0/1790987183 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaa0004730 con 0x560413811db0 2024-09-12T02:09:37.672 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.664+0000 7fbab0c7d740 1 -- 172.21.15.16:0/1790987183 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560413811db0 msgr2=0x5604138121b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:37.672 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.664+0000 7fbaa77fe640 1 --2- 172.21.15.16:0/2778291165 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fba940a2c30 0x7fba940a50f0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fba9800d740 tx=0x7fba9805b350 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:37.672 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.664+0000 7fbab0c7d740 1 --2- 172.21.15.16:0/1790987183 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560413811db0 0x5604138121b0 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7fbaa00097b0 tx=0x7fbaa0005fa0 comp rx=0 tx=0).stop 2024-09-12T02:09:37.672 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.664+0000 7fbab0c7d740 1 -- 172.21.15.16:0/1790987183 shutdown_connections 2024-09-12T02:09:37.672 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.664+0000 7fbab0c7d740 1 --2- 172.21.15.16:0/1790987183 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560413817780 0x560413819c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:37.672 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.664+0000 7fbab0c7d740 1 --2- 172.21.15.16:0/1790987183 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5604138126f0 0x5604138171b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:37.672 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.664+0000 7fbab0c7d740 1 --2- 172.21.15.16:0/1790987183 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560413811db0 0x5604138121b0 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:37.672 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.665+0000 7fbab0c7d740 1 -- 172.21.15.16:0/1790987183 >> 172.21.15.16:0/1790987183 conn(0x560413809010 msgr2=0x560413809410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:37.673 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.665+0000 7fbab0c7d740 1 -- 172.21.15.16:0/1790987183 shutdown_connections 2024-09-12T02:09:37.673 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.665+0000 7fbab0c7d740 1 -- 172.21.15.16:0/1790987183 wait complete. 2024-09-12T02:09:37.673 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.665+0000 7fbab0c7d740 1 -- start start 2024-09-12T02:09:37.673 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.665+0000 7fbab0c7d740 1 Processor -- start 2024-09-12T02:09:37.673 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.665+0000 7fbab0c7d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560413811db0 0x56041381bf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:37.673 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.665+0000 7fbab0c7d740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5604138126f0 0x56041381c460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:37.673 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.665+0000 7fbab0c7d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560413817780 0x5604138234a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:37.673 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.665+0000 7fbab0c7d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56041381a6f0 con 0x560413817780 2024-09-12T02:09:37.673 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.665+0000 7fbab0c7d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5604136fa290 con 0x560413811db0 2024-09-12T02:09:37.673 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.665+0000 7fbab0c7d740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x56041381a870 con 0x5604138126f0 2024-09-12T02:09:37.673 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.665+0000 7fbab0c7d740 10 client.0 _ll_register_callbacks cb 0x56041372b140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:09:37.673 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.666+0000 7fbaa7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560413817780 0x5604138234a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:37.674 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.666+0000 7fbaa7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560413817780 0x5604138234a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39834/0 (socket says 172.21.15.16:39834) 2024-09-12T02:09:37.674 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.666+0000 7fbaa7fff640 1 -- 172.21.15.16:0/4084163247 learned_addr learned my addr 172.21.15.16:0/4084163247 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:09:37.674 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.666+0000 7fba9effd640 1 --2- 172.21.15.16:0/4084163247 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5604138126f0 0x56041381c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:37.674 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.666+0000 7fbaa7fff640 1 --2- 172.21.15.16:0/4084163247 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560413811db0 0x56041381bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:37.674 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.666+0000 7fbaa7fff640 1 -- 172.21.15.16:0/4084163247 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5604138126f0 msgr2=0x56041381c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:09:37.674 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.666+0000 7fbaa7fff640 1 --2- 172.21.15.16:0/4084163247 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5604138126f0 0x56041381c460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:37.674 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.666+0000 7fbaa7fff640 1 -- 172.21.15.16:0/4084163247 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560413811db0 msgr2=0x56041381bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:09:37.674 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.666+0000 7fbaa7fff640 1 --2- 172.21.15.16:0/4084163247 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560413811db0 0x56041381bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:37.674 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.666+0000 7fbaa7fff640 1 -- 172.21.15.16:0/4084163247 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560413484210 con 0x560413817780 2024-09-12T02:09:37.674 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.666+0000 7fbaa7fff640 1 --2- 172.21.15.16:0/4084163247 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560413817780 0x5604138234a0 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7fbaa00049d0 tx=0x7fbaa003d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7945eb0dd5a699a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:37.674 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.666+0000 7fba86ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:09:37.674 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.666+0000 7fba86ffd640 1 -- 172.21.15.16:0/4084163247 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaa0003970 con 0x560413817780 2024-09-12T02:09:37.674 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.666+0000 7fba86ffd640 1 -- 172.21.15.16:0/4084163247 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fbaa0003b10 con 0x560413817780 2024-09-12T02:09:37.675 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.666+0000 7fbab0c7d740 1 -- 172.21.15.16:0/4084163247 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbaa0036070 con 0x560413817780 2024-09-12T02:09:37.675 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.667+0000 7fba86ffd640 1 -- 172.21.15.16:0/4084163247 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fbaa0003d10 con 0x560413817780 2024-09-12T02:09:37.675 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.667+0000 7fbab0c7d740 10 client.17805 fetch_fsmap learned FSMap version 25 2024-09-12T02:09:37.675 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.667+0000 7fbab0c7d740 1 -- 172.21.15.16:0/4084163247 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x560413550620 con 0x560413817780 2024-09-12T02:09:37.675 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.667+0000 7fba86ffd640 1 -- 172.21.15.16:0/4084163247 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbaa0036070 con 0x560413817780 2024-09-12T02:09:37.675 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.667+0000 7fba86ffd640 1 -- 172.21.15.16:0/4084163247 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fbaa00306c0 con 0x560413817780 2024-09-12T02:09:37.675 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.667+0000 7fbab0c7d740 10 client.17805 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:09:37.675 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.667+0000 7fbab0c7d740 10 client.17805 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:09:37.675 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.667+0000 7fbab0c7d740 20 client.17805 populate_metadata read hostname 'smithi016' 2024-09-12T02:09:37.675 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.667+0000 7fbab0c7d740 1 --2- 172.21.15.16:0/4084163247 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x560413827710 0x560413829b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:37.675 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.667+0000 7fbab0c7d740 4 client.17805 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:09:37.675 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.667+0000 7fbab0c7d740 1 -- 172.21.15.16:0/4084163247 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56041377ea20 con 0x560413827710 2024-09-12T02:09:37.675 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.667+0000 7fba9effd640 1 --2- 172.21.15.16:0/4084163247 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x560413827710 0x560413829b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:37.675 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.668+0000 7fba86ffd640 10 client.17805 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:09:37.676 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.668+0000 7fba9effd640 1 --2- 172.21.15.16:0/4084163247 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x560413827710 0x560413829b00 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3315cca16c0482e2 server_cookie=ce7a53a9482f4fc8 in_seq=0 out_seq=0 2024-09-12T02:09:37.676 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.668+0000 7fba86ffd640 1 -- 172.21.15.16:0/4084163247 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56041377ea20 con 0x560413827710 2024-09-12T02:09:37.676 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.668+0000 7fba86ffd640 10 client.17805 handle_command_reply: tid=0 2024-09-12T02:09:37.676 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.669+0000 7fbab0c7d740 1 client.17805 shutdown 2024-09-12T02:09:37.676 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.669+0000 7fbab0c7d740 1 -- 172.21.15.16:0/4084163247 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560413817780 msgr2=0x5604138234a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:37.676 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.669+0000 7fbab0c7d740 1 --2- 172.21.15.16:0/4084163247 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560413817780 0x5604138234a0 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7fbaa00049d0 tx=0x7fbaa003d040 comp rx=0 tx=0).stop 2024-09-12T02:09:37.676 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.669+0000 7fbab0c7d740 1 -- 172.21.15.16:0/4084163247 shutdown_connections 2024-09-12T02:09:37.676 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.669+0000 7fbab0c7d740 1 --2- 172.21.15.16:0/4084163247 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x560413827710 0x560413829b00 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:37.676 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.669+0000 7fbab0c7d740 1 --2- 172.21.15.16:0/4084163247 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560413817780 0x5604138234a0 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:37.676 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.669+0000 7fbab0c7d740 1 --2- 172.21.15.16:0/4084163247 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5604138126f0 0x56041381c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:37.676 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.669+0000 7fba86ffd640 0 client.17805 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:09:37.676 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.669+0000 7fbab0c7d740 1 --2- 172.21.15.16:0/4084163247 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560413811db0 0x56041381bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:37.676 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.669+0000 7fbab0c7d740 1 -- 172.21.15.16:0/4084163247 >> 172.21.15.16:0/4084163247 conn(0x560413809010 msgr2=0x56041380a820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:37.677 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.669+0000 7fbab0c7d740 1 -- 172.21.15.16:0/4084163247 shutdown_connections 2024-09-12T02:09:37.677 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.669+0000 7fbab0c7d740 1 -- 172.21.15.16:0/4084163247 wait complete. 2024-09-12T02:09:37.677 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.669+0000 7fbab0c7d740 20 client.17805 trim_cache size 0 max 16384 2024-09-12T02:09:37.688 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.683+0000 7fbab0c7d740 1 Processor -- start 2024-09-12T02:09:37.688 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.683+0000 7fbab0c7d740 1 -- start start 2024-09-12T02:09:37.689 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.683+0000 7fbab0c7d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560413811db0 0x5604138080b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:37.689 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.683+0000 7fbab0c7d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5604138126f0 0x560413823460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:37.689 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.683+0000 7fbab0c7d740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560413817780 0x56041382e280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:37.689 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.683+0000 7fbab0c7d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbaa81a9490 con 0x5604138126f0 2024-09-12T02:09:37.689 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.683+0000 7fbab0c7d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbaa81a6d60 con 0x560413811db0 2024-09-12T02:09:37.689 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.683+0000 7fbab0c7d740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fbaa81a4bb0 con 0x560413817780 2024-09-12T02:09:37.689 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.684+0000 7fba9effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5604138126f0 0x560413823460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:37.689 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.684+0000 7fba9effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5604138126f0 0x560413823460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39840/0 (socket says 172.21.15.16:39840) 2024-09-12T02:09:37.689 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.684+0000 7fba9effd640 1 -- 172.21.15.16:0/266560243 learned_addr learned my addr 172.21.15.16:0/266560243 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:09:37.689 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.684+0000 7fba9effd640 1 -- 172.21.15.16:0/266560243 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560413817780 msgr2=0x56041382e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:09:37.689 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.684+0000 7fbaa7fff640 1 --2- 172.21.15.16:0/266560243 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560413817780 0x56041382e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:37.689 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.684+0000 7fba9effd640 1 --2- 172.21.15.16:0/266560243 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560413817780 0x56041382e280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:37.690 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.684+0000 7fba9effd640 1 -- 172.21.15.16:0/266560243 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560413811db0 msgr2=0x5604138080b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:09:37.690 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.684+0000 7fba9effd640 1 --2- 172.21.15.16:0/266560243 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560413811db0 0x5604138080b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:37.690 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.684+0000 7fba9effd640 1 -- 172.21.15.16:0/266560243 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560413812bd0 con 0x5604138126f0 2024-09-12T02:09:37.690 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.684+0000 7fbaa7fff640 1 --2- 172.21.15.16:0/266560243 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560413817780 0x56041382e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:09:37.690 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.684+0000 7fba9effd640 1 --2- 172.21.15.16:0/266560243 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5604138126f0 0x560413823460 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x56041382a230 tx=0x7fba90013830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3add4e33c2fdf6b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:37.690 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.684+0000 7fba9d7fa640 1 -- 172.21.15.16:0/266560243 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba90004020 con 0x5604138126f0 2024-09-12T02:09:37.690 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.684+0000 7fba9d7fa640 1 -- 172.21.15.16:0/266560243 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba900041c0 con 0x5604138126f0 2024-09-12T02:09:37.690 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.684+0000 7fba9d7fa640 1 -- 172.21.15.16:0/266560243 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba90011730 con 0x5604138126f0 2024-09-12T02:09:37.690 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.685+0000 7fbab0c7d740 1 -- 172.21.15.16:0/266560243 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5604138126f0 msgr2=0x560413823460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:37.690 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.685+0000 7fbab0c7d740 1 --2- 172.21.15.16:0/266560243 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5604138126f0 0x560413823460 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x56041382a230 tx=0x7fba90013830 comp rx=0 tx=0).stop 2024-09-12T02:09:37.690 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.685+0000 7fbab0c7d740 1 -- 172.21.15.16:0/266560243 shutdown_connections 2024-09-12T02:09:37.690 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.685+0000 7fbab0c7d740 1 --2- 172.21.15.16:0/266560243 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560413817780 0x56041382e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:37.691 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.685+0000 7fbab0c7d740 1 --2- 172.21.15.16:0/266560243 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5604138126f0 0x560413823460 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:37.691 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.685+0000 7fbab0c7d740 1 --2- 172.21.15.16:0/266560243 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560413811db0 0x5604138080b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:37.691 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.685+0000 7fbab0c7d740 1 -- 172.21.15.16:0/266560243 >> 172.21.15.16:0/266560243 conn(0x560413809010 msgr2=0x5604138079a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:37.691 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.685+0000 7fbab0c7d740 1 -- 172.21.15.16:0/266560243 shutdown_connections 2024-09-12T02:09:37.691 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.685+0000 7fbab0c7d740 1 -- 172.21.15.16:0/266560243 wait complete. 2024-09-12T02:09:37.691 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.685+0000 7fbab0c7d740 1 -- start start 2024-09-12T02:09:37.691 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.685+0000 7fbab0c7d740 1 Processor -- start 2024-09-12T02:09:37.691 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.686+0000 7fbab0c7d740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560413811db0 0x560413820c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:37.691 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.686+0000 7fbab0c7d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5604138126f0 0x560413821180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:37.691 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.686+0000 7fbab0c7d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560413817780 0x560413822140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:37.691 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.686+0000 7fbab0c7d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbaa81a5220 con 0x560413817780 2024-09-12T02:09:37.691 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.686+0000 7fbab0c7d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbaa81a6690 con 0x5604138126f0 2024-09-12T02:09:37.691 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.686+0000 7fbab0c7d740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fbaa81a58f0 con 0x560413811db0 2024-09-12T02:09:37.692 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.686+0000 7fbab0c7d740 10 client.0 _ll_register_callbacks cb 0x7fbaa0005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:09:37.692 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.686+0000 7fbaa7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560413817780 0x560413822140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:37.692 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.686+0000 7fbaa7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560413817780 0x560413822140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39848/0 (socket says 172.21.15.16:39848) 2024-09-12T02:09:37.692 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.686+0000 7fbaa7fff640 1 -- 172.21.15.16:0/3260684934 learned_addr learned my addr 172.21.15.16:0/3260684934 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:09:37.692 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.686+0000 7fbaa7fff640 1 --2- 172.21.15.16:0/3260684934 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560413811db0 0x560413820c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:37.692 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.686+0000 7fba9effd640 1 --2- 172.21.15.16:0/3260684934 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5604138126f0 0x560413821180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:37.692 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.686+0000 7fbaa7fff640 1 -- 172.21.15.16:0/3260684934 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560413811db0 msgr2=0x560413820c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:09:37.692 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.686+0000 7fbaa7fff640 1 --2- 172.21.15.16:0/3260684934 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560413811db0 0x560413820c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:37.692 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.686+0000 7fbaa7fff640 1 -- 172.21.15.16:0/3260684934 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5604138126f0 msgr2=0x560413821180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:09:37.692 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.686+0000 7fbaa7fff640 1 --2- 172.21.15.16:0/3260684934 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5604138126f0 0x560413821180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:37.692 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.686+0000 7fbaa7fff640 1 -- 172.21.15.16:0/3260684934 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560413822750 con 0x560413817780 2024-09-12T02:09:37.692 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.686+0000 7fbaa7fff640 1 --2- 172.21.15.16:0/3260684934 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560413817780 0x560413822140 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7fbaa002ade0 tx=0x7fbaa003d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13637deb0431fc78 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:37.693 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.686+0000 7fba86ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:09:37.693 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.686+0000 7fba86ffd640 1 -- 172.21.15.16:0/3260684934 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaa00040c0 con 0x560413817780 2024-09-12T02:09:37.693 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.686+0000 7fba86ffd640 1 -- 172.21.15.16:0/3260684934 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fbaa0004260 con 0x560413817780 2024-09-12T02:09:37.693 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.686+0000 7fbab0c7d740 1 -- 172.21.15.16:0/3260684934 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fba900041c0 con 0x560413817780 2024-09-12T02:09:37.693 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.687+0000 7fba86ffd640 1 -- 172.21.15.16:0/3260684934 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fbaa0030c30 con 0x560413817780 2024-09-12T02:09:37.693 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.687+0000 7fbab0c7d740 10 client.17811 fetch_fsmap learned FSMap version 25 2024-09-12T02:09:37.693 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.687+0000 7fbab0c7d740 1 -- 172.21.15.16:0/3260684934 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5604138922c0 con 0x560413817780 2024-09-12T02:09:37.693 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.687+0000 7fba86ffd640 1 -- 172.21.15.16:0/3260684934 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba900041c0 con 0x560413817780 2024-09-12T02:09:37.693 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.687+0000 7fba86ffd640 1 -- 172.21.15.16:0/3260684934 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fbaa00305c0 con 0x560413817780 2024-09-12T02:09:37.693 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.687+0000 7fbab0c7d740 10 client.17811 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:09:37.693 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.687+0000 7fbab0c7d740 10 client.17811 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:09:37.693 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.687+0000 7fbab0c7d740 20 client.17811 populate_metadata read hostname 'smithi016' 2024-09-12T02:09:37.694 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.687+0000 7fbab0c7d740 1 --2- 172.21.15.16:0/3260684934 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x560413892c40 0x560413895060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:37.694 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.687+0000 7fbab0c7d740 4 client.17811 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:09:37.694 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.687+0000 7fbab0c7d740 1 -- 172.21.15.16:0/3260684934 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560413778d70 con 0x560413892c40 2024-09-12T02:09:37.694 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.687+0000 7fba9effd640 1 --2- 172.21.15.16:0/3260684934 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x560413892c40 0x560413895060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:37.694 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.688+0000 7fba86ffd640 10 client.17811 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:09:37.694 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.688+0000 7fba9effd640 1 --2- 172.21.15.16:0/3260684934 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x560413892c40 0x560413895060 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f9fcb7f69537372 server_cookie=53abb5620d74f0c0 in_seq=0 out_seq=0 2024-09-12T02:09:37.703 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.702+0000 7fba86ffd640 1 -- 172.21.15.16:0/3260684934 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560413778d70 con 0x560413892c40 2024-09-12T02:09:37.703 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.702+0000 7fba86ffd640 10 client.17811 handle_command_reply: tid=0 2024-09-12T02:09:37.703 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.702+0000 7fbab0c7d740 1 client.17811 shutdown 2024-09-12T02:09:37.703 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.702+0000 7fbab0c7d740 1 -- 172.21.15.16:0/3260684934 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560413817780 msgr2=0x560413822140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:37.703 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.702+0000 7fbab0c7d740 1 --2- 172.21.15.16:0/3260684934 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560413817780 0x560413822140 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7fbaa002ade0 tx=0x7fbaa003d040 comp rx=0 tx=0).stop 2024-09-12T02:09:37.703 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.702+0000 7fbab0c7d740 1 -- 172.21.15.16:0/3260684934 shutdown_connections 2024-09-12T02:09:37.703 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.702+0000 7fbab0c7d740 1 --2- 172.21.15.16:0/3260684934 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x560413892c40 0x560413895060 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:37.703 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.702+0000 7fbab0c7d740 1 --2- 172.21.15.16:0/3260684934 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560413817780 0x560413822140 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:37.703 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.702+0000 7fbab0c7d740 1 --2- 172.21.15.16:0/3260684934 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5604138126f0 0x560413821180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:37.703 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.702+0000 7fbab0c7d740 1 --2- 172.21.15.16:0/3260684934 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560413811db0 0x560413820c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:37.703 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.702+0000 7fba86ffd640 0 client.17811 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:09:37.704 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.702+0000 7fbab0c7d740 1 -- 172.21.15.16:0/3260684934 >> 172.21.15.16:0/3260684934 conn(0x560413809010 msgr2=0x5604138081c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:37.704 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.702+0000 7fbab0c7d740 1 -- 172.21.15.16:0/3260684934 shutdown_connections 2024-09-12T02:09:37.704 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.702+0000 7fbab0c7d740 1 -- 172.21.15.16:0/3260684934 wait complete. 2024-09-12T02:09:37.704 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.702+0000 7fbab0c7d740 20 client.17811 trim_cache size 0 max 16384 2024-09-12T02:09:37.704 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-09-12T02:09:37.704 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-09-12T02:09:37.704 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-09-12T02:09:37.704 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-09-12T02:09:37.705 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.703+0000 7fbaaddfd640 1 -- 172.21.15.16:0/2778291165 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fba940a2c30 msgr2=0x7fba940a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:37.705 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.703+0000 7fbaaddfd640 1 --2- 172.21.15.16:0/2778291165 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fba940a2c30 0x7fba940a50f0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fba9800d740 tx=0x7fba9805b350 comp rx=0 tx=0).stop 2024-09-12T02:09:37.705 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.703+0000 7fbaaddfd640 1 -- 172.21.15.16:0/2778291165 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbaa8100a60 msgr2=0x7fbaa8197da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:37.705 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.703+0000 7fbaaddfd640 1 --2- 172.21.15.16:0/2778291165 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbaa8100a60 0x7fbaa8197da0 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7fba9800d770 tx=0x7fba9800dc30 comp rx=0 tx=0).stop 2024-09-12T02:09:37.705 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.703+0000 7fbaaddfd640 1 -- 172.21.15.16:0/2778291165 shutdown_connections 2024-09-12T02:09:37.706 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.703+0000 7fbaaddfd640 1 --2- 172.21.15.16:0/2778291165 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fba940a2c30 0x7fba940a50f0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:37.706 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.703+0000 7fbaaddfd640 1 --2- 172.21.15.16:0/2778291165 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbaa8102610 0x7fbaa819f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:37.706 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.703+0000 7fbaaddfd640 1 --2- 172.21.15.16:0/2778291165 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbaa8101c50 0x7fbaa81982e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:37.706 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.703+0000 7fbaaddfd640 1 --2- 172.21.15.16:0/2778291165 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbaa8100a60 0x7fbaa8197da0 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:37.706 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.703+0000 7fbaaddfd640 1 -- 172.21.15.16:0/2778291165 >> 172.21.15.16:0/2778291165 conn(0x7fbaa80fc1e0 msgr2=0x7fbaa80fdc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:37.706 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.703+0000 7fbaaddfd640 1 -- 172.21.15.16:0/2778291165 shutdown_connections 2024-09-12T02:09:37.706 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:37.703+0000 7fbaaddfd640 1 -- 172.21.15.16:0/2778291165 wait complete. 2024-09-12T02:09:37.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:09:37 smithi184 ceph-mon[35593]: pgmap v566: 97 pgs: 97 active+clean; 4.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 5.2 KiB/s wr, 10 op/s 2024-09-12T02:09:37.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:09:37 smithi033 ceph-mon[35332]: pgmap v566: 97 pgs: 97 active+clean; 4.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 5.2 KiB/s wr, 10 op/s 2024-09-12T02:09:38.626 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-12T02:09:38.626 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-12T02:09:38.895 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:09:39.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.391+0000 7f7c5d4cd640 1 -- 172.21.15.16:0/36967934 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c58100a80 msgr2=0x7f7c58100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:39.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.391+0000 7f7c5d4cd640 1 --2- 172.21.15.16:0/36967934 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c58100a80 0x7f7c58100ee0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f7c40009930 tx=0x7f7c4002a420 comp rx=0 tx=0).stop 2024-09-12T02:09:39.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.391+0000 7f7c5d4cd640 1 -- 172.21.15.16:0/36967934 shutdown_connections 2024-09-12T02:09:39.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.391+0000 7f7c5d4cd640 1 --2- 172.21.15.16:0/36967934 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7c58101420 0x7f7c5810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:39.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.392+0000 7f7c5d4cd640 1 --2- 172.21.15.16:0/36967934 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c58100a80 0x7f7c58100ee0 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:39.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.392+0000 7f7c5d4cd640 1 --2- 172.21.15.16:0/36967934 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c58106a80 0x7f7c58106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:39.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.392+0000 7f7c5d4cd640 1 -- 172.21.15.16:0/36967934 >> 172.21.15.16:0/36967934 conn(0x7f7c580fc7a0 msgr2=0x7f7c580febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:39.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.392+0000 7f7c5d4cd640 1 -- 172.21.15.16:0/36967934 shutdown_connections 2024-09-12T02:09:39.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.392+0000 7f7c5d4cd640 1 -- 172.21.15.16:0/36967934 wait complete. 2024-09-12T02:09:39.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.392+0000 7f7c5d4cd640 1 Processor -- start 2024-09-12T02:09:39.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.392+0000 7f7c5d4cd640 1 -- start start 2024-09-12T02:09:39.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.392+0000 7f7c5d4cd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c58100a80 0x7f7c58197330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:39.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.392+0000 7f7c5d4cd640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7c58101420 0x7f7c58197870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:39.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.392+0000 7f7c5d4cd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c58106a80 0x7f7c5819bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:39.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.392+0000 7f7c5d4cd640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7c58110e80 con 0x7f7c58100a80 2024-09-12T02:09:39.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.392+0000 7f7c5d4cd640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7c58110d00 con 0x7f7c58106a80 2024-09-12T02:09:39.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.392+0000 7f7c5d4cd640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7c58111000 con 0x7f7c58101420 2024-09-12T02:09:39.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.393+0000 7f7c56ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c58100a80 0x7f7c58197330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:39.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.393+0000 7f7c56ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c58100a80 0x7f7c58197330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39866/0 (socket says 172.21.15.16:39866) 2024-09-12T02:09:39.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.393+0000 7f7c56ffd640 1 -- 172.21.15.16:0/1939023367 learned_addr learned my addr 172.21.15.16:0/1939023367 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:09:39.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.393+0000 7f7c4ffff640 1 --2- 172.21.15.16:0/1939023367 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7c58101420 0x7f7c58197870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:39.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.393+0000 7f7c577fe640 1 --2- 172.21.15.16:0/1939023367 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c58106a80 0x7f7c5819bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:39.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.393+0000 7f7c56ffd640 1 -- 172.21.15.16:0/1939023367 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7c58101420 msgr2=0x7f7c58197870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:39.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.393+0000 7f7c56ffd640 1 --2- 172.21.15.16:0/1939023367 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7c58101420 0x7f7c58197870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:39.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.393+0000 7f7c56ffd640 1 -- 172.21.15.16:0/1939023367 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c58106a80 msgr2=0x7f7c5819bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:39.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.393+0000 7f7c56ffd640 1 --2- 172.21.15.16:0/1939023367 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c58106a80 0x7f7c5819bc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:39.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.393+0000 7f7c56ffd640 1 -- 172.21.15.16:0/1939023367 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c5819c2e0 con 0x7f7c58100a80 2024-09-12T02:09:39.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.393+0000 7f7c56ffd640 1 --2- 172.21.15.16:0/1939023367 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c58100a80 0x7f7c58197330 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f7c48009970 tx=0x7f7c48009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:39.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.393+0000 7f7c54ff9640 1 -- 172.21.15.16:0/1939023367 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c4800d020 con 0x7f7c58100a80 2024-09-12T02:09:39.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.393+0000 7f7c54ff9640 1 -- 172.21.15.16:0/1939023367 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c48007d40 con 0x7f7c58100a80 2024-09-12T02:09:39.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.393+0000 7f7c5d4cd640 1 -- 172.21.15.16:0/1939023367 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c5819c5d0 con 0x7f7c58100a80 2024-09-12T02:09:39.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.393+0000 7f7c54ff9640 1 -- 172.21.15.16:0/1939023367 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c4800ccc0 con 0x7f7c58100a80 2024-09-12T02:09:39.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.393+0000 7f7c5d4cd640 1 -- 172.21.15.16:0/1939023367 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c5806a790 con 0x7f7c58100a80 2024-09-12T02:09:39.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.396+0000 7f7c54ff9640 1 -- 172.21.15.16:0/1939023367 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f7c4800c460 con 0x7f7c58100a80 2024-09-12T02:09:39.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.396+0000 7f7c5f8a6740 1 Processor -- start 2024-09-12T02:09:39.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.396+0000 7f7c5f8a6740 1 -- start start 2024-09-12T02:09:39.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.396+0000 7f7c5f8a6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55edf054db60 0x55edf054df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:39.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.396+0000 7f7c5f8a6740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55edf054e480 0x55edf0558800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:39.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.396+0000 7f7c5f8a6740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55edf0558d40 0x55edf055b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:39.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.396+0000 7f7c5f8a6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55edf04b3280 con 0x55edf054db60 2024-09-12T02:09:39.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.396+0000 7f7c5f8a6740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55edf045ab10 con 0x55edf054e480 2024-09-12T02:09:39.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.396+0000 7f7c5f8a6740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55edf04af240 con 0x55edf0558d40 2024-09-12T02:09:39.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.397+0000 7f7c54ff9640 1 --2- 172.21.15.16:0/1939023367 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f7c440a2c00 0x7f7c440a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:39.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.397+0000 7f7c577fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55edf054db60 0x55edf054df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:39.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.397+0000 7f7c577fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55edf054db60 0x55edf054df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39876/0 (socket says 172.21.15.16:39876) 2024-09-12T02:09:39.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.397+0000 7f7c577fe640 1 -- 172.21.15.16:0/2837718404 learned_addr learned my addr 172.21.15.16:0/2837718404 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:09:39.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.397+0000 7f7c56ffd640 1 --2- 172.21.15.16:0/2837718404 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55edf054e480 0x55edf0558800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:39.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.397+0000 7f7c4ffff640 1 --2- 172.21.15.16:0/2837718404 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55edf0558d40 0x55edf055b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:39.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.397+0000 7f7c54ff9640 1 -- 172.21.15.16:0/1939023367 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f7c480d0230 con 0x7f7c58100a80 2024-09-12T02:09:39.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.397+0000 7f7c577fe640 1 --2- 172.21.15.16:0/1939023367 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f7c440a2c00 0x7f7c440a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:39.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.397+0000 7f7c577fe640 1 -- 172.21.15.16:0/2837718404 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55edf0558d40 msgr2=0x55edf055b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:09:39.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.397+0000 7f7c577fe640 1 --2- 172.21.15.16:0/2837718404 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55edf0558d40 0x55edf055b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:39.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.397+0000 7f7c577fe640 1 -- 172.21.15.16:0/2837718404 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55edf054e480 msgr2=0x55edf0558800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:09:39.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.397+0000 7f7c577fe640 1 --2- 172.21.15.16:0/2837718404 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55edf054e480 0x55edf0558800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:39.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.397+0000 7f7c577fe640 1 -- 172.21.15.16:0/2837718404 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55edf04671f0 con 0x55edf054db60 2024-09-12T02:09:39.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.397+0000 7f7c56ffd640 1 --2- 172.21.15.16:0/2837718404 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55edf054e480 0x55edf0558800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:09:39.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.397+0000 7f7c577fe640 1 --2- 172.21.15.16:0/2837718404 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55edf054db60 0x55edf054df40 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f7c50005f40 tx=0x7f7c50012760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f61ab6c14f41a703 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:39.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.397+0000 7f7c577fe640 1 --2- 172.21.15.16:0/1939023367 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f7c440a2c00 0x7f7c440a50c0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f7c50012c60 tx=0x7f7c5000b000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:39.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.398+0000 7f7c4d7fa640 1 -- 172.21.15.16:0/2837718404 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c50024070 con 0x55edf054db60 2024-09-12T02:09:39.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.398+0000 7f7c4d7fa640 1 -- 172.21.15.16:0/2837718404 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c50014590 con 0x55edf054db60 2024-09-12T02:09:39.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.398+0000 7f7c4d7fa640 1 -- 172.21.15.16:0/2837718404 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c5000ee10 con 0x55edf054db60 2024-09-12T02:09:39.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.398+0000 7f7c5f8a6740 1 -- 172.21.15.16:0/2837718404 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55edf054db60 msgr2=0x55edf054df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:39.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.398+0000 7f7c5f8a6740 1 --2- 172.21.15.16:0/2837718404 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55edf054db60 0x55edf054df40 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f7c50005f40 tx=0x7f7c50012760 comp rx=0 tx=0).stop 2024-09-12T02:09:39.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.398+0000 7f7c5f8a6740 1 -- 172.21.15.16:0/2837718404 shutdown_connections 2024-09-12T02:09:39.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.398+0000 7f7c5f8a6740 1 --2- 172.21.15.16:0/2837718404 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55edf0558d40 0x55edf055b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:39.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.398+0000 7f7c5f8a6740 1 --2- 172.21.15.16:0/2837718404 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55edf054e480 0x55edf0558800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:39.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.398+0000 7f7c5f8a6740 1 --2- 172.21.15.16:0/2837718404 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55edf054db60 0x55edf054df40 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:39.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.398+0000 7f7c5f8a6740 1 -- 172.21.15.16:0/2837718404 >> 172.21.15.16:0/2837718404 conn(0x55edf054a0f0 msgr2=0x55edf054a4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:39.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.398+0000 7f7c5f8a6740 1 -- 172.21.15.16:0/2837718404 shutdown_connections 2024-09-12T02:09:39.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.398+0000 7f7c5f8a6740 1 -- 172.21.15.16:0/2837718404 wait complete. 2024-09-12T02:09:39.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.399+0000 7f7c5f8a6740 1 -- start start 2024-09-12T02:09:39.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.399+0000 7f7c5f8a6740 1 Processor -- start 2024-09-12T02:09:39.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.399+0000 7f7c5f8a6740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55edf054db60 0x55edf0555d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:39.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.399+0000 7f7c5f8a6740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55edf054e480 0x55edf0556240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:39.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.399+0000 7f7c5f8a6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55edf0558d40 0x55edf0556780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:39.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.399+0000 7f7c5f8a6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55edf055b910 con 0x55edf0558d40 2024-09-12T02:09:39.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.399+0000 7f7c5f8a6740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55edf046df10 con 0x55edf054db60 2024-09-12T02:09:39.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.399+0000 7f7c5f8a6740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55edf055ba90 con 0x55edf054e480 2024-09-12T02:09:39.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.399+0000 7f7c5f8a6740 10 client.0 _ll_register_callbacks cb 0x55edf0436a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:09:39.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.399+0000 7f7c577fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55edf0558d40 0x55edf0556780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:39.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.399+0000 7f7c577fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55edf0558d40 0x55edf0556780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39892/0 (socket says 172.21.15.16:39892) 2024-09-12T02:09:39.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.399+0000 7f7c577fe640 1 -- 172.21.15.16:0/1473988762 learned_addr learned my addr 172.21.15.16:0/1473988762 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:09:39.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.399+0000 7f7c56ffd640 1 --2- 172.21.15.16:0/1473988762 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55edf054db60 0x55edf0555d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:39.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.399+0000 7f7c4ffff640 1 --2- 172.21.15.16:0/1473988762 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55edf054e480 0x55edf0556240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:39.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.400+0000 7f7c577fe640 1 -- 172.21.15.16:0/1473988762 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55edf054e480 msgr2=0x55edf0556240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:09:39.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.400+0000 7f7c577fe640 1 --2- 172.21.15.16:0/1473988762 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55edf054e480 0x55edf0556240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:39.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.400+0000 7f7c577fe640 1 -- 172.21.15.16:0/1473988762 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55edf054db60 msgr2=0x55edf0555d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:09:39.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.400+0000 7f7c577fe640 1 --2- 172.21.15.16:0/1473988762 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55edf054db60 0x55edf0555d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:39.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.400+0000 7f7c577fe640 1 -- 172.21.15.16:0/1473988762 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55edf02246a0 con 0x55edf0558d40 2024-09-12T02:09:39.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.400+0000 7f7c577fe640 1 --2- 172.21.15.16:0/1473988762 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55edf0558d40 0x55edf0556780 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f7c50005f10 tx=0x7f7c50014050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c71147546281f6f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:39.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.400+0000 7f7c3e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:09:39.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.400+0000 7f7c3e7fc640 1 -- 172.21.15.16:0/1473988762 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c5002e020 con 0x55edf0558d40 2024-09-12T02:09:39.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.400+0000 7f7c3e7fc640 1 -- 172.21.15.16:0/1473988762 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f7c5000ebf0 con 0x55edf0558d40 2024-09-12T02:09:39.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.400+0000 7f7c5f8a6740 1 -- 172.21.15.16:0/1473988762 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55edf04c07f0 con 0x55edf0558d40 2024-09-12T02:09:39.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.400+0000 7f7c3e7fc640 1 -- 172.21.15.16:0/1473988762 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7c50024040 con 0x55edf0558d40 2024-09-12T02:09:39.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.400+0000 7f7c5f8a6740 10 client.17823 fetch_fsmap learned FSMap version 25 2024-09-12T02:09:39.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.400+0000 7f7c5f8a6740 1 -- 172.21.15.16:0/1473988762 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55edf05675f0 con 0x55edf0558d40 2024-09-12T02:09:39.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.400+0000 7f7c3e7fc640 1 -- 172.21.15.16:0/1473988762 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55edf04c07f0 con 0x55edf0558d40 2024-09-12T02:09:39.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.400+0000 7f7c3e7fc640 1 -- 172.21.15.16:0/1473988762 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f7c5000f040 con 0x55edf0558d40 2024-09-12T02:09:39.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.401+0000 7f7c5f8a6740 10 client.17823 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:09:39.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.401+0000 7f7c5f8a6740 10 client.17823 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:09:39.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.401+0000 7f7c5f8a6740 20 client.17823 populate_metadata read hostname 'smithi016' 2024-09-12T02:09:39.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.401+0000 7f7c5f8a6740 1 --2- 172.21.15.16:0/1473988762 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55edf0568ab0 0x55edf056aea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:39.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.401+0000 7f7c5f8a6740 4 client.17823 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:09:39.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.401+0000 7f7c5f8a6740 1 -- 172.21.15.16:0/1473988762 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55edf04bad00 con 0x55edf0568ab0 2024-09-12T02:09:39.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.401+0000 7f7c56ffd640 1 --2- 172.21.15.16:0/1473988762 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55edf0568ab0 0x55edf056aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:39.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.401+0000 7f7c56ffd640 1 --2- 172.21.15.16:0/1473988762 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55edf0568ab0 0x55edf056aea0 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4286f121c8672305 server_cookie=7420ca45d8fe0999 in_seq=0 out_seq=0 2024-09-12T02:09:39.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.401+0000 7f7c3e7fc640 10 client.17823 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:09:39.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.402+0000 7f7c3e7fc640 1 -- 172.21.15.16:0/1473988762 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55edf04bad00 con 0x55edf0568ab0 2024-09-12T02:09:39.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.402+0000 7f7c3e7fc640 10 client.17823 handle_command_reply: tid=0 2024-09-12T02:09:39.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.402+0000 7f7c5f8a6740 1 client.17823 shutdown 2024-09-12T02:09:39.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.402+0000 7f7c5f8a6740 1 -- 172.21.15.16:0/1473988762 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55edf0558d40 msgr2=0x55edf0556780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:39.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.402+0000 7f7c5f8a6740 1 --2- 172.21.15.16:0/1473988762 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55edf0558d40 0x55edf0556780 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f7c50005f10 tx=0x7f7c50014050 comp rx=0 tx=0).stop 2024-09-12T02:09:39.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.402+0000 7f7c5f8a6740 1 -- 172.21.15.16:0/1473988762 shutdown_connections 2024-09-12T02:09:39.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.402+0000 7f7c5f8a6740 1 --2- 172.21.15.16:0/1473988762 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55edf0568ab0 0x55edf056aea0 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:39.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.402+0000 7f7c5f8a6740 1 --2- 172.21.15.16:0/1473988762 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55edf0558d40 0x55edf0556780 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:39.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.402+0000 7f7c5f8a6740 1 --2- 172.21.15.16:0/1473988762 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55edf054e480 0x55edf0556240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:39.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.402+0000 7f7c3e7fc640 0 client.17823 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:09:39.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.402+0000 7f7c5f8a6740 1 --2- 172.21.15.16:0/1473988762 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55edf054db60 0x55edf0555d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:39.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.402+0000 7f7c5f8a6740 1 -- 172.21.15.16:0/1473988762 >> 172.21.15.16:0/1473988762 conn(0x55edf054a0f0 msgr2=0x55edf0549270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:39.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.402+0000 7f7c5f8a6740 1 -- 172.21.15.16:0/1473988762 shutdown_connections 2024-09-12T02:09:39.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.403+0000 7f7c5f8a6740 1 -- 172.21.15.16:0/1473988762 wait complete. 2024-09-12T02:09:39.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.403+0000 7f7c5f8a6740 20 client.17823 trim_cache size 0 max 16384 2024-09-12T02:09:39.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.420+0000 7f7c5f8a6740 1 Processor -- start 2024-09-12T02:09:39.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.420+0000 7f7c5f8a6740 1 -- start start 2024-09-12T02:09:39.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.420+0000 7f7c5f8a6740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55edf054db60 0x55edf0550610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:39.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.420+0000 7f7c5f8a6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55edf054e480 0x55edf0570560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:39.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.420+0000 7f7c5f8a6740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55edf0558d40 0x55edf0570aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:39.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.420+0000 7f7c5f8a6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7c581ada50 con 0x55edf054e480 2024-09-12T02:09:39.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.420+0000 7f7c5f8a6740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7c581ab320 con 0x55edf0558d40 2024-09-12T02:09:39.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.420+0000 7f7c5f8a6740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7c581a91b0 con 0x55edf054db60 2024-09-12T02:09:39.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.421+0000 7f7c4ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55edf054e480 0x55edf0570560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:39.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.421+0000 7f7c4ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55edf054e480 0x55edf0570560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39906/0 (socket says 172.21.15.16:39906) 2024-09-12T02:09:39.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.421+0000 7f7c4ffff640 1 -- 172.21.15.16:0/2386304462 learned_addr learned my addr 172.21.15.16:0/2386304462 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:09:39.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.421+0000 7f7c56ffd640 1 --2- 172.21.15.16:0/2386304462 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55edf054db60 0x55edf0550610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:39.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.421+0000 7f7c577fe640 1 --2- 172.21.15.16:0/2386304462 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55edf0558d40 0x55edf0570aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:39.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.421+0000 7f7c4ffff640 1 -- 172.21.15.16:0/2386304462 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55edf054db60 msgr2=0x55edf0550610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:09:39.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.421+0000 7f7c4ffff640 1 --2- 172.21.15.16:0/2386304462 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55edf054db60 0x55edf0550610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:39.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.421+0000 7f7c4ffff640 1 -- 172.21.15.16:0/2386304462 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55edf0558d40 msgr2=0x55edf0570aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:09:39.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.421+0000 7f7c4ffff640 1 --2- 172.21.15.16:0/2386304462 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55edf0558d40 0x55edf0570aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:39.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.421+0000 7f7c4ffff640 1 -- 172.21.15.16:0/2386304462 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55edf054b1a0 con 0x55edf054e480 2024-09-12T02:09:39.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.421+0000 7f7c577fe640 1 --2- 172.21.15.16:0/2386304462 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55edf0558d40 0x55edf0570aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:09:39.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.421+0000 7f7c4ffff640 1 --2- 172.21.15.16:0/2386304462 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55edf054e480 0x55edf0570560 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f7c4002a940 tx=0x7f7c40031660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24aa94c9e372c7a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:39.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.421+0000 7f7c4cff9640 1 -- 172.21.15.16:0/2386304462 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c40035040 con 0x55edf054e480 2024-09-12T02:09:39.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.421+0000 7f7c4cff9640 1 -- 172.21.15.16:0/2386304462 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c400035f0 con 0x55edf054e480 2024-09-12T02:09:39.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.421+0000 7f7c4cff9640 1 -- 172.21.15.16:0/2386304462 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c4003d6b0 con 0x55edf054e480 2024-09-12T02:09:39.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.422+0000 7f7c5f8a6740 1 -- 172.21.15.16:0/2386304462 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55edf054e480 msgr2=0x55edf0570560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:39.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.422+0000 7f7c5f8a6740 1 --2- 172.21.15.16:0/2386304462 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55edf054e480 0x55edf0570560 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f7c4002a940 tx=0x7f7c40031660 comp rx=0 tx=0).stop 2024-09-12T02:09:39.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.422+0000 7f7c5f8a6740 1 -- 172.21.15.16:0/2386304462 shutdown_connections 2024-09-12T02:09:39.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.422+0000 7f7c5f8a6740 1 --2- 172.21.15.16:0/2386304462 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55edf0558d40 0x55edf0570aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:39.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.422+0000 7f7c5f8a6740 1 --2- 172.21.15.16:0/2386304462 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55edf054e480 0x55edf0570560 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:39.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.422+0000 7f7c5f8a6740 1 --2- 172.21.15.16:0/2386304462 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55edf054db60 0x55edf0550610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:39.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.422+0000 7f7c5f8a6740 1 -- 172.21.15.16:0/2386304462 >> 172.21.15.16:0/2386304462 conn(0x55edf054a0f0 msgr2=0x55edf0557c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:39.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.422+0000 7f7c5f8a6740 1 -- 172.21.15.16:0/2386304462 shutdown_connections 2024-09-12T02:09:39.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.422+0000 7f7c5f8a6740 1 -- 172.21.15.16:0/2386304462 wait complete. 2024-09-12T02:09:39.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.422+0000 7f7c5f8a6740 1 -- start start 2024-09-12T02:09:39.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.423+0000 7f7c5f8a6740 1 Processor -- start 2024-09-12T02:09:39.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.423+0000 7f7c5f8a6740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55edf054db60 0x55edf055cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:39.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.423+0000 7f7c5f8a6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55edf054e480 0x55edf055d180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:39.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.423+0000 7f7c5f8a6740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55edf0558d40 0x55edf055d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:39.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.423+0000 7f7c5f8a6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7c581a9800 con 0x55edf054e480 2024-09-12T02:09:39.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.423+0000 7f7c5f8a6740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7c581aac50 con 0x55edf0558d40 2024-09-12T02:09:39.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.423+0000 7f7c5f8a6740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7c581a9eb0 con 0x55edf054db60 2024-09-12T02:09:39.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.423+0000 7f7c5f8a6740 10 client.0 _ll_register_callbacks cb 0x7f7c50022070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:09:39.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.423+0000 7f7c4ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55edf054e480 0x55edf055d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:39.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.423+0000 7f7c4ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55edf054e480 0x55edf055d180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39918/0 (socket says 172.21.15.16:39918) 2024-09-12T02:09:39.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.423+0000 7f7c56ffd640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55edf054db60 0x55edf055cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:39.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.423+0000 7f7c4ffff640 1 -- 172.21.15.16:0/1236345354 learned_addr learned my addr 172.21.15.16:0/1236345354 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:09:39.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.423+0000 7f7c577fe640 1 --2- 172.21.15.16:0/1236345354 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55edf0558d40 0x55edf055d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:39.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.423+0000 7f7c4ffff640 1 -- 172.21.15.16:0/1236345354 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55edf054db60 msgr2=0x55edf055cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:09:39.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.423+0000 7f7c4ffff640 1 --2- 172.21.15.16:0/1236345354 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55edf054db60 0x55edf055cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:39.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.423+0000 7f7c4ffff640 1 -- 172.21.15.16:0/1236345354 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55edf0558d40 msgr2=0x55edf055d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:09:39.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.423+0000 7f7c4ffff640 1 --2- 172.21.15.16:0/1236345354 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55edf0558d40 0x55edf055d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:39.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.423+0000 7f7c4ffff640 1 -- 172.21.15.16:0/1236345354 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55edf05d2730 con 0x55edf054e480 2024-09-12T02:09:39.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.423+0000 7f7c577fe640 1 --2- 172.21.15.16:0/1236345354 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55edf0558d40 0x55edf055d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:09:39.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.424+0000 7f7c4ffff640 1 --2- 172.21.15.16:0/1236345354 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55edf054e480 0x55edf055d180 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f7c40006010 tx=0x7f7c40004230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7217d06b2a8c058 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:39.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.424+0000 7f7c3e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:09:39.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.424+0000 7f7c3e7fc640 1 -- 172.21.15.16:0/1236345354 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c40035030 con 0x55edf054e480 2024-09-12T02:09:39.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.424+0000 7f7c3e7fc640 1 -- 172.21.15.16:0/1236345354 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f7c400039a0 con 0x55edf054e480 2024-09-12T02:09:39.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.424+0000 7f7c5f8a6740 1 -- 172.21.15.16:0/1236345354 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c400035f0 con 0x55edf054e480 2024-09-12T02:09:39.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.424+0000 7f7c3e7fc640 1 -- 172.21.15.16:0/1236345354 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7c40003ba0 con 0x55edf054e480 2024-09-12T02:09:39.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.424+0000 7f7c5f8a6740 10 client.17829 fetch_fsmap learned FSMap version 25 2024-09-12T02:09:39.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.424+0000 7f7c5f8a6740 1 -- 172.21.15.16:0/1236345354 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55edf05d3a20 con 0x55edf054e480 2024-09-12T02:09:39.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.424+0000 7f7c3e7fc640 1 -- 172.21.15.16:0/1236345354 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c400035f0 con 0x55edf054e480 2024-09-12T02:09:39.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.424+0000 7f7c3e7fc640 1 -- 172.21.15.16:0/1236345354 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f7c40004bb0 con 0x55edf054e480 2024-09-12T02:09:39.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.424+0000 7f7c5f8a6740 10 client.17829 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:09:39.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.424+0000 7f7c5f8a6740 10 client.17829 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:09:39.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.424+0000 7f7c5f8a6740 20 client.17829 populate_metadata read hostname 'smithi016' 2024-09-12T02:09:39.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.424+0000 7f7c5f8a6740 1 --2- 172.21.15.16:0/1236345354 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55edf05d4120 0x55edf05d6540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:39.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.424+0000 7f7c5f8a6740 4 client.17829 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-12T02:09:39.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.424+0000 7f7c5f8a6740 1 -- 172.21.15.16:0/1236345354 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55edf0437110 con 0x55edf05d4120 2024-09-12T02:09:39.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.425+0000 7f7c56ffd640 1 --2- 172.21.15.16:0/1236345354 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55edf05d4120 0x55edf05d6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:39.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.425+0000 7f7c3e7fc640 10 client.17829 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:09:39.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.425+0000 7f7c56ffd640 1 --2- 172.21.15.16:0/1236345354 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55edf05d4120 0x55edf05d6540 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d5ca7e0a6b272e0 server_cookie=d946e8d717a8c282 in_seq=0 out_seq=0 2024-09-12T02:09:39.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.426+0000 7f7c3e7fc640 1 -- 172.21.15.16:0/1236345354 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55edf0437110 con 0x55edf05d4120 2024-09-12T02:09:39.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.426+0000 7f7c3e7fc640 10 client.17829 handle_command_reply: tid=0 2024-09-12T02:09:39.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.426+0000 7f7c5f8a6740 1 client.17829 shutdown 2024-09-12T02:09:39.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.426+0000 7f7c5f8a6740 1 -- 172.21.15.16:0/1236345354 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55edf054e480 msgr2=0x55edf055d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:39.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.426+0000 7f7c5f8a6740 1 --2- 172.21.15.16:0/1236345354 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55edf054e480 0x55edf055d180 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f7c40006010 tx=0x7f7c40004230 comp rx=0 tx=0).stop 2024-09-12T02:09:39.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.426+0000 7f7c5f8a6740 1 -- 172.21.15.16:0/1236345354 shutdown_connections 2024-09-12T02:09:39.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.426+0000 7f7c5f8a6740 1 --2- 172.21.15.16:0/1236345354 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55edf05d4120 0x55edf05d6540 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:39.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.426+0000 7f7c5f8a6740 1 --2- 172.21.15.16:0/1236345354 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55edf0558d40 0x55edf055d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:39.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.426+0000 7f7c5f8a6740 1 --2- 172.21.15.16:0/1236345354 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55edf054e480 0x55edf055d180 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:39.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.426+0000 7f7c5f8a6740 1 --2- 172.21.15.16:0/1236345354 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55edf054db60 0x55edf055cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:39.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.426+0000 7f7c5f8a6740 1 -- 172.21.15.16:0/1236345354 >> 172.21.15.16:0/1236345354 conn(0x55edf054a0f0 msgr2=0x55edf0549270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:39.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.426+0000 7f7c3e7fc640 0 client.17829 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:09:39.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.426+0000 7f7c5f8a6740 1 -- 172.21.15.16:0/1236345354 shutdown_connections 2024-09-12T02:09:39.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.426+0000 7f7c5f8a6740 1 -- 172.21.15.16:0/1236345354 wait complete. 2024-09-12T02:09:39.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.427+0000 7f7c5f8a6740 20 client.17829 trim_cache size 0 max 16384 2024-09-12T02:09:39.439 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T02:09:39.439 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-09-12T02:09:39.439 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "c9b98abe-56b9-4ec8-a213-2e79d55cf1a3", 2024-09-12T02:09:39.439 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-09-12T02:09:39.439 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T02:09:39.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.427+0000 7f7c5d4cd640 1 -- 172.21.15.16:0/1939023367 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f7c440a2c00 msgr2=0x7f7c440a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:39.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.427+0000 7f7c5d4cd640 1 --2- 172.21.15.16:0/1939023367 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f7c440a2c00 0x7f7c440a50c0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f7c50012c60 tx=0x7f7c5000b000 comp rx=0 tx=0).stop 2024-09-12T02:09:39.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.427+0000 7f7c5d4cd640 1 -- 172.21.15.16:0/1939023367 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c58100a80 msgr2=0x7f7c58197330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:39.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.427+0000 7f7c5d4cd640 1 --2- 172.21.15.16:0/1939023367 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c58100a80 0x7f7c58197330 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f7c48009970 tx=0x7f7c48009e30 comp rx=0 tx=0).stop 2024-09-12T02:09:39.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.427+0000 7f7c5d4cd640 1 -- 172.21.15.16:0/1939023367 shutdown_connections 2024-09-12T02:09:39.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.427+0000 7f7c5d4cd640 1 --2- 172.21.15.16:0/1939023367 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f7c440a2c00 0x7f7c440a50c0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:39.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.427+0000 7f7c5d4cd640 1 --2- 172.21.15.16:0/1939023367 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c58106a80 0x7f7c5819bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:39.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.427+0000 7f7c5d4cd640 1 --2- 172.21.15.16:0/1939023367 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7c58101420 0x7f7c58197870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:39.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.427+0000 7f7c5d4cd640 1 --2- 172.21.15.16:0/1939023367 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c58100a80 0x7f7c58197330 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:39.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.427+0000 7f7c5d4cd640 1 -- 172.21.15.16:0/1939023367 >> 172.21.15.16:0/1939023367 conn(0x7f7c580fc7a0 msgr2=0x7f7c580fcfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:39.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.427+0000 7f7c5d4cd640 1 -- 172.21.15.16:0/1939023367 shutdown_connections 2024-09-12T02:09:39.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:39.427+0000 7f7c5d4cd640 1 -- 172.21.15.16:0/1939023367 wait complete. 2024-09-12T02:09:39.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:09:39 smithi016 ceph-mon[30947]: pgmap v567: 97 pgs: 97 active+clean; 4.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 5.0 KiB/s wr, 7 op/s 2024-09-12T02:09:39.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:09:39 smithi184 ceph-mon[35593]: pgmap v567: 97 pgs: 97 active+clean; 4.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 5.0 KiB/s wr, 7 op/s 2024-09-12T02:09:39.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:09:39 smithi033 ceph-mon[35332]: pgmap v567: 97 pgs: 97 active+clean; 4.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 5.0 KiB/s wr, 7 op/s 2024-09-12T02:09:40.116 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T02:09:40.373 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:09:40.814 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:09:40 smithi016 ceph-mon[30947]: scrub queued for path: / 2024-09-12T02:09:40.814 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:09:40 smithi016 ceph-mon[30947]: scrub summary: idle+waiting paths [/] 2024-09-12T02:09:40.814 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:09:40 smithi016 ceph-mon[30947]: scrub summary: active paths [/] 2024-09-12T02:09:40.814 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:09:40 smithi016 ceph-mon[30947]: scrub summary: idle+waiting paths [/] 2024-09-12T02:09:40.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:09:40 smithi184 ceph-mon[35593]: scrub queued for path: / 2024-09-12T02:09:40.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:09:40 smithi184 ceph-mon[35593]: scrub summary: idle+waiting paths [/] 2024-09-12T02:09:40.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:09:40 smithi184 ceph-mon[35593]: scrub summary: active paths [/] 2024-09-12T02:09:40.878 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:09:40 smithi184 ceph-mon[35593]: scrub summary: idle+waiting paths [/] 2024-09-12T02:09:40.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.897+0000 7f6b67d11640 1 -- 172.21.15.16:0/2918295293 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b60101de0 msgr2=0x7f6b601021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:40.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.897+0000 7f6b67d11640 1 --2- 172.21.15.16:0/2918295293 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b60101de0 0x7f6b601021c0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c009900 tx=0x7f6b5c02a410 comp rx=0 tx=0).stop 2024-09-12T02:09:40.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.897+0000 7f6b67d11640 1 -- 172.21.15.16:0/2918295293 shutdown_connections 2024-09-12T02:09:40.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.897+0000 7f6b67d11640 1 --2- 172.21.15.16:0/2918295293 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6b60106f10 0x7f6b6010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:40.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.897+0000 7f6b67d11640 1 --2- 172.21.15.16:0/2918295293 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b60102790 0x7f6b601067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:40.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.897+0000 7f6b67d11640 1 --2- 172.21.15.16:0/2918295293 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b60101de0 0x7f6b601021c0 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:40.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.897+0000 7f6b67d11640 1 -- 172.21.15.16:0/2918295293 >> 172.21.15.16:0/2918295293 conn(0x7f6b600fd8c0 msgr2=0x7f6b600ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:40.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.897+0000 7f6b67d11640 1 -- 172.21.15.16:0/2918295293 shutdown_connections 2024-09-12T02:09:40.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.897+0000 7f6b67d11640 1 -- 172.21.15.16:0/2918295293 wait complete. 2024-09-12T02:09:40.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.898+0000 7f6b67d11640 1 Processor -- start 2024-09-12T02:09:40.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.898+0000 7f6b67d11640 1 -- start start 2024-09-12T02:09:40.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.898+0000 7f6b67d11640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6b60101de0 0x7f6b600744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:40.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.898+0000 7f6b67d11640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b60102790 0x7f6b60074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:40.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.898+0000 7f6b67d11640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b60106f10 0x7f6b60074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:40.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.898+0000 7f6b67d11640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6b60111f30 con 0x7f6b60102790 2024-09-12T02:09:40.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.898+0000 7f6b67d11640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6b60111db0 con 0x7f6b60106f10 2024-09-12T02:09:40.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.898+0000 7f6b67d11640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f6b601120b0 con 0x7f6b60101de0 2024-09-12T02:09:40.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.898+0000 7f6b652ac640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b60102790 0x7f6b60074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:40.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.898+0000 7f6b652ac640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b60102790 0x7f6b60074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39942/0 (socket says 172.21.15.16:39942) 2024-09-12T02:09:40.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.898+0000 7f6b652ac640 1 -- 172.21.15.16:0/4221173552 learned_addr learned my addr 172.21.15.16:0/4221173552 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:09:40.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.899+0000 7f6b65aad640 1 --2- 172.21.15.16:0/4221173552 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6b60101de0 0x7f6b600744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:40.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.899+0000 7f6b662ae640 1 --2- 172.21.15.16:0/4221173552 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b60106f10 0x7f6b60074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:40.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.899+0000 7f6b652ac640 1 -- 172.21.15.16:0/4221173552 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6b60101de0 msgr2=0x7f6b600744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:40.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.899+0000 7f6b652ac640 1 --2- 172.21.15.16:0/4221173552 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6b60101de0 0x7f6b600744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:40.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.899+0000 7f6b652ac640 1 -- 172.21.15.16:0/4221173552 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b60106f10 msgr2=0x7f6b60074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:40.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.899+0000 7f6b652ac640 1 --2- 172.21.15.16:0/4221173552 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b60106f10 0x7f6b60074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:40.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.899+0000 7f6b652ac640 1 -- 172.21.15.16:0/4221173552 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b601a6980 con 0x7f6b60102790 2024-09-12T02:09:40.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.899+0000 7f6b65aad640 1 --2- 172.21.15.16:0/4221173552 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6b60101de0 0x7f6b600744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:09:40.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.899+0000 7f6b662ae640 1 --2- 172.21.15.16:0/4221173552 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b60106f10 0x7f6b60074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:09:40.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.899+0000 7f6b652ac640 1 --2- 172.21.15.16:0/4221173552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b60102790 0x7f6b60074a30 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f6b5000d950 tx=0x7f6b5000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:40.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.899+0000 7f6b56ffd640 1 -- 172.21.15.16:0/4221173552 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b50017030 con 0x7f6b60102790 2024-09-12T02:09:40.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.899+0000 7f6b56ffd640 1 -- 172.21.15.16:0/4221173552 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b50004020 con 0x7f6b60102790 2024-09-12T02:09:40.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.899+0000 7f6b67d11640 1 -- 172.21.15.16:0/4221173552 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b601a6c70 con 0x7f6b60102790 2024-09-12T02:09:40.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.899+0000 7f6b56ffd640 1 -- 172.21.15.16:0/4221173552 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b500047a0 con 0x7f6b60102790 2024-09-12T02:09:40.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.899+0000 7f6b67d11640 1 -- 172.21.15.16:0/4221173552 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b601a71b0 con 0x7f6b60102790 2024-09-12T02:09:40.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.901+0000 7f6b56ffd640 1 -- 172.21.15.16:0/4221173552 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6b500041d0 con 0x7f6b60102790 2024-09-12T02:09:40.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.901+0000 7f6b6a0ea740 1 Processor -- start 2024-09-12T02:09:40.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.902+0000 7f6b6a0ea740 1 -- start start 2024-09-12T02:09:40.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.902+0000 7f6b6a0ea740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56524cf6d4a0 0x56524cf6d880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:40.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.902+0000 7f6b6a0ea740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56524cf6ddc0 0x56524cf78140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:40.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.902+0000 7f6b6a0ea740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56524cf78680 0x56524cf7aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:40.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.902+0000 7f6b6a0ea740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56524ced32e0 con 0x56524cf78680 2024-09-12T02:09:40.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.902+0000 7f6b6a0ea740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56524ce7ace0 con 0x56524cf6ddc0 2024-09-12T02:09:40.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.902+0000 7f6b6a0ea740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x56524cecf2a0 con 0x56524cf6d4a0 2024-09-12T02:09:40.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.902+0000 7f6b652ac640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56524cf78680 0x56524cf7aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:40.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.902+0000 7f6b652ac640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56524cf78680 0x56524cf7aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39952/0 (socket says 172.21.15.16:39952) 2024-09-12T02:09:40.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.902+0000 7f6b652ac640 1 -- 172.21.15.16:0/3586579776 learned_addr learned my addr 172.21.15.16:0/3586579776 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:09:40.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.902+0000 7f6b56ffd640 1 --2- 172.21.15.16:0/4221173552 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f6b4c0a2b80 0x7f6b4c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:40.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.902+0000 7f6b65aad640 1 --2- 172.21.15.16:0/3586579776 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56524cf6ddc0 0x56524cf78140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:40.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.902+0000 7f6b662ae640 1 --2- 172.21.15.16:0/3586579776 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56524cf6d4a0 0x56524cf6d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:40.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.903+0000 7f6b662ae640 1 --2- 172.21.15.16:0/4221173552 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f6b4c0a2b80 0x7f6b4c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:40.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.903+0000 7f6b56ffd640 1 -- 172.21.15.16:0/4221173552 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f6b500cea40 con 0x7f6b60102790 2024-09-12T02:09:40.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.903+0000 7f6b652ac640 1 -- 172.21.15.16:0/3586579776 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56524cf6d4a0 msgr2=0x56524cf6d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:09:40.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.903+0000 7f6b652ac640 1 --2- 172.21.15.16:0/3586579776 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56524cf6d4a0 0x56524cf6d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:40.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.903+0000 7f6b652ac640 1 -- 172.21.15.16:0/3586579776 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56524cf6ddc0 msgr2=0x56524cf78140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:09:40.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.903+0000 7f6b652ac640 1 --2- 172.21.15.16:0/3586579776 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56524cf6ddc0 0x56524cf78140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:40.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.903+0000 7f6b652ac640 1 -- 172.21.15.16:0/3586579776 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56524ce5f440 con 0x56524cf78680 2024-09-12T02:09:40.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.903+0000 7f6b65aad640 1 --2- 172.21.15.16:0/3586579776 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56524cf6ddc0 0x56524cf78140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:09:40.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.903+0000 7f6b652ac640 1 --2- 172.21.15.16:0/3586579776 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56524cf78680 0x56524cf7aa70 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f6b500977a0 tx=0x7f6b50000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7874ec4226b41687 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:40.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.903+0000 7f6b662ae640 1 --2- 172.21.15.16:0/4221173552 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f6b4c0a2b80 0x7f6b4c0a5040 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f6b58005c40 tx=0x7f6b58009200 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:40.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.903+0000 7f6b4bfff640 1 -- 172.21.15.16:0/3586579776 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b5009ae80 con 0x56524cf78680 2024-09-12T02:09:40.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.903+0000 7f6b4bfff640 1 -- 172.21.15.16:0/3586579776 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b5009ba40 con 0x56524cf78680 2024-09-12T02:09:40.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.903+0000 7f6b4bfff640 1 -- 172.21.15.16:0/3586579776 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b5009aca0 con 0x56524cf78680 2024-09-12T02:09:40.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.904+0000 7f6b6a0ea740 1 -- 172.21.15.16:0/3586579776 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56524cf78680 msgr2=0x56524cf7aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:40.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.904+0000 7f6b6a0ea740 1 --2- 172.21.15.16:0/3586579776 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56524cf78680 0x56524cf7aa70 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f6b500977a0 tx=0x7f6b50000bc0 comp rx=0 tx=0).stop 2024-09-12T02:09:40.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.904+0000 7f6b6a0ea740 1 -- 172.21.15.16:0/3586579776 shutdown_connections 2024-09-12T02:09:40.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.904+0000 7f6b6a0ea740 1 --2- 172.21.15.16:0/3586579776 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56524cf78680 0x56524cf7aa70 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:40.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.904+0000 7f6b6a0ea740 1 --2- 172.21.15.16:0/3586579776 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56524cf6ddc0 0x56524cf78140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:40.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.904+0000 7f6b6a0ea740 1 --2- 172.21.15.16:0/3586579776 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56524cf6d4a0 0x56524cf6d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:40.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.904+0000 7f6b6a0ea740 1 -- 172.21.15.16:0/3586579776 >> 172.21.15.16:0/3586579776 conn(0x56524cf69a30 msgr2=0x56524cf69e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:40.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.904+0000 7f6b6a0ea740 1 -- 172.21.15.16:0/3586579776 shutdown_connections 2024-09-12T02:09:40.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.904+0000 7f6b6a0ea740 1 -- 172.21.15.16:0/3586579776 wait complete. 2024-09-12T02:09:40.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.904+0000 7f6b6a0ea740 1 -- start start 2024-09-12T02:09:40.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.904+0000 7f6b6a0ea740 1 Processor -- start 2024-09-12T02:09:40.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.905+0000 7f6b6a0ea740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56524cf6d4a0 0x56524cf77fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:40.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.905+0000 7f6b6a0ea740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56524cf6ddc0 0x56524cf75280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:40.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.905+0000 7f6b6a0ea740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56524cf78680 0x56524cf757e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:40.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.905+0000 7f6b6a0ea740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56524cf7b1f0 con 0x56524cf6ddc0 2024-09-12T02:09:40.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.905+0000 7f6b6a0ea740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56524ce8e050 con 0x56524cf6d4a0 2024-09-12T02:09:40.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.905+0000 7f6b6a0ea740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x56524cf7b370 con 0x56524cf78680 2024-09-12T02:09:40.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.905+0000 7f6b6a0ea740 10 client.0 _ll_register_callbacks cb 0x56524cece160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:09:40.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.905+0000 7f6b662ae640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56524cf78680 0x56524cf757e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:40.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.905+0000 7f6b652ac640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56524cf6ddc0 0x56524cf75280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:40.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.905+0000 7f6b662ae640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56524cf78680 0x56524cf757e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:57030/0 (socket says 172.21.15.16:57030) 2024-09-12T02:09:40.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.905+0000 7f6b652ac640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56524cf6ddc0 0x56524cf75280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39968/0 (socket says 172.21.15.16:39968) 2024-09-12T02:09:40.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.905+0000 7f6b662ae640 1 -- 172.21.15.16:0/2052681128 learned_addr learned my addr 172.21.15.16:0/2052681128 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:09:40.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.905+0000 7f6b65aad640 1 --2- 172.21.15.16:0/2052681128 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56524cf6d4a0 0x56524cf77fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:40.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.905+0000 7f6b652ac640 1 -- 172.21.15.16:0/2052681128 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56524cf78680 msgr2=0x56524cf757e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:09:40.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.905+0000 7f6b652ac640 1 --2- 172.21.15.16:0/2052681128 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56524cf78680 0x56524cf757e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:40.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.905+0000 7f6b652ac640 1 -- 172.21.15.16:0/2052681128 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56524cf6d4a0 msgr2=0x56524cf77fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:09:40.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.905+0000 7f6b652ac640 1 --2- 172.21.15.16:0/2052681128 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56524cf6d4a0 0x56524cf77fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:40.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.906+0000 7f6b652ac640 1 -- 172.21.15.16:0/2052681128 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56524cc443c0 con 0x56524cf6ddc0 2024-09-12T02:09:40.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.906+0000 7f6b65aad640 1 --2- 172.21.15.16:0/2052681128 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56524cf6d4a0 0x56524cf77fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:09:40.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.906+0000 7f6b652ac640 1 --2- 172.21.15.16:0/2052681128 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56524cf6ddc0 0x56524cf75280 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f6b500cc630 tx=0x7f6b5009f810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c895ef30b4d30c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:40.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.906+0000 7f6b497fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:09:40.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.906+0000 7f6b497fa640 1 -- 172.21.15.16:0/2052681128 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b500b3020 con 0x56524cf6ddc0 2024-09-12T02:09:40.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.906+0000 7f6b497fa640 1 -- 172.21.15.16:0/2052681128 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f6b500b4250 con 0x56524cf6ddc0 2024-09-12T02:09:40.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.906+0000 7f6b6a0ea740 1 -- 172.21.15.16:0/2052681128 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b5009ba40 con 0x56524cf6ddc0 2024-09-12T02:09:40.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.906+0000 7f6b497fa640 1 -- 172.21.15.16:0/2052681128 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6b5009aca0 con 0x56524cf6ddc0 2024-09-12T02:09:40.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.906+0000 7f6b6a0ea740 10 client.17841 fetch_fsmap learned FSMap version 25 2024-09-12T02:09:40.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.906+0000 7f6b6a0ea740 1 -- 172.21.15.16:0/2052681128 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56524cf7b1f0 con 0x56524cf6ddc0 2024-09-12T02:09:40.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.906+0000 7f6b497fa640 1 -- 172.21.15.16:0/2052681128 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b5009ba40 con 0x56524cf6ddc0 2024-09-12T02:09:40.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.906+0000 7f6b497fa640 1 -- 172.21.15.16:0/2052681128 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f6b500a3b20 con 0x56524cf6ddc0 2024-09-12T02:09:40.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.906+0000 7f6b6a0ea740 10 client.17841 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:09:40.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.906+0000 7f6b6a0ea740 10 client.17841 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:09:40.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.907+0000 7f6b6a0ea740 20 client.17841 populate_metadata read hostname 'smithi016' 2024-09-12T02:09:40.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.907+0000 7f6b6a0ea740 1 --2- 172.21.15.16:0/2052681128 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x56524cf87d70 0x56524cf8a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:40.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.907+0000 7f6b6a0ea740 4 client.17841 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:09:40.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.907+0000 7f6b6a0ea740 1 -- 172.21.15.16:0/2052681128 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56524cee0850 con 0x56524cf87d70 2024-09-12T02:09:40.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.907+0000 7f6b65aad640 1 --2- 172.21.15.16:0/2052681128 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x56524cf87d70 0x56524cf8a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:40.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.907+0000 7f6b65aad640 1 --2- 172.21.15.16:0/2052681128 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x56524cf87d70 0x56524cf8a160 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11f286c04b0fc935 server_cookie=ab0029e5837f496f in_seq=0 out_seq=0 2024-09-12T02:09:40.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.907+0000 7f6b497fa640 10 client.17841 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:09:40.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.908+0000 7f6b497fa640 1 -- 172.21.15.16:0/2052681128 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56524cee0850 con 0x56524cf87d70 2024-09-12T02:09:40.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.908+0000 7f6b497fa640 10 client.17841 handle_command_reply: tid=0 2024-09-12T02:09:40.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.908+0000 7f6b6a0ea740 1 client.17841 shutdown 2024-09-12T02:09:40.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.908+0000 7f6b6a0ea740 1 -- 172.21.15.16:0/2052681128 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56524cf6ddc0 msgr2=0x56524cf75280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:40.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.908+0000 7f6b6a0ea740 1 --2- 172.21.15.16:0/2052681128 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56524cf6ddc0 0x56524cf75280 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f6b500cc630 tx=0x7f6b5009f810 comp rx=0 tx=0).stop 2024-09-12T02:09:40.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.908+0000 7f6b6a0ea740 1 -- 172.21.15.16:0/2052681128 shutdown_connections 2024-09-12T02:09:40.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.908+0000 7f6b6a0ea740 1 --2- 172.21.15.16:0/2052681128 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x56524cf87d70 0x56524cf8a160 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:40.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.908+0000 7f6b6a0ea740 1 --2- 172.21.15.16:0/2052681128 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56524cf78680 0x56524cf757e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:40.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.908+0000 7f6b6a0ea740 1 --2- 172.21.15.16:0/2052681128 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56524cf6ddc0 0x56524cf75280 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:40.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.908+0000 7f6b6a0ea740 1 --2- 172.21.15.16:0/2052681128 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56524cf6d4a0 0x56524cf77fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:40.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.908+0000 7f6b6a0ea740 1 -- 172.21.15.16:0/2052681128 >> 172.21.15.16:0/2052681128 conn(0x56524cf69a30 msgr2=0x56524cf6f710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:40.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.908+0000 7f6b497fa640 0 client.17841 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:09:40.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.908+0000 7f6b6a0ea740 1 -- 172.21.15.16:0/2052681128 shutdown_connections 2024-09-12T02:09:40.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.908+0000 7f6b6a0ea740 1 -- 172.21.15.16:0/2052681128 wait complete. 2024-09-12T02:09:40.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.909+0000 7f6b6a0ea740 20 client.17841 trim_cache size 0 max 16384 2024-09-12T02:09:40.918 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:09:40 smithi033 ceph-mon[35332]: scrub queued for path: / 2024-09-12T02:09:40.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:09:40 smithi033 ceph-mon[35332]: scrub summary: idle+waiting paths [/] 2024-09-12T02:09:40.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:09:40 smithi033 ceph-mon[35332]: scrub summary: active paths [/] 2024-09-12T02:09:40.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:09:40 smithi033 ceph-mon[35332]: scrub summary: idle+waiting paths [/] 2024-09-12T02:09:40.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.925+0000 7f6b6a0ea740 1 Processor -- start 2024-09-12T02:09:40.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.926+0000 7f6b6a0ea740 1 -- start start 2024-09-12T02:09:40.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.926+0000 7f6b6a0ea740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56524cf6d4a0 0x56524cf84480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:40.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.926+0000 7f6b6a0ea740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56524cf6ddc0 0x56524cf89fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:40.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.926+0000 7f6b6a0ea740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56524cf78680 0x56524cf771f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:40.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.926+0000 7f6b6a0ea740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6b601b1f80 con 0x56524cf6d4a0 2024-09-12T02:09:40.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.926+0000 7f6b6a0ea740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6b601af850 con 0x56524cf78680 2024-09-12T02:09:40.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.926+0000 7f6b6a0ea740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f6b601ad6a0 con 0x56524cf6ddc0 2024-09-12T02:09:40.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.926+0000 7f6b65aad640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56524cf6d4a0 0x56524cf84480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:40.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.926+0000 7f6b65aad640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56524cf6d4a0 0x56524cf84480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39978/0 (socket says 172.21.15.16:39978) 2024-09-12T02:09:40.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.926+0000 7f6b65aad640 1 -- 172.21.15.16:0/3858742483 learned_addr learned my addr 172.21.15.16:0/3858742483 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:09:40.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.926+0000 7f6b652ac640 1 --2- 172.21.15.16:0/3858742483 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56524cf6ddc0 0x56524cf89fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:40.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.926+0000 7f6b65aad640 1 -- 172.21.15.16:0/3858742483 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56524cf6ddc0 msgr2=0x56524cf89fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:09:40.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.926+0000 7f6b65aad640 1 --2- 172.21.15.16:0/3858742483 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56524cf6ddc0 0x56524cf89fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:40.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.926+0000 7f6b65aad640 1 -- 172.21.15.16:0/3858742483 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56524cf78680 msgr2=0x56524cf771f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:09:40.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.926+0000 7f6b65aad640 1 --2- 172.21.15.16:0/3858742483 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56524cf78680 0x56524cf771f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:40.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.926+0000 7f6b65aad640 1 -- 172.21.15.16:0/3858742483 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56524cf6d2d0 con 0x56524cf6d4a0 2024-09-12T02:09:40.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.926+0000 7f6b652ac640 1 --2- 172.21.15.16:0/3858742483 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56524cf6ddc0 0x56524cf89fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:09:40.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.926+0000 7f6b65aad640 1 --2- 172.21.15.16:0/3858742483 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56524cf6d4a0 0x56524cf84480 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c02ac00 tx=0x7f6b5c02ab50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54ada44ad5453c23 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:40.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.926+0000 7f6b4b7fe640 1 -- 172.21.15.16:0/3858742483 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b5c047070 con 0x56524cf6d4a0 2024-09-12T02:09:40.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.927+0000 7f6b4b7fe640 1 -- 172.21.15.16:0/3858742483 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b5c005520 con 0x56524cf6d4a0 2024-09-12T02:09:40.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.927+0000 7f6b4b7fe640 1 -- 172.21.15.16:0/3858742483 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b5c035040 con 0x56524cf6d4a0 2024-09-12T02:09:40.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.927+0000 7f6b6a0ea740 1 -- 172.21.15.16:0/3858742483 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56524cf6d4a0 msgr2=0x56524cf84480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:40.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.927+0000 7f6b6a0ea740 1 --2- 172.21.15.16:0/3858742483 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56524cf6d4a0 0x56524cf84480 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c02ac00 tx=0x7f6b5c02ab50 comp rx=0 tx=0).stop 2024-09-12T02:09:40.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.927+0000 7f6b6a0ea740 1 -- 172.21.15.16:0/3858742483 shutdown_connections 2024-09-12T02:09:40.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.927+0000 7f6b6a0ea740 1 --2- 172.21.15.16:0/3858742483 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56524cf78680 0x56524cf771f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:40.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.927+0000 7f6b6a0ea740 1 --2- 172.21.15.16:0/3858742483 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56524cf6ddc0 0x56524cf89fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:40.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.927+0000 7f6b6a0ea740 1 --2- 172.21.15.16:0/3858742483 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56524cf6d4a0 0x56524cf84480 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:40.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.927+0000 7f6b6a0ea740 1 -- 172.21.15.16:0/3858742483 >> 172.21.15.16:0/3858742483 conn(0x56524cf69a30 msgr2=0x56524cf76ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:40.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.927+0000 7f6b6a0ea740 1 -- 172.21.15.16:0/3858742483 shutdown_connections 2024-09-12T02:09:40.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.927+0000 7f6b6a0ea740 1 -- 172.21.15.16:0/3858742483 wait complete. 2024-09-12T02:09:40.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.927+0000 7f6b6a0ea740 1 -- start start 2024-09-12T02:09:40.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.928+0000 7f6b6a0ea740 1 Processor -- start 2024-09-12T02:09:40.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.928+0000 7f6b6a0ea740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56524cf6d4a0 0x56524cf7c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:40.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.928+0000 7f6b6a0ea740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56524cf6ddc0 0x56524cf7c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:40.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.928+0000 7f6b6a0ea740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56524cf78680 0x56524cf7cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:40.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.928+0000 7f6b6a0ea740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6b601add10 con 0x56524cf6ddc0 2024-09-12T02:09:40.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.928+0000 7f6b6a0ea740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6b601af180 con 0x56524cf78680 2024-09-12T02:09:40.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.928+0000 7f6b6a0ea740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f6b601ae3e0 con 0x56524cf6d4a0 2024-09-12T02:09:40.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.928+0000 7f6b6a0ea740 10 client.0 _ll_register_callbacks cb 0x7f6b58009dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:09:40.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.928+0000 7f6b652ac640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56524cf6ddc0 0x56524cf7c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:40.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.928+0000 7f6b652ac640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56524cf6ddc0 0x56524cf7c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39992/0 (socket says 172.21.15.16:39992) 2024-09-12T02:09:40.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.928+0000 7f6b652ac640 1 -- 172.21.15.16:0/97049882 learned_addr learned my addr 172.21.15.16:0/97049882 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:09:40.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.928+0000 7f6b65aad640 1 --2- 172.21.15.16:0/97049882 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56524cf6d4a0 0x56524cf7c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:40.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.928+0000 7f6b662ae640 1 --2- 172.21.15.16:0/97049882 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56524cf78680 0x56524cf7cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:40.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.929+0000 7f6b652ac640 1 -- 172.21.15.16:0/97049882 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56524cf6d4a0 msgr2=0x56524cf7c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:09:40.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.929+0000 7f6b652ac640 1 --2- 172.21.15.16:0/97049882 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56524cf6d4a0 0x56524cf7c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:40.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.929+0000 7f6b652ac640 1 -- 172.21.15.16:0/97049882 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56524cf78680 msgr2=0x56524cf7cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:09:40.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.929+0000 7f6b652ac640 1 --2- 172.21.15.16:0/97049882 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56524cf78680 0x56524cf7cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:40.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.929+0000 7f6b652ac640 1 -- 172.21.15.16:0/97049882 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56524cf6a6c0 con 0x56524cf6ddc0 2024-09-12T02:09:40.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.929+0000 7f6b652ac640 1 --2- 172.21.15.16:0/97049882 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56524cf6ddc0 0x56524cf7c9e0 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f6b500977a0 tx=0x7f6b500057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a29db4db35250c4c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:40.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.929+0000 7f6b497fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:09:40.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.929+0000 7f6b497fa640 1 -- 172.21.15.16:0/97049882 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b500b3050 con 0x56524cf6ddc0 2024-09-12T02:09:40.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.929+0000 7f6b497fa640 1 -- 172.21.15.16:0/97049882 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f6b500972a0 con 0x56524cf6ddc0 2024-09-12T02:09:40.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.929+0000 7f6b6a0ea740 1 -- 172.21.15.16:0/97049882 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b5c005520 con 0x56524cf6ddc0 2024-09-12T02:09:40.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.929+0000 7f6b497fa640 1 -- 172.21.15.16:0/97049882 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6b5009ae80 con 0x56524cf6ddc0 2024-09-12T02:09:40.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.929+0000 7f6b6a0ea740 10 client.17847 fetch_fsmap learned FSMap version 25 2024-09-12T02:09:40.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.929+0000 7f6b6a0ea740 1 -- 172.21.15.16:0/97049882 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56524cff2a40 con 0x56524cf6ddc0 2024-09-12T02:09:40.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.929+0000 7f6b497fa640 1 -- 172.21.15.16:0/97049882 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b5c005520 con 0x56524cf6ddc0 2024-09-12T02:09:40.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.929+0000 7f6b497fa640 1 -- 172.21.15.16:0/97049882 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f6b500662b0 con 0x56524cf6ddc0 2024-09-12T02:09:40.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.929+0000 7f6b6a0ea740 10 client.17847 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:09:40.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.929+0000 7f6b6a0ea740 10 client.17847 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:09:40.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.929+0000 7f6b6a0ea740 20 client.17847 populate_metadata read hostname 'smithi016' 2024-09-12T02:09:40.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.930+0000 7f6b6a0ea740 1 --2- 172.21.15.16:0/97049882 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x56524cff2ff0 0x56524cff5410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:40.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.930+0000 7f6b6a0ea740 4 client.17847 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T02:09:40.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.930+0000 7f6b6a0ea740 1 -- 172.21.15.16:0/97049882 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56524cf8a690 con 0x56524cff2ff0 2024-09-12T02:09:40.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.930+0000 7f6b65aad640 1 --2- 172.21.15.16:0/97049882 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x56524cff2ff0 0x56524cff5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:40.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.930+0000 7f6b65aad640 1 --2- 172.21.15.16:0/97049882 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x56524cff2ff0 0x56524cff5410 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f6ce90cf0bdb8b4 server_cookie=dad73488e21abb91 in_seq=0 out_seq=0 2024-09-12T02:09:40.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.930+0000 7f6b497fa640 10 client.17847 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:09:40.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.930+0000 7f6b497fa640 1 -- 172.21.15.16:0/97049882 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x56524cf8a690 con 0x56524cff2ff0 2024-09-12T02:09:40.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.930+0000 7f6b497fa640 10 client.17847 handle_command_reply: tid=0 2024-09-12T02:09:40.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.930+0000 7f6b6a0ea740 1 client.17847 shutdown 2024-09-12T02:09:40.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.930+0000 7f6b6a0ea740 1 -- 172.21.15.16:0/97049882 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56524cf6ddc0 msgr2=0x56524cf7c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:40.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.930+0000 7f6b6a0ea740 1 --2- 172.21.15.16:0/97049882 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56524cf6ddc0 0x56524cf7c9e0 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f6b500977a0 tx=0x7f6b500057b0 comp rx=0 tx=0).stop 2024-09-12T02:09:40.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.931+0000 7f6b6a0ea740 1 -- 172.21.15.16:0/97049882 shutdown_connections 2024-09-12T02:09:40.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.931+0000 7f6b6a0ea740 1 --2- 172.21.15.16:0/97049882 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x56524cff2ff0 0x56524cff5410 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:40.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.931+0000 7f6b6a0ea740 1 --2- 172.21.15.16:0/97049882 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56524cf78680 0x56524cf7cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:40.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.931+0000 7f6b6a0ea740 1 --2- 172.21.15.16:0/97049882 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56524cf6ddc0 0x56524cf7c9e0 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:40.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.931+0000 7f6b6a0ea740 1 --2- 172.21.15.16:0/97049882 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56524cf6d4a0 0x56524cf7c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:40.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.931+0000 7f6b497fa640 0 client.17847 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:09:40.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.931+0000 7f6b6a0ea740 1 -- 172.21.15.16:0/97049882 >> 172.21.15.16:0/97049882 conn(0x56524cf69a30 msgr2=0x56524cf77300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:40.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.931+0000 7f6b6a0ea740 1 -- 172.21.15.16:0/97049882 shutdown_connections 2024-09-12T02:09:40.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.931+0000 7f6b6a0ea740 1 -- 172.21.15.16:0/97049882 wait complete. 2024-09-12T02:09:40.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.931+0000 7f6b6a0ea740 20 client.17847 trim_cache size 0 max 16384 2024-09-12T02:09:40.936 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T02:09:40.936 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-12T02:09:40.937 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": { 2024-09-12T02:09:40.937 INFO:teuthology.orchestra.run.smithi016.stdout: "c9b98abe-56b9-4ec8-a213-2e79d55cf1a3": { 2024-09-12T02:09:40.937 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-12T02:09:40.937 INFO:teuthology.orchestra.run.smithi016.stdout: "tag": "c9b98abe-56b9-4ec8-a213-2e79d55cf1a3", 2024-09-12T02:09:40.937 INFO:teuthology.orchestra.run.smithi016.stdout: "options": "recursive,force" 2024-09-12T02:09:40.937 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:09:40.937 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:09:40.937 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T02:09:40.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.931+0000 7f6b67d11640 1 -- 172.21.15.16:0/4221173552 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f6b4c0a2b80 msgr2=0x7f6b4c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:40.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.931+0000 7f6b67d11640 1 --2- 172.21.15.16:0/4221173552 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f6b4c0a2b80 0x7f6b4c0a5040 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f6b58005c40 tx=0x7f6b58009200 comp rx=0 tx=0).stop 2024-09-12T02:09:40.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.931+0000 7f6b67d11640 1 -- 172.21.15.16:0/4221173552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b60102790 msgr2=0x7f6b60074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:40.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.931+0000 7f6b67d11640 1 --2- 172.21.15.16:0/4221173552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b60102790 0x7f6b60074a30 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f6b5000d950 tx=0x7f6b5000de10 comp rx=0 tx=0).stop 2024-09-12T02:09:40.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.931+0000 7f6b67d11640 1 -- 172.21.15.16:0/4221173552 shutdown_connections 2024-09-12T02:09:40.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.931+0000 7f6b67d11640 1 --2- 172.21.15.16:0/4221173552 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f6b4c0a2b80 0x7f6b4c0a5040 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:40.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.931+0000 7f6b67d11640 1 --2- 172.21.15.16:0/4221173552 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b60106f10 0x7f6b60074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:40.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.931+0000 7f6b67d11640 1 --2- 172.21.15.16:0/4221173552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b60102790 0x7f6b60074a30 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:40.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.931+0000 7f6b67d11640 1 --2- 172.21.15.16:0/4221173552 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6b60101de0 0x7f6b600744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:40.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.931+0000 7f6b67d11640 1 -- 172.21.15.16:0/4221173552 >> 172.21.15.16:0/4221173552 conn(0x7f6b600fd8c0 msgr2=0x7f6b600fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:40.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.932+0000 7f6b67d11640 1 -- 172.21.15.16:0/4221173552 shutdown_connections 2024-09-12T02:09:40.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:40.932+0000 7f6b67d11640 1 -- 172.21.15.16:0/4221173552 wait complete. 2024-09-12T02:09:41.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:09:41 smithi016 ceph-mon[30947]: pgmap v568: 97 pgs: 97 active+clean; 4.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 6.5 KiB/s wr, 11 op/s 2024-09-12T02:09:41.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:09:41 smithi016 ceph-mon[30947]: scrub summary: idle 2024-09-12T02:09:41.785 INFO:tasks.cephfs.filesystem:scrub status for tag:c9b98abe-56b9-4ec8-a213-2e79d55cf1a3 - {'path': '/', 'tag': 'c9b98abe-56b9-4ec8-a213-2e79d55cf1a3', 'options': 'recursive,force'} 2024-09-12T02:09:41.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:09:41 smithi184 ceph-mon[35593]: pgmap v568: 97 pgs: 97 active+clean; 4.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 6.5 KiB/s wr, 11 op/s 2024-09-12T02:09:41.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:09:41 smithi184 ceph-mon[35593]: scrub summary: idle 2024-09-12T02:09:41.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:09:41 smithi033 ceph-mon[35332]: pgmap v568: 97 pgs: 97 active+clean; 4.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 6.5 KiB/s wr, 11 op/s 2024-09-12T02:09:41.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:09:41 smithi033 ceph-mon[35332]: scrub summary: idle 2024-09-12T02:09:43.365 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:09:43.636 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:09:43.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:09:43 smithi016 ceph-mon[30947]: pgmap v569: 97 pgs: 97 active+clean; 4.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 2.8 KiB/s wr, 9 op/s 2024-09-12T02:09:43.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:09:43 smithi184 ceph-mon[35593]: pgmap v569: 97 pgs: 97 active+clean; 4.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 2.8 KiB/s wr, 9 op/s 2024-09-12T02:09:43.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:09:43 smithi033 ceph-mon[35332]: pgmap v569: 97 pgs: 97 active+clean; 4.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 2.8 KiB/s wr, 9 op/s 2024-09-12T02:09:44.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:44.103+0000 7f1965151640 1 -- 172.21.15.16:0/142622558 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19601036e0 msgr2=0x7f1960107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:44.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:44.103+0000 7f1965151640 1 --2- 172.21.15.16:0/142622558 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19601036e0 0x7f1960107b90 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f1948009930 tx=0x7f194802a420 comp rx=0 tx=0).stop 2024-09-12T02:09:44.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:44.104+0000 7f1965151640 1 -- 172.21.15.16:0/142622558 shutdown_connections 2024-09-12T02:09:44.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:44.104+0000 7f1965151640 1 --2- 172.21.15.16:0/142622558 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f19601082a0 0x7f196010ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:44.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:44.104+0000 7f1965151640 1 --2- 172.21.15.16:0/142622558 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19601036e0 0x7f1960107b90 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:44.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:44.104+0000 7f1965151640 1 --2- 172.21.15.16:0/142622558 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1960102d30 0x7f1960103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:44.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:44.104+0000 7f1965151640 1 -- 172.21.15.16:0/142622558 >> 172.21.15.16:0/142622558 conn(0x7f19600fc7d0 msgr2=0x7f19600febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:44.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:44.104+0000 7f1965151640 1 -- 172.21.15.16:0/142622558 shutdown_connections 2024-09-12T02:09:44.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:44.104+0000 7f1965151640 1 -- 172.21.15.16:0/142622558 wait complete. 2024-09-12T02:09:44.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:44.104+0000 7f1965151640 1 Processor -- start 2024-09-12T02:09:44.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:44.104+0000 7f1965151640 1 -- start start 2024-09-12T02:09:44.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:44.105+0000 7f1965151640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1960102d30 0x7f1960198600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:44.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:44.105+0000 7f1965151640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19601036e0 0x7f1960198b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:44.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:44.105+0000 7f1965151640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f19601082a0 0x7f196019ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:44.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:44.105+0000 7f1965151640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f19601120c0 con 0x7f19601036e0 2024-09-12T02:09:44.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:44.105+0000 7f1965151640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1960111f40 con 0x7f19601082a0 2024-09-12T02:09:44.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:44.105+0000 7f1965151640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1960112240 con 0x7f1960102d30 2024-09-12T02:09:44.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:44.105+0000 7f195e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19601036e0 0x7f1960198b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:44.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:44.105+0000 7f195e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19601036e0 0x7f1960198b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38116/0 (socket says 172.21.15.16:38116) 2024-09-12T02:09:44.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:44.105+0000 7f195e59c640 1 -- 172.21.15.16:0/3056753437 learned_addr learned my addr 172.21.15.16:0/3056753437 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:09:44.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:44.105+0000 7f195f59e640 1 --2- 172.21.15.16:0/3056753437 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f19601082a0 0x7f196019ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:44.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:44.105+0000 7f195ed9d640 1 --2- 172.21.15.16:0/3056753437 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1960102d30 0x7f1960198600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:44.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:44.105+0000 7f195e59c640 1 -- 172.21.15.16:0/3056753437 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1960102d30 msgr2=0x7f1960198600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:44.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:44.105+0000 7f195e59c640 1 --2- 172.21.15.16:0/3056753437 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1960102d30 0x7f1960198600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:44.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:44.105+0000 7f195e59c640 1 -- 172.21.15.16:0/3056753437 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f19601082a0 msgr2=0x7f196019ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:44.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:44.105+0000 7f195e59c640 1 --2- 172.21.15.16:0/3056753437 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f19601082a0 0x7f196019ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:44.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:44.105+0000 7f195e59c640 1 -- 172.21.15.16:0/3056753437 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f196019d5b0 con 0x7f19601036e0 2024-09-12T02:09:44.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:44.105+0000 7f195f59e640 1 --2- 172.21.15.16:0/3056753437 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f19601082a0 0x7f196019ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:09:44.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:44.105+0000 7f195e59c640 1 --2- 172.21.15.16:0/3056753437 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19601036e0 0x7f1960198b40 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f1948005940 tx=0x7f19480042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:44.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:44.106+0000 7f1957fff640 1 -- 172.21.15.16:0/3056753437 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f194803f030 con 0x7f19601036e0 2024-09-12T02:09:44.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:44.106+0000 7f1965151640 1 -- 172.21.15.16:0/3056753437 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f196019d840 con 0x7f19601036e0 2024-09-12T02:09:44.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:44.106+0000 7f1957fff640 1 -- 172.21.15.16:0/3056753437 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1948003ce0 con 0x7f19601036e0 2024-09-12T02:09:44.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:44.106+0000 7f1957fff640 1 -- 172.21.15.16:0/3056753437 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1948035070 con 0x7f19601036e0 2024-09-12T02:09:44.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:44.106+0000 7f1965151640 1 -- 172.21.15.16:0/3056753437 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19601a5180 con 0x7f19601036e0 2024-09-12T02:09:44.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:44.107+0000 7f1965151640 1 -- 172.21.15.16:0/3056753437 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1960103110 con 0x7f19601036e0 2024-09-12T02:09:44.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:44.108+0000 7f1957fff640 1 -- 172.21.15.16:0/3056753437 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f1948042030 con 0x7f19601036e0 2024-09-12T02:09:44.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:44.109+0000 7f1957fff640 1 --2- 172.21.15.16:0/3056753437 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f194c0a2b80 0x7f194c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:44.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:44.109+0000 7f195ed9d640 1 --2- 172.21.15.16:0/3056753437 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f194c0a2b80 0x7f194c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:44.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:44.109+0000 7f1957fff640 1 -- 172.21.15.16:0/3056753437 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f19480ef4d0 con 0x7f19601036e0 2024-09-12T02:09:44.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:44.109+0000 7f195ed9d640 1 --2- 172.21.15.16:0/3056753437 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f194c0a2b80 0x7f194c0a5040 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f1950005fd0 tx=0x7f1950005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:44.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:44.113+0000 7f1957fff640 1 -- 172.21.15.16:0/3056753437 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f19480bc4d0 con 0x7f19601036e0 2024-09-12T02:09:44.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:44.324+0000 7f1965151640 1 -- 172.21.15.16:0/3056753437 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f196010a2b0 con 0x7f19601036e0 2024-09-12T02:09:44.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:44.325+0000 7f1957fff640 1 -- 172.21.15.16:0/3056753437 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11911 (secure 0 0 0) 0x7f19480bbdd0 con 0x7f19601036e0 2024-09-12T02:09:44.325 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T02:09:44.325 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":25,"btime":"2024-09-12T02:01:37:192021+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3279123279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3279123279},{"type":"v1","addr":"172.21.15.16:6835","nonce":3279123279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4119452382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4119452382},{"type":"v1","addr":"172.21.15.16:6837","nonce":4119452382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/3263823165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3263823165},{"type":"v1","addr":"172.21.15.16:6839","nonce":3263823165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/1510290732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":1510290732},{"type":"v1","addr":"172.21.15.16:6841","nonce":1510290732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/279674603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":279674603},{"type":"v1","addr":"172.21.15.33:6833","nonce":279674603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3555728028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3555728028},{"type":"v1","addr":"172.21.15.33:6835","nonce":3555728028}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T01:59:56.117608+0000","modified":"2024-09-12T02:01:36.294760+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24482,"mds_1":24481,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6833/1563374106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1563374106},{"type":"v1","addr":"172.21.15.184:6833","nonce":1563374106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6837/1892833868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1892833868},{"type":"v1","addr":"172.21.15.33:6837","nonce":1892833868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/1457679686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1457679686},{"type":"v1","addr":"172.21.15.184:6835","nonce":1457679686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.184:6837/1793393087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":1793393087},{"type":"v1","addr":"172.21.15.184:6837","nonce":1793393087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.184:6839/4203347941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":4203347941},{"type":"v1","addr":"172.21.15.184:6839","nonce":4203347941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.33:6839/4136670204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":4136670204},{"type":"v1","addr":"172.21.15.33:6839","nonce":4136670204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24481,24482,24469]},"id":1}]} 2024-09-12T02:09:44.325 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 25 2024-09-12T02:09:44.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:44.327+0000 7f1965151640 1 -- 172.21.15.16:0/3056753437 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f194c0a2b80 msgr2=0x7f194c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:44.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:44.327+0000 7f1965151640 1 --2- 172.21.15.16:0/3056753437 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f194c0a2b80 0x7f194c0a5040 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f1950005fd0 tx=0x7f1950005850 comp rx=0 tx=0).stop 2024-09-12T02:09:44.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:44.327+0000 7f1965151640 1 -- 172.21.15.16:0/3056753437 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19601036e0 msgr2=0x7f1960198b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:44.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:44.327+0000 7f1965151640 1 --2- 172.21.15.16:0/3056753437 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19601036e0 0x7f1960198b40 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f1948005940 tx=0x7f19480042f0 comp rx=0 tx=0).stop 2024-09-12T02:09:44.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:44.327+0000 7f1965151640 1 -- 172.21.15.16:0/3056753437 shutdown_connections 2024-09-12T02:09:44.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:44.327+0000 7f1965151640 1 --2- 172.21.15.16:0/3056753437 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f194c0a2b80 0x7f194c0a5040 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:44.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:44.327+0000 7f1965151640 1 --2- 172.21.15.16:0/3056753437 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f19601082a0 0x7f196019ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:44.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:44.327+0000 7f1965151640 1 --2- 172.21.15.16:0/3056753437 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19601036e0 0x7f1960198b40 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:44.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:44.327+0000 7f1965151640 1 --2- 172.21.15.16:0/3056753437 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1960102d30 0x7f1960198600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:44.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:44.327+0000 7f1965151640 1 -- 172.21.15.16:0/3056753437 >> 172.21.15.16:0/3056753437 conn(0x7f19600fc7d0 msgr2=0x7f1960105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:44.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:44.327+0000 7f1965151640 1 -- 172.21.15.16:0/3056753437 shutdown_connections 2024-09-12T02:09:44.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:44.327+0000 7f1965151640 1 -- 172.21.15.16:0/3056753437 wait complete. 2024-09-12T02:09:44.651 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:09:44 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/3056753437' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:09:44.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:09:44 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/3056753437' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:09:44.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:09:44 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/3056753437' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:09:44.965 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24482 quiesce db --set-id d676e596 --release --await' 2024-09-12T02:09:44.965 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24482 quiesce db --set-id d676e596 --release --await 2024-09-12T02:09:45.223 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:09:45.647 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:09:45 smithi016 ceph-mon[30947]: pgmap v570: 97 pgs: 97 active+clean; 4.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 2.9 KiB/s wr, 10 op/s 2024-09-12T02:09:45.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.736+0000 7f16d8481640 1 -- 172.21.15.16:0/3099106159 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16d0102790 msgr2=0x7f16d01067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:45.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.736+0000 7f16d8481640 1 --2- 172.21.15.16:0/3099106159 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16d0102790 0x7f16d01067e0 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f16c0009900 tx=0x7f16c002a410 comp rx=0 tx=0).stop 2024-09-12T02:09:45.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.736+0000 7f16d8481640 1 -- 172.21.15.16:0/3099106159 shutdown_connections 2024-09-12T02:09:45.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.736+0000 7f16d8481640 1 --2- 172.21.15.16:0/3099106159 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f16d0106f10 0x7f16d010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:45.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.736+0000 7f16d8481640 1 --2- 172.21.15.16:0/3099106159 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16d0102790 0x7f16d01067e0 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:45.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.736+0000 7f16d8481640 1 --2- 172.21.15.16:0/3099106159 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f16d0101de0 0x7f16d01021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:45.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.736+0000 7f16d8481640 1 -- 172.21.15.16:0/3099106159 >> 172.21.15.16:0/3099106159 conn(0x7f16d00fd8c0 msgr2=0x7f16d00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:45.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.736+0000 7f16d8481640 1 -- 172.21.15.16:0/3099106159 shutdown_connections 2024-09-12T02:09:45.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.736+0000 7f16d8481640 1 -- 172.21.15.16:0/3099106159 wait complete. 2024-09-12T02:09:45.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.737+0000 7f16d8481640 1 Processor -- start 2024-09-12T02:09:45.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.737+0000 7f16d8481640 1 -- start start 2024-09-12T02:09:45.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.737+0000 7f16d8481640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16d0101de0 0x7f16d00744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:45.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.737+0000 7f16d8481640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f16d0102790 0x7f16d0074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:45.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.737+0000 7f16d8481640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f16d0106f10 0x7f16d0074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:45.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.737+0000 7f16d8481640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f16d0111fc0 con 0x7f16d0101de0 2024-09-12T02:09:45.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.737+0000 7f16d8481640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f16d0111e40 con 0x7f16d0106f10 2024-09-12T02:09:45.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.737+0000 7f16d8481640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f16d0112140 con 0x7f16d0102790 2024-09-12T02:09:45.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.737+0000 7f16d621d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16d0101de0 0x7f16d00744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:45.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.737+0000 7f16d621d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16d0101de0 0x7f16d00744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38128/0 (socket says 172.21.15.16:38128) 2024-09-12T02:09:45.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.737+0000 7f16d621d640 1 -- 172.21.15.16:0/1070670117 learned_addr learned my addr 172.21.15.16:0/1070670117 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:09:45.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.737+0000 7f16d6a1e640 1 --2- 172.21.15.16:0/1070670117 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f16d0106f10 0x7f16d0074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:45.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.737+0000 7f16d5a1c640 1 --2- 172.21.15.16:0/1070670117 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f16d0102790 0x7f16d0074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:45.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.737+0000 7f16d621d640 1 -- 172.21.15.16:0/1070670117 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f16d0102790 msgr2=0x7f16d0074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:45.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.737+0000 7f16d621d640 1 --2- 172.21.15.16:0/1070670117 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f16d0102790 0x7f16d0074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:45.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.737+0000 7f16d621d640 1 -- 172.21.15.16:0/1070670117 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f16d0106f10 msgr2=0x7f16d0074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:45.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.737+0000 7f16d621d640 1 --2- 172.21.15.16:0/1070670117 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f16d0106f10 0x7f16d0074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:45.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.738+0000 7f16d621d640 1 -- 172.21.15.16:0/1070670117 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16d01a68c0 con 0x7f16d0101de0 2024-09-12T02:09:45.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.738+0000 7f16d621d640 1 --2- 172.21.15.16:0/1070670117 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16d0101de0 0x7f16d00744f0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f16cc00d950 tx=0x7f16cc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:45.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.738+0000 7f16c77fe640 1 -- 172.21.15.16:0/1070670117 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16cc017030 con 0x7f16d0101de0 2024-09-12T02:09:45.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.738+0000 7f16d8481640 1 -- 172.21.15.16:0/1070670117 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16d01a6bb0 con 0x7f16d0101de0 2024-09-12T02:09:45.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.738+0000 7f16c77fe640 1 -- 172.21.15.16:0/1070670117 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16cc004020 con 0x7f16d0101de0 2024-09-12T02:09:45.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.738+0000 7f16c77fe640 1 -- 172.21.15.16:0/1070670117 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16cc0047a0 con 0x7f16d0101de0 2024-09-12T02:09:45.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.738+0000 7f16d8481640 1 -- 172.21.15.16:0/1070670117 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16d01a70c0 con 0x7f16d0101de0 2024-09-12T02:09:45.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.740+0000 7f16c77fe640 1 -- 172.21.15.16:0/1070670117 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f16cc0041d0 con 0x7f16d0101de0 2024-09-12T02:09:45.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.740+0000 7f16da85a740 1 Processor -- start 2024-09-12T02:09:45.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.741+0000 7f16da85a740 1 -- start start 2024-09-12T02:09:45.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.741+0000 7f16da85a740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5582157cc060 0x5582157cc440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:45.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.741+0000 7f16da85a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5582157cc980 0x5582157d6d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:45.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.741+0000 7f16da85a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582157d7240 0x5582157d9630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:45.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.741+0000 7f16da85a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x558215731620 con 0x5582157d7240 2024-09-12T02:09:45.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.741+0000 7f16da85a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5582156d8e50 con 0x5582157cc980 2024-09-12T02:09:45.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.741+0000 7f16da85a740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55821572d5e0 con 0x5582157cc060 2024-09-12T02:09:45.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.741+0000 7f16d5a1c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582157d7240 0x5582157d9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:45.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.741+0000 7f16d5a1c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582157d7240 0x5582157d9630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38134/0 (socket says 172.21.15.16:38134) 2024-09-12T02:09:45.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.741+0000 7f16d5a1c640 1 -- 172.21.15.16:0/2818312347 learned_addr learned my addr 172.21.15.16:0/2818312347 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:09:45.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.741+0000 7f16c77fe640 1 --2- 172.21.15.16:0/1070670117 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f16bc0a2b80 0x7f16bc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:45.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.741+0000 7f16d6a1e640 1 --2- 172.21.15.16:0/2818312347 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5582157cc060 0x5582157cc440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:45.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.741+0000 7f16d621d640 1 --2- 172.21.15.16:0/2818312347 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5582157cc980 0x5582157d6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:45.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.741+0000 7f16d6a1e640 1 --2- 172.21.15.16:0/1070670117 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f16bc0a2b80 0x7f16bc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:45.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.741+0000 7f16d5a1c640 1 -- 172.21.15.16:0/2818312347 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5582157cc060 msgr2=0x5582157cc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:09:45.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.741+0000 7f16d5a1c640 1 --2- 172.21.15.16:0/2818312347 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5582157cc060 0x5582157cc440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:45.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.741+0000 7f16d5a1c640 1 -- 172.21.15.16:0/2818312347 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5582157cc980 msgr2=0x5582157d6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:09:45.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.741+0000 7f16d5a1c640 1 --2- 172.21.15.16:0/2818312347 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5582157cc980 0x5582157d6d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:45.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.742+0000 7f16d5a1c640 1 -- 172.21.15.16:0/2818312347 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5582156c9d40 con 0x5582157d7240 2024-09-12T02:09:45.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.742+0000 7f16c77fe640 1 -- 172.21.15.16:0/1070670117 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f16cc0ce8c0 con 0x7f16d0101de0 2024-09-12T02:09:45.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.742+0000 7f16d5a1c640 1 --2- 172.21.15.16:0/2818312347 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582157d7240 0x5582157d9630 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f16c0009900 tx=0x7f16c00040f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=208a4874ab0d33da server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:45.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.742+0000 7f16d6a1e640 1 --2- 172.21.15.16:0/1070670117 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f16bc0a2b80 0x7f16bc0a5040 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f16c8005fd0 tx=0x7f16c8009600 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:45.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.742+0000 7f16c4ff9640 1 -- 172.21.15.16:0/2818312347 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16c0035040 con 0x5582157d7240 2024-09-12T02:09:45.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.742+0000 7f16c4ff9640 1 -- 172.21.15.16:0/2818312347 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16c0005520 con 0x5582157d7240 2024-09-12T02:09:45.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.742+0000 7f16c4ff9640 1 -- 172.21.15.16:0/2818312347 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16c0004e60 con 0x5582157d7240 2024-09-12T02:09:45.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.743+0000 7f16da85a740 1 -- 172.21.15.16:0/2818312347 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582157d7240 msgr2=0x5582157d9630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:45.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.743+0000 7f16da85a740 1 --2- 172.21.15.16:0/2818312347 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582157d7240 0x5582157d9630 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f16c0009900 tx=0x7f16c00040f0 comp rx=0 tx=0).stop 2024-09-12T02:09:45.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.743+0000 7f16da85a740 1 -- 172.21.15.16:0/2818312347 shutdown_connections 2024-09-12T02:09:45.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.743+0000 7f16da85a740 1 --2- 172.21.15.16:0/2818312347 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582157d7240 0x5582157d9630 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:45.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.743+0000 7f16da85a740 1 --2- 172.21.15.16:0/2818312347 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5582157cc980 0x5582157d6d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:45.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.743+0000 7f16da85a740 1 --2- 172.21.15.16:0/2818312347 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5582157cc060 0x5582157cc440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:45.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.743+0000 7f16da85a740 1 -- 172.21.15.16:0/2818312347 >> 172.21.15.16:0/2818312347 conn(0x5582157c85f0 msgr2=0x5582157c89d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:45.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.743+0000 7f16da85a740 1 -- 172.21.15.16:0/2818312347 shutdown_connections 2024-09-12T02:09:45.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.743+0000 7f16da85a740 1 -- 172.21.15.16:0/2818312347 wait complete. 2024-09-12T02:09:45.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.743+0000 7f16da85a740 1 -- start start 2024-09-12T02:09:45.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.743+0000 7f16da85a740 1 Processor -- start 2024-09-12T02:09:45.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.744+0000 7f16da85a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5582157cc060 0x5582157d40d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:45.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.744+0000 7f16da85a740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5582157cc980 0x5582157d4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:45.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.744+0000 7f16da85a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582157d7240 0x5582157d4b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:45.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.744+0000 7f16da85a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5582157d9e10 con 0x5582157d7240 2024-09-12T02:09:45.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.744+0000 7f16da85a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5582156b9f10 con 0x5582157cc060 2024-09-12T02:09:45.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.744+0000 7f16da85a740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5582157d9f90 con 0x5582157cc980 2024-09-12T02:09:45.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.744+0000 7f16da85a740 10 client.0 _ll_register_callbacks cb 0x5582156dddb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:09:45.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.744+0000 7f16d6a1e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582157d7240 0x5582157d4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:45.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.744+0000 7f16d6a1e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582157d7240 0x5582157d4b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38148/0 (socket says 172.21.15.16:38148) 2024-09-12T02:09:45.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.744+0000 7f16d6a1e640 1 -- 172.21.15.16:0/3055226666 learned_addr learned my addr 172.21.15.16:0/3055226666 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:09:45.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.744+0000 7f16d621d640 1 --2- 172.21.15.16:0/3055226666 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5582157cc060 0x5582157d40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:45.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.744+0000 7f16d5a1c640 1 --2- 172.21.15.16:0/3055226666 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5582157cc980 0x5582157d4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:45.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.744+0000 7f16d6a1e640 1 -- 172.21.15.16:0/3055226666 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5582157cc980 msgr2=0x5582157d4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:09:45.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.744+0000 7f16d6a1e640 1 --2- 172.21.15.16:0/3055226666 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5582157cc980 0x5582157d4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:45.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.744+0000 7f16d6a1e640 1 -- 172.21.15.16:0/3055226666 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5582157cc060 msgr2=0x5582157d40d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:09:45.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.744+0000 7f16d6a1e640 1 --2- 172.21.15.16:0/3055226666 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5582157cc060 0x5582157d40d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:45.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.744+0000 7f16d6a1e640 1 -- 172.21.15.16:0/3055226666 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558215440580 con 0x5582157d7240 2024-09-12T02:09:45.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.744+0000 7f16d6a1e640 1 --2- 172.21.15.16:0/3055226666 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582157d7240 0x5582157d4b50 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f16c8010d10 tx=0x7f16c8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ff0599b195696fe server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:45.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.744+0000 7f16b9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:09:45.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.744+0000 7f16b9ffb640 1 -- 172.21.15.16:0/3055226666 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16c8004d20 con 0x5582157d7240 2024-09-12T02:09:45.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.744+0000 7f16b9ffb640 1 -- 172.21.15.16:0/3055226666 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f16c8005020 con 0x5582157d7240 2024-09-12T02:09:45.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.744+0000 7f16da85a740 1 -- 172.21.15.16:0/3055226666 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16c0005520 con 0x5582157d7240 2024-09-12T02:09:45.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.745+0000 7f16b9ffb640 1 -- 172.21.15.16:0/3055226666 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f16c8005220 con 0x5582157d7240 2024-09-12T02:09:45.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.745+0000 7f16da85a740 10 client.17865 fetch_fsmap learned FSMap version 25 2024-09-12T02:09:45.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.745+0000 7f16da85a740 1 -- 172.21.15.16:0/3055226666 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55821550a960 con 0x5582157d7240 2024-09-12T02:09:45.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.745+0000 7f16b9ffb640 1 -- 172.21.15.16:0/3055226666 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16c0005520 con 0x5582157d7240 2024-09-12T02:09:45.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.745+0000 7f16b9ffb640 1 -- 172.21.15.16:0/3055226666 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f16c800f040 con 0x5582157d7240 2024-09-12T02:09:45.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.745+0000 7f16da85a740 10 client.17865 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:09:45.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.745+0000 7f16da85a740 10 client.17865 resolve_mds: validated gid 24482 aka daemon mds.b 2024-09-12T02:09:45.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.745+0000 7f16da85a740 20 client.17865 populate_metadata read hostname 'smithi016' 2024-09-12T02:09:45.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.745+0000 7f16da85a740 1 --2- 172.21.15.16:0/3055226666 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5582157e6dd0 0x5582157e9240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:45.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.745+0000 7f16da85a740 4 client.17865 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:09:45.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.745+0000 7f16da85a740 1 -- 172.21.15.16:0/3055226666 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55821573eb90 con 0x5582157e6dd0 2024-09-12T02:09:45.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.745+0000 7f16d621d640 1 --2- 172.21.15.16:0/3055226666 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5582157e6dd0 0x5582157e9240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:45.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.746+0000 7f16d621d640 1 --2- 172.21.15.16:0/3055226666 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5582157e6dd0 0x5582157e9240 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=edc107ea80dd0f4c server_cookie=ce9862f4b2b31c94 in_seq=0 out_seq=0 2024-09-12T02:09:45.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.746+0000 7f16b9ffb640 10 client.17865 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:09:45.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.746+0000 7f16b9ffb640 1 -- 172.21.15.16:0/3055226666 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55821573eb90 con 0x5582157e6dd0 2024-09-12T02:09:45.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.746+0000 7f16b9ffb640 10 client.17865 handle_command_reply: tid=0 2024-09-12T02:09:45.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.746+0000 7f16da85a740 1 client.17865 shutdown 2024-09-12T02:09:45.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.747+0000 7f16da85a740 1 -- 172.21.15.16:0/3055226666 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582157d7240 msgr2=0x5582157d4b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:45.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.747+0000 7f16da85a740 1 --2- 172.21.15.16:0/3055226666 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582157d7240 0x5582157d4b50 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f16c8010d10 tx=0x7f16c8004a40 comp rx=0 tx=0).stop 2024-09-12T02:09:45.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.747+0000 7f16da85a740 1 -- 172.21.15.16:0/3055226666 shutdown_connections 2024-09-12T02:09:45.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.747+0000 7f16da85a740 1 --2- 172.21.15.16:0/3055226666 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5582157e6dd0 0x5582157e9240 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:45.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.747+0000 7f16b9ffb640 0 client.17865 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:09:45.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.747+0000 7f16da85a740 1 --2- 172.21.15.16:0/3055226666 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582157d7240 0x5582157d4b50 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:45.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.747+0000 7f16da85a740 1 --2- 172.21.15.16:0/3055226666 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5582157cc980 0x5582157d4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:45.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.747+0000 7f16da85a740 1 --2- 172.21.15.16:0/3055226666 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5582157cc060 0x5582157d40d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:45.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.747+0000 7f16da85a740 1 -- 172.21.15.16:0/3055226666 >> 172.21.15.16:0/3055226666 conn(0x5582157c85f0 msgr2=0x5582157ce2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:45.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.747+0000 7f16da85a740 1 -- 172.21.15.16:0/3055226666 shutdown_connections 2024-09-12T02:09:45.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.747+0000 7f16da85a740 1 -- 172.21.15.16:0/3055226666 wait complete. 2024-09-12T02:09:45.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.747+0000 7f16da85a740 20 client.17865 trim_cache size 0 max 16384 2024-09-12T02:09:45.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.764+0000 7f16da85a740 1 Processor -- start 2024-09-12T02:09:45.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.764+0000 7f16da85a740 1 -- start start 2024-09-12T02:09:45.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.764+0000 7f16da85a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582157cc060 0x5582157ce1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:45.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.764+0000 7f16da85a740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5582157cc980 0x5582157eeb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:45.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.764+0000 7f16da85a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5582157d7240 0x5582157ef050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:45.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.764+0000 7f16da85a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f16d01b1ec0 con 0x5582157cc060 2024-09-12T02:09:45.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.764+0000 7f16da85a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f16d01af790 con 0x5582157d7240 2024-09-12T02:09:45.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.764+0000 7f16da85a740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f16d01ad5e0 con 0x5582157cc980 2024-09-12T02:09:45.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.764+0000 7f16d621d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582157cc060 0x5582157ce1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:45.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.764+0000 7f16d621d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582157cc060 0x5582157ce1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38158/0 (socket says 172.21.15.16:38158) 2024-09-12T02:09:45.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.764+0000 7f16d621d640 1 -- 172.21.15.16:0/541384960 learned_addr learned my addr 172.21.15.16:0/541384960 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:09:45.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.764+0000 7f16d6a1e640 1 --2- 172.21.15.16:0/541384960 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5582157d7240 0x5582157ef050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:45.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.764+0000 7f16d621d640 1 -- 172.21.15.16:0/541384960 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5582157cc980 msgr2=0x5582157eeb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:09:45.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.764+0000 7f16d5a1c640 1 --2- 172.21.15.16:0/541384960 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5582157cc980 0x5582157eeb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:45.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.764+0000 7f16d621d640 1 --2- 172.21.15.16:0/541384960 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5582157cc980 0x5582157eeb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:45.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.765+0000 7f16d621d640 1 -- 172.21.15.16:0/541384960 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5582157d7240 msgr2=0x5582157ef050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:09:45.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.765+0000 7f16d621d640 1 --2- 172.21.15.16:0/541384960 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5582157d7240 0x5582157ef050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:45.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.765+0000 7f16d621d640 1 -- 172.21.15.16:0/541384960 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5582157ceed0 con 0x5582157cc060 2024-09-12T02:09:45.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.765+0000 7f16d6a1e640 1 --2- 172.21.15.16:0/541384960 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5582157d7240 0x5582157ef050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:09:45.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.765+0000 7f16d621d640 1 --2- 172.21.15.16:0/541384960 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582157cc060 0x5582157ce1c0 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f16cc005250 tx=0x7f16cc06e280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e46f5dc1055553 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:45.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.765+0000 7f16bbfff640 1 -- 172.21.15.16:0/541384960 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16cc09b8c0 con 0x5582157cc060 2024-09-12T02:09:45.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.765+0000 7f16bbfff640 1 -- 172.21.15.16:0/541384960 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16cc06d020 con 0x5582157cc060 2024-09-12T02:09:45.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.765+0000 7f16bbfff640 1 -- 172.21.15.16:0/541384960 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16cc0992f0 con 0x5582157cc060 2024-09-12T02:09:45.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.765+0000 7f16da85a740 1 -- 172.21.15.16:0/541384960 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582157cc060 msgr2=0x5582157ce1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:45.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.765+0000 7f16da85a740 1 --2- 172.21.15.16:0/541384960 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582157cc060 0x5582157ce1c0 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f16cc005250 tx=0x7f16cc06e280 comp rx=0 tx=0).stop 2024-09-12T02:09:45.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.766+0000 7f16da85a740 1 -- 172.21.15.16:0/541384960 shutdown_connections 2024-09-12T02:09:45.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.766+0000 7f16da85a740 1 --2- 172.21.15.16:0/541384960 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5582157d7240 0x5582157ef050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:45.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.766+0000 7f16da85a740 1 --2- 172.21.15.16:0/541384960 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5582157cc980 0x5582157eeb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:45.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.766+0000 7f16da85a740 1 --2- 172.21.15.16:0/541384960 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582157cc060 0x5582157ce1c0 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:45.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.766+0000 7f16da85a740 1 -- 172.21.15.16:0/541384960 >> 172.21.15.16:0/541384960 conn(0x5582157c85f0 msgr2=0x5582157cdab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:45.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.766+0000 7f16da85a740 1 -- 172.21.15.16:0/541384960 shutdown_connections 2024-09-12T02:09:45.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.766+0000 7f16da85a740 1 -- 172.21.15.16:0/541384960 wait complete. 2024-09-12T02:09:45.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.766+0000 7f16da85a740 1 -- start start 2024-09-12T02:09:45.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.766+0000 7f16da85a740 1 Processor -- start 2024-09-12T02:09:45.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.766+0000 7f16da85a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582157cc060 0x5582157df580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:45.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.766+0000 7f16da85a740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5582157cc980 0x5582157dfac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:45.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.766+0000 7f16da85a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5582157d7240 0x5582157dc430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:45.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.766+0000 7f16da85a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f16d01adc50 con 0x5582157cc060 2024-09-12T02:09:45.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.766+0000 7f16da85a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f16d01af0c0 con 0x5582157d7240 2024-09-12T02:09:45.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.766+0000 7f16da85a740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f16d01ae320 con 0x5582157cc980 2024-09-12T02:09:45.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.766+0000 7f16da85a740 10 client.0 _ll_register_callbacks cb 0x7f16c80109c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:09:45.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.766+0000 7f16d621d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582157cc060 0x5582157df580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:45.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.766+0000 7f16d621d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582157cc060 0x5582157df580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38172/0 (socket says 172.21.15.16:38172) 2024-09-12T02:09:45.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.766+0000 7f16d621d640 1 -- 172.21.15.16:0/712743986 learned_addr learned my addr 172.21.15.16:0/712743986 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:09:45.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.767+0000 7f16d5a1c640 1 --2- 172.21.15.16:0/712743986 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5582157cc980 0x5582157dfac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:45.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.767+0000 7f16d621d640 1 -- 172.21.15.16:0/712743986 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5582157cc980 msgr2=0x5582157dfac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:09:45.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.767+0000 7f16d621d640 1 --2- 172.21.15.16:0/712743986 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5582157cc980 0x5582157dfac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:45.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.767+0000 7f16d621d640 1 -- 172.21.15.16:0/712743986 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5582157d7240 msgr2=0x5582157dc430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:09:45.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.767+0000 7f16d6a1e640 1 --2- 172.21.15.16:0/712743986 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5582157d7240 0x5582157dc430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:45.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.767+0000 7f16d621d640 1 --2- 172.21.15.16:0/712743986 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5582157d7240 0x5582157dc430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:45.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.767+0000 7f16d621d640 1 -- 172.21.15.16:0/712743986 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5582157dcc30 con 0x5582157cc060 2024-09-12T02:09:45.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.767+0000 7f16d621d640 1 --2- 172.21.15.16:0/712743986 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582157cc060 0x5582157df580 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f16cc0a3900 tx=0x7f16cc09b1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7ab8d6540f4475f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:45.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.767+0000 7f16b9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:09:45.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.767+0000 7f16b9ffb640 1 -- 172.21.15.16:0/712743986 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16cc09b8c0 con 0x5582157cc060 2024-09-12T02:09:45.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.767+0000 7f16b9ffb640 1 -- 172.21.15.16:0/712743986 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f16cc0b9e00 con 0x5582157cc060 2024-09-12T02:09:45.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.767+0000 7f16da85a740 1 -- 172.21.15.16:0/712743986 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16cc06d020 con 0x5582157cc060 2024-09-12T02:09:45.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.767+0000 7f16b9ffb640 1 -- 172.21.15.16:0/712743986 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f16cc0cc570 con 0x5582157cc060 2024-09-12T02:09:45.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.767+0000 7f16da85a740 10 client.17871 fetch_fsmap learned FSMap version 25 2024-09-12T02:09:45.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.767+0000 7f16da85a740 1 -- 172.21.15.16:0/712743986 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5582158527f0 con 0x5582157cc060 2024-09-12T02:09:45.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.767+0000 7f16b9ffb640 1 -- 172.21.15.16:0/712743986 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16cc06d020 con 0x5582157cc060 2024-09-12T02:09:45.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.768+0000 7f16b9ffb640 1 -- 172.21.15.16:0/712743986 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f16cc09f120 con 0x5582157cc060 2024-09-12T02:09:45.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.768+0000 7f16da85a740 10 client.17871 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:09:45.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.768+0000 7f16da85a740 10 client.17871 resolve_mds: validated gid 24482 aka daemon mds.b 2024-09-12T02:09:45.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.768+0000 7f16da85a740 20 client.17871 populate_metadata read hostname 'smithi016' 2024-09-12T02:09:45.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.768+0000 7f16da85a740 1 --2- 172.21.15.16:0/712743986 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5582157e6dd0 0x558215854dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:45.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.768+0000 7f16da85a740 4 client.17871 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "d676e596", "release": true, "await": true}] 2024-09-12T02:09:45.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.768+0000 7f16da85a740 1 -- 172.21.15.16:0/712743986 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "quiesce db", "set_id": "d676e596", "release": true, "await": true}) -- 0x5582157390a0 con 0x5582157e6dd0 2024-09-12T02:09:45.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.768+0000 7f16d5a1c640 1 --2- 172.21.15.16:0/712743986 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5582157e6dd0 0x558215854dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:45.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.768+0000 7f16d5a1c640 1 --2- 172.21.15.16:0/712743986 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5582157e6dd0 0x558215854dc0 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b27b45912c3e132e server_cookie=fdf5504671011e83 in_seq=0 out_seq=0 2024-09-12T02:09:45.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.768+0000 7f16b9ffb640 10 client.17871 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:09:45.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.771+0000 7f16b9ffb640 1 -- 172.21.15.16:0/712743986 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5582157390a0 con 0x5582157e6dd0 2024-09-12T02:09:45.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.771+0000 7f16b9ffb640 10 client.17871 handle_command_reply: tid=0 2024-09-12T02:09:45.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.771+0000 7f16da85a740 1 client.17871 shutdown 2024-09-12T02:09:45.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.771+0000 7f16da85a740 1 -- 172.21.15.16:0/712743986 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582157cc060 msgr2=0x5582157df580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:45.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.771+0000 7f16da85a740 1 --2- 172.21.15.16:0/712743986 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582157cc060 0x5582157df580 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f16cc0a3900 tx=0x7f16cc09b1c0 comp rx=0 tx=0).stop 2024-09-12T02:09:45.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.771+0000 7f16da85a740 1 -- 172.21.15.16:0/712743986 shutdown_connections 2024-09-12T02:09:45.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.771+0000 7f16da85a740 1 --2- 172.21.15.16:0/712743986 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5582157e6dd0 0x558215854dc0 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:45.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.771+0000 7f16da85a740 1 --2- 172.21.15.16:0/712743986 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5582157d7240 0x5582157dc430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:45.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.771+0000 7f16da85a740 1 --2- 172.21.15.16:0/712743986 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5582157cc980 0x5582157dfac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:45.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.771+0000 7f16da85a740 1 --2- 172.21.15.16:0/712743986 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582157cc060 0x5582157df580 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:45.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.771+0000 7f16da85a740 1 -- 172.21.15.16:0/712743986 >> 172.21.15.16:0/712743986 conn(0x5582157c85f0 msgr2=0x5582157ce2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:45.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.771+0000 7f16b9ffb640 0 client.17871 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:09:45.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.771+0000 7f16da85a740 1 -- 172.21.15.16:0/712743986 shutdown_connections 2024-09-12T02:09:45.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.771+0000 7f16da85a740 1 -- 172.21.15.16:0/712743986 wait complete. 2024-09-12T02:09:45.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.771+0000 7f16da85a740 20 client.17871 trim_cache size 0 max 16384 2024-09-12T02:09:45.780 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T02:09:45.780 INFO:teuthology.orchestra.run.smithi016.stdout: "epoch": 25, 2024-09-12T02:09:45.780 INFO:teuthology.orchestra.run.smithi016.stdout: "leader": 24482, 2024-09-12T02:09:45.780 INFO:teuthology.orchestra.run.smithi016.stdout: "set_version": 32, 2024-09-12T02:09:45.780 INFO:teuthology.orchestra.run.smithi016.stdout: "sets": { 2024-09-12T02:09:45.780 INFO:teuthology.orchestra.run.smithi016.stdout: "d676e596": { 2024-09-12T02:09:45.780 INFO:teuthology.orchestra.run.smithi016.stdout: "version": 32, 2024-09-12T02:09:45.780 INFO:teuthology.orchestra.run.smithi016.stdout: "age_ref": 0.0, 2024-09-12T02:09:45.780 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-12T02:09:45.780 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RELEASED", 2024-09-12T02:09:45.780 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-12T02:09:45.780 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:09:45.781 INFO:teuthology.orchestra.run.smithi016.stdout: "timeout": 90.0, 2024-09-12T02:09:45.781 INFO:teuthology.orchestra.run.smithi016.stdout: "expiration": 139.4, 2024-09-12T02:09:45.781 INFO:teuthology.orchestra.run.smithi016.stdout: "members": { 2024-09-12T02:09:45.781 INFO:teuthology.orchestra.run.smithi016.stdout: "file:/": { 2024-09-12T02:09:45.781 INFO:teuthology.orchestra.run.smithi016.stdout: "excluded": false, 2024-09-12T02:09:45.781 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-12T02:09:45.781 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RELEASED", 2024-09-12T02:09:45.781 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-12T02:09:45.781 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:09:45.781 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:09:45.781 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:09:45.781 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:09:45.781 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:09:45.782 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T02:09:45.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.772+0000 7f16d8481640 1 -- 172.21.15.16:0/1070670117 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f16bc0a2b80 msgr2=0x7f16bc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:45.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.772+0000 7f16d8481640 1 --2- 172.21.15.16:0/1070670117 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f16bc0a2b80 0x7f16bc0a5040 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f16c8005fd0 tx=0x7f16c8009600 comp rx=0 tx=0).stop 2024-09-12T02:09:45.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.772+0000 7f16d8481640 1 -- 172.21.15.16:0/1070670117 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16d0101de0 msgr2=0x7f16d00744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:45.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.772+0000 7f16d8481640 1 --2- 172.21.15.16:0/1070670117 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16d0101de0 0x7f16d00744f0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f16cc00d950 tx=0x7f16cc00de10 comp rx=0 tx=0).stop 2024-09-12T02:09:45.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.772+0000 7f16d8481640 1 -- 172.21.15.16:0/1070670117 shutdown_connections 2024-09-12T02:09:45.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.772+0000 7f16d8481640 1 --2- 172.21.15.16:0/1070670117 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f16bc0a2b80 0x7f16bc0a5040 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:45.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.772+0000 7f16d8481640 1 --2- 172.21.15.16:0/1070670117 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f16d0106f10 0x7f16d0074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:45.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.772+0000 7f16d8481640 1 --2- 172.21.15.16:0/1070670117 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f16d0102790 0x7f16d0074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:45.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.772+0000 7f16d8481640 1 --2- 172.21.15.16:0/1070670117 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16d0101de0 0x7f16d00744f0 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:45.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.772+0000 7f16d8481640 1 -- 172.21.15.16:0/1070670117 >> 172.21.15.16:0/1070670117 conn(0x7f16d00fd8c0 msgr2=0x7f16d00fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:45.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.772+0000 7f16d8481640 1 -- 172.21.15.16:0/1070670117 shutdown_connections 2024-09-12T02:09:45.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:45.772+0000 7f16d8481640 1 -- 172.21.15.16:0/1070670117 wait complete. 2024-09-12T02:09:45.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:09:45 smithi184 ceph-mon[35593]: pgmap v570: 97 pgs: 97 active+clean; 4.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 2.9 KiB/s wr, 10 op/s 2024-09-12T02:09:45.918 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:09:45 smithi033 ceph-mon[35332]: pgmap v570: 97 pgs: 97 active+clean; 4.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 2.9 KiB/s wr, 10 op/s 2024-09-12T02:09:46.440 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'd676e596', total seconds elapsed: 22.5 2024-09-12T02:09:46.440 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 19.4 2024-09-12T02:09:46.440 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:09:46.699 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:09:47.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:47.194+0000 7fc3ba82d640 1 -- 172.21.15.16:0/1081696692 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3b4102d40 msgr2=0x7fc3b4103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:47.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:47.194+0000 7fc3ba82d640 1 --2- 172.21.15.16:0/1081696692 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3b4102d40 0x7fc3b4103120 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7fc3a4009900 tx=0x7fc3a402a480 comp rx=0 tx=0).stop 2024-09-12T02:09:47.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:47.194+0000 7fc3ba82d640 1 -- 172.21.15.16:0/1081696692 shutdown_connections 2024-09-12T02:09:47.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:47.194+0000 7fc3ba82d640 1 --2- 172.21.15.16:0/1081696692 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3b410c170 0x7fc3b410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:47.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:47.194+0000 7fc3ba82d640 1 --2- 172.21.15.16:0/1081696692 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc3b4103660 0x7fc3b410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:47.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:47.194+0000 7fc3ba82d640 1 --2- 172.21.15.16:0/1081696692 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3b4102d40 0x7fc3b4103120 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:47.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:47.194+0000 7fc3ba82d640 1 -- 172.21.15.16:0/1081696692 >> 172.21.15.16:0/1081696692 conn(0x7fc3b40fc7a0 msgr2=0x7fc3b40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:47.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:47.194+0000 7fc3ba82d640 1 -- 172.21.15.16:0/1081696692 shutdown_connections 2024-09-12T02:09:47.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:47.194+0000 7fc3ba82d640 1 -- 172.21.15.16:0/1081696692 wait complete. 2024-09-12T02:09:47.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:47.195+0000 7fc3ba82d640 1 Processor -- start 2024-09-12T02:09:47.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:47.195+0000 7fc3ba82d640 1 -- start start 2024-09-12T02:09:47.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:47.196+0000 7fc3ba82d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3b4102d40 0x7fc3b41984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:47.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:47.196+0000 7fc3ba82d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc3b4103660 0x7fc3b4198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:47.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:47.196+0000 7fc3ba82d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3b410c170 0x7fc3b419cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:47.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:47.196+0000 7fc3ba82d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc3b4111f70 con 0x7fc3b410c170 2024-09-12T02:09:47.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:47.196+0000 7fc3ba82d640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc3b4111df0 con 0x7fc3b4102d40 2024-09-12T02:09:47.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:47.196+0000 7fc3ba82d640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc3b41120f0 con 0x7fc3b4103660 2024-09-12T02:09:47.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:47.196+0000 7fc3b8dca640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3b410c170 0x7fc3b419cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:47.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:47.196+0000 7fc3b8dca640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3b410c170 0x7fc3b419cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38188/0 (socket says 172.21.15.16:38188) 2024-09-12T02:09:47.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:47.196+0000 7fc3b8dca640 1 -- 172.21.15.16:0/400987530 learned_addr learned my addr 172.21.15.16:0/400987530 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:09:47.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:47.196+0000 7fc3b3fff640 1 --2- 172.21.15.16:0/400987530 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3b4102d40 0x7fc3b41984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:47.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:47.196+0000 7fc3abfff640 1 --2- 172.21.15.16:0/400987530 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc3b4103660 0x7fc3b4198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:47.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:47.196+0000 7fc3b8dca640 1 -- 172.21.15.16:0/400987530 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc3b4103660 msgr2=0x7fc3b4198a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:47.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:47.196+0000 7fc3b8dca640 1 --2- 172.21.15.16:0/400987530 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc3b4103660 0x7fc3b4198a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:47.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:47.196+0000 7fc3b8dca640 1 -- 172.21.15.16:0/400987530 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3b4102d40 msgr2=0x7fc3b41984f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:47.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:47.196+0000 7fc3b8dca640 1 --2- 172.21.15.16:0/400987530 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3b4102d40 0x7fc3b41984f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:47.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:47.196+0000 7fc3b8dca640 1 -- 172.21.15.16:0/400987530 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3b419d4a0 con 0x7fc3b410c170 2024-09-12T02:09:47.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:47.196+0000 7fc3abfff640 1 --2- 172.21.15.16:0/400987530 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc3b4103660 0x7fc3b4198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:09:47.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:47.196+0000 7fc3b8dca640 1 --2- 172.21.15.16:0/400987530 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3b410c170 0x7fc3b419cdc0 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7fc3ac00cad0 tx=0x7fc3ac00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:47.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:47.196+0000 7fc3b1ffb640 1 -- 172.21.15.16:0/400987530 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3ac00ae20 con 0x7fc3b410c170 2024-09-12T02:09:47.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:47.196+0000 7fc3b1ffb640 1 -- 172.21.15.16:0/400987530 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3ac005020 con 0x7fc3b410c170 2024-09-12T02:09:47.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:47.196+0000 7fc3ba82d640 1 -- 172.21.15.16:0/400987530 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3b419d790 con 0x7fc3b410c170 2024-09-12T02:09:47.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:47.197+0000 7fc3b1ffb640 1 -- 172.21.15.16:0/400987530 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3ac00f6f0 con 0x7fc3b410c170 2024-09-12T02:09:47.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:47.197+0000 7fc3ba82d640 1 -- 172.21.15.16:0/400987530 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3b41a5070 con 0x7fc3b410c170 2024-09-12T02:09:47.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:47.198+0000 7fc3ba82d640 1 -- 172.21.15.16:0/400987530 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3b4103120 con 0x7fc3b410c170 2024-09-12T02:09:47.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:47.199+0000 7fc3b1ffb640 1 -- 172.21.15.16:0/400987530 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fc3ac0051d0 con 0x7fc3b410c170 2024-09-12T02:09:47.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:47.199+0000 7fc3b1ffb640 1 --2- 172.21.15.16:0/400987530 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc3a00a2b80 0x7fc3a00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:47.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:47.199+0000 7fc3b3fff640 1 --2- 172.21.15.16:0/400987530 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc3a00a2b80 0x7fc3a00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:47.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:47.200+0000 7fc3b1ffb640 1 -- 172.21.15.16:0/400987530 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fc3ac0cefd0 con 0x7fc3b410c170 2024-09-12T02:09:47.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:47.200+0000 7fc3b3fff640 1 --2- 172.21.15.16:0/400987530 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc3a00a2b80 0x7fc3a00a5040 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fc3a4000c00 tx=0x7fc3a40058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:47.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:47.203+0000 7fc3b1ffb640 1 -- 172.21.15.16:0/400987530 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc3ac014030 con 0x7fc3b410c170 2024-09-12T02:09:47.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:47.417+0000 7fc3ba82d640 1 -- 172.21.15.16:0/400987530 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc3b4106c50 con 0x7fc3b410c170 2024-09-12T02:09:47.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:47.418+0000 7fc3b1ffb640 1 -- 172.21.15.16:0/400987530 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11911 (secure 0 0 0) 0x7fc3ac09bfd0 con 0x7fc3b410c170 2024-09-12T02:09:47.418 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T02:09:47.419 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":25,"btime":"2024-09-12T02:01:37:192021+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3279123279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3279123279},{"type":"v1","addr":"172.21.15.16:6835","nonce":3279123279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4119452382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4119452382},{"type":"v1","addr":"172.21.15.16:6837","nonce":4119452382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/3263823165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3263823165},{"type":"v1","addr":"172.21.15.16:6839","nonce":3263823165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/1510290732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":1510290732},{"type":"v1","addr":"172.21.15.16:6841","nonce":1510290732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/279674603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":279674603},{"type":"v1","addr":"172.21.15.33:6833","nonce":279674603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3555728028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3555728028},{"type":"v1","addr":"172.21.15.33:6835","nonce":3555728028}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T01:59:56.117608+0000","modified":"2024-09-12T02:01:36.294760+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24482,"mds_1":24481,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6833/1563374106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1563374106},{"type":"v1","addr":"172.21.15.184:6833","nonce":1563374106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6837/1892833868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1892833868},{"type":"v1","addr":"172.21.15.33:6837","nonce":1892833868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/1457679686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1457679686},{"type":"v1","addr":"172.21.15.184:6835","nonce":1457679686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.184:6837/1793393087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":1793393087},{"type":"v1","addr":"172.21.15.184:6837","nonce":1793393087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.184:6839/4203347941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":4203347941},{"type":"v1","addr":"172.21.15.184:6839","nonce":4203347941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.33:6839/4136670204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":4136670204},{"type":"v1","addr":"172.21.15.33:6839","nonce":4136670204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24481,24482,24469]},"id":1}]} 2024-09-12T02:09:47.419 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 25 2024-09-12T02:09:47.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:47.421+0000 7fc3ba82d640 1 -- 172.21.15.16:0/400987530 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc3a00a2b80 msgr2=0x7fc3a00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:47.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:47.421+0000 7fc3ba82d640 1 --2- 172.21.15.16:0/400987530 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc3a00a2b80 0x7fc3a00a5040 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fc3a4000c00 tx=0x7fc3a40058d0 comp rx=0 tx=0).stop 2024-09-12T02:09:47.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:47.421+0000 7fc3ba82d640 1 -- 172.21.15.16:0/400987530 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3b410c170 msgr2=0x7fc3b419cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:47.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:47.421+0000 7fc3ba82d640 1 --2- 172.21.15.16:0/400987530 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3b410c170 0x7fc3b419cdc0 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7fc3ac00cad0 tx=0x7fc3ac00cf90 comp rx=0 tx=0).stop 2024-09-12T02:09:47.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:47.421+0000 7fc3ba82d640 1 -- 172.21.15.16:0/400987530 shutdown_connections 2024-09-12T02:09:47.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:47.421+0000 7fc3ba82d640 1 --2- 172.21.15.16:0/400987530 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc3a00a2b80 0x7fc3a00a5040 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:47.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:47.421+0000 7fc3ba82d640 1 --2- 172.21.15.16:0/400987530 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3b410c170 0x7fc3b419cdc0 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:47.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:47.421+0000 7fc3ba82d640 1 --2- 172.21.15.16:0/400987530 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc3b4103660 0x7fc3b4198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:47.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:47.421+0000 7fc3ba82d640 1 --2- 172.21.15.16:0/400987530 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3b4102d40 0x7fc3b41984f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:47.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:47.421+0000 7fc3ba82d640 1 -- 172.21.15.16:0/400987530 >> 172.21.15.16:0/400987530 conn(0x7fc3b40fc7a0 msgr2=0x7fc3b40fd4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:47.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:47.421+0000 7fc3ba82d640 1 -- 172.21.15.16:0/400987530 shutdown_connections 2024-09-12T02:09:47.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:47.421+0000 7fc3ba82d640 1 -- 172.21.15.16:0/400987530 wait complete. 2024-09-12T02:09:47.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:09:47 smithi016 ceph-mon[30947]: pgmap v571: 97 pgs: 97 active+clean; 4.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 2.2 KiB/s wr, 10 op/s 2024-09-12T02:09:47.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:09:47 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/400987530' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:09:47.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:09:47 smithi184 ceph-mon[35593]: pgmap v571: 97 pgs: 97 active+clean; 4.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 2.2 KiB/s wr, 10 op/s 2024-09-12T02:09:47.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:09:47 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/400987530' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:09:47.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:09:47 smithi033 ceph-mon[35332]: pgmap v571: 97 pgs: 97 active+clean; 4.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 2.2 KiB/s wr, 10 op/s 2024-09-12T02:09:47.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:09:47 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/400987530' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:09:48.080 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24482 quiesce db / --timeout 90 --expiration 139.4 --await-for 45.0' 2024-09-12T02:09:48.080 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24482 quiesce db / --timeout 90 --expiration 139.4 --await-for 45.0 2024-09-12T02:09:48.339 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:09:48.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.863+0000 7f315a42c640 1 -- 172.21.15.16:0/2970370187 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3154106f10 msgr2=0x7f315410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:48.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.863+0000 7f315a42c640 1 --2- 172.21.15.16:0/2970370187 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3154106f10 0x7f315410ea90 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f314c009ca0 tx=0x7f314c02a8c0 comp rx=0 tx=0).stop 2024-09-12T02:09:48.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.863+0000 7f315a42c640 1 -- 172.21.15.16:0/2970370187 shutdown_connections 2024-09-12T02:09:48.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.863+0000 7f315a42c640 1 --2- 172.21.15.16:0/2970370187 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3154106f10 0x7f315410ea90 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:48.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.863+0000 7f315a42c640 1 --2- 172.21.15.16:0/2970370187 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3154102790 0x7f31541067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:48.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.863+0000 7f315a42c640 1 --2- 172.21.15.16:0/2970370187 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3154101de0 0x7f31541021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:48.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.863+0000 7f315a42c640 1 -- 172.21.15.16:0/2970370187 >> 172.21.15.16:0/2970370187 conn(0x7f31540fd8c0 msgr2=0x7f31540ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:48.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.863+0000 7f315a42c640 1 -- 172.21.15.16:0/2970370187 shutdown_connections 2024-09-12T02:09:48.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.863+0000 7f315a42c640 1 -- 172.21.15.16:0/2970370187 wait complete. 2024-09-12T02:09:48.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.864+0000 7f315a42c640 1 Processor -- start 2024-09-12T02:09:48.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.864+0000 7f315a42c640 1 -- start start 2024-09-12T02:09:48.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.864+0000 7f315a42c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3154101de0 0x7f315419a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:48.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.864+0000 7f315a42c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3154102790 0x7f315419ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:48.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.864+0000 7f315a42c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3154106f10 0x7f315419eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:48.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.864+0000 7f315a42c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3154111f70 con 0x7f3154101de0 2024-09-12T02:09:48.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.864+0000 7f315a42c640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3154111df0 con 0x7f3154106f10 2024-09-12T02:09:48.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.864+0000 7f315a42c640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f31541120f0 con 0x7f3154102790 2024-09-12T02:09:48.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.864+0000 7f3153fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3154101de0 0x7f315419a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:48.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.864+0000 7f3153fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3154101de0 0x7f315419a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38206/0 (socket says 172.21.15.16:38206) 2024-09-12T02:09:48.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.864+0000 7f3153fff640 1 -- 172.21.15.16:0/3978377244 learned_addr learned my addr 172.21.15.16:0/3978377244 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:09:48.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.864+0000 7f31589c9640 1 --2- 172.21.15.16:0/3978377244 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3154106f10 0x7f315419eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:48.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.864+0000 7f314bfff640 1 --2- 172.21.15.16:0/3978377244 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3154102790 0x7f315419ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:48.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.864+0000 7f3153fff640 1 -- 172.21.15.16:0/3978377244 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3154102790 msgr2=0x7f315419ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:48.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.864+0000 7f3153fff640 1 --2- 172.21.15.16:0/3978377244 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3154102790 0x7f315419ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:48.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.865+0000 7f3153fff640 1 -- 172.21.15.16:0/3978377244 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3154106f10 msgr2=0x7f315419eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:48.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.865+0000 7f3153fff640 1 --2- 172.21.15.16:0/3978377244 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3154106f10 0x7f315419eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:48.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.865+0000 7f3153fff640 1 -- 172.21.15.16:0/3978377244 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f315419f590 con 0x7f3154101de0 2024-09-12T02:09:48.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.865+0000 7f31589c9640 1 --2- 172.21.15.16:0/3978377244 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3154106f10 0x7f315419eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:09:48.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.865+0000 7f3153fff640 1 --2- 172.21.15.16:0/3978377244 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3154101de0 0x7f315419a5e0 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f314400c920 tx=0x7f314400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:48.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.865+0000 7f3151ffb640 1 -- 172.21.15.16:0/3978377244 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3144007c70 con 0x7f3154101de0 2024-09-12T02:09:48.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.865+0000 7f3151ffb640 1 -- 172.21.15.16:0/3978377244 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3144007e10 con 0x7f3154101de0 2024-09-12T02:09:48.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.865+0000 7f315a42c640 1 -- 172.21.15.16:0/3978377244 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f315419f880 con 0x7f3154101de0 2024-09-12T02:09:48.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.865+0000 7f3151ffb640 1 -- 172.21.15.16:0/3978377244 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3144018890 con 0x7f3154101de0 2024-09-12T02:09:48.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.865+0000 7f315a42c640 1 -- 172.21.15.16:0/3978377244 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31541a7160 con 0x7f3154101de0 2024-09-12T02:09:48.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.867+0000 7f3151ffb640 1 -- 172.21.15.16:0/3978377244 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f3144018a70 con 0x7f3154101de0 2024-09-12T02:09:48.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.868+0000 7f315c805740 1 Processor -- start 2024-09-12T02:09:48.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.868+0000 7f315c805740 1 -- start start 2024-09-12T02:09:48.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.868+0000 7f315c805740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5555ec5b9760 0x5555ec5b9b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:48.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.868+0000 7f315c805740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5555ec5ba080 0x5555ec5c4400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:48.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.868+0000 7f315c805740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5555ec5c4940 0x5555ec5c6d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:48.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.868+0000 7f315c805740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5555ec51f2a0 con 0x5555ec5ba080 2024-09-12T02:09:48.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.868+0000 7f315c805740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5555ec4c6b10 con 0x5555ec5c4940 2024-09-12T02:09:48.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.868+0000 7f315c805740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5555ec51b260 con 0x5555ec5b9760 2024-09-12T02:09:48.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.868+0000 7f3153fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5555ec5ba080 0x5555ec5c4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:48.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.868+0000 7f3153fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5555ec5ba080 0x5555ec5c4400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38214/0 (socket says 172.21.15.16:38214) 2024-09-12T02:09:48.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.868+0000 7f3153fff640 1 -- 172.21.15.16:0/471907514 learned_addr learned my addr 172.21.15.16:0/471907514 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:09:48.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.868+0000 7f31589c9640 1 --2- 172.21.15.16:0/471907514 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5555ec5b9760 0x5555ec5b9b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:48.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.868+0000 7f314bfff640 1 --2- 172.21.15.16:0/471907514 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5555ec5c4940 0x5555ec5c6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:48.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.868+0000 7f3151ffb640 1 --2- 172.21.15.16:0/3978377244 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f31400a2c00 0x7f31400a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:48.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.868+0000 7f3153fff640 1 -- 172.21.15.16:0/471907514 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5555ec5b9760 msgr2=0x5555ec5b9b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:09:48.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.868+0000 7f3153fff640 1 --2- 172.21.15.16:0/471907514 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5555ec5b9760 0x5555ec5b9b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:48.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.868+0000 7f3153fff640 1 -- 172.21.15.16:0/471907514 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5555ec5c4940 msgr2=0x5555ec5c6d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:09:48.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.868+0000 7f3153fff640 1 --2- 172.21.15.16:0/471907514 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5555ec5c4940 0x5555ec5c6d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:48.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.868+0000 7f3153fff640 1 -- 172.21.15.16:0/471907514 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5555ec290b90 con 0x5555ec5ba080 2024-09-12T02:09:48.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.868+0000 7f314bfff640 1 --2- 172.21.15.16:0/471907514 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5555ec5c4940 0x5555ec5c6d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:09:48.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.869+0000 7f31589c9640 1 --2- 172.21.15.16:0/3978377244 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f31400a2c00 0x7f31400a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:48.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.869+0000 7f3151ffb640 1 -- 172.21.15.16:0/3978377244 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f31440cf170 con 0x7f3154101de0 2024-09-12T02:09:48.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.869+0000 7f3153fff640 1 --2- 172.21.15.16:0/471907514 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5555ec5ba080 0x5555ec5c4400 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f3144094a30 tx=0x7f3144090f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6572288fbdc2d848 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:48.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.869+0000 7f314a7fc640 1 -- 172.21.15.16:0/471907514 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f314409b5b0 con 0x5555ec5ba080 2024-09-12T02:09:48.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.869+0000 7f314a7fc640 1 -- 172.21.15.16:0/471907514 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3144094a90 con 0x5555ec5ba080 2024-09-12T02:09:48.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.869+0000 7f314a7fc640 1 -- 172.21.15.16:0/471907514 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31440d2d80 con 0x5555ec5ba080 2024-09-12T02:09:48.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.869+0000 7f31589c9640 1 --2- 172.21.15.16:0/3978377244 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f31400a2c00 0x7f31400a50c0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f314c006010 tx=0x7f314c03e040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:48.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.870+0000 7f315c805740 1 -- 172.21.15.16:0/471907514 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5555ec5ba080 msgr2=0x5555ec5c4400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:48.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.870+0000 7f315c805740 1 --2- 172.21.15.16:0/471907514 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5555ec5ba080 0x5555ec5c4400 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f3144094a30 tx=0x7f3144090f50 comp rx=0 tx=0).stop 2024-09-12T02:09:48.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.870+0000 7f315c805740 1 -- 172.21.15.16:0/471907514 shutdown_connections 2024-09-12T02:09:48.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.870+0000 7f315c805740 1 --2- 172.21.15.16:0/471907514 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5555ec5c4940 0x5555ec5c6d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:48.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.870+0000 7f315c805740 1 --2- 172.21.15.16:0/471907514 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5555ec5ba080 0x5555ec5c4400 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:48.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.870+0000 7f315c805740 1 --2- 172.21.15.16:0/471907514 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5555ec5b9760 0x5555ec5b9b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:48.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.870+0000 7f315c805740 1 -- 172.21.15.16:0/471907514 >> 172.21.15.16:0/471907514 conn(0x5555ec5b5cf0 msgr2=0x5555ec5b60d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:48.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.870+0000 7f315c805740 1 -- 172.21.15.16:0/471907514 shutdown_connections 2024-09-12T02:09:48.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.870+0000 7f315c805740 1 -- 172.21.15.16:0/471907514 wait complete. 2024-09-12T02:09:48.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.870+0000 7f315c805740 1 -- start start 2024-09-12T02:09:48.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.870+0000 7f315c805740 1 Processor -- start 2024-09-12T02:09:48.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.871+0000 7f315c805740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5555ec5b9760 0x5555ec5c17e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:48.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.871+0000 7f315c805740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5555ec5ba080 0x5555ec5c1d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:48.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.871+0000 7f315c805740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5555ec5c4940 0x5555ec5c2260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:48.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.871+0000 7f315c805740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5555ec5c7510 con 0x5555ec5c4940 2024-09-12T02:09:48.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.871+0000 7f315c805740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5555ec4a76e0 con 0x5555ec5b9760 2024-09-12T02:09:48.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.871+0000 7f315c805740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5555ec5c7690 con 0x5555ec5ba080 2024-09-12T02:09:48.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.871+0000 7f315c805740 10 client.0 _ll_register_callbacks cb 0x5555ec4cba80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:09:48.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.871+0000 7f31589c9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5555ec5c4940 0x5555ec5c2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:48.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.871+0000 7f314bfff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5555ec5ba080 0x5555ec5c1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:48.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.871+0000 7f31589c9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5555ec5c4940 0x5555ec5c2260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38228/0 (socket says 172.21.15.16:38228) 2024-09-12T02:09:48.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.871+0000 7f31589c9640 1 -- 172.21.15.16:0/3974065837 learned_addr learned my addr 172.21.15.16:0/3974065837 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:09:48.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.871+0000 7f3153fff640 1 --2- 172.21.15.16:0/3974065837 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5555ec5b9760 0x5555ec5c17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:48.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.871+0000 7f31589c9640 1 -- 172.21.15.16:0/3974065837 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5555ec5ba080 msgr2=0x5555ec5c1d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:09:48.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.871+0000 7f31589c9640 1 --2- 172.21.15.16:0/3974065837 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5555ec5ba080 0x5555ec5c1d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:48.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.871+0000 7f31589c9640 1 -- 172.21.15.16:0/3974065837 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5555ec5b9760 msgr2=0x5555ec5c17e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:09:48.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.871+0000 7f31589c9640 1 --2- 172.21.15.16:0/3974065837 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5555ec5b9760 0x5555ec5c17e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:48.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.871+0000 7f31589c9640 1 -- 172.21.15.16:0/3974065837 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5555ec5d09d0 con 0x5555ec5c4940 2024-09-12T02:09:48.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.871+0000 7f3153fff640 1 --2- 172.21.15.16:0/3974065837 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5555ec5b9760 0x5555ec5c17e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:09:48.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.872+0000 7f31589c9640 1 --2- 172.21.15.16:0/3974065837 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5555ec5c4940 0x5555ec5c2260 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f314c003eb0 tx=0x7f314c0095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afc4b154f59c830e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:48.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.872+0000 7f31337fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:09:48.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.872+0000 7f31337fe640 1 -- 172.21.15.16:0/3974065837 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f314c004450 con 0x5555ec5c4940 2024-09-12T02:09:48.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.872+0000 7f315c805740 1 -- 172.21.15.16:0/3974065837 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5555ec52c9d0 con 0x5555ec5c4940 2024-09-12T02:09:48.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.872+0000 7f31337fe640 1 -- 172.21.15.16:0/3974065837 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f314c0045f0 con 0x5555ec5c4940 2024-09-12T02:09:48.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.872+0000 7f31337fe640 1 -- 172.21.15.16:0/3974065837 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f314c0319d0 con 0x5555ec5c4940 2024-09-12T02:09:48.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.872+0000 7f315c805740 10 client.17889 fetch_fsmap learned FSMap version 25 2024-09-12T02:09:48.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.872+0000 7f315c805740 1 -- 172.21.15.16:0/3974065837 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5555ec5d3340 con 0x5555ec5c4940 2024-09-12T02:09:48.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.872+0000 7f31337fe640 1 -- 172.21.15.16:0/3974065837 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5555ec52c9d0 con 0x5555ec5c4940 2024-09-12T02:09:48.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.872+0000 7f31337fe640 1 -- 172.21.15.16:0/3974065837 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f314c0306e0 con 0x5555ec5c4940 2024-09-12T02:09:48.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.872+0000 7f315c805740 10 client.17889 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:09:48.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.872+0000 7f315c805740 10 client.17889 resolve_mds: validated gid 24482 aka daemon mds.b 2024-09-12T02:09:48.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.872+0000 7f315c805740 20 client.17889 populate_metadata read hostname 'smithi016' 2024-09-12T02:09:48.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.872+0000 7f315c805740 1 --2- 172.21.15.16:0/3974065837 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5555ec5d47f0 0x5555ec5d6c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:48.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.872+0000 7f315c805740 4 client.17889 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:09:48.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.872+0000 7f315c805740 1 -- 172.21.15.16:0/3974065837 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5555ec526d20 con 0x5555ec5d47f0 2024-09-12T02:09:48.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.873+0000 7f3153fff640 1 --2- 172.21.15.16:0/3974065837 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5555ec5d47f0 0x5555ec5d6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:48.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.873+0000 7f3153fff640 1 --2- 172.21.15.16:0/3974065837 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5555ec5d47f0 0x5555ec5d6c70 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0f704ceeee0c815 server_cookie=fd1c8847c4001294 in_seq=0 out_seq=0 2024-09-12T02:09:48.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.873+0000 7f31337fe640 10 client.17889 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:09:48.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.874+0000 7f31337fe640 1 -- 172.21.15.16:0/3974065837 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5555ec526d20 con 0x5555ec5d47f0 2024-09-12T02:09:48.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.874+0000 7f31337fe640 10 client.17889 handle_command_reply: tid=0 2024-09-12T02:09:48.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.874+0000 7f315c805740 1 client.17889 shutdown 2024-09-12T02:09:48.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.874+0000 7f315c805740 1 -- 172.21.15.16:0/3974065837 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5555ec5c4940 msgr2=0x5555ec5c2260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:48.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.874+0000 7f315c805740 1 --2- 172.21.15.16:0/3974065837 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5555ec5c4940 0x5555ec5c2260 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f314c003eb0 tx=0x7f314c0095c0 comp rx=0 tx=0).stop 2024-09-12T02:09:48.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.874+0000 7f315c805740 1 -- 172.21.15.16:0/3974065837 shutdown_connections 2024-09-12T02:09:48.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.874+0000 7f315c805740 1 --2- 172.21.15.16:0/3974065837 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5555ec5d47f0 0x5555ec5d6c70 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:48.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.874+0000 7f315c805740 1 --2- 172.21.15.16:0/3974065837 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5555ec5c4940 0x5555ec5c2260 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:48.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.874+0000 7f315c805740 1 --2- 172.21.15.16:0/3974065837 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5555ec5ba080 0x5555ec5c1d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:48.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.874+0000 7f315c805740 1 --2- 172.21.15.16:0/3974065837 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5555ec5b9760 0x5555ec5c17e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:48.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.874+0000 7f31337fe640 0 client.17889 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:09:48.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.874+0000 7f315c805740 1 -- 172.21.15.16:0/3974065837 >> 172.21.15.16:0/3974065837 conn(0x5555ec5b5cf0 msgr2=0x5555ec5bba20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:48.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.874+0000 7f315c805740 1 -- 172.21.15.16:0/3974065837 shutdown_connections 2024-09-12T02:09:48.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.874+0000 7f315c805740 1 -- 172.21.15.16:0/3974065837 wait complete. 2024-09-12T02:09:48.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.874+0000 7f315c805740 20 client.17889 trim_cache size 0 max 16384 2024-09-12T02:09:48.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.891+0000 7f315c805740 1 Processor -- start 2024-09-12T02:09:48.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.891+0000 7f315c805740 1 -- start start 2024-09-12T02:09:48.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.891+0000 7f315c805740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5555ec5b9760 0x5555ec5bc1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:48.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.891+0000 7f315c805740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5555ec5ba080 0x5555ec5b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:48.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.891+0000 7f315c805740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5555ec5c4940 0x5555ec5dc540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:48.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.891+0000 7f315c805740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f31541b1eb0 con 0x5555ec5b9760 2024-09-12T02:09:48.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.891+0000 7f315c805740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f31541af780 con 0x5555ec5ba080 2024-09-12T02:09:48.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.891+0000 7f315c805740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f31541ad5d0 con 0x5555ec5c4940 2024-09-12T02:09:48.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.891+0000 7f3153fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5555ec5b9760 0x5555ec5bc1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:48.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.891+0000 7f3153fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5555ec5b9760 0x5555ec5bc1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38240/0 (socket says 172.21.15.16:38240) 2024-09-12T02:09:48.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.891+0000 7f3153fff640 1 -- 172.21.15.16:0/4035123616 learned_addr learned my addr 172.21.15.16:0/4035123616 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:09:48.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.892+0000 7f314bfff640 1 --2- 172.21.15.16:0/4035123616 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5555ec5ba080 0x5555ec5b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:48.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.892+0000 7f3153fff640 1 -- 172.21.15.16:0/4035123616 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5555ec5c4940 msgr2=0x5555ec5dc540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:09:48.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.892+0000 7f3153fff640 1 --2- 172.21.15.16:0/4035123616 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5555ec5c4940 0x5555ec5dc540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:48.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.892+0000 7f3153fff640 1 -- 172.21.15.16:0/4035123616 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5555ec5ba080 msgr2=0x5555ec5b4d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:09:48.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.892+0000 7f3153fff640 1 --2- 172.21.15.16:0/4035123616 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5555ec5ba080 0x5555ec5b4d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:48.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.892+0000 7f3153fff640 1 -- 172.21.15.16:0/4035123616 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5555ec5b5b20 con 0x5555ec5b9760 2024-09-12T02:09:48.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.892+0000 7f3153fff640 1 --2- 172.21.15.16:0/4035123616 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5555ec5b9760 0x5555ec5bc1c0 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f314409ba00 tx=0x7f3144090f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=195262c67a5c7c81 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:48.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.892+0000 7f3149ffb640 1 -- 172.21.15.16:0/4035123616 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3144066de0 con 0x5555ec5b9760 2024-09-12T02:09:48.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.892+0000 7f3149ffb640 1 -- 172.21.15.16:0/4035123616 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f314409c170 con 0x5555ec5b9760 2024-09-12T02:09:48.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.892+0000 7f3149ffb640 1 -- 172.21.15.16:0/4035123616 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f314409b3d0 con 0x5555ec5b9760 2024-09-12T02:09:48.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.893+0000 7f315c805740 1 -- 172.21.15.16:0/4035123616 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5555ec5b9760 msgr2=0x5555ec5bc1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:48.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.893+0000 7f315c805740 1 --2- 172.21.15.16:0/4035123616 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5555ec5b9760 0x5555ec5bc1c0 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f314409ba00 tx=0x7f3144090f50 comp rx=0 tx=0).stop 2024-09-12T02:09:48.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.893+0000 7f315c805740 1 -- 172.21.15.16:0/4035123616 shutdown_connections 2024-09-12T02:09:48.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.893+0000 7f315c805740 1 --2- 172.21.15.16:0/4035123616 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5555ec5c4940 0x5555ec5dc540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:48.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.893+0000 7f315c805740 1 --2- 172.21.15.16:0/4035123616 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5555ec5ba080 0x5555ec5b4d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:48.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.893+0000 7f315c805740 1 --2- 172.21.15.16:0/4035123616 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5555ec5b9760 0x5555ec5bc1c0 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:48.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.893+0000 7f315c805740 1 -- 172.21.15.16:0/4035123616 >> 172.21.15.16:0/4035123616 conn(0x5555ec5b5cf0 msgr2=0x5555ec5b4650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:48.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.893+0000 7f315c805740 1 -- 172.21.15.16:0/4035123616 shutdown_connections 2024-09-12T02:09:48.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.893+0000 7f315c805740 1 -- 172.21.15.16:0/4035123616 wait complete. 2024-09-12T02:09:48.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.893+0000 7f315c805740 1 -- start start 2024-09-12T02:09:48.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.893+0000 7f315c805740 1 Processor -- start 2024-09-12T02:09:48.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.894+0000 7f315c805740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5555ec5b9760 0x5555ec5c87a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:48.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.894+0000 7f315c805740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5555ec5ba080 0x5555ec5c8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:48.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.894+0000 7f315c805740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5555ec5c4940 0x5555ec5c9220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:48.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.894+0000 7f315c805740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f31541adc40 con 0x5555ec5ba080 2024-09-12T02:09:48.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.894+0000 7f315c805740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f31541af0b0 con 0x5555ec5b9760 2024-09-12T02:09:48.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.894+0000 7f315c805740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f31541ae310 con 0x5555ec5c4940 2024-09-12T02:09:48.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.894+0000 7f315c805740 10 client.0 _ll_register_callbacks cb 0x7f314c003de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:09:48.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.894+0000 7f314bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5555ec5ba080 0x5555ec5c8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:48.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.894+0000 7f314bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5555ec5ba080 0x5555ec5c8ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38250/0 (socket says 172.21.15.16:38250) 2024-09-12T02:09:48.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.894+0000 7f314bfff640 1 -- 172.21.15.16:0/3597529642 learned_addr learned my addr 172.21.15.16:0/3597529642 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:09:48.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.894+0000 7f3153fff640 1 --2- 172.21.15.16:0/3597529642 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5555ec5b9760 0x5555ec5c87a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:48.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.894+0000 7f31589c9640 1 --2- 172.21.15.16:0/3597529642 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5555ec5c4940 0x5555ec5c9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:48.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.894+0000 7f314bfff640 1 -- 172.21.15.16:0/3597529642 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5555ec5c4940 msgr2=0x5555ec5c9220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:09:48.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.894+0000 7f314bfff640 1 --2- 172.21.15.16:0/3597529642 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5555ec5c4940 0x5555ec5c9220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:48.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.894+0000 7f314bfff640 1 -- 172.21.15.16:0/3597529642 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5555ec5b9760 msgr2=0x5555ec5c87a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:09:48.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.894+0000 7f314bfff640 1 --2- 172.21.15.16:0/3597529642 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5555ec5b9760 0x5555ec5c87a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:48.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.894+0000 7f314bfff640 1 -- 172.21.15.16:0/3597529642 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5555ec5b6980 con 0x5555ec5ba080 2024-09-12T02:09:48.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.894+0000 7f314bfff640 1 --2- 172.21.15.16:0/3597529642 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5555ec5ba080 0x5555ec5c8ce0 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x5555ec5b7070 tx=0x7f313c013830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=481e83fc765e73 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:48.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.894+0000 7f31337fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:09:48.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.894+0000 7f31337fe640 1 -- 172.21.15.16:0/3597529642 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f313c004240 con 0x5555ec5ba080 2024-09-12T02:09:48.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.894+0000 7f31337fe640 1 -- 172.21.15.16:0/3597529642 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f313c0043e0 con 0x5555ec5ba080 2024-09-12T02:09:48.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.894+0000 7f315c805740 1 -- 172.21.15.16:0/3597529642 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f314409c170 con 0x5555ec5ba080 2024-09-12T02:09:48.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.895+0000 7f31337fe640 1 -- 172.21.15.16:0/3597529642 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f313c0035f0 con 0x5555ec5ba080 2024-09-12T02:09:48.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.895+0000 7f315c805740 10 client.17895 fetch_fsmap learned FSMap version 25 2024-09-12T02:09:48.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.895+0000 7f315c805740 1 -- 172.21.15.16:0/3597529642 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5555ec63f7e0 con 0x5555ec5ba080 2024-09-12T02:09:48.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.895+0000 7f31337fe640 1 -- 172.21.15.16:0/3597529642 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f314409c170 con 0x5555ec5ba080 2024-09-12T02:09:48.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.895+0000 7f31337fe640 1 -- 172.21.15.16:0/3597529642 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f313c01da10 con 0x5555ec5ba080 2024-09-12T02:09:48.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.895+0000 7f315c805740 10 client.17895 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:09:48.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.895+0000 7f315c805740 10 client.17895 resolve_mds: validated gid 24482 aka daemon mds.b 2024-09-12T02:09:48.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.895+0000 7f315c805740 20 client.17895 populate_metadata read hostname 'smithi016' 2024-09-12T02:09:48.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.895+0000 7f315c805740 1 --2- 172.21.15.16:0/3597529642 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5555ec640030 0x5555ec642450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:48.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.895+0000 7f315c805740 4 client.17895 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 139.4, "await_for": 45.0}] 2024-09-12T02:09:48.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.895+0000 7f315c805740 1 -- 172.21.15.16:0/3597529642 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 139.4, "await_for": 45.0}) -- 0x5555ec4a2ec0 con 0x5555ec640030 2024-09-12T02:09:48.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.895+0000 7f3153fff640 1 --2- 172.21.15.16:0/3597529642 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5555ec640030 0x5555ec642450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:48.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.896+0000 7f3153fff640 1 --2- 172.21.15.16:0/3597529642 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5555ec640030 0x5555ec642450 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20968ebe769121a5 server_cookie=f76ab67c203e03c4 in_seq=0 out_seq=0 2024-09-12T02:09:48.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.896+0000 7f31337fe640 10 client.17895 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:09:48.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.907+0000 7f31337fe640 1 -- 172.21.15.16:0/3597529642 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5555ec4a2ec0 con 0x5555ec640030 2024-09-12T02:09:48.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.907+0000 7f31337fe640 10 client.17895 handle_command_reply: tid=0 2024-09-12T02:09:48.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.907+0000 7f315c805740 1 client.17895 shutdown 2024-09-12T02:09:48.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.907+0000 7f315c805740 1 -- 172.21.15.16:0/3597529642 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5555ec5ba080 msgr2=0x5555ec5c8ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:48.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.907+0000 7f315c805740 1 --2- 172.21.15.16:0/3597529642 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5555ec5ba080 0x5555ec5c8ce0 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x5555ec5b7070 tx=0x7f313c013830 comp rx=0 tx=0).stop 2024-09-12T02:09:48.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.907+0000 7f315c805740 1 -- 172.21.15.16:0/3597529642 shutdown_connections 2024-09-12T02:09:48.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.907+0000 7f315c805740 1 --2- 172.21.15.16:0/3597529642 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5555ec640030 0x5555ec642450 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:48.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.907+0000 7f315c805740 1 --2- 172.21.15.16:0/3597529642 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5555ec5c4940 0x5555ec5c9220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:48.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.907+0000 7f315c805740 1 --2- 172.21.15.16:0/3597529642 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5555ec5ba080 0x5555ec5c8ce0 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:48.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.907+0000 7f315c805740 1 --2- 172.21.15.16:0/3597529642 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5555ec5b9760 0x5555ec5c87a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:48.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.907+0000 7f315c805740 1 -- 172.21.15.16:0/3597529642 >> 172.21.15.16:0/3597529642 conn(0x5555ec5b5cf0 msgr2=0x5555ec5b4e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:48.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.907+0000 7f31337fe640 0 client.17895 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:09:48.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.907+0000 7f315c805740 1 -- 172.21.15.16:0/3597529642 shutdown_connections 2024-09-12T02:09:48.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.907+0000 7f315c805740 1 -- 172.21.15.16:0/3597529642 wait complete. 2024-09-12T02:09:48.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.907+0000 7f315c805740 20 client.17895 trim_cache size 0 max 16384 2024-09-12T02:09:48.909 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T02:09:48.909 INFO:teuthology.orchestra.run.smithi016.stdout: "epoch": 25, 2024-09-12T02:09:48.909 INFO:teuthology.orchestra.run.smithi016.stdout: "leader": 24482, 2024-09-12T02:09:48.909 INFO:teuthology.orchestra.run.smithi016.stdout: "set_version": 34, 2024-09-12T02:09:48.909 INFO:teuthology.orchestra.run.smithi016.stdout: "sets": { 2024-09-12T02:09:48.910 INFO:teuthology.orchestra.run.smithi016.stdout: "f9fc3d8a": { 2024-09-12T02:09:48.910 INFO:teuthology.orchestra.run.smithi016.stdout: "version": 34, 2024-09-12T02:09:48.910 INFO:teuthology.orchestra.run.smithi016.stdout: "age_ref": 0.0, 2024-09-12T02:09:48.910 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-12T02:09:48.910 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "QUIESCED", 2024-09-12T02:09:48.910 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-12T02:09:48.910 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:09:48.910 INFO:teuthology.orchestra.run.smithi016.stdout: "timeout": 90.0, 2024-09-12T02:09:48.910 INFO:teuthology.orchestra.run.smithi016.stdout: "expiration": 139.4, 2024-09-12T02:09:48.910 INFO:teuthology.orchestra.run.smithi016.stdout: "members": { 2024-09-12T02:09:48.910 INFO:teuthology.orchestra.run.smithi016.stdout: "file:/": { 2024-09-12T02:09:48.910 INFO:teuthology.orchestra.run.smithi016.stdout: "excluded": false, 2024-09-12T02:09:48.910 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-12T02:09:48.910 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "QUIESCED", 2024-09-12T02:09:48.911 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-12T02:09:48.911 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:09:48.911 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:09:48.911 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:09:48.911 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:09:48.911 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:09:48.911 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T02:09:48.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.908+0000 7f315a42c640 1 -- 172.21.15.16:0/3978377244 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f31400a2c00 msgr2=0x7f31400a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:48.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.908+0000 7f315a42c640 1 --2- 172.21.15.16:0/3978377244 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f31400a2c00 0x7f31400a50c0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f314c006010 tx=0x7f314c03e040 comp rx=0 tx=0).stop 2024-09-12T02:09:48.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.908+0000 7f315a42c640 1 -- 172.21.15.16:0/3978377244 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3154101de0 msgr2=0x7f315419a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:48.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.908+0000 7f315a42c640 1 --2- 172.21.15.16:0/3978377244 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3154101de0 0x7f315419a5e0 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f314400c920 tx=0x7f314400cde0 comp rx=0 tx=0).stop 2024-09-12T02:09:48.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.908+0000 7f315a42c640 1 -- 172.21.15.16:0/3978377244 shutdown_connections 2024-09-12T02:09:48.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.908+0000 7f315a42c640 1 --2- 172.21.15.16:0/3978377244 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f31400a2c00 0x7f31400a50c0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:48.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.908+0000 7f315a42c640 1 --2- 172.21.15.16:0/3978377244 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3154106f10 0x7f315419eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:48.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.908+0000 7f315a42c640 1 --2- 172.21.15.16:0/3978377244 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3154102790 0x7f315419ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:48.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.908+0000 7f315a42c640 1 --2- 172.21.15.16:0/3978377244 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3154101de0 0x7f315419a5e0 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:48.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.908+0000 7f315a42c640 1 -- 172.21.15.16:0/3978377244 >> 172.21.15.16:0/3978377244 conn(0x7f31540fd8c0 msgr2=0x7f31540ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:48.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.908+0000 7f315a42c640 1 -- 172.21.15.16:0/3978377244 shutdown_connections 2024-09-12T02:09:48.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:09:48.908+0000 7f315a42c640 1 -- 172.21.15.16:0/3978377244 wait complete. 2024-09-12T02:09:49.675 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'f9fc3d8a', quiesce took 3.2 seconds. Will release after: 16.2 2024-09-12T02:09:49.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:09:49 smithi184 ceph-mon[35593]: pgmap v572: 97 pgs: 97 active+clean; 4.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 2.2 KiB/s wr, 9 op/s 2024-09-12T02:09:49.918 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:09:49 smithi033 ceph-mon[35332]: pgmap v572: 97 pgs: 97 active+clean; 4.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 2.2 KiB/s wr, 9 op/s 2024-09-12T02:09:49.933 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:09:49 smithi016 ceph-mon[30947]: pgmap v572: 97 pgs: 97 active+clean; 4.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 2.2 KiB/s wr, 9 op/s 2024-09-12T02:09:50.805 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.803+0000 7fb8842e6640 1 -- 172.21.15.16:0/1210244546 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb87c104f10 msgr2=0x7fb87c107300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:50.805 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.803+0000 7fb8842e6640 1 --2- 172.21.15.16:0/1210244546 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb87c104f10 0x7fb87c107300 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7fb86c009900 tx=0x7fb86c02a410 comp rx=0 tx=0).stop 2024-09-12T02:09:50.806 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.803+0000 7fb8842e6640 1 -- 172.21.15.16:0/1210244546 shutdown_connections 2024-09-12T02:09:50.806 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.803+0000 7fb8842e6640 1 --2- 172.21.15.16:0/1210244546 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb87c107840 0x7fb87c109cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:50.806 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.803+0000 7fb8842e6640 1 --2- 172.21.15.16:0/1210244546 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb87c104f10 0x7fb87c107300 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:50.806 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.803+0000 7fb8842e6640 1 --2- 172.21.15.16:0/1210244546 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb87c1025e0 0x7fb87c1049d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:50.806 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.803+0000 7fb8842e6640 1 -- 172.21.15.16:0/1210244546 >> 172.21.15.16:0/1210244546 conn(0x7fb87c0fc1e0 msgr2=0x7fb87c0fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:50.806 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.803+0000 7fb8842e6640 1 -- 172.21.15.16:0/1210244546 shutdown_connections 2024-09-12T02:09:50.806 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.803+0000 7fb8842e6640 1 -- 172.21.15.16:0/1210244546 wait complete. 2024-09-12T02:09:50.806 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.804+0000 7fb8842e6640 1 Processor -- start 2024-09-12T02:09:50.806 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.804+0000 7fb8842e6640 1 -- start start 2024-09-12T02:09:50.806 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.804+0000 7fb8842e6640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb87c1025e0 0x7fb87c197e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:50.806 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.804+0000 7fb8842e6640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb87c104f10 0x7fb87c1983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:50.806 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.804+0000 7fb8842e6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb87c107840 0x7fb87c19f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:50.806 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.804+0000 7fb8842e6640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb87c10d810 con 0x7fb87c107840 2024-09-12T02:09:50.806 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.804+0000 7fb8842e6640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb87c10d690 con 0x7fb87c1025e0 2024-09-12T02:09:50.807 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.804+0000 7fb8842e6640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb87c10d990 con 0x7fb87c104f10 2024-09-12T02:09:50.807 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.804+0000 7fb882883640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb87c107840 0x7fb87c19f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:50.807 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.804+0000 7fb882883640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb87c107840 0x7fb87c19f440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38264/0 (socket says 172.21.15.16:38264) 2024-09-12T02:09:50.807 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.804+0000 7fb882883640 1 -- 172.21.15.16:0/3872886196 learned_addr learned my addr 172.21.15.16:0/3872886196 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:09:50.807 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.804+0000 7fb881881640 1 --2- 172.21.15.16:0/3872886196 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb87c104f10 0x7fb87c1983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:50.807 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.804+0000 7fb882082640 1 --2- 172.21.15.16:0/3872886196 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb87c1025e0 0x7fb87c197e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:50.807 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.804+0000 7fb882883640 1 -- 172.21.15.16:0/3872886196 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb87c104f10 msgr2=0x7fb87c1983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:50.807 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.804+0000 7fb882883640 1 --2- 172.21.15.16:0/3872886196 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb87c104f10 0x7fb87c1983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:50.807 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.804+0000 7fb882883640 1 -- 172.21.15.16:0/3872886196 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb87c1025e0 msgr2=0x7fb87c197e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:50.807 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.804+0000 7fb882883640 1 --2- 172.21.15.16:0/3872886196 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb87c1025e0 0x7fb87c197e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:50.807 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.804+0000 7fb882883640 1 -- 172.21.15.16:0/3872886196 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb87c19fab0 con 0x7fb87c107840 2024-09-12T02:09:50.807 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.805+0000 7fb882883640 1 --2- 172.21.15.16:0/3872886196 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb87c107840 0x7fb87c19f440 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7fb87400e970 tx=0x7fb87400ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:50.807 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.805+0000 7fb8737fe640 1 -- 172.21.15.16:0/3872886196 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb87400ccc0 con 0x7fb87c107840 2024-09-12T02:09:50.807 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.805+0000 7fb8842e6640 1 -- 172.21.15.16:0/3872886196 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb87c19fda0 con 0x7fb87c107840 2024-09-12T02:09:50.808 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.805+0000 7fb8737fe640 1 -- 172.21.15.16:0/3872886196 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb874005500 con 0x7fb87c107840 2024-09-12T02:09:50.808 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.805+0000 7fb8737fe640 1 -- 172.21.15.16:0/3872886196 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb874010690 con 0x7fb87c107840 2024-09-12T02:09:50.808 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.805+0000 7fb8842e6640 1 -- 172.21.15.16:0/3872886196 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb87c1a0210 con 0x7fb87c107840 2024-09-12T02:09:50.813 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.807+0000 7fb8737fe640 1 -- 172.21.15.16:0/3872886196 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fb8740046c0 con 0x7fb87c107840 2024-09-12T02:09:50.813 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.808+0000 7fb887082740 1 Processor -- start 2024-09-12T02:09:50.814 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.808+0000 7fb887082740 1 -- start start 2024-09-12T02:09:50.814 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.808+0000 7fb887082740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c809280db0 0x55c8092811b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:50.814 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.808+0000 7fb887082740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c8092816f0 0x55c8092861b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:50.814 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.808+0000 7fb887082740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c809286780 0x55c809288c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:50.814 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.808+0000 7fb887082740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55c8091e02f0 con 0x55c8092816f0 2024-09-12T02:09:50.814 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.808+0000 7fb887082740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55c809188270 con 0x55c809280db0 2024-09-12T02:09:50.814 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.808+0000 7fb887082740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55c8091dc2b0 con 0x55c809286780 2024-09-12T02:09:50.814 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.808+0000 7fb882082640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c8092816f0 0x55c8092861b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:50.814 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.808+0000 7fb882082640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c8092816f0 0x55c8092861b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38280/0 (socket says 172.21.15.16:38280) 2024-09-12T02:09:50.814 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.808+0000 7fb882082640 1 -- 172.21.15.16:0/2021237018 learned_addr learned my addr 172.21.15.16:0/2021237018 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:09:50.814 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.809+0000 7fb881881640 1 --2- 172.21.15.16:0/2021237018 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c809280db0 0x55c8092811b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:50.814 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.809+0000 7fb881881640 1 --2- 172.21.15.16:0/2021237018 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c809286780 0x55c809288c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:50.815 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.809+0000 7fb882082640 1 -- 172.21.15.16:0/2021237018 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c809286780 msgr2=0x55c809288c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:09:50.815 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.809+0000 7fb882082640 1 --2- 172.21.15.16:0/2021237018 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c809286780 0x55c809288c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:50.815 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.809+0000 7fb882082640 1 -- 172.21.15.16:0/2021237018 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c809280db0 msgr2=0x55c8092811b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:09:50.815 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.809+0000 7fb882082640 1 --2- 172.21.15.16:0/2021237018 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c809280db0 0x55c8092811b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:50.815 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.809+0000 7fb8737fe640 1 --2- 172.21.15.16:0/3872886196 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fb8680a2c30 0x7fb8680a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:50.815 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.809+0000 7fb882082640 1 -- 172.21.15.16:0/2021237018 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c809187340 con 0x55c8092816f0 2024-09-12T02:09:50.815 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.809+0000 7fb881881640 1 --2- 172.21.15.16:0/2021237018 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c809280db0 0x55c8092811b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:09:50.815 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.809+0000 7fb882883640 1 --2- 172.21.15.16:0/3872886196 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fb8680a2c30 0x7fb8680a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:50.815 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.809+0000 7fb8737fe640 1 -- 172.21.15.16:0/3872886196 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fb8740cf390 con 0x7fb87c107840 2024-09-12T02:09:50.815 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.809+0000 7fb882082640 1 --2- 172.21.15.16:0/2021237018 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c8092816f0 0x55c8092861b0 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7fb8780103f0 tx=0x7fb8780108b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d91f291fb5196a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:50.815 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.809+0000 7fb870bfe640 1 -- 172.21.15.16:0/2021237018 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb878016070 con 0x55c8092816f0 2024-09-12T02:09:50.815 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.809+0000 7fb870bfe640 1 -- 172.21.15.16:0/2021237018 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb878004d20 con 0x55c8092816f0 2024-09-12T02:09:50.816 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.809+0000 7fb870bfe640 1 -- 172.21.15.16:0/2021237018 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb878005400 con 0x55c8092816f0 2024-09-12T02:09:50.816 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.810+0000 7fb887082740 1 -- 172.21.15.16:0/2021237018 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c8092816f0 msgr2=0x55c8092861b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:50.816 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.810+0000 7fb887082740 1 --2- 172.21.15.16:0/2021237018 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c8092816f0 0x55c8092861b0 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7fb8780103f0 tx=0x7fb8780108b0 comp rx=0 tx=0).stop 2024-09-12T02:09:50.816 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.810+0000 7fb882883640 1 --2- 172.21.15.16:0/3872886196 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fb8680a2c30 0x7fb8680a50f0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fb8740ccf30 tx=0x7fb874066910 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:50.816 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.810+0000 7fb887082740 1 -- 172.21.15.16:0/2021237018 shutdown_connections 2024-09-12T02:09:50.816 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.810+0000 7fb887082740 1 --2- 172.21.15.16:0/2021237018 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c809286780 0x55c809288c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:50.816 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.810+0000 7fb887082740 1 --2- 172.21.15.16:0/2021237018 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c8092816f0 0x55c8092861b0 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:50.816 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.810+0000 7fb887082740 1 --2- 172.21.15.16:0/2021237018 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c809280db0 0x55c8092811b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:50.816 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.810+0000 7fb887082740 1 -- 172.21.15.16:0/2021237018 >> 172.21.15.16:0/2021237018 conn(0x55c809278010 msgr2=0x55c809278410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:50.816 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.810+0000 7fb887082740 1 -- 172.21.15.16:0/2021237018 shutdown_connections 2024-09-12T02:09:50.816 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.810+0000 7fb887082740 1 -- 172.21.15.16:0/2021237018 wait complete. 2024-09-12T02:09:50.816 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.811+0000 7fb887082740 1 -- start start 2024-09-12T02:09:50.816 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.811+0000 7fb887082740 1 Processor -- start 2024-09-12T02:09:50.817 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.811+0000 7fb887082740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c809280db0 0x55c80928b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:50.817 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.811+0000 7fb887082740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c8092816f0 0x55c80928b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:50.817 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.811+0000 7fb887082740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c809286780 0x55c809292540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:50.817 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.811+0000 7fb887082740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55c809289750 con 0x55c809280db0 2024-09-12T02:09:50.817 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.811+0000 7fb887082740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55c809169290 con 0x55c809286780 2024-09-12T02:09:50.817 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.811+0000 7fb887082740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55c8092898d0 con 0x55c8092816f0 2024-09-12T02:09:50.817 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.811+0000 7fb887082740 10 client.0 _ll_register_callbacks cb 0x55c80919a140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:09:50.817 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.811+0000 7fb881881640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c809280db0 0x55c80928b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:50.817 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.811+0000 7fb881881640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c809280db0 0x55c80928b030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38292/0 (socket says 172.21.15.16:38292) 2024-09-12T02:09:50.817 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.811+0000 7fb881881640 1 -- 172.21.15.16:0/523046261 learned_addr learned my addr 172.21.15.16:0/523046261 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:09:50.817 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.811+0000 7fb882082640 1 --2- 172.21.15.16:0/523046261 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c8092816f0 0x55c80928b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:50.817 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.811+0000 7fb881881640 1 -- 172.21.15.16:0/523046261 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c8092816f0 msgr2=0x55c80928b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:09:50.818 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.811+0000 7fb881881640 1 --2- 172.21.15.16:0/523046261 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c8092816f0 0x55c80928b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:50.818 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.811+0000 7fb881881640 1 -- 172.21.15.16:0/523046261 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c809286780 msgr2=0x55c809292540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:09:50.818 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.811+0000 7fb881881640 1 --2- 172.21.15.16:0/523046261 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c809286780 0x55c809292540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:50.818 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.811+0000 7fb881881640 1 -- 172.21.15.16:0/523046261 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c808ef3210 con 0x55c809280db0 2024-09-12T02:09:50.818 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.811+0000 7fb882082640 1 --2- 172.21.15.16:0/523046261 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c8092816f0 0x55c80928b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:09:50.818 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.812+0000 7fb881881640 1 --2- 172.21.15.16:0/523046261 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c809280db0 0x55c80928b030 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7fb87c06ae60 tx=0x7fb86c005e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe905f1d766dfed server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:50.818 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.812+0000 7fb865ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:09:50.818 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.812+0000 7fb865ffb640 1 -- 172.21.15.16:0/523046261 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb86c046020 con 0x55c809280db0 2024-09-12T02:09:50.818 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.812+0000 7fb865ffb640 1 -- 172.21.15.16:0/523046261 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fb86c005240 con 0x55c809280db0 2024-09-12T02:09:50.818 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.812+0000 7fb887082740 1 -- 172.21.15.16:0/523046261 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb878004d20 con 0x55c809280db0 2024-09-12T02:09:50.818 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.812+0000 7fb865ffb640 1 -- 172.21.15.16:0/523046261 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fb86c035070 con 0x55c809280db0 2024-09-12T02:09:50.818 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.812+0000 7fb887082740 10 client.17907 fetch_fsmap learned FSMap version 25 2024-09-12T02:09:50.819 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.812+0000 7fb887082740 1 -- 172.21.15.16:0/523046261 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c808fbf620 con 0x55c809280db0 2024-09-12T02:09:50.819 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.812+0000 7fb865ffb640 1 -- 172.21.15.16:0/523046261 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb878004d20 con 0x55c809280db0 2024-09-12T02:09:50.819 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.812+0000 7fb865ffb640 1 -- 172.21.15.16:0/523046261 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fb86c03d570 con 0x55c809280db0 2024-09-12T02:09:50.819 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.812+0000 7fb887082740 10 client.17907 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:09:50.819 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.812+0000 7fb887082740 10 client.17907 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:09:50.819 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.812+0000 7fb887082740 20 client.17907 populate_metadata read hostname 'smithi016' 2024-09-12T02:09:50.819 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.812+0000 7fb887082740 1 --2- 172.21.15.16:0/523046261 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55c809296c30 0x55c809299040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:50.819 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.812+0000 7fb887082740 4 client.17907 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:09:50.819 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.812+0000 7fb887082740 1 -- 172.21.15.16:0/523046261 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c8091eda20 con 0x55c809296c30 2024-09-12T02:09:50.819 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.813+0000 7fb882082640 1 --2- 172.21.15.16:0/523046261 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55c809296c30 0x55c809299040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:50.819 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.813+0000 7fb882082640 1 --2- 172.21.15.16:0/523046261 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55c809296c30 0x55c809299040 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=354c325565c9072f server_cookie=ad99474e9dce8773 in_seq=0 out_seq=0 2024-09-12T02:09:50.819 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.813+0000 7fb865ffb640 10 client.17907 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:09:50.820 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.814+0000 7fb865ffb640 1 -- 172.21.15.16:0/523046261 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c8091eda20 con 0x55c809296c30 2024-09-12T02:09:50.820 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.814+0000 7fb865ffb640 10 client.17907 handle_command_reply: tid=0 2024-09-12T02:09:50.821 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.814+0000 7fb887082740 1 client.17907 shutdown 2024-09-12T02:09:50.821 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.814+0000 7fb887082740 1 -- 172.21.15.16:0/523046261 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c809280db0 msgr2=0x55c80928b030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:50.821 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.814+0000 7fb887082740 1 --2- 172.21.15.16:0/523046261 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c809280db0 0x55c80928b030 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7fb87c06ae60 tx=0x7fb86c005e70 comp rx=0 tx=0).stop 2024-09-12T02:09:50.821 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.814+0000 7fb887082740 1 -- 172.21.15.16:0/523046261 shutdown_connections 2024-09-12T02:09:50.821 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.814+0000 7fb887082740 1 --2- 172.21.15.16:0/523046261 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55c809296c30 0x55c809299040 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:50.821 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.814+0000 7fb887082740 1 --2- 172.21.15.16:0/523046261 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c809286780 0x55c809292540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:50.821 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.814+0000 7fb887082740 1 --2- 172.21.15.16:0/523046261 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c8092816f0 0x55c80928b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:50.821 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.814+0000 7fb865ffb640 0 client.17907 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:09:50.821 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.814+0000 7fb887082740 1 --2- 172.21.15.16:0/523046261 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c809280db0 0x55c80928b030 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:50.821 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.814+0000 7fb887082740 1 -- 172.21.15.16:0/523046261 >> 172.21.15.16:0/523046261 conn(0x55c809278010 msgr2=0x55c809279820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:50.821 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.814+0000 7fb887082740 1 -- 172.21.15.16:0/523046261 shutdown_connections 2024-09-12T02:09:50.821 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.814+0000 7fb887082740 1 -- 172.21.15.16:0/523046261 wait complete. 2024-09-12T02:09:50.822 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.814+0000 7fb887082740 20 client.17907 trim_cache size 0 max 16384 2024-09-12T02:09:50.833 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.829+0000 7fb887082740 1 Processor -- start 2024-09-12T02:09:50.834 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.829+0000 7fb887082740 1 -- start start 2024-09-12T02:09:50.834 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.829+0000 7fb887082740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c809278010 0x55c80928b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:50.834 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.829+0000 7fb887082740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c809280db0 0x55c809292500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:50.834 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.829+0000 7fb887082740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c8092816f0 0x55c8092770b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:50.834 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.829+0000 7fb887082740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb87c1a9550 con 0x55c809280db0 2024-09-12T02:09:50.834 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.829+0000 7fb887082740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb87c1a6e20 con 0x55c809278010 2024-09-12T02:09:50.834 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.829+0000 7fb887082740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb87c1a4ce0 con 0x55c8092816f0 2024-09-12T02:09:50.834 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.830+0000 7fb882082640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c809280db0 0x55c809292500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:50.835 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.830+0000 7fb882082640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c809280db0 0x55c809292500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38308/0 (socket says 172.21.15.16:38308) 2024-09-12T02:09:50.835 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.830+0000 7fb882082640 1 -- 172.21.15.16:0/1643607846 learned_addr learned my addr 172.21.15.16:0/1643607846 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:09:50.835 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.830+0000 7fb881881640 1 --2- 172.21.15.16:0/1643607846 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c809278010 0x55c80928b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:50.835 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.830+0000 7fb881881640 1 --2- 172.21.15.16:0/1643607846 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c8092816f0 0x55c8092770b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:50.835 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.830+0000 7fb882082640 1 -- 172.21.15.16:0/1643607846 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c8092816f0 msgr2=0x55c8092770b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:09:50.835 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.830+0000 7fb882082640 1 --2- 172.21.15.16:0/1643607846 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c8092816f0 0x55c8092770b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:50.835 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.830+0000 7fb882082640 1 -- 172.21.15.16:0/1643607846 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c809278010 msgr2=0x55c80928b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:09:50.835 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.830+0000 7fb882082640 1 --2- 172.21.15.16:0/1643607846 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c809278010 0x55c80928b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:50.835 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.830+0000 7fb882082640 1 -- 172.21.15.16:0/1643607846 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c809276650 con 0x55c809280db0 2024-09-12T02:09:50.835 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.830+0000 7fb881881640 1 --2- 172.21.15.16:0/1643607846 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c809278010 0x55c80928b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:09:50.835 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.830+0000 7fb882082640 1 --2- 172.21.15.16:0/1643607846 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c809280db0 0x55c809292500 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7fb8780103f0 tx=0x7fb878009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ee0ca02f45d216 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:50.835 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.830+0000 7fb867fff640 1 -- 172.21.15.16:0/1643607846 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb878016040 con 0x55c809280db0 2024-09-12T02:09:50.836 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.830+0000 7fb867fff640 1 -- 172.21.15.16:0/1643607846 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8780129f0 con 0x55c809280db0 2024-09-12T02:09:50.836 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.830+0000 7fb867fff640 1 -- 172.21.15.16:0/1643607846 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb878005770 con 0x55c809280db0 2024-09-12T02:09:50.836 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.831+0000 7fb887082740 1 -- 172.21.15.16:0/1643607846 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c809280db0 msgr2=0x55c809292500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:50.836 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.831+0000 7fb887082740 1 --2- 172.21.15.16:0/1643607846 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c809280db0 0x55c809292500 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7fb8780103f0 tx=0x7fb878009c30 comp rx=0 tx=0).stop 2024-09-12T02:09:50.836 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.831+0000 7fb887082740 1 -- 172.21.15.16:0/1643607846 shutdown_connections 2024-09-12T02:09:50.836 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.831+0000 7fb887082740 1 --2- 172.21.15.16:0/1643607846 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c8092816f0 0x55c8092770b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:50.836 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.831+0000 7fb887082740 1 --2- 172.21.15.16:0/1643607846 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c809280db0 0x55c809292500 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:50.836 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.831+0000 7fb887082740 1 --2- 172.21.15.16:0/1643607846 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c809278010 0x55c80928b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:50.836 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.831+0000 7fb887082740 1 -- 172.21.15.16:0/1643607846 >> 172.21.15.16:0/1643607846 conn(0x55c8092862a0 msgr2=0x55c8092769a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:50.836 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.831+0000 7fb887082740 1 -- 172.21.15.16:0/1643607846 shutdown_connections 2024-09-12T02:09:50.836 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.831+0000 7fb887082740 1 -- 172.21.15.16:0/1643607846 wait complete. 2024-09-12T02:09:50.836 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.831+0000 7fb887082740 1 -- start start 2024-09-12T02:09:50.837 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.831+0000 7fb887082740 1 Processor -- start 2024-09-12T02:09:50.837 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.831+0000 7fb887082740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c809278010 0x55c80928fe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:50.837 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.831+0000 7fb887082740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c809280db0 0x55c8092903d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:50.837 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.831+0000 7fb887082740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c8092816f0 0x55c8092913e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:50.837 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.831+0000 7fb887082740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb87c1a52e0 con 0x55c809278010 2024-09-12T02:09:50.837 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.831+0000 7fb887082740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb87c1a6750 con 0x55c809280db0 2024-09-12T02:09:50.837 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.831+0000 7fb887082740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb87c1a59b0 con 0x55c8092816f0 2024-09-12T02:09:50.837 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.832+0000 7fb887082740 10 client.0 _ll_register_callbacks cb 0x7fb86c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:09:50.837 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.832+0000 7fb881881640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c809278010 0x55c80928fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:50.837 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.832+0000 7fb881881640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c809278010 0x55c80928fe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38314/0 (socket says 172.21.15.16:38314) 2024-09-12T02:09:50.837 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.832+0000 7fb881881640 1 -- 172.21.15.16:0/3757832891 learned_addr learned my addr 172.21.15.16:0/3757832891 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:09:50.837 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.832+0000 7fb882082640 1 --2- 172.21.15.16:0/3757832891 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c809280db0 0x55c8092903d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:50.837 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.832+0000 7fb881881640 1 --2- 172.21.15.16:0/3757832891 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c8092816f0 0x55c8092913e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:50.838 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.832+0000 7fb881881640 1 -- 172.21.15.16:0/3757832891 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c8092816f0 msgr2=0x55c8092913e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:09:50.838 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.832+0000 7fb881881640 1 --2- 172.21.15.16:0/3757832891 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c8092816f0 0x55c8092913e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:50.838 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.832+0000 7fb881881640 1 -- 172.21.15.16:0/3757832891 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c809280db0 msgr2=0x55c8092903d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:09:50.838 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.832+0000 7fb881881640 1 --2- 172.21.15.16:0/3757832891 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c809280db0 0x55c8092903d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:50.838 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.832+0000 7fb881881640 1 -- 172.21.15.16:0/3757832891 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c8092919f0 con 0x55c809278010 2024-09-12T02:09:50.838 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.832+0000 7fb881881640 1 --2- 172.21.15.16:0/3757832891 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c809278010 0x55c80928fe90 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7fb86c005ab0 tx=0x7fb86c031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44e79a8def0aa453 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:50.838 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.832+0000 7fb865ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:09:50.838 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.832+0000 7fb865ffb640 1 -- 172.21.15.16:0/3757832891 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb86c046070 con 0x55c809278010 2024-09-12T02:09:50.838 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.832+0000 7fb865ffb640 1 -- 172.21.15.16:0/3757832891 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fb86c03dd60 con 0x55c809278010 2024-09-12T02:09:50.838 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.832+0000 7fb887082740 1 -- 172.21.15.16:0/3757832891 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8780129f0 con 0x55c809278010 2024-09-12T02:09:50.838 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.832+0000 7fb865ffb640 1 -- 172.21.15.16:0/3757832891 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fb86c005020 con 0x55c809278010 2024-09-12T02:09:50.838 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.833+0000 7fb887082740 10 client.17913 fetch_fsmap learned FSMap version 25 2024-09-12T02:09:50.838 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.833+0000 7fb887082740 1 -- 172.21.15.16:0/3757832891 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c809301b40 con 0x55c809278010 2024-09-12T02:09:50.839 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.833+0000 7fb865ffb640 1 -- 172.21.15.16:0/3757832891 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8780129f0 con 0x55c809278010 2024-09-12T02:09:50.839 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.833+0000 7fb865ffb640 1 -- 172.21.15.16:0/3757832891 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fb86c057040 con 0x55c809278010 2024-09-12T02:09:50.839 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.833+0000 7fb887082740 10 client.17913 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:09:50.839 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.833+0000 7fb887082740 10 client.17913 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:09:50.839 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.833+0000 7fb887082740 20 client.17913 populate_metadata read hostname 'smithi016' 2024-09-12T02:09:50.839 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.833+0000 7fb887082740 1 --2- 172.21.15.16:0/3757832891 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55c809286780 0x55c809304500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:50.839 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.833+0000 7fb887082740 4 client.17913 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:09:50.839 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.833+0000 7fb887082740 1 -- 172.21.15.16:0/3757832891 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c8091e7d70 con 0x55c809286780 2024-09-12T02:09:50.839 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.833+0000 7fb882082640 1 --2- 172.21.15.16:0/3757832891 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55c809286780 0x55c809304500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:50.839 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.834+0000 7fb882082640 1 --2- 172.21.15.16:0/3757832891 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55c809286780 0x55c809304500 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14aa25480a42f5a9 server_cookie=11fe73fce8e8523e in_seq=0 out_seq=0 2024-09-12T02:09:50.839 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.834+0000 7fb865ffb640 10 client.17913 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:09:50.847 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.847+0000 7fb865ffb640 1 -- 172.21.15.16:0/3757832891 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c8091e7d70 con 0x55c809286780 2024-09-12T02:09:50.848 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.847+0000 7fb865ffb640 10 client.17913 handle_command_reply: tid=0 2024-09-12T02:09:50.848 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.847+0000 7fb887082740 1 client.17913 shutdown 2024-09-12T02:09:50.848 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.847+0000 7fb887082740 1 -- 172.21.15.16:0/3757832891 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c809278010 msgr2=0x55c80928fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:50.848 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.847+0000 7fb887082740 1 --2- 172.21.15.16:0/3757832891 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c809278010 0x55c80928fe90 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7fb86c005ab0 tx=0x7fb86c031400 comp rx=0 tx=0).stop 2024-09-12T02:09:50.848 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.847+0000 7fb887082740 1 -- 172.21.15.16:0/3757832891 shutdown_connections 2024-09-12T02:09:50.848 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.847+0000 7fb887082740 1 --2- 172.21.15.16:0/3757832891 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55c809286780 0x55c809304500 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:50.848 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.847+0000 7fb887082740 1 --2- 172.21.15.16:0/3757832891 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c8092816f0 0x55c8092913e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:50.848 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.847+0000 7fb887082740 1 --2- 172.21.15.16:0/3757832891 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c809280db0 0x55c8092903d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:50.848 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.847+0000 7fb865ffb640 0 client.17913 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:09:50.848 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.847+0000 7fb887082740 1 --2- 172.21.15.16:0/3757832891 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c809278010 0x55c80928fe90 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:50.848 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.847+0000 7fb887082740 1 -- 172.21.15.16:0/3757832891 >> 172.21.15.16:0/3757832891 conn(0x55c8092862a0 msgr2=0x55c8092769a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:50.848 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.847+0000 7fb887082740 1 -- 172.21.15.16:0/3757832891 shutdown_connections 2024-09-12T02:09:50.848 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.847+0000 7fb887082740 1 -- 172.21.15.16:0/3757832891 wait complete. 2024-09-12T02:09:50.849 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.847+0000 7fb887082740 20 client.17913 trim_cache size 0 max 16384 2024-09-12T02:09:50.849 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-09-12T02:09:50.849 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-09-12T02:09:50.849 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-09-12T02:09:50.849 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-09-12T02:09:50.850 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.848+0000 7fb8842e6640 1 -- 172.21.15.16:0/3872886196 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fb8680a2c30 msgr2=0x7fb8680a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:50.850 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.848+0000 7fb8842e6640 1 --2- 172.21.15.16:0/3872886196 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fb8680a2c30 0x7fb8680a50f0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fb8740ccf30 tx=0x7fb874066910 comp rx=0 tx=0).stop 2024-09-12T02:09:50.850 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.848+0000 7fb8842e6640 1 -- 172.21.15.16:0/3872886196 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb87c107840 msgr2=0x7fb87c19f440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:50.850 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.848+0000 7fb8842e6640 1 --2- 172.21.15.16:0/3872886196 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb87c107840 0x7fb87c19f440 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7fb87400e970 tx=0x7fb87400ee30 comp rx=0 tx=0).stop 2024-09-12T02:09:50.850 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.848+0000 7fb8842e6640 1 -- 172.21.15.16:0/3872886196 shutdown_connections 2024-09-12T02:09:50.850 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.848+0000 7fb8842e6640 1 --2- 172.21.15.16:0/3872886196 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fb8680a2c30 0x7fb8680a50f0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:50.850 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.848+0000 7fb8842e6640 1 --2- 172.21.15.16:0/3872886196 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb87c107840 0x7fb87c19f440 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:50.850 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.848+0000 7fb8842e6640 1 --2- 172.21.15.16:0/3872886196 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb87c104f10 0x7fb87c1983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:50.850 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.848+0000 7fb8842e6640 1 --2- 172.21.15.16:0/3872886196 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb87c1025e0 0x7fb87c197e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:50.851 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.848+0000 7fb8842e6640 1 -- 172.21.15.16:0/3872886196 >> 172.21.15.16:0/3872886196 conn(0x7fb87c0fc1e0 msgr2=0x7fb87c103040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:50.851 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.848+0000 7fb8842e6640 1 -- 172.21.15.16:0/3872886196 shutdown_connections 2024-09-12T02:09:50.851 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:09:50.848+0000 7fb8842e6640 1 -- 172.21.15.16:0/3872886196 wait complete. 2024-09-12T02:09:51.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:09:51 smithi184 ceph-mon[35593]: pgmap v573: 97 pgs: 97 active+clean; 4.2 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 6.7 KiB/s wr, 12 op/s 2024-09-12T02:09:51.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:09:51 smithi033 ceph-mon[35332]: pgmap v573: 97 pgs: 97 active+clean; 4.2 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 6.7 KiB/s wr, 12 op/s 2024-09-12T02:09:51.933 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:09:51 smithi016 ceph-mon[30947]: pgmap v573: 97 pgs: 97 active+clean; 4.2 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 6.7 KiB/s wr, 12 op/s 2024-09-12T02:09:53.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:09:53 smithi184 ceph-mon[35593]: pgmap v574: 97 pgs: 97 active+clean; 4.2 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 5.3 KiB/s wr, 9 op/s 2024-09-12T02:09:53.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:09:53 smithi033 ceph-mon[35332]: pgmap v574: 97 pgs: 97 active+clean; 4.2 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 5.3 KiB/s wr, 9 op/s 2024-09-12T02:09:53.933 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:09:53 smithi016 ceph-mon[30947]: pgmap v574: 97 pgs: 97 active+clean; 4.2 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 5.3 KiB/s wr, 9 op/s 2024-09-12T02:09:55.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:09:55 smithi184 ceph-mon[35593]: pgmap v575: 97 pgs: 97 active+clean; 4.2 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 6.7 KiB/s wr, 10 op/s 2024-09-12T02:09:55.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:09:55 smithi033 ceph-mon[35332]: pgmap v575: 97 pgs: 97 active+clean; 4.2 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 6.7 KiB/s wr, 10 op/s 2024-09-12T02:09:55.933 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:09:55 smithi016 ceph-mon[30947]: pgmap v575: 97 pgs: 97 active+clean; 4.2 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 6.7 KiB/s wr, 10 op/s 2024-09-12T02:09:57.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:09:57 smithi184 ceph-mon[35593]: pgmap v576: 97 pgs: 97 active+clean; 4.2 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 6.8 KiB/s wr, 10 op/s 2024-09-12T02:09:57.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:09:57 smithi033 ceph-mon[35332]: pgmap v576: 97 pgs: 97 active+clean; 4.2 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 6.8 KiB/s wr, 10 op/s 2024-09-12T02:09:57.933 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:09:57 smithi016 ceph-mon[30947]: pgmap v576: 97 pgs: 97 active+clean; 4.2 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 6.8 KiB/s wr, 10 op/s 2024-09-12T02:09:59.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:09:59 smithi184 ceph-mon[35593]: pgmap v577: 97 pgs: 97 active+clean; 4.2 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 6.8 KiB/s wr, 9 op/s 2024-09-12T02:09:59.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:09:59 smithi033 ceph-mon[35332]: pgmap v577: 97 pgs: 97 active+clean; 4.2 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 6.8 KiB/s wr, 9 op/s 2024-09-12T02:09:59.933 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:09:59 smithi016 ceph-mon[30947]: pgmap v577: 97 pgs: 97 active+clean; 4.2 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 6.8 KiB/s wr, 9 op/s 2024-09-12T02:10:00.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:10:00 smithi184 ceph-mon[35593]: overall HEALTH_OK 2024-09-12T02:10:00.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:10:00 smithi033 ceph-mon[35332]: overall HEALTH_OK 2024-09-12T02:10:00.933 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:10:00 smithi016 ceph-mon[30947]: overall HEALTH_OK 2024-09-12T02:10:01.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:10:01 smithi184 ceph-mon[35593]: pgmap v578: 97 pgs: 97 active+clean; 4.2 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 6.8 KiB/s wr, 11 op/s 2024-09-12T02:10:01.920 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:10:01 smithi033 ceph-mon[35332]: pgmap v578: 97 pgs: 97 active+clean; 4.2 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 6.8 KiB/s wr, 11 op/s 2024-09-12T02:10:01.934 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:10:01 smithi016 ceph-mon[30947]: pgmap v578: 97 pgs: 97 active+clean; 4.2 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 6.8 KiB/s wr, 11 op/s 2024-09-12T02:10:03.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:10:03 smithi184 ceph-mon[35593]: pgmap v579: 97 pgs: 97 active+clean; 4.2 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.1 KiB/s wr, 8 op/s 2024-09-12T02:10:03.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:10:03 smithi033 ceph-mon[35332]: pgmap v579: 97 pgs: 97 active+clean; 4.2 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.1 KiB/s wr, 8 op/s 2024-09-12T02:10:03.933 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:10:03 smithi016 ceph-mon[30947]: pgmap v579: 97 pgs: 97 active+clean; 4.2 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.1 KiB/s wr, 8 op/s 2024-09-12T02:10:03.959 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.958+0000 7f6e57db9640 1 -- 172.21.15.16:0/2591882347 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e50104f10 msgr2=0x7f6e50107300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:03.959 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.958+0000 7f6e57db9640 1 --2- 172.21.15.16:0/2591882347 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e50104f10 0x7f6e50107300 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f6e40009900 tx=0x7f6e4002a410 comp rx=0 tx=0).stop 2024-09-12T02:10:03.959 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.958+0000 7f6e57db9640 1 -- 172.21.15.16:0/2591882347 shutdown_connections 2024-09-12T02:10:03.959 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.958+0000 7f6e57db9640 1 --2- 172.21.15.16:0/2591882347 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6e50107840 0x7f6e50109cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:03.959 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.958+0000 7f6e57db9640 1 --2- 172.21.15.16:0/2591882347 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e50104f10 0x7f6e50107300 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:03.959 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.958+0000 7f6e57db9640 1 --2- 172.21.15.16:0/2591882347 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6e501025e0 0x7f6e501049d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:03.959 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.958+0000 7f6e57db9640 1 -- 172.21.15.16:0/2591882347 >> 172.21.15.16:0/2591882347 conn(0x7f6e500fc1e0 msgr2=0x7f6e500fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:10:03.959 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.958+0000 7f6e57db9640 1 -- 172.21.15.16:0/2591882347 shutdown_connections 2024-09-12T02:10:03.959 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.958+0000 7f6e57db9640 1 -- 172.21.15.16:0/2591882347 wait complete. 2024-09-12T02:10:03.959 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.959+0000 7f6e57db9640 1 Processor -- start 2024-09-12T02:10:03.959 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.959+0000 7f6e57db9640 1 -- start start 2024-09-12T02:10:03.961 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.959+0000 7f6e57db9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6e501025e0 0x7f6e50101f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:03.961 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.959+0000 7f6e57db9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e50104f10 0x7f6e50100590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:03.962 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.959+0000 7f6e57db9640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6e50107840 0x7f6e50100ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:03.962 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.959+0000 7f6e57db9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6e5010d810 con 0x7f6e50104f10 2024-09-12T02:10:03.962 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.959+0000 7f6e57db9640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6e5010d690 con 0x7f6e501025e0 2024-09-12T02:10:03.962 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.959+0000 7f6e57db9640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f6e5010d990 con 0x7f6e50107840 2024-09-12T02:10:03.962 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.959+0000 7f6e55354640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e50104f10 0x7f6e50100590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:03.962 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.959+0000 7f6e55354640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e50104f10 0x7f6e50100590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57236/0 (socket says 172.21.15.16:57236) 2024-09-12T02:10:03.962 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.959+0000 7f6e55354640 1 -- 172.21.15.16:0/3451686239 learned_addr learned my addr 172.21.15.16:0/3451686239 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:10:03.962 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.960+0000 7f6e56356640 1 --2- 172.21.15.16:0/3451686239 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6e50107840 0x7f6e50100ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:03.962 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.960+0000 7f6e55b55640 1 --2- 172.21.15.16:0/3451686239 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6e501025e0 0x7f6e50101f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:03.962 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.960+0000 7f6e55354640 1 -- 172.21.15.16:0/3451686239 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6e50107840 msgr2=0x7f6e50100ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:03.962 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.960+0000 7f6e55354640 1 --2- 172.21.15.16:0/3451686239 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6e50107840 0x7f6e50100ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:03.962 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.960+0000 7f6e55354640 1 -- 172.21.15.16:0/3451686239 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6e501025e0 msgr2=0x7f6e50101f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:03.962 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.960+0000 7f6e55354640 1 --2- 172.21.15.16:0/3451686239 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6e501025e0 0x7f6e50101f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:03.962 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.960+0000 7f6e55354640 1 -- 172.21.15.16:0/3451686239 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e50101360 con 0x7f6e50104f10 2024-09-12T02:10:03.963 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.960+0000 7f6e55354640 1 --2- 172.21.15.16:0/3451686239 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e50104f10 0x7f6e50100590 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f6e40031610 tx=0x7f6e400043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:10:03.963 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.960+0000 7f6e46ffd640 1 -- 172.21.15.16:0/3451686239 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e4003f030 con 0x7f6e50104f10 2024-09-12T02:10:03.963 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.960+0000 7f6e46ffd640 1 -- 172.21.15.16:0/3451686239 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e40003ce0 con 0x7f6e50104f10 2024-09-12T02:10:03.963 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.960+0000 7f6e57db9640 1 -- 172.21.15.16:0/3451686239 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e501a1f50 con 0x7f6e50104f10 2024-09-12T02:10:03.963 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.960+0000 7f6e46ffd640 1 -- 172.21.15.16:0/3451686239 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e40035070 con 0x7f6e50104f10 2024-09-12T02:10:03.963 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.960+0000 7f6e57db9640 1 -- 172.21.15.16:0/3451686239 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e501a23c0 con 0x7f6e50104f10 2024-09-12T02:10:03.966 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.963+0000 7f6e46ffd640 1 -- 172.21.15.16:0/3451686239 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6e40042030 con 0x7f6e50104f10 2024-09-12T02:10:03.966 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.963+0000 7f6e5ac26740 1 Processor -- start 2024-09-12T02:10:03.966 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.963+0000 7f6e5ac26740 1 -- start start 2024-09-12T02:10:03.966 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.964+0000 7f6e5ac26740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be8355b9a0 0x55be8355bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:03.966 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.964+0000 7f6e5ac26740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55be8355c2e0 0x55be83560da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:03.966 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.964+0000 7f6e5ac26740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be83561370 0x55be83563830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:03.966 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.964+0000 7f6e5ac26740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55be834bf2f0 con 0x55be83561370 2024-09-12T02:10:03.966 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.964+0000 7f6e5ac26740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55be83467270 con 0x55be8355b9a0 2024-09-12T02:10:03.966 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.964+0000 7f6e5ac26740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55be834bb2b0 con 0x55be8355c2e0 2024-09-12T02:10:03.967 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.964+0000 7f6e55354640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be83561370 0x55be83563830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:03.967 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.964+0000 7f6e56356640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be8355b9a0 0x55be8355bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:03.967 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.964+0000 7f6e55b55640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55be8355c2e0 0x55be83560da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:03.967 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.964+0000 7f6e55b55640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55be8355c2e0 0x55be83560da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:47878/0 (socket says 172.21.15.16:47878) 2024-09-12T02:10:03.967 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.964+0000 7f6e56356640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be8355b9a0 0x55be8355bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.16:38304/0 (socket says 172.21.15.16:38304) 2024-09-12T02:10:03.967 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.964+0000 7f6e55b55640 1 -- 172.21.15.16:0/3996052899 learned_addr learned my addr 172.21.15.16:0/3996052899 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:10:03.967 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.964+0000 7f6e46ffd640 1 --2- 172.21.15.16:0/3451686239 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f6e3c0a2cb0 0x7f6e3c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:03.967 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.964+0000 7f6e56356640 1 --2- 172.21.15.16:0/3451686239 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f6e3c0a2cb0 0x7f6e3c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:03.967 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.964+0000 7f6e55b55640 1 -- 172.21.15.16:0/3996052899 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be8355b9a0 msgr2=0x55be8355bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:03.967 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.964+0000 7f6e55b55640 1 --2- 172.21.15.16:0/3996052899 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be8355b9a0 0x55be8355bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:03.967 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.964+0000 7f6e55b55640 1 -- 172.21.15.16:0/3996052899 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be83561370 msgr2=0x55be83563830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:03.967 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.964+0000 7f6e55b55640 1 --2- 172.21.15.16:0/3996052899 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be83561370 0x55be83563830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:03.967 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.964+0000 7f6e55b55640 1 -- 172.21.15.16:0/3996052899 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be83466340 con 0x55be8355c2e0 2024-09-12T02:10:03.968 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.964+0000 7f6e46ffd640 1 -- 172.21.15.16:0/3451686239 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f6e400ef090 con 0x7f6e50104f10 2024-09-12T02:10:03.968 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.965+0000 7f6e55b55640 1 --2- 172.21.15.16:0/3996052899 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55be8355c2e0 0x55be83560da0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6e4c0103f0 tx=0x7f6e4c0108b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=341d22dc7943a1fe server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:10:03.968 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.965+0000 7f6e56356640 1 --2- 172.21.15.16:0/3451686239 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f6e3c0a2cb0 0x7f6e3c0a5170 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f6e48005d00 tx=0x7f6e48004730 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:10:03.968 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.965+0000 7f6e3bfff640 1 -- 172.21.15.16:0/3996052899 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e4c016070 con 0x55be8355c2e0 2024-09-12T02:10:03.968 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.965+0000 7f6e3bfff640 1 -- 172.21.15.16:0/3996052899 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e4c004d20 con 0x55be8355c2e0 2024-09-12T02:10:03.968 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.965+0000 7f6e3bfff640 1 -- 172.21.15.16:0/3996052899 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e4c005450 con 0x55be8355c2e0 2024-09-12T02:10:03.968 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.966+0000 7f6e5ac26740 1 -- 172.21.15.16:0/3996052899 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55be8355c2e0 msgr2=0x55be83560da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:03.968 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.966+0000 7f6e5ac26740 1 --2- 172.21.15.16:0/3996052899 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55be8355c2e0 0x55be83560da0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6e4c0103f0 tx=0x7f6e4c0108b0 comp rx=0 tx=0).stop 2024-09-12T02:10:03.968 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.966+0000 7f6e5ac26740 1 -- 172.21.15.16:0/3996052899 shutdown_connections 2024-09-12T02:10:03.968 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.966+0000 7f6e5ac26740 1 --2- 172.21.15.16:0/3996052899 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be83561370 0x55be83563830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:03.968 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.966+0000 7f6e5ac26740 1 --2- 172.21.15.16:0/3996052899 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55be8355c2e0 0x55be83560da0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:03.968 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.966+0000 7f6e5ac26740 1 --2- 172.21.15.16:0/3996052899 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be8355b9a0 0x55be8355bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:03.968 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.966+0000 7f6e5ac26740 1 -- 172.21.15.16:0/3996052899 >> 172.21.15.16:0/3996052899 conn(0x55be83557010 msgr2=0x55be83557410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:10:03.969 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.966+0000 7f6e5ac26740 1 -- 172.21.15.16:0/3996052899 shutdown_connections 2024-09-12T02:10:03.969 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.966+0000 7f6e5ac26740 1 -- 172.21.15.16:0/3996052899 wait complete. 2024-09-12T02:10:03.969 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.966+0000 7f6e5ac26740 1 -- start start 2024-09-12T02:10:03.969 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.966+0000 7f6e5ac26740 1 Processor -- start 2024-09-12T02:10:03.972 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.967+0000 7f6e5ac26740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be8355b9a0 0x55be83565ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:03.972 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.967+0000 7f6e5ac26740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be8355c2e0 0x55be835660e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:03.972 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.967+0000 7f6e5ac26740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55be83561370 0x55be8356d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:03.972 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.967+0000 7f6e5ac26740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55be83564340 con 0x55be8355c2e0 2024-09-12T02:10:03.973 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.967+0000 7f6e5ac26740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55be83448290 con 0x55be8355b9a0 2024-09-12T02:10:03.973 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.967+0000 7f6e5ac26740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55be835644c0 con 0x55be83561370 2024-09-12T02:10:03.973 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.967+0000 7f6e5ac26740 10 client.0 _ll_register_callbacks cb 0x55be83479140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:10:03.973 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.967+0000 7f6e55354640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be8355c2e0 0x55be835660e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:03.973 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.967+0000 7f6e55354640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be8355c2e0 0x55be835660e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57246/0 (socket says 172.21.15.16:57246) 2024-09-12T02:10:03.973 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.967+0000 7f6e55354640 1 -- 172.21.15.16:0/1505897897 learned_addr learned my addr 172.21.15.16:0/1505897897 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:10:03.973 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.967+0000 7f6e56356640 1 --2- 172.21.15.16:0/1505897897 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55be83561370 0x55be8356d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:03.973 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.967+0000 7f6e55b55640 1 --2- 172.21.15.16:0/1505897897 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be8355b9a0 0x55be83565ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:03.973 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.967+0000 7f6e55354640 1 -- 172.21.15.16:0/1505897897 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55be83561370 msgr2=0x55be8356d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:03.973 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.967+0000 7f6e55354640 1 --2- 172.21.15.16:0/1505897897 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55be83561370 0x55be8356d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:03.973 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.967+0000 7f6e55354640 1 -- 172.21.15.16:0/1505897897 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be8355b9a0 msgr2=0x55be83565ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:03.973 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.967+0000 7f6e55354640 1 --2- 172.21.15.16:0/1505897897 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be8355b9a0 0x55be83565ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:03.973 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.967+0000 7f6e55354640 1 -- 172.21.15.16:0/1505897897 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be831d2210 con 0x55be8355c2e0 2024-09-12T02:10:03.974 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.967+0000 7f6e55354640 1 --2- 172.21.15.16:0/1505897897 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be8355c2e0 0x55be835660e0 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f6e40031440 tx=0x7f6e40003f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=832293ba199933d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:10:03.974 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.967+0000 7f6e397fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:10:03.974 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.967+0000 7f6e397fa640 1 -- 172.21.15.16:0/1505897897 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e40049040 con 0x55be8355c2e0 2024-09-12T02:10:03.974 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.967+0000 7f6e397fa640 1 -- 172.21.15.16:0/1505897897 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f6e400b80b0 con 0x55be8355c2e0 2024-09-12T02:10:03.974 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.967+0000 7f6e5ac26740 1 -- 172.21.15.16:0/1505897897 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6e4c004d20 con 0x55be8355c2e0 2024-09-12T02:10:03.974 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.968+0000 7f6e397fa640 1 -- 172.21.15.16:0/1505897897 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6e400bb3a0 con 0x55be8355c2e0 2024-09-12T02:10:03.974 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.968+0000 7f6e5ac26740 10 client.17925 fetch_fsmap learned FSMap version 25 2024-09-12T02:10:03.974 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.968+0000 7f6e5ac26740 1 -- 172.21.15.16:0/1505897897 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55be8329e620 con 0x55be8355c2e0 2024-09-12T02:10:03.974 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.968+0000 7f6e397fa640 1 -- 172.21.15.16:0/1505897897 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e4c004d20 con 0x55be8355c2e0 2024-09-12T02:10:03.974 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.968+0000 7f6e397fa640 1 -- 172.21.15.16:0/1505897897 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f6e400b7790 con 0x55be8355c2e0 2024-09-12T02:10:03.974 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.968+0000 7f6e5ac26740 10 client.17925 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:10:03.974 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.968+0000 7f6e5ac26740 10 client.17925 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:10:03.974 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.968+0000 7f6e5ac26740 20 client.17925 populate_metadata read hostname 'smithi016' 2024-09-12T02:10:03.974 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.968+0000 7f6e5ac26740 1 --2- 172.21.15.16:0/1505897897 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55be835718d0 0x55be83573d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:03.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.968+0000 7f6e5ac26740 4 client.17925 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:10:03.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.968+0000 7f6e5ac26740 1 -- 172.21.15.16:0/1505897897 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55be834cca20 con 0x55be835718d0 2024-09-12T02:10:03.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.968+0000 7f6e55b55640 1 --2- 172.21.15.16:0/1505897897 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55be835718d0 0x55be83573d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:03.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.969+0000 7f6e55b55640 1 --2- 172.21.15.16:0/1505897897 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55be835718d0 0x55be83573d20 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3d681e8207f00e7 server_cookie=fe6dccee3b9f1f8f in_seq=0 out_seq=0 2024-09-12T02:10:03.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.969+0000 7f6e397fa640 10 client.17925 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:10:03.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.970+0000 7f6e397fa640 1 -- 172.21.15.16:0/1505897897 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55be834cca20 con 0x55be835718d0 2024-09-12T02:10:03.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.970+0000 7f6e397fa640 10 client.17925 handle_command_reply: tid=0 2024-09-12T02:10:03.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.970+0000 7f6e5ac26740 1 client.17925 shutdown 2024-09-12T02:10:03.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.970+0000 7f6e5ac26740 1 -- 172.21.15.16:0/1505897897 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be8355c2e0 msgr2=0x55be835660e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:03.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.970+0000 7f6e5ac26740 1 --2- 172.21.15.16:0/1505897897 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be8355c2e0 0x55be835660e0 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f6e40031440 tx=0x7f6e40003f50 comp rx=0 tx=0).stop 2024-09-12T02:10:03.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.970+0000 7f6e5ac26740 1 -- 172.21.15.16:0/1505897897 shutdown_connections 2024-09-12T02:10:03.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.970+0000 7f6e5ac26740 1 --2- 172.21.15.16:0/1505897897 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55be835718d0 0x55be83573d20 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:03.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.970+0000 7f6e5ac26740 1 --2- 172.21.15.16:0/1505897897 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55be83561370 0x55be8356d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:03.976 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.970+0000 7f6e5ac26740 1 --2- 172.21.15.16:0/1505897897 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be8355c2e0 0x55be835660e0 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:03.976 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.970+0000 7f6e397fa640 0 client.17925 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:10:03.976 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.970+0000 7f6e5ac26740 1 --2- 172.21.15.16:0/1505897897 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be8355b9a0 0x55be83565ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:03.976 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.970+0000 7f6e5ac26740 1 -- 172.21.15.16:0/1505897897 >> 172.21.15.16:0/1505897897 conn(0x55be83557010 msgr2=0x55be83558820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:10:03.976 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.970+0000 7f6e5ac26740 1 -- 172.21.15.16:0/1505897897 shutdown_connections 2024-09-12T02:10:03.976 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.970+0000 7f6e5ac26740 1 -- 172.21.15.16:0/1505897897 wait complete. 2024-09-12T02:10:03.976 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.970+0000 7f6e5ac26740 20 client.17925 trim_cache size 0 max 16384 2024-09-12T02:10:03.989 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.984+0000 7f6e5ac26740 1 Processor -- start 2024-09-12T02:10:03.990 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.984+0000 7f6e5ac26740 1 -- start start 2024-09-12T02:10:03.990 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.985+0000 7f6e5ac26740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be8355b9a0 0x55be83557ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:03.990 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.985+0000 7f6e5ac26740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55be8355c2e0 0x55be8356d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:03.990 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.985+0000 7f6e5ac26740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be83561370 0x55be83558400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:03.990 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.985+0000 7f6e5ac26740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6e501ad8e0 con 0x55be8355b9a0 2024-09-12T02:10:03.990 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.985+0000 7f6e5ac26740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6e501ab1b0 con 0x55be83561370 2024-09-12T02:10:03.990 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.985+0000 7f6e5ac26740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f6e501a9000 con 0x55be8355c2e0 2024-09-12T02:10:03.990 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.985+0000 7f6e55b55640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be8355b9a0 0x55be83557ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:03.990 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.985+0000 7f6e55b55640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be8355b9a0 0x55be83557ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57254/0 (socket says 172.21.15.16:57254) 2024-09-12T02:10:03.990 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.985+0000 7f6e55b55640 1 -- 172.21.15.16:0/2361061074 learned_addr learned my addr 172.21.15.16:0/2361061074 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:10:03.990 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.985+0000 7f6e56356640 1 --2- 172.21.15.16:0/2361061074 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be83561370 0x55be83558400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:03.991 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.985+0000 7f6e55354640 1 --2- 172.21.15.16:0/2361061074 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55be8355c2e0 0x55be8356d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:03.991 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.985+0000 7f6e55b55640 1 -- 172.21.15.16:0/2361061074 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55be8355c2e0 msgr2=0x55be8356d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:03.991 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.985+0000 7f6e55b55640 1 --2- 172.21.15.16:0/2361061074 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55be8355c2e0 0x55be8356d0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:03.991 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.985+0000 7f6e55b55640 1 -- 172.21.15.16:0/2361061074 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be83561370 msgr2=0x55be83558400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:03.991 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.985+0000 7f6e55b55640 1 --2- 172.21.15.16:0/2361061074 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be83561370 0x55be83558400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:03.991 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.985+0000 7f6e55b55640 1 -- 172.21.15.16:0/2361061074 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be83557ca0 con 0x55be8355b9a0 2024-09-12T02:10:03.991 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.985+0000 7f6e56356640 1 --2- 172.21.15.16:0/2361061074 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be83561370 0x55be83558400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:10:03.991 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.985+0000 7f6e55b55640 1 --2- 172.21.15.16:0/2361061074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be8355b9a0 0x55be83557ec0 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f6e4c009e30 tx=0x7f6e4c009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b68591fa1c51e8ad server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:10:03.991 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.985+0000 7f6e3b7fe640 1 -- 172.21.15.16:0/2361061074 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e4c016050 con 0x55be8355b9a0 2024-09-12T02:10:03.991 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.985+0000 7f6e3b7fe640 1 -- 172.21.15.16:0/2361061074 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e4c003e50 con 0x55be8355b9a0 2024-09-12T02:10:03.991 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.985+0000 7f6e3b7fe640 1 -- 172.21.15.16:0/2361061074 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e4c012510 con 0x55be8355b9a0 2024-09-12T02:10:03.991 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.986+0000 7f6e5ac26740 1 -- 172.21.15.16:0/2361061074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be8355b9a0 msgr2=0x55be83557ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:03.992 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.986+0000 7f6e5ac26740 1 --2- 172.21.15.16:0/2361061074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be8355b9a0 0x55be83557ec0 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f6e4c009e30 tx=0x7f6e4c009680 comp rx=0 tx=0).stop 2024-09-12T02:10:03.992 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.986+0000 7f6e5ac26740 1 -- 172.21.15.16:0/2361061074 shutdown_connections 2024-09-12T02:10:03.992 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.986+0000 7f6e5ac26740 1 --2- 172.21.15.16:0/2361061074 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be83561370 0x55be83558400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:03.992 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.986+0000 7f6e5ac26740 1 --2- 172.21.15.16:0/2361061074 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55be8355c2e0 0x55be8356d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:03.992 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.986+0000 7f6e5ac26740 1 --2- 172.21.15.16:0/2361061074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be8355b9a0 0x55be83557ec0 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:03.992 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.986+0000 7f6e5ac26740 1 -- 172.21.15.16:0/2361061074 >> 172.21.15.16:0/2361061074 conn(0x55be83557010 msgr2=0x55be83565ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:10:03.992 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.986+0000 7f6e5ac26740 1 -- 172.21.15.16:0/2361061074 shutdown_connections 2024-09-12T02:10:03.992 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.986+0000 7f6e5ac26740 1 -- 172.21.15.16:0/2361061074 wait complete. 2024-09-12T02:10:03.992 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.986+0000 7f6e5ac26740 1 -- start start 2024-09-12T02:10:03.992 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.986+0000 7f6e5ac26740 1 Processor -- start 2024-09-12T02:10:03.992 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.987+0000 7f6e5ac26740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be8355b9a0 0x55be8356aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:03.992 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.987+0000 7f6e5ac26740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55be8355c2e0 0x55be8356b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:03.992 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.987+0000 7f6e5ac26740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be83561370 0x55be8356c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:03.993 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.987+0000 7f6e5ac26740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6e501a9670 con 0x55be83561370 2024-09-12T02:10:03.993 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.987+0000 7f6e5ac26740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6e501aaae0 con 0x55be8355b9a0 2024-09-12T02:10:03.993 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.987+0000 7f6e5ac26740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f6e501a9d40 con 0x55be8355c2e0 2024-09-12T02:10:03.993 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.987+0000 7f6e5ac26740 10 client.0 _ll_register_callbacks cb 0x7f6e400b6420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:10:03.993 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.987+0000 7f6e56356640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be83561370 0x55be8356c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:03.993 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.987+0000 7f6e56356640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be83561370 0x55be8356c0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57260/0 (socket says 172.21.15.16:57260) 2024-09-12T02:10:03.993 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.987+0000 7f6e56356640 1 -- 172.21.15.16:0/2213434046 learned_addr learned my addr 172.21.15.16:0/2213434046 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:10:03.993 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.987+0000 7f6e55354640 1 --2- 172.21.15.16:0/2213434046 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55be8355c2e0 0x55be8356b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:03.993 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.987+0000 7f6e55b55640 1 --2- 172.21.15.16:0/2213434046 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be8355b9a0 0x55be8356aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:03.993 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.987+0000 7f6e56356640 1 -- 172.21.15.16:0/2213434046 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55be8355c2e0 msgr2=0x55be8356b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:03.993 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.987+0000 7f6e56356640 1 --2- 172.21.15.16:0/2213434046 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55be8355c2e0 0x55be8356b0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:03.993 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.987+0000 7f6e56356640 1 -- 172.21.15.16:0/2213434046 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be8355b9a0 msgr2=0x55be8356aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:03.994 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.987+0000 7f6e56356640 1 --2- 172.21.15.16:0/2213434046 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be8355b9a0 0x55be8356aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:03.994 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.987+0000 7f6e56356640 1 -- 172.21.15.16:0/2213434046 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be8356c700 con 0x55be83561370 2024-09-12T02:10:03.994 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.987+0000 7f6e56356640 1 --2- 172.21.15.16:0/2213434046 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be83561370 0x55be8356c0f0 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f6e4800bf70 tx=0x7f6e480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84a72d15e4675c53 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:10:03.994 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.987+0000 7f6e397fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:10:03.994 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.988+0000 7f6e397fa640 1 -- 172.21.15.16:0/2213434046 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e48021690 con 0x55be83561370 2024-09-12T02:10:03.994 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.988+0000 7f6e397fa640 1 -- 172.21.15.16:0/2213434046 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f6e48021830 con 0x55be83561370 2024-09-12T02:10:03.994 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.988+0000 7f6e5ac26740 1 -- 172.21.15.16:0/2213434046 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6e4c003e50 con 0x55be83561370 2024-09-12T02:10:03.994 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.988+0000 7f6e397fa640 1 -- 172.21.15.16:0/2213434046 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6e48021a30 con 0x55be83561370 2024-09-12T02:10:03.994 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.988+0000 7f6e5ac26740 10 client.17931 fetch_fsmap learned FSMap version 25 2024-09-12T02:10:03.994 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.988+0000 7f6e5ac26740 1 -- 172.21.15.16:0/2213434046 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55be835dcac0 con 0x55be83561370 2024-09-12T02:10:03.994 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.988+0000 7f6e397fa640 1 -- 172.21.15.16:0/2213434046 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e4c003e50 con 0x55be83561370 2024-09-12T02:10:03.994 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.988+0000 7f6e397fa640 1 -- 172.21.15.16:0/2213434046 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f6e4801e040 con 0x55be83561370 2024-09-12T02:10:03.994 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.988+0000 7f6e5ac26740 10 client.17931 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:10:03.994 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.988+0000 7f6e5ac26740 10 client.17931 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:10:03.995 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.988+0000 7f6e5ac26740 20 client.17931 populate_metadata read hostname 'smithi016' 2024-09-12T02:10:03.995 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.988+0000 7f6e5ac26740 1 --2- 172.21.15.16:0/2213434046 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55be835dd4d0 0x55be835df8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:03.995 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.988+0000 7f6e5ac26740 4 client.17931 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:10:03.995 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.988+0000 7f6e5ac26740 1 -- 172.21.15.16:0/2213434046 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55be834c6d70 con 0x55be835dd4d0 2024-09-12T02:10:03.995 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.989+0000 7f6e55b55640 1 --2- 172.21.15.16:0/2213434046 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55be835dd4d0 0x55be835df8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:03.995 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.989+0000 7f6e397fa640 10 client.17931 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:10:03.995 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:03.989+0000 7f6e55b55640 1 --2- 172.21.15.16:0/2213434046 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55be835dd4d0 0x55be835df8f0 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2c71f4b129637d8 server_cookie=b3721d14c3e0ec54 in_seq=0 out_seq=0 2024-09-12T02:10:04.003 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:04.001+0000 7f6e397fa640 1 -- 172.21.15.16:0/2213434046 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55be834c6d70 con 0x55be835dd4d0 2024-09-12T02:10:04.004 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:04.001+0000 7f6e397fa640 10 client.17931 handle_command_reply: tid=0 2024-09-12T02:10:04.004 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:04.001+0000 7f6e5ac26740 1 client.17931 shutdown 2024-09-12T02:10:04.004 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:04.001+0000 7f6e5ac26740 1 -- 172.21.15.16:0/2213434046 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be83561370 msgr2=0x55be8356c0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:04.004 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:04.001+0000 7f6e5ac26740 1 --2- 172.21.15.16:0/2213434046 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be83561370 0x55be8356c0f0 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f6e4800bf70 tx=0x7f6e480046c0 comp rx=0 tx=0).stop 2024-09-12T02:10:04.004 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:04.001+0000 7f6e5ac26740 1 -- 172.21.15.16:0/2213434046 shutdown_connections 2024-09-12T02:10:04.004 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:04.001+0000 7f6e5ac26740 1 --2- 172.21.15.16:0/2213434046 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55be835dd4d0 0x55be835df8f0 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:04.004 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:04.001+0000 7f6e5ac26740 1 --2- 172.21.15.16:0/2213434046 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be83561370 0x55be8356c0f0 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:04.004 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:04.001+0000 7f6e5ac26740 1 --2- 172.21.15.16:0/2213434046 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55be8355c2e0 0x55be8356b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:04.004 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:04.001+0000 7f6e5ac26740 1 --2- 172.21.15.16:0/2213434046 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be8355b9a0 0x55be8356aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:04.004 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:04.001+0000 7f6e397fa640 0 client.17931 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:10:04.004 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:04.001+0000 7f6e5ac26740 1 -- 172.21.15.16:0/2213434046 >> 172.21.15.16:0/2213434046 conn(0x55be83557010 msgr2=0x55be835559a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:10:04.005 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:04.001+0000 7f6e5ac26740 1 -- 172.21.15.16:0/2213434046 shutdown_connections 2024-09-12T02:10:04.005 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:04.001+0000 7f6e5ac26740 1 -- 172.21.15.16:0/2213434046 wait complete. 2024-09-12T02:10:04.005 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:04.002+0000 7f6e5ac26740 20 client.17931 trim_cache size 0 max 16384 2024-09-12T02:10:04.005 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-09-12T02:10:04.005 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-09-12T02:10:04.005 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-09-12T02:10:04.005 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-09-12T02:10:04.006 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:04.002+0000 7f6e57db9640 1 -- 172.21.15.16:0/3451686239 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f6e3c0a2cb0 msgr2=0x7f6e3c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:04.006 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:04.002+0000 7f6e57db9640 1 --2- 172.21.15.16:0/3451686239 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f6e3c0a2cb0 0x7f6e3c0a5170 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f6e48005d00 tx=0x7f6e48004730 comp rx=0 tx=0).stop 2024-09-12T02:10:04.006 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:04.002+0000 7f6e57db9640 1 -- 172.21.15.16:0/3451686239 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e50104f10 msgr2=0x7f6e50100590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:04.006 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:04.002+0000 7f6e57db9640 1 --2- 172.21.15.16:0/3451686239 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e50104f10 0x7f6e50100590 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f6e40031610 tx=0x7f6e400043b0 comp rx=0 tx=0).stop 2024-09-12T02:10:04.006 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:04.002+0000 7f6e57db9640 1 -- 172.21.15.16:0/3451686239 shutdown_connections 2024-09-12T02:10:04.007 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:04.002+0000 7f6e57db9640 1 --2- 172.21.15.16:0/3451686239 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f6e3c0a2cb0 0x7f6e3c0a5170 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:04.007 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:04.002+0000 7f6e57db9640 1 --2- 172.21.15.16:0/3451686239 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6e50107840 0x7f6e50100ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:04.007 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:04.002+0000 7f6e57db9640 1 --2- 172.21.15.16:0/3451686239 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e50104f10 0x7f6e50100590 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:04.007 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:04.002+0000 7f6e57db9640 1 --2- 172.21.15.16:0/3451686239 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6e501025e0 0x7f6e50101f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:04.007 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:04.002+0000 7f6e57db9640 1 -- 172.21.15.16:0/3451686239 >> 172.21.15.16:0/3451686239 conn(0x7f6e500fc1e0 msgr2=0x7f6e50103040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:10:04.007 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:04.002+0000 7f6e57db9640 1 -- 172.21.15.16:0/3451686239 shutdown_connections 2024-09-12T02:10:04.007 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:04.002+0000 7f6e57db9640 1 -- 172.21.15.16:0/3451686239 wait complete. 2024-09-12T02:10:05.876 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:10:05.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:10:05 smithi184 ceph-mon[35593]: pgmap v580: 97 pgs: 97 active+clean; 4.2 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 3.0 KiB/s wr, 9 op/s 2024-09-12T02:10:05.906 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:10:05 smithi016 ceph-mon[30947]: pgmap v580: 97 pgs: 97 active+clean; 4.2 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 3.0 KiB/s wr, 9 op/s 2024-09-12T02:10:05.918 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:10:05 smithi033 ceph-mon[35332]: pgmap v580: 97 pgs: 97 active+clean; 4.2 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 3.0 KiB/s wr, 9 op/s 2024-09-12T02:10:06.154 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:10:06.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:06.640+0000 7f7eeefa1640 1 -- 172.21.15.16:0/3347305637 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ee81068d0 msgr2=0x7f7ee8106cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:06.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:06.640+0000 7f7eeefa1640 1 --2- 172.21.15.16:0/3347305637 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ee81068d0 0x7f7ee8106cb0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f7edc009930 tx=0x7f7edc02a420 comp rx=0 tx=0).stop 2024-09-12T02:10:06.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:06.640+0000 7f7eeefa1640 1 -- 172.21.15.16:0/3347305637 shutdown_connections 2024-09-12T02:10:06.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:06.640+0000 7f7eeefa1640 1 --2- 172.21.15.16:0/3347305637 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7ee8101270 0x7f7ee810d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:06.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:06.640+0000 7f7eeefa1640 1 --2- 172.21.15.16:0/3347305637 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7ee81008d0 0x7f7ee8100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:06.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:06.640+0000 7f7eeefa1640 1 --2- 172.21.15.16:0/3347305637 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ee81068d0 0x7f7ee8106cb0 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:06.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:06.640+0000 7f7eeefa1640 1 -- 172.21.15.16:0/3347305637 >> 172.21.15.16:0/3347305637 conn(0x7f7ee80fc610 msgr2=0x7f7ee80fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:10:06.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:06.640+0000 7f7eeefa1640 1 -- 172.21.15.16:0/3347305637 shutdown_connections 2024-09-12T02:10:06.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:06.640+0000 7f7eeefa1640 1 -- 172.21.15.16:0/3347305637 wait complete. 2024-09-12T02:10:06.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:06.641+0000 7f7eeefa1640 1 Processor -- start 2024-09-12T02:10:06.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:06.641+0000 7f7eeefa1640 1 -- start start 2024-09-12T02:10:06.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:06.641+0000 7f7eeefa1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7ee81008d0 0x7f7ee81982c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:06.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:06.641+0000 7f7eeefa1640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7ee8101270 0x7f7ee8198800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:06.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:06.641+0000 7f7eeefa1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ee81068d0 0x7f7ee819cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:06.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:06.641+0000 7f7eeefa1640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7ee8111df0 con 0x7f7ee81068d0 2024-09-12T02:10:06.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:06.641+0000 7f7eeefa1640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7ee8111c70 con 0x7f7ee81008d0 2024-09-12T02:10:06.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:06.641+0000 7f7eeefa1640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7ee8111f70 con 0x7f7ee8101270 2024-09-12T02:10:06.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:06.641+0000 7f7eee7a0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ee81068d0 0x7f7ee819cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:06.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:06.641+0000 7f7eee7a0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ee81068d0 0x7f7ee819cb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57278/0 (socket says 172.21.15.16:57278) 2024-09-12T02:10:06.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:06.641+0000 7f7eee7a0640 1 -- 172.21.15.16:0/3859036715 learned_addr learned my addr 172.21.15.16:0/3859036715 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:10:06.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:06.641+0000 7f7eedf9f640 1 --2- 172.21.15.16:0/3859036715 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7ee81008d0 0x7f7ee81982c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:06.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:06.641+0000 7f7eed79e640 1 --2- 172.21.15.16:0/3859036715 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7ee8101270 0x7f7ee8198800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:06.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:06.642+0000 7f7eee7a0640 1 -- 172.21.15.16:0/3859036715 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7ee8101270 msgr2=0x7f7ee8198800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:06.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:06.642+0000 7f7eee7a0640 1 --2- 172.21.15.16:0/3859036715 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7ee8101270 0x7f7ee8198800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:06.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:06.642+0000 7f7eee7a0640 1 -- 172.21.15.16:0/3859036715 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7ee81008d0 msgr2=0x7f7ee81982c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:06.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:06.642+0000 7f7eee7a0640 1 --2- 172.21.15.16:0/3859036715 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7ee81008d0 0x7f7ee81982c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:06.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:06.642+0000 7f7eee7a0640 1 -- 172.21.15.16:0/3859036715 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ee819d270 con 0x7f7ee81068d0 2024-09-12T02:10:06.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:06.642+0000 7f7eed79e640 1 --2- 172.21.15.16:0/3859036715 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7ee8101270 0x7f7ee8198800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:10:06.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:06.642+0000 7f7eee7a0640 1 --2- 172.21.15.16:0/3859036715 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ee81068d0 0x7f7ee819cb90 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f7ee000eaa0 tx=0x7f7ee000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:10:06.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:06.642+0000 7f7edaffd640 1 -- 172.21.15.16:0/3859036715 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ee000cdb0 con 0x7f7ee81068d0 2024-09-12T02:10:06.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:06.642+0000 7f7edaffd640 1 -- 172.21.15.16:0/3859036715 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ee0005020 con 0x7f7ee81068d0 2024-09-12T02:10:06.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:06.642+0000 7f7eeefa1640 1 -- 172.21.15.16:0/3859036715 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ee819d560 con 0x7f7ee81068d0 2024-09-12T02:10:06.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:06.642+0000 7f7edaffd640 1 -- 172.21.15.16:0/3859036715 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ee00116b0 con 0x7f7ee81068d0 2024-09-12T02:10:06.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:06.642+0000 7f7eeefa1640 1 -- 172.21.15.16:0/3859036715 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ee81a4e40 con 0x7f7ee81068d0 2024-09-12T02:10:06.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:06.643+0000 7f7eeefa1640 1 -- 172.21.15.16:0/3859036715 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ee8100d30 con 0x7f7ee81068d0 2024-09-12T02:10:06.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:06.644+0000 7f7edaffd640 1 -- 172.21.15.16:0/3859036715 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f7ee0005560 con 0x7f7ee81068d0 2024-09-12T02:10:06.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:06.645+0000 7f7edaffd640 1 --2- 172.21.15.16:0/3859036715 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f7ed40a2b80 0x7f7ed40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:06.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:06.645+0000 7f7edaffd640 1 -- 172.21.15.16:0/3859036715 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f7ee00cf220 con 0x7f7ee81068d0 2024-09-12T02:10:06.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:06.645+0000 7f7eedf9f640 1 --2- 172.21.15.16:0/3859036715 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f7ed40a2b80 0x7f7ed40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:06.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:06.645+0000 7f7eedf9f640 1 --2- 172.21.15.16:0/3859036715 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f7ed40a2b80 0x7f7ed40a5040 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f7edc006010 tx=0x7f7edc000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:10:06.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:06.649+0000 7f7edaffd640 1 -- 172.21.15.16:0/3859036715 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7ee0016030 con 0x7f7ee81068d0 2024-09-12T02:10:06.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:06.859+0000 7f7eeefa1640 1 -- 172.21.15.16:0/3859036715 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7ee802d7c0 con 0x7f7ee81068d0 2024-09-12T02:10:06.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:06.860+0000 7f7edaffd640 1 -- 172.21.15.16:0/3859036715 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11911 (secure 0 0 0) 0x7f7ee009c180 con 0x7f7ee81068d0 2024-09-12T02:10:06.861 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T02:10:06.861 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":25,"btime":"2024-09-12T02:01:37:192021+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3279123279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3279123279},{"type":"v1","addr":"172.21.15.16:6835","nonce":3279123279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4119452382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4119452382},{"type":"v1","addr":"172.21.15.16:6837","nonce":4119452382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/3263823165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3263823165},{"type":"v1","addr":"172.21.15.16:6839","nonce":3263823165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/1510290732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":1510290732},{"type":"v1","addr":"172.21.15.16:6841","nonce":1510290732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/279674603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":279674603},{"type":"v1","addr":"172.21.15.33:6833","nonce":279674603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3555728028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3555728028},{"type":"v1","addr":"172.21.15.33:6835","nonce":3555728028}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T01:59:56.117608+0000","modified":"2024-09-12T02:01:36.294760+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24482,"mds_1":24481,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6833/1563374106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1563374106},{"type":"v1","addr":"172.21.15.184:6833","nonce":1563374106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6837/1892833868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1892833868},{"type":"v1","addr":"172.21.15.33:6837","nonce":1892833868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/1457679686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1457679686},{"type":"v1","addr":"172.21.15.184:6835","nonce":1457679686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.184:6837/1793393087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":1793393087},{"type":"v1","addr":"172.21.15.184:6837","nonce":1793393087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.184:6839/4203347941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":4203347941},{"type":"v1","addr":"172.21.15.184:6839","nonce":4203347941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.33:6839/4136670204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":4136670204},{"type":"v1","addr":"172.21.15.33:6839","nonce":4136670204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24481,24482,24469]},"id":1}]} 2024-09-12T02:10:06.861 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 25 2024-09-12T02:10:06.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:06.862+0000 7f7eeefa1640 1 -- 172.21.15.16:0/3859036715 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f7ed40a2b80 msgr2=0x7f7ed40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:06.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:06.862+0000 7f7eeefa1640 1 --2- 172.21.15.16:0/3859036715 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f7ed40a2b80 0x7f7ed40a5040 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f7edc006010 tx=0x7f7edc000bc0 comp rx=0 tx=0).stop 2024-09-12T02:10:06.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:06.862+0000 7f7eeefa1640 1 -- 172.21.15.16:0/3859036715 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ee81068d0 msgr2=0x7f7ee819cb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:06.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:06.862+0000 7f7eeefa1640 1 --2- 172.21.15.16:0/3859036715 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ee81068d0 0x7f7ee819cb90 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f7ee000eaa0 tx=0x7f7ee000ef60 comp rx=0 tx=0).stop 2024-09-12T02:10:06.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:06.862+0000 7f7eeefa1640 1 -- 172.21.15.16:0/3859036715 shutdown_connections 2024-09-12T02:10:06.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:06.862+0000 7f7eeefa1640 1 --2- 172.21.15.16:0/3859036715 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f7ed40a2b80 0x7f7ed40a5040 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:06.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:06.862+0000 7f7eeefa1640 1 --2- 172.21.15.16:0/3859036715 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ee81068d0 0x7f7ee819cb90 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:06.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:06.862+0000 7f7eeefa1640 1 --2- 172.21.15.16:0/3859036715 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7ee8101270 0x7f7ee8198800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:06.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:06.862+0000 7f7eeefa1640 1 --2- 172.21.15.16:0/3859036715 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7ee81008d0 0x7f7ee81982c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:06.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:06.862+0000 7f7eeefa1640 1 -- 172.21.15.16:0/3859036715 >> 172.21.15.16:0/3859036715 conn(0x7f7ee80fc610 msgr2=0x7f7ee80fce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:10:06.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:06.862+0000 7f7eeefa1640 1 -- 172.21.15.16:0/3859036715 shutdown_connections 2024-09-12T02:10:06.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:06.862+0000 7f7eeefa1640 1 -- 172.21.15.16:0/3859036715 wait complete. 2024-09-12T02:10:07.614 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24482 quiesce db --set-id f9fc3d8a --release --await' 2024-09-12T02:10:07.614 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24482 quiesce db --set-id f9fc3d8a --release --await 2024-09-12T02:10:07.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:10:07 smithi184 ceph-mon[35593]: pgmap v581: 97 pgs: 97 active+clean; 4.2 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-09-12T02:10:07.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:10:07 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/3859036715' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:10:07.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:10:07 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:10:07.917 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:10:07.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:10:07 smithi033 ceph-mon[35332]: pgmap v581: 97 pgs: 97 active+clean; 4.2 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-09-12T02:10:07.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:10:07 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/3859036715' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:10:07.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:10:07 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:10:07.933 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:10:07 smithi016 ceph-mon[30947]: pgmap v581: 97 pgs: 97 active+clean; 4.2 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-09-12T02:10:07.933 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:10:07 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/3859036715' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:10:07.933 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:10:07 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:10:08.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.361+0000 7f6d0d5c5640 1 -- 172.21.15.16:0/2817093562 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d08102790 msgr2=0x7f6d081067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:08.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.361+0000 7f6d0d5c5640 1 --2- 172.21.15.16:0/2817093562 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d08102790 0x7f6d081067e0 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f6cfc009900 tx=0x7f6cfc02a410 comp rx=0 tx=0).stop 2024-09-12T02:10:08.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.361+0000 7f6d0d5c5640 1 -- 172.21.15.16:0/2817093562 shutdown_connections 2024-09-12T02:10:08.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.361+0000 7f6d0d5c5640 1 --2- 172.21.15.16:0/2817093562 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d08106f10 0x7f6d0810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:08.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.361+0000 7f6d0d5c5640 1 --2- 172.21.15.16:0/2817093562 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d08102790 0x7f6d081067e0 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:08.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.361+0000 7f6d0d5c5640 1 --2- 172.21.15.16:0/2817093562 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6d08101de0 0x7f6d081021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:08.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.361+0000 7f6d0d5c5640 1 -- 172.21.15.16:0/2817093562 >> 172.21.15.16:0/2817093562 conn(0x7f6d080fd8c0 msgr2=0x7f6d080ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:10:08.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.362+0000 7f6d0d5c5640 1 -- 172.21.15.16:0/2817093562 shutdown_connections 2024-09-12T02:10:08.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.362+0000 7f6d0d5c5640 1 -- 172.21.15.16:0/2817093562 wait complete. 2024-09-12T02:10:08.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.362+0000 7f6d0d5c5640 1 Processor -- start 2024-09-12T02:10:08.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.362+0000 7f6d0d5c5640 1 -- start start 2024-09-12T02:10:08.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.363+0000 7f6d0d5c5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d08101de0 0x7f6d081984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:08.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.363+0000 7f6d0d5c5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d08102790 0x7f6d081989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:08.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.363+0000 7f6d0d5c5640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6d08106f10 0x7f6d0819cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:08.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.363+0000 7f6d0d5c5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6d08111f70 con 0x7f6d08101de0 2024-09-12T02:10:08.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.363+0000 7f6d0d5c5640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6d08111df0 con 0x7f6d08102790 2024-09-12T02:10:08.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.363+0000 7f6d0d5c5640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f6d081120f0 con 0x7f6d08106f10 2024-09-12T02:10:08.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.363+0000 7f6d06ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d08101de0 0x7f6d081984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:08.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.363+0000 7f6d06ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d08101de0 0x7f6d081984b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57300/0 (socket says 172.21.15.16:57300) 2024-09-12T02:10:08.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.363+0000 7f6d06ffd640 1 -- 172.21.15.16:0/2747332876 learned_addr learned my addr 172.21.15.16:0/2747332876 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:10:08.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.363+0000 7f6d067fc640 1 --2- 172.21.15.16:0/2747332876 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d08102790 0x7f6d081989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:08.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.363+0000 7f6d077fe640 1 --2- 172.21.15.16:0/2747332876 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6d08106f10 0x7f6d0819cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:08.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.363+0000 7f6d06ffd640 1 -- 172.21.15.16:0/2747332876 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6d08106f10 msgr2=0x7f6d0819cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:08.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.363+0000 7f6d06ffd640 1 --2- 172.21.15.16:0/2747332876 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6d08106f10 0x7f6d0819cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:08.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.363+0000 7f6d06ffd640 1 -- 172.21.15.16:0/2747332876 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d08102790 msgr2=0x7f6d081989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:08.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.363+0000 7f6d06ffd640 1 --2- 172.21.15.16:0/2747332876 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d08102790 0x7f6d081989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:08.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.363+0000 7f6d06ffd640 1 -- 172.21.15.16:0/2747332876 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d0819d460 con 0x7f6d08101de0 2024-09-12T02:10:08.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.363+0000 7f6d077fe640 1 --2- 172.21.15.16:0/2747332876 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6d08106f10 0x7f6d0819cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:10:08.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.363+0000 7f6d06ffd640 1 --2- 172.21.15.16:0/2747332876 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d08101de0 0x7f6d081984b0 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f6cf800c970 tx=0x7f6cf800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:10:08.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.364+0000 7f6ceffff640 1 -- 172.21.15.16:0/2747332876 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cf8007bb0 con 0x7f6d08101de0 2024-09-12T02:10:08.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.364+0000 7f6ceffff640 1 -- 172.21.15.16:0/2747332876 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6cf8007d50 con 0x7f6d08101de0 2024-09-12T02:10:08.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.364+0000 7f6ceffff640 1 -- 172.21.15.16:0/2747332876 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cf800f450 con 0x7f6d08101de0 2024-09-12T02:10:08.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.364+0000 7f6d0d5c5640 1 -- 172.21.15.16:0/2747332876 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d0819d750 con 0x7f6d08101de0 2024-09-12T02:10:08.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.364+0000 7f6d0d5c5640 1 -- 172.21.15.16:0/2747332876 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d08074420 con 0x7f6d08101de0 2024-09-12T02:10:08.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.366+0000 7f6ceffff640 1 -- 172.21.15.16:0/2747332876 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6cf8005020 con 0x7f6d08101de0 2024-09-12T02:10:08.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.366+0000 7f6d0f99e740 1 Processor -- start 2024-09-12T02:10:08.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.367+0000 7f6d0f99e740 1 -- start start 2024-09-12T02:10:08.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.367+0000 7f6d0f99e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc36ba5060 0x55dc36ba5440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:08.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.367+0000 7f6d0f99e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc36ba5980 0x55dc36bafd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:08.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.367+0000 7f6d0f99e740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dc36bb0240 0x55dc36bb2630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:08.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.367+0000 7f6d0f99e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55dc36b0a620 con 0x55dc36ba5980 2024-09-12T02:10:08.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.367+0000 7f6d0f99e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55dc36ab1e50 con 0x55dc36ba5060 2024-09-12T02:10:08.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.367+0000 7f6d0f99e740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55dc36b065e0 con 0x55dc36bb0240 2024-09-12T02:10:08.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.367+0000 7f6d06ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc36ba5980 0x55dc36bafd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:08.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.367+0000 7f6ceffff640 1 --2- 172.21.15.16:0/2747332876 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f6cf00a2c00 0x7f6cf00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:08.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.367+0000 7f6d06ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc36ba5980 0x55dc36bafd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57310/0 (socket says 172.21.15.16:57310) 2024-09-12T02:10:08.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.367+0000 7f6d06ffd640 1 -- 172.21.15.16:0/2356545134 learned_addr learned my addr 172.21.15.16:0/2356545134 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:10:08.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.367+0000 7f6d077fe640 1 --2- 172.21.15.16:0/2356545134 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc36ba5060 0x55dc36ba5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:08.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.367+0000 7f6d077fe640 1 --2- 172.21.15.16:0/2747332876 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f6cf00a2c00 0x7f6cf00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:08.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.367+0000 7f6d067fc640 1 --2- 172.21.15.16:0/2356545134 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dc36bb0240 0x55dc36bb2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:08.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.367+0000 7f6ceffff640 1 -- 172.21.15.16:0/2747332876 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f6cf80ced00 con 0x7f6d08101de0 2024-09-12T02:10:08.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.367+0000 7f6d06ffd640 1 -- 172.21.15.16:0/2356545134 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dc36bb0240 msgr2=0x55dc36bb2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:08.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.367+0000 7f6d06ffd640 1 --2- 172.21.15.16:0/2356545134 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dc36bb0240 0x55dc36bb2630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:08.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.367+0000 7f6d06ffd640 1 -- 172.21.15.16:0/2356545134 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc36ba5060 msgr2=0x55dc36ba5440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:08.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.367+0000 7f6d06ffd640 1 --2- 172.21.15.16:0/2356545134 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc36ba5060 0x55dc36ba5440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:08.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.367+0000 7f6d06ffd640 1 -- 172.21.15.16:0/2356545134 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc36aa2d40 con 0x55dc36ba5980 2024-09-12T02:10:08.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.368+0000 7f6d06ffd640 1 --2- 172.21.15.16:0/2356545134 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc36ba5980 0x55dc36bafd00 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f6cf800c940 tx=0x7f6cf8004a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30c2bcbdbee07e3d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:10:08.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.368+0000 7f6d077fe640 1 --2- 172.21.15.16:0/2747332876 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f6cf00a2c00 0x7f6cf00a50c0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f6d00005c40 tx=0x7f6d00009280 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:10:08.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.368+0000 7f6ced7fa640 1 -- 172.21.15.16:0/2356545134 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cf801d070 con 0x55dc36ba5980 2024-09-12T02:10:08.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.368+0000 7f6ced7fa640 1 -- 172.21.15.16:0/2356545134 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6cf809bd00 con 0x55dc36ba5980 2024-09-12T02:10:08.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.368+0000 7f6ced7fa640 1 -- 172.21.15.16:0/2356545134 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cf8097050 con 0x55dc36ba5980 2024-09-12T02:10:08.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.368+0000 7f6d0f99e740 1 -- 172.21.15.16:0/2356545134 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc36ba5980 msgr2=0x55dc36bafd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:08.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.368+0000 7f6d0f99e740 1 --2- 172.21.15.16:0/2356545134 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc36ba5980 0x55dc36bafd00 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f6cf800c940 tx=0x7f6cf8004a20 comp rx=0 tx=0).stop 2024-09-12T02:10:08.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.369+0000 7f6d0f99e740 1 -- 172.21.15.16:0/2356545134 shutdown_connections 2024-09-12T02:10:08.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.369+0000 7f6d0f99e740 1 --2- 172.21.15.16:0/2356545134 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dc36bb0240 0x55dc36bb2630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:08.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.369+0000 7f6d0f99e740 1 --2- 172.21.15.16:0/2356545134 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc36ba5980 0x55dc36bafd00 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:08.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.369+0000 7f6d0f99e740 1 --2- 172.21.15.16:0/2356545134 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc36ba5060 0x55dc36ba5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:08.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.369+0000 7f6d0f99e740 1 -- 172.21.15.16:0/2356545134 >> 172.21.15.16:0/2356545134 conn(0x55dc36ba15f0 msgr2=0x55dc36ba19d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:10:08.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.369+0000 7f6d0f99e740 1 -- 172.21.15.16:0/2356545134 shutdown_connections 2024-09-12T02:10:08.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.369+0000 7f6d0f99e740 1 -- 172.21.15.16:0/2356545134 wait complete. 2024-09-12T02:10:08.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.369+0000 7f6d0f99e740 1 -- start start 2024-09-12T02:10:08.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.369+0000 7f6d0f99e740 1 Processor -- start 2024-09-12T02:10:08.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.369+0000 7f6d0f99e740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dc36ba5060 0x55dc36bad140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:08.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.369+0000 7f6d0f99e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc36ba5980 0x55dc36bad680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:08.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.369+0000 7f6d0f99e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc36bb0240 0x55dc36badbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:08.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.369+0000 7f6d0f99e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55dc36bb2db0 con 0x55dc36bb0240 2024-09-12T02:10:08.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.369+0000 7f6d0f99e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55dc36a92f10 con 0x55dc36ba5980 2024-09-12T02:10:08.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.369+0000 7f6d0f99e740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55dc36bb2f30 con 0x55dc36ba5060 2024-09-12T02:10:08.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.369+0000 7f6d0f99e740 10 client.0 _ll_register_callbacks cb 0x55dc36ab6db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:10:08.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.369+0000 7f6d077fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc36bb0240 0x55dc36badbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:08.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.369+0000 7f6d077fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc36bb0240 0x55dc36badbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57312/0 (socket says 172.21.15.16:57312) 2024-09-12T02:10:08.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.369+0000 7f6d077fe640 1 -- 172.21.15.16:0/4147970349 learned_addr learned my addr 172.21.15.16:0/4147970349 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:10:08.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.370+0000 7f6d06ffd640 1 --2- 172.21.15.16:0/4147970349 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dc36ba5060 0x55dc36bad140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:08.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.370+0000 7f6d067fc640 1 --2- 172.21.15.16:0/4147970349 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc36ba5980 0x55dc36bad680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:08.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.370+0000 7f6d077fe640 1 -- 172.21.15.16:0/4147970349 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dc36ba5060 msgr2=0x55dc36bad140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:08.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.370+0000 7f6d077fe640 1 --2- 172.21.15.16:0/4147970349 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dc36ba5060 0x55dc36bad140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:08.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.370+0000 7f6d077fe640 1 -- 172.21.15.16:0/4147970349 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc36ba5980 msgr2=0x55dc36bad680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:08.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.370+0000 7f6d077fe640 1 --2- 172.21.15.16:0/4147970349 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc36ba5980 0x55dc36bad680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:08.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.370+0000 7f6d077fe640 1 -- 172.21.15.16:0/4147970349 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc36819580 con 0x55dc36bb0240 2024-09-12T02:10:08.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.370+0000 7f6d06ffd640 1 --2- 172.21.15.16:0/4147970349 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dc36ba5060 0x55dc36bad140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:10:08.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.370+0000 7f6d077fe640 1 --2- 172.21.15.16:0/4147970349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc36bb0240 0x55dc36badbc0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f6d00012fb0 tx=0x7f6d00004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=201434109c7c2279 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:10:08.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.370+0000 7f6ceaff5640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:10:08.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.370+0000 7f6ceaff5640 1 -- 172.21.15.16:0/4147970349 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d00025070 con 0x55dc36bb0240 2024-09-12T02:10:08.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.370+0000 7f6d0f99e740 1 -- 172.21.15.16:0/4147970349 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6cf809bd00 con 0x55dc36bb0240 2024-09-12T02:10:08.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.370+0000 7f6ceaff5640 1 -- 172.21.15.16:0/4147970349 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f6d00010ce0 con 0x55dc36bb0240 2024-09-12T02:10:08.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.370+0000 7f6ceaff5640 1 -- 172.21.15.16:0/4147970349 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6d0002a460 con 0x55dc36bb0240 2024-09-12T02:10:08.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.370+0000 7f6d0f99e740 10 client.17949 fetch_fsmap learned FSMap version 25 2024-09-12T02:10:08.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.370+0000 7f6d0f99e740 1 -- 172.21.15.16:0/4147970349 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55dc368e3960 con 0x55dc36bb0240 2024-09-12T02:10:08.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.371+0000 7f6ceaff5640 1 -- 172.21.15.16:0/4147970349 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6cf809bd00 con 0x55dc36bb0240 2024-09-12T02:10:08.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.371+0000 7f6ceaff5640 1 -- 172.21.15.16:0/4147970349 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f6d0001e040 con 0x55dc36bb0240 2024-09-12T02:10:08.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.371+0000 7f6d0f99e740 10 client.17949 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:10:08.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.371+0000 7f6d0f99e740 10 client.17949 resolve_mds: validated gid 24482 aka daemon mds.b 2024-09-12T02:10:08.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.371+0000 7f6d0f99e740 20 client.17949 populate_metadata read hostname 'smithi016' 2024-09-12T02:10:08.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.371+0000 7f6d0f99e740 1 --2- 172.21.15.16:0/4147970349 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55dc36bbfd60 0x55dc36bc2150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:08.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.371+0000 7f6d0f99e740 4 client.17949 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:10:08.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.371+0000 7f6d0f99e740 1 -- 172.21.15.16:0/4147970349 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dc36b17b90 con 0x55dc36bbfd60 2024-09-12T02:10:08.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.371+0000 7f6d06ffd640 1 --2- 172.21.15.16:0/4147970349 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55dc36bbfd60 0x55dc36bc2150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:08.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.371+0000 7f6d06ffd640 1 --2- 172.21.15.16:0/4147970349 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55dc36bbfd60 0x55dc36bc2150 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d4357b7eb5ae2db server_cookie=3bbd8b7a432ac17d in_seq=0 out_seq=0 2024-09-12T02:10:08.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.371+0000 7f6ceaff5640 10 client.17949 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:10:08.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.372+0000 7f6ceaff5640 1 -- 172.21.15.16:0/4147970349 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55dc36b17b90 con 0x55dc36bbfd60 2024-09-12T02:10:08.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.372+0000 7f6ceaff5640 10 client.17949 handle_command_reply: tid=0 2024-09-12T02:10:08.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.372+0000 7f6d0f99e740 1 client.17949 shutdown 2024-09-12T02:10:08.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.372+0000 7f6d0f99e740 1 -- 172.21.15.16:0/4147970349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc36bb0240 msgr2=0x55dc36badbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:08.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.372+0000 7f6d0f99e740 1 --2- 172.21.15.16:0/4147970349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc36bb0240 0x55dc36badbc0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f6d00012fb0 tx=0x7f6d00004a40 comp rx=0 tx=0).stop 2024-09-12T02:10:08.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.372+0000 7f6d0f99e740 1 -- 172.21.15.16:0/4147970349 shutdown_connections 2024-09-12T02:10:08.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.372+0000 7f6d0f99e740 1 --2- 172.21.15.16:0/4147970349 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55dc36bbfd60 0x55dc36bc2150 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:08.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.372+0000 7f6ceaff5640 0 client.17949 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:10:08.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.372+0000 7f6d0f99e740 1 --2- 172.21.15.16:0/4147970349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc36bb0240 0x55dc36badbc0 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:08.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.372+0000 7f6d0f99e740 1 --2- 172.21.15.16:0/4147970349 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc36ba5980 0x55dc36bad680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:08.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.372+0000 7f6d0f99e740 1 --2- 172.21.15.16:0/4147970349 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dc36ba5060 0x55dc36bad140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:08.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.372+0000 7f6d0f99e740 1 -- 172.21.15.16:0/4147970349 >> 172.21.15.16:0/4147970349 conn(0x55dc36ba15f0 msgr2=0x55dc36ba7320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:10:08.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.373+0000 7f6d0f99e740 1 -- 172.21.15.16:0/4147970349 shutdown_connections 2024-09-12T02:10:08.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.373+0000 7f6d0f99e740 1 -- 172.21.15.16:0/4147970349 wait complete. 2024-09-12T02:10:08.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.373+0000 7f6d0f99e740 20 client.17949 trim_cache size 0 max 16384 2024-09-12T02:10:08.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.390+0000 7f6d0f99e740 1 Processor -- start 2024-09-12T02:10:08.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.390+0000 7f6d0f99e740 1 -- start start 2024-09-12T02:10:08.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.390+0000 7f6d0f99e740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dc36ba5060 0x55dc36ba7b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:08.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.390+0000 7f6d0f99e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc36ba5980 0x55dc36ba0640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:08.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.390+0000 7f6d0f99e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc36bb0240 0x55dc36bc7a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:08.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.390+0000 7f6d0f99e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6d081b1f80 con 0x55dc36bb0240 2024-09-12T02:10:08.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.390+0000 7f6d0f99e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6d081af850 con 0x55dc36ba5980 2024-09-12T02:10:08.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.390+0000 7f6d0f99e740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f6d081ad6a0 con 0x55dc36ba5060 2024-09-12T02:10:08.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.390+0000 7f6d077fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc36bb0240 0x55dc36bc7a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:08.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.390+0000 7f6d077fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc36bb0240 0x55dc36bc7a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57326/0 (socket says 172.21.15.16:57326) 2024-09-12T02:10:08.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.390+0000 7f6d077fe640 1 -- 172.21.15.16:0/1454940773 learned_addr learned my addr 172.21.15.16:0/1454940773 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:10:08.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.391+0000 7f6d06ffd640 1 --2- 172.21.15.16:0/1454940773 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dc36ba5060 0x55dc36ba7b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:08.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.391+0000 7f6d067fc640 1 --2- 172.21.15.16:0/1454940773 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc36ba5980 0x55dc36ba0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:08.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.391+0000 7f6d077fe640 1 -- 172.21.15.16:0/1454940773 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dc36ba5060 msgr2=0x55dc36ba7b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:08.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.391+0000 7f6d077fe640 1 --2- 172.21.15.16:0/1454940773 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dc36ba5060 0x55dc36ba7b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:08.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.391+0000 7f6d077fe640 1 -- 172.21.15.16:0/1454940773 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc36ba5980 msgr2=0x55dc36ba0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:08.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.391+0000 7f6d077fe640 1 --2- 172.21.15.16:0/1454940773 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc36ba5980 0x55dc36ba0640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:08.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.391+0000 7f6d077fe640 1 -- 172.21.15.16:0/1454940773 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc36ba1420 con 0x55dc36bb0240 2024-09-12T02:10:08.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.391+0000 7f6d077fe640 1 --2- 172.21.15.16:0/1454940773 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc36bb0240 0x55dc36bc7a00 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f6d00006010 tx=0x7f6d00010820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed8eb7cfb354164a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:10:08.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.391+0000 7f6cecff9640 1 -- 172.21.15.16:0/1454940773 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d00025040 con 0x55dc36bb0240 2024-09-12T02:10:08.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.391+0000 7f6cecff9640 1 -- 172.21.15.16:0/1454940773 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d00039020 con 0x55dc36bb0240 2024-09-12T02:10:08.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.391+0000 7f6cecff9640 1 -- 172.21.15.16:0/1454940773 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d00021ac0 con 0x55dc36bb0240 2024-09-12T02:10:08.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.392+0000 7f6d0f99e740 1 -- 172.21.15.16:0/1454940773 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc36bb0240 msgr2=0x55dc36bc7a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:08.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.392+0000 7f6d0f99e740 1 --2- 172.21.15.16:0/1454940773 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc36bb0240 0x55dc36bc7a00 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f6d00006010 tx=0x7f6d00010820 comp rx=0 tx=0).stop 2024-09-12T02:10:08.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.392+0000 7f6d0f99e740 1 -- 172.21.15.16:0/1454940773 shutdown_connections 2024-09-12T02:10:08.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.392+0000 7f6d0f99e740 1 --2- 172.21.15.16:0/1454940773 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc36bb0240 0x55dc36bc7a00 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:08.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.392+0000 7f6d0f99e740 1 --2- 172.21.15.16:0/1454940773 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc36ba5980 0x55dc36ba0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:08.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.392+0000 7f6d0f99e740 1 --2- 172.21.15.16:0/1454940773 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dc36ba5060 0x55dc36ba7b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:08.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.392+0000 7f6d0f99e740 1 -- 172.21.15.16:0/1454940773 >> 172.21.15.16:0/1454940773 conn(0x55dc36ba15f0 msgr2=0x55dc36b9ff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:10:08.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.392+0000 7f6d0f99e740 1 -- 172.21.15.16:0/1454940773 shutdown_connections 2024-09-12T02:10:08.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.392+0000 7f6d0f99e740 1 -- 172.21.15.16:0/1454940773 wait complete. 2024-09-12T02:10:08.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.392+0000 7f6d0f99e740 1 -- start start 2024-09-12T02:10:08.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.392+0000 7f6d0f99e740 1 Processor -- start 2024-09-12T02:10:08.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.392+0000 7f6d0f99e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc36ba5060 0x55dc36bb46f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:08.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.392+0000 7f6d0f99e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc36ba5980 0x55dc36bb4c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:08.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.392+0000 7f6d0f99e740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dc36bb0240 0x55dc36c299d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:08.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.393+0000 7f6d0f99e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6d081add10 con 0x55dc36ba5980 2024-09-12T02:10:08.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.393+0000 7f6d0f99e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6d081af180 con 0x55dc36ba5060 2024-09-12T02:10:08.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.393+0000 7f6d0f99e740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f6d081ae3e0 con 0x55dc36bb0240 2024-09-12T02:10:08.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.393+0000 7f6d0f99e740 10 client.0 _ll_register_callbacks cb 0x7f6d00012470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:10:08.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.393+0000 7f6d067fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc36ba5980 0x55dc36bb4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:08.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.393+0000 7f6d067fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc36ba5980 0x55dc36bb4c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57338/0 (socket says 172.21.15.16:57338) 2024-09-12T02:10:08.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.393+0000 7f6d067fc640 1 -- 172.21.15.16:0/527630638 learned_addr learned my addr 172.21.15.16:0/527630638 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:10:08.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.393+0000 7f6d06ffd640 1 --2- 172.21.15.16:0/527630638 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc36ba5060 0x55dc36bb46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:08.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.393+0000 7f6d077fe640 1 --2- 172.21.15.16:0/527630638 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dc36bb0240 0x55dc36c299d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:08.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.393+0000 7f6d067fc640 1 -- 172.21.15.16:0/527630638 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dc36bb0240 msgr2=0x55dc36c299d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:08.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.393+0000 7f6d067fc640 1 --2- 172.21.15.16:0/527630638 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dc36bb0240 0x55dc36c299d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:08.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.393+0000 7f6d067fc640 1 -- 172.21.15.16:0/527630638 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc36ba5060 msgr2=0x55dc36bb46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:08.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.393+0000 7f6d067fc640 1 --2- 172.21.15.16:0/527630638 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc36ba5060 0x55dc36bb46f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:08.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.393+0000 7f6d067fc640 1 -- 172.21.15.16:0/527630638 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc36c2a010 con 0x55dc36ba5980 2024-09-12T02:10:08.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.393+0000 7f6d06ffd640 1 --2- 172.21.15.16:0/527630638 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc36ba5060 0x55dc36bb46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:10:08.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.393+0000 7f6d067fc640 1 --2- 172.21.15.16:0/527630638 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc36ba5980 0x55dc36bb4c30 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f6cfc02a930 tx=0x7f6cfc031af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4504b95cde3937a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:10:08.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.393+0000 7f6ceaff5640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:10:08.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.393+0000 7f6ceaff5640 1 -- 172.21.15.16:0/527630638 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cfc035040 con 0x55dc36ba5980 2024-09-12T02:10:08.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.394+0000 7f6ceaff5640 1 -- 172.21.15.16:0/527630638 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f6cfc0035f0 con 0x55dc36ba5980 2024-09-12T02:10:08.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.394+0000 7f6d0f99e740 1 -- 172.21.15.16:0/527630638 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d00039020 con 0x55dc36ba5980 2024-09-12T02:10:08.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.394+0000 7f6ceaff5640 1 -- 172.21.15.16:0/527630638 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6cfc0044d0 con 0x55dc36ba5980 2024-09-12T02:10:08.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.394+0000 7f6d0f99e740 10 client.17955 fetch_fsmap learned FSMap version 25 2024-09-12T02:10:08.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.394+0000 7f6d0f99e740 1 -- 172.21.15.16:0/527630638 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55dc36c2b360 con 0x55dc36ba5980 2024-09-12T02:10:08.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.394+0000 7f6ceaff5640 1 -- 172.21.15.16:0/527630638 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d00039020 con 0x55dc36ba5980 2024-09-12T02:10:08.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.394+0000 7f6ceaff5640 1 -- 172.21.15.16:0/527630638 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f6cfc044070 con 0x55dc36ba5980 2024-09-12T02:10:08.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.394+0000 7f6d0f99e740 10 client.17955 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:10:08.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.394+0000 7f6d0f99e740 10 client.17955 resolve_mds: validated gid 24482 aka daemon mds.b 2024-09-12T02:10:08.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.394+0000 7f6d0f99e740 20 client.17955 populate_metadata read hostname 'smithi016' 2024-09-12T02:10:08.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.394+0000 7f6d0f99e740 1 --2- 172.21.15.16:0/527630638 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55dc36c2bb40 0x55dc36c2df60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:08.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.394+0000 7f6d0f99e740 4 client.17955 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "f9fc3d8a", "release": true, "await": true}] 2024-09-12T02:10:08.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.394+0000 7f6d0f99e740 1 -- 172.21.15.16:0/527630638 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "quiesce db", "set_id": "f9fc3d8a", "release": true, "await": true}) -- 0x55dc36b120a0 con 0x55dc36c2bb40 2024-09-12T02:10:08.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.394+0000 7f6d06ffd640 1 --2- 172.21.15.16:0/527630638 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55dc36c2bb40 0x55dc36c2df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:08.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.395+0000 7f6d06ffd640 1 --2- 172.21.15.16:0/527630638 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55dc36c2bb40 0x55dc36c2df60 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b80af17b4cbeca2c server_cookie=2fce97f4897abe83 in_seq=0 out_seq=0 2024-09-12T02:10:08.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.395+0000 7f6ceaff5640 10 client.17955 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:10:08.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.398+0000 7f6ceaff5640 1 -- 172.21.15.16:0/527630638 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55dc36b120a0 con 0x55dc36c2bb40 2024-09-12T02:10:08.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.398+0000 7f6ceaff5640 10 client.17955 handle_command_reply: tid=0 2024-09-12T02:10:08.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.398+0000 7f6d0f99e740 1 client.17955 shutdown 2024-09-12T02:10:08.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.398+0000 7f6d0f99e740 1 -- 172.21.15.16:0/527630638 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc36ba5980 msgr2=0x55dc36bb4c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:08.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.398+0000 7f6d0f99e740 1 --2- 172.21.15.16:0/527630638 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc36ba5980 0x55dc36bb4c30 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f6cfc02a930 tx=0x7f6cfc031af0 comp rx=0 tx=0).stop 2024-09-12T02:10:08.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.398+0000 7f6d0f99e740 1 -- 172.21.15.16:0/527630638 shutdown_connections 2024-09-12T02:10:08.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.398+0000 7f6d0f99e740 1 --2- 172.21.15.16:0/527630638 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55dc36c2bb40 0x55dc36c2df60 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:08.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.398+0000 7f6d0f99e740 1 --2- 172.21.15.16:0/527630638 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dc36bb0240 0x55dc36c299d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:08.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.398+0000 7f6ceaff5640 0 client.17955 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:10:08.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.398+0000 7f6d0f99e740 1 --2- 172.21.15.16:0/527630638 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc36ba5980 0x55dc36bb4c30 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:08.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.398+0000 7f6d0f99e740 1 --2- 172.21.15.16:0/527630638 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc36ba5060 0x55dc36bb46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:08.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.398+0000 7f6d0f99e740 1 -- 172.21.15.16:0/527630638 >> 172.21.15.16:0/527630638 conn(0x55dc36ba15f0 msgr2=0x55dc36bab5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:10:08.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.398+0000 7f6d0f99e740 1 -- 172.21.15.16:0/527630638 shutdown_connections 2024-09-12T02:10:08.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.398+0000 7f6d0f99e740 1 -- 172.21.15.16:0/527630638 wait complete. 2024-09-12T02:10:08.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.398+0000 7f6d0f99e740 20 client.17955 trim_cache size 0 max 16384 2024-09-12T02:10:08.402 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T02:10:08.402 INFO:teuthology.orchestra.run.smithi016.stdout: "epoch": 25, 2024-09-12T02:10:08.402 INFO:teuthology.orchestra.run.smithi016.stdout: "leader": 24482, 2024-09-12T02:10:08.402 INFO:teuthology.orchestra.run.smithi016.stdout: "set_version": 36, 2024-09-12T02:10:08.402 INFO:teuthology.orchestra.run.smithi016.stdout: "sets": { 2024-09-12T02:10:08.402 INFO:teuthology.orchestra.run.smithi016.stdout: "f9fc3d8a": { 2024-09-12T02:10:08.402 INFO:teuthology.orchestra.run.smithi016.stdout: "version": 36, 2024-09-12T02:10:08.402 INFO:teuthology.orchestra.run.smithi016.stdout: "age_ref": 0.0, 2024-09-12T02:10:08.403 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-12T02:10:08.403 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RELEASED", 2024-09-12T02:10:08.403 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-12T02:10:08.403 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:10:08.403 INFO:teuthology.orchestra.run.smithi016.stdout: "timeout": 90.0, 2024-09-12T02:10:08.403 INFO:teuthology.orchestra.run.smithi016.stdout: "expiration": 139.4, 2024-09-12T02:10:08.403 INFO:teuthology.orchestra.run.smithi016.stdout: "members": { 2024-09-12T02:10:08.403 INFO:teuthology.orchestra.run.smithi016.stdout: "file:/": { 2024-09-12T02:10:08.403 INFO:teuthology.orchestra.run.smithi016.stdout: "excluded": false, 2024-09-12T02:10:08.403 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-12T02:10:08.404 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RELEASED", 2024-09-12T02:10:08.404 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-12T02:10:08.404 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:10:08.404 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:10:08.404 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:10:08.404 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:10:08.404 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:10:08.404 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T02:10:08.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.398+0000 7f6d0d5c5640 1 -- 172.21.15.16:0/2747332876 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f6cf00a2c00 msgr2=0x7f6cf00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:08.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.398+0000 7f6d0d5c5640 1 --2- 172.21.15.16:0/2747332876 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f6cf00a2c00 0x7f6cf00a50c0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f6d00005c40 tx=0x7f6d00009280 comp rx=0 tx=0).stop 2024-09-12T02:10:08.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.398+0000 7f6d0d5c5640 1 -- 172.21.15.16:0/2747332876 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d08101de0 msgr2=0x7f6d081984b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:08.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.398+0000 7f6d0d5c5640 1 --2- 172.21.15.16:0/2747332876 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d08101de0 0x7f6d081984b0 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f6cf800c970 tx=0x7f6cf800ce30 comp rx=0 tx=0).stop 2024-09-12T02:10:08.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.399+0000 7f6d0d5c5640 1 -- 172.21.15.16:0/2747332876 shutdown_connections 2024-09-12T02:10:08.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.399+0000 7f6d0d5c5640 1 --2- 172.21.15.16:0/2747332876 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f6cf00a2c00 0x7f6cf00a50c0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:08.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.399+0000 7f6d0d5c5640 1 --2- 172.21.15.16:0/2747332876 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6d08106f10 0x7f6d0819cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:08.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.399+0000 7f6d0d5c5640 1 --2- 172.21.15.16:0/2747332876 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d08102790 0x7f6d081989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:08.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.399+0000 7f6d0d5c5640 1 --2- 172.21.15.16:0/2747332876 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d08101de0 0x7f6d081984b0 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:08.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.399+0000 7f6d0d5c5640 1 -- 172.21.15.16:0/2747332876 >> 172.21.15.16:0/2747332876 conn(0x7f6d080fd8c0 msgr2=0x7f6d080febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:10:08.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.399+0000 7f6d0d5c5640 1 -- 172.21.15.16:0/2747332876 shutdown_connections 2024-09-12T02:10:08.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:08.399+0000 7f6d0d5c5640 1 -- 172.21.15.16:0/2747332876 wait complete. 2024-09-12T02:10:08.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:10:08 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:10:08.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:10:08 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:10:08.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:10:08 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T02:10:08.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:10:08 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:10:08.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:10:08 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:10:08.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:10:08 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T02:10:08.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:10:08 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:10:08.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:10:08 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:10:08.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:10:08 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T02:10:09.030 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'f9fc3d8a', total seconds elapsed: 22.6 2024-09-12T02:10:10.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:10:09 smithi184 ceph-mon[35593]: pgmap v582: 97 pgs: 97 active+clean; 4.2 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 1.5 KiB/s wr, 7 op/s 2024-09-12T02:10:10.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:10:09 smithi033 ceph-mon[35332]: pgmap v582: 97 pgs: 97 active+clean; 4.2 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 1.5 KiB/s wr, 7 op/s 2024-09-12T02:10:10.183 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:10:09 smithi016 ceph-mon[30947]: pgmap v582: 97 pgs: 97 active+clean; 4.2 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 1.5 KiB/s wr, 7 op/s 2024-09-12T02:10:11.786 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T02:10:12.059 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:10:12.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:10:11 smithi184 ceph-mon[35593]: pgmap v583: 97 pgs: 97 active+clean; 4.2 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 2.6 KiB/s wr, 9 op/s 2024-09-12T02:10:12.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:10:11 smithi033 ceph-mon[35332]: pgmap v583: 97 pgs: 97 active+clean; 4.2 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 2.6 KiB/s wr, 9 op/s 2024-09-12T02:10:12.180 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:10:11 smithi016 ceph-mon[30947]: pgmap v583: 97 pgs: 97 active+clean; 4.2 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 2.6 KiB/s wr, 9 op/s 2024-09-12T02:10:12.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.521+0000 7fc34a5a2640 1 -- 172.21.15.16:0/3645258476 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc344106a80 msgr2=0x7fc344106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:12.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.521+0000 7fc34a5a2640 1 --2- 172.21.15.16:0/3645258476 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc344106a80 0x7fc344106e60 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7fc334009930 tx=0x7fc33402a420 comp rx=0 tx=0).stop 2024-09-12T02:10:12.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.521+0000 7fc34a5a2640 1 -- 172.21.15.16:0/3645258476 shutdown_connections 2024-09-12T02:10:12.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.521+0000 7fc34a5a2640 1 --2- 172.21.15.16:0/3645258476 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc344101420 0x7fc34410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:12.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.521+0000 7fc34a5a2640 1 --2- 172.21.15.16:0/3645258476 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc344100a80 0x7fc344100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:12.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.521+0000 7fc34a5a2640 1 --2- 172.21.15.16:0/3645258476 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc344106a80 0x7fc344106e60 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:12.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.521+0000 7fc34a5a2640 1 -- 172.21.15.16:0/3645258476 >> 172.21.15.16:0/3645258476 conn(0x7fc3440fc7a0 msgr2=0x7fc3440febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:10:12.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.521+0000 7fc34a5a2640 1 -- 172.21.15.16:0/3645258476 shutdown_connections 2024-09-12T02:10:12.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.521+0000 7fc34a5a2640 1 -- 172.21.15.16:0/3645258476 wait complete. 2024-09-12T02:10:12.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.522+0000 7fc34a5a2640 1 Processor -- start 2024-09-12T02:10:12.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.522+0000 7fc34a5a2640 1 -- start start 2024-09-12T02:10:12.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.522+0000 7fc34a5a2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc344100a80 0x7fc344102a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:12.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.522+0000 7fc34a5a2640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc344101420 0x7fc344102f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:12.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.522+0000 7fc34a5a2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc344106a80 0x7fc3441034b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:12.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.522+0000 7fc34a5a2640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc344110ea0 con 0x7fc344106a80 2024-09-12T02:10:12.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.522+0000 7fc34a5a2640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc344110d20 con 0x7fc344100a80 2024-09-12T02:10:12.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.522+0000 7fc34a5a2640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc344111020 con 0x7fc344101420 2024-09-12T02:10:12.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.522+0000 7fc348b3f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc344106a80 0x7fc3441034b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:12.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.522+0000 7fc348b3f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc344106a80 0x7fc3441034b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45744/0 (socket says 172.21.15.16:45744) 2024-09-12T02:10:12.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.522+0000 7fc348b3f640 1 -- 172.21.15.16:0/1998227644 learned_addr learned my addr 172.21.15.16:0/1998227644 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:10:12.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.523+0000 7fc343fff640 1 --2- 172.21.15.16:0/1998227644 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc344100a80 0x7fc344102a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:12.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.523+0000 7fc3437fe640 1 --2- 172.21.15.16:0/1998227644 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc344101420 0x7fc344102f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:12.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.523+0000 7fc348b3f640 1 -- 172.21.15.16:0/1998227644 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc344101420 msgr2=0x7fc344102f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:12.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.523+0000 7fc348b3f640 1 --2- 172.21.15.16:0/1998227644 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc344101420 0x7fc344102f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:12.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.523+0000 7fc348b3f640 1 -- 172.21.15.16:0/1998227644 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc344100a80 msgr2=0x7fc344102a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:12.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.523+0000 7fc348b3f640 1 --2- 172.21.15.16:0/1998227644 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc344100a80 0x7fc344102a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:12.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.523+0000 7fc348b3f640 1 -- 172.21.15.16:0/1998227644 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3441a6990 con 0x7fc344106a80 2024-09-12T02:10:12.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.523+0000 7fc348b3f640 1 --2- 172.21.15.16:0/1998227644 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc344106a80 0x7fc3441034b0 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7fc33c00b970 tx=0x7fc33c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:10:12.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.523+0000 7fc3417fa640 1 -- 172.21.15.16:0/1998227644 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc33c019070 con 0x7fc344106a80 2024-09-12T02:10:12.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.523+0000 7fc3417fa640 1 -- 172.21.15.16:0/1998227644 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc33c00cb40 con 0x7fc344106a80 2024-09-12T02:10:12.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.523+0000 7fc34a5a2640 1 -- 172.21.15.16:0/1998227644 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3441a6c80 con 0x7fc344106a80 2024-09-12T02:10:12.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.523+0000 7fc3417fa640 1 -- 172.21.15.16:0/1998227644 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc33c007650 con 0x7fc344106a80 2024-09-12T02:10:12.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.523+0000 7fc34a5a2640 1 -- 172.21.15.16:0/1998227644 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3441a71c0 con 0x7fc344106a80 2024-09-12T02:10:12.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.526+0000 7fc3417fa640 1 -- 172.21.15.16:0/1998227644 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fc33c01c070 con 0x7fc344106a80 2024-09-12T02:10:12.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.526+0000 7fc34c97b740 1 Processor -- start 2024-09-12T02:10:12.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.526+0000 7fc34c97b740 1 -- start start 2024-09-12T02:10:12.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.526+0000 7fc34c97b740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560ea4bd34a0 0x560ea4bd3880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:12.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.527+0000 7fc34c97b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ea4bd3dc0 0x560ea4bde140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:12.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.527+0000 7fc34c97b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560ea4bde680 0x560ea4be0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:12.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.527+0000 7fc34c97b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x560ea4b392e0 con 0x560ea4bde680 2024-09-12T02:10:12.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.527+0000 7fc34c97b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x560ea4ae0ce0 con 0x560ea4bd3dc0 2024-09-12T02:10:12.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.527+0000 7fc34c97b740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x560ea4b352a0 con 0x560ea4bd34a0 2024-09-12T02:10:12.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.527+0000 7fc3437fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560ea4bde680 0x560ea4be0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:12.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.527+0000 7fc343fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ea4bd3dc0 0x560ea4bde140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:12.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.527+0000 7fc3437fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560ea4bde680 0x560ea4be0a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45750/0 (socket says 172.21.15.16:45750) 2024-09-12T02:10:12.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.527+0000 7fc3437fe640 1 -- 172.21.15.16:0/2705350466 learned_addr learned my addr 172.21.15.16:0/2705350466 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:10:12.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.527+0000 7fc3437fe640 1 --2- 172.21.15.16:0/2705350466 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560ea4bd34a0 0x560ea4bd3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:12.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.527+0000 7fc3417fa640 1 --2- 172.21.15.16:0/1998227644 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc32c0a2b80 0x7fc32c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:12.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.527+0000 7fc348b3f640 1 --2- 172.21.15.16:0/1998227644 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc32c0a2b80 0x7fc32c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:12.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.527+0000 7fc343fff640 1 -- 172.21.15.16:0/2705350466 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560ea4bd34a0 msgr2=0x560ea4bd3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:12.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.527+0000 7fc343fff640 1 --2- 172.21.15.16:0/2705350466 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560ea4bd34a0 0x560ea4bd3880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:12.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.527+0000 7fc343fff640 1 -- 172.21.15.16:0/2705350466 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560ea4bde680 msgr2=0x560ea4be0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:12.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.527+0000 7fc343fff640 1 --2- 172.21.15.16:0/2705350466 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560ea4bde680 0x560ea4be0a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:12.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.527+0000 7fc343fff640 1 -- 172.21.15.16:0/2705350466 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ea4ac5440 con 0x560ea4bd3dc0 2024-09-12T02:10:12.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.527+0000 7fc3437fe640 1 --2- 172.21.15.16:0/2705350466 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560ea4bde680 0x560ea4be0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:10:12.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.527+0000 7fc3417fa640 1 -- 172.21.15.16:0/1998227644 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fc33c0ce680 con 0x7fc344106a80 2024-09-12T02:10:12.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.527+0000 7fc343fff640 1 --2- 172.21.15.16:0/2705350466 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ea4bd3dc0 0x560ea4bde140 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc334009900 tx=0x7fc334032ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cc9ed7ab6e4a7c74 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:10:12.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.528+0000 7fc348b3f640 1 --2- 172.21.15.16:0/1998227644 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc32c0a2b80 0x7fc32c0a5040 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fc33c096bc0 tx=0x7fc33c095910 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:10:12.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.528+0000 7fc32a7fc640 1 -- 172.21.15.16:0/2705350466 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc33403f070 con 0x560ea4bd3dc0 2024-09-12T02:10:12.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.528+0000 7fc32a7fc640 1 -- 172.21.15.16:0/2705350466 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3340317b0 con 0x560ea4bd3dc0 2024-09-12T02:10:12.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.528+0000 7fc32a7fc640 1 -- 172.21.15.16:0/2705350466 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc334035070 con 0x560ea4bd3dc0 2024-09-12T02:10:12.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.528+0000 7fc34c97b740 1 -- 172.21.15.16:0/2705350466 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ea4bd3dc0 msgr2=0x560ea4bde140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:12.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.528+0000 7fc34c97b740 1 --2- 172.21.15.16:0/2705350466 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ea4bd3dc0 0x560ea4bde140 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc334009900 tx=0x7fc334032ce0 comp rx=0 tx=0).stop 2024-09-12T02:10:12.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.528+0000 7fc34c97b740 1 -- 172.21.15.16:0/2705350466 shutdown_connections 2024-09-12T02:10:12.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.528+0000 7fc34c97b740 1 --2- 172.21.15.16:0/2705350466 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560ea4bde680 0x560ea4be0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:12.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.528+0000 7fc34c97b740 1 --2- 172.21.15.16:0/2705350466 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ea4bd3dc0 0x560ea4bde140 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:12.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.528+0000 7fc34c97b740 1 --2- 172.21.15.16:0/2705350466 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560ea4bd34a0 0x560ea4bd3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:12.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.528+0000 7fc34c97b740 1 -- 172.21.15.16:0/2705350466 >> 172.21.15.16:0/2705350466 conn(0x560ea4bcfa30 msgr2=0x560ea4bcfe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:10:12.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.528+0000 7fc34c97b740 1 -- 172.21.15.16:0/2705350466 shutdown_connections 2024-09-12T02:10:12.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.529+0000 7fc34c97b740 1 -- 172.21.15.16:0/2705350466 wait complete. 2024-09-12T02:10:12.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.529+0000 7fc34c97b740 1 -- start start 2024-09-12T02:10:12.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.529+0000 7fc34c97b740 1 Processor -- start 2024-09-12T02:10:12.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.529+0000 7fc34c97b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ea4bd34a0 0x560ea4bddf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:12.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.529+0000 7fc34c97b740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560ea4bd3dc0 0x560ea4bdb210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:12.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.529+0000 7fc34c97b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560ea4bde680 0x560ea4bdb750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:12.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.529+0000 7fc34c97b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x560ea4be1250 con 0x560ea4bde680 2024-09-12T02:10:12.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.529+0000 7fc34c97b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x560ea4af4050 con 0x560ea4bd34a0 2024-09-12T02:10:12.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.529+0000 7fc34c97b740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x560ea4be13d0 con 0x560ea4bd3dc0 2024-09-12T02:10:12.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.529+0000 7fc34c97b740 10 client.0 _ll_register_callbacks cb 0x560ea4b34160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:10:12.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.529+0000 7fc3437fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560ea4bde680 0x560ea4bdb750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:12.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.529+0000 7fc3437fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560ea4bde680 0x560ea4bdb750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45756/0 (socket says 172.21.15.16:45756) 2024-09-12T02:10:12.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.530+0000 7fc3437fe640 1 -- 172.21.15.16:0/4249400604 learned_addr learned my addr 172.21.15.16:0/4249400604 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:10:12.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.530+0000 7fc3437fe640 1 --2- 172.21.15.16:0/4249400604 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ea4bd34a0 0x560ea4bddf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:12.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.530+0000 7fc343fff640 1 --2- 172.21.15.16:0/4249400604 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560ea4bd3dc0 0x560ea4bdb210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:12.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.530+0000 7fc3437fe640 1 -- 172.21.15.16:0/4249400604 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560ea4bd3dc0 msgr2=0x560ea4bdb210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:12.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.530+0000 7fc3437fe640 1 --2- 172.21.15.16:0/4249400604 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560ea4bd3dc0 0x560ea4bdb210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:12.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.530+0000 7fc3437fe640 1 -- 172.21.15.16:0/4249400604 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ea4bd34a0 msgr2=0x560ea4bddf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:12.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.530+0000 7fc3437fe640 1 --2- 172.21.15.16:0/4249400604 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ea4bd34a0 0x560ea4bddf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:12.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.530+0000 7fc3437fe640 1 -- 172.21.15.16:0/4249400604 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ea48aa3c0 con 0x560ea4bde680 2024-09-12T02:10:12.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.530+0000 7fc3437fe640 1 --2- 172.21.15.16:0/4249400604 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560ea4bde680 0x560ea4bdb750 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7fc338014f80 tx=0x7fc33800cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f73056eeb0793dc7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:10:12.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.530+0000 7fc3237fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:10:12.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.530+0000 7fc3237fe640 1 -- 172.21.15.16:0/4249400604 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc338004230 con 0x560ea4bde680 2024-09-12T02:10:12.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.530+0000 7fc3237fe640 1 -- 172.21.15.16:0/4249400604 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fc3380043d0 con 0x560ea4bde680 2024-09-12T02:10:12.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.530+0000 7fc34c97b740 1 -- 172.21.15.16:0/4249400604 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc3340317b0 con 0x560ea4bde680 2024-09-12T02:10:12.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.530+0000 7fc3237fe640 1 -- 172.21.15.16:0/4249400604 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc3380035f0 con 0x560ea4bde680 2024-09-12T02:10:12.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.530+0000 7fc34c97b740 10 client.17967 fetch_fsmap learned FSMap version 25 2024-09-12T02:10:12.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.530+0000 7fc34c97b740 1 -- 172.21.15.16:0/4249400604 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x560ea4becc50 con 0x560ea4bde680 2024-09-12T02:10:12.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.531+0000 7fc3237fe640 1 -- 172.21.15.16:0/4249400604 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3340317b0 con 0x560ea4bde680 2024-09-12T02:10:12.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.531+0000 7fc3237fe640 1 -- 172.21.15.16:0/4249400604 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fc33800f670 con 0x560ea4bde680 2024-09-12T02:10:12.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.531+0000 7fc34c97b740 10 client.17967 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:10:12.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.531+0000 7fc34c97b740 10 client.17967 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:10:12.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.531+0000 7fc34c97b740 20 client.17967 populate_metadata read hostname 'smithi016' 2024-09-12T02:10:12.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.531+0000 7fc34c97b740 1 --2- 172.21.15.16:0/4249400604 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x560ea4bee110 0x560ea4bf0500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:12.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.531+0000 7fc34c97b740 4 client.17967 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:10:12.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.531+0000 7fc34c97b740 1 -- 172.21.15.16:0/4249400604 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560ea4b46850 con 0x560ea4bee110 2024-09-12T02:10:12.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.531+0000 7fc343fff640 1 --2- 172.21.15.16:0/4249400604 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x560ea4bee110 0x560ea4bf0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:12.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.531+0000 7fc343fff640 1 --2- 172.21.15.16:0/4249400604 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x560ea4bee110 0x560ea4bf0500 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=28b5834226a36422 server_cookie=daff7b7f0a0f20d6 in_seq=0 out_seq=0 2024-09-12T02:10:12.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.531+0000 7fc3237fe640 10 client.17967 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:10:12.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.532+0000 7fc3237fe640 1 -- 172.21.15.16:0/4249400604 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560ea4b46850 con 0x560ea4bee110 2024-09-12T02:10:12.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.532+0000 7fc3237fe640 10 client.17967 handle_command_reply: tid=0 2024-09-12T02:10:12.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.532+0000 7fc34c97b740 1 client.17967 shutdown 2024-09-12T02:10:12.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.532+0000 7fc34c97b740 1 -- 172.21.15.16:0/4249400604 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560ea4bde680 msgr2=0x560ea4bdb750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:12.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.532+0000 7fc34c97b740 1 --2- 172.21.15.16:0/4249400604 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560ea4bde680 0x560ea4bdb750 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7fc338014f80 tx=0x7fc33800cc10 comp rx=0 tx=0).stop 2024-09-12T02:10:12.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.532+0000 7fc34c97b740 1 -- 172.21.15.16:0/4249400604 shutdown_connections 2024-09-12T02:10:12.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.532+0000 7fc34c97b740 1 --2- 172.21.15.16:0/4249400604 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x560ea4bee110 0x560ea4bf0500 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:12.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.532+0000 7fc34c97b740 1 --2- 172.21.15.16:0/4249400604 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560ea4bde680 0x560ea4bdb750 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:12.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.532+0000 7fc34c97b740 1 --2- 172.21.15.16:0/4249400604 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560ea4bd3dc0 0x560ea4bdb210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:12.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.532+0000 7fc34c97b740 1 --2- 172.21.15.16:0/4249400604 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ea4bd34a0 0x560ea4bddf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:12.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.532+0000 7fc34c97b740 1 -- 172.21.15.16:0/4249400604 >> 172.21.15.16:0/4249400604 conn(0x560ea4bcfa30 msgr2=0x560ea4bd5710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:10:12.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.532+0000 7fc3237fe640 0 client.17967 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:10:12.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.533+0000 7fc34c97b740 1 -- 172.21.15.16:0/4249400604 shutdown_connections 2024-09-12T02:10:12.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.533+0000 7fc34c97b740 1 -- 172.21.15.16:0/4249400604 wait complete. 2024-09-12T02:10:12.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.533+0000 7fc34c97b740 20 client.17967 trim_cache size 0 max 16384 2024-09-12T02:10:12.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.550+0000 7fc34c97b740 1 Processor -- start 2024-09-12T02:10:12.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.550+0000 7fc34c97b740 1 -- start start 2024-09-12T02:10:12.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.551+0000 7fc34c97b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560ea4bcfa30 0x560ea4bd9ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:12.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.551+0000 7fc34c97b740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560ea4bd34a0 0x560ea4bf5d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:12.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.551+0000 7fc34c97b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ea4bd3dc0 0x560ea4bf6270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:12.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.551+0000 7fc34c97b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc3441b1f10 con 0x560ea4bcfa30 2024-09-12T02:10:12.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.551+0000 7fc34c97b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc3441af7e0 con 0x560ea4bd3dc0 2024-09-12T02:10:12.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.551+0000 7fc34c97b740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc3441ad630 con 0x560ea4bd34a0 2024-09-12T02:10:12.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.551+0000 7fc3437fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560ea4bcfa30 0x560ea4bd9ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:12.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.551+0000 7fc3437fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560ea4bcfa30 0x560ea4bd9ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45768/0 (socket says 172.21.15.16:45768) 2024-09-12T02:10:12.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.551+0000 7fc3437fe640 1 -- 172.21.15.16:0/123429711 learned_addr learned my addr 172.21.15.16:0/123429711 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:10:12.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.551+0000 7fc343fff640 1 --2- 172.21.15.16:0/123429711 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560ea4bd34a0 0x560ea4bf5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:12.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.551+0000 7fc3437fe640 1 --2- 172.21.15.16:0/123429711 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ea4bd3dc0 0x560ea4bf6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:12.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.551+0000 7fc3437fe640 1 -- 172.21.15.16:0/123429711 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560ea4bd34a0 msgr2=0x560ea4bf5d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:12.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.551+0000 7fc3437fe640 1 --2- 172.21.15.16:0/123429711 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560ea4bd34a0 0x560ea4bf5d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:12.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.551+0000 7fc3437fe640 1 -- 172.21.15.16:0/123429711 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ea4bd3dc0 msgr2=0x560ea4bf6270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:12.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.551+0000 7fc3437fe640 1 --2- 172.21.15.16:0/123429711 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ea4bd3dc0 0x560ea4bf6270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:12.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.551+0000 7fc3437fe640 1 -- 172.21.15.16:0/123429711 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ea4bda1c0 con 0x560ea4bcfa30 2024-09-12T02:10:12.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.551+0000 7fc3437fe640 1 --2- 172.21.15.16:0/123429711 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560ea4bcfa30 0x560ea4bd9ae0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7fc33800cd80 tx=0x7fc338001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5aaf3c81e2d2b0f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:10:12.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.551+0000 7fc329ffb640 1 -- 172.21.15.16:0/123429711 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc33800fe10 con 0x560ea4bcfa30 2024-09-12T02:10:12.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.552+0000 7fc329ffb640 1 -- 172.21.15.16:0/123429711 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc338030020 con 0x560ea4bcfa30 2024-09-12T02:10:12.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.552+0000 7fc329ffb640 1 -- 172.21.15.16:0/123429711 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc338004680 con 0x560ea4bcfa30 2024-09-12T02:10:12.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.552+0000 7fc34c97b740 1 -- 172.21.15.16:0/123429711 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560ea4bcfa30 msgr2=0x560ea4bd9ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:12.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.552+0000 7fc34c97b740 1 --2- 172.21.15.16:0/123429711 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560ea4bcfa30 0x560ea4bd9ae0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7fc33800cd80 tx=0x7fc338001e50 comp rx=0 tx=0).stop 2024-09-12T02:10:12.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.552+0000 7fc34c97b740 1 -- 172.21.15.16:0/123429711 shutdown_connections 2024-09-12T02:10:12.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.552+0000 7fc34c97b740 1 --2- 172.21.15.16:0/123429711 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ea4bd3dc0 0x560ea4bf6270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:12.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.552+0000 7fc34c97b740 1 --2- 172.21.15.16:0/123429711 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560ea4bd34a0 0x560ea4bf5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:12.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.552+0000 7fc34c97b740 1 --2- 172.21.15.16:0/123429711 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560ea4bcfa30 0x560ea4bd9ae0 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:12.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.552+0000 7fc34c97b740 1 -- 172.21.15.16:0/123429711 >> 172.21.15.16:0/123429711 conn(0x560ea4bdb8b0 msgr2=0x560ea4bdd290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:10:12.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.552+0000 7fc34c97b740 1 -- 172.21.15.16:0/123429711 shutdown_connections 2024-09-12T02:10:12.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.552+0000 7fc34c97b740 1 -- 172.21.15.16:0/123429711 wait complete. 2024-09-12T02:10:12.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.553+0000 7fc34c97b740 1 -- start start 2024-09-12T02:10:12.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.553+0000 7fc34c97b740 1 Processor -- start 2024-09-12T02:10:12.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.553+0000 7fc34c97b740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560ea4bcfa30 0x560ea4be4390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:12.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.553+0000 7fc34c97b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560ea4bd34a0 0x560ea4be48d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:12.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.553+0000 7fc34c97b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ea4bd3dc0 0x560ea4be4e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:12.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.553+0000 7fc34c97b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc3441adca0 con 0x560ea4bd34a0 2024-09-12T02:10:12.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.553+0000 7fc34c97b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc3441af110 con 0x560ea4bd3dc0 2024-09-12T02:10:12.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.553+0000 7fc34c97b740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc3441ae370 con 0x560ea4bcfa30 2024-09-12T02:10:12.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.553+0000 7fc34c97b740 10 client.0 _ll_register_callbacks cb 0x7fc33800ce10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:10:12.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.553+0000 7fc343fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560ea4bd34a0 0x560ea4be48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:12.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.553+0000 7fc343fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560ea4bd34a0 0x560ea4be48d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45774/0 (socket says 172.21.15.16:45774) 2024-09-12T02:10:12.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.553+0000 7fc343fff640 1 -- 172.21.15.16:0/2897271770 learned_addr learned my addr 172.21.15.16:0/2897271770 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:10:12.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.553+0000 7fc3437fe640 1 --2- 172.21.15.16:0/2897271770 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560ea4bcfa30 0x560ea4be4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:12.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.554+0000 7fc3437fe640 1 --2- 172.21.15.16:0/2897271770 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ea4bd3dc0 0x560ea4be4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:12.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.554+0000 7fc343fff640 1 -- 172.21.15.16:0/2897271770 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560ea4bcfa30 msgr2=0x560ea4be4390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:12.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.554+0000 7fc343fff640 1 --2- 172.21.15.16:0/2897271770 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560ea4bcfa30 0x560ea4be4390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:12.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.554+0000 7fc343fff640 1 -- 172.21.15.16:0/2897271770 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ea4bd3dc0 msgr2=0x560ea4be4e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:12.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.554+0000 7fc343fff640 1 --2- 172.21.15.16:0/2897271770 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ea4bd3dc0 0x560ea4be4e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:12.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.554+0000 7fc343fff640 1 -- 172.21.15.16:0/2897271770 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ea4bde2d0 con 0x560ea4bd34a0 2024-09-12T02:10:12.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.554+0000 7fc3437fe640 1 --2- 172.21.15.16:0/2897271770 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ea4bd3dc0 0x560ea4be4e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:10:12.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.554+0000 7fc343fff640 1 --2- 172.21.15.16:0/2897271770 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560ea4bd34a0 0x560ea4be48d0 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7fc334009930 tx=0x7fc334005230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70daf35629d114dd server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:10:12.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.554+0000 7fc3237fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:10:12.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.554+0000 7fc3237fe640 1 -- 172.21.15.16:0/2897271770 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc334052020 con 0x560ea4bd34a0 2024-09-12T02:10:12.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.554+0000 7fc34c97b740 1 -- 172.21.15.16:0/2897271770 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc338030020 con 0x560ea4bd34a0 2024-09-12T02:10:12.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.554+0000 7fc3237fe640 1 -- 172.21.15.16:0/2897271770 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fc334004030 con 0x560ea4bd34a0 2024-09-12T02:10:12.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.554+0000 7fc3237fe640 1 -- 172.21.15.16:0/2897271770 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc334035040 con 0x560ea4bd34a0 2024-09-12T02:10:12.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.554+0000 7fc34c97b740 10 client.17973 fetch_fsmap learned FSMap version 25 2024-09-12T02:10:12.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.555+0000 7fc34c97b740 1 -- 172.21.15.16:0/2897271770 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x560ea4c58e00 con 0x560ea4bd34a0 2024-09-12T02:10:12.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.555+0000 7fc3237fe640 1 -- 172.21.15.16:0/2897271770 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc338030020 con 0x560ea4bd34a0 2024-09-12T02:10:12.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.555+0000 7fc3237fe640 1 -- 172.21.15.16:0/2897271770 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fc334003ad0 con 0x560ea4bd34a0 2024-09-12T02:10:12.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.555+0000 7fc34c97b740 10 client.17973 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:10:12.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.555+0000 7fc34c97b740 10 client.17973 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:10:12.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.555+0000 7fc34c97b740 20 client.17973 populate_metadata read hostname 'smithi016' 2024-09-12T02:10:12.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.555+0000 7fc34c97b740 1 --2- 172.21.15.16:0/2897271770 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x560ea4bde680 0x560ea4c5b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:12.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.555+0000 7fc34c97b740 4 client.17973 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T02:10:12.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.555+0000 7fc34c97b740 1 -- 172.21.15.16:0/2897271770 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560ea4b40d60 con 0x560ea4bde680 2024-09-12T02:10:12.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.555+0000 7fc3437fe640 1 --2- 172.21.15.16:0/2897271770 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x560ea4bde680 0x560ea4c5b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:12.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.556+0000 7fc3437fe640 1 --2- 172.21.15.16:0/2897271770 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x560ea4bde680 0x560ea4c5b0f0 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f4105582e89f1ec server_cookie=fa62407da76af164 in_seq=0 out_seq=0 2024-09-12T02:10:12.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.556+0000 7fc3237fe640 10 client.17973 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:10:12.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.556+0000 7fc3237fe640 1 -- 172.21.15.16:0/2897271770 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x560ea4b40d60 con 0x560ea4bde680 2024-09-12T02:10:12.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.556+0000 7fc3237fe640 10 client.17973 handle_command_reply: tid=0 2024-09-12T02:10:12.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.556+0000 7fc34c97b740 1 client.17973 shutdown 2024-09-12T02:10:12.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.556+0000 7fc34c97b740 1 -- 172.21.15.16:0/2897271770 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560ea4bd34a0 msgr2=0x560ea4be48d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:12.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.556+0000 7fc34c97b740 1 --2- 172.21.15.16:0/2897271770 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560ea4bd34a0 0x560ea4be48d0 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7fc334009930 tx=0x7fc334005230 comp rx=0 tx=0).stop 2024-09-12T02:10:12.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.556+0000 7fc34c97b740 1 -- 172.21.15.16:0/2897271770 shutdown_connections 2024-09-12T02:10:12.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.556+0000 7fc34c97b740 1 --2- 172.21.15.16:0/2897271770 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x560ea4bde680 0x560ea4c5b0f0 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:12.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.556+0000 7fc3237fe640 0 client.17973 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:10:12.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.556+0000 7fc34c97b740 1 --2- 172.21.15.16:0/2897271770 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ea4bd3dc0 0x560ea4be4e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:12.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.556+0000 7fc34c97b740 1 --2- 172.21.15.16:0/2897271770 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560ea4bd34a0 0x560ea4be48d0 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:12.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.556+0000 7fc34c97b740 1 --2- 172.21.15.16:0/2897271770 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560ea4bcfa30 0x560ea4be4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:12.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.556+0000 7fc34c97b740 1 -- 172.21.15.16:0/2897271770 >> 172.21.15.16:0/2897271770 conn(0x560ea4bdb8b0 msgr2=0x560ea4bdd290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:10:12.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.556+0000 7fc34c97b740 1 -- 172.21.15.16:0/2897271770 shutdown_connections 2024-09-12T02:10:12.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.556+0000 7fc34c97b740 1 -- 172.21.15.16:0/2897271770 wait complete. 2024-09-12T02:10:12.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.556+0000 7fc34c97b740 20 client.17973 trim_cache size 0 max 16384 2024-09-12T02:10:12.563 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T02:10:12.563 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-09-12T02:10:12.563 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {} 2024-09-12T02:10:12.564 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T02:10:12.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.557+0000 7fc34a5a2640 1 -- 172.21.15.16:0/1998227644 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc32c0a2b80 msgr2=0x7fc32c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:12.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.557+0000 7fc34a5a2640 1 --2- 172.21.15.16:0/1998227644 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc32c0a2b80 0x7fc32c0a5040 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fc33c096bc0 tx=0x7fc33c095910 comp rx=0 tx=0).stop 2024-09-12T02:10:12.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.557+0000 7fc34a5a2640 1 -- 172.21.15.16:0/1998227644 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc344106a80 msgr2=0x7fc3441034b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:12.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.557+0000 7fc34a5a2640 1 --2- 172.21.15.16:0/1998227644 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc344106a80 0x7fc3441034b0 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7fc33c00b970 tx=0x7fc33c00be30 comp rx=0 tx=0).stop 2024-09-12T02:10:12.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.557+0000 7fc34a5a2640 1 -- 172.21.15.16:0/1998227644 shutdown_connections 2024-09-12T02:10:12.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.557+0000 7fc34a5a2640 1 --2- 172.21.15.16:0/1998227644 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc32c0a2b80 0x7fc32c0a5040 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:12.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.557+0000 7fc34a5a2640 1 --2- 172.21.15.16:0/1998227644 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc344106a80 0x7fc3441034b0 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:12.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.557+0000 7fc34a5a2640 1 --2- 172.21.15.16:0/1998227644 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc344101420 0x7fc344102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:12.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.557+0000 7fc34a5a2640 1 --2- 172.21.15.16:0/1998227644 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc344100a80 0x7fc344102a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:12.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.557+0000 7fc34a5a2640 1 -- 172.21.15.16:0/1998227644 >> 172.21.15.16:0/1998227644 conn(0x7fc3440fc7a0 msgr2=0x7fc344104c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:10:12.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.557+0000 7fc34a5a2640 1 -- 172.21.15.16:0/1998227644 shutdown_connections 2024-09-12T02:10:12.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:12.557+0000 7fc34a5a2640 1 -- 172.21.15.16:0/1998227644 wait complete. 2024-09-12T02:10:13.240 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-12T02:10:13.241 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:10:13.498 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:10:13.904 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:10:13 smithi016 ceph-mon[30947]: pgmap v584: 97 pgs: 97 active+clean; 4.2 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.6 KiB/s wr, 8 op/s 2024-09-12T02:10:14.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:13.987+0000 7f151c77a640 1 -- 172.21.15.16:0/2994203159 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15141036e0 msgr2=0x7f1514107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:14.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:13.987+0000 7f151c77a640 1 --2- 172.21.15.16:0/2994203159 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15141036e0 0x7f1514107b90 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f1504009930 tx=0x7f150402a420 comp rx=0 tx=0).stop 2024-09-12T02:10:14.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:13.987+0000 7f151c77a640 1 -- 172.21.15.16:0/2994203159 shutdown_connections 2024-09-12T02:10:14.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:13.987+0000 7f151c77a640 1 --2- 172.21.15.16:0/2994203159 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f15141082a0 0x7f151410ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:14.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:13.987+0000 7f151c77a640 1 --2- 172.21.15.16:0/2994203159 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15141036e0 0x7f1514107b90 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:14.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:13.987+0000 7f151c77a640 1 --2- 172.21.15.16:0/2994203159 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1514102d30 0x7f1514103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:14.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:13.987+0000 7f151c77a640 1 -- 172.21.15.16:0/2994203159 >> 172.21.15.16:0/2994203159 conn(0x7f15140fc7d0 msgr2=0x7f15140febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:10:14.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:13.987+0000 7f151c77a640 1 -- 172.21.15.16:0/2994203159 shutdown_connections 2024-09-12T02:10:14.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:13.987+0000 7f151c77a640 1 -- 172.21.15.16:0/2994203159 wait complete. 2024-09-12T02:10:14.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:13.987+0000 7f151c77a640 1 Processor -- start 2024-09-12T02:10:14.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:13.988+0000 7f151c77a640 1 -- start start 2024-09-12T02:10:14.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:13.988+0000 7f151c77a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1514102d30 0x7f1514198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:14.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:13.988+0000 7f151c77a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f15141036e0 0x7f1514198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:14.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:13.988+0000 7f151c77a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f15141082a0 0x7f151419ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:14.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:13.988+0000 7f151c77a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f15141120a0 con 0x7f1514102d30 2024-09-12T02:10:14.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:13.988+0000 7f151c77a640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1514111f20 con 0x7f15141036e0 2024-09-12T02:10:14.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:13.988+0000 7f151c77a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1514112220 con 0x7f15141082a0 2024-09-12T02:10:14.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:13.988+0000 7f151a516640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1514102d30 0x7f1514198580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:14.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:13.988+0000 7f151a516640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1514102d30 0x7f1514198580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45794/0 (socket says 172.21.15.16:45794) 2024-09-12T02:10:14.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:13.988+0000 7f151a516640 1 -- 172.21.15.16:0/873521910 learned_addr learned my addr 172.21.15.16:0/873521910 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:10:14.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:13.988+0000 7f1519d15640 1 --2- 172.21.15.16:0/873521910 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f15141036e0 0x7f1514198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:14.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:13.988+0000 7f151ad17640 1 --2- 172.21.15.16:0/873521910 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f15141082a0 0x7f151419ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:14.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:13.988+0000 7f151a516640 1 -- 172.21.15.16:0/873521910 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f15141082a0 msgr2=0x7f151419ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:14.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:13.988+0000 7f151a516640 1 --2- 172.21.15.16:0/873521910 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f15141082a0 0x7f151419ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:14.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:13.988+0000 7f151a516640 1 -- 172.21.15.16:0/873521910 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f15141036e0 msgr2=0x7f1514198ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:14.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:13.988+0000 7f151a516640 1 --2- 172.21.15.16:0/873521910 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f15141036e0 0x7f1514198ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:14.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:13.988+0000 7f151a516640 1 -- 172.21.15.16:0/873521910 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f151419d530 con 0x7f1514102d30 2024-09-12T02:10:14.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:13.988+0000 7f151a516640 1 --2- 172.21.15.16:0/873521910 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1514102d30 0x7f1514198580 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f151000c970 tx=0x7f151000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:10:14.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:13.989+0000 7f150b7fe640 1 -- 172.21.15.16:0/873521910 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1510007bb0 con 0x7f1514102d30 2024-09-12T02:10:14.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:13.989+0000 7f151c77a640 1 -- 172.21.15.16:0/873521910 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f151419d820 con 0x7f1514102d30 2024-09-12T02:10:14.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:13.989+0000 7f150b7fe640 1 -- 172.21.15.16:0/873521910 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1510007d50 con 0x7f1514102d30 2024-09-12T02:10:14.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:13.989+0000 7f150b7fe640 1 -- 172.21.15.16:0/873521910 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f151000f450 con 0x7f1514102d30 2024-09-12T02:10:14.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:13.989+0000 7f151c77a640 1 -- 172.21.15.16:0/873521910 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15141a5100 con 0x7f1514102d30 2024-09-12T02:10:14.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:13.990+0000 7f151c77a640 1 -- 172.21.15.16:0/873521910 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1514103110 con 0x7f1514102d30 2024-09-12T02:10:14.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:13.991+0000 7f150b7fe640 1 -- 172.21.15.16:0/873521910 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f1510004fe0 con 0x7f1514102d30 2024-09-12T02:10:14.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:13.992+0000 7f150b7fe640 1 --2- 172.21.15.16:0/873521910 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f15000a2b80 0x7f15000a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:14.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:13.992+0000 7f1519d15640 1 --2- 172.21.15.16:0/873521910 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f15000a2b80 0x7f15000a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:14.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:13.992+0000 7f150b7fe640 1 -- 172.21.15.16:0/873521910 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f15100cf170 con 0x7f1514102d30 2024-09-12T02:10:14.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:13.992+0000 7f1519d15640 1 --2- 172.21.15.16:0/873521910 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f15000a2b80 0x7f15000a5040 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f1504005940 tx=0x7f15040058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:10:14.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:13.995+0000 7f150b7fe640 1 -- 172.21.15.16:0/873521910 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1510014030 con 0x7f1514102d30 2024-09-12T02:10:14.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:10:13 smithi184 ceph-mon[35593]: pgmap v584: 97 pgs: 97 active+clean; 4.2 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.6 KiB/s wr, 8 op/s 2024-09-12T02:10:14.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:10:13 smithi033 ceph-mon[35332]: pgmap v584: 97 pgs: 97 active+clean; 4.2 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.6 KiB/s wr, 8 op/s 2024-09-12T02:10:14.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:14.207+0000 7f151c77a640 1 -- 172.21.15.16:0/873521910 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f15141998e0 con 0x7f1514102d30 2024-09-12T02:10:14.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:14.208+0000 7f150b7fe640 1 -- 172.21.15.16:0/873521910 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11911 (secure 0 0 0) 0x7f15141998e0 con 0x7f1514102d30 2024-09-12T02:10:14.209 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T02:10:14.209 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":25,"btime":"2024-09-12T02:01:37:192021+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3279123279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3279123279},{"type":"v1","addr":"172.21.15.16:6835","nonce":3279123279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4119452382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4119452382},{"type":"v1","addr":"172.21.15.16:6837","nonce":4119452382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/3263823165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3263823165},{"type":"v1","addr":"172.21.15.16:6839","nonce":3263823165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/1510290732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":1510290732},{"type":"v1","addr":"172.21.15.16:6841","nonce":1510290732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/279674603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":279674603},{"type":"v1","addr":"172.21.15.33:6833","nonce":279674603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3555728028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3555728028},{"type":"v1","addr":"172.21.15.33:6835","nonce":3555728028}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T01:59:56.117608+0000","modified":"2024-09-12T02:01:36.294760+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24482,"mds_1":24481,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6833/1563374106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1563374106},{"type":"v1","addr":"172.21.15.184:6833","nonce":1563374106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6837/1892833868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1892833868},{"type":"v1","addr":"172.21.15.33:6837","nonce":1892833868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/1457679686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1457679686},{"type":"v1","addr":"172.21.15.184:6835","nonce":1457679686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.184:6837/1793393087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":1793393087},{"type":"v1","addr":"172.21.15.184:6837","nonce":1793393087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.184:6839/4203347941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":4203347941},{"type":"v1","addr":"172.21.15.184:6839","nonce":4203347941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.33:6839/4136670204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":4136670204},{"type":"v1","addr":"172.21.15.33:6839","nonce":4136670204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24481,24482,24469]},"id":1}]} 2024-09-12T02:10:14.210 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 25 2024-09-12T02:10:14.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:14.210+0000 7f151c77a640 1 -- 172.21.15.16:0/873521910 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f15000a2b80 msgr2=0x7f15000a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:14.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:14.210+0000 7f151c77a640 1 --2- 172.21.15.16:0/873521910 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f15000a2b80 0x7f15000a5040 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f1504005940 tx=0x7f15040058d0 comp rx=0 tx=0).stop 2024-09-12T02:10:14.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:14.210+0000 7f151c77a640 1 -- 172.21.15.16:0/873521910 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1514102d30 msgr2=0x7f1514198580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:14.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:14.210+0000 7f151c77a640 1 --2- 172.21.15.16:0/873521910 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1514102d30 0x7f1514198580 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f151000c970 tx=0x7f151000ce30 comp rx=0 tx=0).stop 2024-09-12T02:10:14.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:14.210+0000 7f151c77a640 1 -- 172.21.15.16:0/873521910 shutdown_connections 2024-09-12T02:10:14.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:14.210+0000 7f151c77a640 1 --2- 172.21.15.16:0/873521910 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f15000a2b80 0x7f15000a5040 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:14.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:14.210+0000 7f151c77a640 1 --2- 172.21.15.16:0/873521910 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f15141082a0 0x7f151419ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:14.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:14.210+0000 7f151c77a640 1 --2- 172.21.15.16:0/873521910 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f15141036e0 0x7f1514198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:14.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:14.210+0000 7f151c77a640 1 --2- 172.21.15.16:0/873521910 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1514102d30 0x7f1514198580 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:14.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:14.210+0000 7f151c77a640 1 -- 172.21.15.16:0/873521910 >> 172.21.15.16:0/873521910 conn(0x7f15140fc7d0 msgr2=0x7f1514105d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:10:14.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:14.211+0000 7f151c77a640 1 -- 172.21.15.16:0/873521910 shutdown_connections 2024-09-12T02:10:14.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:14.211+0000 7f151c77a640 1 -- 172.21.15.16:0/873521910 wait complete. 2024-09-12T02:10:14.818 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:10:14 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/873521910' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:10:14.848 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-12T02:10:15.105 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:10:15.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:10:14 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/873521910' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:10:15.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:10:14 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/873521910' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:10:15.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.613+0000 7efcd77d3640 1 -- 172.21.15.16:0/1600626396 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efcd0100c10 msgr2=0x7efcd0110840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:15.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.613+0000 7efcd77d3640 1 --2- 172.21.15.16:0/1600626396 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efcd0100c10 0x7efcd0110840 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7efcc8009ca0 tx=0x7efcc802a950 comp rx=0 tx=0).stop 2024-09-12T02:10:15.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.613+0000 7efcd77d3640 1 -- 172.21.15.16:0/1600626396 shutdown_connections 2024-09-12T02:10:15.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.613+0000 7efcd77d3640 1 --2- 172.21.15.16:0/1600626396 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efcd0100c10 0x7efcd0110840 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:15.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.613+0000 7efcd77d3640 1 --2- 172.21.15.16:0/1600626396 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efcd0102ea0 0x7efcd01005f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:15.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.613+0000 7efcd77d3640 1 --2- 172.21.15.16:0/1600626396 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efcd0102ac0 0x7efcd01000b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:15.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.613+0000 7efcd77d3640 1 -- 172.21.15.16:0/1600626396 >> 172.21.15.16:0/1600626396 conn(0x7efcd0077210 msgr2=0x7efcd00fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:10:15.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.613+0000 7efcd77d3640 1 -- 172.21.15.16:0/1600626396 shutdown_connections 2024-09-12T02:10:15.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.613+0000 7efcd77d3640 1 -- 172.21.15.16:0/1600626396 wait complete. 2024-09-12T02:10:15.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.614+0000 7efcd77d3640 1 Processor -- start 2024-09-12T02:10:15.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.614+0000 7efcd77d3640 1 -- start start 2024-09-12T02:10:15.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.614+0000 7efcd77d3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efcd0100c10 0x7efcd019a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:15.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.614+0000 7efcd77d3640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efcd0102ac0 0x7efcd019ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:15.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.614+0000 7efcd77d3640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efcd0102ea0 0x7efcd019eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:15.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.614+0000 7efcd77d3640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7efcd0114180 con 0x7efcd0100c10 2024-09-12T02:10:15.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.614+0000 7efcd77d3640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7efcd0114000 con 0x7efcd0102ea0 2024-09-12T02:10:15.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.614+0000 7efcd77d3640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7efcd0114300 con 0x7efcd0102ac0 2024-09-12T02:10:15.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.614+0000 7efcd556f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efcd0100c10 0x7efcd019a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:15.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.614+0000 7efcd556f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efcd0100c10 0x7efcd019a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45810/0 (socket says 172.21.15.16:45810) 2024-09-12T02:10:15.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.614+0000 7efcd556f640 1 -- 172.21.15.16:0/803626164 learned_addr learned my addr 172.21.15.16:0/803626164 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:10:15.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.614+0000 7efcd4d6e640 1 --2- 172.21.15.16:0/803626164 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efcd0102ac0 0x7efcd019ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:15.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.614+0000 7efcd5d70640 1 --2- 172.21.15.16:0/803626164 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efcd0102ea0 0x7efcd019eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:15.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.615+0000 7efcd556f640 1 -- 172.21.15.16:0/803626164 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efcd0102ac0 msgr2=0x7efcd019ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:15.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.615+0000 7efcd556f640 1 --2- 172.21.15.16:0/803626164 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efcd0102ac0 0x7efcd019ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:15.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.615+0000 7efcd556f640 1 -- 172.21.15.16:0/803626164 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efcd0102ea0 msgr2=0x7efcd019eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:15.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.615+0000 7efcd556f640 1 --2- 172.21.15.16:0/803626164 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efcd0102ea0 0x7efcd019eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:15.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.615+0000 7efcd556f640 1 -- 172.21.15.16:0/803626164 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcd019f5a0 con 0x7efcd0100c10 2024-09-12T02:10:15.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.615+0000 7efcd556f640 1 --2- 172.21.15.16:0/803626164 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efcd0100c10 0x7efcd019a5f0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7efcc000e9f0 tx=0x7efcc000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:10:15.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.615+0000 7efcc67fc640 1 -- 172.21.15.16:0/803626164 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcc000cd80 con 0x7efcd0100c10 2024-09-12T02:10:15.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.615+0000 7efcd77d3640 1 -- 172.21.15.16:0/803626164 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcd019f890 con 0x7efcd0100c10 2024-09-12T02:10:15.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.615+0000 7efcc67fc640 1 -- 172.21.15.16:0/803626164 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efcc0010910 con 0x7efcd0100c10 2024-09-12T02:10:15.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.615+0000 7efcc67fc640 1 -- 172.21.15.16:0/803626164 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcc00188f0 con 0x7efcd0100c10 2024-09-12T02:10:15.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.615+0000 7efcd77d3640 1 -- 172.21.15.16:0/803626164 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efcd01a7170 con 0x7efcd0100c10 2024-09-12T02:10:15.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.618+0000 7efcc67fc640 1 -- 172.21.15.16:0/803626164 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7efcc0010ac0 con 0x7efcd0100c10 2024-09-12T02:10:15.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.618+0000 7efcd9bac740 1 Processor -- start 2024-09-12T02:10:15.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.618+0000 7efcd9bac740 1 -- start start 2024-09-12T02:10:15.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.618+0000 7efcd9bac740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56508df27b40 0x56508df26090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:15.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.618+0000 7efcd9bac740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56508df265d0 0x56508df26a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:15.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.618+0000 7efcd9bac740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56508df31370 0x56508df33760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:15.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.618+0000 7efcd9bac740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56508de8c2f0 con 0x56508df27b40 2024-09-12T02:10:15.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.618+0000 7efcd9bac740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56508de33cf0 con 0x56508df265d0 2024-09-12T02:10:15.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.618+0000 7efcd9bac740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x56508de882b0 con 0x56508df31370 2024-09-12T02:10:15.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.618+0000 7efcd5d70640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56508df27b40 0x56508df26090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:15.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.618+0000 7efcd5d70640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56508df27b40 0x56508df26090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45824/0 (socket says 172.21.15.16:45824) 2024-09-12T02:10:15.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.618+0000 7efcd5d70640 1 -- 172.21.15.16:0/4237753992 learned_addr learned my addr 172.21.15.16:0/4237753992 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:10:15.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.618+0000 7efcd556f640 1 --2- 172.21.15.16:0/4237753992 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56508df265d0 0x56508df26a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:15.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.619+0000 7efcd4d6e640 1 --2- 172.21.15.16:0/4237753992 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56508df31370 0x56508df33760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:15.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.619+0000 7efcd5d70640 1 -- 172.21.15.16:0/4237753992 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56508df31370 msgr2=0x56508df33760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:15.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.619+0000 7efcd5d70640 1 --2- 172.21.15.16:0/4237753992 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56508df31370 0x56508df33760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:15.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.619+0000 7efcd5d70640 1 -- 172.21.15.16:0/4237753992 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56508df265d0 msgr2=0x56508df26a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:15.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.619+0000 7efcd5d70640 1 --2- 172.21.15.16:0/4237753992 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56508df265d0 0x56508df26a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:15.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.619+0000 7efcc67fc640 1 --2- 172.21.15.16:0/803626164 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7efcbc0a2b80 0x7efcbc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:15.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.619+0000 7efcd5d70640 1 -- 172.21.15.16:0/4237753992 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56508de183a0 con 0x56508df27b40 2024-09-12T02:10:15.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.619+0000 7efcd4d6e640 1 --2- 172.21.15.16:0/803626164 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7efcbc0a2b80 0x7efcbc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:15.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.619+0000 7efcd5d70640 1 --2- 172.21.15.16:0/4237753992 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56508df27b40 0x56508df26090 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7efcc8035000 tx=0x7efcc80093e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30524ddd95a38f64 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:10:15.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.619+0000 7efcc67fc640 1 -- 172.21.15.16:0/803626164 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7efcc00cf330 con 0x7efcd0100c10 2024-09-12T02:10:15.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.619+0000 7efcbb7fe640 1 -- 172.21.15.16:0/4237753992 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcc8034040 con 0x56508df27b40 2024-09-12T02:10:15.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.619+0000 7efcbb7fe640 1 -- 172.21.15.16:0/4237753992 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efcc802f940 con 0x56508df27b40 2024-09-12T02:10:15.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.619+0000 7efcbb7fe640 1 -- 172.21.15.16:0/4237753992 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcc80042a0 con 0x56508df27b40 2024-09-12T02:10:15.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.620+0000 7efcd9bac740 1 -- 172.21.15.16:0/4237753992 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56508df27b40 msgr2=0x56508df26090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:15.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.620+0000 7efcd4d6e640 1 --2- 172.21.15.16:0/803626164 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7efcbc0a2b80 0x7efcbc0a5040 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x56508df27070 tx=0x7efccc005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:10:15.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.620+0000 7efcd9bac740 1 --2- 172.21.15.16:0/4237753992 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56508df27b40 0x56508df26090 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7efcc8035000 tx=0x7efcc80093e0 comp rx=0 tx=0).stop 2024-09-12T02:10:15.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.620+0000 7efcd9bac740 1 -- 172.21.15.16:0/4237753992 shutdown_connections 2024-09-12T02:10:15.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.620+0000 7efcd9bac740 1 --2- 172.21.15.16:0/4237753992 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56508df31370 0x56508df33760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:15.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.620+0000 7efcd9bac740 1 --2- 172.21.15.16:0/4237753992 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56508df265d0 0x56508df26a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:15.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.620+0000 7efcd9bac740 1 --2- 172.21.15.16:0/4237753992 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56508df27b40 0x56508df26090 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:15.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.620+0000 7efcd9bac740 1 -- 172.21.15.16:0/4237753992 >> 172.21.15.16:0/4237753992 conn(0x56508deef310 msgr2=0x56508df22c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:10:15.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.620+0000 7efcd9bac740 1 -- 172.21.15.16:0/4237753992 shutdown_connections 2024-09-12T02:10:15.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.620+0000 7efcd9bac740 1 -- 172.21.15.16:0/4237753992 wait complete. 2024-09-12T02:10:15.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.620+0000 7efcd9bac740 1 -- start start 2024-09-12T02:10:15.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.621+0000 7efcd9bac740 1 Processor -- start 2024-09-12T02:10:15.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.621+0000 7efcd9bac740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56508df265d0 0x56508df30d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:15.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.621+0000 7efcd9bac740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56508df27b40 0x56508df2dff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:15.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.621+0000 7efcd9bac740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56508df31370 0x56508df2e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:15.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.621+0000 7efcd9bac740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56508df34010 con 0x56508df265d0 2024-09-12T02:10:15.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.621+0000 7efcd9bac740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56508de47060 con 0x56508df31370 2024-09-12T02:10:15.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.621+0000 7efcd9bac740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x56508df34190 con 0x56508df27b40 2024-09-12T02:10:15.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.621+0000 7efcd9bac740 10 client.0 _ll_register_callbacks cb 0x56508db9a8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:10:15.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.621+0000 7efcd556f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56508df265d0 0x56508df30d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:15.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.621+0000 7efcd556f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56508df265d0 0x56508df30d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45836/0 (socket says 172.21.15.16:45836) 2024-09-12T02:10:15.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.621+0000 7efcd556f640 1 -- 172.21.15.16:0/947436283 learned_addr learned my addr 172.21.15.16:0/947436283 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:10:15.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.621+0000 7efcd5d70640 1 --2- 172.21.15.16:0/947436283 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56508df31370 0x56508df2e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:15.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.621+0000 7efcd4d6e640 1 --2- 172.21.15.16:0/947436283 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56508df27b40 0x56508df2dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:15.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.621+0000 7efcd556f640 1 -- 172.21.15.16:0/947436283 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56508df27b40 msgr2=0x56508df2dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:15.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.621+0000 7efcd556f640 1 --2- 172.21.15.16:0/947436283 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56508df27b40 0x56508df2dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:15.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.621+0000 7efcd556f640 1 -- 172.21.15.16:0/947436283 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56508df31370 msgr2=0x56508df2e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:15.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.621+0000 7efcd556f640 1 --2- 172.21.15.16:0/947436283 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56508df31370 0x56508df2e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:15.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.621+0000 7efcd556f640 1 -- 172.21.15.16:0/947436283 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56508dbfd3c0 con 0x56508df265d0 2024-09-12T02:10:15.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.622+0000 7efcd5d70640 1 --2- 172.21.15.16:0/947436283 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56508df31370 0x56508df2e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:10:15.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.622+0000 7efcd556f640 1 --2- 172.21.15.16:0/947436283 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56508df265d0 0x56508df30d40 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7efcc000c6a0 tx=0x7efcc0098040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2bdfd084369f707 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:10:15.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.622+0000 7efcb8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:10:15.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.622+0000 7efcb8ff9640 1 -- 172.21.15.16:0/947436283 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcc009b330 con 0x56508df265d0 2024-09-12T02:10:15.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.622+0000 7efcb8ff9640 1 -- 172.21.15.16:0/947436283 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7efcc0010430 con 0x56508df265d0 2024-09-12T02:10:15.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.622+0000 7efcd9bac740 1 -- 172.21.15.16:0/947436283 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efcc802f940 con 0x56508df265d0 2024-09-12T02:10:15.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.622+0000 7efcb8ff9640 1 -- 172.21.15.16:0/947436283 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7efcc009bce0 con 0x56508df265d0 2024-09-12T02:10:15.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.622+0000 7efcd9bac740 10 client.17991 fetch_fsmap learned FSMap version 25 2024-09-12T02:10:15.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.622+0000 7efcd9bac740 1 -- 172.21.15.16:0/947436283 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56508df3fad0 con 0x56508df265d0 2024-09-12T02:10:15.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.622+0000 7efcb8ff9640 1 -- 172.21.15.16:0/947436283 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efcc802f940 con 0x56508df265d0 2024-09-12T02:10:15.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.622+0000 7efcb8ff9640 1 -- 172.21.15.16:0/947436283 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7efcc00ce030 con 0x56508df265d0 2024-09-12T02:10:15.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.622+0000 7efcd9bac740 10 client.17991 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:10:15.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.622+0000 7efcd9bac740 10 client.17991 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-12T02:10:15.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.622+0000 7efcd9bac740 20 client.17991 populate_metadata read hostname 'smithi016' 2024-09-12T02:10:15.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.622+0000 7efcd9bac740 1 --2- 172.21.15.16:0/947436283 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x56508df40ee0 0x56508df43310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:15.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.622+0000 7efcd9bac740 4 client.17991 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:10:15.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.622+0000 7efcd9bac740 1 -- 172.21.15.16:0/947436283 --> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56508de99860 con 0x56508df40ee0 2024-09-12T02:10:15.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.623+0000 7efcd4d6e640 1 --2- 172.21.15.16:0/947436283 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x56508df40ee0 0x56508df43310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:15.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.623+0000 7efcb8ff9640 10 client.17991 ms_handle_connect on v2:172.21.15.184:6836/1793393087 2024-09-12T02:10:15.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.623+0000 7efcd4d6e640 1 --2- 172.21.15.16:0/947436283 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x56508df40ee0 0x56508df43310 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ba1bb53979c19b36 server_cookie=9fa043bd26fffc8f in_seq=0 out_seq=0 2024-09-12T02:10:15.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.624+0000 7efcb8ff9640 1 -- 172.21.15.16:0/947436283 <== mds.2 v2:172.21.15.184:6836/1793393087 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56508de99860 con 0x56508df40ee0 2024-09-12T02:10:15.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.624+0000 7efcb8ff9640 10 client.17991 handle_command_reply: tid=0 2024-09-12T02:10:15.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.624+0000 7efcd9bac740 1 client.17991 shutdown 2024-09-12T02:10:15.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.624+0000 7efcd9bac740 1 -- 172.21.15.16:0/947436283 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56508df265d0 msgr2=0x56508df30d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:15.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.624+0000 7efcd9bac740 1 --2- 172.21.15.16:0/947436283 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56508df265d0 0x56508df30d40 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7efcc000c6a0 tx=0x7efcc0098040 comp rx=0 tx=0).stop 2024-09-12T02:10:15.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.624+0000 7efcd9bac740 1 -- 172.21.15.16:0/947436283 shutdown_connections 2024-09-12T02:10:15.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.624+0000 7efcd9bac740 1 --2- 172.21.15.16:0/947436283 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56508df31370 0x56508df2e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:15.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.624+0000 7efcd9bac740 1 --2- 172.21.15.16:0/947436283 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56508df27b40 0x56508df2dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:15.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.624+0000 7efcd9bac740 1 --2- 172.21.15.16:0/947436283 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x56508df40ee0 0x56508df43310 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:15.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.624+0000 7efcd9bac740 1 --2- 172.21.15.16:0/947436283 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56508df265d0 0x56508df30d40 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:15.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.624+0000 7efcd9bac740 1 -- 172.21.15.16:0/947436283 >> 172.21.15.16:0/947436283 conn(0x56508deef310 msgr2=0x56508df28930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:10:15.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.624+0000 7efcb8ff9640 0 client.17991 ms_handle_reset on v2:172.21.15.184:6836/1793393087 2024-09-12T02:10:15.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.624+0000 7efcd9bac740 1 -- 172.21.15.16:0/947436283 shutdown_connections 2024-09-12T02:10:15.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.624+0000 7efcd9bac740 1 -- 172.21.15.16:0/947436283 wait complete. 2024-09-12T02:10:15.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.625+0000 7efcd9bac740 20 client.17991 trim_cache size 0 max 16384 2024-09-12T02:10:15.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.638+0000 7efcd9bac740 1 Processor -- start 2024-09-12T02:10:15.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.638+0000 7efcd9bac740 1 -- start start 2024-09-12T02:10:15.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.638+0000 7efcd9bac740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56508df265d0 0x56508df22e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:15.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.638+0000 7efcd9bac740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56508df27b40 0x56508df3d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:15.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.638+0000 7efcd9bac740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56508df31370 0x56508df29120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:15.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.638+0000 7efcd9bac740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7efcd01b1e00 con 0x56508df265d0 2024-09-12T02:10:15.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.638+0000 7efcd9bac740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7efcd01af6d0 con 0x56508df27b40 2024-09-12T02:10:15.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.638+0000 7efcd9bac740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7efcd01ad520 con 0x56508df31370 2024-09-12T02:10:15.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.638+0000 7efcd556f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56508df265d0 0x56508df22e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:15.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.638+0000 7efcd556f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56508df265d0 0x56508df22e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45840/0 (socket says 172.21.15.16:45840) 2024-09-12T02:10:15.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.638+0000 7efcd556f640 1 -- 172.21.15.16:0/685963100 learned_addr learned my addr 172.21.15.16:0/685963100 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:10:15.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.638+0000 7efcd4d6e640 1 --2- 172.21.15.16:0/685963100 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56508df27b40 0x56508df3d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:15.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.638+0000 7efcd5d70640 1 --2- 172.21.15.16:0/685963100 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56508df31370 0x56508df29120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:15.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.639+0000 7efcd556f640 1 -- 172.21.15.16:0/685963100 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56508df31370 msgr2=0x56508df29120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:15.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.639+0000 7efcd556f640 1 --2- 172.21.15.16:0/685963100 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56508df31370 0x56508df29120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:15.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.639+0000 7efcd556f640 1 -- 172.21.15.16:0/685963100 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56508df27b40 msgr2=0x56508df3d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:15.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.639+0000 7efcd556f640 1 --2- 172.21.15.16:0/685963100 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56508df27b40 0x56508df3d2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:15.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.639+0000 7efcd556f640 1 -- 172.21.15.16:0/685963100 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56508df22510 con 0x56508df265d0 2024-09-12T02:10:15.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.639+0000 7efcd4d6e640 1 --2- 172.21.15.16:0/685963100 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56508df27b40 0x56508df3d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:10:15.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.639+0000 7efcd5d70640 1 --2- 172.21.15.16:0/685963100 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56508df31370 0x56508df29120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:10:15.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.639+0000 7efcd556f640 1 --2- 172.21.15.16:0/685963100 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56508df265d0 0x56508df22e10 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7efcc0023040 tx=0x7efcc001f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=622e7f5b2c9e0e42 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:10:15.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.639+0000 7efcbaffd640 1 -- 172.21.15.16:0/685963100 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcc009ffe0 con 0x56508df265d0 2024-09-12T02:10:15.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.639+0000 7efcbaffd640 1 -- 172.21.15.16:0/685963100 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efcc00a48a0 con 0x56508df265d0 2024-09-12T02:10:15.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.639+0000 7efcbaffd640 1 -- 172.21.15.16:0/685963100 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcc009aba0 con 0x56508df265d0 2024-09-12T02:10:15.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.640+0000 7efcd9bac740 1 -- 172.21.15.16:0/685963100 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56508df265d0 msgr2=0x56508df22e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:15.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.640+0000 7efcd9bac740 1 --2- 172.21.15.16:0/685963100 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56508df265d0 0x56508df22e10 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7efcc0023040 tx=0x7efcc001f000 comp rx=0 tx=0).stop 2024-09-12T02:10:15.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.640+0000 7efcd9bac740 1 -- 172.21.15.16:0/685963100 shutdown_connections 2024-09-12T02:10:15.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.640+0000 7efcd9bac740 1 --2- 172.21.15.16:0/685963100 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56508df31370 0x56508df29120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:15.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.640+0000 7efcd9bac740 1 --2- 172.21.15.16:0/685963100 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56508df27b40 0x56508df3d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:15.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.640+0000 7efcd9bac740 1 --2- 172.21.15.16:0/685963100 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56508df265d0 0x56508df22e10 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:15.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.640+0000 7efcd9bac740 1 -- 172.21.15.16:0/685963100 >> 172.21.15.16:0/685963100 conn(0x56508deef310 msgr2=0x56508df22700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:10:15.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.640+0000 7efcd9bac740 1 -- 172.21.15.16:0/685963100 shutdown_connections 2024-09-12T02:10:15.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.640+0000 7efcd9bac740 1 -- 172.21.15.16:0/685963100 wait complete. 2024-09-12T02:10:15.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.640+0000 7efcd9bac740 1 -- start start 2024-09-12T02:10:15.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.640+0000 7efcd9bac740 1 Processor -- start 2024-09-12T02:10:15.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.641+0000 7efcd9bac740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56508df265d0 0x56508df39420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:15.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.641+0000 7efcd9bac740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56508df27b40 0x56508df39960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:15.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.641+0000 7efcd9bac740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56508df31370 0x56508df2a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:15.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.641+0000 7efcd9bac740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7efcd01adb90 con 0x56508df27b40 2024-09-12T02:10:15.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.641+0000 7efcd9bac740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7efcd01af000 con 0x56508df265d0 2024-09-12T02:10:15.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.641+0000 7efcd9bac740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7efcd01ae260 con 0x56508df31370 2024-09-12T02:10:15.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.641+0000 7efcd9bac740 10 client.0 _ll_register_callbacks cb 0x7efcc00d2b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:10:15.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.641+0000 7efcd4d6e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56508df27b40 0x56508df39960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:15.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.641+0000 7efcd4d6e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56508df27b40 0x56508df39960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45846/0 (socket says 172.21.15.16:45846) 2024-09-12T02:10:15.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.641+0000 7efcd4d6e640 1 -- 172.21.15.16:0/931575141 learned_addr learned my addr 172.21.15.16:0/931575141 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:10:15.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.641+0000 7efcd5d70640 1 --2- 172.21.15.16:0/931575141 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56508df31370 0x56508df2a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:15.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.641+0000 7efcd556f640 1 --2- 172.21.15.16:0/931575141 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56508df265d0 0x56508df39420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:15.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.641+0000 7efcd4d6e640 1 -- 172.21.15.16:0/931575141 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56508df31370 msgr2=0x56508df2a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:15.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.641+0000 7efcd4d6e640 1 --2- 172.21.15.16:0/931575141 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56508df31370 0x56508df2a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:15.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.641+0000 7efcd4d6e640 1 -- 172.21.15.16:0/931575141 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56508df265d0 msgr2=0x56508df39420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:15.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.641+0000 7efcd4d6e640 1 --2- 172.21.15.16:0/931575141 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56508df265d0 0x56508df39420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:15.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.641+0000 7efcd4d6e640 1 -- 172.21.15.16:0/931575141 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56508df39fd0 con 0x56508df27b40 2024-09-12T02:10:15.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.641+0000 7efcd556f640 1 --2- 172.21.15.16:0/931575141 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56508df265d0 0x56508df39420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:10:15.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.641+0000 7efcd4d6e640 1 --2- 172.21.15.16:0/931575141 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56508df27b40 0x56508df39960 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x56508df2d880 tx=0x7efccc009ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8381e5dd7c65fdc server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:10:15.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.641+0000 7efcb8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:10:15.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.642+0000 7efcb8ff9640 1 -- 172.21.15.16:0/931575141 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efccc005030 con 0x56508df27b40 2024-09-12T02:10:15.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.642+0000 7efcb8ff9640 1 -- 172.21.15.16:0/931575141 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7efccc0051d0 con 0x56508df27b40 2024-09-12T02:10:15.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.642+0000 7efcd9bac740 1 -- 172.21.15.16:0/931575141 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efcc00a48a0 con 0x56508df27b40 2024-09-12T02:10:15.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.642+0000 7efcb8ff9640 1 -- 172.21.15.16:0/931575141 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7efccc0053d0 con 0x56508df27b40 2024-09-12T02:10:15.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.642+0000 7efcd9bac740 10 client.17997 fetch_fsmap learned FSMap version 25 2024-09-12T02:10:15.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.642+0000 7efcd9bac740 1 -- 172.21.15.16:0/931575141 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56508dfab1b0 con 0x56508df27b40 2024-09-12T02:10:15.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.642+0000 7efcb8ff9640 1 -- 172.21.15.16:0/931575141 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efcc00a48a0 con 0x56508df27b40 2024-09-12T02:10:15.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.642+0000 7efcb8ff9640 1 -- 172.21.15.16:0/931575141 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7efccc003a70 con 0x56508df27b40 2024-09-12T02:10:15.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.642+0000 7efcd9bac740 10 client.17997 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:10:15.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.642+0000 7efcd9bac740 10 client.17997 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-12T02:10:15.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.642+0000 7efcd9bac740 20 client.17997 populate_metadata read hostname 'smithi016' 2024-09-12T02:10:15.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.642+0000 7efcd9bac740 1 --2- 172.21.15.16:0/931575141 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x56508dfab990 0x56508dfadd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:15.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.642+0000 7efcd9bac740 4 client.17997 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T02:10:15.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.642+0000 7efcd9bac740 1 -- 172.21.15.16:0/931575141 --> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56508de93d70 con 0x56508dfab990 2024-09-12T02:10:15.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.643+0000 7efcd556f640 1 --2- 172.21.15.16:0/931575141 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x56508dfab990 0x56508dfadd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:15.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.643+0000 7efcd556f640 1 --2- 172.21.15.16:0/931575141 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x56508dfab990 0x56508dfadd80 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5d030ad4716d44ee server_cookie=83f35c152c79aff6 in_seq=0 out_seq=0 2024-09-12T02:10:15.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.643+0000 7efcb8ff9640 10 client.17997 ms_handle_connect on v2:172.21.15.184:6836/1793393087 2024-09-12T02:10:15.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.643+0000 7efcb8ff9640 1 -- 172.21.15.16:0/931575141 <== mds.2 v2:172.21.15.184:6836/1793393087 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56508de93d70 con 0x56508dfab990 2024-09-12T02:10:15.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.643+0000 7efcb8ff9640 10 client.17997 handle_command_reply: tid=0 2024-09-12T02:10:15.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.643+0000 7efcd9bac740 1 client.17997 shutdown 2024-09-12T02:10:15.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.643+0000 7efcd9bac740 1 -- 172.21.15.16:0/931575141 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56508df27b40 msgr2=0x56508df39960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:15.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.643+0000 7efcd9bac740 1 --2- 172.21.15.16:0/931575141 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56508df27b40 0x56508df39960 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x56508df2d880 tx=0x7efccc009ef0 comp rx=0 tx=0).stop 2024-09-12T02:10:15.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.643+0000 7efcd9bac740 1 -- 172.21.15.16:0/931575141 shutdown_connections 2024-09-12T02:10:15.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.643+0000 7efcd9bac740 1 --2- 172.21.15.16:0/931575141 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56508df31370 0x56508df2a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:15.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.643+0000 7efcd9bac740 1 --2- 172.21.15.16:0/931575141 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x56508dfab990 0x56508dfadd80 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:15.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.643+0000 7efcd9bac740 1 --2- 172.21.15.16:0/931575141 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56508df27b40 0x56508df39960 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:15.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.643+0000 7efcd9bac740 1 --2- 172.21.15.16:0/931575141 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56508df265d0 0x56508df39420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:15.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.643+0000 7efcd9bac740 1 -- 172.21.15.16:0/931575141 >> 172.21.15.16:0/931575141 conn(0x56508deef310 msgr2=0x56508df2cce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:10:15.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.643+0000 7efcb8ff9640 0 client.17997 ms_handle_reset on v2:172.21.15.184:6836/1793393087 2024-09-12T02:10:15.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.644+0000 7efcd9bac740 1 -- 172.21.15.16:0/931575141 shutdown_connections 2024-09-12T02:10:15.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.644+0000 7efcd9bac740 1 -- 172.21.15.16:0/931575141 wait complete. 2024-09-12T02:10:15.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.644+0000 7efcd9bac740 20 client.17997 trim_cache size 0 max 16384 2024-09-12T02:10:15.660 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T02:10:15.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.644+0000 7efcd77d3640 1 -- 172.21.15.16:0/803626164 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7efcbc0a2b80 msgr2=0x7efcbc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:15.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.644+0000 7efcd77d3640 1 --2- 172.21.15.16:0/803626164 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7efcbc0a2b80 0x7efcbc0a5040 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x56508df27070 tx=0x7efccc005f90 comp rx=0 tx=0).stop 2024-09-12T02:10:15.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.644+0000 7efcd77d3640 1 -- 172.21.15.16:0/803626164 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efcd0100c10 msgr2=0x7efcd019a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:15.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.644+0000 7efcd77d3640 1 --2- 172.21.15.16:0/803626164 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efcd0100c10 0x7efcd019a5f0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7efcc000e9f0 tx=0x7efcc000eeb0 comp rx=0 tx=0).stop 2024-09-12T02:10:15.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.644+0000 7efcd77d3640 1 -- 172.21.15.16:0/803626164 shutdown_connections 2024-09-12T02:10:15.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.644+0000 7efcd77d3640 1 --2- 172.21.15.16:0/803626164 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7efcbc0a2b80 0x7efcbc0a5040 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:15.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.644+0000 7efcd77d3640 1 --2- 172.21.15.16:0/803626164 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efcd0102ea0 0x7efcd019eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:15.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.644+0000 7efcd77d3640 1 --2- 172.21.15.16:0/803626164 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efcd0102ac0 0x7efcd019ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:15.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.644+0000 7efcd77d3640 1 --2- 172.21.15.16:0/803626164 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efcd0100c10 0x7efcd019a5f0 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:15.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.644+0000 7efcd77d3640 1 -- 172.21.15.16:0/803626164 >> 172.21.15.16:0/803626164 conn(0x7efcd0077210 msgr2=0x7efcd00fd890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:10:15.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.644+0000 7efcd77d3640 1 -- 172.21.15.16:0/803626164 shutdown_connections 2024-09-12T02:10:15.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:15.644+0000 7efcd77d3640 1 -- 172.21.15.16:0/803626164 wait complete. 2024-09-12T02:10:15.934 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:10:15 smithi016 ceph-mon[30947]: pgmap v585: 97 pgs: 97 active+clean; 4.2 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 8.2 KiB/s wr, 10 op/s 2024-09-12T02:10:16.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:10:15 smithi184 ceph-mon[35593]: pgmap v585: 97 pgs: 97 active+clean; 4.2 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 8.2 KiB/s wr, 10 op/s 2024-09-12T02:10:16.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:10:15 smithi033 ceph-mon[35332]: pgmap v585: 97 pgs: 97 active+clean; 4.2 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 8.2 KiB/s wr, 10 op/s 2024-09-12T02:10:16.384 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-12T02:10:16.653 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:10:17.095 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.093+0000 7f9f6093b640 1 -- 172.21.15.16:0/796588667 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f5c1025f0 msgr2=0x7f9f5c108e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:17.095 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.093+0000 7f9f6093b640 1 --2- 172.21.15.16:0/796588667 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f5c1025f0 0x7f9f5c108e80 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f9f54009ca0 tx=0x7f9f5402a8c0 comp rx=0 tx=0).stop 2024-09-12T02:10:17.095 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.093+0000 7f9f6093b640 1 -- 172.21.15.16:0/796588667 shutdown_connections 2024-09-12T02:10:17.095 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.093+0000 7f9f6093b640 1 --2- 172.21.15.16:0/796588667 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f5c1025f0 0x7f9f5c108e80 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:17.095 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.093+0000 7f9f6093b640 1 --2- 172.21.15.16:0/796588667 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9f5c101c30 0x7f9f5c1020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:17.095 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.093+0000 7f9f6093b640 1 --2- 172.21.15.16:0/796588667 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f5c100a40 0x7f9f5c100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:17.096 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.093+0000 7f9f6093b640 1 -- 172.21.15.16:0/796588667 >> 172.21.15.16:0/796588667 conn(0x7f9f5c0fc1e0 msgr2=0x7f9f5c0fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:10:17.096 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.093+0000 7f9f6093b640 1 -- 172.21.15.16:0/796588667 shutdown_connections 2024-09-12T02:10:17.096 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.093+0000 7f9f6093b640 1 -- 172.21.15.16:0/796588667 wait complete. 2024-09-12T02:10:17.097 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.094+0000 7f9f6093b640 1 Processor -- start 2024-09-12T02:10:17.097 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.094+0000 7f9f6093b640 1 -- start start 2024-09-12T02:10:17.097 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.094+0000 7f9f6093b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f5c100a40 0x7f9f5c06a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:17.097 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.095+0000 7f9f6093b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f5c101c30 0x7f9f5c19cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:17.097 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.095+0000 7f9f6093b640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9f5c1025f0 0x7f9f5c19f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:17.097 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.095+0000 7f9f6093b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9f5c10c720 con 0x7f9f5c101c30 2024-09-12T02:10:17.097 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.095+0000 7f9f6093b640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9f5c10c5a0 con 0x7f9f5c100a40 2024-09-12T02:10:17.097 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.095+0000 7f9f6093b640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f9f5c10c8a0 con 0x7f9f5c1025f0 2024-09-12T02:10:17.098 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.095+0000 7f9f59d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f5c101c30 0x7f9f5c19cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:17.099 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.095+0000 7f9f59d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f5c101c30 0x7f9f5c19cb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45872/0 (socket says 172.21.15.16:45872) 2024-09-12T02:10:17.099 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.095+0000 7f9f59d9b640 1 -- 172.21.15.16:0/158459585 learned_addr learned my addr 172.21.15.16:0/158459585 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:10:17.099 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.095+0000 7f9f5a59c640 1 --2- 172.21.15.16:0/158459585 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f5c100a40 0x7f9f5c06a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:17.099 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.095+0000 7f9f5ad9d640 1 --2- 172.21.15.16:0/158459585 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9f5c1025f0 0x7f9f5c19f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:17.099 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.095+0000 7f9f59d9b640 1 -- 172.21.15.16:0/158459585 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9f5c1025f0 msgr2=0x7f9f5c19f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:17.099 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.095+0000 7f9f59d9b640 1 --2- 172.21.15.16:0/158459585 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9f5c1025f0 0x7f9f5c19f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:17.099 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.095+0000 7f9f59d9b640 1 -- 172.21.15.16:0/158459585 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f5c100a40 msgr2=0x7f9f5c06a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:17.099 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.095+0000 7f9f59d9b640 1 --2- 172.21.15.16:0/158459585 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f5c100a40 0x7f9f5c06a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:17.099 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.095+0000 7f9f59d9b640 1 -- 172.21.15.16:0/158459585 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f5c19fa60 con 0x7f9f5c101c30 2024-09-12T02:10:17.099 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.095+0000 7f9f5ad9d640 1 --2- 172.21.15.16:0/158459585 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9f5c1025f0 0x7f9f5c19f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:10:17.099 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.095+0000 7f9f59d9b640 1 --2- 172.21.15.16:0/158459585 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f5c101c30 0x7f9f5c19cb30 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f9f5000b4a0 tx=0x7f9f5000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:10:17.099 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.095+0000 7f9f4b7fe640 1 -- 172.21.15.16:0/158459585 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f50014070 con 0x7f9f5c101c30 2024-09-12T02:10:17.100 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.095+0000 7f9f6093b640 1 -- 172.21.15.16:0/158459585 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f5c19fcf0 con 0x7f9f5c101c30 2024-09-12T02:10:17.100 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.095+0000 7f9f4b7fe640 1 -- 172.21.15.16:0/158459585 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f50010af0 con 0x7f9f5c101c30 2024-09-12T02:10:17.100 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.095+0000 7f9f4b7fe640 1 -- 172.21.15.16:0/158459585 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f50017d10 con 0x7f9f5c101c30 2024-09-12T02:10:17.100 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.095+0000 7f9f6093b640 1 -- 172.21.15.16:0/158459585 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f5c1a0260 con 0x7f9f5c101c30 2024-09-12T02:10:17.107 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.098+0000 7f9f4b7fe640 1 -- 172.21.15.16:0/158459585 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f9f50017070 con 0x7f9f5c101c30 2024-09-12T02:10:17.107 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.098+0000 7f9f636d7740 1 Processor -- start 2024-09-12T02:10:17.107 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.098+0000 7f9f636d7740 1 -- start start 2024-09-12T02:10:17.107 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.099+0000 7f9f636d7740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562f3dc549a0 0x562f3dc54da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:17.107 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.099+0000 7f9f636d7740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562f3dc552e0 0x562f3dc59da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:17.107 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.099+0000 7f9f636d7740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f3dc5a370 0x562f3dc5c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:17.107 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.099+0000 7f9f636d7740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x562f3dbb82f0 con 0x562f3dc5a370 2024-09-12T02:10:17.107 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.099+0000 7f9f636d7740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x562f3db60270 con 0x562f3dc552e0 2024-09-12T02:10:17.107 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.099+0000 7f9f636d7740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x562f3dbb42b0 con 0x562f3dc549a0 2024-09-12T02:10:17.107 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.099+0000 7f9f59d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f3dc5a370 0x562f3dc5c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:17.108 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.099+0000 7f9f5a59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562f3dc552e0 0x562f3dc59da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:17.108 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.099+0000 7f9f59d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f3dc5a370 0x562f3dc5c830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45882/0 (socket says 172.21.15.16:45882) 2024-09-12T02:10:17.108 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.099+0000 7f9f59d9b640 1 -- 172.21.15.16:0/1358686521 learned_addr learned my addr 172.21.15.16:0/1358686521 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:10:17.108 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.099+0000 7f9f5ad9d640 1 --2- 172.21.15.16:0/1358686521 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562f3dc549a0 0x562f3dc54da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:17.108 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.099+0000 7f9f4b7fe640 1 --2- 172.21.15.16:0/158459585 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f9f440a2c30 0x7f9f440a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:17.108 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.099+0000 7f9f5a59c640 1 -- 172.21.15.16:0/1358686521 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562f3dc549a0 msgr2=0x562f3dc54da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:17.108 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.099+0000 7f9f5a59c640 1 --2- 172.21.15.16:0/1358686521 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562f3dc549a0 0x562f3dc54da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:17.108 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.099+0000 7f9f5a59c640 1 -- 172.21.15.16:0/1358686521 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f3dc5a370 msgr2=0x562f3dc5c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:17.108 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.099+0000 7f9f5a59c640 1 --2- 172.21.15.16:0/1358686521 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f3dc5a370 0x562f3dc5c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:17.108 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.099+0000 7f9f5a59c640 1 -- 172.21.15.16:0/1358686521 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f3db5f340 con 0x562f3dc552e0 2024-09-12T02:10:17.108 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.099+0000 7f9f59d9b640 1 --2- 172.21.15.16:0/1358686521 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f3dc5a370 0x562f3dc5c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:10:17.108 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.099+0000 7f9f5ad9d640 1 --2- 172.21.15.16:0/158459585 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f9f440a2c30 0x7f9f440a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:17.109 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.099+0000 7f9f4b7fe640 1 -- 172.21.15.16:0/158459585 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f9f500ce2c0 con 0x7f9f5c101c30 2024-09-12T02:10:17.109 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.099+0000 7f9f5a59c640 1 --2- 172.21.15.16:0/1358686521 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562f3dc552e0 0x562f3dc59da0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c00ab20 tx=0x7f9f4c00f3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9dd397e9e5e91513 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:10:17.109 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.100+0000 7f9f48ff9640 1 -- 172.21.15.16:0/1358686521 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f4c004020 con 0x562f3dc552e0 2024-09-12T02:10:17.109 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.100+0000 7f9f5ad9d640 1 --2- 172.21.15.16:0/158459585 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f9f440a2c30 0x7f9f440a50f0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f9f54031ae0 tx=0x7f9f5403e040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:10:17.109 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.100+0000 7f9f48ff9640 1 -- 172.21.15.16:0/1358686521 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f4c005500 con 0x562f3dc552e0 2024-09-12T02:10:17.109 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.100+0000 7f9f48ff9640 1 -- 172.21.15.16:0/1358686521 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f4c004e00 con 0x562f3dc552e0 2024-09-12T02:10:17.109 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.100+0000 7f9f636d7740 1 -- 172.21.15.16:0/1358686521 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562f3dc552e0 msgr2=0x562f3dc59da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:17.109 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.100+0000 7f9f636d7740 1 --2- 172.21.15.16:0/1358686521 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562f3dc552e0 0x562f3dc59da0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c00ab20 tx=0x7f9f4c00f3c0 comp rx=0 tx=0).stop 2024-09-12T02:10:17.109 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.100+0000 7f9f636d7740 1 -- 172.21.15.16:0/1358686521 shutdown_connections 2024-09-12T02:10:17.109 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.100+0000 7f9f636d7740 1 --2- 172.21.15.16:0/1358686521 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f3dc5a370 0x562f3dc5c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:17.109 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.100+0000 7f9f636d7740 1 --2- 172.21.15.16:0/1358686521 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562f3dc552e0 0x562f3dc59da0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:17.109 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.100+0000 7f9f636d7740 1 --2- 172.21.15.16:0/1358686521 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562f3dc549a0 0x562f3dc54da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:17.109 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.100+0000 7f9f636d7740 1 -- 172.21.15.16:0/1358686521 >> 172.21.15.16:0/1358686521 conn(0x562f3dc50010 msgr2=0x562f3dc50410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:10:17.109 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.101+0000 7f9f636d7740 1 -- 172.21.15.16:0/1358686521 shutdown_connections 2024-09-12T02:10:17.110 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.101+0000 7f9f636d7740 1 -- 172.21.15.16:0/1358686521 wait complete. 2024-09-12T02:10:17.110 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.101+0000 7f9f636d7740 1 -- start start 2024-09-12T02:10:17.110 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.101+0000 7f9f636d7740 1 Processor -- start 2024-09-12T02:10:17.110 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.101+0000 7f9f636d7740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f3dc549a0 0x562f3dc5eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:17.110 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.101+0000 7f9f636d7740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562f3dc552e0 0x562f3dc5f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:17.110 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.101+0000 7f9f636d7740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562f3dc5a370 0x562f3dc660e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:17.110 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.101+0000 7f9f636d7740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x562f3dc5d340 con 0x562f3dc549a0 2024-09-12T02:10:17.110 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.101+0000 7f9f636d7740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x562f3db41290 con 0x562f3dc5a370 2024-09-12T02:10:17.110 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.101+0000 7f9f636d7740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x562f3dc5d4c0 con 0x562f3dc552e0 2024-09-12T02:10:17.110 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.101+0000 7f9f636d7740 10 client.0 _ll_register_callbacks cb 0x562f3db72140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:10:17.110 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.101+0000 7f9f5a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f3dc549a0 0x562f3dc5eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:17.110 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.101+0000 7f9f5a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f3dc549a0 0x562f3dc5eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45890/0 (socket says 172.21.15.16:45890) 2024-09-12T02:10:17.110 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.101+0000 7f9f5a59c640 1 -- 172.21.15.16:0/3096417235 learned_addr learned my addr 172.21.15.16:0/3096417235 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:10:17.111 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.101+0000 7f9f5ad9d640 1 --2- 172.21.15.16:0/3096417235 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562f3dc5a370 0x562f3dc660e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:17.111 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.102+0000 7f9f59d9b640 1 --2- 172.21.15.16:0/3096417235 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562f3dc552e0 0x562f3dc5f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:17.111 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.102+0000 7f9f5a59c640 1 -- 172.21.15.16:0/3096417235 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562f3dc552e0 msgr2=0x562f3dc5f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:17.111 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.102+0000 7f9f5a59c640 1 --2- 172.21.15.16:0/3096417235 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562f3dc552e0 0x562f3dc5f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:17.111 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.102+0000 7f9f5a59c640 1 -- 172.21.15.16:0/3096417235 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562f3dc5a370 msgr2=0x562f3dc660e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:17.111 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.102+0000 7f9f5a59c640 1 --2- 172.21.15.16:0/3096417235 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562f3dc5a370 0x562f3dc660e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:17.111 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.102+0000 7f9f5a59c640 1 -- 172.21.15.16:0/3096417235 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f3d8cb210 con 0x562f3dc549a0 2024-09-12T02:10:17.111 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.102+0000 7f9f5a59c640 1 --2- 172.21.15.16:0/3096417235 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f3dc549a0 0x562f3dc5eb60 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c01a040 tx=0x7f9f4c005290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8cead710e81b002 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:10:17.111 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.102+0000 7f9f41ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:10:17.111 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.102+0000 7f9f41ffb640 1 -- 172.21.15.16:0/3096417235 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f4c027020 con 0x562f3dc549a0 2024-09-12T02:10:17.111 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.102+0000 7f9f41ffb640 1 -- 172.21.15.16:0/3096417235 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f9f4c0046c0 con 0x562f3dc549a0 2024-09-12T02:10:17.111 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.102+0000 7f9f636d7740 1 -- 172.21.15.16:0/3096417235 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f4c005500 con 0x562f3dc549a0 2024-09-12T02:10:17.111 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.102+0000 7f9f41ffb640 1 -- 172.21.15.16:0/3096417235 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9f4c01c030 con 0x562f3dc549a0 2024-09-12T02:10:17.112 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.102+0000 7f9f636d7740 10 client.18009 fetch_fsmap learned FSMap version 25 2024-09-12T02:10:17.112 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.102+0000 7f9f636d7740 1 -- 172.21.15.16:0/3096417235 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x562f3d997620 con 0x562f3dc549a0 2024-09-12T02:10:17.112 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.102+0000 7f9f41ffb640 1 -- 172.21.15.16:0/3096417235 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f4c005500 con 0x562f3dc549a0 2024-09-12T02:10:17.112 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.103+0000 7f9f41ffb640 1 -- 172.21.15.16:0/3096417235 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f9f4c022070 con 0x562f3dc549a0 2024-09-12T02:10:17.112 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.103+0000 7f9f636d7740 10 client.18009 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:10:17.112 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.103+0000 7f9f636d7740 10 client.18009 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:10:17.112 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.103+0000 7f9f636d7740 20 client.18009 populate_metadata read hostname 'smithi016' 2024-09-12T02:10:17.112 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.103+0000 7f9f636d7740 1 --2- 172.21.15.16:0/3096417235 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x562f3dc6a500 0x562f3dc6c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:17.112 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.103+0000 7f9f636d7740 4 client.18009 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:10:17.112 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.103+0000 7f9f636d7740 1 -- 172.21.15.16:0/3096417235 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562f3dbc5a20 con 0x562f3dc6a500 2024-09-12T02:10:17.112 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.103+0000 7f9f59d9b640 1 --2- 172.21.15.16:0/3096417235 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x562f3dc6a500 0x562f3dc6c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:17.112 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.104+0000 7f9f59d9b640 1 --2- 172.21.15.16:0/3096417235 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x562f3dc6a500 0x562f3dc6c8f0 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d73f17f1819d649 server_cookie=2e93b6200ce5b853 in_seq=0 out_seq=0 2024-09-12T02:10:17.113 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.104+0000 7f9f41ffb640 10 client.18009 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:10:17.113 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.104+0000 7f9f41ffb640 1 -- 172.21.15.16:0/3096417235 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562f3dbc5a20 con 0x562f3dc6a500 2024-09-12T02:10:17.113 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.104+0000 7f9f41ffb640 10 client.18009 handle_command_reply: tid=0 2024-09-12T02:10:17.113 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.104+0000 7f9f636d7740 1 client.18009 shutdown 2024-09-12T02:10:17.113 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.104+0000 7f9f636d7740 1 -- 172.21.15.16:0/3096417235 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f3dc549a0 msgr2=0x562f3dc5eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:17.113 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.104+0000 7f9f636d7740 1 --2- 172.21.15.16:0/3096417235 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f3dc549a0 0x562f3dc5eb60 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c01a040 tx=0x7f9f4c005290 comp rx=0 tx=0).stop 2024-09-12T02:10:17.113 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.104+0000 7f9f636d7740 1 -- 172.21.15.16:0/3096417235 shutdown_connections 2024-09-12T02:10:17.113 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.104+0000 7f9f636d7740 1 --2- 172.21.15.16:0/3096417235 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x562f3dc6a500 0x562f3dc6c8f0 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:17.113 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.104+0000 7f9f636d7740 1 --2- 172.21.15.16:0/3096417235 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562f3dc5a370 0x562f3dc660e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:17.113 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.104+0000 7f9f636d7740 1 --2- 172.21.15.16:0/3096417235 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562f3dc552e0 0x562f3dc5f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:17.113 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.104+0000 7f9f636d7740 1 --2- 172.21.15.16:0/3096417235 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f3dc549a0 0x562f3dc5eb60 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:17.113 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.104+0000 7f9f636d7740 1 -- 172.21.15.16:0/3096417235 >> 172.21.15.16:0/3096417235 conn(0x562f3dc50010 msgr2=0x562f3dc51820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:10:17.113 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.104+0000 7f9f41ffb640 0 client.18009 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:10:17.114 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.105+0000 7f9f636d7740 1 -- 172.21.15.16:0/3096417235 shutdown_connections 2024-09-12T02:10:17.114 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.105+0000 7f9f636d7740 1 -- 172.21.15.16:0/3096417235 wait complete. 2024-09-12T02:10:17.114 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.105+0000 7f9f636d7740 20 client.18009 trim_cache size 0 max 16384 2024-09-12T02:10:17.125 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.119+0000 7f9f636d7740 1 Processor -- start 2024-09-12T02:10:17.125 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.119+0000 7f9f636d7740 1 -- start start 2024-09-12T02:10:17.126 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.119+0000 7f9f636d7740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562f3dc549a0 0x562f3dc503f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:17.126 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.119+0000 7f9f636d7740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f3dc552e0 0x562f3dc66000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:17.126 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.119+0000 7f9f636d7740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562f3dc5a370 0x562f3dc50930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:17.126 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.119+0000 7f9f636d7740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9f5c1ab7b0 con 0x562f3dc552e0 2024-09-12T02:10:17.126 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.119+0000 7f9f636d7740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9f5c1a9080 con 0x562f3dc549a0 2024-09-12T02:10:17.126 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.119+0000 7f9f636d7740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f9f5c1a6ef0 con 0x562f3dc5a370 2024-09-12T02:10:17.126 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.119+0000 7f9f59d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f3dc552e0 0x562f3dc66000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:17.126 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.119+0000 7f9f59d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f3dc552e0 0x562f3dc66000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45904/0 (socket says 172.21.15.16:45904) 2024-09-12T02:10:17.126 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.119+0000 7f9f59d9b640 1 -- 172.21.15.16:0/4131660639 learned_addr learned my addr 172.21.15.16:0/4131660639 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:10:17.126 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.119+0000 7f9f5ad9d640 1 --2- 172.21.15.16:0/4131660639 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562f3dc5a370 0x562f3dc50930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:17.126 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.119+0000 7f9f5a59c640 1 --2- 172.21.15.16:0/4131660639 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562f3dc549a0 0x562f3dc503f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:17.126 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.120+0000 7f9f59d9b640 1 -- 172.21.15.16:0/4131660639 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562f3dc5a370 msgr2=0x562f3dc50930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:17.127 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.120+0000 7f9f59d9b640 1 --2- 172.21.15.16:0/4131660639 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562f3dc5a370 0x562f3dc50930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:17.127 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.120+0000 7f9f59d9b640 1 -- 172.21.15.16:0/4131660639 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562f3dc549a0 msgr2=0x562f3dc503f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:17.127 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.120+0000 7f9f59d9b640 1 --2- 172.21.15.16:0/4131660639 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562f3dc549a0 0x562f3dc503f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:17.127 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.120+0000 7f9f59d9b640 1 -- 172.21.15.16:0/4131660639 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f3dc4b2d0 con 0x562f3dc552e0 2024-09-12T02:10:17.127 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.120+0000 7f9f59d9b640 1 --2- 172.21.15.16:0/4131660639 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f3dc552e0 0x562f3dc66000 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f9f5009a8c0 tx=0x7f9f500ce5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bac1763bfb6fae19 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:10:17.127 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.120+0000 7f9f43fff640 1 -- 172.21.15.16:0/4131660639 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f5001e040 con 0x562f3dc552e0 2024-09-12T02:10:17.127 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.120+0000 7f9f43fff640 1 -- 172.21.15.16:0/4131660639 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f5009b2c0 con 0x562f3dc552e0 2024-09-12T02:10:17.127 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.120+0000 7f9f43fff640 1 -- 172.21.15.16:0/4131660639 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f50098f50 con 0x562f3dc552e0 2024-09-12T02:10:17.127 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.121+0000 7f9f636d7740 1 -- 172.21.15.16:0/4131660639 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f3dc552e0 msgr2=0x562f3dc66000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:17.127 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.121+0000 7f9f636d7740 1 --2- 172.21.15.16:0/4131660639 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f3dc552e0 0x562f3dc66000 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f9f5009a8c0 tx=0x7f9f500ce5c0 comp rx=0 tx=0).stop 2024-09-12T02:10:17.127 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.121+0000 7f9f636d7740 1 -- 172.21.15.16:0/4131660639 shutdown_connections 2024-09-12T02:10:17.127 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.121+0000 7f9f636d7740 1 --2- 172.21.15.16:0/4131660639 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562f3dc5a370 0x562f3dc50930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:17.127 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.121+0000 7f9f636d7740 1 --2- 172.21.15.16:0/4131660639 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f3dc552e0 0x562f3dc66000 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:17.128 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.121+0000 7f9f636d7740 1 --2- 172.21.15.16:0/4131660639 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562f3dc549a0 0x562f3dc503f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:17.128 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.121+0000 7f9f636d7740 1 -- 172.21.15.16:0/4131660639 >> 172.21.15.16:0/4131660639 conn(0x562f3dc50010 msgr2=0x562f3dc5e910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:10:17.128 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.121+0000 7f9f636d7740 1 -- 172.21.15.16:0/4131660639 shutdown_connections 2024-09-12T02:10:17.128 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.121+0000 7f9f636d7740 1 -- 172.21.15.16:0/4131660639 wait complete. 2024-09-12T02:10:17.128 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.121+0000 7f9f636d7740 1 -- start start 2024-09-12T02:10:17.128 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.121+0000 7f9f636d7740 1 Processor -- start 2024-09-12T02:10:17.128 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.121+0000 7f9f636d7740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562f3dc549a0 0x562f3dc63290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:17.128 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.121+0000 7f9f636d7740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f3dc552e0 0x562f3dc637d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:17.128 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.121+0000 7f9f636d7740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562f3dc5a370 0x562f3dc64790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:17.128 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.121+0000 7f9f636d7740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9f5c1a7540 con 0x562f3dc552e0 2024-09-12T02:10:17.128 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.121+0000 7f9f636d7740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9f5c1a89b0 con 0x562f3dc549a0 2024-09-12T02:10:17.128 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.121+0000 7f9f636d7740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f9f5c1a7c10 con 0x562f3dc5a370 2024-09-12T02:10:17.128 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.121+0000 7f9f636d7740 10 client.0 _ll_register_callbacks cb 0x7f9f500d1cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:10:17.129 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.121+0000 7f9f59d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f3dc552e0 0x562f3dc637d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:17.129 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.122+0000 7f9f59d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f3dc552e0 0x562f3dc637d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45916/0 (socket says 172.21.15.16:45916) 2024-09-12T02:10:17.129 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.122+0000 7f9f59d9b640 1 -- 172.21.15.16:0/86655248 learned_addr learned my addr 172.21.15.16:0/86655248 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:10:17.129 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.122+0000 7f9f5a59c640 1 --2- 172.21.15.16:0/86655248 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562f3dc549a0 0x562f3dc63290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:17.129 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.122+0000 7f9f5ad9d640 1 --2- 172.21.15.16:0/86655248 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562f3dc5a370 0x562f3dc64790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:17.129 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.122+0000 7f9f59d9b640 1 -- 172.21.15.16:0/86655248 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562f3dc5a370 msgr2=0x562f3dc64790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:17.129 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.122+0000 7f9f59d9b640 1 --2- 172.21.15.16:0/86655248 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562f3dc5a370 0x562f3dc64790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:17.129 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.122+0000 7f9f59d9b640 1 -- 172.21.15.16:0/86655248 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562f3dc549a0 msgr2=0x562f3dc63290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:17.129 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.122+0000 7f9f59d9b640 1 --2- 172.21.15.16:0/86655248 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562f3dc549a0 0x562f3dc63290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:17.129 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.122+0000 7f9f59d9b640 1 -- 172.21.15.16:0/86655248 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f3dc5a0d0 con 0x562f3dc552e0 2024-09-12T02:10:17.129 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.122+0000 7f9f59d9b640 1 --2- 172.21.15.16:0/86655248 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f3dc552e0 0x562f3dc637d0 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f9f5000be60 tx=0x7f9f5009abc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d43b341ea4cc1680 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:10:17.129 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.122+0000 7f9f41ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:10:17.130 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.122+0000 7f9f41ffb640 1 -- 172.21.15.16:0/86655248 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f5001e070 con 0x562f3dc552e0 2024-09-12T02:10:17.130 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.122+0000 7f9f41ffb640 1 -- 172.21.15.16:0/86655248 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f9f50017360 con 0x562f3dc552e0 2024-09-12T02:10:17.130 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.122+0000 7f9f636d7740 1 -- 172.21.15.16:0/86655248 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f5009b2c0 con 0x562f3dc552e0 2024-09-12T02:10:17.130 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.122+0000 7f9f41ffb640 1 -- 172.21.15.16:0/86655248 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9f5009ef70 con 0x562f3dc552e0 2024-09-12T02:10:17.130 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.122+0000 7f9f636d7740 10 client.18015 fetch_fsmap learned FSMap version 25 2024-09-12T02:10:17.130 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.122+0000 7f9f636d7740 1 -- 172.21.15.16:0/86655248 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x562f3dc65f50 con 0x562f3dc552e0 2024-09-12T02:10:17.131 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.123+0000 7f9f41ffb640 1 -- 172.21.15.16:0/86655248 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f5009b2c0 con 0x562f3dc552e0 2024-09-12T02:10:17.131 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.123+0000 7f9f41ffb640 1 -- 172.21.15.16:0/86655248 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f9f5009a010 con 0x562f3dc552e0 2024-09-12T02:10:17.131 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.123+0000 7f9f636d7740 10 client.18015 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:10:17.131 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.123+0000 7f9f636d7740 10 client.18015 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:10:17.131 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.123+0000 7f9f636d7740 20 client.18015 populate_metadata read hostname 'smithi016' 2024-09-12T02:10:17.131 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.123+0000 7f9f636d7740 1 --2- 172.21.15.16:0/86655248 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x562f3dcd54d0 0x562f3dcd78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:17.131 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.123+0000 7f9f636d7740 4 client.18015 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:10:17.131 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.123+0000 7f9f636d7740 1 -- 172.21.15.16:0/86655248 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562f3dbbfd70 con 0x562f3dcd54d0 2024-09-12T02:10:17.131 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.123+0000 7f9f5a59c640 1 --2- 172.21.15.16:0/86655248 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x562f3dcd54d0 0x562f3dcd78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:17.131 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.124+0000 7f9f5a59c640 1 --2- 172.21.15.16:0/86655248 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x562f3dcd54d0 0x562f3dcd78f0 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62e1658eec20f45c server_cookie=ae5c2ca0917dc7ef in_seq=0 out_seq=0 2024-09-12T02:10:17.131 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.124+0000 7f9f41ffb640 10 client.18015 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:10:17.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.125+0000 7f9d6baee640 1 -- 172.21.15.16:0/1601349308 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d64102ef0 msgr2=0x7f9d6410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:17.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.125+0000 7f9d6baee640 1 --2- 172.21.15.16:0/1601349308 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d64102ef0 0x7f9d6410da40 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f9d5c009ca0 tx=0x7f9d5c02a950 comp rx=0 tx=0).stop 2024-09-12T02:10:17.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.125+0000 7f9d6baee640 1 -- 172.21.15.16:0/1601349308 shutdown_connections 2024-09-12T02:10:17.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.125+0000 7f9d6baee640 1 --2- 172.21.15.16:0/1601349308 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d64102ef0 0x7f9d6410da40 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:17.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.125+0000 7f9d6baee640 1 --2- 172.21.15.16:0/1601349308 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9d64102570 0x7f9d641029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:17.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.125+0000 7f9d6baee640 1 --2- 172.21.15.16:0/1601349308 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9d64100e20 0x7f9d64101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:17.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.125+0000 7f9d6baee640 1 -- 172.21.15.16:0/1601349308 >> 172.21.15.16:0/1601349308 conn(0x7f9d640fc7d0 msgr2=0x7f9d640febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:10:17.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.125+0000 7f9d6baee640 1 -- 172.21.15.16:0/1601349308 shutdown_connections 2024-09-12T02:10:17.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.125+0000 7f9d6baee640 1 -- 172.21.15.16:0/1601349308 wait complete. 2024-09-12T02:10:17.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.126+0000 7f9d6baee640 1 Processor -- start 2024-09-12T02:10:17.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.126+0000 7f9d6baee640 1 -- start start 2024-09-12T02:10:17.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.126+0000 7f9d6baee640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9d64100e20 0x7f9d64198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:17.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.126+0000 7f9d6baee640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d64102570 0x7f9d641989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:17.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.126+0000 7f9d6baee640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9d64102ef0 0x7f9d6419cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:17.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.126+0000 7f9d6baee640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9d64110f80 con 0x7f9d64102570 2024-09-12T02:10:17.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.126+0000 7f9d6baee640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9d64110e00 con 0x7f9d64100e20 2024-09-12T02:10:17.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.126+0000 7f9d6baee640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f9d64111100 con 0x7f9d64102ef0 2024-09-12T02:10:17.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.126+0000 7f9d6988a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9d64100e20 0x7f9d64198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:17.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.126+0000 7f9d69089640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d64102570 0x7f9d641989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:17.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.127+0000 7f9d69089640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d64102570 0x7f9d641989b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45932/0 (socket says 172.21.15.16:45932) 2024-09-12T02:10:17.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.127+0000 7f9d69089640 1 -- 172.21.15.16:0/1910217680 learned_addr learned my addr 172.21.15.16:0/1910217680 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:10:17.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.127+0000 7f9d6a08b640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9d64102ef0 0x7f9d6419cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:17.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.127+0000 7f9d69089640 1 -- 172.21.15.16:0/1910217680 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9d64102ef0 msgr2=0x7f9d6419cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:17.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.127+0000 7f9d69089640 1 --2- 172.21.15.16:0/1910217680 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9d64102ef0 0x7f9d6419cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:17.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.127+0000 7f9d69089640 1 -- 172.21.15.16:0/1910217680 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9d64100e20 msgr2=0x7f9d64198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:17.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.127+0000 7f9d69089640 1 --2- 172.21.15.16:0/1910217680 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9d64100e20 0x7f9d64198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:17.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.127+0000 7f9d69089640 1 -- 172.21.15.16:0/1910217680 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d6419d420 con 0x7f9d64102570 2024-09-12T02:10:17.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.127+0000 7f9d69089640 1 --2- 172.21.15.16:0/1910217680 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d64102570 0x7f9d641989b0 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f9d5400e9f0 tx=0x7f9d5400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:10:17.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.127+0000 7f9d6988a640 1 --2- 172.21.15.16:0/1910217680 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9d64100e20 0x7f9d64198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:10:17.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.128+0000 7f9d5affd640 1 -- 172.21.15.16:0/1910217680 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d5400cd80 con 0x7f9d64102570 2024-09-12T02:10:17.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.128+0000 7f9d5affd640 1 -- 172.21.15.16:0/1910217680 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d54010910 con 0x7f9d64102570 2024-09-12T02:10:17.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.128+0000 7f9d5affd640 1 -- 172.21.15.16:0/1910217680 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d540188f0 con 0x7f9d64102570 2024-09-12T02:10:17.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.128+0000 7f9d6baee640 1 -- 172.21.15.16:0/1910217680 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d6419d710 con 0x7f9d64102570 2024-09-12T02:10:17.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.128+0000 7f9d6baee640 1 -- 172.21.15.16:0/1910217680 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d641a4ff0 con 0x7f9d64102570 2024-09-12T02:10:17.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.131+0000 7f9d6dec7740 1 Processor -- start 2024-09-12T02:10:17.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.131+0000 7f9d6dec7740 1 -- start start 2024-09-12T02:10:17.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.131+0000 7f9d5affd640 1 -- 172.21.15.16:0/1910217680 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f9d54010430 con 0x7f9d64102570 2024-09-12T02:10:17.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.131+0000 7f9d6dec7740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c2d13e8b40 0x55c2d13e7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:17.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.131+0000 7f9d6dec7740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c2d13e75d0 0x55c2d13e7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:17.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.132+0000 7f9d6dec7740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c2d13f2370 0x55c2d13f4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:17.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.132+0000 7f9d6dec7740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55c2d134d2f0 con 0x55c2d13e8b40 2024-09-12T02:10:17.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.132+0000 7f9d6dec7740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55c2d12f4cf0 con 0x55c2d13f2370 2024-09-12T02:10:17.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.132+0000 7f9d6dec7740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55c2d13492b0 con 0x55c2d13e75d0 2024-09-12T02:10:17.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.132+0000 7f9d6a08b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c2d13e8b40 0x55c2d13e7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:17.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.132+0000 7f9d6a08b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c2d13e8b40 0x55c2d13e7090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45944/0 (socket says 172.21.15.16:45944) 2024-09-12T02:10:17.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.132+0000 7f9d6a08b640 1 -- 172.21.15.16:0/3739441579 learned_addr learned my addr 172.21.15.16:0/3739441579 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:10:17.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.132+0000 7f9d6988a640 1 --2- 172.21.15.16:0/3739441579 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c2d13e75d0 0x55c2d13e7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:17.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.132+0000 7f9d69089640 1 --2- 172.21.15.16:0/3739441579 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c2d13f2370 0x55c2d13f4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:17.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.132+0000 7f9d6a08b640 1 -- 172.21.15.16:0/3739441579 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c2d13e75d0 msgr2=0x55c2d13e7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:17.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.132+0000 7f9d6a08b640 1 --2- 172.21.15.16:0/3739441579 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c2d13e75d0 0x55c2d13e7a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:17.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.132+0000 7f9d6a08b640 1 -- 172.21.15.16:0/3739441579 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c2d13f2370 msgr2=0x55c2d13f4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:17.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.132+0000 7f9d6a08b640 1 --2- 172.21.15.16:0/3739441579 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c2d13f2370 0x55c2d13f4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:17.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.132+0000 7f9d6a08b640 1 -- 172.21.15.16:0/3739441579 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2d12d93a0 con 0x55c2d13e8b40 2024-09-12T02:10:17.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.132+0000 7f9d6a08b640 1 --2- 172.21.15.16:0/3739441579 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c2d13e8b40 0x55c2d13e7090 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f9d5c035000 tx=0x7f9d5c0093e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15d691531c6621f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:10:17.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.133+0000 7f9d4ffff640 1 -- 172.21.15.16:0/3739441579 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d5c048070 con 0x55c2d13e8b40 2024-09-12T02:10:17.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.133+0000 7f9d5affd640 1 --2- 172.21.15.16:0/1910217680 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f9d500a2c00 0x7f9d500a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:17.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.133+0000 7f9d4ffff640 1 -- 172.21.15.16:0/3739441579 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d5c02f930 con 0x55c2d13e8b40 2024-09-12T02:10:17.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.133+0000 7f9d4ffff640 1 -- 172.21.15.16:0/3739441579 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d5c034040 con 0x55c2d13e8b40 2024-09-12T02:10:17.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.133+0000 7f9d6988a640 1 --2- 172.21.15.16:0/1910217680 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f9d500a2c00 0x7f9d500a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:17.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.133+0000 7f9d5affd640 1 -- 172.21.15.16:0/1910217680 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f9d540cf450 con 0x7f9d64102570 2024-09-12T02:10:17.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.133+0000 7f9d6dec7740 1 -- 172.21.15.16:0/3739441579 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c2d13e8b40 msgr2=0x55c2d13e7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:17.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.133+0000 7f9d6dec7740 1 --2- 172.21.15.16:0/3739441579 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c2d13e8b40 0x55c2d13e7090 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f9d5c035000 tx=0x7f9d5c0093e0 comp rx=0 tx=0).stop 2024-09-12T02:10:17.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.133+0000 7f9d6988a640 1 --2- 172.21.15.16:0/1910217680 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f9d500a2c00 0x7f9d500a50c0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x55c2d13e4ac0 tx=0x7f9d60005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:10:17.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.134+0000 7f9d6dec7740 1 -- 172.21.15.16:0/3739441579 shutdown_connections 2024-09-12T02:10:17.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.134+0000 7f9d6dec7740 1 --2- 172.21.15.16:0/3739441579 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c2d13f2370 0x55c2d13f4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:17.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.134+0000 7f9d6dec7740 1 --2- 172.21.15.16:0/3739441579 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c2d13e75d0 0x55c2d13e7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:17.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.134+0000 7f9d6dec7740 1 --2- 172.21.15.16:0/3739441579 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c2d13e8b40 0x55c2d13e7090 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:17.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.134+0000 7f9d6dec7740 1 -- 172.21.15.16:0/3739441579 >> 172.21.15.16:0/3739441579 conn(0x55c2d13b0310 msgr2=0x55c2d13e3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:10:17.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.134+0000 7f9d6dec7740 1 -- 172.21.15.16:0/3739441579 shutdown_connections 2024-09-12T02:10:17.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.134+0000 7f9d6dec7740 1 -- 172.21.15.16:0/3739441579 wait complete. 2024-09-12T02:10:17.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.134+0000 7f9d6dec7740 1 -- start start 2024-09-12T02:10:17.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.134+0000 7f9d6dec7740 1 Processor -- start 2024-09-12T02:10:17.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.134+0000 7f9d6dec7740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c2d13e75d0 0x55c2d13ef100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:17.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.134+0000 7f9d6dec7740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c2d13e8b40 0x55c2d13ef640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:17.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.135+0000 7f9d6dec7740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c2d13f2370 0x55c2d13efb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:17.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.135+0000 7f9d6dec7740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55c2d13f93c0 con 0x55c2d13f2370 2024-09-12T02:10:17.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.135+0000 7f9d6dec7740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55c2d1308060 con 0x55c2d13e8b40 2024-09-12T02:10:17.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.135+0000 7f9d6dec7740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55c2d13f9540 con 0x55c2d13e75d0 2024-09-12T02:10:17.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.135+0000 7f9d6dec7740 10 client.0 _ll_register_callbacks cb 0x55c2d105b8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:10:17.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.135+0000 7f9d6a08b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c2d13f2370 0x55c2d13efb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:17.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.135+0000 7f9d6a08b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c2d13f2370 0x55c2d13efb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45958/0 (socket says 172.21.15.16:45958) 2024-09-12T02:10:17.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.135+0000 7f9d69089640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c2d13e8b40 0x55c2d13ef640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:17.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.135+0000 7f9d6a08b640 1 -- 172.21.15.16:0/134292754 learned_addr learned my addr 172.21.15.16:0/134292754 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:10:17.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.135+0000 7f9d6988a640 1 --2- 172.21.15.16:0/134292754 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c2d13e75d0 0x55c2d13ef100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:17.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.135+0000 7f9d6a08b640 1 -- 172.21.15.16:0/134292754 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c2d13e75d0 msgr2=0x55c2d13ef100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:17.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.135+0000 7f9d6a08b640 1 --2- 172.21.15.16:0/134292754 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c2d13e75d0 0x55c2d13ef100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:17.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.135+0000 7f9d6a08b640 1 -- 172.21.15.16:0/134292754 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c2d13e8b40 msgr2=0x55c2d13ef640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:17.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.135+0000 7f9d6a08b640 1 --2- 172.21.15.16:0/134292754 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c2d13e8b40 0x55c2d13ef640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:17.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.135+0000 7f9d6a08b640 1 -- 172.21.15.16:0/134292754 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2d10be3c0 con 0x55c2d13f2370 2024-09-12T02:10:17.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.135+0000 7f9d6988a640 1 --2- 172.21.15.16:0/134292754 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c2d13e75d0 0x55c2d13ef100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:10:17.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.135+0000 7f9d69089640 1 --2- 172.21.15.16:0/134292754 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c2d13e8b40 0x55c2d13ef640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:10:17.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.135+0000 7f9d6a08b640 1 --2- 172.21.15.16:0/134292754 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c2d13f2370 0x55c2d13efb80 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f9d5c02ae70 tx=0x7f9d5c003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d43e97d561d6882 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:10:17.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.135+0000 7f9d4d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:10:17.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.136+0000 7f9d4d7fa640 1 -- 172.21.15.16:0/134292754 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d5c048030 con 0x55c2d13f2370 2024-09-12T02:10:17.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.136+0000 7f9d6dec7740 1 -- 172.21.15.16:0/134292754 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d5c02f930 con 0x55c2d13f2370 2024-09-12T02:10:17.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.136+0000 7f9d4d7fa640 1 -- 172.21.15.16:0/134292754 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f9d5c040050 con 0x55c2d13f2370 2024-09-12T02:10:17.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.136+0000 7f9d4d7fa640 1 -- 172.21.15.16:0/134292754 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9d5c034040 con 0x55c2d13f2370 2024-09-12T02:10:17.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.136+0000 7f9d6dec7740 10 client.18027 fetch_fsmap learned FSMap version 25 2024-09-12T02:10:17.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.136+0000 7f9d6dec7740 1 -- 172.21.15.16:0/134292754 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c2d1404ed0 con 0x55c2d13f2370 2024-09-12T02:10:17.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.136+0000 7f9d4d7fa640 1 -- 172.21.15.16:0/134292754 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d5c02f930 con 0x55c2d13f2370 2024-09-12T02:10:17.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.136+0000 7f9d4d7fa640 1 -- 172.21.15.16:0/134292754 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f9d5c03a430 con 0x55c2d13f2370 2024-09-12T02:10:17.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.136+0000 7f9d6dec7740 10 client.18027 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:10:17.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.136+0000 7f9d6dec7740 10 client.18027 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T02:10:17.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.136+0000 7f9d6dec7740 20 client.18027 populate_metadata read hostname 'smithi016' 2024-09-12T02:10:17.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.136+0000 7f9d6dec7740 1 --2- 172.21.15.16:0/134292754 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x55c2d1406220 0x55c2d1408610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:17.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.136+0000 7f9d6dec7740 4 client.18027 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:10:17.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.136+0000 7f9d6dec7740 1 -- 172.21.15.16:0/134292754 --> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c2d135a860 con 0x55c2d1406220 2024-09-12T02:10:17.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.137+0000 7f9d6988a640 1 --2- 172.21.15.16:0/134292754 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x55c2d1406220 0x55c2d1408610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:17.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.137+0000 7f9d4d7fa640 10 client.18027 ms_handle_connect on v2:172.21.15.184:6838/4203347941 2024-09-12T02:10:17.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.137+0000 7f9d6988a640 1 --2- 172.21.15.16:0/134292754 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x55c2d1406220 0x55c2d1408610 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a24381e872fff8b1 server_cookie=bb8d6514919b82ff in_seq=0 out_seq=0 2024-09-12T02:10:17.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.138+0000 7f9d4d7fa640 1 -- 172.21.15.16:0/134292754 <== mds.1 v2:172.21.15.184:6838/4203347941 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c2d135a860 con 0x55c2d1406220 2024-09-12T02:10:17.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.138+0000 7f9d4d7fa640 10 client.18027 handle_command_reply: tid=0 2024-09-12T02:10:17.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.138+0000 7f9d6dec7740 1 client.18027 shutdown 2024-09-12T02:10:17.148 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.138+0000 7f9f41ffb640 1 -- 172.21.15.16:0/86655248 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562f3dbbfd70 con 0x562f3dcd54d0 2024-09-12T02:10:17.148 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.138+0000 7f9f41ffb640 10 client.18015 handle_command_reply: tid=0 2024-09-12T02:10:17.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.138+0000 7f9d6dec7740 1 -- 172.21.15.16:0/134292754 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c2d13f2370 msgr2=0x55c2d13efb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:17.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.138+0000 7f9d6dec7740 1 --2- 172.21.15.16:0/134292754 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c2d13f2370 0x55c2d13efb80 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f9d5c02ae70 tx=0x7f9d5c003970 comp rx=0 tx=0).stop 2024-09-12T02:10:17.149 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.138+0000 7f9f636d7740 1 client.18015 shutdown 2024-09-12T02:10:17.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.138+0000 7f9d6dec7740 1 -- 172.21.15.16:0/134292754 shutdown_connections 2024-09-12T02:10:17.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.138+0000 7f9d6dec7740 1 --2- 172.21.15.16:0/134292754 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x55c2d1406220 0x55c2d1408610 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:17.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.138+0000 7f9d6dec7740 1 --2- 172.21.15.16:0/134292754 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c2d13f2370 0x55c2d13efb80 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:17.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.138+0000 7f9d6dec7740 1 --2- 172.21.15.16:0/134292754 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c2d13e8b40 0x55c2d13ef640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:17.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.138+0000 7f9d4d7fa640 0 client.18027 ms_handle_reset on v2:172.21.15.184:6838/4203347941 2024-09-12T02:10:17.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.138+0000 7f9d6dec7740 1 --2- 172.21.15.16:0/134292754 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c2d13e75d0 0x55c2d13ef100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:17.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.138+0000 7f9d6dec7740 1 -- 172.21.15.16:0/134292754 >> 172.21.15.16:0/134292754 conn(0x55c2d13b0310 msgr2=0x55c2d13e9960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:10:17.150 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.138+0000 7f9f636d7740 1 -- 172.21.15.16:0/86655248 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f3dc552e0 msgr2=0x562f3dc637d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:17.150 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.138+0000 7f9f636d7740 1 --2- 172.21.15.16:0/86655248 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f3dc552e0 0x562f3dc637d0 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f9f5000be60 tx=0x7f9f5009abc0 comp rx=0 tx=0).stop 2024-09-12T02:10:17.151 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.138+0000 7f9f636d7740 1 -- 172.21.15.16:0/86655248 shutdown_connections 2024-09-12T02:10:17.151 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.138+0000 7f9f636d7740 1 --2- 172.21.15.16:0/86655248 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x562f3dcd54d0 0x562f3dcd78f0 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:17.151 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.138+0000 7f9f636d7740 1 --2- 172.21.15.16:0/86655248 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562f3dc5a370 0x562f3dc64790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:17.151 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.138+0000 7f9f636d7740 1 --2- 172.21.15.16:0/86655248 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f3dc552e0 0x562f3dc637d0 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:17.151 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.138+0000 7f9f41ffb640 0 client.18015 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:10:17.151 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.138+0000 7f9f636d7740 1 --2- 172.21.15.16:0/86655248 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562f3dc549a0 0x562f3dc63290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:17.151 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.138+0000 7f9f636d7740 1 -- 172.21.15.16:0/86655248 >> 172.21.15.16:0/86655248 conn(0x562f3dc50010 msgr2=0x562f3dc4e9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:10:17.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.138+0000 7f9d6dec7740 1 -- 172.21.15.16:0/134292754 shutdown_connections 2024-09-12T02:10:17.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.138+0000 7f9d6dec7740 1 -- 172.21.15.16:0/134292754 wait complete. 2024-09-12T02:10:17.152 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.138+0000 7f9f636d7740 1 -- 172.21.15.16:0/86655248 shutdown_connections 2024-09-12T02:10:17.152 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.138+0000 7f9f636d7740 1 -- 172.21.15.16:0/86655248 wait complete. 2024-09-12T02:10:17.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.138+0000 7f9d6dec7740 20 client.18027 trim_cache size 0 max 16384 2024-09-12T02:10:17.153 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.139+0000 7f9f636d7740 20 client.18015 trim_cache size 0 max 16384 2024-09-12T02:10:17.153 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-09-12T02:10:17.153 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-09-12T02:10:17.153 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-09-12T02:10:17.153 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-09-12T02:10:17.154 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.139+0000 7f9f6093b640 1 -- 172.21.15.16:0/158459585 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f9f440a2c30 msgr2=0x7f9f440a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:17.154 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.139+0000 7f9f6093b640 1 --2- 172.21.15.16:0/158459585 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f9f440a2c30 0x7f9f440a50f0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f9f54031ae0 tx=0x7f9f5403e040 comp rx=0 tx=0).stop 2024-09-12T02:10:17.154 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.139+0000 7f9f6093b640 1 -- 172.21.15.16:0/158459585 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f5c101c30 msgr2=0x7f9f5c19cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:17.154 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.139+0000 7f9f6093b640 1 --2- 172.21.15.16:0/158459585 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f5c101c30 0x7f9f5c19cb30 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f9f5000b4a0 tx=0x7f9f5000b960 comp rx=0 tx=0).stop 2024-09-12T02:10:17.154 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.139+0000 7f9f6093b640 1 -- 172.21.15.16:0/158459585 shutdown_connections 2024-09-12T02:10:17.154 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.139+0000 7f9f6093b640 1 --2- 172.21.15.16:0/158459585 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f9f440a2c30 0x7f9f440a50f0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:17.155 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.139+0000 7f9f6093b640 1 --2- 172.21.15.16:0/158459585 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9f5c1025f0 0x7f9f5c19f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:17.155 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.139+0000 7f9f6093b640 1 --2- 172.21.15.16:0/158459585 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f5c101c30 0x7f9f5c19cb30 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:17.155 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.139+0000 7f9f6093b640 1 --2- 172.21.15.16:0/158459585 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f5c100a40 0x7f9f5c06a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:17.155 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.139+0000 7f9f6093b640 1 -- 172.21.15.16:0/158459585 >> 172.21.15.16:0/158459585 conn(0x7f9f5c0fc1e0 msgr2=0x7f9f5c104fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:10:17.155 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.139+0000 7f9f6093b640 1 -- 172.21.15.16:0/158459585 shutdown_connections 2024-09-12T02:10:17.155 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:17.139+0000 7f9f6093b640 1 -- 172.21.15.16:0/158459585 wait complete. 2024-09-12T02:10:17.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.151+0000 7f9d6dec7740 1 Processor -- start 2024-09-12T02:10:17.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.151+0000 7f9d6dec7740 1 -- start start 2024-09-12T02:10:17.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.152+0000 7f9d6dec7740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c2d13e75d0 0x55c2d13ef9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:17.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.152+0000 7f9d6dec7740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c2d13e8b40 0x55c2d13ec360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:17.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.152+0000 7f9d6dec7740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c2d13f2370 0x55c2d13ec8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:17.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.152+0000 7f9d6dec7740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9d641ad950 con 0x55c2d13e8b40 2024-09-12T02:10:17.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.152+0000 7f9d6dec7740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9d641ab220 con 0x55c2d13f2370 2024-09-12T02:10:17.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.152+0000 7f9d6dec7740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f9d641a90b0 con 0x55c2d13e75d0 2024-09-12T02:10:17.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.152+0000 7f9d69089640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c2d13e8b40 0x55c2d13ec360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:17.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.152+0000 7f9d69089640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c2d13e8b40 0x55c2d13ec360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45972/0 (socket says 172.21.15.16:45972) 2024-09-12T02:10:17.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.152+0000 7f9d69089640 1 -- 172.21.15.16:0/3933672820 learned_addr learned my addr 172.21.15.16:0/3933672820 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:10:17.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.152+0000 7f9d6988a640 1 --2- 172.21.15.16:0/3933672820 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c2d13e75d0 0x55c2d13ef9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:17.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.152+0000 7f9d6a08b640 1 --2- 172.21.15.16:0/3933672820 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c2d13f2370 0x55c2d13ec8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:17.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.152+0000 7f9d69089640 1 -- 172.21.15.16:0/3933672820 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c2d13e75d0 msgr2=0x55c2d13ef9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:17.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.152+0000 7f9d69089640 1 --2- 172.21.15.16:0/3933672820 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c2d13e75d0 0x55c2d13ef9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:17.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.152+0000 7f9d69089640 1 -- 172.21.15.16:0/3933672820 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c2d13f2370 msgr2=0x55c2d13ec8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:17.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.152+0000 7f9d69089640 1 --2- 172.21.15.16:0/3933672820 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c2d13f2370 0x55c2d13ec8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:17.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.152+0000 7f9d69089640 1 -- 172.21.15.16:0/3933672820 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2d13ea120 con 0x55c2d13e8b40 2024-09-12T02:10:17.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.152+0000 7f9d6a08b640 1 --2- 172.21.15.16:0/3933672820 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c2d13f2370 0x55c2d13ec8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:10:17.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.153+0000 7f9d69089640 1 --2- 172.21.15.16:0/3933672820 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c2d13e8b40 0x55c2d13ec360 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f9d540d2d10 tx=0x7f9d540095b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89006c7bd1a4e645 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:10:17.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.153+0000 7f9d4f7fe640 1 -- 172.21.15.16:0/3933672820 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d5409c450 con 0x55c2d13e8b40 2024-09-12T02:10:17.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.153+0000 7f9d4f7fe640 1 -- 172.21.15.16:0/3933672820 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d5409bd50 con 0x55c2d13e8b40 2024-09-12T02:10:17.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.153+0000 7f9d4f7fe640 1 -- 172.21.15.16:0/3933672820 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d540cf0f0 con 0x55c2d13e8b40 2024-09-12T02:10:17.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.154+0000 7f9d6dec7740 1 -- 172.21.15.16:0/3933672820 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c2d13e8b40 msgr2=0x55c2d13ec360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:17.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.154+0000 7f9d6dec7740 1 --2- 172.21.15.16:0/3933672820 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c2d13e8b40 0x55c2d13ec360 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f9d540d2d10 tx=0x7f9d540095b0 comp rx=0 tx=0).stop 2024-09-12T02:10:17.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.154+0000 7f9d6dec7740 1 -- 172.21.15.16:0/3933672820 shutdown_connections 2024-09-12T02:10:17.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.154+0000 7f9d6dec7740 1 --2- 172.21.15.16:0/3933672820 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c2d13f2370 0x55c2d13ec8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:17.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.154+0000 7f9d6dec7740 1 --2- 172.21.15.16:0/3933672820 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c2d13e8b40 0x55c2d13ec360 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:17.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.154+0000 7f9d6dec7740 1 --2- 172.21.15.16:0/3933672820 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c2d13e75d0 0x55c2d13ef9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:17.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.154+0000 7f9d6dec7740 1 -- 172.21.15.16:0/3933672820 >> 172.21.15.16:0/3933672820 conn(0x55c2d13b0310 msgr2=0x55c2d1401fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:10:17.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.154+0000 7f9d6dec7740 1 -- 172.21.15.16:0/3933672820 shutdown_connections 2024-09-12T02:10:17.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.154+0000 7f9d6dec7740 1 -- 172.21.15.16:0/3933672820 wait complete. 2024-09-12T02:10:17.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.154+0000 7f9d6dec7740 1 -- start start 2024-09-12T02:10:17.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.154+0000 7f9d6dec7740 1 Processor -- start 2024-09-12T02:10:17.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.155+0000 7f9d6dec7740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c2d13e75d0 0x55c2d13f7240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:17.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.155+0000 7f9d6dec7740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c2d13e8b40 0x55c2d13f7780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:17.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.155+0000 7f9d6dec7740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c2d13f2370 0x55c2d13f7cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:17.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.155+0000 7f9d6dec7740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9d641a9700 con 0x55c2d13e75d0 2024-09-12T02:10:17.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.155+0000 7f9d6dec7740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9d641aab50 con 0x55c2d13f2370 2024-09-12T02:10:17.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.155+0000 7f9d6dec7740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f9d641a9db0 con 0x55c2d13e8b40 2024-09-12T02:10:17.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.155+0000 7f9d6dec7740 10 client.0 _ll_register_callbacks cb 0x7f9d5c046070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:10:17.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.155+0000 7f9d6988a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c2d13e75d0 0x55c2d13f7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:17.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.155+0000 7f9d6988a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c2d13e75d0 0x55c2d13f7240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45974/0 (socket says 172.21.15.16:45974) 2024-09-12T02:10:17.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.155+0000 7f9d6988a640 1 -- 172.21.15.16:0/2789858250 learned_addr learned my addr 172.21.15.16:0/2789858250 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:10:17.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.155+0000 7f9d69089640 1 --2- 172.21.15.16:0/2789858250 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c2d13e8b40 0x55c2d13f7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:17.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.155+0000 7f9d6a08b640 1 --2- 172.21.15.16:0/2789858250 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c2d13f2370 0x55c2d13f7cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:17.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.155+0000 7f9d6988a640 1 -- 172.21.15.16:0/2789858250 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c2d13e8b40 msgr2=0x55c2d13f7780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:17.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.155+0000 7f9d6988a640 1 --2- 172.21.15.16:0/2789858250 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c2d13e8b40 0x55c2d13f7780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:17.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.155+0000 7f9d6988a640 1 -- 172.21.15.16:0/2789858250 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c2d13f2370 msgr2=0x55c2d13f7cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:17.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.155+0000 7f9d6988a640 1 --2- 172.21.15.16:0/2789858250 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c2d13f2370 0x55c2d13f7cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:17.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.155+0000 7f9d6988a640 1 -- 172.21.15.16:0/2789858250 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2d140a7f0 con 0x55c2d13e75d0 2024-09-12T02:10:17.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.155+0000 7f9d6a08b640 1 --2- 172.21.15.16:0/2789858250 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c2d13f2370 0x55c2d13f7cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:10:17.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.155+0000 7f9d6988a640 1 --2- 172.21.15.16:0/2789858250 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c2d13e75d0 0x55c2d13f7240 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f9d60009f80 tx=0x7f9d600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d9f893453553f5e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:10:17.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.155+0000 7f9d4d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:10:17.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.155+0000 7f9d4d7fa640 1 -- 172.21.15.16:0/2789858250 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d60020620 con 0x55c2d13e75d0 2024-09-12T02:10:17.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.156+0000 7f9d4d7fa640 1 -- 172.21.15.16:0/2789858250 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f9d600207c0 con 0x55c2d13e75d0 2024-09-12T02:10:17.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.156+0000 7f9d6dec7740 1 -- 172.21.15.16:0/2789858250 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d5409bd50 con 0x55c2d13e75d0 2024-09-12T02:10:17.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.156+0000 7f9d4d7fa640 1 -- 172.21.15.16:0/2789858250 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9d60003e40 con 0x55c2d13e75d0 2024-09-12T02:10:17.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.156+0000 7f9d6dec7740 10 client.18033 fetch_fsmap learned FSMap version 25 2024-09-12T02:10:17.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.156+0000 7f9d6dec7740 1 -- 172.21.15.16:0/2789858250 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c2d140ba80 con 0x55c2d13e75d0 2024-09-12T02:10:17.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.156+0000 7f9d4d7fa640 1 -- 172.21.15.16:0/2789858250 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d5409bd50 con 0x55c2d13e75d0 2024-09-12T02:10:17.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.156+0000 7f9d4d7fa640 1 -- 172.21.15.16:0/2789858250 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f9d6000f040 con 0x55c2d13e75d0 2024-09-12T02:10:17.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.156+0000 7f9d6dec7740 10 client.18033 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:10:17.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.156+0000 7f9d6dec7740 10 client.18033 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T02:10:17.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.156+0000 7f9d6dec7740 20 client.18033 populate_metadata read hostname 'smithi016' 2024-09-12T02:10:17.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.156+0000 7f9d6dec7740 1 --2- 172.21.15.16:0/2789858250 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x55c2d1406220 0x55c2d1477350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:17.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.156+0000 7f9d6dec7740 4 client.18033 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T02:10:17.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.156+0000 7f9d6dec7740 1 -- 172.21.15.16:0/2789858250 --> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c2d1354d70 con 0x55c2d1406220 2024-09-12T02:10:17.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.157+0000 7f9d69089640 1 --2- 172.21.15.16:0/2789858250 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x55c2d1406220 0x55c2d1477350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:17.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.157+0000 7f9d4d7fa640 10 client.18033 ms_handle_connect on v2:172.21.15.184:6838/4203347941 2024-09-12T02:10:17.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.157+0000 7f9d69089640 1 --2- 172.21.15.16:0/2789858250 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x55c2d1406220 0x55c2d1477350 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f1b2571ca0248dcb server_cookie=10ae50a985aa6042 in_seq=0 out_seq=0 2024-09-12T02:10:17.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.157+0000 7f9d4d7fa640 1 -- 172.21.15.16:0/2789858250 <== mds.1 v2:172.21.15.184:6838/4203347941 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c2d1354d70 con 0x55c2d1406220 2024-09-12T02:10:17.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.157+0000 7f9d4d7fa640 10 client.18033 handle_command_reply: tid=0 2024-09-12T02:10:17.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.157+0000 7f9d6dec7740 1 client.18033 shutdown 2024-09-12T02:10:17.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.157+0000 7f9d6dec7740 1 -- 172.21.15.16:0/2789858250 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c2d13e75d0 msgr2=0x55c2d13f7240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:17.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.157+0000 7f9d6dec7740 1 --2- 172.21.15.16:0/2789858250 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c2d13e75d0 0x55c2d13f7240 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f9d60009f80 tx=0x7f9d600046c0 comp rx=0 tx=0).stop 2024-09-12T02:10:17.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.157+0000 7f9d6dec7740 1 -- 172.21.15.16:0/2789858250 shutdown_connections 2024-09-12T02:10:17.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.157+0000 7f9d6dec7740 1 --2- 172.21.15.16:0/2789858250 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x55c2d1406220 0x55c2d1477350 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:17.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.158+0000 7f9d6dec7740 1 --2- 172.21.15.16:0/2789858250 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c2d13f2370 0x55c2d13f7cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:17.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.158+0000 7f9d6dec7740 1 --2- 172.21.15.16:0/2789858250 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c2d13e8b40 0x55c2d13f7780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:17.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.158+0000 7f9d6dec7740 1 --2- 172.21.15.16:0/2789858250 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c2d13e75d0 0x55c2d13f7240 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:17.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.158+0000 7f9d4d7fa640 0 client.18033 ms_handle_reset on v2:172.21.15.184:6838/4203347941 2024-09-12T02:10:17.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.158+0000 7f9d6dec7740 1 -- 172.21.15.16:0/2789858250 >> 172.21.15.16:0/2789858250 conn(0x55c2d13b0310 msgr2=0x55c2d13f0960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:10:17.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.158+0000 7f9d6dec7740 1 -- 172.21.15.16:0/2789858250 shutdown_connections 2024-09-12T02:10:17.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.158+0000 7f9d6dec7740 1 -- 172.21.15.16:0/2789858250 wait complete. 2024-09-12T02:10:17.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.158+0000 7f9d6dec7740 20 client.18033 trim_cache size 0 max 16384 2024-09-12T02:10:17.168 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T02:10:17.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.158+0000 7f9d6baee640 1 -- 172.21.15.16:0/1910217680 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f9d500a2c00 msgr2=0x7f9d500a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:17.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.158+0000 7f9d6baee640 1 --2- 172.21.15.16:0/1910217680 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f9d500a2c00 0x7f9d500a50c0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x55c2d13e4ac0 tx=0x7f9d60005f90 comp rx=0 tx=0).stop 2024-09-12T02:10:17.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.158+0000 7f9d6baee640 1 -- 172.21.15.16:0/1910217680 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d64102570 msgr2=0x7f9d641989b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:17.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.158+0000 7f9d6baee640 1 --2- 172.21.15.16:0/1910217680 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d64102570 0x7f9d641989b0 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f9d5400e9f0 tx=0x7f9d5400eeb0 comp rx=0 tx=0).stop 2024-09-12T02:10:17.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.158+0000 7f9d6baee640 1 -- 172.21.15.16:0/1910217680 shutdown_connections 2024-09-12T02:10:17.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.158+0000 7f9d6baee640 1 --2- 172.21.15.16:0/1910217680 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f9d500a2c00 0x7f9d500a50c0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:17.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.158+0000 7f9d6baee640 1 --2- 172.21.15.16:0/1910217680 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9d64102ef0 0x7f9d6419cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:17.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.158+0000 7f9d6baee640 1 --2- 172.21.15.16:0/1910217680 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d64102570 0x7f9d641989b0 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:17.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.158+0000 7f9d6baee640 1 --2- 172.21.15.16:0/1910217680 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9d64100e20 0x7f9d64198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:17.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.158+0000 7f9d6baee640 1 -- 172.21.15.16:0/1910217680 >> 172.21.15.16:0/1910217680 conn(0x7f9d640fc7d0 msgr2=0x7f9d6406a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:10:17.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.158+0000 7f9d6baee640 1 -- 172.21.15.16:0/1910217680 shutdown_connections 2024-09-12T02:10:17.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:17.159+0000 7f9d6baee640 1 -- 172.21.15.16:0/1910217680 wait complete. 2024-09-12T02:10:17.933 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:10:17 smithi016 ceph-mon[30947]: pgmap v586: 97 pgs: 97 active+clean; 4.2 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 7.2 KiB/s wr, 10 op/s 2024-09-12T02:10:17.975 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-12T02:10:18.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:10:17 smithi184 ceph-mon[35593]: pgmap v586: 97 pgs: 97 active+clean; 4.2 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 7.2 KiB/s wr, 10 op/s 2024-09-12T02:10:18.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:10:17 smithi033 ceph-mon[35332]: pgmap v586: 97 pgs: 97 active+clean; 4.2 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 7.2 KiB/s wr, 10 op/s 2024-09-12T02:10:18.276 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:10:18.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.757+0000 7fbb26584640 1 -- 172.21.15.16:0/1825549405 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb20103660 msgr2=0x7fbb2010bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:18.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.757+0000 7fbb26584640 1 --2- 172.21.15.16:0/1825549405 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb20103660 0x7fbb2010bb50 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7fbb14009900 tx=0x7fbb1402a480 comp rx=0 tx=0).stop 2024-09-12T02:10:18.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.757+0000 7fbb26584640 1 -- 172.21.15.16:0/1825549405 shutdown_connections 2024-09-12T02:10:18.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.757+0000 7fbb26584640 1 --2- 172.21.15.16:0/1825549405 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbb2010c170 0x7fbb2010e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:18.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.757+0000 7fbb26584640 1 --2- 172.21.15.16:0/1825549405 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb20103660 0x7fbb2010bb50 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:18.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.757+0000 7fbb26584640 1 --2- 172.21.15.16:0/1825549405 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbb20102d40 0x7fbb20103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:18.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.757+0000 7fbb26584640 1 -- 172.21.15.16:0/1825549405 >> 172.21.15.16:0/1825549405 conn(0x7fbb200fc7a0 msgr2=0x7fbb200febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:10:18.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.757+0000 7fbb26584640 1 -- 172.21.15.16:0/1825549405 shutdown_connections 2024-09-12T02:10:18.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.757+0000 7fbb26584640 1 -- 172.21.15.16:0/1825549405 wait complete. 2024-09-12T02:10:18.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.758+0000 7fbb26584640 1 Processor -- start 2024-09-12T02:10:18.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.758+0000 7fbb26584640 1 -- start start 2024-09-12T02:10:18.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.758+0000 7fbb26584640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbb20102d40 0x7fbb201984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:18.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.758+0000 7fbb26584640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbb20103660 0x7fbb201989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:18.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.758+0000 7fbb26584640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb2010c170 0x7fbb2019cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:18.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.758+0000 7fbb26584640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbb20111fb0 con 0x7fbb2010c170 2024-09-12T02:10:18.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.758+0000 7fbb26584640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbb20111e30 con 0x7fbb20103660 2024-09-12T02:10:18.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.758+0000 7fbb26584640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fbb20112130 con 0x7fbb20102d40 2024-09-12T02:10:18.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.758+0000 7fbb24b21640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb2010c170 0x7fbb2019cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:18.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.759+0000 7fbb24b21640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb2010c170 0x7fbb2019cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45990/0 (socket says 172.21.15.16:45990) 2024-09-12T02:10:18.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.759+0000 7fbb24b21640 1 -- 172.21.15.16:0/2177325883 learned_addr learned my addr 172.21.15.16:0/2177325883 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:10:18.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.759+0000 7fbb1f7fe640 1 --2- 172.21.15.16:0/2177325883 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbb20103660 0x7fbb201989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:18.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.759+0000 7fbb1ffff640 1 --2- 172.21.15.16:0/2177325883 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbb20102d40 0x7fbb201984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:18.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.759+0000 7fbb24b21640 1 -- 172.21.15.16:0/2177325883 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbb20102d40 msgr2=0x7fbb201984a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:18.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.759+0000 7fbb24b21640 1 --2- 172.21.15.16:0/2177325883 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbb20102d40 0x7fbb201984a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:18.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.759+0000 7fbb24b21640 1 -- 172.21.15.16:0/2177325883 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbb20103660 msgr2=0x7fbb201989e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:18.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.759+0000 7fbb24b21640 1 --2- 172.21.15.16:0/2177325883 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbb20103660 0x7fbb201989e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:18.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.759+0000 7fbb24b21640 1 -- 172.21.15.16:0/2177325883 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb2019d450 con 0x7fbb2010c170 2024-09-12T02:10:18.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.759+0000 7fbb1ffff640 1 --2- 172.21.15.16:0/2177325883 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbb20102d40 0x7fbb201984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:10:18.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.759+0000 7fbb24b21640 1 --2- 172.21.15.16:0/2177325883 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb2010c170 0x7fbb2019cd70 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7fbb1800b970 tx=0x7fbb1800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:10:18.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.759+0000 7fbb1d7fa640 1 -- 172.21.15.16:0/2177325883 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb18019070 con 0x7fbb2010c170 2024-09-12T02:10:18.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.759+0000 7fbb26584640 1 -- 172.21.15.16:0/2177325883 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb2019d740 con 0x7fbb2010c170 2024-09-12T02:10:18.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.759+0000 7fbb1d7fa640 1 -- 172.21.15.16:0/2177325883 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb1800cb40 con 0x7fbb2010c170 2024-09-12T02:10:18.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.759+0000 7fbb1d7fa640 1 -- 172.21.15.16:0/2177325883 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb18007650 con 0x7fbb2010c170 2024-09-12T02:10:18.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.759+0000 7fbb26584640 1 -- 172.21.15.16:0/2177325883 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb201a5020 con 0x7fbb2010c170 2024-09-12T02:10:18.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.762+0000 7fbb2895d740 1 Processor -- start 2024-09-12T02:10:18.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.762+0000 7fbb1d7fa640 1 -- 172.21.15.16:0/2177325883 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fbb1801c070 con 0x7fbb2010c170 2024-09-12T02:10:18.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.762+0000 7fbb2895d740 1 -- start start 2024-09-12T02:10:18.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.762+0000 7fbb2895d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5631f32edb40 0x5631f32ec090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:18.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.762+0000 7fbb2895d740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5631f32ec5d0 0x5631f32eca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:18.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.762+0000 7fbb2895d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5631f32fb780 0x5631f32fdb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:18.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.762+0000 7fbb2895d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5631f32522f0 con 0x5631f32fb780 2024-09-12T02:10:18.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.762+0000 7fbb2895d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5631f31f9cf0 con 0x5631f32edb40 2024-09-12T02:10:18.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.762+0000 7fbb2895d740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5631f324e2b0 con 0x5631f32ec5d0 2024-09-12T02:10:18.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.762+0000 7fbb1f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5631f32fb780 0x5631f32fdb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:18.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.762+0000 7fbb1f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5631f32fb780 0x5631f32fdb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46006/0 (socket says 172.21.15.16:46006) 2024-09-12T02:10:18.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.762+0000 7fbb1f7fe640 1 -- 172.21.15.16:0/3584634607 learned_addr learned my addr 172.21.15.16:0/3584634607 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:10:18.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.762+0000 7fbb1ffff640 1 --2- 172.21.15.16:0/3584634607 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5631f32ec5d0 0x5631f32eca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:18.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.762+0000 7fbb1f7fe640 1 --2- 172.21.15.16:0/3584634607 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5631f32edb40 0x5631f32ec090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:18.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.762+0000 7fbb1d7fa640 1 --2- 172.21.15.16:0/2177325883 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fbb080a2b80 0x7fbb080a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:18.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.763+0000 7fbb24b21640 1 --2- 172.21.15.16:0/2177325883 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fbb080a2b80 0x7fbb080a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:18.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.763+0000 7fbb1d7fa640 1 -- 172.21.15.16:0/2177325883 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fbb180ce570 con 0x7fbb2010c170 2024-09-12T02:10:18.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.763+0000 7fbb1ffff640 1 -- 172.21.15.16:0/3584634607 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5631f32edb40 msgr2=0x5631f32ec090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:18.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.763+0000 7fbb1ffff640 1 --2- 172.21.15.16:0/3584634607 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5631f32edb40 0x5631f32ec090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:18.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.763+0000 7fbb1ffff640 1 -- 172.21.15.16:0/3584634607 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5631f32fb780 msgr2=0x5631f32fdb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:18.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.763+0000 7fbb1ffff640 1 --2- 172.21.15.16:0/3584634607 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5631f32fb780 0x5631f32fdb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:18.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.763+0000 7fbb1ffff640 1 -- 172.21.15.16:0/3584634607 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5631f31de3a0 con 0x5631f32ec5d0 2024-09-12T02:10:18.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.763+0000 7fbb1f7fe640 1 --2- 172.21.15.16:0/3584634607 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5631f32fb780 0x5631f32fdb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:10:18.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.763+0000 7fbb1ffff640 1 --2- 172.21.15.16:0/3584634607 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5631f32ec5d0 0x5631f32eca10 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fbb1000bab0 tx=0x7fbb1000bf70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aa57e3c136f654b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:10:18.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.763+0000 7fbb24b21640 1 --2- 172.21.15.16:0/2177325883 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fbb080a2b80 0x7fbb080a5040 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fbb1800bf60 tx=0x7fbb18065ba0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:10:18.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.763+0000 7fbb067fc640 1 -- 172.21.15.16:0/3584634607 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb10017420 con 0x5631f32ec5d0 2024-09-12T02:10:18.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.763+0000 7fbb067fc640 1 -- 172.21.15.16:0/3584634607 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb10017aa0 con 0x5631f32ec5d0 2024-09-12T02:10:18.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.763+0000 7fbb067fc640 1 -- 172.21.15.16:0/3584634607 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb1000fa10 con 0x5631f32ec5d0 2024-09-12T02:10:18.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.764+0000 7fbb2895d740 1 -- 172.21.15.16:0/3584634607 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5631f32ec5d0 msgr2=0x5631f32eca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:18.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.764+0000 7fbb2895d740 1 --2- 172.21.15.16:0/3584634607 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5631f32ec5d0 0x5631f32eca10 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fbb1000bab0 tx=0x7fbb1000bf70 comp rx=0 tx=0).stop 2024-09-12T02:10:18.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.764+0000 7fbb2895d740 1 -- 172.21.15.16:0/3584634607 shutdown_connections 2024-09-12T02:10:18.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.764+0000 7fbb2895d740 1 --2- 172.21.15.16:0/3584634607 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5631f32fb780 0x5631f32fdb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:18.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.764+0000 7fbb2895d740 1 --2- 172.21.15.16:0/3584634607 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5631f32ec5d0 0x5631f32eca10 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:18.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.764+0000 7fbb2895d740 1 --2- 172.21.15.16:0/3584634607 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5631f32edb40 0x5631f32ec090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:18.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.764+0000 7fbb2895d740 1 -- 172.21.15.16:0/3584634607 >> 172.21.15.16:0/3584634607 conn(0x5631f32b5310 msgr2=0x5631f32e8c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:10:18.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.764+0000 7fbb2895d740 1 -- 172.21.15.16:0/3584634607 shutdown_connections 2024-09-12T02:10:18.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.764+0000 7fbb2895d740 1 -- 172.21.15.16:0/3584634607 wait complete. 2024-09-12T02:10:18.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.764+0000 7fbb2895d740 1 -- start start 2024-09-12T02:10:18.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.765+0000 7fbb2895d740 1 Processor -- start 2024-09-12T02:10:18.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.765+0000 7fbb2895d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5631f32ec5d0 0x5631f32fb0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:18.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.765+0000 7fbb2895d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5631f32edb40 0x5631f32f8390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:18.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.765+0000 7fbb2895d740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5631f32fb780 0x5631f32f88f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:18.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.765+0000 7fbb2895d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5631f32fe480 con 0x5631f32edb40 2024-09-12T02:10:18.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.765+0000 7fbb2895d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5631f320d060 con 0x5631f32ec5d0 2024-09-12T02:10:18.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.765+0000 7fbb2895d740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5631f32fe600 con 0x5631f32fb780 2024-09-12T02:10:18.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.765+0000 7fbb2895d740 10 client.0 _ll_register_callbacks cb 0x5631f2f608c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:10:18.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.765+0000 7fbb1ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5631f32edb40 0x5631f32f8390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:18.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.765+0000 7fbb1ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5631f32edb40 0x5631f32f8390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46016/0 (socket says 172.21.15.16:46016) 2024-09-12T02:10:18.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.765+0000 7fbb1ffff640 1 -- 172.21.15.16:0/56195328 learned_addr learned my addr 172.21.15.16:0/56195328 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:10:18.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.765+0000 7fbb1f7fe640 1 --2- 172.21.15.16:0/56195328 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5631f32fb780 0x5631f32f88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:18.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.765+0000 7fbb1f7fe640 1 --2- 172.21.15.16:0/56195328 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5631f32ec5d0 0x5631f32fb0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:18.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.765+0000 7fbb1ffff640 1 -- 172.21.15.16:0/56195328 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5631f32fb780 msgr2=0x5631f32f88f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:18.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.765+0000 7fbb1ffff640 1 --2- 172.21.15.16:0/56195328 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5631f32fb780 0x5631f32f88f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:18.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.765+0000 7fbb1ffff640 1 -- 172.21.15.16:0/56195328 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5631f32ec5d0 msgr2=0x5631f32fb0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:18.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.765+0000 7fbb1ffff640 1 --2- 172.21.15.16:0/56195328 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5631f32ec5d0 0x5631f32fb0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:18.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.765+0000 7fbb1ffff640 1 -- 172.21.15.16:0/56195328 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5631f2fc33c0 con 0x5631f32edb40 2024-09-12T02:10:18.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.765+0000 7fbb1f7fe640 1 --2- 172.21.15.16:0/56195328 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5631f32fb780 0x5631f32f88f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:10:18.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.765+0000 7fbb1ffff640 1 --2- 172.21.15.16:0/56195328 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5631f32edb40 0x5631f32f8390 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7fbb1000f630 tx=0x7fbb10005ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b48a0cfc548fbcde server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:10:18.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.765+0000 7fbaff7fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:10:18.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.766+0000 7fbaff7fe640 1 -- 172.21.15.16:0/56195328 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb10027020 con 0x5631f32edb40 2024-09-12T02:10:18.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.766+0000 7fbaff7fe640 1 -- 172.21.15.16:0/56195328 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fbb10017830 con 0x5631f32edb40 2024-09-12T02:10:18.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.766+0000 7fbb2895d740 1 -- 172.21.15.16:0/56195328 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb10017aa0 con 0x5631f32edb40 2024-09-12T02:10:18.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.766+0000 7fbaff7fe640 1 -- 172.21.15.16:0/56195328 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fbb1000f790 con 0x5631f32edb40 2024-09-12T02:10:18.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.766+0000 7fbb2895d740 10 client.18045 fetch_fsmap learned FSMap version 25 2024-09-12T02:10:18.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.766+0000 7fbb2895d740 1 -- 172.21.15.16:0/56195328 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5631f3309e90 con 0x5631f32edb40 2024-09-12T02:10:18.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.766+0000 7fbaff7fe640 1 -- 172.21.15.16:0/56195328 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb10017aa0 con 0x5631f32edb40 2024-09-12T02:10:18.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.766+0000 7fbaff7fe640 1 -- 172.21.15.16:0/56195328 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fbb10023a70 con 0x5631f32edb40 2024-09-12T02:10:18.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.766+0000 7fbb2895d740 10 client.18045 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:10:18.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.766+0000 7fbb2895d740 10 client.18045 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:10:18.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.766+0000 7fbb2895d740 20 client.18045 populate_metadata read hostname 'smithi016' 2024-09-12T02:10:18.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.766+0000 7fbb2895d740 1 --2- 172.21.15.16:0/56195328 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5631f330b240 0x5631f330d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:18.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.766+0000 7fbb2895d740 4 client.18045 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:10:18.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.766+0000 7fbb2895d740 1 -- 172.21.15.16:0/56195328 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5631f325f860 con 0x5631f330b240 2024-09-12T02:10:18.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.767+0000 7fbb1f7fe640 1 --2- 172.21.15.16:0/56195328 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5631f330b240 0x5631f330d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:18.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.767+0000 7fbaff7fe640 10 client.18045 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:10:18.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.767+0000 7fbb1f7fe640 1 --2- 172.21.15.16:0/56195328 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5631f330b240 0x5631f330d630 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1528d26181750c0a server_cookie=49197400a7b91b39 in_seq=0 out_seq=0 2024-09-12T02:10:18.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.768+0000 7fbaff7fe640 1 -- 172.21.15.16:0/56195328 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5631f325f860 con 0x5631f330b240 2024-09-12T02:10:18.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.768+0000 7fbaff7fe640 10 client.18045 handle_command_reply: tid=0 2024-09-12T02:10:18.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.768+0000 7fbb2895d740 1 client.18045 shutdown 2024-09-12T02:10:18.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.768+0000 7fbb2895d740 1 -- 172.21.15.16:0/56195328 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5631f32edb40 msgr2=0x5631f32f8390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:18.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.768+0000 7fbb2895d740 1 --2- 172.21.15.16:0/56195328 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5631f32edb40 0x5631f32f8390 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7fbb1000f630 tx=0x7fbb10005ed0 comp rx=0 tx=0).stop 2024-09-12T02:10:18.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.768+0000 7fbb2895d740 1 -- 172.21.15.16:0/56195328 shutdown_connections 2024-09-12T02:10:18.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.768+0000 7fbb2895d740 1 --2- 172.21.15.16:0/56195328 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5631f330b240 0x5631f330d630 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:18.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.768+0000 7fbb2895d740 1 --2- 172.21.15.16:0/56195328 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5631f32fb780 0x5631f32f88f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:18.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.768+0000 7fbb2895d740 1 --2- 172.21.15.16:0/56195328 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5631f32edb40 0x5631f32f8390 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:18.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.768+0000 7fbaff7fe640 0 client.18045 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:10:18.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.768+0000 7fbb2895d740 1 --2- 172.21.15.16:0/56195328 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5631f32ec5d0 0x5631f32fb0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:18.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.768+0000 7fbb2895d740 1 -- 172.21.15.16:0/56195328 >> 172.21.15.16:0/56195328 conn(0x5631f32b5310 msgr2=0x5631f32ee960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:10:18.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.768+0000 7fbb2895d740 1 -- 172.21.15.16:0/56195328 shutdown_connections 2024-09-12T02:10:18.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.768+0000 7fbb2895d740 1 -- 172.21.15.16:0/56195328 wait complete. 2024-09-12T02:10:18.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.768+0000 7fbb2895d740 20 client.18045 trim_cache size 0 max 16384 2024-09-12T02:10:18.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.781+0000 7fbb2895d740 1 Processor -- start 2024-09-12T02:10:18.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.781+0000 7fbb2895d740 1 -- start start 2024-09-12T02:10:18.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.781+0000 7fbb2895d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5631f32ec5d0 0x5631f3307720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:18.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.781+0000 7fbb2895d740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5631f32edb40 0x5631f32fa300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:18.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.781+0000 7fbb2895d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5631f32fb780 0x5631f32fcb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:18.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.781+0000 7fbb2895d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbb201adbd0 con 0x5631f32fb780 2024-09-12T02:10:18.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.781+0000 7fbb2895d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbb201ab4a0 con 0x5631f32ec5d0 2024-09-12T02:10:18.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.781+0000 7fbb2895d740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fbb201a92f0 con 0x5631f32edb40 2024-09-12T02:10:18.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.782+0000 7fbb1f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5631f32fb780 0x5631f32fcb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:18.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.782+0000 7fbb1f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5631f32fb780 0x5631f32fcb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46026/0 (socket says 172.21.15.16:46026) 2024-09-12T02:10:18.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.782+0000 7fbb1f7fe640 1 -- 172.21.15.16:0/351522141 learned_addr learned my addr 172.21.15.16:0/351522141 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:10:18.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.782+0000 7fbb1f7fe640 1 --2- 172.21.15.16:0/351522141 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5631f32ec5d0 0x5631f3307720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:18.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.782+0000 7fbb1ffff640 1 --2- 172.21.15.16:0/351522141 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5631f32edb40 0x5631f32fa300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:18.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.782+0000 7fbb1f7fe640 1 -- 172.21.15.16:0/351522141 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5631f32edb40 msgr2=0x5631f32fa300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:18.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.782+0000 7fbb1f7fe640 1 --2- 172.21.15.16:0/351522141 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5631f32edb40 0x5631f32fa300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:18.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.782+0000 7fbb1f7fe640 1 -- 172.21.15.16:0/351522141 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5631f32ec5d0 msgr2=0x5631f3307720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:18.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.782+0000 7fbb1f7fe640 1 --2- 172.21.15.16:0/351522141 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5631f32ec5d0 0x5631f3307720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:18.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.782+0000 7fbb1f7fe640 1 -- 172.21.15.16:0/351522141 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5631f32e9960 con 0x5631f32fb780 2024-09-12T02:10:18.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.782+0000 7fbb1ffff640 1 --2- 172.21.15.16:0/351522141 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5631f32edb40 0x5631f32fa300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:10:18.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.782+0000 7fbb1f7fe640 1 --2- 172.21.15.16:0/351522141 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5631f32fb780 0x5631f32fcb70 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7fbb14006010 tx=0x7fbb14009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5b4ae49fb6d06f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:10:18.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.782+0000 7fbb05ffb640 1 -- 172.21.15.16:0/351522141 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb14035040 con 0x5631f32fb780 2024-09-12T02:10:18.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.782+0000 7fbb05ffb640 1 -- 172.21.15.16:0/351522141 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb1403f0d0 con 0x5631f32fb780 2024-09-12T02:10:18.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.782+0000 7fbb05ffb640 1 -- 172.21.15.16:0/351522141 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb14005020 con 0x5631f32fb780 2024-09-12T02:10:18.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.783+0000 7fbb2895d740 1 -- 172.21.15.16:0/351522141 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5631f32fb780 msgr2=0x5631f32fcb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:18.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.783+0000 7fbb2895d740 1 --2- 172.21.15.16:0/351522141 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5631f32fb780 0x5631f32fcb70 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7fbb14006010 tx=0x7fbb14009260 comp rx=0 tx=0).stop 2024-09-12T02:10:18.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.783+0000 7fbb2895d740 1 -- 172.21.15.16:0/351522141 shutdown_connections 2024-09-12T02:10:18.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.783+0000 7fbb2895d740 1 --2- 172.21.15.16:0/351522141 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5631f32fb780 0x5631f32fcb70 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:18.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.783+0000 7fbb2895d740 1 --2- 172.21.15.16:0/351522141 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5631f32edb40 0x5631f32fa300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:18.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.783+0000 7fbb2895d740 1 --2- 172.21.15.16:0/351522141 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5631f32ec5d0 0x5631f3307720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:18.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.783+0000 7fbb2895d740 1 -- 172.21.15.16:0/351522141 >> 172.21.15.16:0/351522141 conn(0x5631f32b5310 msgr2=0x5631f32f9bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:10:18.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.783+0000 7fbb2895d740 1 -- 172.21.15.16:0/351522141 shutdown_connections 2024-09-12T02:10:18.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.783+0000 7fbb2895d740 1 -- 172.21.15.16:0/351522141 wait complete. 2024-09-12T02:10:18.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.783+0000 7fbb2895d740 1 -- start start 2024-09-12T02:10:18.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.784+0000 7fbb2895d740 1 Processor -- start 2024-09-12T02:10:18.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.784+0000 7fbb2895d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5631f32ec5d0 0x5631f33038a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:18.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.784+0000 7fbb2895d740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5631f32edb40 0x5631f3303de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:18.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.784+0000 7fbb2895d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5631f32fb780 0x5631f32fe850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:18.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.784+0000 7fbb2895d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbb201a9960 con 0x5631f32fb780 2024-09-12T02:10:18.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.784+0000 7fbb2895d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbb201aadd0 con 0x5631f32ec5d0 2024-09-12T02:10:18.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.784+0000 7fbb2895d740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fbb201aa030 con 0x5631f32edb40 2024-09-12T02:10:18.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.784+0000 7fbb2895d740 10 client.0 _ll_register_callbacks cb 0x7fbb14009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:10:18.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.784+0000 7fbb1f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5631f32fb780 0x5631f32fe850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:18.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.784+0000 7fbb1f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5631f32fb780 0x5631f32fe850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46030/0 (socket says 172.21.15.16:46030) 2024-09-12T02:10:18.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.784+0000 7fbb1f7fe640 1 -- 172.21.15.16:0/2309320100 learned_addr learned my addr 172.21.15.16:0/2309320100 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:10:18.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.784+0000 7fbb1f7fe640 1 --2- 172.21.15.16:0/2309320100 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5631f32ec5d0 0x5631f33038a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:18.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.784+0000 7fbb1ffff640 1 --2- 172.21.15.16:0/2309320100 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5631f32edb40 0x5631f3303de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:18.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.784+0000 7fbb1f7fe640 1 -- 172.21.15.16:0/2309320100 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5631f32edb40 msgr2=0x5631f3303de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:18.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.784+0000 7fbb1f7fe640 1 --2- 172.21.15.16:0/2309320100 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5631f32edb40 0x5631f3303de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:18.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.784+0000 7fbb1f7fe640 1 -- 172.21.15.16:0/2309320100 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5631f32ec5d0 msgr2=0x5631f33038a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:18.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.784+0000 7fbb1f7fe640 1 --2- 172.21.15.16:0/2309320100 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5631f32ec5d0 0x5631f33038a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:18.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.784+0000 7fbb1f7fe640 1 -- 172.21.15.16:0/2309320100 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5631f32fed90 con 0x5631f32fb780 2024-09-12T02:10:18.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.784+0000 7fbb1ffff640 1 --2- 172.21.15.16:0/2309320100 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5631f32edb40 0x5631f3303de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:10:18.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.785+0000 7fbb1f7fe640 1 --2- 172.21.15.16:0/2309320100 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5631f32fb780 0x5631f32fe850 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7fbb14001eb0 tx=0x7fbb14004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9733da183d99fb39 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:10:18.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.785+0000 7fbaff7fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:10:18.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.785+0000 7fbaff7fe640 1 -- 172.21.15.16:0/2309320100 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb14035040 con 0x5631f32fb780 2024-09-12T02:10:18.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.785+0000 7fbb2895d740 1 -- 172.21.15.16:0/2309320100 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb1403f0d0 con 0x5631f32fb780 2024-09-12T02:10:18.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.785+0000 7fbaff7fe640 1 -- 172.21.15.16:0/2309320100 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fbb14004050 con 0x5631f32fb780 2024-09-12T02:10:18.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.785+0000 7fbaff7fe640 1 -- 172.21.15.16:0/2309320100 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fbb14031a30 con 0x5631f32fb780 2024-09-12T02:10:18.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.785+0000 7fbb2895d740 10 client.18051 fetch_fsmap learned FSMap version 25 2024-09-12T02:10:18.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.785+0000 7fbb2895d740 1 -- 172.21.15.16:0/2309320100 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5631f3376430 con 0x5631f32fb780 2024-09-12T02:10:18.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.785+0000 7fbaff7fe640 1 -- 172.21.15.16:0/2309320100 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb1403f0d0 con 0x5631f32fb780 2024-09-12T02:10:18.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.785+0000 7fbaff7fe640 1 -- 172.21.15.16:0/2309320100 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fbb1403f520 con 0x5631f32fb780 2024-09-12T02:10:18.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.785+0000 7fbb2895d740 10 client.18051 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:10:18.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.785+0000 7fbb2895d740 10 client.18051 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:10:18.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.785+0000 7fbb2895d740 20 client.18051 populate_metadata read hostname 'smithi016' 2024-09-12T02:10:18.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.785+0000 7fbb2895d740 1 --2- 172.21.15.16:0/2309320100 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5631f3376a50 0x5631f3378e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:18.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.785+0000 7fbb2895d740 4 client.18051 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T02:10:18.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.785+0000 7fbb2895d740 1 -- 172.21.15.16:0/2309320100 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5631f3259d70 con 0x5631f3376a50 2024-09-12T02:10:18.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.786+0000 7fbb1ffff640 1 --2- 172.21.15.16:0/2309320100 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5631f3376a50 0x5631f3378e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:18.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.786+0000 7fbaff7fe640 10 client.18051 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:10:18.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.786+0000 7fbb1ffff640 1 --2- 172.21.15.16:0/2309320100 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5631f3376a50 0x5631f3378e40 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4320ee0b4183611 server_cookie=258a8039b9a726ed in_seq=0 out_seq=0 2024-09-12T02:10:18.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.786+0000 7fbaff7fe640 1 -- 172.21.15.16:0/2309320100 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5631f3259d70 con 0x5631f3376a50 2024-09-12T02:10:18.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.786+0000 7fbaff7fe640 10 client.18051 handle_command_reply: tid=0 2024-09-12T02:10:18.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.786+0000 7fbb2895d740 1 client.18051 shutdown 2024-09-12T02:10:18.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.786+0000 7fbb2895d740 1 -- 172.21.15.16:0/2309320100 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5631f32fb780 msgr2=0x5631f32fe850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:18.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.786+0000 7fbb2895d740 1 --2- 172.21.15.16:0/2309320100 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5631f32fb780 0x5631f32fe850 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7fbb14001eb0 tx=0x7fbb14004ac0 comp rx=0 tx=0).stop 2024-09-12T02:10:18.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.786+0000 7fbb2895d740 1 -- 172.21.15.16:0/2309320100 shutdown_connections 2024-09-12T02:10:18.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.786+0000 7fbb2895d740 1 --2- 172.21.15.16:0/2309320100 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5631f3376a50 0x5631f3378e40 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:18.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.786+0000 7fbb2895d740 1 --2- 172.21.15.16:0/2309320100 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5631f32fb780 0x5631f32fe850 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:18.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.787+0000 7fbb2895d740 1 --2- 172.21.15.16:0/2309320100 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5631f32edb40 0x5631f3303de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:18.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.787+0000 7fbb2895d740 1 --2- 172.21.15.16:0/2309320100 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5631f32ec5d0 0x5631f33038a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:18.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.787+0000 7fbb2895d740 1 -- 172.21.15.16:0/2309320100 >> 172.21.15.16:0/2309320100 conn(0x5631f32b5310 msgr2=0x5631f32fa410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:10:18.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.787+0000 7fbaff7fe640 0 client.18051 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:10:18.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.787+0000 7fbb2895d740 1 -- 172.21.15.16:0/2309320100 shutdown_connections 2024-09-12T02:10:18.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.787+0000 7fbb2895d740 1 -- 172.21.15.16:0/2309320100 wait complete. 2024-09-12T02:10:18.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.787+0000 7fbb2895d740 20 client.18051 trim_cache size 0 max 16384 2024-09-12T02:10:18.794 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T02:10:18.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.787+0000 7fbb26584640 1 -- 172.21.15.16:0/2177325883 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fbb080a2b80 msgr2=0x7fbb080a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:18.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.787+0000 7fbb26584640 1 --2- 172.21.15.16:0/2177325883 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fbb080a2b80 0x7fbb080a5040 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fbb1800bf60 tx=0x7fbb18065ba0 comp rx=0 tx=0).stop 2024-09-12T02:10:18.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.787+0000 7fbb26584640 1 -- 172.21.15.16:0/2177325883 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb2010c170 msgr2=0x7fbb2019cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:18.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.787+0000 7fbb26584640 1 --2- 172.21.15.16:0/2177325883 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb2010c170 0x7fbb2019cd70 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7fbb1800b970 tx=0x7fbb1800be30 comp rx=0 tx=0).stop 2024-09-12T02:10:18.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.787+0000 7fbb26584640 1 -- 172.21.15.16:0/2177325883 shutdown_connections 2024-09-12T02:10:18.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.787+0000 7fbb26584640 1 --2- 172.21.15.16:0/2177325883 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fbb080a2b80 0x7fbb080a5040 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:18.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.787+0000 7fbb26584640 1 --2- 172.21.15.16:0/2177325883 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb2010c170 0x7fbb2019cd70 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:18.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.787+0000 7fbb26584640 1 --2- 172.21.15.16:0/2177325883 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbb20103660 0x7fbb201989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:18.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.787+0000 7fbb26584640 1 --2- 172.21.15.16:0/2177325883 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbb20102d40 0x7fbb201984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:18.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.787+0000 7fbb26584640 1 -- 172.21.15.16:0/2177325883 >> 172.21.15.16:0/2177325883 conn(0x7fbb200fc7a0 msgr2=0x7fbb200fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:10:18.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.788+0000 7fbb26584640 1 -- 172.21.15.16:0/2177325883 shutdown_connections 2024-09-12T02:10:18.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:18.788+0000 7fbb26584640 1 -- 172.21.15.16:0/2177325883 wait complete. 2024-09-12T02:10:20.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:10:19 smithi184 ceph-mon[35593]: pgmap v587: 97 pgs: 97 active+clean; 4.2 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 7.0 KiB/s wr, 8 op/s 2024-09-12T02:10:20.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:10:19 smithi033 ceph-mon[35332]: pgmap v587: 97 pgs: 97 active+clean; 4.2 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 7.0 KiB/s wr, 8 op/s 2024-09-12T02:10:20.183 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:10:19 smithi016 ceph-mon[30947]: pgmap v587: 97 pgs: 97 active+clean; 4.2 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 7.0 KiB/s wr, 8 op/s 2024-09-12T02:10:20.509 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-12T02:10:20.510 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-12T02:10:20.782 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:10:21.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.241+0000 7f18f7a5d640 1 -- 172.21.15.16:0/3080397125 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18f0104000 msgr2=0x7f18f01043e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:21.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.241+0000 7f18f7a5d640 1 --2- 172.21.15.16:0/3080397125 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18f0104000 0x7f18f01043e0 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f18ec009900 tx=0x7f18ec02a410 comp rx=0 tx=0).stop 2024-09-12T02:10:21.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.241+0000 7f18f7a5d640 1 -- 172.21.15.16:0/3080397125 shutdown_connections 2024-09-12T02:10:21.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.241+0000 7f18f7a5d640 1 --2- 172.21.15.16:0/3080397125 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f18f0109130 0x7f18f0110cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:21.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.241+0000 7f18f7a5d640 1 --2- 172.21.15.16:0/3080397125 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18f01049b0 0x7f18f0108a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:21.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.241+0000 7f18f7a5d640 1 --2- 172.21.15.16:0/3080397125 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18f0104000 0x7f18f01043e0 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:21.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.241+0000 7f18f7a5d640 1 -- 172.21.15.16:0/3080397125 >> 172.21.15.16:0/3080397125 conn(0x7f18f00ffa20 msgr2=0x7f18f0101e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:10:21.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.241+0000 7f18f7a5d640 1 -- 172.21.15.16:0/3080397125 shutdown_connections 2024-09-12T02:10:21.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.241+0000 7f18f7a5d640 1 -- 172.21.15.16:0/3080397125 wait complete. 2024-09-12T02:10:21.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.242+0000 7f18f7a5d640 1 Processor -- start 2024-09-12T02:10:21.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.242+0000 7f18f7a5d640 1 -- start start 2024-09-12T02:10:21.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.242+0000 7f18f7a5d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18f0104000 0x7f18f019a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:21.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.242+0000 7f18f7a5d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f18f01049b0 0x7f18f019ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:21.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.242+0000 7f18f7a5d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18f0109130 0x7f18f019ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:21.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.242+0000 7f18f7a5d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f18f0114130 con 0x7f18f0109130 2024-09-12T02:10:21.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.242+0000 7f18f7a5d640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f18f0113fb0 con 0x7f18f0104000 2024-09-12T02:10:21.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.242+0000 7f18f7a5d640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f18f01142b0 con 0x7f18f01049b0 2024-09-12T02:10:21.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.242+0000 7f18f5ffa640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18f0109130 0x7f18f019ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:21.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.242+0000 7f18f5ffa640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18f0109130 0x7f18f019ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60678/0 (socket says 172.21.15.16:60678) 2024-09-12T02:10:21.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.242+0000 7f18f5ffa640 1 -- 172.21.15.16:0/1316554366 learned_addr learned my addr 172.21.15.16:0/1316554366 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:10:21.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.243+0000 7f18f4ff8640 1 --2- 172.21.15.16:0/1316554366 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f18f01049b0 0x7f18f019ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:21.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.243+0000 7f18f57f9640 1 --2- 172.21.15.16:0/1316554366 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18f0104000 0x7f18f019a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:21.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.243+0000 7f18f5ffa640 1 -- 172.21.15.16:0/1316554366 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f18f01049b0 msgr2=0x7f18f019ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:21.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.243+0000 7f18f5ffa640 1 --2- 172.21.15.16:0/1316554366 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f18f01049b0 0x7f18f019ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:21.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.243+0000 7f18f5ffa640 1 -- 172.21.15.16:0/1316554366 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18f0104000 msgr2=0x7f18f019a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:21.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.243+0000 7f18f5ffa640 1 --2- 172.21.15.16:0/1316554366 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18f0104000 0x7f18f019a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:21.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.243+0000 7f18f5ffa640 1 -- 172.21.15.16:0/1316554366 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18f019f5e0 con 0x7f18f0109130 2024-09-12T02:10:21.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.243+0000 7f18f57f9640 1 --2- 172.21.15.16:0/1316554366 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18f0104000 0x7f18f019a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:10:21.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.243+0000 7f18f5ffa640 1 --2- 172.21.15.16:0/1316554366 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18f0109130 0x7f18f019ef00 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f18e800eaa0 tx=0x7f18e800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:10:21.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.243+0000 7f18e67fc640 1 -- 172.21.15.16:0/1316554366 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18e800cdb0 con 0x7f18f0109130 2024-09-12T02:10:21.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.243+0000 7f18f7a5d640 1 -- 172.21.15.16:0/1316554366 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18f019f8d0 con 0x7f18f0109130 2024-09-12T02:10:21.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.243+0000 7f18e67fc640 1 -- 172.21.15.16:0/1316554366 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18e8005020 con 0x7f18f0109130 2024-09-12T02:10:21.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.243+0000 7f18e67fc640 1 -- 172.21.15.16:0/1316554366 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18e80116b0 con 0x7f18f0109130 2024-09-12T02:10:21.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.243+0000 7f18f7a5d640 1 -- 172.21.15.16:0/1316554366 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18f01a71b0 con 0x7f18f0109130 2024-09-12T02:10:21.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.245+0000 7f18e67fc640 1 -- 172.21.15.16:0/1316554366 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f18e8005560 con 0x7f18f0109130 2024-09-12T02:10:21.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.246+0000 7f18f9e36740 1 Processor -- start 2024-09-12T02:10:21.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.246+0000 7f18f9e36740 1 -- start start 2024-09-12T02:10:21.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.246+0000 7f18f9e36740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a0f4e4db60 0x55a0f4e4df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:21.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.246+0000 7f18f9e36740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a0f4e4e480 0x55a0f4e58800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:21.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.246+0000 7f18f9e36740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a0f4e58d40 0x55a0f4e5b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:21.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.246+0000 7f18f9e36740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55a0f4db3280 con 0x55a0f4e4e480 2024-09-12T02:10:21.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.246+0000 7f18f9e36740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55a0f4d5ab10 con 0x55a0f4e4db60 2024-09-12T02:10:21.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.246+0000 7f18f9e36740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55a0f4daf240 con 0x55a0f4e58d40 2024-09-12T02:10:21.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.246+0000 7f18e67fc640 1 --2- 172.21.15.16:0/1316554366 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f18dc0a2c00 0x7f18dc0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:21.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.246+0000 7f18f57f9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a0f4e4e480 0x55a0f4e58800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:21.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.246+0000 7f18f57f9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a0f4e4e480 0x55a0f4e58800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60690/0 (socket says 172.21.15.16:60690) 2024-09-12T02:10:21.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.246+0000 7f18f57f9640 1 -- 172.21.15.16:0/4017898221 learned_addr learned my addr 172.21.15.16:0/4017898221 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:10:21.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.247+0000 7f18f5ffa640 1 --2- 172.21.15.16:0/1316554366 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f18dc0a2c00 0x7f18dc0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:21.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.247+0000 7f18f4ff8640 1 --2- 172.21.15.16:0/4017898221 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a0f4e4db60 0x55a0f4e4df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:21.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.247+0000 7f18f4ff8640 1 --2- 172.21.15.16:0/4017898221 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a0f4e58d40 0x55a0f4e5b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:21.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.247+0000 7f18f57f9640 1 -- 172.21.15.16:0/4017898221 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a0f4e58d40 msgr2=0x55a0f4e5b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:21.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.247+0000 7f18f57f9640 1 --2- 172.21.15.16:0/4017898221 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a0f4e58d40 0x55a0f4e5b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:21.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.247+0000 7f18f57f9640 1 -- 172.21.15.16:0/4017898221 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a0f4e4db60 msgr2=0x55a0f4e4df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:21.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.247+0000 7f18f57f9640 1 --2- 172.21.15.16:0/4017898221 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a0f4e4db60 0x55a0f4e4df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:21.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.247+0000 7f18f57f9640 1 -- 172.21.15.16:0/4017898221 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0f4d671f0 con 0x55a0f4e4e480 2024-09-12T02:10:21.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.247+0000 7f18f4ff8640 1 --2- 172.21.15.16:0/4017898221 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a0f4e58d40 0x55a0f4e5b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:10:21.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.247+0000 7f18e67fc640 1 -- 172.21.15.16:0/1316554366 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f18e80cf270 con 0x7f18f0109130 2024-09-12T02:10:21.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.247+0000 7f18f57f9640 1 --2- 172.21.15.16:0/4017898221 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a0f4e4e480 0x55a0f4e58800 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f18ec031550 tx=0x7f18ec032cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=827c4cfb535502f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:10:21.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.247+0000 7f18db7fe640 1 -- 172.21.15.16:0/4017898221 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18ec03f070 con 0x55a0f4e4e480 2024-09-12T02:10:21.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.247+0000 7f18db7fe640 1 -- 172.21.15.16:0/4017898221 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18ec0317b0 con 0x55a0f4e4e480 2024-09-12T02:10:21.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.247+0000 7f18db7fe640 1 -- 172.21.15.16:0/4017898221 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18ec035070 con 0x55a0f4e4e480 2024-09-12T02:10:21.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.248+0000 7f18f9e36740 1 -- 172.21.15.16:0/4017898221 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a0f4e4e480 msgr2=0x55a0f4e58800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:21.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.248+0000 7f18f9e36740 1 --2- 172.21.15.16:0/4017898221 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a0f4e4e480 0x55a0f4e58800 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f18ec031550 tx=0x7f18ec032cd0 comp rx=0 tx=0).stop 2024-09-12T02:10:21.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.248+0000 7f18f5ffa640 1 --2- 172.21.15.16:0/1316554366 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f18dc0a2c00 0x7f18dc0a50c0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f18e80984a0 tx=0x7f18e80054d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:10:21.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.248+0000 7f18f9e36740 1 -- 172.21.15.16:0/4017898221 shutdown_connections 2024-09-12T02:10:21.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.248+0000 7f18f9e36740 1 --2- 172.21.15.16:0/4017898221 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a0f4e58d40 0x55a0f4e5b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:21.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.248+0000 7f18f9e36740 1 --2- 172.21.15.16:0/4017898221 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a0f4e4e480 0x55a0f4e58800 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:21.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.248+0000 7f18f9e36740 1 --2- 172.21.15.16:0/4017898221 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a0f4e4db60 0x55a0f4e4df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:21.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.248+0000 7f18f9e36740 1 -- 172.21.15.16:0/4017898221 >> 172.21.15.16:0/4017898221 conn(0x55a0f4e4a0f0 msgr2=0x55a0f4e4a4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:10:21.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.248+0000 7f18f9e36740 1 -- 172.21.15.16:0/4017898221 shutdown_connections 2024-09-12T02:10:21.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.248+0000 7f18f9e36740 1 -- 172.21.15.16:0/4017898221 wait complete. 2024-09-12T02:10:21.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.249+0000 7f18f9e36740 1 -- start start 2024-09-12T02:10:21.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.249+0000 7f18f9e36740 1 Processor -- start 2024-09-12T02:10:21.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.249+0000 7f18f9e36740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a0f4e4db60 0x55a0f4e55c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:21.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.249+0000 7f18f9e36740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a0f4e4e480 0x55a0f4e56140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:21.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.249+0000 7f18f9e36740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a0f4e58d40 0x55a0f4e56680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:21.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.249+0000 7f18f9e36740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55a0f4e5b910 con 0x55a0f4e58d40 2024-09-12T02:10:21.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.249+0000 7f18f9e36740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55a0f4d6df10 con 0x55a0f4e4e480 2024-09-12T02:10:21.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.249+0000 7f18f9e36740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55a0f4e5ba90 con 0x55a0f4e4db60 2024-09-12T02:10:21.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.249+0000 7f18f9e36740 10 client.0 _ll_register_callbacks cb 0x55a0f4d36a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:10:21.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.249+0000 7f18f4ff8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a0f4e58d40 0x55a0f4e56680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:21.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.249+0000 7f18f4ff8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a0f4e58d40 0x55a0f4e56680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60706/0 (socket says 172.21.15.16:60706) 2024-09-12T02:10:21.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.249+0000 7f18f4ff8640 1 -- 172.21.15.16:0/1199884697 learned_addr learned my addr 172.21.15.16:0/1199884697 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:10:21.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.249+0000 7f18f57f9640 1 --2- 172.21.15.16:0/1199884697 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a0f4e4e480 0x55a0f4e56140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:21.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.249+0000 7f18f4ff8640 1 --2- 172.21.15.16:0/1199884697 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a0f4e4db60 0x55a0f4e55c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:21.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.250+0000 7f18f4ff8640 1 -- 172.21.15.16:0/1199884697 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a0f4e4db60 msgr2=0x55a0f4e55c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:21.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.250+0000 7f18f4ff8640 1 --2- 172.21.15.16:0/1199884697 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a0f4e4db60 0x55a0f4e55c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:21.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.250+0000 7f18f4ff8640 1 -- 172.21.15.16:0/1199884697 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a0f4e4e480 msgr2=0x55a0f4e56140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:21.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.250+0000 7f18f4ff8640 1 --2- 172.21.15.16:0/1199884697 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a0f4e4e480 0x55a0f4e56140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:21.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.250+0000 7f18f4ff8640 1 -- 172.21.15.16:0/1199884697 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0f4b246a0 con 0x55a0f4e58d40 2024-09-12T02:10:21.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.250+0000 7f18f4ff8640 1 --2- 172.21.15.16:0/1199884697 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a0f4e58d40 0x55a0f4e56680 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f18f00ffe40 tx=0x7f18e0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=639bde288dbe9fa0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:10:21.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.250+0000 7f18d8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:10:21.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.250+0000 7f18d8ff9640 1 -- 172.21.15.16:0/1199884697 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18e0011d70 con 0x55a0f4e58d40 2024-09-12T02:10:21.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.250+0000 7f18d8ff9640 1 -- 172.21.15.16:0/1199884697 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f18e0005020 con 0x55a0f4e58d40 2024-09-12T02:10:21.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.250+0000 7f18f9e36740 1 -- 172.21.15.16:0/1199884697 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18ec0317b0 con 0x55a0f4e58d40 2024-09-12T02:10:21.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.250+0000 7f18d8ff9640 1 -- 172.21.15.16:0/1199884697 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f18e0026050 con 0x55a0f4e58d40 2024-09-12T02:10:21.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.250+0000 7f18f9e36740 10 client.18063 fetch_fsmap learned FSMap version 25 2024-09-12T02:10:21.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.250+0000 7f18f9e36740 1 -- 172.21.15.16:0/1199884697 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a0f4e674c0 con 0x55a0f4e58d40 2024-09-12T02:10:21.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.250+0000 7f18d8ff9640 1 -- 172.21.15.16:0/1199884697 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18ec0317b0 con 0x55a0f4e58d40 2024-09-12T02:10:21.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.251+0000 7f18d8ff9640 1 -- 172.21.15.16:0/1199884697 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f18e000f040 con 0x55a0f4e58d40 2024-09-12T02:10:21.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.251+0000 7f18f9e36740 10 client.18063 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:10:21.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.251+0000 7f18f9e36740 10 client.18063 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:10:21.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.251+0000 7f18f9e36740 20 client.18063 populate_metadata read hostname 'smithi016' 2024-09-12T02:10:21.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.251+0000 7f18f9e36740 1 --2- 172.21.15.16:0/1199884697 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55a0f4e688d0 0x55a0f4e6ace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:21.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.251+0000 7f18f9e36740 4 client.18063 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:10:21.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.251+0000 7f18f9e36740 1 -- 172.21.15.16:0/1199884697 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a0f4dc07f0 con 0x55a0f4e688d0 2024-09-12T02:10:21.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.251+0000 7f18f57f9640 1 --2- 172.21.15.16:0/1199884697 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55a0f4e688d0 0x55a0f4e6ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:21.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.252+0000 7f18f57f9640 1 --2- 172.21.15.16:0/1199884697 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55a0f4e688d0 0x55a0f4e6ace0 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d46d74f96a83ac0 server_cookie=56bd3e74e9ab4c64 in_seq=0 out_seq=0 2024-09-12T02:10:21.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.252+0000 7f18d8ff9640 10 client.18063 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:10:21.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.252+0000 7f18d8ff9640 1 -- 172.21.15.16:0/1199884697 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a0f4dc07f0 con 0x55a0f4e688d0 2024-09-12T02:10:21.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.252+0000 7f18d8ff9640 10 client.18063 handle_command_reply: tid=0 2024-09-12T02:10:21.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.252+0000 7f18f9e36740 1 client.18063 shutdown 2024-09-12T02:10:21.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.252+0000 7f18f9e36740 1 -- 172.21.15.16:0/1199884697 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a0f4e58d40 msgr2=0x55a0f4e56680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:21.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.252+0000 7f18f9e36740 1 --2- 172.21.15.16:0/1199884697 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a0f4e58d40 0x55a0f4e56680 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f18f00ffe40 tx=0x7f18e0004730 comp rx=0 tx=0).stop 2024-09-12T02:10:21.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.252+0000 7f18f9e36740 1 -- 172.21.15.16:0/1199884697 shutdown_connections 2024-09-12T02:10:21.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.252+0000 7f18f9e36740 1 --2- 172.21.15.16:0/1199884697 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55a0f4e688d0 0x55a0f4e6ace0 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:21.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.252+0000 7f18f9e36740 1 --2- 172.21.15.16:0/1199884697 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a0f4e58d40 0x55a0f4e56680 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:21.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.252+0000 7f18f9e36740 1 --2- 172.21.15.16:0/1199884697 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a0f4e4e480 0x55a0f4e56140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:21.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.252+0000 7f18f9e36740 1 --2- 172.21.15.16:0/1199884697 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a0f4e4db60 0x55a0f4e55c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:21.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.252+0000 7f18f9e36740 1 -- 172.21.15.16:0/1199884697 >> 172.21.15.16:0/1199884697 conn(0x55a0f4e4a0f0 msgr2=0x55a0f4e49270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:10:21.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.253+0000 7f18d8ff9640 0 client.18063 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:10:21.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.253+0000 7f18f9e36740 1 -- 172.21.15.16:0/1199884697 shutdown_connections 2024-09-12T02:10:21.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.253+0000 7f18f9e36740 1 -- 172.21.15.16:0/1199884697 wait complete. 2024-09-12T02:10:21.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.253+0000 7f18f9e36740 20 client.18063 trim_cache size 0 max 16384 2024-09-12T02:10:21.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.270+0000 7f18f9e36740 1 Processor -- start 2024-09-12T02:10:21.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.270+0000 7f18f9e36740 1 -- start start 2024-09-12T02:10:21.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.270+0000 7f18f9e36740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a0f4e4db60 0x55a0f4e4fd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:21.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.270+0000 7f18f9e36740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a0f4e4e480 0x55a0f4e705b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:21.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.270+0000 7f18f9e36740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a0f4e58d40 0x55a0f4e70af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:21.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.270+0000 7f18f9e36740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f18f01b1f30 con 0x55a0f4e58d40 2024-09-12T02:10:21.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.270+0000 7f18f9e36740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f18f01af800 con 0x55a0f4e4e480 2024-09-12T02:10:21.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.270+0000 7f18f9e36740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f18f01ad650 con 0x55a0f4e4db60 2024-09-12T02:10:21.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.270+0000 7f18f4ff8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a0f4e58d40 0x55a0f4e70af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:21.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.271+0000 7f18f4ff8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a0f4e58d40 0x55a0f4e70af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60714/0 (socket says 172.21.15.16:60714) 2024-09-12T02:10:21.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.271+0000 7f18f4ff8640 1 -- 172.21.15.16:0/1207667874 learned_addr learned my addr 172.21.15.16:0/1207667874 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:10:21.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.271+0000 7f18f4ff8640 1 --2- 172.21.15.16:0/1207667874 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a0f4e4db60 0x55a0f4e4fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:21.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.271+0000 7f18f57f9640 1 --2- 172.21.15.16:0/1207667874 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a0f4e4e480 0x55a0f4e705b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:21.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.271+0000 7f18f4ff8640 1 -- 172.21.15.16:0/1207667874 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a0f4e4db60 msgr2=0x55a0f4e4fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:21.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.271+0000 7f18f4ff8640 1 --2- 172.21.15.16:0/1207667874 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a0f4e4db60 0x55a0f4e4fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:21.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.271+0000 7f18f4ff8640 1 -- 172.21.15.16:0/1207667874 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a0f4e4e480 msgr2=0x55a0f4e705b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:21.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.271+0000 7f18f4ff8640 1 --2- 172.21.15.16:0/1207667874 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a0f4e4e480 0x55a0f4e705b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:21.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.271+0000 7f18f4ff8640 1 -- 172.21.15.16:0/1207667874 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0f4e54150 con 0x55a0f4e58d40 2024-09-12T02:10:21.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.271+0000 7f18f57f9640 1 --2- 172.21.15.16:0/1207667874 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a0f4e4e480 0x55a0f4e705b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:10:21.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.271+0000 7f18f4ff8640 1 --2- 172.21.15.16:0/1207667874 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a0f4e58d40 0x55a0f4e70af0 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f18e0004700 tx=0x7f18e000e7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbc79dba8d6d68ed server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:10:21.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.271+0000 7f18daffd640 1 -- 172.21.15.16:0/1207667874 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18e002f070 con 0x55a0f4e58d40 2024-09-12T02:10:21.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.271+0000 7f18daffd640 1 -- 172.21.15.16:0/1207667874 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18e0010e20 con 0x55a0f4e58d40 2024-09-12T02:10:21.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.271+0000 7f18daffd640 1 -- 172.21.15.16:0/1207667874 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18e0000e60 con 0x55a0f4e58d40 2024-09-12T02:10:21.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.272+0000 7f18f9e36740 1 -- 172.21.15.16:0/1207667874 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a0f4e58d40 msgr2=0x55a0f4e70af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:21.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.272+0000 7f18f9e36740 1 --2- 172.21.15.16:0/1207667874 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a0f4e58d40 0x55a0f4e70af0 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f18e0004700 tx=0x7f18e000e7a0 comp rx=0 tx=0).stop 2024-09-12T02:10:21.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.272+0000 7f18f9e36740 1 -- 172.21.15.16:0/1207667874 shutdown_connections 2024-09-12T02:10:21.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.272+0000 7f18f9e36740 1 --2- 172.21.15.16:0/1207667874 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a0f4e58d40 0x55a0f4e70af0 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:21.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.272+0000 7f18f9e36740 1 --2- 172.21.15.16:0/1207667874 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a0f4e4e480 0x55a0f4e705b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:21.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.272+0000 7f18f9e36740 1 --2- 172.21.15.16:0/1207667874 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a0f4e4db60 0x55a0f4e4fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:21.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.272+0000 7f18f9e36740 1 -- 172.21.15.16:0/1207667874 >> 172.21.15.16:0/1207667874 conn(0x55a0f4e4a0f0 msgr2=0x55a0f4e4f600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:10:21.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.272+0000 7f18f9e36740 1 -- 172.21.15.16:0/1207667874 shutdown_connections 2024-09-12T02:10:21.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.272+0000 7f18f9e36740 1 -- 172.21.15.16:0/1207667874 wait complete. 2024-09-12T02:10:21.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.272+0000 7f18f9e36740 1 -- start start 2024-09-12T02:10:21.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.272+0000 7f18f9e36740 1 Processor -- start 2024-09-12T02:10:21.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.273+0000 7f18f9e36740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a0f4e4db60 0x55a0f4e5d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:21.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.273+0000 7f18f9e36740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a0f4e4e480 0x55a0f4e5d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:21.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.273+0000 7f18f9e36740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a0f4e58d40 0x55a0f4ed2580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:21.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.273+0000 7f18f9e36740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f18f01adcc0 con 0x55a0f4e4e480 2024-09-12T02:10:21.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.273+0000 7f18f9e36740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f18f01af130 con 0x55a0f4e58d40 2024-09-12T02:10:21.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.273+0000 7f18f9e36740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f18f01ae390 con 0x55a0f4e4db60 2024-09-12T02:10:21.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.273+0000 7f18f9e36740 10 client.0 _ll_register_callbacks cb 0x7f18e000e650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:10:21.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.273+0000 7f18f57f9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a0f4e4e480 0x55a0f4e5d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:21.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.273+0000 7f18f57f9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a0f4e4e480 0x55a0f4e5d730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60724/0 (socket says 172.21.15.16:60724) 2024-09-12T02:10:21.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.273+0000 7f18f57f9640 1 -- 172.21.15.16:0/3231380630 learned_addr learned my addr 172.21.15.16:0/3231380630 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:10:21.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.273+0000 7f18f4ff8640 1 --2- 172.21.15.16:0/3231380630 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a0f4e4db60 0x55a0f4e5d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:21.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.273+0000 7f18f4ff8640 1 --2- 172.21.15.16:0/3231380630 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a0f4e58d40 0x55a0f4ed2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:21.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.273+0000 7f18f57f9640 1 -- 172.21.15.16:0/3231380630 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a0f4e4db60 msgr2=0x55a0f4e5d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:21.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.273+0000 7f18f57f9640 1 --2- 172.21.15.16:0/3231380630 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a0f4e4db60 0x55a0f4e5d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:21.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.273+0000 7f18f57f9640 1 -- 172.21.15.16:0/3231380630 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a0f4e58d40 msgr2=0x55a0f4ed2580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:21.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.273+0000 7f18f57f9640 1 --2- 172.21.15.16:0/3231380630 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a0f4e58d40 0x55a0f4ed2580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:21.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.273+0000 7f18f57f9640 1 -- 172.21.15.16:0/3231380630 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0f4ed2af0 con 0x55a0f4e4e480 2024-09-12T02:10:21.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.273+0000 7f18f57f9640 1 --2- 172.21.15.16:0/3231380630 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a0f4e4e480 0x55a0f4e5d730 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f18ec009900 tx=0x7f18ec032de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=817a8b98a2ddb3f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:10:21.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.273+0000 7f18d8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:10:21.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.274+0000 7f18d8ff9640 1 -- 172.21.15.16:0/3231380630 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18ec035070 con 0x55a0f4e4e480 2024-09-12T02:10:21.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.274+0000 7f18d8ff9640 1 -- 172.21.15.16:0/3231380630 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f18ec031830 con 0x55a0f4e4e480 2024-09-12T02:10:21.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.274+0000 7f18f9e36740 1 -- 172.21.15.16:0/3231380630 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18e0010e20 con 0x55a0f4e4e480 2024-09-12T02:10:21.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.274+0000 7f18d8ff9640 1 -- 172.21.15.16:0/3231380630 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f18ec060050 con 0x55a0f4e4e480 2024-09-12T02:10:21.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.274+0000 7f18f9e36740 10 client.18069 fetch_fsmap learned FSMap version 25 2024-09-12T02:10:21.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.274+0000 7f18f9e36740 1 -- 172.21.15.16:0/3231380630 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a0f4ed3de0 con 0x55a0f4e4e480 2024-09-12T02:10:21.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.274+0000 7f18d8ff9640 1 -- 172.21.15.16:0/3231380630 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18e0010e20 con 0x55a0f4e4e480 2024-09-12T02:10:21.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.274+0000 7f18d8ff9640 1 -- 172.21.15.16:0/3231380630 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f18ec04e040 con 0x55a0f4e4e480 2024-09-12T02:10:21.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.274+0000 7f18f9e36740 10 client.18069 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:10:21.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.274+0000 7f18f9e36740 10 client.18069 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:10:21.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.274+0000 7f18f9e36740 20 client.18069 populate_metadata read hostname 'smithi016' 2024-09-12T02:10:21.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.274+0000 7f18f9e36740 1 --2- 172.21.15.16:0/3231380630 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55a0f4ed4120 0x55a0f4ed6510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:21.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.274+0000 7f18f9e36740 4 client.18069 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-12T02:10:21.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.274+0000 7f18f9e36740 1 -- 172.21.15.16:0/3231380630 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55a0f4dbad00 con 0x55a0f4ed4120 2024-09-12T02:10:21.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.275+0000 7f18f4ff8640 1 --2- 172.21.15.16:0/3231380630 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55a0f4ed4120 0x55a0f4ed6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:21.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.275+0000 7f18f4ff8640 1 --2- 172.21.15.16:0/3231380630 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55a0f4ed4120 0x55a0f4ed6510 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d76837c918af688b server_cookie=92be0365a6ac4de7 in_seq=0 out_seq=0 2024-09-12T02:10:21.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.275+0000 7f18d8ff9640 10 client.18069 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:10:21.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.276+0000 7f18d8ff9640 1 -- 172.21.15.16:0/3231380630 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55a0f4dbad00 con 0x55a0f4ed4120 2024-09-12T02:10:21.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.276+0000 7f18d8ff9640 10 client.18069 handle_command_reply: tid=0 2024-09-12T02:10:21.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.276+0000 7f18f9e36740 1 client.18069 shutdown 2024-09-12T02:10:21.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.276+0000 7f18f9e36740 1 -- 172.21.15.16:0/3231380630 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a0f4e4e480 msgr2=0x55a0f4e5d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:21.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.276+0000 7f18f9e36740 1 --2- 172.21.15.16:0/3231380630 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a0f4e4e480 0x55a0f4e5d730 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f18ec009900 tx=0x7f18ec032de0 comp rx=0 tx=0).stop 2024-09-12T02:10:21.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.276+0000 7f18f9e36740 1 -- 172.21.15.16:0/3231380630 shutdown_connections 2024-09-12T02:10:21.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.276+0000 7f18f9e36740 1 --2- 172.21.15.16:0/3231380630 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55a0f4ed4120 0x55a0f4ed6510 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:21.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.276+0000 7f18f9e36740 1 --2- 172.21.15.16:0/3231380630 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a0f4e58d40 0x55a0f4ed2580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:21.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.276+0000 7f18f9e36740 1 --2- 172.21.15.16:0/3231380630 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a0f4e4e480 0x55a0f4e5d730 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:21.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.276+0000 7f18f9e36740 1 --2- 172.21.15.16:0/3231380630 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a0f4e4db60 0x55a0f4e5d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:21.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.276+0000 7f18f9e36740 1 -- 172.21.15.16:0/3231380630 >> 172.21.15.16:0/3231380630 conn(0x55a0f4e4a0f0 msgr2=0x55a0f4e4fe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:10:21.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.276+0000 7f18d8ff9640 0 client.18069 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:10:21.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.277+0000 7f18f9e36740 1 -- 172.21.15.16:0/3231380630 shutdown_connections 2024-09-12T02:10:21.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.277+0000 7f18f9e36740 1 -- 172.21.15.16:0/3231380630 wait complete. 2024-09-12T02:10:21.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.277+0000 7f18f9e36740 20 client.18069 trim_cache size 0 max 16384 2024-09-12T02:10:21.285 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T02:10:21.285 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-09-12T02:10:21.285 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "ca0a9334-6276-41ec-a5ff-01a6150d2a6c", 2024-09-12T02:10:21.285 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-09-12T02:10:21.285 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T02:10:21.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.277+0000 7f18f7a5d640 1 -- 172.21.15.16:0/1316554366 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f18dc0a2c00 msgr2=0x7f18dc0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:21.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.277+0000 7f18f7a5d640 1 --2- 172.21.15.16:0/1316554366 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f18dc0a2c00 0x7f18dc0a50c0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f18e80984a0 tx=0x7f18e80054d0 comp rx=0 tx=0).stop 2024-09-12T02:10:21.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.277+0000 7f18f7a5d640 1 -- 172.21.15.16:0/1316554366 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18f0109130 msgr2=0x7f18f019ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:21.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.277+0000 7f18f7a5d640 1 --2- 172.21.15.16:0/1316554366 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18f0109130 0x7f18f019ef00 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f18e800eaa0 tx=0x7f18e800ef60 comp rx=0 tx=0).stop 2024-09-12T02:10:21.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.277+0000 7f18f7a5d640 1 -- 172.21.15.16:0/1316554366 shutdown_connections 2024-09-12T02:10:21.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.277+0000 7f18f7a5d640 1 --2- 172.21.15.16:0/1316554366 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f18dc0a2c00 0x7f18dc0a50c0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:21.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.277+0000 7f18f7a5d640 1 --2- 172.21.15.16:0/1316554366 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18f0109130 0x7f18f019ef00 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:21.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.277+0000 7f18f7a5d640 1 --2- 172.21.15.16:0/1316554366 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f18f01049b0 0x7f18f019ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:21.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.277+0000 7f18f7a5d640 1 --2- 172.21.15.16:0/1316554366 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18f0104000 0x7f18f019a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:21.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.277+0000 7f18f7a5d640 1 -- 172.21.15.16:0/1316554366 >> 172.21.15.16:0/1316554366 conn(0x7f18f00ffa20 msgr2=0x7f18f0100da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:10:21.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.277+0000 7f18f7a5d640 1 -- 172.21.15.16:0/1316554366 shutdown_connections 2024-09-12T02:10:21.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:21.277+0000 7f18f7a5d640 1 -- 172.21.15.16:0/1316554366 wait complete. 2024-09-12T02:10:21.894 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:10:21 smithi016 ceph-mon[30947]: pgmap v588: 97 pgs: 97 active+clean; 4.3 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-12T02:10:21.894 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:10:21 smithi016 ceph-mon[30947]: scrub queued for path: / 2024-09-12T02:10:21.894 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:10:21 smithi016 ceph-mon[30947]: scrub summary: idle+waiting paths [/] 2024-09-12T02:10:21.894 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:10:21 smithi016 ceph-mon[30947]: scrub summary: active paths [/] 2024-09-12T02:10:21.894 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:10:21 smithi016 ceph-mon[30947]: scrub summary: idle+waiting paths [/] 2024-09-12T02:10:21.923 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T02:10:22.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:10:21 smithi184 ceph-mon[35593]: pgmap v588: 97 pgs: 97 active+clean; 4.3 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-12T02:10:22.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:10:21 smithi184 ceph-mon[35593]: scrub queued for path: / 2024-09-12T02:10:22.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:10:21 smithi184 ceph-mon[35593]: scrub summary: idle+waiting paths [/] 2024-09-12T02:10:22.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:10:21 smithi184 ceph-mon[35593]: scrub summary: active paths [/] 2024-09-12T02:10:22.128 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:10:21 smithi184 ceph-mon[35593]: scrub summary: idle+waiting paths [/] 2024-09-12T02:10:22.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:10:21 smithi033 ceph-mon[35332]: pgmap v588: 97 pgs: 97 active+clean; 4.3 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-12T02:10:22.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:10:21 smithi033 ceph-mon[35332]: scrub queued for path: / 2024-09-12T02:10:22.170 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:10:21 smithi033 ceph-mon[35332]: scrub summary: idle+waiting paths [/] 2024-09-12T02:10:22.170 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:10:21 smithi033 ceph-mon[35332]: scrub summary: active paths [/] 2024-09-12T02:10:22.170 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:10:21 smithi033 ceph-mon[35332]: scrub summary: idle+waiting paths [/] 2024-09-12T02:10:22.187 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:10:22.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.638+0000 7fb8d2cc8640 1 -- 172.21.15.16:0/805592578 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8cc102ef0 msgr2=0x7fb8cc10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:22.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.638+0000 7fb8d2cc8640 1 --2- 172.21.15.16:0/805592578 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8cc102ef0 0x7fb8cc10da40 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7fb8c4009ca0 tx=0x7fb8c402a950 comp rx=0 tx=0).stop 2024-09-12T02:10:22.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.638+0000 7fb8d2cc8640 1 -- 172.21.15.16:0/805592578 shutdown_connections 2024-09-12T02:10:22.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.638+0000 7fb8d2cc8640 1 --2- 172.21.15.16:0/805592578 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8cc102ef0 0x7fb8cc10da40 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:22.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.638+0000 7fb8d2cc8640 1 --2- 172.21.15.16:0/805592578 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb8cc102570 0x7fb8cc1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:22.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.638+0000 7fb8d2cc8640 1 --2- 172.21.15.16:0/805592578 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb8cc100e20 0x7fb8cc101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:22.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.638+0000 7fb8d2cc8640 1 -- 172.21.15.16:0/805592578 >> 172.21.15.16:0/805592578 conn(0x7fb8cc0fc7d0 msgr2=0x7fb8cc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:10:22.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.638+0000 7fb8d2cc8640 1 -- 172.21.15.16:0/805592578 shutdown_connections 2024-09-12T02:10:22.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.638+0000 7fb8d2cc8640 1 -- 172.21.15.16:0/805592578 wait complete. 2024-09-12T02:10:22.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.638+0000 7fb8d2cc8640 1 Processor -- start 2024-09-12T02:10:22.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.639+0000 7fb8d2cc8640 1 -- start start 2024-09-12T02:10:22.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.639+0000 7fb8d2cc8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb8cc100e20 0x7fb8cc1985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:22.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.639+0000 7fb8d2cc8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8cc102570 0x7fb8cc198b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:22.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.639+0000 7fb8d2cc8640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb8cc102ef0 0x7fb8cc19ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:22.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.639+0000 7fb8d2cc8640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb8cc111020 con 0x7fb8cc102570 2024-09-12T02:10:22.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.639+0000 7fb8d2cc8640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb8cc110ea0 con 0x7fb8cc100e20 2024-09-12T02:10:22.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.639+0000 7fb8d2cc8640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb8cc1111a0 con 0x7fb8cc102ef0 2024-09-12T02:10:22.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.639+0000 7fb8cbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8cc102570 0x7fb8cc198b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:22.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.639+0000 7fb8cbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8cc102570 0x7fb8cc198b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60736/0 (socket says 172.21.15.16:60736) 2024-09-12T02:10:22.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.639+0000 7fb8cbfff640 1 -- 172.21.15.16:0/4131727408 learned_addr learned my addr 172.21.15.16:0/4131727408 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:10:22.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.639+0000 7fb8d1265640 1 --2- 172.21.15.16:0/4131727408 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb8cc102ef0 0x7fb8cc19ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:22.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.639+0000 7fb8d0a64640 1 --2- 172.21.15.16:0/4131727408 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb8cc100e20 0x7fb8cc1985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:22.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.639+0000 7fb8cbfff640 1 -- 172.21.15.16:0/4131727408 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb8cc102ef0 msgr2=0x7fb8cc19ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:22.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.639+0000 7fb8cbfff640 1 --2- 172.21.15.16:0/4131727408 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb8cc102ef0 0x7fb8cc19ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:22.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.639+0000 7fb8cbfff640 1 -- 172.21.15.16:0/4131727408 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb8cc100e20 msgr2=0x7fb8cc1985e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:22.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.639+0000 7fb8cbfff640 1 --2- 172.21.15.16:0/4131727408 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb8cc100e20 0x7fb8cc1985e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:22.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.639+0000 7fb8cbfff640 1 -- 172.21.15.16:0/4131727408 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8cc19d590 con 0x7fb8cc102570 2024-09-12T02:10:22.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.639+0000 7fb8d0a64640 1 --2- 172.21.15.16:0/4131727408 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb8cc100e20 0x7fb8cc1985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:10:22.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.639+0000 7fb8cbfff640 1 --2- 172.21.15.16:0/4131727408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8cc102570 0x7fb8cc198b20 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7fb8b800b4a0 tx=0x7fb8b800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:10:22.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.640+0000 7fb8c9ffb640 1 -- 172.21.15.16:0/4131727408 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8b8014070 con 0x7fb8cc102570 2024-09-12T02:10:22.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.640+0000 7fb8c9ffb640 1 -- 172.21.15.16:0/4131727408 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8b8010af0 con 0x7fb8cc102570 2024-09-12T02:10:22.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.640+0000 7fb8d2cc8640 1 -- 172.21.15.16:0/4131727408 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8cc19d880 con 0x7fb8cc102570 2024-09-12T02:10:22.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.640+0000 7fb8c9ffb640 1 -- 172.21.15.16:0/4131727408 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8b8017d10 con 0x7fb8cc102570 2024-09-12T02:10:22.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.640+0000 7fb8d2cc8640 1 -- 172.21.15.16:0/4131727408 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8cc1a5160 con 0x7fb8cc102570 2024-09-12T02:10:22.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.642+0000 7fb8c9ffb640 1 -- 172.21.15.16:0/4131727408 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fb8b8017070 con 0x7fb8cc102570 2024-09-12T02:10:22.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.642+0000 7fb8d50a1740 1 Processor -- start 2024-09-12T02:10:22.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.642+0000 7fb8d50a1740 1 -- start start 2024-09-12T02:10:22.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.643+0000 7fb8d50a1740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aec6f52380 0x55aec6f52760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:22.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.643+0000 7fb8d50a1740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55aec6f52ca0 0x55aec6f61550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:22.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.643+0000 7fb8c9ffb640 1 --2- 172.21.15.16:0/4131727408 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fb8bc0a2b80 0x7fb8bc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:22.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.643+0000 7fb8d50a1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55aec6f61a90 0x55aec6f63e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:22.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.643+0000 7fb8d50a1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55aec6eb82e0 con 0x55aec6f61a90 2024-09-12T02:10:22.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.643+0000 7fb8d50a1740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55aec6e5fce0 con 0x55aec6f52380 2024-09-12T02:10:22.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.643+0000 7fb8d50a1740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55aec6eb42a0 con 0x55aec6f52ca0 2024-09-12T02:10:22.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.643+0000 7fb8d1265640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55aec6f61a90 0x55aec6f63e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:22.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.643+0000 7fb8cbfff640 1 --2- 172.21.15.16:0/4131727408 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fb8bc0a2b80 0x7fb8bc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:22.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.643+0000 7fb8d1265640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55aec6f61a90 0x55aec6f63e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60742/0 (socket says 172.21.15.16:60742) 2024-09-12T02:10:22.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.643+0000 7fb8d1265640 1 -- 172.21.15.16:0/2789036625 learned_addr learned my addr 172.21.15.16:0/2789036625 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:10:22.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.643+0000 7fb8d0a64640 1 --2- 172.21.15.16:0/2789036625 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55aec6f52ca0 0x55aec6f61550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:22.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.643+0000 7fb8c9ffb640 1 -- 172.21.15.16:0/4131727408 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fb8b80ce4c0 con 0x7fb8cc102570 2024-09-12T02:10:22.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.643+0000 7fb8d1265640 1 --2- 172.21.15.16:0/2789036625 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aec6f52380 0x55aec6f52760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:22.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.643+0000 7fb8d0a64640 1 -- 172.21.15.16:0/2789036625 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aec6f52380 msgr2=0x55aec6f52760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:22.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.643+0000 7fb8d0a64640 1 --2- 172.21.15.16:0/2789036625 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aec6f52380 0x55aec6f52760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:22.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.643+0000 7fb8d0a64640 1 -- 172.21.15.16:0/2789036625 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55aec6f61a90 msgr2=0x55aec6f63e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:22.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.643+0000 7fb8d0a64640 1 --2- 172.21.15.16:0/2789036625 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55aec6f61a90 0x55aec6f63e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:22.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.643+0000 7fb8d0a64640 1 -- 172.21.15.16:0/2789036625 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aec6e44440 con 0x55aec6f52ca0 2024-09-12T02:10:22.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.643+0000 7fb8d1265640 1 --2- 172.21.15.16:0/2789036625 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aec6f52380 0x55aec6f52760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:10:22.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.643+0000 7fb8cbfff640 1 --2- 172.21.15.16:0/4131727408 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fb8bc0a2b80 0x7fb8bc0a5040 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fb8b8097180 tx=0x7fb8b804a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:10:22.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.644+0000 7fb8d0a64640 1 --2- 172.21.15.16:0/2789036625 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55aec6f52ca0 0x55aec6f61550 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb8c000a940 tx=0x7fb8c000ae00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=644c8e6e2b0524ec server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:10:22.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.644+0000 7fb8b6ffd640 1 -- 172.21.15.16:0/2789036625 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8c0015070 con 0x55aec6f52ca0 2024-09-12T02:10:22.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.644+0000 7fb8b6ffd640 1 -- 172.21.15.16:0/2789036625 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8c000cc00 con 0x55aec6f52ca0 2024-09-12T02:10:22.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.644+0000 7fb8b6ffd640 1 -- 172.21.15.16:0/2789036625 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8c00185c0 con 0x55aec6f52ca0 2024-09-12T02:10:22.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.644+0000 7fb8d50a1740 1 -- 172.21.15.16:0/2789036625 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55aec6f52ca0 msgr2=0x55aec6f61550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:22.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.644+0000 7fb8d50a1740 1 --2- 172.21.15.16:0/2789036625 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55aec6f52ca0 0x55aec6f61550 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb8c000a940 tx=0x7fb8c000ae00 comp rx=0 tx=0).stop 2024-09-12T02:10:22.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.644+0000 7fb8d50a1740 1 -- 172.21.15.16:0/2789036625 shutdown_connections 2024-09-12T02:10:22.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.644+0000 7fb8d50a1740 1 --2- 172.21.15.16:0/2789036625 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55aec6f61a90 0x55aec6f63e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:22.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.644+0000 7fb8d50a1740 1 --2- 172.21.15.16:0/2789036625 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55aec6f52ca0 0x55aec6f61550 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:22.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.644+0000 7fb8d50a1740 1 --2- 172.21.15.16:0/2789036625 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aec6f52380 0x55aec6f52760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:22.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.644+0000 7fb8d50a1740 1 -- 172.21.15.16:0/2789036625 >> 172.21.15.16:0/2789036625 conn(0x55aec6f4ea30 msgr2=0x55aec6f4ee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:10:22.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.645+0000 7fb8d50a1740 1 -- 172.21.15.16:0/2789036625 shutdown_connections 2024-09-12T02:10:22.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.645+0000 7fb8d50a1740 1 -- 172.21.15.16:0/2789036625 wait complete. 2024-09-12T02:10:22.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.645+0000 7fb8d50a1740 1 -- start start 2024-09-12T02:10:22.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.645+0000 7fb8d50a1740 1 Processor -- start 2024-09-12T02:10:22.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.645+0000 7fb8d50a1740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55aec6f52380 0x55aec6f61310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:22.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.645+0000 7fb8d50a1740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aec6f52ca0 0x55aec6f5e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:22.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.645+0000 7fb8d50a1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55aec6f61a90 0x55aec6f5eb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:22.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.645+0000 7fb8d50a1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55aec6f64660 con 0x55aec6f61a90 2024-09-12T02:10:22.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.645+0000 7fb8d50a1740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55aec6e73050 con 0x55aec6f52ca0 2024-09-12T02:10:22.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.645+0000 7fb8d50a1740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55aec6f647e0 con 0x55aec6f52380 2024-09-12T02:10:22.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.645+0000 7fb8d50a1740 10 client.0 _ll_register_callbacks cb 0x55aec6eb3160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:10:22.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.646+0000 7fb8d1265640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55aec6f61a90 0x55aec6f5eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:22.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.646+0000 7fb8d1265640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55aec6f61a90 0x55aec6f5eb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60754/0 (socket says 172.21.15.16:60754) 2024-09-12T02:10:22.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.646+0000 7fb8d1265640 1 -- 172.21.15.16:0/3760462094 learned_addr learned my addr 172.21.15.16:0/3760462094 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:10:22.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.646+0000 7fb8d1265640 1 --2- 172.21.15.16:0/3760462094 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55aec6f52380 0x55aec6f61310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:22.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.646+0000 7fb8d0a64640 1 --2- 172.21.15.16:0/3760462094 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aec6f52ca0 0x55aec6f5e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:22.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.646+0000 7fb8d1265640 1 -- 172.21.15.16:0/3760462094 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55aec6f52380 msgr2=0x55aec6f61310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:22.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.646+0000 7fb8d1265640 1 --2- 172.21.15.16:0/3760462094 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55aec6f52380 0x55aec6f61310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:22.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.646+0000 7fb8d1265640 1 -- 172.21.15.16:0/3760462094 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aec6f52ca0 msgr2=0x55aec6f5e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:22.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.646+0000 7fb8d1265640 1 --2- 172.21.15.16:0/3760462094 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aec6f52ca0 0x55aec6f5e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:22.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.646+0000 7fb8d1265640 1 -- 172.21.15.16:0/3760462094 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aec6c293c0 con 0x55aec6f61a90 2024-09-12T02:10:22.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.646+0000 7fb8d1265640 1 --2- 172.21.15.16:0/3760462094 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55aec6f61a90 0x55aec6f5eb50 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7fb8c4006010 tx=0x7fb8c4009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f99385ea222f876d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:10:22.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.646+0000 7fb8abfff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:10:22.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.646+0000 7fb8abfff640 1 -- 172.21.15.16:0/3760462094 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8c4034040 con 0x55aec6f61a90 2024-09-12T02:10:22.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.646+0000 7fb8abfff640 1 -- 172.21.15.16:0/3760462094 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fb8c4003b80 con 0x55aec6f61a90 2024-09-12T02:10:22.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.646+0000 7fb8d50a1740 1 -- 172.21.15.16:0/3760462094 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8c000cc00 con 0x55aec6f61a90 2024-09-12T02:10:22.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.646+0000 7fb8abfff640 1 -- 172.21.15.16:0/3760462094 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fb8c40307b0 con 0x55aec6f61a90 2024-09-12T02:10:22.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.646+0000 7fb8d50a1740 10 client.18081 fetch_fsmap learned FSMap version 25 2024-09-12T02:10:22.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.646+0000 7fb8d50a1740 1 -- 172.21.15.16:0/3760462094 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55aec6f70020 con 0x55aec6f61a90 2024-09-12T02:10:22.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.647+0000 7fb8abfff640 1 -- 172.21.15.16:0/3760462094 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8c000cc00 con 0x55aec6f61a90 2024-09-12T02:10:22.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.647+0000 7fb8abfff640 1 -- 172.21.15.16:0/3760462094 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fb8c4039a20 con 0x55aec6f61a90 2024-09-12T02:10:22.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.647+0000 7fb8d50a1740 10 client.18081 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:10:22.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.647+0000 7fb8d50a1740 10 client.18081 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:10:22.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.647+0000 7fb8d50a1740 20 client.18081 populate_metadata read hostname 'smithi016' 2024-09-12T02:10:22.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.647+0000 7fb8d50a1740 1 --2- 172.21.15.16:0/3760462094 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55aec6f71430 0x55aec6f73860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:22.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.647+0000 7fb8d50a1740 4 client.18081 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:10:22.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.647+0000 7fb8d50a1740 1 -- 172.21.15.16:0/3760462094 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aec6ec5850 con 0x55aec6f71430 2024-09-12T02:10:22.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.647+0000 7fb8d0a64640 1 --2- 172.21.15.16:0/3760462094 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55aec6f71430 0x55aec6f73860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:22.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.648+0000 7fb8d0a64640 1 --2- 172.21.15.16:0/3760462094 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55aec6f71430 0x55aec6f73860 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b43311f03942c2 server_cookie=cc4b845b32923559 in_seq=0 out_seq=0 2024-09-12T02:10:22.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.648+0000 7fb8abfff640 10 client.18081 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:10:22.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.648+0000 7fb8abfff640 1 -- 172.21.15.16:0/3760462094 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55aec6ec5850 con 0x55aec6f71430 2024-09-12T02:10:22.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.648+0000 7fb8abfff640 10 client.18081 handle_command_reply: tid=0 2024-09-12T02:10:22.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.648+0000 7fb8d50a1740 1 client.18081 shutdown 2024-09-12T02:10:22.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.648+0000 7fb8d50a1740 1 -- 172.21.15.16:0/3760462094 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55aec6f61a90 msgr2=0x55aec6f5eb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:22.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.648+0000 7fb8d50a1740 1 --2- 172.21.15.16:0/3760462094 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55aec6f61a90 0x55aec6f5eb50 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7fb8c4006010 tx=0x7fb8c4009800 comp rx=0 tx=0).stop 2024-09-12T02:10:22.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.648+0000 7fb8d50a1740 1 -- 172.21.15.16:0/3760462094 shutdown_connections 2024-09-12T02:10:22.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.648+0000 7fb8d50a1740 1 --2- 172.21.15.16:0/3760462094 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55aec6f71430 0x55aec6f73860 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:22.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.648+0000 7fb8d50a1740 1 --2- 172.21.15.16:0/3760462094 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55aec6f61a90 0x55aec6f5eb50 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:22.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.649+0000 7fb8d50a1740 1 --2- 172.21.15.16:0/3760462094 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aec6f52ca0 0x55aec6f5e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:22.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.649+0000 7fb8abfff640 0 client.18081 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:10:22.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.649+0000 7fb8d50a1740 1 --2- 172.21.15.16:0/3760462094 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55aec6f52380 0x55aec6f61310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:22.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.649+0000 7fb8d50a1740 1 -- 172.21.15.16:0/3760462094 >> 172.21.15.16:0/3760462094 conn(0x55aec6f4ea30 msgr2=0x55aec6f54790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:10:22.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.649+0000 7fb8d50a1740 1 -- 172.21.15.16:0/3760462094 shutdown_connections 2024-09-12T02:10:22.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.649+0000 7fb8d50a1740 1 -- 172.21.15.16:0/3760462094 wait complete. 2024-09-12T02:10:22.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.649+0000 7fb8d50a1740 20 client.18081 trim_cache size 0 max 16384 2024-09-12T02:10:22.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.666+0000 7fb8d50a1740 1 Processor -- start 2024-09-12T02:10:22.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.666+0000 7fb8d50a1740 1 -- start start 2024-09-12T02:10:22.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.666+0000 7fb8d50a1740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aec6f52380 0x55aec6f6d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:22.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.666+0000 7fb8d50a1740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55aec6f52ca0 0x55aec6f59d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:22.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.666+0000 7fb8d50a1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55aec6f61a90 0x55aec6f5d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:22.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.666+0000 7fb8d50a1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb8cc1adbd0 con 0x55aec6f61a90 2024-09-12T02:10:22.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.666+0000 7fb8d50a1740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb8cc1ab4a0 con 0x55aec6f52380 2024-09-12T02:10:22.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.666+0000 7fb8d50a1740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb8cc1a9310 con 0x55aec6f52ca0 2024-09-12T02:10:22.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.666+0000 7fb8d1265640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55aec6f61a90 0x55aec6f5d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:22.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.666+0000 7fb8d1265640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55aec6f61a90 0x55aec6f5d5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60766/0 (socket says 172.21.15.16:60766) 2024-09-12T02:10:22.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.666+0000 7fb8d1265640 1 -- 172.21.15.16:0/3399522678 learned_addr learned my addr 172.21.15.16:0/3399522678 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:10:22.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.667+0000 7fb8d1265640 1 --2- 172.21.15.16:0/3399522678 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aec6f52380 0x55aec6f6d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:22.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.667+0000 7fb8d0a64640 1 --2- 172.21.15.16:0/3399522678 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55aec6f52ca0 0x55aec6f59d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:22.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.667+0000 7fb8d1265640 1 -- 172.21.15.16:0/3399522678 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55aec6f52ca0 msgr2=0x55aec6f59d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:22.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.667+0000 7fb8d1265640 1 --2- 172.21.15.16:0/3399522678 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55aec6f52ca0 0x55aec6f59d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:22.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.667+0000 7fb8d1265640 1 -- 172.21.15.16:0/3399522678 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aec6f52380 msgr2=0x55aec6f6d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:22.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.667+0000 7fb8d1265640 1 --2- 172.21.15.16:0/3399522678 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aec6f52380 0x55aec6f6d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:22.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.667+0000 7fb8d1265640 1 -- 172.21.15.16:0/3399522678 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aec6f4e4d0 con 0x55aec6f61a90 2024-09-12T02:10:22.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.667+0000 7fb8d0a64640 1 --2- 172.21.15.16:0/3399522678 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55aec6f52ca0 0x55aec6f59d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:10:22.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.667+0000 7fb8d1265640 1 --2- 172.21.15.16:0/3399522678 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55aec6f61a90 0x55aec6f5d5f0 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7fb8c4002230 tx=0x7fb8c4005c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9748ea280b1d204 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:10:22.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.667+0000 7fb8b67fc640 1 -- 172.21.15.16:0/3399522678 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8c4034070 con 0x55aec6f61a90 2024-09-12T02:10:22.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.667+0000 7fb8b67fc640 1 -- 172.21.15.16:0/3399522678 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8c405b040 con 0x55aec6f61a90 2024-09-12T02:10:22.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.667+0000 7fb8b67fc640 1 -- 172.21.15.16:0/3399522678 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8c405a040 con 0x55aec6f61a90 2024-09-12T02:10:22.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.668+0000 7fb8d50a1740 1 -- 172.21.15.16:0/3399522678 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55aec6f61a90 msgr2=0x55aec6f5d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:22.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.668+0000 7fb8d50a1740 1 --2- 172.21.15.16:0/3399522678 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55aec6f61a90 0x55aec6f5d5f0 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7fb8c4002230 tx=0x7fb8c4005c30 comp rx=0 tx=0).stop 2024-09-12T02:10:22.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.668+0000 7fb8d50a1740 1 -- 172.21.15.16:0/3399522678 shutdown_connections 2024-09-12T02:10:22.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.668+0000 7fb8d50a1740 1 --2- 172.21.15.16:0/3399522678 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55aec6f61a90 0x55aec6f5d5f0 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:22.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.668+0000 7fb8d50a1740 1 --2- 172.21.15.16:0/3399522678 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55aec6f52ca0 0x55aec6f59d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:22.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.668+0000 7fb8d50a1740 1 --2- 172.21.15.16:0/3399522678 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aec6f52380 0x55aec6f6d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:22.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.668+0000 7fb8d50a1740 1 -- 172.21.15.16:0/3399522678 >> 172.21.15.16:0/3399522678 conn(0x55aec6f4ea30 msgr2=0x55aec6f5cee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:10:22.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.668+0000 7fb8d50a1740 1 -- 172.21.15.16:0/3399522678 shutdown_connections 2024-09-12T02:10:22.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.668+0000 7fb8d50a1740 1 -- 172.21.15.16:0/3399522678 wait complete. 2024-09-12T02:10:22.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.668+0000 7fb8d50a1740 1 -- start start 2024-09-12T02:10:22.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.668+0000 7fb8d50a1740 1 Processor -- start 2024-09-12T02:10:22.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.669+0000 7fb8d50a1740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aec6f52380 0x55aec6f69f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:22.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.669+0000 7fb8d50a1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55aec6f52ca0 0x55aec6f65230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:22.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.669+0000 7fb8d50a1740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55aec6f61a90 0x55aec6f65770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:22.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.669+0000 7fb8d50a1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb8cc1a9960 con 0x55aec6f52ca0 2024-09-12T02:10:22.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.669+0000 7fb8d50a1740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb8cc1aadd0 con 0x55aec6f52380 2024-09-12T02:10:22.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.669+0000 7fb8d50a1740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb8cc1aa030 con 0x55aec6f61a90 2024-09-12T02:10:22.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.669+0000 7fb8d50a1740 10 client.0 _ll_register_callbacks cb 0x7fb8c4030440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:10:22.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.669+0000 7fb8d0a64640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55aec6f52ca0 0x55aec6f65230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:22.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.669+0000 7fb8d0a64640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55aec6f52ca0 0x55aec6f65230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60774/0 (socket says 172.21.15.16:60774) 2024-09-12T02:10:22.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.669+0000 7fb8d0a64640 1 -- 172.21.15.16:0/279763308 learned_addr learned my addr 172.21.15.16:0/279763308 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:10:22.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.669+0000 7fb8d1265640 1 --2- 172.21.15.16:0/279763308 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aec6f52380 0x55aec6f69f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:22.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.669+0000 7fb8d1265640 1 --2- 172.21.15.16:0/279763308 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55aec6f61a90 0x55aec6f65770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:22.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.669+0000 7fb8d0a64640 1 -- 172.21.15.16:0/279763308 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55aec6f61a90 msgr2=0x55aec6f65770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:22.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.669+0000 7fb8d0a64640 1 --2- 172.21.15.16:0/279763308 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55aec6f61a90 0x55aec6f65770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:22.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.669+0000 7fb8d0a64640 1 -- 172.21.15.16:0/279763308 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aec6f52380 msgr2=0x55aec6f69f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:22.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.669+0000 7fb8d0a64640 1 --2- 172.21.15.16:0/279763308 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aec6f52380 0x55aec6f69f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:22.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.669+0000 7fb8d0a64640 1 -- 172.21.15.16:0/279763308 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aec6f4f6c0 con 0x55aec6f52ca0 2024-09-12T02:10:22.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.669+0000 7fb8d1265640 1 --2- 172.21.15.16:0/279763308 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55aec6f61a90 0x55aec6f65770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:10:22.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.669+0000 7fb8d0a64640 1 --2- 172.21.15.16:0/279763308 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55aec6f52ca0 0x55aec6f65230 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7fb8c0000c00 tx=0x7fb8c000a1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2647c35fc9f5461 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:10:22.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.669+0000 7fb8abfff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:10:22.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.670+0000 7fb8abfff640 1 -- 172.21.15.16:0/279763308 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8c0015040 con 0x55aec6f52ca0 2024-09-12T02:10:22.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.670+0000 7fb8abfff640 1 -- 172.21.15.16:0/279763308 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fb8c000ca00 con 0x55aec6f52ca0 2024-09-12T02:10:22.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.670+0000 7fb8d50a1740 1 -- 172.21.15.16:0/279763308 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8c405b040 con 0x55aec6f52ca0 2024-09-12T02:10:22.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.670+0000 7fb8abfff640 1 -- 172.21.15.16:0/279763308 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fb8c0037050 con 0x55aec6f52ca0 2024-09-12T02:10:22.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.670+0000 7fb8d50a1740 10 client.18087 fetch_fsmap learned FSMap version 25 2024-09-12T02:10:22.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.670+0000 7fb8d50a1740 1 -- 172.21.15.16:0/279763308 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55aec6fdbc50 con 0x55aec6f52ca0 2024-09-12T02:10:22.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.670+0000 7fb8abfff640 1 -- 172.21.15.16:0/279763308 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8c405b040 con 0x55aec6f52ca0 2024-09-12T02:10:22.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.670+0000 7fb8abfff640 1 -- 172.21.15.16:0/279763308 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fb8c002e070 con 0x55aec6f52ca0 2024-09-12T02:10:22.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.670+0000 7fb8d50a1740 10 client.18087 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:10:22.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.670+0000 7fb8d50a1740 10 client.18087 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:10:22.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.670+0000 7fb8d50a1740 20 client.18087 populate_metadata read hostname 'smithi016' 2024-09-12T02:10:22.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.670+0000 7fb8d50a1740 1 --2- 172.21.15.16:0/279763308 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55aec6f71430 0x55aec6fde100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:22.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.670+0000 7fb8d50a1740 4 client.18087 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T02:10:22.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.670+0000 7fb8d50a1740 1 -- 172.21.15.16:0/279763308 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55aec6f5eee0 con 0x55aec6f71430 2024-09-12T02:10:22.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.671+0000 7fb8d1265640 1 --2- 172.21.15.16:0/279763308 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55aec6f71430 0x55aec6fde100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:22.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.671+0000 7fb8d1265640 1 --2- 172.21.15.16:0/279763308 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55aec6f71430 0x55aec6fde100 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f1e5906e656af5f server_cookie=e3f18078f890c9ed in_seq=0 out_seq=0 2024-09-12T02:10:22.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.671+0000 7fb8abfff640 10 client.18087 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:10:22.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.671+0000 7fb8abfff640 1 -- 172.21.15.16:0/279763308 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55aec6f5eee0 con 0x55aec6f71430 2024-09-12T02:10:22.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.671+0000 7fb8abfff640 10 client.18087 handle_command_reply: tid=0 2024-09-12T02:10:22.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.671+0000 7fb8d50a1740 1 client.18087 shutdown 2024-09-12T02:10:22.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.671+0000 7fb8d50a1740 1 -- 172.21.15.16:0/279763308 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55aec6f52ca0 msgr2=0x55aec6f65230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:22.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.671+0000 7fb8d50a1740 1 --2- 172.21.15.16:0/279763308 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55aec6f52ca0 0x55aec6f65230 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7fb8c0000c00 tx=0x7fb8c000a1f0 comp rx=0 tx=0).stop 2024-09-12T02:10:22.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.671+0000 7fb8d50a1740 1 -- 172.21.15.16:0/279763308 shutdown_connections 2024-09-12T02:10:22.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.671+0000 7fb8d50a1740 1 --2- 172.21.15.16:0/279763308 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55aec6f71430 0x55aec6fde100 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:22.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.671+0000 7fb8d50a1740 1 --2- 172.21.15.16:0/279763308 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55aec6f61a90 0x55aec6f65770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:22.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.671+0000 7fb8d50a1740 1 --2- 172.21.15.16:0/279763308 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55aec6f52ca0 0x55aec6f65230 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:22.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.671+0000 7fb8abfff640 0 client.18087 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:10:22.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.671+0000 7fb8d50a1740 1 --2- 172.21.15.16:0/279763308 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aec6f52380 0x55aec6f69f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:22.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.671+0000 7fb8d50a1740 1 -- 172.21.15.16:0/279763308 >> 172.21.15.16:0/279763308 conn(0x55aec6f4ea30 msgr2=0x55aec6f5d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:10:22.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.672+0000 7fb8d50a1740 1 -- 172.21.15.16:0/279763308 shutdown_connections 2024-09-12T02:10:22.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.672+0000 7fb8d50a1740 1 -- 172.21.15.16:0/279763308 wait complete. 2024-09-12T02:10:22.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.672+0000 7fb8d50a1740 20 client.18087 trim_cache size 0 max 16384 2024-09-12T02:10:22.680 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T02:10:22.680 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-12T02:10:22.680 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": { 2024-09-12T02:10:22.680 INFO:teuthology.orchestra.run.smithi016.stdout: "ca0a9334-6276-41ec-a5ff-01a6150d2a6c": { 2024-09-12T02:10:22.680 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-12T02:10:22.680 INFO:teuthology.orchestra.run.smithi016.stdout: "tag": "ca0a9334-6276-41ec-a5ff-01a6150d2a6c", 2024-09-12T02:10:22.680 INFO:teuthology.orchestra.run.smithi016.stdout: "options": "recursive,force" 2024-09-12T02:10:22.681 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:10:22.681 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:10:22.681 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T02:10:22.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.672+0000 7fb8d2cc8640 1 -- 172.21.15.16:0/4131727408 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fb8bc0a2b80 msgr2=0x7fb8bc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:22.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.672+0000 7fb8d2cc8640 1 --2- 172.21.15.16:0/4131727408 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fb8bc0a2b80 0x7fb8bc0a5040 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fb8b8097180 tx=0x7fb8b804a040 comp rx=0 tx=0).stop 2024-09-12T02:10:22.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.672+0000 7fb8d2cc8640 1 -- 172.21.15.16:0/4131727408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8cc102570 msgr2=0x7fb8cc198b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:22.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.672+0000 7fb8d2cc8640 1 --2- 172.21.15.16:0/4131727408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8cc102570 0x7fb8cc198b20 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7fb8b800b4a0 tx=0x7fb8b800b960 comp rx=0 tx=0).stop 2024-09-12T02:10:22.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.672+0000 7fb8d2cc8640 1 -- 172.21.15.16:0/4131727408 shutdown_connections 2024-09-12T02:10:22.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.672+0000 7fb8d2cc8640 1 --2- 172.21.15.16:0/4131727408 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fb8bc0a2b80 0x7fb8bc0a5040 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:22.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.672+0000 7fb8d2cc8640 1 --2- 172.21.15.16:0/4131727408 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb8cc102ef0 0x7fb8cc19ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:22.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.672+0000 7fb8d2cc8640 1 --2- 172.21.15.16:0/4131727408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8cc102570 0x7fb8cc198b20 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:22.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.672+0000 7fb8d2cc8640 1 --2- 172.21.15.16:0/4131727408 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb8cc100e20 0x7fb8cc1985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:22.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.672+0000 7fb8d2cc8640 1 -- 172.21.15.16:0/4131727408 >> 172.21.15.16:0/4131727408 conn(0x7fb8cc0fc7d0 msgr2=0x7fb8cc06a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:10:22.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.673+0000 7fb8d2cc8640 1 -- 172.21.15.16:0/4131727408 shutdown_connections 2024-09-12T02:10:22.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:22.673+0000 7fb8d2cc8640 1 -- 172.21.15.16:0/4131727408 wait complete. 2024-09-12T02:10:23.389 INFO:tasks.cephfs.filesystem:scrub status for tag:ca0a9334-6276-41ec-a5ff-01a6150d2a6c - {'path': '/', 'tag': 'ca0a9334-6276-41ec-a5ff-01a6150d2a6c', 'options': 'recursive,force'} 2024-09-12T02:10:24.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:10:23 smithi184 ceph-mon[35593]: pgmap v589: 97 pgs: 97 active+clean; 4.3 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-09-12T02:10:24.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:10:23 smithi033 ceph-mon[35332]: pgmap v589: 97 pgs: 97 active+clean; 4.3 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-09-12T02:10:24.183 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:10:23 smithi016 ceph-mon[30947]: pgmap v589: 97 pgs: 97 active+clean; 4.3 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-09-12T02:10:26.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:10:25 smithi184 ceph-mon[35593]: pgmap v590: 97 pgs: 97 active+clean; 4.4 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-09-12T02:10:26.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:10:25 smithi033 ceph-mon[35332]: pgmap v590: 97 pgs: 97 active+clean; 4.4 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-09-12T02:10:26.183 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:10:25 smithi016 ceph-mon[30947]: pgmap v590: 97 pgs: 97 active+clean; 4.4 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-09-12T02:10:27.085 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:10:26 smithi016 ceph-mon[30947]: scrub summary: idle 2024-09-12T02:10:27.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:10:26 smithi184 ceph-mon[35593]: scrub summary: idle 2024-09-12T02:10:27.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:10:26 smithi033 ceph-mon[35332]: scrub summary: idle 2024-09-12T02:10:28.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:10:27 smithi184 ceph-mon[35593]: pgmap v591: 97 pgs: 97 active+clean; 4.4 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-09-12T02:10:28.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:10:27 smithi033 ceph-mon[35332]: pgmap v591: 97 pgs: 97 active+clean; 4.4 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-09-12T02:10:28.183 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:10:27 smithi016 ceph-mon[30947]: pgmap v591: 97 pgs: 97 active+clean; 4.4 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-09-12T02:10:30.242 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.241+0000 7feff90a6640 1 -- 172.21.15.16:0/3590472847 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feff4100d90 msgr2=0x7feff41011f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:30.242 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.241+0000 7feff90a6640 1 --2- 172.21.15.16:0/3590472847 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feff4100d90 0x7feff41011f0 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7fefe8009900 tx=0x7fefe802a480 comp rx=0 tx=0).stop 2024-09-12T02:10:30.243 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.242+0000 7feff90a6640 1 -- 172.21.15.16:0/3590472847 shutdown_connections 2024-09-12T02:10:30.243 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.242+0000 7feff90a6640 1 --2- 172.21.15.16:0/3590472847 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feff4101730 0x7feff4101be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:30.243 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.242+0000 7feff90a6640 1 --2- 172.21.15.16:0/3590472847 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feff4100d90 0x7feff41011f0 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:30.243 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.242+0000 7feff90a6640 1 --2- 172.21.15.16:0/3590472847 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feff4100450 0x7feff4100850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:30.243 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.242+0000 7feff90a6640 1 -- 172.21.15.16:0/3590472847 >> 172.21.15.16:0/3590472847 conn(0x7feff40fc1e0 msgr2=0x7feff40fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:10:30.243 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.242+0000 7feff90a6640 1 -- 172.21.15.16:0/3590472847 shutdown_connections 2024-09-12T02:10:30.243 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.242+0000 7feff90a6640 1 -- 172.21.15.16:0/3590472847 wait complete. 2024-09-12T02:10:30.243 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.242+0000 7feff90a6640 1 Processor -- start 2024-09-12T02:10:30.245 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.243+0000 7feff90a6640 1 -- start start 2024-09-12T02:10:30.245 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.243+0000 7feff90a6640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feff4100450 0x7feff4199fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:30.245 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.243+0000 7feff90a6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feff4100d90 0x7feff419a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:30.245 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.243+0000 7feff90a6640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feff4101730 0x7feff41a1590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:30.246 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.243+0000 7feff90a6640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7feff410f9c0 con 0x7feff4100d90 2024-09-12T02:10:30.246 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.243+0000 7feff90a6640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7feff410f840 con 0x7feff4101730 2024-09-12T02:10:30.246 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.243+0000 7feff90a6640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7feff410fb40 con 0x7feff4100450 2024-09-12T02:10:30.246 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.243+0000 7feff259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feff4100d90 0x7feff419a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:30.246 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.243+0000 7feff259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feff4100d90 0x7feff419a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60788/0 (socket says 172.21.15.16:60788) 2024-09-12T02:10:30.246 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.243+0000 7feff259c640 1 -- 172.21.15.16:0/2792304456 learned_addr learned my addr 172.21.15.16:0/2792304456 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:10:30.246 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.243+0000 7feff2d9d640 1 --2- 172.21.15.16:0/2792304456 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feff4100450 0x7feff4199fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:30.246 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.243+0000 7feff359e640 1 --2- 172.21.15.16:0/2792304456 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feff4101730 0x7feff41a1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:30.246 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.243+0000 7feff259c640 1 -- 172.21.15.16:0/2792304456 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feff4100450 msgr2=0x7feff4199fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:30.246 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.243+0000 7feff259c640 1 --2- 172.21.15.16:0/2792304456 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feff4100450 0x7feff4199fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:30.246 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.243+0000 7feff259c640 1 -- 172.21.15.16:0/2792304456 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feff4101730 msgr2=0x7feff41a1590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:30.247 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.243+0000 7feff259c640 1 --2- 172.21.15.16:0/2792304456 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feff4101730 0x7feff41a1590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:30.247 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.243+0000 7feff259c640 1 -- 172.21.15.16:0/2792304456 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feff41a1c00 con 0x7feff4100d90 2024-09-12T02:10:30.247 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.244+0000 7feff259c640 1 --2- 172.21.15.16:0/2792304456 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feff4100d90 0x7feff419a520 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7fefe8009230 tx=0x7fefe8004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:10:30.247 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.244+0000 7fefdbfff640 1 -- 172.21.15.16:0/2792304456 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefe803f030 con 0x7feff4100d90 2024-09-12T02:10:30.247 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.244+0000 7fefdbfff640 1 -- 172.21.15.16:0/2792304456 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fefe8003ce0 con 0x7feff4100d90 2024-09-12T02:10:30.247 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.244+0000 7feff90a6640 1 -- 172.21.15.16:0/2792304456 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feff41a1e90 con 0x7feff4100d90 2024-09-12T02:10:30.247 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.244+0000 7fefdbfff640 1 -- 172.21.15.16:0/2792304456 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefe8035070 con 0x7feff4100d90 2024-09-12T02:10:30.247 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.244+0000 7feff90a6640 1 -- 172.21.15.16:0/2792304456 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feff41a2300 con 0x7feff4100d90 2024-09-12T02:10:30.254 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.247+0000 7feffbe39740 1 Processor -- start 2024-09-12T02:10:30.254 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.247+0000 7fefdbfff640 1 -- 172.21.15.16:0/2792304456 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fefe8042030 con 0x7feff4100d90 2024-09-12T02:10:30.254 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.247+0000 7feffbe39740 1 -- start start 2024-09-12T02:10:30.254 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.247+0000 7feffbe39740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5591749e29a0 0x5591749e2da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:30.254 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.247+0000 7feffbe39740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5591749e32e0 0x5591749e7da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:30.254 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.247+0000 7feffbe39740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5591749e8370 0x5591749ea830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:30.255 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.247+0000 7feffbe39740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5591749462f0 con 0x5591749e8370 2024-09-12T02:10:30.255 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.247+0000 7feffbe39740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5591748ee270 con 0x5591749e32e0 2024-09-12T02:10:30.255 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.247+0000 7feffbe39740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5591749422b0 con 0x5591749e29a0 2024-09-12T02:10:30.255 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.247+0000 7feff259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5591749e8370 0x5591749ea830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:30.255 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.247+0000 7feff259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5591749e8370 0x5591749ea830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60792/0 (socket says 172.21.15.16:60792) 2024-09-12T02:10:30.255 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.247+0000 7feff259c640 1 -- 172.21.15.16:0/573657748 learned_addr learned my addr 172.21.15.16:0/573657748 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:10:30.255 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.247+0000 7feff2d9d640 1 --2- 172.21.15.16:0/573657748 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5591749e32e0 0x5591749e7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:30.255 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.247+0000 7feff359e640 1 --2- 172.21.15.16:0/573657748 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5591749e29a0 0x5591749e2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:30.255 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.248+0000 7feff259c640 1 -- 172.21.15.16:0/573657748 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5591749e29a0 msgr2=0x5591749e2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:30.255 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.248+0000 7feff259c640 1 --2- 172.21.15.16:0/573657748 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5591749e29a0 0x5591749e2da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:30.255 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.248+0000 7feff259c640 1 -- 172.21.15.16:0/573657748 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5591749e32e0 msgr2=0x5591749e7da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:30.255 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.248+0000 7feff259c640 1 --2- 172.21.15.16:0/573657748 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5591749e32e0 0x5591749e7da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:30.256 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.248+0000 7fefdbfff640 1 --2- 172.21.15.16:0/2792304456 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fefdc0a2cb0 0x7fefdc0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:30.256 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.248+0000 7feff259c640 1 -- 172.21.15.16:0/573657748 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591748ed340 con 0x5591749e8370 2024-09-12T02:10:30.256 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.248+0000 7feff2d9d640 1 --2- 172.21.15.16:0/573657748 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5591749e32e0 0x5591749e7da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:10:30.256 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.248+0000 7feff359e640 1 --2- 172.21.15.16:0/2792304456 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fefdc0a2cb0 0x7fefdc0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:30.256 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.248+0000 7feff259c640 1 --2- 172.21.15.16:0/573657748 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5591749e8370 0x5591749ea830 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7fefe80f5bf0 tx=0x7fefe80b8490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fd74ea7417ed202 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:10:30.256 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.248+0000 7fefdbfff640 1 -- 172.21.15.16:0/2792304456 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fefe80ef840 con 0x7feff4100d90 2024-09-12T02:10:30.256 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.248+0000 7fefd97fa640 1 -- 172.21.15.16:0/573657748 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefe80bbfc0 con 0x5591749e8370 2024-09-12T02:10:30.256 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.248+0000 7fefd97fa640 1 -- 172.21.15.16:0/573657748 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fefe80c3870 con 0x5591749e8370 2024-09-12T02:10:30.256 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.248+0000 7fefd97fa640 1 -- 172.21.15.16:0/573657748 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefe80c4920 con 0x5591749e8370 2024-09-12T02:10:30.256 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.249+0000 7feffbe39740 1 -- 172.21.15.16:0/573657748 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5591749e8370 msgr2=0x5591749ea830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:30.256 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.249+0000 7feff359e640 1 --2- 172.21.15.16:0/2792304456 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fefdc0a2cb0 0x7fefdc0a5170 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fefec005fd0 tx=0x7fefec009600 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:10:30.257 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.249+0000 7feffbe39740 1 --2- 172.21.15.16:0/573657748 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5591749e8370 0x5591749ea830 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7fefe80f5bf0 tx=0x7fefe80b8490 comp rx=0 tx=0).stop 2024-09-12T02:10:30.257 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.249+0000 7feffbe39740 1 -- 172.21.15.16:0/573657748 shutdown_connections 2024-09-12T02:10:30.257 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.249+0000 7feffbe39740 1 --2- 172.21.15.16:0/573657748 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5591749e8370 0x5591749ea830 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:30.257 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.249+0000 7feffbe39740 1 --2- 172.21.15.16:0/573657748 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5591749e32e0 0x5591749e7da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:30.257 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.249+0000 7feffbe39740 1 --2- 172.21.15.16:0/573657748 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5591749e29a0 0x5591749e2da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:30.257 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.249+0000 7feffbe39740 1 -- 172.21.15.16:0/573657748 >> 172.21.15.16:0/573657748 conn(0x5591749de010 msgr2=0x5591749de410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:10:30.257 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.249+0000 7feffbe39740 1 -- 172.21.15.16:0/573657748 shutdown_connections 2024-09-12T02:10:30.257 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.249+0000 7feffbe39740 1 -- 172.21.15.16:0/573657748 wait complete. 2024-09-12T02:10:30.257 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.249+0000 7feffbe39740 1 -- start start 2024-09-12T02:10:30.257 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.249+0000 7feffbe39740 1 Processor -- start 2024-09-12T02:10:30.257 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.250+0000 7feffbe39740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5591749e29a0 0x5591749ec940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:30.257 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.250+0000 7feffbe39740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5591749e32e0 0x5591749ece80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:30.257 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.250+0000 7feffbe39740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5591749e8370 0x5591749f3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:30.258 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.250+0000 7feffbe39740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5591749eb100 con 0x5591749e29a0 2024-09-12T02:10:30.258 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.250+0000 7feffbe39740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5591748cf290 con 0x5591749e32e0 2024-09-12T02:10:30.258 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.250+0000 7feffbe39740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5591749eb280 con 0x5591749e8370 2024-09-12T02:10:30.258 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.250+0000 7feffbe39740 10 client.0 _ll_register_callbacks cb 0x559174900140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:10:30.258 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.250+0000 7feff2d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5591749e29a0 0x5591749ec940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:30.258 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.250+0000 7feff2d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5591749e29a0 0x5591749ec940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60806/0 (socket says 172.21.15.16:60806) 2024-09-12T02:10:30.258 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.250+0000 7feff2d9d640 1 -- 172.21.15.16:0/2498178512 learned_addr learned my addr 172.21.15.16:0/2498178512 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:10:30.258 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.250+0000 7feff359e640 1 --2- 172.21.15.16:0/2498178512 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5591749e8370 0x5591749f3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:30.258 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.250+0000 7feff259c640 1 --2- 172.21.15.16:0/2498178512 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5591749e32e0 0x5591749ece80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:30.258 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.250+0000 7feff2d9d640 1 -- 172.21.15.16:0/2498178512 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5591749e8370 msgr2=0x5591749f3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:30.258 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.250+0000 7feff2d9d640 1 --2- 172.21.15.16:0/2498178512 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5591749e8370 0x5591749f3ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:30.259 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.250+0000 7feff2d9d640 1 -- 172.21.15.16:0/2498178512 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5591749e32e0 msgr2=0x5591749ece80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:30.259 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.250+0000 7feff2d9d640 1 --2- 172.21.15.16:0/2498178512 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5591749e32e0 0x5591749ece80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:30.259 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.250+0000 7feff2d9d640 1 -- 172.21.15.16:0/2498178512 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559174659210 con 0x5591749e29a0 2024-09-12T02:10:30.259 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.250+0000 7feff259c640 1 --2- 172.21.15.16:0/2498178512 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5591749e32e0 0x5591749ece80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:10:30.259 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.250+0000 7feff2d9d640 1 --2- 172.21.15.16:0/2498178512 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5591749e29a0 0x5591749ec940 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7fefe400bad0 tx=0x7fefe400bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b979b74ae603d30 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:10:30.259 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.250+0000 7fefd6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:10:30.259 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.251+0000 7fefd6ff5640 1 -- 172.21.15.16:0/2498178512 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefe4003d60 con 0x5591749e29a0 2024-09-12T02:10:30.259 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.251+0000 7fefd6ff5640 1 -- 172.21.15.16:0/2498178512 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fefe4011440 con 0x5591749e29a0 2024-09-12T02:10:30.259 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.251+0000 7feffbe39740 1 -- 172.21.15.16:0/2498178512 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x559174953a20 con 0x5591749e29a0 2024-09-12T02:10:30.259 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.251+0000 7fefd6ff5640 1 -- 172.21.15.16:0/2498178512 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fefe4011640 con 0x5591749e29a0 2024-09-12T02:10:30.259 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.251+0000 7feffbe39740 10 client.18099 fetch_fsmap learned FSMap version 25 2024-09-12T02:10:30.259 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.251+0000 7feffbe39740 1 -- 172.21.15.16:0/2498178512 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x559174725620 con 0x5591749e29a0 2024-09-12T02:10:30.259 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.251+0000 7fefd6ff5640 1 -- 172.21.15.16:0/2498178512 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x559174953a20 con 0x5591749e29a0 2024-09-12T02:10:30.259 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.251+0000 7fefd6ff5640 1 -- 172.21.15.16:0/2498178512 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fefe401b910 con 0x5591749e29a0 2024-09-12T02:10:30.260 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.251+0000 7feffbe39740 10 client.18099 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:10:30.260 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.251+0000 7feffbe39740 10 client.18099 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:10:30.260 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.251+0000 7feffbe39740 20 client.18099 populate_metadata read hostname 'smithi016' 2024-09-12T02:10:30.260 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.251+0000 7feffbe39740 1 --2- 172.21.15.16:0/2498178512 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5591749f85e0 0x5591749faa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:30.260 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.251+0000 7feffbe39740 4 client.18099 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:10:30.260 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.251+0000 7feffbe39740 1 -- 172.21.15.16:0/2498178512 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55917494dd70 con 0x5591749f85e0 2024-09-12T02:10:30.260 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.252+0000 7feff259c640 1 --2- 172.21.15.16:0/2498178512 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5591749f85e0 0x5591749faa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:30.260 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.252+0000 7fefd6ff5640 10 client.18099 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:10:30.260 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.252+0000 7feff259c640 1 --2- 172.21.15.16:0/2498178512 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5591749f85e0 0x5591749faa30 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7bef7041d9aa0073 server_cookie=8f25033a515c3cc9 in_seq=0 out_seq=0 2024-09-12T02:10:30.260 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.253+0000 7fefd6ff5640 1 -- 172.21.15.16:0/2498178512 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55917494dd70 con 0x5591749f85e0 2024-09-12T02:10:30.260 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.253+0000 7fefd6ff5640 10 client.18099 handle_command_reply: tid=0 2024-09-12T02:10:30.260 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.253+0000 7feffbe39740 1 client.18099 shutdown 2024-09-12T02:10:30.260 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.253+0000 7feffbe39740 1 -- 172.21.15.16:0/2498178512 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5591749e29a0 msgr2=0x5591749ec940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:30.261 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.253+0000 7feffbe39740 1 --2- 172.21.15.16:0/2498178512 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5591749e29a0 0x5591749ec940 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7fefe400bad0 tx=0x7fefe400bed0 comp rx=0 tx=0).stop 2024-09-12T02:10:30.261 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.253+0000 7feffbe39740 1 -- 172.21.15.16:0/2498178512 shutdown_connections 2024-09-12T02:10:30.261 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.253+0000 7feffbe39740 1 --2- 172.21.15.16:0/2498178512 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5591749f85e0 0x5591749faa30 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:30.261 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.253+0000 7feffbe39740 1 --2- 172.21.15.16:0/2498178512 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5591749e8370 0x5591749f3ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:30.261 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.253+0000 7fefd6ff5640 0 client.18099 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:10:30.261 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.253+0000 7feffbe39740 1 --2- 172.21.15.16:0/2498178512 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5591749e32e0 0x5591749ece80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:30.261 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.253+0000 7feffbe39740 1 --2- 172.21.15.16:0/2498178512 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5591749e29a0 0x5591749ec940 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:30.261 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.253+0000 7feffbe39740 1 -- 172.21.15.16:0/2498178512 >> 172.21.15.16:0/2498178512 conn(0x5591749de010 msgr2=0x5591749df820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:10:30.261 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.253+0000 7feffbe39740 1 -- 172.21.15.16:0/2498178512 shutdown_connections 2024-09-12T02:10:30.261 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.253+0000 7feffbe39740 1 -- 172.21.15.16:0/2498178512 wait complete. 2024-09-12T02:10:30.261 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.253+0000 7feffbe39740 20 client.18099 trim_cache size 0 max 16384 2024-09-12T02:10:30.273 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.268+0000 7feffbe39740 1 Processor -- start 2024-09-12T02:10:30.273 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.268+0000 7feffbe39740 1 -- start start 2024-09-12T02:10:30.273 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.268+0000 7feffbe39740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5591749e29a0 0x5591749ecd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:30.273 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.268+0000 7feffbe39740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5591749e32e0 0x5591749f3e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:30.273 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.268+0000 7feffbe39740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5591749e8370 0x5591749df400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:30.274 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.268+0000 7feffbe39740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7feff41ad8e0 con 0x5591749e32e0 2024-09-12T02:10:30.274 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.268+0000 7feffbe39740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7feff41ab1b0 con 0x5591749e29a0 2024-09-12T02:10:30.274 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.268+0000 7feffbe39740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7feff41a9020 con 0x5591749e8370 2024-09-12T02:10:30.274 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.268+0000 7feff259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5591749e32e0 0x5591749f3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:30.274 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.268+0000 7feff259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5591749e32e0 0x5591749f3e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60820/0 (socket says 172.21.15.16:60820) 2024-09-12T02:10:30.274 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.268+0000 7feff259c640 1 -- 172.21.15.16:0/367716700 learned_addr learned my addr 172.21.15.16:0/367716700 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:10:30.274 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.268+0000 7feff359e640 1 --2- 172.21.15.16:0/367716700 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5591749e8370 0x5591749df400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:30.274 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.268+0000 7feff2d9d640 1 --2- 172.21.15.16:0/367716700 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5591749e29a0 0x5591749ecd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:30.274 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.268+0000 7feff259c640 1 -- 172.21.15.16:0/367716700 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5591749e8370 msgr2=0x5591749df400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:30.274 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.268+0000 7feff259c640 1 --2- 172.21.15.16:0/367716700 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5591749e8370 0x5591749df400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:30.274 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.268+0000 7feff259c640 1 -- 172.21.15.16:0/367716700 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5591749e29a0 msgr2=0x5591749ecd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:30.275 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.268+0000 7feff259c640 1 --2- 172.21.15.16:0/367716700 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5591749e29a0 0x5591749ecd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:30.275 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.269+0000 7feff259c640 1 -- 172.21.15.16:0/367716700 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591749ec480 con 0x5591749e32e0 2024-09-12T02:10:30.275 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.269+0000 7feff359e640 1 --2- 172.21.15.16:0/367716700 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5591749e8370 0x5591749df400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:10:30.275 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.269+0000 7feff259c640 1 --2- 172.21.15.16:0/367716700 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5591749e32e0 0x5591749f3e80 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7fefe8003fb0 tx=0x7fefe80f5530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33680e209b4f6eec server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:10:30.275 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.269+0000 7fefd8ff9640 1 -- 172.21.15.16:0/367716700 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefe8049040 con 0x5591749e32e0 2024-09-12T02:10:30.275 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.269+0000 7fefd8ff9640 1 -- 172.21.15.16:0/367716700 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fefe80c4db0 con 0x5591749e32e0 2024-09-12T02:10:30.275 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.269+0000 7fefd8ff9640 1 -- 172.21.15.16:0/367716700 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefe80bbb50 con 0x5591749e32e0 2024-09-12T02:10:30.275 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.269+0000 7feffbe39740 1 -- 172.21.15.16:0/367716700 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5591749e32e0 msgr2=0x5591749f3e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:30.275 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.269+0000 7feffbe39740 1 --2- 172.21.15.16:0/367716700 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5591749e32e0 0x5591749f3e80 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7fefe8003fb0 tx=0x7fefe80f5530 comp rx=0 tx=0).stop 2024-09-12T02:10:30.275 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.269+0000 7feffbe39740 1 -- 172.21.15.16:0/367716700 shutdown_connections 2024-09-12T02:10:30.275 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.269+0000 7feffbe39740 1 --2- 172.21.15.16:0/367716700 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5591749e8370 0x5591749df400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:30.275 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.269+0000 7feffbe39740 1 --2- 172.21.15.16:0/367716700 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5591749e32e0 0x5591749f3e80 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:30.276 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.269+0000 7feffbe39740 1 --2- 172.21.15.16:0/367716700 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5591749e29a0 0x5591749ecd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:30.276 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.269+0000 7feffbe39740 1 -- 172.21.15.16:0/367716700 >> 172.21.15.16:0/367716700 conn(0x5591749de010 msgr2=0x5591749dd1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:10:30.276 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.269+0000 7feffbe39740 1 -- 172.21.15.16:0/367716700 shutdown_connections 2024-09-12T02:10:30.276 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.270+0000 7feffbe39740 1 -- 172.21.15.16:0/367716700 wait complete. 2024-09-12T02:10:30.276 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.270+0000 7feffbe39740 1 -- start start 2024-09-12T02:10:30.276 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.270+0000 7feffbe39740 1 Processor -- start 2024-09-12T02:10:30.276 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.270+0000 7feffbe39740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5591749e29a0 0x5591749f17a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:30.276 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.270+0000 7feffbe39740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5591749e32e0 0x5591749f1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:30.276 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.270+0000 7feffbe39740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5591749e8370 0x5591749f2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:30.276 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.270+0000 7feffbe39740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7feff41a9670 con 0x5591749e8370 2024-09-12T02:10:30.276 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.270+0000 7feffbe39740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7feff41aaae0 con 0x5591749e29a0 2024-09-12T02:10:30.277 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.270+0000 7feffbe39740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7feff41a9d40 con 0x5591749e32e0 2024-09-12T02:10:30.277 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.270+0000 7feffbe39740 10 client.0 _ll_register_callbacks cb 0x7fefe4004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:10:30.277 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.270+0000 7feff359e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5591749e8370 0x5591749f2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:30.277 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.270+0000 7feff359e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5591749e8370 0x5591749f2c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60830/0 (socket says 172.21.15.16:60830) 2024-09-12T02:10:30.277 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.270+0000 7feff2d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5591749e29a0 0x5591749f17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:30.277 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.270+0000 7feff359e640 1 -- 172.21.15.16:0/1037976183 learned_addr learned my addr 172.21.15.16:0/1037976183 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:10:30.277 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.270+0000 7feff259c640 1 --2- 172.21.15.16:0/1037976183 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5591749e32e0 0x5591749f1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:30.277 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.271+0000 7feff359e640 1 -- 172.21.15.16:0/1037976183 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5591749e32e0 msgr2=0x5591749f1ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:30.277 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.271+0000 7feff359e640 1 --2- 172.21.15.16:0/1037976183 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5591749e32e0 0x5591749f1ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:30.277 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.271+0000 7feff359e640 1 -- 172.21.15.16:0/1037976183 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5591749e29a0 msgr2=0x5591749f17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:30.277 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.271+0000 7feff359e640 1 --2- 172.21.15.16:0/1037976183 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5591749e29a0 0x5591749f17a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:30.277 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.271+0000 7feff359e640 1 -- 172.21.15.16:0/1037976183 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5591749f3220 con 0x5591749e8370 2024-09-12T02:10:30.278 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.271+0000 7feff2d9d640 1 --2- 172.21.15.16:0/1037976183 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5591749e29a0 0x5591749f17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:10:30.278 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.271+0000 7feff359e640 1 --2- 172.21.15.16:0/1037976183 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5591749e8370 0x5591749f2c10 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7fefec011e90 tx=0x7fefec004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4f9349575f50e40 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:10:30.278 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.271+0000 7fefd6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:10:30.278 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.271+0000 7fefd6ff5640 1 -- 172.21.15.16:0/1037976183 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefec02f060 con 0x5591749e8370 2024-09-12T02:10:30.278 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.271+0000 7fefd6ff5640 1 -- 172.21.15.16:0/1037976183 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fefec020db0 con 0x5591749e8370 2024-09-12T02:10:30.278 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.271+0000 7feffbe39740 1 -- 172.21.15.16:0/1037976183 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fefe80c4db0 con 0x5591749e8370 2024-09-12T02:10:30.278 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.271+0000 7fefd6ff5640 1 -- 172.21.15.16:0/1037976183 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fefec003970 con 0x5591749e8370 2024-09-12T02:10:30.278 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.271+0000 7feffbe39740 10 client.18105 fetch_fsmap learned FSMap version 25 2024-09-12T02:10:30.278 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.271+0000 7feffbe39740 1 -- 172.21.15.16:0/1037976183 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x559174a63560 con 0x5591749e8370 2024-09-12T02:10:30.278 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.271+0000 7fefd6ff5640 1 -- 172.21.15.16:0/1037976183 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fefe80c4db0 con 0x5591749e8370 2024-09-12T02:10:30.278 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.272+0000 7fefd6ff5640 1 -- 172.21.15.16:0/1037976183 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fefec034070 con 0x5591749e8370 2024-09-12T02:10:30.278 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.272+0000 7feffbe39740 10 client.18105 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:10:30.278 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.272+0000 7feffbe39740 10 client.18105 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:10:30.278 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.272+0000 7feffbe39740 20 client.18105 populate_metadata read hostname 'smithi016' 2024-09-12T02:10:30.279 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.272+0000 7feffbe39740 1 --2- 172.21.15.16:0/1037976183 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x559174a63f40 0x559174a66360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:30.279 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.272+0000 7feffbe39740 4 client.18105 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:10:30.279 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.272+0000 7feffbe39740 1 -- 172.21.15.16:0/1037976183 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5591748ca900 con 0x559174a63f40 2024-09-12T02:10:30.279 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.272+0000 7feff2d9d640 1 --2- 172.21.15.16:0/1037976183 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x559174a63f40 0x559174a66360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:30.279 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.272+0000 7feff2d9d640 1 --2- 172.21.15.16:0/1037976183 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x559174a63f40 0x559174a66360 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb96bd23dc5fec9c server_cookie=72089230d94b1261 in_seq=0 out_seq=0 2024-09-12T02:10:30.279 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.272+0000 7fefd6ff5640 10 client.18105 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:10:30.288 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.287+0000 7fefd6ff5640 1 -- 172.21.15.16:0/1037976183 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5591748ca900 con 0x559174a63f40 2024-09-12T02:10:30.288 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.287+0000 7fefd6ff5640 10 client.18105 handle_command_reply: tid=0 2024-09-12T02:10:30.288 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.287+0000 7feffbe39740 1 client.18105 shutdown 2024-09-12T02:10:30.288 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.287+0000 7feffbe39740 1 -- 172.21.15.16:0/1037976183 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5591749e8370 msgr2=0x5591749f2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:30.288 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.287+0000 7feffbe39740 1 --2- 172.21.15.16:0/1037976183 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5591749e8370 0x5591749f2c10 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7fefec011e90 tx=0x7fefec004a40 comp rx=0 tx=0).stop 2024-09-12T02:10:30.288 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.287+0000 7feffbe39740 1 -- 172.21.15.16:0/1037976183 shutdown_connections 2024-09-12T02:10:30.288 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.287+0000 7feffbe39740 1 --2- 172.21.15.16:0/1037976183 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x559174a63f40 0x559174a66360 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:30.288 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.287+0000 7feffbe39740 1 --2- 172.21.15.16:0/1037976183 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5591749e8370 0x5591749f2c10 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:30.288 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.287+0000 7feffbe39740 1 --2- 172.21.15.16:0/1037976183 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5591749e32e0 0x5591749f1ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:30.289 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.287+0000 7feffbe39740 1 --2- 172.21.15.16:0/1037976183 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5591749e29a0 0x5591749f17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:30.289 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.287+0000 7fefd6ff5640 0 client.18105 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:10:30.289 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.287+0000 7feffbe39740 1 -- 172.21.15.16:0/1037976183 >> 172.21.15.16:0/1037976183 conn(0x5591749de010 msgr2=0x5591749dd1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:10:30.289 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.287+0000 7feffbe39740 1 -- 172.21.15.16:0/1037976183 shutdown_connections 2024-09-12T02:10:30.289 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.287+0000 7feffbe39740 1 -- 172.21.15.16:0/1037976183 wait complete. 2024-09-12T02:10:30.289 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.288+0000 7feffbe39740 20 client.18105 trim_cache size 0 max 16384 2024-09-12T02:10:30.289 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-09-12T02:10:30.289 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-09-12T02:10:30.289 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-09-12T02:10:30.289 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-09-12T02:10:30.290 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.288+0000 7feff90a6640 1 -- 172.21.15.16:0/2792304456 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fefdc0a2cb0 msgr2=0x7fefdc0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:30.290 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.288+0000 7feff90a6640 1 --2- 172.21.15.16:0/2792304456 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fefdc0a2cb0 0x7fefdc0a5170 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fefec005fd0 tx=0x7fefec009600 comp rx=0 tx=0).stop 2024-09-12T02:10:30.290 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.288+0000 7feff90a6640 1 -- 172.21.15.16:0/2792304456 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feff4100d90 msgr2=0x7feff419a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:30.290 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.288+0000 7feff90a6640 1 --2- 172.21.15.16:0/2792304456 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feff4100d90 0x7feff419a520 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7fefe8009230 tx=0x7fefe8004360 comp rx=0 tx=0).stop 2024-09-12T02:10:30.291 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.288+0000 7feff90a6640 1 -- 172.21.15.16:0/2792304456 shutdown_connections 2024-09-12T02:10:30.291 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.288+0000 7feff90a6640 1 --2- 172.21.15.16:0/2792304456 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fefdc0a2cb0 0x7fefdc0a5170 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:30.291 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.288+0000 7feff90a6640 1 --2- 172.21.15.16:0/2792304456 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feff4101730 0x7feff41a1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:30.291 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.288+0000 7feff90a6640 1 --2- 172.21.15.16:0/2792304456 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feff4100d90 0x7feff419a520 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:30.291 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.288+0000 7feff90a6640 1 --2- 172.21.15.16:0/2792304456 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feff4100450 0x7feff4199fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:30.291 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.288+0000 7feff90a6640 1 -- 172.21.15.16:0/2792304456 >> 172.21.15.16:0/2792304456 conn(0x7feff40fc1e0 msgr2=0x7feff40fdc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:10:30.291 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.288+0000 7feff90a6640 1 -- 172.21.15.16:0/2792304456 shutdown_connections 2024-09-12T02:10:30.291 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:30.288+0000 7feff90a6640 1 -- 172.21.15.16:0/2792304456 wait complete. 2024-09-12T02:10:30.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:10:29 smithi184 ceph-mon[35593]: pgmap v592: 97 pgs: 97 active+clean; 4.4 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-09-12T02:10:30.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:10:29 smithi033 ceph-mon[35332]: pgmap v592: 97 pgs: 97 active+clean; 4.4 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-09-12T02:10:30.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:10:29 smithi016 ceph-mon[30947]: pgmap v592: 97 pgs: 97 active+clean; 4.4 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-09-12T02:10:32.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:10:31 smithi184 ceph-mon[35593]: pgmap v593: 97 pgs: 97 active+clean; 4.4 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 17 op/s 2024-09-12T02:10:32.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:10:31 smithi033 ceph-mon[35332]: pgmap v593: 97 pgs: 97 active+clean; 4.4 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 17 op/s 2024-09-12T02:10:32.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:10:31 smithi016 ceph-mon[30947]: pgmap v593: 97 pgs: 97 active+clean; 4.4 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 17 op/s 2024-09-12T02:10:34.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:10:33 smithi184 ceph-mon[35593]: pgmap v594: 97 pgs: 97 active+clean; 4.4 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-09-12T02:10:34.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:10:33 smithi033 ceph-mon[35332]: pgmap v594: 97 pgs: 97 active+clean; 4.4 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-09-12T02:10:34.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:10:33 smithi016 ceph-mon[30947]: pgmap v594: 97 pgs: 97 active+clean; 4.4 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-09-12T02:10:36.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:10:35 smithi184 ceph-mon[35593]: pgmap v595: 97 pgs: 97 active+clean; 4.5 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-09-12T02:10:36.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:10:35 smithi033 ceph-mon[35332]: pgmap v595: 97 pgs: 97 active+clean; 4.5 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-09-12T02:10:36.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:10:35 smithi016 ceph-mon[30947]: pgmap v595: 97 pgs: 97 active+clean; 4.5 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-09-12T02:10:38.183 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:10:37 smithi016 ceph-mon[30947]: pgmap v596: 97 pgs: 97 active+clean; 4.5 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-09-12T02:10:38.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:10:37 smithi184 ceph-mon[35593]: pgmap v596: 97 pgs: 97 active+clean; 4.5 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-09-12T02:10:38.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:10:37 smithi033 ceph-mon[35332]: pgmap v596: 97 pgs: 97 active+clean; 4.5 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-09-12T02:10:40.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:10:39 smithi184 ceph-mon[35593]: pgmap v597: 97 pgs: 97 active+clean; 4.5 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-09-12T02:10:40.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:10:39 smithi033 ceph-mon[35332]: pgmap v597: 97 pgs: 97 active+clean; 4.5 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-09-12T02:10:40.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:10:39 smithi016 ceph-mon[30947]: pgmap v597: 97 pgs: 97 active+clean; 4.5 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-09-12T02:10:42.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:10:41 smithi184 ceph-mon[35593]: pgmap v598: 97 pgs: 97 active+clean; 4.5 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-09-12T02:10:42.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:10:41 smithi033 ceph-mon[35332]: pgmap v598: 97 pgs: 97 active+clean; 4.5 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-09-12T02:10:42.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:10:41 smithi016 ceph-mon[30947]: pgmap v598: 97 pgs: 97 active+clean; 4.5 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-09-12T02:10:43.389 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.389+0000 7f88b795f640 1 -- 172.21.15.16:0/2183440777 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88b0101d90 msgr2=0x7f88b0108620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:43.389 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.389+0000 7f88b795f640 1 --2- 172.21.15.16:0/2183440777 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88b0101d90 0x7f88b0108620 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f88a800adb0 tx=0x7f88a802b7a0 comp rx=0 tx=0).stop 2024-09-12T02:10:43.389 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.389+0000 7f88b795f640 1 -- 172.21.15.16:0/2183440777 shutdown_connections 2024-09-12T02:10:43.389 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.389+0000 7f88b795f640 1 --2- 172.21.15.16:0/2183440777 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88b0101d90 0x7f88b0108620 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:43.390 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.389+0000 7f88b795f640 1 --2- 172.21.15.16:0/2183440777 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f88b01013d0 0x7f88b0101850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:43.390 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.389+0000 7f88b795f640 1 --2- 172.21.15.16:0/2183440777 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f88b01001e0 0x7f88b01005e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:43.390 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.389+0000 7f88b795f640 1 -- 172.21.15.16:0/2183440777 >> 172.21.15.16:0/2183440777 conn(0x7f88b00fb980 msgr2=0x7f88b00fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:10:43.390 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.389+0000 7f88b795f640 1 -- 172.21.15.16:0/2183440777 shutdown_connections 2024-09-12T02:10:43.390 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.389+0000 7f88b795f640 1 -- 172.21.15.16:0/2183440777 wait complete. 2024-09-12T02:10:43.390 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.389+0000 7f88b795f640 1 Processor -- start 2024-09-12T02:10:43.390 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.390+0000 7f88b795f640 1 -- start start 2024-09-12T02:10:43.392 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.390+0000 7f88b795f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f88b01001e0 0x7f88b0195ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:43.392 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.390+0000 7f88b795f640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f88b01013d0 0x7f88b01960e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:43.392 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.390+0000 7f88b795f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88b0101d90 0x7f88b019d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:43.392 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.390+0000 7f88b795f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f88b010b5d0 con 0x7f88b0101d90 2024-09-12T02:10:43.392 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.390+0000 7f88b795f640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f88b010b450 con 0x7f88b01001e0 2024-09-12T02:10:43.392 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.390+0000 7f88b795f640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f88b010b750 con 0x7f88b01013d0 2024-09-12T02:10:43.392 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.390+0000 7f88b5efc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88b0101d90 0x7f88b019d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:43.392 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.390+0000 7f88b5efc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88b0101d90 0x7f88b019d150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60734/0 (socket says 172.21.15.16:60734) 2024-09-12T02:10:43.392 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.390+0000 7f88b5efc640 1 -- 172.21.15.16:0/1094403718 learned_addr learned my addr 172.21.15.16:0/1094403718 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:10:43.392 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.390+0000 7f88b4efa640 1 --2- 172.21.15.16:0/1094403718 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f88b01013d0 0x7f88b01960e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:43.393 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.390+0000 7f88b56fb640 1 --2- 172.21.15.16:0/1094403718 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f88b01001e0 0x7f88b0195ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:43.393 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.390+0000 7f88b5efc640 1 -- 172.21.15.16:0/1094403718 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f88b01013d0 msgr2=0x7f88b01960e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:43.393 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.390+0000 7f88b5efc640 1 --2- 172.21.15.16:0/1094403718 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f88b01013d0 0x7f88b01960e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:43.393 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.390+0000 7f88b5efc640 1 -- 172.21.15.16:0/1094403718 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f88b01001e0 msgr2=0x7f88b0195ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:43.393 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.390+0000 7f88b5efc640 1 --2- 172.21.15.16:0/1094403718 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f88b01001e0 0x7f88b0195ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:43.393 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.390+0000 7f88b5efc640 1 -- 172.21.15.16:0/1094403718 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88b019d7c0 con 0x7f88b0101d90 2024-09-12T02:10:43.393 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.390+0000 7f88b56fb640 1 --2- 172.21.15.16:0/1094403718 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f88b01001e0 0x7f88b0195ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:10:43.393 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.390+0000 7f88b5efc640 1 --2- 172.21.15.16:0/1094403718 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88b0101d90 0x7f88b019d150 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f88a8032b50 tx=0x7f88a8033f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:10:43.393 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.391+0000 7f88a67fc640 1 -- 172.21.15.16:0/1094403718 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88a8041070 con 0x7f88b0101d90 2024-09-12T02:10:43.393 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.391+0000 7f88a67fc640 1 -- 172.21.15.16:0/1094403718 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88a8032890 con 0x7f88b0101d90 2024-09-12T02:10:43.394 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.391+0000 7f88b795f640 1 -- 172.21.15.16:0/1094403718 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88b019da50 con 0x7f88b0101d90 2024-09-12T02:10:43.394 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.391+0000 7f88a67fc640 1 -- 172.21.15.16:0/1094403718 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88a80047a0 con 0x7f88b0101d90 2024-09-12T02:10:43.394 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.391+0000 7f88b795f640 1 -- 172.21.15.16:0/1094403718 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88b019df40 con 0x7f88b0101d90 2024-09-12T02:10:43.399 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.393+0000 7f88a67fc640 1 -- 172.21.15.16:0/1094403718 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f88a803a040 con 0x7f88b0101d90 2024-09-12T02:10:43.399 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.393+0000 7f88ba7d5740 1 Processor -- start 2024-09-12T02:10:43.399 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.394+0000 7f88ba7d5740 1 -- start start 2024-09-12T02:10:43.399 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.394+0000 7f88ba7d5740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564565f6abb0 0x564565f6afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:43.399 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.394+0000 7f88ba7d5740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564565f6b4f0 0x564565f6ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:43.399 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.394+0000 7f88ba7d5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564565f70580 0x564565f72a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:43.400 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.394+0000 7f88ba7d5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x564565ecc2f0 con 0x564565f70580 2024-09-12T02:10:43.400 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.394+0000 7f88ba7d5740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x564565e74270 con 0x564565f6b4f0 2024-09-12T02:10:43.400 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.394+0000 7f88ba7d5740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x564565ec82b0 con 0x564565f6abb0 2024-09-12T02:10:43.400 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.394+0000 7f88b4efa640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564565f70580 0x564565f72a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:43.400 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.394+0000 7f88b4efa640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564565f70580 0x564565f72a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60742/0 (socket says 172.21.15.16:60742) 2024-09-12T02:10:43.400 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.394+0000 7f88b4efa640 1 -- 172.21.15.16:0/1892455729 learned_addr learned my addr 172.21.15.16:0/1892455729 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:10:43.400 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.394+0000 7f88a67fc640 1 --2- 172.21.15.16:0/1094403718 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f889c0a2c30 0x7f889c0a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:43.400 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.394+0000 7f88b56fb640 1 --2- 172.21.15.16:0/1892455729 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564565f6b4f0 0x564565f6ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:43.400 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.394+0000 7f88b4efa640 1 --2- 172.21.15.16:0/1892455729 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564565f6abb0 0x564565f6afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:43.400 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.394+0000 7f88b5efc640 1 --2- 172.21.15.16:0/1094403718 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f889c0a2c30 0x7f889c0a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:43.400 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.394+0000 7f88b4efa640 1 -- 172.21.15.16:0/1892455729 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564565f6abb0 msgr2=0x564565f6afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:43.400 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.395+0000 7f88b4efa640 1 --2- 172.21.15.16:0/1892455729 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564565f6abb0 0x564565f6afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:43.401 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.395+0000 7f88b4efa640 1 -- 172.21.15.16:0/1892455729 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564565f6b4f0 msgr2=0x564565f6ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:43.401 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.395+0000 7f88b4efa640 1 --2- 172.21.15.16:0/1892455729 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564565f6b4f0 0x564565f6ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:43.401 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.395+0000 7f88a67fc640 1 -- 172.21.15.16:0/1094403718 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f88a80f14e0 con 0x7f88b0101d90 2024-09-12T02:10:43.401 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.395+0000 7f88b4efa640 1 -- 172.21.15.16:0/1892455729 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564565e73340 con 0x564565f70580 2024-09-12T02:10:43.401 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.395+0000 7f88b56fb640 1 --2- 172.21.15.16:0/1892455729 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564565f6b4f0 0x564565f6ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:10:43.401 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.395+0000 7f88b4efa640 1 --2- 172.21.15.16:0/1892455729 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564565f70580 0x564565f72a40 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f88a00058c0 tx=0x7f88a0005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d840d89d319aa457 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:10:43.401 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.395+0000 7f889b7fe640 1 -- 172.21.15.16:0/1892455729 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88a0004030 con 0x564565f70580 2024-09-12T02:10:43.401 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.395+0000 7f889b7fe640 1 -- 172.21.15.16:0/1892455729 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88a0005500 con 0x564565f70580 2024-09-12T02:10:43.401 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.395+0000 7f889b7fe640 1 -- 172.21.15.16:0/1892455729 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88a0004c10 con 0x564565f70580 2024-09-12T02:10:43.401 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.395+0000 7f88b5efc640 1 --2- 172.21.15.16:0/1094403718 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f889c0a2c30 0x7f889c0a50f0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f88a8045000 tx=0x7f88a80b3220 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:10:43.401 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.396+0000 7f88ba7d5740 1 -- 172.21.15.16:0/1892455729 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564565f70580 msgr2=0x564565f72a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:43.401 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.396+0000 7f88ba7d5740 1 --2- 172.21.15.16:0/1892455729 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564565f70580 0x564565f72a40 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f88a00058c0 tx=0x7f88a0005850 comp rx=0 tx=0).stop 2024-09-12T02:10:43.401 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.396+0000 7f88ba7d5740 1 -- 172.21.15.16:0/1892455729 shutdown_connections 2024-09-12T02:10:43.401 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.396+0000 7f88ba7d5740 1 --2- 172.21.15.16:0/1892455729 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564565f70580 0x564565f72a40 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:43.402 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.396+0000 7f88ba7d5740 1 --2- 172.21.15.16:0/1892455729 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564565f6b4f0 0x564565f6ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:43.402 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.396+0000 7f88ba7d5740 1 --2- 172.21.15.16:0/1892455729 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564565f6abb0 0x564565f6afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:43.402 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.396+0000 7f88ba7d5740 1 -- 172.21.15.16:0/1892455729 >> 172.21.15.16:0/1892455729 conn(0x564565f64010 msgr2=0x564565f64410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:10:43.402 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.396+0000 7f88ba7d5740 1 -- 172.21.15.16:0/1892455729 shutdown_connections 2024-09-12T02:10:43.402 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.396+0000 7f88ba7d5740 1 -- 172.21.15.16:0/1892455729 wait complete. 2024-09-12T02:10:43.402 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.396+0000 7f88ba7d5740 1 -- start start 2024-09-12T02:10:43.402 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.396+0000 7f88ba7d5740 1 Processor -- start 2024-09-12T02:10:43.402 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.397+0000 7f88ba7d5740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564565f6abb0 0x564565f74e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:43.402 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.397+0000 7f88ba7d5740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564565f6b4f0 0x564565f75370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:43.403 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.397+0000 7f88ba7d5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564565f70580 0x564565f7c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:43.403 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.397+0000 7f88ba7d5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x564565f73550 con 0x564565f70580 2024-09-12T02:10:43.403 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.397+0000 7f88ba7d5740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x564565e55290 con 0x564565f6abb0 2024-09-12T02:10:43.403 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.397+0000 7f88ba7d5740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x564565f736d0 con 0x564565f6b4f0 2024-09-12T02:10:43.403 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.397+0000 7f88ba7d5740 10 client.0 _ll_register_callbacks cb 0x564565e86140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:10:43.403 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.397+0000 7f88b4efa640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564565f70580 0x564565f7c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:43.403 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.397+0000 7f88b4efa640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564565f70580 0x564565f7c3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60756/0 (socket says 172.21.15.16:60756) 2024-09-12T02:10:43.403 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.397+0000 7f88b4efa640 1 -- 172.21.15.16:0/1254271157 learned_addr learned my addr 172.21.15.16:0/1254271157 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:10:43.403 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.397+0000 7f88b56fb640 1 --2- 172.21.15.16:0/1254271157 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564565f6b4f0 0x564565f75370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:43.403 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.397+0000 7f88b4efa640 1 --2- 172.21.15.16:0/1254271157 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564565f6abb0 0x564565f74e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:43.403 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.397+0000 7f88b4efa640 1 -- 172.21.15.16:0/1254271157 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564565f6b4f0 msgr2=0x564565f75370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:43.404 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.397+0000 7f88b4efa640 1 --2- 172.21.15.16:0/1254271157 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564565f6b4f0 0x564565f75370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:43.404 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.397+0000 7f88b4efa640 1 -- 172.21.15.16:0/1254271157 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564565f6abb0 msgr2=0x564565f74e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:43.404 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.397+0000 7f88b4efa640 1 --2- 172.21.15.16:0/1254271157 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564565f6abb0 0x564565f74e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:43.404 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.397+0000 7f88b4efa640 1 -- 172.21.15.16:0/1254271157 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564565bdf210 con 0x564565f70580 2024-09-12T02:10:43.404 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.397+0000 7f88b4efa640 1 --2- 172.21.15.16:0/1254271157 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564565f70580 0x564565f7c3b0 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f88a0009e70 tx=0x7f88a0009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=344c6400b542243c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:10:43.404 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.397+0000 7f8898ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:10:43.404 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.398+0000 7f8898ff9640 1 -- 172.21.15.16:0/1254271157 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88a0004990 con 0x564565f70580 2024-09-12T02:10:43.404 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.398+0000 7f8898ff9640 1 -- 172.21.15.16:0/1254271157 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f88a0019040 con 0x564565f70580 2024-09-12T02:10:43.404 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.398+0000 7f88ba7d5740 1 -- 172.21.15.16:0/1254271157 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f88a0005500 con 0x564565f70580 2024-09-12T02:10:43.404 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.398+0000 7f8898ff9640 1 -- 172.21.15.16:0/1254271157 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f88a0004b30 con 0x564565f70580 2024-09-12T02:10:43.404 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.398+0000 7f88ba7d5740 10 client.18117 fetch_fsmap learned FSMap version 25 2024-09-12T02:10:43.404 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.398+0000 7f88ba7d5740 1 -- 172.21.15.16:0/1254271157 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x564565cab620 con 0x564565f70580 2024-09-12T02:10:43.404 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.398+0000 7f8898ff9640 1 -- 172.21.15.16:0/1254271157 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88a0005500 con 0x564565f70580 2024-09-12T02:10:43.405 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.398+0000 7f8898ff9640 1 -- 172.21.15.16:0/1254271157 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f88a001f5d0 con 0x564565f70580 2024-09-12T02:10:43.405 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.398+0000 7f88ba7d5740 10 client.18117 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:10:43.405 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.398+0000 7f88ba7d5740 10 client.18117 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:10:43.405 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.398+0000 7f88ba7d5740 20 client.18117 populate_metadata read hostname 'smithi016' 2024-09-12T02:10:43.405 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.398+0000 7f88ba7d5740 1 --2- 172.21.15.16:0/1254271157 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x564565f80830 0x564565f82c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:43.405 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.398+0000 7f88ba7d5740 4 client.18117 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:10:43.405 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.398+0000 7f88ba7d5740 1 -- 172.21.15.16:0/1254271157 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564565ed9a20 con 0x564565f80830 2024-09-12T02:10:43.405 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.399+0000 7f88b56fb640 1 --2- 172.21.15.16:0/1254271157 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x564565f80830 0x564565f82c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:43.405 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.399+0000 7f8898ff9640 10 client.18117 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:10:43.405 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.399+0000 7f88b56fb640 1 --2- 172.21.15.16:0/1254271157 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x564565f80830 0x564565f82c20 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f890e6c27ffc0cd3 server_cookie=1479178be20dc91f in_seq=0 out_seq=0 2024-09-12T02:10:43.406 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.400+0000 7f8898ff9640 1 -- 172.21.15.16:0/1254271157 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564565ed9a20 con 0x564565f80830 2024-09-12T02:10:43.406 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.400+0000 7f8898ff9640 10 client.18117 handle_command_reply: tid=0 2024-09-12T02:10:43.406 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.400+0000 7f88ba7d5740 1 client.18117 shutdown 2024-09-12T02:10:43.407 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.400+0000 7f88ba7d5740 1 -- 172.21.15.16:0/1254271157 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564565f70580 msgr2=0x564565f7c3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:43.407 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.400+0000 7f88ba7d5740 1 --2- 172.21.15.16:0/1254271157 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564565f70580 0x564565f7c3b0 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f88a0009e70 tx=0x7f88a0009da0 comp rx=0 tx=0).stop 2024-09-12T02:10:43.407 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.400+0000 7f88ba7d5740 1 -- 172.21.15.16:0/1254271157 shutdown_connections 2024-09-12T02:10:43.407 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.400+0000 7f88ba7d5740 1 --2- 172.21.15.16:0/1254271157 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x564565f80830 0x564565f82c20 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:43.407 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.400+0000 7f88ba7d5740 1 --2- 172.21.15.16:0/1254271157 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564565f70580 0x564565f7c3b0 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:43.407 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.400+0000 7f88ba7d5740 1 --2- 172.21.15.16:0/1254271157 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564565f6b4f0 0x564565f75370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:43.407 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.400+0000 7f88ba7d5740 1 --2- 172.21.15.16:0/1254271157 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564565f6abb0 0x564565f74e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:43.407 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.400+0000 7f8898ff9640 0 client.18117 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:10:43.407 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.400+0000 7f88ba7d5740 1 -- 172.21.15.16:0/1254271157 >> 172.21.15.16:0/1254271157 conn(0x564565f64010 msgr2=0x564565f65820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:10:43.407 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.400+0000 7f88ba7d5740 1 -- 172.21.15.16:0/1254271157 shutdown_connections 2024-09-12T02:10:43.407 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.400+0000 7f88ba7d5740 1 -- 172.21.15.16:0/1254271157 wait complete. 2024-09-12T02:10:43.407 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.400+0000 7f88ba7d5740 20 client.18117 trim_cache size 0 max 16384 2024-09-12T02:10:43.419 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.415+0000 7f88ba7d5740 1 Processor -- start 2024-09-12T02:10:43.419 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.415+0000 7f88ba7d5740 1 -- start start 2024-09-12T02:10:43.419 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.415+0000 7f88ba7d5740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564565f64010 0x564565f630b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:43.419 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.415+0000 7f88ba7d5740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564565f6abb0 0x564565f87500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:43.419 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.415+0000 7f88ba7d5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564565f6b4f0 0x564565f87a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:43.419 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.415+0000 7f88ba7d5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f88b01a94f0 con 0x564565f6b4f0 2024-09-12T02:10:43.419 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.415+0000 7f88ba7d5740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f88b01a6dc0 con 0x564565f64010 2024-09-12T02:10:43.419 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.415+0000 7f88ba7d5740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f88b01a4c30 con 0x564565f6abb0 2024-09-12T02:10:43.419 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.415+0000 7f88b4efa640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564565f6b4f0 0x564565f87a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:43.420 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.415+0000 7f88b4efa640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564565f6b4f0 0x564565f87a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60768/0 (socket says 172.21.15.16:60768) 2024-09-12T02:10:43.420 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.415+0000 7f88b4efa640 1 -- 172.21.15.16:0/1990492931 learned_addr learned my addr 172.21.15.16:0/1990492931 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:10:43.420 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.415+0000 7f88b56fb640 1 --2- 172.21.15.16:0/1990492931 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564565f6abb0 0x564565f87500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:43.420 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.415+0000 7f88b4efa640 1 -- 172.21.15.16:0/1990492931 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564565f6abb0 msgr2=0x564565f87500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:43.420 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.415+0000 7f88b4efa640 1 --2- 172.21.15.16:0/1990492931 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564565f6abb0 0x564565f87500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:43.420 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.415+0000 7f88b4efa640 1 -- 172.21.15.16:0/1990492931 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564565f64010 msgr2=0x564565f630b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:43.420 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.415+0000 7f88b4efa640 1 --2- 172.21.15.16:0/1990492931 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564565f64010 0x564565f630b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:43.420 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.415+0000 7f88b4efa640 1 -- 172.21.15.16:0/1990492931 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564565f83280 con 0x564565f6b4f0 2024-09-12T02:10:43.420 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.416+0000 7f88b4efa640 1 --2- 172.21.15.16:0/1990492931 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564565f6b4f0 0x564565f87a40 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f88a001f9a0 tx=0x7f88a0011570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4575862737d4ddc server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:10:43.420 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.416+0000 7f889affd640 1 -- 172.21.15.16:0/1990492931 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88a0004880 con 0x564565f6b4f0 2024-09-12T02:10:43.421 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.416+0000 7f889affd640 1 -- 172.21.15.16:0/1990492931 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88a000fd90 con 0x564565f6b4f0 2024-09-12T02:10:43.421 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.416+0000 7f889affd640 1 -- 172.21.15.16:0/1990492931 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88a000f790 con 0x564565f6b4f0 2024-09-12T02:10:43.421 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.416+0000 7f88ba7d5740 1 -- 172.21.15.16:0/1990492931 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564565f6b4f0 msgr2=0x564565f87a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:43.421 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.416+0000 7f88ba7d5740 1 --2- 172.21.15.16:0/1990492931 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564565f6b4f0 0x564565f87a40 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f88a001f9a0 tx=0x7f88a0011570 comp rx=0 tx=0).stop 2024-09-12T02:10:43.421 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.416+0000 7f88ba7d5740 1 -- 172.21.15.16:0/1990492931 shutdown_connections 2024-09-12T02:10:43.421 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.416+0000 7f88ba7d5740 1 --2- 172.21.15.16:0/1990492931 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564565f6b4f0 0x564565f87a40 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:43.421 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.416+0000 7f88ba7d5740 1 --2- 172.21.15.16:0/1990492931 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564565f6abb0 0x564565f87500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:43.421 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.416+0000 7f88ba7d5740 1 --2- 172.21.15.16:0/1990492931 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564565f64010 0x564565f630b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:43.421 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.416+0000 7f88ba7d5740 1 -- 172.21.15.16:0/1990492931 >> 172.21.15.16:0/1990492931 conn(0x564565f700d0 msgr2=0x564565f629a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:10:43.421 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.416+0000 7f88ba7d5740 1 -- 172.21.15.16:0/1990492931 shutdown_connections 2024-09-12T02:10:43.421 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.416+0000 7f88ba7d5740 1 -- 172.21.15.16:0/1990492931 wait complete. 2024-09-12T02:10:43.421 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.417+0000 7f88ba7d5740 1 -- start start 2024-09-12T02:10:43.421 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.417+0000 7f88ba7d5740 1 Processor -- start 2024-09-12T02:10:43.422 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.417+0000 7f88ba7d5740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564565f64010 0x564565f6a420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:43.422 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.417+0000 7f88ba7d5740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564565f6abb0 0x564565fe94d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:43.422 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.417+0000 7f88ba7d5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564565f6b4f0 0x564565febaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:43.422 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.417+0000 7f88ba7d5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f88b01a5280 con 0x564565f6b4f0 2024-09-12T02:10:43.422 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.417+0000 7f88ba7d5740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f88b01a66f0 con 0x564565f64010 2024-09-12T02:10:43.422 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.417+0000 7f88ba7d5740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f88b01a5950 con 0x564565f6abb0 2024-09-12T02:10:43.422 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.417+0000 7f88ba7d5740 10 client.0 _ll_register_callbacks cb 0x7f88ac00b440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:10:43.422 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.417+0000 7f88b4efa640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564565f6b4f0 0x564565febaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:43.422 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.417+0000 7f88b4efa640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564565f6b4f0 0x564565febaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60770/0 (socket says 172.21.15.16:60770) 2024-09-12T02:10:43.422 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.417+0000 7f88b4efa640 1 -- 172.21.15.16:0/1882734425 learned_addr learned my addr 172.21.15.16:0/1882734425 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:10:43.422 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.417+0000 7f88b4efa640 1 --2- 172.21.15.16:0/1882734425 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564565f64010 0x564565f6a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:43.422 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.418+0000 7f88b56fb640 1 --2- 172.21.15.16:0/1882734425 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564565f6abb0 0x564565fe94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:43.422 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.418+0000 7f88b4efa640 1 -- 172.21.15.16:0/1882734425 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564565f6abb0 msgr2=0x564565fe94d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:43.422 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.418+0000 7f88b4efa640 1 --2- 172.21.15.16:0/1882734425 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564565f6abb0 0x564565fe94d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:43.423 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.418+0000 7f88b4efa640 1 -- 172.21.15.16:0/1882734425 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564565f64010 msgr2=0x564565f6a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:43.423 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.418+0000 7f88b4efa640 1 --2- 172.21.15.16:0/1882734425 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564565f64010 0x564565f6a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:43.423 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.418+0000 7f88b4efa640 1 -- 172.21.15.16:0/1882734425 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564565fec320 con 0x564565f6b4f0 2024-09-12T02:10:43.423 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.418+0000 7f88b4efa640 1 --2- 172.21.15.16:0/1882734425 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564565f6b4f0 0x564565febaf0 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f88a0005390 tx=0x7f88a0005970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a40b0a5b118044c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:10:43.423 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.418+0000 7f8898ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:10:43.423 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.418+0000 7f8898ff9640 1 -- 172.21.15.16:0/1882734425 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88a003e020 con 0x564565f6b4f0 2024-09-12T02:10:43.423 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.418+0000 7f8898ff9640 1 -- 172.21.15.16:0/1882734425 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f88a002a050 con 0x564565f6b4f0 2024-09-12T02:10:43.423 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.418+0000 7f88ba7d5740 1 -- 172.21.15.16:0/1882734425 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f88a000fd90 con 0x564565f6b4f0 2024-09-12T02:10:43.423 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.418+0000 7f8898ff9640 1 -- 172.21.15.16:0/1882734425 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f88a00048f0 con 0x564565f6b4f0 2024-09-12T02:10:43.423 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.418+0000 7f88ba7d5740 10 client.18123 fetch_fsmap learned FSMap version 25 2024-09-12T02:10:43.423 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.418+0000 7f88ba7d5740 1 -- 172.21.15.16:0/1882734425 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x564565fed630 con 0x564565f6b4f0 2024-09-12T02:10:43.423 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.418+0000 7f8898ff9640 1 -- 172.21.15.16:0/1882734425 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88a000fd90 con 0x564565f6b4f0 2024-09-12T02:10:43.423 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.419+0000 7f8898ff9640 1 -- 172.21.15.16:0/1882734425 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f88a000f9a0 con 0x564565f6b4f0 2024-09-12T02:10:43.424 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.419+0000 7f88ba7d5740 10 client.18123 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:10:43.424 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.419+0000 7f88ba7d5740 10 client.18123 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:10:43.424 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.419+0000 7f88ba7d5740 20 client.18123 populate_metadata read hostname 'smithi016' 2024-09-12T02:10:43.424 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.419+0000 7f88ba7d5740 1 --2- 172.21.15.16:0/1882734425 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x564565f70580 0x564565fefd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:43.424 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.419+0000 7f88ba7d5740 4 client.18123 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:10:43.424 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.419+0000 7f88ba7d5740 1 -- 172.21.15.16:0/1882734425 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564565ed3d70 con 0x564565f70580 2024-09-12T02:10:43.424 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.419+0000 7f88b56fb640 1 --2- 172.21.15.16:0/1882734425 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x564565f70580 0x564565fefd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:43.424 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.419+0000 7f88b56fb640 1 --2- 172.21.15.16:0/1882734425 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x564565f70580 0x564565fefd50 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94d79e5d5c6003bf server_cookie=cb5f0e806de953be in_seq=0 out_seq=0 2024-09-12T02:10:43.424 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.419+0000 7f8898ff9640 10 client.18123 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:10:43.437 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.435+0000 7f8898ff9640 1 -- 172.21.15.16:0/1882734425 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564565ed3d70 con 0x564565f70580 2024-09-12T02:10:43.437 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.435+0000 7f8898ff9640 10 client.18123 handle_command_reply: tid=0 2024-09-12T02:10:43.437 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.435+0000 7f88ba7d5740 1 client.18123 shutdown 2024-09-12T02:10:43.437 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.435+0000 7f88ba7d5740 1 -- 172.21.15.16:0/1882734425 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564565f6b4f0 msgr2=0x564565febaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:43.438 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.435+0000 7f88ba7d5740 1 --2- 172.21.15.16:0/1882734425 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564565f6b4f0 0x564565febaf0 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f88a0005390 tx=0x7f88a0005970 comp rx=0 tx=0).stop 2024-09-12T02:10:43.438 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.435+0000 7f88ba7d5740 1 -- 172.21.15.16:0/1882734425 shutdown_connections 2024-09-12T02:10:43.438 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.435+0000 7f88ba7d5740 1 --2- 172.21.15.16:0/1882734425 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x564565f70580 0x564565fefd50 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:43.438 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.435+0000 7f88ba7d5740 1 --2- 172.21.15.16:0/1882734425 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564565f6b4f0 0x564565febaf0 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:43.438 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.435+0000 7f88ba7d5740 1 --2- 172.21.15.16:0/1882734425 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564565f6abb0 0x564565fe94d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:43.438 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.435+0000 7f8898ff9640 0 client.18123 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:10:43.438 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.435+0000 7f88ba7d5740 1 --2- 172.21.15.16:0/1882734425 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564565f64010 0x564565f6a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:43.438 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.435+0000 7f88ba7d5740 1 -- 172.21.15.16:0/1882734425 >> 172.21.15.16:0/1882734425 conn(0x564565f700d0 msgr2=0x564565f631c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:10:43.439 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.436+0000 7f88ba7d5740 1 -- 172.21.15.16:0/1882734425 shutdown_connections 2024-09-12T02:10:43.439 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.436+0000 7f88ba7d5740 1 -- 172.21.15.16:0/1882734425 wait complete. 2024-09-12T02:10:43.439 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.436+0000 7f88ba7d5740 20 client.18123 trim_cache size 0 max 16384 2024-09-12T02:10:43.439 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-09-12T02:10:43.439 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-09-12T02:10:43.439 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-09-12T02:10:43.439 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-09-12T02:10:43.440 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.436+0000 7f88b795f640 1 -- 172.21.15.16:0/1094403718 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f889c0a2c30 msgr2=0x7f889c0a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:43.440 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.436+0000 7f88b795f640 1 --2- 172.21.15.16:0/1094403718 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f889c0a2c30 0x7f889c0a50f0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f88a8045000 tx=0x7f88a80b3220 comp rx=0 tx=0).stop 2024-09-12T02:10:43.440 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.436+0000 7f88b795f640 1 -- 172.21.15.16:0/1094403718 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88b0101d90 msgr2=0x7f88b019d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:43.440 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.436+0000 7f88b795f640 1 --2- 172.21.15.16:0/1094403718 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88b0101d90 0x7f88b019d150 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f88a8032b50 tx=0x7f88a8033f40 comp rx=0 tx=0).stop 2024-09-12T02:10:43.440 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.436+0000 7f88b795f640 1 -- 172.21.15.16:0/1094403718 shutdown_connections 2024-09-12T02:10:43.440 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.436+0000 7f88b795f640 1 --2- 172.21.15.16:0/1094403718 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f889c0a2c30 0x7f889c0a50f0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:43.440 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.436+0000 7f88b795f640 1 --2- 172.21.15.16:0/1094403718 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88b0101d90 0x7f88b019d150 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:43.441 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.436+0000 7f88b795f640 1 --2- 172.21.15.16:0/1094403718 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f88b01013d0 0x7f88b01960e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:43.441 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.436+0000 7f88b795f640 1 --2- 172.21.15.16:0/1094403718 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f88b01001e0 0x7f88b0195ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:43.441 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.436+0000 7f88b795f640 1 -- 172.21.15.16:0/1094403718 >> 172.21.15.16:0/1094403718 conn(0x7f88b00fb980 msgr2=0x7f88b01047a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:10:43.441 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.436+0000 7f88b795f640 1 -- 172.21.15.16:0/1094403718 shutdown_connections 2024-09-12T02:10:43.441 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:43.436+0000 7f88b795f640 1 -- 172.21.15.16:0/1094403718 wait complete. 2024-09-12T02:10:43.685 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:10:43.685 INFO:tasks.workunit.client.0.smithi016.stdout:251/600: open fd = 3 2024-09-12T02:10:43.685 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:10:43.685 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:10:43.685 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:10:43.685 INFO:tasks.workunit.client.0.smithi016.stdout:252/600: open fd = 3 2024-09-12T02:10:43.685 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:10:43.685 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:10:43.686 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:10:43.686 INFO:tasks.workunit.client.0.smithi016.stdout:253/600: open fd = 3 2024-09-12T02:10:43.686 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:10:43.686 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:10:43.686 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:10:43.686 INFO:tasks.workunit.client.0.smithi016.stdout:254/600: open fd = 3 2024-09-12T02:10:43.686 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:10:43.686 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:10:43.686 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:10:43.686 INFO:tasks.workunit.client.0.smithi016.stdout:255/600: open fd = 3 2024-09-12T02:10:43.686 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:10:43.686 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:10:43.686 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:10:43.686 INFO:tasks.workunit.client.0.smithi016.stdout:256/600: open fd = 3 2024-09-12T02:10:43.687 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:10:43.687 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:10:43.687 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:10:43.687 INFO:tasks.workunit.client.0.smithi016.stdout:257/600: open fd = 3 2024-09-12T02:10:43.687 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:10:43.687 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:10:43.687 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:10:43.687 INFO:tasks.workunit.client.0.smithi016.stdout:258/600: open fd = 3 2024-09-12T02:10:43.687 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:10:43.687 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:10:43.687 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:10:43.687 INFO:tasks.workunit.client.0.smithi016.stdout:259/600: open fd = 3 2024-09-12T02:10:43.687 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:10:43.688 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:10:43.688 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:10:43.688 INFO:tasks.workunit.client.0.smithi016.stdout:260/600: open fd = 3 2024-09-12T02:10:43.688 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:10:43.688 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:10:43.690 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:10:43.691 INFO:tasks.workunit.client.0.smithi016.stdout:261/600: open fd = 3 2024-09-12T02:10:43.691 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:10:43.691 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:10:43.691 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:10:43.691 INFO:tasks.workunit.client.0.smithi016.stdout:262/600: open fd = 3 2024-09-12T02:10:43.691 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:10:43.691 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:10:43.691 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:10:43.691 INFO:tasks.workunit.client.0.smithi016.stdout:263/600: open fd = 3 2024-09-12T02:10:43.691 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:10:43.691 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:10:43.691 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:10:43.691 INFO:tasks.workunit.client.0.smithi016.stdout:264/600: open fd = 3 2024-09-12T02:10:43.692 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:10:43.692 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:10:43.692 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:10:43.692 INFO:tasks.workunit.client.0.smithi016.stdout:265/600: open fd = 3 2024-09-12T02:10:43.692 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:10:43.692 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:10:43.692 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:10:43.692 INFO:tasks.workunit.client.0.smithi016.stdout:266/600: open fd = 3 2024-09-12T02:10:43.692 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:10:43.692 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:10:43.692 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:10:43.692 INFO:tasks.workunit.client.0.smithi016.stdout:267/600: open fd = 3 2024-09-12T02:10:43.692 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:10:43.693 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:10:43.693 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:10:43.693 INFO:tasks.workunit.client.0.smithi016.stdout:268/600: open fd = 3 2024-09-12T02:10:43.693 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:10:43.693 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:10:43.693 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:10:43.693 INFO:tasks.workunit.client.0.smithi016.stdout:269/600: open fd = 3 2024-09-12T02:10:43.693 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:10:43.693 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:10:43.693 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:10:43.693 INFO:tasks.workunit.client.0.smithi016.stdout:270/600: open fd = 3 2024-09-12T02:10:43.693 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:10:43.693 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:10:43.693 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:10:43.694 INFO:tasks.workunit.client.0.smithi016.stdout:271/600: open fd = 3 2024-09-12T02:10:43.694 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:10:43.694 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:10:43.694 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:10:43.694 INFO:tasks.workunit.client.0.smithi016.stdout:272/600: open fd = 3 2024-09-12T02:10:43.694 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:10:43.694 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:10:43.694 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:10:43.694 INFO:tasks.workunit.client.0.smithi016.stdout:273/600: open fd = 3 2024-09-12T02:10:43.695 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:10:43.695 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:10:43.695 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:10:43.695 INFO:tasks.workunit.client.0.smithi016.stdout:274/600: open fd = 3 2024-09-12T02:10:43.695 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:10:43.695 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:10:43.695 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:10:43.695 INFO:tasks.workunit.client.0.smithi016.stdout:275/600: open fd = 3 2024-09-12T02:10:43.695 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:10:43.695 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:10:43.695 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:10:43.696 INFO:tasks.workunit.client.0.smithi016.stdout:276/600: open fd = 3 2024-09-12T02:10:43.696 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:10:43.696 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:10:43.696 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:10:43.696 INFO:tasks.workunit.client.0.smithi016.stdout:277/600: open fd = 3 2024-09-12T02:10:43.696 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:10:43.696 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:10:43.696 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:10:43.696 INFO:tasks.workunit.client.0.smithi016.stdout:278/600: open fd = 3 2024-09-12T02:10:43.696 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:10:43.696 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:10:43.696 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:10:43.696 INFO:tasks.workunit.client.0.smithi016.stdout:279/600: open fd = 3 2024-09-12T02:10:43.697 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:10:43.697 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:10:43.697 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:10:43.697 INFO:tasks.workunit.client.0.smithi016.stdout:280/600: open fd = 3 2024-09-12T02:10:43.697 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:10:43.697 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:10:43.697 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:10:43.697 INFO:tasks.workunit.client.0.smithi016.stdout:281/600: open fd = 3 2024-09-12T02:10:43.697 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:10:43.697 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:10:43.697 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:10:43.697 INFO:tasks.workunit.client.0.smithi016.stdout:282/600: open fd = 3 2024-09-12T02:10:43.697 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:10:43.697 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:10:43.698 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:10:43.698 INFO:tasks.workunit.client.0.smithi016.stdout:283/600: open fd = 3 2024-09-12T02:10:43.698 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:10:43.698 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:10:43.698 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:10:43.698 INFO:tasks.workunit.client.0.smithi016.stdout:284/600: open fd = 3 2024-09-12T02:10:43.698 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:10:43.698 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:10:43.698 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:10:43.698 INFO:tasks.workunit.client.0.smithi016.stdout:285/600: open fd = 3 2024-09-12T02:10:43.699 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:10:43.699 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:10:43.699 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:10:43.699 INFO:tasks.workunit.client.0.smithi016.stdout:286/600: open fd = 3 2024-09-12T02:10:43.699 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:10:43.699 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:10:43.699 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:10:43.699 INFO:tasks.workunit.client.0.smithi016.stdout:287/600: open fd = 3 2024-09-12T02:10:43.699 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:10:43.699 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:10:43.700 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:10:43.700 INFO:tasks.workunit.client.0.smithi016.stdout:288/600: open fd = 3 2024-09-12T02:10:43.700 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:10:43.700 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:10:43.700 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:10:43.700 INFO:tasks.workunit.client.0.smithi016.stdout:289/600: open fd = 3 2024-09-12T02:10:43.700 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:10:43.700 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:10:43.700 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:10:43.700 INFO:tasks.workunit.client.0.smithi016.stdout:290/600: open fd = 3 2024-09-12T02:10:43.701 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:10:43.701 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:10:43.701 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:10:43.701 INFO:tasks.workunit.client.0.smithi016.stdout:291/600: open fd = 3 2024-09-12T02:10:43.701 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:10:43.701 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:10:43.701 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:10:43.701 INFO:tasks.workunit.client.0.smithi016.stdout:292/600: open fd = 3 2024-09-12T02:10:43.701 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:10:43.701 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:10:43.701 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:10:43.701 INFO:tasks.workunit.client.0.smithi016.stdout:293/600: open fd = 3 2024-09-12T02:10:43.701 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:10:43.701 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:10:43.702 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:10:43.702 INFO:tasks.workunit.client.0.smithi016.stdout:294/600: open fd = 3 2024-09-12T02:10:43.702 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:10:43.702 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:10:43.702 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:10:43.702 INFO:tasks.workunit.client.0.smithi016.stdout:295/600: open fd = 3 2024-09-12T02:10:43.702 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:10:43.702 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:10:43.702 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:10:43.702 INFO:tasks.workunit.client.0.smithi016.stdout:296/600: open fd = 3 2024-09-12T02:10:43.702 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:10:43.702 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:10:43.702 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:10:43.703 INFO:tasks.workunit.client.0.smithi016.stdout:297/600: open fd = 3 2024-09-12T02:10:43.703 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:10:43.703 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:10:43.703 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:10:43.703 INFO:tasks.workunit.client.0.smithi016.stdout:298/600: open fd = 3 2024-09-12T02:10:43.703 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:10:43.703 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:10:43.703 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:10:43.704 INFO:tasks.workunit.client.0.smithi016.stdout:299/600: open fd = 3 2024-09-12T02:10:43.704 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:10:43.704 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:10:43.704 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:10:43.704 INFO:tasks.workunit.client.0.smithi016.stdout:300/600: open fd = 3 2024-09-12T02:10:43.704 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:10:43.704 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:10:43.704 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:10:43.704 INFO:tasks.workunit.client.0.smithi016.stdout:301/600: open fd = 3 2024-09-12T02:10:43.705 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:10:43.705 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:10:43.705 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:10:43.705 INFO:tasks.workunit.client.0.smithi016.stdout:302/600: open fd = 3 2024-09-12T02:10:43.705 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:10:43.705 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:10:43.705 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:10:43.705 INFO:tasks.workunit.client.0.smithi016.stdout:303/600: open fd = 3 2024-09-12T02:10:43.705 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:10:43.705 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:10:43.705 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:10:43.705 INFO:tasks.workunit.client.0.smithi016.stdout:304/600: open fd = 3 2024-09-12T02:10:43.705 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:10:43.705 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:10:43.706 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:10:43.706 INFO:tasks.workunit.client.0.smithi016.stdout:305/600: open fd = 3 2024-09-12T02:10:43.706 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:10:43.706 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:10:43.706 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:10:43.706 INFO:tasks.workunit.client.0.smithi016.stdout:306/600: open fd = 3 2024-09-12T02:10:43.706 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:10:43.706 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:10:43.706 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:10:43.706 INFO:tasks.workunit.client.0.smithi016.stdout:307/600: open fd = 3 2024-09-12T02:10:43.706 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:10:43.706 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:10:43.706 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:10:43.706 INFO:tasks.workunit.client.0.smithi016.stdout:308/600: open fd = 3 2024-09-12T02:10:43.707 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:10:43.707 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:10:43.707 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:10:43.707 INFO:tasks.workunit.client.0.smithi016.stdout:309/600: open fd = 3 2024-09-12T02:10:43.707 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:10:43.707 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:10:43.707 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:10:43.707 INFO:tasks.workunit.client.0.smithi016.stdout:310/600: open fd = 3 2024-09-12T02:10:43.707 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:10:43.707 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:10:43.708 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:10:43.708 INFO:tasks.workunit.client.0.smithi016.stdout:311/600: open fd = 3 2024-09-12T02:10:43.708 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:10:43.708 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:10:43.708 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:10:43.708 INFO:tasks.workunit.client.0.smithi016.stdout:312/600: open fd = 3 2024-09-12T02:10:43.708 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:10:43.708 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:10:44.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:10:43 smithi184 ceph-mon[35593]: pgmap v599: 97 pgs: 97 active+clean; 4.6 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-12T02:10:44.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:10:43 smithi033 ceph-mon[35332]: pgmap v599: 97 pgs: 97 active+clean; 4.6 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-12T02:10:44.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:10:43 smithi016 ceph-mon[30947]: pgmap v599: 97 pgs: 97 active+clean; 4.6 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-12T02:10:46.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:10:45 smithi184 ceph-mon[35593]: pgmap v600: 97 pgs: 97 active+clean; 584 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 16 KiB/s wr, 14 op/s 2024-09-12T02:10:46.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:10:45 smithi033 ceph-mon[35332]: pgmap v600: 97 pgs: 97 active+clean; 584 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 16 KiB/s wr, 14 op/s 2024-09-12T02:10:46.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:10:45 smithi016 ceph-mon[30947]: pgmap v600: 97 pgs: 97 active+clean; 584 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 16 KiB/s wr, 14 op/s 2024-09-12T02:10:48.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:10:47 smithi184 ceph-mon[35593]: pgmap v601: 97 pgs: 97 active+clean; 654 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 16 KiB/s wr, 15 op/s 2024-09-12T02:10:48.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:10:47 smithi033 ceph-mon[35332]: pgmap v601: 97 pgs: 97 active+clean; 654 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 16 KiB/s wr, 15 op/s 2024-09-12T02:10:48.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:10:47 smithi016 ceph-mon[30947]: pgmap v601: 97 pgs: 97 active+clean; 654 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 16 KiB/s wr, 15 op/s 2024-09-12T02:10:50.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:10:49 smithi184 ceph-mon[35593]: pgmap v602: 97 pgs: 97 active+clean; 654 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 16 KiB/s wr, 14 op/s 2024-09-12T02:10:50.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:10:49 smithi033 ceph-mon[35332]: pgmap v602: 97 pgs: 97 active+clean; 654 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 16 KiB/s wr, 14 op/s 2024-09-12T02:10:50.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:10:49 smithi016 ceph-mon[30947]: pgmap v602: 97 pgs: 97 active+clean; 654 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 16 KiB/s wr, 14 op/s 2024-09-12T02:10:52.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:10:51 smithi184 ceph-mon[35593]: pgmap v603: 97 pgs: 97 active+clean; 654 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 16 KiB/s wr, 15 op/s 2024-09-12T02:10:52.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:10:51 smithi033 ceph-mon[35332]: pgmap v603: 97 pgs: 97 active+clean; 654 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 16 KiB/s wr, 15 op/s 2024-09-12T02:10:52.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:10:51 smithi016 ceph-mon[30947]: pgmap v603: 97 pgs: 97 active+clean; 654 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 16 KiB/s wr, 15 op/s 2024-09-12T02:10:53.390 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T02:10:53.666 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:10:54.038 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:10:53 smithi016 ceph-mon[30947]: pgmap v604: 97 pgs: 97 active+clean; 712 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 16 KiB/s wr, 14 op/s 2024-09-12T02:10:54.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.127+0000 7fb0c1817640 1 -- 172.21.15.16:0/3229980200 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0bc1036c0 msgr2=0x7fb0bc10bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:54.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.127+0000 7fb0c1817640 1 --2- 172.21.15.16:0/3229980200 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0bc1036c0 0x7fb0bc10bbb0 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7fb0b0009900 tx=0x7fb0b002a480 comp rx=0 tx=0).stop 2024-09-12T02:10:54.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.127+0000 7fb0c1817640 1 -- 172.21.15.16:0/3229980200 shutdown_connections 2024-09-12T02:10:54.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.127+0000 7fb0c1817640 1 --2- 172.21.15.16:0/3229980200 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb0bc10c180 0x7fb0bc10e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:54.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.127+0000 7fb0c1817640 1 --2- 172.21.15.16:0/3229980200 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0bc1036c0 0x7fb0bc10bbb0 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:54.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.127+0000 7fb0c1817640 1 --2- 172.21.15.16:0/3229980200 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb0bc102da0 0x7fb0bc103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:54.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.127+0000 7fb0c1817640 1 -- 172.21.15.16:0/3229980200 >> 172.21.15.16:0/3229980200 conn(0x7fb0bc0fc7a0 msgr2=0x7fb0bc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:10:54.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.127+0000 7fb0c1817640 1 -- 172.21.15.16:0/3229980200 shutdown_connections 2024-09-12T02:10:54.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.127+0000 7fb0c1817640 1 -- 172.21.15.16:0/3229980200 wait complete. 2024-09-12T02:10:54.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.128+0000 7fb0c1817640 1 Processor -- start 2024-09-12T02:10:54.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.128+0000 7fb0c1817640 1 -- start start 2024-09-12T02:10:54.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.128+0000 7fb0c1817640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb0bc102da0 0x7fb0bc1984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:54.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.128+0000 7fb0c1817640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb0bc1036c0 0x7fb0bc1989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:54.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.128+0000 7fb0c1817640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0bc10c180 0x7fb0bc19cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:54.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.128+0000 7fb0c1817640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb0bc111fa0 con 0x7fb0bc10c180 2024-09-12T02:10:54.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.128+0000 7fb0c1817640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb0bc111e20 con 0x7fb0bc102da0 2024-09-12T02:10:54.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.128+0000 7fb0c1817640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb0bc112120 con 0x7fb0bc1036c0 2024-09-12T02:10:54.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.129+0000 7fb0bb7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0bc10c180 0x7fb0bc19cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:54.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.129+0000 7fb0ba7fc640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb0bc1036c0 0x7fb0bc1989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:54.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.129+0000 7fb0bb7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0bc10c180 0x7fb0bc19cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39682/0 (socket says 172.21.15.16:39682) 2024-09-12T02:10:54.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.129+0000 7fb0bb7fe640 1 -- 172.21.15.16:0/3612403740 learned_addr learned my addr 172.21.15.16:0/3612403740 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:10:54.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.129+0000 7fb0baffd640 1 --2- 172.21.15.16:0/3612403740 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb0bc102da0 0x7fb0bc1984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:54.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.129+0000 7fb0bb7fe640 1 -- 172.21.15.16:0/3612403740 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb0bc1036c0 msgr2=0x7fb0bc1989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:54.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.129+0000 7fb0bb7fe640 1 --2- 172.21.15.16:0/3612403740 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb0bc1036c0 0x7fb0bc1989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:54.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.129+0000 7fb0bb7fe640 1 -- 172.21.15.16:0/3612403740 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb0bc102da0 msgr2=0x7fb0bc1984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:54.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.129+0000 7fb0bb7fe640 1 --2- 172.21.15.16:0/3612403740 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb0bc102da0 0x7fb0bc1984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:54.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.129+0000 7fb0bb7fe640 1 -- 172.21.15.16:0/3612403740 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0bc19d460 con 0x7fb0bc10c180 2024-09-12T02:10:54.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.129+0000 7fb0bb7fe640 1 --2- 172.21.15.16:0/3612403740 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0bc10c180 0x7fb0bc19cd80 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7fb0b400cce0 tx=0x7fb0b400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:10:54.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.129+0000 7fb0c0815640 1 -- 172.21.15.16:0/3612403740 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0b400f450 con 0x7fb0bc10c180 2024-09-12T02:10:54.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.129+0000 7fb0c1817640 1 -- 172.21.15.16:0/3612403740 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0bc19d750 con 0x7fb0bc10c180 2024-09-12T02:10:54.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.129+0000 7fb0c0815640 1 -- 172.21.15.16:0/3612403740 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0b400fad0 con 0x7fb0bc10c180 2024-09-12T02:10:54.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.129+0000 7fb0c0815640 1 -- 172.21.15.16:0/3612403740 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0b4018b60 con 0x7fb0bc10c180 2024-09-12T02:10:54.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.129+0000 7fb0c1817640 1 -- 172.21.15.16:0/3612403740 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0bc1a5030 con 0x7fb0bc10c180 2024-09-12T02:10:54.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.132+0000 7fb0c0815640 1 -- 172.21.15.16:0/3612403740 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fb0b4018d00 con 0x7fb0bc10c180 2024-09-12T02:10:54.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.132+0000 7fb0c3bf0740 1 Processor -- start 2024-09-12T02:10:54.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.132+0000 7fb0c3bf0740 1 -- start start 2024-09-12T02:10:54.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.132+0000 7fb0c3bf0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a5dc783c0 0x560a5dc787a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:54.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.132+0000 7fb0c3bf0740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560a5dc78ce0 0x560a5dc87550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:54.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.132+0000 7fb0c3bf0740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560a5dc87a90 0x560a5dc89e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:54.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.132+0000 7fb0c3bf0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x560a5dbde2e0 con 0x560a5dc783c0 2024-09-12T02:10:54.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.132+0000 7fb0c3bf0740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x560a5db85ce0 con 0x560a5dc87a90 2024-09-12T02:10:54.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.132+0000 7fb0c3bf0740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x560a5dbda2a0 con 0x560a5dc78ce0 2024-09-12T02:10:54.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.133+0000 7fb0ba7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a5dc783c0 0x560a5dc787a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:54.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.133+0000 7fb0ba7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a5dc783c0 0x560a5dc787a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39692/0 (socket says 172.21.15.16:39692) 2024-09-12T02:10:54.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.133+0000 7fb0ba7fc640 1 -- 172.21.15.16:0/1310909404 learned_addr learned my addr 172.21.15.16:0/1310909404 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:10:54.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.133+0000 7fb0baffd640 1 --2- 172.21.15.16:0/1310909404 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560a5dc78ce0 0x560a5dc87550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:54.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.133+0000 7fb0ba7fc640 1 --2- 172.21.15.16:0/1310909404 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560a5dc87a90 0x560a5dc89e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:54.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.133+0000 7fb0c0815640 1 --2- 172.21.15.16:0/3612403740 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fb0a40a2b80 0x7fb0a40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:54.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.133+0000 7fb0ba7fc640 1 -- 172.21.15.16:0/1310909404 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560a5dc78ce0 msgr2=0x560a5dc87550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:54.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.133+0000 7fb0ba7fc640 1 --2- 172.21.15.16:0/1310909404 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560a5dc78ce0 0x560a5dc87550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:54.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.133+0000 7fb0ba7fc640 1 -- 172.21.15.16:0/1310909404 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560a5dc87a90 msgr2=0x560a5dc89e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:54.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.133+0000 7fb0ba7fc640 1 --2- 172.21.15.16:0/1310909404 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560a5dc87a90 0x560a5dc89e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:54.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.133+0000 7fb0ba7fc640 1 -- 172.21.15.16:0/1310909404 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a5db6a440 con 0x560a5dc783c0 2024-09-12T02:10:54.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.133+0000 7fb0bb7fe640 1 --2- 172.21.15.16:0/3612403740 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fb0a40a2b80 0x7fb0a40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:54.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.133+0000 7fb0c0815640 1 -- 172.21.15.16:0/3612403740 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fb0b40cf500 con 0x7fb0bc10c180 2024-09-12T02:10:54.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.133+0000 7fb0ba7fc640 1 --2- 172.21.15.16:0/1310909404 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a5dc783c0 0x560a5dc787a0 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7fb0b002a9a0 tx=0x7fb0b0005d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=502f84df7ebf757 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:10:54.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.133+0000 7fb0a1ffb640 1 -- 172.21.15.16:0/1310909404 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0b0035040 con 0x560a5dc783c0 2024-09-12T02:10:54.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.134+0000 7fb0a1ffb640 1 -- 172.21.15.16:0/1310909404 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0b00035f0 con 0x560a5dc783c0 2024-09-12T02:10:54.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.134+0000 7fb0a1ffb640 1 -- 172.21.15.16:0/1310909404 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0b003a5f0 con 0x560a5dc783c0 2024-09-12T02:10:54.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.134+0000 7fb0c3bf0740 1 -- 172.21.15.16:0/1310909404 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a5dc783c0 msgr2=0x560a5dc787a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:54.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.134+0000 7fb0c3bf0740 1 --2- 172.21.15.16:0/1310909404 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a5dc783c0 0x560a5dc787a0 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7fb0b002a9a0 tx=0x7fb0b0005d90 comp rx=0 tx=0).stop 2024-09-12T02:10:54.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.134+0000 7fb0bb7fe640 1 --2- 172.21.15.16:0/3612403740 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fb0a40a2b80 0x7fb0a40a5040 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fb0b400afd0 tx=0x7fb0b4091240 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:10:54.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.134+0000 7fb0c3bf0740 1 -- 172.21.15.16:0/1310909404 shutdown_connections 2024-09-12T02:10:54.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.134+0000 7fb0c3bf0740 1 --2- 172.21.15.16:0/1310909404 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560a5dc87a90 0x560a5dc89e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:54.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.134+0000 7fb0c3bf0740 1 --2- 172.21.15.16:0/1310909404 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560a5dc78ce0 0x560a5dc87550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:54.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.134+0000 7fb0c3bf0740 1 --2- 172.21.15.16:0/1310909404 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a5dc783c0 0x560a5dc787a0 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:54.169 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T02:10:54.169 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-09-12T02:10:54.169 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {} 2024-09-12T02:10:54.169 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T02:10:54.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.134+0000 7fb0c3bf0740 1 -- 172.21.15.16:0/1310909404 >> 172.21.15.16:0/1310909404 conn(0x560a5dc74a30 msgr2=0x560a5dc74e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:10:54.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.134+0000 7fb0c3bf0740 1 -- 172.21.15.16:0/1310909404 shutdown_connections 2024-09-12T02:10:54.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.134+0000 7fb0c3bf0740 1 -- 172.21.15.16:0/1310909404 wait complete. 2024-09-12T02:10:54.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.135+0000 7fb0c3bf0740 1 -- start start 2024-09-12T02:10:54.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.135+0000 7fb0c3bf0740 1 Processor -- start 2024-09-12T02:10:54.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.135+0000 7fb0c3bf0740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560a5dc783c0 0x560a5dc87470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:54.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.135+0000 7fb0c3bf0740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560a5dc78ce0 0x560a5dc84720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:54.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.135+0000 7fb0c3bf0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a5dc87a90 0x560a5dc84cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:54.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.135+0000 7fb0c3bf0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x560a5dc8a5a0 con 0x560a5dc87a90 2024-09-12T02:10:54.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.135+0000 7fb0c3bf0740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x560a5db99050 con 0x560a5dc783c0 2024-09-12T02:10:54.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.135+0000 7fb0c3bf0740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x560a5dc8a720 con 0x560a5dc78ce0 2024-09-12T02:10:54.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.135+0000 7fb0c3bf0740 10 client.0 _ll_register_callbacks cb 0x560a5dbd9160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:10:54.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.135+0000 7fb0ba7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a5dc87a90 0x560a5dc84cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:54.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.135+0000 7fb0ba7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a5dc87a90 0x560a5dc84cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39694/0 (socket says 172.21.15.16:39694) 2024-09-12T02:10:54.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.135+0000 7fb0ba7fc640 1 -- 172.21.15.16:0/143255073 learned_addr learned my addr 172.21.15.16:0/143255073 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:10:54.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.136+0000 7fb0ba7fc640 1 --2- 172.21.15.16:0/143255073 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560a5dc783c0 0x560a5dc87470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:54.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.136+0000 7fb0baffd640 1 --2- 172.21.15.16:0/143255073 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560a5dc78ce0 0x560a5dc84720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:54.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.136+0000 7fb0ba7fc640 1 -- 172.21.15.16:0/143255073 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560a5dc78ce0 msgr2=0x560a5dc84720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:54.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.136+0000 7fb0ba7fc640 1 --2- 172.21.15.16:0/143255073 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560a5dc78ce0 0x560a5dc84720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:54.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.136+0000 7fb0ba7fc640 1 -- 172.21.15.16:0/143255073 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560a5dc783c0 msgr2=0x560a5dc87470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:54.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.136+0000 7fb0ba7fc640 1 --2- 172.21.15.16:0/143255073 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560a5dc783c0 0x560a5dc87470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:54.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.136+0000 7fb0ba7fc640 1 -- 172.21.15.16:0/143255073 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a5d94f3c0 con 0x560a5dc87a90 2024-09-12T02:10:54.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.136+0000 7fb0baffd640 1 --2- 172.21.15.16:0/143255073 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560a5dc78ce0 0x560a5dc84720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:10:54.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.136+0000 7fb0ba7fc640 1 --2- 172.21.15.16:0/143255073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a5dc87a90 0x560a5dc84cf0 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7fb0b0004310 tx=0x7fb0b0003f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70486a5a035c876f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:10:54.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.136+0000 7fb09affd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:10:54.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.136+0000 7fb09affd640 1 -- 172.21.15.16:0/143255073 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0b0035040 con 0x560a5dc87a90 2024-09-12T02:10:54.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.136+0000 7fb09affd640 1 -- 172.21.15.16:0/143255073 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fb0b0045040 con 0x560a5dc87a90 2024-09-12T02:10:54.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.136+0000 7fb0c3bf0740 1 -- 172.21.15.16:0/143255073 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0b00035f0 con 0x560a5dc87a90 2024-09-12T02:10:54.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.136+0000 7fb09affd640 1 -- 172.21.15.16:0/143255073 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fb0b00037a0 con 0x560a5dc87a90 2024-09-12T02:10:54.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.136+0000 7fb0c3bf0740 10 client.18135 fetch_fsmap learned FSMap version 25 2024-09-12T02:10:54.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.136+0000 7fb0c3bf0740 1 -- 172.21.15.16:0/143255073 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x560a5dc96020 con 0x560a5dc87a90 2024-09-12T02:10:54.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.137+0000 7fb09affd640 1 -- 172.21.15.16:0/143255073 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0b00035f0 con 0x560a5dc87a90 2024-09-12T02:10:54.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.137+0000 7fb09affd640 1 -- 172.21.15.16:0/143255073 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fb0b0049410 con 0x560a5dc87a90 2024-09-12T02:10:54.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.137+0000 7fb0c3bf0740 10 client.18135 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:10:54.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.137+0000 7fb0c3bf0740 10 client.18135 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:10:54.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.137+0000 7fb0c3bf0740 20 client.18135 populate_metadata read hostname 'smithi016' 2024-09-12T02:10:54.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.137+0000 7fb0c3bf0740 1 --2- 172.21.15.16:0/143255073 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x560a5dc97370 0x560a5dc99760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:54.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.137+0000 7fb0c3bf0740 4 client.18135 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:10:54.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.137+0000 7fb0c3bf0740 1 -- 172.21.15.16:0/143255073 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560a5dbeb850 con 0x560a5dc97370 2024-09-12T02:10:54.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.137+0000 7fb0baffd640 1 --2- 172.21.15.16:0/143255073 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x560a5dc97370 0x560a5dc99760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:54.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.138+0000 7fb09affd640 10 client.18135 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:10:54.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.138+0000 7fb0baffd640 1 --2- 172.21.15.16:0/143255073 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x560a5dc97370 0x560a5dc99760 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0f5ca52cd986c3c server_cookie=4c6ea093b4273a22 in_seq=0 out_seq=0 2024-09-12T02:10:54.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.138+0000 7fb09affd640 1 -- 172.21.15.16:0/143255073 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560a5dbeb850 con 0x560a5dc97370 2024-09-12T02:10:54.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.138+0000 7fb09affd640 10 client.18135 handle_command_reply: tid=0 2024-09-12T02:10:54.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.138+0000 7fb0c3bf0740 1 client.18135 shutdown 2024-09-12T02:10:54.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.139+0000 7fb0c3bf0740 1 -- 172.21.15.16:0/143255073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a5dc87a90 msgr2=0x560a5dc84cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:54.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.139+0000 7fb0c3bf0740 1 --2- 172.21.15.16:0/143255073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a5dc87a90 0x560a5dc84cf0 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7fb0b0004310 tx=0x7fb0b0003f40 comp rx=0 tx=0).stop 2024-09-12T02:10:54.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.139+0000 7fb0c3bf0740 1 -- 172.21.15.16:0/143255073 shutdown_connections 2024-09-12T02:10:54.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.139+0000 7fb0c3bf0740 1 --2- 172.21.15.16:0/143255073 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x560a5dc97370 0x560a5dc99760 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:54.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.139+0000 7fb0c3bf0740 1 --2- 172.21.15.16:0/143255073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a5dc87a90 0x560a5dc84cf0 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:54.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.139+0000 7fb09affd640 0 client.18135 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:10:54.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.139+0000 7fb0c3bf0740 1 --2- 172.21.15.16:0/143255073 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560a5dc78ce0 0x560a5dc84720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:54.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.139+0000 7fb0c3bf0740 1 --2- 172.21.15.16:0/143255073 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560a5dc783c0 0x560a5dc87470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:54.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.139+0000 7fb0c3bf0740 1 -- 172.21.15.16:0/143255073 >> 172.21.15.16:0/143255073 conn(0x560a5dc74a30 msgr2=0x560a5dc73b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:10:54.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.139+0000 7fb0c3bf0740 1 -- 172.21.15.16:0/143255073 shutdown_connections 2024-09-12T02:10:54.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.139+0000 7fb0c3bf0740 1 -- 172.21.15.16:0/143255073 wait complete. 2024-09-12T02:10:54.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.139+0000 7fb0c3bf0740 20 client.18135 trim_cache size 0 max 16384 2024-09-12T02:10:54.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.156+0000 7fb0c3bf0740 1 Processor -- start 2024-09-12T02:10:54.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.156+0000 7fb0c3bf0740 1 -- start start 2024-09-12T02:10:54.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.156+0000 7fb0c3bf0740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560a5dc783c0 0x560a5dc73a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:54.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.156+0000 7fb0c3bf0740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560a5dc78ce0 0x560a5dc819a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:54.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.156+0000 7fb0c3bf0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a5dc87a90 0x560a5dc81ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:54.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.156+0000 7fb0c3bf0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb0bc1ad9b0 con 0x560a5dc87a90 2024-09-12T02:10:54.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.156+0000 7fb0c3bf0740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb0bc1ab280 con 0x560a5dc78ce0 2024-09-12T02:10:54.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.156+0000 7fb0c3bf0740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb0bc1a90f0 con 0x560a5dc783c0 2024-09-12T02:10:54.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.156+0000 7fb0ba7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a5dc87a90 0x560a5dc81ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:54.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.156+0000 7fb0ba7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a5dc87a90 0x560a5dc81ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39702/0 (socket says 172.21.15.16:39702) 2024-09-12T02:10:54.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.156+0000 7fb0ba7fc640 1 -- 172.21.15.16:0/25261917 learned_addr learned my addr 172.21.15.16:0/25261917 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:10:54.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.157+0000 7fb0baffd640 1 --2- 172.21.15.16:0/25261917 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560a5dc78ce0 0x560a5dc819a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:54.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.157+0000 7fb0ba7fc640 1 --2- 172.21.15.16:0/25261917 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560a5dc783c0 0x560a5dc73a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:54.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.157+0000 7fb0ba7fc640 1 -- 172.21.15.16:0/25261917 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560a5dc783c0 msgr2=0x560a5dc73a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:54.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.157+0000 7fb0ba7fc640 1 --2- 172.21.15.16:0/25261917 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560a5dc783c0 0x560a5dc73a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:54.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.157+0000 7fb0ba7fc640 1 -- 172.21.15.16:0/25261917 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560a5dc78ce0 msgr2=0x560a5dc819a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:54.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.157+0000 7fb0ba7fc640 1 --2- 172.21.15.16:0/25261917 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560a5dc78ce0 0x560a5dc819a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:54.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.157+0000 7fb0ba7fc640 1 -- 172.21.15.16:0/25261917 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a5dc82c70 con 0x560a5dc87a90 2024-09-12T02:10:54.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.157+0000 7fb0ba7fc640 1 --2- 172.21.15.16:0/25261917 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a5dc87a90 0x560a5dc81ee0 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7fb0b0009850 tx=0x7fb0b00318e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7e6b7d178c4c502 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:10:54.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.157+0000 7fb0a17fa640 1 -- 172.21.15.16:0/25261917 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0b0035040 con 0x560a5dc87a90 2024-09-12T02:10:54.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.157+0000 7fb0a17fa640 1 -- 172.21.15.16:0/25261917 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0b00035f0 con 0x560a5dc87a90 2024-09-12T02:10:54.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.157+0000 7fb0a17fa640 1 -- 172.21.15.16:0/25261917 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0b0049b10 con 0x560a5dc87a90 2024-09-12T02:10:54.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.158+0000 7fb0c3bf0740 1 -- 172.21.15.16:0/25261917 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a5dc87a90 msgr2=0x560a5dc81ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:54.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.158+0000 7fb0c3bf0740 1 --2- 172.21.15.16:0/25261917 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a5dc87a90 0x560a5dc81ee0 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7fb0b0009850 tx=0x7fb0b00318e0 comp rx=0 tx=0).stop 2024-09-12T02:10:54.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.158+0000 7fb0c3bf0740 1 -- 172.21.15.16:0/25261917 shutdown_connections 2024-09-12T02:10:54.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.158+0000 7fb0c3bf0740 1 --2- 172.21.15.16:0/25261917 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a5dc87a90 0x560a5dc81ee0 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:54.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.158+0000 7fb0c3bf0740 1 --2- 172.21.15.16:0/25261917 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560a5dc78ce0 0x560a5dc819a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:54.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.158+0000 7fb0c3bf0740 1 --2- 172.21.15.16:0/25261917 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560a5dc783c0 0x560a5dc73a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:54.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.158+0000 7fb0c3bf0740 1 -- 172.21.15.16:0/25261917 >> 172.21.15.16:0/25261917 conn(0x560a5dc74a30 msgr2=0x560a5dc73370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:10:54.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.158+0000 7fb0c3bf0740 1 -- 172.21.15.16:0/25261917 shutdown_connections 2024-09-12T02:10:54.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.158+0000 7fb0c3bf0740 1 -- 172.21.15.16:0/25261917 wait complete. 2024-09-12T02:10:54.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.158+0000 7fb0c3bf0740 1 -- start start 2024-09-12T02:10:54.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.158+0000 7fb0c3bf0740 1 Processor -- start 2024-09-12T02:10:54.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.159+0000 7fb0c3bf0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a5dc783c0 0x560a5dc79ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:54.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.159+0000 7fb0c3bf0740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560a5dc78ce0 0x560a5dc7a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:54.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.159+0000 7fb0c3bf0740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560a5dc87a90 0x560a5dc7ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:54.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.159+0000 7fb0c3bf0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb0bc1a9740 con 0x560a5dc783c0 2024-09-12T02:10:54.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.159+0000 7fb0c3bf0740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb0bc1aabb0 con 0x560a5dc87a90 2024-09-12T02:10:54.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.159+0000 7fb0c3bf0740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb0bc1a9e10 con 0x560a5dc78ce0 2024-09-12T02:10:54.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.159+0000 7fb0c3bf0740 10 client.0 _ll_register_callbacks cb 0x7fb0b0031970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:10:54.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.159+0000 7fb0ba7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a5dc783c0 0x560a5dc79ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:54.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.159+0000 7fb0ba7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a5dc783c0 0x560a5dc79ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39716/0 (socket says 172.21.15.16:39716) 2024-09-12T02:10:54.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.159+0000 7fb0ba7fc640 1 -- 172.21.15.16:0/335055941 learned_addr learned my addr 172.21.15.16:0/335055941 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:10:54.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.159+0000 7fb0baffd640 1 --2- 172.21.15.16:0/335055941 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560a5dc78ce0 0x560a5dc7a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:54.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.159+0000 7fb0ba7fc640 1 --2- 172.21.15.16:0/335055941 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560a5dc87a90 0x560a5dc7ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:54.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.159+0000 7fb0ba7fc640 1 -- 172.21.15.16:0/335055941 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560a5dc78ce0 msgr2=0x560a5dc7a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:54.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.159+0000 7fb0ba7fc640 1 --2- 172.21.15.16:0/335055941 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560a5dc78ce0 0x560a5dc7a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:54.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.159+0000 7fb0ba7fc640 1 -- 172.21.15.16:0/335055941 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560a5dc87a90 msgr2=0x560a5dc7ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:54.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.159+0000 7fb0ba7fc640 1 --2- 172.21.15.16:0/335055941 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560a5dc87a90 0x560a5dc7ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:54.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.159+0000 7fb0ba7fc640 1 -- 172.21.15.16:0/335055941 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a5dc7f4a0 con 0x560a5dc783c0 2024-09-12T02:10:54.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.159+0000 7fb0ba7fc640 1 --2- 172.21.15.16:0/335055941 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a5dc783c0 0x560a5dc79ff0 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7fb0b0005a80 tx=0x7fb0b0005d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67d2c874df8341ea server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:10:54.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.159+0000 7fb09affd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:10:54.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.160+0000 7fb09affd640 1 -- 172.21.15.16:0/335055941 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0b0035070 con 0x560a5dc783c0 2024-09-12T02:10:54.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.160+0000 7fb09affd640 1 -- 172.21.15.16:0/335055941 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fb0b0053040 con 0x560a5dc783c0 2024-09-12T02:10:54.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.160+0000 7fb0c3bf0740 1 -- 172.21.15.16:0/335055941 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0b00035f0 con 0x560a5dc783c0 2024-09-12T02:10:54.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.160+0000 7fb09affd640 1 -- 172.21.15.16:0/335055941 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fb0b0049cd0 con 0x560a5dc783c0 2024-09-12T02:10:54.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.160+0000 7fb0c3bf0740 10 client.18141 fetch_fsmap learned FSMap version 25 2024-09-12T02:10:54.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.160+0000 7fb0c3bf0740 1 -- 172.21.15.16:0/335055941 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x560a5dd06ae0 con 0x560a5dc783c0 2024-09-12T02:10:54.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.160+0000 7fb09affd640 1 -- 172.21.15.16:0/335055941 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0b00035f0 con 0x560a5dc783c0 2024-09-12T02:10:54.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.160+0000 7fb09affd640 1 -- 172.21.15.16:0/335055941 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fb0b0003940 con 0x560a5dc783c0 2024-09-12T02:10:54.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.160+0000 7fb0c3bf0740 10 client.18141 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:10:54.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.160+0000 7fb0c3bf0740 10 client.18141 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:10:54.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.160+0000 7fb0c3bf0740 20 client.18141 populate_metadata read hostname 'smithi016' 2024-09-12T02:10:54.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.160+0000 7fb0c3bf0740 1 --2- 172.21.15.16:0/335055941 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x560a5dc97370 0x560a5dd08ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:54.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.160+0000 7fb0c3bf0740 4 client.18141 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T02:10:54.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.160+0000 7fb0c3bf0740 1 -- 172.21.15.16:0/335055941 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560a5dbe5d60 con 0x560a5dc97370 2024-09-12T02:10:54.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.161+0000 7fb0baffd640 1 --2- 172.21.15.16:0/335055941 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x560a5dc97370 0x560a5dd08ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:54.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.161+0000 7fb0baffd640 1 --2- 172.21.15.16:0/335055941 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x560a5dc97370 0x560a5dd08ed0 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a35c33aae719e38a server_cookie=b249b68441b753df in_seq=0 out_seq=0 2024-09-12T02:10:54.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.161+0000 7fb09affd640 10 client.18141 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:10:54.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.161+0000 7fb09affd640 1 -- 172.21.15.16:0/335055941 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x560a5dbe5d60 con 0x560a5dc97370 2024-09-12T02:10:54.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.161+0000 7fb09affd640 10 client.18141 handle_command_reply: tid=0 2024-09-12T02:10:54.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.161+0000 7fb0c3bf0740 1 client.18141 shutdown 2024-09-12T02:10:54.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.161+0000 7fb0c3bf0740 1 -- 172.21.15.16:0/335055941 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a5dc783c0 msgr2=0x560a5dc79ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:54.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.161+0000 7fb0c3bf0740 1 --2- 172.21.15.16:0/335055941 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a5dc783c0 0x560a5dc79ff0 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7fb0b0005a80 tx=0x7fb0b0005d90 comp rx=0 tx=0).stop 2024-09-12T02:10:54.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.161+0000 7fb0c3bf0740 1 -- 172.21.15.16:0/335055941 shutdown_connections 2024-09-12T02:10:54.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.161+0000 7fb0c3bf0740 1 --2- 172.21.15.16:0/335055941 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x560a5dc97370 0x560a5dd08ed0 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:54.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.161+0000 7fb0c3bf0740 1 --2- 172.21.15.16:0/335055941 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560a5dc87a90 0x560a5dc7ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:54.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.161+0000 7fb0c3bf0740 1 --2- 172.21.15.16:0/335055941 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560a5dc78ce0 0x560a5dc7a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:54.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.161+0000 7fb0c3bf0740 1 --2- 172.21.15.16:0/335055941 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a5dc783c0 0x560a5dc79ff0 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:54.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.161+0000 7fb0c3bf0740 1 -- 172.21.15.16:0/335055941 >> 172.21.15.16:0/335055941 conn(0x560a5dc74a30 msgr2=0x560a5dc85f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:10:54.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.162+0000 7fb09affd640 0 client.18141 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:10:54.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.162+0000 7fb0c3bf0740 1 -- 172.21.15.16:0/335055941 shutdown_connections 2024-09-12T02:10:54.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.162+0000 7fb0c3bf0740 1 -- 172.21.15.16:0/335055941 wait complete. 2024-09-12T02:10:54.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.162+0000 7fb0c3bf0740 20 client.18141 trim_cache size 0 max 16384 2024-09-12T02:10:54.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.162+0000 7fb0c1817640 1 -- 172.21.15.16:0/3612403740 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fb0a40a2b80 msgr2=0x7fb0a40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:54.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.162+0000 7fb0c1817640 1 --2- 172.21.15.16:0/3612403740 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fb0a40a2b80 0x7fb0a40a5040 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fb0b400afd0 tx=0x7fb0b4091240 comp rx=0 tx=0).stop 2024-09-12T02:10:54.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.162+0000 7fb0c1817640 1 -- 172.21.15.16:0/3612403740 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0bc10c180 msgr2=0x7fb0bc19cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:54.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.162+0000 7fb0c1817640 1 --2- 172.21.15.16:0/3612403740 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0bc10c180 0x7fb0bc19cd80 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7fb0b400cce0 tx=0x7fb0b400a620 comp rx=0 tx=0).stop 2024-09-12T02:10:54.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.162+0000 7fb0c1817640 1 -- 172.21.15.16:0/3612403740 shutdown_connections 2024-09-12T02:10:54.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.162+0000 7fb0c1817640 1 --2- 172.21.15.16:0/3612403740 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fb0a40a2b80 0x7fb0a40a5040 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:54.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.162+0000 7fb0c1817640 1 --2- 172.21.15.16:0/3612403740 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0bc10c180 0x7fb0bc19cd80 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:54.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.162+0000 7fb0c1817640 1 --2- 172.21.15.16:0/3612403740 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb0bc1036c0 0x7fb0bc1989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:54.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.162+0000 7fb0c1817640 1 --2- 172.21.15.16:0/3612403740 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb0bc102da0 0x7fb0bc1984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:54.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.162+0000 7fb0c1817640 1 -- 172.21.15.16:0/3612403740 >> 172.21.15.16:0/3612403740 conn(0x7fb0bc0fc7a0 msgr2=0x7fb0bc0fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:10:54.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.163+0000 7fb0c1817640 1 -- 172.21.15.16:0/3612403740 shutdown_connections 2024-09-12T02:10:54.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:54.163+0000 7fb0c1817640 1 -- 172.21.15.16:0/3612403740 wait complete. 2024-09-12T02:10:54.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:10:53 smithi184 ceph-mon[35593]: pgmap v604: 97 pgs: 97 active+clean; 712 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 16 KiB/s wr, 14 op/s 2024-09-12T02:10:54.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:10:53 smithi033 ceph-mon[35332]: pgmap v604: 97 pgs: 97 active+clean; 712 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 16 KiB/s wr, 14 op/s 2024-09-12T02:10:55.063 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-12T02:10:55.064 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:10:55.327 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:10:55.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:55.816+0000 7f99e49be640 1 -- 172.21.15.16:0/829059040 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99e01082a0 msgr2=0x7f99e010ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:55.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:55.816+0000 7f99e49be640 1 --2- 172.21.15.16:0/829059040 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99e01082a0 0x7f99e010ec40 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f99d8009930 tx=0x7f99d802a420 comp rx=0 tx=0).stop 2024-09-12T02:10:55.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:55.816+0000 7f99e49be640 1 -- 172.21.15.16:0/829059040 shutdown_connections 2024-09-12T02:10:55.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:55.816+0000 7f99e49be640 1 --2- 172.21.15.16:0/829059040 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99e01082a0 0x7f99e010ec40 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:55.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:55.816+0000 7f99e49be640 1 --2- 172.21.15.16:0/829059040 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f99e01036e0 0x7f99e0107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:55.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:55.816+0000 7f99e49be640 1 --2- 172.21.15.16:0/829059040 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99e0102d30 0x7f99e0103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:55.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:55.816+0000 7f99e49be640 1 -- 172.21.15.16:0/829059040 >> 172.21.15.16:0/829059040 conn(0x7f99e00fc7d0 msgr2=0x7f99e00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:10:55.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:55.816+0000 7f99e49be640 1 -- 172.21.15.16:0/829059040 shutdown_connections 2024-09-12T02:10:55.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:55.816+0000 7f99e49be640 1 -- 172.21.15.16:0/829059040 wait complete. 2024-09-12T02:10:55.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:55.817+0000 7f99e49be640 1 Processor -- start 2024-09-12T02:10:55.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:55.817+0000 7f99e49be640 1 -- start start 2024-09-12T02:10:55.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:55.817+0000 7f99e49be640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99e0102d30 0x7f99e01985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:55.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:55.817+0000 7f99e49be640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99e01036e0 0x7f99e0198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:55.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:55.817+0000 7f99e49be640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f99e01082a0 0x7f99e019ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:55.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:55.817+0000 7f99e49be640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f99e01120d0 con 0x7f99e01036e0 2024-09-12T02:10:55.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:55.817+0000 7f99e49be640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f99e0111f50 con 0x7f99e0102d30 2024-09-12T02:10:55.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:55.817+0000 7f99e49be640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f99e0112250 con 0x7f99e01082a0 2024-09-12T02:10:55.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:55.818+0000 7f99ddd9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99e01036e0 0x7f99e0198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:55.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:55.818+0000 7f99ddd9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99e01036e0 0x7f99e0198af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39738/0 (socket says 172.21.15.16:39738) 2024-09-12T02:10:55.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:55.818+0000 7f99ddd9b640 1 -- 172.21.15.16:0/461785770 learned_addr learned my addr 172.21.15.16:0/461785770 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:10:55.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:55.818+0000 7f99de59c640 1 --2- 172.21.15.16:0/461785770 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99e0102d30 0x7f99e01985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:55.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:55.818+0000 7f99ddd9b640 1 -- 172.21.15.16:0/461785770 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f99e01082a0 msgr2=0x7f99e019ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:55.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:55.818+0000 7f99ddd9b640 1 --2- 172.21.15.16:0/461785770 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f99e01082a0 0x7f99e019ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:55.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:55.818+0000 7f99ddd9b640 1 -- 172.21.15.16:0/461785770 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99e0102d30 msgr2=0x7f99e01985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:55.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:55.818+0000 7f99ddd9b640 1 --2- 172.21.15.16:0/461785770 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99e0102d30 0x7f99e01985b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:55.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:55.818+0000 7f99ddd9b640 1 -- 172.21.15.16:0/461785770 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99e019d560 con 0x7f99e01036e0 2024-09-12T02:10:55.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:55.818+0000 7f99ddd9b640 1 --2- 172.21.15.16:0/461785770 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99e01036e0 0x7f99e0198af0 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f99c800d950 tx=0x7f99c800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:10:55.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:55.818+0000 7f99d77fe640 1 -- 172.21.15.16:0/461785770 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99c8017070 con 0x7f99e01036e0 2024-09-12T02:10:55.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:55.818+0000 7f99e49be640 1 -- 172.21.15.16:0/461785770 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99e019d850 con 0x7f99e01036e0 2024-09-12T02:10:55.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:55.818+0000 7f99d77fe640 1 -- 172.21.15.16:0/461785770 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99c8009c30 con 0x7f99e01036e0 2024-09-12T02:10:55.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:55.818+0000 7f99d77fe640 1 -- 172.21.15.16:0/461785770 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99c80047a0 con 0x7f99e01036e0 2024-09-12T02:10:55.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:55.818+0000 7f99e49be640 1 -- 172.21.15.16:0/461785770 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99e01a5130 con 0x7f99e01036e0 2024-09-12T02:10:55.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:55.820+0000 7f99e49be640 1 -- 172.21.15.16:0/461785770 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99e0103110 con 0x7f99e01036e0 2024-09-12T02:10:55.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:55.821+0000 7f99d77fe640 1 -- 172.21.15.16:0/461785770 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f99c8004020 con 0x7f99e01036e0 2024-09-12T02:10:55.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:55.822+0000 7f99d77fe640 1 --2- 172.21.15.16:0/461785770 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f99cc0a2b80 0x7f99cc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:55.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:55.822+0000 7f99de59c640 1 --2- 172.21.15.16:0/461785770 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f99cc0a2b80 0x7f99cc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:55.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:55.822+0000 7f99d77fe640 1 -- 172.21.15.16:0/461785770 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f99c80cebb0 con 0x7f99e01036e0 2024-09-12T02:10:55.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:55.822+0000 7f99de59c640 1 --2- 172.21.15.16:0/461785770 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f99cc0a2b80 0x7f99cc0a5040 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f99d0005c40 tx=0x7f99d000a590 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:10:55.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:55.826+0000 7f99d77fe640 1 -- 172.21.15.16:0/461785770 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f99c8020020 con 0x7f99e01036e0 2024-09-12T02:10:56.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:56.040+0000 7f99e49be640 1 -- 172.21.15.16:0/461785770 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f99e010a2e0 con 0x7f99e01036e0 2024-09-12T02:10:56.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:56.041+0000 7f99d77fe640 1 -- 172.21.15.16:0/461785770 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11911 (secure 0 0 0) 0x7f99c809bbb0 con 0x7f99e01036e0 2024-09-12T02:10:56.041 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T02:10:56.042 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":25,"btime":"2024-09-12T02:01:37:192021+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3279123279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3279123279},{"type":"v1","addr":"172.21.15.16:6835","nonce":3279123279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4119452382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4119452382},{"type":"v1","addr":"172.21.15.16:6837","nonce":4119452382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/3263823165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3263823165},{"type":"v1","addr":"172.21.15.16:6839","nonce":3263823165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/1510290732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":1510290732},{"type":"v1","addr":"172.21.15.16:6841","nonce":1510290732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/279674603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":279674603},{"type":"v1","addr":"172.21.15.33:6833","nonce":279674603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3555728028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3555728028},{"type":"v1","addr":"172.21.15.33:6835","nonce":3555728028}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T01:59:56.117608+0000","modified":"2024-09-12T02:01:36.294760+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24482,"mds_1":24481,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6833/1563374106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1563374106},{"type":"v1","addr":"172.21.15.184:6833","nonce":1563374106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6837/1892833868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1892833868},{"type":"v1","addr":"172.21.15.33:6837","nonce":1892833868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/1457679686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1457679686},{"type":"v1","addr":"172.21.15.184:6835","nonce":1457679686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.184:6837/1793393087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":1793393087},{"type":"v1","addr":"172.21.15.184:6837","nonce":1793393087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.184:6839/4203347941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":4203347941},{"type":"v1","addr":"172.21.15.184:6839","nonce":4203347941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.33:6839/4136670204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":4136670204},{"type":"v1","addr":"172.21.15.33:6839","nonce":4136670204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24481,24482,24469]},"id":1}]} 2024-09-12T02:10:56.042 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 25 2024-09-12T02:10:56.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:56.043+0000 7f99e49be640 1 -- 172.21.15.16:0/461785770 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f99cc0a2b80 msgr2=0x7f99cc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:56.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:56.043+0000 7f99e49be640 1 --2- 172.21.15.16:0/461785770 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f99cc0a2b80 0x7f99cc0a5040 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f99d0005c40 tx=0x7f99d000a590 comp rx=0 tx=0).stop 2024-09-12T02:10:56.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:56.043+0000 7f99e49be640 1 -- 172.21.15.16:0/461785770 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99e01036e0 msgr2=0x7f99e0198af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:56.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:56.043+0000 7f99e49be640 1 --2- 172.21.15.16:0/461785770 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99e01036e0 0x7f99e0198af0 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f99c800d950 tx=0x7f99c800de10 comp rx=0 tx=0).stop 2024-09-12T02:10:56.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:56.043+0000 7f99e49be640 1 -- 172.21.15.16:0/461785770 shutdown_connections 2024-09-12T02:10:56.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:56.043+0000 7f99e49be640 1 --2- 172.21.15.16:0/461785770 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f99cc0a2b80 0x7f99cc0a5040 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:56.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:56.043+0000 7f99e49be640 1 --2- 172.21.15.16:0/461785770 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f99e01082a0 0x7f99e019ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:56.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:56.043+0000 7f99e49be640 1 --2- 172.21.15.16:0/461785770 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99e01036e0 0x7f99e0198af0 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:56.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:56.043+0000 7f99e49be640 1 --2- 172.21.15.16:0/461785770 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99e0102d30 0x7f99e01985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:56.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:56.043+0000 7f99e49be640 1 -- 172.21.15.16:0/461785770 >> 172.21.15.16:0/461785770 conn(0x7f99e00fc7d0 msgr2=0x7f99e0106380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:10:56.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:56.043+0000 7f99e49be640 1 -- 172.21.15.16:0/461785770 shutdown_connections 2024-09-12T02:10:56.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:56.043+0000 7f99e49be640 1 -- 172.21.15.16:0/461785770 wait complete. 2024-09-12T02:10:56.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:10:55 smithi184 ceph-mon[35593]: pgmap v605: 97 pgs: 97 active+clean; 712 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 16 KiB/s wr, 14 op/s 2024-09-12T02:10:56.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:10:55 smithi033 ceph-mon[35332]: pgmap v605: 97 pgs: 97 active+clean; 712 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 16 KiB/s wr, 14 op/s 2024-09-12T02:10:56.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:10:55 smithi016 ceph-mon[30947]: pgmap v605: 97 pgs: 97 active+clean; 712 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 16 KiB/s wr, 14 op/s 2024-09-12T02:10:56.533 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.530+0000 7f0d54a77640 1 -- 172.21.15.16:0/1160718440 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d50102610 msgr2=0x7f0d50108ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:56.533 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.530+0000 7f0d54a77640 1 --2- 172.21.15.16:0/1160718440 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d50102610 0x7f0d50108ea0 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f0d48009ca0 tx=0x7f0d4802a8c0 comp rx=0 tx=0).stop 2024-09-12T02:10:56.533 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.530+0000 7f0d54a77640 1 -- 172.21.15.16:0/1160718440 shutdown_connections 2024-09-12T02:10:56.533 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.530+0000 7f0d54a77640 1 --2- 172.21.15.16:0/1160718440 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d50102610 0x7f0d50108ea0 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:56.533 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.530+0000 7f0d54a77640 1 --2- 172.21.15.16:0/1160718440 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0d50101c50 0x7f0d501020d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:56.533 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.530+0000 7f0d54a77640 1 --2- 172.21.15.16:0/1160718440 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0d50100a60 0x7f0d50100e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:56.533 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.530+0000 7f0d54a77640 1 -- 172.21.15.16:0/1160718440 >> 172.21.15.16:0/1160718440 conn(0x7f0d500fc1e0 msgr2=0x7f0d500fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:10:56.533 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.530+0000 7f0d54a77640 1 -- 172.21.15.16:0/1160718440 shutdown_connections 2024-09-12T02:10:56.533 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.530+0000 7f0d54a77640 1 -- 172.21.15.16:0/1160718440 wait complete. 2024-09-12T02:10:56.533 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.531+0000 7f0d54a77640 1 Processor -- start 2024-09-12T02:10:56.533 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.531+0000 7f0d54a77640 1 -- start start 2024-09-12T02:10:56.534 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.531+0000 7f0d54a77640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0d50100a60 0x7f0d50197db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:56.534 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.531+0000 7f0d54a77640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0d50101c50 0x7f0d501982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:56.534 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.531+0000 7f0d54a77640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d50102610 0x7f0d5019f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:56.534 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.531+0000 7f0d54a77640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0d5010d830 con 0x7f0d50102610 2024-09-12T02:10:56.534 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.531+0000 7f0d54a77640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0d5010d6b0 con 0x7f0d50101c50 2024-09-12T02:10:56.534 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.531+0000 7f0d54a77640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f0d5010d9b0 con 0x7f0d50100a60 2024-09-12T02:10:56.534 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.532+0000 7f0d4ed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d50102610 0x7f0d5019f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:56.534 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.532+0000 7f0d4ed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d50102610 0x7f0d5019f360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39762/0 (socket says 172.21.15.16:39762) 2024-09-12T02:10:56.534 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.532+0000 7f0d4ed9d640 1 -- 172.21.15.16:0/693005213 learned_addr learned my addr 172.21.15.16:0/693005213 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:10:56.534 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.532+0000 7f0d4dd9b640 1 --2- 172.21.15.16:0/693005213 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0d50101c50 0x7f0d501982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:56.535 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.532+0000 7f0d4e59c640 1 --2- 172.21.15.16:0/693005213 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0d50100a60 0x7f0d50197db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:56.535 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.532+0000 7f0d4ed9d640 1 -- 172.21.15.16:0/693005213 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0d50100a60 msgr2=0x7f0d50197db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:56.535 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.532+0000 7f0d4ed9d640 1 --2- 172.21.15.16:0/693005213 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0d50100a60 0x7f0d50197db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:56.535 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.532+0000 7f0d4ed9d640 1 -- 172.21.15.16:0/693005213 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0d50101c50 msgr2=0x7f0d501982f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:56.535 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.532+0000 7f0d4ed9d640 1 --2- 172.21.15.16:0/693005213 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0d50101c50 0x7f0d501982f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:56.535 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.532+0000 7f0d4ed9d640 1 -- 172.21.15.16:0/693005213 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d5019f9d0 con 0x7f0d50102610 2024-09-12T02:10:56.535 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.532+0000 7f0d4dd9b640 1 --2- 172.21.15.16:0/693005213 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0d50101c50 0x7f0d501982f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:10:56.535 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.532+0000 7f0d4e59c640 1 --2- 172.21.15.16:0/693005213 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0d50100a60 0x7f0d50197db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:10:56.535 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.532+0000 7f0d4ed9d640 1 --2- 172.21.15.16:0/693005213 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d50102610 0x7f0d5019f360 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f0d48005e30 tx=0x7f0d48031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:10:56.535 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.532+0000 7f0d3f7fe640 1 -- 172.21.15.16:0/693005213 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d48003c30 con 0x7f0d50102610 2024-09-12T02:10:56.536 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.532+0000 7f0d3f7fe640 1 -- 172.21.15.16:0/693005213 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d48003dd0 con 0x7f0d50102610 2024-09-12T02:10:56.536 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.532+0000 7f0d54a77640 1 -- 172.21.15.16:0/693005213 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d5019fc60 con 0x7f0d50102610 2024-09-12T02:10:56.536 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.532+0000 7f0d3f7fe640 1 -- 172.21.15.16:0/693005213 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d4802fde0 con 0x7f0d50102610 2024-09-12T02:10:56.536 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.532+0000 7f0d54a77640 1 -- 172.21.15.16:0/693005213 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d501a01b0 con 0x7f0d50102610 2024-09-12T02:10:56.544 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.534+0000 7f0d3f7fe640 1 -- 172.21.15.16:0/693005213 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f0d48004050 con 0x7f0d50102610 2024-09-12T02:10:56.544 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.535+0000 7f0d5780a740 1 Processor -- start 2024-09-12T02:10:56.545 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.535+0000 7f0d5780a740 1 -- start start 2024-09-12T02:10:56.545 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.535+0000 7f0d5780a740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55863988d9a0 0x55863988dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:56.545 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.535+0000 7f0d5780a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55863988e2e0 0x558639892da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:56.545 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.535+0000 7f0d5780a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558639893370 0x558639895830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:56.545 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.535+0000 7f0d5780a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5586397f12f0 con 0x558639893370 2024-09-12T02:10:56.545 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.535+0000 7f0d5780a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x558639799270 con 0x55863988e2e0 2024-09-12T02:10:56.545 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.535+0000 7f0d5780a740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5586397ed2b0 con 0x55863988d9a0 2024-09-12T02:10:56.545 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.535+0000 7f0d4dd9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558639893370 0x558639895830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:56.545 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.535+0000 7f0d4dd9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558639893370 0x558639895830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39770/0 (socket says 172.21.15.16:39770) 2024-09-12T02:10:56.545 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.535+0000 7f0d4dd9b640 1 -- 172.21.15.16:0/548533554 learned_addr learned my addr 172.21.15.16:0/548533554 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:10:56.545 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.535+0000 7f0d3f7fe640 1 --2- 172.21.15.16:0/693005213 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f0d380a2c30 0x7f0d380a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:56.545 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.535+0000 7f0d4e59c640 1 --2- 172.21.15.16:0/548533554 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55863988e2e0 0x558639892da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:56.546 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.535+0000 7f0d4dd9b640 1 --2- 172.21.15.16:0/548533554 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55863988d9a0 0x55863988dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:56.546 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.536+0000 7f0d4ed9d640 1 --2- 172.21.15.16:0/693005213 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f0d380a2c30 0x7f0d380a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:56.546 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.536+0000 7f0d4dd9b640 1 -- 172.21.15.16:0/548533554 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55863988d9a0 msgr2=0x55863988dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:56.546 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.536+0000 7f0d3f7fe640 1 -- 172.21.15.16:0/693005213 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f0d480f0600 con 0x7f0d50102610 2024-09-12T02:10:56.546 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.536+0000 7f0d4dd9b640 1 --2- 172.21.15.16:0/548533554 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55863988d9a0 0x55863988dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:56.546 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.536+0000 7f0d4dd9b640 1 -- 172.21.15.16:0/548533554 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55863988e2e0 msgr2=0x558639892da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:56.546 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.536+0000 7f0d4dd9b640 1 --2- 172.21.15.16:0/548533554 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55863988e2e0 0x558639892da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:56.546 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.536+0000 7f0d4dd9b640 1 -- 172.21.15.16:0/548533554 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558639798340 con 0x558639893370 2024-09-12T02:10:56.546 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.536+0000 7f0d4dd9b640 1 --2- 172.21.15.16:0/548533554 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558639893370 0x558639895830 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f0d440149d0 tx=0x7f0d4400c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57c01803a9d25dc4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:10:56.546 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.536+0000 7f0d4ed9d640 1 --2- 172.21.15.16:0/693005213 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f0d380a2c30 0x7f0d380a50f0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f0d480b9950 tx=0x7f0d48088110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:10:56.547 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.536+0000 7f0d3cff9640 1 -- 172.21.15.16:0/548533554 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d440142d0 con 0x558639893370 2024-09-12T02:10:56.547 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.536+0000 7f0d3cff9640 1 -- 172.21.15.16:0/548533554 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d4400fc40 con 0x558639893370 2024-09-12T02:10:56.547 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.536+0000 7f0d3cff9640 1 -- 172.21.15.16:0/548533554 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d44012b00 con 0x558639893370 2024-09-12T02:10:56.547 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.537+0000 7f0d5780a740 1 -- 172.21.15.16:0/548533554 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558639893370 msgr2=0x558639895830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:56.547 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.537+0000 7f0d5780a740 1 --2- 172.21.15.16:0/548533554 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558639893370 0x558639895830 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f0d440149d0 tx=0x7f0d4400c4f0 comp rx=0 tx=0).stop 2024-09-12T02:10:56.547 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.537+0000 7f0d5780a740 1 -- 172.21.15.16:0/548533554 shutdown_connections 2024-09-12T02:10:56.547 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.537+0000 7f0d5780a740 1 --2- 172.21.15.16:0/548533554 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558639893370 0x558639895830 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:56.547 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.537+0000 7f0d5780a740 1 --2- 172.21.15.16:0/548533554 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55863988e2e0 0x558639892da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:56.547 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.537+0000 7f0d5780a740 1 --2- 172.21.15.16:0/548533554 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55863988d9a0 0x55863988dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:56.547 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.537+0000 7f0d5780a740 1 -- 172.21.15.16:0/548533554 >> 172.21.15.16:0/548533554 conn(0x558639889010 msgr2=0x558639889410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:10:56.547 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.537+0000 7f0d5780a740 1 -- 172.21.15.16:0/548533554 shutdown_connections 2024-09-12T02:10:56.547 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.537+0000 7f0d5780a740 1 -- 172.21.15.16:0/548533554 wait complete. 2024-09-12T02:10:56.548 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.537+0000 7f0d5780a740 1 -- start start 2024-09-12T02:10:56.548 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.537+0000 7f0d5780a740 1 Processor -- start 2024-09-12T02:10:56.548 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.538+0000 7f0d5780a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55863988d9a0 0x558639897c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:56.548 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.538+0000 7f0d5780a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55863988e2e0 0x558639898170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:56.548 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.538+0000 7f0d5780a740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558639893370 0x55863989f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:56.548 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.538+0000 7f0d5780a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5586398962e0 con 0x55863988d9a0 2024-09-12T02:10:56.548 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.538+0000 7f0d5780a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55863977a290 con 0x55863988e2e0 2024-09-12T02:10:56.548 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.538+0000 7f0d5780a740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x558639896460 con 0x558639893370 2024-09-12T02:10:56.548 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.538+0000 7f0d5780a740 10 client.0 _ll_register_callbacks cb 0x5586397ab140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:10:56.548 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.538+0000 7f0d4dd9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55863988d9a0 0x558639897c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:56.548 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.538+0000 7f0d4dd9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55863988d9a0 0x558639897c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39776/0 (socket says 172.21.15.16:39776) 2024-09-12T02:10:56.548 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.538+0000 7f0d4dd9b640 1 -- 172.21.15.16:0/2138583461 learned_addr learned my addr 172.21.15.16:0/2138583461 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:10:56.549 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.538+0000 7f0d4e59c640 1 --2- 172.21.15.16:0/2138583461 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55863988e2e0 0x558639898170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:56.549 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.538+0000 7f0d4dd9b640 1 -- 172.21.15.16:0/2138583461 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558639893370 msgr2=0x55863989f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:56.549 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.538+0000 7f0d4dd9b640 1 --2- 172.21.15.16:0/2138583461 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558639893370 0x55863989f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:56.549 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.538+0000 7f0d4dd9b640 1 -- 172.21.15.16:0/2138583461 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55863988e2e0 msgr2=0x558639898170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:56.549 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.538+0000 7f0d4dd9b640 1 --2- 172.21.15.16:0/2138583461 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55863988e2e0 0x558639898170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:56.549 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.538+0000 7f0d4dd9b640 1 -- 172.21.15.16:0/2138583461 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558639504210 con 0x55863988d9a0 2024-09-12T02:10:56.549 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.538+0000 7f0d4dd9b640 1 --2- 172.21.15.16:0/2138583461 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55863988d9a0 0x558639897c30 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f0d44005bc0 tx=0x7f0d4400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2de298edb7c26f84 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:10:56.549 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.538+0000 7f0d35ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:10:56.549 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.538+0000 7f0d35ffb640 1 -- 172.21.15.16:0/2138583461 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d4400f930 con 0x55863988d9a0 2024-09-12T02:10:56.549 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.538+0000 7f0d35ffb640 1 -- 172.21.15.16:0/2138583461 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f0d44005030 con 0x55863988d9a0 2024-09-12T02:10:56.549 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.538+0000 7f0d5780a740 1 -- 172.21.15.16:0/2138583461 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d4400fc40 con 0x55863988d9a0 2024-09-12T02:10:56.549 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.539+0000 7f0d35ffb640 1 -- 172.21.15.16:0/2138583461 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f0d44012450 con 0x55863988d9a0 2024-09-12T02:10:56.549 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.539+0000 7f0d5780a740 10 client.18159 fetch_fsmap learned FSMap version 25 2024-09-12T02:10:56.550 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.539+0000 7f0d5780a740 1 -- 172.21.15.16:0/2138583461 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5586395d0620 con 0x55863988d9a0 2024-09-12T02:10:56.550 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.539+0000 7f0d35ffb640 1 -- 172.21.15.16:0/2138583461 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d4400fc40 con 0x55863988d9a0 2024-09-12T02:10:56.550 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.539+0000 7f0d35ffb640 1 -- 172.21.15.16:0/2138583461 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f0d440046c0 con 0x55863988d9a0 2024-09-12T02:10:56.550 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.539+0000 7f0d5780a740 10 client.18159 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:10:56.550 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.539+0000 7f0d5780a740 10 client.18159 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:10:56.550 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.539+0000 7f0d5780a740 20 client.18159 populate_metadata read hostname 'smithi016' 2024-09-12T02:10:56.550 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.539+0000 7f0d5780a740 1 --2- 172.21.15.16:0/2138583461 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5586398a3710 0x5586398a5b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:56.550 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.539+0000 7f0d5780a740 4 client.18159 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:10:56.550 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.539+0000 7f0d5780a740 1 -- 172.21.15.16:0/2138583461 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5586397fea20 con 0x5586398a3710 2024-09-12T02:10:56.550 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.539+0000 7f0d4e59c640 1 --2- 172.21.15.16:0/2138583461 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5586398a3710 0x5586398a5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:56.550 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.540+0000 7f0d4e59c640 1 --2- 172.21.15.16:0/2138583461 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5586398a3710 0x5586398a5b50 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=851b1a29583adeed server_cookie=196a0ebf1e349d94 in_seq=0 out_seq=0 2024-09-12T02:10:56.550 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.540+0000 7f0d35ffb640 10 client.18159 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:10:56.550 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.540+0000 7f0d35ffb640 1 -- 172.21.15.16:0/2138583461 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5586397fea20 con 0x5586398a3710 2024-09-12T02:10:56.551 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.540+0000 7f0d35ffb640 10 client.18159 handle_command_reply: tid=0 2024-09-12T02:10:56.551 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.540+0000 7f0d5780a740 1 client.18159 shutdown 2024-09-12T02:10:56.551 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.541+0000 7f0d5780a740 1 -- 172.21.15.16:0/2138583461 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55863988d9a0 msgr2=0x558639897c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:56.551 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.541+0000 7f0d5780a740 1 --2- 172.21.15.16:0/2138583461 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55863988d9a0 0x558639897c30 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f0d44005bc0 tx=0x7f0d4400cd50 comp rx=0 tx=0).stop 2024-09-12T02:10:56.551 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.541+0000 7f0d5780a740 1 -- 172.21.15.16:0/2138583461 shutdown_connections 2024-09-12T02:10:56.551 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.541+0000 7f0d5780a740 1 --2- 172.21.15.16:0/2138583461 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5586398a3710 0x5586398a5b50 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:56.551 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.541+0000 7f0d5780a740 1 --2- 172.21.15.16:0/2138583461 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558639893370 0x55863989f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:56.551 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.541+0000 7f0d35ffb640 0 client.18159 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:10:56.551 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.541+0000 7f0d5780a740 1 --2- 172.21.15.16:0/2138583461 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55863988e2e0 0x558639898170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:56.551 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.541+0000 7f0d5780a740 1 --2- 172.21.15.16:0/2138583461 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55863988d9a0 0x558639897c30 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:56.551 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.541+0000 7f0d5780a740 1 -- 172.21.15.16:0/2138583461 >> 172.21.15.16:0/2138583461 conn(0x558639889010 msgr2=0x55863988a820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:10:56.551 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.541+0000 7f0d5780a740 1 -- 172.21.15.16:0/2138583461 shutdown_connections 2024-09-12T02:10:56.552 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.541+0000 7f0d5780a740 1 -- 172.21.15.16:0/2138583461 wait complete. 2024-09-12T02:10:56.552 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.541+0000 7f0d5780a740 20 client.18159 trim_cache size 0 max 16384 2024-09-12T02:10:56.561 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.555+0000 7f0d5780a740 1 Processor -- start 2024-09-12T02:10:56.561 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.555+0000 7f0d5780a740 1 -- start start 2024-09-12T02:10:56.561 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.555+0000 7f0d5780a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55863988d9a0 0x558639898200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:56.561 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.555+0000 7f0d5780a740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55863988e2e0 0x55863989f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:56.561 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.555+0000 7f0d5780a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558639893370 0x5586398880b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:56.561 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.555+0000 7f0d5780a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0d501ab6d0 con 0x55863988d9a0 2024-09-12T02:10:56.561 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.555+0000 7f0d5780a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0d501a8fa0 con 0x558639893370 2024-09-12T02:10:56.562 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.555+0000 7f0d5780a740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f0d501a6df0 con 0x55863988e2e0 2024-09-12T02:10:56.562 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.555+0000 7f0d4dd9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55863988d9a0 0x558639898200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:56.562 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.555+0000 7f0d4dd9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55863988d9a0 0x558639898200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39786/0 (socket says 172.21.15.16:39786) 2024-09-12T02:10:56.562 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.555+0000 7f0d4dd9b640 1 -- 172.21.15.16:0/1075496265 learned_addr learned my addr 172.21.15.16:0/1075496265 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:10:56.562 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.555+0000 7f0d4e59c640 1 --2- 172.21.15.16:0/1075496265 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55863988e2e0 0x55863989f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:56.562 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.556+0000 7f0d4dd9b640 1 --2- 172.21.15.16:0/1075496265 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558639893370 0x5586398880b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:56.562 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.556+0000 7f0d4dd9b640 1 -- 172.21.15.16:0/1075496265 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55863988e2e0 msgr2=0x55863989f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:56.562 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.556+0000 7f0d4dd9b640 1 --2- 172.21.15.16:0/1075496265 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55863988e2e0 0x55863989f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:56.562 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.556+0000 7f0d4dd9b640 1 -- 172.21.15.16:0/1075496265 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558639893370 msgr2=0x5586398880b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:56.562 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.556+0000 7f0d4dd9b640 1 --2- 172.21.15.16:0/1075496265 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558639893370 0x5586398880b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:56.562 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.556+0000 7f0d4dd9b640 1 -- 172.21.15.16:0/1075496265 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558639897770 con 0x55863988d9a0 2024-09-12T02:10:56.562 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.556+0000 7f0d4dd9b640 1 --2- 172.21.15.16:0/1075496265 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55863988d9a0 0x558639898200 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f0d4400cb10 tx=0x7f0d4400c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e599d4e8167b99 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:10:56.562 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.556+0000 7f0d37fff640 1 -- 172.21.15.16:0/1075496265 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d44004680 con 0x55863988d9a0 2024-09-12T02:10:56.563 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.556+0000 7f0d37fff640 1 -- 172.21.15.16:0/1075496265 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d44004820 con 0x55863988d9a0 2024-09-12T02:10:56.563 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.556+0000 7f0d37fff640 1 -- 172.21.15.16:0/1075496265 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d44004cd0 con 0x55863988d9a0 2024-09-12T02:10:56.563 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.557+0000 7f0d5780a740 1 -- 172.21.15.16:0/1075496265 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55863988d9a0 msgr2=0x558639898200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:56.563 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.557+0000 7f0d5780a740 1 --2- 172.21.15.16:0/1075496265 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55863988d9a0 0x558639898200 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f0d4400cb10 tx=0x7f0d4400c970 comp rx=0 tx=0).stop 2024-09-12T02:10:56.563 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.557+0000 7f0d5780a740 1 -- 172.21.15.16:0/1075496265 shutdown_connections 2024-09-12T02:10:56.563 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.557+0000 7f0d5780a740 1 --2- 172.21.15.16:0/1075496265 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558639893370 0x5586398880b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:56.563 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.557+0000 7f0d5780a740 1 --2- 172.21.15.16:0/1075496265 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55863988e2e0 0x55863989f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:56.563 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.557+0000 7f0d5780a740 1 --2- 172.21.15.16:0/1075496265 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55863988d9a0 0x558639898200 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:56.563 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.557+0000 7f0d5780a740 1 -- 172.21.15.16:0/1075496265 >> 172.21.15.16:0/1075496265 conn(0x558639889010 msgr2=0x5586398879a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:10:56.563 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.557+0000 7f0d5780a740 1 -- 172.21.15.16:0/1075496265 shutdown_connections 2024-09-12T02:10:56.563 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.557+0000 7f0d5780a740 1 -- 172.21.15.16:0/1075496265 wait complete. 2024-09-12T02:10:56.564 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.557+0000 7f0d5780a740 1 -- start start 2024-09-12T02:10:56.564 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.557+0000 7f0d5780a740 1 Processor -- start 2024-09-12T02:10:56.564 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.557+0000 7f0d5780a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55863988d9a0 0x55863989c900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:56.564 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.557+0000 7f0d5780a740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55863988e2e0 0x55863989ce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:56.564 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.557+0000 7f0d5780a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558639893370 0x55863989de00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:56.564 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.557+0000 7f0d5780a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0d501a7460 con 0x558639893370 2024-09-12T02:10:56.564 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.557+0000 7f0d5780a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0d501a88d0 con 0x55863988d9a0 2024-09-12T02:10:56.564 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.557+0000 7f0d5780a740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f0d501a7b30 con 0x55863988e2e0 2024-09-12T02:10:56.564 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.557+0000 7f0d5780a740 10 client.0 _ll_register_callbacks cb 0x7f0d40009400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:10:56.565 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.557+0000 7f0d4dd9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558639893370 0x55863989de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:56.565 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.558+0000 7f0d4dd9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558639893370 0x55863989de00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39788/0 (socket says 172.21.15.16:39788) 2024-09-12T02:10:56.565 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.558+0000 7f0d4dd9b640 1 -- 172.21.15.16:0/1219907883 learned_addr learned my addr 172.21.15.16:0/1219907883 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:10:56.565 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.558+0000 7f0d4e59c640 1 --2- 172.21.15.16:0/1219907883 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55863988e2e0 0x55863989ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:56.566 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.558+0000 7f0d4dd9b640 1 --2- 172.21.15.16:0/1219907883 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55863988d9a0 0x55863989c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:56.566 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.558+0000 7f0d4e59c640 1 -- 172.21.15.16:0/1219907883 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55863988d9a0 msgr2=0x55863989c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:56.566 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.558+0000 7f0d4e59c640 1 --2- 172.21.15.16:0/1219907883 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55863988d9a0 0x55863989c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:56.566 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.558+0000 7f0d4e59c640 1 -- 172.21.15.16:0/1219907883 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558639893370 msgr2=0x55863989de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:56.566 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.558+0000 7f0d4e59c640 1 --2- 172.21.15.16:0/1219907883 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558639893370 0x55863989de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:56.566 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.558+0000 7f0d4e59c640 1 -- 172.21.15.16:0/1219907883 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55863989e410 con 0x55863988e2e0 2024-09-12T02:10:56.566 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.558+0000 7f0d4e59c640 1 --2- 172.21.15.16:0/1219907883 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55863988e2e0 0x55863989ce40 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x558639889e90 tx=0x7f0d4001e9e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9700d5c393a33b72 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:10:56.566 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.558+0000 7f0d35ffb640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-09-12T02:10:56.566 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.558+0000 7f0d35ffb640 1 -- 172.21.15.16:0/1219907883 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d40005220 con 0x55863988e2e0 2024-09-12T02:10:56.567 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.558+0000 7f0d5780a740 1 -- 172.21.15.16:0/1219907883 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d44004820 con 0x55863988e2e0 2024-09-12T02:10:56.567 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.559+0000 7f0d35ffb640 1 -- 172.21.15.16:0/1219907883 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f0d400053c0 con 0x55863988e2e0 2024-09-12T02:10:56.567 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.559+0000 7f0d35ffb640 1 -- 172.21.15.16:0/1219907883 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f0d400055c0 con 0x55863988e2e0 2024-09-12T02:10:56.567 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.559+0000 7f0d5780a740 10 client.27151 fetch_fsmap learned FSMap version 25 2024-09-12T02:10:56.567 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.559+0000 7f0d5780a740 1 -- 172.21.15.16:0/1219907883 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55863990e5b0 con 0x55863988e2e0 2024-09-12T02:10:56.567 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.559+0000 7f0d35ffb640 1 -- 172.21.15.16:0/1219907883 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d44004820 con 0x55863988e2e0 2024-09-12T02:10:56.567 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.559+0000 7f0d35ffb640 1 -- 172.21.15.16:0/1219907883 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f0d4002f050 con 0x55863988e2e0 2024-09-12T02:10:56.567 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.559+0000 7f0d5780a740 10 client.27151 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:10:56.567 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.559+0000 7f0d5780a740 10 client.27151 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:10:56.567 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.559+0000 7f0d5780a740 20 client.27151 populate_metadata read hostname 'smithi016' 2024-09-12T02:10:56.568 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.559+0000 7f0d5780a740 1 --2- 172.21.15.16:0/1219907883 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55863990eee0 0x558639911300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:56.568 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.559+0000 7f0d5780a740 4 client.27151 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:10:56.568 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.559+0000 7f0d5780a740 1 -- 172.21.15.16:0/1219907883 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5586397f8d70 con 0x55863990eee0 2024-09-12T02:10:56.568 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.560+0000 7f0d4dd9b640 1 --2- 172.21.15.16:0/1219907883 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55863990eee0 0x558639911300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:56.568 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.560+0000 7f0d4dd9b640 1 --2- 172.21.15.16:0/1219907883 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55863990eee0 0x558639911300 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85d7eec6af9cedb6 server_cookie=2b4383c18df0ca1c in_seq=0 out_seq=0 2024-09-12T02:10:56.568 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.560+0000 7f0d35ffb640 10 client.27151 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:10:56.574 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.573+0000 7f0d35ffb640 1 -- 172.21.15.16:0/1219907883 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5586397f8d70 con 0x55863990eee0 2024-09-12T02:10:56.574 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.573+0000 7f0d35ffb640 10 client.27151 handle_command_reply: tid=0 2024-09-12T02:10:56.574 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.573+0000 7f0d5780a740 1 client.27151 shutdown 2024-09-12T02:10:56.574 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.573+0000 7f0d5780a740 1 -- 172.21.15.16:0/1219907883 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55863988e2e0 msgr2=0x55863989ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:56.574 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.573+0000 7f0d5780a740 1 --2- 172.21.15.16:0/1219907883 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55863988e2e0 0x55863989ce40 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x558639889e90 tx=0x7f0d4001e9e0 comp rx=0 tx=0).stop 2024-09-12T02:10:56.574 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.573+0000 7f0d5780a740 1 -- 172.21.15.16:0/1219907883 shutdown_connections 2024-09-12T02:10:56.574 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.573+0000 7f0d5780a740 1 --2- 172.21.15.16:0/1219907883 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55863990eee0 0x558639911300 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:56.574 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.573+0000 7f0d5780a740 1 --2- 172.21.15.16:0/1219907883 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558639893370 0x55863989de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:56.574 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.573+0000 7f0d5780a740 1 --2- 172.21.15.16:0/1219907883 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55863988e2e0 0x55863989ce40 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:56.575 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.573+0000 7f0d35ffb640 0 client.27151 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:10:56.575 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.573+0000 7f0d5780a740 1 --2- 172.21.15.16:0/1219907883 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55863988d9a0 0x55863989c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:56.575 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.573+0000 7f0d5780a740 1 -- 172.21.15.16:0/1219907883 >> 172.21.15.16:0/1219907883 conn(0x558639889010 msgr2=0x5586398881c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:10:56.575 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.573+0000 7f0d5780a740 1 -- 172.21.15.16:0/1219907883 shutdown_connections 2024-09-12T02:10:56.575 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.573+0000 7f0d5780a740 1 -- 172.21.15.16:0/1219907883 wait complete. 2024-09-12T02:10:56.575 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.573+0000 7f0d5780a740 20 client.27151 trim_cache size 0 max 16384 2024-09-12T02:10:56.575 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-09-12T02:10:56.575 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-09-12T02:10:56.575 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-09-12T02:10:56.576 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-09-12T02:10:56.576 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.574+0000 7f0d54a77640 1 -- 172.21.15.16:0/693005213 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f0d380a2c30 msgr2=0x7f0d380a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:56.577 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.574+0000 7f0d54a77640 1 --2- 172.21.15.16:0/693005213 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f0d380a2c30 0x7f0d380a50f0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f0d480b9950 tx=0x7f0d48088110 comp rx=0 tx=0).stop 2024-09-12T02:10:56.577 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.574+0000 7f0d54a77640 1 -- 172.21.15.16:0/693005213 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d50102610 msgr2=0x7f0d5019f360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:56.577 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.574+0000 7f0d54a77640 1 --2- 172.21.15.16:0/693005213 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d50102610 0x7f0d5019f360 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f0d48005e30 tx=0x7f0d48031ee0 comp rx=0 tx=0).stop 2024-09-12T02:10:56.577 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.574+0000 7f0d54a77640 1 -- 172.21.15.16:0/693005213 shutdown_connections 2024-09-12T02:10:56.577 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.574+0000 7f0d54a77640 1 --2- 172.21.15.16:0/693005213 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f0d380a2c30 0x7f0d380a50f0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:56.577 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.574+0000 7f0d54a77640 1 --2- 172.21.15.16:0/693005213 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d50102610 0x7f0d5019f360 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:56.577 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.574+0000 7f0d54a77640 1 --2- 172.21.15.16:0/693005213 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0d50101c50 0x7f0d501982f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:56.577 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.574+0000 7f0d54a77640 1 --2- 172.21.15.16:0/693005213 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0d50100a60 0x7f0d50197db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:56.577 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.574+0000 7f0d54a77640 1 -- 172.21.15.16:0/693005213 >> 172.21.15.16:0/693005213 conn(0x7f0d500fc1e0 msgr2=0x7f0d50104fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:10:56.577 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.574+0000 7f0d54a77640 1 -- 172.21.15.16:0/693005213 shutdown_connections 2024-09-12T02:10:56.577 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:10:56.574+0000 7f0d54a77640 1 -- 172.21.15.16:0/693005213 wait complete. 2024-09-12T02:10:56.922 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-12T02:10:57.181 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:10:57.183 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:10:56 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/461785770' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:10:57.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:10:56 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/461785770' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:10:57.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:10:56 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/461785770' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:10:57.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.766+0000 7f3af81be640 1 -- 172.21.15.16:0/1399360550 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3af0103720 msgr2=0x7f3af0107b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:57.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.766+0000 7f3af81be640 1 --2- 172.21.15.16:0/1399360550 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3af0103720 0x7f3af0107b60 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f3ae4009930 tx=0x7f3ae402a420 comp rx=0 tx=0).stop 2024-09-12T02:10:57.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.766+0000 7f3af81be640 1 -- 172.21.15.16:0/1399360550 shutdown_connections 2024-09-12T02:10:57.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.766+0000 7f3af81be640 1 --2- 172.21.15.16:0/1399360550 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3af01082a0 0x7f3af010ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:57.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.766+0000 7f3af81be640 1 --2- 172.21.15.16:0/1399360550 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3af0103720 0x7f3af0107b60 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:57.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.766+0000 7f3af81be640 1 --2- 172.21.15.16:0/1399360550 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3af0102d70 0x7f3af0103150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:57.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.766+0000 7f3af81be640 1 -- 172.21.15.16:0/1399360550 >> 172.21.15.16:0/1399360550 conn(0x7f3af00fc7d0 msgr2=0x7f3af00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:10:57.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.766+0000 7f3af81be640 1 -- 172.21.15.16:0/1399360550 shutdown_connections 2024-09-12T02:10:57.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.766+0000 7f3af81be640 1 -- 172.21.15.16:0/1399360550 wait complete. 2024-09-12T02:10:57.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.766+0000 7f3af81be640 1 Processor -- start 2024-09-12T02:10:57.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.766+0000 7f3af81be640 1 -- start start 2024-09-12T02:10:57.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.767+0000 7f3af81be640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3af0102d70 0x7f3af019a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:57.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.767+0000 7f3af81be640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3af0103720 0x7f3af019ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:57.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.767+0000 7f3af81be640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3af01082a0 0x7f3af019f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:57.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.767+0000 7f3af81be640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3af0112040 con 0x7f3af0103720 2024-09-12T02:10:57.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.767+0000 7f3af81be640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3af0111ec0 con 0x7f3af01082a0 2024-09-12T02:10:57.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.767+0000 7f3af81be640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f3af01121c0 con 0x7f3af0102d70 2024-09-12T02:10:57.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.767+0000 7f3af5759640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3af0103720 0x7f3af019ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:57.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.767+0000 7f3af5759640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3af0103720 0x7f3af019ad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39806/0 (socket says 172.21.15.16:39806) 2024-09-12T02:10:57.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.767+0000 7f3af5759640 1 -- 172.21.15.16:0/4162060307 learned_addr learned my addr 172.21.15.16:0/4162060307 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:10:57.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.767+0000 7f3af5f5a640 1 --2- 172.21.15.16:0/4162060307 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3af0102d70 0x7f3af019a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:57.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.767+0000 7f3af675b640 1 --2- 172.21.15.16:0/4162060307 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3af01082a0 0x7f3af019f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:57.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.767+0000 7f3af5759640 1 -- 172.21.15.16:0/4162060307 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3af0102d70 msgr2=0x7f3af019a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:57.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.767+0000 7f3af5759640 1 --2- 172.21.15.16:0/4162060307 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3af0102d70 0x7f3af019a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:57.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.767+0000 7f3af5759640 1 -- 172.21.15.16:0/4162060307 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3af01082a0 msgr2=0x7f3af019f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:57.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.767+0000 7f3af5759640 1 --2- 172.21.15.16:0/4162060307 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3af01082a0 0x7f3af019f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:57.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.767+0000 7f3af5759640 1 -- 172.21.15.16:0/4162060307 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3af019f6e0 con 0x7f3af0103720 2024-09-12T02:10:57.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.767+0000 7f3af5759640 1 --2- 172.21.15.16:0/4162060307 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3af0103720 0x7f3af019ad10 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f3ae4005940 tx=0x7f3ae40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:10:57.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.768+0000 7f3aeeffd640 1 -- 172.21.15.16:0/4162060307 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae403f030 con 0x7f3af0103720 2024-09-12T02:10:57.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.768+0000 7f3af81be640 1 -- 172.21.15.16:0/4162060307 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3af019f970 con 0x7f3af0103720 2024-09-12T02:10:57.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.768+0000 7f3aeeffd640 1 -- 172.21.15.16:0/4162060307 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ae4003ce0 con 0x7f3af0103720 2024-09-12T02:10:57.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.768+0000 7f3aeeffd640 1 -- 172.21.15.16:0/4162060307 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae4035070 con 0x7f3af0103720 2024-09-12T02:10:57.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.768+0000 7f3af81be640 1 -- 172.21.15.16:0/4162060307 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3af01a72b0 con 0x7f3af0103720 2024-09-12T02:10:57.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.770+0000 7f3aeeffd640 1 -- 172.21.15.16:0/4162060307 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f3ae4042030 con 0x7f3af0103720 2024-09-12T02:10:57.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.770+0000 7f3afa597740 1 Processor -- start 2024-09-12T02:10:57.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.771+0000 7f3afa597740 1 -- start start 2024-09-12T02:10:57.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.771+0000 7f3aeeffd640 1 --2- 172.21.15.16:0/4162060307 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f3ad80a2b80 0x7f3ad80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:57.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.771+0000 7f3af675b640 1 --2- 172.21.15.16:0/4162060307 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f3ad80a2b80 0x7f3ad80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:57.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.771+0000 7f3aeeffd640 1 -- 172.21.15.16:0/4162060307 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f3ae40ef3a0 con 0x7f3af0103720 2024-09-12T02:10:57.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.771+0000 7f3afa597740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a5e971cb40 0x55a5e971b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:57.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.771+0000 7f3afa597740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a5e971b5d0 0x55a5e971ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:57.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.771+0000 7f3afa597740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a5e9726370 0x55a5e9728760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:57.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.771+0000 7f3afa597740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55a5e96812f0 con 0x55a5e9726370 2024-09-12T02:10:57.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.771+0000 7f3afa597740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55a5e9628cf0 con 0x55a5e971cb40 2024-09-12T02:10:57.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.771+0000 7f3afa597740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55a5e967d2b0 con 0x55a5e971b5d0 2024-09-12T02:10:57.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.771+0000 7f3af675b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a5e9726370 0x55a5e9728760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:57.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.771+0000 7f3af5f5a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a5e971cb40 0x55a5e971b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:57.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.771+0000 7f3af675b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a5e9726370 0x55a5e9728760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39812/0 (socket says 172.21.15.16:39812) 2024-09-12T02:10:57.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.771+0000 7f3af675b640 1 -- 172.21.15.16:0/1644541578 learned_addr learned my addr 172.21.15.16:0/1644541578 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:10:57.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.771+0000 7f3af5f5a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a5e971cb40 0x55a5e971b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.16:49330/0 (socket says 172.21.15.16:49330) 2024-09-12T02:10:57.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.771+0000 7f3af5759640 1 --2- 172.21.15.16:0/1644541578 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a5e971b5d0 0x55a5e971ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:57.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.772+0000 7f3af5f5a640 1 -- 172.21.15.16:0/1644541578 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a5e971b5d0 msgr2=0x55a5e971ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:57.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.772+0000 7f3af5f5a640 1 --2- 172.21.15.16:0/1644541578 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a5e971b5d0 0x55a5e971ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:57.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.772+0000 7f3af5f5a640 1 -- 172.21.15.16:0/1644541578 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a5e9726370 msgr2=0x55a5e9728760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:57.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.772+0000 7f3af5f5a640 1 --2- 172.21.15.16:0/1644541578 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a5e9726370 0x55a5e9728760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:57.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.772+0000 7f3af675b640 1 --2- 172.21.15.16:0/4162060307 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f3ad80a2b80 0x7f3ad80a5040 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f3ae800bab0 tx=0x7f3ae800beb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:10:57.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.772+0000 7f3af5f5a640 1 -- 172.21.15.16:0/1644541578 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5e960d3a0 con 0x55a5e971cb40 2024-09-12T02:10:57.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.772+0000 7f3af5759640 1 --2- 172.21.15.16:0/1644541578 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a5e971b5d0 0x55a5e971ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:10:57.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.772+0000 7f3af5f5a640 1 --2- 172.21.15.16:0/1644541578 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a5e971cb40 0x55a5e971b090 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f3ae0007ed0 tx=0x7f3ae000f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f40237bdab524bd server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:10:57.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.772+0000 7f3ad7fff640 1 -- 172.21.15.16:0/1644541578 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae0004d20 con 0x55a5e971cb40 2024-09-12T02:10:57.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.772+0000 7f3ad7fff640 1 -- 172.21.15.16:0/1644541578 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ae0010b20 con 0x55a5e971cb40 2024-09-12T02:10:57.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.772+0000 7f3ad7fff640 1 -- 172.21.15.16:0/1644541578 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae0005720 con 0x55a5e971cb40 2024-09-12T02:10:57.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.773+0000 7f3afa597740 1 -- 172.21.15.16:0/1644541578 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a5e971cb40 msgr2=0x55a5e971b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:57.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.773+0000 7f3afa597740 1 --2- 172.21.15.16:0/1644541578 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a5e971cb40 0x55a5e971b090 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f3ae0007ed0 tx=0x7f3ae000f790 comp rx=0 tx=0).stop 2024-09-12T02:10:57.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.773+0000 7f3afa597740 1 -- 172.21.15.16:0/1644541578 shutdown_connections 2024-09-12T02:10:57.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.773+0000 7f3afa597740 1 --2- 172.21.15.16:0/1644541578 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a5e9726370 0x55a5e9728760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:57.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.773+0000 7f3afa597740 1 --2- 172.21.15.16:0/1644541578 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a5e971b5d0 0x55a5e971ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:57.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.773+0000 7f3afa597740 1 --2- 172.21.15.16:0/1644541578 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a5e971cb40 0x55a5e971b090 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:57.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.773+0000 7f3afa597740 1 -- 172.21.15.16:0/1644541578 >> 172.21.15.16:0/1644541578 conn(0x55a5e96e4310 msgr2=0x55a5e9717c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:10:57.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.773+0000 7f3afa597740 1 -- 172.21.15.16:0/1644541578 shutdown_connections 2024-09-12T02:10:57.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.773+0000 7f3afa597740 1 -- 172.21.15.16:0/1644541578 wait complete. 2024-09-12T02:10:57.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.773+0000 7f3afa597740 1 -- start start 2024-09-12T02:10:57.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.773+0000 7f3afa597740 1 Processor -- start 2024-09-12T02:10:57.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.774+0000 7f3afa597740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a5e971b5d0 0x55a5e9725cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:57.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.774+0000 7f3afa597740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a5e971cb40 0x55a5e9722f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:57.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.774+0000 7f3afa597740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a5e9726370 0x55a5e97234a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:57.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.774+0000 7f3afa597740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55a5e9729070 con 0x55a5e971b5d0 2024-09-12T02:10:57.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.774+0000 7f3afa597740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55a5e963c060 con 0x55a5e971cb40 2024-09-12T02:10:57.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.774+0000 7f3afa597740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55a5e97291f0 con 0x55a5e9726370 2024-09-12T02:10:57.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.774+0000 7f3afa597740 10 client.0 _ll_register_callbacks cb 0x55a5e938f8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:10:57.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.774+0000 7f3af5f5a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a5e971b5d0 0x55a5e9725cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:57.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.774+0000 7f3af5f5a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a5e971b5d0 0x55a5e9725cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39822/0 (socket says 172.21.15.16:39822) 2024-09-12T02:10:57.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.774+0000 7f3af5f5a640 1 -- 172.21.15.16:0/803559249 learned_addr learned my addr 172.21.15.16:0/803559249 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:10:57.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.774+0000 7f3af5759640 1 --2- 172.21.15.16:0/803559249 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a5e971cb40 0x55a5e9722f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:57.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.774+0000 7f3af675b640 1 --2- 172.21.15.16:0/803559249 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a5e9726370 0x55a5e97234a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:57.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.774+0000 7f3af5f5a640 1 -- 172.21.15.16:0/803559249 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a5e9726370 msgr2=0x55a5e97234a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:57.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.774+0000 7f3af5f5a640 1 --2- 172.21.15.16:0/803559249 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a5e9726370 0x55a5e97234a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:57.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.774+0000 7f3af5f5a640 1 -- 172.21.15.16:0/803559249 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a5e971cb40 msgr2=0x55a5e9722f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:57.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.774+0000 7f3af5f5a640 1 --2- 172.21.15.16:0/803559249 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a5e971cb40 0x55a5e9722f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:57.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.774+0000 7f3af5f5a640 1 -- 172.21.15.16:0/803559249 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5e93f23c0 con 0x55a5e971b5d0 2024-09-12T02:10:57.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.775+0000 7f3af5f5a640 1 --2- 172.21.15.16:0/803559249 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a5e971b5d0 0x55a5e9725cb0 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f3ae0005400 tx=0x7f3ae00106c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c50c41033ad2ae7a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:10:57.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.775+0000 7f3ad57fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:10:57.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.775+0000 7f3ad57fa640 1 -- 172.21.15.16:0/803559249 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae001d040 con 0x55a5e971b5d0 2024-09-12T02:10:57.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.775+0000 7f3afa597740 1 -- 172.21.15.16:0/803559249 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ae0010b20 con 0x55a5e971b5d0 2024-09-12T02:10:57.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.775+0000 7f3ad57fa640 1 -- 172.21.15.16:0/803559249 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f3ae0004c40 con 0x55a5e971b5d0 2024-09-12T02:10:57.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.775+0000 7f3ad57fa640 1 -- 172.21.15.16:0/803559249 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f3ae0010970 con 0x55a5e971b5d0 2024-09-12T02:10:57.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.775+0000 7f3afa597740 10 client.18177 fetch_fsmap learned FSMap version 25 2024-09-12T02:10:57.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.775+0000 7f3afa597740 1 -- 172.21.15.16:0/803559249 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a5e9734a60 con 0x55a5e971b5d0 2024-09-12T02:10:57.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.775+0000 7f3ad57fa640 1 -- 172.21.15.16:0/803559249 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ae0010b20 con 0x55a5e971b5d0 2024-09-12T02:10:57.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.775+0000 7f3ad57fa640 1 -- 172.21.15.16:0/803559249 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f3ae0022890 con 0x55a5e971b5d0 2024-09-12T02:10:57.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.775+0000 7f3afa597740 10 client.18177 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:10:57.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.775+0000 7f3afa597740 10 client.18177 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-12T02:10:57.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.775+0000 7f3afa597740 20 client.18177 populate_metadata read hostname 'smithi016' 2024-09-12T02:10:57.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.775+0000 7f3afa597740 1 --2- 172.21.15.16:0/803559249 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55a5e9735e60 0x55a5e9738250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:57.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.775+0000 7f3afa597740 4 client.18177 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:10:57.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.775+0000 7f3afa597740 1 -- 172.21.15.16:0/803559249 --> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a5e968e860 con 0x55a5e9735e60 2024-09-12T02:10:57.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.776+0000 7f3af5759640 1 --2- 172.21.15.16:0/803559249 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55a5e9735e60 0x55a5e9738250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:57.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.776+0000 7f3af5759640 1 --2- 172.21.15.16:0/803559249 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55a5e9735e60 0x55a5e9738250 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cf80156a85ff575 server_cookie=c460e3edd8a7c513 in_seq=0 out_seq=0 2024-09-12T02:10:57.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.776+0000 7f3ad57fa640 10 client.18177 ms_handle_connect on v2:172.21.15.184:6836/1793393087 2024-09-12T02:10:57.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.777+0000 7f3ad57fa640 1 -- 172.21.15.16:0/803559249 <== mds.2 v2:172.21.15.184:6836/1793393087 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a5e968e860 con 0x55a5e9735e60 2024-09-12T02:10:57.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.777+0000 7f3ad57fa640 10 client.18177 handle_command_reply: tid=0 2024-09-12T02:10:57.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.777+0000 7f3afa597740 1 client.18177 shutdown 2024-09-12T02:10:57.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.777+0000 7f3afa597740 1 -- 172.21.15.16:0/803559249 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a5e971b5d0 msgr2=0x55a5e9725cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:57.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.777+0000 7f3afa597740 1 --2- 172.21.15.16:0/803559249 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a5e971b5d0 0x55a5e9725cb0 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f3ae0005400 tx=0x7f3ae00106c0 comp rx=0 tx=0).stop 2024-09-12T02:10:57.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.777+0000 7f3afa597740 1 -- 172.21.15.16:0/803559249 shutdown_connections 2024-09-12T02:10:57.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.777+0000 7f3afa597740 1 --2- 172.21.15.16:0/803559249 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a5e9726370 0x55a5e97234a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:57.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.777+0000 7f3afa597740 1 --2- 172.21.15.16:0/803559249 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a5e971cb40 0x55a5e9722f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:57.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.777+0000 7f3afa597740 1 --2- 172.21.15.16:0/803559249 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55a5e9735e60 0x55a5e9738250 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:57.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.777+0000 7f3afa597740 1 --2- 172.21.15.16:0/803559249 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a5e971b5d0 0x55a5e9725cb0 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:57.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.777+0000 7f3ad57fa640 0 client.18177 ms_handle_reset on v2:172.21.15.184:6836/1793393087 2024-09-12T02:10:57.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.777+0000 7f3afa597740 1 -- 172.21.15.16:0/803559249 >> 172.21.15.16:0/803559249 conn(0x55a5e96e4310 msgr2=0x55a5e971d960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:10:57.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.777+0000 7f3afa597740 1 -- 172.21.15.16:0/803559249 shutdown_connections 2024-09-12T02:10:57.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.777+0000 7f3afa597740 1 -- 172.21.15.16:0/803559249 wait complete. 2024-09-12T02:10:57.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.777+0000 7f3afa597740 20 client.18177 trim_cache size 0 max 16384 2024-09-12T02:10:57.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.791+0000 7f3afa597740 1 Processor -- start 2024-09-12T02:10:57.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.791+0000 7f3afa597740 1 -- start start 2024-09-12T02:10:57.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.792+0000 7f3afa597740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a5e971b5d0 0x55a5e9732310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:57.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.792+0000 7f3afa597740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a5e971cb40 0x55a5e973d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:57.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.792+0000 7f3afa597740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a5e9726370 0x55a5e973ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:57.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.792+0000 7f3afa597740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3af01b1f50 con 0x55a5e9726370 2024-09-12T02:10:57.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.792+0000 7f3afa597740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3af01af820 con 0x55a5e971cb40 2024-09-12T02:10:57.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.792+0000 7f3afa597740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f3af01ad670 con 0x55a5e971b5d0 2024-09-12T02:10:57.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.792+0000 7f3af675b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a5e9726370 0x55a5e973ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:57.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.792+0000 7f3af675b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a5e9726370 0x55a5e973ddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39826/0 (socket says 172.21.15.16:39826) 2024-09-12T02:10:57.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.792+0000 7f3af5f5a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a5e971b5d0 0x55a5e9732310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:57.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.792+0000 7f3af675b640 1 -- 172.21.15.16:0/3551852410 learned_addr learned my addr 172.21.15.16:0/3551852410 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:10:57.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.792+0000 7f3af5759640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a5e971cb40 0x55a5e973d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:57.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.792+0000 7f3af675b640 1 -- 172.21.15.16:0/3551852410 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a5e971b5d0 msgr2=0x55a5e9732310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:57.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.792+0000 7f3af675b640 1 --2- 172.21.15.16:0/3551852410 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a5e971b5d0 0x55a5e9732310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:57.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.792+0000 7f3af675b640 1 -- 172.21.15.16:0/3551852410 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a5e971cb40 msgr2=0x55a5e973d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:57.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.792+0000 7f3af675b640 1 --2- 172.21.15.16:0/3551852410 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a5e971cb40 0x55a5e973d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:57.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.792+0000 7f3af675b640 1 -- 172.21.15.16:0/3551852410 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5e9718960 con 0x55a5e9726370 2024-09-12T02:10:57.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.793+0000 7f3af675b640 1 --2- 172.21.15.16:0/3551852410 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a5e9726370 0x55a5e973ddf0 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f3ae8003fd0 tx=0x7f3ae801d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=267a11dd966e2d4b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:10:57.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.793+0000 7f3ad77fe640 1 -- 172.21.15.16:0/3551852410 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae8005220 con 0x55a5e9726370 2024-09-12T02:10:57.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.793+0000 7f3ad77fe640 1 -- 172.21.15.16:0/3551852410 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ae8025070 con 0x55a5e9726370 2024-09-12T02:10:57.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.793+0000 7f3ad77fe640 1 -- 172.21.15.16:0/3551852410 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae8021740 con 0x55a5e9726370 2024-09-12T02:10:57.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.793+0000 7f3afa597740 1 -- 172.21.15.16:0/3551852410 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a5e9726370 msgr2=0x55a5e973ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:57.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.793+0000 7f3afa597740 1 --2- 172.21.15.16:0/3551852410 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a5e9726370 0x55a5e973ddf0 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f3ae8003fd0 tx=0x7f3ae801d000 comp rx=0 tx=0).stop 2024-09-12T02:10:57.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.794+0000 7f3afa597740 1 -- 172.21.15.16:0/3551852410 shutdown_connections 2024-09-12T02:10:57.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.794+0000 7f3afa597740 1 --2- 172.21.15.16:0/3551852410 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a5e9726370 0x55a5e973ddf0 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:57.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.794+0000 7f3afa597740 1 --2- 172.21.15.16:0/3551852410 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a5e971cb40 0x55a5e973d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:57.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.794+0000 7f3afa597740 1 --2- 172.21.15.16:0/3551852410 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a5e971b5d0 0x55a5e9732310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:57.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.794+0000 7f3afa597740 1 -- 172.21.15.16:0/3551852410 >> 172.21.15.16:0/3551852410 conn(0x55a5e96e4310 msgr2=0x55a5e9721830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:10:57.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.794+0000 7f3afa597740 1 -- 172.21.15.16:0/3551852410 shutdown_connections 2024-09-12T02:10:57.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.794+0000 7f3afa597740 1 -- 172.21.15.16:0/3551852410 wait complete. 2024-09-12T02:10:57.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.794+0000 7f3afa597740 1 -- start start 2024-09-12T02:10:57.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.794+0000 7f3afa597740 1 Processor -- start 2024-09-12T02:10:57.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.795+0000 7f3afa597740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a5e971b5d0 0x55a5e972a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:57.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.795+0000 7f3afa597740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a5e971cb40 0x55a5e972aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:57.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.795+0000 7f3afa597740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a5e9726370 0x55a5e979f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:57.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.795+0000 7f3afa597740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3af01adce0 con 0x55a5e971b5d0 2024-09-12T02:10:57.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.795+0000 7f3afa597740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3af01af150 con 0x55a5e971cb40 2024-09-12T02:10:57.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.795+0000 7f3afa597740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f3af01ae3b0 con 0x55a5e9726370 2024-09-12T02:10:57.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.795+0000 7f3afa597740 10 client.0 _ll_register_callbacks cb 0x7f3ae8003d90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:10:57.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.795+0000 7f3af5f5a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a5e971b5d0 0x55a5e972a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:57.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.795+0000 7f3af5f5a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a5e971b5d0 0x55a5e972a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39828/0 (socket says 172.21.15.16:39828) 2024-09-12T02:10:57.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.795+0000 7f3af5f5a640 1 -- 172.21.15.16:0/816987159 learned_addr learned my addr 172.21.15.16:0/816987159 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:10:57.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.795+0000 7f3af5759640 1 --2- 172.21.15.16:0/816987159 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a5e971cb40 0x55a5e972aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:57.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.795+0000 7f3af675b640 1 --2- 172.21.15.16:0/816987159 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a5e9726370 0x55a5e979f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:57.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.795+0000 7f3af5f5a640 1 -- 172.21.15.16:0/816987159 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a5e9726370 msgr2=0x55a5e979f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:57.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.795+0000 7f3af5f5a640 1 --2- 172.21.15.16:0/816987159 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a5e9726370 0x55a5e979f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:57.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.795+0000 7f3af5f5a640 1 -- 172.21.15.16:0/816987159 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a5e971cb40 msgr2=0x55a5e972aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:57.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.795+0000 7f3af5f5a640 1 --2- 172.21.15.16:0/816987159 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a5e971cb40 0x55a5e972aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:57.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.795+0000 7f3af5f5a640 1 -- 172.21.15.16:0/816987159 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5e972b340 con 0x55a5e971b5d0 2024-09-12T02:10:57.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.795+0000 7f3af5f5a640 1 --2- 172.21.15.16:0/816987159 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a5e971b5d0 0x55a5e972a540 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f3ae0022d30 tx=0x7f3ae00107c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7024d6a5c47eb943 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:10:57.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.795+0000 7f3ad57fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:10:57.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.796+0000 7f3ad57fa640 1 -- 172.21.15.16:0/816987159 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae0004be0 con 0x55a5e971b5d0 2024-09-12T02:10:57.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.796+0000 7f3afa597740 1 -- 172.21.15.16:0/816987159 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ae8025070 con 0x55a5e971b5d0 2024-09-12T02:10:57.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.796+0000 7f3ad57fa640 1 -- 172.21.15.16:0/816987159 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f3ae0004d80 con 0x55a5e971b5d0 2024-09-12T02:10:57.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.796+0000 7f3ad57fa640 1 -- 172.21.15.16:0/816987159 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f3ae0045050 con 0x55a5e971b5d0 2024-09-12T02:10:57.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.796+0000 7f3afa597740 10 client.18183 fetch_fsmap learned FSMap version 25 2024-09-12T02:10:57.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.796+0000 7f3afa597740 1 -- 172.21.15.16:0/816987159 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a5e97a0ec0 con 0x55a5e971b5d0 2024-09-12T02:10:57.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.796+0000 7f3ad57fa640 1 -- 172.21.15.16:0/816987159 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ae8025070 con 0x55a5e971b5d0 2024-09-12T02:10:57.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.796+0000 7f3ad57fa640 1 -- 172.21.15.16:0/816987159 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f3ae00226e0 con 0x55a5e971b5d0 2024-09-12T02:10:57.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.796+0000 7f3afa597740 10 client.18183 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:10:57.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.796+0000 7f3afa597740 10 client.18183 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-12T02:10:57.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.796+0000 7f3afa597740 20 client.18183 populate_metadata read hostname 'smithi016' 2024-09-12T02:10:57.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.796+0000 7f3afa597740 1 --2- 172.21.15.16:0/816987159 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55a5e97a15d0 0x55a5e97a39f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:57.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.796+0000 7f3afa597740 4 client.18183 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T02:10:57.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.796+0000 7f3afa597740 1 -- 172.21.15.16:0/816987159 --> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a5e9688d70 con 0x55a5e97a15d0 2024-09-12T02:10:57.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.797+0000 7f3af5759640 1 --2- 172.21.15.16:0/816987159 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55a5e97a15d0 0x55a5e97a39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:57.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.797+0000 7f3af5759640 1 --2- 172.21.15.16:0/816987159 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55a5e97a15d0 0x55a5e97a39f0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bd7fae3b9d13e0ea server_cookie=86041d365825689e in_seq=0 out_seq=0 2024-09-12T02:10:57.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.797+0000 7f3ad57fa640 10 client.18183 ms_handle_connect on v2:172.21.15.184:6836/1793393087 2024-09-12T02:10:57.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.797+0000 7f3ad57fa640 1 -- 172.21.15.16:0/816987159 <== mds.2 v2:172.21.15.184:6836/1793393087 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a5e9688d70 con 0x55a5e97a15d0 2024-09-12T02:10:57.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.797+0000 7f3ad57fa640 10 client.18183 handle_command_reply: tid=0 2024-09-12T02:10:57.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.797+0000 7f3afa597740 1 client.18183 shutdown 2024-09-12T02:10:57.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.797+0000 7f3afa597740 1 -- 172.21.15.16:0/816987159 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a5e971b5d0 msgr2=0x55a5e972a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:57.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.798+0000 7f3afa597740 1 --2- 172.21.15.16:0/816987159 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a5e971b5d0 0x55a5e972a540 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f3ae0022d30 tx=0x7f3ae00107c0 comp rx=0 tx=0).stop 2024-09-12T02:10:57.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.798+0000 7f3afa597740 1 -- 172.21.15.16:0/816987159 shutdown_connections 2024-09-12T02:10:57.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.798+0000 7f3afa597740 1 --2- 172.21.15.16:0/816987159 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a5e9726370 0x55a5e979f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:57.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.798+0000 7f3afa597740 1 --2- 172.21.15.16:0/816987159 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a5e971cb40 0x55a5e972aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:57.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.798+0000 7f3afa597740 1 --2- 172.21.15.16:0/816987159 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55a5e97a15d0 0x55a5e97a39f0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:57.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.798+0000 7f3afa597740 1 --2- 172.21.15.16:0/816987159 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a5e971b5d0 0x55a5e972a540 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:57.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.798+0000 7f3ad57fa640 0 client.18183 ms_handle_reset on v2:172.21.15.184:6836/1793393087 2024-09-12T02:10:57.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.798+0000 7f3afa597740 1 -- 172.21.15.16:0/816987159 >> 172.21.15.16:0/816987159 conn(0x55a5e96e4310 msgr2=0x55a5e97164e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:10:57.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.798+0000 7f3afa597740 1 -- 172.21.15.16:0/816987159 shutdown_connections 2024-09-12T02:10:57.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.798+0000 7f3afa597740 1 -- 172.21.15.16:0/816987159 wait complete. 2024-09-12T02:10:57.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.798+0000 7f3afa597740 20 client.18183 trim_cache size 0 max 16384 2024-09-12T02:10:57.809 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T02:10:57.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.798+0000 7f3af81be640 1 -- 172.21.15.16:0/4162060307 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f3ad80a2b80 msgr2=0x7f3ad80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:57.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.798+0000 7f3af81be640 1 --2- 172.21.15.16:0/4162060307 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f3ad80a2b80 0x7f3ad80a5040 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f3ae800bab0 tx=0x7f3ae800beb0 comp rx=0 tx=0).stop 2024-09-12T02:10:57.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.798+0000 7f3af81be640 1 -- 172.21.15.16:0/4162060307 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3af0103720 msgr2=0x7f3af019ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:57.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.798+0000 7f3af81be640 1 --2- 172.21.15.16:0/4162060307 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3af0103720 0x7f3af019ad10 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f3ae4005940 tx=0x7f3ae40042f0 comp rx=0 tx=0).stop 2024-09-12T02:10:57.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.799+0000 7f3af81be640 1 -- 172.21.15.16:0/4162060307 shutdown_connections 2024-09-12T02:10:57.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.799+0000 7f3af81be640 1 --2- 172.21.15.16:0/4162060307 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f3ad80a2b80 0x7f3ad80a5040 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:57.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.799+0000 7f3af81be640 1 --2- 172.21.15.16:0/4162060307 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3af01082a0 0x7f3af019f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:57.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.799+0000 7f3af81be640 1 --2- 172.21.15.16:0/4162060307 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3af0103720 0x7f3af019ad10 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:57.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.799+0000 7f3af81be640 1 --2- 172.21.15.16:0/4162060307 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3af0102d70 0x7f3af019a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:57.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.799+0000 7f3af81be640 1 -- 172.21.15.16:0/4162060307 >> 172.21.15.16:0/4162060307 conn(0x7f3af00fc7d0 msgr2=0x7f3af01009a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:10:57.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.799+0000 7f3af81be640 1 -- 172.21.15.16:0/4162060307 shutdown_connections 2024-09-12T02:10:57.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:57.799+0000 7f3af81be640 1 -- 172.21.15.16:0/4162060307 wait complete. 2024-09-12T02:10:58.183 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:10:57 smithi016 ceph-mon[30947]: pgmap v606: 97 pgs: 97 active+clean; 773 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-09-12T02:10:58.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:10:57 smithi184 ceph-mon[35593]: pgmap v606: 97 pgs: 97 active+clean; 773 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-09-12T02:10:58.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:10:57 smithi033 ceph-mon[35332]: pgmap v606: 97 pgs: 97 active+clean; 773 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-09-12T02:10:58.553 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-12T02:10:58.818 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:10:59.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.293+0000 7fec6f35c640 1 -- 172.21.15.16:0/1091998113 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec6810c170 msgr2=0x7fec6810e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:59.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.293+0000 7fec6f35c640 1 --2- 172.21.15.16:0/1091998113 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec6810c170 0x7fec6810e610 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7fec60009ca0 tx=0x7fec6002a950 comp rx=0 tx=0).stop 2024-09-12T02:10:59.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.293+0000 7fec6f35c640 1 -- 172.21.15.16:0/1091998113 shutdown_connections 2024-09-12T02:10:59.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.293+0000 7fec6f35c640 1 --2- 172.21.15.16:0/1091998113 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec6810c170 0x7fec6810e610 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:59.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.293+0000 7fec6f35c640 1 --2- 172.21.15.16:0/1091998113 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fec68103660 0x7fec6810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:59.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.293+0000 7fec6f35c640 1 --2- 172.21.15.16:0/1091998113 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fec68102d40 0x7fec68103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:59.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.293+0000 7fec6f35c640 1 -- 172.21.15.16:0/1091998113 >> 172.21.15.16:0/1091998113 conn(0x7fec680fc7a0 msgr2=0x7fec680febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:10:59.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.293+0000 7fec6f35c640 1 -- 172.21.15.16:0/1091998113 shutdown_connections 2024-09-12T02:10:59.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.293+0000 7fec6f35c640 1 -- 172.21.15.16:0/1091998113 wait complete. 2024-09-12T02:10:59.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.294+0000 7fec6f35c640 1 Processor -- start 2024-09-12T02:10:59.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.294+0000 7fec6f35c640 1 -- start start 2024-09-12T02:10:59.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.294+0000 7fec6f35c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fec68102d40 0x7fec68198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:59.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.294+0000 7fec6f35c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec68103660 0x7fec68198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:59.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.294+0000 7fec6f35c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fec6810c170 0x7fec6819cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:59.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.294+0000 7fec6f35c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fec68111f70 con 0x7fec68103660 2024-09-12T02:10:59.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.294+0000 7fec6f35c640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fec68111df0 con 0x7fec68102d40 2024-09-12T02:10:59.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.294+0000 7fec6f35c640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fec681120f0 con 0x7fec6810c170 2024-09-12T02:10:59.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.294+0000 7fec6c8f7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec68103660 0x7fec68198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:59.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.294+0000 7fec6c8f7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec68103660 0x7fec68198870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39846/0 (socket says 172.21.15.16:39846) 2024-09-12T02:10:59.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.294+0000 7fec6d0f8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fec68102d40 0x7fec68198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:59.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.294+0000 7fec6c8f7640 1 -- 172.21.15.16:0/3073604865 learned_addr learned my addr 172.21.15.16:0/3073604865 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:10:59.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.294+0000 7fec6d8f9640 1 --2- 172.21.15.16:0/3073604865 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fec6810c170 0x7fec6819cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:59.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.294+0000 7fec6c8f7640 1 -- 172.21.15.16:0/3073604865 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fec6810c170 msgr2=0x7fec6819cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:59.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.294+0000 7fec6c8f7640 1 --2- 172.21.15.16:0/3073604865 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fec6810c170 0x7fec6819cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:59.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.294+0000 7fec6c8f7640 1 -- 172.21.15.16:0/3073604865 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fec68102d40 msgr2=0x7fec68198330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:59.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.294+0000 7fec6c8f7640 1 --2- 172.21.15.16:0/3073604865 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fec68102d40 0x7fec68198330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:59.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.295+0000 7fec6c8f7640 1 -- 172.21.15.16:0/3073604865 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec6819d2e0 con 0x7fec68103660 2024-09-12T02:10:59.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.295+0000 7fec6d8f9640 1 --2- 172.21.15.16:0/3073604865 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fec6810c170 0x7fec6819cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:10:59.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.295+0000 7fec6d0f8640 1 --2- 172.21.15.16:0/3073604865 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fec68102d40 0x7fec68198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:10:59.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.295+0000 7fec6c8f7640 1 --2- 172.21.15.16:0/3073604865 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec68103660 0x7fec68198870 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7fec5800c970 tx=0x7fec5800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:10:59.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.295+0000 7fec5e7fc640 1 -- 172.21.15.16:0/3073604865 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec58007bb0 con 0x7fec68103660 2024-09-12T02:10:59.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.295+0000 7fec6f35c640 1 -- 172.21.15.16:0/3073604865 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec6819d5d0 con 0x7fec68103660 2024-09-12T02:10:59.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.295+0000 7fec5e7fc640 1 -- 172.21.15.16:0/3073604865 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec58007d50 con 0x7fec68103660 2024-09-12T02:10:59.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.295+0000 7fec5e7fc640 1 -- 172.21.15.16:0/3073604865 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec58018820 con 0x7fec68103660 2024-09-12T02:10:59.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.295+0000 7fec6f35c640 1 -- 172.21.15.16:0/3073604865 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec681a4eb0 con 0x7fec68103660 2024-09-12T02:10:59.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.297+0000 7fec5e7fc640 1 -- 172.21.15.16:0/3073604865 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fec58018a80 con 0x7fec68103660 2024-09-12T02:10:59.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.298+0000 7fec71735740 1 Processor -- start 2024-09-12T02:10:59.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.298+0000 7fec71735740 1 -- start start 2024-09-12T02:10:59.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.298+0000 7fec71735740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b302ff2b40 0x55b302ff1090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:59.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.298+0000 7fec71735740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b302ff15d0 0x55b302ff1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:59.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.298+0000 7fec71735740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b302ffc370 0x55b302ffe760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:59.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.298+0000 7fec71735740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55b302f572f0 con 0x55b302ffc370 2024-09-12T02:10:59.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.298+0000 7fec71735740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55b302efecf0 con 0x55b302ff15d0 2024-09-12T02:10:59.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.298+0000 7fec71735740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55b302f532b0 con 0x55b302ff2b40 2024-09-12T02:10:59.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.298+0000 7fec5e7fc640 1 --2- 172.21.15.16:0/3073604865 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fec540a2b80 0x7fec540a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:59.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.298+0000 7fec6c8f7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b302ffc370 0x55b302ffe760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:59.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.298+0000 7fec6d8f9640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b302ff2b40 0x55b302ff1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:59.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.298+0000 7fec6d8f9640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b302ff2b40 0x55b302ff1090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:41346/0 (socket says 172.21.15.16:41346) 2024-09-12T02:10:59.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.298+0000 7fec6c8f7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b302ffc370 0x55b302ffe760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39858/0 (socket says 172.21.15.16:39858) 2024-09-12T02:10:59.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.298+0000 7fec6d8f9640 1 -- 172.21.15.16:0/100207382 learned_addr learned my addr 172.21.15.16:0/100207382 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:10:59.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.299+0000 7fec6d0f8640 1 --2- 172.21.15.16:0/100207382 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b302ff15d0 0x55b302ff1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:59.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.299+0000 7fec6d8f9640 1 --2- 172.21.15.16:0/3073604865 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fec540a2b80 0x7fec540a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:59.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.299+0000 7fec5e7fc640 1 -- 172.21.15.16:0/3073604865 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fec580cf020 con 0x7fec68103660 2024-09-12T02:10:59.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.299+0000 7fec6c8f7640 1 -- 172.21.15.16:0/100207382 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b302ff2b40 msgr2=0x55b302ff1090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:59.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.299+0000 7fec6c8f7640 1 --2- 172.21.15.16:0/100207382 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b302ff2b40 0x55b302ff1090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:59.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.299+0000 7fec6c8f7640 1 -- 172.21.15.16:0/100207382 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b302ff15d0 msgr2=0x55b302ff1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:59.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.299+0000 7fec6c8f7640 1 --2- 172.21.15.16:0/100207382 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b302ff15d0 0x55b302ff1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:59.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.299+0000 7fec6c8f7640 1 -- 172.21.15.16:0/100207382 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b302ee33a0 con 0x55b302ffc370 2024-09-12T02:10:59.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.299+0000 7fec6d0f8640 1 --2- 172.21.15.16:0/100207382 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b302ff15d0 0x55b302ff1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:10:59.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.299+0000 7fec6d8f9640 1 --2- 172.21.15.16:0/3073604865 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fec540a2b80 0x7fec540a5040 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fec6002f480 tx=0x7fec60005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:10:59.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.299+0000 7fec6c8f7640 1 --2- 172.21.15.16:0/100207382 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b302ffc370 0x55b302ffe760 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7fec580ccc10 tx=0x7fec5806e9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37cedc02fc61de69 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:10:59.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.299+0000 7fec537fe640 1 -- 172.21.15.16:0/100207382 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec58066a00 con 0x55b302ffc370 2024-09-12T02:10:59.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.299+0000 7fec537fe640 1 -- 172.21.15.16:0/100207382 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec5809c020 con 0x55b302ffc370 2024-09-12T02:10:59.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.299+0000 7fec537fe640 1 -- 172.21.15.16:0/100207382 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec58018e40 con 0x55b302ffc370 2024-09-12T02:10:59.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.300+0000 7fec71735740 1 -- 172.21.15.16:0/100207382 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b302ffc370 msgr2=0x55b302ffe760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:59.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.300+0000 7fec71735740 1 --2- 172.21.15.16:0/100207382 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b302ffc370 0x55b302ffe760 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7fec580ccc10 tx=0x7fec5806e9e0 comp rx=0 tx=0).stop 2024-09-12T02:10:59.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.300+0000 7fec71735740 1 -- 172.21.15.16:0/100207382 shutdown_connections 2024-09-12T02:10:59.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.300+0000 7fec71735740 1 --2- 172.21.15.16:0/100207382 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b302ffc370 0x55b302ffe760 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:59.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.300+0000 7fec71735740 1 --2- 172.21.15.16:0/100207382 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b302ff15d0 0x55b302ff1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:59.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.300+0000 7fec71735740 1 --2- 172.21.15.16:0/100207382 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b302ff2b40 0x55b302ff1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:59.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.300+0000 7fec71735740 1 -- 172.21.15.16:0/100207382 >> 172.21.15.16:0/100207382 conn(0x55b302fba310 msgr2=0x55b302fedc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:10:59.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.300+0000 7fec71735740 1 -- 172.21.15.16:0/100207382 shutdown_connections 2024-09-12T02:10:59.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.300+0000 7fec71735740 1 -- 172.21.15.16:0/100207382 wait complete. 2024-09-12T02:10:59.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.300+0000 7fec71735740 1 -- start start 2024-09-12T02:10:59.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.301+0000 7fec71735740 1 Processor -- start 2024-09-12T02:10:59.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.301+0000 7fec71735740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b302ff15d0 0x55b302ffbe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:59.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.301+0000 7fec71735740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b302ff2b40 0x55b302ff90e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:59.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.301+0000 7fec71735740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b302ffc370 0x55b302ff9870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:59.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.301+0000 7fec71735740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55b302fff070 con 0x55b302ff15d0 2024-09-12T02:10:59.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.301+0000 7fec71735740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55b302f12060 con 0x55b302ff2b40 2024-09-12T02:10:59.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.301+0000 7fec71735740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55b302fff1f0 con 0x55b302ffc370 2024-09-12T02:10:59.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.301+0000 7fec71735740 10 client.0 _ll_register_callbacks cb 0x55b302c658c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:10:59.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.301+0000 7fec6d0f8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b302ff15d0 0x55b302ffbe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:59.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.301+0000 7fec6d0f8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b302ff15d0 0x55b302ffbe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39860/0 (socket says 172.21.15.16:39860) 2024-09-12T02:10:59.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.301+0000 7fec6d0f8640 1 -- 172.21.15.16:0/1614048762 learned_addr learned my addr 172.21.15.16:0/1614048762 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:10:59.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.301+0000 7fec6d8f9640 1 --2- 172.21.15.16:0/1614048762 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b302ffc370 0x55b302ff9870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:59.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.301+0000 7fec6c8f7640 1 --2- 172.21.15.16:0/1614048762 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b302ff2b40 0x55b302ff90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:59.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.301+0000 7fec6d0f8640 1 -- 172.21.15.16:0/1614048762 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b302ffc370 msgr2=0x55b302ff9870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:59.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.301+0000 7fec6d0f8640 1 --2- 172.21.15.16:0/1614048762 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b302ffc370 0x55b302ff9870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:59.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.301+0000 7fec6d0f8640 1 -- 172.21.15.16:0/1614048762 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b302ff2b40 msgr2=0x55b302ff90e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:59.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.301+0000 7fec6d0f8640 1 --2- 172.21.15.16:0/1614048762 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b302ff2b40 0x55b302ff90e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:59.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.301+0000 7fec6d0f8640 1 -- 172.21.15.16:0/1614048762 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b302cc83c0 con 0x55b302ff15d0 2024-09-12T02:10:59.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.301+0000 7fec6d8f9640 1 --2- 172.21.15.16:0/1614048762 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b302ffc370 0x55b302ff9870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:10:59.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.301+0000 7fec6c8f7640 1 --2- 172.21.15.16:0/1614048762 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b302ff2b40 0x55b302ff90e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:10:59.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.302+0000 7fec6d0f8640 1 --2- 172.21.15.16:0/1614048762 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b302ff15d0 0x55b302ffbe30 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7fec640117e0 tx=0x7fec64011ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e3e514b01a0032 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:10:59.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.302+0000 7fec50ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:10:59.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.302+0000 7fec50ff9640 1 -- 172.21.15.16:0/1614048762 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec6401f060 con 0x55b302ff15d0 2024-09-12T02:10:59.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.302+0000 7fec71735740 1 -- 172.21.15.16:0/1614048762 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fec5809c020 con 0x55b302ff15d0 2024-09-12T02:10:59.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.302+0000 7fec50ff9640 1 -- 172.21.15.16:0/1614048762 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fec64010ba0 con 0x55b302ff15d0 2024-09-12T02:10:59.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.302+0000 7fec50ff9640 1 -- 172.21.15.16:0/1614048762 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fec64010da0 con 0x55b302ff15d0 2024-09-12T02:10:59.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.302+0000 7fec71735740 10 client.18195 fetch_fsmap learned FSMap version 25 2024-09-12T02:10:59.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.302+0000 7fec71735740 1 -- 172.21.15.16:0/1614048762 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b30300ac40 con 0x55b302ff15d0 2024-09-12T02:10:59.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.302+0000 7fec50ff9640 1 -- 172.21.15.16:0/1614048762 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec5809c020 con 0x55b302ff15d0 2024-09-12T02:10:59.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.302+0000 7fec50ff9640 1 -- 172.21.15.16:0/1614048762 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fec64024070 con 0x55b302ff15d0 2024-09-12T02:10:59.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.302+0000 7fec71735740 10 client.18195 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:10:59.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.302+0000 7fec71735740 10 client.18195 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T02:10:59.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.302+0000 7fec71735740 20 client.18195 populate_metadata read hostname 'smithi016' 2024-09-12T02:10:59.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.302+0000 7fec71735740 1 --2- 172.21.15.16:0/1614048762 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x55b30300c100 0x55b30300e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:59.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.302+0000 7fec71735740 4 client.18195 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:10:59.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.302+0000 7fec71735740 1 -- 172.21.15.16:0/1614048762 --> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b302f64860 con 0x55b30300c100 2024-09-12T02:10:59.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.303+0000 7fec6c8f7640 1 --2- 172.21.15.16:0/1614048762 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x55b30300c100 0x55b30300e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:59.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.303+0000 7fec50ff9640 10 client.18195 ms_handle_connect on v2:172.21.15.184:6838/4203347941 2024-09-12T02:10:59.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.303+0000 7fec6c8f7640 1 --2- 172.21.15.16:0/1614048762 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x55b30300c100 0x55b30300e4f0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bb32697ce8904257 server_cookie=38154456181e7faf in_seq=0 out_seq=0 2024-09-12T02:10:59.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.304+0000 7fec50ff9640 1 -- 172.21.15.16:0/1614048762 <== mds.1 v2:172.21.15.184:6838/4203347941 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b302f64860 con 0x55b30300c100 2024-09-12T02:10:59.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.304+0000 7fec50ff9640 10 client.18195 handle_command_reply: tid=0 2024-09-12T02:10:59.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.304+0000 7fec71735740 1 client.18195 shutdown 2024-09-12T02:10:59.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.304+0000 7fec71735740 1 -- 172.21.15.16:0/1614048762 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b302ff15d0 msgr2=0x55b302ffbe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:59.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.304+0000 7fec71735740 1 --2- 172.21.15.16:0/1614048762 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b302ff15d0 0x55b302ffbe30 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7fec640117e0 tx=0x7fec64011ca0 comp rx=0 tx=0).stop 2024-09-12T02:10:59.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.305+0000 7fec71735740 1 -- 172.21.15.16:0/1614048762 shutdown_connections 2024-09-12T02:10:59.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.305+0000 7fec71735740 1 --2- 172.21.15.16:0/1614048762 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x55b30300c100 0x55b30300e4f0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:59.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.305+0000 7fec71735740 1 --2- 172.21.15.16:0/1614048762 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b302ffc370 0x55b302ff9870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:59.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.305+0000 7fec71735740 1 --2- 172.21.15.16:0/1614048762 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b302ff2b40 0x55b302ff90e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:59.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.305+0000 7fec50ff9640 0 client.18195 ms_handle_reset on v2:172.21.15.184:6838/4203347941 2024-09-12T02:10:59.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.305+0000 7fec71735740 1 --2- 172.21.15.16:0/1614048762 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b302ff15d0 0x55b302ffbe30 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:59.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.305+0000 7fec71735740 1 -- 172.21.15.16:0/1614048762 >> 172.21.15.16:0/1614048762 conn(0x55b302fba310 msgr2=0x55b302ff3990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:10:59.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.305+0000 7fec71735740 1 -- 172.21.15.16:0/1614048762 shutdown_connections 2024-09-12T02:10:59.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.305+0000 7fec71735740 1 -- 172.21.15.16:0/1614048762 wait complete. 2024-09-12T02:10:59.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.305+0000 7fec71735740 20 client.18195 trim_cache size 0 max 16384 2024-09-12T02:10:59.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.318+0000 7fec71735740 1 Processor -- start 2024-09-12T02:10:59.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.318+0000 7fec71735740 1 -- start start 2024-09-12T02:10:59.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.318+0000 7fec71735740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b302ff15d0 0x55b302fede10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:59.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.318+0000 7fec71735740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b302ff2b40 0x55b302ff89a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:59.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.318+0000 7fec71735740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b302ffc370 0x55b303013da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:59.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.318+0000 7fec71735740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fec681afc00 con 0x55b302ff2b40 2024-09-12T02:10:59.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.318+0000 7fec71735740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fec681ad4d0 con 0x55b302ffc370 2024-09-12T02:10:59.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.318+0000 7fec71735740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fec681ab320 con 0x55b302ff15d0 2024-09-12T02:10:59.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.318+0000 7fec6c8f7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b302ff2b40 0x55b302ff89a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:59.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.318+0000 7fec6c8f7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b302ff2b40 0x55b302ff89a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39862/0 (socket says 172.21.15.16:39862) 2024-09-12T02:10:59.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.318+0000 7fec6c8f7640 1 -- 172.21.15.16:0/2777665485 learned_addr learned my addr 172.21.15.16:0/2777665485 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:10:59.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.318+0000 7fec6d0f8640 1 --2- 172.21.15.16:0/2777665485 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b302ff15d0 0x55b302fede10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:59.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.318+0000 7fec6d8f9640 1 --2- 172.21.15.16:0/2777665485 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b302ffc370 0x55b303013da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:59.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.319+0000 7fec6c8f7640 1 -- 172.21.15.16:0/2777665485 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b302ff15d0 msgr2=0x55b302fede10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:59.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.319+0000 7fec6c8f7640 1 --2- 172.21.15.16:0/2777665485 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b302ff15d0 0x55b302fede10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:59.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.319+0000 7fec6c8f7640 1 -- 172.21.15.16:0/2777665485 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b302ffc370 msgr2=0x55b303013da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:59.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.319+0000 7fec6c8f7640 1 --2- 172.21.15.16:0/2777665485 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b302ffc370 0x55b303013da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:59.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.319+0000 7fec6c8f7640 1 -- 172.21.15.16:0/2777665485 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b302ff9be0 con 0x55b302ff2b40 2024-09-12T02:10:59.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.319+0000 7fec6d8f9640 1 --2- 172.21.15.16:0/2777665485 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b302ffc370 0x55b303013da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:10:59.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.319+0000 7fec6d0f8640 1 --2- 172.21.15.16:0/2777665485 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b302ff15d0 0x55b302fede10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:10:59.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.319+0000 7fec6c8f7640 1 --2- 172.21.15.16:0/2777665485 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b302ff2b40 0x55b302ff89a0 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7fec5809b620 tx=0x7fec580c64b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb3d542861ced0a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:10:59.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.319+0000 7fec52ffd640 1 -- 172.21.15.16:0/2777665485 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec580a4590 con 0x55b302ff2b40 2024-09-12T02:10:59.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.319+0000 7fec52ffd640 1 -- 172.21.15.16:0/2777665485 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec5807f1b0 con 0x55b302ff2b40 2024-09-12T02:10:59.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.319+0000 7fec52ffd640 1 -- 172.21.15.16:0/2777665485 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec58074ba0 con 0x55b302ff2b40 2024-09-12T02:10:59.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.320+0000 7fec71735740 1 -- 172.21.15.16:0/2777665485 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b302ff2b40 msgr2=0x55b302ff89a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:59.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.320+0000 7fec71735740 1 --2- 172.21.15.16:0/2777665485 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b302ff2b40 0x55b302ff89a0 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7fec5809b620 tx=0x7fec580c64b0 comp rx=0 tx=0).stop 2024-09-12T02:10:59.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.320+0000 7fec71735740 1 -- 172.21.15.16:0/2777665485 shutdown_connections 2024-09-12T02:10:59.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.320+0000 7fec71735740 1 --2- 172.21.15.16:0/2777665485 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b302ffc370 0x55b303013da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:59.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.320+0000 7fec71735740 1 --2- 172.21.15.16:0/2777665485 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b302ff2b40 0x55b302ff89a0 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:59.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.320+0000 7fec71735740 1 --2- 172.21.15.16:0/2777665485 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b302ff15d0 0x55b302fede10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:59.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.320+0000 7fec71735740 1 -- 172.21.15.16:0/2777665485 >> 172.21.15.16:0/2777665485 conn(0x55b302fba310 msgr2=0x55b302fed700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:10:59.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.320+0000 7fec71735740 1 -- 172.21.15.16:0/2777665485 shutdown_connections 2024-09-12T02:10:59.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.320+0000 7fec71735740 1 -- 172.21.15.16:0/2777665485 wait complete. 2024-09-12T02:10:59.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.320+0000 7fec71735740 1 -- start start 2024-09-12T02:10:59.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.320+0000 7fec71735740 1 Processor -- start 2024-09-12T02:10:59.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.321+0000 7fec71735740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b302ff15d0 0x55b302ff6030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:59.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.321+0000 7fec71735740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b302ff2b40 0x55b302ff6570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:59.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.321+0000 7fec71735740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b302ffc370 0x55b302ff6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:59.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.321+0000 7fec71735740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fec681ab990 con 0x55b302ff15d0 2024-09-12T02:10:59.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.321+0000 7fec71735740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fec681ace00 con 0x55b302ffc370 2024-09-12T02:10:59.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.321+0000 7fec71735740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fec681ac060 con 0x55b302ff2b40 2024-09-12T02:10:59.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.321+0000 7fec6d0f8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b302ff15d0 0x55b302ff6030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:59.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.321+0000 7fec6d0f8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b302ff15d0 0x55b302ff6030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39870/0 (socket says 172.21.15.16:39870) 2024-09-12T02:10:59.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.321+0000 7fec6d0f8640 1 -- 172.21.15.16:0/927261302 learned_addr learned my addr 172.21.15.16:0/927261302 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:10:59.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.321+0000 7fec71735740 10 client.0 _ll_register_callbacks cb 0x7fec6400bc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:10:59.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.321+0000 7fec6c8f7640 1 --2- 172.21.15.16:0/927261302 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b302ff2b40 0x55b302ff6570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:59.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.321+0000 7fec6d8f9640 1 --2- 172.21.15.16:0/927261302 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b302ffc370 0x55b302ff6ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:59.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.321+0000 7fec6d0f8640 1 -- 172.21.15.16:0/927261302 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b302ff2b40 msgr2=0x55b302ff6570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:59.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.321+0000 7fec6d0f8640 1 --2- 172.21.15.16:0/927261302 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b302ff2b40 0x55b302ff6570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:59.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.321+0000 7fec6d0f8640 1 -- 172.21.15.16:0/927261302 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b302ffc370 msgr2=0x55b302ff6ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:59.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.321+0000 7fec6d0f8640 1 --2- 172.21.15.16:0/927261302 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b302ffc370 0x55b302ff6ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:59.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.321+0000 7fec6d0f8640 1 -- 172.21.15.16:0/927261302 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b303075d70 con 0x55b302ff15d0 2024-09-12T02:10:59.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.321+0000 7fec6d8f9640 1 --2- 172.21.15.16:0/927261302 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b302ffc370 0x55b302ff6ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:10:59.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.321+0000 7fec6d0f8640 1 --2- 172.21.15.16:0/927261302 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b302ff15d0 0x55b302ff6030 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7fec6400f810 tx=0x7fec6400fba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71acd152cb13b7d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:10:59.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.321+0000 7fec50ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:10:59.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.322+0000 7fec50ff9640 1 -- 172.21.15.16:0/927261302 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec6402a040 con 0x55b302ff15d0 2024-09-12T02:10:59.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.322+0000 7fec50ff9640 1 -- 172.21.15.16:0/927261302 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fec64010ba0 con 0x55b302ff15d0 2024-09-12T02:10:59.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.322+0000 7fec71735740 1 -- 172.21.15.16:0/927261302 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fec5807f1b0 con 0x55b302ff15d0 2024-09-12T02:10:59.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.322+0000 7fec50ff9640 1 -- 172.21.15.16:0/927261302 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fec6400fc10 con 0x55b302ff15d0 2024-09-12T02:10:59.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.322+0000 7fec71735740 10 client.18201 fetch_fsmap learned FSMap version 25 2024-09-12T02:10:59.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.322+0000 7fec71735740 1 -- 172.21.15.16:0/927261302 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b303077010 con 0x55b302ff15d0 2024-09-12T02:10:59.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.322+0000 7fec50ff9640 1 -- 172.21.15.16:0/927261302 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec5807f1b0 con 0x55b302ff15d0 2024-09-12T02:10:59.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.322+0000 7fec50ff9640 1 -- 172.21.15.16:0/927261302 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fec64024040 con 0x55b302ff15d0 2024-09-12T02:10:59.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.322+0000 7fec71735740 10 client.18201 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:10:59.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.322+0000 7fec71735740 10 client.18201 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T02:10:59.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.322+0000 7fec71735740 20 client.18201 populate_metadata read hostname 'smithi016' 2024-09-12T02:10:59.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.322+0000 7fec71735740 1 --2- 172.21.15.16:0/927261302 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x55b303077790 0x55b303079bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:59.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.322+0000 7fec71735740 4 client.18201 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T02:10:59.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.322+0000 7fec71735740 1 -- 172.21.15.16:0/927261302 --> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b302f5ed70 con 0x55b303077790 2024-09-12T02:10:59.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.323+0000 7fec6c8f7640 1 --2- 172.21.15.16:0/927261302 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x55b303077790 0x55b303079bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:59.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.323+0000 7fec50ff9640 10 client.18201 ms_handle_connect on v2:172.21.15.184:6838/4203347941 2024-09-12T02:10:59.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.323+0000 7fec6c8f7640 1 --2- 172.21.15.16:0/927261302 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x55b303077790 0x55b303079bb0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2fa746485f521ea3 server_cookie=7d2efa0439865b74 in_seq=0 out_seq=0 2024-09-12T02:10:59.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.323+0000 7fec50ff9640 1 -- 172.21.15.16:0/927261302 <== mds.1 v2:172.21.15.184:6838/4203347941 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b302f5ed70 con 0x55b303077790 2024-09-12T02:10:59.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.323+0000 7fec50ff9640 10 client.18201 handle_command_reply: tid=0 2024-09-12T02:10:59.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.324+0000 7fec71735740 1 client.18201 shutdown 2024-09-12T02:10:59.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.324+0000 7fec71735740 1 -- 172.21.15.16:0/927261302 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b302ff15d0 msgr2=0x55b302ff6030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:59.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.324+0000 7fec71735740 1 --2- 172.21.15.16:0/927261302 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b302ff15d0 0x55b302ff6030 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7fec6400f810 tx=0x7fec6400fba0 comp rx=0 tx=0).stop 2024-09-12T02:10:59.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.324+0000 7fec71735740 1 -- 172.21.15.16:0/927261302 shutdown_connections 2024-09-12T02:10:59.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.324+0000 7fec71735740 1 --2- 172.21.15.16:0/927261302 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x55b303077790 0x55b303079bb0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:59.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.324+0000 7fec71735740 1 --2- 172.21.15.16:0/927261302 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b302ffc370 0x55b302ff6ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:59.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.324+0000 7fec71735740 1 --2- 172.21.15.16:0/927261302 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b302ff2b40 0x55b302ff6570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:59.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.324+0000 7fec71735740 1 --2- 172.21.15.16:0/927261302 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b302ff15d0 0x55b302ff6030 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:59.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.324+0000 7fec71735740 1 -- 172.21.15.16:0/927261302 >> 172.21.15.16:0/927261302 conn(0x55b302fba310 msgr2=0x55b302ff3110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:10:59.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.324+0000 7fec50ff9640 0 client.18201 ms_handle_reset on v2:172.21.15.184:6838/4203347941 2024-09-12T02:10:59.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.324+0000 7fec71735740 1 -- 172.21.15.16:0/927261302 shutdown_connections 2024-09-12T02:10:59.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.324+0000 7fec71735740 1 -- 172.21.15.16:0/927261302 wait complete. 2024-09-12T02:10:59.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.324+0000 7fec71735740 20 client.18201 trim_cache size 0 max 16384 2024-09-12T02:10:59.340 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T02:10:59.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.324+0000 7fec6f35c640 1 -- 172.21.15.16:0/3073604865 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fec540a2b80 msgr2=0x7fec540a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:59.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.325+0000 7fec6f35c640 1 --2- 172.21.15.16:0/3073604865 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fec540a2b80 0x7fec540a5040 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fec6002f480 tx=0x7fec60005f90 comp rx=0 tx=0).stop 2024-09-12T02:10:59.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.325+0000 7fec6f35c640 1 -- 172.21.15.16:0/3073604865 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec68103660 msgr2=0x7fec68198870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:59.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.325+0000 7fec6f35c640 1 --2- 172.21.15.16:0/3073604865 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec68103660 0x7fec68198870 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7fec5800c970 tx=0x7fec5800ce30 comp rx=0 tx=0).stop 2024-09-12T02:10:59.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.325+0000 7fec6f35c640 1 -- 172.21.15.16:0/3073604865 shutdown_connections 2024-09-12T02:10:59.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.325+0000 7fec6f35c640 1 --2- 172.21.15.16:0/3073604865 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fec540a2b80 0x7fec540a5040 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:59.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.325+0000 7fec6f35c640 1 --2- 172.21.15.16:0/3073604865 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fec6810c170 0x7fec6819cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:59.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.325+0000 7fec6f35c640 1 --2- 172.21.15.16:0/3073604865 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec68103660 0x7fec68198870 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:59.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.325+0000 7fec6f35c640 1 --2- 172.21.15.16:0/3073604865 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fec68102d40 0x7fec68198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:59.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.325+0000 7fec6f35c640 1 -- 172.21.15.16:0/3073604865 >> 172.21.15.16:0/3073604865 conn(0x7fec680fc7a0 msgr2=0x7fec680fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:10:59.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.325+0000 7fec6f35c640 1 -- 172.21.15.16:0/3073604865 shutdown_connections 2024-09-12T02:10:59.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:10:59.325+0000 7fec6f35c640 1 -- 172.21.15.16:0/3073604865 wait complete. 2024-09-12T02:11:00.150 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-12T02:11:00.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:10:59 smithi184 ceph-mon[35593]: pgmap v607: 97 pgs: 97 active+clean; 773 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-12T02:11:00.409 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:11:00.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:10:59 smithi033 ceph-mon[35332]: pgmap v607: 97 pgs: 97 active+clean; 773 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-12T02:11:00.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:10:59 smithi016 ceph-mon[30947]: pgmap v607: 97 pgs: 97 active+clean; 773 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-12T02:11:00.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.884+0000 7f9f613a1640 1 -- 172.21.15.16:0/3625206219 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f5c100b50 msgr2=0x7f9f5c106690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:00.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.884+0000 7f9f613a1640 1 --2- 172.21.15.16:0/3625206219 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f5c100b50 0x7f9f5c106690 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f9f50009930 tx=0x7f9f5002a420 comp rx=0 tx=0).stop 2024-09-12T02:11:00.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.884+0000 7f9f613a1640 1 -- 172.21.15.16:0/3625206219 shutdown_connections 2024-09-12T02:11:00.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.884+0000 7f9f613a1640 1 --2- 172.21.15.16:0/3625206219 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9f5c106df0 0x7f9f5c10e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:00.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.884+0000 7f9f613a1640 1 --2- 172.21.15.16:0/3625206219 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f5c100b50 0x7f9f5c106690 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:00.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.884+0000 7f9f613a1640 1 --2- 172.21.15.16:0/3625206219 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f5c1001a0 0x7f9f5c100580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:00.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.884+0000 7f9f613a1640 1 -- 172.21.15.16:0/3625206219 >> 172.21.15.16:0/3625206219 conn(0x7f9f5c077070 msgr2=0x7f9f5c0fe480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:00.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.884+0000 7f9f613a1640 1 -- 172.21.15.16:0/3625206219 shutdown_connections 2024-09-12T02:11:00.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.884+0000 7f9f613a1640 1 -- 172.21.15.16:0/3625206219 wait complete. 2024-09-12T02:11:00.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.885+0000 7f9f613a1640 1 Processor -- start 2024-09-12T02:11:00.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.885+0000 7f9f613a1640 1 -- start start 2024-09-12T02:11:00.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.885+0000 7f9f613a1640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9f5c1001a0 0x7f9f5c101120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:00.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.885+0000 7f9f613a1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f5c100b50 0x7f9f5c101660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:00.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.885+0000 7f9f613a1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f5c106df0 0x7f9f5c101ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:00.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.885+0000 7f9f613a1640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9f5c111dd0 con 0x7f9f5c106df0 2024-09-12T02:11:00.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.885+0000 7f9f613a1640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9f5c111c50 con 0x7f9f5c100b50 2024-09-12T02:11:00.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.885+0000 7f9f613a1640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f9f5c111f50 con 0x7f9f5c1001a0 2024-09-12T02:11:00.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.885+0000 7f9f60ba0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f5c106df0 0x7f9f5c101ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:00.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.885+0000 7f9f60ba0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f5c106df0 0x7f9f5c101ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39886/0 (socket says 172.21.15.16:39886) 2024-09-12T02:11:00.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.885+0000 7f9f60ba0640 1 -- 172.21.15.16:0/3955011901 learned_addr learned my addr 172.21.15.16:0/3955011901 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:11:00.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.886+0000 7f9f5bfff640 1 --2- 172.21.15.16:0/3955011901 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9f5c1001a0 0x7f9f5c101120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:00.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.886+0000 7f9f5b7fe640 1 --2- 172.21.15.16:0/3955011901 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f5c100b50 0x7f9f5c101660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:00.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.886+0000 7f9f60ba0640 1 -- 172.21.15.16:0/3955011901 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9f5c1001a0 msgr2=0x7f9f5c101120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:00.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.886+0000 7f9f60ba0640 1 --2- 172.21.15.16:0/3955011901 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9f5c1001a0 0x7f9f5c101120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:00.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.886+0000 7f9f60ba0640 1 -- 172.21.15.16:0/3955011901 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f5c100b50 msgr2=0x7f9f5c101660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:00.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.886+0000 7f9f60ba0640 1 --2- 172.21.15.16:0/3955011901 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f5c100b50 0x7f9f5c101660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:00.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.886+0000 7f9f60ba0640 1 -- 172.21.15.16:0/3955011901 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f5c1a6710 con 0x7f9f5c106df0 2024-09-12T02:11:00.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.886+0000 7f9f60ba0640 1 --2- 172.21.15.16:0/3955011901 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f5c106df0 0x7f9f5c101ba0 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f9f5400eaa0 tx=0x7f9f5400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:00.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.886+0000 7f9f597fa640 1 -- 172.21.15.16:0/3955011901 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f5400cdb0 con 0x7f9f5c106df0 2024-09-12T02:11:00.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.886+0000 7f9f597fa640 1 -- 172.21.15.16:0/3955011901 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f54005020 con 0x7f9f5c106df0 2024-09-12T02:11:00.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.886+0000 7f9f613a1640 1 -- 172.21.15.16:0/3955011901 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f5c1a6a00 con 0x7f9f5c106df0 2024-09-12T02:11:00.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.886+0000 7f9f597fa640 1 -- 172.21.15.16:0/3955011901 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f540116b0 con 0x7f9f5c106df0 2024-09-12T02:11:00.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.886+0000 7f9f613a1640 1 -- 172.21.15.16:0/3955011901 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f5c1a6f10 con 0x7f9f5c106df0 2024-09-12T02:11:00.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.889+0000 7f9f597fa640 1 -- 172.21.15.16:0/3955011901 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f9f54005560 con 0x7f9f5c106df0 2024-09-12T02:11:00.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.889+0000 7f9f649dc740 1 Processor -- start 2024-09-12T02:11:00.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.889+0000 7f9f649dc740 1 -- start start 2024-09-12T02:11:00.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.889+0000 7f9f649dc740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e968938d20 0x55e968937270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:00.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.889+0000 7f9f649dc740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9689377b0 0x55e968937bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:00.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.889+0000 7f9f649dc740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e968942550 0x55e968944940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:00.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.889+0000 7f9f649dc740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e96889d2f0 con 0x55e968938d20 2024-09-12T02:11:00.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.889+0000 7f9f649dc740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55e968844cf0 con 0x55e9689377b0 2024-09-12T02:11:00.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.889+0000 7f9f649dc740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55e9688992b0 con 0x55e968942550 2024-09-12T02:11:00.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.889+0000 7f9f5b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e968938d20 0x55e968937270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:00.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.889+0000 7f9f5b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e968938d20 0x55e968937270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39894/0 (socket says 172.21.15.16:39894) 2024-09-12T02:11:00.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.889+0000 7f9f5b7fe640 1 -- 172.21.15.16:0/2262238038 learned_addr learned my addr 172.21.15.16:0/2262238038 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:11:00.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.889+0000 7f9f5bfff640 1 --2- 172.21.15.16:0/2262238038 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9689377b0 0x55e968937bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:00.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.890+0000 7f9f597fa640 1 --2- 172.21.15.16:0/3955011901 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f9f440a2b80 0x7f9f440a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:00.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.890+0000 7f9f5b7fe640 1 --2- 172.21.15.16:0/2262238038 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e968942550 0x55e968944940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:00.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.890+0000 7f9f5b7fe640 1 -- 172.21.15.16:0/2262238038 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e968942550 msgr2=0x55e968944940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:00.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.890+0000 7f9f5b7fe640 1 --2- 172.21.15.16:0/2262238038 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e968942550 0x55e968944940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:00.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.890+0000 7f9f5b7fe640 1 -- 172.21.15.16:0/2262238038 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9689377b0 msgr2=0x55e968937bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:00.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.890+0000 7f9f5b7fe640 1 --2- 172.21.15.16:0/2262238038 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9689377b0 0x55e968937bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:00.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.890+0000 7f9f5b7fe640 1 -- 172.21.15.16:0/2262238038 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9688293a0 con 0x55e968938d20 2024-09-12T02:11:00.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.890+0000 7f9f5bfff640 1 --2- 172.21.15.16:0/2262238038 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9689377b0 0x55e968937bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:11:00.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.890+0000 7f9f597fa640 1 -- 172.21.15.16:0/3955011901 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f9f540cf0d0 con 0x7f9f5c106df0 2024-09-12T02:11:00.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.890+0000 7f9f60ba0640 1 --2- 172.21.15.16:0/3955011901 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f9f440a2b80 0x7f9f440a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:00.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.890+0000 7f9f5b7fe640 1 --2- 172.21.15.16:0/2262238038 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e968938d20 0x55e968937270 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f9f50000c00 tx=0x7f9f50005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=642ed1f36af9b83f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:00.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.890+0000 7f9f427fc640 1 -- 172.21.15.16:0/2262238038 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f50035040 con 0x55e968938d20 2024-09-12T02:11:00.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.890+0000 7f9f427fc640 1 -- 172.21.15.16:0/2262238038 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f50042070 con 0x55e968938d20 2024-09-12T02:11:00.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.890+0000 7f9f427fc640 1 -- 172.21.15.16:0/2262238038 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f50004a00 con 0x55e968938d20 2024-09-12T02:11:00.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.891+0000 7f9f649dc740 1 -- 172.21.15.16:0/2262238038 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e968938d20 msgr2=0x55e968937270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:00.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.891+0000 7f9f60ba0640 1 --2- 172.21.15.16:0/3955011901 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f9f440a2b80 0x7f9f440a5040 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f9f5409b780 tx=0x7f9f54000f40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:00.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.891+0000 7f9f649dc740 1 --2- 172.21.15.16:0/2262238038 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e968938d20 0x55e968937270 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f9f50000c00 tx=0x7f9f50005fd0 comp rx=0 tx=0).stop 2024-09-12T02:11:00.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.891+0000 7f9f649dc740 1 -- 172.21.15.16:0/2262238038 shutdown_connections 2024-09-12T02:11:00.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.891+0000 7f9f649dc740 1 --2- 172.21.15.16:0/2262238038 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e968942550 0x55e968944940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:00.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.891+0000 7f9f649dc740 1 --2- 172.21.15.16:0/2262238038 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9689377b0 0x55e968937bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:00.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.891+0000 7f9f649dc740 1 --2- 172.21.15.16:0/2262238038 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e968938d20 0x55e968937270 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:00.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.891+0000 7f9f649dc740 1 -- 172.21.15.16:0/2262238038 >> 172.21.15.16:0/2262238038 conn(0x55e968900310 msgr2=0x55e968933e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:00.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.891+0000 7f9f649dc740 1 -- 172.21.15.16:0/2262238038 shutdown_connections 2024-09-12T02:11:00.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.891+0000 7f9f649dc740 1 -- 172.21.15.16:0/2262238038 wait complete. 2024-09-12T02:11:00.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.891+0000 7f9f649dc740 1 -- start start 2024-09-12T02:11:00.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.892+0000 7f9f649dc740 1 Processor -- start 2024-09-12T02:11:00.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.892+0000 7f9f649dc740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9689377b0 0x55e968941ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:00.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.892+0000 7f9f649dc740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e968938d20 0x55e96893f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:00.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.892+0000 7f9f649dc740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e968942550 0x55e96893f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:00.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.892+0000 7f9f649dc740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e9689451f0 con 0x55e968942550 2024-09-12T02:11:00.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.892+0000 7f9f649dc740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55e968858060 con 0x55e9689377b0 2024-09-12T02:11:00.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.892+0000 7f9f649dc740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55e968945370 con 0x55e968938d20 2024-09-12T02:11:00.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.892+0000 7f9f649dc740 10 client.0 _ll_register_callbacks cb 0x55e9685ab8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:11:00.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.892+0000 7f9f5b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e968942550 0x55e96893f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:00.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.892+0000 7f9f5b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e968942550 0x55e96893f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39908/0 (socket says 172.21.15.16:39908) 2024-09-12T02:11:00.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.892+0000 7f9f5b7fe640 1 -- 172.21.15.16:0/2209323875 learned_addr learned my addr 172.21.15.16:0/2209323875 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:11:00.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.892+0000 7f9f5bfff640 1 --2- 172.21.15.16:0/2209323875 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e968938d20 0x55e96893f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:00.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.892+0000 7f9f5b7fe640 1 --2- 172.21.15.16:0/2209323875 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9689377b0 0x55e968941ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:00.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.892+0000 7f9f5b7fe640 1 -- 172.21.15.16:0/2209323875 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e968938d20 msgr2=0x55e96893f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:00.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.892+0000 7f9f5b7fe640 1 --2- 172.21.15.16:0/2209323875 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e968938d20 0x55e96893f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:00.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.892+0000 7f9f5b7fe640 1 -- 172.21.15.16:0/2209323875 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9689377b0 msgr2=0x55e968941ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:00.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.892+0000 7f9f5b7fe640 1 --2- 172.21.15.16:0/2209323875 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9689377b0 0x55e968941ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:00.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.892+0000 7f9f5b7fe640 1 -- 172.21.15.16:0/2209323875 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e96860e3c0 con 0x55e968942550 2024-09-12T02:11:00.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.893+0000 7f9f5b7fe640 1 --2- 172.21.15.16:0/2209323875 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e968942550 0x55e96893f6b0 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f9f50031b30 tx=0x7f9f50040040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99742bb74df5f27e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:00.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.893+0000 7f9f3b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:11:00.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.893+0000 7f9f3b7fe640 1 -- 172.21.15.16:0/2209323875 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f50004020 con 0x55e968942550 2024-09-12T02:11:00.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.893+0000 7f9f3b7fe640 1 -- 172.21.15.16:0/2209323875 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f9f500041c0 con 0x55e968942550 2024-09-12T02:11:00.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.893+0000 7f9f649dc740 1 -- 172.21.15.16:0/2209323875 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f50042070 con 0x55e968942550 2024-09-12T02:11:00.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.893+0000 7f9f3b7fe640 1 -- 172.21.15.16:0/2209323875 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9f5005a050 con 0x55e968942550 2024-09-12T02:11:00.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.893+0000 7f9f649dc740 10 client.18213 fetch_fsmap learned FSMap version 25 2024-09-12T02:11:00.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.893+0000 7f9f649dc740 1 -- 172.21.15.16:0/2209323875 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e968950c00 con 0x55e968942550 2024-09-12T02:11:00.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.893+0000 7f9f3b7fe640 1 -- 172.21.15.16:0/2209323875 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f50042070 con 0x55e968942550 2024-09-12T02:11:00.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.893+0000 7f9f3b7fe640 1 -- 172.21.15.16:0/2209323875 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f9f500464d0 con 0x55e968942550 2024-09-12T02:11:00.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.893+0000 7f9f649dc740 10 client.18213 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:11:00.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.893+0000 7f9f649dc740 10 client.18213 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:11:00.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.893+0000 7f9f649dc740 20 client.18213 populate_metadata read hostname 'smithi016' 2024-09-12T02:11:00.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.893+0000 7f9f649dc740 1 --2- 172.21.15.16:0/2209323875 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55e968951f50 0x55e968954340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:00.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.894+0000 7f9f649dc740 4 client.18213 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:11:00.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.894+0000 7f9f649dc740 1 -- 172.21.15.16:0/2209323875 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e9688aa860 con 0x55e968951f50 2024-09-12T02:11:00.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.894+0000 7f9f5bfff640 1 --2- 172.21.15.16:0/2209323875 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55e968951f50 0x55e968954340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:00.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.894+0000 7f9f5bfff640 1 --2- 172.21.15.16:0/2209323875 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55e968951f50 0x55e968954340 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6065fa200c7d2a9 server_cookie=497b95a07b4bb3de in_seq=0 out_seq=0 2024-09-12T02:11:00.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.894+0000 7f9f3b7fe640 10 client.18213 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:11:00.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.895+0000 7f9f3b7fe640 1 -- 172.21.15.16:0/2209323875 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e9688aa860 con 0x55e968951f50 2024-09-12T02:11:00.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.895+0000 7f9f3b7fe640 10 client.18213 handle_command_reply: tid=0 2024-09-12T02:11:00.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.895+0000 7f9f649dc740 1 client.18213 shutdown 2024-09-12T02:11:00.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.895+0000 7f9f649dc740 1 -- 172.21.15.16:0/2209323875 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e968942550 msgr2=0x55e96893f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:00.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.895+0000 7f9f649dc740 1 --2- 172.21.15.16:0/2209323875 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e968942550 0x55e96893f6b0 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f9f50031b30 tx=0x7f9f50040040 comp rx=0 tx=0).stop 2024-09-12T02:11:00.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.895+0000 7f9f649dc740 1 -- 172.21.15.16:0/2209323875 shutdown_connections 2024-09-12T02:11:00.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.895+0000 7f9f649dc740 1 --2- 172.21.15.16:0/2209323875 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55e968951f50 0x55e968954340 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:00.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.895+0000 7f9f649dc740 1 --2- 172.21.15.16:0/2209323875 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e968942550 0x55e96893f6b0 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:00.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.895+0000 7f9f649dc740 1 --2- 172.21.15.16:0/2209323875 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e968938d20 0x55e96893f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:00.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.895+0000 7f9f649dc740 1 --2- 172.21.15.16:0/2209323875 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9689377b0 0x55e968941ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:00.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.895+0000 7f9f649dc740 1 -- 172.21.15.16:0/2209323875 >> 172.21.15.16:0/2209323875 conn(0x55e968900310 msgr2=0x55e968939b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:00.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.895+0000 7f9f3b7fe640 0 client.18213 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:11:00.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.895+0000 7f9f649dc740 1 -- 172.21.15.16:0/2209323875 shutdown_connections 2024-09-12T02:11:00.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.895+0000 7f9f649dc740 1 -- 172.21.15.16:0/2209323875 wait complete. 2024-09-12T02:11:00.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.895+0000 7f9f649dc740 20 client.18213 trim_cache size 0 max 16384 2024-09-12T02:11:00.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.908+0000 7f9f649dc740 1 Processor -- start 2024-09-12T02:11:00.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.908+0000 7f9f649dc740 1 -- start start 2024-09-12T02:11:00.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.908+0000 7f9f649dc740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9689377b0 0x55e968956f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:00.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.908+0000 7f9f649dc740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e968938d20 0x55e9689574c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:00.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.908+0000 7f9f649dc740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e968942550 0x55e96893c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:00.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.908+0000 7f9f649dc740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9f5c1b1c70 con 0x55e968938d20 2024-09-12T02:11:00.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.908+0000 7f9f649dc740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9f5c1af540 con 0x55e9689377b0 2024-09-12T02:11:00.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.908+0000 7f9f649dc740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f9f5c1ad3b0 con 0x55e968942550 2024-09-12T02:11:00.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.909+0000 7f9f5bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e968938d20 0x55e9689574c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:00.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.909+0000 7f9f5bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e968938d20 0x55e9689574c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39910/0 (socket says 172.21.15.16:39910) 2024-09-12T02:11:00.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.909+0000 7f9f5bfff640 1 -- 172.21.15.16:0/2935533277 learned_addr learned my addr 172.21.15.16:0/2935533277 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:11:00.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.909+0000 7f9f5b7fe640 1 --2- 172.21.15.16:0/2935533277 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9689377b0 0x55e968956f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:00.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.909+0000 7f9f5bfff640 1 -- 172.21.15.16:0/2935533277 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e968942550 msgr2=0x55e96893c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:00.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.909+0000 7f9f5bfff640 1 --2- 172.21.15.16:0/2935533277 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e968942550 0x55e96893c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:00.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.909+0000 7f9f5bfff640 1 -- 172.21.15.16:0/2935533277 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9689377b0 msgr2=0x55e968956f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:00.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.909+0000 7f9f5bfff640 1 --2- 172.21.15.16:0/2935533277 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9689377b0 0x55e968956f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:00.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.909+0000 7f9f5bfff640 1 -- 172.21.15.16:0/2935533277 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e968941ac0 con 0x55e968938d20 2024-09-12T02:11:00.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.909+0000 7f9f5b7fe640 1 --2- 172.21.15.16:0/2935533277 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9689377b0 0x55e968956f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:11:00.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.909+0000 7f9f5bfff640 1 --2- 172.21.15.16:0/2935533277 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e968938d20 0x55e9689574c0 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x55e96893e9e0 tx=0x7f9f4c00b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bca4f322701bbe2a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:00.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.909+0000 7f9f41ffb640 1 -- 172.21.15.16:0/2935533277 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f4c003d20 con 0x55e968938d20 2024-09-12T02:11:00.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.909+0000 7f9f41ffb640 1 -- 172.21.15.16:0/2935533277 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f4c011920 con 0x55e968938d20 2024-09-12T02:11:00.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.909+0000 7f9f41ffb640 1 -- 172.21.15.16:0/2935533277 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f4c01d810 con 0x55e968938d20 2024-09-12T02:11:00.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.910+0000 7f9f649dc740 1 -- 172.21.15.16:0/2935533277 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e968938d20 msgr2=0x55e9689574c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:00.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.910+0000 7f9f649dc740 1 --2- 172.21.15.16:0/2935533277 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e968938d20 0x55e9689574c0 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x55e96893e9e0 tx=0x7f9f4c00b390 comp rx=0 tx=0).stop 2024-09-12T02:11:00.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.910+0000 7f9f649dc740 1 -- 172.21.15.16:0/2935533277 shutdown_connections 2024-09-12T02:11:00.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.910+0000 7f9f649dc740 1 --2- 172.21.15.16:0/2935533277 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e968942550 0x55e96893c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:00.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.910+0000 7f9f649dc740 1 --2- 172.21.15.16:0/2935533277 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e968938d20 0x55e9689574c0 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:00.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.910+0000 7f9f649dc740 1 --2- 172.21.15.16:0/2935533277 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9689377b0 0x55e968956f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:00.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.910+0000 7f9f649dc740 1 -- 172.21.15.16:0/2935533277 >> 172.21.15.16:0/2935533277 conn(0x55e968900310 msgr2=0x55e96893f110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:00.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.910+0000 7f9f649dc740 1 -- 172.21.15.16:0/2935533277 shutdown_connections 2024-09-12T02:11:00.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.910+0000 7f9f649dc740 1 -- 172.21.15.16:0/2935533277 wait complete. 2024-09-12T02:11:00.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.910+0000 7f9f649dc740 1 -- start start 2024-09-12T02:11:00.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.910+0000 7f9f649dc740 1 Processor -- start 2024-09-12T02:11:00.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.911+0000 7f9f649dc740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9689377b0 0x55e968953e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:00.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.911+0000 7f9f649dc740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e968938d20 0x55e968954390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:00.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.911+0000 7f9f649dc740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e968942550 0x55e9689548d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:00.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.911+0000 7f9f649dc740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9f5c1ada00 con 0x55e968942550 2024-09-12T02:11:00.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.911+0000 7f9f649dc740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9f5c1aee70 con 0x55e9689377b0 2024-09-12T02:11:00.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.911+0000 7f9f649dc740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f9f5c1ae0d0 con 0x55e968938d20 2024-09-12T02:11:00.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.911+0000 7f9f649dc740 10 client.0 _ll_register_callbacks cb 0x7f9f500056a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:11:00.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.911+0000 7f9f5b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e968942550 0x55e9689548d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:00.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.911+0000 7f9f5b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e968942550 0x55e9689548d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39926/0 (socket says 172.21.15.16:39926) 2024-09-12T02:11:00.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.911+0000 7f9f5b7fe640 1 -- 172.21.15.16:0/3252498968 learned_addr learned my addr 172.21.15.16:0/3252498968 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:11:00.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.911+0000 7f9f5bfff640 1 --2- 172.21.15.16:0/3252498968 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e968938d20 0x55e968954390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:00.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.911+0000 7f9f5b7fe640 1 --2- 172.21.15.16:0/3252498968 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9689377b0 0x55e968953e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:00.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.911+0000 7f9f5b7fe640 1 -- 172.21.15.16:0/3252498968 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e968938d20 msgr2=0x55e968954390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:00.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.911+0000 7f9f5b7fe640 1 --2- 172.21.15.16:0/3252498968 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e968938d20 0x55e968954390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:00.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.911+0000 7f9f5b7fe640 1 -- 172.21.15.16:0/3252498968 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9689377b0 msgr2=0x55e968953e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:00.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.911+0000 7f9f5b7fe640 1 --2- 172.21.15.16:0/3252498968 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9689377b0 0x55e968953e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:00.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.911+0000 7f9f5b7fe640 1 -- 172.21.15.16:0/3252498968 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9689ba980 con 0x55e968942550 2024-09-12T02:11:00.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.911+0000 7f9f5b7fe640 1 --2- 172.21.15.16:0/3252498968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e968942550 0x55e9689548d0 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f9f50005e90 tx=0x7f9f5002a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bfb0961ac5f51b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:00.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.911+0000 7f9f3b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:11:00.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.912+0000 7f9f3b7fe640 1 -- 172.21.15.16:0/3252498968 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f50035040 con 0x55e968942550 2024-09-12T02:11:00.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.912+0000 7f9f3b7fe640 1 -- 172.21.15.16:0/3252498968 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f9f50053450 con 0x55e968942550 2024-09-12T02:11:00.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.912+0000 7f9f649dc740 1 -- 172.21.15.16:0/3252498968 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f4c011920 con 0x55e968942550 2024-09-12T02:11:00.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.912+0000 7f9f3b7fe640 1 -- 172.21.15.16:0/3252498968 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9f50005500 con 0x55e968942550 2024-09-12T02:11:00.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.912+0000 7f9f649dc740 10 client.18219 fetch_fsmap learned FSMap version 25 2024-09-12T02:11:00.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.912+0000 7f9f649dc740 1 -- 172.21.15.16:0/3252498968 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e9689bbbd0 con 0x55e968942550 2024-09-12T02:11:00.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.912+0000 7f9f3b7fe640 1 -- 172.21.15.16:0/3252498968 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f4c011920 con 0x55e968942550 2024-09-12T02:11:00.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.912+0000 7f9f3b7fe640 1 -- 172.21.15.16:0/3252498968 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f9f5005e430 con 0x55e968942550 2024-09-12T02:11:00.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.912+0000 7f9f649dc740 10 client.18219 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:11:00.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.912+0000 7f9f649dc740 10 client.18219 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:11:00.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.912+0000 7f9f649dc740 20 client.18219 populate_metadata read hostname 'smithi016' 2024-09-12T02:11:00.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.912+0000 7f9f649dc740 1 --2- 172.21.15.16:0/3252498968 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55e9689bc2f0 0x55e9689be710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:00.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.912+0000 7f9f649dc740 4 client.18219 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T02:11:00.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.912+0000 7f9f649dc740 1 -- 172.21.15.16:0/3252498968 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e9688a4d70 con 0x55e9689bc2f0 2024-09-12T02:11:00.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.913+0000 7f9f5bfff640 1 --2- 172.21.15.16:0/3252498968 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55e9689bc2f0 0x55e9689be710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:00.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.913+0000 7f9f5bfff640 1 --2- 172.21.15.16:0/3252498968 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55e9689bc2f0 0x55e9689be710 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8b67be1a626edee server_cookie=5df9d82983d71f7d in_seq=0 out_seq=0 2024-09-12T02:11:00.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.913+0000 7f9f3b7fe640 10 client.18219 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:11:00.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.913+0000 7f9f3b7fe640 1 -- 172.21.15.16:0/3252498968 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e9688a4d70 con 0x55e9689bc2f0 2024-09-12T02:11:00.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.913+0000 7f9f3b7fe640 10 client.18219 handle_command_reply: tid=0 2024-09-12T02:11:00.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.913+0000 7f9f649dc740 1 client.18219 shutdown 2024-09-12T02:11:00.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.913+0000 7f9f649dc740 1 -- 172.21.15.16:0/3252498968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e968942550 msgr2=0x55e9689548d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:00.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.913+0000 7f9f649dc740 1 --2- 172.21.15.16:0/3252498968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e968942550 0x55e9689548d0 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f9f50005e90 tx=0x7f9f5002a420 comp rx=0 tx=0).stop 2024-09-12T02:11:00.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.913+0000 7f9f649dc740 1 -- 172.21.15.16:0/3252498968 shutdown_connections 2024-09-12T02:11:00.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.913+0000 7f9f649dc740 1 --2- 172.21.15.16:0/3252498968 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55e9689bc2f0 0x55e9689be710 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:00.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.914+0000 7f9f649dc740 1 --2- 172.21.15.16:0/3252498968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e968942550 0x55e9689548d0 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:00.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.914+0000 7f9f3b7fe640 0 client.18219 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:11:00.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.914+0000 7f9f649dc740 1 --2- 172.21.15.16:0/3252498968 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e968938d20 0x55e968954390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:00.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.914+0000 7f9f649dc740 1 --2- 172.21.15.16:0/3252498968 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9689377b0 0x55e968953e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:00.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.914+0000 7f9f649dc740 1 -- 172.21.15.16:0/3252498968 >> 172.21.15.16:0/3252498968 conn(0x55e968900310 msgr2=0x55e96893da20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:00.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.914+0000 7f9f649dc740 1 -- 172.21.15.16:0/3252498968 shutdown_connections 2024-09-12T02:11:00.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.914+0000 7f9f649dc740 1 -- 172.21.15.16:0/3252498968 wait complete. 2024-09-12T02:11:00.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.914+0000 7f9f649dc740 20 client.18219 trim_cache size 0 max 16384 2024-09-12T02:11:00.922 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T02:11:00.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.914+0000 7f9f613a1640 1 -- 172.21.15.16:0/3955011901 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f9f440a2b80 msgr2=0x7f9f440a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:00.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.914+0000 7f9f613a1640 1 --2- 172.21.15.16:0/3955011901 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f9f440a2b80 0x7f9f440a5040 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f9f5409b780 tx=0x7f9f54000f40 comp rx=0 tx=0).stop 2024-09-12T02:11:00.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.914+0000 7f9f613a1640 1 -- 172.21.15.16:0/3955011901 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f5c106df0 msgr2=0x7f9f5c101ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:00.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.914+0000 7f9f613a1640 1 --2- 172.21.15.16:0/3955011901 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f5c106df0 0x7f9f5c101ba0 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f9f5400eaa0 tx=0x7f9f5400ef60 comp rx=0 tx=0).stop 2024-09-12T02:11:00.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.914+0000 7f9f613a1640 1 -- 172.21.15.16:0/3955011901 shutdown_connections 2024-09-12T02:11:00.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.914+0000 7f9f613a1640 1 --2- 172.21.15.16:0/3955011901 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f9f440a2b80 0x7f9f440a5040 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:00.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.914+0000 7f9f613a1640 1 --2- 172.21.15.16:0/3955011901 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f5c106df0 0x7f9f5c101ba0 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:00.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.914+0000 7f9f613a1640 1 --2- 172.21.15.16:0/3955011901 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f5c100b50 0x7f9f5c101660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:00.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.914+0000 7f9f613a1640 1 --2- 172.21.15.16:0/3955011901 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9f5c1001a0 0x7f9f5c101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:00.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.914+0000 7f9f613a1640 1 -- 172.21.15.16:0/3955011901 >> 172.21.15.16:0/3955011901 conn(0x7f9f5c077070 msgr2=0x7f9f5c0fc6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:00.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.915+0000 7f9f613a1640 1 -- 172.21.15.16:0/3955011901 shutdown_connections 2024-09-12T02:11:00.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:00.915+0000 7f9f613a1640 1 -- 172.21.15.16:0/3955011901 wait complete. 2024-09-12T02:11:02.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:11:02 smithi184 ceph-mon[35593]: pgmap v608: 97 pgs: 97 active+clean; 773 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-09-12T02:11:02.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:11:02 smithi033 ceph-mon[35332]: pgmap v608: 97 pgs: 97 active+clean; 773 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-09-12T02:11:02.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:11:02 smithi016 ceph-mon[30947]: pgmap v608: 97 pgs: 97 active+clean; 773 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-09-12T02:11:02.643 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-12T02:11:02.643 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-12T02:11:02.913 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:11:03.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.393+0000 7f303c3fb640 1 -- 172.21.15.16:0/2050606399 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3034106a80 msgr2=0x7f3034106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:03.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.393+0000 7f303c3fb640 1 --2- 172.21.15.16:0/2050606399 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3034106a80 0x7f3034106e60 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f3030009930 tx=0x7f303002a420 comp rx=0 tx=0).stop 2024-09-12T02:11:03.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.393+0000 7f303c3fb640 1 -- 172.21.15.16:0/2050606399 shutdown_connections 2024-09-12T02:11:03.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.393+0000 7f303c3fb640 1 --2- 172.21.15.16:0/2050606399 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3034101420 0x7f303410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:03.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.393+0000 7f303c3fb640 1 --2- 172.21.15.16:0/2050606399 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3034100a80 0x7f3034100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:03.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.393+0000 7f303c3fb640 1 --2- 172.21.15.16:0/2050606399 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3034106a80 0x7f3034106e60 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:03.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.393+0000 7f303c3fb640 1 -- 172.21.15.16:0/2050606399 >> 172.21.15.16:0/2050606399 conn(0x7f30340fc7a0 msgr2=0x7f30340febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:03.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.393+0000 7f303c3fb640 1 -- 172.21.15.16:0/2050606399 shutdown_connections 2024-09-12T02:11:03.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.393+0000 7f303c3fb640 1 -- 172.21.15.16:0/2050606399 wait complete. 2024-09-12T02:11:03.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.394+0000 7f303c3fb640 1 Processor -- start 2024-09-12T02:11:03.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.394+0000 7f303c3fb640 1 -- start start 2024-09-12T02:11:03.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.394+0000 7f303c3fb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3034100a80 0x7f3034198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:03.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.394+0000 7f303c3fb640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3034101420 0x7f3034198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:03.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.394+0000 7f303c3fb640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3034106a80 0x7f303419cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:03.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.394+0000 7f303c3fb640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3034110e80 con 0x7f3034100a80 2024-09-12T02:11:03.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.394+0000 7f303c3fb640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3034110d00 con 0x7f3034101420 2024-09-12T02:11:03.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.394+0000 7f303c3fb640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f3034111000 con 0x7f3034106a80 2024-09-12T02:11:03.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.394+0000 7f303a197640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3034100a80 0x7f3034198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:03.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.394+0000 7f303a197640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3034100a80 0x7f3034198450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39218/0 (socket says 172.21.15.16:39218) 2024-09-12T02:11:03.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.394+0000 7f303a197640 1 -- 172.21.15.16:0/3269231629 learned_addr learned my addr 172.21.15.16:0/3269231629 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:11:03.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.395+0000 7f3039996640 1 --2- 172.21.15.16:0/3269231629 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3034101420 0x7f3034198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:03.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.395+0000 7f303a998640 1 --2- 172.21.15.16:0/3269231629 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3034106a80 0x7f303419cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:03.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.395+0000 7f303a197640 1 -- 172.21.15.16:0/3269231629 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3034106a80 msgr2=0x7f303419cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:03.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.395+0000 7f303a197640 1 --2- 172.21.15.16:0/3269231629 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3034106a80 0x7f303419cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:03.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.395+0000 7f303a197640 1 -- 172.21.15.16:0/3269231629 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3034101420 msgr2=0x7f3034198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:03.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.395+0000 7f303a197640 1 --2- 172.21.15.16:0/3269231629 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3034101420 0x7f3034198990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:03.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.395+0000 7f303a197640 1 -- 172.21.15.16:0/3269231629 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f303419d400 con 0x7f3034100a80 2024-09-12T02:11:03.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.395+0000 7f303a998640 1 --2- 172.21.15.16:0/3269231629 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3034106a80 0x7f303419cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:11:03.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.395+0000 7f3039996640 1 --2- 172.21.15.16:0/3269231629 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3034101420 0x7f3034198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:11:03.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.395+0000 7f303a197640 1 --2- 172.21.15.16:0/3269231629 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3034100a80 0x7f3034198450 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f3030009230 tx=0x7f3030004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:03.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.395+0000 7f30277fe640 1 -- 172.21.15.16:0/3269231629 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f303003f030 con 0x7f3034100a80 2024-09-12T02:11:03.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.395+0000 7f30277fe640 1 -- 172.21.15.16:0/3269231629 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3030003ad0 con 0x7f3034100a80 2024-09-12T02:11:03.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.395+0000 7f303c3fb640 1 -- 172.21.15.16:0/3269231629 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f303419d690 con 0x7f3034100a80 2024-09-12T02:11:03.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.395+0000 7f30277fe640 1 -- 172.21.15.16:0/3269231629 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3030035070 con 0x7f3034100a80 2024-09-12T02:11:03.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.395+0000 7f303c3fb640 1 -- 172.21.15.16:0/3269231629 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30341a4fd0 con 0x7f3034100a80 2024-09-12T02:11:03.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.398+0000 7f30277fe640 1 -- 172.21.15.16:0/3269231629 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f3030042030 con 0x7f3034100a80 2024-09-12T02:11:03.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.398+0000 7f303e7d4740 1 Processor -- start 2024-09-12T02:11:03.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.399+0000 7f303e7d4740 1 -- start start 2024-09-12T02:11:03.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.399+0000 7f303e7d4740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555cc43dfa60 0x555cc43dfe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:03.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.399+0000 7f303e7d4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555cc43e0380 0x555cc43eec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:03.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.399+0000 7f303e7d4740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555cc43ef150 0x555cc43f1540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:03.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.399+0000 7f303e7d4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x555cc4345280 con 0x555cc43e0380 2024-09-12T02:11:03.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.399+0000 7f303e7d4740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x555cc42ecb10 con 0x555cc43ef150 2024-09-12T02:11:03.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.399+0000 7f303e7d4740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x555cc4341240 con 0x555cc43dfa60 2024-09-12T02:11:03.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.399+0000 7f303a197640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555cc43e0380 0x555cc43eec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:03.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.399+0000 7f303a197640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555cc43e0380 0x555cc43eec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39232/0 (socket says 172.21.15.16:39232) 2024-09-12T02:11:03.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.399+0000 7f303a197640 1 -- 172.21.15.16:0/292185956 learned_addr learned my addr 172.21.15.16:0/292185956 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:11:03.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.399+0000 7f303a998640 1 --2- 172.21.15.16:0/292185956 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555cc43dfa60 0x555cc43dfe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:03.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.399+0000 7f3039996640 1 --2- 172.21.15.16:0/292185956 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555cc43ef150 0x555cc43f1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:03.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.399+0000 7f30277fe640 1 --2- 172.21.15.16:0/3269231629 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f30200a2b80 0x7f30200a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:03.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.399+0000 7f303a197640 1 -- 172.21.15.16:0/292185956 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555cc43dfa60 msgr2=0x555cc43dfe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:03.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.399+0000 7f303a197640 1 --2- 172.21.15.16:0/292185956 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555cc43dfa60 0x555cc43dfe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:03.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.399+0000 7f303a197640 1 -- 172.21.15.16:0/292185956 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555cc43ef150 msgr2=0x555cc43f1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:03.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.400+0000 7f303a197640 1 --2- 172.21.15.16:0/292185956 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555cc43ef150 0x555cc43f1540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:03.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.400+0000 7f303a197640 1 -- 172.21.15.16:0/292185956 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555cc42f91f0 con 0x555cc43e0380 2024-09-12T02:11:03.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.400+0000 7f303a998640 1 --2- 172.21.15.16:0/3269231629 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f30200a2b80 0x7f30200a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:03.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.400+0000 7f3039996640 1 --2- 172.21.15.16:0/292185956 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555cc43ef150 0x555cc43f1540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:11:03.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.400+0000 7f30277fe640 1 -- 172.21.15.16:0/3269231629 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f30300ef540 con 0x7f3034100a80 2024-09-12T02:11:03.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.400+0000 7f303a197640 1 --2- 172.21.15.16:0/292185956 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555cc43e0380 0x555cc43eec10 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f30300b0810 tx=0x7f30300b1320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dc944271a105750 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:03.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.400+0000 7f303a998640 1 --2- 172.21.15.16:0/3269231629 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f30200a2b80 0x7f30200a5040 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f302c005fd0 tx=0x7f302c004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:03.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.400+0000 7f3024dea640 1 -- 172.21.15.16:0/292185956 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30300bb980 con 0x555cc43e0380 2024-09-12T02:11:03.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.400+0000 7f3024dea640 1 -- 172.21.15.16:0/292185956 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30300b1fb0 con 0x555cc43e0380 2024-09-12T02:11:03.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.400+0000 7f3024dea640 1 -- 172.21.15.16:0/292185956 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30300bb7a0 con 0x555cc43e0380 2024-09-12T02:11:03.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.401+0000 7f303e7d4740 1 -- 172.21.15.16:0/292185956 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555cc43e0380 msgr2=0x555cc43eec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:03.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.401+0000 7f303e7d4740 1 --2- 172.21.15.16:0/292185956 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555cc43e0380 0x555cc43eec10 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f30300b0810 tx=0x7f30300b1320 comp rx=0 tx=0).stop 2024-09-12T02:11:03.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.401+0000 7f303e7d4740 1 -- 172.21.15.16:0/292185956 shutdown_connections 2024-09-12T02:11:03.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.401+0000 7f303e7d4740 1 --2- 172.21.15.16:0/292185956 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555cc43ef150 0x555cc43f1540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:03.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.401+0000 7f303e7d4740 1 --2- 172.21.15.16:0/292185956 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555cc43e0380 0x555cc43eec10 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:03.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.401+0000 7f303e7d4740 1 --2- 172.21.15.16:0/292185956 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555cc43dfa60 0x555cc43dfe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:03.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.401+0000 7f303e7d4740 1 -- 172.21.15.16:0/292185956 >> 172.21.15.16:0/292185956 conn(0x555cc43dc0f0 msgr2=0x555cc43dc4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:03.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.401+0000 7f303e7d4740 1 -- 172.21.15.16:0/292185956 shutdown_connections 2024-09-12T02:11:03.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.401+0000 7f303e7d4740 1 -- 172.21.15.16:0/292185956 wait complete. 2024-09-12T02:11:03.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.401+0000 7f303e7d4740 1 -- start start 2024-09-12T02:11:03.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.402+0000 7f303e7d4740 1 Processor -- start 2024-09-12T02:11:03.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.402+0000 7f303e7d4740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555cc43dfa60 0x555cc43ec040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:03.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.402+0000 7f303e7d4740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555cc43e0380 0x555cc43ec580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:03.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.402+0000 7f303e7d4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555cc43ef150 0x555cc43ecac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:03.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.402+0000 7f303e7d4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x555cc43f1b40 con 0x555cc43ef150 2024-09-12T02:11:03.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.402+0000 7f303e7d4740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x555cc42fff10 con 0x555cc43dfa60 2024-09-12T02:11:03.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.402+0000 7f303e7d4740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x555cc43f1cc0 con 0x555cc43e0380 2024-09-12T02:11:03.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.402+0000 7f303e7d4740 10 client.0 _ll_register_callbacks cb 0x555cc42c8a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:11:03.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.402+0000 7f303a998640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555cc43ef150 0x555cc43ecac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:03.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.402+0000 7f303a998640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555cc43ef150 0x555cc43ecac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39238/0 (socket says 172.21.15.16:39238) 2024-09-12T02:11:03.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.402+0000 7f303a998640 1 -- 172.21.15.16:0/4060058087 learned_addr learned my addr 172.21.15.16:0/4060058087 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:11:03.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.402+0000 7f3039996640 1 --2- 172.21.15.16:0/4060058087 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555cc43e0380 0x555cc43ec580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:03.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.402+0000 7f303a197640 1 --2- 172.21.15.16:0/4060058087 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555cc43dfa60 0x555cc43ec040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:03.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.402+0000 7f303a998640 1 -- 172.21.15.16:0/4060058087 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555cc43e0380 msgr2=0x555cc43ec580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:03.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.402+0000 7f303a998640 1 --2- 172.21.15.16:0/4060058087 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555cc43e0380 0x555cc43ec580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:03.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.402+0000 7f303a998640 1 -- 172.21.15.16:0/4060058087 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555cc43dfa60 msgr2=0x555cc43ec040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:03.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.402+0000 7f303a998640 1 --2- 172.21.15.16:0/4060058087 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555cc43dfa60 0x555cc43ec040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:03.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.402+0000 7f303a998640 1 -- 172.21.15.16:0/4060058087 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555cc40b66a0 con 0x555cc43ef150 2024-09-12T02:11:03.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.403+0000 7f303a998640 1 --2- 172.21.15.16:0/4060058087 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555cc43ef150 0x555cc43ecac0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f302c0138e0 tx=0x7f302c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d13371724befc9ad server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:03.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.403+0000 7f301dffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:11:03.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.403+0000 7f301dffb640 1 -- 172.21.15.16:0/4060058087 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f302c013e60 con 0x555cc43ef150 2024-09-12T02:11:03.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.403+0000 7f301dffb640 1 -- 172.21.15.16:0/4060058087 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f302c004d20 con 0x555cc43ef150 2024-09-12T02:11:03.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.403+0000 7f303e7d4740 1 -- 172.21.15.16:0/4060058087 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x555cc43527f0 con 0x555cc43ef150 2024-09-12T02:11:03.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.403+0000 7f301dffb640 1 -- 172.21.15.16:0/4060058087 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f302c01eaf0 con 0x555cc43ef150 2024-09-12T02:11:03.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.403+0000 7f303e7d4740 10 client.18231 fetch_fsmap learned FSMap version 25 2024-09-12T02:11:03.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.403+0000 7f303e7d4740 1 -- 172.21.15.16:0/4060058087 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555cc43fd6e0 con 0x555cc43ef150 2024-09-12T02:11:03.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.403+0000 7f301dffb640 1 -- 172.21.15.16:0/4060058087 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x555cc43527f0 con 0x555cc43ef150 2024-09-12T02:11:03.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.403+0000 7f301dffb640 1 -- 172.21.15.16:0/4060058087 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f302c00f040 con 0x555cc43ef150 2024-09-12T02:11:03.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.403+0000 7f303e7d4740 10 client.18231 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:11:03.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.403+0000 7f303e7d4740 10 client.18231 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:11:03.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.403+0000 7f303e7d4740 20 client.18231 populate_metadata read hostname 'smithi016' 2024-09-12T02:11:03.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.403+0000 7f303e7d4740 1 --2- 172.21.15.16:0/4060058087 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x555cc43febf0 0x555cc4401030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:03.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.403+0000 7f303e7d4740 4 client.18231 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:11:03.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.403+0000 7f303e7d4740 1 -- 172.21.15.16:0/4060058087 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555cc434cd00 con 0x555cc43febf0 2024-09-12T02:11:03.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.404+0000 7f303a197640 1 --2- 172.21.15.16:0/4060058087 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x555cc43febf0 0x555cc4401030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:03.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.404+0000 7f303a197640 1 --2- 172.21.15.16:0/4060058087 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x555cc43febf0 0x555cc4401030 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b5e0a60e880fd8a server_cookie=24203319eb757a39 in_seq=0 out_seq=0 2024-09-12T02:11:03.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.404+0000 7f301dffb640 10 client.18231 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:11:03.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.405+0000 7f301dffb640 1 -- 172.21.15.16:0/4060058087 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555cc434cd00 con 0x555cc43febf0 2024-09-12T02:11:03.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.405+0000 7f301dffb640 10 client.18231 handle_command_reply: tid=0 2024-09-12T02:11:03.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.405+0000 7f303e7d4740 1 client.18231 shutdown 2024-09-12T02:11:03.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.405+0000 7f303e7d4740 1 -- 172.21.15.16:0/4060058087 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555cc43ef150 msgr2=0x555cc43ecac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:03.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.405+0000 7f303e7d4740 1 --2- 172.21.15.16:0/4060058087 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555cc43ef150 0x555cc43ecac0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f302c0138e0 tx=0x7f302c004ab0 comp rx=0 tx=0).stop 2024-09-12T02:11:03.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.405+0000 7f303e7d4740 1 -- 172.21.15.16:0/4060058087 shutdown_connections 2024-09-12T02:11:03.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.405+0000 7f303e7d4740 1 --2- 172.21.15.16:0/4060058087 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x555cc43febf0 0x555cc4401030 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:03.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.405+0000 7f303e7d4740 1 --2- 172.21.15.16:0/4060058087 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555cc43ef150 0x555cc43ecac0 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:03.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.405+0000 7f303e7d4740 1 --2- 172.21.15.16:0/4060058087 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555cc43e0380 0x555cc43ec580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:03.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.405+0000 7f301dffb640 0 client.18231 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:11:03.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.405+0000 7f303e7d4740 1 --2- 172.21.15.16:0/4060058087 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555cc43dfa60 0x555cc43ec040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:03.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.405+0000 7f303e7d4740 1 -- 172.21.15.16:0/4060058087 >> 172.21.15.16:0/4060058087 conn(0x555cc43dc0f0 msgr2=0x555cc43e1e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:03.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.405+0000 7f303e7d4740 1 -- 172.21.15.16:0/4060058087 shutdown_connections 2024-09-12T02:11:03.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.405+0000 7f303e7d4740 1 -- 172.21.15.16:0/4060058087 wait complete. 2024-09-12T02:11:03.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.405+0000 7f303e7d4740 20 client.18231 trim_cache size 0 max 16384 2024-09-12T02:11:03.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.423+0000 7f303e7d4740 1 Processor -- start 2024-09-12T02:11:03.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.423+0000 7f303e7d4740 1 -- start start 2024-09-12T02:11:03.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.423+0000 7f303e7d4740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555cc43dfa60 0x555cc43e1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:03.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.423+0000 7f303e7d4740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555cc43e0380 0x555cc4406900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:03.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.423+0000 7f303e7d4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555cc43ef150 0x555cc4406e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:03.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.423+0000 7f303e7d4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f30341adb50 con 0x555cc43ef150 2024-09-12T02:11:03.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.423+0000 7f303e7d4740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f30341ab420 con 0x555cc43e0380 2024-09-12T02:11:03.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.423+0000 7f303e7d4740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f30341a9290 con 0x555cc43dfa60 2024-09-12T02:11:03.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.423+0000 7f303a998640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555cc43ef150 0x555cc4406e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:03.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.423+0000 7f303a998640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555cc43ef150 0x555cc4406e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39254/0 (socket says 172.21.15.16:39254) 2024-09-12T02:11:03.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.423+0000 7f303a998640 1 -- 172.21.15.16:0/471696502 learned_addr learned my addr 172.21.15.16:0/471696502 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:11:03.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.423+0000 7f3039996640 1 --2- 172.21.15.16:0/471696502 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555cc43e0380 0x555cc4406900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:03.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.423+0000 7f303a197640 1 --2- 172.21.15.16:0/471696502 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555cc43dfa60 0x555cc43e1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:03.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.423+0000 7f303a998640 1 -- 172.21.15.16:0/471696502 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555cc43dfa60 msgr2=0x555cc43e1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:03.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.423+0000 7f303a998640 1 --2- 172.21.15.16:0/471696502 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555cc43dfa60 0x555cc43e1d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:03.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.423+0000 7f303a998640 1 -- 172.21.15.16:0/471696502 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555cc43e0380 msgr2=0x555cc4406900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:03.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.423+0000 7f303a998640 1 --2- 172.21.15.16:0/471696502 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555cc43e0380 0x555cc4406900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:03.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.423+0000 7f303a998640 1 -- 172.21.15.16:0/471696502 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555cc43eaff0 con 0x555cc43ef150 2024-09-12T02:11:03.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.423+0000 7f303a197640 1 --2- 172.21.15.16:0/471696502 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555cc43dfa60 0x555cc43e1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:11:03.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.424+0000 7f303a998640 1 --2- 172.21.15.16:0/471696502 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555cc43ef150 0x555cc4406e40 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f302c01e830 tx=0x7f302c01e860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e76bdd23175a94a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:03.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.424+0000 7f301ffff640 1 -- 172.21.15.16:0/471696502 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f302c01ec90 con 0x555cc43ef150 2024-09-12T02:11:03.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.424+0000 7f301ffff640 1 -- 172.21.15.16:0/471696502 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f302c023070 con 0x555cc43ef150 2024-09-12T02:11:03.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.424+0000 7f301ffff640 1 -- 172.21.15.16:0/471696502 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f302c027870 con 0x555cc43ef150 2024-09-12T02:11:03.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.424+0000 7f303e7d4740 1 -- 172.21.15.16:0/471696502 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555cc43ef150 msgr2=0x555cc4406e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:03.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.424+0000 7f303e7d4740 1 --2- 172.21.15.16:0/471696502 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555cc43ef150 0x555cc4406e40 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f302c01e830 tx=0x7f302c01e860 comp rx=0 tx=0).stop 2024-09-12T02:11:03.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.424+0000 7f303e7d4740 1 -- 172.21.15.16:0/471696502 shutdown_connections 2024-09-12T02:11:03.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.424+0000 7f303e7d4740 1 --2- 172.21.15.16:0/471696502 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555cc43ef150 0x555cc4406e40 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:03.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.424+0000 7f303e7d4740 1 --2- 172.21.15.16:0/471696502 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555cc43e0380 0x555cc4406900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:03.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.424+0000 7f303e7d4740 1 --2- 172.21.15.16:0/471696502 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555cc43dfa60 0x555cc43e1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:03.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.424+0000 7f303e7d4740 1 -- 172.21.15.16:0/471696502 >> 172.21.15.16:0/471696502 conn(0x555cc43dc0f0 msgr2=0x555cc43e1630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:03.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.424+0000 7f303e7d4740 1 -- 172.21.15.16:0/471696502 shutdown_connections 2024-09-12T02:11:03.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.424+0000 7f303e7d4740 1 -- 172.21.15.16:0/471696502 wait complete. 2024-09-12T02:11:03.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.425+0000 7f303e7d4740 1 -- start start 2024-09-12T02:11:03.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.425+0000 7f303e7d4740 1 Processor -- start 2024-09-12T02:11:03.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.425+0000 7f303e7d4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555cc43dfa60 0x555cc43f2c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:03.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.425+0000 7f303e7d4740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555cc43e0380 0x555cc43f3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:03.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.425+0000 7f303e7d4740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555cc43ef150 0x555cc43f36d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:03.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.425+0000 7f303e7d4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f30341a98e0 con 0x555cc43dfa60 2024-09-12T02:11:03.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.425+0000 7f303e7d4740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f30341aad50 con 0x555cc43ef150 2024-09-12T02:11:03.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.425+0000 7f303e7d4740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f30341a9fb0 con 0x555cc43e0380 2024-09-12T02:11:03.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.425+0000 7f303e7d4740 10 client.0 _ll_register_callbacks cb 0x7f302c009cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:11:03.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.425+0000 7f303a197640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555cc43dfa60 0x555cc43f2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:03.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.425+0000 7f303a197640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555cc43dfa60 0x555cc43f2c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39262/0 (socket says 172.21.15.16:39262) 2024-09-12T02:11:03.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.425+0000 7f303a197640 1 -- 172.21.15.16:0/3947822713 learned_addr learned my addr 172.21.15.16:0/3947822713 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:11:03.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.425+0000 7f303a998640 1 --2- 172.21.15.16:0/3947822713 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555cc43ef150 0x555cc43f36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:03.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.425+0000 7f3039996640 1 --2- 172.21.15.16:0/3947822713 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555cc43e0380 0x555cc43f3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:03.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.426+0000 7f303a197640 1 -- 172.21.15.16:0/3947822713 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555cc43e0380 msgr2=0x555cc43f3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:03.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.426+0000 7f303a197640 1 --2- 172.21.15.16:0/3947822713 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555cc43e0380 0x555cc43f3190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:03.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.426+0000 7f303a197640 1 -- 172.21.15.16:0/3947822713 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555cc43ef150 msgr2=0x555cc43f36d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:03.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.426+0000 7f303a197640 1 --2- 172.21.15.16:0/3947822713 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555cc43ef150 0x555cc43f36d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:03.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.426+0000 7f303a197640 1 -- 172.21.15.16:0/3947822713 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555cc4468a70 con 0x555cc43dfa60 2024-09-12T02:11:03.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.426+0000 7f303a998640 1 --2- 172.21.15.16:0/3947822713 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555cc43ef150 0x555cc43f36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:11:03.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.426+0000 7f303a197640 1 --2- 172.21.15.16:0/3947822713 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555cc43dfa60 0x555cc43f2c50 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f30300bbdd0 tx=0x7f303002a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=432ee850ec71a447 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:03.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.426+0000 7f301dffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:11:03.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.426+0000 7f301dffb640 1 -- 172.21.15.16:0/3947822713 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30300be070 con 0x555cc43dfa60 2024-09-12T02:11:03.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.426+0000 7f301dffb640 1 -- 172.21.15.16:0/3947822713 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f30300b9ee0 con 0x555cc43dfa60 2024-09-12T02:11:03.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.426+0000 7f303e7d4740 1 -- 172.21.15.16:0/3947822713 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f302c023070 con 0x555cc43dfa60 2024-09-12T02:11:03.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.426+0000 7f301dffb640 1 -- 172.21.15.16:0/3947822713 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f30300b3020 con 0x555cc43dfa60 2024-09-12T02:11:03.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.426+0000 7f303e7d4740 10 client.18237 fetch_fsmap learned FSMap version 25 2024-09-12T02:11:03.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.426+0000 7f303e7d4740 1 -- 172.21.15.16:0/3947822713 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555cc4469d00 con 0x555cc43dfa60 2024-09-12T02:11:03.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.426+0000 7f301dffb640 1 -- 172.21.15.16:0/3947822713 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f302c023070 con 0x555cc43dfa60 2024-09-12T02:11:03.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.427+0000 7f301dffb640 1 -- 172.21.15.16:0/3947822713 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f30300b84b0 con 0x555cc43dfa60 2024-09-12T02:11:03.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.427+0000 7f303e7d4740 10 client.18237 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:11:03.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.427+0000 7f303e7d4740 10 client.18237 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:11:03.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.427+0000 7f303e7d4740 20 client.18237 populate_metadata read hostname 'smithi016' 2024-09-12T02:11:03.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.427+0000 7f303e7d4740 1 --2- 172.21.15.16:0/3947822713 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x555cc446a290 0x555cc446c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:03.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.427+0000 7f303e7d4740 4 client.18237 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-12T02:11:03.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.427+0000 7f303e7d4740 1 -- 172.21.15.16:0/3947822713 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x555cc42c9110 con 0x555cc446a290 2024-09-12T02:11:03.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.427+0000 7f3039996640 1 --2- 172.21.15.16:0/3947822713 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x555cc446a290 0x555cc446c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:03.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.427+0000 7f3039996640 1 --2- 172.21.15.16:0/3947822713 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x555cc446a290 0x555cc446c680 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f367215221fc573 server_cookie=d4de9c95727d2e8e in_seq=0 out_seq=0 2024-09-12T02:11:03.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.427+0000 7f301dffb640 10 client.18237 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:11:03.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.428+0000 7f301dffb640 1 -- 172.21.15.16:0/3947822713 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x555cc42c9110 con 0x555cc446a290 2024-09-12T02:11:03.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.428+0000 7f301dffb640 10 client.18237 handle_command_reply: tid=0 2024-09-12T02:11:03.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.428+0000 7f303e7d4740 1 client.18237 shutdown 2024-09-12T02:11:03.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.428+0000 7f303e7d4740 1 -- 172.21.15.16:0/3947822713 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555cc43dfa60 msgr2=0x555cc43f2c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:03.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.428+0000 7f303e7d4740 1 --2- 172.21.15.16:0/3947822713 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555cc43dfa60 0x555cc43f2c50 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f30300bbdd0 tx=0x7f303002a420 comp rx=0 tx=0).stop 2024-09-12T02:11:03.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.428+0000 7f303e7d4740 1 -- 172.21.15.16:0/3947822713 shutdown_connections 2024-09-12T02:11:03.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.428+0000 7f303e7d4740 1 --2- 172.21.15.16:0/3947822713 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x555cc446a290 0x555cc446c680 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:03.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.428+0000 7f303e7d4740 1 --2- 172.21.15.16:0/3947822713 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555cc43ef150 0x555cc43f36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:03.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.429+0000 7f303e7d4740 1 --2- 172.21.15.16:0/3947822713 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555cc43e0380 0x555cc43f3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:03.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.429+0000 7f301dffb640 0 client.18237 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:11:03.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.429+0000 7f303e7d4740 1 --2- 172.21.15.16:0/3947822713 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555cc43dfa60 0x555cc43f2c50 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:03.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.429+0000 7f303e7d4740 1 -- 172.21.15.16:0/3947822713 >> 172.21.15.16:0/3947822713 conn(0x555cc43dc0f0 msgr2=0x555cc43ec820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:03.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.429+0000 7f303e7d4740 1 -- 172.21.15.16:0/3947822713 shutdown_connections 2024-09-12T02:11:03.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.429+0000 7f303e7d4740 1 -- 172.21.15.16:0/3947822713 wait complete. 2024-09-12T02:11:03.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.429+0000 7f303e7d4740 20 client.18237 trim_cache size 0 max 16384 2024-09-12T02:11:03.434 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T02:11:03.434 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-09-12T02:11:03.434 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "9f18ab0a-8b76-4f31-b2f8-382ba5bede51", 2024-09-12T02:11:03.434 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-09-12T02:11:03.434 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T02:11:03.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.429+0000 7f303c3fb640 1 -- 172.21.15.16:0/3269231629 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f30200a2b80 msgr2=0x7f30200a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:03.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.429+0000 7f303c3fb640 1 --2- 172.21.15.16:0/3269231629 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f30200a2b80 0x7f30200a5040 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f302c005fd0 tx=0x7f302c004a40 comp rx=0 tx=0).stop 2024-09-12T02:11:03.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.429+0000 7f303c3fb640 1 -- 172.21.15.16:0/3269231629 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3034100a80 msgr2=0x7f3034198450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:03.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.429+0000 7f303c3fb640 1 --2- 172.21.15.16:0/3269231629 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3034100a80 0x7f3034198450 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f3030009230 tx=0x7f3030004270 comp rx=0 tx=0).stop 2024-09-12T02:11:03.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.429+0000 7f303c3fb640 1 -- 172.21.15.16:0/3269231629 shutdown_connections 2024-09-12T02:11:03.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.429+0000 7f303c3fb640 1 --2- 172.21.15.16:0/3269231629 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f30200a2b80 0x7f30200a5040 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:03.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.429+0000 7f303c3fb640 1 --2- 172.21.15.16:0/3269231629 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3034106a80 0x7f303419cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:03.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.429+0000 7f303c3fb640 1 --2- 172.21.15.16:0/3269231629 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3034101420 0x7f3034198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:03.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.429+0000 7f303c3fb640 1 --2- 172.21.15.16:0/3269231629 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3034100a80 0x7f3034198450 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:03.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.429+0000 7f303c3fb640 1 -- 172.21.15.16:0/3269231629 >> 172.21.15.16:0/3269231629 conn(0x7f30340fc7a0 msgr2=0x7f303406a8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:03.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.429+0000 7f303c3fb640 1 -- 172.21.15.16:0/3269231629 shutdown_connections 2024-09-12T02:11:03.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:03.429+0000 7f303c3fb640 1 -- 172.21.15.16:0/3269231629 wait complete. 2024-09-12T02:11:04.140 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T02:11:04.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:11:04 smithi184 ceph-mon[35593]: pgmap v609: 97 pgs: 97 active+clean; 844 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-09-12T02:11:04.397 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:11:04.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:11:04 smithi033 ceph-mon[35332]: pgmap v609: 97 pgs: 97 active+clean; 844 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-09-12T02:11:04.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:11:04 smithi016 ceph-mon[30947]: pgmap v609: 97 pgs: 97 active+clean; 844 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-09-12T02:11:04.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.876+0000 7f9a1b837640 1 -- 172.21.15.16:0/374861144 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a14074590 msgr2=0x7f9a14074970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:04.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.876+0000 7f9a1b837640 1 --2- 172.21.15.16:0/374861144 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a14074590 0x7f9a14074970 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f9a10009900 tx=0x7f9a1002a480 comp rx=0 tx=0).stop 2024-09-12T02:11:04.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.876+0000 7f9a1b837640 1 -- 172.21.15.16:0/374861144 shutdown_connections 2024-09-12T02:11:04.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.876+0000 7f9a1b837640 1 --2- 172.21.15.16:0/374861144 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9a14112b90 0x7f9a14114f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:04.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.876+0000 7f9a1b837640 1 --2- 172.21.15.16:0/374861144 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a14074eb0 0x7f9a14112650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:04.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.876+0000 7f9a1b837640 1 --2- 172.21.15.16:0/374861144 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a14074590 0x7f9a14074970 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:04.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.876+0000 7f9a1b837640 1 -- 172.21.15.16:0/374861144 >> 172.21.15.16:0/374861144 conn(0x7f9a140fd8c0 msgr2=0x7f9a140ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:04.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.876+0000 7f9a1b837640 1 -- 172.21.15.16:0/374861144 shutdown_connections 2024-09-12T02:11:04.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.876+0000 7f9a1b837640 1 -- 172.21.15.16:0/374861144 wait complete. 2024-09-12T02:11:04.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.877+0000 7f9a1b837640 1 Processor -- start 2024-09-12T02:11:04.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.877+0000 7f9a1b837640 1 -- start start 2024-09-12T02:11:04.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.878+0000 7f9a1b837640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9a14074590 0x7f9a141a4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:04.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.878+0000 7f9a1b837640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a14074eb0 0x7f9a141a4c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:04.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.878+0000 7f9a1b837640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a14112b90 0x7f9a1419e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:04.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.878+0000 7f9a1b837640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9a14118570 con 0x7f9a14074eb0 2024-09-12T02:11:04.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.878+0000 7f9a1b837640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9a141183f0 con 0x7f9a14112b90 2024-09-12T02:11:04.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.878+0000 7f9a1b837640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f9a141186f0 con 0x7f9a14074590 2024-09-12T02:11:04.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.878+0000 7f9a18dd2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a14074eb0 0x7f9a141a4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:04.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.878+0000 7f9a18dd2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a14074eb0 0x7f9a141a4c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39272/0 (socket says 172.21.15.16:39272) 2024-09-12T02:11:04.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.878+0000 7f9a18dd2640 1 -- 172.21.15.16:0/1481915865 learned_addr learned my addr 172.21.15.16:0/1481915865 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:11:04.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.878+0000 7f9a195d3640 1 --2- 172.21.15.16:0/1481915865 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9a14074590 0x7f9a141a4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:04.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.878+0000 7f9a19dd4640 1 --2- 172.21.15.16:0/1481915865 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a14112b90 0x7f9a1419e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:04.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.878+0000 7f9a18dd2640 1 -- 172.21.15.16:0/1481915865 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9a14074590 msgr2=0x7f9a141a4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:04.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.878+0000 7f9a18dd2640 1 --2- 172.21.15.16:0/1481915865 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9a14074590 0x7f9a141a4710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:04.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.878+0000 7f9a18dd2640 1 -- 172.21.15.16:0/1481915865 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a14112b90 msgr2=0x7f9a1419e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:04.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.878+0000 7f9a18dd2640 1 --2- 172.21.15.16:0/1481915865 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a14112b90 0x7f9a1419e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:04.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.878+0000 7f9a18dd2640 1 -- 172.21.15.16:0/1481915865 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a1419f000 con 0x7f9a14074eb0 2024-09-12T02:11:04.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.878+0000 7f9a18dd2640 1 --2- 172.21.15.16:0/1481915865 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a14074eb0 0x7f9a141a4c50 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f9a0400d950 tx=0x7f9a0400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:04.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.878+0000 7f9a0a7fc640 1 -- 172.21.15.16:0/1481915865 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a04017070 con 0x7f9a14074eb0 2024-09-12T02:11:04.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.878+0000 7f9a0a7fc640 1 -- 172.21.15.16:0/1481915865 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a04009c30 con 0x7f9a14074eb0 2024-09-12T02:11:04.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.878+0000 7f9a1b837640 1 -- 172.21.15.16:0/1481915865 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a1419f2f0 con 0x7f9a14074eb0 2024-09-12T02:11:04.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.878+0000 7f9a0a7fc640 1 -- 172.21.15.16:0/1481915865 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a0400f6b0 con 0x7f9a14074eb0 2024-09-12T02:11:04.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.878+0000 7f9a1b837640 1 -- 172.21.15.16:0/1481915865 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a141ab5e0 con 0x7f9a14074eb0 2024-09-12T02:11:04.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.882+0000 7f9a0a7fc640 1 -- 172.21.15.16:0/1481915865 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f9a040109d0 con 0x7f9a14074eb0 2024-09-12T02:11:04.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.882+0000 7f9a1dc10740 1 Processor -- start 2024-09-12T02:11:04.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.882+0000 7f9a1dc10740 1 -- start start 2024-09-12T02:11:04.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.882+0000 7f9a1dc10740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560534c174a0 0x560534c17880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:04.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.882+0000 7f9a1dc10740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560534c17dc0 0x560534c22140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:04.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.882+0000 7f9a1dc10740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560534c22680 0x560534c24a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:04.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.882+0000 7f9a1dc10740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x560534b7d2e0 con 0x560534c174a0 2024-09-12T02:11:04.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.882+0000 7f9a1dc10740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x560534b24ce0 con 0x560534c17dc0 2024-09-12T02:11:04.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.882+0000 7f9a1dc10740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x560534b792a0 con 0x560534c22680 2024-09-12T02:11:04.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.882+0000 7f9a19dd4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560534c174a0 0x560534c17880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:04.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.882+0000 7f9a19dd4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560534c174a0 0x560534c17880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39274/0 (socket says 172.21.15.16:39274) 2024-09-12T02:11:04.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.882+0000 7f9a19dd4640 1 -- 172.21.15.16:0/4117612293 learned_addr learned my addr 172.21.15.16:0/4117612293 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:11:04.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.882+0000 7f9a195d3640 1 --2- 172.21.15.16:0/4117612293 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560534c17dc0 0x560534c22140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:04.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.882+0000 7f9a18dd2640 1 --2- 172.21.15.16:0/4117612293 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560534c22680 0x560534c24a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:04.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.882+0000 7f9a195d3640 1 -- 172.21.15.16:0/4117612293 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560534c22680 msgr2=0x560534c24a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:04.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.882+0000 7f9a195d3640 1 --2- 172.21.15.16:0/4117612293 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560534c22680 0x560534c24a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:04.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.882+0000 7f9a195d3640 1 -- 172.21.15.16:0/4117612293 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560534c174a0 msgr2=0x560534c17880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:04.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.882+0000 7f9a195d3640 1 --2- 172.21.15.16:0/4117612293 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560534c174a0 0x560534c17880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:04.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.882+0000 7f9a195d3640 1 -- 172.21.15.16:0/4117612293 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560534b09440 con 0x560534c17dc0 2024-09-12T02:11:04.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.882+0000 7f9a19dd4640 1 --2- 172.21.15.16:0/4117612293 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560534c174a0 0x560534c17880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:11:04.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.882+0000 7f9a0a7fc640 1 --2- 172.21.15.16:0/1481915865 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f9a000a2c00 0x7f9a000a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:04.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.883+0000 7f9a19dd4640 1 --2- 172.21.15.16:0/1481915865 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f9a000a2c00 0x7f9a000a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:04.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.883+0000 7f9a0a7fc640 1 -- 172.21.15.16:0/1481915865 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f9a040ceae0 con 0x7f9a14074eb0 2024-09-12T02:11:04.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.883+0000 7f9a195d3640 1 --2- 172.21.15.16:0/4117612293 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560534c17dc0 0x560534c22140 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f9a10005940 tx=0x7f9a10005e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e213aa96e9fe0d09 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:04.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.883+0000 7f99ff7fe640 1 -- 172.21.15.16:0/4117612293 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a10040070 con 0x560534c17dc0 2024-09-12T02:11:04.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.883+0000 7f99ff7fe640 1 -- 172.21.15.16:0/4117612293 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a100319f0 con 0x560534c17dc0 2024-09-12T02:11:04.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.883+0000 7f99ff7fe640 1 -- 172.21.15.16:0/4117612293 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a10035040 con 0x560534c17dc0 2024-09-12T02:11:04.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.884+0000 7f9a1dc10740 1 -- 172.21.15.16:0/4117612293 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560534c17dc0 msgr2=0x560534c22140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:04.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.884+0000 7f9a19dd4640 1 --2- 172.21.15.16:0/1481915865 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f9a000a2c00 0x7f9a000a50c0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c0136f0 tx=0x7f9a0c009600 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:04.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.884+0000 7f9a1dc10740 1 --2- 172.21.15.16:0/4117612293 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560534c17dc0 0x560534c22140 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f9a10005940 tx=0x7f9a10005e80 comp rx=0 tx=0).stop 2024-09-12T02:11:04.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.884+0000 7f9a1dc10740 1 -- 172.21.15.16:0/4117612293 shutdown_connections 2024-09-12T02:11:04.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.884+0000 7f9a1dc10740 1 --2- 172.21.15.16:0/4117612293 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560534c22680 0x560534c24a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:04.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.884+0000 7f9a1dc10740 1 --2- 172.21.15.16:0/4117612293 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560534c17dc0 0x560534c22140 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:04.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.884+0000 7f9a1dc10740 1 --2- 172.21.15.16:0/4117612293 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560534c174a0 0x560534c17880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:04.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.884+0000 7f9a1dc10740 1 -- 172.21.15.16:0/4117612293 >> 172.21.15.16:0/4117612293 conn(0x560534c13a30 msgr2=0x560534c13e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:04.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.884+0000 7f9a1dc10740 1 -- 172.21.15.16:0/4117612293 shutdown_connections 2024-09-12T02:11:04.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.884+0000 7f9a1dc10740 1 -- 172.21.15.16:0/4117612293 wait complete. 2024-09-12T02:11:04.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.884+0000 7f9a1dc10740 1 -- start start 2024-09-12T02:11:04.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.884+0000 7f9a1dc10740 1 Processor -- start 2024-09-12T02:11:04.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.885+0000 7f9a1dc10740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560534c174a0 0x560534c21f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:04.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.885+0000 7f9a1dc10740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560534c17dc0 0x560534c1f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:04.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.885+0000 7f9a1dc10740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560534c22680 0x560534c1f720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:04.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.885+0000 7f9a1dc10740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x560534c25250 con 0x560534c22680 2024-09-12T02:11:04.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.885+0000 7f9a1dc10740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x560534b38050 con 0x560534c174a0 2024-09-12T02:11:04.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.885+0000 7f9a1dc10740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x560534c253d0 con 0x560534c17dc0 2024-09-12T02:11:04.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.885+0000 7f9a1dc10740 10 client.0 _ll_register_callbacks cb 0x560534b78160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:11:04.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.885+0000 7f9a19dd4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560534c22680 0x560534c1f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:04.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.885+0000 7f9a19dd4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560534c22680 0x560534c1f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39286/0 (socket says 172.21.15.16:39286) 2024-09-12T02:11:04.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.885+0000 7f9a19dd4640 1 -- 172.21.15.16:0/462907025 learned_addr learned my addr 172.21.15.16:0/462907025 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:11:04.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.885+0000 7f9a195d3640 1 --2- 172.21.15.16:0/462907025 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560534c174a0 0x560534c21f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:04.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.885+0000 7f9a18dd2640 1 --2- 172.21.15.16:0/462907025 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560534c17dc0 0x560534c1f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:04.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.885+0000 7f9a19dd4640 1 -- 172.21.15.16:0/462907025 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560534c17dc0 msgr2=0x560534c1f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:04.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.885+0000 7f9a19dd4640 1 --2- 172.21.15.16:0/462907025 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560534c17dc0 0x560534c1f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:04.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.885+0000 7f9a19dd4640 1 -- 172.21.15.16:0/462907025 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560534c174a0 msgr2=0x560534c21f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:04.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.885+0000 7f9a19dd4640 1 --2- 172.21.15.16:0/462907025 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560534c174a0 0x560534c21f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:04.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.885+0000 7f9a19dd4640 1 -- 172.21.15.16:0/462907025 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5605348ee3c0 con 0x560534c22680 2024-09-12T02:11:04.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.885+0000 7f9a19dd4640 1 --2- 172.21.15.16:0/462907025 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560534c22680 0x560534c1f720 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c009850 tx=0x7f9a0c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57a4a85c17c11f17 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:04.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.885+0000 7f99fcff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:11:04.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.886+0000 7f99fcff9640 1 -- 172.21.15.16:0/462907025 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a0c020620 con 0x560534c22680 2024-09-12T02:11:04.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.886+0000 7f99fcff9640 1 -- 172.21.15.16:0/462907025 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f9a0c0207c0 con 0x560534c22680 2024-09-12T02:11:04.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.886+0000 7f9a1dc10740 1 -- 172.21.15.16:0/462907025 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9a100319f0 con 0x560534c22680 2024-09-12T02:11:04.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.886+0000 7f99fcff9640 1 -- 172.21.15.16:0/462907025 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9a0c0209c0 con 0x560534c22680 2024-09-12T02:11:04.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.886+0000 7f9a1dc10740 10 client.18249 fetch_fsmap learned FSMap version 25 2024-09-12T02:11:04.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.886+0000 7f9a1dc10740 1 -- 172.21.15.16:0/462907025 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x560534c30c20 con 0x560534c22680 2024-09-12T02:11:04.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.886+0000 7f99fcff9640 1 -- 172.21.15.16:0/462907025 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a100319f0 con 0x560534c22680 2024-09-12T02:11:04.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.886+0000 7f99fcff9640 1 -- 172.21.15.16:0/462907025 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f9a0c00f040 con 0x560534c22680 2024-09-12T02:11:04.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.886+0000 7f9a1dc10740 10 client.18249 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:11:04.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.886+0000 7f9a1dc10740 10 client.18249 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:11:04.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.886+0000 7f9a1dc10740 20 client.18249 populate_metadata read hostname 'smithi016' 2024-09-12T02:11:04.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.886+0000 7f9a1dc10740 1 --2- 172.21.15.16:0/462907025 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x560534c320e0 0x560534c34580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:04.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.886+0000 7f9a1dc10740 4 client.18249 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:11:04.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.886+0000 7f9a1dc10740 1 -- 172.21.15.16:0/462907025 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560534b8a850 con 0x560534c320e0 2024-09-12T02:11:04.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.887+0000 7f9a195d3640 1 --2- 172.21.15.16:0/462907025 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x560534c320e0 0x560534c34580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:04.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.887+0000 7f9a195d3640 1 --2- 172.21.15.16:0/462907025 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x560534c320e0 0x560534c34580 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dfce8a8262984748 server_cookie=53478235aa4ea1b in_seq=0 out_seq=0 2024-09-12T02:11:04.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.887+0000 7f99fcff9640 10 client.18249 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:11:04.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.888+0000 7f99fcff9640 1 -- 172.21.15.16:0/462907025 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560534b8a850 con 0x560534c320e0 2024-09-12T02:11:04.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.888+0000 7f99fcff9640 10 client.18249 handle_command_reply: tid=0 2024-09-12T02:11:04.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.888+0000 7f9a1dc10740 1 client.18249 shutdown 2024-09-12T02:11:04.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.888+0000 7f9a1dc10740 1 -- 172.21.15.16:0/462907025 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560534c22680 msgr2=0x560534c1f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:04.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.888+0000 7f9a1dc10740 1 --2- 172.21.15.16:0/462907025 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560534c22680 0x560534c1f720 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c009850 tx=0x7f9a0c004a40 comp rx=0 tx=0).stop 2024-09-12T02:11:04.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.888+0000 7f9a1dc10740 1 -- 172.21.15.16:0/462907025 shutdown_connections 2024-09-12T02:11:04.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.888+0000 7f9a1dc10740 1 --2- 172.21.15.16:0/462907025 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x560534c320e0 0x560534c34580 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:04.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.888+0000 7f9a1dc10740 1 --2- 172.21.15.16:0/462907025 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560534c22680 0x560534c1f720 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:04.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.888+0000 7f9a1dc10740 1 --2- 172.21.15.16:0/462907025 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560534c17dc0 0x560534c1f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:04.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.888+0000 7f99fcff9640 0 client.18249 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:11:04.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.888+0000 7f9a1dc10740 1 --2- 172.21.15.16:0/462907025 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560534c174a0 0x560534c21f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:04.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.888+0000 7f9a1dc10740 1 -- 172.21.15.16:0/462907025 >> 172.21.15.16:0/462907025 conn(0x560534c13a30 msgr2=0x560534c19760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:04.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.888+0000 7f9a1dc10740 1 -- 172.21.15.16:0/462907025 shutdown_connections 2024-09-12T02:11:04.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.888+0000 7f9a1dc10740 1 -- 172.21.15.16:0/462907025 wait complete. 2024-09-12T02:11:04.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.888+0000 7f9a1dc10740 20 client.18249 trim_cache size 0 max 16384 2024-09-12T02:11:04.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.906+0000 7f9a1dc10740 1 Processor -- start 2024-09-12T02:11:04.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.906+0000 7f9a1dc10740 1 -- start start 2024-09-12T02:11:04.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.906+0000 7f9a1dc10740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560534c13a30 0x560534c2e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:04.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.906+0000 7f9a1dc10740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560534c174a0 0x560534c39ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:04.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.906+0000 7f9a1dc10740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560534c17dc0 0x560534c3a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:04.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.906+0000 7f9a1dc10740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9a141b6300 con 0x560534c174a0 2024-09-12T02:11:04.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.906+0000 7f9a1dc10740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9a141b3bd0 con 0x560534c13a30 2024-09-12T02:11:04.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.906+0000 7f9a1dc10740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f9a141b1a20 con 0x560534c17dc0 2024-09-12T02:11:04.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.906+0000 7f9a18dd2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560534c174a0 0x560534c39ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:04.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.906+0000 7f9a18dd2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560534c174a0 0x560534c39ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39298/0 (socket says 172.21.15.16:39298) 2024-09-12T02:11:04.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.906+0000 7f9a18dd2640 1 -- 172.21.15.16:0/2133401937 learned_addr learned my addr 172.21.15.16:0/2133401937 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:11:04.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.906+0000 7f9a19dd4640 1 --2- 172.21.15.16:0/2133401937 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560534c17dc0 0x560534c3a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:04.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.906+0000 7f9a195d3640 1 --2- 172.21.15.16:0/2133401937 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560534c13a30 0x560534c2e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:04.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.906+0000 7f9a18dd2640 1 -- 172.21.15.16:0/2133401937 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560534c17dc0 msgr2=0x560534c3a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:04.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.906+0000 7f9a18dd2640 1 --2- 172.21.15.16:0/2133401937 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560534c17dc0 0x560534c3a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:04.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.906+0000 7f9a18dd2640 1 -- 172.21.15.16:0/2133401937 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560534c13a30 msgr2=0x560534c2e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:04.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.906+0000 7f9a18dd2640 1 --2- 172.21.15.16:0/2133401937 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560534c13a30 0x560534c2e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:04.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.906+0000 7f9a18dd2640 1 -- 172.21.15.16:0/2133401937 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560534c13860 con 0x560534c174a0 2024-09-12T02:11:04.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.906+0000 7f9a195d3640 1 --2- 172.21.15.16:0/2133401937 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560534c13a30 0x560534c2e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:11:04.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.906+0000 7f9a18dd2640 1 --2- 172.21.15.16:0/2133401937 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560534c174a0 0x560534c39ea0 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f9a04097e60 tx=0x7f9a0409b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=504a3592aedecccd server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:04.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.907+0000 7f99feffd640 1 -- 172.21.15.16:0/2133401937 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a04020040 con 0x560534c174a0 2024-09-12T02:11:04.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.907+0000 7f99feffd640 1 -- 172.21.15.16:0/2133401937 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a04009de0 con 0x560534c174a0 2024-09-12T02:11:04.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.907+0000 7f99feffd640 1 -- 172.21.15.16:0/2133401937 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a0409a8e0 con 0x560534c174a0 2024-09-12T02:11:04.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.907+0000 7f9a1dc10740 1 -- 172.21.15.16:0/2133401937 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560534c174a0 msgr2=0x560534c39ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:04.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.907+0000 7f9a1dc10740 1 --2- 172.21.15.16:0/2133401937 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560534c174a0 0x560534c39ea0 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f9a04097e60 tx=0x7f9a0409b260 comp rx=0 tx=0).stop 2024-09-12T02:11:04.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.907+0000 7f9a1dc10740 1 -- 172.21.15.16:0/2133401937 shutdown_connections 2024-09-12T02:11:04.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.907+0000 7f9a1dc10740 1 --2- 172.21.15.16:0/2133401937 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560534c17dc0 0x560534c3a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:04.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.907+0000 7f9a1dc10740 1 --2- 172.21.15.16:0/2133401937 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560534c174a0 0x560534c39ea0 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:04.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.907+0000 7f9a1dc10740 1 --2- 172.21.15.16:0/2133401937 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560534c13a30 0x560534c2e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:04.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.907+0000 7f9a1dc10740 1 -- 172.21.15.16:0/2133401937 >> 172.21.15.16:0/2133401937 conn(0x560534c1f880 msgr2=0x560534c1dab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:04.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.907+0000 7f9a1dc10740 1 -- 172.21.15.16:0/2133401937 shutdown_connections 2024-09-12T02:11:04.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.907+0000 7f9a1dc10740 1 -- 172.21.15.16:0/2133401937 wait complete. 2024-09-12T02:11:04.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.908+0000 7f9a1dc10740 1 -- start start 2024-09-12T02:11:04.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.908+0000 7f9a1dc10740 1 Processor -- start 2024-09-12T02:11:04.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.908+0000 7f9a1dc10740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560534c13a30 0x560534c26a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:04.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.908+0000 7f9a1dc10740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560534c174a0 0x560534c26fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:04.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.908+0000 7f9a1dc10740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560534c17dc0 0x560534c9be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:04.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.908+0000 7f9a1dc10740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9a141b2090 con 0x560534c13a30 2024-09-12T02:11:04.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.908+0000 7f9a1dc10740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9a141b3500 con 0x560534c174a0 2024-09-12T02:11:04.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.908+0000 7f9a1dc10740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f9a141b2760 con 0x560534c17dc0 2024-09-12T02:11:04.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.908+0000 7f9a1dc10740 10 client.0 _ll_register_callbacks cb 0x7f9a0c009ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:11:04.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.908+0000 7f9a195d3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560534c13a30 0x560534c26a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:04.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.908+0000 7f9a195d3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560534c13a30 0x560534c26a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39314/0 (socket says 172.21.15.16:39314) 2024-09-12T02:11:04.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.908+0000 7f9a18dd2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560534c174a0 0x560534c26fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:04.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.908+0000 7f9a195d3640 1 -- 172.21.15.16:0/2294923996 learned_addr learned my addr 172.21.15.16:0/2294923996 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:11:04.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.908+0000 7f9a19dd4640 1 --2- 172.21.15.16:0/2294923996 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560534c17dc0 0x560534c9be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:04.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.909+0000 7f9a195d3640 1 -- 172.21.15.16:0/2294923996 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560534c17dc0 msgr2=0x560534c9be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:04.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.909+0000 7f9a195d3640 1 --2- 172.21.15.16:0/2294923996 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560534c17dc0 0x560534c9be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:04.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.909+0000 7f9a195d3640 1 -- 172.21.15.16:0/2294923996 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560534c174a0 msgr2=0x560534c26fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:04.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.909+0000 7f9a195d3640 1 --2- 172.21.15.16:0/2294923996 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560534c174a0 0x560534c26fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:04.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.909+0000 7f9a195d3640 1 -- 172.21.15.16:0/2294923996 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560534c146c0 con 0x560534c13a30 2024-09-12T02:11:04.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.909+0000 7f9a19dd4640 1 --2- 172.21.15.16:0/2294923996 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560534c17dc0 0x560534c9be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:11:04.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.909+0000 7f9a195d3640 1 --2- 172.21.15.16:0/2294923996 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560534c13a30 0x560534c26a90 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f9a10031550 tx=0x7f9a1002a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d81c760308ac5c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:04.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.909+0000 7f99fcff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:11:04.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.909+0000 7f99fcff9640 1 -- 172.21.15.16:0/2294923996 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a10040040 con 0x560534c13a30 2024-09-12T02:11:04.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.909+0000 7f99fcff9640 1 -- 172.21.15.16:0/2294923996 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f9a10004240 con 0x560534c13a30 2024-09-12T02:11:04.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.909+0000 7f9a1dc10740 1 -- 172.21.15.16:0/2294923996 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9a04009de0 con 0x560534c13a30 2024-09-12T02:11:04.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.909+0000 7f99fcff9640 1 -- 172.21.15.16:0/2294923996 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9a10035040 con 0x560534c13a30 2024-09-12T02:11:04.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.909+0000 7f9a1dc10740 10 client.18255 fetch_fsmap learned FSMap version 25 2024-09-12T02:11:04.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.909+0000 7f9a1dc10740 1 -- 172.21.15.16:0/2294923996 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x560534c9d430 con 0x560534c13a30 2024-09-12T02:11:04.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.909+0000 7f99fcff9640 1 -- 172.21.15.16:0/2294923996 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a04009de0 con 0x560534c13a30 2024-09-12T02:11:04.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.910+0000 7f99fcff9640 1 -- 172.21.15.16:0/2294923996 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f9a100504b0 con 0x560534c13a30 2024-09-12T02:11:04.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.910+0000 7f9a1dc10740 10 client.18255 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:11:04.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.910+0000 7f9a1dc10740 10 client.18255 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:11:04.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.910+0000 7f9a1dc10740 20 client.18255 populate_metadata read hostname 'smithi016' 2024-09-12T02:11:04.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.910+0000 7f9a1dc10740 1 --2- 172.21.15.16:0/2294923996 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x560534c22680 0x560534c9f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:04.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.910+0000 7f9a1dc10740 4 client.18255 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T02:11:04.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.910+0000 7f9a1dc10740 1 -- 172.21.15.16:0/2294923996 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560534b84d60 con 0x560534c22680 2024-09-12T02:11:04.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.910+0000 7f9a18dd2640 1 --2- 172.21.15.16:0/2294923996 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x560534c22680 0x560534c9f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:04.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.910+0000 7f99fcff9640 10 client.18255 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:11:04.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.910+0000 7f9a18dd2640 1 --2- 172.21.15.16:0/2294923996 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x560534c22680 0x560534c9f780 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fca9d96b735eb0e1 server_cookie=df8102361f6ffd01 in_seq=0 out_seq=0 2024-09-12T02:11:04.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.910+0000 7f99fcff9640 1 -- 172.21.15.16:0/2294923996 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x560534b84d60 con 0x560534c22680 2024-09-12T02:11:04.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.910+0000 7f99fcff9640 10 client.18255 handle_command_reply: tid=0 2024-09-12T02:11:04.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.910+0000 7f9a1dc10740 1 client.18255 shutdown 2024-09-12T02:11:04.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.911+0000 7f9a1dc10740 1 -- 172.21.15.16:0/2294923996 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560534c13a30 msgr2=0x560534c26a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:04.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.911+0000 7f9a1dc10740 1 --2- 172.21.15.16:0/2294923996 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560534c13a30 0x560534c26a90 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f9a10031550 tx=0x7f9a1002a480 comp rx=0 tx=0).stop 2024-09-12T02:11:04.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.911+0000 7f9a1dc10740 1 -- 172.21.15.16:0/2294923996 shutdown_connections 2024-09-12T02:11:04.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.911+0000 7f9a1dc10740 1 --2- 172.21.15.16:0/2294923996 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x560534c22680 0x560534c9f780 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:04.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.911+0000 7f9a1dc10740 1 --2- 172.21.15.16:0/2294923996 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560534c17dc0 0x560534c9be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:04.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.911+0000 7f9a1dc10740 1 --2- 172.21.15.16:0/2294923996 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560534c174a0 0x560534c26fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:04.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.911+0000 7f9a1dc10740 1 --2- 172.21.15.16:0/2294923996 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560534c13a30 0x560534c26a90 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:04.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.911+0000 7f9a1dc10740 1 -- 172.21.15.16:0/2294923996 >> 172.21.15.16:0/2294923996 conn(0x560534c1f880 msgr2=0x560534c20a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:04.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.911+0000 7f99fcff9640 0 client.18255 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:11:04.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.911+0000 7f9a1dc10740 1 -- 172.21.15.16:0/2294923996 shutdown_connections 2024-09-12T02:11:04.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.911+0000 7f9a1dc10740 1 -- 172.21.15.16:0/2294923996 wait complete. 2024-09-12T02:11:04.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.911+0000 7f9a1dc10740 20 client.18255 trim_cache size 0 max 16384 2024-09-12T02:11:04.921 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T02:11:04.921 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-12T02:11:04.921 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": { 2024-09-12T02:11:04.921 INFO:teuthology.orchestra.run.smithi016.stdout: "9f18ab0a-8b76-4f31-b2f8-382ba5bede51": { 2024-09-12T02:11:04.921 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-12T02:11:04.921 INFO:teuthology.orchestra.run.smithi016.stdout: "tag": "9f18ab0a-8b76-4f31-b2f8-382ba5bede51", 2024-09-12T02:11:04.921 INFO:teuthology.orchestra.run.smithi016.stdout: "options": "recursive,force" 2024-09-12T02:11:04.921 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:11:04.921 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:11:04.921 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T02:11:04.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.911+0000 7f9a1b837640 1 -- 172.21.15.16:0/1481915865 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f9a000a2c00 msgr2=0x7f9a000a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:04.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.911+0000 7f9a1b837640 1 --2- 172.21.15.16:0/1481915865 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f9a000a2c00 0x7f9a000a50c0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c0136f0 tx=0x7f9a0c009600 comp rx=0 tx=0).stop 2024-09-12T02:11:04.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.911+0000 7f9a1b837640 1 -- 172.21.15.16:0/1481915865 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a14074eb0 msgr2=0x7f9a141a4c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:04.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.911+0000 7f9a1b837640 1 --2- 172.21.15.16:0/1481915865 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a14074eb0 0x7f9a141a4c50 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f9a0400d950 tx=0x7f9a0400de10 comp rx=0 tx=0).stop 2024-09-12T02:11:04.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.912+0000 7f9a1b837640 1 -- 172.21.15.16:0/1481915865 shutdown_connections 2024-09-12T02:11:04.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.912+0000 7f9a1b837640 1 --2- 172.21.15.16:0/1481915865 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f9a000a2c00 0x7f9a000a50c0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:04.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.912+0000 7f9a1b837640 1 --2- 172.21.15.16:0/1481915865 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a14112b90 0x7f9a1419e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:04.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.912+0000 7f9a1b837640 1 --2- 172.21.15.16:0/1481915865 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a14074eb0 0x7f9a141a4c50 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:04.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.912+0000 7f9a1b837640 1 --2- 172.21.15.16:0/1481915865 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9a14074590 0x7f9a141a4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:04.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.912+0000 7f9a1b837640 1 -- 172.21.15.16:0/1481915865 >> 172.21.15.16:0/1481915865 conn(0x7f9a140fd8c0 msgr2=0x7f9a140ff0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:04.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.912+0000 7f9a1b837640 1 -- 172.21.15.16:0/1481915865 shutdown_connections 2024-09-12T02:11:04.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:04.912+0000 7f9a1b837640 1 -- 172.21.15.16:0/1481915865 wait complete. 2024-09-12T02:11:05.183 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:11:05 smithi016 ceph-mon[30947]: scrub queued for path: / 2024-09-12T02:11:05.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:11:05 smithi016 ceph-mon[30947]: scrub summary: idle+waiting paths [/] 2024-09-12T02:11:05.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:11:05 smithi016 ceph-mon[30947]: scrub summary: active paths [/] 2024-09-12T02:11:05.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:11:05 smithi016 ceph-mon[30947]: scrub summary: idle+waiting paths [/] 2024-09-12T02:11:05.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:11:05 smithi184 ceph-mon[35593]: scrub queued for path: / 2024-09-12T02:11:05.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:11:05 smithi184 ceph-mon[35593]: scrub summary: idle+waiting paths [/] 2024-09-12T02:11:05.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:11:05 smithi184 ceph-mon[35593]: scrub summary: active paths [/] 2024-09-12T02:11:05.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:11:05 smithi184 ceph-mon[35593]: scrub summary: idle+waiting paths [/] 2024-09-12T02:11:05.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:11:05 smithi033 ceph-mon[35332]: scrub queued for path: / 2024-09-12T02:11:05.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:11:05 smithi033 ceph-mon[35332]: scrub summary: idle+waiting paths [/] 2024-09-12T02:11:05.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:11:05 smithi033 ceph-mon[35332]: scrub summary: active paths [/] 2024-09-12T02:11:05.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:11:05 smithi033 ceph-mon[35332]: scrub summary: idle+waiting paths [/] 2024-09-12T02:11:05.657 INFO:tasks.cephfs.filesystem:scrub status for tag:9f18ab0a-8b76-4f31-b2f8-382ba5bede51 - {'path': '/', 'tag': '9f18ab0a-8b76-4f31-b2f8-382ba5bede51', 'options': 'recursive,force'} 2024-09-12T02:11:06.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:11:06 smithi184 ceph-mon[35593]: pgmap v610: 97 pgs: 97 active+clean; 844 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-12T02:11:06.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:11:06 smithi033 ceph-mon[35332]: pgmap v610: 97 pgs: 97 active+clean; 844 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-12T02:11:06.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:11:06 smithi016 ceph-mon[30947]: pgmap v610: 97 pgs: 97 active+clean; 844 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-12T02:11:07.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:11:07 smithi184 ceph-mon[35593]: scrub summary: idle 2024-09-12T02:11:07.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:11:07 smithi033 ceph-mon[35332]: scrub summary: idle 2024-09-12T02:11:07.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:11:07 smithi016 ceph-mon[30947]: scrub summary: idle 2024-09-12T02:11:08.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:11:08 smithi184 ceph-mon[35593]: pgmap v611: 97 pgs: 97 active+clean; 900 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 17 op/s 2024-09-12T02:11:08.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:11:08 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:11:08.418 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:11:08 smithi033 ceph-mon[35332]: pgmap v611: 97 pgs: 97 active+clean; 900 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 17 op/s 2024-09-12T02:11:08.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:11:08 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:11:08.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:11:08 smithi016 ceph-mon[30947]: pgmap v611: 97 pgs: 97 active+clean; 900 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 17 op/s 2024-09-12T02:11:08.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:11:08 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:11:09.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:11:09 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:11:09.500 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:11:09 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:11:09.500 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:11:09 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T02:11:09.501 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:11:09 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:11:09.501 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:11:09 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:11:09.502 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:11:09 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T02:11:09.502 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:11:09 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:11:09.502 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:11:09 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:11:09.502 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:11:09 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T02:11:09.675 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.674+0000 7f5da4eb7640 1 -- 172.21.15.16:0/3996818530 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5da0100a40 msgr2=0x7f5da0100e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:09.675 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.674+0000 7f5da4eb7640 1 --2- 172.21.15.16:0/3996818530 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5da0100a40 0x7f5da0100e40 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f5d90009900 tx=0x7f5d9002a410 comp rx=0 tx=0).stop 2024-09-12T02:11:09.675 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.674+0000 7f5da4eb7640 1 -- 172.21.15.16:0/3996818530 shutdown_connections 2024-09-12T02:11:09.675 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.674+0000 7f5da4eb7640 1 --2- 172.21.15.16:0/3996818530 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5da01025f0 0x7f5da0108e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:09.675 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.674+0000 7f5da4eb7640 1 --2- 172.21.15.16:0/3996818530 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5da0101c30 0x7f5da01020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:09.675 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.674+0000 7f5da4eb7640 1 --2- 172.21.15.16:0/3996818530 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5da0100a40 0x7f5da0100e40 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:09.675 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.674+0000 7f5da4eb7640 1 -- 172.21.15.16:0/3996818530 >> 172.21.15.16:0/3996818530 conn(0x7f5da00fc1e0 msgr2=0x7f5da00fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:09.675 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.674+0000 7f5da4eb7640 1 -- 172.21.15.16:0/3996818530 shutdown_connections 2024-09-12T02:11:09.675 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.674+0000 7f5da4eb7640 1 -- 172.21.15.16:0/3996818530 wait complete. 2024-09-12T02:11:09.678 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.675+0000 7f5da4eb7640 1 Processor -- start 2024-09-12T02:11:09.678 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.675+0000 7f5da4eb7640 1 -- start start 2024-09-12T02:11:09.678 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.676+0000 7f5da4eb7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5da0100a40 0x7f5da006a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:09.678 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.676+0000 7f5da4eb7640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5da0101c30 0x7f5da019ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:09.678 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.676+0000 7f5da4eb7640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5da01025f0 0x7f5da019f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:09.678 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.676+0000 7f5da4eb7640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5da010c6a0 con 0x7f5da0100a40 2024-09-12T02:11:09.678 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.676+0000 7f5da4eb7640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5da010c520 con 0x7f5da0101c30 2024-09-12T02:11:09.678 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.676+0000 7f5da4eb7640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5da010c820 con 0x7f5da01025f0 2024-09-12T02:11:09.678 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.676+0000 7f5d9e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5da0100a40 0x7f5da006a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:09.678 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.676+0000 7f5d9e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5da0100a40 0x7f5da006a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39344/0 (socket says 172.21.15.16:39344) 2024-09-12T02:11:09.678 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.676+0000 7f5d9e59c640 1 -- 172.21.15.16:0/2347860609 learned_addr learned my addr 172.21.15.16:0/2347860609 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:11:09.679 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.676+0000 7f5d9ed9d640 1 --2- 172.21.15.16:0/2347860609 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5da01025f0 0x7f5da019f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:09.679 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.676+0000 7f5d97fff640 1 --2- 172.21.15.16:0/2347860609 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5da0101c30 0x7f5da019ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:09.679 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.676+0000 7f5d9e59c640 1 -- 172.21.15.16:0/2347860609 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5da01025f0 msgr2=0x7f5da019f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:09.679 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.676+0000 7f5d9e59c640 1 --2- 172.21.15.16:0/2347860609 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5da01025f0 0x7f5da019f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:09.679 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.676+0000 7f5d9e59c640 1 -- 172.21.15.16:0/2347860609 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5da0101c30 msgr2=0x7f5da019ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:09.679 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.676+0000 7f5d9e59c640 1 --2- 172.21.15.16:0/2347860609 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5da0101c30 0x7f5da019ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:09.679 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.676+0000 7f5d9e59c640 1 -- 172.21.15.16:0/2347860609 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5da019f9b0 con 0x7f5da0100a40 2024-09-12T02:11:09.679 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.676+0000 7f5d9e59c640 1 --2- 172.21.15.16:0/2347860609 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5da0100a40 0x7f5da006a410 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f5d90031610 tx=0x7f5d900043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:09.679 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.677+0000 7f5d977fe640 1 -- 172.21.15.16:0/2347860609 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d9003f030 con 0x7f5da0100a40 2024-09-12T02:11:09.679 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.677+0000 7f5d977fe640 1 -- 172.21.15.16:0/2347860609 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d90003ce0 con 0x7f5da0100a40 2024-09-12T02:11:09.679 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.677+0000 7f5da4eb7640 1 -- 172.21.15.16:0/2347860609 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5da019fbe0 con 0x7f5da0100a40 2024-09-12T02:11:09.679 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.677+0000 7f5d977fe640 1 -- 172.21.15.16:0/2347860609 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d90035070 con 0x7f5da0100a40 2024-09-12T02:11:09.679 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.677+0000 7f5da4eb7640 1 -- 172.21.15.16:0/2347860609 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5da01a00d0 con 0x7f5da0100a40 2024-09-12T02:11:09.685 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.679+0000 7f5d977fe640 1 -- 172.21.15.16:0/2347860609 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5d90042030 con 0x7f5da0100a40 2024-09-12T02:11:09.685 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.679+0000 7f5da7c53740 1 Processor -- start 2024-09-12T02:11:09.685 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.680+0000 7f5da7c53740 1 -- start start 2024-09-12T02:11:09.685 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.680+0000 7f5da7c53740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5567a548adb0 0x5567a548b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:09.685 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.680+0000 7f5da7c53740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5567a548b6f0 0x5567a54901b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:09.685 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.680+0000 7f5da7c53740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5567a5490780 0x5567a5492c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:09.685 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.680+0000 7f5da7c53740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5567a53ea2f0 con 0x5567a5490780 2024-09-12T02:11:09.685 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.680+0000 7f5da7c53740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5567a5392270 con 0x5567a548b6f0 2024-09-12T02:11:09.685 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.680+0000 7f5da7c53740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5567a53e62b0 con 0x5567a548adb0 2024-09-12T02:11:09.685 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.680+0000 7f5d97fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5567a5490780 0x5567a5492c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:09.685 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.680+0000 7f5d977fe640 1 --2- 172.21.15.16:0/2347860609 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f5d8c0a2c30 0x7f5d8c0a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:09.686 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.680+0000 7f5d97fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5567a5490780 0x5567a5492c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39350/0 (socket says 172.21.15.16:39350) 2024-09-12T02:11:09.686 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.680+0000 7f5d97fff640 1 -- 172.21.15.16:0/2771883242 learned_addr learned my addr 172.21.15.16:0/2771883242 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:11:09.686 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.680+0000 7f5d9ed9d640 1 --2- 172.21.15.16:0/2771883242 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5567a548adb0 0x5567a548b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:09.686 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.680+0000 7f5d9e59c640 1 --2- 172.21.15.16:0/2771883242 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5567a548b6f0 0x5567a54901b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:09.686 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.680+0000 7f5d9ed9d640 1 --2- 172.21.15.16:0/2347860609 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f5d8c0a2c30 0x7f5d8c0a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:09.686 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.680+0000 7f5d97fff640 1 -- 172.21.15.16:0/2771883242 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5567a548adb0 msgr2=0x5567a548b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:09.686 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.680+0000 7f5d97fff640 1 --2- 172.21.15.16:0/2771883242 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5567a548adb0 0x5567a548b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:09.686 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.680+0000 7f5d977fe640 1 -- 172.21.15.16:0/2347860609 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f5d900ef5e0 con 0x7f5da0100a40 2024-09-12T02:11:09.686 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.680+0000 7f5d97fff640 1 -- 172.21.15.16:0/2771883242 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5567a548b6f0 msgr2=0x5567a54901b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:09.686 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.680+0000 7f5d97fff640 1 --2- 172.21.15.16:0/2771883242 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5567a548b6f0 0x5567a54901b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:09.686 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.680+0000 7f5d97fff640 1 -- 172.21.15.16:0/2771883242 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567a5391340 con 0x5567a5490780 2024-09-12T02:11:09.686 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.681+0000 7f5d97fff640 1 --2- 172.21.15.16:0/2771883242 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5567a5490780 0x5567a5492c40 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f5d8800cc40 tx=0x7f5d8800e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddb74e9ee43b8ae0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:09.686 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.681+0000 7f5d9ed9d640 1 --2- 172.21.15.16:0/2347860609 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f5d8c0a2c30 0x7f5d8c0a50f0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f5d98009fd0 tx=0x7f5d98009600 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:09.687 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.681+0000 7f5d94ff9640 1 -- 172.21.15.16:0/2771883242 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d88014070 con 0x5567a5490780 2024-09-12T02:11:09.687 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.681+0000 7f5d94ff9640 1 -- 172.21.15.16:0/2771883242 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d880047a0 con 0x5567a5490780 2024-09-12T02:11:09.687 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.681+0000 7f5d94ff9640 1 -- 172.21.15.16:0/2771883242 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d88010690 con 0x5567a5490780 2024-09-12T02:11:09.687 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.682+0000 7f5da7c53740 1 -- 172.21.15.16:0/2771883242 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5567a5490780 msgr2=0x5567a5492c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:09.687 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.682+0000 7f5da7c53740 1 --2- 172.21.15.16:0/2771883242 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5567a5490780 0x5567a5492c40 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f5d8800cc40 tx=0x7f5d8800e720 comp rx=0 tx=0).stop 2024-09-12T02:11:09.687 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.682+0000 7f5da7c53740 1 -- 172.21.15.16:0/2771883242 shutdown_connections 2024-09-12T02:11:09.687 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.682+0000 7f5da7c53740 1 --2- 172.21.15.16:0/2771883242 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5567a5490780 0x5567a5492c40 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:09.687 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.682+0000 7f5da7c53740 1 --2- 172.21.15.16:0/2771883242 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5567a548b6f0 0x5567a54901b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:09.687 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.682+0000 7f5da7c53740 1 --2- 172.21.15.16:0/2771883242 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5567a548adb0 0x5567a548b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:09.687 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.682+0000 7f5da7c53740 1 -- 172.21.15.16:0/2771883242 >> 172.21.15.16:0/2771883242 conn(0x5567a5482010 msgr2=0x5567a5482410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:09.687 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.682+0000 7f5da7c53740 1 -- 172.21.15.16:0/2771883242 shutdown_connections 2024-09-12T02:11:09.687 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.682+0000 7f5da7c53740 1 -- 172.21.15.16:0/2771883242 wait complete. 2024-09-12T02:11:09.687 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.682+0000 7f5da7c53740 1 -- start start 2024-09-12T02:11:09.688 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.682+0000 7f5da7c53740 1 Processor -- start 2024-09-12T02:11:09.688 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.682+0000 7f5da7c53740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5567a548adb0 0x5567a5494fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:09.688 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.683+0000 7f5da7c53740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5567a548b6f0 0x5567a54954e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:09.688 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.683+0000 7f5da7c53740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5567a5490780 0x5567a549c520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:09.688 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.683+0000 7f5da7c53740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5567a5493750 con 0x5567a548adb0 2024-09-12T02:11:09.688 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.683+0000 7f5da7c53740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5567a5373290 con 0x5567a548b6f0 2024-09-12T02:11:09.688 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.683+0000 7f5da7c53740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5567a54938d0 con 0x5567a5490780 2024-09-12T02:11:09.688 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.683+0000 7f5da7c53740 10 client.0 _ll_register_callbacks cb 0x5567a53a4140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:11:09.688 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.683+0000 7f5d9e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5567a548adb0 0x5567a5494fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:09.688 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.683+0000 7f5d9e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5567a548adb0 0x5567a5494fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39362/0 (socket says 172.21.15.16:39362) 2024-09-12T02:11:09.688 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.683+0000 7f5d9e59c640 1 -- 172.21.15.16:0/3489212812 learned_addr learned my addr 172.21.15.16:0/3489212812 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:11:09.688 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.683+0000 7f5d97fff640 1 --2- 172.21.15.16:0/3489212812 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5567a548b6f0 0x5567a54954e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:09.689 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.683+0000 7f5d9ed9d640 1 --2- 172.21.15.16:0/3489212812 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5567a5490780 0x5567a549c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:09.689 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.683+0000 7f5d9e59c640 1 -- 172.21.15.16:0/3489212812 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5567a5490780 msgr2=0x5567a549c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:09.689 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.683+0000 7f5d9e59c640 1 --2- 172.21.15.16:0/3489212812 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5567a5490780 0x5567a549c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:09.689 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.683+0000 7f5d9e59c640 1 -- 172.21.15.16:0/3489212812 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5567a548b6f0 msgr2=0x5567a54954e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:09.689 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.683+0000 7f5d9e59c640 1 --2- 172.21.15.16:0/3489212812 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5567a548b6f0 0x5567a54954e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:09.689 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.683+0000 7f5d9e59c640 1 -- 172.21.15.16:0/3489212812 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5567a50fd210 con 0x5567a548adb0 2024-09-12T02:11:09.689 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.683+0000 7f5d9ed9d640 1 --2- 172.21.15.16:0/3489212812 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5567a5490780 0x5567a549c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:11:09.689 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.683+0000 7f5d9e59c640 1 --2- 172.21.15.16:0/3489212812 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5567a548adb0 0x5567a5494fa0 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f5d900b82a0 tx=0x7f5d9008adc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a41da8d89e05475 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:09.689 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.683+0000 7f5d85ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:11:09.689 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.683+0000 7f5d85ffb640 1 -- 172.21.15.16:0/3489212812 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d900be030 con 0x5567a548adb0 2024-09-12T02:11:09.689 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.683+0000 7f5d85ffb640 1 -- 172.21.15.16:0/3489212812 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f5d900ed1f0 con 0x5567a548adb0 2024-09-12T02:11:09.689 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.683+0000 7f5da7c53740 1 -- 172.21.15.16:0/3489212812 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d880047a0 con 0x5567a548adb0 2024-09-12T02:11:09.689 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.683+0000 7f5d85ffb640 1 -- 172.21.15.16:0/3489212812 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5d90049040 con 0x5567a548adb0 2024-09-12T02:11:09.690 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.683+0000 7f5da7c53740 10 client.18267 fetch_fsmap learned FSMap version 25 2024-09-12T02:11:09.690 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.683+0000 7f5da7c53740 1 -- 172.21.15.16:0/3489212812 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5567a51c9620 con 0x5567a548adb0 2024-09-12T02:11:09.690 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.684+0000 7f5d85ffb640 1 -- 172.21.15.16:0/3489212812 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d880047a0 con 0x5567a548adb0 2024-09-12T02:11:09.690 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.684+0000 7f5d85ffb640 1 -- 172.21.15.16:0/3489212812 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f5d90003800 con 0x5567a548adb0 2024-09-12T02:11:09.690 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.684+0000 7f5da7c53740 10 client.18267 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:11:09.690 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.684+0000 7f5da7c53740 10 client.18267 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:11:09.690 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.684+0000 7f5da7c53740 20 client.18267 populate_metadata read hostname 'smithi016' 2024-09-12T02:11:09.690 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.684+0000 7f5da7c53740 1 --2- 172.21.15.16:0/3489212812 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5567a54a0c50 0x5567a54a30c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:09.690 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.684+0000 7f5da7c53740 4 client.18267 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:11:09.690 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.684+0000 7f5da7c53740 1 -- 172.21.15.16:0/3489212812 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5567a53f7a20 con 0x5567a54a0c50 2024-09-12T02:11:09.690 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.684+0000 7f5d97fff640 1 --2- 172.21.15.16:0/3489212812 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5567a54a0c50 0x5567a54a30c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:09.690 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.685+0000 7f5d85ffb640 10 client.18267 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:11:09.690 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.685+0000 7f5d97fff640 1 --2- 172.21.15.16:0/3489212812 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5567a54a0c50 0x5567a54a30c0 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4dbaec931f5700c server_cookie=7344d85b8c9f1b6e in_seq=0 out_seq=0 2024-09-12T02:11:09.692 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.685+0000 7f5d85ffb640 1 -- 172.21.15.16:0/3489212812 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5567a53f7a20 con 0x5567a54a0c50 2024-09-12T02:11:09.692 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.685+0000 7f5d85ffb640 10 client.18267 handle_command_reply: tid=0 2024-09-12T02:11:09.692 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.686+0000 7f5da7c53740 1 client.18267 shutdown 2024-09-12T02:11:09.692 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.686+0000 7f5da7c53740 1 -- 172.21.15.16:0/3489212812 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5567a548adb0 msgr2=0x5567a5494fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:09.692 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.686+0000 7f5da7c53740 1 --2- 172.21.15.16:0/3489212812 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5567a548adb0 0x5567a5494fa0 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f5d900b82a0 tx=0x7f5d9008adc0 comp rx=0 tx=0).stop 2024-09-12T02:11:09.692 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.686+0000 7f5da7c53740 1 -- 172.21.15.16:0/3489212812 shutdown_connections 2024-09-12T02:11:09.692 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.686+0000 7f5da7c53740 1 --2- 172.21.15.16:0/3489212812 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5567a54a0c50 0x5567a54a30c0 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:09.692 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.686+0000 7f5da7c53740 1 --2- 172.21.15.16:0/3489212812 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5567a5490780 0x5567a549c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:09.692 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.686+0000 7f5da7c53740 1 --2- 172.21.15.16:0/3489212812 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5567a548b6f0 0x5567a54954e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:09.692 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.686+0000 7f5d85ffb640 0 client.18267 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:11:09.692 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.686+0000 7f5da7c53740 1 --2- 172.21.15.16:0/3489212812 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5567a548adb0 0x5567a5494fa0 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:09.692 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.686+0000 7f5da7c53740 1 -- 172.21.15.16:0/3489212812 >> 172.21.15.16:0/3489212812 conn(0x5567a5482010 msgr2=0x5567a5483820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:09.693 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.686+0000 7f5da7c53740 1 -- 172.21.15.16:0/3489212812 shutdown_connections 2024-09-12T02:11:09.693 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.686+0000 7f5da7c53740 1 -- 172.21.15.16:0/3489212812 wait complete. 2024-09-12T02:11:09.693 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.686+0000 7f5da7c53740 20 client.18267 trim_cache size 0 max 16384 2024-09-12T02:11:09.706 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.701+0000 7f5da7c53740 1 Processor -- start 2024-09-12T02:11:09.706 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.701+0000 7f5da7c53740 1 -- start start 2024-09-12T02:11:09.706 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.701+0000 7f5da7c53740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5567a548adb0 0x5567a54953c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:09.706 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.701+0000 7f5da7c53740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5567a548b6f0 0x5567a549c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:09.706 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.701+0000 7f5da7c53740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5567a5490780 0x5567a54a7990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:09.706 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.701+0000 7f5da7c53740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5da01a9440 con 0x5567a5490780 2024-09-12T02:11:09.706 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.701+0000 7f5da7c53740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5da01a6d10 con 0x5567a548b6f0 2024-09-12T02:11:09.707 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.701+0000 7f5da7c53740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5da01a4b80 con 0x5567a548adb0 2024-09-12T02:11:09.707 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.701+0000 7f5d9ed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5567a5490780 0x5567a54a7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:09.707 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.701+0000 7f5d9ed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5567a5490780 0x5567a54a7990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39366/0 (socket says 172.21.15.16:39366) 2024-09-12T02:11:09.707 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.701+0000 7f5d9ed9d640 1 -- 172.21.15.16:0/2042330035 learned_addr learned my addr 172.21.15.16:0/2042330035 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:11:09.707 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.701+0000 7f5d97fff640 1 --2- 172.21.15.16:0/2042330035 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5567a548b6f0 0x5567a549c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:09.707 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.701+0000 7f5d9e59c640 1 --2- 172.21.15.16:0/2042330035 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5567a548adb0 0x5567a54953c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:09.707 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.701+0000 7f5d9ed9d640 1 -- 172.21.15.16:0/2042330035 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5567a548adb0 msgr2=0x5567a54953c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:09.707 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.701+0000 7f5d9ed9d640 1 --2- 172.21.15.16:0/2042330035 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5567a548adb0 0x5567a54953c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:09.707 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.701+0000 7f5d9ed9d640 1 -- 172.21.15.16:0/2042330035 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5567a548b6f0 msgr2=0x5567a549c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:09.707 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.701+0000 7f5d9ed9d640 1 --2- 172.21.15.16:0/2042330035 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5567a548b6f0 0x5567a549c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:09.707 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.701+0000 7f5d9ed9d640 1 -- 172.21.15.16:0/2042330035 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567a5494ae0 con 0x5567a5490780 2024-09-12T02:11:09.707 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.701+0000 7f5d9e59c640 1 --2- 172.21.15.16:0/2042330035 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5567a548adb0 0x5567a54953c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:11:09.707 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.701+0000 7f5d9ed9d640 1 --2- 172.21.15.16:0/2042330035 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5567a5490780 0x5567a54a7990 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f5d98012d60 tx=0x7f5d98009a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e5cee8e94f0e59b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:09.708 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.702+0000 7f5d87fff640 1 -- 172.21.15.16:0/2042330035 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d9802e070 con 0x5567a5490780 2024-09-12T02:11:09.708 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.702+0000 7f5d87fff640 1 -- 172.21.15.16:0/2042330035 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d98021e50 con 0x5567a5490780 2024-09-12T02:11:09.708 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.702+0000 7f5d87fff640 1 -- 172.21.15.16:0/2042330035 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d9801f400 con 0x5567a5490780 2024-09-12T02:11:09.708 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.702+0000 7f5da7c53740 1 -- 172.21.15.16:0/2042330035 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5567a5490780 msgr2=0x5567a54a7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:09.708 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.702+0000 7f5da7c53740 1 --2- 172.21.15.16:0/2042330035 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5567a5490780 0x5567a54a7990 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f5d98012d60 tx=0x7f5d98009a00 comp rx=0 tx=0).stop 2024-09-12T02:11:09.708 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.702+0000 7f5da7c53740 1 -- 172.21.15.16:0/2042330035 shutdown_connections 2024-09-12T02:11:09.708 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.702+0000 7f5da7c53740 1 --2- 172.21.15.16:0/2042330035 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5567a5490780 0x5567a54a7990 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:09.708 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.702+0000 7f5da7c53740 1 --2- 172.21.15.16:0/2042330035 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5567a548b6f0 0x5567a549c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:09.708 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.702+0000 7f5da7c53740 1 --2- 172.21.15.16:0/2042330035 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5567a548adb0 0x5567a54953c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:09.708 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.702+0000 7f5da7c53740 1 -- 172.21.15.16:0/2042330035 >> 172.21.15.16:0/2042330035 conn(0x5567a5482010 msgr2=0x5567a54811c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:09.708 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.702+0000 7f5da7c53740 1 -- 172.21.15.16:0/2042330035 shutdown_connections 2024-09-12T02:11:09.708 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.702+0000 7f5da7c53740 1 -- 172.21.15.16:0/2042330035 wait complete. 2024-09-12T02:11:09.709 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.702+0000 7f5da7c53740 1 -- start start 2024-09-12T02:11:09.709 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.703+0000 7f5da7c53740 1 Processor -- start 2024-09-12T02:11:09.709 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.703+0000 7f5da7c53740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5567a548adb0 0x5567a549bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:09.709 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.703+0000 7f5da7c53740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5567a548b6f0 0x5567a549c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:09.709 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.703+0000 7f5da7c53740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5567a5490780 0x5567a549a560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:09.709 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.703+0000 7f5da7c53740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5da01a51d0 con 0x5567a548b6f0 2024-09-12T02:11:09.709 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.703+0000 7f5da7c53740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5da01a6640 con 0x5567a548adb0 2024-09-12T02:11:09.709 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.703+0000 7f5da7c53740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5da01a58a0 con 0x5567a5490780 2024-09-12T02:11:09.709 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.703+0000 7f5da7c53740 10 client.0 _ll_register_callbacks cb 0x7f5d90009290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:11:09.709 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.703+0000 7f5d97fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5567a548b6f0 0x5567a549c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:09.709 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.703+0000 7f5d97fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5567a548b6f0 0x5567a549c470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39380/0 (socket says 172.21.15.16:39380) 2024-09-12T02:11:09.709 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.703+0000 7f5d97fff640 1 -- 172.21.15.16:0/3465089097 learned_addr learned my addr 172.21.15.16:0/3465089097 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:11:09.709 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.703+0000 7f5d9e59c640 1 --2- 172.21.15.16:0/3465089097 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5567a548adb0 0x5567a549bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:09.710 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.703+0000 7f5d9ed9d640 1 --2- 172.21.15.16:0/3465089097 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5567a5490780 0x5567a549a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:09.710 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.703+0000 7f5d97fff640 1 -- 172.21.15.16:0/3465089097 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5567a5490780 msgr2=0x5567a549a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:09.710 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.703+0000 7f5d97fff640 1 --2- 172.21.15.16:0/3465089097 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5567a5490780 0x5567a549a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:09.710 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.703+0000 7f5d97fff640 1 -- 172.21.15.16:0/3465089097 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5567a548adb0 msgr2=0x5567a549bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:09.710 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.703+0000 7f5d97fff640 1 --2- 172.21.15.16:0/3465089097 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5567a548adb0 0x5567a549bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:09.710 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.703+0000 7f5d97fff640 1 -- 172.21.15.16:0/3465089097 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5567a549ad90 con 0x5567a548b6f0 2024-09-12T02:11:09.710 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.703+0000 7f5d9ed9d640 1 --2- 172.21.15.16:0/3465089097 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5567a5490780 0x5567a549a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:11:09.710 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.703+0000 7f5d97fff640 1 --2- 172.21.15.16:0/3465089097 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5567a548b6f0 0x5567a549c470 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f5d88000c00 tx=0x7f5d8800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4ec0576d308e44b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:09.710 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.703+0000 7f5d85ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:11:09.710 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.704+0000 7f5d85ffb640 1 -- 172.21.15.16:0/3465089097 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d88014040 con 0x5567a548b6f0 2024-09-12T02:11:09.710 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.704+0000 7f5d85ffb640 1 -- 172.21.15.16:0/3465089097 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f5d880054d0 con 0x5567a548b6f0 2024-09-12T02:11:09.710 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.704+0000 7f5da7c53740 1 -- 172.21.15.16:0/3465089097 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d98021e50 con 0x5567a548b6f0 2024-09-12T02:11:09.710 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.704+0000 7f5d85ffb640 1 -- 172.21.15.16:0/3465089097 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5d88010650 con 0x5567a548b6f0 2024-09-12T02:11:09.710 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.704+0000 7f5da7c53740 10 client.18273 fetch_fsmap learned FSMap version 25 2024-09-12T02:11:09.711 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.704+0000 7f5da7c53740 1 -- 172.21.15.16:0/3465089097 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5567a550c530 con 0x5567a548b6f0 2024-09-12T02:11:09.711 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.704+0000 7f5d85ffb640 1 -- 172.21.15.16:0/3465089097 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d98021e50 con 0x5567a548b6f0 2024-09-12T02:11:09.711 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.704+0000 7f5d85ffb640 1 -- 172.21.15.16:0/3465089097 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f5d88003a20 con 0x5567a548b6f0 2024-09-12T02:11:09.711 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.704+0000 7f5da7c53740 10 client.18273 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:11:09.711 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.704+0000 7f5da7c53740 10 client.18273 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:11:09.711 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.704+0000 7f5da7c53740 20 client.18273 populate_metadata read hostname 'smithi016' 2024-09-12T02:11:09.711 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.704+0000 7f5da7c53740 1 --2- 172.21.15.16:0/3465089097 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5567a550cc30 0x5567a550f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:09.711 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.704+0000 7f5da7c53740 4 client.18273 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:11:09.711 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.704+0000 7f5da7c53740 1 -- 172.21.15.16:0/3465089097 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5567a53f1d70 con 0x5567a550cc30 2024-09-12T02:11:09.711 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.705+0000 7f5d9e59c640 1 --2- 172.21.15.16:0/3465089097 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5567a550cc30 0x5567a550f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:09.711 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.705+0000 7f5d85ffb640 10 client.18273 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:11:09.711 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.705+0000 7f5d9e59c640 1 --2- 172.21.15.16:0/3465089097 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5567a550cc30 0x5567a550f050 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b88c7d81bcd89812 server_cookie=f9059a02d971df70 in_seq=0 out_seq=0 2024-09-12T02:11:09.717 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.717+0000 7f5d85ffb640 1 -- 172.21.15.16:0/3465089097 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5567a53f1d70 con 0x5567a550cc30 2024-09-12T02:11:09.718 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.717+0000 7f5d85ffb640 10 client.18273 handle_command_reply: tid=0 2024-09-12T02:11:09.718 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.717+0000 7f5da7c53740 1 client.18273 shutdown 2024-09-12T02:11:09.718 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.717+0000 7f5da7c53740 1 -- 172.21.15.16:0/3465089097 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5567a548b6f0 msgr2=0x5567a549c470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:09.718 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.717+0000 7f5da7c53740 1 --2- 172.21.15.16:0/3465089097 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5567a548b6f0 0x5567a549c470 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f5d88000c00 tx=0x7f5d8800c990 comp rx=0 tx=0).stop 2024-09-12T02:11:09.718 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.717+0000 7f5da7c53740 1 -- 172.21.15.16:0/3465089097 shutdown_connections 2024-09-12T02:11:09.718 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.717+0000 7f5da7c53740 1 --2- 172.21.15.16:0/3465089097 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5567a550cc30 0x5567a550f050 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:09.718 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.717+0000 7f5da7c53740 1 --2- 172.21.15.16:0/3465089097 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5567a5490780 0x5567a549a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:09.718 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.717+0000 7f5d85ffb640 0 client.18273 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:11:09.718 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.717+0000 7f5da7c53740 1 --2- 172.21.15.16:0/3465089097 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5567a548b6f0 0x5567a549c470 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:09.718 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.717+0000 7f5da7c53740 1 --2- 172.21.15.16:0/3465089097 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5567a548adb0 0x5567a549bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:09.718 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.717+0000 7f5da7c53740 1 -- 172.21.15.16:0/3465089097 >> 172.21.15.16:0/3465089097 conn(0x5567a5482010 msgr2=0x5567a54811c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:09.718 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.717+0000 7f5da7c53740 1 -- 172.21.15.16:0/3465089097 shutdown_connections 2024-09-12T02:11:09.718 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.717+0000 7f5da7c53740 1 -- 172.21.15.16:0/3465089097 wait complete. 2024-09-12T02:11:09.719 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.717+0000 7f5da7c53740 20 client.18273 trim_cache size 0 max 16384 2024-09-12T02:11:09.719 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-09-12T02:11:09.719 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-09-12T02:11:09.719 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-09-12T02:11:09.719 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-09-12T02:11:09.720 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.718+0000 7f5da4eb7640 1 -- 172.21.15.16:0/2347860609 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f5d8c0a2c30 msgr2=0x7f5d8c0a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:09.720 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.718+0000 7f5da4eb7640 1 --2- 172.21.15.16:0/2347860609 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f5d8c0a2c30 0x7f5d8c0a50f0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f5d98009fd0 tx=0x7f5d98009600 comp rx=0 tx=0).stop 2024-09-12T02:11:09.720 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.718+0000 7f5da4eb7640 1 -- 172.21.15.16:0/2347860609 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5da0100a40 msgr2=0x7f5da006a410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:09.720 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.718+0000 7f5da4eb7640 1 --2- 172.21.15.16:0/2347860609 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5da0100a40 0x7f5da006a410 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f5d90031610 tx=0x7f5d900043b0 comp rx=0 tx=0).stop 2024-09-12T02:11:09.720 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.718+0000 7f5da4eb7640 1 -- 172.21.15.16:0/2347860609 shutdown_connections 2024-09-12T02:11:09.720 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.718+0000 7f5da4eb7640 1 --2- 172.21.15.16:0/2347860609 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f5d8c0a2c30 0x7f5d8c0a50f0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:09.720 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.718+0000 7f5da4eb7640 1 --2- 172.21.15.16:0/2347860609 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5da01025f0 0x7f5da019f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:09.720 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.718+0000 7f5da4eb7640 1 --2- 172.21.15.16:0/2347860609 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5da0101c30 0x7f5da019ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:09.720 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.718+0000 7f5da4eb7640 1 --2- 172.21.15.16:0/2347860609 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5da0100a40 0x7f5da006a410 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:09.720 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.718+0000 7f5da4eb7640 1 -- 172.21.15.16:0/2347860609 >> 172.21.15.16:0/2347860609 conn(0x7f5da00fc1e0 msgr2=0x7f5da0104f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:09.721 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.718+0000 7f5da4eb7640 1 -- 172.21.15.16:0/2347860609 shutdown_connections 2024-09-12T02:11:09.721 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:09.718+0000 7f5da4eb7640 1 -- 172.21.15.16:0/2347860609 wait complete. 2024-09-12T02:11:10.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:11:10 smithi184 ceph-mon[35593]: pgmap v612: 97 pgs: 97 active+clean; 900 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-12T02:11:10.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:11:10 smithi033 ceph-mon[35332]: pgmap v612: 97 pgs: 97 active+clean; 900 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-12T02:11:10.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:11:10 smithi016 ceph-mon[30947]: pgmap v612: 97 pgs: 97 active+clean; 900 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-12T02:11:12.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:11:12 smithi184 ceph-mon[35593]: pgmap v613: 97 pgs: 97 active+clean; 900 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-09-12T02:11:12.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:11:12 smithi033 ceph-mon[35332]: pgmap v613: 97 pgs: 97 active+clean; 900 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-09-12T02:11:12.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:11:12 smithi016 ceph-mon[30947]: pgmap v613: 97 pgs: 97 active+clean; 900 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-09-12T02:11:14.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:11:14 smithi184 ceph-mon[35593]: pgmap v614: 97 pgs: 97 active+clean; 972 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 21 KiB/s wr, 16 op/s 2024-09-12T02:11:14.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:11:14 smithi033 ceph-mon[35332]: pgmap v614: 97 pgs: 97 active+clean; 972 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 21 KiB/s wr, 16 op/s 2024-09-12T02:11:14.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:11:14 smithi016 ceph-mon[30947]: pgmap v614: 97 pgs: 97 active+clean; 972 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 21 KiB/s wr, 16 op/s 2024-09-12T02:11:16.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:11:16 smithi184 ceph-mon[35593]: pgmap v615: 97 pgs: 97 active+clean; 972 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-09-12T02:11:16.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:11:16 smithi033 ceph-mon[35332]: pgmap v615: 97 pgs: 97 active+clean; 972 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-09-12T02:11:16.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:11:16 smithi016 ceph-mon[30947]: pgmap v615: 97 pgs: 97 active+clean; 972 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-09-12T02:11:18.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:11:18 smithi184 ceph-mon[35593]: pgmap v616: 97 pgs: 97 active+clean; 1.0 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 17 op/s 2024-09-12T02:11:18.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:11:18 smithi033 ceph-mon[35332]: pgmap v616: 97 pgs: 97 active+clean; 1.0 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 17 op/s 2024-09-12T02:11:18.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:11:18 smithi016 ceph-mon[30947]: pgmap v616: 97 pgs: 97 active+clean; 1.0 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 17 op/s 2024-09-12T02:11:20.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:11:20 smithi184 ceph-mon[35593]: pgmap v617: 97 pgs: 97 active+clean; 1.0 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-09-12T02:11:20.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:11:20 smithi033 ceph-mon[35332]: pgmap v617: 97 pgs: 97 active+clean; 1.0 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-09-12T02:11:20.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:11:20 smithi016 ceph-mon[30947]: pgmap v617: 97 pgs: 97 active+clean; 1.0 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-09-12T02:11:21.089 INFO:tasks.quiescer.fs.[cephfs]:Total duration (45.1) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-12T02:11:21.089 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 22.55 2024-09-12T02:11:21.089 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:11:21.362 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:11:21.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:21.812+0000 7f97f5e35640 1 -- 172.21.15.16:0/1381140087 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97f0106a80 msgr2=0x7f97f0106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:21.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:21.812+0000 7f97f5e35640 1 --2- 172.21.15.16:0/1381140087 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97f0106a80 0x7f97f0106e60 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f97e0009930 tx=0x7f97e002a420 comp rx=0 tx=0).stop 2024-09-12T02:11:21.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:21.813+0000 7f97f5e35640 1 -- 172.21.15.16:0/1381140087 shutdown_connections 2024-09-12T02:11:21.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:21.813+0000 7f97f5e35640 1 --2- 172.21.15.16:0/1381140087 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97f0101420 0x7f97f010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:21.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:21.813+0000 7f97f5e35640 1 --2- 172.21.15.16:0/1381140087 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f97f0100a80 0x7f97f0100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:21.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:21.813+0000 7f97f5e35640 1 --2- 172.21.15.16:0/1381140087 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97f0106a80 0x7f97f0106e60 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:21.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:21.813+0000 7f97f5e35640 1 -- 172.21.15.16:0/1381140087 >> 172.21.15.16:0/1381140087 conn(0x7f97f00fc7a0 msgr2=0x7f97f00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:21.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:21.813+0000 7f97f5e35640 1 -- 172.21.15.16:0/1381140087 shutdown_connections 2024-09-12T02:11:21.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:21.813+0000 7f97f5e35640 1 -- 172.21.15.16:0/1381140087 wait complete. 2024-09-12T02:11:21.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:21.813+0000 7f97f5e35640 1 Processor -- start 2024-09-12T02:11:21.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:21.814+0000 7f97f5e35640 1 -- start start 2024-09-12T02:11:21.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:21.814+0000 7f97f5e35640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f97f0100a80 0x7f97f01992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:21.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:21.814+0000 7f97f5e35640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97f0101420 0x7f97f0199830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:21.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:21.814+0000 7f97f5e35640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97f0106a80 0x7f97f01a4370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:21.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:21.814+0000 7f97f5e35640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f97f0110e80 con 0x7f97f0106a80 2024-09-12T02:11:21.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:21.814+0000 7f97f5e35640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f97f0110d00 con 0x7f97f0101420 2024-09-12T02:11:21.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:21.814+0000 7f97f5e35640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f97f0111000 con 0x7f97f0100a80 2024-09-12T02:11:21.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:21.814+0000 7f97effff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97f0106a80 0x7f97f01a4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:21.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:21.814+0000 7f97effff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97f0106a80 0x7f97f01a4370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41396/0 (socket says 172.21.15.16:41396) 2024-09-12T02:11:21.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:21.814+0000 7f97effff640 1 -- 172.21.15.16:0/2410448778 learned_addr learned my addr 172.21.15.16:0/2410448778 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:11:21.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:21.814+0000 7f97ef7fe640 1 --2- 172.21.15.16:0/2410448778 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f97f0100a80 0x7f97f01992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:21.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:21.814+0000 7f97eeffd640 1 --2- 172.21.15.16:0/2410448778 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97f0101420 0x7f97f0199830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:21.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:21.814+0000 7f97effff640 1 -- 172.21.15.16:0/2410448778 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f97f0100a80 msgr2=0x7f97f01992f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:21.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:21.814+0000 7f97effff640 1 --2- 172.21.15.16:0/2410448778 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f97f0100a80 0x7f97f01992f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:21.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:21.814+0000 7f97effff640 1 -- 172.21.15.16:0/2410448778 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97f0101420 msgr2=0x7f97f0199830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:21.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:21.814+0000 7f97effff640 1 --2- 172.21.15.16:0/2410448778 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97f0101420 0x7f97f0199830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:21.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:21.814+0000 7f97effff640 1 -- 172.21.15.16:0/2410448778 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97f01a48b0 con 0x7f97f0106a80 2024-09-12T02:11:21.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:21.814+0000 7f97effff640 1 --2- 172.21.15.16:0/2410448778 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97f0106a80 0x7f97f01a4370 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f97e8009fd0 tx=0x7f97e800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:21.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:21.815+0000 7f97ecff9640 1 -- 172.21.15.16:0/2410448778 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97e8017070 con 0x7f97f0106a80 2024-09-12T02:11:21.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:21.815+0000 7f97ecff9640 1 -- 172.21.15.16:0/2410448778 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f97e8003e50 con 0x7f97f0106a80 2024-09-12T02:11:21.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:21.815+0000 7f97f5e35640 1 -- 172.21.15.16:0/2410448778 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97f01a4a80 con 0x7f97f0106a80 2024-09-12T02:11:21.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:21.815+0000 7f97ecff9640 1 -- 172.21.15.16:0/2410448778 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97e80106c0 con 0x7f97f0106a80 2024-09-12T02:11:21.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:21.815+0000 7f97f5e35640 1 -- 172.21.15.16:0/2410448778 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97f01a4f40 con 0x7f97f0106a80 2024-09-12T02:11:21.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:21.816+0000 7f97f5e35640 1 -- 172.21.15.16:0/2410448778 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97f0100ee0 con 0x7f97f0106a80 2024-09-12T02:11:21.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:21.817+0000 7f97ecff9640 1 -- 172.21.15.16:0/2410448778 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f97e8003970 con 0x7f97f0106a80 2024-09-12T02:11:21.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:21.818+0000 7f97ecff9640 1 --2- 172.21.15.16:0/2410448778 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f97d80a2b80 0x7f97d80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:21.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:21.818+0000 7f97ecff9640 1 -- 172.21.15.16:0/2410448778 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f97e80cec60 con 0x7f97f0106a80 2024-09-12T02:11:21.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:21.818+0000 7f97ef7fe640 1 --2- 172.21.15.16:0/2410448778 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f97d80a2b80 0x7f97d80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:21.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:21.818+0000 7f97ef7fe640 1 --2- 172.21.15.16:0/2410448778 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f97d80a2b80 0x7f97d80a5040 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f97e0009900 tx=0x7f97e00058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:21.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:21.821+0000 7f97ecff9640 1 -- 172.21.15.16:0/2410448778 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f97e809bc60 con 0x7f97f0106a80 2024-09-12T02:11:22.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:22.031+0000 7f97f5e35640 1 -- 172.21.15.16:0/2410448778 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f97f0108ff0 con 0x7f97f0106a80 2024-09-12T02:11:22.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:22.032+0000 7f97ecff9640 1 -- 172.21.15.16:0/2410448778 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11911 (secure 0 0 0) 0x7f97e809b560 con 0x7f97f0106a80 2024-09-12T02:11:22.033 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T02:11:22.034 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":25,"btime":"2024-09-12T02:01:37:192021+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3279123279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3279123279},{"type":"v1","addr":"172.21.15.16:6835","nonce":3279123279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4119452382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4119452382},{"type":"v1","addr":"172.21.15.16:6837","nonce":4119452382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/3263823165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3263823165},{"type":"v1","addr":"172.21.15.16:6839","nonce":3263823165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/1510290732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":1510290732},{"type":"v1","addr":"172.21.15.16:6841","nonce":1510290732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/279674603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":279674603},{"type":"v1","addr":"172.21.15.33:6833","nonce":279674603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3555728028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3555728028},{"type":"v1","addr":"172.21.15.33:6835","nonce":3555728028}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T01:59:56.117608+0000","modified":"2024-09-12T02:01:36.294760+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24482,"mds_1":24481,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6833/1563374106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1563374106},{"type":"v1","addr":"172.21.15.184:6833","nonce":1563374106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6837/1892833868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1892833868},{"type":"v1","addr":"172.21.15.33:6837","nonce":1892833868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/1457679686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1457679686},{"type":"v1","addr":"172.21.15.184:6835","nonce":1457679686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.184:6837/1793393087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":1793393087},{"type":"v1","addr":"172.21.15.184:6837","nonce":1793393087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.184:6839/4203347941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":4203347941},{"type":"v1","addr":"172.21.15.184:6839","nonce":4203347941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.33:6839/4136670204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":4136670204},{"type":"v1","addr":"172.21.15.33:6839","nonce":4136670204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24481,24482,24469]},"id":1}]} 2024-09-12T02:11:22.034 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 25 2024-09-12T02:11:22.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:22.034+0000 7f97f5e35640 1 -- 172.21.15.16:0/2410448778 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f97d80a2b80 msgr2=0x7f97d80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:22.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:22.034+0000 7f97f5e35640 1 --2- 172.21.15.16:0/2410448778 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f97d80a2b80 0x7f97d80a5040 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f97e0009900 tx=0x7f97e00058d0 comp rx=0 tx=0).stop 2024-09-12T02:11:22.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:22.034+0000 7f97f5e35640 1 -- 172.21.15.16:0/2410448778 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97f0106a80 msgr2=0x7f97f01a4370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:22.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:22.034+0000 7f97f5e35640 1 --2- 172.21.15.16:0/2410448778 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97f0106a80 0x7f97f01a4370 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f97e8009fd0 tx=0x7f97e800b260 comp rx=0 tx=0).stop 2024-09-12T02:11:22.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:22.034+0000 7f97f5e35640 1 -- 172.21.15.16:0/2410448778 shutdown_connections 2024-09-12T02:11:22.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:22.034+0000 7f97f5e35640 1 --2- 172.21.15.16:0/2410448778 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f97d80a2b80 0x7f97d80a5040 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:22.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:22.034+0000 7f97f5e35640 1 --2- 172.21.15.16:0/2410448778 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97f0106a80 0x7f97f01a4370 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:22.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:22.034+0000 7f97f5e35640 1 --2- 172.21.15.16:0/2410448778 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97f0101420 0x7f97f0199830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:22.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:22.034+0000 7f97f5e35640 1 --2- 172.21.15.16:0/2410448778 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f97f0100a80 0x7f97f01992f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:22.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:22.034+0000 7f97f5e35640 1 -- 172.21.15.16:0/2410448778 >> 172.21.15.16:0/2410448778 conn(0x7f97f00fc7a0 msgr2=0x7f97f00fcf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:22.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:22.034+0000 7f97f5e35640 1 -- 172.21.15.16:0/2410448778 shutdown_connections 2024-09-12T02:11:22.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:22.034+0000 7f97f5e35640 1 -- 172.21.15.16:0/2410448778 wait complete. 2024-09-12T02:11:22.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:11:22 smithi184 ceph-mon[35593]: pgmap v618: 97 pgs: 97 active+clean; 1.0 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-12T02:11:22.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:11:22 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/2410448778' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:11:22.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:11:22 smithi033 ceph-mon[35332]: pgmap v618: 97 pgs: 97 active+clean; 1.0 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-12T02:11:22.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:11:22 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/2410448778' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:11:22.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:11:22 smithi016 ceph-mon[30947]: pgmap v618: 97 pgs: 97 active+clean; 1.0 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-12T02:11:22.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:11:22 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/2410448778' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:11:22.819 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.818+0000 7ff2973d6640 1 -- 172.21.15.16:0/859112117 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff290101c30 msgr2=0x7ff2901020b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:22.819 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.818+0000 7ff2973d6640 1 --2- 172.21.15.16:0/859112117 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff290101c30 0x7ff2901020b0 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7ff280009900 tx=0x7ff28002a410 comp rx=0 tx=0).stop 2024-09-12T02:11:22.820 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.818+0000 7ff2973d6640 1 -- 172.21.15.16:0/859112117 shutdown_connections 2024-09-12T02:11:22.820 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.818+0000 7ff2973d6640 1 --2- 172.21.15.16:0/859112117 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff2901025f0 0x7ff290108e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:22.820 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.818+0000 7ff2973d6640 1 --2- 172.21.15.16:0/859112117 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff290101c30 0x7ff2901020b0 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:22.820 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.818+0000 7ff2973d6640 1 --2- 172.21.15.16:0/859112117 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff290100a40 0x7ff290100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:22.820 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.818+0000 7ff2973d6640 1 -- 172.21.15.16:0/859112117 >> 172.21.15.16:0/859112117 conn(0x7ff2900fc1e0 msgr2=0x7ff2900fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:22.820 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.818+0000 7ff2973d6640 1 -- 172.21.15.16:0/859112117 shutdown_connections 2024-09-12T02:11:22.820 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.818+0000 7ff2973d6640 1 -- 172.21.15.16:0/859112117 wait complete. 2024-09-12T02:11:22.821 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24482 quiesce db / --timeout 90 --expiration 142.55 --await-for 45.0' 2024-09-12T02:11:22.822 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24482 quiesce db / --timeout 90 --expiration 142.55 --await-for 45.0 2024-09-12T02:11:22.826 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.818+0000 7ff2973d6640 1 Processor -- start 2024-09-12T02:11:22.826 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.819+0000 7ff2973d6640 1 -- start start 2024-09-12T02:11:22.826 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.819+0000 7ff2973d6640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff290100a40 0x7ff290197d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:22.826 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.819+0000 7ff2973d6640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff290101c30 0x7ff290198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:22.827 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.819+0000 7ff2973d6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2901025f0 0x7ff29019f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:22.827 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.819+0000 7ff2973d6640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff29010c6a0 con 0x7ff2901025f0 2024-09-12T02:11:22.827 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.819+0000 7ff2973d6640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff29010c520 con 0x7ff290101c30 2024-09-12T02:11:22.827 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.819+0000 7ff2973d6640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff29010c820 con 0x7ff290100a40 2024-09-12T02:11:22.827 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.819+0000 7ff295973640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2901025f0 0x7ff29019f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:22.827 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.819+0000 7ff295973640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2901025f0 0x7ff29019f300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41412/0 (socket says 172.21.15.16:41412) 2024-09-12T02:11:22.827 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.819+0000 7ff295973640 1 -- 172.21.15.16:0/2982785158 learned_addr learned my addr 172.21.15.16:0/2982785158 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:11:22.827 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.819+0000 7ff295172640 1 --2- 172.21.15.16:0/2982785158 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff290100a40 0x7ff290197d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:22.827 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.819+0000 7ff294971640 1 --2- 172.21.15.16:0/2982785158 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff290101c30 0x7ff290198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:22.828 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.819+0000 7ff295973640 1 -- 172.21.15.16:0/2982785158 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff290100a40 msgr2=0x7ff290197d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:22.828 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.819+0000 7ff295973640 1 --2- 172.21.15.16:0/2982785158 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff290100a40 0x7ff290197d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:22.828 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.819+0000 7ff295973640 1 -- 172.21.15.16:0/2982785158 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff290101c30 msgr2=0x7ff290198290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:22.828 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.819+0000 7ff295973640 1 --2- 172.21.15.16:0/2982785158 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff290101c30 0x7ff290198290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:22.828 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.819+0000 7ff295973640 1 -- 172.21.15.16:0/2982785158 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff29019f970 con 0x7ff2901025f0 2024-09-12T02:11:22.828 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.820+0000 7ff295973640 1 --2- 172.21.15.16:0/2982785158 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2901025f0 0x7ff29019f300 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7ff28800cce0 tx=0x7ff28800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:22.828 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.820+0000 7ff2867fc640 1 -- 172.21.15.16:0/2982785158 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff28800f450 con 0x7ff2901025f0 2024-09-12T02:11:22.828 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.820+0000 7ff2867fc640 1 -- 172.21.15.16:0/2982785158 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff28800fad0 con 0x7ff2901025f0 2024-09-12T02:11:22.828 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.820+0000 7ff2867fc640 1 -- 172.21.15.16:0/2982785158 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff288018c10 con 0x7ff2901025f0 2024-09-12T02:11:22.828 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.820+0000 7ff2973d6640 1 -- 172.21.15.16:0/2982785158 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff29019fc60 con 0x7ff2901025f0 2024-09-12T02:11:22.828 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.820+0000 7ff2973d6640 1 -- 172.21.15.16:0/2982785158 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2901a0180 con 0x7ff2901025f0 2024-09-12T02:11:22.829 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.822+0000 7ff2867fc640 1 -- 172.21.15.16:0/2982785158 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff28800f5f0 con 0x7ff2901025f0 2024-09-12T02:11:22.829 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.823+0000 7ff29a252740 1 Processor -- start 2024-09-12T02:11:22.829 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.823+0000 7ff29a252740 1 -- start start 2024-09-12T02:11:22.829 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.823+0000 7ff29a252740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558b46ee39a0 0x558b46ee3da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:22.829 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.823+0000 7ff29a252740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b46ee42e0 0x558b46ee8da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:22.829 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.823+0000 7ff29a252740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b46ee9370 0x558b46eeb830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:22.829 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.823+0000 7ff29a252740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x558b46e472f0 con 0x558b46ee9370 2024-09-12T02:11:22.829 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.823+0000 7ff29a252740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x558b46def270 con 0x558b46ee42e0 2024-09-12T02:11:22.829 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.823+0000 7ff29a252740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x558b46e432b0 con 0x558b46ee39a0 2024-09-12T02:11:22.829 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.823+0000 7ff294971640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b46ee9370 0x558b46eeb830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:22.829 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.823+0000 7ff2867fc640 1 --2- 172.21.15.16:0/2982785158 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7ff27c0a2cb0 0x7ff27c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:22.829 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.823+0000 7ff294971640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b46ee9370 0x558b46eeb830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41428/0 (socket says 172.21.15.16:41428) 2024-09-12T02:11:22.829 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.823+0000 7ff294971640 1 -- 172.21.15.16:0/3053277179 learned_addr learned my addr 172.21.15.16:0/3053277179 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:11:22.830 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.823+0000 7ff295172640 1 --2- 172.21.15.16:0/3053277179 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b46ee42e0 0x558b46ee8da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:22.830 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.823+0000 7ff295973640 1 --2- 172.21.15.16:0/2982785158 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7ff27c0a2cb0 0x7ff27c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:22.830 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.823+0000 7ff294971640 1 --2- 172.21.15.16:0/3053277179 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558b46ee39a0 0x558b46ee3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:22.830 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.823+0000 7ff2867fc640 1 -- 172.21.15.16:0/2982785158 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7ff2880cf600 con 0x7ff2901025f0 2024-09-12T02:11:22.830 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.823+0000 7ff294971640 1 -- 172.21.15.16:0/3053277179 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558b46ee39a0 msgr2=0x558b46ee3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:22.830 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.823+0000 7ff294971640 1 --2- 172.21.15.16:0/3053277179 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558b46ee39a0 0x558b46ee3da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:22.830 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.823+0000 7ff294971640 1 -- 172.21.15.16:0/3053277179 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b46ee42e0 msgr2=0x558b46ee8da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:22.830 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.823+0000 7ff294971640 1 --2- 172.21.15.16:0/3053277179 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b46ee42e0 0x558b46ee8da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:22.830 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.823+0000 7ff294971640 1 -- 172.21.15.16:0/3053277179 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b46dee340 con 0x558b46ee9370 2024-09-12T02:11:22.830 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.823+0000 7ff295172640 1 --2- 172.21.15.16:0/3053277179 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b46ee42e0 0x558b46ee8da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:11:22.830 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.824+0000 7ff294971640 1 --2- 172.21.15.16:0/3053277179 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b46ee9370 0x558b46eeb830 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7ff280009900 tx=0x7ff280005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc55d3f29c480fbe server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:22.830 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.824+0000 7ff295973640 1 --2- 172.21.15.16:0/2982785158 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7ff27c0a2cb0 0x7ff27c0a5170 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7ff2880cd100 tx=0x7ff288066ae0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:22.835 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.824+0000 7ff27b7fe640 1 -- 172.21.15.16:0/3053277179 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff280044020 con 0x558b46ee9370 2024-09-12T02:11:22.835 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.824+0000 7ff27b7fe640 1 -- 172.21.15.16:0/3053277179 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff280005030 con 0x558b46ee9370 2024-09-12T02:11:22.835 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.824+0000 7ff27b7fe640 1 -- 172.21.15.16:0/3053277179 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff280035070 con 0x558b46ee9370 2024-09-12T02:11:22.835 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.825+0000 7ff29a252740 1 -- 172.21.15.16:0/3053277179 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b46ee9370 msgr2=0x558b46eeb830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:22.836 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.825+0000 7ff29a252740 1 --2- 172.21.15.16:0/3053277179 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b46ee9370 0x558b46eeb830 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7ff280009900 tx=0x7ff280005fd0 comp rx=0 tx=0).stop 2024-09-12T02:11:22.836 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.825+0000 7ff29a252740 1 -- 172.21.15.16:0/3053277179 shutdown_connections 2024-09-12T02:11:22.836 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.825+0000 7ff29a252740 1 --2- 172.21.15.16:0/3053277179 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b46ee9370 0x558b46eeb830 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:22.836 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.825+0000 7ff29a252740 1 --2- 172.21.15.16:0/3053277179 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b46ee42e0 0x558b46ee8da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:22.836 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.825+0000 7ff29a252740 1 --2- 172.21.15.16:0/3053277179 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558b46ee39a0 0x558b46ee3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:22.836 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.825+0000 7ff29a252740 1 -- 172.21.15.16:0/3053277179 >> 172.21.15.16:0/3053277179 conn(0x558b46edf010 msgr2=0x558b46edf410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:22.836 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.825+0000 7ff29a252740 1 -- 172.21.15.16:0/3053277179 shutdown_connections 2024-09-12T02:11:22.836 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.825+0000 7ff29a252740 1 -- 172.21.15.16:0/3053277179 wait complete. 2024-09-12T02:11:22.836 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.825+0000 7ff29a252740 1 -- start start 2024-09-12T02:11:22.836 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.825+0000 7ff29a252740 1 Processor -- start 2024-09-12T02:11:22.836 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.826+0000 7ff29a252740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558b46ee39a0 0x558b46eedbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:22.836 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.826+0000 7ff29a252740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b46ee42e0 0x558b46eee100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:22.837 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.826+0000 7ff29a252740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b46ee9370 0x558b46ef5140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:22.837 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.826+0000 7ff29a252740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x558b46eec2e0 con 0x558b46ee42e0 2024-09-12T02:11:22.837 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.826+0000 7ff29a252740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x558b46dd0290 con 0x558b46ee9370 2024-09-12T02:11:22.837 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.826+0000 7ff29a252740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x558b46eec460 con 0x558b46ee39a0 2024-09-12T02:11:22.837 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.826+0000 7ff29a252740 10 client.0 _ll_register_callbacks cb 0x558b46e01140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:11:22.837 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.826+0000 7ff295172640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b46ee42e0 0x558b46eee100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:22.837 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.826+0000 7ff295172640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b46ee42e0 0x558b46eee100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41442/0 (socket says 172.21.15.16:41442) 2024-09-12T02:11:22.837 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.826+0000 7ff295172640 1 -- 172.21.15.16:0/3219833874 learned_addr learned my addr 172.21.15.16:0/3219833874 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:11:22.837 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.826+0000 7ff294971640 1 --2- 172.21.15.16:0/3219833874 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558b46ee39a0 0x558b46eedbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:22.837 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.826+0000 7ff294971640 1 --2- 172.21.15.16:0/3219833874 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b46ee9370 0x558b46ef5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:22.837 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.826+0000 7ff295172640 1 -- 172.21.15.16:0/3219833874 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558b46ee39a0 msgr2=0x558b46eedbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:22.837 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.826+0000 7ff295172640 1 --2- 172.21.15.16:0/3219833874 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558b46ee39a0 0x558b46eedbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:22.837 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.826+0000 7ff295172640 1 -- 172.21.15.16:0/3219833874 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b46ee9370 msgr2=0x558b46ef5140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:22.838 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.826+0000 7ff295172640 1 --2- 172.21.15.16:0/3219833874 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b46ee9370 0x558b46ef5140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:22.838 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.826+0000 7ff295172640 1 -- 172.21.15.16:0/3219833874 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b46b5a210 con 0x558b46ee42e0 2024-09-12T02:11:22.838 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.826+0000 7ff294971640 1 --2- 172.21.15.16:0/3219833874 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558b46ee39a0 0x558b46eedbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:11:22.838 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.826+0000 7ff295172640 1 --2- 172.21.15.16:0/3219833874 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b46ee42e0 0x558b46eee100 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7ff28c00bf50 tx=0x7ff28c0118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9558703eed0313f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:22.838 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.826+0000 7ff278ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:11:22.838 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.827+0000 7ff278ff9640 1 -- 172.21.15.16:0/3219833874 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff28c004030 con 0x558b46ee42e0 2024-09-12T02:11:22.838 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.827+0000 7ff278ff9640 1 -- 172.21.15.16:0/3219833874 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7ff28c0041d0 con 0x558b46ee42e0 2024-09-12T02:11:22.838 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.827+0000 7ff29a252740 1 -- 172.21.15.16:0/3219833874 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff280005030 con 0x558b46ee42e0 2024-09-12T02:11:22.838 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.827+0000 7ff278ff9640 1 -- 172.21.15.16:0/3219833874 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff28c0043d0 con 0x558b46ee42e0 2024-09-12T02:11:22.838 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.827+0000 7ff29a252740 10 client.18291 fetch_fsmap learned FSMap version 25 2024-09-12T02:11:22.838 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.827+0000 7ff29a252740 1 -- 172.21.15.16:0/3219833874 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x558b46c26620 con 0x558b46ee42e0 2024-09-12T02:11:22.838 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.827+0000 7ff278ff9640 1 -- 172.21.15.16:0/3219833874 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff280005030 con 0x558b46ee42e0 2024-09-12T02:11:22.838 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.827+0000 7ff278ff9640 1 -- 172.21.15.16:0/3219833874 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7ff28c003a00 con 0x558b46ee42e0 2024-09-12T02:11:22.839 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.827+0000 7ff29a252740 10 client.18291 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:11:22.839 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.827+0000 7ff29a252740 10 client.18291 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:11:22.839 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.827+0000 7ff29a252740 20 client.18291 populate_metadata read hostname 'smithi016' 2024-09-12T02:11:22.839 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.827+0000 7ff29a252740 1 --2- 172.21.15.16:0/3219833874 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x558b46ef98f0 0x558b46efbd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:22.839 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.827+0000 7ff29a252740 4 client.18291 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:11:22.839 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.827+0000 7ff29a252740 1 -- 172.21.15.16:0/3219833874 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558b46e54a20 con 0x558b46ef98f0 2024-09-12T02:11:22.839 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.828+0000 7ff294971640 1 --2- 172.21.15.16:0/3219833874 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x558b46ef98f0 0x558b46efbd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:22.839 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.828+0000 7ff294971640 1 --2- 172.21.15.16:0/3219833874 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x558b46ef98f0 0x558b46efbd10 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dbefd86a4f168060 server_cookie=a17a50ff44d09989 in_seq=0 out_seq=0 2024-09-12T02:11:22.839 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.828+0000 7ff278ff9640 10 client.18291 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:11:22.839 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.829+0000 7ff278ff9640 1 -- 172.21.15.16:0/3219833874 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558b46e54a20 con 0x558b46ef98f0 2024-09-12T02:11:22.839 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.829+0000 7ff278ff9640 10 client.18291 handle_command_reply: tid=0 2024-09-12T02:11:22.839 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.829+0000 7ff29a252740 1 client.18291 shutdown 2024-09-12T02:11:22.839 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.829+0000 7ff29a252740 1 -- 172.21.15.16:0/3219833874 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b46ee42e0 msgr2=0x558b46eee100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:22.840 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.829+0000 7ff29a252740 1 --2- 172.21.15.16:0/3219833874 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b46ee42e0 0x558b46eee100 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7ff28c00bf50 tx=0x7ff28c0118d0 comp rx=0 tx=0).stop 2024-09-12T02:11:22.840 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.829+0000 7ff29a252740 1 -- 172.21.15.16:0/3219833874 shutdown_connections 2024-09-12T02:11:22.840 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.829+0000 7ff29a252740 1 --2- 172.21.15.16:0/3219833874 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x558b46ef98f0 0x558b46efbd10 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:22.840 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.829+0000 7ff29a252740 1 --2- 172.21.15.16:0/3219833874 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b46ee9370 0x558b46ef5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:22.840 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.829+0000 7ff29a252740 1 --2- 172.21.15.16:0/3219833874 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b46ee42e0 0x558b46eee100 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:22.840 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.829+0000 7ff278ff9640 0 client.18291 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:11:22.840 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.829+0000 7ff29a252740 1 --2- 172.21.15.16:0/3219833874 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558b46ee39a0 0x558b46eedbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:22.840 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.829+0000 7ff29a252740 1 -- 172.21.15.16:0/3219833874 >> 172.21.15.16:0/3219833874 conn(0x558b46edf010 msgr2=0x558b46ee0820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:22.840 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.829+0000 7ff29a252740 1 -- 172.21.15.16:0/3219833874 shutdown_connections 2024-09-12T02:11:22.840 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.829+0000 7ff29a252740 1 -- 172.21.15.16:0/3219833874 wait complete. 2024-09-12T02:11:22.840 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.829+0000 7ff29a252740 20 client.18291 trim_cache size 0 max 16384 2024-09-12T02:11:22.846 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.844+0000 7ff29a252740 1 Processor -- start 2024-09-12T02:11:22.846 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.844+0000 7ff29a252740 1 -- start start 2024-09-12T02:11:22.847 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.844+0000 7ff29a252740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b46ee39a0 0x558b46eedbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:22.847 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.844+0000 7ff29a252740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b46ee42e0 0x558b46eee100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:22.847 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.844+0000 7ff29a252740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558b46ee9370 0x558b46ef5100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:22.847 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.844+0000 7ff29a252740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff2901ab610 con 0x558b46ee42e0 2024-09-12T02:11:22.847 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.844+0000 7ff29a252740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff2901a8ee0 con 0x558b46ee39a0 2024-09-12T02:11:22.847 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.844+0000 7ff29a252740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff2901a6d30 con 0x558b46ee9370 2024-09-12T02:11:22.847 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.845+0000 7ff295172640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b46ee42e0 0x558b46eee100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:22.847 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.845+0000 7ff295172640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b46ee42e0 0x558b46eee100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41444/0 (socket says 172.21.15.16:41444) 2024-09-12T02:11:22.847 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.845+0000 7ff295172640 1 -- 172.21.15.16:0/3823592780 learned_addr learned my addr 172.21.15.16:0/3823592780 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:11:22.847 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.845+0000 7ff294971640 1 --2- 172.21.15.16:0/3823592780 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b46ee39a0 0x558b46eedbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:22.847 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.845+0000 7ff294971640 1 --2- 172.21.15.16:0/3823592780 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558b46ee9370 0x558b46ef5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:22.847 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.845+0000 7ff295172640 1 -- 172.21.15.16:0/3823592780 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558b46ee9370 msgr2=0x558b46ef5100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:22.848 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.845+0000 7ff295172640 1 --2- 172.21.15.16:0/3823592780 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558b46ee9370 0x558b46ef5100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:22.848 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.845+0000 7ff295172640 1 -- 172.21.15.16:0/3823592780 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b46ee39a0 msgr2=0x558b46eedbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:22.848 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.845+0000 7ff295172640 1 --2- 172.21.15.16:0/3823592780 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b46ee39a0 0x558b46eedbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:22.848 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.845+0000 7ff295172640 1 -- 172.21.15.16:0/3823592780 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b46edee30 con 0x558b46ee42e0 2024-09-12T02:11:22.848 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.845+0000 7ff294971640 1 --2- 172.21.15.16:0/3823592780 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558b46ee9370 0x558b46ef5100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:11:22.848 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.845+0000 7ff295172640 1 --2- 172.21.15.16:0/3823592780 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b46ee42e0 0x558b46eee100 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7ff28c003ec0 tx=0x7ff28c01a690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7190dfc67664ff9d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:22.848 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.845+0000 7ff27affd640 1 -- 172.21.15.16:0/3823592780 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff28c029040 con 0x558b46ee42e0 2024-09-12T02:11:22.848 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.845+0000 7ff27affd640 1 -- 172.21.15.16:0/3823592780 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff28c003ad0 con 0x558b46ee42e0 2024-09-12T02:11:22.848 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.845+0000 7ff27affd640 1 -- 172.21.15.16:0/3823592780 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff28c005040 con 0x558b46ee42e0 2024-09-12T02:11:22.852 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.846+0000 7ff29a252740 1 -- 172.21.15.16:0/3823592780 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b46ee42e0 msgr2=0x558b46eee100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:22.852 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.846+0000 7ff29a252740 1 --2- 172.21.15.16:0/3823592780 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b46ee42e0 0x558b46eee100 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7ff28c003ec0 tx=0x7ff28c01a690 comp rx=0 tx=0).stop 2024-09-12T02:11:22.852 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.847+0000 7ff29a252740 1 -- 172.21.15.16:0/3823592780 shutdown_connections 2024-09-12T02:11:22.852 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.847+0000 7ff29a252740 1 --2- 172.21.15.16:0/3823592780 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558b46ee9370 0x558b46ef5100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:22.852 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.847+0000 7ff29a252740 1 --2- 172.21.15.16:0/3823592780 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b46ee42e0 0x558b46eee100 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:22.852 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.847+0000 7ff29a252740 1 --2- 172.21.15.16:0/3823592780 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b46ee39a0 0x558b46eedbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:22.852 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.847+0000 7ff29a252740 1 -- 172.21.15.16:0/3823592780 >> 172.21.15.16:0/3823592780 conn(0x558b46edf010 msgr2=0x558b46ee4a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:22.852 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.847+0000 7ff29a252740 1 -- 172.21.15.16:0/3823592780 shutdown_connections 2024-09-12T02:11:22.852 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.847+0000 7ff29a252740 1 -- 172.21.15.16:0/3823592780 wait complete. 2024-09-12T02:11:22.852 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.847+0000 7ff29a252740 1 -- start start 2024-09-12T02:11:22.852 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.847+0000 7ff29a252740 1 Processor -- start 2024-09-12T02:11:22.852 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.847+0000 7ff29a252740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558b46ee39a0 0x558b46ef24a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:22.853 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.847+0000 7ff29a252740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b46ee42e0 0x558b46ef29e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:22.853 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.847+0000 7ff29a252740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b46ee9370 0x558b46ef39f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:22.853 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.847+0000 7ff29a252740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff2901a73a0 con 0x558b46ee42e0 2024-09-12T02:11:22.853 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.847+0000 7ff29a252740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff2901a8810 con 0x558b46ee9370 2024-09-12T02:11:22.853 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.848+0000 7ff29a252740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff2901a7a70 con 0x558b46ee39a0 2024-09-12T02:11:22.853 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.848+0000 7ff29a252740 10 client.0 _ll_register_callbacks cb 0x7ff28c00b820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:11:22.853 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.848+0000 7ff295172640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b46ee42e0 0x558b46ef29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:22.853 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.848+0000 7ff295172640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b46ee42e0 0x558b46ef29e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41458/0 (socket says 172.21.15.16:41458) 2024-09-12T02:11:22.853 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.848+0000 7ff294971640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b46ee9370 0x558b46ef39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:22.853 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.848+0000 7ff295172640 1 -- 172.21.15.16:0/2497189033 learned_addr learned my addr 172.21.15.16:0/2497189033 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:11:22.853 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.848+0000 7ff294971640 1 --2- 172.21.15.16:0/2497189033 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558b46ee39a0 0x558b46ef24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:22.853 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.848+0000 7ff295172640 1 -- 172.21.15.16:0/2497189033 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558b46ee39a0 msgr2=0x558b46ef24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:22.853 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.848+0000 7ff295172640 1 --2- 172.21.15.16:0/2497189033 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558b46ee39a0 0x558b46ef24a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:22.854 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.848+0000 7ff295172640 1 -- 172.21.15.16:0/2497189033 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b46ee9370 msgr2=0x558b46ef39f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:22.854 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.848+0000 7ff295172640 1 --2- 172.21.15.16:0/2497189033 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b46ee9370 0x558b46ef39f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:22.854 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.848+0000 7ff295172640 1 -- 172.21.15.16:0/2497189033 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b46ef4000 con 0x558b46ee42e0 2024-09-12T02:11:22.854 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.848+0000 7ff295172640 1 --2- 172.21.15.16:0/2497189033 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b46ee42e0 0x558b46ef29e0 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7ff28c0059f0 tx=0x7ff28c01a400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb53c4db63f6fd79 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:22.854 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.848+0000 7ff278ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:11:22.854 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.848+0000 7ff278ff9640 1 -- 172.21.15.16:0/2497189033 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff28c03b020 con 0x558b46ee42e0 2024-09-12T02:11:22.854 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.848+0000 7ff29a252740 1 -- 172.21.15.16:0/2497189033 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff28c003ad0 con 0x558b46ee42e0 2024-09-12T02:11:22.854 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.848+0000 7ff278ff9640 1 -- 172.21.15.16:0/2497189033 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7ff28c0035f0 con 0x558b46ee42e0 2024-09-12T02:11:22.854 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.849+0000 7ff278ff9640 1 -- 172.21.15.16:0/2497189033 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff28c029040 con 0x558b46ee42e0 2024-09-12T02:11:22.854 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.849+0000 7ff29a252740 10 client.18297 fetch_fsmap learned FSMap version 25 2024-09-12T02:11:22.854 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.849+0000 7ff29a252740 1 -- 172.21.15.16:0/2497189033 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x558b46ef5280 con 0x558b46ee42e0 2024-09-12T02:11:22.854 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.849+0000 7ff278ff9640 1 -- 172.21.15.16:0/2497189033 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff28c003ad0 con 0x558b46ee42e0 2024-09-12T02:11:22.854 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.849+0000 7ff278ff9640 1 -- 172.21.15.16:0/2497189033 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7ff28c040070 con 0x558b46ee42e0 2024-09-12T02:11:22.854 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.849+0000 7ff29a252740 10 client.18297 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:11:22.855 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.849+0000 7ff29a252740 10 client.18297 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:11:22.855 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.849+0000 7ff29a252740 20 client.18297 populate_metadata read hostname 'smithi016' 2024-09-12T02:11:22.855 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.849+0000 7ff29a252740 1 --2- 172.21.15.16:0/2497189033 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x558b46f64a80 0x558b46f66ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:22.855 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.849+0000 7ff29a252740 4 client.18297 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:11:22.855 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.849+0000 7ff29a252740 1 -- 172.21.15.16:0/2497189033 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558b46e4ed70 con 0x558b46f64a80 2024-09-12T02:11:22.855 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.849+0000 7ff294971640 1 --2- 172.21.15.16:0/2497189033 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x558b46f64a80 0x558b46f66ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:22.855 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.850+0000 7ff278ff9640 10 client.18297 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:11:22.855 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.850+0000 7ff294971640 1 --2- 172.21.15.16:0/2497189033 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x558b46f64a80 0x558b46f66ea0 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8675489ae210619f server_cookie=c6cbc3493bd82a81 in_seq=0 out_seq=0 2024-09-12T02:11:22.865 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.864+0000 7ff278ff9640 1 -- 172.21.15.16:0/2497189033 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558b46e4ed70 con 0x558b46f64a80 2024-09-12T02:11:22.865 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.864+0000 7ff278ff9640 10 client.18297 handle_command_reply: tid=0 2024-09-12T02:11:22.865 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.864+0000 7ff29a252740 1 client.18297 shutdown 2024-09-12T02:11:22.865 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.864+0000 7ff29a252740 1 -- 172.21.15.16:0/2497189033 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b46ee42e0 msgr2=0x558b46ef29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:22.866 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.864+0000 7ff29a252740 1 --2- 172.21.15.16:0/2497189033 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b46ee42e0 0x558b46ef29e0 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7ff28c0059f0 tx=0x7ff28c01a400 comp rx=0 tx=0).stop 2024-09-12T02:11:22.866 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.864+0000 7ff29a252740 1 -- 172.21.15.16:0/2497189033 shutdown_connections 2024-09-12T02:11:22.866 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.864+0000 7ff29a252740 1 --2- 172.21.15.16:0/2497189033 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x558b46f64a80 0x558b46f66ea0 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:22.866 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.864+0000 7ff29a252740 1 --2- 172.21.15.16:0/2497189033 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b46ee9370 0x558b46ef39f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:22.866 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.864+0000 7ff29a252740 1 --2- 172.21.15.16:0/2497189033 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b46ee42e0 0x558b46ef29e0 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:22.866 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.864+0000 7ff278ff9640 0 client.18297 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:11:22.866 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.864+0000 7ff29a252740 1 --2- 172.21.15.16:0/2497189033 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558b46ee39a0 0x558b46ef24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:22.866 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.864+0000 7ff29a252740 1 -- 172.21.15.16:0/2497189033 >> 172.21.15.16:0/2497189033 conn(0x558b46edf010 msgr2=0x558b46eedb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:22.866 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.864+0000 7ff29a252740 1 -- 172.21.15.16:0/2497189033 shutdown_connections 2024-09-12T02:11:22.866 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.864+0000 7ff29a252740 1 -- 172.21.15.16:0/2497189033 wait complete. 2024-09-12T02:11:22.866 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.864+0000 7ff29a252740 20 client.18297 trim_cache size 0 max 16384 2024-09-12T02:11:22.867 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-09-12T02:11:22.867 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-09-12T02:11:22.867 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-09-12T02:11:22.867 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-09-12T02:11:22.868 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.865+0000 7ff2973d6640 1 -- 172.21.15.16:0/2982785158 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7ff27c0a2cb0 msgr2=0x7ff27c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:22.868 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.865+0000 7ff2973d6640 1 --2- 172.21.15.16:0/2982785158 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7ff27c0a2cb0 0x7ff27c0a5170 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7ff2880cd100 tx=0x7ff288066ae0 comp rx=0 tx=0).stop 2024-09-12T02:11:22.868 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.865+0000 7ff2973d6640 1 -- 172.21.15.16:0/2982785158 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2901025f0 msgr2=0x7ff29019f300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:22.868 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.865+0000 7ff2973d6640 1 --2- 172.21.15.16:0/2982785158 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2901025f0 0x7ff29019f300 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7ff28800cce0 tx=0x7ff28800a620 comp rx=0 tx=0).stop 2024-09-12T02:11:22.868 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.865+0000 7ff2973d6640 1 -- 172.21.15.16:0/2982785158 shutdown_connections 2024-09-12T02:11:22.868 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.865+0000 7ff2973d6640 1 --2- 172.21.15.16:0/2982785158 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7ff27c0a2cb0 0x7ff27c0a5170 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:22.868 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.865+0000 7ff2973d6640 1 --2- 172.21.15.16:0/2982785158 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2901025f0 0x7ff29019f300 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:22.868 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.865+0000 7ff2973d6640 1 --2- 172.21.15.16:0/2982785158 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff290101c30 0x7ff290198290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:22.868 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.865+0000 7ff2973d6640 1 --2- 172.21.15.16:0/2982785158 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff290100a40 0x7ff290197d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:22.868 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.865+0000 7ff2973d6640 1 -- 172.21.15.16:0/2982785158 >> 172.21.15.16:0/2982785158 conn(0x7ff2900fc1e0 msgr2=0x7ff2900fdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:22.868 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.865+0000 7ff2973d6640 1 -- 172.21.15.16:0/2982785158 shutdown_connections 2024-09-12T02:11:22.868 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:22.865+0000 7ff2973d6640 1 -- 172.21.15.16:0/2982785158 wait complete. 2024-09-12T02:11:23.083 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:11:23.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.567+0000 7f591e50a640 1 -- 172.21.15.16:0/2166577317 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5918102790 msgr2=0x7f59181067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:23.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.567+0000 7f591e50a640 1 --2- 172.21.15.16:0/2166577317 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5918102790 0x7f59181067e0 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f590c009900 tx=0x7f590c02a3f0 comp rx=0 tx=0).stop 2024-09-12T02:11:23.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.567+0000 7f591e50a640 1 -- 172.21.15.16:0/2166577317 shutdown_connections 2024-09-12T02:11:23.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.567+0000 7f591e50a640 1 --2- 172.21.15.16:0/2166577317 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5918106f10 0x7f591810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:23.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.567+0000 7f591e50a640 1 --2- 172.21.15.16:0/2166577317 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5918102790 0x7f59181067e0 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:23.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.567+0000 7f591e50a640 1 --2- 172.21.15.16:0/2166577317 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5918101de0 0x7f59181021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:23.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.567+0000 7f591e50a640 1 -- 172.21.15.16:0/2166577317 >> 172.21.15.16:0/2166577317 conn(0x7f59180fd8c0 msgr2=0x7f59180ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:23.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.567+0000 7f591e50a640 1 -- 172.21.15.16:0/2166577317 shutdown_connections 2024-09-12T02:11:23.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.567+0000 7f591e50a640 1 -- 172.21.15.16:0/2166577317 wait complete. 2024-09-12T02:11:23.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.568+0000 7f591e50a640 1 Processor -- start 2024-09-12T02:11:23.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.568+0000 7f591e50a640 1 -- start start 2024-09-12T02:11:23.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.568+0000 7f591e50a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5918101de0 0x7f59181983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:23.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.568+0000 7f591e50a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5918102790 0x7f59181988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:23.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.568+0000 7f591e50a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5918106f10 0x7f591819cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:23.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.568+0000 7f591e50a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5918111f70 con 0x7f5918102790 2024-09-12T02:11:23.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.568+0000 7f591e50a640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5918111df0 con 0x7f5918106f10 2024-09-12T02:11:23.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.568+0000 7f591e50a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f59181120f0 con 0x7f5918101de0 2024-09-12T02:11:23.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.568+0000 7f59177fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5918102790 0x7f59181988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:23.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.568+0000 7f59177fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5918102790 0x7f59181988f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41464/0 (socket says 172.21.15.16:41464) 2024-09-12T02:11:23.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.568+0000 7f59177fe640 1 -- 172.21.15.16:0/3018115782 learned_addr learned my addr 172.21.15.16:0/3018115782 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:11:23.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.569+0000 7f591caa7640 1 --2- 172.21.15.16:0/3018115782 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5918106f10 0x7f591819cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:23.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.569+0000 7f5917fff640 1 --2- 172.21.15.16:0/3018115782 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5918101de0 0x7f59181983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:23.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.569+0000 7f59177fe640 1 -- 172.21.15.16:0/3018115782 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5918101de0 msgr2=0x7f59181983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:23.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.569+0000 7f59177fe640 1 --2- 172.21.15.16:0/3018115782 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5918101de0 0x7f59181983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:23.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.569+0000 7f59177fe640 1 -- 172.21.15.16:0/3018115782 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5918106f10 msgr2=0x7f591819cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:23.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.569+0000 7f59177fe640 1 --2- 172.21.15.16:0/3018115782 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5918106f10 0x7f591819cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:23.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.569+0000 7f59177fe640 1 -- 172.21.15.16:0/3018115782 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f591819d360 con 0x7f5918102790 2024-09-12T02:11:23.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.569+0000 7f591caa7640 1 --2- 172.21.15.16:0/3018115782 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5918106f10 0x7f591819cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:11:23.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.569+0000 7f5917fff640 1 --2- 172.21.15.16:0/3018115782 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5918101de0 0x7f59181983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:11:23.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.569+0000 7f59177fe640 1 --2- 172.21.15.16:0/3018115782 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5918102790 0x7f59181988f0 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f590c006010 tx=0x7f590c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:23.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.569+0000 7f59157fa640 1 -- 172.21.15.16:0/3018115782 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f590c03f030 con 0x7f5918102790 2024-09-12T02:11:23.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.569+0000 7f59157fa640 1 -- 172.21.15.16:0/3018115782 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f590c003ce0 con 0x7f5918102790 2024-09-12T02:11:23.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.569+0000 7f591e50a640 1 -- 172.21.15.16:0/3018115782 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f591819d5f0 con 0x7f5918102790 2024-09-12T02:11:23.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.569+0000 7f59157fa640 1 -- 172.21.15.16:0/3018115782 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f590c035070 con 0x7f5918102790 2024-09-12T02:11:23.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.569+0000 7f591e50a640 1 -- 172.21.15.16:0/3018115782 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59181a4f30 con 0x7f5918102790 2024-09-12T02:11:23.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.571+0000 7f59157fa640 1 -- 172.21.15.16:0/3018115782 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f590c042030 con 0x7f5918102790 2024-09-12T02:11:23.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.572+0000 7f59208e3740 1 Processor -- start 2024-09-12T02:11:23.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.572+0000 7f59208e3740 1 -- start start 2024-09-12T02:11:23.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.572+0000 7f59208e3740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a64466e750 0x55a64466eb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:23.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.572+0000 7f59208e3740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a64466f070 0x55a64467d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:23.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.572+0000 7f59157fa640 1 --2- 172.21.15.16:0/3018115782 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f59000a2b80 0x7f59000a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:23.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.572+0000 7f59208e3740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a64467de40 0x55a644680230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:23.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.572+0000 7f59208e3740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55a6445d3160 con 0x55a64467de40 2024-09-12T02:11:23.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.572+0000 7f59208e3740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55a64457a720 con 0x55a64466e750 2024-09-12T02:11:23.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.572+0000 7f59208e3740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55a6445cf120 con 0x55a64466f070 2024-09-12T02:11:23.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.572+0000 7f591caa7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a64467de40 0x55a644680230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:23.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.572+0000 7f591caa7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a64467de40 0x55a644680230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41472/0 (socket says 172.21.15.16:41472) 2024-09-12T02:11:23.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.572+0000 7f591caa7640 1 -- 172.21.15.16:0/3461564563 learned_addr learned my addr 172.21.15.16:0/3461564563 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:11:23.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.572+0000 7f59177fe640 1 --2- 172.21.15.16:0/3018115782 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f59000a2b80 0x7f59000a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:23.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.572+0000 7f59157fa640 1 -- 172.21.15.16:0/3018115782 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f590c0ef7b0 con 0x7f5918102790 2024-09-12T02:11:23.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.572+0000 7f591caa7640 1 --2- 172.21.15.16:0/3461564563 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a64466e750 0x55a64466eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:23.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.573+0000 7f5917fff640 1 --2- 172.21.15.16:0/3461564563 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a64466f070 0x55a64467d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:23.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.573+0000 7f591caa7640 1 -- 172.21.15.16:0/3461564563 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a64466f070 msgr2=0x55a64467d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:23.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.573+0000 7f591caa7640 1 --2- 172.21.15.16:0/3461564563 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a64466f070 0x55a64467d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:23.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.573+0000 7f591caa7640 1 -- 172.21.15.16:0/3461564563 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a64466e750 msgr2=0x55a64466eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:23.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.573+0000 7f591caa7640 1 --2- 172.21.15.16:0/3461564563 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a64466e750 0x55a64466eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:23.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.573+0000 7f591caa7640 1 -- 172.21.15.16:0/3461564563 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a644344ba0 con 0x55a64467de40 2024-09-12T02:11:23.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.573+0000 7f59177fe640 1 --2- 172.21.15.16:0/3018115782 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f59000a2b80 0x7f59000a5040 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f590c0b8aa0 tx=0x7f590c086d60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:23.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.573+0000 7f591caa7640 1 --2- 172.21.15.16:0/3461564563 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a64467de40 0x55a644680230 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f5910012a10 tx=0x7f5910009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ad6c9b52b4e8c26 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:23.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.573+0000 7f58fe7fc640 1 -- 172.21.15.16:0/3461564563 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5910004b20 con 0x55a64467de40 2024-09-12T02:11:23.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.573+0000 7f58fe7fc640 1 -- 172.21.15.16:0/3461564563 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5910010d10 con 0x55a64467de40 2024-09-12T02:11:23.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.573+0000 7f58fe7fc640 1 -- 172.21.15.16:0/3461564563 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5910005580 con 0x55a64467de40 2024-09-12T02:11:23.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.574+0000 7f59208e3740 1 -- 172.21.15.16:0/3461564563 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a64467de40 msgr2=0x55a644680230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:23.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.574+0000 7f59208e3740 1 --2- 172.21.15.16:0/3461564563 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a64467de40 0x55a644680230 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f5910012a10 tx=0x7f5910009750 comp rx=0 tx=0).stop 2024-09-12T02:11:23.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.574+0000 7f59208e3740 1 -- 172.21.15.16:0/3461564563 shutdown_connections 2024-09-12T02:11:23.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.574+0000 7f59208e3740 1 --2- 172.21.15.16:0/3461564563 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a64467de40 0x55a644680230 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:23.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.574+0000 7f59208e3740 1 --2- 172.21.15.16:0/3461564563 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a64466f070 0x55a64467d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:23.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.574+0000 7f59208e3740 1 --2- 172.21.15.16:0/3461564563 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a64466e750 0x55a64466eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:23.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.574+0000 7f59208e3740 1 -- 172.21.15.16:0/3461564563 >> 172.21.15.16:0/3461564563 conn(0x55a64466ade0 msgr2=0x55a64466b1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:23.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.574+0000 7f59208e3740 1 -- 172.21.15.16:0/3461564563 shutdown_connections 2024-09-12T02:11:23.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.574+0000 7f59208e3740 1 -- 172.21.15.16:0/3461564563 wait complete. 2024-09-12T02:11:23.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.574+0000 7f59208e3740 1 -- start start 2024-09-12T02:11:23.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.574+0000 7f59208e3740 1 Processor -- start 2024-09-12T02:11:23.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.575+0000 7f59208e3740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a64466e750 0x55a64467ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:23.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.575+0000 7f59208e3740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a64466f070 0x55a64467b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:23.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.575+0000 7f59208e3740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a64467de40 0x55a64467b890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:23.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.575+0000 7f59208e3740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55a644680a70 con 0x55a64466e750 2024-09-12T02:11:23.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.575+0000 7f59208e3740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55a64455b900 con 0x55a64466f070 2024-09-12T02:11:23.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.575+0000 7f59208e3740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55a644680bf0 con 0x55a64467de40 2024-09-12T02:11:23.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.575+0000 7f59208e3740 10 client.0 _ll_register_callbacks cb 0x55a644559780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:11:23.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.575+0000 7f591caa7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a64466e750 0x55a64467ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:23.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.575+0000 7f591caa7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a64466e750 0x55a64467ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41486/0 (socket says 172.21.15.16:41486) 2024-09-12T02:11:23.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.575+0000 7f591caa7640 1 -- 172.21.15.16:0/1364268751 learned_addr learned my addr 172.21.15.16:0/1364268751 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:11:23.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.575+0000 7f5917fff640 1 --2- 172.21.15.16:0/1364268751 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a64466f070 0x55a64467b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:23.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.575+0000 7f591caa7640 1 --2- 172.21.15.16:0/1364268751 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a64467de40 0x55a64467b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:23.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.575+0000 7f591caa7640 1 -- 172.21.15.16:0/1364268751 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a64467de40 msgr2=0x55a64467b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:23.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.575+0000 7f591caa7640 1 --2- 172.21.15.16:0/1364268751 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a64467de40 0x55a64467b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:23.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.575+0000 7f591caa7640 1 -- 172.21.15.16:0/1364268751 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a64466f070 msgr2=0x55a64467b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:23.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.575+0000 7f591caa7640 1 --2- 172.21.15.16:0/1364268751 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a64466f070 0x55a64467b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:23.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.575+0000 7f591caa7640 1 -- 172.21.15.16:0/1364268751 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a64459bc50 con 0x55a64466e750 2024-09-12T02:11:23.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.575+0000 7f591caa7640 1 --2- 172.21.15.16:0/1364268751 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a64466e750 0x55a64467ae10 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f5910004fd0 tx=0x7f5910005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7b6fe3adf00c9c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:23.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.575+0000 7f58f77fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:11:23.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.575+0000 7f58f77fe640 1 -- 172.21.15.16:0/1364268751 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5910004d10 con 0x55a64466e750 2024-09-12T02:11:23.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.575+0000 7f58f77fe640 1 -- 172.21.15.16:0/1364268751 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f5910010830 con 0x55a64466e750 2024-09-12T02:11:23.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.575+0000 7f59208e3740 1 -- 172.21.15.16:0/1364268751 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5910010d10 con 0x55a64466e750 2024-09-12T02:11:23.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.576+0000 7f58f77fe640 1 -- 172.21.15.16:0/1364268751 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5910010a30 con 0x55a64466e750 2024-09-12T02:11:23.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.576+0000 7f59208e3740 10 client.18309 fetch_fsmap learned FSMap version 25 2024-09-12T02:11:23.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.576+0000 7f59208e3740 1 -- 172.21.15.16:0/1364268751 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a64468c830 con 0x55a64466e750 2024-09-12T02:11:23.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.576+0000 7f58f77fe640 1 -- 172.21.15.16:0/1364268751 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5910010d10 con 0x55a64466e750 2024-09-12T02:11:23.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.576+0000 7f58f77fe640 1 -- 172.21.15.16:0/1364268751 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f5910030050 con 0x55a64466e750 2024-09-12T02:11:23.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.576+0000 7f59208e3740 10 client.18309 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:11:23.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.576+0000 7f59208e3740 10 client.18309 resolve_mds: validated gid 24482 aka daemon mds.b 2024-09-12T02:11:23.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.576+0000 7f59208e3740 20 client.18309 populate_metadata read hostname 'smithi016' 2024-09-12T02:11:23.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.576+0000 7f59208e3740 1 --2- 172.21.15.16:0/1364268751 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55a64468db80 0x55a64468ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:23.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.576+0000 7f59208e3740 4 client.18309 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:11:23.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.576+0000 7f59208e3740 1 -- 172.21.15.16:0/1364268751 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a6445e0890 con 0x55a64468db80 2024-09-12T02:11:23.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.576+0000 7f5917fff640 1 --2- 172.21.15.16:0/1364268751 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55a64468db80 0x55a64468ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:23.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.577+0000 7f5917fff640 1 --2- 172.21.15.16:0/1364268751 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55a64468db80 0x55a64468ff70 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f1d86c20a7e7ad1a server_cookie=5ce90b5a9c2e3f35 in_seq=0 out_seq=0 2024-09-12T02:11:23.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.577+0000 7f58f77fe640 10 client.18309 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:11:23.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.578+0000 7f58f77fe640 1 -- 172.21.15.16:0/1364268751 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a6445e0890 con 0x55a64468db80 2024-09-12T02:11:23.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.578+0000 7f58f77fe640 10 client.18309 handle_command_reply: tid=0 2024-09-12T02:11:23.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.578+0000 7f59208e3740 1 client.18309 shutdown 2024-09-12T02:11:23.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.578+0000 7f59208e3740 1 -- 172.21.15.16:0/1364268751 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a64466e750 msgr2=0x55a64467ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:23.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.578+0000 7f59208e3740 1 --2- 172.21.15.16:0/1364268751 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a64466e750 0x55a64467ae10 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f5910004fd0 tx=0x7f5910005020 comp rx=0 tx=0).stop 2024-09-12T02:11:23.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.578+0000 7f59208e3740 1 -- 172.21.15.16:0/1364268751 shutdown_connections 2024-09-12T02:11:23.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.578+0000 7f59208e3740 1 --2- 172.21.15.16:0/1364268751 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55a64468db80 0x55a64468ff70 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:23.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.578+0000 7f59208e3740 1 --2- 172.21.15.16:0/1364268751 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a64467de40 0x55a64467b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:23.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.578+0000 7f59208e3740 1 --2- 172.21.15.16:0/1364268751 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a64466f070 0x55a64467b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:23.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.578+0000 7f58f77fe640 0 client.18309 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:11:23.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.578+0000 7f59208e3740 1 --2- 172.21.15.16:0/1364268751 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a64466e750 0x55a64467ae10 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:23.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.578+0000 7f59208e3740 1 -- 172.21.15.16:0/1364268751 >> 172.21.15.16:0/1364268751 conn(0x55a64466ade0 msgr2=0x55a644670b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:23.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.578+0000 7f59208e3740 1 -- 172.21.15.16:0/1364268751 shutdown_connections 2024-09-12T02:11:23.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.578+0000 7f59208e3740 1 -- 172.21.15.16:0/1364268751 wait complete. 2024-09-12T02:11:23.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.578+0000 7f59208e3740 20 client.18309 trim_cache size 0 max 16384 2024-09-12T02:11:23.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.596+0000 7f59208e3740 1 Processor -- start 2024-09-12T02:11:23.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.596+0000 7f59208e3740 1 -- start start 2024-09-12T02:11:23.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.596+0000 7f59208e3740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a64466e750 0x55a644695630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:23.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.596+0000 7f59208e3740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a64466f070 0x55a644695b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:23.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.596+0000 7f59208e3740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a64467de40 0x55a644678090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:23.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.596+0000 7f59208e3740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f59181adac0 con 0x55a64466f070 2024-09-12T02:11:23.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.596+0000 7f59208e3740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f59181ab390 con 0x55a64467de40 2024-09-12T02:11:23.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.596+0000 7f59208e3740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f59181a9200 con 0x55a64466e750 2024-09-12T02:11:23.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.596+0000 7f5917fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a64466f070 0x55a644695b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:23.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.596+0000 7f5917fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a64466f070 0x55a644695b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41490/0 (socket says 172.21.15.16:41490) 2024-09-12T02:11:23.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.596+0000 7f5917fff640 1 -- 172.21.15.16:0/1296902609 learned_addr learned my addr 172.21.15.16:0/1296902609 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:11:23.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.596+0000 7f591caa7640 1 --2- 172.21.15.16:0/1296902609 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a64466e750 0x55a644695630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:23.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.596+0000 7f591caa7640 1 --2- 172.21.15.16:0/1296902609 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a64467de40 0x55a644678090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:23.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.596+0000 7f5917fff640 1 -- 172.21.15.16:0/1296902609 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a64466e750 msgr2=0x55a644695630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:23.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.596+0000 7f5917fff640 1 --2- 172.21.15.16:0/1296902609 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a64466e750 0x55a644695630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:23.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.596+0000 7f5917fff640 1 -- 172.21.15.16:0/1296902609 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a64467de40 msgr2=0x55a644678090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:23.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.596+0000 7f5917fff640 1 --2- 172.21.15.16:0/1296902609 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a64467de40 0x55a644678090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:23.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.596+0000 7f5917fff640 1 -- 172.21.15.16:0/1296902609 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a64466bf70 con 0x55a64466f070 2024-09-12T02:11:23.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.596+0000 7f591caa7640 1 --2- 172.21.15.16:0/1296902609 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a64467de40 0x55a644678090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:11:23.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.597+0000 7f5917fff640 1 --2- 172.21.15.16:0/1296902609 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a64466f070 0x55a644695b70 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x55a64467a6a0 tx=0x7f5908010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=630524d941595f8a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:23.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.597+0000 7f58fdffb640 1 -- 172.21.15.16:0/1296902609 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5908005020 con 0x55a64466f070 2024-09-12T02:11:23.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.597+0000 7f58fdffb640 1 -- 172.21.15.16:0/1296902609 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f59080047a0 con 0x55a64466f070 2024-09-12T02:11:23.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.597+0000 7f58fdffb640 1 -- 172.21.15.16:0/1296902609 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59080246d0 con 0x55a64466f070 2024-09-12T02:11:23.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.598+0000 7f59208e3740 1 -- 172.21.15.16:0/1296902609 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a64466f070 msgr2=0x55a644695b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:23.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.598+0000 7f59208e3740 1 --2- 172.21.15.16:0/1296902609 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a64466f070 0x55a644695b70 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x55a64467a6a0 tx=0x7f5908010800 comp rx=0 tx=0).stop 2024-09-12T02:11:23.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.598+0000 7f59208e3740 1 -- 172.21.15.16:0/1296902609 shutdown_connections 2024-09-12T02:11:23.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.598+0000 7f59208e3740 1 --2- 172.21.15.16:0/1296902609 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a64467de40 0x55a644678090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:23.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.598+0000 7f59208e3740 1 --2- 172.21.15.16:0/1296902609 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a64466f070 0x55a644695b70 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:23.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.598+0000 7f59208e3740 1 --2- 172.21.15.16:0/1296902609 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a64466e750 0x55a644695630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:23.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.598+0000 7f59208e3740 1 -- 172.21.15.16:0/1296902609 >> 172.21.15.16:0/1296902609 conn(0x55a64466ade0 msgr2=0x55a64467ce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:23.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.598+0000 7f59208e3740 1 -- 172.21.15.16:0/1296902609 shutdown_connections 2024-09-12T02:11:23.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.598+0000 7f59208e3740 1 -- 172.21.15.16:0/1296902609 wait complete. 2024-09-12T02:11:23.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.598+0000 7f59208e3740 1 -- start start 2024-09-12T02:11:23.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.598+0000 7f59208e3740 1 Processor -- start 2024-09-12T02:11:23.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.599+0000 7f59208e3740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a64466e750 0x55a644685e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:23.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.599+0000 7f59208e3740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a64466f070 0x55a6446863c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:23.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.599+0000 7f59208e3740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a64467de40 0x55a644682e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:23.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.599+0000 7f59208e3740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f59181a9850 con 0x55a64466f070 2024-09-12T02:11:23.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.599+0000 7f59208e3740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f59181aacc0 con 0x55a64466e750 2024-09-12T02:11:23.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.599+0000 7f59208e3740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f59181a9f20 con 0x55a64467de40 2024-09-12T02:11:23.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.599+0000 7f59208e3740 10 client.0 _ll_register_callbacks cb 0x7f59100124d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:11:23.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.599+0000 7f5917fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a64466f070 0x55a6446863c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:23.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.599+0000 7f5917fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a64466f070 0x55a6446863c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41498/0 (socket says 172.21.15.16:41498) 2024-09-12T02:11:23.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.599+0000 7f591caa7640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a64466e750 0x55a644685e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:23.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.599+0000 7f5917fff640 1 -- 172.21.15.16:0/3417211197 learned_addr learned my addr 172.21.15.16:0/3417211197 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:11:23.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.599+0000 7f591caa7640 1 --2- 172.21.15.16:0/3417211197 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a64467de40 0x55a644682e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:23.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.599+0000 7f5917fff640 1 -- 172.21.15.16:0/3417211197 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a64467de40 msgr2=0x55a644682e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:23.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.599+0000 7f5917fff640 1 --2- 172.21.15.16:0/3417211197 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a64467de40 0x55a644682e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:23.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.599+0000 7f5917fff640 1 -- 172.21.15.16:0/3417211197 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a64466e750 msgr2=0x55a644685e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:23.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.599+0000 7f5917fff640 1 --2- 172.21.15.16:0/3417211197 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a64466e750 0x55a644685e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:23.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.599+0000 7f5917fff640 1 -- 172.21.15.16:0/3417211197 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a644683390 con 0x55a64466f070 2024-09-12T02:11:23.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.600+0000 7f5917fff640 1 --2- 172.21.15.16:0/3417211197 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a64466f070 0x55a6446863c0 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f590800dbc0 tx=0x7f5908005250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ecb2e325f6099ee server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:23.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.600+0000 7f58f77fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:11:23.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.600+0000 7f58f77fe640 1 -- 172.21.15.16:0/3417211197 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5908032060 con 0x55a64466f070 2024-09-12T02:11:23.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.600+0000 7f58f77fe640 1 -- 172.21.15.16:0/3417211197 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f5908020040 con 0x55a64466f070 2024-09-12T02:11:23.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.600+0000 7f59208e3740 1 -- 172.21.15.16:0/3417211197 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f59080047a0 con 0x55a64466f070 2024-09-12T02:11:23.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.601+0000 7f58f77fe640 1 -- 172.21.15.16:0/3417211197 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5908004950 con 0x55a64466f070 2024-09-12T02:11:23.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.601+0000 7f59208e3740 10 client.18315 fetch_fsmap learned FSMap version 25 2024-09-12T02:11:23.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.601+0000 7f59208e3740 1 -- 172.21.15.16:0/3417211197 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a6446f8ca0 con 0x55a64466f070 2024-09-12T02:11:23.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.601+0000 7f58f77fe640 1 -- 172.21.15.16:0/3417211197 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f59080047a0 con 0x55a64466f070 2024-09-12T02:11:23.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.601+0000 7f58f77fe640 1 -- 172.21.15.16:0/3417211197 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f5908024cc0 con 0x55a64466f070 2024-09-12T02:11:23.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.601+0000 7f59208e3740 10 client.18315 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:11:23.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.601+0000 7f59208e3740 10 client.18315 resolve_mds: validated gid 24482 aka daemon mds.b 2024-09-12T02:11:23.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.601+0000 7f59208e3740 20 client.18315 populate_metadata read hostname 'smithi016' 2024-09-12T02:11:23.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.601+0000 7f59208e3740 1 --2- 172.21.15.16:0/3417211197 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55a6446f93a0 0x55a6446fb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:23.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.601+0000 7f59208e3740 4 client.18315 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.55, "await_for": 45.0}] 2024-09-12T02:11:23.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.601+0000 7f59208e3740 1 -- 172.21.15.16:0/3417211197 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.55, "await_for": 45.0}) -- 0x55a6445dabe0 con 0x55a6446f93a0 2024-09-12T02:11:23.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.602+0000 7f591caa7640 1 --2- 172.21.15.16:0/3417211197 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55a6446f93a0 0x55a6446fb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:23.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.602+0000 7f591caa7640 1 --2- 172.21.15.16:0/3417211197 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55a6446f93a0 0x55a6446fb7c0 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20d77f96c49589fd server_cookie=9adc38fb74d39597 in_seq=0 out_seq=0 2024-09-12T02:11:23.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.602+0000 7f58f77fe640 10 client.18315 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:11:23.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.616+0000 7f58f77fe640 1 -- 172.21.15.16:0/3417211197 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55a6445dabe0 con 0x55a6446f93a0 2024-09-12T02:11:23.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.616+0000 7f58f77fe640 10 client.18315 handle_command_reply: tid=0 2024-09-12T02:11:23.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.616+0000 7f59208e3740 1 client.18315 shutdown 2024-09-12T02:11:23.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.616+0000 7f59208e3740 1 -- 172.21.15.16:0/3417211197 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a64466f070 msgr2=0x55a6446863c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:23.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.616+0000 7f59208e3740 1 --2- 172.21.15.16:0/3417211197 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a64466f070 0x55a6446863c0 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f590800dbc0 tx=0x7f5908005250 comp rx=0 tx=0).stop 2024-09-12T02:11:23.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.616+0000 7f59208e3740 1 -- 172.21.15.16:0/3417211197 shutdown_connections 2024-09-12T02:11:23.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.616+0000 7f59208e3740 1 --2- 172.21.15.16:0/3417211197 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55a6446f93a0 0x55a6446fb7c0 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:23.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.616+0000 7f59208e3740 1 --2- 172.21.15.16:0/3417211197 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a64467de40 0x55a644682e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:23.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.616+0000 7f59208e3740 1 --2- 172.21.15.16:0/3417211197 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a64466f070 0x55a6446863c0 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:23.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.616+0000 7f58f77fe640 0 client.18315 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:11:23.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.616+0000 7f59208e3740 1 --2- 172.21.15.16:0/3417211197 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a64466e750 0x55a644685e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:23.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.616+0000 7f59208e3740 1 -- 172.21.15.16:0/3417211197 >> 172.21.15.16:0/3417211197 conn(0x55a64466ade0 msgr2=0x55a644670280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:23.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.616+0000 7f59208e3740 1 -- 172.21.15.16:0/3417211197 shutdown_connections 2024-09-12T02:11:23.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.616+0000 7f59208e3740 1 -- 172.21.15.16:0/3417211197 wait complete. 2024-09-12T02:11:23.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.616+0000 7f59208e3740 20 client.18315 trim_cache size 0 max 16384 2024-09-12T02:11:23.619 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T02:11:23.619 INFO:teuthology.orchestra.run.smithi016.stdout: "epoch": 25, 2024-09-12T02:11:23.619 INFO:teuthology.orchestra.run.smithi016.stdout: "leader": 24482, 2024-09-12T02:11:23.619 INFO:teuthology.orchestra.run.smithi016.stdout: "set_version": 38, 2024-09-12T02:11:23.620 INFO:teuthology.orchestra.run.smithi016.stdout: "sets": { 2024-09-12T02:11:23.620 INFO:teuthology.orchestra.run.smithi016.stdout: "998c7ef0": { 2024-09-12T02:11:23.620 INFO:teuthology.orchestra.run.smithi016.stdout: "version": 38, 2024-09-12T02:11:23.620 INFO:teuthology.orchestra.run.smithi016.stdout: "age_ref": 0.0, 2024-09-12T02:11:23.620 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-12T02:11:23.620 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "QUIESCED", 2024-09-12T02:11:23.620 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-12T02:11:23.620 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:11:23.620 INFO:teuthology.orchestra.run.smithi016.stdout: "timeout": 90.0, 2024-09-12T02:11:23.620 INFO:teuthology.orchestra.run.smithi016.stdout: "expiration": 142.6, 2024-09-12T02:11:23.620 INFO:teuthology.orchestra.run.smithi016.stdout: "members": { 2024-09-12T02:11:23.620 INFO:teuthology.orchestra.run.smithi016.stdout: "file:/": { 2024-09-12T02:11:23.621 INFO:teuthology.orchestra.run.smithi016.stdout: "excluded": false, 2024-09-12T02:11:23.621 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-12T02:11:23.621 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "QUIESCED", 2024-09-12T02:11:23.621 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-12T02:11:23.621 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:11:23.621 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:11:23.621 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:11:23.621 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:11:23.621 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:11:23.621 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T02:11:23.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.617+0000 7f591e50a640 1 -- 172.21.15.16:0/3018115782 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f59000a2b80 msgr2=0x7f59000a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:23.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.617+0000 7f591e50a640 1 --2- 172.21.15.16:0/3018115782 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f59000a2b80 0x7f59000a5040 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f590c0b8aa0 tx=0x7f590c086d60 comp rx=0 tx=0).stop 2024-09-12T02:11:23.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.617+0000 7f591e50a640 1 -- 172.21.15.16:0/3018115782 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5918102790 msgr2=0x7f59181988f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:23.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.617+0000 7f591e50a640 1 --2- 172.21.15.16:0/3018115782 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5918102790 0x7f59181988f0 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f590c006010 tx=0x7f590c0042f0 comp rx=0 tx=0).stop 2024-09-12T02:11:23.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.617+0000 7f591e50a640 1 -- 172.21.15.16:0/3018115782 shutdown_connections 2024-09-12T02:11:23.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.617+0000 7f591e50a640 1 --2- 172.21.15.16:0/3018115782 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f59000a2b80 0x7f59000a5040 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:23.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.617+0000 7f591e50a640 1 --2- 172.21.15.16:0/3018115782 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5918106f10 0x7f591819cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:23.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.617+0000 7f591e50a640 1 --2- 172.21.15.16:0/3018115782 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5918102790 0x7f59181988f0 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:23.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.617+0000 7f591e50a640 1 --2- 172.21.15.16:0/3018115782 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5918101de0 0x7f59181983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:23.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.617+0000 7f591e50a640 1 -- 172.21.15.16:0/3018115782 >> 172.21.15.16:0/3018115782 conn(0x7f59180fd8c0 msgr2=0x7f59180feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:23.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.617+0000 7f591e50a640 1 -- 172.21.15.16:0/3018115782 shutdown_connections 2024-09-12T02:11:23.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:23.617+0000 7f591e50a640 1 -- 172.21.15.16:0/3018115782 wait complete. 2024-09-12T02:11:24.363 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '998c7ef0', quiesce took 3.3 seconds. Will release after: 19.25 2024-09-12T02:11:24.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:11:24 smithi184 ceph-mon[35593]: pgmap v619: 97 pgs: 97 active+clean; 1.1 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 14 op/s 2024-09-12T02:11:24.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:11:24 smithi033 ceph-mon[35332]: pgmap v619: 97 pgs: 97 active+clean; 1.1 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 14 op/s 2024-09-12T02:11:24.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:11:24 smithi016 ceph-mon[30947]: pgmap v619: 97 pgs: 97 active+clean; 1.1 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 14 op/s 2024-09-12T02:11:26.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:11:26 smithi184 ceph-mon[35593]: pgmap v620: 97 pgs: 97 active+clean; 1.1 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-09-12T02:11:26.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:11:26 smithi033 ceph-mon[35332]: pgmap v620: 97 pgs: 97 active+clean; 1.1 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-09-12T02:11:26.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:11:26 smithi016 ceph-mon[30947]: pgmap v620: 97 pgs: 97 active+clean; 1.1 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-09-12T02:11:28.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:11:28 smithi184 ceph-mon[35593]: pgmap v621: 97 pgs: 97 active+clean; 1.1 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 17 KiB/s wr, 15 op/s 2024-09-12T02:11:28.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:11:28 smithi033 ceph-mon[35332]: pgmap v621: 97 pgs: 97 active+clean; 1.1 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 17 KiB/s wr, 15 op/s 2024-09-12T02:11:28.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:11:28 smithi016 ceph-mon[30947]: pgmap v621: 97 pgs: 97 active+clean; 1.1 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 17 KiB/s wr, 15 op/s 2024-09-12T02:11:30.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:11:30 smithi184 ceph-mon[35593]: pgmap v622: 97 pgs: 97 active+clean; 1.1 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-09-12T02:11:30.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:11:30 smithi033 ceph-mon[35332]: pgmap v622: 97 pgs: 97 active+clean; 1.1 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-09-12T02:11:30.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:11:30 smithi016 ceph-mon[30947]: pgmap v622: 97 pgs: 97 active+clean; 1.1 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-09-12T02:11:32.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:11:32 smithi184 ceph-mon[35593]: pgmap v623: 97 pgs: 97 active+clean; 1.1 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 12 op/s 2024-09-12T02:11:32.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:11:32 smithi033 ceph-mon[35332]: pgmap v623: 97 pgs: 97 active+clean; 1.1 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 12 op/s 2024-09-12T02:11:32.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:11:32 smithi016 ceph-mon[30947]: pgmap v623: 97 pgs: 97 active+clean; 1.1 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 12 op/s 2024-09-12T02:11:34.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:11:34 smithi184 ceph-mon[35593]: pgmap v624: 97 pgs: 97 active+clean; 1.1 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 KiB/s wr, 12 op/s 2024-09-12T02:11:34.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:11:34 smithi033 ceph-mon[35332]: pgmap v624: 97 pgs: 97 active+clean; 1.1 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 KiB/s wr, 12 op/s 2024-09-12T02:11:34.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:11:34 smithi016 ceph-mon[30947]: pgmap v624: 97 pgs: 97 active+clean; 1.1 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 KiB/s wr, 12 op/s 2024-09-12T02:11:35.658 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T02:11:35.932 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:11:35.966 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.966+0000 7f45ab6dd640 1 -- 172.21.15.16:0/3673321846 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45a41025f0 msgr2=0x7f45a4108e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:35.966 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.966+0000 7f45ab6dd640 1 --2- 172.21.15.16:0/3673321846 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45a41025f0 0x7f45a4108e80 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f4594009900 tx=0x7f459402a410 comp rx=0 tx=0).stop 2024-09-12T02:11:35.966 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.966+0000 7f45ab6dd640 1 -- 172.21.15.16:0/3673321846 shutdown_connections 2024-09-12T02:11:35.966 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.966+0000 7f45ab6dd640 1 --2- 172.21.15.16:0/3673321846 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45a41025f0 0x7f45a4108e80 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:35.966 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.966+0000 7f45ab6dd640 1 --2- 172.21.15.16:0/3673321846 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f45a4101c30 0x7f45a41020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:35.966 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.966+0000 7f45ab6dd640 1 --2- 172.21.15.16:0/3673321846 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f45a4100a40 0x7f45a4100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:35.966 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.966+0000 7f45ab6dd640 1 -- 172.21.15.16:0/3673321846 >> 172.21.15.16:0/3673321846 conn(0x7f45a40fc1e0 msgr2=0x7f45a40fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:35.966 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.966+0000 7f45ab6dd640 1 -- 172.21.15.16:0/3673321846 shutdown_connections 2024-09-12T02:11:35.966 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.966+0000 7f45ab6dd640 1 -- 172.21.15.16:0/3673321846 wait complete. 2024-09-12T02:11:35.968 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.966+0000 7f45ab6dd640 1 Processor -- start 2024-09-12T02:11:35.968 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.966+0000 7f45ab6dd640 1 -- start start 2024-09-12T02:11:35.968 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.966+0000 7f45ab6dd640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f45a4100a40 0x7f45a4197c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:35.969 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.967+0000 7f45ab6dd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f45a4101c30 0x7f45a4198170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:35.969 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.967+0000 7f45ab6dd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45a41025f0 0x7f45a419f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:35.969 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.967+0000 7f45ab6dd640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f45a410c6a0 con 0x7f45a41025f0 2024-09-12T02:11:35.969 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.967+0000 7f45ab6dd640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f45a410c520 con 0x7f45a4101c30 2024-09-12T02:11:35.969 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.967+0000 7f45ab6dd640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f45a410c820 con 0x7f45a4100a40 2024-09-12T02:11:35.969 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.967+0000 7f45a9c7a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45a41025f0 0x7f45a419f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:35.969 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.967+0000 7f45a9c7a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45a41025f0 0x7f45a419f1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52098/0 (socket says 172.21.15.16:52098) 2024-09-12T02:11:35.969 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.967+0000 7f45a9c7a640 1 -- 172.21.15.16:0/3042621418 learned_addr learned my addr 172.21.15.16:0/3042621418 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:11:35.969 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.967+0000 7f45a9479640 1 --2- 172.21.15.16:0/3042621418 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f45a4100a40 0x7f45a4197c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:35.969 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.967+0000 7f45a8c78640 1 --2- 172.21.15.16:0/3042621418 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f45a4101c30 0x7f45a4198170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:35.969 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.967+0000 7f45a9c7a640 1 -- 172.21.15.16:0/3042621418 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f45a4100a40 msgr2=0x7f45a4197c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:35.969 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.967+0000 7f45a9c7a640 1 --2- 172.21.15.16:0/3042621418 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f45a4100a40 0x7f45a4197c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:35.969 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.967+0000 7f45a9c7a640 1 -- 172.21.15.16:0/3042621418 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f45a4101c30 msgr2=0x7f45a4198170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:35.969 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.967+0000 7f45a9c7a640 1 --2- 172.21.15.16:0/3042621418 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f45a4101c30 0x7f45a4198170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:35.970 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.967+0000 7f45a9c7a640 1 -- 172.21.15.16:0/3042621418 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45a419f840 con 0x7f45a41025f0 2024-09-12T02:11:35.970 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.967+0000 7f45a9c7a640 1 --2- 172.21.15.16:0/3042621418 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45a41025f0 0x7f45a419f1a0 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f45940314e0 tx=0x7f45940043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:35.970 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.967+0000 7f459a7fc640 1 -- 172.21.15.16:0/3042621418 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f459403f030 con 0x7f45a41025f0 2024-09-12T02:11:35.970 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.967+0000 7f459a7fc640 1 -- 172.21.15.16:0/3042621418 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4594003ce0 con 0x7f45a41025f0 2024-09-12T02:11:35.970 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.967+0000 7f45ab6dd640 1 -- 172.21.15.16:0/3042621418 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45a419fad0 con 0x7f45a41025f0 2024-09-12T02:11:35.970 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.967+0000 7f459a7fc640 1 -- 172.21.15.16:0/3042621418 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4594035070 con 0x7f45a41025f0 2024-09-12T02:11:35.970 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.968+0000 7f45ab6dd640 1 -- 172.21.15.16:0/3042621418 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45a419ffc0 con 0x7f45a41025f0 2024-09-12T02:11:35.972 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.970+0000 7f459a7fc640 1 -- 172.21.15.16:0/3042621418 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f4594042030 con 0x7f45a41025f0 2024-09-12T02:11:35.972 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.970+0000 7f459a7fc640 1 --2- 172.21.15.16:0/3042621418 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f45900a2c30 0x7f45900a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:35.972 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.971+0000 7f459a7fc640 1 -- 172.21.15.16:0/3042621418 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f45940ef880 con 0x7f45a41025f0 2024-09-12T02:11:35.972 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.971+0000 7f45a9479640 1 --2- 172.21.15.16:0/3042621418 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f45900a2c30 0x7f45900a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:35.972 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.971+0000 7f45ae47a740 1 Processor -- start 2024-09-12T02:11:35.972 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.971+0000 7f45ae47a740 1 -- start start 2024-09-12T02:11:35.973 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.971+0000 7f45a9479640 1 --2- 172.21.15.16:0/3042621418 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f45900a2c30 0x7f45900a50f0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f45a4199100 tx=0x7f45a0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:35.973 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.971+0000 7f45ae47a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555f321629a0 0x555f32162da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:35.973 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.971+0000 7f45ae47a740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555f321632e0 0x555f32167da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:35.973 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.971+0000 7f45ae47a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555f32168370 0x555f3216a830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:35.973 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.971+0000 7f45ae47a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x555f320c62f0 con 0x555f321629a0 2024-09-12T02:11:35.973 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.971+0000 7f45ae47a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x555f3206e270 con 0x555f32168370 2024-09-12T02:11:35.973 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.971+0000 7f45ae47a740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x555f320c22b0 con 0x555f321632e0 2024-09-12T02:11:35.973 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.971+0000 7f45a9479640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555f321629a0 0x555f32162da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:35.973 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.971+0000 7f45a9479640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555f321629a0 0x555f32162da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52106/0 (socket says 172.21.15.16:52106) 2024-09-12T02:11:35.973 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.971+0000 7f45a9479640 1 -- 172.21.15.16:0/3236051663 learned_addr learned my addr 172.21.15.16:0/3236051663 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:11:35.973 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.972+0000 7f45a8c78640 1 --2- 172.21.15.16:0/3236051663 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555f321632e0 0x555f32167da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:35.973 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.972+0000 7f45a9c7a640 1 --2- 172.21.15.16:0/3236051663 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555f32168370 0x555f3216a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:35.973 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.972+0000 7f45a9479640 1 -- 172.21.15.16:0/3236051663 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555f321632e0 msgr2=0x555f32167da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:35.974 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.972+0000 7f45a9479640 1 --2- 172.21.15.16:0/3236051663 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555f321632e0 0x555f32167da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:35.974 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.972+0000 7f45a9479640 1 -- 172.21.15.16:0/3236051663 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555f32168370 msgr2=0x555f3216a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:35.974 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.972+0000 7f45a9479640 1 --2- 172.21.15.16:0/3236051663 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555f32168370 0x555f3216a830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:35.974 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.972+0000 7f45a9479640 1 -- 172.21.15.16:0/3236051663 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f3206d340 con 0x555f321629a0 2024-09-12T02:11:35.974 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.972+0000 7f45a9479640 1 --2- 172.21.15.16:0/3236051663 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555f321629a0 0x555f32162da0 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f45a000ff60 tx=0x7f45a0012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3d882c5a7849970 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:35.974 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.972+0000 7f458f7fe640 1 -- 172.21.15.16:0/3236051663 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45a0016070 con 0x555f321629a0 2024-09-12T02:11:35.974 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.972+0000 7f458f7fe640 1 -- 172.21.15.16:0/3236051663 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45a001b900 con 0x555f321629a0 2024-09-12T02:11:35.974 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.972+0000 7f458f7fe640 1 -- 172.21.15.16:0/3236051663 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45a001c9e0 con 0x555f321629a0 2024-09-12T02:11:35.978 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.973+0000 7f45ae47a740 1 -- 172.21.15.16:0/3236051663 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555f321629a0 msgr2=0x555f32162da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:35.978 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.973+0000 7f45ae47a740 1 --2- 172.21.15.16:0/3236051663 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555f321629a0 0x555f32162da0 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f45a000ff60 tx=0x7f45a0012000 comp rx=0 tx=0).stop 2024-09-12T02:11:35.978 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.973+0000 7f45ae47a740 1 -- 172.21.15.16:0/3236051663 shutdown_connections 2024-09-12T02:11:35.979 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.973+0000 7f45ae47a740 1 --2- 172.21.15.16:0/3236051663 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555f32168370 0x555f3216a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:35.979 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.973+0000 7f45ae47a740 1 --2- 172.21.15.16:0/3236051663 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555f321632e0 0x555f32167da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:35.979 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.973+0000 7f45ae47a740 1 --2- 172.21.15.16:0/3236051663 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555f321629a0 0x555f32162da0 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:35.979 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.973+0000 7f45ae47a740 1 -- 172.21.15.16:0/3236051663 >> 172.21.15.16:0/3236051663 conn(0x555f3215e010 msgr2=0x555f3215e410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:35.979 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.973+0000 7f45ae47a740 1 -- 172.21.15.16:0/3236051663 shutdown_connections 2024-09-12T02:11:35.979 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.973+0000 7f45ae47a740 1 -- 172.21.15.16:0/3236051663 wait complete. 2024-09-12T02:11:35.979 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.973+0000 7f45ae47a740 1 -- start start 2024-09-12T02:11:35.979 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.973+0000 7f45ae47a740 1 Processor -- start 2024-09-12T02:11:35.979 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.974+0000 7f45ae47a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555f321629a0 0x555f3216cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:35.979 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.974+0000 7f45ae47a740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555f321632e0 0x555f3216d100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:35.979 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.974+0000 7f45ae47a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555f32168370 0x555f32174140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:35.979 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.974+0000 7f45ae47a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x555f3216b340 con 0x555f32168370 2024-09-12T02:11:35.979 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.974+0000 7f45ae47a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x555f3204f290 con 0x555f321629a0 2024-09-12T02:11:35.980 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.974+0000 7f45ae47a740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x555f3216b4c0 con 0x555f321632e0 2024-09-12T02:11:35.980 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.974+0000 7f45ae47a740 10 client.0 _ll_register_callbacks cb 0x555f32080140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:11:35.980 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.974+0000 7f45a9c7a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555f32168370 0x555f32174140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:35.980 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.974+0000 7f45a9c7a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555f32168370 0x555f32174140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52112/0 (socket says 172.21.15.16:52112) 2024-09-12T02:11:35.980 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.974+0000 7f45a9c7a640 1 -- 172.21.15.16:0/368617364 learned_addr learned my addr 172.21.15.16:0/368617364 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:11:35.980 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.974+0000 7f45a8c78640 1 --2- 172.21.15.16:0/368617364 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555f321632e0 0x555f3216d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:35.980 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.974+0000 7f45a9479640 1 --2- 172.21.15.16:0/368617364 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555f321629a0 0x555f3216cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:35.980 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.974+0000 7f45a9c7a640 1 -- 172.21.15.16:0/368617364 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555f321632e0 msgr2=0x555f3216d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:35.980 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.974+0000 7f45a9c7a640 1 --2- 172.21.15.16:0/368617364 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555f321632e0 0x555f3216d100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:35.980 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.974+0000 7f45a9c7a640 1 -- 172.21.15.16:0/368617364 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555f321629a0 msgr2=0x555f3216cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:35.980 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.974+0000 7f45a9c7a640 1 --2- 172.21.15.16:0/368617364 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555f321629a0 0x555f3216cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:35.980 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.974+0000 7f45a9c7a640 1 -- 172.21.15.16:0/368617364 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f31dd9210 con 0x555f32168370 2024-09-12T02:11:35.980 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.974+0000 7f45a9c7a640 1 --2- 172.21.15.16:0/368617364 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555f32168370 0x555f32174140 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f4594006010 tx=0x7f4594031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=741373198187112e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:35.980 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.974+0000 7f458cff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:11:35.981 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.974+0000 7f458cff9640 1 -- 172.21.15.16:0/368617364 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4594049040 con 0x555f32168370 2024-09-12T02:11:35.981 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.974+0000 7f458cff9640 1 -- 172.21.15.16:0/368617364 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f45940b8820 con 0x555f32168370 2024-09-12T02:11:35.981 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.974+0000 7f45ae47a740 1 -- 172.21.15.16:0/368617364 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45a001b900 con 0x555f32168370 2024-09-12T02:11:35.981 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.975+0000 7f458cff9640 1 -- 172.21.15.16:0/368617364 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f45940bc240 con 0x555f32168370 2024-09-12T02:11:35.981 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.975+0000 7f45ae47a740 10 client.18327 fetch_fsmap learned FSMap version 25 2024-09-12T02:11:35.981 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.975+0000 7f45ae47a740 1 -- 172.21.15.16:0/368617364 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555f31ea5620 con 0x555f32168370 2024-09-12T02:11:35.981 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.975+0000 7f458cff9640 1 -- 172.21.15.16:0/368617364 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45a001b900 con 0x555f32168370 2024-09-12T02:11:35.981 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.975+0000 7f458cff9640 1 -- 172.21.15.16:0/368617364 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f45940bc4a0 con 0x555f32168370 2024-09-12T02:11:35.981 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.975+0000 7f45ae47a740 10 client.18327 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:11:35.981 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.975+0000 7f45ae47a740 10 client.18327 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:11:35.981 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.975+0000 7f45ae47a740 20 client.18327 populate_metadata read hostname 'smithi016' 2024-09-12T02:11:35.981 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.975+0000 7f45ae47a740 1 --2- 172.21.15.16:0/368617364 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x555f32178820 0x555f3217aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:35.981 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.975+0000 7f45ae47a740 4 client.18327 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:11:35.982 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.975+0000 7f45ae47a740 1 -- 172.21.15.16:0/368617364 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555f320d3a20 con 0x555f32178820 2024-09-12T02:11:35.982 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.975+0000 7f45a9479640 1 --2- 172.21.15.16:0/368617364 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x555f32178820 0x555f3217aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:35.982 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.976+0000 7f458cff9640 10 client.18327 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:11:35.982 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.976+0000 7f45a9479640 1 --2- 172.21.15.16:0/368617364 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x555f32178820 0x555f3217aca0 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5ad975db8a65741 server_cookie=4e4406dc6f2e281e in_seq=0 out_seq=0 2024-09-12T02:11:35.982 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.976+0000 7f458cff9640 1 -- 172.21.15.16:0/368617364 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555f320d3a20 con 0x555f32178820 2024-09-12T02:11:35.982 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.976+0000 7f458cff9640 10 client.18327 handle_command_reply: tid=0 2024-09-12T02:11:35.982 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.976+0000 7f45ae47a740 1 client.18327 shutdown 2024-09-12T02:11:35.982 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.977+0000 7f45ae47a740 1 -- 172.21.15.16:0/368617364 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555f32168370 msgr2=0x555f32174140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:35.982 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.977+0000 7f45ae47a740 1 --2- 172.21.15.16:0/368617364 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555f32168370 0x555f32174140 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f4594006010 tx=0x7f4594031400 comp rx=0 tx=0).stop 2024-09-12T02:11:35.982 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.977+0000 7f45ae47a740 1 -- 172.21.15.16:0/368617364 shutdown_connections 2024-09-12T02:11:35.982 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.977+0000 7f45ae47a740 1 --2- 172.21.15.16:0/368617364 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x555f32178820 0x555f3217aca0 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:35.982 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.977+0000 7f45ae47a740 1 --2- 172.21.15.16:0/368617364 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555f32168370 0x555f32174140 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:35.982 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.977+0000 7f45ae47a740 1 --2- 172.21.15.16:0/368617364 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555f321632e0 0x555f3216d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:35.982 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.977+0000 7f458cff9640 0 client.18327 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:11:35.983 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.977+0000 7f45ae47a740 1 --2- 172.21.15.16:0/368617364 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555f321629a0 0x555f3216cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:35.983 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.977+0000 7f45ae47a740 1 -- 172.21.15.16:0/368617364 >> 172.21.15.16:0/368617364 conn(0x555f3215e010 msgr2=0x555f3215f820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:35.983 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.977+0000 7f45ae47a740 1 -- 172.21.15.16:0/368617364 shutdown_connections 2024-09-12T02:11:35.983 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.977+0000 7f45ae47a740 1 -- 172.21.15.16:0/368617364 wait complete. 2024-09-12T02:11:35.983 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.977+0000 7f45ae47a740 20 client.18327 trim_cache size 0 max 16384 2024-09-12T02:11:35.992 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.991+0000 7f45ae47a740 1 Processor -- start 2024-09-12T02:11:35.993 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.991+0000 7f45ae47a740 1 -- start start 2024-09-12T02:11:35.993 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.992+0000 7f45ae47a740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555f321629a0 0x555f3215eec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:35.993 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.992+0000 7f45ae47a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555f321632e0 0x555f32174100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:35.993 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.992+0000 7f45ae47a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555f32168370 0x555f3215f400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:35.993 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.992+0000 7f45ae47a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f45a41ab480 con 0x555f321632e0 2024-09-12T02:11:35.993 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.992+0000 7f45ae47a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f45a41a8d50 con 0x555f32168370 2024-09-12T02:11:35.993 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.992+0000 7f45ae47a740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f45a41a6ba0 con 0x555f321629a0 2024-09-12T02:11:35.993 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.992+0000 7f45a8c78640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555f321632e0 0x555f32174100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:35.994 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.992+0000 7f45a8c78640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555f321632e0 0x555f32174100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52114/0 (socket says 172.21.15.16:52114) 2024-09-12T02:11:35.994 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.992+0000 7f45a8c78640 1 -- 172.21.15.16:0/3250064744 learned_addr learned my addr 172.21.15.16:0/3250064744 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:11:35.994 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.992+0000 7f45a9479640 1 --2- 172.21.15.16:0/3250064744 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555f321629a0 0x555f3215eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:35.994 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.992+0000 7f45a9c7a640 1 --2- 172.21.15.16:0/3250064744 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555f32168370 0x555f3215f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:35.994 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.992+0000 7f45a8c78640 1 -- 172.21.15.16:0/3250064744 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555f321629a0 msgr2=0x555f3215eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:35.994 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.992+0000 7f45a8c78640 1 --2- 172.21.15.16:0/3250064744 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555f321629a0 0x555f3215eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:35.994 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.992+0000 7f45a8c78640 1 -- 172.21.15.16:0/3250064744 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555f32168370 msgr2=0x555f3215f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:35.994 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.992+0000 7f45a8c78640 1 --2- 172.21.15.16:0/3250064744 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555f32168370 0x555f3215f400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:35.994 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.992+0000 7f45a8c78640 1 -- 172.21.15.16:0/3250064744 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f32163f70 con 0x555f321632e0 2024-09-12T02:11:35.994 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.992+0000 7f45a8c78640 1 --2- 172.21.15.16:0/3250064744 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555f321632e0 0x555f32174100 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f459c007d30 tx=0x7f459c012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=977a0507b9daf33c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:35.994 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.992+0000 7f458effd640 1 -- 172.21.15.16:0/3250064744 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f459c004d20 con 0x555f321632e0 2024-09-12T02:11:35.994 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.993+0000 7f458effd640 1 -- 172.21.15.16:0/3250064744 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f459c010b30 con 0x555f321632e0 2024-09-12T02:11:35.994 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.993+0000 7f458effd640 1 -- 172.21.15.16:0/3250064744 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f459c0056d0 con 0x555f321632e0 2024-09-12T02:11:35.997 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.993+0000 7f45ae47a740 1 -- 172.21.15.16:0/3250064744 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555f321632e0 msgr2=0x555f32174100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:35.998 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.993+0000 7f45ae47a740 1 --2- 172.21.15.16:0/3250064744 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555f321632e0 0x555f32174100 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f459c007d30 tx=0x7f459c012820 comp rx=0 tx=0).stop 2024-09-12T02:11:35.998 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.993+0000 7f45ae47a740 1 -- 172.21.15.16:0/3250064744 shutdown_connections 2024-09-12T02:11:35.998 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.993+0000 7f45ae47a740 1 --2- 172.21.15.16:0/3250064744 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555f32168370 0x555f3215f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:35.998 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.993+0000 7f45ae47a740 1 --2- 172.21.15.16:0/3250064744 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555f321632e0 0x555f32174100 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:35.998 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.993+0000 7f45ae47a740 1 --2- 172.21.15.16:0/3250064744 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555f321629a0 0x555f3215eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:35.998 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.993+0000 7f45ae47a740 1 -- 172.21.15.16:0/3250064744 >> 172.21.15.16:0/3250064744 conn(0x555f3215e010 msgr2=0x555f32174bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:35.998 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.993+0000 7f45ae47a740 1 -- 172.21.15.16:0/3250064744 shutdown_connections 2024-09-12T02:11:35.998 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.993+0000 7f45ae47a740 1 -- 172.21.15.16:0/3250064744 wait complete. 2024-09-12T02:11:35.998 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.993+0000 7f45ae47a740 1 -- start start 2024-09-12T02:11:35.998 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.993+0000 7f45ae47a740 1 Processor -- start 2024-09-12T02:11:35.998 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.994+0000 7f45ae47a740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555f321629a0 0x555f32171990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:35.998 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.994+0000 7f45ae47a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555f321632e0 0x555f32171ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:35.999 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.994+0000 7f45ae47a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555f32168370 0x555f32172e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:35.999 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.994+0000 7f45ae47a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f45a41a7210 con 0x555f321632e0 2024-09-12T02:11:35.999 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.994+0000 7f45ae47a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f45a41a8680 con 0x555f32168370 2024-09-12T02:11:35.999 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.994+0000 7f45ae47a740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f45a41a78e0 con 0x555f321629a0 2024-09-12T02:11:35.999 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.994+0000 7f45ae47a740 10 client.0 _ll_register_callbacks cb 0x7f4594005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:11:35.999 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.994+0000 7f45a8c78640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555f321632e0 0x555f32171ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:35.999 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.994+0000 7f45a8c78640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555f321632e0 0x555f32171ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52118/0 (socket says 172.21.15.16:52118) 2024-09-12T02:11:35.999 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.994+0000 7f45a8c78640 1 -- 172.21.15.16:0/4167192095 learned_addr learned my addr 172.21.15.16:0/4167192095 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:11:35.999 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.994+0000 7f45a9c7a640 1 --2- 172.21.15.16:0/4167192095 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555f32168370 0x555f32172e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:35.999 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.994+0000 7f45a9479640 1 --2- 172.21.15.16:0/4167192095 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555f321629a0 0x555f32171990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:35.999 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.994+0000 7f45a8c78640 1 -- 172.21.15.16:0/4167192095 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555f321629a0 msgr2=0x555f32171990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:35.999 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.994+0000 7f45a8c78640 1 --2- 172.21.15.16:0/4167192095 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555f321629a0 0x555f32171990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:35.999 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.994+0000 7f45a8c78640 1 -- 172.21.15.16:0/4167192095 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555f32168370 msgr2=0x555f32172e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:35.999 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.994+0000 7f45a8c78640 1 --2- 172.21.15.16:0/4167192095 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555f32168370 0x555f32172e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:36.000 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.994+0000 7f45a8c78640 1 -- 172.21.15.16:0/4167192095 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f321592d0 con 0x555f321632e0 2024-09-12T02:11:36.000 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.995+0000 7f45a8c78640 1 --2- 172.21.15.16:0/4167192095 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555f321632e0 0x555f32171ed0 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f459c000f80 tx=0x7f459c0071c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c5cf068c1e5a549 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:36.000 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.995+0000 7f458cff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:11:36.000 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.995+0000 7f458cff9640 1 -- 172.21.15.16:0/4167192095 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f459c004a40 con 0x555f321632e0 2024-09-12T02:11:36.000 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.995+0000 7f45ae47a740 1 -- 172.21.15.16:0/4167192095 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f459c010b30 con 0x555f321632e0 2024-09-12T02:11:36.000 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.995+0000 7f458cff9640 1 -- 172.21.15.16:0/4167192095 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f459c004be0 con 0x555f321632e0 2024-09-12T02:11:36.000 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.995+0000 7f458cff9640 1 -- 172.21.15.16:0/4167192095 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f459c003970 con 0x555f321632e0 2024-09-12T02:11:36.000 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.995+0000 7f45ae47a740 10 client.18333 fetch_fsmap learned FSMap version 25 2024-09-12T02:11:36.000 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.995+0000 7f45ae47a740 1 -- 172.21.15.16:0/4167192095 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555f321e3580 con 0x555f321632e0 2024-09-12T02:11:36.000 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.995+0000 7f458cff9640 1 -- 172.21.15.16:0/4167192095 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f459c010b30 con 0x555f321632e0 2024-09-12T02:11:36.000 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.995+0000 7f458cff9640 1 -- 172.21.15.16:0/4167192095 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f459c019070 con 0x555f321632e0 2024-09-12T02:11:36.000 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.995+0000 7f45ae47a740 10 client.18333 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:11:36.000 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.995+0000 7f45ae47a740 10 client.18333 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:11:36.000 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.995+0000 7f45ae47a740 20 client.18333 populate_metadata read hostname 'smithi016' 2024-09-12T02:11:36.000 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.995+0000 7f45ae47a740 1 --2- 172.21.15.16:0/4167192095 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x555f321e3e90 0x555f321e62b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:36.001 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.995+0000 7f45ae47a740 4 client.18333 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:11:36.001 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.995+0000 7f45ae47a740 1 -- 172.21.15.16:0/4167192095 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555f320cdd70 con 0x555f321e3e90 2024-09-12T02:11:36.001 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.996+0000 7f45a9479640 1 --2- 172.21.15.16:0/4167192095 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x555f321e3e90 0x555f321e62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:36.001 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.996+0000 7f458cff9640 10 client.18333 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:11:36.001 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:35.996+0000 7f45a9479640 1 --2- 172.21.15.16:0/4167192095 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x555f321e3e90 0x555f321e62b0 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86094b6c004a6640 server_cookie=207824b2dd8297b3 in_seq=0 out_seq=0 2024-09-12T02:11:36.010 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:36.009+0000 7f458cff9640 1 -- 172.21.15.16:0/4167192095 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555f320cdd70 con 0x555f321e3e90 2024-09-12T02:11:36.010 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:36.009+0000 7f458cff9640 10 client.18333 handle_command_reply: tid=0 2024-09-12T02:11:36.010 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:36.009+0000 7f45ae47a740 1 client.18333 shutdown 2024-09-12T02:11:36.010 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:36.009+0000 7f45ae47a740 1 -- 172.21.15.16:0/4167192095 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555f321632e0 msgr2=0x555f32171ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:36.010 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:36.009+0000 7f45ae47a740 1 --2- 172.21.15.16:0/4167192095 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555f321632e0 0x555f32171ed0 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f459c000f80 tx=0x7f459c0071c0 comp rx=0 tx=0).stop 2024-09-12T02:11:36.010 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:36.009+0000 7f45ae47a740 1 -- 172.21.15.16:0/4167192095 shutdown_connections 2024-09-12T02:11:36.010 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:36.009+0000 7f45ae47a740 1 --2- 172.21.15.16:0/4167192095 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x555f321e3e90 0x555f321e62b0 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:36.010 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:36.009+0000 7f45ae47a740 1 --2- 172.21.15.16:0/4167192095 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555f32168370 0x555f32172e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:36.010 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:36.009+0000 7f458cff9640 0 client.18333 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:11:36.010 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:36.009+0000 7f45ae47a740 1 --2- 172.21.15.16:0/4167192095 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555f321632e0 0x555f32171ed0 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:36.010 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:36.009+0000 7f45ae47a740 1 --2- 172.21.15.16:0/4167192095 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555f321629a0 0x555f32171990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:36.010 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:36.009+0000 7f45ae47a740 1 -- 172.21.15.16:0/4167192095 >> 172.21.15.16:0/4167192095 conn(0x555f3215e010 msgr2=0x555f3215c9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:36.011 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:36.009+0000 7f45ae47a740 1 -- 172.21.15.16:0/4167192095 shutdown_connections 2024-09-12T02:11:36.011 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:36.009+0000 7f45ae47a740 1 -- 172.21.15.16:0/4167192095 wait complete. 2024-09-12T02:11:36.011 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:36.009+0000 7f45ae47a740 20 client.18333 trim_cache size 0 max 16384 2024-09-12T02:11:36.011 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-09-12T02:11:36.011 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-09-12T02:11:36.011 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-09-12T02:11:36.011 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-09-12T02:11:36.012 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:36.010+0000 7f45ab6dd640 1 -- 172.21.15.16:0/3042621418 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f45900a2c30 msgr2=0x7f45900a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:36.012 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:36.010+0000 7f45ab6dd640 1 --2- 172.21.15.16:0/3042621418 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f45900a2c30 0x7f45900a50f0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f45a4199100 tx=0x7f45a0005f90 comp rx=0 tx=0).stop 2024-09-12T02:11:36.012 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:36.010+0000 7f45ab6dd640 1 -- 172.21.15.16:0/3042621418 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45a41025f0 msgr2=0x7f45a419f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:36.012 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:36.010+0000 7f45ab6dd640 1 --2- 172.21.15.16:0/3042621418 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45a41025f0 0x7f45a419f1a0 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f45940314e0 tx=0x7f45940043c0 comp rx=0 tx=0).stop 2024-09-12T02:11:36.012 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:36.010+0000 7f45ab6dd640 1 -- 172.21.15.16:0/3042621418 shutdown_connections 2024-09-12T02:11:36.012 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:36.010+0000 7f45ab6dd640 1 --2- 172.21.15.16:0/3042621418 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f45900a2c30 0x7f45900a50f0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:36.012 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:36.010+0000 7f45ab6dd640 1 --2- 172.21.15.16:0/3042621418 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45a41025f0 0x7f45a419f1a0 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:36.013 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:36.010+0000 7f45ab6dd640 1 --2- 172.21.15.16:0/3042621418 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f45a4101c30 0x7f45a4198170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:36.013 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:36.010+0000 7f45ab6dd640 1 --2- 172.21.15.16:0/3042621418 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f45a4100a40 0x7f45a4197c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:36.013 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:36.010+0000 7f45ab6dd640 1 -- 172.21.15.16:0/3042621418 >> 172.21.15.16:0/3042621418 conn(0x7f45a40fc1e0 msgr2=0x7f45a406a0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:36.013 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:36.010+0000 7f45ab6dd640 1 -- 172.21.15.16:0/3042621418 shutdown_connections 2024-09-12T02:11:36.013 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:36.010+0000 7f45ab6dd640 1 -- 172.21.15.16:0/3042621418 wait complete. 2024-09-12T02:11:36.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.424+0000 7f2f16edd640 1 -- 172.21.15.16:0/623113998 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f10101de0 msgr2=0x7f2f101021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:36.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.424+0000 7f2f16edd640 1 --2- 172.21.15.16:0/623113998 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f10101de0 0x7f2f101021c0 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f2f04009900 tx=0x7f2f0402a410 comp rx=0 tx=0).stop 2024-09-12T02:11:36.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.424+0000 7f2f16edd640 1 -- 172.21.15.16:0/623113998 shutdown_connections 2024-09-12T02:11:36.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.424+0000 7f2f16edd640 1 --2- 172.21.15.16:0/623113998 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2f10106f10 0x7f2f1010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:36.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.424+0000 7f2f16edd640 1 --2- 172.21.15.16:0/623113998 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f10102790 0x7f2f101067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:36.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.424+0000 7f2f16edd640 1 --2- 172.21.15.16:0/623113998 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f10101de0 0x7f2f101021c0 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:36.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.424+0000 7f2f16edd640 1 -- 172.21.15.16:0/623113998 >> 172.21.15.16:0/623113998 conn(0x7f2f100fd8c0 msgr2=0x7f2f100ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:36.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.425+0000 7f2f16edd640 1 -- 172.21.15.16:0/623113998 shutdown_connections 2024-09-12T02:11:36.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.425+0000 7f2f16edd640 1 -- 172.21.15.16:0/623113998 wait complete. 2024-09-12T02:11:36.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.425+0000 7f2f16edd640 1 Processor -- start 2024-09-12T02:11:36.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.425+0000 7f2f16edd640 1 -- start start 2024-09-12T02:11:36.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.426+0000 7f2f16edd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f10101de0 0x7f2f10198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:36.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.426+0000 7f2f16edd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f10102790 0x7f2f101989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:36.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.426+0000 7f2f16edd640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2f10106f10 0x7f2f1019cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:36.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.426+0000 7f2f16edd640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2f10111f90 con 0x7f2f10102790 2024-09-12T02:11:36.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.426+0000 7f2f16edd640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2f10111e10 con 0x7f2f10101de0 2024-09-12T02:11:36.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.426+0000 7f2f16edd640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f2f10112110 con 0x7f2f10106f10 2024-09-12T02:11:36.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.426+0000 7f2f0ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f10102790 0x7f2f101989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:36.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.426+0000 7f2f0ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f10102790 0x7f2f101989c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52136/0 (socket says 172.21.15.16:52136) 2024-09-12T02:11:36.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.426+0000 7f2f0ffff640 1 -- 172.21.15.16:0/955627393 learned_addr learned my addr 172.21.15.16:0/955627393 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:11:36.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.426+0000 7f2f1547a640 1 --2- 172.21.15.16:0/955627393 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2f10106f10 0x7f2f1019cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:36.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.426+0000 7f2f14c79640 1 --2- 172.21.15.16:0/955627393 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f10101de0 0x7f2f10198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:36.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.426+0000 7f2f0ffff640 1 -- 172.21.15.16:0/955627393 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2f10106f10 msgr2=0x7f2f1019cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:36.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.426+0000 7f2f0ffff640 1 --2- 172.21.15.16:0/955627393 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2f10106f10 0x7f2f1019cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:36.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.426+0000 7f2f0ffff640 1 -- 172.21.15.16:0/955627393 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f10101de0 msgr2=0x7f2f10198480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:36.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.426+0000 7f2f0ffff640 1 --2- 172.21.15.16:0/955627393 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f10101de0 0x7f2f10198480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:36.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.426+0000 7f2f0ffff640 1 -- 172.21.15.16:0/955627393 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f1019d430 con 0x7f2f10102790 2024-09-12T02:11:36.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.426+0000 7f2f0ffff640 1 --2- 172.21.15.16:0/955627393 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f10102790 0x7f2f101989c0 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f2efc00e9f0 tx=0x7f2efc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:36.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.426+0000 7f2f0dffb640 1 -- 172.21.15.16:0/955627393 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2efc00cd80 con 0x7f2f10102790 2024-09-12T02:11:36.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.426+0000 7f2f0dffb640 1 -- 172.21.15.16:0/955627393 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2efc010910 con 0x7f2f10102790 2024-09-12T02:11:36.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.427+0000 7f2f16edd640 1 -- 172.21.15.16:0/955627393 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f1019d720 con 0x7f2f10102790 2024-09-12T02:11:36.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.427+0000 7f2f0dffb640 1 -- 172.21.15.16:0/955627393 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2efc018990 con 0x7f2f10102790 2024-09-12T02:11:36.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.427+0000 7f2f16edd640 1 -- 172.21.15.16:0/955627393 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f101a5000 con 0x7f2f10102790 2024-09-12T02:11:36.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.430+0000 7f2f0dffb640 1 -- 172.21.15.16:0/955627393 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f2efc010ac0 con 0x7f2f10102790 2024-09-12T02:11:36.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.430+0000 7f2f192b6740 1 Processor -- start 2024-09-12T02:11:36.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.430+0000 7f2f192b6740 1 -- start start 2024-09-12T02:11:36.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.430+0000 7f2f192b6740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5587a11074a0 0x5587a1107880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:36.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.430+0000 7f2f192b6740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5587a1107dc0 0x5587a1112140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:36.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.430+0000 7f2f192b6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587a1112680 0x5587a1114a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:36.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.430+0000 7f2f192b6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5587a106d2e0 con 0x5587a1112680 2024-09-12T02:11:36.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.430+0000 7f2f192b6740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5587a1014ce0 con 0x5587a1107dc0 2024-09-12T02:11:36.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.430+0000 7f2f192b6740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5587a10692a0 con 0x5587a11074a0 2024-09-12T02:11:36.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.430+0000 7f2f0ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587a1112680 0x5587a1114a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:36.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.430+0000 7f2f0ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587a1112680 0x5587a1114a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52140/0 (socket says 172.21.15.16:52140) 2024-09-12T02:11:36.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.430+0000 7f2f0ffff640 1 -- 172.21.15.16:0/3996378107 learned_addr learned my addr 172.21.15.16:0/3996378107 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:11:36.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.430+0000 7f2f14c79640 1 --2- 172.21.15.16:0/3996378107 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5587a1107dc0 0x5587a1112140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:36.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.430+0000 7f2f1547a640 1 --2- 172.21.15.16:0/3996378107 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5587a11074a0 0x5587a1107880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:36.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.431+0000 7f2f0dffb640 1 --2- 172.21.15.16:0/955627393 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f2f000a2c00 0x7f2f000a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:36.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.431+0000 7f2f0ffff640 1 -- 172.21.15.16:0/3996378107 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5587a11074a0 msgr2=0x5587a1107880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:36.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.431+0000 7f2f0ffff640 1 --2- 172.21.15.16:0/3996378107 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5587a11074a0 0x5587a1107880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:36.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.431+0000 7f2f0ffff640 1 -- 172.21.15.16:0/3996378107 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5587a1107dc0 msgr2=0x5587a1112140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:36.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.431+0000 7f2f0ffff640 1 --2- 172.21.15.16:0/3996378107 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5587a1107dc0 0x5587a1112140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:36.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.431+0000 7f2f0ffff640 1 -- 172.21.15.16:0/3996378107 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587a0ff9440 con 0x5587a1112680 2024-09-12T02:11:36.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.431+0000 7f2f1547a640 1 --2- 172.21.15.16:0/3996378107 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5587a11074a0 0x5587a1107880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:11:36.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.431+0000 7f2f1547a640 1 --2- 172.21.15.16:0/955627393 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f2f000a2c00 0x7f2f000a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:36.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.431+0000 7f2f0dffb640 1 -- 172.21.15.16:0/955627393 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f2efc0cf210 con 0x7f2f10102790 2024-09-12T02:11:36.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.431+0000 7f2f0ffff640 1 --2- 172.21.15.16:0/3996378107 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587a1112680 0x5587a1114a70 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f2efc015690 tx=0x7f2efc01d9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f67500a11d2f4d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:36.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.431+0000 7f2efaffd640 1 -- 172.21.15.16:0/3996378107 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2efc09b700 con 0x5587a1112680 2024-09-12T02:11:36.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.431+0000 7f2efaffd640 1 -- 172.21.15.16:0/3996378107 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2efc015960 con 0x5587a1112680 2024-09-12T02:11:36.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.431+0000 7f2efaffd640 1 -- 172.21.15.16:0/3996378107 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2efc09b520 con 0x5587a1112680 2024-09-12T02:11:36.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.431+0000 7f2f1547a640 1 --2- 172.21.15.16:0/955627393 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f2f000a2c00 0x7f2f000a50c0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f2f0800a5a0 tx=0x7f2f08004b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:36.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.432+0000 7f2f192b6740 1 -- 172.21.15.16:0/3996378107 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587a1112680 msgr2=0x5587a1114a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:36.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.432+0000 7f2f192b6740 1 --2- 172.21.15.16:0/3996378107 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587a1112680 0x5587a1114a70 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f2efc015690 tx=0x7f2efc01d9c0 comp rx=0 tx=0).stop 2024-09-12T02:11:36.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.432+0000 7f2f192b6740 1 -- 172.21.15.16:0/3996378107 shutdown_connections 2024-09-12T02:11:36.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.432+0000 7f2f192b6740 1 --2- 172.21.15.16:0/3996378107 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587a1112680 0x5587a1114a70 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:36.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.432+0000 7f2f192b6740 1 --2- 172.21.15.16:0/3996378107 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5587a1107dc0 0x5587a1112140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:36.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.432+0000 7f2f192b6740 1 --2- 172.21.15.16:0/3996378107 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5587a11074a0 0x5587a1107880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:36.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.432+0000 7f2f192b6740 1 -- 172.21.15.16:0/3996378107 >> 172.21.15.16:0/3996378107 conn(0x5587a1103a30 msgr2=0x5587a1103e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:36.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.432+0000 7f2f192b6740 1 -- 172.21.15.16:0/3996378107 shutdown_connections 2024-09-12T02:11:36.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.432+0000 7f2f192b6740 1 -- 172.21.15.16:0/3996378107 wait complete. 2024-09-12T02:11:36.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.432+0000 7f2f192b6740 1 -- start start 2024-09-12T02:11:36.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.432+0000 7f2f192b6740 1 Processor -- start 2024-09-12T02:11:36.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.433+0000 7f2f192b6740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5587a11074a0 0x5587a1111fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:36.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.433+0000 7f2f192b6740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5587a1107dc0 0x5587a110f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:36.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.433+0000 7f2f192b6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587a1112680 0x5587a110f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:36.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.433+0000 7f2f192b6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5587a1115250 con 0x5587a1112680 2024-09-12T02:11:36.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.433+0000 7f2f192b6740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5587a1028050 con 0x5587a1107dc0 2024-09-12T02:11:36.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.433+0000 7f2f192b6740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5587a11153d0 con 0x5587a11074a0 2024-09-12T02:11:36.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.433+0000 7f2f192b6740 10 client.0 _ll_register_callbacks cb 0x5587a1068160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:11:36.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.433+0000 7f2f1547a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587a1112680 0x5587a110f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:36.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.433+0000 7f2f1547a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587a1112680 0x5587a110f7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52152/0 (socket says 172.21.15.16:52152) 2024-09-12T02:11:36.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.433+0000 7f2f1547a640 1 -- 172.21.15.16:0/1847748195 learned_addr learned my addr 172.21.15.16:0/1847748195 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:11:36.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.433+0000 7f2f0ffff640 1 --2- 172.21.15.16:0/1847748195 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5587a1107dc0 0x5587a110f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:36.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.433+0000 7f2f14c79640 1 --2- 172.21.15.16:0/1847748195 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5587a11074a0 0x5587a1111fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:36.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.433+0000 7f2f1547a640 1 -- 172.21.15.16:0/1847748195 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5587a11074a0 msgr2=0x5587a1111fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:36.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.433+0000 7f2f1547a640 1 --2- 172.21.15.16:0/1847748195 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5587a11074a0 0x5587a1111fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:36.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.433+0000 7f2f1547a640 1 -- 172.21.15.16:0/1847748195 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5587a1107dc0 msgr2=0x5587a110f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:36.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.433+0000 7f2f1547a640 1 --2- 172.21.15.16:0/1847748195 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5587a1107dc0 0x5587a110f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:36.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.433+0000 7f2f1547a640 1 -- 172.21.15.16:0/1847748195 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5587a0dde3c0 con 0x5587a1112680 2024-09-12T02:11:36.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.433+0000 7f2f0ffff640 1 --2- 172.21.15.16:0/1847748195 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5587a1107dc0 0x5587a110f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:11:36.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.434+0000 7f2f1547a640 1 --2- 172.21.15.16:0/1847748195 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587a1112680 0x5587a110f7e0 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f2f08007e20 tx=0x7f2f0800d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0326c985c288192 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:36.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.434+0000 7f2eeffff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:11:36.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.434+0000 7f2eeffff640 1 -- 172.21.15.16:0/1847748195 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f08027060 con 0x5587a1112680 2024-09-12T02:11:36.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.434+0000 7f2eeffff640 1 -- 172.21.15.16:0/1847748195 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f2f08015560 con 0x5587a1112680 2024-09-12T02:11:36.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.434+0000 7f2f192b6740 1 -- 172.21.15.16:0/1847748195 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5587a107a850 con 0x5587a1112680 2024-09-12T02:11:36.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.434+0000 7f2eeffff640 1 -- 172.21.15.16:0/1847748195 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2f08007e50 con 0x5587a1112680 2024-09-12T02:11:36.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.434+0000 7f2f192b6740 10 client.18345 fetch_fsmap learned FSMap version 25 2024-09-12T02:11:36.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.434+0000 7f2f192b6740 1 -- 172.21.15.16:0/1847748195 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5587a1120cd0 con 0x5587a1112680 2024-09-12T02:11:36.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.434+0000 7f2eeffff640 1 -- 172.21.15.16:0/1847748195 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5587a107a850 con 0x5587a1112680 2024-09-12T02:11:36.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.434+0000 7f2eeffff640 1 -- 172.21.15.16:0/1847748195 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f2f0802c070 con 0x5587a1112680 2024-09-12T02:11:36.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.435+0000 7f2f192b6740 10 client.18345 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:11:36.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.435+0000 7f2f192b6740 10 client.18345 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:11:36.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.435+0000 7f2f192b6740 20 client.18345 populate_metadata read hostname 'smithi016' 2024-09-12T02:11:36.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.435+0000 7f2f192b6740 1 --2- 172.21.15.16:0/1847748195 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5587a1122190 0x5587a11245c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:36.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.435+0000 7f2f192b6740 4 client.18345 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:11:36.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.435+0000 7f2f192b6740 1 -- 172.21.15.16:0/1847748195 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5587a1074d60 con 0x5587a1122190 2024-09-12T02:11:36.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.435+0000 7f2f14c79640 1 --2- 172.21.15.16:0/1847748195 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5587a1122190 0x5587a11245c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:36.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.435+0000 7f2eeffff640 10 client.18345 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:11:36.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.435+0000 7f2f14c79640 1 --2- 172.21.15.16:0/1847748195 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5587a1122190 0x5587a11245c0 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=77577dd744b16974 server_cookie=a4e6e3ac51418a0e in_seq=0 out_seq=0 2024-09-12T02:11:36.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.436+0000 7f2eeffff640 1 -- 172.21.15.16:0/1847748195 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5587a1074d60 con 0x5587a1122190 2024-09-12T02:11:36.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.436+0000 7f2eeffff640 10 client.18345 handle_command_reply: tid=0 2024-09-12T02:11:36.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.436+0000 7f2f192b6740 1 client.18345 shutdown 2024-09-12T02:11:36.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.436+0000 7f2f192b6740 1 -- 172.21.15.16:0/1847748195 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587a1112680 msgr2=0x5587a110f7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:36.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.436+0000 7f2f192b6740 1 --2- 172.21.15.16:0/1847748195 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587a1112680 0x5587a110f7e0 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f2f08007e20 tx=0x7f2f0800d040 comp rx=0 tx=0).stop 2024-09-12T02:11:36.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.436+0000 7f2f192b6740 1 -- 172.21.15.16:0/1847748195 shutdown_connections 2024-09-12T02:11:36.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.436+0000 7f2f192b6740 1 --2- 172.21.15.16:0/1847748195 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5587a1122190 0x5587a11245c0 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:36.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.436+0000 7f2f192b6740 1 --2- 172.21.15.16:0/1847748195 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587a1112680 0x5587a110f7e0 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:36.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.436+0000 7f2eeffff640 0 client.18345 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:11:36.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.436+0000 7f2f192b6740 1 --2- 172.21.15.16:0/1847748195 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5587a1107dc0 0x5587a110f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:36.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.436+0000 7f2f192b6740 1 --2- 172.21.15.16:0/1847748195 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5587a11074a0 0x5587a1111fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:36.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.436+0000 7f2f192b6740 1 -- 172.21.15.16:0/1847748195 >> 172.21.15.16:0/1847748195 conn(0x5587a1103a30 msgr2=0x5587a1109760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:36.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.436+0000 7f2f192b6740 1 -- 172.21.15.16:0/1847748195 shutdown_connections 2024-09-12T02:11:36.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.436+0000 7f2f192b6740 1 -- 172.21.15.16:0/1847748195 wait complete. 2024-09-12T02:11:36.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.437+0000 7f2f192b6740 20 client.18345 trim_cache size 0 max 16384 2024-09-12T02:11:36.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.454+0000 7f2f192b6740 1 Processor -- start 2024-09-12T02:11:36.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.454+0000 7f2f192b6740 1 -- start start 2024-09-12T02:11:36.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.454+0000 7f2f192b6740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5587a11074a0 0x5587a111e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:36.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.454+0000 7f2f192b6740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5587a1107dc0 0x5587a1129e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:36.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.454+0000 7f2f192b6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587a1112680 0x5587a112a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:36.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.454+0000 7f2f192b6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2f101afd00 con 0x5587a1112680 2024-09-12T02:11:36.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.454+0000 7f2f192b6740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2f101ad5d0 con 0x5587a11074a0 2024-09-12T02:11:36.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.455+0000 7f2f192b6740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f2f101ab420 con 0x5587a1107dc0 2024-09-12T02:11:36.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.455+0000 7f2f1547a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587a1112680 0x5587a112a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:36.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.455+0000 7f2f1547a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587a1112680 0x5587a112a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52162/0 (socket says 172.21.15.16:52162) 2024-09-12T02:11:36.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.455+0000 7f2f1547a640 1 -- 172.21.15.16:0/3521425599 learned_addr learned my addr 172.21.15.16:0/3521425599 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:11:36.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.455+0000 7f2f0ffff640 1 --2- 172.21.15.16:0/3521425599 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5587a1107dc0 0x5587a1129e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:36.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.455+0000 7f2f1547a640 1 -- 172.21.15.16:0/3521425599 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5587a1107dc0 msgr2=0x5587a1129e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:36.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.455+0000 7f2f14c79640 1 --2- 172.21.15.16:0/3521425599 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5587a11074a0 0x5587a111e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:36.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.455+0000 7f2f1547a640 1 --2- 172.21.15.16:0/3521425599 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5587a1107dc0 0x5587a1129e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:36.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.455+0000 7f2f1547a640 1 -- 172.21.15.16:0/3521425599 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5587a11074a0 msgr2=0x5587a111e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:36.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.455+0000 7f2f1547a640 1 --2- 172.21.15.16:0/3521425599 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5587a11074a0 0x5587a111e4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:36.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.455+0000 7f2f1547a640 1 -- 172.21.15.16:0/3521425599 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587a110fb50 con 0x5587a1112680 2024-09-12T02:11:36.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.455+0000 7f2f0ffff640 1 --2- 172.21.15.16:0/3521425599 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5587a1107dc0 0x5587a1129e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:11:36.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.455+0000 7f2f1547a640 1 --2- 172.21.15.16:0/3521425599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587a1112680 0x5587a112a3d0 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f2f08009690 tx=0x7f2f08022f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39a5df488b3562e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:36.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.455+0000 7f2efa7fc640 1 -- 172.21.15.16:0/3521425599 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f08032070 con 0x5587a1112680 2024-09-12T02:11:36.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.455+0000 7f2efa7fc640 1 -- 172.21.15.16:0/3521425599 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f0801d050 con 0x5587a1112680 2024-09-12T02:11:36.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.455+0000 7f2efa7fc640 1 -- 172.21.15.16:0/3521425599 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f0802e460 con 0x5587a1112680 2024-09-12T02:11:36.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.456+0000 7f2f192b6740 1 -- 172.21.15.16:0/3521425599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587a1112680 msgr2=0x5587a112a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:36.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.456+0000 7f2f192b6740 1 --2- 172.21.15.16:0/3521425599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587a1112680 0x5587a112a3d0 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f2f08009690 tx=0x7f2f08022f60 comp rx=0 tx=0).stop 2024-09-12T02:11:36.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.456+0000 7f2f192b6740 1 -- 172.21.15.16:0/3521425599 shutdown_connections 2024-09-12T02:11:36.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.456+0000 7f2f192b6740 1 --2- 172.21.15.16:0/3521425599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587a1112680 0x5587a112a3d0 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:36.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.456+0000 7f2f192b6740 1 --2- 172.21.15.16:0/3521425599 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5587a1107dc0 0x5587a1129e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:36.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.456+0000 7f2f192b6740 1 --2- 172.21.15.16:0/3521425599 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5587a11074a0 0x5587a111e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:36.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.456+0000 7f2f192b6740 1 -- 172.21.15.16:0/3521425599 >> 172.21.15.16:0/3521425599 conn(0x5587a1103a30 msgr2=0x5587a110db50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:36.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.456+0000 7f2f192b6740 1 -- 172.21.15.16:0/3521425599 shutdown_connections 2024-09-12T02:11:36.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.456+0000 7f2f192b6740 1 -- 172.21.15.16:0/3521425599 wait complete. 2024-09-12T02:11:36.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.457+0000 7f2f192b6740 1 -- start start 2024-09-12T02:11:36.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.457+0000 7f2f192b6740 1 Processor -- start 2024-09-12T02:11:36.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.457+0000 7f2f192b6740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5587a11074a0 0x5587a111aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:36.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.457+0000 7f2f192b6740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5587a1107dc0 0x5587a1115dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:36.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.457+0000 7f2f192b6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587a1112680 0x5587a1116310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:36.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.457+0000 7f2f192b6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2f101aba90 con 0x5587a1112680 2024-09-12T02:11:36.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.457+0000 7f2f192b6740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2f101acf00 con 0x5587a1107dc0 2024-09-12T02:11:36.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.457+0000 7f2f192b6740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f2f101ac160 con 0x5587a11074a0 2024-09-12T02:11:36.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.457+0000 7f2f192b6740 10 client.0 _ll_register_callbacks cb 0x7f2f08019040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:11:36.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.457+0000 7f2f1547a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587a1112680 0x5587a1116310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:36.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.457+0000 7f2f1547a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587a1112680 0x5587a1116310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52166/0 (socket says 172.21.15.16:52166) 2024-09-12T02:11:36.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.457+0000 7f2f1547a640 1 -- 172.21.15.16:0/538951856 learned_addr learned my addr 172.21.15.16:0/538951856 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:11:36.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.457+0000 7f2f14c79640 1 --2- 172.21.15.16:0/538951856 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5587a11074a0 0x5587a111aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:36.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.457+0000 7f2f0ffff640 1 --2- 172.21.15.16:0/538951856 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5587a1107dc0 0x5587a1115dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:36.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.458+0000 7f2f1547a640 1 -- 172.21.15.16:0/538951856 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5587a11074a0 msgr2=0x5587a111aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:36.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.458+0000 7f2f1547a640 1 --2- 172.21.15.16:0/538951856 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5587a11074a0 0x5587a111aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:36.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.458+0000 7f2f1547a640 1 -- 172.21.15.16:0/538951856 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5587a1107dc0 msgr2=0x5587a1115dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:36.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.458+0000 7f2f1547a640 1 --2- 172.21.15.16:0/538951856 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5587a1107dc0 0x5587a1115dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:36.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.458+0000 7f2f1547a640 1 -- 172.21.15.16:0/538951856 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5587a1116850 con 0x5587a1112680 2024-09-12T02:11:36.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.458+0000 7f2f1547a640 1 --2- 172.21.15.16:0/538951856 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587a1112680 0x5587a1116310 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f2f0800f040 tx=0x7f2f08016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59d9632536943c1a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:36.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.458+0000 7f2eeffff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:11:36.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.458+0000 7f2eeffff640 1 -- 172.21.15.16:0/538951856 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f08032040 con 0x5587a1112680 2024-09-12T02:11:36.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.458+0000 7f2eeffff640 1 -- 172.21.15.16:0/538951856 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f2f08015970 con 0x5587a1112680 2024-09-12T02:11:36.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.458+0000 7f2f192b6740 1 -- 172.21.15.16:0/538951856 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2f0801d050 con 0x5587a1112680 2024-09-12T02:11:36.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.458+0000 7f2eeffff640 1 -- 172.21.15.16:0/538951856 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2f0802e460 con 0x5587a1112680 2024-09-12T02:11:36.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.458+0000 7f2f192b6740 10 client.18351 fetch_fsmap learned FSMap version 25 2024-09-12T02:11:36.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.458+0000 7f2f192b6740 1 -- 172.21.15.16:0/538951856 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5587a1190f20 con 0x5587a1112680 2024-09-12T02:11:36.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.458+0000 7f2eeffff640 1 -- 172.21.15.16:0/538951856 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f0801d050 con 0x5587a1112680 2024-09-12T02:11:36.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.459+0000 7f2eeffff640 1 -- 172.21.15.16:0/538951856 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f2f0802c070 con 0x5587a1112680 2024-09-12T02:11:36.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.459+0000 7f2f192b6740 10 client.18351 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:11:36.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.459+0000 7f2f192b6740 10 client.18351 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:11:36.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.459+0000 7f2f192b6740 20 client.18351 populate_metadata read hostname 'smithi016' 2024-09-12T02:11:36.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.459+0000 7f2f192b6740 1 --2- 172.21.15.16:0/538951856 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5587a1122190 0x5587a11934e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:36.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.459+0000 7f2f192b6740 4 client.18351 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T02:11:36.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.459+0000 7f2f192b6740 1 -- 172.21.15.16:0/538951856 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5587a0ff1210 con 0x5587a1122190 2024-09-12T02:11:36.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.459+0000 7f2f14c79640 1 --2- 172.21.15.16:0/538951856 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5587a1122190 0x5587a11934e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:36.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.459+0000 7f2eeffff640 10 client.18351 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:11:36.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.459+0000 7f2f14c79640 1 --2- 172.21.15.16:0/538951856 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5587a1122190 0x5587a11934e0 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2c37de815f2d135 server_cookie=368d25217ecd9b70 in_seq=0 out_seq=0 2024-09-12T02:11:36.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.459+0000 7f2eeffff640 1 -- 172.21.15.16:0/538951856 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5587a0ff1210 con 0x5587a1122190 2024-09-12T02:11:36.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.459+0000 7f2eeffff640 10 client.18351 handle_command_reply: tid=0 2024-09-12T02:11:36.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.459+0000 7f2f192b6740 1 client.18351 shutdown 2024-09-12T02:11:36.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.460+0000 7f2f192b6740 1 -- 172.21.15.16:0/538951856 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587a1112680 msgr2=0x5587a1116310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:36.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.460+0000 7f2f192b6740 1 --2- 172.21.15.16:0/538951856 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587a1112680 0x5587a1116310 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f2f0800f040 tx=0x7f2f08016000 comp rx=0 tx=0).stop 2024-09-12T02:11:36.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.460+0000 7f2f192b6740 1 -- 172.21.15.16:0/538951856 shutdown_connections 2024-09-12T02:11:36.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.460+0000 7f2f192b6740 1 --2- 172.21.15.16:0/538951856 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5587a1122190 0x5587a11934e0 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:36.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.460+0000 7f2f192b6740 1 --2- 172.21.15.16:0/538951856 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587a1112680 0x5587a1116310 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:36.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.460+0000 7f2f192b6740 1 --2- 172.21.15.16:0/538951856 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5587a1107dc0 0x5587a1115dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:36.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.460+0000 7f2f192b6740 1 --2- 172.21.15.16:0/538951856 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5587a11074a0 0x5587a111aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:36.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.460+0000 7f2f192b6740 1 -- 172.21.15.16:0/538951856 >> 172.21.15.16:0/538951856 conn(0x5587a1103a30 msgr2=0x5587a1108ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:36.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.460+0000 7f2eeffff640 0 client.18351 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:11:36.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.460+0000 7f2f192b6740 1 -- 172.21.15.16:0/538951856 shutdown_connections 2024-09-12T02:11:36.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.460+0000 7f2f192b6740 1 -- 172.21.15.16:0/538951856 wait complete. 2024-09-12T02:11:36.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.460+0000 7f2f192b6740 20 client.18351 trim_cache size 0 max 16384 2024-09-12T02:11:36.465 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T02:11:36.465 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-09-12T02:11:36.465 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {} 2024-09-12T02:11:36.465 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T02:11:36.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.460+0000 7f2f16edd640 1 -- 172.21.15.16:0/955627393 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f2f000a2c00 msgr2=0x7f2f000a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:36.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.460+0000 7f2f16edd640 1 --2- 172.21.15.16:0/955627393 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f2f000a2c00 0x7f2f000a50c0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f2f0800a5a0 tx=0x7f2f08004b50 comp rx=0 tx=0).stop 2024-09-12T02:11:36.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.460+0000 7f2f16edd640 1 -- 172.21.15.16:0/955627393 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f10102790 msgr2=0x7f2f101989c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:36.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.460+0000 7f2f16edd640 1 --2- 172.21.15.16:0/955627393 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f10102790 0x7f2f101989c0 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f2efc00e9f0 tx=0x7f2efc00eeb0 comp rx=0 tx=0).stop 2024-09-12T02:11:36.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.461+0000 7f2f16edd640 1 -- 172.21.15.16:0/955627393 shutdown_connections 2024-09-12T02:11:36.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.461+0000 7f2f16edd640 1 --2- 172.21.15.16:0/955627393 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f2f000a2c00 0x7f2f000a50c0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:36.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.461+0000 7f2f16edd640 1 --2- 172.21.15.16:0/955627393 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2f10106f10 0x7f2f1019cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:36.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.461+0000 7f2f16edd640 1 --2- 172.21.15.16:0/955627393 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f10102790 0x7f2f101989c0 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:36.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.461+0000 7f2f16edd640 1 --2- 172.21.15.16:0/955627393 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f10101de0 0x7f2f10198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:36.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.461+0000 7f2f16edd640 1 -- 172.21.15.16:0/955627393 >> 172.21.15.16:0/955627393 conn(0x7f2f100fd8c0 msgr2=0x7f2f100fecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:36.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.461+0000 7f2f16edd640 1 -- 172.21.15.16:0/955627393 shutdown_connections 2024-09-12T02:11:36.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:36.461+0000 7f2f16edd640 1 -- 172.21.15.16:0/955627393 wait complete. 2024-09-12T02:11:36.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:11:36 smithi184 ceph-mon[35593]: pgmap v625: 97 pgs: 97 active+clean; 1.1 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 4.7 KiB/s wr, 9 op/s 2024-09-12T02:11:36.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:11:36 smithi033 ceph-mon[35332]: pgmap v625: 97 pgs: 97 active+clean; 1.1 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 4.7 KiB/s wr, 9 op/s 2024-09-12T02:11:36.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:11:36 smithi016 ceph-mon[30947]: pgmap v625: 97 pgs: 97 active+clean; 1.1 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 4.7 KiB/s wr, 9 op/s 2024-09-12T02:11:37.236 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-12T02:11:37.236 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:11:37.497 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:11:37.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:37.965+0000 7f4195fca640 1 -- 172.21.15.16:0/3920327718 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4190106f10 msgr2=0x7f419010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:37.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:37.965+0000 7f4195fca640 1 --2- 172.21.15.16:0/3920327718 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4190106f10 0x7f419010ea90 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f4188009ca0 tx=0x7f418802a8c0 comp rx=0 tx=0).stop 2024-09-12T02:11:37.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:37.965+0000 7f4195fca640 1 -- 172.21.15.16:0/3920327718 shutdown_connections 2024-09-12T02:11:37.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:37.965+0000 7f4195fca640 1 --2- 172.21.15.16:0/3920327718 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4190106f10 0x7f419010ea90 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:37.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:37.965+0000 7f4195fca640 1 --2- 172.21.15.16:0/3920327718 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4190100ca0 0x7f41901067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:37.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:37.965+0000 7f4195fca640 1 --2- 172.21.15.16:0/3920327718 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f41901002f0 0x7f41901006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:37.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:37.965+0000 7f4195fca640 1 -- 172.21.15.16:0/3920327718 >> 172.21.15.16:0/3920327718 conn(0x7f41900fbee0 msgr2=0x7f41900fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:37.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:37.965+0000 7f4195fca640 1 -- 172.21.15.16:0/3920327718 shutdown_connections 2024-09-12T02:11:37.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:37.965+0000 7f4195fca640 1 -- 172.21.15.16:0/3920327718 wait complete. 2024-09-12T02:11:37.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:37.966+0000 7f4195fca640 1 Processor -- start 2024-09-12T02:11:37.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:37.966+0000 7f4195fca640 1 -- start start 2024-09-12T02:11:37.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:37.966+0000 7f4195fca640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41901002f0 0x7f41901983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:37.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:37.966+0000 7f4195fca640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4190100ca0 0x7f4190198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:37.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:37.966+0000 7f4195fca640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4190106f10 0x7f419019cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:37.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:37.966+0000 7f4195fca640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4190111f70 con 0x7f41901002f0 2024-09-12T02:11:37.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:37.966+0000 7f4195fca640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4190111df0 con 0x7f4190106f10 2024-09-12T02:11:37.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:37.966+0000 7f4195fca640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f41901120f0 con 0x7f4190100ca0 2024-09-12T02:11:37.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:37.966+0000 7f418f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41901002f0 0x7f41901983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:37.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:37.967+0000 7f418f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41901002f0 0x7f41901983d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52180/0 (socket says 172.21.15.16:52180) 2024-09-12T02:11:37.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:37.967+0000 7f418f7fe640 1 -- 172.21.15.16:0/387131407 learned_addr learned my addr 172.21.15.16:0/387131407 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:11:37.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:37.967+0000 7f418effd640 1 --2- 172.21.15.16:0/387131407 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4190100ca0 0x7f4190198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:37.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:37.967+0000 7f418ffff640 1 --2- 172.21.15.16:0/387131407 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4190106f10 0x7f419019cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:37.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:37.967+0000 7f418f7fe640 1 -- 172.21.15.16:0/387131407 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4190100ca0 msgr2=0x7f4190198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:37.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:37.967+0000 7f418f7fe640 1 --2- 172.21.15.16:0/387131407 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4190100ca0 0x7f4190198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:37.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:37.967+0000 7f418f7fe640 1 -- 172.21.15.16:0/387131407 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4190106f10 msgr2=0x7f419019cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:37.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:37.967+0000 7f418f7fe640 1 --2- 172.21.15.16:0/387131407 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4190106f10 0x7f419019cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:37.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:37.967+0000 7f418f7fe640 1 -- 172.21.15.16:0/387131407 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f419019d380 con 0x7f41901002f0 2024-09-12T02:11:37.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:37.967+0000 7f418f7fe640 1 --2- 172.21.15.16:0/387131407 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41901002f0 0x7f41901983d0 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f418000c890 tx=0x7f418000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:37.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:37.967+0000 7f418cff9640 1 -- 172.21.15.16:0/387131407 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4180004220 con 0x7f41901002f0 2024-09-12T02:11:37.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:37.967+0000 7f418cff9640 1 -- 172.21.15.16:0/387131407 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41800043c0 con 0x7f41901002f0 2024-09-12T02:11:37.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:37.967+0000 7f4195fca640 1 -- 172.21.15.16:0/387131407 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f419019d670 con 0x7f41901002f0 2024-09-12T02:11:37.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:37.967+0000 7f418cff9640 1 -- 172.21.15.16:0/387131407 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41800049e0 con 0x7f41901002f0 2024-09-12T02:11:37.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:37.968+0000 7f4195fca640 1 -- 172.21.15.16:0/387131407 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41901a4f50 con 0x7f41901002f0 2024-09-12T02:11:37.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:37.969+0000 7f4195fca640 1 -- 172.21.15.16:0/387131407 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41901006d0 con 0x7f41901002f0 2024-09-12T02:11:37.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:37.970+0000 7f418cff9640 1 -- 172.21.15.16:0/387131407 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f41800079b0 con 0x7f41901002f0 2024-09-12T02:11:37.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:37.970+0000 7f418cff9640 1 --2- 172.21.15.16:0/387131407 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f41780a2b80 0x7f41780a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:37.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:37.971+0000 7f418cff9640 1 -- 172.21.15.16:0/387131407 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f41800cef50 con 0x7f41901002f0 2024-09-12T02:11:37.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:37.971+0000 7f418effd640 1 --2- 172.21.15.16:0/387131407 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f41780a2b80 0x7f41780a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:37.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:37.971+0000 7f418effd640 1 --2- 172.21.15.16:0/387131407 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f41780a2b80 0x7f41780a5040 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f41900fc790 tx=0x7f418400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:37.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:37.975+0000 7f418cff9640 1 -- 172.21.15.16:0/387131407 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4180014030 con 0x7f41901002f0 2024-09-12T02:11:38.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:38.185+0000 7f4195fca640 1 -- 172.21.15.16:0/387131407 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4190199730 con 0x7f41901002f0 2024-09-12T02:11:38.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:38.186+0000 7f418cff9640 1 -- 172.21.15.16:0/387131407 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11911 (secure 0 0 0) 0x7f418009b900 con 0x7f41901002f0 2024-09-12T02:11:38.186 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T02:11:38.186 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":25,"btime":"2024-09-12T02:01:37:192021+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3279123279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3279123279},{"type":"v1","addr":"172.21.15.16:6835","nonce":3279123279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4119452382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4119452382},{"type":"v1","addr":"172.21.15.16:6837","nonce":4119452382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/3263823165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3263823165},{"type":"v1","addr":"172.21.15.16:6839","nonce":3263823165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/1510290732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":1510290732},{"type":"v1","addr":"172.21.15.16:6841","nonce":1510290732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/279674603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":279674603},{"type":"v1","addr":"172.21.15.33:6833","nonce":279674603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3555728028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3555728028},{"type":"v1","addr":"172.21.15.33:6835","nonce":3555728028}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T01:59:56.117608+0000","modified":"2024-09-12T02:01:36.294760+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24482,"mds_1":24481,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6833/1563374106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1563374106},{"type":"v1","addr":"172.21.15.184:6833","nonce":1563374106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6837/1892833868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1892833868},{"type":"v1","addr":"172.21.15.33:6837","nonce":1892833868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/1457679686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1457679686},{"type":"v1","addr":"172.21.15.184:6835","nonce":1457679686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.184:6837/1793393087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":1793393087},{"type":"v1","addr":"172.21.15.184:6837","nonce":1793393087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.184:6839/4203347941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":4203347941},{"type":"v1","addr":"172.21.15.184:6839","nonce":4203347941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.33:6839/4136670204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":4136670204},{"type":"v1","addr":"172.21.15.33:6839","nonce":4136670204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24481,24482,24469]},"id":1}]} 2024-09-12T02:11:38.187 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 25 2024-09-12T02:11:38.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:38.188+0000 7f4195fca640 1 -- 172.21.15.16:0/387131407 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f41780a2b80 msgr2=0x7f41780a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:38.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:38.188+0000 7f4195fca640 1 --2- 172.21.15.16:0/387131407 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f41780a2b80 0x7f41780a5040 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f41900fc790 tx=0x7f418400b390 comp rx=0 tx=0).stop 2024-09-12T02:11:38.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:38.188+0000 7f4195fca640 1 -- 172.21.15.16:0/387131407 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41901002f0 msgr2=0x7f41901983d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:38.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:38.188+0000 7f4195fca640 1 --2- 172.21.15.16:0/387131407 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41901002f0 0x7f41901983d0 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f418000c890 tx=0x7f418000cd50 comp rx=0 tx=0).stop 2024-09-12T02:11:38.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:38.188+0000 7f4195fca640 1 -- 172.21.15.16:0/387131407 shutdown_connections 2024-09-12T02:11:38.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:38.188+0000 7f4195fca640 1 --2- 172.21.15.16:0/387131407 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f41780a2b80 0x7f41780a5040 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:38.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:38.188+0000 7f4195fca640 1 --2- 172.21.15.16:0/387131407 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4190106f10 0x7f419019cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:38.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:38.188+0000 7f4195fca640 1 --2- 172.21.15.16:0/387131407 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4190100ca0 0x7f4190198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:38.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:38.188+0000 7f4195fca640 1 --2- 172.21.15.16:0/387131407 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41901002f0 0x7f41901983d0 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:38.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:38.188+0000 7f4195fca640 1 -- 172.21.15.16:0/387131407 >> 172.21.15.16:0/387131407 conn(0x7f41900fbee0 msgr2=0x7f41900fd1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:38.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:38.188+0000 7f4195fca640 1 -- 172.21.15.16:0/387131407 shutdown_connections 2024-09-12T02:11:38.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:38.188+0000 7f4195fca640 1 -- 172.21.15.16:0/387131407 wait complete. 2024-09-12T02:11:38.510 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:11:38 smithi016 ceph-mon[30947]: pgmap v626: 97 pgs: 97 active+clean; 1.1 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 6.0 KiB/s wr, 10 op/s 2024-09-12T02:11:38.510 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:11:38 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/387131407' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:11:38.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:11:38 smithi184 ceph-mon[35593]: pgmap v626: 97 pgs: 97 active+clean; 1.1 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 6.0 KiB/s wr, 10 op/s 2024-09-12T02:11:38.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:11:38 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/387131407' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:11:38.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:11:38 smithi033 ceph-mon[35332]: pgmap v626: 97 pgs: 97 active+clean; 1.1 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 6.0 KiB/s wr, 10 op/s 2024-09-12T02:11:38.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:11:38 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/387131407' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:11:38.927 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-12T02:11:39.186 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:11:39.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.691+0000 7fefaa44c640 1 -- 172.21.15.16:0/2510290950 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fefa4101420 msgr2=0x7fefa410d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:39.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.691+0000 7fefaa44c640 1 --2- 172.21.15.16:0/2510290950 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fefa4101420 0x7fefa410d950 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7fef9c009cd0 tx=0x7fef9c02a8e0 comp rx=0 tx=0).stop 2024-09-12T02:11:39.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.691+0000 7fefaa44c640 1 -- 172.21.15.16:0/2510290950 shutdown_connections 2024-09-12T02:11:39.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.691+0000 7fefaa44c640 1 --2- 172.21.15.16:0/2510290950 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fefa4101420 0x7fefa410d950 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:39.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.691+0000 7fefaa44c640 1 --2- 172.21.15.16:0/2510290950 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fefa4100a80 0x7fefa4100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:39.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.691+0000 7fefaa44c640 1 --2- 172.21.15.16:0/2510290950 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fefa4106a80 0x7fefa4106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:39.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.691+0000 7fefaa44c640 1 -- 172.21.15.16:0/2510290950 >> 172.21.15.16:0/2510290950 conn(0x7fefa40fc7a0 msgr2=0x7fefa40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:39.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.691+0000 7fefaa44c640 1 -- 172.21.15.16:0/2510290950 shutdown_connections 2024-09-12T02:11:39.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.691+0000 7fefaa44c640 1 -- 172.21.15.16:0/2510290950 wait complete. 2024-09-12T02:11:39.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.692+0000 7fefaa44c640 1 Processor -- start 2024-09-12T02:11:39.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.692+0000 7fefaa44c640 1 -- start start 2024-09-12T02:11:39.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.692+0000 7fefaa44c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fefa4100a80 0x7fefa41992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:39.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.692+0000 7fefaa44c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fefa4101420 0x7fefa4199830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:39.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.692+0000 7fefaa44c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fefa4106a80 0x7fefa41a4370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:39.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.692+0000 7fefaa44c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fefa4110ea0 con 0x7fefa4106a80 2024-09-12T02:11:39.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.692+0000 7fefaa44c640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fefa4110d20 con 0x7fefa4101420 2024-09-12T02:11:39.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.692+0000 7fefaa44c640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fefa4111020 con 0x7fefa4100a80 2024-09-12T02:11:39.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.692+0000 7fefa89e9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fefa4106a80 0x7fefa41a4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:39.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.692+0000 7fefa89e9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fefa4106a80 0x7fefa41a4370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52212/0 (socket says 172.21.15.16:52212) 2024-09-12T02:11:39.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.692+0000 7fefa89e9640 1 -- 172.21.15.16:0/3841394677 learned_addr learned my addr 172.21.15.16:0/3841394677 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:11:39.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.692+0000 7fefa37fe640 1 --2- 172.21.15.16:0/3841394677 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fefa4101420 0x7fefa4199830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:39.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.692+0000 7fefa3fff640 1 --2- 172.21.15.16:0/3841394677 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fefa4100a80 0x7fefa41992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:39.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.693+0000 7fefa89e9640 1 -- 172.21.15.16:0/3841394677 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fefa4100a80 msgr2=0x7fefa41992f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:39.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.693+0000 7fefa89e9640 1 --2- 172.21.15.16:0/3841394677 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fefa4100a80 0x7fefa41992f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:39.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.693+0000 7fefa89e9640 1 -- 172.21.15.16:0/3841394677 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fefa4101420 msgr2=0x7fefa4199830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:39.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.693+0000 7fefa89e9640 1 --2- 172.21.15.16:0/3841394677 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fefa4101420 0x7fefa4199830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:39.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.693+0000 7fefa89e9640 1 -- 172.21.15.16:0/3841394677 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefa41a48b0 con 0x7fefa4106a80 2024-09-12T02:11:39.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.693+0000 7fefa89e9640 1 --2- 172.21.15.16:0/3841394677 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fefa4106a80 0x7fefa41a4370 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7fef9c005cc0 tx=0x7fef9c003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:39.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.693+0000 7fefa17fa640 1 -- 172.21.15.16:0/3841394677 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef9c004040 con 0x7fefa4106a80 2024-09-12T02:11:39.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.693+0000 7fefaa44c640 1 -- 172.21.15.16:0/3841394677 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefa41a4a80 con 0x7fefa4106a80 2024-09-12T02:11:39.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.693+0000 7fefa17fa640 1 -- 172.21.15.16:0/3841394677 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef9c0046c0 con 0x7fefa4106a80 2024-09-12T02:11:39.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.693+0000 7fefa17fa640 1 -- 172.21.15.16:0/3841394677 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef9c030e70 con 0x7fefa4106a80 2024-09-12T02:11:39.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.693+0000 7fefaa44c640 1 -- 172.21.15.16:0/3841394677 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefa41a4f30 con 0x7fefa4106a80 2024-09-12T02:11:39.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.696+0000 7fefa17fa640 1 -- 172.21.15.16:0/3841394677 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fef9c0305a0 con 0x7fefa4106a80 2024-09-12T02:11:39.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.696+0000 7fefac825740 1 Processor -- start 2024-09-12T02:11:39.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.696+0000 7fefac825740 1 -- start start 2024-09-12T02:11:39.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.696+0000 7fefac825740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555ab94bdb40 0x555ab94bc090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:39.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.696+0000 7fefac825740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555ab94bc5d0 0x555ab94bca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:39.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.696+0000 7fefac825740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ab94c7370 0x555ab94c9760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:39.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.696+0000 7fefac825740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x555ab94222f0 con 0x555ab94c7370 2024-09-12T02:11:39.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.696+0000 7fefac825740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x555ab93c9cf0 con 0x555ab94bdb40 2024-09-12T02:11:39.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.696+0000 7fefac825740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x555ab941e2b0 con 0x555ab94bc5d0 2024-09-12T02:11:39.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.696+0000 7fefa37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ab94c7370 0x555ab94c9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:39.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.697+0000 7fefa37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ab94c7370 0x555ab94c9760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52218/0 (socket says 172.21.15.16:52218) 2024-09-12T02:11:39.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.697+0000 7fefa37fe640 1 -- 172.21.15.16:0/3246717953 learned_addr learned my addr 172.21.15.16:0/3246717953 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:11:39.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.697+0000 7fefa17fa640 1 --2- 172.21.15.16:0/3841394677 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fef8c0a2b80 0x7fef8c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:39.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.697+0000 7fefa37fe640 1 --2- 172.21.15.16:0/3246717953 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555ab94bdb40 0x555ab94bc090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:39.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.697+0000 7fefa3fff640 1 --2- 172.21.15.16:0/3246717953 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555ab94bc5d0 0x555ab94bca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:39.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.697+0000 7fefa89e9640 1 --2- 172.21.15.16:0/3841394677 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fef8c0a2b80 0x7fef8c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:39.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.697+0000 7fefa17fa640 1 -- 172.21.15.16:0/3841394677 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fef9c02e030 con 0x7fefa4106a80 2024-09-12T02:11:39.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.697+0000 7fefa37fe640 1 -- 172.21.15.16:0/3246717953 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555ab94bc5d0 msgr2=0x555ab94bca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:39.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.697+0000 7fefa37fe640 1 --2- 172.21.15.16:0/3246717953 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555ab94bc5d0 0x555ab94bca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:39.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.697+0000 7fefa37fe640 1 -- 172.21.15.16:0/3246717953 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555ab94bdb40 msgr2=0x555ab94bc090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:39.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.697+0000 7fefa37fe640 1 --2- 172.21.15.16:0/3246717953 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555ab94bdb40 0x555ab94bc090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:39.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.697+0000 7fefa37fe640 1 -- 172.21.15.16:0/3246717953 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ab93ae3a0 con 0x555ab94c7370 2024-09-12T02:11:39.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.697+0000 7fefa3fff640 1 --2- 172.21.15.16:0/3246717953 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555ab94bc5d0 0x555ab94bca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:11:39.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.697+0000 7fefa37fe640 1 --2- 172.21.15.16:0/3246717953 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ab94c7370 0x555ab94c9760 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7fef980127c0 tx=0x7fef9800b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2570e3b79764a01 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:39.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.697+0000 7fefa89e9640 1 --2- 172.21.15.16:0/3841394677 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fef8c0a2b80 0x7fef8c0a5040 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fef9c0bc8a0 tx=0x7fef9c08b890 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:39.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.697+0000 7fef8a7fc640 1 -- 172.21.15.16:0/3246717953 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef98012e00 con 0x555ab94c7370 2024-09-12T02:11:39.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.697+0000 7fef8a7fc640 1 -- 172.21.15.16:0/3246717953 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef98010c50 con 0x555ab94c7370 2024-09-12T02:11:39.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.697+0000 7fef8a7fc640 1 -- 172.21.15.16:0/3246717953 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef9801eb30 con 0x555ab94c7370 2024-09-12T02:11:39.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.698+0000 7fefac825740 1 -- 172.21.15.16:0/3246717953 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ab94c7370 msgr2=0x555ab94c9760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:39.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.698+0000 7fefac825740 1 --2- 172.21.15.16:0/3246717953 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ab94c7370 0x555ab94c9760 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7fef980127c0 tx=0x7fef9800b7c0 comp rx=0 tx=0).stop 2024-09-12T02:11:39.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.698+0000 7fefac825740 1 -- 172.21.15.16:0/3246717953 shutdown_connections 2024-09-12T02:11:39.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.698+0000 7fefac825740 1 --2- 172.21.15.16:0/3246717953 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ab94c7370 0x555ab94c9760 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:39.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.698+0000 7fefac825740 1 --2- 172.21.15.16:0/3246717953 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555ab94bc5d0 0x555ab94bca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:39.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.698+0000 7fefac825740 1 --2- 172.21.15.16:0/3246717953 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555ab94bdb40 0x555ab94bc090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:39.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.698+0000 7fefac825740 1 -- 172.21.15.16:0/3246717953 >> 172.21.15.16:0/3246717953 conn(0x555ab9485310 msgr2=0x555ab94b8c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:39.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.698+0000 7fefac825740 1 -- 172.21.15.16:0/3246717953 shutdown_connections 2024-09-12T02:11:39.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.698+0000 7fefac825740 1 -- 172.21.15.16:0/3246717953 wait complete. 2024-09-12T02:11:39.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.699+0000 7fefac825740 1 -- start start 2024-09-12T02:11:39.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.699+0000 7fefac825740 1 Processor -- start 2024-09-12T02:11:39.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.699+0000 7fefac825740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555ab94bc5d0 0x555ab94c6d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:39.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.699+0000 7fefac825740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555ab94bdb40 0x555ab94c3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:39.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.699+0000 7fefac825740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ab94c7370 0x555ab94c4570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:39.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.699+0000 7fefac825740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x555ab94ca070 con 0x555ab94c7370 2024-09-12T02:11:39.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.699+0000 7fefac825740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x555ab93dd060 con 0x555ab94bc5d0 2024-09-12T02:11:39.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.699+0000 7fefac825740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x555ab94ca1f0 con 0x555ab94bdb40 2024-09-12T02:11:39.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.699+0000 7fefac825740 10 client.0 _ll_register_callbacks cb 0x555ab91308c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:11:39.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.699+0000 7fefa37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ab94c7370 0x555ab94c4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:39.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.699+0000 7fefa37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ab94c7370 0x555ab94c4570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52230/0 (socket says 172.21.15.16:52230) 2024-09-12T02:11:39.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.699+0000 7fefa37fe640 1 -- 172.21.15.16:0/2951693162 learned_addr learned my addr 172.21.15.16:0/2951693162 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:11:39.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.700+0000 7fefa37fe640 1 --2- 172.21.15.16:0/2951693162 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555ab94bc5d0 0x555ab94c6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:39.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.700+0000 7fefa3fff640 1 --2- 172.21.15.16:0/2951693162 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555ab94bdb40 0x555ab94c3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:39.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.700+0000 7fefa37fe640 1 -- 172.21.15.16:0/2951693162 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555ab94bdb40 msgr2=0x555ab94c3fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:39.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.700+0000 7fefa37fe640 1 --2- 172.21.15.16:0/2951693162 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555ab94bdb40 0x555ab94c3fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:39.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.700+0000 7fefa37fe640 1 -- 172.21.15.16:0/2951693162 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555ab94bc5d0 msgr2=0x555ab94c6d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:39.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.700+0000 7fefa37fe640 1 --2- 172.21.15.16:0/2951693162 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555ab94bc5d0 0x555ab94c6d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:39.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.700+0000 7fefa37fe640 1 -- 172.21.15.16:0/2951693162 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ab91933c0 con 0x555ab94c7370 2024-09-12T02:11:39.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.700+0000 7fefa37fe640 1 --2- 172.21.15.16:0/2951693162 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ab94c7370 0x555ab94c4570 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7fef98001eb0 tx=0x7fef98014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fda2c6bf9cdd632 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:39.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.700+0000 7fef837fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:11:39.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.700+0000 7fef837fe640 1 -- 172.21.15.16:0/2951693162 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef98024040 con 0x555ab94c7370 2024-09-12T02:11:39.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.700+0000 7fefac825740 1 -- 172.21.15.16:0/2951693162 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef98010c50 con 0x555ab94c7370 2024-09-12T02:11:39.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.700+0000 7fef837fe640 1 -- 172.21.15.16:0/2951693162 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fef9801e8b0 con 0x555ab94c7370 2024-09-12T02:11:39.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.700+0000 7fef837fe640 1 -- 172.21.15.16:0/2951693162 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fef98010e00 con 0x555ab94c7370 2024-09-12T02:11:39.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.700+0000 7fefac825740 10 client.18369 fetch_fsmap learned FSMap version 25 2024-09-12T02:11:39.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.701+0000 7fefac825740 1 -- 172.21.15.16:0/2951693162 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555ab94d5b20 con 0x555ab94c7370 2024-09-12T02:11:39.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.701+0000 7fef837fe640 1 -- 172.21.15.16:0/2951693162 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef98010c50 con 0x555ab94c7370 2024-09-12T02:11:39.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.701+0000 7fef837fe640 1 -- 172.21.15.16:0/2951693162 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fef9802a040 con 0x555ab94c7370 2024-09-12T02:11:39.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.701+0000 7fefac825740 10 client.18369 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:11:39.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.701+0000 7fefac825740 10 client.18369 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-12T02:11:39.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.701+0000 7fefac825740 20 client.18369 populate_metadata read hostname 'smithi016' 2024-09-12T02:11:39.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.701+0000 7fefac825740 1 --2- 172.21.15.16:0/2951693162 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x555ab94d6f80 0x555ab94d9370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:39.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.701+0000 7fefac825740 4 client.18369 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:11:39.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.701+0000 7fefac825740 1 -- 172.21.15.16:0/2951693162 --> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555ab942f860 con 0x555ab94d6f80 2024-09-12T02:11:39.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.701+0000 7fefa3fff640 1 --2- 172.21.15.16:0/2951693162 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x555ab94d6f80 0x555ab94d9370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:39.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.702+0000 7fefa3fff640 1 --2- 172.21.15.16:0/2951693162 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x555ab94d6f80 0x555ab94d9370 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7498eea67655ef79 server_cookie=f67d4c7c5e5fa076 in_seq=0 out_seq=0 2024-09-12T02:11:39.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.702+0000 7fef837fe640 10 client.18369 ms_handle_connect on v2:172.21.15.184:6836/1793393087 2024-09-12T02:11:39.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.702+0000 7fef837fe640 1 -- 172.21.15.16:0/2951693162 <== mds.2 v2:172.21.15.184:6836/1793393087 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555ab942f860 con 0x555ab94d6f80 2024-09-12T02:11:39.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.702+0000 7fef837fe640 10 client.18369 handle_command_reply: tid=0 2024-09-12T02:11:39.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.702+0000 7fefac825740 1 client.18369 shutdown 2024-09-12T02:11:39.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.703+0000 7fefac825740 1 -- 172.21.15.16:0/2951693162 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ab94c7370 msgr2=0x555ab94c4570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:39.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.703+0000 7fefac825740 1 --2- 172.21.15.16:0/2951693162 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ab94c7370 0x555ab94c4570 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7fef98001eb0 tx=0x7fef98014040 comp rx=0 tx=0).stop 2024-09-12T02:11:39.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.703+0000 7fefac825740 1 -- 172.21.15.16:0/2951693162 shutdown_connections 2024-09-12T02:11:39.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.703+0000 7fefac825740 1 --2- 172.21.15.16:0/2951693162 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x555ab94d6f80 0x555ab94d9370 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:39.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.703+0000 7fefac825740 1 --2- 172.21.15.16:0/2951693162 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ab94c7370 0x555ab94c4570 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:39.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.703+0000 7fefac825740 1 --2- 172.21.15.16:0/2951693162 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555ab94bdb40 0x555ab94c3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:39.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.703+0000 7fefac825740 1 --2- 172.21.15.16:0/2951693162 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555ab94bc5d0 0x555ab94c6d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:39.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.703+0000 7fefac825740 1 -- 172.21.15.16:0/2951693162 >> 172.21.15.16:0/2951693162 conn(0x555ab9485310 msgr2=0x555ab94be960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:39.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.703+0000 7fef837fe640 0 client.18369 ms_handle_reset on v2:172.21.15.184:6836/1793393087 2024-09-12T02:11:39.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.703+0000 7fefac825740 1 -- 172.21.15.16:0/2951693162 shutdown_connections 2024-09-12T02:11:39.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.703+0000 7fefac825740 1 -- 172.21.15.16:0/2951693162 wait complete. 2024-09-12T02:11:39.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.703+0000 7fefac825740 20 client.18369 trim_cache size 0 max 16384 2024-09-12T02:11:39.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.716+0000 7fefac825740 1 Processor -- start 2024-09-12T02:11:39.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.716+0000 7fefac825740 1 -- start start 2024-09-12T02:11:39.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.716+0000 7fefac825740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555ab94bc5d0 0x555ab94d3370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:39.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.716+0000 7fefac825740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555ab94bdb40 0x555ab94c2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:39.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.716+0000 7fefac825740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ab94c7370 0x555ab94bf120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:39.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.716+0000 7fefac825740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fefa41afca0 con 0x555ab94c7370 2024-09-12T02:11:39.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.716+0000 7fefac825740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fefa41ad570 con 0x555ab94bdb40 2024-09-12T02:11:39.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.716+0000 7fefac825740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fefa41ab3c0 con 0x555ab94bc5d0 2024-09-12T02:11:39.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.716+0000 7fefa37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ab94c7370 0x555ab94bf120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:39.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.716+0000 7fefa37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ab94c7370 0x555ab94bf120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52240/0 (socket says 172.21.15.16:52240) 2024-09-12T02:11:39.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.716+0000 7fefa37fe640 1 -- 172.21.15.16:0/3029083754 learned_addr learned my addr 172.21.15.16:0/3029083754 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:11:39.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.717+0000 7fefa37fe640 1 --2- 172.21.15.16:0/3029083754 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555ab94bc5d0 0x555ab94d3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:39.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.717+0000 7fefa3fff640 1 --2- 172.21.15.16:0/3029083754 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555ab94bdb40 0x555ab94c2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:39.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.717+0000 7fefa37fe640 1 -- 172.21.15.16:0/3029083754 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555ab94bc5d0 msgr2=0x555ab94d3370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:39.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.717+0000 7fefa37fe640 1 --2- 172.21.15.16:0/3029083754 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555ab94bc5d0 0x555ab94d3370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:39.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.717+0000 7fefa37fe640 1 -- 172.21.15.16:0/3029083754 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555ab94bdb40 msgr2=0x555ab94c2fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:39.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.717+0000 7fefa37fe640 1 --2- 172.21.15.16:0/3029083754 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555ab94bdb40 0x555ab94c2fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:39.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.717+0000 7fefa37fe640 1 -- 172.21.15.16:0/3029083754 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ab94b8de0 con 0x555ab94c7370 2024-09-12T02:11:39.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.717+0000 7fefa3fff640 1 --2- 172.21.15.16:0/3029083754 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555ab94bdb40 0x555ab94c2fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:11:39.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.717+0000 7fefa37fe640 1 --2- 172.21.15.16:0/3029083754 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ab94c7370 0x555ab94bf120 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7fef9802dda0 tx=0x7fef98012470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=655d8671a8d6a187 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:39.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.717+0000 7fef89ffb640 1 -- 172.21.15.16:0/3029083754 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef98010c50 con 0x555ab94c7370 2024-09-12T02:11:39.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.717+0000 7fef89ffb640 1 -- 172.21.15.16:0/3029083754 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef98016040 con 0x555ab94c7370 2024-09-12T02:11:39.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.717+0000 7fef89ffb640 1 -- 172.21.15.16:0/3029083754 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef98005290 con 0x555ab94c7370 2024-09-12T02:11:39.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.718+0000 7fefac825740 1 -- 172.21.15.16:0/3029083754 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ab94c7370 msgr2=0x555ab94bf120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:39.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.718+0000 7fefac825740 1 --2- 172.21.15.16:0/3029083754 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ab94c7370 0x555ab94bf120 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7fef9802dda0 tx=0x7fef98012470 comp rx=0 tx=0).stop 2024-09-12T02:11:39.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.718+0000 7fefac825740 1 -- 172.21.15.16:0/3029083754 shutdown_connections 2024-09-12T02:11:39.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.718+0000 7fefac825740 1 --2- 172.21.15.16:0/3029083754 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ab94c7370 0x555ab94bf120 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:39.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.718+0000 7fefac825740 1 --2- 172.21.15.16:0/3029083754 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555ab94bdb40 0x555ab94c2fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:39.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.718+0000 7fefac825740 1 --2- 172.21.15.16:0/3029083754 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555ab94bc5d0 0x555ab94d3370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:39.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.718+0000 7fefac825740 1 -- 172.21.15.16:0/3029083754 >> 172.21.15.16:0/3029083754 conn(0x555ab9485310 msgr2=0x555ab94c28a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:39.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.718+0000 7fefac825740 1 -- 172.21.15.16:0/3029083754 shutdown_connections 2024-09-12T02:11:39.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.718+0000 7fefac825740 1 -- 172.21.15.16:0/3029083754 wait complete. 2024-09-12T02:11:39.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.718+0000 7fefac825740 1 -- start start 2024-09-12T02:11:39.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.718+0000 7fefac825740 1 Processor -- start 2024-09-12T02:11:39.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.719+0000 7fefac825740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555ab94bc5d0 0x555ab94ca4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:39.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.719+0000 7fefac825740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ab94bdb40 0x555ab94ca9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:39.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.719+0000 7fefac825740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555ab94c7370 0x555ab94caf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:39.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.719+0000 7fefac825740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fefa41aba30 con 0x555ab94bdb40 2024-09-12T02:11:39.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.719+0000 7fefac825740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fefa41acea0 con 0x555ab94bc5d0 2024-09-12T02:11:39.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.719+0000 7fefac825740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fefa41ac100 con 0x555ab94c7370 2024-09-12T02:11:39.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.719+0000 7fefac825740 10 client.0 _ll_register_callbacks cb 0x7fef9800be90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:11:39.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.719+0000 7fefa3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ab94bdb40 0x555ab94ca9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:39.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.719+0000 7fefa3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ab94bdb40 0x555ab94ca9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52256/0 (socket says 172.21.15.16:52256) 2024-09-12T02:11:39.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.719+0000 7fefa3fff640 1 -- 172.21.15.16:0/1353118288 learned_addr learned my addr 172.21.15.16:0/1353118288 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:11:39.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.719+0000 7fefa37fe640 1 --2- 172.21.15.16:0/1353118288 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555ab94bc5d0 0x555ab94ca4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:39.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.719+0000 7fefa37fe640 1 --2- 172.21.15.16:0/1353118288 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555ab94c7370 0x555ab94caf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:39.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.719+0000 7fefa3fff640 1 -- 172.21.15.16:0/1353118288 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555ab94c7370 msgr2=0x555ab94caf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:39.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.719+0000 7fefa3fff640 1 --2- 172.21.15.16:0/1353118288 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555ab94c7370 0x555ab94caf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:39.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.719+0000 7fefa3fff640 1 -- 172.21.15.16:0/1353118288 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555ab94bc5d0 msgr2=0x555ab94ca4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:39.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.719+0000 7fefa3fff640 1 --2- 172.21.15.16:0/1353118288 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555ab94bc5d0 0x555ab94ca4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:39.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.719+0000 7fefa3fff640 1 -- 172.21.15.16:0/1353118288 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ab953fbd0 con 0x555ab94bdb40 2024-09-12T02:11:39.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.719+0000 7fefa3fff640 1 --2- 172.21.15.16:0/1353118288 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ab94bdb40 0x555ab94ca9e0 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7fef94004810 tx=0x7fef94018820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=403410b0b3986cf9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:39.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.719+0000 7fef837fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:11:39.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.719+0000 7fef837fe640 1 -- 172.21.15.16:0/1353118288 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef940180d0 con 0x555ab94bdb40 2024-09-12T02:11:39.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.719+0000 7fef837fe640 1 -- 172.21.15.16:0/1353118288 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fef94016430 con 0x555ab94bdb40 2024-09-12T02:11:39.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.719+0000 7fefac825740 1 -- 172.21.15.16:0/1353118288 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef98016040 con 0x555ab94bdb40 2024-09-12T02:11:39.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.720+0000 7fef837fe640 1 -- 172.21.15.16:0/1353118288 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fef9402d050 con 0x555ab94bdb40 2024-09-12T02:11:39.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.720+0000 7fefac825740 10 client.18375 fetch_fsmap learned FSMap version 25 2024-09-12T02:11:39.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.720+0000 7fefac825740 1 -- 172.21.15.16:0/1353118288 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555ab9540e80 con 0x555ab94bdb40 2024-09-12T02:11:39.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.720+0000 7fef837fe640 1 -- 172.21.15.16:0/1353118288 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef98016040 con 0x555ab94bdb40 2024-09-12T02:11:39.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.720+0000 7fef837fe640 1 -- 172.21.15.16:0/1353118288 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fef94025620 con 0x555ab94bdb40 2024-09-12T02:11:39.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.720+0000 7fefac825740 10 client.18375 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:11:39.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.720+0000 7fefac825740 10 client.18375 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-12T02:11:39.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.720+0000 7fefac825740 20 client.18375 populate_metadata read hostname 'smithi016' 2024-09-12T02:11:39.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.720+0000 7fefac825740 1 --2- 172.21.15.16:0/1353118288 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x555ab95415e0 0x555ab9543a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:39.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.720+0000 7fefac825740 4 client.18375 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T02:11:39.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.720+0000 7fefac825740 1 -- 172.21.15.16:0/1353118288 --> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555ab9429d70 con 0x555ab95415e0 2024-09-12T02:11:39.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.720+0000 7fefa37fe640 1 --2- 172.21.15.16:0/1353118288 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x555ab95415e0 0x555ab9543a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:39.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.721+0000 7fefa37fe640 1 --2- 172.21.15.16:0/1353118288 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x555ab95415e0 0x555ab9543a00 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4279b5aaff264f37 server_cookie=8acadf8def0ca2f5 in_seq=0 out_seq=0 2024-09-12T02:11:39.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.721+0000 7fef837fe640 10 client.18375 ms_handle_connect on v2:172.21.15.184:6836/1793393087 2024-09-12T02:11:39.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.721+0000 7fef837fe640 1 -- 172.21.15.16:0/1353118288 <== mds.2 v2:172.21.15.184:6836/1793393087 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555ab9429d70 con 0x555ab95415e0 2024-09-12T02:11:39.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.721+0000 7fef837fe640 10 client.18375 handle_command_reply: tid=0 2024-09-12T02:11:39.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.721+0000 7fefac825740 1 client.18375 shutdown 2024-09-12T02:11:39.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.721+0000 7fefac825740 1 -- 172.21.15.16:0/1353118288 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ab94bdb40 msgr2=0x555ab94ca9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:39.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.721+0000 7fefac825740 1 --2- 172.21.15.16:0/1353118288 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ab94bdb40 0x555ab94ca9e0 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7fef94004810 tx=0x7fef94018820 comp rx=0 tx=0).stop 2024-09-12T02:11:39.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.721+0000 7fefac825740 1 -- 172.21.15.16:0/1353118288 shutdown_connections 2024-09-12T02:11:39.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.721+0000 7fefac825740 1 --2- 172.21.15.16:0/1353118288 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555ab94c7370 0x555ab94caf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:39.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.721+0000 7fefac825740 1 --2- 172.21.15.16:0/1353118288 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x555ab95415e0 0x555ab9543a00 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:39.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.721+0000 7fefac825740 1 --2- 172.21.15.16:0/1353118288 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ab94bdb40 0x555ab94ca9e0 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:39.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.721+0000 7fefac825740 1 --2- 172.21.15.16:0/1353118288 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555ab94bc5d0 0x555ab94ca4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:39.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.721+0000 7fefac825740 1 -- 172.21.15.16:0/1353118288 >> 172.21.15.16:0/1353118288 conn(0x555ab9485310 msgr2=0x555ab94c30c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:39.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.721+0000 7fef837fe640 0 client.18375 ms_handle_reset on v2:172.21.15.184:6836/1793393087 2024-09-12T02:11:39.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.721+0000 7fefac825740 1 -- 172.21.15.16:0/1353118288 shutdown_connections 2024-09-12T02:11:39.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.721+0000 7fefac825740 1 -- 172.21.15.16:0/1353118288 wait complete. 2024-09-12T02:11:39.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.721+0000 7fefac825740 20 client.18375 trim_cache size 0 max 16384 2024-09-12T02:11:39.727 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T02:11:39.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.722+0000 7fefaa44c640 1 -- 172.21.15.16:0/3841394677 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fef8c0a2b80 msgr2=0x7fef8c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:39.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.722+0000 7fefaa44c640 1 --2- 172.21.15.16:0/3841394677 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fef8c0a2b80 0x7fef8c0a5040 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fef9c0bc8a0 tx=0x7fef9c08b890 comp rx=0 tx=0).stop 2024-09-12T02:11:39.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.722+0000 7fefaa44c640 1 -- 172.21.15.16:0/3841394677 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fefa4106a80 msgr2=0x7fefa41a4370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:39.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.722+0000 7fefaa44c640 1 --2- 172.21.15.16:0/3841394677 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fefa4106a80 0x7fefa41a4370 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7fef9c005cc0 tx=0x7fef9c003970 comp rx=0 tx=0).stop 2024-09-12T02:11:39.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.722+0000 7fefaa44c640 1 -- 172.21.15.16:0/3841394677 shutdown_connections 2024-09-12T02:11:39.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.722+0000 7fefaa44c640 1 --2- 172.21.15.16:0/3841394677 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fef8c0a2b80 0x7fef8c0a5040 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:39.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.722+0000 7fefaa44c640 1 --2- 172.21.15.16:0/3841394677 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fefa4106a80 0x7fefa41a4370 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:39.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.722+0000 7fefaa44c640 1 --2- 172.21.15.16:0/3841394677 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fefa4101420 0x7fefa4199830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:39.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.722+0000 7fefaa44c640 1 --2- 172.21.15.16:0/3841394677 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fefa4100a80 0x7fefa41992f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:39.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.722+0000 7fefaa44c640 1 -- 172.21.15.16:0/3841394677 >> 172.21.15.16:0/3841394677 conn(0x7fefa40fc7a0 msgr2=0x7fefa40fcec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:39.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.722+0000 7fefaa44c640 1 -- 172.21.15.16:0/3841394677 shutdown_connections 2024-09-12T02:11:39.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:39.722+0000 7fefaa44c640 1 -- 172.21.15.16:0/3841394677 wait complete. 2024-09-12T02:11:40.537 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-12T02:11:40.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:11:40 smithi184 ceph-mon[35593]: pgmap v627: 97 pgs: 97 active+clean; 1.1 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.9 KiB/s wr, 6 op/s 2024-09-12T02:11:40.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:11:40 smithi033 ceph-mon[35332]: pgmap v627: 97 pgs: 97 active+clean; 1.1 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.9 KiB/s wr, 6 op/s 2024-09-12T02:11:40.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:11:40 smithi016 ceph-mon[30947]: pgmap v627: 97 pgs: 97 active+clean; 1.1 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.9 KiB/s wr, 6 op/s 2024-09-12T02:11:40.799 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:11:41.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.290+0000 7f0e5aadd640 1 -- 172.21.15.16:0/464714515 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e54103660 msgr2=0x7f0e5410bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:41.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.290+0000 7f0e5aadd640 1 --2- 172.21.15.16:0/464714515 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e54103660 0x7f0e5410bb50 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f0e40009900 tx=0x7f0e4002a480 comp rx=0 tx=0).stop 2024-09-12T02:11:41.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.290+0000 7f0e5aadd640 1 -- 172.21.15.16:0/464714515 shutdown_connections 2024-09-12T02:11:41.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.290+0000 7f0e5aadd640 1 --2- 172.21.15.16:0/464714515 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0e5410c170 0x7f0e5410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:41.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.290+0000 7f0e5aadd640 1 --2- 172.21.15.16:0/464714515 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e54103660 0x7f0e5410bb50 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:41.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.290+0000 7f0e5aadd640 1 --2- 172.21.15.16:0/464714515 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e54102d40 0x7f0e54103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:41.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.290+0000 7f0e5aadd640 1 -- 172.21.15.16:0/464714515 >> 172.21.15.16:0/464714515 conn(0x7f0e540fc7a0 msgr2=0x7f0e540febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:41.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.291+0000 7f0e5aadd640 1 -- 172.21.15.16:0/464714515 shutdown_connections 2024-09-12T02:11:41.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.291+0000 7f0e5aadd640 1 -- 172.21.15.16:0/464714515 wait complete. 2024-09-12T02:11:41.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.291+0000 7f0e5aadd640 1 Processor -- start 2024-09-12T02:11:41.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.291+0000 7f0e5aadd640 1 -- start start 2024-09-12T02:11:41.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.291+0000 7f0e5aadd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e54102d40 0x7f0e54198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:41.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.291+0000 7f0e5aadd640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0e54103660 0x7f0e541989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:41.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.291+0000 7f0e5aadd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e5410c170 0x7f0e5419cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:41.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.291+0000 7f0e5aadd640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0e54111f70 con 0x7f0e54102d40 2024-09-12T02:11:41.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.291+0000 7f0e5aadd640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0e54111df0 con 0x7f0e5410c170 2024-09-12T02:11:41.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.291+0000 7f0e5aadd640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f0e541120f0 con 0x7f0e54103660 2024-09-12T02:11:41.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.292+0000 7f0e58879640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e54102d40 0x7f0e54198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:41.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.292+0000 7f0e58879640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e54102d40 0x7f0e54198470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35824/0 (socket says 172.21.15.16:35824) 2024-09-12T02:11:41.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.292+0000 7f0e58879640 1 -- 172.21.15.16:0/850180284 learned_addr learned my addr 172.21.15.16:0/850180284 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:11:41.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.292+0000 7f0e53fff640 1 --2- 172.21.15.16:0/850180284 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0e54103660 0x7f0e541989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:41.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.292+0000 7f0e5907a640 1 --2- 172.21.15.16:0/850180284 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e5410c170 0x7f0e5419cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:41.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.292+0000 7f0e58879640 1 -- 172.21.15.16:0/850180284 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0e54103660 msgr2=0x7f0e541989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:41.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.292+0000 7f0e58879640 1 --2- 172.21.15.16:0/850180284 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0e54103660 0x7f0e541989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:41.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.292+0000 7f0e58879640 1 -- 172.21.15.16:0/850180284 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e5410c170 msgr2=0x7f0e5419cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:41.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.292+0000 7f0e58879640 1 --2- 172.21.15.16:0/850180284 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e5410c170 0x7f0e5419cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:41.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.292+0000 7f0e58879640 1 -- 172.21.15.16:0/850180284 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e5419d420 con 0x7f0e54102d40 2024-09-12T02:11:41.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.292+0000 7f0e53fff640 1 --2- 172.21.15.16:0/850180284 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0e54103660 0x7f0e541989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:11:41.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.292+0000 7f0e5907a640 1 --2- 172.21.15.16:0/850180284 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e5410c170 0x7f0e5419cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:11:41.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.292+0000 7f0e58879640 1 --2- 172.21.15.16:0/850180284 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e54102d40 0x7f0e54198470 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f0e4800d950 tx=0x7f0e4800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:41.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.292+0000 7f0e51ffb640 1 -- 172.21.15.16:0/850180284 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e48017070 con 0x7f0e54102d40 2024-09-12T02:11:41.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.292+0000 7f0e51ffb640 1 -- 172.21.15.16:0/850180284 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e48009c30 con 0x7f0e54102d40 2024-09-12T02:11:41.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.292+0000 7f0e5aadd640 1 -- 172.21.15.16:0/850180284 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e5419d710 con 0x7f0e54102d40 2024-09-12T02:11:41.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.292+0000 7f0e51ffb640 1 -- 172.21.15.16:0/850180284 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e4800f6b0 con 0x7f0e54102d40 2024-09-12T02:11:41.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.292+0000 7f0e5aadd640 1 -- 172.21.15.16:0/850180284 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e541a4ff0 con 0x7f0e54102d40 2024-09-12T02:11:41.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.295+0000 7f0e51ffb640 1 -- 172.21.15.16:0/850180284 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f0e480109d0 con 0x7f0e54102d40 2024-09-12T02:11:41.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.295+0000 7f0e5ceb6740 1 Processor -- start 2024-09-12T02:11:41.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.295+0000 7f0e5ceb6740 1 -- start start 2024-09-12T02:11:41.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.295+0000 7f0e5ceb6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e401fd4b40 0x55e401fd3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:41.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.295+0000 7f0e5ceb6740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e401fd35d0 0x55e401fd3a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:41.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.295+0000 7f0e5ceb6740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e401fe2780 0x55e401fe4b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:41.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.295+0000 7f0e5ceb6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e401f392f0 con 0x55e401fd4b40 2024-09-12T02:11:41.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.295+0000 7f0e5ceb6740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55e401ee0cf0 con 0x55e401fd35d0 2024-09-12T02:11:41.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.295+0000 7f0e5ceb6740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55e401f352b0 con 0x55e401fe2780 2024-09-12T02:11:41.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.295+0000 7f0e5907a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e401fd4b40 0x55e401fd3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:41.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.295+0000 7f0e5907a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e401fd4b40 0x55e401fd3090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35838/0 (socket says 172.21.15.16:35838) 2024-09-12T02:11:41.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.295+0000 7f0e5907a640 1 -- 172.21.15.16:0/1184965811 learned_addr learned my addr 172.21.15.16:0/1184965811 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:11:41.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.295+0000 7f0e58879640 1 --2- 172.21.15.16:0/1184965811 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e401fd35d0 0x55e401fd3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:41.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.295+0000 7f0e51ffb640 1 --2- 172.21.15.16:0/850180284 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f0e440a2b80 0x7f0e440a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:41.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.296+0000 7f0e53fff640 1 --2- 172.21.15.16:0/1184965811 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e401fe2780 0x55e401fe4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:41.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.296+0000 7f0e5907a640 1 -- 172.21.15.16:0/1184965811 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e401fe2780 msgr2=0x55e401fe4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:41.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.296+0000 7f0e5907a640 1 --2- 172.21.15.16:0/1184965811 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e401fe2780 0x55e401fe4b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:41.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.296+0000 7f0e5907a640 1 -- 172.21.15.16:0/1184965811 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e401fd35d0 msgr2=0x55e401fd3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:41.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.296+0000 7f0e5907a640 1 --2- 172.21.15.16:0/1184965811 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e401fd35d0 0x55e401fd3a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:41.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.296+0000 7f0e5907a640 1 -- 172.21.15.16:0/1184965811 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e401ec53a0 con 0x55e401fd4b40 2024-09-12T02:11:41.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.296+0000 7f0e51ffb640 1 -- 172.21.15.16:0/850180284 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f0e480ceca0 con 0x7f0e54102d40 2024-09-12T02:11:41.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.296+0000 7f0e5907a640 1 --2- 172.21.15.16:0/850180284 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f0e440a2b80 0x7f0e440a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:41.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.296+0000 7f0e5907a640 1 --2- 172.21.15.16:0/1184965811 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e401fd4b40 0x55e401fd3090 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f0e4c008ed0 tx=0x7f0e4c00d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bb81c7259c780be server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:41.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.296+0000 7f0e3effd640 1 -- 172.21.15.16:0/1184965811 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e4c011e60 con 0x55e401fd4b40 2024-09-12T02:11:41.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.296+0000 7f0e3effd640 1 -- 172.21.15.16:0/1184965811 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e4c01adc0 con 0x55e401fd4b40 2024-09-12T02:11:41.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.296+0000 7f0e3effd640 1 -- 172.21.15.16:0/1184965811 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e4c019cf0 con 0x55e401fd4b40 2024-09-12T02:11:41.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.297+0000 7f0e5ceb6740 1 -- 172.21.15.16:0/1184965811 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e401fd4b40 msgr2=0x55e401fd3090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:41.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.297+0000 7f0e5907a640 1 --2- 172.21.15.16:0/850180284 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f0e440a2b80 0x7f0e440a5040 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f0e4c01e000 tx=0x7f0e4c01f000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:41.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.297+0000 7f0e5ceb6740 1 --2- 172.21.15.16:0/1184965811 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e401fd4b40 0x55e401fd3090 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f0e4c008ed0 tx=0x7f0e4c00d710 comp rx=0 tx=0).stop 2024-09-12T02:11:41.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.297+0000 7f0e5ceb6740 1 -- 172.21.15.16:0/1184965811 shutdown_connections 2024-09-12T02:11:41.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.297+0000 7f0e5ceb6740 1 --2- 172.21.15.16:0/1184965811 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e401fe2780 0x55e401fe4b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:41.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.297+0000 7f0e5ceb6740 1 --2- 172.21.15.16:0/1184965811 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e401fd35d0 0x55e401fd3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:41.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.297+0000 7f0e5ceb6740 1 --2- 172.21.15.16:0/1184965811 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e401fd4b40 0x55e401fd3090 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:41.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.297+0000 7f0e5ceb6740 1 -- 172.21.15.16:0/1184965811 >> 172.21.15.16:0/1184965811 conn(0x55e401f9c310 msgr2=0x55e401fcfc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:41.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.297+0000 7f0e5ceb6740 1 -- 172.21.15.16:0/1184965811 shutdown_connections 2024-09-12T02:11:41.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.297+0000 7f0e5ceb6740 1 -- 172.21.15.16:0/1184965811 wait complete. 2024-09-12T02:11:41.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.297+0000 7f0e5ceb6740 1 -- start start 2024-09-12T02:11:41.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.297+0000 7f0e5ceb6740 1 Processor -- start 2024-09-12T02:11:41.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.298+0000 7f0e5ceb6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e401fd35d0 0x55e401fe2170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:41.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.298+0000 7f0e5ceb6740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e401fd4b40 0x55e401fdf420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:41.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.298+0000 7f0e5ceb6740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e401fe2780 0x55e401fdfa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:41.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.298+0000 7f0e5ceb6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e401fe5480 con 0x55e401fd35d0 2024-09-12T02:11:41.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.298+0000 7f0e5ceb6740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55e401ef4060 con 0x55e401fe2780 2024-09-12T02:11:41.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.298+0000 7f0e5ceb6740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55e401fe5600 con 0x55e401fd4b40 2024-09-12T02:11:41.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.298+0000 7f0e5ceb6740 10 client.0 _ll_register_callbacks cb 0x55e401c478c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:11:41.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.298+0000 7f0e58879640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e401fd35d0 0x55e401fe2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:41.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.298+0000 7f0e58879640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e401fd35d0 0x55e401fe2170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35842/0 (socket says 172.21.15.16:35842) 2024-09-12T02:11:41.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.298+0000 7f0e58879640 1 -- 172.21.15.16:0/260954907 learned_addr learned my addr 172.21.15.16:0/260954907 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:11:41.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.298+0000 7f0e58879640 1 -- 172.21.15.16:0/260954907 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e401fd4b40 msgr2=0x55e401fdf420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:41.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.298+0000 7f0e53fff640 1 --2- 172.21.15.16:0/260954907 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e401fd4b40 0x55e401fdf420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:41.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.298+0000 7f0e58879640 1 --2- 172.21.15.16:0/260954907 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e401fd4b40 0x55e401fdf420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:41.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.298+0000 7f0e5907a640 1 --2- 172.21.15.16:0/260954907 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e401fe2780 0x55e401fdfa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:41.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.298+0000 7f0e58879640 1 -- 172.21.15.16:0/260954907 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e401fe2780 msgr2=0x55e401fdfa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:41.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.298+0000 7f0e58879640 1 --2- 172.21.15.16:0/260954907 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e401fe2780 0x55e401fdfa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:41.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.298+0000 7f0e58879640 1 -- 172.21.15.16:0/260954907 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e401caa3c0 con 0x55e401fd35d0 2024-09-12T02:11:41.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.299+0000 7f0e58879640 1 --2- 172.21.15.16:0/260954907 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e401fd35d0 0x55e401fe2170 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f0e48097f70 tx=0x7f0e48095fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98fa600c578891a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:41.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.299+0000 7f0e33fff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:11:41.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.299+0000 7f0e33fff640 1 -- 172.21.15.16:0/260954907 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e48020040 con 0x55e401fd35d0 2024-09-12T02:11:41.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.299+0000 7f0e33fff640 1 -- 172.21.15.16:0/260954907 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f0e48097c10 con 0x55e401fd35d0 2024-09-12T02:11:41.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.299+0000 7f0e5ceb6740 1 -- 172.21.15.16:0/260954907 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e4c01adc0 con 0x55e401fd35d0 2024-09-12T02:11:41.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.299+0000 7f0e33fff640 1 -- 172.21.15.16:0/260954907 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f0e4809b5a0 con 0x55e401fd35d0 2024-09-12T02:11:41.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.299+0000 7f0e5ceb6740 10 client.18387 fetch_fsmap learned FSMap version 25 2024-09-12T02:11:41.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.299+0000 7f0e5ceb6740 1 -- 172.21.15.16:0/260954907 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e401ff0fd0 con 0x55e401fd35d0 2024-09-12T02:11:41.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.299+0000 7f0e33fff640 1 -- 172.21.15.16:0/260954907 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e4c01adc0 con 0x55e401fd35d0 2024-09-12T02:11:41.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.299+0000 7f0e33fff640 1 -- 172.21.15.16:0/260954907 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f0e480d3040 con 0x55e401fd35d0 2024-09-12T02:11:41.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.299+0000 7f0e5ceb6740 10 client.18387 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:11:41.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.299+0000 7f0e5ceb6740 10 client.18387 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T02:11:41.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.299+0000 7f0e5ceb6740 20 client.18387 populate_metadata read hostname 'smithi016' 2024-09-12T02:11:41.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.299+0000 7f0e5ceb6740 1 --2- 172.21.15.16:0/260954907 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x55e401ff2490 0x55e401ff4880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:41.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.299+0000 7f0e5ceb6740 4 client.18387 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:11:41.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.299+0000 7f0e5ceb6740 1 -- 172.21.15.16:0/260954907 --> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e401f46860 con 0x55e401ff2490 2024-09-12T02:11:41.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.300+0000 7f0e53fff640 1 --2- 172.21.15.16:0/260954907 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x55e401ff2490 0x55e401ff4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:41.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.300+0000 7f0e33fff640 10 client.18387 ms_handle_connect on v2:172.21.15.184:6838/4203347941 2024-09-12T02:11:41.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.300+0000 7f0e53fff640 1 --2- 172.21.15.16:0/260954907 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x55e401ff2490 0x55e401ff4880 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9de15bc6f51dca03 server_cookie=266681a88dfb5ee5 in_seq=0 out_seq=0 2024-09-12T02:11:41.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.301+0000 7f0e33fff640 1 -- 172.21.15.16:0/260954907 <== mds.1 v2:172.21.15.184:6838/4203347941 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e401f46860 con 0x55e401ff2490 2024-09-12T02:11:41.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.301+0000 7f0e33fff640 10 client.18387 handle_command_reply: tid=0 2024-09-12T02:11:41.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.301+0000 7f0e5ceb6740 1 client.18387 shutdown 2024-09-12T02:11:41.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.301+0000 7f0e5ceb6740 1 -- 172.21.15.16:0/260954907 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e401fd35d0 msgr2=0x55e401fe2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:41.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.301+0000 7f0e5ceb6740 1 --2- 172.21.15.16:0/260954907 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e401fd35d0 0x55e401fe2170 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f0e48097f70 tx=0x7f0e48095fd0 comp rx=0 tx=0).stop 2024-09-12T02:11:41.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.301+0000 7f0e5ceb6740 1 -- 172.21.15.16:0/260954907 shutdown_connections 2024-09-12T02:11:41.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.301+0000 7f0e5ceb6740 1 --2- 172.21.15.16:0/260954907 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x55e401ff2490 0x55e401ff4880 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:41.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.301+0000 7f0e5ceb6740 1 --2- 172.21.15.16:0/260954907 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e401fe2780 0x55e401fdfa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:41.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.301+0000 7f0e5ceb6740 1 --2- 172.21.15.16:0/260954907 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e401fd4b40 0x55e401fdf420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:41.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.301+0000 7f0e5ceb6740 1 --2- 172.21.15.16:0/260954907 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e401fd35d0 0x55e401fe2170 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:41.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.301+0000 7f0e5ceb6740 1 -- 172.21.15.16:0/260954907 >> 172.21.15.16:0/260954907 conn(0x55e401f9c310 msgr2=0x55e401fd5930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:41.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.301+0000 7f0e33fff640 0 client.18387 ms_handle_reset on v2:172.21.15.184:6838/4203347941 2024-09-12T02:11:41.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.301+0000 7f0e5ceb6740 1 -- 172.21.15.16:0/260954907 shutdown_connections 2024-09-12T02:11:41.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.301+0000 7f0e5ceb6740 1 -- 172.21.15.16:0/260954907 wait complete. 2024-09-12T02:11:41.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.302+0000 7f0e5ceb6740 20 client.18387 trim_cache size 0 max 16384 2024-09-12T02:11:41.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.315+0000 7f0e5ceb6740 1 Processor -- start 2024-09-12T02:11:41.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.315+0000 7f0e5ceb6740 1 -- start start 2024-09-12T02:11:41.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.315+0000 7f0e5ceb6740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e401fd35d0 0x55e401fee720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:41.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.315+0000 7f0e5ceb6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e401fd4b40 0x55e401fde960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:41.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.315+0000 7f0e5ceb6740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e401fe2780 0x55e401ff9f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:41.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.315+0000 7f0e5ceb6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0e541adb80 con 0x55e401fd4b40 2024-09-12T02:11:41.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.315+0000 7f0e5ceb6740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0e541ab450 con 0x55e401fe2780 2024-09-12T02:11:41.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.315+0000 7f0e5ceb6740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f0e541a92a0 con 0x55e401fd35d0 2024-09-12T02:11:41.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.315+0000 7f0e53fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e401fd4b40 0x55e401fde960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:41.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.315+0000 7f0e53fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e401fd4b40 0x55e401fde960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35858/0 (socket says 172.21.15.16:35858) 2024-09-12T02:11:41.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.315+0000 7f0e53fff640 1 -- 172.21.15.16:0/580789942 learned_addr learned my addr 172.21.15.16:0/580789942 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:11:41.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.315+0000 7f0e5907a640 1 --2- 172.21.15.16:0/580789942 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e401fe2780 0x55e401ff9f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:41.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.315+0000 7f0e53fff640 1 -- 172.21.15.16:0/580789942 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e401fd35d0 msgr2=0x55e401fee720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:41.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.315+0000 7f0e53fff640 1 --2- 172.21.15.16:0/580789942 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e401fd35d0 0x55e401fee720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:41.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.315+0000 7f0e53fff640 1 -- 172.21.15.16:0/580789942 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e401fe2780 msgr2=0x55e401ff9f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:41.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.315+0000 7f0e53fff640 1 --2- 172.21.15.16:0/580789942 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e401fe2780 0x55e401ff9f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:41.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.315+0000 7f0e53fff640 1 -- 172.21.15.16:0/580789942 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e401fcfed0 con 0x55e401fd4b40 2024-09-12T02:11:41.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.316+0000 7f0e53fff640 1 --2- 172.21.15.16:0/580789942 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e401fd4b40 0x55e401fde960 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f0e40005ab0 tx=0x7f0e400042b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20259c980a01b036 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:41.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.316+0000 7f0e3e7fc640 1 -- 172.21.15.16:0/580789942 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e40049070 con 0x55e401fd4b40 2024-09-12T02:11:41.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.316+0000 7f0e3e7fc640 1 -- 172.21.15.16:0/580789942 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e40005520 con 0x55e401fd4b40 2024-09-12T02:11:41.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.316+0000 7f0e3e7fc640 1 -- 172.21.15.16:0/580789942 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e40035040 con 0x55e401fd4b40 2024-09-12T02:11:41.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.316+0000 7f0e5ceb6740 1 -- 172.21.15.16:0/580789942 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e401fd4b40 msgr2=0x55e401fde960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:41.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.316+0000 7f0e5ceb6740 1 --2- 172.21.15.16:0/580789942 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e401fd4b40 0x55e401fde960 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f0e40005ab0 tx=0x7f0e400042b0 comp rx=0 tx=0).stop 2024-09-12T02:11:41.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.317+0000 7f0e5ceb6740 1 -- 172.21.15.16:0/580789942 shutdown_connections 2024-09-12T02:11:41.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.317+0000 7f0e5ceb6740 1 --2- 172.21.15.16:0/580789942 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e401fe2780 0x55e401ff9f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:41.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.317+0000 7f0e5ceb6740 1 --2- 172.21.15.16:0/580789942 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e401fd4b40 0x55e401fde960 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:41.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.317+0000 7f0e5ceb6740 1 --2- 172.21.15.16:0/580789942 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e401fd35d0 0x55e401fee720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:41.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.317+0000 7f0e5ceb6740 1 -- 172.21.15.16:0/580789942 >> 172.21.15.16:0/580789942 conn(0x55e401f9c310 msgr2=0x55e401fddcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:41.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.317+0000 7f0e5ceb6740 1 -- 172.21.15.16:0/580789942 shutdown_connections 2024-09-12T02:11:41.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.317+0000 7f0e5ceb6740 1 -- 172.21.15.16:0/580789942 wait complete. 2024-09-12T02:11:41.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.317+0000 7f0e5ceb6740 1 -- start start 2024-09-12T02:11:41.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.317+0000 7f0e5ceb6740 1 Processor -- start 2024-09-12T02:11:41.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.317+0000 7f0e5ceb6740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e401fd35d0 0x55e401feb180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:41.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.317+0000 7f0e5ceb6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e401fd4b40 0x55e401fe6500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:41.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.317+0000 7f0e5ceb6740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e401fe2780 0x55e401fe6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:41.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.317+0000 7f0e5ceb6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0e541a9910 con 0x55e401fd4b40 2024-09-12T02:11:41.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.317+0000 7f0e5ceb6740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0e541aad80 con 0x55e401fd35d0 2024-09-12T02:11:41.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.317+0000 7f0e5ceb6740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f0e541a9fe0 con 0x55e401fe2780 2024-09-12T02:11:41.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.317+0000 7f0e5ceb6740 10 client.0 _ll_register_callbacks cb 0x7f0e48086050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:11:41.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.318+0000 7f0e53fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e401fd4b40 0x55e401fe6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:41.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.318+0000 7f0e53fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e401fd4b40 0x55e401fe6500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35870/0 (socket says 172.21.15.16:35870) 2024-09-12T02:11:41.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.318+0000 7f0e53fff640 1 -- 172.21.15.16:0/1263668762 learned_addr learned my addr 172.21.15.16:0/1263668762 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:11:41.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.318+0000 7f0e58879640 1 --2- 172.21.15.16:0/1263668762 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e401fd35d0 0x55e401feb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:41.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.318+0000 7f0e5907a640 1 --2- 172.21.15.16:0/1263668762 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e401fe2780 0x55e401fe6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:41.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.318+0000 7f0e53fff640 1 -- 172.21.15.16:0/1263668762 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e401fe2780 msgr2=0x55e401fe6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:41.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.318+0000 7f0e53fff640 1 --2- 172.21.15.16:0/1263668762 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e401fe2780 0x55e401fe6c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:41.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.318+0000 7f0e53fff640 1 -- 172.21.15.16:0/1263668762 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e401fd35d0 msgr2=0x55e401feb180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:41.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.318+0000 7f0e53fff640 1 --2- 172.21.15.16:0/1263668762 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e401fd35d0 0x55e401feb180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:41.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.318+0000 7f0e53fff640 1 -- 172.21.15.16:0/1263668762 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e40205bf60 con 0x55e401fd4b40 2024-09-12T02:11:41.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.318+0000 7f0e5907a640 1 --2- 172.21.15.16:0/1263668762 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e401fe2780 0x55e401fe6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:11:41.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.318+0000 7f0e53fff640 1 --2- 172.21.15.16:0/1263668762 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e401fd4b40 0x55e401fe6500 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f0e40047040 tx=0x7f0e40009520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=935c947a78bffd9c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:41.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.318+0000 7f0e33fff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:11:41.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.318+0000 7f0e33fff640 1 -- 172.21.15.16:0/1263668762 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e40052060 con 0x55e401fd4b40 2024-09-12T02:11:41.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.318+0000 7f0e33fff640 1 -- 172.21.15.16:0/1263668762 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f0e400043a0 con 0x55e401fd4b40 2024-09-12T02:11:41.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.318+0000 7f0e5ceb6740 1 -- 172.21.15.16:0/1263668762 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e40005520 con 0x55e401fd4b40 2024-09-12T02:11:41.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.319+0000 7f0e33fff640 1 -- 172.21.15.16:0/1263668762 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f0e40035050 con 0x55e401fd4b40 2024-09-12T02:11:41.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.319+0000 7f0e5ceb6740 10 client.18393 fetch_fsmap learned FSMap version 25 2024-09-12T02:11:41.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.319+0000 7f0e5ceb6740 1 -- 172.21.15.16:0/1263668762 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e40205d100 con 0x55e401fd4b40 2024-09-12T02:11:41.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.319+0000 7f0e33fff640 1 -- 172.21.15.16:0/1263668762 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e40005520 con 0x55e401fd4b40 2024-09-12T02:11:41.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.319+0000 7f0e33fff640 1 -- 172.21.15.16:0/1263668762 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f0e40031770 con 0x55e401fd4b40 2024-09-12T02:11:41.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.319+0000 7f0e5ceb6740 10 client.18393 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:11:41.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.319+0000 7f0e5ceb6740 10 client.18393 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T02:11:41.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.319+0000 7f0e5ceb6740 20 client.18393 populate_metadata read hostname 'smithi016' 2024-09-12T02:11:41.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.319+0000 7f0e5ceb6740 1 --2- 172.21.15.16:0/1263668762 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x55e40205d7d0 0x55e40205fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:41.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.319+0000 7f0e5ceb6740 4 client.18393 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T02:11:41.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.319+0000 7f0e5ceb6740 1 -- 172.21.15.16:0/1263668762 --> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e401f40d70 con 0x55e40205d7d0 2024-09-12T02:11:41.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.319+0000 7f0e58879640 1 --2- 172.21.15.16:0/1263668762 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x55e40205d7d0 0x55e40205fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:41.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.320+0000 7f0e58879640 1 --2- 172.21.15.16:0/1263668762 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x55e40205d7d0 0x55e40205fbf0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8db47a26d0afa12c server_cookie=893046505bf1dd09 in_seq=0 out_seq=0 2024-09-12T02:11:41.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.320+0000 7f0e33fff640 10 client.18393 ms_handle_connect on v2:172.21.15.184:6838/4203347941 2024-09-12T02:11:41.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.320+0000 7f0e33fff640 1 -- 172.21.15.16:0/1263668762 <== mds.1 v2:172.21.15.184:6838/4203347941 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e401f40d70 con 0x55e40205d7d0 2024-09-12T02:11:41.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.320+0000 7f0e33fff640 10 client.18393 handle_command_reply: tid=0 2024-09-12T02:11:41.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.320+0000 7f0e5ceb6740 1 client.18393 shutdown 2024-09-12T02:11:41.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.320+0000 7f0e5ceb6740 1 -- 172.21.15.16:0/1263668762 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e401fd4b40 msgr2=0x55e401fe6500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:41.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.320+0000 7f0e5ceb6740 1 --2- 172.21.15.16:0/1263668762 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e401fd4b40 0x55e401fe6500 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f0e40047040 tx=0x7f0e40009520 comp rx=0 tx=0).stop 2024-09-12T02:11:41.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.320+0000 7f0e5ceb6740 1 -- 172.21.15.16:0/1263668762 shutdown_connections 2024-09-12T02:11:41.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.320+0000 7f0e5ceb6740 1 --2- 172.21.15.16:0/1263668762 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x55e40205d7d0 0x55e40205fbf0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:41.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.320+0000 7f0e33fff640 0 client.18393 ms_handle_reset on v2:172.21.15.184:6838/4203347941 2024-09-12T02:11:41.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.320+0000 7f0e5ceb6740 1 --2- 172.21.15.16:0/1263668762 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e401fe2780 0x55e401fe6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:41.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.320+0000 7f0e5ceb6740 1 --2- 172.21.15.16:0/1263668762 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e401fd4b40 0x55e401fe6500 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:41.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.320+0000 7f0e5ceb6740 1 --2- 172.21.15.16:0/1263668762 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e401fd35d0 0x55e401feb180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:41.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.320+0000 7f0e5ceb6740 1 -- 172.21.15.16:0/1263668762 >> 172.21.15.16:0/1263668762 conn(0x55e401f9c310 msgr2=0x55e401fce4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:41.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.320+0000 7f0e5ceb6740 1 -- 172.21.15.16:0/1263668762 shutdown_connections 2024-09-12T02:11:41.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.320+0000 7f0e5ceb6740 1 -- 172.21.15.16:0/1263668762 wait complete. 2024-09-12T02:11:41.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.320+0000 7f0e5ceb6740 20 client.18393 trim_cache size 0 max 16384 2024-09-12T02:11:41.330 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T02:11:41.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.321+0000 7f0e5aadd640 1 -- 172.21.15.16:0/850180284 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f0e440a2b80 msgr2=0x7f0e440a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:41.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.321+0000 7f0e5aadd640 1 --2- 172.21.15.16:0/850180284 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f0e440a2b80 0x7f0e440a5040 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f0e4c01e000 tx=0x7f0e4c01f000 comp rx=0 tx=0).stop 2024-09-12T02:11:41.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.321+0000 7f0e5aadd640 1 -- 172.21.15.16:0/850180284 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e54102d40 msgr2=0x7f0e54198470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:41.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.321+0000 7f0e5aadd640 1 --2- 172.21.15.16:0/850180284 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e54102d40 0x7f0e54198470 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f0e4800d950 tx=0x7f0e4800de10 comp rx=0 tx=0).stop 2024-09-12T02:11:41.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.321+0000 7f0e5aadd640 1 -- 172.21.15.16:0/850180284 shutdown_connections 2024-09-12T02:11:41.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.321+0000 7f0e5aadd640 1 --2- 172.21.15.16:0/850180284 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f0e440a2b80 0x7f0e440a5040 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:41.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.321+0000 7f0e5aadd640 1 --2- 172.21.15.16:0/850180284 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e5410c170 0x7f0e5419cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:41.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.321+0000 7f0e5aadd640 1 --2- 172.21.15.16:0/850180284 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0e54103660 0x7f0e541989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:41.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.321+0000 7f0e5aadd640 1 --2- 172.21.15.16:0/850180284 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e54102d40 0x7f0e54198470 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:41.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.321+0000 7f0e5aadd640 1 -- 172.21.15.16:0/850180284 >> 172.21.15.16:0/850180284 conn(0x7f0e540fc7a0 msgr2=0x7f0e540fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:41.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.321+0000 7f0e5aadd640 1 -- 172.21.15.16:0/850180284 shutdown_connections 2024-09-12T02:11:41.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:41.321+0000 7f0e5aadd640 1 -- 172.21.15.16:0/850180284 wait complete. 2024-09-12T02:11:42.044 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-12T02:11:42.302 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:11:42.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:11:42 smithi184 ceph-mon[35593]: pgmap v628: 97 pgs: 97 active+clean; 1.1 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 2.5 KiB/s wr, 9 op/s 2024-09-12T02:11:42.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:11:42 smithi033 ceph-mon[35332]: pgmap v628: 97 pgs: 97 active+clean; 1.1 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 2.5 KiB/s wr, 9 op/s 2024-09-12T02:11:42.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:11:42 smithi016 ceph-mon[30947]: pgmap v628: 97 pgs: 97 active+clean; 1.1 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 2.5 KiB/s wr, 9 op/s 2024-09-12T02:11:42.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.772+0000 7f2c95123640 1 -- 172.21.15.16:0/3127799230 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c901036c0 msgr2=0x7f2c9010bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:42.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.772+0000 7f2c95123640 1 --2- 172.21.15.16:0/3127799230 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c901036c0 0x7f2c9010bbb0 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f2c84009900 tx=0x7f2c8402a480 comp rx=0 tx=0).stop 2024-09-12T02:11:42.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.772+0000 7f2c95123640 1 -- 172.21.15.16:0/3127799230 shutdown_connections 2024-09-12T02:11:42.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.772+0000 7f2c95123640 1 --2- 172.21.15.16:0/3127799230 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c9010c180 0x7f2c9010e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:42.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.772+0000 7f2c95123640 1 --2- 172.21.15.16:0/3127799230 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c901036c0 0x7f2c9010bbb0 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:42.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.772+0000 7f2c95123640 1 --2- 172.21.15.16:0/3127799230 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2c90102da0 0x7f2c90103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:42.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.772+0000 7f2c95123640 1 -- 172.21.15.16:0/3127799230 >> 172.21.15.16:0/3127799230 conn(0x7f2c900fc7a0 msgr2=0x7f2c900febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:42.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.772+0000 7f2c95123640 1 -- 172.21.15.16:0/3127799230 shutdown_connections 2024-09-12T02:11:42.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.772+0000 7f2c95123640 1 -- 172.21.15.16:0/3127799230 wait complete. 2024-09-12T02:11:42.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.773+0000 7f2c95123640 1 Processor -- start 2024-09-12T02:11:42.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.773+0000 7f2c95123640 1 -- start start 2024-09-12T02:11:42.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.773+0000 7f2c95123640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c90102da0 0x7f2c90198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:42.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.773+0000 7f2c95123640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c901036c0 0x7f2c90198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:42.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.773+0000 7f2c95123640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2c9010c180 0x7f2c9019cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:42.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.773+0000 7f2c95123640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2c90111fa0 con 0x7f2c90102da0 2024-09-12T02:11:42.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.773+0000 7f2c95123640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2c90111e20 con 0x7f2c901036c0 2024-09-12T02:11:42.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.773+0000 7f2c95123640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f2c90112120 con 0x7f2c9010c180 2024-09-12T02:11:42.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.774+0000 7f2c8ed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c90102da0 0x7f2c90198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:42.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.774+0000 7f2c8ed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c90102da0 0x7f2c90198500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35886/0 (socket says 172.21.15.16:35886) 2024-09-12T02:11:42.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.774+0000 7f2c8ed9d640 1 -- 172.21.15.16:0/3758969677 learned_addr learned my addr 172.21.15.16:0/3758969677 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:11:42.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.774+0000 7f2c8e59c640 1 --2- 172.21.15.16:0/3758969677 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c901036c0 0x7f2c90198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:42.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.774+0000 7f2c8f59e640 1 --2- 172.21.15.16:0/3758969677 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2c9010c180 0x7f2c9019cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:42.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.774+0000 7f2c8ed9d640 1 -- 172.21.15.16:0/3758969677 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2c9010c180 msgr2=0x7f2c9019cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:42.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.774+0000 7f2c8ed9d640 1 --2- 172.21.15.16:0/3758969677 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2c9010c180 0x7f2c9019cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:42.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.774+0000 7f2c8ed9d640 1 -- 172.21.15.16:0/3758969677 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c901036c0 msgr2=0x7f2c90198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:42.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.774+0000 7f2c8ed9d640 1 --2- 172.21.15.16:0/3758969677 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c901036c0 0x7f2c90198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:42.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.774+0000 7f2c8ed9d640 1 -- 172.21.15.16:0/3758969677 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c9019d4b0 con 0x7f2c90102da0 2024-09-12T02:11:42.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.774+0000 7f2c8f59e640 1 --2- 172.21.15.16:0/3758969677 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2c9010c180 0x7f2c9019cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:11:42.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.774+0000 7f2c8e59c640 1 --2- 172.21.15.16:0/3758969677 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c901036c0 0x7f2c90198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:11:42.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.774+0000 7f2c8ed9d640 1 --2- 172.21.15.16:0/3758969677 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c90102da0 0x7f2c90198500 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f2c8000b4a0 tx=0x7f2c8000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:42.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.774+0000 7f2c77fff640 1 -- 172.21.15.16:0/3758969677 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c80014070 con 0x7f2c90102da0 2024-09-12T02:11:42.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.774+0000 7f2c95123640 1 -- 172.21.15.16:0/3758969677 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c9019d7a0 con 0x7f2c90102da0 2024-09-12T02:11:42.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.774+0000 7f2c77fff640 1 -- 172.21.15.16:0/3758969677 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c80010af0 con 0x7f2c90102da0 2024-09-12T02:11:42.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.774+0000 7f2c77fff640 1 -- 172.21.15.16:0/3758969677 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c80017d10 con 0x7f2c90102da0 2024-09-12T02:11:42.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.774+0000 7f2c95123640 1 -- 172.21.15.16:0/3758969677 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c901a5080 con 0x7f2c90102da0 2024-09-12T02:11:42.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.777+0000 7f2c974fc740 1 Processor -- start 2024-09-12T02:11:42.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.777+0000 7f2c77fff640 1 -- 172.21.15.16:0/3758969677 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f2c80017070 con 0x7f2c90102da0 2024-09-12T02:11:42.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.777+0000 7f2c974fc740 1 -- start start 2024-09-12T02:11:42.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.777+0000 7f2c974fc740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bdb73f8b40 0x55bdb73f7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:42.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.777+0000 7f2c974fc740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bdb73f75d0 0x55bdb73f7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:42.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.777+0000 7f2c974fc740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bdb7402370 0x55bdb7404760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:42.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.777+0000 7f2c974fc740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55bdb735d2f0 con 0x55bdb7402370 2024-09-12T02:11:42.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.777+0000 7f2c974fc740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55bdb7304cf0 con 0x55bdb73f8b40 2024-09-12T02:11:42.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.777+0000 7f2c974fc740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55bdb73592b0 con 0x55bdb73f75d0 2024-09-12T02:11:42.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.777+0000 7f2c8e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bdb7402370 0x55bdb7404760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:42.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.777+0000 7f2c8e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bdb7402370 0x55bdb7404760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35888/0 (socket says 172.21.15.16:35888) 2024-09-12T02:11:42.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.777+0000 7f2c8e59c640 1 -- 172.21.15.16:0/2826953864 learned_addr learned my addr 172.21.15.16:0/2826953864 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:11:42.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.777+0000 7f2c8f59e640 1 --2- 172.21.15.16:0/2826953864 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bdb73f8b40 0x55bdb73f7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:42.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.778+0000 7f2c8ed9d640 1 --2- 172.21.15.16:0/2826953864 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bdb73f75d0 0x55bdb73f7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:42.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.778+0000 7f2c8e59c640 1 -- 172.21.15.16:0/2826953864 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bdb73f75d0 msgr2=0x55bdb73f7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:42.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.778+0000 7f2c8e59c640 1 --2- 172.21.15.16:0/2826953864 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bdb73f75d0 0x55bdb73f7a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:42.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.778+0000 7f2c8e59c640 1 -- 172.21.15.16:0/2826953864 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bdb73f8b40 msgr2=0x55bdb73f7090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:42.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.778+0000 7f2c8e59c640 1 --2- 172.21.15.16:0/2826953864 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bdb73f8b40 0x55bdb73f7090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:42.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.778+0000 7f2c8e59c640 1 -- 172.21.15.16:0/2826953864 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bdb72e93a0 con 0x55bdb7402370 2024-09-12T02:11:42.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.778+0000 7f2c77fff640 1 --2- 172.21.15.16:0/3758969677 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f2c780a2b80 0x7f2c780a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:42.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.778+0000 7f2c8f59e640 1 --2- 172.21.15.16:0/2826953864 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bdb73f8b40 0x55bdb73f7090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:11:42.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.778+0000 7f2c8f59e640 1 --2- 172.21.15.16:0/3758969677 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f2c780a2b80 0x7f2c780a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:42.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.778+0000 7f2c77fff640 1 -- 172.21.15.16:0/3758969677 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f2c800ce3c0 con 0x7f2c90102da0 2024-09-12T02:11:42.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.778+0000 7f2c8e59c640 1 --2- 172.21.15.16:0/2826953864 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bdb7402370 0x55bdb7404760 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7f2c8402a9a0 tx=0x7f2c840316e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d91f54b14b68bf03 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:42.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.778+0000 7f2c757fa640 1 -- 172.21.15.16:0/2826953864 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c8403f070 con 0x55bdb7402370 2024-09-12T02:11:42.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.778+0000 7f2c757fa640 1 -- 172.21.15.16:0/2826953864 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c840035f0 con 0x55bdb7402370 2024-09-12T02:11:42.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.778+0000 7f2c757fa640 1 -- 172.21.15.16:0/2826953864 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c84035070 con 0x55bdb7402370 2024-09-12T02:11:42.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.779+0000 7f2c974fc740 1 -- 172.21.15.16:0/2826953864 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bdb7402370 msgr2=0x55bdb7404760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:42.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.779+0000 7f2c974fc740 1 --2- 172.21.15.16:0/2826953864 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bdb7402370 0x55bdb7404760 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7f2c8402a9a0 tx=0x7f2c840316e0 comp rx=0 tx=0).stop 2024-09-12T02:11:42.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.779+0000 7f2c8f59e640 1 --2- 172.21.15.16:0/3758969677 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f2c780a2b80 0x7f2c780a5040 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f2c88005fd0 tx=0x7f2c88008630 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:42.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.779+0000 7f2c974fc740 1 -- 172.21.15.16:0/2826953864 shutdown_connections 2024-09-12T02:11:42.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.779+0000 7f2c974fc740 1 --2- 172.21.15.16:0/2826953864 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bdb7402370 0x55bdb7404760 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:42.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.779+0000 7f2c974fc740 1 --2- 172.21.15.16:0/2826953864 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bdb73f75d0 0x55bdb73f7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:42.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.779+0000 7f2c974fc740 1 --2- 172.21.15.16:0/2826953864 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bdb73f8b40 0x55bdb73f7090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:42.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.779+0000 7f2c974fc740 1 -- 172.21.15.16:0/2826953864 >> 172.21.15.16:0/2826953864 conn(0x55bdb73c0310 msgr2=0x55bdb73f3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:42.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.779+0000 7f2c974fc740 1 -- 172.21.15.16:0/2826953864 shutdown_connections 2024-09-12T02:11:42.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.779+0000 7f2c974fc740 1 -- 172.21.15.16:0/2826953864 wait complete. 2024-09-12T02:11:42.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.779+0000 7f2c974fc740 1 -- start start 2024-09-12T02:11:42.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.779+0000 7f2c974fc740 1 Processor -- start 2024-09-12T02:11:42.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.780+0000 7f2c974fc740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bdb73f75d0 0x55bdb7401d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:42.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.780+0000 7f2c974fc740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bdb73f8b40 0x55bdb73ff040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:42.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.780+0000 7f2c974fc740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bdb7402370 0x55bdb73ff650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:42.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.780+0000 7f2c974fc740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55bdb7405010 con 0x55bdb73f8b40 2024-09-12T02:11:42.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.780+0000 7f2c974fc740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55bdb7318060 con 0x55bdb7402370 2024-09-12T02:11:42.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.780+0000 7f2c974fc740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55bdb7405190 con 0x55bdb73f75d0 2024-09-12T02:11:42.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.780+0000 7f2c974fc740 10 client.0 _ll_register_callbacks cb 0x55bdb706b8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:11:42.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.780+0000 7f2c8e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bdb73f8b40 0x55bdb73ff040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:42.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.780+0000 7f2c8e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bdb73f8b40 0x55bdb73ff040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35896/0 (socket says 172.21.15.16:35896) 2024-09-12T02:11:42.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.780+0000 7f2c8e59c640 1 -- 172.21.15.16:0/1941613659 learned_addr learned my addr 172.21.15.16:0/1941613659 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:11:42.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.780+0000 7f2c8ed9d640 1 --2- 172.21.15.16:0/1941613659 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bdb73f75d0 0x55bdb7401d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:42.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.780+0000 7f2c8f59e640 1 --2- 172.21.15.16:0/1941613659 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bdb7402370 0x55bdb73ff650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:42.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.780+0000 7f2c8e59c640 1 -- 172.21.15.16:0/1941613659 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bdb73f75d0 msgr2=0x55bdb7401d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:42.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.780+0000 7f2c8e59c640 1 --2- 172.21.15.16:0/1941613659 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bdb73f75d0 0x55bdb7401d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:42.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.780+0000 7f2c8e59c640 1 -- 172.21.15.16:0/1941613659 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bdb7402370 msgr2=0x55bdb73ff650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:42.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.780+0000 7f2c8e59c640 1 --2- 172.21.15.16:0/1941613659 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bdb7402370 0x55bdb73ff650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:42.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.780+0000 7f2c8e59c640 1 -- 172.21.15.16:0/1941613659 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bdb70ce3c0 con 0x55bdb73f8b40 2024-09-12T02:11:42.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.780+0000 7f2c8e59c640 1 --2- 172.21.15.16:0/1941613659 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bdb73f8b40 0x55bdb73ff040 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f2c84004640 tx=0x7f2c840318f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=306f12af56abea36 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:42.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.780+0000 7f2c72ff5640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:11:42.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.781+0000 7f2c72ff5640 1 -- 172.21.15.16:0/1941613659 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c84035070 con 0x55bdb73f8b40 2024-09-12T02:11:42.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.781+0000 7f2c72ff5640 1 -- 172.21.15.16:0/1941613659 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f2c84003bb0 con 0x55bdb73f8b40 2024-09-12T02:11:42.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.781+0000 7f2c974fc740 1 -- 172.21.15.16:0/1941613659 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c840035f0 con 0x55bdb73f8b40 2024-09-12T02:11:42.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.781+0000 7f2c72ff5640 1 -- 172.21.15.16:0/1941613659 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2c8403f040 con 0x55bdb73f8b40 2024-09-12T02:11:42.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.781+0000 7f2c974fc740 10 client.18405 fetch_fsmap learned FSMap version 25 2024-09-12T02:11:42.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.781+0000 7f2c974fc740 1 -- 172.21.15.16:0/1941613659 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55bdb7410ae0 con 0x55bdb73f8b40 2024-09-12T02:11:42.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.781+0000 7f2c72ff5640 1 -- 172.21.15.16:0/1941613659 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c840035f0 con 0x55bdb73f8b40 2024-09-12T02:11:42.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.781+0000 7f2c72ff5640 1 -- 172.21.15.16:0/1941613659 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f2c84050660 con 0x55bdb73f8b40 2024-09-12T02:11:42.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.781+0000 7f2c974fc740 10 client.18405 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:11:42.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.781+0000 7f2c974fc740 10 client.18405 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:11:42.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.781+0000 7f2c974fc740 20 client.18405 populate_metadata read hostname 'smithi016' 2024-09-12T02:11:42.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.781+0000 7f2c974fc740 1 --2- 172.21.15.16:0/1941613659 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55bdb7411e30 0x55bdb7414220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:42.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.781+0000 7f2c974fc740 4 client.18405 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:11:42.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.781+0000 7f2c974fc740 1 -- 172.21.15.16:0/1941613659 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bdb736a860 con 0x55bdb7411e30 2024-09-12T02:11:42.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.782+0000 7f2c8ed9d640 1 --2- 172.21.15.16:0/1941613659 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55bdb7411e30 0x55bdb7414220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:42.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.782+0000 7f2c8ed9d640 1 --2- 172.21.15.16:0/1941613659 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55bdb7411e30 0x55bdb7414220 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b88b03f47b37cb1 server_cookie=77794b8a9ab97854 in_seq=0 out_seq=0 2024-09-12T02:11:42.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.782+0000 7f2c72ff5640 10 client.18405 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:11:42.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.783+0000 7f2c72ff5640 1 -- 172.21.15.16:0/1941613659 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bdb736a860 con 0x55bdb7411e30 2024-09-12T02:11:42.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.783+0000 7f2c72ff5640 10 client.18405 handle_command_reply: tid=0 2024-09-12T02:11:42.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.783+0000 7f2c974fc740 1 client.18405 shutdown 2024-09-12T02:11:42.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.783+0000 7f2c974fc740 1 -- 172.21.15.16:0/1941613659 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bdb73f8b40 msgr2=0x55bdb73ff040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:42.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.783+0000 7f2c974fc740 1 --2- 172.21.15.16:0/1941613659 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bdb73f8b40 0x55bdb73ff040 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f2c84004640 tx=0x7f2c840318f0 comp rx=0 tx=0).stop 2024-09-12T02:11:42.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.783+0000 7f2c974fc740 1 -- 172.21.15.16:0/1941613659 shutdown_connections 2024-09-12T02:11:42.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.783+0000 7f2c974fc740 1 --2- 172.21.15.16:0/1941613659 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55bdb7411e30 0x55bdb7414220 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:42.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.783+0000 7f2c974fc740 1 --2- 172.21.15.16:0/1941613659 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bdb7402370 0x55bdb73ff650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:42.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.783+0000 7f2c974fc740 1 --2- 172.21.15.16:0/1941613659 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bdb73f8b40 0x55bdb73ff040 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:42.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.783+0000 7f2c72ff5640 0 client.18405 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:11:42.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.783+0000 7f2c974fc740 1 --2- 172.21.15.16:0/1941613659 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bdb73f75d0 0x55bdb7401d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:42.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.783+0000 7f2c974fc740 1 -- 172.21.15.16:0/1941613659 >> 172.21.15.16:0/1941613659 conn(0x55bdb73c0310 msgr2=0x55bdb73f9960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:42.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.783+0000 7f2c974fc740 1 -- 172.21.15.16:0/1941613659 shutdown_connections 2024-09-12T02:11:42.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.783+0000 7f2c974fc740 1 -- 172.21.15.16:0/1941613659 wait complete. 2024-09-12T02:11:42.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.783+0000 7f2c974fc740 20 client.18405 trim_cache size 0 max 16384 2024-09-12T02:11:42.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.796+0000 7f2c974fc740 1 Processor -- start 2024-09-12T02:11:42.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.796+0000 7f2c974fc740 1 -- start start 2024-09-12T02:11:42.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.796+0000 7f2c974fc740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bdb73f75d0 0x55bdb7401370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:42.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.796+0000 7f2c974fc740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bdb73f8b40 0x55bdb73fa120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:42.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.796+0000 7f2c974fc740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bdb7402370 0x55bdb73fa660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:42.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.796+0000 7f2c974fc740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2c901afd70 con 0x55bdb73f8b40 2024-09-12T02:11:42.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.796+0000 7f2c974fc740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2c901ad640 con 0x55bdb73f75d0 2024-09-12T02:11:42.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.796+0000 7f2c974fc740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f2c901ab4e0 con 0x55bdb7402370 2024-09-12T02:11:42.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.796+0000 7f2c8e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bdb73f8b40 0x55bdb73fa120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:42.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.796+0000 7f2c8e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bdb73f8b40 0x55bdb73fa120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35904/0 (socket says 172.21.15.16:35904) 2024-09-12T02:11:42.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.796+0000 7f2c8e59c640 1 -- 172.21.15.16:0/3358886032 learned_addr learned my addr 172.21.15.16:0/3358886032 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:11:42.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.797+0000 7f2c8f59e640 1 --2- 172.21.15.16:0/3358886032 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bdb7402370 0x55bdb73fa660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:42.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.797+0000 7f2c8e59c640 1 -- 172.21.15.16:0/3358886032 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bdb7402370 msgr2=0x55bdb73fa660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:42.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.797+0000 7f2c8ed9d640 1 --2- 172.21.15.16:0/3358886032 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bdb73f75d0 0x55bdb7401370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:42.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.797+0000 7f2c8e59c640 1 --2- 172.21.15.16:0/3358886032 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bdb7402370 0x55bdb73fa660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:42.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.797+0000 7f2c8e59c640 1 -- 172.21.15.16:0/3358886032 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bdb73f75d0 msgr2=0x55bdb7401370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:42.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.797+0000 7f2c8e59c640 1 --2- 172.21.15.16:0/3358886032 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bdb73f75d0 0x55bdb7401370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:42.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.797+0000 7f2c8e59c640 1 -- 172.21.15.16:0/3358886032 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bdb73fffe0 con 0x55bdb73f8b40 2024-09-12T02:11:42.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.797+0000 7f2c8f59e640 1 --2- 172.21.15.16:0/3358886032 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bdb7402370 0x55bdb73fa660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:11:42.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.797+0000 7f2c8e59c640 1 --2- 172.21.15.16:0/3358886032 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bdb73f8b40 0x55bdb73fa120 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f2c84031630 tx=0x7f2c840316e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=756ecaf523e3ca85 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:42.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.797+0000 7f2c74ff9640 1 -- 172.21.15.16:0/3358886032 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c84035070 con 0x55bdb73f8b40 2024-09-12T02:11:42.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.797+0000 7f2c74ff9640 1 -- 172.21.15.16:0/3358886032 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c8404c070 con 0x55bdb73f8b40 2024-09-12T02:11:42.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.797+0000 7f2c74ff9640 1 -- 172.21.15.16:0/3358886032 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c84005280 con 0x55bdb73f8b40 2024-09-12T02:11:42.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.798+0000 7f2c974fc740 1 -- 172.21.15.16:0/3358886032 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bdb73f8b40 msgr2=0x55bdb73fa120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:42.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.798+0000 7f2c974fc740 1 --2- 172.21.15.16:0/3358886032 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bdb73f8b40 0x55bdb73fa120 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f2c84031630 tx=0x7f2c840316e0 comp rx=0 tx=0).stop 2024-09-12T02:11:42.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.798+0000 7f2c974fc740 1 -- 172.21.15.16:0/3358886032 shutdown_connections 2024-09-12T02:11:42.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.798+0000 7f2c974fc740 1 --2- 172.21.15.16:0/3358886032 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bdb7402370 0x55bdb73fa660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:42.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.798+0000 7f2c974fc740 1 --2- 172.21.15.16:0/3358886032 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bdb73f8b40 0x55bdb73fa120 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:42.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.798+0000 7f2c974fc740 1 --2- 172.21.15.16:0/3358886032 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bdb73f75d0 0x55bdb7401370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:42.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.798+0000 7f2c974fc740 1 -- 172.21.15.16:0/3358886032 >> 172.21.15.16:0/3358886032 conn(0x55bdb73c0310 msgr2=0x55bdb73f2d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:42.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.798+0000 7f2c974fc740 1 -- 172.21.15.16:0/3358886032 shutdown_connections 2024-09-12T02:11:42.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.798+0000 7f2c974fc740 1 -- 172.21.15.16:0/3358886032 wait complete. 2024-09-12T02:11:42.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.798+0000 7f2c974fc740 1 -- start start 2024-09-12T02:11:42.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.798+0000 7f2c974fc740 1 Processor -- start 2024-09-12T02:11:42.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.798+0000 7f2c974fc740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bdb73f75d0 0x55bdb7415010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:42.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.798+0000 7f2c974fc740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bdb73f8b40 0x55bdb7415550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:42.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.798+0000 7f2c974fc740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bdb7402370 0x55bdb7415a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:42.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.798+0000 7f2c974fc740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2c901abb50 con 0x55bdb73f75d0 2024-09-12T02:11:42.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.798+0000 7f2c974fc740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2c901acf70 con 0x55bdb73f8b40 2024-09-12T02:11:42.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.798+0000 7f2c974fc740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f2c901ac1d0 con 0x55bdb7402370 2024-09-12T02:11:42.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.798+0000 7f2c974fc740 10 client.0 _ll_register_callbacks cb 0x7f2c880084d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:11:42.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.799+0000 7f2c8ed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bdb73f75d0 0x55bdb7415010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:42.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.799+0000 7f2c8ed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bdb73f75d0 0x55bdb7415010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35918/0 (socket says 172.21.15.16:35918) 2024-09-12T02:11:42.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.799+0000 7f2c8ed9d640 1 -- 172.21.15.16:0/2344358202 learned_addr learned my addr 172.21.15.16:0/2344358202 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:11:42.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.799+0000 7f2c8e59c640 1 --2- 172.21.15.16:0/2344358202 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bdb73f8b40 0x55bdb7415550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:42.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.799+0000 7f2c8f59e640 1 --2- 172.21.15.16:0/2344358202 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bdb7402370 0x55bdb7415a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:42.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.799+0000 7f2c8ed9d640 1 -- 172.21.15.16:0/2344358202 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bdb7402370 msgr2=0x55bdb7415a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:42.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.799+0000 7f2c8ed9d640 1 --2- 172.21.15.16:0/2344358202 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bdb7402370 0x55bdb7415a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:42.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.799+0000 7f2c8ed9d640 1 -- 172.21.15.16:0/2344358202 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bdb73f8b40 msgr2=0x55bdb7415550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:42.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.799+0000 7f2c8ed9d640 1 --2- 172.21.15.16:0/2344358202 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bdb73f8b40 0x55bdb7415550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:42.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.799+0000 7f2c8ed9d640 1 -- 172.21.15.16:0/2344358202 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bdb747fb80 con 0x55bdb73f75d0 2024-09-12T02:11:42.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.799+0000 7f2c8e59c640 1 --2- 172.21.15.16:0/2344358202 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bdb73f8b40 0x55bdb7415550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:11:42.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.799+0000 7f2c8ed9d640 1 --2- 172.21.15.16:0/2344358202 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bdb73f75d0 0x55bdb7415010 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f2c80097690 tx=0x7f2c8000bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fee349417c338905 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:42.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.799+0000 7f2c72ff5640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:11:42.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.799+0000 7f2c72ff5640 1 -- 172.21.15.16:0/2344358202 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c8001e040 con 0x55bdb73f75d0 2024-09-12T02:11:42.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.799+0000 7f2c72ff5640 1 -- 172.21.15.16:0/2344358202 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f2c80017360 con 0x55bdb73f75d0 2024-09-12T02:11:42.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.799+0000 7f2c974fc740 1 -- 172.21.15.16:0/2344358202 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c8404c070 con 0x55bdb73f75d0 2024-09-12T02:11:42.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.799+0000 7f2c72ff5640 1 -- 172.21.15.16:0/2344358202 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2c8009f070 con 0x55bdb73f75d0 2024-09-12T02:11:42.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.799+0000 7f2c974fc740 10 client.18411 fetch_fsmap learned FSMap version 25 2024-09-12T02:11:42.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.799+0000 7f2c974fc740 1 -- 172.21.15.16:0/2344358202 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55bdb7480e30 con 0x55bdb73f75d0 2024-09-12T02:11:42.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.800+0000 7f2c72ff5640 1 -- 172.21.15.16:0/2344358202 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c8404c070 con 0x55bdb73f75d0 2024-09-12T02:11:42.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.800+0000 7f2c72ff5640 1 -- 172.21.15.16:0/2344358202 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f2c800a3530 con 0x55bdb73f75d0 2024-09-12T02:11:42.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.800+0000 7f2c974fc740 10 client.18411 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:11:42.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.800+0000 7f2c974fc740 10 client.18411 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:11:42.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.800+0000 7f2c974fc740 20 client.18411 populate_metadata read hostname 'smithi016' 2024-09-12T02:11:42.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.800+0000 7f2c974fc740 1 --2- 172.21.15.16:0/2344358202 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55bdb7481550 0x55bdb7483970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:42.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.800+0000 7f2c974fc740 4 client.18411 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T02:11:42.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.800+0000 7f2c974fc740 1 -- 172.21.15.16:0/2344358202 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bdb7364d70 con 0x55bdb7481550 2024-09-12T02:11:42.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.800+0000 7f2c8e59c640 1 --2- 172.21.15.16:0/2344358202 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55bdb7481550 0x55bdb7483970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:42.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.801+0000 7f2c8e59c640 1 --2- 172.21.15.16:0/2344358202 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55bdb7481550 0x55bdb7483970 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c4e8daa593353ecf server_cookie=8a3e0070b0280b5d in_seq=0 out_seq=0 2024-09-12T02:11:42.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.801+0000 7f2c72ff5640 10 client.18411 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:11:42.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.801+0000 7f2c72ff5640 1 -- 172.21.15.16:0/2344358202 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bdb7364d70 con 0x55bdb7481550 2024-09-12T02:11:42.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.801+0000 7f2c72ff5640 10 client.18411 handle_command_reply: tid=0 2024-09-12T02:11:42.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.801+0000 7f2c974fc740 1 client.18411 shutdown 2024-09-12T02:11:42.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.801+0000 7f2c974fc740 1 -- 172.21.15.16:0/2344358202 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bdb73f75d0 msgr2=0x55bdb7415010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:42.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.801+0000 7f2c974fc740 1 --2- 172.21.15.16:0/2344358202 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bdb73f75d0 0x55bdb7415010 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f2c80097690 tx=0x7f2c8000bf90 comp rx=0 tx=0).stop 2024-09-12T02:11:42.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.801+0000 7f2c974fc740 1 -- 172.21.15.16:0/2344358202 shutdown_connections 2024-09-12T02:11:42.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.801+0000 7f2c974fc740 1 --2- 172.21.15.16:0/2344358202 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55bdb7481550 0x55bdb7483970 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:42.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.801+0000 7f2c974fc740 1 --2- 172.21.15.16:0/2344358202 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bdb7402370 0x55bdb7415a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:42.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.801+0000 7f2c974fc740 1 --2- 172.21.15.16:0/2344358202 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bdb73f8b40 0x55bdb7415550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:42.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.801+0000 7f2c974fc740 1 --2- 172.21.15.16:0/2344358202 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bdb73f75d0 0x55bdb7415010 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:42.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.801+0000 7f2c974fc740 1 -- 172.21.15.16:0/2344358202 >> 172.21.15.16:0/2344358202 conn(0x55bdb73c0310 msgr2=0x55bdb73fd7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:42.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.801+0000 7f2c72ff5640 0 client.18411 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:11:42.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.801+0000 7f2c974fc740 1 -- 172.21.15.16:0/2344358202 shutdown_connections 2024-09-12T02:11:42.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.801+0000 7f2c974fc740 1 -- 172.21.15.16:0/2344358202 wait complete. 2024-09-12T02:11:42.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.801+0000 7f2c974fc740 20 client.18411 trim_cache size 0 max 16384 2024-09-12T02:11:42.812 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T02:11:42.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.802+0000 7f2c95123640 1 -- 172.21.15.16:0/3758969677 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f2c780a2b80 msgr2=0x7f2c780a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:42.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.802+0000 7f2c95123640 1 --2- 172.21.15.16:0/3758969677 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f2c780a2b80 0x7f2c780a5040 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f2c88005fd0 tx=0x7f2c88008630 comp rx=0 tx=0).stop 2024-09-12T02:11:42.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.802+0000 7f2c95123640 1 -- 172.21.15.16:0/3758969677 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c90102da0 msgr2=0x7f2c90198500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:42.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.802+0000 7f2c95123640 1 --2- 172.21.15.16:0/3758969677 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c90102da0 0x7f2c90198500 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f2c8000b4a0 tx=0x7f2c8000b960 comp rx=0 tx=0).stop 2024-09-12T02:11:42.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.802+0000 7f2c95123640 1 -- 172.21.15.16:0/3758969677 shutdown_connections 2024-09-12T02:11:42.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.802+0000 7f2c95123640 1 --2- 172.21.15.16:0/3758969677 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f2c780a2b80 0x7f2c780a5040 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:42.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.802+0000 7f2c95123640 1 --2- 172.21.15.16:0/3758969677 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2c9010c180 0x7f2c9019cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:42.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.802+0000 7f2c95123640 1 --2- 172.21.15.16:0/3758969677 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c901036c0 0x7f2c90198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:42.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.802+0000 7f2c95123640 1 --2- 172.21.15.16:0/3758969677 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c90102da0 0x7f2c90198500 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:42.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.802+0000 7f2c95123640 1 -- 172.21.15.16:0/3758969677 >> 172.21.15.16:0/3758969677 conn(0x7f2c900fc7a0 msgr2=0x7f2c900fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:42.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.802+0000 7f2c95123640 1 -- 172.21.15.16:0/3758969677 shutdown_connections 2024-09-12T02:11:42.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:42.802+0000 7f2c95123640 1 -- 172.21.15.16:0/3758969677 wait complete. 2024-09-12T02:11:43.616 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:11:43.876 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:11:44.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:44.337+0000 7ff0b525c640 1 -- 172.21.15.16:0/3507768909 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0b0100a60 msgr2=0x7ff0b0100ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:44.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:44.337+0000 7ff0b525c640 1 --2- 172.21.15.16:0/3507768909 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0b0100a60 0x7ff0b0100ec0 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7ff098009930 tx=0x7ff09802a420 comp rx=0 tx=0).stop 2024-09-12T02:11:44.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:44.337+0000 7ff0b525c640 1 -- 172.21.15.16:0/3507768909 shutdown_connections 2024-09-12T02:11:44.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:44.337+0000 7ff0b525c640 1 --2- 172.21.15.16:0/3507768909 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0b0101400 0x7ff0b010d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:44.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:44.337+0000 7ff0b525c640 1 --2- 172.21.15.16:0/3507768909 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0b0100a60 0x7ff0b0100ec0 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:44.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:44.337+0000 7ff0b525c640 1 --2- 172.21.15.16:0/3507768909 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff0b0106a60 0x7ff0b0106e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:44.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:44.337+0000 7ff0b525c640 1 -- 172.21.15.16:0/3507768909 >> 172.21.15.16:0/3507768909 conn(0x7ff0b00fc780 msgr2=0x7ff0b00feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:44.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:44.338+0000 7ff0b525c640 1 -- 172.21.15.16:0/3507768909 shutdown_connections 2024-09-12T02:11:44.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:44.338+0000 7ff0b525c640 1 -- 172.21.15.16:0/3507768909 wait complete. 2024-09-12T02:11:44.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:44.338+0000 7ff0b525c640 1 Processor -- start 2024-09-12T02:11:44.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:44.338+0000 7ff0b525c640 1 -- start start 2024-09-12T02:11:44.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:44.339+0000 7ff0b525c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0b0100a60 0x7ff0b01985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:44.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:44.339+0000 7ff0b525c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0b0101400 0x7ff0b0198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:44.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:44.339+0000 7ff0b525c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff0b0106a60 0x7ff0b019ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:44.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:44.339+0000 7ff0b525c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff0b0110f50 con 0x7ff0b0100a60 2024-09-12T02:11:44.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:44.339+0000 7ff0b525c640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff0b0110dd0 con 0x7ff0b0101400 2024-09-12T02:11:44.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:44.339+0000 7ff0b525c640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff0b01110d0 con 0x7ff0b0106a60 2024-09-12T02:11:44.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:44.339+0000 7ff0aed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0b0100a60 0x7ff0b01985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:44.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:44.339+0000 7ff0aed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0b0100a60 0x7ff0b01985c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35934/0 (socket says 172.21.15.16:35934) 2024-09-12T02:11:44.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:44.339+0000 7ff0aed9d640 1 -- 172.21.15.16:0/2870295197 learned_addr learned my addr 172.21.15.16:0/2870295197 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:11:44.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:44.339+0000 7ff0a7fff640 1 --2- 172.21.15.16:0/2870295197 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0b0101400 0x7ff0b0198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:44.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:44.339+0000 7ff0af59e640 1 --2- 172.21.15.16:0/2870295197 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff0b0106a60 0x7ff0b019ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:44.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:44.339+0000 7ff0aed9d640 1 -- 172.21.15.16:0/2870295197 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff0b0106a60 msgr2=0x7ff0b019ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:44.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:44.339+0000 7ff0aed9d640 1 --2- 172.21.15.16:0/2870295197 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff0b0106a60 0x7ff0b019ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:44.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:44.339+0000 7ff0aed9d640 1 -- 172.21.15.16:0/2870295197 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0b0101400 msgr2=0x7ff0b0198b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:44.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:44.339+0000 7ff0aed9d640 1 --2- 172.21.15.16:0/2870295197 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0b0101400 0x7ff0b0198b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:44.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:44.339+0000 7ff0aed9d640 1 -- 172.21.15.16:0/2870295197 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0b019d570 con 0x7ff0b0100a60 2024-09-12T02:11:44.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:44.339+0000 7ff0aed9d640 1 --2- 172.21.15.16:0/2870295197 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0b0100a60 0x7ff0b01985c0 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7ff0a000b4a0 tx=0x7ff0a000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:44.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:44.339+0000 7ff0acd99640 1 -- 172.21.15.16:0/2870295197 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0a0014070 con 0x7ff0b0100a60 2024-09-12T02:11:44.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:44.339+0000 7ff0acd99640 1 -- 172.21.15.16:0/2870295197 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0a0010af0 con 0x7ff0b0100a60 2024-09-12T02:11:44.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:44.339+0000 7ff0b525c640 1 -- 172.21.15.16:0/2870295197 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0b019d860 con 0x7ff0b0100a60 2024-09-12T02:11:44.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:44.339+0000 7ff0acd99640 1 -- 172.21.15.16:0/2870295197 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0a0017d10 con 0x7ff0b0100a60 2024-09-12T02:11:44.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:44.339+0000 7ff0b525c640 1 -- 172.21.15.16:0/2870295197 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0b01a5140 con 0x7ff0b0100a60 2024-09-12T02:11:44.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:44.340+0000 7ff0b525c640 1 -- 172.21.15.16:0/2870295197 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0b0100f40 con 0x7ff0b0100a60 2024-09-12T02:11:44.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:44.342+0000 7ff0acd99640 1 -- 172.21.15.16:0/2870295197 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff0a0017070 con 0x7ff0b0100a60 2024-09-12T02:11:44.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:44.342+0000 7ff0acd99640 1 --2- 172.21.15.16:0/2870295197 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7ff09c0a2b80 0x7ff09c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:44.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:44.342+0000 7ff0a7fff640 1 --2- 172.21.15.16:0/2870295197 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7ff09c0a2b80 0x7ff09c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:44.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:44.342+0000 7ff0acd99640 1 -- 172.21.15.16:0/2870295197 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7ff0a00ce700 con 0x7ff0b0100a60 2024-09-12T02:11:44.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:44.343+0000 7ff0a7fff640 1 --2- 172.21.15.16:0/2870295197 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7ff09c0a2b80 0x7ff09c0a5040 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7ff098006010 tx=0x7ff098000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:44.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:44.346+0000 7ff0acd99640 1 -- 172.21.15.16:0/2870295197 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff0a009b700 con 0x7ff0b0100a60 2024-09-12T02:11:44.541 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-12T02:11:44.541 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-12T02:11:44.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:44.559+0000 7ff0b525c640 1 -- 172.21.15.16:0/2870295197 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff0b0199920 con 0x7ff0b0100a60 2024-09-12T02:11:44.559 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:11:44 smithi016 ceph-mon[30947]: pgmap v629: 97 pgs: 97 active+clean; 1.1 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.7 KiB/s wr, 8 op/s 2024-09-12T02:11:44.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:44.559+0000 7ff0acd99640 1 -- 172.21.15.16:0/2870295197 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11911 (secure 0 0 0) 0x7ff0b0199920 con 0x7ff0b0100a60 2024-09-12T02:11:44.560 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T02:11:44.560 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":25,"btime":"2024-09-12T02:01:37:192021+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3279123279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3279123279},{"type":"v1","addr":"172.21.15.16:6835","nonce":3279123279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4119452382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4119452382},{"type":"v1","addr":"172.21.15.16:6837","nonce":4119452382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/3263823165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3263823165},{"type":"v1","addr":"172.21.15.16:6839","nonce":3263823165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/1510290732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":1510290732},{"type":"v1","addr":"172.21.15.16:6841","nonce":1510290732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/279674603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":279674603},{"type":"v1","addr":"172.21.15.33:6833","nonce":279674603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3555728028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3555728028},{"type":"v1","addr":"172.21.15.33:6835","nonce":3555728028}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T01:59:56.117608+0000","modified":"2024-09-12T02:01:36.294760+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24482,"mds_1":24481,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6833/1563374106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1563374106},{"type":"v1","addr":"172.21.15.184:6833","nonce":1563374106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6837/1892833868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1892833868},{"type":"v1","addr":"172.21.15.33:6837","nonce":1892833868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/1457679686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1457679686},{"type":"v1","addr":"172.21.15.184:6835","nonce":1457679686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.184:6837/1793393087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":1793393087},{"type":"v1","addr":"172.21.15.184:6837","nonce":1793393087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.184:6839/4203347941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":4203347941},{"type":"v1","addr":"172.21.15.184:6839","nonce":4203347941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.33:6839/4136670204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":4136670204},{"type":"v1","addr":"172.21.15.33:6839","nonce":4136670204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24481,24482,24469]},"id":1}]} 2024-09-12T02:11:44.560 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 25 2024-09-12T02:11:44.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:44.562+0000 7ff0b525c640 1 -- 172.21.15.16:0/2870295197 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7ff09c0a2b80 msgr2=0x7ff09c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:44.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:44.562+0000 7ff0b525c640 1 --2- 172.21.15.16:0/2870295197 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7ff09c0a2b80 0x7ff09c0a5040 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7ff098006010 tx=0x7ff098000bc0 comp rx=0 tx=0).stop 2024-09-12T02:11:44.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:44.562+0000 7ff0b525c640 1 -- 172.21.15.16:0/2870295197 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0b0100a60 msgr2=0x7ff0b01985c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:44.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:44.562+0000 7ff0b525c640 1 --2- 172.21.15.16:0/2870295197 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0b0100a60 0x7ff0b01985c0 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7ff0a000b4a0 tx=0x7ff0a000b960 comp rx=0 tx=0).stop 2024-09-12T02:11:44.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:44.562+0000 7ff0b525c640 1 -- 172.21.15.16:0/2870295197 shutdown_connections 2024-09-12T02:11:44.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:44.562+0000 7ff0b525c640 1 --2- 172.21.15.16:0/2870295197 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7ff09c0a2b80 0x7ff09c0a5040 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:44.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:44.562+0000 7ff0b525c640 1 --2- 172.21.15.16:0/2870295197 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff0b0106a60 0x7ff0b019ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:44.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:44.562+0000 7ff0b525c640 1 --2- 172.21.15.16:0/2870295197 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0b0101400 0x7ff0b0198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:44.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:44.562+0000 7ff0b525c640 1 --2- 172.21.15.16:0/2870295197 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0b0100a60 0x7ff0b01985c0 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:44.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:44.562+0000 7ff0b525c640 1 -- 172.21.15.16:0/2870295197 >> 172.21.15.16:0/2870295197 conn(0x7ff0b00fc780 msgr2=0x7ff0b01048d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:44.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:44.562+0000 7ff0b525c640 1 -- 172.21.15.16:0/2870295197 shutdown_connections 2024-09-12T02:11:44.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:44.562+0000 7ff0b525c640 1 -- 172.21.15.16:0/2870295197 wait complete. 2024-09-12T02:11:44.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:11:44 smithi033 ceph-mon[35332]: pgmap v629: 97 pgs: 97 active+clean; 1.1 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.7 KiB/s wr, 8 op/s 2024-09-12T02:11:44.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:11:44 smithi184 ceph-mon[35593]: pgmap v629: 97 pgs: 97 active+clean; 1.1 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.7 KiB/s wr, 8 op/s 2024-09-12T02:11:45.285 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24482 quiesce db --set-id 998c7ef0 --release --await' 2024-09-12T02:11:45.285 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24482 quiesce db --set-id 998c7ef0 --release --await 2024-09-12T02:11:45.314 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:11:45.592 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:11:45 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/2870295197' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:11:45.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:11:45 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/2870295197' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:11:45.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.828+0000 7f193951b640 1 -- 172.21.15.16:0/2307665020 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1934106a90 msgr2=0x7f1934106e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:45.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.828+0000 7f193951b640 1 --2- 172.21.15.16:0/2307665020 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1934106a90 0x7f1934106e70 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f1924009930 tx=0x7f192402a420 comp rx=0 tx=0).stop 2024-09-12T02:11:45.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.828+0000 7f193951b640 1 -- 172.21.15.16:0/2307665020 shutdown_connections 2024-09-12T02:11:45.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.828+0000 7f193951b640 1 --2- 172.21.15.16:0/2307665020 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1934101430 0x7f193410d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:45.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.828+0000 7f193951b640 1 --2- 172.21.15.16:0/2307665020 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1934100a90 0x7f1934100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:45.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.828+0000 7f193951b640 1 --2- 172.21.15.16:0/2307665020 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1934106a90 0x7f1934106e70 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:45.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.828+0000 7f193951b640 1 -- 172.21.15.16:0/2307665020 >> 172.21.15.16:0/2307665020 conn(0x7f19340fc7d0 msgr2=0x7f19340febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:45.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.828+0000 7f193951b640 1 -- 172.21.15.16:0/2307665020 shutdown_connections 2024-09-12T02:11:45.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.828+0000 7f193951b640 1 -- 172.21.15.16:0/2307665020 wait complete. 2024-09-12T02:11:45.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.828+0000 7f193951b640 1 Processor -- start 2024-09-12T02:11:45.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.829+0000 7f193951b640 1 -- start start 2024-09-12T02:11:45.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.829+0000 7f193951b640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1934100a90 0x7f1934198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:45.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.829+0000 7f193951b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1934101430 0x7f19341989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:45.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.829+0000 7f193951b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1934106a90 0x7f193419cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:45.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.829+0000 7f193951b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1934110e90 con 0x7f1934101430 2024-09-12T02:11:45.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.829+0000 7f193951b640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1934110d10 con 0x7f1934106a90 2024-09-12T02:11:45.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.829+0000 7f193951b640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1934111010 con 0x7f1934100a90 2024-09-12T02:11:45.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.829+0000 7f19327fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1934101430 0x7f19341989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:45.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.829+0000 7f19327fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1934101430 0x7f19341989b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35954/0 (socket says 172.21.15.16:35954) 2024-09-12T02:11:45.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.829+0000 7f19327fc640 1 -- 172.21.15.16:0/905542030 learned_addr learned my addr 172.21.15.16:0/905542030 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:11:45.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.829+0000 7f1932ffd640 1 --2- 172.21.15.16:0/905542030 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1934100a90 0x7f1934198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:45.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.829+0000 7f19337fe640 1 --2- 172.21.15.16:0/905542030 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1934106a90 0x7f193419cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:45.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.829+0000 7f19327fc640 1 -- 172.21.15.16:0/905542030 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1934100a90 msgr2=0x7f1934198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:45.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.829+0000 7f19327fc640 1 --2- 172.21.15.16:0/905542030 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1934100a90 0x7f1934198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:45.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.829+0000 7f19327fc640 1 -- 172.21.15.16:0/905542030 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1934106a90 msgr2=0x7f193419cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:45.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.829+0000 7f19327fc640 1 --2- 172.21.15.16:0/905542030 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1934106a90 0x7f193419cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:45.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.829+0000 7f19327fc640 1 -- 172.21.15.16:0/905542030 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f193419d420 con 0x7f1934101430 2024-09-12T02:11:45.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.830+0000 7f19327fc640 1 --2- 172.21.15.16:0/905542030 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1934101430 0x7f19341989b0 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f192800d950 tx=0x7f192800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:45.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.830+0000 7f191bfff640 1 -- 172.21.15.16:0/905542030 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1928017070 con 0x7f1934101430 2024-09-12T02:11:45.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.830+0000 7f193951b640 1 -- 172.21.15.16:0/905542030 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f193419d710 con 0x7f1934101430 2024-09-12T02:11:45.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.830+0000 7f191bfff640 1 -- 172.21.15.16:0/905542030 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1928009c30 con 0x7f1934101430 2024-09-12T02:11:45.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.830+0000 7f191bfff640 1 -- 172.21.15.16:0/905542030 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f192800f6b0 con 0x7f1934101430 2024-09-12T02:11:45.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.830+0000 7f193951b640 1 -- 172.21.15.16:0/905542030 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19341a4ff0 con 0x7f1934101430 2024-09-12T02:11:45.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.832+0000 7f191bfff640 1 -- 172.21.15.16:0/905542030 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f19280109d0 con 0x7f1934101430 2024-09-12T02:11:45.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.833+0000 7f193b8f4740 1 Processor -- start 2024-09-12T02:11:45.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.833+0000 7f193b8f4740 1 -- start start 2024-09-12T02:11:45.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.833+0000 7f193b8f4740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564f552f5a20 0x564f552f5e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:45.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.833+0000 7f193b8f4740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564f552f6340 0x564f55304c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:45.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.833+0000 7f193b8f4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564f55305150 0x564f55307540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:45.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.833+0000 7f193b8f4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x564f5525b280 con 0x564f55305150 2024-09-12T02:11:45.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.833+0000 7f193b8f4740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x564f55202b10 con 0x564f552f6340 2024-09-12T02:11:45.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.833+0000 7f193b8f4740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x564f55257240 con 0x564f552f5a20 2024-09-12T02:11:45.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.833+0000 7f19327fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564f55305150 0x564f55307540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:45.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.833+0000 7f19327fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564f55305150 0x564f55307540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35958/0 (socket says 172.21.15.16:35958) 2024-09-12T02:11:45.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.833+0000 7f19327fc640 1 -- 172.21.15.16:0/3318334363 learned_addr learned my addr 172.21.15.16:0/3318334363 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:11:45.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.833+0000 7f19337fe640 1 --2- 172.21.15.16:0/3318334363 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564f552f5a20 0x564f552f5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:45.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.833+0000 7f1932ffd640 1 --2- 172.21.15.16:0/3318334363 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564f552f6340 0x564f55304c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:45.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.833+0000 7f191bfff640 1 --2- 172.21.15.16:0/905542030 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f191c0a2b80 0x7f191c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:45.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.834+0000 7f19337fe640 1 -- 172.21.15.16:0/3318334363 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564f552f6340 msgr2=0x564f55304c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:45.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.834+0000 7f19337fe640 1 --2- 172.21.15.16:0/3318334363 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564f552f6340 0x564f55304c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:45.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.834+0000 7f19337fe640 1 -- 172.21.15.16:0/3318334363 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564f55305150 msgr2=0x564f55307540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:45.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.834+0000 7f19337fe640 1 --2- 172.21.15.16:0/3318334363 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564f55305150 0x564f55307540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:45.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.834+0000 7f19337fe640 1 -- 172.21.15.16:0/3318334363 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f5520f1f0 con 0x564f552f5a20 2024-09-12T02:11:45.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.834+0000 7f1932ffd640 1 --2- 172.21.15.16:0/3318334363 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564f552f6340 0x564f55304c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:11:45.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.834+0000 7f191bfff640 1 -- 172.21.15.16:0/905542030 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f19280cecb0 con 0x7f1934101430 2024-09-12T02:11:45.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.834+0000 7f19337fe640 1 --2- 172.21.15.16:0/905542030 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f191c0a2b80 0x7f191c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:45.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.834+0000 7f19337fe640 1 --2- 172.21.15.16:0/3318334363 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564f552f5a20 0x564f552f5e00 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f192c008e00 tx=0x7f192c00e720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f21baa53242b7f78 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:45.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.834+0000 7f19197fa640 1 -- 172.21.15.16:0/3318334363 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f192c022070 con 0x564f552f5a20 2024-09-12T02:11:45.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.834+0000 7f19197fa640 1 -- 172.21.15.16:0/3318334363 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f192c004b60 con 0x564f552f5a20 2024-09-12T02:11:45.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.834+0000 7f19197fa640 1 -- 172.21.15.16:0/3318334363 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f192c01c8b0 con 0x564f552f5a20 2024-09-12T02:11:45.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.834+0000 7f19337fe640 1 --2- 172.21.15.16:0/905542030 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f191c0a2b80 0x7f191c0a5040 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f192c01f000 tx=0x7f192c020000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:45.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.835+0000 7f193b8f4740 1 -- 172.21.15.16:0/3318334363 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564f552f5a20 msgr2=0x564f552f5e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:45.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.835+0000 7f193b8f4740 1 --2- 172.21.15.16:0/3318334363 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564f552f5a20 0x564f552f5e00 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f192c008e00 tx=0x7f192c00e720 comp rx=0 tx=0).stop 2024-09-12T02:11:45.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.835+0000 7f193b8f4740 1 -- 172.21.15.16:0/3318334363 shutdown_connections 2024-09-12T02:11:45.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.835+0000 7f193b8f4740 1 --2- 172.21.15.16:0/3318334363 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564f55305150 0x564f55307540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:45.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.835+0000 7f193b8f4740 1 --2- 172.21.15.16:0/3318334363 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564f552f6340 0x564f55304c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:45.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.835+0000 7f193b8f4740 1 --2- 172.21.15.16:0/3318334363 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564f552f5a20 0x564f552f5e00 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:45.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.835+0000 7f193b8f4740 1 -- 172.21.15.16:0/3318334363 >> 172.21.15.16:0/3318334363 conn(0x564f552f20f0 msgr2=0x564f552f24d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:45.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.835+0000 7f193b8f4740 1 -- 172.21.15.16:0/3318334363 shutdown_connections 2024-09-12T02:11:45.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.835+0000 7f193b8f4740 1 -- 172.21.15.16:0/3318334363 wait complete. 2024-09-12T02:11:45.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.836+0000 7f193b8f4740 1 -- start start 2024-09-12T02:11:45.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.836+0000 7f193b8f4740 1 Processor -- start 2024-09-12T02:11:45.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.836+0000 7f193b8f4740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564f552f5a20 0x564f553020f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:45.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.836+0000 7f193b8f4740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564f552f6340 0x564f55302630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:45.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.836+0000 7f193b8f4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564f55305150 0x564f55302b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:45.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.836+0000 7f193b8f4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x564f55307d20 con 0x564f55305150 2024-09-12T02:11:45.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.836+0000 7f193b8f4740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x564f55215f10 con 0x564f552f5a20 2024-09-12T02:11:45.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.836+0000 7f193b8f4740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x564f55307ea0 con 0x564f552f6340 2024-09-12T02:11:45.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.836+0000 7f193b8f4740 10 client.0 _ll_register_callbacks cb 0x564f551dea40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:11:45.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.836+0000 7f19327fc640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564f552f6340 0x564f55302630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:45.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.836+0000 7f19337fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564f55305150 0x564f55302b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:45.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.836+0000 7f19327fc640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564f552f6340 0x564f55302630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:41976/0 (socket says 172.21.15.16:41976) 2024-09-12T02:11:45.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.836+0000 7f19327fc640 1 -- 172.21.15.16:0/2038847025 learned_addr learned my addr 172.21.15.16:0/2038847025 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:11:45.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.836+0000 7f19337fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564f55305150 0x564f55302b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35968/0 (socket says 172.21.15.16:35968) 2024-09-12T02:11:45.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.836+0000 7f1932ffd640 1 --2- 172.21.15.16:0/2038847025 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564f552f5a20 0x564f553020f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:45.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.837+0000 7f19337fe640 1 -- 172.21.15.16:0/2038847025 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564f552f6340 msgr2=0x564f55302630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:45.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.837+0000 7f19337fe640 1 --2- 172.21.15.16:0/2038847025 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564f552f6340 0x564f55302630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:45.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.837+0000 7f19337fe640 1 -- 172.21.15.16:0/2038847025 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564f552f5a20 msgr2=0x564f553020f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:45.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.837+0000 7f19337fe640 1 --2- 172.21.15.16:0/2038847025 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564f552f5a20 0x564f553020f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:45.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.837+0000 7f19337fe640 1 -- 172.21.15.16:0/2038847025 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f54fcc6a0 con 0x564f55305150 2024-09-12T02:11:45.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.837+0000 7f19337fe640 1 --2- 172.21.15.16:0/2038847025 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564f55305150 0x564f55302b70 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f192c014010 tx=0x7f192c008570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70f66f25421e0d75 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:45.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.837+0000 7f1916ff5640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:11:45.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.837+0000 7f1916ff5640 1 -- 172.21.15.16:0/2038847025 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f192c022030 con 0x564f55305150 2024-09-12T02:11:45.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.837+0000 7f193b8f4740 1 -- 172.21.15.16:0/2038847025 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f192c004b60 con 0x564f55305150 2024-09-12T02:11:45.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.837+0000 7f1916ff5640 1 -- 172.21.15.16:0/2038847025 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f192c011a30 con 0x564f55305150 2024-09-12T02:11:45.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.837+0000 7f1916ff5640 1 -- 172.21.15.16:0/2038847025 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f192c008900 con 0x564f55305150 2024-09-12T02:11:45.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.837+0000 7f193b8f4740 10 client.18429 fetch_fsmap learned FSMap version 25 2024-09-12T02:11:45.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.837+0000 7f193b8f4740 1 -- 172.21.15.16:0/2038847025 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x564f55313a20 con 0x564f55305150 2024-09-12T02:11:45.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.838+0000 7f1916ff5640 1 -- 172.21.15.16:0/2038847025 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f192c004b60 con 0x564f55305150 2024-09-12T02:11:45.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.838+0000 7f1916ff5640 1 -- 172.21.15.16:0/2038847025 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f192c02f020 con 0x564f55305150 2024-09-12T02:11:45.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.838+0000 7f193b8f4740 10 client.18429 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:11:45.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.838+0000 7f193b8f4740 10 client.18429 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:11:45.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.838+0000 7f193b8f4740 20 client.18429 populate_metadata read hostname 'smithi016' 2024-09-12T02:11:45.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.838+0000 7f193b8f4740 1 --2- 172.21.15.16:0/2038847025 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x564f55314d70 0x564f55317160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:45.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.838+0000 7f193b8f4740 4 client.18429 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:11:45.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.838+0000 7f193b8f4740 1 -- 172.21.15.16:0/2038847025 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564f552687f0 con 0x564f55314d70 2024-09-12T02:11:45.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.838+0000 7f1932ffd640 1 --2- 172.21.15.16:0/2038847025 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x564f55314d70 0x564f55317160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:45.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.839+0000 7f1916ff5640 10 client.18429 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:11:45.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.839+0000 7f1932ffd640 1 --2- 172.21.15.16:0/2038847025 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x564f55314d70 0x564f55317160 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fbea678f44c43285 server_cookie=923d31f99645802b in_seq=0 out_seq=0 2024-09-12T02:11:45.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.839+0000 7f1916ff5640 1 -- 172.21.15.16:0/2038847025 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564f552687f0 con 0x564f55314d70 2024-09-12T02:11:45.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.839+0000 7f1916ff5640 10 client.18429 handle_command_reply: tid=0 2024-09-12T02:11:45.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.840+0000 7f193b8f4740 1 client.18429 shutdown 2024-09-12T02:11:45.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.840+0000 7f193b8f4740 1 -- 172.21.15.16:0/2038847025 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564f55305150 msgr2=0x564f55302b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:45.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.840+0000 7f193b8f4740 1 --2- 172.21.15.16:0/2038847025 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564f55305150 0x564f55302b70 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f192c014010 tx=0x7f192c008570 comp rx=0 tx=0).stop 2024-09-12T02:11:45.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.840+0000 7f193b8f4740 1 -- 172.21.15.16:0/2038847025 shutdown_connections 2024-09-12T02:11:45.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.840+0000 7f193b8f4740 1 --2- 172.21.15.16:0/2038847025 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x564f55314d70 0x564f55317160 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:45.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.840+0000 7f193b8f4740 1 --2- 172.21.15.16:0/2038847025 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564f55305150 0x564f55302b70 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:45.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.840+0000 7f193b8f4740 1 --2- 172.21.15.16:0/2038847025 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564f552f6340 0x564f55302630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:45.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.840+0000 7f1916ff5640 0 client.18429 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:11:45.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.840+0000 7f193b8f4740 1 --2- 172.21.15.16:0/2038847025 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564f552f5a20 0x564f553020f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:45.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.840+0000 7f193b8f4740 1 -- 172.21.15.16:0/2038847025 >> 172.21.15.16:0/2038847025 conn(0x564f552f20f0 msgr2=0x564f552f7e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:45.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.840+0000 7f193b8f4740 1 -- 172.21.15.16:0/2038847025 shutdown_connections 2024-09-12T02:11:45.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.840+0000 7f193b8f4740 1 -- 172.21.15.16:0/2038847025 wait complete. 2024-09-12T02:11:45.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.840+0000 7f193b8f4740 20 client.18429 trim_cache size 0 max 16384 2024-09-12T02:11:45.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.858+0000 7f193b8f4740 1 Processor -- start 2024-09-12T02:11:45.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.858+0000 7f193b8f4740 1 -- start start 2024-09-12T02:11:45.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.859+0000 7f193b8f4740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564f552f5a20 0x564f553029c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:45.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.859+0000 7f193b8f4740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564f552f6340 0x564f5531c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:45.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.859+0000 7f193b8f4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564f55305150 0x564f5531cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:45.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.859+0000 7f193b8f4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f19341adbb0 con 0x564f55305150 2024-09-12T02:11:45.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.859+0000 7f193b8f4740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f19341ab480 con 0x564f552f6340 2024-09-12T02:11:45.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.859+0000 7f193b8f4740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f19341a92f0 con 0x564f552f5a20 2024-09-12T02:11:45.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.859+0000 7f19337fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564f55305150 0x564f5531cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:45.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.859+0000 7f19337fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564f55305150 0x564f5531cf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35984/0 (socket says 172.21.15.16:35984) 2024-09-12T02:11:45.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.859+0000 7f19337fe640 1 -- 172.21.15.16:0/4074700192 learned_addr learned my addr 172.21.15.16:0/4074700192 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:11:45.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.859+0000 7f1932ffd640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564f552f5a20 0x564f553029c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:45.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.859+0000 7f19327fc640 1 --2- 172.21.15.16:0/4074700192 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564f552f6340 0x564f5531c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:45.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.859+0000 7f1932ffd640 1 -- 172.21.15.16:0/4074700192 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564f552f6340 msgr2=0x564f5531c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:45.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.859+0000 7f1932ffd640 1 --2- 172.21.15.16:0/4074700192 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564f552f6340 0x564f5531c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:45.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.859+0000 7f1932ffd640 1 -- 172.21.15.16:0/4074700192 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564f55305150 msgr2=0x564f5531cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:45.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.859+0000 7f1932ffd640 1 --2- 172.21.15.16:0/4074700192 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564f55305150 0x564f5531cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:45.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.859+0000 7f1932ffd640 1 -- 172.21.15.16:0/4074700192 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f553049c0 con 0x564f552f5a20 2024-09-12T02:11:45.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.859+0000 7f19337fe640 1 --2- 172.21.15.16:0/4074700192 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564f55305150 0x564f5531cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:11:45.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.859+0000 7f19327fc640 1 --2- 172.21.15.16:0/4074700192 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564f552f6340 0x564f5531c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:11:45.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.860+0000 7f1932ffd640 1 --2- 172.21.15.16:0/4074700192 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564f552f5a20 0x564f553029c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1924005940 tx=0x7f1924000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dbfc929bd60ba73c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:45.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.860+0000 7f1918ff9640 1 -- 172.21.15.16:0/4074700192 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f192404c070 con 0x564f552f5a20 2024-09-12T02:11:45.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.860+0000 7f1918ff9640 1 -- 172.21.15.16:0/4074700192 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1924005500 con 0x564f552f5a20 2024-09-12T02:11:45.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.860+0000 7f1918ff9640 1 -- 172.21.15.16:0/4074700192 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1924035040 con 0x564f552f5a20 2024-09-12T02:11:45.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.861+0000 7f193b8f4740 1 -- 172.21.15.16:0/4074700192 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564f552f5a20 msgr2=0x564f553029c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:45.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.861+0000 7f193b8f4740 1 --2- 172.21.15.16:0/4074700192 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564f552f5a20 0x564f553029c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1924005940 tx=0x7f1924000bc0 comp rx=0 tx=0).stop 2024-09-12T02:11:45.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.861+0000 7f193b8f4740 1 -- 172.21.15.16:0/4074700192 shutdown_connections 2024-09-12T02:11:45.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.861+0000 7f193b8f4740 1 --2- 172.21.15.16:0/4074700192 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564f55305150 0x564f5531cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:45.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.861+0000 7f193b8f4740 1 --2- 172.21.15.16:0/4074700192 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564f552f6340 0x564f5531c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:45.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.861+0000 7f193b8f4740 1 --2- 172.21.15.16:0/4074700192 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564f552f5a20 0x564f553029c0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:45.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.861+0000 7f193b8f4740 1 -- 172.21.15.16:0/4074700192 >> 172.21.15.16:0/4074700192 conn(0x564f552f20f0 msgr2=0x564f55304060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:45.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.861+0000 7f193b8f4740 1 -- 172.21.15.16:0/4074700192 shutdown_connections 2024-09-12T02:11:45.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.861+0000 7f193b8f4740 1 -- 172.21.15.16:0/4074700192 wait complete. 2024-09-12T02:11:45.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.861+0000 7f193b8f4740 1 -- start start 2024-09-12T02:11:45.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.861+0000 7f193b8f4740 1 Processor -- start 2024-09-12T02:11:45.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.862+0000 7f193b8f4740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564f552f5a20 0x564f55308ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:45.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.862+0000 7f193b8f4740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564f552f6340 0x564f55309410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:45.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.862+0000 7f193b8f4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564f55305150 0x564f55309950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:45.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.862+0000 7f193b8f4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f19341a9940 con 0x564f55305150 2024-09-12T02:11:45.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.862+0000 7f193b8f4740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f19341aadb0 con 0x564f552f5a20 2024-09-12T02:11:45.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.862+0000 7f193b8f4740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f19341aa010 con 0x564f552f6340 2024-09-12T02:11:45.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.862+0000 7f1932ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564f552f5a20 0x564f55308ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:45.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.862+0000 7f193b8f4740 10 client.0 _ll_register_callbacks cb 0x7f192c020070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:11:45.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.862+0000 7f19337fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564f55305150 0x564f55309950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:45.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.862+0000 7f19327fc640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564f552f6340 0x564f55309410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:45.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.862+0000 7f19327fc640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564f552f6340 0x564f55309410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:41994/0 (socket says 172.21.15.16:41994) 2024-09-12T02:11:45.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.862+0000 7f19337fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564f55305150 0x564f55309950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35996/0 (socket says 172.21.15.16:35996) 2024-09-12T02:11:45.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.862+0000 7f19327fc640 1 -- 172.21.15.16:0/2914263927 learned_addr learned my addr 172.21.15.16:0/2914263927 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:11:45.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.862+0000 7f1932ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564f552f5a20 0x564f55308ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.16:51144/0 (socket says 172.21.15.16:51144) 2024-09-12T02:11:45.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.862+0000 7f19337fe640 1 -- 172.21.15.16:0/2914263927 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564f552f6340 msgr2=0x564f55309410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:45.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.862+0000 7f19337fe640 1 --2- 172.21.15.16:0/2914263927 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564f552f6340 0x564f55309410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:45.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.862+0000 7f19337fe640 1 -- 172.21.15.16:0/2914263927 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564f552f5a20 msgr2=0x564f55308ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:45.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.862+0000 7f19337fe640 1 --2- 172.21.15.16:0/2914263927 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564f552f5a20 0x564f55308ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:45.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.862+0000 7f19337fe640 1 -- 172.21.15.16:0/2914263927 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f5537ea00 con 0x564f55305150 2024-09-12T02:11:45.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.862+0000 7f19327fc640 1 --2- 172.21.15.16:0/2914263927 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564f552f6340 0x564f55309410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:11:45.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.862+0000 7f19337fe640 1 --2- 172.21.15.16:0/2914263927 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564f55305150 0x564f55309950 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f192c008bc0 tx=0x7f192c01c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce9b9fad5bb55512 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:45.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.862+0000 7f1916ff5640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:11:45.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.863+0000 7f1916ff5640 1 -- 172.21.15.16:0/2914263927 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f192c026880 con 0x564f55305150 2024-09-12T02:11:45.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.863+0000 7f1916ff5640 1 -- 172.21.15.16:0/2914263927 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f192c026a20 con 0x564f55305150 2024-09-12T02:11:45.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.863+0000 7f193b8f4740 1 -- 172.21.15.16:0/2914263927 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1924005500 con 0x564f55305150 2024-09-12T02:11:45.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.863+0000 7f1916ff5640 1 -- 172.21.15.16:0/2914263927 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f192c047050 con 0x564f55305150 2024-09-12T02:11:45.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.863+0000 7f193b8f4740 10 client.18435 fetch_fsmap learned FSMap version 25 2024-09-12T02:11:45.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.863+0000 7f193b8f4740 1 -- 172.21.15.16:0/2914263927 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x564f5537fd20 con 0x564f55305150 2024-09-12T02:11:45.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.863+0000 7f1916ff5640 1 -- 172.21.15.16:0/2914263927 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1924005500 con 0x564f55305150 2024-09-12T02:11:45.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.864+0000 7f1916ff5640 1 -- 172.21.15.16:0/2914263927 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f192c03e070 con 0x564f55305150 2024-09-12T02:11:45.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.864+0000 7f193b8f4740 10 client.18435 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:11:45.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.864+0000 7f193b8f4740 10 client.18435 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:11:45.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.864+0000 7f193b8f4740 20 client.18435 populate_metadata read hostname 'smithi016' 2024-09-12T02:11:45.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.864+0000 7f193b8f4740 1 --2- 172.21.15.16:0/2914263927 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x564f55380380 0x564f553827a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:45.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.864+0000 7f193b8f4740 4 client.18435 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-12T02:11:45.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.864+0000 7f193b8f4740 1 -- 172.21.15.16:0/2914263927 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x564f55262d00 con 0x564f55380380 2024-09-12T02:11:45.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.864+0000 7f1932ffd640 1 --2- 172.21.15.16:0/2914263927 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x564f55380380 0x564f553827a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:45.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.864+0000 7f1932ffd640 1 --2- 172.21.15.16:0/2914263927 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x564f55380380 0x564f553827a0 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e20afb10f5c155f server_cookie=15258d1d54113a5f in_seq=0 out_seq=0 2024-09-12T02:11:45.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.865+0000 7f1916ff5640 10 client.18435 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:11:45.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.865+0000 7f1916ff5640 1 -- 172.21.15.16:0/2914263927 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x564f55262d00 con 0x564f55380380 2024-09-12T02:11:45.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.866+0000 7f1916ff5640 10 client.18435 handle_command_reply: tid=0 2024-09-12T02:11:45.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.866+0000 7f193b8f4740 1 client.18435 shutdown 2024-09-12T02:11:45.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.866+0000 7f193b8f4740 1 -- 172.21.15.16:0/2914263927 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564f55305150 msgr2=0x564f55309950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:45.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.866+0000 7f193b8f4740 1 --2- 172.21.15.16:0/2914263927 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564f55305150 0x564f55309950 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f192c008bc0 tx=0x7f192c01c9a0 comp rx=0 tx=0).stop 2024-09-12T02:11:45.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.866+0000 7f193b8f4740 1 -- 172.21.15.16:0/2914263927 shutdown_connections 2024-09-12T02:11:45.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.866+0000 7f193b8f4740 1 --2- 172.21.15.16:0/2914263927 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x564f55380380 0x564f553827a0 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:45.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.866+0000 7f193b8f4740 1 --2- 172.21.15.16:0/2914263927 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564f55305150 0x564f55309950 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:45.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.866+0000 7f1916ff5640 0 client.18435 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:11:45.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.866+0000 7f193b8f4740 1 --2- 172.21.15.16:0/2914263927 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564f552f6340 0x564f55309410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:45.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.866+0000 7f193b8f4740 1 --2- 172.21.15.16:0/2914263927 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564f552f5a20 0x564f55308ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:45.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.866+0000 7f193b8f4740 1 -- 172.21.15.16:0/2914263927 >> 172.21.15.16:0/2914263927 conn(0x564f552f20f0 msgr2=0x564f552f0a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:45.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.866+0000 7f193b8f4740 1 -- 172.21.15.16:0/2914263927 shutdown_connections 2024-09-12T02:11:45.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.866+0000 7f193b8f4740 1 -- 172.21.15.16:0/2914263927 wait complete. 2024-09-12T02:11:45.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.866+0000 7f193b8f4740 20 client.18435 trim_cache size 0 max 16384 2024-09-12T02:11:45.872 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T02:11:45.872 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-09-12T02:11:45.872 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "a1df0cf1-b6e1-4c70-9b70-b69e675b14e5", 2024-09-12T02:11:45.872 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-09-12T02:11:45.872 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T02:11:45.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.867+0000 7f193951b640 1 -- 172.21.15.16:0/905542030 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f191c0a2b80 msgr2=0x7f191c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:45.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.867+0000 7f193951b640 1 --2- 172.21.15.16:0/905542030 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f191c0a2b80 0x7f191c0a5040 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f192c01f000 tx=0x7f192c020000 comp rx=0 tx=0).stop 2024-09-12T02:11:45.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.867+0000 7f193951b640 1 -- 172.21.15.16:0/905542030 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1934101430 msgr2=0x7f19341989b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:45.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.867+0000 7f193951b640 1 --2- 172.21.15.16:0/905542030 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1934101430 0x7f19341989b0 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f192800d950 tx=0x7f192800de10 comp rx=0 tx=0).stop 2024-09-12T02:11:45.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.867+0000 7f193951b640 1 -- 172.21.15.16:0/905542030 shutdown_connections 2024-09-12T02:11:45.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.867+0000 7f193951b640 1 --2- 172.21.15.16:0/905542030 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f191c0a2b80 0x7f191c0a5040 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:45.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.867+0000 7f193951b640 1 --2- 172.21.15.16:0/905542030 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1934106a90 0x7f193419cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:45.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.867+0000 7f193951b640 1 --2- 172.21.15.16:0/905542030 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1934101430 0x7f19341989b0 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:45.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.867+0000 7f193951b640 1 --2- 172.21.15.16:0/905542030 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1934100a90 0x7f1934198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:45.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.867+0000 7f193951b640 1 -- 172.21.15.16:0/905542030 >> 172.21.15.16:0/905542030 conn(0x7f19340fc7d0 msgr2=0x7f193406aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:45.874 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:11:45.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.867+0000 7f193951b640 1 -- 172.21.15.16:0/905542030 shutdown_connections 2024-09-12T02:11:45.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:45.867+0000 7f193951b640 1 -- 172.21.15.16:0/905542030 wait complete. 2024-09-12T02:11:45.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:11:45 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/2870295197' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:11:46.473 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:11:46 smithi016 ceph-mon[30947]: pgmap v630: 97 pgs: 97 active+clean; 1.1 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 2.7 KiB/s wr, 8 op/s 2024-09-12T02:11:46.474 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:11:46 smithi016 ceph-mon[30947]: scrub queued for path: / 2024-09-12T02:11:46.474 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:11:46 smithi016 ceph-mon[30947]: scrub summary: idle+waiting paths [/] 2024-09-12T02:11:46.474 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:11:46 smithi016 ceph-mon[30947]: scrub summary: active paths [/] 2024-09-12T02:11:46.474 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:11:46 smithi016 ceph-mon[30947]: scrub summary: idle+waiting paths [/] 2024-09-12T02:11:46.474 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:11:46 smithi016 ceph-mon[30947]: scrub summary: idle 2024-09-12T02:11:46.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:11:46 smithi033 ceph-mon[35332]: pgmap v630: 97 pgs: 97 active+clean; 1.1 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 2.7 KiB/s wr, 8 op/s 2024-09-12T02:11:46.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:11:46 smithi033 ceph-mon[35332]: scrub queued for path: / 2024-09-12T02:11:46.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:11:46 smithi033 ceph-mon[35332]: scrub summary: idle+waiting paths [/] 2024-09-12T02:11:46.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:11:46 smithi033 ceph-mon[35332]: scrub summary: active paths [/] 2024-09-12T02:11:46.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:11:46 smithi033 ceph-mon[35332]: scrub summary: idle+waiting paths [/] 2024-09-12T02:11:46.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:11:46 smithi033 ceph-mon[35332]: scrub summary: idle 2024-09-12T02:11:46.795 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T02:11:46.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:11:46 smithi184 ceph-mon[35593]: pgmap v630: 97 pgs: 97 active+clean; 1.1 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 2.7 KiB/s wr, 8 op/s 2024-09-12T02:11:46.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:11:46 smithi184 ceph-mon[35593]: scrub queued for path: / 2024-09-12T02:11:46.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:11:46 smithi184 ceph-mon[35593]: scrub summary: idle+waiting paths [/] 2024-09-12T02:11:46.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:11:46 smithi184 ceph-mon[35593]: scrub summary: active paths [/] 2024-09-12T02:11:46.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:11:46 smithi184 ceph-mon[35593]: scrub summary: idle+waiting paths [/] 2024-09-12T02:11:46.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:11:46 smithi184 ceph-mon[35593]: scrub summary: idle 2024-09-12T02:11:47.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.001+0000 7fe999cc3640 1 -- 172.21.15.16:0/1307610134 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe994102780 msgr2=0x7fe9941067d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:47.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.001+0000 7fe999cc3640 1 --2- 172.21.15.16:0/1307610134 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe994102780 0x7fe9941067d0 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7fe988009900 tx=0x7fe98802a410 comp rx=0 tx=0).stop 2024-09-12T02:11:47.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.001+0000 7fe999cc3640 1 -- 172.21.15.16:0/1307610134 shutdown_connections 2024-09-12T02:11:47.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.001+0000 7fe999cc3640 1 --2- 172.21.15.16:0/1307610134 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe994106f00 0x7fe99410ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:47.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.001+0000 7fe999cc3640 1 --2- 172.21.15.16:0/1307610134 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe994102780 0x7fe9941067d0 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:47.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.001+0000 7fe999cc3640 1 --2- 172.21.15.16:0/1307610134 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe994101dd0 0x7fe9941021b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:47.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.001+0000 7fe999cc3640 1 -- 172.21.15.16:0/1307610134 >> 172.21.15.16:0/1307610134 conn(0x7fe9940fd8b0 msgr2=0x7fe9940ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:47.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.001+0000 7fe999cc3640 1 -- 172.21.15.16:0/1307610134 shutdown_connections 2024-09-12T02:11:47.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.001+0000 7fe999cc3640 1 -- 172.21.15.16:0/1307610134 wait complete. 2024-09-12T02:11:47.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.002+0000 7fe999cc3640 1 Processor -- start 2024-09-12T02:11:47.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.002+0000 7fe999cc3640 1 -- start start 2024-09-12T02:11:47.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.002+0000 7fe999cc3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe994101dd0 0x7fe9940744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:47.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.002+0000 7fe999cc3640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe994102780 0x7fe994074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:47.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.002+0000 7fe999cc3640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe994106f00 0x7fe994074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:47.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.002+0000 7fe999cc3640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe994111f60 con 0x7fe994101dd0 2024-09-12T02:11:47.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.002+0000 7fe999cc3640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe994111de0 con 0x7fe994102780 2024-09-12T02:11:47.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.002+0000 7fe999cc3640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe9941120e0 con 0x7fe994106f00 2024-09-12T02:11:47.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.002+0000 7fe9937fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe994101dd0 0x7fe9940744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:47.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.002+0000 7fe9937fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe994101dd0 0x7fe9940744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36000/0 (socket says 172.21.15.16:36000) 2024-09-12T02:11:47.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.002+0000 7fe9937fe640 1 -- 172.21.15.16:0/3399231386 learned_addr learned my addr 172.21.15.16:0/3399231386 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:11:47.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.002+0000 7fe992ffd640 1 --2- 172.21.15.16:0/3399231386 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe994102780 0x7fe994074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:47.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.003+0000 7fe993fff640 1 --2- 172.21.15.16:0/3399231386 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe994106f00 0x7fe994074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:47.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.003+0000 7fe9937fe640 1 -- 172.21.15.16:0/3399231386 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe994106f00 msgr2=0x7fe994074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:47.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.003+0000 7fe9937fe640 1 --2- 172.21.15.16:0/3399231386 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe994106f00 0x7fe994074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:47.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.003+0000 7fe9937fe640 1 -- 172.21.15.16:0/3399231386 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe994102780 msgr2=0x7fe994074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:47.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.003+0000 7fe9937fe640 1 --2- 172.21.15.16:0/3399231386 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe994102780 0x7fe994074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:47.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.003+0000 7fe9937fe640 1 -- 172.21.15.16:0/3399231386 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9941a68c0 con 0x7fe994101dd0 2024-09-12T02:11:47.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.003+0000 7fe9937fe640 1 --2- 172.21.15.16:0/3399231386 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe994101dd0 0x7fe9940744f0 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7fe98400d950 tx=0x7fe98400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:47.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.003+0000 7fe990ff9640 1 -- 172.21.15.16:0/3399231386 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe984017070 con 0x7fe994101dd0 2024-09-12T02:11:47.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.003+0000 7fe990ff9640 1 -- 172.21.15.16:0/3399231386 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe984009c30 con 0x7fe994101dd0 2024-09-12T02:11:47.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.003+0000 7fe999cc3640 1 -- 172.21.15.16:0/3399231386 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9941a6bb0 con 0x7fe994101dd0 2024-09-12T02:11:47.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.003+0000 7fe990ff9640 1 -- 172.21.15.16:0/3399231386 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe98400f6b0 con 0x7fe994101dd0 2024-09-12T02:11:47.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.003+0000 7fe999cc3640 1 -- 172.21.15.16:0/3399231386 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9941a70c0 con 0x7fe994101dd0 2024-09-12T02:11:47.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.006+0000 7fe99c09c740 1 Processor -- start 2024-09-12T02:11:47.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.006+0000 7fe99c09c740 1 -- start start 2024-09-12T02:11:47.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.006+0000 7fe990ff9640 1 -- 172.21.15.16:0/3399231386 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fe98400f850 con 0x7fe994101dd0 2024-09-12T02:11:47.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.006+0000 7fe99c09c740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d6175c5060 0x55d6175c5440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:47.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.006+0000 7fe99c09c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d6175c5980 0x55d6175cfd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:47.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.006+0000 7fe99c09c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d6175d0240 0x55d6175d2630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:47.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.006+0000 7fe99c09c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55d61752a620 con 0x55d6175c5980 2024-09-12T02:11:47.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.006+0000 7fe99c09c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55d6174d1e50 con 0x55d6175d0240 2024-09-12T02:11:47.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.006+0000 7fe99c09c740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55d6175265e0 con 0x55d6175c5060 2024-09-12T02:11:47.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.007+0000 7fe993fff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d6175c5060 0x55d6175c5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:47.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.007+0000 7fe993fff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d6175c5060 0x55d6175c5440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:42036/0 (socket says 172.21.15.16:42036) 2024-09-12T02:11:47.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.007+0000 7fe993fff640 1 -- 172.21.15.16:0/1946755982 learned_addr learned my addr 172.21.15.16:0/1946755982 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:11:47.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.007+0000 7fe992ffd640 1 --2- 172.21.15.16:0/1946755982 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d6175d0240 0x55d6175d2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:47.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.007+0000 7fe9937fe640 1 --2- 172.21.15.16:0/1946755982 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d6175c5980 0x55d6175cfd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:47.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.007+0000 7fe993fff640 1 -- 172.21.15.16:0/1946755982 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d6175d0240 msgr2=0x55d6175d2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:47.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.007+0000 7fe993fff640 1 --2- 172.21.15.16:0/1946755982 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d6175d0240 0x55d6175d2630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:47.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.007+0000 7fe993fff640 1 -- 172.21.15.16:0/1946755982 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d6175c5980 msgr2=0x55d6175cfd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:47.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.007+0000 7fe993fff640 1 --2- 172.21.15.16:0/1946755982 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d6175c5980 0x55d6175cfd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:47.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.007+0000 7fe993fff640 1 -- 172.21.15.16:0/1946755982 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d6174c2d40 con 0x55d6175c5060 2024-09-12T02:11:47.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.007+0000 7fe990ff9640 1 --2- 172.21.15.16:0/3399231386 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fe97c0a2c00 0x7fe97c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:47.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.007+0000 7fe993fff640 1 --2- 172.21.15.16:0/1946755982 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d6175c5060 0x55d6175c5440 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fe98c009d70 tx=0x7fe98c00f730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=13c2617aae1232d4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:47.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.007+0000 7fe9937fe640 1 --2- 172.21.15.16:0/3399231386 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fe97c0a2c00 0x7fe97c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:47.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.008+0000 7fe990ff9640 1 -- 172.21.15.16:0/3399231386 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fe9840ceb50 con 0x7fe994101dd0 2024-09-12T02:11:47.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.008+0000 7fe979ffb640 1 -- 172.21.15.16:0/1946755982 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe98c003970 con 0x55d6175c5060 2024-09-12T02:11:47.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.008+0000 7fe979ffb640 1 -- 172.21.15.16:0/1946755982 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe98c004030 con 0x55d6175c5060 2024-09-12T02:11:47.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.008+0000 7fe979ffb640 1 -- 172.21.15.16:0/1946755982 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe98c011840 con 0x55d6175c5060 2024-09-12T02:11:47.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.008+0000 7fe9937fe640 1 --2- 172.21.15.16:0/3399231386 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fe97c0a2c00 0x7fe97c0a50c0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fe9840978b0 tx=0x7fe9840d2670 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:47.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.008+0000 7fe99c09c740 1 -- 172.21.15.16:0/1946755982 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d6175c5060 msgr2=0x55d6175c5440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:47.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.008+0000 7fe99c09c740 1 --2- 172.21.15.16:0/1946755982 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d6175c5060 0x55d6175c5440 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fe98c009d70 tx=0x7fe98c00f730 comp rx=0 tx=0).stop 2024-09-12T02:11:47.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.008+0000 7fe99c09c740 1 -- 172.21.15.16:0/1946755982 shutdown_connections 2024-09-12T02:11:47.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.008+0000 7fe99c09c740 1 --2- 172.21.15.16:0/1946755982 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d6175d0240 0x55d6175d2630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:47.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.008+0000 7fe99c09c740 1 --2- 172.21.15.16:0/1946755982 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d6175c5980 0x55d6175cfd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:47.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.008+0000 7fe99c09c740 1 --2- 172.21.15.16:0/1946755982 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d6175c5060 0x55d6175c5440 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:47.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.008+0000 7fe99c09c740 1 -- 172.21.15.16:0/1946755982 >> 172.21.15.16:0/1946755982 conn(0x55d6175c15f0 msgr2=0x55d6175c19d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:47.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.009+0000 7fe99c09c740 1 -- 172.21.15.16:0/1946755982 shutdown_connections 2024-09-12T02:11:47.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.009+0000 7fe99c09c740 1 -- 172.21.15.16:0/1946755982 wait complete. 2024-09-12T02:11:47.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.009+0000 7fe99c09c740 1 -- start start 2024-09-12T02:11:47.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.009+0000 7fe99c09c740 1 Processor -- start 2024-09-12T02:11:47.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.009+0000 7fe99c09c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d6175c5060 0x55d6175cd150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:47.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.009+0000 7fe99c09c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d6175c5980 0x55d6175cd690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:47.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.009+0000 7fe99c09c740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d6175d0240 0x55d6175cdbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:47.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.009+0000 7fe99c09c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55d6175d2e10 con 0x55d6175c5060 2024-09-12T02:11:47.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.009+0000 7fe99c09c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55d6174b2f10 con 0x55d6175c5980 2024-09-12T02:11:47.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.009+0000 7fe99c09c740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55d6175d2f90 con 0x55d6175d0240 2024-09-12T02:11:47.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.009+0000 7fe99c09c740 10 client.0 _ll_register_callbacks cb 0x55d6174d6db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:11:47.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.010+0000 7fe993fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d6175c5060 0x55d6175cd150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:47.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.010+0000 7fe993fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d6175c5060 0x55d6175cd150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36026/0 (socket says 172.21.15.16:36026) 2024-09-12T02:11:47.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.010+0000 7fe993fff640 1 -- 172.21.15.16:0/1460586262 learned_addr learned my addr 172.21.15.16:0/1460586262 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:11:47.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.010+0000 7fe992ffd640 1 --2- 172.21.15.16:0/1460586262 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d6175c5980 0x55d6175cd690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:47.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.010+0000 7fe993fff640 1 -- 172.21.15.16:0/1460586262 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d6175d0240 msgr2=0x55d6175cdbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:47.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.010+0000 7fe993fff640 1 --2- 172.21.15.16:0/1460586262 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d6175d0240 0x55d6175cdbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:47.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.010+0000 7fe993fff640 1 -- 172.21.15.16:0/1460586262 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d6175c5980 msgr2=0x55d6175cd690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:47.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.010+0000 7fe993fff640 1 --2- 172.21.15.16:0/1460586262 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d6175c5980 0x55d6175cd690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:47.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.010+0000 7fe993fff640 1 -- 172.21.15.16:0/1460586262 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d617239580 con 0x55d6175c5060 2024-09-12T02:11:47.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.010+0000 7fe993fff640 1 --2- 172.21.15.16:0/1460586262 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d6175c5060 0x55d6175cd150 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7fe98c0114a0 tx=0x7fe98c004410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8147a47b080fad3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:47.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.010+0000 7fe972ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:11:47.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.010+0000 7fe972ffd640 1 -- 172.21.15.16:0/1460586262 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe98c02a060 con 0x55d6175c5060 2024-09-12T02:11:47.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.011+0000 7fe99c09c740 1 -- 172.21.15.16:0/1460586262 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe98c004030 con 0x55d6175c5060 2024-09-12T02:11:47.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.011+0000 7fe972ffd640 1 -- 172.21.15.16:0/1460586262 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fe98c00fde0 con 0x55d6175c5060 2024-09-12T02:11:47.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.011+0000 7fe972ffd640 1 -- 172.21.15.16:0/1460586262 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe98c009950 con 0x55d6175c5060 2024-09-12T02:11:47.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.011+0000 7fe99c09c740 10 client.18447 fetch_fsmap learned FSMap version 25 2024-09-12T02:11:47.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.011+0000 7fe99c09c740 1 -- 172.21.15.16:0/1460586262 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d617303960 con 0x55d6175c5060 2024-09-12T02:11:47.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.011+0000 7fe972ffd640 1 -- 172.21.15.16:0/1460586262 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe98c004030 con 0x55d6175c5060 2024-09-12T02:11:47.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.011+0000 7fe972ffd640 1 -- 172.21.15.16:0/1460586262 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fe98c02f070 con 0x55d6175c5060 2024-09-12T02:11:47.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.011+0000 7fe99c09c740 10 client.18447 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:11:47.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.011+0000 7fe99c09c740 10 client.18447 resolve_mds: validated gid 24482 aka daemon mds.b 2024-09-12T02:11:47.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.011+0000 7fe99c09c740 20 client.18447 populate_metadata read hostname 'smithi016' 2024-09-12T02:11:47.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.011+0000 7fe99c09c740 1 --2- 172.21.15.16:0/1460586262 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55d6175e01c0 0x55d6175e25e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:47.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.012+0000 7fe99c09c740 4 client.18447 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:11:47.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.012+0000 7fe99c09c740 1 -- 172.21.15.16:0/1460586262 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d617537b90 con 0x55d6175e01c0 2024-09-12T02:11:47.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.012+0000 7fe992ffd640 1 --2- 172.21.15.16:0/1460586262 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55d6175e01c0 0x55d6175e25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:47.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.012+0000 7fe972ffd640 10 client.18447 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:11:47.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.012+0000 7fe992ffd640 1 --2- 172.21.15.16:0/1460586262 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55d6175e01c0 0x55d6175e25e0 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e6855226dba06c9 server_cookie=18114881cd19d5eb in_seq=0 out_seq=0 2024-09-12T02:11:47.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.013+0000 7fe972ffd640 1 -- 172.21.15.16:0/1460586262 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d617537b90 con 0x55d6175e01c0 2024-09-12T02:11:47.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.013+0000 7fe972ffd640 10 client.18447 handle_command_reply: tid=0 2024-09-12T02:11:47.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.013+0000 7fe99c09c740 1 client.18447 shutdown 2024-09-12T02:11:47.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.013+0000 7fe99c09c740 1 -- 172.21.15.16:0/1460586262 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d6175c5060 msgr2=0x55d6175cd150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:47.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.013+0000 7fe99c09c740 1 --2- 172.21.15.16:0/1460586262 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d6175c5060 0x55d6175cd150 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7fe98c0114a0 tx=0x7fe98c004410 comp rx=0 tx=0).stop 2024-09-12T02:11:47.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.013+0000 7fe99c09c740 1 -- 172.21.15.16:0/1460586262 shutdown_connections 2024-09-12T02:11:47.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.013+0000 7fe99c09c740 1 --2- 172.21.15.16:0/1460586262 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55d6175e01c0 0x55d6175e25e0 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:47.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.013+0000 7fe99c09c740 1 --2- 172.21.15.16:0/1460586262 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d6175d0240 0x55d6175cdbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:47.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.013+0000 7fe99c09c740 1 --2- 172.21.15.16:0/1460586262 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d6175c5980 0x55d6175cd690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:47.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.013+0000 7fe99c09c740 1 --2- 172.21.15.16:0/1460586262 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d6175c5060 0x55d6175cd150 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:47.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.013+0000 7fe99c09c740 1 -- 172.21.15.16:0/1460586262 >> 172.21.15.16:0/1460586262 conn(0x55d6175c15f0 msgr2=0x55d6175c7320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:47.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.014+0000 7fe972ffd640 0 client.18447 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:11:47.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.014+0000 7fe99c09c740 1 -- 172.21.15.16:0/1460586262 shutdown_connections 2024-09-12T02:11:47.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.014+0000 7fe99c09c740 1 -- 172.21.15.16:0/1460586262 wait complete. 2024-09-12T02:11:47.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.014+0000 7fe99c09c740 20 client.18447 trim_cache size 0 max 16384 2024-09-12T02:11:47.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.037+0000 7fe99c09c740 1 Processor -- start 2024-09-12T02:11:47.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.037+0000 7fe99c09c740 1 -- start start 2024-09-12T02:11:47.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.037+0000 7fe99c09c740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d6175c5060 0x55d6175c0640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:47.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.037+0000 7fe99c09c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d6175c5980 0x55d6175e7eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:47.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.037+0000 7fe99c09c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d6175d0240 0x55d6175e83f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:47.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.037+0000 7fe99c09c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe9941b1ed0 con 0x55d6175d0240 2024-09-12T02:11:47.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.037+0000 7fe99c09c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe9941af7a0 con 0x55d6175c5980 2024-09-12T02:11:47.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.037+0000 7fe99c09c740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe9941ad610 con 0x55d6175c5060 2024-09-12T02:11:47.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.037+0000 7fe993fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d6175d0240 0x55d6175e83f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:47.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.037+0000 7fe993fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d6175d0240 0x55d6175e83f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36030/0 (socket says 172.21.15.16:36030) 2024-09-12T02:11:47.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.037+0000 7fe993fff640 1 -- 172.21.15.16:0/3250591688 learned_addr learned my addr 172.21.15.16:0/3250591688 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:11:47.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.038+0000 7fe993fff640 1 --2- 172.21.15.16:0/3250591688 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d6175c5060 0x55d6175c0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:47.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.038+0000 7fe992ffd640 1 --2- 172.21.15.16:0/3250591688 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d6175c5980 0x55d6175e7eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:47.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.038+0000 7fe993fff640 1 -- 172.21.15.16:0/3250591688 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d6175c5060 msgr2=0x55d6175c0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:47.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.038+0000 7fe993fff640 1 --2- 172.21.15.16:0/3250591688 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d6175c5060 0x55d6175c0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:47.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.038+0000 7fe993fff640 1 -- 172.21.15.16:0/3250591688 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d6175c5980 msgr2=0x55d6175e7eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:47.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.038+0000 7fe993fff640 1 --2- 172.21.15.16:0/3250591688 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d6175c5980 0x55d6175e7eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:47.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.038+0000 7fe993fff640 1 -- 172.21.15.16:0/3250591688 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d6175c57b0 con 0x55d6175d0240 2024-09-12T02:11:47.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.038+0000 7fe993fff640 1 --2- 172.21.15.16:0/3250591688 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d6175d0240 0x55d6175e83f0 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7fe98c0048d0 tx=0x7fe98c013040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aff72e765a3fa5a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:47.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.038+0000 7fe9797fa640 1 -- 172.21.15.16:0/3250591688 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe98c035020 con 0x55d6175d0240 2024-09-12T02:11:47.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.038+0000 7fe9797fa640 1 -- 172.21.15.16:0/3250591688 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe98c036040 con 0x55d6175d0240 2024-09-12T02:11:47.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.038+0000 7fe9797fa640 1 -- 172.21.15.16:0/3250591688 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe98c0118d0 con 0x55d6175d0240 2024-09-12T02:11:47.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.039+0000 7fe99c09c740 1 -- 172.21.15.16:0/3250591688 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d6175d0240 msgr2=0x55d6175e83f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:47.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.039+0000 7fe99c09c740 1 --2- 172.21.15.16:0/3250591688 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d6175d0240 0x55d6175e83f0 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7fe98c0048d0 tx=0x7fe98c013040 comp rx=0 tx=0).stop 2024-09-12T02:11:47.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.039+0000 7fe99c09c740 1 -- 172.21.15.16:0/3250591688 shutdown_connections 2024-09-12T02:11:47.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.039+0000 7fe99c09c740 1 --2- 172.21.15.16:0/3250591688 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d6175d0240 0x55d6175e83f0 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:47.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.039+0000 7fe99c09c740 1 --2- 172.21.15.16:0/3250591688 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d6175c5980 0x55d6175e7eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:47.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.039+0000 7fe99c09c740 1 --2- 172.21.15.16:0/3250591688 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d6175c5060 0x55d6175c0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:47.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.039+0000 7fe99c09c740 1 -- 172.21.15.16:0/3250591688 >> 172.21.15.16:0/3250591688 conn(0x55d6175c15f0 msgr2=0x55d6175bff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:47.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.039+0000 7fe99c09c740 1 -- 172.21.15.16:0/3250591688 shutdown_connections 2024-09-12T02:11:47.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.039+0000 7fe99c09c740 1 -- 172.21.15.16:0/3250591688 wait complete. 2024-09-12T02:11:47.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.039+0000 7fe99c09c740 1 -- start start 2024-09-12T02:11:47.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.040+0000 7fe99c09c740 1 Processor -- start 2024-09-12T02:11:47.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.040+0000 7fe99c09c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d6175c5060 0x55d6175d87a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:47.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.040+0000 7fe99c09c740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d6175c5980 0x55d6175d3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:47.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.040+0000 7fe99c09c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d6175d0240 0x55d6175d4010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:47.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.040+0000 7fe99c09c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe9941adc60 con 0x55d6175d0240 2024-09-12T02:11:47.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.040+0000 7fe99c09c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe9941af0d0 con 0x55d6175c5060 2024-09-12T02:11:47.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.040+0000 7fe99c09c740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe9941ae330 con 0x55d6175c5980 2024-09-12T02:11:47.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.040+0000 7fe99c09c740 10 client.0 _ll_register_callbacks cb 0x7fe98c009bc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:11:47.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.040+0000 7fe993fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d6175d0240 0x55d6175d4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:47.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.040+0000 7fe993fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d6175d0240 0x55d6175d4010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36046/0 (socket says 172.21.15.16:36046) 2024-09-12T02:11:47.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.040+0000 7fe993fff640 1 -- 172.21.15.16:0/3672798964 learned_addr learned my addr 172.21.15.16:0/3672798964 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:11:47.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.040+0000 7fe993fff640 1 --2- 172.21.15.16:0/3672798964 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d6175c5060 0x55d6175d87a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:47.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.040+0000 7fe992ffd640 1 --2- 172.21.15.16:0/3672798964 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d6175c5980 0x55d6175d3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:47.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.041+0000 7fe993fff640 1 -- 172.21.15.16:0/3672798964 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d6175c5980 msgr2=0x55d6175d3ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:47.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.041+0000 7fe993fff640 1 --2- 172.21.15.16:0/3672798964 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d6175c5980 0x55d6175d3ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:47.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.041+0000 7fe993fff640 1 -- 172.21.15.16:0/3672798964 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d6175c5060 msgr2=0x55d6175d87a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:47.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.041+0000 7fe993fff640 1 --2- 172.21.15.16:0/3672798964 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d6175c5060 0x55d6175d87a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:47.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.041+0000 7fe993fff640 1 -- 172.21.15.16:0/3672798964 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d6175d4680 con 0x55d6175d0240 2024-09-12T02:11:47.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.041+0000 7fe993fff640 1 --2- 172.21.15.16:0/3672798964 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d6175d0240 0x55d6175d4010 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7fe98c002230 tx=0x7fe98c01e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90a430ead480c4d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:47.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.041+0000 7fe972ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:11:47.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.041+0000 7fe972ffd640 1 -- 172.21.15.16:0/3672798964 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe98c035070 con 0x55d6175d0240 2024-09-12T02:11:47.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.041+0000 7fe972ffd640 1 -- 172.21.15.16:0/3672798964 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fe98c003b80 con 0x55d6175d0240 2024-09-12T02:11:47.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.041+0000 7fe99c09c740 1 -- 172.21.15.16:0/3672798964 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe98c036040 con 0x55d6175d0240 2024-09-12T02:11:47.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.041+0000 7fe972ffd640 1 -- 172.21.15.16:0/3672798964 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe98c02a070 con 0x55d6175d0240 2024-09-12T02:11:47.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.041+0000 7fe99c09c740 10 client.18453 fetch_fsmap learned FSMap version 25 2024-09-12T02:11:47.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.041+0000 7fe99c09c740 1 -- 172.21.15.16:0/3672798964 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d61764ad30 con 0x55d6175d0240 2024-09-12T02:11:47.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.041+0000 7fe972ffd640 1 -- 172.21.15.16:0/3672798964 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe98c036040 con 0x55d6175d0240 2024-09-12T02:11:47.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.042+0000 7fe972ffd640 1 -- 172.21.15.16:0/3672798964 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fe98c03b030 con 0x55d6175d0240 2024-09-12T02:11:47.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.042+0000 7fe99c09c740 10 client.18453 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:11:47.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.042+0000 7fe99c09c740 10 client.18453 resolve_mds: validated gid 24482 aka daemon mds.b 2024-09-12T02:11:47.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.042+0000 7fe99c09c740 20 client.18453 populate_metadata read hostname 'smithi016' 2024-09-12T02:11:47.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.042+0000 7fe99c09c740 1 --2- 172.21.15.16:0/3672798964 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55d61764b2e0 0x55d61764d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:47.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.042+0000 7fe99c09c740 4 client.18453 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "998c7ef0", "release": true, "await": true}] 2024-09-12T02:11:47.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.042+0000 7fe99c09c740 1 -- 172.21.15.16:0/3672798964 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "quiesce db", "set_id": "998c7ef0", "release": true, "await": true}) -- 0x55d6175320a0 con 0x55d61764b2e0 2024-09-12T02:11:47.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.042+0000 7fe992ffd640 1 --2- 172.21.15.16:0/3672798964 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55d61764b2e0 0x55d61764d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:47.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.042+0000 7fe992ffd640 1 --2- 172.21.15.16:0/3672798964 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55d61764b2e0 0x55d61764d6d0 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6506f718447664cf server_cookie=9ad7243159b8352e in_seq=0 out_seq=0 2024-09-12T02:11:47.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.042+0000 7fe972ffd640 10 client.18453 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:11:47.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.045+0000 7fe972ffd640 1 -- 172.21.15.16:0/3672798964 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55d6175320a0 con 0x55d61764b2e0 2024-09-12T02:11:47.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.045+0000 7fe972ffd640 10 client.18453 handle_command_reply: tid=0 2024-09-12T02:11:47.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.045+0000 7fe99c09c740 1 client.18453 shutdown 2024-09-12T02:11:47.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.045+0000 7fe99c09c740 1 -- 172.21.15.16:0/3672798964 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d6175d0240 msgr2=0x55d6175d4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:47.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.045+0000 7fe99c09c740 1 --2- 172.21.15.16:0/3672798964 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d6175d0240 0x55d6175d4010 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7fe98c002230 tx=0x7fe98c01e040 comp rx=0 tx=0).stop 2024-09-12T02:11:47.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.045+0000 7fe99c09c740 1 -- 172.21.15.16:0/3672798964 shutdown_connections 2024-09-12T02:11:47.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.045+0000 7fe99c09c740 1 --2- 172.21.15.16:0/3672798964 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55d61764b2e0 0x55d61764d6d0 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:47.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.045+0000 7fe99c09c740 1 --2- 172.21.15.16:0/3672798964 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d6175d0240 0x55d6175d4010 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:47.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.045+0000 7fe99c09c740 1 --2- 172.21.15.16:0/3672798964 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d6175c5980 0x55d6175d3ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:47.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.045+0000 7fe99c09c740 1 --2- 172.21.15.16:0/3672798964 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d6175c5060 0x55d6175d87a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:47.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.045+0000 7fe99c09c740 1 -- 172.21.15.16:0/3672798964 >> 172.21.15.16:0/3672798964 conn(0x55d6175c15f0 msgr2=0x55d6175c0750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:47.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.045+0000 7fe972ffd640 0 client.18453 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:11:47.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.045+0000 7fe99c09c740 1 -- 172.21.15.16:0/3672798964 shutdown_connections 2024-09-12T02:11:47.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.045+0000 7fe99c09c740 1 -- 172.21.15.16:0/3672798964 wait complete. 2024-09-12T02:11:47.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.045+0000 7fe99c09c740 20 client.18453 trim_cache size 0 max 16384 2024-09-12T02:11:47.059 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T02:11:47.059 INFO:teuthology.orchestra.run.smithi016.stdout: "epoch": 25, 2024-09-12T02:11:47.059 INFO:teuthology.orchestra.run.smithi016.stdout: "leader": 24482, 2024-09-12T02:11:47.059 INFO:teuthology.orchestra.run.smithi016.stdout: "set_version": 40, 2024-09-12T02:11:47.059 INFO:teuthology.orchestra.run.smithi016.stdout: "sets": { 2024-09-12T02:11:47.059 INFO:teuthology.orchestra.run.smithi016.stdout: "998c7ef0": { 2024-09-12T02:11:47.060 INFO:teuthology.orchestra.run.smithi016.stdout: "version": 40, 2024-09-12T02:11:47.060 INFO:teuthology.orchestra.run.smithi016.stdout: "age_ref": 0.0, 2024-09-12T02:11:47.060 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-12T02:11:47.060 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RELEASED", 2024-09-12T02:11:47.060 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-12T02:11:47.060 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:11:47.060 INFO:teuthology.orchestra.run.smithi016.stdout: "timeout": 90.0, 2024-09-12T02:11:47.060 INFO:teuthology.orchestra.run.smithi016.stdout: "expiration": 142.6, 2024-09-12T02:11:47.060 INFO:teuthology.orchestra.run.smithi016.stdout: "members": { 2024-09-12T02:11:47.060 INFO:teuthology.orchestra.run.smithi016.stdout: "file:/": { 2024-09-12T02:11:47.060 INFO:teuthology.orchestra.run.smithi016.stdout: "excluded": false, 2024-09-12T02:11:47.060 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-12T02:11:47.060 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RELEASED", 2024-09-12T02:11:47.060 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-12T02:11:47.061 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:11:47.061 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:11:47.061 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:11:47.061 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:11:47.061 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:11:47.061 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T02:11:47.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.046+0000 7fe999cc3640 1 -- 172.21.15.16:0/3399231386 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fe97c0a2c00 msgr2=0x7fe97c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:47.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.046+0000 7fe999cc3640 1 --2- 172.21.15.16:0/3399231386 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fe97c0a2c00 0x7fe97c0a50c0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fe9840978b0 tx=0x7fe9840d2670 comp rx=0 tx=0).stop 2024-09-12T02:11:47.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.046+0000 7fe999cc3640 1 -- 172.21.15.16:0/3399231386 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe994101dd0 msgr2=0x7fe9940744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:47.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.046+0000 7fe999cc3640 1 --2- 172.21.15.16:0/3399231386 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe994101dd0 0x7fe9940744f0 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7fe98400d950 tx=0x7fe98400de10 comp rx=0 tx=0).stop 2024-09-12T02:11:47.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.046+0000 7fe999cc3640 1 -- 172.21.15.16:0/3399231386 shutdown_connections 2024-09-12T02:11:47.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.046+0000 7fe999cc3640 1 --2- 172.21.15.16:0/3399231386 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fe97c0a2c00 0x7fe97c0a50c0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:47.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.046+0000 7fe999cc3640 1 --2- 172.21.15.16:0/3399231386 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe994106f00 0x7fe994074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:47.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.046+0000 7fe999cc3640 1 --2- 172.21.15.16:0/3399231386 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe994102780 0x7fe994074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:47.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.046+0000 7fe999cc3640 1 --2- 172.21.15.16:0/3399231386 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe994101dd0 0x7fe9940744f0 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:47.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.046+0000 7fe999cc3640 1 -- 172.21.15.16:0/3399231386 >> 172.21.15.16:0/3399231386 conn(0x7fe9940fd8b0 msgr2=0x7fe9940fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:47.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.046+0000 7fe999cc3640 1 -- 172.21.15.16:0/3399231386 shutdown_connections 2024-09-12T02:11:47.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:47.046+0000 7fe999cc3640 1 -- 172.21.15.16:0/3399231386 wait complete. 2024-09-12T02:11:47.673 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '998c7ef0', total seconds elapsed: 26.6 2024-09-12T02:11:47.673 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 22.55 2024-09-12T02:11:47.673 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:11:47.696 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:11:48.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.172+0000 7fc745df6640 1 -- 172.21.15.16:0/2451479158 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc740101de0 msgr2=0x7fc7401021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:48.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.172+0000 7fc745df6640 1 --2- 172.21.15.16:0/2451479158 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc740101de0 0x7fc7401021c0 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7fc730009900 tx=0x7fc73002a410 comp rx=0 tx=0).stop 2024-09-12T02:11:48.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.172+0000 7fc745df6640 1 -- 172.21.15.16:0/2451479158 shutdown_connections 2024-09-12T02:11:48.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.172+0000 7fc745df6640 1 --2- 172.21.15.16:0/2451479158 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc740106f10 0x7fc74010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:48.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.172+0000 7fc745df6640 1 --2- 172.21.15.16:0/2451479158 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc740102790 0x7fc7401067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:48.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.172+0000 7fc745df6640 1 --2- 172.21.15.16:0/2451479158 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc740101de0 0x7fc7401021c0 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:48.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.172+0000 7fc745df6640 1 -- 172.21.15.16:0/2451479158 >> 172.21.15.16:0/2451479158 conn(0x7fc7400fd8c0 msgr2=0x7fc7400ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:48.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.173+0000 7fc745df6640 1 -- 172.21.15.16:0/2451479158 shutdown_connections 2024-09-12T02:11:48.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.173+0000 7fc745df6640 1 -- 172.21.15.16:0/2451479158 wait complete. 2024-09-12T02:11:48.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.173+0000 7fc745df6640 1 Processor -- start 2024-09-12T02:11:48.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.173+0000 7fc745df6640 1 -- start start 2024-09-12T02:11:48.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.174+0000 7fc745df6640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc740101de0 0x7fc7400744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:48.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.174+0000 7fc745df6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc740102790 0x7fc740074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:48.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.174+0000 7fc745df6640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc740106f10 0x7fc740074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:48.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.174+0000 7fc745df6640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc740111f30 con 0x7fc740102790 2024-09-12T02:11:48.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.174+0000 7fc745df6640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc740111db0 con 0x7fc740106f10 2024-09-12T02:11:48.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.174+0000 7fc745df6640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc7401120b0 con 0x7fc740101de0 2024-09-12T02:11:48.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.174+0000 7fc73effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc740102790 0x7fc740074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:48.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.174+0000 7fc73effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc740102790 0x7fc740074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36076/0 (socket says 172.21.15.16:36076) 2024-09-12T02:11:48.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.174+0000 7fc73effd640 1 -- 172.21.15.16:0/1649660018 learned_addr learned my addr 172.21.15.16:0/1649660018 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:11:48.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.174+0000 7fc73f7fe640 1 --2- 172.21.15.16:0/1649660018 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc740101de0 0x7fc7400744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:48.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.174+0000 7fc73ffff640 1 --2- 172.21.15.16:0/1649660018 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc740106f10 0x7fc740074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:48.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.174+0000 7fc73effd640 1 -- 172.21.15.16:0/1649660018 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc740101de0 msgr2=0x7fc7400744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:48.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.174+0000 7fc73effd640 1 --2- 172.21.15.16:0/1649660018 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc740101de0 0x7fc7400744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:48.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.174+0000 7fc73effd640 1 -- 172.21.15.16:0/1649660018 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc740106f10 msgr2=0x7fc740074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:48.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.174+0000 7fc73effd640 1 --2- 172.21.15.16:0/1649660018 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc740106f10 0x7fc740074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:48.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.174+0000 7fc73effd640 1 -- 172.21.15.16:0/1649660018 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7401a6980 con 0x7fc740102790 2024-09-12T02:11:48.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.174+0000 7fc73effd640 1 --2- 172.21.15.16:0/1649660018 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc740102790 0x7fc740074a30 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7fc73400d950 tx=0x7fc73400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:48.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.174+0000 7fc73cff9640 1 -- 172.21.15.16:0/1649660018 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc734017030 con 0x7fc740102790 2024-09-12T02:11:48.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.175+0000 7fc73cff9640 1 -- 172.21.15.16:0/1649660018 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc734004020 con 0x7fc740102790 2024-09-12T02:11:48.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.175+0000 7fc745df6640 1 -- 172.21.15.16:0/1649660018 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7401a6c70 con 0x7fc740102790 2024-09-12T02:11:48.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.175+0000 7fc73cff9640 1 -- 172.21.15.16:0/1649660018 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7340047a0 con 0x7fc740102790 2024-09-12T02:11:48.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.175+0000 7fc745df6640 1 -- 172.21.15.16:0/1649660018 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7401a71b0 con 0x7fc740102790 2024-09-12T02:11:48.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.177+0000 7fc73cff9640 1 -- 172.21.15.16:0/1649660018 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fc7340041d0 con 0x7fc740102790 2024-09-12T02:11:48.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.177+0000 7fc7481cf740 1 Processor -- start 2024-09-12T02:11:48.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.177+0000 7fc7481cf740 1 -- start start 2024-09-12T02:11:48.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.177+0000 7fc73cff9640 1 --2- 172.21.15.16:0/1649660018 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc7280a2b80 0x7fc7280a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:48.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.178+0000 7fc7481cf740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df50a914a0 0x55df50a91880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:48.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.178+0000 7fc73ffff640 1 --2- 172.21.15.16:0/1649660018 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc7280a2b80 0x7fc7280a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:48.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.178+0000 7fc73cff9640 1 -- 172.21.15.16:0/1649660018 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fc7340ce8c0 con 0x7fc740102790 2024-09-12T02:11:48.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.178+0000 7fc7481cf740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55df50a91dc0 0x55df50a9c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:48.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.178+0000 7fc7481cf740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55df50a9c680 0x55df50a9ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:48.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.178+0000 7fc7481cf740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55df509f72e0 con 0x55df50a914a0 2024-09-12T02:11:48.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.178+0000 7fc7481cf740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55df5099ece0 con 0x55df50a9c680 2024-09-12T02:11:48.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.178+0000 7fc7481cf740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55df509f32a0 con 0x55df50a91dc0 2024-09-12T02:11:48.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.178+0000 7fc73f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df50a914a0 0x55df50a91880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:48.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.178+0000 7fc73f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df50a914a0 0x55df50a91880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36088/0 (socket says 172.21.15.16:36088) 2024-09-12T02:11:48.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.178+0000 7fc73f7fe640 1 -- 172.21.15.16:0/330174936 learned_addr learned my addr 172.21.15.16:0/330174936 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:11:48.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.178+0000 7fc73effd640 1 --2- 172.21.15.16:0/330174936 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55df50a91dc0 0x55df50a9c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:48.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.178+0000 7fc73ffff640 1 --2- 172.21.15.16:0/330174936 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55df50a9c680 0x55df50a9ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:48.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.178+0000 7fc73f7fe640 1 -- 172.21.15.16:0/330174936 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55df50a91dc0 msgr2=0x55df50a9c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:48.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.178+0000 7fc73f7fe640 1 --2- 172.21.15.16:0/330174936 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55df50a91dc0 0x55df50a9c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:48.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.178+0000 7fc73f7fe640 1 -- 172.21.15.16:0/330174936 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55df50a9c680 msgr2=0x55df50a9ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:48.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.178+0000 7fc73f7fe640 1 --2- 172.21.15.16:0/330174936 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55df50a9c680 0x55df50a9ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:48.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.178+0000 7fc73f7fe640 1 -- 172.21.15.16:0/330174936 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df50983440 con 0x55df50a914a0 2024-09-12T02:11:48.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.178+0000 7fc73effd640 1 --2- 172.21.15.16:0/330174936 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55df50a91dc0 0x55df50a9c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:11:48.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.178+0000 7fc73f7fe640 1 --2- 172.21.15.16:0/330174936 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df50a914a0 0x55df50a91880 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7fc730009900 tx=0x7fc730005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=735f86c7bb970c67 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:48.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.179+0000 7fc73ffff640 1 --2- 172.21.15.16:0/1649660018 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc7280a2b80 0x7fc7280a5040 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x55df50a8edb0 tx=0x7fc738009a50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:48.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.179+0000 7fc725ffb640 1 -- 172.21.15.16:0/330174936 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc730040070 con 0x55df50a914a0 2024-09-12T02:11:48.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.179+0000 7fc725ffb640 1 -- 172.21.15.16:0/330174936 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc7300319f0 con 0x55df50a914a0 2024-09-12T02:11:48.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.179+0000 7fc725ffb640 1 -- 172.21.15.16:0/330174936 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc730035040 con 0x55df50a914a0 2024-09-12T02:11:48.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.179+0000 7fc7481cf740 1 -- 172.21.15.16:0/330174936 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df50a914a0 msgr2=0x55df50a91880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:48.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.179+0000 7fc7481cf740 1 --2- 172.21.15.16:0/330174936 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df50a914a0 0x55df50a91880 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7fc730009900 tx=0x7fc730005e80 comp rx=0 tx=0).stop 2024-09-12T02:11:48.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.179+0000 7fc7481cf740 1 -- 172.21.15.16:0/330174936 shutdown_connections 2024-09-12T02:11:48.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.179+0000 7fc7481cf740 1 --2- 172.21.15.16:0/330174936 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55df50a9c680 0x55df50a9ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:48.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.179+0000 7fc7481cf740 1 --2- 172.21.15.16:0/330174936 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55df50a91dc0 0x55df50a9c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:48.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.179+0000 7fc7481cf740 1 --2- 172.21.15.16:0/330174936 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df50a914a0 0x55df50a91880 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:48.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.179+0000 7fc7481cf740 1 -- 172.21.15.16:0/330174936 >> 172.21.15.16:0/330174936 conn(0x55df50a8da30 msgr2=0x55df50a8de10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:48.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.179+0000 7fc7481cf740 1 -- 172.21.15.16:0/330174936 shutdown_connections 2024-09-12T02:11:48.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.179+0000 7fc7481cf740 1 -- 172.21.15.16:0/330174936 wait complete. 2024-09-12T02:11:48.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.180+0000 7fc7481cf740 1 -- start start 2024-09-12T02:11:48.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.180+0000 7fc7481cf740 1 Processor -- start 2024-09-12T02:11:48.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.180+0000 7fc7481cf740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df50a914a0 0x55df50a9beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:48.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.180+0000 7fc7481cf740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55df50a91dc0 0x55df50a99160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:48.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.180+0000 7fc7481cf740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55df50a9c680 0x55df50a996a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:48.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.180+0000 7fc7481cf740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55df50a9f250 con 0x55df50a914a0 2024-09-12T02:11:48.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.180+0000 7fc7481cf740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55df509b2050 con 0x55df50a91dc0 2024-09-12T02:11:48.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.180+0000 7fc7481cf740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55df50a9f3d0 con 0x55df50a9c680 2024-09-12T02:11:48.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.180+0000 7fc7481cf740 10 client.0 _ll_register_callbacks cb 0x55df509f2160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:11:48.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.180+0000 7fc73f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df50a914a0 0x55df50a9beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:48.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.181+0000 7fc73f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df50a914a0 0x55df50a9beb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36102/0 (socket says 172.21.15.16:36102) 2024-09-12T02:11:48.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.181+0000 7fc73f7fe640 1 -- 172.21.15.16:0/1208871470 learned_addr learned my addr 172.21.15.16:0/1208871470 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:11:48.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.181+0000 7fc73effd640 1 --2- 172.21.15.16:0/1208871470 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55df50a91dc0 0x55df50a99160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:48.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.181+0000 7fc73f7fe640 1 -- 172.21.15.16:0/1208871470 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55df50a9c680 msgr2=0x55df50a996a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:48.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.181+0000 7fc73f7fe640 1 --2- 172.21.15.16:0/1208871470 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55df50a9c680 0x55df50a996a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:48.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.181+0000 7fc73f7fe640 1 -- 172.21.15.16:0/1208871470 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55df50a91dc0 msgr2=0x55df50a99160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:48.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.181+0000 7fc73f7fe640 1 --2- 172.21.15.16:0/1208871470 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55df50a91dc0 0x55df50a99160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:48.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.181+0000 7fc73f7fe640 1 -- 172.21.15.16:0/1208871470 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df507683c0 con 0x55df50a914a0 2024-09-12T02:11:48.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.181+0000 7fc73f7fe640 1 --2- 172.21.15.16:0/1208871470 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df50a914a0 0x55df50a9beb0 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7fc73002a450 tx=0x7fc730005250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18e75a7e162efd9e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:48.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.181+0000 7fc71effd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:11:48.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.181+0000 7fc71effd640 1 -- 172.21.15.16:0/1208871470 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc730040030 con 0x55df50a914a0 2024-09-12T02:11:48.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.181+0000 7fc71effd640 1 -- 172.21.15.16:0/1208871470 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fc730004250 con 0x55df50a914a0 2024-09-12T02:11:48.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.181+0000 7fc7481cf740 1 -- 172.21.15.16:0/1208871470 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc7300319f0 con 0x55df50a914a0 2024-09-12T02:11:48.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.181+0000 7fc71effd640 1 -- 172.21.15.16:0/1208871470 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc730035040 con 0x55df50a914a0 2024-09-12T02:11:48.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.181+0000 7fc7481cf740 10 client.18465 fetch_fsmap learned FSMap version 25 2024-09-12T02:11:48.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.181+0000 7fc7481cf740 1 -- 172.21.15.16:0/1208871470 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55df50aaab00 con 0x55df50a914a0 2024-09-12T02:11:48.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.182+0000 7fc71effd640 1 -- 172.21.15.16:0/1208871470 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc7300319f0 con 0x55df50a914a0 2024-09-12T02:11:48.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.182+0000 7fc71effd640 1 -- 172.21.15.16:0/1208871470 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fc730003a10 con 0x55df50a914a0 2024-09-12T02:11:48.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.182+0000 7fc7481cf740 10 client.18465 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:11:48.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.182+0000 7fc7481cf740 10 client.18465 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:11:48.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.182+0000 7fc7481cf740 20 client.18465 populate_metadata read hostname 'smithi016' 2024-09-12T02:11:48.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.182+0000 7fc7481cf740 1 --2- 172.21.15.16:0/1208871470 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55df50aabe50 0x55df50aae240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:48.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.182+0000 7fc7481cf740 4 client.18465 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:11:48.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.182+0000 7fc7481cf740 1 -- 172.21.15.16:0/1208871470 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55df50a04850 con 0x55df50aabe50 2024-09-12T02:11:48.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.182+0000 7fc73effd640 1 --2- 172.21.15.16:0/1208871470 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55df50aabe50 0x55df50aae240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:48.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.182+0000 7fc71effd640 10 client.18465 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:11:48.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.182+0000 7fc73effd640 1 --2- 172.21.15.16:0/1208871470 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55df50aabe50 0x55df50aae240 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e53dba3cda422d2 server_cookie=c393d8bcb0a87517 in_seq=0 out_seq=0 2024-09-12T02:11:48.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.183+0000 7fc71effd640 1 -- 172.21.15.16:0/1208871470 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55df50a04850 con 0x55df50aabe50 2024-09-12T02:11:48.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.183+0000 7fc71effd640 10 client.18465 handle_command_reply: tid=0 2024-09-12T02:11:48.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.183+0000 7fc7481cf740 1 client.18465 shutdown 2024-09-12T02:11:48.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.183+0000 7fc7481cf740 1 -- 172.21.15.16:0/1208871470 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df50a914a0 msgr2=0x55df50a9beb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:48.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.183+0000 7fc7481cf740 1 --2- 172.21.15.16:0/1208871470 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df50a914a0 0x55df50a9beb0 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7fc73002a450 tx=0x7fc730005250 comp rx=0 tx=0).stop 2024-09-12T02:11:48.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.183+0000 7fc7481cf740 1 -- 172.21.15.16:0/1208871470 shutdown_connections 2024-09-12T02:11:48.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.183+0000 7fc7481cf740 1 --2- 172.21.15.16:0/1208871470 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55df50aabe50 0x55df50aae240 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:48.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.183+0000 7fc7481cf740 1 --2- 172.21.15.16:0/1208871470 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55df50a9c680 0x55df50a996a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:48.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.183+0000 7fc7481cf740 1 --2- 172.21.15.16:0/1208871470 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55df50a91dc0 0x55df50a99160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:48.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.183+0000 7fc71effd640 0 client.18465 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:11:48.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.183+0000 7fc7481cf740 1 --2- 172.21.15.16:0/1208871470 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df50a914a0 0x55df50a9beb0 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:48.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.183+0000 7fc7481cf740 1 -- 172.21.15.16:0/1208871470 >> 172.21.15.16:0/1208871470 conn(0x55df50a8da30 msgr2=0x55df50a8cb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:48.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.184+0000 7fc7481cf740 1 -- 172.21.15.16:0/1208871470 shutdown_connections 2024-09-12T02:11:48.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.184+0000 7fc7481cf740 1 -- 172.21.15.16:0/1208871470 wait complete. 2024-09-12T02:11:48.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.184+0000 7fc7481cf740 20 client.18465 trim_cache size 0 max 16384 2024-09-12T02:11:48.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.201+0000 7fc7481cf740 1 Processor -- start 2024-09-12T02:11:48.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.201+0000 7fc7481cf740 1 -- start start 2024-09-12T02:11:48.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.201+0000 7fc7481cf740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55df50a914a0 0x55df50a94900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:48.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.201+0000 7fc7481cf740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55df50a91dc0 0x55df50a98140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:48.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.201+0000 7fc7481cf740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df50a9c680 0x55df50ab3870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:48.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.201+0000 7fc7481cf740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc7401b1e50 con 0x55df50a9c680 2024-09-12T02:11:48.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.201+0000 7fc7481cf740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc7401af720 con 0x55df50a91dc0 2024-09-12T02:11:48.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.201+0000 7fc7481cf740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc7401ad5e0 con 0x55df50a914a0 2024-09-12T02:11:48.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.201+0000 7fc73ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df50a9c680 0x55df50ab3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:48.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.201+0000 7fc73ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df50a9c680 0x55df50ab3870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36108/0 (socket says 172.21.15.16:36108) 2024-09-12T02:11:48.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.201+0000 7fc73ffff640 1 -- 172.21.15.16:0/103297210 learned_addr learned my addr 172.21.15.16:0/103297210 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:11:48.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.201+0000 7fc73f7fe640 1 --2- 172.21.15.16:0/103297210 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55df50a914a0 0x55df50a94900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:48.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.201+0000 7fc73ffff640 1 -- 172.21.15.16:0/103297210 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55df50a914a0 msgr2=0x55df50a94900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:48.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.201+0000 7fc73ffff640 1 --2- 172.21.15.16:0/103297210 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55df50a914a0 0x55df50a94900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:48.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.201+0000 7fc73ffff640 1 -- 172.21.15.16:0/103297210 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55df50a91dc0 msgr2=0x55df50a98140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:48.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.201+0000 7fc73effd640 1 --2- 172.21.15.16:0/103297210 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55df50a91dc0 0x55df50a98140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:48.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.201+0000 7fc73ffff640 1 --2- 172.21.15.16:0/103297210 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55df50a91dc0 0x55df50a98140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:48.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.201+0000 7fc73ffff640 1 -- 172.21.15.16:0/103297210 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df50a8ebb0 con 0x55df50a9c680 2024-09-12T02:11:48.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.201+0000 7fc73f7fe640 1 --2- 172.21.15.16:0/103297210 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55df50a914a0 0x55df50a94900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:11:48.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.202+0000 7fc73ffff640 1 --2- 172.21.15.16:0/103297210 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df50a9c680 0x55df50ab3870 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7fc738009c90 tx=0x7fc738004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1c2ca89cff6918b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:48.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.202+0000 7fc7257fa640 1 -- 172.21.15.16:0/103297210 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc738024030 con 0x55df50a9c680 2024-09-12T02:11:48.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.202+0000 7fc7257fa640 1 -- 172.21.15.16:0/103297210 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc738010d00 con 0x55df50a9c680 2024-09-12T02:11:48.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.202+0000 7fc7257fa640 1 -- 172.21.15.16:0/103297210 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc738020790 con 0x55df50a9c680 2024-09-12T02:11:48.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.202+0000 7fc7481cf740 1 -- 172.21.15.16:0/103297210 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df50a9c680 msgr2=0x55df50ab3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:48.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.202+0000 7fc7481cf740 1 --2- 172.21.15.16:0/103297210 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df50a9c680 0x55df50ab3870 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7fc738009c90 tx=0x7fc738004ab0 comp rx=0 tx=0).stop 2024-09-12T02:11:48.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.203+0000 7fc7481cf740 1 -- 172.21.15.16:0/103297210 shutdown_connections 2024-09-12T02:11:48.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.203+0000 7fc7481cf740 1 --2- 172.21.15.16:0/103297210 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df50a9c680 0x55df50ab3870 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:48.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.203+0000 7fc7481cf740 1 --2- 172.21.15.16:0/103297210 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55df50a91dc0 0x55df50a98140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:48.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.203+0000 7fc7481cf740 1 --2- 172.21.15.16:0/103297210 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55df50a914a0 0x55df50a94900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:48.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.203+0000 7fc7481cf740 1 -- 172.21.15.16:0/103297210 >> 172.21.15.16:0/103297210 conn(0x55df50a8da30 msgr2=0x55df50a97a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:48.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.203+0000 7fc7481cf740 1 -- 172.21.15.16:0/103297210 shutdown_connections 2024-09-12T02:11:48.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.203+0000 7fc7481cf740 1 -- 172.21.15.16:0/103297210 wait complete. 2024-09-12T02:11:48.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.203+0000 7fc7481cf740 1 -- start start 2024-09-12T02:11:48.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.203+0000 7fc7481cf740 1 Processor -- start 2024-09-12T02:11:48.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.203+0000 7fc7481cf740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df50a914a0 0x55df50aa4e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:48.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.203+0000 7fc7481cf740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55df50a91dc0 0x55df50a95f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:48.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.203+0000 7fc7481cf740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55df50a9c680 0x55df50a965a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:48.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.203+0000 7fc7481cf740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc7401adc30 con 0x55df50a914a0 2024-09-12T02:11:48.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.203+0000 7fc7481cf740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc7401af050 con 0x55df50a9c680 2024-09-12T02:11:48.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.203+0000 7fc7481cf740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc7401ae2b0 con 0x55df50a91dc0 2024-09-12T02:11:48.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.204+0000 7fc7481cf740 10 client.0 _ll_register_callbacks cb 0x7fc730005d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:11:48.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.204+0000 7fc73f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df50a914a0 0x55df50aa4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:48.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.204+0000 7fc73f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df50a914a0 0x55df50aa4e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36120/0 (socket says 172.21.15.16:36120) 2024-09-12T02:11:48.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.204+0000 7fc73f7fe640 1 -- 172.21.15.16:0/2432427593 learned_addr learned my addr 172.21.15.16:0/2432427593 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:11:48.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.204+0000 7fc73effd640 1 --2- 172.21.15.16:0/2432427593 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55df50a91dc0 0x55df50a95f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:48.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.204+0000 7fc73ffff640 1 --2- 172.21.15.16:0/2432427593 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55df50a9c680 0x55df50a965a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:48.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.204+0000 7fc73f7fe640 1 -- 172.21.15.16:0/2432427593 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55df50a91dc0 msgr2=0x55df50a95f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:48.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.204+0000 7fc73f7fe640 1 --2- 172.21.15.16:0/2432427593 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55df50a91dc0 0x55df50a95f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:48.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.204+0000 7fc73f7fe640 1 -- 172.21.15.16:0/2432427593 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55df50a9c680 msgr2=0x55df50a965a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:48.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.204+0000 7fc73f7fe640 1 --2- 172.21.15.16:0/2432427593 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55df50a9c680 0x55df50a965a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:48.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.204+0000 7fc73f7fe640 1 -- 172.21.15.16:0/2432427593 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df50a96da0 con 0x55df50a914a0 2024-09-12T02:11:48.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.204+0000 7fc73ffff640 1 --2- 172.21.15.16:0/2432427593 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55df50a9c680 0x55df50a965a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:11:48.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.204+0000 7fc73effd640 1 --2- 172.21.15.16:0/2432427593 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55df50a91dc0 0x55df50a95f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:11:48.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.204+0000 7fc73f7fe640 1 --2- 172.21.15.16:0/2432427593 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df50a914a0 0x55df50aa4e50 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7fc730003db0 tx=0x7fc7300095b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32395d075f30717c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:48.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.204+0000 7fc71effd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:11:48.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.204+0000 7fc71effd640 1 -- 172.21.15.16:0/2432427593 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc730060060 con 0x55df50a914a0 2024-09-12T02:11:48.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.204+0000 7fc71effd640 1 -- 172.21.15.16:0/2432427593 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fc7300043e0 con 0x55df50a914a0 2024-09-12T02:11:48.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.204+0000 7fc7481cf740 1 -- 172.21.15.16:0/2432427593 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc738010d00 con 0x55df50a914a0 2024-09-12T02:11:48.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.204+0000 7fc71effd640 1 -- 172.21.15.16:0/2432427593 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc730040030 con 0x55df50a914a0 2024-09-12T02:11:48.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.204+0000 7fc7481cf740 10 client.18471 fetch_fsmap learned FSMap version 25 2024-09-12T02:11:48.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.204+0000 7fc7481cf740 1 -- 172.21.15.16:0/2432427593 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55df50b168a0 con 0x55df50a914a0 2024-09-12T02:11:48.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.205+0000 7fc71effd640 1 -- 172.21.15.16:0/2432427593 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc738010d00 con 0x55df50a914a0 2024-09-12T02:11:48.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.205+0000 7fc71effd640 1 -- 172.21.15.16:0/2432427593 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fc730065070 con 0x55df50a914a0 2024-09-12T02:11:48.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.205+0000 7fc7481cf740 10 client.18471 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:11:48.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.205+0000 7fc7481cf740 10 client.18471 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:11:48.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.205+0000 7fc7481cf740 20 client.18471 populate_metadata read hostname 'smithi016' 2024-09-12T02:11:48.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.205+0000 7fc7481cf740 1 --2- 172.21.15.16:0/2432427593 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55df50b16ff0 0x55df50b19410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:48.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.205+0000 7fc7481cf740 4 client.18471 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T02:11:48.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.205+0000 7fc7481cf740 1 -- 172.21.15.16:0/2432427593 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55df509fed60 con 0x55df50b16ff0 2024-09-12T02:11:48.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.205+0000 7fc73effd640 1 --2- 172.21.15.16:0/2432427593 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55df50b16ff0 0x55df50b19410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:48.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.206+0000 7fc73effd640 1 --2- 172.21.15.16:0/2432427593 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55df50b16ff0 0x55df50b19410 crc :-1 s=READY pgs=291 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e40bc3b87091b8e3 server_cookie=340ec6109ba06af7 in_seq=0 out_seq=0 2024-09-12T02:11:48.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.206+0000 7fc71effd640 10 client.18471 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:11:48.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.206+0000 7fc71effd640 1 -- 172.21.15.16:0/2432427593 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55df509fed60 con 0x55df50b16ff0 2024-09-12T02:11:48.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.206+0000 7fc71effd640 10 client.18471 handle_command_reply: tid=0 2024-09-12T02:11:48.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.206+0000 7fc7481cf740 1 client.18471 shutdown 2024-09-12T02:11:48.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.206+0000 7fc7481cf740 1 -- 172.21.15.16:0/2432427593 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df50a914a0 msgr2=0x55df50aa4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:48.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.206+0000 7fc7481cf740 1 --2- 172.21.15.16:0/2432427593 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df50a914a0 0x55df50aa4e50 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7fc730003db0 tx=0x7fc7300095b0 comp rx=0 tx=0).stop 2024-09-12T02:11:48.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.206+0000 7fc7481cf740 1 -- 172.21.15.16:0/2432427593 shutdown_connections 2024-09-12T02:11:48.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.206+0000 7fc7481cf740 1 --2- 172.21.15.16:0/2432427593 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55df50b16ff0 0x55df50b19410 crc :-1 s=READY pgs=291 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:48.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.206+0000 7fc7481cf740 1 --2- 172.21.15.16:0/2432427593 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55df50a9c680 0x55df50a965a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:48.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.206+0000 7fc7481cf740 1 --2- 172.21.15.16:0/2432427593 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55df50a91dc0 0x55df50a95f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:48.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.206+0000 7fc7481cf740 1 --2- 172.21.15.16:0/2432427593 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df50a914a0 0x55df50aa4e50 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:48.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.206+0000 7fc71effd640 0 client.18471 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:11:48.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.206+0000 7fc7481cf740 1 -- 172.21.15.16:0/2432427593 >> 172.21.15.16:0/2432427593 conn(0x55df50a8da30 msgr2=0x55df50a94760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:48.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.206+0000 7fc7481cf740 1 -- 172.21.15.16:0/2432427593 shutdown_connections 2024-09-12T02:11:48.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.206+0000 7fc7481cf740 1 -- 172.21.15.16:0/2432427593 wait complete. 2024-09-12T02:11:48.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.206+0000 7fc7481cf740 20 client.18471 trim_cache size 0 max 16384 2024-09-12T02:11:48.215 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T02:11:48.216 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-09-12T02:11:48.216 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {} 2024-09-12T02:11:48.216 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T02:11:48.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.207+0000 7fc745df6640 1 -- 172.21.15.16:0/1649660018 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc7280a2b80 msgr2=0x7fc7280a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:48.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.207+0000 7fc745df6640 1 --2- 172.21.15.16:0/1649660018 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc7280a2b80 0x7fc7280a5040 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x55df50a8edb0 tx=0x7fc738009a50 comp rx=0 tx=0).stop 2024-09-12T02:11:48.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.207+0000 7fc745df6640 1 -- 172.21.15.16:0/1649660018 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc740102790 msgr2=0x7fc740074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:48.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.207+0000 7fc745df6640 1 --2- 172.21.15.16:0/1649660018 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc740102790 0x7fc740074a30 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7fc73400d950 tx=0x7fc73400de10 comp rx=0 tx=0).stop 2024-09-12T02:11:48.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.207+0000 7fc745df6640 1 -- 172.21.15.16:0/1649660018 shutdown_connections 2024-09-12T02:11:48.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.207+0000 7fc745df6640 1 --2- 172.21.15.16:0/1649660018 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc7280a2b80 0x7fc7280a5040 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:48.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.207+0000 7fc745df6640 1 --2- 172.21.15.16:0/1649660018 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc740106f10 0x7fc740074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:48.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.207+0000 7fc745df6640 1 --2- 172.21.15.16:0/1649660018 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc740102790 0x7fc740074a30 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:48.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.207+0000 7fc745df6640 1 --2- 172.21.15.16:0/1649660018 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc740101de0 0x7fc7400744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:48.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.207+0000 7fc745df6640 1 -- 172.21.15.16:0/1649660018 >> 172.21.15.16:0/1649660018 conn(0x7fc7400fd8c0 msgr2=0x7fc7400fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:48.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.207+0000 7fc745df6640 1 -- 172.21.15.16:0/1649660018 shutdown_connections 2024-09-12T02:11:48.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:48.207+0000 7fc745df6640 1 -- 172.21.15.16:0/1649660018 wait complete. 2024-09-12T02:11:48.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:11:48 smithi033 ceph-mon[35332]: pgmap v631: 97 pgs: 97 active+clean; 1.1 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 3.2 KiB/s wr, 10 op/s 2024-09-12T02:11:48.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:11:48 smithi016 ceph-mon[30947]: pgmap v631: 97 pgs: 97 active+clean; 1.1 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 3.2 KiB/s wr, 10 op/s 2024-09-12T02:11:48.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:11:48 smithi184 ceph-mon[35593]: pgmap v631: 97 pgs: 97 active+clean; 1.1 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 3.2 KiB/s wr, 10 op/s 2024-09-12T02:11:48.935 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-12T02:11:48.935 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:11:48.973 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:11:49.101 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.101+0000 7f0f19230640 1 -- 172.21.15.16:0/3610697420 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f14107820 msgr2=0x7f0f14109c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:49.101 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.101+0000 7f0f19230640 1 --2- 172.21.15.16:0/3610697420 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f14107820 0x7f0f14109c90 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f0f0c009ca0 tx=0x7f0f0c02a8c0 comp rx=0 tx=0).stop 2024-09-12T02:11:49.101 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.101+0000 7f0f19230640 1 -- 172.21.15.16:0/3610697420 shutdown_connections 2024-09-12T02:11:49.102 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.101+0000 7f0f19230640 1 --2- 172.21.15.16:0/3610697420 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f14107820 0x7f0f14109c90 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:49.102 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.101+0000 7f0f19230640 1 --2- 172.21.15.16:0/3610697420 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f14104ef0 0x7f0f141072e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:49.102 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.101+0000 7f0f19230640 1 --2- 172.21.15.16:0/3610697420 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0f141025c0 0x7f0f141049b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:49.102 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.101+0000 7f0f19230640 1 -- 172.21.15.16:0/3610697420 >> 172.21.15.16:0/3610697420 conn(0x7f0f140fc1e0 msgr2=0x7f0f140fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:49.102 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.101+0000 7f0f19230640 1 -- 172.21.15.16:0/3610697420 shutdown_connections 2024-09-12T02:11:49.102 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.101+0000 7f0f19230640 1 -- 172.21.15.16:0/3610697420 wait complete. 2024-09-12T02:11:49.102 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.102+0000 7f0f19230640 1 Processor -- start 2024-09-12T02:11:49.104 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.102+0000 7f0f19230640 1 -- start start 2024-09-12T02:11:49.104 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.102+0000 7f0f19230640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f141025c0 0x7f0f14101f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:49.104 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.102+0000 7f0f19230640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0f14104ef0 0x7f0f14100570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:49.104 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.102+0000 7f0f19230640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f14107820 0x7f0f14100ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:49.105 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.102+0000 7f0f19230640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0f1410d800 con 0x7f0f14107820 2024-09-12T02:11:49.105 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.102+0000 7f0f19230640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0f1410d680 con 0x7f0f141025c0 2024-09-12T02:11:49.105 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.102+0000 7f0f19230640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f0f1410d980 con 0x7f0f14104ef0 2024-09-12T02:11:49.105 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.102+0000 7f0f1359e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f14107820 0x7f0f14100ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:49.105 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.102+0000 7f0f1259c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0f14104ef0 0x7f0f14100570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:49.105 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.102+0000 7f0f1259c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0f14104ef0 0x7f0f14100570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:42160/0 (socket says 172.21.15.16:42160) 2024-09-12T02:11:49.105 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.102+0000 7f0f1359e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f14107820 0x7f0f14100ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36146/0 (socket says 172.21.15.16:36146) 2024-09-12T02:11:49.105 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.102+0000 7f0f1259c640 1 -- 172.21.15.16:0/211823397 learned_addr learned my addr 172.21.15.16:0/211823397 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:11:49.105 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.102+0000 7f0f12d9d640 1 --2- 172.21.15.16:0/211823397 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f141025c0 0x7f0f14101f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:49.105 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.103+0000 7f0f1359e640 1 -- 172.21.15.16:0/211823397 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0f14104ef0 msgr2=0x7f0f14100570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:49.105 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.103+0000 7f0f1359e640 1 --2- 172.21.15.16:0/211823397 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0f14104ef0 0x7f0f14100570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:49.105 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.103+0000 7f0f1359e640 1 -- 172.21.15.16:0/211823397 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f141025c0 msgr2=0x7f0f14101f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:49.105 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.103+0000 7f0f1359e640 1 --2- 172.21.15.16:0/211823397 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f141025c0 0x7f0f14101f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:49.106 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.103+0000 7f0f1359e640 1 -- 172.21.15.16:0/211823397 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f14101340 con 0x7f0f14107820 2024-09-12T02:11:49.106 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.103+0000 7f0f12d9d640 1 --2- 172.21.15.16:0/211823397 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f141025c0 0x7f0f14101f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:11:49.106 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.103+0000 7f0f1359e640 1 --2- 172.21.15.16:0/211823397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f14107820 0x7f0f14100ab0 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f0f0c02ade0 tx=0x7f0f0c031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:49.106 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.103+0000 7f0efbfff640 1 -- 172.21.15.16:0/211823397 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f0c003c30 con 0x7f0f14107820 2024-09-12T02:11:49.106 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.103+0000 7f0efbfff640 1 -- 172.21.15.16:0/211823397 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f0c003dd0 con 0x7f0f14107820 2024-09-12T02:11:49.106 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.103+0000 7f0f19230640 1 -- 172.21.15.16:0/211823397 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f141a1f70 con 0x7f0f14107820 2024-09-12T02:11:49.106 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.103+0000 7f0efbfff640 1 -- 172.21.15.16:0/211823397 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f0c02fd40 con 0x7f0f14107820 2024-09-12T02:11:49.106 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.103+0000 7f0f19230640 1 -- 172.21.15.16:0/211823397 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f141a2430 con 0x7f0f14107820 2024-09-12T02:11:49.109 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.106+0000 7f0f1bfc8740 1 Processor -- start 2024-09-12T02:11:49.109 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.106+0000 7f0efbfff640 1 -- 172.21.15.16:0/211823397 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f0f0c004050 con 0x7f0f14107820 2024-09-12T02:11:49.109 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.107+0000 7f0f1bfc8740 1 -- start start 2024-09-12T02:11:49.109 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.107+0000 7f0f1bfc8740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5650761119a0 0x565076111da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:49.109 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.107+0000 7f0f1bfc8740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5650761122e0 0x565076116da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:49.109 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.107+0000 7f0f1bfc8740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565076117370 0x565076119830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:49.109 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.107+0000 7f0f1bfc8740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5650760752f0 con 0x565076117370 2024-09-12T02:11:49.109 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.107+0000 7f0f1bfc8740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56507601d270 con 0x5650761119a0 2024-09-12T02:11:49.110 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.107+0000 7f0f1bfc8740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5650760712b0 con 0x5650761122e0 2024-09-12T02:11:49.110 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.107+0000 7f0f1259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565076117370 0x565076119830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:49.110 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.107+0000 7f0f1259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565076117370 0x565076119830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36156/0 (socket says 172.21.15.16:36156) 2024-09-12T02:11:49.110 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.107+0000 7f0f1259c640 1 -- 172.21.15.16:0/234554678 learned_addr learned my addr 172.21.15.16:0/234554678 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:11:49.110 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.107+0000 7f0f1259c640 1 --2- 172.21.15.16:0/234554678 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5650761119a0 0x565076111da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:49.110 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.107+0000 7f0f12d9d640 1 --2- 172.21.15.16:0/234554678 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5650761122e0 0x565076116da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:49.110 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.107+0000 7f0efbfff640 1 --2- 172.21.15.16:0/211823397 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f0efc0a2cb0 0x7f0efc0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:49.110 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.107+0000 7f0f1259c640 1 -- 172.21.15.16:0/234554678 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5650761122e0 msgr2=0x565076116da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:49.110 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.107+0000 7f0f1259c640 1 --2- 172.21.15.16:0/234554678 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5650761122e0 0x565076116da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:49.110 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.107+0000 7f0f1259c640 1 -- 172.21.15.16:0/234554678 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5650761119a0 msgr2=0x565076111da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:49.110 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.107+0000 7f0f1259c640 1 --2- 172.21.15.16:0/234554678 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5650761119a0 0x565076111da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:49.110 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.107+0000 7f0f1259c640 1 -- 172.21.15.16:0/234554678 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56507601c340 con 0x565076117370 2024-09-12T02:11:49.110 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.108+0000 7f0f1359e640 1 --2- 172.21.15.16:0/211823397 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f0efc0a2cb0 0x7f0efc0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:49.111 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.108+0000 7f0efbfff640 1 -- 172.21.15.16:0/211823397 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f0f0c0f00b0 con 0x7f0f14107820 2024-09-12T02:11:49.111 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.108+0000 7f0f1259c640 1 --2- 172.21.15.16:0/234554678 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565076117370 0x565076119830 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f0f080117a0 tx=0x7f0f0800a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9346e9eedc374eb3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:49.111 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.108+0000 7f0ef97fa640 1 -- 172.21.15.16:0/234554678 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f08004020 con 0x565076117370 2024-09-12T02:11:49.111 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.108+0000 7f0ef97fa640 1 -- 172.21.15.16:0/234554678 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f08005500 con 0x565076117370 2024-09-12T02:11:49.111 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.108+0000 7f0ef97fa640 1 -- 172.21.15.16:0/234554678 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f08004b90 con 0x565076117370 2024-09-12T02:11:49.111 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.109+0000 7f0f1bfc8740 1 -- 172.21.15.16:0/234554678 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565076117370 msgr2=0x565076119830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:49.111 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.109+0000 7f0f1bfc8740 1 --2- 172.21.15.16:0/234554678 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565076117370 0x565076119830 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f0f080117a0 tx=0x7f0f0800a730 comp rx=0 tx=0).stop 2024-09-12T02:11:49.111 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.109+0000 7f0f1359e640 1 --2- 172.21.15.16:0/211823397 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f0efc0a2cb0 0x7f0efc0a5170 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f0f0c0b8e10 tx=0x7f0f0c0b1e90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:49.111 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.109+0000 7f0f1bfc8740 1 -- 172.21.15.16:0/234554678 shutdown_connections 2024-09-12T02:11:49.111 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.109+0000 7f0f1bfc8740 1 --2- 172.21.15.16:0/234554678 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565076117370 0x565076119830 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:49.111 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.109+0000 7f0f1bfc8740 1 --2- 172.21.15.16:0/234554678 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5650761122e0 0x565076116da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:49.111 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.109+0000 7f0f1bfc8740 1 --2- 172.21.15.16:0/234554678 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5650761119a0 0x565076111da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:49.111 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.109+0000 7f0f1bfc8740 1 -- 172.21.15.16:0/234554678 >> 172.21.15.16:0/234554678 conn(0x56507610d010 msgr2=0x56507610d410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:49.111 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.109+0000 7f0f1bfc8740 1 -- 172.21.15.16:0/234554678 shutdown_connections 2024-09-12T02:11:49.112 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.109+0000 7f0f1bfc8740 1 -- 172.21.15.16:0/234554678 wait complete. 2024-09-12T02:11:49.112 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.109+0000 7f0f1bfc8740 1 -- start start 2024-09-12T02:11:49.112 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.109+0000 7f0f1bfc8740 1 Processor -- start 2024-09-12T02:11:49.115 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.109+0000 7f0f1bfc8740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5650761119a0 0x56507611bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:49.115 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.109+0000 7f0f1bfc8740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5650761122e0 0x56507611c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:49.115 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.109+0000 7f0f1bfc8740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565076117370 0x565076123140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:49.115 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.109+0000 7f0f1bfc8740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56507611a340 con 0x565076117370 2024-09-12T02:11:49.115 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.109+0000 7f0f1bfc8740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x565075ffe290 con 0x5650761119a0 2024-09-12T02:11:49.115 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.109+0000 7f0f1bfc8740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x56507611a4c0 con 0x5650761122e0 2024-09-12T02:11:49.115 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.110+0000 7f0f1bfc8740 10 client.0 _ll_register_callbacks cb 0x56507602f140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:11:49.116 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.110+0000 7f0f1259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565076117370 0x565076123140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:49.116 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.110+0000 7f0f1259c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5650761119a0 0x56507611bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:49.116 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.110+0000 7f0f1259c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5650761119a0 0x56507611bbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.16:51288/0 (socket says 172.21.15.16:51288) 2024-09-12T02:11:49.116 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.110+0000 7f0f1259c640 1 -- 172.21.15.16:0/1339079506 learned_addr learned my addr 172.21.15.16:0/1339079506 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:11:49.116 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.110+0000 7f0f12d9d640 1 --2- 172.21.15.16:0/1339079506 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5650761122e0 0x56507611c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:49.116 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.110+0000 7f0f1259c640 1 -- 172.21.15.16:0/1339079506 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5650761122e0 msgr2=0x56507611c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:49.116 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.110+0000 7f0f1259c640 1 --2- 172.21.15.16:0/1339079506 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5650761122e0 0x56507611c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:49.116 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.110+0000 7f0f1259c640 1 -- 172.21.15.16:0/1339079506 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565076117370 msgr2=0x565076123140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:49.116 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.110+0000 7f0f1259c640 1 --2- 172.21.15.16:0/1339079506 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565076117370 0x565076123140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:49.116 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.110+0000 7f0f1259c640 1 -- 172.21.15.16:0/1339079506 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565075d88210 con 0x5650761119a0 2024-09-12T02:11:49.116 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.110+0000 7f0f1259c640 1 --2- 172.21.15.16:0/1339079506 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5650761119a0 0x56507611bbc0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f0f08005ce0 tx=0x7f0f0800a730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad5094e2557e74f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:49.117 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.110+0000 7f0ef6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-12T02:11:49.117 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.110+0000 7f0ef6ff5640 1 -- 172.21.15.16:0/1339079506 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f08022070 con 0x5650761119a0 2024-09-12T02:11:49.117 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.110+0000 7f0ef6ff5640 1 -- 172.21.15.16:0/1339079506 <== mon.2 v2:172.21.15.33:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f0f0800f790 con 0x5650761119a0 2024-09-12T02:11:49.117 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.110+0000 7f0f1bfc8740 1 -- 172.21.15.16:0/1339079506 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f08005500 con 0x5650761119a0 2024-09-12T02:11:49.117 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.111+0000 7f0ef6ff5640 1 -- 172.21.15.16:0/1339079506 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f0f08028040 con 0x5650761119a0 2024-09-12T02:11:49.117 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.111+0000 7f0f1bfc8740 10 client.27638 fetch_fsmap learned FSMap version 25 2024-09-12T02:11:49.117 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.111+0000 7f0f1bfc8740 1 -- 172.21.15.16:0/1339079506 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x565075e54620 con 0x5650761119a0 2024-09-12T02:11:49.117 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.111+0000 7f0ef6ff5640 1 -- 172.21.15.16:0/1339079506 <== mon.2 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f08005500 con 0x5650761119a0 2024-09-12T02:11:49.117 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.111+0000 7f0ef6ff5640 1 -- 172.21.15.16:0/1339079506 <== mon.2 v2:172.21.15.33:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f0f0801ea30 con 0x5650761119a0 2024-09-12T02:11:49.117 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.111+0000 7f0f1bfc8740 10 client.27638 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:11:49.117 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.111+0000 7f0f1bfc8740 10 client.27638 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:11:49.117 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.111+0000 7f0f1bfc8740 20 client.27638 populate_metadata read hostname 'smithi016' 2024-09-12T02:11:49.118 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.111+0000 7f0f1bfc8740 1 --2- 172.21.15.16:0/1339079506 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x565076127590 0x565076129980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:49.118 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.111+0000 7f0f1bfc8740 4 client.27638 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:11:49.118 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.111+0000 7f0f1bfc8740 1 -- 172.21.15.16:0/1339079506 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565076082a20 con 0x565076127590 2024-09-12T02:11:49.118 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.111+0000 7f0f12d9d640 1 --2- 172.21.15.16:0/1339079506 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x565076127590 0x565076129980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:49.118 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.112+0000 7f0f12d9d640 1 --2- 172.21.15.16:0/1339079506 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x565076127590 0x565076129980 crc :-1 s=READY pgs=292 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7cdf18f342439938 server_cookie=1b71eecd3c228db5 in_seq=0 out_seq=0 2024-09-12T02:11:49.118 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.112+0000 7f0ef6ff5640 10 client.27638 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:11:49.118 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.112+0000 7f0ef6ff5640 1 -- 172.21.15.16:0/1339079506 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x565076082a20 con 0x565076127590 2024-09-12T02:11:49.118 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.112+0000 7f0ef6ff5640 10 client.27638 handle_command_reply: tid=0 2024-09-12T02:11:49.118 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.112+0000 7f0f1bfc8740 1 client.27638 shutdown 2024-09-12T02:11:49.118 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.113+0000 7f0f1bfc8740 1 -- 172.21.15.16:0/1339079506 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5650761119a0 msgr2=0x56507611bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:49.118 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.113+0000 7f0f1bfc8740 1 --2- 172.21.15.16:0/1339079506 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5650761119a0 0x56507611bbc0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f0f08005ce0 tx=0x7f0f0800a730 comp rx=0 tx=0).stop 2024-09-12T02:11:49.118 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.113+0000 7f0f1bfc8740 1 -- 172.21.15.16:0/1339079506 shutdown_connections 2024-09-12T02:11:49.118 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.113+0000 7f0f1bfc8740 1 --2- 172.21.15.16:0/1339079506 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x565076127590 0x565076129980 crc :-1 s=READY pgs=292 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:49.119 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.113+0000 7f0f1bfc8740 1 --2- 172.21.15.16:0/1339079506 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565076117370 0x565076123140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:49.119 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.113+0000 7f0f1bfc8740 1 --2- 172.21.15.16:0/1339079506 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5650761122e0 0x56507611c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:49.119 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.113+0000 7f0f1bfc8740 1 --2- 172.21.15.16:0/1339079506 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5650761119a0 0x56507611bbc0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:49.119 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.113+0000 7f0ef6ff5640 0 client.27638 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:11:49.119 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.113+0000 7f0f1bfc8740 1 -- 172.21.15.16:0/1339079506 >> 172.21.15.16:0/1339079506 conn(0x56507610d010 msgr2=0x56507610e820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:49.119 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.113+0000 7f0f1bfc8740 1 -- 172.21.15.16:0/1339079506 shutdown_connections 2024-09-12T02:11:49.119 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.113+0000 7f0f1bfc8740 1 -- 172.21.15.16:0/1339079506 wait complete. 2024-09-12T02:11:49.119 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.113+0000 7f0f1bfc8740 20 client.27638 trim_cache size 0 max 16384 2024-09-12T02:11:49.128 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.127+0000 7f0f1bfc8740 1 Processor -- start 2024-09-12T02:11:49.128 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.127+0000 7f0f1bfc8740 1 -- start start 2024-09-12T02:11:49.129 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.127+0000 7f0f1bfc8740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5650761119a0 0x56507610dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:49.129 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.127+0000 7f0f1bfc8740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650761122e0 0x565076123100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:49.129 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.127+0000 7f0f1bfc8740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565076117370 0x56507610e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:49.129 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.127+0000 7f0f1bfc8740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0f141ad990 con 0x5650761122e0 2024-09-12T02:11:49.129 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.127+0000 7f0f1bfc8740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0f141ab260 con 0x565076117370 2024-09-12T02:11:49.129 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.127+0000 7f0f1bfc8740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f0f141a90d0 con 0x5650761119a0 2024-09-12T02:11:49.129 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.127+0000 7f0f12d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650761122e0 0x565076123100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:49.130 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.128+0000 7f0f12d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650761122e0 0x565076123100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36170/0 (socket says 172.21.15.16:36170) 2024-09-12T02:11:49.130 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.128+0000 7f0f12d9d640 1 -- 172.21.15.16:0/2141946201 learned_addr learned my addr 172.21.15.16:0/2141946201 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:11:49.130 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.128+0000 7f0f1259c640 1 --2- 172.21.15.16:0/2141946201 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565076117370 0x56507610e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:49.130 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.128+0000 7f0f1259c640 1 --2- 172.21.15.16:0/2141946201 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5650761119a0 0x56507610dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:49.130 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.128+0000 7f0f12d9d640 1 -- 172.21.15.16:0/2141946201 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5650761119a0 msgr2=0x56507610dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:49.130 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.128+0000 7f0f12d9d640 1 --2- 172.21.15.16:0/2141946201 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5650761119a0 0x56507610dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:49.130 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.128+0000 7f0f12d9d640 1 -- 172.21.15.16:0/2141946201 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565076117370 msgr2=0x56507610e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:49.130 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.128+0000 7f0f12d9d640 1 --2- 172.21.15.16:0/2141946201 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565076117370 0x56507610e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:49.130 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.128+0000 7f0f12d9d640 1 -- 172.21.15.16:0/2141946201 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565076129fb0 con 0x5650761122e0 2024-09-12T02:11:49.130 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.128+0000 7f0f12d9d640 1 --2- 172.21.15.16:0/2141946201 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650761122e0 0x565076123100 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x56507610de90 tx=0x7f0f04005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22a879893ae4e373 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:49.130 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.128+0000 7f0ef8ff9640 1 -- 172.21.15.16:0/2141946201 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f040142d0 con 0x5650761122e0 2024-09-12T02:11:49.130 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.128+0000 7f0ef8ff9640 1 -- 172.21.15.16:0/2141946201 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f040119e0 con 0x5650761122e0 2024-09-12T02:11:49.130 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.128+0000 7f0ef8ff9640 1 -- 172.21.15.16:0/2141946201 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f0401b9a0 con 0x5650761122e0 2024-09-12T02:11:49.134 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.129+0000 7f0f1bfc8740 1 -- 172.21.15.16:0/2141946201 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650761122e0 msgr2=0x565076123100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:49.134 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.129+0000 7f0f1bfc8740 1 --2- 172.21.15.16:0/2141946201 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650761122e0 0x565076123100 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x56507610de90 tx=0x7f0f04005850 comp rx=0 tx=0).stop 2024-09-12T02:11:49.134 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.129+0000 7f0f1bfc8740 1 -- 172.21.15.16:0/2141946201 shutdown_connections 2024-09-12T02:11:49.134 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.129+0000 7f0f1bfc8740 1 --2- 172.21.15.16:0/2141946201 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565076117370 0x56507610e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:49.135 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.129+0000 7f0f1bfc8740 1 --2- 172.21.15.16:0/2141946201 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650761122e0 0x565076123100 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:49.135 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.129+0000 7f0f1bfc8740 1 --2- 172.21.15.16:0/2141946201 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5650761119a0 0x56507610dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:49.135 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.129+0000 7f0f1bfc8740 1 -- 172.21.15.16:0/2141946201 >> 172.21.15.16:0/2141946201 conn(0x56507610d010 msgr2=0x56507611b970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:49.135 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.129+0000 7f0f1bfc8740 1 -- 172.21.15.16:0/2141946201 shutdown_connections 2024-09-12T02:11:49.135 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.129+0000 7f0f1bfc8740 1 -- 172.21.15.16:0/2141946201 wait complete. 2024-09-12T02:11:49.135 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.129+0000 7f0f1bfc8740 1 -- start start 2024-09-12T02:11:49.135 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.129+0000 7f0f1bfc8740 1 Processor -- start 2024-09-12T02:11:49.135 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.130+0000 7f0f1bfc8740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5650761119a0 0x5650761204c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:49.135 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.130+0000 7f0f1bfc8740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5650761122e0 0x565076120a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:49.135 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.130+0000 7f0f1bfc8740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565076117370 0x565076121a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:49.135 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.130+0000 7f0f1bfc8740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0f141a9720 con 0x565076117370 2024-09-12T02:11:49.135 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.130+0000 7f0f1bfc8740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0f141aab90 con 0x5650761122e0 2024-09-12T02:11:49.135 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.130+0000 7f0f1bfc8740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f0f141a9df0 con 0x5650761119a0 2024-09-12T02:11:49.136 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.130+0000 7f0f1bfc8740 10 client.0 _ll_register_callbacks cb 0x7f0f0800ae40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:11:49.136 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.130+0000 7f0f1259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565076117370 0x565076121a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:49.136 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.130+0000 7f0f1259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565076117370 0x565076121a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36176/0 (socket says 172.21.15.16:36176) 2024-09-12T02:11:49.136 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.130+0000 7f0f1259c640 1 -- 172.21.15.16:0/4288422998 learned_addr learned my addr 172.21.15.16:0/4288422998 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:11:49.136 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.130+0000 7f0f1259c640 1 --2- 172.21.15.16:0/4288422998 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5650761119a0 0x5650761204c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:49.136 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.130+0000 7f0f12d9d640 1 --2- 172.21.15.16:0/4288422998 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5650761122e0 0x565076120a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:49.136 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.130+0000 7f0f1259c640 1 -- 172.21.15.16:0/4288422998 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5650761119a0 msgr2=0x5650761204c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:49.136 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.130+0000 7f0f1259c640 1 --2- 172.21.15.16:0/4288422998 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5650761119a0 0x5650761204c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:49.136 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.130+0000 7f0f1259c640 1 -- 172.21.15.16:0/4288422998 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5650761122e0 msgr2=0x565076120a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:49.136 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.130+0000 7f0f1259c640 1 --2- 172.21.15.16:0/4288422998 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5650761122e0 0x565076120a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:49.136 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.130+0000 7f0f1259c640 1 -- 172.21.15.16:0/4288422998 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565076122020 con 0x565076117370 2024-09-12T02:11:49.136 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.130+0000 7f0f12d9d640 1 --2- 172.21.15.16:0/4288422998 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5650761122e0 0x565076120a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:11:49.137 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.130+0000 7f0f1259c640 1 --2- 172.21.15.16:0/4288422998 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565076117370 0x565076121a10 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f0f0801eed0 tx=0x7f0f08004420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76863d6f743f928d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:49.137 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.130+0000 7f0ef6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:11:49.137 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.131+0000 7f0ef6ff5640 1 -- 172.21.15.16:0/4288422998 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f08022040 con 0x565076117370 2024-09-12T02:11:49.137 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.131+0000 7f0ef6ff5640 1 -- 172.21.15.16:0/4288422998 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f0f0800aa60 con 0x565076117370 2024-09-12T02:11:49.137 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.131+0000 7f0f1bfc8740 1 -- 172.21.15.16:0/4288422998 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f040119e0 con 0x565076117370 2024-09-12T02:11:49.137 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.131+0000 7f0ef6ff5640 1 -- 172.21.15.16:0/4288422998 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f0f0800ac60 con 0x565076117370 2024-09-12T02:11:49.137 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.131+0000 7f0f1bfc8740 10 client.18489 fetch_fsmap learned FSMap version 25 2024-09-12T02:11:49.137 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.131+0000 7f0f1bfc8740 1 -- 172.21.15.16:0/4288422998 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5650761233a0 con 0x565076117370 2024-09-12T02:11:49.137 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.131+0000 7f0ef6ff5640 1 -- 172.21.15.16:0/4288422998 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f040119e0 con 0x565076117370 2024-09-12T02:11:49.137 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.131+0000 7f0ef6ff5640 1 -- 172.21.15.16:0/4288422998 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f0f08011030 con 0x565076117370 2024-09-12T02:11:49.137 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.131+0000 7f0f1bfc8740 10 client.18489 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:11:49.137 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.131+0000 7f0f1bfc8740 10 client.18489 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:11:49.138 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.131+0000 7f0f1bfc8740 20 client.18489 populate_metadata read hostname 'smithi016' 2024-09-12T02:11:49.138 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.131+0000 7f0f1bfc8740 1 --2- 172.21.15.16:0/4288422998 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5650761928d0 0x565076194cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:49.138 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.131+0000 7f0f1bfc8740 4 client.18489 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:11:49.138 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.131+0000 7f0f1bfc8740 1 -- 172.21.15.16:0/4288422998 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56507607cd70 con 0x5650761928d0 2024-09-12T02:11:49.138 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.132+0000 7f0f12d9d640 1 --2- 172.21.15.16:0/4288422998 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5650761928d0 0x565076194cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:49.138 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.132+0000 7f0ef6ff5640 10 client.18489 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:11:49.138 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.132+0000 7f0f12d9d640 1 --2- 172.21.15.16:0/4288422998 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5650761928d0 0x565076194cf0 crc :-1 s=READY pgs=293 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f12d888e0448a7 server_cookie=da04a9a4e82be8d1 in_seq=0 out_seq=0 2024-09-12T02:11:49.143 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.142+0000 7f0ef6ff5640 1 -- 172.21.15.16:0/4288422998 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56507607cd70 con 0x5650761928d0 2024-09-12T02:11:49.143 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.142+0000 7f0ef6ff5640 10 client.18489 handle_command_reply: tid=0 2024-09-12T02:11:49.143 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.143+0000 7f0f1bfc8740 1 client.18489 shutdown 2024-09-12T02:11:49.143 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.143+0000 7f0f1bfc8740 1 -- 172.21.15.16:0/4288422998 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565076117370 msgr2=0x565076121a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:49.143 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.143+0000 7f0f1bfc8740 1 --2- 172.21.15.16:0/4288422998 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565076117370 0x565076121a10 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f0f0801eed0 tx=0x7f0f08004420 comp rx=0 tx=0).stop 2024-09-12T02:11:49.144 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.143+0000 7f0f1bfc8740 1 -- 172.21.15.16:0/4288422998 shutdown_connections 2024-09-12T02:11:49.144 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.143+0000 7f0f1bfc8740 1 --2- 172.21.15.16:0/4288422998 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5650761928d0 0x565076194cf0 crc :-1 s=READY pgs=293 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:49.144 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.143+0000 7f0f1bfc8740 1 --2- 172.21.15.16:0/4288422998 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565076117370 0x565076121a10 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:49.144 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.143+0000 7f0ef6ff5640 0 client.18489 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:11:49.144 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.143+0000 7f0f1bfc8740 1 --2- 172.21.15.16:0/4288422998 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5650761122e0 0x565076120a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:49.144 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.143+0000 7f0f1bfc8740 1 --2- 172.21.15.16:0/4288422998 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5650761119a0 0x5650761204c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:49.144 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.143+0000 7f0f1bfc8740 1 -- 172.21.15.16:0/4288422998 >> 172.21.15.16:0/4288422998 conn(0x56507610d010 msgr2=0x56507610b9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:49.144 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.143+0000 7f0f1bfc8740 1 -- 172.21.15.16:0/4288422998 shutdown_connections 2024-09-12T02:11:49.144 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.143+0000 7f0f1bfc8740 1 -- 172.21.15.16:0/4288422998 wait complete. 2024-09-12T02:11:49.144 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.143+0000 7f0f1bfc8740 20 client.18489 trim_cache size 0 max 16384 2024-09-12T02:11:49.145 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-09-12T02:11:49.145 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-09-12T02:11:49.145 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-09-12T02:11:49.145 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-09-12T02:11:49.146 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.144+0000 7f0f19230640 1 -- 172.21.15.16:0/211823397 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f0efc0a2cb0 msgr2=0x7f0efc0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:49.146 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.144+0000 7f0f19230640 1 --2- 172.21.15.16:0/211823397 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f0efc0a2cb0 0x7f0efc0a5170 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f0f0c0b8e10 tx=0x7f0f0c0b1e90 comp rx=0 tx=0).stop 2024-09-12T02:11:49.146 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.144+0000 7f0f19230640 1 -- 172.21.15.16:0/211823397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f14107820 msgr2=0x7f0f14100ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:49.146 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.144+0000 7f0f19230640 1 --2- 172.21.15.16:0/211823397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f14107820 0x7f0f14100ab0 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f0f0c02ade0 tx=0x7f0f0c031ee0 comp rx=0 tx=0).stop 2024-09-12T02:11:49.146 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.144+0000 7f0f19230640 1 -- 172.21.15.16:0/211823397 shutdown_connections 2024-09-12T02:11:49.146 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.144+0000 7f0f19230640 1 --2- 172.21.15.16:0/211823397 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f0efc0a2cb0 0x7f0efc0a5170 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:49.146 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.144+0000 7f0f19230640 1 --2- 172.21.15.16:0/211823397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f14107820 0x7f0f14100ab0 unknown :-1 s=CLOSED pgs=1396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:49.146 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.144+0000 7f0f19230640 1 --2- 172.21.15.16:0/211823397 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0f14104ef0 0x7f0f14100570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:49.146 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.144+0000 7f0f19230640 1 --2- 172.21.15.16:0/211823397 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f141025c0 0x7f0f14101f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:49.146 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.144+0000 7f0f19230640 1 -- 172.21.15.16:0/211823397 >> 172.21.15.16:0/211823397 conn(0x7f0f140fc1e0 msgr2=0x7f0f14102f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:49.146 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.144+0000 7f0f19230640 1 -- 172.21.15.16:0/211823397 shutdown_connections 2024-09-12T02:11:49.146 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:11:49.144+0000 7f0f19230640 1 -- 172.21.15.16:0/211823397 wait complete. 2024-09-12T02:11:49.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:49.472+0000 7fa757e29640 1 -- 172.21.15.16:0/1510932451 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa750102570 msgr2=0x7fa7501029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:49.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:49.472+0000 7fa757e29640 1 --2- 172.21.15.16:0/1510932451 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa750102570 0x7fa7501029b0 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7fa740009900 tx=0x7fa74002a480 comp rx=0 tx=0).stop 2024-09-12T02:11:49.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:49.472+0000 7fa757e29640 1 -- 172.21.15.16:0/1510932451 shutdown_connections 2024-09-12T02:11:49.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:49.472+0000 7fa757e29640 1 --2- 172.21.15.16:0/1510932451 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa750102ef0 0x7fa75010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:49.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:49.472+0000 7fa757e29640 1 --2- 172.21.15.16:0/1510932451 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa750102570 0x7fa7501029b0 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:49.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:49.472+0000 7fa757e29640 1 --2- 172.21.15.16:0/1510932451 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa750100e20 0x7fa750101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:49.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:49.472+0000 7fa757e29640 1 -- 172.21.15.16:0/1510932451 >> 172.21.15.16:0/1510932451 conn(0x7fa7500fc7d0 msgr2=0x7fa7500febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:49.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:49.472+0000 7fa757e29640 1 -- 172.21.15.16:0/1510932451 shutdown_connections 2024-09-12T02:11:49.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:49.472+0000 7fa757e29640 1 -- 172.21.15.16:0/1510932451 wait complete. 2024-09-12T02:11:49.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:49.473+0000 7fa757e29640 1 Processor -- start 2024-09-12T02:11:49.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:49.473+0000 7fa757e29640 1 -- start start 2024-09-12T02:11:49.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:49.473+0000 7fa757e29640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa750100e20 0x7fa750199350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:49.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:49.473+0000 7fa757e29640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa750102570 0x7fa750199890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:49.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:49.473+0000 7fa757e29640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa750102ef0 0x7fa7501a43d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:49.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:49.473+0000 7fa757e29640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa750110f80 con 0x7fa750102570 2024-09-12T02:11:49.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:49.473+0000 7fa757e29640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa750110e00 con 0x7fa750102ef0 2024-09-12T02:11:49.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:49.473+0000 7fa757e29640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa750111100 con 0x7fa750100e20 2024-09-12T02:11:49.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:49.473+0000 7fa7553c4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa750102570 0x7fa750199890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:49.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:49.474+0000 7fa7553c4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa750102570 0x7fa750199890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36198/0 (socket says 172.21.15.16:36198) 2024-09-12T02:11:49.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:49.474+0000 7fa7553c4640 1 -- 172.21.15.16:0/4169501887 learned_addr learned my addr 172.21.15.16:0/4169501887 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:11:49.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:49.474+0000 7fa755bc5640 1 --2- 172.21.15.16:0/4169501887 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa750100e20 0x7fa750199350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:49.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:49.474+0000 7fa7563c6640 1 --2- 172.21.15.16:0/4169501887 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa750102ef0 0x7fa7501a43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:49.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:49.474+0000 7fa7553c4640 1 -- 172.21.15.16:0/4169501887 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa750100e20 msgr2=0x7fa750199350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:49.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:49.474+0000 7fa7553c4640 1 --2- 172.21.15.16:0/4169501887 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa750100e20 0x7fa750199350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:49.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:49.474+0000 7fa7553c4640 1 -- 172.21.15.16:0/4169501887 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa750102ef0 msgr2=0x7fa7501a43d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:49.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:49.474+0000 7fa7553c4640 1 --2- 172.21.15.16:0/4169501887 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa750102ef0 0x7fa7501a43d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:49.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:49.474+0000 7fa7553c4640 1 -- 172.21.15.16:0/4169501887 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7501a4910 con 0x7fa750102570 2024-09-12T02:11:49.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:49.474+0000 7fa755bc5640 1 --2- 172.21.15.16:0/4169501887 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa750100e20 0x7fa750199350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:11:49.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:49.474+0000 7fa7563c6640 1 --2- 172.21.15.16:0/4169501887 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa750102ef0 0x7fa7501a43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:11:49.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:49.474+0000 7fa7553c4640 1 --2- 172.21.15.16:0/4169501887 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa750102570 0x7fa750199890 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7fa740006010 tx=0x7fa740003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:49.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:49.474+0000 7fa746ffd640 1 -- 172.21.15.16:0/4169501887 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa74003f070 con 0x7fa750102570 2024-09-12T02:11:49.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:49.474+0000 7fa757e29640 1 -- 172.21.15.16:0/4169501887 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7501a4ae0 con 0x7fa750102570 2024-09-12T02:11:49.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:49.475+0000 7fa746ffd640 1 -- 172.21.15.16:0/4169501887 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa740031c30 con 0x7fa750102570 2024-09-12T02:11:49.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:49.475+0000 7fa746ffd640 1 -- 172.21.15.16:0/4169501887 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa740035030 con 0x7fa750102570 2024-09-12T02:11:49.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:49.475+0000 7fa757e29640 1 -- 172.21.15.16:0/4169501887 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7501a4f90 con 0x7fa750102570 2024-09-12T02:11:49.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:49.476+0000 7fa757e29640 1 -- 172.21.15.16:0/4169501887 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa750101200 con 0x7fa750102570 2024-09-12T02:11:49.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:49.477+0000 7fa746ffd640 1 -- 172.21.15.16:0/4169501887 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa740042070 con 0x7fa750102570 2024-09-12T02:11:49.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:49.478+0000 7fa746ffd640 1 --2- 172.21.15.16:0/4169501887 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fa73c0a2b80 0x7fa73c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:49.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:49.478+0000 7fa755bc5640 1 --2- 172.21.15.16:0/4169501887 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fa73c0a2b80 0x7fa73c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:49.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:49.478+0000 7fa746ffd640 1 -- 172.21.15.16:0/4169501887 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fa7400ef3d0 con 0x7fa750102570 2024-09-12T02:11:49.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:49.478+0000 7fa755bc5640 1 --2- 172.21.15.16:0/4169501887 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fa73c0a2b80 0x7fa73c0a5040 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fa74c009fc0 tx=0x7fa74c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:49.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:49.482+0000 7fa746ffd640 1 -- 172.21.15.16:0/4169501887 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa7400bc330 con 0x7fa750102570 2024-09-12T02:11:49.503 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:11:49.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:49.704+0000 7fa757e29640 1 -- 172.21.15.16:0/4169501887 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa750109090 con 0x7fa750102570 2024-09-12T02:11:49.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:49.705+0000 7fa746ffd640 1 -- 172.21.15.16:0/4169501887 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11911 (secure 0 0 0) 0x7fa740031de0 con 0x7fa750102570 2024-09-12T02:11:49.706 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T02:11:49.706 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":25,"btime":"2024-09-12T02:01:37:192021+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3279123279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3279123279},{"type":"v1","addr":"172.21.15.16:6835","nonce":3279123279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4119452382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4119452382},{"type":"v1","addr":"172.21.15.16:6837","nonce":4119452382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/3263823165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3263823165},{"type":"v1","addr":"172.21.15.16:6839","nonce":3263823165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/1510290732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":1510290732},{"type":"v1","addr":"172.21.15.16:6841","nonce":1510290732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/279674603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":279674603},{"type":"v1","addr":"172.21.15.33:6833","nonce":279674603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3555728028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3555728028},{"type":"v1","addr":"172.21.15.33:6835","nonce":3555728028}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T01:59:56.117608+0000","modified":"2024-09-12T02:01:36.294760+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24482,"mds_1":24481,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6833/1563374106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1563374106},{"type":"v1","addr":"172.21.15.184:6833","nonce":1563374106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6837/1892833868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1892833868},{"type":"v1","addr":"172.21.15.33:6837","nonce":1892833868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/1457679686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1457679686},{"type":"v1","addr":"172.21.15.184:6835","nonce":1457679686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.184:6837/1793393087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":1793393087},{"type":"v1","addr":"172.21.15.184:6837","nonce":1793393087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.184:6839/4203347941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":4203347941},{"type":"v1","addr":"172.21.15.184:6839","nonce":4203347941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.33:6839/4136670204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":4136670204},{"type":"v1","addr":"172.21.15.33:6839","nonce":4136670204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24481,24482,24469]},"id":1}]} 2024-09-12T02:11:49.706 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 25 2024-09-12T02:11:49.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:49.707+0000 7fa757e29640 1 -- 172.21.15.16:0/4169501887 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fa73c0a2b80 msgr2=0x7fa73c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:49.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:49.707+0000 7fa757e29640 1 --2- 172.21.15.16:0/4169501887 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fa73c0a2b80 0x7fa73c0a5040 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fa74c009fc0 tx=0x7fa74c005f90 comp rx=0 tx=0).stop 2024-09-12T02:11:49.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:49.707+0000 7fa757e29640 1 -- 172.21.15.16:0/4169501887 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa750102570 msgr2=0x7fa750199890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:49.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:49.707+0000 7fa757e29640 1 --2- 172.21.15.16:0/4169501887 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa750102570 0x7fa750199890 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7fa740006010 tx=0x7fa740003c10 comp rx=0 tx=0).stop 2024-09-12T02:11:49.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:49.707+0000 7fa757e29640 1 -- 172.21.15.16:0/4169501887 shutdown_connections 2024-09-12T02:11:49.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:49.707+0000 7fa757e29640 1 --2- 172.21.15.16:0/4169501887 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fa73c0a2b80 0x7fa73c0a5040 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:49.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:49.707+0000 7fa757e29640 1 --2- 172.21.15.16:0/4169501887 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa750102ef0 0x7fa7501a43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:49.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:49.707+0000 7fa757e29640 1 --2- 172.21.15.16:0/4169501887 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa750102570 0x7fa750199890 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:49.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:49.707+0000 7fa757e29640 1 --2- 172.21.15.16:0/4169501887 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa750100e20 0x7fa750199350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:49.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:49.707+0000 7fa757e29640 1 -- 172.21.15.16:0/4169501887 >> 172.21.15.16:0/4169501887 conn(0x7fa7500fc7d0 msgr2=0x7fa7501068b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:49.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:49.707+0000 7fa757e29640 1 -- 172.21.15.16:0/4169501887 shutdown_connections 2024-09-12T02:11:49.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:49.708+0000 7fa757e29640 1 -- 172.21.15.16:0/4169501887 wait complete. 2024-09-12T02:11:50.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:50.108+0000 7f1061b0a640 1 -- 172.21.15.16:0/3390720317 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f105c10c170 msgr2=0x7f105c10e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:50.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:50.108+0000 7f1061b0a640 1 --2- 172.21.15.16:0/3390720317 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f105c10c170 0x7f105c10e610 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f1054009ca0 tx=0x7f105402a950 comp rx=0 tx=0).stop 2024-09-12T02:11:50.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:50.108+0000 7f1061b0a640 1 -- 172.21.15.16:0/3390720317 shutdown_connections 2024-09-12T02:11:50.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:50.108+0000 7f1061b0a640 1 --2- 172.21.15.16:0/3390720317 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f105c10c170 0x7f105c10e610 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:50.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:50.108+0000 7f1061b0a640 1 --2- 172.21.15.16:0/3390720317 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f105c103660 0x7f105c10bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:50.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:50.108+0000 7f1061b0a640 1 --2- 172.21.15.16:0/3390720317 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f105c102d40 0x7f105c103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:50.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:50.108+0000 7f1061b0a640 1 -- 172.21.15.16:0/3390720317 >> 172.21.15.16:0/3390720317 conn(0x7f105c0fc7a0 msgr2=0x7f105c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:50.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:50.108+0000 7f1061b0a640 1 -- 172.21.15.16:0/3390720317 shutdown_connections 2024-09-12T02:11:50.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:50.108+0000 7f1061b0a640 1 -- 172.21.15.16:0/3390720317 wait complete. 2024-09-12T02:11:50.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:50.109+0000 7f1061b0a640 1 Processor -- start 2024-09-12T02:11:50.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:50.109+0000 7f1061b0a640 1 -- start start 2024-09-12T02:11:50.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:50.109+0000 7f1061b0a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f105c102d40 0x7f105c198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:50.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:50.109+0000 7f1061b0a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f105c103660 0x7f105c198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:50.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:50.109+0000 7f1061b0a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f105c10c170 0x7f105c19cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:50.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:50.109+0000 7f1061b0a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f105c111f70 con 0x7f105c103660 2024-09-12T02:11:50.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:50.109+0000 7f1061b0a640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f105c111df0 con 0x7f105c102d40 2024-09-12T02:11:50.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:50.109+0000 7f1061b0a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f105c1120f0 con 0x7f105c10c170 2024-09-12T02:11:50.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:50.109+0000 7f105affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f105c103660 0x7f105c198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:50.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:50.109+0000 7f105affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f105c103660 0x7f105c198950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36228/0 (socket says 172.21.15.16:36228) 2024-09-12T02:11:50.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:50.109+0000 7f105affd640 1 -- 172.21.15.16:0/1095957651 learned_addr learned my addr 172.21.15.16:0/1095957651 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:11:50.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:50.109+0000 7f105b7fe640 1 --2- 172.21.15.16:0/1095957651 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f105c102d40 0x7f105c198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:50.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:50.109+0000 7f105bfff640 1 --2- 172.21.15.16:0/1095957651 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f105c10c170 0x7f105c19cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:50.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:50.109+0000 7f105affd640 1 -- 172.21.15.16:0/1095957651 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f105c10c170 msgr2=0x7f105c19cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:50.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:50.109+0000 7f105affd640 1 --2- 172.21.15.16:0/1095957651 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f105c10c170 0x7f105c19cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:50.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:50.109+0000 7f105affd640 1 -- 172.21.15.16:0/1095957651 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f105c102d40 msgr2=0x7f105c198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:50.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:50.109+0000 7f105affd640 1 --2- 172.21.15.16:0/1095957651 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f105c102d40 0x7f105c198410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:50.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:50.109+0000 7f105affd640 1 -- 172.21.15.16:0/1095957651 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f105c19d3c0 con 0x7f105c103660 2024-09-12T02:11:50.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:50.109+0000 7f105bfff640 1 --2- 172.21.15.16:0/1095957651 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f105c10c170 0x7f105c19cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:11:50.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:50.110+0000 7f105affd640 1 --2- 172.21.15.16:0/1095957651 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f105c103660 0x7f105c198950 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f105000c940 tx=0x7f105000ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:50.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:50.110+0000 7f1058ff9640 1 -- 172.21.15.16:0/1095957651 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1050004220 con 0x7f105c103660 2024-09-12T02:11:50.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:50.110+0000 7f1058ff9640 1 -- 172.21.15.16:0/1095957651 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f10500043c0 con 0x7f105c103660 2024-09-12T02:11:50.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:50.110+0000 7f1058ff9640 1 -- 172.21.15.16:0/1095957651 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1050004a00 con 0x7f105c103660 2024-09-12T02:11:50.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:50.110+0000 7f1061b0a640 1 -- 172.21.15.16:0/1095957651 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f105c19d6b0 con 0x7f105c103660 2024-09-12T02:11:50.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:50.110+0000 7f1061b0a640 1 -- 172.21.15.16:0/1095957651 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f105c1a4f90 con 0x7f105c103660 2024-09-12T02:11:50.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:50.111+0000 7f1061b0a640 1 -- 172.21.15.16:0/1095957651 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f105c103120 con 0x7f105c103660 2024-09-12T02:11:50.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:50.112+0000 7f1058ff9640 1 -- 172.21.15.16:0/1095957651 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f10500079b0 con 0x7f105c103660 2024-09-12T02:11:50.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:50.113+0000 7f1058ff9640 1 --2- 172.21.15.16:0/1095957651 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f10440a2b80 0x7f10440a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:50.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:50.113+0000 7f105b7fe640 1 --2- 172.21.15.16:0/1095957651 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f10440a2b80 0x7f10440a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:50.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:50.113+0000 7f1058ff9640 1 -- 172.21.15.16:0/1095957651 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f10500cefa0 con 0x7f105c103660 2024-09-12T02:11:50.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:50.113+0000 7f105b7fe640 1 --2- 172.21.15.16:0/1095957651 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f10440a2b80 0x7f10440a5040 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f104c005fd0 tx=0x7f104c005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:50.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:50.117+0000 7f1058ff9640 1 -- 172.21.15.16:0/1095957651 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1050014030 con 0x7f105c103660 2024-09-12T02:11:50.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:50.329+0000 7f1061b0a640 1 -- 172.21.15.16:0/1095957651 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f105c1a5280 con 0x7f105c103660 2024-09-12T02:11:50.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:50.329+0000 7f1058ff9640 1 -- 172.21.15.16:0/1095957651 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11911 (secure 0 0 0) 0x7f105009bfa0 con 0x7f105c103660 2024-09-12T02:11:50.330 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T02:11:50.330 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":25,"btime":"2024-09-12T02:01:37:192021+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3279123279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3279123279},{"type":"v1","addr":"172.21.15.16:6835","nonce":3279123279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4119452382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4119452382},{"type":"v1","addr":"172.21.15.16:6837","nonce":4119452382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/3263823165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3263823165},{"type":"v1","addr":"172.21.15.16:6839","nonce":3263823165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/1510290732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":1510290732},{"type":"v1","addr":"172.21.15.16:6841","nonce":1510290732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/279674603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":279674603},{"type":"v1","addr":"172.21.15.33:6833","nonce":279674603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3555728028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3555728028},{"type":"v1","addr":"172.21.15.33:6835","nonce":3555728028}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T01:59:56.117608+0000","modified":"2024-09-12T02:01:36.294760+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24482,"mds_1":24481,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6833/1563374106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1563374106},{"type":"v1","addr":"172.21.15.184:6833","nonce":1563374106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6837/1892833868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1892833868},{"type":"v1","addr":"172.21.15.33:6837","nonce":1892833868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/1457679686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1457679686},{"type":"v1","addr":"172.21.15.184:6835","nonce":1457679686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.184:6837/1793393087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":1793393087},{"type":"v1","addr":"172.21.15.184:6837","nonce":1793393087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.184:6839/4203347941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":4203347941},{"type":"v1","addr":"172.21.15.184:6839","nonce":4203347941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.33:6839/4136670204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":4136670204},{"type":"v1","addr":"172.21.15.33:6839","nonce":4136670204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24481,24482,24469]},"id":1}]} 2024-09-12T02:11:50.330 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 25 2024-09-12T02:11:50.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:50.331+0000 7f1061b0a640 1 -- 172.21.15.16:0/1095957651 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f10440a2b80 msgr2=0x7f10440a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:50.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:50.331+0000 7f1061b0a640 1 --2- 172.21.15.16:0/1095957651 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f10440a2b80 0x7f10440a5040 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f104c005fd0 tx=0x7f104c005850 comp rx=0 tx=0).stop 2024-09-12T02:11:50.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:50.331+0000 7f1061b0a640 1 -- 172.21.15.16:0/1095957651 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f105c103660 msgr2=0x7f105c198950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:50.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:50.331+0000 7f1061b0a640 1 --2- 172.21.15.16:0/1095957651 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f105c103660 0x7f105c198950 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f105000c940 tx=0x7f105000ce00 comp rx=0 tx=0).stop 2024-09-12T02:11:50.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:50.332+0000 7f1061b0a640 1 -- 172.21.15.16:0/1095957651 shutdown_connections 2024-09-12T02:11:50.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:50.332+0000 7f1061b0a640 1 --2- 172.21.15.16:0/1095957651 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f10440a2b80 0x7f10440a5040 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:50.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:50.332+0000 7f1061b0a640 1 --2- 172.21.15.16:0/1095957651 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f105c10c170 0x7f105c19cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:50.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:50.332+0000 7f1061b0a640 1 --2- 172.21.15.16:0/1095957651 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f105c103660 0x7f105c198950 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:50.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:50.332+0000 7f1061b0a640 1 --2- 172.21.15.16:0/1095957651 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f105c102d40 0x7f105c198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:50.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:50.332+0000 7f1061b0a640 1 -- 172.21.15.16:0/1095957651 >> 172.21.15.16:0/1095957651 conn(0x7f105c0fc7a0 msgr2=0x7f105c0fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:50.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:50.332+0000 7f1061b0a640 1 -- 172.21.15.16:0/1095957651 shutdown_connections 2024-09-12T02:11:50.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:50.332+0000 7f1061b0a640 1 -- 172.21.15.16:0/1095957651 wait complete. 2024-09-12T02:11:50.582 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:11:50 smithi016 ceph-mon[30947]: pgmap v632: 97 pgs: 97 active+clean; 1.1 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.5 KiB/s wr, 8 op/s 2024-09-12T02:11:50.582 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:11:50 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/4169501887' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:11:50.582 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:11:50 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/1095957651' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:11:50.583 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24482 quiesce db / --timeout 90 --expiration 142.55 --await-for 45.0' 2024-09-12T02:11:50.583 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24482 quiesce db / --timeout 90 --expiration 142.55 --await-for 45.0 2024-09-12T02:11:50.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:11:50 smithi033 ceph-mon[35332]: pgmap v632: 97 pgs: 97 active+clean; 1.1 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.5 KiB/s wr, 8 op/s 2024-09-12T02:11:50.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:11:50 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/4169501887' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:11:50.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:11:50 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/1095957651' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:11:50.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:11:50 smithi184 ceph-mon[35593]: pgmap v632: 97 pgs: 97 active+clean; 1.1 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.5 KiB/s wr, 8 op/s 2024-09-12T02:11:50.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:11:50 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/4169501887' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:11:50.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:11:50 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/1095957651' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:11:51.231 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-12T02:11:51.274 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:11:51.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.746+0000 7ff15009f640 1 -- 172.21.15.16:0/544791575 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff148101e20 msgr2=0x7ff1481056b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:51.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.746+0000 7ff15009f640 1 --2- 172.21.15.16:0/544791575 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff148101e20 0x7ff1481056b0 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7ff138009900 tx=0x7ff13802a410 comp rx=0 tx=0).stop 2024-09-12T02:11:51.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.746+0000 7ff15009f640 1 -- 172.21.15.16:0/544791575 shutdown_connections 2024-09-12T02:11:51.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.746+0000 7ff15009f640 1 --2- 172.21.15.16:0/544791575 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff148105de0 0x7ff14810d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:51.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.746+0000 7ff15009f640 1 --2- 172.21.15.16:0/544791575 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff148101e20 0x7ff1481056b0 unknown :-1 s=CLOSED pgs=1404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:51.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.746+0000 7ff15009f640 1 --2- 172.21.15.16:0/544791575 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff148101470 0x7ff148101850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:51.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.746+0000 7ff15009f640 1 -- 172.21.15.16:0/544791575 >> 172.21.15.16:0/544791575 conn(0x7ff148077230 msgr2=0x7ff1480fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:51.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.747+0000 7ff15009f640 1 -- 172.21.15.16:0/544791575 shutdown_connections 2024-09-12T02:11:51.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.747+0000 7ff15009f640 1 -- 172.21.15.16:0/544791575 wait complete. 2024-09-12T02:11:51.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.747+0000 7ff15009f640 1 Processor -- start 2024-09-12T02:11:51.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.747+0000 7ff15009f640 1 -- start start 2024-09-12T02:11:51.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.748+0000 7ff15009f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff148101470 0x7ff1481992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:51.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.748+0000 7ff15009f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff148101e20 0x7ff1481997f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:51.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.748+0000 7ff15009f640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff148105de0 0x7ff1481a4330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:51.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.748+0000 7ff15009f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff148110e40 con 0x7ff148101e20 2024-09-12T02:11:51.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.748+0000 7ff15009f640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff148110cc0 con 0x7ff148101470 2024-09-12T02:11:51.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.748+0000 7ff15009f640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff148110fc0 con 0x7ff148105de0 2024-09-12T02:11:51.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.748+0000 7ff14d63a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff148101e20 0x7ff1481997f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:51.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.748+0000 7ff14d63a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff148101e20 0x7ff1481997f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53900/0 (socket says 172.21.15.16:53900) 2024-09-12T02:11:51.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.748+0000 7ff14d63a640 1 -- 172.21.15.16:0/3724048948 learned_addr learned my addr 172.21.15.16:0/3724048948 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:11:51.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.748+0000 7ff14de3b640 1 --2- 172.21.15.16:0/3724048948 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff148101470 0x7ff1481992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:51.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.748+0000 7ff14e63c640 1 --2- 172.21.15.16:0/3724048948 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff148105de0 0x7ff1481a4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:51.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.748+0000 7ff14d63a640 1 -- 172.21.15.16:0/3724048948 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff148105de0 msgr2=0x7ff1481a4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:51.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.748+0000 7ff14d63a640 1 --2- 172.21.15.16:0/3724048948 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff148105de0 0x7ff1481a4330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:51.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.748+0000 7ff14d63a640 1 -- 172.21.15.16:0/3724048948 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff148101470 msgr2=0x7ff1481992b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:51.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.748+0000 7ff14d63a640 1 --2- 172.21.15.16:0/3724048948 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff148101470 0x7ff1481992b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:51.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.748+0000 7ff14d63a640 1 -- 172.21.15.16:0/3724048948 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1481a4870 con 0x7ff148101e20 2024-09-12T02:11:51.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.748+0000 7ff14de3b640 1 --2- 172.21.15.16:0/3724048948 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff148101470 0x7ff1481992b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:11:51.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.749+0000 7ff14d63a640 1 --2- 172.21.15.16:0/3724048948 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff148101e20 0x7ff1481997f0 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7ff138006010 tx=0x7ff1380042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:51.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.749+0000 7ff13effd640 1 -- 172.21.15.16:0/3724048948 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff13803f030 con 0x7ff148101e20 2024-09-12T02:11:51.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.749+0000 7ff13effd640 1 -- 172.21.15.16:0/3724048948 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff138003ce0 con 0x7ff148101e20 2024-09-12T02:11:51.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.749+0000 7ff13effd640 1 -- 172.21.15.16:0/3724048948 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff138035070 con 0x7ff148101e20 2024-09-12T02:11:51.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.749+0000 7ff15009f640 1 -- 172.21.15.16:0/3724048948 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1481a4a40 con 0x7ff148101e20 2024-09-12T02:11:51.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.749+0000 7ff15009f640 1 -- 172.21.15.16:0/3724048948 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1481a4ef0 con 0x7ff148101e20 2024-09-12T02:11:51.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.751+0000 7ff152478740 1 Processor -- start 2024-09-12T02:11:51.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.751+0000 7ff13effd640 1 -- 172.21.15.16:0/3724048948 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff138042030 con 0x7ff148101e20 2024-09-12T02:11:51.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.752+0000 7ff152478740 1 -- start start 2024-09-12T02:11:51.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.752+0000 7ff152478740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dd924c9750 0x55dd924c9b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:51.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.752+0000 7ff152478740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dd924ca070 0x55dd924d8900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:51.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.752+0000 7ff152478740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dd924d8e40 0x55dd924db230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:51.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.752+0000 7ff152478740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55dd9242e160 con 0x55dd924c9750 2024-09-12T02:11:51.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.752+0000 7ff152478740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55dd923d5720 con 0x55dd924d8e40 2024-09-12T02:11:51.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.752+0000 7ff152478740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55dd9242a120 con 0x55dd924ca070 2024-09-12T02:11:51.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.752+0000 7ff14e63c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dd924c9750 0x55dd924c9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:51.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.752+0000 7ff14e63c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dd924c9750 0x55dd924c9b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53908/0 (socket says 172.21.15.16:53908) 2024-09-12T02:11:51.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.752+0000 7ff14e63c640 1 -- 172.21.15.16:0/1548095995 learned_addr learned my addr 172.21.15.16:0/1548095995 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:11:51.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.752+0000 7ff14de3b640 1 --2- 172.21.15.16:0/1548095995 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dd924ca070 0x55dd924d8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:51.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.752+0000 7ff14d63a640 1 --2- 172.21.15.16:0/1548095995 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dd924d8e40 0x55dd924db230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:51.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.753+0000 7ff14e63c640 1 -- 172.21.15.16:0/1548095995 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dd924ca070 msgr2=0x55dd924d8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:51.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.753+0000 7ff14e63c640 1 --2- 172.21.15.16:0/1548095995 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dd924ca070 0x55dd924d8900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:51.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.753+0000 7ff14e63c640 1 -- 172.21.15.16:0/1548095995 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dd924d8e40 msgr2=0x55dd924db230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:51.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.753+0000 7ff14e63c640 1 --2- 172.21.15.16:0/1548095995 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dd924d8e40 0x55dd924db230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:51.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.753+0000 7ff14e63c640 1 -- 172.21.15.16:0/1548095995 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd9219fba0 con 0x55dd924c9750 2024-09-12T02:11:51.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.753+0000 7ff14d63a640 1 --2- 172.21.15.16:0/1548095995 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dd924d8e40 0x55dd924db230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:11:51.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.753+0000 7ff14e63c640 1 --2- 172.21.15.16:0/1548095995 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dd924c9750 0x55dd924c9b30 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7ff14000ae10 tx=0x7ff14000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d12f253717af4df5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:51.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.753+0000 7ff13effd640 1 --2- 172.21.15.16:0/3724048948 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7ff1340a2c00 0x7ff1340a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:51.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.753+0000 7ff14de3b640 1 --2- 172.21.15.16:0/3724048948 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7ff1340a2c00 0x7ff1340a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:51.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.753+0000 7ff13effd640 1 -- 172.21.15.16:0/3724048948 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7ff1380ef450 con 0x7ff148101e20 2024-09-12T02:11:51.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.753+0000 7ff133fff640 1 -- 172.21.15.16:0/1548095995 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff140003b70 con 0x55dd924c9750 2024-09-12T02:11:51.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.753+0000 7ff133fff640 1 -- 172.21.15.16:0/1548095995 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff140003d10 con 0x55dd924c9750 2024-09-12T02:11:51.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.753+0000 7ff133fff640 1 -- 172.21.15.16:0/1548095995 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff140010b40 con 0x55dd924c9750 2024-09-12T02:11:51.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.754+0000 7ff152478740 1 -- 172.21.15.16:0/1548095995 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dd924c9750 msgr2=0x55dd924c9b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:51.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.754+0000 7ff14de3b640 1 --2- 172.21.15.16:0/3724048948 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7ff1340a2c00 0x7ff1340a50c0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x55dd924cc440 tx=0x7ff144005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:51.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.754+0000 7ff152478740 1 --2- 172.21.15.16:0/1548095995 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dd924c9750 0x55dd924c9b30 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7ff14000ae10 tx=0x7ff14000f790 comp rx=0 tx=0).stop 2024-09-12T02:11:51.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.754+0000 7ff152478740 1 -- 172.21.15.16:0/1548095995 shutdown_connections 2024-09-12T02:11:51.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.754+0000 7ff152478740 1 --2- 172.21.15.16:0/1548095995 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dd924d8e40 0x55dd924db230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:51.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.754+0000 7ff152478740 1 --2- 172.21.15.16:0/1548095995 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dd924ca070 0x55dd924d8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:51.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.754+0000 7ff152478740 1 --2- 172.21.15.16:0/1548095995 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dd924c9750 0x55dd924c9b30 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:51.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.754+0000 7ff152478740 1 -- 172.21.15.16:0/1548095995 >> 172.21.15.16:0/1548095995 conn(0x55dd924c5de0 msgr2=0x55dd924c61c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:51.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.754+0000 7ff152478740 1 -- 172.21.15.16:0/1548095995 shutdown_connections 2024-09-12T02:11:51.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.754+0000 7ff152478740 1 -- 172.21.15.16:0/1548095995 wait complete. 2024-09-12T02:11:51.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.755+0000 7ff152478740 1 -- start start 2024-09-12T02:11:51.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.755+0000 7ff152478740 1 Processor -- start 2024-09-12T02:11:51.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.755+0000 7ff152478740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dd924c9750 0x55dd924d5d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:51.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.755+0000 7ff152478740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dd924ca070 0x55dd924d62b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:51.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.755+0000 7ff152478740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dd924d8e40 0x55dd924d67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:51.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.755+0000 7ff152478740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55dd924dba70 con 0x55dd924c9750 2024-09-12T02:11:51.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.755+0000 7ff152478740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55dd923b6900 con 0x55dd924d8e40 2024-09-12T02:11:51.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.755+0000 7ff152478740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55dd924dbbf0 con 0x55dd924ca070 2024-09-12T02:11:51.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.755+0000 7ff152478740 10 client.0 _ll_register_callbacks cb 0x55dd923b4780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:11:51.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.755+0000 7ff14de3b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dd924c9750 0x55dd924d5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:51.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.755+0000 7ff14de3b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dd924c9750 0x55dd924d5d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53910/0 (socket says 172.21.15.16:53910) 2024-09-12T02:11:51.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.756+0000 7ff14de3b640 1 -- 172.21.15.16:0/3868859930 learned_addr learned my addr 172.21.15.16:0/3868859930 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:11:51.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.756+0000 7ff14e63c640 1 --2- 172.21.15.16:0/3868859930 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dd924d8e40 0x55dd924d67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:51.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.756+0000 7ff14d63a640 1 --2- 172.21.15.16:0/3868859930 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dd924ca070 0x55dd924d62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:51.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.756+0000 7ff14de3b640 1 -- 172.21.15.16:0/3868859930 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dd924ca070 msgr2=0x55dd924d62b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:51.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.756+0000 7ff14de3b640 1 --2- 172.21.15.16:0/3868859930 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dd924ca070 0x55dd924d62b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:51.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.756+0000 7ff14de3b640 1 -- 172.21.15.16:0/3868859930 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dd924d8e40 msgr2=0x55dd924d67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:51.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.756+0000 7ff14de3b640 1 --2- 172.21.15.16:0/3868859930 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dd924d8e40 0x55dd924d67f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:51.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.756+0000 7ff14de3b640 1 -- 172.21.15.16:0/3868859930 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dd923f6c50 con 0x55dd924c9750 2024-09-12T02:11:51.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.756+0000 7ff14de3b640 1 --2- 172.21.15.16:0/3868859930 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dd924c9750 0x55dd924d5d70 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7ff14400fb20 tx=0x7ff144004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8706aa99713c32eb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:51.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.756+0000 7ff1317fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:11:51.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.756+0000 7ff1317fa640 1 -- 172.21.15.16:0/3868859930 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff144004020 con 0x55dd924c9750 2024-09-12T02:11:51.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.756+0000 7ff152478740 1 -- 172.21.15.16:0/3868859930 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff140003d10 con 0x55dd924c9750 2024-09-12T02:11:51.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.756+0000 7ff1317fa640 1 -- 172.21.15.16:0/3868859930 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7ff144018030 con 0x55dd924c9750 2024-09-12T02:11:51.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.757+0000 7ff1317fa640 1 -- 172.21.15.16:0/3868859930 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff1440041c0 con 0x55dd924c9750 2024-09-12T02:11:51.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.757+0000 7ff152478740 10 client.18513 fetch_fsmap learned FSMap version 25 2024-09-12T02:11:51.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.757+0000 7ff152478740 1 -- 172.21.15.16:0/3868859930 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55dd924e7840 con 0x55dd924c9750 2024-09-12T02:11:51.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.757+0000 7ff1317fa640 1 -- 172.21.15.16:0/3868859930 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff140003d10 con 0x55dd924c9750 2024-09-12T02:11:51.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.757+0000 7ff1317fa640 1 -- 172.21.15.16:0/3868859930 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7ff1440037f0 con 0x55dd924c9750 2024-09-12T02:11:51.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.757+0000 7ff152478740 10 client.18513 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:11:51.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.757+0000 7ff152478740 10 client.18513 resolve_mds: validated gid 24482 aka daemon mds.b 2024-09-12T02:11:51.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.757+0000 7ff152478740 20 client.18513 populate_metadata read hostname 'smithi016' 2024-09-12T02:11:51.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.757+0000 7ff152478740 1 --2- 172.21.15.16:0/3868859930 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55dd924e8d00 0x55dd924eb110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:51.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.757+0000 7ff152478740 4 client.18513 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:11:51.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.757+0000 7ff152478740 1 -- 172.21.15.16:0/3868859930 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dd9243b890 con 0x55dd924e8d00 2024-09-12T02:11:51.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.758+0000 7ff14d63a640 1 --2- 172.21.15.16:0/3868859930 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55dd924e8d00 0x55dd924eb110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:51.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.758+0000 7ff1317fa640 10 client.18513 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:11:51.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.758+0000 7ff14d63a640 1 --2- 172.21.15.16:0/3868859930 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55dd924e8d00 0x55dd924eb110 crc :-1 s=READY pgs=294 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c32dcc22dadc37d server_cookie=9d3461683eff0e81 in_seq=0 out_seq=0 2024-09-12T02:11:51.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.759+0000 7ff1317fa640 1 -- 172.21.15.16:0/3868859930 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55dd9243b890 con 0x55dd924e8d00 2024-09-12T02:11:51.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.759+0000 7ff1317fa640 10 client.18513 handle_command_reply: tid=0 2024-09-12T02:11:51.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.759+0000 7ff152478740 1 client.18513 shutdown 2024-09-12T02:11:51.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.759+0000 7ff152478740 1 -- 172.21.15.16:0/3868859930 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dd924c9750 msgr2=0x55dd924d5d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:51.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.759+0000 7ff152478740 1 --2- 172.21.15.16:0/3868859930 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dd924c9750 0x55dd924d5d70 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7ff14400fb20 tx=0x7ff144004730 comp rx=0 tx=0).stop 2024-09-12T02:11:51.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.759+0000 7ff152478740 1 -- 172.21.15.16:0/3868859930 shutdown_connections 2024-09-12T02:11:51.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.759+0000 7ff152478740 1 --2- 172.21.15.16:0/3868859930 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55dd924e8d00 0x55dd924eb110 crc :-1 s=READY pgs=294 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:51.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.759+0000 7ff152478740 1 --2- 172.21.15.16:0/3868859930 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dd924d8e40 0x55dd924d67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:51.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.759+0000 7ff152478740 1 --2- 172.21.15.16:0/3868859930 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dd924ca070 0x55dd924d62b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:51.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.759+0000 7ff152478740 1 --2- 172.21.15.16:0/3868859930 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dd924c9750 0x55dd924d5d70 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:51.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.759+0000 7ff152478740 1 -- 172.21.15.16:0/3868859930 >> 172.21.15.16:0/3868859930 conn(0x55dd924c5de0 msgr2=0x55dd924cbb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:51.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.760+0000 7ff1317fa640 0 client.18513 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:11:51.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.760+0000 7ff152478740 1 -- 172.21.15.16:0/3868859930 shutdown_connections 2024-09-12T02:11:51.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.760+0000 7ff152478740 1 -- 172.21.15.16:0/3868859930 wait complete. 2024-09-12T02:11:51.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.760+0000 7ff152478740 20 client.18513 trim_cache size 0 max 16384 2024-09-12T02:11:51.781 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:11:51.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.783+0000 7ff152478740 1 Processor -- start 2024-09-12T02:11:51.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.783+0000 7ff152478740 1 -- start start 2024-09-12T02:11:51.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.783+0000 7ff152478740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dd924c9750 0x55dd924d12b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:51.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.783+0000 7ff152478740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dd924ca070 0x55dd924cba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:51.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.783+0000 7ff152478740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dd924d8e40 0x55dd924f09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:51.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.783+0000 7ff152478740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff1481adb30 con 0x55dd924d8e40 2024-09-12T02:11:51.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.783+0000 7ff152478740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff1481ab400 con 0x55dd924c9750 2024-09-12T02:11:51.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.783+0000 7ff152478740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff1481a9270 con 0x55dd924ca070 2024-09-12T02:11:51.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.783+0000 7ff14e63c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dd924d8e40 0x55dd924f09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:51.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.784+0000 7ff14e63c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dd924d8e40 0x55dd924f09e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53920/0 (socket says 172.21.15.16:53920) 2024-09-12T02:11:51.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.784+0000 7ff14e63c640 1 -- 172.21.15.16:0/1911821208 learned_addr learned my addr 172.21.15.16:0/1911821208 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:11:51.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.784+0000 7ff14de3b640 1 --2- 172.21.15.16:0/1911821208 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dd924c9750 0x55dd924d12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:51.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.784+0000 7ff14e63c640 1 -- 172.21.15.16:0/1911821208 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dd924ca070 msgr2=0x55dd924cba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:51.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.784+0000 7ff14e63c640 1 --2- 172.21.15.16:0/1911821208 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dd924ca070 0x55dd924cba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:51.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.784+0000 7ff14e63c640 1 -- 172.21.15.16:0/1911821208 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dd924c9750 msgr2=0x55dd924d12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:51.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.784+0000 7ff14e63c640 1 --2- 172.21.15.16:0/1911821208 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dd924c9750 0x55dd924d12b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:51.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.784+0000 7ff14e63c640 1 -- 172.21.15.16:0/1911821208 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd924d4d20 con 0x55dd924d8e40 2024-09-12T02:11:51.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.784+0000 7ff14de3b640 1 --2- 172.21.15.16:0/1911821208 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dd924c9750 0x55dd924d12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:11:51.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.784+0000 7ff14e63c640 1 --2- 172.21.15.16:0/1911821208 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dd924d8e40 0x55dd924f09e0 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7ff140004990 tx=0x7ff14000a870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efbfb2bee5e53045 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:51.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.784+0000 7ff1337fe640 1 -- 172.21.15.16:0/1911821208 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff140004a00 con 0x55dd924d8e40 2024-09-12T02:11:51.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.784+0000 7ff1337fe640 1 -- 172.21.15.16:0/1911821208 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff140004ba0 con 0x55dd924d8e40 2024-09-12T02:11:51.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.784+0000 7ff1337fe640 1 -- 172.21.15.16:0/1911821208 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1400186a0 con 0x55dd924d8e40 2024-09-12T02:11:51.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.785+0000 7ff152478740 1 -- 172.21.15.16:0/1911821208 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dd924d8e40 msgr2=0x55dd924f09e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:51.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.785+0000 7ff152478740 1 --2- 172.21.15.16:0/1911821208 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dd924d8e40 0x55dd924f09e0 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7ff140004990 tx=0x7ff14000a870 comp rx=0 tx=0).stop 2024-09-12T02:11:51.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.785+0000 7ff152478740 1 -- 172.21.15.16:0/1911821208 shutdown_connections 2024-09-12T02:11:51.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.785+0000 7ff152478740 1 --2- 172.21.15.16:0/1911821208 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dd924d8e40 0x55dd924f09e0 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:51.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.785+0000 7ff152478740 1 --2- 172.21.15.16:0/1911821208 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dd924ca070 0x55dd924cba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:51.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.785+0000 7ff152478740 1 --2- 172.21.15.16:0/1911821208 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dd924c9750 0x55dd924d12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:51.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.785+0000 7ff152478740 1 -- 172.21.15.16:0/1911821208 >> 172.21.15.16:0/1911821208 conn(0x55dd924c5de0 msgr2=0x55dd924cb320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:51.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.785+0000 7ff152478740 1 -- 172.21.15.16:0/1911821208 shutdown_connections 2024-09-12T02:11:51.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.785+0000 7ff152478740 1 -- 172.21.15.16:0/1911821208 wait complete. 2024-09-12T02:11:51.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.786+0000 7ff152478740 1 -- start start 2024-09-12T02:11:51.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.786+0000 7ff152478740 1 Processor -- start 2024-09-12T02:11:51.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.786+0000 7ff152478740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dd924c9750 0x55dd924ccee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:51.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.786+0000 7ff152478740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dd924ca070 0x55dd924cd420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:51.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.786+0000 7ff152478740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dd924d8e40 0x55dd924dfea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:51.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.786+0000 7ff152478740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff1481a98c0 con 0x55dd924d8e40 2024-09-12T02:11:51.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.786+0000 7ff152478740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff1481aad30 con 0x55dd924c9750 2024-09-12T02:11:51.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.786+0000 7ff152478740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff1481a9f90 con 0x55dd924ca070 2024-09-12T02:11:51.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.786+0000 7ff152478740 10 client.0 _ll_register_callbacks cb 0x7ff144009ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:11:51.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.787+0000 7ff14e63c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dd924d8e40 0x55dd924dfea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:51.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.787+0000 7ff14e63c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dd924d8e40 0x55dd924dfea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53922/0 (socket says 172.21.15.16:53922) 2024-09-12T02:11:51.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.787+0000 7ff14e63c640 1 -- 172.21.15.16:0/3303858695 learned_addr learned my addr 172.21.15.16:0/3303858695 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:11:51.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.787+0000 7ff14de3b640 1 --2- 172.21.15.16:0/3303858695 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dd924c9750 0x55dd924ccee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:51.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.787+0000 7ff14d63a640 1 --2- 172.21.15.16:0/3303858695 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dd924ca070 0x55dd924cd420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:51.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.787+0000 7ff14e63c640 1 -- 172.21.15.16:0/3303858695 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dd924ca070 msgr2=0x55dd924cd420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:51.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.787+0000 7ff14e63c640 1 --2- 172.21.15.16:0/3303858695 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dd924ca070 0x55dd924cd420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:51.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.787+0000 7ff14e63c640 1 -- 172.21.15.16:0/3303858695 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dd924c9750 msgr2=0x55dd924ccee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:51.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.787+0000 7ff14e63c640 1 --2- 172.21.15.16:0/3303858695 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dd924c9750 0x55dd924ccee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:51.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.787+0000 7ff14e63c640 1 -- 172.21.15.16:0/3303858695 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dd924e03e0 con 0x55dd924d8e40 2024-09-12T02:11:51.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.787+0000 7ff14e63c640 1 --2- 172.21.15.16:0/3303858695 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dd924d8e40 0x55dd924dfea0 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7ff14000aba0 tx=0x7ff140018670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d04933fdaad01fa server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:51.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.787+0000 7ff1317fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:11:51.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.787+0000 7ff1317fa640 1 -- 172.21.15.16:0/3303858695 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff140004a00 con 0x55dd924d8e40 2024-09-12T02:11:51.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.787+0000 7ff1317fa640 1 -- 172.21.15.16:0/3303858695 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7ff140018bf0 con 0x55dd924d8e40 2024-09-12T02:11:51.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.788+0000 7ff152478740 1 -- 172.21.15.16:0/3303858695 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff140004ba0 con 0x55dd924d8e40 2024-09-12T02:11:51.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.788+0000 7ff1317fa640 1 -- 172.21.15.16:0/3303858695 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff140010b40 con 0x55dd924d8e40 2024-09-12T02:11:51.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.788+0000 7ff152478740 10 client.18519 fetch_fsmap learned FSMap version 25 2024-09-12T02:11:51.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.788+0000 7ff152478740 1 -- 172.21.15.16:0/3303858695 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55dd924e16c0 con 0x55dd924d8e40 2024-09-12T02:11:51.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.788+0000 7ff1317fa640 1 -- 172.21.15.16:0/3303858695 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff140018df0 con 0x55dd924d8e40 2024-09-12T02:11:51.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.788+0000 7ff1317fa640 1 -- 172.21.15.16:0/3303858695 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7ff140023550 con 0x55dd924d8e40 2024-09-12T02:11:51.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.788+0000 7ff152478740 10 client.18519 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:11:51.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.788+0000 7ff152478740 10 client.18519 resolve_mds: validated gid 24482 aka daemon mds.b 2024-09-12T02:11:51.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.788+0000 7ff152478740 20 client.18519 populate_metadata read hostname 'smithi016' 2024-09-12T02:11:51.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.788+0000 7ff152478740 1 --2- 172.21.15.16:0/3303858695 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55dd925581a0 0x55dd9255a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:51.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.788+0000 7ff152478740 4 client.18519 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.55, "await_for": 45.0}] 2024-09-12T02:11:51.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.788+0000 7ff152478740 1 -- 172.21.15.16:0/3303858695 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.55, "await_for": 45.0}) -- 0x7ff140004ba0 con 0x55dd925581a0 2024-09-12T02:11:51.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.789+0000 7ff14de3b640 1 --2- 172.21.15.16:0/3303858695 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55dd925581a0 0x55dd9255a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:51.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.789+0000 7ff1317fa640 10 client.18519 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:11:51.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.789+0000 7ff14de3b640 1 --2- 172.21.15.16:0/3303858695 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55dd925581a0 0x55dd9255a590 crc :-1 s=READY pgs=295 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ce10b8dc1f7c2cb server_cookie=8dd802c589de3aa1 in_seq=0 out_seq=0 2024-09-12T02:11:51.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.804+0000 7ff1317fa640 1 -- 172.21.15.16:0/3303858695 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x7ff140004ba0 con 0x55dd925581a0 2024-09-12T02:11:51.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.804+0000 7ff1317fa640 10 client.18519 handle_command_reply: tid=0 2024-09-12T02:11:51.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.804+0000 7ff152478740 1 client.18519 shutdown 2024-09-12T02:11:51.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.804+0000 7ff152478740 1 -- 172.21.15.16:0/3303858695 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dd924d8e40 msgr2=0x55dd924dfea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:51.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.804+0000 7ff152478740 1 --2- 172.21.15.16:0/3303858695 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dd924d8e40 0x55dd924dfea0 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7ff14000aba0 tx=0x7ff140018670 comp rx=0 tx=0).stop 2024-09-12T02:11:51.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.805+0000 7ff152478740 1 -- 172.21.15.16:0/3303858695 shutdown_connections 2024-09-12T02:11:51.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.805+0000 7ff152478740 1 --2- 172.21.15.16:0/3303858695 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55dd925581a0 0x55dd9255a590 crc :-1 s=READY pgs=295 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:51.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.805+0000 7ff152478740 1 --2- 172.21.15.16:0/3303858695 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dd924d8e40 0x55dd924dfea0 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:51.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.805+0000 7ff152478740 1 --2- 172.21.15.16:0/3303858695 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dd924ca070 0x55dd924cd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:51.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.805+0000 7ff152478740 1 --2- 172.21.15.16:0/3303858695 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dd924c9750 0x55dd924ccee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:51.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.805+0000 7ff152478740 1 -- 172.21.15.16:0/3303858695 >> 172.21.15.16:0/3303858695 conn(0x55dd924c5de0 msgr2=0x55dd924cbb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:51.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.805+0000 7ff1317fa640 0 client.18519 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:11:51.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.805+0000 7ff152478740 1 -- 172.21.15.16:0/3303858695 shutdown_connections 2024-09-12T02:11:51.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.805+0000 7ff152478740 1 -- 172.21.15.16:0/3303858695 wait complete. 2024-09-12T02:11:51.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.805+0000 7ff152478740 20 client.18519 trim_cache size 0 max 16384 2024-09-12T02:11:51.806 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T02:11:51.807 INFO:teuthology.orchestra.run.smithi016.stdout: "epoch": 25, 2024-09-12T02:11:51.807 INFO:teuthology.orchestra.run.smithi016.stdout: "leader": 24482, 2024-09-12T02:11:51.807 INFO:teuthology.orchestra.run.smithi016.stdout: "set_version": 42, 2024-09-12T02:11:51.807 INFO:teuthology.orchestra.run.smithi016.stdout: "sets": { 2024-09-12T02:11:51.807 INFO:teuthology.orchestra.run.smithi016.stdout: "44342e0a": { 2024-09-12T02:11:51.807 INFO:teuthology.orchestra.run.smithi016.stdout: "version": 42, 2024-09-12T02:11:51.807 INFO:teuthology.orchestra.run.smithi016.stdout: "age_ref": 0.0, 2024-09-12T02:11:51.807 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-12T02:11:51.807 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "QUIESCED", 2024-09-12T02:11:51.807 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-12T02:11:51.807 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:11:51.808 INFO:teuthology.orchestra.run.smithi016.stdout: "timeout": 90.0, 2024-09-12T02:11:51.808 INFO:teuthology.orchestra.run.smithi016.stdout: "expiration": 142.6, 2024-09-12T02:11:51.808 INFO:teuthology.orchestra.run.smithi016.stdout: "members": { 2024-09-12T02:11:51.808 INFO:teuthology.orchestra.run.smithi016.stdout: "file:/": { 2024-09-12T02:11:51.808 INFO:teuthology.orchestra.run.smithi016.stdout: "excluded": false, 2024-09-12T02:11:51.808 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-12T02:11:51.808 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "QUIESCED", 2024-09-12T02:11:51.808 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-12T02:11:51.808 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:11:51.808 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:11:51.808 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:11:51.808 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:11:51.808 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:11:51.808 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T02:11:51.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.806+0000 7ff15009f640 1 -- 172.21.15.16:0/3724048948 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7ff1340a2c00 msgr2=0x7ff1340a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:51.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.806+0000 7ff15009f640 1 --2- 172.21.15.16:0/3724048948 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7ff1340a2c00 0x7ff1340a50c0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x55dd924cc440 tx=0x7ff144005f90 comp rx=0 tx=0).stop 2024-09-12T02:11:51.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.806+0000 7ff15009f640 1 -- 172.21.15.16:0/3724048948 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff148101e20 msgr2=0x7ff1481997f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:51.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.806+0000 7ff15009f640 1 --2- 172.21.15.16:0/3724048948 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff148101e20 0x7ff1481997f0 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7ff138006010 tx=0x7ff1380042f0 comp rx=0 tx=0).stop 2024-09-12T02:11:51.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.806+0000 7ff15009f640 1 -- 172.21.15.16:0/3724048948 shutdown_connections 2024-09-12T02:11:51.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.806+0000 7ff15009f640 1 --2- 172.21.15.16:0/3724048948 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7ff1340a2c00 0x7ff1340a50c0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:51.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.806+0000 7ff15009f640 1 --2- 172.21.15.16:0/3724048948 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff148105de0 0x7ff1481a4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:51.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.806+0000 7ff15009f640 1 --2- 172.21.15.16:0/3724048948 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff148101e20 0x7ff1481997f0 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:51.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.806+0000 7ff15009f640 1 --2- 172.21.15.16:0/3724048948 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff148101470 0x7ff1481992b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:51.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.806+0000 7ff15009f640 1 -- 172.21.15.16:0/3724048948 >> 172.21.15.16:0/3724048948 conn(0x7ff148077230 msgr2=0x7ff148104060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:51.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.806+0000 7ff15009f640 1 -- 172.21.15.16:0/3724048948 shutdown_connections 2024-09-12T02:11:51.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:51.806+0000 7ff15009f640 1 -- 172.21.15.16:0/3724048948 wait complete. 2024-09-12T02:11:52.455 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '44342e0a', quiesce took 4.8 seconds. Will release after: 17.75 2024-09-12T02:11:52.659 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:11:52 smithi016 ceph-mon[30947]: pgmap v633: 97 pgs: 97 active+clean; 1.1 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.9 KiB/s wr, 11 op/s 2024-09-12T02:11:52.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:11:52 smithi033 ceph-mon[35332]: pgmap v633: 97 pgs: 97 active+clean; 1.1 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.9 KiB/s wr, 11 op/s 2024-09-12T02:11:52.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:11:52 smithi184 ceph-mon[35593]: pgmap v633: 97 pgs: 97 active+clean; 1.1 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.9 KiB/s wr, 11 op/s 2024-09-12T02:11:52.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.864+0000 7f8266c7d640 1 -- 172.21.15.16:0/3934108079 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8260102790 msgr2=0x7f82601067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:52.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.864+0000 7f8266c7d640 1 --2- 172.21.15.16:0/3934108079 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8260102790 0x7f82601067e0 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f8254009900 tx=0x7f825402a3f0 comp rx=0 tx=0).stop 2024-09-12T02:11:52.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.864+0000 7f8266c7d640 1 -- 172.21.15.16:0/3934108079 shutdown_connections 2024-09-12T02:11:52.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.864+0000 7f8266c7d640 1 --2- 172.21.15.16:0/3934108079 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8260106f10 0x7f826010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:52.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.864+0000 7f8266c7d640 1 --2- 172.21.15.16:0/3934108079 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8260102790 0x7f82601067e0 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:52.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.864+0000 7f8266c7d640 1 --2- 172.21.15.16:0/3934108079 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8260101de0 0x7f82601021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:52.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.864+0000 7f8266c7d640 1 -- 172.21.15.16:0/3934108079 >> 172.21.15.16:0/3934108079 conn(0x7f82600fd8c0 msgr2=0x7f82600ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:52.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.864+0000 7f8266c7d640 1 -- 172.21.15.16:0/3934108079 shutdown_connections 2024-09-12T02:11:52.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.864+0000 7f8266c7d640 1 -- 172.21.15.16:0/3934108079 wait complete. 2024-09-12T02:11:52.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.864+0000 7f8266c7d640 1 Processor -- start 2024-09-12T02:11:52.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.864+0000 7f8266c7d640 1 -- start start 2024-09-12T02:11:52.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.865+0000 7f8266c7d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8260101de0 0x7f82600744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:52.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.865+0000 7f8266c7d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8260102790 0x7f8260074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:52.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.865+0000 7f8266c7d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8260106f10 0x7f8260074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:52.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.865+0000 7f8266c7d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8260111fc0 con 0x7f8260106f10 2024-09-12T02:11:52.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.865+0000 7f8266c7d640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8260111e40 con 0x7f8260101de0 2024-09-12T02:11:52.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.865+0000 7f8266c7d640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8260112140 con 0x7f8260102790 2024-09-12T02:11:52.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.865+0000 7f826521a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8260106f10 0x7f8260074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:52.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.865+0000 7f826521a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8260106f10 0x7f8260074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53942/0 (socket says 172.21.15.16:53942) 2024-09-12T02:11:52.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.865+0000 7f826521a640 1 -- 172.21.15.16:0/582439859 learned_addr learned my addr 172.21.15.16:0/582439859 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:11:52.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.865+0000 7f8264a19640 1 --2- 172.21.15.16:0/582439859 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8260101de0 0x7f82600744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:52.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.865+0000 7f825ffff640 1 --2- 172.21.15.16:0/582439859 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8260102790 0x7f8260074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:52.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.865+0000 7f826521a640 1 -- 172.21.15.16:0/582439859 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8260102790 msgr2=0x7f8260074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:52.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.865+0000 7f826521a640 1 --2- 172.21.15.16:0/582439859 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8260102790 0x7f8260074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:52.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.865+0000 7f826521a640 1 -- 172.21.15.16:0/582439859 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8260101de0 msgr2=0x7f82600744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:52.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.865+0000 7f826521a640 1 --2- 172.21.15.16:0/582439859 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8260101de0 0x7f82600744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:52.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.865+0000 7f826521a640 1 -- 172.21.15.16:0/582439859 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82601a68e0 con 0x7f8260106f10 2024-09-12T02:11:52.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.865+0000 7f825ffff640 1 --2- 172.21.15.16:0/582439859 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8260102790 0x7f8260074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:11:52.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.865+0000 7f826521a640 1 --2- 172.21.15.16:0/582439859 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8260106f10 0x7f8260074f70 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f825800eaa0 tx=0x7f825800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:52.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.865+0000 7f825dffb640 1 -- 172.21.15.16:0/582439859 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f825800cdb0 con 0x7f8260106f10 2024-09-12T02:11:52.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.865+0000 7f8266c7d640 1 -- 172.21.15.16:0/582439859 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82601a6bd0 con 0x7f8260106f10 2024-09-12T02:11:52.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.865+0000 7f825dffb640 1 -- 172.21.15.16:0/582439859 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8258005020 con 0x7f8260106f10 2024-09-12T02:11:52.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.865+0000 7f825dffb640 1 -- 172.21.15.16:0/582439859 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82580116b0 con 0x7f8260106f10 2024-09-12T02:11:52.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.866+0000 7f8266c7d640 1 -- 172.21.15.16:0/582439859 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82601a7090 con 0x7f8260106f10 2024-09-12T02:11:52.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.868+0000 7f825dffb640 1 -- 172.21.15.16:0/582439859 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f8258005560 con 0x7f8260106f10 2024-09-12T02:11:52.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.868+0000 7f8269056740 1 Processor -- start 2024-09-12T02:11:52.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.868+0000 7f8269056740 1 -- start start 2024-09-12T02:11:52.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.869+0000 7f8269056740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564a86b09b40 0x564a86b08090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:52.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.869+0000 7f8269056740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564a86b085d0 0x564a86b08a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:52.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.869+0000 7f8269056740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564a86b13370 0x564a86b15760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:52.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.869+0000 7f825dffb640 1 --2- 172.21.15.16:0/582439859 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f82480a2b80 0x7f82480a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:52.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.869+0000 7f8269056740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x564a86a6e2f0 con 0x564a86b09b40 2024-09-12T02:11:52.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.869+0000 7f8269056740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x564a86a15cf0 con 0x564a86b085d0 2024-09-12T02:11:52.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.869+0000 7f8269056740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x564a86a6a2b0 con 0x564a86b13370 2024-09-12T02:11:52.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.869+0000 7f826521a640 1 --2- 172.21.15.16:0/582439859 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f82480a2b80 0x7f82480a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:52.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.869+0000 7f825ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564a86b09b40 0x564a86b08090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:52.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.869+0000 7f825ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564a86b09b40 0x564a86b08090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53956/0 (socket says 172.21.15.16:53956) 2024-09-12T02:11:52.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.869+0000 7f825ffff640 1 -- 172.21.15.16:0/3706249384 learned_addr learned my addr 172.21.15.16:0/3706249384 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:11:52.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.869+0000 7f825dffb640 1 -- 172.21.15.16:0/582439859 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f82580cf250 con 0x7f8260106f10 2024-09-12T02:11:52.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.869+0000 7f8264a19640 1 --2- 172.21.15.16:0/3706249384 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564a86b085d0 0x564a86b08a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:52.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.869+0000 7f825ffff640 1 --2- 172.21.15.16:0/3706249384 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564a86b13370 0x564a86b15760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:52.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.869+0000 7f8264a19640 1 -- 172.21.15.16:0/3706249384 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564a86b13370 msgr2=0x564a86b15760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:52.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.869+0000 7f8264a19640 1 --2- 172.21.15.16:0/3706249384 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564a86b13370 0x564a86b15760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:52.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.869+0000 7f8264a19640 1 -- 172.21.15.16:0/3706249384 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564a86b09b40 msgr2=0x564a86b08090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:52.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.869+0000 7f8264a19640 1 --2- 172.21.15.16:0/3706249384 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564a86b09b40 0x564a86b08090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:52.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.869+0000 7f8264a19640 1 -- 172.21.15.16:0/3706249384 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a869fa3a0 con 0x564a86b085d0 2024-09-12T02:11:52.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.869+0000 7f825ffff640 1 --2- 172.21.15.16:0/3706249384 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564a86b13370 0x564a86b15760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:11:52.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.869+0000 7f826521a640 1 --2- 172.21.15.16:0/582439859 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f82480a2b80 0x7f82480a5040 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f8258097f10 tx=0x7f8258066730 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:52.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.870+0000 7f8264a19640 1 --2- 172.21.15.16:0/3706249384 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564a86b085d0 0x564a86b08a10 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f825000bbf0 tx=0x7f82500107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df2b6b593a7f49c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:52.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.870+0000 7f8246ffd640 1 -- 172.21.15.16:0/3706249384 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8250004030 con 0x564a86b085d0 2024-09-12T02:11:52.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.870+0000 7f8246ffd640 1 -- 172.21.15.16:0/3706249384 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82500041d0 con 0x564a86b085d0 2024-09-12T02:11:52.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.870+0000 7f8246ffd640 1 -- 172.21.15.16:0/3706249384 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82500117b0 con 0x564a86b085d0 2024-09-12T02:11:52.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.871+0000 7f8269056740 1 -- 172.21.15.16:0/3706249384 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564a86b085d0 msgr2=0x564a86b08a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:52.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.871+0000 7f8269056740 1 --2- 172.21.15.16:0/3706249384 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564a86b085d0 0x564a86b08a10 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f825000bbf0 tx=0x7f82500107a0 comp rx=0 tx=0).stop 2024-09-12T02:11:52.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.871+0000 7f8269056740 1 -- 172.21.15.16:0/3706249384 shutdown_connections 2024-09-12T02:11:52.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.871+0000 7f8269056740 1 --2- 172.21.15.16:0/3706249384 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564a86b13370 0x564a86b15760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:52.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.871+0000 7f8269056740 1 --2- 172.21.15.16:0/3706249384 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564a86b085d0 0x564a86b08a10 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:52.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.871+0000 7f8269056740 1 --2- 172.21.15.16:0/3706249384 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564a86b09b40 0x564a86b08090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:52.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.871+0000 7f8269056740 1 -- 172.21.15.16:0/3706249384 >> 172.21.15.16:0/3706249384 conn(0x564a86ad1310 msgr2=0x564a86b04c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:52.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.871+0000 7f8269056740 1 -- 172.21.15.16:0/3706249384 shutdown_connections 2024-09-12T02:11:52.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.871+0000 7f8269056740 1 -- 172.21.15.16:0/3706249384 wait complete. 2024-09-12T02:11:52.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.871+0000 7f8269056740 1 -- start start 2024-09-12T02:11:52.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.871+0000 7f8269056740 1 Processor -- start 2024-09-12T02:11:52.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.872+0000 7f8269056740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564a86b085d0 0x564a86b12d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:52.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.872+0000 7f8269056740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564a86b09b40 0x564a86b0fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:52.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.872+0000 7f8269056740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564a86b13370 0x564a86b105e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:52.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.872+0000 7f8269056740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x564a86b16070 con 0x564a86b13370 2024-09-12T02:11:52.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.872+0000 7f8269056740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x564a86a29060 con 0x564a86b09b40 2024-09-12T02:11:52.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.872+0000 7f8269056740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x564a86b161f0 con 0x564a86b085d0 2024-09-12T02:11:52.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.872+0000 7f8269056740 10 client.0 _ll_register_callbacks cb 0x564a8677c8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:11:52.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.872+0000 7f825ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564a86b13370 0x564a86b105e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:52.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.872+0000 7f825ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564a86b13370 0x564a86b105e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53958/0 (socket says 172.21.15.16:53958) 2024-09-12T02:11:52.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.872+0000 7f825ffff640 1 -- 172.21.15.16:0/3576950688 learned_addr learned my addr 172.21.15.16:0/3576950688 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:11:52.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.872+0000 7f8264a19640 1 --2- 172.21.15.16:0/3576950688 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564a86b09b40 0x564a86b0fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:52.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.872+0000 7f825ffff640 1 --2- 172.21.15.16:0/3576950688 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564a86b085d0 0x564a86b12d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:52.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.872+0000 7f825ffff640 1 -- 172.21.15.16:0/3576950688 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564a86b085d0 msgr2=0x564a86b12d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:52.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.872+0000 7f825ffff640 1 --2- 172.21.15.16:0/3576950688 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564a86b085d0 0x564a86b12d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:52.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.872+0000 7f825ffff640 1 -- 172.21.15.16:0/3576950688 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564a86b09b40 msgr2=0x564a86b0fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:52.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.872+0000 7f825ffff640 1 --2- 172.21.15.16:0/3576950688 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564a86b09b40 0x564a86b0fff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:52.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.872+0000 7f825ffff640 1 -- 172.21.15.16:0/3576950688 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a867df3c0 con 0x564a86b13370 2024-09-12T02:11:52.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.872+0000 7f8264a19640 1 --2- 172.21.15.16:0/3576950688 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564a86b09b40 0x564a86b0fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:11:52.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.872+0000 7f825ffff640 1 --2- 172.21.15.16:0/3576950688 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564a86b13370 0x564a86b105e0 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f825402a910 tx=0x7f8254005b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11788163b70ba4d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:52.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.872+0000 7f823ffff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:11:52.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.873+0000 7f823ffff640 1 -- 172.21.15.16:0/3576950688 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8254035070 con 0x564a86b13370 2024-09-12T02:11:52.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.873+0000 7f823ffff640 1 -- 172.21.15.16:0/3576950688 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f8254004020 con 0x564a86b13370 2024-09-12T02:11:52.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.873+0000 7f8269056740 1 -- 172.21.15.16:0/3576950688 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f82500041d0 con 0x564a86b13370 2024-09-12T02:11:52.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.873+0000 7f823ffff640 1 -- 172.21.15.16:0/3576950688 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f8254004220 con 0x564a86b13370 2024-09-12T02:11:52.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.873+0000 7f8269056740 10 client.18531 fetch_fsmap learned FSMap version 25 2024-09-12T02:11:52.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.873+0000 7f8269056740 1 -- 172.21.15.16:0/3576950688 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x564a86b21a80 con 0x564a86b13370 2024-09-12T02:11:52.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.873+0000 7f823ffff640 1 -- 172.21.15.16:0/3576950688 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82500041d0 con 0x564a86b13370 2024-09-12T02:11:52.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.873+0000 7f823ffff640 1 -- 172.21.15.16:0/3576950688 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f82540035f0 con 0x564a86b13370 2024-09-12T02:11:52.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.873+0000 7f8269056740 10 client.18531 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:11:52.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.873+0000 7f8269056740 10 client.18531 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-12T02:11:52.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.873+0000 7f8269056740 20 client.18531 populate_metadata read hostname 'smithi016' 2024-09-12T02:11:52.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.873+0000 7f8269056740 1 --2- 172.21.15.16:0/3576950688 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x564a86b22e90 0x564a86b252e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:52.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.873+0000 7f8269056740 4 client.18531 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:11:52.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.873+0000 7f8269056740 1 -- 172.21.15.16:0/3576950688 --> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564a86a7b860 con 0x564a86b22e90 2024-09-12T02:11:52.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.874+0000 7f8264a19640 1 --2- 172.21.15.16:0/3576950688 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x564a86b22e90 0x564a86b252e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:52.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.874+0000 7f823ffff640 10 client.18531 ms_handle_connect on v2:172.21.15.184:6836/1793393087 2024-09-12T02:11:52.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.874+0000 7f8264a19640 1 --2- 172.21.15.16:0/3576950688 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x564a86b22e90 0x564a86b252e0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=945f1a5c8dfd57f9 server_cookie=fe0f1573285a24a0 in_seq=0 out_seq=0 2024-09-12T02:11:52.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.875+0000 7f823ffff640 1 -- 172.21.15.16:0/3576950688 <== mds.2 v2:172.21.15.184:6836/1793393087 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564a86a7b860 con 0x564a86b22e90 2024-09-12T02:11:52.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.875+0000 7f823ffff640 10 client.18531 handle_command_reply: tid=0 2024-09-12T02:11:52.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.875+0000 7f8269056740 1 client.18531 shutdown 2024-09-12T02:11:52.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.875+0000 7f8269056740 1 -- 172.21.15.16:0/3576950688 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564a86b13370 msgr2=0x564a86b105e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:52.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.875+0000 7f8269056740 1 --2- 172.21.15.16:0/3576950688 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564a86b13370 0x564a86b105e0 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f825402a910 tx=0x7f8254005b90 comp rx=0 tx=0).stop 2024-09-12T02:11:52.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.875+0000 7f8269056740 1 -- 172.21.15.16:0/3576950688 shutdown_connections 2024-09-12T02:11:52.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.875+0000 7f8269056740 1 --2- 172.21.15.16:0/3576950688 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x564a86b22e90 0x564a86b252e0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:52.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.875+0000 7f8269056740 1 --2- 172.21.15.16:0/3576950688 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564a86b13370 0x564a86b105e0 unknown :-1 s=CLOSED pgs=1412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:52.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.875+0000 7f8269056740 1 --2- 172.21.15.16:0/3576950688 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564a86b09b40 0x564a86b0fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:52.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.875+0000 7f8269056740 1 --2- 172.21.15.16:0/3576950688 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564a86b085d0 0x564a86b12d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:52.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.875+0000 7f823ffff640 0 client.18531 ms_handle_reset on v2:172.21.15.184:6836/1793393087 2024-09-12T02:11:52.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.875+0000 7f8269056740 1 -- 172.21.15.16:0/3576950688 >> 172.21.15.16:0/3576950688 conn(0x564a86ad1310 msgr2=0x564a86b0a930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:52.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.875+0000 7f8269056740 1 -- 172.21.15.16:0/3576950688 shutdown_connections 2024-09-12T02:11:52.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.875+0000 7f8269056740 1 -- 172.21.15.16:0/3576950688 wait complete. 2024-09-12T02:11:52.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.875+0000 7f8269056740 20 client.18531 trim_cache size 0 max 16384 2024-09-12T02:11:52.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.888+0000 7f8269056740 1 Processor -- start 2024-09-12T02:11:52.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.889+0000 7f8269056740 1 -- start start 2024-09-12T02:11:52.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.889+0000 7f8269056740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564a86b085d0 0x564a86b04e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:52.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.889+0000 7f8269056740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564a86b09b40 0x564a86b1f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:52.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.889+0000 7f8269056740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564a86b13370 0x564a86b0b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:52.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.889+0000 7f8269056740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f82601b1f20 con 0x564a86b13370 2024-09-12T02:11:52.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.889+0000 7f8269056740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f82601af7f0 con 0x564a86b085d0 2024-09-12T02:11:52.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.889+0000 7f8269056740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f82601ad640 con 0x564a86b09b40 2024-09-12T02:11:52.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.889+0000 7f825ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564a86b13370 0x564a86b0b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:52.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.889+0000 7f825ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564a86b13370 0x564a86b0b120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53964/0 (socket says 172.21.15.16:53964) 2024-09-12T02:11:52.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.889+0000 7f825ffff640 1 -- 172.21.15.16:0/3849898125 learned_addr learned my addr 172.21.15.16:0/3849898125 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:11:52.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.889+0000 7f8264a19640 1 --2- 172.21.15.16:0/3849898125 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564a86b09b40 0x564a86b1f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:52.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.889+0000 7f825ffff640 1 --2- 172.21.15.16:0/3849898125 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564a86b085d0 0x564a86b04e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:52.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.889+0000 7f825ffff640 1 -- 172.21.15.16:0/3849898125 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564a86b09b40 msgr2=0x564a86b1f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:52.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.889+0000 7f825ffff640 1 --2- 172.21.15.16:0/3849898125 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564a86b09b40 0x564a86b1f340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:52.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.889+0000 7f825ffff640 1 -- 172.21.15.16:0/3849898125 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564a86b085d0 msgr2=0x564a86b04e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:52.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.889+0000 7f825ffff640 1 --2- 172.21.15.16:0/3849898125 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564a86b085d0 0x564a86b04e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:52.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.889+0000 7f825ffff640 1 -- 172.21.15.16:0/3849898125 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a86b25960 con 0x564a86b13370 2024-09-12T02:11:52.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.889+0000 7f825ffff640 1 --2- 172.21.15.16:0/3849898125 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564a86b13370 0x564a86b0b120 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f8254009850 tx=0x7f825402a3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=807f89ec6873f191 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:52.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.890+0000 7f82467fc640 1 -- 172.21.15.16:0/3849898125 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8254035040 con 0x564a86b13370 2024-09-12T02:11:52.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.890+0000 7f82467fc640 1 -- 172.21.15.16:0/3849898125 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82540035f0 con 0x564a86b13370 2024-09-12T02:11:52.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.890+0000 7f82467fc640 1 -- 172.21.15.16:0/3849898125 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8254031ba0 con 0x564a86b13370 2024-09-12T02:11:52.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.890+0000 7f8269056740 1 -- 172.21.15.16:0/3849898125 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564a86b13370 msgr2=0x564a86b0b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:52.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.890+0000 7f8269056740 1 --2- 172.21.15.16:0/3849898125 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564a86b13370 0x564a86b0b120 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f8254009850 tx=0x7f825402a3f0 comp rx=0 tx=0).stop 2024-09-12T02:11:52.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.890+0000 7f8269056740 1 -- 172.21.15.16:0/3849898125 shutdown_connections 2024-09-12T02:11:52.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.890+0000 7f8269056740 1 --2- 172.21.15.16:0/3849898125 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564a86b13370 0x564a86b0b120 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:52.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.890+0000 7f8269056740 1 --2- 172.21.15.16:0/3849898125 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564a86b09b40 0x564a86b1f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:52.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.890+0000 7f8269056740 1 --2- 172.21.15.16:0/3849898125 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564a86b085d0 0x564a86b04e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:52.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.890+0000 7f8269056740 1 -- 172.21.15.16:0/3849898125 >> 172.21.15.16:0/3849898125 conn(0x564a86ad1310 msgr2=0x564a86b04700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:52.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.890+0000 7f8269056740 1 -- 172.21.15.16:0/3849898125 shutdown_connections 2024-09-12T02:11:52.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.890+0000 7f8269056740 1 -- 172.21.15.16:0/3849898125 wait complete. 2024-09-12T02:11:52.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.891+0000 7f8269056740 1 -- start start 2024-09-12T02:11:52.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.891+0000 7f8269056740 1 Processor -- start 2024-09-12T02:11:52.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.891+0000 7f8269056740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564a86b085d0 0x564a86b1ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:52.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.891+0000 7f8269056740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564a86b09b40 0x564a86b0cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:52.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.891+0000 7f8269056740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564a86b13370 0x564a86b0d110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:52.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.891+0000 7f8269056740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f82601adcb0 con 0x564a86b085d0 2024-09-12T02:11:52.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.891+0000 7f8269056740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f82601af120 con 0x564a86b09b40 2024-09-12T02:11:52.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.891+0000 7f8269056740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f82601ae380 con 0x564a86b13370 2024-09-12T02:11:52.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.891+0000 7f8269056740 10 client.0 _ll_register_callbacks cb 0x7f8254004ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:11:52.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.891+0000 7f825ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564a86b085d0 0x564a86b1ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:52.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.891+0000 7f825ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564a86b085d0 0x564a86b1ba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53974/0 (socket says 172.21.15.16:53974) 2024-09-12T02:11:52.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.891+0000 7f825ffff640 1 -- 172.21.15.16:0/3537912023 learned_addr learned my addr 172.21.15.16:0/3537912023 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:11:52.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.892+0000 7f8264a19640 1 --2- 172.21.15.16:0/3537912023 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564a86b09b40 0x564a86b0cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:52.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.892+0000 7f825ffff640 1 -- 172.21.15.16:0/3537912023 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564a86b13370 msgr2=0x564a86b0d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:52.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.892+0000 7f825ffff640 1 --2- 172.21.15.16:0/3537912023 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564a86b13370 0x564a86b0d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:52.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.892+0000 7f825ffff640 1 -- 172.21.15.16:0/3537912023 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564a86b09b40 msgr2=0x564a86b0cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:52.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.892+0000 7f825ffff640 1 --2- 172.21.15.16:0/3537912023 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564a86b09b40 0x564a86b0cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:52.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.892+0000 7f825ffff640 1 -- 172.21.15.16:0/3537912023 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a86b0d780 con 0x564a86b085d0 2024-09-12T02:11:52.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.892+0000 7f8264a19640 1 --2- 172.21.15.16:0/3537912023 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564a86b09b40 0x564a86b0cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:11:52.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.892+0000 7f825ffff640 1 --2- 172.21.15.16:0/3537912023 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564a86b085d0 0x564a86b1ba70 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f8254009900 tx=0x7f8254004f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8d750097c36439 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:52.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.892+0000 7f823ffff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:11:52.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.892+0000 7f823ffff640 1 -- 172.21.15.16:0/3537912023 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8254035040 con 0x564a86b085d0 2024-09-12T02:11:52.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.892+0000 7f823ffff640 1 -- 172.21.15.16:0/3537912023 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f8254003e40 con 0x564a86b085d0 2024-09-12T02:11:52.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.892+0000 7f8269056740 1 -- 172.21.15.16:0/3537912023 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f82540035f0 con 0x564a86b085d0 2024-09-12T02:11:52.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.892+0000 7f823ffff640 1 -- 172.21.15.16:0/3537912023 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f8254004040 con 0x564a86b085d0 2024-09-12T02:11:52.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.892+0000 7f8269056740 10 client.18537 fetch_fsmap learned FSMap version 25 2024-09-12T02:11:52.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.892+0000 7f8269056740 1 -- 172.21.15.16:0/3537912023 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x564a86b8d9f0 con 0x564a86b085d0 2024-09-12T02:11:52.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.892+0000 7f823ffff640 1 -- 172.21.15.16:0/3537912023 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82540035f0 con 0x564a86b085d0 2024-09-12T02:11:52.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.893+0000 7f823ffff640 1 -- 172.21.15.16:0/3537912023 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f8254057a00 con 0x564a86b085d0 2024-09-12T02:11:52.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.893+0000 7f8269056740 10 client.18537 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:11:52.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.893+0000 7f8269056740 10 client.18537 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-12T02:11:52.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.893+0000 7f8269056740 20 client.18537 populate_metadata read hostname 'smithi016' 2024-09-12T02:11:52.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.893+0000 7f8269056740 1 --2- 172.21.15.16:0/3537912023 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x564a86b8e0b0 0x564a86b904a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:52.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.893+0000 7f8269056740 4 client.18537 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T02:11:52.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.893+0000 7f8269056740 1 -- 172.21.15.16:0/3537912023 --> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564a86a75d70 con 0x564a86b8e0b0 2024-09-12T02:11:52.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.893+0000 7f8264a19640 1 --2- 172.21.15.16:0/3537912023 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x564a86b8e0b0 0x564a86b904a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:52.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.893+0000 7f823ffff640 10 client.18537 ms_handle_connect on v2:172.21.15.184:6836/1793393087 2024-09-12T02:11:52.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.893+0000 7f8264a19640 1 --2- 172.21.15.16:0/3537912023 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x564a86b8e0b0 0x564a86b904a0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e57f1acd617863cc server_cookie=d38617d642f910f7 in_seq=0 out_seq=0 2024-09-12T02:11:52.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.894+0000 7f823ffff640 1 -- 172.21.15.16:0/3537912023 <== mds.2 v2:172.21.15.184:6836/1793393087 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564a86a75d70 con 0x564a86b8e0b0 2024-09-12T02:11:52.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.894+0000 7f823ffff640 10 client.18537 handle_command_reply: tid=0 2024-09-12T02:11:52.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.894+0000 7f8269056740 1 client.18537 shutdown 2024-09-12T02:11:52.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.894+0000 7f8269056740 1 -- 172.21.15.16:0/3537912023 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564a86b085d0 msgr2=0x564a86b1ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:52.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.894+0000 7f8269056740 1 --2- 172.21.15.16:0/3537912023 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564a86b085d0 0x564a86b1ba70 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f8254009900 tx=0x7f8254004f80 comp rx=0 tx=0).stop 2024-09-12T02:11:52.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.894+0000 7f8269056740 1 -- 172.21.15.16:0/3537912023 shutdown_connections 2024-09-12T02:11:52.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.894+0000 7f8269056740 1 --2- 172.21.15.16:0/3537912023 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564a86b13370 0x564a86b0d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:52.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.894+0000 7f8269056740 1 --2- 172.21.15.16:0/3537912023 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564a86b09b40 0x564a86b0cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:52.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.894+0000 7f8269056740 1 --2- 172.21.15.16:0/3537912023 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x564a86b8e0b0 0x564a86b904a0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:52.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.894+0000 7f8269056740 1 --2- 172.21.15.16:0/3537912023 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564a86b085d0 0x564a86b1ba70 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:52.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.894+0000 7f8269056740 1 -- 172.21.15.16:0/3537912023 >> 172.21.15.16:0/3537912023 conn(0x564a86ad1310 msgr2=0x564a86b034e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:52.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.894+0000 7f823ffff640 0 client.18537 ms_handle_reset on v2:172.21.15.184:6836/1793393087 2024-09-12T02:11:52.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.894+0000 7f8269056740 1 -- 172.21.15.16:0/3537912023 shutdown_connections 2024-09-12T02:11:52.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.894+0000 7f8269056740 1 -- 172.21.15.16:0/3537912023 wait complete. 2024-09-12T02:11:52.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.894+0000 7f8269056740 20 client.18537 trim_cache size 0 max 16384 2024-09-12T02:11:52.908 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T02:11:52.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.895+0000 7f8266c7d640 1 -- 172.21.15.16:0/582439859 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f82480a2b80 msgr2=0x7f82480a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:52.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.895+0000 7f8266c7d640 1 --2- 172.21.15.16:0/582439859 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f82480a2b80 0x7f82480a5040 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f8258097f10 tx=0x7f8258066730 comp rx=0 tx=0).stop 2024-09-12T02:11:52.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.895+0000 7f8266c7d640 1 -- 172.21.15.16:0/582439859 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8260106f10 msgr2=0x7f8260074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:52.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.895+0000 7f8266c7d640 1 --2- 172.21.15.16:0/582439859 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8260106f10 0x7f8260074f70 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f825800eaa0 tx=0x7f825800ef60 comp rx=0 tx=0).stop 2024-09-12T02:11:52.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.895+0000 7f8266c7d640 1 -- 172.21.15.16:0/582439859 shutdown_connections 2024-09-12T02:11:52.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.895+0000 7f8266c7d640 1 --2- 172.21.15.16:0/582439859 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f82480a2b80 0x7f82480a5040 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:52.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.895+0000 7f8266c7d640 1 --2- 172.21.15.16:0/582439859 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8260106f10 0x7f8260074f70 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:52.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.895+0000 7f8266c7d640 1 --2- 172.21.15.16:0/582439859 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8260102790 0x7f8260074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:52.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.895+0000 7f8266c7d640 1 --2- 172.21.15.16:0/582439859 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8260101de0 0x7f82600744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:52.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.895+0000 7f8266c7d640 1 -- 172.21.15.16:0/582439859 >> 172.21.15.16:0/582439859 conn(0x7f82600fd8c0 msgr2=0x7f82600fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:52.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.895+0000 7f8266c7d640 1 -- 172.21.15.16:0/582439859 shutdown_connections 2024-09-12T02:11:52.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:52.895+0000 7f8266c7d640 1 -- 172.21.15.16:0/582439859 wait complete. 2024-09-12T02:11:53.659 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-12T02:11:53.921 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:11:54.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.354+0000 7f210eb26640 1 -- 172.21.15.16:0/2936282299 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2108106a60 msgr2=0x7f2108106e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:54.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.354+0000 7f210eb26640 1 --2- 172.21.15.16:0/2936282299 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2108106a60 0x7f2108106e40 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f2104009930 tx=0x7f210402a420 comp rx=0 tx=0).stop 2024-09-12T02:11:54.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.354+0000 7f210eb26640 1 -- 172.21.15.16:0/2936282299 shutdown_connections 2024-09-12T02:11:54.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.354+0000 7f210eb26640 1 --2- 172.21.15.16:0/2936282299 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2108101400 0x7f210810d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:54.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.354+0000 7f210eb26640 1 --2- 172.21.15.16:0/2936282299 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2108100a60 0x7f2108100ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:54.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.354+0000 7f210eb26640 1 --2- 172.21.15.16:0/2936282299 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2108106a60 0x7f2108106e40 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:54.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.354+0000 7f210eb26640 1 -- 172.21.15.16:0/2936282299 >> 172.21.15.16:0/2936282299 conn(0x7f21080fc780 msgr2=0x7f21080feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:54.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.354+0000 7f210eb26640 1 -- 172.21.15.16:0/2936282299 shutdown_connections 2024-09-12T02:11:54.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.354+0000 7f210eb26640 1 -- 172.21.15.16:0/2936282299 wait complete. 2024-09-12T02:11:54.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.355+0000 7f210eb26640 1 Processor -- start 2024-09-12T02:11:54.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.355+0000 7f210eb26640 1 -- start start 2024-09-12T02:11:54.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.355+0000 7f210eb26640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2108100a60 0x7f2108198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:54.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.355+0000 7f210eb26640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2108101400 0x7f2108198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:54.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.355+0000 7f210eb26640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2108106a60 0x7f210819cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:54.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.355+0000 7f210eb26640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2108110e80 con 0x7f2108100a60 2024-09-12T02:11:54.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.355+0000 7f210eb26640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2108110d00 con 0x7f2108101400 2024-09-12T02:11:54.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.355+0000 7f210eb26640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f2108111000 con 0x7f2108106a60 2024-09-12T02:11:54.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.355+0000 7f210c8c2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2108100a60 0x7f2108198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:54.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.355+0000 7f210c8c2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2108100a60 0x7f2108198440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53992/0 (socket says 172.21.15.16:53992) 2024-09-12T02:11:54.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.355+0000 7f210c8c2640 1 -- 172.21.15.16:0/3669994777 learned_addr learned my addr 172.21.15.16:0/3669994777 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:11:54.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.355+0000 7f210d0c3640 1 --2- 172.21.15.16:0/3669994777 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2108106a60 0x7f210819cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:54.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.355+0000 7f20fffff640 1 --2- 172.21.15.16:0/3669994777 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2108101400 0x7f2108198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:54.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.355+0000 7f210c8c2640 1 -- 172.21.15.16:0/3669994777 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2108106a60 msgr2=0x7f210819cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:54.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.355+0000 7f210c8c2640 1 --2- 172.21.15.16:0/3669994777 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2108106a60 0x7f210819cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:54.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.355+0000 7f210c8c2640 1 -- 172.21.15.16:0/3669994777 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2108101400 msgr2=0x7f2108198980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:54.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.355+0000 7f210c8c2640 1 --2- 172.21.15.16:0/3669994777 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2108101400 0x7f2108198980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:54.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.355+0000 7f210c8c2640 1 -- 172.21.15.16:0/3669994777 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f210819d3f0 con 0x7f2108100a60 2024-09-12T02:11:54.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.355+0000 7f20fffff640 1 --2- 172.21.15.16:0/3669994777 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2108101400 0x7f2108198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:11:54.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.356+0000 7f210c8c2640 1 --2- 172.21.15.16:0/3669994777 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2108100a60 0x7f2108198440 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f2104009a50 tx=0x7f2104004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:54.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.356+0000 7f20fdffb640 1 -- 172.21.15.16:0/3669994777 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f210403f030 con 0x7f2108100a60 2024-09-12T02:11:54.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.356+0000 7f210eb26640 1 -- 172.21.15.16:0/3669994777 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f210819d680 con 0x7f2108100a60 2024-09-12T02:11:54.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.356+0000 7f20fdffb640 1 -- 172.21.15.16:0/3669994777 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2104003ce0 con 0x7f2108100a60 2024-09-12T02:11:54.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.356+0000 7f20fdffb640 1 -- 172.21.15.16:0/3669994777 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2104035070 con 0x7f2108100a60 2024-09-12T02:11:54.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.356+0000 7f210eb26640 1 -- 172.21.15.16:0/3669994777 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21081a4fc0 con 0x7f2108100a60 2024-09-12T02:11:54.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.358+0000 7f2110eff740 1 Processor -- start 2024-09-12T02:11:54.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.358+0000 7f20fdffb640 1 -- 172.21.15.16:0/3669994777 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f2104042030 con 0x7f2108100a60 2024-09-12T02:11:54.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.359+0000 7f2110eff740 1 -- start start 2024-09-12T02:11:54.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.359+0000 7f2110eff740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6b8faeb40 0x55c6b8fad090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:54.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.359+0000 7f2110eff740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c6b8fad5d0 0x55c6b8fada10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:54.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.359+0000 7f2110eff740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c6b8fb8370 0x55c6b8fba760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:54.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.359+0000 7f2110eff740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55c6b8f132f0 con 0x55c6b8fad5d0 2024-09-12T02:11:54.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.359+0000 7f2110eff740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55c6b8ebacf0 con 0x55c6b8faeb40 2024-09-12T02:11:54.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.359+0000 7f2110eff740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55c6b8f0f2b0 con 0x55c6b8fb8370 2024-09-12T02:11:54.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.359+0000 7f210c8c2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c6b8fad5d0 0x55c6b8fada10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:54.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.359+0000 7f210c8c2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c6b8fad5d0 0x55c6b8fada10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54002/0 (socket says 172.21.15.16:54002) 2024-09-12T02:11:54.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.359+0000 7f210c8c2640 1 -- 172.21.15.16:0/1859691920 learned_addr learned my addr 172.21.15.16:0/1859691920 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:11:54.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.359+0000 7f210d0c3640 1 --2- 172.21.15.16:0/1859691920 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6b8faeb40 0x55c6b8fad090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:54.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.359+0000 7f20fffff640 1 --2- 172.21.15.16:0/1859691920 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c6b8fb8370 0x55c6b8fba760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:54.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.359+0000 7f20fdffb640 1 --2- 172.21.15.16:0/3669994777 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f20f00a2b80 0x7f20f00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:54.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.359+0000 7f210c8c2640 1 -- 172.21.15.16:0/1859691920 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c6b8fb8370 msgr2=0x55c6b8fba760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:54.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.359+0000 7f210c8c2640 1 --2- 172.21.15.16:0/1859691920 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c6b8fb8370 0x55c6b8fba760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:54.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.359+0000 7f210c8c2640 1 -- 172.21.15.16:0/1859691920 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6b8faeb40 msgr2=0x55c6b8fad090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:54.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.359+0000 7f210c8c2640 1 --2- 172.21.15.16:0/1859691920 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6b8faeb40 0x55c6b8fad090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:54.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.360+0000 7f210c8c2640 1 -- 172.21.15.16:0/1859691920 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6b8e9f3a0 con 0x55c6b8fad5d0 2024-09-12T02:11:54.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.360+0000 7f20fffff640 1 --2- 172.21.15.16:0/1859691920 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c6b8fb8370 0x55c6b8fba760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:11:54.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.360+0000 7f210d0c3640 1 --2- 172.21.15.16:0/3669994777 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f20f00a2b80 0x7f20f00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:54.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.360+0000 7f20fdffb640 1 -- 172.21.15.16:0/3669994777 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f21040ef6b0 con 0x7f2108100a60 2024-09-12T02:11:54.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.360+0000 7f210c8c2640 1 --2- 172.21.15.16:0/1859691920 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c6b8fad5d0 0x55c6b8fada10 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f21040d8780 tx=0x7f2104086ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3728a853c01c479 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:54.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.360+0000 7f20eeffd640 1 -- 172.21.15.16:0/1859691920 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21040bbce0 con 0x55c6b8fad5d0 2024-09-12T02:11:54.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.360+0000 7f210d0c3640 1 --2- 172.21.15.16:0/3669994777 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f20f00a2b80 0x7f20f00a5040 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f2100007e50 tx=0x7f2100004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:54.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.360+0000 7f20eeffd640 1 -- 172.21.15.16:0/1859691920 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f21040d8d00 con 0x55c6b8fad5d0 2024-09-12T02:11:54.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.360+0000 7f20eeffd640 1 -- 172.21.15.16:0/1859691920 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21040ba4d0 con 0x55c6b8fad5d0 2024-09-12T02:11:54.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.361+0000 7f2110eff740 1 -- 172.21.15.16:0/1859691920 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c6b8fad5d0 msgr2=0x55c6b8fada10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:54.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.361+0000 7f2110eff740 1 --2- 172.21.15.16:0/1859691920 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c6b8fad5d0 0x55c6b8fada10 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f21040d8780 tx=0x7f2104086ae0 comp rx=0 tx=0).stop 2024-09-12T02:11:54.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.361+0000 7f2110eff740 1 -- 172.21.15.16:0/1859691920 shutdown_connections 2024-09-12T02:11:54.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.361+0000 7f2110eff740 1 --2- 172.21.15.16:0/1859691920 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c6b8fb8370 0x55c6b8fba760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:54.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.361+0000 7f2110eff740 1 --2- 172.21.15.16:0/1859691920 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c6b8fad5d0 0x55c6b8fada10 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:54.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.361+0000 7f2110eff740 1 --2- 172.21.15.16:0/1859691920 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6b8faeb40 0x55c6b8fad090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:54.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.361+0000 7f2110eff740 1 -- 172.21.15.16:0/1859691920 >> 172.21.15.16:0/1859691920 conn(0x55c6b8f76310 msgr2=0x55c6b8fa9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:54.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.361+0000 7f2110eff740 1 -- 172.21.15.16:0/1859691920 shutdown_connections 2024-09-12T02:11:54.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.361+0000 7f2110eff740 1 -- 172.21.15.16:0/1859691920 wait complete. 2024-09-12T02:11:54.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.361+0000 7f2110eff740 1 -- start start 2024-09-12T02:11:54.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.361+0000 7f2110eff740 1 Processor -- start 2024-09-12T02:11:54.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.362+0000 7f2110eff740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6b8fad5d0 0x55c6b8fb7d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:54.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.362+0000 7f2110eff740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c6b8faeb40 0x55c6b8fb4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:54.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.362+0000 7f2110eff740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c6b8fb8370 0x55c6b8fb5560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:54.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.362+0000 7f2110eff740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55c6b8fbae30 con 0x55c6b8faeb40 2024-09-12T02:11:54.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.362+0000 7f2110eff740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55c6b8ece060 con 0x55c6b8fad5d0 2024-09-12T02:11:54.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.362+0000 7f2110eff740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55c6b8fbafb0 con 0x55c6b8fb8370 2024-09-12T02:11:54.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.362+0000 7f2110eff740 10 client.0 _ll_register_callbacks cb 0x55c6b8c218c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:11:54.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.362+0000 7f20fffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c6b8faeb40 0x55c6b8fb4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:54.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.362+0000 7f20fffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c6b8faeb40 0x55c6b8fb4fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54010/0 (socket says 172.21.15.16:54010) 2024-09-12T02:11:54.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.362+0000 7f20fffff640 1 -- 172.21.15.16:0/179189083 learned_addr learned my addr 172.21.15.16:0/179189083 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:11:54.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.362+0000 7f210d0c3640 1 --2- 172.21.15.16:0/179189083 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c6b8fb8370 0x55c6b8fb5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:54.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.362+0000 7f210c8c2640 1 --2- 172.21.15.16:0/179189083 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6b8fad5d0 0x55c6b8fb7d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:54.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.362+0000 7f20fffff640 1 -- 172.21.15.16:0/179189083 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c6b8fb8370 msgr2=0x55c6b8fb5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:54.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.362+0000 7f20fffff640 1 --2- 172.21.15.16:0/179189083 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c6b8fb8370 0x55c6b8fb5560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:54.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.362+0000 7f20fffff640 1 -- 172.21.15.16:0/179189083 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6b8fad5d0 msgr2=0x55c6b8fb7d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:54.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.362+0000 7f20fffff640 1 --2- 172.21.15.16:0/179189083 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6b8fad5d0 0x55c6b8fb7d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:54.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.362+0000 7f20fffff640 1 -- 172.21.15.16:0/179189083 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6b8c843c0 con 0x55c6b8faeb40 2024-09-12T02:11:54.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.362+0000 7f210c8c2640 1 --2- 172.21.15.16:0/179189083 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6b8fad5d0 0x55c6b8fb7d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:11:54.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.362+0000 7f210d0c3640 1 --2- 172.21.15.16:0/179189083 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c6b8fb8370 0x55c6b8fb5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:11:54.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.362+0000 7f20fffff640 1 --2- 172.21.15.16:0/179189083 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c6b8faeb40 0x55c6b8fb4fc0 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f20f800ded0 tx=0x7f20f800f5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=312a934d7faff2a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:54.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.362+0000 7f20e7fff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:11:54.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.362+0000 7f20e7fff640 1 -- 172.21.15.16:0/179189083 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20f8018070 con 0x55c6b8faeb40 2024-09-12T02:11:54.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.363+0000 7f20e7fff640 1 -- 172.21.15.16:0/179189083 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f20f8014680 con 0x55c6b8faeb40 2024-09-12T02:11:54.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.363+0000 7f2110eff740 1 -- 172.21.15.16:0/179189083 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55c6b8f20860 con 0x55c6b8faeb40 2024-09-12T02:11:54.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.363+0000 7f20e7fff640 1 -- 172.21.15.16:0/179189083 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f20f8024050 con 0x55c6b8faeb40 2024-09-12T02:11:54.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.363+0000 7f2110eff740 10 client.18549 fetch_fsmap learned FSMap version 25 2024-09-12T02:11:54.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.363+0000 7f2110eff740 1 -- 172.21.15.16:0/179189083 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c6b8fc67d0 con 0x55c6b8faeb40 2024-09-12T02:11:54.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.363+0000 7f20e7fff640 1 -- 172.21.15.16:0/179189083 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55c6b8f20860 con 0x55c6b8faeb40 2024-09-12T02:11:54.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.363+0000 7f20e7fff640 1 -- 172.21.15.16:0/179189083 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f20f8013b30 con 0x55c6b8faeb40 2024-09-12T02:11:54.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.363+0000 7f2110eff740 10 client.18549 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:11:54.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.363+0000 7f2110eff740 10 client.18549 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T02:11:54.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.363+0000 7f2110eff740 20 client.18549 populate_metadata read hostname 'smithi016' 2024-09-12T02:11:54.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.363+0000 7f2110eff740 1 --2- 172.21.15.16:0/179189083 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x55c6b8fc7c30 0x55c6b8fca090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:54.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.363+0000 7f2110eff740 4 client.18549 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:11:54.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.363+0000 7f2110eff740 1 -- 172.21.15.16:0/179189083 --> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c6b8f1ad70 con 0x55c6b8fc7c30 2024-09-12T02:11:54.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.363+0000 7f210c8c2640 1 --2- 172.21.15.16:0/179189083 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x55c6b8fc7c30 0x55c6b8fca090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:54.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.364+0000 7f20e7fff640 10 client.18549 ms_handle_connect on v2:172.21.15.184:6838/4203347941 2024-09-12T02:11:54.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.364+0000 7f210c8c2640 1 --2- 172.21.15.16:0/179189083 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x55c6b8fc7c30 0x55c6b8fca090 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=49f35097629424bf server_cookie=e6e11e326f8a510f in_seq=0 out_seq=0 2024-09-12T02:11:54.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.365+0000 7f20e7fff640 1 -- 172.21.15.16:0/179189083 <== mds.1 v2:172.21.15.184:6838/4203347941 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c6b8f1ad70 con 0x55c6b8fc7c30 2024-09-12T02:11:54.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.365+0000 7f20e7fff640 10 client.18549 handle_command_reply: tid=0 2024-09-12T02:11:54.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.365+0000 7f2110eff740 1 client.18549 shutdown 2024-09-12T02:11:54.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.365+0000 7f2110eff740 1 -- 172.21.15.16:0/179189083 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c6b8faeb40 msgr2=0x55c6b8fb4fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:54.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.365+0000 7f2110eff740 1 --2- 172.21.15.16:0/179189083 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c6b8faeb40 0x55c6b8fb4fc0 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f20f800ded0 tx=0x7f20f800f5b0 comp rx=0 tx=0).stop 2024-09-12T02:11:54.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.365+0000 7f2110eff740 1 -- 172.21.15.16:0/179189083 shutdown_connections 2024-09-12T02:11:54.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.365+0000 7f2110eff740 1 --2- 172.21.15.16:0/179189083 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x55c6b8fc7c30 0x55c6b8fca090 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:54.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.365+0000 7f2110eff740 1 --2- 172.21.15.16:0/179189083 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c6b8fb8370 0x55c6b8fb5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:54.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.365+0000 7f2110eff740 1 --2- 172.21.15.16:0/179189083 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c6b8faeb40 0x55c6b8fb4fc0 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:54.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.365+0000 7f2110eff740 1 --2- 172.21.15.16:0/179189083 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6b8fad5d0 0x55c6b8fb7d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:54.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.365+0000 7f2110eff740 1 -- 172.21.15.16:0/179189083 >> 172.21.15.16:0/179189083 conn(0x55c6b8f76310 msgr2=0x55c6b8faf930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:54.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.365+0000 7f20e7fff640 0 client.18549 ms_handle_reset on v2:172.21.15.184:6838/4203347941 2024-09-12T02:11:54.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.365+0000 7f2110eff740 1 -- 172.21.15.16:0/179189083 shutdown_connections 2024-09-12T02:11:54.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.365+0000 7f2110eff740 1 -- 172.21.15.16:0/179189083 wait complete. 2024-09-12T02:11:54.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.365+0000 7f2110eff740 20 client.18549 trim_cache size 0 max 16384 2024-09-12T02:11:54.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.378+0000 7f2110eff740 1 Processor -- start 2024-09-12T02:11:54.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.378+0000 7f2110eff740 1 -- start start 2024-09-12T02:11:54.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.378+0000 7f2110eff740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c6b8fad5d0 0x55c6b8fc3f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:54.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.378+0000 7f2110eff740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6b8faeb40 0x55c6b8fb0120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:54.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.378+0000 7f2110eff740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c6b8fb8370 0x55c6b8fb0660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:54.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.378+0000 7f2110eff740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f21081afd10 con 0x55c6b8fb8370 2024-09-12T02:11:54.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.378+0000 7f2110eff740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f21081ad5e0 con 0x55c6b8faeb40 2024-09-12T02:11:54.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.378+0000 7f2110eff740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f21081ab430 con 0x55c6b8fad5d0 2024-09-12T02:11:54.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.379+0000 7f210d0c3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c6b8fb8370 0x55c6b8fb0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:54.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.379+0000 7f210d0c3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c6b8fb8370 0x55c6b8fb0660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54014/0 (socket says 172.21.15.16:54014) 2024-09-12T02:11:54.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.379+0000 7f210d0c3640 1 -- 172.21.15.16:0/2610423233 learned_addr learned my addr 172.21.15.16:0/2610423233 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:11:54.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.379+0000 7f20fffff640 1 --2- 172.21.15.16:0/2610423233 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6b8faeb40 0x55c6b8fb0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:54.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.379+0000 7f210c8c2640 1 --2- 172.21.15.16:0/2610423233 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c6b8fad5d0 0x55c6b8fc3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:54.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.379+0000 7f210d0c3640 1 -- 172.21.15.16:0/2610423233 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c6b8fad5d0 msgr2=0x55c6b8fc3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:54.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.379+0000 7f210d0c3640 1 --2- 172.21.15.16:0/2610423233 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c6b8fad5d0 0x55c6b8fc3f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:54.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.379+0000 7f210d0c3640 1 -- 172.21.15.16:0/2610423233 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6b8faeb40 msgr2=0x55c6b8fb0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:54.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.379+0000 7f210d0c3640 1 --2- 172.21.15.16:0/2610423233 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6b8faeb40 0x55c6b8fb0120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:54.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.379+0000 7f210d0c3640 1 -- 172.21.15.16:0/2610423233 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6b8fb0ba0 con 0x55c6b8fb8370 2024-09-12T02:11:54.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.379+0000 7f210d0c3640 1 --2- 172.21.15.16:0/2610423233 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c6b8fb8370 0x55c6b8fb0660 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f2100007630 tx=0x7f2100013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a9aad95b78b00d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:54.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.379+0000 7f20ee7fc640 1 -- 172.21.15.16:0/2610423233 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f210001d880 con 0x55c6b8fb8370 2024-09-12T02:11:54.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.379+0000 7f20ee7fc640 1 -- 172.21.15.16:0/2610423233 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f210001b440 con 0x55c6b8fb8370 2024-09-12T02:11:54.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.379+0000 7f20ee7fc640 1 -- 172.21.15.16:0/2610423233 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2100026600 con 0x55c6b8fb8370 2024-09-12T02:11:54.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.380+0000 7f2110eff740 1 -- 172.21.15.16:0/2610423233 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c6b8fb8370 msgr2=0x55c6b8fb0660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:54.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.380+0000 7f2110eff740 1 --2- 172.21.15.16:0/2610423233 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c6b8fb8370 0x55c6b8fb0660 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f2100007630 tx=0x7f2100013000 comp rx=0 tx=0).stop 2024-09-12T02:11:54.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.380+0000 7f2110eff740 1 -- 172.21.15.16:0/2610423233 shutdown_connections 2024-09-12T02:11:54.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.380+0000 7f2110eff740 1 --2- 172.21.15.16:0/2610423233 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c6b8fb8370 0x55c6b8fb0660 unknown :-1 s=CLOSED pgs=1419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:54.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.380+0000 7f2110eff740 1 --2- 172.21.15.16:0/2610423233 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6b8faeb40 0x55c6b8fb0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:54.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.380+0000 7f2110eff740 1 --2- 172.21.15.16:0/2610423233 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c6b8fad5d0 0x55c6b8fc3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:54.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.380+0000 7f2110eff740 1 -- 172.21.15.16:0/2610423233 >> 172.21.15.16:0/2610423233 conn(0x55c6b8f76310 msgr2=0x55c6b8fb40b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:54.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.380+0000 7f2110eff740 1 -- 172.21.15.16:0/2610423233 shutdown_connections 2024-09-12T02:11:54.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.380+0000 7f2110eff740 1 -- 172.21.15.16:0/2610423233 wait complete. 2024-09-12T02:11:54.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.380+0000 7f2110eff740 1 -- start start 2024-09-12T02:11:54.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.381+0000 7f2110eff740 1 Processor -- start 2024-09-12T02:11:54.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.381+0000 7f2110eff740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c6b8fad5d0 0x55c6b8fc0790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:54.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.381+0000 7f2110eff740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6b8faeb40 0x55c6b8fbbac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:54.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.381+0000 7f2110eff740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c6b8fb8370 0x55c6b8fbc000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:54.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.381+0000 7f2110eff740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f21081abaa0 con 0x55c6b8fb8370 2024-09-12T02:11:54.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.381+0000 7f2110eff740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f21081acf10 con 0x55c6b8faeb40 2024-09-12T02:11:54.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.381+0000 7f2110eff740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f21081ac170 con 0x55c6b8fad5d0 2024-09-12T02:11:54.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.381+0000 7f2110eff740 10 client.0 _ll_register_callbacks cb 0x7f20f800d180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:11:54.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.381+0000 7f210d0c3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c6b8fb8370 0x55c6b8fbc000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:54.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.381+0000 7f210d0c3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c6b8fb8370 0x55c6b8fbc000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54028/0 (socket says 172.21.15.16:54028) 2024-09-12T02:11:54.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.381+0000 7f210d0c3640 1 -- 172.21.15.16:0/1604445251 learned_addr learned my addr 172.21.15.16:0/1604445251 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:11:54.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.381+0000 7f20fffff640 1 --2- 172.21.15.16:0/1604445251 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6b8faeb40 0x55c6b8fbbac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:54.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.381+0000 7f210c8c2640 1 --2- 172.21.15.16:0/1604445251 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c6b8fad5d0 0x55c6b8fc0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:54.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.381+0000 7f210d0c3640 1 -- 172.21.15.16:0/1604445251 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c6b8fad5d0 msgr2=0x55c6b8fc0790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:54.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.381+0000 7f210d0c3640 1 --2- 172.21.15.16:0/1604445251 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c6b8fad5d0 0x55c6b8fc0790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:54.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.381+0000 7f210d0c3640 1 -- 172.21.15.16:0/1604445251 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6b8faeb40 msgr2=0x55c6b8fbbac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:54.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.381+0000 7f210d0c3640 1 --2- 172.21.15.16:0/1604445251 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6b8faeb40 0x55c6b8fbbac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:54.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.381+0000 7f210d0c3640 1 -- 172.21.15.16:0/1604445251 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6b8fbc830 con 0x55c6b8fb8370 2024-09-12T02:11:54.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.382+0000 7f210d0c3640 1 --2- 172.21.15.16:0/1604445251 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c6b8fb8370 0x55c6b8fbc000 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f2100007600 tx=0x7f210001b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=748d761614532428 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:54.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.382+0000 7f20e7fff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:11:54.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.382+0000 7f20e7fff640 1 -- 172.21.15.16:0/1604445251 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f210001d880 con 0x55c6b8fb8370 2024-09-12T02:11:54.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.382+0000 7f20e7fff640 1 -- 172.21.15.16:0/1604445251 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f2100023040 con 0x55c6b8fb8370 2024-09-12T02:11:54.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.382+0000 7f2110eff740 1 -- 172.21.15.16:0/1604445251 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f210001b440 con 0x55c6b8fb8370 2024-09-12T02:11:54.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.382+0000 7f20e7fff640 1 -- 172.21.15.16:0/1604445251 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f210001da20 con 0x55c6b8fb8370 2024-09-12T02:11:54.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.382+0000 7f2110eff740 10 client.18555 fetch_fsmap learned FSMap version 25 2024-09-12T02:11:54.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.382+0000 7f2110eff740 1 -- 172.21.15.16:0/1604445251 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c6b90328c0 con 0x55c6b8fb8370 2024-09-12T02:11:54.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.382+0000 7f20e7fff640 1 -- 172.21.15.16:0/1604445251 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f210001b440 con 0x55c6b8fb8370 2024-09-12T02:11:54.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.382+0000 7f20e7fff640 1 -- 172.21.15.16:0/1604445251 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f2100030c30 con 0x55c6b8fb8370 2024-09-12T02:11:54.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.382+0000 7f2110eff740 10 client.18555 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:11:54.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.382+0000 7f2110eff740 10 client.18555 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T02:11:54.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.383+0000 7f2110eff740 20 client.18555 populate_metadata read hostname 'smithi016' 2024-09-12T02:11:54.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.383+0000 7f2110eff740 1 --2- 172.21.15.16:0/1604445251 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x55c6b9032f70 0x55c6b9035390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:54.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.383+0000 7f2110eff740 4 client.18555 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T02:11:54.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.383+0000 7f2110eff740 1 -- 172.21.15.16:0/1604445251 --> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c6b8e97170 con 0x55c6b9032f70 2024-09-12T02:11:54.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.383+0000 7f210c8c2640 1 --2- 172.21.15.16:0/1604445251 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x55c6b9032f70 0x55c6b9035390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:54.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.383+0000 7f210c8c2640 1 --2- 172.21.15.16:0/1604445251 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x55c6b9032f70 0x55c6b9035390 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7683de57bce70f40 server_cookie=8ea4a5a7fc5f386c in_seq=0 out_seq=0 2024-09-12T02:11:54.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.383+0000 7f20e7fff640 10 client.18555 ms_handle_connect on v2:172.21.15.184:6838/4203347941 2024-09-12T02:11:54.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.383+0000 7f20e7fff640 1 -- 172.21.15.16:0/1604445251 <== mds.1 v2:172.21.15.184:6838/4203347941 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c6b8e97170 con 0x55c6b9032f70 2024-09-12T02:11:54.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.383+0000 7f20e7fff640 10 client.18555 handle_command_reply: tid=0 2024-09-12T02:11:54.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.383+0000 7f2110eff740 1 client.18555 shutdown 2024-09-12T02:11:54.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.383+0000 7f2110eff740 1 -- 172.21.15.16:0/1604445251 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c6b8fb8370 msgr2=0x55c6b8fbc000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:54.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.384+0000 7f2110eff740 1 --2- 172.21.15.16:0/1604445251 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c6b8fb8370 0x55c6b8fbc000 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f2100007600 tx=0x7f210001b820 comp rx=0 tx=0).stop 2024-09-12T02:11:54.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.384+0000 7f2110eff740 1 -- 172.21.15.16:0/1604445251 shutdown_connections 2024-09-12T02:11:54.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.384+0000 7f2110eff740 1 --2- 172.21.15.16:0/1604445251 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x55c6b9032f70 0x55c6b9035390 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:54.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.384+0000 7f2110eff740 1 --2- 172.21.15.16:0/1604445251 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c6b8fb8370 0x55c6b8fbc000 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:54.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.384+0000 7f2110eff740 1 --2- 172.21.15.16:0/1604445251 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6b8faeb40 0x55c6b8fbbac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:54.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.384+0000 7f2110eff740 1 --2- 172.21.15.16:0/1604445251 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c6b8fad5d0 0x55c6b8fc0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:54.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.384+0000 7f2110eff740 1 -- 172.21.15.16:0/1604445251 >> 172.21.15.16:0/1604445251 conn(0x55c6b8f76310 msgr2=0x55c6b8fa8310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:54.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.384+0000 7f20e7fff640 0 client.18555 ms_handle_reset on v2:172.21.15.184:6838/4203347941 2024-09-12T02:11:54.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.384+0000 7f2110eff740 1 -- 172.21.15.16:0/1604445251 shutdown_connections 2024-09-12T02:11:54.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.384+0000 7f2110eff740 1 -- 172.21.15.16:0/1604445251 wait complete. 2024-09-12T02:11:54.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.384+0000 7f2110eff740 20 client.18555 trim_cache size 0 max 16384 2024-09-12T02:11:54.393 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T02:11:54.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.384+0000 7f210eb26640 1 -- 172.21.15.16:0/3669994777 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f20f00a2b80 msgr2=0x7f20f00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:54.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.384+0000 7f210eb26640 1 --2- 172.21.15.16:0/3669994777 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f20f00a2b80 0x7f20f00a5040 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f2100007e50 tx=0x7f2100004a40 comp rx=0 tx=0).stop 2024-09-12T02:11:54.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.384+0000 7f210eb26640 1 -- 172.21.15.16:0/3669994777 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2108100a60 msgr2=0x7f2108198440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:54.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.384+0000 7f210eb26640 1 --2- 172.21.15.16:0/3669994777 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2108100a60 0x7f2108198440 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f2104009a50 tx=0x7f2104004270 comp rx=0 tx=0).stop 2024-09-12T02:11:54.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.385+0000 7f210eb26640 1 -- 172.21.15.16:0/3669994777 shutdown_connections 2024-09-12T02:11:54.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.385+0000 7f210eb26640 1 --2- 172.21.15.16:0/3669994777 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f20f00a2b80 0x7f20f00a5040 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:54.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.385+0000 7f210eb26640 1 --2- 172.21.15.16:0/3669994777 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2108106a60 0x7f210819cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:54.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.385+0000 7f210eb26640 1 --2- 172.21.15.16:0/3669994777 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2108101400 0x7f2108198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:54.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.385+0000 7f210eb26640 1 --2- 172.21.15.16:0/3669994777 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2108100a60 0x7f2108198440 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:54.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.385+0000 7f210eb26640 1 -- 172.21.15.16:0/3669994777 >> 172.21.15.16:0/3669994777 conn(0x7f21080fc780 msgr2=0x7f210806aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:54.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.385+0000 7f210eb26640 1 -- 172.21.15.16:0/3669994777 shutdown_connections 2024-09-12T02:11:54.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:54.385+0000 7f210eb26640 1 -- 172.21.15.16:0/3669994777 wait complete. 2024-09-12T02:11:54.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:11:54 smithi033 ceph-mon[35332]: pgmap v634: 97 pgs: 97 active+clean; 1.2 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 7.6 KiB/s wr, 12 op/s 2024-09-12T02:11:54.679 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:11:54 smithi184 ceph-mon[35593]: pgmap v634: 97 pgs: 97 active+clean; 1.2 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 7.6 KiB/s wr, 12 op/s 2024-09-12T02:11:54.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:11:54 smithi016 ceph-mon[30947]: pgmap v634: 97 pgs: 97 active+clean; 1.2 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 7.6 KiB/s wr, 12 op/s 2024-09-12T02:11:55.056 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-12T02:11:55.318 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:11:55.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.803+0000 7f01b3f70640 1 -- 172.21.15.16:0/3351589708 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01ac1037a0 msgr2=0x7f01ac10bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:55.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.803+0000 7f01b3f70640 1 --2- 172.21.15.16:0/3351589708 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01ac1037a0 0x7f01ac10bc00 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f019c009900 tx=0x7f019c02a410 comp rx=0 tx=0).stop 2024-09-12T02:11:55.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.803+0000 7f01b3f70640 1 -- 172.21.15.16:0/3351589708 shutdown_connections 2024-09-12T02:11:55.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.803+0000 7f01b3f70640 1 --2- 172.21.15.16:0/3351589708 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f01ac10c2f0 0x7f01ac10e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:55.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.803+0000 7f01b3f70640 1 --2- 172.21.15.16:0/3351589708 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01ac1037a0 0x7f01ac10bc00 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:55.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.803+0000 7f01b3f70640 1 --2- 172.21.15.16:0/3351589708 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01ac102df0 0x7f01ac1031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:55.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.803+0000 7f01b3f70640 1 -- 172.21.15.16:0/3351589708 >> 172.21.15.16:0/3351589708 conn(0x7f01ac0fc7d0 msgr2=0x7f01ac0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:55.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.803+0000 7f01b3f70640 1 -- 172.21.15.16:0/3351589708 shutdown_connections 2024-09-12T02:11:55.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.803+0000 7f01b3f70640 1 -- 172.21.15.16:0/3351589708 wait complete. 2024-09-12T02:11:55.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.803+0000 7f01b3f70640 1 Processor -- start 2024-09-12T02:11:55.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.803+0000 7f01b3f70640 1 -- start start 2024-09-12T02:11:55.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.804+0000 7f01b3f70640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01ac102df0 0x7f01ac198630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:55.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.804+0000 7f01b3f70640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01ac1037a0 0x7f01ac198b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:55.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.804+0000 7f01b3f70640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f01ac10c2f0 0x7f01ac19cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:55.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.804+0000 7f01b3f70640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f01ac1120b0 con 0x7f01ac102df0 2024-09-12T02:11:55.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.804+0000 7f01b3f70640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f01ac111f30 con 0x7f01ac1037a0 2024-09-12T02:11:55.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.804+0000 7f01b3f70640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f01ac112230 con 0x7f01ac10c2f0 2024-09-12T02:11:55.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.804+0000 7f01b1d0c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01ac102df0 0x7f01ac198630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:55.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.804+0000 7f01b1d0c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01ac102df0 0x7f01ac198630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54048/0 (socket says 172.21.15.16:54048) 2024-09-12T02:11:55.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.804+0000 7f01b1d0c640 1 -- 172.21.15.16:0/1958679822 learned_addr learned my addr 172.21.15.16:0/1958679822 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:11:55.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.804+0000 7f01b150b640 1 --2- 172.21.15.16:0/1958679822 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01ac1037a0 0x7f01ac198b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:55.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.804+0000 7f01b250d640 1 --2- 172.21.15.16:0/1958679822 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f01ac10c2f0 0x7f01ac19cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:55.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.804+0000 7f01b1d0c640 1 -- 172.21.15.16:0/1958679822 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f01ac10c2f0 msgr2=0x7f01ac19cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:55.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.804+0000 7f01b1d0c640 1 --2- 172.21.15.16:0/1958679822 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f01ac10c2f0 0x7f01ac19cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:55.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.804+0000 7f01b1d0c640 1 -- 172.21.15.16:0/1958679822 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01ac1037a0 msgr2=0x7f01ac198b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:55.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.804+0000 7f01b1d0c640 1 --2- 172.21.15.16:0/1958679822 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01ac1037a0 0x7f01ac198b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:55.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.804+0000 7f01b1d0c640 1 -- 172.21.15.16:0/1958679822 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01ac19d5e0 con 0x7f01ac102df0 2024-09-12T02:11:55.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.804+0000 7f01b250d640 1 --2- 172.21.15.16:0/1958679822 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f01ac10c2f0 0x7f01ac19cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:11:55.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.804+0000 7f01b1d0c640 1 --2- 172.21.15.16:0/1958679822 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01ac102df0 0x7f01ac198630 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f01a800e9f0 tx=0x7f01a800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:55.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.804+0000 7f01a2ffd640 1 -- 172.21.15.16:0/1958679822 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01a800cd80 con 0x7f01ac102df0 2024-09-12T02:11:55.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.805+0000 7f01b3f70640 1 -- 172.21.15.16:0/1958679822 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01ac19d8d0 con 0x7f01ac102df0 2024-09-12T02:11:55.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.805+0000 7f01a2ffd640 1 -- 172.21.15.16:0/1958679822 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f01a8010910 con 0x7f01ac102df0 2024-09-12T02:11:55.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.805+0000 7f01a2ffd640 1 -- 172.21.15.16:0/1958679822 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01a8018990 con 0x7f01ac102df0 2024-09-12T02:11:55.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.805+0000 7f01b3f70640 1 -- 172.21.15.16:0/1958679822 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01ac1a51b0 con 0x7f01ac102df0 2024-09-12T02:11:55.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.807+0000 7f01a2ffd640 1 -- 172.21.15.16:0/1958679822 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f01a8010ac0 con 0x7f01ac102df0 2024-09-12T02:11:55.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.807+0000 7f01b6349740 1 Processor -- start 2024-09-12T02:11:55.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.807+0000 7f01b6349740 1 -- start start 2024-09-12T02:11:55.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.808+0000 7f01b6349740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562145a9fb40 0x562145a9e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:55.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.808+0000 7f01b6349740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562145a9e5d0 0x562145a9ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:55.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.808+0000 7f01b6349740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562145aad780 0x562145aafb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:55.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.808+0000 7f01b6349740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x562145a042f0 con 0x562145a9e5d0 2024-09-12T02:11:55.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.808+0000 7f01b6349740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5621459abcf0 con 0x562145a9fb40 2024-09-12T02:11:55.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.808+0000 7f01b6349740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x562145a002b0 con 0x562145aad780 2024-09-12T02:11:55.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.808+0000 7f01b1d0c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562145a9e5d0 0x562145a9ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:55.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.808+0000 7f01b1d0c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562145a9e5d0 0x562145a9ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54058/0 (socket says 172.21.15.16:54058) 2024-09-12T02:11:55.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.808+0000 7f01b1d0c640 1 -- 172.21.15.16:0/2267712827 learned_addr learned my addr 172.21.15.16:0/2267712827 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:11:55.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.808+0000 7f01a2ffd640 1 --2- 172.21.15.16:0/1958679822 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f01980a2b80 0x7f01980a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:55.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.808+0000 7f01b150b640 1 --2- 172.21.15.16:0/2267712827 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562145aad780 0x562145aafb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:55.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.808+0000 7f01b250d640 1 --2- 172.21.15.16:0/2267712827 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562145a9fb40 0x562145a9e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:55.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.808+0000 7f01b250d640 1 --2- 172.21.15.16:0/1958679822 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f01980a2b80 0x7f01980a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:55.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.808+0000 7f01a2ffd640 1 -- 172.21.15.16:0/1958679822 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f01a80cf420 con 0x7f01ac102df0 2024-09-12T02:11:55.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.808+0000 7f01b1d0c640 1 -- 172.21.15.16:0/2267712827 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562145aad780 msgr2=0x562145aafb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:55.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.808+0000 7f01b1d0c640 1 --2- 172.21.15.16:0/2267712827 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562145aad780 0x562145aafb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:55.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.808+0000 7f01b1d0c640 1 -- 172.21.15.16:0/2267712827 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562145a9fb40 msgr2=0x562145a9e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:55.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.808+0000 7f01b1d0c640 1 --2- 172.21.15.16:0/2267712827 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562145a9fb40 0x562145a9e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:55.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.808+0000 7f01b1d0c640 1 -- 172.21.15.16:0/2267712827 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5621459903a0 con 0x562145a9e5d0 2024-09-12T02:11:55.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.809+0000 7f01b1d0c640 1 --2- 172.21.15.16:0/2267712827 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562145a9e5d0 0x562145a9ea10 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f01a8098650 tx=0x7f01a8018420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c00f1fc6c1a5185f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:55.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.809+0000 7f01b250d640 1 --2- 172.21.15.16:0/1958679822 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f01980a2b80 0x7f01980a5040 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f01a4005fd0 tx=0x7f01a4005c80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:55.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.809+0000 7f0197fff640 1 -- 172.21.15.16:0/2267712827 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01a809bc80 con 0x562145a9e5d0 2024-09-12T02:11:55.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.809+0000 7f0197fff640 1 -- 172.21.15.16:0/2267712827 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f01a80a0030 con 0x562145a9e5d0 2024-09-12T02:11:55.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.809+0000 7f0197fff640 1 -- 172.21.15.16:0/2267712827 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01a80a4460 con 0x562145a9e5d0 2024-09-12T02:11:55.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.809+0000 7f01b6349740 1 -- 172.21.15.16:0/2267712827 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562145a9e5d0 msgr2=0x562145a9ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:55.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.809+0000 7f01b6349740 1 --2- 172.21.15.16:0/2267712827 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562145a9e5d0 0x562145a9ea10 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f01a8098650 tx=0x7f01a8018420 comp rx=0 tx=0).stop 2024-09-12T02:11:55.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.809+0000 7f01b6349740 1 -- 172.21.15.16:0/2267712827 shutdown_connections 2024-09-12T02:11:55.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.809+0000 7f01b6349740 1 --2- 172.21.15.16:0/2267712827 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562145aad780 0x562145aafb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:55.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.809+0000 7f01b6349740 1 --2- 172.21.15.16:0/2267712827 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562145a9e5d0 0x562145a9ea10 unknown :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:55.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.809+0000 7f01b6349740 1 --2- 172.21.15.16:0/2267712827 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562145a9fb40 0x562145a9e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:55.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.809+0000 7f01b6349740 1 -- 172.21.15.16:0/2267712827 >> 172.21.15.16:0/2267712827 conn(0x562145a67310 msgr2=0x562145a9ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:55.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.810+0000 7f01b6349740 1 -- 172.21.15.16:0/2267712827 shutdown_connections 2024-09-12T02:11:55.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.810+0000 7f01b6349740 1 -- 172.21.15.16:0/2267712827 wait complete. 2024-09-12T02:11:55.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.810+0000 7f01b6349740 1 -- start start 2024-09-12T02:11:55.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.810+0000 7f01b6349740 1 Processor -- start 2024-09-12T02:11:55.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.810+0000 7f01b6349740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562145a9e5d0 0x562145aad1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:55.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.810+0000 7f01b6349740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562145a9fb40 0x562145aaa490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:55.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.810+0000 7f01b6349740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562145aad780 0x562145aaac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:55.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.810+0000 7f01b6349740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x562145ab0420 con 0x562145a9e5d0 2024-09-12T02:11:55.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.810+0000 7f01b6349740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5621459bf060 con 0x562145aad780 2024-09-12T02:11:55.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.810+0000 7f01b6349740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x562145ab05a0 con 0x562145a9fb40 2024-09-12T02:11:55.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.810+0000 7f01b6349740 10 client.0 _ll_register_callbacks cb 0x5621457128c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:11:55.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.811+0000 7f01b1d0c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562145a9e5d0 0x562145aad1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:55.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.811+0000 7f01b1d0c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562145a9e5d0 0x562145aad1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54062/0 (socket says 172.21.15.16:54062) 2024-09-12T02:11:55.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.811+0000 7f01b1d0c640 1 -- 172.21.15.16:0/2199729100 learned_addr learned my addr 172.21.15.16:0/2199729100 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:11:55.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.811+0000 7f01b250d640 1 --2- 172.21.15.16:0/2199729100 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562145aad780 0x562145aaac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:55.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.811+0000 7f01b150b640 1 --2- 172.21.15.16:0/2199729100 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562145a9fb40 0x562145aaa490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:55.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.811+0000 7f01b1d0c640 1 -- 172.21.15.16:0/2199729100 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562145a9fb40 msgr2=0x562145aaa490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:55.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.811+0000 7f01b1d0c640 1 --2- 172.21.15.16:0/2199729100 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562145a9fb40 0x562145aaa490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:55.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.811+0000 7f01b1d0c640 1 -- 172.21.15.16:0/2199729100 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562145aad780 msgr2=0x562145aaac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:55.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.811+0000 7f01b1d0c640 1 --2- 172.21.15.16:0/2199729100 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562145aad780 0x562145aaac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:55.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.811+0000 7f01b1d0c640 1 -- 172.21.15.16:0/2199729100 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5621457753c0 con 0x562145a9e5d0 2024-09-12T02:11:55.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.811+0000 7f01b250d640 1 --2- 172.21.15.16:0/2199729100 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562145aad780 0x562145aaac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:11:55.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.811+0000 7f01b150b640 1 --2- 172.21.15.16:0/2199729100 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562145a9fb40 0x562145aaa490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:11:55.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.811+0000 7f01b1d0c640 1 --2- 172.21.15.16:0/2199729100 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562145a9e5d0 0x562145aad1e0 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f01a80a43c0 tx=0x7f01a80983e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97b6b9e75bf59a95 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:55.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.811+0000 7f01957fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:11:55.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.811+0000 7f01957fa640 1 -- 172.21.15.16:0/2199729100 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01a809bc80 con 0x562145a9e5d0 2024-09-12T02:11:55.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.811+0000 7f01957fa640 1 -- 172.21.15.16:0/2199729100 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f01a80a4670 con 0x562145a9e5d0 2024-09-12T02:11:55.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.811+0000 7f01b6349740 1 -- 172.21.15.16:0/2199729100 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f01a80a0030 con 0x562145a9e5d0 2024-09-12T02:11:55.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.811+0000 7f01957fa640 1 -- 172.21.15.16:0/2199729100 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f01a8018b30 con 0x562145a9e5d0 2024-09-12T02:11:55.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.811+0000 7f01b6349740 10 client.18567 fetch_fsmap learned FSMap version 25 2024-09-12T02:11:55.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.811+0000 7f01b6349740 1 -- 172.21.15.16:0/2199729100 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x562145ab0420 con 0x562145a9e5d0 2024-09-12T02:11:55.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.812+0000 7f01957fa640 1 -- 172.21.15.16:0/2199729100 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f01a80a0030 con 0x562145a9e5d0 2024-09-12T02:11:55.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.812+0000 7f01957fa640 1 -- 172.21.15.16:0/2199729100 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f01a809abd0 con 0x562145a9e5d0 2024-09-12T02:11:55.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.812+0000 7f01b6349740 10 client.18567 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:11:55.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.812+0000 7f01b6349740 10 client.18567 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:11:55.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.812+0000 7f01b6349740 20 client.18567 populate_metadata read hostname 'smithi016' 2024-09-12T02:11:55.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.812+0000 7f01b6349740 1 --2- 172.21.15.16:0/2199729100 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x562145abd0c0 0x562145abf4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:55.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.812+0000 7f01b6349740 4 client.18567 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:11:55.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.812+0000 7f01b6349740 1 -- 172.21.15.16:0/2199729100 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562145a11860 con 0x562145abd0c0 2024-09-12T02:11:55.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.812+0000 7f01b150b640 1 --2- 172.21.15.16:0/2199729100 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x562145abd0c0 0x562145abf4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:55.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.813+0000 7f01b150b640 1 --2- 172.21.15.16:0/2199729100 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x562145abd0c0 0x562145abf4b0 crc :-1 s=READY pgs=296 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=472eccd9865bd2fc server_cookie=e6d5f4d03622f30a in_seq=0 out_seq=0 2024-09-12T02:11:55.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.813+0000 7f01957fa640 10 client.18567 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:11:55.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.813+0000 7f01957fa640 1 -- 172.21.15.16:0/2199729100 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562145a11860 con 0x562145abd0c0 2024-09-12T02:11:55.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.813+0000 7f01957fa640 10 client.18567 handle_command_reply: tid=0 2024-09-12T02:11:55.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.813+0000 7f01b6349740 1 client.18567 shutdown 2024-09-12T02:11:55.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.813+0000 7f01b6349740 1 -- 172.21.15.16:0/2199729100 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562145a9e5d0 msgr2=0x562145aad1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:55.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.813+0000 7f01b6349740 1 --2- 172.21.15.16:0/2199729100 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562145a9e5d0 0x562145aad1e0 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f01a80a43c0 tx=0x7f01a80983e0 comp rx=0 tx=0).stop 2024-09-12T02:11:55.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.814+0000 7f01b6349740 1 -- 172.21.15.16:0/2199729100 shutdown_connections 2024-09-12T02:11:55.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.814+0000 7f01b6349740 1 --2- 172.21.15.16:0/2199729100 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x562145abd0c0 0x562145abf4b0 crc :-1 s=READY pgs=296 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:55.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.814+0000 7f01b6349740 1 --2- 172.21.15.16:0/2199729100 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562145aad780 0x562145aaac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:55.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.814+0000 7f01b6349740 1 --2- 172.21.15.16:0/2199729100 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562145a9fb40 0x562145aaa490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:55.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.814+0000 7f01b6349740 1 --2- 172.21.15.16:0/2199729100 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562145a9e5d0 0x562145aad1e0 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:55.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.814+0000 7f01957fa640 0 client.18567 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:11:55.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.814+0000 7f01b6349740 1 -- 172.21.15.16:0/2199729100 >> 172.21.15.16:0/2199729100 conn(0x562145a67310 msgr2=0x562145aa0990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:55.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.814+0000 7f01b6349740 1 -- 172.21.15.16:0/2199729100 shutdown_connections 2024-09-12T02:11:55.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.814+0000 7f01b6349740 1 -- 172.21.15.16:0/2199729100 wait complete. 2024-09-12T02:11:55.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.814+0000 7f01b6349740 20 client.18567 trim_cache size 0 max 16384 2024-09-12T02:11:55.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.827+0000 7f01b6349740 1 Processor -- start 2024-09-12T02:11:55.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.827+0000 7f01b6349740 1 -- start start 2024-09-12T02:11:55.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.827+0000 7f01b6349740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562145a9e5d0 0x562145a9ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:55.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.827+0000 7f01b6349740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562145a9fb40 0x562145ac4c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:55.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.827+0000 7f01b6349740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562145aad780 0x562145ac5160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:55.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.827+0000 7f01b6349740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f01ac1adc10 con 0x562145a9fb40 2024-09-12T02:11:55.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.827+0000 7f01b6349740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f01ac1ab4e0 con 0x562145aad780 2024-09-12T02:11:55.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.827+0000 7f01b6349740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f01ac1a9330 con 0x562145a9e5d0 2024-09-12T02:11:55.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.827+0000 7f01b150b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562145a9fb40 0x562145ac4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:55.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.827+0000 7f01b150b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562145a9fb40 0x562145ac4c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54074/0 (socket says 172.21.15.16:54074) 2024-09-12T02:11:55.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.827+0000 7f01b150b640 1 -- 172.21.15.16:0/3798503348 learned_addr learned my addr 172.21.15.16:0/3798503348 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:11:55.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.827+0000 7f01b250d640 1 --2- 172.21.15.16:0/3798503348 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562145aad780 0x562145ac5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:55.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.827+0000 7f01b150b640 1 -- 172.21.15.16:0/3798503348 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562145a9e5d0 msgr2=0x562145a9ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:55.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.827+0000 7f01b150b640 1 --2- 172.21.15.16:0/3798503348 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562145a9e5d0 0x562145a9ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:55.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.827+0000 7f01b150b640 1 -- 172.21.15.16:0/3798503348 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562145aad780 msgr2=0x562145ac5160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:55.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.827+0000 7f01b150b640 1 --2- 172.21.15.16:0/3798503348 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562145aad780 0x562145ac5160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:55.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.827+0000 7f01b150b640 1 -- 172.21.15.16:0/3798503348 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562145ab96c0 con 0x562145a9fb40 2024-09-12T02:11:55.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.827+0000 7f01b250d640 1 --2- 172.21.15.16:0/3798503348 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562145aad780 0x562145ac5160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:11:55.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.827+0000 7f01b150b640 1 --2- 172.21.15.16:0/3798503348 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562145a9fb40 0x562145ac4c20 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f019c009230 tx=0x7f019c0058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15be0328048ab984 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:55.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.828+0000 7f01977fe640 1 -- 172.21.15.16:0/3798503348 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f019c04a070 con 0x562145a9fb40 2024-09-12T02:11:55.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.828+0000 7f01977fe640 1 -- 172.21.15.16:0/3798503348 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f019c005520 con 0x562145a9fb40 2024-09-12T02:11:55.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.828+0000 7f01977fe640 1 -- 172.21.15.16:0/3798503348 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f019c035040 con 0x562145a9fb40 2024-09-12T02:11:55.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.828+0000 7f01b6349740 1 -- 172.21.15.16:0/3798503348 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562145a9fb40 msgr2=0x562145ac4c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:55.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.828+0000 7f01b6349740 1 --2- 172.21.15.16:0/3798503348 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562145a9fb40 0x562145ac4c20 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f019c009230 tx=0x7f019c0058d0 comp rx=0 tx=0).stop 2024-09-12T02:11:55.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.828+0000 7f01b6349740 1 -- 172.21.15.16:0/3798503348 shutdown_connections 2024-09-12T02:11:55.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.828+0000 7f01b6349740 1 --2- 172.21.15.16:0/3798503348 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562145aad780 0x562145ac5160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:55.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.828+0000 7f01b6349740 1 --2- 172.21.15.16:0/3798503348 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562145a9fb40 0x562145ac4c20 unknown :-1 s=CLOSED pgs=1425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:55.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.828+0000 7f01b6349740 1 --2- 172.21.15.16:0/3798503348 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562145a9e5d0 0x562145a9ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:55.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.828+0000 7f01b6349740 1 -- 172.21.15.16:0/3798503348 >> 172.21.15.16:0/3798503348 conn(0x562145a67310 msgr2=0x562145aa9580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:55.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.829+0000 7f01b6349740 1 -- 172.21.15.16:0/3798503348 shutdown_connections 2024-09-12T02:11:55.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.829+0000 7f01b6349740 1 -- 172.21.15.16:0/3798503348 wait complete. 2024-09-12T02:11:55.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.829+0000 7f01b6349740 1 -- start start 2024-09-12T02:11:55.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.829+0000 7f01b6349740 1 Processor -- start 2024-09-12T02:11:55.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.829+0000 7f01b6349740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562145a9e5d0 0x562145ab1d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:55.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.829+0000 7f01b6349740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562145a9fb40 0x562145ab22d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:55.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.829+0000 7f01b6349740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562145aad780 0x562145b26bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:55.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.829+0000 7f01b6349740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f01ac1a99a0 con 0x562145aad780 2024-09-12T02:11:55.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.829+0000 7f01b6349740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f01ac1aae10 con 0x562145a9e5d0 2024-09-12T02:11:55.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.829+0000 7f01b6349740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f01ac1aa070 con 0x562145a9fb40 2024-09-12T02:11:55.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.829+0000 7f01b6349740 10 client.0 _ll_register_callbacks cb 0x7f01a400acc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:11:55.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.830+0000 7f01b250d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562145aad780 0x562145b26bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:55.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.830+0000 7f01b250d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562145aad780 0x562145b26bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54084/0 (socket says 172.21.15.16:54084) 2024-09-12T02:11:55.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.830+0000 7f01b250d640 1 -- 172.21.15.16:0/3892945224 learned_addr learned my addr 172.21.15.16:0/3892945224 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:11:55.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.830+0000 7f01b1d0c640 1 --2- 172.21.15.16:0/3892945224 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562145a9e5d0 0x562145ab1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:55.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.830+0000 7f01b150b640 1 --2- 172.21.15.16:0/3892945224 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562145a9fb40 0x562145ab22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:55.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.830+0000 7f01b250d640 1 -- 172.21.15.16:0/3892945224 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562145a9fb40 msgr2=0x562145ab22d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:55.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.830+0000 7f01b250d640 1 --2- 172.21.15.16:0/3892945224 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562145a9fb40 0x562145ab22d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:55.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.830+0000 7f01b250d640 1 -- 172.21.15.16:0/3892945224 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562145a9e5d0 msgr2=0x562145ab1d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:55.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.830+0000 7f01b250d640 1 --2- 172.21.15.16:0/3892945224 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562145a9e5d0 0x562145ab1d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:55.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.830+0000 7f01b250d640 1 -- 172.21.15.16:0/3892945224 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562145b27230 con 0x562145aad780 2024-09-12T02:11:55.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.830+0000 7f01b150b640 1 --2- 172.21.15.16:0/3892945224 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562145a9fb40 0x562145ab22d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:11:55.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.830+0000 7f01b1d0c640 1 --2- 172.21.15.16:0/3892945224 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562145a9e5d0 0x562145ab1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:11:55.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.830+0000 7f01b250d640 1 --2- 172.21.15.16:0/3892945224 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562145aad780 0x562145b26bf0 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f01a400a3a0 tx=0x7f01a4004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5c252ca9a8ce960 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:55.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.830+0000 7f01957fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:11:55.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.830+0000 7f01957fa640 1 -- 172.21.15.16:0/3892945224 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01a402c060 con 0x562145aad780 2024-09-12T02:11:55.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.830+0000 7f01957fa640 1 -- 172.21.15.16:0/3892945224 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f01a401dd50 con 0x562145aad780 2024-09-12T02:11:55.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.830+0000 7f01b6349740 1 -- 172.21.15.16:0/3892945224 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f019c005520 con 0x562145aad780 2024-09-12T02:11:55.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.830+0000 7f01957fa640 1 -- 172.21.15.16:0/3892945224 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f01a4027410 con 0x562145aad780 2024-09-12T02:11:55.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.831+0000 7f01b6349740 10 client.18573 fetch_fsmap learned FSMap version 25 2024-09-12T02:11:55.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.831+0000 7f01b6349740 1 -- 172.21.15.16:0/3892945224 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x562145b28550 con 0x562145aad780 2024-09-12T02:11:55.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.831+0000 7f01957fa640 1 -- 172.21.15.16:0/3892945224 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f019c005520 con 0x562145aad780 2024-09-12T02:11:55.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.831+0000 7f01957fa640 1 -- 172.21.15.16:0/3892945224 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f01a4031070 con 0x562145aad780 2024-09-12T02:11:55.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.831+0000 7f01b6349740 10 client.18573 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:11:55.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.831+0000 7f01b6349740 10 client.18573 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:11:55.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.831+0000 7f01b6349740 20 client.18573 populate_metadata read hostname 'smithi016' 2024-09-12T02:11:55.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.831+0000 7f01b6349740 1 --2- 172.21.15.16:0/3892945224 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x562145b28cd0 0x562145b2b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:55.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.831+0000 7f01b6349740 4 client.18573 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T02:11:55.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.831+0000 7f01b6349740 1 -- 172.21.15.16:0/3892945224 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562145a0bd70 con 0x562145b28cd0 2024-09-12T02:11:55.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.831+0000 7f01b1d0c640 1 --2- 172.21.15.16:0/3892945224 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x562145b28cd0 0x562145b2b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:55.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.832+0000 7f01b1d0c640 1 --2- 172.21.15.16:0/3892945224 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x562145b28cd0 0x562145b2b0f0 crc :-1 s=READY pgs=297 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c0af7d437fdbf58 server_cookie=eee6302e59358021 in_seq=0 out_seq=0 2024-09-12T02:11:55.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.832+0000 7f01957fa640 10 client.18573 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:11:55.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.832+0000 7f01957fa640 1 -- 172.21.15.16:0/3892945224 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562145a0bd70 con 0x562145b28cd0 2024-09-12T02:11:55.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.832+0000 7f01957fa640 10 client.18573 handle_command_reply: tid=0 2024-09-12T02:11:55.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.832+0000 7f01b6349740 1 client.18573 shutdown 2024-09-12T02:11:55.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.832+0000 7f01b6349740 1 -- 172.21.15.16:0/3892945224 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562145aad780 msgr2=0x562145b26bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:55.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.832+0000 7f01b6349740 1 --2- 172.21.15.16:0/3892945224 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562145aad780 0x562145b26bf0 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f01a400a3a0 tx=0x7f01a4004ab0 comp rx=0 tx=0).stop 2024-09-12T02:11:55.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.832+0000 7f01b6349740 1 -- 172.21.15.16:0/3892945224 shutdown_connections 2024-09-12T02:11:55.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.832+0000 7f01b6349740 1 --2- 172.21.15.16:0/3892945224 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x562145b28cd0 0x562145b2b0f0 crc :-1 s=READY pgs=297 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:55.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.832+0000 7f01b6349740 1 --2- 172.21.15.16:0/3892945224 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562145aad780 0x562145b26bf0 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:55.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.832+0000 7f01b6349740 1 --2- 172.21.15.16:0/3892945224 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562145a9fb40 0x562145ab22d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:55.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.832+0000 7f01b6349740 1 --2- 172.21.15.16:0/3892945224 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562145a9e5d0 0x562145ab1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:55.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.832+0000 7f01957fa640 0 client.18573 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:11:55.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.832+0000 7f01b6349740 1 -- 172.21.15.16:0/3892945224 >> 172.21.15.16:0/3892945224 conn(0x562145a67310 msgr2=0x562145ab86b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:55.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.832+0000 7f01b6349740 1 -- 172.21.15.16:0/3892945224 shutdown_connections 2024-09-12T02:11:55.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.832+0000 7f01b6349740 1 -- 172.21.15.16:0/3892945224 wait complete. 2024-09-12T02:11:55.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.832+0000 7f01b6349740 20 client.18573 trim_cache size 0 max 16384 2024-09-12T02:11:55.851 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T02:11:55.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.833+0000 7f01b3f70640 1 -- 172.21.15.16:0/1958679822 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f01980a2b80 msgr2=0x7f01980a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:55.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.833+0000 7f01b3f70640 1 --2- 172.21.15.16:0/1958679822 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f01980a2b80 0x7f01980a5040 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f01a4005fd0 tx=0x7f01a4005c80 comp rx=0 tx=0).stop 2024-09-12T02:11:55.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.833+0000 7f01b3f70640 1 -- 172.21.15.16:0/1958679822 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01ac102df0 msgr2=0x7f01ac198630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:55.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.833+0000 7f01b3f70640 1 --2- 172.21.15.16:0/1958679822 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01ac102df0 0x7f01ac198630 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f01a800e9f0 tx=0x7f01a800eeb0 comp rx=0 tx=0).stop 2024-09-12T02:11:55.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.833+0000 7f01b3f70640 1 -- 172.21.15.16:0/1958679822 shutdown_connections 2024-09-12T02:11:55.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.833+0000 7f01b3f70640 1 --2- 172.21.15.16:0/1958679822 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f01980a2b80 0x7f01980a5040 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:55.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.833+0000 7f01b3f70640 1 --2- 172.21.15.16:0/1958679822 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f01ac10c2f0 0x7f01ac19cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:55.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.833+0000 7f01b3f70640 1 --2- 172.21.15.16:0/1958679822 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01ac1037a0 0x7f01ac198b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:55.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.833+0000 7f01b3f70640 1 --2- 172.21.15.16:0/1958679822 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01ac102df0 0x7f01ac198630 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:55.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.833+0000 7f01b3f70640 1 -- 172.21.15.16:0/1958679822 >> 172.21.15.16:0/1958679822 conn(0x7f01ac0fc7d0 msgr2=0x7f01ac0fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:55.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.833+0000 7f01b3f70640 1 -- 172.21.15.16:0/1958679822 shutdown_connections 2024-09-12T02:11:55.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:55.833+0000 7f01b3f70640 1 -- 172.21.15.16:0/1958679822 wait complete. 2024-09-12T02:11:56.591 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:11:56 smithi016 ceph-mon[30947]: pgmap v635: 97 pgs: 97 active+clean; 1.2 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 7.6 KiB/s wr, 11 op/s 2024-09-12T02:11:56.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:11:56 smithi033 ceph-mon[35332]: pgmap v635: 97 pgs: 97 active+clean; 1.2 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 7.6 KiB/s wr, 11 op/s 2024-09-12T02:11:56.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:11:56 smithi184 ceph-mon[35593]: pgmap v635: 97 pgs: 97 active+clean; 1.2 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 7.6 KiB/s wr, 11 op/s 2024-09-12T02:11:57.628 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-12T02:11:57.628 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-12T02:11:57.900 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:11:58.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.348+0000 7f7af5a32640 1 -- 172.21.15.16:0/2587409586 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7af0106f10 msgr2=0x7f7af010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:58.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.348+0000 7f7af5a32640 1 --2- 172.21.15.16:0/2587409586 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7af0106f10 0x7f7af010ea90 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f7ae8009900 tx=0x7f7ae802a410 comp rx=0 tx=0).stop 2024-09-12T02:11:58.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.348+0000 7f7af5a32640 1 -- 172.21.15.16:0/2587409586 shutdown_connections 2024-09-12T02:11:58.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.348+0000 7f7af5a32640 1 --2- 172.21.15.16:0/2587409586 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7af0106f10 0x7f7af010ea90 unknown :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:58.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.348+0000 7f7af5a32640 1 --2- 172.21.15.16:0/2587409586 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7af0102790 0x7f7af01067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:58.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.348+0000 7f7af5a32640 1 --2- 172.21.15.16:0/2587409586 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7af0101de0 0x7f7af01021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:58.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.348+0000 7f7af5a32640 1 -- 172.21.15.16:0/2587409586 >> 172.21.15.16:0/2587409586 conn(0x7f7af00fd8c0 msgr2=0x7f7af00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:58.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.348+0000 7f7af5a32640 1 -- 172.21.15.16:0/2587409586 shutdown_connections 2024-09-12T02:11:58.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.348+0000 7f7af5a32640 1 -- 172.21.15.16:0/2587409586 wait complete. 2024-09-12T02:11:58.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.349+0000 7f7af5a32640 1 Processor -- start 2024-09-12T02:11:58.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.349+0000 7f7af5a32640 1 -- start start 2024-09-12T02:11:58.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.349+0000 7f7af5a32640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7af0101de0 0x7f7af0198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:58.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.349+0000 7f7af5a32640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7af0102790 0x7f7af01989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:58.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.349+0000 7f7af5a32640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7af0106f10 0x7f7af019cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:58.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.349+0000 7f7af5a32640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7af0111f90 con 0x7f7af0106f10 2024-09-12T02:11:58.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.349+0000 7f7af5a32640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7af0111e10 con 0x7f7af0102790 2024-09-12T02:11:58.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.349+0000 7f7af5a32640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7af0112110 con 0x7f7af0101de0 2024-09-12T02:11:58.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.349+0000 7f7aef7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7af0106f10 0x7f7af019cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:58.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.349+0000 7f7aef7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7af0106f10 0x7f7af019cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54108/0 (socket says 172.21.15.16:54108) 2024-09-12T02:11:58.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.349+0000 7f7aef7fe640 1 -- 172.21.15.16:0/1172456720 learned_addr learned my addr 172.21.15.16:0/1172456720 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:11:58.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.349+0000 7f7aee7fc640 1 --2- 172.21.15.16:0/1172456720 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7af0102790 0x7f7af01989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:58.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.350+0000 7f7aeeffd640 1 --2- 172.21.15.16:0/1172456720 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7af0101de0 0x7f7af0198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:58.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.350+0000 7f7aef7fe640 1 -- 172.21.15.16:0/1172456720 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7af0101de0 msgr2=0x7f7af0198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:58.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.350+0000 7f7aef7fe640 1 --2- 172.21.15.16:0/1172456720 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7af0101de0 0x7f7af0198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:58.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.350+0000 7f7aef7fe640 1 -- 172.21.15.16:0/1172456720 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7af0102790 msgr2=0x7f7af01989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:58.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.350+0000 7f7aef7fe640 1 --2- 172.21.15.16:0/1172456720 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7af0102790 0x7f7af01989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:58.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.350+0000 7f7aef7fe640 1 -- 172.21.15.16:0/1172456720 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7af019d410 con 0x7f7af0106f10 2024-09-12T02:11:58.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.350+0000 7f7aeeffd640 1 --2- 172.21.15.16:0/1172456720 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7af0101de0 0x7f7af0198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:11:58.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.350+0000 7f7aef7fe640 1 --2- 172.21.15.16:0/1172456720 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7af0106f10 0x7f7af019cd30 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f7ae8009230 tx=0x7f7ae80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:58.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.350+0000 7f7af4a30640 1 -- 172.21.15.16:0/1172456720 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ae803f030 con 0x7f7af0106f10 2024-09-12T02:11:58.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.350+0000 7f7af5a32640 1 -- 172.21.15.16:0/1172456720 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7af019d6a0 con 0x7f7af0106f10 2024-09-12T02:11:58.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.350+0000 7f7af4a30640 1 -- 172.21.15.16:0/1172456720 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ae8003ce0 con 0x7f7af0106f10 2024-09-12T02:11:58.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.350+0000 7f7af4a30640 1 -- 172.21.15.16:0/1172456720 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ae8035070 con 0x7f7af0106f10 2024-09-12T02:11:58.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.350+0000 7f7af5a32640 1 -- 172.21.15.16:0/1172456720 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7af01a4fe0 con 0x7f7af0106f10 2024-09-12T02:11:58.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.352+0000 7f7af4a30640 1 -- 172.21.15.16:0/1172456720 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f7ae8042030 con 0x7f7af0106f10 2024-09-12T02:11:58.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.353+0000 7f7af7e0b740 1 Processor -- start 2024-09-12T02:11:58.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.353+0000 7f7af7e0b740 1 -- start start 2024-09-12T02:11:58.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.353+0000 7f7af7e0b740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561d15330f70 0x561d15331350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:58.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.353+0000 7f7af7e0b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561d15331890 0x561d1533bc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:58.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.353+0000 7f7af7e0b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561d1533c150 0x561d1533e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:58.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.353+0000 7f7af7e0b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x561d15292280 con 0x561d1533c150 2024-09-12T02:11:58.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.353+0000 7f7af7e0b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x561d15239b10 con 0x561d15331890 2024-09-12T02:11:58.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.353+0000 7f7af7e0b740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x561d1528e240 con 0x561d15330f70 2024-09-12T02:11:58.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.353+0000 7f7aee7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561d1533c150 0x561d1533e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:58.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.353+0000 7f7aee7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561d1533c150 0x561d1533e540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54112/0 (socket says 172.21.15.16:54112) 2024-09-12T02:11:58.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.353+0000 7f7aee7fc640 1 -- 172.21.15.16:0/1108285268 learned_addr learned my addr 172.21.15.16:0/1108285268 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:11:58.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.353+0000 7f7aeeffd640 1 --2- 172.21.15.16:0/1108285268 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561d15331890 0x561d1533bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:58.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.353+0000 7f7aee7fc640 1 --2- 172.21.15.16:0/1108285268 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561d15330f70 0x561d15331350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:58.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.354+0000 7f7aee7fc640 1 -- 172.21.15.16:0/1108285268 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561d15330f70 msgr2=0x561d15331350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:58.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.354+0000 7f7aee7fc640 1 --2- 172.21.15.16:0/1108285268 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561d15330f70 0x561d15331350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:58.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.354+0000 7f7af4a30640 1 --2- 172.21.15.16:0/1172456720 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f7adc0a2b80 0x7f7adc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:58.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.354+0000 7f7aee7fc640 1 -- 172.21.15.16:0/1108285268 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561d15331890 msgr2=0x561d1533bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:58.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.354+0000 7f7aee7fc640 1 --2- 172.21.15.16:0/1108285268 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561d15331890 0x561d1533bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:58.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.354+0000 7f7aee7fc640 1 -- 172.21.15.16:0/1108285268 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d152461f0 con 0x561d1533c150 2024-09-12T02:11:58.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.354+0000 7f7aee7fc640 1 --2- 172.21.15.16:0/1108285268 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561d1533c150 0x561d1533e540 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f7ad8011730 tx=0x7f7ad800a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40499bd8400af0d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:58.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.354+0000 7f7af4a30640 1 -- 172.21.15.16:0/1172456720 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f7ae80ef880 con 0x7f7af0106f10 2024-09-12T02:11:58.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.354+0000 7f7aef7fe640 1 --2- 172.21.15.16:0/1172456720 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f7adc0a2b80 0x7f7adc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:58.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.354+0000 7f7ae5ffb640 1 -- 172.21.15.16:0/1108285268 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ad8004020 con 0x561d1533c150 2024-09-12T02:11:58.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.354+0000 7f7ae5ffb640 1 -- 172.21.15.16:0/1108285268 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ad8005500 con 0x561d1533c150 2024-09-12T02:11:58.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.354+0000 7f7ae5ffb640 1 -- 172.21.15.16:0/1108285268 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ad8004c30 con 0x561d1533c150 2024-09-12T02:11:58.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.355+0000 7f7af7e0b740 1 -- 172.21.15.16:0/1108285268 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561d1533c150 msgr2=0x561d1533e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:58.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.355+0000 7f7aef7fe640 1 --2- 172.21.15.16:0/1172456720 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f7adc0a2b80 0x7f7adc0a5040 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f7ae8032c00 tx=0x7f7ae802a410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:58.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.355+0000 7f7af7e0b740 1 --2- 172.21.15.16:0/1108285268 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561d1533c150 0x561d1533e540 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f7ad8011730 tx=0x7f7ad800a6a0 comp rx=0 tx=0).stop 2024-09-12T02:11:58.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.355+0000 7f7af7e0b740 1 -- 172.21.15.16:0/1108285268 shutdown_connections 2024-09-12T02:11:58.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.355+0000 7f7af7e0b740 1 --2- 172.21.15.16:0/1108285268 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561d1533c150 0x561d1533e540 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:58.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.355+0000 7f7af7e0b740 1 --2- 172.21.15.16:0/1108285268 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561d15331890 0x561d1533bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:58.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.355+0000 7f7af7e0b740 1 --2- 172.21.15.16:0/1108285268 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561d15330f70 0x561d15331350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:58.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.355+0000 7f7af7e0b740 1 -- 172.21.15.16:0/1108285268 >> 172.21.15.16:0/1108285268 conn(0x561d1532d500 msgr2=0x561d1532d8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:58.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.355+0000 7f7af7e0b740 1 -- 172.21.15.16:0/1108285268 shutdown_connections 2024-09-12T02:11:58.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.355+0000 7f7af7e0b740 1 -- 172.21.15.16:0/1108285268 wait complete. 2024-09-12T02:11:58.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.355+0000 7f7af7e0b740 1 -- start start 2024-09-12T02:11:58.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.356+0000 7f7af7e0b740 1 Processor -- start 2024-09-12T02:11:58.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.356+0000 7f7af7e0b740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561d15330f70 0x561d153390c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:58.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.356+0000 7f7af7e0b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561d15331890 0x561d15339600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:58.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.356+0000 7f7af7e0b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561d1533c150 0x561d15339b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:58.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.356+0000 7f7af7e0b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x561d1533ed20 con 0x561d1533c150 2024-09-12T02:11:58.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.356+0000 7f7af7e0b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x561d1524cf10 con 0x561d15331890 2024-09-12T02:11:58.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.356+0000 7f7af7e0b740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x561d1533eea0 con 0x561d15330f70 2024-09-12T02:11:58.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.356+0000 7f7af7e0b740 10 client.0 _ll_register_callbacks cb 0x561d15215a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:11:58.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.356+0000 7f7aee7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561d1533c150 0x561d15339b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:58.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.356+0000 7f7aee7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561d1533c150 0x561d15339b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54124/0 (socket says 172.21.15.16:54124) 2024-09-12T02:11:58.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.356+0000 7f7aee7fc640 1 -- 172.21.15.16:0/548775342 learned_addr learned my addr 172.21.15.16:0/548775342 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:11:58.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.356+0000 7f7aeeffd640 1 --2- 172.21.15.16:0/548775342 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561d15331890 0x561d15339600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:58.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.356+0000 7f7aee7fc640 1 --2- 172.21.15.16:0/548775342 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561d15330f70 0x561d153390c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:58.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.357+0000 7f7aee7fc640 1 -- 172.21.15.16:0/548775342 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561d15330f70 msgr2=0x561d153390c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:58.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.357+0000 7f7aee7fc640 1 --2- 172.21.15.16:0/548775342 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561d15330f70 0x561d153390c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:58.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.357+0000 7f7aee7fc640 1 -- 172.21.15.16:0/548775342 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561d15331890 msgr2=0x561d15339600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:58.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.357+0000 7f7aee7fc640 1 --2- 172.21.15.16:0/548775342 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561d15331890 0x561d15339600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:58.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.357+0000 7f7aee7fc640 1 -- 172.21.15.16:0/548775342 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d150036a0 con 0x561d1533c150 2024-09-12T02:11:58.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.357+0000 7f7aee7fc640 1 --2- 172.21.15.16:0/548775342 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561d1533c150 0x561d15339b40 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f7ad8004910 tx=0x7f7ad8016040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3e29139efd16e79 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:58.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.357+0000 7f7aceffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:11:58.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.357+0000 7f7aceffd640 1 -- 172.21.15.16:0/548775342 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ad80046c0 con 0x561d1533c150 2024-09-12T02:11:58.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.357+0000 7f7aceffd640 1 -- 172.21.15.16:0/548775342 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f7ad8004bc0 con 0x561d1533c150 2024-09-12T02:11:58.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.357+0000 7f7af7e0b740 1 -- 172.21.15.16:0/548775342 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ad8005500 con 0x561d1533c150 2024-09-12T02:11:58.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.357+0000 7f7aceffd640 1 -- 172.21.15.16:0/548775342 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7ad8004dc0 con 0x561d1533c150 2024-09-12T02:11:58.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.357+0000 7f7af7e0b740 10 client.18585 fetch_fsmap learned FSMap version 25 2024-09-12T02:11:58.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.357+0000 7f7af7e0b740 1 -- 172.21.15.16:0/548775342 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x561d1534a9b0 con 0x561d1533c150 2024-09-12T02:11:58.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.357+0000 7f7aceffd640 1 -- 172.21.15.16:0/548775342 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ad8005500 con 0x561d1533c150 2024-09-12T02:11:58.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.358+0000 7f7aceffd640 1 -- 172.21.15.16:0/548775342 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f7ad802e770 con 0x561d1533c150 2024-09-12T02:11:58.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.358+0000 7f7af7e0b740 10 client.18585 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:11:58.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.358+0000 7f7af7e0b740 10 client.18585 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:11:58.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.358+0000 7f7af7e0b740 20 client.18585 populate_metadata read hostname 'smithi016' 2024-09-12T02:11:58.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.358+0000 7f7af7e0b740 1 --2- 172.21.15.16:0/548775342 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x561d1534bd00 0x561d1534e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:58.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.358+0000 7f7af7e0b740 4 client.18585 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:11:58.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.358+0000 7f7af7e0b740 1 -- 172.21.15.16:0/548775342 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561d1529f7f0 con 0x561d1534bd00 2024-09-12T02:11:58.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.358+0000 7f7aeeffd640 1 --2- 172.21.15.16:0/548775342 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x561d1534bd00 0x561d1534e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:58.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.358+0000 7f7aeeffd640 1 --2- 172.21.15.16:0/548775342 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x561d1534bd00 0x561d1534e0f0 crc :-1 s=READY pgs=298 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=259560e1cdc20f88 server_cookie=2694baa1c27f420e in_seq=0 out_seq=0 2024-09-12T02:11:58.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.358+0000 7f7aceffd640 10 client.18585 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:11:58.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.359+0000 7f7aceffd640 1 -- 172.21.15.16:0/548775342 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561d1529f7f0 con 0x561d1534bd00 2024-09-12T02:11:58.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.359+0000 7f7aceffd640 10 client.18585 handle_command_reply: tid=0 2024-09-12T02:11:58.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.359+0000 7f7af7e0b740 1 client.18585 shutdown 2024-09-12T02:11:58.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.359+0000 7f7af7e0b740 1 -- 172.21.15.16:0/548775342 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561d1533c150 msgr2=0x561d15339b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:58.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.359+0000 7f7af7e0b740 1 --2- 172.21.15.16:0/548775342 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561d1533c150 0x561d15339b40 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f7ad8004910 tx=0x7f7ad8016040 comp rx=0 tx=0).stop 2024-09-12T02:11:58.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.360+0000 7f7af7e0b740 1 -- 172.21.15.16:0/548775342 shutdown_connections 2024-09-12T02:11:58.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.360+0000 7f7af7e0b740 1 --2- 172.21.15.16:0/548775342 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x561d1534bd00 0x561d1534e0f0 crc :-1 s=READY pgs=298 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:58.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.360+0000 7f7aceffd640 0 client.18585 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:11:58.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.360+0000 7f7af7e0b740 1 --2- 172.21.15.16:0/548775342 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561d1533c150 0x561d15339b40 unknown :-1 s=CLOSED pgs=1430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:58.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.360+0000 7f7af7e0b740 1 --2- 172.21.15.16:0/548775342 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561d15331890 0x561d15339600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:58.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.360+0000 7f7af7e0b740 1 --2- 172.21.15.16:0/548775342 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561d15330f70 0x561d153390c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:58.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.360+0000 7f7af7e0b740 1 -- 172.21.15.16:0/548775342 >> 172.21.15.16:0/548775342 conn(0x561d1532d500 msgr2=0x561d15333230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:58.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.360+0000 7f7af7e0b740 1 -- 172.21.15.16:0/548775342 shutdown_connections 2024-09-12T02:11:58.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.360+0000 7f7af7e0b740 1 -- 172.21.15.16:0/548775342 wait complete. 2024-09-12T02:11:58.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.360+0000 7f7af7e0b740 20 client.18585 trim_cache size 0 max 16384 2024-09-12T02:11:58.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.377+0000 7f7af7e0b740 1 Processor -- start 2024-09-12T02:11:58.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.377+0000 7f7af7e0b740 1 -- start start 2024-09-12T02:11:58.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.377+0000 7f7af7e0b740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561d15330f70 0x561d15328520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:58.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.377+0000 7f7af7e0b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561d15331890 0x561d153537a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:58.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.377+0000 7f7af7e0b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561d1533c150 0x561d15353ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:58.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.377+0000 7f7af7e0b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7af01adaf0 con 0x561d15331890 2024-09-12T02:11:58.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.377+0000 7f7af7e0b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7af01ab3c0 con 0x561d1533c150 2024-09-12T02:11:58.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.377+0000 7f7af7e0b740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7af01a9250 con 0x561d15330f70 2024-09-12T02:11:58.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.378+0000 7f7aeeffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561d15331890 0x561d153537a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:58.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.378+0000 7f7aeeffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561d15331890 0x561d153537a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54130/0 (socket says 172.21.15.16:54130) 2024-09-12T02:11:58.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.378+0000 7f7aeeffd640 1 -- 172.21.15.16:0/605011354 learned_addr learned my addr 172.21.15.16:0/605011354 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:11:58.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.378+0000 7f7aeeffd640 1 -- 172.21.15.16:0/605011354 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561d15330f70 msgr2=0x561d15328520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:58.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.378+0000 7f7aeeffd640 1 --2- 172.21.15.16:0/605011354 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561d15330f70 0x561d15328520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:58.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.378+0000 7f7aee7fc640 1 --2- 172.21.15.16:0/605011354 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561d1533c150 0x561d15353ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:58.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.378+0000 7f7aeeffd640 1 -- 172.21.15.16:0/605011354 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561d1533c150 msgr2=0x561d15353ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:58.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.378+0000 7f7aeeffd640 1 --2- 172.21.15.16:0/605011354 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561d1533c150 0x561d15353ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:58.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.378+0000 7f7aeeffd640 1 -- 172.21.15.16:0/605011354 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d153316c0 con 0x561d15331890 2024-09-12T02:11:58.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.378+0000 7f7aeeffd640 1 --2- 172.21.15.16:0/605011354 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561d15331890 0x561d153537a0 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x561d1533a700 tx=0x7f7ae00098d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4be425bfaf7ff5d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:58.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.378+0000 7f7ae57fa640 1 -- 172.21.15.16:0/605011354 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ae0027620 con 0x561d15331890 2024-09-12T02:11:58.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.378+0000 7f7ae57fa640 1 -- 172.21.15.16:0/605011354 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ae0022030 con 0x561d15331890 2024-09-12T02:11:58.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.378+0000 7f7ae57fa640 1 -- 172.21.15.16:0/605011354 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ae00186e0 con 0x561d15331890 2024-09-12T02:11:58.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.379+0000 7f7af7e0b740 1 -- 172.21.15.16:0/605011354 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561d15331890 msgr2=0x561d153537a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:58.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.379+0000 7f7af7e0b740 1 --2- 172.21.15.16:0/605011354 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561d15331890 0x561d153537a0 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x561d1533a700 tx=0x7f7ae00098d0 comp rx=0 tx=0).stop 2024-09-12T02:11:58.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.379+0000 7f7af7e0b740 1 -- 172.21.15.16:0/605011354 shutdown_connections 2024-09-12T02:11:58.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.379+0000 7f7af7e0b740 1 --2- 172.21.15.16:0/605011354 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561d1533c150 0x561d15353ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:58.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.379+0000 7f7af7e0b740 1 --2- 172.21.15.16:0/605011354 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561d15331890 0x561d153537a0 unknown :-1 s=CLOSED pgs=1431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:58.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.379+0000 7f7af7e0b740 1 --2- 172.21.15.16:0/605011354 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561d15330f70 0x561d15328520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:58.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.379+0000 7f7af7e0b740 1 -- 172.21.15.16:0/605011354 >> 172.21.15.16:0/605011354 conn(0x561d1532d500 msgr2=0x561d1533b030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:58.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.379+0000 7f7af7e0b740 1 -- 172.21.15.16:0/605011354 shutdown_connections 2024-09-12T02:11:58.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.379+0000 7f7af7e0b740 1 -- 172.21.15.16:0/605011354 wait complete. 2024-09-12T02:11:58.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.379+0000 7f7af7e0b740 1 -- start start 2024-09-12T02:11:58.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.379+0000 7f7af7e0b740 1 Processor -- start 2024-09-12T02:11:58.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.380+0000 7f7af7e0b740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561d15330f70 0x561d15336160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:58.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.380+0000 7f7af7e0b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561d15331890 0x561d153366a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:58.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.380+0000 7f7af7e0b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561d1533c150 0x561d15336be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:58.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.380+0000 7f7af7e0b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7af01a98a0 con 0x561d15331890 2024-09-12T02:11:58.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.380+0000 7f7af7e0b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7af01aacf0 con 0x561d1533c150 2024-09-12T02:11:58.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.380+0000 7f7af7e0b740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7af01a9f50 con 0x561d15330f70 2024-09-12T02:11:58.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.380+0000 7f7af7e0b740 10 client.0 _ll_register_callbacks cb 0x7f7ae0009770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:11:58.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.380+0000 7f7aeeffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561d15331890 0x561d153366a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:58.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.380+0000 7f7aeeffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561d15331890 0x561d153366a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54134/0 (socket says 172.21.15.16:54134) 2024-09-12T02:11:58.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.380+0000 7f7aeeffd640 1 -- 172.21.15.16:0/3252943984 learned_addr learned my addr 172.21.15.16:0/3252943984 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:11:58.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.380+0000 7f7aee7fc640 1 --2- 172.21.15.16:0/3252943984 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561d15330f70 0x561d15336160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:58.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.380+0000 7f7aee7fc640 1 --2- 172.21.15.16:0/3252943984 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561d1533c150 0x561d15336be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:58.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.380+0000 7f7aeeffd640 1 -- 172.21.15.16:0/3252943984 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561d15330f70 msgr2=0x561d15336160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:58.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.380+0000 7f7aeeffd640 1 --2- 172.21.15.16:0/3252943984 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561d15330f70 0x561d15336160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:58.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.380+0000 7f7aeeffd640 1 -- 172.21.15.16:0/3252943984 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561d1533c150 msgr2=0x561d15336be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:58.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.380+0000 7f7aeeffd640 1 --2- 172.21.15.16:0/3252943984 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561d1533c150 0x561d15336be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:58.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.380+0000 7f7aeeffd640 1 -- 172.21.15.16:0/3252943984 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d153b57c0 con 0x561d15331890 2024-09-12T02:11:58.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.380+0000 7f7aeeffd640 1 --2- 172.21.15.16:0/3252943984 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561d15331890 0x561d153366a0 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f7ae0009dc0 tx=0x7f7ae00185a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e29bed0ecb64ac45 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:58.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.380+0000 7f7aceffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:11:58.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.381+0000 7f7aceffd640 1 -- 172.21.15.16:0/3252943984 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ae0018be0 con 0x561d15331890 2024-09-12T02:11:58.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.381+0000 7f7aceffd640 1 -- 172.21.15.16:0/3252943984 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f7ae0018d80 con 0x561d15331890 2024-09-12T02:11:58.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.381+0000 7f7af7e0b740 1 -- 172.21.15.16:0/3252943984 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ae0022030 con 0x561d15331890 2024-09-12T02:11:58.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.381+0000 7f7aceffd640 1 -- 172.21.15.16:0/3252943984 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7ae0030470 con 0x561d15331890 2024-09-12T02:11:58.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.381+0000 7f7af7e0b740 10 client.18591 fetch_fsmap learned FSMap version 25 2024-09-12T02:11:58.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.381+0000 7f7af7e0b740 1 -- 172.21.15.16:0/3252943984 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x561d153b6b00 con 0x561d15331890 2024-09-12T02:11:58.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.381+0000 7f7aceffd640 1 -- 172.21.15.16:0/3252943984 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ae0022030 con 0x561d15331890 2024-09-12T02:11:58.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.381+0000 7f7aceffd640 1 -- 172.21.15.16:0/3252943984 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f7ae00276c0 con 0x561d15331890 2024-09-12T02:11:58.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.381+0000 7f7af7e0b740 10 client.18591 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:11:58.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.381+0000 7f7af7e0b740 10 client.18591 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:11:58.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.381+0000 7f7af7e0b740 20 client.18591 populate_metadata read hostname 'smithi016' 2024-09-12T02:11:58.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.381+0000 7f7af7e0b740 1 --2- 172.21.15.16:0/3252943984 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x561d1534bd00 0x561d153b9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:58.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.381+0000 7f7af7e0b740 4 client.18591 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-12T02:11:58.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.381+0000 7f7af7e0b740 1 -- 172.21.15.16:0/3252943984 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x561d15299d00 con 0x561d1534bd00 2024-09-12T02:11:58.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.382+0000 7f7aee7fc640 1 --2- 172.21.15.16:0/3252943984 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x561d1534bd00 0x561d153b9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:58.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.382+0000 7f7aceffd640 10 client.18591 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:11:58.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.382+0000 7f7aee7fc640 1 --2- 172.21.15.16:0/3252943984 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x561d1534bd00 0x561d153b9280 crc :-1 s=READY pgs=299 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f31389c6432a325a server_cookie=a0996f65f05dd5c3 in_seq=0 out_seq=0 2024-09-12T02:11:58.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.383+0000 7f7aceffd640 1 -- 172.21.15.16:0/3252943984 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x561d15299d00 con 0x561d1534bd00 2024-09-12T02:11:58.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.383+0000 7f7aceffd640 10 client.18591 handle_command_reply: tid=0 2024-09-12T02:11:58.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.383+0000 7f7af7e0b740 1 client.18591 shutdown 2024-09-12T02:11:58.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.383+0000 7f7af7e0b740 1 -- 172.21.15.16:0/3252943984 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561d15331890 msgr2=0x561d153366a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:58.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.383+0000 7f7af7e0b740 1 --2- 172.21.15.16:0/3252943984 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561d15331890 0x561d153366a0 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f7ae0009dc0 tx=0x7f7ae00185a0 comp rx=0 tx=0).stop 2024-09-12T02:11:58.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.383+0000 7f7af7e0b740 1 -- 172.21.15.16:0/3252943984 shutdown_connections 2024-09-12T02:11:58.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.383+0000 7f7af7e0b740 1 --2- 172.21.15.16:0/3252943984 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x561d1534bd00 0x561d153b9280 crc :-1 s=READY pgs=299 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:58.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.383+0000 7f7af7e0b740 1 --2- 172.21.15.16:0/3252943984 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561d1533c150 0x561d15336be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:58.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.383+0000 7f7af7e0b740 1 --2- 172.21.15.16:0/3252943984 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561d15331890 0x561d153366a0 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:58.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.383+0000 7f7af7e0b740 1 --2- 172.21.15.16:0/3252943984 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561d15330f70 0x561d15336160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:58.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.383+0000 7f7af7e0b740 1 -- 172.21.15.16:0/3252943984 >> 172.21.15.16:0/3252943984 conn(0x561d1532d500 msgr2=0x561d15339080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:58.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.383+0000 7f7aceffd640 0 client.18591 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:11:58.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.384+0000 7f7af7e0b740 1 -- 172.21.15.16:0/3252943984 shutdown_connections 2024-09-12T02:11:58.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.384+0000 7f7af7e0b740 1 -- 172.21.15.16:0/3252943984 wait complete. 2024-09-12T02:11:58.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.384+0000 7f7af7e0b740 20 client.18591 trim_cache size 0 max 16384 2024-09-12T02:11:58.402 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T02:11:58.403 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-09-12T02:11:58.403 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "8b803ff1-9d56-4e42-81ec-2b28458d0df9", 2024-09-12T02:11:58.403 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-09-12T02:11:58.403 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T02:11:58.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.384+0000 7f7af5a32640 1 -- 172.21.15.16:0/1172456720 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f7adc0a2b80 msgr2=0x7f7adc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:58.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.384+0000 7f7af5a32640 1 --2- 172.21.15.16:0/1172456720 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f7adc0a2b80 0x7f7adc0a5040 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f7ae8032c00 tx=0x7f7ae802a410 comp rx=0 tx=0).stop 2024-09-12T02:11:58.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.384+0000 7f7af5a32640 1 -- 172.21.15.16:0/1172456720 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7af0106f10 msgr2=0x7f7af019cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:58.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.384+0000 7f7af5a32640 1 --2- 172.21.15.16:0/1172456720 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7af0106f10 0x7f7af019cd30 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f7ae8009230 tx=0x7f7ae80042f0 comp rx=0 tx=0).stop 2024-09-12T02:11:58.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.384+0000 7f7af5a32640 1 -- 172.21.15.16:0/1172456720 shutdown_connections 2024-09-12T02:11:58.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.384+0000 7f7af5a32640 1 --2- 172.21.15.16:0/1172456720 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f7adc0a2b80 0x7f7adc0a5040 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:58.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.384+0000 7f7af5a32640 1 --2- 172.21.15.16:0/1172456720 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7af0106f10 0x7f7af019cd30 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:58.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.384+0000 7f7af5a32640 1 --2- 172.21.15.16:0/1172456720 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7af0102790 0x7f7af01989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:58.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.384+0000 7f7af5a32640 1 --2- 172.21.15.16:0/1172456720 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7af0101de0 0x7f7af0198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:58.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.384+0000 7f7af5a32640 1 -- 172.21.15.16:0/1172456720 >> 172.21.15.16:0/1172456720 conn(0x7f7af00fd8c0 msgr2=0x7f7af00fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:58.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.384+0000 7f7af5a32640 1 -- 172.21.15.16:0/1172456720 shutdown_connections 2024-09-12T02:11:58.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:58.385+0000 7f7af5a32640 1 -- 172.21.15.16:0/1172456720 wait complete. 2024-09-12T02:11:58.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:11:58 smithi033 ceph-mon[35332]: pgmap v636: 97 pgs: 97 active+clean; 1.2 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.9 KiB/s wr, 13 op/s 2024-09-12T02:11:58.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:11:58 smithi016 ceph-mon[30947]: pgmap v636: 97 pgs: 97 active+clean; 1.2 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.9 KiB/s wr, 13 op/s 2024-09-12T02:11:58.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:11:58 smithi184 ceph-mon[35593]: pgmap v636: 97 pgs: 97 active+clean; 1.2 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.9 KiB/s wr, 13 op/s 2024-09-12T02:11:59.107 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T02:11:59.368 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:11:59.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:11:59 smithi033 ceph-mon[35332]: scrub queued for path: / 2024-09-12T02:11:59.670 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:11:59 smithi033 ceph-mon[35332]: scrub summary: idle+waiting paths [/] 2024-09-12T02:11:59.670 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:11:59 smithi033 ceph-mon[35332]: scrub summary: active paths [/] 2024-09-12T02:11:59.670 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:11:59 smithi033 ceph-mon[35332]: scrub summary: idle+waiting paths [/] 2024-09-12T02:11:59.674 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:11:59 smithi016 ceph-mon[30947]: scrub queued for path: / 2024-09-12T02:11:59.674 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:11:59 smithi016 ceph-mon[30947]: scrub summary: idle+waiting paths [/] 2024-09-12T02:11:59.674 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:11:59 smithi016 ceph-mon[30947]: scrub summary: active paths [/] 2024-09-12T02:11:59.675 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:11:59 smithi016 ceph-mon[30947]: scrub summary: idle+waiting paths [/] 2024-09-12T02:11:59.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:11:59 smithi184 ceph-mon[35593]: scrub queued for path: / 2024-09-12T02:11:59.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:11:59 smithi184 ceph-mon[35593]: scrub summary: idle+waiting paths [/] 2024-09-12T02:11:59.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:11:59 smithi184 ceph-mon[35593]: scrub summary: active paths [/] 2024-09-12T02:11:59.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:11:59 smithi184 ceph-mon[35593]: scrub summary: idle+waiting paths [/] 2024-09-12T02:11:59.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.871+0000 7f3a7f47f640 1 -- 172.21.15.16:0/46163348 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a78106f50 msgr2=0x7f3a7810eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:59.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.871+0000 7f3a7f47f640 1 --2- 172.21.15.16:0/46163348 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a78106f50 0x7f3a7810eaf0 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f3a70009930 tx=0x7f3a7002a420 comp rx=0 tx=0).stop 2024-09-12T02:11:59.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.872+0000 7f3a7f47f640 1 -- 172.21.15.16:0/46163348 shutdown_connections 2024-09-12T02:11:59.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.872+0000 7f3a7f47f640 1 --2- 172.21.15.16:0/46163348 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a78106f50 0x7f3a7810eaf0 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:59.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.872+0000 7f3a7f47f640 1 --2- 172.21.15.16:0/46163348 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a78100d00 0x7f3a78106840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:59.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.872+0000 7f3a7f47f640 1 --2- 172.21.15.16:0/46163348 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3a78100350 0x7f3a78100730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:59.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.872+0000 7f3a7f47f640 1 -- 172.21.15.16:0/46163348 >> 172.21.15.16:0/46163348 conn(0x7f3a78077210 msgr2=0x7f3a780fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:59.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.872+0000 7f3a7f47f640 1 -- 172.21.15.16:0/46163348 shutdown_connections 2024-09-12T02:11:59.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.872+0000 7f3a7f47f640 1 -- 172.21.15.16:0/46163348 wait complete. 2024-09-12T02:11:59.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.872+0000 7f3a7f47f640 1 Processor -- start 2024-09-12T02:11:59.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.872+0000 7f3a7f47f640 1 -- start start 2024-09-12T02:11:59.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.872+0000 7f3a7f47f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a78100350 0x7f3a781984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:59.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.872+0000 7f3a7f47f640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3a78100d00 0x7f3a781989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:59.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.872+0000 7f3a7f47f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a78106f50 0x7f3a7819cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:59.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.873+0000 7f3a7f47f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3a78112000 con 0x7f3a78100350 2024-09-12T02:11:59.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.873+0000 7f3a7f47f640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3a78111e80 con 0x7f3a78106f50 2024-09-12T02:11:59.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.873+0000 7f3a7f47f640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f3a78112180 con 0x7f3a78100d00 2024-09-12T02:11:59.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.873+0000 7f3a7d21b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a78100350 0x7f3a781984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:59.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.873+0000 7f3a7d21b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a78100350 0x7f3a781984a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54166/0 (socket says 172.21.15.16:54166) 2024-09-12T02:11:59.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.873+0000 7f3a7d21b640 1 -- 172.21.15.16:0/1944566867 learned_addr learned my addr 172.21.15.16:0/1944566867 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:11:59.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.873+0000 7f3a7ca1a640 1 --2- 172.21.15.16:0/1944566867 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3a78100d00 0x7f3a781989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:59.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.873+0000 7f3a7da1c640 1 --2- 172.21.15.16:0/1944566867 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a78106f50 0x7f3a7819cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:59.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.873+0000 7f3a7d21b640 1 -- 172.21.15.16:0/1944566867 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3a78100d00 msgr2=0x7f3a781989e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:59.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.873+0000 7f3a7d21b640 1 --2- 172.21.15.16:0/1944566867 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3a78100d00 0x7f3a781989e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:59.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.873+0000 7f3a7d21b640 1 -- 172.21.15.16:0/1944566867 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a78106f50 msgr2=0x7f3a7819cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:59.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.873+0000 7f3a7d21b640 1 --2- 172.21.15.16:0/1944566867 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a78106f50 0x7f3a7819cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:59.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.873+0000 7f3a7d21b640 1 -- 172.21.15.16:0/1944566867 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a7819d450 con 0x7f3a78100350 2024-09-12T02:11:59.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.873+0000 7f3a7da1c640 1 --2- 172.21.15.16:0/1944566867 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a78106f50 0x7f3a7819cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:11:59.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.873+0000 7f3a7ca1a640 1 --2- 172.21.15.16:0/1944566867 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3a78100d00 0x7f3a781989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:11:59.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.873+0000 7f3a7d21b640 1 --2- 172.21.15.16:0/1944566867 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a78100350 0x7f3a781984a0 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f3a7400ba50 tx=0x7f3a7400bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:59.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.873+0000 7f3a6a7fc640 1 -- 172.21.15.16:0/1944566867 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a74019070 con 0x7f3a78100350 2024-09-12T02:11:59.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.873+0000 7f3a7f47f640 1 -- 172.21.15.16:0/1944566867 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a7819d740 con 0x7f3a78100350 2024-09-12T02:11:59.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.873+0000 7f3a6a7fc640 1 -- 172.21.15.16:0/1944566867 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a7400cbe0 con 0x7f3a78100350 2024-09-12T02:11:59.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.873+0000 7f3a6a7fc640 1 -- 172.21.15.16:0/1944566867 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a740076e0 con 0x7f3a78100350 2024-09-12T02:11:59.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.873+0000 7f3a7f47f640 1 -- 172.21.15.16:0/1944566867 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a781a5020 con 0x7f3a78100350 2024-09-12T02:11:59.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.876+0000 7f3a6a7fc640 1 -- 172.21.15.16:0/1944566867 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f3a7401c070 con 0x7f3a78100350 2024-09-12T02:11:59.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.876+0000 7f3a81858740 1 Processor -- start 2024-09-12T02:11:59.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.876+0000 7f3a81858740 1 -- start start 2024-09-12T02:11:59.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.876+0000 7f3a81858740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5638209c18b0 0x5638209c1c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:59.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.876+0000 7f3a81858740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638209c21d0 0x5638209cc550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:59.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.876+0000 7f3a81858740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5638209cca90 0x5638209cee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:59.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.876+0000 7f3a81858740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5638209232e0 con 0x5638209c21d0 2024-09-12T02:11:59.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.876+0000 7f3a81858740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5638208cace0 con 0x5638209c18b0 2024-09-12T02:11:59.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.876+0000 7f3a81858740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x56382091f2a0 con 0x5638209cca90 2024-09-12T02:11:59.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.876+0000 7f3a7d21b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638209c21d0 0x5638209cc550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:59.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.876+0000 7f3a7d21b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638209c21d0 0x5638209cc550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54180/0 (socket says 172.21.15.16:54180) 2024-09-12T02:11:59.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.876+0000 7f3a7d21b640 1 -- 172.21.15.16:0/1068938091 learned_addr learned my addr 172.21.15.16:0/1068938091 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:11:59.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.876+0000 7f3a6a7fc640 1 --2- 172.21.15.16:0/1944566867 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f3a640a2b80 0x7f3a640a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:59.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.876+0000 7f3a7da1c640 1 --2- 172.21.15.16:0/1068938091 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5638209c18b0 0x5638209c1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:59.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.877+0000 7f3a7ca1a640 1 --2- 172.21.15.16:0/1068938091 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5638209cca90 0x5638209cee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:59.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.877+0000 7f3a6a7fc640 1 -- 172.21.15.16:0/1944566867 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f3a740ce630 con 0x7f3a78100350 2024-09-12T02:11:59.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.877+0000 7f3a7da1c640 1 --2- 172.21.15.16:0/1944566867 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f3a640a2b80 0x7f3a640a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:59.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.877+0000 7f3a7d21b640 1 -- 172.21.15.16:0/1068938091 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5638209cca90 msgr2=0x5638209cee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:59.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.877+0000 7f3a7d21b640 1 --2- 172.21.15.16:0/1068938091 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5638209cca90 0x5638209cee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:59.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.877+0000 7f3a7d21b640 1 -- 172.21.15.16:0/1068938091 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5638209c18b0 msgr2=0x5638209c1c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:59.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.877+0000 7f3a7d21b640 1 --2- 172.21.15.16:0/1068938091 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5638209c18b0 0x5638209c1c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:59.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.877+0000 7f3a7d21b640 1 -- 172.21.15.16:0/1068938091 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638208af440 con 0x5638209c21d0 2024-09-12T02:11:59.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.877+0000 7f3a7ca1a640 1 --2- 172.21.15.16:0/1068938091 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5638209cca90 0x5638209cee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:11:59.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.877+0000 7f3a7d21b640 1 --2- 172.21.15.16:0/1068938091 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638209c21d0 0x5638209cc550 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f3a7409ac30 tx=0x7f3a74004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4144f071252c6a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:59.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.877+0000 7f3a637fe640 1 -- 172.21.15.16:0/1068938091 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a7409af30 con 0x5638209c21d0 2024-09-12T02:11:59.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.877+0000 7f3a637fe640 1 -- 172.21.15.16:0/1068938091 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a7409f2e0 con 0x5638209c21d0 2024-09-12T02:11:59.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.877+0000 7f3a637fe640 1 -- 172.21.15.16:0/1068938091 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a740a3710 con 0x5638209c21d0 2024-09-12T02:11:59.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.877+0000 7f3a7da1c640 1 --2- 172.21.15.16:0/1944566867 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f3a640a2b80 0x7f3a640a5040 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f3a70031550 tx=0x7f3a7002a420 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:59.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.878+0000 7f3a81858740 1 -- 172.21.15.16:0/1068938091 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638209c21d0 msgr2=0x5638209cc550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:59.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.878+0000 7f3a81858740 1 --2- 172.21.15.16:0/1068938091 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638209c21d0 0x5638209cc550 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f3a7409ac30 tx=0x7f3a74004a40 comp rx=0 tx=0).stop 2024-09-12T02:11:59.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.878+0000 7f3a81858740 1 -- 172.21.15.16:0/1068938091 shutdown_connections 2024-09-12T02:11:59.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.878+0000 7f3a81858740 1 --2- 172.21.15.16:0/1068938091 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5638209cca90 0x5638209cee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:59.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.878+0000 7f3a81858740 1 --2- 172.21.15.16:0/1068938091 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638209c21d0 0x5638209cc550 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:59.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.878+0000 7f3a81858740 1 --2- 172.21.15.16:0/1068938091 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5638209c18b0 0x5638209c1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:59.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.878+0000 7f3a81858740 1 -- 172.21.15.16:0/1068938091 >> 172.21.15.16:0/1068938091 conn(0x5638209bde40 msgr2=0x5638209be220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:59.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.878+0000 7f3a81858740 1 -- 172.21.15.16:0/1068938091 shutdown_connections 2024-09-12T02:11:59.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.878+0000 7f3a81858740 1 -- 172.21.15.16:0/1068938091 wait complete. 2024-09-12T02:11:59.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.878+0000 7f3a81858740 1 -- start start 2024-09-12T02:11:59.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.879+0000 7f3a81858740 1 Processor -- start 2024-09-12T02:11:59.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.879+0000 7f3a81858740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638209c18b0 0x5638209cc400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:59.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.879+0000 7f3a81858740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5638209c21d0 0x5638209c96b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:59.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.879+0000 7f3a81858740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5638209cca90 0x5638209c9c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:59.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.879+0000 7f3a81858740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5638209cf660 con 0x5638209c18b0 2024-09-12T02:11:59.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.879+0000 7f3a81858740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5638208de050 con 0x5638209c21d0 2024-09-12T02:11:59.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.879+0000 7f3a81858740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5638209cf7e0 con 0x5638209cca90 2024-09-12T02:11:59.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.879+0000 7f3a81858740 10 client.0 _ll_register_callbacks cb 0x56382091e160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:11:59.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.879+0000 7f3a7d21b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638209c18b0 0x5638209cc400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:59.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.879+0000 7f3a7d21b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638209c18b0 0x5638209cc400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54194/0 (socket says 172.21.15.16:54194) 2024-09-12T02:11:59.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.879+0000 7f3a7d21b640 1 -- 172.21.15.16:0/4034202959 learned_addr learned my addr 172.21.15.16:0/4034202959 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:11:59.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.879+0000 7f3a7ca1a640 1 --2- 172.21.15.16:0/4034202959 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5638209c21d0 0x5638209c96b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:59.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.879+0000 7f3a7d21b640 1 -- 172.21.15.16:0/4034202959 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5638209cca90 msgr2=0x5638209c9c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:59.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.879+0000 7f3a7da1c640 1 --2- 172.21.15.16:0/4034202959 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5638209cca90 0x5638209c9c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:59.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.879+0000 7f3a7d21b640 1 --2- 172.21.15.16:0/4034202959 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5638209cca90 0x5638209c9c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:59.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.880+0000 7f3a7d21b640 1 -- 172.21.15.16:0/4034202959 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5638209c21d0 msgr2=0x5638209c96b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:59.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.880+0000 7f3a7d21b640 1 --2- 172.21.15.16:0/4034202959 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5638209c21d0 0x5638209c96b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:59.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.880+0000 7f3a7d21b640 1 -- 172.21.15.16:0/4034202959 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638206943c0 con 0x5638209c18b0 2024-09-12T02:11:59.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.880+0000 7f3a7ca1a640 1 --2- 172.21.15.16:0/4034202959 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5638209c21d0 0x5638209c96b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:11:59.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.880+0000 7f3a7d21b640 1 --2- 172.21.15.16:0/4034202959 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638209c18b0 0x5638209cc400 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f3a740a3670 tx=0x7f3a740d41f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5064eac0e14df7da server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:59.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.880+0000 7f3a60ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:11:59.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.880+0000 7f3a60ff9640 1 -- 172.21.15.16:0/4034202959 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a740a3910 con 0x5638209c18b0 2024-09-12T02:11:59.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.880+0000 7f3a60ff9640 1 -- 172.21.15.16:0/4034202959 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f3a740b9b70 con 0x5638209c18b0 2024-09-12T02:11:59.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.880+0000 7f3a81858740 1 -- 172.21.15.16:0/4034202959 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3a7409f2e0 con 0x5638209c18b0 2024-09-12T02:11:59.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.880+0000 7f3a60ff9640 1 -- 172.21.15.16:0/4034202959 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f3a740b7030 con 0x5638209c18b0 2024-09-12T02:11:59.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.880+0000 7f3a81858740 10 client.18603 fetch_fsmap learned FSMap version 25 2024-09-12T02:11:59.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.880+0000 7f3a81858740 1 -- 172.21.15.16:0/4034202959 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5638209db030 con 0x5638209c18b0 2024-09-12T02:11:59.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.880+0000 7f3a60ff9640 1 -- 172.21.15.16:0/4034202959 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a740b6030 con 0x5638209c18b0 2024-09-12T02:11:59.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.880+0000 7f3a60ff9640 1 -- 172.21.15.16:0/4034202959 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f3a74096c80 con 0x5638209c18b0 2024-09-12T02:11:59.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.880+0000 7f3a81858740 10 client.18603 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:11:59.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.880+0000 7f3a81858740 10 client.18603 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:11:59.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.880+0000 7f3a81858740 20 client.18603 populate_metadata read hostname 'smithi016' 2024-09-12T02:11:59.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.880+0000 7f3a81858740 1 --2- 172.21.15.16:0/4034202959 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5638209dc380 0x5638209de770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:59.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.880+0000 7f3a81858740 4 client.18603 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:11:59.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.880+0000 7f3a81858740 1 -- 172.21.15.16:0/4034202959 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563820930850 con 0x5638209dc380 2024-09-12T02:11:59.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.881+0000 7f3a7ca1a640 1 --2- 172.21.15.16:0/4034202959 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5638209dc380 0x5638209de770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:59.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.881+0000 7f3a60ff9640 10 client.18603 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:11:59.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.881+0000 7f3a7ca1a640 1 --2- 172.21.15.16:0/4034202959 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5638209dc380 0x5638209de770 crc :-1 s=READY pgs=300 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=27c2d1c3afa85918 server_cookie=26ae5a37e691672a in_seq=0 out_seq=0 2024-09-12T02:11:59.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.882+0000 7f3a60ff9640 1 -- 172.21.15.16:0/4034202959 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563820930850 con 0x5638209dc380 2024-09-12T02:11:59.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.882+0000 7f3a60ff9640 10 client.18603 handle_command_reply: tid=0 2024-09-12T02:11:59.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.882+0000 7f3a81858740 1 client.18603 shutdown 2024-09-12T02:11:59.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.882+0000 7f3a81858740 1 -- 172.21.15.16:0/4034202959 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638209c18b0 msgr2=0x5638209cc400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:59.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.882+0000 7f3a81858740 1 --2- 172.21.15.16:0/4034202959 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638209c18b0 0x5638209cc400 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f3a740a3670 tx=0x7f3a740d41f0 comp rx=0 tx=0).stop 2024-09-12T02:11:59.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.882+0000 7f3a81858740 1 -- 172.21.15.16:0/4034202959 shutdown_connections 2024-09-12T02:11:59.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.882+0000 7f3a81858740 1 --2- 172.21.15.16:0/4034202959 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5638209dc380 0x5638209de770 crc :-1 s=READY pgs=300 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:59.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.882+0000 7f3a81858740 1 --2- 172.21.15.16:0/4034202959 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5638209cca90 0x5638209c9c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:59.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.882+0000 7f3a81858740 1 --2- 172.21.15.16:0/4034202959 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5638209c21d0 0x5638209c96b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:59.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.882+0000 7f3a81858740 1 --2- 172.21.15.16:0/4034202959 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638209c18b0 0x5638209cc400 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:59.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.882+0000 7f3a60ff9640 0 client.18603 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:11:59.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.882+0000 7f3a81858740 1 -- 172.21.15.16:0/4034202959 >> 172.21.15.16:0/4034202959 conn(0x5638209bde40 msgr2=0x5638209c3b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:59.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.882+0000 7f3a81858740 1 -- 172.21.15.16:0/4034202959 shutdown_connections 2024-09-12T02:11:59.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.882+0000 7f3a81858740 1 -- 172.21.15.16:0/4034202959 wait complete. 2024-09-12T02:11:59.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.882+0000 7f3a81858740 20 client.18603 trim_cache size 0 max 16384 2024-09-12T02:11:59.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.899+0000 7f3a81858740 1 Processor -- start 2024-09-12T02:11:59.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.899+0000 7f3a81858740 1 -- start start 2024-09-12T02:11:59.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.900+0000 7f3a81858740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638209c18b0 0x5638209d88c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:59.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.900+0000 7f3a81858740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5638209c21d0 0x5638209e3d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:59.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.900+0000 7f3a81858740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5638209cca90 0x5638209e4250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:59.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.900+0000 7f3a81858740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3a781adbe0 con 0x5638209c18b0 2024-09-12T02:11:59.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.900+0000 7f3a81858740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3a781ab4b0 con 0x5638209cca90 2024-09-12T02:11:59.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.900+0000 7f3a81858740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f3a781a9320 con 0x5638209c21d0 2024-09-12T02:11:59.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.900+0000 7f3a7d21b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638209c18b0 0x5638209d88c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:59.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.900+0000 7f3a7d21b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638209c18b0 0x5638209d88c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54208/0 (socket says 172.21.15.16:54208) 2024-09-12T02:11:59.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.900+0000 7f3a7d21b640 1 -- 172.21.15.16:0/3319293038 learned_addr learned my addr 172.21.15.16:0/3319293038 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:11:59.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.900+0000 7f3a7ca1a640 1 --2- 172.21.15.16:0/3319293038 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5638209c21d0 0x5638209e3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:59.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.900+0000 7f3a7da1c640 1 --2- 172.21.15.16:0/3319293038 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5638209cca90 0x5638209e4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:59.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.900+0000 7f3a7d21b640 1 -- 172.21.15.16:0/3319293038 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5638209c21d0 msgr2=0x5638209e3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:59.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.900+0000 7f3a7d21b640 1 --2- 172.21.15.16:0/3319293038 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5638209c21d0 0x5638209e3d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:59.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.900+0000 7f3a7d21b640 1 -- 172.21.15.16:0/3319293038 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5638209cca90 msgr2=0x5638209e4250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:59.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.900+0000 7f3a7d21b640 1 --2- 172.21.15.16:0/3319293038 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5638209cca90 0x5638209e4250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:59.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.900+0000 7f3a7d21b640 1 -- 172.21.15.16:0/3319293038 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638209deae0 con 0x5638209c18b0 2024-09-12T02:11:59.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.900+0000 7f3a7ca1a640 1 --2- 172.21.15.16:0/3319293038 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5638209c21d0 0x5638209e3d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:11:59.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.900+0000 7f3a7da1c640 1 --2- 172.21.15.16:0/3319293038 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5638209cca90 0x5638209e4250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:11:59.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.900+0000 7f3a7d21b640 1 --2- 172.21.15.16:0/3319293038 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638209c18b0 0x5638209d88c0 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f3a740978b0 tx=0x7f3a74004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4edd7d262338f11 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:59.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.900+0000 7f3a62ffd640 1 -- 172.21.15.16:0/3319293038 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a7407d030 con 0x5638209c18b0 2024-09-12T02:11:59.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.901+0000 7f3a62ffd640 1 -- 172.21.15.16:0/3319293038 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a740b6070 con 0x5638209c18b0 2024-09-12T02:11:59.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.901+0000 7f3a62ffd640 1 -- 172.21.15.16:0/3319293038 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a7409a4b0 con 0x5638209c18b0 2024-09-12T02:11:59.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.901+0000 7f3a81858740 1 -- 172.21.15.16:0/3319293038 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638209c18b0 msgr2=0x5638209d88c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:59.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.901+0000 7f3a81858740 1 --2- 172.21.15.16:0/3319293038 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638209c18b0 0x5638209d88c0 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f3a740978b0 tx=0x7f3a74004a40 comp rx=0 tx=0).stop 2024-09-12T02:11:59.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.901+0000 7f3a81858740 1 -- 172.21.15.16:0/3319293038 shutdown_connections 2024-09-12T02:11:59.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.901+0000 7f3a81858740 1 --2- 172.21.15.16:0/3319293038 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5638209cca90 0x5638209e4250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:59.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.901+0000 7f3a81858740 1 --2- 172.21.15.16:0/3319293038 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5638209c21d0 0x5638209e3d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:59.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.901+0000 7f3a81858740 1 --2- 172.21.15.16:0/3319293038 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638209c18b0 0x5638209d88c0 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:59.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.901+0000 7f3a81858740 1 -- 172.21.15.16:0/3319293038 >> 172.21.15.16:0/3319293038 conn(0x5638209bde40 msgr2=0x5638209c87a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:59.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.901+0000 7f3a81858740 1 -- 172.21.15.16:0/3319293038 shutdown_connections 2024-09-12T02:11:59.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.901+0000 7f3a81858740 1 -- 172.21.15.16:0/3319293038 wait complete. 2024-09-12T02:11:59.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.901+0000 7f3a81858740 1 -- start start 2024-09-12T02:11:59.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.902+0000 7f3a81858740 1 Processor -- start 2024-09-12T02:11:59.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.902+0000 7f3a81858740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5638209c18b0 0x5638209d4e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:59.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.902+0000 7f3a81858740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5638209c21d0 0x5638209d2190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:59.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.902+0000 7f3a81858740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638209cca90 0x5638209d26d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:59.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.902+0000 7f3a81858740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3a781a9970 con 0x5638209cca90 2024-09-12T02:11:59.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.902+0000 7f3a81858740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3a781aade0 con 0x5638209c18b0 2024-09-12T02:11:59.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.902+0000 7f3a81858740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f3a781aa040 con 0x5638209c21d0 2024-09-12T02:11:59.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.902+0000 7f3a81858740 10 client.0 _ll_register_callbacks cb 0x7f3a70009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:11:59.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.902+0000 7f3a7da1c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638209cca90 0x5638209d26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:59.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.902+0000 7f3a7da1c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638209cca90 0x5638209d26d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54222/0 (socket says 172.21.15.16:54222) 2024-09-12T02:11:59.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.902+0000 7f3a7da1c640 1 -- 172.21.15.16:0/3448170588 learned_addr learned my addr 172.21.15.16:0/3448170588 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:11:59.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.902+0000 7f3a7d21b640 1 --2- 172.21.15.16:0/3448170588 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5638209c18b0 0x5638209d4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:59.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.902+0000 7f3a7ca1a640 1 --2- 172.21.15.16:0/3448170588 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5638209c21d0 0x5638209d2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:59.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.903+0000 7f3a7da1c640 1 -- 172.21.15.16:0/3448170588 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5638209c21d0 msgr2=0x5638209d2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:59.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.903+0000 7f3a7da1c640 1 --2- 172.21.15.16:0/3448170588 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5638209c21d0 0x5638209d2190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:59.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.903+0000 7f3a7da1c640 1 -- 172.21.15.16:0/3448170588 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5638209c18b0 msgr2=0x5638209d4e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:59.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.903+0000 7f3a7da1c640 1 --2- 172.21.15.16:0/3448170588 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5638209c18b0 0x5638209d4e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:59.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.903+0000 7f3a7da1c640 1 -- 172.21.15.16:0/3448170588 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638209cc770 con 0x5638209cca90 2024-09-12T02:11:59.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.903+0000 7f3a7da1c640 1 --2- 172.21.15.16:0/3448170588 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638209cca90 0x5638209d26d0 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f3a70006010 tx=0x7f3a70009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2cc152abdd63616 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:59.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.903+0000 7f3a60ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:11:59.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.903+0000 7f3a60ff9640 1 -- 172.21.15.16:0/3448170588 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a70035070 con 0x5638209cca90 2024-09-12T02:11:59.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.903+0000 7f3a60ff9640 1 -- 172.21.15.16:0/3448170588 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f3a700048c0 con 0x5638209cca90 2024-09-12T02:11:59.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.903+0000 7f3a81858740 1 -- 172.21.15.16:0/3448170588 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3a740b6070 con 0x5638209cca90 2024-09-12T02:11:59.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.903+0000 7f3a60ff9640 1 -- 172.21.15.16:0/3448170588 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f3a70004ac0 con 0x5638209cca90 2024-09-12T02:11:59.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.903+0000 7f3a81858740 10 client.18609 fetch_fsmap learned FSMap version 25 2024-09-12T02:11:59.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.903+0000 7f3a81858740 1 -- 172.21.15.16:0/3448170588 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x563820a46770 con 0x5638209cca90 2024-09-12T02:11:59.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.903+0000 7f3a60ff9640 1 -- 172.21.15.16:0/3448170588 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a740b6070 con 0x5638209cca90 2024-09-12T02:11:59.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.904+0000 7f3a60ff9640 1 -- 172.21.15.16:0/3448170588 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f3a7003a040 con 0x5638209cca90 2024-09-12T02:11:59.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.904+0000 7f3a81858740 10 client.18609 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:11:59.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.904+0000 7f3a81858740 10 client.18609 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:11:59.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.904+0000 7f3a81858740 20 client.18609 populate_metadata read hostname 'smithi016' 2024-09-12T02:11:59.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.904+0000 7f3a81858740 1 --2- 172.21.15.16:0/3448170588 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x563820a46f60 0x563820a49380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:59.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.904+0000 7f3a81858740 4 client.18609 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T02:11:59.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.904+0000 7f3a81858740 1 -- 172.21.15.16:0/3448170588 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56382092ad60 con 0x563820a46f60 2024-09-12T02:11:59.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.904+0000 7f3a7d21b640 1 --2- 172.21.15.16:0/3448170588 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x563820a46f60 0x563820a49380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:59.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.904+0000 7f3a7d21b640 1 --2- 172.21.15.16:0/3448170588 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x563820a46f60 0x563820a49380 crc :-1 s=READY pgs=301 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ebdb77237b76241 server_cookie=ea620b756c383305 in_seq=0 out_seq=0 2024-09-12T02:11:59.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.904+0000 7f3a60ff9640 10 client.18609 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:11:59.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.905+0000 7f3a60ff9640 1 -- 172.21.15.16:0/3448170588 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x56382092ad60 con 0x563820a46f60 2024-09-12T02:11:59.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.905+0000 7f3a60ff9640 10 client.18609 handle_command_reply: tid=0 2024-09-12T02:11:59.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.905+0000 7f3a81858740 1 client.18609 shutdown 2024-09-12T02:11:59.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.905+0000 7f3a81858740 1 -- 172.21.15.16:0/3448170588 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638209cca90 msgr2=0x5638209d26d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:59.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.905+0000 7f3a81858740 1 --2- 172.21.15.16:0/3448170588 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638209cca90 0x5638209d26d0 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f3a70006010 tx=0x7f3a70009260 comp rx=0 tx=0).stop 2024-09-12T02:11:59.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.905+0000 7f3a81858740 1 -- 172.21.15.16:0/3448170588 shutdown_connections 2024-09-12T02:11:59.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.905+0000 7f3a81858740 1 --2- 172.21.15.16:0/3448170588 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x563820a46f60 0x563820a49380 crc :-1 s=READY pgs=301 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:59.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.905+0000 7f3a81858740 1 --2- 172.21.15.16:0/3448170588 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638209cca90 0x5638209d26d0 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:59.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.905+0000 7f3a81858740 1 --2- 172.21.15.16:0/3448170588 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5638209c21d0 0x5638209d2190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:59.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.905+0000 7f3a60ff9640 0 client.18609 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:11:59.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.905+0000 7f3a81858740 1 --2- 172.21.15.16:0/3448170588 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5638209c18b0 0x5638209d4e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:59.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.905+0000 7f3a81858740 1 -- 172.21.15.16:0/3448170588 >> 172.21.15.16:0/3448170588 conn(0x5638209bde40 msgr2=0x5638209b8220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:59.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.905+0000 7f3a81858740 1 -- 172.21.15.16:0/3448170588 shutdown_connections 2024-09-12T02:11:59.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.905+0000 7f3a81858740 1 -- 172.21.15.16:0/3448170588 wait complete. 2024-09-12T02:11:59.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.905+0000 7f3a81858740 20 client.18609 trim_cache size 0 max 16384 2024-09-12T02:11:59.911 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T02:11:59.911 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-12T02:11:59.911 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": { 2024-09-12T02:11:59.911 INFO:teuthology.orchestra.run.smithi016.stdout: "8b803ff1-9d56-4e42-81ec-2b28458d0df9": { 2024-09-12T02:11:59.911 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-12T02:11:59.911 INFO:teuthology.orchestra.run.smithi016.stdout: "tag": "8b803ff1-9d56-4e42-81ec-2b28458d0df9", 2024-09-12T02:11:59.911 INFO:teuthology.orchestra.run.smithi016.stdout: "options": "recursive,force" 2024-09-12T02:11:59.912 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:11:59.912 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:11:59.912 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T02:11:59.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.906+0000 7f3a7f47f640 1 -- 172.21.15.16:0/1944566867 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f3a640a2b80 msgr2=0x7f3a640a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:59.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.906+0000 7f3a7f47f640 1 --2- 172.21.15.16:0/1944566867 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f3a640a2b80 0x7f3a640a5040 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f3a70031550 tx=0x7f3a7002a420 comp rx=0 tx=0).stop 2024-09-12T02:11:59.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.906+0000 7f3a7f47f640 1 -- 172.21.15.16:0/1944566867 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a78100350 msgr2=0x7f3a781984a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:59.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.906+0000 7f3a7f47f640 1 --2- 172.21.15.16:0/1944566867 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a78100350 0x7f3a781984a0 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f3a7400ba50 tx=0x7f3a7400bf10 comp rx=0 tx=0).stop 2024-09-12T02:11:59.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.906+0000 7f3a7f47f640 1 -- 172.21.15.16:0/1944566867 shutdown_connections 2024-09-12T02:11:59.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.906+0000 7f3a7f47f640 1 --2- 172.21.15.16:0/1944566867 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f3a640a2b80 0x7f3a640a5040 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:59.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.906+0000 7f3a7f47f640 1 --2- 172.21.15.16:0/1944566867 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a78106f50 0x7f3a7819cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:59.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.906+0000 7f3a7f47f640 1 --2- 172.21.15.16:0/1944566867 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3a78100d00 0x7f3a781989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:59.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.906+0000 7f3a7f47f640 1 --2- 172.21.15.16:0/1944566867 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a78100350 0x7f3a781984a0 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:59.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.906+0000 7f3a7f47f640 1 -- 172.21.15.16:0/1944566867 >> 172.21.15.16:0/1944566867 conn(0x7f3a78077210 msgr2=0x7f3a78105710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:59.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.906+0000 7f3a7f47f640 1 -- 172.21.15.16:0/1944566867 shutdown_connections 2024-09-12T02:11:59.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:11:59.906+0000 7f3a7f47f640 1 -- 172.21.15.16:0/1944566867 wait complete. 2024-09-12T02:12:00.616 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:12:00 smithi016 ceph-mon[30947]: pgmap v637: 97 pgs: 97 active+clean; 1.2 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 7.3 KiB/s wr, 10 op/s 2024-09-12T02:12:00.647 INFO:tasks.cephfs.filesystem:scrub status for tag:8b803ff1-9d56-4e42-81ec-2b28458d0df9 - {'path': '/', 'tag': '8b803ff1-9d56-4e42-81ec-2b28458d0df9', 'options': 'recursive,force'} 2024-09-12T02:12:00.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:12:00 smithi033 ceph-mon[35332]: pgmap v637: 97 pgs: 97 active+clean; 1.2 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 7.3 KiB/s wr, 10 op/s 2024-09-12T02:12:00.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:12:00 smithi184 ceph-mon[35593]: pgmap v637: 97 pgs: 97 active+clean; 1.2 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 7.3 KiB/s wr, 10 op/s 2024-09-12T02:12:01.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:12:01 smithi033 ceph-mon[35332]: scrub summary: idle 2024-09-12T02:12:01.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:12:01 smithi016 ceph-mon[30947]: scrub summary: idle 2024-09-12T02:12:01.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:12:01 smithi184 ceph-mon[35593]: scrub summary: idle 2024-09-12T02:12:02.245 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.245+0000 7feea8c9b640 1 -- 172.21.15.16:0/61328846 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feea4101c40 msgr2=0x7feea41020c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:02.246 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.245+0000 7feea8c9b640 1 --2- 172.21.15.16:0/61328846 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feea4101c40 0x7feea41020c0 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7fee8c009900 tx=0x7fee8c02a410 comp rx=0 tx=0).stop 2024-09-12T02:12:02.246 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.245+0000 7feea8c9b640 1 -- 172.21.15.16:0/61328846 shutdown_connections 2024-09-12T02:12:02.246 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.245+0000 7feea8c9b640 1 --2- 172.21.15.16:0/61328846 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feea4102600 0x7feea4108e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:02.246 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.245+0000 7feea8c9b640 1 --2- 172.21.15.16:0/61328846 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feea4101c40 0x7feea41020c0 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:02.246 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.245+0000 7feea8c9b640 1 --2- 172.21.15.16:0/61328846 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feea4100a50 0x7feea4100e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:02.246 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.245+0000 7feea8c9b640 1 -- 172.21.15.16:0/61328846 >> 172.21.15.16:0/61328846 conn(0x7feea40fc1b0 msgr2=0x7feea40fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:02.246 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.245+0000 7feea8c9b640 1 -- 172.21.15.16:0/61328846 shutdown_connections 2024-09-12T02:12:02.246 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.245+0000 7feea8c9b640 1 -- 172.21.15.16:0/61328846 wait complete. 2024-09-12T02:12:02.248 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.246+0000 7feea8c9b640 1 Processor -- start 2024-09-12T02:12:02.248 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.246+0000 7feea8c9b640 1 -- start start 2024-09-12T02:12:02.249 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.246+0000 7feea8c9b640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feea4100a50 0x7feea4197dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:02.249 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.246+0000 7feea8c9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feea4101c40 0x7feea4198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:02.249 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.246+0000 7feea8c9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feea4102600 0x7feea419f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:02.249 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.246+0000 7feea8c9b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7feea410d820 con 0x7feea4102600 2024-09-12T02:12:02.249 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.246+0000 7feea8c9b640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7feea410d6a0 con 0x7feea4101c40 2024-09-12T02:12:02.249 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.246+0000 7feea8c9b640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7feea410d9a0 con 0x7feea4100a50 2024-09-12T02:12:02.249 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.246+0000 7feea2d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feea4102600 0x7feea419f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:02.249 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.246+0000 7feea2d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feea4102600 0x7feea419f380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38136/0 (socket says 172.21.15.16:38136) 2024-09-12T02:12:02.249 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.246+0000 7feea2d9d640 1 -- 172.21.15.16:0/3476368847 learned_addr learned my addr 172.21.15.16:0/3476368847 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:12:02.250 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.246+0000 7feea1d9b640 1 --2- 172.21.15.16:0/3476368847 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feea4101c40 0x7feea4198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:02.250 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.247+0000 7feea259c640 1 --2- 172.21.15.16:0/3476368847 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feea4100a50 0x7feea4197dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:02.250 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.247+0000 7feea2d9d640 1 -- 172.21.15.16:0/3476368847 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feea4100a50 msgr2=0x7feea4197dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:02.250 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.247+0000 7feea2d9d640 1 --2- 172.21.15.16:0/3476368847 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feea4100a50 0x7feea4197dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:02.250 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.247+0000 7feea2d9d640 1 -- 172.21.15.16:0/3476368847 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feea4101c40 msgr2=0x7feea4198310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:02.250 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.247+0000 7feea2d9d640 1 --2- 172.21.15.16:0/3476368847 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feea4101c40 0x7feea4198310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:02.250 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.247+0000 7feea2d9d640 1 -- 172.21.15.16:0/3476368847 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feea419f9f0 con 0x7feea4102600 2024-09-12T02:12:02.250 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.247+0000 7feea259c640 1 --2- 172.21.15.16:0/3476368847 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feea4100a50 0x7feea4197dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:12:02.250 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.247+0000 7feea2d9d640 1 --2- 172.21.15.16:0/3476368847 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feea4102600 0x7feea419f380 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7fee9c00cce0 tx=0x7fee9c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:02.251 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.247+0000 7fee9b7fe640 1 -- 172.21.15.16:0/3476368847 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee9c00f450 con 0x7feea4102600 2024-09-12T02:12:02.251 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.247+0000 7fee9b7fe640 1 -- 172.21.15.16:0/3476368847 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee9c00fad0 con 0x7feea4102600 2024-09-12T02:12:02.251 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.247+0000 7fee9b7fe640 1 -- 172.21.15.16:0/3476368847 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee9c018c10 con 0x7feea4102600 2024-09-12T02:12:02.251 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.247+0000 7feea8c9b640 1 -- 172.21.15.16:0/3476368847 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feea419fce0 con 0x7feea4102600 2024-09-12T02:12:02.251 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.247+0000 7feea8c9b640 1 -- 172.21.15.16:0/3476368847 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feea41a01b0 con 0x7feea4102600 2024-09-12T02:12:02.261 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.250+0000 7fee9b7fe640 1 -- 172.21.15.16:0/3476368847 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fee9c00f5f0 con 0x7feea4102600 2024-09-12T02:12:02.261 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.250+0000 7feeaba31740 1 Processor -- start 2024-09-12T02:12:02.262 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.250+0000 7feeaba31740 1 -- start start 2024-09-12T02:12:02.262 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.250+0000 7feeaba31740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557351db49a0 0x557351db4da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:02.262 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.250+0000 7feeaba31740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557351db52e0 0x557351db9da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:02.262 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.250+0000 7feeaba31740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557351dba370 0x557351dbc830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:02.262 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.251+0000 7feeaba31740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x557351d182f0 con 0x557351dba370 2024-09-12T02:12:02.262 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.251+0000 7feeaba31740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x557351cc0270 con 0x557351db52e0 2024-09-12T02:12:02.262 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.251+0000 7feeaba31740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x557351d142b0 con 0x557351db49a0 2024-09-12T02:12:02.263 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.251+0000 7fee9b7fe640 1 --2- 172.21.15.16:0/3476368847 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fee900a2cb0 0x7fee900a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:02.263 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.251+0000 7feea1d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557351dba370 0x557351dbc830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:02.263 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.251+0000 7feea259c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557351db52e0 0x557351db9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:02.263 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.251+0000 7feea1d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557351dba370 0x557351dbc830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38140/0 (socket says 172.21.15.16:38140) 2024-09-12T02:12:02.263 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.251+0000 7feea259c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557351db52e0 0x557351db9da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.16:43576/0 (socket says 172.21.15.16:43576) 2024-09-12T02:12:02.263 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.251+0000 7feea1d9b640 1 -- 172.21.15.16:0/2355554300 learned_addr learned my addr 172.21.15.16:0/2355554300 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:12:02.263 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.251+0000 7feea1d9b640 1 --2- 172.21.15.16:0/2355554300 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557351db49a0 0x557351db4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:02.263 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.251+0000 7feea2d9d640 1 --2- 172.21.15.16:0/3476368847 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fee900a2cb0 0x7fee900a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:02.264 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.251+0000 7fee9b7fe640 1 -- 172.21.15.16:0/3476368847 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fee9c0cf5f0 con 0x7feea4102600 2024-09-12T02:12:02.264 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.251+0000 7feea259c640 1 -- 172.21.15.16:0/2355554300 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557351db49a0 msgr2=0x557351db4da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:02.264 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.251+0000 7feea259c640 1 --2- 172.21.15.16:0/2355554300 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557351db49a0 0x557351db4da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:02.264 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.251+0000 7feea259c640 1 -- 172.21.15.16:0/2355554300 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557351dba370 msgr2=0x557351dbc830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:02.264 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.251+0000 7feea259c640 1 --2- 172.21.15.16:0/2355554300 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557351dba370 0x557351dbc830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:02.264 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.251+0000 7feea259c640 1 -- 172.21.15.16:0/2355554300 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557351cbf340 con 0x557351db52e0 2024-09-12T02:12:02.264 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.251+0000 7feea1d9b640 1 --2- 172.21.15.16:0/2355554300 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557351db49a0 0x557351db4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:12:02.264 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.251+0000 7feea259c640 1 --2- 172.21.15.16:0/2355554300 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557351db52e0 0x557351db9da0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7feea4101aa0 tx=0x7fee9400f730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f675409cbdf8d353 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:02.265 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.251+0000 7feea2d9d640 1 --2- 172.21.15.16:0/3476368847 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fee900a2cb0 0x7fee900a5170 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fee9c09be80 tx=0x7fee9c066ac0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:02.265 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.252+0000 7fee98ff9640 1 -- 172.21.15.16:0/2355554300 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee94015070 con 0x557351db52e0 2024-09-12T02:12:02.265 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.252+0000 7fee98ff9640 1 -- 172.21.15.16:0/2355554300 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee9401a940 con 0x557351db52e0 2024-09-12T02:12:02.265 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.252+0000 7fee98ff9640 1 -- 172.21.15.16:0/2355554300 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee940118c0 con 0x557351db52e0 2024-09-12T02:12:02.265 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.252+0000 7feeaba31740 1 -- 172.21.15.16:0/2355554300 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557351db52e0 msgr2=0x557351db9da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:02.265 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.252+0000 7feeaba31740 1 --2- 172.21.15.16:0/2355554300 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557351db52e0 0x557351db9da0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7feea4101aa0 tx=0x7fee9400f730 comp rx=0 tx=0).stop 2024-09-12T02:12:02.265 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.252+0000 7feeaba31740 1 -- 172.21.15.16:0/2355554300 shutdown_connections 2024-09-12T02:12:02.265 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.252+0000 7feeaba31740 1 --2- 172.21.15.16:0/2355554300 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557351dba370 0x557351dbc830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:02.266 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.252+0000 7feeaba31740 1 --2- 172.21.15.16:0/2355554300 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557351db52e0 0x557351db9da0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:02.266 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.252+0000 7feeaba31740 1 --2- 172.21.15.16:0/2355554300 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557351db49a0 0x557351db4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:02.266 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.252+0000 7feeaba31740 1 -- 172.21.15.16:0/2355554300 >> 172.21.15.16:0/2355554300 conn(0x557351db0010 msgr2=0x557351db0410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:02.266 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.253+0000 7feeaba31740 1 -- 172.21.15.16:0/2355554300 shutdown_connections 2024-09-12T02:12:02.266 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.253+0000 7feeaba31740 1 -- 172.21.15.16:0/2355554300 wait complete. 2024-09-12T02:12:02.266 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.253+0000 7feeaba31740 1 -- start start 2024-09-12T02:12:02.266 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.253+0000 7feeaba31740 1 Processor -- start 2024-09-12T02:12:02.267 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.253+0000 7feeaba31740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557351db49a0 0x557351dbec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:02.267 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.253+0000 7feeaba31740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557351db52e0 0x557351dbf190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:02.267 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.253+0000 7feeaba31740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557351dba370 0x557351dc61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:02.267 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.253+0000 7feeaba31740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x557351dbd340 con 0x557351db49a0 2024-09-12T02:12:02.267 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.253+0000 7feeaba31740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x557351ca1290 con 0x557351db52e0 2024-09-12T02:12:02.267 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.253+0000 7feeaba31740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x557351dbd4c0 con 0x557351dba370 2024-09-12T02:12:02.267 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.253+0000 7feeaba31740 10 client.0 _ll_register_callbacks cb 0x557351cd2140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:12:02.267 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.253+0000 7feea1d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557351db49a0 0x557351dbec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:02.268 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.253+0000 7feea1d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557351db49a0 0x557351dbec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38150/0 (socket says 172.21.15.16:38150) 2024-09-12T02:12:02.268 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.253+0000 7feea1d9b640 1 -- 172.21.15.16:0/2718207548 learned_addr learned my addr 172.21.15.16:0/2718207548 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:12:02.268 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.253+0000 7feea259c640 1 --2- 172.21.15.16:0/2718207548 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557351db52e0 0x557351dbf190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:02.268 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.254+0000 7feea1d9b640 1 --2- 172.21.15.16:0/2718207548 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557351dba370 0x557351dc61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:02.268 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.254+0000 7feea259c640 1 -- 172.21.15.16:0/2718207548 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557351dba370 msgr2=0x557351dc61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:02.268 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.254+0000 7feea259c640 1 --2- 172.21.15.16:0/2718207548 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557351dba370 0x557351dc61d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:02.268 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.254+0000 7feea259c640 1 -- 172.21.15.16:0/2718207548 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557351db49a0 msgr2=0x557351dbec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:02.268 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.254+0000 7feea259c640 1 --2- 172.21.15.16:0/2718207548 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557351db49a0 0x557351dbec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:02.268 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.254+0000 7feea259c640 1 -- 172.21.15.16:0/2718207548 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557351a2b210 con 0x557351db52e0 2024-09-12T02:12:02.268 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.254+0000 7feea1d9b640 1 --2- 172.21.15.16:0/2718207548 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557351db49a0 0x557351dbec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:12:02.268 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.254+0000 7feea259c640 1 --2- 172.21.15.16:0/2718207548 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557351db52e0 0x557351dbf190 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x557351db0e90 tx=0x7fee9401a690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=246ab8d6a36fda28 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:02.269 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.254+0000 7fee89ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-12T02:12:02.269 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.254+0000 7fee89ffb640 1 -- 172.21.15.16:0/2718207548 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee94015070 con 0x557351db52e0 2024-09-12T02:12:02.269 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.254+0000 7feeaba31740 1 -- 172.21.15.16:0/2718207548 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee9401a940 con 0x557351db52e0 2024-09-12T02:12:02.269 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.254+0000 7fee89ffb640 1 -- 172.21.15.16:0/2718207548 <== mon.2 v2:172.21.15.33:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fee9401ad00 con 0x557351db52e0 2024-09-12T02:12:02.269 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.254+0000 7fee89ffb640 1 -- 172.21.15.16:0/2718207548 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fee94004020 con 0x557351db52e0 2024-09-12T02:12:02.269 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.254+0000 7feeaba31740 10 client.27737 fetch_fsmap learned FSMap version 25 2024-09-12T02:12:02.269 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.254+0000 7feeaba31740 1 -- 172.21.15.16:0/2718207548 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x557351af7620 con 0x557351db52e0 2024-09-12T02:12:02.269 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.255+0000 7fee89ffb640 1 -- 172.21.15.16:0/2718207548 <== mon.2 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee9401a940 con 0x557351db52e0 2024-09-12T02:12:02.269 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.255+0000 7fee89ffb640 1 -- 172.21.15.16:0/2718207548 <== mon.2 v2:172.21.15.33:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fee9402c050 con 0x557351db52e0 2024-09-12T02:12:02.269 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.255+0000 7feeaba31740 10 client.27737 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:12:02.269 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.255+0000 7feeaba31740 10 client.27737 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:12:02.269 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.255+0000 7feeaba31740 20 client.27737 populate_metadata read hostname 'smithi016' 2024-09-12T02:12:02.270 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.255+0000 7feeaba31740 1 --2- 172.21.15.16:0/2718207548 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x557351dca590 0x557351dcc980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:02.270 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.255+0000 7feeaba31740 4 client.27737 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:12:02.270 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.255+0000 7feeaba31740 1 -- 172.21.15.16:0/2718207548 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557351d25a20 con 0x557351dca590 2024-09-12T02:12:02.270 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.255+0000 7feea1d9b640 1 --2- 172.21.15.16:0/2718207548 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x557351dca590 0x557351dcc980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:02.270 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.256+0000 7fee89ffb640 10 client.27737 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:12:02.270 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.256+0000 7feea1d9b640 1 --2- 172.21.15.16:0/2718207548 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x557351dca590 0x557351dcc980 crc :-1 s=READY pgs=302 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=69d07f9cc793ad85 server_cookie=12d93c8d9cc83e63 in_seq=0 out_seq=0 2024-09-12T02:12:02.270 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.256+0000 7fee89ffb640 1 -- 172.21.15.16:0/2718207548 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557351d25a20 con 0x557351dca590 2024-09-12T02:12:02.270 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.256+0000 7fee89ffb640 10 client.27737 handle_command_reply: tid=0 2024-09-12T02:12:02.270 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.256+0000 7feeaba31740 1 client.27737 shutdown 2024-09-12T02:12:02.270 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.256+0000 7feeaba31740 1 -- 172.21.15.16:0/2718207548 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557351db52e0 msgr2=0x557351dbf190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:02.270 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.256+0000 7feeaba31740 1 --2- 172.21.15.16:0/2718207548 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557351db52e0 0x557351dbf190 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x557351db0e90 tx=0x7fee9401a690 comp rx=0 tx=0).stop 2024-09-12T02:12:02.271 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.256+0000 7feeaba31740 1 -- 172.21.15.16:0/2718207548 shutdown_connections 2024-09-12T02:12:02.271 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.256+0000 7feeaba31740 1 --2- 172.21.15.16:0/2718207548 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x557351dca590 0x557351dcc980 crc :-1 s=READY pgs=302 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:02.271 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.256+0000 7feeaba31740 1 --2- 172.21.15.16:0/2718207548 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557351dba370 0x557351dc61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:02.271 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.256+0000 7feeaba31740 1 --2- 172.21.15.16:0/2718207548 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557351db52e0 0x557351dbf190 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:02.271 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.256+0000 7feeaba31740 1 --2- 172.21.15.16:0/2718207548 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557351db49a0 0x557351dbec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:02.271 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.256+0000 7fee89ffb640 0 client.27737 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:12:02.271 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.256+0000 7feeaba31740 1 -- 172.21.15.16:0/2718207548 >> 172.21.15.16:0/2718207548 conn(0x557351db0010 msgr2=0x557351db1820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:02.271 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.257+0000 7feeaba31740 1 -- 172.21.15.16:0/2718207548 shutdown_connections 2024-09-12T02:12:02.271 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.257+0000 7feeaba31740 1 -- 172.21.15.16:0/2718207548 wait complete. 2024-09-12T02:12:02.271 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.257+0000 7feeaba31740 20 client.27737 trim_cache size 0 max 16384 2024-09-12T02:12:02.277 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.271+0000 7feeaba31740 1 Processor -- start 2024-09-12T02:12:02.277 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.271+0000 7feeaba31740 1 -- start start 2024-09-12T02:12:02.277 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.271+0000 7feeaba31740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557351db0010 0x557351daf0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:02.277 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.271+0000 7feeaba31740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557351db49a0 0x557351dd1150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:02.277 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.271+0000 7feeaba31740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557351db52e0 0x557351dd1690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:02.277 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.271+0000 7feeaba31740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7feea41ab680 con 0x557351db49a0 2024-09-12T02:12:02.277 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.271+0000 7feeaba31740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7feea41a8f50 con 0x557351db52e0 2024-09-12T02:12:02.277 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.271+0000 7feeaba31740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7feea41a6da0 con 0x557351db0010 2024-09-12T02:12:02.277 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.271+0000 7feea259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557351db49a0 0x557351dd1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:02.277 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.272+0000 7feea259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557351db49a0 0x557351dd1150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38152/0 (socket says 172.21.15.16:38152) 2024-09-12T02:12:02.277 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.272+0000 7feea259c640 1 -- 172.21.15.16:0/1215216584 learned_addr learned my addr 172.21.15.16:0/1215216584 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:12:02.278 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.272+0000 7feea1d9b640 1 --2- 172.21.15.16:0/1215216584 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557351db0010 0x557351daf0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:02.278 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.272+0000 7feea259c640 1 -- 172.21.15.16:0/1215216584 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557351db0010 msgr2=0x557351daf0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:02.278 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.272+0000 7feea259c640 1 --2- 172.21.15.16:0/1215216584 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557351db0010 0x557351daf0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:02.278 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.272+0000 7feea259c640 1 -- 172.21.15.16:0/1215216584 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557351db52e0 msgr2=0x557351dd1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:02.278 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.272+0000 7feea259c640 1 --2- 172.21.15.16:0/1215216584 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557351db52e0 0x557351dd1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:02.278 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.272+0000 7feea259c640 1 -- 172.21.15.16:0/1215216584 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557351dab2d0 con 0x557351db49a0 2024-09-12T02:12:02.278 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.272+0000 7feea1d9b640 1 --2- 172.21.15.16:0/1215216584 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557351db0010 0x557351daf0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:12:02.278 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.272+0000 7feea259c640 1 --2- 172.21.15.16:0/1215216584 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557351db49a0 0x557351dd1150 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7fee940045c0 tx=0x7fee9400bf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78e15744c0cd4d2f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:02.278 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.272+0000 7fee8bfff640 1 -- 172.21.15.16:0/1215216584 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee94015070 con 0x557351db49a0 2024-09-12T02:12:02.278 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.272+0000 7fee8bfff640 1 -- 172.21.15.16:0/1215216584 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee9401ad00 con 0x557351db49a0 2024-09-12T02:12:02.278 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.272+0000 7fee8bfff640 1 -- 172.21.15.16:0/1215216584 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee940048e0 con 0x557351db49a0 2024-09-12T02:12:02.278 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.273+0000 7feeaba31740 1 -- 172.21.15.16:0/1215216584 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557351db49a0 msgr2=0x557351dd1150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:02.278 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.273+0000 7feeaba31740 1 --2- 172.21.15.16:0/1215216584 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557351db49a0 0x557351dd1150 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7fee940045c0 tx=0x7fee9400bf00 comp rx=0 tx=0).stop 2024-09-12T02:12:02.279 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.273+0000 7feeaba31740 1 -- 172.21.15.16:0/1215216584 shutdown_connections 2024-09-12T02:12:02.279 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.273+0000 7feeaba31740 1 --2- 172.21.15.16:0/1215216584 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557351db52e0 0x557351dd1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:02.279 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.273+0000 7feeaba31740 1 --2- 172.21.15.16:0/1215216584 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557351db49a0 0x557351dd1150 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:02.279 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.273+0000 7feeaba31740 1 --2- 172.21.15.16:0/1215216584 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557351db0010 0x557351daf0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:02.279 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.273+0000 7feeaba31740 1 -- 172.21.15.16:0/1215216584 >> 172.21.15.16:0/1215216584 conn(0x557351db9ec0 msgr2=0x557351dae9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:02.279 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.273+0000 7feeaba31740 1 -- 172.21.15.16:0/1215216584 shutdown_connections 2024-09-12T02:12:02.280 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.273+0000 7feeaba31740 1 -- 172.21.15.16:0/1215216584 wait complete. 2024-09-12T02:12:02.280 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.273+0000 7feeaba31740 1 -- start start 2024-09-12T02:12:02.280 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.273+0000 7feeaba31740 1 Processor -- start 2024-09-12T02:12:02.280 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.273+0000 7feeaba31740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557351db0010 0x557351e30720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:02.280 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.273+0000 7feeaba31740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557351db49a0 0x557351e30c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:02.280 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.273+0000 7feeaba31740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557351db52e0 0x557351e37ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:02.280 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.273+0000 7feeaba31740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7feea41a7410 con 0x557351db0010 2024-09-12T02:12:02.280 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.273+0000 7feeaba31740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7feea41a8880 con 0x557351db52e0 2024-09-12T02:12:02.280 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.273+0000 7feeaba31740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7feea41a7ae0 con 0x557351db49a0 2024-09-12T02:12:02.280 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.273+0000 7feeaba31740 10 client.0 _ll_register_callbacks cb 0x7fee8c005e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:12:02.280 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.274+0000 7feea1d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557351db0010 0x557351e30720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:02.280 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.274+0000 7feea1d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557351db0010 0x557351e30720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38158/0 (socket says 172.21.15.16:38158) 2024-09-12T02:12:02.281 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.274+0000 7feea1d9b640 1 -- 172.21.15.16:0/3580281461 learned_addr learned my addr 172.21.15.16:0/3580281461 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:12:02.281 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.274+0000 7feea1d9b640 1 -- 172.21.15.16:0/3580281461 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557351db49a0 msgr2=0x557351e30c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:02.281 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.274+0000 7feea1d9b640 1 --2- 172.21.15.16:0/3580281461 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557351db49a0 0x557351e30c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:02.281 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.274+0000 7feea1d9b640 1 -- 172.21.15.16:0/3580281461 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557351db52e0 msgr2=0x557351e37ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:02.281 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.274+0000 7feea1d9b640 1 --2- 172.21.15.16:0/3580281461 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557351db52e0 0x557351e37ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:02.281 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.274+0000 7feea1d9b640 1 -- 172.21.15.16:0/3580281461 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557351e382b0 con 0x557351db0010 2024-09-12T02:12:02.281 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.274+0000 7feea1d9b640 1 --2- 172.21.15.16:0/3580281461 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557351db0010 0x557351e30720 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7fee8c006010 tx=0x7fee8c0092d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f07660be15bcf9b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:02.281 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.274+0000 7fee89ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:12:02.281 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.274+0000 7fee89ffb640 1 -- 172.21.15.16:0/3580281461 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee8c051060 con 0x557351db0010 2024-09-12T02:12:02.281 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.274+0000 7fee89ffb640 1 -- 172.21.15.16:0/3580281461 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fee8c04de80 con 0x557351db0010 2024-09-12T02:12:02.281 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.274+0000 7feeaba31740 1 -- 172.21.15.16:0/3580281461 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee9401ad00 con 0x557351db0010 2024-09-12T02:12:02.281 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.274+0000 7fee89ffb640 1 -- 172.21.15.16:0/3580281461 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fee8c035040 con 0x557351db0010 2024-09-12T02:12:02.281 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.274+0000 7feeaba31740 10 client.18627 fetch_fsmap learned FSMap version 25 2024-09-12T02:12:02.282 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.274+0000 7feeaba31740 1 -- 172.21.15.16:0/3580281461 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x557351e395d0 con 0x557351db0010 2024-09-12T02:12:02.282 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.275+0000 7fee89ffb640 1 -- 172.21.15.16:0/3580281461 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee9401ad00 con 0x557351db0010 2024-09-12T02:12:02.282 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.275+0000 7fee89ffb640 1 -- 172.21.15.16:0/3580281461 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fee8c03d450 con 0x557351db0010 2024-09-12T02:12:02.282 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.275+0000 7feeaba31740 10 client.18627 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:12:02.282 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.275+0000 7feeaba31740 10 client.18627 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:12:02.282 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.275+0000 7feeaba31740 20 client.18627 populate_metadata read hostname 'smithi016' 2024-09-12T02:12:02.282 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.275+0000 7feeaba31740 1 --2- 172.21.15.16:0/3580281461 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x557351dba370 0x557351e3bf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:02.282 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.275+0000 7feeaba31740 4 client.18627 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:12:02.282 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.275+0000 7feeaba31740 1 -- 172.21.15.16:0/3580281461 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557351db5fd0 con 0x557351dba370 2024-09-12T02:12:02.282 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.275+0000 7feea259c640 1 --2- 172.21.15.16:0/3580281461 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x557351dba370 0x557351e3bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:02.282 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.276+0000 7fee89ffb640 10 client.18627 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:12:02.282 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.276+0000 7feea259c640 1 --2- 172.21.15.16:0/3580281461 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x557351dba370 0x557351e3bf50 crc :-1 s=READY pgs=303 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d19fc74d5b490fa server_cookie=35d2fc75062d9354 in_seq=0 out_seq=0 2024-09-12T02:12:02.287 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.287+0000 7fee89ffb640 1 -- 172.21.15.16:0/3580281461 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557351db5fd0 con 0x557351dba370 2024-09-12T02:12:02.288 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.287+0000 7fee89ffb640 10 client.18627 handle_command_reply: tid=0 2024-09-12T02:12:02.288 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.287+0000 7feeaba31740 1 client.18627 shutdown 2024-09-12T02:12:02.288 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.287+0000 7feeaba31740 1 -- 172.21.15.16:0/3580281461 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557351db0010 msgr2=0x557351e30720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:02.288 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.287+0000 7feeaba31740 1 --2- 172.21.15.16:0/3580281461 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557351db0010 0x557351e30720 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7fee8c006010 tx=0x7fee8c0092d0 comp rx=0 tx=0).stop 2024-09-12T02:12:02.288 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.287+0000 7feeaba31740 1 -- 172.21.15.16:0/3580281461 shutdown_connections 2024-09-12T02:12:02.288 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.287+0000 7feeaba31740 1 --2- 172.21.15.16:0/3580281461 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x557351dba370 0x557351e3bf50 crc :-1 s=READY pgs=303 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:02.288 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.287+0000 7feeaba31740 1 --2- 172.21.15.16:0/3580281461 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557351db52e0 0x557351e37ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:02.288 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.287+0000 7feeaba31740 1 --2- 172.21.15.16:0/3580281461 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557351db49a0 0x557351e30c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:02.288 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.287+0000 7feeaba31740 1 --2- 172.21.15.16:0/3580281461 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557351db0010 0x557351e30720 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:02.288 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.287+0000 7fee89ffb640 0 client.18627 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:12:02.288 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.287+0000 7feeaba31740 1 -- 172.21.15.16:0/3580281461 >> 172.21.15.16:0/3580281461 conn(0x557351db9ec0 msgr2=0x557351daf1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:02.289 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.287+0000 7feeaba31740 1 -- 172.21.15.16:0/3580281461 shutdown_connections 2024-09-12T02:12:02.289 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.287+0000 7feeaba31740 1 -- 172.21.15.16:0/3580281461 wait complete. 2024-09-12T02:12:02.289 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.287+0000 7feeaba31740 20 client.18627 trim_cache size 0 max 16384 2024-09-12T02:12:02.289 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-09-12T02:12:02.289 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-09-12T02:12:02.289 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-09-12T02:12:02.289 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-09-12T02:12:02.290 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.288+0000 7feea8c9b640 1 -- 172.21.15.16:0/3476368847 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fee900a2cb0 msgr2=0x7fee900a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:02.290 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.288+0000 7feea8c9b640 1 --2- 172.21.15.16:0/3476368847 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fee900a2cb0 0x7fee900a5170 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fee9c09be80 tx=0x7fee9c066ac0 comp rx=0 tx=0).stop 2024-09-12T02:12:02.290 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.288+0000 7feea8c9b640 1 -- 172.21.15.16:0/3476368847 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feea4102600 msgr2=0x7feea419f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:02.290 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.288+0000 7feea8c9b640 1 --2- 172.21.15.16:0/3476368847 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feea4102600 0x7feea419f380 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7fee9c00cce0 tx=0x7fee9c00a620 comp rx=0 tx=0).stop 2024-09-12T02:12:02.291 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.288+0000 7feea8c9b640 1 -- 172.21.15.16:0/3476368847 shutdown_connections 2024-09-12T02:12:02.291 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.288+0000 7feea8c9b640 1 --2- 172.21.15.16:0/3476368847 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fee900a2cb0 0x7fee900a5170 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:02.291 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.288+0000 7feea8c9b640 1 --2- 172.21.15.16:0/3476368847 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feea4102600 0x7feea419f380 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:02.291 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.288+0000 7feea8c9b640 1 --2- 172.21.15.16:0/3476368847 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feea4101c40 0x7feea4198310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:02.291 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.288+0000 7feea8c9b640 1 --2- 172.21.15.16:0/3476368847 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feea4100a50 0x7feea4197dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:02.291 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.288+0000 7feea8c9b640 1 -- 172.21.15.16:0/3476368847 >> 172.21.15.16:0/3476368847 conn(0x7feea40fc1b0 msgr2=0x7feea4104ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:02.291 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.288+0000 7feea8c9b640 1 -- 172.21.15.16:0/3476368847 shutdown_connections 2024-09-12T02:12:02.291 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:02.288+0000 7feea8c9b640 1 -- 172.21.15.16:0/3476368847 wait complete. 2024-09-12T02:12:02.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:12:02 smithi033 ceph-mon[35332]: pgmap v638: 97 pgs: 97 active+clean; 1.2 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.3 KiB/s wr, 12 op/s 2024-09-12T02:12:02.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:12:02 smithi016 ceph-mon[30947]: pgmap v638: 97 pgs: 97 active+clean; 1.2 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.3 KiB/s wr, 12 op/s 2024-09-12T02:12:02.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:12:02 smithi184 ceph-mon[35593]: pgmap v638: 97 pgs: 97 active+clean; 1.2 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.3 KiB/s wr, 12 op/s 2024-09-12T02:12:03.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:12:03 smithi016 ceph-mon[30947]: pgmap v639: 97 pgs: 97 active+clean; 1.2 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 7.4 KiB/s wr, 10 op/s 2024-09-12T02:12:03.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:12:03 smithi184 ceph-mon[35593]: pgmap v639: 97 pgs: 97 active+clean; 1.2 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 7.4 KiB/s wr, 10 op/s 2024-09-12T02:12:03.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:12:03 smithi033 ceph-mon[35332]: pgmap v639: 97 pgs: 97 active+clean; 1.2 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 7.4 KiB/s wr, 10 op/s 2024-09-12T02:12:05.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:12:05 smithi184 ceph-mon[35593]: pgmap v640: 97 pgs: 97 active+clean; 1.2 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-09-12T02:12:05.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:12:05 smithi033 ceph-mon[35332]: pgmap v640: 97 pgs: 97 active+clean; 1.2 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-09-12T02:12:05.934 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:12:05 smithi016 ceph-mon[30947]: pgmap v640: 97 pgs: 97 active+clean; 1.2 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-09-12T02:12:07.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:12:07 smithi184 ceph-mon[35593]: pgmap v641: 97 pgs: 97 active+clean; 1.2 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 2.7 KiB/s wr, 10 op/s 2024-09-12T02:12:07.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:12:07 smithi033 ceph-mon[35332]: pgmap v641: 97 pgs: 97 active+clean; 1.2 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 2.7 KiB/s wr, 10 op/s 2024-09-12T02:12:07.933 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:12:07 smithi016 ceph-mon[30947]: pgmap v641: 97 pgs: 97 active+clean; 1.2 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 2.7 KiB/s wr, 10 op/s 2024-09-12T02:12:08.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:12:08 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:12:08.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:12:08 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:12:08.933 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:12:08 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:12:09.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:12:09 smithi184 ceph-mon[35593]: pgmap v642: 97 pgs: 97 active+clean; 1.2 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 2.4 KiB/s wr, 9 op/s 2024-09-12T02:12:09.918 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:12:09 smithi033 ceph-mon[35332]: pgmap v642: 97 pgs: 97 active+clean; 1.2 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 2.4 KiB/s wr, 9 op/s 2024-09-12T02:12:09.933 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:12:09 smithi016 ceph-mon[30947]: pgmap v642: 97 pgs: 97 active+clean; 1.2 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 2.4 KiB/s wr, 9 op/s 2024-09-12T02:12:10.207 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:12:10.469 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:12:11.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:10.999+0000 7fee42d1e640 1 -- 172.21.15.16:0/494517232 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee3c102d30 msgr2=0x7fee3c103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:11.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:10.999+0000 7fee42d1e640 1 --2- 172.21.15.16:0/494517232 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee3c102d30 0x7fee3c103110 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7fee30009930 tx=0x7fee3002a420 comp rx=0 tx=0).stop 2024-09-12T02:12:11.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:10.999+0000 7fee42d1e640 1 -- 172.21.15.16:0/494517232 shutdown_connections 2024-09-12T02:12:11.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:10.999+0000 7fee42d1e640 1 --2- 172.21.15.16:0/494517232 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fee3c1082a0 0x7fee3c10ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:11.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:10.999+0000 7fee42d1e640 1 --2- 172.21.15.16:0/494517232 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fee3c1036e0 0x7fee3c107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:11.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:10.999+0000 7fee42d1e640 1 --2- 172.21.15.16:0/494517232 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee3c102d30 0x7fee3c103110 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:11.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:10.999+0000 7fee42d1e640 1 -- 172.21.15.16:0/494517232 >> 172.21.15.16:0/494517232 conn(0x7fee3c0fc7d0 msgr2=0x7fee3c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:11.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:10.999+0000 7fee42d1e640 1 -- 172.21.15.16:0/494517232 shutdown_connections 2024-09-12T02:12:11.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:10.999+0000 7fee42d1e640 1 -- 172.21.15.16:0/494517232 wait complete. 2024-09-12T02:12:11.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:10.999+0000 7fee42d1e640 1 Processor -- start 2024-09-12T02:12:11.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:10.999+0000 7fee42d1e640 1 -- start start 2024-09-12T02:12:11.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:11.000+0000 7fee42d1e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fee3c102d30 0x7fee3c100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:11.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:11.000+0000 7fee42d1e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee3c1036e0 0x7fee3c100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:11.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:11.000+0000 7fee42d1e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fee3c1082a0 0x7fee3c101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:11.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:11.000+0000 7fee42d1e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fee3c1120a0 con 0x7fee3c1036e0 2024-09-12T02:12:11.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:11.000+0000 7fee42d1e640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fee3c111f20 con 0x7fee3c102d30 2024-09-12T02:12:11.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:11.000+0000 7fee42d1e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fee3c112220 con 0x7fee3c1082a0 2024-09-12T02:12:11.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:11.000+0000 7fee3bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee3c1036e0 0x7fee3c100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:11.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:11.000+0000 7fee3bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee3c1036e0 0x7fee3c100ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38180/0 (socket says 172.21.15.16:38180) 2024-09-12T02:12:11.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:11.000+0000 7fee3bfff640 1 -- 172.21.15.16:0/4091268491 learned_addr learned my addr 172.21.15.16:0/4091268491 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:12:11.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:11.000+0000 7fee40aba640 1 --2- 172.21.15.16:0/4091268491 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fee3c102d30 0x7fee3c100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:11.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:11.000+0000 7fee412bb640 1 --2- 172.21.15.16:0/4091268491 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fee3c1082a0 0x7fee3c101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:11.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:11.000+0000 7fee3bfff640 1 -- 172.21.15.16:0/4091268491 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fee3c1082a0 msgr2=0x7fee3c101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:11.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:11.000+0000 7fee3bfff640 1 --2- 172.21.15.16:0/4091268491 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fee3c1082a0 0x7fee3c101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:11.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:11.000+0000 7fee3bfff640 1 -- 172.21.15.16:0/4091268491 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fee3c102d30 msgr2=0x7fee3c100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:11.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:11.000+0000 7fee3bfff640 1 --2- 172.21.15.16:0/4091268491 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fee3c102d30 0x7fee3c100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:11.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:11.000+0000 7fee3bfff640 1 -- 172.21.15.16:0/4091268491 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee3c1a6a00 con 0x7fee3c1036e0 2024-09-12T02:12:11.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:11.000+0000 7fee3bfff640 1 --2- 172.21.15.16:0/4091268491 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee3c1036e0 0x7fee3c100ed0 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7fee2800c8c0 tx=0x7fee2800cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:11.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:11.000+0000 7fee39ffb640 1 -- 172.21.15.16:0/4091268491 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee28007bb0 con 0x7fee3c1036e0 2024-09-12T02:12:11.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:11.001+0000 7fee39ffb640 1 -- 172.21.15.16:0/4091268491 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee28007d50 con 0x7fee3c1036e0 2024-09-12T02:12:11.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:11.001+0000 7fee42d1e640 1 -- 172.21.15.16:0/4091268491 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee3c1a6cf0 con 0x7fee3c1036e0 2024-09-12T02:12:11.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:11.001+0000 7fee39ffb640 1 -- 172.21.15.16:0/4091268491 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee28018830 con 0x7fee3c1036e0 2024-09-12T02:12:11.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:11.001+0000 7fee42d1e640 1 -- 172.21.15.16:0/4091268491 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee3c1a71d0 con 0x7fee3c1036e0 2024-09-12T02:12:11.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:11.002+0000 7fee42d1e640 1 -- 172.21.15.16:0/4091268491 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee3c06a850 con 0x7fee3c1036e0 2024-09-12T02:12:11.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:11.003+0000 7fee39ffb640 1 -- 172.21.15.16:0/4091268491 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fee28018a10 con 0x7fee3c1036e0 2024-09-12T02:12:11.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:11.004+0000 7fee39ffb640 1 --2- 172.21.15.16:0/4091268491 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fee2c0a2b80 0x7fee2c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:11.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:11.004+0000 7fee40aba640 1 --2- 172.21.15.16:0/4091268491 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fee2c0a2b80 0x7fee2c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:11.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:11.004+0000 7fee39ffb640 1 -- 172.21.15.16:0/4091268491 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fee280ceff0 con 0x7fee3c1036e0 2024-09-12T02:12:11.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:11.004+0000 7fee40aba640 1 --2- 172.21.15.16:0/4091268491 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fee2c0a2b80 0x7fee2c0a5040 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fee30009900 tx=0x7fee300058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:11.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:11.008+0000 7fee39ffb640 1 -- 172.21.15.16:0/4091268491 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fee28014030 con 0x7fee3c1036e0 2024-09-12T02:12:11.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:12:10 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T02:12:11.128 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:12:10 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T02:12:11.128 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:12:10 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T02:12:11.128 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:12:10 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T02:12:11.128 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:12:10 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T02:12:11.128 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:12:10 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T02:12:11.128 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:12:10 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:12:11.128 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:12:10 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:12:11.128 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:12:10 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T02:12:11.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:12:10 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T02:12:11.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:12:10 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T02:12:11.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:12:10 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T02:12:11.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:12:10 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T02:12:11.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:12:10 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T02:12:11.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:12:10 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T02:12:11.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:12:10 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:12:11.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:12:10 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:12:11.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:12:10 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T02:12:11.183 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:12:10 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T02:12:11.183 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:12:10 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T02:12:11.183 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:12:10 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T02:12:11.183 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:12:10 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T02:12:11.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:12:10 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T02:12:11.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:12:10 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T02:12:11.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:12:10 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:12:11.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:12:10 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:12:11.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:12:10 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T02:12:11.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:11.222+0000 7fee42d1e640 1 -- 172.21.15.16:0/4091268491 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fee3c02d7c0 con 0x7fee3c1036e0 2024-09-12T02:12:11.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:11.223+0000 7fee39ffb640 1 -- 172.21.15.16:0/4091268491 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11911 (secure 0 0 0) 0x7fee2809bff0 con 0x7fee3c1036e0 2024-09-12T02:12:11.224 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T02:12:11.224 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":25,"btime":"2024-09-12T02:01:37:192021+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3279123279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3279123279},{"type":"v1","addr":"172.21.15.16:6835","nonce":3279123279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4119452382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4119452382},{"type":"v1","addr":"172.21.15.16:6837","nonce":4119452382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/3263823165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3263823165},{"type":"v1","addr":"172.21.15.16:6839","nonce":3263823165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/1510290732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":1510290732},{"type":"v1","addr":"172.21.15.16:6841","nonce":1510290732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/279674603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":279674603},{"type":"v1","addr":"172.21.15.33:6833","nonce":279674603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3555728028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3555728028},{"type":"v1","addr":"172.21.15.33:6835","nonce":3555728028}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T01:59:56.117608+0000","modified":"2024-09-12T02:01:36.294760+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24482,"mds_1":24481,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6833/1563374106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1563374106},{"type":"v1","addr":"172.21.15.184:6833","nonce":1563374106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6837/1892833868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1892833868},{"type":"v1","addr":"172.21.15.33:6837","nonce":1892833868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/1457679686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1457679686},{"type":"v1","addr":"172.21.15.184:6835","nonce":1457679686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.184:6837/1793393087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":1793393087},{"type":"v1","addr":"172.21.15.184:6837","nonce":1793393087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.184:6839/4203347941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":4203347941},{"type":"v1","addr":"172.21.15.184:6839","nonce":4203347941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.33:6839/4136670204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":4136670204},{"type":"v1","addr":"172.21.15.33:6839","nonce":4136670204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24481,24482,24469]},"id":1}]} 2024-09-12T02:12:11.224 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 25 2024-09-12T02:12:11.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:11.225+0000 7fee42d1e640 1 -- 172.21.15.16:0/4091268491 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fee2c0a2b80 msgr2=0x7fee2c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:11.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:11.225+0000 7fee42d1e640 1 --2- 172.21.15.16:0/4091268491 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fee2c0a2b80 0x7fee2c0a5040 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fee30009900 tx=0x7fee300058d0 comp rx=0 tx=0).stop 2024-09-12T02:12:11.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:11.226+0000 7fee42d1e640 1 -- 172.21.15.16:0/4091268491 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee3c1036e0 msgr2=0x7fee3c100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:11.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:11.226+0000 7fee42d1e640 1 --2- 172.21.15.16:0/4091268491 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee3c1036e0 0x7fee3c100ed0 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7fee2800c8c0 tx=0x7fee2800cd80 comp rx=0 tx=0).stop 2024-09-12T02:12:11.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:11.226+0000 7fee42d1e640 1 -- 172.21.15.16:0/4091268491 shutdown_connections 2024-09-12T02:12:11.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:11.226+0000 7fee42d1e640 1 --2- 172.21.15.16:0/4091268491 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fee2c0a2b80 0x7fee2c0a5040 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:11.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:11.226+0000 7fee42d1e640 1 --2- 172.21.15.16:0/4091268491 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fee3c1082a0 0x7fee3c101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:11.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:11.226+0000 7fee42d1e640 1 --2- 172.21.15.16:0/4091268491 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee3c1036e0 0x7fee3c100ed0 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:11.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:11.226+0000 7fee42d1e640 1 --2- 172.21.15.16:0/4091268491 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fee3c102d30 0x7fee3c100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:11.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:11.226+0000 7fee42d1e640 1 -- 172.21.15.16:0/4091268491 >> 172.21.15.16:0/4091268491 conn(0x7fee3c0fc7d0 msgr2=0x7fee3c105d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:11.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:11.226+0000 7fee42d1e640 1 -- 172.21.15.16:0/4091268491 shutdown_connections 2024-09-12T02:12:11.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:11.226+0000 7fee42d1e640 1 -- 172.21.15.16:0/4091268491 wait complete. 2024-09-12T02:12:11.889 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:12:11 smithi016 ceph-mon[30947]: pgmap v643: 97 pgs: 97 active+clean; 1.2 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.4 KiB/s wr, 10 op/s 2024-09-12T02:12:11.889 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:12:11 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/4091268491' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:12:11.891 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24482 quiesce db --set-id 44342e0a --release --await' 2024-09-12T02:12:11.891 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24482 quiesce db --set-id 44342e0a --release --await 2024-09-12T02:12:12.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:12:11 smithi184 ceph-mon[35593]: pgmap v643: 97 pgs: 97 active+clean; 1.2 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.4 KiB/s wr, 10 op/s 2024-09-12T02:12:12.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:12:11 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/4091268491' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:12:12.151 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:12:12.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:12:11 smithi033 ceph-mon[35332]: pgmap v643: 97 pgs: 97 active+clean; 1.2 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.4 KiB/s wr, 10 op/s 2024-09-12T02:12:12.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:12:11 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/4091268491' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:12:12.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.621+0000 7f629b50e640 1 -- 172.21.15.16:0/3754122800 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6294102570 msgr2=0x7f62941029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:12.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.621+0000 7f629b50e640 1 --2- 172.21.15.16:0/3754122800 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6294102570 0x7f62941029b0 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f6288009900 tx=0x7f628802a480 comp rx=0 tx=0).stop 2024-09-12T02:12:12.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.621+0000 7f629b50e640 1 -- 172.21.15.16:0/3754122800 shutdown_connections 2024-09-12T02:12:12.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.621+0000 7f629b50e640 1 --2- 172.21.15.16:0/3754122800 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6294102ef0 0x7f629410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:12.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.621+0000 7f629b50e640 1 --2- 172.21.15.16:0/3754122800 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6294102570 0x7f62941029b0 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:12.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.621+0000 7f629b50e640 1 --2- 172.21.15.16:0/3754122800 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6294100e20 0x7f6294101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:12.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.621+0000 7f629b50e640 1 -- 172.21.15.16:0/3754122800 >> 172.21.15.16:0/3754122800 conn(0x7f62940fc7d0 msgr2=0x7f62940febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:12.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.621+0000 7f629b50e640 1 -- 172.21.15.16:0/3754122800 shutdown_connections 2024-09-12T02:12:12.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.621+0000 7f629b50e640 1 -- 172.21.15.16:0/3754122800 wait complete. 2024-09-12T02:12:12.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.621+0000 7f629b50e640 1 Processor -- start 2024-09-12T02:12:12.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.621+0000 7f629b50e640 1 -- start start 2024-09-12T02:12:12.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.622+0000 7f629b50e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6294100e20 0x7f6294199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:12.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.622+0000 7f629b50e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6294102570 0x7f62941997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:12.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.622+0000 7f629b50e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6294102ef0 0x7f62941a42e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:12.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.622+0000 7f629b50e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6294110f80 con 0x7f6294102570 2024-09-12T02:12:12.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.622+0000 7f629b50e640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6294110e00 con 0x7f6294102ef0 2024-09-12T02:12:12.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.622+0000 7f629b50e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f6294111100 con 0x7f6294100e20 2024-09-12T02:12:12.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.622+0000 7f6298aa9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6294102570 0x7f62941997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:12.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.622+0000 7f6298aa9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6294102570 0x7f62941997a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38512/0 (socket says 172.21.15.16:38512) 2024-09-12T02:12:12.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.622+0000 7f6298aa9640 1 -- 172.21.15.16:0/1115919388 learned_addr learned my addr 172.21.15.16:0/1115919388 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:12:12.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.622+0000 7f62992aa640 1 --2- 172.21.15.16:0/1115919388 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6294100e20 0x7f6294199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:12.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.622+0000 7f6299aab640 1 --2- 172.21.15.16:0/1115919388 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6294102ef0 0x7f62941a42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:12.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.622+0000 7f6298aa9640 1 -- 172.21.15.16:0/1115919388 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6294100e20 msgr2=0x7f6294199260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:12.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.622+0000 7f6298aa9640 1 --2- 172.21.15.16:0/1115919388 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6294100e20 0x7f6294199260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:12.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.622+0000 7f6298aa9640 1 -- 172.21.15.16:0/1115919388 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6294102ef0 msgr2=0x7f62941a42e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:12.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.622+0000 7f6298aa9640 1 --2- 172.21.15.16:0/1115919388 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6294102ef0 0x7f62941a42e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:12.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.622+0000 7f6298aa9640 1 -- 172.21.15.16:0/1115919388 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62941a4820 con 0x7f6294102570 2024-09-12T02:12:12.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.622+0000 7f6298aa9640 1 --2- 172.21.15.16:0/1115919388 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6294102570 0x7f62941997a0 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f6288006010 tx=0x7f6288003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:12.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.623+0000 7f62927fc640 1 -- 172.21.15.16:0/1115919388 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f628803f070 con 0x7f6294102570 2024-09-12T02:12:12.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.623+0000 7f62927fc640 1 -- 172.21.15.16:0/1115919388 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6288031c30 con 0x7f6294102570 2024-09-12T02:12:12.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.623+0000 7f629b50e640 1 -- 172.21.15.16:0/1115919388 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62941a49f0 con 0x7f6294102570 2024-09-12T02:12:12.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.623+0000 7f62927fc640 1 -- 172.21.15.16:0/1115919388 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6288035030 con 0x7f6294102570 2024-09-12T02:12:12.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.623+0000 7f629b50e640 1 -- 172.21.15.16:0/1115919388 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62941a4ed0 con 0x7f6294102570 2024-09-12T02:12:12.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.626+0000 7f62927fc640 1 -- 172.21.15.16:0/1115919388 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6288042070 con 0x7f6294102570 2024-09-12T02:12:12.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.626+0000 7f629d8e7740 1 Processor -- start 2024-09-12T02:12:12.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.626+0000 7f629d8e7740 1 -- start start 2024-09-12T02:12:12.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.626+0000 7f629d8e7740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b845227060 0x55b845227440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:12.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.626+0000 7f629d8e7740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b845227980 0x55b845231d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:12.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.626+0000 7f629d8e7740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b845232240 0x55b845234630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:12.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.626+0000 7f629d8e7740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55b84518c620 con 0x55b845227060 2024-09-12T02:12:12.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.626+0000 7f629d8e7740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55b845133e50 con 0x55b845227980 2024-09-12T02:12:12.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.626+0000 7f629d8e7740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55b8451885e0 con 0x55b845232240 2024-09-12T02:12:12.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.626+0000 7f6299aab640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b845227060 0x55b845227440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:12.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.627+0000 7f6299aab640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b845227060 0x55b845227440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38524/0 (socket says 172.21.15.16:38524) 2024-09-12T02:12:12.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.627+0000 7f6299aab640 1 -- 172.21.15.16:0/3790868130 learned_addr learned my addr 172.21.15.16:0/3790868130 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:12:12.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.627+0000 7f62992aa640 1 --2- 172.21.15.16:0/3790868130 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b845227980 0x55b845231d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:12.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.627+0000 7f6298aa9640 1 --2- 172.21.15.16:0/3790868130 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b845232240 0x55b845234630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:12.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.627+0000 7f62927fc640 1 --2- 172.21.15.16:0/1115919388 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f627c0a2c00 0x7f627c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:12.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.627+0000 7f6299aab640 1 -- 172.21.15.16:0/3790868130 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b845232240 msgr2=0x55b845234630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:12.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.627+0000 7f6299aab640 1 --2- 172.21.15.16:0/3790868130 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b845232240 0x55b845234630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:12.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.627+0000 7f6299aab640 1 -- 172.21.15.16:0/3790868130 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b845227980 msgr2=0x55b845231d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:12.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.627+0000 7f6299aab640 1 --2- 172.21.15.16:0/3790868130 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b845227980 0x55b845231d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:12.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.627+0000 7f6299aab640 1 -- 172.21.15.16:0/3790868130 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b845124d40 con 0x55b845227060 2024-09-12T02:12:12.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.627+0000 7f62927fc640 1 -- 172.21.15.16:0/1115919388 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f62880ef840 con 0x7f6294102570 2024-09-12T02:12:12.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.627+0000 7f6299aab640 1 --2- 172.21.15.16:0/3790868130 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b845227060 0x55b845227440 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f6284007dd0 tx=0x7f628400e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e553212ed406204e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:12.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.627+0000 7f6299aab640 1 --2- 172.21.15.16:0/1115919388 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f627c0a2c00 0x7f627c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:12.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.627+0000 7f627b7fe640 1 -- 172.21.15.16:0/3790868130 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6284014070 con 0x55b845227060 2024-09-12T02:12:12.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.627+0000 7f627b7fe640 1 -- 172.21.15.16:0/3790868130 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6284010430 con 0x55b845227060 2024-09-12T02:12:12.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.627+0000 7f627b7fe640 1 -- 172.21.15.16:0/3790868130 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62840187a0 con 0x55b845227060 2024-09-12T02:12:12.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.628+0000 7f629d8e7740 1 -- 172.21.15.16:0/3790868130 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b845227060 msgr2=0x55b845227440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:12.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.628+0000 7f6299aab640 1 --2- 172.21.15.16:0/1115919388 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f627c0a2c00 0x7f627c0a50c0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f6284018000 tx=0x7f6284018ec0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:12.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.628+0000 7f629d8e7740 1 --2- 172.21.15.16:0/3790868130 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b845227060 0x55b845227440 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f6284007dd0 tx=0x7f628400e720 comp rx=0 tx=0).stop 2024-09-12T02:12:12.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.628+0000 7f629d8e7740 1 -- 172.21.15.16:0/3790868130 shutdown_connections 2024-09-12T02:12:12.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.628+0000 7f629d8e7740 1 --2- 172.21.15.16:0/3790868130 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b845232240 0x55b845234630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:12.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.628+0000 7f629d8e7740 1 --2- 172.21.15.16:0/3790868130 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b845227980 0x55b845231d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:12.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.628+0000 7f629d8e7740 1 --2- 172.21.15.16:0/3790868130 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b845227060 0x55b845227440 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:12.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.628+0000 7f629d8e7740 1 -- 172.21.15.16:0/3790868130 >> 172.21.15.16:0/3790868130 conn(0x55b8452235f0 msgr2=0x55b8452239d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:12.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.628+0000 7f629d8e7740 1 -- 172.21.15.16:0/3790868130 shutdown_connections 2024-09-12T02:12:12.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.628+0000 7f629d8e7740 1 -- 172.21.15.16:0/3790868130 wait complete. 2024-09-12T02:12:12.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.629+0000 7f629d8e7740 1 -- start start 2024-09-12T02:12:12.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.629+0000 7f629d8e7740 1 Processor -- start 2024-09-12T02:12:12.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.629+0000 7f629d8e7740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b845227060 0x55b84522f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:12.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.629+0000 7f629d8e7740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b845227980 0x55b84522f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:12.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.629+0000 7f629d8e7740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b845232240 0x55b84522fb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:12.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.629+0000 7f629d8e7740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55b845234e10 con 0x55b845227980 2024-09-12T02:12:12.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.629+0000 7f629d8e7740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55b845114f10 con 0x55b845227060 2024-09-12T02:12:12.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.629+0000 7f629d8e7740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55b845234f90 con 0x55b845232240 2024-09-12T02:12:12.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.629+0000 7f629d8e7740 10 client.0 _ll_register_callbacks cb 0x55b845138db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:12:12.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.629+0000 7f6298aa9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b845227980 0x55b84522f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:12.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.629+0000 7f6298aa9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b845227980 0x55b84522f5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38540/0 (socket says 172.21.15.16:38540) 2024-09-12T02:12:12.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.629+0000 7f6298aa9640 1 -- 172.21.15.16:0/1075281870 learned_addr learned my addr 172.21.15.16:0/1075281870 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:12:12.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.629+0000 7f62992aa640 1 --2- 172.21.15.16:0/1075281870 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b845227060 0x55b84522f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:12.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.629+0000 7f6299aab640 1 --2- 172.21.15.16:0/1075281870 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b845232240 0x55b84522fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:12.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.630+0000 7f6298aa9640 1 -- 172.21.15.16:0/1075281870 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b845232240 msgr2=0x55b84522fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:12.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.630+0000 7f6298aa9640 1 --2- 172.21.15.16:0/1075281870 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b845232240 0x55b84522fb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:12.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.630+0000 7f6298aa9640 1 -- 172.21.15.16:0/1075281870 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b845227060 msgr2=0x55b84522f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:12.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.630+0000 7f6298aa9640 1 --2- 172.21.15.16:0/1075281870 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b845227060 0x55b84522f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:12.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.630+0000 7f6298aa9640 1 -- 172.21.15.16:0/1075281870 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b844e9b580 con 0x55b845227980 2024-09-12T02:12:12.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.630+0000 7f6298aa9640 1 --2- 172.21.15.16:0/1075281870 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b845227980 0x55b84522f5c0 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f62880b8b10 tx=0x7f62880bbfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd95e8f2238c8f52 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:12.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.630+0000 7f6278ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:12:12.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.630+0000 7f6278ff9640 1 -- 172.21.15.16:0/1075281870 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6288049070 con 0x55b845227980 2024-09-12T02:12:12.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.630+0000 7f6278ff9640 1 -- 172.21.15.16:0/1075281870 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f62880b87b0 con 0x55b845227980 2024-09-12T02:12:12.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.630+0000 7f629d8e7740 1 -- 172.21.15.16:0/1075281870 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6284010430 con 0x55b845227980 2024-09-12T02:12:12.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.630+0000 7f6278ff9640 1 -- 172.21.15.16:0/1075281870 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f62880bc840 con 0x55b845227980 2024-09-12T02:12:12.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.630+0000 7f629d8e7740 10 client.18645 fetch_fsmap learned FSMap version 25 2024-09-12T02:12:12.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.630+0000 7f629d8e7740 1 -- 172.21.15.16:0/1075281870 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b844f65960 con 0x55b845227980 2024-09-12T02:12:12.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.630+0000 7f6278ff9640 1 -- 172.21.15.16:0/1075281870 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6284010430 con 0x55b845227980 2024-09-12T02:12:12.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.631+0000 7f6278ff9640 1 -- 172.21.15.16:0/1075281870 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f62880c0130 con 0x55b845227980 2024-09-12T02:12:12.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.631+0000 7f629d8e7740 10 client.18645 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:12:12.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.631+0000 7f629d8e7740 10 client.18645 resolve_mds: validated gid 24482 aka daemon mds.b 2024-09-12T02:12:12.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.631+0000 7f629d8e7740 20 client.18645 populate_metadata read hostname 'smithi016' 2024-09-12T02:12:12.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.631+0000 7f629d8e7740 1 --2- 172.21.15.16:0/1075281870 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55b845241bc0 0x55b845243fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:12.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.631+0000 7f629d8e7740 4 client.18645 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:12:12.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.631+0000 7f629d8e7740 1 -- 172.21.15.16:0/1075281870 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b845199b90 con 0x55b845241bc0 2024-09-12T02:12:12.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.631+0000 7f62992aa640 1 --2- 172.21.15.16:0/1075281870 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55b845241bc0 0x55b845243fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:12.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.631+0000 7f6278ff9640 10 client.18645 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:12:12.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.631+0000 7f62992aa640 1 --2- 172.21.15.16:0/1075281870 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55b845241bc0 0x55b845243fd0 crc :-1 s=READY pgs=304 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d03a5df80ef877c4 server_cookie=c27cc1ac0f2e57b4 in_seq=0 out_seq=0 2024-09-12T02:12:12.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.632+0000 7f6278ff9640 1 -- 172.21.15.16:0/1075281870 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b845199b90 con 0x55b845241bc0 2024-09-12T02:12:12.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.632+0000 7f6278ff9640 10 client.18645 handle_command_reply: tid=0 2024-09-12T02:12:12.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.632+0000 7f629d8e7740 1 client.18645 shutdown 2024-09-12T02:12:12.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.632+0000 7f629d8e7740 1 -- 172.21.15.16:0/1075281870 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b845227980 msgr2=0x55b84522f5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:12.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.632+0000 7f629d8e7740 1 --2- 172.21.15.16:0/1075281870 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b845227980 0x55b84522f5c0 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f62880b8b10 tx=0x7f62880bbfd0 comp rx=0 tx=0).stop 2024-09-12T02:12:12.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.632+0000 7f629d8e7740 1 -- 172.21.15.16:0/1075281870 shutdown_connections 2024-09-12T02:12:12.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.632+0000 7f629d8e7740 1 --2- 172.21.15.16:0/1075281870 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55b845241bc0 0x55b845243fd0 crc :-1 s=READY pgs=304 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:12.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.632+0000 7f629d8e7740 1 --2- 172.21.15.16:0/1075281870 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b845232240 0x55b84522fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:12.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.632+0000 7f629d8e7740 1 --2- 172.21.15.16:0/1075281870 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b845227980 0x55b84522f5c0 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:12.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.632+0000 7f629d8e7740 1 --2- 172.21.15.16:0/1075281870 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b845227060 0x55b84522f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:12.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.632+0000 7f629d8e7740 1 -- 172.21.15.16:0/1075281870 >> 172.21.15.16:0/1075281870 conn(0x55b8452235f0 msgr2=0x55b845222750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:12.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.632+0000 7f6278ff9640 0 client.18645 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:12:12.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.633+0000 7f629d8e7740 1 -- 172.21.15.16:0/1075281870 shutdown_connections 2024-09-12T02:12:12.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.633+0000 7f629d8e7740 1 -- 172.21.15.16:0/1075281870 wait complete. 2024-09-12T02:12:12.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.633+0000 7f629d8e7740 20 client.18645 trim_cache size 0 max 16384 2024-09-12T02:12:12.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.650+0000 7f629d8e7740 1 Processor -- start 2024-09-12T02:12:12.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.650+0000 7f629d8e7740 1 -- start start 2024-09-12T02:12:12.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.650+0000 7f629d8e7740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b845227060 0x55b8452498a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:12.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.650+0000 7f629d8e7740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b845227980 0x55b845249de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:12.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.650+0000 7f629d8e7740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b845232240 0x55b84522c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:12.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.650+0000 7f629d8e7740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f62941b1eb0 con 0x55b845227980 2024-09-12T02:12:12.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.650+0000 7f629d8e7740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f62941af780 con 0x55b845232240 2024-09-12T02:12:12.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.650+0000 7f629d8e7740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f62941ad5f0 con 0x55b845227060 2024-09-12T02:12:12.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.650+0000 7f6298aa9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b845227980 0x55b845249de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:12.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.650+0000 7f6298aa9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b845227980 0x55b845249de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38542/0 (socket says 172.21.15.16:38542) 2024-09-12T02:12:12.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.650+0000 7f6298aa9640 1 -- 172.21.15.16:0/3360837397 learned_addr learned my addr 172.21.15.16:0/3360837397 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:12:12.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.650+0000 7f62992aa640 1 --2- 172.21.15.16:0/3360837397 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b845227060 0x55b8452498a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:12.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.650+0000 7f6298aa9640 1 -- 172.21.15.16:0/3360837397 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b845227060 msgr2=0x55b8452498a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:12.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.650+0000 7f6298aa9640 1 --2- 172.21.15.16:0/3360837397 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b845227060 0x55b8452498a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:12.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.650+0000 7f6298aa9640 1 -- 172.21.15.16:0/3360837397 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b845232240 msgr2=0x55b84522c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:12.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.650+0000 7f6298aa9640 1 --2- 172.21.15.16:0/3360837397 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b845232240 0x55b84522c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:12.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.650+0000 7f6298aa9640 1 -- 172.21.15.16:0/3360837397 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b845223420 con 0x55b845227980 2024-09-12T02:12:12.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.650+0000 7f6298aa9640 1 --2- 172.21.15.16:0/3360837397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b845227980 0x55b845249de0 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f62880b85a0 tx=0x7f62880b83e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec4e2452cb307bdf server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:12.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.651+0000 7f627affd640 1 -- 172.21.15.16:0/3360837397 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6288049070 con 0x55b845227980 2024-09-12T02:12:12.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.651+0000 7f627affd640 1 -- 172.21.15.16:0/3360837397 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62880c04f0 con 0x55b845227980 2024-09-12T02:12:12.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.651+0000 7f627affd640 1 -- 172.21.15.16:0/3360837397 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62880bc3f0 con 0x55b845227980 2024-09-12T02:12:12.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.651+0000 7f629d8e7740 1 -- 172.21.15.16:0/3360837397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b845227980 msgr2=0x55b845249de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:12.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.651+0000 7f629d8e7740 1 --2- 172.21.15.16:0/3360837397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b845227980 0x55b845249de0 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f62880b85a0 tx=0x7f62880b83e0 comp rx=0 tx=0).stop 2024-09-12T02:12:12.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.651+0000 7f629d8e7740 1 -- 172.21.15.16:0/3360837397 shutdown_connections 2024-09-12T02:12:12.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.651+0000 7f629d8e7740 1 --2- 172.21.15.16:0/3360837397 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b845232240 0x55b84522c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:12.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.651+0000 7f629d8e7740 1 --2- 172.21.15.16:0/3360837397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b845227980 0x55b845249de0 unknown :-1 s=CLOSED pgs=1449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:12.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.651+0000 7f629d8e7740 1 --2- 172.21.15.16:0/3360837397 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b845227060 0x55b8452498a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:12.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.651+0000 7f629d8e7740 1 -- 172.21.15.16:0/3360837397 >> 172.21.15.16:0/3360837397 conn(0x55b8452235f0 msgr2=0x55b84522d950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:12.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.652+0000 7f629d8e7740 1 -- 172.21.15.16:0/3360837397 shutdown_connections 2024-09-12T02:12:12.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.652+0000 7f629d8e7740 1 -- 172.21.15.16:0/3360837397 wait complete. 2024-09-12T02:12:12.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.652+0000 7f629d8e7740 1 -- start start 2024-09-12T02:12:12.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.652+0000 7f629d8e7740 1 Processor -- start 2024-09-12T02:12:12.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.652+0000 7f629d8e7740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b845227060 0x55b8452366b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:12.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.652+0000 7f629d8e7740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b845227980 0x55b845236bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:12.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.652+0000 7f629d8e7740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b845232240 0x55b8452ab870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:12.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.652+0000 7f629d8e7740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f62941adc40 con 0x55b845227060 2024-09-12T02:12:12.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.653+0000 7f629d8e7740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f62941af0b0 con 0x55b845227980 2024-09-12T02:12:12.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.653+0000 7f629d8e7740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f62941ae310 con 0x55b845232240 2024-09-12T02:12:12.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.653+0000 7f629d8e7740 10 client.0 _ll_register_callbacks cb 0x7f6288009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:12:12.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.653+0000 7f6298aa9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b845227980 0x55b845236bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:12.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.653+0000 7f62992aa640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b845227060 0x55b8452366b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:12.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.653+0000 7f6298aa9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b845227980 0x55b845236bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.16:50696/0 (socket says 172.21.15.16:50696) 2024-09-12T02:12:12.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.653+0000 7f62992aa640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b845227060 0x55b8452366b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38548/0 (socket says 172.21.15.16:38548) 2024-09-12T02:12:12.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.653+0000 7f6298aa9640 1 -- 172.21.15.16:0/1442400053 learned_addr learned my addr 172.21.15.16:0/1442400053 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:12:12.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.653+0000 7f6299aab640 1 --2- 172.21.15.16:0/1442400053 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b845232240 0x55b8452ab870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:12.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.653+0000 7f6298aa9640 1 -- 172.21.15.16:0/1442400053 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b845232240 msgr2=0x55b8452ab870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:12.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.653+0000 7f6298aa9640 1 --2- 172.21.15.16:0/1442400053 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b845232240 0x55b8452ab870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:12.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.653+0000 7f6298aa9640 1 -- 172.21.15.16:0/1442400053 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b845227060 msgr2=0x55b8452366b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:12.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.653+0000 7f6298aa9640 1 --2- 172.21.15.16:0/1442400053 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b845227060 0x55b8452366b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:12.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.653+0000 7f6298aa9640 1 -- 172.21.15.16:0/1442400053 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b8452241a0 con 0x55b845227980 2024-09-12T02:12:12.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.653+0000 7f62992aa640 1 --2- 172.21.15.16:0/1442400053 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b845227060 0x55b8452366b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:12:12.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.653+0000 7f6298aa9640 1 --2- 172.21.15.16:0/1442400053 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b845227980 0x55b845236bf0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f62880d52b0 tx=0x7f62880bc140 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d5cd143cbea67c6e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:12.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.653+0000 7f6278ff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-12T02:12:12.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.653+0000 7f6278ff9640 1 -- 172.21.15.16:0/1442400053 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6288049070 con 0x55b845227980 2024-09-12T02:12:12.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.653+0000 7f6278ff9640 1 -- 172.21.15.16:0/1442400053 <== mon.2 v2:172.21.15.33:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f62880dc0b0 con 0x55b845227980 2024-09-12T02:12:12.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.653+0000 7f629d8e7740 1 -- 172.21.15.16:0/1442400053 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f62880c04f0 con 0x55b845227980 2024-09-12T02:12:12.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.654+0000 7f6278ff9640 1 -- 172.21.15.16:0/1442400053 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f62880c4a10 con 0x55b845227980 2024-09-12T02:12:12.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.654+0000 7f629d8e7740 10 client.27755 fetch_fsmap learned FSMap version 25 2024-09-12T02:12:12.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.654+0000 7f629d8e7740 1 -- 172.21.15.16:0/1442400053 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b8452241a0 con 0x55b845227980 2024-09-12T02:12:12.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.654+0000 7f6278ff9640 1 -- 172.21.15.16:0/1442400053 <== mon.2 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62880c04f0 con 0x55b845227980 2024-09-12T02:12:12.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.654+0000 7f6278ff9640 1 -- 172.21.15.16:0/1442400053 <== mon.2 v2:172.21.15.33:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f62880bb590 con 0x55b845227980 2024-09-12T02:12:12.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.654+0000 7f629d8e7740 10 client.27755 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:12:12.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.654+0000 7f629d8e7740 10 client.27755 resolve_mds: validated gid 24482 aka daemon mds.b 2024-09-12T02:12:12.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.654+0000 7f629d8e7740 20 client.27755 populate_metadata read hostname 'smithi016' 2024-09-12T02:12:12.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.654+0000 7f629d8e7740 1 --2- 172.21.15.16:0/1442400053 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55b8452ad200 0x55b8452af620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:12.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.654+0000 7f629d8e7740 4 client.27755 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "44342e0a", "release": true, "await": true}] 2024-09-12T02:12:12.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.654+0000 7f629d8e7740 1 -- 172.21.15.16:0/1442400053 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "quiesce db", "set_id": "44342e0a", "release": true, "await": true}) -- 0x55b8451940a0 con 0x55b8452ad200 2024-09-12T02:12:12.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.654+0000 7f62992aa640 1 --2- 172.21.15.16:0/1442400053 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55b8452ad200 0x55b8452af620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:12.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.655+0000 7f62992aa640 1 --2- 172.21.15.16:0/1442400053 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55b8452ad200 0x55b8452af620 crc :-1 s=READY pgs=305 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85e3f5d65ab3e6bd server_cookie=d65ec146744e5e42 in_seq=0 out_seq=0 2024-09-12T02:12:12.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.655+0000 7f6278ff9640 10 client.27755 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:12:12.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.657+0000 7f6278ff9640 1 -- 172.21.15.16:0/1442400053 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55b8451940a0 con 0x55b8452ad200 2024-09-12T02:12:12.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.657+0000 7f6278ff9640 10 client.27755 handle_command_reply: tid=0 2024-09-12T02:12:12.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.657+0000 7f629d8e7740 1 client.27755 shutdown 2024-09-12T02:12:12.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.657+0000 7f629d8e7740 1 -- 172.21.15.16:0/1442400053 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b845227980 msgr2=0x55b845236bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:12.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.657+0000 7f629d8e7740 1 --2- 172.21.15.16:0/1442400053 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b845227980 0x55b845236bf0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f62880d52b0 tx=0x7f62880bc140 comp rx=0 tx=0).stop 2024-09-12T02:12:12.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.657+0000 7f629d8e7740 1 -- 172.21.15.16:0/1442400053 shutdown_connections 2024-09-12T02:12:12.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.658+0000 7f629d8e7740 1 --2- 172.21.15.16:0/1442400053 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55b8452ad200 0x55b8452af620 crc :-1 s=READY pgs=305 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:12.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.658+0000 7f629d8e7740 1 --2- 172.21.15.16:0/1442400053 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b845232240 0x55b8452ab870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:12.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.658+0000 7f629d8e7740 1 --2- 172.21.15.16:0/1442400053 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b845227980 0x55b845236bf0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:12.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.658+0000 7f6278ff9640 0 client.27755 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:12:12.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.658+0000 7f629d8e7740 1 --2- 172.21.15.16:0/1442400053 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b845227060 0x55b8452366b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:12.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.658+0000 7f629d8e7740 1 -- 172.21.15.16:0/1442400053 >> 172.21.15.16:0/1442400053 conn(0x55b8452235f0 msgr2=0x55b845221f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:12.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.658+0000 7f629d8e7740 1 -- 172.21.15.16:0/1442400053 shutdown_connections 2024-09-12T02:12:12.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.658+0000 7f629d8e7740 1 -- 172.21.15.16:0/1442400053 wait complete. 2024-09-12T02:12:12.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.658+0000 7f629d8e7740 20 client.27755 trim_cache size 0 max 16384 2024-09-12T02:12:12.661 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T02:12:12.661 INFO:teuthology.orchestra.run.smithi016.stdout: "epoch": 25, 2024-09-12T02:12:12.662 INFO:teuthology.orchestra.run.smithi016.stdout: "leader": 24482, 2024-09-12T02:12:12.662 INFO:teuthology.orchestra.run.smithi016.stdout: "set_version": 44, 2024-09-12T02:12:12.662 INFO:teuthology.orchestra.run.smithi016.stdout: "sets": { 2024-09-12T02:12:12.662 INFO:teuthology.orchestra.run.smithi016.stdout: "44342e0a": { 2024-09-12T02:12:12.662 INFO:teuthology.orchestra.run.smithi016.stdout: "version": 44, 2024-09-12T02:12:12.662 INFO:teuthology.orchestra.run.smithi016.stdout: "age_ref": 0.0, 2024-09-12T02:12:12.662 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-12T02:12:12.662 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RELEASED", 2024-09-12T02:12:12.662 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-12T02:12:12.662 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:12:12.662 INFO:teuthology.orchestra.run.smithi016.stdout: "timeout": 90.0, 2024-09-12T02:12:12.662 INFO:teuthology.orchestra.run.smithi016.stdout: "expiration": 142.6, 2024-09-12T02:12:12.662 INFO:teuthology.orchestra.run.smithi016.stdout: "members": { 2024-09-12T02:12:12.663 INFO:teuthology.orchestra.run.smithi016.stdout: "file:/": { 2024-09-12T02:12:12.663 INFO:teuthology.orchestra.run.smithi016.stdout: "excluded": false, 2024-09-12T02:12:12.663 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-12T02:12:12.663 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RELEASED", 2024-09-12T02:12:12.663 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-12T02:12:12.663 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:12:12.663 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:12:12.663 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:12:12.663 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:12:12.663 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:12:12.663 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T02:12:12.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.658+0000 7f629b50e640 1 -- 172.21.15.16:0/1115919388 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f627c0a2c00 msgr2=0x7f627c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:12.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.658+0000 7f629b50e640 1 --2- 172.21.15.16:0/1115919388 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f627c0a2c00 0x7f627c0a50c0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f6284018000 tx=0x7f6284018ec0 comp rx=0 tx=0).stop 2024-09-12T02:12:12.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.658+0000 7f629b50e640 1 -- 172.21.15.16:0/1115919388 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6294102570 msgr2=0x7f62941997a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:12.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.658+0000 7f629b50e640 1 --2- 172.21.15.16:0/1115919388 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6294102570 0x7f62941997a0 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f6288006010 tx=0x7f6288003c10 comp rx=0 tx=0).stop 2024-09-12T02:12:12.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.658+0000 7f629b50e640 1 -- 172.21.15.16:0/1115919388 shutdown_connections 2024-09-12T02:12:12.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.658+0000 7f629b50e640 1 --2- 172.21.15.16:0/1115919388 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f627c0a2c00 0x7f627c0a50c0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:12.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.658+0000 7f629b50e640 1 --2- 172.21.15.16:0/1115919388 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6294102ef0 0x7f62941a42e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:12.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.658+0000 7f629b50e640 1 --2- 172.21.15.16:0/1115919388 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6294102570 0x7f62941997a0 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:12.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.658+0000 7f629b50e640 1 --2- 172.21.15.16:0/1115919388 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6294100e20 0x7f6294199260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:12.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.658+0000 7f629b50e640 1 -- 172.21.15.16:0/1115919388 >> 172.21.15.16:0/1115919388 conn(0x7f62940fc7d0 msgr2=0x7f6294105f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:12.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.659+0000 7f629b50e640 1 -- 172.21.15.16:0/1115919388 shutdown_connections 2024-09-12T02:12:12.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:12.659+0000 7f629b50e640 1 -- 172.21.15.16:0/1115919388 wait complete. 2024-09-12T02:12:13.495 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '44342e0a', total seconds elapsed: 25.8 2024-09-12T02:12:14.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:12:13 smithi184 ceph-mon[35593]: pgmap v644: 97 pgs: 97 active+clean; 1.2 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.4 KiB/s wr, 9 op/s 2024-09-12T02:12:14.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:12:13 smithi033 ceph-mon[35332]: pgmap v644: 97 pgs: 97 active+clean; 1.2 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.4 KiB/s wr, 9 op/s 2024-09-12T02:12:14.183 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:12:13 smithi016 ceph-mon[30947]: pgmap v644: 97 pgs: 97 active+clean; 1.2 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.4 KiB/s wr, 9 op/s 2024-09-12T02:12:15.392 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.391+0000 7f070544c640 1 -- 172.21.15.16:0/237864356 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0700101c40 msgr2=0x7f07001020c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:15.392 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.391+0000 7f070544c640 1 --2- 172.21.15.16:0/237864356 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0700101c40 0x7f07001020c0 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f06f4009900 tx=0x7f06f402a410 comp rx=0 tx=0).stop 2024-09-12T02:12:15.392 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.391+0000 7f070544c640 1 -- 172.21.15.16:0/237864356 shutdown_connections 2024-09-12T02:12:15.392 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.391+0000 7f070544c640 1 --2- 172.21.15.16:0/237864356 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0700102600 0x7f0700108e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:15.392 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.391+0000 7f070544c640 1 --2- 172.21.15.16:0/237864356 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0700101c40 0x7f07001020c0 unknown :-1 s=CLOSED pgs=1450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:15.392 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.391+0000 7f070544c640 1 --2- 172.21.15.16:0/237864356 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0700100a50 0x7f0700100e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:15.392 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.391+0000 7f070544c640 1 -- 172.21.15.16:0/237864356 >> 172.21.15.16:0/237864356 conn(0x7f07000fc1b0 msgr2=0x7f07000fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:15.392 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.391+0000 7f070544c640 1 -- 172.21.15.16:0/237864356 shutdown_connections 2024-09-12T02:12:15.392 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.391+0000 7f070544c640 1 -- 172.21.15.16:0/237864356 wait complete. 2024-09-12T02:12:15.392 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.392+0000 7f070544c640 1 Processor -- start 2024-09-12T02:12:15.393 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.392+0000 7f070544c640 1 -- start start 2024-09-12T02:12:15.394 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.392+0000 7f070544c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0700100a50 0x7f0700197de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:15.394 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.392+0000 7f070544c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0700101c40 0x7f0700198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:15.395 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.392+0000 7f070544c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0700102600 0x7f070019f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:15.395 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.392+0000 7f070544c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f070010d820 con 0x7f0700101c40 2024-09-12T02:12:15.395 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.392+0000 7f070544c640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f070010d6a0 con 0x7f0700100a50 2024-09-12T02:12:15.395 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.392+0000 7f070544c640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f070010d9a0 con 0x7f0700102600 2024-09-12T02:12:15.395 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.392+0000 7f06fe7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0700101c40 0x7f0700198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:15.395 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.392+0000 7f06fe7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0700101c40 0x7f0700198320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38564/0 (socket says 172.21.15.16:38564) 2024-09-12T02:12:15.395 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.392+0000 7f06fe7fc640 1 -- 172.21.15.16:0/1930524779 learned_addr learned my addr 172.21.15.16:0/1930524779 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:12:15.395 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.393+0000 7f06ff7fe640 1 --2- 172.21.15.16:0/1930524779 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0700102600 0x7f070019f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:15.395 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.393+0000 7f06feffd640 1 --2- 172.21.15.16:0/1930524779 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0700100a50 0x7f0700197de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:15.395 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.393+0000 7f06fe7fc640 1 -- 172.21.15.16:0/1930524779 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0700102600 msgr2=0x7f070019f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:15.395 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.393+0000 7f06fe7fc640 1 --2- 172.21.15.16:0/1930524779 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0700102600 0x7f070019f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:15.395 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.393+0000 7f06fe7fc640 1 -- 172.21.15.16:0/1930524779 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0700100a50 msgr2=0x7f0700197de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:15.396 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.393+0000 7f06fe7fc640 1 --2- 172.21.15.16:0/1930524779 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0700100a50 0x7f0700197de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:15.396 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.393+0000 7f06fe7fc640 1 -- 172.21.15.16:0/1930524779 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f070019fa00 con 0x7f0700101c40 2024-09-12T02:12:15.396 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.393+0000 7f06ff7fe640 1 --2- 172.21.15.16:0/1930524779 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0700102600 0x7f070019f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:12:15.396 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.393+0000 7f06feffd640 1 --2- 172.21.15.16:0/1930524779 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0700100a50 0x7f0700197de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:12:15.396 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.393+0000 7f06fe7fc640 1 --2- 172.21.15.16:0/1930524779 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0700101c40 0x7f0700198320 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f06f4009230 tx=0x7f06f40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:15.396 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.393+0000 7f06e7fff640 1 -- 172.21.15.16:0/1930524779 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06f403f030 con 0x7f0700101c40 2024-09-12T02:12:15.396 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.393+0000 7f070544c640 1 -- 172.21.15.16:0/1930524779 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f070019fc90 con 0x7f0700101c40 2024-09-12T02:12:15.396 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.393+0000 7f06e7fff640 1 -- 172.21.15.16:0/1930524779 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06f4003ce0 con 0x7f0700101c40 2024-09-12T02:12:15.396 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.393+0000 7f06e7fff640 1 -- 172.21.15.16:0/1930524779 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06f4035070 con 0x7f0700101c40 2024-09-12T02:12:15.396 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.393+0000 7f070544c640 1 -- 172.21.15.16:0/1930524779 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07001a01d0 con 0x7f0700101c40 2024-09-12T02:12:15.402 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.396+0000 7f06e7fff640 1 -- 172.21.15.16:0/1930524779 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f06f4042030 con 0x7f0700101c40 2024-09-12T02:12:15.403 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.396+0000 7f07081e7740 1 Processor -- start 2024-09-12T02:12:15.403 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.396+0000 7f07081e7740 1 -- start start 2024-09-12T02:12:15.403 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.396+0000 7f07081e7740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a358a1f9a0 0x55a358a1fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:15.403 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.396+0000 7f07081e7740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a358a202e0 0x55a358a24da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:15.403 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.396+0000 7f07081e7740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a358a25370 0x55a358a27830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:15.403 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.396+0000 7f07081e7740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55a3589832f0 con 0x55a358a202e0 2024-09-12T02:12:15.403 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.396+0000 7f07081e7740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55a35892b270 con 0x55a358a25370 2024-09-12T02:12:15.403 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.396+0000 7f07081e7740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55a35897f2b0 con 0x55a358a1f9a0 2024-09-12T02:12:15.403 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.396+0000 7f06feffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a358a202e0 0x55a358a24da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:15.403 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.396+0000 7f06feffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a358a202e0 0x55a358a24da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38568/0 (socket says 172.21.15.16:38568) 2024-09-12T02:12:15.403 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.396+0000 7f06feffd640 1 -- 172.21.15.16:0/2763939278 learned_addr learned my addr 172.21.15.16:0/2763939278 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:12:15.403 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.396+0000 7f06e7fff640 1 --2- 172.21.15.16:0/1930524779 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f06e80a2c30 0x7f06e80a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:15.403 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.396+0000 7f06fe7fc640 1 --2- 172.21.15.16:0/2763939278 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a358a25370 0x55a358a27830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:15.404 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.396+0000 7f06ff7fe640 1 --2- 172.21.15.16:0/2763939278 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a358a1f9a0 0x55a358a1fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:15.404 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.396+0000 7f06feffd640 1 -- 172.21.15.16:0/2763939278 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a358a1f9a0 msgr2=0x55a358a1fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:15.404 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.397+0000 7f06feffd640 1 --2- 172.21.15.16:0/2763939278 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a358a1f9a0 0x55a358a1fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:15.404 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.397+0000 7f06feffd640 1 -- 172.21.15.16:0/2763939278 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a358a25370 msgr2=0x55a358a27830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:15.404 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.397+0000 7f06feffd640 1 --2- 172.21.15.16:0/2763939278 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a358a25370 0x55a358a27830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:15.404 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.397+0000 7f06feffd640 1 -- 172.21.15.16:0/2763939278 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a35892a340 con 0x55a358a202e0 2024-09-12T02:12:15.404 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.397+0000 7f06ff7fe640 1 --2- 172.21.15.16:0/2763939278 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a358a1f9a0 0x55a358a1fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:12:15.404 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.397+0000 7f06fe7fc640 1 --2- 172.21.15.16:0/2763939278 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a358a25370 0x55a358a27830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:12:15.404 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.397+0000 7f06ff7fe640 1 --2- 172.21.15.16:0/1930524779 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f06e80a2c30 0x7f06e80a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:15.404 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.397+0000 7f06e7fff640 1 -- 172.21.15.16:0/1930524779 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f06f40ef5a0 con 0x7f0700101c40 2024-09-12T02:12:15.404 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.397+0000 7f06feffd640 1 --2- 172.21.15.16:0/2763939278 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a358a202e0 0x55a358a24da0 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f06f0009cf0 tx=0x7f06f00117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85e2ae92b2ddc2b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:15.404 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.397+0000 7f06e57fa640 1 -- 172.21.15.16:0/2763939278 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06f0012450 con 0x55a358a202e0 2024-09-12T02:12:15.404 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.397+0000 7f06ff7fe640 1 --2- 172.21.15.16:0/1930524779 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f06e80a2c30 0x7f06e80a50f0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f06f80099a0 tx=0x7f06f8004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:15.405 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.397+0000 7f06e57fa640 1 -- 172.21.15.16:0/2763939278 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06f0012ad0 con 0x55a358a202e0 2024-09-12T02:12:15.405 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.397+0000 7f06e57fa640 1 -- 172.21.15.16:0/2763939278 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06f000fca0 con 0x55a358a202e0 2024-09-12T02:12:15.405 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.398+0000 7f07081e7740 1 -- 172.21.15.16:0/2763939278 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a358a202e0 msgr2=0x55a358a24da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:15.405 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.398+0000 7f07081e7740 1 --2- 172.21.15.16:0/2763939278 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a358a202e0 0x55a358a24da0 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f06f0009cf0 tx=0x7f06f00117b0 comp rx=0 tx=0).stop 2024-09-12T02:12:15.405 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.398+0000 7f07081e7740 1 -- 172.21.15.16:0/2763939278 shutdown_connections 2024-09-12T02:12:15.405 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.398+0000 7f07081e7740 1 --2- 172.21.15.16:0/2763939278 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a358a25370 0x55a358a27830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:15.405 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.398+0000 7f07081e7740 1 --2- 172.21.15.16:0/2763939278 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a358a202e0 0x55a358a24da0 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:15.405 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.398+0000 7f07081e7740 1 --2- 172.21.15.16:0/2763939278 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a358a1f9a0 0x55a358a1fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:15.405 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.398+0000 7f07081e7740 1 -- 172.21.15.16:0/2763939278 >> 172.21.15.16:0/2763939278 conn(0x55a358a1b010 msgr2=0x55a358a1b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:15.405 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.398+0000 7f07081e7740 1 -- 172.21.15.16:0/2763939278 shutdown_connections 2024-09-12T02:12:15.405 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.398+0000 7f07081e7740 1 -- 172.21.15.16:0/2763939278 wait complete. 2024-09-12T02:12:15.405 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.398+0000 7f07081e7740 1 -- start start 2024-09-12T02:12:15.405 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.398+0000 7f07081e7740 1 Processor -- start 2024-09-12T02:12:15.406 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.399+0000 7f07081e7740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a358a1f9a0 0x55a358a29ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:15.406 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.399+0000 7f07081e7740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a358a202e0 0x55a358a2a010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:15.406 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.399+0000 7f07081e7740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a358a25370 0x55a358a31050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:15.406 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.399+0000 7f07081e7740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55a358a28340 con 0x55a358a1f9a0 2024-09-12T02:12:15.406 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.399+0000 7f07081e7740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55a35890c290 con 0x55a358a202e0 2024-09-12T02:12:15.406 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.399+0000 7f07081e7740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55a358a284c0 con 0x55a358a25370 2024-09-12T02:12:15.406 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.399+0000 7f07081e7740 10 client.0 _ll_register_callbacks cb 0x55a35893d140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:12:15.406 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.399+0000 7f06feffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a358a1f9a0 0x55a358a29ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:15.406 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.399+0000 7f06feffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a358a1f9a0 0x55a358a29ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38570/0 (socket says 172.21.15.16:38570) 2024-09-12T02:12:15.406 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.399+0000 7f06feffd640 1 -- 172.21.15.16:0/2415155226 learned_addr learned my addr 172.21.15.16:0/2415155226 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:12:15.406 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.399+0000 7f06ff7fe640 1 --2- 172.21.15.16:0/2415155226 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a358a25370 0x55a358a31050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:15.406 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.399+0000 7f06fe7fc640 1 --2- 172.21.15.16:0/2415155226 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a358a202e0 0x55a358a2a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:15.406 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.399+0000 7f06feffd640 1 -- 172.21.15.16:0/2415155226 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a358a25370 msgr2=0x55a358a31050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:15.407 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.399+0000 7f06feffd640 1 --2- 172.21.15.16:0/2415155226 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a358a25370 0x55a358a31050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:15.407 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.399+0000 7f06feffd640 1 -- 172.21.15.16:0/2415155226 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a358a202e0 msgr2=0x55a358a2a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:15.407 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.399+0000 7f06feffd640 1 --2- 172.21.15.16:0/2415155226 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a358a202e0 0x55a358a2a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:15.407 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.399+0000 7f06feffd640 1 -- 172.21.15.16:0/2415155226 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a358696210 con 0x55a358a1f9a0 2024-09-12T02:12:15.407 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.399+0000 7f06fe7fc640 1 --2- 172.21.15.16:0/2415155226 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a358a202e0 0x55a358a2a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:12:15.407 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.399+0000 7f06feffd640 1 --2- 172.21.15.16:0/2415155226 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a358a1f9a0 0x55a358a29ad0 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f06f0005830 tx=0x7f06f0009a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e98e2d4d4465aac server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:15.407 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.399+0000 7f06e2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:12:15.407 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.399+0000 7f06e2ff5640 1 -- 172.21.15.16:0/2415155226 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06f0028020 con 0x55a358a1f9a0 2024-09-12T02:12:15.407 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.399+0000 7f06e2ff5640 1 -- 172.21.15.16:0/2415155226 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f06f00042d0 con 0x55a358a1f9a0 2024-09-12T02:12:15.407 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.399+0000 7f07081e7740 1 -- 172.21.15.16:0/2415155226 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f06f0012ad0 con 0x55a358a1f9a0 2024-09-12T02:12:15.407 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.400+0000 7f06e2ff5640 1 -- 172.21.15.16:0/2415155226 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f06f0012450 con 0x55a358a1f9a0 2024-09-12T02:12:15.407 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.400+0000 7f07081e7740 10 client.18663 fetch_fsmap learned FSMap version 25 2024-09-12T02:12:15.407 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.400+0000 7f07081e7740 1 -- 172.21.15.16:0/2415155226 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a358762620 con 0x55a358a1f9a0 2024-09-12T02:12:15.407 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.400+0000 7f06e2ff5640 1 -- 172.21.15.16:0/2415155226 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06f0012ad0 con 0x55a358a1f9a0 2024-09-12T02:12:15.408 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.400+0000 7f06e2ff5640 1 -- 172.21.15.16:0/2415155226 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f06f00051e0 con 0x55a358a1f9a0 2024-09-12T02:12:15.408 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.400+0000 7f07081e7740 10 client.18663 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:12:15.408 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.400+0000 7f07081e7740 10 client.18663 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:12:15.408 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.400+0000 7f07081e7740 20 client.18663 populate_metadata read hostname 'smithi016' 2024-09-12T02:12:15.408 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.400+0000 7f07081e7740 1 --2- 172.21.15.16:0/2415155226 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55a358a353e0 0x55a358a377d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:15.408 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.400+0000 7f07081e7740 4 client.18663 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:12:15.408 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.400+0000 7f07081e7740 1 -- 172.21.15.16:0/2415155226 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a358990a20 con 0x55a358a353e0 2024-09-12T02:12:15.408 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.400+0000 7f06fe7fc640 1 --2- 172.21.15.16:0/2415155226 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55a358a353e0 0x55a358a377d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:15.408 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.401+0000 7f06e2ff5640 10 client.18663 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:12:15.408 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.401+0000 7f06fe7fc640 1 --2- 172.21.15.16:0/2415155226 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55a358a353e0 0x55a358a377d0 crc :-1 s=READY pgs=306 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=69cb0ba77ba9c9ae server_cookie=ff23c0e48c0ec90b in_seq=0 out_seq=0 2024-09-12T02:12:15.408 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.401+0000 7f06e2ff5640 1 -- 172.21.15.16:0/2415155226 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a358990a20 con 0x55a358a353e0 2024-09-12T02:12:15.408 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.401+0000 7f06e2ff5640 10 client.18663 handle_command_reply: tid=0 2024-09-12T02:12:15.408 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.402+0000 7f07081e7740 1 client.18663 shutdown 2024-09-12T02:12:15.409 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.402+0000 7f07081e7740 1 -- 172.21.15.16:0/2415155226 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a358a1f9a0 msgr2=0x55a358a29ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:15.409 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.402+0000 7f07081e7740 1 --2- 172.21.15.16:0/2415155226 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a358a1f9a0 0x55a358a29ad0 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f06f0005830 tx=0x7f06f0009a10 comp rx=0 tx=0).stop 2024-09-12T02:12:15.409 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.402+0000 7f07081e7740 1 -- 172.21.15.16:0/2415155226 shutdown_connections 2024-09-12T02:12:15.409 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.402+0000 7f07081e7740 1 --2- 172.21.15.16:0/2415155226 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55a358a353e0 0x55a358a377d0 crc :-1 s=READY pgs=306 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:15.409 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.402+0000 7f07081e7740 1 --2- 172.21.15.16:0/2415155226 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a358a25370 0x55a358a31050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:15.409 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.402+0000 7f07081e7740 1 --2- 172.21.15.16:0/2415155226 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a358a202e0 0x55a358a2a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:15.409 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.402+0000 7f07081e7740 1 --2- 172.21.15.16:0/2415155226 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a358a1f9a0 0x55a358a29ad0 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:15.409 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.402+0000 7f06e2ff5640 0 client.18663 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:12:15.409 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.402+0000 7f07081e7740 1 -- 172.21.15.16:0/2415155226 >> 172.21.15.16:0/2415155226 conn(0x55a358a1b010 msgr2=0x55a358a1c820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:15.409 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.402+0000 7f07081e7740 1 -- 172.21.15.16:0/2415155226 shutdown_connections 2024-09-12T02:12:15.409 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.402+0000 7f07081e7740 1 -- 172.21.15.16:0/2415155226 wait complete. 2024-09-12T02:12:15.409 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.402+0000 7f07081e7740 20 client.18663 trim_cache size 0 max 16384 2024-09-12T02:12:15.421 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.417+0000 7f07081e7740 1 Processor -- start 2024-09-12T02:12:15.421 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.417+0000 7f07081e7740 1 -- start start 2024-09-12T02:12:15.421 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.417+0000 7f07081e7740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a358a1f9a0 0x55a358a2a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:15.421 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.417+0000 7f07081e7740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a358a202e0 0x55a358a30fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:15.422 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.417+0000 7f07081e7740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a358a25370 0x55a358a1a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:15.422 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.417+0000 7f07081e7740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f07001ab720 con 0x55a358a1f9a0 2024-09-12T02:12:15.422 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.417+0000 7f07081e7740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f07001a8ff0 con 0x55a358a202e0 2024-09-12T02:12:15.422 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.417+0000 7f07081e7740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f07001a6e60 con 0x55a358a25370 2024-09-12T02:12:15.422 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.417+0000 7f06feffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a358a1f9a0 0x55a358a2a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:15.422 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.417+0000 7f06feffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a358a1f9a0 0x55a358a2a0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38586/0 (socket says 172.21.15.16:38586) 2024-09-12T02:12:15.422 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.417+0000 7f06feffd640 1 -- 172.21.15.16:0/3461848303 learned_addr learned my addr 172.21.15.16:0/3461848303 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:12:15.422 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.417+0000 7f06ff7fe640 1 --2- 172.21.15.16:0/3461848303 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a358a25370 0x55a358a1a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:15.422 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.417+0000 7f06fe7fc640 1 --2- 172.21.15.16:0/3461848303 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a358a202e0 0x55a358a30fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:15.422 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.417+0000 7f06feffd640 1 -- 172.21.15.16:0/3461848303 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a358a25370 msgr2=0x55a358a1a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:15.422 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.417+0000 7f06feffd640 1 --2- 172.21.15.16:0/3461848303 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a358a25370 0x55a358a1a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:15.422 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.417+0000 7f06feffd640 1 -- 172.21.15.16:0/3461848303 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a358a202e0 msgr2=0x55a358a30fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:15.422 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.417+0000 7f06feffd640 1 --2- 172.21.15.16:0/3461848303 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a358a202e0 0x55a358a30fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:15.423 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.417+0000 7f06feffd640 1 -- 172.21.15.16:0/3461848303 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a358a207c0 con 0x55a358a1f9a0 2024-09-12T02:12:15.423 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.417+0000 7f06fe7fc640 1 --2- 172.21.15.16:0/3461848303 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a358a202e0 0x55a358a30fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:12:15.423 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.417+0000 7f06feffd640 1 --2- 172.21.15.16:0/3461848303 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a358a1f9a0 0x55a358a2a0a0 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f06f0009c10 tx=0x7f06f0009fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a161651cfce7c41c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:15.423 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.418+0000 7f06e4ff9640 1 -- 172.21.15.16:0/3461848303 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06f0028040 con 0x55a358a1f9a0 2024-09-12T02:12:15.423 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.418+0000 7f06e4ff9640 1 -- 172.21.15.16:0/3461848303 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06f003c070 con 0x55a358a1f9a0 2024-09-12T02:12:15.423 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.418+0000 7f06e4ff9640 1 -- 172.21.15.16:0/3461848303 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06f000fad0 con 0x55a358a1f9a0 2024-09-12T02:12:15.423 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.418+0000 7f07081e7740 1 -- 172.21.15.16:0/3461848303 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a358a1f9a0 msgr2=0x55a358a2a0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:15.423 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.418+0000 7f07081e7740 1 --2- 172.21.15.16:0/3461848303 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a358a1f9a0 0x55a358a2a0a0 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f06f0009c10 tx=0x7f06f0009fa0 comp rx=0 tx=0).stop 2024-09-12T02:12:15.423 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.418+0000 7f07081e7740 1 -- 172.21.15.16:0/3461848303 shutdown_connections 2024-09-12T02:12:15.423 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.418+0000 7f07081e7740 1 --2- 172.21.15.16:0/3461848303 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a358a25370 0x55a358a1a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:15.423 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.418+0000 7f07081e7740 1 --2- 172.21.15.16:0/3461848303 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a358a202e0 0x55a358a30fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:15.423 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.418+0000 7f07081e7740 1 --2- 172.21.15.16:0/3461848303 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a358a1f9a0 0x55a358a2a0a0 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:15.423 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.418+0000 7f07081e7740 1 -- 172.21.15.16:0/3461848303 >> 172.21.15.16:0/3461848303 conn(0x55a358a1b010 msgr2=0x55a358a199a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:15.423 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.418+0000 7f07081e7740 1 -- 172.21.15.16:0/3461848303 shutdown_connections 2024-09-12T02:12:15.424 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.418+0000 7f07081e7740 1 -- 172.21.15.16:0/3461848303 wait complete. 2024-09-12T02:12:15.424 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.418+0000 7f07081e7740 1 -- start start 2024-09-12T02:12:15.424 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.419+0000 7f07081e7740 1 Processor -- start 2024-09-12T02:12:15.424 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.419+0000 7f07081e7740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a358a1f9a0 0x55a358a2ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:15.424 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.419+0000 7f07081e7740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a358a202e0 0x55a358a2f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:15.424 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.419+0000 7f07081e7740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a358a25370 0x55a358a30200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:15.424 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.419+0000 7f07081e7740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f07001a74b0 con 0x55a358a25370 2024-09-12T02:12:15.424 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.419+0000 7f07081e7740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f07001a8920 con 0x55a358a202e0 2024-09-12T02:12:15.424 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.419+0000 7f07081e7740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f07001a7b80 con 0x55a358a1f9a0 2024-09-12T02:12:15.424 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.419+0000 7f07081e7740 10 client.0 _ll_register_callbacks cb 0x7f06f0009b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:12:15.424 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.419+0000 7f06ff7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a358a25370 0x55a358a30200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:15.424 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.419+0000 7f06ff7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a358a25370 0x55a358a30200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38592/0 (socket says 172.21.15.16:38592) 2024-09-12T02:12:15.424 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.419+0000 7f06ff7fe640 1 -- 172.21.15.16:0/716833128 learned_addr learned my addr 172.21.15.16:0/716833128 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:12:15.425 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.419+0000 7f06feffd640 1 --2- 172.21.15.16:0/716833128 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a358a1f9a0 0x55a358a2ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:15.425 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.419+0000 7f06fe7fc640 1 --2- 172.21.15.16:0/716833128 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a358a202e0 0x55a358a2f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:15.425 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.419+0000 7f06ff7fe640 1 -- 172.21.15.16:0/716833128 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a358a1f9a0 msgr2=0x55a358a2ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:15.425 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.419+0000 7f06ff7fe640 1 --2- 172.21.15.16:0/716833128 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a358a1f9a0 0x55a358a2ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:15.425 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.419+0000 7f06ff7fe640 1 -- 172.21.15.16:0/716833128 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a358a202e0 msgr2=0x55a358a2f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:15.425 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.419+0000 7f06ff7fe640 1 --2- 172.21.15.16:0/716833128 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a358a202e0 0x55a358a2f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:15.425 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.419+0000 7f06ff7fe640 1 -- 172.21.15.16:0/716833128 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a358a30810 con 0x55a358a25370 2024-09-12T02:12:15.425 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.419+0000 7f06fe7fc640 1 --2- 172.21.15.16:0/716833128 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a358a202e0 0x55a358a2f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:12:15.425 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.419+0000 7f06feffd640 1 --2- 172.21.15.16:0/716833128 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a358a1f9a0 0x55a358a2ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:12:15.425 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.420+0000 7f06ff7fe640 1 --2- 172.21.15.16:0/716833128 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a358a25370 0x55a358a30200 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f06f80043c0 tx=0x7f06f8009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8f469b888c2c3bd server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:15.425 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.420+0000 7f06e2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:12:15.425 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.420+0000 7f06e2ff5640 1 -- 172.21.15.16:0/716833128 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06f800fbf0 con 0x55a358a25370 2024-09-12T02:12:15.425 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.420+0000 7f06e2ff5640 1 -- 172.21.15.16:0/716833128 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f06f8009040 con 0x55a358a25370 2024-09-12T02:12:15.425 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.420+0000 7f07081e7740 1 -- 172.21.15.16:0/716833128 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f06f003c070 con 0x55a358a25370 2024-09-12T02:12:15.426 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.420+0000 7f06e2ff5640 1 -- 172.21.15.16:0/716833128 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f06f800fd90 con 0x55a358a25370 2024-09-12T02:12:15.426 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.420+0000 7f07081e7740 10 client.18669 fetch_fsmap learned FSMap version 25 2024-09-12T02:12:15.426 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.420+0000 7f07081e7740 1 -- 172.21.15.16:0/716833128 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a358aa06d0 con 0x55a358a25370 2024-09-12T02:12:15.426 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.420+0000 7f06e2ff5640 1 -- 172.21.15.16:0/716833128 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06f003c070 con 0x55a358a25370 2024-09-12T02:12:15.426 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.420+0000 7f06e2ff5640 1 -- 172.21.15.16:0/716833128 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f06f8029c00 con 0x55a358a25370 2024-09-12T02:12:15.426 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.420+0000 7f07081e7740 10 client.18669 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:12:15.426 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.420+0000 7f07081e7740 10 client.18669 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:12:15.426 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.420+0000 7f07081e7740 20 client.18669 populate_metadata read hostname 'smithi016' 2024-09-12T02:12:15.426 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.420+0000 7f07081e7740 1 --2- 172.21.15.16:0/716833128 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55a358aa1020 0x55a358aa3440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:15.426 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.420+0000 7f07081e7740 4 client.18669 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:12:15.426 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.420+0000 7f07081e7740 1 -- 172.21.15.16:0/716833128 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a35898ad70 con 0x55a358aa1020 2024-09-12T02:12:15.426 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.421+0000 7f06feffd640 1 --2- 172.21.15.16:0/716833128 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55a358aa1020 0x55a358aa3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:15.426 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.421+0000 7f06e2ff5640 10 client.18669 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:12:15.426 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.421+0000 7f06feffd640 1 --2- 172.21.15.16:0/716833128 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55a358aa1020 0x55a358aa3440 crc :-1 s=READY pgs=307 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e00883f69e0100b1 server_cookie=6356831afc1a64f2 in_seq=0 out_seq=0 2024-09-12T02:12:15.432 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.431+0000 7f06e2ff5640 1 -- 172.21.15.16:0/716833128 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a35898ad70 con 0x55a358aa1020 2024-09-12T02:12:15.432 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.431+0000 7f06e2ff5640 10 client.18669 handle_command_reply: tid=0 2024-09-12T02:12:15.432 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.431+0000 7f07081e7740 1 client.18669 shutdown 2024-09-12T02:12:15.432 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.432+0000 7f07081e7740 1 -- 172.21.15.16:0/716833128 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a358a25370 msgr2=0x55a358a30200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:15.432 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.432+0000 7f07081e7740 1 --2- 172.21.15.16:0/716833128 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a358a25370 0x55a358a30200 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f06f80043c0 tx=0x7f06f8009500 comp rx=0 tx=0).stop 2024-09-12T02:12:15.432 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.432+0000 7f07081e7740 1 -- 172.21.15.16:0/716833128 shutdown_connections 2024-09-12T02:12:15.432 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.432+0000 7f07081e7740 1 --2- 172.21.15.16:0/716833128 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55a358aa1020 0x55a358aa3440 crc :-1 s=READY pgs=307 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:15.433 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.432+0000 7f07081e7740 1 --2- 172.21.15.16:0/716833128 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a358a25370 0x55a358a30200 unknown :-1 s=CLOSED pgs=1455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:15.433 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.432+0000 7f07081e7740 1 --2- 172.21.15.16:0/716833128 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a358a202e0 0x55a358a2f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:15.433 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.432+0000 7f07081e7740 1 --2- 172.21.15.16:0/716833128 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a358a1f9a0 0x55a358a2ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:15.433 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.432+0000 7f06e2ff5640 0 client.18669 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:12:15.433 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.432+0000 7f07081e7740 1 -- 172.21.15.16:0/716833128 >> 172.21.15.16:0/716833128 conn(0x55a358a1b010 msgr2=0x55a358a1a1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:15.433 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.432+0000 7f07081e7740 1 -- 172.21.15.16:0/716833128 shutdown_connections 2024-09-12T02:12:15.433 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.432+0000 7f07081e7740 1 -- 172.21.15.16:0/716833128 wait complete. 2024-09-12T02:12:15.433 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.432+0000 7f07081e7740 20 client.18669 trim_cache size 0 max 16384 2024-09-12T02:12:15.433 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-09-12T02:12:15.433 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-09-12T02:12:15.434 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-09-12T02:12:15.434 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-09-12T02:12:15.434 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.432+0000 7f070544c640 1 -- 172.21.15.16:0/1930524779 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f06e80a2c30 msgr2=0x7f06e80a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:15.434 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.432+0000 7f070544c640 1 --2- 172.21.15.16:0/1930524779 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f06e80a2c30 0x7f06e80a50f0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f06f80099a0 tx=0x7f06f8004a40 comp rx=0 tx=0).stop 2024-09-12T02:12:15.435 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.432+0000 7f070544c640 1 -- 172.21.15.16:0/1930524779 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0700101c40 msgr2=0x7f0700198320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:15.435 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.432+0000 7f070544c640 1 --2- 172.21.15.16:0/1930524779 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0700101c40 0x7f0700198320 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f06f4009230 tx=0x7f06f40042f0 comp rx=0 tx=0).stop 2024-09-12T02:12:15.435 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.432+0000 7f070544c640 1 -- 172.21.15.16:0/1930524779 shutdown_connections 2024-09-12T02:12:15.435 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.432+0000 7f070544c640 1 --2- 172.21.15.16:0/1930524779 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f06e80a2c30 0x7f06e80a50f0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:15.435 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.432+0000 7f070544c640 1 --2- 172.21.15.16:0/1930524779 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0700102600 0x7f070019f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:15.435 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.432+0000 7f070544c640 1 --2- 172.21.15.16:0/1930524779 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0700101c40 0x7f0700198320 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:15.435 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.432+0000 7f070544c640 1 --2- 172.21.15.16:0/1930524779 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0700100a50 0x7f0700197de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:15.435 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.432+0000 7f070544c640 1 -- 172.21.15.16:0/1930524779 >> 172.21.15.16:0/1930524779 conn(0x7f07000fc1b0 msgr2=0x7f0700104f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:15.435 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.433+0000 7f070544c640 1 -- 172.21.15.16:0/1930524779 shutdown_connections 2024-09-12T02:12:15.435 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:15.433+0000 7f070544c640 1 -- 172.21.15.16:0/1930524779 wait complete. 2024-09-12T02:12:16.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:12:15 smithi184 ceph-mon[35593]: pgmap v645: 97 pgs: 97 active+clean; 1.2 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-09-12T02:12:16.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:12:15 smithi033 ceph-mon[35332]: pgmap v645: 97 pgs: 97 active+clean; 1.2 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-09-12T02:12:16.183 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:12:15 smithi016 ceph-mon[30947]: pgmap v645: 97 pgs: 97 active+clean; 1.2 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-09-12T02:12:18.128 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:12:17 smithi184 ceph-mon[35593]: pgmap v646: 97 pgs: 97 active+clean; 1.3 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 6.5 KiB/s wr, 10 op/s 2024-09-12T02:12:18.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:12:17 smithi033 ceph-mon[35332]: pgmap v646: 97 pgs: 97 active+clean; 1.3 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 6.5 KiB/s wr, 10 op/s 2024-09-12T02:12:18.183 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:12:17 smithi016 ceph-mon[30947]: pgmap v646: 97 pgs: 97 active+clean; 1.3 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 6.5 KiB/s wr, 10 op/s 2024-09-12T02:12:20.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:12:19 smithi184 ceph-mon[35593]: pgmap v647: 97 pgs: 97 active+clean; 1.3 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 5.4 KiB/s wr, 7 op/s 2024-09-12T02:12:20.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:12:19 smithi033 ceph-mon[35332]: pgmap v647: 97 pgs: 97 active+clean; 1.3 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 5.4 KiB/s wr, 7 op/s 2024-09-12T02:12:20.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:12:19 smithi016 ceph-mon[30947]: pgmap v647: 97 pgs: 97 active+clean; 1.3 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 5.4 KiB/s wr, 7 op/s 2024-09-12T02:12:22.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:12:21 smithi184 ceph-mon[35593]: pgmap v648: 97 pgs: 97 active+clean; 1.3 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.0 KiB/s wr, 10 op/s 2024-09-12T02:12:22.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:12:21 smithi033 ceph-mon[35332]: pgmap v648: 97 pgs: 97 active+clean; 1.3 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.0 KiB/s wr, 10 op/s 2024-09-12T02:12:22.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:12:21 smithi016 ceph-mon[30947]: pgmap v648: 97 pgs: 97 active+clean; 1.3 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.0 KiB/s wr, 10 op/s 2024-09-12T02:12:24.240 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:12:23 smithi033 ceph-mon[35332]: pgmap v649: 97 pgs: 97 active+clean; 1.3 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-09-12T02:12:24.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:12:23 smithi184 ceph-mon[35593]: pgmap v649: 97 pgs: 97 active+clean; 1.3 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-09-12T02:12:24.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:12:23 smithi016 ceph-mon[30947]: pgmap v649: 97 pgs: 97 active+clean; 1.3 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-09-12T02:12:26.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:12:25 smithi184 ceph-mon[35593]: pgmap v650: 97 pgs: 97 active+clean; 1.3 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-09-12T02:12:26.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:12:25 smithi033 ceph-mon[35332]: pgmap v650: 97 pgs: 97 active+clean; 1.3 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-09-12T02:12:26.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:12:25 smithi016 ceph-mon[30947]: pgmap v650: 97 pgs: 97 active+clean; 1.3 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-09-12T02:12:28.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:12:27 smithi184 ceph-mon[35593]: pgmap v651: 97 pgs: 97 active+clean; 1.4 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 15 op/s 2024-09-12T02:12:28.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:12:27 smithi033 ceph-mon[35332]: pgmap v651: 97 pgs: 97 active+clean; 1.4 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 15 op/s 2024-09-12T02:12:28.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:12:27 smithi016 ceph-mon[30947]: pgmap v651: 97 pgs: 97 active+clean; 1.4 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 15 op/s 2024-09-12T02:12:28.533 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.532+0000 7f1dba067640 1 -- 172.21.15.16:0/897304994 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1db4102610 msgr2=0x7f1db4108ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:28.533 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.532+0000 7f1dba067640 1 --2- 172.21.15.16:0/897304994 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1db4102610 0x7f1db4108ea0 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f1dac009ca0 tx=0x7f1dac02a8c0 comp rx=0 tx=0).stop 2024-09-12T02:12:28.533 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.532+0000 7f1dba067640 1 -- 172.21.15.16:0/897304994 shutdown_connections 2024-09-12T02:12:28.533 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.532+0000 7f1dba067640 1 --2- 172.21.15.16:0/897304994 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1db4102610 0x7f1db4108ea0 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:28.534 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.532+0000 7f1dba067640 1 --2- 172.21.15.16:0/897304994 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1db4101c50 0x7f1db41020d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:28.534 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.532+0000 7f1dba067640 1 --2- 172.21.15.16:0/897304994 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1db4100a60 0x7f1db4100e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:28.534 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.532+0000 7f1dba067640 1 -- 172.21.15.16:0/897304994 >> 172.21.15.16:0/897304994 conn(0x7f1db40fc1e0 msgr2=0x7f1db40fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:28.534 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.532+0000 7f1dba067640 1 -- 172.21.15.16:0/897304994 shutdown_connections 2024-09-12T02:12:28.534 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.532+0000 7f1dba067640 1 -- 172.21.15.16:0/897304994 wait complete. 2024-09-12T02:12:28.534 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.532+0000 7f1dba067640 1 Processor -- start 2024-09-12T02:12:28.534 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.532+0000 7f1dba067640 1 -- start start 2024-09-12T02:12:28.536 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.533+0000 7f1dba067640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1db4100a60 0x7f1db4199fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:28.536 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.533+0000 7f1dba067640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1db4101c50 0x7f1db419a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:28.536 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.533+0000 7f1dba067640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1db4102610 0x7f1db41a1560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:28.536 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.533+0000 7f1dba067640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1db410d7f0 con 0x7f1db4101c50 2024-09-12T02:12:28.536 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.533+0000 7f1dba067640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1db410d670 con 0x7f1db4102610 2024-09-12T02:12:28.537 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.533+0000 7f1dba067640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1db410d970 con 0x7f1db4100a60 2024-09-12T02:12:28.537 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.533+0000 7f1daaffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1db4101c50 0x7f1db419a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:28.537 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.533+0000 7f1daaffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1db4101c50 0x7f1db419a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58602/0 (socket says 172.21.15.16:58602) 2024-09-12T02:12:28.537 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.533+0000 7f1daaffd640 1 -- 172.21.15.16:0/1261371153 learned_addr learned my addr 172.21.15.16:0/1261371153 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:12:28.537 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.533+0000 7f1db37fe640 1 --2- 172.21.15.16:0/1261371153 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1db4100a60 0x7f1db4199fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:28.537 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.533+0000 7f1db3fff640 1 --2- 172.21.15.16:0/1261371153 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1db4102610 0x7f1db41a1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:28.537 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.533+0000 7f1daaffd640 1 -- 172.21.15.16:0/1261371153 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1db4100a60 msgr2=0x7f1db4199fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:28.537 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.533+0000 7f1daaffd640 1 --2- 172.21.15.16:0/1261371153 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1db4100a60 0x7f1db4199fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:28.537 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.533+0000 7f1daaffd640 1 -- 172.21.15.16:0/1261371153 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1db4102610 msgr2=0x7f1db41a1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:28.537 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.533+0000 7f1daaffd640 1 --2- 172.21.15.16:0/1261371153 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1db4102610 0x7f1db41a1560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:28.537 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.533+0000 7f1daaffd640 1 -- 172.21.15.16:0/1261371153 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1db41a1bd0 con 0x7f1db4101c50 2024-09-12T02:12:28.537 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.533+0000 7f1db3fff640 1 --2- 172.21.15.16:0/1261371153 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1db4102610 0x7f1db41a1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:12:28.538 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.533+0000 7f1daaffd640 1 --2- 172.21.15.16:0/1261371153 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1db4101c50 0x7f1db419a4f0 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f1d9c00e9f0 tx=0x7f1d9c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:28.538 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.534+0000 7f1db17fa640 1 -- 172.21.15.16:0/1261371153 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d9c00cd80 con 0x7f1db4101c50 2024-09-12T02:12:28.538 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.534+0000 7f1db17fa640 1 -- 172.21.15.16:0/1261371153 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d9c010910 con 0x7f1db4101c50 2024-09-12T02:12:28.538 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.534+0000 7f1dba067640 1 -- 172.21.15.16:0/1261371153 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1db41a1ec0 con 0x7f1db4101c50 2024-09-12T02:12:28.538 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.534+0000 7f1db17fa640 1 -- 172.21.15.16:0/1261371153 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d9c018990 con 0x7f1db4101c50 2024-09-12T02:12:28.538 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.534+0000 7f1dba067640 1 -- 172.21.15.16:0/1261371153 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1db41a23d0 con 0x7f1db4101c50 2024-09-12T02:12:28.544 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.536+0000 7f1db17fa640 1 -- 172.21.15.16:0/1261371153 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f1d9c010ac0 con 0x7f1db4101c50 2024-09-12T02:12:28.544 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.536+0000 7f1dbcdff740 1 Processor -- start 2024-09-12T02:12:28.544 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.537+0000 7f1dbcdff740 1 -- start start 2024-09-12T02:12:28.544 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.537+0000 7f1dbcdff740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5651293eb9a0 0x5651293ebda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:28.545 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.537+0000 7f1dbcdff740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5651293ec2e0 0x5651293f0da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:28.545 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.537+0000 7f1dbcdff740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5651293f1370 0x5651293f3830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:28.545 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.537+0000 7f1dbcdff740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56512934f2f0 con 0x5651293f1370 2024-09-12T02:12:28.545 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.537+0000 7f1dbcdff740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5651292f7270 con 0x5651293eb9a0 2024-09-12T02:12:28.545 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.537+0000 7f1dbcdff740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x56512934b2b0 con 0x5651293ec2e0 2024-09-12T02:12:28.545 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.537+0000 7f1daaffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5651293f1370 0x5651293f3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:28.545 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.537+0000 7f1daaffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5651293f1370 0x5651293f3830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58612/0 (socket says 172.21.15.16:58612) 2024-09-12T02:12:28.545 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.537+0000 7f1daaffd640 1 -- 172.21.15.16:0/3322360403 learned_addr learned my addr 172.21.15.16:0/3322360403 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:12:28.545 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.537+0000 7f1db3fff640 1 --2- 172.21.15.16:0/3322360403 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5651293eb9a0 0x5651293ebda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:28.545 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.537+0000 7f1db37fe640 1 --2- 172.21.15.16:0/3322360403 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5651293ec2e0 0x5651293f0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:28.545 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.537+0000 7f1daaffd640 1 -- 172.21.15.16:0/3322360403 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5651293ec2e0 msgr2=0x5651293f0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:28.545 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.537+0000 7f1daaffd640 1 --2- 172.21.15.16:0/3322360403 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5651293ec2e0 0x5651293f0da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:28.546 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.537+0000 7f1daaffd640 1 -- 172.21.15.16:0/3322360403 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5651293eb9a0 msgr2=0x5651293ebda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:28.546 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.537+0000 7f1daaffd640 1 --2- 172.21.15.16:0/3322360403 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5651293eb9a0 0x5651293ebda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:28.546 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.537+0000 7f1daaffd640 1 -- 172.21.15.16:0/3322360403 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651292f6340 con 0x5651293f1370 2024-09-12T02:12:28.546 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.537+0000 7f1db17fa640 1 --2- 172.21.15.16:0/1261371153 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f1da00a2cb0 0x7f1da00a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:28.546 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.538+0000 7f1daaffd640 1 --2- 172.21.15.16:0/3322360403 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5651293f1370 0x5651293f3830 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f1d9c0d74f0 tx=0x7f1d9c0d79b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=737baaae2a181eff server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:28.546 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.538+0000 7f1db3fff640 1 --2- 172.21.15.16:0/1261371153 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f1da00a2cb0 0x7f1da00a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:28.546 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.538+0000 7f1da9ffb640 1 -- 172.21.15.16:0/3322360403 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d9c097e70 con 0x5651293f1370 2024-09-12T02:12:28.546 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.538+0000 7f1db17fa640 1 -- 172.21.15.16:0/1261371153 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f1d9c0cf370 con 0x7f1db4101c50 2024-09-12T02:12:28.546 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.538+0000 7f1da9ffb640 1 -- 172.21.15.16:0/3322360403 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d9c09c2d0 con 0x5651293f1370 2024-09-12T02:12:28.546 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.538+0000 7f1da9ffb640 1 -- 172.21.15.16:0/3322360403 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d9c075860 con 0x5651293f1370 2024-09-12T02:12:28.546 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.539+0000 7f1dbcdff740 1 -- 172.21.15.16:0/3322360403 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5651293f1370 msgr2=0x5651293f3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:28.546 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.539+0000 7f1db3fff640 1 --2- 172.21.15.16:0/1261371153 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f1da00a2cb0 0x7f1da00a5170 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f1dac006010 tx=0x7f1dac031bd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:28.546 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.539+0000 7f1dbcdff740 1 --2- 172.21.15.16:0/3322360403 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5651293f1370 0x5651293f3830 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f1d9c0d74f0 tx=0x7f1d9c0d79b0 comp rx=0 tx=0).stop 2024-09-12T02:12:28.547 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.539+0000 7f1dbcdff740 1 -- 172.21.15.16:0/3322360403 shutdown_connections 2024-09-12T02:12:28.547 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.539+0000 7f1dbcdff740 1 --2- 172.21.15.16:0/3322360403 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5651293f1370 0x5651293f3830 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:28.547 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.539+0000 7f1dbcdff740 1 --2- 172.21.15.16:0/3322360403 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5651293ec2e0 0x5651293f0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:28.547 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.539+0000 7f1dbcdff740 1 --2- 172.21.15.16:0/3322360403 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5651293eb9a0 0x5651293ebda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:28.547 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.539+0000 7f1dbcdff740 1 -- 172.21.15.16:0/3322360403 >> 172.21.15.16:0/3322360403 conn(0x5651293e7010 msgr2=0x5651293e7410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:28.547 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.539+0000 7f1dbcdff740 1 -- 172.21.15.16:0/3322360403 shutdown_connections 2024-09-12T02:12:28.547 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.539+0000 7f1dbcdff740 1 -- 172.21.15.16:0/3322360403 wait complete. 2024-09-12T02:12:28.547 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.539+0000 7f1dbcdff740 1 -- start start 2024-09-12T02:12:28.547 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.539+0000 7f1dbcdff740 1 Processor -- start 2024-09-12T02:12:28.547 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.540+0000 7f1dbcdff740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5651293eb9a0 0x5651293f5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:28.547 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.540+0000 7f1dbcdff740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5651293ec2e0 0x5651293f6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:28.547 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.540+0000 7f1dbcdff740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5651293f1370 0x5651293fd140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:28.547 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.540+0000 7f1dbcdff740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5651293f4340 con 0x5651293eb9a0 2024-09-12T02:12:28.548 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.540+0000 7f1dbcdff740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5651292d8290 con 0x5651293f1370 2024-09-12T02:12:28.548 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.540+0000 7f1dbcdff740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5651293f44c0 con 0x5651293ec2e0 2024-09-12T02:12:28.548 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.540+0000 7f1dbcdff740 10 client.0 _ll_register_callbacks cb 0x565129309140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:12:28.548 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.540+0000 7f1db37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5651293eb9a0 0x5651293f5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:28.548 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.540+0000 7f1db37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5651293eb9a0 0x5651293f5bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58624/0 (socket says 172.21.15.16:58624) 2024-09-12T02:12:28.548 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.540+0000 7f1db37fe640 1 -- 172.21.15.16:0/3724626037 learned_addr learned my addr 172.21.15.16:0/3724626037 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:12:28.548 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.540+0000 7f1daaffd640 1 --2- 172.21.15.16:0/3724626037 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5651293ec2e0 0x5651293f6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:28.548 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.540+0000 7f1db3fff640 1 --2- 172.21.15.16:0/3724626037 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5651293f1370 0x5651293fd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:28.548 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.540+0000 7f1db37fe640 1 -- 172.21.15.16:0/3724626037 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5651293ec2e0 msgr2=0x5651293f6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:28.548 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.540+0000 7f1db37fe640 1 --2- 172.21.15.16:0/3724626037 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5651293ec2e0 0x5651293f6100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:28.548 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.540+0000 7f1db37fe640 1 -- 172.21.15.16:0/3724626037 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5651293f1370 msgr2=0x5651293fd140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:28.548 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.540+0000 7f1db37fe640 1 --2- 172.21.15.16:0/3724626037 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5651293f1370 0x5651293fd140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:28.549 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.540+0000 7f1db37fe640 1 -- 172.21.15.16:0/3724626037 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565129062210 con 0x5651293eb9a0 2024-09-12T02:12:28.549 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.540+0000 7f1daaffd640 1 --2- 172.21.15.16:0/3724626037 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5651293ec2e0 0x5651293f6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:12:28.549 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.540+0000 7f1db3fff640 1 --2- 172.21.15.16:0/3724626037 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5651293f1370 0x5651293fd140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:12:28.549 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.540+0000 7f1db37fe640 1 --2- 172.21.15.16:0/3724626037 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5651293eb9a0 0x5651293f5bc0 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7f1da400bf80 tx=0x7f1da40128e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=802dc966fb4dfa26 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:28.549 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.540+0000 7f1d92ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:12:28.549 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.541+0000 7f1d92ffd640 1 -- 172.21.15.16:0/3724626037 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1da4021060 con 0x5651293eb9a0 2024-09-12T02:12:28.549 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.541+0000 7f1d92ffd640 1 -- 172.21.15.16:0/3724626037 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f1da40106f0 con 0x5651293eb9a0 2024-09-12T02:12:28.549 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.541+0000 7f1dbcdff740 1 -- 172.21.15.16:0/3724626037 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d9c09c2d0 con 0x5651293eb9a0 2024-09-12T02:12:28.549 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.541+0000 7f1d92ffd640 1 -- 172.21.15.16:0/3724626037 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1da40108f0 con 0x5651293eb9a0 2024-09-12T02:12:28.549 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.541+0000 7f1dbcdff740 10 client.18681 fetch_fsmap learned FSMap version 25 2024-09-12T02:12:28.549 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.541+0000 7f1dbcdff740 1 -- 172.21.15.16:0/3724626037 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56512912e620 con 0x5651293eb9a0 2024-09-12T02:12:28.549 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.541+0000 7f1d92ffd640 1 -- 172.21.15.16:0/3724626037 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d9c09c2d0 con 0x5651293eb9a0 2024-09-12T02:12:28.549 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.541+0000 7f1d92ffd640 1 -- 172.21.15.16:0/3724626037 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f1da4026070 con 0x5651293eb9a0 2024-09-12T02:12:28.550 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.541+0000 7f1dbcdff740 10 client.18681 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:12:28.550 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.541+0000 7f1dbcdff740 10 client.18681 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:12:28.550 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.541+0000 7f1dbcdff740 20 client.18681 populate_metadata read hostname 'smithi016' 2024-09-12T02:12:28.550 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.541+0000 7f1dbcdff740 1 --2- 172.21.15.16:0/3724626037 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5651294018e0 0x565129403da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:28.550 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.541+0000 7f1dbcdff740 4 client.18681 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:12:28.550 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.541+0000 7f1dbcdff740 1 -- 172.21.15.16:0/3724626037 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56512935ca20 con 0x5651294018e0 2024-09-12T02:12:28.550 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.541+0000 7f1daaffd640 1 --2- 172.21.15.16:0/3724626037 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5651294018e0 0x565129403da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:28.550 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.542+0000 7f1daaffd640 1 --2- 172.21.15.16:0/3724626037 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5651294018e0 0x565129403da0 crc :-1 s=READY pgs=308 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f03c6126575ec8 server_cookie=a992e4272f38a367 in_seq=0 out_seq=0 2024-09-12T02:12:28.550 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.542+0000 7f1d92ffd640 10 client.18681 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:12:28.550 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.543+0000 7f1d92ffd640 1 -- 172.21.15.16:0/3724626037 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56512935ca20 con 0x5651294018e0 2024-09-12T02:12:28.550 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.543+0000 7f1d92ffd640 10 client.18681 handle_command_reply: tid=0 2024-09-12T02:12:28.550 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.543+0000 7f1dbcdff740 1 client.18681 shutdown 2024-09-12T02:12:28.550 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.543+0000 7f1dbcdff740 1 -- 172.21.15.16:0/3724626037 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5651293eb9a0 msgr2=0x5651293f5bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:28.551 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.543+0000 7f1dbcdff740 1 --2- 172.21.15.16:0/3724626037 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5651293eb9a0 0x5651293f5bc0 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7f1da400bf80 tx=0x7f1da40128e0 comp rx=0 tx=0).stop 2024-09-12T02:12:28.551 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.543+0000 7f1dbcdff740 1 -- 172.21.15.16:0/3724626037 shutdown_connections 2024-09-12T02:12:28.551 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.543+0000 7f1dbcdff740 1 --2- 172.21.15.16:0/3724626037 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5651294018e0 0x565129403da0 crc :-1 s=READY pgs=308 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:28.551 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.543+0000 7f1dbcdff740 1 --2- 172.21.15.16:0/3724626037 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5651293f1370 0x5651293fd140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:28.551 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.543+0000 7f1dbcdff740 1 --2- 172.21.15.16:0/3724626037 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5651293ec2e0 0x5651293f6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:28.551 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.543+0000 7f1dbcdff740 1 --2- 172.21.15.16:0/3724626037 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5651293eb9a0 0x5651293f5bc0 unknown :-1 s=CLOSED pgs=1459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:28.551 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.543+0000 7f1d92ffd640 0 client.18681 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:12:28.551 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.543+0000 7f1dbcdff740 1 -- 172.21.15.16:0/3724626037 >> 172.21.15.16:0/3724626037 conn(0x5651293e7010 msgr2=0x5651293e8820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:28.551 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.543+0000 7f1dbcdff740 1 -- 172.21.15.16:0/3724626037 shutdown_connections 2024-09-12T02:12:28.551 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.543+0000 7f1dbcdff740 1 -- 172.21.15.16:0/3724626037 wait complete. 2024-09-12T02:12:28.551 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.543+0000 7f1dbcdff740 20 client.18681 trim_cache size 0 max 16384 2024-09-12T02:12:28.561 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.558+0000 7f1dbcdff740 1 Processor -- start 2024-09-12T02:12:28.561 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.558+0000 7f1dbcdff740 1 -- start start 2024-09-12T02:12:28.561 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.558+0000 7f1dbcdff740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5651293eb9a0 0x5651293f6080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:28.561 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.558+0000 7f1dbcdff740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5651293ec2e0 0x5651293fd0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:28.561 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.558+0000 7f1dbcdff740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5651293f1370 0x5651293e7e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:28.561 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.558+0000 7f1dbcdff740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1db41ad980 con 0x5651293ec2e0 2024-09-12T02:12:28.561 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.558+0000 7f1dbcdff740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1db41ab250 con 0x5651293f1370 2024-09-12T02:12:28.561 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.558+0000 7f1dbcdff740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1db41a90a0 con 0x5651293eb9a0 2024-09-12T02:12:28.561 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.558+0000 7f1daaffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5651293ec2e0 0x5651293fd0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:28.561 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.558+0000 7f1daaffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5651293ec2e0 0x5651293fd0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58640/0 (socket says 172.21.15.16:58640) 2024-09-12T02:12:28.563 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.558+0000 7f1daaffd640 1 -- 172.21.15.16:0/3551314315 learned_addr learned my addr 172.21.15.16:0/3551314315 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:12:28.563 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.558+0000 7f1db37fe640 1 --2- 172.21.15.16:0/3551314315 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5651293eb9a0 0x5651293f6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:28.564 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.558+0000 7f1db3fff640 1 --2- 172.21.15.16:0/3551314315 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5651293f1370 0x5651293e7e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:28.564 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.558+0000 7f1daaffd640 1 -- 172.21.15.16:0/3551314315 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5651293eb9a0 msgr2=0x5651293f6080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:28.564 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.558+0000 7f1daaffd640 1 --2- 172.21.15.16:0/3551314315 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5651293eb9a0 0x5651293f6080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:28.564 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.558+0000 7f1daaffd640 1 -- 172.21.15.16:0/3551314315 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5651293f1370 msgr2=0x5651293e7e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:28.564 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.558+0000 7f1daaffd640 1 --2- 172.21.15.16:0/3551314315 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5651293f1370 0x5651293e7e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:28.564 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.558+0000 7f1daaffd640 1 -- 172.21.15.16:0/3551314315 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651293f5700 con 0x5651293ec2e0 2024-09-12T02:12:28.564 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.558+0000 7f1db3fff640 1 --2- 172.21.15.16:0/3551314315 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5651293f1370 0x5651293e7e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:12:28.564 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.559+0000 7f1daaffd640 1 --2- 172.21.15.16:0/3551314315 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5651293ec2e0 0x5651293fd0b0 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f1d9c00c6a0 tx=0x7f1d9c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c3a8907bd0b2355 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:28.564 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.559+0000 7f1da97fa640 1 -- 172.21.15.16:0/3551314315 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d9c076450 con 0x5651293ec2e0 2024-09-12T02:12:28.564 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.559+0000 7f1da97fa640 1 -- 172.21.15.16:0/3551314315 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d9c09c2d0 con 0x5651293ec2e0 2024-09-12T02:12:28.564 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.559+0000 7f1da97fa640 1 -- 172.21.15.16:0/3551314315 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d9c076620 con 0x5651293ec2e0 2024-09-12T02:12:28.564 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.559+0000 7f1dbcdff740 1 -- 172.21.15.16:0/3551314315 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5651293ec2e0 msgr2=0x5651293fd0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:28.565 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.559+0000 7f1dbcdff740 1 --2- 172.21.15.16:0/3551314315 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5651293ec2e0 0x5651293fd0b0 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f1d9c00c6a0 tx=0x7f1d9c000bc0 comp rx=0 tx=0).stop 2024-09-12T02:12:28.565 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.559+0000 7f1dbcdff740 1 -- 172.21.15.16:0/3551314315 shutdown_connections 2024-09-12T02:12:28.565 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.559+0000 7f1dbcdff740 1 --2- 172.21.15.16:0/3551314315 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5651293f1370 0x5651293e7e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:28.565 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.559+0000 7f1dbcdff740 1 --2- 172.21.15.16:0/3551314315 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5651293ec2e0 0x5651293fd0b0 unknown :-1 s=CLOSED pgs=1460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:28.565 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.559+0000 7f1dbcdff740 1 --2- 172.21.15.16:0/3551314315 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5651293eb9a0 0x5651293f6080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:28.565 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.559+0000 7f1dbcdff740 1 -- 172.21.15.16:0/3551314315 >> 172.21.15.16:0/3551314315 conn(0x5651293e7010 msgr2=0x5651293f5970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:28.565 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.559+0000 7f1dbcdff740 1 -- 172.21.15.16:0/3551314315 shutdown_connections 2024-09-12T02:12:28.565 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.559+0000 7f1dbcdff740 1 -- 172.21.15.16:0/3551314315 wait complete. 2024-09-12T02:12:28.565 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.560+0000 7f1dbcdff740 1 -- start start 2024-09-12T02:12:28.565 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.560+0000 7f1dbcdff740 1 Processor -- start 2024-09-12T02:12:28.568 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.560+0000 7f1dbcdff740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5651293eb9a0 0x5651293fa690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:28.568 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.560+0000 7f1dbcdff740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5651293ec2e0 0x5651293fabd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:28.569 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.560+0000 7f1dbcdff740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5651293f1370 0x5651293fbb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:28.569 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.560+0000 7f1dbcdff740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1db41a9710 con 0x5651293ec2e0 2024-09-12T02:12:28.569 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.560+0000 7f1dbcdff740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1db41aab80 con 0x5651293f1370 2024-09-12T02:12:28.569 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.560+0000 7f1dbcdff740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1db41a9de0 con 0x5651293eb9a0 2024-09-12T02:12:28.569 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.560+0000 7f1dbcdff740 10 client.0 _ll_register_callbacks cb 0x7f1da400b7b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:12:28.569 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.560+0000 7f1daaffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5651293ec2e0 0x5651293fabd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:28.569 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.560+0000 7f1daaffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5651293ec2e0 0x5651293fabd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58650/0 (socket says 172.21.15.16:58650) 2024-09-12T02:12:28.569 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.560+0000 7f1daaffd640 1 -- 172.21.15.16:0/2007022661 learned_addr learned my addr 172.21.15.16:0/2007022661 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:12:28.569 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.561+0000 7f1db37fe640 1 --2- 172.21.15.16:0/2007022661 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5651293eb9a0 0x5651293fa690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:28.569 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.561+0000 7f1db3fff640 1 --2- 172.21.15.16:0/2007022661 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5651293f1370 0x5651293fbb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:28.569 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.561+0000 7f1daaffd640 1 -- 172.21.15.16:0/2007022661 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5651293eb9a0 msgr2=0x5651293fa690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:28.569 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.561+0000 7f1daaffd640 1 --2- 172.21.15.16:0/2007022661 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5651293eb9a0 0x5651293fa690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:28.569 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.561+0000 7f1daaffd640 1 -- 172.21.15.16:0/2007022661 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5651293f1370 msgr2=0x5651293fbb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:28.570 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.561+0000 7f1daaffd640 1 --2- 172.21.15.16:0/2007022661 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5651293f1370 0x5651293fbb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:28.570 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.561+0000 7f1daaffd640 1 -- 172.21.15.16:0/2007022661 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651293fc1a0 con 0x5651293ec2e0 2024-09-12T02:12:28.570 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.561+0000 7f1db3fff640 1 --2- 172.21.15.16:0/2007022661 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5651293f1370 0x5651293fbb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:12:28.570 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.561+0000 7f1daaffd640 1 --2- 172.21.15.16:0/2007022661 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5651293ec2e0 0x5651293fabd0 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f1d9c0a4310 tx=0x7f1d9c09bc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=944e77f0861120b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:28.570 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.561+0000 7f1d92ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:12:28.570 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.561+0000 7f1d92ffd640 1 -- 172.21.15.16:0/2007022661 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d9c076d60 con 0x5651293ec2e0 2024-09-12T02:12:28.570 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.561+0000 7f1d92ffd640 1 -- 172.21.15.16:0/2007022661 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f1d9c0ccef0 con 0x5651293ec2e0 2024-09-12T02:12:28.570 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.561+0000 7f1dbcdff740 1 -- 172.21.15.16:0/2007022661 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d9c09c2d0 con 0x5651293ec2e0 2024-09-12T02:12:28.570 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.561+0000 7f1d92ffd640 1 -- 172.21.15.16:0/2007022661 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1d9c09b530 con 0x5651293ec2e0 2024-09-12T02:12:28.570 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.561+0000 7f1dbcdff740 10 client.18687 fetch_fsmap learned FSMap version 25 2024-09-12T02:12:28.570 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.561+0000 7f1dbcdff740 1 -- 172.21.15.16:0/2007022661 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5651293fd420 con 0x5651293ec2e0 2024-09-12T02:12:28.570 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.561+0000 7f1d92ffd640 1 -- 172.21.15.16:0/2007022661 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d9c09c2d0 con 0x5651293ec2e0 2024-09-12T02:12:28.570 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.562+0000 7f1d92ffd640 1 -- 172.21.15.16:0/2007022661 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f1d9c0a43b0 con 0x5651293ec2e0 2024-09-12T02:12:28.571 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.562+0000 7f1dbcdff740 10 client.18687 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:12:28.571 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.562+0000 7f1dbcdff740 10 client.18687 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:12:28.571 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.562+0000 7f1dbcdff740 20 client.18687 populate_metadata read hostname 'smithi016' 2024-09-12T02:12:28.571 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.562+0000 7f1dbcdff740 1 --2- 172.21.15.16:0/2007022661 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x56512946c910 0x56512946ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:28.571 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.562+0000 7f1dbcdff740 4 client.18687 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:12:28.571 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.562+0000 7f1dbcdff740 1 -- 172.21.15.16:0/2007022661 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x565129356d70 con 0x56512946c910 2024-09-12T02:12:28.571 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.562+0000 7f1db37fe640 1 --2- 172.21.15.16:0/2007022661 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x56512946c910 0x56512946ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:28.571 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.562+0000 7f1db37fe640 1 --2- 172.21.15.16:0/2007022661 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x56512946c910 0x56512946ed30 crc :-1 s=READY pgs=309 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85c3387290533d59 server_cookie=e9aecc8bff36864e in_seq=0 out_seq=0 2024-09-12T02:12:28.571 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.562+0000 7f1d92ffd640 10 client.18687 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:12:28.577 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.576+0000 7f1d92ffd640 1 -- 172.21.15.16:0/2007022661 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x565129356d70 con 0x56512946c910 2024-09-12T02:12:28.577 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.576+0000 7f1d92ffd640 10 client.18687 handle_command_reply: tid=0 2024-09-12T02:12:28.577 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.576+0000 7f1dbcdff740 1 client.18687 shutdown 2024-09-12T02:12:28.577 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.576+0000 7f1dbcdff740 1 -- 172.21.15.16:0/2007022661 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5651293ec2e0 msgr2=0x5651293fabd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:28.578 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.576+0000 7f1dbcdff740 1 --2- 172.21.15.16:0/2007022661 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5651293ec2e0 0x5651293fabd0 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f1d9c0a4310 tx=0x7f1d9c09bc50 comp rx=0 tx=0).stop 2024-09-12T02:12:28.578 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.576+0000 7f1dbcdff740 1 -- 172.21.15.16:0/2007022661 shutdown_connections 2024-09-12T02:12:28.578 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.576+0000 7f1dbcdff740 1 --2- 172.21.15.16:0/2007022661 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x56512946c910 0x56512946ed30 crc :-1 s=READY pgs=309 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:28.578 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.576+0000 7f1dbcdff740 1 --2- 172.21.15.16:0/2007022661 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5651293f1370 0x5651293fbb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:28.578 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.576+0000 7f1dbcdff740 1 --2- 172.21.15.16:0/2007022661 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5651293ec2e0 0x5651293fabd0 unknown :-1 s=CLOSED pgs=1461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:28.578 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.576+0000 7f1d92ffd640 0 client.18687 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:12:28.578 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.576+0000 7f1dbcdff740 1 --2- 172.21.15.16:0/2007022661 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5651293eb9a0 0x5651293fa690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:28.578 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.576+0000 7f1dbcdff740 1 -- 172.21.15.16:0/2007022661 >> 172.21.15.16:0/2007022661 conn(0x5651293e7010 msgr2=0x5651293e59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:28.578 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.576+0000 7f1dbcdff740 1 -- 172.21.15.16:0/2007022661 shutdown_connections 2024-09-12T02:12:28.578 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.576+0000 7f1dbcdff740 1 -- 172.21.15.16:0/2007022661 wait complete. 2024-09-12T02:12:28.578 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.576+0000 7f1dbcdff740 20 client.18687 trim_cache size 0 max 16384 2024-09-12T02:12:28.579 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-09-12T02:12:28.579 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-09-12T02:12:28.579 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-09-12T02:12:28.579 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-09-12T02:12:28.580 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.577+0000 7f1dba067640 1 -- 172.21.15.16:0/1261371153 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f1da00a2cb0 msgr2=0x7f1da00a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:28.580 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.577+0000 7f1dba067640 1 --2- 172.21.15.16:0/1261371153 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f1da00a2cb0 0x7f1da00a5170 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f1dac006010 tx=0x7f1dac031bd0 comp rx=0 tx=0).stop 2024-09-12T02:12:28.580 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.577+0000 7f1dba067640 1 -- 172.21.15.16:0/1261371153 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1db4101c50 msgr2=0x7f1db419a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:28.580 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.577+0000 7f1dba067640 1 --2- 172.21.15.16:0/1261371153 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1db4101c50 0x7f1db419a4f0 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f1d9c00e9f0 tx=0x7f1d9c00eeb0 comp rx=0 tx=0).stop 2024-09-12T02:12:28.580 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.577+0000 7f1dba067640 1 -- 172.21.15.16:0/1261371153 shutdown_connections 2024-09-12T02:12:28.580 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.577+0000 7f1dba067640 1 --2- 172.21.15.16:0/1261371153 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f1da00a2cb0 0x7f1da00a5170 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:28.580 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.577+0000 7f1dba067640 1 --2- 172.21.15.16:0/1261371153 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1db4102610 0x7f1db41a1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:28.580 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.577+0000 7f1dba067640 1 --2- 172.21.15.16:0/1261371153 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1db4101c50 0x7f1db419a4f0 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:28.580 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.577+0000 7f1dba067640 1 --2- 172.21.15.16:0/1261371153 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1db4100a60 0x7f1db4199fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:28.580 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.577+0000 7f1dba067640 1 -- 172.21.15.16:0/1261371153 >> 172.21.15.16:0/1261371153 conn(0x7f1db40fc1e0 msgr2=0x7f1db4109860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:28.580 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.577+0000 7f1dba067640 1 -- 172.21.15.16:0/1261371153 shutdown_connections 2024-09-12T02:12:28.581 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:28.577+0000 7f1dba067640 1 -- 172.21.15.16:0/1261371153 wait complete. 2024-09-12T02:12:29.793 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:12:29.793 INFO:tasks.workunit.client.0.smithi016.stdout:313/600: open fd = 3 2024-09-12T02:12:29.793 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:12:29.794 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:12:29.794 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:12:29.794 INFO:tasks.workunit.client.0.smithi016.stdout:314/600: open fd = 3 2024-09-12T02:12:29.794 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:12:29.794 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:12:29.794 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:12:29.795 INFO:tasks.workunit.client.0.smithi016.stdout:315/600: open fd = 3 2024-09-12T02:12:29.795 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:12:29.795 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:12:29.795 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:12:29.795 INFO:tasks.workunit.client.0.smithi016.stdout:316/600: open fd = 3 2024-09-12T02:12:29.796 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:12:29.796 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:12:29.796 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:12:29.796 INFO:tasks.workunit.client.0.smithi016.stdout:317/600: open fd = 3 2024-09-12T02:12:29.796 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:12:29.797 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:12:29.797 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:12:29.797 INFO:tasks.workunit.client.0.smithi016.stdout:318/600: open fd = 3 2024-09-12T02:12:29.797 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:12:29.797 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:12:29.797 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:12:29.798 INFO:tasks.workunit.client.0.smithi016.stdout:319/600: open fd = 3 2024-09-12T02:12:29.798 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:12:29.799 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:12:29.799 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:12:29.799 INFO:tasks.workunit.client.0.smithi016.stdout:320/600: open fd = 3 2024-09-12T02:12:29.799 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:12:29.799 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:12:29.799 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:12:29.800 INFO:tasks.workunit.client.0.smithi016.stdout:321/600: open fd = 3 2024-09-12T02:12:29.800 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:12:29.800 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:12:29.800 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:12:29.800 INFO:tasks.workunit.client.0.smithi016.stdout:322/600: open fd = 3 2024-09-12T02:12:29.801 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:12:29.801 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:12:29.801 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:12:29.801 INFO:tasks.workunit.client.0.smithi016.stdout:323/600: open fd = 3 2024-09-12T02:12:29.801 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:12:29.801 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:12:29.802 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:12:29.802 INFO:tasks.workunit.client.0.smithi016.stdout:324/600: open fd = 3 2024-09-12T02:12:29.802 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:12:29.802 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:12:29.802 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:12:29.803 INFO:tasks.workunit.client.0.smithi016.stdout:325/600: open fd = 3 2024-09-12T02:12:29.803 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:12:29.803 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:12:29.803 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:12:29.803 INFO:tasks.workunit.client.0.smithi016.stdout:326/600: open fd = 3 2024-09-12T02:12:29.803 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:12:29.804 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:12:29.804 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:12:29.804 INFO:tasks.workunit.client.0.smithi016.stdout:327/600: open fd = 3 2024-09-12T02:12:29.804 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:12:29.804 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:12:29.805 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:12:29.805 INFO:tasks.workunit.client.0.smithi016.stdout:328/600: open fd = 3 2024-09-12T02:12:29.805 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:12:29.805 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:12:29.805 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:12:29.805 INFO:tasks.workunit.client.0.smithi016.stdout:329/600: open fd = 3 2024-09-12T02:12:29.806 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:12:29.806 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:12:29.806 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:12:29.806 INFO:tasks.workunit.client.0.smithi016.stdout:330/600: open fd = 3 2024-09-12T02:12:29.806 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:12:29.807 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:12:29.807 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:12:29.807 INFO:tasks.workunit.client.0.smithi016.stdout:331/600: open fd = 3 2024-09-12T02:12:29.807 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:12:29.807 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:12:29.807 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:12:29.808 INFO:tasks.workunit.client.0.smithi016.stdout:332/600: open fd = 3 2024-09-12T02:12:29.808 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:12:29.808 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:12:29.808 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:12:29.809 INFO:tasks.workunit.client.0.smithi016.stdout:333/600: open fd = 3 2024-09-12T02:12:29.809 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:12:29.809 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:12:29.809 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:12:29.809 INFO:tasks.workunit.client.0.smithi016.stdout:334/600: open fd = 3 2024-09-12T02:12:29.810 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:12:29.810 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:12:29.810 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:12:29.810 INFO:tasks.workunit.client.0.smithi016.stdout:335/600: open fd = 3 2024-09-12T02:12:29.810 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:12:29.810 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:12:29.811 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:12:29.811 INFO:tasks.workunit.client.0.smithi016.stdout:336/600: open fd = 3 2024-09-12T02:12:29.811 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:12:29.811 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:12:29.811 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:12:29.811 INFO:tasks.workunit.client.0.smithi016.stdout:337/600: open fd = 3 2024-09-12T02:12:29.812 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:12:29.812 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:12:29.812 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:12:29.812 INFO:tasks.workunit.client.0.smithi016.stdout:338/600: open fd = 3 2024-09-12T02:12:29.812 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:12:29.813 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:12:29.813 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:12:29.813 INFO:tasks.workunit.client.0.smithi016.stdout:339/600: open fd = 3 2024-09-12T02:12:29.813 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:12:29.813 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:12:29.814 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:12:29.814 INFO:tasks.workunit.client.0.smithi016.stdout:340/600: open fd = 3 2024-09-12T02:12:29.814 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:12:29.814 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:12:29.814 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:12:29.814 INFO:tasks.workunit.client.0.smithi016.stdout:341/600: open fd = 3 2024-09-12T02:12:29.815 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:12:29.815 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:12:29.815 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:12:29.815 INFO:tasks.workunit.client.0.smithi016.stdout:342/600: open fd = 3 2024-09-12T02:12:29.815 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:12:29.816 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:12:29.816 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:12:29.816 INFO:tasks.workunit.client.0.smithi016.stdout:343/600: open fd = 3 2024-09-12T02:12:29.816 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:12:29.816 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:12:29.817 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:12:29.817 INFO:tasks.workunit.client.0.smithi016.stdout:344/600: open fd = 3 2024-09-12T02:12:29.817 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:12:29.817 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:12:29.817 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:12:29.817 INFO:tasks.workunit.client.0.smithi016.stdout:345/600: open fd = 3 2024-09-12T02:12:29.818 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:12:29.818 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:12:29.818 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:12:29.818 INFO:tasks.workunit.client.0.smithi016.stdout:346/600: open fd = 3 2024-09-12T02:12:29.818 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:12:29.819 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:12:29.819 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:12:29.819 INFO:tasks.workunit.client.0.smithi016.stdout:347/600: open fd = 3 2024-09-12T02:12:29.819 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:12:29.819 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:12:29.820 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:12:29.820 INFO:tasks.workunit.client.0.smithi016.stdout:348/600: open fd = 3 2024-09-12T02:12:29.820 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:12:29.820 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:12:29.820 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:12:29.820 INFO:tasks.workunit.client.0.smithi016.stdout:349/600: open fd = 3 2024-09-12T02:12:29.821 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:12:29.821 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:12:29.821 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:12:29.821 INFO:tasks.workunit.client.0.smithi016.stdout:350/600: open fd = 3 2024-09-12T02:12:29.821 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:12:29.821 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:12:29.822 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:12:29.822 INFO:tasks.workunit.client.0.smithi016.stdout:351/600: open fd = 3 2024-09-12T02:12:29.822 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:12:29.822 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:12:29.822 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:12:29.823 INFO:tasks.workunit.client.0.smithi016.stdout:352/600: open fd = 3 2024-09-12T02:12:29.823 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:12:29.823 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:12:29.823 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:12:29.823 INFO:tasks.workunit.client.0.smithi016.stdout:353/600: open fd = 3 2024-09-12T02:12:29.823 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:12:29.824 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:12:29.824 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:12:29.824 INFO:tasks.workunit.client.0.smithi016.stdout:354/600: open fd = 3 2024-09-12T02:12:29.824 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:12:29.824 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:12:29.824 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:12:29.825 INFO:tasks.workunit.client.0.smithi016.stdout:355/600: open fd = 3 2024-09-12T02:12:29.825 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:12:29.825 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:12:29.825 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:12:29.825 INFO:tasks.workunit.client.0.smithi016.stdout:356/600: open fd = 3 2024-09-12T02:12:29.826 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:12:29.826 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:12:29.826 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:12:29.826 INFO:tasks.workunit.client.0.smithi016.stdout:357/600: open fd = 3 2024-09-12T02:12:29.826 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:12:29.827 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:12:29.827 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:12:29.827 INFO:tasks.workunit.client.0.smithi016.stdout:358/600: open fd = 3 2024-09-12T02:12:29.827 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:12:29.827 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:12:29.828 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:12:29.828 INFO:tasks.workunit.client.0.smithi016.stdout:359/600: open fd = 3 2024-09-12T02:12:29.828 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:12:29.828 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:12:29.828 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:12:29.828 INFO:tasks.workunit.client.0.smithi016.stdout:360/600: open fd = 3 2024-09-12T02:12:29.829 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:12:29.829 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:12:29.829 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:12:29.829 INFO:tasks.workunit.client.0.smithi016.stdout:361/600: open fd = 3 2024-09-12T02:12:29.829 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:12:29.830 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:12:29.830 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:12:29.830 INFO:tasks.workunit.client.0.smithi016.stdout:362/600: open fd = 3 2024-09-12T02:12:29.830 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:12:29.830 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:12:29.831 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:12:29.831 INFO:tasks.workunit.client.0.smithi016.stdout:363/600: open fd = 3 2024-09-12T02:12:29.831 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:12:29.831 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:12:29.831 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:12:29.832 INFO:tasks.workunit.client.0.smithi016.stdout:364/600: open fd = 3 2024-09-12T02:12:29.832 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:12:29.832 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:12:29.832 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:12:29.832 INFO:tasks.workunit.client.0.smithi016.stdout:365/600: open fd = 3 2024-09-12T02:12:29.833 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:12:29.833 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:12:29.833 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:12:29.833 INFO:tasks.workunit.client.0.smithi016.stdout:366/600: open fd = 3 2024-09-12T02:12:29.833 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:12:29.833 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:12:29.834 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:12:29.834 INFO:tasks.workunit.client.0.smithi016.stdout:367/600: open fd = 3 2024-09-12T02:12:29.834 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:12:29.834 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:12:29.834 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:12:29.835 INFO:tasks.workunit.client.0.smithi016.stdout:368/600: open fd = 3 2024-09-12T02:12:29.835 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:12:29.835 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:12:29.835 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:12:29.835 INFO:tasks.workunit.client.0.smithi016.stdout:369/600: open fd = 3 2024-09-12T02:12:29.836 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:12:29.836 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:12:29.836 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:12:29.836 INFO:tasks.workunit.client.0.smithi016.stdout:370/600: open fd = 3 2024-09-12T02:12:29.836 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:12:29.836 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:12:29.837 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:12:29.837 INFO:tasks.workunit.client.0.smithi016.stdout:371/600: open fd = 3 2024-09-12T02:12:29.837 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:12:29.837 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:12:29.837 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:12:29.838 INFO:tasks.workunit.client.0.smithi016.stdout:372/600: open fd = 3 2024-09-12T02:12:29.838 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:12:29.838 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:12:29.838 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:12:29.838 INFO:tasks.workunit.client.0.smithi016.stdout:373/600: open fd = 3 2024-09-12T02:12:29.838 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:12:29.839 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:12:29.839 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:12:29.840 INFO:tasks.workunit.client.0.smithi016.stdout:374/600: open fd = 3 2024-09-12T02:12:29.840 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:12:29.840 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:12:29.840 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:12:30.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:12:29 smithi184 ceph-mon[35593]: pgmap v652: 97 pgs: 97 active+clean; 1.4 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 11 op/s 2024-09-12T02:12:30.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:12:29 smithi033 ceph-mon[35332]: pgmap v652: 97 pgs: 97 active+clean; 1.4 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 11 op/s 2024-09-12T02:12:30.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:12:29 smithi016 ceph-mon[30947]: pgmap v652: 97 pgs: 97 active+clean; 1.4 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 11 op/s 2024-09-12T02:12:30.648 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T02:12:30.919 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:12:31.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.405+0000 7fd917b68640 1 -- 172.21.15.16:0/3716440113 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd910108300 msgr2=0x7fd91010ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:31.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.405+0000 7fd917b68640 1 --2- 172.21.15.16:0/3716440113 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd910108300 0x7fd91010ec70 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7fd908009cd0 tx=0x7fd90802a8f0 comp rx=0 tx=0).stop 2024-09-12T02:12:31.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.405+0000 7fd917b68640 1 -- 172.21.15.16:0/3716440113 shutdown_connections 2024-09-12T02:12:31.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.405+0000 7fd917b68640 1 --2- 172.21.15.16:0/3716440113 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd910108300 0x7fd91010ec70 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:31.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.405+0000 7fd917b68640 1 --2- 172.21.15.16:0/3716440113 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd910103780 0x7fd910107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:31.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.405+0000 7fd917b68640 1 --2- 172.21.15.16:0/3716440113 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd910102dd0 0x7fd9101031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:31.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.405+0000 7fd917b68640 1 -- 172.21.15.16:0/3716440113 >> 172.21.15.16:0/3716440113 conn(0x7fd9100fc7d0 msgr2=0x7fd9100febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:31.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.405+0000 7fd917b68640 1 -- 172.21.15.16:0/3716440113 shutdown_connections 2024-09-12T02:12:31.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.405+0000 7fd917b68640 1 -- 172.21.15.16:0/3716440113 wait complete. 2024-09-12T02:12:31.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.406+0000 7fd917b68640 1 Processor -- start 2024-09-12T02:12:31.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.406+0000 7fd917b68640 1 -- start start 2024-09-12T02:12:31.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.406+0000 7fd917b68640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd910102dd0 0x7fd9101985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:31.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.406+0000 7fd917b68640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd910103780 0x7fd910198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:31.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.406+0000 7fd917b68640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd910108300 0x7fd91019ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:31.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.406+0000 7fd917b68640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd9101120f0 con 0x7fd910103780 2024-09-12T02:12:31.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.406+0000 7fd917b68640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd910111f70 con 0x7fd910108300 2024-09-12T02:12:31.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.406+0000 7fd917b68640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd910112270 con 0x7fd910102dd0 2024-09-12T02:12:31.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.406+0000 7fd915103640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd910103780 0x7fd910198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:31.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.406+0000 7fd915103640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd910103780 0x7fd910198b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36478/0 (socket says 172.21.15.16:36478) 2024-09-12T02:12:31.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.406+0000 7fd915103640 1 -- 172.21.15.16:0/1530454567 learned_addr learned my addr 172.21.15.16:0/1530454567 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:12:31.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.406+0000 7fd916105640 1 --2- 172.21.15.16:0/1530454567 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd910108300 0x7fd91019ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:31.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.406+0000 7fd915904640 1 --2- 172.21.15.16:0/1530454567 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd910102dd0 0x7fd9101985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:31.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.407+0000 7fd915103640 1 -- 172.21.15.16:0/1530454567 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd910102dd0 msgr2=0x7fd9101985c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:31.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.407+0000 7fd915103640 1 --2- 172.21.15.16:0/1530454567 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd910102dd0 0x7fd9101985c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:31.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.407+0000 7fd915103640 1 -- 172.21.15.16:0/1530454567 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd910108300 msgr2=0x7fd91019ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:31.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.407+0000 7fd915103640 1 --2- 172.21.15.16:0/1530454567 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd910108300 0x7fd91019ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:31.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.407+0000 7fd915103640 1 -- 172.21.15.16:0/1530454567 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd91019d570 con 0x7fd910103780 2024-09-12T02:12:31.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.407+0000 7fd915103640 1 --2- 172.21.15.16:0/1530454567 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd910103780 0x7fd910198b00 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7fd900006fd0 tx=0x7fd90000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:31.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.407+0000 7fd906ffd640 1 -- 172.21.15.16:0/1530454567 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd900011830 con 0x7fd910103780 2024-09-12T02:12:31.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.407+0000 7fd906ffd640 1 -- 172.21.15.16:0/1530454567 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd900016070 con 0x7fd910103780 2024-09-12T02:12:31.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.407+0000 7fd917b68640 1 -- 172.21.15.16:0/1530454567 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd91019d860 con 0x7fd910103780 2024-09-12T02:12:31.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.407+0000 7fd906ffd640 1 -- 172.21.15.16:0/1530454567 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd90000f2e0 con 0x7fd910103780 2024-09-12T02:12:31.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.407+0000 7fd917b68640 1 -- 172.21.15.16:0/1530454567 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9101a5140 con 0x7fd910103780 2024-09-12T02:12:31.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.409+0000 7fd906ffd640 1 -- 172.21.15.16:0/1530454567 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fd90001b460 con 0x7fd910103780 2024-09-12T02:12:31.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.410+0000 7fd919f41740 1 Processor -- start 2024-09-12T02:12:31.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.410+0000 7fd919f41740 1 -- start start 2024-09-12T02:12:31.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.410+0000 7fd919f41740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561c219db380 0x561c219db760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:31.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.410+0000 7fd919f41740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561c219dbca0 0x561c219ea550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:31.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.410+0000 7fd919f41740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561c219eaa90 0x561c219ece80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:31.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.410+0000 7fd919f41740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x561c219412e0 con 0x561c219dbca0 2024-09-12T02:12:31.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.410+0000 7fd919f41740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x561c218e8ce0 con 0x561c219eaa90 2024-09-12T02:12:31.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.410+0000 7fd919f41740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x561c2193d2a0 con 0x561c219db380 2024-09-12T02:12:31.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.410+0000 7fd915904640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561c219dbca0 0x561c219ea550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:31.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.410+0000 7fd915904640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561c219dbca0 0x561c219ea550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36482/0 (socket says 172.21.15.16:36482) 2024-09-12T02:12:31.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.410+0000 7fd915904640 1 -- 172.21.15.16:0/544486489 learned_addr learned my addr 172.21.15.16:0/544486489 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:12:31.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.410+0000 7fd906ffd640 1 --2- 172.21.15.16:0/1530454567 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fd8fc0a2b80 0x7fd8fc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:31.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.410+0000 7fd916105640 1 --2- 172.21.15.16:0/544486489 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561c219db380 0x561c219db760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:31.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.411+0000 7fd916105640 1 --2- 172.21.15.16:0/1530454567 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fd8fc0a2b80 0x7fd8fc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:31.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.411+0000 7fd915904640 1 -- 172.21.15.16:0/544486489 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561c219db380 msgr2=0x561c219db760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:31.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.411+0000 7fd915904640 1 --2- 172.21.15.16:0/544486489 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561c219db380 0x561c219db760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:31.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.411+0000 7fd915904640 1 -- 172.21.15.16:0/544486489 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561c219eaa90 msgr2=0x561c219ece80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:31.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.411+0000 7fd915904640 1 --2- 172.21.15.16:0/544486489 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561c219eaa90 0x561c219ece80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:31.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.411+0000 7fd915904640 1 -- 172.21.15.16:0/544486489 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c218cd440 con 0x561c219dbca0 2024-09-12T02:12:31.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.411+0000 7fd906ffd640 1 -- 172.21.15.16:0/1530454567 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fd9000ce5d0 con 0x7fd910103780 2024-09-12T02:12:31.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.411+0000 7fd915904640 1 --2- 172.21.15.16:0/544486489 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561c219dbca0 0x561c219ea550 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7fd90c00bd10 tx=0x7fd90c0107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78e3a711f04e4573 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:31.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.411+0000 7fd8fbfff640 1 -- 172.21.15.16:0/544486489 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd90c004030 con 0x561c219dbca0 2024-09-12T02:12:31.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.411+0000 7fd8fbfff640 1 -- 172.21.15.16:0/544486489 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd90c0041d0 con 0x561c219dbca0 2024-09-12T02:12:31.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.411+0000 7fd8fbfff640 1 -- 172.21.15.16:0/544486489 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd90c011840 con 0x561c219dbca0 2024-09-12T02:12:31.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.411+0000 7fd916105640 1 --2- 172.21.15.16:0/1530454567 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fd8fc0a2b80 0x7fd8fc0a5040 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fd908009a60 tx=0x7fd9080095c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:31.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.412+0000 7fd919f41740 1 -- 172.21.15.16:0/544486489 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561c219dbca0 msgr2=0x561c219ea550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:31.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.412+0000 7fd919f41740 1 --2- 172.21.15.16:0/544486489 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561c219dbca0 0x561c219ea550 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7fd90c00bd10 tx=0x7fd90c0107a0 comp rx=0 tx=0).stop 2024-09-12T02:12:31.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.412+0000 7fd919f41740 1 -- 172.21.15.16:0/544486489 shutdown_connections 2024-09-12T02:12:31.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.412+0000 7fd919f41740 1 --2- 172.21.15.16:0/544486489 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561c219eaa90 0x561c219ece80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:31.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.412+0000 7fd919f41740 1 --2- 172.21.15.16:0/544486489 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561c219dbca0 0x561c219ea550 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:31.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.412+0000 7fd919f41740 1 --2- 172.21.15.16:0/544486489 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561c219db380 0x561c219db760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:31.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.412+0000 7fd919f41740 1 -- 172.21.15.16:0/544486489 >> 172.21.15.16:0/544486489 conn(0x561c219d7a30 msgr2=0x561c219d7e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:31.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.412+0000 7fd919f41740 1 -- 172.21.15.16:0/544486489 shutdown_connections 2024-09-12T02:12:31.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.412+0000 7fd919f41740 1 -- 172.21.15.16:0/544486489 wait complete. 2024-09-12T02:12:31.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.412+0000 7fd919f41740 1 -- start start 2024-09-12T02:12:31.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.412+0000 7fd919f41740 1 Processor -- start 2024-09-12T02:12:31.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.413+0000 7fd919f41740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561c219db380 0x561c219ea510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:31.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.413+0000 7fd919f41740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561c219dbca0 0x561c219e77c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:31.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.413+0000 7fd919f41740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561c219eaa90 0x561c219e7f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:31.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.413+0000 7fd919f41740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x561c219ed660 con 0x561c219dbca0 2024-09-12T02:12:31.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.413+0000 7fd919f41740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x561c218fc050 con 0x561c219eaa90 2024-09-12T02:12:31.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.413+0000 7fd919f41740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x561c219ed7e0 con 0x561c219db380 2024-09-12T02:12:31.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.413+0000 7fd919f41740 10 client.0 _ll_register_callbacks cb 0x561c2193c160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:12:31.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.413+0000 7fd915103640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561c219dbca0 0x561c219e77c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:31.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.413+0000 7fd915103640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561c219dbca0 0x561c219e77c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36494/0 (socket says 172.21.15.16:36494) 2024-09-12T02:12:31.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.413+0000 7fd915103640 1 -- 172.21.15.16:0/3634276459 learned_addr learned my addr 172.21.15.16:0/3634276459 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:12:31.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.413+0000 7fd916105640 1 --2- 172.21.15.16:0/3634276459 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561c219eaa90 0x561c219e7f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:31.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.413+0000 7fd915904640 1 --2- 172.21.15.16:0/3634276459 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561c219db380 0x561c219ea510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:31.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.413+0000 7fd915103640 1 -- 172.21.15.16:0/3634276459 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561c219db380 msgr2=0x561c219ea510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:31.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.413+0000 7fd915103640 1 --2- 172.21.15.16:0/3634276459 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561c219db380 0x561c219ea510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:31.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.413+0000 7fd915103640 1 -- 172.21.15.16:0/3634276459 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561c219eaa90 msgr2=0x561c219e7f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:31.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.413+0000 7fd915103640 1 --2- 172.21.15.16:0/3634276459 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561c219eaa90 0x561c219e7f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:31.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.413+0000 7fd915103640 1 -- 172.21.15.16:0/3634276459 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c216b23c0 con 0x561c219dbca0 2024-09-12T02:12:31.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.414+0000 7fd915103640 1 --2- 172.21.15.16:0/3634276459 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561c219dbca0 0x561c219e77c0 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7fd90000bde0 tx=0x7fd90009aa90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f268ced3b44b3f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:31.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.414+0000 7fd8f97fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:12:31.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.414+0000 7fd8f97fa640 1 -- 172.21.15.16:0/3634276459 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd90009aed0 con 0x561c219dbca0 2024-09-12T02:12:31.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.414+0000 7fd8f97fa640 1 -- 172.21.15.16:0/3634276459 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fd900097540 con 0x561c219dbca0 2024-09-12T02:12:31.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.414+0000 7fd919f41740 1 -- 172.21.15.16:0/3634276459 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd90c0041d0 con 0x561c219dbca0 2024-09-12T02:12:31.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.414+0000 7fd8f97fa640 1 -- 172.21.15.16:0/3634276459 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd90009f280 con 0x561c219dbca0 2024-09-12T02:12:31.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.414+0000 7fd919f41740 10 client.18699 fetch_fsmap learned FSMap version 25 2024-09-12T02:12:31.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.414+0000 7fd919f41740 1 -- 172.21.15.16:0/3634276459 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x561c219f9190 con 0x561c219dbca0 2024-09-12T02:12:31.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.414+0000 7fd8f97fa640 1 -- 172.21.15.16:0/3634276459 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd90c0041d0 con 0x561c219dbca0 2024-09-12T02:12:31.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.414+0000 7fd8f97fa640 1 -- 172.21.15.16:0/3634276459 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fd9000ce0d0 con 0x561c219dbca0 2024-09-12T02:12:31.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.414+0000 7fd919f41740 10 client.18699 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:12:31.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.414+0000 7fd919f41740 10 client.18699 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:12:31.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.414+0000 7fd919f41740 20 client.18699 populate_metadata read hostname 'smithi016' 2024-09-12T02:12:31.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.414+0000 7fd919f41740 1 --2- 172.21.15.16:0/3634276459 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x561c219fa5a0 0x561c219fc9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:31.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.415+0000 7fd919f41740 4 client.18699 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:12:31.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.415+0000 7fd919f41740 1 -- 172.21.15.16:0/3634276459 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561c2194e850 con 0x561c219fa5a0 2024-09-12T02:12:31.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.415+0000 7fd915904640 1 --2- 172.21.15.16:0/3634276459 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x561c219fa5a0 0x561c219fc9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:31.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.415+0000 7fd915904640 1 --2- 172.21.15.16:0/3634276459 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x561c219fa5a0 0x561c219fc9b0 crc :-1 s=READY pgs=310 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a83a1c635d55975d server_cookie=433216390ddae300 in_seq=0 out_seq=0 2024-09-12T02:12:31.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.415+0000 7fd8f97fa640 10 client.18699 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:12:31.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.416+0000 7fd8f97fa640 1 -- 172.21.15.16:0/3634276459 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561c2194e850 con 0x561c219fa5a0 2024-09-12T02:12:31.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.416+0000 7fd8f97fa640 10 client.18699 handle_command_reply: tid=0 2024-09-12T02:12:31.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.416+0000 7fd919f41740 1 client.18699 shutdown 2024-09-12T02:12:31.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.416+0000 7fd919f41740 1 -- 172.21.15.16:0/3634276459 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561c219dbca0 msgr2=0x561c219e77c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:31.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.416+0000 7fd919f41740 1 --2- 172.21.15.16:0/3634276459 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561c219dbca0 0x561c219e77c0 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7fd90000bde0 tx=0x7fd90009aa90 comp rx=0 tx=0).stop 2024-09-12T02:12:31.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.416+0000 7fd919f41740 1 -- 172.21.15.16:0/3634276459 shutdown_connections 2024-09-12T02:12:31.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.416+0000 7fd919f41740 1 --2- 172.21.15.16:0/3634276459 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x561c219fa5a0 0x561c219fc9b0 crc :-1 s=READY pgs=310 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:31.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.416+0000 7fd919f41740 1 --2- 172.21.15.16:0/3634276459 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561c219eaa90 0x561c219e7f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:31.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.416+0000 7fd919f41740 1 --2- 172.21.15.16:0/3634276459 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561c219dbca0 0x561c219e77c0 unknown :-1 s=CLOSED pgs=1465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:31.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.416+0000 7fd919f41740 1 --2- 172.21.15.16:0/3634276459 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561c219db380 0x561c219ea510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:31.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.416+0000 7fd8f97fa640 0 client.18699 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:12:31.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.416+0000 7fd919f41740 1 -- 172.21.15.16:0/3634276459 >> 172.21.15.16:0/3634276459 conn(0x561c219d7a30 msgr2=0x561c219dd790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:31.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.416+0000 7fd919f41740 1 -- 172.21.15.16:0/3634276459 shutdown_connections 2024-09-12T02:12:31.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.416+0000 7fd919f41740 1 -- 172.21.15.16:0/3634276459 wait complete. 2024-09-12T02:12:31.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.416+0000 7fd919f41740 20 client.18699 trim_cache size 0 max 16384 2024-09-12T02:12:31.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.434+0000 7fd919f41740 1 Processor -- start 2024-09-12T02:12:31.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.434+0000 7fd919f41740 1 -- start start 2024-09-12T02:12:31.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.434+0000 7fd919f41740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561c219db380 0x561c219f6900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:31.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.434+0000 7fd919f41740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561c219dbca0 0x561c21a02280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:31.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.434+0000 7fd919f41740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561c219eaa90 0x561c21a027c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:31.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.434+0000 7fd919f41740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd9101adb10 con 0x561c219eaa90 2024-09-12T02:12:31.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.434+0000 7fd919f41740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd9101ab3e0 con 0x561c219dbca0 2024-09-12T02:12:31.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.434+0000 7fd919f41740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd9101a9270 con 0x561c219db380 2024-09-12T02:12:31.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.434+0000 7fd916105640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561c219eaa90 0x561c21a027c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:31.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.434+0000 7fd916105640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561c219eaa90 0x561c21a027c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36502/0 (socket says 172.21.15.16:36502) 2024-09-12T02:12:31.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.434+0000 7fd916105640 1 -- 172.21.15.16:0/1559283968 learned_addr learned my addr 172.21.15.16:0/1559283968 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:12:31.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.434+0000 7fd915103640 1 --2- 172.21.15.16:0/1559283968 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561c219dbca0 0x561c21a02280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:31.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.434+0000 7fd915904640 1 --2- 172.21.15.16:0/1559283968 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561c219db380 0x561c219f6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:31.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.434+0000 7fd916105640 1 -- 172.21.15.16:0/1559283968 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561c219db380 msgr2=0x561c219f6900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:31.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.434+0000 7fd916105640 1 --2- 172.21.15.16:0/1559283968 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561c219db380 0x561c219f6900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:31.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.434+0000 7fd916105640 1 -- 172.21.15.16:0/1559283968 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561c219dbca0 msgr2=0x561c21a02280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:31.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.434+0000 7fd916105640 1 --2- 172.21.15.16:0/1559283968 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561c219dbca0 0x561c21a02280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:31.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.434+0000 7fd916105640 1 -- 172.21.15.16:0/1559283968 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c219d8ba0 con 0x561c219eaa90 2024-09-12T02:12:31.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.434+0000 7fd916105640 1 --2- 172.21.15.16:0/1559283968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561c219eaa90 0x561c21a027c0 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7fd908035000 tx=0x7fd908042000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ca9b0cb7e879927 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:31.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.435+0000 7fd8fb7fe640 1 -- 172.21.15.16:0/1559283968 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd908056070 con 0x561c219eaa90 2024-09-12T02:12:31.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.435+0000 7fd8fb7fe640 1 -- 172.21.15.16:0/1559283968 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd908036070 con 0x561c219eaa90 2024-09-12T02:12:31.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.435+0000 7fd8fb7fe640 1 -- 172.21.15.16:0/1559283968 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9080047a0 con 0x561c219eaa90 2024-09-12T02:12:31.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.435+0000 7fd919f41740 1 -- 172.21.15.16:0/1559283968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561c219eaa90 msgr2=0x561c21a027c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:31.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.435+0000 7fd919f41740 1 --2- 172.21.15.16:0/1559283968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561c219eaa90 0x561c21a027c0 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7fd908035000 tx=0x7fd908042000 comp rx=0 tx=0).stop 2024-09-12T02:12:31.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.435+0000 7fd919f41740 1 -- 172.21.15.16:0/1559283968 shutdown_connections 2024-09-12T02:12:31.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.435+0000 7fd919f41740 1 --2- 172.21.15.16:0/1559283968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561c219eaa90 0x561c21a027c0 unknown :-1 s=CLOSED pgs=1466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:31.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.435+0000 7fd919f41740 1 --2- 172.21.15.16:0/1559283968 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561c219dbca0 0x561c21a02280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:31.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.435+0000 7fd919f41740 1 --2- 172.21.15.16:0/1559283968 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561c219db380 0x561c219f6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:31.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.435+0000 7fd919f41740 1 -- 172.21.15.16:0/1559283968 >> 172.21.15.16:0/1559283968 conn(0x561c219d7a30 msgr2=0x561c219d6b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:31.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.435+0000 7fd919f41740 1 -- 172.21.15.16:0/1559283968 shutdown_connections 2024-09-12T02:12:31.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.435+0000 7fd919f41740 1 -- 172.21.15.16:0/1559283968 wait complete. 2024-09-12T02:12:31.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.436+0000 7fd919f41740 1 -- start start 2024-09-12T02:12:31.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.436+0000 7fd919f41740 1 Processor -- start 2024-09-12T02:12:31.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.436+0000 7fd919f41740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561c219db380 0x561c219eebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:31.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.436+0000 7fd919f41740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561c219dbca0 0x561c219ef130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:31.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.436+0000 7fd919f41740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561c219eaa90 0x561c21a64250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:31.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.436+0000 7fd919f41740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd9101a98c0 con 0x561c219dbca0 2024-09-12T02:12:31.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.436+0000 7fd919f41740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd9101aad10 con 0x561c219db380 2024-09-12T02:12:31.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.436+0000 7fd919f41740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd9101a9f70 con 0x561c219eaa90 2024-09-12T02:12:31.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.436+0000 7fd919f41740 10 client.0 _ll_register_callbacks cb 0x7fd900010380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:12:31.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.436+0000 7fd915103640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561c219dbca0 0x561c219ef130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:31.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.436+0000 7fd915103640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561c219dbca0 0x561c219ef130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36518/0 (socket says 172.21.15.16:36518) 2024-09-12T02:12:31.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.436+0000 7fd915103640 1 -- 172.21.15.16:0/1026790666 learned_addr learned my addr 172.21.15.16:0/1026790666 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:12:31.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.436+0000 7fd915904640 1 --2- 172.21.15.16:0/1026790666 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561c219db380 0x561c219eebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:31.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.436+0000 7fd916105640 1 --2- 172.21.15.16:0/1026790666 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561c219eaa90 0x561c21a64250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:31.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.437+0000 7fd915103640 1 -- 172.21.15.16:0/1026790666 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561c219eaa90 msgr2=0x561c21a64250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:31.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.437+0000 7fd915103640 1 --2- 172.21.15.16:0/1026790666 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561c219eaa90 0x561c21a64250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:31.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.437+0000 7fd915103640 1 -- 172.21.15.16:0/1026790666 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561c219db380 msgr2=0x561c219eebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:31.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.437+0000 7fd915103640 1 --2- 172.21.15.16:0/1026790666 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561c219db380 0x561c219eebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:31.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.437+0000 7fd915103640 1 -- 172.21.15.16:0/1026790666 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c21a64790 con 0x561c219dbca0 2024-09-12T02:12:31.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.437+0000 7fd915103640 1 --2- 172.21.15.16:0/1026790666 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561c219dbca0 0x561c219ef130 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7fd9000cc1c0 tx=0x7fd90000d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d51a162bf30bc88 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:31.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.437+0000 7fd8f97fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:12:31.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.437+0000 7fd8f97fa640 1 -- 172.21.15.16:0/1026790666 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd90009aca0 con 0x561c219dbca0 2024-09-12T02:12:31.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.437+0000 7fd8f97fa640 1 -- 172.21.15.16:0/1026790666 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fd9000b7020 con 0x561c219dbca0 2024-09-12T02:12:31.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.437+0000 7fd919f41740 1 -- 172.21.15.16:0/1026790666 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd908036070 con 0x561c219dbca0 2024-09-12T02:12:31.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.437+0000 7fd8f97fa640 1 -- 172.21.15.16:0/1026790666 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd9000a3b40 con 0x561c219dbca0 2024-09-12T02:12:31.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.437+0000 7fd919f41740 10 client.18705 fetch_fsmap learned FSMap version 25 2024-09-12T02:12:31.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.437+0000 7fd919f41740 1 -- 172.21.15.16:0/1026790666 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x561c21a65960 con 0x561c219dbca0 2024-09-12T02:12:31.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.437+0000 7fd8f97fa640 1 -- 172.21.15.16:0/1026790666 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd908036070 con 0x561c219dbca0 2024-09-12T02:12:31.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.437+0000 7fd8f97fa640 1 -- 172.21.15.16:0/1026790666 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fd900083030 con 0x561c219dbca0 2024-09-12T02:12:31.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.438+0000 7fd919f41740 10 client.18705 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:12:31.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.438+0000 7fd919f41740 10 client.18705 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:12:31.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.438+0000 7fd919f41740 20 client.18705 populate_metadata read hostname 'smithi016' 2024-09-12T02:12:31.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.438+0000 7fd919f41740 1 --2- 172.21.15.16:0/1026790666 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x561c21a66130 0x561c21a68550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:31.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.438+0000 7fd919f41740 4 client.18705 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T02:12:31.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.438+0000 7fd919f41740 1 -- 172.21.15.16:0/1026790666 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561c21948d60 con 0x561c21a66130 2024-09-12T02:12:31.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.438+0000 7fd915904640 1 --2- 172.21.15.16:0/1026790666 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x561c21a66130 0x561c21a68550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:31.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.438+0000 7fd915904640 1 --2- 172.21.15.16:0/1026790666 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x561c21a66130 0x561c21a68550 crc :-1 s=READY pgs=311 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bcd59eec6615c4c0 server_cookie=d3577901da3869da in_seq=0 out_seq=0 2024-09-12T02:12:31.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.438+0000 7fd8f97fa640 10 client.18705 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:12:31.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.439+0000 7fd8f97fa640 1 -- 172.21.15.16:0/1026790666 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x561c21948d60 con 0x561c21a66130 2024-09-12T02:12:31.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.439+0000 7fd8f97fa640 10 client.18705 handle_command_reply: tid=0 2024-09-12T02:12:31.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.439+0000 7fd919f41740 1 client.18705 shutdown 2024-09-12T02:12:31.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.439+0000 7fd919f41740 1 -- 172.21.15.16:0/1026790666 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561c219dbca0 msgr2=0x561c219ef130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:31.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.439+0000 7fd919f41740 1 --2- 172.21.15.16:0/1026790666 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561c219dbca0 0x561c219ef130 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7fd9000cc1c0 tx=0x7fd90000d040 comp rx=0 tx=0).stop 2024-09-12T02:12:31.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.439+0000 7fd919f41740 1 -- 172.21.15.16:0/1026790666 shutdown_connections 2024-09-12T02:12:31.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.439+0000 7fd919f41740 1 --2- 172.21.15.16:0/1026790666 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x561c21a66130 0x561c21a68550 crc :-1 s=READY pgs=311 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:31.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.439+0000 7fd919f41740 1 --2- 172.21.15.16:0/1026790666 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561c219eaa90 0x561c21a64250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:31.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.439+0000 7fd8f97fa640 0 client.18705 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:12:31.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.439+0000 7fd919f41740 1 --2- 172.21.15.16:0/1026790666 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561c219dbca0 0x561c219ef130 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:31.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.439+0000 7fd919f41740 1 --2- 172.21.15.16:0/1026790666 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561c219db380 0x561c219eebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:31.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.439+0000 7fd919f41740 1 -- 172.21.15.16:0/1026790666 >> 172.21.15.16:0/1026790666 conn(0x561c219d7a30 msgr2=0x561c219dce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:31.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.439+0000 7fd919f41740 1 -- 172.21.15.16:0/1026790666 shutdown_connections 2024-09-12T02:12:31.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.439+0000 7fd919f41740 1 -- 172.21.15.16:0/1026790666 wait complete. 2024-09-12T02:12:31.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.439+0000 7fd919f41740 20 client.18705 trim_cache size 0 max 16384 2024-09-12T02:12:31.447 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T02:12:31.447 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-09-12T02:12:31.447 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {} 2024-09-12T02:12:31.448 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T02:12:31.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.439+0000 7fd917b68640 1 -- 172.21.15.16:0/1530454567 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fd8fc0a2b80 msgr2=0x7fd8fc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:31.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.439+0000 7fd917b68640 1 --2- 172.21.15.16:0/1530454567 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fd8fc0a2b80 0x7fd8fc0a5040 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fd908009a60 tx=0x7fd9080095c0 comp rx=0 tx=0).stop 2024-09-12T02:12:31.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.439+0000 7fd917b68640 1 -- 172.21.15.16:0/1530454567 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd910103780 msgr2=0x7fd910198b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:31.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.440+0000 7fd917b68640 1 --2- 172.21.15.16:0/1530454567 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd910103780 0x7fd910198b00 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7fd900006fd0 tx=0x7fd90000b8e0 comp rx=0 tx=0).stop 2024-09-12T02:12:31.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.440+0000 7fd917b68640 1 -- 172.21.15.16:0/1530454567 shutdown_connections 2024-09-12T02:12:31.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.440+0000 7fd917b68640 1 --2- 172.21.15.16:0/1530454567 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fd8fc0a2b80 0x7fd8fc0a5040 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:31.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.440+0000 7fd917b68640 1 --2- 172.21.15.16:0/1530454567 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd910108300 0x7fd91019ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:31.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.440+0000 7fd917b68640 1 --2- 172.21.15.16:0/1530454567 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd910103780 0x7fd910198b00 unknown :-1 s=CLOSED pgs=1463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:31.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.440+0000 7fd917b68640 1 --2- 172.21.15.16:0/1530454567 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd910102dd0 0x7fd9101985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:31.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.440+0000 7fd917b68640 1 -- 172.21.15.16:0/1530454567 >> 172.21.15.16:0/1530454567 conn(0x7fd9100fc7d0 msgr2=0x7fd910105e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:31.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.440+0000 7fd917b68640 1 -- 172.21.15.16:0/1530454567 shutdown_connections 2024-09-12T02:12:31.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:31.440+0000 7fd917b68640 1 -- 172.21.15.16:0/1530454567 wait complete. 2024-09-12T02:12:32.151 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:12:31 smithi016 ceph-mon[30947]: pgmap v653: 97 pgs: 97 active+clean; 1.4 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-09-12T02:12:32.182 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-12T02:12:32.182 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:12:32.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:12:31 smithi184 ceph-mon[35593]: pgmap v653: 97 pgs: 97 active+clean; 1.4 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-09-12T02:12:32.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:12:31 smithi033 ceph-mon[35332]: pgmap v653: 97 pgs: 97 active+clean; 1.4 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-09-12T02:12:32.441 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:12:32.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:32.936+0000 7fe8f444f640 1 -- 172.21.15.16:0/2694351686 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8ec102790 msgr2=0x7fe8ec1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:32.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:32.936+0000 7fe8f444f640 1 --2- 172.21.15.16:0/2694351686 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8ec102790 0x7fe8ec1067e0 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7fe8e0009900 tx=0x7fe8e002a410 comp rx=0 tx=0).stop 2024-09-12T02:12:32.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:32.937+0000 7fe8f444f640 1 -- 172.21.15.16:0/2694351686 shutdown_connections 2024-09-12T02:12:32.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:32.937+0000 7fe8f444f640 1 --2- 172.21.15.16:0/2694351686 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe8ec106f10 0x7fe8ec10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:32.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:32.937+0000 7fe8f444f640 1 --2- 172.21.15.16:0/2694351686 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8ec102790 0x7fe8ec1067e0 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:32.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:32.937+0000 7fe8f444f640 1 --2- 172.21.15.16:0/2694351686 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe8ec101de0 0x7fe8ec1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:32.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:32.937+0000 7fe8f444f640 1 -- 172.21.15.16:0/2694351686 >> 172.21.15.16:0/2694351686 conn(0x7fe8ec0fd8c0 msgr2=0x7fe8ec0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:32.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:32.937+0000 7fe8f444f640 1 -- 172.21.15.16:0/2694351686 shutdown_connections 2024-09-12T02:12:32.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:32.937+0000 7fe8f444f640 1 -- 172.21.15.16:0/2694351686 wait complete. 2024-09-12T02:12:32.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:32.937+0000 7fe8f444f640 1 Processor -- start 2024-09-12T02:12:32.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:32.937+0000 7fe8f444f640 1 -- start start 2024-09-12T02:12:32.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:32.938+0000 7fe8f444f640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe8ec101de0 0x7fe8ec0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:32.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:32.938+0000 7fe8f444f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8ec102790 0x7fe8ec074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:32.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:32.938+0000 7fe8f444f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe8ec106f10 0x7fe8ec074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:32.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:32.938+0000 7fe8f444f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe8ec111fc0 con 0x7fe8ec102790 2024-09-12T02:12:32.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:32.938+0000 7fe8f444f640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe8ec111e40 con 0x7fe8ec106f10 2024-09-12T02:12:32.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:32.938+0000 7fe8f444f640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe8ec112140 con 0x7fe8ec101de0 2024-09-12T02:12:32.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:32.938+0000 7fe8f19ea640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8ec102790 0x7fe8ec074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:32.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:32.938+0000 7fe8f19ea640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8ec102790 0x7fe8ec074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36528/0 (socket says 172.21.15.16:36528) 2024-09-12T02:12:32.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:32.938+0000 7fe8f19ea640 1 -- 172.21.15.16:0/2637725145 learned_addr learned my addr 172.21.15.16:0/2637725145 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:12:32.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:32.938+0000 7fe8f21eb640 1 --2- 172.21.15.16:0/2637725145 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe8ec101de0 0x7fe8ec0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:32.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:32.938+0000 7fe8f29ec640 1 --2- 172.21.15.16:0/2637725145 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe8ec106f10 0x7fe8ec074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:32.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:32.938+0000 7fe8f19ea640 1 -- 172.21.15.16:0/2637725145 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe8ec101de0 msgr2=0x7fe8ec0744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:32.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:32.938+0000 7fe8f19ea640 1 --2- 172.21.15.16:0/2637725145 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe8ec101de0 0x7fe8ec0744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:32.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:32.938+0000 7fe8f19ea640 1 -- 172.21.15.16:0/2637725145 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe8ec106f10 msgr2=0x7fe8ec074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:32.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:32.938+0000 7fe8f19ea640 1 --2- 172.21.15.16:0/2637725145 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe8ec106f10 0x7fe8ec074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:32.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:32.938+0000 7fe8f19ea640 1 -- 172.21.15.16:0/2637725145 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8ec1a6920 con 0x7fe8ec102790 2024-09-12T02:12:32.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:32.938+0000 7fe8f19ea640 1 --2- 172.21.15.16:0/2637725145 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8ec102790 0x7fe8ec074a30 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7fe8e0004770 tx=0x7fe8e0003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:32.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:32.938+0000 7fe8df7fe640 1 -- 172.21.15.16:0/2637725145 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e003f070 con 0x7fe8ec102790 2024-09-12T02:12:32.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:32.938+0000 7fe8df7fe640 1 -- 172.21.15.16:0/2637725145 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe8e0031ca0 con 0x7fe8ec102790 2024-09-12T02:12:32.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:32.938+0000 7fe8df7fe640 1 -- 172.21.15.16:0/2637725145 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e0035030 con 0x7fe8ec102790 2024-09-12T02:12:32.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:32.939+0000 7fe8f444f640 1 -- 172.21.15.16:0/2637725145 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8ec1a6bb0 con 0x7fe8ec102790 2024-09-12T02:12:32.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:32.939+0000 7fe8f444f640 1 -- 172.21.15.16:0/2637725145 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8ec1a70e0 con 0x7fe8ec102790 2024-09-12T02:12:32.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:32.939+0000 7fe8f444f640 1 -- 172.21.15.16:0/2637725145 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8ec1021c0 con 0x7fe8ec102790 2024-09-12T02:12:32.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:32.941+0000 7fe8df7fe640 1 -- 172.21.15.16:0/2637725145 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fe8e0042070 con 0x7fe8ec102790 2024-09-12T02:12:32.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:32.941+0000 7fe8df7fe640 1 --2- 172.21.15.16:0/2637725145 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fe8d80a2c00 0x7fe8d80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:32.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:32.941+0000 7fe8f21eb640 1 --2- 172.21.15.16:0/2637725145 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fe8d80a2c00 0x7fe8d80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:32.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:32.941+0000 7fe8df7fe640 1 -- 172.21.15.16:0/2637725145 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fe8e00ef710 con 0x7fe8ec102790 2024-09-12T02:12:32.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:32.942+0000 7fe8f21eb640 1 --2- 172.21.15.16:0/2637725145 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fe8d80a2c00 0x7fe8d80a50c0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fe8e800bbe0 tx=0x7fe8e800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:32.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:32.946+0000 7fe8df7fe640 1 -- 172.21.15.16:0/2637725145 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe8e00bc710 con 0x7fe8ec102790 2024-09-12T02:12:33.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:33.158+0000 7fe8f444f640 1 -- 172.21.15.16:0/2637725145 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe8ec10a150 con 0x7fe8ec102790 2024-09-12T02:12:33.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:33.159+0000 7fe8df7fe640 1 -- 172.21.15.16:0/2637725145 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11911 (secure 0 0 0) 0x7fe8e00bc010 con 0x7fe8ec102790 2024-09-12T02:12:33.161 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T02:12:33.161 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":25,"btime":"2024-09-12T02:01:37:192021+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3279123279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3279123279},{"type":"v1","addr":"172.21.15.16:6835","nonce":3279123279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4119452382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4119452382},{"type":"v1","addr":"172.21.15.16:6837","nonce":4119452382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/3263823165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3263823165},{"type":"v1","addr":"172.21.15.16:6839","nonce":3263823165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/1510290732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":1510290732},{"type":"v1","addr":"172.21.15.16:6841","nonce":1510290732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/279674603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":279674603},{"type":"v1","addr":"172.21.15.33:6833","nonce":279674603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3555728028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3555728028},{"type":"v1","addr":"172.21.15.33:6835","nonce":3555728028}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T01:59:56.117608+0000","modified":"2024-09-12T02:01:36.294760+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24482,"mds_1":24481,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6833/1563374106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1563374106},{"type":"v1","addr":"172.21.15.184:6833","nonce":1563374106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6837/1892833868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1892833868},{"type":"v1","addr":"172.21.15.33:6837","nonce":1892833868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/1457679686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1457679686},{"type":"v1","addr":"172.21.15.184:6835","nonce":1457679686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.184:6837/1793393087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":1793393087},{"type":"v1","addr":"172.21.15.184:6837","nonce":1793393087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.184:6839/4203347941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":4203347941},{"type":"v1","addr":"172.21.15.184:6839","nonce":4203347941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.33:6839/4136670204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":4136670204},{"type":"v1","addr":"172.21.15.33:6839","nonce":4136670204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24481,24482,24469]},"id":1}]} 2024-09-12T02:12:33.161 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 25 2024-09-12T02:12:33.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:33.162+0000 7fe8f444f640 1 -- 172.21.15.16:0/2637725145 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fe8d80a2c00 msgr2=0x7fe8d80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:33.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:33.162+0000 7fe8f444f640 1 --2- 172.21.15.16:0/2637725145 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fe8d80a2c00 0x7fe8d80a50c0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fe8e800bbe0 tx=0x7fe8e800b390 comp rx=0 tx=0).stop 2024-09-12T02:12:33.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:33.162+0000 7fe8f444f640 1 -- 172.21.15.16:0/2637725145 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8ec102790 msgr2=0x7fe8ec074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:33.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:33.162+0000 7fe8f444f640 1 --2- 172.21.15.16:0/2637725145 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8ec102790 0x7fe8ec074a30 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7fe8e0004770 tx=0x7fe8e0003c10 comp rx=0 tx=0).stop 2024-09-12T02:12:33.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:33.162+0000 7fe8f444f640 1 -- 172.21.15.16:0/2637725145 shutdown_connections 2024-09-12T02:12:33.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:33.162+0000 7fe8f444f640 1 --2- 172.21.15.16:0/2637725145 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fe8d80a2c00 0x7fe8d80a50c0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:33.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:33.162+0000 7fe8f444f640 1 --2- 172.21.15.16:0/2637725145 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe8ec106f10 0x7fe8ec074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:33.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:33.162+0000 7fe8f444f640 1 --2- 172.21.15.16:0/2637725145 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8ec102790 0x7fe8ec074a30 unknown :-1 s=CLOSED pgs=1469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:33.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:33.162+0000 7fe8f444f640 1 --2- 172.21.15.16:0/2637725145 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe8ec101de0 0x7fe8ec0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:33.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:33.162+0000 7fe8f444f640 1 -- 172.21.15.16:0/2637725145 >> 172.21.15.16:0/2637725145 conn(0x7fe8ec0fd8c0 msgr2=0x7fe8ec0fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:33.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:33.162+0000 7fe8f444f640 1 -- 172.21.15.16:0/2637725145 shutdown_connections 2024-09-12T02:12:33.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:33.162+0000 7fe8f444f640 1 -- 172.21.15.16:0/2637725145 wait complete. 2024-09-12T02:12:33.900 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-12T02:12:34.161 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:12:34.183 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:12:33 smithi016 ceph-mon[30947]: pgmap v654: 97 pgs: 97 active+clean; 1.4 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-09-12T02:12:34.183 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:12:33 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/2637725145' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:12:34.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:12:33 smithi184 ceph-mon[35593]: pgmap v654: 97 pgs: 97 active+clean; 1.4 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-09-12T02:12:34.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:12:33 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/2637725145' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:12:34.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:12:33 smithi033 ceph-mon[35332]: pgmap v654: 97 pgs: 97 active+clean; 1.4 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-09-12T02:12:34.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:12:33 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/2637725145' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:12:34.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.564+0000 7f0128465640 1 -- 172.21.15.16:0/3389580993 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0120103660 msgr2=0x7f012010bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:34.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.564+0000 7f0128465640 1 --2- 172.21.15.16:0/3389580993 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0120103660 0x7f012010bb50 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f0110009900 tx=0x7f011002a480 comp rx=0 tx=0).stop 2024-09-12T02:12:34.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.564+0000 7f0128465640 1 -- 172.21.15.16:0/3389580993 shutdown_connections 2024-09-12T02:12:34.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.564+0000 7f0128465640 1 --2- 172.21.15.16:0/3389580993 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f012010c170 0x7f012010e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:34.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.564+0000 7f0128465640 1 --2- 172.21.15.16:0/3389580993 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0120103660 0x7f012010bb50 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:34.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.564+0000 7f0128465640 1 --2- 172.21.15.16:0/3389580993 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0120102d40 0x7f0120103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:34.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.564+0000 7f0128465640 1 -- 172.21.15.16:0/3389580993 >> 172.21.15.16:0/3389580993 conn(0x7f01200fc7a0 msgr2=0x7f01200febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:34.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.564+0000 7f0128465640 1 -- 172.21.15.16:0/3389580993 shutdown_connections 2024-09-12T02:12:34.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.564+0000 7f0128465640 1 -- 172.21.15.16:0/3389580993 wait complete. 2024-09-12T02:12:34.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.566+0000 7f0128465640 1 Processor -- start 2024-09-12T02:12:34.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.566+0000 7f0128465640 1 -- start start 2024-09-12T02:12:34.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.566+0000 7f0128465640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0120102d40 0x7f0120198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:34.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.566+0000 7f0128465640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0120103660 0x7f0120198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:34.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.566+0000 7f0128465640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f012010c170 0x7f012019cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:34.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.566+0000 7f0128465640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0120111f70 con 0x7f012010c170 2024-09-12T02:12:34.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.566+0000 7f0128465640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0120111df0 con 0x7f0120102d40 2024-09-12T02:12:34.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.566+0000 7f0128465640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f01201120f0 con 0x7f0120103660 2024-09-12T02:12:34.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.566+0000 7f0126a02640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f012010c170 0x7f012019cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:34.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.566+0000 7f0126a02640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f012010c170 0x7f012019cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36554/0 (socket says 172.21.15.16:36554) 2024-09-12T02:12:34.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.566+0000 7f0126201640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0120102d40 0x7f0120198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:34.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.566+0000 7f0126a02640 1 -- 172.21.15.16:0/4268866423 learned_addr learned my addr 172.21.15.16:0/4268866423 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:12:34.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.566+0000 7f0125a00640 1 --2- 172.21.15.16:0/4268866423 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0120103660 0x7f0120198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:34.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.566+0000 7f0126a02640 1 -- 172.21.15.16:0/4268866423 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0120103660 msgr2=0x7f0120198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:34.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.566+0000 7f0126a02640 1 --2- 172.21.15.16:0/4268866423 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0120103660 0x7f0120198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:34.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.566+0000 7f0126a02640 1 -- 172.21.15.16:0/4268866423 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0120102d40 msgr2=0x7f0120198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:34.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.566+0000 7f0126a02640 1 --2- 172.21.15.16:0/4268866423 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0120102d40 0x7f0120198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:34.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.566+0000 7f0126a02640 1 -- 172.21.15.16:0/4268866423 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f012019d3e0 con 0x7f012010c170 2024-09-12T02:12:34.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.566+0000 7f0126a02640 1 --2- 172.21.15.16:0/4268866423 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f012010c170 0x7f012019cd00 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f011800cad0 tx=0x7f011800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:34.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.566+0000 7f01177fe640 1 -- 172.21.15.16:0/4268866423 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f011800ae20 con 0x7f012010c170 2024-09-12T02:12:34.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.566+0000 7f01177fe640 1 -- 172.21.15.16:0/4268866423 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0118005020 con 0x7f012010c170 2024-09-12T02:12:34.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.566+0000 7f01177fe640 1 -- 172.21.15.16:0/4268866423 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f011800f6f0 con 0x7f012010c170 2024-09-12T02:12:34.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.566+0000 7f0128465640 1 -- 172.21.15.16:0/4268866423 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f012019d6d0 con 0x7f012010c170 2024-09-12T02:12:34.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.566+0000 7f0128465640 1 -- 172.21.15.16:0/4268866423 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01201a4fb0 con 0x7f012010c170 2024-09-12T02:12:34.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.568+0000 7f01177fe640 1 -- 172.21.15.16:0/4268866423 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f01180051d0 con 0x7f012010c170 2024-09-12T02:12:34.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.569+0000 7f012a83e740 1 Processor -- start 2024-09-12T02:12:34.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.569+0000 7f012a83e740 1 -- start start 2024-09-12T02:12:34.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.569+0000 7f012a83e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3802a2b40 0x55c3802a1090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:34.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.569+0000 7f012a83e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3802a15d0 0x55c3802a1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:34.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.570+0000 7f012a83e740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c3802ac370 0x55c3802ae760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:34.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.570+0000 7f012a83e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55c3802072f0 con 0x55c3802a15d0 2024-09-12T02:12:34.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.570+0000 7f012a83e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55c3801aecf0 con 0x55c3802a2b40 2024-09-12T02:12:34.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.570+0000 7f012a83e740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55c3802032b0 con 0x55c3802ac370 2024-09-12T02:12:34.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.570+0000 7f0126201640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3802a15d0 0x55c3802a1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:34.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.570+0000 7f0126201640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3802a15d0 0x55c3802a1a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36566/0 (socket says 172.21.15.16:36566) 2024-09-12T02:12:34.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.570+0000 7f01177fe640 1 --2- 172.21.15.16:0/4268866423 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f010c0a2c00 0x7f010c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:34.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.570+0000 7f0126201640 1 -- 172.21.15.16:0/2172611522 learned_addr learned my addr 172.21.15.16:0/2172611522 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:12:34.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.570+0000 7f0125a00640 1 --2- 172.21.15.16:0/2172611522 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3802a2b40 0x55c3802a1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:34.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.570+0000 7f0125a00640 1 --2- 172.21.15.16:0/2172611522 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c3802ac370 0x55c3802ae760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:34.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.570+0000 7f01177fe640 1 -- 172.21.15.16:0/4268866423 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f01180cf220 con 0x7f012010c170 2024-09-12T02:12:34.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.570+0000 7f0126a02640 1 --2- 172.21.15.16:0/4268866423 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f010c0a2c00 0x7f010c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:34.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.570+0000 7f0125a00640 1 -- 172.21.15.16:0/2172611522 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c3802ac370 msgr2=0x55c3802ae760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:34.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.570+0000 7f0125a00640 1 --2- 172.21.15.16:0/2172611522 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c3802ac370 0x55c3802ae760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:34.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.570+0000 7f0125a00640 1 -- 172.21.15.16:0/2172611522 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3802a15d0 msgr2=0x55c3802a1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:34.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.570+0000 7f0125a00640 1 --2- 172.21.15.16:0/2172611522 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3802a15d0 0x55c3802a1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:34.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.570+0000 7f0125a00640 1 -- 172.21.15.16:0/2172611522 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c3801933a0 con 0x55c3802a2b40 2024-09-12T02:12:34.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.570+0000 7f0126201640 1 --2- 172.21.15.16:0/2172611522 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3802a15d0 0x55c3802a1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:12:34.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.570+0000 7f0125a00640 1 --2- 172.21.15.16:0/2172611522 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3802a2b40 0x55c3802a1090 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f0110009230 tx=0x7f01100056d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=70f3ad5dd828f347 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:34.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.571+0000 7f0126a02640 1 --2- 172.21.15.16:0/4268866423 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f010c0a2c00 0x7f010c0a50c0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f011800caa0 tx=0x7f0118000f40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:34.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.571+0000 7f0114dea640 1 -- 172.21.15.16:0/2172611522 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0110035040 con 0x55c3802a2b40 2024-09-12T02:12:34.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.571+0000 7f0114dea640 1 -- 172.21.15.16:0/2172611522 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f01100035f0 con 0x55c3802a2b40 2024-09-12T02:12:34.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.571+0000 7f0114dea640 1 -- 172.21.15.16:0/2172611522 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f011003d6a0 con 0x55c3802a2b40 2024-09-12T02:12:34.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.571+0000 7f012a83e740 1 -- 172.21.15.16:0/2172611522 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3802a2b40 msgr2=0x55c3802a1090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:34.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.571+0000 7f012a83e740 1 --2- 172.21.15.16:0/2172611522 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3802a2b40 0x55c3802a1090 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f0110009230 tx=0x7f01100056d0 comp rx=0 tx=0).stop 2024-09-12T02:12:34.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.571+0000 7f012a83e740 1 -- 172.21.15.16:0/2172611522 shutdown_connections 2024-09-12T02:12:34.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.571+0000 7f012a83e740 1 --2- 172.21.15.16:0/2172611522 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c3802ac370 0x55c3802ae760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:34.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.571+0000 7f012a83e740 1 --2- 172.21.15.16:0/2172611522 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3802a15d0 0x55c3802a1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:34.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.571+0000 7f012a83e740 1 --2- 172.21.15.16:0/2172611522 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3802a2b40 0x55c3802a1090 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:34.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.571+0000 7f012a83e740 1 -- 172.21.15.16:0/2172611522 >> 172.21.15.16:0/2172611522 conn(0x55c38026a310 msgr2=0x55c38029dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:34.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.572+0000 7f012a83e740 1 -- 172.21.15.16:0/2172611522 shutdown_connections 2024-09-12T02:12:34.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.572+0000 7f012a83e740 1 -- 172.21.15.16:0/2172611522 wait complete. 2024-09-12T02:12:34.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.572+0000 7f012a83e740 1 -- start start 2024-09-12T02:12:34.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.572+0000 7f012a83e740 1 Processor -- start 2024-09-12T02:12:34.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.572+0000 7f012a83e740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c3802a15d0 0x55c3802abdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:34.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.572+0000 7f012a83e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3802a2b40 0x55c3802a9060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:34.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.572+0000 7f012a83e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3802ac370 0x55c3802a97f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:34.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.572+0000 7f012a83e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55c3802af010 con 0x55c3802a2b40 2024-09-12T02:12:34.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.572+0000 7f012a83e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55c3801c2060 con 0x55c3802ac370 2024-09-12T02:12:34.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.572+0000 7f012a83e740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55c3802af190 con 0x55c3802a15d0 2024-09-12T02:12:34.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.572+0000 7f012a83e740 10 client.0 _ll_register_callbacks cb 0x55c37ff158c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:12:34.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.572+0000 7f0126201640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3802a2b40 0x55c3802a9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:34.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.572+0000 7f0126201640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3802a2b40 0x55c3802a9060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36576/0 (socket says 172.21.15.16:36576) 2024-09-12T02:12:34.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.573+0000 7f0126201640 1 -- 172.21.15.16:0/202994613 learned_addr learned my addr 172.21.15.16:0/202994613 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:12:34.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.573+0000 7f0125a00640 1 --2- 172.21.15.16:0/202994613 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3802ac370 0x55c3802a97f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:34.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.573+0000 7f0125a00640 1 --2- 172.21.15.16:0/202994613 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c3802a15d0 0x55c3802abdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:34.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.573+0000 7f0126201640 1 -- 172.21.15.16:0/202994613 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c3802a15d0 msgr2=0x55c3802abdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:34.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.573+0000 7f0126201640 1 --2- 172.21.15.16:0/202994613 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c3802a15d0 0x55c3802abdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:34.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.573+0000 7f0126201640 1 -- 172.21.15.16:0/202994613 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3802ac370 msgr2=0x55c3802a97f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:34.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.573+0000 7f0126201640 1 --2- 172.21.15.16:0/202994613 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3802ac370 0x55c3802a97f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:34.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.573+0000 7f0126201640 1 -- 172.21.15.16:0/202994613 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c37ff783c0 con 0x55c3802a2b40 2024-09-12T02:12:34.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.573+0000 7f0126201640 1 --2- 172.21.15.16:0/202994613 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3802a2b40 0x55c3802a9060 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7f011c00bac0 tx=0x7f011c004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d289859f3b24a70e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:34.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.573+0000 7f0109ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:12:34.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.573+0000 7f0109ffb640 1 -- 172.21.15.16:0/202994613 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f011c011440 con 0x55c3802a2b40 2024-09-12T02:12:34.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.573+0000 7f0109ffb640 1 -- 172.21.15.16:0/202994613 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f011c0115e0 con 0x55c3802a2b40 2024-09-12T02:12:34.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.573+0000 7f012a83e740 1 -- 172.21.15.16:0/202994613 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f01100035f0 con 0x55c3802a2b40 2024-09-12T02:12:34.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.573+0000 7f0109ffb640 1 -- 172.21.15.16:0/202994613 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f011c0117e0 con 0x55c3802a2b40 2024-09-12T02:12:34.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.573+0000 7f012a83e740 10 client.18723 fetch_fsmap learned FSMap version 25 2024-09-12T02:12:34.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.573+0000 7f012a83e740 1 -- 172.21.15.16:0/202994613 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c3802bab60 con 0x55c3802a2b40 2024-09-12T02:12:34.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.574+0000 7f0109ffb640 1 -- 172.21.15.16:0/202994613 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f01100035f0 con 0x55c3802a2b40 2024-09-12T02:12:34.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.574+0000 7f0109ffb640 1 -- 172.21.15.16:0/202994613 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f011c01bac0 con 0x55c3802a2b40 2024-09-12T02:12:34.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.574+0000 7f012a83e740 10 client.18723 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:12:34.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.574+0000 7f012a83e740 10 client.18723 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-12T02:12:34.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.574+0000 7f012a83e740 20 client.18723 populate_metadata read hostname 'smithi016' 2024-09-12T02:12:34.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.574+0000 7f012a83e740 1 --2- 172.21.15.16:0/202994613 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55c3802bc010 0x55c3802be430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:34.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.574+0000 7f012a83e740 4 client.18723 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:12:34.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.574+0000 7f012a83e740 1 -- 172.21.15.16:0/202994613 --> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c380214860 con 0x55c3802bc010 2024-09-12T02:12:34.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.574+0000 7f0125a00640 1 --2- 172.21.15.16:0/202994613 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55c3802bc010 0x55c3802be430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:34.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.575+0000 7f0125a00640 1 --2- 172.21.15.16:0/202994613 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55c3802bc010 0x55c3802be430 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d6eee887e2103c45 server_cookie=226e4c4d64ee1bdf in_seq=0 out_seq=0 2024-09-12T02:12:34.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.575+0000 7f0109ffb640 10 client.18723 ms_handle_connect on v2:172.21.15.184:6836/1793393087 2024-09-12T02:12:34.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.575+0000 7f0109ffb640 1 -- 172.21.15.16:0/202994613 <== mds.2 v2:172.21.15.184:6836/1793393087 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c380214860 con 0x55c3802bc010 2024-09-12T02:12:34.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.575+0000 7f0109ffb640 10 client.18723 handle_command_reply: tid=0 2024-09-12T02:12:34.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.575+0000 7f012a83e740 1 client.18723 shutdown 2024-09-12T02:12:34.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.576+0000 7f012a83e740 1 -- 172.21.15.16:0/202994613 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3802a2b40 msgr2=0x55c3802a9060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:34.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.576+0000 7f012a83e740 1 --2- 172.21.15.16:0/202994613 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3802a2b40 0x55c3802a9060 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7f011c00bac0 tx=0x7f011c004600 comp rx=0 tx=0).stop 2024-09-12T02:12:34.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.576+0000 7f012a83e740 1 -- 172.21.15.16:0/202994613 shutdown_connections 2024-09-12T02:12:34.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.576+0000 7f012a83e740 1 --2- 172.21.15.16:0/202994613 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3802ac370 0x55c3802a97f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:34.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.576+0000 7f012a83e740 1 --2- 172.21.15.16:0/202994613 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55c3802bc010 0x55c3802be430 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:34.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.576+0000 7f0109ffb640 0 client.18723 ms_handle_reset on v2:172.21.15.184:6836/1793393087 2024-09-12T02:12:34.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.576+0000 7f012a83e740 1 --2- 172.21.15.16:0/202994613 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3802a2b40 0x55c3802a9060 unknown :-1 s=CLOSED pgs=1472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:34.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.576+0000 7f012a83e740 1 --2- 172.21.15.16:0/202994613 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c3802a15d0 0x55c3802abdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:34.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.576+0000 7f012a83e740 1 -- 172.21.15.16:0/202994613 >> 172.21.15.16:0/202994613 conn(0x55c38026a310 msgr2=0x55c3802a3960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:34.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.576+0000 7f012a83e740 1 -- 172.21.15.16:0/202994613 shutdown_connections 2024-09-12T02:12:34.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.576+0000 7f012a83e740 1 -- 172.21.15.16:0/202994613 wait complete. 2024-09-12T02:12:34.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.576+0000 7f012a83e740 20 client.18723 trim_cache size 0 max 16384 2024-09-12T02:12:34.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.590+0000 7f012a83e740 1 Processor -- start 2024-09-12T02:12:34.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.590+0000 7f012a83e740 1 -- start start 2024-09-12T02:12:34.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.590+0000 7f012a83e740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c3802a15d0 0x55c3802a2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:34.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.590+0000 7f012a83e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3802a2b40 0x55c3802a8040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:34.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.590+0000 7f012a83e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3802ac370 0x55c3802c3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:34.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.590+0000 7f012a83e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f01201afd00 con 0x55c3802a2b40 2024-09-12T02:12:34.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.590+0000 7f012a83e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f01201ad5d0 con 0x55c3802ac370 2024-09-12T02:12:34.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.590+0000 7f012a83e740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f01201ab440 con 0x55c3802a15d0 2024-09-12T02:12:34.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.590+0000 7f0126201640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3802a2b40 0x55c3802a8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:34.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.590+0000 7f0126201640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3802a2b40 0x55c3802a8040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36592/0 (socket says 172.21.15.16:36592) 2024-09-12T02:12:34.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.590+0000 7f0126201640 1 -- 172.21.15.16:0/963168841 learned_addr learned my addr 172.21.15.16:0/963168841 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:12:34.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.590+0000 7f0125a00640 1 --2- 172.21.15.16:0/963168841 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c3802a15d0 0x55c3802a2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:34.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.590+0000 7f0125a00640 1 --2- 172.21.15.16:0/963168841 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3802ac370 0x55c3802c3ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:34.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.590+0000 7f0126201640 1 -- 172.21.15.16:0/963168841 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c3802a15d0 msgr2=0x55c3802a2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:34.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.590+0000 7f0126201640 1 --2- 172.21.15.16:0/963168841 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c3802a15d0 0x55c3802a2300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:34.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.590+0000 7f0126201640 1 -- 172.21.15.16:0/963168841 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3802ac370 msgr2=0x55c3802c3ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:34.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.590+0000 7f0126201640 1 --2- 172.21.15.16:0/963168841 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3802ac370 0x55c3802c3ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:34.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.590+0000 7f0126201640 1 -- 172.21.15.16:0/963168841 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c3802be7a0 con 0x55c3802a2b40 2024-09-12T02:12:34.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.590+0000 7f0125a00640 1 --2- 172.21.15.16:0/963168841 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3802ac370 0x55c3802c3ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:12:34.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.591+0000 7f0126201640 1 --2- 172.21.15.16:0/963168841 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3802a2b40 0x55c3802a8040 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f011c0254f0 tx=0x7f011c004050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d83901f42c7e1c75 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:34.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.591+0000 7f010bfff640 1 -- 172.21.15.16:0/963168841 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f011c011440 con 0x55c3802a2b40 2024-09-12T02:12:34.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.591+0000 7f010bfff640 1 -- 172.21.15.16:0/963168841 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f011c0115e0 con 0x55c3802a2b40 2024-09-12T02:12:34.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.591+0000 7f010bfff640 1 -- 172.21.15.16:0/963168841 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f011c0048e0 con 0x55c3802a2b40 2024-09-12T02:12:34.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.592+0000 7f012a83e740 1 -- 172.21.15.16:0/963168841 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3802a2b40 msgr2=0x55c3802a8040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:34.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.592+0000 7f012a83e740 1 --2- 172.21.15.16:0/963168841 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3802a2b40 0x55c3802a8040 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f011c0254f0 tx=0x7f011c004050 comp rx=0 tx=0).stop 2024-09-12T02:12:34.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.592+0000 7f012a83e740 1 -- 172.21.15.16:0/963168841 shutdown_connections 2024-09-12T02:12:34.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.592+0000 7f012a83e740 1 --2- 172.21.15.16:0/963168841 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3802ac370 0x55c3802c3ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:34.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.592+0000 7f012a83e740 1 --2- 172.21.15.16:0/963168841 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3802a2b40 0x55c3802a8040 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:34.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.592+0000 7f012a83e740 1 --2- 172.21.15.16:0/963168841 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c3802a15d0 0x55c3802a2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:34.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.592+0000 7f012a83e740 1 -- 172.21.15.16:0/963168841 >> 172.21.15.16:0/963168841 conn(0x55c38026a310 msgr2=0x55c3802a7930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:34.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.592+0000 7f012a83e740 1 -- 172.21.15.16:0/963168841 shutdown_connections 2024-09-12T02:12:34.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.592+0000 7f012a83e740 1 -- 172.21.15.16:0/963168841 wait complete. 2024-09-12T02:12:34.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.592+0000 7f012a83e740 1 -- start start 2024-09-12T02:12:34.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.592+0000 7f012a83e740 1 Processor -- start 2024-09-12T02:12:34.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.592+0000 7f012a83e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3802a15d0 0x55c3802b4c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:34.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.592+0000 7f012a83e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3802a2b40 0x55c3802aff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:34.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.592+0000 7f012a83e740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c3802ac370 0x55c3802b04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:34.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.592+0000 7f012a83e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f01201aba90 con 0x55c3802a15d0 2024-09-12T02:12:34.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.592+0000 7f012a83e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f01201acf00 con 0x55c3802a2b40 2024-09-12T02:12:34.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.592+0000 7f012a83e740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f01201ac160 con 0x55c3802ac370 2024-09-12T02:12:34.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.592+0000 7f012a83e740 10 client.0 _ll_register_callbacks cb 0x7f011c0040c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:12:34.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.592+0000 7f0125a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3802a15d0 0x55c3802b4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:34.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.592+0000 7f0125a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3802a15d0 0x55c3802b4c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36604/0 (socket says 172.21.15.16:36604) 2024-09-12T02:12:34.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.592+0000 7f0125a00640 1 -- 172.21.15.16:0/2410141408 learned_addr learned my addr 172.21.15.16:0/2410141408 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:12:34.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.592+0000 7f0126201640 1 --2- 172.21.15.16:0/2410141408 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3802a2b40 0x55c3802aff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:34.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.592+0000 7f0125a00640 1 -- 172.21.15.16:0/2410141408 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c3802ac370 msgr2=0x55c3802b04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:34.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.592+0000 7f0125a00640 1 --2- 172.21.15.16:0/2410141408 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c3802ac370 0x55c3802b04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:34.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.592+0000 7f0125a00640 1 -- 172.21.15.16:0/2410141408 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3802a2b40 msgr2=0x55c3802aff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:34.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.592+0000 7f0125a00640 1 --2- 172.21.15.16:0/2410141408 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3802a2b40 0x55c3802aff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:34.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.592+0000 7f0125a00640 1 -- 172.21.15.16:0/2410141408 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c3802b0d00 con 0x55c3802a15d0 2024-09-12T02:12:34.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.592+0000 7f0125a00640 1 --2- 172.21.15.16:0/2410141408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3802a15d0 0x55c3802b4c20 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f0110031ba0 tx=0x7f01100095e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b0b859556b1d3c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:34.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.592+0000 7f0109ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:12:34.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.592+0000 7f0109ffb640 1 -- 172.21.15.16:0/2410141408 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0110035040 con 0x55c3802a15d0 2024-09-12T02:12:34.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.592+0000 7f0109ffb640 1 -- 172.21.15.16:0/2410141408 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f0110052050 con 0x55c3802a15d0 2024-09-12T02:12:34.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.592+0000 7f012a83e740 1 -- 172.21.15.16:0/2410141408 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f011c0115e0 con 0x55c3802a15d0 2024-09-12T02:12:34.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.592+0000 7f0109ffb640 1 -- 172.21.15.16:0/2410141408 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f0110066050 con 0x55c3802a15d0 2024-09-12T02:12:34.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.592+0000 7f012a83e740 10 client.18729 fetch_fsmap learned FSMap version 25 2024-09-12T02:12:34.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.594+0000 7f012a83e740 1 -- 172.21.15.16:0/2410141408 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c380326cc0 con 0x55c3802a15d0 2024-09-12T02:12:34.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.594+0000 7f0109ffb640 1 -- 172.21.15.16:0/2410141408 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f011c0115e0 con 0x55c3802a15d0 2024-09-12T02:12:34.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.594+0000 7f0109ffb640 1 -- 172.21.15.16:0/2410141408 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f011003a040 con 0x55c3802a15d0 2024-09-12T02:12:34.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.594+0000 7f012a83e740 10 client.18729 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:12:34.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.594+0000 7f012a83e740 10 client.18729 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-12T02:12:34.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.594+0000 7f012a83e740 20 client.18729 populate_metadata read hostname 'smithi016' 2024-09-12T02:12:34.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.594+0000 7f012a83e740 1 --2- 172.21.15.16:0/2410141408 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55c3802bc010 0x55c380329010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:34.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.594+0000 7f012a83e740 4 client.18729 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T02:12:34.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.594+0000 7f012a83e740 1 -- 172.21.15.16:0/2410141408 --> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c38020ed70 con 0x55c3802bc010 2024-09-12T02:12:34.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.594+0000 7f0126201640 1 --2- 172.21.15.16:0/2410141408 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55c3802bc010 0x55c380329010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:34.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.595+0000 7f0126201640 1 --2- 172.21.15.16:0/2410141408 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55c3802bc010 0x55c380329010 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fd690e9282d703eb server_cookie=70893784bf14d720 in_seq=0 out_seq=0 2024-09-12T02:12:34.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.595+0000 7f0109ffb640 10 client.18729 ms_handle_connect on v2:172.21.15.184:6836/1793393087 2024-09-12T02:12:34.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.595+0000 7f0109ffb640 1 -- 172.21.15.16:0/2410141408 <== mds.2 v2:172.21.15.184:6836/1793393087 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c38020ed70 con 0x55c3802bc010 2024-09-12T02:12:34.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.595+0000 7f0109ffb640 10 client.18729 handle_command_reply: tid=0 2024-09-12T02:12:34.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.595+0000 7f012a83e740 1 client.18729 shutdown 2024-09-12T02:12:34.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.595+0000 7f012a83e740 1 -- 172.21.15.16:0/2410141408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3802a15d0 msgr2=0x55c3802b4c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:34.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.595+0000 7f012a83e740 1 --2- 172.21.15.16:0/2410141408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3802a15d0 0x55c3802b4c20 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f0110031ba0 tx=0x7f01100095e0 comp rx=0 tx=0).stop 2024-09-12T02:12:34.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.595+0000 7f012a83e740 1 -- 172.21.15.16:0/2410141408 shutdown_connections 2024-09-12T02:12:34.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.595+0000 7f012a83e740 1 --2- 172.21.15.16:0/2410141408 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c3802ac370 0x55c3802b04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:34.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.595+0000 7f012a83e740 1 --2- 172.21.15.16:0/2410141408 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3802a2b40 0x55c3802aff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:34.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.595+0000 7f012a83e740 1 --2- 172.21.15.16:0/2410141408 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55c3802bc010 0x55c380329010 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:34.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.595+0000 7f012a83e740 1 --2- 172.21.15.16:0/2410141408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3802a15d0 0x55c3802b4c20 unknown :-1 s=CLOSED pgs=1474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:34.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.595+0000 7f012a83e740 1 -- 172.21.15.16:0/2410141408 >> 172.21.15.16:0/2410141408 conn(0x55c38026a310 msgr2=0x55c3802a3110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:34.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.595+0000 7f0109ffb640 0 client.18729 ms_handle_reset on v2:172.21.15.184:6836/1793393087 2024-09-12T02:12:34.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.595+0000 7f012a83e740 1 -- 172.21.15.16:0/2410141408 shutdown_connections 2024-09-12T02:12:34.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.595+0000 7f012a83e740 1 -- 172.21.15.16:0/2410141408 wait complete. 2024-09-12T02:12:34.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.595+0000 7f012a83e740 20 client.18729 trim_cache size 0 max 16384 2024-09-12T02:12:34.601 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T02:12:34.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.595+0000 7f0128465640 1 -- 172.21.15.16:0/4268866423 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f010c0a2c00 msgr2=0x7f010c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:34.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.595+0000 7f0128465640 1 --2- 172.21.15.16:0/4268866423 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f010c0a2c00 0x7f010c0a50c0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f011800caa0 tx=0x7f0118000f40 comp rx=0 tx=0).stop 2024-09-12T02:12:34.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.595+0000 7f0128465640 1 -- 172.21.15.16:0/4268866423 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f012010c170 msgr2=0x7f012019cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:34.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.595+0000 7f0128465640 1 --2- 172.21.15.16:0/4268866423 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f012010c170 0x7f012019cd00 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f011800cad0 tx=0x7f011800cf90 comp rx=0 tx=0).stop 2024-09-12T02:12:34.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.595+0000 7f0128465640 1 -- 172.21.15.16:0/4268866423 shutdown_connections 2024-09-12T02:12:34.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.595+0000 7f0128465640 1 --2- 172.21.15.16:0/4268866423 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f010c0a2c00 0x7f010c0a50c0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:34.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.595+0000 7f0128465640 1 --2- 172.21.15.16:0/4268866423 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f012010c170 0x7f012019cd00 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:34.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.595+0000 7f0128465640 1 --2- 172.21.15.16:0/4268866423 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0120103660 0x7f0120198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:34.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.595+0000 7f0128465640 1 --2- 172.21.15.16:0/4268866423 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0120102d40 0x7f0120198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:34.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.595+0000 7f0128465640 1 -- 172.21.15.16:0/4268866423 >> 172.21.15.16:0/4268866423 conn(0x7f01200fc7a0 msgr2=0x7f01200fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:34.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.595+0000 7f0128465640 1 -- 172.21.15.16:0/4268866423 shutdown_connections 2024-09-12T02:12:34.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:34.595+0000 7f0128465640 1 -- 172.21.15.16:0/4268866423 wait complete. 2024-09-12T02:12:35.370 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-12T02:12:35.636 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:12:36.080 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:12:36 smithi016 ceph-mon[30947]: pgmap v655: 97 pgs: 97 active+clean; 1.4 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-12T02:12:36.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.163+0000 7f6ccbb4d640 1 -- 172.21.15.16:0/101490610 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6cc4102790 msgr2=0x7f6cc41067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:36.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.163+0000 7f6ccbb4d640 1 --2- 172.21.15.16:0/101490610 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6cc4102790 0x7f6cc41067e0 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f6cb4009900 tx=0x7f6cb402a410 comp rx=0 tx=0).stop 2024-09-12T02:12:36.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.163+0000 7f6ccbb4d640 1 -- 172.21.15.16:0/101490610 shutdown_connections 2024-09-12T02:12:36.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.163+0000 7f6ccbb4d640 1 --2- 172.21.15.16:0/101490610 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6cc4106f10 0x7f6cc410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:36.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.163+0000 7f6ccbb4d640 1 --2- 172.21.15.16:0/101490610 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6cc4102790 0x7f6cc41067e0 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:36.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.163+0000 7f6ccbb4d640 1 --2- 172.21.15.16:0/101490610 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6cc4101de0 0x7f6cc41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:36.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.163+0000 7f6ccbb4d640 1 -- 172.21.15.16:0/101490610 >> 172.21.15.16:0/101490610 conn(0x7f6cc40fd8c0 msgr2=0x7f6cc40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:36.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.164+0000 7f6ccbb4d640 1 -- 172.21.15.16:0/101490610 shutdown_connections 2024-09-12T02:12:36.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.164+0000 7f6ccbb4d640 1 -- 172.21.15.16:0/101490610 wait complete. 2024-09-12T02:12:36.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.164+0000 7f6ccbb4d640 1 Processor -- start 2024-09-12T02:12:36.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.164+0000 7f6ccbb4d640 1 -- start start 2024-09-12T02:12:36.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.164+0000 7f6ccbb4d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6cc4101de0 0x7f6cc41983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:36.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.164+0000 7f6ccbb4d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6cc4102790 0x7f6cc41988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:36.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.164+0000 7f6ccbb4d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6cc4106f10 0x7f6cc419cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:36.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.164+0000 7f6ccbb4d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6cc4111f70 con 0x7f6cc4101de0 2024-09-12T02:12:36.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.164+0000 7f6ccbb4d640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6cc4111df0 con 0x7f6cc4102790 2024-09-12T02:12:36.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.164+0000 7f6ccbb4d640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f6cc41120f0 con 0x7f6cc4106f10 2024-09-12T02:12:36.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.165+0000 7f6cc98e9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6cc4101de0 0x7f6cc41983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:36.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.165+0000 7f6cc98e9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6cc4101de0 0x7f6cc41983b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36614/0 (socket says 172.21.15.16:36614) 2024-09-12T02:12:36.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.165+0000 7f6cc98e9640 1 -- 172.21.15.16:0/2401038634 learned_addr learned my addr 172.21.15.16:0/2401038634 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:12:36.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.165+0000 7f6cc90e8640 1 --2- 172.21.15.16:0/2401038634 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6cc4102790 0x7f6cc41988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:36.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.165+0000 7f6cca0ea640 1 --2- 172.21.15.16:0/2401038634 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6cc4106f10 0x7f6cc419cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:36.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.165+0000 7f6cc98e9640 1 -- 172.21.15.16:0/2401038634 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6cc4106f10 msgr2=0x7f6cc419cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:36.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.165+0000 7f6cc98e9640 1 --2- 172.21.15.16:0/2401038634 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6cc4106f10 0x7f6cc419cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:36.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.165+0000 7f6cc98e9640 1 -- 172.21.15.16:0/2401038634 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6cc4102790 msgr2=0x7f6cc41988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:36.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.165+0000 7f6cc98e9640 1 --2- 172.21.15.16:0/2401038634 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6cc4102790 0x7f6cc41988f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:36.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.165+0000 7f6cc98e9640 1 -- 172.21.15.16:0/2401038634 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cc419d360 con 0x7f6cc4101de0 2024-09-12T02:12:36.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.165+0000 7f6cca0ea640 1 --2- 172.21.15.16:0/2401038634 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6cc4106f10 0x7f6cc419cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:12:36.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.165+0000 7f6cc98e9640 1 --2- 172.21.15.16:0/2401038634 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6cc4101de0 0x7f6cc41983b0 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f6cc000c890 tx=0x7f6cc000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:36.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.165+0000 7f6cbaffd640 1 -- 172.21.15.16:0/2401038634 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cc0004220 con 0x7f6cc4101de0 2024-09-12T02:12:36.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.165+0000 7f6ccbb4d640 1 -- 172.21.15.16:0/2401038634 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cc419d650 con 0x7f6cc4101de0 2024-09-12T02:12:36.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.165+0000 7f6cbaffd640 1 -- 172.21.15.16:0/2401038634 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6cc00043c0 con 0x7f6cc4101de0 2024-09-12T02:12:36.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.165+0000 7f6cbaffd640 1 -- 172.21.15.16:0/2401038634 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cc00049e0 con 0x7f6cc4101de0 2024-09-12T02:12:36.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.165+0000 7f6ccbb4d640 1 -- 172.21.15.16:0/2401038634 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cc41a4f30 con 0x7f6cc4101de0 2024-09-12T02:12:36.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.167+0000 7f6cbaffd640 1 -- 172.21.15.16:0/2401038634 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6cc00079b0 con 0x7f6cc4101de0 2024-09-12T02:12:36.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.168+0000 7f6ccdf26740 1 Processor -- start 2024-09-12T02:12:36.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.168+0000 7f6ccdf26740 1 -- start start 2024-09-12T02:12:36.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.168+0000 7f6ccdf26740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556dda36fb40 0x556dda36e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:36.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.168+0000 7f6ccdf26740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556dda36e5d0 0x556dda36ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:36.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.168+0000 7f6ccdf26740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556dda37d780 0x556dda37fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:36.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.168+0000 7f6ccdf26740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x556dda2d42f0 con 0x556dda36fb40 2024-09-12T02:12:36.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.168+0000 7f6ccdf26740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x556dda27bcf0 con 0x556dda37d780 2024-09-12T02:12:36.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.168+0000 7f6ccdf26740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x556dda2d02b0 con 0x556dda36e5d0 2024-09-12T02:12:36.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.168+0000 7f6cbaffd640 1 --2- 172.21.15.16:0/2401038634 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f6cb00a2b80 0x7f6cb00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:36.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.169+0000 7f6cca0ea640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556dda36fb40 0x556dda36e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:36.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.169+0000 7f6cca0ea640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556dda36fb40 0x556dda36e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36622/0 (socket says 172.21.15.16:36622) 2024-09-12T02:12:36.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.169+0000 7f6cca0ea640 1 -- 172.21.15.16:0/1536817557 learned_addr learned my addr 172.21.15.16:0/1536817557 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:12:36.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.169+0000 7f6cca0ea640 1 --2- 172.21.15.16:0/2401038634 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f6cb00a2b80 0x7f6cb00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:36.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.169+0000 7f6cc90e8640 1 --2- 172.21.15.16:0/1536817557 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556dda37d780 0x556dda37fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:36.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.169+0000 7f6cc98e9640 1 --2- 172.21.15.16:0/1536817557 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556dda36e5d0 0x556dda36ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:36.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.169+0000 7f6cbaffd640 1 -- 172.21.15.16:0/2401038634 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f6cc00cf000 con 0x7f6cc4101de0 2024-09-12T02:12:36.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.169+0000 7f6cc90e8640 1 -- 172.21.15.16:0/1536817557 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556dda36e5d0 msgr2=0x556dda36ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:36.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.169+0000 7f6cc90e8640 1 --2- 172.21.15.16:0/1536817557 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556dda36e5d0 0x556dda36ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:36.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.169+0000 7f6cc90e8640 1 -- 172.21.15.16:0/1536817557 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556dda36fb40 msgr2=0x556dda36e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:36.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.169+0000 7f6cc90e8640 1 --2- 172.21.15.16:0/1536817557 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556dda36fb40 0x556dda36e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:36.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.169+0000 7f6cc90e8640 1 -- 172.21.15.16:0/1536817557 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556dda2603a0 con 0x556dda37d780 2024-09-12T02:12:36.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.169+0000 7f6cca0ea640 1 --2- 172.21.15.16:0/2401038634 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f6cb00a2b80 0x7f6cb00a5040 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f6cbc012b20 tx=0x7f6cbc00a940 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:36.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.169+0000 7f6cc90e8640 1 --2- 172.21.15.16:0/1536817557 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556dda37d780 0x556dda37fb70 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f6cb4005940 tx=0x7f6cb40316e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b3f5cc4eccb5633 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:36.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.169+0000 7f6caffff640 1 -- 172.21.15.16:0/1536817557 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cb4040070 con 0x556dda37d780 2024-09-12T02:12:36.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.170+0000 7f6caffff640 1 -- 172.21.15.16:0/1536817557 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6cb4005020 con 0x556dda37d780 2024-09-12T02:12:36.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.170+0000 7f6caffff640 1 -- 172.21.15.16:0/1536817557 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cb4035040 con 0x556dda37d780 2024-09-12T02:12:36.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.170+0000 7f6ccdf26740 1 -- 172.21.15.16:0/1536817557 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556dda37d780 msgr2=0x556dda37fb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:36.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.170+0000 7f6ccdf26740 1 --2- 172.21.15.16:0/1536817557 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556dda37d780 0x556dda37fb70 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f6cb4005940 tx=0x7f6cb40316e0 comp rx=0 tx=0).stop 2024-09-12T02:12:36.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.170+0000 7f6ccdf26740 1 -- 172.21.15.16:0/1536817557 shutdown_connections 2024-09-12T02:12:36.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.170+0000 7f6ccdf26740 1 --2- 172.21.15.16:0/1536817557 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556dda37d780 0x556dda37fb70 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:36.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.170+0000 7f6ccdf26740 1 --2- 172.21.15.16:0/1536817557 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556dda36e5d0 0x556dda36ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:36.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.170+0000 7f6ccdf26740 1 --2- 172.21.15.16:0/1536817557 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556dda36fb40 0x556dda36e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:36.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.170+0000 7f6ccdf26740 1 -- 172.21.15.16:0/1536817557 >> 172.21.15.16:0/1536817557 conn(0x556dda337310 msgr2=0x556dda36ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:36.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.170+0000 7f6ccdf26740 1 -- 172.21.15.16:0/1536817557 shutdown_connections 2024-09-12T02:12:36.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.170+0000 7f6ccdf26740 1 -- 172.21.15.16:0/1536817557 wait complete. 2024-09-12T02:12:36.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.171+0000 7f6ccdf26740 1 -- start start 2024-09-12T02:12:36.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.171+0000 7f6ccdf26740 1 Processor -- start 2024-09-12T02:12:36.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.171+0000 7f6ccdf26740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556dda36e5d0 0x556dda37d110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:36.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.171+0000 7f6ccdf26740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556dda36fb40 0x556dda37a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:36.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.171+0000 7f6ccdf26740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556dda37d780 0x556dda37a940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:36.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.171+0000 7f6ccdf26740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x556dda380420 con 0x556dda37d780 2024-09-12T02:12:36.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.171+0000 7f6ccdf26740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x556dda28f060 con 0x556dda36fb40 2024-09-12T02:12:36.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.171+0000 7f6ccdf26740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x556dda3805a0 con 0x556dda36e5d0 2024-09-12T02:12:36.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.171+0000 7f6ccdf26740 10 client.0 _ll_register_callbacks cb 0x556dd9fe28c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:12:36.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.171+0000 7f6cca0ea640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556dda37d780 0x556dda37a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:36.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.172+0000 7f6cca0ea640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556dda37d780 0x556dda37a940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36632/0 (socket says 172.21.15.16:36632) 2024-09-12T02:12:36.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.172+0000 7f6cca0ea640 1 -- 172.21.15.16:0/1623316441 learned_addr learned my addr 172.21.15.16:0/1623316441 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:12:36.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.172+0000 7f6cc98e9640 1 --2- 172.21.15.16:0/1623316441 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556dda36e5d0 0x556dda37d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:36.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.172+0000 7f6cc90e8640 1 --2- 172.21.15.16:0/1623316441 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556dda36fb40 0x556dda37a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:36.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.172+0000 7f6cca0ea640 1 -- 172.21.15.16:0/1623316441 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556dda36e5d0 msgr2=0x556dda37d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:36.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.172+0000 7f6cca0ea640 1 --2- 172.21.15.16:0/1623316441 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556dda36e5d0 0x556dda37d110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:36.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.172+0000 7f6cca0ea640 1 -- 172.21.15.16:0/1623316441 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556dda36fb40 msgr2=0x556dda37a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:36.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.172+0000 7f6cca0ea640 1 --2- 172.21.15.16:0/1623316441 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556dda36fb40 0x556dda37a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:36.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.172+0000 7f6cca0ea640 1 -- 172.21.15.16:0/1623316441 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556dda0453c0 con 0x556dda37d780 2024-09-12T02:12:36.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.172+0000 7f6cc90e8640 1 --2- 172.21.15.16:0/1623316441 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556dda36fb40 0x556dda37a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:12:36.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.172+0000 7f6cca0ea640 1 --2- 172.21.15.16:0/1623316441 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556dda37d780 0x556dda37a940 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f6cbc00afd0 tx=0x7f6cbc00a640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f201d98c66df927d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:36.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.172+0000 7f6cad7fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:12:36.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.172+0000 7f6cad7fa640 1 -- 172.21.15.16:0/1623316441 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cbc004b20 con 0x556dda37d780 2024-09-12T02:12:36.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.172+0000 7f6cad7fa640 1 -- 172.21.15.16:0/1623316441 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f6cbc004cc0 con 0x556dda37d780 2024-09-12T02:12:36.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.172+0000 7f6ccdf26740 1 -- 172.21.15.16:0/1623316441 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6cb4005020 con 0x556dda37d780 2024-09-12T02:12:36.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.172+0000 7f6cad7fa640 1 -- 172.21.15.16:0/1623316441 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6cbc01e600 con 0x556dda37d780 2024-09-12T02:12:36.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.172+0000 7f6ccdf26740 10 client.18741 fetch_fsmap learned FSMap version 25 2024-09-12T02:12:36.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.172+0000 7f6ccdf26740 1 -- 172.21.15.16:0/1623316441 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x556dda38be50 con 0x556dda37d780 2024-09-12T02:12:36.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.173+0000 7f6cad7fa640 1 -- 172.21.15.16:0/1623316441 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6cb4005020 con 0x556dda37d780 2024-09-12T02:12:36.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.173+0000 7f6cad7fa640 1 -- 172.21.15.16:0/1623316441 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f6cbc005380 con 0x556dda37d780 2024-09-12T02:12:36.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.173+0000 7f6ccdf26740 10 client.18741 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:12:36.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.173+0000 7f6ccdf26740 10 client.18741 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T02:12:36.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.173+0000 7f6ccdf26740 20 client.18741 populate_metadata read hostname 'smithi016' 2024-09-12T02:12:36.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.173+0000 7f6ccdf26740 1 --2- 172.21.15.16:0/1623316441 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x556dda38d310 0x556dda38f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:36.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.173+0000 7f6ccdf26740 4 client.18741 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:12:36.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.173+0000 7f6ccdf26740 1 -- 172.21.15.16:0/1623316441 --> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556dda2e1860 con 0x556dda38d310 2024-09-12T02:12:36.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.173+0000 7f6cc98e9640 1 --2- 172.21.15.16:0/1623316441 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x556dda38d310 0x556dda38f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:36.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.174+0000 7f6cad7fa640 10 client.18741 ms_handle_connect on v2:172.21.15.184:6838/4203347941 2024-09-12T02:12:36.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.174+0000 7f6cc98e9640 1 --2- 172.21.15.16:0/1623316441 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x556dda38d310 0x556dda38f780 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=616b1fb4fa3233d8 server_cookie=b240549f71e8d8a3 in_seq=0 out_seq=0 2024-09-12T02:12:36.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.175+0000 7f6cad7fa640 1 -- 172.21.15.16:0/1623316441 <== mds.1 v2:172.21.15.184:6838/4203347941 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556dda2e1860 con 0x556dda38d310 2024-09-12T02:12:36.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.175+0000 7f6cad7fa640 10 client.18741 handle_command_reply: tid=0 2024-09-12T02:12:36.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.175+0000 7f6ccdf26740 1 client.18741 shutdown 2024-09-12T02:12:36.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.175+0000 7f6ccdf26740 1 -- 172.21.15.16:0/1623316441 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556dda37d780 msgr2=0x556dda37a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:36.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.175+0000 7f6ccdf26740 1 --2- 172.21.15.16:0/1623316441 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556dda37d780 0x556dda37a940 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f6cbc00afd0 tx=0x7f6cbc00a640 comp rx=0 tx=0).stop 2024-09-12T02:12:36.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.175+0000 7f6ccdf26740 1 -- 172.21.15.16:0/1623316441 shutdown_connections 2024-09-12T02:12:36.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.175+0000 7f6ccdf26740 1 --2- 172.21.15.16:0/1623316441 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x556dda38d310 0x556dda38f780 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:36.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.175+0000 7f6ccdf26740 1 --2- 172.21.15.16:0/1623316441 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556dda37d780 0x556dda37a940 unknown :-1 s=CLOSED pgs=1477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:36.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.175+0000 7f6ccdf26740 1 --2- 172.21.15.16:0/1623316441 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556dda36fb40 0x556dda37a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:36.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.175+0000 7f6ccdf26740 1 --2- 172.21.15.16:0/1623316441 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556dda36e5d0 0x556dda37d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:36.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.175+0000 7f6ccdf26740 1 -- 172.21.15.16:0/1623316441 >> 172.21.15.16:0/1623316441 conn(0x556dda337310 msgr2=0x556dda370960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:36.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.175+0000 7f6cad7fa640 0 client.18741 ms_handle_reset on v2:172.21.15.184:6838/4203347941 2024-09-12T02:12:36.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.175+0000 7f6ccdf26740 1 -- 172.21.15.16:0/1623316441 shutdown_connections 2024-09-12T02:12:36.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.175+0000 7f6ccdf26740 1 -- 172.21.15.16:0/1623316441 wait complete. 2024-09-12T02:12:36.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.175+0000 7f6ccdf26740 20 client.18741 trim_cache size 0 max 16384 2024-09-12T02:12:36.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.188+0000 7f6ccdf26740 1 Processor -- start 2024-09-12T02:12:36.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.188+0000 7f6ccdf26740 1 -- start start 2024-09-12T02:12:36.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.188+0000 7f6ccdf26740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556dda36e5d0 0x556dda36ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:36.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.188+0000 7f6ccdf26740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556dda36fb40 0x556dda3896c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:36.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.188+0000 7f6ccdf26740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556dda37d780 0x556dda379c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:36.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.188+0000 7f6ccdf26740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6cc41ada60 con 0x556dda36fb40 2024-09-12T02:12:36.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.188+0000 7f6ccdf26740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6cc41ab330 con 0x556dda37d780 2024-09-12T02:12:36.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.188+0000 7f6ccdf26740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f6cc41a9180 con 0x556dda36e5d0 2024-09-12T02:12:36.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.189+0000 7f6cc90e8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556dda36fb40 0x556dda3896c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:36.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.189+0000 7f6cc90e8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556dda36fb40 0x556dda3896c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36644/0 (socket says 172.21.15.16:36644) 2024-09-12T02:12:36.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.189+0000 7f6cc90e8640 1 -- 172.21.15.16:0/3491490045 learned_addr learned my addr 172.21.15.16:0/3491490045 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:12:36.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.189+0000 7f6cca0ea640 1 --2- 172.21.15.16:0/3491490045 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556dda37d780 0x556dda379c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:36.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.189+0000 7f6cc98e9640 1 --2- 172.21.15.16:0/3491490045 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556dda36e5d0 0x556dda36ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:36.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.189+0000 7f6cc90e8640 1 -- 172.21.15.16:0/3491490045 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556dda36e5d0 msgr2=0x556dda36ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:36.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.189+0000 7f6cc90e8640 1 --2- 172.21.15.16:0/3491490045 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556dda36e5d0 0x556dda36ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:36.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.189+0000 7f6cc90e8640 1 -- 172.21.15.16:0/3491490045 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556dda37d780 msgr2=0x556dda379c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:36.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.189+0000 7f6cc90e8640 1 --2- 172.21.15.16:0/3491490045 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556dda37d780 0x556dda379c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:36.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.189+0000 7f6cc90e8640 1 -- 172.21.15.16:0/3491490045 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556dda379370 con 0x556dda36fb40 2024-09-12T02:12:36.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.189+0000 7f6cca0ea640 1 --2- 172.21.15.16:0/3491490045 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556dda37d780 0x556dda379c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:12:36.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.189+0000 7f6cc90e8640 1 --2- 172.21.15.16:0/3491490045 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556dda36fb40 0x556dda3896c0 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f6cb4005940 tx=0x7f6cb4000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e7bbab4204fbd31 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:36.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.189+0000 7f6caf7fe640 1 -- 172.21.15.16:0/3491490045 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cb4035050 con 0x556dda36fb40 2024-09-12T02:12:36.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.189+0000 7f6caf7fe640 1 -- 172.21.15.16:0/3491490045 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6cb4031890 con 0x556dda36fb40 2024-09-12T02:12:36.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.189+0000 7f6caf7fe640 1 -- 172.21.15.16:0/3491490045 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cb4040040 con 0x556dda36fb40 2024-09-12T02:12:36.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.190+0000 7f6ccdf26740 1 -- 172.21.15.16:0/3491490045 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556dda36fb40 msgr2=0x556dda3896c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:36.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.190+0000 7f6ccdf26740 1 --2- 172.21.15.16:0/3491490045 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556dda36fb40 0x556dda3896c0 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f6cb4005940 tx=0x7f6cb4000bc0 comp rx=0 tx=0).stop 2024-09-12T02:12:36.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.190+0000 7f6ccdf26740 1 -- 172.21.15.16:0/3491490045 shutdown_connections 2024-09-12T02:12:36.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.190+0000 7f6ccdf26740 1 --2- 172.21.15.16:0/3491490045 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556dda37d780 0x556dda379c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:36.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.190+0000 7f6ccdf26740 1 --2- 172.21.15.16:0/3491490045 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556dda36fb40 0x556dda3896c0 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:36.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.190+0000 7f6ccdf26740 1 --2- 172.21.15.16:0/3491490045 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556dda36e5d0 0x556dda36ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:36.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.190+0000 7f6ccdf26740 1 -- 172.21.15.16:0/3491490045 >> 172.21.15.16:0/3491490045 conn(0x556dda337310 msgr2=0x556dda36a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:36.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.190+0000 7f6ccdf26740 1 -- 172.21.15.16:0/3491490045 shutdown_connections 2024-09-12T02:12:36.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.190+0000 7f6ccdf26740 1 -- 172.21.15.16:0/3491490045 wait complete. 2024-09-12T02:12:36.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.190+0000 7f6ccdf26740 1 -- start start 2024-09-12T02:12:36.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.190+0000 7f6ccdf26740 1 Processor -- start 2024-09-12T02:12:36.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.191+0000 7f6ccdf26740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556dda36e5d0 0x556dda385f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:36.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.191+0000 7f6ccdf26740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556dda36fb40 0x556dda381290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:36.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.191+0000 7f6ccdf26740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556dda37d780 0x556dda3817d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:36.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.191+0000 7f6ccdf26740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6cc41a97f0 con 0x556dda36fb40 2024-09-12T02:12:36.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.191+0000 7f6ccdf26740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6cc41aac60 con 0x556dda37d780 2024-09-12T02:12:36.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.191+0000 7f6ccdf26740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f6cc41a9ec0 con 0x556dda36e5d0 2024-09-12T02:12:36.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.191+0000 7f6ccdf26740 10 client.0 _ll_register_callbacks cb 0x7f6cbc00a560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:12:36.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.191+0000 7f6cc90e8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556dda36fb40 0x556dda381290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:36.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.191+0000 7f6cc90e8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556dda36fb40 0x556dda381290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36648/0 (socket says 172.21.15.16:36648) 2024-09-12T02:12:36.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.191+0000 7f6cc90e8640 1 -- 172.21.15.16:0/2370261040 learned_addr learned my addr 172.21.15.16:0/2370261040 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:12:36.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.191+0000 7f6cc98e9640 1 --2- 172.21.15.16:0/2370261040 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556dda36e5d0 0x556dda385f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:36.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.191+0000 7f6cca0ea640 1 --2- 172.21.15.16:0/2370261040 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556dda37d780 0x556dda3817d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:36.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.191+0000 7f6cc90e8640 1 -- 172.21.15.16:0/2370261040 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556dda36e5d0 msgr2=0x556dda385f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:36.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.191+0000 7f6cc90e8640 1 --2- 172.21.15.16:0/2370261040 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556dda36e5d0 0x556dda385f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:36.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.191+0000 7f6cc90e8640 1 -- 172.21.15.16:0/2370261040 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556dda37d780 msgr2=0x556dda3817d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:36.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.191+0000 7f6cc90e8640 1 --2- 172.21.15.16:0/2370261040 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556dda37d780 0x556dda3817d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:36.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.191+0000 7f6cc90e8640 1 -- 172.21.15.16:0/2370261040 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556dda382000 con 0x556dda36fb40 2024-09-12T02:12:36.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.191+0000 7f6cc90e8640 1 --2- 172.21.15.16:0/2370261040 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556dda36fb40 0x556dda381290 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f6cb4050040 tx=0x7f6cb402a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58a067b43370b92f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:36.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.191+0000 7f6cad7fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:12:36.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.192+0000 7f6cad7fa640 1 -- 172.21.15.16:0/2370261040 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cb4040040 con 0x556dda36fb40 2024-09-12T02:12:36.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.192+0000 7f6cad7fa640 1 -- 172.21.15.16:0/2370261040 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f6cb404a040 con 0x556dda36fb40 2024-09-12T02:12:36.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.192+0000 7f6ccdf26740 1 -- 172.21.15.16:0/2370261040 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6cb4031890 con 0x556dda36fb40 2024-09-12T02:12:36.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.192+0000 7f6cad7fa640 1 -- 172.21.15.16:0/2370261040 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6cb4035030 con 0x556dda36fb40 2024-09-12T02:12:36.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.192+0000 7f6ccdf26740 10 client.18747 fetch_fsmap learned FSMap version 25 2024-09-12T02:12:36.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.192+0000 7f6ccdf26740 1 -- 172.21.15.16:0/2370261040 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x556dda3f8010 con 0x556dda36fb40 2024-09-12T02:12:36.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.192+0000 7f6cad7fa640 1 -- 172.21.15.16:0/2370261040 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6cb4031890 con 0x556dda36fb40 2024-09-12T02:12:36.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.192+0000 7f6cad7fa640 1 -- 172.21.15.16:0/2370261040 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f6cb4003680 con 0x556dda36fb40 2024-09-12T02:12:36.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.192+0000 7f6ccdf26740 10 client.18747 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:12:36.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.192+0000 7f6ccdf26740 10 client.18747 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T02:12:36.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.192+0000 7f6ccdf26740 20 client.18747 populate_metadata read hostname 'smithi016' 2024-09-12T02:12:36.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.192+0000 7f6ccdf26740 1 --2- 172.21.15.16:0/2370261040 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x556dda3f82e0 0x556dda3fa6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:36.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.192+0000 7f6ccdf26740 4 client.18747 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T02:12:36.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.192+0000 7f6ccdf26740 1 -- 172.21.15.16:0/2370261040 --> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556dda2dbd70 con 0x556dda3f82e0 2024-09-12T02:12:36.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.192+0000 7f6cc98e9640 1 --2- 172.21.15.16:0/2370261040 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x556dda3f82e0 0x556dda3fa6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:36.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.193+0000 7f6cc98e9640 1 --2- 172.21.15.16:0/2370261040 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x556dda3f82e0 0x556dda3fa6d0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=73c3122648b248c server_cookie=e8f620b9c9e82337 in_seq=0 out_seq=0 2024-09-12T02:12:36.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.193+0000 7f6cad7fa640 10 client.18747 ms_handle_connect on v2:172.21.15.184:6838/4203347941 2024-09-12T02:12:36.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.193+0000 7f6cad7fa640 1 -- 172.21.15.16:0/2370261040 <== mds.1 v2:172.21.15.184:6838/4203347941 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556dda2dbd70 con 0x556dda3f82e0 2024-09-12T02:12:36.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.193+0000 7f6cad7fa640 10 client.18747 handle_command_reply: tid=0 2024-09-12T02:12:36.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.193+0000 7f6ccdf26740 1 client.18747 shutdown 2024-09-12T02:12:36.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.193+0000 7f6ccdf26740 1 -- 172.21.15.16:0/2370261040 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556dda36fb40 msgr2=0x556dda381290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:36.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.193+0000 7f6ccdf26740 1 --2- 172.21.15.16:0/2370261040 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556dda36fb40 0x556dda381290 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f6cb4050040 tx=0x7f6cb402a410 comp rx=0 tx=0).stop 2024-09-12T02:12:36.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.193+0000 7f6ccdf26740 1 -- 172.21.15.16:0/2370261040 shutdown_connections 2024-09-12T02:12:36.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.193+0000 7f6ccdf26740 1 --2- 172.21.15.16:0/2370261040 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x556dda3f82e0 0x556dda3fa6d0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:36.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.193+0000 7f6ccdf26740 1 --2- 172.21.15.16:0/2370261040 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556dda37d780 0x556dda3817d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:36.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.193+0000 7f6ccdf26740 1 --2- 172.21.15.16:0/2370261040 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556dda36fb40 0x556dda381290 unknown :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:36.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.193+0000 7f6cad7fa640 0 client.18747 ms_handle_reset on v2:172.21.15.184:6838/4203347941 2024-09-12T02:12:36.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.193+0000 7f6ccdf26740 1 --2- 172.21.15.16:0/2370261040 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556dda36e5d0 0x556dda385f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:36.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.193+0000 7f6ccdf26740 1 -- 172.21.15.16:0/2370261040 >> 172.21.15.16:0/2370261040 conn(0x556dda337310 msgr2=0x556dda36a610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:36.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.193+0000 7f6ccdf26740 1 -- 172.21.15.16:0/2370261040 shutdown_connections 2024-09-12T02:12:36.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.194+0000 7f6ccdf26740 1 -- 172.21.15.16:0/2370261040 wait complete. 2024-09-12T02:12:36.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.194+0000 7f6ccdf26740 20 client.18747 trim_cache size 0 max 16384 2024-09-12T02:12:36.199 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T02:12:36.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.194+0000 7f6ccbb4d640 1 -- 172.21.15.16:0/2401038634 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f6cb00a2b80 msgr2=0x7f6cb00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:36.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.194+0000 7f6ccbb4d640 1 --2- 172.21.15.16:0/2401038634 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f6cb00a2b80 0x7f6cb00a5040 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f6cbc012b20 tx=0x7f6cbc00a940 comp rx=0 tx=0).stop 2024-09-12T02:12:36.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.194+0000 7f6ccbb4d640 1 -- 172.21.15.16:0/2401038634 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6cc4101de0 msgr2=0x7f6cc41983b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:36.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.194+0000 7f6ccbb4d640 1 --2- 172.21.15.16:0/2401038634 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6cc4101de0 0x7f6cc41983b0 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f6cc000c890 tx=0x7f6cc000cd50 comp rx=0 tx=0).stop 2024-09-12T02:12:36.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.194+0000 7f6ccbb4d640 1 -- 172.21.15.16:0/2401038634 shutdown_connections 2024-09-12T02:12:36.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.194+0000 7f6ccbb4d640 1 --2- 172.21.15.16:0/2401038634 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f6cb00a2b80 0x7f6cb00a5040 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:36.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.194+0000 7f6ccbb4d640 1 --2- 172.21.15.16:0/2401038634 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6cc4106f10 0x7f6cc419cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:36.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.194+0000 7f6ccbb4d640 1 --2- 172.21.15.16:0/2401038634 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6cc4102790 0x7f6cc41988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:36.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.194+0000 7f6ccbb4d640 1 --2- 172.21.15.16:0/2401038634 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6cc4101de0 0x7f6cc41983b0 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:36.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.194+0000 7f6ccbb4d640 1 -- 172.21.15.16:0/2401038634 >> 172.21.15.16:0/2401038634 conn(0x7f6cc40fd8c0 msgr2=0x7f6cc40fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:36.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.194+0000 7f6ccbb4d640 1 -- 172.21.15.16:0/2401038634 shutdown_connections 2024-09-12T02:12:36.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:36.194+0000 7f6ccbb4d640 1 -- 172.21.15.16:0/2401038634 wait complete. 2024-09-12T02:12:36.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:12:36 smithi184 ceph-mon[35593]: pgmap v655: 97 pgs: 97 active+clean; 1.4 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-12T02:12:36.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:12:36 smithi033 ceph-mon[35332]: pgmap v655: 97 pgs: 97 active+clean; 1.4 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-12T02:12:36.887 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-12T02:12:37.149 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:12:37.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.664+0000 7fb839766640 1 -- 172.21.15.16:0/2377263376 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb834106f10 msgr2=0x7fb83410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:37.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.664+0000 7fb839766640 1 --2- 172.21.15.16:0/2377263376 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb834106f10 0x7fb83410ea90 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7fb82c009ca0 tx=0x7fb82c02a8c0 comp rx=0 tx=0).stop 2024-09-12T02:12:37.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.664+0000 7fb839766640 1 -- 172.21.15.16:0/2377263376 shutdown_connections 2024-09-12T02:12:37.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.664+0000 7fb839766640 1 --2- 172.21.15.16:0/2377263376 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb834106f10 0x7fb83410ea90 unknown :-1 s=CLOSED pgs=1480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:37.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.664+0000 7fb839766640 1 --2- 172.21.15.16:0/2377263376 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb834102790 0x7fb8341067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:37.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.664+0000 7fb839766640 1 --2- 172.21.15.16:0/2377263376 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb834101de0 0x7fb8341021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:37.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.664+0000 7fb839766640 1 -- 172.21.15.16:0/2377263376 >> 172.21.15.16:0/2377263376 conn(0x7fb8340fd8c0 msgr2=0x7fb8340ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:37.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.664+0000 7fb839766640 1 -- 172.21.15.16:0/2377263376 shutdown_connections 2024-09-12T02:12:37.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.664+0000 7fb839766640 1 -- 172.21.15.16:0/2377263376 wait complete. 2024-09-12T02:12:37.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.665+0000 7fb839766640 1 Processor -- start 2024-09-12T02:12:37.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.665+0000 7fb839766640 1 -- start start 2024-09-12T02:12:37.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.665+0000 7fb839766640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb834101de0 0x7fb8341983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:37.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.666+0000 7fb839766640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb834102790 0x7fb834198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:37.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.666+0000 7fb839766640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb834106f10 0x7fb83419ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:37.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.666+0000 7fb839766640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb834111f70 con 0x7fb834101de0 2024-09-12T02:12:37.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.666+0000 7fb839766640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb834111df0 con 0x7fb834106f10 2024-09-12T02:12:37.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.666+0000 7fb839766640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb8341120f0 con 0x7fb834102790 2024-09-12T02:12:37.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.666+0000 7fb832ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb834101de0 0x7fb8341983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:37.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.666+0000 7fb832ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb834101de0 0x7fb8341983f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36660/0 (socket says 172.21.15.16:36660) 2024-09-12T02:12:37.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.666+0000 7fb832ffd640 1 -- 172.21.15.16:0/3979875350 learned_addr learned my addr 172.21.15.16:0/3979875350 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:12:37.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.666+0000 7fb82bfff640 1 --2- 172.21.15.16:0/3979875350 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb834102790 0x7fb834198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:37.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.666+0000 7fb8337fe640 1 --2- 172.21.15.16:0/3979875350 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb834106f10 0x7fb83419ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:37.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.666+0000 7fb832ffd640 1 -- 172.21.15.16:0/3979875350 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb834102790 msgr2=0x7fb834198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:37.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.666+0000 7fb832ffd640 1 --2- 172.21.15.16:0/3979875350 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb834102790 0x7fb834198930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:37.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.666+0000 7fb832ffd640 1 -- 172.21.15.16:0/3979875350 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb834106f10 msgr2=0x7fb83419ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:37.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.666+0000 7fb832ffd640 1 --2- 172.21.15.16:0/3979875350 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb834106f10 0x7fb83419ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:37.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.666+0000 7fb832ffd640 1 -- 172.21.15.16:0/3979875350 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb83419d3a0 con 0x7fb834101de0 2024-09-12T02:12:37.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.666+0000 7fb82bfff640 1 --2- 172.21.15.16:0/3979875350 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb834102790 0x7fb834198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:12:37.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.666+0000 7fb832ffd640 1 --2- 172.21.15.16:0/3979875350 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb834101de0 0x7fb8341983f0 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7fb82400c970 tx=0x7fb82400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:37.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.666+0000 7fb830ff9640 1 -- 172.21.15.16:0/3979875350 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb824007bb0 con 0x7fb834101de0 2024-09-12T02:12:37.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.666+0000 7fb830ff9640 1 -- 172.21.15.16:0/3979875350 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb824007d50 con 0x7fb834101de0 2024-09-12T02:12:37.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.666+0000 7fb839766640 1 -- 172.21.15.16:0/3979875350 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb83419d690 con 0x7fb834101de0 2024-09-12T02:12:37.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.666+0000 7fb830ff9640 1 -- 172.21.15.16:0/3979875350 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb82400f450 con 0x7fb834101de0 2024-09-12T02:12:37.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.667+0000 7fb839766640 1 -- 172.21.15.16:0/3979875350 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8341a4f70 con 0x7fb834101de0 2024-09-12T02:12:37.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.669+0000 7fb830ff9640 1 -- 172.21.15.16:0/3979875350 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fb824005020 con 0x7fb834101de0 2024-09-12T02:12:37.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.669+0000 7fb83bb3f740 1 Processor -- start 2024-09-12T02:12:37.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.669+0000 7fb83bb3f740 1 -- start start 2024-09-12T02:12:37.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.670+0000 7fb83bb3f740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b3a29ceb40 0x55b3a29cd090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:37.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.670+0000 7fb83bb3f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3a29cd5d0 0x55b3a29cda10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:37.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.670+0000 7fb83bb3f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b3a29d8370 0x55b3a29da760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:37.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.670+0000 7fb83bb3f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55b3a29332f0 con 0x55b3a29cd5d0 2024-09-12T02:12:37.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.670+0000 7fb83bb3f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55b3a28dacf0 con 0x55b3a29d8370 2024-09-12T02:12:37.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.670+0000 7fb83bb3f740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55b3a292f2b0 con 0x55b3a29ceb40 2024-09-12T02:12:37.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.670+0000 7fb832ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3a29cd5d0 0x55b3a29cda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:37.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.670+0000 7fb830ff9640 1 --2- 172.21.15.16:0/3979875350 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fb8200a2b80 0x7fb8200a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:37.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.670+0000 7fb832ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3a29cd5d0 0x55b3a29cda10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36668/0 (socket says 172.21.15.16:36668) 2024-09-12T02:12:37.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.670+0000 7fb832ffd640 1 -- 172.21.15.16:0/2012772114 learned_addr learned my addr 172.21.15.16:0/2012772114 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:12:37.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.670+0000 7fb8337fe640 1 --2- 172.21.15.16:0/2012772114 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b3a29ceb40 0x55b3a29cd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:37.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.670+0000 7fb82bfff640 1 --2- 172.21.15.16:0/2012772114 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b3a29d8370 0x55b3a29da760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:37.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.670+0000 7fb8337fe640 1 --2- 172.21.15.16:0/3979875350 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fb8200a2b80 0x7fb8200a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:37.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.670+0000 7fb830ff9640 1 -- 172.21.15.16:0/3979875350 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fb8240cee30 con 0x7fb834101de0 2024-09-12T02:12:37.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.670+0000 7fb832ffd640 1 -- 172.21.15.16:0/2012772114 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b3a29ceb40 msgr2=0x55b3a29cd090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:37.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.670+0000 7fb832ffd640 1 --2- 172.21.15.16:0/2012772114 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b3a29ceb40 0x55b3a29cd090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:37.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.670+0000 7fb832ffd640 1 -- 172.21.15.16:0/2012772114 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b3a29d8370 msgr2=0x55b3a29da760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:37.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.670+0000 7fb832ffd640 1 --2- 172.21.15.16:0/2012772114 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b3a29d8370 0x55b3a29da760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:37.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.670+0000 7fb832ffd640 1 -- 172.21.15.16:0/2012772114 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b3a28bf3a0 con 0x55b3a29cd5d0 2024-09-12T02:12:37.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.670+0000 7fb832ffd640 1 --2- 172.21.15.16:0/2012772114 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3a29cd5d0 0x55b3a29cda10 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7fb8240cc910 tx=0x7fb824004a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51f265b33a1d9b0b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:37.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.671+0000 7fb8337fe640 1 --2- 172.21.15.16:0/3979875350 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fb8200a2b80 0x7fb8200a5040 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fb82c031f80 tx=0x7fb82c0095c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:37.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.671+0000 7fb8297fa640 1 -- 172.21.15.16:0/2012772114 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb82401d070 con 0x55b3a29cd5d0 2024-09-12T02:12:37.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.671+0000 7fb8297fa640 1 -- 172.21.15.16:0/2012772114 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb82409b730 con 0x55b3a29cd5d0 2024-09-12T02:12:37.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.671+0000 7fb8297fa640 1 -- 172.21.15.16:0/2012772114 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb824099860 con 0x55b3a29cd5d0 2024-09-12T02:12:37.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.671+0000 7fb83bb3f740 1 -- 172.21.15.16:0/2012772114 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3a29cd5d0 msgr2=0x55b3a29cda10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:37.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.671+0000 7fb83bb3f740 1 --2- 172.21.15.16:0/2012772114 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3a29cd5d0 0x55b3a29cda10 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7fb8240cc910 tx=0x7fb824004a20 comp rx=0 tx=0).stop 2024-09-12T02:12:37.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.672+0000 7fb83bb3f740 1 -- 172.21.15.16:0/2012772114 shutdown_connections 2024-09-12T02:12:37.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.672+0000 7fb83bb3f740 1 --2- 172.21.15.16:0/2012772114 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b3a29d8370 0x55b3a29da760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:37.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.672+0000 7fb83bb3f740 1 --2- 172.21.15.16:0/2012772114 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3a29cd5d0 0x55b3a29cda10 unknown :-1 s=CLOSED pgs=1482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:37.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.672+0000 7fb83bb3f740 1 --2- 172.21.15.16:0/2012772114 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b3a29ceb40 0x55b3a29cd090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:37.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.672+0000 7fb83bb3f740 1 -- 172.21.15.16:0/2012772114 >> 172.21.15.16:0/2012772114 conn(0x55b3a2996310 msgr2=0x55b3a29c9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:37.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.672+0000 7fb83bb3f740 1 -- 172.21.15.16:0/2012772114 shutdown_connections 2024-09-12T02:12:37.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.672+0000 7fb83bb3f740 1 -- 172.21.15.16:0/2012772114 wait complete. 2024-09-12T02:12:37.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.672+0000 7fb83bb3f740 1 -- start start 2024-09-12T02:12:37.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.672+0000 7fb83bb3f740 1 Processor -- start 2024-09-12T02:12:37.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.672+0000 7fb83bb3f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3a29cd5d0 0x55b3a29d7da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:37.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.672+0000 7fb83bb3f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b3a29ceb40 0x55b3a29d5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:37.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.672+0000 7fb83bb3f740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b3a29d8370 0x55b3a29d57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:37.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.672+0000 7fb83bb3f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55b3a29df3c0 con 0x55b3a29cd5d0 2024-09-12T02:12:37.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.672+0000 7fb83bb3f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55b3a28ee060 con 0x55b3a29ceb40 2024-09-12T02:12:37.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.672+0000 7fb83bb3f740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55b3a29df540 con 0x55b3a29d8370 2024-09-12T02:12:37.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.672+0000 7fb83bb3f740 10 client.0 _ll_register_callbacks cb 0x55b3a26418c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:12:37.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.672+0000 7fb832ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3a29cd5d0 0x55b3a29d7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:37.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.672+0000 7fb832ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3a29cd5d0 0x55b3a29d7da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36674/0 (socket says 172.21.15.16:36674) 2024-09-12T02:12:37.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.673+0000 7fb832ffd640 1 -- 172.21.15.16:0/4266322163 learned_addr learned my addr 172.21.15.16:0/4266322163 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:12:37.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.673+0000 7fb8337fe640 1 --2- 172.21.15.16:0/4266322163 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b3a29d8370 0x55b3a29d57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:37.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.673+0000 7fb82bfff640 1 --2- 172.21.15.16:0/4266322163 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b3a29ceb40 0x55b3a29d5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:37.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.673+0000 7fb832ffd640 1 -- 172.21.15.16:0/4266322163 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b3a29d8370 msgr2=0x55b3a29d57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:37.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.673+0000 7fb832ffd640 1 --2- 172.21.15.16:0/4266322163 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b3a29d8370 0x55b3a29d57e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:37.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.673+0000 7fb832ffd640 1 -- 172.21.15.16:0/4266322163 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b3a29ceb40 msgr2=0x55b3a29d5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:37.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.673+0000 7fb832ffd640 1 --2- 172.21.15.16:0/4266322163 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b3a29ceb40 0x55b3a29d5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:37.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.673+0000 7fb832ffd640 1 -- 172.21.15.16:0/4266322163 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b3a26a43c0 con 0x55b3a29cd5d0 2024-09-12T02:12:37.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.673+0000 7fb832ffd640 1 --2- 172.21.15.16:0/4266322163 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3a29cd5d0 0x55b3a29d7da0 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7fb824007590 tx=0x7fb824097a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=feccd52078491464 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:37.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.673+0000 7fb81a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:12:37.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.673+0000 7fb81a7fc640 1 -- 172.21.15.16:0/4266322163 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8240b3020 con 0x55b3a29cd5d0 2024-09-12T02:12:37.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.673+0000 7fb81a7fc640 1 -- 172.21.15.16:0/4266322163 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fb824097da0 con 0x55b3a29cd5d0 2024-09-12T02:12:37.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.673+0000 7fb83bb3f740 1 -- 172.21.15.16:0/4266322163 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb82409b730 con 0x55b3a29cd5d0 2024-09-12T02:12:37.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.673+0000 7fb81a7fc640 1 -- 172.21.15.16:0/4266322163 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fb82401d040 con 0x55b3a29cd5d0 2024-09-12T02:12:37.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.673+0000 7fb83bb3f740 10 client.18759 fetch_fsmap learned FSMap version 25 2024-09-12T02:12:37.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.673+0000 7fb83bb3f740 1 -- 172.21.15.16:0/4266322163 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b3a29eaed0 con 0x55b3a29cd5d0 2024-09-12T02:12:37.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.674+0000 7fb81a7fc640 1 -- 172.21.15.16:0/4266322163 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb82409b730 con 0x55b3a29cd5d0 2024-09-12T02:12:37.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.674+0000 7fb81a7fc640 1 -- 172.21.15.16:0/4266322163 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fb82409ab80 con 0x55b3a29cd5d0 2024-09-12T02:12:37.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.674+0000 7fb83bb3f740 10 client.18759 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:12:37.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.674+0000 7fb83bb3f740 10 client.18759 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:12:37.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.674+0000 7fb83bb3f740 20 client.18759 populate_metadata read hostname 'smithi016' 2024-09-12T02:12:37.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.674+0000 7fb83bb3f740 1 --2- 172.21.15.16:0/4266322163 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55b3a29ec220 0x55b3a29ee610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:37.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.674+0000 7fb83bb3f740 4 client.18759 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:12:37.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.674+0000 7fb83bb3f740 1 -- 172.21.15.16:0/4266322163 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b3a2940860 con 0x55b3a29ec220 2024-09-12T02:12:37.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.674+0000 7fb82bfff640 1 --2- 172.21.15.16:0/4266322163 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55b3a29ec220 0x55b3a29ee610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:37.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.675+0000 7fb82bfff640 1 --2- 172.21.15.16:0/4266322163 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55b3a29ec220 0x55b3a29ee610 crc :-1 s=READY pgs=312 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3900b0d454b098b2 server_cookie=50df05520031d941 in_seq=0 out_seq=0 2024-09-12T02:12:37.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.675+0000 7fb81a7fc640 10 client.18759 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:12:37.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.675+0000 7fb81a7fc640 1 -- 172.21.15.16:0/4266322163 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b3a2940860 con 0x55b3a29ec220 2024-09-12T02:12:37.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.675+0000 7fb81a7fc640 10 client.18759 handle_command_reply: tid=0 2024-09-12T02:12:37.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.675+0000 7fb83bb3f740 1 client.18759 shutdown 2024-09-12T02:12:37.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.676+0000 7fb83bb3f740 1 -- 172.21.15.16:0/4266322163 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3a29cd5d0 msgr2=0x55b3a29d7da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:37.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.676+0000 7fb83bb3f740 1 --2- 172.21.15.16:0/4266322163 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3a29cd5d0 0x55b3a29d7da0 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7fb824007590 tx=0x7fb824097a50 comp rx=0 tx=0).stop 2024-09-12T02:12:37.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.676+0000 7fb83bb3f740 1 -- 172.21.15.16:0/4266322163 shutdown_connections 2024-09-12T02:12:37.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.676+0000 7fb83bb3f740 1 --2- 172.21.15.16:0/4266322163 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55b3a29ec220 0x55b3a29ee610 crc :-1 s=READY pgs=312 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:37.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.676+0000 7fb83bb3f740 1 --2- 172.21.15.16:0/4266322163 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b3a29d8370 0x55b3a29d57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:37.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.676+0000 7fb83bb3f740 1 --2- 172.21.15.16:0/4266322163 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b3a29ceb40 0x55b3a29d5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:37.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.676+0000 7fb81a7fc640 0 client.18759 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:12:37.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.676+0000 7fb83bb3f740 1 --2- 172.21.15.16:0/4266322163 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3a29cd5d0 0x55b3a29d7da0 unknown :-1 s=CLOSED pgs=1483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:37.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.676+0000 7fb83bb3f740 1 -- 172.21.15.16:0/4266322163 >> 172.21.15.16:0/4266322163 conn(0x55b3a2996310 msgr2=0x55b3a29cf990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:37.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.676+0000 7fb83bb3f740 1 -- 172.21.15.16:0/4266322163 shutdown_connections 2024-09-12T02:12:37.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.676+0000 7fb83bb3f740 1 -- 172.21.15.16:0/4266322163 wait complete. 2024-09-12T02:12:37.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.676+0000 7fb83bb3f740 20 client.18759 trim_cache size 0 max 16384 2024-09-12T02:12:37.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.689+0000 7fb83bb3f740 1 Processor -- start 2024-09-12T02:12:37.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.689+0000 7fb83bb3f740 1 -- start start 2024-09-12T02:12:37.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.689+0000 7fb83bb3f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b3a29cd5d0 0x55b3a29f1630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:37.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.689+0000 7fb83bb3f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3a29ceb40 0x55b3a29f1b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:37.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.689+0000 7fb83bb3f740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b3a29d8370 0x55b3a29d22b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:37.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.689+0000 7fb83bb3f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb8341adb80 con 0x55b3a29ceb40 2024-09-12T02:12:37.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.689+0000 7fb83bb3f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb8341ab450 con 0x55b3a29cd5d0 2024-09-12T02:12:37.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.689+0000 7fb83bb3f740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb8341a92c0 con 0x55b3a29d8370 2024-09-12T02:12:37.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.689+0000 7fb82bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3a29ceb40 0x55b3a29f1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:37.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.690+0000 7fb82bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3a29ceb40 0x55b3a29f1b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36678/0 (socket says 172.21.15.16:36678) 2024-09-12T02:12:37.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.690+0000 7fb82bfff640 1 -- 172.21.15.16:0/1204204394 learned_addr learned my addr 172.21.15.16:0/1204204394 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:12:37.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.690+0000 7fb82bfff640 1 -- 172.21.15.16:0/1204204394 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b3a29d8370 msgr2=0x55b3a29d22b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:37.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.690+0000 7fb832ffd640 1 --2- 172.21.15.16:0/1204204394 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b3a29cd5d0 0x55b3a29f1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:37.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.690+0000 7fb8337fe640 1 --2- 172.21.15.16:0/1204204394 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b3a29d8370 0x55b3a29d22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:37.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.690+0000 7fb82bfff640 1 --2- 172.21.15.16:0/1204204394 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b3a29d8370 0x55b3a29d22b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:37.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.690+0000 7fb82bfff640 1 -- 172.21.15.16:0/1204204394 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b3a29cd5d0 msgr2=0x55b3a29f1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:37.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.690+0000 7fb82bfff640 1 --2- 172.21.15.16:0/1204204394 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b3a29cd5d0 0x55b3a29f1630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:37.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.690+0000 7fb82bfff640 1 -- 172.21.15.16:0/1204204394 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b3a29e8b70 con 0x55b3a29ceb40 2024-09-12T02:12:37.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.690+0000 7fb832ffd640 1 --2- 172.21.15.16:0/1204204394 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b3a29cd5d0 0x55b3a29f1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:12:37.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.690+0000 7fb82bfff640 1 --2- 172.21.15.16:0/1204204394 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3a29ceb40 0x55b3a29f1b70 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x55b3a29ce070 tx=0x7fb81c014840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa9c29ec3a9a9e69 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:37.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.690+0000 7fb828ff9640 1 -- 172.21.15.16:0/1204204394 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb81c009e50 con 0x55b3a29ceb40 2024-09-12T02:12:37.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.690+0000 7fb828ff9640 1 -- 172.21.15.16:0/1204204394 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb81c0140d0 con 0x55b3a29ceb40 2024-09-12T02:12:37.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.690+0000 7fb828ff9640 1 -- 172.21.15.16:0/1204204394 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb81c01d6a0 con 0x55b3a29ceb40 2024-09-12T02:12:37.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.691+0000 7fb83bb3f740 1 -- 172.21.15.16:0/1204204394 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3a29ceb40 msgr2=0x55b3a29f1b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:37.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.691+0000 7fb83bb3f740 1 --2- 172.21.15.16:0/1204204394 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3a29ceb40 0x55b3a29f1b70 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x55b3a29ce070 tx=0x7fb81c014840 comp rx=0 tx=0).stop 2024-09-12T02:12:37.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.691+0000 7fb83bb3f740 1 -- 172.21.15.16:0/1204204394 shutdown_connections 2024-09-12T02:12:37.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.691+0000 7fb83bb3f740 1 --2- 172.21.15.16:0/1204204394 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b3a29d8370 0x55b3a29d22b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:37.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.691+0000 7fb83bb3f740 1 --2- 172.21.15.16:0/1204204394 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3a29ceb40 0x55b3a29f1b70 unknown :-1 s=CLOSED pgs=1484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:37.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.691+0000 7fb83bb3f740 1 --2- 172.21.15.16:0/1204204394 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b3a29cd5d0 0x55b3a29f1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:37.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.691+0000 7fb83bb3f740 1 -- 172.21.15.16:0/1204204394 >> 172.21.15.16:0/1204204394 conn(0x55b3a2996310 msgr2=0x55b3a29d4140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:37.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.691+0000 7fb83bb3f740 1 -- 172.21.15.16:0/1204204394 shutdown_connections 2024-09-12T02:12:37.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.691+0000 7fb83bb3f740 1 -- 172.21.15.16:0/1204204394 wait complete. 2024-09-12T02:12:37.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.691+0000 7fb83bb3f740 1 -- start start 2024-09-12T02:12:37.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.691+0000 7fb83bb3f740 1 Processor -- start 2024-09-12T02:12:37.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.692+0000 7fb83bb3f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b3a29cd5d0 0x55b3a29ef5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:37.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.692+0000 7fb83bb3f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3a29ceb40 0x55b3a29efb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:37.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.692+0000 7fb83bb3f740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b3a29d8370 0x55b3a29f0060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:37.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.692+0000 7fb83bb3f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb8341a9910 con 0x55b3a29ceb40 2024-09-12T02:12:37.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.692+0000 7fb83bb3f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb8341aad80 con 0x55b3a29cd5d0 2024-09-12T02:12:37.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.692+0000 7fb83bb3f740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb8341a9fe0 con 0x55b3a29d8370 2024-09-12T02:12:37.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.692+0000 7fb83bb3f740 10 client.0 _ll_register_callbacks cb 0x7fb81c009320 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:12:37.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.692+0000 7fb82bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3a29ceb40 0x55b3a29efb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:37.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.692+0000 7fb82bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3a29ceb40 0x55b3a29efb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36692/0 (socket says 172.21.15.16:36692) 2024-09-12T02:12:37.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.692+0000 7fb82bfff640 1 -- 172.21.15.16:0/1156570759 learned_addr learned my addr 172.21.15.16:0/1156570759 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:12:37.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.692+0000 7fb8337fe640 1 --2- 172.21.15.16:0/1156570759 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b3a29d8370 0x55b3a29f0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:37.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.692+0000 7fb832ffd640 1 --2- 172.21.15.16:0/1156570759 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b3a29cd5d0 0x55b3a29ef5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:37.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.692+0000 7fb82bfff640 1 -- 172.21.15.16:0/1156570759 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b3a29d8370 msgr2=0x55b3a29f0060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:37.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.692+0000 7fb82bfff640 1 --2- 172.21.15.16:0/1156570759 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b3a29d8370 0x55b3a29f0060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:37.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.692+0000 7fb82bfff640 1 -- 172.21.15.16:0/1156570759 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b3a29cd5d0 msgr2=0x55b3a29ef5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:37.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.692+0000 7fb82bfff640 1 --2- 172.21.15.16:0/1156570759 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b3a29cd5d0 0x55b3a29ef5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:37.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.692+0000 7fb82bfff640 1 -- 172.21.15.16:0/1156570759 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b3a2a55d00 con 0x55b3a29ceb40 2024-09-12T02:12:37.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.692+0000 7fb82bfff640 1 --2- 172.21.15.16:0/1156570759 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3a29ceb40 0x55b3a29efb20 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x55b3a29c8330 tx=0x7fb81c0143c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6b26f93feb0bb8e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:37.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.692+0000 7fb81a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:12:37.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.693+0000 7fb81a7fc640 1 -- 172.21.15.16:0/1156570759 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb81c036060 con 0x55b3a29ceb40 2024-09-12T02:12:37.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.693+0000 7fb81a7fc640 1 -- 172.21.15.16:0/1156570759 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fb81c01e4d0 con 0x55b3a29ceb40 2024-09-12T02:12:37.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.693+0000 7fb83bb3f740 1 -- 172.21.15.16:0/1156570759 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb81c0140d0 con 0x55b3a29ceb40 2024-09-12T02:12:37.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.693+0000 7fb81a7fc640 1 -- 172.21.15.16:0/1156570759 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fb81c01e6d0 con 0x55b3a29ceb40 2024-09-12T02:12:37.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.693+0000 7fb83bb3f740 10 client.18765 fetch_fsmap learned FSMap version 25 2024-09-12T02:12:37.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.693+0000 7fb83bb3f740 1 -- 172.21.15.16:0/1156570759 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b3a2a56f30 con 0x55b3a29ceb40 2024-09-12T02:12:37.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.693+0000 7fb81a7fc640 1 -- 172.21.15.16:0/1156570759 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb81c0140d0 con 0x55b3a29ceb40 2024-09-12T02:12:37.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.693+0000 7fb81a7fc640 1 -- 172.21.15.16:0/1156570759 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fb81c03b070 con 0x55b3a29ceb40 2024-09-12T02:12:37.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.693+0000 7fb83bb3f740 10 client.18765 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:12:37.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.693+0000 7fb83bb3f740 10 client.18765 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:12:37.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.693+0000 7fb83bb3f740 20 client.18765 populate_metadata read hostname 'smithi016' 2024-09-12T02:12:37.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.693+0000 7fb83bb3f740 1 --2- 172.21.15.16:0/1156570759 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55b3a2a57660 0x55b3a2a59a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:37.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.693+0000 7fb83bb3f740 4 client.18765 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T02:12:37.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.693+0000 7fb83bb3f740 1 -- 172.21.15.16:0/1156570759 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b3a293ad70 con 0x55b3a2a57660 2024-09-12T02:12:37.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.694+0000 7fb832ffd640 1 --2- 172.21.15.16:0/1156570759 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55b3a2a57660 0x55b3a2a59a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:37.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.694+0000 7fb832ffd640 1 --2- 172.21.15.16:0/1156570759 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55b3a2a57660 0x55b3a2a59a80 crc :-1 s=READY pgs=313 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b71a196548a7fdf server_cookie=f0c7443a91877186 in_seq=0 out_seq=0 2024-09-12T02:12:37.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.694+0000 7fb81a7fc640 10 client.18765 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:12:37.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.694+0000 7fb81a7fc640 1 -- 172.21.15.16:0/1156570759 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b3a293ad70 con 0x55b3a2a57660 2024-09-12T02:12:37.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.694+0000 7fb81a7fc640 10 client.18765 handle_command_reply: tid=0 2024-09-12T02:12:37.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.694+0000 7fb83bb3f740 1 client.18765 shutdown 2024-09-12T02:12:37.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.694+0000 7fb83bb3f740 1 -- 172.21.15.16:0/1156570759 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3a29ceb40 msgr2=0x55b3a29efb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:37.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.694+0000 7fb83bb3f740 1 --2- 172.21.15.16:0/1156570759 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3a29ceb40 0x55b3a29efb20 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x55b3a29c8330 tx=0x7fb81c0143c0 comp rx=0 tx=0).stop 2024-09-12T02:12:37.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.694+0000 7fb83bb3f740 1 -- 172.21.15.16:0/1156570759 shutdown_connections 2024-09-12T02:12:37.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.694+0000 7fb83bb3f740 1 --2- 172.21.15.16:0/1156570759 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55b3a2a57660 0x55b3a2a59a80 crc :-1 s=READY pgs=313 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:37.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.694+0000 7fb83bb3f740 1 --2- 172.21.15.16:0/1156570759 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b3a29d8370 0x55b3a29f0060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:37.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.694+0000 7fb83bb3f740 1 --2- 172.21.15.16:0/1156570759 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3a29ceb40 0x55b3a29efb20 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:37.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.694+0000 7fb83bb3f740 1 --2- 172.21.15.16:0/1156570759 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b3a29cd5d0 0x55b3a29ef5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:37.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.694+0000 7fb83bb3f740 1 -- 172.21.15.16:0/1156570759 >> 172.21.15.16:0/1156570759 conn(0x55b3a2996310 msgr2=0x55b3a29e7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:37.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.695+0000 7fb81a7fc640 0 client.18765 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:12:37.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.695+0000 7fb83bb3f740 1 -- 172.21.15.16:0/1156570759 shutdown_connections 2024-09-12T02:12:37.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.695+0000 7fb83bb3f740 1 -- 172.21.15.16:0/1156570759 wait complete. 2024-09-12T02:12:37.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.695+0000 7fb83bb3f740 20 client.18765 trim_cache size 0 max 16384 2024-09-12T02:12:37.700 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T02:12:37.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.695+0000 7fb839766640 1 -- 172.21.15.16:0/3979875350 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fb8200a2b80 msgr2=0x7fb8200a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:37.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.695+0000 7fb839766640 1 --2- 172.21.15.16:0/3979875350 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fb8200a2b80 0x7fb8200a5040 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fb82c031f80 tx=0x7fb82c0095c0 comp rx=0 tx=0).stop 2024-09-12T02:12:37.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.695+0000 7fb839766640 1 -- 172.21.15.16:0/3979875350 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb834101de0 msgr2=0x7fb8341983f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:37.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.695+0000 7fb839766640 1 --2- 172.21.15.16:0/3979875350 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb834101de0 0x7fb8341983f0 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7fb82400c970 tx=0x7fb82400ce30 comp rx=0 tx=0).stop 2024-09-12T02:12:37.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.695+0000 7fb839766640 1 -- 172.21.15.16:0/3979875350 shutdown_connections 2024-09-12T02:12:37.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.695+0000 7fb839766640 1 --2- 172.21.15.16:0/3979875350 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fb8200a2b80 0x7fb8200a5040 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:37.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.695+0000 7fb839766640 1 --2- 172.21.15.16:0/3979875350 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb834106f10 0x7fb83419ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:37.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.695+0000 7fb839766640 1 --2- 172.21.15.16:0/3979875350 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb834102790 0x7fb834198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:37.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.695+0000 7fb839766640 1 --2- 172.21.15.16:0/3979875350 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb834101de0 0x7fb8341983f0 unknown :-1 s=CLOSED pgs=1481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:37.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.695+0000 7fb839766640 1 -- 172.21.15.16:0/3979875350 >> 172.21.15.16:0/3979875350 conn(0x7fb8340fd8c0 msgr2=0x7fb8340fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:37.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.695+0000 7fb839766640 1 -- 172.21.15.16:0/3979875350 shutdown_connections 2024-09-12T02:12:37.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:37.695+0000 7fb839766640 1 -- 172.21.15.16:0/3979875350 wait complete. 2024-09-12T02:12:38.306 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:12:38 smithi016 ceph-mon[30947]: pgmap v656: 97 pgs: 97 active+clean; 1.5 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-09-12T02:12:38.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:12:38 smithi184 ceph-mon[35593]: pgmap v656: 97 pgs: 97 active+clean; 1.5 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-09-12T02:12:38.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:12:38 smithi033 ceph-mon[35332]: pgmap v656: 97 pgs: 97 active+clean; 1.5 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-09-12T02:12:39.416 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-12T02:12:39.417 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-12T02:12:39.688 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:12:40.081 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:12:40 smithi016 ceph-mon[30947]: pgmap v657: 97 pgs: 97 active+clean; 1.5 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 11 op/s 2024-09-12T02:12:40.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.166+0000 7fa306134640 1 -- 172.21.15.16:0/4020034003 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa300102790 msgr2=0x7fa3001067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:40.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.166+0000 7fa306134640 1 --2- 172.21.15.16:0/4020034003 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa300102790 0x7fa3001067e0 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7fa2e8009900 tx=0x7fa2e802a410 comp rx=0 tx=0).stop 2024-09-12T02:12:40.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.166+0000 7fa306134640 1 -- 172.21.15.16:0/4020034003 shutdown_connections 2024-09-12T02:12:40.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.166+0000 7fa306134640 1 --2- 172.21.15.16:0/4020034003 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa300106f10 0x7fa30010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:40.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.166+0000 7fa306134640 1 --2- 172.21.15.16:0/4020034003 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa300102790 0x7fa3001067e0 unknown :-1 s=CLOSED pgs=1486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:40.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.166+0000 7fa306134640 1 --2- 172.21.15.16:0/4020034003 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa300101de0 0x7fa3001021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:40.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.166+0000 7fa306134640 1 -- 172.21.15.16:0/4020034003 >> 172.21.15.16:0/4020034003 conn(0x7fa3000fd8c0 msgr2=0x7fa3000ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:40.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.166+0000 7fa306134640 1 -- 172.21.15.16:0/4020034003 shutdown_connections 2024-09-12T02:12:40.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.166+0000 7fa306134640 1 -- 172.21.15.16:0/4020034003 wait complete. 2024-09-12T02:12:40.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.167+0000 7fa306134640 1 Processor -- start 2024-09-12T02:12:40.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.167+0000 7fa306134640 1 -- start start 2024-09-12T02:12:40.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.167+0000 7fa306134640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa300101de0 0x7fa30019a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:40.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.167+0000 7fa306134640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa300102790 0x7fa30019abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:40.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.167+0000 7fa306134640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa300106f10 0x7fa30019ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:40.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.167+0000 7fa306134640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa300111f30 con 0x7fa300101de0 2024-09-12T02:12:40.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.167+0000 7fa306134640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa300111db0 con 0x7fa300102790 2024-09-12T02:12:40.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.167+0000 7fa306134640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa3001120b0 con 0x7fa300106f10 2024-09-12T02:12:40.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.167+0000 7fa2ff7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa300101de0 0x7fa30019a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:40.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.168+0000 7fa2ff7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa300101de0 0x7fa30019a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36716/0 (socket says 172.21.15.16:36716) 2024-09-12T02:12:40.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.168+0000 7fa2ff7fe640 1 -- 172.21.15.16:0/1632400306 learned_addr learned my addr 172.21.15.16:0/1632400306 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:12:40.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.168+0000 7fa2f7fff640 1 --2- 172.21.15.16:0/1632400306 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa300102790 0x7fa30019abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:40.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.168+0000 7fa2fffff640 1 --2- 172.21.15.16:0/1632400306 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa300106f10 0x7fa30019ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:40.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.168+0000 7fa2ff7fe640 1 -- 172.21.15.16:0/1632400306 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa300106f10 msgr2=0x7fa30019ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:40.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.168+0000 7fa2ff7fe640 1 --2- 172.21.15.16:0/1632400306 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa300106f10 0x7fa30019ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:40.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.168+0000 7fa2ff7fe640 1 -- 172.21.15.16:0/1632400306 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa300102790 msgr2=0x7fa30019abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:40.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.168+0000 7fa2ff7fe640 1 --2- 172.21.15.16:0/1632400306 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa300102790 0x7fa30019abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:40.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.168+0000 7fa2ff7fe640 1 -- 172.21.15.16:0/1632400306 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa30019f620 con 0x7fa300101de0 2024-09-12T02:12:40.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.168+0000 7fa2f7fff640 1 --2- 172.21.15.16:0/1632400306 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa300102790 0x7fa30019abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:12:40.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.168+0000 7fa2ff7fe640 1 --2- 172.21.15.16:0/1632400306 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa300101de0 0x7fa30019a670 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7fa2f000d950 tx=0x7fa2f000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:40.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.168+0000 7fa2fd7fa640 1 -- 172.21.15.16:0/1632400306 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2f0017070 con 0x7fa300101de0 2024-09-12T02:12:40.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.168+0000 7fa2fd7fa640 1 -- 172.21.15.16:0/1632400306 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2f0009c30 con 0x7fa300101de0 2024-09-12T02:12:40.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.168+0000 7fa306134640 1 -- 172.21.15.16:0/1632400306 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa30019f910 con 0x7fa300101de0 2024-09-12T02:12:40.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.168+0000 7fa2fd7fa640 1 -- 172.21.15.16:0/1632400306 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2f000f6b0 con 0x7fa300101de0 2024-09-12T02:12:40.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.168+0000 7fa306134640 1 -- 172.21.15.16:0/1632400306 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3001a71f0 con 0x7fa300101de0 2024-09-12T02:12:40.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.170+0000 7fa2fd7fa640 1 -- 172.21.15.16:0/1632400306 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa2f00109d0 con 0x7fa300101de0 2024-09-12T02:12:40.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.171+0000 7fa30850d740 1 Processor -- start 2024-09-12T02:12:40.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.171+0000 7fa30850d740 1 -- start start 2024-09-12T02:12:40.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.171+0000 7fa30850d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fb4014bb60 0x55fb4014bf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:40.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.171+0000 7fa30850d740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fb4014c480 0x55fb40156800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:40.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.171+0000 7fa30850d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fb40156d40 0x55fb40159130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:40.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.171+0000 7fa30850d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55fb400b1280 con 0x55fb40156d40 2024-09-12T02:12:40.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.171+0000 7fa30850d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55fb40058b10 con 0x55fb4014bb60 2024-09-12T02:12:40.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.171+0000 7fa30850d740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55fb400ad240 con 0x55fb4014c480 2024-09-12T02:12:40.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.171+0000 7fa2f7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fb40156d40 0x55fb40159130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:40.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.171+0000 7fa2f7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fb40156d40 0x55fb40159130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36718/0 (socket says 172.21.15.16:36718) 2024-09-12T02:12:40.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.171+0000 7fa2f7fff640 1 -- 172.21.15.16:0/3616682186 learned_addr learned my addr 172.21.15.16:0/3616682186 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:12:40.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.171+0000 7fa2fffff640 1 --2- 172.21.15.16:0/3616682186 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fb4014bb60 0x55fb4014bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:40.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.171+0000 7fa2fd7fa640 1 --2- 172.21.15.16:0/1632400306 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fa2ec0a2c00 0x7fa2ec0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:40.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.171+0000 7fa2ff7fe640 1 --2- 172.21.15.16:0/3616682186 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fb4014c480 0x55fb40156800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:40.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.171+0000 7fa2f7fff640 1 -- 172.21.15.16:0/3616682186 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fb4014c480 msgr2=0x55fb40156800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:40.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.171+0000 7fa2f7fff640 1 --2- 172.21.15.16:0/3616682186 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fb4014c480 0x55fb40156800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:40.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.171+0000 7fa2f7fff640 1 -- 172.21.15.16:0/3616682186 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fb4014bb60 msgr2=0x55fb4014bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:40.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.171+0000 7fa2f7fff640 1 --2- 172.21.15.16:0/3616682186 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fb4014bb60 0x55fb4014bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:40.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.171+0000 7fa2f7fff640 1 -- 172.21.15.16:0/3616682186 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb400651f0 con 0x55fb40156d40 2024-09-12T02:12:40.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.172+0000 7fa2fffff640 1 --2- 172.21.15.16:0/1632400306 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fa2ec0a2c00 0x7fa2ec0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:40.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.172+0000 7fa2fd7fa640 1 -- 172.21.15.16:0/1632400306 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fa2f00ceb90 con 0x7fa300101de0 2024-09-12T02:12:40.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.172+0000 7fa2f7fff640 1 --2- 172.21.15.16:0/3616682186 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fb40156d40 0x55fb40159130 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7fa2e8009900 tx=0x7fa2e8005e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ab14064cb32645b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:40.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.172+0000 7fa2f5ffb640 1 -- 172.21.15.16:0/3616682186 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2e803f070 con 0x55fb40156d40 2024-09-12T02:12:40.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.172+0000 7fa2f5ffb640 1 -- 172.21.15.16:0/3616682186 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2e8004030 con 0x55fb40156d40 2024-09-12T02:12:40.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.172+0000 7fa2f5ffb640 1 -- 172.21.15.16:0/3616682186 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2e8035070 con 0x55fb40156d40 2024-09-12T02:12:40.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.173+0000 7fa30850d740 1 -- 172.21.15.16:0/3616682186 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fb40156d40 msgr2=0x55fb40159130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:40.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.173+0000 7fa30850d740 1 --2- 172.21.15.16:0/3616682186 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fb40156d40 0x55fb40159130 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7fa2e8009900 tx=0x7fa2e8005e60 comp rx=0 tx=0).stop 2024-09-12T02:12:40.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.173+0000 7fa2fffff640 1 --2- 172.21.15.16:0/1632400306 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fa2ec0a2c00 0x7fa2ec0a50c0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fa2f8005c40 tx=0x7fa2f8004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:40.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.173+0000 7fa30850d740 1 -- 172.21.15.16:0/3616682186 shutdown_connections 2024-09-12T02:12:40.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.173+0000 7fa30850d740 1 --2- 172.21.15.16:0/3616682186 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fb40156d40 0x55fb40159130 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:40.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.173+0000 7fa30850d740 1 --2- 172.21.15.16:0/3616682186 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fb4014c480 0x55fb40156800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:40.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.173+0000 7fa30850d740 1 --2- 172.21.15.16:0/3616682186 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fb4014bb60 0x55fb4014bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:40.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.173+0000 7fa30850d740 1 -- 172.21.15.16:0/3616682186 >> 172.21.15.16:0/3616682186 conn(0x55fb401480f0 msgr2=0x55fb401484d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:40.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.173+0000 7fa30850d740 1 -- 172.21.15.16:0/3616682186 shutdown_connections 2024-09-12T02:12:40.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.173+0000 7fa30850d740 1 -- 172.21.15.16:0/3616682186 wait complete. 2024-09-12T02:12:40.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.174+0000 7fa30850d740 1 -- start start 2024-09-12T02:12:40.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.174+0000 7fa30850d740 1 Processor -- start 2024-09-12T02:12:40.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.174+0000 7fa30850d740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fb4014bb60 0x55fb40153c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:40.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.174+0000 7fa30850d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fb4014c480 0x55fb40154140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:40.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.174+0000 7fa30850d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fb40156d40 0x55fb40154680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:40.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.174+0000 7fa30850d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55fb40159910 con 0x55fb40156d40 2024-09-12T02:12:40.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.174+0000 7fa30850d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55fb4006bf10 con 0x55fb4014c480 2024-09-12T02:12:40.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.174+0000 7fa30850d740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55fb40159a90 con 0x55fb4014bb60 2024-09-12T02:12:40.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.174+0000 7fa30850d740 10 client.0 _ll_register_callbacks cb 0x55fb40034a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:12:40.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.174+0000 7fa2fffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fb40156d40 0x55fb40154680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:40.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.174+0000 7fa2fffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fb40156d40 0x55fb40154680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36720/0 (socket says 172.21.15.16:36720) 2024-09-12T02:12:40.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.174+0000 7fa2fffff640 1 -- 172.21.15.16:0/301356027 learned_addr learned my addr 172.21.15.16:0/301356027 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:12:40.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.175+0000 7fa2f7fff640 1 --2- 172.21.15.16:0/301356027 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fb4014c480 0x55fb40154140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:40.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.175+0000 7fa2ff7fe640 1 --2- 172.21.15.16:0/301356027 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fb4014bb60 0x55fb40153c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:40.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.175+0000 7fa2fffff640 1 -- 172.21.15.16:0/301356027 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fb4014bb60 msgr2=0x55fb40153c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:40.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.175+0000 7fa2fffff640 1 --2- 172.21.15.16:0/301356027 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fb4014bb60 0x55fb40153c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:40.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.175+0000 7fa2fffff640 1 -- 172.21.15.16:0/301356027 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fb4014c480 msgr2=0x55fb40154140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:40.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.175+0000 7fa2fffff640 1 --2- 172.21.15.16:0/301356027 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fb4014c480 0x55fb40154140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:40.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.175+0000 7fa2fffff640 1 -- 172.21.15.16:0/301356027 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb3fe226a0 con 0x55fb40156d40 2024-09-12T02:12:40.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.175+0000 7fa2fffff640 1 --2- 172.21.15.16:0/301356027 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fb40156d40 0x55fb40154680 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7fa2f800fac0 tx=0x7fa2f8004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d641520edbe2d199 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:40.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.175+0000 7fa2deffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:12:40.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.175+0000 7fa2deffd640 1 -- 172.21.15.16:0/301356027 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2f802b060 con 0x55fb40156d40 2024-09-12T02:12:40.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.175+0000 7fa2deffd640 1 -- 172.21.15.16:0/301356027 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fa2f801d5f0 con 0x55fb40156d40 2024-09-12T02:12:40.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.175+0000 7fa30850d740 1 -- 172.21.15.16:0/301356027 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2e8004030 con 0x55fb40156d40 2024-09-12T02:12:40.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.175+0000 7fa2deffd640 1 -- 172.21.15.16:0/301356027 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa2f801d7f0 con 0x55fb40156d40 2024-09-12T02:12:40.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.175+0000 7fa30850d740 10 client.18777 fetch_fsmap learned FSMap version 25 2024-09-12T02:12:40.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.175+0000 7fa30850d740 1 -- 172.21.15.16:0/301356027 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55fb40165550 con 0x55fb40156d40 2024-09-12T02:12:40.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.176+0000 7fa2deffd640 1 -- 172.21.15.16:0/301356027 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2e8004030 con 0x55fb40156d40 2024-09-12T02:12:40.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.176+0000 7fa2deffd640 1 -- 172.21.15.16:0/301356027 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fa2f8030070 con 0x55fb40156d40 2024-09-12T02:12:40.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.176+0000 7fa30850d740 10 client.18777 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:12:40.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.176+0000 7fa30850d740 10 client.18777 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:12:40.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.176+0000 7fa30850d740 20 client.18777 populate_metadata read hostname 'smithi016' 2024-09-12T02:12:40.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.176+0000 7fa30850d740 1 --2- 172.21.15.16:0/301356027 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55fb40166a10 0x55fb40168e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:40.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.176+0000 7fa30850d740 4 client.18777 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:12:40.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.176+0000 7fa30850d740 1 -- 172.21.15.16:0/301356027 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fb400be7f0 con 0x55fb40166a10 2024-09-12T02:12:40.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.176+0000 7fa2ff7fe640 1 --2- 172.21.15.16:0/301356027 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55fb40166a10 0x55fb40168e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:40.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.177+0000 7fa2ff7fe640 1 --2- 172.21.15.16:0/301356027 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55fb40166a10 0x55fb40168e20 crc :-1 s=READY pgs=314 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7c1515d0a86e58f server_cookie=781ca3c160172b2 in_seq=0 out_seq=0 2024-09-12T02:12:40.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.177+0000 7fa2deffd640 10 client.18777 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:12:40.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.177+0000 7fa2deffd640 1 -- 172.21.15.16:0/301356027 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fb400be7f0 con 0x55fb40166a10 2024-09-12T02:12:40.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.177+0000 7fa2deffd640 10 client.18777 handle_command_reply: tid=0 2024-09-12T02:12:40.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.177+0000 7fa30850d740 1 client.18777 shutdown 2024-09-12T02:12:40.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.178+0000 7fa30850d740 1 -- 172.21.15.16:0/301356027 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fb40156d40 msgr2=0x55fb40154680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:40.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.178+0000 7fa30850d740 1 --2- 172.21.15.16:0/301356027 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fb40156d40 0x55fb40154680 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7fa2f800fac0 tx=0x7fa2f8004ab0 comp rx=0 tx=0).stop 2024-09-12T02:12:40.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.178+0000 7fa30850d740 1 -- 172.21.15.16:0/301356027 shutdown_connections 2024-09-12T02:12:40.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.178+0000 7fa30850d740 1 --2- 172.21.15.16:0/301356027 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55fb40166a10 0x55fb40168e20 crc :-1 s=READY pgs=314 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:40.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.178+0000 7fa2deffd640 0 client.18777 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:12:40.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.178+0000 7fa30850d740 1 --2- 172.21.15.16:0/301356027 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fb40156d40 0x55fb40154680 unknown :-1 s=CLOSED pgs=1489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:40.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.178+0000 7fa30850d740 1 --2- 172.21.15.16:0/301356027 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fb4014c480 0x55fb40154140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:40.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.178+0000 7fa30850d740 1 --2- 172.21.15.16:0/301356027 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fb4014bb60 0x55fb40153c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:40.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.178+0000 7fa30850d740 1 -- 172.21.15.16:0/301356027 >> 172.21.15.16:0/301356027 conn(0x55fb401480f0 msgr2=0x55fb4014ddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:40.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.178+0000 7fa30850d740 1 -- 172.21.15.16:0/301356027 shutdown_connections 2024-09-12T02:12:40.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.178+0000 7fa30850d740 1 -- 172.21.15.16:0/301356027 wait complete. 2024-09-12T02:12:40.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.178+0000 7fa30850d740 20 client.18777 trim_cache size 0 max 16384 2024-09-12T02:12:40.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.195+0000 7fa30850d740 1 Processor -- start 2024-09-12T02:12:40.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.195+0000 7fa30850d740 1 -- start start 2024-09-12T02:12:40.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.195+0000 7fa30850d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fb4014bb60 0x55fb4016e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:40.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.195+0000 7fa30850d740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fb4014c480 0x55fb4016ec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:40.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.195+0000 7fa30850d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fb40156d40 0x55fb40155350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:40.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.195+0000 7fa30850d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa3001b1f10 con 0x55fb40156d40 2024-09-12T02:12:40.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.195+0000 7fa30850d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa3001af7e0 con 0x55fb4014bb60 2024-09-12T02:12:40.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.195+0000 7fa30850d740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa3001ad630 con 0x55fb4014c480 2024-09-12T02:12:40.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.196+0000 7fa2fffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fb40156d40 0x55fb40155350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:40.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.196+0000 7fa2fffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fb40156d40 0x55fb40155350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36724/0 (socket says 172.21.15.16:36724) 2024-09-12T02:12:40.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.196+0000 7fa2fffff640 1 -- 172.21.15.16:0/260156510 learned_addr learned my addr 172.21.15.16:0/260156510 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:12:40.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.196+0000 7fa2f7fff640 1 --2- 172.21.15.16:0/260156510 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fb4014c480 0x55fb4016ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:40.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.196+0000 7fa2ff7fe640 1 --2- 172.21.15.16:0/260156510 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fb4014bb60 0x55fb4016e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:40.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.196+0000 7fa2fffff640 1 -- 172.21.15.16:0/260156510 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fb4014c480 msgr2=0x55fb4016ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:40.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.196+0000 7fa2fffff640 1 --2- 172.21.15.16:0/260156510 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fb4014c480 0x55fb4016ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:40.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.196+0000 7fa2fffff640 1 -- 172.21.15.16:0/260156510 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fb4014bb60 msgr2=0x55fb4016e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:40.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.196+0000 7fa2fffff640 1 --2- 172.21.15.16:0/260156510 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fb4014bb60 0x55fb4016e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:40.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.196+0000 7fa2fffff640 1 -- 172.21.15.16:0/260156510 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb40159960 con 0x55fb40156d40 2024-09-12T02:12:40.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.196+0000 7fa2ff7fe640 1 --2- 172.21.15.16:0/260156510 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fb4014bb60 0x55fb4016e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:12:40.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.196+0000 7fa2fffff640 1 --2- 172.21.15.16:0/260156510 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fb40156d40 0x55fb40155350 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7fa2f8011fd0 tx=0x7fa2f8009320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=960968902f3c90b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:40.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.196+0000 7fa2f57fa640 1 -- 172.21.15.16:0/260156510 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2f8036070 con 0x55fb40156d40 2024-09-12T02:12:40.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.196+0000 7fa2f57fa640 1 -- 172.21.15.16:0/260156510 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2f801d5f0 con 0x55fb40156d40 2024-09-12T02:12:40.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.196+0000 7fa2f57fa640 1 -- 172.21.15.16:0/260156510 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2f802b070 con 0x55fb40156d40 2024-09-12T02:12:40.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.197+0000 7fa30850d740 1 -- 172.21.15.16:0/260156510 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fb40156d40 msgr2=0x55fb40155350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:40.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.197+0000 7fa30850d740 1 --2- 172.21.15.16:0/260156510 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fb40156d40 0x55fb40155350 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7fa2f8011fd0 tx=0x7fa2f8009320 comp rx=0 tx=0).stop 2024-09-12T02:12:40.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.197+0000 7fa30850d740 1 -- 172.21.15.16:0/260156510 shutdown_connections 2024-09-12T02:12:40.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.197+0000 7fa30850d740 1 --2- 172.21.15.16:0/260156510 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fb40156d40 0x55fb40155350 unknown :-1 s=CLOSED pgs=1490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:40.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.197+0000 7fa30850d740 1 --2- 172.21.15.16:0/260156510 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fb4014c480 0x55fb4016ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:40.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.197+0000 7fa30850d740 1 --2- 172.21.15.16:0/260156510 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fb4014bb60 0x55fb4016e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:40.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.197+0000 7fa30850d740 1 -- 172.21.15.16:0/260156510 >> 172.21.15.16:0/260156510 conn(0x55fb401480f0 msgr2=0x55fb40152cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:40.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.197+0000 7fa30850d740 1 -- 172.21.15.16:0/260156510 shutdown_connections 2024-09-12T02:12:40.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.197+0000 7fa30850d740 1 -- 172.21.15.16:0/260156510 wait complete. 2024-09-12T02:12:40.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.197+0000 7fa30850d740 1 -- start start 2024-09-12T02:12:40.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.197+0000 7fa30850d740 1 Processor -- start 2024-09-12T02:12:40.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.198+0000 7fa30850d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fb4014bb60 0x55fb4015f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:40.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.198+0000 7fa30850d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fb4014c480 0x55fb40150990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:40.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.198+0000 7fa30850d740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fb40156d40 0x55fb40150ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:40.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.198+0000 7fa30850d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa3001adca0 con 0x55fb4014bb60 2024-09-12T02:12:40.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.198+0000 7fa30850d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa3001af110 con 0x55fb4014c480 2024-09-12T02:12:40.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.198+0000 7fa30850d740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa3001ae370 con 0x55fb40156d40 2024-09-12T02:12:40.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.198+0000 7fa30850d740 10 client.0 _ll_register_callbacks cb 0x7fa2f8009e40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:12:40.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.198+0000 7fa2ff7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fb4014bb60 0x55fb4015f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:40.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.198+0000 7fa2ff7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fb4014bb60 0x55fb4015f470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36728/0 (socket says 172.21.15.16:36728) 2024-09-12T02:12:40.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.198+0000 7fa2ff7fe640 1 -- 172.21.15.16:0/2680117235 learned_addr learned my addr 172.21.15.16:0/2680117235 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:12:40.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.198+0000 7fa2fffff640 1 --2- 172.21.15.16:0/2680117235 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fb40156d40 0x55fb40150ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:40.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.198+0000 7fa2f7fff640 1 --2- 172.21.15.16:0/2680117235 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fb4014c480 0x55fb40150990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:40.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.198+0000 7fa2ff7fe640 1 -- 172.21.15.16:0/2680117235 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fb40156d40 msgr2=0x55fb40150ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:40.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.198+0000 7fa2ff7fe640 1 --2- 172.21.15.16:0/2680117235 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fb40156d40 0x55fb40150ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:40.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.198+0000 7fa2ff7fe640 1 -- 172.21.15.16:0/2680117235 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fb4014c480 msgr2=0x55fb40150990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:40.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.198+0000 7fa2ff7fe640 1 --2- 172.21.15.16:0/2680117235 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fb4014c480 0x55fb40150990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:40.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.198+0000 7fa2ff7fe640 1 -- 172.21.15.16:0/2680117235 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb40151700 con 0x55fb4014bb60 2024-09-12T02:12:40.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.198+0000 7fa2f7fff640 1 --2- 172.21.15.16:0/2680117235 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fb4014c480 0x55fb40150990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:12:40.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.198+0000 7fa2ff7fe640 1 --2- 172.21.15.16:0/2680117235 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fb4014bb60 0x55fb4015f470 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7fa2f009b190 tx=0x7fa2f0097130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc2ed8185e22d565 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:40.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.198+0000 7fa2deffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:12:40.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.198+0000 7fa2deffd640 1 -- 172.21.15.16:0/2680117235 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2f0020040 con 0x55fb4014bb60 2024-09-12T02:12:40.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.199+0000 7fa2deffd640 1 -- 172.21.15.16:0/2680117235 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fa2f0099530 con 0x55fb4014bb60 2024-09-12T02:12:40.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.199+0000 7fa30850d740 1 -- 172.21.15.16:0/2680117235 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2f801d5f0 con 0x55fb4014bb60 2024-09-12T02:12:40.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.199+0000 7fa2deffd640 1 -- 172.21.15.16:0/2680117235 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa2f00a4100 con 0x55fb4014bb60 2024-09-12T02:12:40.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.199+0000 7fa30850d740 10 client.18783 fetch_fsmap learned FSMap version 25 2024-09-12T02:12:40.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.199+0000 7fa30850d740 1 -- 172.21.15.16:0/2680117235 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55fb401d16d0 con 0x55fb4014bb60 2024-09-12T02:12:40.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.199+0000 7fa2deffd640 1 -- 172.21.15.16:0/2680117235 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2f801d5f0 con 0x55fb4014bb60 2024-09-12T02:12:40.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.199+0000 7fa2deffd640 1 -- 172.21.15.16:0/2680117235 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fa2f00ccc70 con 0x55fb4014bb60 2024-09-12T02:12:40.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.199+0000 7fa30850d740 10 client.18783 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:12:40.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.199+0000 7fa30850d740 10 client.18783 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:12:40.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.199+0000 7fa30850d740 20 client.18783 populate_metadata read hostname 'smithi016' 2024-09-12T02:12:40.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.199+0000 7fa30850d740 1 --2- 172.21.15.16:0/2680117235 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55fb40166a10 0x55fb401d3e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:40.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.199+0000 7fa30850d740 4 client.18783 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-12T02:12:40.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.199+0000 7fa30850d740 1 -- 172.21.15.16:0/2680117235 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55fb400b8d00 con 0x55fb40166a10 2024-09-12T02:12:40.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.199+0000 7fa2f7fff640 1 --2- 172.21.15.16:0/2680117235 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55fb40166a10 0x55fb401d3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:40.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.200+0000 7fa2f7fff640 1 --2- 172.21.15.16:0/2680117235 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55fb40166a10 0x55fb401d3e00 crc :-1 s=READY pgs=315 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c2114afef286628 server_cookie=57d0b0a52cb92104 in_seq=0 out_seq=0 2024-09-12T02:12:40.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.200+0000 7fa2deffd640 10 client.18783 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:12:40.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.201+0000 7fa2deffd640 1 -- 172.21.15.16:0/2680117235 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55fb400b8d00 con 0x55fb40166a10 2024-09-12T02:12:40.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.201+0000 7fa2deffd640 10 client.18783 handle_command_reply: tid=0 2024-09-12T02:12:40.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.201+0000 7fa30850d740 1 client.18783 shutdown 2024-09-12T02:12:40.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.201+0000 7fa30850d740 1 -- 172.21.15.16:0/2680117235 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fb4014bb60 msgr2=0x55fb4015f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:40.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.201+0000 7fa30850d740 1 --2- 172.21.15.16:0/2680117235 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fb4014bb60 0x55fb4015f470 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7fa2f009b190 tx=0x7fa2f0097130 comp rx=0 tx=0).stop 2024-09-12T02:12:40.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.201+0000 7fa30850d740 1 -- 172.21.15.16:0/2680117235 shutdown_connections 2024-09-12T02:12:40.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.201+0000 7fa30850d740 1 --2- 172.21.15.16:0/2680117235 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55fb40166a10 0x55fb401d3e00 crc :-1 s=READY pgs=315 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:40.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.201+0000 7fa30850d740 1 --2- 172.21.15.16:0/2680117235 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fb40156d40 0x55fb40150ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:40.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.202+0000 7fa30850d740 1 --2- 172.21.15.16:0/2680117235 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fb4014c480 0x55fb40150990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:40.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.202+0000 7fa30850d740 1 --2- 172.21.15.16:0/2680117235 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fb4014bb60 0x55fb4015f470 unknown :-1 s=CLOSED pgs=1491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:40.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.202+0000 7fa30850d740 1 -- 172.21.15.16:0/2680117235 >> 172.21.15.16:0/2680117235 conn(0x55fb401480f0 msgr2=0x55fb40146a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:40.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.202+0000 7fa2deffd640 0 client.18783 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:12:40.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.202+0000 7fa30850d740 1 -- 172.21.15.16:0/2680117235 shutdown_connections 2024-09-12T02:12:40.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.202+0000 7fa30850d740 1 -- 172.21.15.16:0/2680117235 wait complete. 2024-09-12T02:12:40.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.202+0000 7fa30850d740 20 client.18783 trim_cache size 0 max 16384 2024-09-12T02:12:40.208 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T02:12:40.209 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-09-12T02:12:40.209 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "a2ab018d-f42a-420c-b36a-1844a9e81637", 2024-09-12T02:12:40.209 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-09-12T02:12:40.209 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T02:12:40.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.202+0000 7fa306134640 1 -- 172.21.15.16:0/1632400306 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fa2ec0a2c00 msgr2=0x7fa2ec0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:40.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.202+0000 7fa306134640 1 --2- 172.21.15.16:0/1632400306 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fa2ec0a2c00 0x7fa2ec0a50c0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fa2f8005c40 tx=0x7fa2f8004a40 comp rx=0 tx=0).stop 2024-09-12T02:12:40.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.202+0000 7fa306134640 1 -- 172.21.15.16:0/1632400306 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa300101de0 msgr2=0x7fa30019a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:40.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.202+0000 7fa306134640 1 --2- 172.21.15.16:0/1632400306 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa300101de0 0x7fa30019a670 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7fa2f000d950 tx=0x7fa2f000de10 comp rx=0 tx=0).stop 2024-09-12T02:12:40.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.202+0000 7fa306134640 1 -- 172.21.15.16:0/1632400306 shutdown_connections 2024-09-12T02:12:40.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.202+0000 7fa306134640 1 --2- 172.21.15.16:0/1632400306 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fa2ec0a2c00 0x7fa2ec0a50c0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:40.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.202+0000 7fa306134640 1 --2- 172.21.15.16:0/1632400306 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa300106f10 0x7fa30019ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:40.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.202+0000 7fa306134640 1 --2- 172.21.15.16:0/1632400306 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa300102790 0x7fa30019abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:40.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.202+0000 7fa306134640 1 --2- 172.21.15.16:0/1632400306 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa300101de0 0x7fa30019a670 unknown :-1 s=CLOSED pgs=1487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:40.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.202+0000 7fa306134640 1 -- 172.21.15.16:0/1632400306 >> 172.21.15.16:0/1632400306 conn(0x7fa3000fd8c0 msgr2=0x7fa3000febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:40.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.202+0000 7fa306134640 1 -- 172.21.15.16:0/1632400306 shutdown_connections 2024-09-12T02:12:40.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:40.202+0000 7fa306134640 1 -- 172.21.15.16:0/1632400306 wait complete. 2024-09-12T02:12:40.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:12:40 smithi184 ceph-mon[35593]: pgmap v657: 97 pgs: 97 active+clean; 1.5 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 11 op/s 2024-09-12T02:12:40.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:12:40 smithi033 ceph-mon[35332]: pgmap v657: 97 pgs: 97 active+clean; 1.5 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 11 op/s 2024-09-12T02:12:40.854 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T02:12:41.111 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:12:41.183 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:12:41 smithi016 ceph-mon[30947]: scrub queued for path: / 2024-09-12T02:12:41.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:12:41 smithi016 ceph-mon[30947]: scrub summary: idle+waiting paths [/] 2024-09-12T02:12:41.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:12:41 smithi016 ceph-mon[30947]: scrub summary: active paths [/] 2024-09-12T02:12:41.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:12:41 smithi016 ceph-mon[30947]: scrub summary: idle+waiting paths [/] 2024-09-12T02:12:41.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:12:41 smithi184 ceph-mon[35593]: scrub queued for path: / 2024-09-12T02:12:41.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:12:41 smithi184 ceph-mon[35593]: scrub summary: idle+waiting paths [/] 2024-09-12T02:12:41.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:12:41 smithi184 ceph-mon[35593]: scrub summary: active paths [/] 2024-09-12T02:12:41.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:12:41 smithi184 ceph-mon[35593]: scrub summary: idle+waiting paths [/] 2024-09-12T02:12:41.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:12:41 smithi033 ceph-mon[35332]: scrub queued for path: / 2024-09-12T02:12:41.581 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:12:41 smithi033 ceph-mon[35332]: scrub summary: idle+waiting paths [/] 2024-09-12T02:12:41.581 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:12:41 smithi033 ceph-mon[35332]: scrub summary: active paths [/] 2024-09-12T02:12:41.581 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:12:41 smithi033 ceph-mon[35332]: scrub summary: idle+waiting paths [/] 2024-09-12T02:12:41.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.564+0000 7f9041405640 1 -- 172.21.15.16:0/2447930866 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f903c10c190 msgr2=0x7f903c10e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:41.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.564+0000 7f9041405640 1 --2- 172.21.15.16:0/2447930866 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f903c10c190 0x7f903c10e620 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f9034009900 tx=0x7f903402a480 comp rx=0 tx=0).stop 2024-09-12T02:12:41.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.565+0000 7f9041405640 1 -- 172.21.15.16:0/2447930866 shutdown_connections 2024-09-12T02:12:41.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.565+0000 7f9041405640 1 --2- 172.21.15.16:0/2447930866 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f903c10c190 0x7f903c10e620 unknown :-1 s=CLOSED pgs=1492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:41.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.565+0000 7f9041405640 1 --2- 172.21.15.16:0/2447930866 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f903c103710 0x7f903c10bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:41.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.565+0000 7f9041405640 1 --2- 172.21.15.16:0/2447930866 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f903c102df0 0x7f903c1031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:41.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.565+0000 7f9041405640 1 -- 172.21.15.16:0/2447930866 >> 172.21.15.16:0/2447930866 conn(0x7f903c0fc7d0 msgr2=0x7f903c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:41.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.565+0000 7f9041405640 1 -- 172.21.15.16:0/2447930866 shutdown_connections 2024-09-12T02:12:41.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.565+0000 7f9041405640 1 -- 172.21.15.16:0/2447930866 wait complete. 2024-09-12T02:12:41.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.565+0000 7f9041405640 1 Processor -- start 2024-09-12T02:12:41.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.565+0000 7f9041405640 1 -- start start 2024-09-12T02:12:41.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.566+0000 7f9041405640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f903c102df0 0x7f903c100a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:41.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.566+0000 7f9041405640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f903c103710 0x7f903c100f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:41.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.566+0000 7f9041405640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f903c10c190 0x7f903c1014d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:41.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.566+0000 7f9041405640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f903c111f80 con 0x7f903c103710 2024-09-12T02:12:41.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.566+0000 7f9041405640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f903c111e00 con 0x7f903c102df0 2024-09-12T02:12:41.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.566+0000 7f9041405640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f903c112100 con 0x7f903c10c190 2024-09-12T02:12:41.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.566+0000 7f903a7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f903c103710 0x7f903c100f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:41.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.566+0000 7f903a7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f903c103710 0x7f903c100f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60754/0 (socket says 172.21.15.16:60754) 2024-09-12T02:12:41.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.566+0000 7f903a7fc640 1 -- 172.21.15.16:0/3062009871 learned_addr learned my addr 172.21.15.16:0/3062009871 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:12:41.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.566+0000 7f903b7fe640 1 --2- 172.21.15.16:0/3062009871 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f903c10c190 0x7f903c1014d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:41.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.566+0000 7f903affd640 1 --2- 172.21.15.16:0/3062009871 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f903c102df0 0x7f903c100a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:41.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.566+0000 7f903a7fc640 1 -- 172.21.15.16:0/3062009871 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f903c10c190 msgr2=0x7f903c1014d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:41.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.566+0000 7f903a7fc640 1 --2- 172.21.15.16:0/3062009871 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f903c10c190 0x7f903c1014d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:41.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.566+0000 7f903a7fc640 1 -- 172.21.15.16:0/3062009871 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f903c102df0 msgr2=0x7f903c100a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:41.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.566+0000 7f903a7fc640 1 --2- 172.21.15.16:0/3062009871 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f903c102df0 0x7f903c100a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:41.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.566+0000 7f903a7fc640 1 -- 172.21.15.16:0/3062009871 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f903c1a68c0 con 0x7f903c103710 2024-09-12T02:12:41.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.566+0000 7f903affd640 1 --2- 172.21.15.16:0/3062009871 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f903c102df0 0x7f903c100a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:12:41.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.566+0000 7f903a7fc640 1 --2- 172.21.15.16:0/3062009871 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f903c103710 0x7f903c100f90 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f902400c890 tx=0x7f902400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:41.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.567+0000 7f9033fff640 1 -- 172.21.15.16:0/3062009871 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9024004220 con 0x7f903c103710 2024-09-12T02:12:41.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.567+0000 7f9041405640 1 -- 172.21.15.16:0/3062009871 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f903c1a6bb0 con 0x7f903c103710 2024-09-12T02:12:41.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.567+0000 7f9033fff640 1 -- 172.21.15.16:0/3062009871 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f90240043c0 con 0x7f903c103710 2024-09-12T02:12:41.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.567+0000 7f9033fff640 1 -- 172.21.15.16:0/3062009871 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90240049e0 con 0x7f903c103710 2024-09-12T02:12:41.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.567+0000 7f9041405640 1 -- 172.21.15.16:0/3062009871 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f903c1a70f0 con 0x7f903c103710 2024-09-12T02:12:41.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.569+0000 7f9033fff640 1 -- 172.21.15.16:0/3062009871 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f90240079b0 con 0x7f903c103710 2024-09-12T02:12:41.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.569+0000 7f90437de740 1 Processor -- start 2024-09-12T02:12:41.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.569+0000 7f90437de740 1 -- start start 2024-09-12T02:12:41.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.569+0000 7f90437de740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5589365bb4a0 0x5589365bb880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:41.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.569+0000 7f90437de740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5589365bbdc0 0x5589365c6140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:41.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.569+0000 7f90437de740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5589365c6680 0x5589365c8a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:41.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.570+0000 7f90437de740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5589365212e0 con 0x5589365bb4a0 2024-09-12T02:12:41.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.570+0000 7f90437de740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5589364c8ce0 con 0x5589365c6680 2024-09-12T02:12:41.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.570+0000 7f90437de740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55893651d2a0 con 0x5589365bbdc0 2024-09-12T02:12:41.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.570+0000 7f903b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5589365bb4a0 0x5589365bb880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:41.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.570+0000 7f903b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5589365bb4a0 0x5589365bb880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60756/0 (socket says 172.21.15.16:60756) 2024-09-12T02:12:41.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.570+0000 7f903b7fe640 1 -- 172.21.15.16:0/3944747802 learned_addr learned my addr 172.21.15.16:0/3944747802 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:12:41.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.570+0000 7f903a7fc640 1 --2- 172.21.15.16:0/3944747802 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5589365c6680 0x5589365c8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:41.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.570+0000 7f903affd640 1 --2- 172.21.15.16:0/3944747802 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5589365bbdc0 0x5589365c6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:41.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.570+0000 7f9033fff640 1 --2- 172.21.15.16:0/3062009871 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f90280a2b80 0x7f90280a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:41.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.570+0000 7f903b7fe640 1 -- 172.21.15.16:0/3944747802 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5589365bbdc0 msgr2=0x5589365c6140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:41.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.570+0000 7f903b7fe640 1 --2- 172.21.15.16:0/3944747802 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5589365bbdc0 0x5589365c6140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:41.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.570+0000 7f903b7fe640 1 -- 172.21.15.16:0/3944747802 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5589365c6680 msgr2=0x5589365c8a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:41.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.570+0000 7f903b7fe640 1 --2- 172.21.15.16:0/3944747802 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5589365c6680 0x5589365c8a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:41.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.570+0000 7f903b7fe640 1 -- 172.21.15.16:0/3944747802 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589364ad440 con 0x5589365bb4a0 2024-09-12T02:12:41.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.570+0000 7f903a7fc640 1 --2- 172.21.15.16:0/3944747802 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5589365c6680 0x5589365c8a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:12:41.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.570+0000 7f9033fff640 1 -- 172.21.15.16:0/3062009871 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f90240cefc0 con 0x7f903c103710 2024-09-12T02:12:41.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.570+0000 7f903b7fe640 1 --2- 172.21.15.16:0/3062009871 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f90280a2b80 0x7f90280a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:41.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.570+0000 7f903b7fe640 1 --2- 172.21.15.16:0/3944747802 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5589365bb4a0 0x5589365bb880 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f9034033000 tx=0x7f9034031680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1c66486c8cc9582 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:41.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.571+0000 7f90317fa640 1 -- 172.21.15.16:0/3944747802 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9034035040 con 0x5589365bb4a0 2024-09-12T02:12:41.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.571+0000 7f90317fa640 1 -- 172.21.15.16:0/3944747802 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9034003c10 con 0x5589365bb4a0 2024-09-12T02:12:41.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.571+0000 7f90317fa640 1 -- 172.21.15.16:0/3944747802 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9034046c30 con 0x5589365bb4a0 2024-09-12T02:12:41.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.571+0000 7f903b7fe640 1 --2- 172.21.15.16:0/3062009871 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f90280a2b80 0x7f90280a5040 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f903404b000 tx=0x7f9034048000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:41.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.571+0000 7f90437de740 1 -- 172.21.15.16:0/3944747802 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5589365bb4a0 msgr2=0x5589365bb880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:41.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.571+0000 7f90437de740 1 --2- 172.21.15.16:0/3944747802 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5589365bb4a0 0x5589365bb880 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f9034033000 tx=0x7f9034031680 comp rx=0 tx=0).stop 2024-09-12T02:12:41.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.571+0000 7f90437de740 1 -- 172.21.15.16:0/3944747802 shutdown_connections 2024-09-12T02:12:41.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.571+0000 7f90437de740 1 --2- 172.21.15.16:0/3944747802 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5589365c6680 0x5589365c8a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:41.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.571+0000 7f90437de740 1 --2- 172.21.15.16:0/3944747802 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5589365bbdc0 0x5589365c6140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:41.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.571+0000 7f90437de740 1 --2- 172.21.15.16:0/3944747802 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5589365bb4a0 0x5589365bb880 unknown :-1 s=CLOSED pgs=1494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:41.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.571+0000 7f90437de740 1 -- 172.21.15.16:0/3944747802 >> 172.21.15.16:0/3944747802 conn(0x5589365b7a30 msgr2=0x5589365b7e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:41.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.572+0000 7f90437de740 1 -- 172.21.15.16:0/3944747802 shutdown_connections 2024-09-12T02:12:41.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.572+0000 7f90437de740 1 -- 172.21.15.16:0/3944747802 wait complete. 2024-09-12T02:12:41.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.572+0000 7f90437de740 1 -- start start 2024-09-12T02:12:41.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.572+0000 7f90437de740 1 Processor -- start 2024-09-12T02:12:41.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.572+0000 7f90437de740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5589365bb4a0 0x5589365c6000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:41.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.572+0000 7f90437de740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5589365bbdc0 0x5589365c32b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:41.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.572+0000 7f90437de740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5589365c6680 0x5589365c3830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:41.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.572+0000 7f90437de740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5589365c9250 con 0x5589365c6680 2024-09-12T02:12:41.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.572+0000 7f90437de740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5589364dc050 con 0x5589365bbdc0 2024-09-12T02:12:41.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.572+0000 7f90437de740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5589365c93d0 con 0x5589365bb4a0 2024-09-12T02:12:41.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.572+0000 7f90437de740 10 client.0 _ll_register_callbacks cb 0x55893651c160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:12:41.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.572+0000 7f903b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5589365c6680 0x5589365c3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:41.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.572+0000 7f903b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5589365c6680 0x5589365c3830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60762/0 (socket says 172.21.15.16:60762) 2024-09-12T02:12:41.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.572+0000 7f903b7fe640 1 -- 172.21.15.16:0/3477037773 learned_addr learned my addr 172.21.15.16:0/3477037773 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:12:41.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.573+0000 7f903a7fc640 1 --2- 172.21.15.16:0/3477037773 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5589365bbdc0 0x5589365c32b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:41.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.573+0000 7f903affd640 1 --2- 172.21.15.16:0/3477037773 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5589365bb4a0 0x5589365c6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:41.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.573+0000 7f903b7fe640 1 -- 172.21.15.16:0/3477037773 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5589365bb4a0 msgr2=0x5589365c6000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:41.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.573+0000 7f903b7fe640 1 --2- 172.21.15.16:0/3477037773 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5589365bb4a0 0x5589365c6000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:41.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.573+0000 7f903b7fe640 1 -- 172.21.15.16:0/3477037773 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5589365bbdc0 msgr2=0x5589365c32b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:41.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.573+0000 7f903b7fe640 1 --2- 172.21.15.16:0/3477037773 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5589365bbdc0 0x5589365c32b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:41.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.573+0000 7f903b7fe640 1 -- 172.21.15.16:0/3477037773 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5589362923c0 con 0x5589365c6680 2024-09-12T02:12:41.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.573+0000 7f903b7fe640 1 --2- 172.21.15.16:0/3477037773 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5589365c6680 0x5589365c3830 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f903402a9a0 tx=0x7f9034046950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79478cfe2bd6c1b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:41.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.573+0000 7f90227fc640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:12:41.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.573+0000 7f90227fc640 1 -- 172.21.15.16:0/3477037773 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9034046e90 con 0x5589365c6680 2024-09-12T02:12:41.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.573+0000 7f90227fc640 1 -- 172.21.15.16:0/3477037773 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f903404d040 con 0x5589365c6680 2024-09-12T02:12:41.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.573+0000 7f90437de740 1 -- 172.21.15.16:0/3477037773 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9034003c10 con 0x5589365c6680 2024-09-12T02:12:41.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.573+0000 7f90227fc640 1 -- 172.21.15.16:0/3477037773 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9034031a20 con 0x5589365c6680 2024-09-12T02:12:41.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.573+0000 7f90437de740 10 client.18795 fetch_fsmap learned FSMap version 25 2024-09-12T02:12:41.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.573+0000 7f90437de740 1 -- 172.21.15.16:0/3477037773 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5589365c9250 con 0x5589365c6680 2024-09-12T02:12:41.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.573+0000 7f90227fc640 1 -- 172.21.15.16:0/3477037773 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9034003c10 con 0x5589365c6680 2024-09-12T02:12:41.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.574+0000 7f90227fc640 1 -- 172.21.15.16:0/3477037773 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f9034051a10 con 0x5589365c6680 2024-09-12T02:12:41.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.574+0000 7f90437de740 10 client.18795 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:12:41.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.574+0000 7f90437de740 10 client.18795 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:12:41.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.574+0000 7f90437de740 20 client.18795 populate_metadata read hostname 'smithi016' 2024-09-12T02:12:41.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.574+0000 7f90437de740 1 --2- 172.21.15.16:0/3477037773 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5589365d5dc0 0x5589365d81b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:41.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.574+0000 7f90437de740 4 client.18795 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:12:41.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.574+0000 7f90437de740 1 -- 172.21.15.16:0/3477037773 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55893652e850 con 0x5589365d5dc0 2024-09-12T02:12:41.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.574+0000 7f903affd640 1 --2- 172.21.15.16:0/3477037773 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5589365d5dc0 0x5589365d81b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:41.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.574+0000 7f90227fc640 10 client.18795 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:12:41.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.574+0000 7f903affd640 1 --2- 172.21.15.16:0/3477037773 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5589365d5dc0 0x5589365d81b0 crc :-1 s=READY pgs=316 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59df787ed543347f server_cookie=80ad4f6761bcf84e in_seq=0 out_seq=0 2024-09-12T02:12:41.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.575+0000 7f90227fc640 1 -- 172.21.15.16:0/3477037773 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55893652e850 con 0x5589365d5dc0 2024-09-12T02:12:41.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.575+0000 7f90227fc640 10 client.18795 handle_command_reply: tid=0 2024-09-12T02:12:41.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.575+0000 7f90437de740 1 client.18795 shutdown 2024-09-12T02:12:41.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.575+0000 7f90437de740 1 -- 172.21.15.16:0/3477037773 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5589365c6680 msgr2=0x5589365c3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:41.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.575+0000 7f90437de740 1 --2- 172.21.15.16:0/3477037773 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5589365c6680 0x5589365c3830 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f903402a9a0 tx=0x7f9034046950 comp rx=0 tx=0).stop 2024-09-12T02:12:41.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.575+0000 7f90437de740 1 -- 172.21.15.16:0/3477037773 shutdown_connections 2024-09-12T02:12:41.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.575+0000 7f90437de740 1 --2- 172.21.15.16:0/3477037773 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5589365d5dc0 0x5589365d81b0 crc :-1 s=READY pgs=316 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:41.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.575+0000 7f90437de740 1 --2- 172.21.15.16:0/3477037773 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5589365c6680 0x5589365c3830 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:41.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.575+0000 7f90227fc640 0 client.18795 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:12:41.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.575+0000 7f90437de740 1 --2- 172.21.15.16:0/3477037773 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5589365bbdc0 0x5589365c32b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:41.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.575+0000 7f90437de740 1 --2- 172.21.15.16:0/3477037773 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5589365bb4a0 0x5589365c6000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:41.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.575+0000 7f90437de740 1 -- 172.21.15.16:0/3477037773 >> 172.21.15.16:0/3477037773 conn(0x5589365b7a30 msgr2=0x5589365bd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:41.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.575+0000 7f90437de740 1 -- 172.21.15.16:0/3477037773 shutdown_connections 2024-09-12T02:12:41.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.575+0000 7f90437de740 1 -- 172.21.15.16:0/3477037773 wait complete. 2024-09-12T02:12:41.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.576+0000 7f90437de740 20 client.18795 trim_cache size 0 max 16384 2024-09-12T02:12:41.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.593+0000 7f90437de740 1 Processor -- start 2024-09-12T02:12:41.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.593+0000 7f90437de740 1 -- start start 2024-09-12T02:12:41.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.594+0000 7f90437de740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5589365bb4a0 0x5589365d24b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:41.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.594+0000 7f90437de740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5589365bbdc0 0x5589365be030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:41.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.594+0000 7f90437de740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5589365c6680 0x5589365dd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:41.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.594+0000 7f90437de740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f903c1b1ef0 con 0x5589365bbdc0 2024-09-12T02:12:41.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.594+0000 7f90437de740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f903c1af7c0 con 0x5589365c6680 2024-09-12T02:12:41.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.594+0000 7f90437de740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f903c1ad660 con 0x5589365bb4a0 2024-09-12T02:12:41.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.594+0000 7f903a7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5589365bbdc0 0x5589365be030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:41.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.594+0000 7f903a7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5589365bbdc0 0x5589365be030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60766/0 (socket says 172.21.15.16:60766) 2024-09-12T02:12:41.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.594+0000 7f903a7fc640 1 -- 172.21.15.16:0/993702005 learned_addr learned my addr 172.21.15.16:0/993702005 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:12:41.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.594+0000 7f903b7fe640 1 --2- 172.21.15.16:0/993702005 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5589365c6680 0x5589365dd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:41.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.594+0000 7f903affd640 1 --2- 172.21.15.16:0/993702005 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5589365bb4a0 0x5589365d24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:41.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.594+0000 7f903a7fc640 1 -- 172.21.15.16:0/993702005 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5589365bb4a0 msgr2=0x5589365d24b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:41.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.594+0000 7f903a7fc640 1 --2- 172.21.15.16:0/993702005 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5589365bb4a0 0x5589365d24b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:41.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.594+0000 7f903a7fc640 1 -- 172.21.15.16:0/993702005 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5589365c6680 msgr2=0x5589365dd8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:41.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.594+0000 7f903a7fc640 1 --2- 172.21.15.16:0/993702005 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5589365c6680 0x5589365dd8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:41.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.594+0000 7f903a7fc640 1 -- 172.21.15.16:0/993702005 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589365bbbf0 con 0x5589365bbdc0 2024-09-12T02:12:41.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.594+0000 7f903affd640 1 --2- 172.21.15.16:0/993702005 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5589365bb4a0 0x5589365d24b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:12:41.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.594+0000 7f903b7fe640 1 --2- 172.21.15.16:0/993702005 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5589365c6680 0x5589365dd8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:12:41.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.595+0000 7f903a7fc640 1 --2- 172.21.15.16:0/993702005 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5589365bbdc0 0x5589365be030 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f9024007590 tx=0x7f9024005300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f55d32599dcf3c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:41.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.595+0000 7f9030ff9640 1 -- 172.21.15.16:0/993702005 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f902409b690 con 0x5589365bbdc0 2024-09-12T02:12:41.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.595+0000 7f9030ff9640 1 -- 172.21.15.16:0/993702005 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9024084e50 con 0x5589365bbdc0 2024-09-12T02:12:41.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.595+0000 7f9030ff9640 1 -- 172.21.15.16:0/993702005 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9024007ca0 con 0x5589365bbdc0 2024-09-12T02:12:41.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.596+0000 7f90437de740 1 -- 172.21.15.16:0/993702005 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5589365bbdc0 msgr2=0x5589365be030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:41.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.596+0000 7f90437de740 1 --2- 172.21.15.16:0/993702005 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5589365bbdc0 0x5589365be030 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f9024007590 tx=0x7f9024005300 comp rx=0 tx=0).stop 2024-09-12T02:12:41.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.596+0000 7f90437de740 1 -- 172.21.15.16:0/993702005 shutdown_connections 2024-09-12T02:12:41.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.596+0000 7f90437de740 1 --2- 172.21.15.16:0/993702005 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5589365c6680 0x5589365dd8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:41.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.596+0000 7f90437de740 1 --2- 172.21.15.16:0/993702005 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5589365bbdc0 0x5589365be030 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:41.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.596+0000 7f90437de740 1 --2- 172.21.15.16:0/993702005 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5589365bb4a0 0x5589365d24b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:41.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.596+0000 7f90437de740 1 -- 172.21.15.16:0/993702005 >> 172.21.15.16:0/993702005 conn(0x5589365b7a30 msgr2=0x5589365c5330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:41.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.596+0000 7f90437de740 1 -- 172.21.15.16:0/993702005 shutdown_connections 2024-09-12T02:12:41.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.596+0000 7f90437de740 1 -- 172.21.15.16:0/993702005 wait complete. 2024-09-12T02:12:41.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.596+0000 7f90437de740 1 -- start start 2024-09-12T02:12:41.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.596+0000 7f90437de740 1 Processor -- start 2024-09-12T02:12:41.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.597+0000 7f90437de740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5589365bb4a0 0x5589365ce6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:41.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.597+0000 7f90437de740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5589365bbdc0 0x5589365cec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:41.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.597+0000 7f90437de740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5589365c6680 0x5589365cab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:41.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.597+0000 7f90437de740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f903c1adcd0 con 0x5589365c6680 2024-09-12T02:12:41.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.597+0000 7f90437de740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f903c1af0f0 con 0x5589365bb4a0 2024-09-12T02:12:41.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.597+0000 7f90437de740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f903c1ae350 con 0x5589365bbdc0 2024-09-12T02:12:41.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.597+0000 7f903b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5589365c6680 0x5589365cab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:41.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.597+0000 7f90437de740 10 client.0 _ll_register_callbacks cb 0x7f903404a020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:12:41.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.597+0000 7f903affd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5589365bb4a0 0x5589365ce6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:41.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.597+0000 7f903affd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5589365bb4a0 0x5589365ce6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.16:37278/0 (socket says 172.21.15.16:37278) 2024-09-12T02:12:41.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.597+0000 7f903affd640 1 -- 172.21.15.16:0/3861531538 learned_addr learned my addr 172.21.15.16:0/3861531538 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:12:41.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.597+0000 7f903a7fc640 1 --2- 172.21.15.16:0/3861531538 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5589365bbdc0 0x5589365cec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:41.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.597+0000 7f903affd640 1 -- 172.21.15.16:0/3861531538 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5589365bbdc0 msgr2=0x5589365cec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:41.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.597+0000 7f903affd640 1 --2- 172.21.15.16:0/3861531538 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5589365bbdc0 0x5589365cec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:41.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.597+0000 7f903affd640 1 -- 172.21.15.16:0/3861531538 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5589365c6680 msgr2=0x5589365cab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:41.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.597+0000 7f903affd640 1 --2- 172.21.15.16:0/3861531538 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5589365c6680 0x5589365cab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:41.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.597+0000 7f903affd640 1 -- 172.21.15.16:0/3861531538 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5589365c6370 con 0x5589365bb4a0 2024-09-12T02:12:41.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.598+0000 7f903a7fc640 1 --2- 172.21.15.16:0/3861531538 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5589365bbdc0 0x5589365cec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:12:41.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.598+0000 7f903affd640 1 --2- 172.21.15.16:0/3861531538 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5589365bb4a0 0x5589365ce6e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f902c0092f0 tx=0x7f902c005040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=821c1d237f0be7c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:41.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.598+0000 7f90227fc640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-12T02:12:41.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.598+0000 7f90227fc640 1 -- 172.21.15.16:0/3861531538 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f902c0059d0 con 0x5589365bb4a0 2024-09-12T02:12:41.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.598+0000 7f90437de740 1 -- 172.21.15.16:0/3861531538 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9024084e50 con 0x5589365bb4a0 2024-09-12T02:12:41.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.598+0000 7f90227fc640 1 -- 172.21.15.16:0/3861531538 <== mon.2 v2:172.21.15.33:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f902c013660 con 0x5589365bb4a0 2024-09-12T02:12:41.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.598+0000 7f90227fc640 1 -- 172.21.15.16:0/3861531538 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f902c013860 con 0x5589365bb4a0 2024-09-12T02:12:41.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.598+0000 7f90437de740 10 client.27866 fetch_fsmap learned FSMap version 25 2024-09-12T02:12:41.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.598+0000 7f90437de740 1 -- 172.21.15.16:0/3861531538 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x558936641260 con 0x5589365bb4a0 2024-09-12T02:12:41.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.598+0000 7f90227fc640 1 -- 172.21.15.16:0/3861531538 <== mon.2 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9024084e50 con 0x5589365bb4a0 2024-09-12T02:12:41.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.599+0000 7f90227fc640 1 -- 172.21.15.16:0/3861531538 <== mon.2 v2:172.21.15.33:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f902c022070 con 0x5589365bb4a0 2024-09-12T02:12:41.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.599+0000 7f90437de740 10 client.27866 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:12:41.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.599+0000 7f90437de740 10 client.27866 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:12:41.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.599+0000 7f90437de740 20 client.27866 populate_metadata read hostname 'smithi016' 2024-09-12T02:12:41.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.599+0000 7f90437de740 1 --2- 172.21.15.16:0/3861531538 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x558936641930 0x558936643d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:41.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.599+0000 7f90437de740 4 client.27866 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T02:12:41.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.599+0000 7f90437de740 1 -- 172.21.15.16:0/3861531538 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558936528d60 con 0x558936641930 2024-09-12T02:12:41.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.599+0000 7f903a7fc640 1 --2- 172.21.15.16:0/3861531538 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x558936641930 0x558936643d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:41.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.599+0000 7f903a7fc640 1 --2- 172.21.15.16:0/3861531538 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x558936641930 0x558936643d20 crc :-1 s=READY pgs=317 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f56fe4ff349cb1b server_cookie=1aa0006112721c00 in_seq=0 out_seq=0 2024-09-12T02:12:41.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.599+0000 7f90227fc640 10 client.27866 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:12:41.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.600+0000 7f90227fc640 1 -- 172.21.15.16:0/3861531538 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x558936528d60 con 0x558936641930 2024-09-12T02:12:41.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.600+0000 7f90227fc640 10 client.27866 handle_command_reply: tid=0 2024-09-12T02:12:41.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.600+0000 7f90437de740 1 client.27866 shutdown 2024-09-12T02:12:41.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.600+0000 7f90437de740 1 -- 172.21.15.16:0/3861531538 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5589365bb4a0 msgr2=0x5589365ce6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:41.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.600+0000 7f90437de740 1 --2- 172.21.15.16:0/3861531538 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5589365bb4a0 0x5589365ce6e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f902c0092f0 tx=0x7f902c005040 comp rx=0 tx=0).stop 2024-09-12T02:12:41.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.600+0000 7f90437de740 1 -- 172.21.15.16:0/3861531538 shutdown_connections 2024-09-12T02:12:41.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.600+0000 7f90437de740 1 --2- 172.21.15.16:0/3861531538 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x558936641930 0x558936643d20 crc :-1 s=READY pgs=317 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:41.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.600+0000 7f90437de740 1 --2- 172.21.15.16:0/3861531538 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5589365c6680 0x5589365cab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:41.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.600+0000 7f90437de740 1 --2- 172.21.15.16:0/3861531538 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5589365bbdc0 0x5589365cec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:41.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.600+0000 7f90437de740 1 --2- 172.21.15.16:0/3861531538 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5589365bb4a0 0x5589365ce6e0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:41.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.600+0000 7f90437de740 1 -- 172.21.15.16:0/3861531538 >> 172.21.15.16:0/3861531538 conn(0x5589365b7a30 msgr2=0x5589365c5330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:41.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.600+0000 7f90227fc640 0 client.27866 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:12:41.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.600+0000 7f90437de740 1 -- 172.21.15.16:0/3861531538 shutdown_connections 2024-09-12T02:12:41.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.600+0000 7f90437de740 1 -- 172.21.15.16:0/3861531538 wait complete. 2024-09-12T02:12:41.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.600+0000 7f90437de740 20 client.27866 trim_cache size 0 max 16384 2024-09-12T02:12:41.611 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T02:12:41.611 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-09-12T02:12:41.611 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {} 2024-09-12T02:12:41.611 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T02:12:41.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.601+0000 7f9041405640 1 -- 172.21.15.16:0/3062009871 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f90280a2b80 msgr2=0x7f90280a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:41.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.601+0000 7f9041405640 1 --2- 172.21.15.16:0/3062009871 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f90280a2b80 0x7f90280a5040 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f903404b000 tx=0x7f9034048000 comp rx=0 tx=0).stop 2024-09-12T02:12:41.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.601+0000 7f9041405640 1 -- 172.21.15.16:0/3062009871 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f903c103710 msgr2=0x7f903c100f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:41.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.601+0000 7f9041405640 1 --2- 172.21.15.16:0/3062009871 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f903c103710 0x7f903c100f90 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f902400c890 tx=0x7f902400cd50 comp rx=0 tx=0).stop 2024-09-12T02:12:41.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.601+0000 7f9041405640 1 -- 172.21.15.16:0/3062009871 shutdown_connections 2024-09-12T02:12:41.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.601+0000 7f9041405640 1 --2- 172.21.15.16:0/3062009871 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f90280a2b80 0x7f90280a5040 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:41.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.601+0000 7f9041405640 1 --2- 172.21.15.16:0/3062009871 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f903c10c190 0x7f903c1014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:41.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.601+0000 7f9041405640 1 --2- 172.21.15.16:0/3062009871 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f903c103710 0x7f903c100f90 unknown :-1 s=CLOSED pgs=1493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:41.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.601+0000 7f9041405640 1 --2- 172.21.15.16:0/3062009871 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f903c102df0 0x7f903c100a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:41.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.601+0000 7f9041405640 1 -- 172.21.15.16:0/3062009871 >> 172.21.15.16:0/3062009871 conn(0x7f903c0fc7d0 msgr2=0x7f903c0fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:41.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.601+0000 7f9041405640 1 -- 172.21.15.16:0/3062009871 shutdown_connections 2024-09-12T02:12:41.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:41.601+0000 7f9041405640 1 -- 172.21.15.16:0/3062009871 wait complete. 2024-09-12T02:12:41.679 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.676+0000 7f24edc27640 1 -- 172.21.15.16:0/3550642911 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24e8101c50 msgr2=0x7f24e81020d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:41.679 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.676+0000 7f24edc27640 1 --2- 172.21.15.16:0/3550642911 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24e8101c50 0x7f24e81020d0 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7f24dc009900 tx=0x7f24dc02a410 comp rx=0 tx=0).stop 2024-09-12T02:12:41.679 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.676+0000 7f24edc27640 1 -- 172.21.15.16:0/3550642911 shutdown_connections 2024-09-12T02:12:41.679 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.676+0000 7f24edc27640 1 --2- 172.21.15.16:0/3550642911 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f24e8102610 0x7f24e8108ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:41.679 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.676+0000 7f24edc27640 1 --2- 172.21.15.16:0/3550642911 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24e8101c50 0x7f24e81020d0 unknown :-1 s=CLOSED pgs=1497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:41.679 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.676+0000 7f24edc27640 1 --2- 172.21.15.16:0/3550642911 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f24e8100a60 0x7f24e8100e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:41.679 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.676+0000 7f24edc27640 1 -- 172.21.15.16:0/3550642911 >> 172.21.15.16:0/3550642911 conn(0x7f24e80fc1e0 msgr2=0x7f24e80fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:41.679 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.677+0000 7f24edc27640 1 -- 172.21.15.16:0/3550642911 shutdown_connections 2024-09-12T02:12:41.679 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.677+0000 7f24edc27640 1 -- 172.21.15.16:0/3550642911 wait complete. 2024-09-12T02:12:41.680 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.677+0000 7f24edc27640 1 Processor -- start 2024-09-12T02:12:41.680 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.677+0000 7f24edc27640 1 -- start start 2024-09-12T02:12:41.680 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.677+0000 7f24edc27640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24e8100a60 0x7f24e806ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:41.680 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.677+0000 7f24edc27640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f24e8101c50 0x7f24e819cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:41.680 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.677+0000 7f24edc27640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f24e8102610 0x7f24e819f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:41.680 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.677+0000 7f24edc27640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f24e810c710 con 0x7f24e8100a60 2024-09-12T02:12:41.680 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.677+0000 7f24edc27640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f24e810c590 con 0x7f24e8102610 2024-09-12T02:12:41.680 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.677+0000 7f24edc27640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f24e810c890 con 0x7f24e8101c50 2024-09-12T02:12:41.680 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.678+0000 7f24e77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24e8100a60 0x7f24e806ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:41.680 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.678+0000 7f24e77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24e8100a60 0x7f24e806ae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60788/0 (socket says 172.21.15.16:60788) 2024-09-12T02:12:41.680 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.678+0000 7f24e77fe640 1 -- 172.21.15.16:0/926823566 learned_addr learned my addr 172.21.15.16:0/926823566 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:12:41.680 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.678+0000 7f24e7fff640 1 --2- 172.21.15.16:0/926823566 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f24e8102610 0x7f24e819f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:41.681 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.678+0000 7f24e6ffd640 1 --2- 172.21.15.16:0/926823566 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f24e8101c50 0x7f24e819cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:41.681 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.678+0000 7f24e77fe640 1 -- 172.21.15.16:0/926823566 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f24e8101c50 msgr2=0x7f24e819cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:41.681 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.678+0000 7f24e77fe640 1 --2- 172.21.15.16:0/926823566 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f24e8101c50 0x7f24e819cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:41.681 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.678+0000 7f24e77fe640 1 -- 172.21.15.16:0/926823566 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f24e8102610 msgr2=0x7f24e819f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:41.681 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.678+0000 7f24e77fe640 1 --2- 172.21.15.16:0/926823566 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f24e8102610 0x7f24e819f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:41.681 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.678+0000 7f24e77fe640 1 -- 172.21.15.16:0/926823566 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24e819fa50 con 0x7f24e8100a60 2024-09-12T02:12:41.681 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.678+0000 7f24e77fe640 1 --2- 172.21.15.16:0/926823566 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24e8100a60 0x7f24e806ae70 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f24d800e9c0 tx=0x7f24d800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:41.681 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.678+0000 7f24e4ff9640 1 -- 172.21.15.16:0/926823566 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24d800cd80 con 0x7f24e8100a60 2024-09-12T02:12:41.681 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.678+0000 7f24e4ff9640 1 -- 172.21.15.16:0/926823566 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24d80035f0 con 0x7f24e8100a60 2024-09-12T02:12:41.681 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.678+0000 7f24edc27640 1 -- 172.21.15.16:0/926823566 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24e819fce0 con 0x7f24e8100a60 2024-09-12T02:12:41.681 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.678+0000 7f24e4ff9640 1 -- 172.21.15.16:0/926823566 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24d8010690 con 0x7f24e8100a60 2024-09-12T02:12:41.682 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.678+0000 7f24edc27640 1 -- 172.21.15.16:0/926823566 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24e81a01a0 con 0x7f24e8100a60 2024-09-12T02:12:41.689 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.681+0000 7f24e4ff9640 1 -- 172.21.15.16:0/926823566 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f24d80037a0 con 0x7f24e8100a60 2024-09-12T02:12:41.689 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.681+0000 7f24f09c1740 1 Processor -- start 2024-09-12T02:12:41.689 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.681+0000 7f24f09c1740 1 -- start start 2024-09-12T02:12:41.690 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.681+0000 7f24f09c1740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5621babdd9a0 0x5621babddda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:41.690 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.681+0000 7f24f09c1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5621babde2e0 0x5621babe2da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:41.690 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.681+0000 7f24f09c1740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5621babe3370 0x5621babe5830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:41.690 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.681+0000 7f24f09c1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5621bab412f0 con 0x5621babde2e0 2024-09-12T02:12:41.690 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.681+0000 7f24f09c1740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5621baae9270 con 0x5621babdd9a0 2024-09-12T02:12:41.690 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.681+0000 7f24f09c1740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5621bab3d2b0 con 0x5621babe3370 2024-09-12T02:12:41.690 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.681+0000 7f24e77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5621babde2e0 0x5621babe2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:41.690 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.681+0000 7f24e77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5621babde2e0 0x5621babe2da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60794/0 (socket says 172.21.15.16:60794) 2024-09-12T02:12:41.690 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.681+0000 7f24e77fe640 1 -- 172.21.15.16:0/2637399703 learned_addr learned my addr 172.21.15.16:0/2637399703 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:12:41.690 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.681+0000 7f24e6ffd640 1 --2- 172.21.15.16:0/2637399703 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5621babe3370 0x5621babe5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:41.690 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.681+0000 7f24e7fff640 1 --2- 172.21.15.16:0/2637399703 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5621babdd9a0 0x5621babddda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:41.690 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.682+0000 7f24e4ff9640 1 --2- 172.21.15.16:0/926823566 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f24d00a2cb0 0x7f24d00a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:41.691 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.682+0000 7f24e77fe640 1 -- 172.21.15.16:0/2637399703 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5621babe3370 msgr2=0x5621babe5830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:41.691 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.682+0000 7f24e77fe640 1 --2- 172.21.15.16:0/2637399703 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5621babe3370 0x5621babe5830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:41.691 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.682+0000 7f24e77fe640 1 -- 172.21.15.16:0/2637399703 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5621babdd9a0 msgr2=0x5621babddda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:41.691 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.682+0000 7f24e77fe640 1 --2- 172.21.15.16:0/2637399703 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5621babdd9a0 0x5621babddda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:41.691 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.682+0000 7f24e77fe640 1 -- 172.21.15.16:0/2637399703 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5621baae8340 con 0x5621babde2e0 2024-09-12T02:12:41.691 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.682+0000 7f24e7fff640 1 --2- 172.21.15.16:0/926823566 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f24d00a2cb0 0x7f24d00a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:41.691 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.682+0000 7f24e4ff9640 1 -- 172.21.15.16:0/926823566 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f24d80cf490 con 0x7f24e8100a60 2024-09-12T02:12:41.691 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.682+0000 7f24e77fe640 1 --2- 172.21.15.16:0/2637399703 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5621babde2e0 0x5621babe2da0 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f24d80ccf70 tx=0x7f24d8091270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e269d23e99307bf2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:41.691 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.682+0000 7f24cdffb640 1 -- 172.21.15.16:0/2637399703 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24d809b8d0 con 0x5621babde2e0 2024-09-12T02:12:41.691 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.682+0000 7f24cdffb640 1 -- 172.21.15.16:0/2637399703 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24d8094b90 con 0x5621babde2e0 2024-09-12T02:12:41.691 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.682+0000 7f24e7fff640 1 --2- 172.21.15.16:0/926823566 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f24d00a2cb0 0x7f24d00a5170 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f24e0005fd0 tx=0x7f24e0004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:41.691 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.682+0000 7f24cdffb640 1 -- 172.21.15.16:0/2637399703 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24d806bcb0 con 0x5621babde2e0 2024-09-12T02:12:41.691 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.683+0000 7f24f09c1740 1 -- 172.21.15.16:0/2637399703 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5621babde2e0 msgr2=0x5621babe2da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:41.692 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.683+0000 7f24f09c1740 1 --2- 172.21.15.16:0/2637399703 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5621babde2e0 0x5621babe2da0 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f24d80ccf70 tx=0x7f24d8091270 comp rx=0 tx=0).stop 2024-09-12T02:12:41.692 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.683+0000 7f24f09c1740 1 -- 172.21.15.16:0/2637399703 shutdown_connections 2024-09-12T02:12:41.692 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.683+0000 7f24f09c1740 1 --2- 172.21.15.16:0/2637399703 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5621babe3370 0x5621babe5830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:41.692 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.683+0000 7f24f09c1740 1 --2- 172.21.15.16:0/2637399703 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5621babde2e0 0x5621babe2da0 unknown :-1 s=CLOSED pgs=1499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:41.692 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.683+0000 7f24f09c1740 1 --2- 172.21.15.16:0/2637399703 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5621babdd9a0 0x5621babddda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:41.692 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.683+0000 7f24f09c1740 1 -- 172.21.15.16:0/2637399703 >> 172.21.15.16:0/2637399703 conn(0x5621babd9010 msgr2=0x5621babd9410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:41.692 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.683+0000 7f24f09c1740 1 -- 172.21.15.16:0/2637399703 shutdown_connections 2024-09-12T02:12:41.692 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.683+0000 7f24f09c1740 1 -- 172.21.15.16:0/2637399703 wait complete. 2024-09-12T02:12:41.692 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.683+0000 7f24f09c1740 1 -- start start 2024-09-12T02:12:41.692 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.684+0000 7f24f09c1740 1 Processor -- start 2024-09-12T02:12:41.692 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.684+0000 7f24f09c1740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5621babdd9a0 0x5621babe7af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:41.692 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.684+0000 7f24f09c1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5621babde2e0 0x5621babe8030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:41.693 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.684+0000 7f24f09c1740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5621babe3370 0x5621babef070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:41.693 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.684+0000 7f24f09c1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5621babe62e0 con 0x5621babde2e0 2024-09-12T02:12:41.693 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.684+0000 7f24f09c1740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5621baaca290 con 0x5621babe3370 2024-09-12T02:12:41.693 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.684+0000 7f24f09c1740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5621babe6460 con 0x5621babdd9a0 2024-09-12T02:12:41.693 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.684+0000 7f24f09c1740 10 client.0 _ll_register_callbacks cb 0x5621baafb140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:12:41.693 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.684+0000 7f24e6ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5621babde2e0 0x5621babe8030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:41.693 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.684+0000 7f24e6ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5621babde2e0 0x5621babe8030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60806/0 (socket says 172.21.15.16:60806) 2024-09-12T02:12:41.693 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.684+0000 7f24e6ffd640 1 -- 172.21.15.16:0/2165350954 learned_addr learned my addr 172.21.15.16:0/2165350954 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:12:41.693 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.684+0000 7f24e77fe640 1 --2- 172.21.15.16:0/2165350954 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5621babdd9a0 0x5621babe7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:41.693 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.684+0000 7f24e7fff640 1 --2- 172.21.15.16:0/2165350954 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5621babe3370 0x5621babef070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:41.693 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.684+0000 7f24e6ffd640 1 -- 172.21.15.16:0/2165350954 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5621babdd9a0 msgr2=0x5621babe7af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:41.693 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.684+0000 7f24e6ffd640 1 --2- 172.21.15.16:0/2165350954 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5621babdd9a0 0x5621babe7af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:41.693 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.684+0000 7f24e6ffd640 1 -- 172.21.15.16:0/2165350954 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5621babe3370 msgr2=0x5621babef070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:41.694 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.684+0000 7f24e6ffd640 1 --2- 172.21.15.16:0/2165350954 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5621babe3370 0x5621babef070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:41.694 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.684+0000 7f24e6ffd640 1 -- 172.21.15.16:0/2165350954 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5621ba854210 con 0x5621babde2e0 2024-09-12T02:12:41.694 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.684+0000 7f24e77fe640 1 --2- 172.21.15.16:0/2165350954 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5621babdd9a0 0x5621babe7af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:12:41.694 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.685+0000 7f24e6ffd640 1 --2- 172.21.15.16:0/2165350954 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5621babde2e0 0x5621babe8030 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f24dc005c40 tx=0x7f24dc005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23d2bf9532f5481d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:41.694 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.685+0000 7f24c6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:12:41.694 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.685+0000 7f24c6ffd640 1 -- 172.21.15.16:0/2165350954 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24dc035040 con 0x5621babde2e0 2024-09-12T02:12:41.694 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.685+0000 7f24c6ffd640 1 -- 172.21.15.16:0/2165350954 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f24dc0318b0 con 0x5621babde2e0 2024-09-12T02:12:41.694 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.685+0000 7f24f09c1740 1 -- 172.21.15.16:0/2165350954 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5621bab4ea20 con 0x5621babde2e0 2024-09-12T02:12:41.694 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.685+0000 7f24c6ffd640 1 -- 172.21.15.16:0/2165350954 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f24dc031ab0 con 0x5621babde2e0 2024-09-12T02:12:41.694 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.685+0000 7f24f09c1740 10 client.18813 fetch_fsmap learned FSMap version 25 2024-09-12T02:12:41.694 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.685+0000 7f24f09c1740 1 -- 172.21.15.16:0/2165350954 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5621ba920620 con 0x5621babde2e0 2024-09-12T02:12:41.694 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.685+0000 7f24c6ffd640 1 -- 172.21.15.16:0/2165350954 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5621bab4ea20 con 0x5621babde2e0 2024-09-12T02:12:41.694 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.685+0000 7f24c6ffd640 1 -- 172.21.15.16:0/2165350954 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f24dc0035f0 con 0x5621babde2e0 2024-09-12T02:12:41.695 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.685+0000 7f24f09c1740 10 client.18813 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:12:41.695 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.685+0000 7f24f09c1740 10 client.18813 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:12:41.695 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.685+0000 7f24f09c1740 20 client.18813 populate_metadata read hostname 'smithi016' 2024-09-12T02:12:41.695 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.685+0000 7f24f09c1740 1 --2- 172.21.15.16:0/2165350954 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5621babf3760 0x5621babf5b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:41.695 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.685+0000 7f24f09c1740 4 client.18813 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:12:41.695 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.685+0000 7f24f09c1740 1 -- 172.21.15.16:0/2165350954 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5621bab48d70 con 0x5621babf3760 2024-09-12T02:12:41.695 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.686+0000 7f24e77fe640 1 --2- 172.21.15.16:0/2165350954 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5621babf3760 0x5621babf5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:41.695 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.686+0000 7f24c6ffd640 10 client.18813 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:12:41.695 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.686+0000 7f24e77fe640 1 --2- 172.21.15.16:0/2165350954 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5621babf3760 0x5621babf5b90 crc :-1 s=READY pgs=318 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cee5730cb8360bd3 server_cookie=2c5ebba1cd6691fe in_seq=0 out_seq=0 2024-09-12T02:12:41.695 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.686+0000 7f24c6ffd640 1 -- 172.21.15.16:0/2165350954 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5621bab48d70 con 0x5621babf3760 2024-09-12T02:12:41.695 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.686+0000 7f24c6ffd640 10 client.18813 handle_command_reply: tid=0 2024-09-12T02:12:41.695 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.687+0000 7f24f09c1740 1 client.18813 shutdown 2024-09-12T02:12:41.695 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.687+0000 7f24f09c1740 1 -- 172.21.15.16:0/2165350954 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5621babde2e0 msgr2=0x5621babe8030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:41.696 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.687+0000 7f24f09c1740 1 --2- 172.21.15.16:0/2165350954 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5621babde2e0 0x5621babe8030 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f24dc005c40 tx=0x7f24dc005270 comp rx=0 tx=0).stop 2024-09-12T02:12:41.696 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.687+0000 7f24f09c1740 1 -- 172.21.15.16:0/2165350954 shutdown_connections 2024-09-12T02:12:41.696 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.687+0000 7f24f09c1740 1 --2- 172.21.15.16:0/2165350954 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5621babf3760 0x5621babf5b90 crc :-1 s=READY pgs=318 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:41.696 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.687+0000 7f24f09c1740 1 --2- 172.21.15.16:0/2165350954 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5621babe3370 0x5621babef070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:41.696 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.687+0000 7f24f09c1740 1 --2- 172.21.15.16:0/2165350954 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5621babde2e0 0x5621babe8030 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:41.696 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.687+0000 7f24f09c1740 1 --2- 172.21.15.16:0/2165350954 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5621babdd9a0 0x5621babe7af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:41.696 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.687+0000 7f24c6ffd640 0 client.18813 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:12:41.696 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.687+0000 7f24f09c1740 1 -- 172.21.15.16:0/2165350954 >> 172.21.15.16:0/2165350954 conn(0x5621babd9010 msgr2=0x5621babda820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:41.696 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.687+0000 7f24f09c1740 1 -- 172.21.15.16:0/2165350954 shutdown_connections 2024-09-12T02:12:41.696 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.687+0000 7f24f09c1740 1 -- 172.21.15.16:0/2165350954 wait complete. 2024-09-12T02:12:41.696 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.687+0000 7f24f09c1740 20 client.18813 trim_cache size 0 max 16384 2024-09-12T02:12:41.703 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.701+0000 7f24f09c1740 1 Processor -- start 2024-09-12T02:12:41.703 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.701+0000 7f24f09c1740 1 -- start start 2024-09-12T02:12:41.703 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.701+0000 7f24f09c1740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5621babdd9a0 0x5621babda400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:41.703 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.701+0000 7f24f09c1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5621babde2e0 0x5621babda940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:41.704 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.701+0000 7f24f09c1740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5621babe3370 0x5621babfa4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:41.704 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.701+0000 7f24f09c1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f24e81ab6a0 con 0x5621babde2e0 2024-09-12T02:12:41.704 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.701+0000 7f24f09c1740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f24e81a8f70 con 0x5621babdd9a0 2024-09-12T02:12:41.704 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.701+0000 7f24f09c1740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f24e81a6dc0 con 0x5621babe3370 2024-09-12T02:12:41.704 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.701+0000 7f24e6ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5621babde2e0 0x5621babda940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:41.704 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.701+0000 7f24e6ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5621babde2e0 0x5621babda940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60814/0 (socket says 172.21.15.16:60814) 2024-09-12T02:12:41.704 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.701+0000 7f24e6ffd640 1 -- 172.21.15.16:0/1517066959 learned_addr learned my addr 172.21.15.16:0/1517066959 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:12:41.704 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.702+0000 7f24e6ffd640 1 -- 172.21.15.16:0/1517066959 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5621babe3370 msgr2=0x5621babfa4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:41.704 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.702+0000 7f24e6ffd640 1 --2- 172.21.15.16:0/1517066959 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5621babe3370 0x5621babfa4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:41.705 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.702+0000 7f24e6ffd640 1 -- 172.21.15.16:0/1517066959 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5621babdd9a0 msgr2=0x5621babda400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:41.705 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.702+0000 7f24e6ffd640 1 --2- 172.21.15.16:0/1517066959 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5621babdd9a0 0x5621babda400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:41.705 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.702+0000 7f24e6ffd640 1 -- 172.21.15.16:0/1517066959 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5621babd8e30 con 0x5621babde2e0 2024-09-12T02:12:41.705 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.702+0000 7f24e6ffd640 1 --2- 172.21.15.16:0/1517066959 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5621babde2e0 0x5621babda940 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f24dc003a40 tx=0x7f24dc001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1276e2dc2cadabd server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:41.705 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.702+0000 7f24cd7fa640 1 -- 172.21.15.16:0/1517066959 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24dc035040 con 0x5621babde2e0 2024-09-12T02:12:41.705 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.702+0000 7f24cd7fa640 1 -- 172.21.15.16:0/1517066959 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24dc040070 con 0x5621babde2e0 2024-09-12T02:12:41.705 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.702+0000 7f24cd7fa640 1 -- 172.21.15.16:0/1517066959 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24dc004e00 con 0x5621babde2e0 2024-09-12T02:12:41.705 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.703+0000 7f24f09c1740 1 -- 172.21.15.16:0/1517066959 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5621babde2e0 msgr2=0x5621babda940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:41.705 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.703+0000 7f24f09c1740 1 --2- 172.21.15.16:0/1517066959 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5621babde2e0 0x5621babda940 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f24dc003a40 tx=0x7f24dc001e70 comp rx=0 tx=0).stop 2024-09-12T02:12:41.705 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.703+0000 7f24f09c1740 1 -- 172.21.15.16:0/1517066959 shutdown_connections 2024-09-12T02:12:41.706 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.703+0000 7f24f09c1740 1 --2- 172.21.15.16:0/1517066959 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5621babe3370 0x5621babfa4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:41.706 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.703+0000 7f24f09c1740 1 --2- 172.21.15.16:0/1517066959 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5621babde2e0 0x5621babda940 unknown :-1 s=CLOSED pgs=1501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:41.706 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.703+0000 7f24f09c1740 1 --2- 172.21.15.16:0/1517066959 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5621babdd9a0 0x5621babda400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:41.706 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.703+0000 7f24f09c1740 1 -- 172.21.15.16:0/1517066959 >> 172.21.15.16:0/1517066959 conn(0x5621babd9010 msgr2=0x5621babd81c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:41.706 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.703+0000 7f24f09c1740 1 -- 172.21.15.16:0/1517066959 shutdown_connections 2024-09-12T02:12:41.706 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.703+0000 7f24f09c1740 1 -- 172.21.15.16:0/1517066959 wait complete. 2024-09-12T02:12:41.706 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.703+0000 7f24f09c1740 1 -- start start 2024-09-12T02:12:41.709 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.703+0000 7f24f09c1740 1 Processor -- start 2024-09-12T02:12:41.709 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.704+0000 7f24f09c1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5621babdd9a0 0x5621bac59630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:41.709 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.704+0000 7f24f09c1740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5621babde2e0 0x5621bac59b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:41.709 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.704+0000 7f24f09c1740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5621babe3370 0x5621bac60bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:41.709 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.704+0000 7f24f09c1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f24e81a7430 con 0x5621babdd9a0 2024-09-12T02:12:41.710 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.704+0000 7f24f09c1740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f24e81a88a0 con 0x5621babe3370 2024-09-12T02:12:41.710 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.704+0000 7f24f09c1740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f24e81a7b00 con 0x5621babde2e0 2024-09-12T02:12:41.710 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.704+0000 7f24f09c1740 10 client.0 _ll_register_callbacks cb 0x7f24dc009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:12:41.710 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.704+0000 7f24e77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5621babdd9a0 0x5621bac59630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:41.710 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.704+0000 7f24e77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5621babdd9a0 0x5621bac59630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60824/0 (socket says 172.21.15.16:60824) 2024-09-12T02:12:41.710 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.704+0000 7f24e77fe640 1 -- 172.21.15.16:0/3893392863 learned_addr learned my addr 172.21.15.16:0/3893392863 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:12:41.710 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.704+0000 7f24e7fff640 1 --2- 172.21.15.16:0/3893392863 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5621babe3370 0x5621bac60bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:41.710 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.704+0000 7f24e6ffd640 1 --2- 172.21.15.16:0/3893392863 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5621babde2e0 0x5621bac59b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:41.710 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.704+0000 7f24e77fe640 1 -- 172.21.15.16:0/3893392863 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5621babde2e0 msgr2=0x5621bac59b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:41.710 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.704+0000 7f24e77fe640 1 --2- 172.21.15.16:0/3893392863 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5621babde2e0 0x5621bac59b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:41.710 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.704+0000 7f24e77fe640 1 -- 172.21.15.16:0/3893392863 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5621babe3370 msgr2=0x5621bac60bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:41.710 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.704+0000 7f24e77fe640 1 --2- 172.21.15.16:0/3893392863 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5621babe3370 0x5621bac60bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:41.710 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.704+0000 7f24e77fe640 1 -- 172.21.15.16:0/3893392863 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5621bac611c0 con 0x5621babdd9a0 2024-09-12T02:12:41.711 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.704+0000 7f24e7fff640 1 --2- 172.21.15.16:0/3893392863 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5621babe3370 0x5621bac60bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:12:41.711 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.704+0000 7f24e77fe640 1 --2- 172.21.15.16:0/3893392863 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5621babdd9a0 0x5621bac59630 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f24d80cd080 tx=0x7f24d809b9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0f51234cf2bc926 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:41.711 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.704+0000 7f24c6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:12:41.711 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.704+0000 7f24c6ffd640 1 -- 172.21.15.16:0/3893392863 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24d809b6f0 con 0x5621babdd9a0 2024-09-12T02:12:41.711 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.704+0000 7f24c6ffd640 1 -- 172.21.15.16:0/3893392863 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f24d8099e30 con 0x5621babdd9a0 2024-09-12T02:12:41.711 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.704+0000 7f24f09c1740 1 -- 172.21.15.16:0/3893392863 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24dc040070 con 0x5621babdd9a0 2024-09-12T02:12:41.711 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.704+0000 7f24c6ffd640 1 -- 172.21.15.16:0/3893392863 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f24d807f050 con 0x5621babdd9a0 2024-09-12T02:12:41.711 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.704+0000 7f24f09c1740 10 client.18819 fetch_fsmap learned FSMap version 25 2024-09-12T02:12:41.711 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.704+0000 7f24f09c1740 1 -- 172.21.15.16:0/3893392863 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5621bac625a0 con 0x5621babdd9a0 2024-09-12T02:12:41.711 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.705+0000 7f24c6ffd640 1 -- 172.21.15.16:0/3893392863 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24dc040070 con 0x5621babdd9a0 2024-09-12T02:12:41.711 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.705+0000 7f24c6ffd640 1 -- 172.21.15.16:0/3893392863 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f24d8097ae0 con 0x5621babdd9a0 2024-09-12T02:12:41.711 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.705+0000 7f24f09c1740 10 client.18819 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:12:41.712 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.705+0000 7f24f09c1740 10 client.18819 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:12:41.712 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.705+0000 7f24f09c1740 20 client.18819 populate_metadata read hostname 'smithi016' 2024-09-12T02:12:41.712 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.705+0000 7f24f09c1740 1 --2- 172.21.15.16:0/3893392863 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5621bac62b10 0x5621bac64f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:41.712 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.705+0000 7f24f09c1740 4 client.18819 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:12:41.712 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.705+0000 7f24f09c1740 1 -- 172.21.15.16:0/3893392863 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5621baac5900 con 0x5621bac62b10 2024-09-12T02:12:41.712 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.705+0000 7f24e6ffd640 1 --2- 172.21.15.16:0/3893392863 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5621bac62b10 0x5621bac64f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:41.712 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.706+0000 7f24c6ffd640 10 client.18819 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:12:41.712 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.706+0000 7f24e6ffd640 1 --2- 172.21.15.16:0/3893392863 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5621bac62b10 0x5621bac64f00 crc :-1 s=READY pgs=319 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=690c5649adba41be server_cookie=350e72d1baef606d in_seq=0 out_seq=0 2024-09-12T02:12:41.718 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.717+0000 7f24c6ffd640 1 -- 172.21.15.16:0/3893392863 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5621baac5900 con 0x5621bac62b10 2024-09-12T02:12:41.718 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.717+0000 7f24c6ffd640 10 client.18819 handle_command_reply: tid=0 2024-09-12T02:12:41.719 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.718+0000 7f24f09c1740 1 client.18819 shutdown 2024-09-12T02:12:41.719 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.718+0000 7f24f09c1740 1 -- 172.21.15.16:0/3893392863 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5621babdd9a0 msgr2=0x5621bac59630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:41.719 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.718+0000 7f24f09c1740 1 --2- 172.21.15.16:0/3893392863 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5621babdd9a0 0x5621bac59630 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f24d80cd080 tx=0x7f24d809b9f0 comp rx=0 tx=0).stop 2024-09-12T02:12:41.719 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.718+0000 7f24f09c1740 1 -- 172.21.15.16:0/3893392863 shutdown_connections 2024-09-12T02:12:41.719 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.718+0000 7f24f09c1740 1 --2- 172.21.15.16:0/3893392863 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5621bac62b10 0x5621bac64f00 crc :-1 s=READY pgs=319 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:41.719 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.718+0000 7f24f09c1740 1 --2- 172.21.15.16:0/3893392863 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5621babe3370 0x5621bac60bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:41.719 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.718+0000 7f24f09c1740 1 --2- 172.21.15.16:0/3893392863 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5621babde2e0 0x5621bac59b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:41.719 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.718+0000 7f24c6ffd640 0 client.18819 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:12:41.719 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.718+0000 7f24f09c1740 1 --2- 172.21.15.16:0/3893392863 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5621babdd9a0 0x5621bac59630 unknown :-1 s=CLOSED pgs=1502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:41.719 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.718+0000 7f24f09c1740 1 -- 172.21.15.16:0/3893392863 >> 172.21.15.16:0/3893392863 conn(0x5621babd9010 msgr2=0x5621babd93f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:41.719 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.718+0000 7f24f09c1740 1 -- 172.21.15.16:0/3893392863 shutdown_connections 2024-09-12T02:12:41.720 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.718+0000 7f24f09c1740 1 -- 172.21.15.16:0/3893392863 wait complete. 2024-09-12T02:12:41.720 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.718+0000 7f24f09c1740 20 client.18819 trim_cache size 0 max 16384 2024-09-12T02:12:41.720 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-09-12T02:12:41.720 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-09-12T02:12:41.720 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-09-12T02:12:41.720 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-09-12T02:12:41.721 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.718+0000 7f24edc27640 1 -- 172.21.15.16:0/926823566 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f24d00a2cb0 msgr2=0x7f24d00a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:41.721 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.718+0000 7f24edc27640 1 --2- 172.21.15.16:0/926823566 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f24d00a2cb0 0x7f24d00a5170 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f24e0005fd0 tx=0x7f24e0004a40 comp rx=0 tx=0).stop 2024-09-12T02:12:41.721 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.719+0000 7f24edc27640 1 -- 172.21.15.16:0/926823566 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24e8100a60 msgr2=0x7f24e806ae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:41.721 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.719+0000 7f24edc27640 1 --2- 172.21.15.16:0/926823566 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24e8100a60 0x7f24e806ae70 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f24d800e9c0 tx=0x7f24d800ee80 comp rx=0 tx=0).stop 2024-09-12T02:12:41.722 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.719+0000 7f24edc27640 1 -- 172.21.15.16:0/926823566 shutdown_connections 2024-09-12T02:12:41.722 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.719+0000 7f24edc27640 1 --2- 172.21.15.16:0/926823566 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f24d00a2cb0 0x7f24d00a5170 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:41.722 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.719+0000 7f24edc27640 1 --2- 172.21.15.16:0/926823566 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f24e8102610 0x7f24e819f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:41.722 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.719+0000 7f24edc27640 1 --2- 172.21.15.16:0/926823566 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f24e8101c50 0x7f24e819cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:41.722 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.719+0000 7f24edc27640 1 --2- 172.21.15.16:0/926823566 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24e8100a60 0x7f24e806ae70 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:41.722 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.719+0000 7f24edc27640 1 -- 172.21.15.16:0/926823566 >> 172.21.15.16:0/926823566 conn(0x7f24e80fc1e0 msgr2=0x7f24e8104f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:41.722 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.719+0000 7f24edc27640 1 -- 172.21.15.16:0/926823566 shutdown_connections 2024-09-12T02:12:41.722 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:41.719+0000 7f24edc27640 1 -- 172.21.15.16:0/926823566 wait complete. 2024-09-12T02:12:42.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:12:42 smithi016 ceph-mon[30947]: pgmap v658: 97 pgs: 97 active+clean; 1.5 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-09-12T02:12:42.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:12:42 smithi016 ceph-mon[30947]: scrub summary: idle 2024-09-12T02:12:42.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:12:42 smithi184 ceph-mon[35593]: pgmap v658: 97 pgs: 97 active+clean; 1.5 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-09-12T02:12:42.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:12:42 smithi184 ceph-mon[35593]: scrub summary: idle 2024-09-12T02:12:42.389 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-12T02:12:42.389 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:12:42.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:12:42 smithi033 ceph-mon[35332]: pgmap v658: 97 pgs: 97 active+clean; 1.5 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-09-12T02:12:42.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:12:42 smithi033 ceph-mon[35332]: scrub summary: idle 2024-09-12T02:12:42.647 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:12:43.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:43.138+0000 7fa4acea7640 1 -- 172.21.15.16:0/1078988304 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4a81082a0 msgr2=0x7fa4a810ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:43.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:43.138+0000 7fa4acea7640 1 --2- 172.21.15.16:0/1078988304 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4a81082a0 0x7fa4a810ec40 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7fa4a0009cd0 tx=0x7fa4a002a8e0 comp rx=0 tx=0).stop 2024-09-12T02:12:43.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:43.138+0000 7fa4acea7640 1 -- 172.21.15.16:0/1078988304 shutdown_connections 2024-09-12T02:12:43.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:43.138+0000 7fa4acea7640 1 --2- 172.21.15.16:0/1078988304 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4a81082a0 0x7fa4a810ec40 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:43.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:43.138+0000 7fa4acea7640 1 --2- 172.21.15.16:0/1078988304 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa4a81036e0 0x7fa4a8107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:43.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:43.138+0000 7fa4acea7640 1 --2- 172.21.15.16:0/1078988304 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa4a8102d30 0x7fa4a8103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:43.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:43.138+0000 7fa4acea7640 1 -- 172.21.15.16:0/1078988304 >> 172.21.15.16:0/1078988304 conn(0x7fa4a80fc7d0 msgr2=0x7fa4a80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:43.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:43.138+0000 7fa4acea7640 1 -- 172.21.15.16:0/1078988304 shutdown_connections 2024-09-12T02:12:43.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:43.138+0000 7fa4acea7640 1 -- 172.21.15.16:0/1078988304 wait complete. 2024-09-12T02:12:43.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:43.139+0000 7fa4acea7640 1 Processor -- start 2024-09-12T02:12:43.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:43.139+0000 7fa4acea7640 1 -- start start 2024-09-12T02:12:43.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:43.139+0000 7fa4acea7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4a8102d30 0x7fa4a8198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:43.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:43.139+0000 7fa4acea7640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa4a81036e0 0x7fa4a8198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:43.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:43.139+0000 7fa4acea7640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa4a81082a0 0x7fa4a819ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:43.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:43.139+0000 7fa4acea7640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa4a8112100 con 0x7fa4a8102d30 2024-09-12T02:12:43.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:43.139+0000 7fa4acea7640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa4a8111f80 con 0x7fa4a81082a0 2024-09-12T02:12:43.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:43.139+0000 7fa4acea7640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa4a8112280 con 0x7fa4a81036e0 2024-09-12T02:12:43.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:43.139+0000 7fa4a659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4a8102d30 0x7fa4a8198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:43.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:43.139+0000 7fa4a659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4a8102d30 0x7fa4a8198550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60842/0 (socket says 172.21.15.16:60842) 2024-09-12T02:12:43.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:43.139+0000 7fa4a659c640 1 -- 172.21.15.16:0/715084434 learned_addr learned my addr 172.21.15.16:0/715084434 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:12:43.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:43.139+0000 7fa4a659c640 1 -- 172.21.15.16:0/715084434 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa4a81036e0 msgr2=0x7fa4a8198a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:43.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:43.139+0000 7fa4a659c640 1 --2- 172.21.15.16:0/715084434 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa4a81036e0 0x7fa4a8198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:43.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:43.139+0000 7fa4a659c640 1 -- 172.21.15.16:0/715084434 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa4a81082a0 msgr2=0x7fa4a819ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:43.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:43.139+0000 7fa4a6d9d640 1 --2- 172.21.15.16:0/715084434 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa4a81082a0 0x7fa4a819ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:43.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:43.140+0000 7fa4a659c640 1 --2- 172.21.15.16:0/715084434 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa4a81082a0 0x7fa4a819ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:43.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:43.140+0000 7fa4a659c640 1 -- 172.21.15.16:0/715084434 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4a819d500 con 0x7fa4a8102d30 2024-09-12T02:12:43.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:43.140+0000 7fa4a659c640 1 --2- 172.21.15.16:0/715084434 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4a8102d30 0x7fa4a8198550 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7fa49800b4a0 tx=0x7fa49800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:43.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:43.140+0000 7fa4977fe640 1 -- 172.21.15.16:0/715084434 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa498014070 con 0x7fa4a8102d30 2024-09-12T02:12:43.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:43.140+0000 7fa4977fe640 1 -- 172.21.15.16:0/715084434 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa498010af0 con 0x7fa4a8102d30 2024-09-12T02:12:43.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:43.140+0000 7fa4acea7640 1 -- 172.21.15.16:0/715084434 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4a819d7f0 con 0x7fa4a8102d30 2024-09-12T02:12:43.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:43.140+0000 7fa4977fe640 1 -- 172.21.15.16:0/715084434 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa498017d10 con 0x7fa4a8102d30 2024-09-12T02:12:43.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:43.140+0000 7fa4acea7640 1 -- 172.21.15.16:0/715084434 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4a81a50d0 con 0x7fa4a8102d30 2024-09-12T02:12:43.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:43.141+0000 7fa4acea7640 1 -- 172.21.15.16:0/715084434 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4a806a850 con 0x7fa4a8102d30 2024-09-12T02:12:43.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:43.142+0000 7fa4977fe640 1 -- 172.21.15.16:0/715084434 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa498017070 con 0x7fa4a8102d30 2024-09-12T02:12:43.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:43.143+0000 7fa4977fe640 1 --2- 172.21.15.16:0/715084434 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fa4900a2b80 0x7fa4900a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:43.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:43.143+0000 7fa4a5d9b640 1 --2- 172.21.15.16:0/715084434 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fa4900a2b80 0x7fa4900a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:43.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:43.143+0000 7fa4977fe640 1 -- 172.21.15.16:0/715084434 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fa4980ce530 con 0x7fa4a8102d30 2024-09-12T02:12:43.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:43.144+0000 7fa4a5d9b640 1 --2- 172.21.15.16:0/715084434 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fa4900a2b80 0x7fa4900a5040 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fa4a80fed20 tx=0x7fa49c005c70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:43.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:43.147+0000 7fa4977fe640 1 -- 172.21.15.16:0/715084434 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa49809b490 con 0x7fa4a8102d30 2024-09-12T02:12:43.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:43.361+0000 7fa4acea7640 1 -- 172.21.15.16:0/715084434 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa4a8103110 con 0x7fa4a8102d30 2024-09-12T02:12:43.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:43.362+0000 7fa4977fe640 1 -- 172.21.15.16:0/715084434 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11911 (secure 0 0 0) 0x7fa49809ad90 con 0x7fa4a8102d30 2024-09-12T02:12:43.362 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T02:12:43.362 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":25,"btime":"2024-09-12T02:01:37:192021+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3279123279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3279123279},{"type":"v1","addr":"172.21.15.16:6835","nonce":3279123279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4119452382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4119452382},{"type":"v1","addr":"172.21.15.16:6837","nonce":4119452382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/3263823165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3263823165},{"type":"v1","addr":"172.21.15.16:6839","nonce":3263823165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/1510290732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":1510290732},{"type":"v1","addr":"172.21.15.16:6841","nonce":1510290732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/279674603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":279674603},{"type":"v1","addr":"172.21.15.33:6833","nonce":279674603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3555728028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3555728028},{"type":"v1","addr":"172.21.15.33:6835","nonce":3555728028}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T01:59:56.117608+0000","modified":"2024-09-12T02:01:36.294760+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24482,"mds_1":24481,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6833/1563374106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1563374106},{"type":"v1","addr":"172.21.15.184:6833","nonce":1563374106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6837/1892833868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1892833868},{"type":"v1","addr":"172.21.15.33:6837","nonce":1892833868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/1457679686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1457679686},{"type":"v1","addr":"172.21.15.184:6835","nonce":1457679686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.184:6837/1793393087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":1793393087},{"type":"v1","addr":"172.21.15.184:6837","nonce":1793393087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.184:6839/4203347941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":4203347941},{"type":"v1","addr":"172.21.15.184:6839","nonce":4203347941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.33:6839/4136670204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":4136670204},{"type":"v1","addr":"172.21.15.33:6839","nonce":4136670204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24481,24482,24469]},"id":1}]} 2024-09-12T02:12:43.362 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 25 2024-09-12T02:12:43.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:43.364+0000 7fa4acea7640 1 -- 172.21.15.16:0/715084434 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fa4900a2b80 msgr2=0x7fa4900a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:43.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:43.364+0000 7fa4acea7640 1 --2- 172.21.15.16:0/715084434 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fa4900a2b80 0x7fa4900a5040 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fa4a80fed20 tx=0x7fa49c005c70 comp rx=0 tx=0).stop 2024-09-12T02:12:43.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:43.364+0000 7fa4acea7640 1 -- 172.21.15.16:0/715084434 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4a8102d30 msgr2=0x7fa4a8198550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:43.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:43.364+0000 7fa4acea7640 1 --2- 172.21.15.16:0/715084434 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4a8102d30 0x7fa4a8198550 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7fa49800b4a0 tx=0x7fa49800b960 comp rx=0 tx=0).stop 2024-09-12T02:12:43.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:43.364+0000 7fa4acea7640 1 -- 172.21.15.16:0/715084434 shutdown_connections 2024-09-12T02:12:43.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:43.364+0000 7fa4acea7640 1 --2- 172.21.15.16:0/715084434 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fa4900a2b80 0x7fa4900a5040 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:43.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:43.364+0000 7fa4acea7640 1 --2- 172.21.15.16:0/715084434 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa4a81082a0 0x7fa4a819ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:43.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:43.364+0000 7fa4acea7640 1 --2- 172.21.15.16:0/715084434 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa4a81036e0 0x7fa4a8198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:43.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:43.364+0000 7fa4acea7640 1 --2- 172.21.15.16:0/715084434 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4a8102d30 0x7fa4a8198550 unknown :-1 s=CLOSED pgs=1504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:43.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:43.364+0000 7fa4acea7640 1 -- 172.21.15.16:0/715084434 >> 172.21.15.16:0/715084434 conn(0x7fa4a80fc7d0 msgr2=0x7fa4a8106380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:43.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:43.364+0000 7fa4acea7640 1 -- 172.21.15.16:0/715084434 shutdown_connections 2024-09-12T02:12:43.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:43.364+0000 7fa4acea7640 1 -- 172.21.15.16:0/715084434 wait complete. 2024-09-12T02:12:43.998 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-12T02:12:44.257 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:12:44.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:12:44 smithi184 ceph-mon[35593]: pgmap v659: 97 pgs: 97 active+clean; 1.5 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 14 op/s 2024-09-12T02:12:44.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:12:44 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/715084434' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:12:44.378 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:12:44 smithi016 ceph-mon[30947]: pgmap v659: 97 pgs: 97 active+clean; 1.5 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 14 op/s 2024-09-12T02:12:44.378 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:12:44 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/715084434' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:12:44.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:12:44 smithi033 ceph-mon[35332]: pgmap v659: 97 pgs: 97 active+clean; 1.5 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 14 op/s 2024-09-12T02:12:44.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:12:44 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/715084434' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:12:44.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.679+0000 7fcaa17e4640 1 -- 172.21.15.16:0/3093834851 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca9c1082a0 msgr2=0x7fca9c10ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:44.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.680+0000 7fcaa17e4640 1 --2- 172.21.15.16:0/3093834851 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca9c1082a0 0x7fca9c10ec40 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7fca94009930 tx=0x7fca9402a420 comp rx=0 tx=0).stop 2024-09-12T02:12:44.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.680+0000 7fcaa17e4640 1 -- 172.21.15.16:0/3093834851 shutdown_connections 2024-09-12T02:12:44.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.680+0000 7fcaa17e4640 1 --2- 172.21.15.16:0/3093834851 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca9c1082a0 0x7fca9c10ec40 unknown :-1 s=CLOSED pgs=1505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:44.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.680+0000 7fcaa17e4640 1 --2- 172.21.15.16:0/3093834851 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fca9c1036e0 0x7fca9c107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:44.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.680+0000 7fcaa17e4640 1 --2- 172.21.15.16:0/3093834851 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca9c102d30 0x7fca9c103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:44.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.680+0000 7fcaa17e4640 1 -- 172.21.15.16:0/3093834851 >> 172.21.15.16:0/3093834851 conn(0x7fca9c0fc7d0 msgr2=0x7fca9c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:44.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.680+0000 7fcaa17e4640 1 -- 172.21.15.16:0/3093834851 shutdown_connections 2024-09-12T02:12:44.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.681+0000 7fcaa17e4640 1 -- 172.21.15.16:0/3093834851 wait complete. 2024-09-12T02:12:44.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.681+0000 7fcaa17e4640 1 Processor -- start 2024-09-12T02:12:44.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.681+0000 7fcaa17e4640 1 -- start start 2024-09-12T02:12:44.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.681+0000 7fcaa17e4640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fca9c102d30 0x7fca9c198540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:44.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.681+0000 7fcaa17e4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca9c1036e0 0x7fca9c198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:44.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.681+0000 7fcaa17e4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca9c1082a0 0x7fca9c19ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:44.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.682+0000 7fcaa17e4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fca9c1120a0 con 0x7fca9c1082a0 2024-09-12T02:12:44.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.682+0000 7fcaa17e4640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fca9c111f20 con 0x7fca9c1036e0 2024-09-12T02:12:44.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.682+0000 7fcaa17e4640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fca9c112220 con 0x7fca9c102d30 2024-09-12T02:12:44.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.682+0000 7fca9b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca9c1082a0 0x7fca9c19ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:44.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.682+0000 7fca9b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca9c1082a0 0x7fca9c19ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60862/0 (socket says 172.21.15.16:60862) 2024-09-12T02:12:44.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.682+0000 7fca9b7fe640 1 -- 172.21.15.16:0/207058389 learned_addr learned my addr 172.21.15.16:0/207058389 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:12:44.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.682+0000 7fca9affd640 1 --2- 172.21.15.16:0/207058389 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fca9c102d30 0x7fca9c198540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:44.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.682+0000 7fca9a7fc640 1 --2- 172.21.15.16:0/207058389 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca9c1036e0 0x7fca9c198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:44.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.682+0000 7fca9b7fe640 1 -- 172.21.15.16:0/207058389 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fca9c102d30 msgr2=0x7fca9c198540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:44.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.682+0000 7fca9b7fe640 1 --2- 172.21.15.16:0/207058389 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fca9c102d30 0x7fca9c198540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:44.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.682+0000 7fca9b7fe640 1 -- 172.21.15.16:0/207058389 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca9c1036e0 msgr2=0x7fca9c198a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:44.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.682+0000 7fca9b7fe640 1 --2- 172.21.15.16:0/207058389 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca9c1036e0 0x7fca9c198a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:44.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.682+0000 7fca9b7fe640 1 -- 172.21.15.16:0/207058389 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca9c19d4f0 con 0x7fca9c1082a0 2024-09-12T02:12:44.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.682+0000 7fca9b7fe640 1 --2- 172.21.15.16:0/207058389 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca9c1082a0 0x7fca9c19ce10 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7fca94005ab0 tx=0x7fca940042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:44.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.682+0000 7fca83fff640 1 -- 172.21.15.16:0/207058389 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca9403f030 con 0x7fca9c1082a0 2024-09-12T02:12:44.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.682+0000 7fcaa17e4640 1 -- 172.21.15.16:0/207058389 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca9c19d780 con 0x7fca9c1082a0 2024-09-12T02:12:44.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.682+0000 7fca83fff640 1 -- 172.21.15.16:0/207058389 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca94003ce0 con 0x7fca9c1082a0 2024-09-12T02:12:44.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.682+0000 7fca83fff640 1 -- 172.21.15.16:0/207058389 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca94035070 con 0x7fca9c1082a0 2024-09-12T02:12:44.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.682+0000 7fcaa17e4640 1 -- 172.21.15.16:0/207058389 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca9c1a50c0 con 0x7fca9c1082a0 2024-09-12T02:12:44.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.685+0000 7fca83fff640 1 -- 172.21.15.16:0/207058389 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fca94042030 con 0x7fca9c1082a0 2024-09-12T02:12:44.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.685+0000 7fcaa3bbd740 1 Processor -- start 2024-09-12T02:12:44.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.685+0000 7fcaa3bbd740 1 -- start start 2024-09-12T02:12:44.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.685+0000 7fcaa3bbd740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c0f6c32b40 0x55c0f6c31090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:44.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.685+0000 7fcaa3bbd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c0f6c315d0 0x55c0f6c31a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:44.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.685+0000 7fcaa3bbd740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c0f6c3c370 0x55c0f6c3e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:44.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.685+0000 7fcaa3bbd740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55c0f6b972f0 con 0x55c0f6c32b40 2024-09-12T02:12:44.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.685+0000 7fcaa3bbd740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55c0f6b3ecf0 con 0x55c0f6c315d0 2024-09-12T02:12:44.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.685+0000 7fcaa3bbd740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55c0f6b932b0 con 0x55c0f6c3c370 2024-09-12T02:12:44.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.685+0000 7fca9a7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c0f6c32b40 0x55c0f6c31090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:44.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.685+0000 7fca9a7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c0f6c32b40 0x55c0f6c31090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60864/0 (socket says 172.21.15.16:60864) 2024-09-12T02:12:44.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.685+0000 7fca9a7fc640 1 -- 172.21.15.16:0/1332979949 learned_addr learned my addr 172.21.15.16:0/1332979949 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:12:44.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.685+0000 7fca9affd640 1 --2- 172.21.15.16:0/1332979949 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c0f6c315d0 0x55c0f6c31a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:44.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.686+0000 7fca83fff640 1 --2- 172.21.15.16:0/207058389 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fca840a2b80 0x7fca840a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:44.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.686+0000 7fca9a7fc640 1 -- 172.21.15.16:0/1332979949 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c0f6c3c370 msgr2=0x55c0f6c3e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:44.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.686+0000 7fca9a7fc640 1 --2- 172.21.15.16:0/1332979949 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c0f6c3c370 0x55c0f6c3e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:44.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.686+0000 7fca9a7fc640 1 -- 172.21.15.16:0/1332979949 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c0f6c315d0 msgr2=0x55c0f6c31a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:44.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.686+0000 7fca9a7fc640 1 --2- 172.21.15.16:0/1332979949 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c0f6c315d0 0x55c0f6c31a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:44.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.686+0000 7fca9a7fc640 1 -- 172.21.15.16:0/1332979949 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0f6b233a0 con 0x55c0f6c32b40 2024-09-12T02:12:44.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.686+0000 7fca9b7fe640 1 --2- 172.21.15.16:0/207058389 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fca840a2b80 0x7fca840a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:44.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.686+0000 7fca9a7fc640 1 --2- 172.21.15.16:0/1332979949 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c0f6c32b40 0x55c0f6c31090 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7fca9000a300 tx=0x7fca900117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bb29c2b878c1732 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:44.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.686+0000 7fca83fff640 1 -- 172.21.15.16:0/207058389 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fca940ef390 con 0x7fca9c1082a0 2024-09-12T02:12:44.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.686+0000 7fca817fa640 1 -- 172.21.15.16:0/1332979949 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca90004d20 con 0x55c0f6c32b40 2024-09-12T02:12:44.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.686+0000 7fca817fa640 1 -- 172.21.15.16:0/1332979949 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca90005500 con 0x55c0f6c32b40 2024-09-12T02:12:44.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.686+0000 7fca817fa640 1 -- 172.21.15.16:0/1332979949 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca90012830 con 0x55c0f6c32b40 2024-09-12T02:12:44.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.687+0000 7fcaa3bbd740 1 -- 172.21.15.16:0/1332979949 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c0f6c32b40 msgr2=0x55c0f6c31090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:44.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.687+0000 7fca9b7fe640 1 --2- 172.21.15.16:0/207058389 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fca840a2b80 0x7fca840a5040 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fca940bbcd0 tx=0x7fca94009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:44.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.687+0000 7fcaa3bbd740 1 --2- 172.21.15.16:0/1332979949 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c0f6c32b40 0x55c0f6c31090 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7fca9000a300 tx=0x7fca900117b0 comp rx=0 tx=0).stop 2024-09-12T02:12:44.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.687+0000 7fcaa3bbd740 1 -- 172.21.15.16:0/1332979949 shutdown_connections 2024-09-12T02:12:44.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.687+0000 7fcaa3bbd740 1 --2- 172.21.15.16:0/1332979949 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c0f6c3c370 0x55c0f6c3e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:44.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.687+0000 7fcaa3bbd740 1 --2- 172.21.15.16:0/1332979949 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c0f6c315d0 0x55c0f6c31a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:44.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.687+0000 7fcaa3bbd740 1 --2- 172.21.15.16:0/1332979949 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c0f6c32b40 0x55c0f6c31090 unknown :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:44.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.687+0000 7fcaa3bbd740 1 -- 172.21.15.16:0/1332979949 >> 172.21.15.16:0/1332979949 conn(0x55c0f6bfa310 msgr2=0x55c0f6c2dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:44.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.687+0000 7fcaa3bbd740 1 -- 172.21.15.16:0/1332979949 shutdown_connections 2024-09-12T02:12:44.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.687+0000 7fcaa3bbd740 1 -- 172.21.15.16:0/1332979949 wait complete. 2024-09-12T02:12:44.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.687+0000 7fcaa3bbd740 1 -- start start 2024-09-12T02:12:44.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.687+0000 7fcaa3bbd740 1 Processor -- start 2024-09-12T02:12:44.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.688+0000 7fcaa3bbd740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c0f6c315d0 0x55c0f6c3bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:44.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.688+0000 7fcaa3bbd740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c0f6c32b40 0x55c0f6c38fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:44.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.688+0000 7fcaa3bbd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c0f6c3c370 0x55c0f6c395d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:44.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.688+0000 7fcaa3bbd740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55c0f6c3f070 con 0x55c0f6c315d0 2024-09-12T02:12:44.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.688+0000 7fcaa3bbd740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55c0f6b52060 con 0x55c0f6c3c370 2024-09-12T02:12:44.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.688+0000 7fcaa3bbd740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55c0f6c3f1f0 con 0x55c0f6c32b40 2024-09-12T02:12:44.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.688+0000 7fcaa3bbd740 10 client.0 _ll_register_callbacks cb 0x55c0f68a58c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:12:44.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.688+0000 7fca9a7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c0f6c315d0 0x55c0f6c3bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:44.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.688+0000 7fca9a7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c0f6c315d0 0x55c0f6c3bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60872/0 (socket says 172.21.15.16:60872) 2024-09-12T02:12:44.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.688+0000 7fca9a7fc640 1 -- 172.21.15.16:0/569490011 learned_addr learned my addr 172.21.15.16:0/569490011 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:12:44.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.688+0000 7fca9affd640 1 --2- 172.21.15.16:0/569490011 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c0f6c32b40 0x55c0f6c38fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:44.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.688+0000 7fca9a7fc640 1 --2- 172.21.15.16:0/569490011 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c0f6c3c370 0x55c0f6c395d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:44.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.688+0000 7fca9a7fc640 1 -- 172.21.15.16:0/569490011 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c0f6c32b40 msgr2=0x55c0f6c38fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:44.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.688+0000 7fca9a7fc640 1 --2- 172.21.15.16:0/569490011 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c0f6c32b40 0x55c0f6c38fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:44.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.688+0000 7fca9a7fc640 1 -- 172.21.15.16:0/569490011 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c0f6c3c370 msgr2=0x55c0f6c395d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:44.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.688+0000 7fca9a7fc640 1 --2- 172.21.15.16:0/569490011 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c0f6c3c370 0x55c0f6c395d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:44.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.688+0000 7fca9a7fc640 1 -- 172.21.15.16:0/569490011 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c0f69083c0 con 0x55c0f6c315d0 2024-09-12T02:12:44.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.688+0000 7fca9a7fc640 1 --2- 172.21.15.16:0/569490011 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c0f6c315d0 0x55c0f6c3bd30 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7fca9000acf0 tx=0x7fca900124f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=272d368875fa4871 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:44.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.688+0000 7fca7eff5640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:12:44.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.689+0000 7fca7eff5640 1 -- 172.21.15.16:0/569490011 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca900125d0 con 0x55c0f6c315d0 2024-09-12T02:12:44.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.689+0000 7fca7eff5640 1 -- 172.21.15.16:0/569490011 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fca90012770 con 0x55c0f6c315d0 2024-09-12T02:12:44.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.689+0000 7fcaa3bbd740 1 -- 172.21.15.16:0/569490011 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca90005500 con 0x55c0f6c315d0 2024-09-12T02:12:44.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.689+0000 7fca7eff5640 1 -- 172.21.15.16:0/569490011 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fca90012970 con 0x55c0f6c315d0 2024-09-12T02:12:44.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.689+0000 7fcaa3bbd740 10 client.18837 fetch_fsmap learned FSMap version 25 2024-09-12T02:12:44.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.689+0000 7fcaa3bbd740 1 -- 172.21.15.16:0/569490011 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c0f6c4aad0 con 0x55c0f6c315d0 2024-09-12T02:12:44.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.689+0000 7fca7eff5640 1 -- 172.21.15.16:0/569490011 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca90005500 con 0x55c0f6c315d0 2024-09-12T02:12:44.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.689+0000 7fca7eff5640 1 -- 172.21.15.16:0/569490011 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fca9002aa40 con 0x55c0f6c315d0 2024-09-12T02:12:44.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.689+0000 7fcaa3bbd740 10 client.18837 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:12:44.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.689+0000 7fcaa3bbd740 10 client.18837 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-12T02:12:44.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.689+0000 7fcaa3bbd740 20 client.18837 populate_metadata read hostname 'smithi016' 2024-09-12T02:12:44.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.689+0000 7fcaa3bbd740 1 --2- 172.21.15.16:0/569490011 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55c0f6c4be80 0x55c0f6c4e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:44.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.689+0000 7fcaa3bbd740 4 client.18837 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:12:44.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.689+0000 7fcaa3bbd740 1 -- 172.21.15.16:0/569490011 --> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c0f6ba4860 con 0x55c0f6c4be80 2024-09-12T02:12:44.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.690+0000 7fca9affd640 1 --2- 172.21.15.16:0/569490011 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55c0f6c4be80 0x55c0f6c4e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:44.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.690+0000 7fca9affd640 1 --2- 172.21.15.16:0/569490011 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55c0f6c4be80 0x55c0f6c4e270 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c33bcff40ab75148 server_cookie=57442aaa9b8af4e5 in_seq=0 out_seq=0 2024-09-12T02:12:44.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.690+0000 7fca7eff5640 10 client.18837 ms_handle_connect on v2:172.21.15.184:6836/1793393087 2024-09-12T02:12:44.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.691+0000 7fca7eff5640 1 -- 172.21.15.16:0/569490011 <== mds.2 v2:172.21.15.184:6836/1793393087 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c0f6ba4860 con 0x55c0f6c4be80 2024-09-12T02:12:44.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.691+0000 7fca7eff5640 10 client.18837 handle_command_reply: tid=0 2024-09-12T02:12:44.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.691+0000 7fcaa3bbd740 1 client.18837 shutdown 2024-09-12T02:12:44.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.691+0000 7fcaa3bbd740 1 -- 172.21.15.16:0/569490011 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c0f6c315d0 msgr2=0x55c0f6c3bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:44.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.691+0000 7fcaa3bbd740 1 --2- 172.21.15.16:0/569490011 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c0f6c315d0 0x55c0f6c3bd30 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7fca9000acf0 tx=0x7fca900124f0 comp rx=0 tx=0).stop 2024-09-12T02:12:44.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.691+0000 7fcaa3bbd740 1 -- 172.21.15.16:0/569490011 shutdown_connections 2024-09-12T02:12:44.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.691+0000 7fcaa3bbd740 1 --2- 172.21.15.16:0/569490011 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c0f6c3c370 0x55c0f6c395d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:44.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.691+0000 7fcaa3bbd740 1 --2- 172.21.15.16:0/569490011 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c0f6c32b40 0x55c0f6c38fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:44.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.691+0000 7fcaa3bbd740 1 --2- 172.21.15.16:0/569490011 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55c0f6c4be80 0x55c0f6c4e270 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:44.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.691+0000 7fcaa3bbd740 1 --2- 172.21.15.16:0/569490011 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c0f6c315d0 0x55c0f6c3bd30 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:44.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.691+0000 7fcaa3bbd740 1 -- 172.21.15.16:0/569490011 >> 172.21.15.16:0/569490011 conn(0x55c0f6bfa310 msgr2=0x55c0f6c33930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:44.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.691+0000 7fca7eff5640 0 client.18837 ms_handle_reset on v2:172.21.15.184:6836/1793393087 2024-09-12T02:12:44.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.691+0000 7fcaa3bbd740 1 -- 172.21.15.16:0/569490011 shutdown_connections 2024-09-12T02:12:44.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.691+0000 7fcaa3bbd740 1 -- 172.21.15.16:0/569490011 wait complete. 2024-09-12T02:12:44.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.691+0000 7fcaa3bbd740 20 client.18837 trim_cache size 0 max 16384 2024-09-12T02:12:44.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.704+0000 7fcaa3bbd740 1 Processor -- start 2024-09-12T02:12:44.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.704+0000 7fcaa3bbd740 1 -- start start 2024-09-12T02:12:44.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.704+0000 7fcaa3bbd740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c0f6c315d0 0x55c0f6c48310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:44.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.704+0000 7fcaa3bbd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c0f6c32b40 0x55c0f6c2d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:44.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.704+0000 7fcaa3bbd740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c0f6c3c370 0x55c0f6c53890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:44.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.704+0000 7fcaa3bbd740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fca9c1afd20 con 0x55c0f6c3c370 2024-09-12T02:12:44.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.704+0000 7fcaa3bbd740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fca9c1ad5f0 con 0x55c0f6c32b40 2024-09-12T02:12:44.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.704+0000 7fcaa3bbd740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fca9c1ab440 con 0x55c0f6c315d0 2024-09-12T02:12:44.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.704+0000 7fca9a7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c0f6c3c370 0x55c0f6c53890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:44.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.705+0000 7fca9a7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c0f6c3c370 0x55c0f6c53890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60880/0 (socket says 172.21.15.16:60880) 2024-09-12T02:12:44.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.705+0000 7fca9a7fc640 1 -- 172.21.15.16:0/2060018279 learned_addr learned my addr 172.21.15.16:0/2060018279 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:12:44.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.705+0000 7fca9affd640 1 --2- 172.21.15.16:0/2060018279 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c0f6c32b40 0x55c0f6c2d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:44.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.705+0000 7fca9a7fc640 1 --2- 172.21.15.16:0/2060018279 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c0f6c315d0 0x55c0f6c48310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:44.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.705+0000 7fca9a7fc640 1 -- 172.21.15.16:0/2060018279 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c0f6c315d0 msgr2=0x55c0f6c48310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:44.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.705+0000 7fca9a7fc640 1 --2- 172.21.15.16:0/2060018279 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c0f6c315d0 0x55c0f6c48310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:44.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.705+0000 7fca9a7fc640 1 -- 172.21.15.16:0/2060018279 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c0f6c32b40 msgr2=0x55c0f6c2d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:44.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.705+0000 7fca9a7fc640 1 --2- 172.21.15.16:0/2060018279 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c0f6c32b40 0x55c0f6c2d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:44.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.705+0000 7fca9a7fc640 1 -- 172.21.15.16:0/2060018279 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0f6c37530 con 0x55c0f6c3c370 2024-09-12T02:12:44.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.705+0000 7fca9affd640 1 --2- 172.21.15.16:0/2060018279 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c0f6c32b40 0x55c0f6c2d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:12:44.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.705+0000 7fca9a7fc640 1 --2- 172.21.15.16:0/2060018279 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c0f6c3c370 0x55c0f6c53890 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7fca9000ae90 tx=0x7fca900117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3590af6d85987865 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:44.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.705+0000 7fca80ff9640 1 -- 172.21.15.16:0/2060018279 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca9003a070 con 0x55c0f6c3c370 2024-09-12T02:12:44.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.705+0000 7fca80ff9640 1 -- 172.21.15.16:0/2060018279 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca9001f030 con 0x55c0f6c3c370 2024-09-12T02:12:44.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.705+0000 7fca80ff9640 1 -- 172.21.15.16:0/2060018279 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca90005490 con 0x55c0f6c3c370 2024-09-12T02:12:44.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.706+0000 7fcaa3bbd740 1 -- 172.21.15.16:0/2060018279 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c0f6c3c370 msgr2=0x55c0f6c53890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:44.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.706+0000 7fcaa3bbd740 1 --2- 172.21.15.16:0/2060018279 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c0f6c3c370 0x55c0f6c53890 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7fca9000ae90 tx=0x7fca900117b0 comp rx=0 tx=0).stop 2024-09-12T02:12:44.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.706+0000 7fcaa3bbd740 1 -- 172.21.15.16:0/2060018279 shutdown_connections 2024-09-12T02:12:44.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.706+0000 7fcaa3bbd740 1 --2- 172.21.15.16:0/2060018279 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c0f6c3c370 0x55c0f6c53890 unknown :-1 s=CLOSED pgs=1509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:44.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.706+0000 7fcaa3bbd740 1 --2- 172.21.15.16:0/2060018279 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c0f6c32b40 0x55c0f6c2d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:44.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.706+0000 7fcaa3bbd740 1 --2- 172.21.15.16:0/2060018279 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c0f6c315d0 0x55c0f6c48310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:44.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.706+0000 7fcaa3bbd740 1 -- 172.21.15.16:0/2060018279 >> 172.21.15.16:0/2060018279 conn(0x55c0f6bfa310 msgr2=0x55c0f6c380d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:44.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.706+0000 7fcaa3bbd740 1 -- 172.21.15.16:0/2060018279 shutdown_connections 2024-09-12T02:12:44.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.706+0000 7fcaa3bbd740 1 -- 172.21.15.16:0/2060018279 wait complete. 2024-09-12T02:12:44.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.706+0000 7fcaa3bbd740 1 -- start start 2024-09-12T02:12:44.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.706+0000 7fcaa3bbd740 1 Processor -- start 2024-09-12T02:12:44.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.706+0000 7fcaa3bbd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c0f6c315d0 0x55c0f6c44bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:44.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.707+0000 7fcaa3bbd740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c0f6c32b40 0x55c0f6c3fef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:44.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.707+0000 7fcaa3bbd740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c0f6c3c370 0x55c0f6c40430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:44.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.707+0000 7fcaa3bbd740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fca9c1abab0 con 0x55c0f6c3c370 2024-09-12T02:12:44.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.707+0000 7fcaa3bbd740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fca9c1acf20 con 0x55c0f6c315d0 2024-09-12T02:12:44.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.707+0000 7fcaa3bbd740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fca9c1ac180 con 0x55c0f6c32b40 2024-09-12T02:12:44.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.707+0000 7fcaa3bbd740 10 client.0 _ll_register_callbacks cb 0x7fca8c00b810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:12:44.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.707+0000 7fca9a7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c0f6c3c370 0x55c0f6c40430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:44.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.707+0000 7fca9a7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c0f6c315d0 0x55c0f6c44bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:44.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.707+0000 7fca9a7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c0f6c315d0 0x55c0f6c44bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.16:37410/0 (socket says 172.21.15.16:37410) 2024-09-12T02:12:44.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.707+0000 7fca9a7fc640 1 -- 172.21.15.16:0/4075539397 learned_addr learned my addr 172.21.15.16:0/4075539397 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:12:44.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.707+0000 7fca9affd640 1 --2- 172.21.15.16:0/4075539397 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c0f6c32b40 0x55c0f6c3fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:44.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.707+0000 7fca9a7fc640 1 -- 172.21.15.16:0/4075539397 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c0f6c32b40 msgr2=0x55c0f6c3fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:44.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.707+0000 7fca9a7fc640 1 --2- 172.21.15.16:0/4075539397 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c0f6c32b40 0x55c0f6c3fef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:44.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.707+0000 7fca9a7fc640 1 -- 172.21.15.16:0/4075539397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c0f6c3c370 msgr2=0x55c0f6c40430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:44.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.707+0000 7fca9a7fc640 1 --2- 172.21.15.16:0/4075539397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c0f6c3c370 0x55c0f6c40430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:44.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.707+0000 7fca9a7fc640 1 -- 172.21.15.16:0/4075539397 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c0f6c40c60 con 0x55c0f6c315d0 2024-09-12T02:12:44.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.707+0000 7fca9a7fc640 1 --2- 172.21.15.16:0/4075539397 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c0f6c315d0 0x55c0f6c44bc0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fca90005cc0 tx=0x7fca9000a590 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de59adb729d4921c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:44.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.707+0000 7fca7eff5640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-12T02:12:44.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.707+0000 7fca7eff5640 1 -- 172.21.15.16:0/4075539397 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca9003a070 con 0x55c0f6c315d0 2024-09-12T02:12:44.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.707+0000 7fcaa3bbd740 1 -- 172.21.15.16:0/4075539397 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca9001f030 con 0x55c0f6c315d0 2024-09-12T02:12:44.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.707+0000 7fca7eff5640 1 -- 172.21.15.16:0/4075539397 <== mon.2 v2:172.21.15.33:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fca90005430 con 0x55c0f6c315d0 2024-09-12T02:12:44.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.708+0000 7fca7eff5640 1 -- 172.21.15.16:0/4075539397 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fca90012b10 con 0x55c0f6c315d0 2024-09-12T02:12:44.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.708+0000 7fcaa3bbd740 10 client.27899 fetch_fsmap learned FSMap version 25 2024-09-12T02:12:44.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.708+0000 7fcaa3bbd740 1 -- 172.21.15.16:0/4075539397 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c0f6cb6850 con 0x55c0f6c315d0 2024-09-12T02:12:44.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.708+0000 7fca7eff5640 1 -- 172.21.15.16:0/4075539397 <== mon.2 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca9001f030 con 0x55c0f6c315d0 2024-09-12T02:12:44.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.708+0000 7fca7eff5640 1 -- 172.21.15.16:0/4075539397 <== mon.2 v2:172.21.15.33:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fca9002a430 con 0x55c0f6c315d0 2024-09-12T02:12:44.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.708+0000 7fcaa3bbd740 10 client.27899 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:12:44.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.708+0000 7fcaa3bbd740 10 client.27899 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-12T02:12:44.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.708+0000 7fcaa3bbd740 20 client.27899 populate_metadata read hostname 'smithi016' 2024-09-12T02:12:44.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.708+0000 7fcaa3bbd740 1 --2- 172.21.15.16:0/4075539397 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55c0f6cb6e00 0x55c0f6cb91f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:44.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.708+0000 7fcaa3bbd740 4 client.27899 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T02:12:44.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.708+0000 7fcaa3bbd740 1 -- 172.21.15.16:0/4075539397 --> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c0f6b9ed70 con 0x55c0f6cb6e00 2024-09-12T02:12:44.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.709+0000 7fca9affd640 1 --2- 172.21.15.16:0/4075539397 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55c0f6cb6e00 0x55c0f6cb91f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:44.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.709+0000 7fca9affd640 1 --2- 172.21.15.16:0/4075539397 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55c0f6cb6e00 0x55c0f6cb91f0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3c6a0e85bc178601 server_cookie=78b3eb8375882291 in_seq=0 out_seq=0 2024-09-12T02:12:44.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.709+0000 7fca7eff5640 10 client.27899 ms_handle_connect on v2:172.21.15.184:6836/1793393087 2024-09-12T02:12:44.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.709+0000 7fca7eff5640 1 -- 172.21.15.16:0/4075539397 <== mds.2 v2:172.21.15.184:6836/1793393087 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c0f6b9ed70 con 0x55c0f6cb6e00 2024-09-12T02:12:44.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.709+0000 7fca7eff5640 10 client.27899 handle_command_reply: tid=0 2024-09-12T02:12:44.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.709+0000 7fcaa3bbd740 1 client.27899 shutdown 2024-09-12T02:12:44.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.709+0000 7fcaa3bbd740 1 -- 172.21.15.16:0/4075539397 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c0f6c315d0 msgr2=0x55c0f6c44bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:44.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.709+0000 7fcaa3bbd740 1 --2- 172.21.15.16:0/4075539397 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c0f6c315d0 0x55c0f6c44bc0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fca90005cc0 tx=0x7fca9000a590 comp rx=0 tx=0).stop 2024-09-12T02:12:44.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.709+0000 7fcaa3bbd740 1 -- 172.21.15.16:0/4075539397 shutdown_connections 2024-09-12T02:12:44.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.709+0000 7fcaa3bbd740 1 --2- 172.21.15.16:0/4075539397 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55c0f6cb6e00 0x55c0f6cb91f0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:44.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.709+0000 7fcaa3bbd740 1 --2- 172.21.15.16:0/4075539397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c0f6c3c370 0x55c0f6c40430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:44.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.709+0000 7fca7eff5640 0 client.27899 ms_handle_reset on v2:172.21.15.184:6836/1793393087 2024-09-12T02:12:44.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.709+0000 7fcaa3bbd740 1 --2- 172.21.15.16:0/4075539397 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c0f6c32b40 0x55c0f6c3fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:44.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.709+0000 7fcaa3bbd740 1 --2- 172.21.15.16:0/4075539397 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c0f6c315d0 0x55c0f6c44bc0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:44.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.709+0000 7fcaa3bbd740 1 -- 172.21.15.16:0/4075539397 >> 172.21.15.16:0/4075539397 conn(0x55c0f6bfa310 msgr2=0x55c0f6c380d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:44.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.709+0000 7fcaa3bbd740 1 -- 172.21.15.16:0/4075539397 shutdown_connections 2024-09-12T02:12:44.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.709+0000 7fcaa3bbd740 1 -- 172.21.15.16:0/4075539397 wait complete. 2024-09-12T02:12:44.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.709+0000 7fcaa3bbd740 20 client.27899 trim_cache size 0 max 16384 2024-09-12T02:12:44.720 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T02:12:44.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.710+0000 7fcaa17e4640 1 -- 172.21.15.16:0/207058389 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fca840a2b80 msgr2=0x7fca840a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:44.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.710+0000 7fcaa17e4640 1 --2- 172.21.15.16:0/207058389 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fca840a2b80 0x7fca840a5040 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fca940bbcd0 tx=0x7fca94009260 comp rx=0 tx=0).stop 2024-09-12T02:12:44.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.710+0000 7fcaa17e4640 1 -- 172.21.15.16:0/207058389 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca9c1082a0 msgr2=0x7fca9c19ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:44.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.710+0000 7fcaa17e4640 1 --2- 172.21.15.16:0/207058389 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca9c1082a0 0x7fca9c19ce10 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7fca94005ab0 tx=0x7fca940042f0 comp rx=0 tx=0).stop 2024-09-12T02:12:44.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.710+0000 7fcaa17e4640 1 -- 172.21.15.16:0/207058389 shutdown_connections 2024-09-12T02:12:44.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.710+0000 7fcaa17e4640 1 --2- 172.21.15.16:0/207058389 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fca840a2b80 0x7fca840a5040 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:44.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.710+0000 7fcaa17e4640 1 --2- 172.21.15.16:0/207058389 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca9c1082a0 0x7fca9c19ce10 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:44.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.710+0000 7fcaa17e4640 1 --2- 172.21.15.16:0/207058389 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca9c1036e0 0x7fca9c198a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:44.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.710+0000 7fcaa17e4640 1 --2- 172.21.15.16:0/207058389 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fca9c102d30 0x7fca9c198540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:44.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.710+0000 7fcaa17e4640 1 -- 172.21.15.16:0/207058389 >> 172.21.15.16:0/207058389 conn(0x7fca9c0fc7d0 msgr2=0x7fca9c105d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:44.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.710+0000 7fcaa17e4640 1 -- 172.21.15.16:0/207058389 shutdown_connections 2024-09-12T02:12:44.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:44.710+0000 7fcaa17e4640 1 -- 172.21.15.16:0/207058389 wait complete. 2024-09-12T02:12:45.388 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-12T02:12:45.644 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:12:46.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.118+0000 7f74d4f69640 1 -- 172.21.15.16:0/4138932674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74d0101420 msgr2=0x7f74d010d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:46.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.118+0000 7f74d4f69640 1 --2- 172.21.15.16:0/4138932674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74d0101420 0x7f74d010d950 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7f74c8009cd0 tx=0x7f74c802a8e0 comp rx=0 tx=0).stop 2024-09-12T02:12:46.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.119+0000 7f74d4f69640 1 -- 172.21.15.16:0/4138932674 shutdown_connections 2024-09-12T02:12:46.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.119+0000 7f74d4f69640 1 --2- 172.21.15.16:0/4138932674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74d0101420 0x7f74d010d950 unknown :-1 s=CLOSED pgs=1510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:46.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.119+0000 7f74d4f69640 1 --2- 172.21.15.16:0/4138932674 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f74d0100a80 0x7f74d0100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:46.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.119+0000 7f74d4f69640 1 --2- 172.21.15.16:0/4138932674 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74d0106a80 0x7f74d0106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:46.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.119+0000 7f74d4f69640 1 -- 172.21.15.16:0/4138932674 >> 172.21.15.16:0/4138932674 conn(0x7f74d00fc7a0 msgr2=0x7f74d00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:46.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.119+0000 7f74d4f69640 1 -- 172.21.15.16:0/4138932674 shutdown_connections 2024-09-12T02:12:46.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.119+0000 7f74d4f69640 1 -- 172.21.15.16:0/4138932674 wait complete. 2024-09-12T02:12:46.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.119+0000 7f74d4f69640 1 Processor -- start 2024-09-12T02:12:46.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.119+0000 7f74d4f69640 1 -- start start 2024-09-12T02:12:46.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.120+0000 7f74d4f69640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f74d0100a80 0x7f74d01992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:46.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.120+0000 7f74d4f69640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74d0101420 0x7f74d01997f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:46.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.120+0000 7f74d4f69640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74d0106a80 0x7f74d01a4330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:46.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.120+0000 7f74d4f69640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f74d0110e80 con 0x7f74d0106a80 2024-09-12T02:12:46.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.120+0000 7f74d4f69640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f74d0110d00 con 0x7f74d0101420 2024-09-12T02:12:46.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.120+0000 7f74d4f69640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f74d0111000 con 0x7f74d0100a80 2024-09-12T02:12:46.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.120+0000 7f74ced9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74d0106a80 0x7f74d01a4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:46.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.120+0000 7f74ce59c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f74d0100a80 0x7f74d01992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:46.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.120+0000 7f74cdd9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74d0101420 0x7f74d01997f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:46.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.120+0000 7f74ced9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74d0106a80 0x7f74d01a4330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60898/0 (socket says 172.21.15.16:60898) 2024-09-12T02:12:46.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.120+0000 7f74ced9d640 1 -- 172.21.15.16:0/2941326501 learned_addr learned my addr 172.21.15.16:0/2941326501 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:12:46.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.120+0000 7f74cdd9b640 1 -- 172.21.15.16:0/2941326501 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f74d0100a80 msgr2=0x7f74d01992b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:46.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.120+0000 7f74cdd9b640 1 --2- 172.21.15.16:0/2941326501 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f74d0100a80 0x7f74d01992b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:46.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.120+0000 7f74cdd9b640 1 -- 172.21.15.16:0/2941326501 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74d0106a80 msgr2=0x7f74d01a4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:46.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.120+0000 7f74cdd9b640 1 --2- 172.21.15.16:0/2941326501 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74d0106a80 0x7f74d01a4330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:46.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.120+0000 7f74cdd9b640 1 -- 172.21.15.16:0/2941326501 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74d01a4870 con 0x7f74d0101420 2024-09-12T02:12:46.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.121+0000 7f74cdd9b640 1 --2- 172.21.15.16:0/2941326501 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74d0101420 0x7f74d01997f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f74c400e9f0 tx=0x7f74c400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:46.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.121+0000 7f74bf7fe640 1 -- 172.21.15.16:0/2941326501 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74c400ccc0 con 0x7f74d0101420 2024-09-12T02:12:46.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.121+0000 7f74d4f69640 1 -- 172.21.15.16:0/2941326501 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74d01a4a40 con 0x7f74d0101420 2024-09-12T02:12:46.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.121+0000 7f74bf7fe640 1 -- 172.21.15.16:0/2941326501 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f74c4005020 con 0x7f74d0101420 2024-09-12T02:12:46.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.121+0000 7f74bf7fe640 1 -- 172.21.15.16:0/2941326501 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74c40049e0 con 0x7f74d0101420 2024-09-12T02:12:46.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.121+0000 7f74d4f69640 1 -- 172.21.15.16:0/2941326501 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74d01a4f50 con 0x7f74d0101420 2024-09-12T02:12:46.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.124+0000 7f74bf7fe640 1 -- 172.21.15.16:0/2941326501 <== mon.2 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f74c4005560 con 0x7f74d0101420 2024-09-12T02:12:46.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.124+0000 7f74d7342740 1 Processor -- start 2024-09-12T02:12:46.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.124+0000 7f74d7342740 1 -- start start 2024-09-12T02:12:46.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.124+0000 7f74d7342740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ac2950b40 0x556ac294f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:46.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.124+0000 7f74d7342740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556ac294f5d0 0x556ac294fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:46.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.124+0000 7f74d7342740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556ac295a370 0x556ac295c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:46.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.124+0000 7f74d7342740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x556ac28b52f0 con 0x556ac2950b40 2024-09-12T02:12:46.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.124+0000 7f74d7342740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x556ac285ccf0 con 0x556ac294f5d0 2024-09-12T02:12:46.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.124+0000 7f74d7342740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x556ac28b12b0 con 0x556ac295a370 2024-09-12T02:12:46.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.124+0000 7f74ced9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ac2950b40 0x556ac294f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:46.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.125+0000 7f74ced9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ac2950b40 0x556ac294f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60912/0 (socket says 172.21.15.16:60912) 2024-09-12T02:12:46.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.125+0000 7f74ced9d640 1 -- 172.21.15.16:0/1413387611 learned_addr learned my addr 172.21.15.16:0/1413387611 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:12:46.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.125+0000 7f74ce59c640 1 --2- 172.21.15.16:0/1413387611 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556ac294f5d0 0x556ac294fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:46.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.125+0000 7f74ced9d640 1 -- 172.21.15.16:0/1413387611 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556ac295a370 msgr2=0x556ac295c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:46.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.125+0000 7f74ced9d640 1 --2- 172.21.15.16:0/1413387611 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556ac295a370 0x556ac295c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:46.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.125+0000 7f74ced9d640 1 -- 172.21.15.16:0/1413387611 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556ac294f5d0 msgr2=0x556ac294fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:46.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.125+0000 7f74ced9d640 1 --2- 172.21.15.16:0/1413387611 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556ac294f5d0 0x556ac294fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:46.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.125+0000 7f74ced9d640 1 -- 172.21.15.16:0/1413387611 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ac28413a0 con 0x556ac2950b40 2024-09-12T02:12:46.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.125+0000 7f74bf7fe640 1 --2- 172.21.15.16:0/2941326501 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f74b80a2b80 0x7f74b80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:46.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.125+0000 7f74ced9d640 1 --2- 172.21.15.16:0/1413387611 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ac2950b40 0x556ac294f090 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f74c8005fa0 tx=0x7f74c8009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ada663dbd73530e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:46.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.125+0000 7f74ce59c640 1 --2- 172.21.15.16:0/2941326501 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f74b80a2b80 0x7f74b80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:46.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.125+0000 7f74bf7fe640 1 -- 172.21.15.16:0/2941326501 <== mon.2 v2:172.21.15.33:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f74c40cef10 con 0x7f74d0101420 2024-09-12T02:12:46.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.125+0000 7f74bcff9640 1 -- 172.21.15.16:0/1413387611 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74c8031ac0 con 0x556ac2950b40 2024-09-12T02:12:46.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.125+0000 7f74bcff9640 1 -- 172.21.15.16:0/1413387611 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f74c8036070 con 0x556ac2950b40 2024-09-12T02:12:46.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.125+0000 7f74bcff9640 1 -- 172.21.15.16:0/1413387611 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74c80045e0 con 0x556ac2950b40 2024-09-12T02:12:46.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.126+0000 7f74d7342740 1 -- 172.21.15.16:0/1413387611 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ac2950b40 msgr2=0x556ac294f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:46.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.126+0000 7f74ce59c640 1 --2- 172.21.15.16:0/2941326501 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f74b80a2b80 0x7f74b80a5040 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x556ac294cac0 tx=0x7f74c000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:46.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.126+0000 7f74d7342740 1 --2- 172.21.15.16:0/1413387611 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ac2950b40 0x556ac294f090 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f74c8005fa0 tx=0x7f74c8009a20 comp rx=0 tx=0).stop 2024-09-12T02:12:46.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.126+0000 7f74d7342740 1 -- 172.21.15.16:0/1413387611 shutdown_connections 2024-09-12T02:12:46.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.126+0000 7f74d7342740 1 --2- 172.21.15.16:0/1413387611 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556ac295a370 0x556ac295c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:46.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.126+0000 7f74d7342740 1 --2- 172.21.15.16:0/1413387611 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556ac294f5d0 0x556ac294fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:46.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.126+0000 7f74d7342740 1 --2- 172.21.15.16:0/1413387611 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ac2950b40 0x556ac294f090 unknown :-1 s=CLOSED pgs=1511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:46.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.126+0000 7f74d7342740 1 -- 172.21.15.16:0/1413387611 >> 172.21.15.16:0/1413387611 conn(0x556ac2918310 msgr2=0x556ac294bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:46.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.126+0000 7f74d7342740 1 -- 172.21.15.16:0/1413387611 shutdown_connections 2024-09-12T02:12:46.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.126+0000 7f74d7342740 1 -- 172.21.15.16:0/1413387611 wait complete. 2024-09-12T02:12:46.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.127+0000 7f74d7342740 1 -- start start 2024-09-12T02:12:46.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.127+0000 7f74d7342740 1 Processor -- start 2024-09-12T02:12:46.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.127+0000 7f74d7342740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556ac294f5d0 0x556ac2959d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:46.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.127+0000 7f74d7342740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556ac2950b40 0x556ac2957040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:46.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.127+0000 7f74d7342740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ac295a370 0x556ac2957650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:46.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.127+0000 7f74d7342740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x556ac295d010 con 0x556ac295a370 2024-09-12T02:12:46.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.127+0000 7f74d7342740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x556ac2870060 con 0x556ac2950b40 2024-09-12T02:12:46.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.127+0000 7f74d7342740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x556ac295d190 con 0x556ac294f5d0 2024-09-12T02:12:46.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.127+0000 7f74d7342740 10 client.0 _ll_register_callbacks cb 0x556ac25c38c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:12:46.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.127+0000 7f74ced9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ac295a370 0x556ac2957650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:46.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.127+0000 7f74ced9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ac295a370 0x556ac2957650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60914/0 (socket says 172.21.15.16:60914) 2024-09-12T02:12:46.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.127+0000 7f74ced9d640 1 -- 172.21.15.16:0/4109311155 learned_addr learned my addr 172.21.15.16:0/4109311155 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:12:46.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.127+0000 7f74ce59c640 1 --2- 172.21.15.16:0/4109311155 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556ac294f5d0 0x556ac2959d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:46.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.127+0000 7f74cdd9b640 1 --2- 172.21.15.16:0/4109311155 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556ac2950b40 0x556ac2957040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:46.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.127+0000 7f74ced9d640 1 -- 172.21.15.16:0/4109311155 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556ac294f5d0 msgr2=0x556ac2959d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:46.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.127+0000 7f74ced9d640 1 --2- 172.21.15.16:0/4109311155 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556ac294f5d0 0x556ac2959d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:46.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.127+0000 7f74ced9d640 1 -- 172.21.15.16:0/4109311155 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556ac2950b40 msgr2=0x556ac2957040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:46.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.127+0000 7f74ced9d640 1 --2- 172.21.15.16:0/4109311155 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556ac2950b40 0x556ac2957040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:46.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.127+0000 7f74ced9d640 1 -- 172.21.15.16:0/4109311155 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ac26263c0 con 0x556ac295a370 2024-09-12T02:12:46.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.127+0000 7f74cdd9b640 1 --2- 172.21.15.16:0/4109311155 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556ac2950b40 0x556ac2957040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:12:46.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.128+0000 7f74ced9d640 1 --2- 172.21.15.16:0/4109311155 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ac295a370 0x556ac2957650 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7f74c8009df0 tx=0x7f74c80040e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9850796fb569d228 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:46.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.128+0000 7f74b5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:12:46.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.128+0000 7f74b5ffb640 1 -- 172.21.15.16:0/4109311155 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74c8050070 con 0x556ac295a370 2024-09-12T02:12:46.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.128+0000 7f74b5ffb640 1 -- 172.21.15.16:0/4109311155 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f74c8003970 con 0x556ac295a370 2024-09-12T02:12:46.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.128+0000 7f74d7342740 1 -- 172.21.15.16:0/4109311155 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f74c8036070 con 0x556ac295a370 2024-09-12T02:12:46.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.128+0000 7f74b5ffb640 1 -- 172.21.15.16:0/4109311155 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f74c805c050 con 0x556ac295a370 2024-09-12T02:12:46.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.128+0000 7f74d7342740 10 client.18855 fetch_fsmap learned FSMap version 25 2024-09-12T02:12:46.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.128+0000 7f74d7342740 1 -- 172.21.15.16:0/4109311155 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x556ac295d010 con 0x556ac295a370 2024-09-12T02:12:46.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.128+0000 7f74b5ffb640 1 -- 172.21.15.16:0/4109311155 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f74c8036070 con 0x556ac295a370 2024-09-12T02:12:46.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.128+0000 7f74b5ffb640 1 -- 172.21.15.16:0/4109311155 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f74c8042070 con 0x556ac295a370 2024-09-12T02:12:46.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.128+0000 7f74d7342740 10 client.18855 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:12:46.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.128+0000 7f74d7342740 10 client.18855 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T02:12:46.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.128+0000 7f74d7342740 20 client.18855 populate_metadata read hostname 'smithi016' 2024-09-12T02:12:46.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.128+0000 7f74d7342740 1 --2- 172.21.15.16:0/4109311155 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x556ac2969cc0 0x556ac296c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:46.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.128+0000 7f74d7342740 4 client.18855 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:12:46.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.128+0000 7f74d7342740 1 -- 172.21.15.16:0/4109311155 --> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556ac28c2860 con 0x556ac2969cc0 2024-09-12T02:12:46.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.129+0000 7f74ce59c640 1 --2- 172.21.15.16:0/4109311155 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x556ac2969cc0 0x556ac296c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:46.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.129+0000 7f74ce59c640 1 --2- 172.21.15.16:0/4109311155 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x556ac2969cc0 0x556ac296c0b0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2564954b47793255 server_cookie=f31f8ef76d4d4f8 in_seq=0 out_seq=0 2024-09-12T02:12:46.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.129+0000 7f74b5ffb640 10 client.18855 ms_handle_connect on v2:172.21.15.184:6838/4203347941 2024-09-12T02:12:46.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.130+0000 7f74b5ffb640 1 -- 172.21.15.16:0/4109311155 <== mds.1 v2:172.21.15.184:6838/4203347941 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556ac28c2860 con 0x556ac2969cc0 2024-09-12T02:12:46.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.130+0000 7f74b5ffb640 10 client.18855 handle_command_reply: tid=0 2024-09-12T02:12:46.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.130+0000 7f74d7342740 1 client.18855 shutdown 2024-09-12T02:12:46.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.130+0000 7f74d7342740 1 -- 172.21.15.16:0/4109311155 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ac295a370 msgr2=0x556ac2957650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:46.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.130+0000 7f74d7342740 1 --2- 172.21.15.16:0/4109311155 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ac295a370 0x556ac2957650 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7f74c8009df0 tx=0x7f74c80040e0 comp rx=0 tx=0).stop 2024-09-12T02:12:46.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.130+0000 7f74d7342740 1 -- 172.21.15.16:0/4109311155 shutdown_connections 2024-09-12T02:12:46.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.130+0000 7f74d7342740 1 --2- 172.21.15.16:0/4109311155 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x556ac2969cc0 0x556ac296c0b0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:46.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.130+0000 7f74d7342740 1 --2- 172.21.15.16:0/4109311155 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ac295a370 0x556ac2957650 unknown :-1 s=CLOSED pgs=1512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:46.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.130+0000 7f74d7342740 1 --2- 172.21.15.16:0/4109311155 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556ac2950b40 0x556ac2957040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:46.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.130+0000 7f74b5ffb640 0 client.18855 ms_handle_reset on v2:172.21.15.184:6838/4203347941 2024-09-12T02:12:46.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.130+0000 7f74d7342740 1 --2- 172.21.15.16:0/4109311155 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556ac294f5d0 0x556ac2959d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:46.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.130+0000 7f74d7342740 1 -- 172.21.15.16:0/4109311155 >> 172.21.15.16:0/4109311155 conn(0x556ac2918310 msgr2=0x556ac2951930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:46.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.130+0000 7f74d7342740 1 -- 172.21.15.16:0/4109311155 shutdown_connections 2024-09-12T02:12:46.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.130+0000 7f74d7342740 1 -- 172.21.15.16:0/4109311155 wait complete. 2024-09-12T02:12:46.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.130+0000 7f74d7342740 20 client.18855 trim_cache size 0 max 16384 2024-09-12T02:12:46.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.144+0000 7f74d7342740 1 Processor -- start 2024-09-12T02:12:46.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.144+0000 7f74d7342740 1 -- start start 2024-09-12T02:12:46.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.144+0000 7f74d7342740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556ac294f5d0 0x556ac2959370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:46.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.144+0000 7f74d7342740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556ac2950b40 0x556ac2966310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:46.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.144+0000 7f74d7342740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ac295a370 0x556ac29717d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:46.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.144+0000 7f74d7342740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f74d01afbf0 con 0x556ac295a370 2024-09-12T02:12:46.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.144+0000 7f74d7342740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f74d01ad4c0 con 0x556ac294f5d0 2024-09-12T02:12:46.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.144+0000 7f74d7342740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f74d01ab310 con 0x556ac2950b40 2024-09-12T02:12:46.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.144+0000 7f74ced9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ac295a370 0x556ac29717d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:46.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.144+0000 7f74ced9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ac295a370 0x556ac29717d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60930/0 (socket says 172.21.15.16:60930) 2024-09-12T02:12:46.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.144+0000 7f74ced9d640 1 -- 172.21.15.16:0/2542049352 learned_addr learned my addr 172.21.15.16:0/2542049352 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:12:46.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.144+0000 7f74ce59c640 1 --2- 172.21.15.16:0/2542049352 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556ac294f5d0 0x556ac2959370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:46.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.144+0000 7f74cdd9b640 1 --2- 172.21.15.16:0/2542049352 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556ac2950b40 0x556ac2966310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:46.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.145+0000 7f74ced9d640 1 -- 172.21.15.16:0/2542049352 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556ac2950b40 msgr2=0x556ac2966310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:46.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.145+0000 7f74ced9d640 1 --2- 172.21.15.16:0/2542049352 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556ac2950b40 0x556ac2966310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:46.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.145+0000 7f74ced9d640 1 -- 172.21.15.16:0/2542049352 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556ac294f5d0 msgr2=0x556ac2959370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:46.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.145+0000 7f74ced9d640 1 --2- 172.21.15.16:0/2542049352 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556ac294f5d0 0x556ac2959370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:46.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.145+0000 7f74ced9d640 1 -- 172.21.15.16:0/2542049352 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ac2957fe0 con 0x556ac295a370 2024-09-12T02:12:46.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.145+0000 7f74ced9d640 1 --2- 172.21.15.16:0/2542049352 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ac295a370 0x556ac29717d0 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f74c8000f80 tx=0x7f74c8009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a259ffe4fbf8daeb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:46.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.145+0000 7f74b7fff640 1 -- 172.21.15.16:0/2542049352 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74c8059070 con 0x556ac295a370 2024-09-12T02:12:46.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.145+0000 7f74b7fff640 1 -- 172.21.15.16:0/2542049352 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f74c8036070 con 0x556ac295a370 2024-09-12T02:12:46.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.145+0000 7f74b7fff640 1 -- 172.21.15.16:0/2542049352 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74c8050050 con 0x556ac295a370 2024-09-12T02:12:46.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.146+0000 7f74d7342740 1 -- 172.21.15.16:0/2542049352 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ac295a370 msgr2=0x556ac29717d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:46.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.146+0000 7f74d7342740 1 --2- 172.21.15.16:0/2542049352 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ac295a370 0x556ac29717d0 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f74c8000f80 tx=0x7f74c8009a20 comp rx=0 tx=0).stop 2024-09-12T02:12:46.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.146+0000 7f74d7342740 1 -- 172.21.15.16:0/2542049352 shutdown_connections 2024-09-12T02:12:46.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.146+0000 7f74d7342740 1 --2- 172.21.15.16:0/2542049352 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ac295a370 0x556ac29717d0 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:46.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.146+0000 7f74d7342740 1 --2- 172.21.15.16:0/2542049352 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556ac2950b40 0x556ac2966310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:46.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.146+0000 7f74d7342740 1 --2- 172.21.15.16:0/2542049352 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556ac294f5d0 0x556ac2959370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:46.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.146+0000 7f74d7342740 1 -- 172.21.15.16:0/2542049352 >> 172.21.15.16:0/2542049352 conn(0x556ac2918310 msgr2=0x556ac2956000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:46.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.146+0000 7f74d7342740 1 -- 172.21.15.16:0/2542049352 shutdown_connections 2024-09-12T02:12:46.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.146+0000 7f74d7342740 1 -- 172.21.15.16:0/2542049352 wait complete. 2024-09-12T02:12:46.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.146+0000 7f74d7342740 1 -- start start 2024-09-12T02:12:46.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.147+0000 7f74d7342740 1 Processor -- start 2024-09-12T02:12:46.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.147+0000 7f74d7342740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556ac294f5d0 0x556ac2962a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:46.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.147+0000 7f74d7342740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ac2950b40 0x556ac2953bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:46.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.147+0000 7f74d7342740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556ac295a370 0x556ac2954100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:46.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.147+0000 7f74d7342740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f74d01ab980 con 0x556ac2950b40 2024-09-12T02:12:46.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.147+0000 7f74d7342740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f74d01acdf0 con 0x556ac294f5d0 2024-09-12T02:12:46.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.147+0000 7f74d7342740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f74d01ac050 con 0x556ac295a370 2024-09-12T02:12:46.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.147+0000 7f74d7342740 10 client.0 _ll_register_callbacks cb 0x7f74c000b730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:12:46.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.147+0000 7f74cdd9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ac2950b40 0x556ac2953bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:46.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.147+0000 7f74cdd9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ac2950b40 0x556ac2953bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60938/0 (socket says 172.21.15.16:60938) 2024-09-12T02:12:46.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.147+0000 7f74cdd9b640 1 -- 172.21.15.16:0/782539134 learned_addr learned my addr 172.21.15.16:0/782539134 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:12:46.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.147+0000 7f74ce59c640 1 --2- 172.21.15.16:0/782539134 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556ac294f5d0 0x556ac2962a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:46.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.147+0000 7f74ced9d640 1 --2- 172.21.15.16:0/782539134 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556ac295a370 0x556ac2954100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:46.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.147+0000 7f74cdd9b640 1 -- 172.21.15.16:0/782539134 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556ac295a370 msgr2=0x556ac2954100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:46.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.147+0000 7f74cdd9b640 1 --2- 172.21.15.16:0/782539134 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556ac295a370 0x556ac2954100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:46.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.147+0000 7f74cdd9b640 1 -- 172.21.15.16:0/782539134 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556ac294f5d0 msgr2=0x556ac2962a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:46.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.147+0000 7f74cdd9b640 1 --2- 172.21.15.16:0/782539134 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556ac294f5d0 0x556ac2962a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:46.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.147+0000 7f74cdd9b640 1 -- 172.21.15.16:0/782539134 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ac2952120 con 0x556ac2950b40 2024-09-12T02:12:46.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.147+0000 7f74cdd9b640 1 --2- 172.21.15.16:0/782539134 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ac2950b40 0x556ac2953bc0 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f74c4097c70 tx=0x7f74c409b230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56426a72ebacfd52 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:46.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.147+0000 7f74b5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:12:46.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.148+0000 7f74b5ffb640 1 -- 172.21.15.16:0/782539134 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74c401c040 con 0x556ac2950b40 2024-09-12T02:12:46.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.148+0000 7f74b5ffb640 1 -- 172.21.15.16:0/782539134 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f74c40a3ff0 con 0x556ac2950b40 2024-09-12T02:12:46.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.148+0000 7f74d7342740 1 -- 172.21.15.16:0/782539134 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f74c8036070 con 0x556ac2950b40 2024-09-12T02:12:46.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.148+0000 7f74b5ffb640 1 -- 172.21.15.16:0/782539134 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f74c409b810 con 0x556ac2950b40 2024-09-12T02:12:46.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.148+0000 7f74d7342740 10 client.18861 fetch_fsmap learned FSMap version 25 2024-09-12T02:12:46.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.148+0000 7f74d7342740 1 -- 172.21.15.16:0/782539134 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x556ac29d4490 con 0x556ac2950b40 2024-09-12T02:12:46.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.148+0000 7f74b5ffb640 1 -- 172.21.15.16:0/782539134 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f74c8036070 con 0x556ac2950b40 2024-09-12T02:12:46.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.148+0000 7f74b5ffb640 1 -- 172.21.15.16:0/782539134 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f74c409cb40 con 0x556ac2950b40 2024-09-12T02:12:46.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.148+0000 7f74d7342740 10 client.18861 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:12:46.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.148+0000 7f74d7342740 10 client.18861 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T02:12:46.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.148+0000 7f74d7342740 20 client.18861 populate_metadata read hostname 'smithi016' 2024-09-12T02:12:46.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.148+0000 7f74d7342740 1 --2- 172.21.15.16:0/782539134 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x556ac29d4bc0 0x556ac29d6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:46.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.148+0000 7f74d7342740 4 client.18861 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T02:12:46.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.148+0000 7f74d7342740 1 -- 172.21.15.16:0/782539134 --> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556ac28bcd70 con 0x556ac29d4bc0 2024-09-12T02:12:46.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.148+0000 7f74ce59c640 1 --2- 172.21.15.16:0/782539134 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x556ac29d4bc0 0x556ac29d6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:46.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.149+0000 7f74b5ffb640 10 client.18861 ms_handle_connect on v2:172.21.15.184:6838/4203347941 2024-09-12T02:12:46.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.149+0000 7f74ce59c640 1 --2- 172.21.15.16:0/782539134 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x556ac29d4bc0 0x556ac29d6fe0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=db3ce01e064b940b server_cookie=7a62b13be13b7a2 in_seq=0 out_seq=0 2024-09-12T02:12:46.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.149+0000 7f74b5ffb640 1 -- 172.21.15.16:0/782539134 <== mds.1 v2:172.21.15.184:6838/4203347941 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556ac28bcd70 con 0x556ac29d4bc0 2024-09-12T02:12:46.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.149+0000 7f74b5ffb640 10 client.18861 handle_command_reply: tid=0 2024-09-12T02:12:46.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.149+0000 7f74d7342740 1 client.18861 shutdown 2024-09-12T02:12:46.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.149+0000 7f74d7342740 1 -- 172.21.15.16:0/782539134 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ac2950b40 msgr2=0x556ac2953bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:46.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.149+0000 7f74d7342740 1 --2- 172.21.15.16:0/782539134 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ac2950b40 0x556ac2953bc0 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f74c4097c70 tx=0x7f74c409b230 comp rx=0 tx=0).stop 2024-09-12T02:12:46.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.149+0000 7f74d7342740 1 -- 172.21.15.16:0/782539134 shutdown_connections 2024-09-12T02:12:46.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.149+0000 7f74d7342740 1 --2- 172.21.15.16:0/782539134 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x556ac29d4bc0 0x556ac29d6fe0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:46.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.149+0000 7f74d7342740 1 --2- 172.21.15.16:0/782539134 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556ac295a370 0x556ac2954100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:46.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.149+0000 7f74b5ffb640 0 client.18861 ms_handle_reset on v2:172.21.15.184:6838/4203347941 2024-09-12T02:12:46.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.149+0000 7f74d7342740 1 --2- 172.21.15.16:0/782539134 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ac2950b40 0x556ac2953bc0 unknown :-1 s=CLOSED pgs=1514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:46.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.149+0000 7f74d7342740 1 --2- 172.21.15.16:0/782539134 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556ac294f5d0 0x556ac2962a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:46.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.149+0000 7f74d7342740 1 -- 172.21.15.16:0/782539134 >> 172.21.15.16:0/782539134 conn(0x556ac2918310 msgr2=0x556ac29557e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:46.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.149+0000 7f74d7342740 1 -- 172.21.15.16:0/782539134 shutdown_connections 2024-09-12T02:12:46.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.149+0000 7f74d7342740 1 -- 172.21.15.16:0/782539134 wait complete. 2024-09-12T02:12:46.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.150+0000 7f74d7342740 20 client.18861 trim_cache size 0 max 16384 2024-09-12T02:12:46.160 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T02:12:46.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.150+0000 7f74d4f69640 1 -- 172.21.15.16:0/2941326501 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f74b80a2b80 msgr2=0x7f74b80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:46.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.150+0000 7f74d4f69640 1 --2- 172.21.15.16:0/2941326501 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f74b80a2b80 0x7f74b80a5040 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x556ac294cac0 tx=0x7f74c000b410 comp rx=0 tx=0).stop 2024-09-12T02:12:46.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.150+0000 7f74d4f69640 1 -- 172.21.15.16:0/2941326501 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74d0101420 msgr2=0x7f74d01997f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:46.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.150+0000 7f74d4f69640 1 --2- 172.21.15.16:0/2941326501 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74d0101420 0x7f74d01997f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f74c400e9f0 tx=0x7f74c400eeb0 comp rx=0 tx=0).stop 2024-09-12T02:12:46.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.150+0000 7f74d4f69640 1 -- 172.21.15.16:0/2941326501 shutdown_connections 2024-09-12T02:12:46.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.150+0000 7f74d4f69640 1 --2- 172.21.15.16:0/2941326501 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f74b80a2b80 0x7f74b80a5040 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:46.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.150+0000 7f74d4f69640 1 --2- 172.21.15.16:0/2941326501 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74d0106a80 0x7f74d01a4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:46.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.150+0000 7f74d4f69640 1 --2- 172.21.15.16:0/2941326501 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74d0101420 0x7f74d01997f0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:46.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.150+0000 7f74d4f69640 1 --2- 172.21.15.16:0/2941326501 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f74d0100a80 0x7f74d01992b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:46.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.150+0000 7f74d4f69640 1 -- 172.21.15.16:0/2941326501 >> 172.21.15.16:0/2941326501 conn(0x7f74d00fc7a0 msgr2=0x7f74d00fce70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:46.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.150+0000 7f74d4f69640 1 -- 172.21.15.16:0/2941326501 shutdown_connections 2024-09-12T02:12:46.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:46.150+0000 7f74d4f69640 1 -- 172.21.15.16:0/2941326501 wait complete. 2024-09-12T02:12:46.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:12:46 smithi184 ceph-mon[35593]: pgmap v660: 97 pgs: 97 active+clean; 1.5 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-09-12T02:12:46.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:12:46 smithi033 ceph-mon[35332]: pgmap v660: 97 pgs: 97 active+clean; 1.5 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-09-12T02:12:46.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:12:46 smithi016 ceph-mon[30947]: pgmap v660: 97 pgs: 97 active+clean; 1.5 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-09-12T02:12:46.851 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-12T02:12:47.110 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:12:47.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.556+0000 7f8427a12640 1 -- 172.21.15.16:0/3629616043 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8420100e20 msgr2=0x7f8420101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:47.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.556+0000 7f8427a12640 1 --2- 172.21.15.16:0/3629616043 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8420100e20 0x7f8420101200 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f841c009900 tx=0x7f841c02a480 comp rx=0 tx=0).stop 2024-09-12T02:12:47.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.556+0000 7f8427a12640 1 -- 172.21.15.16:0/3629616043 shutdown_connections 2024-09-12T02:12:47.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.556+0000 7f8427a12640 1 --2- 172.21.15.16:0/3629616043 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8420102ef0 0x7f842010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:47.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.556+0000 7f8427a12640 1 --2- 172.21.15.16:0/3629616043 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8420102570 0x7f84201029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:47.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.556+0000 7f8427a12640 1 --2- 172.21.15.16:0/3629616043 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8420100e20 0x7f8420101200 unknown :-1 s=CLOSED pgs=1515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:47.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.556+0000 7f8427a12640 1 -- 172.21.15.16:0/3629616043 >> 172.21.15.16:0/3629616043 conn(0x7f84200fc7d0 msgr2=0x7f84200febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:47.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.556+0000 7f8427a12640 1 -- 172.21.15.16:0/3629616043 shutdown_connections 2024-09-12T02:12:47.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.556+0000 7f8427a12640 1 -- 172.21.15.16:0/3629616043 wait complete. 2024-09-12T02:12:47.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.556+0000 7f8427a12640 1 Processor -- start 2024-09-12T02:12:47.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.556+0000 7f8427a12640 1 -- start start 2024-09-12T02:12:47.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.557+0000 7f8427a12640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8420100e20 0x7f8420103750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:47.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.557+0000 7f8427a12640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8420102570 0x7f8420103c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:47.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.557+0000 7f8427a12640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8420102ef0 0x7f84201041d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:47.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.557+0000 7f8427a12640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8420110f80 con 0x7f8420102570 2024-09-12T02:12:47.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.557+0000 7f8427a12640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8420110e00 con 0x7f8420102ef0 2024-09-12T02:12:47.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.557+0000 7f8427a12640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8420111100 con 0x7f8420100e20 2024-09-12T02:12:47.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.557+0000 7f8424fad640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8420102570 0x7f8420103c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:47.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.557+0000 7f8424fad640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8420102570 0x7f8420103c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60964/0 (socket says 172.21.15.16:60964) 2024-09-12T02:12:47.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.557+0000 7f8424fad640 1 -- 172.21.15.16:0/2274664834 learned_addr learned my addr 172.21.15.16:0/2274664834 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:12:47.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.557+0000 7f8425faf640 1 --2- 172.21.15.16:0/2274664834 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8420102ef0 0x7f84201041d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:47.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.557+0000 7f84257ae640 1 --2- 172.21.15.16:0/2274664834 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8420100e20 0x7f8420103750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:47.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.557+0000 7f8424fad640 1 -- 172.21.15.16:0/2274664834 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8420100e20 msgr2=0x7f8420103750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:47.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.557+0000 7f8424fad640 1 --2- 172.21.15.16:0/2274664834 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8420100e20 0x7f8420103750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:47.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.557+0000 7f8424fad640 1 -- 172.21.15.16:0/2274664834 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8420102ef0 msgr2=0x7f84201041d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:47.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.557+0000 7f8424fad640 1 --2- 172.21.15.16:0/2274664834 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8420102ef0 0x7f84201041d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:47.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.557+0000 7f8424fad640 1 -- 172.21.15.16:0/2274664834 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84201a6940 con 0x7f8420102570 2024-09-12T02:12:47.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.557+0000 7f8424fad640 1 --2- 172.21.15.16:0/2274664834 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8420102570 0x7f8420103c90 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f841000d950 tx=0x7f841000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:47.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.558+0000 7f84167fc640 1 -- 172.21.15.16:0/2274664834 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8410017030 con 0x7f8420102570 2024-09-12T02:12:47.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.558+0000 7f84167fc640 1 -- 172.21.15.16:0/2274664834 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8410004020 con 0x7f8420102570 2024-09-12T02:12:47.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.558+0000 7f8427a12640 1 -- 172.21.15.16:0/2274664834 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84201a6c30 con 0x7f8420102570 2024-09-12T02:12:47.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.558+0000 7f84167fc640 1 -- 172.21.15.16:0/2274664834 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84100047a0 con 0x7f8420102570 2024-09-12T02:12:47.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.558+0000 7f8427a12640 1 -- 172.21.15.16:0/2274664834 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84201a7170 con 0x7f8420102570 2024-09-12T02:12:47.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.560+0000 7f84167fc640 1 -- 172.21.15.16:0/2274664834 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f84100041d0 con 0x7f8420102570 2024-09-12T02:12:47.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.560+0000 7f8429deb740 1 Processor -- start 2024-09-12T02:12:47.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.560+0000 7f8429deb740 1 -- start start 2024-09-12T02:12:47.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.561+0000 7f8429deb740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d11f466b40 0x55d11f465090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:47.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.561+0000 7f8429deb740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d11f4655d0 0x55d11f465a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:47.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.561+0000 7f8429deb740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d11f470370 0x55d11f472760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:47.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.561+0000 7f8429deb740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55d11f3cb2f0 con 0x55d11f466b40 2024-09-12T02:12:47.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.561+0000 7f8429deb740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55d11f372cf0 con 0x55d11f4655d0 2024-09-12T02:12:47.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.561+0000 7f8429deb740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55d11f3c72b0 con 0x55d11f470370 2024-09-12T02:12:47.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.561+0000 7f84167fc640 1 --2- 172.21.15.16:0/2274664834 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f840c0a2b80 0x7f840c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:47.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.561+0000 7f8425faf640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d11f466b40 0x55d11f465090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:47.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.561+0000 7f84257ae640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d11f4655d0 0x55d11f465a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:47.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.561+0000 7f8425faf640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d11f466b40 0x55d11f465090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60980/0 (socket says 172.21.15.16:60980) 2024-09-12T02:12:47.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.561+0000 7f84257ae640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d11f4655d0 0x55d11f465a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.16:37468/0 (socket says 172.21.15.16:37468) 2024-09-12T02:12:47.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.561+0000 7f8425faf640 1 -- 172.21.15.16:0/998983680 learned_addr learned my addr 172.21.15.16:0/998983680 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:12:47.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.561+0000 7f8424fad640 1 --2- 172.21.15.16:0/998983680 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d11f470370 0x55d11f472760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:47.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.561+0000 7f8425faf640 1 --2- 172.21.15.16:0/2274664834 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f840c0a2b80 0x7f840c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:47.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.561+0000 7f84167fc640 1 -- 172.21.15.16:0/2274664834 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f84100ce8c0 con 0x7f8420102570 2024-09-12T02:12:47.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.561+0000 7f84257ae640 1 -- 172.21.15.16:0/998983680 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d11f470370 msgr2=0x55d11f472760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:47.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.561+0000 7f84257ae640 1 --2- 172.21.15.16:0/998983680 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d11f470370 0x55d11f472760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:47.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.561+0000 7f84257ae640 1 -- 172.21.15.16:0/998983680 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d11f466b40 msgr2=0x55d11f465090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:47.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.561+0000 7f84257ae640 1 --2- 172.21.15.16:0/998983680 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d11f466b40 0x55d11f465090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:47.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.561+0000 7f84257ae640 1 -- 172.21.15.16:0/998983680 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d11f3573a0 con 0x55d11f4655d0 2024-09-12T02:12:47.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.561+0000 7f8425faf640 1 --2- 172.21.15.16:0/998983680 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d11f466b40 0x55d11f465090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:12:47.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.562+0000 7f8425faf640 1 --2- 172.21.15.16:0/2274664834 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f840c0a2b80 0x7f840c0a5040 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f8418007310 tx=0x7f8418007520 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:47.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.562+0000 7f84257ae640 1 --2- 172.21.15.16:0/998983680 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d11f4655d0 0x55d11f465a10 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f841c005940 tx=0x7f841c004170 comp rx=0 tx=0).ready entity=mon.2 client_cookie=47ee6643c940b2b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:47.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.562+0000 7f840b7fe640 1 -- 172.21.15.16:0/998983680 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f841c035030 con 0x55d11f4655d0 2024-09-12T02:12:47.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.562+0000 7f840b7fe640 1 -- 172.21.15.16:0/998983680 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f841c040070 con 0x55d11f4655d0 2024-09-12T02:12:47.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.562+0000 7f840b7fe640 1 -- 172.21.15.16:0/998983680 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f841c031b00 con 0x55d11f4655d0 2024-09-12T02:12:47.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.563+0000 7f8429deb740 1 -- 172.21.15.16:0/998983680 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d11f4655d0 msgr2=0x55d11f465a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:47.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.563+0000 7f8429deb740 1 --2- 172.21.15.16:0/998983680 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d11f4655d0 0x55d11f465a10 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f841c005940 tx=0x7f841c004170 comp rx=0 tx=0).stop 2024-09-12T02:12:47.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.563+0000 7f8429deb740 1 -- 172.21.15.16:0/998983680 shutdown_connections 2024-09-12T02:12:47.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.563+0000 7f8429deb740 1 --2- 172.21.15.16:0/998983680 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d11f470370 0x55d11f472760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:47.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.563+0000 7f8429deb740 1 --2- 172.21.15.16:0/998983680 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d11f4655d0 0x55d11f465a10 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:47.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.563+0000 7f8429deb740 1 --2- 172.21.15.16:0/998983680 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d11f466b40 0x55d11f465090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:47.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.563+0000 7f8429deb740 1 -- 172.21.15.16:0/998983680 >> 172.21.15.16:0/998983680 conn(0x55d11f42e310 msgr2=0x55d11f461c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:47.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.563+0000 7f8429deb740 1 -- 172.21.15.16:0/998983680 shutdown_connections 2024-09-12T02:12:47.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.563+0000 7f8429deb740 1 -- 172.21.15.16:0/998983680 wait complete. 2024-09-12T02:12:47.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.563+0000 7f8429deb740 1 -- start start 2024-09-12T02:12:47.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.563+0000 7f8429deb740 1 Processor -- start 2024-09-12T02:12:47.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.564+0000 7f8429deb740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d11f4655d0 0x55d11f46fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:47.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.564+0000 7f8429deb740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d11f466b40 0x55d11f46d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:47.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.564+0000 7f8429deb740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d11f470370 0x55d11f46d650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:47.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.564+0000 7f8429deb740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55d11f473010 con 0x55d11f466b40 2024-09-12T02:12:47.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.564+0000 7f8429deb740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55d11f386060 con 0x55d11f470370 2024-09-12T02:12:47.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.564+0000 7f8429deb740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55d11f473190 con 0x55d11f4655d0 2024-09-12T02:12:47.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.564+0000 7f8429deb740 10 client.0 _ll_register_callbacks cb 0x55d11f0d98c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:12:47.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.564+0000 7f8424fad640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d11f466b40 0x55d11f46d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:47.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.564+0000 7f8424fad640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d11f466b40 0x55d11f46d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60982/0 (socket says 172.21.15.16:60982) 2024-09-12T02:12:47.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.564+0000 7f8424fad640 1 -- 172.21.15.16:0/1895732699 learned_addr learned my addr 172.21.15.16:0/1895732699 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:12:47.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.564+0000 7f8425faf640 1 --2- 172.21.15.16:0/1895732699 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d11f470370 0x55d11f46d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:47.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.564+0000 7f84257ae640 1 --2- 172.21.15.16:0/1895732699 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d11f4655d0 0x55d11f46fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:47.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.564+0000 7f8424fad640 1 -- 172.21.15.16:0/1895732699 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d11f4655d0 msgr2=0x55d11f46fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:47.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.564+0000 7f8424fad640 1 --2- 172.21.15.16:0/1895732699 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d11f4655d0 0x55d11f46fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:47.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.564+0000 7f8424fad640 1 -- 172.21.15.16:0/1895732699 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d11f470370 msgr2=0x55d11f46d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:47.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.564+0000 7f8424fad640 1 --2- 172.21.15.16:0/1895732699 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d11f470370 0x55d11f46d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:47.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.564+0000 7f8424fad640 1 -- 172.21.15.16:0/1895732699 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d11f13c3c0 con 0x55d11f466b40 2024-09-12T02:12:47.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.564+0000 7f84257ae640 1 --2- 172.21.15.16:0/1895732699 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d11f4655d0 0x55d11f46fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:12:47.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.564+0000 7f8425faf640 1 --2- 172.21.15.16:0/1895732699 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d11f470370 0x55d11f46d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:12:47.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.564+0000 7f8424fad640 1 --2- 172.21.15.16:0/1895732699 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d11f466b40 0x55d11f46d040 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f841009aec0 tx=0x7f841009b0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3628f25e9bdd25ab server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:47.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.564+0000 7f8408ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:12:47.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.564+0000 7f8408ff9640 1 -- 172.21.15.16:0/1895732699 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84100b4060 con 0x55d11f466b40 2024-09-12T02:12:47.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.565+0000 7f8408ff9640 1 -- 172.21.15.16:0/1895732699 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f8410097830 con 0x55d11f466b40 2024-09-12T02:12:47.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.565+0000 7f8429deb740 1 -- 172.21.15.16:0/1895732699 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f841c040070 con 0x55d11f466b40 2024-09-12T02:12:47.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.565+0000 7f8408ff9640 1 -- 172.21.15.16:0/1895732699 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f841009b1c0 con 0x55d11f466b40 2024-09-12T02:12:47.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.565+0000 7f8429deb740 10 client.18873 fetch_fsmap learned FSMap version 25 2024-09-12T02:12:47.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.565+0000 7f8429deb740 1 -- 172.21.15.16:0/1895732699 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d11f47eae0 con 0x55d11f466b40 2024-09-12T02:12:47.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.565+0000 7f8408ff9640 1 -- 172.21.15.16:0/1895732699 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f841c040070 con 0x55d11f466b40 2024-09-12T02:12:47.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.565+0000 7f8408ff9640 1 -- 172.21.15.16:0/1895732699 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f84100bb0b0 con 0x55d11f466b40 2024-09-12T02:12:47.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.565+0000 7f8429deb740 10 client.18873 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:12:47.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.565+0000 7f8429deb740 10 client.18873 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:12:47.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.565+0000 7f8429deb740 20 client.18873 populate_metadata read hostname 'smithi016' 2024-09-12T02:12:47.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.565+0000 7f8429deb740 1 --2- 172.21.15.16:0/1895732699 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55d11f47ffa0 0x55d11f482390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:47.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.565+0000 7f8429deb740 4 client.18873 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:12:47.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.565+0000 7f8429deb740 1 -- 172.21.15.16:0/1895732699 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d11f3d8860 con 0x55d11f47ffa0 2024-09-12T02:12:47.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.565+0000 7f84257ae640 1 --2- 172.21.15.16:0/1895732699 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55d11f47ffa0 0x55d11f482390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:47.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.566+0000 7f8408ff9640 10 client.18873 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:12:47.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.566+0000 7f84257ae640 1 --2- 172.21.15.16:0/1895732699 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55d11f47ffa0 0x55d11f482390 crc :-1 s=READY pgs=320 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b799f175d911f7a1 server_cookie=cb511140c1465f2f in_seq=0 out_seq=0 2024-09-12T02:12:47.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.566+0000 7f8408ff9640 1 -- 172.21.15.16:0/1895732699 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d11f3d8860 con 0x55d11f47ffa0 2024-09-12T02:12:47.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.566+0000 7f8408ff9640 10 client.18873 handle_command_reply: tid=0 2024-09-12T02:12:47.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.567+0000 7f8429deb740 1 client.18873 shutdown 2024-09-12T02:12:47.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.567+0000 7f8429deb740 1 -- 172.21.15.16:0/1895732699 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d11f466b40 msgr2=0x55d11f46d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:47.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.567+0000 7f8429deb740 1 --2- 172.21.15.16:0/1895732699 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d11f466b40 0x55d11f46d040 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f841009aec0 tx=0x7f841009b0c0 comp rx=0 tx=0).stop 2024-09-12T02:12:47.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.567+0000 7f8429deb740 1 -- 172.21.15.16:0/1895732699 shutdown_connections 2024-09-12T02:12:47.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.567+0000 7f8429deb740 1 --2- 172.21.15.16:0/1895732699 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55d11f47ffa0 0x55d11f482390 crc :-1 s=READY pgs=320 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:47.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.567+0000 7f8429deb740 1 --2- 172.21.15.16:0/1895732699 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d11f470370 0x55d11f46d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:47.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.567+0000 7f8429deb740 1 --2- 172.21.15.16:0/1895732699 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d11f466b40 0x55d11f46d040 unknown :-1 s=CLOSED pgs=1517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:47.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.567+0000 7f8429deb740 1 --2- 172.21.15.16:0/1895732699 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d11f4655d0 0x55d11f46fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:47.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.567+0000 7f8429deb740 1 -- 172.21.15.16:0/1895732699 >> 172.21.15.16:0/1895732699 conn(0x55d11f42e310 msgr2=0x55d11f4679c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:47.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.567+0000 7f8408ff9640 0 client.18873 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:12:47.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.567+0000 7f8429deb740 1 -- 172.21.15.16:0/1895732699 shutdown_connections 2024-09-12T02:12:47.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.567+0000 7f8429deb740 1 -- 172.21.15.16:0/1895732699 wait complete. 2024-09-12T02:12:47.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.567+0000 7f8429deb740 20 client.18873 trim_cache size 0 max 16384 2024-09-12T02:12:47.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.580+0000 7f8429deb740 1 Processor -- start 2024-09-12T02:12:47.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.580+0000 7f8429deb740 1 -- start start 2024-09-12T02:12:47.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.580+0000 7f8429deb740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d11f4655d0 0x55d11f461e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:47.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.580+0000 7f8429deb740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d11f466b40 0x55d11f47c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:47.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.580+0000 7f8429deb740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d11f470370 0x55d11f4681b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:47.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.580+0000 7f8429deb740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f84201b1f00 con 0x55d11f466b40 2024-09-12T02:12:47.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.580+0000 7f8429deb740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f84201af7d0 con 0x55d11f4655d0 2024-09-12T02:12:47.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.580+0000 7f8429deb740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f84201ad620 con 0x55d11f470370 2024-09-12T02:12:47.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.580+0000 7f8424fad640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d11f466b40 0x55d11f47c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:47.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.580+0000 7f8424fad640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d11f466b40 0x55d11f47c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60996/0 (socket says 172.21.15.16:60996) 2024-09-12T02:12:47.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.580+0000 7f8424fad640 1 -- 172.21.15.16:0/622604927 learned_addr learned my addr 172.21.15.16:0/622604927 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:12:47.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.580+0000 7f8425faf640 1 --2- 172.21.15.16:0/622604927 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d11f470370 0x55d11f4681b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:47.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.580+0000 7f84257ae640 1 --2- 172.21.15.16:0/622604927 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d11f4655d0 0x55d11f461e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:47.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.580+0000 7f8424fad640 1 -- 172.21.15.16:0/622604927 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d11f470370 msgr2=0x55d11f4681b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:47.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.581+0000 7f8424fad640 1 --2- 172.21.15.16:0/622604927 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d11f470370 0x55d11f4681b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:47.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.581+0000 7f8424fad640 1 -- 172.21.15.16:0/622604927 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d11f4655d0 msgr2=0x55d11f461e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:47.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.581+0000 7f8424fad640 1 --2- 172.21.15.16:0/622604927 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d11f4655d0 0x55d11f461e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:47.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.581+0000 7f8424fad640 1 -- 172.21.15.16:0/622604927 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d11f46b5f0 con 0x55d11f466b40 2024-09-12T02:12:47.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.581+0000 7f8424fad640 1 --2- 172.21.15.16:0/622604927 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d11f466b40 0x55d11f47c2b0 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f84100a3900 tx=0x7f8410005bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a7b8919dee05985 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:47.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.581+0000 7f840affd640 1 -- 172.21.15.16:0/622604927 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f841009ab20 con 0x55d11f466b40 2024-09-12T02:12:47.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.581+0000 7f840affd640 1 -- 172.21.15.16:0/622604927 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f841009f170 con 0x55d11f466b40 2024-09-12T02:12:47.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.581+0000 7f840affd640 1 -- 172.21.15.16:0/622604927 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f841009a780 con 0x55d11f466b40 2024-09-12T02:12:47.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.581+0000 7f8429deb740 1 -- 172.21.15.16:0/622604927 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d11f466b40 msgr2=0x55d11f47c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:47.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.581+0000 7f8429deb740 1 --2- 172.21.15.16:0/622604927 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d11f466b40 0x55d11f47c2b0 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f84100a3900 tx=0x7f8410005bf0 comp rx=0 tx=0).stop 2024-09-12T02:12:47.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.582+0000 7f8429deb740 1 -- 172.21.15.16:0/622604927 shutdown_connections 2024-09-12T02:12:47.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.582+0000 7f8429deb740 1 --2- 172.21.15.16:0/622604927 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d11f470370 0x55d11f4681b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:47.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.582+0000 7f8429deb740 1 --2- 172.21.15.16:0/622604927 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d11f466b40 0x55d11f47c2b0 unknown :-1 s=CLOSED pgs=1518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:47.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.582+0000 7f8429deb740 1 --2- 172.21.15.16:0/622604927 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d11f4655d0 0x55d11f461e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:47.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.582+0000 7f8429deb740 1 -- 172.21.15.16:0/622604927 >> 172.21.15.16:0/622604927 conn(0x55d11f42e310 msgr2=0x55d11f461700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:47.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.582+0000 7f8429deb740 1 -- 172.21.15.16:0/622604927 shutdown_connections 2024-09-12T02:12:47.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.582+0000 7f8429deb740 1 -- 172.21.15.16:0/622604927 wait complete. 2024-09-12T02:12:47.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.582+0000 7f8429deb740 1 -- start start 2024-09-12T02:12:47.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.582+0000 7f8429deb740 1 Processor -- start 2024-09-12T02:12:47.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.582+0000 7f8429deb740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d11f4655d0 0x55d11f4787d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:47.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.582+0000 7f8429deb740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d11f466b40 0x55d11f478d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:47.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.582+0000 7f8429deb740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d11f470370 0x55d11f46a520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:47.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.582+0000 7f8429deb740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f84201adc90 con 0x55d11f4655d0 2024-09-12T02:12:47.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.582+0000 7f8429deb740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f84201af100 con 0x55d11f466b40 2024-09-12T02:12:47.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.582+0000 7f8429deb740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f84201ae360 con 0x55d11f470370 2024-09-12T02:12:47.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.583+0000 7f8429deb740 10 client.0 _ll_register_callbacks cb 0x7f8410097550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:12:47.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.583+0000 7f84257ae640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d11f4655d0 0x55d11f4787d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:47.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.583+0000 7f8424fad640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d11f466b40 0x55d11f478d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:47.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.583+0000 7f84257ae640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d11f4655d0 0x55d11f4787d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60998/0 (socket says 172.21.15.16:60998) 2024-09-12T02:12:47.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.583+0000 7f8424fad640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d11f466b40 0x55d11f478d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.16:37494/0 (socket says 172.21.15.16:37494) 2024-09-12T02:12:47.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.583+0000 7f84257ae640 1 -- 172.21.15.16:0/4061881165 learned_addr learned my addr 172.21.15.16:0/4061881165 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:12:47.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.583+0000 7f8425faf640 1 --2- 172.21.15.16:0/4061881165 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d11f470370 0x55d11f46a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:47.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.583+0000 7f8424fad640 1 -- 172.21.15.16:0/4061881165 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d11f470370 msgr2=0x55d11f46a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:47.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.583+0000 7f8424fad640 1 --2- 172.21.15.16:0/4061881165 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d11f470370 0x55d11f46a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:47.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.583+0000 7f8424fad640 1 -- 172.21.15.16:0/4061881165 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d11f4655d0 msgr2=0x55d11f4787d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:47.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.583+0000 7f8424fad640 1 --2- 172.21.15.16:0/4061881165 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d11f4655d0 0x55d11f4787d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:47.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.583+0000 7f8424fad640 1 -- 172.21.15.16:0/4061881165 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d11f46ad50 con 0x55d11f466b40 2024-09-12T02:12:47.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.583+0000 7f84257ae640 1 --2- 172.21.15.16:0/4061881165 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d11f4655d0 0x55d11f4787d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:12:47.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.583+0000 7f8424fad640 1 --2- 172.21.15.16:0/4061881165 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d11f466b40 0x55d11f478d10 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x55d11f473190 tx=0x7f84100bb300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=17a701f74c4891f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:47.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.583+0000 7f8408ff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-12T02:12:47.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.583+0000 7f8408ff9640 1 -- 172.21.15.16:0/4061881165 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84100b4040 con 0x55d11f466b40 2024-09-12T02:12:47.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.584+0000 7f8429deb740 1 -- 172.21.15.16:0/4061881165 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f841009f170 con 0x55d11f466b40 2024-09-12T02:12:47.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.584+0000 7f8408ff9640 1 -- 172.21.15.16:0/4061881165 <== mon.2 v2:172.21.15.33:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f84100b9e00 con 0x55d11f466b40 2024-09-12T02:12:47.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.584+0000 7f8408ff9640 1 -- 172.21.15.16:0/4061881165 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f841009ab20 con 0x55d11f466b40 2024-09-12T02:12:47.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.584+0000 7f8429deb740 10 client.27932 fetch_fsmap learned FSMap version 25 2024-09-12T02:12:47.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.584+0000 7f8429deb740 1 -- 172.21.15.16:0/4061881165 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d11f487fb0 con 0x55d11f466b40 2024-09-12T02:12:47.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.584+0000 7f8408ff9640 1 -- 172.21.15.16:0/4061881165 <== mon.2 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f841009f170 con 0x55d11f466b40 2024-09-12T02:12:47.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.584+0000 7f8408ff9640 1 -- 172.21.15.16:0/4061881165 <== mon.2 v2:172.21.15.33:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f8410082030 con 0x55d11f466b40 2024-09-12T02:12:47.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.584+0000 7f8429deb740 10 client.27932 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:12:47.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.584+0000 7f8429deb740 10 client.27932 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:12:47.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.584+0000 7f8429deb740 20 client.27932 populate_metadata read hostname 'smithi016' 2024-09-12T02:12:47.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.584+0000 7f8429deb740 1 --2- 172.21.15.16:0/4061881165 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55d11f4eade0 0x55d11f4ed1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:47.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.584+0000 7f8429deb740 4 client.27932 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T02:12:47.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.584+0000 7f8429deb740 1 -- 172.21.15.16:0/4061881165 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d11f3d2d70 con 0x55d11f4eade0 2024-09-12T02:12:47.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.584+0000 7f84257ae640 1 --2- 172.21.15.16:0/4061881165 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55d11f4eade0 0x55d11f4ed1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:47.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.585+0000 7f84257ae640 1 --2- 172.21.15.16:0/4061881165 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55d11f4eade0 0x55d11f4ed1d0 crc :-1 s=READY pgs=321 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30f60d2a0fdd01a9 server_cookie=d1a1634cae7a4121 in_seq=0 out_seq=0 2024-09-12T02:12:47.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.585+0000 7f8408ff9640 10 client.27932 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:12:47.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.585+0000 7f8408ff9640 1 -- 172.21.15.16:0/4061881165 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d11f3d2d70 con 0x55d11f4eade0 2024-09-12T02:12:47.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.585+0000 7f8408ff9640 10 client.27932 handle_command_reply: tid=0 2024-09-12T02:12:47.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.585+0000 7f8429deb740 1 client.27932 shutdown 2024-09-12T02:12:47.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.585+0000 7f8429deb740 1 -- 172.21.15.16:0/4061881165 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d11f466b40 msgr2=0x55d11f478d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:47.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.585+0000 7f8429deb740 1 --2- 172.21.15.16:0/4061881165 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d11f466b40 0x55d11f478d10 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x55d11f473190 tx=0x7f84100bb300 comp rx=0 tx=0).stop 2024-09-12T02:12:47.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.585+0000 7f8429deb740 1 -- 172.21.15.16:0/4061881165 shutdown_connections 2024-09-12T02:12:47.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.585+0000 7f8429deb740 1 --2- 172.21.15.16:0/4061881165 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55d11f4eade0 0x55d11f4ed1d0 crc :-1 s=READY pgs=321 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:47.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.585+0000 7f8429deb740 1 --2- 172.21.15.16:0/4061881165 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d11f470370 0x55d11f46a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:47.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.585+0000 7f8429deb740 1 --2- 172.21.15.16:0/4061881165 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d11f466b40 0x55d11f478d10 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:47.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.585+0000 7f8408ff9640 0 client.27932 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:12:47.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.585+0000 7f8429deb740 1 --2- 172.21.15.16:0/4061881165 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d11f4655d0 0x55d11f4787d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:47.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.585+0000 7f8429deb740 1 -- 172.21.15.16:0/4061881165 >> 172.21.15.16:0/4061881165 conn(0x55d11f42e310 msgr2=0x55d11f4604e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:47.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.585+0000 7f8429deb740 1 -- 172.21.15.16:0/4061881165 shutdown_connections 2024-09-12T02:12:47.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.585+0000 7f8429deb740 1 -- 172.21.15.16:0/4061881165 wait complete. 2024-09-12T02:12:47.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.586+0000 7f8429deb740 20 client.27932 trim_cache size 0 max 16384 2024-09-12T02:12:47.592 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T02:12:47.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.586+0000 7f8427a12640 1 -- 172.21.15.16:0/2274664834 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f840c0a2b80 msgr2=0x7f840c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:47.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.586+0000 7f8427a12640 1 --2- 172.21.15.16:0/2274664834 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f840c0a2b80 0x7f840c0a5040 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f8418007310 tx=0x7f8418007520 comp rx=0 tx=0).stop 2024-09-12T02:12:47.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.586+0000 7f8427a12640 1 -- 172.21.15.16:0/2274664834 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8420102570 msgr2=0x7f8420103c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:47.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.586+0000 7f8427a12640 1 --2- 172.21.15.16:0/2274664834 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8420102570 0x7f8420103c90 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f841000d950 tx=0x7f841000de10 comp rx=0 tx=0).stop 2024-09-12T02:12:47.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.586+0000 7f8427a12640 1 -- 172.21.15.16:0/2274664834 shutdown_connections 2024-09-12T02:12:47.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.586+0000 7f8427a12640 1 --2- 172.21.15.16:0/2274664834 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f840c0a2b80 0x7f840c0a5040 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:47.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.586+0000 7f8427a12640 1 --2- 172.21.15.16:0/2274664834 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8420102ef0 0x7f84201041d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:47.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.586+0000 7f8427a12640 1 --2- 172.21.15.16:0/2274664834 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8420102570 0x7f8420103c90 unknown :-1 s=CLOSED pgs=1516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:47.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.586+0000 7f8427a12640 1 --2- 172.21.15.16:0/2274664834 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8420100e20 0x7f8420103750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:47.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.586+0000 7f8427a12640 1 -- 172.21.15.16:0/2274664834 >> 172.21.15.16:0/2274664834 conn(0x7f84200fc7d0 msgr2=0x7f842006a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:47.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.586+0000 7f8427a12640 1 -- 172.21.15.16:0/2274664834 shutdown_connections 2024-09-12T02:12:47.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:47.586+0000 7f8427a12640 1 -- 172.21.15.16:0/2274664834 wait complete. 2024-09-12T02:12:48.326 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:12:48 smithi016 ceph-mon[30947]: pgmap v661: 97 pgs: 97 active+clean; 1.6 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 17 op/s 2024-09-12T02:12:48.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:12:48 smithi184 ceph-mon[35593]: pgmap v661: 97 pgs: 97 active+clean; 1.6 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 17 op/s 2024-09-12T02:12:48.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:12:48 smithi033 ceph-mon[35332]: pgmap v661: 97 pgs: 97 active+clean; 1.6 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 17 op/s 2024-09-12T02:12:49.359 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-12T02:12:49.359 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-12T02:12:49.633 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:12:50.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.091+0000 7ff8185d5640 1 -- 172.21.15.16:0/2607591615 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff810103660 msgr2=0x7ff81010bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:50.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.091+0000 7ff8185d5640 1 --2- 172.21.15.16:0/2607591615 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff810103660 0x7ff81010bb50 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7ff804009900 tx=0x7ff80402a480 comp rx=0 tx=0).stop 2024-09-12T02:12:50.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.092+0000 7ff8185d5640 1 -- 172.21.15.16:0/2607591615 shutdown_connections 2024-09-12T02:12:50.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.092+0000 7ff8185d5640 1 --2- 172.21.15.16:0/2607591615 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff81010c170 0x7ff81010e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:50.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.092+0000 7ff8185d5640 1 --2- 172.21.15.16:0/2607591615 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff810103660 0x7ff81010bb50 unknown :-1 s=CLOSED pgs=1519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:50.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.092+0000 7ff8185d5640 1 --2- 172.21.15.16:0/2607591615 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff810102d40 0x7ff810103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:50.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.092+0000 7ff8185d5640 1 -- 172.21.15.16:0/2607591615 >> 172.21.15.16:0/2607591615 conn(0x7ff8100fc7a0 msgr2=0x7ff8100febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:50.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.092+0000 7ff8185d5640 1 -- 172.21.15.16:0/2607591615 shutdown_connections 2024-09-12T02:12:50.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.092+0000 7ff8185d5640 1 -- 172.21.15.16:0/2607591615 wait complete. 2024-09-12T02:12:50.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.092+0000 7ff8185d5640 1 Processor -- start 2024-09-12T02:12:50.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.093+0000 7ff8185d5640 1 -- start start 2024-09-12T02:12:50.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.093+0000 7ff8185d5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff810102d40 0x7ff810198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:50.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.093+0000 7ff8185d5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff810103660 0x7ff810198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:50.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.093+0000 7ff8185d5640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff81010c170 0x7ff81019cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:50.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.093+0000 7ff8185d5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff810111f70 con 0x7ff810102d40 2024-09-12T02:12:50.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.093+0000 7ff8185d5640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff810111df0 con 0x7ff810103660 2024-09-12T02:12:50.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.093+0000 7ff8185d5640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff8101120f0 con 0x7ff81010c170 2024-09-12T02:12:50.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.093+0000 7ff816371640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff810102d40 0x7ff810198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:50.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.093+0000 7ff816371640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff810102d40 0x7ff810198410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32776/0 (socket says 172.21.15.16:32776) 2024-09-12T02:12:50.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.093+0000 7ff816371640 1 -- 172.21.15.16:0/248903627 learned_addr learned my addr 172.21.15.16:0/248903627 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:12:50.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.093+0000 7ff816b72640 1 --2- 172.21.15.16:0/248903627 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff81010c170 0x7ff81019cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:50.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.093+0000 7ff815b70640 1 --2- 172.21.15.16:0/248903627 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff810103660 0x7ff810198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:50.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.093+0000 7ff816371640 1 -- 172.21.15.16:0/248903627 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff81010c170 msgr2=0x7ff81019cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:50.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.093+0000 7ff816371640 1 --2- 172.21.15.16:0/248903627 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff81010c170 0x7ff81019cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:50.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.093+0000 7ff816371640 1 -- 172.21.15.16:0/248903627 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff810103660 msgr2=0x7ff810198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:50.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.093+0000 7ff816371640 1 --2- 172.21.15.16:0/248903627 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff810103660 0x7ff810198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:50.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.093+0000 7ff816371640 1 -- 172.21.15.16:0/248903627 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff81019d3c0 con 0x7ff810102d40 2024-09-12T02:12:50.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.094+0000 7ff816371640 1 --2- 172.21.15.16:0/248903627 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff810102d40 0x7ff810198410 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7ff80c00d950 tx=0x7ff80c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:50.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.094+0000 7ff8037fe640 1 -- 172.21.15.16:0/248903627 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff80c017070 con 0x7ff810102d40 2024-09-12T02:12:50.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.094+0000 7ff8185d5640 1 -- 172.21.15.16:0/248903627 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff81019d6b0 con 0x7ff810102d40 2024-09-12T02:12:50.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.094+0000 7ff8037fe640 1 -- 172.21.15.16:0/248903627 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff80c009c30 con 0x7ff810102d40 2024-09-12T02:12:50.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.094+0000 7ff8037fe640 1 -- 172.21.15.16:0/248903627 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff80c00f6b0 con 0x7ff810102d40 2024-09-12T02:12:50.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.094+0000 7ff8185d5640 1 -- 172.21.15.16:0/248903627 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8101a4f90 con 0x7ff810102d40 2024-09-12T02:12:50.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.096+0000 7ff8037fe640 1 -- 172.21.15.16:0/248903627 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff80c0109d0 con 0x7ff810102d40 2024-09-12T02:12:50.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.096+0000 7ff81a9ae740 1 Processor -- start 2024-09-12T02:12:50.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.097+0000 7ff81a9ae740 1 -- start start 2024-09-12T02:12:50.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.097+0000 7ff81a9ae740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562c923d3b60 0x562c923d3f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:50.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.097+0000 7ff81a9ae740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c923d4480 0x562c923de800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:50.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.097+0000 7ff81a9ae740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562c923ded40 0x562c923e1130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:50.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.097+0000 7ff81a9ae740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x562c92339280 con 0x562c923d4480 2024-09-12T02:12:50.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.097+0000 7ff81a9ae740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x562c922e0b10 con 0x562c923d3b60 2024-09-12T02:12:50.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.097+0000 7ff81a9ae740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x562c92335240 con 0x562c923ded40 2024-09-12T02:12:50.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.097+0000 7ff816371640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c923d4480 0x562c923de800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:50.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.097+0000 7ff8037fe640 1 --2- 172.21.15.16:0/248903627 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7ff7fc0a2b80 0x7ff7fc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:50.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.097+0000 7ff816371640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c923d4480 0x562c923de800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32786/0 (socket says 172.21.15.16:32786) 2024-09-12T02:12:50.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.097+0000 7ff816371640 1 -- 172.21.15.16:0/3081647580 learned_addr learned my addr 172.21.15.16:0/3081647580 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:12:50.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.097+0000 7ff815b70640 1 --2- 172.21.15.16:0/3081647580 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562c923ded40 0x562c923e1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:50.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.097+0000 7ff816b72640 1 --2- 172.21.15.16:0/3081647580 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562c923d3b60 0x562c923d3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:50.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.097+0000 7ff816b72640 1 --2- 172.21.15.16:0/248903627 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7ff7fc0a2b80 0x7ff7fc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:50.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.097+0000 7ff8037fe640 1 -- 172.21.15.16:0/248903627 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7ff80c0cec20 con 0x7ff810102d40 2024-09-12T02:12:50.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.098+0000 7ff815b70640 1 -- 172.21.15.16:0/3081647580 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562c923d3b60 msgr2=0x562c923d3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:50.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.098+0000 7ff815b70640 1 --2- 172.21.15.16:0/3081647580 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562c923d3b60 0x562c923d3f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:50.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.098+0000 7ff815b70640 1 -- 172.21.15.16:0/3081647580 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c923d4480 msgr2=0x562c923de800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:50.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.098+0000 7ff815b70640 1 --2- 172.21.15.16:0/3081647580 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c923d4480 0x562c923de800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:50.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.098+0000 7ff815b70640 1 -- 172.21.15.16:0/3081647580 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c922ed1f0 con 0x562c923ded40 2024-09-12T02:12:50.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.098+0000 7ff815b70640 1 --2- 172.21.15.16:0/3081647580 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562c923ded40 0x562c923e1130 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff804005940 tx=0x7ff804005e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d368679209310f8b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:50.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.098+0000 7ff816b72640 1 --2- 172.21.15.16:0/248903627 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7ff7fc0a2b80 0x7ff7fc0a5040 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7ff8080077f0 tx=0x7ff808007520 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:50.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.098+0000 7ff800ff9640 1 -- 172.21.15.16:0/3081647580 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff804040070 con 0x562c923ded40 2024-09-12T02:12:50.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.098+0000 7ff800ff9640 1 -- 172.21.15.16:0/3081647580 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff8040319f0 con 0x562c923ded40 2024-09-12T02:12:50.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.098+0000 7ff800ff9640 1 -- 172.21.15.16:0/3081647580 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff804035040 con 0x562c923ded40 2024-09-12T02:12:50.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.099+0000 7ff81a9ae740 1 -- 172.21.15.16:0/3081647580 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562c923ded40 msgr2=0x562c923e1130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:50.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.099+0000 7ff81a9ae740 1 --2- 172.21.15.16:0/3081647580 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562c923ded40 0x562c923e1130 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff804005940 tx=0x7ff804005e80 comp rx=0 tx=0).stop 2024-09-12T02:12:50.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.099+0000 7ff81a9ae740 1 -- 172.21.15.16:0/3081647580 shutdown_connections 2024-09-12T02:12:50.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.099+0000 7ff81a9ae740 1 --2- 172.21.15.16:0/3081647580 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562c923ded40 0x562c923e1130 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:50.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.099+0000 7ff81a9ae740 1 --2- 172.21.15.16:0/3081647580 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c923d4480 0x562c923de800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:50.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.099+0000 7ff81a9ae740 1 --2- 172.21.15.16:0/3081647580 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562c923d3b60 0x562c923d3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:50.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.099+0000 7ff81a9ae740 1 -- 172.21.15.16:0/3081647580 >> 172.21.15.16:0/3081647580 conn(0x562c923d00f0 msgr2=0x562c923d04d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:50.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.099+0000 7ff81a9ae740 1 -- 172.21.15.16:0/3081647580 shutdown_connections 2024-09-12T02:12:50.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.099+0000 7ff81a9ae740 1 -- 172.21.15.16:0/3081647580 wait complete. 2024-09-12T02:12:50.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.099+0000 7ff81a9ae740 1 -- start start 2024-09-12T02:12:50.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.099+0000 7ff81a9ae740 1 Processor -- start 2024-09-12T02:12:50.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.100+0000 7ff81a9ae740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c923d3b60 0x562c923dbba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:50.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.100+0000 7ff81a9ae740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562c923d4480 0x562c923dc0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:50.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.100+0000 7ff81a9ae740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562c923ded40 0x562c923dc620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:50.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.100+0000 7ff81a9ae740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x562c923e1910 con 0x562c923d3b60 2024-09-12T02:12:50.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.100+0000 7ff81a9ae740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x562c922f3f10 con 0x562c923d4480 2024-09-12T02:12:50.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.100+0000 7ff81a9ae740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x562c923e1a90 con 0x562c923ded40 2024-09-12T02:12:50.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.100+0000 7ff81a9ae740 10 client.0 _ll_register_callbacks cb 0x562c922bca40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:12:50.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.100+0000 7ff816371640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c923d3b60 0x562c923dbba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:50.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.100+0000 7ff816371640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c923d3b60 0x562c923dbba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32792/0 (socket says 172.21.15.16:32792) 2024-09-12T02:12:50.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.100+0000 7ff816371640 1 -- 172.21.15.16:0/2241570276 learned_addr learned my addr 172.21.15.16:0/2241570276 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:12:50.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.100+0000 7ff815b70640 1 --2- 172.21.15.16:0/2241570276 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562c923d4480 0x562c923dc0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:50.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.100+0000 7ff816b72640 1 --2- 172.21.15.16:0/2241570276 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562c923ded40 0x562c923dc620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:50.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.100+0000 7ff816371640 1 -- 172.21.15.16:0/2241570276 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562c923ded40 msgr2=0x562c923dc620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:50.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.100+0000 7ff816371640 1 --2- 172.21.15.16:0/2241570276 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562c923ded40 0x562c923dc620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:50.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.100+0000 7ff816371640 1 -- 172.21.15.16:0/2241570276 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562c923d4480 msgr2=0x562c923dc0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:50.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.100+0000 7ff816371640 1 --2- 172.21.15.16:0/2241570276 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562c923d4480 0x562c923dc0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:50.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.100+0000 7ff816371640 1 -- 172.21.15.16:0/2241570276 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c920aa6a0 con 0x562c923d3b60 2024-09-12T02:12:50.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.100+0000 7ff816b72640 1 --2- 172.21.15.16:0/2241570276 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562c923ded40 0x562c923dc620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:12:50.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.100+0000 7ff816371640 1 --2- 172.21.15.16:0/2241570276 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c923d3b60 0x562c923dbba0 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7ff80c097f00 tx=0x7ff80c00bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64d896837ba27c33 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:50.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.100+0000 7ff7f9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:12:50.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.100+0000 7ff7f9ffb640 1 -- 172.21.15.16:0/2241570276 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff80c020070 con 0x562c923d3b60 2024-09-12T02:12:50.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.100+0000 7ff7f9ffb640 1 -- 172.21.15.16:0/2241570276 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7ff80c0ccb40 con 0x562c923d3b60 2024-09-12T02:12:50.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.100+0000 7ff81a9ae740 1 -- 172.21.15.16:0/2241570276 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff8040319f0 con 0x562c923d3b60 2024-09-12T02:12:50.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.101+0000 7ff7f9ffb640 1 -- 172.21.15.16:0/2241570276 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff80c09b530 con 0x562c923d3b60 2024-09-12T02:12:50.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.101+0000 7ff81a9ae740 10 client.18891 fetch_fsmap learned FSMap version 25 2024-09-12T02:12:50.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.101+0000 7ff81a9ae740 1 -- 172.21.15.16:0/2241570276 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x562c923ed4a0 con 0x562c923d3b60 2024-09-12T02:12:50.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.101+0000 7ff7f9ffb640 1 -- 172.21.15.16:0/2241570276 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff8040319f0 con 0x562c923d3b60 2024-09-12T02:12:50.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.101+0000 7ff7f9ffb640 1 -- 172.21.15.16:0/2241570276 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7ff80c09aa80 con 0x562c923d3b60 2024-09-12T02:12:50.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.101+0000 7ff81a9ae740 10 client.18891 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:12:50.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.101+0000 7ff81a9ae740 10 client.18891 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:12:50.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.101+0000 7ff81a9ae740 20 client.18891 populate_metadata read hostname 'smithi016' 2024-09-12T02:12:50.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.101+0000 7ff81a9ae740 1 --2- 172.21.15.16:0/2241570276 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x562c923ee960 0x562c923f0d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:50.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.101+0000 7ff81a9ae740 4 client.18891 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:12:50.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.101+0000 7ff81a9ae740 1 -- 172.21.15.16:0/2241570276 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562c923467f0 con 0x562c923ee960 2024-09-12T02:12:50.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.101+0000 7ff815b70640 1 --2- 172.21.15.16:0/2241570276 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x562c923ee960 0x562c923f0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:50.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.102+0000 7ff815b70640 1 --2- 172.21.15.16:0/2241570276 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x562c923ee960 0x562c923f0d70 crc :-1 s=READY pgs=322 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b19bacd90075c519 server_cookie=8e304c5f510d4f80 in_seq=0 out_seq=0 2024-09-12T02:12:50.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.102+0000 7ff7f9ffb640 10 client.18891 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:12:50.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.102+0000 7ff7f9ffb640 1 -- 172.21.15.16:0/2241570276 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562c923467f0 con 0x562c923ee960 2024-09-12T02:12:50.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.102+0000 7ff7f9ffb640 10 client.18891 handle_command_reply: tid=0 2024-09-12T02:12:50.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.102+0000 7ff81a9ae740 1 client.18891 shutdown 2024-09-12T02:12:50.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.103+0000 7ff81a9ae740 1 -- 172.21.15.16:0/2241570276 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c923d3b60 msgr2=0x562c923dbba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:50.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.103+0000 7ff81a9ae740 1 --2- 172.21.15.16:0/2241570276 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c923d3b60 0x562c923dbba0 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7ff80c097f00 tx=0x7ff80c00bf60 comp rx=0 tx=0).stop 2024-09-12T02:12:50.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.103+0000 7ff81a9ae740 1 -- 172.21.15.16:0/2241570276 shutdown_connections 2024-09-12T02:12:50.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.103+0000 7ff81a9ae740 1 --2- 172.21.15.16:0/2241570276 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x562c923ee960 0x562c923f0d70 crc :-1 s=READY pgs=322 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:50.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.103+0000 7ff81a9ae740 1 --2- 172.21.15.16:0/2241570276 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562c923ded40 0x562c923dc620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:50.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.103+0000 7ff81a9ae740 1 --2- 172.21.15.16:0/2241570276 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562c923d4480 0x562c923dc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:50.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.103+0000 7ff81a9ae740 1 --2- 172.21.15.16:0/2241570276 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c923d3b60 0x562c923dbba0 unknown :-1 s=CLOSED pgs=1521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:50.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.103+0000 7ff7f9ffb640 0 client.18891 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:12:50.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.103+0000 7ff81a9ae740 1 -- 172.21.15.16:0/2241570276 >> 172.21.15.16:0/2241570276 conn(0x562c923d00f0 msgr2=0x562c923d5e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:50.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.103+0000 7ff81a9ae740 1 -- 172.21.15.16:0/2241570276 shutdown_connections 2024-09-12T02:12:50.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.103+0000 7ff81a9ae740 1 -- 172.21.15.16:0/2241570276 wait complete. 2024-09-12T02:12:50.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.103+0000 7ff81a9ae740 20 client.18891 trim_cache size 0 max 16384 2024-09-12T02:12:50.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.120+0000 7ff81a9ae740 1 Processor -- start 2024-09-12T02:12:50.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.121+0000 7ff81a9ae740 1 -- start start 2024-09-12T02:12:50.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.121+0000 7ff81a9ae740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c923d3b60 0x562c923cf160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:50.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.121+0000 7ff81a9ae740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562c923d4480 0x562c923f6610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:50.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.121+0000 7ff81a9ae740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562c923ded40 0x562c923f6b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:50.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.121+0000 7ff81a9ae740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff8101adaa0 con 0x562c923d3b60 2024-09-12T02:12:50.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.121+0000 7ff81a9ae740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff8101ab370 con 0x562c923ded40 2024-09-12T02:12:50.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.121+0000 7ff81a9ae740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff8101a9200 con 0x562c923d4480 2024-09-12T02:12:50.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.121+0000 7ff816371640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c923d3b60 0x562c923cf160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:50.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.121+0000 7ff816371640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c923d3b60 0x562c923cf160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32804/0 (socket says 172.21.15.16:32804) 2024-09-12T02:12:50.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.121+0000 7ff816371640 1 -- 172.21.15.16:0/758005018 learned_addr learned my addr 172.21.15.16:0/758005018 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:12:50.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.121+0000 7ff816b72640 1 --2- 172.21.15.16:0/758005018 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562c923ded40 0x562c923f6b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:50.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.121+0000 7ff815b70640 1 --2- 172.21.15.16:0/758005018 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562c923d4480 0x562c923f6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:50.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.121+0000 7ff816371640 1 -- 172.21.15.16:0/758005018 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562c923d4480 msgr2=0x562c923f6610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:50.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.121+0000 7ff816371640 1 --2- 172.21.15.16:0/758005018 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562c923d4480 0x562c923f6610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:50.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.121+0000 7ff816371640 1 -- 172.21.15.16:0/758005018 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562c923ded40 msgr2=0x562c923f6b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:50.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.121+0000 7ff816371640 1 --2- 172.21.15.16:0/758005018 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562c923ded40 0x562c923f6b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:50.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.121+0000 7ff816371640 1 -- 172.21.15.16:0/758005018 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c923cff20 con 0x562c923d3b60 2024-09-12T02:12:50.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.121+0000 7ff816b72640 1 --2- 172.21.15.16:0/758005018 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562c923ded40 0x562c923f6b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:12:50.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.121+0000 7ff816371640 1 --2- 172.21.15.16:0/758005018 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c923d3b60 0x562c923cf160 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7ff80c009610 tx=0x7ff80c086050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe9c7c1b2cc1c9c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:50.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.122+0000 7ff7fbfff640 1 -- 172.21.15.16:0/758005018 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff80c020070 con 0x562c923d3b60 2024-09-12T02:12:50.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.122+0000 7ff7fbfff640 1 -- 172.21.15.16:0/758005018 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff80c015070 con 0x562c923d3b60 2024-09-12T02:12:50.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.122+0000 7ff7fbfff640 1 -- 172.21.15.16:0/758005018 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff80c09a980 con 0x562c923d3b60 2024-09-12T02:12:50.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.122+0000 7ff81a9ae740 1 -- 172.21.15.16:0/758005018 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c923d3b60 msgr2=0x562c923cf160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:50.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.122+0000 7ff81a9ae740 1 --2- 172.21.15.16:0/758005018 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c923d3b60 0x562c923cf160 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7ff80c009610 tx=0x7ff80c086050 comp rx=0 tx=0).stop 2024-09-12T02:12:50.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.122+0000 7ff81a9ae740 1 -- 172.21.15.16:0/758005018 shutdown_connections 2024-09-12T02:12:50.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.122+0000 7ff81a9ae740 1 --2- 172.21.15.16:0/758005018 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562c923ded40 0x562c923f6b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:50.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.122+0000 7ff81a9ae740 1 --2- 172.21.15.16:0/758005018 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562c923d4480 0x562c923f6610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:50.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.122+0000 7ff81a9ae740 1 --2- 172.21.15.16:0/758005018 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c923d3b60 0x562c923cf160 unknown :-1 s=CLOSED pgs=1522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:50.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.122+0000 7ff81a9ae740 1 -- 172.21.15.16:0/758005018 >> 172.21.15.16:0/758005018 conn(0x562c923d00f0 msgr2=0x562c923cea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:50.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.122+0000 7ff81a9ae740 1 -- 172.21.15.16:0/758005018 shutdown_connections 2024-09-12T02:12:50.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.122+0000 7ff81a9ae740 1 -- 172.21.15.16:0/758005018 wait complete. 2024-09-12T02:12:50.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.123+0000 7ff81a9ae740 1 -- start start 2024-09-12T02:12:50.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.123+0000 7ff81a9ae740 1 Processor -- start 2024-09-12T02:12:50.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.123+0000 7ff81a9ae740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562c923d3b60 0x562c923e6ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:50.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.123+0000 7ff81a9ae740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c923d4480 0x562c923e7220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:50.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.123+0000 7ff81a9ae740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562c923ded40 0x562c923e3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:50.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.123+0000 7ff81a9ae740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff8101a9850 con 0x562c923d4480 2024-09-12T02:12:50.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.123+0000 7ff81a9ae740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff8101aaca0 con 0x562c923ded40 2024-09-12T02:12:50.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.123+0000 7ff81a9ae740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff8101a9f00 con 0x562c923d3b60 2024-09-12T02:12:50.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.123+0000 7ff81a9ae740 10 client.0 _ll_register_callbacks cb 0x7ff80c09b070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:12:50.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.123+0000 7ff815b70640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c923d4480 0x562c923e7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:50.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.123+0000 7ff815b70640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c923d4480 0x562c923e7220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32808/0 (socket says 172.21.15.16:32808) 2024-09-12T02:12:50.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.123+0000 7ff815b70640 1 -- 172.21.15.16:0/1547445408 learned_addr learned my addr 172.21.15.16:0/1547445408 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:12:50.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.123+0000 7ff816371640 1 --2- 172.21.15.16:0/1547445408 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562c923d3b60 0x562c923e6ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:50.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.123+0000 7ff816b72640 1 --2- 172.21.15.16:0/1547445408 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562c923ded40 0x562c923e3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:50.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.123+0000 7ff815b70640 1 -- 172.21.15.16:0/1547445408 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562c923d3b60 msgr2=0x562c923e6ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:50.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.123+0000 7ff815b70640 1 --2- 172.21.15.16:0/1547445408 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562c923d3b60 0x562c923e6ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:50.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.123+0000 7ff815b70640 1 -- 172.21.15.16:0/1547445408 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562c923ded40 msgr2=0x562c923e3cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:50.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.124+0000 7ff815b70640 1 --2- 172.21.15.16:0/1547445408 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562c923ded40 0x562c923e3cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:50.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.124+0000 7ff815b70640 1 -- 172.21.15.16:0/1547445408 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c923d9b00 con 0x562c923d4480 2024-09-12T02:12:50.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.124+0000 7ff816371640 1 --2- 172.21.15.16:0/1547445408 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562c923d3b60 0x562c923e6ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:12:50.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.124+0000 7ff816b72640 1 --2- 172.21.15.16:0/1547445408 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562c923ded40 0x562c923e3cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:12:50.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.124+0000 7ff815b70640 1 --2- 172.21.15.16:0/1547445408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c923d4480 0x562c923e7220 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x562c923d1470 tx=0x7ff8040316b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2e4cf51cc3da865 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:50.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.124+0000 7ff7f9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:12:50.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.124+0000 7ff7f9ffb640 1 -- 172.21.15.16:0/1547445408 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff804040040 con 0x562c923d4480 2024-09-12T02:12:50.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.124+0000 7ff7f9ffb640 1 -- 172.21.15.16:0/1547445408 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7ff804004440 con 0x562c923d4480 2024-09-12T02:12:50.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.124+0000 7ff81a9ae740 1 -- 172.21.15.16:0/1547445408 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff80c015070 con 0x562c923d4480 2024-09-12T02:12:50.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.124+0000 7ff7f9ffb640 1 -- 172.21.15.16:0/1547445408 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff804060050 con 0x562c923d4480 2024-09-12T02:12:50.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.124+0000 7ff81a9ae740 10 client.18897 fetch_fsmap learned FSMap version 25 2024-09-12T02:12:50.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.124+0000 7ff81a9ae740 1 -- 172.21.15.16:0/1547445408 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x562c92459b70 con 0x562c923d4480 2024-09-12T02:12:50.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.124+0000 7ff7f9ffb640 1 -- 172.21.15.16:0/1547445408 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff80c015070 con 0x562c923d4480 2024-09-12T02:12:50.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.124+0000 7ff7f9ffb640 1 -- 172.21.15.16:0/1547445408 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7ff80404f770 con 0x562c923d4480 2024-09-12T02:12:50.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.124+0000 7ff81a9ae740 10 client.18897 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:12:50.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.124+0000 7ff81a9ae740 10 client.18897 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:12:50.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.124+0000 7ff81a9ae740 20 client.18897 populate_metadata read hostname 'smithi016' 2024-09-12T02:12:50.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.124+0000 7ff81a9ae740 1 --2- 172.21.15.16:0/1547445408 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x562c92459f70 0x562c9245c360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:50.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.124+0000 7ff81a9ae740 4 client.18897 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-12T02:12:50.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.124+0000 7ff81a9ae740 1 -- 172.21.15.16:0/1547445408 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x562c92340d00 con 0x562c92459f70 2024-09-12T02:12:50.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.125+0000 7ff816371640 1 --2- 172.21.15.16:0/1547445408 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x562c92459f70 0x562c9245c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:50.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.125+0000 7ff816371640 1 --2- 172.21.15.16:0/1547445408 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x562c92459f70 0x562c9245c360 crc :-1 s=READY pgs=323 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92051c99d31f73d7 server_cookie=e078863e0f8c404f in_seq=0 out_seq=0 2024-09-12T02:12:50.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.125+0000 7ff7f9ffb640 10 client.18897 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:12:50.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.126+0000 7ff7f9ffb640 1 -- 172.21.15.16:0/1547445408 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x562c92340d00 con 0x562c92459f70 2024-09-12T02:12:50.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.126+0000 7ff7f9ffb640 10 client.18897 handle_command_reply: tid=0 2024-09-12T02:12:50.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.126+0000 7ff81a9ae740 1 client.18897 shutdown 2024-09-12T02:12:50.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.126+0000 7ff81a9ae740 1 -- 172.21.15.16:0/1547445408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c923d4480 msgr2=0x562c923e7220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:50.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.126+0000 7ff81a9ae740 1 --2- 172.21.15.16:0/1547445408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c923d4480 0x562c923e7220 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x562c923d1470 tx=0x7ff8040316b0 comp rx=0 tx=0).stop 2024-09-12T02:12:50.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.127+0000 7ff81a9ae740 1 -- 172.21.15.16:0/1547445408 shutdown_connections 2024-09-12T02:12:50.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.127+0000 7ff81a9ae740 1 --2- 172.21.15.16:0/1547445408 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x562c92459f70 0x562c9245c360 crc :-1 s=READY pgs=323 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:50.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.127+0000 7ff81a9ae740 1 --2- 172.21.15.16:0/1547445408 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562c923ded40 0x562c923e3cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:50.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.127+0000 7ff81a9ae740 1 --2- 172.21.15.16:0/1547445408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c923d4480 0x562c923e7220 unknown :-1 s=CLOSED pgs=1523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:50.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.127+0000 7ff81a9ae740 1 --2- 172.21.15.16:0/1547445408 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562c923d3b60 0x562c923e6ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:50.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.127+0000 7ff7f9ffb640 0 client.18897 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:12:50.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.127+0000 7ff81a9ae740 1 -- 172.21.15.16:0/1547445408 >> 172.21.15.16:0/1547445408 conn(0x562c923d00f0 msgr2=0x562c923cf270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:50.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.127+0000 7ff81a9ae740 1 -- 172.21.15.16:0/1547445408 shutdown_connections 2024-09-12T02:12:50.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.127+0000 7ff81a9ae740 1 -- 172.21.15.16:0/1547445408 wait complete. 2024-09-12T02:12:50.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.127+0000 7ff81a9ae740 20 client.18897 trim_cache size 0 max 16384 2024-09-12T02:12:50.148 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T02:12:50.148 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-09-12T02:12:50.148 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "c17cfabc-4213-4bf5-96ea-f4055ef12a2f", 2024-09-12T02:12:50.148 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-09-12T02:12:50.148 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T02:12:50.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.127+0000 7ff8185d5640 1 -- 172.21.15.16:0/248903627 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7ff7fc0a2b80 msgr2=0x7ff7fc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:50.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.127+0000 7ff8185d5640 1 --2- 172.21.15.16:0/248903627 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7ff7fc0a2b80 0x7ff7fc0a5040 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7ff8080077f0 tx=0x7ff808007520 comp rx=0 tx=0).stop 2024-09-12T02:12:50.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.127+0000 7ff8185d5640 1 -- 172.21.15.16:0/248903627 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff810102d40 msgr2=0x7ff810198410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:50.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.127+0000 7ff8185d5640 1 --2- 172.21.15.16:0/248903627 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff810102d40 0x7ff810198410 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7ff80c00d950 tx=0x7ff80c00de10 comp rx=0 tx=0).stop 2024-09-12T02:12:50.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.127+0000 7ff8185d5640 1 -- 172.21.15.16:0/248903627 shutdown_connections 2024-09-12T02:12:50.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.127+0000 7ff8185d5640 1 --2- 172.21.15.16:0/248903627 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7ff7fc0a2b80 0x7ff7fc0a5040 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:50.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.127+0000 7ff8185d5640 1 --2- 172.21.15.16:0/248903627 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff81010c170 0x7ff81019cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:50.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.127+0000 7ff8185d5640 1 --2- 172.21.15.16:0/248903627 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff810103660 0x7ff810198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:50.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.127+0000 7ff8185d5640 1 --2- 172.21.15.16:0/248903627 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff810102d40 0x7ff810198410 unknown :-1 s=CLOSED pgs=1520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:50.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.127+0000 7ff8185d5640 1 -- 172.21.15.16:0/248903627 >> 172.21.15.16:0/248903627 conn(0x7ff8100fc7a0 msgr2=0x7ff8100fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:50.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.127+0000 7ff8185d5640 1 -- 172.21.15.16:0/248903627 shutdown_connections 2024-09-12T02:12:50.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:50.128+0000 7ff8185d5640 1 -- 172.21.15.16:0/248903627 wait complete. 2024-09-12T02:12:50.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:12:50 smithi184 ceph-mon[35593]: pgmap v662: 97 pgs: 97 active+clean; 1.6 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-12T02:12:50.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:12:50 smithi033 ceph-mon[35332]: pgmap v662: 97 pgs: 97 active+clean; 1.6 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-12T02:12:50.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:12:50 smithi016 ceph-mon[30947]: pgmap v662: 97 pgs: 97 active+clean; 1.6 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-12T02:12:50.830 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T02:12:51.087 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:12:51.352 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:12:51 smithi016 ceph-mon[30947]: scrub queued for path: / 2024-09-12T02:12:51.353 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:12:51 smithi016 ceph-mon[30947]: scrub summary: idle+waiting paths [/] 2024-09-12T02:12:51.353 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:12:51 smithi016 ceph-mon[30947]: scrub summary: active paths [/] 2024-09-12T02:12:51.353 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:12:51 smithi016 ceph-mon[30947]: scrub summary: idle+waiting paths [/] 2024-09-12T02:12:51.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:12:51 smithi184 ceph-mon[35593]: scrub queued for path: / 2024-09-12T02:12:51.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:12:51 smithi184 ceph-mon[35593]: scrub summary: idle+waiting paths [/] 2024-09-12T02:12:51.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:12:51 smithi184 ceph-mon[35593]: scrub summary: active paths [/] 2024-09-12T02:12:51.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:12:51 smithi184 ceph-mon[35593]: scrub summary: idle+waiting paths [/] 2024-09-12T02:12:51.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:12:51 smithi033 ceph-mon[35332]: scrub queued for path: / 2024-09-12T02:12:51.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:12:51 smithi033 ceph-mon[35332]: scrub summary: idle+waiting paths [/] 2024-09-12T02:12:51.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:12:51 smithi033 ceph-mon[35332]: scrub summary: active paths [/] 2024-09-12T02:12:51.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:12:51 smithi033 ceph-mon[35332]: scrub summary: idle+waiting paths [/] 2024-09-12T02:12:51.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.681+0000 7f264e127640 1 -- 172.21.15.16:0/3648684749 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2648106a80 msgr2=0x7f2648106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:51.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.681+0000 7f264e127640 1 --2- 172.21.15.16:0/3648684749 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2648106a80 0x7f2648106e60 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7f2638009930 tx=0x7f263802a420 comp rx=0 tx=0).stop 2024-09-12T02:12:51.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.681+0000 7f264e127640 1 -- 172.21.15.16:0/3648684749 shutdown_connections 2024-09-12T02:12:51.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.681+0000 7f264e127640 1 --2- 172.21.15.16:0/3648684749 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2648101420 0x7f264810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:51.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.681+0000 7f264e127640 1 --2- 172.21.15.16:0/3648684749 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2648100a80 0x7f2648100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:51.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.681+0000 7f264e127640 1 --2- 172.21.15.16:0/3648684749 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2648106a80 0x7f2648106e60 unknown :-1 s=CLOSED pgs=1524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:51.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.681+0000 7f264e127640 1 -- 172.21.15.16:0/3648684749 >> 172.21.15.16:0/3648684749 conn(0x7f26480fc7a0 msgr2=0x7f26480febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:51.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.681+0000 7f264e127640 1 -- 172.21.15.16:0/3648684749 shutdown_connections 2024-09-12T02:12:51.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.681+0000 7f264e127640 1 -- 172.21.15.16:0/3648684749 wait complete. 2024-09-12T02:12:51.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.682+0000 7f264e127640 1 Processor -- start 2024-09-12T02:12:51.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.682+0000 7f264e127640 1 -- start start 2024-09-12T02:12:51.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.682+0000 7f264e127640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2648100a80 0x7f2648198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:51.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.682+0000 7f264e127640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2648101420 0x7f26481989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:51.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.682+0000 7f264e127640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2648106a80 0x7f264819cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:51.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.682+0000 7f264e127640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2648110e80 con 0x7f2648106a80 2024-09-12T02:12:51.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.682+0000 7f264e127640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2648110d00 con 0x7f2648101420 2024-09-12T02:12:51.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.682+0000 7f264e127640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f2648111000 con 0x7f2648100a80 2024-09-12T02:12:51.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.682+0000 7f2647fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2648106a80 0x7f264819cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:51.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.682+0000 7f2647fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2648106a80 0x7f264819cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51620/0 (socket says 172.21.15.16:51620) 2024-09-12T02:12:51.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.682+0000 7f2647fff640 1 -- 172.21.15.16:0/2864373070 learned_addr learned my addr 172.21.15.16:0/2864373070 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:12:51.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.682+0000 7f2646ffd640 1 --2- 172.21.15.16:0/2864373070 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2648101420 0x7f26481989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:51.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.682+0000 7f26477fe640 1 --2- 172.21.15.16:0/2864373070 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2648100a80 0x7f2648198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:51.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.683+0000 7f2647fff640 1 -- 172.21.15.16:0/2864373070 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2648100a80 msgr2=0x7f2648198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:51.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.683+0000 7f2647fff640 1 --2- 172.21.15.16:0/2864373070 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2648100a80 0x7f2648198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:51.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.683+0000 7f2647fff640 1 -- 172.21.15.16:0/2864373070 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2648101420 msgr2=0x7f26481989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:51.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.683+0000 7f2647fff640 1 --2- 172.21.15.16:0/2864373070 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2648101420 0x7f26481989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:51.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.683+0000 7f2647fff640 1 -- 172.21.15.16:0/2864373070 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f264819d420 con 0x7f2648106a80 2024-09-12T02:12:51.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.683+0000 7f2647fff640 1 --2- 172.21.15.16:0/2864373070 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2648106a80 0x7f264819cd40 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f264000cce0 tx=0x7f264000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:51.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.683+0000 7f2644ff9640 1 -- 172.21.15.16:0/2864373070 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f264000f450 con 0x7f2648106a80 2024-09-12T02:12:51.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.683+0000 7f2644ff9640 1 -- 172.21.15.16:0/2864373070 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f264000fad0 con 0x7f2648106a80 2024-09-12T02:12:51.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.683+0000 7f2644ff9640 1 -- 172.21.15.16:0/2864373070 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2640018c10 con 0x7f2648106a80 2024-09-12T02:12:51.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.683+0000 7f264e127640 1 -- 172.21.15.16:0/2864373070 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f264819d710 con 0x7f2648106a80 2024-09-12T02:12:51.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.683+0000 7f264e127640 1 -- 172.21.15.16:0/2864373070 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26481a4ff0 con 0x7f2648106a80 2024-09-12T02:12:51.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.685+0000 7f2644ff9640 1 -- 172.21.15.16:0/2864373070 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f264000f5f0 con 0x7f2648106a80 2024-09-12T02:12:51.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.686+0000 7f2650500740 1 Processor -- start 2024-09-12T02:12:51.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.686+0000 7f2650500740 1 -- start start 2024-09-12T02:12:51.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.686+0000 7f2650500740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556a39a4c4a0 0x556a39a4c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:51.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.686+0000 7f2650500740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a39a4cdc0 0x556a39a57140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:51.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.686+0000 7f2650500740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556a39a57680 0x556a39a59a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:51.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.686+0000 7f2650500740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x556a399b22e0 con 0x556a39a4cdc0 2024-09-12T02:12:51.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.686+0000 7f2650500740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x556a39959ce0 con 0x556a39a57680 2024-09-12T02:12:51.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.686+0000 7f2650500740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x556a399ae2a0 con 0x556a39a4c4a0 2024-09-12T02:12:51.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.686+0000 7f26477fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a39a4cdc0 0x556a39a57140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:51.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.686+0000 7f26477fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a39a4cdc0 0x556a39a57140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51624/0 (socket says 172.21.15.16:51624) 2024-09-12T02:12:51.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.686+0000 7f26477fe640 1 -- 172.21.15.16:0/1702180304 learned_addr learned my addr 172.21.15.16:0/1702180304 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:12:51.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.686+0000 7f2644ff9640 1 --2- 172.21.15.16:0/2864373070 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f26300a2b80 0x7f26300a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:51.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.686+0000 7f2646ffd640 1 --2- 172.21.15.16:0/1702180304 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556a39a57680 0x556a39a59a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:51.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.687+0000 7f2646ffd640 1 --2- 172.21.15.16:0/1702180304 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556a39a4c4a0 0x556a39a4c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:51.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.687+0000 7f2647fff640 1 --2- 172.21.15.16:0/2864373070 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f26300a2b80 0x7f26300a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:51.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.687+0000 7f26477fe640 1 -- 172.21.15.16:0/1702180304 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556a39a4c4a0 msgr2=0x556a39a4c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:51.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.687+0000 7f26477fe640 1 --2- 172.21.15.16:0/1702180304 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556a39a4c4a0 0x556a39a4c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:51.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.687+0000 7f26477fe640 1 -- 172.21.15.16:0/1702180304 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556a39a57680 msgr2=0x556a39a59a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:51.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.687+0000 7f26477fe640 1 --2- 172.21.15.16:0/1702180304 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556a39a57680 0x556a39a59a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:51.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.687+0000 7f2644ff9640 1 -- 172.21.15.16:0/2864373070 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f26400cf5c0 con 0x7f2648106a80 2024-09-12T02:12:51.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.687+0000 7f26477fe640 1 -- 172.21.15.16:0/1702180304 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a3993e440 con 0x556a39a4cdc0 2024-09-12T02:12:51.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.687+0000 7f2646ffd640 1 --2- 172.21.15.16:0/1702180304 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556a39a57680 0x556a39a59a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:12:51.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.687+0000 7f26477fe640 1 --2- 172.21.15.16:0/1702180304 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a39a4cdc0 0x556a39a57140 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f2638009930 tx=0x7f2638031630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=316db1fd6bcae1c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:51.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.687+0000 7f2647fff640 1 --2- 172.21.15.16:0/2864373070 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f26300a2b80 0x7f26300a5040 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f26400987f0 tx=0x7f264001baf0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:51.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.687+0000 7f262dffb640 1 -- 172.21.15.16:0/1702180304 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2638040070 con 0x556a39a4cdc0 2024-09-12T02:12:51.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.687+0000 7f262dffb640 1 -- 172.21.15.16:0/1702180304 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2638005020 con 0x556a39a4cdc0 2024-09-12T02:12:51.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.687+0000 7f262dffb640 1 -- 172.21.15.16:0/1702180304 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2638035040 con 0x556a39a4cdc0 2024-09-12T02:12:51.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.688+0000 7f2650500740 1 -- 172.21.15.16:0/1702180304 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a39a4cdc0 msgr2=0x556a39a57140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:51.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.688+0000 7f2650500740 1 --2- 172.21.15.16:0/1702180304 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a39a4cdc0 0x556a39a57140 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f2638009930 tx=0x7f2638031630 comp rx=0 tx=0).stop 2024-09-12T02:12:51.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.688+0000 7f2650500740 1 -- 172.21.15.16:0/1702180304 shutdown_connections 2024-09-12T02:12:51.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.688+0000 7f2650500740 1 --2- 172.21.15.16:0/1702180304 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556a39a57680 0x556a39a59a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:51.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.688+0000 7f2650500740 1 --2- 172.21.15.16:0/1702180304 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a39a4cdc0 0x556a39a57140 unknown :-1 s=CLOSED pgs=1526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:51.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.688+0000 7f2650500740 1 --2- 172.21.15.16:0/1702180304 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556a39a4c4a0 0x556a39a4c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:51.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.688+0000 7f2650500740 1 -- 172.21.15.16:0/1702180304 >> 172.21.15.16:0/1702180304 conn(0x556a39a48a30 msgr2=0x556a39a48e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:51.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.688+0000 7f2650500740 1 -- 172.21.15.16:0/1702180304 shutdown_connections 2024-09-12T02:12:51.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.688+0000 7f2650500740 1 -- 172.21.15.16:0/1702180304 wait complete. 2024-09-12T02:12:51.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.688+0000 7f2650500740 1 -- start start 2024-09-12T02:12:51.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.689+0000 7f2650500740 1 Processor -- start 2024-09-12T02:12:51.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.689+0000 7f2650500740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556a39a4c4a0 0x556a39a56ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:51.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.689+0000 7f2650500740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556a39a4cdc0 0x556a39a54180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:51.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.689+0000 7f2650500740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a39a57680 0x556a39a546c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:51.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.689+0000 7f2650500740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x556a39a5a250 con 0x556a39a57680 2024-09-12T02:12:51.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.689+0000 7f2650500740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x556a3996d050 con 0x556a39a4cdc0 2024-09-12T02:12:51.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.689+0000 7f2650500740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x556a39a5a3d0 con 0x556a39a4c4a0 2024-09-12T02:12:51.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.689+0000 7f2650500740 10 client.0 _ll_register_callbacks cb 0x556a399ad160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:12:51.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.689+0000 7f2646ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a39a57680 0x556a39a546c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:51.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.689+0000 7f2646ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a39a57680 0x556a39a546c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51626/0 (socket says 172.21.15.16:51626) 2024-09-12T02:12:51.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.689+0000 7f2646ffd640 1 -- 172.21.15.16:0/986736730 learned_addr learned my addr 172.21.15.16:0/986736730 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:12:51.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.689+0000 7f2646ffd640 1 --2- 172.21.15.16:0/986736730 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556a39a4c4a0 0x556a39a56ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:51.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.689+0000 7f26477fe640 1 --2- 172.21.15.16:0/986736730 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556a39a4cdc0 0x556a39a54180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:51.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.689+0000 7f2646ffd640 1 -- 172.21.15.16:0/986736730 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556a39a4c4a0 msgr2=0x556a39a56ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:51.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.689+0000 7f2646ffd640 1 --2- 172.21.15.16:0/986736730 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556a39a4c4a0 0x556a39a56ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:51.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.689+0000 7f2646ffd640 1 -- 172.21.15.16:0/986736730 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556a39a4cdc0 msgr2=0x556a39a54180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:51.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.689+0000 7f2646ffd640 1 --2- 172.21.15.16:0/986736730 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556a39a4cdc0 0x556a39a54180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:51.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.690+0000 7f2646ffd640 1 -- 172.21.15.16:0/986736730 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a397233c0 con 0x556a39a57680 2024-09-12T02:12:51.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.690+0000 7f2646ffd640 1 --2- 172.21.15.16:0/986736730 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a39a57680 0x556a39a546c0 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f263c012a50 tx=0x7f263c014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51359d3968f53905 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:51.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.690+0000 7f2626ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:12:51.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.690+0000 7f2626ffd640 1 -- 172.21.15.16:0/986736730 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f263c004030 con 0x556a39a57680 2024-09-12T02:12:51.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.690+0000 7f2626ffd640 1 -- 172.21.15.16:0/986736730 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f263c0041d0 con 0x556a39a57680 2024-09-12T02:12:51.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.690+0000 7f2650500740 1 -- 172.21.15.16:0/986736730 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2638005020 con 0x556a39a57680 2024-09-12T02:12:51.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.690+0000 7f2626ffd640 1 -- 172.21.15.16:0/986736730 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f263c0043d0 con 0x556a39a57680 2024-09-12T02:12:51.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.690+0000 7f2650500740 10 client.18909 fetch_fsmap learned FSMap version 25 2024-09-12T02:12:51.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.690+0000 7f2650500740 1 -- 172.21.15.16:0/986736730 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x556a39a65b80 con 0x556a39a57680 2024-09-12T02:12:51.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.690+0000 7f2626ffd640 1 -- 172.21.15.16:0/986736730 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2638005020 con 0x556a39a57680 2024-09-12T02:12:51.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.690+0000 7f2626ffd640 1 -- 172.21.15.16:0/986736730 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f263c0047a0 con 0x556a39a57680 2024-09-12T02:12:51.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.690+0000 7f2650500740 10 client.18909 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:12:51.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.690+0000 7f2650500740 10 client.18909 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:12:51.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.691+0000 7f2650500740 20 client.18909 populate_metadata read hostname 'smithi016' 2024-09-12T02:12:51.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.691+0000 7f2650500740 1 --2- 172.21.15.16:0/986736730 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x556a39a67040 0x556a39a69430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:51.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.691+0000 7f2650500740 4 client.18909 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:12:51.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.691+0000 7f2650500740 1 -- 172.21.15.16:0/986736730 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556a399bf850 con 0x556a39a67040 2024-09-12T02:12:51.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.691+0000 7f26477fe640 1 --2- 172.21.15.16:0/986736730 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x556a39a67040 0x556a39a69430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:51.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.691+0000 7f26477fe640 1 --2- 172.21.15.16:0/986736730 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x556a39a67040 0x556a39a69430 crc :-1 s=READY pgs=324 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=140f6c6330ec957e server_cookie=f43c2952628cf602 in_seq=0 out_seq=0 2024-09-12T02:12:51.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.691+0000 7f2626ffd640 10 client.18909 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:12:51.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.692+0000 7f2626ffd640 1 -- 172.21.15.16:0/986736730 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556a399bf850 con 0x556a39a67040 2024-09-12T02:12:51.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.692+0000 7f2626ffd640 10 client.18909 handle_command_reply: tid=0 2024-09-12T02:12:51.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.692+0000 7f2650500740 1 client.18909 shutdown 2024-09-12T02:12:51.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.692+0000 7f2650500740 1 -- 172.21.15.16:0/986736730 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a39a57680 msgr2=0x556a39a546c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:51.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.692+0000 7f2650500740 1 --2- 172.21.15.16:0/986736730 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a39a57680 0x556a39a546c0 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f263c012a50 tx=0x7f263c014040 comp rx=0 tx=0).stop 2024-09-12T02:12:51.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.692+0000 7f2650500740 1 -- 172.21.15.16:0/986736730 shutdown_connections 2024-09-12T02:12:51.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.692+0000 7f2650500740 1 --2- 172.21.15.16:0/986736730 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x556a39a67040 0x556a39a69430 crc :-1 s=READY pgs=324 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:51.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.692+0000 7f2650500740 1 --2- 172.21.15.16:0/986736730 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a39a57680 0x556a39a546c0 unknown :-1 s=CLOSED pgs=1527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:51.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.692+0000 7f2650500740 1 --2- 172.21.15.16:0/986736730 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556a39a4cdc0 0x556a39a54180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:51.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.692+0000 7f2650500740 1 --2- 172.21.15.16:0/986736730 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556a39a4c4a0 0x556a39a56ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:51.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.692+0000 7f2626ffd640 0 client.18909 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:12:51.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.692+0000 7f2650500740 1 -- 172.21.15.16:0/986736730 >> 172.21.15.16:0/986736730 conn(0x556a39a48a30 msgr2=0x556a39a4e710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:51.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.692+0000 7f2650500740 1 -- 172.21.15.16:0/986736730 shutdown_connections 2024-09-12T02:12:51.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.692+0000 7f2650500740 1 -- 172.21.15.16:0/986736730 wait complete. 2024-09-12T02:12:51.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.692+0000 7f2650500740 20 client.18909 trim_cache size 0 max 16384 2024-09-12T02:12:51.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.710+0000 7f2650500740 1 Processor -- start 2024-09-12T02:12:51.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.710+0000 7f2650500740 1 -- start start 2024-09-12T02:12:51.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.710+0000 7f2650500740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556a39a4c4a0 0x556a39a560f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:51.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.710+0000 7f2650500740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a39a4cdc0 0x556a39a6ec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:51.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.710+0000 7f2650500740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556a39a57680 0x556a39a6f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:51.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.710+0000 7f2650500740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f26481afc90 con 0x556a39a4cdc0 2024-09-12T02:12:51.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.710+0000 7f2650500740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f26481ad560 con 0x556a39a57680 2024-09-12T02:12:51.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.710+0000 7f2650500740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f26481ab3b0 con 0x556a39a4c4a0 2024-09-12T02:12:51.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.710+0000 7f26477fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a39a4cdc0 0x556a39a6ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:51.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.710+0000 7f26477fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a39a4cdc0 0x556a39a6ec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51636/0 (socket says 172.21.15.16:51636) 2024-09-12T02:12:51.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.710+0000 7f26477fe640 1 -- 172.21.15.16:0/605832768 learned_addr learned my addr 172.21.15.16:0/605832768 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:12:51.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.710+0000 7f2646ffd640 1 --2- 172.21.15.16:0/605832768 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556a39a57680 0x556a39a6f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:51.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.710+0000 7f2646ffd640 1 --2- 172.21.15.16:0/605832768 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556a39a4c4a0 0x556a39a560f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:51.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.710+0000 7f26477fe640 1 -- 172.21.15.16:0/605832768 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556a39a4c4a0 msgr2=0x556a39a560f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:51.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.710+0000 7f26477fe640 1 --2- 172.21.15.16:0/605832768 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556a39a4c4a0 0x556a39a560f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:51.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.710+0000 7f26477fe640 1 -- 172.21.15.16:0/605832768 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556a39a57680 msgr2=0x556a39a6f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:51.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.710+0000 7f26477fe640 1 --2- 172.21.15.16:0/605832768 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556a39a57680 0x556a39a6f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:51.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.710+0000 7f26477fe640 1 -- 172.21.15.16:0/605832768 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a39a53130 con 0x556a39a4cdc0 2024-09-12T02:12:51.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.710+0000 7f2646ffd640 1 --2- 172.21.15.16:0/605832768 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556a39a4c4a0 0x556a39a560f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:12:51.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.711+0000 7f26477fe640 1 --2- 172.21.15.16:0/605832768 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a39a4cdc0 0x556a39a6ec60 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7f2638009900 tx=0x7f2638031630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd2f512d863b229c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:51.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.711+0000 7f262d7fa640 1 -- 172.21.15.16:0/605832768 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2638040040 con 0x556a39a4cdc0 2024-09-12T02:12:51.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.711+0000 7f262d7fa640 1 -- 172.21.15.16:0/605832768 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2638055070 con 0x556a39a4cdc0 2024-09-12T02:12:51.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.711+0000 7f262d7fa640 1 -- 172.21.15.16:0/605832768 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2638035040 con 0x556a39a4cdc0 2024-09-12T02:12:51.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.712+0000 7f2650500740 1 -- 172.21.15.16:0/605832768 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a39a4cdc0 msgr2=0x556a39a6ec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:51.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.712+0000 7f2650500740 1 --2- 172.21.15.16:0/605832768 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a39a4cdc0 0x556a39a6ec60 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7f2638009900 tx=0x7f2638031630 comp rx=0 tx=0).stop 2024-09-12T02:12:51.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.712+0000 7f2650500740 1 -- 172.21.15.16:0/605832768 shutdown_connections 2024-09-12T02:12:51.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.712+0000 7f2650500740 1 --2- 172.21.15.16:0/605832768 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556a39a57680 0x556a39a6f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:51.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.712+0000 7f2650500740 1 --2- 172.21.15.16:0/605832768 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a39a4cdc0 0x556a39a6ec60 unknown :-1 s=CLOSED pgs=1528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:51.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.712+0000 7f2650500740 1 --2- 172.21.15.16:0/605832768 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556a39a4c4a0 0x556a39a560f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:51.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.712+0000 7f2650500740 1 -- 172.21.15.16:0/605832768 >> 172.21.15.16:0/605832768 conn(0x556a39a48a30 msgr2=0x556a39a559e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:51.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.712+0000 7f2650500740 1 -- 172.21.15.16:0/605832768 shutdown_connections 2024-09-12T02:12:51.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.712+0000 7f2650500740 1 -- 172.21.15.16:0/605832768 wait complete. 2024-09-12T02:12:51.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.712+0000 7f2650500740 1 -- start start 2024-09-12T02:12:51.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.712+0000 7f2650500740 1 Processor -- start 2024-09-12T02:12:51.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.713+0000 7f2650500740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556a39a4c4a0 0x556a39a5fec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:51.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.713+0000 7f2650500740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556a39a4cdc0 0x556a39a5d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:51.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.713+0000 7f2650500740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a39a57680 0x556a39a5da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:51.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.713+0000 7f2650500740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f26481aba20 con 0x556a39a57680 2024-09-12T02:12:51.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.713+0000 7f2650500740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f26481ace90 con 0x556a39a4c4a0 2024-09-12T02:12:51.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.713+0000 7f2650500740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f26481ac0f0 con 0x556a39a4cdc0 2024-09-12T02:12:51.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.713+0000 7f2650500740 10 client.0 _ll_register_callbacks cb 0x7f263c012870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:12:51.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.713+0000 7f2646ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a39a57680 0x556a39a5da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:51.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.713+0000 7f2646ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a39a57680 0x556a39a5da40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51644/0 (socket says 172.21.15.16:51644) 2024-09-12T02:12:51.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.713+0000 7f2646ffd640 1 -- 172.21.15.16:0/178074088 learned_addr learned my addr 172.21.15.16:0/178074088 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:12:51.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.713+0000 7f2646ffd640 1 --2- 172.21.15.16:0/178074088 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556a39a4c4a0 0x556a39a5fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:51.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.713+0000 7f26477fe640 1 --2- 172.21.15.16:0/178074088 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556a39a4cdc0 0x556a39a5d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:51.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.713+0000 7f2646ffd640 1 -- 172.21.15.16:0/178074088 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556a39a4cdc0 msgr2=0x556a39a5d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:51.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.713+0000 7f2646ffd640 1 --2- 172.21.15.16:0/178074088 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556a39a4cdc0 0x556a39a5d2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:51.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.713+0000 7f2646ffd640 1 -- 172.21.15.16:0/178074088 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556a39a4c4a0 msgr2=0x556a39a5fec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:51.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.713+0000 7f2646ffd640 1 --2- 172.21.15.16:0/178074088 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556a39a4c4a0 0x556a39a5fec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:51.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.713+0000 7f2646ffd640 1 -- 172.21.15.16:0/178074088 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a39a57240 con 0x556a39a57680 2024-09-12T02:12:51.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.713+0000 7f26477fe640 1 --2- 172.21.15.16:0/178074088 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556a39a4cdc0 0x556a39a5d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:12:51.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.714+0000 7f2646ffd640 1 --2- 172.21.15.16:0/178074088 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a39a57680 0x556a39a5da40 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7f263c004bf0 tx=0x7f263c014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71a79e08a053c4c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:51.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.714+0000 7f2626ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:12:51.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.714+0000 7f2626ffd640 1 -- 172.21.15.16:0/178074088 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f263c00bcd0 con 0x556a39a57680 2024-09-12T02:12:51.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.714+0000 7f2626ffd640 1 -- 172.21.15.16:0/178074088 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f263c01b040 con 0x556a39a57680 2024-09-12T02:12:51.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.714+0000 7f2650500740 1 -- 172.21.15.16:0/178074088 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2638055070 con 0x556a39a57680 2024-09-12T02:12:51.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.714+0000 7f2626ffd640 1 -- 172.21.15.16:0/178074088 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f263c00dbf0 con 0x556a39a57680 2024-09-12T02:12:51.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.714+0000 7f2650500740 10 client.18915 fetch_fsmap learned FSMap version 25 2024-09-12T02:12:51.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.714+0000 7f2650500740 1 -- 172.21.15.16:0/178074088 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x556a39ad1b40 con 0x556a39a57680 2024-09-12T02:12:51.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.714+0000 7f2626ffd640 1 -- 172.21.15.16:0/178074088 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2638055070 con 0x556a39a57680 2024-09-12T02:12:51.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.714+0000 7f2626ffd640 1 -- 172.21.15.16:0/178074088 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f263c021470 con 0x556a39a57680 2024-09-12T02:12:51.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.714+0000 7f2650500740 10 client.18915 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:12:51.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.714+0000 7f2650500740 10 client.18915 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:12:51.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.714+0000 7f2650500740 20 client.18915 populate_metadata read hostname 'smithi016' 2024-09-12T02:12:51.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.714+0000 7f2650500740 1 --2- 172.21.15.16:0/178074088 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x556a39ad2200 0x556a39ad45f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:51.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.714+0000 7f2650500740 4 client.18915 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T02:12:51.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.714+0000 7f2650500740 1 -- 172.21.15.16:0/178074088 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556a399b9d60 con 0x556a39ad2200 2024-09-12T02:12:51.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.715+0000 7f26477fe640 1 --2- 172.21.15.16:0/178074088 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x556a39ad2200 0x556a39ad45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:51.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.715+0000 7f26477fe640 1 --2- 172.21.15.16:0/178074088 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x556a39ad2200 0x556a39ad45f0 crc :-1 s=READY pgs=325 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dcd2d6e7b8efd40 server_cookie=5be71f15d5b8a33a in_seq=0 out_seq=0 2024-09-12T02:12:51.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.715+0000 7f2626ffd640 10 client.18915 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:12:51.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.715+0000 7f2626ffd640 1 -- 172.21.15.16:0/178074088 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x556a399b9d60 con 0x556a39ad2200 2024-09-12T02:12:51.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.715+0000 7f2626ffd640 10 client.18915 handle_command_reply: tid=0 2024-09-12T02:12:51.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.715+0000 7f2650500740 1 client.18915 shutdown 2024-09-12T02:12:51.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.715+0000 7f2650500740 1 -- 172.21.15.16:0/178074088 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a39a57680 msgr2=0x556a39a5da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:51.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.715+0000 7f2650500740 1 --2- 172.21.15.16:0/178074088 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a39a57680 0x556a39a5da40 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7f263c004bf0 tx=0x7f263c014040 comp rx=0 tx=0).stop 2024-09-12T02:12:51.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.716+0000 7f2650500740 1 -- 172.21.15.16:0/178074088 shutdown_connections 2024-09-12T02:12:51.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.716+0000 7f2650500740 1 --2- 172.21.15.16:0/178074088 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x556a39ad2200 0x556a39ad45f0 crc :-1 s=READY pgs=325 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:51.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.716+0000 7f2626ffd640 0 client.18915 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:12:51.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.716+0000 7f2650500740 1 --2- 172.21.15.16:0/178074088 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a39a57680 0x556a39a5da40 unknown :-1 s=CLOSED pgs=1529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:51.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.716+0000 7f2650500740 1 --2- 172.21.15.16:0/178074088 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556a39a4cdc0 0x556a39a5d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:51.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.716+0000 7f2650500740 1 --2- 172.21.15.16:0/178074088 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556a39a4c4a0 0x556a39a5fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:51.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.716+0000 7f2650500740 1 -- 172.21.15.16:0/178074088 >> 172.21.15.16:0/178074088 conn(0x556a39a48a30 msgr2=0x556a39a56200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:51.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.716+0000 7f2650500740 1 -- 172.21.15.16:0/178074088 shutdown_connections 2024-09-12T02:12:51.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.716+0000 7f2650500740 1 -- 172.21.15.16:0/178074088 wait complete. 2024-09-12T02:12:51.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.716+0000 7f2650500740 20 client.18915 trim_cache size 0 max 16384 2024-09-12T02:12:51.733 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T02:12:51.733 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-09-12T02:12:51.734 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {} 2024-09-12T02:12:51.734 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T02:12:51.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.716+0000 7f264e127640 1 -- 172.21.15.16:0/2864373070 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f26300a2b80 msgr2=0x7f26300a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:51.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.716+0000 7f264e127640 1 --2- 172.21.15.16:0/2864373070 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f26300a2b80 0x7f26300a5040 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f26400987f0 tx=0x7f264001baf0 comp rx=0 tx=0).stop 2024-09-12T02:12:51.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.716+0000 7f264e127640 1 -- 172.21.15.16:0/2864373070 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2648106a80 msgr2=0x7f264819cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:51.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.716+0000 7f264e127640 1 --2- 172.21.15.16:0/2864373070 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2648106a80 0x7f264819cd40 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f264000cce0 tx=0x7f264000a620 comp rx=0 tx=0).stop 2024-09-12T02:12:51.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.716+0000 7f264e127640 1 -- 172.21.15.16:0/2864373070 shutdown_connections 2024-09-12T02:12:51.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.716+0000 7f264e127640 1 --2- 172.21.15.16:0/2864373070 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f26300a2b80 0x7f26300a5040 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:51.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.716+0000 7f264e127640 1 --2- 172.21.15.16:0/2864373070 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2648106a80 0x7f264819cd40 unknown :-1 s=CLOSED pgs=1525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:51.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.716+0000 7f264e127640 1 --2- 172.21.15.16:0/2864373070 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2648101420 0x7f26481989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:51.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.716+0000 7f264e127640 1 --2- 172.21.15.16:0/2864373070 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2648100a80 0x7f2648198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:51.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.716+0000 7f264e127640 1 -- 172.21.15.16:0/2864373070 >> 172.21.15.16:0/2864373070 conn(0x7f26480fc7a0 msgr2=0x7f264806aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:51.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.717+0000 7f264e127640 1 -- 172.21.15.16:0/2864373070 shutdown_connections 2024-09-12T02:12:51.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:51.717+0000 7f264e127640 1 -- 172.21.15.16:0/2864373070 wait complete. 2024-09-12T02:12:52.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:12:52 smithi184 ceph-mon[35593]: pgmap v663: 97 pgs: 97 active+clean; 1.6 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-09-12T02:12:52.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:12:52 smithi184 ceph-mon[35593]: scrub summary: idle 2024-09-12T02:12:52.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:12:52 smithi033 ceph-mon[35332]: pgmap v663: 97 pgs: 97 active+clean; 1.6 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-09-12T02:12:52.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:12:52 smithi033 ceph-mon[35332]: scrub summary: idle 2024-09-12T02:12:52.427 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:12:52 smithi016 ceph-mon[30947]: pgmap v663: 97 pgs: 97 active+clean; 1.6 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-09-12T02:12:52.427 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:12:52 smithi016 ceph-mon[30947]: scrub summary: idle 2024-09-12T02:12:52.457 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-12T02:12:52.458 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:12:52.719 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:12:53.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:53.177+0000 7f69e41f0640 1 -- 172.21.15.16:0/1651543410 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69dc1002f0 msgr2=0x7f69dc1006d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:53.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:53.177+0000 7f69e41f0640 1 --2- 172.21.15.16:0/1651543410 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69dc1002f0 0x7f69dc1006d0 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7f69d8009cd0 tx=0x7f69d802a8e0 comp rx=0 tx=0).stop 2024-09-12T02:12:53.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:53.177+0000 7f69e41f0640 1 -- 172.21.15.16:0/1651543410 shutdown_connections 2024-09-12T02:12:53.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:53.177+0000 7f69e41f0640 1 --2- 172.21.15.16:0/1651543410 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f69dc106f50 0x7f69dc10eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:53.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:53.177+0000 7f69e41f0640 1 --2- 172.21.15.16:0/1651543410 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f69dc100ca0 0x7f69dc106840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:53.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:53.177+0000 7f69e41f0640 1 --2- 172.21.15.16:0/1651543410 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69dc1002f0 0x7f69dc1006d0 unknown :-1 s=CLOSED pgs=1530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:53.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:53.177+0000 7f69e41f0640 1 -- 172.21.15.16:0/1651543410 >> 172.21.15.16:0/1651543410 conn(0x7f69dc077210 msgr2=0x7f69dc0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:53.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:53.177+0000 7f69e41f0640 1 -- 172.21.15.16:0/1651543410 shutdown_connections 2024-09-12T02:12:53.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:53.177+0000 7f69e41f0640 1 -- 172.21.15.16:0/1651543410 wait complete. 2024-09-12T02:12:53.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:53.178+0000 7f69e41f0640 1 Processor -- start 2024-09-12T02:12:53.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:53.178+0000 7f69e41f0640 1 -- start start 2024-09-12T02:12:53.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:53.178+0000 7f69e41f0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f69dc1002f0 0x7f69dc198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:53.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:53.178+0000 7f69e41f0640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f69dc100ca0 0x7f69dc198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:53.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:53.178+0000 7f69e41f0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69dc106f50 0x7f69dc19cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:53.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:53.178+0000 7f69e41f0640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f69dc111fa0 con 0x7f69dc106f50 2024-09-12T02:12:53.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:53.178+0000 7f69e41f0640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f69dc111e20 con 0x7f69dc1002f0 2024-09-12T02:12:53.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:53.178+0000 7f69e41f0640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f69dc112120 con 0x7f69dc100ca0 2024-09-12T02:12:53.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:53.178+0000 7f69e278d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69dc106f50 0x7f69dc19cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:53.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:53.178+0000 7f69e278d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69dc106f50 0x7f69dc19cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51672/0 (socket says 172.21.15.16:51672) 2024-09-12T02:12:53.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:53.178+0000 7f69e278d640 1 -- 172.21.15.16:0/1929518208 learned_addr learned my addr 172.21.15.16:0/1929518208 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:12:53.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:53.178+0000 7f69e178b640 1 --2- 172.21.15.16:0/1929518208 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f69dc100ca0 0x7f69dc198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:53.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:53.178+0000 7f69e1f8c640 1 --2- 172.21.15.16:0/1929518208 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f69dc1002f0 0x7f69dc198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:53.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:53.178+0000 7f69e278d640 1 -- 172.21.15.16:0/1929518208 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f69dc100ca0 msgr2=0x7f69dc198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:53.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:53.178+0000 7f69e278d640 1 --2- 172.21.15.16:0/1929518208 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f69dc100ca0 0x7f69dc198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:53.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:53.178+0000 7f69e278d640 1 -- 172.21.15.16:0/1929518208 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f69dc1002f0 msgr2=0x7f69dc198500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:53.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:53.178+0000 7f69e278d640 1 --2- 172.21.15.16:0/1929518208 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f69dc1002f0 0x7f69dc198500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:53.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:53.178+0000 7f69e278d640 1 -- 172.21.15.16:0/1929518208 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69dc19d4b0 con 0x7f69dc106f50 2024-09-12T02:12:53.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:53.179+0000 7f69e278d640 1 --2- 172.21.15.16:0/1929518208 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69dc106f50 0x7f69dc19cdd0 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7f69d400d750 tx=0x7f69d400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:53.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:53.179+0000 7f69ceffd640 1 -- 172.21.15.16:0/1929518208 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69d4017070 con 0x7f69dc106f50 2024-09-12T02:12:53.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:53.179+0000 7f69ceffd640 1 -- 172.21.15.16:0/1929518208 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69d4005500 con 0x7f69dc106f50 2024-09-12T02:12:53.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:53.179+0000 7f69ceffd640 1 -- 172.21.15.16:0/1929518208 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69d4004d90 con 0x7f69dc106f50 2024-09-12T02:12:53.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:53.179+0000 7f69e41f0640 1 -- 172.21.15.16:0/1929518208 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69dc19d7a0 con 0x7f69dc106f50 2024-09-12T02:12:53.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:53.179+0000 7f69e41f0640 1 -- 172.21.15.16:0/1929518208 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69dc1a5080 con 0x7f69dc106f50 2024-09-12T02:12:53.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:53.180+0000 7f69e41f0640 1 -- 172.21.15.16:0/1929518208 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69dc1006d0 con 0x7f69dc106f50 2024-09-12T02:12:53.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:53.181+0000 7f69ceffd640 1 -- 172.21.15.16:0/1929518208 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f69d4004f30 con 0x7f69dc106f50 2024-09-12T02:12:53.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:53.182+0000 7f69ceffd640 1 --2- 172.21.15.16:0/1929518208 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f69c80a2c00 0x7f69c80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:53.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:53.182+0000 7f69e1f8c640 1 --2- 172.21.15.16:0/1929518208 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f69c80a2c00 0x7f69c80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:53.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:53.182+0000 7f69ceffd640 1 -- 172.21.15.16:0/1929518208 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f69d40ce8a0 con 0x7f69dc106f50 2024-09-12T02:12:53.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:53.182+0000 7f69e1f8c640 1 --2- 172.21.15.16:0/1929518208 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f69c80a2c00 0x7f69c80a50c0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f69d8006010 tx=0x7f69d8009360 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:53.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:53.186+0000 7f69ceffd640 1 -- 172.21.15.16:0/1929518208 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f69d409b1b0 con 0x7f69dc106f50 2024-09-12T02:12:53.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:53.396+0000 7f69e41f0640 1 -- 172.21.15.16:0/1929518208 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f69dc109bb0 con 0x7f69dc106f50 2024-09-12T02:12:53.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:53.397+0000 7f69ceffd640 1 -- 172.21.15.16:0/1929518208 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11911 (secure 0 0 0) 0x7f69d409f4b0 con 0x7f69dc106f50 2024-09-12T02:12:53.397 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T02:12:53.398 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":25,"btime":"2024-09-12T02:01:37:192021+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3279123279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3279123279},{"type":"v1","addr":"172.21.15.16:6835","nonce":3279123279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4119452382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4119452382},{"type":"v1","addr":"172.21.15.16:6837","nonce":4119452382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/3263823165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3263823165},{"type":"v1","addr":"172.21.15.16:6839","nonce":3263823165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/1510290732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":1510290732},{"type":"v1","addr":"172.21.15.16:6841","nonce":1510290732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/279674603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":279674603},{"type":"v1","addr":"172.21.15.33:6833","nonce":279674603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3555728028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3555728028},{"type":"v1","addr":"172.21.15.33:6835","nonce":3555728028}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T01:59:56.117608+0000","modified":"2024-09-12T02:01:36.294760+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24482,"mds_1":24481,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6833/1563374106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1563374106},{"type":"v1","addr":"172.21.15.184:6833","nonce":1563374106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6837/1892833868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1892833868},{"type":"v1","addr":"172.21.15.33:6837","nonce":1892833868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/1457679686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1457679686},{"type":"v1","addr":"172.21.15.184:6835","nonce":1457679686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.184:6837/1793393087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":1793393087},{"type":"v1","addr":"172.21.15.184:6837","nonce":1793393087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.184:6839/4203347941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":4203347941},{"type":"v1","addr":"172.21.15.184:6839","nonce":4203347941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.33:6839/4136670204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":4136670204},{"type":"v1","addr":"172.21.15.33:6839","nonce":4136670204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24481,24482,24469]},"id":1}]} 2024-09-12T02:12:53.398 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 25 2024-09-12T02:12:53.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:53.399+0000 7f69e41f0640 1 -- 172.21.15.16:0/1929518208 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f69c80a2c00 msgr2=0x7f69c80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:53.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:53.399+0000 7f69e41f0640 1 --2- 172.21.15.16:0/1929518208 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f69c80a2c00 0x7f69c80a50c0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f69d8006010 tx=0x7f69d8009360 comp rx=0 tx=0).stop 2024-09-12T02:12:53.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:53.399+0000 7f69e41f0640 1 -- 172.21.15.16:0/1929518208 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69dc106f50 msgr2=0x7f69dc19cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:53.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:53.399+0000 7f69e41f0640 1 --2- 172.21.15.16:0/1929518208 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69dc106f50 0x7f69dc19cdd0 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7f69d400d750 tx=0x7f69d400dc10 comp rx=0 tx=0).stop 2024-09-12T02:12:53.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:53.399+0000 7f69e41f0640 1 -- 172.21.15.16:0/1929518208 shutdown_connections 2024-09-12T02:12:53.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:53.399+0000 7f69e41f0640 1 --2- 172.21.15.16:0/1929518208 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f69c80a2c00 0x7f69c80a50c0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:53.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:53.399+0000 7f69e41f0640 1 --2- 172.21.15.16:0/1929518208 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69dc106f50 0x7f69dc19cdd0 unknown :-1 s=CLOSED pgs=1531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:53.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:53.399+0000 7f69e41f0640 1 --2- 172.21.15.16:0/1929518208 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f69dc100ca0 0x7f69dc198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:53.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:53.399+0000 7f69e41f0640 1 --2- 172.21.15.16:0/1929518208 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f69dc1002f0 0x7f69dc198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:53.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:53.399+0000 7f69e41f0640 1 -- 172.21.15.16:0/1929518208 >> 172.21.15.16:0/1929518208 conn(0x7f69dc077210 msgr2=0x7f69dc1052f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:53.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:53.399+0000 7f69e41f0640 1 -- 172.21.15.16:0/1929518208 shutdown_connections 2024-09-12T02:12:53.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:53.399+0000 7f69e41f0640 1 -- 172.21.15.16:0/1929518208 wait complete. 2024-09-12T02:12:54.114 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-12T02:12:54.374 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:12:54.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:12:54 smithi184 ceph-mon[35593]: pgmap v664: 97 pgs: 97 active+clean; 1.7 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 17 op/s 2024-09-12T02:12:54.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:12:54 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/1929518208' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:12:54.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:12:54 smithi033 ceph-mon[35332]: pgmap v664: 97 pgs: 97 active+clean; 1.7 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 17 op/s 2024-09-12T02:12:54.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:12:54 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/1929518208' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:12:54.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:12:54 smithi016 ceph-mon[30947]: pgmap v664: 97 pgs: 97 active+clean; 1.7 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 17 op/s 2024-09-12T02:12:54.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:12:54 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/1929518208' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:12:54.809 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.808+0000 7f0ea421e640 1 -- 172.21.15.16:0/1620687538 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e9c1025c0 msgr2=0x7f0e9c1049b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:54.809 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.808+0000 7f0ea421e640 1 --2- 172.21.15.16:0/1620687538 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e9c1025c0 0x7f0e9c1049b0 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7f0e98009900 tx=0x7f0e9802a410 comp rx=0 tx=0).stop 2024-09-12T02:12:54.809 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.808+0000 7f0ea421e640 1 -- 172.21.15.16:0/1620687538 shutdown_connections 2024-09-12T02:12:54.809 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.808+0000 7f0ea421e640 1 --2- 172.21.15.16:0/1620687538 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0e9c107820 0x7f0e9c109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:54.809 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.808+0000 7f0ea421e640 1 --2- 172.21.15.16:0/1620687538 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e9c104ef0 0x7f0e9c1072e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:54.809 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.808+0000 7f0ea421e640 1 --2- 172.21.15.16:0/1620687538 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e9c1025c0 0x7f0e9c1049b0 unknown :-1 s=CLOSED pgs=1532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:54.809 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.808+0000 7f0ea421e640 1 -- 172.21.15.16:0/1620687538 >> 172.21.15.16:0/1620687538 conn(0x7f0e9c0fc1e0 msgr2=0x7f0e9c0fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:54.810 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.809+0000 7f0ea421e640 1 -- 172.21.15.16:0/1620687538 shutdown_connections 2024-09-12T02:12:54.810 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.809+0000 7f0ea421e640 1 -- 172.21.15.16:0/1620687538 wait complete. 2024-09-12T02:12:54.812 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.809+0000 7f0ea421e640 1 Processor -- start 2024-09-12T02:12:54.812 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.809+0000 7f0ea421e640 1 -- start start 2024-09-12T02:12:54.812 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.810+0000 7f0ea421e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e9c1025c0 0x7f0e9c197e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:54.812 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.810+0000 7f0ea421e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0e9c104ef0 0x7f0e9c1983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:54.812 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.810+0000 7f0ea421e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e9c107820 0x7f0e9c19f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:54.812 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.810+0000 7f0ea421e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0e9c10d820 con 0x7f0e9c107820 2024-09-12T02:12:54.812 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.810+0000 7f0ea421e640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0e9c10d6a0 con 0x7f0e9c1025c0 2024-09-12T02:12:54.812 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.810+0000 7f0ea421e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f0e9c10d9a0 con 0x7f0e9c104ef0 2024-09-12T02:12:54.812 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.810+0000 7f0ea27bb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e9c107820 0x7f0e9c19f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:54.812 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.810+0000 7f0ea27bb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e9c107820 0x7f0e9c19f430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51688/0 (socket says 172.21.15.16:51688) 2024-09-12T02:12:54.813 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.810+0000 7f0ea27bb640 1 -- 172.21.15.16:0/2134352490 learned_addr learned my addr 172.21.15.16:0/2134352490 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:12:54.813 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.810+0000 7f0ea17b9640 1 --2- 172.21.15.16:0/2134352490 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0e9c104ef0 0x7f0e9c1983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:54.813 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.810+0000 7f0ea1fba640 1 --2- 172.21.15.16:0/2134352490 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e9c1025c0 0x7f0e9c197e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:54.813 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.810+0000 7f0ea27bb640 1 -- 172.21.15.16:0/2134352490 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0e9c104ef0 msgr2=0x7f0e9c1983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:54.813 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.810+0000 7f0ea27bb640 1 --2- 172.21.15.16:0/2134352490 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0e9c104ef0 0x7f0e9c1983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:54.813 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.810+0000 7f0ea27bb640 1 -- 172.21.15.16:0/2134352490 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e9c1025c0 msgr2=0x7f0e9c197e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:54.813 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.810+0000 7f0ea27bb640 1 --2- 172.21.15.16:0/2134352490 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e9c1025c0 0x7f0e9c197e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:54.813 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.810+0000 7f0ea27bb640 1 -- 172.21.15.16:0/2134352490 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e9c19faa0 con 0x7f0e9c107820 2024-09-12T02:12:54.813 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.810+0000 7f0ea27bb640 1 --2- 172.21.15.16:0/2134352490 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e9c107820 0x7f0e9c19f430 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f0e9400eb40 tx=0x7f0e9400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:54.813 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.811+0000 7f0e92ffd640 1 -- 172.21.15.16:0/2134352490 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e94004b20 con 0x7f0e9c107820 2024-09-12T02:12:54.813 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.811+0000 7f0e92ffd640 1 -- 172.21.15.16:0/2134352490 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e94004cc0 con 0x7f0e9c107820 2024-09-12T02:12:54.813 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.811+0000 7f0ea421e640 1 -- 172.21.15.16:0/2134352490 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e9c19fd90 con 0x7f0e9c107820 2024-09-12T02:12:54.813 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.811+0000 7f0e92ffd640 1 -- 172.21.15.16:0/2134352490 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e94010850 con 0x7f0e9c107820 2024-09-12T02:12:54.813 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.811+0000 7f0ea421e640 1 -- 172.21.15.16:0/2134352490 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e9c1a0280 con 0x7f0e9c107820 2024-09-12T02:12:54.814 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.813+0000 7f0ea7090740 1 Processor -- start 2024-09-12T02:12:54.814 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.814+0000 7f0e92ffd640 1 -- 172.21.15.16:0/2134352490 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f0e94005020 con 0x7f0e9c107820 2024-09-12T02:12:54.822 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.815+0000 7f0ea7090740 1 -- start start 2024-09-12T02:12:54.822 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.815+0000 7f0ea7090740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6e092a9a0 0x55c6e092ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:54.822 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.815+0000 7f0ea7090740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c6e092b2e0 0x55c6e092fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:54.822 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.815+0000 7f0ea7090740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c6e0930370 0x55c6e0932830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:54.823 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.815+0000 7f0ea7090740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55c6e088e2f0 con 0x55c6e0930370 2024-09-12T02:12:54.823 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.815+0000 7f0ea7090740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55c6e0836270 con 0x55c6e092a9a0 2024-09-12T02:12:54.823 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.815+0000 7f0ea7090740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55c6e088a2b0 con 0x55c6e092b2e0 2024-09-12T02:12:54.823 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.815+0000 7f0ea17b9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c6e0930370 0x55c6e0932830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:54.823 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.815+0000 7f0ea17b9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c6e0930370 0x55c6e0932830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51692/0 (socket says 172.21.15.16:51692) 2024-09-12T02:12:54.823 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.815+0000 7f0ea17b9640 1 -- 172.21.15.16:0/1459878302 learned_addr learned my addr 172.21.15.16:0/1459878302 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:12:54.823 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.815+0000 7f0ea17b9640 1 --2- 172.21.15.16:0/1459878302 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6e092a9a0 0x55c6e092ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:54.823 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.815+0000 7f0ea1fba640 1 --2- 172.21.15.16:0/1459878302 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c6e092b2e0 0x55c6e092fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:54.823 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.815+0000 7f0ea17b9640 1 -- 172.21.15.16:0/1459878302 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c6e092b2e0 msgr2=0x55c6e092fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:54.823 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.815+0000 7f0ea17b9640 1 --2- 172.21.15.16:0/1459878302 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c6e092b2e0 0x55c6e092fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:54.824 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.815+0000 7f0e92ffd640 1 --2- 172.21.15.16:0/2134352490 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f0e880a2cb0 0x7f0e880a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:54.824 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.815+0000 7f0ea17b9640 1 -- 172.21.15.16:0/1459878302 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6e092a9a0 msgr2=0x55c6e092ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:54.824 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.815+0000 7f0ea17b9640 1 --2- 172.21.15.16:0/1459878302 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6e092a9a0 0x55c6e092ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:54.824 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.815+0000 7f0ea17b9640 1 -- 172.21.15.16:0/1459878302 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6e0835340 con 0x55c6e0930370 2024-09-12T02:12:54.824 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.815+0000 7f0ea27bb640 1 --2- 172.21.15.16:0/2134352490 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f0e880a2cb0 0x7f0e880a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:54.824 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.816+0000 7f0ea17b9640 1 --2- 172.21.15.16:0/1459878302 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c6e0930370 0x55c6e0932830 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7f0e8c0058c0 tx=0x7f0e8c005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0028fe41061f53a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:54.824 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.816+0000 7f0e92ffd640 1 -- 172.21.15.16:0/2134352490 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f0e940cf6f0 con 0x7f0e9c107820 2024-09-12T02:12:54.824 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.816+0000 7f0e87fff640 1 -- 172.21.15.16:0/1459878302 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e8c004030 con 0x55c6e0930370 2024-09-12T02:12:54.824 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.816+0000 7f0e87fff640 1 -- 172.21.15.16:0/1459878302 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e8c005500 con 0x55c6e0930370 2024-09-12T02:12:54.824 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.816+0000 7f0e87fff640 1 -- 172.21.15.16:0/1459878302 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e8c004c60 con 0x55c6e0930370 2024-09-12T02:12:54.824 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.817+0000 7f0ea7090740 1 -- 172.21.15.16:0/1459878302 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c6e0930370 msgr2=0x55c6e0932830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:54.824 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.817+0000 7f0ea27bb640 1 --2- 172.21.15.16:0/2134352490 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f0e880a2cb0 0x7f0e880a5170 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f0e9400c7d0 tx=0x7f0e940c6b80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:54.824 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.817+0000 7f0ea7090740 1 --2- 172.21.15.16:0/1459878302 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c6e0930370 0x55c6e0932830 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7f0e8c0058c0 tx=0x7f0e8c005850 comp rx=0 tx=0).stop 2024-09-12T02:12:54.825 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.817+0000 7f0ea7090740 1 -- 172.21.15.16:0/1459878302 shutdown_connections 2024-09-12T02:12:54.825 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.817+0000 7f0ea7090740 1 --2- 172.21.15.16:0/1459878302 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c6e0930370 0x55c6e0932830 unknown :-1 s=CLOSED pgs=1534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:54.825 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.817+0000 7f0ea7090740 1 --2- 172.21.15.16:0/1459878302 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c6e092b2e0 0x55c6e092fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:54.825 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.817+0000 7f0ea7090740 1 --2- 172.21.15.16:0/1459878302 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6e092a9a0 0x55c6e092ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:54.825 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.817+0000 7f0ea7090740 1 -- 172.21.15.16:0/1459878302 >> 172.21.15.16:0/1459878302 conn(0x55c6e0926010 msgr2=0x55c6e0926410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:54.825 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.817+0000 7f0ea7090740 1 -- 172.21.15.16:0/1459878302 shutdown_connections 2024-09-12T02:12:54.825 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.817+0000 7f0ea7090740 1 -- 172.21.15.16:0/1459878302 wait complete. 2024-09-12T02:12:54.825 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.817+0000 7f0ea7090740 1 -- start start 2024-09-12T02:12:54.825 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.817+0000 7f0ea7090740 1 Processor -- start 2024-09-12T02:12:54.825 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.818+0000 7f0ea7090740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c6e092a9a0 0x55c6e0934b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:54.825 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.818+0000 7f0ea7090740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c6e092b2e0 0x55c6e09350b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:54.826 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.818+0000 7f0ea7090740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6e0930370 0x55c6e093c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:54.826 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.818+0000 7f0ea7090740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55c6e0933340 con 0x55c6e092b2e0 2024-09-12T02:12:54.826 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.818+0000 7f0ea7090740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55c6e0817290 con 0x55c6e0930370 2024-09-12T02:12:54.826 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.818+0000 7f0ea7090740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55c6e09334c0 con 0x55c6e092a9a0 2024-09-12T02:12:54.826 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.818+0000 7f0ea7090740 10 client.0 _ll_register_callbacks cb 0x55c6e0848140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:12:54.826 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.818+0000 7f0ea1fba640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c6e092b2e0 0x55c6e09350b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:54.826 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.818+0000 7f0ea1fba640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c6e092b2e0 0x55c6e09350b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51706/0 (socket says 172.21.15.16:51706) 2024-09-12T02:12:54.826 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.818+0000 7f0ea1fba640 1 -- 172.21.15.16:0/2404329865 learned_addr learned my addr 172.21.15.16:0/2404329865 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:12:54.826 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.818+0000 7f0ea17b9640 1 --2- 172.21.15.16:0/2404329865 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6e0930370 0x55c6e093c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:54.826 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.818+0000 7f0ea17b9640 1 --2- 172.21.15.16:0/2404329865 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c6e092a9a0 0x55c6e0934b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:54.826 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.818+0000 7f0ea1fba640 1 -- 172.21.15.16:0/2404329865 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c6e092a9a0 msgr2=0x55c6e0934b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:54.826 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.818+0000 7f0ea1fba640 1 --2- 172.21.15.16:0/2404329865 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c6e092a9a0 0x55c6e0934b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:54.826 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.818+0000 7f0ea1fba640 1 -- 172.21.15.16:0/2404329865 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6e0930370 msgr2=0x55c6e093c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:54.827 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.818+0000 7f0ea1fba640 1 --2- 172.21.15.16:0/2404329865 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6e0930370 0x55c6e093c0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:54.827 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.818+0000 7f0ea1fba640 1 -- 172.21.15.16:0/2404329865 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6e05a1210 con 0x55c6e092b2e0 2024-09-12T02:12:54.827 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.818+0000 7f0ea1fba640 1 --2- 172.21.15.16:0/2404329865 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c6e092b2e0 0x55c6e09350b0 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f0e98009900 tx=0x7f0e98005e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1c28a4ecf04d94e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:54.827 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.818+0000 7f0e857fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:12:54.827 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.819+0000 7f0e857fa640 1 -- 172.21.15.16:0/2404329865 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e98035040 con 0x55c6e092b2e0 2024-09-12T02:12:54.827 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.819+0000 7f0e857fa640 1 -- 172.21.15.16:0/2404329865 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f0e98004040 con 0x55c6e092b2e0 2024-09-12T02:12:54.827 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.819+0000 7f0ea7090740 1 -- 172.21.15.16:0/2404329865 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e8c005500 con 0x55c6e092b2e0 2024-09-12T02:12:54.827 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.819+0000 7f0e857fa640 1 -- 172.21.15.16:0/2404329865 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f0e980319f0 con 0x55c6e092b2e0 2024-09-12T02:12:54.827 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.819+0000 7f0ea7090740 10 client.18933 fetch_fsmap learned FSMap version 25 2024-09-12T02:12:54.827 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.819+0000 7f0ea7090740 1 -- 172.21.15.16:0/2404329865 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c6e066d620 con 0x55c6e092b2e0 2024-09-12T02:12:54.827 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.819+0000 7f0e857fa640 1 -- 172.21.15.16:0/2404329865 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e8c005500 con 0x55c6e092b2e0 2024-09-12T02:12:54.828 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.819+0000 7f0e857fa640 1 -- 172.21.15.16:0/2404329865 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f0e98004940 con 0x55c6e092b2e0 2024-09-12T02:12:54.828 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.819+0000 7f0ea7090740 10 client.18933 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:12:54.828 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.819+0000 7f0ea7090740 10 client.18933 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:12:54.828 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.819+0000 7f0ea7090740 20 client.18933 populate_metadata read hostname 'smithi016' 2024-09-12T02:12:54.828 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.819+0000 7f0ea7090740 1 --2- 172.21.15.16:0/2404329865 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55c6e09408b0 0x55c6e0942cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:54.828 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.819+0000 7f0ea7090740 4 client.18933 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:12:54.828 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.819+0000 7f0ea7090740 1 -- 172.21.15.16:0/2404329865 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c6e089ba20 con 0x55c6e09408b0 2024-09-12T02:12:54.828 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.820+0000 7f0ea17b9640 1 --2- 172.21.15.16:0/2404329865 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55c6e09408b0 0x55c6e0942cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:54.828 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.820+0000 7f0ea17b9640 1 --2- 172.21.15.16:0/2404329865 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55c6e09408b0 0x55c6e0942cd0 crc :-1 s=READY pgs=326 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf00fa53d664bd1b server_cookie=ef66b32c4163166b in_seq=0 out_seq=0 2024-09-12T02:12:54.828 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.820+0000 7f0e857fa640 10 client.18933 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:12:54.828 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.821+0000 7f0e857fa640 1 -- 172.21.15.16:0/2404329865 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c6e089ba20 con 0x55c6e09408b0 2024-09-12T02:12:54.828 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.821+0000 7f0e857fa640 10 client.18933 handle_command_reply: tid=0 2024-09-12T02:12:54.829 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.821+0000 7f0ea7090740 1 client.18933 shutdown 2024-09-12T02:12:54.829 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.821+0000 7f0ea7090740 1 -- 172.21.15.16:0/2404329865 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c6e092b2e0 msgr2=0x55c6e09350b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:54.829 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.821+0000 7f0ea7090740 1 --2- 172.21.15.16:0/2404329865 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c6e092b2e0 0x55c6e09350b0 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f0e98009900 tx=0x7f0e98005e70 comp rx=0 tx=0).stop 2024-09-12T02:12:54.829 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.821+0000 7f0ea7090740 1 -- 172.21.15.16:0/2404329865 shutdown_connections 2024-09-12T02:12:54.829 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.821+0000 7f0ea7090740 1 --2- 172.21.15.16:0/2404329865 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55c6e09408b0 0x55c6e0942cd0 crc :-1 s=READY pgs=326 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:54.829 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.821+0000 7f0ea7090740 1 --2- 172.21.15.16:0/2404329865 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6e0930370 0x55c6e093c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:54.829 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.821+0000 7f0e857fa640 0 client.18933 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:12:54.829 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.821+0000 7f0ea7090740 1 --2- 172.21.15.16:0/2404329865 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c6e092b2e0 0x55c6e09350b0 unknown :-1 s=CLOSED pgs=1535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:54.829 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.821+0000 7f0ea7090740 1 --2- 172.21.15.16:0/2404329865 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c6e092a9a0 0x55c6e0934b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:54.829 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.821+0000 7f0ea7090740 1 -- 172.21.15.16:0/2404329865 >> 172.21.15.16:0/2404329865 conn(0x55c6e0926010 msgr2=0x55c6e0927820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:54.829 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.821+0000 7f0ea7090740 1 -- 172.21.15.16:0/2404329865 shutdown_connections 2024-09-12T02:12:54.830 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.821+0000 7f0ea7090740 1 -- 172.21.15.16:0/2404329865 wait complete. 2024-09-12T02:12:54.830 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.821+0000 7f0ea7090740 20 client.18933 trim_cache size 0 max 16384 2024-09-12T02:12:54.841 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.836+0000 7f0ea7090740 1 Processor -- start 2024-09-12T02:12:54.842 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.836+0000 7f0ea7090740 1 -- start start 2024-09-12T02:12:54.842 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.836+0000 7f0ea7090740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c6e092a9a0 0x55c6e092ad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:54.842 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.836+0000 7f0ea7090740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6e092b2e0 0x55c6e093c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:54.842 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.836+0000 7f0ea7090740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c6e0930370 0x55c6e0926f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:54.842 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.836+0000 7f0ea7090740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0e9c1ab6a0 con 0x55c6e092a9a0 2024-09-12T02:12:54.842 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.836+0000 7f0ea7090740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0e9c1a8f70 con 0x55c6e092b2e0 2024-09-12T02:12:54.842 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.836+0000 7f0ea7090740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f0e9c1a6dc0 con 0x55c6e0930370 2024-09-12T02:12:54.842 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.836+0000 7f0ea17b9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c6e092a9a0 0x55c6e092ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:54.842 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.836+0000 7f0ea17b9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c6e092a9a0 0x55c6e092ad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51722/0 (socket says 172.21.15.16:51722) 2024-09-12T02:12:54.843 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.836+0000 7f0ea17b9640 1 -- 172.21.15.16:0/2232630908 learned_addr learned my addr 172.21.15.16:0/2232630908 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:12:54.843 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.836+0000 7f0ea1fba640 1 --2- 172.21.15.16:0/2232630908 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6e092b2e0 0x55c6e093c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:54.843 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.836+0000 7f0ea17b9640 1 --2- 172.21.15.16:0/2232630908 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c6e0930370 0x55c6e0926f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:54.843 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.836+0000 7f0ea17b9640 1 -- 172.21.15.16:0/2232630908 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c6e0930370 msgr2=0x55c6e0926f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:54.843 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.836+0000 7f0ea17b9640 1 --2- 172.21.15.16:0/2232630908 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c6e0930370 0x55c6e0926f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:54.843 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.836+0000 7f0ea17b9640 1 -- 172.21.15.16:0/2232630908 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6e092b2e0 msgr2=0x55c6e093c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:54.843 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.836+0000 7f0ea17b9640 1 --2- 172.21.15.16:0/2232630908 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6e092b2e0 0x55c6e093c0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:54.843 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.836+0000 7f0ea17b9640 1 -- 172.21.15.16:0/2232630908 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6e09202f0 con 0x55c6e092a9a0 2024-09-12T02:12:54.843 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.836+0000 7f0ea1fba640 1 --2- 172.21.15.16:0/2232630908 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6e092b2e0 0x55c6e093c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:12:54.843 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.837+0000 7f0ea17b9640 1 --2- 172.21.15.16:0/2232630908 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c6e092a9a0 0x55c6e092ad80 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f0e8c000c00 tx=0x7f0e8c005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=555cac2618e29324 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:54.843 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.837+0000 7f0e877fe640 1 -- 172.21.15.16:0/2232630908 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e8c011200 con 0x55c6e092a9a0 2024-09-12T02:12:54.843 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.837+0000 7f0e877fe640 1 -- 172.21.15.16:0/2232630908 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e8c035070 con 0x55c6e092a9a0 2024-09-12T02:12:54.844 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.837+0000 7f0e877fe640 1 -- 172.21.15.16:0/2232630908 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e8c0318e0 con 0x55c6e092a9a0 2024-09-12T02:12:54.844 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.838+0000 7f0ea7090740 1 -- 172.21.15.16:0/2232630908 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c6e092a9a0 msgr2=0x55c6e092ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:54.844 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.838+0000 7f0ea7090740 1 --2- 172.21.15.16:0/2232630908 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c6e092a9a0 0x55c6e092ad80 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f0e8c000c00 tx=0x7f0e8c005850 comp rx=0 tx=0).stop 2024-09-12T02:12:54.844 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.838+0000 7f0ea7090740 1 -- 172.21.15.16:0/2232630908 shutdown_connections 2024-09-12T02:12:54.844 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.838+0000 7f0ea7090740 1 --2- 172.21.15.16:0/2232630908 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c6e0930370 0x55c6e0926f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:54.844 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.838+0000 7f0ea7090740 1 --2- 172.21.15.16:0/2232630908 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6e092b2e0 0x55c6e093c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:54.844 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.838+0000 7f0ea7090740 1 --2- 172.21.15.16:0/2232630908 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c6e092a9a0 0x55c6e092ad80 unknown :-1 s=CLOSED pgs=1536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:54.844 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.838+0000 7f0ea7090740 1 -- 172.21.15.16:0/2232630908 >> 172.21.15.16:0/2232630908 conn(0x55c6e0926010 msgr2=0x55c6e092c450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:54.844 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.838+0000 7f0ea7090740 1 -- 172.21.15.16:0/2232630908 shutdown_connections 2024-09-12T02:12:54.845 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.838+0000 7f0ea7090740 1 -- 172.21.15.16:0/2232630908 wait complete. 2024-09-12T02:12:54.845 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.838+0000 7f0ea7090740 1 -- start start 2024-09-12T02:12:54.845 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.838+0000 7f0ea7090740 1 Processor -- start 2024-09-12T02:12:54.845 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.838+0000 7f0ea7090740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c6e092a9a0 0x55c6e0939420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:54.845 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.838+0000 7f0ea7090740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6e092b2e0 0x55c6e0939960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:54.845 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.838+0000 7f0ea7090740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c6e0930370 0x55c6e093a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:54.845 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.838+0000 7f0ea7090740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0e9c1a7430 con 0x55c6e0930370 2024-09-12T02:12:54.845 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.839+0000 7f0ea7090740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0e9c1a88a0 con 0x55c6e092b2e0 2024-09-12T02:12:54.845 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.839+0000 7f0ea7090740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f0e9c1a7b00 con 0x55c6e092a9a0 2024-09-12T02:12:54.845 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.839+0000 7f0ea7090740 10 client.0 _ll_register_callbacks cb 0x7f0e98009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:12:54.846 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.839+0000 7f0ea17b9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c6e0930370 0x55c6e093a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:54.846 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.839+0000 7f0ea17b9640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c6e092a9a0 0x55c6e0939420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:54.846 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.839+0000 7f0ea17b9640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c6e092a9a0 0x55c6e0939420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:49728/0 (socket says 172.21.15.16:49728) 2024-09-12T02:12:54.846 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.839+0000 7f0ea17b9640 1 -- 172.21.15.16:0/52987833 learned_addr learned my addr 172.21.15.16:0/52987833 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:12:54.846 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.839+0000 7f0ea1fba640 1 --2- 172.21.15.16:0/52987833 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6e092b2e0 0x55c6e0939960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:54.846 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.839+0000 7f0ea17b9640 1 -- 172.21.15.16:0/52987833 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6e092b2e0 msgr2=0x55c6e0939960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:54.846 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.839+0000 7f0ea17b9640 1 --2- 172.21.15.16:0/52987833 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6e092b2e0 0x55c6e0939960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:54.846 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.839+0000 7f0ea17b9640 1 -- 172.21.15.16:0/52987833 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c6e0930370 msgr2=0x55c6e093a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:54.846 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.839+0000 7f0ea17b9640 1 --2- 172.21.15.16:0/52987833 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c6e0930370 0x55c6e093a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:54.846 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.839+0000 7f0ea17b9640 1 -- 172.21.15.16:0/52987833 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6e093ae50 con 0x55c6e092a9a0 2024-09-12T02:12:54.847 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.839+0000 7f0ea17b9640 1 --2- 172.21.15.16:0/52987833 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c6e092a9a0 0x55c6e0939420 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f0e8c009fd0 tx=0x7f0e8c009dd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aed0d79166b08511 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:54.847 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.839+0000 7f0e857fa640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-09-12T02:12:54.847 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.839+0000 7f0e857fa640 1 -- 172.21.15.16:0/52987833 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e8c031860 con 0x55c6e092a9a0 2024-09-12T02:12:54.847 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.839+0000 7f0ea7090740 1 -- 172.21.15.16:0/52987833 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e8c035070 con 0x55c6e092a9a0 2024-09-12T02:12:54.847 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.839+0000 7f0e857fa640 1 -- 172.21.15.16:0/52987833 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f0e8c029040 con 0x55c6e092a9a0 2024-09-12T02:12:54.847 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.840+0000 7f0e857fa640 1 -- 172.21.15.16:0/52987833 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f0e8c031a00 con 0x55c6e092a9a0 2024-09-12T02:12:54.847 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.840+0000 7f0ea7090740 10 client.27727 fetch_fsmap learned FSMap version 25 2024-09-12T02:12:54.847 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.840+0000 7f0ea7090740 1 -- 172.21.15.16:0/52987833 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c6e093c160 con 0x55c6e092a9a0 2024-09-12T02:12:54.847 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.840+0000 7f0e857fa640 1 -- 172.21.15.16:0/52987833 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e8c035070 con 0x55c6e092a9a0 2024-09-12T02:12:54.847 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.840+0000 7f0e857fa640 1 -- 172.21.15.16:0/52987833 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f0e8c038070 con 0x55c6e092a9a0 2024-09-12T02:12:54.847 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.840+0000 7f0ea7090740 10 client.27727 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:12:54.847 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.840+0000 7f0ea7090740 10 client.27727 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:12:54.848 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.840+0000 7f0ea7090740 20 client.27727 populate_metadata read hostname 'smithi016' 2024-09-12T02:12:54.848 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.840+0000 7f0ea7090740 1 --2- 172.21.15.16:0/52987833 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55c6e09408b0 0x55c6e09ad8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:54.848 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.840+0000 7f0ea7090740 4 client.27727 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:12:54.848 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.840+0000 7f0ea7090740 1 -- 172.21.15.16:0/52987833 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c6e0895d70 con 0x55c6e09408b0 2024-09-12T02:12:54.848 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.840+0000 7f0ea1fba640 1 --2- 172.21.15.16:0/52987833 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55c6e09408b0 0x55c6e09ad8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:54.848 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.841+0000 7f0ea1fba640 1 --2- 172.21.15.16:0/52987833 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55c6e09408b0 0x55c6e09ad8f0 crc :-1 s=READY pgs=327 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=767ecbf423069252 server_cookie=cbba3571ad2fe8c5 in_seq=0 out_seq=0 2024-09-12T02:12:54.848 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.841+0000 7f0e857fa640 10 client.27727 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:12:54.854 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.854+0000 7f0e857fa640 1 -- 172.21.15.16:0/52987833 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c6e0895d70 con 0x55c6e09408b0 2024-09-12T02:12:54.854 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.854+0000 7f0e857fa640 10 client.27727 handle_command_reply: tid=0 2024-09-12T02:12:54.854 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.854+0000 7f0ea7090740 1 client.27727 shutdown 2024-09-12T02:12:54.854 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.854+0000 7f0ea7090740 1 -- 172.21.15.16:0/52987833 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c6e092a9a0 msgr2=0x55c6e0939420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:54.855 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.854+0000 7f0ea7090740 1 --2- 172.21.15.16:0/52987833 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c6e092a9a0 0x55c6e0939420 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f0e8c009fd0 tx=0x7f0e8c009dd0 comp rx=0 tx=0).stop 2024-09-12T02:12:54.855 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.854+0000 7f0ea7090740 1 -- 172.21.15.16:0/52987833 shutdown_connections 2024-09-12T02:12:54.855 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.854+0000 7f0ea7090740 1 --2- 172.21.15.16:0/52987833 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55c6e09408b0 0x55c6e09ad8f0 crc :-1 s=READY pgs=327 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:54.855 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.854+0000 7f0ea7090740 1 --2- 172.21.15.16:0/52987833 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c6e0930370 0x55c6e093a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:54.855 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.854+0000 7f0e857fa640 0 client.27727 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:12:54.855 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.854+0000 7f0ea7090740 1 --2- 172.21.15.16:0/52987833 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6e092b2e0 0x55c6e0939960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:54.855 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.854+0000 7f0ea7090740 1 --2- 172.21.15.16:0/52987833 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c6e092a9a0 0x55c6e0939420 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:54.855 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.854+0000 7f0ea7090740 1 -- 172.21.15.16:0/52987833 >> 172.21.15.16:0/52987833 conn(0x55c6e0926010 msgr2=0x55c6e09249a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:54.855 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.854+0000 7f0ea7090740 1 -- 172.21.15.16:0/52987833 shutdown_connections 2024-09-12T02:12:54.855 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.854+0000 7f0ea7090740 1 -- 172.21.15.16:0/52987833 wait complete. 2024-09-12T02:12:54.855 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.854+0000 7f0ea7090740 20 client.27727 trim_cache size 0 max 16384 2024-09-12T02:12:54.856 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-09-12T02:12:54.856 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-09-12T02:12:54.856 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-09-12T02:12:54.856 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-09-12T02:12:54.856 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.855+0000 7f0ea421e640 1 -- 172.21.15.16:0/2134352490 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f0e880a2cb0 msgr2=0x7f0e880a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:54.857 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.855+0000 7f0ea421e640 1 --2- 172.21.15.16:0/2134352490 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f0e880a2cb0 0x7f0e880a5170 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f0e9400c7d0 tx=0x7f0e940c6b80 comp rx=0 tx=0).stop 2024-09-12T02:12:54.857 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.855+0000 7f0ea421e640 1 -- 172.21.15.16:0/2134352490 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e9c107820 msgr2=0x7f0e9c19f430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:54.857 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.855+0000 7f0ea421e640 1 --2- 172.21.15.16:0/2134352490 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e9c107820 0x7f0e9c19f430 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f0e9400eb40 tx=0x7f0e9400c2d0 comp rx=0 tx=0).stop 2024-09-12T02:12:54.857 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.855+0000 7f0ea421e640 1 -- 172.21.15.16:0/2134352490 shutdown_connections 2024-09-12T02:12:54.857 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.855+0000 7f0ea421e640 1 --2- 172.21.15.16:0/2134352490 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f0e880a2cb0 0x7f0e880a5170 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:54.857 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.855+0000 7f0ea421e640 1 --2- 172.21.15.16:0/2134352490 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e9c107820 0x7f0e9c19f430 unknown :-1 s=CLOSED pgs=1533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:54.857 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.855+0000 7f0ea421e640 1 --2- 172.21.15.16:0/2134352490 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0e9c104ef0 0x7f0e9c1983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:54.857 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.855+0000 7f0ea421e640 1 --2- 172.21.15.16:0/2134352490 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e9c1025c0 0x7f0e9c197e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:54.857 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.855+0000 7f0ea421e640 1 -- 172.21.15.16:0/2134352490 >> 172.21.15.16:0/2134352490 conn(0x7f0e9c0fc1e0 msgr2=0x7f0e9c102fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:54.857 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.855+0000 7f0ea421e640 1 -- 172.21.15.16:0/2134352490 shutdown_connections 2024-09-12T02:12:54.857 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:12:54.855+0000 7f0ea421e640 1 -- 172.21.15.16:0/2134352490 wait complete. 2024-09-12T02:12:54.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.894+0000 7fe99739d640 1 -- 172.21.15.16:0/2898549774 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe990102d40 msgr2=0x7fe990103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:54.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.894+0000 7fe99739d640 1 --2- 172.21.15.16:0/2898549774 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe990102d40 0x7fe990103120 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7fe98c009900 tx=0x7fe98c02a480 comp rx=0 tx=0).stop 2024-09-12T02:12:54.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.894+0000 7fe99739d640 1 -- 172.21.15.16:0/2898549774 shutdown_connections 2024-09-12T02:12:54.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.894+0000 7fe99739d640 1 --2- 172.21.15.16:0/2898549774 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe99010c170 0x7fe99010e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:54.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.894+0000 7fe99739d640 1 --2- 172.21.15.16:0/2898549774 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe990103660 0x7fe99010bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:54.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.894+0000 7fe99739d640 1 --2- 172.21.15.16:0/2898549774 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe990102d40 0x7fe990103120 unknown :-1 s=CLOSED pgs=1537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:54.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.894+0000 7fe99739d640 1 -- 172.21.15.16:0/2898549774 >> 172.21.15.16:0/2898549774 conn(0x7fe9900fc7a0 msgr2=0x7fe9900febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:54.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.894+0000 7fe99739d640 1 -- 172.21.15.16:0/2898549774 shutdown_connections 2024-09-12T02:12:54.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.894+0000 7fe99739d640 1 -- 172.21.15.16:0/2898549774 wait complete. 2024-09-12T02:12:54.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.894+0000 7fe99739d640 1 Processor -- start 2024-09-12T02:12:54.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.895+0000 7fe99739d640 1 -- start start 2024-09-12T02:12:54.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.895+0000 7fe99739d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe990102d40 0x7fe9901984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:54.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.895+0000 7fe99739d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe990103660 0x7fe9901989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:54.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.895+0000 7fe99739d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe99010c170 0x7fe99019cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:54.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.895+0000 7fe99739d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe990111f70 con 0x7fe990103660 2024-09-12T02:12:54.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.895+0000 7fe99739d640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe990111df0 con 0x7fe990102d40 2024-09-12T02:12:54.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.895+0000 7fe99739d640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe9901120f0 con 0x7fe99010c170 2024-09-12T02:12:54.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.895+0000 7fe994938640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe990103660 0x7fe9901989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:54.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.895+0000 7fe994938640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe990103660 0x7fe9901989f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51742/0 (socket says 172.21.15.16:51742) 2024-09-12T02:12:54.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.895+0000 7fe994938640 1 -- 172.21.15.16:0/3323297329 learned_addr learned my addr 172.21.15.16:0/3323297329 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:12:54.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.895+0000 7fe995139640 1 --2- 172.21.15.16:0/3323297329 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe990102d40 0x7fe9901984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:54.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.895+0000 7fe99593a640 1 --2- 172.21.15.16:0/3323297329 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe99010c170 0x7fe99019cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:54.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.895+0000 7fe994938640 1 -- 172.21.15.16:0/3323297329 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe99010c170 msgr2=0x7fe99019cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:54.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.895+0000 7fe994938640 1 --2- 172.21.15.16:0/3323297329 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe99010c170 0x7fe99019cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:54.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.895+0000 7fe994938640 1 -- 172.21.15.16:0/3323297329 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe990102d40 msgr2=0x7fe9901984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:54.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.895+0000 7fe994938640 1 --2- 172.21.15.16:0/3323297329 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe990102d40 0x7fe9901984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:54.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.895+0000 7fe994938640 1 -- 172.21.15.16:0/3323297329 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe99019d460 con 0x7fe990103660 2024-09-12T02:12:54.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.896+0000 7fe994938640 1 --2- 172.21.15.16:0/3323297329 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe990103660 0x7fe9901989f0 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7fe98000d740 tx=0x7fe98000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:54.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.896+0000 7fe9867fc640 1 -- 172.21.15.16:0/3323297329 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe980017070 con 0x7fe990103660 2024-09-12T02:12:54.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.896+0000 7fe99739d640 1 -- 172.21.15.16:0/3323297329 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe99019d750 con 0x7fe990103660 2024-09-12T02:12:54.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.896+0000 7fe9867fc640 1 -- 172.21.15.16:0/3323297329 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe98000de50 con 0x7fe990103660 2024-09-12T02:12:54.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.896+0000 7fe9867fc640 1 -- 172.21.15.16:0/3323297329 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe98000f620 con 0x7fe990103660 2024-09-12T02:12:54.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.896+0000 7fe99739d640 1 -- 172.21.15.16:0/3323297329 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9901a5030 con 0x7fe990103660 2024-09-12T02:12:54.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.898+0000 7fe999776740 1 Processor -- start 2024-09-12T02:12:54.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.898+0000 7fe9867fc640 1 -- 172.21.15.16:0/3323297329 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fe980009c30 con 0x7fe990103660 2024-09-12T02:12:54.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.898+0000 7fe999776740 1 -- start start 2024-09-12T02:12:54.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.899+0000 7fe999776740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e72c1ecb40 0x55e72c1eb090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:54.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.899+0000 7fe999776740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e72c1eb5d0 0x55e72c1eba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:54.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.899+0000 7fe999776740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e72c1f6370 0x55e72c1f8760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:54.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.899+0000 7fe999776740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e72c1512f0 con 0x55e72c1eb5d0 2024-09-12T02:12:54.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.899+0000 7fe999776740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55e72c0f8cf0 con 0x55e72c1f6370 2024-09-12T02:12:54.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.899+0000 7fe999776740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55e72c14d2b0 con 0x55e72c1ecb40 2024-09-12T02:12:54.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.899+0000 7fe995139640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e72c1eb5d0 0x55e72c1eba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:54.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.899+0000 7fe995139640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e72c1eb5d0 0x55e72c1eba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51744/0 (socket says 172.21.15.16:51744) 2024-09-12T02:12:54.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.899+0000 7fe995139640 1 -- 172.21.15.16:0/1856549012 learned_addr learned my addr 172.21.15.16:0/1856549012 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:12:54.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.899+0000 7fe99593a640 1 --2- 172.21.15.16:0/1856549012 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e72c1ecb40 0x55e72c1eb090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:54.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.899+0000 7fe994938640 1 --2- 172.21.15.16:0/1856549012 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e72c1f6370 0x55e72c1f8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:54.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.899+0000 7fe9867fc640 1 --2- 172.21.15.16:0/3323297329 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fe97c0a2b80 0x7fe97c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:54.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.899+0000 7fe995139640 1 -- 172.21.15.16:0/1856549012 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e72c1ecb40 msgr2=0x55e72c1eb090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:54.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.899+0000 7fe995139640 1 --2- 172.21.15.16:0/1856549012 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e72c1ecb40 0x55e72c1eb090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:54.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.899+0000 7fe995139640 1 -- 172.21.15.16:0/1856549012 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e72c1f6370 msgr2=0x55e72c1f8760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:54.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.899+0000 7fe995139640 1 --2- 172.21.15.16:0/1856549012 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e72c1f6370 0x55e72c1f8760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:54.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.899+0000 7fe995139640 1 -- 172.21.15.16:0/1856549012 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e72c0dd3a0 con 0x55e72c1eb5d0 2024-09-12T02:12:54.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.899+0000 7fe99593a640 1 --2- 172.21.15.16:0/3323297329 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fe97c0a2b80 0x7fe97c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:54.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.899+0000 7fe9867fc640 1 -- 172.21.15.16:0/3323297329 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fe9800ceaf0 con 0x7fe990103660 2024-09-12T02:12:54.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.900+0000 7fe995139640 1 --2- 172.21.15.16:0/1856549012 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e72c1eb5d0 0x55e72c1eba10 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7fe98c009900 tx=0x7fe98c031630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3262f953b632839e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:54.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.900+0000 7fe97b7fe640 1 -- 172.21.15.16:0/1856549012 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe98c040070 con 0x55e72c1eb5d0 2024-09-12T02:12:54.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.900+0000 7fe99593a640 1 --2- 172.21.15.16:0/3323297329 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fe97c0a2b80 0x7fe97c0a5040 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fe98800aed0 tx=0x7fe988004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:54.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.900+0000 7fe97b7fe640 1 -- 172.21.15.16:0/1856549012 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe98c005020 con 0x55e72c1eb5d0 2024-09-12T02:12:54.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.900+0000 7fe97b7fe640 1 -- 172.21.15.16:0/1856549012 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe98c035040 con 0x55e72c1eb5d0 2024-09-12T02:12:54.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.900+0000 7fe999776740 1 -- 172.21.15.16:0/1856549012 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e72c1eb5d0 msgr2=0x55e72c1eba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:54.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.900+0000 7fe999776740 1 --2- 172.21.15.16:0/1856549012 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e72c1eb5d0 0x55e72c1eba10 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7fe98c009900 tx=0x7fe98c031630 comp rx=0 tx=0).stop 2024-09-12T02:12:54.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.901+0000 7fe999776740 1 -- 172.21.15.16:0/1856549012 shutdown_connections 2024-09-12T02:12:54.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.901+0000 7fe999776740 1 --2- 172.21.15.16:0/1856549012 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e72c1f6370 0x55e72c1f8760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:54.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.901+0000 7fe999776740 1 --2- 172.21.15.16:0/1856549012 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e72c1eb5d0 0x55e72c1eba10 unknown :-1 s=CLOSED pgs=1539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:54.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.901+0000 7fe999776740 1 --2- 172.21.15.16:0/1856549012 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e72c1ecb40 0x55e72c1eb090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:54.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.901+0000 7fe999776740 1 -- 172.21.15.16:0/1856549012 >> 172.21.15.16:0/1856549012 conn(0x55e72c1b4310 msgr2=0x55e72c1e7c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:54.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.901+0000 7fe999776740 1 -- 172.21.15.16:0/1856549012 shutdown_connections 2024-09-12T02:12:54.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.901+0000 7fe999776740 1 -- 172.21.15.16:0/1856549012 wait complete. 2024-09-12T02:12:54.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.901+0000 7fe999776740 1 -- start start 2024-09-12T02:12:54.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.901+0000 7fe999776740 1 Processor -- start 2024-09-12T02:12:54.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.901+0000 7fe999776740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e72c1eb5d0 0x55e72c1f5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:54.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.901+0000 7fe999776740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e72c1ecb40 0x55e72c1f2f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:54.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.901+0000 7fe999776740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e72c1f6370 0x55e72c1f34f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:54.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.901+0000 7fe999776740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e72c1f9010 con 0x55e72c1f6370 2024-09-12T02:12:54.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.901+0000 7fe999776740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55e72c10c060 con 0x55e72c1eb5d0 2024-09-12T02:12:54.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.901+0000 7fe999776740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55e72c1f9190 con 0x55e72c1ecb40 2024-09-12T02:12:54.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.902+0000 7fe999776740 10 client.0 _ll_register_callbacks cb 0x55e72be5f8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:12:54.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.902+0000 7fe99593a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e72c1f6370 0x55e72c1f34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:54.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.902+0000 7fe99593a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e72c1f6370 0x55e72c1f34f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51756/0 (socket says 172.21.15.16:51756) 2024-09-12T02:12:54.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.902+0000 7fe99593a640 1 -- 172.21.15.16:0/201634444 learned_addr learned my addr 172.21.15.16:0/201634444 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:12:54.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.902+0000 7fe994938640 1 --2- 172.21.15.16:0/201634444 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e72c1ecb40 0x55e72c1f2f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:54.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.902+0000 7fe995139640 1 --2- 172.21.15.16:0/201634444 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e72c1eb5d0 0x55e72c1f5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:54.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.902+0000 7fe99593a640 1 -- 172.21.15.16:0/201634444 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e72c1ecb40 msgr2=0x55e72c1f2f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:54.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.902+0000 7fe99593a640 1 --2- 172.21.15.16:0/201634444 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e72c1ecb40 0x55e72c1f2f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:54.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.902+0000 7fe99593a640 1 -- 172.21.15.16:0/201634444 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e72c1eb5d0 msgr2=0x55e72c1f5ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:54.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.902+0000 7fe99593a640 1 --2- 172.21.15.16:0/201634444 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e72c1eb5d0 0x55e72c1f5ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:54.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.902+0000 7fe99593a640 1 -- 172.21.15.16:0/201634444 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e72bec23c0 con 0x55e72c1f6370 2024-09-12T02:12:54.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.902+0000 7fe995139640 1 --2- 172.21.15.16:0/201634444 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e72c1eb5d0 0x55e72c1f5ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:12:54.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.902+0000 7fe99593a640 1 --2- 172.21.15.16:0/201634444 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e72c1f6370 0x55e72c1f34f0 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7fe988012420 tx=0x7fe988004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1a3424e04226ba9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:54.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.902+0000 7fe978ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:12:54.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.902+0000 7fe978ff9640 1 -- 172.21.15.16:0/201634444 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe988022070 con 0x55e72c1f6370 2024-09-12T02:12:54.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.902+0000 7fe978ff9640 1 -- 172.21.15.16:0/201634444 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fe98800fbf0 con 0x55e72c1f6370 2024-09-12T02:12:54.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.902+0000 7fe999776740 1 -- 172.21.15.16:0/201634444 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe98c005020 con 0x55e72c1f6370 2024-09-12T02:12:54.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.903+0000 7fe978ff9640 1 -- 172.21.15.16:0/201634444 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe98800fdf0 con 0x55e72c1f6370 2024-09-12T02:12:54.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.903+0000 7fe999776740 10 client.18951 fetch_fsmap learned FSMap version 25 2024-09-12T02:12:54.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.903+0000 7fe999776740 1 -- 172.21.15.16:0/201634444 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e72c204a80 con 0x55e72c1f6370 2024-09-12T02:12:54.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.903+0000 7fe978ff9640 1 -- 172.21.15.16:0/201634444 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe98c005020 con 0x55e72c1f6370 2024-09-12T02:12:54.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.903+0000 7fe978ff9640 1 -- 172.21.15.16:0/201634444 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fe98801b040 con 0x55e72c1f6370 2024-09-12T02:12:54.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.903+0000 7fe999776740 10 client.18951 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:12:54.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.903+0000 7fe999776740 10 client.18951 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-12T02:12:54.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.903+0000 7fe999776740 20 client.18951 populate_metadata read hostname 'smithi016' 2024-09-12T02:12:54.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.903+0000 7fe999776740 1 --2- 172.21.15.16:0/201634444 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55e72c205f40 0x55e72c2083b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:54.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.903+0000 7fe999776740 4 client.18951 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:12:54.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.903+0000 7fe999776740 1 -- 172.21.15.16:0/201634444 --> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e72c15e860 con 0x55e72c205f40 2024-09-12T02:12:54.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.903+0000 7fe995139640 1 --2- 172.21.15.16:0/201634444 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55e72c205f40 0x55e72c2083b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:54.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.904+0000 7fe995139640 1 --2- 172.21.15.16:0/201634444 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55e72c205f40 0x55e72c2083b0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=46e42728bdfe5a85 server_cookie=55aedb14bea8718f in_seq=0 out_seq=0 2024-09-12T02:12:54.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.904+0000 7fe978ff9640 10 client.18951 ms_handle_connect on v2:172.21.15.184:6836/1793393087 2024-09-12T02:12:54.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.904+0000 7fe978ff9640 1 -- 172.21.15.16:0/201634444 <== mds.2 v2:172.21.15.184:6836/1793393087 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e72c15e860 con 0x55e72c205f40 2024-09-12T02:12:54.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.904+0000 7fe978ff9640 10 client.18951 handle_command_reply: tid=0 2024-09-12T02:12:54.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.905+0000 7fe999776740 1 client.18951 shutdown 2024-09-12T02:12:54.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.905+0000 7fe999776740 1 -- 172.21.15.16:0/201634444 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e72c1f6370 msgr2=0x55e72c1f34f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:54.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.905+0000 7fe999776740 1 --2- 172.21.15.16:0/201634444 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e72c1f6370 0x55e72c1f34f0 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7fe988012420 tx=0x7fe988004ab0 comp rx=0 tx=0).stop 2024-09-12T02:12:54.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.905+0000 7fe999776740 1 -- 172.21.15.16:0/201634444 shutdown_connections 2024-09-12T02:12:54.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.905+0000 7fe999776740 1 --2- 172.21.15.16:0/201634444 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55e72c205f40 0x55e72c2083b0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:54.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.905+0000 7fe999776740 1 --2- 172.21.15.16:0/201634444 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e72c1f6370 0x55e72c1f34f0 unknown :-1 s=CLOSED pgs=1540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:54.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.905+0000 7fe999776740 1 --2- 172.21.15.16:0/201634444 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e72c1ecb40 0x55e72c1f2f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:54.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.905+0000 7fe999776740 1 --2- 172.21.15.16:0/201634444 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e72c1eb5d0 0x55e72c1f5ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:54.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.905+0000 7fe999776740 1 -- 172.21.15.16:0/201634444 >> 172.21.15.16:0/201634444 conn(0x55e72c1b4310 msgr2=0x55e72c1ed960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:54.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.905+0000 7fe978ff9640 0 client.18951 ms_handle_reset on v2:172.21.15.184:6836/1793393087 2024-09-12T02:12:54.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.905+0000 7fe999776740 1 -- 172.21.15.16:0/201634444 shutdown_connections 2024-09-12T02:12:54.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.905+0000 7fe999776740 1 -- 172.21.15.16:0/201634444 wait complete. 2024-09-12T02:12:54.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.905+0000 7fe999776740 20 client.18951 trim_cache size 0 max 16384 2024-09-12T02:12:54.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.918+0000 7fe999776740 1 Processor -- start 2024-09-12T02:12:54.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.919+0000 7fe999776740 1 -- start start 2024-09-12T02:12:54.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.919+0000 7fe999776740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e72c1eb5d0 0x55e72c2022b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:54.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.919+0000 7fe999776740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e72c1ecb40 0x55e72c1e7700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:54.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.919+0000 7fe999776740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e72c1f6370 0x55e72c1e7c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:54.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.919+0000 7fe999776740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe9901afd70 con 0x55e72c1eb5d0 2024-09-12T02:12:54.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.919+0000 7fe999776740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe9901ad640 con 0x55e72c1ecb40 2024-09-12T02:12:54.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.919+0000 7fe999776740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe9901ab490 con 0x55e72c1f6370 2024-09-12T02:12:54.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.919+0000 7fe995139640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e72c1eb5d0 0x55e72c2022b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:54.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.919+0000 7fe995139640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e72c1eb5d0 0x55e72c2022b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51772/0 (socket says 172.21.15.16:51772) 2024-09-12T02:12:54.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.919+0000 7fe995139640 1 -- 172.21.15.16:0/3249198048 learned_addr learned my addr 172.21.15.16:0/3249198048 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:12:54.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.919+0000 7fe994938640 1 --2- 172.21.15.16:0/3249198048 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e72c1ecb40 0x55e72c1e7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:54.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.919+0000 7fe99593a640 1 --2- 172.21.15.16:0/3249198048 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e72c1f6370 0x55e72c1e7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:54.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.919+0000 7fe995139640 1 -- 172.21.15.16:0/3249198048 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e72c1f6370 msgr2=0x55e72c1e7c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:54.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.919+0000 7fe995139640 1 --2- 172.21.15.16:0/3249198048 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e72c1f6370 0x55e72c1e7c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:54.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.919+0000 7fe995139640 1 -- 172.21.15.16:0/3249198048 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e72c1ecb40 msgr2=0x55e72c1e7700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:54.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.919+0000 7fe995139640 1 --2- 172.21.15.16:0/3249198048 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e72c1ecb40 0x55e72c1e7700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:54.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.919+0000 7fe995139640 1 -- 172.21.15.16:0/3249198048 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e72c1e8180 con 0x55e72c1eb5d0 2024-09-12T02:12:54.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.919+0000 7fe99593a640 1 --2- 172.21.15.16:0/3249198048 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e72c1f6370 0x55e72c1e7c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:12:54.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.920+0000 7fe995139640 1 --2- 172.21.15.16:0/3249198048 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e72c1eb5d0 0x55e72c2022b0 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7fe98c0096c0 tx=0x7fe98c0045b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1d7ea21853cee8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:54.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.920+0000 7fe97affd640 1 -- 172.21.15.16:0/3249198048 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe98c040040 con 0x55e72c1eb5d0 2024-09-12T02:12:54.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.920+0000 7fe97affd640 1 -- 172.21.15.16:0/3249198048 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe98c005330 con 0x55e72c1eb5d0 2024-09-12T02:12:54.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.920+0000 7fe97affd640 1 -- 172.21.15.16:0/3249198048 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe98c035040 con 0x55e72c1eb5d0 2024-09-12T02:12:54.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.920+0000 7fe999776740 1 -- 172.21.15.16:0/3249198048 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e72c1eb5d0 msgr2=0x55e72c2022b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:54.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.920+0000 7fe999776740 1 --2- 172.21.15.16:0/3249198048 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e72c1eb5d0 0x55e72c2022b0 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7fe98c0096c0 tx=0x7fe98c0045b0 comp rx=0 tx=0).stop 2024-09-12T02:12:54.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.920+0000 7fe999776740 1 -- 172.21.15.16:0/3249198048 shutdown_connections 2024-09-12T02:12:54.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.920+0000 7fe999776740 1 --2- 172.21.15.16:0/3249198048 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e72c1f6370 0x55e72c1e7c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:54.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.920+0000 7fe999776740 1 --2- 172.21.15.16:0/3249198048 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e72c1ecb40 0x55e72c1e7700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:54.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.921+0000 7fe999776740 1 --2- 172.21.15.16:0/3249198048 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e72c1eb5d0 0x55e72c2022b0 unknown :-1 s=CLOSED pgs=1541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:54.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.921+0000 7fe999776740 1 -- 172.21.15.16:0/3249198048 >> 172.21.15.16:0/3249198048 conn(0x55e72c1b4310 msgr2=0x55e72c1f2f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:54.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.921+0000 7fe999776740 1 -- 172.21.15.16:0/3249198048 shutdown_connections 2024-09-12T02:12:54.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.921+0000 7fe999776740 1 -- 172.21.15.16:0/3249198048 wait complete. 2024-09-12T02:12:54.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.921+0000 7fe999776740 1 -- start start 2024-09-12T02:12:54.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.921+0000 7fe999776740 1 Processor -- start 2024-09-12T02:12:54.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.921+0000 7fe999776740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e72c1eb5d0 0x55e72c1eff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:54.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.921+0000 7fe999776740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e72c1ecb40 0x55e72c1f0490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:54.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.921+0000 7fe999776740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e72c1f6370 0x55e72c1f09d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:54.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.921+0000 7fe999776740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe9901abb00 con 0x55e72c1eb5d0 2024-09-12T02:12:54.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.921+0000 7fe999776740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe9901acf70 con 0x55e72c1ecb40 2024-09-12T02:12:54.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.921+0000 7fe999776740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe9901ac1d0 con 0x55e72c1f6370 2024-09-12T02:12:54.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.922+0000 7fe999776740 10 client.0 _ll_register_callbacks cb 0x7fe98800ac50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:12:54.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.922+0000 7fe995139640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e72c1eb5d0 0x55e72c1eff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:54.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.922+0000 7fe995139640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e72c1eb5d0 0x55e72c1eff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51780/0 (socket says 172.21.15.16:51780) 2024-09-12T02:12:54.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.922+0000 7fe995139640 1 -- 172.21.15.16:0/2912070574 learned_addr learned my addr 172.21.15.16:0/2912070574 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:12:54.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.922+0000 7fe994938640 1 --2- 172.21.15.16:0/2912070574 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e72c1ecb40 0x55e72c1f0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:54.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.922+0000 7fe99593a640 1 --2- 172.21.15.16:0/2912070574 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e72c1f6370 0x55e72c1f09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:54.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.922+0000 7fe995139640 1 -- 172.21.15.16:0/2912070574 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e72c1f6370 msgr2=0x55e72c1f09d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:54.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.922+0000 7fe995139640 1 --2- 172.21.15.16:0/2912070574 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e72c1f6370 0x55e72c1f09d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:54.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.922+0000 7fe995139640 1 -- 172.21.15.16:0/2912070574 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e72c1ecb40 msgr2=0x55e72c1f0490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:54.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.922+0000 7fe995139640 1 --2- 172.21.15.16:0/2912070574 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e72c1ecb40 0x55e72c1f0490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:54.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.922+0000 7fe995139640 1 -- 172.21.15.16:0/2912070574 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e72c26fc50 con 0x55e72c1eb5d0 2024-09-12T02:12:54.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.922+0000 7fe994938640 1 --2- 172.21.15.16:0/2912070574 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e72c1ecb40 0x55e72c1f0490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:12:54.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.922+0000 7fe99593a640 1 --2- 172.21.15.16:0/2912070574 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e72c1f6370 0x55e72c1f09d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:12:54.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.922+0000 7fe995139640 1 --2- 172.21.15.16:0/2912070574 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e72c1eb5d0 0x55e72c1eff50 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7fe98c005a90 tx=0x7fe98c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d55769b7e6612283 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:54.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.922+0000 7fe978ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:12:54.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.922+0000 7fe978ff9640 1 -- 172.21.15.16:0/2912070574 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe98c035040 con 0x55e72c1eb5d0 2024-09-12T02:12:54.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.922+0000 7fe999776740 1 -- 172.21.15.16:0/2912070574 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe98c005330 con 0x55e72c1eb5d0 2024-09-12T02:12:54.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.922+0000 7fe978ff9640 1 -- 172.21.15.16:0/2912070574 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fe98c004750 con 0x55e72c1eb5d0 2024-09-12T02:12:54.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.923+0000 7fe978ff9640 1 -- 172.21.15.16:0/2912070574 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe98c004950 con 0x55e72c1eb5d0 2024-09-12T02:12:54.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.923+0000 7fe999776740 10 client.18957 fetch_fsmap learned FSMap version 25 2024-09-12T02:12:54.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.923+0000 7fe999776740 1 -- 172.21.15.16:0/2912070574 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e72c270ef0 con 0x55e72c1eb5d0 2024-09-12T02:12:54.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.923+0000 7fe978ff9640 1 -- 172.21.15.16:0/2912070574 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe98c005330 con 0x55e72c1eb5d0 2024-09-12T02:12:54.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.923+0000 7fe978ff9640 1 -- 172.21.15.16:0/2912070574 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fe98c058410 con 0x55e72c1eb5d0 2024-09-12T02:12:54.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.923+0000 7fe999776740 10 client.18957 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:12:54.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.923+0000 7fe999776740 10 client.18957 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-12T02:12:54.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.923+0000 7fe999776740 20 client.18957 populate_metadata read hostname 'smithi016' 2024-09-12T02:12:54.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.923+0000 7fe999776740 1 --2- 172.21.15.16:0/2912070574 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55e72c271180 0x55e72c273570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:54.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.923+0000 7fe999776740 4 client.18957 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T02:12:54.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.923+0000 7fe999776740 1 -- 172.21.15.16:0/2912070574 --> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e72c158d70 con 0x55e72c271180 2024-09-12T02:12:54.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.923+0000 7fe994938640 1 --2- 172.21.15.16:0/2912070574 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55e72c271180 0x55e72c273570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:54.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.924+0000 7fe994938640 1 --2- 172.21.15.16:0/2912070574 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55e72c271180 0x55e72c273570 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b962c7432287ef5d server_cookie=da9a427b209fdaec in_seq=0 out_seq=0 2024-09-12T02:12:54.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.924+0000 7fe978ff9640 10 client.18957 ms_handle_connect on v2:172.21.15.184:6836/1793393087 2024-09-12T02:12:54.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.924+0000 7fe978ff9640 1 -- 172.21.15.16:0/2912070574 <== mds.2 v2:172.21.15.184:6836/1793393087 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e72c158d70 con 0x55e72c271180 2024-09-12T02:12:54.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.924+0000 7fe978ff9640 10 client.18957 handle_command_reply: tid=0 2024-09-12T02:12:54.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.924+0000 7fe999776740 1 client.18957 shutdown 2024-09-12T02:12:54.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.924+0000 7fe999776740 1 -- 172.21.15.16:0/2912070574 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e72c1eb5d0 msgr2=0x55e72c1eff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:54.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.924+0000 7fe999776740 1 --2- 172.21.15.16:0/2912070574 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e72c1eb5d0 0x55e72c1eff50 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7fe98c005a90 tx=0x7fe98c000bc0 comp rx=0 tx=0).stop 2024-09-12T02:12:54.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.924+0000 7fe999776740 1 -- 172.21.15.16:0/2912070574 shutdown_connections 2024-09-12T02:12:54.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.924+0000 7fe999776740 1 --2- 172.21.15.16:0/2912070574 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e72c1f6370 0x55e72c1f09d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:54.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.924+0000 7fe999776740 1 --2- 172.21.15.16:0/2912070574 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e72c1ecb40 0x55e72c1f0490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:54.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.924+0000 7fe999776740 1 --2- 172.21.15.16:0/2912070574 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55e72c271180 0x55e72c273570 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:54.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.924+0000 7fe978ff9640 0 client.18957 ms_handle_reset on v2:172.21.15.184:6836/1793393087 2024-09-12T02:12:54.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.924+0000 7fe999776740 1 --2- 172.21.15.16:0/2912070574 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e72c1eb5d0 0x55e72c1eff50 unknown :-1 s=CLOSED pgs=1542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:54.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.924+0000 7fe999776740 1 -- 172.21.15.16:0/2912070574 >> 172.21.15.16:0/2912070574 conn(0x55e72c1b4310 msgr2=0x55e72c1f2f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:54.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.924+0000 7fe999776740 1 -- 172.21.15.16:0/2912070574 shutdown_connections 2024-09-12T02:12:54.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.924+0000 7fe999776740 1 -- 172.21.15.16:0/2912070574 wait complete. 2024-09-12T02:12:54.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.925+0000 7fe999776740 20 client.18957 trim_cache size 0 max 16384 2024-09-12T02:12:54.935 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T02:12:54.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.925+0000 7fe99739d640 1 -- 172.21.15.16:0/3323297329 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fe97c0a2b80 msgr2=0x7fe97c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:54.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.925+0000 7fe99739d640 1 --2- 172.21.15.16:0/3323297329 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fe97c0a2b80 0x7fe97c0a5040 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fe98800aed0 tx=0x7fe988004a40 comp rx=0 tx=0).stop 2024-09-12T02:12:54.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.925+0000 7fe99739d640 1 -- 172.21.15.16:0/3323297329 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe990103660 msgr2=0x7fe9901989f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:54.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.925+0000 7fe99739d640 1 --2- 172.21.15.16:0/3323297329 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe990103660 0x7fe9901989f0 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7fe98000d740 tx=0x7fe98000dc00 comp rx=0 tx=0).stop 2024-09-12T02:12:54.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.925+0000 7fe99739d640 1 -- 172.21.15.16:0/3323297329 shutdown_connections 2024-09-12T02:12:54.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.925+0000 7fe99739d640 1 --2- 172.21.15.16:0/3323297329 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fe97c0a2b80 0x7fe97c0a5040 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:54.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.925+0000 7fe99739d640 1 --2- 172.21.15.16:0/3323297329 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe99010c170 0x7fe99019cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:54.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.925+0000 7fe99739d640 1 --2- 172.21.15.16:0/3323297329 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe990103660 0x7fe9901989f0 unknown :-1 s=CLOSED pgs=1538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:54.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.925+0000 7fe99739d640 1 --2- 172.21.15.16:0/3323297329 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe990102d40 0x7fe9901984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:54.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.925+0000 7fe99739d640 1 -- 172.21.15.16:0/3323297329 >> 172.21.15.16:0/3323297329 conn(0x7fe9900fc7a0 msgr2=0x7fe9900fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:54.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.925+0000 7fe99739d640 1 -- 172.21.15.16:0/3323297329 shutdown_connections 2024-09-12T02:12:54.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:54.925+0000 7fe99739d640 1 -- 172.21.15.16:0/3323297329 wait complete. 2024-09-12T02:12:55.769 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-12T02:12:56.031 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:12:56.153 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:12:56 smithi016 ceph-mon[30947]: pgmap v665: 97 pgs: 97 active+clean; 1.7 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-09-12T02:12:56.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:12:56 smithi184 ceph-mon[35593]: pgmap v665: 97 pgs: 97 active+clean; 1.7 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-09-12T02:12:56.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:12:56 smithi033 ceph-mon[35332]: pgmap v665: 97 pgs: 97 active+clean; 1.7 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-09-12T02:12:56.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.480+0000 7fd61e9a0640 1 -- 172.21.15.16:0/1761596941 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd618102380 msgr2=0x7fd61810e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:56.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.480+0000 7fd61e9a0640 1 --2- 172.21.15.16:0/1761596941 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd618102380 0x7fd61810e9c0 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7fd610009cd0 tx=0x7fd61002a8e0 comp rx=0 tx=0).stop 2024-09-12T02:12:56.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.480+0000 7fd61e9a0640 1 -- 172.21.15.16:0/1761596941 shutdown_connections 2024-09-12T02:12:56.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.480+0000 7fd61e9a0640 1 --2- 172.21.15.16:0/1761596941 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd618102380 0x7fd61810e9c0 unknown :-1 s=CLOSED pgs=1543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:56.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.480+0000 7fd61e9a0640 1 --2- 172.21.15.16:0/1761596941 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6181019e0 0x7fd618101e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:56.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.480+0000 7fd61e9a0640 1 --2- 172.21.15.16:0/1761596941 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd618107af0 0x7fd618107ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:56.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.480+0000 7fd61e9a0640 1 -- 172.21.15.16:0/1761596941 >> 172.21.15.16:0/1761596941 conn(0x7fd6180fd700 msgr2=0x7fd6180ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:56.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.480+0000 7fd61e9a0640 1 -- 172.21.15.16:0/1761596941 shutdown_connections 2024-09-12T02:12:56.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.480+0000 7fd61e9a0640 1 -- 172.21.15.16:0/1761596941 wait complete. 2024-09-12T02:12:56.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.481+0000 7fd61e9a0640 1 Processor -- start 2024-09-12T02:12:56.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.481+0000 7fd61e9a0640 1 -- start start 2024-09-12T02:12:56.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.481+0000 7fd61e9a0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6181019e0 0x7fd618198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:56.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.481+0000 7fd61e9a0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd618102380 0x7fd618198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:56.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.481+0000 7fd61e9a0640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd618107af0 0x7fd61819cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:56.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.481+0000 7fd61e9a0640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd618111ef0 con 0x7fd618102380 2024-09-12T02:12:56.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.481+0000 7fd61e9a0640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd618111d70 con 0x7fd6181019e0 2024-09-12T02:12:56.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.481+0000 7fd61e9a0640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd618112070 con 0x7fd618107af0 2024-09-12T02:12:56.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.481+0000 7fd61d19d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd618102380 0x7fd618198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:56.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.481+0000 7fd61d19d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd618102380 0x7fd618198870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51794/0 (socket says 172.21.15.16:51794) 2024-09-12T02:12:56.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.481+0000 7fd61d19d640 1 -- 172.21.15.16:0/1296254265 learned_addr learned my addr 172.21.15.16:0/1296254265 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:12:56.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.481+0000 7fd61d99e640 1 --2- 172.21.15.16:0/1296254265 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6181019e0 0x7fd618198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:56.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.482+0000 7fd61e19f640 1 --2- 172.21.15.16:0/1296254265 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd618107af0 0x7fd61819cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:56.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.482+0000 7fd61d19d640 1 -- 172.21.15.16:0/1296254265 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd618107af0 msgr2=0x7fd61819cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:56.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.482+0000 7fd61d19d640 1 --2- 172.21.15.16:0/1296254265 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd618107af0 0x7fd61819cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:56.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.482+0000 7fd61d19d640 1 -- 172.21.15.16:0/1296254265 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6181019e0 msgr2=0x7fd618198330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:56.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.482+0000 7fd61d19d640 1 --2- 172.21.15.16:0/1296254265 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6181019e0 0x7fd618198330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:56.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.482+0000 7fd61d19d640 1 -- 172.21.15.16:0/1296254265 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd61819d2e0 con 0x7fd618102380 2024-09-12T02:12:56.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.482+0000 7fd61d99e640 1 --2- 172.21.15.16:0/1296254265 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6181019e0 0x7fd618198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:12:56.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.482+0000 7fd61e19f640 1 --2- 172.21.15.16:0/1296254265 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd618107af0 0x7fd61819cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:12:56.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.482+0000 7fd61d19d640 1 --2- 172.21.15.16:0/1296254265 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd618102380 0x7fd618198870 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7fd60800d950 tx=0x7fd60800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:56.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.482+0000 7fd60effd640 1 -- 172.21.15.16:0/1296254265 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd608017030 con 0x7fd618102380 2024-09-12T02:12:56.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.482+0000 7fd61e9a0640 1 -- 172.21.15.16:0/1296254265 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd61819d5d0 con 0x7fd618102380 2024-09-12T02:12:56.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.482+0000 7fd60effd640 1 -- 172.21.15.16:0/1296254265 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd608004020 con 0x7fd618102380 2024-09-12T02:12:56.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.482+0000 7fd60effd640 1 -- 172.21.15.16:0/1296254265 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6080047a0 con 0x7fd618102380 2024-09-12T02:12:56.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.482+0000 7fd61e9a0640 1 -- 172.21.15.16:0/1296254265 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6181a4eb0 con 0x7fd618102380 2024-09-12T02:12:56.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.484+0000 7fd60effd640 1 -- 172.21.15.16:0/1296254265 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fd6080041d0 con 0x7fd618102380 2024-09-12T02:12:56.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.485+0000 7fd621fdb740 1 Processor -- start 2024-09-12T02:12:56.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.485+0000 7fd621fdb740 1 -- start start 2024-09-12T02:12:56.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.485+0000 7fd621fdb740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56015c3acd20 0x56015c3ab270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:56.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.485+0000 7fd621fdb740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56015c3ab7b0 0x56015c3abbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:56.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.485+0000 7fd621fdb740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56015c3b6550 0x56015c3b8940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:56.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.485+0000 7fd621fdb740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56015c3112f0 con 0x56015c3ab7b0 2024-09-12T02:12:56.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.485+0000 7fd621fdb740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56015c2b8cf0 con 0x56015c3b6550 2024-09-12T02:12:56.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.485+0000 7fd621fdb740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x56015c30d2b0 con 0x56015c3acd20 2024-09-12T02:12:56.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.485+0000 7fd60effd640 1 --2- 172.21.15.16:0/1296254265 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fd6040a2b80 0x7fd6040a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:56.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.485+0000 7fd61d99e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56015c3ab7b0 0x56015c3abbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:56.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.485+0000 7fd61d99e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56015c3ab7b0 0x56015c3abbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51806/0 (socket says 172.21.15.16:51806) 2024-09-12T02:12:56.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.485+0000 7fd61d99e640 1 -- 172.21.15.16:0/360446272 learned_addr learned my addr 172.21.15.16:0/360446272 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:12:56.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.485+0000 7fd61e19f640 1 --2- 172.21.15.16:0/1296254265 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fd6040a2b80 0x7fd6040a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:56.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.485+0000 7fd60effd640 1 -- 172.21.15.16:0/1296254265 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fd6080cea50 con 0x7fd618102380 2024-09-12T02:12:56.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.485+0000 7fd61d19d640 1 --2- 172.21.15.16:0/360446272 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56015c3b6550 0x56015c3b8940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:56.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.486+0000 7fd61d99e640 1 -- 172.21.15.16:0/360446272 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56015c3acd20 msgr2=0x56015c3ab270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:56.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.486+0000 7fd61d99e640 1 --2- 172.21.15.16:0/360446272 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56015c3acd20 0x56015c3ab270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:56.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.486+0000 7fd61d99e640 1 -- 172.21.15.16:0/360446272 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56015c3b6550 msgr2=0x56015c3b8940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:56.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.486+0000 7fd61d99e640 1 --2- 172.21.15.16:0/360446272 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56015c3b6550 0x56015c3b8940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:56.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.486+0000 7fd61d99e640 1 -- 172.21.15.16:0/360446272 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56015c29d3a0 con 0x56015c3ab7b0 2024-09-12T02:12:56.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.486+0000 7fd61d99e640 1 --2- 172.21.15.16:0/360446272 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56015c3ab7b0 0x56015c3abbf0 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7fd614008be0 tx=0x7fd61400e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b04ec4a34577ef0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:56.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.486+0000 7fd603fff640 1 -- 172.21.15.16:0/360446272 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd614014070 con 0x56015c3ab7b0 2024-09-12T02:12:56.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.486+0000 7fd61e19f640 1 --2- 172.21.15.16:0/1296254265 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fd6040a2b80 0x7fd6040a5040 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fd610005e30 tx=0x7fd610005da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:56.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.486+0000 7fd603fff640 1 -- 172.21.15.16:0/360446272 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd6140047a0 con 0x56015c3ab7b0 2024-09-12T02:12:56.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.486+0000 7fd603fff640 1 -- 172.21.15.16:0/360446272 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd614010650 con 0x56015c3ab7b0 2024-09-12T02:12:56.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.487+0000 7fd621fdb740 1 -- 172.21.15.16:0/360446272 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56015c3ab7b0 msgr2=0x56015c3abbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:56.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.487+0000 7fd621fdb740 1 --2- 172.21.15.16:0/360446272 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56015c3ab7b0 0x56015c3abbf0 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7fd614008be0 tx=0x7fd61400e720 comp rx=0 tx=0).stop 2024-09-12T02:12:56.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.487+0000 7fd621fdb740 1 -- 172.21.15.16:0/360446272 shutdown_connections 2024-09-12T02:12:56.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.487+0000 7fd621fdb740 1 --2- 172.21.15.16:0/360446272 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56015c3b6550 0x56015c3b8940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:56.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.487+0000 7fd621fdb740 1 --2- 172.21.15.16:0/360446272 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56015c3ab7b0 0x56015c3abbf0 unknown :-1 s=CLOSED pgs=1545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:56.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.487+0000 7fd621fdb740 1 --2- 172.21.15.16:0/360446272 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56015c3acd20 0x56015c3ab270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:56.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.487+0000 7fd621fdb740 1 -- 172.21.15.16:0/360446272 >> 172.21.15.16:0/360446272 conn(0x56015c374310 msgr2=0x56015c3a7e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:56.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.487+0000 7fd621fdb740 1 -- 172.21.15.16:0/360446272 shutdown_connections 2024-09-12T02:12:56.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.487+0000 7fd621fdb740 1 -- 172.21.15.16:0/360446272 wait complete. 2024-09-12T02:12:56.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.487+0000 7fd621fdb740 1 -- start start 2024-09-12T02:12:56.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.488+0000 7fd621fdb740 1 Processor -- start 2024-09-12T02:12:56.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.488+0000 7fd621fdb740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56015c3ab7b0 0x56015c3b5f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:56.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.488+0000 7fd621fdb740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56015c3acd20 0x56015c3b31b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:56.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.488+0000 7fd621fdb740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56015c3b6550 0x56015c3b3750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:56.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.488+0000 7fd621fdb740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56015c3b9250 con 0x56015c3acd20 2024-09-12T02:12:56.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.488+0000 7fd621fdb740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56015c2cc060 con 0x56015c3ab7b0 2024-09-12T02:12:56.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.488+0000 7fd621fdb740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x56015c3b93d0 con 0x56015c3b6550 2024-09-12T02:12:56.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.488+0000 7fd621fdb740 10 client.0 _ll_register_callbacks cb 0x56015c01f8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:12:56.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.488+0000 7fd61d19d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56015c3acd20 0x56015c3b31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:56.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.488+0000 7fd61d19d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56015c3acd20 0x56015c3b31b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51816/0 (socket says 172.21.15.16:51816) 2024-09-12T02:12:56.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.488+0000 7fd61d19d640 1 -- 172.21.15.16:0/1477706451 learned_addr learned my addr 172.21.15.16:0/1477706451 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:12:56.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.488+0000 7fd61d99e640 1 --2- 172.21.15.16:0/1477706451 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56015c3ab7b0 0x56015c3b5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:56.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.488+0000 7fd61e19f640 1 --2- 172.21.15.16:0/1477706451 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56015c3b6550 0x56015c3b3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:56.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.489+0000 7fd61d19d640 1 -- 172.21.15.16:0/1477706451 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56015c3b6550 msgr2=0x56015c3b3750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:56.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.489+0000 7fd61d19d640 1 --2- 172.21.15.16:0/1477706451 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56015c3b6550 0x56015c3b3750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:56.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.489+0000 7fd61d19d640 1 -- 172.21.15.16:0/1477706451 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56015c3ab7b0 msgr2=0x56015c3b5f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:56.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.489+0000 7fd61d19d640 1 --2- 172.21.15.16:0/1477706451 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56015c3ab7b0 0x56015c3b5f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:56.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.489+0000 7fd61d19d640 1 -- 172.21.15.16:0/1477706451 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56015c0823c0 con 0x56015c3acd20 2024-09-12T02:12:56.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.489+0000 7fd61e19f640 1 --2- 172.21.15.16:0/1477706451 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56015c3b6550 0x56015c3b3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:12:56.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.489+0000 7fd61d19d640 1 --2- 172.21.15.16:0/1477706451 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56015c3acd20 0x56015c3b31b0 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7fd60809b2e0 tx=0x7fd60809af50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c05d29bf4771044 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:56.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.489+0000 7fd6017fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:12:56.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.489+0000 7fd6017fa640 1 -- 172.21.15.16:0/1477706451 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6080b4060 con 0x56015c3acd20 2024-09-12T02:12:56.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.489+0000 7fd6017fa640 1 -- 172.21.15.16:0/1477706451 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fd6080979c0 con 0x56015c3acd20 2024-09-12T02:12:56.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.489+0000 7fd621fdb740 1 -- 172.21.15.16:0/1477706451 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd6140047a0 con 0x56015c3acd20 2024-09-12T02:12:56.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.489+0000 7fd6017fa640 1 -- 172.21.15.16:0/1477706451 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd60809f700 con 0x56015c3acd20 2024-09-12T02:12:56.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.489+0000 7fd621fdb740 10 client.18969 fetch_fsmap learned FSMap version 25 2024-09-12T02:12:56.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.489+0000 7fd621fdb740 1 -- 172.21.15.16:0/1477706451 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56015c3c9080 con 0x56015c3acd20 2024-09-12T02:12:56.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.489+0000 7fd6017fa640 1 -- 172.21.15.16:0/1477706451 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd6140047a0 con 0x56015c3acd20 2024-09-12T02:12:56.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.489+0000 7fd6017fa640 1 -- 172.21.15.16:0/1477706451 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fd60809f170 con 0x56015c3acd20 2024-09-12T02:12:56.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.490+0000 7fd621fdb740 10 client.18969 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:12:56.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.490+0000 7fd621fdb740 10 client.18969 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T02:12:56.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.490+0000 7fd621fdb740 20 client.18969 populate_metadata read hostname 'smithi016' 2024-09-12T02:12:56.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.490+0000 7fd621fdb740 1 --2- 172.21.15.16:0/1477706451 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x56015c3ca490 0x56015c3cc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:56.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.490+0000 7fd621fdb740 4 client.18969 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:12:56.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.490+0000 7fd621fdb740 1 -- 172.21.15.16:0/1477706451 --> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56015c31e860 con 0x56015c3ca490 2024-09-12T02:12:56.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.490+0000 7fd61d99e640 1 --2- 172.21.15.16:0/1477706451 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x56015c3ca490 0x56015c3cc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:56.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.490+0000 7fd61d99e640 1 --2- 172.21.15.16:0/1477706451 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x56015c3ca490 0x56015c3cc8c0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=48f2b3fac8428755 server_cookie=b1df8e23505e0468 in_seq=0 out_seq=0 2024-09-12T02:12:56.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.490+0000 7fd6017fa640 10 client.18969 ms_handle_connect on v2:172.21.15.184:6838/4203347941 2024-09-12T02:12:56.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.491+0000 7fd6017fa640 1 -- 172.21.15.16:0/1477706451 <== mds.1 v2:172.21.15.184:6838/4203347941 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56015c31e860 con 0x56015c3ca490 2024-09-12T02:12:56.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.491+0000 7fd6017fa640 10 client.18969 handle_command_reply: tid=0 2024-09-12T02:12:56.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.491+0000 7fd621fdb740 1 client.18969 shutdown 2024-09-12T02:12:56.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.491+0000 7fd621fdb740 1 -- 172.21.15.16:0/1477706451 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56015c3acd20 msgr2=0x56015c3b31b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:56.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.491+0000 7fd621fdb740 1 --2- 172.21.15.16:0/1477706451 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56015c3acd20 0x56015c3b31b0 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7fd60809b2e0 tx=0x7fd60809af50 comp rx=0 tx=0).stop 2024-09-12T02:12:56.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.491+0000 7fd621fdb740 1 -- 172.21.15.16:0/1477706451 shutdown_connections 2024-09-12T02:12:56.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.491+0000 7fd621fdb740 1 --2- 172.21.15.16:0/1477706451 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x56015c3ca490 0x56015c3cc8c0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:56.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.491+0000 7fd621fdb740 1 --2- 172.21.15.16:0/1477706451 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56015c3b6550 0x56015c3b3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:56.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.491+0000 7fd621fdb740 1 --2- 172.21.15.16:0/1477706451 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56015c3acd20 0x56015c3b31b0 unknown :-1 s=CLOSED pgs=1546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:56.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.491+0000 7fd6017fa640 0 client.18969 ms_handle_reset on v2:172.21.15.184:6838/4203347941 2024-09-12T02:12:56.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.491+0000 7fd621fdb740 1 --2- 172.21.15.16:0/1477706451 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56015c3ab7b0 0x56015c3b5f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:56.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.491+0000 7fd621fdb740 1 -- 172.21.15.16:0/1477706451 >> 172.21.15.16:0/1477706451 conn(0x56015c374310 msgr2=0x56015c3adb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:56.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.491+0000 7fd621fdb740 1 -- 172.21.15.16:0/1477706451 shutdown_connections 2024-09-12T02:12:56.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.492+0000 7fd621fdb740 1 -- 172.21.15.16:0/1477706451 wait complete. 2024-09-12T02:12:56.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.492+0000 7fd621fdb740 20 client.18969 trim_cache size 0 max 16384 2024-09-12T02:12:56.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.505+0000 7fd621fdb740 1 Processor -- start 2024-09-12T02:12:56.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.505+0000 7fd621fdb740 1 -- start start 2024-09-12T02:12:56.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.505+0000 7fd621fdb740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56015c3ab7b0 0x56015c3c6980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:56.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.505+0000 7fd621fdb740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56015c3acd20 0x56015c3ae300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:56.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.505+0000 7fd621fdb740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56015c3b6550 0x56015c3ae840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:56.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.505+0000 7fd621fdb740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd6181ad920 con 0x56015c3ab7b0 2024-09-12T02:12:56.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.505+0000 7fd621fdb740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd6181ab1f0 con 0x56015c3acd20 2024-09-12T02:12:56.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.505+0000 7fd621fdb740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd6181a9040 con 0x56015c3b6550 2024-09-12T02:12:56.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.505+0000 7fd61d99e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56015c3ab7b0 0x56015c3c6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:56.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.505+0000 7fd61d99e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56015c3ab7b0 0x56015c3c6980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51820/0 (socket says 172.21.15.16:51820) 2024-09-12T02:12:56.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.505+0000 7fd61d99e640 1 -- 172.21.15.16:0/991022602 learned_addr learned my addr 172.21.15.16:0/991022602 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:12:56.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.505+0000 7fd61e19f640 1 --2- 172.21.15.16:0/991022602 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56015c3b6550 0x56015c3ae840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:56.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.505+0000 7fd61d19d640 1 --2- 172.21.15.16:0/991022602 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56015c3acd20 0x56015c3ae300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:56.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.505+0000 7fd61d99e640 1 -- 172.21.15.16:0/991022602 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56015c3b6550 msgr2=0x56015c3ae840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:56.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.505+0000 7fd61d99e640 1 --2- 172.21.15.16:0/991022602 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56015c3b6550 0x56015c3ae840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:56.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.505+0000 7fd61d99e640 1 -- 172.21.15.16:0/991022602 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56015c3acd20 msgr2=0x56015c3ae300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:56.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.505+0000 7fd61d99e640 1 --2- 172.21.15.16:0/991022602 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56015c3acd20 0x56015c3ae300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:56.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.505+0000 7fd61d99e640 1 -- 172.21.15.16:0/991022602 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56015c3a8f00 con 0x56015c3ab7b0 2024-09-12T02:12:56.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.505+0000 7fd61d19d640 1 --2- 172.21.15.16:0/991022602 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56015c3acd20 0x56015c3ae300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:12:56.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.505+0000 7fd61d99e640 1 --2- 172.21.15.16:0/991022602 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56015c3ab7b0 0x56015c3c6980 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x56015c3a8ca0 tx=0x7fd614008ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae868dd74d7d0701 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:56.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.506+0000 7fd6037fe640 1 -- 172.21.15.16:0/991022602 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd61402b070 con 0x56015c3ab7b0 2024-09-12T02:12:56.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.506+0000 7fd6037fe640 1 -- 172.21.15.16:0/991022602 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd614004bd0 con 0x56015c3ab7b0 2024-09-12T02:12:56.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.506+0000 7fd6037fe640 1 -- 172.21.15.16:0/991022602 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd614014040 con 0x56015c3ab7b0 2024-09-12T02:12:56.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.506+0000 7fd621fdb740 1 -- 172.21.15.16:0/991022602 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56015c3ab7b0 msgr2=0x56015c3c6980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:56.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.506+0000 7fd621fdb740 1 --2- 172.21.15.16:0/991022602 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56015c3ab7b0 0x56015c3c6980 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x56015c3a8ca0 tx=0x7fd614008ee0 comp rx=0 tx=0).stop 2024-09-12T02:12:56.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.506+0000 7fd621fdb740 1 -- 172.21.15.16:0/991022602 shutdown_connections 2024-09-12T02:12:56.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.506+0000 7fd621fdb740 1 --2- 172.21.15.16:0/991022602 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56015c3b6550 0x56015c3ae840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:56.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.506+0000 7fd621fdb740 1 --2- 172.21.15.16:0/991022602 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56015c3acd20 0x56015c3ae300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:56.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.506+0000 7fd621fdb740 1 --2- 172.21.15.16:0/991022602 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56015c3ab7b0 0x56015c3c6980 unknown :-1 s=CLOSED pgs=1547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:56.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.506+0000 7fd621fdb740 1 -- 172.21.15.16:0/991022602 >> 172.21.15.16:0/991022602 conn(0x56015c374310 msgr2=0x56015c3a2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:56.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.506+0000 7fd621fdb740 1 -- 172.21.15.16:0/991022602 shutdown_connections 2024-09-12T02:12:56.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.506+0000 7fd621fdb740 1 -- 172.21.15.16:0/991022602 wait complete. 2024-09-12T02:12:56.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.507+0000 7fd621fdb740 1 -- start start 2024-09-12T02:12:56.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.507+0000 7fd621fdb740 1 Processor -- start 2024-09-12T02:12:56.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.507+0000 7fd621fdb740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56015c3ab7b0 0x56015c3b0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:56.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.507+0000 7fd621fdb740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56015c3acd20 0x56015c3b0690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:56.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.507+0000 7fd621fdb740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56015c3b6550 0x56015c3b0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:56.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.507+0000 7fd621fdb740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd6181a96b0 con 0x56015c3acd20 2024-09-12T02:12:56.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.507+0000 7fd621fdb740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd6181aab20 con 0x56015c3ab7b0 2024-09-12T02:12:56.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.507+0000 7fd621fdb740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd6181a9d80 con 0x56015c3b6550 2024-09-12T02:12:56.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.507+0000 7fd621fdb740 10 client.0 _ll_register_callbacks cb 0x7fd608004540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:12:56.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.507+0000 7fd61d19d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56015c3acd20 0x56015c3b0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:56.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.507+0000 7fd61d99e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56015c3ab7b0 0x56015c3b0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:56.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.507+0000 7fd61d19d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56015c3acd20 0x56015c3b0690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51822/0 (socket says 172.21.15.16:51822) 2024-09-12T02:12:56.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.507+0000 7fd61d19d640 1 -- 172.21.15.16:0/4194332457 learned_addr learned my addr 172.21.15.16:0/4194332457 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:12:56.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.507+0000 7fd61e19f640 1 --2- 172.21.15.16:0/4194332457 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56015c3b6550 0x56015c3b0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:56.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.508+0000 7fd61d19d640 1 -- 172.21.15.16:0/4194332457 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56015c3b6550 msgr2=0x56015c3b0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:56.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.508+0000 7fd61d19d640 1 --2- 172.21.15.16:0/4194332457 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56015c3b6550 0x56015c3b0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:56.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.508+0000 7fd61d19d640 1 -- 172.21.15.16:0/4194332457 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56015c3ab7b0 msgr2=0x56015c3b0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:56.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.508+0000 7fd61d19d640 1 --2- 172.21.15.16:0/4194332457 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56015c3ab7b0 0x56015c3b0150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:56.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.508+0000 7fd61d19d640 1 -- 172.21.15.16:0/4194332457 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56015c434160 con 0x56015c3acd20 2024-09-12T02:12:56.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.508+0000 7fd61d19d640 1 --2- 172.21.15.16:0/4194332457 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56015c3acd20 0x56015c3b0690 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7fd608009610 tx=0x7fd60809a9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3eea8c9c35c17e93 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:56.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.508+0000 7fd6017fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:12:56.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.508+0000 7fd6017fa640 1 -- 172.21.15.16:0/4194332457 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd60809b120 con 0x56015c3acd20 2024-09-12T02:12:56.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.508+0000 7fd6017fa640 1 -- 172.21.15.16:0/4194332457 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fd6080bb380 con 0x56015c3acd20 2024-09-12T02:12:56.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.508+0000 7fd621fdb740 1 -- 172.21.15.16:0/4194332457 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd614004bd0 con 0x56015c3acd20 2024-09-12T02:12:56.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.508+0000 7fd6017fa640 1 -- 172.21.15.16:0/4194332457 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd6080b4020 con 0x56015c3acd20 2024-09-12T02:12:56.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.508+0000 7fd621fdb740 10 client.18975 fetch_fsmap learned FSMap version 25 2024-09-12T02:12:56.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.508+0000 7fd621fdb740 1 -- 172.21.15.16:0/4194332457 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56015c435330 con 0x56015c3acd20 2024-09-12T02:12:56.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.508+0000 7fd6017fa640 1 -- 172.21.15.16:0/4194332457 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd614004bd0 con 0x56015c3acd20 2024-09-12T02:12:56.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.509+0000 7fd6017fa640 1 -- 172.21.15.16:0/4194332457 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fd60809a3a0 con 0x56015c3acd20 2024-09-12T02:12:56.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.509+0000 7fd621fdb740 10 client.18975 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:12:56.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.509+0000 7fd621fdb740 10 client.18975 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T02:12:56.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.509+0000 7fd621fdb740 20 client.18975 populate_metadata read hostname 'smithi016' 2024-09-12T02:12:56.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.509+0000 7fd621fdb740 1 --2- 172.21.15.16:0/4194332457 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x56015c435ab0 0x56015c437ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:56.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.509+0000 7fd621fdb740 4 client.18975 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T02:12:56.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.509+0000 7fd621fdb740 1 -- 172.21.15.16:0/4194332457 --> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56015c318d70 con 0x56015c435ab0 2024-09-12T02:12:56.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.509+0000 7fd61d99e640 1 --2- 172.21.15.16:0/4194332457 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x56015c435ab0 0x56015c437ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:56.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.509+0000 7fd61d99e640 1 --2- 172.21.15.16:0/4194332457 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x56015c435ab0 0x56015c437ed0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a947861b884f9147 server_cookie=f477c6a3f549244e in_seq=0 out_seq=0 2024-09-12T02:12:56.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.509+0000 7fd6017fa640 10 client.18975 ms_handle_connect on v2:172.21.15.184:6838/4203347941 2024-09-12T02:12:56.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.510+0000 7fd6017fa640 1 -- 172.21.15.16:0/4194332457 <== mds.1 v2:172.21.15.184:6838/4203347941 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56015c318d70 con 0x56015c435ab0 2024-09-12T02:12:56.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.510+0000 7fd6017fa640 10 client.18975 handle_command_reply: tid=0 2024-09-12T02:12:56.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.510+0000 7fd621fdb740 1 client.18975 shutdown 2024-09-12T02:12:56.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.510+0000 7fd621fdb740 1 -- 172.21.15.16:0/4194332457 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56015c3acd20 msgr2=0x56015c3b0690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:56.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.510+0000 7fd621fdb740 1 --2- 172.21.15.16:0/4194332457 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56015c3acd20 0x56015c3b0690 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7fd608009610 tx=0x7fd60809a9d0 comp rx=0 tx=0).stop 2024-09-12T02:12:56.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.510+0000 7fd621fdb740 1 -- 172.21.15.16:0/4194332457 shutdown_connections 2024-09-12T02:12:56.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.510+0000 7fd621fdb740 1 --2- 172.21.15.16:0/4194332457 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x56015c435ab0 0x56015c437ed0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:56.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.510+0000 7fd621fdb740 1 --2- 172.21.15.16:0/4194332457 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56015c3b6550 0x56015c3b0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:56.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.510+0000 7fd621fdb740 1 --2- 172.21.15.16:0/4194332457 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56015c3acd20 0x56015c3b0690 unknown :-1 s=CLOSED pgs=1548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:56.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.510+0000 7fd6017fa640 0 client.18975 ms_handle_reset on v2:172.21.15.184:6838/4203347941 2024-09-12T02:12:56.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.510+0000 7fd621fdb740 1 --2- 172.21.15.16:0/4194332457 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56015c3ab7b0 0x56015c3b0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:56.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.510+0000 7fd621fdb740 1 -- 172.21.15.16:0/4194332457 >> 172.21.15.16:0/4194332457 conn(0x56015c374310 msgr2=0x56015c3b4a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:56.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.510+0000 7fd621fdb740 1 -- 172.21.15.16:0/4194332457 shutdown_connections 2024-09-12T02:12:56.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.510+0000 7fd621fdb740 1 -- 172.21.15.16:0/4194332457 wait complete. 2024-09-12T02:12:56.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.510+0000 7fd621fdb740 20 client.18975 trim_cache size 0 max 16384 2024-09-12T02:12:56.515 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T02:12:56.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.511+0000 7fd61e9a0640 1 -- 172.21.15.16:0/1296254265 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fd6040a2b80 msgr2=0x7fd6040a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:56.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.511+0000 7fd61e9a0640 1 --2- 172.21.15.16:0/1296254265 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fd6040a2b80 0x7fd6040a5040 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fd610005e30 tx=0x7fd610005da0 comp rx=0 tx=0).stop 2024-09-12T02:12:56.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.511+0000 7fd61e9a0640 1 -- 172.21.15.16:0/1296254265 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd618102380 msgr2=0x7fd618198870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:56.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.511+0000 7fd61e9a0640 1 --2- 172.21.15.16:0/1296254265 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd618102380 0x7fd618198870 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7fd60800d950 tx=0x7fd60800de10 comp rx=0 tx=0).stop 2024-09-12T02:12:56.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.511+0000 7fd61e9a0640 1 -- 172.21.15.16:0/1296254265 shutdown_connections 2024-09-12T02:12:56.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.511+0000 7fd61e9a0640 1 --2- 172.21.15.16:0/1296254265 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fd6040a2b80 0x7fd6040a5040 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:56.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.511+0000 7fd61e9a0640 1 --2- 172.21.15.16:0/1296254265 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd618107af0 0x7fd61819cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:56.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.511+0000 7fd61e9a0640 1 --2- 172.21.15.16:0/1296254265 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd618102380 0x7fd618198870 unknown :-1 s=CLOSED pgs=1544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:56.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.511+0000 7fd61e9a0640 1 --2- 172.21.15.16:0/1296254265 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6181019e0 0x7fd618198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:56.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.511+0000 7fd61e9a0640 1 -- 172.21.15.16:0/1296254265 >> 172.21.15.16:0/1296254265 conn(0x7fd6180fd700 msgr2=0x7fd6180fddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:56.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.511+0000 7fd61e9a0640 1 -- 172.21.15.16:0/1296254265 shutdown_connections 2024-09-12T02:12:56.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:56.511+0000 7fd61e9a0640 1 -- 172.21.15.16:0/1296254265 wait complete. 2024-09-12T02:12:57.292 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-12T02:12:57.554 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:12:58.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.013+0000 7fa93052c640 1 -- 172.21.15.16:0/225932311 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa928101de0 msgr2=0x7fa9281021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:58.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.013+0000 7fa93052c640 1 --2- 172.21.15.16:0/225932311 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa928101de0 0x7fa9281021c0 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7fa924009900 tx=0x7fa92402a410 comp rx=0 tx=0).stop 2024-09-12T02:12:58.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.013+0000 7fa93052c640 1 -- 172.21.15.16:0/225932311 shutdown_connections 2024-09-12T02:12:58.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.013+0000 7fa93052c640 1 --2- 172.21.15.16:0/225932311 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa928106f10 0x7fa92810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:58.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.013+0000 7fa93052c640 1 --2- 172.21.15.16:0/225932311 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa928102790 0x7fa9281067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:58.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.013+0000 7fa93052c640 1 --2- 172.21.15.16:0/225932311 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa928101de0 0x7fa9281021c0 unknown :-1 s=CLOSED pgs=1549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:58.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.013+0000 7fa93052c640 1 -- 172.21.15.16:0/225932311 >> 172.21.15.16:0/225932311 conn(0x7fa9280fd8c0 msgr2=0x7fa9280ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:58.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.013+0000 7fa93052c640 1 -- 172.21.15.16:0/225932311 shutdown_connections 2024-09-12T02:12:58.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.013+0000 7fa93052c640 1 -- 172.21.15.16:0/225932311 wait complete. 2024-09-12T02:12:58.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.014+0000 7fa93052c640 1 Processor -- start 2024-09-12T02:12:58.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.014+0000 7fa93052c640 1 -- start start 2024-09-12T02:12:58.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.014+0000 7fa93052c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa928101de0 0x7fa9281983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:58.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.014+0000 7fa93052c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa928102790 0x7fa928198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:58.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.014+0000 7fa93052c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa928106f10 0x7fa92819cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:58.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.014+0000 7fa93052c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa928111f70 con 0x7fa928106f10 2024-09-12T02:12:58.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.014+0000 7fa93052c640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa928111df0 con 0x7fa928101de0 2024-09-12T02:12:58.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.014+0000 7fa93052c640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa9281120f0 con 0x7fa928102790 2024-09-12T02:12:58.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.014+0000 7fa92eac9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa928106f10 0x7fa92819cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:58.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.014+0000 7fa92eac9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa928106f10 0x7fa92819cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51842/0 (socket says 172.21.15.16:51842) 2024-09-12T02:12:58.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.014+0000 7fa92eac9640 1 -- 172.21.15.16:0/2679292880 learned_addr learned my addr 172.21.15.16:0/2679292880 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:12:58.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.014+0000 7fa92dac7640 1 --2- 172.21.15.16:0/2679292880 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa928102790 0x7fa928198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:58.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.015+0000 7fa92e2c8640 1 --2- 172.21.15.16:0/2679292880 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa928101de0 0x7fa9281983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:58.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.015+0000 7fa92eac9640 1 -- 172.21.15.16:0/2679292880 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa928102790 msgr2=0x7fa928198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:58.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.015+0000 7fa92eac9640 1 --2- 172.21.15.16:0/2679292880 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa928102790 0x7fa928198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:58.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.015+0000 7fa92eac9640 1 -- 172.21.15.16:0/2679292880 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa928101de0 msgr2=0x7fa9281983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:58.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.015+0000 7fa92eac9640 1 --2- 172.21.15.16:0/2679292880 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa928101de0 0x7fa9281983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:58.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.015+0000 7fa92eac9640 1 -- 172.21.15.16:0/2679292880 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa92819d380 con 0x7fa928106f10 2024-09-12T02:12:58.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.015+0000 7fa92eac9640 1 --2- 172.21.15.16:0/2679292880 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa928106f10 0x7fa92819cca0 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7fa92000cce0 tx=0x7fa92000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:58.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.015+0000 7fa91f7fe640 1 -- 172.21.15.16:0/2679292880 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa92000f450 con 0x7fa928106f10 2024-09-12T02:12:58.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.015+0000 7fa91f7fe640 1 -- 172.21.15.16:0/2679292880 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa92000fad0 con 0x7fa928106f10 2024-09-12T02:12:58.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.015+0000 7fa93052c640 1 -- 172.21.15.16:0/2679292880 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa92819d670 con 0x7fa928106f10 2024-09-12T02:12:58.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.015+0000 7fa91f7fe640 1 -- 172.21.15.16:0/2679292880 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa920018c10 con 0x7fa928106f10 2024-09-12T02:12:58.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.015+0000 7fa93052c640 1 -- 172.21.15.16:0/2679292880 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9281a4f50 con 0x7fa928106f10 2024-09-12T02:12:58.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.017+0000 7fa91f7fe640 1 -- 172.21.15.16:0/2679292880 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa92000f5f0 con 0x7fa928106f10 2024-09-12T02:12:58.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.018+0000 7fa932905740 1 Processor -- start 2024-09-12T02:12:58.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.018+0000 7fa932905740 1 -- start start 2024-09-12T02:12:58.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.018+0000 7fa932905740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561784592b40 0x561784591090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:58.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.018+0000 7fa932905740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5617845915d0 0x561784591a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:58.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.018+0000 7fa932905740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56178459c370 0x56178459e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:58.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.018+0000 7fa932905740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5617844f72f0 con 0x5617845915d0 2024-09-12T02:12:58.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.018+0000 7fa932905740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56178449ecf0 con 0x56178459c370 2024-09-12T02:12:58.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.018+0000 7fa932905740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5617844f32b0 con 0x561784592b40 2024-09-12T02:12:58.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.018+0000 7fa91f7fe640 1 --2- 172.21.15.16:0/2679292880 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fa9140a2b80 0x7fa9140a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:58.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.018+0000 7fa92e2c8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5617845915d0 0x561784591a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:58.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.018+0000 7fa92e2c8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5617845915d0 0x561784591a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51846/0 (socket says 172.21.15.16:51846) 2024-09-12T02:12:58.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.018+0000 7fa92e2c8640 1 -- 172.21.15.16:0/990971890 learned_addr learned my addr 172.21.15.16:0/990971890 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:12:58.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.018+0000 7fa92dac7640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561784592b40 0x561784591090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:58.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.018+0000 7fa92dac7640 1 --2- 172.21.15.16:0/990971890 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56178459c370 0x56178459e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:58.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.018+0000 7fa92eac9640 1 --2- 172.21.15.16:0/2679292880 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fa9140a2b80 0x7fa9140a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:58.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.019+0000 7fa92e2c8640 1 -- 172.21.15.16:0/990971890 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561784592b40 msgr2=0x561784591090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:58.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.019+0000 7fa92e2c8640 1 --2- 172.21.15.16:0/990971890 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561784592b40 0x561784591090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:58.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.019+0000 7fa92e2c8640 1 -- 172.21.15.16:0/990971890 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56178459c370 msgr2=0x56178459e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:58.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.019+0000 7fa92e2c8640 1 --2- 172.21.15.16:0/990971890 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56178459c370 0x56178459e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:58.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.019+0000 7fa91f7fe640 1 -- 172.21.15.16:0/2679292880 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fa9200cf6c0 con 0x7fa928106f10 2024-09-12T02:12:58.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.019+0000 7fa92e2c8640 1 -- 172.21.15.16:0/990971890 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5617844833a0 con 0x5617845915d0 2024-09-12T02:12:58.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.019+0000 7fa92e2c8640 1 --2- 172.21.15.16:0/990971890 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5617845915d0 0x561784591a10 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7fa924033000 tx=0x7fa924032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd178c73d5b15865 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:58.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.019+0000 7fa92eac9640 1 --2- 172.21.15.16:0/2679292880 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fa9140a2b80 0x7fa9140a5040 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fa920061e50 tx=0x7fa92006bdd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:58.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.019+0000 7fa91cff9640 1 -- 172.21.15.16:0/990971890 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa92403f070 con 0x5617845915d0 2024-09-12T02:12:58.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.019+0000 7fa91cff9640 1 -- 172.21.15.16:0/990971890 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9240317b0 con 0x5617845915d0 2024-09-12T02:12:58.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.019+0000 7fa91cff9640 1 -- 172.21.15.16:0/990971890 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa924035070 con 0x5617845915d0 2024-09-12T02:12:58.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.020+0000 7fa932905740 1 -- 172.21.15.16:0/990971890 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5617845915d0 msgr2=0x561784591a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:58.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.020+0000 7fa932905740 1 --2- 172.21.15.16:0/990971890 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5617845915d0 0x561784591a10 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7fa924033000 tx=0x7fa924032ce0 comp rx=0 tx=0).stop 2024-09-12T02:12:58.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.020+0000 7fa932905740 1 -- 172.21.15.16:0/990971890 shutdown_connections 2024-09-12T02:12:58.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.020+0000 7fa932905740 1 --2- 172.21.15.16:0/990971890 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56178459c370 0x56178459e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:58.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.020+0000 7fa932905740 1 --2- 172.21.15.16:0/990971890 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5617845915d0 0x561784591a10 unknown :-1 s=CLOSED pgs=1551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:58.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.020+0000 7fa932905740 1 --2- 172.21.15.16:0/990971890 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561784592b40 0x561784591090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:58.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.020+0000 7fa932905740 1 -- 172.21.15.16:0/990971890 >> 172.21.15.16:0/990971890 conn(0x56178455a310 msgr2=0x56178458dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:58.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.020+0000 7fa932905740 1 -- 172.21.15.16:0/990971890 shutdown_connections 2024-09-12T02:12:58.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.020+0000 7fa932905740 1 -- 172.21.15.16:0/990971890 wait complete. 2024-09-12T02:12:58.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.020+0000 7fa932905740 1 -- start start 2024-09-12T02:12:58.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.020+0000 7fa932905740 1 Processor -- start 2024-09-12T02:12:58.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.021+0000 7fa932905740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5617845915d0 0x56178459bd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:58.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.021+0000 7fa932905740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561784592b40 0x561784599010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:58.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.021+0000 7fa932905740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56178459c370 0x561784599600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:58.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.021+0000 7fa932905740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56178459f010 con 0x5617845915d0 2024-09-12T02:12:58.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.021+0000 7fa932905740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5617844b2060 con 0x561784592b40 2024-09-12T02:12:58.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.021+0000 7fa932905740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x56178459f190 con 0x56178459c370 2024-09-12T02:12:58.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.021+0000 7fa932905740 10 client.0 _ll_register_callbacks cb 0x5617842058c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:12:58.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.021+0000 7fa92dac7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5617845915d0 0x56178459bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:58.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.021+0000 7fa92dac7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5617845915d0 0x56178459bd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51858/0 (socket says 172.21.15.16:51858) 2024-09-12T02:12:58.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.021+0000 7fa92dac7640 1 -- 172.21.15.16:0/1763067802 learned_addr learned my addr 172.21.15.16:0/1763067802 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:12:58.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.021+0000 7fa92e2c8640 1 --2- 172.21.15.16:0/1763067802 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561784592b40 0x561784599010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:58.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.021+0000 7fa92dac7640 1 --2- 172.21.15.16:0/1763067802 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56178459c370 0x561784599600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:58.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.021+0000 7fa92dac7640 1 -- 172.21.15.16:0/1763067802 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56178459c370 msgr2=0x561784599600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:58.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.021+0000 7fa92dac7640 1 --2- 172.21.15.16:0/1763067802 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56178459c370 0x561784599600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:58.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.021+0000 7fa92dac7640 1 -- 172.21.15.16:0/1763067802 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561784592b40 msgr2=0x561784599010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:58.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.021+0000 7fa92dac7640 1 --2- 172.21.15.16:0/1763067802 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561784592b40 0x561784599010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:58.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.021+0000 7fa92dac7640 1 -- 172.21.15.16:0/1763067802 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5617842683c0 con 0x5617845915d0 2024-09-12T02:12:58.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.021+0000 7fa92e2c8640 1 --2- 172.21.15.16:0/1763067802 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561784592b40 0x561784599010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:12:58.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.021+0000 7fa92dac7640 1 --2- 172.21.15.16:0/1763067802 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5617845915d0 0x56178459bd60 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7fa918005fd0 tx=0x7fa9180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b56d4dc16082400 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:58.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.021+0000 7fa911ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:12:58.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.022+0000 7fa911ffb640 1 -- 172.21.15.16:0/1763067802 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa918025060 con 0x5617845915d0 2024-09-12T02:12:58.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.022+0000 7fa911ffb640 1 -- 172.21.15.16:0/1763067802 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fa918010c40 con 0x5617845915d0 2024-09-12T02:12:58.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.022+0000 7fa932905740 1 -- 172.21.15.16:0/1763067802 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9240317b0 con 0x5617845915d0 2024-09-12T02:12:58.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.022+0000 7fa911ffb640 1 -- 172.21.15.16:0/1763067802 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa918010e40 con 0x5617845915d0 2024-09-12T02:12:58.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.022+0000 7fa932905740 10 client.18987 fetch_fsmap learned FSMap version 25 2024-09-12T02:12:58.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.022+0000 7fa932905740 1 -- 172.21.15.16:0/1763067802 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5617845aab10 con 0x5617845915d0 2024-09-12T02:12:58.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.022+0000 7fa911ffb640 1 -- 172.21.15.16:0/1763067802 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9240317b0 con 0x5617845915d0 2024-09-12T02:12:58.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.022+0000 7fa911ffb640 1 -- 172.21.15.16:0/1763067802 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fa91802a070 con 0x5617845915d0 2024-09-12T02:12:58.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.022+0000 7fa932905740 10 client.18987 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:12:58.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.022+0000 7fa932905740 10 client.18987 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:12:58.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.022+0000 7fa932905740 20 client.18987 populate_metadata read hostname 'smithi016' 2024-09-12T02:12:58.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.022+0000 7fa932905740 1 --2- 172.21.15.16:0/1763067802 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5617845abfd0 0x5617845ae3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:58.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.022+0000 7fa932905740 4 client.18987 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:12:58.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.022+0000 7fa932905740 1 -- 172.21.15.16:0/1763067802 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561784504860 con 0x5617845abfd0 2024-09-12T02:12:58.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.022+0000 7fa92e2c8640 1 --2- 172.21.15.16:0/1763067802 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5617845abfd0 0x5617845ae3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:58.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.023+0000 7fa92e2c8640 1 --2- 172.21.15.16:0/1763067802 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5617845abfd0 0x5617845ae3c0 crc :-1 s=READY pgs=328 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fbc2eae2848da15a server_cookie=ce686eda1aecfd3a in_seq=0 out_seq=0 2024-09-12T02:12:58.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.023+0000 7fa911ffb640 10 client.18987 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:12:58.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.023+0000 7fa911ffb640 1 -- 172.21.15.16:0/1763067802 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561784504860 con 0x5617845abfd0 2024-09-12T02:12:58.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.023+0000 7fa911ffb640 10 client.18987 handle_command_reply: tid=0 2024-09-12T02:12:58.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.024+0000 7fa932905740 1 client.18987 shutdown 2024-09-12T02:12:58.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.024+0000 7fa932905740 1 -- 172.21.15.16:0/1763067802 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5617845915d0 msgr2=0x56178459bd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:58.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.024+0000 7fa932905740 1 --2- 172.21.15.16:0/1763067802 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5617845915d0 0x56178459bd60 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7fa918005fd0 tx=0x7fa9180046c0 comp rx=0 tx=0).stop 2024-09-12T02:12:58.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.024+0000 7fa932905740 1 -- 172.21.15.16:0/1763067802 shutdown_connections 2024-09-12T02:12:58.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.024+0000 7fa932905740 1 --2- 172.21.15.16:0/1763067802 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5617845abfd0 0x5617845ae3c0 crc :-1 s=READY pgs=328 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:58.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.024+0000 7fa932905740 1 --2- 172.21.15.16:0/1763067802 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56178459c370 0x561784599600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:58.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.024+0000 7fa932905740 1 --2- 172.21.15.16:0/1763067802 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561784592b40 0x561784599010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:58.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.024+0000 7fa911ffb640 0 client.18987 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:12:58.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.024+0000 7fa932905740 1 --2- 172.21.15.16:0/1763067802 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5617845915d0 0x56178459bd60 unknown :-1 s=CLOSED pgs=1552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:58.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.024+0000 7fa932905740 1 -- 172.21.15.16:0/1763067802 >> 172.21.15.16:0/1763067802 conn(0x56178455a310 msgr2=0x561784593960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:58.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.024+0000 7fa932905740 1 -- 172.21.15.16:0/1763067802 shutdown_connections 2024-09-12T02:12:58.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.024+0000 7fa932905740 1 -- 172.21.15.16:0/1763067802 wait complete. 2024-09-12T02:12:58.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.024+0000 7fa932905740 20 client.18987 trim_cache size 0 max 16384 2024-09-12T02:12:58.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.037+0000 7fa932905740 1 Processor -- start 2024-09-12T02:12:58.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.037+0000 7fa932905740 1 -- start start 2024-09-12T02:12:58.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.037+0000 7fa932905740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5617845915d0 0x561784594120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:58.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.037+0000 7fa932905740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561784592b40 0x561784594660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:58.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.037+0000 7fa932905740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56178459c370 0x5617845b3c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:58.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.037+0000 7fa932905740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa9281afc00 con 0x5617845915d0 2024-09-12T02:12:58.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.037+0000 7fa932905740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa9281ad4d0 con 0x561784592b40 2024-09-12T02:12:58.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.037+0000 7fa932905740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa9281ab360 con 0x56178459c370 2024-09-12T02:12:58.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.037+0000 7fa92dac7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5617845915d0 0x561784594120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:58.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.038+0000 7fa92e2c8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561784592b40 0x561784594660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:58.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.038+0000 7fa92e2c8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561784592b40 0x561784594660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.16:51134/0 (socket says 172.21.15.16:51134) 2024-09-12T02:12:58.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.038+0000 7fa92e2c8640 1 -- 172.21.15.16:0/537680073 learned_addr learned my addr 172.21.15.16:0/537680073 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:12:58.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.038+0000 7fa92e2c8640 1 -- 172.21.15.16:0/537680073 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56178459c370 msgr2=0x5617845b3c40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T02:12:58.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.038+0000 7fa92e2c8640 1 --2- 172.21.15.16:0/537680073 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56178459c370 0x5617845b3c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:58.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.038+0000 7fa92e2c8640 1 -- 172.21.15.16:0/537680073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5617845915d0 msgr2=0x561784594120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:58.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.038+0000 7fa92e2c8640 1 --2- 172.21.15.16:0/537680073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5617845915d0 0x561784594120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:58.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.038+0000 7fa92e2c8640 1 -- 172.21.15.16:0/537680073 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561784599970 con 0x561784592b40 2024-09-12T02:12:58.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.039+0000 7fa92e2c8640 1 --2- 172.21.15.16:0/537680073 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561784592b40 0x561784594660 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa924000c00 tx=0x7fa924032f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e1341726c11a4e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:58.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.039+0000 7fa92dac7640 1 --2- 172.21.15.16:0/537680073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5617845915d0 0x561784594120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:12:58.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.041+0000 7fa913fff640 1 -- 172.21.15.16:0/537680073 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa924053070 con 0x561784592b40 2024-09-12T02:12:58.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.041+0000 7fa913fff640 1 -- 172.21.15.16:0/537680073 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa924031790 con 0x561784592b40 2024-09-12T02:12:58.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.041+0000 7fa913fff640 1 -- 172.21.15.16:0/537680073 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa92403f040 con 0x561784592b40 2024-09-12T02:12:58.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.042+0000 7fa932905740 1 -- 172.21.15.16:0/537680073 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561784592b40 msgr2=0x561784594660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:58.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.042+0000 7fa932905740 1 --2- 172.21.15.16:0/537680073 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561784592b40 0x561784594660 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa924000c00 tx=0x7fa924032f90 comp rx=0 tx=0).stop 2024-09-12T02:12:58.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.042+0000 7fa932905740 1 -- 172.21.15.16:0/537680073 shutdown_connections 2024-09-12T02:12:58.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.042+0000 7fa932905740 1 --2- 172.21.15.16:0/537680073 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56178459c370 0x5617845b3c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:58.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.042+0000 7fa932905740 1 --2- 172.21.15.16:0/537680073 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561784592b40 0x561784594660 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:58.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.042+0000 7fa932905740 1 --2- 172.21.15.16:0/537680073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5617845915d0 0x561784594120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:58.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.042+0000 7fa932905740 1 -- 172.21.15.16:0/537680073 >> 172.21.15.16:0/537680073 conn(0x56178455a310 msgr2=0x56178458cd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:58.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.042+0000 7fa932905740 1 -- 172.21.15.16:0/537680073 shutdown_connections 2024-09-12T02:12:58.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.042+0000 7fa932905740 1 -- 172.21.15.16:0/537680073 wait complete. 2024-09-12T02:12:58.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.042+0000 7fa932905740 1 -- start start 2024-09-12T02:12:58.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.043+0000 7fa932905740 1 Processor -- start 2024-09-12T02:12:58.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.043+0000 7fa932905740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5617845915d0 0x5617845960e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:58.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.043+0000 7fa932905740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561784592b40 0x561784596620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:58.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.043+0000 7fa932905740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56178459c370 0x561784596b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:58.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.043+0000 7fa932905740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa9281ab9b0 con 0x561784592b40 2024-09-12T02:12:58.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.043+0000 7fa932905740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa9281ace00 con 0x56178459c370 2024-09-12T02:12:58.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.043+0000 7fa932905740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa9281ac060 con 0x5617845915d0 2024-09-12T02:12:58.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.043+0000 7fa932905740 10 client.0 _ll_register_callbacks cb 0x7fa91800bd20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:12:58.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.043+0000 7fa92e2c8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561784592b40 0x561784596620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:58.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.043+0000 7fa92e2c8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561784592b40 0x561784596620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51882/0 (socket says 172.21.15.16:51882) 2024-09-12T02:12:58.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.043+0000 7fa92e2c8640 1 -- 172.21.15.16:0/88924442 learned_addr learned my addr 172.21.15.16:0/88924442 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:12:58.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.043+0000 7fa92dac7640 1 --2- 172.21.15.16:0/88924442 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5617845915d0 0x5617845960e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:58.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.043+0000 7fa92dac7640 1 --2- 172.21.15.16:0/88924442 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56178459c370 0x561784596b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:58.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.043+0000 7fa92e2c8640 1 -- 172.21.15.16:0/88924442 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5617845915d0 msgr2=0x5617845960e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:58.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.043+0000 7fa92e2c8640 1 --2- 172.21.15.16:0/88924442 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5617845915d0 0x5617845960e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:58.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.043+0000 7fa92e2c8640 1 -- 172.21.15.16:0/88924442 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56178459c370 msgr2=0x561784596b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:58.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.043+0000 7fa92e2c8640 1 --2- 172.21.15.16:0/88924442 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56178459c370 0x561784596b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:58.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.043+0000 7fa92e2c8640 1 -- 172.21.15.16:0/88924442 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56178461a0d0 con 0x561784592b40 2024-09-12T02:12:58.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.044+0000 7fa92e2c8640 1 --2- 172.21.15.16:0/88924442 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561784592b40 0x561784596620 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7fa924009900 tx=0x7fa924005770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1eb3b9fad85947a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:58.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.044+0000 7fa911ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:12:58.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.044+0000 7fa911ffb640 1 -- 172.21.15.16:0/88924442 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa92403f070 con 0x561784592b40 2024-09-12T02:12:58.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.044+0000 7fa911ffb640 1 -- 172.21.15.16:0/88924442 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fa924004350 con 0x561784592b40 2024-09-12T02:12:58.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.044+0000 7fa932905740 1 -- 172.21.15.16:0/88924442 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa924031790 con 0x561784592b40 2024-09-12T02:12:58.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.044+0000 7fa911ffb640 1 -- 172.21.15.16:0/88924442 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa924053040 con 0x561784592b40 2024-09-12T02:12:58.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.044+0000 7fa932905740 10 client.18990 fetch_fsmap learned FSMap version 25 2024-09-12T02:12:58.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.044+0000 7fa932905740 1 -- 172.21.15.16:0/88924442 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56178461b300 con 0x561784592b40 2024-09-12T02:12:58.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.044+0000 7fa911ffb640 1 -- 172.21.15.16:0/88924442 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa924031790 con 0x561784592b40 2024-09-12T02:12:58.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.044+0000 7fa911ffb640 1 -- 172.21.15.16:0/88924442 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fa924003680 con 0x561784592b40 2024-09-12T02:12:58.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.044+0000 7fa932905740 10 client.18990 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:12:58.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.044+0000 7fa932905740 10 client.18990 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:12:58.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.044+0000 7fa932905740 20 client.18990 populate_metadata read hostname 'smithi016' 2024-09-12T02:12:58.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.044+0000 7fa932905740 1 --2- 172.21.15.16:0/88924442 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x56178461b5d0 0x56178461d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:58.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.044+0000 7fa932905740 4 client.18990 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T02:12:58.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.044+0000 7fa932905740 1 -- 172.21.15.16:0/88924442 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5617844fed70 con 0x56178461b5d0 2024-09-12T02:12:58.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.045+0000 7fa92dac7640 1 --2- 172.21.15.16:0/88924442 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x56178461b5d0 0x56178461d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:58.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.045+0000 7fa92dac7640 1 --2- 172.21.15.16:0/88924442 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x56178461b5d0 0x56178461d9c0 crc :-1 s=READY pgs=329 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=96d9163d31244c8e server_cookie=de6352311bc88b6a in_seq=0 out_seq=0 2024-09-12T02:12:58.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.045+0000 7fa911ffb640 10 client.18990 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:12:58.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.045+0000 7fa911ffb640 1 -- 172.21.15.16:0/88924442 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5617844fed70 con 0x56178461b5d0 2024-09-12T02:12:58.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.045+0000 7fa911ffb640 10 client.18990 handle_command_reply: tid=0 2024-09-12T02:12:58.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.045+0000 7fa932905740 1 client.18990 shutdown 2024-09-12T02:12:58.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.045+0000 7fa932905740 1 -- 172.21.15.16:0/88924442 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561784592b40 msgr2=0x561784596620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:58.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.045+0000 7fa932905740 1 --2- 172.21.15.16:0/88924442 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561784592b40 0x561784596620 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7fa924009900 tx=0x7fa924005770 comp rx=0 tx=0).stop 2024-09-12T02:12:58.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.046+0000 7fa932905740 1 -- 172.21.15.16:0/88924442 shutdown_connections 2024-09-12T02:12:58.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.046+0000 7fa932905740 1 --2- 172.21.15.16:0/88924442 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x56178461b5d0 0x56178461d9c0 crc :-1 s=READY pgs=329 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:58.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.046+0000 7fa932905740 1 --2- 172.21.15.16:0/88924442 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56178459c370 0x561784596b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:58.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.046+0000 7fa932905740 1 --2- 172.21.15.16:0/88924442 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561784592b40 0x561784596620 unknown :-1 s=CLOSED pgs=1553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:58.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.046+0000 7fa932905740 1 --2- 172.21.15.16:0/88924442 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5617845915d0 0x5617845960e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:58.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.046+0000 7fa932905740 1 -- 172.21.15.16:0/88924442 >> 172.21.15.16:0/88924442 conn(0x56178455a310 msgr2=0x56178459a870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:58.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.046+0000 7fa911ffb640 0 client.18990 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:12:58.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.046+0000 7fa932905740 1 -- 172.21.15.16:0/88924442 shutdown_connections 2024-09-12T02:12:58.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.046+0000 7fa932905740 1 -- 172.21.15.16:0/88924442 wait complete. 2024-09-12T02:12:58.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.046+0000 7fa932905740 20 client.18990 trim_cache size 0 max 16384 2024-09-12T02:12:58.059 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T02:12:58.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.046+0000 7fa93052c640 1 -- 172.21.15.16:0/2679292880 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fa9140a2b80 msgr2=0x7fa9140a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:58.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.046+0000 7fa93052c640 1 --2- 172.21.15.16:0/2679292880 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fa9140a2b80 0x7fa9140a5040 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fa920061e50 tx=0x7fa92006bdd0 comp rx=0 tx=0).stop 2024-09-12T02:12:58.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.046+0000 7fa93052c640 1 -- 172.21.15.16:0/2679292880 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa928106f10 msgr2=0x7fa92819cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:58.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.046+0000 7fa93052c640 1 --2- 172.21.15.16:0/2679292880 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa928106f10 0x7fa92819cca0 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7fa92000cce0 tx=0x7fa92000a620 comp rx=0 tx=0).stop 2024-09-12T02:12:58.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.046+0000 7fa93052c640 1 -- 172.21.15.16:0/2679292880 shutdown_connections 2024-09-12T02:12:58.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.046+0000 7fa93052c640 1 --2- 172.21.15.16:0/2679292880 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fa9140a2b80 0x7fa9140a5040 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:58.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.046+0000 7fa93052c640 1 --2- 172.21.15.16:0/2679292880 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa928106f10 0x7fa92819cca0 unknown :-1 s=CLOSED pgs=1550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:58.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.046+0000 7fa93052c640 1 --2- 172.21.15.16:0/2679292880 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa928102790 0x7fa928198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:58.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.046+0000 7fa93052c640 1 --2- 172.21.15.16:0/2679292880 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa928101de0 0x7fa9281983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:58.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.046+0000 7fa93052c640 1 -- 172.21.15.16:0/2679292880 >> 172.21.15.16:0/2679292880 conn(0x7fa9280fd8c0 msgr2=0x7fa9280febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:58.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.046+0000 7fa93052c640 1 -- 172.21.15.16:0/2679292880 shutdown_connections 2024-09-12T02:12:58.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:12:58.047+0000 7fa93052c640 1 -- 172.21.15.16:0/2679292880 wait complete. 2024-09-12T02:12:58.183 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:12:58 smithi016 ceph-mon[30947]: pgmap v666: 97 pgs: 97 active+clean; 1.7 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 21 KiB/s wr, 18 op/s 2024-09-12T02:12:58.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:12:58 smithi184 ceph-mon[35593]: pgmap v666: 97 pgs: 97 active+clean; 1.7 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 21 KiB/s wr, 18 op/s 2024-09-12T02:12:58.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:12:58 smithi033 ceph-mon[35332]: pgmap v666: 97 pgs: 97 active+clean; 1.7 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 21 KiB/s wr, 18 op/s 2024-09-12T02:12:59.862 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-12T02:12:59.863 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-12T02:13:00.135 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:13:00.183 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:13:00 smithi016 ceph-mon[30947]: pgmap v667: 97 pgs: 97 active+clean; 1.7 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-09-12T02:13:00.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:13:00 smithi184 ceph-mon[35593]: pgmap v667: 97 pgs: 97 active+clean; 1.7 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-09-12T02:13:00.418 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:13:00 smithi033 ceph-mon[35332]: pgmap v667: 97 pgs: 97 active+clean; 1.7 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-09-12T02:13:00.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.657+0000 7f236727c640 1 -- 172.21.15.16:0/548029729 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2360100e20 msgr2=0x7f2360101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:00.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.657+0000 7f236727c640 1 --2- 172.21.15.16:0/548029729 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2360100e20 0x7f2360101200 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7f235c009ca0 tx=0x7f235c02a950 comp rx=0 tx=0).stop 2024-09-12T02:13:00.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.657+0000 7f236727c640 1 -- 172.21.15.16:0/548029729 shutdown_connections 2024-09-12T02:13:00.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.657+0000 7f236727c640 1 --2- 172.21.15.16:0/548029729 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2360102ef0 0x7f236010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:00.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.657+0000 7f236727c640 1 --2- 172.21.15.16:0/548029729 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2360102570 0x7f23601029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:00.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.657+0000 7f236727c640 1 --2- 172.21.15.16:0/548029729 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2360100e20 0x7f2360101200 unknown :-1 s=CLOSED pgs=1554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:00.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.657+0000 7f236727c640 1 -- 172.21.15.16:0/548029729 >> 172.21.15.16:0/548029729 conn(0x7f23600fc7d0 msgr2=0x7f23600febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:00.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.657+0000 7f236727c640 1 -- 172.21.15.16:0/548029729 shutdown_connections 2024-09-12T02:13:00.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.657+0000 7f236727c640 1 -- 172.21.15.16:0/548029729 wait complete. 2024-09-12T02:13:00.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.657+0000 7f236727c640 1 Processor -- start 2024-09-12T02:13:00.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.657+0000 7f236727c640 1 -- start start 2024-09-12T02:13:00.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.658+0000 7f236727c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2360100e20 0x7f2360103750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:00.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.658+0000 7f236727c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2360102570 0x7f2360103c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:00.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.658+0000 7f236727c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2360102ef0 0x7f23601041d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:00.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.658+0000 7f236727c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2360110fd0 con 0x7f2360102570 2024-09-12T02:13:00.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.658+0000 7f236727c640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2360110e50 con 0x7f2360102ef0 2024-09-12T02:13:00.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.658+0000 7f236727c640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f2360111150 con 0x7f2360100e20 2024-09-12T02:13:00.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.658+0000 7f2364817640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2360102570 0x7f2360103c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:00.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.658+0000 7f2364817640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2360102570 0x7f2360103c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51894/0 (socket says 172.21.15.16:51894) 2024-09-12T02:13:00.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.658+0000 7f2364817640 1 -- 172.21.15.16:0/853514205 learned_addr learned my addr 172.21.15.16:0/853514205 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:13:00.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.658+0000 7f2365018640 1 --2- 172.21.15.16:0/853514205 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2360100e20 0x7f2360103750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:00.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.658+0000 7f2365819640 1 --2- 172.21.15.16:0/853514205 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2360102ef0 0x7f23601041d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:00.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.658+0000 7f2364817640 1 -- 172.21.15.16:0/853514205 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2360100e20 msgr2=0x7f2360103750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:00.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.658+0000 7f2364817640 1 --2- 172.21.15.16:0/853514205 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2360100e20 0x7f2360103750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:00.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.658+0000 7f2364817640 1 -- 172.21.15.16:0/853514205 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2360102ef0 msgr2=0x7f23601041d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:00.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.658+0000 7f2364817640 1 --2- 172.21.15.16:0/853514205 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2360102ef0 0x7f23601041d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:00.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.658+0000 7f2364817640 1 -- 172.21.15.16:0/853514205 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23601a6980 con 0x7f2360102570 2024-09-12T02:13:00.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.658+0000 7f2364817640 1 --2- 172.21.15.16:0/853514205 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2360102570 0x7f2360103c90 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f235000c890 tx=0x7f235000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:00.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.658+0000 7f23567fc640 1 -- 172.21.15.16:0/853514205 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2350004220 con 0x7f2360102570 2024-09-12T02:13:00.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.659+0000 7f23567fc640 1 -- 172.21.15.16:0/853514205 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f23500043c0 con 0x7f2360102570 2024-09-12T02:13:00.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.659+0000 7f236727c640 1 -- 172.21.15.16:0/853514205 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23601a6c70 con 0x7f2360102570 2024-09-12T02:13:00.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.659+0000 7f23567fc640 1 -- 172.21.15.16:0/853514205 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23500049e0 con 0x7f2360102570 2024-09-12T02:13:00.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.659+0000 7f236727c640 1 -- 172.21.15.16:0/853514205 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23601a71b0 con 0x7f2360102570 2024-09-12T02:13:00.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.661+0000 7f23567fc640 1 -- 172.21.15.16:0/853514205 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f23500079b0 con 0x7f2360102570 2024-09-12T02:13:00.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.661+0000 7f2369655740 1 Processor -- start 2024-09-12T02:13:00.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.662+0000 7f2369655740 1 -- start start 2024-09-12T02:13:00.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.662+0000 7f2369655740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b4ff62b60 0x560b4ff62f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:00.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.662+0000 7f2369655740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b4ff63480 0x560b4ff6d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:00.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.662+0000 7f2369655740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560b4ff6dd40 0x560b4ff70130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:00.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.662+0000 7f2369655740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x560b4fec8280 con 0x560b4ff62b60 2024-09-12T02:13:00.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.662+0000 7f2369655740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x560b4fe6fb10 con 0x560b4ff63480 2024-09-12T02:13:00.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.662+0000 7f2369655740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x560b4fec4240 con 0x560b4ff6dd40 2024-09-12T02:13:00.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.662+0000 7f2365819640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b4ff62b60 0x560b4ff62f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:00.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.662+0000 7f2365819640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b4ff62b60 0x560b4ff62f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51902/0 (socket says 172.21.15.16:51902) 2024-09-12T02:13:00.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.662+0000 7f2365819640 1 -- 172.21.15.16:0/2076772705 learned_addr learned my addr 172.21.15.16:0/2076772705 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:13:00.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.662+0000 7f23567fc640 1 --2- 172.21.15.16:0/853514205 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f234c0a2b80 0x7f234c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:00.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.662+0000 7f2365018640 1 --2- 172.21.15.16:0/2076772705 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b4ff63480 0x560b4ff6d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:00.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.662+0000 7f2364817640 1 --2- 172.21.15.16:0/2076772705 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560b4ff6dd40 0x560b4ff70130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:00.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.662+0000 7f2365819640 1 --2- 172.21.15.16:0/853514205 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f234c0a2b80 0x7f234c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:00.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.662+0000 7f2365819640 1 -- 172.21.15.16:0/2076772705 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560b4ff6dd40 msgr2=0x560b4ff70130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:00.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.662+0000 7f2365819640 1 --2- 172.21.15.16:0/2076772705 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560b4ff6dd40 0x560b4ff70130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:00.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.662+0000 7f2365819640 1 -- 172.21.15.16:0/2076772705 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b4ff63480 msgr2=0x560b4ff6d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:00.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.662+0000 7f2365819640 1 --2- 172.21.15.16:0/2076772705 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b4ff63480 0x560b4ff6d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:00.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.662+0000 7f2365819640 1 -- 172.21.15.16:0/2076772705 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b4fe7c1f0 con 0x560b4ff62b60 2024-09-12T02:13:00.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.662+0000 7f2364817640 1 --2- 172.21.15.16:0/2076772705 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560b4ff6dd40 0x560b4ff70130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:13:00.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.662+0000 7f23567fc640 1 -- 172.21.15.16:0/853514205 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f23500ceff0 con 0x7f2360102570 2024-09-12T02:13:00.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.663+0000 7f2365819640 1 --2- 172.21.15.16:0/2076772705 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b4ff62b60 0x560b4ff62f40 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f2358013820 tx=0x7f2358013ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7472c060619ebd16 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:00.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.663+0000 7f2365819640 1 --2- 172.21.15.16:0/853514205 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f234c0a2b80 0x7f234c0a5040 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f2358013fd0 tx=0x7f235801e000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:00.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.663+0000 7f234b7fe640 1 -- 172.21.15.16:0/2076772705 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f235801d9d0 con 0x560b4ff62b60 2024-09-12T02:13:00.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.663+0000 7f234b7fe640 1 -- 172.21.15.16:0/2076772705 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2358021070 con 0x560b4ff62b60 2024-09-12T02:13:00.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.663+0000 7f234b7fe640 1 -- 172.21.15.16:0/2076772705 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2358025d10 con 0x560b4ff62b60 2024-09-12T02:13:00.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.664+0000 7f2369655740 1 -- 172.21.15.16:0/2076772705 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b4ff62b60 msgr2=0x560b4ff62f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:00.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.664+0000 7f2369655740 1 --2- 172.21.15.16:0/2076772705 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b4ff62b60 0x560b4ff62f40 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f2358013820 tx=0x7f2358013ce0 comp rx=0 tx=0).stop 2024-09-12T02:13:00.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.664+0000 7f2369655740 1 -- 172.21.15.16:0/2076772705 shutdown_connections 2024-09-12T02:13:00.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.664+0000 7f2369655740 1 --2- 172.21.15.16:0/2076772705 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560b4ff6dd40 0x560b4ff70130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:00.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.664+0000 7f2369655740 1 --2- 172.21.15.16:0/2076772705 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b4ff63480 0x560b4ff6d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:00.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.664+0000 7f2369655740 1 --2- 172.21.15.16:0/2076772705 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b4ff62b60 0x560b4ff62f40 unknown :-1 s=CLOSED pgs=1556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:00.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.664+0000 7f2369655740 1 -- 172.21.15.16:0/2076772705 >> 172.21.15.16:0/2076772705 conn(0x560b4ff5f0f0 msgr2=0x560b4ff5f4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:00.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.664+0000 7f2369655740 1 -- 172.21.15.16:0/2076772705 shutdown_connections 2024-09-12T02:13:00.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.664+0000 7f2369655740 1 -- 172.21.15.16:0/2076772705 wait complete. 2024-09-12T02:13:00.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.664+0000 7f2369655740 1 -- start start 2024-09-12T02:13:00.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.664+0000 7f2369655740 1 Processor -- start 2024-09-12T02:13:00.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.665+0000 7f2369655740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560b4ff62b60 0x560b4ff6abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:00.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.665+0000 7f2369655740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b4ff63480 0x560b4ff6b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:00.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.665+0000 7f2369655740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b4ff6dd40 0x560b4ff6b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:00.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.665+0000 7f2369655740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x560b4ff70910 con 0x560b4ff6dd40 2024-09-12T02:13:00.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.665+0000 7f2369655740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x560b4fe82f10 con 0x560b4ff63480 2024-09-12T02:13:00.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.665+0000 7f2369655740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x560b4ff70a90 con 0x560b4ff62b60 2024-09-12T02:13:00.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.665+0000 7f2369655740 10 client.0 _ll_register_callbacks cb 0x560b4fe4ba40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:13:00.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.665+0000 7f2365819640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b4ff6dd40 0x560b4ff6b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:00.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.665+0000 7f2365819640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b4ff6dd40 0x560b4ff6b660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51906/0 (socket says 172.21.15.16:51906) 2024-09-12T02:13:00.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.665+0000 7f2365819640 1 -- 172.21.15.16:0/3672870067 learned_addr learned my addr 172.21.15.16:0/3672870067 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:13:00.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.665+0000 7f2365018640 1 --2- 172.21.15.16:0/3672870067 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560b4ff62b60 0x560b4ff6abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:00.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.665+0000 7f2364817640 1 --2- 172.21.15.16:0/3672870067 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b4ff63480 0x560b4ff6b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:00.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.665+0000 7f2365819640 1 -- 172.21.15.16:0/3672870067 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560b4ff62b60 msgr2=0x560b4ff6abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:00.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.665+0000 7f2365819640 1 --2- 172.21.15.16:0/3672870067 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560b4ff62b60 0x560b4ff6abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:00.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.665+0000 7f2365819640 1 -- 172.21.15.16:0/3672870067 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b4ff63480 msgr2=0x560b4ff6b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:00.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.665+0000 7f2365819640 1 --2- 172.21.15.16:0/3672870067 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b4ff63480 0x560b4ff6b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:00.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.665+0000 7f2365819640 1 -- 172.21.15.16:0/3672870067 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b4fc396a0 con 0x560b4ff6dd40 2024-09-12T02:13:00.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.665+0000 7f2365819640 1 --2- 172.21.15.16:0/3672870067 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b4ff6dd40 0x560b4ff6b660 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7f23580256c0 tx=0x7f23580256f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=214bf1d6fcb5929a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:00.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.665+0000 7f2348ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:13:00.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.666+0000 7f2348ff9640 1 -- 172.21.15.16:0/3672870067 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2358035060 con 0x560b4ff6dd40 2024-09-12T02:13:00.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.666+0000 7f2348ff9640 1 -- 172.21.15.16:0/3672870067 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f2358013250 con 0x560b4ff6dd40 2024-09-12T02:13:00.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.666+0000 7f2369655740 1 -- 172.21.15.16:0/3672870067 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2358021070 con 0x560b4ff6dd40 2024-09-12T02:13:00.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.666+0000 7f2348ff9640 1 -- 172.21.15.16:0/3672870067 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2358029040 con 0x560b4ff6dd40 2024-09-12T02:13:00.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.666+0000 7f2369655740 10 client.19002 fetch_fsmap learned FSMap version 25 2024-09-12T02:13:00.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.666+0000 7f2369655740 1 -- 172.21.15.16:0/3672870067 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x560b4ff7c4e0 con 0x560b4ff6dd40 2024-09-12T02:13:00.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.666+0000 7f2348ff9640 1 -- 172.21.15.16:0/3672870067 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2358021070 con 0x560b4ff6dd40 2024-09-12T02:13:00.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.666+0000 7f2348ff9640 1 -- 172.21.15.16:0/3672870067 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f23580051a0 con 0x560b4ff6dd40 2024-09-12T02:13:00.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.666+0000 7f2369655740 10 client.19002 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:13:00.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.666+0000 7f2369655740 10 client.19002 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:13:00.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.666+0000 7f2369655740 20 client.19002 populate_metadata read hostname 'smithi016' 2024-09-12T02:13:00.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.666+0000 7f2369655740 1 --2- 172.21.15.16:0/3672870067 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x560b4ff7d890 0x560b4ff7fc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:00.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.666+0000 7f2369655740 4 client.19002 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:13:00.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.666+0000 7f2369655740 1 -- 172.21.15.16:0/3672870067 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560b4fed57f0 con 0x560b4ff7d890 2024-09-12T02:13:00.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.666+0000 7f2365018640 1 --2- 172.21.15.16:0/3672870067 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x560b4ff7d890 0x560b4ff7fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:00.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.667+0000 7f2348ff9640 10 client.19002 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:13:00.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.667+0000 7f2365018640 1 --2- 172.21.15.16:0/3672870067 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x560b4ff7d890 0x560b4ff7fc80 crc :-1 s=READY pgs=330 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae39de9c79a72ea4 server_cookie=9a6c6a259daa8aee in_seq=0 out_seq=0 2024-09-12T02:13:00.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.668+0000 7f2348ff9640 1 -- 172.21.15.16:0/3672870067 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560b4fed57f0 con 0x560b4ff7d890 2024-09-12T02:13:00.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.668+0000 7f2348ff9640 10 client.19002 handle_command_reply: tid=0 2024-09-12T02:13:00.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.668+0000 7f2369655740 1 client.19002 shutdown 2024-09-12T02:13:00.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.668+0000 7f2369655740 1 -- 172.21.15.16:0/3672870067 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b4ff6dd40 msgr2=0x560b4ff6b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:00.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.668+0000 7f2369655740 1 --2- 172.21.15.16:0/3672870067 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b4ff6dd40 0x560b4ff6b660 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7f23580256c0 tx=0x7f23580256f0 comp rx=0 tx=0).stop 2024-09-12T02:13:00.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.668+0000 7f2369655740 1 -- 172.21.15.16:0/3672870067 shutdown_connections 2024-09-12T02:13:00.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.668+0000 7f2369655740 1 --2- 172.21.15.16:0/3672870067 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x560b4ff7d890 0x560b4ff7fc80 crc :-1 s=READY pgs=330 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:00.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.668+0000 7f2369655740 1 --2- 172.21.15.16:0/3672870067 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b4ff6dd40 0x560b4ff6b660 unknown :-1 s=CLOSED pgs=1557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:00.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.668+0000 7f2369655740 1 --2- 172.21.15.16:0/3672870067 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b4ff63480 0x560b4ff6b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:00.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.668+0000 7f2369655740 1 --2- 172.21.15.16:0/3672870067 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560b4ff62b60 0x560b4ff6abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:00.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.668+0000 7f2348ff9640 0 client.19002 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:13:00.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.668+0000 7f2369655740 1 -- 172.21.15.16:0/3672870067 >> 172.21.15.16:0/3672870067 conn(0x560b4ff5f0f0 msgr2=0x560b4ff64e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:00.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.668+0000 7f2369655740 1 -- 172.21.15.16:0/3672870067 shutdown_connections 2024-09-12T02:13:00.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.668+0000 7f2369655740 1 -- 172.21.15.16:0/3672870067 wait complete. 2024-09-12T02:13:00.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.668+0000 7f2369655740 20 client.19002 trim_cache size 0 max 16384 2024-09-12T02:13:00.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.685+0000 7f2369655740 1 Processor -- start 2024-09-12T02:13:00.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.685+0000 7f2369655740 1 -- start start 2024-09-12T02:13:00.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.686+0000 7f2369655740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b4ff62b60 0x560b4ff655c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:00.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.686+0000 7f2369655740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b4ff63480 0x560b4ff69ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:00.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.686+0000 7f2369655740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560b4ff6dd40 0x560b4ff85250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:00.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.686+0000 7f2369655740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f23601b1ef0 con 0x560b4ff62b60 2024-09-12T02:13:00.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.686+0000 7f2369655740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f23601af7c0 con 0x560b4ff63480 2024-09-12T02:13:00.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.686+0000 7f2369655740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f23601ad610 con 0x560b4ff6dd40 2024-09-12T02:13:00.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.686+0000 7f2365018640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b4ff62b60 0x560b4ff655c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:00.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.686+0000 7f2365018640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b4ff62b60 0x560b4ff655c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51918/0 (socket says 172.21.15.16:51918) 2024-09-12T02:13:00.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.686+0000 7f2365018640 1 -- 172.21.15.16:0/2194617581 learned_addr learned my addr 172.21.15.16:0/2194617581 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:13:00.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.686+0000 7f2364817640 1 --2- 172.21.15.16:0/2194617581 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b4ff63480 0x560b4ff69ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:00.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.686+0000 7f2365819640 1 --2- 172.21.15.16:0/2194617581 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560b4ff6dd40 0x560b4ff85250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:00.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.686+0000 7f2365018640 1 -- 172.21.15.16:0/2194617581 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560b4ff6dd40 msgr2=0x560b4ff85250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:00.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.686+0000 7f2365018640 1 --2- 172.21.15.16:0/2194617581 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560b4ff6dd40 0x560b4ff85250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:00.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.686+0000 7f2365018640 1 -- 172.21.15.16:0/2194617581 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b4ff63480 msgr2=0x560b4ff69ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:00.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.686+0000 7f2365018640 1 --2- 172.21.15.16:0/2194617581 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b4ff63480 0x560b4ff69ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:00.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.686+0000 7f2365018640 1 -- 172.21.15.16:0/2194617581 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b4ff601a0 con 0x560b4ff62b60 2024-09-12T02:13:00.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.686+0000 7f2365819640 1 --2- 172.21.15.16:0/2194617581 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560b4ff6dd40 0x560b4ff85250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:13:00.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.686+0000 7f2365018640 1 --2- 172.21.15.16:0/2194617581 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b4ff62b60 0x560b4ff655c0 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f235c02af10 tx=0x7f235c02f810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0b397923a7cecc4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:00.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.686+0000 7f234affd640 1 -- 172.21.15.16:0/2194617581 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f235c04a070 con 0x560b4ff62b60 2024-09-12T02:13:00.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.686+0000 7f234affd640 1 -- 172.21.15.16:0/2194617581 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f235c02fe10 con 0x560b4ff62b60 2024-09-12T02:13:00.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.686+0000 7f234affd640 1 -- 172.21.15.16:0/2194617581 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f235c034040 con 0x560b4ff62b60 2024-09-12T02:13:00.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.687+0000 7f2369655740 1 -- 172.21.15.16:0/2194617581 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b4ff62b60 msgr2=0x560b4ff655c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:00.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.687+0000 7f2369655740 1 --2- 172.21.15.16:0/2194617581 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b4ff62b60 0x560b4ff655c0 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f235c02af10 tx=0x7f235c02f810 comp rx=0 tx=0).stop 2024-09-12T02:13:00.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.687+0000 7f2369655740 1 -- 172.21.15.16:0/2194617581 shutdown_connections 2024-09-12T02:13:00.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.687+0000 7f2369655740 1 --2- 172.21.15.16:0/2194617581 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560b4ff6dd40 0x560b4ff85250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:00.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.687+0000 7f2369655740 1 --2- 172.21.15.16:0/2194617581 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b4ff63480 0x560b4ff69ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:00.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.687+0000 7f2369655740 1 --2- 172.21.15.16:0/2194617581 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b4ff62b60 0x560b4ff655c0 unknown :-1 s=CLOSED pgs=1558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:00.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.687+0000 7f2369655740 1 -- 172.21.15.16:0/2194617581 >> 172.21.15.16:0/2194617581 conn(0x560b4ff5f0f0 msgr2=0x560b4ff693a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:00.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.687+0000 7f2369655740 1 -- 172.21.15.16:0/2194617581 shutdown_connections 2024-09-12T02:13:00.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.687+0000 7f2369655740 1 -- 172.21.15.16:0/2194617581 wait complete. 2024-09-12T02:13:00.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.687+0000 7f2369655740 1 -- start start 2024-09-12T02:13:00.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.688+0000 7f2369655740 1 Processor -- start 2024-09-12T02:13:00.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.688+0000 7f2369655740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560b4ff62b60 0x560b4ff76320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:00.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.688+0000 7f2369655740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b4ff63480 0x560b4ff67870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:00.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.688+0000 7f2369655740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b4ff6dd40 0x560b4ff67db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:00.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.688+0000 7f2369655740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f23601adc80 con 0x560b4ff6dd40 2024-09-12T02:13:00.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.688+0000 7f2369655740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f23601af0f0 con 0x560b4ff63480 2024-09-12T02:13:00.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.688+0000 7f2369655740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f23601ae350 con 0x560b4ff62b60 2024-09-12T02:13:00.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.688+0000 7f2369655740 10 client.0 _ll_register_callbacks cb 0x7f235c005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:13:00.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.688+0000 7f2365819640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b4ff6dd40 0x560b4ff67db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:00.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.688+0000 7f2365819640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b4ff6dd40 0x560b4ff67db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51924/0 (socket says 172.21.15.16:51924) 2024-09-12T02:13:00.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.688+0000 7f2365819640 1 -- 172.21.15.16:0/2307967019 learned_addr learned my addr 172.21.15.16:0/2307967019 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:13:00.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.688+0000 7f2365018640 1 --2- 172.21.15.16:0/2307967019 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560b4ff62b60 0x560b4ff76320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:00.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.688+0000 7f2364817640 1 --2- 172.21.15.16:0/2307967019 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b4ff63480 0x560b4ff67870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:00.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.688+0000 7f2365819640 1 -- 172.21.15.16:0/2307967019 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560b4ff62b60 msgr2=0x560b4ff76320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:00.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.688+0000 7f2365819640 1 --2- 172.21.15.16:0/2307967019 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560b4ff62b60 0x560b4ff76320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:00.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.688+0000 7f2365819640 1 -- 172.21.15.16:0/2307967019 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b4ff63480 msgr2=0x560b4ff67870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:00.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.688+0000 7f2365819640 1 --2- 172.21.15.16:0/2307967019 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b4ff63480 0x560b4ff67870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:00.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.688+0000 7f2365819640 1 -- 172.21.15.16:0/2307967019 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b4ff684f0 con 0x560b4ff6dd40 2024-09-12T02:13:00.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.688+0000 7f2365018640 1 --2- 172.21.15.16:0/2307967019 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560b4ff62b60 0x560b4ff76320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:13:00.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.689+0000 7f2365819640 1 --2- 172.21.15.16:0/2307967019 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b4ff6dd40 0x560b4ff67db0 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f2358005570 tx=0x7f2358025760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31ad2743f725dbe9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:00.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.689+0000 7f2348ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:13:00.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.689+0000 7f2348ff9640 1 -- 172.21.15.16:0/2307967019 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2358003800 con 0x560b4ff6dd40 2024-09-12T02:13:00.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.689+0000 7f2348ff9640 1 -- 172.21.15.16:0/2307967019 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f2358005110 con 0x560b4ff6dd40 2024-09-12T02:13:00.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.689+0000 7f2369655740 1 -- 172.21.15.16:0/2307967019 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f235c02fe10 con 0x560b4ff6dd40 2024-09-12T02:13:00.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.689+0000 7f2348ff9640 1 -- 172.21.15.16:0/2307967019 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f235800a600 con 0x560b4ff6dd40 2024-09-12T02:13:00.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.689+0000 7f2369655740 10 client.19008 fetch_fsmap learned FSMap version 25 2024-09-12T02:13:00.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.689+0000 7f2369655740 1 -- 172.21.15.16:0/2307967019 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x560b4ffe8100 con 0x560b4ff6dd40 2024-09-12T02:13:00.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.689+0000 7f2348ff9640 1 -- 172.21.15.16:0/2307967019 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f235c02fe10 con 0x560b4ff6dd40 2024-09-12T02:13:00.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.689+0000 7f2348ff9640 1 -- 172.21.15.16:0/2307967019 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f2358045020 con 0x560b4ff6dd40 2024-09-12T02:13:00.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.689+0000 7f2369655740 10 client.19008 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:13:00.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.689+0000 7f2369655740 10 client.19008 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:13:00.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.689+0000 7f2369655740 20 client.19008 populate_metadata read hostname 'smithi016' 2024-09-12T02:13:00.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.689+0000 7f2369655740 1 --2- 172.21.15.16:0/2307967019 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x560b4ffe88d0 0x560b4ffeacf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:00.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.689+0000 7f2369655740 4 client.19008 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-12T02:13:00.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.689+0000 7f2369655740 1 -- 172.21.15.16:0/2307967019 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x560b4fecfd00 con 0x560b4ffe88d0 2024-09-12T02:13:00.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.690+0000 7f2365018640 1 --2- 172.21.15.16:0/2307967019 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x560b4ffe88d0 0x560b4ffeacf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:00.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.690+0000 7f2365018640 1 --2- 172.21.15.16:0/2307967019 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x560b4ffe88d0 0x560b4ffeacf0 crc :-1 s=READY pgs=331 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ecb81f40253a8d73 server_cookie=a593464fb98fb882 in_seq=0 out_seq=0 2024-09-12T02:13:00.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.690+0000 7f2348ff9640 10 client.19008 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:13:00.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.691+0000 7f2348ff9640 1 -- 172.21.15.16:0/2307967019 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x560b4fecfd00 con 0x560b4ffe88d0 2024-09-12T02:13:00.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.691+0000 7f2348ff9640 10 client.19008 handle_command_reply: tid=0 2024-09-12T02:13:00.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.691+0000 7f2369655740 1 client.19008 shutdown 2024-09-12T02:13:00.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.691+0000 7f2369655740 1 -- 172.21.15.16:0/2307967019 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b4ff6dd40 msgr2=0x560b4ff67db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:00.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.692+0000 7f2369655740 1 --2- 172.21.15.16:0/2307967019 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b4ff6dd40 0x560b4ff67db0 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f2358005570 tx=0x7f2358025760 comp rx=0 tx=0).stop 2024-09-12T02:13:00.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.692+0000 7f2369655740 1 -- 172.21.15.16:0/2307967019 shutdown_connections 2024-09-12T02:13:00.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.692+0000 7f2369655740 1 --2- 172.21.15.16:0/2307967019 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x560b4ffe88d0 0x560b4ffeacf0 crc :-1 s=READY pgs=331 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:00.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.692+0000 7f2369655740 1 --2- 172.21.15.16:0/2307967019 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b4ff6dd40 0x560b4ff67db0 unknown :-1 s=CLOSED pgs=1559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:00.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.692+0000 7f2369655740 1 --2- 172.21.15.16:0/2307967019 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b4ff63480 0x560b4ff67870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:00.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.692+0000 7f2369655740 1 --2- 172.21.15.16:0/2307967019 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560b4ff62b60 0x560b4ff76320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:00.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.692+0000 7f2369655740 1 -- 172.21.15.16:0/2307967019 >> 172.21.15.16:0/2307967019 conn(0x560b4ff5f0f0 msgr2=0x560b4ff69bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:00.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.692+0000 7f2348ff9640 0 client.19008 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:13:00.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.692+0000 7f2369655740 1 -- 172.21.15.16:0/2307967019 shutdown_connections 2024-09-12T02:13:00.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.692+0000 7f2369655740 1 -- 172.21.15.16:0/2307967019 wait complete. 2024-09-12T02:13:00.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.692+0000 7f2369655740 20 client.19008 trim_cache size 0 max 16384 2024-09-12T02:13:00.702 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T02:13:00.702 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-09-12T02:13:00.702 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "8751ab38-33b0-4e09-8bbc-b15fb7c3cefe", 2024-09-12T02:13:00.702 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-09-12T02:13:00.702 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T02:13:00.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.692+0000 7f236727c640 1 -- 172.21.15.16:0/853514205 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f234c0a2b80 msgr2=0x7f234c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:00.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.692+0000 7f236727c640 1 --2- 172.21.15.16:0/853514205 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f234c0a2b80 0x7f234c0a5040 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f2358013fd0 tx=0x7f235801e000 comp rx=0 tx=0).stop 2024-09-12T02:13:00.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.692+0000 7f236727c640 1 -- 172.21.15.16:0/853514205 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2360102570 msgr2=0x7f2360103c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:00.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.692+0000 7f236727c640 1 --2- 172.21.15.16:0/853514205 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2360102570 0x7f2360103c90 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f235000c890 tx=0x7f235000cd50 comp rx=0 tx=0).stop 2024-09-12T02:13:00.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.693+0000 7f236727c640 1 -- 172.21.15.16:0/853514205 shutdown_connections 2024-09-12T02:13:00.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.693+0000 7f236727c640 1 --2- 172.21.15.16:0/853514205 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f234c0a2b80 0x7f234c0a5040 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:00.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.693+0000 7f236727c640 1 --2- 172.21.15.16:0/853514205 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2360102ef0 0x7f23601041d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:00.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.693+0000 7f236727c640 1 --2- 172.21.15.16:0/853514205 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2360102570 0x7f2360103c90 unknown :-1 s=CLOSED pgs=1555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:00.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.693+0000 7f236727c640 1 --2- 172.21.15.16:0/853514205 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2360100e20 0x7f2360103750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:00.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.693+0000 7f236727c640 1 -- 172.21.15.16:0/853514205 >> 172.21.15.16:0/853514205 conn(0x7f23600fc7d0 msgr2=0x7f236006a110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:00.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.693+0000 7f236727c640 1 -- 172.21.15.16:0/853514205 shutdown_connections 2024-09-12T02:13:00.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:00.693+0000 7f236727c640 1 -- 172.21.15.16:0/853514205 wait complete. 2024-09-12T02:13:01.517 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T02:13:01.776 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:13:02.161 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:13:02 smithi016 ceph-mon[30947]: pgmap v668: 97 pgs: 97 active+clean; 1.7 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-09-12T02:13:02.161 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:13:02 smithi016 ceph-mon[30947]: scrub queued for path: / 2024-09-12T02:13:02.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:13:02 smithi016 ceph-mon[30947]: scrub summary: idle+waiting paths [/] 2024-09-12T02:13:02.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:13:02 smithi016 ceph-mon[30947]: scrub summary: active paths [/] 2024-09-12T02:13:02.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:13:02 smithi016 ceph-mon[30947]: scrub summary: idle+waiting paths [/] 2024-09-12T02:13:02.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:13:02 smithi016 ceph-mon[30947]: scrub summary: idle 2024-09-12T02:13:02.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.364+0000 7f85a766f640 1 -- 172.21.15.16:0/1278101484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85a0102d40 msgr2=0x7f85a0103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:02.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.364+0000 7f85a766f640 1 --2- 172.21.15.16:0/1278101484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85a0102d40 0x7f85a0103120 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f859c009900 tx=0x7f859c02a480 comp rx=0 tx=0).stop 2024-09-12T02:13:02.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.364+0000 7f85a766f640 1 -- 172.21.15.16:0/1278101484 shutdown_connections 2024-09-12T02:13:02.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.364+0000 7f85a766f640 1 --2- 172.21.15.16:0/1278101484 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85a010c170 0x7f85a010e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:02.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.364+0000 7f85a766f640 1 --2- 172.21.15.16:0/1278101484 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f85a0103660 0x7f85a010bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:02.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.364+0000 7f85a766f640 1 --2- 172.21.15.16:0/1278101484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85a0102d40 0x7f85a0103120 unknown :-1 s=CLOSED pgs=1560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:02.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.364+0000 7f85a766f640 1 -- 172.21.15.16:0/1278101484 >> 172.21.15.16:0/1278101484 conn(0x7f85a00fc7a0 msgr2=0x7f85a00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:02.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.364+0000 7f85a766f640 1 -- 172.21.15.16:0/1278101484 shutdown_connections 2024-09-12T02:13:02.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.364+0000 7f85a766f640 1 -- 172.21.15.16:0/1278101484 wait complete. 2024-09-12T02:13:02.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.365+0000 7f85a766f640 1 Processor -- start 2024-09-12T02:13:02.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.365+0000 7f85a766f640 1 -- start start 2024-09-12T02:13:02.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.365+0000 7f85a766f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85a0102d40 0x7f85a0198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:02.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.365+0000 7f85a766f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85a0103660 0x7f85a0198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:02.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.365+0000 7f85a766f640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f85a010c170 0x7f85a019ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:02.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.365+0000 7f85a766f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f85a0111f70 con 0x7f85a0102d40 2024-09-12T02:13:02.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.365+0000 7f85a766f640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f85a0111df0 con 0x7f85a0103660 2024-09-12T02:13:02.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.365+0000 7f85a766f640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f85a01120f0 con 0x7f85a010c170 2024-09-12T02:13:02.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.365+0000 7f85a540b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85a0102d40 0x7f85a0198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:02.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.365+0000 7f85a540b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85a0102d40 0x7f85a0198400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55500/0 (socket says 172.21.15.16:55500) 2024-09-12T02:13:02.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.365+0000 7f85a540b640 1 -- 172.21.15.16:0/3645437945 learned_addr learned my addr 172.21.15.16:0/3645437945 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:13:02.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.365+0000 7f85a4c0a640 1 --2- 172.21.15.16:0/3645437945 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85a0103660 0x7f85a0198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:02.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.366+0000 7f85a5c0c640 1 --2- 172.21.15.16:0/3645437945 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f85a010c170 0x7f85a019ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:02.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.366+0000 7f85a540b640 1 -- 172.21.15.16:0/3645437945 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f85a010c170 msgr2=0x7f85a019ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:02.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.366+0000 7f85a540b640 1 --2- 172.21.15.16:0/3645437945 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f85a010c170 0x7f85a019ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:02.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.366+0000 7f85a540b640 1 -- 172.21.15.16:0/3645437945 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85a0103660 msgr2=0x7f85a0198940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:02.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.366+0000 7f85a540b640 1 --2- 172.21.15.16:0/3645437945 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85a0103660 0x7f85a0198940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:02.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.366+0000 7f85a540b640 1 -- 172.21.15.16:0/3645437945 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85a019d3b0 con 0x7f85a0102d40 2024-09-12T02:13:02.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.366+0000 7f85a540b640 1 --2- 172.21.15.16:0/3645437945 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85a0102d40 0x7f85a0198400 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f859c006010 tx=0x7f859c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:02.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.366+0000 7f85927fc640 1 -- 172.21.15.16:0/3645437945 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f859c03f030 con 0x7f85a0102d40 2024-09-12T02:13:02.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.366+0000 7f85927fc640 1 -- 172.21.15.16:0/3645437945 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f859c003ce0 con 0x7f85a0102d40 2024-09-12T02:13:02.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.366+0000 7f85a766f640 1 -- 172.21.15.16:0/3645437945 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85a019d640 con 0x7f85a0102d40 2024-09-12T02:13:02.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.366+0000 7f85927fc640 1 -- 172.21.15.16:0/3645437945 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f859c035070 con 0x7f85a0102d40 2024-09-12T02:13:02.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.366+0000 7f85a766f640 1 -- 172.21.15.16:0/3645437945 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85a01a4f80 con 0x7f85a0102d40 2024-09-12T02:13:02.380 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:13:02 smithi184 ceph-mon[35593]: pgmap v668: 97 pgs: 97 active+clean; 1.7 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-09-12T02:13:02.381 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:13:02 smithi184 ceph-mon[35593]: scrub queued for path: / 2024-09-12T02:13:02.381 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:13:02 smithi184 ceph-mon[35593]: scrub summary: idle+waiting paths [/] 2024-09-12T02:13:02.381 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:13:02 smithi184 ceph-mon[35593]: scrub summary: active paths [/] 2024-09-12T02:13:02.381 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:13:02 smithi184 ceph-mon[35593]: scrub summary: idle+waiting paths [/] 2024-09-12T02:13:02.381 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:13:02 smithi184 ceph-mon[35593]: scrub summary: idle 2024-09-12T02:13:02.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.369+0000 7f85927fc640 1 -- 172.21.15.16:0/3645437945 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f859c042030 con 0x7f85a0102d40 2024-09-12T02:13:02.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.369+0000 7f85a9a48740 1 Processor -- start 2024-09-12T02:13:02.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.369+0000 7f85a9a48740 1 -- start start 2024-09-12T02:13:02.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.369+0000 7f85a9a48740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5d9ef34a0 0x55f5d9ef3880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:02.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.369+0000 7f85a9a48740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f5d9ef3dc0 0x55f5d9efe140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:02.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.369+0000 7f85a9a48740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f5d9efe680 0x55f5d9f00a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:02.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.369+0000 7f85a9a48740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55f5d9e592e0 con 0x55f5d9ef34a0 2024-09-12T02:13:02.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.369+0000 7f85a9a48740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55f5d9e00ce0 con 0x55f5d9efe680 2024-09-12T02:13:02.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.369+0000 7f85a9a48740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55f5d9e552a0 con 0x55f5d9ef3dc0 2024-09-12T02:13:02.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.369+0000 7f85a5c0c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5d9ef34a0 0x55f5d9ef3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:02.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.370+0000 7f85a5c0c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5d9ef34a0 0x55f5d9ef3880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55506/0 (socket says 172.21.15.16:55506) 2024-09-12T02:13:02.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.370+0000 7f85a5c0c640 1 -- 172.21.15.16:0/3169434836 learned_addr learned my addr 172.21.15.16:0/3169434836 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:13:02.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.370+0000 7f85927fc640 1 --2- 172.21.15.16:0/3645437945 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f858c0a2c00 0x7f858c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:02.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.370+0000 7f85a540b640 1 --2- 172.21.15.16:0/3169434836 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f5d9ef3dc0 0x55f5d9efe140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:02.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.370+0000 7f85a4c0a640 1 --2- 172.21.15.16:0/3169434836 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f5d9efe680 0x55f5d9f00a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:02.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.370+0000 7f85a5c0c640 1 --2- 172.21.15.16:0/3645437945 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f858c0a2c00 0x7f858c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:02.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.370+0000 7f85a5c0c640 1 -- 172.21.15.16:0/3169434836 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f5d9ef3dc0 msgr2=0x55f5d9efe140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:02.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.370+0000 7f85a5c0c640 1 --2- 172.21.15.16:0/3169434836 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f5d9ef3dc0 0x55f5d9efe140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:02.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.370+0000 7f85a5c0c640 1 -- 172.21.15.16:0/3169434836 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f5d9efe680 msgr2=0x55f5d9f00a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:02.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.370+0000 7f85a5c0c640 1 --2- 172.21.15.16:0/3169434836 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f5d9efe680 0x55f5d9f00a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:02.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.370+0000 7f85a5c0c640 1 -- 172.21.15.16:0/3169434836 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5d9de5440 con 0x55f5d9ef34a0 2024-09-12T02:13:02.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.370+0000 7f85927fc640 1 -- 172.21.15.16:0/3645437945 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f859c0ef500 con 0x7f85a0102d40 2024-09-12T02:13:02.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.370+0000 7f85a5c0c640 1 --2- 172.21.15.16:0/3169434836 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5d9ef34a0 0x55f5d9ef3880 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7f85980129f0 tx=0x7f8598012eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3c0196de28dbb6e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:02.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.370+0000 7f85a5c0c640 1 --2- 172.21.15.16:0/3645437945 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f858c0a2c00 0x7f858c0a50c0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f859801c000 tx=0x7f8598020000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:02.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.370+0000 7f858b7fe640 1 -- 172.21.15.16:0/3169434836 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85980120d0 con 0x55f5d9ef34a0 2024-09-12T02:13:02.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.370+0000 7f858b7fe640 1 -- 172.21.15.16:0/3169434836 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8598023020 con 0x55f5d9ef34a0 2024-09-12T02:13:02.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.370+0000 7f858b7fe640 1 -- 172.21.15.16:0/3169434836 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8598004030 con 0x55f5d9ef34a0 2024-09-12T02:13:02.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.371+0000 7f85a9a48740 1 -- 172.21.15.16:0/3169434836 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5d9ef34a0 msgr2=0x55f5d9ef3880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:02.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.371+0000 7f85a9a48740 1 --2- 172.21.15.16:0/3169434836 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5d9ef34a0 0x55f5d9ef3880 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7f85980129f0 tx=0x7f8598012eb0 comp rx=0 tx=0).stop 2024-09-12T02:13:02.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.371+0000 7f85a9a48740 1 -- 172.21.15.16:0/3169434836 shutdown_connections 2024-09-12T02:13:02.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.371+0000 7f85a9a48740 1 --2- 172.21.15.16:0/3169434836 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f5d9efe680 0x55f5d9f00a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:02.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.371+0000 7f85a9a48740 1 --2- 172.21.15.16:0/3169434836 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f5d9ef3dc0 0x55f5d9efe140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:02.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.371+0000 7f85a9a48740 1 --2- 172.21.15.16:0/3169434836 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5d9ef34a0 0x55f5d9ef3880 unknown :-1 s=CLOSED pgs=1562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:02.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.371+0000 7f85a9a48740 1 -- 172.21.15.16:0/3169434836 >> 172.21.15.16:0/3169434836 conn(0x55f5d9eefa30 msgr2=0x55f5d9eefe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:02.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.372+0000 7f85a9a48740 1 -- 172.21.15.16:0/3169434836 shutdown_connections 2024-09-12T02:13:02.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.372+0000 7f85a9a48740 1 -- 172.21.15.16:0/3169434836 wait complete. 2024-09-12T02:13:02.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.372+0000 7f85a9a48740 1 -- start start 2024-09-12T02:13:02.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.372+0000 7f85a9a48740 1 Processor -- start 2024-09-12T02:13:02.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.373+0000 7f85a9a48740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f5d9ef34a0 0x55f5d9efdff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:02.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.373+0000 7f85a9a48740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5d9ef3dc0 0x55f5d9efb2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:02.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.373+0000 7f85a9a48740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f5d9efe680 0x55f5d9efb800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:02.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.373+0000 7f85a9a48740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55f5d9f011f0 con 0x55f5d9ef3dc0 2024-09-12T02:13:02.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.373+0000 7f85a9a48740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55f5d9e14050 con 0x55f5d9efe680 2024-09-12T02:13:02.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.373+0000 7f85a9a48740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55f5d9f01370 con 0x55f5d9ef34a0 2024-09-12T02:13:02.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.373+0000 7f85a9a48740 10 client.0 _ll_register_callbacks cb 0x55f5d9e54160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:13:02.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.373+0000 7f85a4c0a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5d9ef3dc0 0x55f5d9efb2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:02.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.373+0000 7f85a4c0a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5d9ef3dc0 0x55f5d9efb2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55512/0 (socket says 172.21.15.16:55512) 2024-09-12T02:13:02.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.373+0000 7f85a4c0a640 1 -- 172.21.15.16:0/3300699965 learned_addr learned my addr 172.21.15.16:0/3300699965 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:13:02.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.373+0000 7f85a5c0c640 1 --2- 172.21.15.16:0/3300699965 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f5d9efe680 0x55f5d9efb800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:02.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.373+0000 7f85a540b640 1 --2- 172.21.15.16:0/3300699965 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f5d9ef34a0 0x55f5d9efdff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:02.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.373+0000 7f85a4c0a640 1 -- 172.21.15.16:0/3300699965 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f5d9ef34a0 msgr2=0x55f5d9efdff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:02.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.373+0000 7f85a4c0a640 1 --2- 172.21.15.16:0/3300699965 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f5d9ef34a0 0x55f5d9efdff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:02.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.373+0000 7f85a4c0a640 1 -- 172.21.15.16:0/3300699965 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f5d9efe680 msgr2=0x55f5d9efb800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:02.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.373+0000 7f85a4c0a640 1 --2- 172.21.15.16:0/3300699965 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f5d9efe680 0x55f5d9efb800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:02.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.373+0000 7f85a4c0a640 1 -- 172.21.15.16:0/3300699965 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f5d9bca3c0 con 0x55f5d9ef3dc0 2024-09-12T02:13:02.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.373+0000 7f85a4c0a640 1 --2- 172.21.15.16:0/3300699965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5d9ef3dc0 0x55f5d9efb2a0 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f8594005f80 tx=0x7f859400be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9097051093414820 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:02.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.373+0000 7f8588ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:13:02.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.374+0000 7f8588ff9640 1 -- 172.21.15.16:0/3300699965 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f859401b440 con 0x55f5d9ef3dc0 2024-09-12T02:13:02.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.374+0000 7f8588ff9640 1 -- 172.21.15.16:0/3300699965 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f859401b5e0 con 0x55f5d9ef3dc0 2024-09-12T02:13:02.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.374+0000 7f85a9a48740 1 -- 172.21.15.16:0/3300699965 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8598023020 con 0x55f5d9ef3dc0 2024-09-12T02:13:02.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.374+0000 7f8588ff9640 1 -- 172.21.15.16:0/3300699965 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f859401b7e0 con 0x55f5d9ef3dc0 2024-09-12T02:13:02.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.374+0000 7f85a9a48740 10 client.19020 fetch_fsmap learned FSMap version 25 2024-09-12T02:13:02.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.374+0000 7f85a9a48740 1 -- 172.21.15.16:0/3300699965 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f5d9f0cc20 con 0x55f5d9ef3dc0 2024-09-12T02:13:02.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.374+0000 7f8588ff9640 1 -- 172.21.15.16:0/3300699965 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8598023020 con 0x55f5d9ef3dc0 2024-09-12T02:13:02.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.374+0000 7f8588ff9640 1 -- 172.21.15.16:0/3300699965 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f8594019020 con 0x55f5d9ef3dc0 2024-09-12T02:13:02.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.374+0000 7f85a9a48740 10 client.19020 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:13:02.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.374+0000 7f85a9a48740 10 client.19020 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:13:02.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.374+0000 7f85a9a48740 20 client.19020 populate_metadata read hostname 'smithi016' 2024-09-12T02:13:02.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.374+0000 7f85a9a48740 1 --2- 172.21.15.16:0/3300699965 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55f5d9f0e130 0x55f5d9f10520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:02.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.374+0000 7f85a9a48740 4 client.19020 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:13:02.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.374+0000 7f85a9a48740 1 -- 172.21.15.16:0/3300699965 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f5d9e66850 con 0x55f5d9f0e130 2024-09-12T02:13:02.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.375+0000 7f85a540b640 1 --2- 172.21.15.16:0/3300699965 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55f5d9f0e130 0x55f5d9f10520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:02.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.375+0000 7f85a540b640 1 --2- 172.21.15.16:0/3300699965 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55f5d9f0e130 0x55f5d9f10520 crc :-1 s=READY pgs=332 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e62972c5bdb9fbd0 server_cookie=5fe343a90352b474 in_seq=0 out_seq=0 2024-09-12T02:13:02.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.375+0000 7f8588ff9640 10 client.19020 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:13:02.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.376+0000 7f8588ff9640 1 -- 172.21.15.16:0/3300699965 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f5d9e66850 con 0x55f5d9f0e130 2024-09-12T02:13:02.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.376+0000 7f8588ff9640 10 client.19020 handle_command_reply: tid=0 2024-09-12T02:13:02.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.376+0000 7f85a9a48740 1 client.19020 shutdown 2024-09-12T02:13:02.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.376+0000 7f85a9a48740 1 -- 172.21.15.16:0/3300699965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5d9ef3dc0 msgr2=0x55f5d9efb2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:02.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.376+0000 7f85a9a48740 1 --2- 172.21.15.16:0/3300699965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5d9ef3dc0 0x55f5d9efb2a0 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f8594005f80 tx=0x7f859400be70 comp rx=0 tx=0).stop 2024-09-12T02:13:02.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.376+0000 7f85a9a48740 1 -- 172.21.15.16:0/3300699965 shutdown_connections 2024-09-12T02:13:02.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.376+0000 7f85a9a48740 1 --2- 172.21.15.16:0/3300699965 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55f5d9f0e130 0x55f5d9f10520 crc :-1 s=READY pgs=332 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:02.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.376+0000 7f85a9a48740 1 --2- 172.21.15.16:0/3300699965 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f5d9efe680 0x55f5d9efb800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:02.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.376+0000 7f85a9a48740 1 --2- 172.21.15.16:0/3300699965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5d9ef3dc0 0x55f5d9efb2a0 unknown :-1 s=CLOSED pgs=1563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:02.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.376+0000 7f85a9a48740 1 --2- 172.21.15.16:0/3300699965 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f5d9ef34a0 0x55f5d9efdff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:02.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.376+0000 7f85a9a48740 1 -- 172.21.15.16:0/3300699965 >> 172.21.15.16:0/3300699965 conn(0x55f5d9eefa30 msgr2=0x55f5d9eeeb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:02.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.376+0000 7f8588ff9640 0 client.19020 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:13:02.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.376+0000 7f85a9a48740 1 -- 172.21.15.16:0/3300699965 shutdown_connections 2024-09-12T02:13:02.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.376+0000 7f85a9a48740 1 -- 172.21.15.16:0/3300699965 wait complete. 2024-09-12T02:13:02.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.376+0000 7f85a9a48740 20 client.19020 trim_cache size 0 max 16384 2024-09-12T02:13:02.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.394+0000 7f85a9a48740 1 Processor -- start 2024-09-12T02:13:02.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.394+0000 7f85a9a48740 1 -- start start 2024-09-12T02:13:02.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.394+0000 7f85a9a48740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5d9ef34a0 0x55f5d9f0a450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:02.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.394+0000 7f85a9a48740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f5d9ef3dc0 0x55f5d9eefe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:02.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.394+0000 7f85a9a48740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f5d9efe680 0x55f5d9ef67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:02.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.394+0000 7f85a9a48740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f85a01adad0 con 0x55f5d9ef34a0 2024-09-12T02:13:02.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.394+0000 7f85a9a48740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f85a01ab3a0 con 0x55f5d9ef3dc0 2024-09-12T02:13:02.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.394+0000 7f85a9a48740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f85a01a91f0 con 0x55f5d9efe680 2024-09-12T02:13:02.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.394+0000 7f85a540b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5d9ef34a0 0x55f5d9f0a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:02.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.394+0000 7f85a540b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5d9ef34a0 0x55f5d9f0a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55524/0 (socket says 172.21.15.16:55524) 2024-09-12T02:13:02.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.394+0000 7f85a540b640 1 -- 172.21.15.16:0/1074898484 learned_addr learned my addr 172.21.15.16:0/1074898484 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:13:02.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.394+0000 7f85a540b640 1 -- 172.21.15.16:0/1074898484 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f5d9efe680 msgr2=0x55f5d9ef67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:02.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.394+0000 7f85a4c0a640 1 --2- 172.21.15.16:0/1074898484 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f5d9ef3dc0 0x55f5d9eefe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:02.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.394+0000 7f85a5c0c640 1 --2- 172.21.15.16:0/1074898484 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f5d9efe680 0x55f5d9ef67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:02.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.394+0000 7f85a540b640 1 --2- 172.21.15.16:0/1074898484 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f5d9efe680 0x55f5d9ef67f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:02.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.394+0000 7f85a540b640 1 -- 172.21.15.16:0/1074898484 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f5d9ef3dc0 msgr2=0x55f5d9eefe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:02.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.395+0000 7f85a540b640 1 --2- 172.21.15.16:0/1074898484 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f5d9ef3dc0 0x55f5d9eefe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:02.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.395+0000 7f85a540b640 1 -- 172.21.15.16:0/1074898484 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5d9ef97f0 con 0x55f5d9ef34a0 2024-09-12T02:13:02.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.395+0000 7f85a4c0a640 1 --2- 172.21.15.16:0/1074898484 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f5d9ef3dc0 0x55f5d9eefe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:13:02.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.395+0000 7f85a540b640 1 --2- 172.21.15.16:0/1074898484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5d9ef34a0 0x55f5d9f0a450 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f859c0b8890 tx=0x7f859c0ef700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b349f10729b9061e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:02.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.395+0000 7f858affd640 1 -- 172.21.15.16:0/1074898484 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f859c049040 con 0x55f5d9ef34a0 2024-09-12T02:13:02.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.395+0000 7f858affd640 1 -- 172.21.15.16:0/1074898484 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f859c0bc5c0 con 0x55f5d9ef34a0 2024-09-12T02:13:02.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.395+0000 7f858affd640 1 -- 172.21.15.16:0/1074898484 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f859c0c4b30 con 0x55f5d9ef34a0 2024-09-12T02:13:02.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.395+0000 7f85a9a48740 1 -- 172.21.15.16:0/1074898484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5d9ef34a0 msgr2=0x55f5d9f0a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:02.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.395+0000 7f85a9a48740 1 --2- 172.21.15.16:0/1074898484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5d9ef34a0 0x55f5d9f0a450 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f859c0b8890 tx=0x7f859c0ef700 comp rx=0 tx=0).stop 2024-09-12T02:13:02.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.395+0000 7f85a9a48740 1 -- 172.21.15.16:0/1074898484 shutdown_connections 2024-09-12T02:13:02.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.395+0000 7f85a9a48740 1 --2- 172.21.15.16:0/1074898484 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f5d9efe680 0x55f5d9ef67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:02.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.396+0000 7f85a9a48740 1 --2- 172.21.15.16:0/1074898484 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f5d9ef3dc0 0x55f5d9eefe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:02.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.396+0000 7f85a9a48740 1 --2- 172.21.15.16:0/1074898484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5d9ef34a0 0x55f5d9f0a450 unknown :-1 s=CLOSED pgs=1564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:02.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.396+0000 7f85a9a48740 1 -- 172.21.15.16:0/1074898484 >> 172.21.15.16:0/1074898484 conn(0x55f5d9eefa30 msgr2=0x55f5d9ef60e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:02.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.396+0000 7f85a9a48740 1 -- 172.21.15.16:0/1074898484 shutdown_connections 2024-09-12T02:13:02.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.396+0000 7f85a9a48740 1 -- 172.21.15.16:0/1074898484 wait complete. 2024-09-12T02:13:02.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.396+0000 7f85a9a48740 1 -- start start 2024-09-12T02:13:02.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.396+0000 7f85a9a48740 1 Processor -- start 2024-09-12T02:13:02.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.396+0000 7f85a9a48740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5d9ef34a0 0x55f5d9f06df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:02.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.396+0000 7f85a9a48740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f5d9ef3dc0 0x55f5d9f02120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:02.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.396+0000 7f85a9a48740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f5d9efe680 0x55f5d9f02710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:02.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.396+0000 7f85a9a48740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f85a01a9860 con 0x55f5d9ef34a0 2024-09-12T02:13:02.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.396+0000 7f85a9a48740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f85a01aacd0 con 0x55f5d9ef3dc0 2024-09-12T02:13:02.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.396+0000 7f85a9a48740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f85a01a9f30 con 0x55f5d9efe680 2024-09-12T02:13:02.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.396+0000 7f85a9a48740 10 client.0 _ll_register_callbacks cb 0x7f8594005cd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:13:02.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.397+0000 7f85a540b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5d9ef34a0 0x55f5d9f06df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:02.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.397+0000 7f85a540b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5d9ef34a0 0x55f5d9f06df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55528/0 (socket says 172.21.15.16:55528) 2024-09-12T02:13:02.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.397+0000 7f85a540b640 1 -- 172.21.15.16:0/3809033735 learned_addr learned my addr 172.21.15.16:0/3809033735 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:13:02.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.397+0000 7f85a4c0a640 1 --2- 172.21.15.16:0/3809033735 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f5d9ef3dc0 0x55f5d9f02120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:02.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.397+0000 7f85a540b640 1 -- 172.21.15.16:0/3809033735 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f5d9efe680 msgr2=0x55f5d9f02710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:02.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.397+0000 7f85a540b640 1 --2- 172.21.15.16:0/3809033735 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f5d9efe680 0x55f5d9f02710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:02.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.397+0000 7f85a540b640 1 -- 172.21.15.16:0/3809033735 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f5d9ef3dc0 msgr2=0x55f5d9f02120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:02.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.397+0000 7f85a540b640 1 --2- 172.21.15.16:0/3809033735 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f5d9ef3dc0 0x55f5d9f02120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:02.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.397+0000 7f85a540b640 1 -- 172.21.15.16:0/3809033735 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f5d9f02f40 con 0x55f5d9ef34a0 2024-09-12T02:13:02.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.397+0000 7f85a540b640 1 --2- 172.21.15.16:0/3809033735 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5d9ef34a0 0x55f5d9f06df0 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f859c005ab0 tx=0x7f859c003f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a8b7539694da22f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:02.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.397+0000 7f8588ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:13:02.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.397+0000 7f8588ff9640 1 -- 172.21.15.16:0/3809033735 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f859c0aa020 con 0x55f5d9ef34a0 2024-09-12T02:13:02.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.397+0000 7f8588ff9640 1 -- 172.21.15.16:0/3809033735 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f859c0ba1c0 con 0x55f5d9ef34a0 2024-09-12T02:13:02.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.397+0000 7f85a9a48740 1 -- 172.21.15.16:0/3809033735 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f859c0bc5c0 con 0x55f5d9ef34a0 2024-09-12T02:13:02.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.397+0000 7f8588ff9640 1 -- 172.21.15.16:0/3809033735 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f859c049050 con 0x55f5d9ef34a0 2024-09-12T02:13:02.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.397+0000 7f85a9a48740 10 client.19026 fetch_fsmap learned FSMap version 25 2024-09-12T02:13:02.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.397+0000 7f85a9a48740 1 -- 172.21.15.16:0/3809033735 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f5d9f78e20 con 0x55f5d9ef34a0 2024-09-12T02:13:02.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.398+0000 7f8588ff9640 1 -- 172.21.15.16:0/3809033735 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f859c0bc5c0 con 0x55f5d9ef34a0 2024-09-12T02:13:02.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.398+0000 7f8588ff9640 1 -- 172.21.15.16:0/3809033735 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f859c0a6bd0 con 0x55f5d9ef34a0 2024-09-12T02:13:02.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.398+0000 7f85a9a48740 10 client.19026 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:13:02.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.398+0000 7f85a9a48740 10 client.19026 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:13:02.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.398+0000 7f85a9a48740 20 client.19026 populate_metadata read hostname 'smithi016' 2024-09-12T02:13:02.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.398+0000 7f85a9a48740 1 --2- 172.21.15.16:0/3809033735 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55f5d9f793e0 0x55f5d9f7b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:02.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.398+0000 7f85a9a48740 4 client.19026 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T02:13:02.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.398+0000 7f85a9a48740 1 -- 172.21.15.16:0/3809033735 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f5d9e60d60 con 0x55f5d9f793e0 2024-09-12T02:13:02.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.398+0000 7f85a4c0a640 1 --2- 172.21.15.16:0/3809033735 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55f5d9f793e0 0x55f5d9f7b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:02.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.399+0000 7f85a4c0a640 1 --2- 172.21.15.16:0/3809033735 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55f5d9f793e0 0x55f5d9f7b800 crc :-1 s=READY pgs=333 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa10eeb5d30e9c99 server_cookie=71282183bec1f879 in_seq=0 out_seq=0 2024-09-12T02:13:02.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.399+0000 7f8588ff9640 10 client.19026 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:13:02.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.399+0000 7f8588ff9640 1 -- 172.21.15.16:0/3809033735 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55f5d9e60d60 con 0x55f5d9f793e0 2024-09-12T02:13:02.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.399+0000 7f8588ff9640 10 client.19026 handle_command_reply: tid=0 2024-09-12T02:13:02.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.399+0000 7f85a9a48740 1 client.19026 shutdown 2024-09-12T02:13:02.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.399+0000 7f85a9a48740 1 -- 172.21.15.16:0/3809033735 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5d9ef34a0 msgr2=0x55f5d9f06df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:02.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.399+0000 7f85a9a48740 1 --2- 172.21.15.16:0/3809033735 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5d9ef34a0 0x55f5d9f06df0 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f859c005ab0 tx=0x7f859c003f70 comp rx=0 tx=0).stop 2024-09-12T02:13:02.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.399+0000 7f85a9a48740 1 -- 172.21.15.16:0/3809033735 shutdown_connections 2024-09-12T02:13:02.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.399+0000 7f85a9a48740 1 --2- 172.21.15.16:0/3809033735 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55f5d9f793e0 0x55f5d9f7b800 crc :-1 s=READY pgs=333 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:02.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.399+0000 7f85a9a48740 1 --2- 172.21.15.16:0/3809033735 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f5d9efe680 0x55f5d9f02710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:02.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.399+0000 7f85a9a48740 1 --2- 172.21.15.16:0/3809033735 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f5d9ef3dc0 0x55f5d9f02120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:02.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.399+0000 7f8588ff9640 0 client.19026 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:13:02.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.399+0000 7f85a9a48740 1 --2- 172.21.15.16:0/3809033735 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5d9ef34a0 0x55f5d9f06df0 unknown :-1 s=CLOSED pgs=1565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:02.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.399+0000 7f85a9a48740 1 -- 172.21.15.16:0/3809033735 >> 172.21.15.16:0/3809033735 conn(0x55f5d9eefa30 msgr2=0x55f5d9eeeb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:02.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.399+0000 7f85a9a48740 1 -- 172.21.15.16:0/3809033735 shutdown_connections 2024-09-12T02:13:02.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.399+0000 7f85a9a48740 1 -- 172.21.15.16:0/3809033735 wait complete. 2024-09-12T02:13:02.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.399+0000 7f85a9a48740 20 client.19026 trim_cache size 0 max 16384 2024-09-12T02:13:02.411 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T02:13:02.411 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-09-12T02:13:02.411 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {} 2024-09-12T02:13:02.411 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T02:13:02.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.400+0000 7f85a766f640 1 -- 172.21.15.16:0/3645437945 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f858c0a2c00 msgr2=0x7f858c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:02.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.400+0000 7f85a766f640 1 --2- 172.21.15.16:0/3645437945 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f858c0a2c00 0x7f858c0a50c0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f859801c000 tx=0x7f8598020000 comp rx=0 tx=0).stop 2024-09-12T02:13:02.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.400+0000 7f85a766f640 1 -- 172.21.15.16:0/3645437945 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85a0102d40 msgr2=0x7f85a0198400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:02.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.400+0000 7f85a766f640 1 --2- 172.21.15.16:0/3645437945 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85a0102d40 0x7f85a0198400 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f859c006010 tx=0x7f859c0042f0 comp rx=0 tx=0).stop 2024-09-12T02:13:02.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.400+0000 7f85a766f640 1 -- 172.21.15.16:0/3645437945 shutdown_connections 2024-09-12T02:13:02.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.400+0000 7f85a766f640 1 --2- 172.21.15.16:0/3645437945 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f858c0a2c00 0x7f858c0a50c0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:02.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.400+0000 7f85a766f640 1 --2- 172.21.15.16:0/3645437945 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f85a010c170 0x7f85a019ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:02.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.400+0000 7f85a766f640 1 --2- 172.21.15.16:0/3645437945 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85a0103660 0x7f85a0198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:02.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.400+0000 7f85a766f640 1 --2- 172.21.15.16:0/3645437945 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85a0102d40 0x7f85a0198400 unknown :-1 s=CLOSED pgs=1561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:02.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.400+0000 7f85a766f640 1 -- 172.21.15.16:0/3645437945 >> 172.21.15.16:0/3645437945 conn(0x7f85a00fc7a0 msgr2=0x7f85a00fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:02.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.400+0000 7f85a766f640 1 -- 172.21.15.16:0/3645437945 shutdown_connections 2024-09-12T02:13:02.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:02.400+0000 7f85a766f640 1 -- 172.21.15.16:0/3645437945 wait complete. 2024-09-12T02:13:02.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:13:02 smithi033 ceph-mon[35332]: pgmap v668: 97 pgs: 97 active+clean; 1.7 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-09-12T02:13:02.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:13:02 smithi033 ceph-mon[35332]: scrub queued for path: / 2024-09-12T02:13:02.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:13:02 smithi033 ceph-mon[35332]: scrub summary: idle+waiting paths [/] 2024-09-12T02:13:02.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:13:02 smithi033 ceph-mon[35332]: scrub summary: active paths [/] 2024-09-12T02:13:02.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:13:02 smithi033 ceph-mon[35332]: scrub summary: idle+waiting paths [/] 2024-09-12T02:13:02.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:13:02 smithi033 ceph-mon[35332]: scrub summary: idle 2024-09-12T02:13:03.013 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-12T02:13:03.014 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:13:03.273 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:13:03.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:03.794+0000 7f50fe71d640 1 -- 172.21.15.16:0/3971462002 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50f810c170 msgr2=0x7f50f810e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:03.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:03.794+0000 7f50fe71d640 1 --2- 172.21.15.16:0/3971462002 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50f810c170 0x7f50f810e610 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f50f0009be0 tx=0x7f50f002a750 comp rx=0 tx=0).stop 2024-09-12T02:13:03.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:03.794+0000 7f50fe71d640 1 -- 172.21.15.16:0/3971462002 shutdown_connections 2024-09-12T02:13:03.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:03.794+0000 7f50fe71d640 1 --2- 172.21.15.16:0/3971462002 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50f810c170 0x7f50f810e610 unknown :-1 s=CLOSED pgs=1566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:03.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:03.794+0000 7f50fe71d640 1 --2- 172.21.15.16:0/3971462002 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f50f8103660 0x7f50f810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:03.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:03.794+0000 7f50fe71d640 1 --2- 172.21.15.16:0/3971462002 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f50f8102d40 0x7f50f8103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:03.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:03.794+0000 7f50fe71d640 1 -- 172.21.15.16:0/3971462002 >> 172.21.15.16:0/3971462002 conn(0x7f50f80fc7a0 msgr2=0x7f50f80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:03.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:03.795+0000 7f50fe71d640 1 -- 172.21.15.16:0/3971462002 shutdown_connections 2024-09-12T02:13:03.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:03.795+0000 7f50fe71d640 1 -- 172.21.15.16:0/3971462002 wait complete. 2024-09-12T02:13:03.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:03.795+0000 7f50fe71d640 1 Processor -- start 2024-09-12T02:13:03.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:03.795+0000 7f50fe71d640 1 -- start start 2024-09-12T02:13:03.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:03.796+0000 7f50fe71d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f50f8102d40 0x7f50f81009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:03.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:03.796+0000 7f50fe71d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f50f8103660 0x7f50f8100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:03.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:03.796+0000 7f50fe71d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50f810c170 0x7f50f8101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:03.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:03.796+0000 7f50fe71d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f50f8111f20 con 0x7f50f810c170 2024-09-12T02:13:03.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:03.796+0000 7f50fe71d640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f50f8111da0 con 0x7f50f8102d40 2024-09-12T02:13:03.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:03.796+0000 7f50fe71d640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f50f81120a0 con 0x7f50f8103660 2024-09-12T02:13:03.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:03.796+0000 7f50fccba640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50f810c170 0x7f50f8101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:03.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:03.796+0000 7f50fccba640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50f810c170 0x7f50f8101420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55546/0 (socket says 172.21.15.16:55546) 2024-09-12T02:13:03.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:03.796+0000 7f50fccba640 1 -- 172.21.15.16:0/3798183619 learned_addr learned my addr 172.21.15.16:0/3798183619 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:13:03.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:03.796+0000 7f50f77fe640 1 --2- 172.21.15.16:0/3798183619 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f50f8103660 0x7f50f8100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:03.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:03.796+0000 7f50f7fff640 1 --2- 172.21.15.16:0/3798183619 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f50f8102d40 0x7f50f81009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:03.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:03.796+0000 7f50fccba640 1 -- 172.21.15.16:0/3798183619 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f50f8103660 msgr2=0x7f50f8100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:03.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:03.796+0000 7f50fccba640 1 --2- 172.21.15.16:0/3798183619 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f50f8103660 0x7f50f8100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:03.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:03.796+0000 7f50fccba640 1 -- 172.21.15.16:0/3798183619 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f50f8102d40 msgr2=0x7f50f81009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:03.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:03.796+0000 7f50fccba640 1 --2- 172.21.15.16:0/3798183619 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f50f8102d40 0x7f50f81009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:03.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:03.796+0000 7f50fccba640 1 -- 172.21.15.16:0/3798183619 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50f81a68a0 con 0x7f50f810c170 2024-09-12T02:13:03.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:03.796+0000 7f50fccba640 1 --2- 172.21.15.16:0/3798183619 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50f810c170 0x7f50f8101420 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f50f0009d00 tx=0x7f50f0030630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:03.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:03.797+0000 7f50f57fa640 1 -- 172.21.15.16:0/3798183619 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50f0041020 con 0x7f50f810c170 2024-09-12T02:13:03.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:03.797+0000 7f50fe71d640 1 -- 172.21.15.16:0/3798183619 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50f81a6b30 con 0x7f50f810c170 2024-09-12T02:13:03.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:03.797+0000 7f50f57fa640 1 -- 172.21.15.16:0/3798183619 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f50f0037040 con 0x7f50f810c170 2024-09-12T02:13:03.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:03.797+0000 7f50f57fa640 1 -- 172.21.15.16:0/3798183619 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50f0036070 con 0x7f50f810c170 2024-09-12T02:13:03.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:03.797+0000 7f50fe71d640 1 -- 172.21.15.16:0/3798183619 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50f81a7060 con 0x7f50f810c170 2024-09-12T02:13:03.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:03.798+0000 7f50fe71d640 1 -- 172.21.15.16:0/3798183619 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50f8103120 con 0x7f50f810c170 2024-09-12T02:13:03.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:03.799+0000 7f50f57fa640 1 -- 172.21.15.16:0/3798183619 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f50f0044020 con 0x7f50f810c170 2024-09-12T02:13:03.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:03.800+0000 7f50f57fa640 1 --2- 172.21.15.16:0/3798183619 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f50e00a2b80 0x7f50e00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:03.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:03.800+0000 7f50f7fff640 1 --2- 172.21.15.16:0/3798183619 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f50e00a2b80 0x7f50e00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:03.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:03.800+0000 7f50f57fa640 1 -- 172.21.15.16:0/3798183619 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f50f00f0040 con 0x7f50f810c170 2024-09-12T02:13:03.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:03.800+0000 7f50f7fff640 1 --2- 172.21.15.16:0/3798183619 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f50e00a2b80 0x7f50e00a5040 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f50e8009b60 tx=0x7f50e8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:03.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:03.803+0000 7f50f57fa640 1 -- 172.21.15.16:0/3798183619 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f50f004b020 con 0x7f50f810c170 2024-09-12T02:13:04.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:04.017+0000 7f50fe71d640 1 -- 172.21.15.16:0/3798183619 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f50f802d7c0 con 0x7f50f810c170 2024-09-12T02:13:04.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:04.018+0000 7f50f57fa640 1 -- 172.21.15.16:0/3798183619 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11911 (secure 0 0 0) 0x7f50f00bd0c0 con 0x7f50f810c170 2024-09-12T02:13:04.018 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T02:13:04.019 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":25,"btime":"2024-09-12T02:01:37:192021+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3279123279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3279123279},{"type":"v1","addr":"172.21.15.16:6835","nonce":3279123279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4119452382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4119452382},{"type":"v1","addr":"172.21.15.16:6837","nonce":4119452382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/3263823165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3263823165},{"type":"v1","addr":"172.21.15.16:6839","nonce":3263823165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/1510290732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":1510290732},{"type":"v1","addr":"172.21.15.16:6841","nonce":1510290732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/279674603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":279674603},{"type":"v1","addr":"172.21.15.33:6833","nonce":279674603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3555728028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3555728028},{"type":"v1","addr":"172.21.15.33:6835","nonce":3555728028}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T01:59:56.117608+0000","modified":"2024-09-12T02:01:36.294760+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24482,"mds_1":24481,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6833/1563374106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1563374106},{"type":"v1","addr":"172.21.15.184:6833","nonce":1563374106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6837/1892833868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1892833868},{"type":"v1","addr":"172.21.15.33:6837","nonce":1892833868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/1457679686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1457679686},{"type":"v1","addr":"172.21.15.184:6835","nonce":1457679686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.184:6837/1793393087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":1793393087},{"type":"v1","addr":"172.21.15.184:6837","nonce":1793393087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.184:6839/4203347941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":4203347941},{"type":"v1","addr":"172.21.15.184:6839","nonce":4203347941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.33:6839/4136670204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":4136670204},{"type":"v1","addr":"172.21.15.33:6839","nonce":4136670204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24481,24482,24469]},"id":1}]} 2024-09-12T02:13:04.019 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 25 2024-09-12T02:13:04.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:04.020+0000 7f50fe71d640 1 -- 172.21.15.16:0/3798183619 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f50e00a2b80 msgr2=0x7f50e00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:04.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:04.020+0000 7f50fe71d640 1 --2- 172.21.15.16:0/3798183619 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f50e00a2b80 0x7f50e00a5040 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f50e8009b60 tx=0x7f50e8005f90 comp rx=0 tx=0).stop 2024-09-12T02:13:04.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:04.020+0000 7f50fe71d640 1 -- 172.21.15.16:0/3798183619 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50f810c170 msgr2=0x7f50f8101420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:04.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:04.020+0000 7f50fe71d640 1 --2- 172.21.15.16:0/3798183619 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50f810c170 0x7f50f8101420 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f50f0009d00 tx=0x7f50f0030630 comp rx=0 tx=0).stop 2024-09-12T02:13:04.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:04.020+0000 7f50fe71d640 1 -- 172.21.15.16:0/3798183619 shutdown_connections 2024-09-12T02:13:04.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:04.020+0000 7f50fe71d640 1 --2- 172.21.15.16:0/3798183619 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f50e00a2b80 0x7f50e00a5040 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:04.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:04.020+0000 7f50fe71d640 1 --2- 172.21.15.16:0/3798183619 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50f810c170 0x7f50f8101420 unknown :-1 s=CLOSED pgs=1567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:04.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:04.020+0000 7f50fe71d640 1 --2- 172.21.15.16:0/3798183619 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f50f8103660 0x7f50f8100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:04.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:04.020+0000 7f50fe71d640 1 --2- 172.21.15.16:0/3798183619 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f50f8102d40 0x7f50f81009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:04.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:04.020+0000 7f50fe71d640 1 -- 172.21.15.16:0/3798183619 >> 172.21.15.16:0/3798183619 conn(0x7f50f80fc7a0 msgr2=0x7f50f80fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:04.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:04.020+0000 7f50fe71d640 1 -- 172.21.15.16:0/3798183619 shutdown_connections 2024-09-12T02:13:04.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:04.021+0000 7f50fe71d640 1 -- 172.21.15.16:0/3798183619 wait complete. 2024-09-12T02:13:04.340 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:13:04 smithi016 ceph-mon[30947]: pgmap v669: 97 pgs: 97 active+clean; 1.8 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 17 op/s 2024-09-12T02:13:04.340 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:13:04 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/3798183619' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:13:04.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:13:04 smithi184 ceph-mon[35593]: pgmap v669: 97 pgs: 97 active+clean; 1.8 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 17 op/s 2024-09-12T02:13:04.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:13:04 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/3798183619' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:13:04.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:13:04 smithi033 ceph-mon[35332]: pgmap v669: 97 pgs: 97 active+clean; 1.8 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 17 op/s 2024-09-12T02:13:04.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:13:04 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/3798183619' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:13:04.725 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-12T02:13:04.986 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:13:05.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.496+0000 7f3c214b9640 1 -- 172.21.15.16:0/3517899964 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c1c10c180 msgr2=0x7f3c1c10e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:05.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.496+0000 7f3c214b9640 1 --2- 172.21.15.16:0/3517899964 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c1c10c180 0x7f3c1c10e620 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7f3c0c009ca0 tx=0x7f3c0c02a950 comp rx=0 tx=0).stop 2024-09-12T02:13:05.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.496+0000 7f3c214b9640 1 -- 172.21.15.16:0/3517899964 shutdown_connections 2024-09-12T02:13:05.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.496+0000 7f3c214b9640 1 --2- 172.21.15.16:0/3517899964 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c1c10c180 0x7f3c1c10e620 unknown :-1 s=CLOSED pgs=1568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:05.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.496+0000 7f3c214b9640 1 --2- 172.21.15.16:0/3517899964 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3c1c103670 0x7f3c1c10bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:05.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.496+0000 7f3c214b9640 1 --2- 172.21.15.16:0/3517899964 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c1c102d50 0x7f3c1c103130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:05.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.496+0000 7f3c214b9640 1 -- 172.21.15.16:0/3517899964 >> 172.21.15.16:0/3517899964 conn(0x7f3c1c0fc7d0 msgr2=0x7f3c1c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:05.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.496+0000 7f3c214b9640 1 -- 172.21.15.16:0/3517899964 shutdown_connections 2024-09-12T02:13:05.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.496+0000 7f3c214b9640 1 -- 172.21.15.16:0/3517899964 wait complete. 2024-09-12T02:13:05.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.497+0000 7f3c214b9640 1 Processor -- start 2024-09-12T02:13:05.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.497+0000 7f3c214b9640 1 -- start start 2024-09-12T02:13:05.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.497+0000 7f3c214b9640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3c1c102d50 0x7f3c1c1983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:05.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.497+0000 7f3c214b9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c1c103670 0x7f3c1c1988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:05.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.497+0000 7f3c214b9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c1c10c180 0x7f3c1c19cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:05.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.497+0000 7f3c214b9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3c1c111fb0 con 0x7f3c1c103670 2024-09-12T02:13:05.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.497+0000 7f3c214b9640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3c1c111e30 con 0x7f3c1c10c180 2024-09-12T02:13:05.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.497+0000 7f3c214b9640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f3c1c112130 con 0x7f3c1c102d50 2024-09-12T02:13:05.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.498+0000 7f3c1a7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c1c103670 0x7f3c1c1988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:05.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.498+0000 7f3c1affd640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3c1c102d50 0x7f3c1c1983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:05.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.498+0000 7f3c1a7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c1c103670 0x7f3c1c1988f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55562/0 (socket says 172.21.15.16:55562) 2024-09-12T02:13:05.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.498+0000 7f3c1a7fc640 1 -- 172.21.15.16:0/1510267735 learned_addr learned my addr 172.21.15.16:0/1510267735 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:13:05.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.498+0000 7f3c1b7fe640 1 --2- 172.21.15.16:0/1510267735 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c1c10c180 0x7f3c1c19cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:05.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.498+0000 7f3c1a7fc640 1 -- 172.21.15.16:0/1510267735 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3c1c102d50 msgr2=0x7f3c1c1983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:05.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.498+0000 7f3c1a7fc640 1 --2- 172.21.15.16:0/1510267735 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3c1c102d50 0x7f3c1c1983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:05.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.498+0000 7f3c1a7fc640 1 -- 172.21.15.16:0/1510267735 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c1c10c180 msgr2=0x7f3c1c19cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:05.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.498+0000 7f3c1a7fc640 1 --2- 172.21.15.16:0/1510267735 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c1c10c180 0x7f3c1c19cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:05.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.498+0000 7f3c1a7fc640 1 -- 172.21.15.16:0/1510267735 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c1c19d360 con 0x7f3c1c103670 2024-09-12T02:13:05.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.498+0000 7f3c1a7fc640 1 --2- 172.21.15.16:0/1510267735 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c1c103670 0x7f3c1c1988f0 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f3c0400d750 tx=0x7f3c0400db50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:05.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.498+0000 7f3c13fff640 1 -- 172.21.15.16:0/1510267735 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c04017070 con 0x7f3c1c103670 2024-09-12T02:13:05.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.498+0000 7f3c13fff640 1 -- 172.21.15.16:0/1510267735 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c04009820 con 0x7f3c1c103670 2024-09-12T02:13:05.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.498+0000 7f3c13fff640 1 -- 172.21.15.16:0/1510267735 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c04004c60 con 0x7f3c1c103670 2024-09-12T02:13:05.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.498+0000 7f3c214b9640 1 -- 172.21.15.16:0/1510267735 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c1c19d650 con 0x7f3c1c103670 2024-09-12T02:13:05.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.498+0000 7f3c214b9640 1 -- 172.21.15.16:0/1510267735 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c1c1a4f30 con 0x7f3c1c103670 2024-09-12T02:13:05.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.500+0000 7f3c13fff640 1 -- 172.21.15.16:0/1510267735 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f3c04005220 con 0x7f3c1c103670 2024-09-12T02:13:05.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.501+0000 7f3c23892740 1 Processor -- start 2024-09-12T02:13:05.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.501+0000 7f3c23892740 1 -- start start 2024-09-12T02:13:05.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.501+0000 7f3c23892740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55705fa3eb40 0x55705fa3d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:05.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.501+0000 7f3c23892740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55705fa3d5d0 0x55705fa3da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:05.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.501+0000 7f3c23892740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55705fa48370 0x55705fa4a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:05.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.501+0000 7f3c23892740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55705f9a32f0 con 0x55705fa48370 2024-09-12T02:13:05.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.501+0000 7f3c23892740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55705f94acf0 con 0x55705fa3d5d0 2024-09-12T02:13:05.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.501+0000 7f3c23892740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55705f99f2b0 con 0x55705fa3eb40 2024-09-12T02:13:05.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.502+0000 7f3c1a7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55705fa48370 0x55705fa4a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:05.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.502+0000 7f3c1affd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55705fa3d5d0 0x55705fa3da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:05.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.502+0000 7f3c1a7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55705fa48370 0x55705fa4a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55578/0 (socket says 172.21.15.16:55578) 2024-09-12T02:13:05.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.502+0000 7f3c1a7fc640 1 -- 172.21.15.16:0/2480729200 learned_addr learned my addr 172.21.15.16:0/2480729200 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:13:05.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.502+0000 7f3c13fff640 1 --2- 172.21.15.16:0/1510267735 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f3c080a2b80 0x7f3c080a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:05.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.502+0000 7f3c1b7fe640 1 --2- 172.21.15.16:0/2480729200 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55705fa3eb40 0x55705fa3d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:05.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.502+0000 7f3c1b7fe640 1 --2- 172.21.15.16:0/1510267735 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f3c080a2b80 0x7f3c080a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:05.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.502+0000 7f3c1affd640 1 -- 172.21.15.16:0/2480729200 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55705fa3eb40 msgr2=0x55705fa3d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:05.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.502+0000 7f3c1affd640 1 --2- 172.21.15.16:0/2480729200 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55705fa3eb40 0x55705fa3d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:05.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.502+0000 7f3c1affd640 1 -- 172.21.15.16:0/2480729200 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55705fa48370 msgr2=0x55705fa4a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:05.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.502+0000 7f3c1affd640 1 --2- 172.21.15.16:0/2480729200 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55705fa48370 0x55705fa4a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:05.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.502+0000 7f3c1affd640 1 -- 172.21.15.16:0/2480729200 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55705f92f3a0 con 0x55705fa3d5d0 2024-09-12T02:13:05.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.502+0000 7f3c13fff640 1 -- 172.21.15.16:0/1510267735 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f3c040ce8b0 con 0x7f3c1c103670 2024-09-12T02:13:05.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.502+0000 7f3c1affd640 1 --2- 172.21.15.16:0/2480729200 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55705fa3d5d0 0x55705fa3da10 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f3c1400abf0 tx=0x7f3c1400f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=114d9429b33e63e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:05.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.502+0000 7f3c1b7fe640 1 --2- 172.21.15.16:0/1510267735 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f3c080a2b80 0x7f3c080a5040 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f3c0c006010 tx=0x7f3c0c005d80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:05.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.503+0000 7f3c117fa640 1 -- 172.21.15.16:0/2480729200 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c14005220 con 0x55705fa3d5d0 2024-09-12T02:13:05.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.503+0000 7f3c117fa640 1 -- 172.21.15.16:0/2480729200 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c14004c80 con 0x55705fa3d5d0 2024-09-12T02:13:05.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.503+0000 7f3c117fa640 1 -- 172.21.15.16:0/2480729200 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c14018940 con 0x55705fa3d5d0 2024-09-12T02:13:05.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.503+0000 7f3c23892740 1 -- 172.21.15.16:0/2480729200 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55705fa3d5d0 msgr2=0x55705fa3da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:05.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.503+0000 7f3c23892740 1 --2- 172.21.15.16:0/2480729200 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55705fa3d5d0 0x55705fa3da10 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f3c1400abf0 tx=0x7f3c1400f790 comp rx=0 tx=0).stop 2024-09-12T02:13:05.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.504+0000 7f3c23892740 1 -- 172.21.15.16:0/2480729200 shutdown_connections 2024-09-12T02:13:05.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.504+0000 7f3c23892740 1 --2- 172.21.15.16:0/2480729200 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55705fa48370 0x55705fa4a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:05.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.504+0000 7f3c23892740 1 --2- 172.21.15.16:0/2480729200 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55705fa3d5d0 0x55705fa3da10 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:05.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.504+0000 7f3c23892740 1 --2- 172.21.15.16:0/2480729200 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55705fa3eb40 0x55705fa3d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:05.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.504+0000 7f3c23892740 1 -- 172.21.15.16:0/2480729200 >> 172.21.15.16:0/2480729200 conn(0x55705fa06310 msgr2=0x55705fa39c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:05.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.504+0000 7f3c23892740 1 -- 172.21.15.16:0/2480729200 shutdown_connections 2024-09-12T02:13:05.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.504+0000 7f3c23892740 1 -- 172.21.15.16:0/2480729200 wait complete. 2024-09-12T02:13:05.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.504+0000 7f3c23892740 1 -- start start 2024-09-12T02:13:05.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.504+0000 7f3c23892740 1 Processor -- start 2024-09-12T02:13:05.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.504+0000 7f3c23892740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55705fa3d5d0 0x55705fa47d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:05.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.504+0000 7f3c23892740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55705fa3eb40 0x55705fa44fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:05.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.504+0000 7f3c23892740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55705fa48370 0x55705fa45530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:05.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.504+0000 7f3c23892740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55705fa4b110 con 0x55705fa3eb40 2024-09-12T02:13:05.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.504+0000 7f3c23892740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55705f95e060 con 0x55705fa3d5d0 2024-09-12T02:13:05.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.504+0000 7f3c23892740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55705fa4b290 con 0x55705fa48370 2024-09-12T02:13:05.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.504+0000 7f3c23892740 10 client.0 _ll_register_callbacks cb 0x55705f6b18c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:13:05.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.504+0000 7f3c1a7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55705fa3eb40 0x55705fa44fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:05.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.504+0000 7f3c1affd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55705fa3d5d0 0x55705fa47d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:05.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.504+0000 7f3c1a7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55705fa3eb40 0x55705fa44fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55586/0 (socket says 172.21.15.16:55586) 2024-09-12T02:13:05.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.504+0000 7f3c1a7fc640 1 -- 172.21.15.16:0/2433031294 learned_addr learned my addr 172.21.15.16:0/2433031294 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:13:05.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.505+0000 7f3c1b7fe640 1 --2- 172.21.15.16:0/2433031294 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55705fa48370 0x55705fa45530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:05.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.505+0000 7f3c1a7fc640 1 -- 172.21.15.16:0/2433031294 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55705fa48370 msgr2=0x55705fa45530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:05.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.505+0000 7f3c1a7fc640 1 --2- 172.21.15.16:0/2433031294 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55705fa48370 0x55705fa45530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:05.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.505+0000 7f3c1a7fc640 1 -- 172.21.15.16:0/2433031294 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55705fa3d5d0 msgr2=0x55705fa47d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:05.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.505+0000 7f3c1a7fc640 1 --2- 172.21.15.16:0/2433031294 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55705fa3d5d0 0x55705fa47d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:05.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.505+0000 7f3c1a7fc640 1 -- 172.21.15.16:0/2433031294 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55705f7143c0 con 0x55705fa3eb40 2024-09-12T02:13:05.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.505+0000 7f3c1b7fe640 1 --2- 172.21.15.16:0/2433031294 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55705fa48370 0x55705fa45530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:13:05.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.505+0000 7f3c1affd640 1 --2- 172.21.15.16:0/2433031294 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55705fa3d5d0 0x55705fa47d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:13:05.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.505+0000 7f3c1a7fc640 1 --2- 172.21.15.16:0/2433031294 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55705fa3eb40 0x55705fa44fb0 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7f3c0400d720 tx=0x7f3c0409ac50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c279e884994c006 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:05.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.505+0000 7f3c027fc640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:13:05.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.505+0000 7f3c027fc640 1 -- 172.21.15.16:0/2433031294 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c04020040 con 0x55705fa3eb40 2024-09-12T02:13:05.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.505+0000 7f3c027fc640 1 -- 172.21.15.16:0/2433031294 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f3c040cc4c0 con 0x55705fa3eb40 2024-09-12T02:13:05.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.505+0000 7f3c23892740 1 -- 172.21.15.16:0/2433031294 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c14004c80 con 0x55705fa3eb40 2024-09-12T02:13:05.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.505+0000 7f3c027fc640 1 -- 172.21.15.16:0/2433031294 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f3c0409b110 con 0x55705fa3eb40 2024-09-12T02:13:05.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.505+0000 7f3c23892740 10 client.19044 fetch_fsmap learned FSMap version 25 2024-09-12T02:13:05.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.505+0000 7f3c23892740 1 -- 172.21.15.16:0/2433031294 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55705fa56b40 con 0x55705fa3eb40 2024-09-12T02:13:05.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.506+0000 7f3c027fc640 1 -- 172.21.15.16:0/2433031294 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c14004c80 con 0x55705fa3eb40 2024-09-12T02:13:05.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.506+0000 7f3c027fc640 1 -- 172.21.15.16:0/2433031294 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f3c04097780 con 0x55705fa3eb40 2024-09-12T02:13:05.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.506+0000 7f3c23892740 10 client.19044 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:13:05.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.506+0000 7f3c23892740 10 client.19044 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-12T02:13:05.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.506+0000 7f3c23892740 20 client.19044 populate_metadata read hostname 'smithi016' 2024-09-12T02:13:05.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.506+0000 7f3c23892740 1 --2- 172.21.15.16:0/2433031294 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55705fa58000 0x55705fa5a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:05.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.506+0000 7f3c23892740 4 client.19044 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:13:05.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.506+0000 7f3c23892740 1 -- 172.21.15.16:0/2433031294 --> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55705f9b0860 con 0x55705fa58000 2024-09-12T02:13:05.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.506+0000 7f3c1affd640 1 --2- 172.21.15.16:0/2433031294 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55705fa58000 0x55705fa5a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:05.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.507+0000 7f3c027fc640 10 client.19044 ms_handle_connect on v2:172.21.15.184:6836/1793393087 2024-09-12T02:13:05.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.507+0000 7f3c1affd640 1 --2- 172.21.15.16:0/2433031294 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55705fa58000 0x55705fa5a3f0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=298d6a6de057bbb8 server_cookie=98eefc85804fb781 in_seq=0 out_seq=0 2024-09-12T02:13:05.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.507+0000 7f3c027fc640 1 -- 172.21.15.16:0/2433031294 <== mds.2 v2:172.21.15.184:6836/1793393087 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55705f9b0860 con 0x55705fa58000 2024-09-12T02:13:05.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.507+0000 7f3c027fc640 10 client.19044 handle_command_reply: tid=0 2024-09-12T02:13:05.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.507+0000 7f3c23892740 1 client.19044 shutdown 2024-09-12T02:13:05.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.508+0000 7f3c23892740 1 -- 172.21.15.16:0/2433031294 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55705fa3eb40 msgr2=0x55705fa44fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:05.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.508+0000 7f3c23892740 1 --2- 172.21.15.16:0/2433031294 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55705fa3eb40 0x55705fa44fb0 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7f3c0400d720 tx=0x7f3c0409ac50 comp rx=0 tx=0).stop 2024-09-12T02:13:05.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.508+0000 7f3c23892740 1 -- 172.21.15.16:0/2433031294 shutdown_connections 2024-09-12T02:13:05.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.508+0000 7f3c23892740 1 --2- 172.21.15.16:0/2433031294 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55705fa48370 0x55705fa45530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:05.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.508+0000 7f3c23892740 1 --2- 172.21.15.16:0/2433031294 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55705fa58000 0x55705fa5a3f0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:05.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.508+0000 7f3c23892740 1 --2- 172.21.15.16:0/2433031294 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55705fa3eb40 0x55705fa44fb0 unknown :-1 s=CLOSED pgs=1570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:05.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.508+0000 7f3c027fc640 0 client.19044 ms_handle_reset on v2:172.21.15.184:6836/1793393087 2024-09-12T02:13:05.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.508+0000 7f3c23892740 1 --2- 172.21.15.16:0/2433031294 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55705fa3d5d0 0x55705fa47d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:05.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.508+0000 7f3c23892740 1 -- 172.21.15.16:0/2433031294 >> 172.21.15.16:0/2433031294 conn(0x55705fa06310 msgr2=0x55705fa3f960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:05.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.508+0000 7f3c23892740 1 -- 172.21.15.16:0/2433031294 shutdown_connections 2024-09-12T02:13:05.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.508+0000 7f3c23892740 1 -- 172.21.15.16:0/2433031294 wait complete. 2024-09-12T02:13:05.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.508+0000 7f3c23892740 20 client.19044 trim_cache size 0 max 16384 2024-09-12T02:13:05.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.521+0000 7f3c23892740 1 Processor -- start 2024-09-12T02:13:05.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.521+0000 7f3c23892740 1 -- start start 2024-09-12T02:13:05.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.521+0000 7f3c23892740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55705fa3d5d0 0x55705fa39df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:05.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.521+0000 7f3c23892740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55705fa3eb40 0x55705fa543b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:05.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.521+0000 7f3c23892740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55705fa48370 0x55705fa5fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:05.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.521+0000 7f3c23892740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3c1c1afd20 con 0x55705fa3d5d0 2024-09-12T02:13:05.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.521+0000 7f3c23892740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3c1c1ad5f0 con 0x55705fa3eb40 2024-09-12T02:13:05.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.521+0000 7f3c23892740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f3c1c1ab440 con 0x55705fa48370 2024-09-12T02:13:05.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.521+0000 7f3c1affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55705fa3d5d0 0x55705fa39df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:05.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.521+0000 7f3c1affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55705fa3d5d0 0x55705fa39df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55592/0 (socket says 172.21.15.16:55592) 2024-09-12T02:13:05.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.521+0000 7f3c1affd640 1 -- 172.21.15.16:0/1111235807 learned_addr learned my addr 172.21.15.16:0/1111235807 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:13:05.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.522+0000 7f3c1b7fe640 1 --2- 172.21.15.16:0/1111235807 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55705fa48370 0x55705fa5fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:05.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.522+0000 7f3c1a7fc640 1 --2- 172.21.15.16:0/1111235807 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55705fa3eb40 0x55705fa543b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:05.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.522+0000 7f3c1affd640 1 -- 172.21.15.16:0/1111235807 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55705fa48370 msgr2=0x55705fa5fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:05.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.522+0000 7f3c1affd640 1 --2- 172.21.15.16:0/1111235807 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55705fa48370 0x55705fa5fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:05.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.522+0000 7f3c1affd640 1 -- 172.21.15.16:0/1111235807 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55705fa3eb40 msgr2=0x55705fa543b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:05.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.522+0000 7f3c1affd640 1 --2- 172.21.15.16:0/1111235807 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55705fa3eb40 0x55705fa543b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:05.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.522+0000 7f3c1affd640 1 -- 172.21.15.16:0/1111235807 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55705fa48070 con 0x55705fa3d5d0 2024-09-12T02:13:05.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.522+0000 7f3c1a7fc640 1 --2- 172.21.15.16:0/1111235807 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55705fa3eb40 0x55705fa543b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:13:05.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.522+0000 7f3c1b7fe640 1 --2- 172.21.15.16:0/1111235807 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55705fa48370 0x55705fa5fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:13:05.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.522+0000 7f3c1affd640 1 --2- 172.21.15.16:0/1111235807 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55705fa3d5d0 0x55705fa39df0 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f3c1400abf0 tx=0x7f3c140057e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ed0d5a2a5b24682 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:05.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.522+0000 7f3c10ff9640 1 -- 172.21.15.16:0/1111235807 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c14033070 con 0x55705fa3d5d0 2024-09-12T02:13:05.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.522+0000 7f3c10ff9640 1 -- 172.21.15.16:0/1111235807 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c14005ba0 con 0x55705fa3d5d0 2024-09-12T02:13:05.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.522+0000 7f3c10ff9640 1 -- 172.21.15.16:0/1111235807 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c1402e9e0 con 0x55705fa3d5d0 2024-09-12T02:13:05.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.523+0000 7f3c23892740 1 -- 172.21.15.16:0/1111235807 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55705fa3d5d0 msgr2=0x55705fa39df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:05.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.523+0000 7f3c23892740 1 --2- 172.21.15.16:0/1111235807 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55705fa3d5d0 0x55705fa39df0 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f3c1400abf0 tx=0x7f3c140057e0 comp rx=0 tx=0).stop 2024-09-12T02:13:05.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.523+0000 7f3c23892740 1 -- 172.21.15.16:0/1111235807 shutdown_connections 2024-09-12T02:13:05.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.523+0000 7f3c23892740 1 --2- 172.21.15.16:0/1111235807 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55705fa48370 0x55705fa5fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:05.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.523+0000 7f3c23892740 1 --2- 172.21.15.16:0/1111235807 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55705fa3eb40 0x55705fa543b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:05.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.523+0000 7f3c23892740 1 --2- 172.21.15.16:0/1111235807 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55705fa3d5d0 0x55705fa39df0 unknown :-1 s=CLOSED pgs=1571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:05.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.523+0000 7f3c23892740 1 -- 172.21.15.16:0/1111235807 >> 172.21.15.16:0/1111235807 conn(0x55705fa06310 msgr2=0x55705fa396e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:05.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.523+0000 7f3c23892740 1 -- 172.21.15.16:0/1111235807 shutdown_connections 2024-09-12T02:13:05.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.523+0000 7f3c23892740 1 -- 172.21.15.16:0/1111235807 wait complete. 2024-09-12T02:13:05.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.523+0000 7f3c23892740 1 -- start start 2024-09-12T02:13:05.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.523+0000 7f3c23892740 1 Processor -- start 2024-09-12T02:13:05.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.524+0000 7f3c23892740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55705fa3d5d0 0x55705fa50de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:05.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.524+0000 7f3c23892740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55705fa3eb40 0x55705fa4c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:05.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.524+0000 7f3c23892740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55705fa48370 0x55705fa4c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:05.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.524+0000 7f3c23892740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3c1c1abab0 con 0x55705fa48370 2024-09-12T02:13:05.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.524+0000 7f3c23892740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3c1c1acf20 con 0x55705fa3d5d0 2024-09-12T02:13:05.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.524+0000 7f3c23892740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f3c1c1ac180 con 0x55705fa3eb40 2024-09-12T02:13:05.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.524+0000 7f3c1b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55705fa48370 0x55705fa4c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:05.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.524+0000 7f3c23892740 10 client.0 _ll_register_callbacks cb 0x7f3c040057a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:13:05.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.524+0000 7f3c1b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55705fa48370 0x55705fa4c8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55604/0 (socket says 172.21.15.16:55604) 2024-09-12T02:13:05.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.524+0000 7f3c1b7fe640 1 -- 172.21.15.16:0/1173855370 learned_addr learned my addr 172.21.15.16:0/1173855370 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:13:05.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.524+0000 7f3c1affd640 1 --2- 172.21.15.16:0/1173855370 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55705fa3d5d0 0x55705fa50de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:05.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.524+0000 7f3c1a7fc640 1 --2- 172.21.15.16:0/1173855370 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55705fa3eb40 0x55705fa4c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:05.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.524+0000 7f3c1b7fe640 1 -- 172.21.15.16:0/1173855370 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55705fa3eb40 msgr2=0x55705fa4c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:05.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.524+0000 7f3c1b7fe640 1 --2- 172.21.15.16:0/1173855370 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55705fa3eb40 0x55705fa4c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:05.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.524+0000 7f3c1b7fe640 1 -- 172.21.15.16:0/1173855370 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55705fa3d5d0 msgr2=0x55705fa50de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:05.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.524+0000 7f3c1b7fe640 1 --2- 172.21.15.16:0/1173855370 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55705fa3d5d0 0x55705fa50de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:05.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.524+0000 7f3c1b7fe640 1 -- 172.21.15.16:0/1173855370 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55705fac1bd0 con 0x55705fa48370 2024-09-12T02:13:05.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.524+0000 7f3c1a7fc640 1 --2- 172.21.15.16:0/1173855370 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55705fa3eb40 0x55705fa4c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:13:05.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.524+0000 7f3c1b7fe640 1 --2- 172.21.15.16:0/1173855370 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55705fa48370 0x55705fa4c8f0 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f3c0c009ca0 tx=0x7f3c0c0095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2def8532d8664c29 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:05.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.524+0000 7f3c027fc640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:13:05.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.525+0000 7f3c027fc640 1 -- 172.21.15.16:0/1173855370 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c0c04e070 con 0x55705fa48370 2024-09-12T02:13:05.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.525+0000 7f3c027fc640 1 -- 172.21.15.16:0/1173855370 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f3c0c004790 con 0x55705fa48370 2024-09-12T02:13:05.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.525+0000 7f3c23892740 1 -- 172.21.15.16:0/1173855370 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55705f9aad70 con 0x55705fa48370 2024-09-12T02:13:05.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.525+0000 7f3c027fc640 1 -- 172.21.15.16:0/1173855370 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f3c0c034040 con 0x55705fa48370 2024-09-12T02:13:05.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.525+0000 7f3c23892740 10 client.19050 fetch_fsmap learned FSMap version 25 2024-09-12T02:13:05.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.525+0000 7f3c23892740 1 -- 172.21.15.16:0/1173855370 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55705fac2cc0 con 0x55705fa48370 2024-09-12T02:13:05.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.525+0000 7f3c027fc640 1 -- 172.21.15.16:0/1173855370 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55705f9aad70 con 0x55705fa48370 2024-09-12T02:13:05.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.525+0000 7f3c027fc640 1 -- 172.21.15.16:0/1173855370 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f3c0c03a430 con 0x55705fa48370 2024-09-12T02:13:05.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.525+0000 7f3c23892740 10 client.19050 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:13:05.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.525+0000 7f3c23892740 10 client.19050 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-12T02:13:05.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.525+0000 7f3c23892740 20 client.19050 populate_metadata read hostname 'smithi016' 2024-09-12T02:13:05.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.525+0000 7f3c23892740 1 --2- 172.21.15.16:0/1173855370 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55705fa58000 0x55705fac5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:05.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.525+0000 7f3c23892740 4 client.19050 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T02:13:05.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.525+0000 7f3c23892740 1 -- 172.21.15.16:0/1173855370 --> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55705f927170 con 0x55705fa58000 2024-09-12T02:13:05.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.525+0000 7f3c1affd640 1 --2- 172.21.15.16:0/1173855370 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55705fa58000 0x55705fac5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:05.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.526+0000 7f3c1affd640 1 --2- 172.21.15.16:0/1173855370 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55705fa58000 0x55705fac5450 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=723f9a4e125a41fb server_cookie=8dbdcdb59923b3e6 in_seq=0 out_seq=0 2024-09-12T02:13:05.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.526+0000 7f3c027fc640 10 client.19050 ms_handle_connect on v2:172.21.15.184:6836/1793393087 2024-09-12T02:13:05.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.526+0000 7f3c027fc640 1 -- 172.21.15.16:0/1173855370 <== mds.2 v2:172.21.15.184:6836/1793393087 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55705f927170 con 0x55705fa58000 2024-09-12T02:13:05.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.526+0000 7f3c027fc640 10 client.19050 handle_command_reply: tid=0 2024-09-12T02:13:05.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.526+0000 7f3c23892740 1 client.19050 shutdown 2024-09-12T02:13:05.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.526+0000 7f3c23892740 1 -- 172.21.15.16:0/1173855370 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55705fa48370 msgr2=0x55705fa4c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:05.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.526+0000 7f3c23892740 1 --2- 172.21.15.16:0/1173855370 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55705fa48370 0x55705fa4c8f0 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f3c0c009ca0 tx=0x7f3c0c0095c0 comp rx=0 tx=0).stop 2024-09-12T02:13:05.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.526+0000 7f3c23892740 1 -- 172.21.15.16:0/1173855370 shutdown_connections 2024-09-12T02:13:05.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.526+0000 7f3c23892740 1 --2- 172.21.15.16:0/1173855370 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55705fa58000 0x55705fac5450 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:05.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.526+0000 7f3c23892740 1 --2- 172.21.15.16:0/1173855370 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55705fa48370 0x55705fa4c8f0 unknown :-1 s=CLOSED pgs=1572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:05.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.526+0000 7f3c23892740 1 --2- 172.21.15.16:0/1173855370 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55705fa3eb40 0x55705fa4c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:05.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.526+0000 7f3c23892740 1 --2- 172.21.15.16:0/1173855370 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55705fa3d5d0 0x55705fa50de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:05.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.526+0000 7f3c23892740 1 -- 172.21.15.16:0/1173855370 >> 172.21.15.16:0/1173855370 conn(0x55705fa06310 msgr2=0x55705fa46810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:05.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.526+0000 7f3c027fc640 0 client.19050 ms_handle_reset on v2:172.21.15.184:6836/1793393087 2024-09-12T02:13:05.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.526+0000 7f3c23892740 1 -- 172.21.15.16:0/1173855370 shutdown_connections 2024-09-12T02:13:05.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.526+0000 7f3c23892740 1 -- 172.21.15.16:0/1173855370 wait complete. 2024-09-12T02:13:05.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.526+0000 7f3c23892740 20 client.19050 trim_cache size 0 max 16384 2024-09-12T02:13:05.533 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T02:13:05.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.527+0000 7f3c214b9640 1 -- 172.21.15.16:0/1510267735 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f3c080a2b80 msgr2=0x7f3c080a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:05.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.527+0000 7f3c214b9640 1 --2- 172.21.15.16:0/1510267735 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f3c080a2b80 0x7f3c080a5040 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f3c0c006010 tx=0x7f3c0c005d80 comp rx=0 tx=0).stop 2024-09-12T02:13:05.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.527+0000 7f3c214b9640 1 -- 172.21.15.16:0/1510267735 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c1c103670 msgr2=0x7f3c1c1988f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:05.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.527+0000 7f3c214b9640 1 --2- 172.21.15.16:0/1510267735 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c1c103670 0x7f3c1c1988f0 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f3c0400d750 tx=0x7f3c0400db50 comp rx=0 tx=0).stop 2024-09-12T02:13:05.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.527+0000 7f3c214b9640 1 -- 172.21.15.16:0/1510267735 shutdown_connections 2024-09-12T02:13:05.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.527+0000 7f3c214b9640 1 --2- 172.21.15.16:0/1510267735 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f3c080a2b80 0x7f3c080a5040 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:05.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.527+0000 7f3c214b9640 1 --2- 172.21.15.16:0/1510267735 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c1c10c180 0x7f3c1c19cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:05.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.527+0000 7f3c214b9640 1 --2- 172.21.15.16:0/1510267735 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c1c103670 0x7f3c1c1988f0 unknown :-1 s=CLOSED pgs=1569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:05.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.527+0000 7f3c214b9640 1 --2- 172.21.15.16:0/1510267735 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3c1c102d50 0x7f3c1c1983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:05.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.527+0000 7f3c214b9640 1 -- 172.21.15.16:0/1510267735 >> 172.21.15.16:0/1510267735 conn(0x7f3c1c0fc7d0 msgr2=0x7f3c1c0fd3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:05.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.527+0000 7f3c214b9640 1 -- 172.21.15.16:0/1510267735 shutdown_connections 2024-09-12T02:13:05.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:05.527+0000 7f3c214b9640 1 -- 172.21.15.16:0/1510267735 wait complete. 2024-09-12T02:13:06.183 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:13:06 smithi016 ceph-mon[30947]: pgmap v670: 97 pgs: 97 active+clean; 1.8 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-09-12T02:13:06.241 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-12T02:13:06.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:13:06 smithi184 ceph-mon[35593]: pgmap v670: 97 pgs: 97 active+clean; 1.8 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-09-12T02:13:06.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:13:06 smithi033 ceph-mon[35332]: pgmap v670: 97 pgs: 97 active+clean; 1.8 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-09-12T02:13:06.501 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:13:06.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.961+0000 7f5021770640 1 -- 172.21.15.16:0/1728485160 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f501c106f10 msgr2=0x7f501c10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:06.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.961+0000 7f5021770640 1 --2- 172.21.15.16:0/1728485160 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f501c106f10 0x7f501c10ea90 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7f5014009ca0 tx=0x7f501402a8c0 comp rx=0 tx=0).stop 2024-09-12T02:13:06.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.961+0000 7f5021770640 1 -- 172.21.15.16:0/1728485160 shutdown_connections 2024-09-12T02:13:06.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.961+0000 7f5021770640 1 --2- 172.21.15.16:0/1728485160 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f501c106f10 0x7f501c10ea90 unknown :-1 s=CLOSED pgs=1573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:06.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.961+0000 7f5021770640 1 --2- 172.21.15.16:0/1728485160 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f501c102790 0x7f501c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:06.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.961+0000 7f5021770640 1 --2- 172.21.15.16:0/1728485160 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f501c101de0 0x7f501c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:06.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.961+0000 7f5021770640 1 -- 172.21.15.16:0/1728485160 >> 172.21.15.16:0/1728485160 conn(0x7f501c0fd8c0 msgr2=0x7f501c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:06.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.961+0000 7f5021770640 1 -- 172.21.15.16:0/1728485160 shutdown_connections 2024-09-12T02:13:06.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.961+0000 7f5021770640 1 -- 172.21.15.16:0/1728485160 wait complete. 2024-09-12T02:13:06.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.962+0000 7f5021770640 1 Processor -- start 2024-09-12T02:13:06.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.962+0000 7f5021770640 1 -- start start 2024-09-12T02:13:06.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.962+0000 7f5021770640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f501c101de0 0x7f501c198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:06.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.962+0000 7f5021770640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f501c102790 0x7f501c198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:06.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.962+0000 7f5021770640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f501c106f10 0x7f501c19ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:06.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.962+0000 7f5021770640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f501c111f90 con 0x7f501c106f10 2024-09-12T02:13:06.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.962+0000 7f5021770640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f501c111e10 con 0x7f501c101de0 2024-09-12T02:13:06.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.962+0000 7f5021770640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f501c112110 con 0x7f501c102790 2024-09-12T02:13:06.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.962+0000 7f501b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f501c106f10 0x7f501c19ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:06.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.962+0000 7f501b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f501c106f10 0x7f501c19ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55614/0 (socket says 172.21.15.16:55614) 2024-09-12T02:13:06.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.962+0000 7f501b7fe640 1 -- 172.21.15.16:0/1150185429 learned_addr learned my addr 172.21.15.16:0/1150185429 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:13:06.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.962+0000 7f5013fff640 1 --2- 172.21.15.16:0/1150185429 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f501c102790 0x7f501c198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:06.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.962+0000 7f501affd640 1 --2- 172.21.15.16:0/1150185429 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f501c101de0 0x7f501c198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:06.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.962+0000 7f501b7fe640 1 -- 172.21.15.16:0/1150185429 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f501c102790 msgr2=0x7f501c198a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:06.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.962+0000 7f501b7fe640 1 --2- 172.21.15.16:0/1150185429 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f501c102790 0x7f501c198a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:06.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.962+0000 7f501b7fe640 1 -- 172.21.15.16:0/1150185429 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f501c101de0 msgr2=0x7f501c198550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:06.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.962+0000 7f501b7fe640 1 --2- 172.21.15.16:0/1150185429 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f501c101de0 0x7f501c198550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:06.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.962+0000 7f501b7fe640 1 -- 172.21.15.16:0/1150185429 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f501c19d500 con 0x7f501c106f10 2024-09-12T02:13:06.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.963+0000 7f501affd640 1 --2- 172.21.15.16:0/1150185429 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f501c101de0 0x7f501c198550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:13:06.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.963+0000 7f501b7fe640 1 --2- 172.21.15.16:0/1150185429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f501c106f10 0x7f501c19ce20 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f5014031a20 tx=0x7f5014031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:06.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.963+0000 7f5018ff9640 1 -- 172.21.15.16:0/1150185429 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5014003c30 con 0x7f501c106f10 2024-09-12T02:13:06.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.963+0000 7f5018ff9640 1 -- 172.21.15.16:0/1150185429 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5014003dd0 con 0x7f501c106f10 2024-09-12T02:13:06.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.963+0000 7f5021770640 1 -- 172.21.15.16:0/1150185429 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f501c19d790 con 0x7f501c106f10 2024-09-12T02:13:06.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.963+0000 7f5018ff9640 1 -- 172.21.15.16:0/1150185429 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f501402fc40 con 0x7f501c106f10 2024-09-12T02:13:06.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.963+0000 7f5021770640 1 -- 172.21.15.16:0/1150185429 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f501c1a50d0 con 0x7f501c106f10 2024-09-12T02:13:06.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.965+0000 7f5018ff9640 1 -- 172.21.15.16:0/1150185429 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5014004050 con 0x7f501c106f10 2024-09-12T02:13:06.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.966+0000 7f5023b49740 1 Processor -- start 2024-09-12T02:13:06.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.966+0000 7f5023b49740 1 -- start start 2024-09-12T02:13:06.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.966+0000 7f5023b49740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ee1ed03b40 0x55ee1ed02090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:06.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.966+0000 7f5023b49740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ee1ed025d0 0x55ee1ed02a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:06.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.966+0000 7f5023b49740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee1ed11780 0x55ee1ed13b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:06.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.966+0000 7f5023b49740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55ee1ec682f0 con 0x55ee1ed11780 2024-09-12T02:13:06.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.966+0000 7f5023b49740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55ee1ec0fcf0 con 0x55ee1ed025d0 2024-09-12T02:13:06.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.966+0000 7f5023b49740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55ee1ec642b0 con 0x55ee1ed03b40 2024-09-12T02:13:06.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.966+0000 7f5013fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee1ed11780 0x55ee1ed13b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:06.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.966+0000 7f5013fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee1ed11780 0x55ee1ed13b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55628/0 (socket says 172.21.15.16:55628) 2024-09-12T02:13:06.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.966+0000 7f5013fff640 1 -- 172.21.15.16:0/4001813182 learned_addr learned my addr 172.21.15.16:0/4001813182 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:13:06.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.966+0000 7f501affd640 1 --2- 172.21.15.16:0/4001813182 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ee1ed025d0 0x55ee1ed02a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:06.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.966+0000 7f5013fff640 1 --2- 172.21.15.16:0/4001813182 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ee1ed03b40 0x55ee1ed02090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:06.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.966+0000 7f5018ff9640 1 --2- 172.21.15.16:0/1150185429 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f50080a2b80 0x7f50080a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:06.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.967+0000 7f5013fff640 1 -- 172.21.15.16:0/4001813182 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ee1ed03b40 msgr2=0x55ee1ed02090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:06.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.967+0000 7f5013fff640 1 --2- 172.21.15.16:0/4001813182 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ee1ed03b40 0x55ee1ed02090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:06.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.967+0000 7f5013fff640 1 -- 172.21.15.16:0/4001813182 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ee1ed025d0 msgr2=0x55ee1ed02a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:06.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.967+0000 7f5013fff640 1 --2- 172.21.15.16:0/4001813182 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ee1ed025d0 0x55ee1ed02a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:06.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.967+0000 7f5013fff640 1 -- 172.21.15.16:0/4001813182 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee1ebf43a0 con 0x55ee1ed11780 2024-09-12T02:13:06.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.967+0000 7f501b7fe640 1 --2- 172.21.15.16:0/1150185429 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f50080a2b80 0x7f50080a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:06.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.967+0000 7f5013fff640 1 --2- 172.21.15.16:0/4001813182 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee1ed11780 0x55ee1ed13b70 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f500400ad80 tx=0x7f500400a490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fa5785532699fdd server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:06.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.967+0000 7f5018ff9640 1 -- 172.21.15.16:0/1150185429 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f50140f0330 con 0x7f501c106f10 2024-09-12T02:13:06.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.967+0000 7f50117fa640 1 -- 172.21.15.16:0/4001813182 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f500401b070 con 0x55ee1ed11780 2024-09-12T02:13:06.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.967+0000 7f50117fa640 1 -- 172.21.15.16:0/4001813182 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f500400fdf0 con 0x55ee1ed11780 2024-09-12T02:13:06.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.967+0000 7f50117fa640 1 -- 172.21.15.16:0/4001813182 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50040146e0 con 0x55ee1ed11780 2024-09-12T02:13:06.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.968+0000 7f5023b49740 1 -- 172.21.15.16:0/4001813182 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee1ed11780 msgr2=0x55ee1ed13b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:06.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.968+0000 7f501b7fe640 1 --2- 172.21.15.16:0/1150185429 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f50080a2b80 0x7f50080a5040 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f50140b95f0 tx=0x7f501408fce0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:06.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.968+0000 7f5023b49740 1 --2- 172.21.15.16:0/4001813182 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee1ed11780 0x55ee1ed13b70 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f500400ad80 tx=0x7f500400a490 comp rx=0 tx=0).stop 2024-09-12T02:13:06.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.968+0000 7f5023b49740 1 -- 172.21.15.16:0/4001813182 shutdown_connections 2024-09-12T02:13:06.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.968+0000 7f5023b49740 1 --2- 172.21.15.16:0/4001813182 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee1ed11780 0x55ee1ed13b70 unknown :-1 s=CLOSED pgs=1575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:06.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.968+0000 7f5023b49740 1 --2- 172.21.15.16:0/4001813182 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ee1ed025d0 0x55ee1ed02a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:06.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.968+0000 7f5023b49740 1 --2- 172.21.15.16:0/4001813182 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ee1ed03b40 0x55ee1ed02090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:06.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.968+0000 7f5023b49740 1 -- 172.21.15.16:0/4001813182 >> 172.21.15.16:0/4001813182 conn(0x55ee1eccb310 msgr2=0x55ee1ecfec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:06.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.968+0000 7f5023b49740 1 -- 172.21.15.16:0/4001813182 shutdown_connections 2024-09-12T02:13:06.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.968+0000 7f5023b49740 1 -- 172.21.15.16:0/4001813182 wait complete. 2024-09-12T02:13:06.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.968+0000 7f5023b49740 1 -- start start 2024-09-12T02:13:06.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.968+0000 7f5023b49740 1 Processor -- start 2024-09-12T02:13:06.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.969+0000 7f5023b49740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ee1ed025d0 0x55ee1ed111d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:06.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.969+0000 7f5023b49740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee1ed03b40 0x55ee1ed0e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:06.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.969+0000 7f5023b49740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ee1ed11780 0x55ee1ed0ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:06.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.969+0000 7f5023b49740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55ee1ed14480 con 0x55ee1ed03b40 2024-09-12T02:13:06.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.969+0000 7f5023b49740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55ee1ec23060 con 0x55ee1ed11780 2024-09-12T02:13:06.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.969+0000 7f5023b49740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55ee1ed14600 con 0x55ee1ed025d0 2024-09-12T02:13:06.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.969+0000 7f5023b49740 10 client.0 _ll_register_callbacks cb 0x55ee1e9768c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:13:06.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.969+0000 7f501affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee1ed03b40 0x55ee1ed0e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:06.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.969+0000 7f501affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee1ed03b40 0x55ee1ed0e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55630/0 (socket says 172.21.15.16:55630) 2024-09-12T02:13:06.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.969+0000 7f501affd640 1 -- 172.21.15.16:0/4046657405 learned_addr learned my addr 172.21.15.16:0/4046657405 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:13:06.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.969+0000 7f5013fff640 1 --2- 172.21.15.16:0/4046657405 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ee1ed11780 0x55ee1ed0ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:06.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.969+0000 7f5013fff640 1 --2- 172.21.15.16:0/4046657405 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ee1ed025d0 0x55ee1ed111d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:06.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.969+0000 7f501affd640 1 -- 172.21.15.16:0/4046657405 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ee1ed025d0 msgr2=0x55ee1ed111d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:06.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.969+0000 7f501affd640 1 --2- 172.21.15.16:0/4046657405 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ee1ed025d0 0x55ee1ed111d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:06.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.969+0000 7f501affd640 1 -- 172.21.15.16:0/4046657405 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ee1ed11780 msgr2=0x55ee1ed0ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:06.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.969+0000 7f501affd640 1 --2- 172.21.15.16:0/4046657405 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ee1ed11780 0x55ee1ed0ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:06.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.969+0000 7f501affd640 1 -- 172.21.15.16:0/4046657405 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee1e9d93c0 con 0x55ee1ed03b40 2024-09-12T02:13:06.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.969+0000 7f5013fff640 1 --2- 172.21.15.16:0/4046657405 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ee1ed025d0 0x55ee1ed111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:13:06.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.969+0000 7f501affd640 1 --2- 172.21.15.16:0/4046657405 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee1ed03b40 0x55ee1ed0e480 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f500c012670 tx=0x7f500c012b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a14af9edfd9dfd6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:06.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.969+0000 7f50027fc640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:13:06.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.969+0000 7f50027fc640 1 -- 172.21.15.16:0/4046657405 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f500c01f060 con 0x55ee1ed03b40 2024-09-12T02:13:06.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.969+0000 7f50027fc640 1 -- 172.21.15.16:0/4046657405 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f500c010d60 con 0x55ee1ed03b40 2024-09-12T02:13:06.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.969+0000 7f5023b49740 1 -- 172.21.15.16:0/4046657405 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f500400fdf0 con 0x55ee1ed03b40 2024-09-12T02:13:06.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.970+0000 7f50027fc640 1 -- 172.21.15.16:0/4046657405 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f500c01a3f0 con 0x55ee1ed03b40 2024-09-12T02:13:06.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.970+0000 7f5023b49740 10 client.19062 fetch_fsmap learned FSMap version 25 2024-09-12T02:13:06.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.970+0000 7f5023b49740 1 -- 172.21.15.16:0/4046657405 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ee1ed1fff0 con 0x55ee1ed03b40 2024-09-12T02:13:06.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.970+0000 7f50027fc640 1 -- 172.21.15.16:0/4046657405 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f500400fdf0 con 0x55ee1ed03b40 2024-09-12T02:13:06.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.970+0000 7f50027fc640 1 -- 172.21.15.16:0/4046657405 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f500c024070 con 0x55ee1ed03b40 2024-09-12T02:13:06.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.970+0000 7f5023b49740 10 client.19062 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:13:06.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.970+0000 7f5023b49740 10 client.19062 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T02:13:06.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.970+0000 7f5023b49740 20 client.19062 populate_metadata read hostname 'smithi016' 2024-09-12T02:13:06.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.970+0000 7f5023b49740 1 --2- 172.21.15.16:0/4046657405 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x55ee1ed21500 0x55ee1ed23920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:06.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.970+0000 7f5023b49740 4 client.19062 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:13:06.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.970+0000 7f5023b49740 1 -- 172.21.15.16:0/4046657405 --> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ee1ec75860 con 0x55ee1ed21500 2024-09-12T02:13:06.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.970+0000 7f5013fff640 1 --2- 172.21.15.16:0/4046657405 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x55ee1ed21500 0x55ee1ed23920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:06.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.971+0000 7f5013fff640 1 --2- 172.21.15.16:0/4046657405 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x55ee1ed21500 0x55ee1ed23920 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=93a902ea2b5c40f server_cookie=7ee46a07904a88b8 in_seq=0 out_seq=0 2024-09-12T02:13:06.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.971+0000 7f50027fc640 10 client.19062 ms_handle_connect on v2:172.21.15.184:6838/4203347941 2024-09-12T02:13:06.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.972+0000 7f50027fc640 1 -- 172.21.15.16:0/4046657405 <== mds.1 v2:172.21.15.184:6838/4203347941 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ee1ec75860 con 0x55ee1ed21500 2024-09-12T02:13:06.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.972+0000 7f50027fc640 10 client.19062 handle_command_reply: tid=0 2024-09-12T02:13:06.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.972+0000 7f5023b49740 1 client.19062 shutdown 2024-09-12T02:13:06.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.972+0000 7f5023b49740 1 -- 172.21.15.16:0/4046657405 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee1ed03b40 msgr2=0x55ee1ed0e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:06.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.972+0000 7f5023b49740 1 --2- 172.21.15.16:0/4046657405 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee1ed03b40 0x55ee1ed0e480 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f500c012670 tx=0x7f500c012b30 comp rx=0 tx=0).stop 2024-09-12T02:13:06.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.972+0000 7f5023b49740 1 -- 172.21.15.16:0/4046657405 shutdown_connections 2024-09-12T02:13:06.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.972+0000 7f5023b49740 1 --2- 172.21.15.16:0/4046657405 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x55ee1ed21500 0x55ee1ed23920 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:06.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.972+0000 7f5023b49740 1 --2- 172.21.15.16:0/4046657405 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ee1ed11780 0x55ee1ed0ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:06.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.972+0000 7f5023b49740 1 --2- 172.21.15.16:0/4046657405 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee1ed03b40 0x55ee1ed0e480 unknown :-1 s=CLOSED pgs=1576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:06.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.972+0000 7f5023b49740 1 --2- 172.21.15.16:0/4046657405 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ee1ed025d0 0x55ee1ed111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:06.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.972+0000 7f50027fc640 0 client.19062 ms_handle_reset on v2:172.21.15.184:6838/4203347941 2024-09-12T02:13:06.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.972+0000 7f5023b49740 1 -- 172.21.15.16:0/4046657405 >> 172.21.15.16:0/4046657405 conn(0x55ee1eccb310 msgr2=0x55ee1ed04960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:06.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.972+0000 7f5023b49740 1 -- 172.21.15.16:0/4046657405 shutdown_connections 2024-09-12T02:13:06.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.972+0000 7f5023b49740 1 -- 172.21.15.16:0/4046657405 wait complete. 2024-09-12T02:13:06.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.972+0000 7f5023b49740 20 client.19062 trim_cache size 0 max 16384 2024-09-12T02:13:06.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.985+0000 7f5023b49740 1 Processor -- start 2024-09-12T02:13:06.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.985+0000 7f5023b49740 1 -- start start 2024-09-12T02:13:06.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.985+0000 7f5023b49740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee1ed025d0 0x55ee1ecfed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:06.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.985+0000 7f5023b49740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ee1ed03b40 0x55ee1ed03300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:06.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.985+0000 7f5023b49740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ee1ed11780 0x55ee1ed0d460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:06.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.985+0000 7f5023b49740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f501c1adb30 con 0x55ee1ed025d0 2024-09-12T02:13:06.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.985+0000 7f5023b49740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f501c1ab400 con 0x55ee1ed11780 2024-09-12T02:13:06.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.985+0000 7f5023b49740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f501c1a92b0 con 0x55ee1ed03b40 2024-09-12T02:13:06.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.985+0000 7f5013fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee1ed025d0 0x55ee1ecfed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:06.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.985+0000 7f5013fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee1ed025d0 0x55ee1ecfed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55632/0 (socket says 172.21.15.16:55632) 2024-09-12T02:13:06.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.985+0000 7f5013fff640 1 -- 172.21.15.16:0/2160977415 learned_addr learned my addr 172.21.15.16:0/2160977415 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:13:06.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.986+0000 7f501affd640 1 --2- 172.21.15.16:0/2160977415 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ee1ed03b40 0x55ee1ed03300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:06.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.986+0000 7f5013fff640 1 -- 172.21.15.16:0/2160977415 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ee1ed03b40 msgr2=0x55ee1ed03300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:06.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.986+0000 7f5013fff640 1 --2- 172.21.15.16:0/2160977415 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ee1ed03b40 0x55ee1ed03300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:06.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.986+0000 7f5013fff640 1 -- 172.21.15.16:0/2160977415 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ee1ed11780 msgr2=0x55ee1ed0d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:06.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.986+0000 7f5013fff640 1 --2- 172.21.15.16:0/2160977415 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ee1ed11780 0x55ee1ed0d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:06.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.986+0000 7f5013fff640 1 -- 172.21.15.16:0/2160977415 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee1ed0ef80 con 0x55ee1ed025d0 2024-09-12T02:13:06.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.986+0000 7f501affd640 1 --2- 172.21.15.16:0/2160977415 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ee1ed03b40 0x55ee1ed03300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:13:06.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.986+0000 7f5013fff640 1 --2- 172.21.15.16:0/2160977415 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee1ed025d0 0x55ee1ecfed70 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7f5004019040 tx=0x7f500400aba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dba7f8c072dbdaf2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:06.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.986+0000 7f5010ff9640 1 -- 172.21.15.16:0/2160977415 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f500401b050 con 0x55ee1ed025d0 2024-09-12T02:13:06.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.986+0000 7f5010ff9640 1 -- 172.21.15.16:0/2160977415 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f500401c040 con 0x55ee1ed025d0 2024-09-12T02:13:06.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.986+0000 7f5010ff9640 1 -- 172.21.15.16:0/2160977415 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f500402fa60 con 0x55ee1ed025d0 2024-09-12T02:13:06.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.987+0000 7f5023b49740 1 -- 172.21.15.16:0/2160977415 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee1ed025d0 msgr2=0x55ee1ecfed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:06.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.987+0000 7f5023b49740 1 --2- 172.21.15.16:0/2160977415 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee1ed025d0 0x55ee1ecfed70 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7f5004019040 tx=0x7f500400aba0 comp rx=0 tx=0).stop 2024-09-12T02:13:06.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.987+0000 7f5023b49740 1 -- 172.21.15.16:0/2160977415 shutdown_connections 2024-09-12T02:13:06.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.987+0000 7f5023b49740 1 --2- 172.21.15.16:0/2160977415 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ee1ed11780 0x55ee1ed0d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:06.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.987+0000 7f5023b49740 1 --2- 172.21.15.16:0/2160977415 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ee1ed03b40 0x55ee1ed03300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:06.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.987+0000 7f5023b49740 1 --2- 172.21.15.16:0/2160977415 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee1ed025d0 0x55ee1ecfed70 unknown :-1 s=CLOSED pgs=1577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:06.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.987+0000 7f5023b49740 1 -- 172.21.15.16:0/2160977415 >> 172.21.15.16:0/2160977415 conn(0x55ee1eccb310 msgr2=0x55ee1ed0cd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:06.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.987+0000 7f5023b49740 1 -- 172.21.15.16:0/2160977415 shutdown_connections 2024-09-12T02:13:06.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.987+0000 7f5023b49740 1 -- 172.21.15.16:0/2160977415 wait complete. 2024-09-12T02:13:06.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.987+0000 7f5023b49740 1 -- start start 2024-09-12T02:13:06.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.987+0000 7f5023b49740 1 Processor -- start 2024-09-12T02:13:06.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.988+0000 7f5023b49740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee1ed025d0 0x55ee1ed19e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:06.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.988+0000 7f5023b49740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ee1ed03b40 0x55ee1ed0b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:06.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.988+0000 7f5023b49740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ee1ed11780 0x55ee1ed0b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:06.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.988+0000 7f5023b49740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f501c1a9900 con 0x55ee1ed025d0 2024-09-12T02:13:06.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.988+0000 7f5023b49740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f501c1aad30 con 0x55ee1ed03b40 2024-09-12T02:13:06.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.988+0000 7f5023b49740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f501c1a9f90 con 0x55ee1ed11780 2024-09-12T02:13:06.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.988+0000 7f5023b49740 10 client.0 _ll_register_callbacks cb 0x7f500c009610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:13:06.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.988+0000 7f5013fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee1ed025d0 0x55ee1ed19e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:06.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.988+0000 7f5013fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee1ed025d0 0x55ee1ed19e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55648/0 (socket says 172.21.15.16:55648) 2024-09-12T02:13:06.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.988+0000 7f5013fff640 1 -- 172.21.15.16:0/2875030756 learned_addr learned my addr 172.21.15.16:0/2875030756 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:13:06.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.988+0000 7f5013fff640 1 --2- 172.21.15.16:0/2875030756 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ee1ed11780 0x55ee1ed0b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:06.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.988+0000 7f501affd640 1 --2- 172.21.15.16:0/2875030756 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ee1ed03b40 0x55ee1ed0b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:06.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.988+0000 7f5013fff640 1 -- 172.21.15.16:0/2875030756 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ee1ed11780 msgr2=0x55ee1ed0b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:06.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.988+0000 7f5013fff640 1 --2- 172.21.15.16:0/2875030756 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ee1ed11780 0x55ee1ed0b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:06.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.988+0000 7f5013fff640 1 -- 172.21.15.16:0/2875030756 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ee1ed03b40 msgr2=0x55ee1ed0b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:06.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.988+0000 7f5013fff640 1 --2- 172.21.15.16:0/2875030756 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ee1ed03b40 0x55ee1ed0b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:06.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.988+0000 7f5013fff640 1 -- 172.21.15.16:0/2875030756 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee1ed0bc20 con 0x55ee1ed025d0 2024-09-12T02:13:06.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.988+0000 7f5013fff640 1 --2- 172.21.15.16:0/2875030756 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee1ed025d0 0x55ee1ed19e60 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f5004004700 tx=0x7f500402f5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65510b504238e54b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:06.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.988+0000 7f50027fc640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:13:06.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.988+0000 7f50027fc640 1 -- 172.21.15.16:0/2875030756 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5004041060 con 0x55ee1ed025d0 2024-09-12T02:13:06.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.989+0000 7f50027fc640 1 -- 172.21.15.16:0/2875030756 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f5004030ad0 con 0x55ee1ed025d0 2024-09-12T02:13:06.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.989+0000 7f5023b49740 1 -- 172.21.15.16:0/2875030756 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f500401c040 con 0x55ee1ed025d0 2024-09-12T02:13:06.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.989+0000 7f50027fc640 1 -- 172.21.15.16:0/2875030756 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f500401b070 con 0x55ee1ed025d0 2024-09-12T02:13:06.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.989+0000 7f5023b49740 10 client.19068 fetch_fsmap learned FSMap version 25 2024-09-12T02:13:06.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.989+0000 7f5023b49740 1 -- 172.21.15.16:0/2875030756 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ee1ed8bfa0 con 0x55ee1ed025d0 2024-09-12T02:13:06.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.989+0000 7f50027fc640 1 -- 172.21.15.16:0/2875030756 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f500401c040 con 0x55ee1ed025d0 2024-09-12T02:13:06.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.989+0000 7f50027fc640 1 -- 172.21.15.16:0/2875030756 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f5004015040 con 0x55ee1ed025d0 2024-09-12T02:13:06.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.989+0000 7f5023b49740 10 client.19068 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:13:06.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.989+0000 7f5023b49740 10 client.19068 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T02:13:06.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.989+0000 7f5023b49740 20 client.19068 populate_metadata read hostname 'smithi016' 2024-09-12T02:13:06.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.989+0000 7f5023b49740 1 --2- 172.21.15.16:0/2875030756 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x55ee1ed8c320 0x55ee1ed8e710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:06.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.989+0000 7f5023b49740 4 client.19068 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T02:13:06.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.989+0000 7f5023b49740 1 -- 172.21.15.16:0/2875030756 --> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ee1ec6fd70 con 0x55ee1ed8c320 2024-09-12T02:13:06.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.990+0000 7f501affd640 1 --2- 172.21.15.16:0/2875030756 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x55ee1ed8c320 0x55ee1ed8e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:06.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.990+0000 7f50027fc640 10 client.19068 ms_handle_connect on v2:172.21.15.184:6838/4203347941 2024-09-12T02:13:06.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.990+0000 7f501affd640 1 --2- 172.21.15.16:0/2875030756 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x55ee1ed8c320 0x55ee1ed8e710 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=49e1bbcbde5c060 server_cookie=643fd655a523644a in_seq=0 out_seq=0 2024-09-12T02:13:06.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.990+0000 7f50027fc640 1 -- 172.21.15.16:0/2875030756 <== mds.1 v2:172.21.15.184:6838/4203347941 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ee1ec6fd70 con 0x55ee1ed8c320 2024-09-12T02:13:06.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.990+0000 7f50027fc640 10 client.19068 handle_command_reply: tid=0 2024-09-12T02:13:06.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.990+0000 7f5023b49740 1 client.19068 shutdown 2024-09-12T02:13:06.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.990+0000 7f5023b49740 1 -- 172.21.15.16:0/2875030756 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee1ed025d0 msgr2=0x55ee1ed19e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:06.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.990+0000 7f5023b49740 1 --2- 172.21.15.16:0/2875030756 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee1ed025d0 0x55ee1ed19e60 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f5004004700 tx=0x7f500402f5c0 comp rx=0 tx=0).stop 2024-09-12T02:13:06.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.990+0000 7f5023b49740 1 -- 172.21.15.16:0/2875030756 shutdown_connections 2024-09-12T02:13:06.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.990+0000 7f5023b49740 1 --2- 172.21.15.16:0/2875030756 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x55ee1ed8c320 0x55ee1ed8e710 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:06.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.990+0000 7f5023b49740 1 --2- 172.21.15.16:0/2875030756 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ee1ed11780 0x55ee1ed0b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:06.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.990+0000 7f5023b49740 1 --2- 172.21.15.16:0/2875030756 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ee1ed03b40 0x55ee1ed0b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:06.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.990+0000 7f5023b49740 1 --2- 172.21.15.16:0/2875030756 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee1ed025d0 0x55ee1ed19e60 unknown :-1 s=CLOSED pgs=1578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:06.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.990+0000 7f5023b49740 1 -- 172.21.15.16:0/2875030756 >> 172.21.15.16:0/2875030756 conn(0x55ee1eccb310 msgr2=0x55ee1ed0d570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:06.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.990+0000 7f50027fc640 0 client.19068 ms_handle_reset on v2:172.21.15.184:6838/4203347941 2024-09-12T02:13:06.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.990+0000 7f5023b49740 1 -- 172.21.15.16:0/2875030756 shutdown_connections 2024-09-12T02:13:06.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.990+0000 7f5023b49740 1 -- 172.21.15.16:0/2875030756 wait complete. 2024-09-12T02:13:06.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.990+0000 7f5023b49740 20 client.19068 trim_cache size 0 max 16384 2024-09-12T02:13:06.996 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T02:13:06.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.991+0000 7f5021770640 1 -- 172.21.15.16:0/1150185429 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f50080a2b80 msgr2=0x7f50080a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:06.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.991+0000 7f5021770640 1 --2- 172.21.15.16:0/1150185429 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f50080a2b80 0x7f50080a5040 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f50140b95f0 tx=0x7f501408fce0 comp rx=0 tx=0).stop 2024-09-12T02:13:06.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.991+0000 7f5021770640 1 -- 172.21.15.16:0/1150185429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f501c106f10 msgr2=0x7f501c19ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:06.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.991+0000 7f5021770640 1 --2- 172.21.15.16:0/1150185429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f501c106f10 0x7f501c19ce20 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f5014031a20 tx=0x7f5014031a50 comp rx=0 tx=0).stop 2024-09-12T02:13:06.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.991+0000 7f5021770640 1 -- 172.21.15.16:0/1150185429 shutdown_connections 2024-09-12T02:13:06.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.991+0000 7f5021770640 1 --2- 172.21.15.16:0/1150185429 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f50080a2b80 0x7f50080a5040 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:06.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.991+0000 7f5021770640 1 --2- 172.21.15.16:0/1150185429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f501c106f10 0x7f501c19ce20 unknown :-1 s=CLOSED pgs=1574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:06.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.991+0000 7f5021770640 1 --2- 172.21.15.16:0/1150185429 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f501c102790 0x7f501c198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:06.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.991+0000 7f5021770640 1 --2- 172.21.15.16:0/1150185429 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f501c101de0 0x7f501c198550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:06.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.991+0000 7f5021770640 1 -- 172.21.15.16:0/1150185429 >> 172.21.15.16:0/1150185429 conn(0x7f501c0fd8c0 msgr2=0x7f501c0fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:06.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.991+0000 7f5021770640 1 -- 172.21.15.16:0/1150185429 shutdown_connections 2024-09-12T02:13:06.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:06.991+0000 7f5021770640 1 -- 172.21.15.16:0/1150185429 wait complete. 2024-09-12T02:13:07.673 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-12T02:13:07.937 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:13:07.955 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.951+0000 7f680f52c640 1 -- 172.21.15.16:0/2921905532 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6808103710 msgr2=0x7f6808109fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:07.955 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.951+0000 7f680f52c640 1 --2- 172.21.15.16:0/2921905532 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6808103710 0x7f6808109fa0 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f6800009ca0 tx=0x7f680002a8c0 comp rx=0 tx=0).stop 2024-09-12T02:13:07.956 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.951+0000 7f680f52c640 1 -- 172.21.15.16:0/2921905532 shutdown_connections 2024-09-12T02:13:07.956 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.951+0000 7f680f52c640 1 --2- 172.21.15.16:0/2921905532 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6808103710 0x7f6808109fa0 unknown :-1 s=CLOSED pgs=1579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:07.956 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.951+0000 7f680f52c640 1 --2- 172.21.15.16:0/2921905532 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6808102d50 0x7f68081031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:07.956 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.951+0000 7f680f52c640 1 --2- 172.21.15.16:0/2921905532 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6808101b60 0x7f6808101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:07.956 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.951+0000 7f680f52c640 1 -- 172.21.15.16:0/2921905532 >> 172.21.15.16:0/2921905532 conn(0x7f68080fd2e0 msgr2=0x7f68080ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:07.956 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.951+0000 7f680f52c640 1 -- 172.21.15.16:0/2921905532 shutdown_connections 2024-09-12T02:13:07.957 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.951+0000 7f680f52c640 1 -- 172.21.15.16:0/2921905532 wait complete. 2024-09-12T02:13:07.957 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.952+0000 7f680f52c640 1 Processor -- start 2024-09-12T02:13:07.957 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.952+0000 7f680f52c640 1 -- start start 2024-09-12T02:13:07.957 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.952+0000 7f680f52c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6808101b60 0x7f6808197dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:07.957 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.952+0000 7f680f52c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6808102d50 0x7f6808198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:07.958 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.952+0000 7f680f52c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6808103710 0x7f680819f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:07.958 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.952+0000 7f680f52c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f680810d7e0 con 0x7f6808101b60 2024-09-12T02:13:07.958 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.952+0000 7f680f52c640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f680810d660 con 0x7f6808102d50 2024-09-12T02:13:07.958 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.952+0000 7f680f52c640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f680810d960 con 0x7f6808103710 2024-09-12T02:13:07.958 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.952+0000 7f680d2c8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6808101b60 0x7f6808197dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:07.959 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.952+0000 7f680d2c8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6808101b60 0x7f6808197dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55666/0 (socket says 172.21.15.16:55666) 2024-09-12T02:13:07.959 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.952+0000 7f680d2c8640 1 -- 172.21.15.16:0/900008962 learned_addr learned my addr 172.21.15.16:0/900008962 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:13:07.959 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.952+0000 7f680cac7640 1 --2- 172.21.15.16:0/900008962 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6808102d50 0x7f6808198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:07.959 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.952+0000 7f680dac9640 1 --2- 172.21.15.16:0/900008962 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6808103710 0x7f680819f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:07.959 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.952+0000 7f680d2c8640 1 -- 172.21.15.16:0/900008962 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6808103710 msgr2=0x7f680819f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:07.959 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.953+0000 7f680d2c8640 1 --2- 172.21.15.16:0/900008962 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6808103710 0x7f680819f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:07.960 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.953+0000 7f680d2c8640 1 -- 172.21.15.16:0/900008962 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6808102d50 msgr2=0x7f6808198300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:07.960 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.953+0000 7f680d2c8640 1 --2- 172.21.15.16:0/900008962 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6808102d50 0x7f6808198300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:07.960 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.953+0000 7f680d2c8640 1 -- 172.21.15.16:0/900008962 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f680819f9d0 con 0x7f6808101b60 2024-09-12T02:13:07.960 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.953+0000 7f680d2c8640 1 --2- 172.21.15.16:0/900008962 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6808101b60 0x7f6808197dc0 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7f680400e9c0 tx=0x7f680400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:07.960 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.953+0000 7f67fe7fc640 1 -- 172.21.15.16:0/900008962 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f680400cd80 con 0x7f6808101b60 2024-09-12T02:13:07.961 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.953+0000 7f67fe7fc640 1 -- 172.21.15.16:0/900008962 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68040035f0 con 0x7f6808101b60 2024-09-12T02:13:07.961 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.953+0000 7f67fe7fc640 1 -- 172.21.15.16:0/900008962 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6804010690 con 0x7f6808101b60 2024-09-12T02:13:07.961 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.953+0000 7f680f52c640 1 -- 172.21.15.16:0/900008962 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f680819fcc0 con 0x7f6808101b60 2024-09-12T02:13:07.961 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.953+0000 7f680f52c640 1 -- 172.21.15.16:0/900008962 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68081a01b0 con 0x7f6808101b60 2024-09-12T02:13:07.970 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.956+0000 7f67fe7fc640 1 -- 172.21.15.16:0/900008962 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f68040037a0 con 0x7f6808101b60 2024-09-12T02:13:07.970 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.957+0000 7f68122c2740 1 Processor -- start 2024-09-12T02:13:07.970 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.957+0000 7f68122c2740 1 -- start start 2024-09-12T02:13:07.970 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.957+0000 7f68122c2740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fccb3019a0 0x55fccb301da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:07.970 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.957+0000 7f68122c2740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fccb3022e0 0x55fccb311680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:07.971 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.957+0000 7f68122c2740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fccb311bc0 0x55fccb314030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:07.971 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.957+0000 7f68122c2740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55fccb2672f0 con 0x55fccb3022e0 2024-09-12T02:13:07.971 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.957+0000 7f68122c2740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55fccb20f270 con 0x55fccb3019a0 2024-09-12T02:13:07.971 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.957+0000 7f68122c2740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55fccb2632b0 con 0x55fccb311bc0 2024-09-12T02:13:07.971 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.957+0000 7f680d2c8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fccb3022e0 0x55fccb311680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:07.971 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.957+0000 7f680dac9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fccb3019a0 0x55fccb301da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:07.971 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.957+0000 7f680d2c8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fccb3022e0 0x55fccb311680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55668/0 (socket says 172.21.15.16:55668) 2024-09-12T02:13:07.971 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.957+0000 7f680d2c8640 1 -- 172.21.15.16:0/1449842008 learned_addr learned my addr 172.21.15.16:0/1449842008 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:13:07.971 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.957+0000 7f680cac7640 1 --2- 172.21.15.16:0/1449842008 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fccb311bc0 0x55fccb314030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:07.971 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.958+0000 7f67fe7fc640 1 --2- 172.21.15.16:0/900008962 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f67f40a2cb0 0x7f67f40a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:07.971 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.958+0000 7f680d2c8640 1 -- 172.21.15.16:0/1449842008 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fccb311bc0 msgr2=0x55fccb314030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:07.971 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.958+0000 7f680d2c8640 1 --2- 172.21.15.16:0/1449842008 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fccb311bc0 0x55fccb314030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:07.972 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.958+0000 7f680d2c8640 1 -- 172.21.15.16:0/1449842008 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fccb3019a0 msgr2=0x55fccb301da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:07.972 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.958+0000 7f680d2c8640 1 --2- 172.21.15.16:0/1449842008 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fccb3019a0 0x55fccb301da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:07.972 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.958+0000 7f680d2c8640 1 -- 172.21.15.16:0/1449842008 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fccb20e340 con 0x55fccb3022e0 2024-09-12T02:13:07.972 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.958+0000 7f680dac9640 1 --2- 172.21.15.16:0/1449842008 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fccb3019a0 0x55fccb301da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:13:07.972 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.958+0000 7f680dac9640 1 --2- 172.21.15.16:0/900008962 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f67f40a2cb0 0x7f67f40a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:07.972 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.958+0000 7f67fe7fc640 1 -- 172.21.15.16:0/900008962 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f68040cf5c0 con 0x7f6808101b60 2024-09-12T02:13:07.972 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.958+0000 7f680d2c8640 1 --2- 172.21.15.16:0/1449842008 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fccb3022e0 0x55fccb311680 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f6804003dd0 tx=0x7f6804020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60111cb086a2474e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:07.972 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.958+0000 7f680dac9640 1 --2- 172.21.15.16:0/900008962 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f67f40a2cb0 0x7f67f40a5170 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f6800006010 tx=0x7f68000095c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:07.972 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.958+0000 7f67f37fe640 1 -- 172.21.15.16:0/1449842008 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f680409bbf0 con 0x55fccb3022e0 2024-09-12T02:13:07.972 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.958+0000 7f67f37fe640 1 -- 172.21.15.16:0/1449842008 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f680401fe20 con 0x55fccb3022e0 2024-09-12T02:13:07.972 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.958+0000 7f67f37fe640 1 -- 172.21.15.16:0/1449842008 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f680400a250 con 0x55fccb3022e0 2024-09-12T02:13:07.972 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.959+0000 7f68122c2740 1 -- 172.21.15.16:0/1449842008 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fccb3022e0 msgr2=0x55fccb311680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:07.972 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.959+0000 7f68122c2740 1 --2- 172.21.15.16:0/1449842008 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fccb3022e0 0x55fccb311680 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f6804003dd0 tx=0x7f6804020000 comp rx=0 tx=0).stop 2024-09-12T02:13:07.972 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.959+0000 7f68122c2740 1 -- 172.21.15.16:0/1449842008 shutdown_connections 2024-09-12T02:13:07.973 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.959+0000 7f68122c2740 1 --2- 172.21.15.16:0/1449842008 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fccb311bc0 0x55fccb314030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:07.973 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.959+0000 7f68122c2740 1 --2- 172.21.15.16:0/1449842008 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fccb3022e0 0x55fccb311680 unknown :-1 s=CLOSED pgs=1581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:07.973 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.959+0000 7f68122c2740 1 --2- 172.21.15.16:0/1449842008 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fccb3019a0 0x55fccb301da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:07.973 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.959+0000 7f68122c2740 1 -- 172.21.15.16:0/1449842008 >> 172.21.15.16:0/1449842008 conn(0x55fccb2fef90 msgr2=0x55fccb2ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:07.973 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.959+0000 7f68122c2740 1 -- 172.21.15.16:0/1449842008 shutdown_connections 2024-09-12T02:13:07.973 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.959+0000 7f68122c2740 1 -- 172.21.15.16:0/1449842008 wait complete. 2024-09-12T02:13:07.973 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.960+0000 7f68122c2740 1 -- start start 2024-09-12T02:13:07.973 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.960+0000 7f68122c2740 1 Processor -- start 2024-09-12T02:13:07.973 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.960+0000 7f68122c2740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fccb3019a0 0x55fccb316330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:07.973 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.960+0000 7f68122c2740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fccb3022e0 0x55fccb316870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:07.973 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.960+0000 7f68122c2740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fccb311bc0 0x55fccb31d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:07.973 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.960+0000 7f68122c2740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55fccb314ae0 con 0x55fccb3022e0 2024-09-12T02:13:07.973 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.960+0000 7f68122c2740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55fccb1f0290 con 0x55fccb3019a0 2024-09-12T02:13:07.974 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.960+0000 7f68122c2740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55fccb314c60 con 0x55fccb311bc0 2024-09-12T02:13:07.974 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.960+0000 7f68122c2740 10 client.0 _ll_register_callbacks cb 0x55fccb221140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:13:07.974 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.960+0000 7f680cac7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fccb3022e0 0x55fccb316870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:07.974 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.960+0000 7f680cac7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fccb3022e0 0x55fccb316870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55680/0 (socket says 172.21.15.16:55680) 2024-09-12T02:13:07.974 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.960+0000 7f680cac7640 1 -- 172.21.15.16:0/17737007 learned_addr learned my addr 172.21.15.16:0/17737007 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:13:07.974 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.960+0000 7f680dac9640 1 --2- 172.21.15.16:0/17737007 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fccb311bc0 0x55fccb31d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:07.974 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.960+0000 7f680d2c8640 1 --2- 172.21.15.16:0/17737007 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fccb3019a0 0x55fccb316330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:07.974 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.961+0000 7f680cac7640 1 -- 172.21.15.16:0/17737007 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fccb311bc0 msgr2=0x55fccb31d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:07.974 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.961+0000 7f680cac7640 1 --2- 172.21.15.16:0/17737007 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fccb311bc0 0x55fccb31d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:07.974 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.961+0000 7f680cac7640 1 -- 172.21.15.16:0/17737007 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fccb3019a0 msgr2=0x55fccb316330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:07.974 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.961+0000 7f680cac7640 1 --2- 172.21.15.16:0/17737007 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fccb3019a0 0x55fccb316330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:07.974 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.961+0000 7f680cac7640 1 -- 172.21.15.16:0/17737007 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fccaf7a210 con 0x55fccb3022e0 2024-09-12T02:13:07.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.961+0000 7f680cac7640 1 --2- 172.21.15.16:0/17737007 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fccb3022e0 0x55fccb316870 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7f67f800bf00 tx=0x7f67f8012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9985b3904b8b424a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:07.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.961+0000 7f67f0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:13:07.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.961+0000 7f67f0ff9640 1 -- 172.21.15.16:0/17737007 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67f80108a0 con 0x55fccb3022e0 2024-09-12T02:13:07.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.961+0000 7f67f0ff9640 1 -- 172.21.15.16:0/17737007 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f67f8010a40 con 0x55fccb3022e0 2024-09-12T02:13:07.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.961+0000 7f68122c2740 1 -- 172.21.15.16:0/17737007 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f680401fe20 con 0x55fccb3022e0 2024-09-12T02:13:07.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.961+0000 7f67f0ff9640 1 -- 172.21.15.16:0/17737007 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f67f8010c40 con 0x55fccb3022e0 2024-09-12T02:13:07.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.961+0000 7f68122c2740 10 client.19080 fetch_fsmap learned FSMap version 25 2024-09-12T02:13:07.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.961+0000 7f68122c2740 1 -- 172.21.15.16:0/17737007 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55fccb046620 con 0x55fccb3022e0 2024-09-12T02:13:07.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.961+0000 7f67f0ff9640 1 -- 172.21.15.16:0/17737007 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f680401fe20 con 0x55fccb3022e0 2024-09-12T02:13:07.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.962+0000 7f67f0ff9640 1 -- 172.21.15.16:0/17737007 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f67f8019070 con 0x55fccb3022e0 2024-09-12T02:13:07.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.962+0000 7f68122c2740 10 client.19080 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:13:07.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.962+0000 7f68122c2740 10 client.19080 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:13:07.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.962+0000 7f68122c2740 20 client.19080 populate_metadata read hostname 'smithi016' 2024-09-12T02:13:07.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.962+0000 7f68122c2740 1 --2- 172.21.15.16:0/17737007 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55fccb321fb0 0x55fccb324400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:07.976 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.962+0000 7f68122c2740 4 client.19080 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:13:07.976 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.962+0000 7f68122c2740 1 -- 172.21.15.16:0/17737007 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fccb274a20 con 0x55fccb321fb0 2024-09-12T02:13:07.976 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.962+0000 7f680d2c8640 1 --2- 172.21.15.16:0/17737007 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55fccb321fb0 0x55fccb324400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:07.976 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.962+0000 7f680d2c8640 1 --2- 172.21.15.16:0/17737007 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55fccb321fb0 0x55fccb324400 crc :-1 s=READY pgs=334 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a554771c036599e server_cookie=9e30c250ad8a0c39 in_seq=0 out_seq=0 2024-09-12T02:13:07.976 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.962+0000 7f67f0ff9640 10 client.19080 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:13:07.976 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.963+0000 7f67f0ff9640 1 -- 172.21.15.16:0/17737007 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fccb274a20 con 0x55fccb321fb0 2024-09-12T02:13:07.976 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.963+0000 7f67f0ff9640 10 client.19080 handle_command_reply: tid=0 2024-09-12T02:13:07.976 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.963+0000 7f68122c2740 1 client.19080 shutdown 2024-09-12T02:13:07.976 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.963+0000 7f68122c2740 1 -- 172.21.15.16:0/17737007 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fccb3022e0 msgr2=0x55fccb316870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:07.976 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.963+0000 7f68122c2740 1 --2- 172.21.15.16:0/17737007 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fccb3022e0 0x55fccb316870 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7f67f800bf00 tx=0x7f67f8012820 comp rx=0 tx=0).stop 2024-09-12T02:13:07.976 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.963+0000 7f68122c2740 1 -- 172.21.15.16:0/17737007 shutdown_connections 2024-09-12T02:13:07.976 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.963+0000 7f68122c2740 1 --2- 172.21.15.16:0/17737007 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55fccb321fb0 0x55fccb324400 crc :-1 s=READY pgs=334 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:07.976 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.963+0000 7f68122c2740 1 --2- 172.21.15.16:0/17737007 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fccb311bc0 0x55fccb31d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:07.977 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.963+0000 7f67f0ff9640 0 client.19080 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:13:07.977 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.963+0000 7f68122c2740 1 --2- 172.21.15.16:0/17737007 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fccb3022e0 0x55fccb316870 unknown :-1 s=CLOSED pgs=1582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:07.977 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.963+0000 7f68122c2740 1 --2- 172.21.15.16:0/17737007 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fccb3019a0 0x55fccb316330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:07.977 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.963+0000 7f68122c2740 1 -- 172.21.15.16:0/17737007 >> 172.21.15.16:0/17737007 conn(0x55fccb2fef90 msgr2=0x55fccb3007a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:07.977 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.964+0000 7f68122c2740 1 -- 172.21.15.16:0/17737007 shutdown_connections 2024-09-12T02:13:07.977 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.964+0000 7f68122c2740 1 -- 172.21.15.16:0/17737007 wait complete. 2024-09-12T02:13:07.977 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.964+0000 7f68122c2740 20 client.19080 trim_cache size 0 max 16384 2024-09-12T02:13:07.979 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.978+0000 7f68122c2740 1 Processor -- start 2024-09-12T02:13:07.980 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.978+0000 7f68122c2740 1 -- start start 2024-09-12T02:13:07.980 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.979+0000 7f68122c2740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fccb3019a0 0x55fccb2ffea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:07.980 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.979+0000 7f68122c2740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fccb3022e0 0x55fccb31d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:07.980 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.979+0000 7f68122c2740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fccb311bc0 0x55fccb3003e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:07.980 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.979+0000 7f68122c2740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f68081a95a0 con 0x55fccb3022e0 2024-09-12T02:13:07.980 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.979+0000 7f68122c2740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f68081a6e70 con 0x55fccb3019a0 2024-09-12T02:13:07.980 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.979+0000 7f68122c2740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f68081a4cc0 con 0x55fccb311bc0 2024-09-12T02:13:07.980 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.979+0000 7f680cac7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fccb3022e0 0x55fccb31d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:07.980 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.979+0000 7f680cac7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fccb3022e0 0x55fccb31d870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55684/0 (socket says 172.21.15.16:55684) 2024-09-12T02:13:07.980 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.979+0000 7f680cac7640 1 -- 172.21.15.16:0/3780879016 learned_addr learned my addr 172.21.15.16:0/3780879016 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:13:07.980 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.979+0000 7f680d2c8640 1 --2- 172.21.15.16:0/3780879016 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fccb3019a0 0x55fccb2ffea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:07.980 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.979+0000 7f680dac9640 1 --2- 172.21.15.16:0/3780879016 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fccb311bc0 0x55fccb3003e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:07.981 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.979+0000 7f680cac7640 1 -- 172.21.15.16:0/3780879016 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fccb311bc0 msgr2=0x55fccb3003e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:07.981 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.979+0000 7f680cac7640 1 --2- 172.21.15.16:0/3780879016 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fccb311bc0 0x55fccb3003e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:07.981 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.979+0000 7f680cac7640 1 -- 172.21.15.16:0/3780879016 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fccb3019a0 msgr2=0x55fccb2ffea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:07.981 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.979+0000 7f680cac7640 1 --2- 172.21.15.16:0/3780879016 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fccb3019a0 0x55fccb2ffea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:07.981 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.979+0000 7f680cac7640 1 -- 172.21.15.16:0/3780879016 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fccb311000 con 0x55fccb3022e0 2024-09-12T02:13:07.981 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.979+0000 7f680d2c8640 1 --2- 172.21.15.16:0/3780879016 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fccb3019a0 0x55fccb2ffea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:13:07.981 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.979+0000 7f680cac7640 1 --2- 172.21.15.16:0/3780879016 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fccb3022e0 0x55fccb31d870 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7f67f8000c00 tx=0x7f67f8025480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1275cb941d39e95e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:07.981 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.979+0000 7f67f2ffd640 1 -- 172.21.15.16:0/3780879016 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67f80106a0 con 0x55fccb3022e0 2024-09-12T02:13:07.981 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.979+0000 7f67f2ffd640 1 -- 172.21.15.16:0/3780879016 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67f8025ce0 con 0x55fccb3022e0 2024-09-12T02:13:07.981 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.980+0000 7f67f2ffd640 1 -- 172.21.15.16:0/3780879016 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67f801bb80 con 0x55fccb3022e0 2024-09-12T02:13:07.984 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.980+0000 7f68122c2740 1 -- 172.21.15.16:0/3780879016 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fccb3022e0 msgr2=0x55fccb31d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:07.985 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.980+0000 7f68122c2740 1 --2- 172.21.15.16:0/3780879016 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fccb3022e0 0x55fccb31d870 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7f67f8000c00 tx=0x7f67f8025480 comp rx=0 tx=0).stop 2024-09-12T02:13:07.985 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.980+0000 7f68122c2740 1 -- 172.21.15.16:0/3780879016 shutdown_connections 2024-09-12T02:13:07.985 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.980+0000 7f68122c2740 1 --2- 172.21.15.16:0/3780879016 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fccb311bc0 0x55fccb3003e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:07.985 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.980+0000 7f68122c2740 1 --2- 172.21.15.16:0/3780879016 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fccb3022e0 0x55fccb31d870 unknown :-1 s=CLOSED pgs=1583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:07.985 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.980+0000 7f68122c2740 1 --2- 172.21.15.16:0/3780879016 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fccb3019a0 0x55fccb2ffea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:07.985 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.980+0000 7f68122c2740 1 -- 172.21.15.16:0/3780879016 >> 172.21.15.16:0/3780879016 conn(0x55fccb2fef90 msgr2=0x55fccb316330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:07.985 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.980+0000 7f68122c2740 1 -- 172.21.15.16:0/3780879016 shutdown_connections 2024-09-12T02:13:07.985 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.980+0000 7f68122c2740 1 -- 172.21.15.16:0/3780879016 wait complete. 2024-09-12T02:13:07.985 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.981+0000 7f68122c2740 1 -- start start 2024-09-12T02:13:07.985 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.981+0000 7f68122c2740 1 Processor -- start 2024-09-12T02:13:07.985 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.981+0000 7f68122c2740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fccb3019a0 0x55fccb3191b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:07.985 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.981+0000 7f68122c2740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fccb3022e0 0x55fccb3196f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:07.986 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.981+0000 7f68122c2740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fccb311bc0 0x55fccb31c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:07.986 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.981+0000 7f68122c2740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f68081a5330 con 0x55fccb311bc0 2024-09-12T02:13:07.986 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.981+0000 7f68122c2740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f68081a67a0 con 0x55fccb3022e0 2024-09-12T02:13:07.986 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.981+0000 7f68122c2740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f68081a5a00 con 0x55fccb3019a0 2024-09-12T02:13:07.986 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.981+0000 7f68122c2740 10 client.0 _ll_register_callbacks cb 0x7f67f800b730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:13:07.986 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.981+0000 7f680dac9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fccb311bc0 0x55fccb31c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:07.986 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.981+0000 7f680dac9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fccb311bc0 0x55fccb31c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55700/0 (socket says 172.21.15.16:55700) 2024-09-12T02:13:07.986 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.981+0000 7f680dac9640 1 -- 172.21.15.16:0/3717380870 learned_addr learned my addr 172.21.15.16:0/3717380870 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:13:07.986 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.981+0000 7f680d2c8640 1 --2- 172.21.15.16:0/3717380870 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fccb3019a0 0x55fccb3191b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:07.986 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.981+0000 7f680cac7640 1 --2- 172.21.15.16:0/3717380870 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fccb3022e0 0x55fccb3196f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:07.986 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.981+0000 7f680dac9640 1 -- 172.21.15.16:0/3717380870 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fccb3019a0 msgr2=0x55fccb3191b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:07.986 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.981+0000 7f680dac9640 1 --2- 172.21.15.16:0/3717380870 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fccb3019a0 0x55fccb3191b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:07.986 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.981+0000 7f680dac9640 1 -- 172.21.15.16:0/3717380870 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fccb3022e0 msgr2=0x55fccb3196f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:07.987 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.981+0000 7f680dac9640 1 --2- 172.21.15.16:0/3717380870 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fccb3022e0 0x55fccb3196f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:07.987 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.981+0000 7f680dac9640 1 -- 172.21.15.16:0/3717380870 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fccb31cc80 con 0x55fccb311bc0 2024-09-12T02:13:07.987 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.982+0000 7f680dac9640 1 --2- 172.21.15.16:0/3717380870 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fccb311bc0 0x55fccb31c670 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f6800035000 tx=0x7f680002a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9527ebfc8b8b561 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:07.987 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.982+0000 7f67f0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:13:07.987 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.982+0000 7f67f0ff9640 1 -- 172.21.15.16:0/3717380870 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6800031db0 con 0x55fccb311bc0 2024-09-12T02:13:07.987 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.982+0000 7f67f0ff9640 1 -- 172.21.15.16:0/3717380870 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f6800004040 con 0x55fccb311bc0 2024-09-12T02:13:07.987 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.982+0000 7f68122c2740 1 -- 172.21.15.16:0/3717380870 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f67f8025ce0 con 0x55fccb311bc0 2024-09-12T02:13:07.987 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.982+0000 7f67f0ff9640 1 -- 172.21.15.16:0/3717380870 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6800004240 con 0x55fccb311bc0 2024-09-12T02:13:07.987 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.982+0000 7f68122c2740 10 client.19086 fetch_fsmap learned FSMap version 25 2024-09-12T02:13:07.987 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.982+0000 7f68122c2740 1 -- 172.21.15.16:0/3717380870 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55fccb38cea0 con 0x55fccb311bc0 2024-09-12T02:13:07.987 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.982+0000 7f67f0ff9640 1 -- 172.21.15.16:0/3717380870 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67f8025ce0 con 0x55fccb311bc0 2024-09-12T02:13:07.987 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.982+0000 7f67f0ff9640 1 -- 172.21.15.16:0/3717380870 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f68000305c0 con 0x55fccb311bc0 2024-09-12T02:13:07.987 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.982+0000 7f68122c2740 10 client.19086 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:13:07.988 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.982+0000 7f68122c2740 10 client.19086 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:13:07.988 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.982+0000 7f68122c2740 20 client.19086 populate_metadata read hostname 'smithi016' 2024-09-12T02:13:07.988 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.982+0000 7f68122c2740 1 --2- 172.21.15.16:0/3717380870 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55fccb38d8d0 0x55fccb38fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:07.988 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.982+0000 7f68122c2740 4 client.19086 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:13:07.988 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.982+0000 7f68122c2740 1 -- 172.21.15.16:0/3717380870 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fccb26ed70 con 0x55fccb38d8d0 2024-09-12T02:13:07.988 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.983+0000 7f680d2c8640 1 --2- 172.21.15.16:0/3717380870 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55fccb38d8d0 0x55fccb38fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:07.988 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.983+0000 7f680d2c8640 1 --2- 172.21.15.16:0/3717380870 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55fccb38d8d0 0x55fccb38fcf0 crc :-1 s=READY pgs=335 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=926cd4b78451c548 server_cookie=2bd490b2f17e1d00 in_seq=0 out_seq=0 2024-09-12T02:13:07.988 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.983+0000 7f67f0ff9640 10 client.19086 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:13:07.996 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.996+0000 7f67f0ff9640 1 -- 172.21.15.16:0/3717380870 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55fccb26ed70 con 0x55fccb38d8d0 2024-09-12T02:13:07.997 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.996+0000 7f67f0ff9640 10 client.19086 handle_command_reply: tid=0 2024-09-12T02:13:07.997 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.996+0000 7f68122c2740 1 client.19086 shutdown 2024-09-12T02:13:07.997 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.996+0000 7f68122c2740 1 -- 172.21.15.16:0/3717380870 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fccb311bc0 msgr2=0x55fccb31c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:07.997 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.996+0000 7f68122c2740 1 --2- 172.21.15.16:0/3717380870 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fccb311bc0 0x55fccb31c670 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f6800035000 tx=0x7f680002a8c0 comp rx=0 tx=0).stop 2024-09-12T02:13:07.997 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.996+0000 7f68122c2740 1 -- 172.21.15.16:0/3717380870 shutdown_connections 2024-09-12T02:13:07.997 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.996+0000 7f68122c2740 1 --2- 172.21.15.16:0/3717380870 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55fccb38d8d0 0x55fccb38fcf0 crc :-1 s=READY pgs=335 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:07.997 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.996+0000 7f68122c2740 1 --2- 172.21.15.16:0/3717380870 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fccb311bc0 0x55fccb31c670 unknown :-1 s=CLOSED pgs=1584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:07.997 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.996+0000 7f68122c2740 1 --2- 172.21.15.16:0/3717380870 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fccb3022e0 0x55fccb3196f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:07.997 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.996+0000 7f67f0ff9640 0 client.19086 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:13:07.997 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.996+0000 7f68122c2740 1 --2- 172.21.15.16:0/3717380870 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fccb3019a0 0x55fccb3191b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:07.997 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.996+0000 7f68122c2740 1 -- 172.21.15.16:0/3717380870 >> 172.21.15.16:0/3717380870 conn(0x55fccb2fef90 msgr2=0x55fccb2fd9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:07.997 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.996+0000 7f68122c2740 1 -- 172.21.15.16:0/3717380870 shutdown_connections 2024-09-12T02:13:07.997 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.996+0000 7f68122c2740 1 -- 172.21.15.16:0/3717380870 wait complete. 2024-09-12T02:13:07.998 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.996+0000 7f68122c2740 20 client.19086 trim_cache size 0 max 16384 2024-09-12T02:13:07.998 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-09-12T02:13:07.998 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-09-12T02:13:07.998 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-09-12T02:13:07.998 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-09-12T02:13:07.999 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.997+0000 7f680f52c640 1 -- 172.21.15.16:0/900008962 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f67f40a2cb0 msgr2=0x7f67f40a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:07.999 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.997+0000 7f680f52c640 1 --2- 172.21.15.16:0/900008962 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f67f40a2cb0 0x7f67f40a5170 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f6800006010 tx=0x7f68000095c0 comp rx=0 tx=0).stop 2024-09-12T02:13:07.999 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.997+0000 7f680f52c640 1 -- 172.21.15.16:0/900008962 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6808101b60 msgr2=0x7f6808197dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:07.999 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.997+0000 7f680f52c640 1 --2- 172.21.15.16:0/900008962 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6808101b60 0x7f6808197dc0 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7f680400e9c0 tx=0x7f680400ee80 comp rx=0 tx=0).stop 2024-09-12T02:13:07.999 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.997+0000 7f680f52c640 1 -- 172.21.15.16:0/900008962 shutdown_connections 2024-09-12T02:13:07.999 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.997+0000 7f680f52c640 1 --2- 172.21.15.16:0/900008962 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f67f40a2cb0 0x7f67f40a5170 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:07.999 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.997+0000 7f680f52c640 1 --2- 172.21.15.16:0/900008962 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6808103710 0x7f680819f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:08.000 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.997+0000 7f680f52c640 1 --2- 172.21.15.16:0/900008962 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6808102d50 0x7f6808198300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:08.000 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.997+0000 7f680f52c640 1 --2- 172.21.15.16:0/900008962 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6808101b60 0x7f6808197dc0 unknown :-1 s=CLOSED pgs=1580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:08.000 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.997+0000 7f680f52c640 1 -- 172.21.15.16:0/900008962 >> 172.21.15.16:0/900008962 conn(0x7f68080fd2e0 msgr2=0x7f68081060d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:08.000 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.997+0000 7f680f52c640 1 -- 172.21.15.16:0/900008962 shutdown_connections 2024-09-12T02:13:08.000 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:07.997+0000 7f680f52c640 1 -- 172.21.15.16:0/900008962 wait complete. 2024-09-12T02:13:08.335 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:13:08 smithi016 ceph-mon[30947]: pgmap v671: 97 pgs: 97 active+clean; 1.9 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 18 op/s 2024-09-12T02:13:08.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:13:08 smithi184 ceph-mon[35593]: pgmap v671: 97 pgs: 97 active+clean; 1.9 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 18 op/s 2024-09-12T02:13:08.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:13:08 smithi033 ceph-mon[35332]: pgmap v671: 97 pgs: 97 active+clean; 1.9 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 18 op/s 2024-09-12T02:13:08.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.423+0000 7fc7cc95a640 1 -- 172.21.15.16:0/997870228 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7c81049c0 msgr2=0x7fc7c8108a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:08.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.423+0000 7fc7cc95a640 1 --2- 172.21.15.16:0/997870228 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7c81049c0 0x7fc7c8108a10 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7fc7bc009900 tx=0x7fc7bc02a410 comp rx=0 tx=0).stop 2024-09-12T02:13:08.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.424+0000 7fc7cc95a640 1 -- 172.21.15.16:0/997870228 shutdown_connections 2024-09-12T02:13:08.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.424+0000 7fc7cc95a640 1 --2- 172.21.15.16:0/997870228 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc7c8109140 0x7fc7c8110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:08.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.424+0000 7fc7cc95a640 1 --2- 172.21.15.16:0/997870228 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7c81049c0 0x7fc7c8108a10 unknown :-1 s=CLOSED pgs=1585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:08.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.424+0000 7fc7cc95a640 1 --2- 172.21.15.16:0/997870228 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc7c8104010 0x7fc7c81043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:08.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.424+0000 7fc7cc95a640 1 -- 172.21.15.16:0/997870228 >> 172.21.15.16:0/997870228 conn(0x7fc7c80ffa50 msgr2=0x7fc7c8101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:08.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.424+0000 7fc7cc95a640 1 -- 172.21.15.16:0/997870228 shutdown_connections 2024-09-12T02:13:08.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.424+0000 7fc7cc95a640 1 -- 172.21.15.16:0/997870228 wait complete. 2024-09-12T02:13:08.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.424+0000 7fc7cc95a640 1 Processor -- start 2024-09-12T02:13:08.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.424+0000 7fc7cc95a640 1 -- start start 2024-09-12T02:13:08.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.425+0000 7fc7cc95a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc7c8104010 0x7fc7c819a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:08.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.425+0000 7fc7cc95a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7c81049c0 0x7fc7c819ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:08.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.425+0000 7fc7cc95a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc7c8109140 0x7fc7c819eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:08.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.425+0000 7fc7cc95a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc7c8114140 con 0x7fc7c81049c0 2024-09-12T02:13:08.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.425+0000 7fc7cc95a640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc7c8113fc0 con 0x7fc7c8104010 2024-09-12T02:13:08.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.425+0000 7fc7cc95a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc7c81142c0 con 0x7fc7c8109140 2024-09-12T02:13:08.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.425+0000 7fc7c5d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7c81049c0 0x7fc7c819ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:08.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.425+0000 7fc7c5d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7c81049c0 0x7fc7c819ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55728/0 (socket says 172.21.15.16:55728) 2024-09-12T02:13:08.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.425+0000 7fc7c5d9b640 1 -- 172.21.15.16:0/1343381222 learned_addr learned my addr 172.21.15.16:0/1343381222 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:13:08.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.425+0000 7fc7c659c640 1 --2- 172.21.15.16:0/1343381222 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc7c8104010 0x7fc7c819a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:08.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.425+0000 7fc7c6d9d640 1 --2- 172.21.15.16:0/1343381222 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc7c8109140 0x7fc7c819eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:08.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.425+0000 7fc7c5d9b640 1 -- 172.21.15.16:0/1343381222 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc7c8109140 msgr2=0x7fc7c819eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:08.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.425+0000 7fc7c5d9b640 1 --2- 172.21.15.16:0/1343381222 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc7c8109140 0x7fc7c819eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:08.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.425+0000 7fc7c5d9b640 1 -- 172.21.15.16:0/1343381222 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc7c8104010 msgr2=0x7fc7c819a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:08.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.425+0000 7fc7c5d9b640 1 --2- 172.21.15.16:0/1343381222 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc7c8104010 0x7fc7c819a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:08.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.425+0000 7fc7c5d9b640 1 -- 172.21.15.16:0/1343381222 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7c819f5d0 con 0x7fc7c81049c0 2024-09-12T02:13:08.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.425+0000 7fc7c5d9b640 1 --2- 172.21.15.16:0/1343381222 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7c81049c0 0x7fc7c819ab60 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7fc7bc006010 tx=0x7fc7bc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:08.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.425+0000 7fc7b77fe640 1 -- 172.21.15.16:0/1343381222 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7bc03f030 con 0x7fc7c81049c0 2024-09-12T02:13:08.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.426+0000 7fc7cc95a640 1 -- 172.21.15.16:0/1343381222 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7c819f860 con 0x7fc7c81049c0 2024-09-12T02:13:08.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.426+0000 7fc7b77fe640 1 -- 172.21.15.16:0/1343381222 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc7bc003ce0 con 0x7fc7c81049c0 2024-09-12T02:13:08.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.426+0000 7fc7b77fe640 1 -- 172.21.15.16:0/1343381222 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7bc035070 con 0x7fc7c81049c0 2024-09-12T02:13:08.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.426+0000 7fc7cc95a640 1 -- 172.21.15.16:0/1343381222 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7c81a71a0 con 0x7fc7c81049c0 2024-09-12T02:13:08.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.428+0000 7fc7b77fe640 1 -- 172.21.15.16:0/1343381222 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fc7bc042030 con 0x7fc7c81049c0 2024-09-12T02:13:08.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.428+0000 7fc7ced33740 1 Processor -- start 2024-09-12T02:13:08.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.429+0000 7fc7ced33740 1 -- start start 2024-09-12T02:13:08.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.429+0000 7fc7ced33740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558334f2cb40 0x558334f2b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:08.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.429+0000 7fc7ced33740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558334f2b5d0 0x558334f2ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:08.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.429+0000 7fc7ced33740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558334f36370 0x558334f38760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:08.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.429+0000 7fc7ced33740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x558334e912f0 con 0x558334f2b5d0 2024-09-12T02:13:08.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.429+0000 7fc7ced33740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x558334e38cf0 con 0x558334f2cb40 2024-09-12T02:13:08.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.429+0000 7fc7ced33740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x558334e8d2b0 con 0x558334f36370 2024-09-12T02:13:08.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.429+0000 7fc7c659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558334f2b5d0 0x558334f2ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:08.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.429+0000 7fc7c659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558334f2b5d0 0x558334f2ba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55736/0 (socket says 172.21.15.16:55736) 2024-09-12T02:13:08.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.429+0000 7fc7c659c640 1 -- 172.21.15.16:0/2459995448 learned_addr learned my addr 172.21.15.16:0/2459995448 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:13:08.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.429+0000 7fc7b77fe640 1 --2- 172.21.15.16:0/1343381222 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc7b00a2b80 0x7fc7b00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:08.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.429+0000 7fc7c5d9b640 1 --2- 172.21.15.16:0/2459995448 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558334f36370 0x558334f38760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:08.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.429+0000 7fc7c6d9d640 1 --2- 172.21.15.16:0/2459995448 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558334f2cb40 0x558334f2b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:08.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.429+0000 7fc7c6d9d640 1 --2- 172.21.15.16:0/1343381222 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc7b00a2b80 0x7fc7b00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:08.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.429+0000 7fc7c659c640 1 -- 172.21.15.16:0/2459995448 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558334f36370 msgr2=0x558334f38760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:08.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.429+0000 7fc7c659c640 1 --2- 172.21.15.16:0/2459995448 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558334f36370 0x558334f38760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:08.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.429+0000 7fc7c659c640 1 -- 172.21.15.16:0/2459995448 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558334f2cb40 msgr2=0x558334f2b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:08.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.429+0000 7fc7c659c640 1 --2- 172.21.15.16:0/2459995448 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558334f2cb40 0x558334f2b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:08.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.429+0000 7fc7c659c640 1 -- 172.21.15.16:0/2459995448 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558334e1d3a0 con 0x558334f2b5d0 2024-09-12T02:13:08.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.429+0000 7fc7c6d9d640 1 --2- 172.21.15.16:0/2459995448 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558334f2cb40 0x558334f2b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:13:08.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.429+0000 7fc7b77fe640 1 -- 172.21.15.16:0/1343381222 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fc7bc0ef650 con 0x7fc7c81049c0 2024-09-12T02:13:08.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.430+0000 7fc7c659c640 1 --2- 172.21.15.16:0/2459995448 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558334f2b5d0 0x558334f2ba10 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7fc7b800bcd0 tx=0x7fc7b80107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f14558d03a99bf1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:08.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.430+0000 7fc7c6d9d640 1 --2- 172.21.15.16:0/1343381222 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc7b00a2b80 0x7fc7b00a5040 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fc7c00077f0 tx=0x7fc7c0007520 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:08.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.430+0000 7fc7b4ff9640 1 -- 172.21.15.16:0/2459995448 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7b8005220 con 0x558334f2b5d0 2024-09-12T02:13:08.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.430+0000 7fc7b4ff9640 1 -- 172.21.15.16:0/2459995448 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc7b8004c80 con 0x558334f2b5d0 2024-09-12T02:13:08.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.430+0000 7fc7b4ff9640 1 -- 172.21.15.16:0/2459995448 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7b8011780 con 0x558334f2b5d0 2024-09-12T02:13:08.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.430+0000 7fc7ced33740 1 -- 172.21.15.16:0/2459995448 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558334f2b5d0 msgr2=0x558334f2ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:08.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.430+0000 7fc7ced33740 1 --2- 172.21.15.16:0/2459995448 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558334f2b5d0 0x558334f2ba10 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7fc7b800bcd0 tx=0x7fc7b80107a0 comp rx=0 tx=0).stop 2024-09-12T02:13:08.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.430+0000 7fc7ced33740 1 -- 172.21.15.16:0/2459995448 shutdown_connections 2024-09-12T02:13:08.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.430+0000 7fc7ced33740 1 --2- 172.21.15.16:0/2459995448 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558334f36370 0x558334f38760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:08.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.430+0000 7fc7ced33740 1 --2- 172.21.15.16:0/2459995448 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558334f2b5d0 0x558334f2ba10 unknown :-1 s=CLOSED pgs=1587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:08.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.430+0000 7fc7ced33740 1 --2- 172.21.15.16:0/2459995448 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558334f2cb40 0x558334f2b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:08.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.430+0000 7fc7ced33740 1 -- 172.21.15.16:0/2459995448 >> 172.21.15.16:0/2459995448 conn(0x558334ef4310 msgr2=0x558334f27c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:08.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.431+0000 7fc7ced33740 1 -- 172.21.15.16:0/2459995448 shutdown_connections 2024-09-12T02:13:08.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.431+0000 7fc7ced33740 1 -- 172.21.15.16:0/2459995448 wait complete. 2024-09-12T02:13:08.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.431+0000 7fc7ced33740 1 -- start start 2024-09-12T02:13:08.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.431+0000 7fc7ced33740 1 Processor -- start 2024-09-12T02:13:08.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.431+0000 7fc7ced33740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558334f2b5d0 0x558334f35d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:08.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.431+0000 7fc7ced33740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558334f2cb40 0x558334f32fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:08.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.431+0000 7fc7ced33740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558334f36370 0x558334f335d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:08.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.431+0000 7fc7ced33740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x558334f39070 con 0x558334f2b5d0 2024-09-12T02:13:08.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.431+0000 7fc7ced33740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x558334e4c060 con 0x558334f2cb40 2024-09-12T02:13:08.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.431+0000 7fc7ced33740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x558334f391f0 con 0x558334f36370 2024-09-12T02:13:08.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.431+0000 7fc7ced33740 10 client.0 _ll_register_callbacks cb 0x558334b9f8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:13:08.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.431+0000 7fc7c659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558334f2b5d0 0x558334f35d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:08.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.431+0000 7fc7c659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558334f2b5d0 0x558334f35d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55748/0 (socket says 172.21.15.16:55748) 2024-09-12T02:13:08.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.431+0000 7fc7c659c640 1 -- 172.21.15.16:0/3609636897 learned_addr learned my addr 172.21.15.16:0/3609636897 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:13:08.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.432+0000 7fc7c5d9b640 1 --2- 172.21.15.16:0/3609636897 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558334f2cb40 0x558334f32fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:08.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.432+0000 7fc7c6d9d640 1 --2- 172.21.15.16:0/3609636897 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558334f36370 0x558334f335d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:08.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.432+0000 7fc7c659c640 1 -- 172.21.15.16:0/3609636897 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558334f36370 msgr2=0x558334f335d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:08.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.432+0000 7fc7c659c640 1 --2- 172.21.15.16:0/3609636897 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558334f36370 0x558334f335d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:08.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.432+0000 7fc7c659c640 1 -- 172.21.15.16:0/3609636897 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558334f2cb40 msgr2=0x558334f32fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:08.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.432+0000 7fc7c659c640 1 --2- 172.21.15.16:0/3609636897 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558334f2cb40 0x558334f32fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:08.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.432+0000 7fc7c659c640 1 -- 172.21.15.16:0/3609636897 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558334c023c0 con 0x558334f2b5d0 2024-09-12T02:13:08.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.432+0000 7fc7c659c640 1 --2- 172.21.15.16:0/3609636897 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558334f2b5d0 0x558334f35d30 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7fc7b8005720 tx=0x7fc7b80048d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76f4dc672f926eaf server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:08.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.432+0000 7fc7adffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:13:08.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.432+0000 7fc7adffb640 1 -- 172.21.15.16:0/3609636897 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7b800be80 con 0x558334f2b5d0 2024-09-12T02:13:08.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.432+0000 7fc7adffb640 1 -- 172.21.15.16:0/3609636897 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fc7b8005040 con 0x558334f2b5d0 2024-09-12T02:13:08.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.432+0000 7fc7ced33740 1 -- 172.21.15.16:0/3609636897 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc7b8004c80 con 0x558334f2b5d0 2024-09-12T02:13:08.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.432+0000 7fc7adffb640 1 -- 172.21.15.16:0/3609636897 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc7b8004ac0 con 0x558334f2b5d0 2024-09-12T02:13:08.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.432+0000 7fc7ced33740 10 client.19098 fetch_fsmap learned FSMap version 25 2024-09-12T02:13:08.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.432+0000 7fc7ced33740 1 -- 172.21.15.16:0/3609636897 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x558334f44ad0 con 0x558334f2b5d0 2024-09-12T02:13:08.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.433+0000 7fc7adffb640 1 -- 172.21.15.16:0/3609636897 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc7b8004c80 con 0x558334f2b5d0 2024-09-12T02:13:08.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.433+0000 7fc7adffb640 1 -- 172.21.15.16:0/3609636897 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fc7b802e050 con 0x558334f2b5d0 2024-09-12T02:13:08.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.433+0000 7fc7ced33740 10 client.19098 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:13:08.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.433+0000 7fc7ced33740 10 client.19098 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:13:08.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.433+0000 7fc7ced33740 20 client.19098 populate_metadata read hostname 'smithi016' 2024-09-12T02:13:08.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.433+0000 7fc7ced33740 1 --2- 172.21.15.16:0/3609636897 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x558334f45e20 0x558334f48210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:08.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.433+0000 7fc7ced33740 4 client.19098 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:13:08.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.433+0000 7fc7ced33740 1 -- 172.21.15.16:0/3609636897 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558334e9e860 con 0x558334f45e20 2024-09-12T02:13:08.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.433+0000 7fc7c5d9b640 1 --2- 172.21.15.16:0/3609636897 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x558334f45e20 0x558334f48210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:08.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.434+0000 7fc7adffb640 10 client.19098 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:13:08.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.433+0000 7fc7c5d9b640 1 --2- 172.21.15.16:0/3609636897 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x558334f45e20 0x558334f48210 crc :-1 s=READY pgs=336 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=520e8b4029775b44 server_cookie=ca31ea2b7d4552f7 in_seq=0 out_seq=0 2024-09-12T02:13:08.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.434+0000 7fc7adffb640 1 -- 172.21.15.16:0/3609636897 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558334e9e860 con 0x558334f45e20 2024-09-12T02:13:08.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.434+0000 7fc7adffb640 10 client.19098 handle_command_reply: tid=0 2024-09-12T02:13:08.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.434+0000 7fc7ced33740 1 client.19098 shutdown 2024-09-12T02:13:08.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.434+0000 7fc7ced33740 1 -- 172.21.15.16:0/3609636897 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558334f2b5d0 msgr2=0x558334f35d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:08.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.434+0000 7fc7ced33740 1 --2- 172.21.15.16:0/3609636897 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558334f2b5d0 0x558334f35d30 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7fc7b8005720 tx=0x7fc7b80048d0 comp rx=0 tx=0).stop 2024-09-12T02:13:08.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.434+0000 7fc7ced33740 1 -- 172.21.15.16:0/3609636897 shutdown_connections 2024-09-12T02:13:08.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.434+0000 7fc7ced33740 1 --2- 172.21.15.16:0/3609636897 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x558334f45e20 0x558334f48210 crc :-1 s=READY pgs=336 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:08.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.434+0000 7fc7ced33740 1 --2- 172.21.15.16:0/3609636897 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558334f36370 0x558334f335d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:08.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.434+0000 7fc7adffb640 0 client.19098 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:13:08.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.434+0000 7fc7ced33740 1 --2- 172.21.15.16:0/3609636897 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558334f2cb40 0x558334f32fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:08.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.434+0000 7fc7ced33740 1 --2- 172.21.15.16:0/3609636897 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558334f2b5d0 0x558334f35d30 unknown :-1 s=CLOSED pgs=1588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:08.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.434+0000 7fc7ced33740 1 -- 172.21.15.16:0/3609636897 >> 172.21.15.16:0/3609636897 conn(0x558334ef4310 msgr2=0x558334f2d960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:08.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.434+0000 7fc7ced33740 1 -- 172.21.15.16:0/3609636897 shutdown_connections 2024-09-12T02:13:08.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.435+0000 7fc7ced33740 1 -- 172.21.15.16:0/3609636897 wait complete. 2024-09-12T02:13:08.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.435+0000 7fc7ced33740 20 client.19098 trim_cache size 0 max 16384 2024-09-12T02:13:08.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.448+0000 7fc7ced33740 1 Processor -- start 2024-09-12T02:13:08.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.448+0000 7fc7ced33740 1 -- start start 2024-09-12T02:13:08.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.448+0000 7fc7ced33740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558334f2b5d0 0x558334f42310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:08.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.448+0000 7fc7ced33740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558334f2cb40 0x558334f31fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:08.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.448+0000 7fc7ced33740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558334f36370 0x558334f4d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:08.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.448+0000 7fc7ced33740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc7c81b1ef0 con 0x558334f2cb40 2024-09-12T02:13:08.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.448+0000 7fc7ced33740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc7c81af7c0 con 0x558334f36370 2024-09-12T02:13:08.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.448+0000 7fc7ced33740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc7c81ad630 con 0x558334f2b5d0 2024-09-12T02:13:08.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.448+0000 7fc7c5d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558334f2cb40 0x558334f31fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:08.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.448+0000 7fc7c5d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558334f2cb40 0x558334f31fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55760/0 (socket says 172.21.15.16:55760) 2024-09-12T02:13:08.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.448+0000 7fc7c5d9b640 1 -- 172.21.15.16:0/2642876203 learned_addr learned my addr 172.21.15.16:0/2642876203 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:13:08.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.448+0000 7fc7c6d9d640 1 --2- 172.21.15.16:0/2642876203 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558334f36370 0x558334f4d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:08.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.448+0000 7fc7c659c640 1 --2- 172.21.15.16:0/2642876203 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558334f2b5d0 0x558334f42310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:08.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.448+0000 7fc7c5d9b640 1 -- 172.21.15.16:0/2642876203 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558334f2b5d0 msgr2=0x558334f42310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:08.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.448+0000 7fc7c5d9b640 1 --2- 172.21.15.16:0/2642876203 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558334f2b5d0 0x558334f42310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:08.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.448+0000 7fc7c5d9b640 1 -- 172.21.15.16:0/2642876203 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558334f36370 msgr2=0x558334f4d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:08.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.448+0000 7fc7c5d9b640 1 --2- 172.21.15.16:0/2642876203 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558334f36370 0x558334f4d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:08.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.448+0000 7fc7c5d9b640 1 -- 172.21.15.16:0/2642876203 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558334f31530 con 0x558334f2cb40 2024-09-12T02:13:08.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.448+0000 7fc7c659c640 1 --2- 172.21.15.16:0/2642876203 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558334f2b5d0 0x558334f42310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:13:08.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.449+0000 7fc7c5d9b640 1 --2- 172.21.15.16:0/2642876203 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558334f2cb40 0x558334f31fc0 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7fc7bc0bbd20 tx=0x7fc7bc0e6a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb7692ef7d7a68ec server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:08.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.449+0000 7fc7affff640 1 -- 172.21.15.16:0/2642876203 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7bc049040 con 0x558334f2cb40 2024-09-12T02:13:08.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.449+0000 7fc7affff640 1 -- 172.21.15.16:0/2642876203 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc7bc0bc020 con 0x558334f2cb40 2024-09-12T02:13:08.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.449+0000 7fc7affff640 1 -- 172.21.15.16:0/2642876203 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7bc0c4be0 con 0x558334f2cb40 2024-09-12T02:13:08.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.449+0000 7fc7ced33740 1 -- 172.21.15.16:0/2642876203 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558334f2cb40 msgr2=0x558334f31fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:08.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.449+0000 7fc7ced33740 1 --2- 172.21.15.16:0/2642876203 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558334f2cb40 0x558334f31fc0 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7fc7bc0bbd20 tx=0x7fc7bc0e6a40 comp rx=0 tx=0).stop 2024-09-12T02:13:08.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.449+0000 7fc7ced33740 1 -- 172.21.15.16:0/2642876203 shutdown_connections 2024-09-12T02:13:08.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.449+0000 7fc7ced33740 1 --2- 172.21.15.16:0/2642876203 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558334f36370 0x558334f4d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:08.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.449+0000 7fc7ced33740 1 --2- 172.21.15.16:0/2642876203 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558334f2cb40 0x558334f31fc0 unknown :-1 s=CLOSED pgs=1589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:08.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.449+0000 7fc7ced33740 1 --2- 172.21.15.16:0/2642876203 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558334f2b5d0 0x558334f42310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:08.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.449+0000 7fc7ced33740 1 -- 172.21.15.16:0/2642876203 >> 172.21.15.16:0/2642876203 conn(0x558334ef4310 msgr2=0x558334f318b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:08.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.449+0000 7fc7ced33740 1 -- 172.21.15.16:0/2642876203 shutdown_connections 2024-09-12T02:13:08.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.449+0000 7fc7ced33740 1 -- 172.21.15.16:0/2642876203 wait complete. 2024-09-12T02:13:08.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.450+0000 7fc7ced33740 1 -- start start 2024-09-12T02:13:08.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.450+0000 7fc7ced33740 1 Processor -- start 2024-09-12T02:13:08.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.450+0000 7fc7ced33740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558334f2b5d0 0x558334f3ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:08.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.450+0000 7fc7ced33740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558334f2cb40 0x558334f39d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:08.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.450+0000 7fc7ced33740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558334f36370 0x558334f3a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:08.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.450+0000 7fc7ced33740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc7c81adc80 con 0x558334f36370 2024-09-12T02:13:08.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.450+0000 7fc7ced33740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc7c81af0f0 con 0x558334f2b5d0 2024-09-12T02:13:08.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.450+0000 7fc7ced33740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc7c81ae350 con 0x558334f2cb40 2024-09-12T02:13:08.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.450+0000 7fc7ced33740 10 client.0 _ll_register_callbacks cb 0x7fc7bc0a5440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:13:08.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.450+0000 7fc7c6d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558334f36370 0x558334f3a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:08.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.450+0000 7fc7c6d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558334f36370 0x558334f3a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55766/0 (socket says 172.21.15.16:55766) 2024-09-12T02:13:08.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.450+0000 7fc7c6d9d640 1 -- 172.21.15.16:0/3484474929 learned_addr learned my addr 172.21.15.16:0/3484474929 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:13:08.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.450+0000 7fc7c659c640 1 --2- 172.21.15.16:0/3484474929 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558334f2b5d0 0x558334f3ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:08.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.451+0000 7fc7c5d9b640 1 --2- 172.21.15.16:0/3484474929 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558334f2cb40 0x558334f39d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:08.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.451+0000 7fc7c6d9d640 1 -- 172.21.15.16:0/3484474929 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558334f2cb40 msgr2=0x558334f39d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:08.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.451+0000 7fc7c6d9d640 1 --2- 172.21.15.16:0/3484474929 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558334f2cb40 0x558334f39d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:08.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.451+0000 7fc7c6d9d640 1 -- 172.21.15.16:0/3484474929 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558334f2b5d0 msgr2=0x558334f3ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:08.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.451+0000 7fc7c6d9d640 1 --2- 172.21.15.16:0/3484474929 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558334f2b5d0 0x558334f3ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:08.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.451+0000 7fc7c6d9d640 1 -- 172.21.15.16:0/3484474929 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558334f3a920 con 0x558334f36370 2024-09-12T02:13:08.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.451+0000 7fc7c5d9b640 1 --2- 172.21.15.16:0/3484474929 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558334f2cb40 0x558334f39d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:13:08.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.451+0000 7fc7c6d9d640 1 --2- 172.21.15.16:0/3484474929 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558334f36370 0x558334f3a2b0 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7fc7c0004310 tx=0x7fc7c0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3016ea84a0a24c6d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:08.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.451+0000 7fc7adffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:13:08.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.451+0000 7fc7adffb640 1 -- 172.21.15.16:0/3484474929 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7c0004b20 con 0x558334f36370 2024-09-12T02:13:08.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.451+0000 7fc7adffb640 1 -- 172.21.15.16:0/3484474929 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fc7c0004cc0 con 0x558334f36370 2024-09-12T02:13:08.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.451+0000 7fc7ced33740 1 -- 172.21.15.16:0/3484474929 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc7bc0bc020 con 0x558334f36370 2024-09-12T02:13:08.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.451+0000 7fc7adffb640 1 -- 172.21.15.16:0/3484474929 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc7c0005020 con 0x558334f36370 2024-09-12T02:13:08.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.451+0000 7fc7ced33740 10 client.19104 fetch_fsmap learned FSMap version 25 2024-09-12T02:13:08.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.451+0000 7fc7ced33740 1 -- 172.21.15.16:0/3484474929 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x558334fb0620 con 0x558334f36370 2024-09-12T02:13:08.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.451+0000 7fc7adffb640 1 -- 172.21.15.16:0/3484474929 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc7bc0bc020 con 0x558334f36370 2024-09-12T02:13:08.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.452+0000 7fc7adffb640 1 -- 172.21.15.16:0/3484474929 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fc7c000f5c0 con 0x558334f36370 2024-09-12T02:13:08.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.452+0000 7fc7ced33740 10 client.19104 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:13:08.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.452+0000 7fc7ced33740 10 client.19104 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:13:08.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.452+0000 7fc7ced33740 20 client.19104 populate_metadata read hostname 'smithi016' 2024-09-12T02:13:08.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.452+0000 7fc7ced33740 1 --2- 172.21.15.16:0/3484474929 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x558334fb0dc0 0x558334fb31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:08.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.452+0000 7fc7ced33740 4 client.19104 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T02:13:08.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.452+0000 7fc7ced33740 1 -- 172.21.15.16:0/3484474929 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558334e98d70 con 0x558334fb0dc0 2024-09-12T02:13:08.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.452+0000 7fc7c659c640 1 --2- 172.21.15.16:0/3484474929 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x558334fb0dc0 0x558334fb31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:08.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.452+0000 7fc7c659c640 1 --2- 172.21.15.16:0/3484474929 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x558334fb0dc0 0x558334fb31e0 crc :-1 s=READY pgs=337 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c34e65d992cb1632 server_cookie=5f1b2d25e5f6e723 in_seq=0 out_seq=0 2024-09-12T02:13:08.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.452+0000 7fc7adffb640 10 client.19104 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:13:08.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.452+0000 7fc7adffb640 1 -- 172.21.15.16:0/3484474929 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558334e98d70 con 0x558334fb0dc0 2024-09-12T02:13:08.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.452+0000 7fc7adffb640 10 client.19104 handle_command_reply: tid=0 2024-09-12T02:13:08.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.452+0000 7fc7ced33740 1 client.19104 shutdown 2024-09-12T02:13:08.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.453+0000 7fc7ced33740 1 -- 172.21.15.16:0/3484474929 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558334f36370 msgr2=0x558334f3a2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:08.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.453+0000 7fc7ced33740 1 --2- 172.21.15.16:0/3484474929 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558334f36370 0x558334f3a2b0 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7fc7c0004310 tx=0x7fc7c0004a40 comp rx=0 tx=0).stop 2024-09-12T02:13:08.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.453+0000 7fc7ced33740 1 -- 172.21.15.16:0/3484474929 shutdown_connections 2024-09-12T02:13:08.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.453+0000 7fc7ced33740 1 --2- 172.21.15.16:0/3484474929 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x558334fb0dc0 0x558334fb31e0 crc :-1 s=READY pgs=337 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:08.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.453+0000 7fc7ced33740 1 --2- 172.21.15.16:0/3484474929 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558334f36370 0x558334f3a2b0 unknown :-1 s=CLOSED pgs=1590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:08.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.453+0000 7fc7ced33740 1 --2- 172.21.15.16:0/3484474929 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558334f2cb40 0x558334f39d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:08.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.453+0000 7fc7ced33740 1 --2- 172.21.15.16:0/3484474929 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558334f2b5d0 0x558334f3ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:08.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.453+0000 7fc7ced33740 1 -- 172.21.15.16:0/3484474929 >> 172.21.15.16:0/3484474929 conn(0x558334ef4310 msgr2=0x558334f320d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:08.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.453+0000 7fc7adffb640 0 client.19104 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:13:08.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.453+0000 7fc7ced33740 1 -- 172.21.15.16:0/3484474929 shutdown_connections 2024-09-12T02:13:08.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.453+0000 7fc7ced33740 1 -- 172.21.15.16:0/3484474929 wait complete. 2024-09-12T02:13:08.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.453+0000 7fc7ced33740 20 client.19104 trim_cache size 0 max 16384 2024-09-12T02:13:08.463 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T02:13:08.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.453+0000 7fc7cc95a640 1 -- 172.21.15.16:0/1343381222 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc7b00a2b80 msgr2=0x7fc7b00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:08.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.453+0000 7fc7cc95a640 1 --2- 172.21.15.16:0/1343381222 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc7b00a2b80 0x7fc7b00a5040 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fc7c00077f0 tx=0x7fc7c0007520 comp rx=0 tx=0).stop 2024-09-12T02:13:08.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.453+0000 7fc7cc95a640 1 -- 172.21.15.16:0/1343381222 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7c81049c0 msgr2=0x7fc7c819ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:08.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.453+0000 7fc7cc95a640 1 --2- 172.21.15.16:0/1343381222 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7c81049c0 0x7fc7c819ab60 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7fc7bc006010 tx=0x7fc7bc0042f0 comp rx=0 tx=0).stop 2024-09-12T02:13:08.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.454+0000 7fc7cc95a640 1 -- 172.21.15.16:0/1343381222 shutdown_connections 2024-09-12T02:13:08.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.454+0000 7fc7cc95a640 1 --2- 172.21.15.16:0/1343381222 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc7b00a2b80 0x7fc7b00a5040 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:08.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.454+0000 7fc7cc95a640 1 --2- 172.21.15.16:0/1343381222 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc7c8109140 0x7fc7c819eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:08.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.454+0000 7fc7cc95a640 1 --2- 172.21.15.16:0/1343381222 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7c81049c0 0x7fc7c819ab60 unknown :-1 s=CLOSED pgs=1586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:08.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.454+0000 7fc7cc95a640 1 --2- 172.21.15.16:0/1343381222 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc7c8104010 0x7fc7c819a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:08.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.454+0000 7fc7cc95a640 1 -- 172.21.15.16:0/1343381222 >> 172.21.15.16:0/1343381222 conn(0x7fc7c80ffa50 msgr2=0x7fc7c8100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:08.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.454+0000 7fc7cc95a640 1 -- 172.21.15.16:0/1343381222 shutdown_connections 2024-09-12T02:13:08.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:08.454+0000 7fc7cc95a640 1 -- 172.21.15.16:0/1343381222 wait complete. 2024-09-12T02:13:10.104 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-12T02:13:10.104 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-12T02:13:10.374 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:13:10.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:13:10 smithi184 ceph-mon[35593]: pgmap v672: 97 pgs: 97 active+clean; 1.9 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-09-12T02:13:10.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:13:10 smithi033 ceph-mon[35332]: pgmap v672: 97 pgs: 97 active+clean; 1.9 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-09-12T02:13:10.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:13:10 smithi016 ceph-mon[30947]: pgmap v672: 97 pgs: 97 active+clean; 1.9 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-09-12T02:13:10.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.887+0000 7f6beee77640 1 -- 172.21.15.16:0/3743103838 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6be81009d0 msgr2=0x7f6be8100db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:10.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.887+0000 7f6beee77640 1 --2- 172.21.15.16:0/3743103838 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6be81009d0 0x7f6be8100db0 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7f6bdc009900 tx=0x7f6bdc02a410 comp rx=0 tx=0).stop 2024-09-12T02:13:10.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.887+0000 7f6beee77640 1 -- 172.21.15.16:0/3743103838 shutdown_connections 2024-09-12T02:13:10.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.887+0000 7f6beee77640 1 --2- 172.21.15.16:0/3743103838 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6be810e4f0 0x7f6be8110900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:10.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.887+0000 7f6beee77640 1 --2- 172.21.15.16:0/3743103838 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6be8101380 0x7f6be810dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:10.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.887+0000 7f6beee77640 1 --2- 172.21.15.16:0/3743103838 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6be81009d0 0x7f6be8100db0 unknown :-1 s=CLOSED pgs=1591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:10.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.887+0000 7f6beee77640 1 -- 172.21.15.16:0/3743103838 >> 172.21.15.16:0/3743103838 conn(0x7f6be80fc7d0 msgr2=0x7f6be80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:10.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.887+0000 7f6beee77640 1 -- 172.21.15.16:0/3743103838 shutdown_connections 2024-09-12T02:13:10.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.887+0000 7f6beee77640 1 -- 172.21.15.16:0/3743103838 wait complete. 2024-09-12T02:13:10.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.888+0000 7f6beee77640 1 Processor -- start 2024-09-12T02:13:10.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.888+0000 7f6beee77640 1 -- start start 2024-09-12T02:13:10.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.888+0000 7f6beee77640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6be81009d0 0x7f6be819a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:10.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.888+0000 7f6beee77640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6be8101380 0x7f6be819ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:10.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.888+0000 7f6beee77640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6be810e4f0 0x7f6be819f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:10.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.888+0000 7f6beee77640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6be8114250 con 0x7f6be8101380 2024-09-12T02:13:10.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.888+0000 7f6beee77640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6be81140d0 con 0x7f6be81009d0 2024-09-12T02:13:10.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.888+0000 7f6beee77640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f6be81143d0 con 0x7f6be810e4f0 2024-09-12T02:13:10.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.888+0000 7f6be7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6be8101380 0x7f6be819ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:10.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.888+0000 7f6be7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6be8101380 0x7f6be819ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55792/0 (socket says 172.21.15.16:55792) 2024-09-12T02:13:10.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.888+0000 7f6be7fff640 1 -- 172.21.15.16:0/3219151366 learned_addr learned my addr 172.21.15.16:0/3219151366 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:13:10.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.888+0000 7f6bed414640 1 --2- 172.21.15.16:0/3219151366 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6be810e4f0 0x7f6be819f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:10.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.888+0000 7f6becc13640 1 --2- 172.21.15.16:0/3219151366 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6be81009d0 0x7f6be819a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:10.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.888+0000 7f6be7fff640 1 -- 172.21.15.16:0/3219151366 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6be810e4f0 msgr2=0x7f6be819f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:10.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.888+0000 7f6be7fff640 1 --2- 172.21.15.16:0/3219151366 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6be810e4f0 0x7f6be819f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:10.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.888+0000 7f6be7fff640 1 -- 172.21.15.16:0/3219151366 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6be81009d0 msgr2=0x7f6be819a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:10.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.888+0000 7f6be7fff640 1 --2- 172.21.15.16:0/3219151366 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6be81009d0 0x7f6be819a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:10.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.888+0000 7f6be7fff640 1 -- 172.21.15.16:0/3219151366 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6be819f700 con 0x7f6be8101380 2024-09-12T02:13:10.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.889+0000 7f6be7fff640 1 --2- 172.21.15.16:0/3219151366 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6be8101380 0x7f6be819ac90 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7f6bd400e9f0 tx=0x7f6bd400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:10.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.889+0000 7f6be5ffb640 1 -- 172.21.15.16:0/3219151366 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bd400cd80 con 0x7f6be8101380 2024-09-12T02:13:10.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.889+0000 7f6beee77640 1 -- 172.21.15.16:0/3219151366 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6be819f9f0 con 0x7f6be8101380 2024-09-12T02:13:10.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.889+0000 7f6be5ffb640 1 -- 172.21.15.16:0/3219151366 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6bd4010910 con 0x7f6be8101380 2024-09-12T02:13:10.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.889+0000 7f6be5ffb640 1 -- 172.21.15.16:0/3219151366 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bd40188f0 con 0x7f6be8101380 2024-09-12T02:13:10.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.889+0000 7f6beee77640 1 -- 172.21.15.16:0/3219151366 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6be81a72d0 con 0x7f6be8101380 2024-09-12T02:13:10.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.892+0000 7f6be5ffb640 1 -- 172.21.15.16:0/3219151366 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6bd4010ac0 con 0x7f6be8101380 2024-09-12T02:13:10.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.892+0000 7f6bf1250740 1 Processor -- start 2024-09-12T02:13:10.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.892+0000 7f6bf1250740 1 -- start start 2024-09-12T02:13:10.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.893+0000 7f6bf1250740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cbb1adbb60 0x55cbb1adbf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:10.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.893+0000 7f6bf1250740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cbb1adc480 0x55cbb1ae6800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:10.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.893+0000 7f6bf1250740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cbb1ae6d40 0x55cbb1ae9130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:10.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.893+0000 7f6bf1250740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55cbb1a41280 con 0x55cbb1adbb60 2024-09-12T02:13:10.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.893+0000 7f6bf1250740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55cbb19e8b10 con 0x55cbb1ae6d40 2024-09-12T02:13:10.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.893+0000 7f6bf1250740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55cbb1a3d240 con 0x55cbb1adc480 2024-09-12T02:13:10.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.893+0000 7f6bed414640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cbb1adbb60 0x55cbb1adbf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:10.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.893+0000 7f6bed414640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cbb1adbb60 0x55cbb1adbf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55802/0 (socket says 172.21.15.16:55802) 2024-09-12T02:13:10.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.893+0000 7f6bed414640 1 -- 172.21.15.16:0/2806656768 learned_addr learned my addr 172.21.15.16:0/2806656768 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:13:10.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.893+0000 7f6becc13640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cbb1adc480 0x55cbb1ae6800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:10.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.893+0000 7f6be5ffb640 1 --2- 172.21.15.16:0/3219151366 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f6bd80a2b80 0x7f6bd80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:10.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.893+0000 7f6be7fff640 1 --2- 172.21.15.16:0/2806656768 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cbb1ae6d40 0x55cbb1ae9130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:10.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.893+0000 7f6becc13640 1 -- 172.21.15.16:0/2806656768 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cbb1ae6d40 msgr2=0x55cbb1ae9130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:10.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.893+0000 7f6becc13640 1 --2- 172.21.15.16:0/2806656768 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cbb1ae6d40 0x55cbb1ae9130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:10.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.893+0000 7f6bed414640 1 --2- 172.21.15.16:0/3219151366 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f6bd80a2b80 0x7f6bd80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:10.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.893+0000 7f6becc13640 1 -- 172.21.15.16:0/2806656768 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cbb1adbb60 msgr2=0x55cbb1adbf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:10.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.893+0000 7f6becc13640 1 --2- 172.21.15.16:0/2806656768 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cbb1adbb60 0x55cbb1adbf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:10.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.893+0000 7f6becc13640 1 -- 172.21.15.16:0/2806656768 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cbb19f51f0 con 0x55cbb1adc480 2024-09-12T02:13:10.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.893+0000 7f6be5ffb640 1 -- 172.21.15.16:0/3219151366 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f6bd40cf3e0 con 0x7f6be8101380 2024-09-12T02:13:10.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.894+0000 7f6becc13640 1 --2- 172.21.15.16:0/2806656768 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cbb1adc480 0x55cbb1ae6800 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6bdc009230 tx=0x7f6bdc032c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9e53421edee967e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:10.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.894+0000 7f6bed414640 1 --2- 172.21.15.16:0/3219151366 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f6bd80a2b80 0x7f6bd80a5040 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f6be0007fd0 tx=0x7f6be0004ab0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:10.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.894+0000 7f6bd2ffd640 1 -- 172.21.15.16:0/2806656768 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bdc035050 con 0x55cbb1adc480 2024-09-12T02:13:10.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.894+0000 7f6bd2ffd640 1 -- 172.21.15.16:0/2806656768 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6bdc040d50 con 0x55cbb1adc480 2024-09-12T02:13:10.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.894+0000 7f6bd2ffd640 1 -- 172.21.15.16:0/2806656768 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bdc0052a0 con 0x55cbb1adc480 2024-09-12T02:13:10.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.895+0000 7f6bf1250740 1 -- 172.21.15.16:0/2806656768 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cbb1adc480 msgr2=0x55cbb1ae6800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:10.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.895+0000 7f6bf1250740 1 --2- 172.21.15.16:0/2806656768 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cbb1adc480 0x55cbb1ae6800 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6bdc009230 tx=0x7f6bdc032c30 comp rx=0 tx=0).stop 2024-09-12T02:13:10.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.895+0000 7f6bf1250740 1 -- 172.21.15.16:0/2806656768 shutdown_connections 2024-09-12T02:13:10.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.895+0000 7f6bf1250740 1 --2- 172.21.15.16:0/2806656768 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cbb1ae6d40 0x55cbb1ae9130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:10.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.895+0000 7f6bf1250740 1 --2- 172.21.15.16:0/2806656768 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cbb1adc480 0x55cbb1ae6800 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:10.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.895+0000 7f6bf1250740 1 --2- 172.21.15.16:0/2806656768 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cbb1adbb60 0x55cbb1adbf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:10.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.895+0000 7f6bf1250740 1 -- 172.21.15.16:0/2806656768 >> 172.21.15.16:0/2806656768 conn(0x55cbb1ad80f0 msgr2=0x55cbb1ad84d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:10.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.895+0000 7f6bf1250740 1 -- 172.21.15.16:0/2806656768 shutdown_connections 2024-09-12T02:13:10.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.895+0000 7f6bf1250740 1 -- 172.21.15.16:0/2806656768 wait complete. 2024-09-12T02:13:10.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.895+0000 7f6bf1250740 1 -- start start 2024-09-12T02:13:10.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.895+0000 7f6bf1250740 1 Processor -- start 2024-09-12T02:13:10.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.896+0000 7f6bf1250740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cbb1adbb60 0x55cbb1ae3c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:10.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.896+0000 7f6bf1250740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cbb1adc480 0x55cbb1ae4140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:10.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.896+0000 7f6bf1250740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cbb1ae6d40 0x55cbb1ae4680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:10.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.896+0000 7f6bf1250740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55cbb1ae9910 con 0x55cbb1adc480 2024-09-12T02:13:10.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.896+0000 7f6bf1250740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55cbb19fbf10 con 0x55cbb1adbb60 2024-09-12T02:13:10.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.896+0000 7f6bf1250740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55cbb1ae9a90 con 0x55cbb1ae6d40 2024-09-12T02:13:10.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.896+0000 7f6becc13640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cbb1adbb60 0x55cbb1ae3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:10.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.896+0000 7f6be7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cbb1adc480 0x55cbb1ae4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:10.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.896+0000 7f6becc13640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cbb1adbb60 0x55cbb1ae3c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.16:37354/0 (socket says 172.21.15.16:37354) 2024-09-12T02:13:10.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.896+0000 7f6becc13640 1 -- 172.21.15.16:0/4073854041 learned_addr learned my addr 172.21.15.16:0/4073854041 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:13:10.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.896+0000 7f6bf1250740 10 client.0 _ll_register_callbacks cb 0x55cbb19c4a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:13:10.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.896+0000 7f6bed414640 1 --2- 172.21.15.16:0/4073854041 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cbb1ae6d40 0x55cbb1ae4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:10.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.896+0000 7f6becc13640 1 -- 172.21.15.16:0/4073854041 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cbb1ae6d40 msgr2=0x55cbb1ae4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:10.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.896+0000 7f6becc13640 1 --2- 172.21.15.16:0/4073854041 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cbb1ae6d40 0x55cbb1ae4680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:10.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.896+0000 7f6becc13640 1 -- 172.21.15.16:0/4073854041 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cbb1adc480 msgr2=0x55cbb1ae4140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:10.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.896+0000 7f6becc13640 1 --2- 172.21.15.16:0/4073854041 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cbb1adc480 0x55cbb1ae4140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:10.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.896+0000 7f6becc13640 1 -- 172.21.15.16:0/4073854041 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cbb17b26a0 con 0x55cbb1adbb60 2024-09-12T02:13:10.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.896+0000 7f6bed414640 1 --2- 172.21.15.16:0/4073854041 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cbb1ae6d40 0x55cbb1ae4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:13:10.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.896+0000 7f6be7fff640 1 --2- 172.21.15.16:0/4073854041 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cbb1adc480 0x55cbb1ae4140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:13:10.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.897+0000 7f6becc13640 1 --2- 172.21.15.16:0/4073854041 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cbb1adbb60 0x55cbb1ae3c00 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f6bdc009900 tx=0x7f6bdc004390 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4cd6bced98c17a91 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:10.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.897+0000 7f6bc7fff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-12T02:13:10.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.897+0000 7f6bc7fff640 1 -- 172.21.15.16:0/4073854041 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bdc005030 con 0x55cbb1adbb60 2024-09-12T02:13:10.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.897+0000 7f6bc7fff640 1 -- 172.21.15.16:0/4073854041 <== mon.2 v2:172.21.15.33:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f6bdc0051d0 con 0x55cbb1adbb60 2024-09-12T02:13:10.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.897+0000 7f6bf1250740 1 -- 172.21.15.16:0/4073854041 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6bdc040d50 con 0x55cbb1adbb60 2024-09-12T02:13:10.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.897+0000 7f6bc7fff640 1 -- 172.21.15.16:0/4073854041 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6bdc0053d0 con 0x55cbb1adbb60 2024-09-12T02:13:10.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.897+0000 7f6bf1250740 10 client.28133 fetch_fsmap learned FSMap version 25 2024-09-12T02:13:10.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.897+0000 7f6bf1250740 1 -- 172.21.15.16:0/4073854041 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55cbb1ae9910 con 0x55cbb1adbb60 2024-09-12T02:13:10.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.897+0000 7f6bc7fff640 1 -- 172.21.15.16:0/4073854041 <== mon.2 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6bdc040d50 con 0x55cbb1adbb60 2024-09-12T02:13:10.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.897+0000 7f6bc7fff640 1 -- 172.21.15.16:0/4073854041 <== mon.2 v2:172.21.15.33:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f6bdc003760 con 0x55cbb1adbb60 2024-09-12T02:13:10.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.897+0000 7f6bf1250740 10 client.28133 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:13:10.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.897+0000 7f6bf1250740 10 client.28133 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:13:10.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.897+0000 7f6bf1250740 20 client.28133 populate_metadata read hostname 'smithi016' 2024-09-12T02:13:10.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.897+0000 7f6bf1250740 1 --2- 172.21.15.16:0/4073854041 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55cbb1af6650 0x55cbb1af8a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:10.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.897+0000 7f6bf1250740 4 client.28133 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:13:10.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.897+0000 7f6bf1250740 1 -- 172.21.15.16:0/4073854041 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cbb1a4e7f0 con 0x55cbb1af6650 2024-09-12T02:13:10.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.898+0000 7f6be7fff640 1 --2- 172.21.15.16:0/4073854041 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55cbb1af6650 0x55cbb1af8a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:10.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.898+0000 7f6bc7fff640 10 client.28133 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:13:10.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.898+0000 7f6be7fff640 1 --2- 172.21.15.16:0/4073854041 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55cbb1af6650 0x55cbb1af8a40 crc :-1 s=READY pgs=338 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=496cae2c568c2ea1 server_cookie=3067e70d294525b5 in_seq=0 out_seq=0 2024-09-12T02:13:10.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.899+0000 7f6bc7fff640 1 -- 172.21.15.16:0/4073854041 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cbb1a4e7f0 con 0x55cbb1af6650 2024-09-12T02:13:10.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.899+0000 7f6bc7fff640 10 client.28133 handle_command_reply: tid=0 2024-09-12T02:13:10.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.899+0000 7f6bf1250740 1 client.28133 shutdown 2024-09-12T02:13:10.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.899+0000 7f6bf1250740 1 -- 172.21.15.16:0/4073854041 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cbb1adbb60 msgr2=0x55cbb1ae3c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:10.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.899+0000 7f6bf1250740 1 --2- 172.21.15.16:0/4073854041 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cbb1adbb60 0x55cbb1ae3c00 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f6bdc009900 tx=0x7f6bdc004390 comp rx=0 tx=0).stop 2024-09-12T02:13:10.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.899+0000 7f6bf1250740 1 -- 172.21.15.16:0/4073854041 shutdown_connections 2024-09-12T02:13:10.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.899+0000 7f6bf1250740 1 --2- 172.21.15.16:0/4073854041 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55cbb1af6650 0x55cbb1af8a40 crc :-1 s=READY pgs=338 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:10.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.899+0000 7f6bf1250740 1 --2- 172.21.15.16:0/4073854041 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cbb1ae6d40 0x55cbb1ae4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:10.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.899+0000 7f6bc7fff640 0 client.28133 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:13:10.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.899+0000 7f6bf1250740 1 --2- 172.21.15.16:0/4073854041 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cbb1adc480 0x55cbb1ae4140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:10.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.899+0000 7f6bf1250740 1 --2- 172.21.15.16:0/4073854041 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cbb1adbb60 0x55cbb1ae3c00 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:10.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.899+0000 7f6bf1250740 1 -- 172.21.15.16:0/4073854041 >> 172.21.15.16:0/4073854041 conn(0x55cbb1ad80f0 msgr2=0x55cbb1adde20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:10.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.899+0000 7f6bf1250740 1 -- 172.21.15.16:0/4073854041 shutdown_connections 2024-09-12T02:13:10.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.899+0000 7f6bf1250740 1 -- 172.21.15.16:0/4073854041 wait complete. 2024-09-12T02:13:10.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.900+0000 7f6bf1250740 20 client.28133 trim_cache size 0 max 16384 2024-09-12T02:13:10.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.917+0000 7f6bf1250740 1 Processor -- start 2024-09-12T02:13:10.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.917+0000 7f6bf1250740 1 -- start start 2024-09-12T02:13:10.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.917+0000 7f6bf1250740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cbb1adbb60 0x55cbb1ae2de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:10.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.917+0000 7f6bf1250740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cbb1adc480 0x55cbb1addcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:10.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.917+0000 7f6bf1250740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cbb1ae6d40 0x55cbb1afe1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:10.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.917+0000 7f6bf1250740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6be81ad960 con 0x55cbb1adbb60 2024-09-12T02:13:10.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.917+0000 7f6bf1250740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6be81ab230 con 0x55cbb1ae6d40 2024-09-12T02:13:10.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.917+0000 7f6bf1250740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f6be81a90c0 con 0x55cbb1adc480 2024-09-12T02:13:10.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.917+0000 7f6becc13640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cbb1adbb60 0x55cbb1ae2de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:10.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.918+0000 7f6becc13640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cbb1adbb60 0x55cbb1ae2de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55826/0 (socket says 172.21.15.16:55826) 2024-09-12T02:13:10.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.918+0000 7f6becc13640 1 -- 172.21.15.16:0/3544991097 learned_addr learned my addr 172.21.15.16:0/3544991097 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:13:10.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.918+0000 7f6bed414640 1 --2- 172.21.15.16:0/3544991097 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cbb1ae6d40 0x55cbb1afe1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:10.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.918+0000 7f6be7fff640 1 --2- 172.21.15.16:0/3544991097 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cbb1adc480 0x55cbb1addcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:10.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.918+0000 7f6becc13640 1 -- 172.21.15.16:0/3544991097 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cbb1adc480 msgr2=0x55cbb1addcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:10.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.918+0000 7f6becc13640 1 --2- 172.21.15.16:0/3544991097 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cbb1adc480 0x55cbb1addcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:10.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.918+0000 7f6becc13640 1 -- 172.21.15.16:0/3544991097 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cbb1ae6d40 msgr2=0x55cbb1afe1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:10.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.918+0000 7f6becc13640 1 --2- 172.21.15.16:0/3544991097 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cbb1ae6d40 0x55cbb1afe1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:10.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.918+0000 7f6becc13640 1 -- 172.21.15.16:0/3544991097 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cbb1ad7f20 con 0x55cbb1adbb60 2024-09-12T02:13:10.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.918+0000 7f6becc13640 1 --2- 172.21.15.16:0/3544991097 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cbb1adbb60 0x55cbb1ae2de0 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7f6bdc02a450 tx=0x7f6bdc004480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=351b263e3e6bc357 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:10.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.918+0000 7f6bd27fc640 1 -- 172.21.15.16:0/3544991097 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bdc035040 con 0x55cbb1adbb60 2024-09-12T02:13:10.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.918+0000 7f6bd27fc640 1 -- 172.21.15.16:0/3544991097 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6bdc04e070 con 0x55cbb1adbb60 2024-09-12T02:13:10.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.918+0000 7f6bd27fc640 1 -- 172.21.15.16:0/3544991097 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bdc04a480 con 0x55cbb1adbb60 2024-09-12T02:13:10.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.919+0000 7f6bf1250740 1 -- 172.21.15.16:0/3544991097 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cbb1adbb60 msgr2=0x55cbb1ae2de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:10.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.919+0000 7f6bf1250740 1 --2- 172.21.15.16:0/3544991097 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cbb1adbb60 0x55cbb1ae2de0 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7f6bdc02a450 tx=0x7f6bdc004480 comp rx=0 tx=0).stop 2024-09-12T02:13:10.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.919+0000 7f6bf1250740 1 -- 172.21.15.16:0/3544991097 shutdown_connections 2024-09-12T02:13:10.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.919+0000 7f6bf1250740 1 --2- 172.21.15.16:0/3544991097 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cbb1ae6d40 0x55cbb1afe1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:10.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.919+0000 7f6bf1250740 1 --2- 172.21.15.16:0/3544991097 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cbb1adc480 0x55cbb1addcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:10.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.919+0000 7f6bf1250740 1 --2- 172.21.15.16:0/3544991097 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cbb1adbb60 0x55cbb1ae2de0 unknown :-1 s=CLOSED pgs=1593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:10.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.919+0000 7f6bf1250740 1 -- 172.21.15.16:0/3544991097 >> 172.21.15.16:0/3544991097 conn(0x55cbb1ad80f0 msgr2=0x55cbb1add5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:10.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.919+0000 7f6bf1250740 1 -- 172.21.15.16:0/3544991097 shutdown_connections 2024-09-12T02:13:10.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.919+0000 7f6bf1250740 1 -- 172.21.15.16:0/3544991097 wait complete. 2024-09-12T02:13:10.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.920+0000 7f6bf1250740 1 -- start start 2024-09-12T02:13:10.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.920+0000 7f6bf1250740 1 Processor -- start 2024-09-12T02:13:10.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.920+0000 7f6bf1250740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cbb1adbb60 0x55cbb1ae0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:10.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.920+0000 7f6bf1250740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cbb1adc480 0x55cbb1ae11e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:10.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.920+0000 7f6bf1250740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cbb1ae6d40 0x55cbb1ae1720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:10.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.920+0000 7f6bf1250740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6be81a9710 con 0x55cbb1adc480 2024-09-12T02:13:10.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.920+0000 7f6bf1250740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6be81aab60 con 0x55cbb1adbb60 2024-09-12T02:13:10.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.920+0000 7f6bf1250740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f6be81a9dc0 con 0x55cbb1ae6d40 2024-09-12T02:13:10.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.920+0000 7f6bf1250740 10 client.0 _ll_register_callbacks cb 0x7f6be00078d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:13:10.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.920+0000 7f6be7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cbb1adc480 0x55cbb1ae11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:10.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.920+0000 7f6be7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cbb1adc480 0x55cbb1ae11e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55828/0 (socket says 172.21.15.16:55828) 2024-09-12T02:13:10.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.920+0000 7f6be7fff640 1 -- 172.21.15.16:0/13353823 learned_addr learned my addr 172.21.15.16:0/13353823 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:13:10.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.920+0000 7f6becc13640 1 --2- 172.21.15.16:0/13353823 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cbb1adbb60 0x55cbb1ae0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:10.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.921+0000 7f6bed414640 1 --2- 172.21.15.16:0/13353823 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cbb1ae6d40 0x55cbb1ae1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:10.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.921+0000 7f6be7fff640 1 -- 172.21.15.16:0/13353823 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cbb1ae6d40 msgr2=0x55cbb1ae1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:10.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.921+0000 7f6be7fff640 1 --2- 172.21.15.16:0/13353823 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cbb1ae6d40 0x55cbb1ae1720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:10.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.921+0000 7f6be7fff640 1 -- 172.21.15.16:0/13353823 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cbb1adbb60 msgr2=0x55cbb1ae0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:10.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.921+0000 7f6be7fff640 1 --2- 172.21.15.16:0/13353823 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cbb1adbb60 0x55cbb1ae0ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:10.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.921+0000 7f6be7fff640 1 -- 172.21.15.16:0/13353823 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cbb1b60320 con 0x55cbb1adc480 2024-09-12T02:13:10.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.921+0000 7f6becc13640 1 --2- 172.21.15.16:0/13353823 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cbb1adbb60 0x55cbb1ae0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:13:10.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.921+0000 7f6be7fff640 1 --2- 172.21.15.16:0/13353823 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cbb1adc480 0x55cbb1ae11e0 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7f6bd4098660 tx=0x7f6bd401f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6f68c7aed2a4502 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:10.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.921+0000 7f6bc7fff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:13:10.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.921+0000 7f6bc7fff640 1 -- 172.21.15.16:0/13353823 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bd40b9060 con 0x55cbb1adc480 2024-09-12T02:13:10.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.921+0000 7f6bc7fff640 1 -- 172.21.15.16:0/13353823 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f6bd40a44c0 con 0x55cbb1adc480 2024-09-12T02:13:10.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.921+0000 7f6bf1250740 1 -- 172.21.15.16:0/13353823 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6bdc04e070 con 0x55cbb1adc480 2024-09-12T02:13:10.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.921+0000 7f6bc7fff640 1 -- 172.21.15.16:0/13353823 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6bd40a4950 con 0x55cbb1adc480 2024-09-12T02:13:10.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.921+0000 7f6bf1250740 10 client.19122 fetch_fsmap learned FSMap version 25 2024-09-12T02:13:10.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.921+0000 7f6bf1250740 1 -- 172.21.15.16:0/13353823 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55cbb1b61680 con 0x55cbb1adc480 2024-09-12T02:13:10.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.922+0000 7f6bc7fff640 1 -- 172.21.15.16:0/13353823 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6bdc04e070 con 0x55cbb1adc480 2024-09-12T02:13:10.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.922+0000 7f6bc7fff640 1 -- 172.21.15.16:0/13353823 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f6bd409b130 con 0x55cbb1adc480 2024-09-12T02:13:10.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.922+0000 7f6bf1250740 10 client.19122 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:13:10.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.922+0000 7f6bf1250740 10 client.19122 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:13:10.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.922+0000 7f6bf1250740 20 client.19122 populate_metadata read hostname 'smithi016' 2024-09-12T02:13:10.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.922+0000 7f6bf1250740 1 --2- 172.21.15.16:0/13353823 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55cbb1b61d90 0x55cbb1b641b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:10.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.922+0000 7f6bf1250740 4 client.19122 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-12T02:13:10.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.922+0000 7f6bf1250740 1 -- 172.21.15.16:0/13353823 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55cbb1a48d00 con 0x55cbb1b61d90 2024-09-12T02:13:10.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.922+0000 7f6becc13640 1 --2- 172.21.15.16:0/13353823 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55cbb1b61d90 0x55cbb1b641b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:10.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.923+0000 7f6becc13640 1 --2- 172.21.15.16:0/13353823 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55cbb1b61d90 0x55cbb1b641b0 crc :-1 s=READY pgs=339 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fee5bcfd3e223160 server_cookie=e6e5a60bc25da0cd in_seq=0 out_seq=0 2024-09-12T02:13:10.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.923+0000 7f6bc7fff640 10 client.19122 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:13:10.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.924+0000 7f6bc7fff640 1 -- 172.21.15.16:0/13353823 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55cbb1a48d00 con 0x55cbb1b61d90 2024-09-12T02:13:10.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.924+0000 7f6bc7fff640 10 client.19122 handle_command_reply: tid=0 2024-09-12T02:13:10.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.924+0000 7f6bf1250740 1 client.19122 shutdown 2024-09-12T02:13:10.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.924+0000 7f6bf1250740 1 -- 172.21.15.16:0/13353823 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cbb1adc480 msgr2=0x55cbb1ae11e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:10.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.924+0000 7f6bf1250740 1 --2- 172.21.15.16:0/13353823 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cbb1adc480 0x55cbb1ae11e0 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7f6bd4098660 tx=0x7f6bd401f000 comp rx=0 tx=0).stop 2024-09-12T02:13:10.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.924+0000 7f6bf1250740 1 -- 172.21.15.16:0/13353823 shutdown_connections 2024-09-12T02:13:10.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.924+0000 7f6bf1250740 1 --2- 172.21.15.16:0/13353823 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55cbb1b61d90 0x55cbb1b641b0 crc :-1 s=READY pgs=339 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:10.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.924+0000 7f6bf1250740 1 --2- 172.21.15.16:0/13353823 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cbb1ae6d40 0x55cbb1ae1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:10.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.924+0000 7f6bf1250740 1 --2- 172.21.15.16:0/13353823 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cbb1adc480 0x55cbb1ae11e0 unknown :-1 s=CLOSED pgs=1594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:10.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.924+0000 7f6bf1250740 1 --2- 172.21.15.16:0/13353823 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cbb1adbb60 0x55cbb1ae0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:10.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.924+0000 7f6bf1250740 1 -- 172.21.15.16:0/13353823 >> 172.21.15.16:0/13353823 conn(0x55cbb1ad80f0 msgr2=0x55cbb1ad7270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:10.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.924+0000 7f6bc7fff640 0 client.19122 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:13:10.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.924+0000 7f6bf1250740 1 -- 172.21.15.16:0/13353823 shutdown_connections 2024-09-12T02:13:10.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.924+0000 7f6bf1250740 1 -- 172.21.15.16:0/13353823 wait complete. 2024-09-12T02:13:10.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.924+0000 7f6bf1250740 20 client.19122 trim_cache size 0 max 16384 2024-09-12T02:13:10.928 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T02:13:10.929 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-09-12T02:13:10.929 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "4aa01b9a-4000-4879-bf32-34d27c524a9b", 2024-09-12T02:13:10.929 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-09-12T02:13:10.929 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T02:13:10.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.925+0000 7f6beee77640 1 -- 172.21.15.16:0/3219151366 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f6bd80a2b80 msgr2=0x7f6bd80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:10.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.925+0000 7f6beee77640 1 --2- 172.21.15.16:0/3219151366 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f6bd80a2b80 0x7f6bd80a5040 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f6be0007fd0 tx=0x7f6be0004ab0 comp rx=0 tx=0).stop 2024-09-12T02:13:10.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.925+0000 7f6beee77640 1 -- 172.21.15.16:0/3219151366 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6be8101380 msgr2=0x7f6be819ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:10.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.925+0000 7f6beee77640 1 --2- 172.21.15.16:0/3219151366 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6be8101380 0x7f6be819ac90 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7f6bd400e9f0 tx=0x7f6bd400eeb0 comp rx=0 tx=0).stop 2024-09-12T02:13:10.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.925+0000 7f6beee77640 1 -- 172.21.15.16:0/3219151366 shutdown_connections 2024-09-12T02:13:10.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.925+0000 7f6beee77640 1 --2- 172.21.15.16:0/3219151366 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f6bd80a2b80 0x7f6bd80a5040 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:10.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.925+0000 7f6beee77640 1 --2- 172.21.15.16:0/3219151366 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6be810e4f0 0x7f6be819f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:10.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.925+0000 7f6beee77640 1 --2- 172.21.15.16:0/3219151366 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6be8101380 0x7f6be819ac90 unknown :-1 s=CLOSED pgs=1592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:10.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.925+0000 7f6beee77640 1 --2- 172.21.15.16:0/3219151366 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6be81009d0 0x7f6be819a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:10.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.925+0000 7f6beee77640 1 -- 172.21.15.16:0/3219151366 >> 172.21.15.16:0/3219151366 conn(0x7f6be80fc7d0 msgr2=0x7f6be80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:10.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.925+0000 7f6beee77640 1 -- 172.21.15.16:0/3219151366 shutdown_connections 2024-09-12T02:13:10.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:10.925+0000 7f6beee77640 1 -- 172.21.15.16:0/3219151366 wait complete. 2024-09-12T02:13:11.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:13:11 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:13:11.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:13:11 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T02:13:11.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:13:11 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T02:13:11.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:13:11 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:13:11.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:13:11 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T02:13:11.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:13:11 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T02:13:11.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:13:11 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:13:11.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:13:11 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T02:13:11.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:13:11 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T02:13:11.593 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T02:13:11.848 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:13:12.233 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:13:12 smithi016 ceph-mon[30947]: pgmap v673: 97 pgs: 97 active+clean; 1.9 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-09-12T02:13:12.234 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:13:12 smithi016 ceph-mon[30947]: scrub queued for path: / 2024-09-12T02:13:12.234 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:13:12 smithi016 ceph-mon[30947]: scrub summary: idle+waiting paths [/] 2024-09-12T02:13:12.234 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:13:12 smithi016 ceph-mon[30947]: scrub summary: active paths [/] 2024-09-12T02:13:12.234 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:13:12 smithi016 ceph-mon[30947]: scrub summary: idle+waiting paths [/] 2024-09-12T02:13:12.234 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:13:12 smithi016 ceph-mon[30947]: scrub summary: idle 2024-09-12T02:13:12.234 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:13:12 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:13:12.234 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:13:12 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:13:12.234 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:13:12 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T02:13:12.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.317+0000 7f164621a640 1 -- 172.21.15.16:0/1101201270 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1640102d40 msgr2=0x7f1640103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:12.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.317+0000 7f164621a640 1 --2- 172.21.15.16:0/1101201270 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1640102d40 0x7f1640103120 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7f1630009900 tx=0x7f163002a480 comp rx=0 tx=0).stop 2024-09-12T02:13:12.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.317+0000 7f164621a640 1 -- 172.21.15.16:0/1101201270 shutdown_connections 2024-09-12T02:13:12.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.317+0000 7f164621a640 1 --2- 172.21.15.16:0/1101201270 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f164010c170 0x7f164010e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:12.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.317+0000 7f164621a640 1 --2- 172.21.15.16:0/1101201270 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1640103660 0x7f164010bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:12.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.317+0000 7f164621a640 1 --2- 172.21.15.16:0/1101201270 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1640102d40 0x7f1640103120 unknown :-1 s=CLOSED pgs=1595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:12.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.317+0000 7f164621a640 1 -- 172.21.15.16:0/1101201270 >> 172.21.15.16:0/1101201270 conn(0x7f16400fc7a0 msgr2=0x7f16400febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:12.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.317+0000 7f164621a640 1 -- 172.21.15.16:0/1101201270 shutdown_connections 2024-09-12T02:13:12.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.317+0000 7f164621a640 1 -- 172.21.15.16:0/1101201270 wait complete. 2024-09-12T02:13:12.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.318+0000 7f164621a640 1 Processor -- start 2024-09-12T02:13:12.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.318+0000 7f164621a640 1 -- start start 2024-09-12T02:13:12.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.319+0000 7f164621a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1640102d40 0x7f164019a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:12.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.319+0000 7f164621a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1640103660 0x7f164019aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:12.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.319+0000 7f164621a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f164010c170 0x7f164019ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:12.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.319+0000 7f164621a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1640111fb0 con 0x7f1640103660 2024-09-12T02:13:12.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.319+0000 7f164621a640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1640111e30 con 0x7f164010c170 2024-09-12T02:13:12.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.319+0000 7f164621a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1640112130 con 0x7f1640102d40 2024-09-12T02:13:12.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.319+0000 7f163effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1640103660 0x7f164019aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:12.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.319+0000 7f163effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1640103660 0x7f164019aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39364/0 (socket says 172.21.15.16:39364) 2024-09-12T02:13:12.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.319+0000 7f163effd640 1 -- 172.21.15.16:0/4002644438 learned_addr learned my addr 172.21.15.16:0/4002644438 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:13:12.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.319+0000 7f163ffff640 1 --2- 172.21.15.16:0/4002644438 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f164010c170 0x7f164019ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:12.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.319+0000 7f163f7fe640 1 --2- 172.21.15.16:0/4002644438 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1640102d40 0x7f164019a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:12.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.319+0000 7f163effd640 1 -- 172.21.15.16:0/4002644438 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1640102d40 msgr2=0x7f164019a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:12.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.319+0000 7f163effd640 1 --2- 172.21.15.16:0/4002644438 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1640102d40 0x7f164019a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:12.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.319+0000 7f163effd640 1 -- 172.21.15.16:0/4002644438 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f164010c170 msgr2=0x7f164019ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:12.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.319+0000 7f163effd640 1 --2- 172.21.15.16:0/4002644438 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f164010c170 0x7f164019ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:12.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.319+0000 7f163effd640 1 -- 172.21.15.16:0/4002644438 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f164019f610 con 0x7f1640103660 2024-09-12T02:13:12.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.319+0000 7f163f7fe640 1 --2- 172.21.15.16:0/4002644438 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1640102d40 0x7f164019a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:13:12.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.319+0000 7f163effd640 1 --2- 172.21.15.16:0/4002644438 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1640103660 0x7f164019aba0 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7f163400d740 tx=0x7f163400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:12.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.319+0000 7f163cff9640 1 -- 172.21.15.16:0/4002644438 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1634017070 con 0x7f1640103660 2024-09-12T02:13:12.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.319+0000 7f163cff9640 1 -- 172.21.15.16:0/4002644438 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f163400de50 con 0x7f1640103660 2024-09-12T02:13:12.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.319+0000 7f164621a640 1 -- 172.21.15.16:0/4002644438 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f164019f900 con 0x7f1640103660 2024-09-12T02:13:12.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.319+0000 7f163cff9640 1 -- 172.21.15.16:0/4002644438 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f163400f620 con 0x7f1640103660 2024-09-12T02:13:12.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.320+0000 7f164621a640 1 -- 172.21.15.16:0/4002644438 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16401a71e0 con 0x7f1640103660 2024-09-12T02:13:12.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.322+0000 7f163cff9640 1 -- 172.21.15.16:0/4002644438 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f1634009c30 con 0x7f1640103660 2024-09-12T02:13:12.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.322+0000 7f16485f3740 1 Processor -- start 2024-09-12T02:13:12.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.322+0000 7f16485f3740 1 -- start start 2024-09-12T02:13:12.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.322+0000 7f16485f3740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563acc4a74a0 0x563acc4a7880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:12.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.322+0000 7f16485f3740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563acc4a7dc0 0x563acc4b2140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:12.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.322+0000 7f16485f3740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563acc4b2680 0x563acc4b4a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:12.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.322+0000 7f16485f3740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x563acc40d2e0 con 0x563acc4a74a0 2024-09-12T02:13:12.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.322+0000 7f16485f3740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x563acc3b4ce0 con 0x563acc4b2680 2024-09-12T02:13:12.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.322+0000 7f16485f3740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x563acc4092a0 con 0x563acc4a7dc0 2024-09-12T02:13:12.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.322+0000 7f163ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563acc4a74a0 0x563acc4a7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:12.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.323+0000 7f163ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563acc4a74a0 0x563acc4a7880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39366/0 (socket says 172.21.15.16:39366) 2024-09-12T02:13:12.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.323+0000 7f163ffff640 1 -- 172.21.15.16:0/1345293284 learned_addr learned my addr 172.21.15.16:0/1345293284 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:13:12.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.323+0000 7f163effd640 1 --2- 172.21.15.16:0/1345293284 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563acc4b2680 0x563acc4b4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:12.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.323+0000 7f163f7fe640 1 --2- 172.21.15.16:0/1345293284 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563acc4a7dc0 0x563acc4b2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:12.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.323+0000 7f163cff9640 1 --2- 172.21.15.16:0/4002644438 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f16280a2c00 0x7f16280a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:12.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.323+0000 7f163ffff640 1 -- 172.21.15.16:0/1345293284 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563acc4a7dc0 msgr2=0x563acc4b2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:12.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.323+0000 7f163ffff640 1 --2- 172.21.15.16:0/1345293284 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563acc4a7dc0 0x563acc4b2140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:12.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.323+0000 7f163ffff640 1 -- 172.21.15.16:0/1345293284 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563acc4b2680 msgr2=0x563acc4b4a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:12.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.323+0000 7f163ffff640 1 --2- 172.21.15.16:0/1345293284 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563acc4b2680 0x563acc4b4a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:12.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.323+0000 7f163ffff640 1 -- 172.21.15.16:0/1345293284 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563acc399440 con 0x563acc4a74a0 2024-09-12T02:13:12.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.323+0000 7f163f7fe640 1 --2- 172.21.15.16:0/1345293284 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563acc4a7dc0 0x563acc4b2140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:13:12.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.323+0000 7f163cff9640 1 -- 172.21.15.16:0/4002644438 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f16340ceb20 con 0x7f1640103660 2024-09-12T02:13:12.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.323+0000 7f163ffff640 1 --2- 172.21.15.16:0/4002644438 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f16280a2c00 0x7f16280a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:12.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.323+0000 7f163ffff640 1 --2- 172.21.15.16:0/1345293284 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563acc4a74a0 0x563acc4a7880 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7f163800a5d0 tx=0x7f163800aa90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a897696e36ba7deb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:12.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.323+0000 7f1625ffb640 1 -- 172.21.15.16:0/1345293284 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f163801a040 con 0x563acc4a74a0 2024-09-12T02:13:12.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.323+0000 7f1625ffb640 1 -- 172.21.15.16:0/1345293284 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1638021ba0 con 0x563acc4a74a0 2024-09-12T02:13:12.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.323+0000 7f1625ffb640 1 -- 172.21.15.16:0/1345293284 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f163801e560 con 0x563acc4a74a0 2024-09-12T02:13:12.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.324+0000 7f16485f3740 1 -- 172.21.15.16:0/1345293284 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563acc4a74a0 msgr2=0x563acc4a7880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:12.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.324+0000 7f163ffff640 1 --2- 172.21.15.16:0/4002644438 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f16280a2c00 0x7f16280a50c0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f1638020000 tx=0x7f1638021000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:12.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.324+0000 7f16485f3740 1 --2- 172.21.15.16:0/1345293284 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563acc4a74a0 0x563acc4a7880 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7f163800a5d0 tx=0x7f163800aa90 comp rx=0 tx=0).stop 2024-09-12T02:13:12.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.324+0000 7f16485f3740 1 -- 172.21.15.16:0/1345293284 shutdown_connections 2024-09-12T02:13:12.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.324+0000 7f16485f3740 1 --2- 172.21.15.16:0/1345293284 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563acc4b2680 0x563acc4b4a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:12.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.324+0000 7f16485f3740 1 --2- 172.21.15.16:0/1345293284 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563acc4a7dc0 0x563acc4b2140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:12.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.324+0000 7f16485f3740 1 --2- 172.21.15.16:0/1345293284 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563acc4a74a0 0x563acc4a7880 unknown :-1 s=CLOSED pgs=1597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:12.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.324+0000 7f16485f3740 1 -- 172.21.15.16:0/1345293284 >> 172.21.15.16:0/1345293284 conn(0x563acc4a3a30 msgr2=0x563acc4a3e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:12.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.324+0000 7f16485f3740 1 -- 172.21.15.16:0/1345293284 shutdown_connections 2024-09-12T02:13:12.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.324+0000 7f16485f3740 1 -- 172.21.15.16:0/1345293284 wait complete. 2024-09-12T02:13:12.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.325+0000 7f16485f3740 1 -- start start 2024-09-12T02:13:12.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.325+0000 7f16485f3740 1 Processor -- start 2024-09-12T02:13:12.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.325+0000 7f16485f3740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563acc4a74a0 0x563acc4b1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:12.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.325+0000 7f16485f3740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563acc4a7dc0 0x563acc4af270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:12.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.325+0000 7f16485f3740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563acc4b2680 0x563acc4af7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:12.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.325+0000 7f16485f3740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x563acc4b5250 con 0x563acc4a7dc0 2024-09-12T02:13:12.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.325+0000 7f16485f3740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x563acc3c8050 con 0x563acc4a74a0 2024-09-12T02:13:12.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.325+0000 7f16485f3740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x563acc4b53d0 con 0x563acc4b2680 2024-09-12T02:13:12.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.325+0000 7f16485f3740 10 client.0 _ll_register_callbacks cb 0x563acc408160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:13:12.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.325+0000 7f163effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563acc4a7dc0 0x563acc4af270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:12.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.325+0000 7f163effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563acc4a7dc0 0x563acc4af270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39370/0 (socket says 172.21.15.16:39370) 2024-09-12T02:13:12.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.325+0000 7f163effd640 1 -- 172.21.15.16:0/1214155431 learned_addr learned my addr 172.21.15.16:0/1214155431 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:13:12.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.326+0000 7f163f7fe640 1 --2- 172.21.15.16:0/1214155431 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563acc4a74a0 0x563acc4b1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:12.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.326+0000 7f163effd640 1 -- 172.21.15.16:0/1214155431 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563acc4b2680 msgr2=0x563acc4af7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:12.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.326+0000 7f163effd640 1 --2- 172.21.15.16:0/1214155431 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563acc4b2680 0x563acc4af7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:12.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.326+0000 7f163effd640 1 -- 172.21.15.16:0/1214155431 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563acc4a74a0 msgr2=0x563acc4b1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:12.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.326+0000 7f163effd640 1 --2- 172.21.15.16:0/1214155431 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563acc4a74a0 0x563acc4b1fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:12.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.326+0000 7f163effd640 1 -- 172.21.15.16:0/1214155431 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563acc17e3c0 con 0x563acc4a7dc0 2024-09-12T02:13:12.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.326+0000 7f163f7fe640 1 --2- 172.21.15.16:0/1214155431 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563acc4a74a0 0x563acc4b1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:13:12.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.326+0000 7f163effd640 1 --2- 172.21.15.16:0/1214155431 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563acc4a7dc0 0x563acc4af270 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7f1634005fd0 tx=0x7f16340cee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20841277fba121b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:12.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.326+0000 7f161effd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:13:12.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.326+0000 7f161effd640 1 -- 172.21.15.16:0/1214155431 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1634020040 con 0x563acc4a7dc0 2024-09-12T02:13:12.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.326+0000 7f161effd640 1 -- 172.21.15.16:0/1214155431 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f163409a870 con 0x563acc4a7dc0 2024-09-12T02:13:12.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.326+0000 7f16485f3740 1 -- 172.21.15.16:0/1214155431 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1638021ba0 con 0x563acc4a7dc0 2024-09-12T02:13:12.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.326+0000 7f161effd640 1 -- 172.21.15.16:0/1214155431 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f16340d3040 con 0x563acc4a7dc0 2024-09-12T02:13:12.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.326+0000 7f16485f3740 10 client.19134 fetch_fsmap learned FSMap version 25 2024-09-12T02:13:12.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.326+0000 7f16485f3740 1 -- 172.21.15.16:0/1214155431 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x563acc4c0c50 con 0x563acc4a7dc0 2024-09-12T02:13:12.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.326+0000 7f161effd640 1 -- 172.21.15.16:0/1214155431 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1638021ba0 con 0x563acc4a7dc0 2024-09-12T02:13:12.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.327+0000 7f161effd640 1 -- 172.21.15.16:0/1214155431 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f16340b4190 con 0x563acc4a7dc0 2024-09-12T02:13:12.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.327+0000 7f16485f3740 10 client.19134 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:13:12.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.327+0000 7f16485f3740 10 client.19134 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:13:12.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.327+0000 7f16485f3740 20 client.19134 populate_metadata read hostname 'smithi016' 2024-09-12T02:13:12.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.327+0000 7f16485f3740 1 --2- 172.21.15.16:0/1214155431 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x563acc4c2110 0x563acc4c45d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:12.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.327+0000 7f16485f3740 4 client.19134 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:13:12.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.327+0000 7f16485f3740 1 -- 172.21.15.16:0/1214155431 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563acc41a850 con 0x563acc4c2110 2024-09-12T02:13:12.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.327+0000 7f163f7fe640 1 --2- 172.21.15.16:0/1214155431 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x563acc4c2110 0x563acc4c45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:12.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.327+0000 7f163f7fe640 1 --2- 172.21.15.16:0/1214155431 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x563acc4c2110 0x563acc4c45d0 crc :-1 s=READY pgs=340 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3bbad47ddee624e server_cookie=88dbc11203545d21 in_seq=0 out_seq=0 2024-09-12T02:13:12.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.327+0000 7f161effd640 10 client.19134 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:13:12.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.328+0000 7f161effd640 1 -- 172.21.15.16:0/1214155431 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563acc41a850 con 0x563acc4c2110 2024-09-12T02:13:12.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.328+0000 7f161effd640 10 client.19134 handle_command_reply: tid=0 2024-09-12T02:13:12.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.328+0000 7f16485f3740 1 client.19134 shutdown 2024-09-12T02:13:12.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.328+0000 7f16485f3740 1 -- 172.21.15.16:0/1214155431 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563acc4a7dc0 msgr2=0x563acc4af270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:12.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.328+0000 7f16485f3740 1 --2- 172.21.15.16:0/1214155431 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563acc4a7dc0 0x563acc4af270 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7f1634005fd0 tx=0x7f16340cee20 comp rx=0 tx=0).stop 2024-09-12T02:13:12.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.328+0000 7f16485f3740 1 -- 172.21.15.16:0/1214155431 shutdown_connections 2024-09-12T02:13:12.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.328+0000 7f16485f3740 1 --2- 172.21.15.16:0/1214155431 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x563acc4c2110 0x563acc4c45d0 crc :-1 s=READY pgs=340 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:12.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.328+0000 7f16485f3740 1 --2- 172.21.15.16:0/1214155431 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563acc4b2680 0x563acc4af7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:12.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.328+0000 7f16485f3740 1 --2- 172.21.15.16:0/1214155431 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563acc4a7dc0 0x563acc4af270 unknown :-1 s=CLOSED pgs=1598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:12.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.328+0000 7f16485f3740 1 --2- 172.21.15.16:0/1214155431 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563acc4a74a0 0x563acc4b1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:12.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.328+0000 7f16485f3740 1 -- 172.21.15.16:0/1214155431 >> 172.21.15.16:0/1214155431 conn(0x563acc4a3a30 msgr2=0x563acc4a9710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:12.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.328+0000 7f161effd640 0 client.19134 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:13:12.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.328+0000 7f16485f3740 1 -- 172.21.15.16:0/1214155431 shutdown_connections 2024-09-12T02:13:12.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.328+0000 7f16485f3740 1 -- 172.21.15.16:0/1214155431 wait complete. 2024-09-12T02:13:12.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.329+0000 7f16485f3740 20 client.19134 trim_cache size 0 max 16384 2024-09-12T02:13:12.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.346+0000 7f16485f3740 1 Processor -- start 2024-09-12T02:13:12.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.346+0000 7f16485f3740 1 -- start start 2024-09-12T02:13:12.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.346+0000 7f16485f3740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563acc4a74a0 0x563acc4be4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:12.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.346+0000 7f16485f3740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563acc4a7dc0 0x563acc4c9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:12.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.346+0000 7f16485f3740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563acc4b2680 0x563acc4ca430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:12.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.346+0000 7f16485f3740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f16401b1f60 con 0x563acc4a7dc0 2024-09-12T02:13:12.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.346+0000 7f16485f3740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f16401af830 con 0x563acc4a74a0 2024-09-12T02:13:12.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.346+0000 7f16485f3740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f16401ad680 con 0x563acc4b2680 2024-09-12T02:13:12.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.346+0000 7f163effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563acc4a7dc0 0x563acc4c9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:12.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.346+0000 7f163effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563acc4a7dc0 0x563acc4c9ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39378/0 (socket says 172.21.15.16:39378) 2024-09-12T02:13:12.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.346+0000 7f163effd640 1 -- 172.21.15.16:0/1889783708 learned_addr learned my addr 172.21.15.16:0/1889783708 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:13:12.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.346+0000 7f163f7fe640 1 --2- 172.21.15.16:0/1889783708 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563acc4a74a0 0x563acc4be4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:12.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.346+0000 7f163ffff640 1 --2- 172.21.15.16:0/1889783708 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563acc4b2680 0x563acc4ca430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:12.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.346+0000 7f163effd640 1 -- 172.21.15.16:0/1889783708 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563acc4b2680 msgr2=0x563acc4ca430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:12.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.346+0000 7f163effd640 1 --2- 172.21.15.16:0/1889783708 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563acc4b2680 0x563acc4ca430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:12.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.346+0000 7f163effd640 1 -- 172.21.15.16:0/1889783708 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563acc4a74a0 msgr2=0x563acc4be4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:12.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.346+0000 7f163effd640 1 --2- 172.21.15.16:0/1889783708 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563acc4a74a0 0x563acc4be4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:12.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.346+0000 7f163effd640 1 -- 172.21.15.16:0/1889783708 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563acc4a7290 con 0x563acc4a7dc0 2024-09-12T02:13:12.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.346+0000 7f163ffff640 1 --2- 172.21.15.16:0/1889783708 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563acc4b2680 0x563acc4ca430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:13:12.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.347+0000 7f163effd640 1 --2- 172.21.15.16:0/1889783708 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563acc4a7dc0 0x563acc4c9ef0 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7f163409b3b0 tx=0x7f1634099930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2daacf2b9b65a2d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:12.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.347+0000 7f16257fa640 1 -- 172.21.15.16:0/1889783708 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16340b9030 con 0x563acc4a7dc0 2024-09-12T02:13:12.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.347+0000 7f16257fa640 1 -- 172.21.15.16:0/1889783708 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f163409b420 con 0x563acc4a7dc0 2024-09-12T02:13:12.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.347+0000 7f16257fa640 1 -- 172.21.15.16:0/1889783708 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1634020050 con 0x563acc4a7dc0 2024-09-12T02:13:12.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.347+0000 7f16485f3740 1 -- 172.21.15.16:0/1889783708 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563acc4a7dc0 msgr2=0x563acc4c9ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:12.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.347+0000 7f16485f3740 1 --2- 172.21.15.16:0/1889783708 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563acc4a7dc0 0x563acc4c9ef0 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7f163409b3b0 tx=0x7f1634099930 comp rx=0 tx=0).stop 2024-09-12T02:13:12.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.347+0000 7f16485f3740 1 -- 172.21.15.16:0/1889783708 shutdown_connections 2024-09-12T02:13:12.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.347+0000 7f16485f3740 1 --2- 172.21.15.16:0/1889783708 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563acc4b2680 0x563acc4ca430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:12.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.347+0000 7f16485f3740 1 --2- 172.21.15.16:0/1889783708 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563acc4a7dc0 0x563acc4c9ef0 unknown :-1 s=CLOSED pgs=1599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:12.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.347+0000 7f16485f3740 1 --2- 172.21.15.16:0/1889783708 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563acc4a74a0 0x563acc4be4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:12.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.347+0000 7f16485f3740 1 -- 172.21.15.16:0/1889783708 >> 172.21.15.16:0/1889783708 conn(0x563acc4a3a30 msgr2=0x563acc4ae360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:12.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.347+0000 7f16485f3740 1 -- 172.21.15.16:0/1889783708 shutdown_connections 2024-09-12T02:13:12.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.347+0000 7f16485f3740 1 -- 172.21.15.16:0/1889783708 wait complete. 2024-09-12T02:13:12.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.348+0000 7f16485f3740 1 -- start start 2024-09-12T02:13:12.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.348+0000 7f16485f3740 1 Processor -- start 2024-09-12T02:13:12.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.348+0000 7f16485f3740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563acc4a74a0 0x563acc5252e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:12.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.348+0000 7f16485f3740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563acc4a7dc0 0x563acc525820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:12.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.348+0000 7f16485f3740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563acc4b2680 0x563acc4b7c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:12.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.348+0000 7f16485f3740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f16401adcf0 con 0x563acc4a74a0 2024-09-12T02:13:12.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.348+0000 7f16485f3740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f16401af160 con 0x563acc4b2680 2024-09-12T02:13:12.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.348+0000 7f16485f3740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f16401ae3c0 con 0x563acc4a7dc0 2024-09-12T02:13:12.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.348+0000 7f163f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563acc4a74a0 0x563acc5252e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:12.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.348+0000 7f16485f3740 10 client.0 _ll_register_callbacks cb 0x7f163409b2e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:13:12.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.348+0000 7f163f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563acc4a74a0 0x563acc5252e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39380/0 (socket says 172.21.15.16:39380) 2024-09-12T02:13:12.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.348+0000 7f163f7fe640 1 -- 172.21.15.16:0/61488171 learned_addr learned my addr 172.21.15.16:0/61488171 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:13:12.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.349+0000 7f163ffff640 1 --2- 172.21.15.16:0/61488171 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563acc4b2680 0x563acc4b7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:12.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.349+0000 7f163f7fe640 1 -- 172.21.15.16:0/61488171 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563acc4a7dc0 msgr2=0x563acc525820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:12.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.349+0000 7f163f7fe640 1 --2- 172.21.15.16:0/61488171 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563acc4a7dc0 0x563acc525820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:12.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.349+0000 7f163f7fe640 1 -- 172.21.15.16:0/61488171 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563acc4b2680 msgr2=0x563acc4b7c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:12.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.349+0000 7f163f7fe640 1 --2- 172.21.15.16:0/61488171 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563acc4b2680 0x563acc4b7c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:12.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.349+0000 7f163f7fe640 1 -- 172.21.15.16:0/61488171 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563acc4b8410 con 0x563acc4a74a0 2024-09-12T02:13:12.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.349+0000 7f163f7fe640 1 --2- 172.21.15.16:0/61488171 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563acc4a74a0 0x563acc5252e0 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f1630009230 tx=0x7f1630004100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99d2e26ce0d5a3d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:12.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.349+0000 7f161effd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:13:12.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.349+0000 7f161effd640 1 -- 172.21.15.16:0/61488171 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f163004b060 con 0x563acc4a74a0 2024-09-12T02:13:12.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.349+0000 7f161effd640 1 -- 172.21.15.16:0/61488171 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f1630005240 con 0x563acc4a74a0 2024-09-12T02:13:12.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.349+0000 7f16485f3740 1 -- 172.21.15.16:0/61488171 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f163409b420 con 0x563acc4a74a0 2024-09-12T02:13:12.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.349+0000 7f161effd640 1 -- 172.21.15.16:0/61488171 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1630035040 con 0x563acc4a74a0 2024-09-12T02:13:12.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.349+0000 7f16485f3740 10 client.19140 fetch_fsmap learned FSMap version 25 2024-09-12T02:13:12.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.349+0000 7f16485f3740 1 -- 172.21.15.16:0/61488171 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x563acc52ca20 con 0x563acc4a74a0 2024-09-12T02:13:12.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.350+0000 7f161effd640 1 -- 172.21.15.16:0/61488171 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f163409b420 con 0x563acc4a74a0 2024-09-12T02:13:12.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.350+0000 7f161effd640 1 -- 172.21.15.16:0/61488171 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f1630050070 con 0x563acc4a74a0 2024-09-12T02:13:12.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.350+0000 7f16485f3740 10 client.19140 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:13:12.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.350+0000 7f16485f3740 10 client.19140 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:13:12.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.350+0000 7f16485f3740 20 client.19140 populate_metadata read hostname 'smithi016' 2024-09-12T02:13:12.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.350+0000 7f16485f3740 1 --2- 172.21.15.16:0/61488171 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x563acc52d0f0 0x563acc52f530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:12.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.350+0000 7f16485f3740 4 client.19140 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T02:13:12.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.350+0000 7f16485f3740 1 -- 172.21.15.16:0/61488171 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563acc414d60 con 0x563acc52d0f0 2024-09-12T02:13:12.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.350+0000 7f163effd640 1 --2- 172.21.15.16:0/61488171 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x563acc52d0f0 0x563acc52f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:12.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.351+0000 7f163effd640 1 --2- 172.21.15.16:0/61488171 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x563acc52d0f0 0x563acc52f530 crc :-1 s=READY pgs=341 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6cb5d0117608d150 server_cookie=e5c8d100af03697b in_seq=0 out_seq=0 2024-09-12T02:13:12.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.351+0000 7f161effd640 10 client.19140 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:13:12.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.351+0000 7f161effd640 1 -- 172.21.15.16:0/61488171 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x563acc414d60 con 0x563acc52d0f0 2024-09-12T02:13:12.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.351+0000 7f161effd640 10 client.19140 handle_command_reply: tid=0 2024-09-12T02:13:12.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.351+0000 7f16485f3740 1 client.19140 shutdown 2024-09-12T02:13:12.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.351+0000 7f16485f3740 1 -- 172.21.15.16:0/61488171 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563acc4a74a0 msgr2=0x563acc5252e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:12.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.351+0000 7f16485f3740 1 --2- 172.21.15.16:0/61488171 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563acc4a74a0 0x563acc5252e0 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f1630009230 tx=0x7f1630004100 comp rx=0 tx=0).stop 2024-09-12T02:13:12.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.351+0000 7f16485f3740 1 -- 172.21.15.16:0/61488171 shutdown_connections 2024-09-12T02:13:12.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.351+0000 7f16485f3740 1 --2- 172.21.15.16:0/61488171 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x563acc52d0f0 0x563acc52f530 crc :-1 s=READY pgs=341 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:12.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.351+0000 7f16485f3740 1 --2- 172.21.15.16:0/61488171 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563acc4b2680 0x563acc4b7c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:12.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.351+0000 7f16485f3740 1 --2- 172.21.15.16:0/61488171 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563acc4a7dc0 0x563acc525820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:12.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.351+0000 7f16485f3740 1 --2- 172.21.15.16:0/61488171 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563acc4a74a0 0x563acc5252e0 unknown :-1 s=CLOSED pgs=1600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:12.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.351+0000 7f16485f3740 1 -- 172.21.15.16:0/61488171 >> 172.21.15.16:0/61488171 conn(0x563acc4a3a30 msgr2=0x563acc4b0ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:12.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.351+0000 7f161effd640 0 client.19140 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:13:12.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.351+0000 7f16485f3740 1 -- 172.21.15.16:0/61488171 shutdown_connections 2024-09-12T02:13:12.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.351+0000 7f16485f3740 1 -- 172.21.15.16:0/61488171 wait complete. 2024-09-12T02:13:12.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.351+0000 7f16485f3740 20 client.19140 trim_cache size 0 max 16384 2024-09-12T02:13:12.356 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T02:13:12.356 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-09-12T02:13:12.356 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {} 2024-09-12T02:13:12.357 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T02:13:12.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.352+0000 7f164621a640 1 -- 172.21.15.16:0/4002644438 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f16280a2c00 msgr2=0x7f16280a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:12.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.352+0000 7f164621a640 1 --2- 172.21.15.16:0/4002644438 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f16280a2c00 0x7f16280a50c0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f1638020000 tx=0x7f1638021000 comp rx=0 tx=0).stop 2024-09-12T02:13:12.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.352+0000 7f164621a640 1 -- 172.21.15.16:0/4002644438 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1640103660 msgr2=0x7f164019aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:12.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.352+0000 7f164621a640 1 --2- 172.21.15.16:0/4002644438 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1640103660 0x7f164019aba0 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7f163400d740 tx=0x7f163400dc00 comp rx=0 tx=0).stop 2024-09-12T02:13:12.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.352+0000 7f164621a640 1 -- 172.21.15.16:0/4002644438 shutdown_connections 2024-09-12T02:13:12.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.352+0000 7f164621a640 1 --2- 172.21.15.16:0/4002644438 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f16280a2c00 0x7f16280a50c0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:12.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.352+0000 7f164621a640 1 --2- 172.21.15.16:0/4002644438 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f164010c170 0x7f164019ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:12.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.352+0000 7f164621a640 1 --2- 172.21.15.16:0/4002644438 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1640103660 0x7f164019aba0 unknown :-1 s=CLOSED pgs=1596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:12.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.352+0000 7f164621a640 1 --2- 172.21.15.16:0/4002644438 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1640102d40 0x7f164019a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:12.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.352+0000 7f164621a640 1 -- 172.21.15.16:0/4002644438 >> 172.21.15.16:0/4002644438 conn(0x7f16400fc7a0 msgr2=0x7f16400fcb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:12.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.352+0000 7f164621a640 1 -- 172.21.15.16:0/4002644438 shutdown_connections 2024-09-12T02:13:12.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:12.352+0000 7f164621a640 1 -- 172.21.15.16:0/4002644438 wait complete. 2024-09-12T02:13:12.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:13:12 smithi184 ceph-mon[35593]: pgmap v673: 97 pgs: 97 active+clean; 1.9 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-09-12T02:13:12.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:13:12 smithi184 ceph-mon[35593]: scrub queued for path: / 2024-09-12T02:13:12.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:13:12 smithi184 ceph-mon[35593]: scrub summary: idle+waiting paths [/] 2024-09-12T02:13:12.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:13:12 smithi184 ceph-mon[35593]: scrub summary: active paths [/] 2024-09-12T02:13:12.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:13:12 smithi184 ceph-mon[35593]: scrub summary: idle+waiting paths [/] 2024-09-12T02:13:12.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:13:12 smithi184 ceph-mon[35593]: scrub summary: idle 2024-09-12T02:13:12.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:13:12 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:13:12.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:13:12 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:13:12.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:13:12 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T02:13:12.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:13:12 smithi033 ceph-mon[35332]: pgmap v673: 97 pgs: 97 active+clean; 1.9 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-09-12T02:13:12.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:13:12 smithi033 ceph-mon[35332]: scrub queued for path: / 2024-09-12T02:13:12.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:13:12 smithi033 ceph-mon[35332]: scrub summary: idle+waiting paths [/] 2024-09-12T02:13:12.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:13:12 smithi033 ceph-mon[35332]: scrub summary: active paths [/] 2024-09-12T02:13:12.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:13:12 smithi033 ceph-mon[35332]: scrub summary: idle+waiting paths [/] 2024-09-12T02:13:12.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:13:12 smithi033 ceph-mon[35332]: scrub summary: idle 2024-09-12T02:13:12.420 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:13:12 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:13:12.420 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:13:12 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:13:12.420 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:13:12 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T02:13:13.023 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-12T02:13:13.023 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:13:13.282 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:13:13.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:13.731+0000 7f5ecc186640 1 -- 172.21.15.16:0/3779903711 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ec4100b30 msgr2=0x7f5ec4100f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:13.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:13.731+0000 7f5ecc186640 1 --2- 172.21.15.16:0/3779903711 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ec4100b30 0x7f5ec4100f10 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7f5ec0009900 tx=0x7f5ec002a470 comp rx=0 tx=0).stop 2024-09-12T02:13:13.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:13.731+0000 7f5ecc186640 1 -- 172.21.15.16:0/3779903711 shutdown_connections 2024-09-12T02:13:13.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:13.731+0000 7f5ecc186640 1 --2- 172.21.15.16:0/3779903711 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ec4105f40 0x7f5ec41063d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:13.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:13.731+0000 7f5ecc186640 1 --2- 172.21.15.16:0/3779903711 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5ec4101450 0x7f5ec4105920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:13.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:13.731+0000 7f5ecc186640 1 --2- 172.21.15.16:0/3779903711 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ec4100b30 0x7f5ec4100f10 unknown :-1 s=CLOSED pgs=1601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:13.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:13.731+0000 7f5ecc186640 1 -- 172.21.15.16:0/3779903711 >> 172.21.15.16:0/3779903711 conn(0x7f5ec40fc7a0 msgr2=0x7f5ec40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:13.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:13.731+0000 7f5ecc186640 1 -- 172.21.15.16:0/3779903711 shutdown_connections 2024-09-12T02:13:13.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:13.731+0000 7f5ecc186640 1 -- 172.21.15.16:0/3779903711 wait complete. 2024-09-12T02:13:13.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:13.732+0000 7f5ecc186640 1 Processor -- start 2024-09-12T02:13:13.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:13.732+0000 7f5ecc186640 1 -- start start 2024-09-12T02:13:13.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:13.732+0000 7f5ecc186640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ec4100b30 0x7f5ec41a4840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:13.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:13.732+0000 7f5ecc186640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5ec4101450 0x7f5ec41a4d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:13.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:13.732+0000 7f5ecc186640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ec4105f40 0x7f5ec419ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:13.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:13.732+0000 7f5ecc186640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5ec4118640 con 0x7f5ec4105f40 2024-09-12T02:13:13.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:13.732+0000 7f5ecc186640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5ec41184c0 con 0x7f5ec4100b30 2024-09-12T02:13:13.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:13.732+0000 7f5ecc186640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5ec41187c0 con 0x7f5ec4101450 2024-09-12T02:13:13.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:13.732+0000 7f5eca723640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ec4105f40 0x7f5ec419ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:13.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:13.733+0000 7f5eca723640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ec4105f40 0x7f5ec419ea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39400/0 (socket says 172.21.15.16:39400) 2024-09-12T02:13:13.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:13.733+0000 7f5eca723640 1 -- 172.21.15.16:0/321136412 learned_addr learned my addr 172.21.15.16:0/321136412 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:13:13.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:13.733+0000 7f5ec9721640 1 --2- 172.21.15.16:0/321136412 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5ec4101450 0x7f5ec41a4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:13.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:13.733+0000 7f5ec9f22640 1 --2- 172.21.15.16:0/321136412 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ec4100b30 0x7f5ec41a4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:13.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:13.733+0000 7f5eca723640 1 -- 172.21.15.16:0/321136412 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5ec4101450 msgr2=0x7f5ec41a4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:13.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:13.733+0000 7f5eca723640 1 --2- 172.21.15.16:0/321136412 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5ec4101450 0x7f5ec41a4d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:13.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:13.733+0000 7f5eca723640 1 -- 172.21.15.16:0/321136412 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ec4100b30 msgr2=0x7f5ec41a4840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:13.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:13.733+0000 7f5eca723640 1 --2- 172.21.15.16:0/321136412 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ec4100b30 0x7f5ec41a4840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:13.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:13.733+0000 7f5eca723640 1 -- 172.21.15.16:0/321136412 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ec419f130 con 0x7f5ec4105f40 2024-09-12T02:13:13.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:13.733+0000 7f5ec9721640 1 --2- 172.21.15.16:0/321136412 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5ec4101450 0x7f5ec41a4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:13:13.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:13.733+0000 7f5eca723640 1 --2- 172.21.15.16:0/321136412 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ec4105f40 0x7f5ec419ea00 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc00b970 tx=0x7f5ebc00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:13.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:13.733+0000 7f5ebaffd640 1 -- 172.21.15.16:0/321136412 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ebc019070 con 0x7f5ec4105f40 2024-09-12T02:13:13.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:13.733+0000 7f5ebaffd640 1 -- 172.21.15.16:0/321136412 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ebc00cb40 con 0x7f5ec4105f40 2024-09-12T02:13:13.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:13.733+0000 7f5ecc186640 1 -- 172.21.15.16:0/321136412 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ec419f420 con 0x7f5ec4105f40 2024-09-12T02:13:13.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:13.733+0000 7f5ebaffd640 1 -- 172.21.15.16:0/321136412 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ebc007650 con 0x7f5ec4105f40 2024-09-12T02:13:13.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:13.733+0000 7f5ecc186640 1 -- 172.21.15.16:0/321136412 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ec41ab710 con 0x7f5ec4105f40 2024-09-12T02:13:13.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:13.734+0000 7f5ecc186640 1 -- 172.21.15.16:0/321136412 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ec41059a0 con 0x7f5ec4105f40 2024-09-12T02:13:13.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:13.736+0000 7f5ebaffd640 1 -- 172.21.15.16:0/321136412 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5ebc01c070 con 0x7f5ec4105f40 2024-09-12T02:13:13.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:13.737+0000 7f5ebaffd640 1 --2- 172.21.15.16:0/321136412 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f5eb00a2c00 0x7f5eb00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:13.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:13.737+0000 7f5ec9f22640 1 --2- 172.21.15.16:0/321136412 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f5eb00a2c00 0x7f5eb00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:13.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:13.737+0000 7f5ebaffd640 1 -- 172.21.15.16:0/321136412 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f5ebc0ceae0 con 0x7f5ec4105f40 2024-09-12T02:13:13.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:13.738+0000 7f5ec9f22640 1 --2- 172.21.15.16:0/321136412 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f5eb00a2c00 0x7f5eb00a50c0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f5ec0000c00 tx=0x7f5ec00314b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:13.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:13.741+0000 7f5ebaffd640 1 -- 172.21.15.16:0/321136412 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5ebc09ba40 con 0x7f5ec4105f40 2024-09-12T02:13:13.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:13.953+0000 7f5ecc186640 1 -- 172.21.15.16:0/321136412 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5ec419ff30 con 0x7f5ec4105f40 2024-09-12T02:13:13.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:13.954+0000 7f5ebaffd640 1 -- 172.21.15.16:0/321136412 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11911 (secure 0 0 0) 0x7f5ec419ff30 con 0x7f5ec4105f40 2024-09-12T02:13:13.954 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T02:13:13.954 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":25,"btime":"2024-09-12T02:01:37:192021+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3279123279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3279123279},{"type":"v1","addr":"172.21.15.16:6835","nonce":3279123279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4119452382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4119452382},{"type":"v1","addr":"172.21.15.16:6837","nonce":4119452382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/3263823165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3263823165},{"type":"v1","addr":"172.21.15.16:6839","nonce":3263823165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/1510290732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":1510290732},{"type":"v1","addr":"172.21.15.16:6841","nonce":1510290732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/279674603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":279674603},{"type":"v1","addr":"172.21.15.33:6833","nonce":279674603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3555728028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3555728028},{"type":"v1","addr":"172.21.15.33:6835","nonce":3555728028}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T01:59:56.117608+0000","modified":"2024-09-12T02:01:36.294760+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24482,"mds_1":24481,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6833/1563374106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1563374106},{"type":"v1","addr":"172.21.15.184:6833","nonce":1563374106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6837/1892833868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1892833868},{"type":"v1","addr":"172.21.15.33:6837","nonce":1892833868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/1457679686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1457679686},{"type":"v1","addr":"172.21.15.184:6835","nonce":1457679686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.184:6837/1793393087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":1793393087},{"type":"v1","addr":"172.21.15.184:6837","nonce":1793393087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.184:6839/4203347941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":4203347941},{"type":"v1","addr":"172.21.15.184:6839","nonce":4203347941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.33:6839/4136670204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":4136670204},{"type":"v1","addr":"172.21.15.33:6839","nonce":4136670204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24481,24482,24469]},"id":1}]} 2024-09-12T02:13:13.954 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 25 2024-09-12T02:13:13.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:13.956+0000 7f5ecc186640 1 -- 172.21.15.16:0/321136412 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f5eb00a2c00 msgr2=0x7f5eb00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:13.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:13.956+0000 7f5ecc186640 1 --2- 172.21.15.16:0/321136412 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f5eb00a2c00 0x7f5eb00a50c0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f5ec0000c00 tx=0x7f5ec00314b0 comp rx=0 tx=0).stop 2024-09-12T02:13:13.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:13.956+0000 7f5ecc186640 1 -- 172.21.15.16:0/321136412 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ec4105f40 msgr2=0x7f5ec419ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:13.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:13.956+0000 7f5ecc186640 1 --2- 172.21.15.16:0/321136412 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ec4105f40 0x7f5ec419ea00 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc00b970 tx=0x7f5ebc00be30 comp rx=0 tx=0).stop 2024-09-12T02:13:13.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:13.956+0000 7f5ecc186640 1 -- 172.21.15.16:0/321136412 shutdown_connections 2024-09-12T02:13:13.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:13.956+0000 7f5ecc186640 1 --2- 172.21.15.16:0/321136412 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f5eb00a2c00 0x7f5eb00a50c0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:13.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:13.956+0000 7f5ecc186640 1 --2- 172.21.15.16:0/321136412 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ec4105f40 0x7f5ec419ea00 unknown :-1 s=CLOSED pgs=1602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:13.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:13.956+0000 7f5ecc186640 1 --2- 172.21.15.16:0/321136412 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5ec4101450 0x7f5ec41a4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:13.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:13.956+0000 7f5ecc186640 1 --2- 172.21.15.16:0/321136412 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ec4100b30 0x7f5ec41a4840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:13.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:13.956+0000 7f5ecc186640 1 -- 172.21.15.16:0/321136412 >> 172.21.15.16:0/321136412 conn(0x7f5ec40fc7a0 msgr2=0x7f5ec410f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:13.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:13.956+0000 7f5ecc186640 1 -- 172.21.15.16:0/321136412 shutdown_connections 2024-09-12T02:13:13.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:13.956+0000 7f5ecc186640 1 -- 172.21.15.16:0/321136412 wait complete. 2024-09-12T02:13:14.336 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:13:14 smithi016 ceph-mon[30947]: pgmap v674: 97 pgs: 97 active+clean; 1.9 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-09-12T02:13:14.336 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:13:14 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/321136412' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:13:14.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:13:14 smithi184 ceph-mon[35593]: pgmap v674: 97 pgs: 97 active+clean; 1.9 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-09-12T02:13:14.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:13:14 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/321136412' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:13:14.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:13:14 smithi033 ceph-mon[35332]: pgmap v674: 97 pgs: 97 active+clean; 1.9 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-09-12T02:13:14.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:13:14 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/321136412' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:13:14.661 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-12T02:13:14.920 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:13:15.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.357+0000 7fcb63547640 1 -- 172.21.15.16:0/1049886165 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb5c102790 msgr2=0x7fcb5c1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:15.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.358+0000 7fcb63547640 1 --2- 172.21.15.16:0/1049886165 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb5c102790 0x7fcb5c1067e0 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c009900 tx=0x7fcb4c02a410 comp rx=0 tx=0).stop 2024-09-12T02:13:15.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.358+0000 7fcb63547640 1 -- 172.21.15.16:0/1049886165 shutdown_connections 2024-09-12T02:13:15.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.358+0000 7fcb63547640 1 --2- 172.21.15.16:0/1049886165 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcb5c106f10 0x7fcb5c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:15.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.358+0000 7fcb63547640 1 --2- 172.21.15.16:0/1049886165 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb5c102790 0x7fcb5c1067e0 unknown :-1 s=CLOSED pgs=1603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:15.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.358+0000 7fcb63547640 1 --2- 172.21.15.16:0/1049886165 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb5c101de0 0x7fcb5c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:15.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.358+0000 7fcb63547640 1 -- 172.21.15.16:0/1049886165 >> 172.21.15.16:0/1049886165 conn(0x7fcb5c0fd8c0 msgr2=0x7fcb5c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:15.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.358+0000 7fcb63547640 1 -- 172.21.15.16:0/1049886165 shutdown_connections 2024-09-12T02:13:15.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.358+0000 7fcb63547640 1 -- 172.21.15.16:0/1049886165 wait complete. 2024-09-12T02:13:15.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.358+0000 7fcb63547640 1 Processor -- start 2024-09-12T02:13:15.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.358+0000 7fcb63547640 1 -- start start 2024-09-12T02:13:15.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.359+0000 7fcb63547640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb5c101de0 0x7fcb5c19a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:15.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.359+0000 7fcb63547640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb5c102790 0x7fcb5c19ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:15.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.359+0000 7fcb63547640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcb5c106f10 0x7fcb5c19ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:15.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.359+0000 7fcb63547640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fcb5c111f70 con 0x7fcb5c101de0 2024-09-12T02:13:15.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.359+0000 7fcb63547640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fcb5c111df0 con 0x7fcb5c102790 2024-09-12T02:13:15.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.359+0000 7fcb63547640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fcb5c1120f0 con 0x7fcb5c106f10 2024-09-12T02:13:15.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.359+0000 7fcb612e3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb5c101de0 0x7fcb5c19a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:15.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.359+0000 7fcb612e3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb5c101de0 0x7fcb5c19a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39422/0 (socket says 172.21.15.16:39422) 2024-09-12T02:13:15.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.359+0000 7fcb612e3640 1 -- 172.21.15.16:0/1414702950 learned_addr learned my addr 172.21.15.16:0/1414702950 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:13:15.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.359+0000 7fcb60ae2640 1 --2- 172.21.15.16:0/1414702950 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb5c102790 0x7fcb5c19ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:15.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.359+0000 7fcb61ae4640 1 --2- 172.21.15.16:0/1414702950 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcb5c106f10 0x7fcb5c19ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:15.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.359+0000 7fcb612e3640 1 -- 172.21.15.16:0/1414702950 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcb5c106f10 msgr2=0x7fcb5c19ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:15.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.359+0000 7fcb612e3640 1 --2- 172.21.15.16:0/1414702950 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcb5c106f10 0x7fcb5c19ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:15.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.359+0000 7fcb612e3640 1 -- 172.21.15.16:0/1414702950 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb5c102790 msgr2=0x7fcb5c19ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:15.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.359+0000 7fcb612e3640 1 --2- 172.21.15.16:0/1414702950 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb5c102790 0x7fcb5c19ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:15.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.359+0000 7fcb612e3640 1 -- 172.21.15.16:0/1414702950 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb5c19f5f0 con 0x7fcb5c101de0 2024-09-12T02:13:15.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.359+0000 7fcb612e3640 1 --2- 172.21.15.16:0/1414702950 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb5c101de0 0x7fcb5c19a640 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7fcb5800d950 tx=0x7fcb5800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:15.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.360+0000 7fcb527fc640 1 -- 172.21.15.16:0/1414702950 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb58017030 con 0x7fcb5c101de0 2024-09-12T02:13:15.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.360+0000 7fcb527fc640 1 -- 172.21.15.16:0/1414702950 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb58004020 con 0x7fcb5c101de0 2024-09-12T02:13:15.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.360+0000 7fcb63547640 1 -- 172.21.15.16:0/1414702950 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb5c19f8e0 con 0x7fcb5c101de0 2024-09-12T02:13:15.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.360+0000 7fcb527fc640 1 -- 172.21.15.16:0/1414702950 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb580047a0 con 0x7fcb5c101de0 2024-09-12T02:13:15.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.360+0000 7fcb63547640 1 -- 172.21.15.16:0/1414702950 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb5c1a71c0 con 0x7fcb5c101de0 2024-09-12T02:13:15.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.362+0000 7fcb527fc640 1 -- 172.21.15.16:0/1414702950 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fcb580041d0 con 0x7fcb5c101de0 2024-09-12T02:13:15.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.362+0000 7fcb65920740 1 Processor -- start 2024-09-12T02:13:15.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.362+0000 7fcb65920740 1 -- start start 2024-09-12T02:13:15.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.363+0000 7fcb65920740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee8b8d6b40 0x55ee8b8d5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:15.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.363+0000 7fcb65920740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ee8b8d55d0 0x55ee8b8d5a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:15.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.363+0000 7fcb65920740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ee8b8e0370 0x55ee8b8e2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:15.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.363+0000 7fcb65920740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55ee8b83b2f0 con 0x55ee8b8d6b40 2024-09-12T02:13:15.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.363+0000 7fcb65920740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55ee8b7e2cf0 con 0x55ee8b8e0370 2024-09-12T02:13:15.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.363+0000 7fcb65920740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55ee8b8372b0 con 0x55ee8b8d55d0 2024-09-12T02:13:15.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.363+0000 7fcb61ae4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee8b8d6b40 0x55ee8b8d5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:15.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.363+0000 7fcb61ae4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee8b8d6b40 0x55ee8b8d5090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39426/0 (socket says 172.21.15.16:39426) 2024-09-12T02:13:15.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.363+0000 7fcb61ae4640 1 -- 172.21.15.16:0/827359575 learned_addr learned my addr 172.21.15.16:0/827359575 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:13:15.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.363+0000 7fcb612e3640 1 --2- 172.21.15.16:0/827359575 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ee8b8d55d0 0x55ee8b8d5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:15.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.363+0000 7fcb60ae2640 1 --2- 172.21.15.16:0/827359575 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ee8b8e0370 0x55ee8b8e2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:15.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.363+0000 7fcb527fc640 1 --2- 172.21.15.16:0/1414702950 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fcb480a2c00 0x7fcb480a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:15.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.363+0000 7fcb61ae4640 1 -- 172.21.15.16:0/827359575 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ee8b8d55d0 msgr2=0x55ee8b8d5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:15.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.363+0000 7fcb61ae4640 1 --2- 172.21.15.16:0/827359575 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ee8b8d55d0 0x55ee8b8d5a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:15.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.363+0000 7fcb61ae4640 1 -- 172.21.15.16:0/827359575 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ee8b8e0370 msgr2=0x55ee8b8e2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:15.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.363+0000 7fcb61ae4640 1 --2- 172.21.15.16:0/827359575 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ee8b8e0370 0x55ee8b8e2760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:15.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.363+0000 7fcb61ae4640 1 -- 172.21.15.16:0/827359575 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee8b7c73a0 con 0x55ee8b8d6b40 2024-09-12T02:13:15.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.363+0000 7fcb612e3640 1 --2- 172.21.15.16:0/827359575 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ee8b8d55d0 0x55ee8b8d5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:13:15.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.363+0000 7fcb60ae2640 1 --2- 172.21.15.16:0/827359575 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ee8b8e0370 0x55ee8b8e2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:13:15.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.363+0000 7fcb527fc640 1 -- 172.21.15.16:0/1414702950 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fcb580ce860 con 0x7fcb5c101de0 2024-09-12T02:13:15.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.364+0000 7fcb61ae4640 1 --2- 172.21.15.16:0/827359575 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee8b8d6b40 0x55ee8b8d5090 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7fcb54009f70 tx=0x7fcb54010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de697199bfa01925 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:15.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.364+0000 7fcb61ae4640 1 --2- 172.21.15.16:0/1414702950 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fcb480a2c00 0x7fcb480a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:15.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.364+0000 7fcb477fe640 1 -- 172.21.15.16:0/827359575 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb54022070 con 0x55ee8b8d6b40 2024-09-12T02:13:15.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.364+0000 7fcb477fe640 1 -- 172.21.15.16:0/827359575 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb540113e0 con 0x55ee8b8d6b40 2024-09-12T02:13:15.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.364+0000 7fcb477fe640 1 -- 172.21.15.16:0/827359575 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb54016070 con 0x55ee8b8d6b40 2024-09-12T02:13:15.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.365+0000 7fcb65920740 1 -- 172.21.15.16:0/827359575 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee8b8d6b40 msgr2=0x55ee8b8d5090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:15.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.365+0000 7fcb61ae4640 1 --2- 172.21.15.16:0/1414702950 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fcb480a2c00 0x7fcb480a50c0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fcb5401f000 tx=0x7fcb54022000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:15.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.365+0000 7fcb65920740 1 --2- 172.21.15.16:0/827359575 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee8b8d6b40 0x55ee8b8d5090 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7fcb54009f70 tx=0x7fcb54010800 comp rx=0 tx=0).stop 2024-09-12T02:13:15.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.365+0000 7fcb65920740 1 -- 172.21.15.16:0/827359575 shutdown_connections 2024-09-12T02:13:15.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.365+0000 7fcb65920740 1 --2- 172.21.15.16:0/827359575 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ee8b8e0370 0x55ee8b8e2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:15.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.365+0000 7fcb65920740 1 --2- 172.21.15.16:0/827359575 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ee8b8d55d0 0x55ee8b8d5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:15.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.365+0000 7fcb65920740 1 --2- 172.21.15.16:0/827359575 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee8b8d6b40 0x55ee8b8d5090 unknown :-1 s=CLOSED pgs=1605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:15.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.365+0000 7fcb65920740 1 -- 172.21.15.16:0/827359575 >> 172.21.15.16:0/827359575 conn(0x55ee8b89e310 msgr2=0x55ee8b8d1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:15.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.365+0000 7fcb65920740 1 -- 172.21.15.16:0/827359575 shutdown_connections 2024-09-12T02:13:15.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.365+0000 7fcb65920740 1 -- 172.21.15.16:0/827359575 wait complete. 2024-09-12T02:13:15.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.365+0000 7fcb65920740 1 -- start start 2024-09-12T02:13:15.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.365+0000 7fcb65920740 1 Processor -- start 2024-09-12T02:13:15.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.366+0000 7fcb65920740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ee8b8d55d0 0x55ee8b8dfd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:15.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.366+0000 7fcb65920740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee8b8d6b40 0x55ee8b8dd030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:15.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.366+0000 7fcb65920740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ee8b8e0370 0x55ee8b8dd640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:15.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.366+0000 7fcb65920740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55ee8b8e3070 con 0x55ee8b8d6b40 2024-09-12T02:13:15.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.366+0000 7fcb65920740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55ee8b7f6060 con 0x55ee8b8d55d0 2024-09-12T02:13:15.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.366+0000 7fcb65920740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55ee8b8e31f0 con 0x55ee8b8e0370 2024-09-12T02:13:15.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.366+0000 7fcb65920740 10 client.0 _ll_register_callbacks cb 0x55ee8b5498c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:13:15.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.366+0000 7fcb60ae2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee8b8d6b40 0x55ee8b8dd030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:15.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.366+0000 7fcb60ae2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee8b8d6b40 0x55ee8b8dd030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39434/0 (socket says 172.21.15.16:39434) 2024-09-12T02:13:15.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.366+0000 7fcb60ae2640 1 -- 172.21.15.16:0/1038126748 learned_addr learned my addr 172.21.15.16:0/1038126748 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:13:15.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.366+0000 7fcb61ae4640 1 --2- 172.21.15.16:0/1038126748 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ee8b8e0370 0x55ee8b8dd640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:15.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.366+0000 7fcb612e3640 1 --2- 172.21.15.16:0/1038126748 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ee8b8d55d0 0x55ee8b8dfd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:15.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.366+0000 7fcb60ae2640 1 -- 172.21.15.16:0/1038126748 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ee8b8e0370 msgr2=0x55ee8b8dd640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:15.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.366+0000 7fcb60ae2640 1 --2- 172.21.15.16:0/1038126748 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ee8b8e0370 0x55ee8b8dd640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:15.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.366+0000 7fcb60ae2640 1 -- 172.21.15.16:0/1038126748 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ee8b8d55d0 msgr2=0x55ee8b8dfd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:15.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.366+0000 7fcb60ae2640 1 --2- 172.21.15.16:0/1038126748 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ee8b8d55d0 0x55ee8b8dfd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:15.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.366+0000 7fcb60ae2640 1 -- 172.21.15.16:0/1038126748 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee8b5ac3c0 con 0x55ee8b8d6b40 2024-09-12T02:13:15.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.366+0000 7fcb60ae2640 1 --2- 172.21.15.16:0/1038126748 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee8b8d6b40 0x55ee8b8dd030 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c02a930 tx=0x7fcb4c0318d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe711f4c6fbb08be server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:15.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.366+0000 7fcb44ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:13:15.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.366+0000 7fcb44ff9640 1 -- 172.21.15.16:0/1038126748 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb4c035040 con 0x55ee8b8d6b40 2024-09-12T02:13:15.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.367+0000 7fcb44ff9640 1 -- 172.21.15.16:0/1038126748 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fcb4c004240 con 0x55ee8b8d6b40 2024-09-12T02:13:15.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.367+0000 7fcb65920740 1 -- 172.21.15.16:0/1038126748 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb540113e0 con 0x55ee8b8d6b40 2024-09-12T02:13:15.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.367+0000 7fcb44ff9640 1 -- 172.21.15.16:0/1038126748 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fcb4c004440 con 0x55ee8b8d6b40 2024-09-12T02:13:15.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.367+0000 7fcb65920740 10 client.19158 fetch_fsmap learned FSMap version 25 2024-09-12T02:13:15.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.367+0000 7fcb65920740 1 -- 172.21.15.16:0/1038126748 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ee8b8eeb40 con 0x55ee8b8d6b40 2024-09-12T02:13:15.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.367+0000 7fcb44ff9640 1 -- 172.21.15.16:0/1038126748 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb540113e0 con 0x55ee8b8d6b40 2024-09-12T02:13:15.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.367+0000 7fcb44ff9640 1 -- 172.21.15.16:0/1038126748 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fcb4c0035f0 con 0x55ee8b8d6b40 2024-09-12T02:13:15.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.367+0000 7fcb65920740 10 client.19158 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:13:15.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.367+0000 7fcb65920740 10 client.19158 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-12T02:13:15.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.367+0000 7fcb65920740 20 client.19158 populate_metadata read hostname 'smithi016' 2024-09-12T02:13:15.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.367+0000 7fcb65920740 1 --2- 172.21.15.16:0/1038126748 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55ee8b8f0000 0x55ee8b8f2430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:15.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.367+0000 7fcb65920740 4 client.19158 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:13:15.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.367+0000 7fcb65920740 1 -- 172.21.15.16:0/1038126748 --> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ee8b848860 con 0x55ee8b8f0000 2024-09-12T02:13:15.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.368+0000 7fcb612e3640 1 --2- 172.21.15.16:0/1038126748 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55ee8b8f0000 0x55ee8b8f2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:15.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.368+0000 7fcb612e3640 1 --2- 172.21.15.16:0/1038126748 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55ee8b8f0000 0x55ee8b8f2430 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e0cd033ce3ee9db6 server_cookie=6746cdf715e1f61 in_seq=0 out_seq=0 2024-09-12T02:13:15.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.368+0000 7fcb44ff9640 10 client.19158 ms_handle_connect on v2:172.21.15.184:6836/1793393087 2024-09-12T02:13:15.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.369+0000 7fcb44ff9640 1 -- 172.21.15.16:0/1038126748 <== mds.2 v2:172.21.15.184:6836/1793393087 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ee8b848860 con 0x55ee8b8f0000 2024-09-12T02:13:15.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.369+0000 7fcb44ff9640 10 client.19158 handle_command_reply: tid=0 2024-09-12T02:13:15.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.369+0000 7fcb65920740 1 client.19158 shutdown 2024-09-12T02:13:15.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.369+0000 7fcb65920740 1 -- 172.21.15.16:0/1038126748 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee8b8d6b40 msgr2=0x55ee8b8dd030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:15.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.369+0000 7fcb65920740 1 --2- 172.21.15.16:0/1038126748 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee8b8d6b40 0x55ee8b8dd030 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c02a930 tx=0x7fcb4c0318d0 comp rx=0 tx=0).stop 2024-09-12T02:13:15.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.369+0000 7fcb65920740 1 -- 172.21.15.16:0/1038126748 shutdown_connections 2024-09-12T02:13:15.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.369+0000 7fcb65920740 1 --2- 172.21.15.16:0/1038126748 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ee8b8e0370 0x55ee8b8dd640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:15.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.369+0000 7fcb65920740 1 --2- 172.21.15.16:0/1038126748 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55ee8b8f0000 0x55ee8b8f2430 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:15.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.369+0000 7fcb65920740 1 --2- 172.21.15.16:0/1038126748 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee8b8d6b40 0x55ee8b8dd030 unknown :-1 s=CLOSED pgs=1606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:15.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.369+0000 7fcb65920740 1 --2- 172.21.15.16:0/1038126748 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ee8b8d55d0 0x55ee8b8dfd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:15.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.369+0000 7fcb44ff9640 0 client.19158 ms_handle_reset on v2:172.21.15.184:6836/1793393087 2024-09-12T02:13:15.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.369+0000 7fcb65920740 1 -- 172.21.15.16:0/1038126748 >> 172.21.15.16:0/1038126748 conn(0x55ee8b89e310 msgr2=0x55ee8b8d7960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:15.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.369+0000 7fcb65920740 1 -- 172.21.15.16:0/1038126748 shutdown_connections 2024-09-12T02:13:15.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.369+0000 7fcb65920740 1 -- 172.21.15.16:0/1038126748 wait complete. 2024-09-12T02:13:15.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.369+0000 7fcb65920740 20 client.19158 trim_cache size 0 max 16384 2024-09-12T02:13:15.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.382+0000 7fcb65920740 1 Processor -- start 2024-09-12T02:13:15.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.382+0000 7fcb65920740 1 -- start start 2024-09-12T02:13:15.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.382+0000 7fcb65920740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ee8b8d55d0 0x55ee8b8dc8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:15.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.382+0000 7fcb65920740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee8b8d6b40 0x55ee8b8defa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:15.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.382+0000 7fcb65920740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ee8b8e0370 0x55ee8b8d8120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:15.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.382+0000 7fcb65920740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fcb5c1b1fe0 con 0x55ee8b8d6b40 2024-09-12T02:13:15.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.382+0000 7fcb65920740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fcb5c1af8b0 con 0x55ee8b8d55d0 2024-09-12T02:13:15.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.382+0000 7fcb65920740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fcb5c1ad700 con 0x55ee8b8e0370 2024-09-12T02:13:15.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.383+0000 7fcb60ae2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee8b8d6b40 0x55ee8b8defa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:15.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.383+0000 7fcb60ae2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee8b8d6b40 0x55ee8b8defa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39442/0 (socket says 172.21.15.16:39442) 2024-09-12T02:13:15.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.383+0000 7fcb60ae2640 1 -- 172.21.15.16:0/4191465001 learned_addr learned my addr 172.21.15.16:0/4191465001 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:13:15.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.383+0000 7fcb612e3640 1 --2- 172.21.15.16:0/4191465001 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ee8b8d55d0 0x55ee8b8dc8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:15.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.383+0000 7fcb61ae4640 1 --2- 172.21.15.16:0/4191465001 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ee8b8e0370 0x55ee8b8d8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:15.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.383+0000 7fcb60ae2640 1 -- 172.21.15.16:0/4191465001 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ee8b8e0370 msgr2=0x55ee8b8d8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:15.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.383+0000 7fcb60ae2640 1 --2- 172.21.15.16:0/4191465001 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ee8b8e0370 0x55ee8b8d8120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:15.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.383+0000 7fcb60ae2640 1 -- 172.21.15.16:0/4191465001 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ee8b8d55d0 msgr2=0x55ee8b8dc8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:15.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.383+0000 7fcb60ae2640 1 --2- 172.21.15.16:0/4191465001 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ee8b8d55d0 0x55ee8b8dc8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:15.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.383+0000 7fcb60ae2640 1 -- 172.21.15.16:0/4191465001 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee8b8d52d0 con 0x55ee8b8d6b40 2024-09-12T02:13:15.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.383+0000 7fcb60ae2640 1 --2- 172.21.15.16:0/4191465001 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee8b8d6b40 0x55ee8b8defa0 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c009850 tx=0x7fcb4c02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88aeab19640dd99e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:15.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.383+0000 7fcb46ffd640 1 -- 172.21.15.16:0/4191465001 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb4c035030 con 0x55ee8b8d6b40 2024-09-12T02:13:15.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.383+0000 7fcb46ffd640 1 -- 172.21.15.16:0/4191465001 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb4c003d80 con 0x55ee8b8d6b40 2024-09-12T02:13:15.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.383+0000 7fcb46ffd640 1 -- 172.21.15.16:0/4191465001 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb4c0042a0 con 0x55ee8b8d6b40 2024-09-12T02:13:15.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.384+0000 7fcb65920740 1 -- 172.21.15.16:0/4191465001 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee8b8d6b40 msgr2=0x55ee8b8defa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:15.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.384+0000 7fcb65920740 1 --2- 172.21.15.16:0/4191465001 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee8b8d6b40 0x55ee8b8defa0 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c009850 tx=0x7fcb4c02a410 comp rx=0 tx=0).stop 2024-09-12T02:13:15.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.384+0000 7fcb65920740 1 -- 172.21.15.16:0/4191465001 shutdown_connections 2024-09-12T02:13:15.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.384+0000 7fcb65920740 1 --2- 172.21.15.16:0/4191465001 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ee8b8e0370 0x55ee8b8d8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:15.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.384+0000 7fcb65920740 1 --2- 172.21.15.16:0/4191465001 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee8b8d6b40 0x55ee8b8defa0 unknown :-1 s=CLOSED pgs=1607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:15.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.384+0000 7fcb65920740 1 --2- 172.21.15.16:0/4191465001 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ee8b8d55d0 0x55ee8b8dc8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:15.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.384+0000 7fcb65920740 1 -- 172.21.15.16:0/4191465001 >> 172.21.15.16:0/4191465001 conn(0x55ee8b89e310 msgr2=0x55ee8b8de890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:15.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.384+0000 7fcb65920740 1 -- 172.21.15.16:0/4191465001 shutdown_connections 2024-09-12T02:13:15.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.384+0000 7fcb65920740 1 -- 172.21.15.16:0/4191465001 wait complete. 2024-09-12T02:13:15.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.384+0000 7fcb65920740 1 -- start start 2024-09-12T02:13:15.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.385+0000 7fcb65920740 1 Processor -- start 2024-09-12T02:13:15.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.385+0000 7fcb65920740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ee8b8d55d0 0x55ee8b8e88a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:15.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.385+0000 7fcb65920740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ee8b8d6b40 0x55ee8b8d99d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:15.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.385+0000 7fcb65920740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee8b8e0370 0x55ee8b8d9f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:15.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.385+0000 7fcb65920740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fcb5c1add70 con 0x55ee8b8e0370 2024-09-12T02:13:15.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.385+0000 7fcb65920740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fcb5c1af1e0 con 0x55ee8b8d6b40 2024-09-12T02:13:15.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.385+0000 7fcb65920740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fcb5c1ae440 con 0x55ee8b8d55d0 2024-09-12T02:13:15.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.385+0000 7fcb65920740 10 client.0 _ll_register_callbacks cb 0x7fcb4c009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:13:15.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.385+0000 7fcb61ae4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee8b8e0370 0x55ee8b8d9f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:15.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.385+0000 7fcb61ae4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee8b8e0370 0x55ee8b8d9f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39446/0 (socket says 172.21.15.16:39446) 2024-09-12T02:13:15.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.385+0000 7fcb61ae4640 1 -- 172.21.15.16:0/3135594810 learned_addr learned my addr 172.21.15.16:0/3135594810 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:13:15.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.385+0000 7fcb60ae2640 1 --2- 172.21.15.16:0/3135594810 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ee8b8d6b40 0x55ee8b8d99d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:15.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.385+0000 7fcb612e3640 1 --2- 172.21.15.16:0/3135594810 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ee8b8d55d0 0x55ee8b8e88a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:15.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.385+0000 7fcb61ae4640 1 -- 172.21.15.16:0/3135594810 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ee8b8d55d0 msgr2=0x55ee8b8e88a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:15.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.385+0000 7fcb61ae4640 1 --2- 172.21.15.16:0/3135594810 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ee8b8d55d0 0x55ee8b8e88a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:15.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.385+0000 7fcb61ae4640 1 -- 172.21.15.16:0/3135594810 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ee8b8d6b40 msgr2=0x55ee8b8d99d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:15.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.385+0000 7fcb61ae4640 1 --2- 172.21.15.16:0/3135594810 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ee8b8d6b40 0x55ee8b8d99d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:15.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.385+0000 7fcb61ae4640 1 -- 172.21.15.16:0/3135594810 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee8b8da450 con 0x55ee8b8e0370 2024-09-12T02:13:15.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.386+0000 7fcb61ae4640 1 --2- 172.21.15.16:0/3135594810 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee8b8e0370 0x55ee8b8d9f10 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7fcb54006010 tx=0x7fcb540129e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8f242b16e2c2f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:15.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.386+0000 7fcb44ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:13:15.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.386+0000 7fcb44ff9640 1 -- 172.21.15.16:0/3135594810 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb54037060 con 0x55ee8b8e0370 2024-09-12T02:13:15.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.386+0000 7fcb44ff9640 1 -- 172.21.15.16:0/3135594810 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fcb5401cbf0 con 0x55ee8b8e0370 2024-09-12T02:13:15.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.386+0000 7fcb65920740 1 -- 172.21.15.16:0/3135594810 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb4c003d80 con 0x55ee8b8e0370 2024-09-12T02:13:15.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.386+0000 7fcb44ff9640 1 -- 172.21.15.16:0/3135594810 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fcb54016040 con 0x55ee8b8e0370 2024-09-12T02:13:15.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.386+0000 7fcb65920740 10 client.19164 fetch_fsmap learned FSMap version 25 2024-09-12T02:13:15.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.386+0000 7fcb65920740 1 -- 172.21.15.16:0/3135594810 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ee8b95a9c0 con 0x55ee8b8e0370 2024-09-12T02:13:15.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.386+0000 7fcb44ff9640 1 -- 172.21.15.16:0/3135594810 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb4c003d80 con 0x55ee8b8e0370 2024-09-12T02:13:15.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.386+0000 7fcb44ff9640 1 -- 172.21.15.16:0/3135594810 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fcb5403c070 con 0x55ee8b8e0370 2024-09-12T02:13:15.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.386+0000 7fcb65920740 10 client.19164 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:13:15.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.386+0000 7fcb65920740 10 client.19164 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-12T02:13:15.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.386+0000 7fcb65920740 20 client.19164 populate_metadata read hostname 'smithi016' 2024-09-12T02:13:15.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.387+0000 7fcb65920740 1 --2- 172.21.15.16:0/3135594810 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55ee8b95ad50 0x55ee8b95d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:15.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.387+0000 7fcb65920740 4 client.19164 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T02:13:15.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.387+0000 7fcb65920740 1 -- 172.21.15.16:0/3135594810 --> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ee8b842d70 con 0x55ee8b95ad50 2024-09-12T02:13:15.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.387+0000 7fcb612e3640 1 --2- 172.21.15.16:0/3135594810 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55ee8b95ad50 0x55ee8b95d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:15.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.387+0000 7fcb612e3640 1 --2- 172.21.15.16:0/3135594810 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55ee8b95ad50 0x55ee8b95d140 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d59bcea5f9cc1995 server_cookie=1b43c5690f9c28f8 in_seq=0 out_seq=0 2024-09-12T02:13:15.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.387+0000 7fcb44ff9640 10 client.19164 ms_handle_connect on v2:172.21.15.184:6836/1793393087 2024-09-12T02:13:15.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.388+0000 7fcb44ff9640 1 -- 172.21.15.16:0/3135594810 <== mds.2 v2:172.21.15.184:6836/1793393087 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ee8b842d70 con 0x55ee8b95ad50 2024-09-12T02:13:15.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.388+0000 7fcb44ff9640 10 client.19164 handle_command_reply: tid=0 2024-09-12T02:13:15.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.388+0000 7fcb65920740 1 client.19164 shutdown 2024-09-12T02:13:15.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.388+0000 7fcb65920740 1 -- 172.21.15.16:0/3135594810 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee8b8e0370 msgr2=0x55ee8b8d9f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:15.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.388+0000 7fcb65920740 1 --2- 172.21.15.16:0/3135594810 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee8b8e0370 0x55ee8b8d9f10 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7fcb54006010 tx=0x7fcb540129e0 comp rx=0 tx=0).stop 2024-09-12T02:13:15.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.388+0000 7fcb65920740 1 -- 172.21.15.16:0/3135594810 shutdown_connections 2024-09-12T02:13:15.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.388+0000 7fcb65920740 1 --2- 172.21.15.16:0/3135594810 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55ee8b95ad50 0x55ee8b95d140 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:15.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.388+0000 7fcb65920740 1 --2- 172.21.15.16:0/3135594810 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee8b8e0370 0x55ee8b8d9f10 unknown :-1 s=CLOSED pgs=1608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:15.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.388+0000 7fcb65920740 1 --2- 172.21.15.16:0/3135594810 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ee8b8d6b40 0x55ee8b8d99d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:15.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.388+0000 7fcb65920740 1 --2- 172.21.15.16:0/3135594810 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ee8b8d55d0 0x55ee8b8e88a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:15.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.388+0000 7fcb65920740 1 -- 172.21.15.16:0/3135594810 >> 172.21.15.16:0/3135594810 conn(0x55ee8b89e310 msgr2=0x55ee8b8df0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:15.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.388+0000 7fcb44ff9640 0 client.19164 ms_handle_reset on v2:172.21.15.184:6836/1793393087 2024-09-12T02:13:15.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.388+0000 7fcb65920740 1 -- 172.21.15.16:0/3135594810 shutdown_connections 2024-09-12T02:13:15.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.388+0000 7fcb65920740 1 -- 172.21.15.16:0/3135594810 wait complete. 2024-09-12T02:13:15.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.388+0000 7fcb65920740 20 client.19164 trim_cache size 0 max 16384 2024-09-12T02:13:15.394 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T02:13:15.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.389+0000 7fcb63547640 1 -- 172.21.15.16:0/1414702950 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fcb480a2c00 msgr2=0x7fcb480a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:15.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.389+0000 7fcb63547640 1 --2- 172.21.15.16:0/1414702950 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fcb480a2c00 0x7fcb480a50c0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fcb5401f000 tx=0x7fcb54022000 comp rx=0 tx=0).stop 2024-09-12T02:13:15.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.389+0000 7fcb63547640 1 -- 172.21.15.16:0/1414702950 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb5c101de0 msgr2=0x7fcb5c19a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:15.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.389+0000 7fcb63547640 1 --2- 172.21.15.16:0/1414702950 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb5c101de0 0x7fcb5c19a640 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7fcb5800d950 tx=0x7fcb5800de10 comp rx=0 tx=0).stop 2024-09-12T02:13:15.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.389+0000 7fcb63547640 1 -- 172.21.15.16:0/1414702950 shutdown_connections 2024-09-12T02:13:15.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.389+0000 7fcb63547640 1 --2- 172.21.15.16:0/1414702950 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fcb480a2c00 0x7fcb480a50c0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:15.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.389+0000 7fcb63547640 1 --2- 172.21.15.16:0/1414702950 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcb5c106f10 0x7fcb5c19ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:15.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.389+0000 7fcb63547640 1 --2- 172.21.15.16:0/1414702950 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb5c102790 0x7fcb5c19ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:15.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.389+0000 7fcb63547640 1 --2- 172.21.15.16:0/1414702950 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb5c101de0 0x7fcb5c19a640 unknown :-1 s=CLOSED pgs=1604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:15.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.389+0000 7fcb63547640 1 -- 172.21.15.16:0/1414702950 >> 172.21.15.16:0/1414702950 conn(0x7fcb5c0fd8c0 msgr2=0x7fcb5c0feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:15.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.389+0000 7fcb63547640 1 -- 172.21.15.16:0/1414702950 shutdown_connections 2024-09-12T02:13:15.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:15.389+0000 7fcb63547640 1 -- 172.21.15.16:0/1414702950 wait complete. 2024-09-12T02:13:16.052 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-12T02:13:16.310 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:13:16.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:13:16 smithi184 ceph-mon[35593]: pgmap v675: 97 pgs: 97 active+clean; 1.9 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-09-12T02:13:16.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:13:16 smithi033 ceph-mon[35332]: pgmap v675: 97 pgs: 97 active+clean; 1.9 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-09-12T02:13:16.429 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:13:16 smithi016 ceph-mon[30947]: pgmap v675: 97 pgs: 97 active+clean; 1.9 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-09-12T02:13:16.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.749+0000 7f05a051b640 1 -- 172.21.15.16:0/3478547895 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0598102790 msgr2=0x7f05981067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:16.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.749+0000 7f05a051b640 1 --2- 172.21.15.16:0/3478547895 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0598102790 0x7f05981067e0 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7f0588009900 tx=0x7f058802a410 comp rx=0 tx=0).stop 2024-09-12T02:13:16.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.749+0000 7f05a051b640 1 -- 172.21.15.16:0/3478547895 shutdown_connections 2024-09-12T02:13:16.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.749+0000 7f05a051b640 1 --2- 172.21.15.16:0/3478547895 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0598106f10 0x7f059810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:16.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.749+0000 7f05a051b640 1 --2- 172.21.15.16:0/3478547895 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0598102790 0x7f05981067e0 unknown :-1 s=CLOSED pgs=1609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:16.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.749+0000 7f05a051b640 1 --2- 172.21.15.16:0/3478547895 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0598101de0 0x7f05981021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:16.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.749+0000 7f05a051b640 1 -- 172.21.15.16:0/3478547895 >> 172.21.15.16:0/3478547895 conn(0x7f05980fd8c0 msgr2=0x7f05980ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:16.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.749+0000 7f05a051b640 1 -- 172.21.15.16:0/3478547895 shutdown_connections 2024-09-12T02:13:16.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.749+0000 7f05a051b640 1 -- 172.21.15.16:0/3478547895 wait complete. 2024-09-12T02:13:16.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.750+0000 7f05a051b640 1 Processor -- start 2024-09-12T02:13:16.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.750+0000 7f05a051b640 1 -- start start 2024-09-12T02:13:16.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.750+0000 7f05a051b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0598101de0 0x7f0598198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:16.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.750+0000 7f05a051b640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0598102790 0x7f0598198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:16.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.751+0000 7f05a051b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0598106f10 0x7f059819cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:16.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.751+0000 7f05a051b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0598111f70 con 0x7f0598101de0 2024-09-12T02:13:16.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.751+0000 7f05a051b640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0598111df0 con 0x7f0598106f10 2024-09-12T02:13:16.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.751+0000 7f05a051b640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f05981120f0 con 0x7f0598102790 2024-09-12T02:13:16.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.751+0000 7f059e2b7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0598101de0 0x7f0598198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:16.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.751+0000 7f059e2b7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0598101de0 0x7f0598198450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39460/0 (socket says 172.21.15.16:39460) 2024-09-12T02:13:16.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.751+0000 7f059e2b7640 1 -- 172.21.15.16:0/1345967116 learned_addr learned my addr 172.21.15.16:0/1345967116 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:13:16.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.751+0000 7f059eab8640 1 --2- 172.21.15.16:0/1345967116 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0598106f10 0x7f059819cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:16.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.751+0000 7f059dab6640 1 --2- 172.21.15.16:0/1345967116 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0598102790 0x7f0598198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:16.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.751+0000 7f059e2b7640 1 -- 172.21.15.16:0/1345967116 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0598102790 msgr2=0x7f0598198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:16.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.751+0000 7f059e2b7640 1 --2- 172.21.15.16:0/1345967116 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0598102790 0x7f0598198990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:16.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.751+0000 7f059e2b7640 1 -- 172.21.15.16:0/1345967116 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0598106f10 msgr2=0x7f059819cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:16.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.751+0000 7f059e2b7640 1 --2- 172.21.15.16:0/1345967116 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0598106f10 0x7f059819cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:16.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.751+0000 7f059e2b7640 1 -- 172.21.15.16:0/1345967116 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f059819d400 con 0x7f0598101de0 2024-09-12T02:13:16.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.751+0000 7f059eab8640 1 --2- 172.21.15.16:0/1345967116 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0598106f10 0x7f059819cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:13:16.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.751+0000 7f059e2b7640 1 --2- 172.21.15.16:0/1345967116 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0598101de0 0x7f0598198450 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7f059400d950 tx=0x7f059400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:16.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.751+0000 7f058f7fe640 1 -- 172.21.15.16:0/1345967116 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0594017070 con 0x7f0598101de0 2024-09-12T02:13:16.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.751+0000 7f058f7fe640 1 -- 172.21.15.16:0/1345967116 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0594009c30 con 0x7f0598101de0 2024-09-12T02:13:16.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.751+0000 7f05a051b640 1 -- 172.21.15.16:0/1345967116 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f059819d6f0 con 0x7f0598101de0 2024-09-12T02:13:16.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.751+0000 7f058f7fe640 1 -- 172.21.15.16:0/1345967116 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f059400f6b0 con 0x7f0598101de0 2024-09-12T02:13:16.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.751+0000 7f05a051b640 1 -- 172.21.15.16:0/1345967116 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05981a4fd0 con 0x7f0598101de0 2024-09-12T02:13:16.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.754+0000 7f058f7fe640 1 -- 172.21.15.16:0/1345967116 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f05940109d0 con 0x7f0598101de0 2024-09-12T02:13:16.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.754+0000 7f05a28f4740 1 Processor -- start 2024-09-12T02:13:16.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.754+0000 7f05a28f4740 1 -- start start 2024-09-12T02:13:16.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.754+0000 7f05a28f4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56414f07db40 0x56414f07c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:16.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.754+0000 7f05a28f4740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56414f07c5d0 0x56414f07ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:16.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.754+0000 7f05a28f4740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56414f08b780 0x56414f08db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:16.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.754+0000 7f05a28f4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56414efe22f0 con 0x56414f07db40 2024-09-12T02:13:16.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.754+0000 7f05a28f4740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56414ef89cf0 con 0x56414f08b780 2024-09-12T02:13:16.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.754+0000 7f05a28f4740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x56414efde2b0 con 0x56414f07c5d0 2024-09-12T02:13:16.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.755+0000 7f058f7fe640 1 --2- 172.21.15.16:0/1345967116 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f05840a2b80 0x7f05840a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:16.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.755+0000 7f059eab8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56414f07db40 0x56414f07c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:16.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.755+0000 7f059eab8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56414f07db40 0x56414f07c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39462/0 (socket says 172.21.15.16:39462) 2024-09-12T02:13:16.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.755+0000 7f059eab8640 1 -- 172.21.15.16:0/2546273406 learned_addr learned my addr 172.21.15.16:0/2546273406 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:13:16.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.755+0000 7f059dab6640 1 --2- 172.21.15.16:0/2546273406 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56414f08b780 0x56414f08db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:16.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.755+0000 7f058f7fe640 1 -- 172.21.15.16:0/1345967116 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f05940ceca0 con 0x7f0598101de0 2024-09-12T02:13:16.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.755+0000 7f059e2b7640 1 --2- 172.21.15.16:0/2546273406 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56414f07c5d0 0x56414f07ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:16.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.755+0000 7f059eab8640 1 --2- 172.21.15.16:0/1345967116 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f05840a2b80 0x7f05840a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:16.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.755+0000 7f059dab6640 1 -- 172.21.15.16:0/2546273406 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56414f07c5d0 msgr2=0x56414f07ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:16.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.755+0000 7f059dab6640 1 --2- 172.21.15.16:0/2546273406 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56414f07c5d0 0x56414f07ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:16.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.755+0000 7f059dab6640 1 -- 172.21.15.16:0/2546273406 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56414f07db40 msgr2=0x56414f07c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:16.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.755+0000 7f059dab6640 1 --2- 172.21.15.16:0/2546273406 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56414f07db40 0x56414f07c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:16.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.755+0000 7f059dab6640 1 -- 172.21.15.16:0/2546273406 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56414ef6e3a0 con 0x56414f08b780 2024-09-12T02:13:16.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.755+0000 7f059eab8640 1 --2- 172.21.15.16:0/2546273406 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56414f07db40 0x56414f07c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:13:16.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.755+0000 7f059dab6640 1 --2- 172.21.15.16:0/2546273406 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56414f08b780 0x56414f08db70 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f0588006010 tx=0x7f0588005e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e240513ad02c249f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:16.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.756+0000 7f059eab8640 1 --2- 172.21.15.16:0/1345967116 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f05840a2b80 0x7f05840a5040 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f0590009fd0 tx=0x7f0590009200 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:16.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.756+0000 7f058cff9640 1 -- 172.21.15.16:0/2546273406 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f058803f030 con 0x56414f08b780 2024-09-12T02:13:16.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.756+0000 7f058cff9640 1 -- 172.21.15.16:0/2546273406 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0588032db0 con 0x56414f08b780 2024-09-12T02:13:16.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.756+0000 7f058cff9640 1 -- 172.21.15.16:0/2546273406 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0588035070 con 0x56414f08b780 2024-09-12T02:13:16.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.756+0000 7f05a28f4740 1 -- 172.21.15.16:0/2546273406 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56414f08b780 msgr2=0x56414f08db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:16.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.756+0000 7f05a28f4740 1 --2- 172.21.15.16:0/2546273406 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56414f08b780 0x56414f08db70 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f0588006010 tx=0x7f0588005e80 comp rx=0 tx=0).stop 2024-09-12T02:13:16.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.756+0000 7f05a28f4740 1 -- 172.21.15.16:0/2546273406 shutdown_connections 2024-09-12T02:13:16.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.756+0000 7f05a28f4740 1 --2- 172.21.15.16:0/2546273406 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56414f08b780 0x56414f08db70 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:16.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.756+0000 7f05a28f4740 1 --2- 172.21.15.16:0/2546273406 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56414f07c5d0 0x56414f07ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:16.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.756+0000 7f05a28f4740 1 --2- 172.21.15.16:0/2546273406 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56414f07db40 0x56414f07c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:16.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.756+0000 7f05a28f4740 1 -- 172.21.15.16:0/2546273406 >> 172.21.15.16:0/2546273406 conn(0x56414f045310 msgr2=0x56414f078c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:16.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.757+0000 7f05a28f4740 1 -- 172.21.15.16:0/2546273406 shutdown_connections 2024-09-12T02:13:16.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.757+0000 7f05a28f4740 1 -- 172.21.15.16:0/2546273406 wait complete. 2024-09-12T02:13:16.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.757+0000 7f05a28f4740 1 -- start start 2024-09-12T02:13:16.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.757+0000 7f05a28f4740 1 Processor -- start 2024-09-12T02:13:16.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.757+0000 7f05a28f4740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56414f07c5d0 0x56414f08b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:16.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.757+0000 7f05a28f4740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56414f07db40 0x56414f0883e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:16.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.757+0000 7f05a28f4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56414f08b780 0x56414f088980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:16.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.757+0000 7f05a28f4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56414f08e420 con 0x56414f08b780 2024-09-12T02:13:16.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.757+0000 7f05a28f4740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56414ef9d060 con 0x56414f07db40 2024-09-12T02:13:16.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.757+0000 7f05a28f4740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x56414f08e5a0 con 0x56414f07c5d0 2024-09-12T02:13:16.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.757+0000 7f05a28f4740 10 client.0 _ll_register_callbacks cb 0x56414ecf08c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:13:16.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.757+0000 7f059eab8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56414f08b780 0x56414f088980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:16.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.758+0000 7f059eab8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56414f08b780 0x56414f088980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39470/0 (socket says 172.21.15.16:39470) 2024-09-12T02:13:16.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.758+0000 7f059eab8640 1 -- 172.21.15.16:0/1677660256 learned_addr learned my addr 172.21.15.16:0/1677660256 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:13:16.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.758+0000 7f059dab6640 1 --2- 172.21.15.16:0/1677660256 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56414f07db40 0x56414f0883e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:16.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.758+0000 7f059e2b7640 1 --2- 172.21.15.16:0/1677660256 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56414f07c5d0 0x56414f08b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:16.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.758+0000 7f059eab8640 1 -- 172.21.15.16:0/1677660256 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56414f07c5d0 msgr2=0x56414f08b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:16.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.758+0000 7f059eab8640 1 --2- 172.21.15.16:0/1677660256 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56414f07c5d0 0x56414f08b130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:16.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.758+0000 7f059eab8640 1 -- 172.21.15.16:0/1677660256 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56414f07db40 msgr2=0x56414f0883e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:16.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.758+0000 7f059eab8640 1 --2- 172.21.15.16:0/1677660256 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56414f07db40 0x56414f0883e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:16.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.758+0000 7f059eab8640 1 -- 172.21.15.16:0/1677660256 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56414ed533c0 con 0x56414f08b780 2024-09-12T02:13:16.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.758+0000 7f059eab8640 1 --2- 172.21.15.16:0/1677660256 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56414f08b780 0x56414f088980 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7f0590011b30 tx=0x7f0590004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b139042b67a03a0d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:16.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.758+0000 7f0581ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:13:16.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.758+0000 7f0581ffb640 1 -- 172.21.15.16:0/1677660256 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0590023070 con 0x56414f08b780 2024-09-12T02:13:16.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.758+0000 7f05a28f4740 1 -- 172.21.15.16:0/1677660256 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0588032db0 con 0x56414f08b780 2024-09-12T02:13:16.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.758+0000 7f0581ffb640 1 -- 172.21.15.16:0/1677660256 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f0590004b20 con 0x56414f08b780 2024-09-12T02:13:16.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.758+0000 7f0581ffb640 1 -- 172.21.15.16:0/1677660256 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f0590019070 con 0x56414f08b780 2024-09-12T02:13:16.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.758+0000 7f05a28f4740 10 client.19176 fetch_fsmap learned FSMap version 25 2024-09-12T02:13:16.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.758+0000 7f05a28f4740 1 -- 172.21.15.16:0/1677660256 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56414f099f20 con 0x56414f08b780 2024-09-12T02:13:16.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.759+0000 7f0581ffb640 1 -- 172.21.15.16:0/1677660256 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0588032db0 con 0x56414f08b780 2024-09-12T02:13:16.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.759+0000 7f0581ffb640 1 -- 172.21.15.16:0/1677660256 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f0590032400 con 0x56414f08b780 2024-09-12T02:13:16.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.759+0000 7f05a28f4740 10 client.19176 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:13:16.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.759+0000 7f05a28f4740 10 client.19176 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T02:13:16.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.759+0000 7f05a28f4740 20 client.19176 populate_metadata read hostname 'smithi016' 2024-09-12T02:13:16.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.759+0000 7f05a28f4740 1 --2- 172.21.15.16:0/1677660256 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x56414f09b3e0 0x56414f09d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:16.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.759+0000 7f05a28f4740 4 client.19176 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:13:16.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.759+0000 7f05a28f4740 1 -- 172.21.15.16:0/1677660256 --> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56414efef860 con 0x56414f09b3e0 2024-09-12T02:13:16.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.759+0000 7f059e2b7640 1 --2- 172.21.15.16:0/1677660256 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x56414f09b3e0 0x56414f09d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:16.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.760+0000 7f059e2b7640 1 --2- 172.21.15.16:0/1677660256 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x56414f09b3e0 0x56414f09d830 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=173593271fc9d7bb server_cookie=62c78bfbaa51d6c4 in_seq=0 out_seq=0 2024-09-12T02:13:16.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.760+0000 7f0581ffb640 10 client.19176 ms_handle_connect on v2:172.21.15.184:6838/4203347941 2024-09-12T02:13:16.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.760+0000 7f0581ffb640 1 -- 172.21.15.16:0/1677660256 <== mds.1 v2:172.21.15.184:6838/4203347941 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56414efef860 con 0x56414f09b3e0 2024-09-12T02:13:16.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.760+0000 7f0581ffb640 10 client.19176 handle_command_reply: tid=0 2024-09-12T02:13:16.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.760+0000 7f05a28f4740 1 client.19176 shutdown 2024-09-12T02:13:16.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.760+0000 7f05a28f4740 1 -- 172.21.15.16:0/1677660256 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56414f08b780 msgr2=0x56414f088980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:16.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.760+0000 7f05a28f4740 1 --2- 172.21.15.16:0/1677660256 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56414f08b780 0x56414f088980 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7f0590011b30 tx=0x7f0590004a40 comp rx=0 tx=0).stop 2024-09-12T02:13:16.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.760+0000 7f05a28f4740 1 -- 172.21.15.16:0/1677660256 shutdown_connections 2024-09-12T02:13:16.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.761+0000 7f05a28f4740 1 --2- 172.21.15.16:0/1677660256 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x56414f09b3e0 0x56414f09d830 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:16.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.761+0000 7f05a28f4740 1 --2- 172.21.15.16:0/1677660256 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56414f08b780 0x56414f088980 unknown :-1 s=CLOSED pgs=1611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:16.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.761+0000 7f05a28f4740 1 --2- 172.21.15.16:0/1677660256 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56414f07db40 0x56414f0883e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:16.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.761+0000 7f0581ffb640 0 client.19176 ms_handle_reset on v2:172.21.15.184:6838/4203347941 2024-09-12T02:13:16.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.761+0000 7f05a28f4740 1 --2- 172.21.15.16:0/1677660256 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56414f07c5d0 0x56414f08b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:16.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.761+0000 7f05a28f4740 1 -- 172.21.15.16:0/1677660256 >> 172.21.15.16:0/1677660256 conn(0x56414f045310 msgr2=0x56414f07e9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:16.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.761+0000 7f05a28f4740 1 -- 172.21.15.16:0/1677660256 shutdown_connections 2024-09-12T02:13:16.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.761+0000 7f05a28f4740 1 -- 172.21.15.16:0/1677660256 wait complete. 2024-09-12T02:13:16.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.761+0000 7f05a28f4740 20 client.19176 trim_cache size 0 max 16384 2024-09-12T02:13:16.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.773+0000 7f05a28f4740 1 Processor -- start 2024-09-12T02:13:16.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.774+0000 7f05a28f4740 1 -- start start 2024-09-12T02:13:16.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.774+0000 7f05a28f4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56414f07c5d0 0x56414f0976c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:16.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.774+0000 7f05a28f4740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56414f07db40 0x56414f0883a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:16.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.774+0000 7f05a28f4740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56414f08b780 0x56414f0888e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:16.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.774+0000 7f05a28f4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f05981adb00 con 0x56414f07c5d0 2024-09-12T02:13:16.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.774+0000 7f05a28f4740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f05981ab3d0 con 0x56414f07db40 2024-09-12T02:13:16.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.774+0000 7f05a28f4740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f05981a9240 con 0x56414f08b780 2024-09-12T02:13:16.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.774+0000 7f059e2b7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56414f07c5d0 0x56414f0976c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:16.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.774+0000 7f059e2b7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56414f07c5d0 0x56414f0976c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39476/0 (socket says 172.21.15.16:39476) 2024-09-12T02:13:16.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.774+0000 7f059e2b7640 1 -- 172.21.15.16:0/4202675716 learned_addr learned my addr 172.21.15.16:0/4202675716 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:13:16.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.774+0000 7f059dab6640 1 --2- 172.21.15.16:0/4202675716 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56414f07db40 0x56414f0883a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:16.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.774+0000 7f059eab8640 1 --2- 172.21.15.16:0/4202675716 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56414f08b780 0x56414f0888e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:16.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.774+0000 7f059e2b7640 1 -- 172.21.15.16:0/4202675716 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56414f08b780 msgr2=0x56414f0888e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:16.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.774+0000 7f059e2b7640 1 --2- 172.21.15.16:0/4202675716 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56414f08b780 0x56414f0888e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:16.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.774+0000 7f059e2b7640 1 -- 172.21.15.16:0/4202675716 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56414f07db40 msgr2=0x56414f0883a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:16.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.774+0000 7f059e2b7640 1 --2- 172.21.15.16:0/4202675716 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56414f07db40 0x56414f0883a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:16.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.774+0000 7f059e2b7640 1 -- 172.21.15.16:0/4202675716 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56414f086930 con 0x56414f07c5d0 2024-09-12T02:13:16.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.774+0000 7f059dab6640 1 --2- 172.21.15.16:0/4202675716 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56414f07db40 0x56414f0883a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:13:16.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.774+0000 7f059e2b7640 1 --2- 172.21.15.16:0/4202675716 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56414f07c5d0 0x56414f0976c0 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7f05940cc840 tx=0x7f05940cc710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a06b6b33d6eb0920 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:16.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.775+0000 7f0583fff640 1 -- 172.21.15.16:0/4202675716 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0594020040 con 0x56414f07c5d0 2024-09-12T02:13:16.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.775+0000 7f0583fff640 1 -- 172.21.15.16:0/4202675716 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f059409f950 con 0x56414f07c5d0 2024-09-12T02:13:16.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.775+0000 7f0583fff640 1 -- 172.21.15.16:0/4202675716 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0594084b30 con 0x56414f07c5d0 2024-09-12T02:13:16.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.775+0000 7f05a28f4740 1 -- 172.21.15.16:0/4202675716 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56414f07c5d0 msgr2=0x56414f0976c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:16.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.775+0000 7f05a28f4740 1 --2- 172.21.15.16:0/4202675716 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56414f07c5d0 0x56414f0976c0 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7f05940cc840 tx=0x7f05940cc710 comp rx=0 tx=0).stop 2024-09-12T02:13:16.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.775+0000 7f05a28f4740 1 -- 172.21.15.16:0/4202675716 shutdown_connections 2024-09-12T02:13:16.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.775+0000 7f05a28f4740 1 --2- 172.21.15.16:0/4202675716 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56414f08b780 0x56414f0888e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:16.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.775+0000 7f05a28f4740 1 --2- 172.21.15.16:0/4202675716 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56414f07db40 0x56414f0883a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:16.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.775+0000 7f05a28f4740 1 --2- 172.21.15.16:0/4202675716 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56414f07c5d0 0x56414f0976c0 unknown :-1 s=CLOSED pgs=1612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:16.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.775+0000 7f05a28f4740 1 -- 172.21.15.16:0/4202675716 >> 172.21.15.16:0/4202675716 conn(0x56414f045310 msgr2=0x56414f083720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:16.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.775+0000 7f05a28f4740 1 -- 172.21.15.16:0/4202675716 shutdown_connections 2024-09-12T02:13:16.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.775+0000 7f05a28f4740 1 -- 172.21.15.16:0/4202675716 wait complete. 2024-09-12T02:13:16.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.776+0000 7f05a28f4740 1 -- start start 2024-09-12T02:13:16.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.776+0000 7f05a28f4740 1 Processor -- start 2024-09-12T02:13:16.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.776+0000 7f05a28f4740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56414f07c5d0 0x56414f093ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:16.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.776+0000 7f05a28f4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56414f07db40 0x56414f093ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:16.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.776+0000 7f05a28f4740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56414f08b780 0x56414f0a26e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:16.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.776+0000 7f05a28f4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f05981a9890 con 0x56414f07db40 2024-09-12T02:13:16.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.776+0000 7f05a28f4740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f05981aad00 con 0x56414f07c5d0 2024-09-12T02:13:16.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.776+0000 7f05a28f4740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f05981a9f60 con 0x56414f08b780 2024-09-12T02:13:16.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.776+0000 7f05a28f4740 10 client.0 _ll_register_callbacks cb 0x7f0590013f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:13:16.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.776+0000 7f059dab6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56414f07db40 0x56414f093ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:16.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.776+0000 7f059dab6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56414f07db40 0x56414f093ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39480/0 (socket says 172.21.15.16:39480) 2024-09-12T02:13:16.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.776+0000 7f059dab6640 1 -- 172.21.15.16:0/386943712 learned_addr learned my addr 172.21.15.16:0/386943712 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:13:16.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.776+0000 7f059e2b7640 1 --2- 172.21.15.16:0/386943712 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56414f07c5d0 0x56414f093ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:16.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.776+0000 7f059eab8640 1 --2- 172.21.15.16:0/386943712 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56414f08b780 0x56414f0a26e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:16.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.776+0000 7f059dab6640 1 -- 172.21.15.16:0/386943712 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56414f08b780 msgr2=0x56414f0a26e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:16.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.776+0000 7f059dab6640 1 --2- 172.21.15.16:0/386943712 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56414f08b780 0x56414f0a26e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:16.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.776+0000 7f059dab6640 1 -- 172.21.15.16:0/386943712 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56414f07c5d0 msgr2=0x56414f093ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:16.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.776+0000 7f059dab6640 1 --2- 172.21.15.16:0/386943712 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56414f07c5d0 0x56414f093ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:16.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.776+0000 7f059dab6640 1 -- 172.21.15.16:0/386943712 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56414f0a2f10 con 0x56414f07db40 2024-09-12T02:13:16.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.776+0000 7f059eab8640 1 --2- 172.21.15.16:0/386943712 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56414f08b780 0x56414f0a26e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:13:16.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.777+0000 7f059dab6640 1 --2- 172.21.15.16:0/386943712 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56414f07db40 0x56414f093ff0 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7f0588005940 tx=0x7f0588032f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f3113fae169fbac server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:16.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.777+0000 7f0581ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:13:16.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.777+0000 7f0581ffb640 1 -- 172.21.15.16:0/386943712 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0588035040 con 0x56414f07db40 2024-09-12T02:13:16.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.777+0000 7f05a28f4740 1 -- 172.21.15.16:0/386943712 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f059409f950 con 0x56414f07db40 2024-09-12T02:13:16.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.777+0000 7f0581ffb640 1 -- 172.21.15.16:0/386943712 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f05880318f0 con 0x56414f07db40 2024-09-12T02:13:16.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.777+0000 7f0581ffb640 1 -- 172.21.15.16:0/386943712 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f0588053050 con 0x56414f07db40 2024-09-12T02:13:16.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.777+0000 7f05a28f4740 10 client.19182 fetch_fsmap learned FSMap version 25 2024-09-12T02:13:16.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.777+0000 7f05a28f4740 1 -- 172.21.15.16:0/386943712 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56414f093350 con 0x56414f07db40 2024-09-12T02:13:16.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.777+0000 7f0581ffb640 1 -- 172.21.15.16:0/386943712 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f059409f950 con 0x56414f07db40 2024-09-12T02:13:16.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.777+0000 7f0581ffb640 1 -- 172.21.15.16:0/386943712 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f05880046c0 con 0x56414f07db40 2024-09-12T02:13:16.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.778+0000 7f05a28f4740 10 client.19182 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:13:16.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.778+0000 7f05a28f4740 10 client.19182 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T02:13:16.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.778+0000 7f05a28f4740 20 client.19182 populate_metadata read hostname 'smithi016' 2024-09-12T02:13:16.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.778+0000 7f05a28f4740 1 --2- 172.21.15.16:0/386943712 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x56414f1060d0 0x56414f1084c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:16.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.778+0000 7f05a28f4740 4 client.19182 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T02:13:16.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.778+0000 7f05a28f4740 1 -- 172.21.15.16:0/386943712 --> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56414efe9d70 con 0x56414f1060d0 2024-09-12T02:13:16.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.778+0000 7f059e2b7640 1 --2- 172.21.15.16:0/386943712 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x56414f1060d0 0x56414f1084c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:16.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.778+0000 7f059e2b7640 1 --2- 172.21.15.16:0/386943712 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x56414f1060d0 0x56414f1084c0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=45a61d08a2e10bff server_cookie=ea69b9f35cbb1d79 in_seq=0 out_seq=0 2024-09-12T02:13:16.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.778+0000 7f0581ffb640 10 client.19182 ms_handle_connect on v2:172.21.15.184:6838/4203347941 2024-09-12T02:13:16.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.778+0000 7f0581ffb640 1 -- 172.21.15.16:0/386943712 <== mds.1 v2:172.21.15.184:6838/4203347941 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56414efe9d70 con 0x56414f1060d0 2024-09-12T02:13:16.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.778+0000 7f0581ffb640 10 client.19182 handle_command_reply: tid=0 2024-09-12T02:13:16.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.778+0000 7f05a28f4740 1 client.19182 shutdown 2024-09-12T02:13:16.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.779+0000 7f05a28f4740 1 -- 172.21.15.16:0/386943712 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56414f07db40 msgr2=0x56414f093ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:16.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.779+0000 7f05a28f4740 1 --2- 172.21.15.16:0/386943712 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56414f07db40 0x56414f093ff0 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7f0588005940 tx=0x7f0588032f60 comp rx=0 tx=0).stop 2024-09-12T02:13:16.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.779+0000 7f05a28f4740 1 -- 172.21.15.16:0/386943712 shutdown_connections 2024-09-12T02:13:16.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.779+0000 7f05a28f4740 1 --2- 172.21.15.16:0/386943712 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x56414f1060d0 0x56414f1084c0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:16.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.779+0000 7f05a28f4740 1 --2- 172.21.15.16:0/386943712 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56414f08b780 0x56414f0a26e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:16.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.779+0000 7f05a28f4740 1 --2- 172.21.15.16:0/386943712 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56414f07db40 0x56414f093ff0 unknown :-1 s=CLOSED pgs=1613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:16.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.779+0000 7f05a28f4740 1 --2- 172.21.15.16:0/386943712 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56414f07c5d0 0x56414f093ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:16.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.779+0000 7f05a28f4740 1 -- 172.21.15.16:0/386943712 >> 172.21.15.16:0/386943712 conn(0x56414f045310 msgr2=0x56414f083720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:16.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.779+0000 7f0581ffb640 0 client.19182 ms_handle_reset on v2:172.21.15.184:6838/4203347941 2024-09-12T02:13:16.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.779+0000 7f05a28f4740 1 -- 172.21.15.16:0/386943712 shutdown_connections 2024-09-12T02:13:16.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.779+0000 7f05a28f4740 1 -- 172.21.15.16:0/386943712 wait complete. 2024-09-12T02:13:16.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.779+0000 7f05a28f4740 20 client.19182 trim_cache size 0 max 16384 2024-09-12T02:13:16.795 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T02:13:16.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.779+0000 7f05a051b640 1 -- 172.21.15.16:0/1345967116 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f05840a2b80 msgr2=0x7f05840a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:16.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.779+0000 7f05a051b640 1 --2- 172.21.15.16:0/1345967116 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f05840a2b80 0x7f05840a5040 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f0590009fd0 tx=0x7f0590009200 comp rx=0 tx=0).stop 2024-09-12T02:13:16.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.779+0000 7f05a051b640 1 -- 172.21.15.16:0/1345967116 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0598101de0 msgr2=0x7f0598198450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:16.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.779+0000 7f05a051b640 1 --2- 172.21.15.16:0/1345967116 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0598101de0 0x7f0598198450 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7f059400d950 tx=0x7f059400de10 comp rx=0 tx=0).stop 2024-09-12T02:13:16.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.779+0000 7f05a051b640 1 -- 172.21.15.16:0/1345967116 shutdown_connections 2024-09-12T02:13:16.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.779+0000 7f05a051b640 1 --2- 172.21.15.16:0/1345967116 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f05840a2b80 0x7f05840a5040 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:16.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.779+0000 7f05a051b640 1 --2- 172.21.15.16:0/1345967116 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0598106f10 0x7f059819cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:16.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.779+0000 7f05a051b640 1 --2- 172.21.15.16:0/1345967116 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0598102790 0x7f0598198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:16.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.779+0000 7f05a051b640 1 --2- 172.21.15.16:0/1345967116 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0598101de0 0x7f0598198450 unknown :-1 s=CLOSED pgs=1610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:16.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.779+0000 7f05a051b640 1 -- 172.21.15.16:0/1345967116 >> 172.21.15.16:0/1345967116 conn(0x7f05980fd8c0 msgr2=0x7f05980febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:16.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.780+0000 7f05a051b640 1 -- 172.21.15.16:0/1345967116 shutdown_connections 2024-09-12T02:13:16.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:16.780+0000 7f05a051b640 1 -- 172.21.15.16:0/1345967116 wait complete. 2024-09-12T02:13:17.556 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-12T02:13:17.821 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:13:18.214 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:13:18 smithi016 ceph-mon[30947]: pgmap v676: 97 pgs: 97 active+clean; 2.0 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-12T02:13:18.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.298+0000 7f107de85640 1 -- 172.21.15.16:0/554838131 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1078103660 msgr2=0x7f107810bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:18.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.298+0000 7f107de85640 1 --2- 172.21.15.16:0/554838131 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1078103660 0x7f107810bb50 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7f1060009900 tx=0x7f106002a480 comp rx=0 tx=0).stop 2024-09-12T02:13:18.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.298+0000 7f107de85640 1 -- 172.21.15.16:0/554838131 shutdown_connections 2024-09-12T02:13:18.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.298+0000 7f107de85640 1 --2- 172.21.15.16:0/554838131 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f107810c170 0x7f107810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:18.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.298+0000 7f107de85640 1 --2- 172.21.15.16:0/554838131 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1078103660 0x7f107810bb50 unknown :-1 s=CLOSED pgs=1614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:18.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.298+0000 7f107de85640 1 --2- 172.21.15.16:0/554838131 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1078102d40 0x7f1078103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:18.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.298+0000 7f107de85640 1 -- 172.21.15.16:0/554838131 >> 172.21.15.16:0/554838131 conn(0x7f10780fc7a0 msgr2=0x7f10780febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:18.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.298+0000 7f107de85640 1 -- 172.21.15.16:0/554838131 shutdown_connections 2024-09-12T02:13:18.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.298+0000 7f107de85640 1 -- 172.21.15.16:0/554838131 wait complete. 2024-09-12T02:13:18.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.299+0000 7f107de85640 1 Processor -- start 2024-09-12T02:13:18.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.299+0000 7f107de85640 1 -- start start 2024-09-12T02:13:18.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.299+0000 7f107de85640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1078102d40 0x7f107819a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:18.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.299+0000 7f107de85640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1078103660 0x7f107819abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:18.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.299+0000 7f107de85640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f107810c170 0x7f107819ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:18.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.299+0000 7f107de85640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1078111f70 con 0x7f107810c170 2024-09-12T02:13:18.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.299+0000 7f107de85640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1078111df0 con 0x7f1078102d40 2024-09-12T02:13:18.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.299+0000 7f107de85640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f10781120f0 con 0x7f1078103660 2024-09-12T02:13:18.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.299+0000 7f1077fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f107810c170 0x7f107819ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:18.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.299+0000 7f1077fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f107810c170 0x7f107819ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39504/0 (socket says 172.21.15.16:39504) 2024-09-12T02:13:18.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.299+0000 7f1077fff640 1 -- 172.21.15.16:0/2495097993 learned_addr learned my addr 172.21.15.16:0/2495097993 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:13:18.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.300+0000 7f10777fe640 1 --2- 172.21.15.16:0/2495097993 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1078102d40 0x7f107819a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:18.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.300+0000 7f106ffff640 1 --2- 172.21.15.16:0/2495097993 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1078103660 0x7f107819abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:18.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.300+0000 7f1077fff640 1 -- 172.21.15.16:0/2495097993 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1078103660 msgr2=0x7f107819abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:18.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.300+0000 7f1077fff640 1 --2- 172.21.15.16:0/2495097993 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1078103660 0x7f107819abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:18.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.300+0000 7f1077fff640 1 -- 172.21.15.16:0/2495097993 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1078102d40 msgr2=0x7f107819a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:18.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.300+0000 7f1077fff640 1 --2- 172.21.15.16:0/2495097993 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1078102d40 0x7f107819a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:18.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.300+0000 7f1077fff640 1 -- 172.21.15.16:0/2495097993 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f107819f630 con 0x7f107810c170 2024-09-12T02:13:18.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.300+0000 7f1077fff640 1 --2- 172.21.15.16:0/2495097993 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f107810c170 0x7f107819ef50 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7f107000eb10 tx=0x7f107000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:18.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.300+0000 7f10757fa640 1 -- 172.21.15.16:0/2495097993 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1070004b20 con 0x7f107810c170 2024-09-12T02:13:18.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.300+0000 7f10757fa640 1 -- 172.21.15.16:0/2495097993 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1070004cc0 con 0x7f107810c170 2024-09-12T02:13:18.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.300+0000 7f107de85640 1 -- 172.21.15.16:0/2495097993 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f107819f920 con 0x7f107810c170 2024-09-12T02:13:18.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.300+0000 7f10757fa640 1 -- 172.21.15.16:0/2495097993 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1070010760 con 0x7f107810c170 2024-09-12T02:13:18.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.300+0000 7f107de85640 1 -- 172.21.15.16:0/2495097993 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10781a7200 con 0x7f107810c170 2024-09-12T02:13:18.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.303+0000 7f10757fa640 1 -- 172.21.15.16:0/2495097993 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f1070005020 con 0x7f107810c170 2024-09-12T02:13:18.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.303+0000 7f108025e740 1 Processor -- start 2024-09-12T02:13:18.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.303+0000 7f108025e740 1 -- start start 2024-09-12T02:13:18.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.303+0000 7f108025e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e56f4f3b40 0x55e56f4f2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:18.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.303+0000 7f108025e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e56f4f25d0 0x55e56f4f2a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:18.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.303+0000 7f108025e740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e56f4fd370 0x55e56f4ff760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:18.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.303+0000 7f108025e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e56f4582f0 con 0x55e56f4f3b40 2024-09-12T02:13:18.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.303+0000 7f108025e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55e56f3ffcf0 con 0x55e56f4f25d0 2024-09-12T02:13:18.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.303+0000 7f108025e740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55e56f4542b0 con 0x55e56f4fd370 2024-09-12T02:13:18.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.303+0000 7f106ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e56f4f3b40 0x55e56f4f2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:18.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.303+0000 7f106ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e56f4f3b40 0x55e56f4f2090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39506/0 (socket says 172.21.15.16:39506) 2024-09-12T02:13:18.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.303+0000 7f106ffff640 1 -- 172.21.15.16:0/1914118178 learned_addr learned my addr 172.21.15.16:0/1914118178 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:13:18.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.304+0000 7f10777fe640 1 --2- 172.21.15.16:0/1914118178 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e56f4f25d0 0x55e56f4f2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:18.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.304+0000 7f106ffff640 1 --2- 172.21.15.16:0/1914118178 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e56f4fd370 0x55e56f4ff760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:18.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.304+0000 7f10757fa640 1 --2- 172.21.15.16:0/2495097993 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f10640a2c00 0x7f10640a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:18.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.304+0000 7f106ffff640 1 -- 172.21.15.16:0/1914118178 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e56f4fd370 msgr2=0x55e56f4ff760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:18.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.304+0000 7f106ffff640 1 --2- 172.21.15.16:0/1914118178 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e56f4fd370 0x55e56f4ff760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:18.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.304+0000 7f106ffff640 1 -- 172.21.15.16:0/1914118178 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e56f4f25d0 msgr2=0x55e56f4f2a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:18.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.304+0000 7f106ffff640 1 --2- 172.21.15.16:0/1914118178 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e56f4f25d0 0x55e56f4f2a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:18.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.304+0000 7f106ffff640 1 -- 172.21.15.16:0/1914118178 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e56f3e43a0 con 0x55e56f4f3b40 2024-09-12T02:13:18.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.304+0000 7f10757fa640 1 -- 172.21.15.16:0/2495097993 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f10700cf4e0 con 0x7f107810c170 2024-09-12T02:13:18.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.304+0000 7f1077fff640 1 --2- 172.21.15.16:0/2495097993 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f10640a2c00 0x7f10640a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:18.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.304+0000 7f106ffff640 1 --2- 172.21.15.16:0/1914118178 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e56f4f3b40 0x55e56f4f2090 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7f1060005940 tx=0x7f1060005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12fff2f5f7507240 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:18.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.304+0000 7f106dffb640 1 -- 172.21.15.16:0/1914118178 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1060035050 con 0x55e56f4f3b40 2024-09-12T02:13:18.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.304+0000 7f106dffb640 1 -- 172.21.15.16:0/1914118178 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1060004030 con 0x55e56f4f3b40 2024-09-12T02:13:18.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.304+0000 7f106dffb640 1 -- 172.21.15.16:0/1914118178 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10600046c0 con 0x55e56f4f3b40 2024-09-12T02:13:18.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.305+0000 7f108025e740 1 -- 172.21.15.16:0/1914118178 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e56f4f3b40 msgr2=0x55e56f4f2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:18.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.305+0000 7f108025e740 1 --2- 172.21.15.16:0/1914118178 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e56f4f3b40 0x55e56f4f2090 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7f1060005940 tx=0x7f1060005b50 comp rx=0 tx=0).stop 2024-09-12T02:13:18.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.305+0000 7f1077fff640 1 --2- 172.21.15.16:0/2495097993 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f10640a2c00 0x7f10640a50c0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f1070005ba0 tx=0x7f107000efd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:18.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.305+0000 7f108025e740 1 -- 172.21.15.16:0/1914118178 shutdown_connections 2024-09-12T02:13:18.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.305+0000 7f108025e740 1 --2- 172.21.15.16:0/1914118178 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e56f4fd370 0x55e56f4ff760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:18.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.305+0000 7f108025e740 1 --2- 172.21.15.16:0/1914118178 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e56f4f25d0 0x55e56f4f2a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:18.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.305+0000 7f108025e740 1 --2- 172.21.15.16:0/1914118178 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e56f4f3b40 0x55e56f4f2090 unknown :-1 s=CLOSED pgs=1616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:18.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.305+0000 7f108025e740 1 -- 172.21.15.16:0/1914118178 >> 172.21.15.16:0/1914118178 conn(0x55e56f4bb310 msgr2=0x55e56f4eec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:18.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.305+0000 7f108025e740 1 -- 172.21.15.16:0/1914118178 shutdown_connections 2024-09-12T02:13:18.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.305+0000 7f108025e740 1 -- 172.21.15.16:0/1914118178 wait complete. 2024-09-12T02:13:18.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.305+0000 7f108025e740 1 -- start start 2024-09-12T02:13:18.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.306+0000 7f108025e740 1 Processor -- start 2024-09-12T02:13:18.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.306+0000 7f108025e740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e56f4f25d0 0x55e56f4fcde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:18.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.306+0000 7f108025e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e56f4f3b40 0x55e56f4fa090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:18.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.306+0000 7f108025e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e56f4fd370 0x55e56f4fa7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:18.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.306+0000 7f108025e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e56f500010 con 0x55e56f4fd370 2024-09-12T02:13:18.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.306+0000 7f108025e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55e56f413060 con 0x55e56f4f3b40 2024-09-12T02:13:18.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.306+0000 7f108025e740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55e56f500190 con 0x55e56f4f25d0 2024-09-12T02:13:18.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.306+0000 7f108025e740 10 client.0 _ll_register_callbacks cb 0x55e56f1668c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:13:18.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.306+0000 7f106ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e56f4fd370 0x55e56f4fa7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:18.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.306+0000 7f106ffff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e56f4f25d0 0x55e56f4fcde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:18.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.306+0000 7f106ffff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e56f4f25d0 0x55e56f4fcde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:53110/0 (socket says 172.21.15.16:53110) 2024-09-12T02:13:18.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.306+0000 7f106ffff640 1 -- 172.21.15.16:0/2523664956 learned_addr learned my addr 172.21.15.16:0/2523664956 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:13:18.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.306+0000 7f10777fe640 1 --2- 172.21.15.16:0/2523664956 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e56f4f3b40 0x55e56f4fa090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:18.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.306+0000 7f106ffff640 1 -- 172.21.15.16:0/2523664956 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e56f4f25d0 msgr2=0x55e56f4fcde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:18.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.306+0000 7f106ffff640 1 --2- 172.21.15.16:0/2523664956 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e56f4f25d0 0x55e56f4fcde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:18.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.306+0000 7f106ffff640 1 -- 172.21.15.16:0/2523664956 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e56f4f3b40 msgr2=0x55e56f4fa090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:18.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.306+0000 7f106ffff640 1 --2- 172.21.15.16:0/2523664956 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e56f4f3b40 0x55e56f4fa090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:18.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.306+0000 7f106ffff640 1 -- 172.21.15.16:0/2523664956 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e56f1c93c0 con 0x55e56f4fd370 2024-09-12T02:13:18.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.306+0000 7f106ffff640 1 --2- 172.21.15.16:0/2523664956 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e56f4fd370 0x55e56f4fa7d0 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f106002a9a0 tx=0x7f1060005280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=327b682abdfad483 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:18.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.306+0000 7f1056ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:13:18.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.307+0000 7f1056ffd640 1 -- 172.21.15.16:0/2523664956 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1060035070 con 0x55e56f4fd370 2024-09-12T02:13:18.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.307+0000 7f108025e740 1 -- 172.21.15.16:0/2523664956 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1060004030 con 0x55e56f4fd370 2024-09-12T02:13:18.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.307+0000 7f1056ffd640 1 -- 172.21.15.16:0/2523664956 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f1060005610 con 0x55e56f4fd370 2024-09-12T02:13:18.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.307+0000 7f1056ffd640 1 -- 172.21.15.16:0/2523664956 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1060031630 con 0x55e56f4fd370 2024-09-12T02:13:18.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.307+0000 7f108025e740 10 client.19194 fetch_fsmap learned FSMap version 25 2024-09-12T02:13:18.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.307+0000 7f108025e740 1 -- 172.21.15.16:0/2523664956 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e56f50bb40 con 0x55e56f4fd370 2024-09-12T02:13:18.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.307+0000 7f1056ffd640 1 -- 172.21.15.16:0/2523664956 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1060004030 con 0x55e56f4fd370 2024-09-12T02:13:18.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.307+0000 7f1056ffd640 1 -- 172.21.15.16:0/2523664956 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f10600035f0 con 0x55e56f4fd370 2024-09-12T02:13:18.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.307+0000 7f108025e740 10 client.19194 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:13:18.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.307+0000 7f108025e740 10 client.19194 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:13:18.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.307+0000 7f108025e740 20 client.19194 populate_metadata read hostname 'smithi016' 2024-09-12T02:13:18.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.307+0000 7f108025e740 1 --2- 172.21.15.16:0/2523664956 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55e56f50ce90 0x55e56f50f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:18.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.307+0000 7f108025e740 4 client.19194 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:13:18.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.307+0000 7f108025e740 1 -- 172.21.15.16:0/2523664956 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e56f465860 con 0x55e56f50ce90 2024-09-12T02:13:18.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.308+0000 7f10777fe640 1 --2- 172.21.15.16:0/2523664956 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55e56f50ce90 0x55e56f50f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:18.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.308+0000 7f1056ffd640 10 client.19194 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:13:18.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.308+0000 7f10777fe640 1 --2- 172.21.15.16:0/2523664956 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55e56f50ce90 0x55e56f50f280 crc :-1 s=READY pgs=342 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c4a8f2f34d2bed8 server_cookie=c938e41a472f1fc in_seq=0 out_seq=0 2024-09-12T02:13:18.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.309+0000 7f1056ffd640 1 -- 172.21.15.16:0/2523664956 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e56f465860 con 0x55e56f50ce90 2024-09-12T02:13:18.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.309+0000 7f1056ffd640 10 client.19194 handle_command_reply: tid=0 2024-09-12T02:13:18.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.309+0000 7f108025e740 1 client.19194 shutdown 2024-09-12T02:13:18.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.309+0000 7f108025e740 1 -- 172.21.15.16:0/2523664956 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e56f4fd370 msgr2=0x55e56f4fa7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:18.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.309+0000 7f108025e740 1 --2- 172.21.15.16:0/2523664956 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e56f4fd370 0x55e56f4fa7d0 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f106002a9a0 tx=0x7f1060005280 comp rx=0 tx=0).stop 2024-09-12T02:13:18.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.309+0000 7f108025e740 1 -- 172.21.15.16:0/2523664956 shutdown_connections 2024-09-12T02:13:18.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.309+0000 7f108025e740 1 --2- 172.21.15.16:0/2523664956 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55e56f50ce90 0x55e56f50f280 crc :-1 s=READY pgs=342 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:18.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.309+0000 7f108025e740 1 --2- 172.21.15.16:0/2523664956 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e56f4fd370 0x55e56f4fa7d0 unknown :-1 s=CLOSED pgs=1617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:18.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.309+0000 7f108025e740 1 --2- 172.21.15.16:0/2523664956 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e56f4f3b40 0x55e56f4fa090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:18.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.309+0000 7f108025e740 1 --2- 172.21.15.16:0/2523664956 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e56f4f25d0 0x55e56f4fcde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:18.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.309+0000 7f1056ffd640 0 client.19194 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:13:18.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.309+0000 7f108025e740 1 -- 172.21.15.16:0/2523664956 >> 172.21.15.16:0/2523664956 conn(0x55e56f4bb310 msgr2=0x55e56f4f4960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:18.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.309+0000 7f108025e740 1 -- 172.21.15.16:0/2523664956 shutdown_connections 2024-09-12T02:13:18.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.309+0000 7f108025e740 1 -- 172.21.15.16:0/2523664956 wait complete. 2024-09-12T02:13:18.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.309+0000 7f108025e740 20 client.19194 trim_cache size 0 max 16384 2024-09-12T02:13:18.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.322+0000 7f108025e740 1 Processor -- start 2024-09-12T02:13:18.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.322+0000 7f108025e740 1 -- start start 2024-09-12T02:13:18.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.322+0000 7f108025e740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e56f4f25d0 0x55e56f509430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:18.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.322+0000 7f108025e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e56f4f3b40 0x55e56f5122a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:18.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.322+0000 7f108025e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e56f4fd370 0x55e56f5127e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:18.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.322+0000 7f108025e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f10781b1ea0 con 0x55e56f4fd370 2024-09-12T02:13:18.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.322+0000 7f108025e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f10781af770 con 0x55e56f4f3b40 2024-09-12T02:13:18.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.322+0000 7f108025e740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f10781ad5c0 con 0x55e56f4f25d0 2024-09-12T02:13:18.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.322+0000 7f106ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e56f4fd370 0x55e56f5127e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:18.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.322+0000 7f106ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e56f4fd370 0x55e56f5127e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39514/0 (socket says 172.21.15.16:39514) 2024-09-12T02:13:18.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.322+0000 7f106ffff640 1 -- 172.21.15.16:0/4131497380 learned_addr learned my addr 172.21.15.16:0/4131497380 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:13:18.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.322+0000 7f10777fe640 1 --2- 172.21.15.16:0/4131497380 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e56f4f3b40 0x55e56f5122a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:18.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.323+0000 7f106ffff640 1 -- 172.21.15.16:0/4131497380 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e56f4f25d0 msgr2=0x55e56f509430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:18.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.323+0000 7f106ffff640 1 --2- 172.21.15.16:0/4131497380 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e56f4f25d0 0x55e56f509430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:18.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.323+0000 7f106ffff640 1 -- 172.21.15.16:0/4131497380 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e56f4f3b40 msgr2=0x55e56f5122a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:18.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.323+0000 7f106ffff640 1 --2- 172.21.15.16:0/4131497380 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e56f4f3b40 0x55e56f5122a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:18.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.323+0000 7f106ffff640 1 -- 172.21.15.16:0/4131497380 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e56f4eeed0 con 0x55e56f4fd370 2024-09-12T02:13:18.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.323+0000 7f106ffff640 1 --2- 172.21.15.16:0/4131497380 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e56f4fd370 0x55e56f5127e0 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7f1060001eb0 tx=0x7f1060005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53348d38f176596c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:18.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.323+0000 7f106d7fa640 1 -- 172.21.15.16:0/4131497380 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1060035070 con 0x55e56f4fd370 2024-09-12T02:13:18.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.323+0000 7f106d7fa640 1 -- 172.21.15.16:0/4131497380 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1060003fb0 con 0x55e56f4fd370 2024-09-12T02:13:18.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.323+0000 7f106d7fa640 1 -- 172.21.15.16:0/4131497380 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f106005c070 con 0x55e56f4fd370 2024-09-12T02:13:18.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.324+0000 7f108025e740 1 -- 172.21.15.16:0/4131497380 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e56f4fd370 msgr2=0x55e56f5127e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:18.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.324+0000 7f108025e740 1 --2- 172.21.15.16:0/4131497380 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e56f4fd370 0x55e56f5127e0 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7f1060001eb0 tx=0x7f1060005d50 comp rx=0 tx=0).stop 2024-09-12T02:13:18.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.324+0000 7f108025e740 1 -- 172.21.15.16:0/4131497380 shutdown_connections 2024-09-12T02:13:18.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.324+0000 7f108025e740 1 --2- 172.21.15.16:0/4131497380 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e56f4fd370 0x55e56f5127e0 unknown :-1 s=CLOSED pgs=1618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:18.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.324+0000 7f108025e740 1 --2- 172.21.15.16:0/4131497380 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e56f4f3b40 0x55e56f5122a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:18.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.324+0000 7f108025e740 1 --2- 172.21.15.16:0/4131497380 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e56f4f25d0 0x55e56f509430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:18.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.324+0000 7f108025e740 1 -- 172.21.15.16:0/4131497380 >> 172.21.15.16:0/4131497380 conn(0x55e56f4bb310 msgr2=0x55e56f4fc110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:18.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.324+0000 7f108025e740 1 -- 172.21.15.16:0/4131497380 shutdown_connections 2024-09-12T02:13:18.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.324+0000 7f108025e740 1 -- 172.21.15.16:0/4131497380 wait complete. 2024-09-12T02:13:18.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.324+0000 7f108025e740 1 -- start start 2024-09-12T02:13:18.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.324+0000 7f108025e740 1 Processor -- start 2024-09-12T02:13:18.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.324+0000 7f108025e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e56f4f25d0 0x55e56f512980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:18.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.325+0000 7f108025e740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e56f4f3b40 0x55e56f50fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:18.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.325+0000 7f108025e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e56f4fd370 0x55e56f510210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:18.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.325+0000 7f108025e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f10781adc30 con 0x55e56f4f25d0 2024-09-12T02:13:18.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.325+0000 7f108025e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f10781af0a0 con 0x55e56f4fd370 2024-09-12T02:13:18.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.325+0000 7f108025e740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f10781ae300 con 0x55e56f4f3b40 2024-09-12T02:13:18.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.325+0000 7f108025e740 10 client.0 _ll_register_callbacks cb 0x7f10700093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:13:18.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.325+0000 7f106ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e56f4f25d0 0x55e56f512980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:18.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.325+0000 7f106ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e56f4f25d0 0x55e56f512980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39526/0 (socket says 172.21.15.16:39526) 2024-09-12T02:13:18.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.325+0000 7f106ffff640 1 -- 172.21.15.16:0/3594025138 learned_addr learned my addr 172.21.15.16:0/3594025138 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:13:18.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.325+0000 7f10777fe640 1 --2- 172.21.15.16:0/3594025138 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e56f4f3b40 0x55e56f50fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:18.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.325+0000 7f106ffff640 1 -- 172.21.15.16:0/3594025138 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e56f4f3b40 msgr2=0x55e56f50fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:18.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.325+0000 7f106ffff640 1 --2- 172.21.15.16:0/3594025138 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e56f4f3b40 0x55e56f50fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:18.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.325+0000 7f106ffff640 1 -- 172.21.15.16:0/3594025138 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e56f4fd370 msgr2=0x55e56f510210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:18.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.325+0000 7f106ffff640 1 --2- 172.21.15.16:0/3594025138 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e56f4fd370 0x55e56f510210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:18.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.325+0000 7f106ffff640 1 -- 172.21.15.16:0/3594025138 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e56f510a40 con 0x55e56f4f25d0 2024-09-12T02:13:18.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.325+0000 7f10777fe640 1 --2- 172.21.15.16:0/3594025138 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e56f4f3b40 0x55e56f50fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:13:18.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.325+0000 7f106ffff640 1 --2- 172.21.15.16:0/3594025138 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e56f4f25d0 0x55e56f512980 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7f1060003a10 tx=0x7f10600092a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6ec87a429cbd000 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:18.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.325+0000 7f1056ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:13:18.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.325+0000 7f1056ffd640 1 -- 172.21.15.16:0/3594025138 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1060035070 con 0x55e56f4f25d0 2024-09-12T02:13:18.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.325+0000 7f1056ffd640 1 -- 172.21.15.16:0/3594025138 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f1060005460 con 0x55e56f4f25d0 2024-09-12T02:13:18.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.325+0000 7f108025e740 1 -- 172.21.15.16:0/3594025138 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1060003fb0 con 0x55e56f4f25d0 2024-09-12T02:13:18.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.325+0000 7f1056ffd640 1 -- 172.21.15.16:0/3594025138 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f106005c040 con 0x55e56f4f25d0 2024-09-12T02:13:18.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.326+0000 7f108025e740 10 client.19200 fetch_fsmap learned FSMap version 25 2024-09-12T02:13:18.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.326+0000 7f108025e740 1 -- 172.21.15.16:0/3594025138 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e56f577760 con 0x55e56f4f25d0 2024-09-12T02:13:18.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.326+0000 7f1056ffd640 1 -- 172.21.15.16:0/3594025138 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1060003fb0 con 0x55e56f4f25d0 2024-09-12T02:13:18.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.326+0000 7f1056ffd640 1 -- 172.21.15.16:0/3594025138 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f1060004d40 con 0x55e56f4f25d0 2024-09-12T02:13:18.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.326+0000 7f108025e740 10 client.19200 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:13:18.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.326+0000 7f108025e740 10 client.19200 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:13:18.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.326+0000 7f108025e740 20 client.19200 populate_metadata read hostname 'smithi016' 2024-09-12T02:13:18.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.326+0000 7f108025e740 1 --2- 172.21.15.16:0/3594025138 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55e56f577b80 0x55e56f579f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:18.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.326+0000 7f108025e740 4 client.19200 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T02:13:18.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.326+0000 7f108025e740 1 -- 172.21.15.16:0/3594025138 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e56f45fd70 con 0x55e56f577b80 2024-09-12T02:13:18.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.326+0000 7f10777fe640 1 --2- 172.21.15.16:0/3594025138 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55e56f577b80 0x55e56f579f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:18.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.327+0000 7f10777fe640 1 --2- 172.21.15.16:0/3594025138 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55e56f577b80 0x55e56f579f70 crc :-1 s=READY pgs=343 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67fc47ec27bc4120 server_cookie=4756a491f54397fa in_seq=0 out_seq=0 2024-09-12T02:13:18.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.327+0000 7f1056ffd640 10 client.19200 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:13:18.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.327+0000 7f1056ffd640 1 -- 172.21.15.16:0/3594025138 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e56f45fd70 con 0x55e56f577b80 2024-09-12T02:13:18.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.327+0000 7f1056ffd640 10 client.19200 handle_command_reply: tid=0 2024-09-12T02:13:18.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.327+0000 7f108025e740 1 client.19200 shutdown 2024-09-12T02:13:18.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.327+0000 7f108025e740 1 -- 172.21.15.16:0/3594025138 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e56f4f25d0 msgr2=0x55e56f512980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:18.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.327+0000 7f108025e740 1 --2- 172.21.15.16:0/3594025138 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e56f4f25d0 0x55e56f512980 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7f1060003a10 tx=0x7f10600092a0 comp rx=0 tx=0).stop 2024-09-12T02:13:18.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.327+0000 7f108025e740 1 -- 172.21.15.16:0/3594025138 shutdown_connections 2024-09-12T02:13:18.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.327+0000 7f108025e740 1 --2- 172.21.15.16:0/3594025138 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55e56f577b80 0x55e56f579f70 crc :-1 s=READY pgs=343 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:18.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.327+0000 7f108025e740 1 --2- 172.21.15.16:0/3594025138 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e56f4fd370 0x55e56f510210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:18.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.327+0000 7f108025e740 1 --2- 172.21.15.16:0/3594025138 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e56f4f3b40 0x55e56f50fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:18.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.327+0000 7f108025e740 1 --2- 172.21.15.16:0/3594025138 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e56f4f25d0 0x55e56f512980 unknown :-1 s=CLOSED pgs=1619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:18.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.327+0000 7f108025e740 1 -- 172.21.15.16:0/3594025138 >> 172.21.15.16:0/3594025138 conn(0x55e56f4bb310 msgr2=0x55e56f4f4110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:18.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.327+0000 7f1056ffd640 0 client.19200 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:13:18.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.327+0000 7f108025e740 1 -- 172.21.15.16:0/3594025138 shutdown_connections 2024-09-12T02:13:18.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.327+0000 7f108025e740 1 -- 172.21.15.16:0/3594025138 wait complete. 2024-09-12T02:13:18.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.327+0000 7f108025e740 20 client.19200 trim_cache size 0 max 16384 2024-09-12T02:13:18.335 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T02:13:18.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.328+0000 7f107de85640 1 -- 172.21.15.16:0/2495097993 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f10640a2c00 msgr2=0x7f10640a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:18.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.328+0000 7f107de85640 1 --2- 172.21.15.16:0/2495097993 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f10640a2c00 0x7f10640a50c0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f1070005ba0 tx=0x7f107000efd0 comp rx=0 tx=0).stop 2024-09-12T02:13:18.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.328+0000 7f107de85640 1 -- 172.21.15.16:0/2495097993 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f107810c170 msgr2=0x7f107819ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:18.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.328+0000 7f107de85640 1 --2- 172.21.15.16:0/2495097993 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f107810c170 0x7f107819ef50 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7f107000eb10 tx=0x7f107000c2d0 comp rx=0 tx=0).stop 2024-09-12T02:13:18.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.328+0000 7f107de85640 1 -- 172.21.15.16:0/2495097993 shutdown_connections 2024-09-12T02:13:18.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.328+0000 7f107de85640 1 --2- 172.21.15.16:0/2495097993 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f10640a2c00 0x7f10640a50c0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:18.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.328+0000 7f107de85640 1 --2- 172.21.15.16:0/2495097993 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f107810c170 0x7f107819ef50 unknown :-1 s=CLOSED pgs=1615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:18.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.328+0000 7f107de85640 1 --2- 172.21.15.16:0/2495097993 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1078103660 0x7f107819abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:18.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.328+0000 7f107de85640 1 --2- 172.21.15.16:0/2495097993 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1078102d40 0x7f107819a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:18.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.328+0000 7f107de85640 1 -- 172.21.15.16:0/2495097993 >> 172.21.15.16:0/2495097993 conn(0x7f10780fc7a0 msgr2=0x7f1078101b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:18.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.328+0000 7f107de85640 1 -- 172.21.15.16:0/2495097993 shutdown_connections 2024-09-12T02:13:18.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:18.328+0000 7f107de85640 1 -- 172.21.15.16:0/2495097993 wait complete. 2024-09-12T02:13:18.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:13:18 smithi184 ceph-mon[35593]: pgmap v676: 97 pgs: 97 active+clean; 2.0 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-12T02:13:18.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:13:18 smithi033 ceph-mon[35332]: pgmap v676: 97 pgs: 97 active+clean; 2.0 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-12T02:13:20.096 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-12T02:13:20.097 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-12T02:13:20.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:13:20 smithi184 ceph-mon[35593]: pgmap v677: 97 pgs: 97 active+clean; 2.0 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-09-12T02:13:20.380 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:13:20.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:13:20 smithi033 ceph-mon[35332]: pgmap v677: 97 pgs: 97 active+clean; 2.0 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-09-12T02:13:20.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:13:20 smithi016 ceph-mon[30947]: pgmap v677: 97 pgs: 97 active+clean; 2.0 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-09-12T02:13:20.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.834+0000 7fbd7f398640 1 -- 172.21.15.16:0/1560299658 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd78106f10 msgr2=0x7fbd7810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:20.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.834+0000 7fbd7f398640 1 --2- 172.21.15.16:0/1560299658 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd78106f10 0x7fbd7810ea90 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7fbd70009ca0 tx=0x7fbd7002a8c0 comp rx=0 tx=0).stop 2024-09-12T02:13:20.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.834+0000 7fbd7f398640 1 -- 172.21.15.16:0/1560299658 shutdown_connections 2024-09-12T02:13:20.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.834+0000 7fbd7f398640 1 --2- 172.21.15.16:0/1560299658 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd78106f10 0x7fbd7810ea90 unknown :-1 s=CLOSED pgs=1620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:20.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.834+0000 7fbd7f398640 1 --2- 172.21.15.16:0/1560299658 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd78102790 0x7fbd781067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:20.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.834+0000 7fbd7f398640 1 --2- 172.21.15.16:0/1560299658 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbd78101de0 0x7fbd781021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:20.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.834+0000 7fbd7f398640 1 -- 172.21.15.16:0/1560299658 >> 172.21.15.16:0/1560299658 conn(0x7fbd780fd8c0 msgr2=0x7fbd780ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:20.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.834+0000 7fbd7f398640 1 -- 172.21.15.16:0/1560299658 shutdown_connections 2024-09-12T02:13:20.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.834+0000 7fbd7f398640 1 -- 172.21.15.16:0/1560299658 wait complete. 2024-09-12T02:13:20.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.834+0000 7fbd7f398640 1 Processor -- start 2024-09-12T02:13:20.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.834+0000 7fbd7f398640 1 -- start start 2024-09-12T02:13:20.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.835+0000 7fbd7f398640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd78101de0 0x7fbd780744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:20.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.835+0000 7fbd7f398640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbd78102790 0x7fbd78074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:20.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.835+0000 7fbd7f398640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd78106f10 0x7fbd78074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:20.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.835+0000 7fbd7f398640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbd78112010 con 0x7fbd78101de0 2024-09-12T02:13:20.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.835+0000 7fbd7f398640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbd78111e90 con 0x7fbd78106f10 2024-09-12T02:13:20.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.835+0000 7fbd7f398640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fbd78112190 con 0x7fbd78102790 2024-09-12T02:13:20.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.835+0000 7fbd7d134640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd78101de0 0x7fbd780744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:20.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.835+0000 7fbd7d134640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd78101de0 0x7fbd780744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39544/0 (socket says 172.21.15.16:39544) 2024-09-12T02:13:20.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.835+0000 7fbd7d134640 1 -- 172.21.15.16:0/628524470 learned_addr learned my addr 172.21.15.16:0/628524470 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:13:20.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.835+0000 7fbd7c933640 1 --2- 172.21.15.16:0/628524470 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbd78102790 0x7fbd78074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:20.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.835+0000 7fbd7d935640 1 --2- 172.21.15.16:0/628524470 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd78106f10 0x7fbd78074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:20.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.835+0000 7fbd7d134640 1 -- 172.21.15.16:0/628524470 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbd78102790 msgr2=0x7fbd78074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:20.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.835+0000 7fbd7d134640 1 --2- 172.21.15.16:0/628524470 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbd78102790 0x7fbd78074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:20.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.835+0000 7fbd7d134640 1 -- 172.21.15.16:0/628524470 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd78106f10 msgr2=0x7fbd78074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:20.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.835+0000 7fbd7d134640 1 --2- 172.21.15.16:0/628524470 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd78106f10 0x7fbd78074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:20.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.835+0000 7fbd7d134640 1 -- 172.21.15.16:0/628524470 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd781a6ab0 con 0x7fbd78101de0 2024-09-12T02:13:20.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.835+0000 7fbd7c933640 1 --2- 172.21.15.16:0/628524470 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbd78102790 0x7fbd78074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:13:20.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.835+0000 7fbd7d134640 1 --2- 172.21.15.16:0/628524470 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd78101de0 0x7fbd780744f0 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7fbd7400b4a0 tx=0x7fbd7400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:20.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.836+0000 7fbd6e7fc640 1 -- 172.21.15.16:0/628524470 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd74014070 con 0x7fbd78101de0 2024-09-12T02:13:20.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.836+0000 7fbd6e7fc640 1 -- 172.21.15.16:0/628524470 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd74010af0 con 0x7fbd78101de0 2024-09-12T02:13:20.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.836+0000 7fbd7f398640 1 -- 172.21.15.16:0/628524470 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd781a6da0 con 0x7fbd78101de0 2024-09-12T02:13:20.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.836+0000 7fbd6e7fc640 1 -- 172.21.15.16:0/628524470 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd74017d10 con 0x7fbd78101de0 2024-09-12T02:13:20.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.836+0000 7fbd7f398640 1 -- 172.21.15.16:0/628524470 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd781a7200 con 0x7fbd78101de0 2024-09-12T02:13:20.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.838+0000 7fbd6e7fc640 1 -- 172.21.15.16:0/628524470 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fbd74017070 con 0x7fbd78101de0 2024-09-12T02:13:20.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.838+0000 7fbd81771740 1 Processor -- start 2024-09-12T02:13:20.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.839+0000 7fbd81771740 1 -- start start 2024-09-12T02:13:20.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.839+0000 7fbd81771740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ae34945b60 0x55ae34945f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:20.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.839+0000 7fbd81771740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ae34946480 0x55ae34950800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:20.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.839+0000 7fbd81771740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ae34950d40 0x55ae34953130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:20.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.839+0000 7fbd81771740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55ae348ab280 con 0x55ae34946480 2024-09-12T02:13:20.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.839+0000 7fbd81771740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55ae34852b10 con 0x55ae34950d40 2024-09-12T02:13:20.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.839+0000 7fbd81771740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55ae348a7240 con 0x55ae34945b60 2024-09-12T02:13:20.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.839+0000 7fbd7d134640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ae34946480 0x55ae34950800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:20.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.839+0000 7fbd6e7fc640 1 --2- 172.21.15.16:0/628524470 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fbd640a2b80 0x7fbd640a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:20.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.839+0000 7fbd7d134640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ae34946480 0x55ae34950800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39548/0 (socket says 172.21.15.16:39548) 2024-09-12T02:13:20.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.839+0000 7fbd7d134640 1 -- 172.21.15.16:0/55056446 learned_addr learned my addr 172.21.15.16:0/55056446 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:13:20.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.839+0000 7fbd7d935640 1 --2- 172.21.15.16:0/628524470 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fbd640a2b80 0x7fbd640a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:20.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.839+0000 7fbd7c933640 1 --2- 172.21.15.16:0/55056446 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ae34950d40 0x55ae34953130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:20.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.839+0000 7fbd7d935640 1 --2- 172.21.15.16:0/55056446 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ae34945b60 0x55ae34945f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:20.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.839+0000 7fbd6e7fc640 1 -- 172.21.15.16:0/628524470 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fbd740ce410 con 0x7fbd78101de0 2024-09-12T02:13:20.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.839+0000 7fbd7d134640 1 -- 172.21.15.16:0/55056446 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ae34945b60 msgr2=0x55ae34945f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:20.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.839+0000 7fbd7d134640 1 --2- 172.21.15.16:0/55056446 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ae34945b60 0x55ae34945f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:20.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.839+0000 7fbd7d134640 1 -- 172.21.15.16:0/55056446 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ae34950d40 msgr2=0x55ae34953130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:20.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.839+0000 7fbd7d134640 1 --2- 172.21.15.16:0/55056446 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ae34950d40 0x55ae34953130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:20.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.839+0000 7fbd7d134640 1 -- 172.21.15.16:0/55056446 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae3485f1f0 con 0x55ae34946480 2024-09-12T02:13:20.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.840+0000 7fbd7d134640 1 --2- 172.21.15.16:0/55056446 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ae34946480 0x55ae34950800 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7fbd740a33b0 tx=0x7fbd7400bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b938f63fb07fcd8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:20.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.840+0000 7fbd7d935640 1 --2- 172.21.15.16:0/628524470 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fbd640a2b80 0x7fbd640a5040 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fbd7002f540 tx=0x7fbd700093f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:20.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.840+0000 7fbd637fe640 1 -- 172.21.15.16:0/55056446 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd7409b370 con 0x55ae34946480 2024-09-12T02:13:20.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.840+0000 7fbd637fe640 1 -- 172.21.15.16:0/55056446 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd7401de50 con 0x55ae34946480 2024-09-12T02:13:20.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.840+0000 7fbd637fe640 1 -- 172.21.15.16:0/55056446 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd74010ca0 con 0x55ae34946480 2024-09-12T02:13:20.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.841+0000 7fbd81771740 1 -- 172.21.15.16:0/55056446 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ae34946480 msgr2=0x55ae34950800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:20.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.841+0000 7fbd81771740 1 --2- 172.21.15.16:0/55056446 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ae34946480 0x55ae34950800 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7fbd740a33b0 tx=0x7fbd7400bf90 comp rx=0 tx=0).stop 2024-09-12T02:13:20.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.841+0000 7fbd81771740 1 -- 172.21.15.16:0/55056446 shutdown_connections 2024-09-12T02:13:20.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.841+0000 7fbd81771740 1 --2- 172.21.15.16:0/55056446 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ae34950d40 0x55ae34953130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:20.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.841+0000 7fbd81771740 1 --2- 172.21.15.16:0/55056446 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ae34946480 0x55ae34950800 unknown :-1 s=CLOSED pgs=1622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:20.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.841+0000 7fbd81771740 1 --2- 172.21.15.16:0/55056446 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ae34945b60 0x55ae34945f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:20.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.841+0000 7fbd81771740 1 -- 172.21.15.16:0/55056446 >> 172.21.15.16:0/55056446 conn(0x55ae349420f0 msgr2=0x55ae349424d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:20.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.841+0000 7fbd81771740 1 -- 172.21.15.16:0/55056446 shutdown_connections 2024-09-12T02:13:20.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.841+0000 7fbd81771740 1 -- 172.21.15.16:0/55056446 wait complete. 2024-09-12T02:13:20.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.841+0000 7fbd81771740 1 -- start start 2024-09-12T02:13:20.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.841+0000 7fbd81771740 1 Processor -- start 2024-09-12T02:13:20.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.841+0000 7fbd81771740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ae34945b60 0x55ae3494dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:20.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.841+0000 7fbd81771740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ae34946480 0x55ae3494e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:20.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.841+0000 7fbd81771740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ae34950d40 0x55ae3494e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:20.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.842+0000 7fbd81771740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55ae34953910 con 0x55ae34950d40 2024-09-12T02:13:20.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.842+0000 7fbd81771740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55ae34865f10 con 0x55ae34945b60 2024-09-12T02:13:20.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.842+0000 7fbd81771740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55ae34953a90 con 0x55ae34946480 2024-09-12T02:13:20.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.842+0000 7fbd81771740 10 client.0 _ll_register_callbacks cb 0x55ae3482ea40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:13:20.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.842+0000 7fbd7d935640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ae34950d40 0x55ae3494e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:20.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.842+0000 7fbd7d935640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ae34950d40 0x55ae3494e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39550/0 (socket says 172.21.15.16:39550) 2024-09-12T02:13:20.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.842+0000 7fbd7d935640 1 -- 172.21.15.16:0/1805425663 learned_addr learned my addr 172.21.15.16:0/1805425663 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:13:20.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.842+0000 7fbd7d134640 1 --2- 172.21.15.16:0/1805425663 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ae34945b60 0x55ae3494dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:20.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.842+0000 7fbd7c933640 1 --2- 172.21.15.16:0/1805425663 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ae34946480 0x55ae3494e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:20.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.842+0000 7fbd7d935640 1 -- 172.21.15.16:0/1805425663 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ae34946480 msgr2=0x55ae3494e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:20.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.842+0000 7fbd7d935640 1 --2- 172.21.15.16:0/1805425663 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ae34946480 0x55ae3494e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:20.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.842+0000 7fbd7d935640 1 -- 172.21.15.16:0/1805425663 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ae34945b60 msgr2=0x55ae3494dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:20.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.842+0000 7fbd7d935640 1 --2- 172.21.15.16:0/1805425663 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ae34945b60 0x55ae3494dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:20.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.842+0000 7fbd7d935640 1 -- 172.21.15.16:0/1805425663 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae3461c6a0 con 0x55ae34950d40 2024-09-12T02:13:20.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.842+0000 7fbd7d134640 1 --2- 172.21.15.16:0/1805425663 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ae34945b60 0x55ae3494dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:13:20.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.842+0000 7fbd7d935640 1 --2- 172.21.15.16:0/1805425663 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ae34950d40 0x55ae3494e640 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7fbd70035000 tx=0x7fbd700095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7be45a63aa7c36b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:20.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.842+0000 7fbd60ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:13:20.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.842+0000 7fbd60ff9640 1 -- 172.21.15.16:0/1805425663 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd70050070 con 0x55ae34950d40 2024-09-12T02:13:20.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.842+0000 7fbd60ff9640 1 -- 172.21.15.16:0/1805425663 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fbd700047c0 con 0x55ae34950d40 2024-09-12T02:13:20.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.842+0000 7fbd81771740 1 -- 172.21.15.16:0/1805425663 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbd7401de50 con 0x55ae34950d40 2024-09-12T02:13:20.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.842+0000 7fbd60ff9640 1 -- 172.21.15.16:0/1805425663 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fbd7005b050 con 0x55ae34950d40 2024-09-12T02:13:20.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.843+0000 7fbd81771740 10 client.19212 fetch_fsmap learned FSMap version 25 2024-09-12T02:13:20.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.843+0000 7fbd81771740 1 -- 172.21.15.16:0/1805425663 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ae3495f4f0 con 0x55ae34950d40 2024-09-12T02:13:20.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.843+0000 7fbd60ff9640 1 -- 172.21.15.16:0/1805425663 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd7401de50 con 0x55ae34950d40 2024-09-12T02:13:20.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.843+0000 7fbd60ff9640 1 -- 172.21.15.16:0/1805425663 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fbd70048040 con 0x55ae34950d40 2024-09-12T02:13:20.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.843+0000 7fbd81771740 10 client.19212 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:13:20.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.843+0000 7fbd81771740 10 client.19212 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:13:20.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.843+0000 7fbd81771740 20 client.19212 populate_metadata read hostname 'smithi016' 2024-09-12T02:13:20.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.843+0000 7fbd81771740 1 --2- 172.21.15.16:0/1805425663 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55ae349609a0 0x55ae34962e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:20.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.843+0000 7fbd81771740 4 client.19212 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:13:20.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.843+0000 7fbd81771740 1 -- 172.21.15.16:0/1805425663 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ae348b87f0 con 0x55ae349609a0 2024-09-12T02:13:20.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.843+0000 7fbd7d134640 1 --2- 172.21.15.16:0/1805425663 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55ae349609a0 0x55ae34962e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:20.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.844+0000 7fbd60ff9640 10 client.19212 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:13:20.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.844+0000 7fbd7d134640 1 --2- 172.21.15.16:0/1805425663 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55ae349609a0 0x55ae34962e00 crc :-1 s=READY pgs=344 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3c85ce83f793e6f server_cookie=cab84b1167b307fd in_seq=0 out_seq=0 2024-09-12T02:13:20.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.844+0000 7fbd60ff9640 1 -- 172.21.15.16:0/1805425663 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ae348b87f0 con 0x55ae349609a0 2024-09-12T02:13:20.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.844+0000 7fbd60ff9640 10 client.19212 handle_command_reply: tid=0 2024-09-12T02:13:20.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.844+0000 7fbd81771740 1 client.19212 shutdown 2024-09-12T02:13:20.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.844+0000 7fbd81771740 1 -- 172.21.15.16:0/1805425663 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ae34950d40 msgr2=0x55ae3494e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:20.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.844+0000 7fbd81771740 1 --2- 172.21.15.16:0/1805425663 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ae34950d40 0x55ae3494e640 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7fbd70035000 tx=0x7fbd700095c0 comp rx=0 tx=0).stop 2024-09-12T02:13:20.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.844+0000 7fbd81771740 1 -- 172.21.15.16:0/1805425663 shutdown_connections 2024-09-12T02:13:20.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.844+0000 7fbd81771740 1 --2- 172.21.15.16:0/1805425663 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55ae349609a0 0x55ae34962e00 crc :-1 s=READY pgs=344 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:20.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.844+0000 7fbd81771740 1 --2- 172.21.15.16:0/1805425663 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ae34950d40 0x55ae3494e640 unknown :-1 s=CLOSED pgs=1623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:20.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.844+0000 7fbd81771740 1 --2- 172.21.15.16:0/1805425663 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ae34946480 0x55ae3494e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:20.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.844+0000 7fbd81771740 1 --2- 172.21.15.16:0/1805425663 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ae34945b60 0x55ae3494dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:20.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.844+0000 7fbd60ff9640 0 client.19212 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:13:20.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.844+0000 7fbd81771740 1 -- 172.21.15.16:0/1805425663 >> 172.21.15.16:0/1805425663 conn(0x55ae349420f0 msgr2=0x55ae34947dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:20.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.845+0000 7fbd81771740 1 -- 172.21.15.16:0/1805425663 shutdown_connections 2024-09-12T02:13:20.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.845+0000 7fbd81771740 1 -- 172.21.15.16:0/1805425663 wait complete. 2024-09-12T02:13:20.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.845+0000 7fbd81771740 20 client.19212 trim_cache size 0 max 16384 2024-09-12T02:13:20.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.862+0000 7fbd81771740 1 Processor -- start 2024-09-12T02:13:20.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.862+0000 7fbd81771740 1 -- start start 2024-09-12T02:13:20.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.862+0000 7fbd81771740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ae34945b60 0x55ae349485c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:20.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.862+0000 7fbd81771740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ae34946480 0x55ae3494fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:20.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.862+0000 7fbd81771740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ae34950d40 0x55ae349686d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:20.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.863+0000 7fbd81771740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbd781b1f40 con 0x55ae34945b60 2024-09-12T02:13:20.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.863+0000 7fbd81771740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbd781af810 con 0x55ae34946480 2024-09-12T02:13:20.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.863+0000 7fbd81771740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fbd781ad6a0 con 0x55ae34950d40 2024-09-12T02:13:20.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.863+0000 7fbd7d134640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ae34945b60 0x55ae349485c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:20.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.863+0000 7fbd7d134640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ae34945b60 0x55ae349485c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39552/0 (socket says 172.21.15.16:39552) 2024-09-12T02:13:20.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.863+0000 7fbd7d134640 1 -- 172.21.15.16:0/3271649503 learned_addr learned my addr 172.21.15.16:0/3271649503 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:13:20.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.863+0000 7fbd7c933640 1 --2- 172.21.15.16:0/3271649503 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ae34946480 0x55ae3494fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:20.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.863+0000 7fbd7d935640 1 --2- 172.21.15.16:0/3271649503 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ae34950d40 0x55ae349686d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:20.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.863+0000 7fbd7d134640 1 -- 172.21.15.16:0/3271649503 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ae34950d40 msgr2=0x55ae349686d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:20.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.863+0000 7fbd7d134640 1 --2- 172.21.15.16:0/3271649503 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ae34950d40 0x55ae349686d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:20.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.863+0000 7fbd7d134640 1 -- 172.21.15.16:0/3271649503 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ae34946480 msgr2=0x55ae3494fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:20.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.863+0000 7fbd7d134640 1 --2- 172.21.15.16:0/3271649503 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ae34946480 0x55ae3494fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:20.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.863+0000 7fbd7d134640 1 -- 172.21.15.16:0/3271649503 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae3493d830 con 0x55ae34945b60 2024-09-12T02:13:20.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.863+0000 7fbd7c933640 1 --2- 172.21.15.16:0/3271649503 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ae34946480 0x55ae3494fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:13:20.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.863+0000 7fbd7d134640 1 --2- 172.21.15.16:0/3271649503 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ae34945b60 0x55ae349485c0 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7fbd74009420 tx=0x7fbd740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=699b043d59fc66d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:20.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.863+0000 7fbd62ffd640 1 -- 172.21.15.16:0/3271649503 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd7401e040 con 0x55ae34945b60 2024-09-12T02:13:20.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.863+0000 7fbd62ffd640 1 -- 172.21.15.16:0/3271649503 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd740b8070 con 0x55ae34945b60 2024-09-12T02:13:20.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.863+0000 7fbd62ffd640 1 -- 172.21.15.16:0/3271649503 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd74010ca0 con 0x55ae34945b60 2024-09-12T02:13:20.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.864+0000 7fbd81771740 1 -- 172.21.15.16:0/3271649503 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ae34945b60 msgr2=0x55ae349485c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:20.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.864+0000 7fbd81771740 1 --2- 172.21.15.16:0/3271649503 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ae34945b60 0x55ae349485c0 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7fbd74009420 tx=0x7fbd740046c0 comp rx=0 tx=0).stop 2024-09-12T02:13:20.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.864+0000 7fbd81771740 1 -- 172.21.15.16:0/3271649503 shutdown_connections 2024-09-12T02:13:20.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.864+0000 7fbd81771740 1 --2- 172.21.15.16:0/3271649503 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ae34950d40 0x55ae349686d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:20.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.864+0000 7fbd81771740 1 --2- 172.21.15.16:0/3271649503 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ae34946480 0x55ae3494fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:20.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.864+0000 7fbd81771740 1 --2- 172.21.15.16:0/3271649503 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ae34945b60 0x55ae349485c0 unknown :-1 s=CLOSED pgs=1624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:20.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.864+0000 7fbd81771740 1 -- 172.21.15.16:0/3271649503 >> 172.21.15.16:0/3271649503 conn(0x55ae349420f0 msgr2=0x55ae3494f310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:20.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.864+0000 7fbd81771740 1 -- 172.21.15.16:0/3271649503 shutdown_connections 2024-09-12T02:13:20.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.864+0000 7fbd81771740 1 -- 172.21.15.16:0/3271649503 wait complete. 2024-09-12T02:13:20.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.864+0000 7fbd81771740 1 -- start start 2024-09-12T02:13:20.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.865+0000 7fbd81771740 1 Processor -- start 2024-09-12T02:13:20.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.865+0000 7fbd81771740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ae34945b60 0x55ae34959500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:20.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.865+0000 7fbd81771740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ae34946480 0x55ae3494a980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:20.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.865+0000 7fbd81771740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ae34950d40 0x55ae3494af00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:20.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.865+0000 7fbd81771740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbd781adcf0 con 0x55ae34946480 2024-09-12T02:13:20.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.865+0000 7fbd81771740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbd781af140 con 0x55ae34950d40 2024-09-12T02:13:20.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.865+0000 7fbd81771740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fbd781ae3a0 con 0x55ae34945b60 2024-09-12T02:13:20.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.865+0000 7fbd81771740 10 client.0 _ll_register_callbacks cb 0x7fbd70009900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:13:20.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.865+0000 7fbd7c933640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ae34946480 0x55ae3494a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:20.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.865+0000 7fbd7c933640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ae34946480 0x55ae3494a980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39554/0 (socket says 172.21.15.16:39554) 2024-09-12T02:13:20.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.865+0000 7fbd7c933640 1 -- 172.21.15.16:0/3198581319 learned_addr learned my addr 172.21.15.16:0/3198581319 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:13:20.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.865+0000 7fbd7d134640 1 --2- 172.21.15.16:0/3198581319 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ae34945b60 0x55ae34959500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:20.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.865+0000 7fbd7d935640 1 --2- 172.21.15.16:0/3198581319 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ae34950d40 0x55ae3494af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:20.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.865+0000 7fbd7c933640 1 -- 172.21.15.16:0/3198581319 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ae34945b60 msgr2=0x55ae34959500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:20.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.865+0000 7fbd7c933640 1 --2- 172.21.15.16:0/3198581319 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ae34945b60 0x55ae34959500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:20.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.865+0000 7fbd7c933640 1 -- 172.21.15.16:0/3198581319 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ae34950d40 msgr2=0x55ae3494af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:20.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.865+0000 7fbd7c933640 1 --2- 172.21.15.16:0/3198581319 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ae34950d40 0x55ae3494af00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:20.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.866+0000 7fbd7c933640 1 -- 172.21.15.16:0/3198581319 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae3494b730 con 0x55ae34946480 2024-09-12T02:13:20.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.866+0000 7fbd7c933640 1 --2- 172.21.15.16:0/3198581319 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ae34946480 0x55ae3494a980 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7fbd78075f80 tx=0x7fbd68015b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=492e17a0bd6b478e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:20.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.866+0000 7fbd60ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:13:20.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.866+0000 7fbd60ff9640 1 -- 172.21.15.16:0/3198581319 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd680138a0 con 0x55ae34946480 2024-09-12T02:13:20.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.866+0000 7fbd60ff9640 1 -- 172.21.15.16:0/3198581319 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fbd68013a40 con 0x55ae34946480 2024-09-12T02:13:20.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.866+0000 7fbd81771740 1 -- 172.21.15.16:0/3198581319 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbd740b8070 con 0x55ae34946480 2024-09-12T02:13:20.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.866+0000 7fbd60ff9640 1 -- 172.21.15.16:0/3198581319 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fbd68013c40 con 0x55ae34946480 2024-09-12T02:13:20.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.866+0000 7fbd81771740 10 client.19218 fetch_fsmap learned FSMap version 25 2024-09-12T02:13:20.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.866+0000 7fbd81771740 1 -- 172.21.15.16:0/3198581319 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ae349cb7e0 con 0x55ae34946480 2024-09-12T02:13:20.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.866+0000 7fbd60ff9640 1 -- 172.21.15.16:0/3198581319 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd740b8070 con 0x55ae34946480 2024-09-12T02:13:20.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.867+0000 7fbd60ff9640 1 -- 172.21.15.16:0/3198581319 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fbd68016430 con 0x55ae34946480 2024-09-12T02:13:20.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.867+0000 7fbd81771740 10 client.19218 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:13:20.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.867+0000 7fbd81771740 10 client.19218 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:13:20.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.867+0000 7fbd81771740 20 client.19218 populate_metadata read hostname 'smithi016' 2024-09-12T02:13:20.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.867+0000 7fbd81771740 1 --2- 172.21.15.16:0/3198581319 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55ae349cbfc0 0x55ae349ce3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:20.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.867+0000 7fbd81771740 4 client.19218 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-12T02:13:20.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.867+0000 7fbd81771740 1 -- 172.21.15.16:0/3198581319 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55ae348b2d00 con 0x55ae349cbfc0 2024-09-12T02:13:20.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.867+0000 7fbd7d134640 1 --2- 172.21.15.16:0/3198581319 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55ae349cbfc0 0x55ae349ce3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:20.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.867+0000 7fbd7d134640 1 --2- 172.21.15.16:0/3198581319 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55ae349cbfc0 0x55ae349ce3e0 crc :-1 s=READY pgs=345 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cfe595e91416f617 server_cookie=9eed619a6992b4b9 in_seq=0 out_seq=0 2024-09-12T02:13:20.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.867+0000 7fbd60ff9640 10 client.19218 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:13:20.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.868+0000 7fbd60ff9640 1 -- 172.21.15.16:0/3198581319 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55ae348b2d00 con 0x55ae349cbfc0 2024-09-12T02:13:20.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.868+0000 7fbd60ff9640 10 client.19218 handle_command_reply: tid=0 2024-09-12T02:13:20.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.868+0000 7fbd81771740 1 client.19218 shutdown 2024-09-12T02:13:20.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.868+0000 7fbd81771740 1 -- 172.21.15.16:0/3198581319 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ae34946480 msgr2=0x55ae3494a980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:20.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.868+0000 7fbd81771740 1 --2- 172.21.15.16:0/3198581319 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ae34946480 0x55ae3494a980 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7fbd78075f80 tx=0x7fbd68015b60 comp rx=0 tx=0).stop 2024-09-12T02:13:20.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.869+0000 7fbd81771740 1 -- 172.21.15.16:0/3198581319 shutdown_connections 2024-09-12T02:13:20.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.869+0000 7fbd81771740 1 --2- 172.21.15.16:0/3198581319 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55ae349cbfc0 0x55ae349ce3e0 crc :-1 s=READY pgs=345 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:20.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.869+0000 7fbd81771740 1 --2- 172.21.15.16:0/3198581319 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ae34950d40 0x55ae3494af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:20.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.869+0000 7fbd81771740 1 --2- 172.21.15.16:0/3198581319 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ae34946480 0x55ae3494a980 unknown :-1 s=CLOSED pgs=1625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:20.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.869+0000 7fbd60ff9640 0 client.19218 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:13:20.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.869+0000 7fbd81771740 1 --2- 172.21.15.16:0/3198581319 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ae34945b60 0x55ae34959500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:20.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.869+0000 7fbd81771740 1 -- 172.21.15.16:0/3198581319 >> 172.21.15.16:0/3198581319 conn(0x55ae349420f0 msgr2=0x55ae34941270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:20.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.869+0000 7fbd81771740 1 -- 172.21.15.16:0/3198581319 shutdown_connections 2024-09-12T02:13:20.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.869+0000 7fbd81771740 1 -- 172.21.15.16:0/3198581319 wait complete. 2024-09-12T02:13:20.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.869+0000 7fbd81771740 20 client.19218 trim_cache size 0 max 16384 2024-09-12T02:13:20.877 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T02:13:20.877 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-09-12T02:13:20.877 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "aff6469d-c856-4e70-8dcd-335b25ea76cf", 2024-09-12T02:13:20.877 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-09-12T02:13:20.877 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T02:13:20.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.869+0000 7fbd7f398640 1 -- 172.21.15.16:0/628524470 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fbd640a2b80 msgr2=0x7fbd640a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:20.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.869+0000 7fbd7f398640 1 --2- 172.21.15.16:0/628524470 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fbd640a2b80 0x7fbd640a5040 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fbd7002f540 tx=0x7fbd700093f0 comp rx=0 tx=0).stop 2024-09-12T02:13:20.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.869+0000 7fbd7f398640 1 -- 172.21.15.16:0/628524470 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd78101de0 msgr2=0x7fbd780744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:20.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.869+0000 7fbd7f398640 1 --2- 172.21.15.16:0/628524470 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd78101de0 0x7fbd780744f0 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7fbd7400b4a0 tx=0x7fbd7400b960 comp rx=0 tx=0).stop 2024-09-12T02:13:20.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.869+0000 7fbd7f398640 1 -- 172.21.15.16:0/628524470 shutdown_connections 2024-09-12T02:13:20.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.869+0000 7fbd7f398640 1 --2- 172.21.15.16:0/628524470 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fbd640a2b80 0x7fbd640a5040 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:20.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.870+0000 7fbd7f398640 1 --2- 172.21.15.16:0/628524470 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd78106f10 0x7fbd78074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:20.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.870+0000 7fbd7f398640 1 --2- 172.21.15.16:0/628524470 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbd78102790 0x7fbd78074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:20.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.870+0000 7fbd7f398640 1 --2- 172.21.15.16:0/628524470 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd78101de0 0x7fbd780744f0 unknown :-1 s=CLOSED pgs=1621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:20.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.870+0000 7fbd7f398640 1 -- 172.21.15.16:0/628524470 >> 172.21.15.16:0/628524470 conn(0x7fbd780fd8c0 msgr2=0x7fbd780fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:20.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.870+0000 7fbd7f398640 1 -- 172.21.15.16:0/628524470 shutdown_connections 2024-09-12T02:13:20.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:20.870+0000 7fbd7f398640 1 -- 172.21.15.16:0/628524470 wait complete. 2024-09-12T02:13:21.091 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.089+0000 7fefa759e640 1 -- 172.21.15.16:0/3563535389 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fefa81023f0 msgr2=0x7fefa8108c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:21.091 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.089+0000 7fefa759e640 1 --2- 172.21.15.16:0/3563535389 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fefa81023f0 0x7fefa8108c80 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7fefa0009ca0 tx=0x7fefa002a8c0 comp rx=0 tx=0).stop 2024-09-12T02:13:21.091 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.089+0000 7fefa759e640 1 -- 172.21.15.16:0/3563535389 shutdown_connections 2024-09-12T02:13:21.091 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.089+0000 7fefa759e640 1 --2- 172.21.15.16:0/3563535389 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fefa81023f0 0x7fefa8108c80 unknown :-1 s=CLOSED pgs=1626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:21.091 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.089+0000 7fefa759e640 1 --2- 172.21.15.16:0/3563535389 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fefa8101a30 0x7fefa8101eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:21.091 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.089+0000 7fefa759e640 1 --2- 172.21.15.16:0/3563535389 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fefa8100840 0x7fefa8100c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:21.091 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.089+0000 7fefa759e640 1 -- 172.21.15.16:0/3563535389 >> 172.21.15.16:0/3563535389 conn(0x7fefa80fbfa0 msgr2=0x7fefa80fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:21.092 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.089+0000 7fefa759e640 1 -- 172.21.15.16:0/3563535389 shutdown_connections 2024-09-12T02:13:21.092 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.089+0000 7fefa759e640 1 -- 172.21.15.16:0/3563535389 wait complete. 2024-09-12T02:13:21.092 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.090+0000 7fefa759e640 1 Processor -- start 2024-09-12T02:13:21.092 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.090+0000 7fefa759e640 1 -- start start 2024-09-12T02:13:21.092 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.090+0000 7fefa759e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fefa8100840 0x7fefa8104760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:21.092 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.090+0000 7fefa759e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fefa8101a30 0x7fefa8102e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:21.092 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.090+0000 7fefa759e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fefa81023f0 0x7fefa8103350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:21.092 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.090+0000 7fefa759e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fefa810c4c0 con 0x7fefa8100840 2024-09-12T02:13:21.092 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.090+0000 7fefa759e640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fefa810c340 con 0x7fefa8101a30 2024-09-12T02:13:21.092 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.090+0000 7fefa759e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fefa810c640 con 0x7fefa81023f0 2024-09-12T02:13:21.092 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.090+0000 7fefa659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fefa8100840 0x7fefa8104760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:21.093 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.090+0000 7fefa659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fefa8100840 0x7fefa8104760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39560/0 (socket says 172.21.15.16:39560) 2024-09-12T02:13:21.093 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.090+0000 7fefa659c640 1 -- 172.21.15.16:0/3107601085 learned_addr learned my addr 172.21.15.16:0/3107601085 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:13:21.093 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.090+0000 7fefa6d9d640 1 --2- 172.21.15.16:0/3107601085 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fefa81023f0 0x7fefa8103350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:21.093 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.090+0000 7fefa5d9b640 1 --2- 172.21.15.16:0/3107601085 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fefa8101a30 0x7fefa8102e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:21.093 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.090+0000 7fefa659c640 1 -- 172.21.15.16:0/3107601085 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fefa81023f0 msgr2=0x7fefa8103350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:21.093 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.090+0000 7fefa659c640 1 --2- 172.21.15.16:0/3107601085 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fefa81023f0 0x7fefa8103350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:21.093 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.090+0000 7fefa659c640 1 -- 172.21.15.16:0/3107601085 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fefa8101a30 msgr2=0x7fefa8102e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:21.093 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.090+0000 7fefa659c640 1 --2- 172.21.15.16:0/3107601085 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fefa8101a30 0x7fefa8102e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:21.093 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.090+0000 7fefa659c640 1 -- 172.21.15.16:0/3107601085 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefa8103b70 con 0x7fefa8100840 2024-09-12T02:13:21.093 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.090+0000 7fefa6d9d640 1 --2- 172.21.15.16:0/3107601085 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fefa81023f0 0x7fefa8103350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:13:21.093 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.090+0000 7fefa659c640 1 --2- 172.21.15.16:0/3107601085 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fefa8100840 0x7fefa8104760 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7fef9800c970 tx=0x7fef9800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:21.093 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.091+0000 7fef977fe640 1 -- 172.21.15.16:0/3107601085 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef98004220 con 0x7fefa8100840 2024-09-12T02:13:21.094 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.091+0000 7fef977fe640 1 -- 172.21.15.16:0/3107601085 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef980043c0 con 0x7fefa8100840 2024-09-12T02:13:21.094 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.091+0000 7fefa759e640 1 -- 172.21.15.16:0/3107601085 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefa81a1d20 con 0x7fefa8100840 2024-09-12T02:13:21.094 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.091+0000 7fef977fe640 1 -- 172.21.15.16:0/3107601085 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef98004a00 con 0x7fefa8100840 2024-09-12T02:13:21.094 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.091+0000 7fefa759e640 1 -- 172.21.15.16:0/3107601085 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefa81a21c0 con 0x7fefa8100840 2024-09-12T02:13:21.100 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.093+0000 7fefafa75740 1 Processor -- start 2024-09-12T02:13:21.100 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.094+0000 7fefafa75740 1 -- start start 2024-09-12T02:13:21.100 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.094+0000 7fef977fe640 1 -- 172.21.15.16:0/3107601085 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fef98004ba0 con 0x7fefa8100840 2024-09-12T02:13:21.100 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.094+0000 7fefafa75740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5625602b7b70 0x5625602b7f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:21.100 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.094+0000 7fefafa75740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5625602b84b0 0x5625602bcf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:21.100 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.094+0000 7fefafa75740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5625602bd540 0x5625602bfa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:21.100 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.094+0000 7fefafa75740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56256021b2f0 con 0x5625602bd540 2024-09-12T02:13:21.100 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.094+0000 7fefafa75740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5625601c3270 con 0x5625602b84b0 2024-09-12T02:13:21.100 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.094+0000 7fefafa75740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5625602172b0 con 0x5625602b7b70 2024-09-12T02:13:21.100 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.094+0000 7fefa5d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5625602bd540 0x5625602bfa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:21.101 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.094+0000 7fefa5d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5625602bd540 0x5625602bfa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39574/0 (socket says 172.21.15.16:39574) 2024-09-12T02:13:21.101 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.094+0000 7fefa5d9b640 1 -- 172.21.15.16:0/1811568464 learned_addr learned my addr 172.21.15.16:0/1811568464 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:13:21.101 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.094+0000 7fefa659c640 1 --2- 172.21.15.16:0/1811568464 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5625602b84b0 0x5625602bcf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:21.101 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.094+0000 7fefa6d9d640 1 --2- 172.21.15.16:0/1811568464 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5625602b7b70 0x5625602b7f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:21.101 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.094+0000 7fefa5d9b640 1 -- 172.21.15.16:0/1811568464 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5625602b7b70 msgr2=0x5625602b7f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:21.101 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.094+0000 7fefa5d9b640 1 --2- 172.21.15.16:0/1811568464 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5625602b7b70 0x5625602b7f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:21.101 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.094+0000 7fefa5d9b640 1 -- 172.21.15.16:0/1811568464 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5625602b84b0 msgr2=0x5625602bcf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:21.101 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.094+0000 7fefa5d9b640 1 --2- 172.21.15.16:0/1811568464 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5625602b84b0 0x5625602bcf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:21.101 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.094+0000 7fefa5d9b640 1 -- 172.21.15.16:0/1811568464 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5625601c2340 con 0x5625602bd540 2024-09-12T02:13:21.101 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.094+0000 7fefa659c640 1 --2- 172.21.15.16:0/1811568464 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5625602b84b0 0x5625602bcf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:13:21.101 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.095+0000 7fefa5d9b640 1 --2- 172.21.15.16:0/1811568464 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5625602bd540 0x5625602bfa00 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7fef9c00cc40 tx=0x7fef9c00e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed402a4f6a36aefa server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:21.101 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.095+0000 7fef977fe640 1 --2- 172.21.15.16:0/3107601085 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fef900a2c00 0x7fef900a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:21.102 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.095+0000 7fefa6d9d640 1 --2- 172.21.15.16:0/3107601085 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fef900a2c00 0x7fef900a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:21.102 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.095+0000 7fef94ff9640 1 -- 172.21.15.16:0/1811568464 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef9c014070 con 0x5625602bd540 2024-09-12T02:13:21.102 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.095+0000 7fef94ff9640 1 -- 172.21.15.16:0/1811568464 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef9c0047a0 con 0x5625602bd540 2024-09-12T02:13:21.102 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.095+0000 7fef94ff9640 1 -- 172.21.15.16:0/1811568464 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef9c010760 con 0x5625602bd540 2024-09-12T02:13:21.102 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.095+0000 7fef977fe640 1 -- 172.21.15.16:0/3107601085 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fef980ced50 con 0x7fefa8100840 2024-09-12T02:13:21.102 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.096+0000 7fefafa75740 1 -- 172.21.15.16:0/1811568464 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5625602bd540 msgr2=0x5625602bfa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:21.102 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.096+0000 7fefa6d9d640 1 --2- 172.21.15.16:0/3107601085 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fef900a2c00 0x7fef900a50c0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fefa0006010 tx=0x7fefa0005fa0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:21.102 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.096+0000 7fefafa75740 1 --2- 172.21.15.16:0/1811568464 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5625602bd540 0x5625602bfa00 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7fef9c00cc40 tx=0x7fef9c00e720 comp rx=0 tx=0).stop 2024-09-12T02:13:21.102 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.096+0000 7fefafa75740 1 -- 172.21.15.16:0/1811568464 shutdown_connections 2024-09-12T02:13:21.102 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.096+0000 7fefafa75740 1 --2- 172.21.15.16:0/1811568464 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5625602bd540 0x5625602bfa00 unknown :-1 s=CLOSED pgs=1628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:21.102 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.096+0000 7fefafa75740 1 --2- 172.21.15.16:0/1811568464 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5625602b84b0 0x5625602bcf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:21.102 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.096+0000 7fefafa75740 1 --2- 172.21.15.16:0/1811568464 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5625602b7b70 0x5625602b7f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:21.103 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.096+0000 7fefafa75740 1 -- 172.21.15.16:0/1811568464 >> 172.21.15.16:0/1811568464 conn(0x5625602b31e0 msgr2=0x5625602b35e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:21.103 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.096+0000 7fefafa75740 1 -- 172.21.15.16:0/1811568464 shutdown_connections 2024-09-12T02:13:21.103 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.096+0000 7fefafa75740 1 -- 172.21.15.16:0/1811568464 wait complete. 2024-09-12T02:13:21.103 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.096+0000 7fefafa75740 1 -- start start 2024-09-12T02:13:21.103 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.096+0000 7fefafa75740 1 Processor -- start 2024-09-12T02:13:21.103 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.096+0000 7fefafa75740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5625602b7b70 0x5625602c1d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:21.103 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.096+0000 7fefafa75740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5625602b84b0 0x5625602c2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:21.103 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.096+0000 7fefafa75740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5625602bd540 0x5625602c92b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:21.103 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.096+0000 7fefafa75740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5625602c0510 con 0x5625602b7b70 2024-09-12T02:13:21.103 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.096+0000 7fefafa75740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5625601a4290 con 0x5625602b84b0 2024-09-12T02:13:21.103 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.096+0000 7fefafa75740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5625602c0690 con 0x5625602bd540 2024-09-12T02:13:21.103 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.096+0000 7fefafa75740 10 client.0 _ll_register_callbacks cb 0x5625601d5140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:13:21.103 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.096+0000 7fefa659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5625602b7b70 0x5625602c1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:21.104 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.097+0000 7fefa659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5625602b7b70 0x5625602c1d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39584/0 (socket says 172.21.15.16:39584) 2024-09-12T02:13:21.104 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.097+0000 7fefa659c640 1 -- 172.21.15.16:0/3761711261 learned_addr learned my addr 172.21.15.16:0/3761711261 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:13:21.104 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.097+0000 7fefa5d9b640 1 --2- 172.21.15.16:0/3761711261 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5625602b84b0 0x5625602c2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:21.104 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.097+0000 7fefa659c640 1 -- 172.21.15.16:0/3761711261 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5625602bd540 msgr2=0x5625602c92b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:21.104 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.097+0000 7fefa6d9d640 1 --2- 172.21.15.16:0/3761711261 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5625602bd540 0x5625602c92b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:21.104 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.097+0000 7fefa659c640 1 --2- 172.21.15.16:0/3761711261 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5625602bd540 0x5625602c92b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:21.104 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.097+0000 7fefa659c640 1 -- 172.21.15.16:0/3761711261 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5625602b84b0 msgr2=0x5625602c2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:21.104 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.097+0000 7fefa659c640 1 --2- 172.21.15.16:0/3761711261 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5625602b84b0 0x5625602c2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:21.104 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.097+0000 7fefa659c640 1 -- 172.21.15.16:0/3761711261 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56255ff2e210 con 0x5625602b7b70 2024-09-12T02:13:21.104 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.097+0000 7fefa6d9d640 1 --2- 172.21.15.16:0/3761711261 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5625602bd540 0x5625602c92b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:13:21.104 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.097+0000 7fefa659c640 1 --2- 172.21.15.16:0/3761711261 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5625602b7b70 0x5625602c1d30 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7fef98097ab0 tx=0x7fef980d2640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=568e66cbe3c6810 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:21.104 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.097+0000 7fef8dffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:13:21.104 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.097+0000 7fef8dffb640 1 -- 172.21.15.16:0/3761711261 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef9801d040 con 0x5625602b7b70 2024-09-12T02:13:21.105 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.097+0000 7fef8dffb640 1 -- 172.21.15.16:0/3761711261 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fef9809bab0 con 0x5625602b7b70 2024-09-12T02:13:21.105 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.097+0000 7fefafa75740 1 -- 172.21.15.16:0/3761711261 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef9c0047a0 con 0x5625602b7b70 2024-09-12T02:13:21.105 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.097+0000 7fef8dffb640 1 -- 172.21.15.16:0/3761711261 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fef9809afb0 con 0x5625602b7b70 2024-09-12T02:13:21.105 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.097+0000 7fefafa75740 10 client.19230 fetch_fsmap learned FSMap version 25 2024-09-12T02:13:21.105 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.097+0000 7fefafa75740 1 -- 172.21.15.16:0/3761711261 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56255fffa620 con 0x5625602b7b70 2024-09-12T02:13:21.105 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.098+0000 7fef8dffb640 1 -- 172.21.15.16:0/3761711261 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef9c0047a0 con 0x5625602b7b70 2024-09-12T02:13:21.105 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.098+0000 7fef8dffb640 1 -- 172.21.15.16:0/3761711261 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fef980d2020 con 0x5625602b7b70 2024-09-12T02:13:21.105 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.098+0000 7fefafa75740 10 client.19230 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:13:21.105 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.098+0000 7fefafa75740 10 client.19230 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:13:21.105 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.098+0000 7fefafa75740 20 client.19230 populate_metadata read hostname 'smithi016' 2024-09-12T02:13:21.105 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.098+0000 7fefafa75740 1 --2- 172.21.15.16:0/3761711261 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5625602cda00 0x5625602cfdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:21.105 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.098+0000 7fefafa75740 4 client.19230 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:13:21.106 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.098+0000 7fefafa75740 1 -- 172.21.15.16:0/3761711261 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562560228a20 con 0x5625602cda00 2024-09-12T02:13:21.106 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.098+0000 7fefa5d9b640 1 --2- 172.21.15.16:0/3761711261 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5625602cda00 0x5625602cfdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:21.106 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.098+0000 7fef8dffb640 10 client.19230 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:13:21.106 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.098+0000 7fefa5d9b640 1 --2- 172.21.15.16:0/3761711261 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5625602cda00 0x5625602cfdf0 crc :-1 s=READY pgs=346 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b43fa11429d491dc server_cookie=613d5acd813513e4 in_seq=0 out_seq=0 2024-09-12T02:13:21.106 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.099+0000 7fef8dffb640 1 -- 172.21.15.16:0/3761711261 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562560228a20 con 0x5625602cda00 2024-09-12T02:13:21.106 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.099+0000 7fef8dffb640 10 client.19230 handle_command_reply: tid=0 2024-09-12T02:13:21.106 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.099+0000 7fefafa75740 1 client.19230 shutdown 2024-09-12T02:13:21.106 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.099+0000 7fefafa75740 1 -- 172.21.15.16:0/3761711261 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5625602b7b70 msgr2=0x5625602c1d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:21.106 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.099+0000 7fefafa75740 1 --2- 172.21.15.16:0/3761711261 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5625602b7b70 0x5625602c1d30 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7fef98097ab0 tx=0x7fef980d2640 comp rx=0 tx=0).stop 2024-09-12T02:13:21.106 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.099+0000 7fefafa75740 1 -- 172.21.15.16:0/3761711261 shutdown_connections 2024-09-12T02:13:21.106 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.099+0000 7fefafa75740 1 --2- 172.21.15.16:0/3761711261 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5625602cda00 0x5625602cfdf0 crc :-1 s=READY pgs=346 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:21.106 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.099+0000 7fefafa75740 1 --2- 172.21.15.16:0/3761711261 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5625602bd540 0x5625602c92b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:21.107 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.099+0000 7fefafa75740 1 --2- 172.21.15.16:0/3761711261 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5625602b84b0 0x5625602c2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:21.107 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.099+0000 7fef8dffb640 0 client.19230 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:13:21.107 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.099+0000 7fefafa75740 1 --2- 172.21.15.16:0/3761711261 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5625602b7b70 0x5625602c1d30 unknown :-1 s=CLOSED pgs=1629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:21.107 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.099+0000 7fefafa75740 1 -- 172.21.15.16:0/3761711261 >> 172.21.15.16:0/3761711261 conn(0x5625602b31e0 msgr2=0x5625602b49f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:21.107 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.099+0000 7fefafa75740 1 -- 172.21.15.16:0/3761711261 shutdown_connections 2024-09-12T02:13:21.107 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.099+0000 7fefafa75740 1 -- 172.21.15.16:0/3761711261 wait complete. 2024-09-12T02:13:21.107 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.100+0000 7fefafa75740 20 client.19230 trim_cache size 0 max 16384 2024-09-12T02:13:21.115 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.114+0000 7fefafa75740 1 Processor -- start 2024-09-12T02:13:21.115 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.114+0000 7fefafa75740 1 -- start start 2024-09-12T02:13:21.115 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.114+0000 7fefafa75740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5625602b7b70 0x5625602c2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:21.115 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.114+0000 7fefafa75740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5625602b84b0 0x5625602c9270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:21.115 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.114+0000 7fefafa75740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5625602bd540 0x5625602b47a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:21.115 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.114+0000 7fefafa75740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fefa81ad700 con 0x5625602bd540 2024-09-12T02:13:21.115 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.114+0000 7fefafa75740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fefa81aafd0 con 0x5625602b84b0 2024-09-12T02:13:21.116 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.114+0000 7fefafa75740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fefa81a8e20 con 0x5625602b7b70 2024-09-12T02:13:21.116 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.114+0000 7fefa6d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5625602bd540 0x5625602b47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:21.116 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.114+0000 7fefa6d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5625602bd540 0x5625602b47a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39600/0 (socket says 172.21.15.16:39600) 2024-09-12T02:13:21.116 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.114+0000 7fefa6d9d640 1 -- 172.21.15.16:0/2851315306 learned_addr learned my addr 172.21.15.16:0/2851315306 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:13:21.116 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.115+0000 7fefa6d9d640 1 -- 172.21.15.16:0/2851315306 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5625602b7b70 msgr2=0x5625602c2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:21.116 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.115+0000 7fefa6d9d640 1 --2- 172.21.15.16:0/2851315306 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5625602b7b70 0x5625602c2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:21.116 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.115+0000 7fefa6d9d640 1 -- 172.21.15.16:0/2851315306 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5625602b84b0 msgr2=0x5625602c9270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:21.116 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.115+0000 7fefa5d9b640 1 --2- 172.21.15.16:0/2851315306 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5625602b84b0 0x5625602c9270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:21.116 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.115+0000 7fefa6d9d640 1 --2- 172.21.15.16:0/2851315306 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5625602b84b0 0x5625602c9270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:21.116 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.115+0000 7fefa6d9d640 1 -- 172.21.15.16:0/2851315306 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5625602d0160 con 0x5625602bd540 2024-09-12T02:13:21.116 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.115+0000 7fefa6d9d640 1 --2- 172.21.15.16:0/2851315306 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5625602bd540 0x5625602b47a0 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7fefa0035000 tx=0x7fefa0040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9465a3cee860775 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:21.116 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.115+0000 7fef8ffff640 1 -- 172.21.15.16:0/2851315306 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefa0004040 con 0x5625602bd540 2024-09-12T02:13:21.116 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.115+0000 7fef8ffff640 1 -- 172.21.15.16:0/2851315306 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fefa0036040 con 0x5625602bd540 2024-09-12T02:13:21.117 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.115+0000 7fef8ffff640 1 -- 172.21.15.16:0/2851315306 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefa002fd90 con 0x5625602bd540 2024-09-12T02:13:21.120 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.115+0000 7fefafa75740 1 -- 172.21.15.16:0/2851315306 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5625602bd540 msgr2=0x5625602b47a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:21.120 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.115+0000 7fefafa75740 1 --2- 172.21.15.16:0/2851315306 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5625602bd540 0x5625602b47a0 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7fefa0035000 tx=0x7fefa0040000 comp rx=0 tx=0).stop 2024-09-12T02:13:21.120 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.115+0000 7fefafa75740 1 -- 172.21.15.16:0/2851315306 shutdown_connections 2024-09-12T02:13:21.120 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.115+0000 7fefafa75740 1 --2- 172.21.15.16:0/2851315306 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5625602bd540 0x5625602b47a0 unknown :-1 s=CLOSED pgs=1630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:21.120 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.116+0000 7fefafa75740 1 --2- 172.21.15.16:0/2851315306 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5625602b84b0 0x5625602c9270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:21.120 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.116+0000 7fefafa75740 1 --2- 172.21.15.16:0/2851315306 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5625602b7b70 0x5625602c2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:21.120 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.116+0000 7fefafa75740 1 -- 172.21.15.16:0/2851315306 >> 172.21.15.16:0/2851315306 conn(0x5625602b31e0 msgr2=0x5625602b4090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:21.120 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.116+0000 7fefafa75740 1 -- 172.21.15.16:0/2851315306 shutdown_connections 2024-09-12T02:13:21.121 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.116+0000 7fefafa75740 1 -- 172.21.15.16:0/2851315306 wait complete. 2024-09-12T02:13:21.121 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.116+0000 7fefafa75740 1 -- start start 2024-09-12T02:13:21.121 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.116+0000 7fefafa75740 1 Processor -- start 2024-09-12T02:13:21.121 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.116+0000 7fefafa75740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5625602b7b70 0x5625602c6510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:21.121 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.116+0000 7fefafa75740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5625602b84b0 0x5625602c6a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:21.121 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.116+0000 7fefafa75740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5625602bd540 0x5625602c7a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:21.121 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.116+0000 7fefafa75740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fefa81a9490 con 0x5625602bd540 2024-09-12T02:13:21.121 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.116+0000 7fefafa75740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fefa81aa900 con 0x5625602b84b0 2024-09-12T02:13:21.121 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.116+0000 7fefafa75740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fefa81a9b60 con 0x5625602b7b70 2024-09-12T02:13:21.121 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.116+0000 7fefafa75740 10 client.0 _ll_register_callbacks cb 0x7fef9800c8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:13:21.121 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.116+0000 7fefa6d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5625602bd540 0x5625602c7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:21.121 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.116+0000 7fefa6d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5625602bd540 0x5625602c7a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39604/0 (socket says 172.21.15.16:39604) 2024-09-12T02:13:21.121 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.116+0000 7fefa6d9d640 1 -- 172.21.15.16:0/2487003420 learned_addr learned my addr 172.21.15.16:0/2487003420 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:13:21.122 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.116+0000 7fefa659c640 1 --2- 172.21.15.16:0/2487003420 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5625602b7b70 0x5625602c6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:21.122 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.116+0000 7fefa6d9d640 1 -- 172.21.15.16:0/2487003420 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5625602b7b70 msgr2=0x5625602c6510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:21.122 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.116+0000 7fefa6d9d640 1 --2- 172.21.15.16:0/2487003420 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5625602b7b70 0x5625602c6510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:21.122 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.116+0000 7fefa5d9b640 1 --2- 172.21.15.16:0/2487003420 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5625602b84b0 0x5625602c6a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:21.122 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.116+0000 7fefa6d9d640 1 -- 172.21.15.16:0/2487003420 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5625602b84b0 msgr2=0x5625602c6a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:21.122 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.116+0000 7fefa6d9d640 1 --2- 172.21.15.16:0/2487003420 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5625602b84b0 0x5625602c6a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:21.122 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.117+0000 7fefa6d9d640 1 -- 172.21.15.16:0/2487003420 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5625602c8070 con 0x5625602bd540 2024-09-12T02:13:21.122 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.117+0000 7fefa5d9b640 1 --2- 172.21.15.16:0/2487003420 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5625602b84b0 0x5625602c6a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:13:21.122 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.117+0000 7fefa6d9d640 1 --2- 172.21.15.16:0/2487003420 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5625602bd540 0x5625602c7a60 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7fefa0042040 tx=0x7fefa00095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86685122bbae70e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:21.122 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.117+0000 7fef8dffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:13:21.122 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.117+0000 7fef8dffb640 1 -- 172.21.15.16:0/2487003420 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefa0056070 con 0x5625602bd540 2024-09-12T02:13:21.122 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.117+0000 7fef8dffb640 1 -- 172.21.15.16:0/2487003420 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fefa0004000 con 0x5625602bd540 2024-09-12T02:13:21.122 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.117+0000 7fefafa75740 1 -- 172.21.15.16:0/2487003420 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fefa0036040 con 0x5625602bd540 2024-09-12T02:13:21.123 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.117+0000 7fef8dffb640 1 -- 172.21.15.16:0/2487003420 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fefa002fea0 con 0x5625602bd540 2024-09-12T02:13:21.123 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.117+0000 7fefafa75740 10 client.19236 fetch_fsmap learned FSMap version 25 2024-09-12T02:13:21.123 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.117+0000 7fefafa75740 1 -- 172.21.15.16:0/2487003420 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5625602c9380 con 0x5625602bd540 2024-09-12T02:13:21.123 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.117+0000 7fef8dffb640 1 -- 172.21.15.16:0/2487003420 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fefa0036040 con 0x5625602bd540 2024-09-12T02:13:21.123 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.117+0000 7fef8dffb640 1 -- 172.21.15.16:0/2487003420 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fefa0030ab0 con 0x5625602bd540 2024-09-12T02:13:21.123 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.117+0000 7fefafa75740 10 client.19236 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:13:21.123 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.117+0000 7fefafa75740 10 client.19236 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:13:21.123 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.117+0000 7fefafa75740 20 client.19236 populate_metadata read hostname 'smithi016' 2024-09-12T02:13:21.123 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.117+0000 7fefafa75740 1 --2- 172.21.15.16:0/2487003420 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x562560338bb0 0x56256033afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:21.123 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.117+0000 7fefafa75740 4 client.19236 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:13:21.123 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.117+0000 7fefafa75740 1 -- 172.21.15.16:0/2487003420 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562560222d70 con 0x562560338bb0 2024-09-12T02:13:21.123 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.118+0000 7fefa659c640 1 --2- 172.21.15.16:0/2487003420 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x562560338bb0 0x56256033afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:21.124 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.118+0000 7fefa659c640 1 --2- 172.21.15.16:0/2487003420 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x562560338bb0 0x56256033afd0 crc :-1 s=READY pgs=347 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2cbdfe7d28263853 server_cookie=da2f476cc33adccd in_seq=0 out_seq=0 2024-09-12T02:13:21.124 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.118+0000 7fef8dffb640 10 client.19236 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:13:21.132 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.131+0000 7fef8dffb640 1 -- 172.21.15.16:0/2487003420 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562560222d70 con 0x562560338bb0 2024-09-12T02:13:21.132 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.131+0000 7fef8dffb640 10 client.19236 handle_command_reply: tid=0 2024-09-12T02:13:21.132 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.131+0000 7fefafa75740 1 client.19236 shutdown 2024-09-12T02:13:21.132 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.131+0000 7fefafa75740 1 -- 172.21.15.16:0/2487003420 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5625602bd540 msgr2=0x5625602c7a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:21.132 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.131+0000 7fefafa75740 1 --2- 172.21.15.16:0/2487003420 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5625602bd540 0x5625602c7a60 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7fefa0042040 tx=0x7fefa00095c0 comp rx=0 tx=0).stop 2024-09-12T02:13:21.132 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.131+0000 7fefafa75740 1 -- 172.21.15.16:0/2487003420 shutdown_connections 2024-09-12T02:13:21.132 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.131+0000 7fefafa75740 1 --2- 172.21.15.16:0/2487003420 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x562560338bb0 0x56256033afd0 crc :-1 s=READY pgs=347 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:21.132 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.131+0000 7fefafa75740 1 --2- 172.21.15.16:0/2487003420 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5625602bd540 0x5625602c7a60 unknown :-1 s=CLOSED pgs=1631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:21.132 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.131+0000 7fefafa75740 1 --2- 172.21.15.16:0/2487003420 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5625602b84b0 0x5625602c6a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:21.132 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.131+0000 7fefafa75740 1 --2- 172.21.15.16:0/2487003420 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5625602b7b70 0x5625602c6510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:21.133 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.131+0000 7fefafa75740 1 -- 172.21.15.16:0/2487003420 >> 172.21.15.16:0/2487003420 conn(0x5625602b31e0 msgr2=0x5625602b48b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:21.133 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.131+0000 7fef8dffb640 0 client.19236 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:13:21.133 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.131+0000 7fefafa75740 1 -- 172.21.15.16:0/2487003420 shutdown_connections 2024-09-12T02:13:21.133 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.131+0000 7fefafa75740 1 -- 172.21.15.16:0/2487003420 wait complete. 2024-09-12T02:13:21.133 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.131+0000 7fefafa75740 20 client.19236 trim_cache size 0 max 16384 2024-09-12T02:13:21.133 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-09-12T02:13:21.133 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-09-12T02:13:21.133 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-09-12T02:13:21.133 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-09-12T02:13:21.134 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.132+0000 7fefa759e640 1 -- 172.21.15.16:0/3107601085 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fef900a2c00 msgr2=0x7fef900a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:21.134 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.132+0000 7fefa759e640 1 --2- 172.21.15.16:0/3107601085 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fef900a2c00 0x7fef900a50c0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fefa0006010 tx=0x7fefa0005fa0 comp rx=0 tx=0).stop 2024-09-12T02:13:21.134 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.132+0000 7fefa759e640 1 -- 172.21.15.16:0/3107601085 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fefa8100840 msgr2=0x7fefa8104760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:21.134 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.132+0000 7fefa759e640 1 --2- 172.21.15.16:0/3107601085 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fefa8100840 0x7fefa8104760 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7fef9800c970 tx=0x7fef9800ce30 comp rx=0 tx=0).stop 2024-09-12T02:13:21.135 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.132+0000 7fefa759e640 1 -- 172.21.15.16:0/3107601085 shutdown_connections 2024-09-12T02:13:21.135 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.132+0000 7fefa759e640 1 --2- 172.21.15.16:0/3107601085 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fef900a2c00 0x7fef900a50c0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:21.135 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.132+0000 7fefa759e640 1 --2- 172.21.15.16:0/3107601085 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fefa81023f0 0x7fefa8103350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:21.135 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.132+0000 7fefa759e640 1 --2- 172.21.15.16:0/3107601085 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fefa8101a30 0x7fefa8102e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:21.135 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.132+0000 7fefa759e640 1 --2- 172.21.15.16:0/3107601085 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fefa8100840 0x7fefa8104760 unknown :-1 s=CLOSED pgs=1627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:21.135 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.132+0000 7fefa759e640 1 -- 172.21.15.16:0/3107601085 >> 172.21.15.16:0/3107601085 conn(0x7fefa80fbfa0 msgr2=0x7fefa80fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:21.135 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.132+0000 7fefa759e640 1 -- 172.21.15.16:0/3107601085 shutdown_connections 2024-09-12T02:13:21.135 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:21.132+0000 7fefa759e640 1 -- 172.21.15.16:0/3107601085 wait complete. 2024-09-12T02:13:21.499 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T02:13:21.795 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:13:22.183 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:13:22 smithi016 ceph-mon[30947]: pgmap v678: 97 pgs: 97 active+clean; 2.0 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-09-12T02:13:22.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:13:22 smithi016 ceph-mon[30947]: scrub queued for path: / 2024-09-12T02:13:22.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:13:22 smithi016 ceph-mon[30947]: scrub summary: idle+waiting paths [/] 2024-09-12T02:13:22.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:13:22 smithi016 ceph-mon[30947]: scrub summary: active paths [/] 2024-09-12T02:13:22.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:13:22 smithi016 ceph-mon[30947]: scrub summary: idle+waiting paths [/] 2024-09-12T02:13:22.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:13:22 smithi016 ceph-mon[30947]: scrub summary: idle 2024-09-12T02:13:22.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.290+0000 7f6890f8b640 1 -- 172.21.15.16:0/3737684589 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f688c102d40 msgr2=0x7f688c103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:22.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.290+0000 7f6890f8b640 1 --2- 172.21.15.16:0/3737684589 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f688c102d40 0x7f688c103120 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7f687c009900 tx=0x7f687c02a480 comp rx=0 tx=0).stop 2024-09-12T02:13:22.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.290+0000 7f6890f8b640 1 -- 172.21.15.16:0/3737684589 shutdown_connections 2024-09-12T02:13:22.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.290+0000 7f6890f8b640 1 --2- 172.21.15.16:0/3737684589 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f688c10c170 0x7f688c10e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:22.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.290+0000 7f6890f8b640 1 --2- 172.21.15.16:0/3737684589 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f688c103660 0x7f688c10bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:22.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.290+0000 7f6890f8b640 1 --2- 172.21.15.16:0/3737684589 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f688c102d40 0x7f688c103120 unknown :-1 s=CLOSED pgs=1632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:22.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.290+0000 7f6890f8b640 1 -- 172.21.15.16:0/3737684589 >> 172.21.15.16:0/3737684589 conn(0x7f688c0fc7a0 msgr2=0x7f688c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:22.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.291+0000 7f6890f8b640 1 -- 172.21.15.16:0/3737684589 shutdown_connections 2024-09-12T02:13:22.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.291+0000 7f6890f8b640 1 -- 172.21.15.16:0/3737684589 wait complete. 2024-09-12T02:13:22.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.291+0000 7f6890f8b640 1 Processor -- start 2024-09-12T02:13:22.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.291+0000 7f6890f8b640 1 -- start start 2024-09-12T02:13:22.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.292+0000 7f6890f8b640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f688c102d40 0x7f688c198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:22.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.292+0000 7f6890f8b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f688c103660 0x7f688c198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:22.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.292+0000 7f6890f8b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f688c10c170 0x7f688c19cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:22.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.292+0000 7f6890f8b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f688c111f70 con 0x7f688c10c170 2024-09-12T02:13:22.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.292+0000 7f6890f8b640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f688c111df0 con 0x7f688c103660 2024-09-12T02:13:22.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.292+0000 7f6890f8b640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f688c1120f0 con 0x7f688c102d40 2024-09-12T02:13:22.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.292+0000 7f688ad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f688c10c170 0x7f688c19cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:22.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.292+0000 7f688ad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f688c10c170 0x7f688c19cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39332/0 (socket says 172.21.15.16:39332) 2024-09-12T02:13:22.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.292+0000 7f688ad9d640 1 -- 172.21.15.16:0/3915070583 learned_addr learned my addr 172.21.15.16:0/3915070583 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:13:22.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.292+0000 7f6889d9b640 1 --2- 172.21.15.16:0/3915070583 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f688c103660 0x7f688c198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:22.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.292+0000 7f688a59c640 1 --2- 172.21.15.16:0/3915070583 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f688c102d40 0x7f688c198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:22.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.292+0000 7f688ad9d640 1 -- 172.21.15.16:0/3915070583 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f688c102d40 msgr2=0x7f688c198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:22.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.292+0000 7f688ad9d640 1 --2- 172.21.15.16:0/3915070583 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f688c102d40 0x7f688c198450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:22.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.292+0000 7f688ad9d640 1 -- 172.21.15.16:0/3915070583 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f688c103660 msgr2=0x7f688c198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:22.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.292+0000 7f688ad9d640 1 --2- 172.21.15.16:0/3915070583 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f688c103660 0x7f688c198990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:22.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.292+0000 7f688ad9d640 1 -- 172.21.15.16:0/3915070583 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f688c19d400 con 0x7f688c10c170 2024-09-12T02:13:22.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.292+0000 7f688ad9d640 1 --2- 172.21.15.16:0/3915070583 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f688c10c170 0x7f688c19cd20 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7f688400eaa0 tx=0x7f688400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:22.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.292+0000 7f687b7fe640 1 -- 172.21.15.16:0/3915070583 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f688400cdb0 con 0x7f688c10c170 2024-09-12T02:13:22.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.292+0000 7f687b7fe640 1 -- 172.21.15.16:0/3915070583 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6884005020 con 0x7f688c10c170 2024-09-12T02:13:22.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.292+0000 7f6890f8b640 1 -- 172.21.15.16:0/3915070583 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f688c19d6f0 con 0x7f688c10c170 2024-09-12T02:13:22.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.292+0000 7f687b7fe640 1 -- 172.21.15.16:0/3915070583 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68840116b0 con 0x7f688c10c170 2024-09-12T02:13:22.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.292+0000 7f6890f8b640 1 -- 172.21.15.16:0/3915070583 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f688c1a4fd0 con 0x7f688c10c170 2024-09-12T02:13:22.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.295+0000 7f687b7fe640 1 -- 172.21.15.16:0/3915070583 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6884005560 con 0x7f688c10c170 2024-09-12T02:13:22.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.295+0000 7f6893364740 1 Processor -- start 2024-09-12T02:13:22.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.295+0000 7f6893364740 1 -- start start 2024-09-12T02:13:22.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.296+0000 7f6893364740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b3695134a0 0x55b369513880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:22.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.296+0000 7f6893364740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b369513dc0 0x55b36951e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:22.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.296+0000 7f6893364740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b36951e680 0x55b369520a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:22.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.296+0000 7f6893364740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55b3694792e0 con 0x55b36951e680 2024-09-12T02:13:22.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.296+0000 7f6893364740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55b369420ce0 con 0x55b369513dc0 2024-09-12T02:13:22.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.296+0000 7f6893364740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55b3694752a0 con 0x55b3695134a0 2024-09-12T02:13:22.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.296+0000 7f6889d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b36951e680 0x55b369520a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:22.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.296+0000 7f6889d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b36951e680 0x55b369520a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39336/0 (socket says 172.21.15.16:39336) 2024-09-12T02:13:22.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.296+0000 7f6889d9b640 1 -- 172.21.15.16:0/2711255347 learned_addr learned my addr 172.21.15.16:0/2711255347 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:13:22.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.296+0000 7f687b7fe640 1 --2- 172.21.15.16:0/3915070583 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f68740a2b80 0x7f68740a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:22.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.296+0000 7f688a59c640 1 --2- 172.21.15.16:0/2711255347 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b369513dc0 0x55b36951e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:22.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.296+0000 7f6889d9b640 1 --2- 172.21.15.16:0/2711255347 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b3695134a0 0x55b369513880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:22.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.296+0000 7f688ad9d640 1 --2- 172.21.15.16:0/3915070583 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f68740a2b80 0x7f68740a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:22.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.296+0000 7f687b7fe640 1 -- 172.21.15.16:0/3915070583 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f68840cf170 con 0x7f688c10c170 2024-09-12T02:13:22.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.296+0000 7f6889d9b640 1 -- 172.21.15.16:0/2711255347 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b3695134a0 msgr2=0x55b369513880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:22.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.296+0000 7f6889d9b640 1 --2- 172.21.15.16:0/2711255347 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b3695134a0 0x55b369513880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:22.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.296+0000 7f6889d9b640 1 -- 172.21.15.16:0/2711255347 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b369513dc0 msgr2=0x55b36951e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:22.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.296+0000 7f6889d9b640 1 --2- 172.21.15.16:0/2711255347 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b369513dc0 0x55b36951e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:22.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.296+0000 7f6889d9b640 1 -- 172.21.15.16:0/2711255347 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b369405440 con 0x55b36951e680 2024-09-12T02:13:22.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.296+0000 7f6889d9b640 1 --2- 172.21.15.16:0/2711255347 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b36951e680 0x55b369520a70 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7f68800127a0 tx=0x7f688000b9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9315b87421ad0ff2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:22.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.297+0000 7f688ad9d640 1 --2- 172.21.15.16:0/3915070583 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f68740a2b80 0x7f68740a5040 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f6884098450 tx=0x7f688409f040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:22.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.297+0000 7f6878ff9640 1 -- 172.21.15.16:0/2711255347 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68800120d0 con 0x55b36951e680 2024-09-12T02:13:22.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.297+0000 7f6878ff9640 1 -- 172.21.15.16:0/2711255347 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6880005500 con 0x55b36951e680 2024-09-12T02:13:22.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.297+0000 7f6878ff9640 1 -- 172.21.15.16:0/2711255347 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f688000e750 con 0x55b36951e680 2024-09-12T02:13:22.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.297+0000 7f6893364740 1 -- 172.21.15.16:0/2711255347 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b36951e680 msgr2=0x55b369520a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:22.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.297+0000 7f6893364740 1 --2- 172.21.15.16:0/2711255347 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b36951e680 0x55b369520a70 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7f68800127a0 tx=0x7f688000b9e0 comp rx=0 tx=0).stop 2024-09-12T02:13:22.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.297+0000 7f6893364740 1 -- 172.21.15.16:0/2711255347 shutdown_connections 2024-09-12T02:13:22.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.297+0000 7f6893364740 1 --2- 172.21.15.16:0/2711255347 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b36951e680 0x55b369520a70 unknown :-1 s=CLOSED pgs=1634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:22.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.297+0000 7f6893364740 1 --2- 172.21.15.16:0/2711255347 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b369513dc0 0x55b36951e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:22.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.297+0000 7f6893364740 1 --2- 172.21.15.16:0/2711255347 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b3695134a0 0x55b369513880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:22.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.297+0000 7f6893364740 1 -- 172.21.15.16:0/2711255347 >> 172.21.15.16:0/2711255347 conn(0x55b36950fa30 msgr2=0x55b36950fe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:22.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.298+0000 7f6893364740 1 -- 172.21.15.16:0/2711255347 shutdown_connections 2024-09-12T02:13:22.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.298+0000 7f6893364740 1 -- 172.21.15.16:0/2711255347 wait complete. 2024-09-12T02:13:22.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.298+0000 7f6893364740 1 -- start start 2024-09-12T02:13:22.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.298+0000 7f6893364740 1 Processor -- start 2024-09-12T02:13:22.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.298+0000 7f6893364740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b3695134a0 0x55b36951e010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:22.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.298+0000 7f6893364740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b369513dc0 0x55b36951b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:22.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.298+0000 7f6893364740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b36951e680 0x55b36951b840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:22.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.298+0000 7f6893364740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55b369521250 con 0x55b36951e680 2024-09-12T02:13:22.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.298+0000 7f6893364740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55b369434050 con 0x55b3695134a0 2024-09-12T02:13:22.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.298+0000 7f6893364740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55b3695213d0 con 0x55b369513dc0 2024-09-12T02:13:22.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.298+0000 7f6893364740 10 client.0 _ll_register_callbacks cb 0x55b369474160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:13:22.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.298+0000 7f6889d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b36951e680 0x55b36951b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:22.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.298+0000 7f6889d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b36951e680 0x55b36951b840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39352/0 (socket says 172.21.15.16:39352) 2024-09-12T02:13:22.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.298+0000 7f6889d9b640 1 -- 172.21.15.16:0/3246417544 learned_addr learned my addr 172.21.15.16:0/3246417544 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:13:22.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.299+0000 7f6889d9b640 1 --2- 172.21.15.16:0/3246417544 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b3695134a0 0x55b36951e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:22.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.299+0000 7f688a59c640 1 --2- 172.21.15.16:0/3246417544 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b369513dc0 0x55b36951b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:22.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.299+0000 7f6889d9b640 1 -- 172.21.15.16:0/3246417544 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b369513dc0 msgr2=0x55b36951b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:22.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.299+0000 7f6889d9b640 1 --2- 172.21.15.16:0/3246417544 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b369513dc0 0x55b36951b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:22.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.299+0000 7f6889d9b640 1 -- 172.21.15.16:0/3246417544 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b3695134a0 msgr2=0x55b36951e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:22.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.299+0000 7f6889d9b640 1 --2- 172.21.15.16:0/3246417544 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b3695134a0 0x55b36951e010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:22.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.299+0000 7f6889d9b640 1 -- 172.21.15.16:0/3246417544 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b3691ea3c0 con 0x55b36951e680 2024-09-12T02:13:22.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.299+0000 7f6889d9b640 1 --2- 172.21.15.16:0/3246417544 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b36951e680 0x55b36951b840 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7f68800047e0 tx=0x7f6880004a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c99c2a3130d63c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:22.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.299+0000 7f6871ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:13:22.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.299+0000 7f6871ffb640 1 -- 172.21.15.16:0/3246417544 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6880005030 con 0x55b36951e680 2024-09-12T02:13:22.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.299+0000 7f6871ffb640 1 -- 172.21.15.16:0/3246417544 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f68800051d0 con 0x55b36951e680 2024-09-12T02:13:22.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.299+0000 7f6893364740 1 -- 172.21.15.16:0/3246417544 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6880005500 con 0x55b36951e680 2024-09-12T02:13:22.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.299+0000 7f6871ffb640 1 -- 172.21.15.16:0/3246417544 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6880004020 con 0x55b36951e680 2024-09-12T02:13:22.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.299+0000 7f6893364740 10 client.19248 fetch_fsmap learned FSMap version 25 2024-09-12T02:13:22.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.299+0000 7f6893364740 1 -- 172.21.15.16:0/3246417544 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b36952cc40 con 0x55b36951e680 2024-09-12T02:13:22.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.300+0000 7f6871ffb640 1 -- 172.21.15.16:0/3246417544 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6880005500 con 0x55b36951e680 2024-09-12T02:13:22.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.300+0000 7f6871ffb640 1 -- 172.21.15.16:0/3246417544 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f68800035f0 con 0x55b36951e680 2024-09-12T02:13:22.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.300+0000 7f6893364740 10 client.19248 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:13:22.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.300+0000 7f6893364740 10 client.19248 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:13:22.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.300+0000 7f6893364740 20 client.19248 populate_metadata read hostname 'smithi016' 2024-09-12T02:13:22.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.300+0000 7f6893364740 1 --2- 172.21.15.16:0/3246417544 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55b36952df90 0x55b369530380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:22.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.300+0000 7f6893364740 4 client.19248 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:13:22.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.300+0000 7f6893364740 1 -- 172.21.15.16:0/3246417544 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b369486850 con 0x55b36952df90 2024-09-12T02:13:22.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.300+0000 7f688a59c640 1 --2- 172.21.15.16:0/3246417544 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55b36952df90 0x55b369530380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:22.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.301+0000 7f688a59c640 1 --2- 172.21.15.16:0/3246417544 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55b36952df90 0x55b369530380 crc :-1 s=READY pgs=348 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92c631ca58c76622 server_cookie=6d6eb36f47d36394 in_seq=0 out_seq=0 2024-09-12T02:13:22.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.301+0000 7f6871ffb640 10 client.19248 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:13:22.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.301+0000 7f6871ffb640 1 -- 172.21.15.16:0/3246417544 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b369486850 con 0x55b36952df90 2024-09-12T02:13:22.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.301+0000 7f6871ffb640 10 client.19248 handle_command_reply: tid=0 2024-09-12T02:13:22.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.301+0000 7f6893364740 1 client.19248 shutdown 2024-09-12T02:13:22.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.302+0000 7f6893364740 1 -- 172.21.15.16:0/3246417544 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b36951e680 msgr2=0x55b36951b840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:22.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.302+0000 7f6893364740 1 --2- 172.21.15.16:0/3246417544 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b36951e680 0x55b36951b840 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7f68800047e0 tx=0x7f6880004a80 comp rx=0 tx=0).stop 2024-09-12T02:13:22.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.302+0000 7f6893364740 1 -- 172.21.15.16:0/3246417544 shutdown_connections 2024-09-12T02:13:22.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.302+0000 7f6893364740 1 --2- 172.21.15.16:0/3246417544 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55b36952df90 0x55b369530380 crc :-1 s=READY pgs=348 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:22.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.302+0000 7f6893364740 1 --2- 172.21.15.16:0/3246417544 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b36951e680 0x55b36951b840 unknown :-1 s=CLOSED pgs=1635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:22.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.302+0000 7f6893364740 1 --2- 172.21.15.16:0/3246417544 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b369513dc0 0x55b36951b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:22.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.302+0000 7f6893364740 1 --2- 172.21.15.16:0/3246417544 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b3695134a0 0x55b36951e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:22.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.302+0000 7f6893364740 1 -- 172.21.15.16:0/3246417544 >> 172.21.15.16:0/3246417544 conn(0x55b36950fa30 msgr2=0x55b369515710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:22.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.302+0000 7f6871ffb640 0 client.19248 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:13:22.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.302+0000 7f6893364740 1 -- 172.21.15.16:0/3246417544 shutdown_connections 2024-09-12T02:13:22.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.302+0000 7f6893364740 1 -- 172.21.15.16:0/3246417544 wait complete. 2024-09-12T02:13:22.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.302+0000 7f6893364740 20 client.19248 trim_cache size 0 max 16384 2024-09-12T02:13:22.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.319+0000 7f6893364740 1 Processor -- start 2024-09-12T02:13:22.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.319+0000 7f6893364740 1 -- start start 2024-09-12T02:13:22.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.319+0000 7f6893364740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b3695134a0 0x55b36952a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:22.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.319+0000 7f6893364740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b369513dc0 0x55b369535aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:22.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.319+0000 7f6893364740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b36951e680 0x55b369535fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:22.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.319+0000 7f6893364740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f688c1afe10 con 0x55b36951e680 2024-09-12T02:13:22.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.319+0000 7f6893364740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f688c1ad6e0 con 0x55b3695134a0 2024-09-12T02:13:22.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.319+0000 7f6893364740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f688c1ab590 con 0x55b369513dc0 2024-09-12T02:13:22.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.319+0000 7f6889d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b36951e680 0x55b369535fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:22.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.319+0000 7f6889d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b36951e680 0x55b369535fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39362/0 (socket says 172.21.15.16:39362) 2024-09-12T02:13:22.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.319+0000 7f6889d9b640 1 -- 172.21.15.16:0/114026441 learned_addr learned my addr 172.21.15.16:0/114026441 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:13:22.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.319+0000 7f6889d9b640 1 --2- 172.21.15.16:0/114026441 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b3695134a0 0x55b36952a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:22.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.319+0000 7f688a59c640 1 --2- 172.21.15.16:0/114026441 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b369513dc0 0x55b369535aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:22.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.319+0000 7f6889d9b640 1 -- 172.21.15.16:0/114026441 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b369513dc0 msgr2=0x55b369535aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:22.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.319+0000 7f6889d9b640 1 --2- 172.21.15.16:0/114026441 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b369513dc0 0x55b369535aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:22.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.319+0000 7f6889d9b640 1 -- 172.21.15.16:0/114026441 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b3695134a0 msgr2=0x55b36952a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:22.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.319+0000 7f6889d9b640 1 --2- 172.21.15.16:0/114026441 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b3695134a0 0x55b36952a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:22.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.319+0000 7f6889d9b640 1 -- 172.21.15.16:0/114026441 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b369513bf0 con 0x55b36951e680 2024-09-12T02:13:22.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.320+0000 7f6889d9b640 1 --2- 172.21.15.16:0/114026441 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b36951e680 0x55b369535fe0 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7f6880004a50 tx=0x7f688000b9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98973b497f11b1c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:22.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.320+0000 7f6873fff640 1 -- 172.21.15.16:0/114026441 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f688000b360 con 0x55b36951e680 2024-09-12T02:13:22.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.320+0000 7f6873fff640 1 -- 172.21.15.16:0/114026441 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6880003c30 con 0x55b36951e680 2024-09-12T02:13:22.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.320+0000 7f6873fff640 1 -- 172.21.15.16:0/114026441 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68800041c0 con 0x55b36951e680 2024-09-12T02:13:22.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.320+0000 7f6893364740 1 -- 172.21.15.16:0/114026441 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b36951e680 msgr2=0x55b369535fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:22.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.320+0000 7f6893364740 1 --2- 172.21.15.16:0/114026441 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b36951e680 0x55b369535fe0 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7f6880004a50 tx=0x7f688000b9e0 comp rx=0 tx=0).stop 2024-09-12T02:13:22.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.320+0000 7f6893364740 1 -- 172.21.15.16:0/114026441 shutdown_connections 2024-09-12T02:13:22.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.320+0000 7f6893364740 1 --2- 172.21.15.16:0/114026441 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b36951e680 0x55b369535fe0 unknown :-1 s=CLOSED pgs=1636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:22.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.320+0000 7f6893364740 1 --2- 172.21.15.16:0/114026441 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b369513dc0 0x55b369535aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:22.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.320+0000 7f6893364740 1 --2- 172.21.15.16:0/114026441 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b3695134a0 0x55b36952a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:22.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.320+0000 7f6893364740 1 -- 172.21.15.16:0/114026441 >> 172.21.15.16:0/114026441 conn(0x55b36950fa30 msgr2=0x55b3695168b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:22.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.320+0000 7f6893364740 1 -- 172.21.15.16:0/114026441 shutdown_connections 2024-09-12T02:13:22.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.320+0000 7f6893364740 1 -- 172.21.15.16:0/114026441 wait complete. 2024-09-12T02:13:22.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.321+0000 7f6893364740 1 -- start start 2024-09-12T02:13:22.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.321+0000 7f6893364740 1 Processor -- start 2024-09-12T02:13:22.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.321+0000 7f6893364740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b3695134a0 0x55b369521db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:22.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.321+0000 7f6893364740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b369513dc0 0x55b3695222f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:22.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.321+0000 7f6893364740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b36951e680 0x55b369594e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:22.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.321+0000 7f6893364740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f688c1abbe0 con 0x55b36951e680 2024-09-12T02:13:22.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.321+0000 7f6893364740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f688c1ad010 con 0x55b3695134a0 2024-09-12T02:13:22.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.321+0000 7f6893364740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f688c1ac270 con 0x55b369513dc0 2024-09-12T02:13:22.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.321+0000 7f6893364740 10 client.0 _ll_register_callbacks cb 0x7f687c005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:13:22.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.321+0000 7f6889d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b36951e680 0x55b369594e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:22.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.321+0000 7f6889d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b36951e680 0x55b369594e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39376/0 (socket says 172.21.15.16:39376) 2024-09-12T02:13:22.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.321+0000 7f6889d9b640 1 -- 172.21.15.16:0/336996054 learned_addr learned my addr 172.21.15.16:0/336996054 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:13:22.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.321+0000 7f6889d9b640 1 --2- 172.21.15.16:0/336996054 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b3695134a0 0x55b369521db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:22.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.322+0000 7f688a59c640 1 --2- 172.21.15.16:0/336996054 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b369513dc0 0x55b3695222f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:22.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.322+0000 7f6889d9b640 1 -- 172.21.15.16:0/336996054 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b369513dc0 msgr2=0x55b3695222f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:22.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.322+0000 7f6889d9b640 1 --2- 172.21.15.16:0/336996054 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b369513dc0 0x55b3695222f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:22.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.322+0000 7f6889d9b640 1 -- 172.21.15.16:0/336996054 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b3695134a0 msgr2=0x55b369521db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:22.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.322+0000 7f6889d9b640 1 --2- 172.21.15.16:0/336996054 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b3695134a0 0x55b369521db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:22.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.322+0000 7f6889d9b640 1 -- 172.21.15.16:0/336996054 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b36951e380 con 0x55b36951e680 2024-09-12T02:13:22.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.322+0000 7f688a59c640 1 --2- 172.21.15.16:0/336996054 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b369513dc0 0x55b3695222f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:13:22.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.322+0000 7f6889d9b640 1 --2- 172.21.15.16:0/336996054 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b36951e680 0x55b369594e40 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7f688000bbf0 tx=0x7f6880003870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a480fb8e8e05589f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:22.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.322+0000 7f6871ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:13:22.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.322+0000 7f6871ffb640 1 -- 172.21.15.16:0/336996054 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68800043d0 con 0x55b36951e680 2024-09-12T02:13:22.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.322+0000 7f6871ffb640 1 -- 172.21.15.16:0/336996054 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f688000e720 con 0x55b36951e680 2024-09-12T02:13:22.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.322+0000 7f6893364740 1 -- 172.21.15.16:0/336996054 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6880003c30 con 0x55b36951e680 2024-09-12T02:13:22.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.322+0000 7f6871ffb640 1 -- 172.21.15.16:0/336996054 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f688000e920 con 0x55b36951e680 2024-09-12T02:13:22.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.322+0000 7f6893364740 10 client.19254 fetch_fsmap learned FSMap version 25 2024-09-12T02:13:22.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.322+0000 7f6893364740 1 -- 172.21.15.16:0/336996054 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b36959c640 con 0x55b36951e680 2024-09-12T02:13:22.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.322+0000 7f6871ffb640 1 -- 172.21.15.16:0/336996054 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6880003c30 con 0x55b36951e680 2024-09-12T02:13:22.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.323+0000 7f6871ffb640 1 -- 172.21.15.16:0/336996054 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f688003f040 con 0x55b36951e680 2024-09-12T02:13:22.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.323+0000 7f6893364740 10 client.19254 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:13:22.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.323+0000 7f6893364740 10 client.19254 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:13:22.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.323+0000 7f6893364740 20 client.19254 populate_metadata read hostname 'smithi016' 2024-09-12T02:13:22.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.323+0000 7f6893364740 1 --2- 172.21.15.16:0/336996054 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55b36959cbf0 0x55b36959f010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:22.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.323+0000 7f6893364740 4 client.19254 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T02:13:22.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.323+0000 7f6893364740 1 -- 172.21.15.16:0/336996054 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b369480d60 con 0x55b36959cbf0 2024-09-12T02:13:22.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.323+0000 7f688a59c640 1 --2- 172.21.15.16:0/336996054 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55b36959cbf0 0x55b36959f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:22.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.323+0000 7f688a59c640 1 --2- 172.21.15.16:0/336996054 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55b36959cbf0 0x55b36959f010 crc :-1 s=READY pgs=349 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc9fe2809722ef16 server_cookie=c38ebd5a922ba67 in_seq=0 out_seq=0 2024-09-12T02:13:22.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.323+0000 7f6871ffb640 10 client.19254 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:13:22.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.324+0000 7f6871ffb640 1 -- 172.21.15.16:0/336996054 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55b369480d60 con 0x55b36959cbf0 2024-09-12T02:13:22.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.324+0000 7f6871ffb640 10 client.19254 handle_command_reply: tid=0 2024-09-12T02:13:22.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.324+0000 7f6893364740 1 client.19254 shutdown 2024-09-12T02:13:22.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.324+0000 7f6893364740 1 -- 172.21.15.16:0/336996054 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b36951e680 msgr2=0x55b369594e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:22.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.324+0000 7f6893364740 1 --2- 172.21.15.16:0/336996054 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b36951e680 0x55b369594e40 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7f688000bbf0 tx=0x7f6880003870 comp rx=0 tx=0).stop 2024-09-12T02:13:22.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.324+0000 7f6893364740 1 -- 172.21.15.16:0/336996054 shutdown_connections 2024-09-12T02:13:22.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.324+0000 7f6893364740 1 --2- 172.21.15.16:0/336996054 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55b36959cbf0 0x55b36959f010 crc :-1 s=READY pgs=349 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:22.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.324+0000 7f6893364740 1 --2- 172.21.15.16:0/336996054 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b36951e680 0x55b369594e40 unknown :-1 s=CLOSED pgs=1637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:22.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.324+0000 7f6893364740 1 --2- 172.21.15.16:0/336996054 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b369513dc0 0x55b3695222f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:22.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.324+0000 7f6871ffb640 0 client.19254 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:13:22.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.324+0000 7f6893364740 1 --2- 172.21.15.16:0/336996054 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b3695134a0 0x55b369521db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:22.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.324+0000 7f6893364740 1 -- 172.21.15.16:0/336996054 >> 172.21.15.16:0/336996054 conn(0x55b36950fa30 msgr2=0x55b369519a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:22.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.324+0000 7f6893364740 1 -- 172.21.15.16:0/336996054 shutdown_connections 2024-09-12T02:13:22.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.324+0000 7f6893364740 1 -- 172.21.15.16:0/336996054 wait complete. 2024-09-12T02:13:22.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.324+0000 7f6893364740 20 client.19254 trim_cache size 0 max 16384 2024-09-12T02:13:22.330 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T02:13:22.330 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-09-12T02:13:22.330 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {} 2024-09-12T02:13:22.330 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T02:13:22.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.324+0000 7f6890f8b640 1 -- 172.21.15.16:0/3915070583 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f68740a2b80 msgr2=0x7f68740a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:22.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.324+0000 7f6890f8b640 1 --2- 172.21.15.16:0/3915070583 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f68740a2b80 0x7f68740a5040 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f6884098450 tx=0x7f688409f040 comp rx=0 tx=0).stop 2024-09-12T02:13:22.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.324+0000 7f6890f8b640 1 -- 172.21.15.16:0/3915070583 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f688c10c170 msgr2=0x7f688c19cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:22.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.324+0000 7f6890f8b640 1 --2- 172.21.15.16:0/3915070583 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f688c10c170 0x7f688c19cd20 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7f688400eaa0 tx=0x7f688400ef60 comp rx=0 tx=0).stop 2024-09-12T02:13:22.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.325+0000 7f6890f8b640 1 -- 172.21.15.16:0/3915070583 shutdown_connections 2024-09-12T02:13:22.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.325+0000 7f6890f8b640 1 --2- 172.21.15.16:0/3915070583 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f68740a2b80 0x7f68740a5040 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:22.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.325+0000 7f6890f8b640 1 --2- 172.21.15.16:0/3915070583 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f688c10c170 0x7f688c19cd20 unknown :-1 s=CLOSED pgs=1633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:22.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.325+0000 7f6890f8b640 1 --2- 172.21.15.16:0/3915070583 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f688c103660 0x7f688c198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:22.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.325+0000 7f6890f8b640 1 --2- 172.21.15.16:0/3915070583 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f688c102d40 0x7f688c198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:22.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.325+0000 7f6890f8b640 1 -- 172.21.15.16:0/3915070583 >> 172.21.15.16:0/3915070583 conn(0x7f688c0fc7a0 msgr2=0x7f688c0fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:22.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.325+0000 7f6890f8b640 1 -- 172.21.15.16:0/3915070583 shutdown_connections 2024-09-12T02:13:22.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:22.325+0000 7f6890f8b640 1 -- 172.21.15.16:0/3915070583 wait complete. 2024-09-12T02:13:22.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:13:22 smithi184 ceph-mon[35593]: pgmap v678: 97 pgs: 97 active+clean; 2.0 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-09-12T02:13:22.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:13:22 smithi184 ceph-mon[35593]: scrub queued for path: / 2024-09-12T02:13:22.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:13:22 smithi184 ceph-mon[35593]: scrub summary: idle+waiting paths [/] 2024-09-12T02:13:22.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:13:22 smithi184 ceph-mon[35593]: scrub summary: active paths [/] 2024-09-12T02:13:22.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:13:22 smithi184 ceph-mon[35593]: scrub summary: idle+waiting paths [/] 2024-09-12T02:13:22.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:13:22 smithi184 ceph-mon[35593]: scrub summary: idle 2024-09-12T02:13:22.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:13:22 smithi033 ceph-mon[35332]: pgmap v678: 97 pgs: 97 active+clean; 2.0 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-09-12T02:13:22.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:13:22 smithi033 ceph-mon[35332]: scrub queued for path: / 2024-09-12T02:13:22.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:13:22 smithi033 ceph-mon[35332]: scrub summary: idle+waiting paths [/] 2024-09-12T02:13:22.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:13:22 smithi033 ceph-mon[35332]: scrub summary: active paths [/] 2024-09-12T02:13:22.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:13:22 smithi033 ceph-mon[35332]: scrub summary: idle+waiting paths [/] 2024-09-12T02:13:22.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:13:22 smithi033 ceph-mon[35332]: scrub summary: idle 2024-09-12T02:13:22.982 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-12T02:13:22.982 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:13:23.241 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:13:23.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:23.663+0000 7f4640a4f640 1 -- 172.21.15.16:0/3718308216 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f463c102ef0 msgr2=0x7f463c10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:23.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:23.663+0000 7f4640a4f640 1 --2- 172.21.15.16:0/3718308216 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f463c102ef0 0x7f463c10da40 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7f4634009900 tx=0x7f463402a480 comp rx=0 tx=0).stop 2024-09-12T02:13:23.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:23.663+0000 7f4640a4f640 1 -- 172.21.15.16:0/3718308216 shutdown_connections 2024-09-12T02:13:23.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:23.663+0000 7f4640a4f640 1 --2- 172.21.15.16:0/3718308216 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f463c102ef0 0x7f463c10da40 unknown :-1 s=CLOSED pgs=1638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:23.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:23.663+0000 7f4640a4f640 1 --2- 172.21.15.16:0/3718308216 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f463c102570 0x7f463c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:23.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:23.663+0000 7f4640a4f640 1 --2- 172.21.15.16:0/3718308216 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f463c100e20 0x7f463c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:23.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:23.663+0000 7f4640a4f640 1 -- 172.21.15.16:0/3718308216 >> 172.21.15.16:0/3718308216 conn(0x7f463c0fc7d0 msgr2=0x7f463c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:23.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:23.663+0000 7f4640a4f640 1 -- 172.21.15.16:0/3718308216 shutdown_connections 2024-09-12T02:13:23.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:23.663+0000 7f4640a4f640 1 -- 172.21.15.16:0/3718308216 wait complete. 2024-09-12T02:13:23.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:23.664+0000 7f4640a4f640 1 Processor -- start 2024-09-12T02:13:23.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:23.664+0000 7f4640a4f640 1 -- start start 2024-09-12T02:13:23.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:23.664+0000 7f4640a4f640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f463c100e20 0x7f463c198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:23.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:23.664+0000 7f4640a4f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f463c102570 0x7f463c198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:23.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:23.664+0000 7f4640a4f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f463c102ef0 0x7f463c19ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:23.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:23.664+0000 7f4640a4f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f463c1120a0 con 0x7f463c102570 2024-09-12T02:13:23.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:23.664+0000 7f4640a4f640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f463c111f20 con 0x7f463c102ef0 2024-09-12T02:13:23.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:23.664+0000 7f4640a4f640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f463c112220 con 0x7f463c100e20 2024-09-12T02:13:23.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:23.665+0000 7f4639d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f463c102570 0x7f463c198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:23.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:23.665+0000 7f463ad9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f463c102ef0 0x7f463c19ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:23.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:23.665+0000 7f4639d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f463c102570 0x7f463c198960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39402/0 (socket says 172.21.15.16:39402) 2024-09-12T02:13:23.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:23.665+0000 7f4639d9b640 1 -- 172.21.15.16:0/1033871747 learned_addr learned my addr 172.21.15.16:0/1033871747 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:13:23.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:23.665+0000 7f463a59c640 1 --2- 172.21.15.16:0/1033871747 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f463c100e20 0x7f463c198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:23.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:23.665+0000 7f4639d9b640 1 -- 172.21.15.16:0/1033871747 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f463c100e20 msgr2=0x7f463c198420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:23.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:23.665+0000 7f4639d9b640 1 --2- 172.21.15.16:0/1033871747 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f463c100e20 0x7f463c198420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:23.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:23.665+0000 7f4639d9b640 1 -- 172.21.15.16:0/1033871747 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f463c102ef0 msgr2=0x7f463c19ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:23.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:23.665+0000 7f4639d9b640 1 --2- 172.21.15.16:0/1033871747 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f463c102ef0 0x7f463c19ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:23.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:23.665+0000 7f4639d9b640 1 -- 172.21.15.16:0/1033871747 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f463c19d3d0 con 0x7f463c102570 2024-09-12T02:13:23.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:23.665+0000 7f4639d9b640 1 --2- 172.21.15.16:0/1033871747 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f463c102570 0x7f463c198960 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f463000c920 tx=0x7f463000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:23.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:23.665+0000 7f462b7fe640 1 -- 172.21.15.16:0/1033871747 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4630007c70 con 0x7f463c102570 2024-09-12T02:13:23.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:23.665+0000 7f462b7fe640 1 -- 172.21.15.16:0/1033871747 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4630007e10 con 0x7f463c102570 2024-09-12T02:13:23.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:23.665+0000 7f462b7fe640 1 -- 172.21.15.16:0/1033871747 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4630018890 con 0x7f463c102570 2024-09-12T02:13:23.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:23.665+0000 7f4640a4f640 1 -- 172.21.15.16:0/1033871747 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f463c19d6c0 con 0x7f463c102570 2024-09-12T02:13:23.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:23.665+0000 7f4640a4f640 1 -- 172.21.15.16:0/1033871747 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f463c1a4fa0 con 0x7f463c102570 2024-09-12T02:13:23.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:23.667+0000 7f4640a4f640 1 -- 172.21.15.16:0/1033871747 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f463c101200 con 0x7f463c102570 2024-09-12T02:13:23.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:23.668+0000 7f462b7fe640 1 -- 172.21.15.16:0/1033871747 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f4630018a50 con 0x7f463c102570 2024-09-12T02:13:23.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:23.668+0000 7f462b7fe640 1 --2- 172.21.15.16:0/1033871747 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f46240a2c00 0x7f46240a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:23.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:23.669+0000 7f462b7fe640 1 -- 172.21.15.16:0/1033871747 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f46300cf200 con 0x7f463c102570 2024-09-12T02:13:23.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:23.669+0000 7f463a59c640 1 --2- 172.21.15.16:0/1033871747 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f46240a2c00 0x7f46240a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:23.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:23.669+0000 7f463a59c640 1 --2- 172.21.15.16:0/1033871747 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f46240a2c00 0x7f46240a50c0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f462c005fd0 tx=0x7f462c009760 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:23.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:23.672+0000 7f462b7fe640 1 -- 172.21.15.16:0/1033871747 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4630014030 con 0x7f463c102570 2024-09-12T02:13:23.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:23.882+0000 7f4640a4f640 1 -- 172.21.15.16:0/1033871747 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f463c199780 con 0x7f463c102570 2024-09-12T02:13:23.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:23.883+0000 7f462b7fe640 1 -- 172.21.15.16:0/1033871747 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11911 (secure 0 0 0) 0x7f463009c310 con 0x7f463c102570 2024-09-12T02:13:23.883 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T02:13:23.883 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":25,"btime":"2024-09-12T02:01:37:192021+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3279123279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3279123279},{"type":"v1","addr":"172.21.15.16:6835","nonce":3279123279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4119452382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4119452382},{"type":"v1","addr":"172.21.15.16:6837","nonce":4119452382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/3263823165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3263823165},{"type":"v1","addr":"172.21.15.16:6839","nonce":3263823165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/1510290732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":1510290732},{"type":"v1","addr":"172.21.15.16:6841","nonce":1510290732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/279674603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":279674603},{"type":"v1","addr":"172.21.15.33:6833","nonce":279674603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3555728028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3555728028},{"type":"v1","addr":"172.21.15.33:6835","nonce":3555728028}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T01:59:56.117608+0000","modified":"2024-09-12T02:01:36.294760+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24482,"mds_1":24481,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6833/1563374106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1563374106},{"type":"v1","addr":"172.21.15.184:6833","nonce":1563374106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6837/1892833868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1892833868},{"type":"v1","addr":"172.21.15.33:6837","nonce":1892833868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/1457679686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1457679686},{"type":"v1","addr":"172.21.15.184:6835","nonce":1457679686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.184:6837/1793393087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":1793393087},{"type":"v1","addr":"172.21.15.184:6837","nonce":1793393087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.184:6839/4203347941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":4203347941},{"type":"v1","addr":"172.21.15.184:6839","nonce":4203347941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.33:6839/4136670204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":4136670204},{"type":"v1","addr":"172.21.15.33:6839","nonce":4136670204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24481,24482,24469]},"id":1}]} 2024-09-12T02:13:23.884 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 25 2024-09-12T02:13:23.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:23.885+0000 7f4640a4f640 1 -- 172.21.15.16:0/1033871747 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f46240a2c00 msgr2=0x7f46240a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:23.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:23.885+0000 7f4640a4f640 1 --2- 172.21.15.16:0/1033871747 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f46240a2c00 0x7f46240a50c0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f462c005fd0 tx=0x7f462c009760 comp rx=0 tx=0).stop 2024-09-12T02:13:23.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:23.885+0000 7f4640a4f640 1 -- 172.21.15.16:0/1033871747 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f463c102570 msgr2=0x7f463c198960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:23.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:23.885+0000 7f4640a4f640 1 --2- 172.21.15.16:0/1033871747 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f463c102570 0x7f463c198960 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f463000c920 tx=0x7f463000cde0 comp rx=0 tx=0).stop 2024-09-12T02:13:23.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:23.885+0000 7f4640a4f640 1 -- 172.21.15.16:0/1033871747 shutdown_connections 2024-09-12T02:13:23.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:23.885+0000 7f4640a4f640 1 --2- 172.21.15.16:0/1033871747 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f46240a2c00 0x7f46240a50c0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:23.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:23.885+0000 7f4640a4f640 1 --2- 172.21.15.16:0/1033871747 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f463c102ef0 0x7f463c19ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:23.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:23.885+0000 7f4640a4f640 1 --2- 172.21.15.16:0/1033871747 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f463c102570 0x7f463c198960 unknown :-1 s=CLOSED pgs=1639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:23.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:23.885+0000 7f4640a4f640 1 --2- 172.21.15.16:0/1033871747 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f463c100e20 0x7f463c198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:23.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:23.885+0000 7f4640a4f640 1 -- 172.21.15.16:0/1033871747 >> 172.21.15.16:0/1033871747 conn(0x7f463c0fc7d0 msgr2=0x7f463c1067c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:23.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:23.885+0000 7f4640a4f640 1 -- 172.21.15.16:0/1033871747 shutdown_connections 2024-09-12T02:13:23.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:23.885+0000 7f4640a4f640 1 -- 172.21.15.16:0/1033871747 wait complete. 2024-09-12T02:13:24.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:13:24 smithi184 ceph-mon[35593]: pgmap v679: 97 pgs: 97 active+clean; 2.0 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 15 op/s 2024-09-12T02:13:24.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:13:24 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/1033871747' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:13:24.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:13:24 smithi033 ceph-mon[35332]: pgmap v679: 97 pgs: 97 active+clean; 2.0 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 15 op/s 2024-09-12T02:13:24.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:13:24 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/1033871747' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:13:24.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:13:24 smithi016 ceph-mon[30947]: pgmap v679: 97 pgs: 97 active+clean; 2.0 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 15 op/s 2024-09-12T02:13:24.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:13:24 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/1033871747' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:13:24.581 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-12T02:13:24.840 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:13:25.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.254+0000 7f7b9ed2b640 1 -- 172.21.15.16:0/2328489972 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b98101420 msgr2=0x7f7b9810d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:25.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.254+0000 7f7b9ed2b640 1 --2- 172.21.15.16:0/2328489972 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b98101420 0x7f7b9810d950 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7f7b90009cd0 tx=0x7f7b9002a8e0 comp rx=0 tx=0).stop 2024-09-12T02:13:25.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.254+0000 7f7b9ed2b640 1 -- 172.21.15.16:0/2328489972 shutdown_connections 2024-09-12T02:13:25.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.254+0000 7f7b9ed2b640 1 --2- 172.21.15.16:0/2328489972 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b98101420 0x7f7b9810d950 unknown :-1 s=CLOSED pgs=1640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:25.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.254+0000 7f7b9ed2b640 1 --2- 172.21.15.16:0/2328489972 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7b98100a80 0x7f7b98100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:25.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.254+0000 7f7b9ed2b640 1 --2- 172.21.15.16:0/2328489972 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b98106a80 0x7f7b98106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:25.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.254+0000 7f7b9ed2b640 1 -- 172.21.15.16:0/2328489972 >> 172.21.15.16:0/2328489972 conn(0x7f7b980fc7a0 msgr2=0x7f7b980febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:25.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.254+0000 7f7b9ed2b640 1 -- 172.21.15.16:0/2328489972 shutdown_connections 2024-09-12T02:13:25.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.254+0000 7f7b9ed2b640 1 -- 172.21.15.16:0/2328489972 wait complete. 2024-09-12T02:13:25.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.255+0000 7f7b9ed2b640 1 Processor -- start 2024-09-12T02:13:25.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.255+0000 7f7b9ed2b640 1 -- start start 2024-09-12T02:13:25.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.255+0000 7f7b9ed2b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b98100a80 0x7f7b9810e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:25.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.255+0000 7f7b9ed2b640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7b98101420 0x7f7b9810e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:25.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.255+0000 7f7b9ed2b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b98106a80 0x7f7b9810ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:25.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.255+0000 7f7b9ed2b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7b98111fa0 con 0x7f7b98106a80 2024-09-12T02:13:25.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.255+0000 7f7b9ed2b640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7b98111e20 con 0x7f7b98100a80 2024-09-12T02:13:25.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.255+0000 7f7b9ed2b640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7b98112120 con 0x7f7b98101420 2024-09-12T02:13:25.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.255+0000 7f7b97fff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7b98101420 0x7f7b9810e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:25.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.255+0000 7f7b9d2c8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b98106a80 0x7f7b9810ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:25.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.255+0000 7f7b97fff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7b98101420 0x7f7b9810e920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:53246/0 (socket says 172.21.15.16:53246) 2024-09-12T02:13:25.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.255+0000 7f7b97fff640 1 -- 172.21.15.16:0/497629190 learned_addr learned my addr 172.21.15.16:0/497629190 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:13:25.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.255+0000 7f7b9d2c8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b98106a80 0x7f7b9810ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39414/0 (socket says 172.21.15.16:39414) 2024-09-12T02:13:25.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.255+0000 7f7b9cac7640 1 --2- 172.21.15.16:0/497629190 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b98100a80 0x7f7b9810e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:25.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.255+0000 7f7b97fff640 1 -- 172.21.15.16:0/497629190 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b98100a80 msgr2=0x7f7b9810e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:25.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.255+0000 7f7b97fff640 1 --2- 172.21.15.16:0/497629190 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b98100a80 0x7f7b9810e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:25.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.255+0000 7f7b97fff640 1 -- 172.21.15.16:0/497629190 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b98106a80 msgr2=0x7f7b9810ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:25.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.255+0000 7f7b97fff640 1 --2- 172.21.15.16:0/497629190 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b98106a80 0x7f7b9810ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:25.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.256+0000 7f7b97fff640 1 -- 172.21.15.16:0/497629190 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b981a6900 con 0x7f7b98101420 2024-09-12T02:13:25.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.256+0000 7f7b9cac7640 1 --2- 172.21.15.16:0/497629190 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b98100a80 0x7f7b9810e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:13:25.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.256+0000 7f7b9d2c8640 1 --2- 172.21.15.16:0/497629190 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b98106a80 0x7f7b9810ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:13:25.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.256+0000 7f7b97fff640 1 --2- 172.21.15.16:0/497629190 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7b98101420 0x7f7b9810e920 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f7b8400e9f0 tx=0x7f7b8400edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:25.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.256+0000 7f7b95ffb640 1 -- 172.21.15.16:0/497629190 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b8400cd80 con 0x7f7b98101420 2024-09-12T02:13:25.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.256+0000 7f7b95ffb640 1 -- 172.21.15.16:0/497629190 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b84005500 con 0x7f7b98101420 2024-09-12T02:13:25.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.256+0000 7f7b9ed2b640 1 -- 172.21.15.16:0/497629190 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b981a6bf0 con 0x7f7b98101420 2024-09-12T02:13:25.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.256+0000 7f7b95ffb640 1 -- 172.21.15.16:0/497629190 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b84010690 con 0x7f7b98101420 2024-09-12T02:13:25.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.256+0000 7f7b9ed2b640 1 -- 172.21.15.16:0/497629190 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b981a70e0 con 0x7f7b98101420 2024-09-12T02:13:25.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.258+0000 7f7b95ffb640 1 -- 172.21.15.16:0/497629190 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f7b840046c0 con 0x7f7b98101420 2024-09-12T02:13:25.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.259+0000 7f7ba1104740 1 Processor -- start 2024-09-12T02:13:25.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.259+0000 7f7ba1104740 1 -- start start 2024-09-12T02:13:25.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.259+0000 7f7b95ffb640 1 --2- 172.21.15.16:0/497629190 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f7b880a2b80 0x7f7b880a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:25.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.259+0000 7f7ba1104740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b18a18db40 0x55b18a18c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:25.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.259+0000 7f7ba1104740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b18a18c5d0 0x55b18a18ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:25.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.259+0000 7f7ba1104740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b18a197370 0x55b18a199760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:25.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.259+0000 7f7ba1104740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55b18a0f22f0 con 0x55b18a197370 2024-09-12T02:13:25.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.259+0000 7f7ba1104740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55b18a099cf0 con 0x55b18a18c5d0 2024-09-12T02:13:25.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.259+0000 7f7ba1104740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55b18a0ee2b0 con 0x55b18a18db40 2024-09-12T02:13:25.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.259+0000 7f7b9cac7640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b18a18db40 0x55b18a18c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:25.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.259+0000 7f7b9d2c8640 1 --2- 172.21.15.16:0/497629190 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f7b880a2b80 0x7f7b880a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:25.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.259+0000 7f7b9cac7640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b18a18db40 0x55b18a18c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:53252/0 (socket says 172.21.15.16:53252) 2024-09-12T02:13:25.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.259+0000 7f7b9cac7640 1 -- 172.21.15.16:0/79006599 learned_addr learned my addr 172.21.15.16:0/79006599 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:13:25.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.259+0000 7f7b9d2c8640 1 --2- 172.21.15.16:0/79006599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b18a197370 0x55b18a199760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:25.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.259+0000 7f7b95ffb640 1 -- 172.21.15.16:0/497629190 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f7b840cf460 con 0x7f7b98101420 2024-09-12T02:13:25.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.259+0000 7f7b97fff640 1 --2- 172.21.15.16:0/79006599 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b18a18c5d0 0x55b18a18ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:25.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.259+0000 7f7b9cac7640 1 -- 172.21.15.16:0/79006599 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b18a18c5d0 msgr2=0x55b18a18ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:25.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.259+0000 7f7b9cac7640 1 --2- 172.21.15.16:0/79006599 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b18a18c5d0 0x55b18a18ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:25.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.260+0000 7f7b9cac7640 1 -- 172.21.15.16:0/79006599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b18a197370 msgr2=0x55b18a199760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:25.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.260+0000 7f7b9cac7640 1 --2- 172.21.15.16:0/79006599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b18a197370 0x55b18a199760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:25.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.260+0000 7f7b9cac7640 1 -- 172.21.15.16:0/79006599 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b18a07e3a0 con 0x55b18a18db40 2024-09-12T02:13:25.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.260+0000 7f7b9d2c8640 1 --2- 172.21.15.16:0/497629190 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f7b880a2b80 0x7f7b880a5040 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f7b90004ad0 tx=0x7f7b90009890 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:25.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.260+0000 7f7b9cac7640 1 --2- 172.21.15.16:0/79006599 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b18a18db40 0x55b18a18c090 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c0114f0 tx=0x7f7b8c0119b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=44fb82cca4a3c1f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:25.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.260+0000 7f7b82ffd640 1 -- 172.21.15.16:0/79006599 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b8c01e070 con 0x55b18a18db40 2024-09-12T02:13:25.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.260+0000 7f7b82ffd640 1 -- 172.21.15.16:0/79006599 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b8c011030 con 0x55b18a18db40 2024-09-12T02:13:25.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.260+0000 7f7b82ffd640 1 -- 172.21.15.16:0/79006599 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b8c010720 con 0x55b18a18db40 2024-09-12T02:13:25.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.261+0000 7f7ba1104740 1 -- 172.21.15.16:0/79006599 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b18a18db40 msgr2=0x55b18a18c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:25.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.261+0000 7f7ba1104740 1 --2- 172.21.15.16:0/79006599 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b18a18db40 0x55b18a18c090 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c0114f0 tx=0x7f7b8c0119b0 comp rx=0 tx=0).stop 2024-09-12T02:13:25.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.261+0000 7f7ba1104740 1 -- 172.21.15.16:0/79006599 shutdown_connections 2024-09-12T02:13:25.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.261+0000 7f7ba1104740 1 --2- 172.21.15.16:0/79006599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b18a197370 0x55b18a199760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:25.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.261+0000 7f7ba1104740 1 --2- 172.21.15.16:0/79006599 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b18a18c5d0 0x55b18a18ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:25.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.261+0000 7f7ba1104740 1 --2- 172.21.15.16:0/79006599 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b18a18db40 0x55b18a18c090 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:25.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.261+0000 7f7ba1104740 1 -- 172.21.15.16:0/79006599 >> 172.21.15.16:0/79006599 conn(0x55b18a155310 msgr2=0x55b18a188c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:25.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.261+0000 7f7ba1104740 1 -- 172.21.15.16:0/79006599 shutdown_connections 2024-09-12T02:13:25.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.261+0000 7f7ba1104740 1 -- 172.21.15.16:0/79006599 wait complete. 2024-09-12T02:13:25.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.261+0000 7f7ba1104740 1 -- start start 2024-09-12T02:13:25.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.261+0000 7f7ba1104740 1 Processor -- start 2024-09-12T02:13:25.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.262+0000 7f7ba1104740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b18a18c5d0 0x55b18a196d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:25.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.262+0000 7f7ba1104740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b18a18db40 0x55b18a193ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:25.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.262+0000 7f7ba1104740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b18a197370 0x55b18a1945e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:25.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.262+0000 7f7ba1104740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55b18a19a070 con 0x55b18a18db40 2024-09-12T02:13:25.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.262+0000 7f7ba1104740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55b18a0ad060 con 0x55b18a18c5d0 2024-09-12T02:13:25.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.262+0000 7f7ba1104740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55b18a19a1f0 con 0x55b18a197370 2024-09-12T02:13:25.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.262+0000 7f7ba1104740 10 client.0 _ll_register_callbacks cb 0x55b189e008c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:13:25.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.262+0000 7f7b97fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b18a18db40 0x55b18a193ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:25.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.262+0000 7f7b97fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b18a18db40 0x55b18a193ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39424/0 (socket says 172.21.15.16:39424) 2024-09-12T02:13:25.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.262+0000 7f7b97fff640 1 -- 172.21.15.16:0/2537720408 learned_addr learned my addr 172.21.15.16:0/2537720408 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:13:25.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.262+0000 7f7b9cac7640 1 --2- 172.21.15.16:0/2537720408 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b18a18c5d0 0x55b18a196d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:25.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.262+0000 7f7b9d2c8640 1 --2- 172.21.15.16:0/2537720408 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b18a197370 0x55b18a1945e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:25.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.262+0000 7f7b97fff640 1 -- 172.21.15.16:0/2537720408 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b18a197370 msgr2=0x55b18a1945e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:25.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.262+0000 7f7b97fff640 1 --2- 172.21.15.16:0/2537720408 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b18a197370 0x55b18a1945e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:25.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.262+0000 7f7b97fff640 1 -- 172.21.15.16:0/2537720408 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b18a18c5d0 msgr2=0x55b18a196d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:25.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.262+0000 7f7b97fff640 1 --2- 172.21.15.16:0/2537720408 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b18a18c5d0 0x55b18a196d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:25.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.262+0000 7f7b97fff640 1 -- 172.21.15.16:0/2537720408 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b189e633c0 con 0x55b18a18db40 2024-09-12T02:13:25.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.262+0000 7f7b97fff640 1 --2- 172.21.15.16:0/2537720408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b18a18db40 0x55b18a193ff0 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f7b8400c6a0 tx=0x7f7b8409b620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=400a00bca145f446 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:25.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.262+0000 7f7b77fff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:13:25.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.262+0000 7f7b77fff640 1 -- 172.21.15.16:0/2537720408 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b840b15b0 con 0x55b18a18db40 2024-09-12T02:13:25.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.262+0000 7f7b77fff640 1 -- 172.21.15.16:0/2537720408 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f7b840b17a0 con 0x55b18a18db40 2024-09-12T02:13:25.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.262+0000 7f7ba1104740 1 -- 172.21.15.16:0/2537720408 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b8c011030 con 0x55b18a18db40 2024-09-12T02:13:25.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.263+0000 7f7b77fff640 1 -- 172.21.15.16:0/2537720408 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7b8409c3c0 con 0x55b18a18db40 2024-09-12T02:13:25.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.263+0000 7f7ba1104740 10 client.19272 fetch_fsmap learned FSMap version 25 2024-09-12T02:13:25.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.263+0000 7f7ba1104740 1 -- 172.21.15.16:0/2537720408 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b18a1a5b40 con 0x55b18a18db40 2024-09-12T02:13:25.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.263+0000 7f7b77fff640 1 -- 172.21.15.16:0/2537720408 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b8c011030 con 0x55b18a18db40 2024-09-12T02:13:25.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.263+0000 7f7b77fff640 1 -- 172.21.15.16:0/2537720408 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f7b840aeb60 con 0x55b18a18db40 2024-09-12T02:13:25.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.263+0000 7f7ba1104740 10 client.19272 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:13:25.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.263+0000 7f7ba1104740 10 client.19272 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-12T02:13:25.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.263+0000 7f7ba1104740 20 client.19272 populate_metadata read hostname 'smithi016' 2024-09-12T02:13:25.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.263+0000 7f7ba1104740 1 --2- 172.21.15.16:0/2537720408 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55b18a1a6ff0 0x55b18a1a9420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:25.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.263+0000 7f7ba1104740 4 client.19272 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:13:25.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.263+0000 7f7ba1104740 1 -- 172.21.15.16:0/2537720408 --> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b18a0ff860 con 0x55b18a1a6ff0 2024-09-12T02:13:25.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.263+0000 7f7b9cac7640 1 --2- 172.21.15.16:0/2537720408 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55b18a1a6ff0 0x55b18a1a9420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:25.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.264+0000 7f7b77fff640 10 client.19272 ms_handle_connect on v2:172.21.15.184:6836/1793393087 2024-09-12T02:13:25.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.264+0000 7f7b9cac7640 1 --2- 172.21.15.16:0/2537720408 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55b18a1a6ff0 0x55b18a1a9420 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=99b1f69bfea5864a server_cookie=e819ac2b3a8985a4 in_seq=0 out_seq=0 2024-09-12T02:13:25.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.264+0000 7f7b77fff640 1 -- 172.21.15.16:0/2537720408 <== mds.2 v2:172.21.15.184:6836/1793393087 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b18a0ff860 con 0x55b18a1a6ff0 2024-09-12T02:13:25.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.264+0000 7f7b77fff640 10 client.19272 handle_command_reply: tid=0 2024-09-12T02:13:25.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.265+0000 7f7ba1104740 1 client.19272 shutdown 2024-09-12T02:13:25.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.265+0000 7f7ba1104740 1 -- 172.21.15.16:0/2537720408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b18a18db40 msgr2=0x55b18a193ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:25.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.265+0000 7f7ba1104740 1 --2- 172.21.15.16:0/2537720408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b18a18db40 0x55b18a193ff0 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f7b8400c6a0 tx=0x7f7b8409b620 comp rx=0 tx=0).stop 2024-09-12T02:13:25.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.265+0000 7f7ba1104740 1 -- 172.21.15.16:0/2537720408 shutdown_connections 2024-09-12T02:13:25.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.265+0000 7f7ba1104740 1 --2- 172.21.15.16:0/2537720408 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b18a197370 0x55b18a1945e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:25.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.265+0000 7f7ba1104740 1 --2- 172.21.15.16:0/2537720408 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55b18a1a6ff0 0x55b18a1a9420 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:25.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.265+0000 7f7ba1104740 1 --2- 172.21.15.16:0/2537720408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b18a18db40 0x55b18a193ff0 unknown :-1 s=CLOSED pgs=1641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:25.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.265+0000 7f7ba1104740 1 --2- 172.21.15.16:0/2537720408 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b18a18c5d0 0x55b18a196d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:25.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.265+0000 7f7b77fff640 0 client.19272 ms_handle_reset on v2:172.21.15.184:6836/1793393087 2024-09-12T02:13:25.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.265+0000 7f7ba1104740 1 -- 172.21.15.16:0/2537720408 >> 172.21.15.16:0/2537720408 conn(0x55b18a155310 msgr2=0x55b18a18e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:25.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.265+0000 7f7ba1104740 1 -- 172.21.15.16:0/2537720408 shutdown_connections 2024-09-12T02:13:25.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.265+0000 7f7ba1104740 1 -- 172.21.15.16:0/2537720408 wait complete. 2024-09-12T02:13:25.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.265+0000 7f7ba1104740 20 client.19272 trim_cache size 0 max 16384 2024-09-12T02:13:25.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.278+0000 7f7ba1104740 1 Processor -- start 2024-09-12T02:13:25.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.278+0000 7f7ba1104740 1 -- start start 2024-09-12T02:13:25.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.278+0000 7f7ba1104740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b18a18c5d0 0x55b18a1a3310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:25.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.278+0000 7f7ba1104740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b18a18db40 0x55b18a18f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:25.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.278+0000 7f7ba1104740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b18a197370 0x55b18a18f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:25.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.278+0000 7f7ba1104740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7b981b1fa0 con 0x55b18a18db40 2024-09-12T02:13:25.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.278+0000 7f7ba1104740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7b981af870 con 0x55b18a197370 2024-09-12T02:13:25.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.278+0000 7f7ba1104740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7b981ad6e0 con 0x55b18a18c5d0 2024-09-12T02:13:25.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.278+0000 7f7b97fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b18a18db40 0x55b18a18f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:25.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.278+0000 7f7b97fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b18a18db40 0x55b18a18f120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39436/0 (socket says 172.21.15.16:39436) 2024-09-12T02:13:25.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.278+0000 7f7b9cac7640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b18a18c5d0 0x55b18a1a3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:25.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.278+0000 7f7b97fff640 1 -- 172.21.15.16:0/2563389702 learned_addr learned my addr 172.21.15.16:0/2563389702 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:13:25.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.279+0000 7f7b9d2c8640 1 --2- 172.21.15.16:0/2563389702 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b18a197370 0x55b18a18f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:25.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.279+0000 7f7b9cac7640 1 -- 172.21.15.16:0/2563389702 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b18a197370 msgr2=0x55b18a18f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:25.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.279+0000 7f7b9cac7640 1 --2- 172.21.15.16:0/2563389702 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b18a197370 0x55b18a18f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:25.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.279+0000 7f7b9cac7640 1 -- 172.21.15.16:0/2563389702 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b18a18db40 msgr2=0x55b18a18f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:25.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.279+0000 7f7b9cac7640 1 --2- 172.21.15.16:0/2563389702 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b18a18db40 0x55b18a18f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:25.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.279+0000 7f7b9cac7640 1 -- 172.21.15.16:0/2563389702 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b18a196960 con 0x55b18a18c5d0 2024-09-12T02:13:25.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.279+0000 7f7b97fff640 1 --2- 172.21.15.16:0/2563389702 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b18a18db40 0x55b18a18f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:13:25.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.279+0000 7f7b9cac7640 1 --2- 172.21.15.16:0/2563389702 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b18a18c5d0 0x55b18a1a3310 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c011eb0 tx=0x7f7b8c005250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a1fb34f3deb89c3a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:25.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.279+0000 7f7b827fc640 1 -- 172.21.15.16:0/2563389702 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b8c01e040 con 0x55b18a18c5d0 2024-09-12T02:13:25.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.279+0000 7f7b827fc640 1 -- 172.21.15.16:0/2563389702 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b8c00faa0 con 0x55b18a18c5d0 2024-09-12T02:13:25.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.279+0000 7f7b827fc640 1 -- 172.21.15.16:0/2563389702 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b8c0035f0 con 0x55b18a18c5d0 2024-09-12T02:13:25.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.280+0000 7f7ba1104740 1 -- 172.21.15.16:0/2563389702 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b18a18c5d0 msgr2=0x55b18a1a3310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:25.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.280+0000 7f7ba1104740 1 --2- 172.21.15.16:0/2563389702 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b18a18c5d0 0x55b18a1a3310 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c011eb0 tx=0x7f7b8c005250 comp rx=0 tx=0).stop 2024-09-12T02:13:25.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.280+0000 7f7ba1104740 1 -- 172.21.15.16:0/2563389702 shutdown_connections 2024-09-12T02:13:25.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.280+0000 7f7ba1104740 1 --2- 172.21.15.16:0/2563389702 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b18a197370 0x55b18a18f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:25.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.280+0000 7f7ba1104740 1 --2- 172.21.15.16:0/2563389702 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b18a18db40 0x55b18a18f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:25.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.280+0000 7f7ba1104740 1 --2- 172.21.15.16:0/2563389702 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b18a18c5d0 0x55b18a1a3310 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:25.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.280+0000 7f7ba1104740 1 -- 172.21.15.16:0/2563389702 >> 172.21.15.16:0/2563389702 conn(0x55b18a155310 msgr2=0x55b18a1930e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:25.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.280+0000 7f7ba1104740 1 -- 172.21.15.16:0/2563389702 shutdown_connections 2024-09-12T02:13:25.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.280+0000 7f7ba1104740 1 -- 172.21.15.16:0/2563389702 wait complete. 2024-09-12T02:13:25.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.280+0000 7f7ba1104740 1 -- start start 2024-09-12T02:13:25.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.280+0000 7f7ba1104740 1 Processor -- start 2024-09-12T02:13:25.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.281+0000 7f7ba1104740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b18a18c5d0 0x55b18a19f870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:25.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.281+0000 7f7ba1104740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b18a18db40 0x55b18a19fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:25.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.281+0000 7f7ba1104740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b18a197370 0x55b18a19adf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:25.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.281+0000 7f7ba1104740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7b981add30 con 0x55b18a18c5d0 2024-09-12T02:13:25.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.281+0000 7f7ba1104740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7b981af1a0 con 0x55b18a18db40 2024-09-12T02:13:25.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.281+0000 7f7ba1104740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7b981ae400 con 0x55b18a197370 2024-09-12T02:13:25.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.281+0000 7f7ba1104740 10 client.0 _ll_register_callbacks cb 0x7f7b8400cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:13:25.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.281+0000 7f7b9cac7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b18a18c5d0 0x55b18a19f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:25.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.281+0000 7f7b9cac7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b18a18c5d0 0x55b18a19f870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39448/0 (socket says 172.21.15.16:39448) 2024-09-12T02:13:25.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.281+0000 7f7b9cac7640 1 -- 172.21.15.16:0/814930686 learned_addr learned my addr 172.21.15.16:0/814930686 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:13:25.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.281+0000 7f7b9d2c8640 1 --2- 172.21.15.16:0/814930686 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b18a197370 0x55b18a19adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:25.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.281+0000 7f7b97fff640 1 --2- 172.21.15.16:0/814930686 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b18a18db40 0x55b18a19fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:25.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.281+0000 7f7b9cac7640 1 -- 172.21.15.16:0/814930686 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b18a197370 msgr2=0x55b18a19adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:25.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.281+0000 7f7b9cac7640 1 --2- 172.21.15.16:0/814930686 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b18a197370 0x55b18a19adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:25.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.281+0000 7f7b9cac7640 1 -- 172.21.15.16:0/814930686 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b18a18db40 msgr2=0x55b18a19fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:25.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.281+0000 7f7b9cac7640 1 --2- 172.21.15.16:0/814930686 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b18a18db40 0x55b18a19fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:25.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.281+0000 7f7b9cac7640 1 -- 172.21.15.16:0/814930686 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b18a19b620 con 0x55b18a18c5d0 2024-09-12T02:13:25.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.281+0000 7f7b9cac7640 1 --2- 172.21.15.16:0/814930686 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b18a18c5d0 0x55b18a19f870 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c01c040 tx=0x7f7b8c005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=807760b5b9012448 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:25.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.281+0000 7f7b77fff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:13:25.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.282+0000 7f7b77fff640 1 -- 172.21.15.16:0/814930686 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b8c036070 con 0x55b18a18c5d0 2024-09-12T02:13:25.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.282+0000 7f7b77fff640 1 -- 172.21.15.16:0/814930686 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f7b8c00f5c0 con 0x55b18a18c5d0 2024-09-12T02:13:25.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.282+0000 7f7ba1104740 1 -- 172.21.15.16:0/814930686 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b8c00faa0 con 0x55b18a18c5d0 2024-09-12T02:13:25.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.282+0000 7f7b77fff640 1 -- 172.21.15.16:0/814930686 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7b8c01e040 con 0x55b18a18c5d0 2024-09-12T02:13:25.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.282+0000 7f7ba1104740 10 client.19278 fetch_fsmap learned FSMap version 25 2024-09-12T02:13:25.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.282+0000 7f7ba1104740 1 -- 172.21.15.16:0/814930686 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b18a211970 con 0x55b18a18c5d0 2024-09-12T02:13:25.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.282+0000 7f7b77fff640 1 -- 172.21.15.16:0/814930686 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b8c00faa0 con 0x55b18a18c5d0 2024-09-12T02:13:25.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.282+0000 7f7b77fff640 1 -- 172.21.15.16:0/814930686 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f7b8c02a2f0 con 0x55b18a18c5d0 2024-09-12T02:13:25.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.282+0000 7f7ba1104740 10 client.19278 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:13:25.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.282+0000 7f7ba1104740 10 client.19278 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-12T02:13:25.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.282+0000 7f7ba1104740 20 client.19278 populate_metadata read hostname 'smithi016' 2024-09-12T02:13:25.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.282+0000 7f7ba1104740 1 --2- 172.21.15.16:0/814930686 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55b18a211de0 0x55b18a2141d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:25.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.282+0000 7f7ba1104740 4 client.19278 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T02:13:25.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.282+0000 7f7ba1104740 1 -- 172.21.15.16:0/814930686 --> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b18a0f9d70 con 0x55b18a211de0 2024-09-12T02:13:25.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.283+0000 7f7b97fff640 1 --2- 172.21.15.16:0/814930686 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55b18a211de0 0x55b18a2141d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:25.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.283+0000 7f7b77fff640 10 client.19278 ms_handle_connect on v2:172.21.15.184:6836/1793393087 2024-09-12T02:13:25.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.283+0000 7f7b97fff640 1 --2- 172.21.15.16:0/814930686 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55b18a211de0 0x55b18a2141d0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=34ef6be9516e6798 server_cookie=f37f5a5b0e107b35 in_seq=0 out_seq=0 2024-09-12T02:13:25.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.283+0000 7f7b77fff640 1 -- 172.21.15.16:0/814930686 <== mds.2 v2:172.21.15.184:6836/1793393087 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b18a0f9d70 con 0x55b18a211de0 2024-09-12T02:13:25.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.283+0000 7f7b77fff640 10 client.19278 handle_command_reply: tid=0 2024-09-12T02:13:25.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.283+0000 7f7ba1104740 1 client.19278 shutdown 2024-09-12T02:13:25.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.283+0000 7f7ba1104740 1 -- 172.21.15.16:0/814930686 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b18a18c5d0 msgr2=0x55b18a19f870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:25.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.283+0000 7f7ba1104740 1 --2- 172.21.15.16:0/814930686 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b18a18c5d0 0x55b18a19f870 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c01c040 tx=0x7f7b8c005fd0 comp rx=0 tx=0).stop 2024-09-12T02:13:25.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.283+0000 7f7ba1104740 1 -- 172.21.15.16:0/814930686 shutdown_connections 2024-09-12T02:13:25.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.283+0000 7f7ba1104740 1 --2- 172.21.15.16:0/814930686 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b18a197370 0x55b18a19adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:25.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.283+0000 7f7ba1104740 1 --2- 172.21.15.16:0/814930686 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b18a18db40 0x55b18a19fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:25.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.283+0000 7f7ba1104740 1 --2- 172.21.15.16:0/814930686 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55b18a211de0 0x55b18a2141d0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:25.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.283+0000 7f7ba1104740 1 --2- 172.21.15.16:0/814930686 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b18a18c5d0 0x55b18a19f870 unknown :-1 s=CLOSED pgs=1642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:25.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.283+0000 7f7ba1104740 1 -- 172.21.15.16:0/814930686 >> 172.21.15.16:0/814930686 conn(0x55b18a155310 msgr2=0x55b18a195850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:25.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.283+0000 7f7b77fff640 0 client.19278 ms_handle_reset on v2:172.21.15.184:6836/1793393087 2024-09-12T02:13:25.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.283+0000 7f7ba1104740 1 -- 172.21.15.16:0/814930686 shutdown_connections 2024-09-12T02:13:25.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.283+0000 7f7ba1104740 1 -- 172.21.15.16:0/814930686 wait complete. 2024-09-12T02:13:25.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.284+0000 7f7ba1104740 20 client.19278 trim_cache size 0 max 16384 2024-09-12T02:13:25.311 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T02:13:25.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.284+0000 7f7b9ed2b640 1 -- 172.21.15.16:0/497629190 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f7b880a2b80 msgr2=0x7f7b880a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:25.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.284+0000 7f7b9ed2b640 1 --2- 172.21.15.16:0/497629190 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f7b880a2b80 0x7f7b880a5040 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f7b90004ad0 tx=0x7f7b90009890 comp rx=0 tx=0).stop 2024-09-12T02:13:25.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.284+0000 7f7b9ed2b640 1 -- 172.21.15.16:0/497629190 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7b98101420 msgr2=0x7f7b9810e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:25.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.284+0000 7f7b9ed2b640 1 --2- 172.21.15.16:0/497629190 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7b98101420 0x7f7b9810e920 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f7b8400e9f0 tx=0x7f7b8400edf0 comp rx=0 tx=0).stop 2024-09-12T02:13:25.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.284+0000 7f7b9ed2b640 1 -- 172.21.15.16:0/497629190 shutdown_connections 2024-09-12T02:13:25.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.284+0000 7f7b9ed2b640 1 --2- 172.21.15.16:0/497629190 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f7b880a2b80 0x7f7b880a5040 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:25.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.284+0000 7f7b9ed2b640 1 --2- 172.21.15.16:0/497629190 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b98106a80 0x7f7b9810ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:25.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.284+0000 7f7b9ed2b640 1 --2- 172.21.15.16:0/497629190 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7b98101420 0x7f7b9810e920 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:25.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.284+0000 7f7b9ed2b640 1 --2- 172.21.15.16:0/497629190 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b98100a80 0x7f7b9810e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:25.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.284+0000 7f7b9ed2b640 1 -- 172.21.15.16:0/497629190 >> 172.21.15.16:0/497629190 conn(0x7f7b980fc7a0 msgr2=0x7f7b980fcfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:25.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.284+0000 7f7b9ed2b640 1 -- 172.21.15.16:0/497629190 shutdown_connections 2024-09-12T02:13:25.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:25.284+0000 7f7b9ed2b640 1 -- 172.21.15.16:0/497629190 wait complete. 2024-09-12T02:13:26.007 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-12T02:13:26.266 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:13:26.376 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:13:26 smithi016 ceph-mon[30947]: pgmap v680: 97 pgs: 97 active+clean; 2.0 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-09-12T02:13:26.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:13:26 smithi184 ceph-mon[35593]: pgmap v680: 97 pgs: 97 active+clean; 2.0 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-09-12T02:13:26.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:13:26 smithi033 ceph-mon[35332]: pgmap v680: 97 pgs: 97 active+clean; 2.0 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-09-12T02:13:26.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.696+0000 7f65737d4640 1 -- 172.21.15.16:0/2173779705 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f656c101420 msgr2=0x7f656c10d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:26.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.696+0000 7f65737d4640 1 --2- 172.21.15.16:0/2173779705 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f656c101420 0x7f656c10d950 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7f6564009cd0 tx=0x7f656402a8e0 comp rx=0 tx=0).stop 2024-09-12T02:13:26.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.696+0000 7f65737d4640 1 -- 172.21.15.16:0/2173779705 shutdown_connections 2024-09-12T02:13:26.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.696+0000 7f65737d4640 1 --2- 172.21.15.16:0/2173779705 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f656c101420 0x7f656c10d950 unknown :-1 s=CLOSED pgs=1643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:26.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.696+0000 7f65737d4640 1 --2- 172.21.15.16:0/2173779705 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f656c100a80 0x7f656c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:26.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.696+0000 7f65737d4640 1 --2- 172.21.15.16:0/2173779705 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f656c106a80 0x7f656c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:26.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.696+0000 7f65737d4640 1 -- 172.21.15.16:0/2173779705 >> 172.21.15.16:0/2173779705 conn(0x7f656c0fc7a0 msgr2=0x7f656c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:26.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.696+0000 7f65737d4640 1 -- 172.21.15.16:0/2173779705 shutdown_connections 2024-09-12T02:13:26.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.696+0000 7f65737d4640 1 -- 172.21.15.16:0/2173779705 wait complete. 2024-09-12T02:13:26.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.696+0000 7f65737d4640 1 Processor -- start 2024-09-12T02:13:26.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.696+0000 7f65737d4640 1 -- start start 2024-09-12T02:13:26.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.697+0000 7f65737d4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f656c100a80 0x7f656c198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:26.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.697+0000 7f65737d4640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f656c101420 0x7f656c198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:26.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.697+0000 7f65737d4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f656c106a80 0x7f656c19cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:26.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.697+0000 7f65737d4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f656c110e80 con 0x7f656c106a80 2024-09-12T02:13:26.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.697+0000 7f65737d4640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f656c110d00 con 0x7f656c100a80 2024-09-12T02:13:26.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.697+0000 7f65737d4640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f656c111000 con 0x7f656c101420 2024-09-12T02:13:26.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.697+0000 7f6571d71640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f656c106a80 0x7f656c19cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:26.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.697+0000 7f6571d71640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f656c106a80 0x7f656c19cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39466/0 (socket says 172.21.15.16:39466) 2024-09-12T02:13:26.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.697+0000 7f6571d71640 1 -- 172.21.15.16:0/3959782365 learned_addr learned my addr 172.21.15.16:0/3959782365 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:13:26.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.697+0000 7f6571570640 1 --2- 172.21.15.16:0/3959782365 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f656c100a80 0x7f656c198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:26.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.697+0000 7f6570d6f640 1 --2- 172.21.15.16:0/3959782365 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f656c101420 0x7f656c198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:26.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.697+0000 7f6571d71640 1 -- 172.21.15.16:0/3959782365 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f656c101420 msgr2=0x7f656c198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:26.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.697+0000 7f6571d71640 1 --2- 172.21.15.16:0/3959782365 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f656c101420 0x7f656c198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:26.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.697+0000 7f6571d71640 1 -- 172.21.15.16:0/3959782365 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f656c100a80 msgr2=0x7f656c198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:26.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.697+0000 7f6571d71640 1 --2- 172.21.15.16:0/3959782365 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f656c100a80 0x7f656c198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:26.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.697+0000 7f6571d71640 1 -- 172.21.15.16:0/3959782365 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f656c19d3e0 con 0x7f656c106a80 2024-09-12T02:13:26.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.697+0000 7f6571d71640 1 --2- 172.21.15.16:0/3959782365 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f656c106a80 0x7f656c19cd00 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7f6564005cc0 tx=0x7f6564003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:26.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.698+0000 7f655e7fc640 1 -- 172.21.15.16:0/3959782365 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6564004040 con 0x7f656c106a80 2024-09-12T02:13:26.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.698+0000 7f65737d4640 1 -- 172.21.15.16:0/3959782365 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f656c19d670 con 0x7f656c106a80 2024-09-12T02:13:26.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.698+0000 7f655e7fc640 1 -- 172.21.15.16:0/3959782365 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65640046c0 con 0x7f656c106a80 2024-09-12T02:13:26.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.698+0000 7f655e7fc640 1 -- 172.21.15.16:0/3959782365 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6564046480 con 0x7f656c106a80 2024-09-12T02:13:26.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.698+0000 7f65737d4640 1 -- 172.21.15.16:0/3959782365 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f656c1a4fb0 con 0x7f656c106a80 2024-09-12T02:13:26.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.700+0000 7f655e7fc640 1 -- 172.21.15.16:0/3959782365 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f65640305a0 con 0x7f656c106a80 2024-09-12T02:13:26.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.700+0000 7f6575bad740 1 Processor -- start 2024-09-12T02:13:26.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.700+0000 7f6575bad740 1 -- start start 2024-09-12T02:13:26.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.700+0000 7f6575bad740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d83e10cb40 0x55d83e10b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:26.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.700+0000 7f6575bad740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d83e10b5d0 0x55d83e10ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:26.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.700+0000 7f6575bad740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d83e116370 0x55d83e118760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:26.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.700+0000 7f6575bad740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55d83e0712f0 con 0x55d83e10b5d0 2024-09-12T02:13:26.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.700+0000 7f6575bad740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55d83e018cf0 con 0x55d83e116370 2024-09-12T02:13:26.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.700+0000 7f6575bad740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55d83e06d2b0 con 0x55d83e10cb40 2024-09-12T02:13:26.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.701+0000 7f6571570640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d83e10b5d0 0x55d83e10ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:26.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.701+0000 7f6571570640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d83e10b5d0 0x55d83e10ba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39478/0 (socket says 172.21.15.16:39478) 2024-09-12T02:13:26.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.701+0000 7f6571570640 1 -- 172.21.15.16:0/3788449937 learned_addr learned my addr 172.21.15.16:0/3788449937 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:13:26.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.701+0000 7f655e7fc640 1 --2- 172.21.15.16:0/3959782365 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f65580a2c00 0x7f65580a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:26.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.701+0000 7f6570d6f640 1 --2- 172.21.15.16:0/3788449937 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d83e116370 0x55d83e118760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:26.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.701+0000 7f6570d6f640 1 --2- 172.21.15.16:0/3788449937 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d83e10cb40 0x55d83e10b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:26.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.701+0000 7f6571d71640 1 --2- 172.21.15.16:0/3959782365 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f65580a2c00 0x7f65580a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:26.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.701+0000 7f6571570640 1 -- 172.21.15.16:0/3788449937 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d83e10cb40 msgr2=0x55d83e10b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:26.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.701+0000 7f6571570640 1 --2- 172.21.15.16:0/3788449937 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d83e10cb40 0x55d83e10b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:26.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.701+0000 7f6571570640 1 -- 172.21.15.16:0/3788449937 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d83e116370 msgr2=0x55d83e118760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:26.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.701+0000 7f6571570640 1 --2- 172.21.15.16:0/3788449937 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d83e116370 0x55d83e118760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:26.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.701+0000 7f6571570640 1 -- 172.21.15.16:0/3788449937 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d83dffd3a0 con 0x55d83e10b5d0 2024-09-12T02:13:26.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.701+0000 7f6570d6f640 1 --2- 172.21.15.16:0/3788449937 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d83e10cb40 0x55d83e10b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:13:26.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.701+0000 7f655e7fc640 1 -- 172.21.15.16:0/3959782365 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f656402e030 con 0x7f656c106a80 2024-09-12T02:13:26.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.701+0000 7f6571570640 1 --2- 172.21.15.16:0/3788449937 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d83e10b5d0 0x55d83e10ba10 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7f656800b710 tx=0x7f6568004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0fd07ae0415fb37 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:26.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.701+0000 7f65577fe640 1 -- 172.21.15.16:0/3788449937 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6568003bd0 con 0x55d83e10b5d0 2024-09-12T02:13:26.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.701+0000 7f6571d71640 1 --2- 172.21.15.16:0/3959782365 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f65580a2c00 0x7f65580a50c0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f65640b93f0 tx=0x7f65640dd040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:26.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.701+0000 7f65577fe640 1 -- 172.21.15.16:0/3788449937 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6568015070 con 0x55d83e10b5d0 2024-09-12T02:13:26.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.701+0000 7f65577fe640 1 -- 172.21.15.16:0/3788449937 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6568004e40 con 0x55d83e10b5d0 2024-09-12T02:13:26.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.702+0000 7f6575bad740 1 -- 172.21.15.16:0/3788449937 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d83e10b5d0 msgr2=0x55d83e10ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:26.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.702+0000 7f6575bad740 1 --2- 172.21.15.16:0/3788449937 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d83e10b5d0 0x55d83e10ba10 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7f656800b710 tx=0x7f6568004610 comp rx=0 tx=0).stop 2024-09-12T02:13:26.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.702+0000 7f6575bad740 1 -- 172.21.15.16:0/3788449937 shutdown_connections 2024-09-12T02:13:26.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.702+0000 7f6575bad740 1 --2- 172.21.15.16:0/3788449937 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d83e116370 0x55d83e118760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:26.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.702+0000 7f6575bad740 1 --2- 172.21.15.16:0/3788449937 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d83e10b5d0 0x55d83e10ba10 unknown :-1 s=CLOSED pgs=1645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:26.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.702+0000 7f6575bad740 1 --2- 172.21.15.16:0/3788449937 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d83e10cb40 0x55d83e10b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:26.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.702+0000 7f6575bad740 1 -- 172.21.15.16:0/3788449937 >> 172.21.15.16:0/3788449937 conn(0x55d83e0d4310 msgr2=0x55d83e107c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:26.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.702+0000 7f6575bad740 1 -- 172.21.15.16:0/3788449937 shutdown_connections 2024-09-12T02:13:26.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.702+0000 7f6575bad740 1 -- 172.21.15.16:0/3788449937 wait complete. 2024-09-12T02:13:26.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.703+0000 7f6575bad740 1 -- start start 2024-09-12T02:13:26.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.703+0000 7f6575bad740 1 Processor -- start 2024-09-12T02:13:26.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.703+0000 7f6575bad740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d83e10b5d0 0x55d83e115d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:26.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.703+0000 7f6575bad740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d83e10cb40 0x55d83e113030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:26.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.703+0000 7f6575bad740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d83e116370 0x55d83e113640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:26.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.703+0000 7f6575bad740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55d83e119070 con 0x55d83e116370 2024-09-12T02:13:26.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.703+0000 7f6575bad740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55d83e02c060 con 0x55d83e10cb40 2024-09-12T02:13:26.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.703+0000 7f6575bad740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55d83e1191f0 con 0x55d83e10b5d0 2024-09-12T02:13:26.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.703+0000 7f6575bad740 10 client.0 _ll_register_callbacks cb 0x55d83dd7f8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:13:26.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.703+0000 7f6570d6f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d83e116370 0x55d83e113640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:26.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.703+0000 7f6570d6f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d83e116370 0x55d83e113640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39486/0 (socket says 172.21.15.16:39486) 2024-09-12T02:13:26.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.703+0000 7f6570d6f640 1 -- 172.21.15.16:0/2002654135 learned_addr learned my addr 172.21.15.16:0/2002654135 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:13:26.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.703+0000 7f6571570640 1 --2- 172.21.15.16:0/2002654135 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d83e10cb40 0x55d83e113030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:26.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.703+0000 7f6570d6f640 1 --2- 172.21.15.16:0/2002654135 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d83e10b5d0 0x55d83e115d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:26.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.704+0000 7f6570d6f640 1 -- 172.21.15.16:0/2002654135 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d83e10b5d0 msgr2=0x55d83e115d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:26.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.704+0000 7f6570d6f640 1 --2- 172.21.15.16:0/2002654135 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d83e10b5d0 0x55d83e115d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:26.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.704+0000 7f6570d6f640 1 -- 172.21.15.16:0/2002654135 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d83e10cb40 msgr2=0x55d83e113030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:26.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.704+0000 7f6570d6f640 1 --2- 172.21.15.16:0/2002654135 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d83e10cb40 0x55d83e113030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:26.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.704+0000 7f6570d6f640 1 -- 172.21.15.16:0/2002654135 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d83dde23c0 con 0x55d83e116370 2024-09-12T02:13:26.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.704+0000 7f6570d6f640 1 --2- 172.21.15.16:0/2002654135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d83e116370 0x55d83e113640 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7f656000fab0 tx=0x7f65600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc94a03930f6ddd0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:26.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.704+0000 7f6554ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:13:26.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.704+0000 7f6554ff9640 1 -- 172.21.15.16:0/2002654135 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6560005020 con 0x55d83e116370 2024-09-12T02:13:26.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.704+0000 7f6554ff9640 1 -- 172.21.15.16:0/2002654135 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f65600051c0 con 0x55d83e116370 2024-09-12T02:13:26.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.704+0000 7f6575bad740 1 -- 172.21.15.16:0/2002654135 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6568015070 con 0x55d83e116370 2024-09-12T02:13:26.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.704+0000 7f6554ff9640 1 -- 172.21.15.16:0/2002654135 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f65600053c0 con 0x55d83e116370 2024-09-12T02:13:26.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.704+0000 7f6575bad740 10 client.19290 fetch_fsmap learned FSMap version 25 2024-09-12T02:13:26.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.704+0000 7f6575bad740 1 -- 172.21.15.16:0/2002654135 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d83e124b60 con 0x55d83e116370 2024-09-12T02:13:26.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.704+0000 7f6554ff9640 1 -- 172.21.15.16:0/2002654135 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6568015070 con 0x55d83e116370 2024-09-12T02:13:26.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.704+0000 7f6554ff9640 1 -- 172.21.15.16:0/2002654135 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f6560023020 con 0x55d83e116370 2024-09-12T02:13:26.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.705+0000 7f6575bad740 10 client.19290 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:13:26.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.705+0000 7f6575bad740 10 client.19290 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T02:13:26.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.705+0000 7f6575bad740 20 client.19290 populate_metadata read hostname 'smithi016' 2024-09-12T02:13:26.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.705+0000 7f6575bad740 1 --2- 172.21.15.16:0/2002654135 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x55d83e125f70 0x55d83e128360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:26.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.705+0000 7f6575bad740 4 client.19290 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:13:26.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.705+0000 7f6575bad740 1 -- 172.21.15.16:0/2002654135 --> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d83e07e860 con 0x55d83e125f70 2024-09-12T02:13:26.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.705+0000 7f6571570640 1 --2- 172.21.15.16:0/2002654135 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x55d83e125f70 0x55d83e128360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:26.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.705+0000 7f6571570640 1 --2- 172.21.15.16:0/2002654135 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x55d83e125f70 0x55d83e128360 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3fb54554ace14e38 server_cookie=3eb09b3efe7ee0d0 in_seq=0 out_seq=0 2024-09-12T02:13:26.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.705+0000 7f6554ff9640 10 client.19290 ms_handle_connect on v2:172.21.15.184:6838/4203347941 2024-09-12T02:13:26.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.706+0000 7f6554ff9640 1 -- 172.21.15.16:0/2002654135 <== mds.1 v2:172.21.15.184:6838/4203347941 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d83e07e860 con 0x55d83e125f70 2024-09-12T02:13:26.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.706+0000 7f6554ff9640 10 client.19290 handle_command_reply: tid=0 2024-09-12T02:13:26.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.706+0000 7f6575bad740 1 client.19290 shutdown 2024-09-12T02:13:26.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.706+0000 7f6575bad740 1 -- 172.21.15.16:0/2002654135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d83e116370 msgr2=0x55d83e113640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:26.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.706+0000 7f6575bad740 1 --2- 172.21.15.16:0/2002654135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d83e116370 0x55d83e113640 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7f656000fab0 tx=0x7f65600046c0 comp rx=0 tx=0).stop 2024-09-12T02:13:26.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.706+0000 7f6575bad740 1 -- 172.21.15.16:0/2002654135 shutdown_connections 2024-09-12T02:13:26.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.706+0000 7f6575bad740 1 --2- 172.21.15.16:0/2002654135 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x55d83e125f70 0x55d83e128360 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:26.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.706+0000 7f6575bad740 1 --2- 172.21.15.16:0/2002654135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d83e116370 0x55d83e113640 unknown :-1 s=CLOSED pgs=1646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:26.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.706+0000 7f6575bad740 1 --2- 172.21.15.16:0/2002654135 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d83e10cb40 0x55d83e113030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:26.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.706+0000 7f6575bad740 1 --2- 172.21.15.16:0/2002654135 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d83e10b5d0 0x55d83e115d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:26.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.706+0000 7f6554ff9640 0 client.19290 ms_handle_reset on v2:172.21.15.184:6838/4203347941 2024-09-12T02:13:26.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.706+0000 7f6575bad740 1 -- 172.21.15.16:0/2002654135 >> 172.21.15.16:0/2002654135 conn(0x55d83e0d4310 msgr2=0x55d83e10d960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:26.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.706+0000 7f6575bad740 1 -- 172.21.15.16:0/2002654135 shutdown_connections 2024-09-12T02:13:26.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.706+0000 7f6575bad740 1 -- 172.21.15.16:0/2002654135 wait complete. 2024-09-12T02:13:26.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.706+0000 7f6575bad740 20 client.19290 trim_cache size 0 max 16384 2024-09-12T02:13:26.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.719+0000 7f6575bad740 1 Processor -- start 2024-09-12T02:13:26.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.720+0000 7f6575bad740 1 -- start start 2024-09-12T02:13:26.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.720+0000 7f6575bad740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d83e10b5d0 0x55d83e114fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:26.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.720+0000 7f6575bad740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d83e10cb40 0x55d83e11e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:26.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.720+0000 7f6575bad740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d83e116370 0x55d83e11eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:26.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.720+0000 7f6575bad740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f656c1afc40 con 0x55d83e116370 2024-09-12T02:13:26.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.720+0000 7f6575bad740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f656c1ad510 con 0x55d83e10b5d0 2024-09-12T02:13:26.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.720+0000 7f6575bad740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f656c1ab360 con 0x55d83e10cb40 2024-09-12T02:13:26.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.720+0000 7f6570d6f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d83e116370 0x55d83e11eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:26.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.720+0000 7f6570d6f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d83e116370 0x55d83e11eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39498/0 (socket says 172.21.15.16:39498) 2024-09-12T02:13:26.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.720+0000 7f6570d6f640 1 -- 172.21.15.16:0/785629721 learned_addr learned my addr 172.21.15.16:0/785629721 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:13:26.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.720+0000 7f6571570640 1 --2- 172.21.15.16:0/785629721 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d83e10cb40 0x55d83e11e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:26.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.720+0000 7f6570d6f640 1 --2- 172.21.15.16:0/785629721 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d83e10b5d0 0x55d83e114fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:26.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.720+0000 7f6570d6f640 1 -- 172.21.15.16:0/785629721 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d83e10cb40 msgr2=0x55d83e11e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:26.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.720+0000 7f6570d6f640 1 --2- 172.21.15.16:0/785629721 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d83e10cb40 0x55d83e11e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:26.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.720+0000 7f6570d6f640 1 -- 172.21.15.16:0/785629721 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d83e10b5d0 msgr2=0x55d83e114fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:26.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.720+0000 7f6570d6f640 1 --2- 172.21.15.16:0/785629721 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d83e10b5d0 0x55d83e114fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:26.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.720+0000 7f6570d6f640 1 -- 172.21.15.16:0/785629721 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d83e1160f0 con 0x55d83e116370 2024-09-12T02:13:26.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.720+0000 7f6571570640 1 --2- 172.21.15.16:0/785629721 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d83e10cb40 0x55d83e11e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:13:26.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.720+0000 7f6570d6f640 1 --2- 172.21.15.16:0/785629721 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d83e116370 0x55d83e11eda0 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x55d83e0fc380 tx=0x7f6560005d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dad1741b6ef91364 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:26.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.721+0000 7f6556ffd640 1 -- 172.21.15.16:0/785629721 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f656000ecc0 con 0x55d83e116370 2024-09-12T02:13:26.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.721+0000 7f6556ffd640 1 -- 172.21.15.16:0/785629721 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f656000ee60 con 0x55d83e116370 2024-09-12T02:13:26.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.721+0000 7f6556ffd640 1 -- 172.21.15.16:0/785629721 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6560025840 con 0x55d83e116370 2024-09-12T02:13:26.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.721+0000 7f6575bad740 1 -- 172.21.15.16:0/785629721 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d83e116370 msgr2=0x55d83e11eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:26.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.721+0000 7f6575bad740 1 --2- 172.21.15.16:0/785629721 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d83e116370 0x55d83e11eda0 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x55d83e0fc380 tx=0x7f6560005d20 comp rx=0 tx=0).stop 2024-09-12T02:13:26.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.721+0000 7f6575bad740 1 -- 172.21.15.16:0/785629721 shutdown_connections 2024-09-12T02:13:26.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.721+0000 7f6575bad740 1 --2- 172.21.15.16:0/785629721 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d83e116370 0x55d83e11eda0 unknown :-1 s=CLOSED pgs=1647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:26.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.721+0000 7f6575bad740 1 --2- 172.21.15.16:0/785629721 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d83e10cb40 0x55d83e11e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:26.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.721+0000 7f6575bad740 1 --2- 172.21.15.16:0/785629721 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d83e10b5d0 0x55d83e114fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:26.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.721+0000 7f6575bad740 1 -- 172.21.15.16:0/785629721 >> 172.21.15.16:0/785629721 conn(0x55d83e0d4310 msgr2=0x55d83e114890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:26.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.721+0000 7f6575bad740 1 -- 172.21.15.16:0/785629721 shutdown_connections 2024-09-12T02:13:26.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.721+0000 7f6575bad740 1 -- 172.21.15.16:0/785629721 wait complete. 2024-09-12T02:13:26.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.722+0000 7f6575bad740 1 -- start start 2024-09-12T02:13:26.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.722+0000 7f6575bad740 1 Processor -- start 2024-09-12T02:13:26.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.722+0000 7f6575bad740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d83e10b5d0 0x55d83e10f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:26.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.722+0000 7f6575bad740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d83e10cb40 0x55d83e10ff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:26.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.722+0000 7f6575bad740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d83e116370 0x55d83e110440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:26.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.722+0000 7f6575bad740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f656c1ab9d0 con 0x55d83e10b5d0 2024-09-12T02:13:26.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.722+0000 7f6575bad740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f656c1ace40 con 0x55d83e10cb40 2024-09-12T02:13:26.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.722+0000 7f6575bad740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f656c1ac0a0 con 0x55d83e116370 2024-09-12T02:13:26.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.722+0000 7f6575bad740 10 client.0 _ll_register_callbacks cb 0x7f656000a7c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:13:26.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.722+0000 7f6570d6f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d83e10b5d0 0x55d83e10f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:26.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.722+0000 7f6570d6f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d83e10b5d0 0x55d83e10f9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39502/0 (socket says 172.21.15.16:39502) 2024-09-12T02:13:26.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.722+0000 7f6570d6f640 1 -- 172.21.15.16:0/3664882164 learned_addr learned my addr 172.21.15.16:0/3664882164 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:13:26.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.722+0000 7f6571570640 1 --2- 172.21.15.16:0/3664882164 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d83e10cb40 0x55d83e10ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:26.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.723+0000 7f6570d6f640 1 -- 172.21.15.16:0/3664882164 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d83e116370 msgr2=0x55d83e110440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:26.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.723+0000 7f6570d6f640 1 --2- 172.21.15.16:0/3664882164 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d83e116370 0x55d83e110440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:26.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.723+0000 7f6570d6f640 1 -- 172.21.15.16:0/3664882164 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d83e10cb40 msgr2=0x55d83e10ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:26.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.723+0000 7f6570d6f640 1 --2- 172.21.15.16:0/3664882164 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d83e10cb40 0x55d83e10ff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:26.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.723+0000 7f6570d6f640 1 -- 172.21.15.16:0/3664882164 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d83e193be0 con 0x55d83e10b5d0 2024-09-12T02:13:26.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.723+0000 7f6570d6f640 1 --2- 172.21.15.16:0/3664882164 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d83e10b5d0 0x55d83e10f9c0 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7f65600254a0 tx=0x7f65600048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c732ef992557ea0b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:26.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.723+0000 7f6554ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:13:26.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.723+0000 7f6554ff9640 1 -- 172.21.15.16:0/3664882164 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f656000ecc0 con 0x55d83e10b5d0 2024-09-12T02:13:26.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.723+0000 7f6554ff9640 1 -- 172.21.15.16:0/3664882164 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f6560025750 con 0x55d83e10b5d0 2024-09-12T02:13:26.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.723+0000 7f6575bad740 1 -- 172.21.15.16:0/3664882164 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f656000ee60 con 0x55d83e10b5d0 2024-09-12T02:13:26.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.723+0000 7f6554ff9640 1 -- 172.21.15.16:0/3664882164 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6560044050 con 0x55d83e10b5d0 2024-09-12T02:13:26.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.723+0000 7f6575bad740 10 client.19296 fetch_fsmap learned FSMap version 25 2024-09-12T02:13:26.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.723+0000 7f6575bad740 1 -- 172.21.15.16:0/3664882164 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d83e194ec0 con 0x55d83e10b5d0 2024-09-12T02:13:26.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.723+0000 7f6554ff9640 1 -- 172.21.15.16:0/3664882164 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f656000ee60 con 0x55d83e10b5d0 2024-09-12T02:13:26.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.724+0000 7f6554ff9640 1 -- 172.21.15.16:0/3664882164 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f6560033070 con 0x55d83e10b5d0 2024-09-12T02:13:26.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.724+0000 7f6575bad740 10 client.19296 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:13:26.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.724+0000 7f6575bad740 10 client.19296 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T02:13:26.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.724+0000 7f6575bad740 20 client.19296 populate_metadata read hostname 'smithi016' 2024-09-12T02:13:26.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.724+0000 7f6575bad740 1 --2- 172.21.15.16:0/3664882164 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x55d83e195500 0x55d83e197920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:26.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.724+0000 7f6575bad740 4 client.19296 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T02:13:26.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.724+0000 7f6575bad740 1 -- 172.21.15.16:0/3664882164 --> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d83e078d70 con 0x55d83e195500 2024-09-12T02:13:26.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.724+0000 7f6571570640 1 --2- 172.21.15.16:0/3664882164 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x55d83e195500 0x55d83e197920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:26.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.724+0000 7f6571570640 1 --2- 172.21.15.16:0/3664882164 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x55d83e195500 0x55d83e197920 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f8a077b6fba91110 server_cookie=762102d289fe49c3 in_seq=0 out_seq=0 2024-09-12T02:13:26.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.724+0000 7f6554ff9640 10 client.19296 ms_handle_connect on v2:172.21.15.184:6838/4203347941 2024-09-12T02:13:26.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.724+0000 7f6554ff9640 1 -- 172.21.15.16:0/3664882164 <== mds.1 v2:172.21.15.184:6838/4203347941 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d83e078d70 con 0x55d83e195500 2024-09-12T02:13:26.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.724+0000 7f6554ff9640 10 client.19296 handle_command_reply: tid=0 2024-09-12T02:13:26.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.724+0000 7f6575bad740 1 client.19296 shutdown 2024-09-12T02:13:26.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.725+0000 7f6575bad740 1 -- 172.21.15.16:0/3664882164 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d83e10b5d0 msgr2=0x55d83e10f9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:26.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.725+0000 7f6575bad740 1 --2- 172.21.15.16:0/3664882164 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d83e10b5d0 0x55d83e10f9c0 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7f65600254a0 tx=0x7f65600048f0 comp rx=0 tx=0).stop 2024-09-12T02:13:26.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.725+0000 7f6575bad740 1 -- 172.21.15.16:0/3664882164 shutdown_connections 2024-09-12T02:13:26.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.725+0000 7f6575bad740 1 --2- 172.21.15.16:0/3664882164 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x55d83e195500 0x55d83e197920 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:26.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.725+0000 7f6575bad740 1 --2- 172.21.15.16:0/3664882164 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d83e116370 0x55d83e110440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:26.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.725+0000 7f6575bad740 1 --2- 172.21.15.16:0/3664882164 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d83e10cb40 0x55d83e10ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:26.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.725+0000 7f6575bad740 1 --2- 172.21.15.16:0/3664882164 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d83e10b5d0 0x55d83e10f9c0 unknown :-1 s=CLOSED pgs=1648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:26.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.725+0000 7f6575bad740 1 -- 172.21.15.16:0/3664882164 >> 172.21.15.16:0/3664882164 conn(0x55d83e0d4310 msgr2=0x55d83e121300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:26.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.725+0000 7f6554ff9640 0 client.19296 ms_handle_reset on v2:172.21.15.184:6838/4203347941 2024-09-12T02:13:26.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.725+0000 7f6575bad740 1 -- 172.21.15.16:0/3664882164 shutdown_connections 2024-09-12T02:13:26.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.725+0000 7f6575bad740 1 -- 172.21.15.16:0/3664882164 wait complete. 2024-09-12T02:13:26.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.725+0000 7f6575bad740 20 client.19296 trim_cache size 0 max 16384 2024-09-12T02:13:26.732 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T02:13:26.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.725+0000 7f65737d4640 1 -- 172.21.15.16:0/3959782365 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f65580a2c00 msgr2=0x7f65580a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:26.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.725+0000 7f65737d4640 1 --2- 172.21.15.16:0/3959782365 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f65580a2c00 0x7f65580a50c0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f65640b93f0 tx=0x7f65640dd040 comp rx=0 tx=0).stop 2024-09-12T02:13:26.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.725+0000 7f65737d4640 1 -- 172.21.15.16:0/3959782365 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f656c106a80 msgr2=0x7f656c19cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:26.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.725+0000 7f65737d4640 1 --2- 172.21.15.16:0/3959782365 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f656c106a80 0x7f656c19cd00 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7f6564005cc0 tx=0x7f6564003970 comp rx=0 tx=0).stop 2024-09-12T02:13:26.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.725+0000 7f65737d4640 1 -- 172.21.15.16:0/3959782365 shutdown_connections 2024-09-12T02:13:26.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.725+0000 7f65737d4640 1 --2- 172.21.15.16:0/3959782365 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f65580a2c00 0x7f65580a50c0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:26.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.725+0000 7f65737d4640 1 --2- 172.21.15.16:0/3959782365 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f656c106a80 0x7f656c19cd00 unknown :-1 s=CLOSED pgs=1644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:26.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.725+0000 7f65737d4640 1 --2- 172.21.15.16:0/3959782365 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f656c101420 0x7f656c198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:26.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.725+0000 7f65737d4640 1 --2- 172.21.15.16:0/3959782365 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f656c100a80 0x7f656c198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:26.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.725+0000 7f65737d4640 1 -- 172.21.15.16:0/3959782365 >> 172.21.15.16:0/3959782365 conn(0x7f656c0fc7a0 msgr2=0x7f656c06aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:26.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.726+0000 7f65737d4640 1 -- 172.21.15.16:0/3959782365 shutdown_connections 2024-09-12T02:13:26.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:26.726+0000 7f65737d4640 1 -- 172.21.15.16:0/3959782365 wait complete. 2024-09-12T02:13:27.403 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-12T02:13:27.658 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:13:28.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.062+0000 7fb68ebde640 1 -- 172.21.15.16:0/1739008464 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb688101420 msgr2=0x7fb68810d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:28.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.062+0000 7fb68ebde640 1 --2- 172.21.15.16:0/1739008464 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb688101420 0x7fb68810d950 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7fb680009cd0 tx=0x7fb68002a8e0 comp rx=0 tx=0).stop 2024-09-12T02:13:28.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.062+0000 7fb68ebde640 1 -- 172.21.15.16:0/1739008464 shutdown_connections 2024-09-12T02:13:28.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.062+0000 7fb68ebde640 1 --2- 172.21.15.16:0/1739008464 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb688101420 0x7fb68810d950 unknown :-1 s=CLOSED pgs=1649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:28.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.062+0000 7fb68ebde640 1 --2- 172.21.15.16:0/1739008464 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb688100a80 0x7fb688100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:28.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.062+0000 7fb68ebde640 1 --2- 172.21.15.16:0/1739008464 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb688106a80 0x7fb688106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:28.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.062+0000 7fb68ebde640 1 -- 172.21.15.16:0/1739008464 >> 172.21.15.16:0/1739008464 conn(0x7fb6880fc7a0 msgr2=0x7fb6880febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:28.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.063+0000 7fb68ebde640 1 -- 172.21.15.16:0/1739008464 shutdown_connections 2024-09-12T02:13:28.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.063+0000 7fb68ebde640 1 -- 172.21.15.16:0/1739008464 wait complete. 2024-09-12T02:13:28.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.063+0000 7fb68ebde640 1 Processor -- start 2024-09-12T02:13:28.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.063+0000 7fb68ebde640 1 -- start start 2024-09-12T02:13:28.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.063+0000 7fb68ebde640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb688100a80 0x7fb688198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:28.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.063+0000 7fb68ebde640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb688101420 0x7fb6881988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:28.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.064+0000 7fb68ebde640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb688106a80 0x7fb68819cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:28.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.064+0000 7fb68ebde640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb688110e80 con 0x7fb688101420 2024-09-12T02:13:28.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.064+0000 7fb68ebde640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb688110d00 con 0x7fb688106a80 2024-09-12T02:13:28.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.064+0000 7fb68ebde640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb688111000 con 0x7fb688100a80 2024-09-12T02:13:28.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.064+0000 7fb687fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb688101420 0x7fb6881988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:28.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.064+0000 7fb687fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb688101420 0x7fb6881988d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39530/0 (socket says 172.21.15.16:39530) 2024-09-12T02:13:28.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.064+0000 7fb687fff640 1 -- 172.21.15.16:0/2543815971 learned_addr learned my addr 172.21.15.16:0/2543815971 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:13:28.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.064+0000 7fb68d17b640 1 --2- 172.21.15.16:0/2543815971 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb688106a80 0x7fb68819cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:28.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.064+0000 7fb68c97a640 1 --2- 172.21.15.16:0/2543815971 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb688100a80 0x7fb688198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:28.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.064+0000 7fb687fff640 1 -- 172.21.15.16:0/2543815971 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb688100a80 msgr2=0x7fb688198390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:28.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.064+0000 7fb687fff640 1 --2- 172.21.15.16:0/2543815971 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb688100a80 0x7fb688198390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:28.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.064+0000 7fb687fff640 1 -- 172.21.15.16:0/2543815971 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb688106a80 msgr2=0x7fb68819cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:28.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.064+0000 7fb687fff640 1 --2- 172.21.15.16:0/2543815971 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb688106a80 0x7fb68819cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:28.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.064+0000 7fb687fff640 1 -- 172.21.15.16:0/2543815971 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb68819d340 con 0x7fb688101420 2024-09-12T02:13:28.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.064+0000 7fb68d17b640 1 --2- 172.21.15.16:0/2543815971 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb688106a80 0x7fb68819cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:13:28.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.064+0000 7fb687fff640 1 --2- 172.21.15.16:0/2543815971 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb688101420 0x7fb6881988d0 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7fb67c00e9f0 tx=0x7fb67c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:28.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.064+0000 7fb685ffb640 1 -- 172.21.15.16:0/2543815971 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb67c00cd80 con 0x7fb688101420 2024-09-12T02:13:28.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.064+0000 7fb685ffb640 1 -- 172.21.15.16:0/2543815971 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb67c010910 con 0x7fb688101420 2024-09-12T02:13:28.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.064+0000 7fb68ebde640 1 -- 172.21.15.16:0/2543815971 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb68819d630 con 0x7fb688101420 2024-09-12T02:13:28.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.064+0000 7fb685ffb640 1 -- 172.21.15.16:0/2543815971 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb67c018990 con 0x7fb688101420 2024-09-12T02:13:28.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.065+0000 7fb68ebde640 1 -- 172.21.15.16:0/2543815971 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6881a4f10 con 0x7fb688101420 2024-09-12T02:13:28.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.067+0000 7fb685ffb640 1 -- 172.21.15.16:0/2543815971 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fb67c010ac0 con 0x7fb688101420 2024-09-12T02:13:28.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.067+0000 7fb690fb7740 1 Processor -- start 2024-09-12T02:13:28.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.067+0000 7fb690fb7740 1 -- start start 2024-09-12T02:13:28.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.067+0000 7fb690fb7740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fe06bbeb40 0x55fe06bbd090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:28.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.067+0000 7fb690fb7740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe06bbd5d0 0x55fe06bbda10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:28.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.067+0000 7fb690fb7740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fe06bc8370 0x55fe06bca760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:28.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.067+0000 7fb690fb7740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55fe06b232f0 con 0x55fe06bbd5d0 2024-09-12T02:13:28.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.067+0000 7fb690fb7740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55fe06acacf0 con 0x55fe06bc8370 2024-09-12T02:13:28.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.067+0000 7fb690fb7740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55fe06b1f2b0 con 0x55fe06bbeb40 2024-09-12T02:13:28.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.067+0000 7fb685ffb640 1 --2- 172.21.15.16:0/2543815971 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fb6700a2c00 0x7fb6700a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:28.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.068+0000 7fb68c97a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe06bbd5d0 0x55fe06bbda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:28.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.068+0000 7fb68c97a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe06bbd5d0 0x55fe06bbda10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39532/0 (socket says 172.21.15.16:39532) 2024-09-12T02:13:28.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.068+0000 7fb68c97a640 1 -- 172.21.15.16:0/1412844413 learned_addr learned my addr 172.21.15.16:0/1412844413 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:13:28.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.068+0000 7fb68d17b640 1 --2- 172.21.15.16:0/2543815971 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fb6700a2c00 0x7fb6700a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:28.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.068+0000 7fb687fff640 1 --2- 172.21.15.16:0/1412844413 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fe06bc8370 0x55fe06bca760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:28.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.068+0000 7fb68d17b640 1 --2- 172.21.15.16:0/1412844413 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fe06bbeb40 0x55fe06bbd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:28.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.068+0000 7fb685ffb640 1 -- 172.21.15.16:0/2543815971 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fb67c0cf370 con 0x7fb688101420 2024-09-12T02:13:28.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.068+0000 7fb68c97a640 1 -- 172.21.15.16:0/1412844413 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fe06bbeb40 msgr2=0x55fe06bbd090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:28.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.068+0000 7fb68c97a640 1 --2- 172.21.15.16:0/1412844413 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fe06bbeb40 0x55fe06bbd090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:28.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.068+0000 7fb68c97a640 1 -- 172.21.15.16:0/1412844413 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fe06bc8370 msgr2=0x55fe06bca760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:28.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.068+0000 7fb68c97a640 1 --2- 172.21.15.16:0/1412844413 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fe06bc8370 0x55fe06bca760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:28.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.068+0000 7fb68c97a640 1 -- 172.21.15.16:0/1412844413 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe06aaf3a0 con 0x55fe06bbd5d0 2024-09-12T02:13:28.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.068+0000 7fb687fff640 1 --2- 172.21.15.16:0/1412844413 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fe06bc8370 0x55fe06bca760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:13:28.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.068+0000 7fb68d17b640 1 --2- 172.21.15.16:0/1412844413 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fe06bbeb40 0x55fe06bbd090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:13:28.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.068+0000 7fb68c97a640 1 --2- 172.21.15.16:0/1412844413 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe06bbd5d0 0x55fe06bbda10 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7fb67800cbe0 tx=0x7fb67800e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=841bf58b7ebdafe1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:28.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.068+0000 7fb68d17b640 1 --2- 172.21.15.16:0/2543815971 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fb6700a2c00 0x7fb6700a50c0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fb680005fe0 tx=0x7fb680005d80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:28.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.068+0000 7fb66effd640 1 -- 172.21.15.16:0/1412844413 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb678005220 con 0x55fe06bbd5d0 2024-09-12T02:13:28.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.068+0000 7fb66effd640 1 -- 172.21.15.16:0/1412844413 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb678004c80 con 0x55fe06bbd5d0 2024-09-12T02:13:28.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.068+0000 7fb66effd640 1 -- 172.21.15.16:0/1412844413 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb678010870 con 0x55fe06bbd5d0 2024-09-12T02:13:28.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.069+0000 7fb690fb7740 1 -- 172.21.15.16:0/1412844413 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe06bbd5d0 msgr2=0x55fe06bbda10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:28.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.069+0000 7fb690fb7740 1 --2- 172.21.15.16:0/1412844413 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe06bbd5d0 0x55fe06bbda10 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7fb67800cbe0 tx=0x7fb67800e720 comp rx=0 tx=0).stop 2024-09-12T02:13:28.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.069+0000 7fb690fb7740 1 -- 172.21.15.16:0/1412844413 shutdown_connections 2024-09-12T02:13:28.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.069+0000 7fb690fb7740 1 --2- 172.21.15.16:0/1412844413 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fe06bc8370 0x55fe06bca760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:28.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.069+0000 7fb690fb7740 1 --2- 172.21.15.16:0/1412844413 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe06bbd5d0 0x55fe06bbda10 unknown :-1 s=CLOSED pgs=1651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:28.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.069+0000 7fb690fb7740 1 --2- 172.21.15.16:0/1412844413 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fe06bbeb40 0x55fe06bbd090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:28.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.069+0000 7fb690fb7740 1 -- 172.21.15.16:0/1412844413 >> 172.21.15.16:0/1412844413 conn(0x55fe06b86310 msgr2=0x55fe06bb9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:28.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.069+0000 7fb690fb7740 1 -- 172.21.15.16:0/1412844413 shutdown_connections 2024-09-12T02:13:28.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.069+0000 7fb690fb7740 1 -- 172.21.15.16:0/1412844413 wait complete. 2024-09-12T02:13:28.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.069+0000 7fb690fb7740 1 -- start start 2024-09-12T02:13:28.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.070+0000 7fb690fb7740 1 Processor -- start 2024-09-12T02:13:28.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.070+0000 7fb690fb7740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fe06bbd5d0 0x55fe06bc7ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:28.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.070+0000 7fb690fb7740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fe06bbeb40 0x55fe06bc4f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:28.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.070+0000 7fb690fb7740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe06bc8370 0x55fe06bc54f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:28.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.070+0000 7fb690fb7740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55fe06bcb070 con 0x55fe06bc8370 2024-09-12T02:13:28.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.070+0000 7fb690fb7740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55fe06ade060 con 0x55fe06bbeb40 2024-09-12T02:13:28.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.070+0000 7fb690fb7740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55fe06bcb1f0 con 0x55fe06bbd5d0 2024-09-12T02:13:28.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.070+0000 7fb690fb7740 10 client.0 _ll_register_callbacks cb 0x55fe068318c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:13:28.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.070+0000 7fb68d17b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe06bc8370 0x55fe06bc54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:28.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.070+0000 7fb68d17b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe06bc8370 0x55fe06bc54f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39558/0 (socket says 172.21.15.16:39558) 2024-09-12T02:13:28.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.070+0000 7fb68d17b640 1 -- 172.21.15.16:0/2433789008 learned_addr learned my addr 172.21.15.16:0/2433789008 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:13:28.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.070+0000 7fb68c97a640 1 --2- 172.21.15.16:0/2433789008 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fe06bbd5d0 0x55fe06bc7ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:28.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.070+0000 7fb687fff640 1 --2- 172.21.15.16:0/2433789008 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fe06bbeb40 0x55fe06bc4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:28.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.071+0000 7fb68d17b640 1 -- 172.21.15.16:0/2433789008 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fe06bbd5d0 msgr2=0x55fe06bc7ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:28.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.071+0000 7fb68d17b640 1 --2- 172.21.15.16:0/2433789008 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fe06bbd5d0 0x55fe06bc7ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:28.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.071+0000 7fb68d17b640 1 -- 172.21.15.16:0/2433789008 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fe06bbeb40 msgr2=0x55fe06bc4f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:28.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.071+0000 7fb68d17b640 1 --2- 172.21.15.16:0/2433789008 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fe06bbeb40 0x55fe06bc4f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:28.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.071+0000 7fb68d17b640 1 -- 172.21.15.16:0/2433789008 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe068943c0 con 0x55fe06bc8370 2024-09-12T02:13:28.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.071+0000 7fb68d17b640 1 --2- 172.21.15.16:0/2433789008 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe06bc8370 0x55fe06bc54f0 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7fb680035000 tx=0x7fb68002a8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d32672e8b5eddb6e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:28.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.071+0000 7fb667fff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:13:28.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.071+0000 7fb667fff640 1 -- 172.21.15.16:0/2433789008 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb680004450 con 0x55fe06bc8370 2024-09-12T02:13:28.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.071+0000 7fb667fff640 1 -- 172.21.15.16:0/2433789008 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fb6800045f0 con 0x55fe06bc8370 2024-09-12T02:13:28.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.071+0000 7fb690fb7740 1 -- 172.21.15.16:0/2433789008 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb678004c80 con 0x55fe06bc8370 2024-09-12T02:13:28.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.071+0000 7fb667fff640 1 -- 172.21.15.16:0/2433789008 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fb6800047f0 con 0x55fe06bc8370 2024-09-12T02:13:28.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.071+0000 7fb690fb7740 10 client.19308 fetch_fsmap learned FSMap version 25 2024-09-12T02:13:28.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.071+0000 7fb690fb7740 1 -- 172.21.15.16:0/2433789008 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55fe06bd6a80 con 0x55fe06bc8370 2024-09-12T02:13:28.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.071+0000 7fb667fff640 1 -- 172.21.15.16:0/2433789008 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb678004c80 con 0x55fe06bc8370 2024-09-12T02:13:28.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.071+0000 7fb667fff640 1 -- 172.21.15.16:0/2433789008 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fb680030070 con 0x55fe06bc8370 2024-09-12T02:13:28.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.072+0000 7fb690fb7740 10 client.19308 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:13:28.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.072+0000 7fb690fb7740 10 client.19308 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:13:28.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.072+0000 7fb690fb7740 20 client.19308 populate_metadata read hostname 'smithi016' 2024-09-12T02:13:28.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.072+0000 7fb690fb7740 1 --2- 172.21.15.16:0/2433789008 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55fe06bd7f40 0x55fe06bda3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:28.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.072+0000 7fb690fb7740 4 client.19308 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:13:28.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.072+0000 7fb690fb7740 1 -- 172.21.15.16:0/2433789008 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fe06b30860 con 0x55fe06bd7f40 2024-09-12T02:13:28.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.072+0000 7fb68c97a640 1 --2- 172.21.15.16:0/2433789008 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55fe06bd7f40 0x55fe06bda3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:28.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.072+0000 7fb667fff640 10 client.19308 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:13:28.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.072+0000 7fb68c97a640 1 --2- 172.21.15.16:0/2433789008 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55fe06bd7f40 0x55fe06bda3b0 crc :-1 s=READY pgs=350 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f0565e8aab5bfc32 server_cookie=f9e9eb2170f3eeab in_seq=0 out_seq=0 2024-09-12T02:13:28.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.073+0000 7fb667fff640 1 -- 172.21.15.16:0/2433789008 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fe06b30860 con 0x55fe06bd7f40 2024-09-12T02:13:28.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.073+0000 7fb667fff640 10 client.19308 handle_command_reply: tid=0 2024-09-12T02:13:28.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.073+0000 7fb690fb7740 1 client.19308 shutdown 2024-09-12T02:13:28.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.073+0000 7fb690fb7740 1 -- 172.21.15.16:0/2433789008 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe06bc8370 msgr2=0x55fe06bc54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:28.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.073+0000 7fb690fb7740 1 --2- 172.21.15.16:0/2433789008 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe06bc8370 0x55fe06bc54f0 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7fb680035000 tx=0x7fb68002a8e0 comp rx=0 tx=0).stop 2024-09-12T02:13:28.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.073+0000 7fb690fb7740 1 -- 172.21.15.16:0/2433789008 shutdown_connections 2024-09-12T02:13:28.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.073+0000 7fb690fb7740 1 --2- 172.21.15.16:0/2433789008 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55fe06bd7f40 0x55fe06bda3b0 crc :-1 s=READY pgs=350 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:28.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.073+0000 7fb690fb7740 1 --2- 172.21.15.16:0/2433789008 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe06bc8370 0x55fe06bc54f0 unknown :-1 s=CLOSED pgs=1652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:28.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.073+0000 7fb690fb7740 1 --2- 172.21.15.16:0/2433789008 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fe06bbeb40 0x55fe06bc4f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:28.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.073+0000 7fb690fb7740 1 --2- 172.21.15.16:0/2433789008 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fe06bbd5d0 0x55fe06bc7ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:28.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.073+0000 7fb667fff640 0 client.19308 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:13:28.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.073+0000 7fb690fb7740 1 -- 172.21.15.16:0/2433789008 >> 172.21.15.16:0/2433789008 conn(0x55fe06b86310 msgr2=0x55fe06bbf960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:28.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.073+0000 7fb690fb7740 1 -- 172.21.15.16:0/2433789008 shutdown_connections 2024-09-12T02:13:28.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.073+0000 7fb690fb7740 1 -- 172.21.15.16:0/2433789008 wait complete. 2024-09-12T02:13:28.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.073+0000 7fb690fb7740 20 client.19308 trim_cache size 0 max 16384 2024-09-12T02:13:28.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.087+0000 7fb690fb7740 1 Processor -- start 2024-09-12T02:13:28.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.087+0000 7fb690fb7740 1 -- start start 2024-09-12T02:13:28.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.087+0000 7fb690fb7740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe06bbd5d0 0x55fe06bb9e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:28.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.087+0000 7fb690fb7740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fe06bbeb40 0x55fe06bd4310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:28.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.087+0000 7fb690fb7740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fe06bc8370 0x55fe06bdfcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:28.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.087+0000 7fb690fb7740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb6881afcb0 con 0x55fe06bbd5d0 2024-09-12T02:13:28.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.087+0000 7fb690fb7740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb6881ad580 con 0x55fe06bc8370 2024-09-12T02:13:28.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.087+0000 7fb690fb7740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb6881ab3d0 con 0x55fe06bbeb40 2024-09-12T02:13:28.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.087+0000 7fb68c97a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe06bbd5d0 0x55fe06bb9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:28.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.087+0000 7fb68c97a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe06bbd5d0 0x55fe06bb9e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39562/0 (socket says 172.21.15.16:39562) 2024-09-12T02:13:28.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.087+0000 7fb68c97a640 1 -- 172.21.15.16:0/2074674917 learned_addr learned my addr 172.21.15.16:0/2074674917 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:13:28.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.087+0000 7fb687fff640 1 --2- 172.21.15.16:0/2074674917 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fe06bbeb40 0x55fe06bd4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:28.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.087+0000 7fb68d17b640 1 --2- 172.21.15.16:0/2074674917 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fe06bc8370 0x55fe06bdfcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:28.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.087+0000 7fb68c97a640 1 -- 172.21.15.16:0/2074674917 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fe06bbeb40 msgr2=0x55fe06bd4310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:28.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.087+0000 7fb68c97a640 1 --2- 172.21.15.16:0/2074674917 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fe06bbeb40 0x55fe06bd4310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:28.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.087+0000 7fb68c97a640 1 -- 172.21.15.16:0/2074674917 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fe06bc8370 msgr2=0x55fe06bdfcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:28.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.087+0000 7fb68c97a640 1 --2- 172.21.15.16:0/2074674917 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fe06bc8370 0x55fe06bdfcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:28.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.087+0000 7fb68c97a640 1 -- 172.21.15.16:0/2074674917 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe06bb94f0 con 0x55fe06bbd5d0 2024-09-12T02:13:28.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.088+0000 7fb68c97a640 1 --2- 172.21.15.16:0/2074674917 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe06bbd5d0 0x55fe06bb9e10 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7fb67800cb00 tx=0x7fb67800c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=858f3f76e8e7642 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:28.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.088+0000 7fb66e7fc640 1 -- 172.21.15.16:0/2074674917 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6780052d0 con 0x55fe06bbd5d0 2024-09-12T02:13:28.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.088+0000 7fb66e7fc640 1 -- 172.21.15.16:0/2074674917 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb678005470 con 0x55fe06bbd5d0 2024-09-12T02:13:28.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.088+0000 7fb66e7fc640 1 -- 172.21.15.16:0/2074674917 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6780109b0 con 0x55fe06bbd5d0 2024-09-12T02:13:28.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.088+0000 7fb690fb7740 1 -- 172.21.15.16:0/2074674917 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe06bbd5d0 msgr2=0x55fe06bb9e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:28.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.088+0000 7fb690fb7740 1 --2- 172.21.15.16:0/2074674917 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe06bbd5d0 0x55fe06bb9e10 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7fb67800cb00 tx=0x7fb67800c4d0 comp rx=0 tx=0).stop 2024-09-12T02:13:28.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.089+0000 7fb690fb7740 1 -- 172.21.15.16:0/2074674917 shutdown_connections 2024-09-12T02:13:28.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.089+0000 7fb690fb7740 1 --2- 172.21.15.16:0/2074674917 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fe06bc8370 0x55fe06bdfcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:28.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.089+0000 7fb690fb7740 1 --2- 172.21.15.16:0/2074674917 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fe06bbeb40 0x55fe06bd4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:28.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.089+0000 7fb690fb7740 1 --2- 172.21.15.16:0/2074674917 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe06bbd5d0 0x55fe06bb9e10 unknown :-1 s=CLOSED pgs=1653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:28.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.089+0000 7fb690fb7740 1 -- 172.21.15.16:0/2074674917 >> 172.21.15.16:0/2074674917 conn(0x55fe06b86310 msgr2=0x55fe06bb9700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:28.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.089+0000 7fb690fb7740 1 -- 172.21.15.16:0/2074674917 shutdown_connections 2024-09-12T02:13:28.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.089+0000 7fb690fb7740 1 -- 172.21.15.16:0/2074674917 wait complete. 2024-09-12T02:13:28.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.089+0000 7fb690fb7740 1 -- start start 2024-09-12T02:13:28.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.089+0000 7fb690fb7740 1 Processor -- start 2024-09-12T02:13:28.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.089+0000 7fb690fb7740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe06bbd5d0 0x55fe06bd09b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:28.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.090+0000 7fb690fb7740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fe06bbeb40 0x55fe06bc1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:28.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.090+0000 7fb690fb7740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fe06bc8370 0x55fe06bc1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:28.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.090+0000 7fb690fb7740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb6881aba40 con 0x55fe06bbd5d0 2024-09-12T02:13:28.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.090+0000 7fb690fb7740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb6881aceb0 con 0x55fe06bbeb40 2024-09-12T02:13:28.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.090+0000 7fb690fb7740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb6881ac110 con 0x55fe06bc8370 2024-09-12T02:13:28.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.090+0000 7fb690fb7740 10 client.0 _ll_register_callbacks cb 0x7fb67c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:13:28.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.090+0000 7fb68c97a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe06bbd5d0 0x55fe06bd09b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:28.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.090+0000 7fb68c97a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe06bbd5d0 0x55fe06bd09b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39568/0 (socket says 172.21.15.16:39568) 2024-09-12T02:13:28.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.090+0000 7fb68c97a640 1 -- 172.21.15.16:0/3909497340 learned_addr learned my addr 172.21.15.16:0/3909497340 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:13:28.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.090+0000 7fb687fff640 1 --2- 172.21.15.16:0/3909497340 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fe06bbeb40 0x55fe06bc1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:28.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.090+0000 7fb68d17b640 1 --2- 172.21.15.16:0/3909497340 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fe06bc8370 0x55fe06bc1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:28.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.090+0000 7fb68c97a640 1 -- 172.21.15.16:0/3909497340 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fe06bc8370 msgr2=0x55fe06bc1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:28.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.090+0000 7fb68c97a640 1 --2- 172.21.15.16:0/3909497340 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fe06bc8370 0x55fe06bc1ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:28.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.090+0000 7fb68c97a640 1 -- 172.21.15.16:0/3909497340 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fe06bbeb40 msgr2=0x55fe06bc1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:28.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.090+0000 7fb68c97a640 1 --2- 172.21.15.16:0/3909497340 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fe06bbeb40 0x55fe06bc1ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:28.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.090+0000 7fb68c97a640 1 -- 172.21.15.16:0/3909497340 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe06bc25e0 con 0x55fe06bbd5d0 2024-09-12T02:13:28.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.090+0000 7fb68c97a640 1 --2- 172.21.15.16:0/3909497340 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe06bbd5d0 0x55fe06bd09b0 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7fb67800c730 tx=0x7fb67800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c13ea6a2eb64a92e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:28.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.090+0000 7fb667fff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:13:28.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.090+0000 7fb667fff640 1 -- 172.21.15.16:0/3909497340 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6780035f0 con 0x55fe06bbd5d0 2024-09-12T02:13:28.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.090+0000 7fb667fff640 1 -- 172.21.15.16:0/3909497340 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fb678003790 con 0x55fe06bbd5d0 2024-09-12T02:13:28.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.090+0000 7fb690fb7740 1 -- 172.21.15.16:0/3909497340 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb678005470 con 0x55fe06bbd5d0 2024-09-12T02:13:28.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.091+0000 7fb667fff640 1 -- 172.21.15.16:0/3909497340 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fb678003990 con 0x55fe06bbd5d0 2024-09-12T02:13:28.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.091+0000 7fb690fb7740 10 client.19314 fetch_fsmap learned FSMap version 25 2024-09-12T02:13:28.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.091+0000 7fb690fb7740 1 -- 172.21.15.16:0/3909497340 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55fe06c42a80 con 0x55fe06bbd5d0 2024-09-12T02:13:28.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.091+0000 7fb667fff640 1 -- 172.21.15.16:0/3909497340 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb678005470 con 0x55fe06bbd5d0 2024-09-12T02:13:28.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.091+0000 7fb667fff640 1 -- 172.21.15.16:0/3909497340 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fb678026460 con 0x55fe06bbd5d0 2024-09-12T02:13:28.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.091+0000 7fb690fb7740 10 client.19314 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:13:28.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.091+0000 7fb690fb7740 10 client.19314 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:13:28.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.091+0000 7fb690fb7740 20 client.19314 populate_metadata read hostname 'smithi016' 2024-09-12T02:13:28.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.091+0000 7fb690fb7740 1 --2- 172.21.15.16:0/3909497340 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55fe06c42fa0 0x55fe06c45390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:28.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.091+0000 7fb690fb7740 4 client.19314 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T02:13:28.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.091+0000 7fb690fb7740 1 -- 172.21.15.16:0/3909497340 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fe06b2ad70 con 0x55fe06c42fa0 2024-09-12T02:13:28.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.091+0000 7fb687fff640 1 --2- 172.21.15.16:0/3909497340 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55fe06c42fa0 0x55fe06c45390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:28.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.092+0000 7fb667fff640 10 client.19314 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:13:28.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.092+0000 7fb687fff640 1 --2- 172.21.15.16:0/3909497340 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55fe06c42fa0 0x55fe06c45390 crc :-1 s=READY pgs=351 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c481bcbd3122e1a server_cookie=f6fd964e293b6dae in_seq=0 out_seq=0 2024-09-12T02:13:28.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.092+0000 7fb667fff640 1 -- 172.21.15.16:0/3909497340 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fe06b2ad70 con 0x55fe06c42fa0 2024-09-12T02:13:28.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.092+0000 7fb667fff640 10 client.19314 handle_command_reply: tid=0 2024-09-12T02:13:28.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.092+0000 7fb690fb7740 1 client.19314 shutdown 2024-09-12T02:13:28.122 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T02:13:28.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.092+0000 7fb690fb7740 1 -- 172.21.15.16:0/3909497340 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe06bbd5d0 msgr2=0x55fe06bd09b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:28.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.092+0000 7fb690fb7740 1 --2- 172.21.15.16:0/3909497340 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe06bbd5d0 0x55fe06bd09b0 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7fb67800c730 tx=0x7fb67800c860 comp rx=0 tx=0).stop 2024-09-12T02:13:28.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.092+0000 7fb690fb7740 1 -- 172.21.15.16:0/3909497340 shutdown_connections 2024-09-12T02:13:28.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.092+0000 7fb690fb7740 1 --2- 172.21.15.16:0/3909497340 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55fe06c42fa0 0x55fe06c45390 crc :-1 s=READY pgs=351 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:28.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.092+0000 7fb690fb7740 1 --2- 172.21.15.16:0/3909497340 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fe06bc8370 0x55fe06bc1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:28.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.092+0000 7fb690fb7740 1 --2- 172.21.15.16:0/3909497340 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fe06bbeb40 0x55fe06bc1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:28.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.092+0000 7fb690fb7740 1 --2- 172.21.15.16:0/3909497340 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe06bbd5d0 0x55fe06bd09b0 unknown :-1 s=CLOSED pgs=1654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:28.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.092+0000 7fb690fb7740 1 -- 172.21.15.16:0/3909497340 >> 172.21.15.16:0/3909497340 conn(0x55fe06b86310 msgr2=0x55fe06bb8d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:28.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.092+0000 7fb667fff640 0 client.19314 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:13:28.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.092+0000 7fb690fb7740 1 -- 172.21.15.16:0/3909497340 shutdown_connections 2024-09-12T02:13:28.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.092+0000 7fb690fb7740 1 -- 172.21.15.16:0/3909497340 wait complete. 2024-09-12T02:13:28.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.092+0000 7fb690fb7740 20 client.19314 trim_cache size 0 max 16384 2024-09-12T02:13:28.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.093+0000 7fb68ebde640 1 -- 172.21.15.16:0/2543815971 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fb6700a2c00 msgr2=0x7fb6700a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:28.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.093+0000 7fb68ebde640 1 --2- 172.21.15.16:0/2543815971 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fb6700a2c00 0x7fb6700a50c0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fb680005fe0 tx=0x7fb680005d80 comp rx=0 tx=0).stop 2024-09-12T02:13:28.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.093+0000 7fb68ebde640 1 -- 172.21.15.16:0/2543815971 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb688101420 msgr2=0x7fb6881988d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:28.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.093+0000 7fb68ebde640 1 --2- 172.21.15.16:0/2543815971 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb688101420 0x7fb6881988d0 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7fb67c00e9f0 tx=0x7fb67c00eeb0 comp rx=0 tx=0).stop 2024-09-12T02:13:28.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.093+0000 7fb68ebde640 1 -- 172.21.15.16:0/2543815971 shutdown_connections 2024-09-12T02:13:28.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.093+0000 7fb68ebde640 1 --2- 172.21.15.16:0/2543815971 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fb6700a2c00 0x7fb6700a50c0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:28.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.093+0000 7fb68ebde640 1 --2- 172.21.15.16:0/2543815971 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb688106a80 0x7fb68819cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:28.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.093+0000 7fb68ebde640 1 --2- 172.21.15.16:0/2543815971 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb688101420 0x7fb6881988d0 unknown :-1 s=CLOSED pgs=1650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:28.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.093+0000 7fb68ebde640 1 --2- 172.21.15.16:0/2543815971 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb688100a80 0x7fb688198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:28.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.093+0000 7fb68ebde640 1 -- 172.21.15.16:0/2543815971 >> 172.21.15.16:0/2543815971 conn(0x7fb6880fc7a0 msgr2=0x7fb68806a7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:28.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.093+0000 7fb68ebde640 1 -- 172.21.15.16:0/2543815971 shutdown_connections 2024-09-12T02:13:28.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:28.093+0000 7fb68ebde640 1 -- 172.21.15.16:0/2543815971 wait complete. 2024-09-12T02:13:28.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:13:28 smithi184 ceph-mon[35593]: pgmap v681: 97 pgs: 97 active+clean; 2.1 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 18 op/s 2024-09-12T02:13:28.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:13:28 smithi033 ceph-mon[35332]: pgmap v681: 97 pgs: 97 active+clean; 2.1 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 18 op/s 2024-09-12T02:13:28.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:13:28 smithi016 ceph-mon[30947]: pgmap v681: 97 pgs: 97 active+clean; 2.1 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 18 op/s 2024-09-12T02:13:29.883 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-12T02:13:29.883 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-12T02:13:30.163 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:13:30.183 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:13:30 smithi016 ceph-mon[30947]: pgmap v682: 97 pgs: 97 active+clean; 2.1 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-12T02:13:30.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:13:30 smithi184 ceph-mon[35593]: pgmap v682: 97 pgs: 97 active+clean; 2.1 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-12T02:13:30.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:13:30 smithi033 ceph-mon[35332]: pgmap v682: 97 pgs: 97 active+clean; 2.1 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-12T02:13:30.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.584+0000 7f87e67c1640 1 -- 172.21.15.16:0/3786506057 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87e01036e0 msgr2=0x7f87e0107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:30.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.584+0000 7f87e67c1640 1 --2- 172.21.15.16:0/3786506057 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87e01036e0 0x7f87e0107b90 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7f87c8009930 tx=0x7f87c802a420 comp rx=0 tx=0).stop 2024-09-12T02:13:30.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.584+0000 7f87e67c1640 1 -- 172.21.15.16:0/3786506057 shutdown_connections 2024-09-12T02:13:30.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.584+0000 7f87e67c1640 1 --2- 172.21.15.16:0/3786506057 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f87e01082a0 0x7f87e010ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:30.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.584+0000 7f87e67c1640 1 --2- 172.21.15.16:0/3786506057 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87e01036e0 0x7f87e0107b90 unknown :-1 s=CLOSED pgs=1655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:30.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.584+0000 7f87e67c1640 1 --2- 172.21.15.16:0/3786506057 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87e0102d30 0x7f87e0103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:30.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.584+0000 7f87e67c1640 1 -- 172.21.15.16:0/3786506057 >> 172.21.15.16:0/3786506057 conn(0x7f87e00fc7d0 msgr2=0x7f87e00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:30.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.584+0000 7f87e67c1640 1 -- 172.21.15.16:0/3786506057 shutdown_connections 2024-09-12T02:13:30.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.584+0000 7f87e67c1640 1 -- 172.21.15.16:0/3786506057 wait complete. 2024-09-12T02:13:30.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.584+0000 7f87e67c1640 1 Processor -- start 2024-09-12T02:13:30.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.585+0000 7f87e67c1640 1 -- start start 2024-09-12T02:13:30.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.585+0000 7f87e67c1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87e0102d30 0x7f87e0100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:30.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.585+0000 7f87e67c1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87e01036e0 0x7f87e0100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:30.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.585+0000 7f87e67c1640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f87e01082a0 0x7f87e0101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:30.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.585+0000 7f87e67c1640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f87e01120a0 con 0x7f87e01036e0 2024-09-12T02:13:30.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.585+0000 7f87e67c1640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f87e0111f20 con 0x7f87e0102d30 2024-09-12T02:13:30.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.585+0000 7f87e67c1640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f87e0112220 con 0x7f87e01082a0 2024-09-12T02:13:30.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.585+0000 7f87d7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87e01036e0 0x7f87e0100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:30.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.585+0000 7f87d7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87e01036e0 0x7f87e0100ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39582/0 (socket says 172.21.15.16:39582) 2024-09-12T02:13:30.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.585+0000 7f87d7fff640 1 -- 172.21.15.16:0/2465498646 learned_addr learned my addr 172.21.15.16:0/2465498646 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:13:30.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.585+0000 7f87dffff640 1 --2- 172.21.15.16:0/2465498646 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87e0102d30 0x7f87e0100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:30.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.585+0000 7f87e4d5e640 1 --2- 172.21.15.16:0/2465498646 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f87e01082a0 0x7f87e0101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:30.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.585+0000 7f87d7fff640 1 -- 172.21.15.16:0/2465498646 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f87e01082a0 msgr2=0x7f87e0101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:30.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.585+0000 7f87d7fff640 1 --2- 172.21.15.16:0/2465498646 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f87e01082a0 0x7f87e0101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:30.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.585+0000 7f87d7fff640 1 -- 172.21.15.16:0/2465498646 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87e0102d30 msgr2=0x7f87e0100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:30.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.585+0000 7f87d7fff640 1 --2- 172.21.15.16:0/2465498646 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87e0102d30 0x7f87e0100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:30.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.585+0000 7f87d7fff640 1 -- 172.21.15.16:0/2465498646 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87e01a6a40 con 0x7f87e01036e0 2024-09-12T02:13:30.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.585+0000 7f87d7fff640 1 --2- 172.21.15.16:0/2465498646 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87e01036e0 0x7f87e0100ed0 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7f87c8005940 tx=0x7f87c80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:30.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.586+0000 7f87ddffb640 1 -- 172.21.15.16:0/2465498646 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87c803f030 con 0x7f87e01036e0 2024-09-12T02:13:30.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.586+0000 7f87ddffb640 1 -- 172.21.15.16:0/2465498646 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87c8003ce0 con 0x7f87e01036e0 2024-09-12T02:13:30.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.586+0000 7f87e67c1640 1 -- 172.21.15.16:0/2465498646 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87e01a6cd0 con 0x7f87e01036e0 2024-09-12T02:13:30.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.586+0000 7f87ddffb640 1 -- 172.21.15.16:0/2465498646 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87c8035070 con 0x7f87e01036e0 2024-09-12T02:13:30.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.586+0000 7f87e67c1640 1 -- 172.21.15.16:0/2465498646 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87e01a71b0 con 0x7f87e01036e0 2024-09-12T02:13:30.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.588+0000 7f87ddffb640 1 -- 172.21.15.16:0/2465498646 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f87c8042030 con 0x7f87e01036e0 2024-09-12T02:13:30.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.588+0000 7f87e8b9a740 1 Processor -- start 2024-09-12T02:13:30.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.588+0000 7f87e8b9a740 1 -- start start 2024-09-12T02:13:30.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.589+0000 7f87e8b9a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56137ec5bb60 0x56137ec5bf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:30.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.589+0000 7f87e8b9a740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56137ec5c480 0x56137ec66800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:30.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.589+0000 7f87e8b9a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56137ec66d40 0x56137ec69130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:30.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.589+0000 7f87e8b9a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56137ebc1280 con 0x56137ec5bb60 2024-09-12T02:13:30.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.589+0000 7f87e8b9a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56137eb68b10 con 0x56137ec66d40 2024-09-12T02:13:30.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.589+0000 7f87e8b9a740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x56137ebbd240 con 0x56137ec5c480 2024-09-12T02:13:30.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.589+0000 7f87e4d5e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56137ec5bb60 0x56137ec5bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:30.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.589+0000 7f87e4d5e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56137ec5bb60 0x56137ec5bf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39598/0 (socket says 172.21.15.16:39598) 2024-09-12T02:13:30.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.589+0000 7f87e4d5e640 1 -- 172.21.15.16:0/705150679 learned_addr learned my addr 172.21.15.16:0/705150679 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:13:30.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.589+0000 7f87ddffb640 1 --2- 172.21.15.16:0/2465498646 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f87cc0a2b80 0x7f87cc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:30.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.589+0000 7f87d7fff640 1 --2- 172.21.15.16:0/705150679 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56137ec66d40 0x56137ec69130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:30.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.589+0000 7f87dffff640 1 --2- 172.21.15.16:0/705150679 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56137ec5c480 0x56137ec66800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:30.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.589+0000 7f87e4d5e640 1 -- 172.21.15.16:0/705150679 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56137ec5c480 msgr2=0x56137ec66800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:30.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.589+0000 7f87e4d5e640 1 --2- 172.21.15.16:0/705150679 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56137ec5c480 0x56137ec66800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:30.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.589+0000 7f87e4d5e640 1 -- 172.21.15.16:0/705150679 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56137ec66d40 msgr2=0x56137ec69130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:30.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.589+0000 7f87e4d5e640 1 --2- 172.21.15.16:0/705150679 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56137ec66d40 0x56137ec69130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:30.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.589+0000 7f87ddffb640 1 -- 172.21.15.16:0/2465498646 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f87c80ef4f0 con 0x7f87e01036e0 2024-09-12T02:13:30.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.589+0000 7f87e4d5e640 1 -- 172.21.15.16:0/705150679 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56137eb751f0 con 0x56137ec5bb60 2024-09-12T02:13:30.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.589+0000 7f87e4d5e640 1 --2- 172.21.15.16:0/2465498646 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f87cc0a2b80 0x7f87cc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:30.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.590+0000 7f87e4d5e640 1 --2- 172.21.15.16:0/705150679 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56137ec5bb60 0x56137ec5bf40 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7f87d8007f50 tx=0x7f87d800e7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=279fe8d4c85eb6de server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:30.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.590+0000 7f87d67fc640 1 -- 172.21.15.16:0/705150679 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87d8022070 con 0x56137ec5bb60 2024-09-12T02:13:30.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.590+0000 7f87d67fc640 1 -- 172.21.15.16:0/705150679 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87d8011b70 con 0x56137ec5bb60 2024-09-12T02:13:30.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.590+0000 7f87d67fc640 1 -- 172.21.15.16:0/705150679 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87d801a400 con 0x56137ec5bb60 2024-09-12T02:13:30.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.590+0000 7f87e4d5e640 1 --2- 172.21.15.16:0/2465498646 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f87cc0a2b80 0x7f87cc0a5040 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f87d801f000 tx=0x7f87d8020000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:30.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.591+0000 7f87e8b9a740 1 -- 172.21.15.16:0/705150679 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56137ec5bb60 msgr2=0x56137ec5bf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:30.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.591+0000 7f87e8b9a740 1 --2- 172.21.15.16:0/705150679 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56137ec5bb60 0x56137ec5bf40 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7f87d8007f50 tx=0x7f87d800e7e0 comp rx=0 tx=0).stop 2024-09-12T02:13:30.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.591+0000 7f87e8b9a740 1 -- 172.21.15.16:0/705150679 shutdown_connections 2024-09-12T02:13:30.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.591+0000 7f87e8b9a740 1 --2- 172.21.15.16:0/705150679 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56137ec66d40 0x56137ec69130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:30.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.591+0000 7f87e8b9a740 1 --2- 172.21.15.16:0/705150679 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56137ec5c480 0x56137ec66800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:30.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.591+0000 7f87e8b9a740 1 --2- 172.21.15.16:0/705150679 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56137ec5bb60 0x56137ec5bf40 unknown :-1 s=CLOSED pgs=1657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:30.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.591+0000 7f87e8b9a740 1 -- 172.21.15.16:0/705150679 >> 172.21.15.16:0/705150679 conn(0x56137ec580f0 msgr2=0x56137ec584d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:30.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.591+0000 7f87e8b9a740 1 -- 172.21.15.16:0/705150679 shutdown_connections 2024-09-12T02:13:30.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.591+0000 7f87e8b9a740 1 -- 172.21.15.16:0/705150679 wait complete. 2024-09-12T02:13:30.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.591+0000 7f87e8b9a740 1 -- start start 2024-09-12T02:13:30.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.591+0000 7f87e8b9a740 1 Processor -- start 2024-09-12T02:13:30.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.592+0000 7f87e8b9a740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56137ec5bb60 0x56137ec63be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:30.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.592+0000 7f87e8b9a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56137ec5c480 0x56137ec64120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:30.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.592+0000 7f87e8b9a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56137ec66d40 0x56137ec64660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:30.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.592+0000 7f87e8b9a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56137ec69910 con 0x56137ec66d40 2024-09-12T02:13:30.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.592+0000 7f87e8b9a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56137eb7bf10 con 0x56137ec5c480 2024-09-12T02:13:30.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.592+0000 7f87e8b9a740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x56137ec69a90 con 0x56137ec5bb60 2024-09-12T02:13:30.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.592+0000 7f87e8b9a740 10 client.0 _ll_register_callbacks cb 0x56137eb44a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:13:30.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.592+0000 7f87e4d5e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56137ec66d40 0x56137ec64660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:30.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.592+0000 7f87e4d5e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56137ec66d40 0x56137ec64660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39602/0 (socket says 172.21.15.16:39602) 2024-09-12T02:13:30.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.592+0000 7f87e4d5e640 1 -- 172.21.15.16:0/909173593 learned_addr learned my addr 172.21.15.16:0/909173593 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:13:30.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.592+0000 7f87d7fff640 1 --2- 172.21.15.16:0/909173593 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56137ec5c480 0x56137ec64120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:30.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.592+0000 7f87dffff640 1 --2- 172.21.15.16:0/909173593 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56137ec5bb60 0x56137ec63be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:30.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.592+0000 7f87e4d5e640 1 -- 172.21.15.16:0/909173593 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56137ec5bb60 msgr2=0x56137ec63be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:30.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.592+0000 7f87e4d5e640 1 --2- 172.21.15.16:0/909173593 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56137ec5bb60 0x56137ec63be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:30.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.592+0000 7f87e4d5e640 1 -- 172.21.15.16:0/909173593 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56137ec5c480 msgr2=0x56137ec64120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:30.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.592+0000 7f87e4d5e640 1 --2- 172.21.15.16:0/909173593 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56137ec5c480 0x56137ec64120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:30.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.592+0000 7f87e4d5e640 1 -- 172.21.15.16:0/909173593 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56137e9326a0 con 0x56137ec66d40 2024-09-12T02:13:30.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.592+0000 7f87e4d5e640 1 --2- 172.21.15.16:0/909173593 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56137ec66d40 0x56137ec64660 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7f87d801ca00 tx=0x7f87d801ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38e1cff27eaff13e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:30.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.592+0000 7f87bf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:13:30.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.592+0000 7f87bf7fe640 1 -- 172.21.15.16:0/909173593 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87d8034060 con 0x56137ec66d40 2024-09-12T02:13:30.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.593+0000 7f87bf7fe640 1 -- 172.21.15.16:0/909173593 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f87d8011970 con 0x56137ec66d40 2024-09-12T02:13:30.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.593+0000 7f87e8b9a740 1 -- 172.21.15.16:0/909173593 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f87d8011b70 con 0x56137ec66d40 2024-09-12T02:13:30.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.593+0000 7f87bf7fe640 1 -- 172.21.15.16:0/909173593 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f87d8022040 con 0x56137ec66d40 2024-09-12T02:13:30.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.593+0000 7f87e8b9a740 10 client.19326 fetch_fsmap learned FSMap version 25 2024-09-12T02:13:30.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.593+0000 7f87e8b9a740 1 -- 172.21.15.16:0/909173593 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56137ec75510 con 0x56137ec66d40 2024-09-12T02:13:30.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.593+0000 7f87bf7fe640 1 -- 172.21.15.16:0/909173593 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87d8011b70 con 0x56137ec66d40 2024-09-12T02:13:30.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.593+0000 7f87bf7fe640 1 -- 172.21.15.16:0/909173593 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f87d8039070 con 0x56137ec66d40 2024-09-12T02:13:30.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.593+0000 7f87e8b9a740 10 client.19326 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:13:30.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.593+0000 7f87e8b9a740 10 client.19326 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:13:30.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.593+0000 7f87e8b9a740 20 client.19326 populate_metadata read hostname 'smithi016' 2024-09-12T02:13:30.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.593+0000 7f87e8b9a740 1 --2- 172.21.15.16:0/909173593 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x56137ec76970 0x56137ec78d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:30.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.593+0000 7f87e8b9a740 4 client.19326 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:13:30.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.593+0000 7f87e8b9a740 1 -- 172.21.15.16:0/909173593 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56137ebce7f0 con 0x56137ec76970 2024-09-12T02:13:30.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.593+0000 7f87dffff640 1 --2- 172.21.15.16:0/909173593 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x56137ec76970 0x56137ec78d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:30.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.594+0000 7f87dffff640 1 --2- 172.21.15.16:0/909173593 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x56137ec76970 0x56137ec78d60 crc :-1 s=READY pgs=352 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d938cee9dc50fcf2 server_cookie=2ea17a079c1fcfcc in_seq=0 out_seq=0 2024-09-12T02:13:30.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.594+0000 7f87bf7fe640 10 client.19326 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:13:30.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.595+0000 7f87bf7fe640 1 -- 172.21.15.16:0/909173593 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56137ebce7f0 con 0x56137ec76970 2024-09-12T02:13:30.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.595+0000 7f87bf7fe640 10 client.19326 handle_command_reply: tid=0 2024-09-12T02:13:30.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.595+0000 7f87e8b9a740 1 client.19326 shutdown 2024-09-12T02:13:30.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.595+0000 7f87e8b9a740 1 -- 172.21.15.16:0/909173593 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56137ec66d40 msgr2=0x56137ec64660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:30.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.595+0000 7f87e8b9a740 1 --2- 172.21.15.16:0/909173593 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56137ec66d40 0x56137ec64660 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7f87d801ca00 tx=0x7f87d801ca30 comp rx=0 tx=0).stop 2024-09-12T02:13:30.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.595+0000 7f87e8b9a740 1 -- 172.21.15.16:0/909173593 shutdown_connections 2024-09-12T02:13:30.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.595+0000 7f87e8b9a740 1 --2- 172.21.15.16:0/909173593 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x56137ec76970 0x56137ec78d60 crc :-1 s=READY pgs=352 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:30.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.595+0000 7f87e8b9a740 1 --2- 172.21.15.16:0/909173593 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56137ec66d40 0x56137ec64660 unknown :-1 s=CLOSED pgs=1658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:30.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.595+0000 7f87e8b9a740 1 --2- 172.21.15.16:0/909173593 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56137ec5c480 0x56137ec64120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:30.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.595+0000 7f87e8b9a740 1 --2- 172.21.15.16:0/909173593 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56137ec5bb60 0x56137ec63be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:30.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.595+0000 7f87e8b9a740 1 -- 172.21.15.16:0/909173593 >> 172.21.15.16:0/909173593 conn(0x56137ec580f0 msgr2=0x56137ec57270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:30.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.595+0000 7f87bf7fe640 0 client.19326 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:13:30.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.595+0000 7f87e8b9a740 1 -- 172.21.15.16:0/909173593 shutdown_connections 2024-09-12T02:13:30.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.595+0000 7f87e8b9a740 1 -- 172.21.15.16:0/909173593 wait complete. 2024-09-12T02:13:30.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.595+0000 7f87e8b9a740 20 client.19326 trim_cache size 0 max 16384 2024-09-12T02:13:30.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.613+0000 7f87e8b9a740 1 Processor -- start 2024-09-12T02:13:30.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.613+0000 7f87e8b9a740 1 -- start start 2024-09-12T02:13:30.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.613+0000 7f87e8b9a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56137ec5bb60 0x56137ec65e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:30.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.613+0000 7f87e8b9a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56137ec5c480 0x56137ec5e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:30.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.613+0000 7f87e8b9a740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56137ec66d40 0x56137ec7e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:30.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.613+0000 7f87e8b9a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f87e01b1e80 con 0x56137ec5bb60 2024-09-12T02:13:30.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.614+0000 7f87e8b9a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f87e01af750 con 0x56137ec5c480 2024-09-12T02:13:30.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.614+0000 7f87e8b9a740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f87e01ad5e0 con 0x56137ec66d40 2024-09-12T02:13:30.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.614+0000 7f87dffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56137ec5bb60 0x56137ec65e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:30.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.614+0000 7f87dffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56137ec5bb60 0x56137ec65e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39606/0 (socket says 172.21.15.16:39606) 2024-09-12T02:13:30.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.614+0000 7f87dffff640 1 -- 172.21.15.16:0/3966642434 learned_addr learned my addr 172.21.15.16:0/3966642434 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:13:30.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.614+0000 7f87d7fff640 1 --2- 172.21.15.16:0/3966642434 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56137ec5c480 0x56137ec5e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:30.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.614+0000 7f87dffff640 1 -- 172.21.15.16:0/3966642434 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56137ec66d40 msgr2=0x56137ec7e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:30.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.614+0000 7f87dffff640 1 --2- 172.21.15.16:0/3966642434 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56137ec66d40 0x56137ec7e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:30.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.614+0000 7f87dffff640 1 -- 172.21.15.16:0/3966642434 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56137ec5c480 msgr2=0x56137ec5e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:30.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.614+0000 7f87dffff640 1 --2- 172.21.15.16:0/3966642434 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56137ec5c480 0x56137ec5e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:30.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.614+0000 7f87dffff640 1 -- 172.21.15.16:0/3966642434 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56137ec5c2b0 con 0x56137ec5bb60 2024-09-12T02:13:30.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.614+0000 7f87d7fff640 1 --2- 172.21.15.16:0/3966642434 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56137ec5c480 0x56137ec5e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:13:30.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.614+0000 7f87dffff640 1 --2- 172.21.15.16:0/3966642434 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56137ec5bb60 0x56137ec65e00 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7f87d0004700 tx=0x7f87d000bd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e772258e5053248 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:30.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.614+0000 7f87d5ffb640 1 -- 172.21.15.16:0/3966642434 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87d0003cd0 con 0x56137ec5bb60 2024-09-12T02:13:30.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.614+0000 7f87d5ffb640 1 -- 172.21.15.16:0/3966642434 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87d0003e70 con 0x56137ec5bb60 2024-09-12T02:13:30.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.614+0000 7f87d5ffb640 1 -- 172.21.15.16:0/3966642434 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87d001d6d0 con 0x56137ec5bb60 2024-09-12T02:13:30.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.615+0000 7f87e8b9a740 1 -- 172.21.15.16:0/3966642434 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56137ec5bb60 msgr2=0x56137ec65e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:30.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.615+0000 7f87e8b9a740 1 --2- 172.21.15.16:0/3966642434 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56137ec5bb60 0x56137ec65e00 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7f87d0004700 tx=0x7f87d000bd80 comp rx=0 tx=0).stop 2024-09-12T02:13:30.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.615+0000 7f87e8b9a740 1 -- 172.21.15.16:0/3966642434 shutdown_connections 2024-09-12T02:13:30.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.615+0000 7f87e8b9a740 1 --2- 172.21.15.16:0/3966642434 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56137ec66d40 0x56137ec7e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:30.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.615+0000 7f87e8b9a740 1 --2- 172.21.15.16:0/3966642434 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56137ec5c480 0x56137ec5e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:30.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.615+0000 7f87e8b9a740 1 --2- 172.21.15.16:0/3966642434 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56137ec5bb60 0x56137ec65e00 unknown :-1 s=CLOSED pgs=1659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:30.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.615+0000 7f87e8b9a740 1 -- 172.21.15.16:0/3966642434 >> 172.21.15.16:0/3966642434 conn(0x56137ec580f0 msgr2=0x56137ec57270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:30.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.615+0000 7f87e8b9a740 1 -- 172.21.15.16:0/3966642434 shutdown_connections 2024-09-12T02:13:30.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.615+0000 7f87e8b9a740 1 -- 172.21.15.16:0/3966642434 wait complete. 2024-09-12T02:13:30.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.615+0000 7f87e8b9a740 1 -- start start 2024-09-12T02:13:30.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.616+0000 7f87e8b9a740 1 Processor -- start 2024-09-12T02:13:30.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.616+0000 7f87e8b9a740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56137ec5bb60 0x56137ec6f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:30.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.616+0000 7f87e8b9a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56137ec5c480 0x56137ec6a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:30.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.616+0000 7f87e8b9a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56137ec66d40 0x56137ec6abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:30.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.616+0000 7f87e8b9a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f87e01adc30 con 0x56137ec66d40 2024-09-12T02:13:30.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.616+0000 7f87e8b9a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f87e01af080 con 0x56137ec5c480 2024-09-12T02:13:30.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.616+0000 7f87e8b9a740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f87e01ae2e0 con 0x56137ec5bb60 2024-09-12T02:13:30.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.616+0000 7f87e8b9a740 10 client.0 _ll_register_callbacks cb 0x7f87d8020070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:13:30.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.616+0000 7f87e4d5e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56137ec66d40 0x56137ec6abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:30.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.616+0000 7f87e4d5e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56137ec66d40 0x56137ec6abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39612/0 (socket says 172.21.15.16:39612) 2024-09-12T02:13:30.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.616+0000 7f87e4d5e640 1 -- 172.21.15.16:0/660988946 learned_addr learned my addr 172.21.15.16:0/660988946 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:13:30.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.616+0000 7f87dffff640 1 --2- 172.21.15.16:0/660988946 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56137ec5bb60 0x56137ec6f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:30.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.616+0000 7f87d7fff640 1 --2- 172.21.15.16:0/660988946 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56137ec5c480 0x56137ec6a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:30.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.616+0000 7f87e4d5e640 1 -- 172.21.15.16:0/660988946 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56137ec5bb60 msgr2=0x56137ec6f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:30.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.616+0000 7f87e4d5e640 1 --2- 172.21.15.16:0/660988946 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56137ec5bb60 0x56137ec6f2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:30.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.616+0000 7f87e4d5e640 1 -- 172.21.15.16:0/660988946 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56137ec5c480 msgr2=0x56137ec6a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:30.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.616+0000 7f87e4d5e640 1 --2- 172.21.15.16:0/660988946 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56137ec5c480 0x56137ec6a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:30.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.616+0000 7f87e4d5e640 1 -- 172.21.15.16:0/660988946 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56137ec6b250 con 0x56137ec66d40 2024-09-12T02:13:30.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.617+0000 7f87e4d5e640 1 --2- 172.21.15.16:0/660988946 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56137ec66d40 0x56137ec6abb0 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7f87d801caa0 tx=0x7f87d8011c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9a94281e7f91eec server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:30.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.617+0000 7f87bf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:13:30.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.617+0000 7f87bf7fe640 1 -- 172.21.15.16:0/660988946 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87d803f070 con 0x56137ec66d40 2024-09-12T02:13:30.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.617+0000 7f87bf7fe640 1 -- 172.21.15.16:0/660988946 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f87d801a660 con 0x56137ec66d40 2024-09-12T02:13:30.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.617+0000 7f87e8b9a740 1 -- 172.21.15.16:0/660988946 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f87d0003e70 con 0x56137ec66d40 2024-09-12T02:13:30.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.617+0000 7f87bf7fe640 1 -- 172.21.15.16:0/660988946 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f87d801a400 con 0x56137ec66d40 2024-09-12T02:13:30.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.617+0000 7f87e8b9a740 10 client.19332 fetch_fsmap learned FSMap version 25 2024-09-12T02:13:30.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.617+0000 7f87e8b9a740 1 -- 172.21.15.16:0/660988946 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56137ece12e0 con 0x56137ec66d40 2024-09-12T02:13:30.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.617+0000 7f87bf7fe640 1 -- 172.21.15.16:0/660988946 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87d0003e70 con 0x56137ec66d40 2024-09-12T02:13:30.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.617+0000 7f87bf7fe640 1 -- 172.21.15.16:0/660988946 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f87d8042030 con 0x56137ec66d40 2024-09-12T02:13:30.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.617+0000 7f87e8b9a740 10 client.19332 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:13:30.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.617+0000 7f87e8b9a740 10 client.19332 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:13:30.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.617+0000 7f87e8b9a740 20 client.19332 populate_metadata read hostname 'smithi016' 2024-09-12T02:13:30.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.618+0000 7f87e8b9a740 1 --2- 172.21.15.16:0/660988946 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x56137ece1990 0x56137ece3db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:30.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.618+0000 7f87e8b9a740 4 client.19332 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-12T02:13:30.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.618+0000 7f87e8b9a740 1 -- 172.21.15.16:0/660988946 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56137ebc8d00 con 0x56137ece1990 2024-09-12T02:13:30.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.618+0000 7f87dffff640 1 --2- 172.21.15.16:0/660988946 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x56137ece1990 0x56137ece3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:30.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.618+0000 7f87dffff640 1 --2- 172.21.15.16:0/660988946 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x56137ece1990 0x56137ece3db0 crc :-1 s=READY pgs=353 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de6c8d2ad7a14412 server_cookie=542dac8cb48164b4 in_seq=0 out_seq=0 2024-09-12T02:13:30.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.618+0000 7f87bf7fe640 10 client.19332 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:13:30.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.620+0000 7f87bf7fe640 1 -- 172.21.15.16:0/660988946 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56137ebc8d00 con 0x56137ece1990 2024-09-12T02:13:30.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.620+0000 7f87bf7fe640 10 client.19332 handle_command_reply: tid=0 2024-09-12T02:13:30.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.620+0000 7f87e8b9a740 1 client.19332 shutdown 2024-09-12T02:13:30.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.620+0000 7f87e8b9a740 1 -- 172.21.15.16:0/660988946 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56137ec66d40 msgr2=0x56137ec6abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:30.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.620+0000 7f87e8b9a740 1 --2- 172.21.15.16:0/660988946 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56137ec66d40 0x56137ec6abb0 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7f87d801caa0 tx=0x7f87d8011c20 comp rx=0 tx=0).stop 2024-09-12T02:13:30.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.620+0000 7f87e8b9a740 1 -- 172.21.15.16:0/660988946 shutdown_connections 2024-09-12T02:13:30.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.620+0000 7f87e8b9a740 1 --2- 172.21.15.16:0/660988946 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x56137ece1990 0x56137ece3db0 crc :-1 s=READY pgs=353 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:30.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.620+0000 7f87e8b9a740 1 --2- 172.21.15.16:0/660988946 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56137ec66d40 0x56137ec6abb0 unknown :-1 s=CLOSED pgs=1660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:30.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.620+0000 7f87e8b9a740 1 --2- 172.21.15.16:0/660988946 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56137ec5c480 0x56137ec6a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:30.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.620+0000 7f87e8b9a740 1 --2- 172.21.15.16:0/660988946 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56137ec5bb60 0x56137ec6f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:30.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.620+0000 7f87bf7fe640 0 client.19332 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:13:30.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.620+0000 7f87e8b9a740 1 -- 172.21.15.16:0/660988946 >> 172.21.15.16:0/660988946 conn(0x56137ec580f0 msgr2=0x56137ec56a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:30.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.620+0000 7f87e8b9a740 1 -- 172.21.15.16:0/660988946 shutdown_connections 2024-09-12T02:13:30.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.620+0000 7f87e8b9a740 1 -- 172.21.15.16:0/660988946 wait complete. 2024-09-12T02:13:30.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.620+0000 7f87e8b9a740 20 client.19332 trim_cache size 0 max 16384 2024-09-12T02:13:30.631 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T02:13:30.631 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-09-12T02:13:30.631 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "7033def4-2a8b-473a-9c32-15f611413fbd", 2024-09-12T02:13:30.631 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-09-12T02:13:30.631 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T02:13:30.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.621+0000 7f87e67c1640 1 -- 172.21.15.16:0/2465498646 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f87cc0a2b80 msgr2=0x7f87cc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:30.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.621+0000 7f87e67c1640 1 --2- 172.21.15.16:0/2465498646 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f87cc0a2b80 0x7f87cc0a5040 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f87d801f000 tx=0x7f87d8020000 comp rx=0 tx=0).stop 2024-09-12T02:13:30.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.621+0000 7f87e67c1640 1 -- 172.21.15.16:0/2465498646 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87e01036e0 msgr2=0x7f87e0100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:30.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.621+0000 7f87e67c1640 1 --2- 172.21.15.16:0/2465498646 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87e01036e0 0x7f87e0100ed0 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7f87c8005940 tx=0x7f87c80042f0 comp rx=0 tx=0).stop 2024-09-12T02:13:30.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.621+0000 7f87e67c1640 1 -- 172.21.15.16:0/2465498646 shutdown_connections 2024-09-12T02:13:30.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.621+0000 7f87e67c1640 1 --2- 172.21.15.16:0/2465498646 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f87cc0a2b80 0x7f87cc0a5040 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:30.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.621+0000 7f87e67c1640 1 --2- 172.21.15.16:0/2465498646 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f87e01082a0 0x7f87e0101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:30.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.621+0000 7f87e67c1640 1 --2- 172.21.15.16:0/2465498646 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87e01036e0 0x7f87e0100ed0 unknown :-1 s=CLOSED pgs=1656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:30.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.621+0000 7f87e67c1640 1 --2- 172.21.15.16:0/2465498646 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87e0102d30 0x7f87e0100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:30.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.621+0000 7f87e67c1640 1 -- 172.21.15.16:0/2465498646 >> 172.21.15.16:0/2465498646 conn(0x7f87e00fc7d0 msgr2=0x7f87e0105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:30.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.621+0000 7f87e67c1640 1 -- 172.21.15.16:0/2465498646 shutdown_connections 2024-09-12T02:13:30.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:30.621+0000 7f87e67c1640 1 -- 172.21.15.16:0/2465498646 wait complete. 2024-09-12T02:13:31.332 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T02:13:31.592 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:13:32.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.029+0000 7f68bfd73640 1 -- 172.21.15.16:0/2382163060 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68b81036e0 msgr2=0x7f68b8107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:32.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.029+0000 7f68bfd73640 1 --2- 172.21.15.16:0/2382163060 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68b81036e0 0x7f68b8107b90 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7f68a8009930 tx=0x7f68a802a420 comp rx=0 tx=0).stop 2024-09-12T02:13:32.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.029+0000 7f68bfd73640 1 -- 172.21.15.16:0/2382163060 shutdown_connections 2024-09-12T02:13:32.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.029+0000 7f68bfd73640 1 --2- 172.21.15.16:0/2382163060 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68b81082a0 0x7f68b810ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:32.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.029+0000 7f68bfd73640 1 --2- 172.21.15.16:0/2382163060 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68b81036e0 0x7f68b8107b90 unknown :-1 s=CLOSED pgs=1661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:32.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.029+0000 7f68bfd73640 1 --2- 172.21.15.16:0/2382163060 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f68b8102d30 0x7f68b8103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:32.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.029+0000 7f68bfd73640 1 -- 172.21.15.16:0/2382163060 >> 172.21.15.16:0/2382163060 conn(0x7f68b80fc7d0 msgr2=0x7f68b80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:32.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.029+0000 7f68bfd73640 1 -- 172.21.15.16:0/2382163060 shutdown_connections 2024-09-12T02:13:32.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.029+0000 7f68bfd73640 1 -- 172.21.15.16:0/2382163060 wait complete. 2024-09-12T02:13:32.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.029+0000 7f68bfd73640 1 Processor -- start 2024-09-12T02:13:32.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.029+0000 7f68bfd73640 1 -- start start 2024-09-12T02:13:32.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.030+0000 7f68bfd73640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f68b8102d30 0x7f68b8198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:32.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.030+0000 7f68bfd73640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68b81036e0 0x7f68b8198c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:32.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.030+0000 7f68bfd73640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68b81082a0 0x7f68b819d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:32.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.030+0000 7f68bfd73640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f68b8112190 con 0x7f68b81036e0 2024-09-12T02:13:32.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.030+0000 7f68bfd73640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f68b8112010 con 0x7f68b81082a0 2024-09-12T02:13:32.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.030+0000 7f68bfd73640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f68b8112310 con 0x7f68b8102d30 2024-09-12T02:13:32.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.030+0000 7f68bd30e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68b81036e0 0x7f68b8198c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:32.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.030+0000 7f68bd30e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68b81036e0 0x7f68b8198c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52100/0 (socket says 172.21.15.16:52100) 2024-09-12T02:13:32.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.030+0000 7f68bd30e640 1 -- 172.21.15.16:0/4067187488 learned_addr learned my addr 172.21.15.16:0/4067187488 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:13:32.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.030+0000 7f68bdb0f640 1 --2- 172.21.15.16:0/4067187488 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f68b8102d30 0x7f68b8198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:32.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.030+0000 7f68be310640 1 --2- 172.21.15.16:0/4067187488 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68b81082a0 0x7f68b819d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:32.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.030+0000 7f68bd30e640 1 -- 172.21.15.16:0/4067187488 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f68b8102d30 msgr2=0x7f68b8198750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:32.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.030+0000 7f68bd30e640 1 --2- 172.21.15.16:0/4067187488 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f68b8102d30 0x7f68b8198750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:32.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.030+0000 7f68bd30e640 1 -- 172.21.15.16:0/4067187488 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68b81082a0 msgr2=0x7f68b819d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:32.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.030+0000 7f68bd30e640 1 --2- 172.21.15.16:0/4067187488 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68b81082a0 0x7f68b819d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:32.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.030+0000 7f68bd30e640 1 -- 172.21.15.16:0/4067187488 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68b819d700 con 0x7f68b81036e0 2024-09-12T02:13:32.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.030+0000 7f68bd30e640 1 --2- 172.21.15.16:0/4067187488 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68b81036e0 0x7f68b8198c90 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7f68a802a940 tx=0x7f68a803ac80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:32.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.031+0000 7f68aeffd640 1 -- 172.21.15.16:0/4067187488 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68a8035070 con 0x7f68b81036e0 2024-09-12T02:13:32.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.031+0000 7f68aeffd640 1 -- 172.21.15.16:0/4067187488 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68a8003ad0 con 0x7f68b81036e0 2024-09-12T02:13:32.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.031+0000 7f68bfd73640 1 -- 172.21.15.16:0/4067187488 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68b819d990 con 0x7f68b81036e0 2024-09-12T02:13:32.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.031+0000 7f68aeffd640 1 -- 172.21.15.16:0/4067187488 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68a8031de0 con 0x7f68b81036e0 2024-09-12T02:13:32.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.031+0000 7f68bfd73640 1 -- 172.21.15.16:0/4067187488 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68b81008c0 con 0x7f68b81036e0 2024-09-12T02:13:32.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.033+0000 7f68aeffd640 1 -- 172.21.15.16:0/4067187488 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f68a80315b0 con 0x7f68b81036e0 2024-09-12T02:13:32.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.033+0000 7f68c214c740 1 Processor -- start 2024-09-12T02:13:32.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.034+0000 7f68c214c740 1 -- start start 2024-09-12T02:13:32.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.034+0000 7f68c214c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be5cfc44a0 0x55be5cfc4880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:32.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.034+0000 7f68c214c740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55be5cfc4dc0 0x55be5cfcf140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:32.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.034+0000 7f68c214c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be5cfcf680 0x55be5cfd1a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:32.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.034+0000 7f68c214c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55be5cf2a2e0 con 0x55be5cfcf680 2024-09-12T02:13:32.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.034+0000 7f68c214c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55be5ced1ce0 con 0x55be5cfc44a0 2024-09-12T02:13:32.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.034+0000 7f68c214c740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55be5cf262a0 con 0x55be5cfc4dc0 2024-09-12T02:13:32.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.034+0000 7f68bd30e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be5cfcf680 0x55be5cfd1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:32.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.034+0000 7f68bd30e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be5cfcf680 0x55be5cfd1a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52106/0 (socket says 172.21.15.16:52106) 2024-09-12T02:13:32.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.034+0000 7f68bd30e640 1 -- 172.21.15.16:0/2918049293 learned_addr learned my addr 172.21.15.16:0/2918049293 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:13:32.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.034+0000 7f68aeffd640 1 --2- 172.21.15.16:0/4067187488 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f68a40a2c00 0x7f68a40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:32.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.034+0000 7f68bdb0f640 1 --2- 172.21.15.16:0/2918049293 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55be5cfc4dc0 0x55be5cfcf140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:32.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.034+0000 7f68be310640 1 --2- 172.21.15.16:0/2918049293 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be5cfc44a0 0x55be5cfc4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:32.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.034+0000 7f68be310640 1 --2- 172.21.15.16:0/4067187488 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f68a40a2c00 0x7f68a40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:32.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.035+0000 7f68aeffd640 1 -- 172.21.15.16:0/4067187488 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f68a80f3240 con 0x7f68b81036e0 2024-09-12T02:13:32.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.035+0000 7f68bdb0f640 1 -- 172.21.15.16:0/2918049293 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be5cfc44a0 msgr2=0x55be5cfc4880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:32.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.035+0000 7f68bdb0f640 1 --2- 172.21.15.16:0/2918049293 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be5cfc44a0 0x55be5cfc4880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:32.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.035+0000 7f68bdb0f640 1 -- 172.21.15.16:0/2918049293 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be5cfcf680 msgr2=0x55be5cfd1a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:32.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.035+0000 7f68bdb0f640 1 --2- 172.21.15.16:0/2918049293 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be5cfcf680 0x55be5cfd1a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:32.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.035+0000 7f68bdb0f640 1 -- 172.21.15.16:0/2918049293 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be5ceb6440 con 0x55be5cfc4dc0 2024-09-12T02:13:32.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.035+0000 7f68bd30e640 1 --2- 172.21.15.16:0/2918049293 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be5cfcf680 0x55be5cfd1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:13:32.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.035+0000 7f68be310640 1 --2- 172.21.15.16:0/4067187488 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f68a40a2c00 0x7f68a40a50c0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f68b00070b0 tx=0x7f68b00093d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:32.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.035+0000 7f68bdb0f640 1 --2- 172.21.15.16:0/2918049293 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55be5cfc4dc0 0x55be5cfcf140 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f68b40099a0 tx=0x7f68b4009e60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1bedbcb13a47be1c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:32.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.035+0000 7f68a3fff640 1 -- 172.21.15.16:0/2918049293 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68b4017070 con 0x55be5cfc4dc0 2024-09-12T02:13:32.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.035+0000 7f68a3fff640 1 -- 172.21.15.16:0/2918049293 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68b4013470 con 0x55be5cfc4dc0 2024-09-12T02:13:32.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.035+0000 7f68a3fff640 1 -- 172.21.15.16:0/2918049293 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68b4012630 con 0x55be5cfc4dc0 2024-09-12T02:13:32.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.036+0000 7f68c214c740 1 -- 172.21.15.16:0/2918049293 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55be5cfc4dc0 msgr2=0x55be5cfcf140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:32.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.036+0000 7f68c214c740 1 --2- 172.21.15.16:0/2918049293 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55be5cfc4dc0 0x55be5cfcf140 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f68b40099a0 tx=0x7f68b4009e60 comp rx=0 tx=0).stop 2024-09-12T02:13:32.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.036+0000 7f68c214c740 1 -- 172.21.15.16:0/2918049293 shutdown_connections 2024-09-12T02:13:32.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.036+0000 7f68c214c740 1 --2- 172.21.15.16:0/2918049293 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be5cfcf680 0x55be5cfd1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:32.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.036+0000 7f68c214c740 1 --2- 172.21.15.16:0/2918049293 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55be5cfc4dc0 0x55be5cfcf140 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:32.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.036+0000 7f68c214c740 1 --2- 172.21.15.16:0/2918049293 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be5cfc44a0 0x55be5cfc4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:32.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.036+0000 7f68c214c740 1 -- 172.21.15.16:0/2918049293 >> 172.21.15.16:0/2918049293 conn(0x55be5cfc0a30 msgr2=0x55be5cfc0e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:32.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.036+0000 7f68c214c740 1 -- 172.21.15.16:0/2918049293 shutdown_connections 2024-09-12T02:13:32.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.036+0000 7f68c214c740 1 -- 172.21.15.16:0/2918049293 wait complete. 2024-09-12T02:13:32.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.036+0000 7f68c214c740 1 -- start start 2024-09-12T02:13:32.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.036+0000 7f68c214c740 1 Processor -- start 2024-09-12T02:13:32.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.037+0000 7f68c214c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be5cfc44a0 0x55be5cfcefd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:32.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.037+0000 7f68c214c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be5cfc4dc0 0x55be5cfcc280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:32.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.037+0000 7f68c214c740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55be5cfcf680 0x55be5cfcc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:32.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.037+0000 7f68c214c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55be5cfd2250 con 0x55be5cfc4dc0 2024-09-12T02:13:32.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.037+0000 7f68c214c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55be5cee5050 con 0x55be5cfc44a0 2024-09-12T02:13:32.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.037+0000 7f68c214c740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55be5cfd23d0 con 0x55be5cfcf680 2024-09-12T02:13:32.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.037+0000 7f68c214c740 10 client.0 _ll_register_callbacks cb 0x55be5cf25160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:13:32.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.037+0000 7f68bd30e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be5cfc4dc0 0x55be5cfcc280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:32.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.037+0000 7f68bd30e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be5cfc4dc0 0x55be5cfcc280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52118/0 (socket says 172.21.15.16:52118) 2024-09-12T02:13:32.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.037+0000 7f68bd30e640 1 -- 172.21.15.16:0/1898749392 learned_addr learned my addr 172.21.15.16:0/1898749392 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:13:32.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.037+0000 7f68bdb0f640 1 --2- 172.21.15.16:0/1898749392 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be5cfc44a0 0x55be5cfcefd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:32.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.037+0000 7f68be310640 1 --2- 172.21.15.16:0/1898749392 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55be5cfcf680 0x55be5cfcc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:32.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.037+0000 7f68bd30e640 1 -- 172.21.15.16:0/1898749392 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55be5cfcf680 msgr2=0x55be5cfcc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:32.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.037+0000 7f68bd30e640 1 --2- 172.21.15.16:0/1898749392 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55be5cfcf680 0x55be5cfcc7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:32.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.037+0000 7f68bd30e640 1 -- 172.21.15.16:0/1898749392 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be5cfc44a0 msgr2=0x55be5cfcefd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:32.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.037+0000 7f68bd30e640 1 --2- 172.21.15.16:0/1898749392 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be5cfc44a0 0x55be5cfcefd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:32.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.037+0000 7f68bd30e640 1 -- 172.21.15.16:0/1898749392 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be5cc9b3c0 con 0x55be5cfc4dc0 2024-09-12T02:13:32.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.037+0000 7f68be310640 1 --2- 172.21.15.16:0/1898749392 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55be5cfcf680 0x55be5cfcc7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:13:32.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.038+0000 7f68bd30e640 1 --2- 172.21.15.16:0/1898749392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be5cfc4dc0 0x55be5cfcc280 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7f68a8006010 tx=0x7f68a8031510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2f53fa37a7878a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:32.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.038+0000 7f68a17fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:13:32.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.038+0000 7f68a17fa640 1 -- 172.21.15.16:0/1898749392 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68a80bf4a0 con 0x55be5cfc4dc0 2024-09-12T02:13:32.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.038+0000 7f68a17fa640 1 -- 172.21.15.16:0/1898749392 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f68a8003c80 con 0x55be5cfc4dc0 2024-09-12T02:13:32.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.038+0000 7f68c214c740 1 -- 172.21.15.16:0/1898749392 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68b4013470 con 0x55be5cfc4dc0 2024-09-12T02:13:32.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.038+0000 7f68a17fa640 1 -- 172.21.15.16:0/1898749392 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f68a80c0240 con 0x55be5cfc4dc0 2024-09-12T02:13:32.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.038+0000 7f68c214c740 10 client.19344 fetch_fsmap learned FSMap version 25 2024-09-12T02:13:32.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.038+0000 7f68c214c740 1 -- 172.21.15.16:0/1898749392 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55be5cfddc20 con 0x55be5cfc4dc0 2024-09-12T02:13:32.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.038+0000 7f68a17fa640 1 -- 172.21.15.16:0/1898749392 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68b4013470 con 0x55be5cfc4dc0 2024-09-12T02:13:32.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.038+0000 7f68a17fa640 1 -- 172.21.15.16:0/1898749392 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f68a80de2e0 con 0x55be5cfc4dc0 2024-09-12T02:13:32.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.038+0000 7f68c214c740 10 client.19344 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:13:32.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.038+0000 7f68c214c740 10 client.19344 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:13:32.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.039+0000 7f68c214c740 20 client.19344 populate_metadata read hostname 'smithi016' 2024-09-12T02:13:32.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.039+0000 7f68c214c740 1 --2- 172.21.15.16:0/1898749392 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55be5cfdf030 0x55be5cfe1440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:32.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.039+0000 7f68c214c740 4 client.19344 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:13:32.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.039+0000 7f68c214c740 1 -- 172.21.15.16:0/1898749392 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55be5cf37850 con 0x55be5cfdf030 2024-09-12T02:13:32.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.039+0000 7f68bdb0f640 1 --2- 172.21.15.16:0/1898749392 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55be5cfdf030 0x55be5cfe1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:32.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.039+0000 7f68bdb0f640 1 --2- 172.21.15.16:0/1898749392 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55be5cfdf030 0x55be5cfe1440 crc :-1 s=READY pgs=354 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6eb47f34dd469e1f server_cookie=447b6598f0a9f612 in_seq=0 out_seq=0 2024-09-12T02:13:32.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.039+0000 7f68a17fa640 10 client.19344 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:13:32.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.040+0000 7f68a17fa640 1 -- 172.21.15.16:0/1898749392 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55be5cf37850 con 0x55be5cfdf030 2024-09-12T02:13:32.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.040+0000 7f68a17fa640 10 client.19344 handle_command_reply: tid=0 2024-09-12T02:13:32.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.040+0000 7f68c214c740 1 client.19344 shutdown 2024-09-12T02:13:32.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.040+0000 7f68c214c740 1 -- 172.21.15.16:0/1898749392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be5cfc4dc0 msgr2=0x55be5cfcc280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:32.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.040+0000 7f68c214c740 1 --2- 172.21.15.16:0/1898749392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be5cfc4dc0 0x55be5cfcc280 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7f68a8006010 tx=0x7f68a8031510 comp rx=0 tx=0).stop 2024-09-12T02:13:32.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.041+0000 7f68c214c740 1 -- 172.21.15.16:0/1898749392 shutdown_connections 2024-09-12T02:13:32.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.041+0000 7f68c214c740 1 --2- 172.21.15.16:0/1898749392 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55be5cfdf030 0x55be5cfe1440 crc :-1 s=READY pgs=354 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:32.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.041+0000 7f68c214c740 1 --2- 172.21.15.16:0/1898749392 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55be5cfcf680 0x55be5cfcc7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:32.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.041+0000 7f68c214c740 1 --2- 172.21.15.16:0/1898749392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be5cfc4dc0 0x55be5cfcc280 unknown :-1 s=CLOSED pgs=1663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:32.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.041+0000 7f68a17fa640 0 client.19344 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:13:32.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.041+0000 7f68c214c740 1 --2- 172.21.15.16:0/1898749392 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be5cfc44a0 0x55be5cfcefd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:32.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.041+0000 7f68c214c740 1 -- 172.21.15.16:0/1898749392 >> 172.21.15.16:0/1898749392 conn(0x55be5cfc0a30 msgr2=0x55be5cfc6710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:32.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.041+0000 7f68c214c740 1 -- 172.21.15.16:0/1898749392 shutdown_connections 2024-09-12T02:13:32.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.041+0000 7f68c214c740 1 -- 172.21.15.16:0/1898749392 wait complete. 2024-09-12T02:13:32.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.041+0000 7f68c214c740 20 client.19344 trim_cache size 0 max 16384 2024-09-12T02:13:32.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.058+0000 7f68c214c740 1 Processor -- start 2024-09-12T02:13:32.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.058+0000 7f68c214c740 1 -- start start 2024-09-12T02:13:32.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.058+0000 7f68c214c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be5cfc44a0 0x55be5cfdb4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:32.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.058+0000 7f68c214c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be5cfc4dc0 0x55be5cfe6d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:32.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.058+0000 7f68c214c740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55be5cfcf680 0x55be5cfe7290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:32.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.058+0000 7f68c214c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f68b81adce0 con 0x55be5cfc4dc0 2024-09-12T02:13:32.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.058+0000 7f68c214c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f68b81ab5b0 con 0x55be5cfc44a0 2024-09-12T02:13:32.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.058+0000 7f68c214c740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f68b81a9400 con 0x55be5cfcf680 2024-09-12T02:13:32.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.058+0000 7f68bd30e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be5cfc4dc0 0x55be5cfe6d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:32.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.058+0000 7f68bd30e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be5cfc4dc0 0x55be5cfe6d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52120/0 (socket says 172.21.15.16:52120) 2024-09-12T02:13:32.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.058+0000 7f68bd30e640 1 -- 172.21.15.16:0/1237275956 learned_addr learned my addr 172.21.15.16:0/1237275956 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:13:32.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.058+0000 7f68bd30e640 1 -- 172.21.15.16:0/1237275956 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55be5cfcf680 msgr2=0x55be5cfe7290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:32.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.058+0000 7f68bd30e640 1 --2- 172.21.15.16:0/1237275956 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55be5cfcf680 0x55be5cfe7290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:32.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.058+0000 7f68bd30e640 1 -- 172.21.15.16:0/1237275956 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be5cfc44a0 msgr2=0x55be5cfdb4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:32.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.058+0000 7f68bdb0f640 1 --2- 172.21.15.16:0/1237275956 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be5cfc44a0 0x55be5cfdb4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:32.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.058+0000 7f68bd30e640 1 --2- 172.21.15.16:0/1237275956 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be5cfc44a0 0x55be5cfdb4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:32.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.058+0000 7f68bd30e640 1 -- 172.21.15.16:0/1237275956 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be5cfcf340 con 0x55be5cfc4dc0 2024-09-12T02:13:32.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.058+0000 7f68bdb0f640 1 --2- 172.21.15.16:0/1237275956 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be5cfc44a0 0x55be5cfdb4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:13:32.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.059+0000 7f68bd30e640 1 --2- 172.21.15.16:0/1237275956 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be5cfc4dc0 0x55be5cfe6d50 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7f68a80bfad0 tx=0x7f68a80318e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41f32aa0ff1bffc9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:32.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.059+0000 7f68a37fe640 1 -- 172.21.15.16:0/1237275956 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68a80bf4a0 con 0x55be5cfc4dc0 2024-09-12T02:13:32.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.059+0000 7f68a37fe640 1 -- 172.21.15.16:0/1237275956 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68a80bfb40 con 0x55be5cfc4dc0 2024-09-12T02:13:32.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.059+0000 7f68a37fe640 1 -- 172.21.15.16:0/1237275956 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68a80c8410 con 0x55be5cfc4dc0 2024-09-12T02:13:32.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.059+0000 7f68c214c740 1 -- 172.21.15.16:0/1237275956 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be5cfc4dc0 msgr2=0x55be5cfe6d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:32.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.059+0000 7f68c214c740 1 --2- 172.21.15.16:0/1237275956 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be5cfc4dc0 0x55be5cfe6d50 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7f68a80bfad0 tx=0x7f68a80318e0 comp rx=0 tx=0).stop 2024-09-12T02:13:32.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.059+0000 7f68c214c740 1 -- 172.21.15.16:0/1237275956 shutdown_connections 2024-09-12T02:13:32.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.059+0000 7f68c214c740 1 --2- 172.21.15.16:0/1237275956 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55be5cfcf680 0x55be5cfe7290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:32.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.059+0000 7f68c214c740 1 --2- 172.21.15.16:0/1237275956 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be5cfc4dc0 0x55be5cfe6d50 unknown :-1 s=CLOSED pgs=1664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:32.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.059+0000 7f68c214c740 1 --2- 172.21.15.16:0/1237275956 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be5cfc44a0 0x55be5cfdb4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:32.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.059+0000 7f68c214c740 1 -- 172.21.15.16:0/1237275956 >> 172.21.15.16:0/1237275956 conn(0x55be5cfc0a30 msgr2=0x55be5cfcab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:32.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.060+0000 7f68c214c740 1 -- 172.21.15.16:0/1237275956 shutdown_connections 2024-09-12T02:13:32.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.060+0000 7f68c214c740 1 -- 172.21.15.16:0/1237275956 wait complete. 2024-09-12T02:13:32.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.060+0000 7f68c214c740 1 -- start start 2024-09-12T02:13:32.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.060+0000 7f68c214c740 1 Processor -- start 2024-09-12T02:13:32.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.060+0000 7f68c214c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be5cfc44a0 0x55be5cfd3780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:32.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.060+0000 7f68c214c740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55be5cfc4dc0 0x55be5cfd3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:32.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.060+0000 7f68c214c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be5cfcf680 0x55be5d048d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:32.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.060+0000 7f68c214c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f68b81a9a70 con 0x55be5cfc44a0 2024-09-12T02:13:32.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.060+0000 7f68c214c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f68b81aaee0 con 0x55be5cfcf680 2024-09-12T02:13:32.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.060+0000 7f68c214c740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f68b81aa140 con 0x55be5cfc4dc0 2024-09-12T02:13:32.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.060+0000 7f68c214c740 10 client.0 _ll_register_callbacks cb 0x7f68a80bbcc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:13:32.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.060+0000 7f68bdb0f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be5cfc44a0 0x55be5cfd3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:32.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.061+0000 7f68bdb0f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be5cfc44a0 0x55be5cfd3780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52122/0 (socket says 172.21.15.16:52122) 2024-09-12T02:13:32.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.061+0000 7f68bdb0f640 1 -- 172.21.15.16:0/710354206 learned_addr learned my addr 172.21.15.16:0/710354206 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:13:32.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.061+0000 7f68be310640 1 --2- 172.21.15.16:0/710354206 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be5cfcf680 0x55be5d048d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:32.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.061+0000 7f68bd30e640 1 --2- 172.21.15.16:0/710354206 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55be5cfc4dc0 0x55be5cfd3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:32.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.061+0000 7f68bdb0f640 1 -- 172.21.15.16:0/710354206 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55be5cfc4dc0 msgr2=0x55be5cfd3cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:32.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.061+0000 7f68bdb0f640 1 --2- 172.21.15.16:0/710354206 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55be5cfc4dc0 0x55be5cfd3cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:32.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.061+0000 7f68bdb0f640 1 -- 172.21.15.16:0/710354206 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be5cfcf680 msgr2=0x55be5d048d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:32.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.061+0000 7f68bdb0f640 1 --2- 172.21.15.16:0/710354206 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be5cfcf680 0x55be5d048d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:32.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.061+0000 7f68bdb0f640 1 -- 172.21.15.16:0/710354206 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be5d049260 con 0x55be5cfc44a0 2024-09-12T02:13:32.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.061+0000 7f68bdb0f640 1 --2- 172.21.15.16:0/710354206 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be5cfc44a0 0x55be5cfd3780 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7f68b4009310 tx=0x7f68b40098e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c5c09976b8503b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:32.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.061+0000 7f68a17fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:13:32.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.061+0000 7f68a17fa640 1 -- 172.21.15.16:0/710354206 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68b402c060 con 0x55be5cfc44a0 2024-09-12T02:13:32.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.061+0000 7f68a17fa640 1 -- 172.21.15.16:0/710354206 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f68b4005dd0 con 0x55be5cfc44a0 2024-09-12T02:13:32.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.061+0000 7f68c214c740 1 -- 172.21.15.16:0/710354206 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68a80bfb40 con 0x55be5cfc44a0 2024-09-12T02:13:32.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.061+0000 7f68a17fa640 1 -- 172.21.15.16:0/710354206 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f68b4017040 con 0x55be5cfc44a0 2024-09-12T02:13:32.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.061+0000 7f68c214c740 10 client.19350 fetch_fsmap learned FSMap version 25 2024-09-12T02:13:32.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.061+0000 7f68c214c740 1 -- 172.21.15.16:0/710354206 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55be5d04a380 con 0x55be5cfc44a0 2024-09-12T02:13:32.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.062+0000 7f68a17fa640 1 -- 172.21.15.16:0/710354206 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68a80bfb40 con 0x55be5cfc44a0 2024-09-12T02:13:32.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.062+0000 7f68a17fa640 1 -- 172.21.15.16:0/710354206 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f68b4031070 con 0x55be5cfc44a0 2024-09-12T02:13:32.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.062+0000 7f68c214c740 10 client.19350 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:13:32.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.062+0000 7f68c214c740 10 client.19350 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:13:32.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.062+0000 7f68c214c740 20 client.19350 populate_metadata read hostname 'smithi016' 2024-09-12T02:13:32.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.062+0000 7f68c214c740 1 --2- 172.21.15.16:0/710354206 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55be5d04a7d0 0x55be5d04cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:32.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.062+0000 7f68c214c740 4 client.19350 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T02:13:32.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.062+0000 7f68c214c740 1 -- 172.21.15.16:0/710354206 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55be5cf31d60 con 0x55be5d04a7d0 2024-09-12T02:13:32.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.062+0000 7f68bd30e640 1 --2- 172.21.15.16:0/710354206 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55be5d04a7d0 0x55be5d04cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:32.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.062+0000 7f68a17fa640 10 client.19350 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:13:32.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.062+0000 7f68bd30e640 1 --2- 172.21.15.16:0/710354206 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55be5d04a7d0 0x55be5d04cbc0 crc :-1 s=READY pgs=355 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=141eaada824b88f3 server_cookie=6a691579885f1704 in_seq=0 out_seq=0 2024-09-12T02:13:32.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.063+0000 7f68a17fa640 1 -- 172.21.15.16:0/710354206 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55be5cf31d60 con 0x55be5d04a7d0 2024-09-12T02:13:32.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.063+0000 7f68a17fa640 10 client.19350 handle_command_reply: tid=0 2024-09-12T02:13:32.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.063+0000 7f68c214c740 1 client.19350 shutdown 2024-09-12T02:13:32.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.063+0000 7f68c214c740 1 -- 172.21.15.16:0/710354206 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be5cfc44a0 msgr2=0x55be5cfd3780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:32.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.063+0000 7f68c214c740 1 --2- 172.21.15.16:0/710354206 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be5cfc44a0 0x55be5cfd3780 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7f68b4009310 tx=0x7f68b40098e0 comp rx=0 tx=0).stop 2024-09-12T02:13:32.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.063+0000 7f68c214c740 1 -- 172.21.15.16:0/710354206 shutdown_connections 2024-09-12T02:13:32.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.063+0000 7f68c214c740 1 --2- 172.21.15.16:0/710354206 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55be5d04a7d0 0x55be5d04cbc0 crc :-1 s=READY pgs=355 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:32.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.063+0000 7f68c214c740 1 --2- 172.21.15.16:0/710354206 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be5cfcf680 0x55be5d048d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:32.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.063+0000 7f68a17fa640 0 client.19350 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:13:32.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.063+0000 7f68c214c740 1 --2- 172.21.15.16:0/710354206 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55be5cfc4dc0 0x55be5cfd3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:32.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.063+0000 7f68c214c740 1 --2- 172.21.15.16:0/710354206 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be5cfc44a0 0x55be5cfd3780 unknown :-1 s=CLOSED pgs=1665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:32.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.063+0000 7f68c214c740 1 -- 172.21.15.16:0/710354206 >> 172.21.15.16:0/710354206 conn(0x55be5cfc0a30 msgr2=0x55be5cfce300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:32.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.063+0000 7f68c214c740 1 -- 172.21.15.16:0/710354206 shutdown_connections 2024-09-12T02:13:32.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.063+0000 7f68c214c740 1 -- 172.21.15.16:0/710354206 wait complete. 2024-09-12T02:13:32.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.063+0000 7f68c214c740 20 client.19350 trim_cache size 0 max 16384 2024-09-12T02:13:32.071 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T02:13:32.071 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-09-12T02:13:32.071 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {} 2024-09-12T02:13:32.071 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T02:13:32.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.064+0000 7f68bfd73640 1 -- 172.21.15.16:0/4067187488 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f68a40a2c00 msgr2=0x7f68a40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:32.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.064+0000 7f68bfd73640 1 --2- 172.21.15.16:0/4067187488 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f68a40a2c00 0x7f68a40a50c0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f68b00070b0 tx=0x7f68b00093d0 comp rx=0 tx=0).stop 2024-09-12T02:13:32.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.064+0000 7f68bfd73640 1 -- 172.21.15.16:0/4067187488 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68b81036e0 msgr2=0x7f68b8198c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:32.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.064+0000 7f68bfd73640 1 --2- 172.21.15.16:0/4067187488 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68b81036e0 0x7f68b8198c90 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7f68a802a940 tx=0x7f68a803ac80 comp rx=0 tx=0).stop 2024-09-12T02:13:32.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.064+0000 7f68bfd73640 1 -- 172.21.15.16:0/4067187488 shutdown_connections 2024-09-12T02:13:32.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.064+0000 7f68bfd73640 1 --2- 172.21.15.16:0/4067187488 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f68a40a2c00 0x7f68a40a50c0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:32.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.064+0000 7f68bfd73640 1 --2- 172.21.15.16:0/4067187488 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68b81082a0 0x7f68b819d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:32.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.064+0000 7f68bfd73640 1 --2- 172.21.15.16:0/4067187488 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68b81036e0 0x7f68b8198c90 unknown :-1 s=CLOSED pgs=1662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:32.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.064+0000 7f68bfd73640 1 --2- 172.21.15.16:0/4067187488 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f68b8102d30 0x7f68b8198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:32.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.064+0000 7f68bfd73640 1 -- 172.21.15.16:0/4067187488 >> 172.21.15.16:0/4067187488 conn(0x7f68b80fc7d0 msgr2=0x7f68b8105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:32.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.064+0000 7f68bfd73640 1 -- 172.21.15.16:0/4067187488 shutdown_connections 2024-09-12T02:13:32.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:32.064+0000 7f68bfd73640 1 -- 172.21.15.16:0/4067187488 wait complete. 2024-09-12T02:13:32.284 INFO:tasks.workunit.client.0.smithi016.stdout:375/600: open fd = 3 2024-09-12T02:13:32.284 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:13:32.284 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:13:32.284 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:13:32.284 INFO:tasks.workunit.client.0.smithi016.stdout:376/600: open fd = 3 2024-09-12T02:13:32.285 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:13:32.285 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:13:32.285 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:13:32.285 INFO:tasks.workunit.client.0.smithi016.stdout:377/600: open fd = 3 2024-09-12T02:13:32.285 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:13:32.285 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:13:32.285 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:13:32.285 INFO:tasks.workunit.client.0.smithi016.stdout:378/600: open fd = 3 2024-09-12T02:13:32.285 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:13:32.285 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:13:32.285 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:13:32.285 INFO:tasks.workunit.client.0.smithi016.stdout:379/600: open fd = 3 2024-09-12T02:13:32.286 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:13:32.286 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:13:32.286 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:13:32.286 INFO:tasks.workunit.client.0.smithi016.stdout:380/600: open fd = 3 2024-09-12T02:13:32.286 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:13:32.286 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:13:32.286 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:13:32.286 INFO:tasks.workunit.client.0.smithi016.stdout:381/600: open fd = 3 2024-09-12T02:13:32.286 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:13:32.286 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:13:32.286 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:13:32.286 INFO:tasks.workunit.client.0.smithi016.stdout:382/600: open fd = 3 2024-09-12T02:13:32.286 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:13:32.286 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:13:32.287 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:13:32.287 INFO:tasks.workunit.client.0.smithi016.stdout:383/600: open fd = 3 2024-09-12T02:13:32.287 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:13:32.287 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:13:32.287 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:13:32.287 INFO:tasks.workunit.client.0.smithi016.stdout:384/600: open fd = 3 2024-09-12T02:13:32.287 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:13:32.287 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:13:32.287 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:13:32.287 INFO:tasks.workunit.client.0.smithi016.stdout:385/600: open fd = 3 2024-09-12T02:13:32.287 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:13:32.287 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:13:32.287 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:13:32.287 INFO:tasks.workunit.client.0.smithi016.stdout:386/600: open fd = 3 2024-09-12T02:13:32.288 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:13:32.288 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:13:32.288 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:13:32.288 INFO:tasks.workunit.client.0.smithi016.stdout:387/600: open fd = 3 2024-09-12T02:13:32.288 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:13:32.288 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:13:32.288 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:13:32.288 INFO:tasks.workunit.client.0.smithi016.stdout:388/600: open fd = 3 2024-09-12T02:13:32.288 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:13:32.288 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:13:32.288 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:13:32.288 INFO:tasks.workunit.client.0.smithi016.stdout:389/600: open fd = 3 2024-09-12T02:13:32.288 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:13:32.289 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:13:32.289 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:13:32.289 INFO:tasks.workunit.client.0.smithi016.stdout:390/600: open fd = 3 2024-09-12T02:13:32.289 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:13:32.289 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:13:32.289 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:13:32.289 INFO:tasks.workunit.client.0.smithi016.stdout:391/600: open fd = 3 2024-09-12T02:13:32.289 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:13:32.289 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:13:32.289 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:13:32.289 INFO:tasks.workunit.client.0.smithi016.stdout:392/600: open fd = 3 2024-09-12T02:13:32.290 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:13:32.290 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:13:32.290 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:13:32.290 INFO:tasks.workunit.client.0.smithi016.stdout:393/600: open fd = 3 2024-09-12T02:13:32.290 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:13:32.290 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:13:32.290 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:13:32.290 INFO:tasks.workunit.client.0.smithi016.stdout:394/600: open fd = 3 2024-09-12T02:13:32.290 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:13:32.290 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:13:32.290 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:13:32.290 INFO:tasks.workunit.client.0.smithi016.stdout:395/600: open fd = 3 2024-09-12T02:13:32.290 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:13:32.290 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:13:32.291 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:13:32.291 INFO:tasks.workunit.client.0.smithi016.stdout:396/600: open fd = 3 2024-09-12T02:13:32.291 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:13:32.291 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:13:32.291 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:13:32.291 INFO:tasks.workunit.client.0.smithi016.stdout:397/600: open fd = 3 2024-09-12T02:13:32.291 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:13:32.291 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:13:32.291 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:13:32.291 INFO:tasks.workunit.client.0.smithi016.stdout:398/600: open fd = 3 2024-09-12T02:13:32.291 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:13:32.291 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:13:32.291 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:13:32.291 INFO:tasks.workunit.client.0.smithi016.stdout:399/600: open fd = 3 2024-09-12T02:13:32.292 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:13:32.292 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:13:32.292 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:13:32.292 INFO:tasks.workunit.client.0.smithi016.stdout:400/600: open fd = 3 2024-09-12T02:13:32.292 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:13:32.292 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:13:32.292 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:13:32.292 INFO:tasks.workunit.client.0.smithi016.stdout:401/600: open fd = 3 2024-09-12T02:13:32.292 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:13:32.292 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:13:32.292 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:13:32.292 INFO:tasks.workunit.client.0.smithi016.stdout:402/600: open fd = 3 2024-09-12T02:13:32.292 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:13:32.292 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:13:32.293 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:13:32.293 INFO:tasks.workunit.client.0.smithi016.stdout:403/600: open fd = 3 2024-09-12T02:13:32.293 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:13:32.293 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:13:32.293 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:13:32.293 INFO:tasks.workunit.client.0.smithi016.stdout:404/600: open fd = 3 2024-09-12T02:13:32.293 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:13:32.293 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:13:32.293 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:13:32.293 INFO:tasks.workunit.client.0.smithi016.stdout:405/600: open fd = 3 2024-09-12T02:13:32.293 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:13:32.293 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:13:32.293 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:13:32.293 INFO:tasks.workunit.client.0.smithi016.stdout:406/600: open fd = 3 2024-09-12T02:13:32.294 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:13:32.294 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:13:32.294 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:13:32.294 INFO:tasks.workunit.client.0.smithi016.stdout:407/600: open fd = 3 2024-09-12T02:13:32.294 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:13:32.294 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:13:32.294 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:13:32.294 INFO:tasks.workunit.client.0.smithi016.stdout:408/600: open fd = 3 2024-09-12T02:13:32.294 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:13:32.294 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:13:32.294 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:13:32.294 INFO:tasks.workunit.client.0.smithi016.stdout:409/600: open fd = 3 2024-09-12T02:13:32.294 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:13:32.294 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:13:32.294 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:13:32.295 INFO:tasks.workunit.client.0.smithi016.stdout:410/600: open fd = 3 2024-09-12T02:13:32.295 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:13:32.295 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:13:32.295 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:13:32.295 INFO:tasks.workunit.client.0.smithi016.stdout:411/600: open fd = 3 2024-09-12T02:13:32.295 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:13:32.295 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:13:32.295 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:13:32.295 INFO:tasks.workunit.client.0.smithi016.stdout:412/600: open fd = 3 2024-09-12T02:13:32.295 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:13:32.295 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:13:32.295 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:13:32.295 INFO:tasks.workunit.client.0.smithi016.stdout:413/600: open fd = 3 2024-09-12T02:13:32.295 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:13:32.296 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:13:32.296 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:13:32.296 INFO:tasks.workunit.client.0.smithi016.stdout:414/600: open fd = 3 2024-09-12T02:13:32.296 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:13:32.296 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:13:32.296 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:13:32.296 INFO:tasks.workunit.client.0.smithi016.stdout:415/600: open fd = 3 2024-09-12T02:13:32.296 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:13:32.296 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:13:32.296 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:13:32.296 INFO:tasks.workunit.client.0.smithi016.stdout:416/600: open fd = 3 2024-09-12T02:13:32.296 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:13:32.296 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:13:32.296 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:13:32.297 INFO:tasks.workunit.client.0.smithi016.stdout:417/600: open fd = 3 2024-09-12T02:13:32.297 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:13:32.297 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:13:32.297 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:13:32.297 INFO:tasks.workunit.client.0.smithi016.stdout:418/600: open fd = 3 2024-09-12T02:13:32.297 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:13:32.297 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:13:32.297 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:13:32.297 INFO:tasks.workunit.client.0.smithi016.stdout:419/600: open fd = 3 2024-09-12T02:13:32.298 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:13:32.298 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:13:32.298 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:13:32.298 INFO:tasks.workunit.client.0.smithi016.stdout:420/600: open fd = 3 2024-09-12T02:13:32.298 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:13:32.298 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:13:32.298 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:13:32.298 INFO:tasks.workunit.client.0.smithi016.stdout:421/600: open fd = 3 2024-09-12T02:13:32.298 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:13:32.299 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:13:32.299 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:13:32.299 INFO:tasks.workunit.client.0.smithi016.stdout:422/600: open fd = 3 2024-09-12T02:13:32.299 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:13:32.299 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:13:32.299 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:13:32.299 INFO:tasks.workunit.client.0.smithi016.stdout:423/600: open fd = 3 2024-09-12T02:13:32.299 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:13:32.299 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:13:32.299 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:13:32.299 INFO:tasks.workunit.client.0.smithi016.stdout:424/600: open fd = 3 2024-09-12T02:13:32.299 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:13:32.299 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:13:32.299 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:13:32.300 INFO:tasks.workunit.client.0.smithi016.stdout:425/600: open fd = 3 2024-09-12T02:13:32.300 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:13:32.300 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:13:32.300 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:13:32.300 INFO:tasks.workunit.client.0.smithi016.stdout:426/600: open fd = 3 2024-09-12T02:13:32.300 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:13:32.300 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:13:32.300 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:13:32.300 INFO:tasks.workunit.client.0.smithi016.stdout:427/600: open fd = 3 2024-09-12T02:13:32.300 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:13:32.300 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:13:32.300 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:13:32.300 INFO:tasks.workunit.client.0.smithi016.stdout:428/600: open fd = 3 2024-09-12T02:13:32.301 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:13:32.301 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:13:32.301 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:13:32.301 INFO:tasks.workunit.client.0.smithi016.stdout:429/600: open fd = 3 2024-09-12T02:13:32.301 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:13:32.301 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:13:32.301 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:13:32.301 INFO:tasks.workunit.client.0.smithi016.stdout:430/600: open fd = 3 2024-09-12T02:13:32.301 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:13:32.301 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:13:32.301 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:13:32.301 INFO:tasks.workunit.client.0.smithi016.stdout:431/600: open fd = 3 2024-09-12T02:13:32.301 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:13:32.302 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:13:32.302 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:13:32.302 INFO:tasks.workunit.client.0.smithi016.stdout:432/600: open fd = 3 2024-09-12T02:13:32.302 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:13:32.302 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:13:32.302 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:13:32.302 INFO:tasks.workunit.client.0.smithi016.stdout:433/600: open fd = 3 2024-09-12T02:13:32.302 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:13:32.302 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:13:32.302 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:13:32.302 INFO:tasks.workunit.client.0.smithi016.stdout:434/600: open fd = 3 2024-09-12T02:13:32.302 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:13:32.302 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:13:32.302 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:13:32.302 INFO:tasks.workunit.client.0.smithi016.stdout:435/600: open fd = 3 2024-09-12T02:13:32.303 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:13:32.303 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:13:32.303 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:13:32.303 INFO:tasks.workunit.client.0.smithi016.stdout:436/600: open fd = 3 2024-09-12T02:13:32.303 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:13:32.303 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:13:32.303 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:13:32.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:13:32 smithi184 ceph-mon[35593]: pgmap v683: 97 pgs: 97 active+clean; 2.1 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-09-12T02:13:32.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:13:32 smithi184 ceph-mon[35593]: scrub queued for path: / 2024-09-12T02:13:32.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:13:32 smithi184 ceph-mon[35593]: scrub summary: idle+waiting paths [/] 2024-09-12T02:13:32.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:13:32 smithi184 ceph-mon[35593]: scrub summary: active paths [/] 2024-09-12T02:13:32.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:13:32 smithi184 ceph-mon[35593]: scrub summary: idle+waiting paths [/] 2024-09-12T02:13:32.378 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:13:32 smithi184 ceph-mon[35593]: scrub summary: idle 2024-09-12T02:13:32.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:13:32 smithi033 ceph-mon[35332]: pgmap v683: 97 pgs: 97 active+clean; 2.1 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-09-12T02:13:32.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:13:32 smithi033 ceph-mon[35332]: scrub queued for path: / 2024-09-12T02:13:32.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:13:32 smithi033 ceph-mon[35332]: scrub summary: idle+waiting paths [/] 2024-09-12T02:13:32.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:13:32 smithi033 ceph-mon[35332]: scrub summary: active paths [/] 2024-09-12T02:13:32.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:13:32 smithi033 ceph-mon[35332]: scrub summary: idle+waiting paths [/] 2024-09-12T02:13:32.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:13:32 smithi033 ceph-mon[35332]: scrub summary: idle 2024-09-12T02:13:32.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:13:32 smithi016 ceph-mon[30947]: pgmap v683: 97 pgs: 97 active+clean; 2.1 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-09-12T02:13:32.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:13:32 smithi016 ceph-mon[30947]: scrub queued for path: / 2024-09-12T02:13:32.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:13:32 smithi016 ceph-mon[30947]: scrub summary: idle+waiting paths [/] 2024-09-12T02:13:32.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:13:32 smithi016 ceph-mon[30947]: scrub summary: active paths [/] 2024-09-12T02:13:32.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:13:32 smithi016 ceph-mon[30947]: scrub summary: idle+waiting paths [/] 2024-09-12T02:13:32.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:13:32 smithi016 ceph-mon[30947]: scrub summary: idle 2024-09-12T02:13:32.974 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-12T02:13:32.975 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:13:33.238 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:13:33.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:33.710+0000 7fac2b8cd640 1 -- 172.21.15.16:0/3175555506 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac24106a80 msgr2=0x7fac24106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:33.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:33.710+0000 7fac2b8cd640 1 --2- 172.21.15.16:0/3175555506 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac24106a80 0x7fac24106e60 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7fac20009930 tx=0x7fac2002a420 comp rx=0 tx=0).stop 2024-09-12T02:13:33.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:33.710+0000 7fac2b8cd640 1 -- 172.21.15.16:0/3175555506 shutdown_connections 2024-09-12T02:13:33.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:33.710+0000 7fac2b8cd640 1 --2- 172.21.15.16:0/3175555506 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fac24101420 0x7fac2410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:33.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:33.710+0000 7fac2b8cd640 1 --2- 172.21.15.16:0/3175555506 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac24100a80 0x7fac24100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:33.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:33.710+0000 7fac2b8cd640 1 --2- 172.21.15.16:0/3175555506 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac24106a80 0x7fac24106e60 unknown :-1 s=CLOSED pgs=1666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:33.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:33.710+0000 7fac2b8cd640 1 -- 172.21.15.16:0/3175555506 >> 172.21.15.16:0/3175555506 conn(0x7fac240fc7a0 msgr2=0x7fac240febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:33.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:33.710+0000 7fac2b8cd640 1 -- 172.21.15.16:0/3175555506 shutdown_connections 2024-09-12T02:13:33.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:33.710+0000 7fac2b8cd640 1 -- 172.21.15.16:0/3175555506 wait complete. 2024-09-12T02:13:33.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:33.711+0000 7fac2b8cd640 1 Processor -- start 2024-09-12T02:13:33.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:33.711+0000 7fac2b8cd640 1 -- start start 2024-09-12T02:13:33.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:33.711+0000 7fac2b8cd640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fac24100a80 0x7fac241992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:33.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:33.711+0000 7fac2b8cd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac24101420 0x7fac24199830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:33.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:33.711+0000 7fac2b8cd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac24106a80 0x7fac241a4370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:33.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:33.711+0000 7fac2b8cd640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fac24110e40 con 0x7fac24106a80 2024-09-12T02:13:33.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:33.711+0000 7fac2b8cd640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fac24110cc0 con 0x7fac24101420 2024-09-12T02:13:33.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:33.711+0000 7fac2b8cd640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fac24110fc0 con 0x7fac24100a80 2024-09-12T02:13:33.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:33.711+0000 7fac29e6a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac24106a80 0x7fac241a4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:33.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:33.711+0000 7fac29e6a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac24106a80 0x7fac241a4370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52136/0 (socket says 172.21.15.16:52136) 2024-09-12T02:13:33.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:33.711+0000 7fac29e6a640 1 -- 172.21.15.16:0/4278916595 learned_addr learned my addr 172.21.15.16:0/4278916595 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:13:33.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:33.711+0000 7fac28e68640 1 --2- 172.21.15.16:0/4278916595 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac24101420 0x7fac24199830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:33.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:33.711+0000 7fac29669640 1 --2- 172.21.15.16:0/4278916595 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fac24100a80 0x7fac241992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:33.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:33.711+0000 7fac29e6a640 1 -- 172.21.15.16:0/4278916595 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fac24100a80 msgr2=0x7fac241992f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:33.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:33.711+0000 7fac29e6a640 1 --2- 172.21.15.16:0/4278916595 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fac24100a80 0x7fac241992f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:33.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:33.711+0000 7fac29e6a640 1 -- 172.21.15.16:0/4278916595 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac24101420 msgr2=0x7fac24199830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:33.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:33.711+0000 7fac29e6a640 1 --2- 172.21.15.16:0/4278916595 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac24101420 0x7fac24199830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:33.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:33.711+0000 7fac29e6a640 1 -- 172.21.15.16:0/4278916595 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac241a48b0 con 0x7fac24106a80 2024-09-12T02:13:33.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:33.712+0000 7fac29e6a640 1 --2- 172.21.15.16:0/4278916595 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac24106a80 0x7fac241a4370 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7fac1c00cad0 tx=0x7fac1c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:33.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:33.712+0000 7fac1a7fc640 1 -- 172.21.15.16:0/4278916595 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac1c00ae20 con 0x7fac24106a80 2024-09-12T02:13:33.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:33.712+0000 7fac2b8cd640 1 -- 172.21.15.16:0/4278916595 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac241a4a80 con 0x7fac24106a80 2024-09-12T02:13:33.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:33.712+0000 7fac1a7fc640 1 -- 172.21.15.16:0/4278916595 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac1c005020 con 0x7fac24106a80 2024-09-12T02:13:33.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:33.712+0000 7fac1a7fc640 1 -- 172.21.15.16:0/4278916595 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac1c00f6f0 con 0x7fac24106a80 2024-09-12T02:13:33.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:33.712+0000 7fac2b8cd640 1 -- 172.21.15.16:0/4278916595 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac241a4f90 con 0x7fac24106a80 2024-09-12T02:13:33.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:33.713+0000 7fac2b8cd640 1 -- 172.21.15.16:0/4278916595 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac241021c0 con 0x7fac24106a80 2024-09-12T02:13:33.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:33.714+0000 7fac1a7fc640 1 -- 172.21.15.16:0/4278916595 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fac1c0051d0 con 0x7fac24106a80 2024-09-12T02:13:33.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:33.715+0000 7fac1a7fc640 1 --2- 172.21.15.16:0/4278916595 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fac100a2b80 0x7fac100a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:33.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:33.715+0000 7fac1a7fc640 1 -- 172.21.15.16:0/4278916595 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fac1c0cefb0 con 0x7fac24106a80 2024-09-12T02:13:33.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:33.715+0000 7fac29669640 1 --2- 172.21.15.16:0/4278916595 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fac100a2b80 0x7fac100a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:33.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:33.715+0000 7fac29669640 1 --2- 172.21.15.16:0/4278916595 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fac100a2b80 0x7fac100a5040 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fac20000c00 tx=0x7fac20005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:33.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:33.719+0000 7fac1a7fc640 1 -- 172.21.15.16:0/4278916595 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fac1c014030 con 0x7fac24106a80 2024-09-12T02:13:33.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:33.930+0000 7fac2b8cd640 1 -- 172.21.15.16:0/4278916595 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fac24108ff0 con 0x7fac24106a80 2024-09-12T02:13:33.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:33.931+0000 7fac1a7fc640 1 -- 172.21.15.16:0/4278916595 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11911 (secure 0 0 0) 0x7fac1c09bfb0 con 0x7fac24106a80 2024-09-12T02:13:33.932 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T02:13:33.932 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":25,"btime":"2024-09-12T02:01:37:192021+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3279123279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3279123279},{"type":"v1","addr":"172.21.15.16:6835","nonce":3279123279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4119452382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4119452382},{"type":"v1","addr":"172.21.15.16:6837","nonce":4119452382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/3263823165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3263823165},{"type":"v1","addr":"172.21.15.16:6839","nonce":3263823165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/1510290732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":1510290732},{"type":"v1","addr":"172.21.15.16:6841","nonce":1510290732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/279674603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":279674603},{"type":"v1","addr":"172.21.15.33:6833","nonce":279674603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3555728028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3555728028},{"type":"v1","addr":"172.21.15.33:6835","nonce":3555728028}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T01:59:56.117608+0000","modified":"2024-09-12T02:01:36.294760+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24482,"mds_1":24481,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6833/1563374106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1563374106},{"type":"v1","addr":"172.21.15.184:6833","nonce":1563374106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6837/1892833868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1892833868},{"type":"v1","addr":"172.21.15.33:6837","nonce":1892833868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/1457679686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1457679686},{"type":"v1","addr":"172.21.15.184:6835","nonce":1457679686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.184:6837/1793393087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":1793393087},{"type":"v1","addr":"172.21.15.184:6837","nonce":1793393087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.184:6839/4203347941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":4203347941},{"type":"v1","addr":"172.21.15.184:6839","nonce":4203347941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.33:6839/4136670204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":4136670204},{"type":"v1","addr":"172.21.15.33:6839","nonce":4136670204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24481,24482,24469]},"id":1}]} 2024-09-12T02:13:33.932 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 25 2024-09-12T02:13:33.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:33.933+0000 7fac2b8cd640 1 -- 172.21.15.16:0/4278916595 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fac100a2b80 msgr2=0x7fac100a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:33.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:33.933+0000 7fac2b8cd640 1 --2- 172.21.15.16:0/4278916595 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fac100a2b80 0x7fac100a5040 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fac20000c00 tx=0x7fac20005a20 comp rx=0 tx=0).stop 2024-09-12T02:13:33.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:33.933+0000 7fac2b8cd640 1 -- 172.21.15.16:0/4278916595 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac24106a80 msgr2=0x7fac241a4370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:33.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:33.933+0000 7fac2b8cd640 1 --2- 172.21.15.16:0/4278916595 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac24106a80 0x7fac241a4370 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7fac1c00cad0 tx=0x7fac1c00cf90 comp rx=0 tx=0).stop 2024-09-12T02:13:33.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:33.933+0000 7fac2b8cd640 1 -- 172.21.15.16:0/4278916595 shutdown_connections 2024-09-12T02:13:33.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:33.933+0000 7fac2b8cd640 1 --2- 172.21.15.16:0/4278916595 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fac100a2b80 0x7fac100a5040 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:33.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:33.933+0000 7fac2b8cd640 1 --2- 172.21.15.16:0/4278916595 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac24106a80 0x7fac241a4370 unknown :-1 s=CLOSED pgs=1667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:33.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:33.933+0000 7fac2b8cd640 1 --2- 172.21.15.16:0/4278916595 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac24101420 0x7fac24199830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:33.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:33.933+0000 7fac2b8cd640 1 --2- 172.21.15.16:0/4278916595 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fac24100a80 0x7fac241992f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:33.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:33.933+0000 7fac2b8cd640 1 -- 172.21.15.16:0/4278916595 >> 172.21.15.16:0/4278916595 conn(0x7fac240fc7a0 msgr2=0x7fac240fd050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:33.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:33.933+0000 7fac2b8cd640 1 -- 172.21.15.16:0/4278916595 shutdown_connections 2024-09-12T02:13:33.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:33.933+0000 7fac2b8cd640 1 -- 172.21.15.16:0/4278916595 wait complete. 2024-09-12T02:13:34.183 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:13:34 smithi016 ceph-mon[30947]: pgmap v684: 97 pgs: 97 active+clean; 2.2 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 17 op/s 2024-09-12T02:13:34.183 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:13:34 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/4278916595' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:13:34.228 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.225+0000 7f242c2a6640 1 -- 172.21.15.16:0/444702135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2424103710 msgr2=0x7f2424109fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:34.228 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.225+0000 7f242c2a6640 1 --2- 172.21.15.16:0/444702135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2424103710 0x7f2424109fa0 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7f241c009ca0 tx=0x7f241c02a8c0 comp rx=0 tx=0).stop 2024-09-12T02:13:34.228 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.225+0000 7f242c2a6640 1 -- 172.21.15.16:0/444702135 shutdown_connections 2024-09-12T02:13:34.228 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.225+0000 7f242c2a6640 1 --2- 172.21.15.16:0/444702135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2424103710 0x7f2424109fa0 unknown :-1 s=CLOSED pgs=1668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:34.228 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.225+0000 7f242c2a6640 1 --2- 172.21.15.16:0/444702135 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2424102d50 0x7f24241031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:34.228 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.225+0000 7f242c2a6640 1 --2- 172.21.15.16:0/444702135 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2424101b60 0x7f2424101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:34.228 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.225+0000 7f242c2a6640 1 -- 172.21.15.16:0/444702135 >> 172.21.15.16:0/444702135 conn(0x7f24240fd2e0 msgr2=0x7f24240ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:34.229 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.225+0000 7f242c2a6640 1 -- 172.21.15.16:0/444702135 shutdown_connections 2024-09-12T02:13:34.229 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.225+0000 7f242c2a6640 1 -- 172.21.15.16:0/444702135 wait complete. 2024-09-12T02:13:34.229 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.225+0000 7f242c2a6640 1 Processor -- start 2024-09-12T02:13:34.229 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.225+0000 7f242c2a6640 1 -- start start 2024-09-12T02:13:34.229 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.226+0000 7f242c2a6640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2424101b60 0x7f2424075fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:34.229 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.226+0000 7f242c2a6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2424102d50 0x7f24240745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:34.229 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.226+0000 7f242c2a6640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2424103710 0x7f2424074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:34.229 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.226+0000 7f242c2a6640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f242410d7e0 con 0x7f2424102d50 2024-09-12T02:13:34.229 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.226+0000 7f242c2a6640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f242410d660 con 0x7f2424101b60 2024-09-12T02:13:34.229 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.226+0000 7f242c2a6640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f242410d960 con 0x7f2424103710 2024-09-12T02:13:34.229 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.226+0000 7f2429841640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2424102d50 0x7f24240745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:34.230 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.226+0000 7f2429841640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2424102d50 0x7f24240745e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52148/0 (socket says 172.21.15.16:52148) 2024-09-12T02:13:34.230 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.226+0000 7f2429841640 1 -- 172.21.15.16:0/2666935983 learned_addr learned my addr 172.21.15.16:0/2666935983 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:13:34.230 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.226+0000 7f242a843640 1 --2- 172.21.15.16:0/2666935983 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2424103710 0x7f2424074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:34.230 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.226+0000 7f242a042640 1 --2- 172.21.15.16:0/2666935983 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2424101b60 0x7f2424075fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:34.230 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.226+0000 7f2429841640 1 -- 172.21.15.16:0/2666935983 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2424103710 msgr2=0x7f2424074b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:34.230 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.226+0000 7f2429841640 1 --2- 172.21.15.16:0/2666935983 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2424103710 0x7f2424074b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:34.230 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.226+0000 7f2429841640 1 -- 172.21.15.16:0/2666935983 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2424101b60 msgr2=0x7f2424075fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:34.230 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.226+0000 7f2429841640 1 --2- 172.21.15.16:0/2666935983 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2424101b60 0x7f2424075fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:34.230 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.226+0000 7f2429841640 1 -- 172.21.15.16:0/2666935983 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24240753b0 con 0x7f2424102d50 2024-09-12T02:13:34.230 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.226+0000 7f242a042640 1 --2- 172.21.15.16:0/2666935983 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2424101b60 0x7f2424075fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:13:34.230 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.226+0000 7f2429841640 1 --2- 172.21.15.16:0/2666935983 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2424102d50 0x7f24240745e0 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7f241400c970 tx=0x7f241400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:34.230 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.226+0000 7f241b7fe640 1 -- 172.21.15.16:0/2666935983 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2414007bb0 con 0x7f2424102d50 2024-09-12T02:13:34.230 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.227+0000 7f241b7fe640 1 -- 172.21.15.16:0/2666935983 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2414007d50 con 0x7f2424102d50 2024-09-12T02:13:34.230 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.227+0000 7f241b7fe640 1 -- 172.21.15.16:0/2666935983 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f241400f450 con 0x7f2424102d50 2024-09-12T02:13:34.231 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.227+0000 7f242c2a6640 1 -- 172.21.15.16:0/2666935983 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24241a1ef0 con 0x7f2424102d50 2024-09-12T02:13:34.231 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.227+0000 7f242c2a6640 1 -- 172.21.15.16:0/2666935983 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24241a2300 con 0x7f2424102d50 2024-09-12T02:13:34.237 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.229+0000 7f241b7fe640 1 -- 172.21.15.16:0/2666935983 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f2414005020 con 0x7f2424102d50 2024-09-12T02:13:34.237 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.229+0000 7f242f038740 1 Processor -- start 2024-09-12T02:13:34.237 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.230+0000 7f242f038740 1 -- start start 2024-09-12T02:13:34.237 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.230+0000 7f242f038740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555c9e27a9a0 0x555c9e27ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:34.238 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.230+0000 7f242f038740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555c9e27b2e0 0x555c9e27fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:34.238 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.230+0000 7f242f038740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c9e280370 0x555c9e282830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:34.238 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.230+0000 7f242f038740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x555c9e1de2f0 con 0x555c9e280370 2024-09-12T02:13:34.238 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.230+0000 7f242f038740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x555c9e186270 con 0x555c9e27b2e0 2024-09-12T02:13:34.238 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.230+0000 7f242f038740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x555c9e1da2b0 con 0x555c9e27a9a0 2024-09-12T02:13:34.238 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.230+0000 7f2429841640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c9e280370 0x555c9e282830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:34.238 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.230+0000 7f242a042640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555c9e27b2e0 0x555c9e27fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:34.238 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.230+0000 7f2429841640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c9e280370 0x555c9e282830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52154/0 (socket says 172.21.15.16:52154) 2024-09-12T02:13:34.238 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.230+0000 7f2429841640 1 -- 172.21.15.16:0/921394035 learned_addr learned my addr 172.21.15.16:0/921394035 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:13:34.238 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.230+0000 7f242a843640 1 --2- 172.21.15.16:0/921394035 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555c9e27a9a0 0x555c9e27ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:34.238 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.230+0000 7f241b7fe640 1 --2- 172.21.15.16:0/2666935983 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f24100a2c30 0x7f24100a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:34.238 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.230+0000 7f242a843640 1 --2- 172.21.15.16:0/2666935983 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f24100a2c30 0x7f24100a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:34.239 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.230+0000 7f2429841640 1 -- 172.21.15.16:0/921394035 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555c9e27a9a0 msgr2=0x555c9e27ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:34.239 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.230+0000 7f2429841640 1 --2- 172.21.15.16:0/921394035 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555c9e27a9a0 0x555c9e27ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:34.239 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.230+0000 7f2429841640 1 -- 172.21.15.16:0/921394035 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555c9e27b2e0 msgr2=0x555c9e27fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:34.239 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.230+0000 7f2429841640 1 --2- 172.21.15.16:0/921394035 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555c9e27b2e0 0x555c9e27fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:34.239 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.231+0000 7f2429841640 1 -- 172.21.15.16:0/921394035 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c9e185340 con 0x555c9e280370 2024-09-12T02:13:34.239 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.231+0000 7f242a042640 1 --2- 172.21.15.16:0/921394035 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555c9e27b2e0 0x555c9e27fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:13:34.239 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.231+0000 7f241b7fe640 1 -- 172.21.15.16:0/2666935983 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f24140cede0 con 0x7f2424102d50 2024-09-12T02:13:34.239 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.231+0000 7f2429841640 1 --2- 172.21.15.16:0/921394035 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c9e280370 0x555c9e282830 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7f2414053650 tx=0x7f2414004a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a1be393d186ee67 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:34.239 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.231+0000 7f2418bfe640 1 -- 172.21.15.16:0/921394035 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f241401d030 con 0x555c9e280370 2024-09-12T02:13:34.239 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.231+0000 7f2418bfe640 1 -- 172.21.15.16:0/921394035 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f241409bd80 con 0x555c9e280370 2024-09-12T02:13:34.239 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.231+0000 7f242a843640 1 --2- 172.21.15.16:0/2666935983 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f24100a2c30 0x7f24100a50f0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f241c009430 tx=0x7f241c02a8c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:34.239 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.231+0000 7f2418bfe640 1 -- 172.21.15.16:0/921394035 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f241409b680 con 0x555c9e280370 2024-09-12T02:13:34.239 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.232+0000 7f242f038740 1 -- 172.21.15.16:0/921394035 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c9e280370 msgr2=0x555c9e282830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:34.240 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.232+0000 7f242f038740 1 --2- 172.21.15.16:0/921394035 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c9e280370 0x555c9e282830 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7f2414053650 tx=0x7f2414004a20 comp rx=0 tx=0).stop 2024-09-12T02:13:34.240 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.232+0000 7f242f038740 1 -- 172.21.15.16:0/921394035 shutdown_connections 2024-09-12T02:13:34.240 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.232+0000 7f242f038740 1 --2- 172.21.15.16:0/921394035 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c9e280370 0x555c9e282830 unknown :-1 s=CLOSED pgs=1670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:34.240 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.232+0000 7f242f038740 1 --2- 172.21.15.16:0/921394035 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555c9e27b2e0 0x555c9e27fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:34.240 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.232+0000 7f242f038740 1 --2- 172.21.15.16:0/921394035 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555c9e27a9a0 0x555c9e27ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:34.240 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.232+0000 7f242f038740 1 -- 172.21.15.16:0/921394035 >> 172.21.15.16:0/921394035 conn(0x555c9e276010 msgr2=0x555c9e276410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:34.240 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.232+0000 7f242f038740 1 -- 172.21.15.16:0/921394035 shutdown_connections 2024-09-12T02:13:34.240 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.232+0000 7f242f038740 1 -- 172.21.15.16:0/921394035 wait complete. 2024-09-12T02:13:34.240 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.232+0000 7f242f038740 1 -- start start 2024-09-12T02:13:34.240 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.232+0000 7f242f038740 1 Processor -- start 2024-09-12T02:13:34.240 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.232+0000 7f242f038740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555c9e27a9a0 0x555c9e284aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:34.240 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.232+0000 7f242f038740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c9e27b2e0 0x555c9e284fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:34.240 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.232+0000 7f242f038740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555c9e280370 0x555c9e28c020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:34.240 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.232+0000 7f242f038740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x555c9e283280 con 0x555c9e27b2e0 2024-09-12T02:13:34.241 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.232+0000 7f242f038740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x555c9e167290 con 0x555c9e27a9a0 2024-09-12T02:13:34.241 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.232+0000 7f242f038740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x555c9e283400 con 0x555c9e280370 2024-09-12T02:13:34.241 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.232+0000 7f242f038740 10 client.0 _ll_register_callbacks cb 0x555c9e198140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:13:34.241 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.233+0000 7f2429841640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c9e27b2e0 0x555c9e284fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:34.241 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.233+0000 7f2429841640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c9e27b2e0 0x555c9e284fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52168/0 (socket says 172.21.15.16:52168) 2024-09-12T02:13:34.241 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.233+0000 7f2429841640 1 -- 172.21.15.16:0/228239313 learned_addr learned my addr 172.21.15.16:0/228239313 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:13:34.241 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.233+0000 7f242a042640 1 --2- 172.21.15.16:0/228239313 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555c9e27a9a0 0x555c9e284aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:34.242 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.233+0000 7f2429841640 1 -- 172.21.15.16:0/228239313 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555c9e280370 msgr2=0x555c9e28c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:34.242 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.233+0000 7f242a843640 1 --2- 172.21.15.16:0/228239313 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555c9e280370 0x555c9e28c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:34.242 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.233+0000 7f2429841640 1 --2- 172.21.15.16:0/228239313 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555c9e280370 0x555c9e28c020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:34.242 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.233+0000 7f2429841640 1 -- 172.21.15.16:0/228239313 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555c9e27a9a0 msgr2=0x555c9e284aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:34.242 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.233+0000 7f2429841640 1 --2- 172.21.15.16:0/228239313 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555c9e27a9a0 0x555c9e284aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:34.242 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.233+0000 7f2429841640 1 -- 172.21.15.16:0/228239313 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c9def1210 con 0x555c9e27b2e0 2024-09-12T02:13:34.242 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.233+0000 7f242a042640 1 --2- 172.21.15.16:0/228239313 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555c9e27a9a0 0x555c9e284aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:13:34.242 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.233+0000 7f242a843640 1 --2- 172.21.15.16:0/228239313 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555c9e280370 0x555c9e28c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:13:34.242 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.233+0000 7f2429841640 1 --2- 172.21.15.16:0/228239313 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c9e27b2e0 0x555c9e284fe0 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7f2414053620 tx=0x7f24140d2fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ee42bd8d06671b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:34.242 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.233+0000 7f240dffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:13:34.242 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.233+0000 7f240dffb640 1 -- 172.21.15.16:0/228239313 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f241401d040 con 0x555c9e27b2e0 2024-09-12T02:13:34.242 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.233+0000 7f240dffb640 1 -- 172.21.15.16:0/228239313 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f2414099980 con 0x555c9e27b2e0 2024-09-12T02:13:34.242 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.233+0000 7f242f038740 1 -- 172.21.15.16:0/228239313 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f241409bd80 con 0x555c9e27b2e0 2024-09-12T02:13:34.243 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.233+0000 7f240dffb640 1 -- 172.21.15.16:0/228239313 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f241409b450 con 0x555c9e27b2e0 2024-09-12T02:13:34.243 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.233+0000 7f242f038740 10 client.19368 fetch_fsmap learned FSMap version 25 2024-09-12T02:13:34.243 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.233+0000 7f242f038740 1 -- 172.21.15.16:0/228239313 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555c9dfbd620 con 0x555c9e27b2e0 2024-09-12T02:13:34.243 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.234+0000 7f240dffb640 1 -- 172.21.15.16:0/228239313 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f241409bd80 con 0x555c9e27b2e0 2024-09-12T02:13:34.243 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.234+0000 7f240dffb640 1 -- 172.21.15.16:0/228239313 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f2414097cf0 con 0x555c9e27b2e0 2024-09-12T02:13:34.243 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.234+0000 7f242f038740 10 client.19368 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:13:34.243 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.234+0000 7f242f038740 10 client.19368 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:13:34.243 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.234+0000 7f242f038740 20 client.19368 populate_metadata read hostname 'smithi016' 2024-09-12T02:13:34.243 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.234+0000 7f242f038740 1 --2- 172.21.15.16:0/228239313 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x555c9e290370 0x555c9e292760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:34.243 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.234+0000 7f242f038740 4 client.19368 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:13:34.243 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.234+0000 7f242f038740 1 -- 172.21.15.16:0/228239313 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555c9e1eba20 con 0x555c9e290370 2024-09-12T02:13:34.243 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.234+0000 7f242a042640 1 --2- 172.21.15.16:0/228239313 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x555c9e290370 0x555c9e292760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:34.243 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.235+0000 7f242a042640 1 --2- 172.21.15.16:0/228239313 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x555c9e290370 0x555c9e292760 crc :-1 s=READY pgs=356 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=16d0ab88e01909a8 server_cookie=ad25b525589b4c23 in_seq=0 out_seq=0 2024-09-12T02:13:34.243 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.235+0000 7f240dffb640 10 client.19368 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:13:34.244 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.235+0000 7f240dffb640 1 -- 172.21.15.16:0/228239313 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555c9e1eba20 con 0x555c9e290370 2024-09-12T02:13:34.244 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.235+0000 7f240dffb640 10 client.19368 handle_command_reply: tid=0 2024-09-12T02:13:34.244 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.235+0000 7f242f038740 1 client.19368 shutdown 2024-09-12T02:13:34.244 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.235+0000 7f242f038740 1 -- 172.21.15.16:0/228239313 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c9e27b2e0 msgr2=0x555c9e284fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:34.244 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.235+0000 7f242f038740 1 --2- 172.21.15.16:0/228239313 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c9e27b2e0 0x555c9e284fe0 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7f2414053620 tx=0x7f24140d2fd0 comp rx=0 tx=0).stop 2024-09-12T02:13:34.244 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.235+0000 7f242f038740 1 -- 172.21.15.16:0/228239313 shutdown_connections 2024-09-12T02:13:34.244 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.235+0000 7f242f038740 1 --2- 172.21.15.16:0/228239313 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x555c9e290370 0x555c9e292760 crc :-1 s=READY pgs=356 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:34.244 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.235+0000 7f242f038740 1 --2- 172.21.15.16:0/228239313 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555c9e280370 0x555c9e28c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:34.244 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.235+0000 7f240dffb640 0 client.19368 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:13:34.244 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.235+0000 7f242f038740 1 --2- 172.21.15.16:0/228239313 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c9e27b2e0 0x555c9e284fe0 unknown :-1 s=CLOSED pgs=1671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:34.245 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.235+0000 7f242f038740 1 --2- 172.21.15.16:0/228239313 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555c9e27a9a0 0x555c9e284aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:34.245 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.235+0000 7f242f038740 1 -- 172.21.15.16:0/228239313 >> 172.21.15.16:0/228239313 conn(0x555c9e276010 msgr2=0x555c9e277820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:34.245 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.235+0000 7f242f038740 1 -- 172.21.15.16:0/228239313 shutdown_connections 2024-09-12T02:13:34.245 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.235+0000 7f242f038740 1 -- 172.21.15.16:0/228239313 wait complete. 2024-09-12T02:13:34.245 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.236+0000 7f242f038740 20 client.19368 trim_cache size 0 max 16384 2024-09-12T02:13:34.256 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.250+0000 7f242f038740 1 Processor -- start 2024-09-12T02:13:34.256 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.250+0000 7f242f038740 1 -- start start 2024-09-12T02:13:34.256 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.250+0000 7f242f038740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555c9e27a9a0 0x555c9e283f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:34.256 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.250+0000 7f242f038740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c9e27b2e0 0x555c9e28bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:34.256 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.250+0000 7f242f038740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555c9e280370 0x555c9e278ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:34.256 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.250+0000 7f242f038740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f24241ad920 con 0x555c9e27b2e0 2024-09-12T02:13:34.256 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.250+0000 7f242f038740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f24241ab1f0 con 0x555c9e280370 2024-09-12T02:13:34.256 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.250+0000 7f242f038740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f24241a9060 con 0x555c9e27a9a0 2024-09-12T02:13:34.256 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.250+0000 7f2429841640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c9e27b2e0 0x555c9e28bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:34.256 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.250+0000 7f2429841640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c9e27b2e0 0x555c9e28bfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52178/0 (socket says 172.21.15.16:52178) 2024-09-12T02:13:34.257 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.250+0000 7f2429841640 1 -- 172.21.15.16:0/2556207623 learned_addr learned my addr 172.21.15.16:0/2556207623 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:13:34.257 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.251+0000 7f242a042640 1 --2- 172.21.15.16:0/2556207623 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555c9e27a9a0 0x555c9e283f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:34.257 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.251+0000 7f242a843640 1 --2- 172.21.15.16:0/2556207623 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555c9e280370 0x555c9e278ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:34.257 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.251+0000 7f2429841640 1 -- 172.21.15.16:0/2556207623 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555c9e27a9a0 msgr2=0x555c9e283f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:34.257 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.251+0000 7f2429841640 1 --2- 172.21.15.16:0/2556207623 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555c9e27a9a0 0x555c9e283f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:34.257 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.251+0000 7f2429841640 1 -- 172.21.15.16:0/2556207623 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555c9e280370 msgr2=0x555c9e278ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:34.257 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.251+0000 7f2429841640 1 --2- 172.21.15.16:0/2556207623 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555c9e280370 0x555c9e278ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:34.257 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.251+0000 7f2429841640 1 -- 172.21.15.16:0/2556207623 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c9e283580 con 0x555c9e27b2e0 2024-09-12T02:13:34.257 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.251+0000 7f2429841640 1 --2- 172.21.15.16:0/2556207623 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c9e27b2e0 0x555c9e28bfe0 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7f24140d2440 tx=0x7f24140d2900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5108084d8893689e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:34.257 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.251+0000 7f240ffff640 1 -- 172.21.15.16:0/2556207623 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f241401d030 con 0x555c9e27b2e0 2024-09-12T02:13:34.257 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.251+0000 7f240ffff640 1 -- 172.21.15.16:0/2556207623 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24140860b0 con 0x555c9e27b2e0 2024-09-12T02:13:34.258 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.251+0000 7f240ffff640 1 -- 172.21.15.16:0/2556207623 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f241409b1c0 con 0x555c9e27b2e0 2024-09-12T02:13:34.258 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.251+0000 7f242f038740 1 -- 172.21.15.16:0/2556207623 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c9e27b2e0 msgr2=0x555c9e28bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:34.258 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.251+0000 7f242f038740 1 --2- 172.21.15.16:0/2556207623 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c9e27b2e0 0x555c9e28bfe0 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7f24140d2440 tx=0x7f24140d2900 comp rx=0 tx=0).stop 2024-09-12T02:13:34.258 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.252+0000 7f242f038740 1 -- 172.21.15.16:0/2556207623 shutdown_connections 2024-09-12T02:13:34.258 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.252+0000 7f242f038740 1 --2- 172.21.15.16:0/2556207623 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555c9e280370 0x555c9e278ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:34.258 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.252+0000 7f242f038740 1 --2- 172.21.15.16:0/2556207623 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c9e27b2e0 0x555c9e28bfe0 unknown :-1 s=CLOSED pgs=1672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:34.258 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.252+0000 7f242f038740 1 --2- 172.21.15.16:0/2556207623 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555c9e27a9a0 0x555c9e283f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:34.258 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.252+0000 7f242f038740 1 -- 172.21.15.16:0/2556207623 >> 172.21.15.16:0/2556207623 conn(0x555c9e276010 msgr2=0x555c9e283800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:34.258 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.252+0000 7f242f038740 1 -- 172.21.15.16:0/2556207623 shutdown_connections 2024-09-12T02:13:34.258 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.252+0000 7f242f038740 1 -- 172.21.15.16:0/2556207623 wait complete. 2024-09-12T02:13:34.258 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.252+0000 7f242f038740 1 -- start start 2024-09-12T02:13:34.258 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.252+0000 7f242f038740 1 Processor -- start 2024-09-12T02:13:34.258 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.252+0000 7f242f038740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c9e27a9a0 0x555c9e2897e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:34.258 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.252+0000 7f242f038740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555c9e27b2e0 0x555c9e289d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:34.259 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.252+0000 7f242f038740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555c9e280370 0x555c9e28ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:34.259 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.252+0000 7f242f038740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f24241a96b0 con 0x555c9e27a9a0 2024-09-12T02:13:34.259 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.252+0000 7f242f038740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f24241aab20 con 0x555c9e27b2e0 2024-09-12T02:13:34.259 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.252+0000 7f242f038740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f24241a9d80 con 0x555c9e280370 2024-09-12T02:13:34.259 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.252+0000 7f242f038740 10 client.0 _ll_register_callbacks cb 0x7f242000b440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:13:34.259 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.252+0000 7f242a042640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c9e27a9a0 0x555c9e2897e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:34.259 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.252+0000 7f242a042640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c9e27a9a0 0x555c9e2897e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52194/0 (socket says 172.21.15.16:52194) 2024-09-12T02:13:34.259 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.252+0000 7f242a042640 1 -- 172.21.15.16:0/2663384811 learned_addr learned my addr 172.21.15.16:0/2663384811 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:13:34.259 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.252+0000 7f242a843640 1 --2- 172.21.15.16:0/2663384811 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555c9e280370 0x555c9e28ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:34.259 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.252+0000 7f2429841640 1 --2- 172.21.15.16:0/2663384811 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555c9e27b2e0 0x555c9e289d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:34.259 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.253+0000 7f242a042640 1 -- 172.21.15.16:0/2663384811 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555c9e280370 msgr2=0x555c9e28ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:34.259 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.253+0000 7f242a042640 1 --2- 172.21.15.16:0/2663384811 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555c9e280370 0x555c9e28ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:34.259 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.253+0000 7f242a042640 1 -- 172.21.15.16:0/2663384811 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555c9e27b2e0 msgr2=0x555c9e289d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:34.260 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.253+0000 7f242a042640 1 --2- 172.21.15.16:0/2663384811 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555c9e27b2e0 0x555c9e289d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:34.260 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.253+0000 7f242a042640 1 -- 172.21.15.16:0/2663384811 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c9e28b340 con 0x555c9e27a9a0 2024-09-12T02:13:34.260 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.253+0000 7f242a843640 1 --2- 172.21.15.16:0/2663384811 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555c9e280370 0x555c9e28ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:13:34.260 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.253+0000 7f2429841640 1 --2- 172.21.15.16:0/2663384811 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555c9e27b2e0 0x555c9e289d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:13:34.260 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.253+0000 7f242a042640 1 --2- 172.21.15.16:0/2663384811 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c9e27a9a0 0x555c9e2897e0 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7f2420000c00 tx=0x7f2420012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d950f8b240aeaa80 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:34.260 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.253+0000 7f240dffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:13:34.260 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.253+0000 7f240dffb640 1 -- 172.21.15.16:0/2663384811 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f242001b6a0 con 0x555c9e27a9a0 2024-09-12T02:13:34.260 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.253+0000 7f240dffb640 1 -- 172.21.15.16:0/2663384811 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f242001b840 con 0x555c9e27a9a0 2024-09-12T02:13:34.260 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.253+0000 7f242f038740 1 -- 172.21.15.16:0/2663384811 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x555c9e1e5d70 con 0x555c9e27a9a0 2024-09-12T02:13:34.260 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.253+0000 7f240dffb640 1 -- 172.21.15.16:0/2663384811 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f242001ba40 con 0x555c9e27a9a0 2024-09-12T02:13:34.260 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.253+0000 7f242f038740 10 client.19374 fetch_fsmap learned FSMap version 25 2024-09-12T02:13:34.260 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.253+0000 7f242f038740 1 -- 172.21.15.16:0/2663384811 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555c9e2fb190 con 0x555c9e27a9a0 2024-09-12T02:13:34.261 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.253+0000 7f240dffb640 1 -- 172.21.15.16:0/2663384811 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x555c9e1e5d70 con 0x555c9e27a9a0 2024-09-12T02:13:34.261 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.253+0000 7f240dffb640 1 -- 172.21.15.16:0/2663384811 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f242002f490 con 0x555c9e27a9a0 2024-09-12T02:13:34.261 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.254+0000 7f242f038740 10 client.19374 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:13:34.261 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.254+0000 7f242f038740 10 client.19374 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:13:34.261 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.254+0000 7f242f038740 20 client.19374 populate_metadata read hostname 'smithi016' 2024-09-12T02:13:34.261 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.254+0000 7f242f038740 1 --2- 172.21.15.16:0/2663384811 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x555c9e2fbb20 0x555c9e2fdf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:34.261 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.254+0000 7f242f038740 4 client.19374 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:13:34.261 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.254+0000 7f242f038740 1 -- 172.21.15.16:0/2663384811 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555c9e162900 con 0x555c9e2fbb20 2024-09-12T02:13:34.261 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.254+0000 7f2429841640 1 --2- 172.21.15.16:0/2663384811 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x555c9e2fbb20 0x555c9e2fdf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:34.261 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.254+0000 7f240dffb640 10 client.19374 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:13:34.261 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.254+0000 7f2429841640 1 --2- 172.21.15.16:0/2663384811 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x555c9e2fbb20 0x555c9e2fdf40 crc :-1 s=READY pgs=357 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=537342851a5ca5a4 server_cookie=ee2dafbfba27924d in_seq=0 out_seq=0 2024-09-12T02:13:34.268 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.267+0000 7f240dffb640 1 -- 172.21.15.16:0/2663384811 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555c9e162900 con 0x555c9e2fbb20 2024-09-12T02:13:34.268 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.267+0000 7f240dffb640 10 client.19374 handle_command_reply: tid=0 2024-09-12T02:13:34.268 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.267+0000 7f242f038740 1 client.19374 shutdown 2024-09-12T02:13:34.269 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.267+0000 7f242f038740 1 -- 172.21.15.16:0/2663384811 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c9e27a9a0 msgr2=0x555c9e2897e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:34.269 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.267+0000 7f242f038740 1 --2- 172.21.15.16:0/2663384811 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c9e27a9a0 0x555c9e2897e0 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7f2420000c00 tx=0x7f2420012820 comp rx=0 tx=0).stop 2024-09-12T02:13:34.269 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.267+0000 7f242f038740 1 -- 172.21.15.16:0/2663384811 shutdown_connections 2024-09-12T02:13:34.269 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.267+0000 7f242f038740 1 --2- 172.21.15.16:0/2663384811 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x555c9e2fbb20 0x555c9e2fdf40 crc :-1 s=READY pgs=357 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:34.269 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.267+0000 7f242f038740 1 --2- 172.21.15.16:0/2663384811 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555c9e280370 0x555c9e28ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:34.269 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.267+0000 7f240dffb640 0 client.19374 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:13:34.269 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.267+0000 7f242f038740 1 --2- 172.21.15.16:0/2663384811 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555c9e27b2e0 0x555c9e289d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:34.269 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.267+0000 7f242f038740 1 --2- 172.21.15.16:0/2663384811 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c9e27a9a0 0x555c9e2897e0 unknown :-1 s=CLOSED pgs=1673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:34.269 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.267+0000 7f242f038740 1 -- 172.21.15.16:0/2663384811 >> 172.21.15.16:0/2663384811 conn(0x555c9e276010 msgr2=0x555c9e284020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:34.269 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.267+0000 7f242f038740 1 -- 172.21.15.16:0/2663384811 shutdown_connections 2024-09-12T02:13:34.269 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.267+0000 7f242f038740 1 -- 172.21.15.16:0/2663384811 wait complete. 2024-09-12T02:13:34.269 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.267+0000 7f242f038740 20 client.19374 trim_cache size 0 max 16384 2024-09-12T02:13:34.270 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-09-12T02:13:34.270 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-09-12T02:13:34.270 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-09-12T02:13:34.270 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-09-12T02:13:34.271 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.268+0000 7f242c2a6640 1 -- 172.21.15.16:0/2666935983 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f24100a2c30 msgr2=0x7f24100a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:34.271 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.268+0000 7f242c2a6640 1 --2- 172.21.15.16:0/2666935983 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f24100a2c30 0x7f24100a50f0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f241c009430 tx=0x7f241c02a8c0 comp rx=0 tx=0).stop 2024-09-12T02:13:34.271 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.268+0000 7f242c2a6640 1 -- 172.21.15.16:0/2666935983 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2424102d50 msgr2=0x7f24240745e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:34.271 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.268+0000 7f242c2a6640 1 --2- 172.21.15.16:0/2666935983 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2424102d50 0x7f24240745e0 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7f241400c970 tx=0x7f241400ce30 comp rx=0 tx=0).stop 2024-09-12T02:13:34.271 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.268+0000 7f242c2a6640 1 -- 172.21.15.16:0/2666935983 shutdown_connections 2024-09-12T02:13:34.271 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.268+0000 7f242c2a6640 1 --2- 172.21.15.16:0/2666935983 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f24100a2c30 0x7f24100a50f0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:34.271 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.268+0000 7f242c2a6640 1 --2- 172.21.15.16:0/2666935983 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2424103710 0x7f2424074b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:34.271 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.268+0000 7f242c2a6640 1 --2- 172.21.15.16:0/2666935983 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2424102d50 0x7f24240745e0 unknown :-1 s=CLOSED pgs=1669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:34.271 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.268+0000 7f242c2a6640 1 --2- 172.21.15.16:0/2666935983 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2424101b60 0x7f2424075fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:34.271 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.268+0000 7f242c2a6640 1 -- 172.21.15.16:0/2666935983 >> 172.21.15.16:0/2666935983 conn(0x7f24240fd2e0 msgr2=0x7f2424105ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:34.272 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.268+0000 7f242c2a6640 1 -- 172.21.15.16:0/2666935983 shutdown_connections 2024-09-12T02:13:34.272 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:34.268+0000 7f242c2a6640 1 -- 172.21.15.16:0/2666935983 wait complete. 2024-09-12T02:13:34.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:13:34 smithi184 ceph-mon[35593]: pgmap v684: 97 pgs: 97 active+clean; 2.2 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 17 op/s 2024-09-12T02:13:34.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:13:34 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/4278916595' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:13:34.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:13:34 smithi033 ceph-mon[35332]: pgmap v684: 97 pgs: 97 active+clean; 2.2 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 17 op/s 2024-09-12T02:13:34.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:13:34 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/4278916595' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:13:34.656 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-12T02:13:34.910 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:13:35.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.354+0000 7f3f93e6e640 1 -- 172.21.15.16:0/1673511794 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f8c101de0 msgr2=0x7f3f8c1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:35.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.354+0000 7f3f93e6e640 1 --2- 172.21.15.16:0/1673511794 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f8c101de0 0x7f3f8c1021c0 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7f3f88009900 tx=0x7f3f8802a410 comp rx=0 tx=0).stop 2024-09-12T02:13:35.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.354+0000 7f3f93e6e640 1 -- 172.21.15.16:0/1673511794 shutdown_connections 2024-09-12T02:13:35.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.354+0000 7f3f93e6e640 1 --2- 172.21.15.16:0/1673511794 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3f8c106f10 0x7f3f8c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:35.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.354+0000 7f3f93e6e640 1 --2- 172.21.15.16:0/1673511794 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f8c102790 0x7f3f8c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:35.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.354+0000 7f3f93e6e640 1 --2- 172.21.15.16:0/1673511794 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f8c101de0 0x7f3f8c1021c0 unknown :-1 s=CLOSED pgs=1674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:35.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.354+0000 7f3f93e6e640 1 -- 172.21.15.16:0/1673511794 >> 172.21.15.16:0/1673511794 conn(0x7f3f8c0fd8c0 msgr2=0x7f3f8c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:35.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.354+0000 7f3f93e6e640 1 -- 172.21.15.16:0/1673511794 shutdown_connections 2024-09-12T02:13:35.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.354+0000 7f3f93e6e640 1 -- 172.21.15.16:0/1673511794 wait complete. 2024-09-12T02:13:35.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.355+0000 7f3f93e6e640 1 Processor -- start 2024-09-12T02:13:35.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.355+0000 7f3f93e6e640 1 -- start start 2024-09-12T02:13:35.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.355+0000 7f3f93e6e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f8c101de0 0x7f3f8c198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:35.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.355+0000 7f3f93e6e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f8c102790 0x7f3f8c198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:35.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.355+0000 7f3f93e6e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3f8c106f10 0x7f3f8c19cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:35.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.355+0000 7f3f93e6e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3f8c111f70 con 0x7f3f8c101de0 2024-09-12T02:13:35.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.355+0000 7f3f93e6e640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3f8c111df0 con 0x7f3f8c102790 2024-09-12T02:13:35.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.355+0000 7f3f93e6e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f3f8c1120f0 con 0x7f3f8c106f10 2024-09-12T02:13:35.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.355+0000 7f3f91c0a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f8c101de0 0x7f3f8c198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:35.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.355+0000 7f3f91c0a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f8c101de0 0x7f3f8c198430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52214/0 (socket says 172.21.15.16:52214) 2024-09-12T02:13:35.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.355+0000 7f3f91c0a640 1 -- 172.21.15.16:0/2127940546 learned_addr learned my addr 172.21.15.16:0/2127940546 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:13:35.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.356+0000 7f3f91c0a640 1 -- 172.21.15.16:0/2127940546 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3f8c106f10 msgr2=0x7f3f8c19cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:35.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.356+0000 7f3f91409640 1 --2- 172.21.15.16:0/2127940546 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f8c102790 0x7f3f8c198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:35.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.356+0000 7f3f91c0a640 1 --2- 172.21.15.16:0/2127940546 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3f8c106f10 0x7f3f8c19cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:35.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.356+0000 7f3f91c0a640 1 -- 172.21.15.16:0/2127940546 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f8c102790 msgr2=0x7f3f8c198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:35.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.356+0000 7f3f91c0a640 1 --2- 172.21.15.16:0/2127940546 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f8c102790 0x7f3f8c198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:35.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.356+0000 7f3f91c0a640 1 -- 172.21.15.16:0/2127940546 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f8c19d3e0 con 0x7f3f8c101de0 2024-09-12T02:13:35.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.356+0000 7f3f91409640 1 --2- 172.21.15.16:0/2127940546 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f8c102790 0x7f3f8c198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:13:35.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.356+0000 7f3f91c0a640 1 --2- 172.21.15.16:0/2127940546 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f8c101de0 0x7f3f8c198430 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7f3f88006010 tx=0x7f3f880042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:35.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.356+0000 7f3f82ffd640 1 -- 172.21.15.16:0/2127940546 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f8803f030 con 0x7f3f8c101de0 2024-09-12T02:13:35.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.356+0000 7f3f93e6e640 1 -- 172.21.15.16:0/2127940546 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f8c19d670 con 0x7f3f8c101de0 2024-09-12T02:13:35.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.356+0000 7f3f82ffd640 1 -- 172.21.15.16:0/2127940546 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3f88003ce0 con 0x7f3f8c101de0 2024-09-12T02:13:35.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.356+0000 7f3f82ffd640 1 -- 172.21.15.16:0/2127940546 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f88035070 con 0x7f3f8c101de0 2024-09-12T02:13:35.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.356+0000 7f3f93e6e640 1 -- 172.21.15.16:0/2127940546 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f8c1a4fb0 con 0x7f3f8c101de0 2024-09-12T02:13:35.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.358+0000 7f3f82ffd640 1 -- 172.21.15.16:0/2127940546 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f3f88042030 con 0x7f3f8c101de0 2024-09-12T02:13:35.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.359+0000 7f3f96247740 1 Processor -- start 2024-09-12T02:13:35.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.359+0000 7f3f82ffd640 1 --2- 172.21.15.16:0/2127940546 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f3f780a2b80 0x7f3f780a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:35.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.359+0000 7f3f96247740 1 -- start start 2024-09-12T02:13:35.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.359+0000 7f3f9240b640 1 --2- 172.21.15.16:0/2127940546 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f3f780a2b80 0x7f3f780a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:35.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.359+0000 7f3f82ffd640 1 -- 172.21.15.16:0/2127940546 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f3f880ef3d0 con 0x7f3f8c101de0 2024-09-12T02:13:35.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.359+0000 7f3f96247740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ef109f4b40 0x55ef109f3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:35.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.359+0000 7f3f96247740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ef109f35d0 0x55ef109f3a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:35.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.359+0000 7f3f96247740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ef10a02780 0x55ef10a04b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:35.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.359+0000 7f3f96247740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55ef109592f0 con 0x55ef109f4b40 2024-09-12T02:13:35.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.359+0000 7f3f96247740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55ef10900cf0 con 0x55ef109f35d0 2024-09-12T02:13:35.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.359+0000 7f3f96247740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55ef109552b0 con 0x55ef10a02780 2024-09-12T02:13:35.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.360+0000 7f3f91c0a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ef109f4b40 0x55ef109f3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:35.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.360+0000 7f3f91c0a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ef109f4b40 0x55ef109f3090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52218/0 (socket says 172.21.15.16:52218) 2024-09-12T02:13:35.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.360+0000 7f3f91c0a640 1 -- 172.21.15.16:0/2261039608 learned_addr learned my addr 172.21.15.16:0/2261039608 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:13:35.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.360+0000 7f3f91409640 1 --2- 172.21.15.16:0/2261039608 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ef109f35d0 0x55ef109f3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:35.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.360+0000 7f3f9240b640 1 --2- 172.21.15.16:0/2261039608 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ef10a02780 0x55ef10a04b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:35.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.360+0000 7f3f91c0a640 1 -- 172.21.15.16:0/2261039608 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ef10a02780 msgr2=0x55ef10a04b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:35.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.360+0000 7f3f91c0a640 1 --2- 172.21.15.16:0/2261039608 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ef10a02780 0x55ef10a04b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:35.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.360+0000 7f3f91c0a640 1 -- 172.21.15.16:0/2261039608 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ef109f35d0 msgr2=0x55ef109f3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:35.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.360+0000 7f3f91c0a640 1 --2- 172.21.15.16:0/2261039608 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ef109f35d0 0x55ef109f3a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:35.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.360+0000 7f3f91c0a640 1 -- 172.21.15.16:0/2261039608 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef108e53a0 con 0x55ef109f4b40 2024-09-12T02:13:35.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.360+0000 7f3f9240b640 1 --2- 172.21.15.16:0/2261039608 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ef10a02780 0x55ef10a04b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:13:35.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.360+0000 7f3f9240b640 1 --2- 172.21.15.16:0/2127940546 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f3f780a2b80 0x7f3f780a5040 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f3f84005fd0 tx=0x7f3f84007f20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:35.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.360+0000 7f3f91c0a640 1 --2- 172.21.15.16:0/2261039608 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ef109f4b40 0x55ef109f3090 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7f3f880ece40 tx=0x7f3f880bb830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cf65abbafa636c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:35.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.360+0000 7f3f77fff640 1 -- 172.21.15.16:0/2261039608 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f880d9070 con 0x55ef109f4b40 2024-09-12T02:13:35.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.360+0000 7f3f77fff640 1 -- 172.21.15.16:0/2261039608 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3f8803b580 con 0x55ef109f4b40 2024-09-12T02:13:35.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.360+0000 7f3f77fff640 1 -- 172.21.15.16:0/2261039608 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f88049040 con 0x55ef109f4b40 2024-09-12T02:13:35.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.361+0000 7f3f96247740 1 -- 172.21.15.16:0/2261039608 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ef109f4b40 msgr2=0x55ef109f3090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:35.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.361+0000 7f3f96247740 1 --2- 172.21.15.16:0/2261039608 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ef109f4b40 0x55ef109f3090 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7f3f880ece40 tx=0x7f3f880bb830 comp rx=0 tx=0).stop 2024-09-12T02:13:35.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.361+0000 7f3f96247740 1 -- 172.21.15.16:0/2261039608 shutdown_connections 2024-09-12T02:13:35.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.361+0000 7f3f96247740 1 --2- 172.21.15.16:0/2261039608 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ef10a02780 0x55ef10a04b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:35.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.361+0000 7f3f96247740 1 --2- 172.21.15.16:0/2261039608 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ef109f35d0 0x55ef109f3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:35.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.361+0000 7f3f96247740 1 --2- 172.21.15.16:0/2261039608 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ef109f4b40 0x55ef109f3090 unknown :-1 s=CLOSED pgs=1676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:35.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.361+0000 7f3f96247740 1 -- 172.21.15.16:0/2261039608 >> 172.21.15.16:0/2261039608 conn(0x55ef109bc310 msgr2=0x55ef109efc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:35.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.361+0000 7f3f96247740 1 -- 172.21.15.16:0/2261039608 shutdown_connections 2024-09-12T02:13:35.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.361+0000 7f3f96247740 1 -- 172.21.15.16:0/2261039608 wait complete. 2024-09-12T02:13:35.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.362+0000 7f3f96247740 1 -- start start 2024-09-12T02:13:35.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.362+0000 7f3f96247740 1 Processor -- start 2024-09-12T02:13:35.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.362+0000 7f3f96247740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ef109f35d0 0x55ef10a021a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:35.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.362+0000 7f3f96247740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ef109f4b40 0x55ef109ff450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:35.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.362+0000 7f3f96247740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ef10a02780 0x55ef109ffa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:35.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.362+0000 7f3f96247740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55ef10a05240 con 0x55ef109f35d0 2024-09-12T02:13:35.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.362+0000 7f3f96247740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55ef10914060 con 0x55ef109f4b40 2024-09-12T02:13:35.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.362+0000 7f3f91c0a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ef109f35d0 0x55ef10a021a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:35.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.363+0000 7f3f96247740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55ef10a053c0 con 0x55ef10a02780 2024-09-12T02:13:35.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.363+0000 7f3f91c0a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ef109f35d0 0x55ef10a021a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52220/0 (socket says 172.21.15.16:52220) 2024-09-12T02:13:35.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.363+0000 7f3f91c0a640 1 -- 172.21.15.16:0/401392792 learned_addr learned my addr 172.21.15.16:0/401392792 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:13:35.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.363+0000 7f3f96247740 10 client.0 _ll_register_callbacks cb 0x55ef106678c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:13:35.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.363+0000 7f3f91409640 1 --2- 172.21.15.16:0/401392792 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ef109f4b40 0x55ef109ff450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:35.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.363+0000 7f3f9240b640 1 --2- 172.21.15.16:0/401392792 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ef10a02780 0x55ef109ffa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:35.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.363+0000 7f3f91c0a640 1 -- 172.21.15.16:0/401392792 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ef10a02780 msgr2=0x55ef109ffa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:35.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.363+0000 7f3f91c0a640 1 --2- 172.21.15.16:0/401392792 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ef10a02780 0x55ef109ffa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:35.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.363+0000 7f3f91c0a640 1 -- 172.21.15.16:0/401392792 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ef109f4b40 msgr2=0x55ef109ff450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:35.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.363+0000 7f3f91c0a640 1 --2- 172.21.15.16:0/401392792 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ef109f4b40 0x55ef109ff450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:35.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.363+0000 7f3f91c0a640 1 -- 172.21.15.16:0/401392792 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef106ca3c0 con 0x55ef109f35d0 2024-09-12T02:13:35.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.363+0000 7f3f9240b640 1 --2- 172.21.15.16:0/401392792 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ef10a02780 0x55ef109ffa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:13:35.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.363+0000 7f3f91c0a640 1 --2- 172.21.15.16:0/401392792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ef109f35d0 0x55ef10a021a0 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7f3f880c4430 tx=0x7f3f88005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b90b2cf43549ee1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:35.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.363+0000 7f3f757fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:13:35.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.363+0000 7f3f757fa640 1 -- 172.21.15.16:0/401392792 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f880d9030 con 0x55ef109f35d0 2024-09-12T02:13:35.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.363+0000 7f3f757fa640 1 -- 172.21.15.16:0/401392792 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f3f880b9d90 con 0x55ef109f35d0 2024-09-12T02:13:35.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.363+0000 7f3f96247740 1 -- 172.21.15.16:0/401392792 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ef10966860 con 0x55ef109f35d0 2024-09-12T02:13:35.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.364+0000 7f3f757fa640 1 -- 172.21.15.16:0/401392792 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f3f88049040 con 0x55ef109f35d0 2024-09-12T02:13:35.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.364+0000 7f3f96247740 10 client.19386 fetch_fsmap learned FSMap version 25 2024-09-12T02:13:35.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.364+0000 7f3f96247740 1 -- 172.21.15.16:0/401392792 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ef10a05240 con 0x55ef109f35d0 2024-09-12T02:13:35.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.364+0000 7f3f757fa640 1 -- 172.21.15.16:0/401392792 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55ef10966860 con 0x55ef109f35d0 2024-09-12T02:13:35.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.364+0000 7f3f757fa640 1 -- 172.21.15.16:0/401392792 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f3f8804c060 con 0x55ef109f35d0 2024-09-12T02:13:35.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.364+0000 7f3f96247740 10 client.19386 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:13:35.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.364+0000 7f3f96247740 10 client.19386 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-12T02:13:35.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.364+0000 7f3f96247740 20 client.19386 populate_metadata read hostname 'smithi016' 2024-09-12T02:13:35.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.364+0000 7f3f96247740 1 --2- 172.21.15.16:0/401392792 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55ef10a11db0 0x55ef10a141a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:35.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.364+0000 7f3f96247740 4 client.19386 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:13:35.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.364+0000 7f3f96247740 1 -- 172.21.15.16:0/401392792 --> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ef10960d70 con 0x55ef10a11db0 2024-09-12T02:13:35.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.364+0000 7f3f91409640 1 --2- 172.21.15.16:0/401392792 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55ef10a11db0 0x55ef10a141a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:35.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.365+0000 7f3f91409640 1 --2- 172.21.15.16:0/401392792 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55ef10a11db0 0x55ef10a141a0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cd6f988d7d55e49e server_cookie=3aa4de77d1807624 in_seq=0 out_seq=0 2024-09-12T02:13:35.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.365+0000 7f3f757fa640 10 client.19386 ms_handle_connect on v2:172.21.15.184:6836/1793393087 2024-09-12T02:13:35.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.366+0000 7f3f757fa640 1 -- 172.21.15.16:0/401392792 <== mds.2 v2:172.21.15.184:6836/1793393087 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ef10960d70 con 0x55ef10a11db0 2024-09-12T02:13:35.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.366+0000 7f3f757fa640 10 client.19386 handle_command_reply: tid=0 2024-09-12T02:13:35.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.366+0000 7f3f96247740 1 client.19386 shutdown 2024-09-12T02:13:35.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.366+0000 7f3f96247740 1 -- 172.21.15.16:0/401392792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ef109f35d0 msgr2=0x55ef10a021a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:35.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.366+0000 7f3f96247740 1 --2- 172.21.15.16:0/401392792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ef109f35d0 0x55ef10a021a0 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7f3f880c4430 tx=0x7f3f88005b50 comp rx=0 tx=0).stop 2024-09-12T02:13:35.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.366+0000 7f3f96247740 1 -- 172.21.15.16:0/401392792 shutdown_connections 2024-09-12T02:13:35.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.366+0000 7f3f96247740 1 --2- 172.21.15.16:0/401392792 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ef10a02780 0x55ef109ffa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:35.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.366+0000 7f3f96247740 1 --2- 172.21.15.16:0/401392792 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ef109f4b40 0x55ef109ff450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:35.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.366+0000 7f3f96247740 1 --2- 172.21.15.16:0/401392792 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55ef10a11db0 0x55ef10a141a0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:35.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.366+0000 7f3f96247740 1 --2- 172.21.15.16:0/401392792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ef109f35d0 0x55ef10a021a0 unknown :-1 s=CLOSED pgs=1677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:35.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.366+0000 7f3f757fa640 0 client.19386 ms_handle_reset on v2:172.21.15.184:6836/1793393087 2024-09-12T02:13:35.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.366+0000 7f3f96247740 1 -- 172.21.15.16:0/401392792 >> 172.21.15.16:0/401392792 conn(0x55ef109bc310 msgr2=0x55ef109f5930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:35.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.366+0000 7f3f96247740 1 -- 172.21.15.16:0/401392792 shutdown_connections 2024-09-12T02:13:35.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.366+0000 7f3f96247740 1 -- 172.21.15.16:0/401392792 wait complete. 2024-09-12T02:13:35.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.366+0000 7f3f96247740 20 client.19386 trim_cache size 0 max 16384 2024-09-12T02:13:35.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.379+0000 7f3f96247740 1 Processor -- start 2024-09-12T02:13:35.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.379+0000 7f3f96247740 1 -- start start 2024-09-12T02:13:35.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.379+0000 7f3f96247740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ef109f35d0 0x55ef10a0e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:35.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.379+0000 7f3f96247740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ef109f4b40 0x55ef109eec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:35.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.379+0000 7f3f96247740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ef10a02780 0x55ef10a19930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:35.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.379+0000 7f3f96247740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3f8c1adb70 con 0x55ef109f4b40 2024-09-12T02:13:35.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.379+0000 7f3f96247740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3f8c1ab440 con 0x55ef109f35d0 2024-09-12T02:13:35.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.379+0000 7f3f96247740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f3f8c1a92b0 con 0x55ef10a02780 2024-09-12T02:13:35.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.380+0000 7f3f91409640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ef109f4b40 0x55ef109eec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:35.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.380+0000 7f3f91409640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ef109f4b40 0x55ef109eec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52234/0 (socket says 172.21.15.16:52234) 2024-09-12T02:13:35.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.380+0000 7f3f91409640 1 -- 172.21.15.16:0/890590057 learned_addr learned my addr 172.21.15.16:0/890590057 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:13:35.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.380+0000 7f3f91c0a640 1 --2- 172.21.15.16:0/890590057 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ef109f35d0 0x55ef10a0e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:35.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.380+0000 7f3f9240b640 1 --2- 172.21.15.16:0/890590057 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ef10a02780 0x55ef10a19930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:35.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.380+0000 7f3f91409640 1 -- 172.21.15.16:0/890590057 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ef10a02780 msgr2=0x55ef10a19930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:35.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.380+0000 7f3f91409640 1 --2- 172.21.15.16:0/890590057 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ef10a02780 0x55ef10a19930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:35.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.380+0000 7f3f91409640 1 -- 172.21.15.16:0/890590057 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ef109f35d0 msgr2=0x55ef10a0e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:35.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.380+0000 7f3f91409640 1 --2- 172.21.15.16:0/890590057 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ef109f35d0 0x55ef10a0e4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:35.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.380+0000 7f3f91409640 1 -- 172.21.15.16:0/890590057 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef109f6120 con 0x55ef109f4b40 2024-09-12T02:13:35.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.380+0000 7f3f91c0a640 1 --2- 172.21.15.16:0/890590057 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ef109f35d0 0x55ef10a0e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:13:35.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.380+0000 7f3f91409640 1 --2- 172.21.15.16:0/890590057 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ef109f4b40 0x55ef109eec10 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c000c00 tx=0x7f3f7c014840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe999c073342c7f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:35.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.380+0000 7f3f777fe640 1 -- 172.21.15.16:0/890590057 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f7c0140d0 con 0x55ef109f4b40 2024-09-12T02:13:35.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.380+0000 7f3f777fe640 1 -- 172.21.15.16:0/890590057 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3f7c014270 con 0x55ef109f4b40 2024-09-12T02:13:35.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.380+0000 7f3f777fe640 1 -- 172.21.15.16:0/890590057 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f7c012450 con 0x55ef109f4b40 2024-09-12T02:13:35.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.381+0000 7f3f96247740 1 -- 172.21.15.16:0/890590057 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ef109f4b40 msgr2=0x55ef109eec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:35.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.381+0000 7f3f96247740 1 --2- 172.21.15.16:0/890590057 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ef109f4b40 0x55ef109eec10 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c000c00 tx=0x7f3f7c014840 comp rx=0 tx=0).stop 2024-09-12T02:13:35.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.381+0000 7f3f96247740 1 -- 172.21.15.16:0/890590057 shutdown_connections 2024-09-12T02:13:35.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.381+0000 7f3f96247740 1 --2- 172.21.15.16:0/890590057 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ef10a02780 0x55ef10a19930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:35.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.381+0000 7f3f96247740 1 --2- 172.21.15.16:0/890590057 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ef109f4b40 0x55ef109eec10 unknown :-1 s=CLOSED pgs=1678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:35.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.381+0000 7f3f96247740 1 --2- 172.21.15.16:0/890590057 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ef109f35d0 0x55ef10a0e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:35.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.381+0000 7f3f96247740 1 -- 172.21.15.16:0/890590057 >> 172.21.15.16:0/890590057 conn(0x55ef109bc310 msgr2=0x55ef109ee500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:35.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.381+0000 7f3f96247740 1 -- 172.21.15.16:0/890590057 shutdown_connections 2024-09-12T02:13:35.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.381+0000 7f3f96247740 1 -- 172.21.15.16:0/890590057 wait complete. 2024-09-12T02:13:35.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.381+0000 7f3f96247740 1 -- start start 2024-09-12T02:13:35.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.381+0000 7f3f96247740 1 Processor -- start 2024-09-12T02:13:35.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.382+0000 7f3f96247740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ef109f35d0 0x55ef10a0a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:35.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.382+0000 7f3f96247740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ef109f4b40 0x55ef10a0ab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:35.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.382+0000 7f3f96247740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ef10a02780 0x55ef10a075e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:35.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.382+0000 7f3f96247740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3f8c1a9900 con 0x55ef109f4b40 2024-09-12T02:13:35.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.382+0000 7f3f96247740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3f8c1aad70 con 0x55ef10a02780 2024-09-12T02:13:35.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.382+0000 7f3f96247740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f3f8c1a9fd0 con 0x55ef109f35d0 2024-09-12T02:13:35.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.382+0000 7f3f96247740 10 client.0 _ll_register_callbacks cb 0x7f3f880d7070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:13:35.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.382+0000 7f3f91409640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ef109f4b40 0x55ef10a0ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:35.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.382+0000 7f3f91409640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ef109f4b40 0x55ef10a0ab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52248/0 (socket says 172.21.15.16:52248) 2024-09-12T02:13:35.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.382+0000 7f3f91409640 1 -- 172.21.15.16:0/3771055062 learned_addr learned my addr 172.21.15.16:0/3771055062 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:13:35.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.382+0000 7f3f91c0a640 1 --2- 172.21.15.16:0/3771055062 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ef109f35d0 0x55ef10a0a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:35.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.382+0000 7f3f9240b640 1 --2- 172.21.15.16:0/3771055062 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ef10a02780 0x55ef10a075e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:35.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.382+0000 7f3f91409640 1 -- 172.21.15.16:0/3771055062 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ef109f35d0 msgr2=0x55ef10a0a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:35.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.382+0000 7f3f91409640 1 --2- 172.21.15.16:0/3771055062 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ef109f35d0 0x55ef10a0a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:35.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.382+0000 7f3f91409640 1 -- 172.21.15.16:0/3771055062 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ef10a02780 msgr2=0x55ef10a075e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:35.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.382+0000 7f3f91409640 1 --2- 172.21.15.16:0/3771055062 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ef10a02780 0x55ef10a075e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:35.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.382+0000 7f3f91409640 1 -- 172.21.15.16:0/3771055062 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef10a07b20 con 0x55ef109f4b40 2024-09-12T02:13:35.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.383+0000 7f3f91409640 1 --2- 172.21.15.16:0/3771055062 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ef109f4b40 0x55ef10a0ab50 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c014440 tx=0x7f3f7c009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2f238a3cd036237 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:35.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.383+0000 7f3f757fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:13:35.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.383+0000 7f3f757fa640 1 -- 172.21.15.16:0/3771055062 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f7c035060 con 0x55ef109f4b40 2024-09-12T02:13:35.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.383+0000 7f3f757fa640 1 -- 172.21.15.16:0/3771055062 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f3f7c022050 con 0x55ef109f4b40 2024-09-12T02:13:35.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.383+0000 7f3f96247740 1 -- 172.21.15.16:0/3771055062 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3f7c014270 con 0x55ef109f4b40 2024-09-12T02:13:35.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.383+0000 7f3f757fa640 1 -- 172.21.15.16:0/3771055062 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f3f7c0035f0 con 0x55ef109f4b40 2024-09-12T02:13:35.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.383+0000 7f3f96247740 10 client.19392 fetch_fsmap learned FSMap version 25 2024-09-12T02:13:35.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.383+0000 7f3f96247740 1 -- 172.21.15.16:0/3771055062 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ef10a7cf10 con 0x55ef109f4b40 2024-09-12T02:13:35.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.383+0000 7f3f757fa640 1 -- 172.21.15.16:0/3771055062 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3f7c014270 con 0x55ef109f4b40 2024-09-12T02:13:35.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.383+0000 7f3f757fa640 1 -- 172.21.15.16:0/3771055062 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f3f7c00f040 con 0x55ef109f4b40 2024-09-12T02:13:35.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.383+0000 7f3f96247740 10 client.19392 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:13:35.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.383+0000 7f3f96247740 10 client.19392 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-12T02:13:35.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.383+0000 7f3f96247740 20 client.19392 populate_metadata read hostname 'smithi016' 2024-09-12T02:13:35.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.383+0000 7f3f96247740 1 --2- 172.21.15.16:0/3771055062 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55ef10a7d660 0x55ef10a7fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:35.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.383+0000 7f3f96247740 4 client.19392 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T02:13:35.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.383+0000 7f3f96247740 1 -- 172.21.15.16:0/3771055062 --> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ef108dd170 con 0x55ef10a7d660 2024-09-12T02:13:35.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.384+0000 7f3f91c0a640 1 --2- 172.21.15.16:0/3771055062 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55ef10a7d660 0x55ef10a7fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:35.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.384+0000 7f3f91c0a640 1 --2- 172.21.15.16:0/3771055062 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55ef10a7d660 0x55ef10a7fa80 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cd3f17931d506c83 server_cookie=9008ae4e93b6ed7a in_seq=0 out_seq=0 2024-09-12T02:13:35.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.384+0000 7f3f757fa640 10 client.19392 ms_handle_connect on v2:172.21.15.184:6836/1793393087 2024-09-12T02:13:35.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.384+0000 7f3f757fa640 1 -- 172.21.15.16:0/3771055062 <== mds.2 v2:172.21.15.184:6836/1793393087 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ef108dd170 con 0x55ef10a7d660 2024-09-12T02:13:35.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.384+0000 7f3f757fa640 10 client.19392 handle_command_reply: tid=0 2024-09-12T02:13:35.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.384+0000 7f3f96247740 1 client.19392 shutdown 2024-09-12T02:13:35.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.385+0000 7f3f96247740 1 -- 172.21.15.16:0/3771055062 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ef109f4b40 msgr2=0x55ef10a0ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:35.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.385+0000 7f3f96247740 1 --2- 172.21.15.16:0/3771055062 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ef109f4b40 0x55ef10a0ab50 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c014440 tx=0x7f3f7c009b20 comp rx=0 tx=0).stop 2024-09-12T02:13:35.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.385+0000 7f3f96247740 1 -- 172.21.15.16:0/3771055062 shutdown_connections 2024-09-12T02:13:35.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.385+0000 7f3f96247740 1 --2- 172.21.15.16:0/3771055062 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ef10a02780 0x55ef10a075e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:35.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.385+0000 7f3f96247740 1 --2- 172.21.15.16:0/3771055062 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55ef10a7d660 0x55ef10a7fa80 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:35.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.385+0000 7f3f96247740 1 --2- 172.21.15.16:0/3771055062 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ef109f4b40 0x55ef10a0ab50 unknown :-1 s=CLOSED pgs=1679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:35.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.385+0000 7f3f757fa640 0 client.19392 ms_handle_reset on v2:172.21.15.184:6836/1793393087 2024-09-12T02:13:35.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.385+0000 7f3f96247740 1 --2- 172.21.15.16:0/3771055062 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ef109f35d0 0x55ef10a0a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:35.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.385+0000 7f3f96247740 1 -- 172.21.15.16:0/3771055062 >> 172.21.15.16:0/3771055062 conn(0x55ef109bc310 msgr2=0x55ef109f5930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:35.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.385+0000 7f3f96247740 1 -- 172.21.15.16:0/3771055062 shutdown_connections 2024-09-12T02:13:35.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.385+0000 7f3f96247740 1 -- 172.21.15.16:0/3771055062 wait complete. 2024-09-12T02:13:35.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.385+0000 7f3f96247740 20 client.19392 trim_cache size 0 max 16384 2024-09-12T02:13:35.391 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T02:13:35.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.385+0000 7f3f93e6e640 1 -- 172.21.15.16:0/2127940546 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f3f780a2b80 msgr2=0x7f3f780a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:35.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.385+0000 7f3f93e6e640 1 --2- 172.21.15.16:0/2127940546 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f3f780a2b80 0x7f3f780a5040 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f3f84005fd0 tx=0x7f3f84007f20 comp rx=0 tx=0).stop 2024-09-12T02:13:35.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.385+0000 7f3f93e6e640 1 -- 172.21.15.16:0/2127940546 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f8c101de0 msgr2=0x7f3f8c198430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:35.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.385+0000 7f3f93e6e640 1 --2- 172.21.15.16:0/2127940546 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f8c101de0 0x7f3f8c198430 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7f3f88006010 tx=0x7f3f880042f0 comp rx=0 tx=0).stop 2024-09-12T02:13:35.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.386+0000 7f3f93e6e640 1 -- 172.21.15.16:0/2127940546 shutdown_connections 2024-09-12T02:13:35.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.386+0000 7f3f93e6e640 1 --2- 172.21.15.16:0/2127940546 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f3f780a2b80 0x7f3f780a5040 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:35.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.386+0000 7f3f93e6e640 1 --2- 172.21.15.16:0/2127940546 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3f8c106f10 0x7f3f8c19cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:35.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.386+0000 7f3f93e6e640 1 --2- 172.21.15.16:0/2127940546 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f8c102790 0x7f3f8c198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:35.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.386+0000 7f3f93e6e640 1 --2- 172.21.15.16:0/2127940546 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f8c101de0 0x7f3f8c198430 unknown :-1 s=CLOSED pgs=1675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:35.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.386+0000 7f3f93e6e640 1 -- 172.21.15.16:0/2127940546 >> 172.21.15.16:0/2127940546 conn(0x7f3f8c0fd8c0 msgr2=0x7f3f8c0fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:35.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.386+0000 7f3f93e6e640 1 -- 172.21.15.16:0/2127940546 shutdown_connections 2024-09-12T02:13:35.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:35.386+0000 7f3f93e6e640 1 -- 172.21.15.16:0/2127940546 wait complete. 2024-09-12T02:13:36.055 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-12T02:13:36.312 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:13:36.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:13:36 smithi184 ceph-mon[35593]: pgmap v685: 97 pgs: 97 active+clean; 2.2 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-09-12T02:13:36.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:13:36 smithi033 ceph-mon[35332]: pgmap v685: 97 pgs: 97 active+clean; 2.2 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-09-12T02:13:36.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:13:36 smithi016 ceph-mon[30947]: pgmap v685: 97 pgs: 97 active+clean; 2.2 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-09-12T02:13:36.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.762+0000 7f65d0bc8640 1 -- 172.21.15.16:0/1435410214 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65cc102ef0 msgr2=0x7f65cc10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:36.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.762+0000 7f65d0bc8640 1 --2- 172.21.15.16:0/1435410214 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65cc102ef0 0x7f65cc10da40 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7f65c4009ca0 tx=0x7f65c402a950 comp rx=0 tx=0).stop 2024-09-12T02:13:36.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.762+0000 7f65d0bc8640 1 -- 172.21.15.16:0/1435410214 shutdown_connections 2024-09-12T02:13:36.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.762+0000 7f65d0bc8640 1 --2- 172.21.15.16:0/1435410214 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65cc102ef0 0x7f65cc10da40 unknown :-1 s=CLOSED pgs=1680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:36.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.762+0000 7f65d0bc8640 1 --2- 172.21.15.16:0/1435410214 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f65cc102570 0x7f65cc1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:36.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.762+0000 7f65d0bc8640 1 --2- 172.21.15.16:0/1435410214 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f65cc100e20 0x7f65cc101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:36.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.762+0000 7f65d0bc8640 1 -- 172.21.15.16:0/1435410214 >> 172.21.15.16:0/1435410214 conn(0x7f65cc0fc7d0 msgr2=0x7f65cc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:36.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.762+0000 7f65d0bc8640 1 -- 172.21.15.16:0/1435410214 shutdown_connections 2024-09-12T02:13:36.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.762+0000 7f65d0bc8640 1 -- 172.21.15.16:0/1435410214 wait complete. 2024-09-12T02:13:36.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.763+0000 7f65d0bc8640 1 Processor -- start 2024-09-12T02:13:36.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.763+0000 7f65d0bc8640 1 -- start start 2024-09-12T02:13:36.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.763+0000 7f65d0bc8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f65cc100e20 0x7f65cc198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:36.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.763+0000 7f65d0bc8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65cc102570 0x7f65cc198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:36.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.763+0000 7f65d0bc8640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f65cc102ef0 0x7f65cc19ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:36.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.763+0000 7f65d0bc8640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f65cc1120a0 con 0x7f65cc102570 2024-09-12T02:13:36.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.763+0000 7f65d0bc8640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f65cc111f20 con 0x7f65cc100e20 2024-09-12T02:13:36.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.763+0000 7f65d0bc8640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f65cc112220 con 0x7f65cc102ef0 2024-09-12T02:13:36.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.763+0000 7f65c9d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65cc102570 0x7f65cc198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:36.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.763+0000 7f65c9d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65cc102570 0x7f65cc198960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52274/0 (socket says 172.21.15.16:52274) 2024-09-12T02:13:36.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.763+0000 7f65c9d9b640 1 -- 172.21.15.16:0/3694318166 learned_addr learned my addr 172.21.15.16:0/3694318166 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:13:36.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.763+0000 7f65cad9d640 1 --2- 172.21.15.16:0/3694318166 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f65cc102ef0 0x7f65cc19ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:36.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.763+0000 7f65ca59c640 1 --2- 172.21.15.16:0/3694318166 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f65cc100e20 0x7f65cc198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:36.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.763+0000 7f65c9d9b640 1 -- 172.21.15.16:0/3694318166 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f65cc102ef0 msgr2=0x7f65cc19ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:36.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.763+0000 7f65c9d9b640 1 --2- 172.21.15.16:0/3694318166 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f65cc102ef0 0x7f65cc19ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:36.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.763+0000 7f65c9d9b640 1 -- 172.21.15.16:0/3694318166 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f65cc100e20 msgr2=0x7f65cc198420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:36.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.763+0000 7f65c9d9b640 1 --2- 172.21.15.16:0/3694318166 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f65cc100e20 0x7f65cc198420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:36.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.764+0000 7f65c9d9b640 1 -- 172.21.15.16:0/3694318166 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65cc19d3d0 con 0x7f65cc102570 2024-09-12T02:13:36.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.764+0000 7f65c9d9b640 1 --2- 172.21.15.16:0/3694318166 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65cc102570 0x7f65cc198960 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7f65c000c920 tx=0x7f65c000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:36.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.764+0000 7f65bb7fe640 1 -- 172.21.15.16:0/3694318166 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65c0007c70 con 0x7f65cc102570 2024-09-12T02:13:36.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.764+0000 7f65bb7fe640 1 -- 172.21.15.16:0/3694318166 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65c0007e10 con 0x7f65cc102570 2024-09-12T02:13:36.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.764+0000 7f65d0bc8640 1 -- 172.21.15.16:0/3694318166 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65cc19d6c0 con 0x7f65cc102570 2024-09-12T02:13:36.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.764+0000 7f65bb7fe640 1 -- 172.21.15.16:0/3694318166 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65c0018890 con 0x7f65cc102570 2024-09-12T02:13:36.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.764+0000 7f65d0bc8640 1 -- 172.21.15.16:0/3694318166 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65cc1a4fa0 con 0x7f65cc102570 2024-09-12T02:13:36.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.766+0000 7f65bb7fe640 1 -- 172.21.15.16:0/3694318166 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f65c0018a70 con 0x7f65cc102570 2024-09-12T02:13:36.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.767+0000 7f65d2fa1740 1 Processor -- start 2024-09-12T02:13:36.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.767+0000 7f65d2fa1740 1 -- start start 2024-09-12T02:13:36.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.767+0000 7f65d2fa1740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563b7466ab40 0x563b74669090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:36.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.767+0000 7f65d2fa1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b746695d0 0x563b74669a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:36.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.767+0000 7f65d2fa1740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563b74674370 0x563b74676760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:36.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.767+0000 7f65d2fa1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x563b745cf2f0 con 0x563b746695d0 2024-09-12T02:13:36.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.767+0000 7f65d2fa1740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x563b74576cf0 con 0x563b74674370 2024-09-12T02:13:36.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.767+0000 7f65d2fa1740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x563b745cb2b0 con 0x563b7466ab40 2024-09-12T02:13:36.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.767+0000 7f65ca59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b746695d0 0x563b74669a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:36.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.767+0000 7f65ca59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b746695d0 0x563b74669a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52288/0 (socket says 172.21.15.16:52288) 2024-09-12T02:13:36.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.767+0000 7f65ca59c640 1 -- 172.21.15.16:0/3393820256 learned_addr learned my addr 172.21.15.16:0/3393820256 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:13:36.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.767+0000 7f65bb7fe640 1 --2- 172.21.15.16:0/3694318166 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f65b40a2b80 0x7f65b40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:36.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.767+0000 7f65cad9d640 1 --2- 172.21.15.16:0/3393820256 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563b7466ab40 0x563b74669090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:36.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.768+0000 7f65cad9d640 1 --2- 172.21.15.16:0/3694318166 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f65b40a2b80 0x7f65b40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:36.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.768+0000 7f65ca59c640 1 -- 172.21.15.16:0/3393820256 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563b7466ab40 msgr2=0x563b74669090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:36.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.768+0000 7f65ca59c640 1 --2- 172.21.15.16:0/3393820256 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563b7466ab40 0x563b74669090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:36.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.768+0000 7f65bb7fe640 1 -- 172.21.15.16:0/3694318166 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f65c00cf090 con 0x7f65cc102570 2024-09-12T02:13:36.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.768+0000 7f65ca59c640 1 -- 172.21.15.16:0/3393820256 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563b74674370 msgr2=0x563b74676760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:36.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.768+0000 7f65c9d9b640 1 --2- 172.21.15.16:0/3393820256 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563b74674370 0x563b74676760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:36.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.768+0000 7f65ca59c640 1 --2- 172.21.15.16:0/3393820256 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563b74674370 0x563b74676760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:36.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.768+0000 7f65ca59c640 1 -- 172.21.15.16:0/3393820256 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b7455b3a0 con 0x563b746695d0 2024-09-12T02:13:36.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.768+0000 7f65c9d9b640 1 --2- 172.21.15.16:0/3393820256 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563b74674370 0x563b74676760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:13:36.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.768+0000 7f65ca59c640 1 --2- 172.21.15.16:0/3393820256 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b746695d0 0x563b74669a10 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7f65bc00bd10 tx=0x7f65bc0107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18900a954f0dea90 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:36.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.768+0000 7f65cad9d640 1 --2- 172.21.15.16:0/3694318166 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f65b40a2b80 0x7f65b40a5040 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f65c4005e30 tx=0x7f65c4005c50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:36.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.768+0000 7f65b8ff9640 1 -- 172.21.15.16:0/3393820256 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65bc004030 con 0x563b746695d0 2024-09-12T02:13:36.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.768+0000 7f65b8ff9640 1 -- 172.21.15.16:0/3393820256 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65bc0041d0 con 0x563b746695d0 2024-09-12T02:13:36.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.768+0000 7f65b8ff9640 1 -- 172.21.15.16:0/3393820256 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65bc011820 con 0x563b746695d0 2024-09-12T02:13:36.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.769+0000 7f65d2fa1740 1 -- 172.21.15.16:0/3393820256 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b746695d0 msgr2=0x563b74669a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:36.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.769+0000 7f65d2fa1740 1 --2- 172.21.15.16:0/3393820256 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b746695d0 0x563b74669a10 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7f65bc00bd10 tx=0x7f65bc0107a0 comp rx=0 tx=0).stop 2024-09-12T02:13:36.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.769+0000 7f65d2fa1740 1 -- 172.21.15.16:0/3393820256 shutdown_connections 2024-09-12T02:13:36.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.769+0000 7f65d2fa1740 1 --2- 172.21.15.16:0/3393820256 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563b74674370 0x563b74676760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:36.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.769+0000 7f65d2fa1740 1 --2- 172.21.15.16:0/3393820256 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b746695d0 0x563b74669a10 unknown :-1 s=CLOSED pgs=1682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:36.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.769+0000 7f65d2fa1740 1 --2- 172.21.15.16:0/3393820256 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563b7466ab40 0x563b74669090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:36.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.769+0000 7f65d2fa1740 1 -- 172.21.15.16:0/3393820256 >> 172.21.15.16:0/3393820256 conn(0x563b74632310 msgr2=0x563b74665c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:36.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.769+0000 7f65d2fa1740 1 -- 172.21.15.16:0/3393820256 shutdown_connections 2024-09-12T02:13:36.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.769+0000 7f65d2fa1740 1 -- 172.21.15.16:0/3393820256 wait complete. 2024-09-12T02:13:36.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.770+0000 7f65d2fa1740 1 -- start start 2024-09-12T02:13:36.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.770+0000 7f65d2fa1740 1 Processor -- start 2024-09-12T02:13:36.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.770+0000 7f65d2fa1740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563b7466ab40 0x563b74671270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:36.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.770+0000 7f65d2fa1740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563b74674370 0x563b746717b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:36.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.770+0000 7f65d2fa1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b74671cf0 0x563b74680520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:36.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.770+0000 7f65d2fa1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x563b74677070 con 0x563b74671cf0 2024-09-12T02:13:36.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.770+0000 7f65d2fa1740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x563b7458a060 con 0x563b74674370 2024-09-12T02:13:36.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.770+0000 7f65d2fa1740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x563b746771f0 con 0x563b7466ab40 2024-09-12T02:13:36.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.770+0000 7f65d2fa1740 10 client.0 _ll_register_callbacks cb 0x563b742dd8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:13:36.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.770+0000 7f65cad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b74671cf0 0x563b74680520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:36.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.770+0000 7f65cad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b74671cf0 0x563b74680520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52300/0 (socket says 172.21.15.16:52300) 2024-09-12T02:13:36.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.770+0000 7f65cad9d640 1 -- 172.21.15.16:0/920058533 learned_addr learned my addr 172.21.15.16:0/920058533 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:13:36.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.770+0000 7f65c9d9b640 1 --2- 172.21.15.16:0/920058533 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563b74674370 0x563b746717b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:36.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.770+0000 7f65ca59c640 1 --2- 172.21.15.16:0/920058533 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563b7466ab40 0x563b74671270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:36.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.770+0000 7f65cad9d640 1 -- 172.21.15.16:0/920058533 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563b7466ab40 msgr2=0x563b74671270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:36.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.770+0000 7f65cad9d640 1 --2- 172.21.15.16:0/920058533 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563b7466ab40 0x563b74671270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:36.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.770+0000 7f65cad9d640 1 -- 172.21.15.16:0/920058533 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563b74674370 msgr2=0x563b746717b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:36.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.770+0000 7f65cad9d640 1 --2- 172.21.15.16:0/920058533 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563b74674370 0x563b746717b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:36.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.770+0000 7f65cad9d640 1 -- 172.21.15.16:0/920058533 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b743403c0 con 0x563b74671cf0 2024-09-12T02:13:36.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.771+0000 7f65cad9d640 1 --2- 172.21.15.16:0/920058533 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b74671cf0 0x563b74680520 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7f65c402f8c0 tx=0x7f65c40095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e73832b49848425 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:36.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.771+0000 7f65b1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:13:36.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.771+0000 7f65b1ffb640 1 -- 172.21.15.16:0/920058533 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65c404d030 con 0x563b74671cf0 2024-09-12T02:13:36.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.771+0000 7f65b1ffb640 1 -- 172.21.15.16:0/920058533 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f65c4004250 con 0x563b74671cf0 2024-09-12T02:13:36.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.771+0000 7f65d2fa1740 1 -- 172.21.15.16:0/920058533 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f65bc0041d0 con 0x563b74671cf0 2024-09-12T02:13:36.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.771+0000 7f65b1ffb640 1 -- 172.21.15.16:0/920058533 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f65c4034040 con 0x563b74671cf0 2024-09-12T02:13:36.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.771+0000 7f65d2fa1740 10 client.19404 fetch_fsmap learned FSMap version 25 2024-09-12T02:13:36.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.771+0000 7f65d2fa1740 1 -- 172.21.15.16:0/920058533 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x563b746830b0 con 0x563b74671cf0 2024-09-12T02:13:36.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.771+0000 7f65b1ffb640 1 -- 172.21.15.16:0/920058533 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65bc0041d0 con 0x563b74671cf0 2024-09-12T02:13:36.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.772+0000 7f65b1ffb640 1 -- 172.21.15.16:0/920058533 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f65c4048040 con 0x563b74671cf0 2024-09-12T02:13:36.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.772+0000 7f65d2fa1740 10 client.19404 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:13:36.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.772+0000 7f65d2fa1740 10 client.19404 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T02:13:36.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.772+0000 7f65d2fa1740 20 client.19404 populate_metadata read hostname 'smithi016' 2024-09-12T02:13:36.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.772+0000 7f65d2fa1740 1 --2- 172.21.15.16:0/920058533 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x563b74684570 0x563b746869f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:36.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.772+0000 7f65d2fa1740 4 client.19404 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:13:36.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.772+0000 7f65d2fa1740 1 -- 172.21.15.16:0/920058533 --> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563b745dc860 con 0x563b74684570 2024-09-12T02:13:36.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.772+0000 7f65ca59c640 1 --2- 172.21.15.16:0/920058533 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x563b74684570 0x563b746869f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:36.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.772+0000 7f65ca59c640 1 --2- 172.21.15.16:0/920058533 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x563b74684570 0x563b746869f0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1c82aa259ad6ffe2 server_cookie=cf67c75d8f07dec3 in_seq=0 out_seq=0 2024-09-12T02:13:36.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.772+0000 7f65b1ffb640 10 client.19404 ms_handle_connect on v2:172.21.15.184:6838/4203347941 2024-09-12T02:13:36.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.773+0000 7f65b1ffb640 1 -- 172.21.15.16:0/920058533 <== mds.1 v2:172.21.15.184:6838/4203347941 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563b745dc860 con 0x563b74684570 2024-09-12T02:13:36.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.773+0000 7f65b1ffb640 10 client.19404 handle_command_reply: tid=0 2024-09-12T02:13:36.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.773+0000 7f65d2fa1740 1 client.19404 shutdown 2024-09-12T02:13:36.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.773+0000 7f65d2fa1740 1 -- 172.21.15.16:0/920058533 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b74671cf0 msgr2=0x563b74680520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:36.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.773+0000 7f65d2fa1740 1 --2- 172.21.15.16:0/920058533 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b74671cf0 0x563b74680520 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7f65c402f8c0 tx=0x7f65c40095c0 comp rx=0 tx=0).stop 2024-09-12T02:13:36.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.773+0000 7f65d2fa1740 1 -- 172.21.15.16:0/920058533 shutdown_connections 2024-09-12T02:13:36.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.773+0000 7f65d2fa1740 1 --2- 172.21.15.16:0/920058533 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x563b74684570 0x563b746869f0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:36.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.773+0000 7f65d2fa1740 1 --2- 172.21.15.16:0/920058533 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b74671cf0 0x563b74680520 unknown :-1 s=CLOSED pgs=1683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:36.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.773+0000 7f65d2fa1740 1 --2- 172.21.15.16:0/920058533 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563b74674370 0x563b746717b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:36.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.773+0000 7f65b1ffb640 0 client.19404 ms_handle_reset on v2:172.21.15.184:6838/4203347941 2024-09-12T02:13:36.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.773+0000 7f65d2fa1740 1 --2- 172.21.15.16:0/920058533 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563b7466ab40 0x563b74671270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:36.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.773+0000 7f65d2fa1740 1 -- 172.21.15.16:0/920058533 >> 172.21.15.16:0/920058533 conn(0x563b74632310 msgr2=0x563b7466b960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:36.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.774+0000 7f65d2fa1740 1 -- 172.21.15.16:0/920058533 shutdown_connections 2024-09-12T02:13:36.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.774+0000 7f65d2fa1740 1 -- 172.21.15.16:0/920058533 wait complete. 2024-09-12T02:13:36.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.774+0000 7f65d2fa1740 20 client.19404 trim_cache size 0 max 16384 2024-09-12T02:13:36.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.787+0000 7f65d2fa1740 1 Processor -- start 2024-09-12T02:13:36.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.787+0000 7f65d2fa1740 1 -- start start 2024-09-12T02:13:36.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.787+0000 7f65d2fa1740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563b7466ab40 0x563b7466c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:36.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.787+0000 7f65d2fa1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b74671cf0 0x563b74680310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:36.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.787+0000 7f65d2fa1740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563b74674370 0x563b74680850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:36.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.787+0000 7f65d2fa1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f65cc1afb50 con 0x563b74671cf0 2024-09-12T02:13:36.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.787+0000 7f65d2fa1740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f65cc1ad420 con 0x563b74674370 2024-09-12T02:13:36.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.787+0000 7f65d2fa1740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f65cc1ab270 con 0x563b7466ab40 2024-09-12T02:13:36.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.787+0000 7f65c9d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b74671cf0 0x563b74680310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:36.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.787+0000 7f65c9d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b74671cf0 0x563b74680310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52304/0 (socket says 172.21.15.16:52304) 2024-09-12T02:13:36.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.787+0000 7f65c9d9b640 1 -- 172.21.15.16:0/3719705132 learned_addr learned my addr 172.21.15.16:0/3719705132 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:13:36.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.787+0000 7f65ca59c640 1 --2- 172.21.15.16:0/3719705132 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563b7466ab40 0x563b7466c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:36.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.787+0000 7f65cad9d640 1 --2- 172.21.15.16:0/3719705132 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563b74674370 0x563b74680850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:36.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.787+0000 7f65c9d9b640 1 -- 172.21.15.16:0/3719705132 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563b7466ab40 msgr2=0x563b7466c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:36.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.787+0000 7f65c9d9b640 1 --2- 172.21.15.16:0/3719705132 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563b7466ab40 0x563b7466c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:36.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.787+0000 7f65c9d9b640 1 -- 172.21.15.16:0/3719705132 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563b74674370 msgr2=0x563b74680850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:36.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.787+0000 7f65c9d9b640 1 --2- 172.21.15.16:0/3719705132 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563b74674370 0x563b74680850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:36.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.787+0000 7f65c9d9b640 1 -- 172.21.15.16:0/3719705132 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b74670240 con 0x563b74671cf0 2024-09-12T02:13:36.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.787+0000 7f65c9d9b640 1 --2- 172.21.15.16:0/3719705132 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b74671cf0 0x563b74680310 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7f65c00982c0 tx=0x7f65c000ff30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cf1d35add0f6b1e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:36.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.788+0000 7f65b3fff640 1 -- 172.21.15.16:0/3719705132 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65c00a4560 con 0x563b74671cf0 2024-09-12T02:13:36.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.788+0000 7f65b3fff640 1 -- 172.21.15.16:0/3719705132 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65c00ccca0 con 0x563b74671cf0 2024-09-12T02:13:36.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.788+0000 7f65b3fff640 1 -- 172.21.15.16:0/3719705132 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65c009bdb0 con 0x563b74671cf0 2024-09-12T02:13:36.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.788+0000 7f65d2fa1740 1 -- 172.21.15.16:0/3719705132 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b74671cf0 msgr2=0x563b74680310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:36.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.788+0000 7f65d2fa1740 1 --2- 172.21.15.16:0/3719705132 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b74671cf0 0x563b74680310 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7f65c00982c0 tx=0x7f65c000ff30 comp rx=0 tx=0).stop 2024-09-12T02:13:36.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.788+0000 7f65d2fa1740 1 -- 172.21.15.16:0/3719705132 shutdown_connections 2024-09-12T02:13:36.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.788+0000 7f65d2fa1740 1 --2- 172.21.15.16:0/3719705132 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563b74674370 0x563b74680850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:36.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.788+0000 7f65d2fa1740 1 --2- 172.21.15.16:0/3719705132 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b74671cf0 0x563b74680310 unknown :-1 s=CLOSED pgs=1684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:36.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.788+0000 7f65d2fa1740 1 --2- 172.21.15.16:0/3719705132 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563b7466ab40 0x563b7466c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:36.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.788+0000 7f65d2fa1740 1 -- 172.21.15.16:0/3719705132 >> 172.21.15.16:0/3719705132 conn(0x563b74632310 msgr2=0x563b7466c2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:36.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.788+0000 7f65d2fa1740 1 -- 172.21.15.16:0/3719705132 shutdown_connections 2024-09-12T02:13:36.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.788+0000 7f65d2fa1740 1 -- 172.21.15.16:0/3719705132 wait complete. 2024-09-12T02:13:36.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.789+0000 7f65d2fa1740 1 -- start start 2024-09-12T02:13:36.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.789+0000 7f65d2fa1740 1 Processor -- start 2024-09-12T02:13:36.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.789+0000 7f65d2fa1740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563b7466ab40 0x563b7466ef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:36.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.789+0000 7f65d2fa1740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563b74671cf0 0x563b7467c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:36.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.789+0000 7f65d2fa1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b74674370 0x563b7467ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:36.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.789+0000 7f65d2fa1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f65cc1ab8e0 con 0x563b74674370 2024-09-12T02:13:36.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.789+0000 7f65d2fa1740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f65cc1acd50 con 0x563b7466ab40 2024-09-12T02:13:36.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.789+0000 7f65d2fa1740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f65cc1abfb0 con 0x563b74671cf0 2024-09-12T02:13:36.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.789+0000 7f65d2fa1740 10 client.0 _ll_register_callbacks cb 0x7f65c402f7f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:13:36.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.789+0000 7f65cad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b74674370 0x563b7467ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:36.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.789+0000 7f65cad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b74674370 0x563b7467ca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52314/0 (socket says 172.21.15.16:52314) 2024-09-12T02:13:36.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.789+0000 7f65cad9d640 1 -- 172.21.15.16:0/958081252 learned_addr learned my addr 172.21.15.16:0/958081252 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:13:36.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.789+0000 7f65c9d9b640 1 --2- 172.21.15.16:0/958081252 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563b74671cf0 0x563b7467c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:36.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.789+0000 7f65ca59c640 1 --2- 172.21.15.16:0/958081252 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563b7466ab40 0x563b7466ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:36.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.790+0000 7f65cad9d640 1 -- 172.21.15.16:0/958081252 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563b74671cf0 msgr2=0x563b7467c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:36.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.790+0000 7f65cad9d640 1 --2- 172.21.15.16:0/958081252 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563b74671cf0 0x563b7467c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:36.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.790+0000 7f65cad9d640 1 -- 172.21.15.16:0/958081252 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563b7466ab40 msgr2=0x563b7466ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:36.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.790+0000 7f65cad9d640 1 --2- 172.21.15.16:0/958081252 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563b7466ab40 0x563b7466ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:36.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.790+0000 7f65cad9d640 1 -- 172.21.15.16:0/958081252 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b74675320 con 0x563b74674370 2024-09-12T02:13:36.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.790+0000 7f65cad9d640 1 --2- 172.21.15.16:0/958081252 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b74674370 0x563b7467ca00 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7f65c4009ca0 tx=0x7f65c4004460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=631b2bf38d5b037 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:36.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.790+0000 7f65b1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:13:36.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.790+0000 7f65b1ffb640 1 -- 172.21.15.16:0/958081252 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65c4060070 con 0x563b74674370 2024-09-12T02:13:36.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.790+0000 7f65b1ffb640 1 -- 172.21.15.16:0/958081252 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f65c403aa30 con 0x563b74674370 2024-09-12T02:13:36.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.790+0000 7f65d2fa1740 1 -- 172.21.15.16:0/958081252 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f65c00ccca0 con 0x563b74674370 2024-09-12T02:13:36.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.790+0000 7f65b1ffb640 1 -- 172.21.15.16:0/958081252 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f65c406c050 con 0x563b74674370 2024-09-12T02:13:36.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.790+0000 7f65d2fa1740 10 client.19410 fetch_fsmap learned FSMap version 25 2024-09-12T02:13:36.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.790+0000 7f65d2fa1740 1 -- 172.21.15.16:0/958081252 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x563b746f3920 con 0x563b74674370 2024-09-12T02:13:36.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.790+0000 7f65b1ffb640 1 -- 172.21.15.16:0/958081252 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65c00ccca0 con 0x563b74674370 2024-09-12T02:13:36.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.790+0000 7f65b1ffb640 1 -- 172.21.15.16:0/958081252 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f65c4048050 con 0x563b74674370 2024-09-12T02:13:36.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.790+0000 7f65d2fa1740 10 client.19410 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:13:36.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.790+0000 7f65d2fa1740 10 client.19410 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T02:13:36.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.790+0000 7f65d2fa1740 20 client.19410 populate_metadata read hostname 'smithi016' 2024-09-12T02:13:36.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.790+0000 7f65d2fa1740 1 --2- 172.21.15.16:0/958081252 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x563b746f4070 0x563b746f6490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:36.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.791+0000 7f65d2fa1740 4 client.19410 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T02:13:36.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.791+0000 7f65d2fa1740 1 -- 172.21.15.16:0/958081252 --> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563b745d6d70 con 0x563b746f4070 2024-09-12T02:13:36.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.791+0000 7f65ca59c640 1 --2- 172.21.15.16:0/958081252 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x563b746f4070 0x563b746f6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:36.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.791+0000 7f65b1ffb640 10 client.19410 ms_handle_connect on v2:172.21.15.184:6838/4203347941 2024-09-12T02:13:36.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.791+0000 7f65ca59c640 1 --2- 172.21.15.16:0/958081252 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x563b746f4070 0x563b746f6490 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=186b7b028f0430ec server_cookie=4494da82fed97fd0 in_seq=0 out_seq=0 2024-09-12T02:13:36.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.791+0000 7f65b1ffb640 1 -- 172.21.15.16:0/958081252 <== mds.1 v2:172.21.15.184:6838/4203347941 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563b745d6d70 con 0x563b746f4070 2024-09-12T02:13:36.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.791+0000 7f65b1ffb640 10 client.19410 handle_command_reply: tid=0 2024-09-12T02:13:36.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.791+0000 7f65d2fa1740 1 client.19410 shutdown 2024-09-12T02:13:36.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.792+0000 7f65d2fa1740 1 -- 172.21.15.16:0/958081252 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b74674370 msgr2=0x563b7467ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:36.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.792+0000 7f65d2fa1740 1 --2- 172.21.15.16:0/958081252 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b74674370 0x563b7467ca00 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7f65c4009ca0 tx=0x7f65c4004460 comp rx=0 tx=0).stop 2024-09-12T02:13:36.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.792+0000 7f65d2fa1740 1 -- 172.21.15.16:0/958081252 shutdown_connections 2024-09-12T02:13:36.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.792+0000 7f65d2fa1740 1 --2- 172.21.15.16:0/958081252 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x563b746f4070 0x563b746f6490 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:36.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.792+0000 7f65d2fa1740 1 --2- 172.21.15.16:0/958081252 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b74674370 0x563b7467ca00 unknown :-1 s=CLOSED pgs=1685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:36.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.792+0000 7f65d2fa1740 1 --2- 172.21.15.16:0/958081252 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563b74671cf0 0x563b7467c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:36.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.792+0000 7f65d2fa1740 1 --2- 172.21.15.16:0/958081252 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563b7466ab40 0x563b7466ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:36.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.792+0000 7f65d2fa1740 1 -- 172.21.15.16:0/958081252 >> 172.21.15.16:0/958081252 conn(0x563b74632310 msgr2=0x563b74672ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:36.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.792+0000 7f65b1ffb640 0 client.19410 ms_handle_reset on v2:172.21.15.184:6838/4203347941 2024-09-12T02:13:36.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.792+0000 7f65d2fa1740 1 -- 172.21.15.16:0/958081252 shutdown_connections 2024-09-12T02:13:36.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.792+0000 7f65d2fa1740 1 -- 172.21.15.16:0/958081252 wait complete. 2024-09-12T02:13:36.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.792+0000 7f65d2fa1740 20 client.19410 trim_cache size 0 max 16384 2024-09-12T02:13:36.799 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T02:13:36.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.792+0000 7f65d0bc8640 1 -- 172.21.15.16:0/3694318166 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f65b40a2b80 msgr2=0x7f65b40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:36.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.792+0000 7f65d0bc8640 1 --2- 172.21.15.16:0/3694318166 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f65b40a2b80 0x7f65b40a5040 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f65c4005e30 tx=0x7f65c4005c50 comp rx=0 tx=0).stop 2024-09-12T02:13:36.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.792+0000 7f65d0bc8640 1 -- 172.21.15.16:0/3694318166 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65cc102570 msgr2=0x7f65cc198960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:36.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.792+0000 7f65d0bc8640 1 --2- 172.21.15.16:0/3694318166 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65cc102570 0x7f65cc198960 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7f65c000c920 tx=0x7f65c000cde0 comp rx=0 tx=0).stop 2024-09-12T02:13:36.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.792+0000 7f65d0bc8640 1 -- 172.21.15.16:0/3694318166 shutdown_connections 2024-09-12T02:13:36.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.792+0000 7f65d0bc8640 1 --2- 172.21.15.16:0/3694318166 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f65b40a2b80 0x7f65b40a5040 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:36.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.792+0000 7f65d0bc8640 1 --2- 172.21.15.16:0/3694318166 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f65cc102ef0 0x7f65cc19ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:36.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.792+0000 7f65d0bc8640 1 --2- 172.21.15.16:0/3694318166 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65cc102570 0x7f65cc198960 unknown :-1 s=CLOSED pgs=1681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:36.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.792+0000 7f65d0bc8640 1 --2- 172.21.15.16:0/3694318166 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f65cc100e20 0x7f65cc198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:36.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.792+0000 7f65d0bc8640 1 -- 172.21.15.16:0/3694318166 >> 172.21.15.16:0/3694318166 conn(0x7f65cc0fc7d0 msgr2=0x7f65cc1067c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:36.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.792+0000 7f65d0bc8640 1 -- 172.21.15.16:0/3694318166 shutdown_connections 2024-09-12T02:13:36.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:36.792+0000 7f65d0bc8640 1 -- 172.21.15.16:0/3694318166 wait complete. 2024-09-12T02:13:37.253 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 27.2 2024-09-12T02:13:37.254 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:13:37.560 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-12T02:13:37.622 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:13:38.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.111+0000 7f5358254640 1 -- 172.21.15.16:0/3507258315 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5350103740 msgr2=0x7f5350107b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:38.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.111+0000 7f5358254640 1 --2- 172.21.15.16:0/3507258315 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5350103740 0x7f5350107b80 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7f5340009930 tx=0x7f534002a420 comp rx=0 tx=0).stop 2024-09-12T02:13:38.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.111+0000 7f5358254640 1 -- 172.21.15.16:0/3507258315 shutdown_connections 2024-09-12T02:13:38.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.111+0000 7f5358254640 1 --2- 172.21.15.16:0/3507258315 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5350108290 0x7f535010ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:38.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.111+0000 7f5358254640 1 --2- 172.21.15.16:0/3507258315 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5350103740 0x7f5350107b80 unknown :-1 s=CLOSED pgs=1686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:38.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.111+0000 7f5358254640 1 --2- 172.21.15.16:0/3507258315 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5350102d90 0x7f5350103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:38.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.111+0000 7f5358254640 1 -- 172.21.15.16:0/3507258315 >> 172.21.15.16:0/3507258315 conn(0x7f53500fc7d0 msgr2=0x7f53500febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:38.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.111+0000 7f5358254640 1 -- 172.21.15.16:0/3507258315 shutdown_connections 2024-09-12T02:13:38.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.111+0000 7f5358254640 1 -- 172.21.15.16:0/3507258315 wait complete. 2024-09-12T02:13:38.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.112+0000 7f5358254640 1 Processor -- start 2024-09-12T02:13:38.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.112+0000 7f5358254640 1 -- start start 2024-09-12T02:13:38.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.112+0000 7f5358254640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5350102d90 0x7f5350198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:38.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.112+0000 7f5358254640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5350103740 0x7f5350198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:38.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.112+0000 7f5358254640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5350108290 0x7f535019ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:38.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.112+0000 7f5358254640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5350112090 con 0x7f5350103740 2024-09-12T02:13:38.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.112+0000 7f5358254640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5350111f10 con 0x7f5350102d90 2024-09-12T02:13:38.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.112+0000 7f5358254640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5350112210 con 0x7f5350108290 2024-09-12T02:13:38.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.112+0000 7f53557ef640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5350103740 0x7f5350198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:38.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.112+0000 7f53557ef640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5350103740 0x7f5350198a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52320/0 (socket says 172.21.15.16:52320) 2024-09-12T02:13:38.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.112+0000 7f53557ef640 1 -- 172.21.15.16:0/2792053147 learned_addr learned my addr 172.21.15.16:0/2792053147 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:13:38.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.112+0000 7f53567f1640 1 --2- 172.21.15.16:0/2792053147 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5350108290 0x7f535019ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:38.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.112+0000 7f5355ff0640 1 --2- 172.21.15.16:0/2792053147 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5350102d90 0x7f5350198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:38.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.113+0000 7f53567f1640 1 -- 172.21.15.16:0/2792053147 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5350102d90 msgr2=0x7f5350198550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:38.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.113+0000 7f53567f1640 1 --2- 172.21.15.16:0/2792053147 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5350102d90 0x7f5350198550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:38.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.113+0000 7f53567f1640 1 -- 172.21.15.16:0/2792053147 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5350103740 msgr2=0x7f5350198a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:38.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.113+0000 7f53567f1640 1 --2- 172.21.15.16:0/2792053147 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5350103740 0x7f5350198a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:38.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.113+0000 7f53567f1640 1 -- 172.21.15.16:0/2792053147 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f535019d500 con 0x7f5350108290 2024-09-12T02:13:38.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.113+0000 7f53557ef640 1 --2- 172.21.15.16:0/2792053147 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5350103740 0x7f5350198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:13:38.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.113+0000 7f53567f1640 1 --2- 172.21.15.16:0/2792053147 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5350108290 0x7f535019ce20 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f534800cab0 tx=0x7f534800cf70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:38.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.113+0000 7f5346ffd640 1 -- 172.21.15.16:0/2792053147 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f534800ae20 con 0x7f5350108290 2024-09-12T02:13:38.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.113+0000 7f5346ffd640 1 -- 172.21.15.16:0/2792053147 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f534800f930 con 0x7f5350108290 2024-09-12T02:13:38.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.113+0000 7f5346ffd640 1 -- 172.21.15.16:0/2792053147 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5348018b30 con 0x7f5350108290 2024-09-12T02:13:38.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.113+0000 7f5358254640 1 -- 172.21.15.16:0/2792053147 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f535019d7f0 con 0x7f5350108290 2024-09-12T02:13:38.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.113+0000 7f5358254640 1 -- 172.21.15.16:0/2792053147 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5350100920 con 0x7f5350108290 2024-09-12T02:13:38.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.115+0000 7f5358254640 1 -- 172.21.15.16:0/2792053147 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f535006a7f0 con 0x7f5350108290 2024-09-12T02:13:38.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.116+0000 7f5346ffd640 1 -- 172.21.15.16:0/2792053147 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f534800fae0 con 0x7f5350108290 2024-09-12T02:13:38.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.116+0000 7f5346ffd640 1 --2- 172.21.15.16:0/2792053147 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f533c0a2c00 0x7f533c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:38.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.116+0000 7f5346ffd640 1 -- 172.21.15.16:0/2792053147 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f53480cf590 con 0x7f5350108290 2024-09-12T02:13:38.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.117+0000 7f5355ff0640 1 --2- 172.21.15.16:0/2792053147 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f533c0a2c00 0x7f533c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:38.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.117+0000 7f5355ff0640 1 --2- 172.21.15.16:0/2792053147 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f533c0a2c00 0x7f533c0a50c0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f534c005d00 tx=0x7f534c00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:38.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.121+0000 7f5346ffd640 1 -- 172.21.15.16:0/2792053147 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5348014030 con 0x7f5350108290 2024-09-12T02:13:38.133 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:13:38.183 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:13:38 smithi016 ceph-mon[30947]: pgmap v686: 97 pgs: 97 active+clean; 2.2 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 21 KiB/s wr, 19 op/s 2024-09-12T02:13:38.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.342+0000 7f5358254640 1 -- 172.21.15.16:0/2792053147 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f53501998e0 con 0x7f5350108290 2024-09-12T02:13:38.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.343+0000 7f5346ffd640 1 -- 172.21.15.16:0/2792053147 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11911 (secure 0 0 0) 0x7f53501998e0 con 0x7f5350108290 2024-09-12T02:13:38.345 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T02:13:38.345 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":25,"btime":"2024-09-12T02:01:37:192021+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3279123279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3279123279},{"type":"v1","addr":"172.21.15.16:6835","nonce":3279123279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4119452382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4119452382},{"type":"v1","addr":"172.21.15.16:6837","nonce":4119452382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/3263823165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3263823165},{"type":"v1","addr":"172.21.15.16:6839","nonce":3263823165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/1510290732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":1510290732},{"type":"v1","addr":"172.21.15.16:6841","nonce":1510290732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/279674603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":279674603},{"type":"v1","addr":"172.21.15.33:6833","nonce":279674603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3555728028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3555728028},{"type":"v1","addr":"172.21.15.33:6835","nonce":3555728028}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T01:59:56.117608+0000","modified":"2024-09-12T02:01:36.294760+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24482,"mds_1":24481,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6833/1563374106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1563374106},{"type":"v1","addr":"172.21.15.184:6833","nonce":1563374106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6837/1892833868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1892833868},{"type":"v1","addr":"172.21.15.33:6837","nonce":1892833868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/1457679686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1457679686},{"type":"v1","addr":"172.21.15.184:6835","nonce":1457679686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.184:6837/1793393087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":1793393087},{"type":"v1","addr":"172.21.15.184:6837","nonce":1793393087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.184:6839/4203347941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":4203347941},{"type":"v1","addr":"172.21.15.184:6839","nonce":4203347941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.33:6839/4136670204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":4136670204},{"type":"v1","addr":"172.21.15.33:6839","nonce":4136670204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24481,24482,24469]},"id":1}]} 2024-09-12T02:13:38.345 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 25 2024-09-12T02:13:38.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.345+0000 7f5358254640 1 -- 172.21.15.16:0/2792053147 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f533c0a2c00 msgr2=0x7f533c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:38.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.345+0000 7f5358254640 1 --2- 172.21.15.16:0/2792053147 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f533c0a2c00 0x7f533c0a50c0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f534c005d00 tx=0x7f534c00b410 comp rx=0 tx=0).stop 2024-09-12T02:13:38.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.346+0000 7f5358254640 1 -- 172.21.15.16:0/2792053147 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5350108290 msgr2=0x7f535019ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:38.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.346+0000 7f5358254640 1 --2- 172.21.15.16:0/2792053147 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5350108290 0x7f535019ce20 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f534800cab0 tx=0x7f534800cf70 comp rx=0 tx=0).stop 2024-09-12T02:13:38.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.346+0000 7f5358254640 1 -- 172.21.15.16:0/2792053147 shutdown_connections 2024-09-12T02:13:38.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.346+0000 7f5358254640 1 --2- 172.21.15.16:0/2792053147 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f533c0a2c00 0x7f533c0a50c0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:38.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.346+0000 7f5358254640 1 --2- 172.21.15.16:0/2792053147 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5350108290 0x7f535019ce20 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:38.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.346+0000 7f5358254640 1 --2- 172.21.15.16:0/2792053147 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5350103740 0x7f5350198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:38.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.346+0000 7f5358254640 1 --2- 172.21.15.16:0/2792053147 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5350102d90 0x7f5350198550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:38.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.346+0000 7f5358254640 1 -- 172.21.15.16:0/2792053147 >> 172.21.15.16:0/2792053147 conn(0x7f53500fc7d0 msgr2=0x7f5350105d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:38.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.346+0000 7f5358254640 1 -- 172.21.15.16:0/2792053147 shutdown_connections 2024-09-12T02:13:38.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.346+0000 7f5358254640 1 -- 172.21.15.16:0/2792053147 wait complete. 2024-09-12T02:13:38.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:13:38 smithi184 ceph-mon[35593]: pgmap v686: 97 pgs: 97 active+clean; 2.2 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 21 KiB/s wr, 19 op/s 2024-09-12T02:13:38.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:13:38 smithi033 ceph-mon[35332]: pgmap v686: 97 pgs: 97 active+clean; 2.2 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 21 KiB/s wr, 19 op/s 2024-09-12T02:13:38.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.808+0000 7f1802644640 1 -- 172.21.15.16:0/1762562029 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17fc106f10 msgr2=0x7f17fc10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:38.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.808+0000 7f1802644640 1 --2- 172.21.15.16:0/1762562029 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17fc106f10 0x7f17fc10ea90 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7f17f4009ca0 tx=0x7f17f402a8c0 comp rx=0 tx=0).stop 2024-09-12T02:13:38.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.808+0000 7f1802644640 1 -- 172.21.15.16:0/1762562029 shutdown_connections 2024-09-12T02:13:38.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.808+0000 7f1802644640 1 --2- 172.21.15.16:0/1762562029 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17fc106f10 0x7f17fc10ea90 unknown :-1 s=CLOSED pgs=1687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:38.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.808+0000 7f1802644640 1 --2- 172.21.15.16:0/1762562029 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f17fc102790 0x7f17fc1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:38.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.808+0000 7f1802644640 1 --2- 172.21.15.16:0/1762562029 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f17fc101de0 0x7f17fc1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:38.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.808+0000 7f1802644640 1 -- 172.21.15.16:0/1762562029 >> 172.21.15.16:0/1762562029 conn(0x7f17fc0fd8c0 msgr2=0x7f17fc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:38.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.808+0000 7f1802644640 1 -- 172.21.15.16:0/1762562029 shutdown_connections 2024-09-12T02:13:38.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.808+0000 7f1802644640 1 -- 172.21.15.16:0/1762562029 wait complete. 2024-09-12T02:13:38.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.809+0000 7f1802644640 1 Processor -- start 2024-09-12T02:13:38.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.809+0000 7f1802644640 1 -- start start 2024-09-12T02:13:38.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.809+0000 7f1802644640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17fc101de0 0x7f17fc1983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:38.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.809+0000 7f1802644640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f17fc102790 0x7f17fc198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:38.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.809+0000 7f1802644640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f17fc106f10 0x7f17fc19ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:38.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.809+0000 7f1802644640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f17fc111f70 con 0x7f17fc101de0 2024-09-12T02:13:38.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.809+0000 7f1802644640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f17fc111df0 con 0x7f17fc106f10 2024-09-12T02:13:38.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.809+0000 7f1802644640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f17fc1120f0 con 0x7f17fc102790 2024-09-12T02:13:38.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.809+0000 7f17fbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17fc101de0 0x7f17fc1983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:38.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.809+0000 7f17fbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17fc101de0 0x7f17fc1983f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52330/0 (socket says 172.21.15.16:52330) 2024-09-12T02:13:38.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.809+0000 7f17fbfff640 1 -- 172.21.15.16:0/3220405037 learned_addr learned my addr 172.21.15.16:0/3220405037 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:13:38.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.809+0000 7f1800be1640 1 --2- 172.21.15.16:0/3220405037 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f17fc106f10 0x7f17fc19ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:38.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.809+0000 7f17fb7fe640 1 --2- 172.21.15.16:0/3220405037 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f17fc102790 0x7f17fc198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:38.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.809+0000 7f17fbfff640 1 -- 172.21.15.16:0/3220405037 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f17fc102790 msgr2=0x7f17fc198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:38.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.809+0000 7f17fbfff640 1 --2- 172.21.15.16:0/3220405037 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f17fc102790 0x7f17fc198930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:38.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.809+0000 7f17fbfff640 1 -- 172.21.15.16:0/3220405037 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f17fc106f10 msgr2=0x7f17fc19ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:38.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.809+0000 7f17fbfff640 1 --2- 172.21.15.16:0/3220405037 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f17fc106f10 0x7f17fc19ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:38.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.809+0000 7f17fbfff640 1 -- 172.21.15.16:0/3220405037 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17fc19d3a0 con 0x7f17fc101de0 2024-09-12T02:13:38.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.809+0000 7f1800be1640 1 --2- 172.21.15.16:0/3220405037 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f17fc106f10 0x7f17fc19ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:13:38.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.810+0000 7f17fbfff640 1 --2- 172.21.15.16:0/3220405037 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17fc101de0 0x7f17fc1983f0 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7f17ec00e9f0 tx=0x7f17ec00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:38.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.810+0000 7f17f97fa640 1 -- 172.21.15.16:0/3220405037 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17ec00cd80 con 0x7f17fc101de0 2024-09-12T02:13:38.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.810+0000 7f17f97fa640 1 -- 172.21.15.16:0/3220405037 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17ec005500 con 0x7f17fc101de0 2024-09-12T02:13:38.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.810+0000 7f1802644640 1 -- 172.21.15.16:0/3220405037 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17fc19d690 con 0x7f17fc101de0 2024-09-12T02:13:38.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.810+0000 7f17f97fa640 1 -- 172.21.15.16:0/3220405037 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17ec010690 con 0x7f17fc101de0 2024-09-12T02:13:38.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.810+0000 7f1802644640 1 -- 172.21.15.16:0/3220405037 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17fc1a4f70 con 0x7f17fc101de0 2024-09-12T02:13:38.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.812+0000 7f17f97fa640 1 -- 172.21.15.16:0/3220405037 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f17ec0046c0 con 0x7f17fc101de0 2024-09-12T02:13:38.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.812+0000 7f1804a1d740 1 Processor -- start 2024-09-12T02:13:38.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.813+0000 7f1804a1d740 1 -- start start 2024-09-12T02:13:38.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.813+0000 7f1804a1d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56554fa96b40 0x56554fa95090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:38.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.813+0000 7f1804a1d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56554fa955d0 0x56554fa95a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:38.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.813+0000 7f1804a1d740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56554faa4780 0x56554faa6b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:38.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.813+0000 7f1804a1d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56554f9fb2f0 con 0x56554fa955d0 2024-09-12T02:13:38.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.813+0000 7f1804a1d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56554f9a2cf0 con 0x56554fa96b40 2024-09-12T02:13:38.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.813+0000 7f1804a1d740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x56554f9f72b0 con 0x56554faa4780 2024-09-12T02:13:38.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.813+0000 7f17fbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56554fa955d0 0x56554fa95a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:38.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.813+0000 7f17f97fa640 1 --2- 172.21.15.16:0/3220405037 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f17e40a2b80 0x7f17e40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:38.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.813+0000 7f17fbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56554fa955d0 0x56554fa95a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52332/0 (socket says 172.21.15.16:52332) 2024-09-12T02:13:38.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.813+0000 7f17fbfff640 1 -- 172.21.15.16:0/2510825962 learned_addr learned my addr 172.21.15.16:0/2510825962 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:13:38.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.813+0000 7f17fb7fe640 1 --2- 172.21.15.16:0/2510825962 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56554faa4780 0x56554faa6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:38.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.813+0000 7f1800be1640 1 --2- 172.21.15.16:0/2510825962 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56554fa96b40 0x56554fa95090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:38.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.813+0000 7f1800be1640 1 --2- 172.21.15.16:0/3220405037 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f17e40a2b80 0x7f17e40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:38.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.813+0000 7f17f97fa640 1 -- 172.21.15.16:0/3220405037 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f17ec0cf4e0 con 0x7f17fc101de0 2024-09-12T02:13:38.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.813+0000 7f17fbfff640 1 -- 172.21.15.16:0/2510825962 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56554faa4780 msgr2=0x56554faa6b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:38.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.813+0000 7f17fbfff640 1 --2- 172.21.15.16:0/2510825962 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56554faa4780 0x56554faa6b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:38.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.813+0000 7f17fbfff640 1 -- 172.21.15.16:0/2510825962 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56554fa96b40 msgr2=0x56554fa95090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:38.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.813+0000 7f17fbfff640 1 --2- 172.21.15.16:0/2510825962 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56554fa96b40 0x56554fa95090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:38.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.814+0000 7f17fbfff640 1 -- 172.21.15.16:0/2510825962 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56554f9873a0 con 0x56554fa955d0 2024-09-12T02:13:38.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.814+0000 7f17fb7fe640 1 --2- 172.21.15.16:0/2510825962 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56554faa4780 0x56554faa6b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:13:38.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.814+0000 7f17fbfff640 1 --2- 172.21.15.16:0/2510825962 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56554fa955d0 0x56554fa95a10 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7f17ec0cd080 tx=0x7f17ec004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eaf2c3af2452e0b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:38.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.814+0000 7f17e27fc640 1 -- 172.21.15.16:0/2510825962 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17ec09c4e0 con 0x56554fa955d0 2024-09-12T02:13:38.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.814+0000 7f1800be1640 1 --2- 172.21.15.16:0/3220405037 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f17e40a2b80 0x7f17e40a5040 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f17f4005e30 tx=0x7f17f40095c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:38.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.814+0000 7f17e27fc640 1 -- 172.21.15.16:0/2510825962 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17ec09f030 con 0x56554fa955d0 2024-09-12T02:13:38.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.814+0000 7f17e27fc640 1 -- 172.21.15.16:0/2510825962 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17ec09a170 con 0x56554fa955d0 2024-09-12T02:13:38.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.815+0000 7f1804a1d740 1 -- 172.21.15.16:0/2510825962 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56554fa955d0 msgr2=0x56554fa95a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:38.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.815+0000 7f1804a1d740 1 --2- 172.21.15.16:0/2510825962 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56554fa955d0 0x56554fa95a10 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7f17ec0cd080 tx=0x7f17ec004a40 comp rx=0 tx=0).stop 2024-09-12T02:13:38.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.815+0000 7f1804a1d740 1 -- 172.21.15.16:0/2510825962 shutdown_connections 2024-09-12T02:13:38.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.815+0000 7f1804a1d740 1 --2- 172.21.15.16:0/2510825962 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56554faa4780 0x56554faa6b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:38.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.815+0000 7f1804a1d740 1 --2- 172.21.15.16:0/2510825962 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56554fa955d0 0x56554fa95a10 unknown :-1 s=CLOSED pgs=1689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:38.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.815+0000 7f1804a1d740 1 --2- 172.21.15.16:0/2510825962 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56554fa96b40 0x56554fa95090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:38.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.815+0000 7f1804a1d740 1 -- 172.21.15.16:0/2510825962 >> 172.21.15.16:0/2510825962 conn(0x56554fa5e310 msgr2=0x56554fa91c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:38.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.815+0000 7f1804a1d740 1 -- 172.21.15.16:0/2510825962 shutdown_connections 2024-09-12T02:13:38.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.815+0000 7f1804a1d740 1 -- 172.21.15.16:0/2510825962 wait complete. 2024-09-12T02:13:38.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.815+0000 7f1804a1d740 1 -- start start 2024-09-12T02:13:38.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.815+0000 7f1804a1d740 1 Processor -- start 2024-09-12T02:13:38.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.816+0000 7f1804a1d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56554fa955d0 0x56554faa4160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:38.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.816+0000 7f1804a1d740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56554fa96b40 0x56554faa1410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:38.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.816+0000 7f1804a1d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56554faa4780 0x56554faa19e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:38.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.816+0000 7f1804a1d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56554faa7420 con 0x56554fa955d0 2024-09-12T02:13:38.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.816+0000 7f1804a1d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56554f9b6060 con 0x56554faa4780 2024-09-12T02:13:38.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.816+0000 7f1804a1d740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x56554faa75a0 con 0x56554fa96b40 2024-09-12T02:13:38.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.816+0000 7f1804a1d740 10 client.0 _ll_register_callbacks cb 0x56554f7098c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:13:38.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.816+0000 7f17fbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56554fa955d0 0x56554faa4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:38.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.816+0000 7f17fbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56554fa955d0 0x56554faa4160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52340/0 (socket says 172.21.15.16:52340) 2024-09-12T02:13:38.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.816+0000 7f17fbfff640 1 -- 172.21.15.16:0/3561782926 learned_addr learned my addr 172.21.15.16:0/3561782926 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:13:38.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.816+0000 7f1800be1640 1 --2- 172.21.15.16:0/3561782926 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56554faa4780 0x56554faa19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:38.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.816+0000 7f17fb7fe640 1 --2- 172.21.15.16:0/3561782926 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56554fa96b40 0x56554faa1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:38.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.816+0000 7f17fbfff640 1 -- 172.21.15.16:0/3561782926 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56554fa96b40 msgr2=0x56554faa1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:38.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.816+0000 7f17fbfff640 1 --2- 172.21.15.16:0/3561782926 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56554fa96b40 0x56554faa1410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:38.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.816+0000 7f17fbfff640 1 -- 172.21.15.16:0/3561782926 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56554faa4780 msgr2=0x56554faa19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:38.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.816+0000 7f17fbfff640 1 --2- 172.21.15.16:0/3561782926 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56554faa4780 0x56554faa19e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:38.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.816+0000 7f17fbfff640 1 -- 172.21.15.16:0/3561782926 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56554f76c3c0 con 0x56554fa955d0 2024-09-12T02:13:38.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.816+0000 7f17fb7fe640 1 --2- 172.21.15.16:0/3561782926 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56554fa96b40 0x56554faa1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:13:38.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.816+0000 7f17fbfff640 1 --2- 172.21.15.16:0/3561782926 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56554fa955d0 0x56554faa4160 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7f17ec09bae0 tx=0x7f17ec0981b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58d0702ad5a7cf21 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:38.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.816+0000 7f17db7fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:13:38.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.816+0000 7f17db7fe640 1 -- 172.21.15.16:0/3561782926 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17ec0b9070 con 0x56554fa955d0 2024-09-12T02:13:38.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.816+0000 7f17db7fe640 1 -- 172.21.15.16:0/3561782926 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f17ec0cf0d0 con 0x56554fa955d0 2024-09-12T02:13:38.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.816+0000 7f1804a1d740 1 -- 172.21.15.16:0/3561782926 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17ec09f030 con 0x56554fa955d0 2024-09-12T02:13:38.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.816+0000 7f17db7fe640 1 -- 172.21.15.16:0/3561782926 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f17ec0a0190 con 0x56554fa955d0 2024-09-12T02:13:38.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.817+0000 7f1804a1d740 10 client.19428 fetch_fsmap learned FSMap version 25 2024-09-12T02:13:38.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.817+0000 7f1804a1d740 1 -- 172.21.15.16:0/3561782926 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56554fab2ed0 con 0x56554fa955d0 2024-09-12T02:13:38.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.817+0000 7f17db7fe640 1 -- 172.21.15.16:0/3561782926 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17ec09f030 con 0x56554fa955d0 2024-09-12T02:13:38.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.817+0000 7f17db7fe640 1 -- 172.21.15.16:0/3561782926 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f17ec0a6ce0 con 0x56554fa955d0 2024-09-12T02:13:38.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.817+0000 7f1804a1d740 10 client.19428 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:13:38.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.817+0000 7f1804a1d740 10 client.19428 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:13:38.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.817+0000 7f1804a1d740 20 client.19428 populate_metadata read hostname 'smithi016' 2024-09-12T02:13:38.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.817+0000 7f1804a1d740 1 --2- 172.21.15.16:0/3561782926 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x56554fab4220 0x56554fab6610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:38.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.817+0000 7f1804a1d740 4 client.19428 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:13:38.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.817+0000 7f1804a1d740 1 -- 172.21.15.16:0/3561782926 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56554fa08860 con 0x56554fab4220 2024-09-12T02:13:38.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.817+0000 7f17fb7fe640 1 --2- 172.21.15.16:0/3561782926 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x56554fab4220 0x56554fab6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:38.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.818+0000 7f17db7fe640 10 client.19428 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:13:38.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.818+0000 7f17fb7fe640 1 --2- 172.21.15.16:0/3561782926 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x56554fab4220 0x56554fab6610 crc :-1 s=READY pgs=358 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9729fa422d34c553 server_cookie=d09d64bdd80fbf66 in_seq=0 out_seq=0 2024-09-12T02:13:38.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.818+0000 7f17db7fe640 1 -- 172.21.15.16:0/3561782926 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56554fa08860 con 0x56554fab4220 2024-09-12T02:13:38.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.818+0000 7f17db7fe640 10 client.19428 handle_command_reply: tid=0 2024-09-12T02:13:38.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.818+0000 7f1804a1d740 1 client.19428 shutdown 2024-09-12T02:13:38.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.819+0000 7f1804a1d740 1 -- 172.21.15.16:0/3561782926 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56554fa955d0 msgr2=0x56554faa4160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:38.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.819+0000 7f1804a1d740 1 --2- 172.21.15.16:0/3561782926 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56554fa955d0 0x56554faa4160 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7f17ec09bae0 tx=0x7f17ec0981b0 comp rx=0 tx=0).stop 2024-09-12T02:13:38.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.819+0000 7f1804a1d740 1 -- 172.21.15.16:0/3561782926 shutdown_connections 2024-09-12T02:13:38.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.819+0000 7f1804a1d740 1 --2- 172.21.15.16:0/3561782926 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x56554fab4220 0x56554fab6610 crc :-1 s=READY pgs=358 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:38.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.819+0000 7f1804a1d740 1 --2- 172.21.15.16:0/3561782926 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56554faa4780 0x56554faa19e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:38.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.819+0000 7f1804a1d740 1 --2- 172.21.15.16:0/3561782926 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56554fa96b40 0x56554faa1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:38.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.819+0000 7f17db7fe640 0 client.19428 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:13:38.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.819+0000 7f1804a1d740 1 --2- 172.21.15.16:0/3561782926 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56554fa955d0 0x56554faa4160 unknown :-1 s=CLOSED pgs=1690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:38.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.819+0000 7f1804a1d740 1 -- 172.21.15.16:0/3561782926 >> 172.21.15.16:0/3561782926 conn(0x56554fa5e310 msgr2=0x56554fa97990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:38.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.819+0000 7f1804a1d740 1 -- 172.21.15.16:0/3561782926 shutdown_connections 2024-09-12T02:13:38.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.819+0000 7f1804a1d740 1 -- 172.21.15.16:0/3561782926 wait complete. 2024-09-12T02:13:38.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.819+0000 7f1804a1d740 20 client.19428 trim_cache size 0 max 16384 2024-09-12T02:13:38.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.832+0000 7f1804a1d740 1 Processor -- start 2024-09-12T02:13:38.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.832+0000 7f1804a1d740 1 -- start start 2024-09-12T02:13:38.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.832+0000 7f1804a1d740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56554fa955d0 0x56554faa0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:38.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.832+0000 7f1804a1d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56554fa96b40 0x56554fab9630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:38.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.832+0000 7f1804a1d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56554faa4780 0x56554fab9b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:38.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.832+0000 7f1804a1d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f17fc1adb90 con 0x56554faa4780 2024-09-12T02:13:38.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.832+0000 7f1804a1d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f17fc1ab460 con 0x56554fa96b40 2024-09-12T02:13:38.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.832+0000 7f1804a1d740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f17fc1a92b0 con 0x56554fa955d0 2024-09-12T02:13:38.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.832+0000 7f1800be1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56554faa4780 0x56554fab9b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:38.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.832+0000 7f1800be1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56554faa4780 0x56554fab9b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52354/0 (socket says 172.21.15.16:52354) 2024-09-12T02:13:38.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.832+0000 7f1800be1640 1 -- 172.21.15.16:0/2934701559 learned_addr learned my addr 172.21.15.16:0/2934701559 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:13:38.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.832+0000 7f17fbfff640 1 --2- 172.21.15.16:0/2934701559 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56554fa955d0 0x56554faa0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:38.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.832+0000 7f17fb7fe640 1 --2- 172.21.15.16:0/2934701559 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56554fa96b40 0x56554fab9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:38.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.832+0000 7f1800be1640 1 -- 172.21.15.16:0/2934701559 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56554fa955d0 msgr2=0x56554faa0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:38.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.832+0000 7f1800be1640 1 --2- 172.21.15.16:0/2934701559 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56554fa955d0 0x56554faa0cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:38.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.832+0000 7f1800be1640 1 -- 172.21.15.16:0/2934701559 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56554fa96b40 msgr2=0x56554fab9630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:38.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.832+0000 7f1800be1640 1 --2- 172.21.15.16:0/2934701559 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56554fa96b40 0x56554fab9630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:38.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.832+0000 7f1800be1640 1 -- 172.21.15.16:0/2934701559 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56554fa91de0 con 0x56554faa4780 2024-09-12T02:13:38.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.832+0000 7f17fbfff640 1 --2- 172.21.15.16:0/2934701559 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56554fa955d0 0x56554faa0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:13:38.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.833+0000 7f1800be1640 1 --2- 172.21.15.16:0/2934701559 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56554faa4780 0x56554fab9b70 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7f17f4035000 tx=0x7f17f403a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7157d88ab76d1b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:38.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.833+0000 7f17e1ffb640 1 -- 172.21.15.16:0/2934701559 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17f4003970 con 0x56554faa4780 2024-09-12T02:13:38.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.833+0000 7f17e1ffb640 1 -- 172.21.15.16:0/2934701559 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17f4036040 con 0x56554faa4780 2024-09-12T02:13:38.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.833+0000 7f17e1ffb640 1 -- 172.21.15.16:0/2934701559 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17f402f610 con 0x56554faa4780 2024-09-12T02:13:38.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.833+0000 7f1804a1d740 1 -- 172.21.15.16:0/2934701559 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56554faa4780 msgr2=0x56554fab9b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:38.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.833+0000 7f1804a1d740 1 --2- 172.21.15.16:0/2934701559 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56554faa4780 0x56554fab9b70 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7f17f4035000 tx=0x7f17f403a000 comp rx=0 tx=0).stop 2024-09-12T02:13:38.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.833+0000 7f1804a1d740 1 -- 172.21.15.16:0/2934701559 shutdown_connections 2024-09-12T02:13:38.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.833+0000 7f1804a1d740 1 --2- 172.21.15.16:0/2934701559 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56554faa4780 0x56554fab9b70 unknown :-1 s=CLOSED pgs=1691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:38.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.833+0000 7f1804a1d740 1 --2- 172.21.15.16:0/2934701559 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56554fa96b40 0x56554fab9630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:38.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.833+0000 7f1804a1d740 1 --2- 172.21.15.16:0/2934701559 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56554fa955d0 0x56554faa0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:38.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.833+0000 7f1804a1d740 1 -- 172.21.15.16:0/2934701559 >> 172.21.15.16:0/2934701559 conn(0x56554fa5e310 msgr2=0x56554faa3490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:38.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.834+0000 7f1804a1d740 1 -- 172.21.15.16:0/2934701559 shutdown_connections 2024-09-12T02:13:38.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.834+0000 7f1804a1d740 1 -- 172.21.15.16:0/2934701559 wait complete. 2024-09-12T02:13:38.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.834+0000 7f1804a1d740 1 -- start start 2024-09-12T02:13:38.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.834+0000 7f1804a1d740 1 Processor -- start 2024-09-12T02:13:38.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.834+0000 7f1804a1d740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56554fa955d0 0x56554fab9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:38.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.834+0000 7f1804a1d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56554fa96b40 0x56554fab7110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:38.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.834+0000 7f1804a1d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56554faa4780 0x56554fab7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:38.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.834+0000 7f1804a1d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f17fc1a9920 con 0x56554faa4780 2024-09-12T02:13:38.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.834+0000 7f1804a1d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f17fc1aad90 con 0x56554fa96b40 2024-09-12T02:13:38.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.834+0000 7f1804a1d740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f17fc1a9ff0 con 0x56554fa955d0 2024-09-12T02:13:38.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.834+0000 7f1804a1d740 10 client.0 _ll_register_callbacks cb 0x7f17ec005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:13:38.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.835+0000 7f1800be1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56554faa4780 0x56554fab7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:38.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.835+0000 7f1800be1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56554faa4780 0x56554fab7650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52360/0 (socket says 172.21.15.16:52360) 2024-09-12T02:13:38.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.835+0000 7f1800be1640 1 -- 172.21.15.16:0/1027373567 learned_addr learned my addr 172.21.15.16:0/1027373567 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:13:38.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.835+0000 7f17fbfff640 1 --2- 172.21.15.16:0/1027373567 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56554fa955d0 0x56554fab9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:38.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.835+0000 7f17fb7fe640 1 --2- 172.21.15.16:0/1027373567 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56554fa96b40 0x56554fab7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:38.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.835+0000 7f1800be1640 1 -- 172.21.15.16:0/1027373567 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56554fa955d0 msgr2=0x56554fab9dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:38.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.835+0000 7f1800be1640 1 --2- 172.21.15.16:0/1027373567 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56554fa955d0 0x56554fab9dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:38.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.835+0000 7f1800be1640 1 -- 172.21.15.16:0/1027373567 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56554fa96b40 msgr2=0x56554fab7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:38.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.835+0000 7f1800be1640 1 --2- 172.21.15.16:0/1027373567 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56554fa96b40 0x56554fab7110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:38.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.835+0000 7f1800be1640 1 -- 172.21.15.16:0/1027373567 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56554fab7e50 con 0x56554faa4780 2024-09-12T02:13:38.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.835+0000 7f1800be1640 1 --2- 172.21.15.16:0/1027373567 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56554faa4780 0x56554fab7650 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7f17f4009ca0 tx=0x7f17f402fcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e213b0876d12cfc server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:38.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.835+0000 7f17db7fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:13:38.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.835+0000 7f17db7fe640 1 -- 172.21.15.16:0/1027373567 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17f4004470 con 0x56554faa4780 2024-09-12T02:13:38.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.835+0000 7f17db7fe640 1 -- 172.21.15.16:0/1027373567 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f17f402f330 con 0x56554faa4780 2024-09-12T02:13:38.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.835+0000 7f1804a1d740 1 -- 172.21.15.16:0/1027373567 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17f4036040 con 0x56554faa4780 2024-09-12T02:13:38.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.835+0000 7f17db7fe640 1 -- 172.21.15.16:0/1027373567 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f17f4004b40 con 0x56554faa4780 2024-09-12T02:13:38.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.835+0000 7f1804a1d740 10 client.19434 fetch_fsmap learned FSMap version 25 2024-09-12T02:13:38.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.835+0000 7f1804a1d740 1 -- 172.21.15.16:0/1027373567 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56554fb1eb90 con 0x56554faa4780 2024-09-12T02:13:38.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.836+0000 7f17db7fe640 1 -- 172.21.15.16:0/1027373567 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17f4036040 con 0x56554faa4780 2024-09-12T02:13:38.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.836+0000 7f17db7fe640 1 -- 172.21.15.16:0/1027373567 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f17f4003970 con 0x56554faa4780 2024-09-12T02:13:38.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.836+0000 7f1804a1d740 10 client.19434 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:13:38.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.836+0000 7f1804a1d740 10 client.19434 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:13:38.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.836+0000 7f1804a1d740 20 client.19434 populate_metadata read hostname 'smithi016' 2024-09-12T02:13:38.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.836+0000 7f1804a1d740 1 --2- 172.21.15.16:0/1027373567 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x56554fb1f250 0x56554fb21670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:38.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.836+0000 7f1804a1d740 4 client.19434 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T02:13:38.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.836+0000 7f1804a1d740 1 -- 172.21.15.16:0/1027373567 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56554fa02d70 con 0x56554fb1f250 2024-09-12T02:13:38.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.836+0000 7f17fbfff640 1 --2- 172.21.15.16:0/1027373567 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x56554fb1f250 0x56554fb21670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:38.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.836+0000 7f17fbfff640 1 --2- 172.21.15.16:0/1027373567 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x56554fb1f250 0x56554fb21670 crc :-1 s=READY pgs=359 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=35f75c8faddf2c48 server_cookie=5a23d396e3a2d1b5 in_seq=0 out_seq=0 2024-09-12T02:13:38.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.836+0000 7f17db7fe640 10 client.19434 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:13:38.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.837+0000 7f17db7fe640 1 -- 172.21.15.16:0/1027373567 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56554fa02d70 con 0x56554fb1f250 2024-09-12T02:13:38.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.837+0000 7f17db7fe640 10 client.19434 handle_command_reply: tid=0 2024-09-12T02:13:38.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.837+0000 7f1804a1d740 1 client.19434 shutdown 2024-09-12T02:13:38.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.837+0000 7f1804a1d740 1 -- 172.21.15.16:0/1027373567 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56554faa4780 msgr2=0x56554fab7650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:38.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.837+0000 7f1804a1d740 1 --2- 172.21.15.16:0/1027373567 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56554faa4780 0x56554fab7650 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7f17f4009ca0 tx=0x7f17f402fcf0 comp rx=0 tx=0).stop 2024-09-12T02:13:38.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.837+0000 7f1804a1d740 1 -- 172.21.15.16:0/1027373567 shutdown_connections 2024-09-12T02:13:38.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.837+0000 7f1804a1d740 1 --2- 172.21.15.16:0/1027373567 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x56554fb1f250 0x56554fb21670 crc :-1 s=READY pgs=359 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:38.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.837+0000 7f1804a1d740 1 --2- 172.21.15.16:0/1027373567 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56554faa4780 0x56554fab7650 unknown :-1 s=CLOSED pgs=1692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:38.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.837+0000 7f1804a1d740 1 --2- 172.21.15.16:0/1027373567 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56554fa96b40 0x56554fab7110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:38.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.837+0000 7f17db7fe640 0 client.19434 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:13:38.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.837+0000 7f1804a1d740 1 --2- 172.21.15.16:0/1027373567 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56554fa955d0 0x56554fab9dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:38.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.837+0000 7f1804a1d740 1 -- 172.21.15.16:0/1027373567 >> 172.21.15.16:0/1027373567 conn(0x56554fa5e310 msgr2=0x56554fa9fbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:38.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.837+0000 7f1804a1d740 1 -- 172.21.15.16:0/1027373567 shutdown_connections 2024-09-12T02:13:38.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.837+0000 7f1804a1d740 1 -- 172.21.15.16:0/1027373567 wait complete. 2024-09-12T02:13:38.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.837+0000 7f1804a1d740 20 client.19434 trim_cache size 0 max 16384 2024-09-12T02:13:38.849 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T02:13:38.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.838+0000 7f1802644640 1 -- 172.21.15.16:0/3220405037 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f17e40a2b80 msgr2=0x7f17e40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:38.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.838+0000 7f1802644640 1 --2- 172.21.15.16:0/3220405037 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f17e40a2b80 0x7f17e40a5040 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f17f4005e30 tx=0x7f17f40095c0 comp rx=0 tx=0).stop 2024-09-12T02:13:38.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.838+0000 7f1802644640 1 -- 172.21.15.16:0/3220405037 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17fc101de0 msgr2=0x7f17fc1983f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:38.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.838+0000 7f1802644640 1 --2- 172.21.15.16:0/3220405037 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17fc101de0 0x7f17fc1983f0 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7f17ec00e9f0 tx=0x7f17ec00edf0 comp rx=0 tx=0).stop 2024-09-12T02:13:38.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.838+0000 7f1802644640 1 -- 172.21.15.16:0/3220405037 shutdown_connections 2024-09-12T02:13:38.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.838+0000 7f1802644640 1 --2- 172.21.15.16:0/3220405037 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f17e40a2b80 0x7f17e40a5040 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:38.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.838+0000 7f1802644640 1 --2- 172.21.15.16:0/3220405037 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f17fc106f10 0x7f17fc19ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:38.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.838+0000 7f1802644640 1 --2- 172.21.15.16:0/3220405037 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f17fc102790 0x7f17fc198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:38.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.838+0000 7f1802644640 1 --2- 172.21.15.16:0/3220405037 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17fc101de0 0x7f17fc1983f0 unknown :-1 s=CLOSED pgs=1688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:38.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.838+0000 7f1802644640 1 -- 172.21.15.16:0/3220405037 >> 172.21.15.16:0/3220405037 conn(0x7f17fc0fd8c0 msgr2=0x7f17fc0fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:38.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.838+0000 7f1802644640 1 -- 172.21.15.16:0/3220405037 shutdown_connections 2024-09-12T02:13:38.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:38.838+0000 7f1802644640 1 -- 172.21.15.16:0/3220405037 wait complete. 2024-09-12T02:13:39.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:13:39 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/2792053147' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:13:39.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:13:39 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/2792053147' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:13:39.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:13:39 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/2792053147' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:13:40.043 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24482 quiesce db / --timeout 90 --expiration 147.2 --await-for 45.0' 2024-09-12T02:13:40.044 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24482 quiesce db / --timeout 90 --expiration 147.2 --await-for 45.0 2024-09-12T02:13:40.308 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:13:40.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:13:40 smithi184 ceph-mon[35593]: pgmap v687: 97 pgs: 97 active+clean; 2.2 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-12T02:13:40.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:13:40 smithi033 ceph-mon[35332]: pgmap v687: 97 pgs: 97 active+clean; 2.2 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-12T02:13:40.430 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:13:40 smithi016 ceph-mon[30947]: pgmap v687: 97 pgs: 97 active+clean; 2.2 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-12T02:13:40.605 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-12T02:13:40.606 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-12T02:13:40.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.812+0000 7f1feea1c640 1 -- 172.21.15.16:0/390378229 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1fe81027a0 msgr2=0x7f1fe81067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:40.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.812+0000 7f1feea1c640 1 --2- 172.21.15.16:0/390378229 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1fe81027a0 0x7f1fe81067f0 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7f1fdc009900 tx=0x7f1fdc02a410 comp rx=0 tx=0).stop 2024-09-12T02:13:40.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.812+0000 7f1feea1c640 1 -- 172.21.15.16:0/390378229 shutdown_connections 2024-09-12T02:13:40.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.812+0000 7f1feea1c640 1 --2- 172.21.15.16:0/390378229 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1fe8106f20 0x7f1fe810eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:40.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.812+0000 7f1feea1c640 1 --2- 172.21.15.16:0/390378229 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1fe81027a0 0x7f1fe81067f0 unknown :-1 s=CLOSED pgs=1693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:40.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.812+0000 7f1feea1c640 1 --2- 172.21.15.16:0/390378229 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1fe8101df0 0x7f1fe81021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:40.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.812+0000 7f1feea1c640 1 -- 172.21.15.16:0/390378229 >> 172.21.15.16:0/390378229 conn(0x7f1fe80fd8f0 msgr2=0x7f1fe80ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:40.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.812+0000 7f1feea1c640 1 -- 172.21.15.16:0/390378229 shutdown_connections 2024-09-12T02:13:40.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.812+0000 7f1feea1c640 1 -- 172.21.15.16:0/390378229 wait complete. 2024-09-12T02:13:40.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.812+0000 7f1feea1c640 1 Processor -- start 2024-09-12T02:13:40.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.812+0000 7f1feea1c640 1 -- start start 2024-09-12T02:13:40.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.813+0000 7f1feea1c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1fe8101df0 0x7f1fe8198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:40.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.813+0000 7f1feea1c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1fe81027a0 0x7f1fe81989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:40.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.813+0000 7f1feea1c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1fe8106f20 0x7f1fe819cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:40.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.813+0000 7f1feea1c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1fe8111f80 con 0x7f1fe8106f20 2024-09-12T02:13:40.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.813+0000 7f1feea1c640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1fe8111e00 con 0x7f1fe8101df0 2024-09-12T02:13:40.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.813+0000 7f1feea1c640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1fe8112100 con 0x7f1fe81027a0 2024-09-12T02:13:40.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.813+0000 7f1fecfb9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1fe8106f20 0x7f1fe819cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:40.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.813+0000 7f1fecfb9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1fe8106f20 0x7f1fe819cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52380/0 (socket says 172.21.15.16:52380) 2024-09-12T02:13:40.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.813+0000 7f1fecfb9640 1 -- 172.21.15.16:0/1750800540 learned_addr learned my addr 172.21.15.16:0/1750800540 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:13:40.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.813+0000 7f1fe7fff640 1 --2- 172.21.15.16:0/1750800540 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1fe8101df0 0x7f1fe8198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:40.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.813+0000 7f1fe77fe640 1 --2- 172.21.15.16:0/1750800540 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1fe81027a0 0x7f1fe81989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:40.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.813+0000 7f1fe7fff640 1 -- 172.21.15.16:0/1750800540 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1fe81027a0 msgr2=0x7f1fe81989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:40.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.813+0000 7f1fe7fff640 1 --2- 172.21.15.16:0/1750800540 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1fe81027a0 0x7f1fe81989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:40.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.813+0000 7f1fe7fff640 1 -- 172.21.15.16:0/1750800540 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1fe8106f20 msgr2=0x7f1fe819cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:40.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.813+0000 7f1fe7fff640 1 --2- 172.21.15.16:0/1750800540 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1fe8106f20 0x7f1fe819cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:40.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.814+0000 7f1fe7fff640 1 -- 172.21.15.16:0/1750800540 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fe819d410 con 0x7f1fe8101df0 2024-09-12T02:13:40.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.814+0000 7f1fe7fff640 1 --2- 172.21.15.16:0/1750800540 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1fe8101df0 0x7f1fe8198460 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f1fd800d720 tx=0x7f1fd800dbe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:40.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.814+0000 7f1fe57fa640 1 -- 172.21.15.16:0/1750800540 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fd8017070 con 0x7f1fe8101df0 2024-09-12T02:13:40.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.814+0000 7f1feea1c640 1 -- 172.21.15.16:0/1750800540 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fe819d700 con 0x7f1fe8101df0 2024-09-12T02:13:40.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.814+0000 7f1fe57fa640 1 -- 172.21.15.16:0/1750800540 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1fd800de30 con 0x7f1fe8101df0 2024-09-12T02:13:40.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.814+0000 7f1fe57fa640 1 -- 172.21.15.16:0/1750800540 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fd80047a0 con 0x7f1fe8101df0 2024-09-12T02:13:40.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.814+0000 7f1feea1c640 1 -- 172.21.15.16:0/1750800540 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fe81a4fe0 con 0x7f1fe8101df0 2024-09-12T02:13:40.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.817+0000 7f1ff0df5740 1 Processor -- start 2024-09-12T02:13:40.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.817+0000 7f1ff0df5740 1 -- start start 2024-09-12T02:13:40.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.817+0000 7f1fe57fa640 1 -- 172.21.15.16:0/1750800540 <== mon.2 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f1fd8004940 con 0x7f1fe8101df0 2024-09-12T02:13:40.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.817+0000 7f1ff0df5740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dbdda28660 0x55dbdda28a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:40.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.817+0000 7f1ff0df5740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dbdda28f80 0x55dbdda37810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:40.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.817+0000 7f1ff0df5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dbdda37d50 0x55dbdda3a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:40.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.817+0000 7f1ff0df5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55dbdd98e2a0 con 0x55dbdda37d50 2024-09-12T02:13:40.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.817+0000 7f1ff0df5740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55dbdd935b10 con 0x55dbdda28f80 2024-09-12T02:13:40.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.817+0000 7f1ff0df5740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55dbdd98a260 con 0x55dbdda28660 2024-09-12T02:13:40.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.817+0000 7f1fe77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dbdda37d50 0x55dbdda3a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:40.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.817+0000 7f1fe7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dbdda28f80 0x55dbdda37810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:40.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.818+0000 7f1fe77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dbdda37d50 0x55dbdda3a140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52386/0 (socket says 172.21.15.16:52386) 2024-09-12T02:13:40.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.818+0000 7f1fe77fe640 1 -- 172.21.15.16:0/1162157880 learned_addr learned my addr 172.21.15.16:0/1162157880 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:13:40.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.818+0000 7f1fecfb9640 1 --2- 172.21.15.16:0/1162157880 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dbdda28660 0x55dbdda28a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:40.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.818+0000 7f1fe77fe640 1 -- 172.21.15.16:0/1162157880 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dbdda28660 msgr2=0x55dbdda28a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:40.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.818+0000 7f1fe77fe640 1 --2- 172.21.15.16:0/1162157880 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dbdda28660 0x55dbdda28a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:40.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.818+0000 7f1fe77fe640 1 -- 172.21.15.16:0/1162157880 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dbdda28f80 msgr2=0x55dbdda37810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:40.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.818+0000 7f1fe77fe640 1 --2- 172.21.15.16:0/1162157880 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dbdda28f80 0x55dbdda37810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:40.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.818+0000 7f1fe77fe640 1 -- 172.21.15.16:0/1162157880 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dbdd6ffb90 con 0x55dbdda37d50 2024-09-12T02:13:40.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.818+0000 7f1fe7fff640 1 --2- 172.21.15.16:0/1162157880 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dbdda28f80 0x55dbdda37810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:13:40.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.818+0000 7f1fe57fa640 1 --2- 172.21.15.16:0/1750800540 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f1fd00a2c00 0x7f1fd00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:40.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.818+0000 7f1fecfb9640 1 --2- 172.21.15.16:0/1750800540 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f1fd00a2c00 0x7f1fd00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:40.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.818+0000 7f1fe57fa640 1 -- 172.21.15.16:0/1750800540 <== mon.2 v2:172.21.15.33:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f1fd80cec00 con 0x7f1fe8101df0 2024-09-12T02:13:40.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.818+0000 7f1fe77fe640 1 --2- 172.21.15.16:0/1162157880 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dbdda37d50 0x55dbdda3a140 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7f1fdc009900 tx=0x7f1fdc005e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a24ca00e20350f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:40.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.818+0000 7f1fce7fc640 1 -- 172.21.15.16:0/1162157880 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fdc040070 con 0x55dbdda37d50 2024-09-12T02:13:40.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.819+0000 7f1fce7fc640 1 -- 172.21.15.16:0/1162157880 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1fdc031830 con 0x55dbdda37d50 2024-09-12T02:13:40.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.819+0000 7f1fce7fc640 1 -- 172.21.15.16:0/1162157880 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fdc035040 con 0x55dbdda37d50 2024-09-12T02:13:40.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.819+0000 7f1ff0df5740 1 -- 172.21.15.16:0/1162157880 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dbdda37d50 msgr2=0x55dbdda3a140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:40.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.819+0000 7f1fecfb9640 1 --2- 172.21.15.16:0/1750800540 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f1fd00a2c00 0x7f1fd00a50c0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f1fe00049d0 tx=0x7f1fe0009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:40.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.819+0000 7f1ff0df5740 1 --2- 172.21.15.16:0/1162157880 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dbdda37d50 0x55dbdda3a140 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7f1fdc009900 tx=0x7f1fdc005e00 comp rx=0 tx=0).stop 2024-09-12T02:13:40.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.819+0000 7f1ff0df5740 1 -- 172.21.15.16:0/1162157880 shutdown_connections 2024-09-12T02:13:40.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.819+0000 7f1ff0df5740 1 --2- 172.21.15.16:0/1162157880 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dbdda37d50 0x55dbdda3a140 unknown :-1 s=CLOSED pgs=1694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:40.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.820+0000 7f1ff0df5740 1 --2- 172.21.15.16:0/1162157880 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dbdda28f80 0x55dbdda37810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:40.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.820+0000 7f1ff0df5740 1 --2- 172.21.15.16:0/1162157880 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dbdda28660 0x55dbdda28a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:40.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.820+0000 7f1ff0df5740 1 -- 172.21.15.16:0/1162157880 >> 172.21.15.16:0/1162157880 conn(0x55dbdda24cf0 msgr2=0x55dbdda250d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:40.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.820+0000 7f1ff0df5740 1 -- 172.21.15.16:0/1162157880 shutdown_connections 2024-09-12T02:13:40.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.820+0000 7f1ff0df5740 1 -- 172.21.15.16:0/1162157880 wait complete. 2024-09-12T02:13:40.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.820+0000 7f1ff0df5740 1 -- start start 2024-09-12T02:13:40.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.820+0000 7f1ff0df5740 1 Processor -- start 2024-09-12T02:13:40.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.821+0000 7f1ff0df5740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dbdda28660 0x55dbdda34ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:40.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.821+0000 7f1ff0df5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dbdda28f80 0x55dbdda353e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:40.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.821+0000 7f1ff0df5740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dbdda35920 0x55dbdda43e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:40.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.821+0000 7f1ff0df5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55dbdda3a920 con 0x55dbdda28f80 2024-09-12T02:13:40.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.821+0000 7f1ff0df5740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55dbdd9166e0 con 0x55dbdda35920 2024-09-12T02:13:40.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.821+0000 7f1ff0df5740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55dbdda3aaa0 con 0x55dbdda28660 2024-09-12T02:13:40.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.821+0000 7f1ff0df5740 10 client.0 _ll_register_callbacks cb 0x55dbdd93aa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:13:40.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.821+0000 7f1fe77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dbdda28f80 0x55dbdda353e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:40.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.821+0000 7f1fe7fff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dbdda28660 0x55dbdda34ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:40.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.821+0000 7f1fe77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dbdda28f80 0x55dbdda353e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52392/0 (socket says 172.21.15.16:52392) 2024-09-12T02:13:40.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.821+0000 7f1fe77fe640 1 -- 172.21.15.16:0/2345225655 learned_addr learned my addr 172.21.15.16:0/2345225655 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:13:40.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.821+0000 7f1fe7fff640 1 -- 172.21.15.16:0/2345225655 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dbdda35920 msgr2=0x55dbdda43e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:40.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.821+0000 7f1fe7fff640 1 --2- 172.21.15.16:0/2345225655 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dbdda35920 0x55dbdda43e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:40.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.821+0000 7f1fe7fff640 1 -- 172.21.15.16:0/2345225655 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dbdda28f80 msgr2=0x55dbdda353e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:40.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.821+0000 7f1fe7fff640 1 --2- 172.21.15.16:0/2345225655 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dbdda28f80 0x55dbdda353e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:40.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.821+0000 7f1fe7fff640 1 -- 172.21.15.16:0/2345225655 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dbdda444c0 con 0x55dbdda28660 2024-09-12T02:13:40.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.821+0000 7f1fe77fe640 1 --2- 172.21.15.16:0/2345225655 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dbdda28f80 0x55dbdda353e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:13:40.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.822+0000 7f1fe7fff640 1 --2- 172.21.15.16:0/2345225655 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dbdda28660 0x55dbdda34ea0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x55dbdda2b350 tx=0x7f1fd8005240 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eef0a30734ff7c40 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:40.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.822+0000 7f1fc77fe640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-09-12T02:13:40.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.822+0000 7f1fc77fe640 1 -- 172.21.15.16:0/2345225655 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fd80d3040 con 0x55dbdda28660 2024-09-12T02:13:40.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.822+0000 7f1ff0df5740 1 -- 172.21.15.16:0/2345225655 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1fdc031830 con 0x55dbdda28660 2024-09-12T02:13:40.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.822+0000 7f1fc77fe640 1 -- 172.21.15.16:0/2345225655 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f1fd80a3c40 con 0x55dbdda28660 2024-09-12T02:13:40.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.822+0000 7f1fc77fe640 1 -- 172.21.15.16:0/2345225655 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1fd809b230 con 0x55dbdda28660 2024-09-12T02:13:40.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.822+0000 7f1ff0df5740 10 client.28078 fetch_fsmap learned FSMap version 25 2024-09-12T02:13:40.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.822+0000 7f1ff0df5740 1 -- 172.21.15.16:0/2345225655 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55dbdda2b3f0 con 0x55dbdda28660 2024-09-12T02:13:40.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.822+0000 7f1fc77fe640 1 -- 172.21.15.16:0/2345225655 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1fdc031830 con 0x55dbdda28660 2024-09-12T02:13:40.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.823+0000 7f1fc77fe640 1 -- 172.21.15.16:0/2345225655 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f1fd80b4220 con 0x55dbdda28660 2024-09-12T02:13:40.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.823+0000 7f1ff0df5740 10 client.28078 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:13:40.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.823+0000 7f1ff0df5740 10 client.28078 resolve_mds: validated gid 24482 aka daemon mds.b 2024-09-12T02:13:40.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.823+0000 7f1ff0df5740 20 client.28078 populate_metadata read hostname 'smithi016' 2024-09-12T02:13:40.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.823+0000 7f1ff0df5740 1 --2- 172.21.15.16:0/2345225655 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55dbdda2c8b0 0x55dbdda2ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:40.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.823+0000 7f1ff0df5740 4 client.28078 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:13:40.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.823+0000 7f1ff0df5740 1 -- 172.21.15.16:0/2345225655 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dbdd99b9d0 con 0x55dbdda2c8b0 2024-09-12T02:13:40.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.823+0000 7f1fe77fe640 1 --2- 172.21.15.16:0/2345225655 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55dbdda2c8b0 0x55dbdda2ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:40.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.824+0000 7f1fc77fe640 10 client.28078 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:13:40.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.824+0000 7f1fe77fe640 1 --2- 172.21.15.16:0/2345225655 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55dbdda2c8b0 0x55dbdda2ed40 crc :-1 s=READY pgs=360 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ca0f5e5245be3dc server_cookie=b6c2de080bf50ef8 in_seq=0 out_seq=0 2024-09-12T02:13:40.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.824+0000 7f1fc77fe640 1 -- 172.21.15.16:0/2345225655 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55dbdd99b9d0 con 0x55dbdda2c8b0 2024-09-12T02:13:40.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.824+0000 7f1fc77fe640 10 client.28078 handle_command_reply: tid=0 2024-09-12T02:13:40.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.824+0000 7f1ff0df5740 1 client.28078 shutdown 2024-09-12T02:13:40.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.824+0000 7f1ff0df5740 1 -- 172.21.15.16:0/2345225655 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dbdda28660 msgr2=0x55dbdda34ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:40.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.824+0000 7f1ff0df5740 1 --2- 172.21.15.16:0/2345225655 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dbdda28660 0x55dbdda34ea0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x55dbdda2b350 tx=0x7f1fd8005240 comp rx=0 tx=0).stop 2024-09-12T02:13:40.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.824+0000 7f1ff0df5740 1 -- 172.21.15.16:0/2345225655 shutdown_connections 2024-09-12T02:13:40.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.824+0000 7f1ff0df5740 1 --2- 172.21.15.16:0/2345225655 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55dbdda2c8b0 0x55dbdda2ed40 crc :-1 s=READY pgs=360 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:40.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.824+0000 7f1fc77fe640 0 client.28078 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:13:40.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.825+0000 7f1ff0df5740 1 --2- 172.21.15.16:0/2345225655 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dbdda35920 0x55dbdda43e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:40.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.825+0000 7f1ff0df5740 1 --2- 172.21.15.16:0/2345225655 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dbdda28f80 0x55dbdda353e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:40.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.825+0000 7f1ff0df5740 1 --2- 172.21.15.16:0/2345225655 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dbdda28660 0x55dbdda34ea0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:40.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.825+0000 7f1ff0df5740 1 -- 172.21.15.16:0/2345225655 >> 172.21.15.16:0/2345225655 conn(0x55dbdda24cf0 msgr2=0x55dbdda2aa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:40.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.825+0000 7f1ff0df5740 1 -- 172.21.15.16:0/2345225655 shutdown_connections 2024-09-12T02:13:40.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.825+0000 7f1ff0df5740 1 -- 172.21.15.16:0/2345225655 wait complete. 2024-09-12T02:13:40.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.825+0000 7f1ff0df5740 20 client.28078 trim_cache size 0 max 16384 2024-09-12T02:13:40.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.845+0000 7f1ff0df5740 1 Processor -- start 2024-09-12T02:13:40.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.846+0000 7f1ff0df5740 1 -- start start 2024-09-12T02:13:40.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.846+0000 7f1ff0df5740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dbdda24cf0 0x55dbdda43d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:40.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.846+0000 7f1ff0df5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dbdda28660 0x55dbdda2a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:40.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.846+0000 7f1ff0df5740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dbdda28f80 0x55dbdda32100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:40.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.846+0000 7f1ff0df5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1fe81adab0 con 0x55dbdda28660 2024-09-12T02:13:40.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.846+0000 7f1ff0df5740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1fe81ab380 con 0x55dbdda24cf0 2024-09-12T02:13:40.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.846+0000 7f1ff0df5740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1fe81a91f0 con 0x55dbdda28f80 2024-09-12T02:13:40.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.846+0000 7f1fe77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dbdda28660 0x55dbdda2a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:40.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.846+0000 7f1fe77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dbdda28660 0x55dbdda2a8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52406/0 (socket says 172.21.15.16:52406) 2024-09-12T02:13:40.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.846+0000 7f1fe77fe640 1 -- 172.21.15.16:0/284973033 learned_addr learned my addr 172.21.15.16:0/284973033 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:13:40.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.846+0000 7f1fecfb9640 1 --2- 172.21.15.16:0/284973033 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dbdda28f80 0x55dbdda32100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:40.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.846+0000 7f1fe7fff640 1 --2- 172.21.15.16:0/284973033 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dbdda24cf0 0x55dbdda43d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:40.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.846+0000 7f1fe77fe640 1 -- 172.21.15.16:0/284973033 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dbdda28f80 msgr2=0x55dbdda32100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:40.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.846+0000 7f1fe77fe640 1 --2- 172.21.15.16:0/284973033 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dbdda28f80 0x55dbdda32100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:40.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.846+0000 7f1fe77fe640 1 -- 172.21.15.16:0/284973033 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dbdda24cf0 msgr2=0x55dbdda43d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:40.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.846+0000 7f1fe77fe640 1 --2- 172.21.15.16:0/284973033 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dbdda24cf0 0x55dbdda43d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:40.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.846+0000 7f1fe77fe640 1 -- 172.21.15.16:0/284973033 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dbdda35e40 con 0x55dbdda28660 2024-09-12T02:13:40.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.846+0000 7f1fe7fff640 1 --2- 172.21.15.16:0/284973033 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dbdda24cf0 0x55dbdda43d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:13:40.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.847+0000 7f1fe77fe640 1 --2- 172.21.15.16:0/284973033 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dbdda28660 0x55dbdda2a8f0 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7f1fdc005420 tx=0x7f1fdc02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60ceb5f1c2dcd7da server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:40.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.847+0000 7f1fcdffb640 1 -- 172.21.15.16:0/284973033 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fdc058070 con 0x55dbdda28660 2024-09-12T02:13:40.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.847+0000 7f1fcdffb640 1 -- 172.21.15.16:0/284973033 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1fdc0054c0 con 0x55dbdda28660 2024-09-12T02:13:40.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.847+0000 7f1fcdffb640 1 -- 172.21.15.16:0/284973033 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fdc040040 con 0x55dbdda28660 2024-09-12T02:13:40.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.848+0000 7f1ff0df5740 1 -- 172.21.15.16:0/284973033 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dbdda28660 msgr2=0x55dbdda2a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:40.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.848+0000 7f1ff0df5740 1 --2- 172.21.15.16:0/284973033 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dbdda28660 0x55dbdda2a8f0 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7f1fdc005420 tx=0x7f1fdc02a410 comp rx=0 tx=0).stop 2024-09-12T02:13:40.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.848+0000 7f1ff0df5740 1 -- 172.21.15.16:0/284973033 shutdown_connections 2024-09-12T02:13:40.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.848+0000 7f1ff0df5740 1 --2- 172.21.15.16:0/284973033 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dbdda28f80 0x55dbdda32100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:40.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.848+0000 7f1ff0df5740 1 --2- 172.21.15.16:0/284973033 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dbdda28660 0x55dbdda2a8f0 unknown :-1 s=CLOSED pgs=1695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:40.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.848+0000 7f1ff0df5740 1 --2- 172.21.15.16:0/284973033 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dbdda24cf0 0x55dbdda43d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:40.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.848+0000 7f1ff0df5740 1 -- 172.21.15.16:0/284973033 >> 172.21.15.16:0/284973033 conn(0x55dbdda35540 msgr2=0x55dbdda2a1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:40.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.848+0000 7f1ff0df5740 1 -- 172.21.15.16:0/284973033 shutdown_connections 2024-09-12T02:13:40.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.848+0000 7f1ff0df5740 1 -- 172.21.15.16:0/284973033 wait complete. 2024-09-12T02:13:40.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.848+0000 7f1ff0df5740 1 -- start start 2024-09-12T02:13:40.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.848+0000 7f1ff0df5740 1 Processor -- start 2024-09-12T02:13:40.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.849+0000 7f1ff0df5740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dbdda24cf0 0x55dbdda4a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:40.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.849+0000 7f1ff0df5740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dbdda28660 0x55dbdda4a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:40.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.849+0000 7f1ff0df5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dbdda28f80 0x55dbddaa7ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:40.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.849+0000 7f1ff0df5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1fe81a9840 con 0x55dbdda28f80 2024-09-12T02:13:40.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.849+0000 7f1ff0df5740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1fe81aacb0 con 0x55dbdda24cf0 2024-09-12T02:13:40.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.849+0000 7f1ff0df5740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1fe81a9f10 con 0x55dbdda28660 2024-09-12T02:13:40.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.849+0000 7f1fecfb9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dbdda28f80 0x55dbddaa7ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:40.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.849+0000 7f1fecfb9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dbdda28f80 0x55dbddaa7ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52408/0 (socket says 172.21.15.16:52408) 2024-09-12T02:13:40.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.849+0000 7f1fecfb9640 1 -- 172.21.15.16:0/3996053937 learned_addr learned my addr 172.21.15.16:0/3996053937 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:13:40.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.849+0000 7f1ff0df5740 10 client.0 _ll_register_callbacks cb 0x7f1fd8005b90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:13:40.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.849+0000 7f1fe7fff640 1 --2- 172.21.15.16:0/3996053937 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dbdda24cf0 0x55dbdda4a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:40.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.849+0000 7f1fe77fe640 1 --2- 172.21.15.16:0/3996053937 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dbdda28660 0x55dbdda4a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:40.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.850+0000 7f1fecfb9640 1 -- 172.21.15.16:0/3996053937 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dbdda28660 msgr2=0x55dbdda4a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:40.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.850+0000 7f1fecfb9640 1 --2- 172.21.15.16:0/3996053937 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dbdda28660 0x55dbdda4a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:40.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.850+0000 7f1fecfb9640 1 -- 172.21.15.16:0/3996053937 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dbdda24cf0 msgr2=0x55dbdda4a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:40.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.850+0000 7f1fecfb9640 1 --2- 172.21.15.16:0/3996053937 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dbdda24cf0 0x55dbdda4a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:40.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.850+0000 7f1fecfb9640 1 -- 172.21.15.16:0/3996053937 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dbdda20260 con 0x55dbdda28f80 2024-09-12T02:13:40.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.850+0000 7f1fe7fff640 1 --2- 172.21.15.16:0/3996053937 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dbdda24cf0 0x55dbdda4a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:13:40.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.850+0000 7f1fecfb9640 1 --2- 172.21.15.16:0/3996053937 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dbdda28f80 0x55dbddaa7ea0 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7f1fe000fc40 tx=0x7f1fe0004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed0c70d56359a381 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:40.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.850+0000 7f1fc77fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:13:40.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.850+0000 7f1fc77fe640 1 -- 172.21.15.16:0/3996053937 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fe002f060 con 0x55dbdda28f80 2024-09-12T02:13:40.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.850+0000 7f1fc77fe640 1 -- 172.21.15.16:0/3996053937 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f1fe001d910 con 0x55dbdda28f80 2024-09-12T02:13:40.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.850+0000 7f1ff0df5740 1 -- 172.21.15.16:0/3996053937 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1fdc0054c0 con 0x55dbdda28f80 2024-09-12T02:13:40.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.850+0000 7f1fc77fe640 1 -- 172.21.15.16:0/3996053937 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1fe001db10 con 0x55dbdda28f80 2024-09-12T02:13:40.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.850+0000 7f1ff0df5740 10 client.19452 fetch_fsmap learned FSMap version 25 2024-09-12T02:13:40.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.850+0000 7f1ff0df5740 1 -- 172.21.15.16:0/3996053937 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55dbddaaf690 con 0x55dbdda28f80 2024-09-12T02:13:40.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.851+0000 7f1fc77fe640 1 -- 172.21.15.16:0/3996053937 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1fdc0054c0 con 0x55dbdda28f80 2024-09-12T02:13:40.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.851+0000 7f1fc77fe640 1 -- 172.21.15.16:0/3996053937 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f1fe0034070 con 0x55dbdda28f80 2024-09-12T02:13:40.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.851+0000 7f1ff0df5740 10 client.19452 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:13:40.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.851+0000 7f1ff0df5740 10 client.19452 resolve_mds: validated gid 24482 aka daemon mds.b 2024-09-12T02:13:40.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.851+0000 7f1ff0df5740 20 client.19452 populate_metadata read hostname 'smithi016' 2024-09-12T02:13:40.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.851+0000 7f1ff0df5740 1 --2- 172.21.15.16:0/3996053937 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55dbdda2c8b0 0x55dbddab1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:40.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.851+0000 7f1ff0df5740 4 client.19452 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 147.2, "await_for": 45.0}] 2024-09-12T02:13:40.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.851+0000 7f1ff0df5740 1 -- 172.21.15.16:0/3996053937 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 147.2, "await_for": 45.0}) -- 0x55dbdd995d20 con 0x55dbdda2c8b0 2024-09-12T02:13:40.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.851+0000 7f1fe7fff640 1 --2- 172.21.15.16:0/3996053937 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55dbdda2c8b0 0x55dbddab1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:40.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.852+0000 7f1fc77fe640 10 client.19452 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:13:40.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.852+0000 7f1fe7fff640 1 --2- 172.21.15.16:0/3996053937 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55dbdda2c8b0 0x55dbddab1ec0 crc :-1 s=READY pgs=361 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb747618481b7a0e server_cookie=dd0ab0145d34d6a6 in_seq=0 out_seq=0 2024-09-12T02:13:40.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.862+0000 7f1fc77fe640 1 -- 172.21.15.16:0/3996053937 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55dbdd995d20 con 0x55dbdda2c8b0 2024-09-12T02:13:40.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.863+0000 7f1fc77fe640 10 client.19452 handle_command_reply: tid=0 2024-09-12T02:13:40.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.863+0000 7f1ff0df5740 1 client.19452 shutdown 2024-09-12T02:13:40.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.863+0000 7f1ff0df5740 1 -- 172.21.15.16:0/3996053937 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dbdda28f80 msgr2=0x55dbddaa7ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:40.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.863+0000 7f1ff0df5740 1 --2- 172.21.15.16:0/3996053937 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dbdda28f80 0x55dbddaa7ea0 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7f1fe000fc40 tx=0x7f1fe0004ab0 comp rx=0 tx=0).stop 2024-09-12T02:13:40.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.863+0000 7f1ff0df5740 1 -- 172.21.15.16:0/3996053937 shutdown_connections 2024-09-12T02:13:40.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.863+0000 7f1ff0df5740 1 --2- 172.21.15.16:0/3996053937 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55dbdda2c8b0 0x55dbddab1ec0 crc :-1 s=READY pgs=361 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:40.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.863+0000 7f1fc77fe640 0 client.19452 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:13:40.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.863+0000 7f1ff0df5740 1 --2- 172.21.15.16:0/3996053937 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dbdda28f80 0x55dbddaa7ea0 unknown :-1 s=CLOSED pgs=1696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:40.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.863+0000 7f1ff0df5740 1 --2- 172.21.15.16:0/3996053937 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dbdda28660 0x55dbdda4a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:40.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.863+0000 7f1ff0df5740 1 --2- 172.21.15.16:0/3996053937 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dbdda24cf0 0x55dbdda4a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:40.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.863+0000 7f1ff0df5740 1 -- 172.21.15.16:0/3996053937 >> 172.21.15.16:0/3996053937 conn(0x55dbdda35540 msgr2=0x55dbdda2aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:40.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.863+0000 7f1ff0df5740 1 -- 172.21.15.16:0/3996053937 shutdown_connections 2024-09-12T02:13:40.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.863+0000 7f1ff0df5740 1 -- 172.21.15.16:0/3996053937 wait complete. 2024-09-12T02:13:40.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.863+0000 7f1ff0df5740 20 client.19452 trim_cache size 0 max 16384 2024-09-12T02:13:40.865 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T02:13:40.865 INFO:teuthology.orchestra.run.smithi016.stdout: "epoch": 25, 2024-09-12T02:13:40.865 INFO:teuthology.orchestra.run.smithi016.stdout: "leader": 24482, 2024-09-12T02:13:40.865 INFO:teuthology.orchestra.run.smithi016.stdout: "set_version": 46, 2024-09-12T02:13:40.865 INFO:teuthology.orchestra.run.smithi016.stdout: "sets": { 2024-09-12T02:13:40.865 INFO:teuthology.orchestra.run.smithi016.stdout: "336cf4e6": { 2024-09-12T02:13:40.866 INFO:teuthology.orchestra.run.smithi016.stdout: "version": 46, 2024-09-12T02:13:40.866 INFO:teuthology.orchestra.run.smithi016.stdout: "age_ref": 0.0, 2024-09-12T02:13:40.866 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-12T02:13:40.866 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "QUIESCED", 2024-09-12T02:13:40.866 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-12T02:13:40.866 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:13:40.866 INFO:teuthology.orchestra.run.smithi016.stdout: "timeout": 90.0, 2024-09-12T02:13:40.866 INFO:teuthology.orchestra.run.smithi016.stdout: "expiration": 147.2, 2024-09-12T02:13:40.866 INFO:teuthology.orchestra.run.smithi016.stdout: "members": { 2024-09-12T02:13:40.866 INFO:teuthology.orchestra.run.smithi016.stdout: "file:/": { 2024-09-12T02:13:40.866 INFO:teuthology.orchestra.run.smithi016.stdout: "excluded": false, 2024-09-12T02:13:40.866 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-12T02:13:40.866 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "QUIESCED", 2024-09-12T02:13:40.866 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-12T02:13:40.867 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:13:40.867 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:13:40.867 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:13:40.867 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:13:40.867 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:13:40.867 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T02:13:40.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.864+0000 7f1feea1c640 1 -- 172.21.15.16:0/1750800540 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f1fd00a2c00 msgr2=0x7f1fd00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:40.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.864+0000 7f1feea1c640 1 --2- 172.21.15.16:0/1750800540 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f1fd00a2c00 0x7f1fd00a50c0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f1fe00049d0 tx=0x7f1fe0009500 comp rx=0 tx=0).stop 2024-09-12T02:13:40.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.864+0000 7f1feea1c640 1 -- 172.21.15.16:0/1750800540 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1fe8101df0 msgr2=0x7f1fe8198460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:40.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.864+0000 7f1feea1c640 1 --2- 172.21.15.16:0/1750800540 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1fe8101df0 0x7f1fe8198460 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f1fd800d720 tx=0x7f1fd800dbe0 comp rx=0 tx=0).stop 2024-09-12T02:13:40.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.864+0000 7f1feea1c640 1 -- 172.21.15.16:0/1750800540 shutdown_connections 2024-09-12T02:13:40.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.864+0000 7f1feea1c640 1 --2- 172.21.15.16:0/1750800540 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f1fd00a2c00 0x7f1fd00a50c0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:40.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.864+0000 7f1feea1c640 1 --2- 172.21.15.16:0/1750800540 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1fe8106f20 0x7f1fe819cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:40.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.864+0000 7f1feea1c640 1 --2- 172.21.15.16:0/1750800540 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1fe81027a0 0x7f1fe81989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:40.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.864+0000 7f1feea1c640 1 --2- 172.21.15.16:0/1750800540 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1fe8101df0 0x7f1fe8198460 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:40.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.864+0000 7f1feea1c640 1 -- 172.21.15.16:0/1750800540 >> 172.21.15.16:0/1750800540 conn(0x7f1fe80fd8f0 msgr2=0x7f1fe80fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:40.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.864+0000 7f1feea1c640 1 -- 172.21.15.16:0/1750800540 shutdown_connections 2024-09-12T02:13:40.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:40.864+0000 7f1feea1c640 1 -- 172.21.15.16:0/1750800540 wait complete. 2024-09-12T02:13:40.875 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:13:41.757 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '336cf4e6', quiesce took 4.5 seconds. Will release after: 22.7 2024-09-12T02:13:42.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.234+0000 7f6cd7dad640 1 -- 172.21.15.16:0/726005696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6cd0100b20 msgr2=0x7f6cd0100f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:42.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.234+0000 7f6cd7dad640 1 --2- 172.21.15.16:0/726005696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6cd0100b20 0x7f6cd0100f00 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc009900 tx=0x7f6ccc02a410 comp rx=0 tx=0).stop 2024-09-12T02:13:42.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.234+0000 7f6cd7dad640 1 -- 172.21.15.16:0/726005696 shutdown_connections 2024-09-12T02:13:42.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.234+0000 7f6cd7dad640 1 --2- 172.21.15.16:0/726005696 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6cd010e500 0x7f6cd0110910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:42.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.234+0000 7f6cd7dad640 1 --2- 172.21.15.16:0/726005696 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6cd0101440 0x7f6cd010df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:42.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.234+0000 7f6cd7dad640 1 --2- 172.21.15.16:0/726005696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6cd0100b20 0x7f6cd0100f00 unknown :-1 s=CLOSED pgs=1697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:42.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.234+0000 7f6cd7dad640 1 -- 172.21.15.16:0/726005696 >> 172.21.15.16:0/726005696 conn(0x7f6cd00fc7d0 msgr2=0x7f6cd00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:42.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.235+0000 7f6cd7dad640 1 -- 172.21.15.16:0/726005696 shutdown_connections 2024-09-12T02:13:42.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.235+0000 7f6cd7dad640 1 -- 172.21.15.16:0/726005696 wait complete. 2024-09-12T02:13:42.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.235+0000 7f6cd7dad640 1 Processor -- start 2024-09-12T02:13:42.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.235+0000 7f6cd7dad640 1 -- start start 2024-09-12T02:13:42.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.236+0000 7f6cd7dad640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6cd0100b20 0x7f6cd01a0460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:42.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.236+0000 7f6cd7dad640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6cd0101440 0x7f6cd01a09a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:42.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.236+0000 7f6cd7dad640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6cd010e500 0x7f6cd019a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:42.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.236+0000 7f6cd7dad640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6cd0114340 con 0x7f6cd010e500 2024-09-12T02:13:42.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.236+0000 7f6cd7dad640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6cd01141c0 con 0x7f6cd0101440 2024-09-12T02:13:42.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.236+0000 7f6cd7dad640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f6cd01144c0 con 0x7f6cd0100b20 2024-09-12T02:13:42.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.236+0000 7f6cd5348640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6cd0101440 0x7f6cd01a09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:42.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.236+0000 7f6cd634a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6cd010e500 0x7f6cd019a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:42.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.236+0000 7f6cd5348640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6cd0101440 0x7f6cd01a09a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.16:47686/0 (socket says 172.21.15.16:47686) 2024-09-12T02:13:42.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.236+0000 7f6cd634a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6cd010e500 0x7f6cd019a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53752/0 (socket says 172.21.15.16:53752) 2024-09-12T02:13:42.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.236+0000 7f6cd5348640 1 -- 172.21.15.16:0/564863020 learned_addr learned my addr 172.21.15.16:0/564863020 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:13:42.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.236+0000 7f6cd5b49640 1 --2- 172.21.15.16:0/564863020 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6cd0100b20 0x7f6cd01a0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:42.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.236+0000 7f6cd634a640 1 -- 172.21.15.16:0/564863020 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6cd0100b20 msgr2=0x7f6cd01a0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:42.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.236+0000 7f6cd634a640 1 --2- 172.21.15.16:0/564863020 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6cd0100b20 0x7f6cd01a0460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:42.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.236+0000 7f6cd634a640 1 -- 172.21.15.16:0/564863020 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6cd0101440 msgr2=0x7f6cd01a09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:42.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.236+0000 7f6cd634a640 1 --2- 172.21.15.16:0/564863020 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6cd0101440 0x7f6cd01a09a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:42.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.236+0000 7f6cd634a640 1 -- 172.21.15.16:0/564863020 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cd019ad50 con 0x7f6cd010e500 2024-09-12T02:13:42.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.236+0000 7f6cd5348640 1 --2- 172.21.15.16:0/564863020 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6cd0101440 0x7f6cd01a09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:13:42.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.236+0000 7f6cd634a640 1 --2- 172.21.15.16:0/564863020 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6cd010e500 0x7f6cd019a620 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7f6cc8005fd0 tx=0x7f6cc800edd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:42.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.237+0000 7f6cc6ffd640 1 -- 172.21.15.16:0/564863020 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cc8004d20 con 0x7f6cd010e500 2024-09-12T02:13:42.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.237+0000 7f6cc6ffd640 1 -- 172.21.15.16:0/564863020 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6cc8005020 con 0x7f6cd010e500 2024-09-12T02:13:42.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.237+0000 7f6cd7dad640 1 -- 172.21.15.16:0/564863020 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cd019b040 con 0x7f6cd010e500 2024-09-12T02:13:42.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.237+0000 7f6cc6ffd640 1 -- 172.21.15.16:0/564863020 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cc80116d0 con 0x7f6cd010e500 2024-09-12T02:13:42.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.237+0000 7f6cd7dad640 1 -- 172.21.15.16:0/564863020 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cd01a7330 con 0x7f6cd010e500 2024-09-12T02:13:42.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.239+0000 7f6cc6ffd640 1 -- 172.21.15.16:0/564863020 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6cc8005540 con 0x7f6cd010e500 2024-09-12T02:13:42.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.239+0000 7f6cda186740 1 Processor -- start 2024-09-12T02:13:42.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.240+0000 7f6cda186740 1 -- start start 2024-09-12T02:13:42.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.240+0000 7f6cda186740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5592c9935b60 0x5592c9935f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:42.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.240+0000 7f6cda186740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5592c9936480 0x5592c9940800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:42.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.240+0000 7f6cda186740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592c9940d40 0x5592c9943130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:42.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.240+0000 7f6cda186740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5592c989b280 con 0x5592c9940d40 2024-09-12T02:13:42.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.240+0000 7f6cda186740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5592c9842b10 con 0x5592c9936480 2024-09-12T02:13:42.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.240+0000 7f6cda186740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5592c9897240 con 0x5592c9935b60 2024-09-12T02:13:42.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.240+0000 7f6cd5348640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592c9940d40 0x5592c9943130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:42.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.240+0000 7f6cc6ffd640 1 --2- 172.21.15.16:0/564863020 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f6cbc0a2b80 0x7f6cbc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:42.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.240+0000 7f6cd5348640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592c9940d40 0x5592c9943130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53758/0 (socket says 172.21.15.16:53758) 2024-09-12T02:13:42.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.240+0000 7f6cd5348640 1 -- 172.21.15.16:0/2121114707 learned_addr learned my addr 172.21.15.16:0/2121114707 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:13:42.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.240+0000 7f6cd5b49640 1 --2- 172.21.15.16:0/2121114707 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5592c9936480 0x5592c9940800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:42.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.241+0000 7f6cd5348640 1 --2- 172.21.15.16:0/2121114707 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5592c9935b60 0x5592c9935f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:42.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.241+0000 7f6cd634a640 1 --2- 172.21.15.16:0/564863020 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f6cbc0a2b80 0x7f6cbc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:42.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.241+0000 7f6cc6ffd640 1 -- 172.21.15.16:0/564863020 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f6cc80cf0d0 con 0x7f6cd010e500 2024-09-12T02:13:42.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.241+0000 7f6cd5b49640 1 -- 172.21.15.16:0/2121114707 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5592c9935b60 msgr2=0x5592c9935f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:42.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.241+0000 7f6cd5b49640 1 --2- 172.21.15.16:0/2121114707 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5592c9935b60 0x5592c9935f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:42.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.241+0000 7f6cd5b49640 1 -- 172.21.15.16:0/2121114707 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592c9940d40 msgr2=0x5592c9943130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:42.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.241+0000 7f6cd5b49640 1 --2- 172.21.15.16:0/2121114707 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592c9940d40 0x5592c9943130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:42.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.241+0000 7f6cd5b49640 1 -- 172.21.15.16:0/2121114707 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592c984f1f0 con 0x5592c9936480 2024-09-12T02:13:42.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.241+0000 7f6cd5b49640 1 --2- 172.21.15.16:0/2121114707 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5592c9936480 0x5592c9940800 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc02aa00 tx=0x7f6ccc0041a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=822cbcad057d6214 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:42.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.241+0000 7f6cd634a640 1 --2- 172.21.15.16:0/564863020 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f6cbc0a2b80 0x7f6cbc0a5040 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f6cc80983b0 tx=0x7f6cc801eb40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:42.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.241+0000 7f6cbbfff640 1 -- 172.21.15.16:0/2121114707 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ccc035040 con 0x5592c9936480 2024-09-12T02:13:42.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.241+0000 7f6cbbfff640 1 -- 172.21.15.16:0/2121114707 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ccc005520 con 0x5592c9936480 2024-09-12T02:13:42.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.241+0000 7f6cbbfff640 1 -- 172.21.15.16:0/2121114707 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ccc003850 con 0x5592c9936480 2024-09-12T02:13:42.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.242+0000 7f6cda186740 1 -- 172.21.15.16:0/2121114707 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5592c9936480 msgr2=0x5592c9940800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:42.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.242+0000 7f6cda186740 1 --2- 172.21.15.16:0/2121114707 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5592c9936480 0x5592c9940800 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc02aa00 tx=0x7f6ccc0041a0 comp rx=0 tx=0).stop 2024-09-12T02:13:42.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.242+0000 7f6cda186740 1 -- 172.21.15.16:0/2121114707 shutdown_connections 2024-09-12T02:13:42.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.242+0000 7f6cda186740 1 --2- 172.21.15.16:0/2121114707 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592c9940d40 0x5592c9943130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:42.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.242+0000 7f6cda186740 1 --2- 172.21.15.16:0/2121114707 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5592c9936480 0x5592c9940800 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:42.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.242+0000 7f6cda186740 1 --2- 172.21.15.16:0/2121114707 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5592c9935b60 0x5592c9935f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:42.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.242+0000 7f6cda186740 1 -- 172.21.15.16:0/2121114707 >> 172.21.15.16:0/2121114707 conn(0x5592c99320f0 msgr2=0x5592c99324d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:42.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.242+0000 7f6cda186740 1 -- 172.21.15.16:0/2121114707 shutdown_connections 2024-09-12T02:13:42.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.242+0000 7f6cda186740 1 -- 172.21.15.16:0/2121114707 wait complete. 2024-09-12T02:13:42.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.243+0000 7f6cda186740 1 -- start start 2024-09-12T02:13:42.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.243+0000 7f6cda186740 1 Processor -- start 2024-09-12T02:13:42.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.243+0000 7f6cda186740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5592c9935b60 0x5592c993dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:42.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.243+0000 7f6cda186740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5592c9936480 0x5592c993e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:42.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.243+0000 7f6cda186740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592c9940d40 0x5592c993e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:42.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.243+0000 7f6cda186740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5592c9943910 con 0x5592c9940d40 2024-09-12T02:13:42.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.243+0000 7f6cda186740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5592c9855f10 con 0x5592c9936480 2024-09-12T02:13:42.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.243+0000 7f6cda186740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5592c9943a90 con 0x5592c9935b60 2024-09-12T02:13:42.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.243+0000 7f6cda186740 10 client.0 _ll_register_callbacks cb 0x5592c981ea40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:13:42.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.243+0000 7f6cd5348640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592c9940d40 0x5592c993e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:42.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.243+0000 7f6cd5348640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592c9940d40 0x5592c993e6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53766/0 (socket says 172.21.15.16:53766) 2024-09-12T02:13:42.247 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:13:42 smithi016 ceph-mon[30947]: pgmap v688: 97 pgs: 97 active+clean; 2.2 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-09-12T02:13:42.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.243+0000 7f6cd5348640 1 -- 172.21.15.16:0/84407445 learned_addr learned my addr 172.21.15.16:0/84407445 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:13:42.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.243+0000 7f6cd5348640 1 --2- 172.21.15.16:0/84407445 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5592c9935b60 0x5592c993dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:42.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.243+0000 7f6cd5b49640 1 --2- 172.21.15.16:0/84407445 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5592c9936480 0x5592c993e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:42.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.244+0000 7f6cd5348640 1 -- 172.21.15.16:0/84407445 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5592c9935b60 msgr2=0x5592c993dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:42.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.244+0000 7f6cd5348640 1 --2- 172.21.15.16:0/84407445 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5592c9935b60 0x5592c993dc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:42.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.244+0000 7f6cd5348640 1 -- 172.21.15.16:0/84407445 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5592c9936480 msgr2=0x5592c993e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:42.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.244+0000 7f6cd5348640 1 --2- 172.21.15.16:0/84407445 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5592c9936480 0x5592c993e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:42.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.244+0000 7f6cd5348640 1 -- 172.21.15.16:0/84407445 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592c960c6a0 con 0x5592c9940d40 2024-09-12T02:13:42.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.244+0000 7f6cd5348640 1 --2- 172.21.15.16:0/84407445 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592c9940d40 0x5592c993e6e0 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7f6cc0016000 tx=0x7f6cc0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c25fc695325b3fe server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:42.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.244+0000 7f6cb97fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:13:42.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.244+0000 7f6cb97fa640 1 -- 172.21.15.16:0/84407445 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cc001a110 con 0x5592c9940d40 2024-09-12T02:13:42.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.244+0000 7f6cb97fa640 1 -- 172.21.15.16:0/84407445 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f6cc00177e0 con 0x5592c9940d40 2024-09-12T02:13:42.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.244+0000 7f6cda186740 1 -- 172.21.15.16:0/84407445 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ccc005520 con 0x5592c9940d40 2024-09-12T02:13:42.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.244+0000 7f6cb97fa640 1 -- 172.21.15.16:0/84407445 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6cc002d050 con 0x5592c9940d40 2024-09-12T02:13:42.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.244+0000 7f6cda186740 10 client.19464 fetch_fsmap learned FSMap version 25 2024-09-12T02:13:42.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.244+0000 7f6cda186740 1 -- 172.21.15.16:0/84407445 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5592c994f5b0 con 0x5592c9940d40 2024-09-12T02:13:42.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.245+0000 7f6cb97fa640 1 -- 172.21.15.16:0/84407445 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ccc005520 con 0x5592c9940d40 2024-09-12T02:13:42.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.245+0000 7f6cb97fa640 1 -- 172.21.15.16:0/84407445 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f6cc0024070 con 0x5592c9940d40 2024-09-12T02:13:42.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.245+0000 7f6cda186740 10 client.19464 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:13:42.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.245+0000 7f6cda186740 10 client.19464 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:13:42.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.245+0000 7f6cda186740 20 client.19464 populate_metadata read hostname 'smithi016' 2024-09-12T02:13:42.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.245+0000 7f6cda186740 1 --2- 172.21.15.16:0/84407445 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5592c99509c0 0x5592c9952db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:42.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.245+0000 7f6cda186740 4 client.19464 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:13:42.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.245+0000 7f6cda186740 1 -- 172.21.15.16:0/84407445 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5592c98a87f0 con 0x5592c99509c0 2024-09-12T02:13:42.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.245+0000 7f6cd5b49640 1 --2- 172.21.15.16:0/84407445 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5592c99509c0 0x5592c9952db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:42.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.246+0000 7f6cd5b49640 1 --2- 172.21.15.16:0/84407445 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5592c99509c0 0x5592c9952db0 crc :-1 s=READY pgs=362 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a24578be74e0d48f server_cookie=e5a01b6fce55ba67 in_seq=0 out_seq=0 2024-09-12T02:13:42.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.246+0000 7f6cb97fa640 10 client.19464 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:13:42.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.247+0000 7f6cb97fa640 1 -- 172.21.15.16:0/84407445 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5592c98a87f0 con 0x5592c99509c0 2024-09-12T02:13:42.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.247+0000 7f6cb97fa640 10 client.19464 handle_command_reply: tid=0 2024-09-12T02:13:42.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.247+0000 7f6cda186740 1 client.19464 shutdown 2024-09-12T02:13:42.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.247+0000 7f6cda186740 1 -- 172.21.15.16:0/84407445 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592c9940d40 msgr2=0x5592c993e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:42.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.247+0000 7f6cda186740 1 --2- 172.21.15.16:0/84407445 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592c9940d40 0x5592c993e6e0 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7f6cc0016000 tx=0x7f6cc0008040 comp rx=0 tx=0).stop 2024-09-12T02:13:42.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.247+0000 7f6cda186740 1 -- 172.21.15.16:0/84407445 shutdown_connections 2024-09-12T02:13:42.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.247+0000 7f6cda186740 1 --2- 172.21.15.16:0/84407445 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5592c99509c0 0x5592c9952db0 crc :-1 s=READY pgs=362 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:42.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.247+0000 7f6cda186740 1 --2- 172.21.15.16:0/84407445 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592c9940d40 0x5592c993e6e0 unknown :-1 s=CLOSED pgs=1699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:42.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.247+0000 7f6cda186740 1 --2- 172.21.15.16:0/84407445 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5592c9936480 0x5592c993e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:42.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.247+0000 7f6cb97fa640 0 client.19464 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:13:42.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.247+0000 7f6cda186740 1 --2- 172.21.15.16:0/84407445 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5592c9935b60 0x5592c993dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:42.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.247+0000 7f6cda186740 1 -- 172.21.15.16:0/84407445 >> 172.21.15.16:0/84407445 conn(0x5592c99320f0 msgr2=0x5592c9931270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:42.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.247+0000 7f6cda186740 1 -- 172.21.15.16:0/84407445 shutdown_connections 2024-09-12T02:13:42.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.247+0000 7f6cda186740 1 -- 172.21.15.16:0/84407445 wait complete. 2024-09-12T02:13:42.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.247+0000 7f6cda186740 20 client.19464 trim_cache size 0 max 16384 2024-09-12T02:13:42.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.268+0000 7f6cda186740 1 Processor -- start 2024-09-12T02:13:42.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.268+0000 7f6cda186740 1 -- start start 2024-09-12T02:13:42.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.268+0000 7f6cda186740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592c9935b60 0x5592c9958660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:42.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.268+0000 7f6cda186740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5592c9936480 0x5592c9958ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:42.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.268+0000 7f6cda186740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5592c9940d40 0x5592c993aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:42.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.268+0000 7f6cda186740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6cd01b1f30 con 0x5592c9935b60 2024-09-12T02:13:42.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.268+0000 7f6cda186740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6cd01af800 con 0x5592c9936480 2024-09-12T02:13:42.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.268+0000 7f6cda186740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f6cd01ad670 con 0x5592c9940d40 2024-09-12T02:13:42.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.268+0000 7f6cd5348640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592c9935b60 0x5592c9958660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:42.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.268+0000 7f6cd5348640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592c9935b60 0x5592c9958660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53774/0 (socket says 172.21.15.16:53774) 2024-09-12T02:13:42.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.268+0000 7f6cd5348640 1 -- 172.21.15.16:0/351487042 learned_addr learned my addr 172.21.15.16:0/351487042 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:13:42.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.268+0000 7f6cd5b49640 1 --2- 172.21.15.16:0/351487042 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5592c9936480 0x5592c9958ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:42.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.269+0000 7f6cd5348640 1 -- 172.21.15.16:0/351487042 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5592c9940d40 msgr2=0x5592c993aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:42.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.269+0000 7f6cd5348640 1 --2- 172.21.15.16:0/351487042 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5592c9940d40 0x5592c993aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:42.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.269+0000 7f6cd5348640 1 -- 172.21.15.16:0/351487042 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5592c9936480 msgr2=0x5592c9958ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:42.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.269+0000 7f6cd5348640 1 --2- 172.21.15.16:0/351487042 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5592c9936480 0x5592c9958ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:42.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.269+0000 7f6cd5348640 1 -- 172.21.15.16:0/351487042 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592c99437f0 con 0x5592c9935b60 2024-09-12T02:13:42.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.269+0000 7f6cd5b49640 1 --2- 172.21.15.16:0/351487042 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5592c9936480 0x5592c9958ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:13:42.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.269+0000 7f6cd5348640 1 --2- 172.21.15.16:0/351487042 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592c9935b60 0x5592c9958660 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7f6cc0018900 tx=0x7f6cc0009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f44a6bfa32c832a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:42.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.269+0000 7f6cbb7fe640 1 -- 172.21.15.16:0/351487042 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cc001a110 con 0x5592c9935b60 2024-09-12T02:13:42.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.269+0000 7f6cbb7fe640 1 -- 172.21.15.16:0/351487042 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6cc001a2b0 con 0x5592c9935b60 2024-09-12T02:13:42.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.269+0000 7f6cbb7fe640 1 -- 172.21.15.16:0/351487042 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cc0026470 con 0x5592c9935b60 2024-09-12T02:13:42.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.270+0000 7f6cda186740 1 -- 172.21.15.16:0/351487042 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592c9935b60 msgr2=0x5592c9958660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:42.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.270+0000 7f6cda186740 1 --2- 172.21.15.16:0/351487042 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592c9935b60 0x5592c9958660 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7f6cc0018900 tx=0x7f6cc0009900 comp rx=0 tx=0).stop 2024-09-12T02:13:42.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.270+0000 7f6cda186740 1 -- 172.21.15.16:0/351487042 shutdown_connections 2024-09-12T02:13:42.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.270+0000 7f6cda186740 1 --2- 172.21.15.16:0/351487042 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5592c9940d40 0x5592c993aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:42.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.270+0000 7f6cda186740 1 --2- 172.21.15.16:0/351487042 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5592c9936480 0x5592c9958ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:42.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.270+0000 7f6cda186740 1 --2- 172.21.15.16:0/351487042 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592c9935b60 0x5592c9958660 unknown :-1 s=CLOSED pgs=1700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:42.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.270+0000 7f6cda186740 1 -- 172.21.15.16:0/351487042 >> 172.21.15.16:0/351487042 conn(0x5592c99320f0 msgr2=0x5592c993fbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:42.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.270+0000 7f6cda186740 1 -- 172.21.15.16:0/351487042 shutdown_connections 2024-09-12T02:13:42.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.270+0000 7f6cda186740 1 -- 172.21.15.16:0/351487042 wait complete. 2024-09-12T02:13:42.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.270+0000 7f6cda186740 1 -- start start 2024-09-12T02:13:42.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.270+0000 7f6cda186740 1 Processor -- start 2024-09-12T02:13:42.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.270+0000 7f6cda186740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5592c9935b60 0x5592c9948f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:42.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.271+0000 7f6cda186740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592c9936480 0x5592c9949490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:42.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.271+0000 7f6cda186740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5592c9940d40 0x5592c9945d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:42.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.271+0000 7f6cda186740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6cd01adcc0 con 0x5592c9936480 2024-09-12T02:13:42.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.271+0000 7f6cda186740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6cd01af130 con 0x5592c9940d40 2024-09-12T02:13:42.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.271+0000 7f6cda186740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f6cd01ae390 con 0x5592c9935b60 2024-09-12T02:13:42.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.271+0000 7f6cda186740 10 client.0 _ll_register_callbacks cb 0x7f6cc0015720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:13:42.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.271+0000 7f6cd5b49640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592c9936480 0x5592c9949490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:42.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.271+0000 7f6cd5b49640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592c9936480 0x5592c9949490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53776/0 (socket says 172.21.15.16:53776) 2024-09-12T02:13:42.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.271+0000 7f6cd5b49640 1 -- 172.21.15.16:0/84427607 learned_addr learned my addr 172.21.15.16:0/84427607 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:13:42.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.271+0000 7f6cd5348640 1 --2- 172.21.15.16:0/84427607 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5592c9940d40 0x5592c9945d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:42.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.271+0000 7f6cd5348640 1 --2- 172.21.15.16:0/84427607 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5592c9935b60 0x5592c9948f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:42.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.271+0000 7f6cd5b49640 1 -- 172.21.15.16:0/84427607 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5592c9935b60 msgr2=0x5592c9948f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:42.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.271+0000 7f6cd5b49640 1 --2- 172.21.15.16:0/84427607 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5592c9935b60 0x5592c9948f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:42.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.271+0000 7f6cd5b49640 1 -- 172.21.15.16:0/84427607 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5592c9940d40 msgr2=0x5592c9945d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:42.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.271+0000 7f6cd5b49640 1 --2- 172.21.15.16:0/84427607 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5592c9940d40 0x5592c9945d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:42.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.271+0000 7f6cd5b49640 1 -- 172.21.15.16:0/84427607 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592c9946560 con 0x5592c9936480 2024-09-12T02:13:42.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.271+0000 7f6cd5348640 1 --2- 172.21.15.16:0/84427607 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5592c9935b60 0x5592c9948f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:13:42.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.271+0000 7f6cd5b49640 1 --2- 172.21.15.16:0/84427607 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592c9936480 0x5592c9949490 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc004de0 tx=0x7f6ccc009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f2f6e97494ddfef server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:42.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.271+0000 7f6cb97fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:13:42.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.271+0000 7f6cb97fa640 1 -- 172.21.15.16:0/84427607 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ccc035040 con 0x5592c9936480 2024-09-12T02:13:42.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.271+0000 7f6cb97fa640 1 -- 172.21.15.16:0/84427607 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f6ccc004e80 con 0x5592c9936480 2024-09-12T02:13:42.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.271+0000 7f6cda186740 1 -- 172.21.15.16:0/84427607 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5592c98a2d00 con 0x5592c9936480 2024-09-12T02:13:42.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.272+0000 7f6cb97fa640 1 -- 172.21.15.16:0/84427607 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6ccc005080 con 0x5592c9936480 2024-09-12T02:13:42.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.272+0000 7f6cda186740 10 client.19470 fetch_fsmap learned FSMap version 25 2024-09-12T02:13:42.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.272+0000 7f6cda186740 1 -- 172.21.15.16:0/84427607 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5592c99bc220 con 0x5592c9936480 2024-09-12T02:13:42.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.272+0000 7f6cb97fa640 1 -- 172.21.15.16:0/84427607 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5592c98a2d00 con 0x5592c9936480 2024-09-12T02:13:42.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.272+0000 7f6cb97fa640 1 -- 172.21.15.16:0/84427607 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f6ccc031b50 con 0x5592c9936480 2024-09-12T02:13:42.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.272+0000 7f6cda186740 10 client.19470 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:13:42.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.272+0000 7f6cda186740 10 client.19470 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:13:42.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.272+0000 7f6cda186740 20 client.19470 populate_metadata read hostname 'smithi016' 2024-09-12T02:13:42.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.272+0000 7f6cda186740 1 --2- 172.21.15.16:0/84427607 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5592c99bc620 0x5592c99bea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:42.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.272+0000 7f6cda186740 4 client.19470 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-12T02:13:42.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.272+0000 7f6cda186740 1 -- 172.21.15.16:0/84427607 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5592c981f110 con 0x5592c99bc620 2024-09-12T02:13:42.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.272+0000 7f6cd5348640 1 --2- 172.21.15.16:0/84427607 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5592c99bc620 0x5592c99bea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:42.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.273+0000 7f6cd5348640 1 --2- 172.21.15.16:0/84427607 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5592c99bc620 0x5592c99bea10 crc :-1 s=READY pgs=363 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af254413b29e68fb server_cookie=654e77be6ff6c8ba in_seq=0 out_seq=0 2024-09-12T02:13:42.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.273+0000 7f6cb97fa640 10 client.19470 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:13:42.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.274+0000 7f6cb97fa640 1 -- 172.21.15.16:0/84427607 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5592c981f110 con 0x5592c99bc620 2024-09-12T02:13:42.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.274+0000 7f6cb97fa640 10 client.19470 handle_command_reply: tid=0 2024-09-12T02:13:42.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.274+0000 7f6cda186740 1 client.19470 shutdown 2024-09-12T02:13:42.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.274+0000 7f6cda186740 1 -- 172.21.15.16:0/84427607 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592c9936480 msgr2=0x5592c9949490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:42.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.274+0000 7f6cda186740 1 --2- 172.21.15.16:0/84427607 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592c9936480 0x5592c9949490 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc004de0 tx=0x7f6ccc009680 comp rx=0 tx=0).stop 2024-09-12T02:13:42.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.274+0000 7f6cda186740 1 -- 172.21.15.16:0/84427607 shutdown_connections 2024-09-12T02:13:42.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.274+0000 7f6cda186740 1 --2- 172.21.15.16:0/84427607 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5592c99bc620 0x5592c99bea10 crc :-1 s=READY pgs=363 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:42.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.274+0000 7f6cda186740 1 --2- 172.21.15.16:0/84427607 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5592c9940d40 0x5592c9945d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:42.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.274+0000 7f6cb97fa640 0 client.19470 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:13:42.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.274+0000 7f6cda186740 1 --2- 172.21.15.16:0/84427607 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592c9936480 0x5592c9949490 unknown :-1 s=CLOSED pgs=1701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:42.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.274+0000 7f6cda186740 1 --2- 172.21.15.16:0/84427607 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5592c9935b60 0x5592c9948f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:42.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.274+0000 7f6cda186740 1 -- 172.21.15.16:0/84427607 >> 172.21.15.16:0/84427607 conn(0x5592c99320f0 msgr2=0x5592c9930a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:42.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.274+0000 7f6cda186740 1 -- 172.21.15.16:0/84427607 shutdown_connections 2024-09-12T02:13:42.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.274+0000 7f6cda186740 1 -- 172.21.15.16:0/84427607 wait complete. 2024-09-12T02:13:42.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.274+0000 7f6cda186740 20 client.19470 trim_cache size 0 max 16384 2024-09-12T02:13:42.280 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T02:13:42.280 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-09-12T02:13:42.280 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "cf01b15e-a1cb-41c0-9b3d-b89a28b8f55f", 2024-09-12T02:13:42.280 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-09-12T02:13:42.280 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T02:13:42.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.275+0000 7f6cd7dad640 1 -- 172.21.15.16:0/564863020 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f6cbc0a2b80 msgr2=0x7f6cbc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:42.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.275+0000 7f6cd7dad640 1 --2- 172.21.15.16:0/564863020 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f6cbc0a2b80 0x7f6cbc0a5040 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f6cc80983b0 tx=0x7f6cc801eb40 comp rx=0 tx=0).stop 2024-09-12T02:13:42.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.275+0000 7f6cd7dad640 1 -- 172.21.15.16:0/564863020 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6cd010e500 msgr2=0x7f6cd019a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:42.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.275+0000 7f6cd7dad640 1 --2- 172.21.15.16:0/564863020 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6cd010e500 0x7f6cd019a620 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7f6cc8005fd0 tx=0x7f6cc800edd0 comp rx=0 tx=0).stop 2024-09-12T02:13:42.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.275+0000 7f6cd7dad640 1 -- 172.21.15.16:0/564863020 shutdown_connections 2024-09-12T02:13:42.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.275+0000 7f6cd7dad640 1 --2- 172.21.15.16:0/564863020 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f6cbc0a2b80 0x7f6cbc0a5040 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:42.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.275+0000 7f6cd7dad640 1 --2- 172.21.15.16:0/564863020 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6cd010e500 0x7f6cd019a620 unknown :-1 s=CLOSED pgs=1698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:42.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.275+0000 7f6cd7dad640 1 --2- 172.21.15.16:0/564863020 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6cd0101440 0x7f6cd01a09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:42.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.275+0000 7f6cd7dad640 1 --2- 172.21.15.16:0/564863020 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6cd0100b20 0x7f6cd01a0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:42.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.275+0000 7f6cd7dad640 1 -- 172.21.15.16:0/564863020 >> 172.21.15.16:0/564863020 conn(0x7f6cd00fc7d0 msgr2=0x7f6cd00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:42.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.275+0000 7f6cd7dad640 1 -- 172.21.15.16:0/564863020 shutdown_connections 2024-09-12T02:13:42.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:42.275+0000 7f6cd7dad640 1 -- 172.21.15.16:0/564863020 wait complete. 2024-09-12T02:13:42.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:13:42 smithi184 ceph-mon[35593]: pgmap v688: 97 pgs: 97 active+clean; 2.2 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-09-12T02:13:42.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:13:42 smithi033 ceph-mon[35332]: pgmap v688: 97 pgs: 97 active+clean; 2.2 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-09-12T02:13:43.031 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T02:13:43.293 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:13:43.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:13:43 smithi184 ceph-mon[35593]: scrub queued for path: / 2024-09-12T02:13:43.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:13:43 smithi184 ceph-mon[35593]: scrub summary: idle+waiting paths [/] 2024-09-12T02:13:43.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:13:43 smithi184 ceph-mon[35593]: scrub summary: active paths [/] 2024-09-12T02:13:43.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:13:43 smithi184 ceph-mon[35593]: scrub summary: idle+waiting paths [/] 2024-09-12T02:13:43.414 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:13:43 smithi016 ceph-mon[30947]: scrub queued for path: / 2024-09-12T02:13:43.414 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:13:43 smithi016 ceph-mon[30947]: scrub summary: idle+waiting paths [/] 2024-09-12T02:13:43.414 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:13:43 smithi016 ceph-mon[30947]: scrub summary: active paths [/] 2024-09-12T02:13:43.414 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:13:43 smithi016 ceph-mon[30947]: scrub summary: idle+waiting paths [/] 2024-09-12T02:13:43.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:13:43 smithi033 ceph-mon[35332]: scrub queued for path: / 2024-09-12T02:13:43.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:13:43 smithi033 ceph-mon[35332]: scrub summary: idle+waiting paths [/] 2024-09-12T02:13:43.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:13:43 smithi033 ceph-mon[35332]: scrub summary: active paths [/] 2024-09-12T02:13:43.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:13:43 smithi033 ceph-mon[35332]: scrub summary: idle+waiting paths [/] 2024-09-12T02:13:43.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.820+0000 7f86d88db640 1 -- 172.21.15.16:0/2997261380 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86d41027c0 msgr2=0x7f86d4106810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:43.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.820+0000 7f86d88db640 1 --2- 172.21.15.16:0/2997261380 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86d41027c0 0x7f86d4106810 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7f86bc009900 tx=0x7f86bc02a410 comp rx=0 tx=0).stop 2024-09-12T02:13:43.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.820+0000 7f86d88db640 1 -- 172.21.15.16:0/2997261380 shutdown_connections 2024-09-12T02:13:43.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.820+0000 7f86d88db640 1 --2- 172.21.15.16:0/2997261380 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f86d4106f40 0x7f86d410eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:43.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.820+0000 7f86d88db640 1 --2- 172.21.15.16:0/2997261380 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86d41027c0 0x7f86d4106810 unknown :-1 s=CLOSED pgs=1702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:43.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.820+0000 7f86d88db640 1 --2- 172.21.15.16:0/2997261380 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f86d4101e10 0x7f86d41021f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:43.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.820+0000 7f86d88db640 1 -- 172.21.15.16:0/2997261380 >> 172.21.15.16:0/2997261380 conn(0x7f86d40fd8d0 msgr2=0x7f86d40ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:43.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.820+0000 7f86d88db640 1 -- 172.21.15.16:0/2997261380 shutdown_connections 2024-09-12T02:13:43.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.820+0000 7f86d88db640 1 -- 172.21.15.16:0/2997261380 wait complete. 2024-09-12T02:13:43.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.820+0000 7f86d88db640 1 Processor -- start 2024-09-12T02:13:43.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.821+0000 7f86d88db640 1 -- start start 2024-09-12T02:13:43.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.821+0000 7f86d88db640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f86d4101e10 0x7f86d4198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:43.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.821+0000 7f86d88db640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86d41027c0 0x7f86d4198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:43.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.821+0000 7f86d88db640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f86d4106f40 0x7f86d419ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:43.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.821+0000 7f86d88db640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f86d4111fa0 con 0x7f86d41027c0 2024-09-12T02:13:43.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.821+0000 7f86d88db640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f86d4111e20 con 0x7f86d4106f40 2024-09-12T02:13:43.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.821+0000 7f86d88db640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f86d4112120 con 0x7f86d4101e10 2024-09-12T02:13:43.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.821+0000 7f86c9d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86d41027c0 0x7f86d4198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:43.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.821+0000 7f86c9d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86d41027c0 0x7f86d4198940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53806/0 (socket says 172.21.15.16:53806) 2024-09-12T02:13:43.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.821+0000 7f86c9d9b640 1 -- 172.21.15.16:0/1535898845 learned_addr learned my addr 172.21.15.16:0/1535898845 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:13:43.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.821+0000 7f86d259c640 1 --2- 172.21.15.16:0/1535898845 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f86d4101e10 0x7f86d4198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:43.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.821+0000 7f86d2d9d640 1 --2- 172.21.15.16:0/1535898845 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f86d4106f40 0x7f86d419ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:43.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.822+0000 7f86c9d9b640 1 -- 172.21.15.16:0/1535898845 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f86d4101e10 msgr2=0x7f86d4198400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:43.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.822+0000 7f86c9d9b640 1 --2- 172.21.15.16:0/1535898845 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f86d4101e10 0x7f86d4198400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:43.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.822+0000 7f86c9d9b640 1 -- 172.21.15.16:0/1535898845 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f86d4106f40 msgr2=0x7f86d419ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:43.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.822+0000 7f86c9d9b640 1 --2- 172.21.15.16:0/1535898845 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f86d4106f40 0x7f86d419ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:43.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.822+0000 7f86c9d9b640 1 -- 172.21.15.16:0/1535898845 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86d419d3b0 con 0x7f86d41027c0 2024-09-12T02:13:43.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.822+0000 7f86d2d9d640 1 --2- 172.21.15.16:0/1535898845 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f86d4106f40 0x7f86d419ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:13:43.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.822+0000 7f86c9d9b640 1 --2- 172.21.15.16:0/1535898845 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86d41027c0 0x7f86d4198940 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f86bc006010 tx=0x7f86bc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:43.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.822+0000 7f86cbfff640 1 -- 172.21.15.16:0/1535898845 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86bc03f030 con 0x7f86d41027c0 2024-09-12T02:13:43.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.822+0000 7f86cbfff640 1 -- 172.21.15.16:0/1535898845 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86bc003ce0 con 0x7f86d41027c0 2024-09-12T02:13:43.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.822+0000 7f86d88db640 1 -- 172.21.15.16:0/1535898845 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86d419d640 con 0x7f86d41027c0 2024-09-12T02:13:43.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.822+0000 7f86cbfff640 1 -- 172.21.15.16:0/1535898845 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86bc035070 con 0x7f86d41027c0 2024-09-12T02:13:43.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.822+0000 7f86d88db640 1 -- 172.21.15.16:0/1535898845 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86d41a4f80 con 0x7f86d41027c0 2024-09-12T02:13:43.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.825+0000 7f86dacb4740 1 Processor -- start 2024-09-12T02:13:43.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.825+0000 7f86dacb4740 1 -- start start 2024-09-12T02:13:43.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.825+0000 7f86cbfff640 1 -- 172.21.15.16:0/1535898845 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f86bc042030 con 0x7f86d41027c0 2024-09-12T02:13:43.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.826+0000 7f86dacb4740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606e02844a0 0x5606e0284880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:43.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.826+0000 7f86dacb4740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5606e0284dc0 0x5606e028f140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:43.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.826+0000 7f86dacb4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5606e028f680 0x5606e0291a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:43.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.826+0000 7f86dacb4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5606e01ea2e0 con 0x5606e028f680 2024-09-12T02:13:43.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.826+0000 7f86dacb4740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5606e0191ce0 con 0x5606e02844a0 2024-09-12T02:13:43.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.826+0000 7f86dacb4740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5606e01e62a0 con 0x5606e0284dc0 2024-09-12T02:13:43.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.826+0000 7f86d2d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606e02844a0 0x5606e0284880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:43.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.826+0000 7f86c9d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5606e028f680 0x5606e0291a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:43.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.826+0000 7f86d2d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606e02844a0 0x5606e0284880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.16:47758/0 (socket says 172.21.15.16:47758) 2024-09-12T02:13:43.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.826+0000 7f86d2d9d640 1 -- 172.21.15.16:0/2422177663 learned_addr learned my addr 172.21.15.16:0/2422177663 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:13:43.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.826+0000 7f86d259c640 1 --2- 172.21.15.16:0/2422177663 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5606e0284dc0 0x5606e028f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:43.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.826+0000 7f86d2d9d640 1 -- 172.21.15.16:0/2422177663 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5606e0284dc0 msgr2=0x5606e028f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:43.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.826+0000 7f86d2d9d640 1 --2- 172.21.15.16:0/2422177663 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5606e0284dc0 0x5606e028f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:43.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.826+0000 7f86d2d9d640 1 -- 172.21.15.16:0/2422177663 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5606e028f680 msgr2=0x5606e0291a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:43.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.826+0000 7f86d2d9d640 1 --2- 172.21.15.16:0/2422177663 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5606e028f680 0x5606e0291a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:43.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.826+0000 7f86d2d9d640 1 -- 172.21.15.16:0/2422177663 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5606e0176440 con 0x5606e02844a0 2024-09-12T02:13:43.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.826+0000 7f86cbfff640 1 --2- 172.21.15.16:0/1535898845 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f86c00a2c00 0x7f86c00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:43.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.826+0000 7f86d2d9d640 1 --2- 172.21.15.16:0/2422177663 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606e02844a0 0x5606e0284880 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f86cc00f790 tx=0x7f86cc00fc50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7fdc6b0fc081ee06 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:43.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.827+0000 7f86d259c640 1 --2- 172.21.15.16:0/1535898845 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f86c00a2c00 0x7f86c00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:43.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.827+0000 7f86cbfff640 1 -- 172.21.15.16:0/1535898845 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f86bc0eef90 con 0x7f86d41027c0 2024-09-12T02:13:43.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.827+0000 7f86c8d99640 1 -- 172.21.15.16:0/2422177663 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86cc004d20 con 0x5606e02844a0 2024-09-12T02:13:43.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.827+0000 7f86c8d99640 1 -- 172.21.15.16:0/2422177663 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86cc00de00 con 0x5606e02844a0 2024-09-12T02:13:43.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.827+0000 7f86c8d99640 1 -- 172.21.15.16:0/2422177663 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86cc010430 con 0x5606e02844a0 2024-09-12T02:13:43.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.827+0000 7f86dacb4740 1 -- 172.21.15.16:0/2422177663 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606e02844a0 msgr2=0x5606e0284880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:43.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.827+0000 7f86dacb4740 1 --2- 172.21.15.16:0/2422177663 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606e02844a0 0x5606e0284880 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f86cc00f790 tx=0x7f86cc00fc50 comp rx=0 tx=0).stop 2024-09-12T02:13:43.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.827+0000 7f86d259c640 1 --2- 172.21.15.16:0/1535898845 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f86c00a2c00 0x7f86c00a50c0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x5606e0281890 tx=0x7f86c4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:43.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.827+0000 7f86dacb4740 1 -- 172.21.15.16:0/2422177663 shutdown_connections 2024-09-12T02:13:43.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.827+0000 7f86dacb4740 1 --2- 172.21.15.16:0/2422177663 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5606e028f680 0x5606e0291a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:43.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.828+0000 7f86dacb4740 1 --2- 172.21.15.16:0/2422177663 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5606e0284dc0 0x5606e028f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:43.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.828+0000 7f86dacb4740 1 --2- 172.21.15.16:0/2422177663 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606e02844a0 0x5606e0284880 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:43.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.828+0000 7f86dacb4740 1 -- 172.21.15.16:0/2422177663 >> 172.21.15.16:0/2422177663 conn(0x5606e0280a30 msgr2=0x5606e0280e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:43.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.828+0000 7f86dacb4740 1 -- 172.21.15.16:0/2422177663 shutdown_connections 2024-09-12T02:13:43.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.828+0000 7f86dacb4740 1 -- 172.21.15.16:0/2422177663 wait complete. 2024-09-12T02:13:43.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.828+0000 7f86dacb4740 1 -- start start 2024-09-12T02:13:43.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.828+0000 7f86dacb4740 1 Processor -- start 2024-09-12T02:13:43.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.828+0000 7f86dacb4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5606e02844a0 0x5606e028efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:43.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.828+0000 7f86dacb4740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606e0284dc0 0x5606e028c250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:43.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.828+0000 7f86dacb4740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5606e028f680 0x5606e028c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:43.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.828+0000 7f86dacb4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5606e0292250 con 0x5606e02844a0 2024-09-12T02:13:43.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.828+0000 7f86dacb4740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5606e01a5050 con 0x5606e0284dc0 2024-09-12T02:13:43.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.828+0000 7f86dacb4740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5606e02923d0 con 0x5606e028f680 2024-09-12T02:13:43.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.828+0000 7f86d259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5606e02844a0 0x5606e028efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:43.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.828+0000 7f86dacb4740 10 client.0 _ll_register_callbacks cb 0x5606e01e5160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:13:43.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.829+0000 7f86d259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5606e02844a0 0x5606e028efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53824/0 (socket says 172.21.15.16:53824) 2024-09-12T02:13:43.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.829+0000 7f86d259c640 1 -- 172.21.15.16:0/4098847049 learned_addr learned my addr 172.21.15.16:0/4098847049 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:13:43.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.829+0000 7f86c9d9b640 1 --2- 172.21.15.16:0/4098847049 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606e0284dc0 0x5606e028c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:43.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.829+0000 7f86d2d9d640 1 --2- 172.21.15.16:0/4098847049 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5606e028f680 0x5606e028c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:43.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.829+0000 7f86d259c640 1 -- 172.21.15.16:0/4098847049 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5606e028f680 msgr2=0x5606e028c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:43.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.829+0000 7f86d259c640 1 --2- 172.21.15.16:0/4098847049 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5606e028f680 0x5606e028c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:43.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.829+0000 7f86d259c640 1 -- 172.21.15.16:0/4098847049 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606e0284dc0 msgr2=0x5606e028c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:43.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.829+0000 7f86d259c640 1 --2- 172.21.15.16:0/4098847049 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606e0284dc0 0x5606e028c250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:43.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.829+0000 7f86d259c640 1 -- 172.21.15.16:0/4098847049 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5606dff5b3c0 con 0x5606e02844a0 2024-09-12T02:13:43.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.829+0000 7f86d2d9d640 1 --2- 172.21.15.16:0/4098847049 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5606e028f680 0x5606e028c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:13:43.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.829+0000 7f86c9d9b640 1 --2- 172.21.15.16:0/4098847049 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606e0284dc0 0x5606e028c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:13:43.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.829+0000 7f86d259c640 1 --2- 172.21.15.16:0/4098847049 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5606e02844a0 0x5606e028efa0 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7f86c4009730 tx=0x7f86c4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8f507a5d9c0f2e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:43.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.829+0000 7f86b9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:13:43.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.829+0000 7f86b9ffb640 1 -- 172.21.15.16:0/4098847049 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86c401f640 con 0x5606e02844a0 2024-09-12T02:13:43.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.829+0000 7f86b9ffb640 1 -- 172.21.15.16:0/4098847049 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f86c401f7e0 con 0x5606e02844a0 2024-09-12T02:13:43.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.829+0000 7f86dacb4740 1 -- 172.21.15.16:0/4098847049 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86cc00de00 con 0x5606e02844a0 2024-09-12T02:13:43.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.829+0000 7f86b9ffb640 1 -- 172.21.15.16:0/4098847049 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f86c401f9e0 con 0x5606e02844a0 2024-09-12T02:13:43.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.829+0000 7f86dacb4740 10 client.19482 fetch_fsmap learned FSMap version 25 2024-09-12T02:13:43.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.829+0000 7f86dacb4740 1 -- 172.21.15.16:0/4098847049 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5606e029dc30 con 0x5606e02844a0 2024-09-12T02:13:43.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.830+0000 7f86b9ffb640 1 -- 172.21.15.16:0/4098847049 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86cc00de00 con 0x5606e02844a0 2024-09-12T02:13:43.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.830+0000 7f86b9ffb640 1 -- 172.21.15.16:0/4098847049 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f86c4027070 con 0x5606e02844a0 2024-09-12T02:13:43.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.830+0000 7f86dacb4740 10 client.19482 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:13:43.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.830+0000 7f86dacb4740 10 client.19482 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:13:43.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.830+0000 7f86dacb4740 20 client.19482 populate_metadata read hostname 'smithi016' 2024-09-12T02:13:43.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.830+0000 7f86dacb4740 1 --2- 172.21.15.16:0/4098847049 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5606e029f040 0x5606e02a14a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:43.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.830+0000 7f86dacb4740 4 client.19482 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:13:43.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.830+0000 7f86dacb4740 1 -- 172.21.15.16:0/4098847049 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5606e01f7850 con 0x5606e029f040 2024-09-12T02:13:43.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.830+0000 7f86c9d9b640 1 --2- 172.21.15.16:0/4098847049 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5606e029f040 0x5606e02a14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:43.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.831+0000 7f86c9d9b640 1 --2- 172.21.15.16:0/4098847049 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5606e029f040 0x5606e02a14a0 crc :-1 s=READY pgs=364 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=387d57bef822d774 server_cookie=9f7c6761c2df7777 in_seq=0 out_seq=0 2024-09-12T02:13:43.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.831+0000 7f86b9ffb640 10 client.19482 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:13:43.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.831+0000 7f86b9ffb640 1 -- 172.21.15.16:0/4098847049 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5606e01f7850 con 0x5606e029f040 2024-09-12T02:13:43.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.831+0000 7f86b9ffb640 10 client.19482 handle_command_reply: tid=0 2024-09-12T02:13:43.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.831+0000 7f86dacb4740 1 client.19482 shutdown 2024-09-12T02:13:43.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.831+0000 7f86dacb4740 1 -- 172.21.15.16:0/4098847049 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5606e02844a0 msgr2=0x5606e028efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:43.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.831+0000 7f86dacb4740 1 --2- 172.21.15.16:0/4098847049 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5606e02844a0 0x5606e028efa0 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7f86c4009730 tx=0x7f86c4004730 comp rx=0 tx=0).stop 2024-09-12T02:13:43.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.831+0000 7f86dacb4740 1 -- 172.21.15.16:0/4098847049 shutdown_connections 2024-09-12T02:13:43.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.831+0000 7f86dacb4740 1 --2- 172.21.15.16:0/4098847049 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5606e029f040 0x5606e02a14a0 crc :-1 s=READY pgs=364 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:43.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.831+0000 7f86b9ffb640 0 client.19482 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:13:43.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.831+0000 7f86dacb4740 1 --2- 172.21.15.16:0/4098847049 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5606e028f680 0x5606e028c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:43.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.831+0000 7f86dacb4740 1 --2- 172.21.15.16:0/4098847049 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606e0284dc0 0x5606e028c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:43.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.831+0000 7f86dacb4740 1 --2- 172.21.15.16:0/4098847049 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5606e02844a0 0x5606e028efa0 unknown :-1 s=CLOSED pgs=1704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:43.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.832+0000 7f86dacb4740 1 -- 172.21.15.16:0/4098847049 >> 172.21.15.16:0/4098847049 conn(0x5606e0280a30 msgr2=0x5606e0286710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:43.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.832+0000 7f86dacb4740 1 -- 172.21.15.16:0/4098847049 shutdown_connections 2024-09-12T02:13:43.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.832+0000 7f86dacb4740 1 -- 172.21.15.16:0/4098847049 wait complete. 2024-09-12T02:13:43.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.832+0000 7f86dacb4740 20 client.19482 trim_cache size 0 max 16384 2024-09-12T02:13:43.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.849+0000 7f86dacb4740 1 Processor -- start 2024-09-12T02:13:43.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.849+0000 7f86dacb4740 1 -- start start 2024-09-12T02:13:43.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.849+0000 7f86dacb4740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606e0280a30 0x5606e027fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:43.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.849+0000 7f86dacb4740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5606e02844a0 0x5606e0297a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:43.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.849+0000 7f86dacb4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5606e0284dc0 0x5606e0297f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:43.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.849+0000 7f86dacb4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f86d41afcd0 con 0x5606e0284dc0 2024-09-12T02:13:43.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.849+0000 7f86dacb4740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f86d41ad5a0 con 0x5606e0280a30 2024-09-12T02:13:43.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.849+0000 7f86dacb4740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f86d41ab410 con 0x5606e02844a0 2024-09-12T02:13:43.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.849+0000 7f86d2d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5606e0284dc0 0x5606e0297f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:43.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.849+0000 7f86d2d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5606e0284dc0 0x5606e0297f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53834/0 (socket says 172.21.15.16:53834) 2024-09-12T02:13:43.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.849+0000 7f86d2d9d640 1 -- 172.21.15.16:0/2239747218 learned_addr learned my addr 172.21.15.16:0/2239747218 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:13:43.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.849+0000 7f86d259c640 1 --2- 172.21.15.16:0/2239747218 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606e0280a30 0x5606e027fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:43.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.849+0000 7f86c9d9b640 1 --2- 172.21.15.16:0/2239747218 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5606e02844a0 0x5606e0297a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:43.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.849+0000 7f86d2d9d640 1 -- 172.21.15.16:0/2239747218 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5606e02844a0 msgr2=0x5606e0297a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:43.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.849+0000 7f86d2d9d640 1 --2- 172.21.15.16:0/2239747218 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5606e02844a0 0x5606e0297a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:43.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.849+0000 7f86d2d9d640 1 -- 172.21.15.16:0/2239747218 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606e0280a30 msgr2=0x5606e027fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:43.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.849+0000 7f86d2d9d640 1 --2- 172.21.15.16:0/2239747218 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606e0280a30 0x5606e027fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:43.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.849+0000 7f86d2d9d640 1 -- 172.21.15.16:0/2239747218 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5606e028a7a0 con 0x5606e0284dc0 2024-09-12T02:13:43.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.849+0000 7f86c9d9b640 1 --2- 172.21.15.16:0/2239747218 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5606e02844a0 0x5606e0297a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:13:43.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.849+0000 7f86d259c640 1 --2- 172.21.15.16:0/2239747218 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606e0280a30 0x5606e027fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:13:43.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.850+0000 7f86d2d9d640 1 --2- 172.21.15.16:0/2239747218 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5606e0284dc0 0x5606e0297f40 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7f86cc00f6b0 tx=0x7f86cc007cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbcf77b8f4401ae8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:43.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.850+0000 7f86bbfff640 1 -- 172.21.15.16:0/2239747218 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86cc027070 con 0x5606e0284dc0 2024-09-12T02:13:43.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.850+0000 7f86bbfff640 1 -- 172.21.15.16:0/2239747218 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86cc00dda0 con 0x5606e0284dc0 2024-09-12T02:13:43.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.850+0000 7f86bbfff640 1 -- 172.21.15.16:0/2239747218 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86cc01d040 con 0x5606e0284dc0 2024-09-12T02:13:43.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.850+0000 7f86dacb4740 1 -- 172.21.15.16:0/2239747218 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5606e0284dc0 msgr2=0x5606e0297f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:43.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.850+0000 7f86dacb4740 1 --2- 172.21.15.16:0/2239747218 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5606e0284dc0 0x5606e0297f40 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7f86cc00f6b0 tx=0x7f86cc007cb0 comp rx=0 tx=0).stop 2024-09-12T02:13:43.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.850+0000 7f86dacb4740 1 -- 172.21.15.16:0/2239747218 shutdown_connections 2024-09-12T02:13:43.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.850+0000 7f86dacb4740 1 --2- 172.21.15.16:0/2239747218 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5606e0284dc0 0x5606e0297f40 unknown :-1 s=CLOSED pgs=1705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:43.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.850+0000 7f86dacb4740 1 --2- 172.21.15.16:0/2239747218 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5606e02844a0 0x5606e0297a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:43.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.850+0000 7f86dacb4740 1 --2- 172.21.15.16:0/2239747218 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606e0280a30 0x5606e027fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:43.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.850+0000 7f86dacb4740 1 -- 172.21.15.16:0/2239747218 >> 172.21.15.16:0/2239747218 conn(0x5606e028c8f0 msgr2=0x5606e027f370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:43.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.851+0000 7f86dacb4740 1 -- 172.21.15.16:0/2239747218 shutdown_connections 2024-09-12T02:13:43.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.851+0000 7f86dacb4740 1 -- 172.21.15.16:0/2239747218 wait complete. 2024-09-12T02:13:43.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.851+0000 7f86dacb4740 1 -- start start 2024-09-12T02:13:43.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.851+0000 7f86dacb4740 1 Processor -- start 2024-09-12T02:13:43.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.851+0000 7f86dacb4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5606e0280a30 0x5606e0300bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:43.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.851+0000 7f86dacb4740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606e02844a0 0x5606e0301110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:43.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.851+0000 7f86dacb4740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5606e0284dc0 0x5606e03051e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:43.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.851+0000 7f86dacb4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f86d41aba60 con 0x5606e0280a30 2024-09-12T02:13:43.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.851+0000 7f86dacb4740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f86d41aced0 con 0x5606e02844a0 2024-09-12T02:13:43.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.851+0000 7f86dacb4740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f86d41ac130 con 0x5606e0284dc0 2024-09-12T02:13:43.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.851+0000 7f86dacb4740 10 client.0 _ll_register_callbacks cb 0x7f86c4009ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:13:43.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.851+0000 7f86d259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5606e0280a30 0x5606e0300bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:43.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.851+0000 7f86d259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5606e0280a30 0x5606e0300bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53836/0 (socket says 172.21.15.16:53836) 2024-09-12T02:13:43.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.851+0000 7f86d259c640 1 -- 172.21.15.16:0/4253436646 learned_addr learned my addr 172.21.15.16:0/4253436646 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:13:43.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.851+0000 7f86c9d9b640 1 --2- 172.21.15.16:0/4253436646 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606e02844a0 0x5606e0301110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:43.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.852+0000 7f86d259c640 1 -- 172.21.15.16:0/4253436646 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5606e0284dc0 msgr2=0x5606e03051e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:43.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.852+0000 7f86d2d9d640 1 --2- 172.21.15.16:0/4253436646 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5606e0284dc0 0x5606e03051e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:43.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.852+0000 7f86d259c640 1 --2- 172.21.15.16:0/4253436646 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5606e0284dc0 0x5606e03051e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:43.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.852+0000 7f86d259c640 1 -- 172.21.15.16:0/4253436646 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606e02844a0 msgr2=0x5606e0301110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:43.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.852+0000 7f86d259c640 1 --2- 172.21.15.16:0/4253436646 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606e02844a0 0x5606e0301110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:43.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.852+0000 7f86d259c640 1 -- 172.21.15.16:0/4253436646 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5606e0305880 con 0x5606e0280a30 2024-09-12T02:13:43.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.852+0000 7f86c9d9b640 1 --2- 172.21.15.16:0/4253436646 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606e02844a0 0x5606e0301110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:13:43.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.852+0000 7f86d259c640 1 --2- 172.21.15.16:0/4253436646 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5606e0280a30 0x5606e0300bd0 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7f86c40092e0 tx=0x7f86c4033630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=567042196e478722 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:43.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.852+0000 7f86b9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:13:43.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.852+0000 7f86b9ffb640 1 -- 172.21.15.16:0/4253436646 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86c4037030 con 0x5606e0280a30 2024-09-12T02:13:43.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.852+0000 7f86b9ffb640 1 -- 172.21.15.16:0/4253436646 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f86c401fb80 con 0x5606e0280a30 2024-09-12T02:13:43.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.852+0000 7f86dacb4740 1 -- 172.21.15.16:0/4253436646 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86cc00dda0 con 0x5606e0280a30 2024-09-12T02:13:43.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.852+0000 7f86b9ffb640 1 -- 172.21.15.16:0/4253436646 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f86c4003e50 con 0x5606e0280a30 2024-09-12T02:13:43.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.852+0000 7f86dacb4740 10 client.19488 fetch_fsmap learned FSMap version 25 2024-09-12T02:13:43.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.852+0000 7f86dacb4740 1 -- 172.21.15.16:0/4253436646 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5606e030dd90 con 0x5606e0280a30 2024-09-12T02:13:43.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.852+0000 7f86b9ffb640 1 -- 172.21.15.16:0/4253436646 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86cc00dda0 con 0x5606e0280a30 2024-09-12T02:13:43.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.853+0000 7f86b9ffb640 1 -- 172.21.15.16:0/4253436646 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f86c4027040 con 0x5606e0280a30 2024-09-12T02:13:43.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.853+0000 7f86dacb4740 10 client.19488 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:13:43.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.853+0000 7f86dacb4740 10 client.19488 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:13:43.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.853+0000 7f86dacb4740 20 client.19488 populate_metadata read hostname 'smithi016' 2024-09-12T02:13:43.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.853+0000 7f86dacb4740 1 --2- 172.21.15.16:0/4253436646 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5606e028f680 0x5606e0310550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:43.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.853+0000 7f86dacb4740 4 client.19488 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T02:13:43.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.853+0000 7f86dacb4740 1 -- 172.21.15.16:0/4253436646 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5606e01f1d60 con 0x5606e028f680 2024-09-12T02:13:43.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.853+0000 7f86c9d9b640 1 --2- 172.21.15.16:0/4253436646 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5606e028f680 0x5606e0310550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:43.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.853+0000 7f86c9d9b640 1 --2- 172.21.15.16:0/4253436646 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5606e028f680 0x5606e0310550 crc :-1 s=READY pgs=365 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=333957152623a2e9 server_cookie=f7fc7fc8646459f7 in_seq=0 out_seq=0 2024-09-12T02:13:43.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.853+0000 7f86b9ffb640 10 client.19488 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:13:43.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.853+0000 7f86b9ffb640 1 -- 172.21.15.16:0/4253436646 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5606e01f1d60 con 0x5606e028f680 2024-09-12T02:13:43.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.853+0000 7f86b9ffb640 10 client.19488 handle_command_reply: tid=0 2024-09-12T02:13:43.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.854+0000 7f86dacb4740 1 client.19488 shutdown 2024-09-12T02:13:43.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.854+0000 7f86dacb4740 1 -- 172.21.15.16:0/4253436646 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5606e0280a30 msgr2=0x5606e0300bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:43.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.854+0000 7f86dacb4740 1 --2- 172.21.15.16:0/4253436646 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5606e0280a30 0x5606e0300bd0 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7f86c40092e0 tx=0x7f86c4033630 comp rx=0 tx=0).stop 2024-09-12T02:13:43.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.854+0000 7f86dacb4740 1 -- 172.21.15.16:0/4253436646 shutdown_connections 2024-09-12T02:13:43.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.854+0000 7f86dacb4740 1 --2- 172.21.15.16:0/4253436646 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5606e028f680 0x5606e0310550 crc :-1 s=READY pgs=365 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:43.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.854+0000 7f86dacb4740 1 --2- 172.21.15.16:0/4253436646 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5606e0284dc0 0x5606e03051e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:43.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.854+0000 7f86b9ffb640 0 client.19488 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:13:43.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.854+0000 7f86dacb4740 1 --2- 172.21.15.16:0/4253436646 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606e02844a0 0x5606e0301110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:43.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.854+0000 7f86dacb4740 1 --2- 172.21.15.16:0/4253436646 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5606e0280a30 0x5606e0300bd0 unknown :-1 s=CLOSED pgs=1706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:43.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.854+0000 7f86dacb4740 1 -- 172.21.15.16:0/4253436646 >> 172.21.15.16:0/4253436646 conn(0x5606e028c8f0 msgr2=0x5606e027fb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:43.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.854+0000 7f86dacb4740 1 -- 172.21.15.16:0/4253436646 shutdown_connections 2024-09-12T02:13:43.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.854+0000 7f86dacb4740 1 -- 172.21.15.16:0/4253436646 wait complete. 2024-09-12T02:13:43.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.854+0000 7f86dacb4740 20 client.19488 trim_cache size 0 max 16384 2024-09-12T02:13:43.861 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T02:13:43.861 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-12T02:13:43.861 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": { 2024-09-12T02:13:43.861 INFO:teuthology.orchestra.run.smithi016.stdout: "cf01b15e-a1cb-41c0-9b3d-b89a28b8f55f": { 2024-09-12T02:13:43.861 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-12T02:13:43.861 INFO:teuthology.orchestra.run.smithi016.stdout: "tag": "cf01b15e-a1cb-41c0-9b3d-b89a28b8f55f", 2024-09-12T02:13:43.862 INFO:teuthology.orchestra.run.smithi016.stdout: "options": "recursive,force" 2024-09-12T02:13:43.862 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:13:43.862 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:13:43.862 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T02:13:43.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.854+0000 7f86d88db640 1 -- 172.21.15.16:0/1535898845 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f86c00a2c00 msgr2=0x7f86c00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:43.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.854+0000 7f86d88db640 1 --2- 172.21.15.16:0/1535898845 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f86c00a2c00 0x7f86c00a50c0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x5606e0281890 tx=0x7f86c4005f90 comp rx=0 tx=0).stop 2024-09-12T02:13:43.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.854+0000 7f86d88db640 1 -- 172.21.15.16:0/1535898845 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86d41027c0 msgr2=0x7f86d4198940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:43.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.854+0000 7f86d88db640 1 --2- 172.21.15.16:0/1535898845 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86d41027c0 0x7f86d4198940 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f86bc006010 tx=0x7f86bc0042f0 comp rx=0 tx=0).stop 2024-09-12T02:13:43.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.854+0000 7f86d88db640 1 -- 172.21.15.16:0/1535898845 shutdown_connections 2024-09-12T02:13:43.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.854+0000 7f86d88db640 1 --2- 172.21.15.16:0/1535898845 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f86c00a2c00 0x7f86c00a50c0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:43.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.855+0000 7f86d88db640 1 --2- 172.21.15.16:0/1535898845 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f86d4106f40 0x7f86d419ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:43.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.855+0000 7f86d88db640 1 --2- 172.21.15.16:0/1535898845 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86d41027c0 0x7f86d4198940 unknown :-1 s=CLOSED pgs=1703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:43.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.855+0000 7f86d88db640 1 --2- 172.21.15.16:0/1535898845 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f86d4101e10 0x7f86d4198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:43.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.855+0000 7f86d88db640 1 -- 172.21.15.16:0/1535898845 >> 172.21.15.16:0/1535898845 conn(0x7f86d40fd8d0 msgr2=0x7f86d40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:43.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.855+0000 7f86d88db640 1 -- 172.21.15.16:0/1535898845 shutdown_connections 2024-09-12T02:13:43.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:13:43.855+0000 7f86d88db640 1 -- 172.21.15.16:0/1535898845 wait complete. 2024-09-12T02:13:44.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:13:44 smithi184 ceph-mon[35593]: pgmap v689: 97 pgs: 97 active+clean; 2.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-09-12T02:13:44.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:13:44 smithi033 ceph-mon[35332]: pgmap v689: 97 pgs: 97 active+clean; 2.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-09-12T02:13:44.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:13:44 smithi016 ceph-mon[30947]: pgmap v689: 97 pgs: 97 active+clean; 2.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-09-12T02:13:44.556 INFO:tasks.cephfs.filesystem:scrub status for tag:cf01b15e-a1cb-41c0-9b3d-b89a28b8f55f - {'path': '/', 'tag': 'cf01b15e-a1cb-41c0-9b3d-b89a28b8f55f', 'options': 'recursive,force'} 2024-09-12T02:13:46.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:13:46 smithi184 ceph-mon[35593]: pgmap v690: 97 pgs: 97 active+clean; 2.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-09-12T02:13:46.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:13:46 smithi033 ceph-mon[35332]: pgmap v690: 97 pgs: 97 active+clean; 2.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-09-12T02:13:46.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:13:46 smithi016 ceph-mon[30947]: pgmap v690: 97 pgs: 97 active+clean; 2.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-09-12T02:13:47.369 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.368+0000 7fbb3dcd0640 1 -- 172.21.15.16:0/1500513790 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb38102d50 msgr2=0x7fbb381031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:47.369 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.368+0000 7fbb3dcd0640 1 --2- 172.21.15.16:0/1500513790 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb38102d50 0x7fbb381031d0 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c009900 tx=0x7fbb2c02a410 comp rx=0 tx=0).stop 2024-09-12T02:13:47.369 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.369+0000 7fbb3dcd0640 1 -- 172.21.15.16:0/1500513790 shutdown_connections 2024-09-12T02:13:47.369 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.369+0000 7fbb3dcd0640 1 --2- 172.21.15.16:0/1500513790 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbb38103710 0x7fbb38109fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:47.369 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.369+0000 7fbb3dcd0640 1 --2- 172.21.15.16:0/1500513790 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb38102d50 0x7fbb381031d0 unknown :-1 s=CLOSED pgs=1707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:47.369 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.369+0000 7fbb3dcd0640 1 --2- 172.21.15.16:0/1500513790 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbb38101b60 0x7fbb38101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:47.370 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.369+0000 7fbb3dcd0640 1 -- 172.21.15.16:0/1500513790 >> 172.21.15.16:0/1500513790 conn(0x7fbb380fd2e0 msgr2=0x7fbb380ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:47.370 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.369+0000 7fbb3dcd0640 1 -- 172.21.15.16:0/1500513790 shutdown_connections 2024-09-12T02:13:47.370 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.369+0000 7fbb3dcd0640 1 -- 172.21.15.16:0/1500513790 wait complete. 2024-09-12T02:13:47.370 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.369+0000 7fbb3dcd0640 1 Processor -- start 2024-09-12T02:13:47.371 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.370+0000 7fbb3dcd0640 1 -- start start 2024-09-12T02:13:47.371 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.370+0000 7fbb3dcd0640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbb38101b60 0x7fbb38197dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:47.372 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.370+0000 7fbb3dcd0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb38102d50 0x7fbb38198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:47.372 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.370+0000 7fbb3dcd0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbb38103710 0x7fbb3819f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:47.372 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.370+0000 7fbb3dcd0640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbb3810d7e0 con 0x7fbb38102d50 2024-09-12T02:13:47.372 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.370+0000 7fbb3dcd0640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbb3810d660 con 0x7fbb38103710 2024-09-12T02:13:47.372 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.370+0000 7fbb3dcd0640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fbb3810d960 con 0x7fbb38101b60 2024-09-12T02:13:47.372 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.370+0000 7fbb36ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb38102d50 0x7fbb38198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:47.372 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.370+0000 7fbb36ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb38102d50 0x7fbb38198310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53852/0 (socket says 172.21.15.16:53852) 2024-09-12T02:13:47.372 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.370+0000 7fbb36ffd640 1 -- 172.21.15.16:0/466881187 learned_addr learned my addr 172.21.15.16:0/466881187 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:13:47.372 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.370+0000 7fbb377fe640 1 --2- 172.21.15.16:0/466881187 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbb38101b60 0x7fbb38197dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:47.372 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.370+0000 7fbb37fff640 1 --2- 172.21.15.16:0/466881187 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbb38103710 0x7fbb3819f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:47.372 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.370+0000 7fbb36ffd640 1 -- 172.21.15.16:0/466881187 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbb38101b60 msgr2=0x7fbb38197dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:47.373 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.370+0000 7fbb36ffd640 1 --2- 172.21.15.16:0/466881187 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbb38101b60 0x7fbb38197dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:47.373 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.370+0000 7fbb36ffd640 1 -- 172.21.15.16:0/466881187 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbb38103710 msgr2=0x7fbb3819f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:47.373 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.370+0000 7fbb36ffd640 1 --2- 172.21.15.16:0/466881187 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbb38103710 0x7fbb3819f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:47.373 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.370+0000 7fbb36ffd640 1 -- 172.21.15.16:0/466881187 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb3819f9f0 con 0x7fbb38102d50 2024-09-12T02:13:47.373 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.370+0000 7fbb36ffd640 1 --2- 172.21.15.16:0/466881187 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb38102d50 0x7fbb38198310 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c009230 tx=0x7fbb2c031b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:47.373 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.371+0000 7fbb34ff9640 1 -- 172.21.15.16:0/466881187 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb2c03f030 con 0x7fbb38102d50 2024-09-12T02:13:47.373 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.371+0000 7fbb34ff9640 1 -- 172.21.15.16:0/466881187 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb2c003ad0 con 0x7fbb38102d50 2024-09-12T02:13:47.373 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.371+0000 7fbb3dcd0640 1 -- 172.21.15.16:0/466881187 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb3819fc80 con 0x7fbb38102d50 2024-09-12T02:13:47.373 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.371+0000 7fbb34ff9640 1 -- 172.21.15.16:0/466881187 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb2c035070 con 0x7fbb38102d50 2024-09-12T02:13:47.373 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.371+0000 7fbb3dcd0640 1 -- 172.21.15.16:0/466881187 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb381a01c0 con 0x7fbb38102d50 2024-09-12T02:13:47.378 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.373+0000 7fbb34ff9640 1 -- 172.21.15.16:0/466881187 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fbb2c042030 con 0x7fbb38102d50 2024-09-12T02:13:47.379 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.373+0000 7fbb40a6b740 1 Processor -- start 2024-09-12T02:13:47.379 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.373+0000 7fbb40a6b740 1 -- start start 2024-09-12T02:13:47.379 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.374+0000 7fbb40a6b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558592e299a0 0x558592e29da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:47.379 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.374+0000 7fbb40a6b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558592e2a2e0 0x558592e2eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:47.379 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.374+0000 7fbb40a6b740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558592e2f370 0x558592e31830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:47.379 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.374+0000 7fbb40a6b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x558592d8d2f0 con 0x558592e2a2e0 2024-09-12T02:13:47.379 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.374+0000 7fbb40a6b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x558592d35270 con 0x558592e299a0 2024-09-12T02:13:47.379 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.374+0000 7fbb40a6b740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x558592d892b0 con 0x558592e2f370 2024-09-12T02:13:47.379 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.374+0000 7fbb377fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558592e2a2e0 0x558592e2eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:47.379 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.374+0000 7fbb377fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558592e2a2e0 0x558592e2eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53866/0 (socket says 172.21.15.16:53866) 2024-09-12T02:13:47.379 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.374+0000 7fbb377fe640 1 -- 172.21.15.16:0/4164338939 learned_addr learned my addr 172.21.15.16:0/4164338939 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:13:47.379 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.374+0000 7fbb37fff640 1 --2- 172.21.15.16:0/4164338939 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558592e299a0 0x558592e29da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:47.380 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.374+0000 7fbb36ffd640 1 --2- 172.21.15.16:0/4164338939 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558592e2f370 0x558592e31830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:47.380 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.374+0000 7fbb377fe640 1 -- 172.21.15.16:0/4164338939 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558592e2f370 msgr2=0x558592e31830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:47.380 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.374+0000 7fbb377fe640 1 --2- 172.21.15.16:0/4164338939 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558592e2f370 0x558592e31830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:47.380 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.374+0000 7fbb377fe640 1 -- 172.21.15.16:0/4164338939 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558592e299a0 msgr2=0x558592e29da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:47.380 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.374+0000 7fbb377fe640 1 --2- 172.21.15.16:0/4164338939 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558592e299a0 0x558592e29da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:47.380 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.374+0000 7fbb377fe640 1 -- 172.21.15.16:0/4164338939 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558592d34340 con 0x558592e2a2e0 2024-09-12T02:13:47.380 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.374+0000 7fbb34ff9640 1 --2- 172.21.15.16:0/466881187 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fbb200a2cb0 0x7fbb200a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:47.380 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.374+0000 7fbb377fe640 1 --2- 172.21.15.16:0/4164338939 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558592e2a2e0 0x558592e2eda0 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7fbb2800b7d0 tx=0x7fbb28004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bf48d0778dd620f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:47.380 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.374+0000 7fbb37fff640 1 --2- 172.21.15.16:0/466881187 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fbb200a2cb0 0x7fbb200a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:47.380 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.374+0000 7fbb34ff9640 1 -- 172.21.15.16:0/466881187 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fbb2c0ef400 con 0x7fbb38102d50 2024-09-12T02:13:47.380 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.374+0000 7fbb1dffb640 1 -- 172.21.15.16:0/4164338939 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb28003dd0 con 0x558592e2a2e0 2024-09-12T02:13:47.380 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.375+0000 7fbb1dffb640 1 -- 172.21.15.16:0/4164338939 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb28015070 con 0x558592e2a2e0 2024-09-12T02:13:47.380 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.375+0000 7fbb1dffb640 1 -- 172.21.15.16:0/4164338939 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb28011760 con 0x558592e2a2e0 2024-09-12T02:13:47.381 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.375+0000 7fbb40a6b740 1 -- 172.21.15.16:0/4164338939 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558592e2a2e0 msgr2=0x558592e2eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:47.381 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.375+0000 7fbb40a6b740 1 --2- 172.21.15.16:0/4164338939 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558592e2a2e0 0x558592e2eda0 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7fbb2800b7d0 tx=0x7fbb28004610 comp rx=0 tx=0).stop 2024-09-12T02:13:47.381 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.375+0000 7fbb37fff640 1 --2- 172.21.15.16:0/466881187 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fbb200a2cb0 0x7fbb200a5170 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fbb30005fd0 tx=0x7fbb30009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:47.381 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.375+0000 7fbb40a6b740 1 -- 172.21.15.16:0/4164338939 shutdown_connections 2024-09-12T02:13:47.381 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.375+0000 7fbb40a6b740 1 --2- 172.21.15.16:0/4164338939 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558592e2f370 0x558592e31830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:47.381 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.375+0000 7fbb40a6b740 1 --2- 172.21.15.16:0/4164338939 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558592e2a2e0 0x558592e2eda0 unknown :-1 s=CLOSED pgs=1709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:47.381 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.375+0000 7fbb40a6b740 1 --2- 172.21.15.16:0/4164338939 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558592e299a0 0x558592e29da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:47.381 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.375+0000 7fbb40a6b740 1 -- 172.21.15.16:0/4164338939 >> 172.21.15.16:0/4164338939 conn(0x558592e25010 msgr2=0x558592e25410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:47.381 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.375+0000 7fbb40a6b740 1 -- 172.21.15.16:0/4164338939 shutdown_connections 2024-09-12T02:13:47.381 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.375+0000 7fbb40a6b740 1 -- 172.21.15.16:0/4164338939 wait complete. 2024-09-12T02:13:47.381 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.376+0000 7fbb40a6b740 1 -- start start 2024-09-12T02:13:47.381 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.376+0000 7fbb40a6b740 1 Processor -- start 2024-09-12T02:13:47.381 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.376+0000 7fbb40a6b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558592e299a0 0x558592e33b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:47.382 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.376+0000 7fbb40a6b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558592e2a2e0 0x558592e340d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:47.382 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.376+0000 7fbb40a6b740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558592e2f370 0x558592e3b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:47.382 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.376+0000 7fbb40a6b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x558592e32340 con 0x558592e2a2e0 2024-09-12T02:13:47.382 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.376+0000 7fbb40a6b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x558592d16290 con 0x558592e299a0 2024-09-12T02:13:47.382 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.376+0000 7fbb40a6b740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x558592e324c0 con 0x558592e2f370 2024-09-12T02:13:47.382 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.376+0000 7fbb40a6b740 10 client.0 _ll_register_callbacks cb 0x558592d47140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:13:47.382 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.376+0000 7fbb36ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558592e2a2e0 0x558592e340d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:47.382 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.377+0000 7fbb36ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558592e2a2e0 0x558592e340d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53874/0 (socket says 172.21.15.16:53874) 2024-09-12T02:13:47.382 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.377+0000 7fbb377fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558592e299a0 0x558592e33b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:47.382 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.377+0000 7fbb36ffd640 1 -- 172.21.15.16:0/3575890890 learned_addr learned my addr 172.21.15.16:0/3575890890 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:13:47.382 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.377+0000 7fbb37fff640 1 --2- 172.21.15.16:0/3575890890 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558592e2f370 0x558592e3b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:47.382 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.377+0000 7fbb377fe640 1 -- 172.21.15.16:0/3575890890 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558592e2f370 msgr2=0x558592e3b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:47.382 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.377+0000 7fbb377fe640 1 --2- 172.21.15.16:0/3575890890 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558592e2f370 0x558592e3b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:47.383 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.377+0000 7fbb377fe640 1 -- 172.21.15.16:0/3575890890 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558592e2a2e0 msgr2=0x558592e340d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:47.383 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.377+0000 7fbb377fe640 1 --2- 172.21.15.16:0/3575890890 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558592e2a2e0 0x558592e340d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:47.383 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.377+0000 7fbb377fe640 1 -- 172.21.15.16:0/3575890890 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558592aa0210 con 0x558592e299a0 2024-09-12T02:13:47.383 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.377+0000 7fbb36ffd640 1 --2- 172.21.15.16:0/3575890890 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558592e2a2e0 0x558592e340d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:13:47.383 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.377+0000 7fbb37fff640 1 --2- 172.21.15.16:0/3575890890 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558592e2f370 0x558592e3b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:13:47.383 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.377+0000 7fbb377fe640 1 --2- 172.21.15.16:0/3575890890 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558592e299a0 0x558592e33b90 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x558592e25e90 tx=0x7fbb28003e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d888d5f89d0f71ee server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:47.383 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.377+0000 7fbb16ffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-12T02:13:47.383 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.377+0000 7fbb16ffd640 1 -- 172.21.15.16:0/3575890890 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb2801a880 con 0x558592e299a0 2024-09-12T02:13:47.383 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.377+0000 7fbb16ffd640 1 -- 172.21.15.16:0/3575890890 <== mon.2 v2:172.21.15.33:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fbb2801aa20 con 0x558592e299a0 2024-09-12T02:13:47.383 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.377+0000 7fbb40a6b740 1 -- 172.21.15.16:0/3575890890 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb28015070 con 0x558592e299a0 2024-09-12T02:13:47.383 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.377+0000 7fbb16ffd640 1 -- 172.21.15.16:0/3575890890 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fbb2800be70 con 0x558592e299a0 2024-09-12T02:13:47.383 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.377+0000 7fbb40a6b740 10 client.28457 fetch_fsmap learned FSMap version 25 2024-09-12T02:13:47.383 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.377+0000 7fbb40a6b740 1 -- 172.21.15.16:0/3575890890 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x558592b6c620 con 0x558592e299a0 2024-09-12T02:13:47.384 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.378+0000 7fbb16ffd640 1 -- 172.21.15.16:0/3575890890 <== mon.2 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb28015070 con 0x558592e299a0 2024-09-12T02:13:47.384 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.378+0000 7fbb16ffd640 1 -- 172.21.15.16:0/3575890890 <== mon.2 v2:172.21.15.33:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fbb2802f050 con 0x558592e299a0 2024-09-12T02:13:47.384 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.378+0000 7fbb40a6b740 10 client.28457 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:13:47.384 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.378+0000 7fbb40a6b740 10 client.28457 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:13:47.384 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.378+0000 7fbb40a6b740 20 client.28457 populate_metadata read hostname 'smithi016' 2024-09-12T02:13:47.384 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.378+0000 7fbb40a6b740 1 --2- 172.21.15.16:0/3575890890 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x558592e3f4d0 0x558592e418c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:47.384 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.378+0000 7fbb40a6b740 4 client.28457 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:13:47.384 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.378+0000 7fbb40a6b740 1 -- 172.21.15.16:0/3575890890 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558592d9aa20 con 0x558592e3f4d0 2024-09-12T02:13:47.384 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.378+0000 7fbb36ffd640 1 --2- 172.21.15.16:0/3575890890 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x558592e3f4d0 0x558592e418c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:47.384 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.379+0000 7fbb36ffd640 1 --2- 172.21.15.16:0/3575890890 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x558592e3f4d0 0x558592e418c0 crc :-1 s=READY pgs=366 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=779c9986b5d638f6 server_cookie=3289f7573573bb49 in_seq=0 out_seq=0 2024-09-12T02:13:47.384 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.379+0000 7fbb16ffd640 10 client.28457 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:13:47.385 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:13:47 smithi184 ceph-mon[35593]: scrub summary: idle 2024-09-12T02:13:47.386 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.379+0000 7fbb16ffd640 1 -- 172.21.15.16:0/3575890890 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558592d9aa20 con 0x558592e3f4d0 2024-09-12T02:13:47.386 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.379+0000 7fbb16ffd640 10 client.28457 handle_command_reply: tid=0 2024-09-12T02:13:47.386 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.379+0000 7fbb40a6b740 1 client.28457 shutdown 2024-09-12T02:13:47.386 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.380+0000 7fbb40a6b740 1 -- 172.21.15.16:0/3575890890 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558592e299a0 msgr2=0x558592e33b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:47.386 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.380+0000 7fbb40a6b740 1 --2- 172.21.15.16:0/3575890890 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558592e299a0 0x558592e33b90 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x558592e25e90 tx=0x7fbb28003e80 comp rx=0 tx=0).stop 2024-09-12T02:13:47.386 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.380+0000 7fbb40a6b740 1 -- 172.21.15.16:0/3575890890 shutdown_connections 2024-09-12T02:13:47.386 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.380+0000 7fbb40a6b740 1 --2- 172.21.15.16:0/3575890890 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x558592e3f4d0 0x558592e418c0 crc :-1 s=READY pgs=366 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:47.386 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.380+0000 7fbb40a6b740 1 --2- 172.21.15.16:0/3575890890 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558592e2f370 0x558592e3b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:47.387 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.380+0000 7fbb40a6b740 1 --2- 172.21.15.16:0/3575890890 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558592e2a2e0 0x558592e340d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:47.387 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.380+0000 7fbb16ffd640 0 client.28457 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:13:47.387 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.380+0000 7fbb40a6b740 1 --2- 172.21.15.16:0/3575890890 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558592e299a0 0x558592e33b90 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:47.387 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.380+0000 7fbb40a6b740 1 -- 172.21.15.16:0/3575890890 >> 172.21.15.16:0/3575890890 conn(0x558592e25010 msgr2=0x558592e26820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:47.387 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.380+0000 7fbb40a6b740 1 -- 172.21.15.16:0/3575890890 shutdown_connections 2024-09-12T02:13:47.387 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.380+0000 7fbb40a6b740 1 -- 172.21.15.16:0/3575890890 wait complete. 2024-09-12T02:13:47.387 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.380+0000 7fbb40a6b740 20 client.28457 trim_cache size 0 max 16384 2024-09-12T02:13:47.399 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.395+0000 7fbb40a6b740 1 Processor -- start 2024-09-12T02:13:47.399 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.395+0000 7fbb40a6b740 1 -- start start 2024-09-12T02:13:47.399 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.395+0000 7fbb40a6b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558592e299a0 0x558592e266b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:47.399 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.395+0000 7fbb40a6b740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558592e2a2e0 0x558592e3b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:47.400 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.395+0000 7fbb40a6b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558592e2f370 0x558592e46130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:47.400 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.395+0000 7fbb40a6b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbb381ab7c0 con 0x558592e2f370 2024-09-12T02:13:47.400 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.395+0000 7fbb40a6b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbb381a9090 con 0x558592e299a0 2024-09-12T02:13:47.400 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.395+0000 7fbb40a6b740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fbb381a6ee0 con 0x558592e2a2e0 2024-09-12T02:13:47.400 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.395+0000 7fbb37fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558592e2f370 0x558592e46130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:47.400 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.395+0000 7fbb37fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558592e2f370 0x558592e46130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53882/0 (socket says 172.21.15.16:53882) 2024-09-12T02:13:47.400 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.395+0000 7fbb37fff640 1 -- 172.21.15.16:0/108045993 learned_addr learned my addr 172.21.15.16:0/108045993 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:13:47.400 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.395+0000 7fbb377fe640 1 --2- 172.21.15.16:0/108045993 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558592e299a0 0x558592e266b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:47.400 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.395+0000 7fbb36ffd640 1 --2- 172.21.15.16:0/108045993 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558592e2a2e0 0x558592e3b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:47.400 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.395+0000 7fbb37fff640 1 -- 172.21.15.16:0/108045993 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558592e2a2e0 msgr2=0x558592e3b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:47.400 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.395+0000 7fbb37fff640 1 --2- 172.21.15.16:0/108045993 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558592e2a2e0 0x558592e3b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:47.400 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.395+0000 7fbb37fff640 1 -- 172.21.15.16:0/108045993 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558592e299a0 msgr2=0x558592e266b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:47.401 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.395+0000 7fbb37fff640 1 --2- 172.21.15.16:0/108045993 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558592e299a0 0x558592e266b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:47.401 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.395+0000 7fbb37fff640 1 -- 172.21.15.16:0/108045993 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558592e41c30 con 0x558592e2f370 2024-09-12T02:13:47.401 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.395+0000 7fbb36ffd640 1 --2- 172.21.15.16:0/108045993 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558592e2a2e0 0x558592e3b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:13:47.401 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.395+0000 7fbb37fff640 1 --2- 172.21.15.16:0/108045993 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558592e2f370 0x558592e46130 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7fbb30003da0 tx=0x7fbb30016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21324b705524c3b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:47.401 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.396+0000 7fbb1d7fa640 1 -- 172.21.15.16:0/108045993 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb30004540 con 0x558592e2f370 2024-09-12T02:13:47.401 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.396+0000 7fbb1d7fa640 1 -- 172.21.15.16:0/108045993 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb300046e0 con 0x558592e2f370 2024-09-12T02:13:47.401 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.396+0000 7fbb1d7fa640 1 -- 172.21.15.16:0/108045993 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb3001c500 con 0x558592e2f370 2024-09-12T02:13:47.401 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.396+0000 7fbb40a6b740 1 -- 172.21.15.16:0/108045993 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558592e2f370 msgr2=0x558592e46130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:47.401 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.396+0000 7fbb40a6b740 1 --2- 172.21.15.16:0/108045993 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558592e2f370 0x558592e46130 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7fbb30003da0 tx=0x7fbb30016000 comp rx=0 tx=0).stop 2024-09-12T02:13:47.401 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.396+0000 7fbb40a6b740 1 -- 172.21.15.16:0/108045993 shutdown_connections 2024-09-12T02:13:47.401 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.396+0000 7fbb40a6b740 1 --2- 172.21.15.16:0/108045993 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558592e2f370 0x558592e46130 unknown :-1 s=CLOSED pgs=1710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:47.401 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.396+0000 7fbb40a6b740 1 --2- 172.21.15.16:0/108045993 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558592e2a2e0 0x558592e3b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:47.401 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.396+0000 7fbb40a6b740 1 --2- 172.21.15.16:0/108045993 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558592e299a0 0x558592e266b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:47.402 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.396+0000 7fbb40a6b740 1 -- 172.21.15.16:0/108045993 >> 172.21.15.16:0/108045993 conn(0x558592e25010 msgr2=0x558592e25fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:47.402 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.396+0000 7fbb40a6b740 1 -- 172.21.15.16:0/108045993 shutdown_connections 2024-09-12T02:13:47.402 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.396+0000 7fbb40a6b740 1 -- 172.21.15.16:0/108045993 wait complete. 2024-09-12T02:13:47.402 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.397+0000 7fbb40a6b740 1 -- start start 2024-09-12T02:13:47.402 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.397+0000 7fbb40a6b740 1 Processor -- start 2024-09-12T02:13:47.402 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.397+0000 7fbb40a6b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558592e299a0 0x558592e38850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:47.402 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.397+0000 7fbb40a6b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558592e2a2e0 0x558592e38d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:47.402 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.397+0000 7fbb40a6b740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558592e2f370 0x558592e39da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:47.402 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.397+0000 7fbb40a6b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbb381a7550 con 0x558592e299a0 2024-09-12T02:13:47.402 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.397+0000 7fbb40a6b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbb381a89c0 con 0x558592e2a2e0 2024-09-12T02:13:47.402 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.397+0000 7fbb40a6b740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fbb381a7c20 con 0x558592e2f370 2024-09-12T02:13:47.402 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.397+0000 7fbb40a6b740 10 client.0 _ll_register_callbacks cb 0x7fbb30009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:13:47.402 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.397+0000 7fbb377fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558592e299a0 0x558592e38850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:47.403 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.397+0000 7fbb36ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558592e2a2e0 0x558592e38d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:47.403 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.397+0000 7fbb377fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558592e299a0 0x558592e38850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53896/0 (socket says 172.21.15.16:53896) 2024-09-12T02:13:47.403 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.397+0000 7fbb377fe640 1 -- 172.21.15.16:0/3858156424 learned_addr learned my addr 172.21.15.16:0/3858156424 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:13:47.403 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.397+0000 7fbb37fff640 1 --2- 172.21.15.16:0/3858156424 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558592e2f370 0x558592e39da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:47.403 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.398+0000 7fbb36ffd640 1 -- 172.21.15.16:0/3858156424 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558592e2f370 msgr2=0x558592e39da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:47.403 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.398+0000 7fbb36ffd640 1 --2- 172.21.15.16:0/3858156424 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558592e2f370 0x558592e39da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:47.403 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.398+0000 7fbb36ffd640 1 -- 172.21.15.16:0/3858156424 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558592e299a0 msgr2=0x558592e38850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:13:47.403 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.398+0000 7fbb36ffd640 1 --2- 172.21.15.16:0/3858156424 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558592e299a0 0x558592e38850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:47.403 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.398+0000 7fbb36ffd640 1 -- 172.21.15.16:0/3858156424 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558592e3a3b0 con 0x558592e2a2e0 2024-09-12T02:13:47.403 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.398+0000 7fbb377fe640 1 --2- 172.21.15.16:0/3858156424 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558592e299a0 0x558592e38850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:13:47.403 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.398+0000 7fbb36ffd640 1 --2- 172.21.15.16:0/3858156424 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558592e2a2e0 0x558592e38d90 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c0c4440 tx=0x7fbb2c0bbdb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a80b3c86534e47a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:47.403 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.398+0000 7fbb16ffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-12T02:13:47.403 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.398+0000 7fbb16ffd640 1 -- 172.21.15.16:0/3858156424 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb2c049040 con 0x558592e2a2e0 2024-09-12T02:13:47.404 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.398+0000 7fbb40a6b740 1 -- 172.21.15.16:0/3858156424 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb300046e0 con 0x558592e2a2e0 2024-09-12T02:13:47.404 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.398+0000 7fbb16ffd640 1 -- 172.21.15.16:0/3858156424 <== mon.2 v2:172.21.15.33:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fbb2c0f2d70 con 0x558592e2a2e0 2024-09-12T02:13:47.404 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.398+0000 7fbb16ffd640 1 -- 172.21.15.16:0/3858156424 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fbb2c0f2a50 con 0x558592e2a2e0 2024-09-12T02:13:47.404 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.398+0000 7fbb40a6b740 10 client.28463 fetch_fsmap learned FSMap version 25 2024-09-12T02:13:47.404 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.398+0000 7fbb40a6b740 1 -- 172.21.15.16:0/3858156424 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x558592eaa240 con 0x558592e2a2e0 2024-09-12T02:13:47.404 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.398+0000 7fbb16ffd640 1 -- 172.21.15.16:0/3858156424 <== mon.2 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb300046e0 con 0x558592e2a2e0 2024-09-12T02:13:47.404 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.399+0000 7fbb16ffd640 1 -- 172.21.15.16:0/3858156424 <== mon.2 v2:172.21.15.33:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fbb2c03bc70 con 0x558592e2a2e0 2024-09-12T02:13:47.404 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.399+0000 7fbb40a6b740 10 client.28463 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:13:47.404 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.399+0000 7fbb40a6b740 10 client.28463 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:13:47.404 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.399+0000 7fbb40a6b740 20 client.28463 populate_metadata read hostname 'smithi016' 2024-09-12T02:13:47.404 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.399+0000 7fbb40a6b740 1 --2- 172.21.15.16:0/3858156424 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x558592eaabc0 0x558592eacfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:47.404 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.399+0000 7fbb40a6b740 4 client.28463 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:13:47.404 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.399+0000 7fbb40a6b740 1 -- 172.21.15.16:0/3858156424 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558592d94d70 con 0x558592eaabc0 2024-09-12T02:13:47.405 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.399+0000 7fbb377fe640 1 --2- 172.21.15.16:0/3858156424 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x558592eaabc0 0x558592eacfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:47.405 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.399+0000 7fbb16ffd640 10 client.28463 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:13:47.405 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.399+0000 7fbb377fe640 1 --2- 172.21.15.16:0/3858156424 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x558592eaabc0 0x558592eacfe0 crc :-1 s=READY pgs=367 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=66d15f56dc508436 server_cookie=bf39c893585a467a in_seq=0 out_seq=0 2024-09-12T02:13:47.412 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.411+0000 7fbb16ffd640 1 -- 172.21.15.16:0/3858156424 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558592d94d70 con 0x558592eaabc0 2024-09-12T02:13:47.412 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.412+0000 7fbb16ffd640 10 client.28463 handle_command_reply: tid=0 2024-09-12T02:13:47.412 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.412+0000 7fbb40a6b740 1 client.28463 shutdown 2024-09-12T02:13:47.413 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.412+0000 7fbb40a6b740 1 -- 172.21.15.16:0/3858156424 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558592e2a2e0 msgr2=0x558592e38d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:47.413 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.412+0000 7fbb40a6b740 1 --2- 172.21.15.16:0/3858156424 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558592e2a2e0 0x558592e38d90 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c0c4440 tx=0x7fbb2c0bbdb0 comp rx=0 tx=0).stop 2024-09-12T02:13:47.413 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.412+0000 7fbb40a6b740 1 -- 172.21.15.16:0/3858156424 shutdown_connections 2024-09-12T02:13:47.413 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.412+0000 7fbb40a6b740 1 --2- 172.21.15.16:0/3858156424 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x558592eaabc0 0x558592eacfe0 crc :-1 s=READY pgs=367 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:47.413 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.412+0000 7fbb40a6b740 1 --2- 172.21.15.16:0/3858156424 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558592e2f370 0x558592e39da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:47.413 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.412+0000 7fbb40a6b740 1 --2- 172.21.15.16:0/3858156424 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558592e2a2e0 0x558592e38d90 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:47.413 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.412+0000 7fbb40a6b740 1 --2- 172.21.15.16:0/3858156424 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558592e299a0 0x558592e38850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:47.413 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.412+0000 7fbb16ffd640 0 client.28463 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:13:47.413 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.412+0000 7fbb40a6b740 1 -- 172.21.15.16:0/3858156424 >> 172.21.15.16:0/3858156424 conn(0x558592e25010 msgr2=0x558592e239a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:47.413 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.412+0000 7fbb40a6b740 1 -- 172.21.15.16:0/3858156424 shutdown_connections 2024-09-12T02:13:47.413 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.412+0000 7fbb40a6b740 1 -- 172.21.15.16:0/3858156424 wait complete. 2024-09-12T02:13:47.413 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.412+0000 7fbb40a6b740 20 client.28463 trim_cache size 0 max 16384 2024-09-12T02:13:47.414 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-09-12T02:13:47.414 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-09-12T02:13:47.414 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-09-12T02:13:47.414 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-09-12T02:13:47.415 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.412+0000 7fbb3dcd0640 1 -- 172.21.15.16:0/466881187 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fbb200a2cb0 msgr2=0x7fbb200a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:47.415 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.413+0000 7fbb3dcd0640 1 --2- 172.21.15.16:0/466881187 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fbb200a2cb0 0x7fbb200a5170 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fbb30005fd0 tx=0x7fbb30009500 comp rx=0 tx=0).stop 2024-09-12T02:13:47.415 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.413+0000 7fbb3dcd0640 1 -- 172.21.15.16:0/466881187 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb38102d50 msgr2=0x7fbb38198310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:47.415 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.413+0000 7fbb3dcd0640 1 --2- 172.21.15.16:0/466881187 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb38102d50 0x7fbb38198310 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c009230 tx=0x7fbb2c031b60 comp rx=0 tx=0).stop 2024-09-12T02:13:47.415 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.413+0000 7fbb3dcd0640 1 -- 172.21.15.16:0/466881187 shutdown_connections 2024-09-12T02:13:47.415 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.413+0000 7fbb3dcd0640 1 --2- 172.21.15.16:0/466881187 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fbb200a2cb0 0x7fbb200a5170 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:47.415 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.413+0000 7fbb3dcd0640 1 --2- 172.21.15.16:0/466881187 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbb38103710 0x7fbb3819f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:47.415 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.413+0000 7fbb3dcd0640 1 --2- 172.21.15.16:0/466881187 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb38102d50 0x7fbb38198310 unknown :-1 s=CLOSED pgs=1708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:47.415 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.413+0000 7fbb3dcd0640 1 --2- 172.21.15.16:0/466881187 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbb38101b60 0x7fbb38197dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:47.415 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.413+0000 7fbb3dcd0640 1 -- 172.21.15.16:0/466881187 >> 172.21.15.16:0/466881187 conn(0x7fbb380fd2e0 msgr2=0x7fbb381060a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:47.415 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.413+0000 7fbb3dcd0640 1 -- 172.21.15.16:0/466881187 shutdown_connections 2024-09-12T02:13:47.416 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:13:47.413+0000 7fbb3dcd0640 1 -- 172.21.15.16:0/466881187 wait complete. 2024-09-12T02:13:47.418 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:13:47 smithi033 ceph-mon[35332]: scrub summary: idle 2024-09-12T02:13:47.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:13:47 smithi016 ceph-mon[30947]: scrub summary: idle 2024-09-12T02:13:48.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:13:48 smithi184 ceph-mon[35593]: pgmap v691: 97 pgs: 97 active+clean; 2.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 KiB/s wr, 15 op/s 2024-09-12T02:13:48.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:13:48 smithi033 ceph-mon[35332]: pgmap v691: 97 pgs: 97 active+clean; 2.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 KiB/s wr, 15 op/s 2024-09-12T02:13:48.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:13:48 smithi016 ceph-mon[30947]: pgmap v691: 97 pgs: 97 active+clean; 2.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 KiB/s wr, 15 op/s 2024-09-12T02:13:50.377 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:13:50 smithi184 ceph-mon[35593]: pgmap v692: 97 pgs: 97 active+clean; 2.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.7 KiB/s wr, 10 op/s 2024-09-12T02:13:50.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:13:50 smithi033 ceph-mon[35332]: pgmap v692: 97 pgs: 97 active+clean; 2.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.7 KiB/s wr, 10 op/s 2024-09-12T02:13:50.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:13:50 smithi016 ceph-mon[30947]: pgmap v692: 97 pgs: 97 active+clean; 2.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.7 KiB/s wr, 10 op/s 2024-09-12T02:13:52.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:13:52 smithi033 ceph-mon[35332]: pgmap v693: 97 pgs: 97 active+clean; 2.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.1 KiB/s wr, 12 op/s 2024-09-12T02:13:52.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:13:52 smithi016 ceph-mon[30947]: pgmap v693: 97 pgs: 97 active+clean; 2.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.1 KiB/s wr, 12 op/s 2024-09-12T02:13:52.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:13:52 smithi184 ceph-mon[35593]: pgmap v693: 97 pgs: 97 active+clean; 2.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.1 KiB/s wr, 12 op/s 2024-09-12T02:13:54.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:13:54 smithi033 ceph-mon[35332]: pgmap v694: 97 pgs: 97 active+clean; 2.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.9 KiB/s wr, 12 op/s 2024-09-12T02:13:54.429 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:13:54 smithi184 ceph-mon[35593]: pgmap v694: 97 pgs: 97 active+clean; 2.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.9 KiB/s wr, 12 op/s 2024-09-12T02:13:54.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:13:54 smithi016 ceph-mon[30947]: pgmap v694: 97 pgs: 97 active+clean; 2.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.9 KiB/s wr, 12 op/s 2024-09-12T02:13:56.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:13:56 smithi033 ceph-mon[35332]: pgmap v695: 97 pgs: 97 active+clean; 2.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 2.9 KiB/s wr, 10 op/s 2024-09-12T02:13:56.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:13:56 smithi016 ceph-mon[30947]: pgmap v695: 97 pgs: 97 active+clean; 2.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 2.9 KiB/s wr, 10 op/s 2024-09-12T02:13:56.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:13:56 smithi184 ceph-mon[35593]: pgmap v695: 97 pgs: 97 active+clean; 2.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 2.9 KiB/s wr, 10 op/s 2024-09-12T02:13:58.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:13:58 smithi033 ceph-mon[35332]: pgmap v696: 97 pgs: 97 active+clean; 2.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.9 KiB/s wr, 10 op/s 2024-09-12T02:13:58.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:13:58 smithi016 ceph-mon[30947]: pgmap v696: 97 pgs: 97 active+clean; 2.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.9 KiB/s wr, 10 op/s 2024-09-12T02:13:58.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:13:58 smithi184 ceph-mon[35593]: pgmap v696: 97 pgs: 97 active+clean; 2.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.9 KiB/s wr, 10 op/s 2024-09-12T02:14:00.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:14:00 smithi033 ceph-mon[35332]: pgmap v697: 97 pgs: 97 active+clean; 2.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.8 KiB/s wr, 7 op/s 2024-09-12T02:14:00.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:14:00 smithi016 ceph-mon[30947]: pgmap v697: 97 pgs: 97 active+clean; 2.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.8 KiB/s wr, 7 op/s 2024-09-12T02:14:00.514 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.513+0000 7fa73e3c4640 1 -- 172.21.15.16:0/865251448 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa738107840 msgr2=0x7fa738109cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:14:00.514 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.513+0000 7fa73e3c4640 1 --2- 172.21.15.16:0/865251448 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa738107840 0x7fa738109cb0 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7fa730009ca0 tx=0x7fa73002a8c0 comp rx=0 tx=0).stop 2024-09-12T02:14:00.514 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.513+0000 7fa73e3c4640 1 -- 172.21.15.16:0/865251448 shutdown_connections 2024-09-12T02:14:00.514 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.513+0000 7fa73e3c4640 1 --2- 172.21.15.16:0/865251448 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa738107840 0x7fa738109cb0 unknown :-1 s=CLOSED pgs=1711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:00.514 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.513+0000 7fa73e3c4640 1 --2- 172.21.15.16:0/865251448 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa738104f10 0x7fa738107300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:00.515 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.513+0000 7fa73e3c4640 1 --2- 172.21.15.16:0/865251448 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa7381025e0 0x7fa7381049d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:00.515 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.513+0000 7fa73e3c4640 1 -- 172.21.15.16:0/865251448 >> 172.21.15.16:0/865251448 conn(0x7fa7380fc1e0 msgr2=0x7fa7380fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:14:00.515 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.513+0000 7fa73e3c4640 1 -- 172.21.15.16:0/865251448 shutdown_connections 2024-09-12T02:14:00.515 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.513+0000 7fa73e3c4640 1 -- 172.21.15.16:0/865251448 wait complete. 2024-09-12T02:14:00.515 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.514+0000 7fa73e3c4640 1 Processor -- start 2024-09-12T02:14:00.515 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.514+0000 7fa73e3c4640 1 -- start start 2024-09-12T02:14:00.517 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.514+0000 7fa73e3c4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa7381025e0 0x7fa738101f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:00.517 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.514+0000 7fa73e3c4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa738104f10 0x7fa738100590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:00.517 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.514+0000 7fa73e3c4640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa738107840 0x7fa738100ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:00.517 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.515+0000 7fa73e3c4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa73810d800 con 0x7fa738104f10 2024-09-12T02:14:00.517 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.515+0000 7fa73e3c4640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa73810d680 con 0x7fa7381025e0 2024-09-12T02:14:00.517 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.515+0000 7fa73e3c4640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa73810d980 con 0x7fa738107840 2024-09-12T02:14:00.518 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.515+0000 7fa7377fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa738104f10 0x7fa738100590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:00.518 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.515+0000 7fa7377fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa738104f10 0x7fa738100590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37120/0 (socket says 172.21.15.16:37120) 2024-09-12T02:14:00.518 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.515+0000 7fa7377fe640 1 -- 172.21.15.16:0/2889033676 learned_addr learned my addr 172.21.15.16:0/2889033676 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:14:00.518 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.515+0000 7fa737fff640 1 --2- 172.21.15.16:0/2889033676 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa7381025e0 0x7fa738101f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:00.518 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.515+0000 7fa73c961640 1 --2- 172.21.15.16:0/2889033676 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa738107840 0x7fa738100ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:00.518 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.515+0000 7fa7377fe640 1 -- 172.21.15.16:0/2889033676 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa738107840 msgr2=0x7fa738100ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:14:00.518 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.515+0000 7fa7377fe640 1 --2- 172.21.15.16:0/2889033676 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa738107840 0x7fa738100ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:00.518 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.515+0000 7fa7377fe640 1 -- 172.21.15.16:0/2889033676 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa7381025e0 msgr2=0x7fa738101f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:14:00.518 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.515+0000 7fa7377fe640 1 --2- 172.21.15.16:0/2889033676 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa7381025e0 0x7fa738101f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:00.518 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.515+0000 7fa7377fe640 1 -- 172.21.15.16:0/2889033676 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa738101360 con 0x7fa738104f10 2024-09-12T02:14:00.518 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.515+0000 7fa7377fe640 1 --2- 172.21.15.16:0/2889033676 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa738104f10 0x7fa738100590 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7fa72c00c970 tx=0x7fa72c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:14:00.518 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.515+0000 7fa7357fa640 1 -- 172.21.15.16:0/2889033676 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa72c014070 con 0x7fa738104f10 2024-09-12T02:14:00.518 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.515+0000 7fa7357fa640 1 -- 172.21.15.16:0/2889033676 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa72c004440 con 0x7fa738104f10 2024-09-12T02:14:00.519 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.515+0000 7fa73e3c4640 1 -- 172.21.15.16:0/2889033676 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7381a1ec0 con 0x7fa738104f10 2024-09-12T02:14:00.519 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.515+0000 7fa7357fa640 1 -- 172.21.15.16:0/2889033676 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa72c00f450 con 0x7fa738104f10 2024-09-12T02:14:00.519 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.516+0000 7fa73e3c4640 1 -- 172.21.15.16:0/2889033676 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7381a2330 con 0x7fa738104f10 2024-09-12T02:14:00.525 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.518+0000 7fa7357fa640 1 -- 172.21.15.16:0/2889033676 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa72c017070 con 0x7fa738104f10 2024-09-12T02:14:00.525 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.518+0000 7fa74123d740 1 Processor -- start 2024-09-12T02:14:00.525 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.518+0000 7fa74123d740 1 -- start start 2024-09-12T02:14:00.525 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.518+0000 7fa74123d740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55aad78cd9a0 0x55aad78cdda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:00.525 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.518+0000 7fa74123d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55aad78ce2e0 0x55aad78d2da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:00.525 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.518+0000 7fa74123d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aad78d3370 0x55aad78d5830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:00.525 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.518+0000 7fa74123d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55aad78312f0 con 0x55aad78ce2e0 2024-09-12T02:14:00.525 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.518+0000 7fa74123d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55aad77d9270 con 0x55aad78d3370 2024-09-12T02:14:00.525 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.518+0000 7fa74123d740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55aad782d2b0 con 0x55aad78cd9a0 2024-09-12T02:14:00.525 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.519+0000 7fa737fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55aad78ce2e0 0x55aad78d2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:00.526 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.519+0000 7fa737fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55aad78ce2e0 0x55aad78d2da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37130/0 (socket says 172.21.15.16:37130) 2024-09-12T02:14:00.526 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.519+0000 7fa737fff640 1 -- 172.21.15.16:0/1690306605 learned_addr learned my addr 172.21.15.16:0/1690306605 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:14:00.526 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.519+0000 7fa73c961640 1 --2- 172.21.15.16:0/1690306605 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55aad78cd9a0 0x55aad78cdda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:00.526 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.519+0000 7fa7377fe640 1 --2- 172.21.15.16:0/1690306605 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aad78d3370 0x55aad78d5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:00.526 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.519+0000 7fa737fff640 1 -- 172.21.15.16:0/1690306605 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55aad78cd9a0 msgr2=0x55aad78cdda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:14:00.526 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.519+0000 7fa737fff640 1 --2- 172.21.15.16:0/1690306605 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55aad78cd9a0 0x55aad78cdda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:00.526 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.519+0000 7fa737fff640 1 -- 172.21.15.16:0/1690306605 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aad78d3370 msgr2=0x55aad78d5830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:14:00.526 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.519+0000 7fa737fff640 1 --2- 172.21.15.16:0/1690306605 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aad78d3370 0x55aad78d5830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:00.526 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.519+0000 7fa737fff640 1 -- 172.21.15.16:0/1690306605 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aad77d8340 con 0x55aad78ce2e0 2024-09-12T02:14:00.526 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.519+0000 7fa737fff640 1 --2- 172.21.15.16:0/1690306605 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55aad78ce2e0 0x55aad78d2da0 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7fa728009cf0 tx=0x7fa728011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a953b451477a4d4a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:14:00.526 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.519+0000 7fa7357fa640 1 --2- 172.21.15.16:0/2889033676 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fa7200a2cb0 0x7fa7200a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:00.526 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.519+0000 7fa73c961640 1 --2- 172.21.15.16:0/2889033676 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fa7200a2cb0 0x7fa7200a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:00.526 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.519+0000 7fa71e7fc640 1 -- 172.21.15.16:0/1690306605 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa728004230 con 0x55aad78ce2e0 2024-09-12T02:14:00.527 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.519+0000 7fa71e7fc640 1 -- 172.21.15.16:0/1690306605 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7280043d0 con 0x55aad78ce2e0 2024-09-12T02:14:00.527 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.519+0000 7fa71e7fc640 1 -- 172.21.15.16:0/1690306605 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa728019960 con 0x55aad78ce2e0 2024-09-12T02:14:00.527 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.519+0000 7fa7357fa640 1 -- 172.21.15.16:0/2889033676 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fa72c0ced70 con 0x7fa738104f10 2024-09-12T02:14:00.527 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.520+0000 7fa74123d740 1 -- 172.21.15.16:0/1690306605 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55aad78ce2e0 msgr2=0x55aad78d2da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:14:00.527 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.520+0000 7fa73c961640 1 --2- 172.21.15.16:0/2889033676 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fa7200a2cb0 0x7fa7200a5170 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fa730005cc0 tx=0x7fa730031bb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:14:00.527 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.520+0000 7fa74123d740 1 --2- 172.21.15.16:0/1690306605 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55aad78ce2e0 0x55aad78d2da0 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7fa728009cf0 tx=0x7fa728011810 comp rx=0 tx=0).stop 2024-09-12T02:14:00.527 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.520+0000 7fa74123d740 1 -- 172.21.15.16:0/1690306605 shutdown_connections 2024-09-12T02:14:00.527 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.520+0000 7fa74123d740 1 --2- 172.21.15.16:0/1690306605 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aad78d3370 0x55aad78d5830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:00.527 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.520+0000 7fa74123d740 1 --2- 172.21.15.16:0/1690306605 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55aad78ce2e0 0x55aad78d2da0 unknown :-1 s=CLOSED pgs=1713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:00.527 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.520+0000 7fa74123d740 1 --2- 172.21.15.16:0/1690306605 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55aad78cd9a0 0x55aad78cdda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:00.527 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.520+0000 7fa74123d740 1 -- 172.21.15.16:0/1690306605 >> 172.21.15.16:0/1690306605 conn(0x55aad78c9010 msgr2=0x55aad78c9410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:14:00.527 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.520+0000 7fa74123d740 1 -- 172.21.15.16:0/1690306605 shutdown_connections 2024-09-12T02:14:00.527 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.520+0000 7fa74123d740 1 -- 172.21.15.16:0/1690306605 wait complete. 2024-09-12T02:14:00.528 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.521+0000 7fa74123d740 1 -- start start 2024-09-12T02:14:00.528 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.521+0000 7fa74123d740 1 Processor -- start 2024-09-12T02:14:00.528 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.521+0000 7fa74123d740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55aad78cd9a0 0x55aad78d7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:00.528 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.521+0000 7fa74123d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55aad78ce2e0 0x55aad78d8110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:00.528 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.521+0000 7fa74123d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aad78d3370 0x55aad78df150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:00.528 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.521+0000 7fa74123d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55aad78d6340 con 0x55aad78ce2e0 2024-09-12T02:14:00.528 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.521+0000 7fa74123d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55aad77ba290 con 0x55aad78d3370 2024-09-12T02:14:00.528 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.521+0000 7fa74123d740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55aad78d64c0 con 0x55aad78cd9a0 2024-09-12T02:14:00.528 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.521+0000 7fa74123d740 10 client.0 _ll_register_callbacks cb 0x55aad77eb140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:14:00.528 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.521+0000 7fa7377fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55aad78ce2e0 0x55aad78d8110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:00.528 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.521+0000 7fa7377fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55aad78ce2e0 0x55aad78d8110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37140/0 (socket says 172.21.15.16:37140) 2024-09-12T02:14:00.528 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.521+0000 7fa7377fe640 1 -- 172.21.15.16:0/646270721 learned_addr learned my addr 172.21.15.16:0/646270721 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:14:00.529 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.521+0000 7fa73c961640 1 --2- 172.21.15.16:0/646270721 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aad78d3370 0x55aad78df150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:00.529 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.522+0000 7fa737fff640 1 --2- 172.21.15.16:0/646270721 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55aad78cd9a0 0x55aad78d7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:00.529 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.522+0000 7fa7377fe640 1 -- 172.21.15.16:0/646270721 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55aad78cd9a0 msgr2=0x55aad78d7bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:14:00.529 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.522+0000 7fa7377fe640 1 --2- 172.21.15.16:0/646270721 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55aad78cd9a0 0x55aad78d7bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:00.529 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.522+0000 7fa7377fe640 1 -- 172.21.15.16:0/646270721 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aad78d3370 msgr2=0x55aad78df150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:14:00.529 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.522+0000 7fa7377fe640 1 --2- 172.21.15.16:0/646270721 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aad78d3370 0x55aad78df150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:00.529 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.522+0000 7fa7377fe640 1 -- 172.21.15.16:0/646270721 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aad7544210 con 0x55aad78ce2e0 2024-09-12T02:14:00.529 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.522+0000 7fa7377fe640 1 --2- 172.21.15.16:0/646270721 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55aad78ce2e0 0x55aad78d8110 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7fa72c09b600 tx=0x7fa72c097a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5cbae2086fe62af server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:14:00.529 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.522+0000 7fa7177fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:14:00.529 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.522+0000 7fa7177fe640 1 -- 172.21.15.16:0/646270721 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa72c09bd70 con 0x55aad78ce2e0 2024-09-12T02:14:00.529 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.522+0000 7fa7177fe640 1 -- 172.21.15.16:0/646270721 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fa72c0afb90 con 0x55aad78ce2e0 2024-09-12T02:14:00.529 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.522+0000 7fa74123d740 1 -- 172.21.15.16:0/646270721 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7280043d0 con 0x55aad78ce2e0 2024-09-12T02:14:00.529 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.522+0000 7fa7177fe640 1 -- 172.21.15.16:0/646270721 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa72c09b670 con 0x55aad78ce2e0 2024-09-12T02:14:00.530 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.522+0000 7fa74123d740 10 client.19518 fetch_fsmap learned FSMap version 25 2024-09-12T02:14:00.530 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.522+0000 7fa74123d740 1 -- 172.21.15.16:0/646270721 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55aad7610620 con 0x55aad78ce2e0 2024-09-12T02:14:00.530 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.522+0000 7fa7177fe640 1 -- 172.21.15.16:0/646270721 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7280043d0 con 0x55aad78ce2e0 2024-09-12T02:14:00.530 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.523+0000 7fa7177fe640 1 -- 172.21.15.16:0/646270721 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fa72c09aac0 con 0x55aad78ce2e0 2024-09-12T02:14:00.530 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.523+0000 7fa74123d740 10 client.19518 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:14:00.530 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.523+0000 7fa74123d740 10 client.19518 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:14:00.530 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.523+0000 7fa74123d740 20 client.19518 populate_metadata read hostname 'smithi016' 2024-09-12T02:14:00.530 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.523+0000 7fa74123d740 1 --2- 172.21.15.16:0/646270721 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55aad78e3890 0x55aad78e5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:00.530 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.523+0000 7fa74123d740 4 client.19518 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:14:00.530 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.523+0000 7fa74123d740 1 -- 172.21.15.16:0/646270721 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aad783ea20 con 0x55aad78e3890 2024-09-12T02:14:00.530 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.523+0000 7fa737fff640 1 --2- 172.21.15.16:0/646270721 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55aad78e3890 0x55aad78e5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:00.530 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.523+0000 7fa7177fe640 10 client.19518 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:14:00.530 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.523+0000 7fa737fff640 1 --2- 172.21.15.16:0/646270721 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55aad78e3890 0x55aad78e5ce0 crc :-1 s=READY pgs=368 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c06a530c0749409 server_cookie=af8b0faf673c187c in_seq=0 out_seq=0 2024-09-12T02:14:00.530 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.524+0000 7fa7177fe640 1 -- 172.21.15.16:0/646270721 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55aad783ea20 con 0x55aad78e3890 2024-09-12T02:14:00.531 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.524+0000 7fa7177fe640 10 client.19518 handle_command_reply: tid=0 2024-09-12T02:14:00.531 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.524+0000 7fa74123d740 1 client.19518 shutdown 2024-09-12T02:14:00.531 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.524+0000 7fa74123d740 1 -- 172.21.15.16:0/646270721 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55aad78ce2e0 msgr2=0x55aad78d8110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:14:00.531 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.524+0000 7fa74123d740 1 --2- 172.21.15.16:0/646270721 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55aad78ce2e0 0x55aad78d8110 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7fa72c09b600 tx=0x7fa72c097a10 comp rx=0 tx=0).stop 2024-09-12T02:14:00.531 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.524+0000 7fa74123d740 1 -- 172.21.15.16:0/646270721 shutdown_connections 2024-09-12T02:14:00.531 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.524+0000 7fa74123d740 1 --2- 172.21.15.16:0/646270721 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55aad78e3890 0x55aad78e5ce0 crc :-1 s=READY pgs=368 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:00.531 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.524+0000 7fa74123d740 1 --2- 172.21.15.16:0/646270721 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aad78d3370 0x55aad78df150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:00.531 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.524+0000 7fa74123d740 1 --2- 172.21.15.16:0/646270721 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55aad78ce2e0 0x55aad78d8110 unknown :-1 s=CLOSED pgs=1714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:00.531 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.524+0000 7fa7177fe640 0 client.19518 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:14:00.531 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.524+0000 7fa74123d740 1 --2- 172.21.15.16:0/646270721 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55aad78cd9a0 0x55aad78d7bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:00.531 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.524+0000 7fa74123d740 1 -- 172.21.15.16:0/646270721 >> 172.21.15.16:0/646270721 conn(0x55aad78c9010 msgr2=0x55aad78ca820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:14:00.531 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.524+0000 7fa74123d740 1 -- 172.21.15.16:0/646270721 shutdown_connections 2024-09-12T02:14:00.531 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.524+0000 7fa74123d740 1 -- 172.21.15.16:0/646270721 wait complete. 2024-09-12T02:14:00.532 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.525+0000 7fa74123d740 20 client.19518 trim_cache size 0 max 16384 2024-09-12T02:14:00.541 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.539+0000 7fa74123d740 1 Processor -- start 2024-09-12T02:14:00.541 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.539+0000 7fa74123d740 1 -- start start 2024-09-12T02:14:00.541 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.539+0000 7fa74123d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aad78cd9a0 0x55aad78c80b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:00.541 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.539+0000 7fa74123d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55aad78ce2e0 0x55aad78df110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:00.541 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.539+0000 7fa74123d740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55aad78d3370 0x55aad78ea5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:00.541 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.539+0000 7fa74123d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa7381ad980 con 0x55aad78ce2e0 2024-09-12T02:14:00.541 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.539+0000 7fa74123d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa7381ab250 con 0x55aad78cd9a0 2024-09-12T02:14:00.542 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.539+0000 7fa74123d740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa7381a90a0 con 0x55aad78d3370 2024-09-12T02:14:00.542 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.539+0000 7fa7377fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55aad78ce2e0 0x55aad78df110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:00.542 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.539+0000 7fa7377fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55aad78ce2e0 0x55aad78df110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37154/0 (socket says 172.21.15.16:37154) 2024-09-12T02:14:00.542 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.539+0000 7fa7377fe640 1 -- 172.21.15.16:0/2834395821 learned_addr learned my addr 172.21.15.16:0/2834395821 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:14:00.542 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.539+0000 7fa737fff640 1 --2- 172.21.15.16:0/2834395821 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aad78cd9a0 0x55aad78c80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:00.542 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.540+0000 7fa73c961640 1 --2- 172.21.15.16:0/2834395821 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55aad78d3370 0x55aad78ea5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:00.542 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.540+0000 7fa7377fe640 1 -- 172.21.15.16:0/2834395821 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55aad78d3370 msgr2=0x55aad78ea5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:14:00.542 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.540+0000 7fa7377fe640 1 --2- 172.21.15.16:0/2834395821 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55aad78d3370 0x55aad78ea5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:00.542 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.540+0000 7fa7377fe640 1 -- 172.21.15.16:0/2834395821 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aad78cd9a0 msgr2=0x55aad78c80b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:14:00.542 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.540+0000 7fa7377fe640 1 --2- 172.21.15.16:0/2834395821 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aad78cd9a0 0x55aad78c80b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:00.542 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.540+0000 7fa7377fe640 1 -- 172.21.15.16:0/2834395821 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aad78c9ca0 con 0x55aad78ce2e0 2024-09-12T02:14:00.542 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.540+0000 7fa737fff640 1 --2- 172.21.15.16:0/2834395821 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aad78cd9a0 0x55aad78c80b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:14:00.542 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.540+0000 7fa73c961640 1 --2- 172.21.15.16:0/2834395821 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55aad78d3370 0x55aad78ea5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:14:00.543 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.540+0000 7fa7377fe640 1 --2- 172.21.15.16:0/2834395821 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55aad78ce2e0 0x55aad78df110 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7fa72c000c00 tx=0x7fa72c097250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d71cfd5704082d4a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:14:00.543 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.540+0000 7fa71dffb640 1 -- 172.21.15.16:0/2834395821 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa72c09bd70 con 0x55aad78ce2e0 2024-09-12T02:14:00.543 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.540+0000 7fa71dffb640 1 -- 172.21.15.16:0/2834395821 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa72c09fa20 con 0x55aad78ce2e0 2024-09-12T02:14:00.543 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.540+0000 7fa71dffb640 1 -- 172.21.15.16:0/2834395821 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa72c0afb90 con 0x55aad78ce2e0 2024-09-12T02:14:00.543 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.541+0000 7fa74123d740 1 -- 172.21.15.16:0/2834395821 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55aad78ce2e0 msgr2=0x55aad78df110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:14:00.543 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.541+0000 7fa74123d740 1 --2- 172.21.15.16:0/2834395821 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55aad78ce2e0 0x55aad78df110 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7fa72c000c00 tx=0x7fa72c097250 comp rx=0 tx=0).stop 2024-09-12T02:14:00.543 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.541+0000 7fa74123d740 1 -- 172.21.15.16:0/2834395821 shutdown_connections 2024-09-12T02:14:00.543 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.541+0000 7fa74123d740 1 --2- 172.21.15.16:0/2834395821 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55aad78d3370 0x55aad78ea5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:00.543 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.541+0000 7fa74123d740 1 --2- 172.21.15.16:0/2834395821 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55aad78ce2e0 0x55aad78df110 unknown :-1 s=CLOSED pgs=1715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:00.543 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.541+0000 7fa74123d740 1 --2- 172.21.15.16:0/2834395821 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aad78cd9a0 0x55aad78c80b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:00.543 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.541+0000 7fa74123d740 1 -- 172.21.15.16:0/2834395821 >> 172.21.15.16:0/2834395821 conn(0x55aad78c9010 msgr2=0x55aad78c79a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:14:00.543 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.541+0000 7fa74123d740 1 -- 172.21.15.16:0/2834395821 shutdown_connections 2024-09-12T02:14:00.543 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.541+0000 7fa74123d740 1 -- 172.21.15.16:0/2834395821 wait complete. 2024-09-12T02:14:00.544 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.541+0000 7fa74123d740 1 -- start start 2024-09-12T02:14:00.544 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.541+0000 7fa74123d740 1 Processor -- start 2024-09-12T02:14:00.547 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.541+0000 7fa74123d740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55aad78cd9a0 0x55aad78dc9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:00.547 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.542+0000 7fa74123d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55aad78ce2e0 0x55aad78dcf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:00.547 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.542+0000 7fa74123d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aad78d3370 0x55aad78ddec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:00.547 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.542+0000 7fa74123d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa7381a9710 con 0x55aad78ce2e0 2024-09-12T02:14:00.547 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.542+0000 7fa74123d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa7381aab80 con 0x55aad78d3370 2024-09-12T02:14:00.547 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.542+0000 7fa74123d740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa7381a9de0 con 0x55aad78cd9a0 2024-09-12T02:14:00.547 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.542+0000 7fa74123d740 10 client.0 _ll_register_callbacks cb 0x7fa72c0d23c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:14:00.547 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.542+0000 7fa7377fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55aad78ce2e0 0x55aad78dcf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:00.547 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.542+0000 7fa7377fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55aad78ce2e0 0x55aad78dcf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37168/0 (socket says 172.21.15.16:37168) 2024-09-12T02:14:00.548 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.542+0000 7fa7377fe640 1 -- 172.21.15.16:0/3063015387 learned_addr learned my addr 172.21.15.16:0/3063015387 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:14:00.548 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.542+0000 7fa73c961640 1 --2- 172.21.15.16:0/3063015387 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aad78d3370 0x55aad78ddec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:00.548 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.542+0000 7fa737fff640 1 --2- 172.21.15.16:0/3063015387 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55aad78cd9a0 0x55aad78dc9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:00.548 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.542+0000 7fa7377fe640 1 -- 172.21.15.16:0/3063015387 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55aad78cd9a0 msgr2=0x55aad78dc9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:14:00.548 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.542+0000 7fa7377fe640 1 --2- 172.21.15.16:0/3063015387 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55aad78cd9a0 0x55aad78dc9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:00.548 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.542+0000 7fa7377fe640 1 -- 172.21.15.16:0/3063015387 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aad78d3370 msgr2=0x55aad78ddec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:14:00.548 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.542+0000 7fa7377fe640 1 --2- 172.21.15.16:0/3063015387 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aad78d3370 0x55aad78ddec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:00.548 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.542+0000 7fa7377fe640 1 -- 172.21.15.16:0/3063015387 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aad78de4d0 con 0x55aad78ce2e0 2024-09-12T02:14:00.548 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.542+0000 7fa737fff640 1 --2- 172.21.15.16:0/3063015387 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55aad78cd9a0 0x55aad78dc9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:14:00.548 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.542+0000 7fa73c961640 1 --2- 172.21.15.16:0/3063015387 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aad78d3370 0x55aad78ddec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:14:00.548 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.542+0000 7fa7377fe640 1 --2- 172.21.15.16:0/3063015387 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55aad78ce2e0 0x55aad78dcf00 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7fa72c097ad0 tx=0x7fa72c003920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43ae3c95b95f4fbc server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:14:00.548 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.542+0000 7fa7177fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:14:00.548 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.542+0000 7fa7177fe640 1 -- 172.21.15.16:0/3063015387 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa72c09bd70 con 0x55aad78ce2e0 2024-09-12T02:14:00.549 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.542+0000 7fa7177fe640 1 -- 172.21.15.16:0/3063015387 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fa72c0973c0 con 0x55aad78ce2e0 2024-09-12T02:14:00.549 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.542+0000 7fa74123d740 1 -- 172.21.15.16:0/3063015387 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa72c09fa20 con 0x55aad78ce2e0 2024-09-12T02:14:00.549 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.542+0000 7fa7177fe640 1 -- 172.21.15.16:0/3063015387 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa72c0a42e0 con 0x55aad78ce2e0 2024-09-12T02:14:00.549 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.542+0000 7fa74123d740 10 client.19524 fetch_fsmap learned FSMap version 25 2024-09-12T02:14:00.549 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.542+0000 7fa74123d740 1 -- 172.21.15.16:0/3063015387 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55aad794e5c0 con 0x55aad78ce2e0 2024-09-12T02:14:00.549 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.543+0000 7fa7177fe640 1 -- 172.21.15.16:0/3063015387 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa72c09fa20 con 0x55aad78ce2e0 2024-09-12T02:14:00.549 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.543+0000 7fa7177fe640 1 -- 172.21.15.16:0/3063015387 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fa72c0b41a0 con 0x55aad78ce2e0 2024-09-12T02:14:00.549 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.543+0000 7fa74123d740 10 client.19524 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:14:00.549 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.543+0000 7fa74123d740 10 client.19524 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:14:00.549 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.543+0000 7fa74123d740 20 client.19524 populate_metadata read hostname 'smithi016' 2024-09-12T02:14:00.549 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.543+0000 7fa74123d740 1 --2- 172.21.15.16:0/3063015387 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55aad794ecd0 0x55aad79510c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:00.549 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.543+0000 7fa74123d740 4 client.19524 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:14:00.549 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.543+0000 7fa74123d740 1 -- 172.21.15.16:0/3063015387 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55aad7838d70 con 0x55aad794ecd0 2024-09-12T02:14:00.550 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.543+0000 7fa737fff640 1 --2- 172.21.15.16:0/3063015387 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55aad794ecd0 0x55aad79510c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:00.550 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.544+0000 7fa7177fe640 10 client.19524 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:14:00.550 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.544+0000 7fa737fff640 1 --2- 172.21.15.16:0/3063015387 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55aad794ecd0 0x55aad79510c0 crc :-1 s=READY pgs=369 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1cb8bdd5d68bc0f9 server_cookie=c9619ac5a86d2ca5 in_seq=0 out_seq=0 2024-09-12T02:14:00.554 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.553+0000 7fa7177fe640 1 -- 172.21.15.16:0/3063015387 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55aad7838d70 con 0x55aad794ecd0 2024-09-12T02:14:00.554 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.553+0000 7fa7177fe640 10 client.19524 handle_command_reply: tid=0 2024-09-12T02:14:00.554 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.553+0000 7fa74123d740 1 client.19524 shutdown 2024-09-12T02:14:00.554 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.554+0000 7fa74123d740 1 -- 172.21.15.16:0/3063015387 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55aad78ce2e0 msgr2=0x55aad78dcf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:14:00.554 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.554+0000 7fa74123d740 1 --2- 172.21.15.16:0/3063015387 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55aad78ce2e0 0x55aad78dcf00 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7fa72c097ad0 tx=0x7fa72c003920 comp rx=0 tx=0).stop 2024-09-12T02:14:00.554 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.554+0000 7fa74123d740 1 -- 172.21.15.16:0/3063015387 shutdown_connections 2024-09-12T02:14:00.554 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.554+0000 7fa74123d740 1 --2- 172.21.15.16:0/3063015387 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55aad794ecd0 0x55aad79510c0 crc :-1 s=READY pgs=369 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:00.554 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.554+0000 7fa74123d740 1 --2- 172.21.15.16:0/3063015387 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aad78d3370 0x55aad78ddec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:00.555 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.554+0000 7fa7177fe640 0 client.19524 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:14:00.555 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.554+0000 7fa74123d740 1 --2- 172.21.15.16:0/3063015387 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55aad78ce2e0 0x55aad78dcf00 unknown :-1 s=CLOSED pgs=1716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:00.555 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.554+0000 7fa74123d740 1 --2- 172.21.15.16:0/3063015387 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55aad78cd9a0 0x55aad78dc9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:00.555 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.554+0000 7fa74123d740 1 -- 172.21.15.16:0/3063015387 >> 172.21.15.16:0/3063015387 conn(0x55aad78c9010 msgr2=0x55aad78c81c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:14:00.555 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.554+0000 7fa74123d740 1 -- 172.21.15.16:0/3063015387 shutdown_connections 2024-09-12T02:14:00.555 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.554+0000 7fa74123d740 1 -- 172.21.15.16:0/3063015387 wait complete. 2024-09-12T02:14:00.555 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.554+0000 7fa74123d740 20 client.19524 trim_cache size 0 max 16384 2024-09-12T02:14:00.555 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-09-12T02:14:00.555 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-09-12T02:14:00.556 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-09-12T02:14:00.556 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-09-12T02:14:00.556 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.554+0000 7fa73e3c4640 1 -- 172.21.15.16:0/2889033676 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fa7200a2cb0 msgr2=0x7fa7200a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:14:00.556 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.554+0000 7fa73e3c4640 1 --2- 172.21.15.16:0/2889033676 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fa7200a2cb0 0x7fa7200a5170 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fa730005cc0 tx=0x7fa730031bb0 comp rx=0 tx=0).stop 2024-09-12T02:14:00.557 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.554+0000 7fa73e3c4640 1 -- 172.21.15.16:0/2889033676 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa738104f10 msgr2=0x7fa738100590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:14:00.557 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.554+0000 7fa73e3c4640 1 --2- 172.21.15.16:0/2889033676 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa738104f10 0x7fa738100590 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7fa72c00c970 tx=0x7fa72c00ce30 comp rx=0 tx=0).stop 2024-09-12T02:14:00.557 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.554+0000 7fa73e3c4640 1 -- 172.21.15.16:0/2889033676 shutdown_connections 2024-09-12T02:14:00.557 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.555+0000 7fa73e3c4640 1 --2- 172.21.15.16:0/2889033676 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fa7200a2cb0 0x7fa7200a5170 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:00.557 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.555+0000 7fa73e3c4640 1 --2- 172.21.15.16:0/2889033676 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa738107840 0x7fa738100ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:00.557 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.555+0000 7fa73e3c4640 1 --2- 172.21.15.16:0/2889033676 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa738104f10 0x7fa738100590 unknown :-1 s=CLOSED pgs=1712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:00.557 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.555+0000 7fa73e3c4640 1 --2- 172.21.15.16:0/2889033676 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa7381025e0 0x7fa738101f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:00.557 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.555+0000 7fa73e3c4640 1 -- 172.21.15.16:0/2889033676 >> 172.21.15.16:0/2889033676 conn(0x7fa7380fc1e0 msgr2=0x7fa738102e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:14:00.557 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.555+0000 7fa73e3c4640 1 -- 172.21.15.16:0/2889033676 shutdown_connections 2024-09-12T02:14:00.557 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:00.555+0000 7fa73e3c4640 1 -- 172.21.15.16:0/2889033676 wait complete. 2024-09-12T02:14:00.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:14:00 smithi184 ceph-mon[35593]: pgmap v697: 97 pgs: 97 active+clean; 2.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.8 KiB/s wr, 7 op/s 2024-09-12T02:14:02.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:14:02 smithi033 ceph-mon[35332]: pgmap v698: 97 pgs: 97 active+clean; 2.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-09-12T02:14:02.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:14:02 smithi016 ceph-mon[30947]: pgmap v698: 97 pgs: 97 active+clean; 2.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-09-12T02:14:02.629 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:14:02 smithi184 ceph-mon[35593]: pgmap v698: 97 pgs: 97 active+clean; 2.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-09-12T02:14:04.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:14:04 smithi033 ceph-mon[35332]: pgmap v699: 97 pgs: 97 active+clean; 2.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 2.1 KiB/s wr, 7 op/s 2024-09-12T02:14:04.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:14:04 smithi016 ceph-mon[30947]: pgmap v699: 97 pgs: 97 active+clean; 2.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 2.1 KiB/s wr, 7 op/s 2024-09-12T02:14:04.459 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:14:04.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:14:04 smithi184 ceph-mon[35593]: pgmap v699: 97 pgs: 97 active+clean; 2.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 2.1 KiB/s wr, 7 op/s 2024-09-12T02:14:04.730 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:14:05.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:05.241+0000 7f2f357d8640 1 -- 172.21.15.16:0/2370133846 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f30102d40 msgr2=0x7f2f30103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:14:05.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:05.241+0000 7f2f357d8640 1 --2- 172.21.15.16:0/2370133846 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f30102d40 0x7f2f30103120 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7f2f20009900 tx=0x7f2f2002a480 comp rx=0 tx=0).stop 2024-09-12T02:14:05.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:05.241+0000 7f2f357d8640 1 -- 172.21.15.16:0/2370133846 shutdown_connections 2024-09-12T02:14:05.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:05.241+0000 7f2f357d8640 1 --2- 172.21.15.16:0/2370133846 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f3010c170 0x7f2f3010e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:05.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:05.241+0000 7f2f357d8640 1 --2- 172.21.15.16:0/2370133846 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2f30103660 0x7f2f3010bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:05.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:05.241+0000 7f2f357d8640 1 --2- 172.21.15.16:0/2370133846 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f30102d40 0x7f2f30103120 unknown :-1 s=CLOSED pgs=1717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:05.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:05.241+0000 7f2f357d8640 1 -- 172.21.15.16:0/2370133846 >> 172.21.15.16:0/2370133846 conn(0x7f2f300fc7a0 msgr2=0x7f2f300febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:14:05.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:05.242+0000 7f2f357d8640 1 -- 172.21.15.16:0/2370133846 shutdown_connections 2024-09-12T02:14:05.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:05.242+0000 7f2f357d8640 1 -- 172.21.15.16:0/2370133846 wait complete. 2024-09-12T02:14:05.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:05.242+0000 7f2f357d8640 1 Processor -- start 2024-09-12T02:14:05.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:05.242+0000 7f2f357d8640 1 -- start start 2024-09-12T02:14:05.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:05.242+0000 7f2f357d8640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2f30102d40 0x7f2f301009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:05.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:05.242+0000 7f2f357d8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f30103660 0x7f2f30100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:05.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:05.242+0000 7f2f357d8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f3010c170 0x7f2f30101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:05.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:05.242+0000 7f2f357d8640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2f30111f70 con 0x7f2f3010c170 2024-09-12T02:14:05.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:05.242+0000 7f2f357d8640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2f30111df0 con 0x7f2f30103660 2024-09-12T02:14:05.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:05.242+0000 7f2f357d8640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f2f301120f0 con 0x7f2f30102d40 2024-09-12T02:14:05.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:05.243+0000 7f2f2f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f3010c170 0x7f2f30101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:05.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:05.243+0000 7f2f2f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f3010c170 0x7f2f30101420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40002/0 (socket says 172.21.15.16:40002) 2024-09-12T02:14:05.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:05.243+0000 7f2f2f7fe640 1 -- 172.21.15.16:0/3090209994 learned_addr learned my addr 172.21.15.16:0/3090209994 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:14:05.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:05.243+0000 7f2f27fff640 1 --2- 172.21.15.16:0/3090209994 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f30103660 0x7f2f30100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:05.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:05.243+0000 7f2f2effd640 1 --2- 172.21.15.16:0/3090209994 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2f30102d40 0x7f2f301009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:05.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:05.243+0000 7f2f2f7fe640 1 -- 172.21.15.16:0/3090209994 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2f30102d40 msgr2=0x7f2f301009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:14:05.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:05.243+0000 7f2f2f7fe640 1 --2- 172.21.15.16:0/3090209994 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2f30102d40 0x7f2f301009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:05.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:05.243+0000 7f2f2f7fe640 1 -- 172.21.15.16:0/3090209994 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f30103660 msgr2=0x7f2f30100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:14:05.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:05.243+0000 7f2f2f7fe640 1 --2- 172.21.15.16:0/3090209994 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f30103660 0x7f2f30100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:05.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:05.243+0000 7f2f2f7fe640 1 -- 172.21.15.16:0/3090209994 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f301a69c0 con 0x7f2f3010c170 2024-09-12T02:14:05.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:05.243+0000 7f2f2f7fe640 1 --2- 172.21.15.16:0/3090209994 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f3010c170 0x7f2f30101420 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7f2f2800cad0 tx=0x7f2f2800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:14:05.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:05.243+0000 7f2f2cff9640 1 -- 172.21.15.16:0/3090209994 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f2800ae20 con 0x7f2f3010c170 2024-09-12T02:14:05.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:05.243+0000 7f2f357d8640 1 -- 172.21.15.16:0/3090209994 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f301a6cb0 con 0x7f2f3010c170 2024-09-12T02:14:05.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:05.243+0000 7f2f2cff9640 1 -- 172.21.15.16:0/3090209994 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f28005020 con 0x7f2f3010c170 2024-09-12T02:14:05.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:05.243+0000 7f2f2cff9640 1 -- 172.21.15.16:0/3090209994 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f2800f6f0 con 0x7f2f3010c170 2024-09-12T02:14:05.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:05.244+0000 7f2f357d8640 1 -- 172.21.15.16:0/3090209994 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f301a71a0 con 0x7f2f3010c170 2024-09-12T02:14:05.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:05.244+0000 7f2f357d8640 1 -- 172.21.15.16:0/3090209994 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f30103120 con 0x7f2f3010c170 2024-09-12T02:14:05.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:05.246+0000 7f2f2cff9640 1 -- 172.21.15.16:0/3090209994 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f2f280051d0 con 0x7f2f3010c170 2024-09-12T02:14:05.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:05.246+0000 7f2f2cff9640 1 --2- 172.21.15.16:0/3090209994 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f2f1c0a2b80 0x7f2f1c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:05.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:05.246+0000 7f2f2effd640 1 --2- 172.21.15.16:0/3090209994 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f2f1c0a2b80 0x7f2f1c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:05.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:05.247+0000 7f2f2cff9640 1 -- 172.21.15.16:0/3090209994 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f2f280cf350 con 0x7f2f3010c170 2024-09-12T02:14:05.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:05.247+0000 7f2f2effd640 1 --2- 172.21.15.16:0/3090209994 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f2f1c0a2b80 0x7f2f1c0a5040 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f2f200096c0 tx=0x7f2f20031490 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:14:05.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:05.250+0000 7f2f2cff9640 1 -- 172.21.15.16:0/3090209994 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2f28014030 con 0x7f2f3010c170 2024-09-12T02:14:05.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:05.464+0000 7f2f357d8640 1 -- 172.21.15.16:0/3090209994 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2f30101ed0 con 0x7f2f3010c170 2024-09-12T02:14:05.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:05.465+0000 7f2f2cff9640 1 -- 172.21.15.16:0/3090209994 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11911 (secure 0 0 0) 0x7f2f30101ed0 con 0x7f2f3010c170 2024-09-12T02:14:05.465 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T02:14:05.466 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":25,"btime":"2024-09-12T02:01:37:192021+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3279123279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3279123279},{"type":"v1","addr":"172.21.15.16:6835","nonce":3279123279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4119452382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4119452382},{"type":"v1","addr":"172.21.15.16:6837","nonce":4119452382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/3263823165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3263823165},{"type":"v1","addr":"172.21.15.16:6839","nonce":3263823165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/1510290732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":1510290732},{"type":"v1","addr":"172.21.15.16:6841","nonce":1510290732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/279674603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":279674603},{"type":"v1","addr":"172.21.15.33:6833","nonce":279674603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3555728028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3555728028},{"type":"v1","addr":"172.21.15.33:6835","nonce":3555728028}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T01:59:56.117608+0000","modified":"2024-09-12T02:01:36.294760+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24482,"mds_1":24481,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6833/1563374106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1563374106},{"type":"v1","addr":"172.21.15.184:6833","nonce":1563374106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6837/1892833868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1892833868},{"type":"v1","addr":"172.21.15.33:6837","nonce":1892833868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/1457679686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1457679686},{"type":"v1","addr":"172.21.15.184:6835","nonce":1457679686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.184:6837/1793393087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":1793393087},{"type":"v1","addr":"172.21.15.184:6837","nonce":1793393087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.184:6839/4203347941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":4203347941},{"type":"v1","addr":"172.21.15.184:6839","nonce":4203347941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.33:6839/4136670204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":4136670204},{"type":"v1","addr":"172.21.15.33:6839","nonce":4136670204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24481,24482,24469]},"id":1}]} 2024-09-12T02:14:05.466 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 25 2024-09-12T02:14:05.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:05.467+0000 7f2f357d8640 1 -- 172.21.15.16:0/3090209994 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f2f1c0a2b80 msgr2=0x7f2f1c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:14:05.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:05.467+0000 7f2f357d8640 1 --2- 172.21.15.16:0/3090209994 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f2f1c0a2b80 0x7f2f1c0a5040 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f2f200096c0 tx=0x7f2f20031490 comp rx=0 tx=0).stop 2024-09-12T02:14:05.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:05.467+0000 7f2f357d8640 1 -- 172.21.15.16:0/3090209994 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f3010c170 msgr2=0x7f2f30101420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:14:05.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:05.467+0000 7f2f357d8640 1 --2- 172.21.15.16:0/3090209994 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f3010c170 0x7f2f30101420 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7f2f2800cad0 tx=0x7f2f2800cf90 comp rx=0 tx=0).stop 2024-09-12T02:14:05.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:05.467+0000 7f2f357d8640 1 -- 172.21.15.16:0/3090209994 shutdown_connections 2024-09-12T02:14:05.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:05.467+0000 7f2f357d8640 1 --2- 172.21.15.16:0/3090209994 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f2f1c0a2b80 0x7f2f1c0a5040 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:05.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:05.467+0000 7f2f357d8640 1 --2- 172.21.15.16:0/3090209994 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f3010c170 0x7f2f30101420 unknown :-1 s=CLOSED pgs=1718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:05.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:05.467+0000 7f2f357d8640 1 --2- 172.21.15.16:0/3090209994 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f30103660 0x7f2f30100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:05.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:05.467+0000 7f2f357d8640 1 --2- 172.21.15.16:0/3090209994 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2f30102d40 0x7f2f301009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:05.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:05.467+0000 7f2f357d8640 1 -- 172.21.15.16:0/3090209994 >> 172.21.15.16:0/3090209994 conn(0x7f2f300fc7a0 msgr2=0x7f2f300fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:14:05.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:05.467+0000 7f2f357d8640 1 -- 172.21.15.16:0/3090209994 shutdown_connections 2024-09-12T02:14:05.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:05.467+0000 7f2f357d8640 1 -- 172.21.15.16:0/3090209994 wait complete. 2024-09-12T02:14:06.240 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24482 quiesce db --set-id 336cf4e6 --release --await' 2024-09-12T02:14:06.240 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24482 quiesce db --set-id 336cf4e6 --release --await 2024-09-12T02:14:06.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:14:06 smithi033 ceph-mon[35332]: pgmap v700: 97 pgs: 97 active+clean; 2.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-09-12T02:14:06.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:14:06 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/3090209994' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:14:06.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:14:06 smithi016 ceph-mon[30947]: pgmap v700: 97 pgs: 97 active+clean; 2.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-09-12T02:14:06.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:14:06 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/3090209994' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:14:06.502 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:14:06.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:14:06 smithi184 ceph-mon[35593]: pgmap v700: 97 pgs: 97 active+clean; 2.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-09-12T02:14:06.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:14:06 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/3090209994' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:14:07.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.008+0000 7fda8fc92640 1 -- 172.21.15.16:0/1774635733 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda88074590 msgr2=0x7fda88074970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:14:07.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.008+0000 7fda8fc92640 1 --2- 172.21.15.16:0/1774635733 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda88074590 0x7fda88074970 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7fda84009900 tx=0x7fda8402a480 comp rx=0 tx=0).stop 2024-09-12T02:14:07.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.008+0000 7fda8fc92640 1 -- 172.21.15.16:0/1774635733 shutdown_connections 2024-09-12T02:14:07.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.008+0000 7fda8fc92640 1 --2- 172.21.15.16:0/1774635733 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fda88112b90 0x7fda88114f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:07.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.008+0000 7fda8fc92640 1 --2- 172.21.15.16:0/1774635733 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fda88074eb0 0x7fda88112650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:07.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.008+0000 7fda8fc92640 1 --2- 172.21.15.16:0/1774635733 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda88074590 0x7fda88074970 unknown :-1 s=CLOSED pgs=1719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:07.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.008+0000 7fda8fc92640 1 -- 172.21.15.16:0/1774635733 >> 172.21.15.16:0/1774635733 conn(0x7fda880fd8c0 msgr2=0x7fda880ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:14:07.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.008+0000 7fda8fc92640 1 -- 172.21.15.16:0/1774635733 shutdown_connections 2024-09-12T02:14:07.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.008+0000 7fda8fc92640 1 -- 172.21.15.16:0/1774635733 wait complete. 2024-09-12T02:14:07.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.009+0000 7fda8fc92640 1 Processor -- start 2024-09-12T02:14:07.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.009+0000 7fda8fc92640 1 -- start start 2024-09-12T02:14:07.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.009+0000 7fda8fc92640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fda88074590 0x7fda881a46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:07.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.009+0000 7fda8fc92640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda88074eb0 0x7fda881a4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:07.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.009+0000 7fda8fc92640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fda88112b90 0x7fda8819e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:07.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.009+0000 7fda8fc92640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fda88118570 con 0x7fda88074eb0 2024-09-12T02:14:07.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.009+0000 7fda8fc92640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fda881183f0 con 0x7fda88074590 2024-09-12T02:14:07.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.009+0000 7fda8fc92640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fda881186f0 con 0x7fda88112b90 2024-09-12T02:14:07.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.009+0000 7fda8d22d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda88074eb0 0x7fda881a4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:07.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.009+0000 7fda8da2e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fda88074590 0x7fda881a46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:07.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.009+0000 7fda8d22d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda88074eb0 0x7fda881a4c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40014/0 (socket says 172.21.15.16:40014) 2024-09-12T02:14:07.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.009+0000 7fda8d22d640 1 -- 172.21.15.16:0/4048885091 learned_addr learned my addr 172.21.15.16:0/4048885091 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:14:07.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.010+0000 7fda8e22f640 1 --2- 172.21.15.16:0/4048885091 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fda88112b90 0x7fda8819e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:07.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.010+0000 7fda8d22d640 1 -- 172.21.15.16:0/4048885091 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fda88112b90 msgr2=0x7fda8819e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:14:07.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.010+0000 7fda8d22d640 1 --2- 172.21.15.16:0/4048885091 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fda88112b90 0x7fda8819e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:07.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.010+0000 7fda8d22d640 1 -- 172.21.15.16:0/4048885091 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fda88074590 msgr2=0x7fda881a46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:14:07.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.010+0000 7fda8d22d640 1 --2- 172.21.15.16:0/4048885091 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fda88074590 0x7fda881a46d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:07.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.010+0000 7fda8d22d640 1 -- 172.21.15.16:0/4048885091 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda8819efc0 con 0x7fda88074eb0 2024-09-12T02:14:07.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.010+0000 7fda8e22f640 1 --2- 172.21.15.16:0/4048885091 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fda88112b90 0x7fda8819e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:14:07.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.010+0000 7fda8d22d640 1 --2- 172.21.15.16:0/4048885091 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda88074eb0 0x7fda881a4c10 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7fda7800d950 tx=0x7fda7800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:14:07.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.010+0000 7fda7effd640 1 -- 172.21.15.16:0/4048885091 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda78017070 con 0x7fda88074eb0 2024-09-12T02:14:07.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.010+0000 7fda7effd640 1 -- 172.21.15.16:0/4048885091 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda78009c30 con 0x7fda88074eb0 2024-09-12T02:14:07.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.010+0000 7fda8fc92640 1 -- 172.21.15.16:0/4048885091 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda8819f2b0 con 0x7fda88074eb0 2024-09-12T02:14:07.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.010+0000 7fda7effd640 1 -- 172.21.15.16:0/4048885091 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda7800f6b0 con 0x7fda88074eb0 2024-09-12T02:14:07.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.010+0000 7fda8fc92640 1 -- 172.21.15.16:0/4048885091 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda881ab5a0 con 0x7fda88074eb0 2024-09-12T02:14:07.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.012+0000 7fda7effd640 1 -- 172.21.15.16:0/4048885091 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fda780109d0 con 0x7fda88074eb0 2024-09-12T02:14:07.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.013+0000 7fda9206b740 1 Processor -- start 2024-09-12T02:14:07.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.013+0000 7fda9206b740 1 -- start start 2024-09-12T02:14:07.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.013+0000 7fda9206b740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5648243f7060 0x5648243f7440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:07.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.013+0000 7fda9206b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5648243f7980 0x564824401d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:07.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.013+0000 7fda9206b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564824402240 0x564824404630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:07.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.013+0000 7fda9206b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56482435c620 con 0x564824402240 2024-09-12T02:14:07.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.013+0000 7fda9206b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x564824303e50 con 0x5648243f7980 2024-09-12T02:14:07.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.013+0000 7fda9206b740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5648243585e0 con 0x5648243f7060 2024-09-12T02:14:07.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.013+0000 7fda7effd640 1 --2- 172.21.15.16:0/4048885091 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fda740a2b80 0x7fda740a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:07.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.014+0000 7fda8d22d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564824402240 0x564824404630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:07.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.014+0000 7fda8e22f640 1 --2- 172.21.15.16:0/4048885091 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fda740a2b80 0x7fda740a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:07.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.014+0000 7fda8d22d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564824402240 0x564824404630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40020/0 (socket says 172.21.15.16:40020) 2024-09-12T02:14:07.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.014+0000 7fda8d22d640 1 -- 172.21.15.16:0/1515447464 learned_addr learned my addr 172.21.15.16:0/1515447464 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:14:07.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.014+0000 7fda8da2e640 1 --2- 172.21.15.16:0/1515447464 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5648243f7980 0x564824401d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:07.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.014+0000 7fda8e22f640 1 --2- 172.21.15.16:0/1515447464 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5648243f7060 0x5648243f7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:07.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.014+0000 7fda7effd640 1 -- 172.21.15.16:0/4048885091 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fda780cecd0 con 0x7fda88074eb0 2024-09-12T02:14:07.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.014+0000 7fda8d22d640 1 -- 172.21.15.16:0/1515447464 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5648243f7060 msgr2=0x5648243f7440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:14:07.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.014+0000 7fda8d22d640 1 --2- 172.21.15.16:0/1515447464 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5648243f7060 0x5648243f7440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:07.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.014+0000 7fda8d22d640 1 -- 172.21.15.16:0/1515447464 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5648243f7980 msgr2=0x564824401d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:14:07.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.014+0000 7fda8d22d640 1 --2- 172.21.15.16:0/1515447464 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5648243f7980 0x564824401d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:07.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.014+0000 7fda8d22d640 1 -- 172.21.15.16:0/1515447464 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648242f4d40 con 0x564824402240 2024-09-12T02:14:07.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.014+0000 7fda8da2e640 1 --2- 172.21.15.16:0/1515447464 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5648243f7980 0x564824401d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:14:07.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.014+0000 7fda8e22f640 1 --2- 172.21.15.16:0/4048885091 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fda740a2b80 0x7fda740a5040 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fda80005c40 tx=0x7fda8000acb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:14:07.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.014+0000 7fda8d22d640 1 --2- 172.21.15.16:0/1515447464 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564824402240 0x564824404630 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7fda7809b380 tx=0x7fda78066650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db22444b60ddd197 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:14:07.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.014+0000 7fda73fff640 1 -- 172.21.15.16:0/1515447464 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda78020070 con 0x564824402240 2024-09-12T02:14:07.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.014+0000 7fda73fff640 1 -- 172.21.15.16:0/1515447464 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda7809bc30 con 0x564824402240 2024-09-12T02:14:07.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.014+0000 7fda73fff640 1 -- 172.21.15.16:0/1515447464 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda780bb070 con 0x564824402240 2024-09-12T02:14:07.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.015+0000 7fda9206b740 1 -- 172.21.15.16:0/1515447464 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564824402240 msgr2=0x564824404630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:14:07.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.015+0000 7fda9206b740 1 --2- 172.21.15.16:0/1515447464 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564824402240 0x564824404630 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7fda7809b380 tx=0x7fda78066650 comp rx=0 tx=0).stop 2024-09-12T02:14:07.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.015+0000 7fda9206b740 1 -- 172.21.15.16:0/1515447464 shutdown_connections 2024-09-12T02:14:07.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.015+0000 7fda9206b740 1 --2- 172.21.15.16:0/1515447464 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564824402240 0x564824404630 unknown :-1 s=CLOSED pgs=1721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:07.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.015+0000 7fda9206b740 1 --2- 172.21.15.16:0/1515447464 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5648243f7980 0x564824401d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:07.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.015+0000 7fda9206b740 1 --2- 172.21.15.16:0/1515447464 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5648243f7060 0x5648243f7440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:07.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.015+0000 7fda9206b740 1 -- 172.21.15.16:0/1515447464 >> 172.21.15.16:0/1515447464 conn(0x5648243f35f0 msgr2=0x5648243f39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:14:07.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.015+0000 7fda9206b740 1 -- 172.21.15.16:0/1515447464 shutdown_connections 2024-09-12T02:14:07.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.015+0000 7fda9206b740 1 -- 172.21.15.16:0/1515447464 wait complete. 2024-09-12T02:14:07.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.016+0000 7fda9206b740 1 -- start start 2024-09-12T02:14:07.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.016+0000 7fda9206b740 1 Processor -- start 2024-09-12T02:14:07.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.016+0000 7fda9206b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5648243f7060 0x5648243ff140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:07.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.016+0000 7fda9206b740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5648243f7980 0x5648243ff680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:07.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.016+0000 7fda9206b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564824402240 0x5648243ffbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:07.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.016+0000 7fda9206b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x564824404db0 con 0x5648243f7060 2024-09-12T02:14:07.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.016+0000 7fda9206b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5648242e4f10 con 0x564824402240 2024-09-12T02:14:07.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.016+0000 7fda9206b740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x564824404f30 con 0x5648243f7980 2024-09-12T02:14:07.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.016+0000 7fda9206b740 10 client.0 _ll_register_callbacks cb 0x564824308db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:14:07.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.016+0000 7fda8da2e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5648243f7060 0x5648243ff140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:07.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.016+0000 7fda8da2e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5648243f7060 0x5648243ff140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40032/0 (socket says 172.21.15.16:40032) 2024-09-12T02:14:07.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.016+0000 7fda8da2e640 1 -- 172.21.15.16:0/82785617 learned_addr learned my addr 172.21.15.16:0/82785617 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:14:07.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.016+0000 7fda8d22d640 1 --2- 172.21.15.16:0/82785617 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5648243f7980 0x5648243ff680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:07.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.017+0000 7fda8e22f640 1 --2- 172.21.15.16:0/82785617 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564824402240 0x5648243ffbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:07.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.017+0000 7fda8da2e640 1 -- 172.21.15.16:0/82785617 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5648243f7980 msgr2=0x5648243ff680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:14:07.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.017+0000 7fda8da2e640 1 --2- 172.21.15.16:0/82785617 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5648243f7980 0x5648243ff680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:07.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.017+0000 7fda8da2e640 1 -- 172.21.15.16:0/82785617 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564824402240 msgr2=0x5648243ffbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:14:07.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.017+0000 7fda8da2e640 1 --2- 172.21.15.16:0/82785617 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564824402240 0x5648243ffbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:07.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.017+0000 7fda8da2e640 1 -- 172.21.15.16:0/82785617 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56482406b580 con 0x5648243f7060 2024-09-12T02:14:07.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.017+0000 7fda8e22f640 1 --2- 172.21.15.16:0/82785617 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564824402240 0x5648243ffbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:14:07.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.017+0000 7fda8da2e640 1 --2- 172.21.15.16:0/82785617 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5648243f7060 0x5648243ff140 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7fda84009230 tx=0x7fda84004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f49d8df05530e612 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:14:07.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.017+0000 7fda717fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:14:07.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.017+0000 7fda717fa640 1 -- 172.21.15.16:0/82785617 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda84035040 con 0x5648243f7060 2024-09-12T02:14:07.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.017+0000 7fda717fa640 1 -- 172.21.15.16:0/82785617 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fda84005020 con 0x5648243f7060 2024-09-12T02:14:07.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.017+0000 7fda9206b740 1 -- 172.21.15.16:0/82785617 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fda7809bc30 con 0x5648243f7060 2024-09-12T02:14:07.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.017+0000 7fda717fa640 1 -- 172.21.15.16:0/82785617 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fda84005220 con 0x5648243f7060 2024-09-12T02:14:07.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.017+0000 7fda9206b740 10 client.19542 fetch_fsmap learned FSMap version 25 2024-09-12T02:14:07.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.017+0000 7fda9206b740 1 -- 172.21.15.16:0/82785617 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x564824135960 con 0x5648243f7060 2024-09-12T02:14:07.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.018+0000 7fda717fa640 1 -- 172.21.15.16:0/82785617 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda7809bc30 con 0x5648243f7060 2024-09-12T02:14:07.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.018+0000 7fda717fa640 1 -- 172.21.15.16:0/82785617 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fda84003680 con 0x5648243f7060 2024-09-12T02:14:07.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.018+0000 7fda9206b740 10 client.19542 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:14:07.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.018+0000 7fda9206b740 10 client.19542 resolve_mds: validated gid 24482 aka daemon mds.b 2024-09-12T02:14:07.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.018+0000 7fda9206b740 20 client.19542 populate_metadata read hostname 'smithi016' 2024-09-12T02:14:07.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.018+0000 7fda9206b740 1 --2- 172.21.15.16:0/82785617 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x564824411d60 0x564824414150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:07.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.018+0000 7fda9206b740 4 client.19542 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:14:07.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.018+0000 7fda9206b740 1 -- 172.21.15.16:0/82785617 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564824369b90 con 0x564824411d60 2024-09-12T02:14:07.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.018+0000 7fda8d22d640 1 --2- 172.21.15.16:0/82785617 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x564824411d60 0x564824414150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:07.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.018+0000 7fda8d22d640 1 --2- 172.21.15.16:0/82785617 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x564824411d60 0x564824414150 crc :-1 s=READY pgs=370 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f8d026f6701f381c server_cookie=b6a7f135bcfd2429 in_seq=0 out_seq=0 2024-09-12T02:14:07.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.018+0000 7fda717fa640 10 client.19542 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:14:07.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.019+0000 7fda717fa640 1 -- 172.21.15.16:0/82785617 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564824369b90 con 0x564824411d60 2024-09-12T02:14:07.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.019+0000 7fda717fa640 10 client.19542 handle_command_reply: tid=0 2024-09-12T02:14:07.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.019+0000 7fda9206b740 1 client.19542 shutdown 2024-09-12T02:14:07.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.019+0000 7fda9206b740 1 -- 172.21.15.16:0/82785617 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5648243f7060 msgr2=0x5648243ff140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:14:07.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.019+0000 7fda9206b740 1 --2- 172.21.15.16:0/82785617 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5648243f7060 0x5648243ff140 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7fda84009230 tx=0x7fda84004040 comp rx=0 tx=0).stop 2024-09-12T02:14:07.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.019+0000 7fda9206b740 1 -- 172.21.15.16:0/82785617 shutdown_connections 2024-09-12T02:14:07.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.019+0000 7fda9206b740 1 --2- 172.21.15.16:0/82785617 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x564824411d60 0x564824414150 crc :-1 s=READY pgs=370 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:07.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.019+0000 7fda9206b740 1 --2- 172.21.15.16:0/82785617 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564824402240 0x5648243ffbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:07.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.019+0000 7fda9206b740 1 --2- 172.21.15.16:0/82785617 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5648243f7980 0x5648243ff680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:07.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.019+0000 7fda9206b740 1 --2- 172.21.15.16:0/82785617 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5648243f7060 0x5648243ff140 unknown :-1 s=CLOSED pgs=1722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:07.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.019+0000 7fda717fa640 0 client.19542 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:14:07.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.019+0000 7fda9206b740 1 -- 172.21.15.16:0/82785617 >> 172.21.15.16:0/82785617 conn(0x5648243f35f0 msgr2=0x5648243f9320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:14:07.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.020+0000 7fda9206b740 1 -- 172.21.15.16:0/82785617 shutdown_connections 2024-09-12T02:14:07.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.020+0000 7fda9206b740 1 -- 172.21.15.16:0/82785617 wait complete. 2024-09-12T02:14:07.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.020+0000 7fda9206b740 20 client.19542 trim_cache size 0 max 16384 2024-09-12T02:14:07.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.037+0000 7fda9206b740 1 Processor -- start 2024-09-12T02:14:07.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.037+0000 7fda9206b740 1 -- start start 2024-09-12T02:14:07.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.037+0000 7fda9206b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5648243f7060 0x5648243f39d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:07.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.037+0000 7fda9206b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5648243f7980 0x5648243f91c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:07.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.037+0000 7fda9206b740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564824402240 0x564824419a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:07.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.037+0000 7fda9206b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fda881b6390 con 0x5648243f7980 2024-09-12T02:14:07.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.037+0000 7fda9206b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fda881b3c60 con 0x5648243f7060 2024-09-12T02:14:07.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.037+0000 7fda9206b740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fda881b1ad0 con 0x564824402240 2024-09-12T02:14:07.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.037+0000 7fda8d22d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5648243f7980 0x5648243f91c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:07.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.037+0000 7fda8d22d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5648243f7980 0x5648243f91c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40038/0 (socket says 172.21.15.16:40038) 2024-09-12T02:14:07.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.037+0000 7fda8d22d640 1 -- 172.21.15.16:0/4226815678 learned_addr learned my addr 172.21.15.16:0/4226815678 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:14:07.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.037+0000 7fda8e22f640 1 --2- 172.21.15.16:0/4226815678 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564824402240 0x564824419a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:07.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.037+0000 7fda8da2e640 1 --2- 172.21.15.16:0/4226815678 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5648243f7060 0x5648243f39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:07.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.037+0000 7fda8d22d640 1 -- 172.21.15.16:0/4226815678 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564824402240 msgr2=0x564824419a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:14:07.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.037+0000 7fda8d22d640 1 --2- 172.21.15.16:0/4226815678 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564824402240 0x564824419a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:07.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.037+0000 7fda8d22d640 1 -- 172.21.15.16:0/4226815678 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5648243f7060 msgr2=0x5648243f39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:14:07.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.037+0000 7fda8d22d640 1 --2- 172.21.15.16:0/4226815678 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5648243f7060 0x5648243f39d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:07.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.037+0000 7fda8d22d640 1 -- 172.21.15.16:0/4226815678 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564824401080 con 0x5648243f7980 2024-09-12T02:14:07.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.037+0000 7fda8da2e640 1 --2- 172.21.15.16:0/4226815678 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5648243f7060 0x5648243f39d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:14:07.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.037+0000 7fda8e22f640 1 --2- 172.21.15.16:0/4226815678 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564824402240 0x564824419a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:14:07.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.038+0000 7fda8d22d640 1 --2- 172.21.15.16:0/4226815678 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5648243f7980 0x5648243f91c0 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7fda78010fd0 tx=0x7fda78086040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5433e86de6f2d6c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:14:07.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.038+0000 7fda737fe640 1 -- 172.21.15.16:0/4226815678 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda78020040 con 0x5648243f7980 2024-09-12T02:14:07.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.038+0000 7fda737fe640 1 -- 172.21.15.16:0/4226815678 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda7809bc30 con 0x5648243f7980 2024-09-12T02:14:07.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.038+0000 7fda737fe640 1 -- 172.21.15.16:0/4226815678 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda780bb040 con 0x5648243f7980 2024-09-12T02:14:07.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.038+0000 7fda9206b740 1 -- 172.21.15.16:0/4226815678 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5648243f7980 msgr2=0x5648243f91c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:14:07.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.038+0000 7fda9206b740 1 --2- 172.21.15.16:0/4226815678 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5648243f7980 0x5648243f91c0 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7fda78010fd0 tx=0x7fda78086040 comp rx=0 tx=0).stop 2024-09-12T02:14:07.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.038+0000 7fda9206b740 1 -- 172.21.15.16:0/4226815678 shutdown_connections 2024-09-12T02:14:07.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.038+0000 7fda9206b740 1 --2- 172.21.15.16:0/4226815678 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564824402240 0x564824419a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:07.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.038+0000 7fda9206b740 1 --2- 172.21.15.16:0/4226815678 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5648243f7980 0x5648243f91c0 unknown :-1 s=CLOSED pgs=1723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:07.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.038+0000 7fda9206b740 1 --2- 172.21.15.16:0/4226815678 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5648243f7060 0x5648243f39d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:07.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.038+0000 7fda9206b740 1 -- 172.21.15.16:0/4226815678 >> 172.21.15.16:0/4226815678 conn(0x5648243f35f0 msgr2=0x5648243f8ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:14:07.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.038+0000 7fda9206b740 1 -- 172.21.15.16:0/4226815678 shutdown_connections 2024-09-12T02:14:07.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.038+0000 7fda9206b740 1 -- 172.21.15.16:0/4226815678 wait complete. 2024-09-12T02:14:07.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.039+0000 7fda9206b740 1 -- start start 2024-09-12T02:14:07.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.039+0000 7fda9206b740 1 Processor -- start 2024-09-12T02:14:07.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.039+0000 7fda9206b740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5648243f7060 0x564824406560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:07.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.039+0000 7fda9206b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5648243f7980 0x564824406aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:07.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.039+0000 7fda9206b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564824402240 0x56482447b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:07.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.039+0000 7fda9206b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fda881b2120 con 0x564824402240 2024-09-12T02:14:07.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.039+0000 7fda9206b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fda881b3590 con 0x5648243f7980 2024-09-12T02:14:07.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.039+0000 7fda9206b740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fda881b27f0 con 0x5648243f7060 2024-09-12T02:14:07.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.040+0000 7fda8e22f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564824402240 0x56482447b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:07.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.040+0000 7fda9206b740 10 client.0 _ll_register_callbacks cb 0x7fda84031830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:14:07.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.040+0000 7fda8da2e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5648243f7060 0x564824406560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:07.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.040+0000 7fda8e22f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564824402240 0x56482447b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40054/0 (socket says 172.21.15.16:40054) 2024-09-12T02:14:07.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.040+0000 7fda8e22f640 1 -- 172.21.15.16:0/4176792494 learned_addr learned my addr 172.21.15.16:0/4176792494 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:14:07.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.040+0000 7fda8d22d640 1 --2- 172.21.15.16:0/4176792494 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5648243f7980 0x564824406aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:07.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.040+0000 7fda8e22f640 1 -- 172.21.15.16:0/4176792494 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5648243f7060 msgr2=0x564824406560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:14:07.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.040+0000 7fda8e22f640 1 --2- 172.21.15.16:0/4176792494 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5648243f7060 0x564824406560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:07.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.040+0000 7fda8e22f640 1 -- 172.21.15.16:0/4176792494 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5648243f7980 msgr2=0x564824406aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:14:07.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.040+0000 7fda8e22f640 1 --2- 172.21.15.16:0/4176792494 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5648243f7980 0x564824406aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:07.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.040+0000 7fda8e22f640 1 -- 172.21.15.16:0/4176792494 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5648243f77b0 con 0x564824402240 2024-09-12T02:14:07.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.040+0000 7fda8e22f640 1 --2- 172.21.15.16:0/4176792494 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564824402240 0x56482447b9d0 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7fda80010d20 tx=0x7fda8001a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=315f3d7201fc5f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:14:07.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.040+0000 7fda717fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:14:07.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.040+0000 7fda717fa640 1 -- 172.21.15.16:0/4176792494 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda800047c0 con 0x564824402240 2024-09-12T02:14:07.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.040+0000 7fda717fa640 1 -- 172.21.15.16:0/4176792494 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fda8002b440 con 0x564824402240 2024-09-12T02:14:07.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.040+0000 7fda9206b740 1 -- 172.21.15.16:0/4176792494 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fda7809bc30 con 0x564824402240 2024-09-12T02:14:07.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.041+0000 7fda717fa640 1 -- 172.21.15.16:0/4176792494 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fda8002b640 con 0x564824402240 2024-09-12T02:14:07.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.041+0000 7fda9206b740 10 client.19548 fetch_fsmap learned FSMap version 25 2024-09-12T02:14:07.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.041+0000 7fda9206b740 1 -- 172.21.15.16:0/4176792494 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56482447cf30 con 0x564824402240 2024-09-12T02:14:07.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.041+0000 7fda717fa640 1 -- 172.21.15.16:0/4176792494 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda7809bc30 con 0x564824402240 2024-09-12T02:14:07.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.041+0000 7fda717fa640 1 -- 172.21.15.16:0/4176792494 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fda80029020 con 0x564824402240 2024-09-12T02:14:07.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.041+0000 7fda9206b740 10 client.19548 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:14:07.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.041+0000 7fda9206b740 10 client.19548 resolve_mds: validated gid 24482 aka daemon mds.b 2024-09-12T02:14:07.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.041+0000 7fda9206b740 20 client.19548 populate_metadata read hostname 'smithi016' 2024-09-12T02:14:07.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.041+0000 7fda9206b740 1 --2- 172.21.15.16:0/4176792494 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x56482447d6b0 0x56482447fad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:07.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.041+0000 7fda9206b740 4 client.19548 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "336cf4e6", "release": true, "await": true}] 2024-09-12T02:14:07.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.041+0000 7fda9206b740 1 -- 172.21.15.16:0/4176792494 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "quiesce db", "set_id": "336cf4e6", "release": true, "await": true}) -- 0x5648243640a0 con 0x56482447d6b0 2024-09-12T02:14:07.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.041+0000 7fda8da2e640 1 --2- 172.21.15.16:0/4176792494 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x56482447d6b0 0x56482447fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:07.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.042+0000 7fda8da2e640 1 --2- 172.21.15.16:0/4176792494 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x56482447d6b0 0x56482447fad0 crc :-1 s=READY pgs=371 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b74df4b5e0c4a7d3 server_cookie=a4e881d3a5a777fb in_seq=0 out_seq=0 2024-09-12T02:14:07.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.042+0000 7fda717fa640 10 client.19548 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:14:07.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.044+0000 7fda717fa640 1 -- 172.21.15.16:0/4176792494 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5648243640a0 con 0x56482447d6b0 2024-09-12T02:14:07.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.044+0000 7fda717fa640 10 client.19548 handle_command_reply: tid=0 2024-09-12T02:14:07.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.044+0000 7fda9206b740 1 client.19548 shutdown 2024-09-12T02:14:07.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.044+0000 7fda9206b740 1 -- 172.21.15.16:0/4176792494 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564824402240 msgr2=0x56482447b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:14:07.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.044+0000 7fda9206b740 1 --2- 172.21.15.16:0/4176792494 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564824402240 0x56482447b9d0 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7fda80010d20 tx=0x7fda8001a000 comp rx=0 tx=0).stop 2024-09-12T02:14:07.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.045+0000 7fda9206b740 1 -- 172.21.15.16:0/4176792494 shutdown_connections 2024-09-12T02:14:07.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.045+0000 7fda9206b740 1 --2- 172.21.15.16:0/4176792494 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x56482447d6b0 0x56482447fad0 crc :-1 s=READY pgs=371 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:07.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.045+0000 7fda9206b740 1 --2- 172.21.15.16:0/4176792494 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564824402240 0x56482447b9d0 unknown :-1 s=CLOSED pgs=1724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:07.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.045+0000 7fda9206b740 1 --2- 172.21.15.16:0/4176792494 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5648243f7980 0x564824406aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:07.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.045+0000 7fda717fa640 0 client.19548 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:14:07.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.045+0000 7fda9206b740 1 --2- 172.21.15.16:0/4176792494 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5648243f7060 0x564824406560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:07.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.045+0000 7fda9206b740 1 -- 172.21.15.16:0/4176792494 >> 172.21.15.16:0/4176792494 conn(0x5648243f35f0 msgr2=0x5648243f92d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:14:07.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.045+0000 7fda9206b740 1 -- 172.21.15.16:0/4176792494 shutdown_connections 2024-09-12T02:14:07.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.045+0000 7fda9206b740 1 -- 172.21.15.16:0/4176792494 wait complete. 2024-09-12T02:14:07.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.045+0000 7fda9206b740 20 client.19548 trim_cache size 0 max 16384 2024-09-12T02:14:07.049 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T02:14:07.049 INFO:teuthology.orchestra.run.smithi016.stdout: "epoch": 25, 2024-09-12T02:14:07.049 INFO:teuthology.orchestra.run.smithi016.stdout: "leader": 24482, 2024-09-12T02:14:07.049 INFO:teuthology.orchestra.run.smithi016.stdout: "set_version": 48, 2024-09-12T02:14:07.049 INFO:teuthology.orchestra.run.smithi016.stdout: "sets": { 2024-09-12T02:14:07.049 INFO:teuthology.orchestra.run.smithi016.stdout: "336cf4e6": { 2024-09-12T02:14:07.049 INFO:teuthology.orchestra.run.smithi016.stdout: "version": 48, 2024-09-12T02:14:07.049 INFO:teuthology.orchestra.run.smithi016.stdout: "age_ref": 0.0, 2024-09-12T02:14:07.049 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-12T02:14:07.050 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RELEASED", 2024-09-12T02:14:07.050 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-12T02:14:07.050 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:14:07.050 INFO:teuthology.orchestra.run.smithi016.stdout: "timeout": 90.0, 2024-09-12T02:14:07.050 INFO:teuthology.orchestra.run.smithi016.stdout: "expiration": 147.2, 2024-09-12T02:14:07.050 INFO:teuthology.orchestra.run.smithi016.stdout: "members": { 2024-09-12T02:14:07.050 INFO:teuthology.orchestra.run.smithi016.stdout: "file:/": { 2024-09-12T02:14:07.050 INFO:teuthology.orchestra.run.smithi016.stdout: "excluded": false, 2024-09-12T02:14:07.050 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-12T02:14:07.050 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RELEASED", 2024-09-12T02:14:07.050 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-12T02:14:07.051 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:14:07.051 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:14:07.051 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:14:07.051 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:14:07.051 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:14:07.051 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T02:14:07.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.045+0000 7fda8fc92640 1 -- 172.21.15.16:0/4048885091 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fda740a2b80 msgr2=0x7fda740a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:14:07.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.045+0000 7fda8fc92640 1 --2- 172.21.15.16:0/4048885091 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fda740a2b80 0x7fda740a5040 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fda80005c40 tx=0x7fda8000acb0 comp rx=0 tx=0).stop 2024-09-12T02:14:07.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.045+0000 7fda8fc92640 1 -- 172.21.15.16:0/4048885091 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda88074eb0 msgr2=0x7fda881a4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:14:07.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.045+0000 7fda8fc92640 1 --2- 172.21.15.16:0/4048885091 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda88074eb0 0x7fda881a4c10 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7fda7800d950 tx=0x7fda7800de10 comp rx=0 tx=0).stop 2024-09-12T02:14:07.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.046+0000 7fda8fc92640 1 -- 172.21.15.16:0/4048885091 shutdown_connections 2024-09-12T02:14:07.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.046+0000 7fda8fc92640 1 --2- 172.21.15.16:0/4048885091 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fda740a2b80 0x7fda740a5040 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:07.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.046+0000 7fda8fc92640 1 --2- 172.21.15.16:0/4048885091 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fda88112b90 0x7fda8819e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:07.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.046+0000 7fda8fc92640 1 --2- 172.21.15.16:0/4048885091 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda88074eb0 0x7fda881a4c10 unknown :-1 s=CLOSED pgs=1720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:07.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.046+0000 7fda8fc92640 1 --2- 172.21.15.16:0/4048885091 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fda88074590 0x7fda881a46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:07.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.046+0000 7fda8fc92640 1 -- 172.21.15.16:0/4048885091 >> 172.21.15.16:0/4048885091 conn(0x7fda880fd8c0 msgr2=0x7fda880ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:14:07.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.046+0000 7fda8fc92640 1 -- 172.21.15.16:0/4048885091 shutdown_connections 2024-09-12T02:14:07.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:07.046+0000 7fda8fc92640 1 -- 172.21.15.16:0/4048885091 wait complete. 2024-09-12T02:14:07.659 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '336cf4e6', total seconds elapsed: 30.4 2024-09-12T02:14:08.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:14:08 smithi033 ceph-mon[35332]: pgmap v701: 97 pgs: 97 active+clean; 2.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.6 KiB/s wr, 8 op/s 2024-09-12T02:14:08.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:14:08 smithi016 ceph-mon[30947]: pgmap v701: 97 pgs: 97 active+clean; 2.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.6 KiB/s wr, 8 op/s 2024-09-12T02:14:08.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:14:08 smithi184 ceph-mon[35593]: pgmap v701: 97 pgs: 97 active+clean; 2.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.6 KiB/s wr, 8 op/s 2024-09-12T02:14:10.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:14:10 smithi033 ceph-mon[35332]: pgmap v702: 97 pgs: 97 active+clean; 2.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2024-09-12T02:14:10.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:14:10 smithi016 ceph-mon[30947]: pgmap v702: 97 pgs: 97 active+clean; 2.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2024-09-12T02:14:10.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:14:10 smithi184 ceph-mon[35593]: pgmap v702: 97 pgs: 97 active+clean; 2.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2024-09-12T02:14:12.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:14:12 smithi033 ceph-mon[35332]: pgmap v703: 97 pgs: 97 active+clean; 2.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.6 KiB/s wr, 8 op/s 2024-09-12T02:14:12.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:14:12 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:14:12.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:14:12 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T02:14:12.420 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:14:12 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T02:14:12.420 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:14:12 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T02:14:12.420 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:14:12 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T02:14:12.420 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:14:12 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:14:12.420 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:14:12 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:14:12.420 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:14:12 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T02:14:12.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:14:12 smithi016 ceph-mon[30947]: pgmap v703: 97 pgs: 97 active+clean; 2.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.6 KiB/s wr, 8 op/s 2024-09-12T02:14:12.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:14:12 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:14:12.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:14:12 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T02:14:12.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:14:12 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T02:14:12.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:14:12 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T02:14:12.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:14:12 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T02:14:12.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:14:12 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:14:12.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:14:12 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:14:12.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:14:12 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T02:14:12.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:14:12 smithi184 ceph-mon[35593]: pgmap v703: 97 pgs: 97 active+clean; 2.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.6 KiB/s wr, 8 op/s 2024-09-12T02:14:12.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:14:12 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:14:12.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:14:12 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T02:14:12.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:14:12 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T02:14:12.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:14:12 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T02:14:12.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:14:12 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T02:14:12.628 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:14:12 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:14:12.628 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:14:12 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:14:12.628 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:14:12 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T02:14:13.656 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.654+0000 7f688fc74640 1 -- 172.21.15.16:0/2027996214 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6888100cf0 msgr2=0x7f6888101150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:14:13.656 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.654+0000 7f688fc74640 1 --2- 172.21.15.16:0/2027996214 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6888100cf0 0x7f6888101150 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7f6878009900 tx=0x7f687802a410 comp rx=0 tx=0).stop 2024-09-12T02:14:13.656 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.654+0000 7f688fc74640 1 -- 172.21.15.16:0/2027996214 shutdown_connections 2024-09-12T02:14:13.656 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.654+0000 7f688fc74640 1 --2- 172.21.15.16:0/2027996214 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6888101690 0x7f6888101b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:13.656 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.654+0000 7f688fc74640 1 --2- 172.21.15.16:0/2027996214 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6888100cf0 0x7f6888101150 unknown :-1 s=CLOSED pgs=1725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:13.656 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.654+0000 7f688fc74640 1 --2- 172.21.15.16:0/2027996214 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68881003b0 0x7f68881007b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:13.657 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.654+0000 7f688fc74640 1 -- 172.21.15.16:0/2027996214 >> 172.21.15.16:0/2027996214 conn(0x7f68880fc1e0 msgr2=0x7f68880fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:14:13.657 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.654+0000 7f688fc74640 1 -- 172.21.15.16:0/2027996214 shutdown_connections 2024-09-12T02:14:13.657 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.654+0000 7f688fc74640 1 -- 172.21.15.16:0/2027996214 wait complete. 2024-09-12T02:14:13.657 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.654+0000 7f688fc74640 1 Processor -- start 2024-09-12T02:14:13.657 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.655+0000 7f688fc74640 1 -- start start 2024-09-12T02:14:13.657 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.655+0000 7f688fc74640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f68881003b0 0x7f688819a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:13.657 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.655+0000 7f688fc74640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6888100cf0 0x7f688819a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:13.657 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.655+0000 7f688fc74640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6888101690 0x7f68881a15d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:13.657 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.655+0000 7f688fc74640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f688810f990 con 0x7f6888100cf0 2024-09-12T02:14:13.657 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.655+0000 7f688fc74640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f688810f810 con 0x7f6888101690 2024-09-12T02:14:13.657 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.655+0000 7f688fc74640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f688810fb10 con 0x7f68881003b0 2024-09-12T02:14:13.657 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.655+0000 7f688d20f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6888100cf0 0x7f688819a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:13.657 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.655+0000 7f688d20f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6888100cf0 0x7f688819a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53210/0 (socket says 172.21.15.16:53210) 2024-09-12T02:14:13.658 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.655+0000 7f688d20f640 1 -- 172.21.15.16:0/2959488040 learned_addr learned my addr 172.21.15.16:0/2959488040 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:14:13.658 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.655+0000 7f688da10640 1 --2- 172.21.15.16:0/2959488040 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f68881003b0 0x7f688819a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:13.658 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.655+0000 7f688e211640 1 --2- 172.21.15.16:0/2959488040 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6888101690 0x7f68881a15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:13.658 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.655+0000 7f688d20f640 1 -- 172.21.15.16:0/2959488040 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f68881003b0 msgr2=0x7f688819a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:14:13.658 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.655+0000 7f688d20f640 1 --2- 172.21.15.16:0/2959488040 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f68881003b0 0x7f688819a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:13.658 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.655+0000 7f688d20f640 1 -- 172.21.15.16:0/2959488040 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6888101690 msgr2=0x7f68881a15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:14:13.658 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.655+0000 7f688d20f640 1 --2- 172.21.15.16:0/2959488040 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6888101690 0x7f68881a15d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:13.658 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.655+0000 7f688d20f640 1 -- 172.21.15.16:0/2959488040 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68881a1c40 con 0x7f6888100cf0 2024-09-12T02:14:13.658 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.655+0000 7f688e211640 1 --2- 172.21.15.16:0/2959488040 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6888101690 0x7f68881a15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:14:13.658 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.655+0000 7f688d20f640 1 --2- 172.21.15.16:0/2959488040 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6888100cf0 0x7f688819a560 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7f6878006010 tx=0x7f6878004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:14:13.658 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.656+0000 7f687effd640 1 -- 172.21.15.16:0/2959488040 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f687803f030 con 0x7f6888100cf0 2024-09-12T02:14:13.659 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.656+0000 7f687effd640 1 -- 172.21.15.16:0/2959488040 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6878003ce0 con 0x7f6888100cf0 2024-09-12T02:14:13.659 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.656+0000 7f688fc74640 1 -- 172.21.15.16:0/2959488040 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68881a1ed0 con 0x7f6888100cf0 2024-09-12T02:14:13.659 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.656+0000 7f687effd640 1 -- 172.21.15.16:0/2959488040 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6878035070 con 0x7f6888100cf0 2024-09-12T02:14:13.659 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.656+0000 7f688fc74640 1 -- 172.21.15.16:0/2959488040 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68881a2340 con 0x7f6888100cf0 2024-09-12T02:14:13.662 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.658+0000 7f687effd640 1 -- 172.21.15.16:0/2959488040 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6878042030 con 0x7f6888100cf0 2024-09-12T02:14:13.662 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.659+0000 7f6892a15740 1 Processor -- start 2024-09-12T02:14:13.662 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.659+0000 7f6892a15740 1 -- start start 2024-09-12T02:14:13.662 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.659+0000 7f6892a15740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55acb24f99a0 0x55acb24f9da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:13.662 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.659+0000 7f6892a15740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55acb24fa2e0 0x55acb24feda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:13.662 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.659+0000 7f6892a15740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55acb24ff370 0x55acb2501830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:13.662 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.659+0000 7f6892a15740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55acb245d2f0 con 0x55acb24fa2e0 2024-09-12T02:14:13.662 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.659+0000 7f6892a15740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55acb2405270 con 0x55acb24f99a0 2024-09-12T02:14:13.662 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.659+0000 7f6892a15740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55acb24592b0 con 0x55acb24ff370 2024-09-12T02:14:13.662 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.659+0000 7f687effd640 1 --2- 172.21.15.16:0/2959488040 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f68740a2cb0 0x7f68740a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:13.662 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.659+0000 7f688da10640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55acb24fa2e0 0x55acb24feda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:13.662 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.659+0000 7f688da10640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55acb24fa2e0 0x55acb24feda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53224/0 (socket says 172.21.15.16:53224) 2024-09-12T02:14:13.663 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.659+0000 7f688da10640 1 -- 172.21.15.16:0/4073274539 learned_addr learned my addr 172.21.15.16:0/4073274539 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:14:13.663 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.660+0000 7f688e211640 1 --2- 172.21.15.16:0/2959488040 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f68740a2cb0 0x7f68740a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:13.663 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.660+0000 7f687effd640 1 -- 172.21.15.16:0/2959488040 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f68780ef910 con 0x7f6888100cf0 2024-09-12T02:14:13.663 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.660+0000 7f688d20f640 1 --2- 172.21.15.16:0/4073274539 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55acb24ff370 0x55acb2501830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:13.663 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.660+0000 7f688e211640 1 --2- 172.21.15.16:0/4073274539 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55acb24f99a0 0x55acb24f9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:13.663 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.660+0000 7f688da10640 1 -- 172.21.15.16:0/4073274539 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55acb24ff370 msgr2=0x55acb2501830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:14:13.663 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.660+0000 7f688da10640 1 --2- 172.21.15.16:0/4073274539 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55acb24ff370 0x55acb2501830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:13.663 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.660+0000 7f688da10640 1 -- 172.21.15.16:0/4073274539 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55acb24f99a0 msgr2=0x55acb24f9da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:14:13.663 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.660+0000 7f688da10640 1 --2- 172.21.15.16:0/4073274539 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55acb24f99a0 0x55acb24f9da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:13.663 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.660+0000 7f688da10640 1 -- 172.21.15.16:0/4073274539 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55acb2404340 con 0x55acb24fa2e0 2024-09-12T02:14:13.663 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.660+0000 7f688da10640 1 --2- 172.21.15.16:0/4073274539 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55acb24fa2e0 0x55acb24feda0 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7f688400bd50 tx=0x7f68840107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b31cc170fd4f8294 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:14:13.663 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.660+0000 7f688e211640 1 --2- 172.21.15.16:0/2959488040 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f68740a2cb0 0x7f68740a5170 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f6880009fd0 tx=0x7f6880009540 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:14:13.664 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.660+0000 7f6873fff640 1 -- 172.21.15.16:0/4073274539 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6884005220 con 0x55acb24fa2e0 2024-09-12T02:14:13.664 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.660+0000 7f6873fff640 1 -- 172.21.15.16:0/4073274539 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f688400be40 con 0x55acb24fa2e0 2024-09-12T02:14:13.664 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.660+0000 7f6873fff640 1 -- 172.21.15.16:0/4073274539 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6884011730 con 0x55acb24fa2e0 2024-09-12T02:14:13.664 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.661+0000 7f6892a15740 1 -- 172.21.15.16:0/4073274539 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55acb24fa2e0 msgr2=0x55acb24feda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:14:13.664 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.661+0000 7f6892a15740 1 --2- 172.21.15.16:0/4073274539 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55acb24fa2e0 0x55acb24feda0 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7f688400bd50 tx=0x7f68840107a0 comp rx=0 tx=0).stop 2024-09-12T02:14:13.664 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.661+0000 7f6892a15740 1 -- 172.21.15.16:0/4073274539 shutdown_connections 2024-09-12T02:14:13.664 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.661+0000 7f6892a15740 1 --2- 172.21.15.16:0/4073274539 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55acb24ff370 0x55acb2501830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:13.664 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.661+0000 7f6892a15740 1 --2- 172.21.15.16:0/4073274539 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55acb24fa2e0 0x55acb24feda0 unknown :-1 s=CLOSED pgs=1727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:13.664 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.661+0000 7f6892a15740 1 --2- 172.21.15.16:0/4073274539 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55acb24f99a0 0x55acb24f9da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:13.664 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.661+0000 7f6892a15740 1 -- 172.21.15.16:0/4073274539 >> 172.21.15.16:0/4073274539 conn(0x55acb24f5010 msgr2=0x55acb24f5410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:14:13.664 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.661+0000 7f6892a15740 1 -- 172.21.15.16:0/4073274539 shutdown_connections 2024-09-12T02:14:13.664 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.661+0000 7f6892a15740 1 -- 172.21.15.16:0/4073274539 wait complete. 2024-09-12T02:14:13.664 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.662+0000 7f6892a15740 1 -- start start 2024-09-12T02:14:13.665 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.662+0000 7f6892a15740 1 Processor -- start 2024-09-12T02:14:13.668 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.662+0000 7f6892a15740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55acb24f99a0 0x55acb2503c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:13.668 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.662+0000 7f6892a15740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55acb24fa2e0 0x55acb2504170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:13.668 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.662+0000 7f6892a15740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55acb24ff370 0x55acb250b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:13.668 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.662+0000 7f6892a15740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55acb2502340 con 0x55acb24f99a0 2024-09-12T02:14:13.668 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.662+0000 7f6892a15740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55acb23e6290 con 0x55acb24fa2e0 2024-09-12T02:14:13.669 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.662+0000 7f6892a15740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55acb25024c0 con 0x55acb24ff370 2024-09-12T02:14:13.669 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.662+0000 7f6892a15740 10 client.0 _ll_register_callbacks cb 0x55acb2417140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:14:13.669 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.662+0000 7f688da10640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55acb24f99a0 0x55acb2503c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:13.669 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.662+0000 7f688da10640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55acb24f99a0 0x55acb2503c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53230/0 (socket says 172.21.15.16:53230) 2024-09-12T02:14:13.669 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.662+0000 7f688da10640 1 -- 172.21.15.16:0/78779933 learned_addr learned my addr 172.21.15.16:0/78779933 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:14:13.669 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.663+0000 7f688d20f640 1 --2- 172.21.15.16:0/78779933 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55acb24fa2e0 0x55acb2504170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:13.669 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.663+0000 7f688e211640 1 --2- 172.21.15.16:0/78779933 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55acb24ff370 0x55acb250b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:13.669 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.663+0000 7f688da10640 1 -- 172.21.15.16:0/78779933 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55acb24ff370 msgr2=0x55acb250b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:14:13.669 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.663+0000 7f688da10640 1 --2- 172.21.15.16:0/78779933 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55acb24ff370 0x55acb250b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:13.670 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.663+0000 7f688da10640 1 -- 172.21.15.16:0/78779933 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55acb24fa2e0 msgr2=0x55acb2504170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:14:13.670 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.663+0000 7f688da10640 1 --2- 172.21.15.16:0/78779933 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55acb24fa2e0 0x55acb2504170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:13.670 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.663+0000 7f688da10640 1 -- 172.21.15.16:0/78779933 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55acb2170210 con 0x55acb24f99a0 2024-09-12T02:14:13.670 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.663+0000 7f688d20f640 1 --2- 172.21.15.16:0/78779933 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55acb24fa2e0 0x55acb2504170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:14:13.670 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.663+0000 7f688e211640 1 --2- 172.21.15.16:0/78779933 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55acb24ff370 0x55acb250b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:14:13.670 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.663+0000 7f688da10640 1 --2- 172.21.15.16:0/78779933 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55acb24f99a0 0x55acb2503c30 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7f688400bd20 tx=0x7f68840052e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e95e07a6933f0b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:14:13.671 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.663+0000 7f68717fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:14:13.671 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.663+0000 7f68717fa640 1 -- 172.21.15.16:0/78779933 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6884004020 con 0x55acb24f99a0 2024-09-12T02:14:13.671 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.663+0000 7f68717fa640 1 -- 172.21.15.16:0/78779933 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f68840041c0 con 0x55acb24f99a0 2024-09-12T02:14:13.671 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.663+0000 7f6892a15740 1 -- 172.21.15.16:0/78779933 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55acb246aa20 con 0x55acb24f99a0 2024-09-12T02:14:13.671 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.663+0000 7f68717fa640 1 -- 172.21.15.16:0/78779933 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f68840043c0 con 0x55acb24f99a0 2024-09-12T02:14:13.671 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.663+0000 7f6892a15740 10 client.19560 fetch_fsmap learned FSMap version 25 2024-09-12T02:14:13.672 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.663+0000 7f6892a15740 1 -- 172.21.15.16:0/78779933 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55acb223c620 con 0x55acb24f99a0 2024-09-12T02:14:13.672 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.663+0000 7f68717fa640 1 -- 172.21.15.16:0/78779933 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55acb246aa20 con 0x55acb24f99a0 2024-09-12T02:14:13.672 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.664+0000 7f68717fa640 1 -- 172.21.15.16:0/78779933 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f6884003bb0 con 0x55acb24f99a0 2024-09-12T02:14:13.672 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.664+0000 7f6892a15740 10 client.19560 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:14:13.672 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.664+0000 7f6892a15740 10 client.19560 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:14:13.672 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.664+0000 7f6892a15740 20 client.19560 populate_metadata read hostname 'smithi016' 2024-09-12T02:14:13.673 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.664+0000 7f6892a15740 1 --2- 172.21.15.16:0/78779933 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55acb250f5c0 0x55acb25119b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:13.673 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.664+0000 7f6892a15740 4 client.19560 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:14:13.673 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.664+0000 7f6892a15740 1 -- 172.21.15.16:0/78779933 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55acb2464d70 con 0x55acb250f5c0 2024-09-12T02:14:13.673 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.664+0000 7f688d20f640 1 --2- 172.21.15.16:0/78779933 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55acb250f5c0 0x55acb25119b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:13.673 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.664+0000 7f688d20f640 1 --2- 172.21.15.16:0/78779933 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55acb250f5c0 0x55acb25119b0 crc :-1 s=READY pgs=372 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3bde7145a24450f2 server_cookie=1dfa5e26c88d8cad in_seq=0 out_seq=0 2024-09-12T02:14:13.673 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.664+0000 7f68717fa640 10 client.19560 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:14:13.673 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.665+0000 7f68717fa640 1 -- 172.21.15.16:0/78779933 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55acb2464d70 con 0x55acb250f5c0 2024-09-12T02:14:13.674 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.665+0000 7f68717fa640 10 client.19560 handle_command_reply: tid=0 2024-09-12T02:14:13.674 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.665+0000 7f6892a15740 1 client.19560 shutdown 2024-09-12T02:14:13.674 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.666+0000 7f6892a15740 1 -- 172.21.15.16:0/78779933 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55acb24f99a0 msgr2=0x55acb2503c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:14:13.674 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.666+0000 7f6892a15740 1 --2- 172.21.15.16:0/78779933 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55acb24f99a0 0x55acb2503c30 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7f688400bd20 tx=0x7f68840052e0 comp rx=0 tx=0).stop 2024-09-12T02:14:13.674 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.666+0000 7f6892a15740 1 -- 172.21.15.16:0/78779933 shutdown_connections 2024-09-12T02:14:13.674 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.666+0000 7f6892a15740 1 --2- 172.21.15.16:0/78779933 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55acb250f5c0 0x55acb25119b0 crc :-1 s=READY pgs=372 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:13.674 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.666+0000 7f6892a15740 1 --2- 172.21.15.16:0/78779933 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55acb24ff370 0x55acb250b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:13.674 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.666+0000 7f68717fa640 0 client.19560 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:14:13.675 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.666+0000 7f6892a15740 1 --2- 172.21.15.16:0/78779933 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55acb24fa2e0 0x55acb2504170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:13.675 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.666+0000 7f6892a15740 1 --2- 172.21.15.16:0/78779933 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55acb24f99a0 0x55acb2503c30 unknown :-1 s=CLOSED pgs=1728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:13.675 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.666+0000 7f6892a15740 1 -- 172.21.15.16:0/78779933 >> 172.21.15.16:0/78779933 conn(0x55acb24f5010 msgr2=0x55acb24f6820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:14:13.675 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.666+0000 7f6892a15740 1 -- 172.21.15.16:0/78779933 shutdown_connections 2024-09-12T02:14:13.675 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.666+0000 7f6892a15740 1 -- 172.21.15.16:0/78779933 wait complete. 2024-09-12T02:14:13.675 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.666+0000 7f6892a15740 20 client.19560 trim_cache size 0 max 16384 2024-09-12T02:14:13.685 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.680+0000 7f6892a15740 1 Processor -- start 2024-09-12T02:14:13.685 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.680+0000 7f6892a15740 1 -- start start 2024-09-12T02:14:13.685 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.681+0000 7f6892a15740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55acb24f99a0 0x55acb24f8410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:13.685 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.681+0000 7f6892a15740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55acb24fa2e0 0x55acb250b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:13.685 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.681+0000 7f6892a15740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55acb24ff370 0x55acb24f8950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:13.685 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.681+0000 7f6892a15740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f68881ad9f0 con 0x55acb24fa2e0 2024-09-12T02:14:13.685 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.681+0000 7f6892a15740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f68881ab2c0 con 0x55acb24ff370 2024-09-12T02:14:13.685 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.681+0000 7f6892a15740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f68881a9130 con 0x55acb24f99a0 2024-09-12T02:14:13.686 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.681+0000 7f688d20f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55acb24fa2e0 0x55acb250b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:13.686 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.681+0000 7f688d20f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55acb24fa2e0 0x55acb250b0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53238/0 (socket says 172.21.15.16:53238) 2024-09-12T02:14:13.686 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.681+0000 7f688d20f640 1 -- 172.21.15.16:0/1839363200 learned_addr learned my addr 172.21.15.16:0/1839363200 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:14:13.686 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.681+0000 7f688d20f640 1 -- 172.21.15.16:0/1839363200 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55acb24f99a0 msgr2=0x55acb24f8410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:14:13.686 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.681+0000 7f688da10640 1 --2- 172.21.15.16:0/1839363200 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55acb24f99a0 0x55acb24f8410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:13.686 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.681+0000 7f688e211640 1 --2- 172.21.15.16:0/1839363200 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55acb24ff370 0x55acb24f8950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:13.686 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.681+0000 7f688d20f640 1 --2- 172.21.15.16:0/1839363200 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55acb24f99a0 0x55acb24f8410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:13.686 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.681+0000 7f688d20f640 1 -- 172.21.15.16:0/1839363200 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55acb24ff370 msgr2=0x55acb24f8950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:14:13.686 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.681+0000 7f688d20f640 1 --2- 172.21.15.16:0/1839363200 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55acb24ff370 0x55acb24f8950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:13.686 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.681+0000 7f688d20f640 1 -- 172.21.15.16:0/1839363200 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55acb2511d20 con 0x55acb24fa2e0 2024-09-12T02:14:13.686 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.681+0000 7f688da10640 1 --2- 172.21.15.16:0/1839363200 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55acb24f99a0 0x55acb24f8410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:14:13.686 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.681+0000 7f688e211640 1 --2- 172.21.15.16:0/1839363200 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55acb24ff370 0x55acb24f8950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:14:13.686 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.681+0000 7f688d20f640 1 --2- 172.21.15.16:0/1839363200 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55acb24fa2e0 0x55acb250b0d0 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7f68780bbe70 tx=0x7f68780ed2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b04d2fde75ead71 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:14:13.687 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.681+0000 7f68737fe640 1 -- 172.21.15.16:0/1839363200 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6878049040 con 0x55acb24fa2e0 2024-09-12T02:14:13.687 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.681+0000 7f68737fe640 1 -- 172.21.15.16:0/1839363200 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68780c0520 con 0x55acb24fa2e0 2024-09-12T02:14:13.687 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.681+0000 7f68737fe640 1 -- 172.21.15.16:0/1839363200 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68780b87e0 con 0x55acb24fa2e0 2024-09-12T02:14:13.687 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.682+0000 7f6892a15740 1 -- 172.21.15.16:0/1839363200 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55acb24fa2e0 msgr2=0x55acb250b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:14:13.687 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.682+0000 7f6892a15740 1 --2- 172.21.15.16:0/1839363200 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55acb24fa2e0 0x55acb250b0d0 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7f68780bbe70 tx=0x7f68780ed2e0 comp rx=0 tx=0).stop 2024-09-12T02:14:13.687 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.682+0000 7f6892a15740 1 -- 172.21.15.16:0/1839363200 shutdown_connections 2024-09-12T02:14:13.687 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.682+0000 7f6892a15740 1 --2- 172.21.15.16:0/1839363200 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55acb24ff370 0x55acb24f8950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:13.687 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.682+0000 7f6892a15740 1 --2- 172.21.15.16:0/1839363200 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55acb24fa2e0 0x55acb250b0d0 unknown :-1 s=CLOSED pgs=1729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:13.687 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.682+0000 7f6892a15740 1 --2- 172.21.15.16:0/1839363200 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55acb24f99a0 0x55acb24f8410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:13.687 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.682+0000 7f6892a15740 1 -- 172.21.15.16:0/1839363200 >> 172.21.15.16:0/1839363200 conn(0x55acb24f5010 msgr2=0x55acb24f41c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:14:13.687 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.682+0000 7f6892a15740 1 -- 172.21.15.16:0/1839363200 shutdown_connections 2024-09-12T02:14:13.687 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.682+0000 7f6892a15740 1 -- 172.21.15.16:0/1839363200 wait complete. 2024-09-12T02:14:13.687 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.682+0000 7f6892a15740 1 -- start start 2024-09-12T02:14:13.688 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.682+0000 7f6892a15740 1 Processor -- start 2024-09-12T02:14:13.688 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.683+0000 7f6892a15740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55acb24f99a0 0x55acb2508af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:13.688 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.683+0000 7f6892a15740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55acb24fa2e0 0x55acb2509030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:13.688 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.683+0000 7f6892a15740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55acb24ff370 0x55acb250a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:13.688 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.683+0000 7f6892a15740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f68881a9780 con 0x55acb24f99a0 2024-09-12T02:14:13.688 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.683+0000 7f6892a15740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f68881aabf0 con 0x55acb24fa2e0 2024-09-12T02:14:13.688 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.683+0000 7f6892a15740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f68881a9e50 con 0x55acb24ff370 2024-09-12T02:14:13.688 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.683+0000 7f6892a15740 10 client.0 _ll_register_callbacks cb 0x7f68800126d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:14:13.688 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.683+0000 7f688da10640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55acb24f99a0 0x55acb2508af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:13.688 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.683+0000 7f688da10640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55acb24f99a0 0x55acb2508af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53244/0 (socket says 172.21.15.16:53244) 2024-09-12T02:14:13.688 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.683+0000 7f688da10640 1 -- 172.21.15.16:0/124615897 learned_addr learned my addr 172.21.15.16:0/124615897 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:14:13.688 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.683+0000 7f688e211640 1 --2- 172.21.15.16:0/124615897 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55acb24ff370 0x55acb250a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:13.688 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.683+0000 7f688d20f640 1 --2- 172.21.15.16:0/124615897 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55acb24fa2e0 0x55acb2509030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:13.689 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.683+0000 7f688da10640 1 -- 172.21.15.16:0/124615897 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55acb24ff370 msgr2=0x55acb250a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:14:13.689 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.683+0000 7f688da10640 1 --2- 172.21.15.16:0/124615897 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55acb24ff370 0x55acb250a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:13.689 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.683+0000 7f688da10640 1 -- 172.21.15.16:0/124615897 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55acb24fa2e0 msgr2=0x55acb2509030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:14:13.689 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.683+0000 7f688da10640 1 --2- 172.21.15.16:0/124615897 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55acb24fa2e0 0x55acb2509030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:13.689 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.683+0000 7f688da10640 1 -- 172.21.15.16:0/124615897 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55acb250a650 con 0x55acb24f99a0 2024-09-12T02:14:13.689 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.683+0000 7f688d20f640 1 --2- 172.21.15.16:0/124615897 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55acb24fa2e0 0x55acb2509030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:14:13.689 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.683+0000 7f688da10640 1 --2- 172.21.15.16:0/124615897 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55acb24f99a0 0x55acb2508af0 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7f688400bd50 tx=0x7f68840045d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=306f1a46da90a281 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:14:13.689 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.683+0000 7f68717fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:14:13.689 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.684+0000 7f68717fa640 1 -- 172.21.15.16:0/124615897 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f688400b410 con 0x55acb24f99a0 2024-09-12T02:14:13.689 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.684+0000 7f68717fa640 1 -- 172.21.15.16:0/124615897 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f6884005560 con 0x55acb24f99a0 2024-09-12T02:14:13.689 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.684+0000 7f6892a15740 1 -- 172.21.15.16:0/124615897 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68780c0520 con 0x55acb24f99a0 2024-09-12T02:14:13.689 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.684+0000 7f68717fa640 1 -- 172.21.15.16:0/124615897 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6884041050 con 0x55acb24f99a0 2024-09-12T02:14:13.689 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.684+0000 7f6892a15740 10 client.19566 fetch_fsmap learned FSMap version 25 2024-09-12T02:14:13.690 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.684+0000 7f6892a15740 1 -- 172.21.15.16:0/124615897 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55acb257a440 con 0x55acb24f99a0 2024-09-12T02:14:13.690 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.684+0000 7f68717fa640 1 -- 172.21.15.16:0/124615897 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68780c0520 con 0x55acb24f99a0 2024-09-12T02:14:13.690 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.684+0000 7f68717fa640 1 -- 172.21.15.16:0/124615897 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f68840035f0 con 0x55acb24f99a0 2024-09-12T02:14:13.690 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.684+0000 7f6892a15740 10 client.19566 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:14:13.690 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.684+0000 7f6892a15740 10 client.19566 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:14:13.690 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.684+0000 7f6892a15740 20 client.19566 populate_metadata read hostname 'smithi016' 2024-09-12T02:14:13.690 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.684+0000 7f6892a15740 1 --2- 172.21.15.16:0/124615897 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55acb257ae20 0x55acb257d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:13.690 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.684+0000 7f6892a15740 4 client.19566 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:14:13.690 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.684+0000 7f6892a15740 1 -- 172.21.15.16:0/124615897 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55acb23e1900 con 0x55acb257ae20 2024-09-12T02:14:13.690 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.685+0000 7f688d20f640 1 --2- 172.21.15.16:0/124615897 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55acb257ae20 0x55acb257d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:13.690 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.685+0000 7f68717fa640 10 client.19566 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:14:13.690 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.685+0000 7f688d20f640 1 --2- 172.21.15.16:0/124615897 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55acb257ae20 0x55acb257d240 crc :-1 s=READY pgs=373 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b13e1145fede109 server_cookie=955fa4829ed6b9fc in_seq=0 out_seq=0 2024-09-12T02:14:13.697 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.696+0000 7f68717fa640 1 -- 172.21.15.16:0/124615897 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55acb23e1900 con 0x55acb257ae20 2024-09-12T02:14:13.697 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.696+0000 7f68717fa640 10 client.19566 handle_command_reply: tid=0 2024-09-12T02:14:13.697 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.696+0000 7f6892a15740 1 client.19566 shutdown 2024-09-12T02:14:13.697 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.696+0000 7f6892a15740 1 -- 172.21.15.16:0/124615897 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55acb24f99a0 msgr2=0x55acb2508af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:14:13.697 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.696+0000 7f6892a15740 1 --2- 172.21.15.16:0/124615897 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55acb24f99a0 0x55acb2508af0 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7f688400bd50 tx=0x7f68840045d0 comp rx=0 tx=0).stop 2024-09-12T02:14:13.697 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.696+0000 7f6892a15740 1 -- 172.21.15.16:0/124615897 shutdown_connections 2024-09-12T02:14:13.697 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.696+0000 7f6892a15740 1 --2- 172.21.15.16:0/124615897 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55acb257ae20 0x55acb257d240 crc :-1 s=READY pgs=373 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:13.698 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.696+0000 7f6892a15740 1 --2- 172.21.15.16:0/124615897 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55acb24ff370 0x55acb250a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:13.698 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.696+0000 7f6892a15740 1 --2- 172.21.15.16:0/124615897 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55acb24fa2e0 0x55acb2509030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:13.698 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.696+0000 7f6892a15740 1 --2- 172.21.15.16:0/124615897 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55acb24f99a0 0x55acb2508af0 unknown :-1 s=CLOSED pgs=1730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:13.698 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.696+0000 7f68717fa640 0 client.19566 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:14:13.698 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.696+0000 7f6892a15740 1 -- 172.21.15.16:0/124615897 >> 172.21.15.16:0/124615897 conn(0x55acb24f5010 msgr2=0x55acb24f7400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:14:13.698 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.696+0000 7f6892a15740 1 -- 172.21.15.16:0/124615897 shutdown_connections 2024-09-12T02:14:13.699 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.696+0000 7f6892a15740 1 -- 172.21.15.16:0/124615897 wait complete. 2024-09-12T02:14:13.700 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.696+0000 7f6892a15740 20 client.19566 trim_cache size 0 max 16384 2024-09-12T02:14:13.700 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-09-12T02:14:13.700 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-09-12T02:14:13.700 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-09-12T02:14:13.700 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-09-12T02:14:13.701 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.697+0000 7f688fc74640 1 -- 172.21.15.16:0/2959488040 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f68740a2cb0 msgr2=0x7f68740a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:14:13.701 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.697+0000 7f688fc74640 1 --2- 172.21.15.16:0/2959488040 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f68740a2cb0 0x7f68740a5170 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f6880009fd0 tx=0x7f6880009540 comp rx=0 tx=0).stop 2024-09-12T02:14:13.701 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.697+0000 7f688fc74640 1 -- 172.21.15.16:0/2959488040 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6888100cf0 msgr2=0x7f688819a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:14:13.701 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.697+0000 7f688fc74640 1 --2- 172.21.15.16:0/2959488040 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6888100cf0 0x7f688819a560 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7f6878006010 tx=0x7f6878004360 comp rx=0 tx=0).stop 2024-09-12T02:14:13.701 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.697+0000 7f688fc74640 1 -- 172.21.15.16:0/2959488040 shutdown_connections 2024-09-12T02:14:13.701 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.697+0000 7f688fc74640 1 --2- 172.21.15.16:0/2959488040 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f68740a2cb0 0x7f68740a5170 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:13.702 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.697+0000 7f688fc74640 1 --2- 172.21.15.16:0/2959488040 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6888101690 0x7f68881a15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:13.702 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.697+0000 7f688fc74640 1 --2- 172.21.15.16:0/2959488040 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6888100cf0 0x7f688819a560 unknown :-1 s=CLOSED pgs=1726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:13.702 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.697+0000 7f688fc74640 1 --2- 172.21.15.16:0/2959488040 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f68881003b0 0x7f688819a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:13.702 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.697+0000 7f688fc74640 1 -- 172.21.15.16:0/2959488040 >> 172.21.15.16:0/2959488040 conn(0x7f68880fc1e0 msgr2=0x7f68880fdc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:14:13.702 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.697+0000 7f688fc74640 1 -- 172.21.15.16:0/2959488040 shutdown_connections 2024-09-12T02:14:13.702 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:13.697+0000 7f688fc74640 1 -- 172.21.15.16:0/2959488040 wait complete. 2024-09-12T02:14:14.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:14:14 smithi033 ceph-mon[35332]: pgmap v704: 97 pgs: 97 active+clean; 2.4 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 7.0 KiB/s wr, 9 op/s 2024-09-12T02:14:14.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:14:14 smithi016 ceph-mon[30947]: pgmap v704: 97 pgs: 97 active+clean; 2.4 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 7.0 KiB/s wr, 9 op/s 2024-09-12T02:14:14.557 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T02:14:14.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:14:14 smithi184 ceph-mon[35593]: pgmap v704: 97 pgs: 97 active+clean; 2.4 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 7.0 KiB/s wr, 9 op/s 2024-09-12T02:14:14.829 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:14:15.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.411+0000 7f99b40c7640 1 -- 172.21.15.16:0/1005401464 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99ac102570 msgr2=0x7f99ac1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:14:15.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.411+0000 7f99b40c7640 1 --2- 172.21.15.16:0/1005401464 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99ac102570 0x7f99ac1029b0 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7f999c009900 tx=0x7f999c02a480 comp rx=0 tx=0).stop 2024-09-12T02:14:15.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.411+0000 7f99b40c7640 1 -- 172.21.15.16:0/1005401464 shutdown_connections 2024-09-12T02:14:15.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.411+0000 7f99b40c7640 1 --2- 172.21.15.16:0/1005401464 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f99ac102ef0 0x7f99ac10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:15.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.411+0000 7f99b40c7640 1 --2- 172.21.15.16:0/1005401464 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99ac102570 0x7f99ac1029b0 unknown :-1 s=CLOSED pgs=1731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:15.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.411+0000 7f99b40c7640 1 --2- 172.21.15.16:0/1005401464 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99ac100e20 0x7f99ac101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:15.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.411+0000 7f99b40c7640 1 -- 172.21.15.16:0/1005401464 >> 172.21.15.16:0/1005401464 conn(0x7f99ac0fc7d0 msgr2=0x7f99ac0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:14:15.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.411+0000 7f99b40c7640 1 -- 172.21.15.16:0/1005401464 shutdown_connections 2024-09-12T02:14:15.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.412+0000 7f99b40c7640 1 -- 172.21.15.16:0/1005401464 wait complete. 2024-09-12T02:14:15.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.412+0000 7f99b40c7640 1 Processor -- start 2024-09-12T02:14:15.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.412+0000 7f99b40c7640 1 -- start start 2024-09-12T02:14:15.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.413+0000 7f99b40c7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99ac100e20 0x7f99ac198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:15.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.413+0000 7f99b40c7640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99ac102570 0x7f99ac198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:15.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.413+0000 7f99b40c7640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f99ac102ef0 0x7f99ac19cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:15.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.413+0000 7f99b40c7640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f99ac110f80 con 0x7f99ac100e20 2024-09-12T02:14:15.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.413+0000 7f99b40c7640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f99ac110e00 con 0x7f99ac102570 2024-09-12T02:14:15.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.413+0000 7f99b40c7640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f99ac111100 con 0x7f99ac102ef0 2024-09-12T02:14:15.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.413+0000 7f99b1e63640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99ac100e20 0x7f99ac198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:15.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.413+0000 7f99b1e63640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99ac100e20 0x7f99ac198450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53266/0 (socket says 172.21.15.16:53266) 2024-09-12T02:14:15.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.413+0000 7f99b1e63640 1 -- 172.21.15.16:0/1877463645 learned_addr learned my addr 172.21.15.16:0/1877463645 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:14:15.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.413+0000 7f99b1662640 1 --2- 172.21.15.16:0/1877463645 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99ac102570 0x7f99ac198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:15.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.413+0000 7f99b2664640 1 --2- 172.21.15.16:0/1877463645 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f99ac102ef0 0x7f99ac19cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:15.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.413+0000 7f99b1e63640 1 -- 172.21.15.16:0/1877463645 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f99ac102ef0 msgr2=0x7f99ac19cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:14:15.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.413+0000 7f99b1e63640 1 --2- 172.21.15.16:0/1877463645 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f99ac102ef0 0x7f99ac19cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:15.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.413+0000 7f99b1e63640 1 -- 172.21.15.16:0/1877463645 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99ac102570 msgr2=0x7f99ac198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:14:15.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.413+0000 7f99b1e63640 1 --2- 172.21.15.16:0/1877463645 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99ac102570 0x7f99ac198990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:15.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.413+0000 7f99b1e63640 1 -- 172.21.15.16:0/1877463645 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99ac19d400 con 0x7f99ac100e20 2024-09-12T02:14:15.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.413+0000 7f99b2664640 1 --2- 172.21.15.16:0/1877463645 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f99ac102ef0 0x7f99ac19cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:14:15.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.413+0000 7f99b1e63640 1 --2- 172.21.15.16:0/1877463645 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99ac100e20 0x7f99ac198450 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7f99a800c920 tx=0x7f99a800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:14:15.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.414+0000 7f99a2ffd640 1 -- 172.21.15.16:0/1877463645 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99a8007c70 con 0x7f99ac100e20 2024-09-12T02:14:15.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.414+0000 7f99b40c7640 1 -- 172.21.15.16:0/1877463645 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99ac19d6f0 con 0x7f99ac100e20 2024-09-12T02:14:15.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.414+0000 7f99a2ffd640 1 -- 172.21.15.16:0/1877463645 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99a8007e10 con 0x7f99ac100e20 2024-09-12T02:14:15.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.414+0000 7f99a2ffd640 1 -- 172.21.15.16:0/1877463645 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99a8018890 con 0x7f99ac100e20 2024-09-12T02:14:15.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.414+0000 7f99b40c7640 1 -- 172.21.15.16:0/1877463645 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99ac1a4fd0 con 0x7f99ac100e20 2024-09-12T02:14:15.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.417+0000 7f99a2ffd640 1 -- 172.21.15.16:0/1877463645 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f99a8018b30 con 0x7f99ac100e20 2024-09-12T02:14:15.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.417+0000 7f99b64a0740 1 Processor -- start 2024-09-12T02:14:15.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.417+0000 7f99b64a0740 1 -- start start 2024-09-12T02:14:15.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.417+0000 7f99b64a0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561f24f34380 0x561f24f34760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:15.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.418+0000 7f99b64a0740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561f24f34ca0 0x561f24f43550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:15.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.418+0000 7f99b64a0740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561f24f43a90 0x561f24f45e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:15.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.418+0000 7f99b64a0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x561f24e9a2e0 con 0x561f24f34380 2024-09-12T02:14:15.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.418+0000 7f99b64a0740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x561f24e41ce0 con 0x561f24f34ca0 2024-09-12T02:14:15.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.418+0000 7f99b64a0740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x561f24e962a0 con 0x561f24f43a90 2024-09-12T02:14:15.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.418+0000 7f99b2664640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561f24f34380 0x561f24f34760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:15.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.418+0000 7f99b2664640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561f24f34380 0x561f24f34760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53268/0 (socket says 172.21.15.16:53268) 2024-09-12T02:14:15.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.418+0000 7f99b2664640 1 -- 172.21.15.16:0/2840436439 learned_addr learned my addr 172.21.15.16:0/2840436439 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:14:15.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.418+0000 7f99a2ffd640 1 --2- 172.21.15.16:0/1877463645 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f99980a2b80 0x7f99980a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:15.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.418+0000 7f99b1e63640 1 --2- 172.21.15.16:0/2840436439 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561f24f34ca0 0x561f24f43550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:15.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.418+0000 7f99b1662640 1 --2- 172.21.15.16:0/2840436439 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561f24f43a90 0x561f24f45e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:15.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.418+0000 7f99b2664640 1 --2- 172.21.15.16:0/1877463645 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f99980a2b80 0x7f99980a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:15.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.418+0000 7f99a2ffd640 1 -- 172.21.15.16:0/1877463645 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f99a80cf190 con 0x7f99ac100e20 2024-09-12T02:14:15.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.418+0000 7f99b1e63640 1 -- 172.21.15.16:0/2840436439 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561f24f43a90 msgr2=0x561f24f45e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:14:15.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.418+0000 7f99b1e63640 1 --2- 172.21.15.16:0/2840436439 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561f24f43a90 0x561f24f45e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:15.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.418+0000 7f99b1e63640 1 -- 172.21.15.16:0/2840436439 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561f24f34380 msgr2=0x561f24f34760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:14:15.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.418+0000 7f99b1e63640 1 --2- 172.21.15.16:0/2840436439 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561f24f34380 0x561f24f34760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:15.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.418+0000 7f99b1e63640 1 -- 172.21.15.16:0/2840436439 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f24e26440 con 0x561f24f34ca0 2024-09-12T02:14:15.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.418+0000 7f99b2664640 1 --2- 172.21.15.16:0/2840436439 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561f24f34380 0x561f24f34760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:14:15.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.418+0000 7f99b1e63640 1 --2- 172.21.15.16:0/2840436439 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561f24f34ca0 0x561f24f43550 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f99a80a4130 tx=0x7f99a800ff30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8914092ce07ec49 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:14:15.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.419+0000 7f99b2664640 1 --2- 172.21.15.16:0/1877463645 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f99980a2b80 0x7f99980a5040 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f99a4014c00 tx=0x7f99a4004ab0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:14:15.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.419+0000 7f9997fff640 1 -- 172.21.15.16:0/2840436439 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99a8018e20 con 0x561f24f34ca0 2024-09-12T02:14:15.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.419+0000 7f9997fff640 1 -- 172.21.15.16:0/2840436439 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99a809b9f0 con 0x561f24f34ca0 2024-09-12T02:14:15.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.419+0000 7f9997fff640 1 -- 172.21.15.16:0/2840436439 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99a8098060 con 0x561f24f34ca0 2024-09-12T02:14:15.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.419+0000 7f99b64a0740 1 -- 172.21.15.16:0/2840436439 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561f24f34ca0 msgr2=0x561f24f43550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:14:15.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.419+0000 7f99b64a0740 1 --2- 172.21.15.16:0/2840436439 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561f24f34ca0 0x561f24f43550 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f99a80a4130 tx=0x7f99a800ff30 comp rx=0 tx=0).stop 2024-09-12T02:14:15.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.419+0000 7f99b64a0740 1 -- 172.21.15.16:0/2840436439 shutdown_connections 2024-09-12T02:14:15.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.419+0000 7f99b64a0740 1 --2- 172.21.15.16:0/2840436439 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561f24f43a90 0x561f24f45e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:15.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.419+0000 7f99b64a0740 1 --2- 172.21.15.16:0/2840436439 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561f24f34ca0 0x561f24f43550 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:15.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.419+0000 7f99b64a0740 1 --2- 172.21.15.16:0/2840436439 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561f24f34380 0x561f24f34760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:15.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.419+0000 7f99b64a0740 1 -- 172.21.15.16:0/2840436439 >> 172.21.15.16:0/2840436439 conn(0x561f24f30a30 msgr2=0x561f24f30e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:14:15.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.420+0000 7f99b64a0740 1 -- 172.21.15.16:0/2840436439 shutdown_connections 2024-09-12T02:14:15.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.420+0000 7f99b64a0740 1 -- 172.21.15.16:0/2840436439 wait complete. 2024-09-12T02:14:15.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.420+0000 7f99b64a0740 1 -- start start 2024-09-12T02:14:15.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.420+0000 7f99b64a0740 1 Processor -- start 2024-09-12T02:14:15.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.420+0000 7f99b64a0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561f24f34380 0x561f24f43470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:15.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.420+0000 7f99b64a0740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561f24f34ca0 0x561f24f40770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:15.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.420+0000 7f99b64a0740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561f24f43a90 0x561f24f40dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:15.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.421+0000 7f99b64a0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x561f24f46660 con 0x561f24f34380 2024-09-12T02:14:15.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.421+0000 7f99b64a0740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x561f24e55050 con 0x561f24f43a90 2024-09-12T02:14:15.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.421+0000 7f99b64a0740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x561f24f467e0 con 0x561f24f34ca0 2024-09-12T02:14:15.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.421+0000 7f99b64a0740 10 client.0 _ll_register_callbacks cb 0x561f24e95160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:14:15.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.421+0000 7f99b1e63640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561f24f34380 0x561f24f43470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:15.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.421+0000 7f99b1e63640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561f24f34380 0x561f24f43470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53278/0 (socket says 172.21.15.16:53278) 2024-09-12T02:14:15.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.421+0000 7f99b1e63640 1 -- 172.21.15.16:0/112931458 learned_addr learned my addr 172.21.15.16:0/112931458 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:14:15.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.421+0000 7f99b2664640 1 --2- 172.21.15.16:0/112931458 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561f24f43a90 0x561f24f40dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:15.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.421+0000 7f99b1662640 1 --2- 172.21.15.16:0/112931458 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561f24f34ca0 0x561f24f40770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:15.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.421+0000 7f99b1e63640 1 -- 172.21.15.16:0/112931458 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561f24f34ca0 msgr2=0x561f24f40770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:14:15.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.421+0000 7f99b1e63640 1 --2- 172.21.15.16:0/112931458 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561f24f34ca0 0x561f24f40770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:15.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.421+0000 7f99b1e63640 1 -- 172.21.15.16:0/112931458 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561f24f43a90 msgr2=0x561f24f40dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:14:15.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.421+0000 7f99b1e63640 1 --2- 172.21.15.16:0/112931458 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561f24f43a90 0x561f24f40dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:15.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.421+0000 7f99b1e63640 1 -- 172.21.15.16:0/112931458 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f24c0b3c0 con 0x561f24f34380 2024-09-12T02:14:15.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.421+0000 7f99b1662640 1 --2- 172.21.15.16:0/112931458 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561f24f34ca0 0x561f24f40770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:14:15.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.421+0000 7f99b1e63640 1 --2- 172.21.15.16:0/112931458 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561f24f34380 0x561f24f43470 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7f99a809b980 tx=0x7f99a809c1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbc9dc661424f6f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:14:15.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.421+0000 7f99957fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:14:15.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.421+0000 7f99957fa640 1 -- 172.21.15.16:0/112931458 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99a80a4660 con 0x561f24f34380 2024-09-12T02:14:15.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.421+0000 7f99957fa640 1 -- 172.21.15.16:0/112931458 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f99a80a41d0 con 0x561f24f34380 2024-09-12T02:14:15.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.421+0000 7f99b64a0740 1 -- 172.21.15.16:0/112931458 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f99a809b9f0 con 0x561f24f34380 2024-09-12T02:14:15.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.421+0000 7f99957fa640 1 -- 172.21.15.16:0/112931458 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f99a8018e20 con 0x561f24f34380 2024-09-12T02:14:15.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.422+0000 7f99b64a0740 10 client.19578 fetch_fsmap learned FSMap version 25 2024-09-12T02:14:15.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.422+0000 7f99b64a0740 1 -- 172.21.15.16:0/112931458 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x561f24f52130 con 0x561f24f34380 2024-09-12T02:14:15.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.422+0000 7f99957fa640 1 -- 172.21.15.16:0/112931458 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99a809b9f0 con 0x561f24f34380 2024-09-12T02:14:15.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.422+0000 7f99957fa640 1 -- 172.21.15.16:0/112931458 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f99a8086270 con 0x561f24f34380 2024-09-12T02:14:15.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.422+0000 7f99b64a0740 10 client.19578 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:14:15.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.422+0000 7f99b64a0740 10 client.19578 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:14:15.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.422+0000 7f99b64a0740 20 client.19578 populate_metadata read hostname 'smithi016' 2024-09-12T02:14:15.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.422+0000 7f99b64a0740 1 --2- 172.21.15.16:0/112931458 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x561f24f53480 0x561f24f55870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:15.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.422+0000 7f99b64a0740 4 client.19578 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:14:15.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.422+0000 7f99b64a0740 1 -- 172.21.15.16:0/112931458 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561f24ea7850 con 0x561f24f53480 2024-09-12T02:14:15.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.422+0000 7f99b1662640 1 --2- 172.21.15.16:0/112931458 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x561f24f53480 0x561f24f55870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:15.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.423+0000 7f99b1662640 1 --2- 172.21.15.16:0/112931458 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x561f24f53480 0x561f24f55870 crc :-1 s=READY pgs=374 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa5d66d50aa17483 server_cookie=2d44916d2c3d823d in_seq=0 out_seq=0 2024-09-12T02:14:15.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.423+0000 7f99957fa640 10 client.19578 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:14:15.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.423+0000 7f99957fa640 1 -- 172.21.15.16:0/112931458 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561f24ea7850 con 0x561f24f53480 2024-09-12T02:14:15.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.423+0000 7f99957fa640 10 client.19578 handle_command_reply: tid=0 2024-09-12T02:14:15.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.424+0000 7f99b64a0740 1 client.19578 shutdown 2024-09-12T02:14:15.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.424+0000 7f99b64a0740 1 -- 172.21.15.16:0/112931458 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561f24f34380 msgr2=0x561f24f43470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:14:15.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.424+0000 7f99b64a0740 1 --2- 172.21.15.16:0/112931458 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561f24f34380 0x561f24f43470 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7f99a809b980 tx=0x7f99a809c1b0 comp rx=0 tx=0).stop 2024-09-12T02:14:15.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.424+0000 7f99b64a0740 1 -- 172.21.15.16:0/112931458 shutdown_connections 2024-09-12T02:14:15.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.424+0000 7f99b64a0740 1 --2- 172.21.15.16:0/112931458 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x561f24f53480 0x561f24f55870 crc :-1 s=READY pgs=374 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:15.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.424+0000 7f99b64a0740 1 --2- 172.21.15.16:0/112931458 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561f24f43a90 0x561f24f40dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:15.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.424+0000 7f99b64a0740 1 --2- 172.21.15.16:0/112931458 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561f24f34ca0 0x561f24f40770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:15.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.424+0000 7f99957fa640 0 client.19578 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:14:15.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.424+0000 7f99b64a0740 1 --2- 172.21.15.16:0/112931458 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561f24f34380 0x561f24f43470 unknown :-1 s=CLOSED pgs=1733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:15.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.424+0000 7f99b64a0740 1 -- 172.21.15.16:0/112931458 >> 172.21.15.16:0/112931458 conn(0x561f24f30a30 msgr2=0x561f24f36790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:14:15.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.424+0000 7f99b64a0740 1 -- 172.21.15.16:0/112931458 shutdown_connections 2024-09-12T02:14:15.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.424+0000 7f99b64a0740 1 -- 172.21.15.16:0/112931458 wait complete. 2024-09-12T02:14:15.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.424+0000 7f99b64a0740 20 client.19578 trim_cache size 0 max 16384 2024-09-12T02:14:15.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.441+0000 7f99b64a0740 1 Processor -- start 2024-09-12T02:14:15.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.441+0000 7f99b64a0740 1 -- start start 2024-09-12T02:14:15.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.441+0000 7f99b64a0740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561f24f34380 0x561f24f4fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:15.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.441+0000 7f99b64a0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561f24f34ca0 0x561f24f58890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:15.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.441+0000 7f99b64a0740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561f24f43a90 0x561f24f58dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:15.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.441+0000 7f99b64a0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f99ac1ada00 con 0x561f24f34ca0 2024-09-12T02:14:15.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.441+0000 7f99b64a0740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f99ac1ab2d0 con 0x561f24f43a90 2024-09-12T02:14:15.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.441+0000 7f99b64a0740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f99ac1a9120 con 0x561f24f34380 2024-09-12T02:14:15.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.441+0000 7f99b1662640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561f24f34ca0 0x561f24f58890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:15.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.441+0000 7f99b1662640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561f24f34ca0 0x561f24f58890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53280/0 (socket says 172.21.15.16:53280) 2024-09-12T02:14:15.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.441+0000 7f99b1662640 1 -- 172.21.15.16:0/216215704 learned_addr learned my addr 172.21.15.16:0/216215704 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:14:15.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.442+0000 7f99b1662640 1 -- 172.21.15.16:0/216215704 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561f24f34380 msgr2=0x561f24f4fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:14:15.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.442+0000 7f99b1662640 1 --2- 172.21.15.16:0/216215704 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561f24f34380 0x561f24f4fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:15.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.442+0000 7f99b1662640 1 -- 172.21.15.16:0/216215704 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561f24f43a90 msgr2=0x561f24f58dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:14:15.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.442+0000 7f99b1662640 1 --2- 172.21.15.16:0/216215704 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561f24f43a90 0x561f24f58dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:15.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.442+0000 7f99b1662640 1 -- 172.21.15.16:0/216215704 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f24f3ecc0 con 0x561f24f34ca0 2024-09-12T02:14:15.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.442+0000 7f99b1662640 1 --2- 172.21.15.16:0/216215704 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561f24f34ca0 0x561f24f58890 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7f999c009900 tx=0x7f999c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb0e45c83fd9470e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:14:15.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.442+0000 7f99977fe640 1 -- 172.21.15.16:0/216215704 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f999c035040 con 0x561f24f34ca0 2024-09-12T02:14:15.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.442+0000 7f99977fe640 1 -- 172.21.15.16:0/216215704 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f999c0047a0 con 0x561f24f34ca0 2024-09-12T02:14:15.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.442+0000 7f99977fe640 1 -- 172.21.15.16:0/216215704 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f999c003a10 con 0x561f24f34ca0 2024-09-12T02:14:15.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.443+0000 7f99b64a0740 1 -- 172.21.15.16:0/216215704 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561f24f34ca0 msgr2=0x561f24f58890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:14:15.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.443+0000 7f99b64a0740 1 --2- 172.21.15.16:0/216215704 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561f24f34ca0 0x561f24f58890 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7f999c009900 tx=0x7f999c000bc0 comp rx=0 tx=0).stop 2024-09-12T02:14:15.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.443+0000 7f99b64a0740 1 -- 172.21.15.16:0/216215704 shutdown_connections 2024-09-12T02:14:15.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.443+0000 7f99b64a0740 1 --2- 172.21.15.16:0/216215704 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561f24f43a90 0x561f24f58dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:15.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.443+0000 7f99b64a0740 1 --2- 172.21.15.16:0/216215704 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561f24f34ca0 0x561f24f58890 unknown :-1 s=CLOSED pgs=1734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:15.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.443+0000 7f99b64a0740 1 --2- 172.21.15.16:0/216215704 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561f24f34380 0x561f24f4fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:15.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.443+0000 7f99b64a0740 1 -- 172.21.15.16:0/216215704 >> 172.21.15.16:0/216215704 conn(0x561f24f30a30 msgr2=0x561f24f3bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:14:15.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.443+0000 7f99b64a0740 1 -- 172.21.15.16:0/216215704 shutdown_connections 2024-09-12T02:14:15.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.443+0000 7f99b64a0740 1 -- 172.21.15.16:0/216215704 wait complete. 2024-09-12T02:14:15.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.443+0000 7f99b64a0740 1 -- start start 2024-09-12T02:14:15.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.443+0000 7f99b64a0740 1 Processor -- start 2024-09-12T02:14:15.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.444+0000 7f99b64a0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561f24f34380 0x561f24f56640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:15.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.444+0000 7f99b64a0740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561f24f34ca0 0x561f24f56b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:15.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.444+0000 7f99b64a0740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561f24f43a90 0x561f24f570c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:15.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.444+0000 7f99b64a0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f99ac1a9790 con 0x561f24f34380 2024-09-12T02:14:15.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.444+0000 7f99b64a0740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f99ac1aac00 con 0x561f24f34ca0 2024-09-12T02:14:15.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.444+0000 7f99b64a0740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f99ac1a9e60 con 0x561f24f43a90 2024-09-12T02:14:15.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.444+0000 7f99b64a0740 10 client.0 _ll_register_callbacks cb 0x7f99a4009800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:14:15.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.444+0000 7f99b1e63640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561f24f34380 0x561f24f56640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:15.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.444+0000 7f99b1662640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561f24f34ca0 0x561f24f56b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:15.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.444+0000 7f99b1e63640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561f24f34380 0x561f24f56640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53284/0 (socket says 172.21.15.16:53284) 2024-09-12T02:14:15.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.444+0000 7f99b1e63640 1 -- 172.21.15.16:0/2471569012 learned_addr learned my addr 172.21.15.16:0/2471569012 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:14:15.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.444+0000 7f99b2664640 1 --2- 172.21.15.16:0/2471569012 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561f24f43a90 0x561f24f570c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:15.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.444+0000 7f99b1662640 1 -- 172.21.15.16:0/2471569012 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561f24f43a90 msgr2=0x561f24f570c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:14:15.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.444+0000 7f99b1662640 1 --2- 172.21.15.16:0/2471569012 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561f24f43a90 0x561f24f570c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:15.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.444+0000 7f99b1662640 1 -- 172.21.15.16:0/2471569012 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561f24f34380 msgr2=0x561f24f56640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:14:15.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.444+0000 7f99b1662640 1 --2- 172.21.15.16:0/2471569012 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561f24f34380 0x561f24f56640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:15.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.444+0000 7f99b1662640 1 -- 172.21.15.16:0/2471569012 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f24fbcdc0 con 0x561f24f34ca0 2024-09-12T02:14:15.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.444+0000 7f99b1662640 1 --2- 172.21.15.16:0/2471569012 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561f24f34ca0 0x561f24f56b80 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x561f24f520d0 tx=0x7f999c004b30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ae03d3f84299013 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:14:15.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.444+0000 7f99957fa640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-12T02:14:15.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.444+0000 7f99957fa640 1 -- 172.21.15.16:0/2471569012 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f999c054070 con 0x561f24f34ca0 2024-09-12T02:14:15.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.445+0000 7f99b64a0740 1 -- 172.21.15.16:0/2471569012 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f999c0047a0 con 0x561f24f34ca0 2024-09-12T02:14:15.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.445+0000 7f99957fa640 1 -- 172.21.15.16:0/2471569012 <== mon.2 v2:172.21.15.33:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f999c004f70 con 0x561f24f34ca0 2024-09-12T02:14:15.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.445+0000 7f99957fa640 1 -- 172.21.15.16:0/2471569012 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f999c035040 con 0x561f24f34ca0 2024-09-12T02:14:15.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.445+0000 7f99b64a0740 10 client.28520 fetch_fsmap learned FSMap version 25 2024-09-12T02:14:15.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.445+0000 7f99b64a0740 1 -- 172.21.15.16:0/2471569012 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x561f24fbe120 con 0x561f24f34ca0 2024-09-12T02:14:15.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.445+0000 7f99957fa640 1 -- 172.21.15.16:0/2471569012 <== mon.2 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f999c0047a0 con 0x561f24f34ca0 2024-09-12T02:14:15.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.445+0000 7f99957fa640 1 -- 172.21.15.16:0/2471569012 <== mon.2 v2:172.21.15.33:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f999c057020 con 0x561f24f34ca0 2024-09-12T02:14:15.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.445+0000 7f99b64a0740 10 client.28520 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:14:15.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.445+0000 7f99b64a0740 10 client.28520 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:14:15.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.445+0000 7f99b64a0740 20 client.28520 populate_metadata read hostname 'smithi016' 2024-09-12T02:14:15.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.445+0000 7f99b64a0740 1 --2- 172.21.15.16:0/2471569012 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x561f24fbe8c0 0x561f24fc0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:15.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.445+0000 7f99b64a0740 4 client.28520 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T02:14:15.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.445+0000 7f99b64a0740 1 -- 172.21.15.16:0/2471569012 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561f24ea1d60 con 0x561f24fbe8c0 2024-09-12T02:14:15.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.445+0000 7f99b1e63640 1 --2- 172.21.15.16:0/2471569012 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x561f24fbe8c0 0x561f24fc0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:15.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.446+0000 7f99b1e63640 1 --2- 172.21.15.16:0/2471569012 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x561f24fbe8c0 0x561f24fc0cb0 crc :-1 s=READY pgs=375 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15041ada2cede9df server_cookie=88ab90754c85f88d in_seq=0 out_seq=0 2024-09-12T02:14:15.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.446+0000 7f99957fa640 10 client.28520 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:14:15.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.446+0000 7f99957fa640 1 -- 172.21.15.16:0/2471569012 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x561f24ea1d60 con 0x561f24fbe8c0 2024-09-12T02:14:15.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.446+0000 7f99957fa640 10 client.28520 handle_command_reply: tid=0 2024-09-12T02:14:15.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.446+0000 7f99b64a0740 1 client.28520 shutdown 2024-09-12T02:14:15.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.446+0000 7f99b64a0740 1 -- 172.21.15.16:0/2471569012 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561f24f34ca0 msgr2=0x561f24f56b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:14:15.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.446+0000 7f99b64a0740 1 --2- 172.21.15.16:0/2471569012 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561f24f34ca0 0x561f24f56b80 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x561f24f520d0 tx=0x7f999c004b30 comp rx=0 tx=0).stop 2024-09-12T02:14:15.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.446+0000 7f99b64a0740 1 -- 172.21.15.16:0/2471569012 shutdown_connections 2024-09-12T02:14:15.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.446+0000 7f99b64a0740 1 --2- 172.21.15.16:0/2471569012 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x561f24fbe8c0 0x561f24fc0cb0 crc :-1 s=READY pgs=375 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:15.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.446+0000 7f99b64a0740 1 --2- 172.21.15.16:0/2471569012 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561f24f43a90 0x561f24f570c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:15.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.446+0000 7f99b64a0740 1 --2- 172.21.15.16:0/2471569012 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561f24f34ca0 0x561f24f56b80 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:15.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.446+0000 7f99b64a0740 1 --2- 172.21.15.16:0/2471569012 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561f24f34380 0x561f24f56640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:15.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.446+0000 7f99b64a0740 1 -- 172.21.15.16:0/2471569012 >> 172.21.15.16:0/2471569012 conn(0x561f24f30a30 msgr2=0x561f24f2f350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:14:15.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.446+0000 7f99957fa640 0 client.28520 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:14:15.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.446+0000 7f99b64a0740 1 -- 172.21.15.16:0/2471569012 shutdown_connections 2024-09-12T02:14:15.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.447+0000 7f99b64a0740 1 -- 172.21.15.16:0/2471569012 wait complete. 2024-09-12T02:14:15.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.447+0000 7f99b64a0740 20 client.28520 trim_cache size 0 max 16384 2024-09-12T02:14:15.453 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T02:14:15.453 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-09-12T02:14:15.453 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {} 2024-09-12T02:14:15.453 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T02:14:15.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.447+0000 7f99b40c7640 1 -- 172.21.15.16:0/1877463645 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f99980a2b80 msgr2=0x7f99980a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:14:15.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.447+0000 7f99b40c7640 1 --2- 172.21.15.16:0/1877463645 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f99980a2b80 0x7f99980a5040 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f99a4014c00 tx=0x7f99a4004ab0 comp rx=0 tx=0).stop 2024-09-12T02:14:15.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.447+0000 7f99b40c7640 1 -- 172.21.15.16:0/1877463645 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99ac100e20 msgr2=0x7f99ac198450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:14:15.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.447+0000 7f99b40c7640 1 --2- 172.21.15.16:0/1877463645 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99ac100e20 0x7f99ac198450 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7f99a800c920 tx=0x7f99a800cde0 comp rx=0 tx=0).stop 2024-09-12T02:14:15.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.447+0000 7f99b40c7640 1 -- 172.21.15.16:0/1877463645 shutdown_connections 2024-09-12T02:14:15.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.447+0000 7f99b40c7640 1 --2- 172.21.15.16:0/1877463645 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f99980a2b80 0x7f99980a5040 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:15.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.447+0000 7f99b40c7640 1 --2- 172.21.15.16:0/1877463645 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f99ac102ef0 0x7f99ac19cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:15.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.447+0000 7f99b40c7640 1 --2- 172.21.15.16:0/1877463645 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99ac102570 0x7f99ac198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:15.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.447+0000 7f99b40c7640 1 --2- 172.21.15.16:0/1877463645 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99ac100e20 0x7f99ac198450 unknown :-1 s=CLOSED pgs=1732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:15.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.447+0000 7f99b40c7640 1 -- 172.21.15.16:0/1877463645 >> 172.21.15.16:0/1877463645 conn(0x7f99ac0fc7d0 msgr2=0x7f99ac06a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:14:15.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.447+0000 7f99b40c7640 1 -- 172.21.15.16:0/1877463645 shutdown_connections 2024-09-12T02:14:15.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:15.447+0000 7f99b40c7640 1 -- 172.21.15.16:0/1877463645 wait complete. 2024-09-12T02:14:16.183 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-12T02:14:16.183 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:14:16.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:14:16 smithi033 ceph-mon[35332]: pgmap v705: 97 pgs: 97 active+clean; 2.4 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 6.3 KiB/s wr, 9 op/s 2024-09-12T02:14:16.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:14:16 smithi016 ceph-mon[30947]: pgmap v705: 97 pgs: 97 active+clean; 2.4 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 6.3 KiB/s wr, 9 op/s 2024-09-12T02:14:16.444 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:14:16.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:14:16 smithi184 ceph-mon[35593]: pgmap v705: 97 pgs: 97 active+clean; 2.4 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 6.3 KiB/s wr, 9 op/s 2024-09-12T02:14:16.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:16.898+0000 7f67cd193640 1 -- 172.21.15.16:0/2614691757 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67c8100a80 msgr2=0x7f67c8100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:14:16.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:16.898+0000 7f67cd193640 1 --2- 172.21.15.16:0/2614691757 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67c8100a80 0x7f67c8100ee0 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7f67b0009930 tx=0x7f67b002a420 comp rx=0 tx=0).stop 2024-09-12T02:14:16.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:16.898+0000 7f67cd193640 1 -- 172.21.15.16:0/2614691757 shutdown_connections 2024-09-12T02:14:16.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:16.898+0000 7f67cd193640 1 --2- 172.21.15.16:0/2614691757 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f67c8101420 0x7f67c810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:16.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:16.898+0000 7f67cd193640 1 --2- 172.21.15.16:0/2614691757 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67c8100a80 0x7f67c8100ee0 unknown :-1 s=CLOSED pgs=1735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:16.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:16.898+0000 7f67cd193640 1 --2- 172.21.15.16:0/2614691757 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f67c8106a80 0x7f67c8106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:16.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:16.898+0000 7f67cd193640 1 -- 172.21.15.16:0/2614691757 >> 172.21.15.16:0/2614691757 conn(0x7f67c80fc7a0 msgr2=0x7f67c80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:14:16.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:16.898+0000 7f67cd193640 1 -- 172.21.15.16:0/2614691757 shutdown_connections 2024-09-12T02:14:16.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:16.898+0000 7f67cd193640 1 -- 172.21.15.16:0/2614691757 wait complete. 2024-09-12T02:14:16.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:16.899+0000 7f67cd193640 1 Processor -- start 2024-09-12T02:14:16.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:16.899+0000 7f67cd193640 1 -- start start 2024-09-12T02:14:16.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:16.899+0000 7f67cd193640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67c8100a80 0x7f67c80fcdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:16.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:16.899+0000 7f67cd193640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f67c8101420 0x7f67c80fd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:16.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:16.899+0000 7f67cd193640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f67c8106a80 0x7f67c80fd830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:16.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:16.899+0000 7f67cd193640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f67c8110ed0 con 0x7f67c8100a80 2024-09-12T02:14:16.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:16.899+0000 7f67cd193640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f67c8110d50 con 0x7f67c8106a80 2024-09-12T02:14:16.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:16.899+0000 7f67cd193640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f67c8111050 con 0x7f67c8101420 2024-09-12T02:14:16.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:16.899+0000 7f67c6d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67c8100a80 0x7f67c80fcdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:16.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:16.899+0000 7f67c6d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67c8100a80 0x7f67c80fcdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53302/0 (socket says 172.21.15.16:53302) 2024-09-12T02:14:16.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:16.899+0000 7f67c6d9d640 1 -- 172.21.15.16:0/1928296672 learned_addr learned my addr 172.21.15.16:0/1928296672 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:14:16.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:16.900+0000 7f67c759e640 1 --2- 172.21.15.16:0/1928296672 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f67c8106a80 0x7f67c80fd830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:16.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:16.900+0000 7f67bffff640 1 --2- 172.21.15.16:0/1928296672 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f67c8101420 0x7f67c80fd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:16.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:16.900+0000 7f67c6d9d640 1 -- 172.21.15.16:0/1928296672 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f67c8101420 msgr2=0x7f67c80fd2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:14:16.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:16.900+0000 7f67c6d9d640 1 --2- 172.21.15.16:0/1928296672 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f67c8101420 0x7f67c80fd2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:16.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:16.900+0000 7f67c6d9d640 1 -- 172.21.15.16:0/1928296672 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f67c8106a80 msgr2=0x7f67c80fd830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:14:16.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:16.900+0000 7f67c6d9d640 1 --2- 172.21.15.16:0/1928296672 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f67c8106a80 0x7f67c80fd830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:16.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:16.900+0000 7f67c6d9d640 1 -- 172.21.15.16:0/1928296672 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67c81a6970 con 0x7f67c8100a80 2024-09-12T02:14:16.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:16.900+0000 7f67bffff640 1 --2- 172.21.15.16:0/1928296672 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f67c8101420 0x7f67c80fd2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:14:16.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:16.900+0000 7f67c6d9d640 1 --2- 172.21.15.16:0/1928296672 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67c8100a80 0x7f67c80fcdb0 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7f67b800d770 tx=0x7f67b800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:14:16.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:16.900+0000 7f67c4d99640 1 -- 172.21.15.16:0/1928296672 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67b8017070 con 0x7f67c8100a80 2024-09-12T02:14:16.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:16.900+0000 7f67c4d99640 1 -- 172.21.15.16:0/1928296672 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67b8005500 con 0x7f67c8100a80 2024-09-12T02:14:16.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:16.900+0000 7f67cd193640 1 -- 172.21.15.16:0/1928296672 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67c81a6c60 con 0x7f67c8100a80 2024-09-12T02:14:16.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:16.900+0000 7f67c4d99640 1 -- 172.21.15.16:0/1928296672 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67b8004e00 con 0x7f67c8100a80 2024-09-12T02:14:16.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:16.900+0000 7f67cd193640 1 -- 172.21.15.16:0/1928296672 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67c81a71a0 con 0x7f67c8100a80 2024-09-12T02:14:16.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:16.901+0000 7f67cd193640 1 -- 172.21.15.16:0/1928296672 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67c8100ee0 con 0x7f67c8100a80 2024-09-12T02:14:16.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:16.902+0000 7f67c4d99640 1 -- 172.21.15.16:0/1928296672 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f67b8004fa0 con 0x7f67c8100a80 2024-09-12T02:14:16.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:16.903+0000 7f67c4d99640 1 --2- 172.21.15.16:0/1928296672 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f67b40a2c00 0x7f67b40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:16.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:16.903+0000 7f67bffff640 1 --2- 172.21.15.16:0/1928296672 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f67b40a2c00 0x7f67b40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:16.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:16.903+0000 7f67c4d99640 1 -- 172.21.15.16:0/1928296672 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f67b80ce9b0 con 0x7f67c8100a80 2024-09-12T02:14:16.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:16.904+0000 7f67bffff640 1 --2- 172.21.15.16:0/1928296672 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f67b40a2c00 0x7f67b40a50c0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f67b002a940 tx=0x7f67b0005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:14:16.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:16.907+0000 7f67c4d99640 1 -- 172.21.15.16:0/1928296672 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f67b809b9b0 con 0x7f67c8100a80 2024-09-12T02:14:17.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:17.118+0000 7f67cd193640 1 -- 172.21.15.16:0/1928296672 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f67c8109a70 con 0x7f67c8100a80 2024-09-12T02:14:17.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:17.119+0000 7f67c4d99640 1 -- 172.21.15.16:0/1928296672 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11911 (secure 0 0 0) 0x7f67b8095e20 con 0x7f67c8100a80 2024-09-12T02:14:17.119 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T02:14:17.119 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":25,"btime":"2024-09-12T02:01:37:192021+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3279123279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3279123279},{"type":"v1","addr":"172.21.15.16:6835","nonce":3279123279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4119452382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4119452382},{"type":"v1","addr":"172.21.15.16:6837","nonce":4119452382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/3263823165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3263823165},{"type":"v1","addr":"172.21.15.16:6839","nonce":3263823165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/1510290732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":1510290732},{"type":"v1","addr":"172.21.15.16:6841","nonce":1510290732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/279674603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":279674603},{"type":"v1","addr":"172.21.15.33:6833","nonce":279674603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3555728028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3555728028},{"type":"v1","addr":"172.21.15.33:6835","nonce":3555728028}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T01:59:56.117608+0000","modified":"2024-09-12T02:01:36.294760+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24482,"mds_1":24481,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6833/1563374106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1563374106},{"type":"v1","addr":"172.21.15.184:6833","nonce":1563374106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6837/1892833868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1892833868},{"type":"v1","addr":"172.21.15.33:6837","nonce":1892833868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/1457679686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1457679686},{"type":"v1","addr":"172.21.15.184:6835","nonce":1457679686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.184:6837/1793393087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":1793393087},{"type":"v1","addr":"172.21.15.184:6837","nonce":1793393087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.184:6839/4203347941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":4203347941},{"type":"v1","addr":"172.21.15.184:6839","nonce":4203347941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.33:6839/4136670204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":4136670204},{"type":"v1","addr":"172.21.15.33:6839","nonce":4136670204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24481,24482,24469]},"id":1}]} 2024-09-12T02:14:17.119 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 25 2024-09-12T02:14:17.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:17.121+0000 7f67cd193640 1 -- 172.21.15.16:0/1928296672 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f67b40a2c00 msgr2=0x7f67b40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:14:17.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:17.121+0000 7f67cd193640 1 --2- 172.21.15.16:0/1928296672 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f67b40a2c00 0x7f67b40a50c0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f67b002a940 tx=0x7f67b0005fd0 comp rx=0 tx=0).stop 2024-09-12T02:14:17.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:17.121+0000 7f67cd193640 1 -- 172.21.15.16:0/1928296672 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67c8100a80 msgr2=0x7f67c80fcdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:14:17.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:17.121+0000 7f67cd193640 1 --2- 172.21.15.16:0/1928296672 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67c8100a80 0x7f67c80fcdb0 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7f67b800d770 tx=0x7f67b800dc30 comp rx=0 tx=0).stop 2024-09-12T02:14:17.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:17.121+0000 7f67cd193640 1 -- 172.21.15.16:0/1928296672 shutdown_connections 2024-09-12T02:14:17.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:17.121+0000 7f67cd193640 1 --2- 172.21.15.16:0/1928296672 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f67b40a2c00 0x7f67b40a50c0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:17.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:17.121+0000 7f67cd193640 1 --2- 172.21.15.16:0/1928296672 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f67c8106a80 0x7f67c80fd830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:17.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:17.121+0000 7f67cd193640 1 --2- 172.21.15.16:0/1928296672 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f67c8101420 0x7f67c80fd2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:17.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:17.121+0000 7f67cd193640 1 --2- 172.21.15.16:0/1928296672 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67c8100a80 0x7f67c80fcdb0 unknown :-1 s=CLOSED pgs=1736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:17.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:17.121+0000 7f67cd193640 1 -- 172.21.15.16:0/1928296672 >> 172.21.15.16:0/1928296672 conn(0x7f67c80fc7a0 msgr2=0x7f67c806aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:14:17.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:17.121+0000 7f67cd193640 1 -- 172.21.15.16:0/1928296672 shutdown_connections 2024-09-12T02:14:17.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:17.121+0000 7f67cd193640 1 -- 172.21.15.16:0/1928296672 wait complete. 2024-09-12T02:14:17.419 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:14:17 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/1928296672' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:14:17.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:14:17 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/1928296672' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:14:17.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:14:17 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/1928296672' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:14:17.811 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-12T02:14:18.072 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:14:18.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:14:18 smithi016 ceph-mon[30947]: pgmap v706: 97 pgs: 97 active+clean; 2.4 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-09-12T02:14:18.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.526+0000 7fc2456d3640 1 -- 172.21.15.16:0/948800560 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2401036e0 msgr2=0x7fc240107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:14:18.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.526+0000 7fc2456d3640 1 --2- 172.21.15.16:0/948800560 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2401036e0 0x7fc240107b90 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7fc228009930 tx=0x7fc22802a420 comp rx=0 tx=0).stop 2024-09-12T02:14:18.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.527+0000 7fc2456d3640 1 -- 172.21.15.16:0/948800560 shutdown_connections 2024-09-12T02:14:18.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.527+0000 7fc2456d3640 1 --2- 172.21.15.16:0/948800560 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2401082a0 0x7fc24010ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:18.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.527+0000 7fc2456d3640 1 --2- 172.21.15.16:0/948800560 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2401036e0 0x7fc240107b90 unknown :-1 s=CLOSED pgs=1737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:18.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.527+0000 7fc2456d3640 1 --2- 172.21.15.16:0/948800560 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc240102d30 0x7fc240103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:18.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.527+0000 7fc2456d3640 1 -- 172.21.15.16:0/948800560 >> 172.21.15.16:0/948800560 conn(0x7fc2400fc7d0 msgr2=0x7fc2400febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:14:18.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.527+0000 7fc2456d3640 1 -- 172.21.15.16:0/948800560 shutdown_connections 2024-09-12T02:14:18.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.527+0000 7fc2456d3640 1 -- 172.21.15.16:0/948800560 wait complete. 2024-09-12T02:14:18.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.528+0000 7fc2456d3640 1 Processor -- start 2024-09-12T02:14:18.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.528+0000 7fc2456d3640 1 -- start start 2024-09-12T02:14:18.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.528+0000 7fc2456d3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc240102d30 0x7fc240198540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:18.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.528+0000 7fc2456d3640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc2401036e0 0x7fc240198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:18.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.528+0000 7fc2456d3640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2401082a0 0x7fc24019ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:18.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.528+0000 7fc2456d3640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc240112060 con 0x7fc240102d30 2024-09-12T02:14:18.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.528+0000 7fc2456d3640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc240111ee0 con 0x7fc2401082a0 2024-09-12T02:14:18.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.528+0000 7fc2456d3640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc2401121e0 con 0x7fc2401036e0 2024-09-12T02:14:18.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.528+0000 7fc23effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc240102d30 0x7fc240198540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:18.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.528+0000 7fc23effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc240102d30 0x7fc240198540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53318/0 (socket says 172.21.15.16:53318) 2024-09-12T02:14:18.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.528+0000 7fc23effd640 1 -- 172.21.15.16:0/3228845020 learned_addr learned my addr 172.21.15.16:0/3228845020 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:14:18.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.529+0000 7fc23f7fe640 1 --2- 172.21.15.16:0/3228845020 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2401082a0 0x7fc24019ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:18.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.529+0000 7fc23e7fc640 1 --2- 172.21.15.16:0/3228845020 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc2401036e0 0x7fc240198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:18.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.529+0000 7fc23effd640 1 -- 172.21.15.16:0/3228845020 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc2401036e0 msgr2=0x7fc240198a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:14:18.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.529+0000 7fc23effd640 1 --2- 172.21.15.16:0/3228845020 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc2401036e0 0x7fc240198a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:18.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.529+0000 7fc23effd640 1 -- 172.21.15.16:0/3228845020 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2401082a0 msgr2=0x7fc24019ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:14:18.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.529+0000 7fc23effd640 1 --2- 172.21.15.16:0/3228845020 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2401082a0 0x7fc24019ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:18.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.529+0000 7fc23effd640 1 -- 172.21.15.16:0/3228845020 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc24019d4f0 con 0x7fc240102d30 2024-09-12T02:14:18.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.529+0000 7fc23effd640 1 --2- 172.21.15.16:0/3228845020 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc240102d30 0x7fc240198540 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7fc23800cce0 tx=0x7fc23800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:14:18.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.529+0000 7fc237fff640 1 -- 172.21.15.16:0/3228845020 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc23800f650 con 0x7fc240102d30 2024-09-12T02:14:18.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.529+0000 7fc237fff640 1 -- 172.21.15.16:0/3228845020 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc23800fcd0 con 0x7fc240102d30 2024-09-12T02:14:18.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.529+0000 7fc2456d3640 1 -- 172.21.15.16:0/3228845020 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc24019d7e0 con 0x7fc240102d30 2024-09-12T02:14:18.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.529+0000 7fc237fff640 1 -- 172.21.15.16:0/3228845020 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc238018c70 con 0x7fc240102d30 2024-09-12T02:14:18.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.529+0000 7fc2456d3640 1 -- 172.21.15.16:0/3228845020 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2401a50c0 con 0x7fc240102d30 2024-09-12T02:14:18.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.531+0000 7fc237fff640 1 -- 172.21.15.16:0/3228845020 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fc23800f7f0 con 0x7fc240102d30 2024-09-12T02:14:18.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.532+0000 7fc247aac740 1 Processor -- start 2024-09-12T02:14:18.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.532+0000 7fc247aac740 1 -- start start 2024-09-12T02:14:18.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.532+0000 7fc247aac740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636e71c9b40 0x5636e71c8090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:18.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.532+0000 7fc247aac740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636e71c85d0 0x5636e71c8a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:18.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.532+0000 7fc237fff640 1 --2- 172.21.15.16:0/3228845020 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc22c0a2b80 0x7fc22c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:18.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.532+0000 7fc247aac740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5636e71d3370 0x5636e71d5760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:18.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.532+0000 7fc247aac740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5636e712e2f0 con 0x5636e71c85d0 2024-09-12T02:14:18.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.532+0000 7fc247aac740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5636e70d5cf0 con 0x5636e71c9b40 2024-09-12T02:14:18.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.532+0000 7fc247aac740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5636e712a2b0 con 0x5636e71d3370 2024-09-12T02:14:18.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.533+0000 7fc23e7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636e71c85d0 0x5636e71c8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:18.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.533+0000 7fc23e7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636e71c85d0 0x5636e71c8a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53330/0 (socket says 172.21.15.16:53330) 2024-09-12T02:14:18.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.533+0000 7fc23e7fc640 1 -- 172.21.15.16:0/2363840419 learned_addr learned my addr 172.21.15.16:0/2363840419 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:14:18.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.533+0000 7fc237fff640 1 -- 172.21.15.16:0/3228845020 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fc2380cf3a0 con 0x7fc240102d30 2024-09-12T02:14:18.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.533+0000 7fc23f7fe640 1 --2- 172.21.15.16:0/2363840419 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636e71c9b40 0x5636e71c8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:18.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.533+0000 7fc23effd640 1 --2- 172.21.15.16:0/3228845020 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc22c0a2b80 0x7fc22c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:18.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.533+0000 7fc23f7fe640 1 --2- 172.21.15.16:0/2363840419 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5636e71d3370 0x5636e71d5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:18.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.533+0000 7fc23e7fc640 1 -- 172.21.15.16:0/2363840419 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5636e71d3370 msgr2=0x5636e71d5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:14:18.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.533+0000 7fc23e7fc640 1 --2- 172.21.15.16:0/2363840419 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5636e71d3370 0x5636e71d5760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:18.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.533+0000 7fc23e7fc640 1 -- 172.21.15.16:0/2363840419 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636e71c9b40 msgr2=0x5636e71c8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:14:18.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.533+0000 7fc23e7fc640 1 --2- 172.21.15.16:0/2363840419 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636e71c9b40 0x5636e71c8090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:18.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.533+0000 7fc23e7fc640 1 -- 172.21.15.16:0/2363840419 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636e70ba3a0 con 0x5636e71c85d0 2024-09-12T02:14:18.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.533+0000 7fc23effd640 1 --2- 172.21.15.16:0/3228845020 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc22c0a2b80 0x7fc22c0a5040 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fc23800afd0 tx=0x7fc238007f30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:14:18.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.533+0000 7fc23e7fc640 1 --2- 172.21.15.16:0/2363840419 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636e71c85d0 0x5636e71c8a10 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7fc228009900 tx=0x7fc228032cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa398af846c32823 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:14:18.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.533+0000 7fc2357fa640 1 -- 172.21.15.16:0/2363840419 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc228035040 con 0x5636e71c85d0 2024-09-12T02:14:18.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.534+0000 7fc2357fa640 1 -- 172.21.15.16:0/2363840419 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2280317b0 con 0x5636e71c85d0 2024-09-12T02:14:18.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.534+0000 7fc2357fa640 1 -- 172.21.15.16:0/2363840419 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc228004770 con 0x5636e71c85d0 2024-09-12T02:14:18.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.534+0000 7fc247aac740 1 -- 172.21.15.16:0/2363840419 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636e71c85d0 msgr2=0x5636e71c8a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:14:18.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.534+0000 7fc247aac740 1 --2- 172.21.15.16:0/2363840419 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636e71c85d0 0x5636e71c8a10 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7fc228009900 tx=0x7fc228032cb0 comp rx=0 tx=0).stop 2024-09-12T02:14:18.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.534+0000 7fc247aac740 1 -- 172.21.15.16:0/2363840419 shutdown_connections 2024-09-12T02:14:18.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.534+0000 7fc247aac740 1 --2- 172.21.15.16:0/2363840419 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5636e71d3370 0x5636e71d5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:18.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.534+0000 7fc247aac740 1 --2- 172.21.15.16:0/2363840419 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636e71c85d0 0x5636e71c8a10 unknown :-1 s=CLOSED pgs=1739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:18.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.534+0000 7fc247aac740 1 --2- 172.21.15.16:0/2363840419 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636e71c9b40 0x5636e71c8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:18.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.534+0000 7fc247aac740 1 -- 172.21.15.16:0/2363840419 >> 172.21.15.16:0/2363840419 conn(0x5636e7191310 msgr2=0x5636e71c4c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:14:18.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.534+0000 7fc247aac740 1 -- 172.21.15.16:0/2363840419 shutdown_connections 2024-09-12T02:14:18.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.534+0000 7fc247aac740 1 -- 172.21.15.16:0/2363840419 wait complete. 2024-09-12T02:14:18.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.535+0000 7fc247aac740 1 -- start start 2024-09-12T02:14:18.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.535+0000 7fc247aac740 1 Processor -- start 2024-09-12T02:14:18.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.535+0000 7fc247aac740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5636e71c85d0 0x5636e71d2c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:18.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.535+0000 7fc247aac740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636e71c9b40 0x5636e71cfee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:18.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.535+0000 7fc247aac740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636e71d3370 0x5636e71d0420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:18.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.535+0000 7fc247aac740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5636e71d6010 con 0x5636e71c9b40 2024-09-12T02:14:18.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.535+0000 7fc247aac740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5636e70e9060 con 0x5636e71d3370 2024-09-12T02:14:18.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.535+0000 7fc247aac740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5636e71d6190 con 0x5636e71c85d0 2024-09-12T02:14:18.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.535+0000 7fc247aac740 10 client.0 _ll_register_callbacks cb 0x5636e6e3c8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:14:18.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.535+0000 7fc23e7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636e71c9b40 0x5636e71cfee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:18.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.535+0000 7fc23e7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636e71c9b40 0x5636e71cfee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53336/0 (socket says 172.21.15.16:53336) 2024-09-12T02:14:18.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.535+0000 7fc23e7fc640 1 -- 172.21.15.16:0/2323700824 learned_addr learned my addr 172.21.15.16:0/2323700824 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:14:18.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.535+0000 7fc23f7fe640 1 --2- 172.21.15.16:0/2323700824 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5636e71c85d0 0x5636e71d2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:18.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.536+0000 7fc23f7fe640 1 --2- 172.21.15.16:0/2323700824 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636e71d3370 0x5636e71d0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:18.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.536+0000 7fc23e7fc640 1 -- 172.21.15.16:0/2323700824 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5636e71c85d0 msgr2=0x5636e71d2c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:14:18.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.536+0000 7fc23e7fc640 1 --2- 172.21.15.16:0/2323700824 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5636e71c85d0 0x5636e71d2c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:18.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.536+0000 7fc23e7fc640 1 -- 172.21.15.16:0/2323700824 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636e71d3370 msgr2=0x5636e71d0420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:14:18.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.536+0000 7fc23e7fc640 1 --2- 172.21.15.16:0/2323700824 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636e71d3370 0x5636e71d0420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:18.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.536+0000 7fc23e7fc640 1 -- 172.21.15.16:0/2323700824 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636e6e9f3c0 con 0x5636e71c9b40 2024-09-12T02:14:18.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.536+0000 7fc23f7fe640 1 --2- 172.21.15.16:0/2323700824 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5636e71c85d0 0x5636e71d2c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:14:18.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.536+0000 7fc23e7fc640 1 --2- 172.21.15.16:0/2323700824 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636e71c9b40 0x5636e71cfee0 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7fc22802a940 tx=0x7fc2280315b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e0edf398c87f7b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:14:18.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.536+0000 7fc2267fc640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:14:18.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.536+0000 7fc2267fc640 1 -- 172.21.15.16:0/2323700824 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc228035030 con 0x5636e71c9b40 2024-09-12T02:14:18.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.536+0000 7fc2267fc640 1 -- 172.21.15.16:0/2323700824 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fc228004260 con 0x5636e71c9b40 2024-09-12T02:14:18.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.536+0000 7fc247aac740 1 -- 172.21.15.16:0/2323700824 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc2280317b0 con 0x5636e71c9b40 2024-09-12T02:14:18.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.536+0000 7fc2267fc640 1 -- 172.21.15.16:0/2323700824 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc228005630 con 0x5636e71c9b40 2024-09-12T02:14:18.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.536+0000 7fc247aac740 10 client.19602 fetch_fsmap learned FSMap version 25 2024-09-12T02:14:18.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.536+0000 7fc247aac740 1 -- 172.21.15.16:0/2323700824 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5636e71e19a0 con 0x5636e71c9b40 2024-09-12T02:14:18.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.536+0000 7fc2267fc640 1 -- 172.21.15.16:0/2323700824 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2280317b0 con 0x5636e71c9b40 2024-09-12T02:14:18.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.537+0000 7fc2267fc640 1 -- 172.21.15.16:0/2323700824 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fc22803c740 con 0x5636e71c9b40 2024-09-12T02:14:18.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.537+0000 7fc247aac740 10 client.19602 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:14:18.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.537+0000 7fc247aac740 10 client.19602 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-12T02:14:18.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.537+0000 7fc247aac740 20 client.19602 populate_metadata read hostname 'smithi016' 2024-09-12T02:14:18.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.537+0000 7fc247aac740 1 --2- 172.21.15.16:0/2323700824 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x5636e71e2cf0 0x5636e71e50e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:18.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.537+0000 7fc247aac740 4 client.19602 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:14:18.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.537+0000 7fc247aac740 1 -- 172.21.15.16:0/2323700824 --> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5636e713b860 con 0x5636e71e2cf0 2024-09-12T02:14:18.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.537+0000 7fc23f7fe640 1 --2- 172.21.15.16:0/2323700824 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x5636e71e2cf0 0x5636e71e50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:18.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.538+0000 7fc23f7fe640 1 --2- 172.21.15.16:0/2323700824 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x5636e71e2cf0 0x5636e71e50e0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9a84449a59594e33 server_cookie=8b73b20d03907958 in_seq=0 out_seq=0 2024-09-12T02:14:18.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.538+0000 7fc2267fc640 10 client.19602 ms_handle_connect on v2:172.21.15.184:6836/1793393087 2024-09-12T02:14:18.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.538+0000 7fc2267fc640 1 -- 172.21.15.16:0/2323700824 <== mds.2 v2:172.21.15.184:6836/1793393087 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5636e713b860 con 0x5636e71e2cf0 2024-09-12T02:14:18.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.538+0000 7fc2267fc640 10 client.19602 handle_command_reply: tid=0 2024-09-12T02:14:18.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.538+0000 7fc247aac740 1 client.19602 shutdown 2024-09-12T02:14:18.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.539+0000 7fc247aac740 1 -- 172.21.15.16:0/2323700824 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636e71c9b40 msgr2=0x5636e71cfee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:14:18.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.539+0000 7fc247aac740 1 --2- 172.21.15.16:0/2323700824 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636e71c9b40 0x5636e71cfee0 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7fc22802a940 tx=0x7fc2280315b0 comp rx=0 tx=0).stop 2024-09-12T02:14:18.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.539+0000 7fc247aac740 1 -- 172.21.15.16:0/2323700824 shutdown_connections 2024-09-12T02:14:18.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.539+0000 7fc247aac740 1 --2- 172.21.15.16:0/2323700824 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636e71d3370 0x5636e71d0420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:18.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.539+0000 7fc247aac740 1 --2- 172.21.15.16:0/2323700824 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x5636e71e2cf0 0x5636e71e50e0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:18.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.539+0000 7fc2267fc640 0 client.19602 ms_handle_reset on v2:172.21.15.184:6836/1793393087 2024-09-12T02:14:18.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.539+0000 7fc247aac740 1 --2- 172.21.15.16:0/2323700824 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636e71c9b40 0x5636e71cfee0 unknown :-1 s=CLOSED pgs=1740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:18.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.539+0000 7fc247aac740 1 --2- 172.21.15.16:0/2323700824 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5636e71c85d0 0x5636e71d2c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:18.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.539+0000 7fc247aac740 1 -- 172.21.15.16:0/2323700824 >> 172.21.15.16:0/2323700824 conn(0x5636e7191310 msgr2=0x5636e71ca930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:14:18.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.539+0000 7fc247aac740 1 -- 172.21.15.16:0/2323700824 shutdown_connections 2024-09-12T02:14:18.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.539+0000 7fc247aac740 1 -- 172.21.15.16:0/2323700824 wait complete. 2024-09-12T02:14:18.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.539+0000 7fc247aac740 20 client.19602 trim_cache size 0 max 16384 2024-09-12T02:14:18.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.552+0000 7fc247aac740 1 Processor -- start 2024-09-12T02:14:18.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.552+0000 7fc247aac740 1 -- start start 2024-09-12T02:14:18.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.552+0000 7fc247aac740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636e71c85d0 0x5636e71ceec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:18.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.552+0000 7fc247aac740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636e71c9b40 0x5636e71d4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:18.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.552+0000 7fc247aac740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5636e71d3370 0x5636e71d4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:18.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.552+0000 7fc247aac740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc2401afc00 con 0x5636e71c85d0 2024-09-12T02:14:18.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.552+0000 7fc247aac740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc2401ad4d0 con 0x5636e71c9b40 2024-09-12T02:14:18.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.552+0000 7fc247aac740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc2401ab320 con 0x5636e71d3370 2024-09-12T02:14:18.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.553+0000 7fc23f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636e71c85d0 0x5636e71ceec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:18.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.553+0000 7fc23f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636e71c85d0 0x5636e71ceec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53340/0 (socket says 172.21.15.16:53340) 2024-09-12T02:14:18.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.553+0000 7fc23f7fe640 1 -- 172.21.15.16:0/1007823464 learned_addr learned my addr 172.21.15.16:0/1007823464 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:14:18.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.553+0000 7fc23f7fe640 1 --2- 172.21.15.16:0/1007823464 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5636e71d3370 0x5636e71d4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:18.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.553+0000 7fc23e7fc640 1 --2- 172.21.15.16:0/1007823464 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636e71c9b40 0x5636e71d4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:18.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.553+0000 7fc23f7fe640 1 -- 172.21.15.16:0/1007823464 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5636e71d3370 msgr2=0x5636e71d4ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:14:18.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.553+0000 7fc23f7fe640 1 --2- 172.21.15.16:0/1007823464 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5636e71d3370 0x5636e71d4ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:18.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.553+0000 7fc23f7fe640 1 -- 172.21.15.16:0/1007823464 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636e71c9b40 msgr2=0x5636e71d4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:14:18.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.553+0000 7fc23f7fe640 1 --2- 172.21.15.16:0/1007823464 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636e71c9b40 0x5636e71d4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:18.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.553+0000 7fc23f7fe640 1 -- 172.21.15.16:0/1007823464 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636e71cbae0 con 0x5636e71c85d0 2024-09-12T02:14:18.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.553+0000 7fc23f7fe640 1 --2- 172.21.15.16:0/1007823464 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636e71c85d0 0x5636e71ceec0 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x5636e71d3340 tx=0x7fc23000b2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53e0fd8ee0cad4f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:14:18.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.553+0000 7fc234ff9640 1 -- 172.21.15.16:0/1007823464 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc230025040 con 0x5636e71c85d0 2024-09-12T02:14:18.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.553+0000 7fc234ff9640 1 -- 172.21.15.16:0/1007823464 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc230004490 con 0x5636e71c85d0 2024-09-12T02:14:18.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.553+0000 7fc234ff9640 1 -- 172.21.15.16:0/1007823464 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc230022630 con 0x5636e71c85d0 2024-09-12T02:14:18.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.554+0000 7fc247aac740 1 -- 172.21.15.16:0/1007823464 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636e71c85d0 msgr2=0x5636e71ceec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:14:18.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.554+0000 7fc247aac740 1 --2- 172.21.15.16:0/1007823464 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636e71c85d0 0x5636e71ceec0 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x5636e71d3340 tx=0x7fc23000b2e0 comp rx=0 tx=0).stop 2024-09-12T02:14:18.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.554+0000 7fc247aac740 1 -- 172.21.15.16:0/1007823464 shutdown_connections 2024-09-12T02:14:18.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.554+0000 7fc247aac740 1 --2- 172.21.15.16:0/1007823464 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5636e71d3370 0x5636e71d4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:18.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.554+0000 7fc247aac740 1 --2- 172.21.15.16:0/1007823464 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636e71c9b40 0x5636e71d4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:18.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.554+0000 7fc247aac740 1 --2- 172.21.15.16:0/1007823464 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636e71c85d0 0x5636e71ceec0 unknown :-1 s=CLOSED pgs=1741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:18.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.554+0000 7fc247aac740 1 -- 172.21.15.16:0/1007823464 >> 172.21.15.16:0/1007823464 conn(0x5636e7191310 msgr2=0x5636e71ce7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:14:18.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.554+0000 7fc247aac740 1 -- 172.21.15.16:0/1007823464 shutdown_connections 2024-09-12T02:14:18.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.554+0000 7fc247aac740 1 -- 172.21.15.16:0/1007823464 wait complete. 2024-09-12T02:14:18.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.554+0000 7fc247aac740 1 -- start start 2024-09-12T02:14:18.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.555+0000 7fc247aac740 1 Processor -- start 2024-09-12T02:14:18.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.555+0000 7fc247aac740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636e71c85d0 0x5636e71e84d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:18.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.555+0000 7fc247aac740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5636e71c9b40 0x5636e71e8a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:18.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.555+0000 7fc247aac740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636e71d3370 0x5636e71e3a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:18.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.555+0000 7fc247aac740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc2401ab990 con 0x5636e71c85d0 2024-09-12T02:14:18.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.555+0000 7fc247aac740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc2401ace00 con 0x5636e71d3370 2024-09-12T02:14:18.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.555+0000 7fc247aac740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc2401ac060 con 0x5636e71c9b40 2024-09-12T02:14:18.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.555+0000 7fc247aac740 10 client.0 _ll_register_callbacks cb 0x7fc228009610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:14:18.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.555+0000 7fc23f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636e71c85d0 0x5636e71e84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:18.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.555+0000 7fc23f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636e71c85d0 0x5636e71e84d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53348/0 (socket says 172.21.15.16:53348) 2024-09-12T02:14:18.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.555+0000 7fc23f7fe640 1 -- 172.21.15.16:0/3024870917 learned_addr learned my addr 172.21.15.16:0/3024870917 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:14:18.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.555+0000 7fc23e7fc640 1 --2- 172.21.15.16:0/3024870917 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5636e71c9b40 0x5636e71e8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:18.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.555+0000 7fc23f7fe640 1 --2- 172.21.15.16:0/3024870917 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636e71d3370 0x5636e71e3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:18.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.555+0000 7fc23e7fc640 1 -- 172.21.15.16:0/3024870917 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636e71d3370 msgr2=0x5636e71e3a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:14:18.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.555+0000 7fc23e7fc640 1 --2- 172.21.15.16:0/3024870917 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636e71d3370 0x5636e71e3a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:18.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.555+0000 7fc23e7fc640 1 -- 172.21.15.16:0/3024870917 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636e71c85d0 msgr2=0x5636e71e84d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:14:18.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.555+0000 7fc23e7fc640 1 --2- 172.21.15.16:0/3024870917 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636e71c85d0 0x5636e71e84d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:18.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.555+0000 7fc23e7fc640 1 -- 172.21.15.16:0/3024870917 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636e71e4250 con 0x5636e71c9b40 2024-09-12T02:14:18.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.556+0000 7fc23e7fc640 1 --2- 172.21.15.16:0/3024870917 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5636e71c9b40 0x5636e71e8a10 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fc22802abd0 tx=0x7fc228004490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e2af37aab0261171 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:14:18.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.556+0000 7fc2267fc640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-09-12T02:14:18.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.556+0000 7fc2267fc640 1 -- 172.21.15.16:0/3024870917 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc228035040 con 0x5636e71c9b40 2024-09-12T02:14:18.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.556+0000 7fc247aac740 1 -- 172.21.15.16:0/3024870917 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc230004490 con 0x5636e71c9b40 2024-09-12T02:14:18.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.556+0000 7fc2267fc640 1 -- 172.21.15.16:0/3024870917 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fc22803c650 con 0x5636e71c9b40 2024-09-12T02:14:18.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.556+0000 7fc2267fc640 1 -- 172.21.15.16:0/3024870917 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc22803c850 con 0x5636e71c9b40 2024-09-12T02:14:18.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.556+0000 7fc247aac740 10 client.28192 fetch_fsmap learned FSMap version 25 2024-09-12T02:14:18.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.556+0000 7fc247aac740 1 -- 172.21.15.16:0/3024870917 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5636e72516b0 con 0x5636e71c9b40 2024-09-12T02:14:18.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.556+0000 7fc2267fc640 1 -- 172.21.15.16:0/3024870917 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc230004490 con 0x5636e71c9b40 2024-09-12T02:14:18.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.556+0000 7fc2267fc640 1 -- 172.21.15.16:0/3024870917 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fc22804e850 con 0x5636e71c9b40 2024-09-12T02:14:18.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.556+0000 7fc247aac740 10 client.28192 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:14:18.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.556+0000 7fc247aac740 10 client.28192 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-12T02:14:18.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.556+0000 7fc247aac740 20 client.28192 populate_metadata read hostname 'smithi016' 2024-09-12T02:14:18.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.556+0000 7fc247aac740 1 --2- 172.21.15.16:0/3024870917 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x5636e7251b70 0x5636e7253f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:18.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.556+0000 7fc247aac740 4 client.28192 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T02:14:18.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.556+0000 7fc247aac740 1 -- 172.21.15.16:0/3024870917 --> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5636e7135d70 con 0x5636e7251b70 2024-09-12T02:14:18.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.557+0000 7fc23f7fe640 1 --2- 172.21.15.16:0/3024870917 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x5636e7251b70 0x5636e7253f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:18.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.557+0000 7fc23f7fe640 1 --2- 172.21.15.16:0/3024870917 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x5636e7251b70 0x5636e7253f90 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a20d8265c1a45541 server_cookie=d3df673b906ab384 in_seq=0 out_seq=0 2024-09-12T02:14:18.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.557+0000 7fc2267fc640 10 client.28192 ms_handle_connect on v2:172.21.15.184:6836/1793393087 2024-09-12T02:14:18.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.557+0000 7fc2267fc640 1 -- 172.21.15.16:0/3024870917 <== mds.2 v2:172.21.15.184:6836/1793393087 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5636e7135d70 con 0x5636e7251b70 2024-09-12T02:14:18.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.557+0000 7fc2267fc640 10 client.28192 handle_command_reply: tid=0 2024-09-12T02:14:18.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.557+0000 7fc247aac740 1 client.28192 shutdown 2024-09-12T02:14:18.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.557+0000 7fc247aac740 1 -- 172.21.15.16:0/3024870917 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5636e71c9b40 msgr2=0x5636e71e8a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:14:18.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.557+0000 7fc247aac740 1 --2- 172.21.15.16:0/3024870917 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5636e71c9b40 0x5636e71e8a10 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fc22802abd0 tx=0x7fc228004490 comp rx=0 tx=0).stop 2024-09-12T02:14:18.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.557+0000 7fc247aac740 1 -- 172.21.15.16:0/3024870917 shutdown_connections 2024-09-12T02:14:18.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.557+0000 7fc247aac740 1 --2- 172.21.15.16:0/3024870917 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636e71d3370 0x5636e71e3a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:18.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.557+0000 7fc247aac740 1 --2- 172.21.15.16:0/3024870917 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5636e71c9b40 0x5636e71e8a10 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:18.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.557+0000 7fc247aac740 1 --2- 172.21.15.16:0/3024870917 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x5636e7251b70 0x5636e7253f90 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:18.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.557+0000 7fc247aac740 1 --2- 172.21.15.16:0/3024870917 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636e71c85d0 0x5636e71e84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:18.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.557+0000 7fc247aac740 1 -- 172.21.15.16:0/3024870917 >> 172.21.15.16:0/3024870917 conn(0x5636e7191310 msgr2=0x5636e71cefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:14:18.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.557+0000 7fc2267fc640 0 client.28192 ms_handle_reset on v2:172.21.15.184:6836/1793393087 2024-09-12T02:14:18.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.558+0000 7fc247aac740 1 -- 172.21.15.16:0/3024870917 shutdown_connections 2024-09-12T02:14:18.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.558+0000 7fc247aac740 1 -- 172.21.15.16:0/3024870917 wait complete. 2024-09-12T02:14:18.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.558+0000 7fc247aac740 20 client.28192 trim_cache size 0 max 16384 2024-09-12T02:14:18.565 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T02:14:18.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.558+0000 7fc2456d3640 1 -- 172.21.15.16:0/3228845020 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc22c0a2b80 msgr2=0x7fc22c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:14:18.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.558+0000 7fc2456d3640 1 --2- 172.21.15.16:0/3228845020 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc22c0a2b80 0x7fc22c0a5040 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fc23800afd0 tx=0x7fc238007f30 comp rx=0 tx=0).stop 2024-09-12T02:14:18.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.558+0000 7fc2456d3640 1 -- 172.21.15.16:0/3228845020 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc240102d30 msgr2=0x7fc240198540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:14:18.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.558+0000 7fc2456d3640 1 --2- 172.21.15.16:0/3228845020 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc240102d30 0x7fc240198540 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7fc23800cce0 tx=0x7fc23800a620 comp rx=0 tx=0).stop 2024-09-12T02:14:18.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.558+0000 7fc2456d3640 1 -- 172.21.15.16:0/3228845020 shutdown_connections 2024-09-12T02:14:18.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.558+0000 7fc2456d3640 1 --2- 172.21.15.16:0/3228845020 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc22c0a2b80 0x7fc22c0a5040 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:18.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.558+0000 7fc2456d3640 1 --2- 172.21.15.16:0/3228845020 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2401082a0 0x7fc24019ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:18.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.558+0000 7fc2456d3640 1 --2- 172.21.15.16:0/3228845020 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc2401036e0 0x7fc240198a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:18.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.558+0000 7fc2456d3640 1 --2- 172.21.15.16:0/3228845020 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc240102d30 0x7fc240198540 unknown :-1 s=CLOSED pgs=1738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:18.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.558+0000 7fc2456d3640 1 -- 172.21.15.16:0/3228845020 >> 172.21.15.16:0/3228845020 conn(0x7fc2400fc7d0 msgr2=0x7fc240105d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:14:18.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.559+0000 7fc2456d3640 1 -- 172.21.15.16:0/3228845020 shutdown_connections 2024-09-12T02:14:18.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:18.559+0000 7fc2456d3640 1 -- 172.21.15.16:0/3228845020 wait complete. 2024-09-12T02:14:18.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:14:18 smithi184 ceph-mon[35593]: pgmap v706: 97 pgs: 97 active+clean; 2.4 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-09-12T02:14:18.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:14:18 smithi033 ceph-mon[35332]: pgmap v706: 97 pgs: 97 active+clean; 2.4 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-09-12T02:14:19.288 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-12T02:14:19.552 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:14:20.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.017+0000 7fa9e964f640 1 -- 172.21.15.16:0/3499331592 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9e4108300 msgr2=0x7fa9e410ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:14:20.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.017+0000 7fa9e964f640 1 --2- 172.21.15.16:0/3499331592 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9e4108300 0x7fa9e410ec70 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7fa9dc009930 tx=0x7fa9dc02a420 comp rx=0 tx=0).stop 2024-09-12T02:14:20.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.017+0000 7fa9e964f640 1 -- 172.21.15.16:0/3499331592 shutdown_connections 2024-09-12T02:14:20.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.017+0000 7fa9e964f640 1 --2- 172.21.15.16:0/3499331592 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9e4108300 0x7fa9e410ec70 unknown :-1 s=CLOSED pgs=1742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:20.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.017+0000 7fa9e964f640 1 --2- 172.21.15.16:0/3499331592 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa9e4103780 0x7fa9e4107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:20.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.017+0000 7fa9e964f640 1 --2- 172.21.15.16:0/3499331592 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa9e4102dd0 0x7fa9e41031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:20.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.017+0000 7fa9e964f640 1 -- 172.21.15.16:0/3499331592 >> 172.21.15.16:0/3499331592 conn(0x7fa9e40fc7d0 msgr2=0x7fa9e40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:14:20.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.017+0000 7fa9e964f640 1 -- 172.21.15.16:0/3499331592 shutdown_connections 2024-09-12T02:14:20.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.017+0000 7fa9e964f640 1 -- 172.21.15.16:0/3499331592 wait complete. 2024-09-12T02:14:20.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.017+0000 7fa9e964f640 1 Processor -- start 2024-09-12T02:14:20.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.018+0000 7fa9e964f640 1 -- start start 2024-09-12T02:14:20.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.018+0000 7fa9e964f640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa9e4102dd0 0x7fa9e419a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:20.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.018+0000 7fa9e964f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa9e4103780 0x7fa9e419ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:20.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.018+0000 7fa9e964f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9e4108300 0x7fa9e419ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:20.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.018+0000 7fa9e964f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa9e41120d0 con 0x7fa9e4108300 2024-09-12T02:14:20.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.018+0000 7fa9e964f640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa9e4111f50 con 0x7fa9e4103780 2024-09-12T02:14:20.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.018+0000 7fa9e964f640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa9e4112250 con 0x7fa9e4102dd0 2024-09-12T02:14:20.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.018+0000 7fa9e37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9e4108300 0x7fa9e419ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:20.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.018+0000 7fa9e37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9e4108300 0x7fa9e419ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53372/0 (socket says 172.21.15.16:53372) 2024-09-12T02:14:20.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.018+0000 7fa9e37fe640 1 -- 172.21.15.16:0/3808782575 learned_addr learned my addr 172.21.15.16:0/3808782575 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:14:20.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.018+0000 7fa9e2ffd640 1 --2- 172.21.15.16:0/3808782575 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa9e4102dd0 0x7fa9e419a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:20.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.018+0000 7fa9e27fc640 1 --2- 172.21.15.16:0/3808782575 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa9e4103780 0x7fa9e419ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:20.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.018+0000 7fa9e37fe640 1 -- 172.21.15.16:0/3808782575 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa9e4102dd0 msgr2=0x7fa9e419a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:14:20.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.018+0000 7fa9e37fe640 1 --2- 172.21.15.16:0/3808782575 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa9e4102dd0 0x7fa9e419a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:20.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.018+0000 7fa9e37fe640 1 -- 172.21.15.16:0/3808782575 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa9e4103780 msgr2=0x7fa9e419ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:14:20.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.018+0000 7fa9e37fe640 1 --2- 172.21.15.16:0/3808782575 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa9e4103780 0x7fa9e419ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:20.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.018+0000 7fa9e37fe640 1 -- 172.21.15.16:0/3808782575 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9e419f670 con 0x7fa9e4108300 2024-09-12T02:14:20.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.018+0000 7fa9e2ffd640 1 --2- 172.21.15.16:0/3808782575 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa9e4102dd0 0x7fa9e419a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:14:20.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.018+0000 7fa9e27fc640 1 --2- 172.21.15.16:0/3808782575 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa9e4103780 0x7fa9e419ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:14:20.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.019+0000 7fa9e37fe640 1 --2- 172.21.15.16:0/3808782575 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9e4108300 0x7fa9e419ef90 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7fa9dc005ab0 tx=0x7fa9dc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:14:20.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.019+0000 7fa9cbfff640 1 -- 172.21.15.16:0/3808782575 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9dc03f030 con 0x7fa9e4108300 2024-09-12T02:14:20.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.019+0000 7fa9cbfff640 1 -- 172.21.15.16:0/3808782575 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9dc003ce0 con 0x7fa9e4108300 2024-09-12T02:14:20.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.019+0000 7fa9e964f640 1 -- 172.21.15.16:0/3808782575 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9e419f900 con 0x7fa9e4108300 2024-09-12T02:14:20.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.019+0000 7fa9cbfff640 1 -- 172.21.15.16:0/3808782575 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9dc035070 con 0x7fa9e4108300 2024-09-12T02:14:20.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.019+0000 7fa9e964f640 1 -- 172.21.15.16:0/3808782575 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9e41a7240 con 0x7fa9e4108300 2024-09-12T02:14:20.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.022+0000 7fa9cbfff640 1 -- 172.21.15.16:0/3808782575 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa9dc042030 con 0x7fa9e4108300 2024-09-12T02:14:20.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.022+0000 7fa9eba28740 1 Processor -- start 2024-09-12T02:14:20.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.022+0000 7fa9eba28740 1 -- start start 2024-09-12T02:14:20.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.022+0000 7fa9eba28740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5561f7dbeb40 0x5561f7dbd090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:20.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.022+0000 7fa9eba28740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5561f7dbd5d0 0x5561f7dbda10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:20.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.022+0000 7fa9eba28740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5561f7dc8370 0x5561f7dca760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:20.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.022+0000 7fa9eba28740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5561f7d232f0 con 0x5561f7dbeb40 2024-09-12T02:14:20.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.022+0000 7fa9eba28740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5561f7ccacf0 con 0x5561f7dc8370 2024-09-12T02:14:20.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.022+0000 7fa9eba28740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5561f7d1f2b0 con 0x5561f7dbd5d0 2024-09-12T02:14:20.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.022+0000 7fa9e27fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5561f7dbeb40 0x5561f7dbd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:20.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.022+0000 7fa9e27fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5561f7dbeb40 0x5561f7dbd090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53382/0 (socket says 172.21.15.16:53382) 2024-09-12T02:14:20.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.022+0000 7fa9e27fc640 1 -- 172.21.15.16:0/2006557380 learned_addr learned my addr 172.21.15.16:0/2006557380 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:14:20.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.022+0000 7fa9e2ffd640 1 --2- 172.21.15.16:0/2006557380 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5561f7dbd5d0 0x5561f7dbda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:20.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.023+0000 7fa9e27fc640 1 -- 172.21.15.16:0/2006557380 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5561f7dbd5d0 msgr2=0x5561f7dbda10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:14:20.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.023+0000 7fa9cbfff640 1 --2- 172.21.15.16:0/3808782575 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fa9cc0a2c00 0x7fa9cc0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:20.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.023+0000 7fa9e27fc640 1 --2- 172.21.15.16:0/2006557380 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5561f7dbd5d0 0x5561f7dbda10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:20.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.023+0000 7fa9e27fc640 1 -- 172.21.15.16:0/2006557380 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5561f7dc8370 msgr2=0x5561f7dca760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:14:20.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.023+0000 7fa9e27fc640 1 --2- 172.21.15.16:0/2006557380 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5561f7dc8370 0x5561f7dca760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:20.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.023+0000 7fa9e27fc640 1 -- 172.21.15.16:0/2006557380 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561f7caf3a0 con 0x5561f7dbeb40 2024-09-12T02:14:20.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.023+0000 7fa9e37fe640 1 --2- 172.21.15.16:0/3808782575 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fa9cc0a2c00 0x7fa9cc0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:20.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.023+0000 7fa9cbfff640 1 -- 172.21.15.16:0/3808782575 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fa9dc0ef510 con 0x7fa9e4108300 2024-09-12T02:14:20.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.023+0000 7fa9e27fc640 1 --2- 172.21.15.16:0/2006557380 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5561f7dbeb40 0x5561f7dbd090 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7fa9d80058c0 tx=0x7fa9d80106e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ef0ff9da478d399 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:14:20.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.023+0000 7fa9c97fa640 1 -- 172.21.15.16:0/2006557380 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9d8016070 con 0x5561f7dbeb40 2024-09-12T02:14:20.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.023+0000 7fa9c97fa640 1 -- 172.21.15.16:0/2006557380 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9d8009e90 con 0x5561f7dbeb40 2024-09-12T02:14:20.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.023+0000 7fa9c97fa640 1 -- 172.21.15.16:0/2006557380 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9d8012420 con 0x5561f7dbeb40 2024-09-12T02:14:20.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.024+0000 7fa9eba28740 1 -- 172.21.15.16:0/2006557380 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5561f7dbeb40 msgr2=0x5561f7dbd090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:14:20.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.024+0000 7fa9e37fe640 1 --2- 172.21.15.16:0/3808782575 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fa9cc0a2c00 0x7fa9cc0a50c0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fa9dc0ed010 tx=0x7fa9dc086940 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:14:20.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.024+0000 7fa9eba28740 1 --2- 172.21.15.16:0/2006557380 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5561f7dbeb40 0x5561f7dbd090 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7fa9d80058c0 tx=0x7fa9d80106e0 comp rx=0 tx=0).stop 2024-09-12T02:14:20.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.024+0000 7fa9eba28740 1 -- 172.21.15.16:0/2006557380 shutdown_connections 2024-09-12T02:14:20.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.024+0000 7fa9eba28740 1 --2- 172.21.15.16:0/2006557380 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5561f7dc8370 0x5561f7dca760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:20.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.024+0000 7fa9eba28740 1 --2- 172.21.15.16:0/2006557380 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5561f7dbd5d0 0x5561f7dbda10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:20.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.024+0000 7fa9eba28740 1 --2- 172.21.15.16:0/2006557380 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5561f7dbeb40 0x5561f7dbd090 unknown :-1 s=CLOSED pgs=1744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:20.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.024+0000 7fa9eba28740 1 -- 172.21.15.16:0/2006557380 >> 172.21.15.16:0/2006557380 conn(0x5561f7d86310 msgr2=0x5561f7db9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:14:20.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.024+0000 7fa9eba28740 1 -- 172.21.15.16:0/2006557380 shutdown_connections 2024-09-12T02:14:20.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.024+0000 7fa9eba28740 1 -- 172.21.15.16:0/2006557380 wait complete. 2024-09-12T02:14:20.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.024+0000 7fa9eba28740 1 -- start start 2024-09-12T02:14:20.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.024+0000 7fa9eba28740 1 Processor -- start 2024-09-12T02:14:20.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.025+0000 7fa9eba28740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5561f7dbd5d0 0x5561f7dc7df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:20.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.025+0000 7fa9eba28740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5561f7dbeb40 0x5561f7dc50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:20.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.025+0000 7fa9eba28740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5561f7dc8370 0x5561f7dc57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:20.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.025+0000 7fa9eba28740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5561f7dcb070 con 0x5561f7dbeb40 2024-09-12T02:14:20.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.025+0000 7fa9eba28740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5561f7cde060 con 0x5561f7dc8370 2024-09-12T02:14:20.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.025+0000 7fa9eba28740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5561f7dcb1f0 con 0x5561f7dbd5d0 2024-09-12T02:14:20.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.025+0000 7fa9eba28740 10 client.0 _ll_register_callbacks cb 0x5561f7a318c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:14:20.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.025+0000 7fa9e2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5561f7dbeb40 0x5561f7dc50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:20.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.025+0000 7fa9e2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5561f7dbeb40 0x5561f7dc50a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53394/0 (socket says 172.21.15.16:53394) 2024-09-12T02:14:20.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.025+0000 7fa9e2ffd640 1 -- 172.21.15.16:0/2328065099 learned_addr learned my addr 172.21.15.16:0/2328065099 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:14:20.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.025+0000 7fa9e27fc640 1 --2- 172.21.15.16:0/2328065099 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5561f7dbd5d0 0x5561f7dc7df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:20.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.025+0000 7fa9e27fc640 1 --2- 172.21.15.16:0/2328065099 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5561f7dc8370 0x5561f7dc57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:20.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.025+0000 7fa9e2ffd640 1 -- 172.21.15.16:0/2328065099 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5561f7dbd5d0 msgr2=0x5561f7dc7df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:14:20.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.025+0000 7fa9e2ffd640 1 --2- 172.21.15.16:0/2328065099 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5561f7dbd5d0 0x5561f7dc7df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:20.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.025+0000 7fa9e2ffd640 1 -- 172.21.15.16:0/2328065099 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5561f7dc8370 msgr2=0x5561f7dc57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:14:20.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.025+0000 7fa9e2ffd640 1 --2- 172.21.15.16:0/2328065099 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5561f7dc8370 0x5561f7dc57e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:20.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.025+0000 7fa9e2ffd640 1 -- 172.21.15.16:0/2328065099 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561f7a943c0 con 0x5561f7dbeb40 2024-09-12T02:14:20.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.025+0000 7fa9e27fc640 1 --2- 172.21.15.16:0/2328065099 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5561f7dc8370 0x5561f7dc57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:14:20.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.025+0000 7fa9e2ffd640 1 --2- 172.21.15.16:0/2328065099 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5561f7dbeb40 0x5561f7dc50a0 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7fa9d4011bc0 tx=0x7fa9d400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5347dc555886cd3d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:14:20.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.025+0000 7fa9c6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:14:20.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.025+0000 7fa9c6ff5640 1 -- 172.21.15.16:0/2328065099 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9d4004d20 con 0x5561f7dbeb40 2024-09-12T02:14:20.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.025+0000 7fa9c6ff5640 1 -- 172.21.15.16:0/2328065099 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fa9d400e5c0 con 0x5561f7dbeb40 2024-09-12T02:14:20.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.025+0000 7fa9eba28740 1 -- 172.21.15.16:0/2328065099 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9d8009e90 con 0x5561f7dbeb40 2024-09-12T02:14:20.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.026+0000 7fa9c6ff5640 1 -- 172.21.15.16:0/2328065099 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa9d400e7c0 con 0x5561f7dbeb40 2024-09-12T02:14:20.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.026+0000 7fa9eba28740 10 client.19620 fetch_fsmap learned FSMap version 25 2024-09-12T02:14:20.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.026+0000 7fa9eba28740 1 -- 172.21.15.16:0/2328065099 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5561f7dd6c00 con 0x5561f7dbeb40 2024-09-12T02:14:20.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.026+0000 7fa9c6ff5640 1 -- 172.21.15.16:0/2328065099 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9d8009e90 con 0x5561f7dbeb40 2024-09-12T02:14:20.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.026+0000 7fa9c6ff5640 1 -- 172.21.15.16:0/2328065099 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fa9d4018070 con 0x5561f7dbeb40 2024-09-12T02:14:20.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.026+0000 7fa9eba28740 10 client.19620 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:14:20.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.026+0000 7fa9eba28740 10 client.19620 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T02:14:20.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.026+0000 7fa9eba28740 20 client.19620 populate_metadata read hostname 'smithi016' 2024-09-12T02:14:20.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.026+0000 7fa9eba28740 1 --2- 172.21.15.16:0/2328065099 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x5561f7dd80b0 0x5561f7dda4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:20.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.026+0000 7fa9eba28740 4 client.19620 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:14:20.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.026+0000 7fa9eba28740 1 -- 172.21.15.16:0/2328065099 --> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5561f7d30860 con 0x5561f7dd80b0 2024-09-12T02:14:20.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.026+0000 7fa9e27fc640 1 --2- 172.21.15.16:0/2328065099 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x5561f7dd80b0 0x5561f7dda4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:20.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.027+0000 7fa9e27fc640 1 --2- 172.21.15.16:0/2328065099 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x5561f7dd80b0 0x5561f7dda4d0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9cb25f0d92361053 server_cookie=6de3a40ac0c0837f in_seq=0 out_seq=0 2024-09-12T02:14:20.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.027+0000 7fa9c6ff5640 10 client.19620 ms_handle_connect on v2:172.21.15.184:6838/4203347941 2024-09-12T02:14:20.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.027+0000 7fa9c6ff5640 1 -- 172.21.15.16:0/2328065099 <== mds.1 v2:172.21.15.184:6838/4203347941 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5561f7d30860 con 0x5561f7dd80b0 2024-09-12T02:14:20.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.027+0000 7fa9c6ff5640 10 client.19620 handle_command_reply: tid=0 2024-09-12T02:14:20.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.027+0000 7fa9eba28740 1 client.19620 shutdown 2024-09-12T02:14:20.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.028+0000 7fa9eba28740 1 -- 172.21.15.16:0/2328065099 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5561f7dbeb40 msgr2=0x5561f7dc50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:14:20.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.028+0000 7fa9eba28740 1 --2- 172.21.15.16:0/2328065099 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5561f7dbeb40 0x5561f7dc50a0 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7fa9d4011bc0 tx=0x7fa9d400f790 comp rx=0 tx=0).stop 2024-09-12T02:14:20.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.028+0000 7fa9eba28740 1 -- 172.21.15.16:0/2328065099 shutdown_connections 2024-09-12T02:14:20.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.028+0000 7fa9eba28740 1 --2- 172.21.15.16:0/2328065099 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x5561f7dd80b0 0x5561f7dda4d0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:20.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.028+0000 7fa9eba28740 1 --2- 172.21.15.16:0/2328065099 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5561f7dc8370 0x5561f7dc57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:20.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.028+0000 7fa9eba28740 1 --2- 172.21.15.16:0/2328065099 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5561f7dbeb40 0x5561f7dc50a0 unknown :-1 s=CLOSED pgs=1745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:20.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.028+0000 7fa9eba28740 1 --2- 172.21.15.16:0/2328065099 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5561f7dbd5d0 0x5561f7dc7df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:20.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.028+0000 7fa9eba28740 1 -- 172.21.15.16:0/2328065099 >> 172.21.15.16:0/2328065099 conn(0x5561f7d86310 msgr2=0x5561f7dbf960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:14:20.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.028+0000 7fa9c6ff5640 0 client.19620 ms_handle_reset on v2:172.21.15.184:6838/4203347941 2024-09-12T02:14:20.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.028+0000 7fa9eba28740 1 -- 172.21.15.16:0/2328065099 shutdown_connections 2024-09-12T02:14:20.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.028+0000 7fa9eba28740 1 -- 172.21.15.16:0/2328065099 wait complete. 2024-09-12T02:14:20.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.028+0000 7fa9eba28740 20 client.19620 trim_cache size 0 max 16384 2024-09-12T02:14:20.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.041+0000 7fa9eba28740 1 Processor -- start 2024-09-12T02:14:20.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.041+0000 7fa9eba28740 1 -- start start 2024-09-12T02:14:20.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.041+0000 7fa9eba28740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5561f7dbd5d0 0x5561f7dc5070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:20.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.041+0000 7fa9eba28740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5561f7dbeb40 0x5561f7dd4310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:20.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.041+0000 7fa9eba28740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5561f7dc8370 0x5561f7ddfd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:20.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.041+0000 7fa9eba28740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa9e41b1dc0 con 0x5561f7dc8370 2024-09-12T02:14:20.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.041+0000 7fa9eba28740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa9e41af690 con 0x5561f7dbd5d0 2024-09-12T02:14:20.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.041+0000 7fa9eba28740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa9e41ad520 con 0x5561f7dbeb40 2024-09-12T02:14:20.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.041+0000 7fa9e27fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5561f7dc8370 0x5561f7ddfd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:20.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.041+0000 7fa9e27fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5561f7dc8370 0x5561f7ddfd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53402/0 (socket says 172.21.15.16:53402) 2024-09-12T02:14:20.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.041+0000 7fa9e27fc640 1 -- 172.21.15.16:0/2000745396 learned_addr learned my addr 172.21.15.16:0/2000745396 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:14:20.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.041+0000 7fa9e2ffd640 1 --2- 172.21.15.16:0/2000745396 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5561f7dbeb40 0x5561f7dd4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:20.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.041+0000 7fa9e27fc640 1 -- 172.21.15.16:0/2000745396 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5561f7dbeb40 msgr2=0x5561f7dd4310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:14:20.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.041+0000 7fa9e27fc640 1 --2- 172.21.15.16:0/2000745396 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5561f7dbeb40 0x5561f7dd4310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:20.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.041+0000 7fa9e27fc640 1 -- 172.21.15.16:0/2000745396 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5561f7dbd5d0 msgr2=0x5561f7dc5070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:14:20.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.041+0000 7fa9e27fc640 1 --2- 172.21.15.16:0/2000745396 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5561f7dbd5d0 0x5561f7dc5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:20.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.042+0000 7fa9e27fc640 1 -- 172.21.15.16:0/2000745396 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561f7db9ed0 con 0x5561f7dc8370 2024-09-12T02:14:20.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.042+0000 7fa9e27fc640 1 --2- 172.21.15.16:0/2000745396 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5561f7dc8370 0x5561f7ddfd80 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7fa9d8009ba0 tx=0x7fa9d80106e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=170795a889c6d57b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:14:20.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.042+0000 7fa9c8ff9640 1 -- 172.21.15.16:0/2000745396 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9d8016040 con 0x5561f7dc8370 2024-09-12T02:14:20.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.042+0000 7fa9c8ff9640 1 -- 172.21.15.16:0/2000745396 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9d8004b60 con 0x5561f7dc8370 2024-09-12T02:14:20.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.042+0000 7fa9c8ff9640 1 -- 172.21.15.16:0/2000745396 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9d8024040 con 0x5561f7dc8370 2024-09-12T02:14:20.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.042+0000 7fa9eba28740 1 -- 172.21.15.16:0/2000745396 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5561f7dc8370 msgr2=0x5561f7ddfd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:14:20.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.042+0000 7fa9eba28740 1 --2- 172.21.15.16:0/2000745396 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5561f7dc8370 0x5561f7ddfd80 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7fa9d8009ba0 tx=0x7fa9d80106e0 comp rx=0 tx=0).stop 2024-09-12T02:14:20.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.042+0000 7fa9eba28740 1 -- 172.21.15.16:0/2000745396 shutdown_connections 2024-09-12T02:14:20.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.043+0000 7fa9eba28740 1 --2- 172.21.15.16:0/2000745396 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5561f7dc8370 0x5561f7ddfd80 unknown :-1 s=CLOSED pgs=1746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:20.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.043+0000 7fa9eba28740 1 --2- 172.21.15.16:0/2000745396 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5561f7dbeb40 0x5561f7dd4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:20.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.043+0000 7fa9eba28740 1 --2- 172.21.15.16:0/2000745396 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5561f7dbd5d0 0x5561f7dc5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:20.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.043+0000 7fa9eba28740 1 -- 172.21.15.16:0/2000745396 >> 172.21.15.16:0/2000745396 conn(0x5561f7d86310 msgr2=0x5561f7dc4960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:14:20.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.043+0000 7fa9eba28740 1 -- 172.21.15.16:0/2000745396 shutdown_connections 2024-09-12T02:14:20.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.043+0000 7fa9eba28740 1 -- 172.21.15.16:0/2000745396 wait complete. 2024-09-12T02:14:20.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.043+0000 7fa9eba28740 1 -- start start 2024-09-12T02:14:20.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.043+0000 7fa9eba28740 1 Processor -- start 2024-09-12T02:14:20.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.043+0000 7fa9eba28740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5561f7dbd5d0 0x5561f7dd0480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:20.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.043+0000 7fa9eba28740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5561f7dbeb40 0x5561f7dd09c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:20.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.043+0000 7fa9eba28740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5561f7dc8370 0x5561f7dc1430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:20.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.043+0000 7fa9eba28740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa9e41adb70 con 0x5561f7dbd5d0 2024-09-12T02:14:20.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.043+0000 7fa9eba28740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa9e41aefc0 con 0x5561f7dbeb40 2024-09-12T02:14:20.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.043+0000 7fa9eba28740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa9e41ae220 con 0x5561f7dc8370 2024-09-12T02:14:20.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.043+0000 7fa9eba28740 10 client.0 _ll_register_callbacks cb 0x7fa9d4007ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:14:20.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.043+0000 7fa9e27fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5561f7dbd5d0 0x5561f7dd0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:20.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.043+0000 7fa9e27fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5561f7dbd5d0 0x5561f7dd0480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53416/0 (socket says 172.21.15.16:53416) 2024-09-12T02:14:20.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.043+0000 7fa9e27fc640 1 -- 172.21.15.16:0/565112048 learned_addr learned my addr 172.21.15.16:0/565112048 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:14:20.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.044+0000 7fa9e27fc640 1 --2- 172.21.15.16:0/565112048 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5561f7dc8370 0x5561f7dc1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:20.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.044+0000 7fa9e2ffd640 1 --2- 172.21.15.16:0/565112048 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5561f7dbeb40 0x5561f7dd09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:20.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.044+0000 7fa9e27fc640 1 -- 172.21.15.16:0/565112048 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5561f7dc8370 msgr2=0x5561f7dc1430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:14:20.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.044+0000 7fa9e27fc640 1 --2- 172.21.15.16:0/565112048 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5561f7dc8370 0x5561f7dc1430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:20.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.044+0000 7fa9e27fc640 1 -- 172.21.15.16:0/565112048 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5561f7dbeb40 msgr2=0x5561f7dd09c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:14:20.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.044+0000 7fa9e27fc640 1 --2- 172.21.15.16:0/565112048 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5561f7dbeb40 0x5561f7dd09c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:20.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.044+0000 7fa9e27fc640 1 -- 172.21.15.16:0/565112048 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561f7dc1970 con 0x5561f7dbd5d0 2024-09-12T02:14:20.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.044+0000 7fa9e27fc640 1 --2- 172.21.15.16:0/565112048 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5561f7dbd5d0 0x5561f7dd0480 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7fa9d8009e30 tx=0x7fa9d8005900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1398bc98d2a924fb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:14:20.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.044+0000 7fa9c6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:14:20.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.044+0000 7fa9c6ff5640 1 -- 172.21.15.16:0/565112048 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9d8016040 con 0x5561f7dbd5d0 2024-09-12T02:14:20.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.044+0000 7fa9c6ff5640 1 -- 172.21.15.16:0/565112048 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fa9d8005200 con 0x5561f7dbd5d0 2024-09-12T02:14:20.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.044+0000 7fa9eba28740 1 -- 172.21.15.16:0/565112048 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9d8004b60 con 0x5561f7dbd5d0 2024-09-12T02:14:20.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.044+0000 7fa9c6ff5640 1 -- 172.21.15.16:0/565112048 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa9d8005600 con 0x5561f7dbd5d0 2024-09-12T02:14:20.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.044+0000 7fa9eba28740 10 client.19626 fetch_fsmap learned FSMap version 25 2024-09-12T02:14:20.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.044+0000 7fa9eba28740 1 -- 172.21.15.16:0/565112048 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5561f7e42410 con 0x5561f7dbd5d0 2024-09-12T02:14:20.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.045+0000 7fa9c6ff5640 1 -- 172.21.15.16:0/565112048 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9d8004b60 con 0x5561f7dbd5d0 2024-09-12T02:14:20.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.045+0000 7fa9c6ff5640 1 -- 172.21.15.16:0/565112048 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fa9d8003850 con 0x5561f7dbd5d0 2024-09-12T02:14:20.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.045+0000 7fa9eba28740 10 client.19626 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:14:20.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.045+0000 7fa9eba28740 10 client.19626 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T02:14:20.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.045+0000 7fa9eba28740 20 client.19626 populate_metadata read hostname 'smithi016' 2024-09-12T02:14:20.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.045+0000 7fa9eba28740 1 --2- 172.21.15.16:0/565112048 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x5561f7e42880 0x5561f7e44c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:20.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.045+0000 7fa9eba28740 4 client.19626 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T02:14:20.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.045+0000 7fa9eba28740 1 -- 172.21.15.16:0/565112048 --> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5561f7d2ad70 con 0x5561f7e42880 2024-09-12T02:14:20.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.045+0000 7fa9e2ffd640 1 --2- 172.21.15.16:0/565112048 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x5561f7e42880 0x5561f7e44c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:20.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.046+0000 7fa9e2ffd640 1 --2- 172.21.15.16:0/565112048 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x5561f7e42880 0x5561f7e44c70 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b16e769ad0a25cff server_cookie=108235e767395e01 in_seq=0 out_seq=0 2024-09-12T02:14:20.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.046+0000 7fa9c6ff5640 10 client.19626 ms_handle_connect on v2:172.21.15.184:6838/4203347941 2024-09-12T02:14:20.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.046+0000 7fa9c6ff5640 1 -- 172.21.15.16:0/565112048 <== mds.1 v2:172.21.15.184:6838/4203347941 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5561f7d2ad70 con 0x5561f7e42880 2024-09-12T02:14:20.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.046+0000 7fa9c6ff5640 10 client.19626 handle_command_reply: tid=0 2024-09-12T02:14:20.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.046+0000 7fa9eba28740 1 client.19626 shutdown 2024-09-12T02:14:20.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.046+0000 7fa9eba28740 1 -- 172.21.15.16:0/565112048 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5561f7dbd5d0 msgr2=0x5561f7dd0480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:14:20.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.046+0000 7fa9eba28740 1 --2- 172.21.15.16:0/565112048 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5561f7dbd5d0 0x5561f7dd0480 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7fa9d8009e30 tx=0x7fa9d8005900 comp rx=0 tx=0).stop 2024-09-12T02:14:20.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.046+0000 7fa9eba28740 1 -- 172.21.15.16:0/565112048 shutdown_connections 2024-09-12T02:14:20.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.046+0000 7fa9eba28740 1 --2- 172.21.15.16:0/565112048 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x5561f7e42880 0x5561f7e44c70 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:20.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.046+0000 7fa9eba28740 1 --2- 172.21.15.16:0/565112048 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5561f7dc8370 0x5561f7dc1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:20.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.046+0000 7fa9c6ff5640 0 client.19626 ms_handle_reset on v2:172.21.15.184:6838/4203347941 2024-09-12T02:14:20.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.046+0000 7fa9eba28740 1 --2- 172.21.15.16:0/565112048 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5561f7dbeb40 0x5561f7dd09c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:20.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.046+0000 7fa9eba28740 1 --2- 172.21.15.16:0/565112048 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5561f7dbd5d0 0x5561f7dd0480 unknown :-1 s=CLOSED pgs=1747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:20.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.046+0000 7fa9eba28740 1 -- 172.21.15.16:0/565112048 >> 172.21.15.16:0/565112048 conn(0x5561f7d86310 msgr2=0x5561f7db8d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:14:20.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.046+0000 7fa9eba28740 1 -- 172.21.15.16:0/565112048 shutdown_connections 2024-09-12T02:14:20.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.046+0000 7fa9eba28740 1 -- 172.21.15.16:0/565112048 wait complete. 2024-09-12T02:14:20.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.046+0000 7fa9eba28740 20 client.19626 trim_cache size 0 max 16384 2024-09-12T02:14:20.058 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T02:14:20.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.047+0000 7fa9e964f640 1 -- 172.21.15.16:0/3808782575 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fa9cc0a2c00 msgr2=0x7fa9cc0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:14:20.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.047+0000 7fa9e964f640 1 --2- 172.21.15.16:0/3808782575 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fa9cc0a2c00 0x7fa9cc0a50c0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fa9dc0ed010 tx=0x7fa9dc086940 comp rx=0 tx=0).stop 2024-09-12T02:14:20.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.047+0000 7fa9e964f640 1 -- 172.21.15.16:0/3808782575 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9e4108300 msgr2=0x7fa9e419ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:14:20.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.047+0000 7fa9e964f640 1 --2- 172.21.15.16:0/3808782575 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9e4108300 0x7fa9e419ef90 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7fa9dc005ab0 tx=0x7fa9dc0042f0 comp rx=0 tx=0).stop 2024-09-12T02:14:20.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.047+0000 7fa9e964f640 1 -- 172.21.15.16:0/3808782575 shutdown_connections 2024-09-12T02:14:20.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.047+0000 7fa9e964f640 1 --2- 172.21.15.16:0/3808782575 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fa9cc0a2c00 0x7fa9cc0a50c0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:20.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.047+0000 7fa9e964f640 1 --2- 172.21.15.16:0/3808782575 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9e4108300 0x7fa9e419ef90 unknown :-1 s=CLOSED pgs=1743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:20.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.047+0000 7fa9e964f640 1 --2- 172.21.15.16:0/3808782575 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa9e4103780 0x7fa9e419ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:20.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.047+0000 7fa9e964f640 1 --2- 172.21.15.16:0/3808782575 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa9e4102dd0 0x7fa9e419a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:20.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.047+0000 7fa9e964f640 1 -- 172.21.15.16:0/3808782575 >> 172.21.15.16:0/3808782575 conn(0x7fa9e40fc7d0 msgr2=0x7fa9e41012b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:14:20.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.047+0000 7fa9e964f640 1 -- 172.21.15.16:0/3808782575 shutdown_connections 2024-09-12T02:14:20.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:20.047+0000 7fa9e964f640 1 -- 172.21.15.16:0/3808782575 wait complete. 2024-09-12T02:14:20.183 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:14:20 smithi016 ceph-mon[30947]: pgmap v707: 97 pgs: 97 active+clean; 2.4 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T02:14:20.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:14:20 smithi184 ceph-mon[35593]: pgmap v707: 97 pgs: 97 active+clean; 2.4 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T02:14:20.649 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-12T02:14:20.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:14:20 smithi033 ceph-mon[35332]: pgmap v707: 97 pgs: 97 active+clean; 2.4 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T02:14:20.905 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:14:21.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.398+0000 7f4dc67be640 1 -- 172.21.15.16:0/3098002060 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4dc0100a80 msgr2=0x7f4dc0100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:14:21.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.398+0000 7f4dc67be640 1 --2- 172.21.15.16:0/3098002060 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4dc0100a80 0x7f4dc0100ee0 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7f4da8009930 tx=0x7f4da802a420 comp rx=0 tx=0).stop 2024-09-12T02:14:21.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.399+0000 7f4dc67be640 1 -- 172.21.15.16:0/3098002060 shutdown_connections 2024-09-12T02:14:21.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.399+0000 7f4dc67be640 1 --2- 172.21.15.16:0/3098002060 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4dc0101420 0x7f4dc010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:21.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.399+0000 7f4dc67be640 1 --2- 172.21.15.16:0/3098002060 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4dc0100a80 0x7f4dc0100ee0 unknown :-1 s=CLOSED pgs=1748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:21.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.399+0000 7f4dc67be640 1 --2- 172.21.15.16:0/3098002060 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4dc0106a80 0x7f4dc0106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:21.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.399+0000 7f4dc67be640 1 -- 172.21.15.16:0/3098002060 >> 172.21.15.16:0/3098002060 conn(0x7f4dc00fc7a0 msgr2=0x7f4dc00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:14:21.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.399+0000 7f4dc67be640 1 -- 172.21.15.16:0/3098002060 shutdown_connections 2024-09-12T02:14:21.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.399+0000 7f4dc67be640 1 -- 172.21.15.16:0/3098002060 wait complete. 2024-09-12T02:14:21.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.399+0000 7f4dc67be640 1 Processor -- start 2024-09-12T02:14:21.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.399+0000 7f4dc67be640 1 -- start start 2024-09-12T02:14:21.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.399+0000 7f4dc67be640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4dc0100a80 0x7f4dc0198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:21.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.399+0000 7f4dc67be640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4dc0101420 0x7f4dc01989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:21.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.399+0000 7f4dc67be640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4dc0106a80 0x7f4dc019cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:21.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.399+0000 7f4dc67be640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4dc0110ea0 con 0x7f4dc0106a80 2024-09-12T02:14:21.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.399+0000 7f4dc67be640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4dc0110d20 con 0x7f4dc0101420 2024-09-12T02:14:21.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.399+0000 7f4dc67be640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f4dc0111020 con 0x7f4dc0100a80 2024-09-12T02:14:21.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.400+0000 7f4dc4d5b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4dc0106a80 0x7f4dc019cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:21.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.400+0000 7f4dc4d5b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4dc0106a80 0x7f4dc019cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55338/0 (socket says 172.21.15.16:55338) 2024-09-12T02:14:21.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.400+0000 7f4dc4d5b640 1 -- 172.21.15.16:0/307025104 learned_addr learned my addr 172.21.15.16:0/307025104 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:14:21.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.400+0000 7f4db7fff640 1 --2- 172.21.15.16:0/307025104 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4dc0101420 0x7f4dc01989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:21.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.400+0000 7f4dbffff640 1 --2- 172.21.15.16:0/307025104 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4dc0100a80 0x7f4dc0198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:21.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.400+0000 7f4dc4d5b640 1 -- 172.21.15.16:0/307025104 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4dc0100a80 msgr2=0x7f4dc0198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:14:21.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.400+0000 7f4dc4d5b640 1 --2- 172.21.15.16:0/307025104 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4dc0100a80 0x7f4dc0198490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:21.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.400+0000 7f4dc4d5b640 1 -- 172.21.15.16:0/307025104 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4dc0101420 msgr2=0x7f4dc01989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:14:21.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.400+0000 7f4dc4d5b640 1 --2- 172.21.15.16:0/307025104 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4dc0101420 0x7f4dc01989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:21.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.400+0000 7f4dc4d5b640 1 -- 172.21.15.16:0/307025104 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4dc019d440 con 0x7f4dc0106a80 2024-09-12T02:14:21.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.400+0000 7f4dc4d5b640 1 --2- 172.21.15.16:0/307025104 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4dc0106a80 0x7f4dc019cd60 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7f4db800b970 tx=0x7f4db800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:14:21.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.400+0000 7f4dbdffb640 1 -- 172.21.15.16:0/307025104 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4db8019070 con 0x7f4dc0106a80 2024-09-12T02:14:21.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.400+0000 7f4dbdffb640 1 -- 172.21.15.16:0/307025104 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4db800cb40 con 0x7f4dc0106a80 2024-09-12T02:14:21.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.400+0000 7f4dc67be640 1 -- 172.21.15.16:0/307025104 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4dc019d730 con 0x7f4dc0106a80 2024-09-12T02:14:21.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.400+0000 7f4dbdffb640 1 -- 172.21.15.16:0/307025104 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4db8007650 con 0x7f4dc0106a80 2024-09-12T02:14:21.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.400+0000 7f4dc67be640 1 -- 172.21.15.16:0/307025104 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4dc01a5010 con 0x7f4dc0106a80 2024-09-12T02:14:21.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.402+0000 7f4dbdffb640 1 -- 172.21.15.16:0/307025104 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f4db801c070 con 0x7f4dc0106a80 2024-09-12T02:14:21.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.403+0000 7f4dc8b97740 1 Processor -- start 2024-09-12T02:14:21.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.403+0000 7f4dc8b97740 1 -- start start 2024-09-12T02:14:21.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.403+0000 7f4dc8b97740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560a378acb40 0x560a378ab090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:21.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.403+0000 7f4dc8b97740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560a378ab5d0 0x560a378aba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:21.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.403+0000 7f4dc8b97740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a378b6370 0x560a378b8760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:21.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.403+0000 7f4dc8b97740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x560a378112f0 con 0x560a378b6370 2024-09-12T02:14:21.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.403+0000 7f4dc8b97740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x560a377b8cf0 con 0x560a378acb40 2024-09-12T02:14:21.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.403+0000 7f4dc8b97740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x560a3780d2b0 con 0x560a378ab5d0 2024-09-12T02:14:21.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.403+0000 7f4dbdffb640 1 --2- 172.21.15.16:0/307025104 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f4dac0a2c00 0x7f4dac0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:21.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.403+0000 7f4db7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a378b6370 0x560a378b8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:21.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.403+0000 7f4dc4d5b640 1 --2- 172.21.15.16:0/307025104 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f4dac0a2c00 0x7f4dac0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:21.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.403+0000 7f4db7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a378b6370 0x560a378b8760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55342/0 (socket says 172.21.15.16:55342) 2024-09-12T02:14:21.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.403+0000 7f4db7fff640 1 -- 172.21.15.16:0/3743184504 learned_addr learned my addr 172.21.15.16:0/3743184504 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:14:21.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.404+0000 7f4dbffff640 1 --2- 172.21.15.16:0/3743184504 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560a378ab5d0 0x560a378aba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:21.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.404+0000 7f4db7fff640 1 --2- 172.21.15.16:0/3743184504 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560a378acb40 0x560a378ab090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:21.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.404+0000 7f4dbdffb640 1 -- 172.21.15.16:0/307025104 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f4db80ce520 con 0x7f4dc0106a80 2024-09-12T02:14:21.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.404+0000 7f4dbffff640 1 -- 172.21.15.16:0/3743184504 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560a378acb40 msgr2=0x560a378ab090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:14:21.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.404+0000 7f4dbffff640 1 --2- 172.21.15.16:0/3743184504 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560a378acb40 0x560a378ab090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:21.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.404+0000 7f4dbffff640 1 -- 172.21.15.16:0/3743184504 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a378b6370 msgr2=0x560a378b8760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:14:21.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.404+0000 7f4dbffff640 1 --2- 172.21.15.16:0/3743184504 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a378b6370 0x560a378b8760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:21.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.404+0000 7f4dbffff640 1 -- 172.21.15.16:0/3743184504 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a3779d3a0 con 0x560a378ab5d0 2024-09-12T02:14:21.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.404+0000 7f4dc4d5b640 1 --2- 172.21.15.16:0/307025104 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f4dac0a2c00 0x7f4dac0a50c0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f4db8097330 tx=0x7f4db80c59b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:14:21.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.404+0000 7f4dbffff640 1 --2- 172.21.15.16:0/3743184504 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560a378ab5d0 0x560a378aba10 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f4db000bbf0 tx=0x7f4db00107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=69f16444a617be37 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:14:21.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.404+0000 7f4db67fc640 1 -- 172.21.15.16:0/3743184504 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4db0004030 con 0x560a378ab5d0 2024-09-12T02:14:21.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.404+0000 7f4db67fc640 1 -- 172.21.15.16:0/3743184504 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4db00041d0 con 0x560a378ab5d0 2024-09-12T02:14:21.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.404+0000 7f4db67fc640 1 -- 172.21.15.16:0/3743184504 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4db00116a0 con 0x560a378ab5d0 2024-09-12T02:14:21.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.405+0000 7f4dc8b97740 1 -- 172.21.15.16:0/3743184504 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560a378ab5d0 msgr2=0x560a378aba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:14:21.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.405+0000 7f4dc8b97740 1 --2- 172.21.15.16:0/3743184504 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560a378ab5d0 0x560a378aba10 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f4db000bbf0 tx=0x7f4db00107a0 comp rx=0 tx=0).stop 2024-09-12T02:14:21.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.405+0000 7f4dc8b97740 1 -- 172.21.15.16:0/3743184504 shutdown_connections 2024-09-12T02:14:21.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.405+0000 7f4dc8b97740 1 --2- 172.21.15.16:0/3743184504 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a378b6370 0x560a378b8760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:21.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.405+0000 7f4dc8b97740 1 --2- 172.21.15.16:0/3743184504 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560a378ab5d0 0x560a378aba10 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:21.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.405+0000 7f4dc8b97740 1 --2- 172.21.15.16:0/3743184504 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560a378acb40 0x560a378ab090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:21.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.405+0000 7f4dc8b97740 1 -- 172.21.15.16:0/3743184504 >> 172.21.15.16:0/3743184504 conn(0x560a37874310 msgr2=0x560a378a7c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:14:21.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.405+0000 7f4dc8b97740 1 -- 172.21.15.16:0/3743184504 shutdown_connections 2024-09-12T02:14:21.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.405+0000 7f4dc8b97740 1 -- 172.21.15.16:0/3743184504 wait complete. 2024-09-12T02:14:21.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.406+0000 7f4dc8b97740 1 -- start start 2024-09-12T02:14:21.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.406+0000 7f4dc8b97740 1 Processor -- start 2024-09-12T02:14:21.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.406+0000 7f4dc8b97740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560a378ab5d0 0x560a378b5d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:21.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.406+0000 7f4dc8b97740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a378acb40 0x560a378b3010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:21.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.406+0000 7f4dc8b97740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560a378b6370 0x560a378b3600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:21.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.406+0000 7f4dc8b97740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x560a378b9070 con 0x560a378acb40 2024-09-12T02:14:21.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.406+0000 7f4dc8b97740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x560a377cc060 con 0x560a378ab5d0 2024-09-12T02:14:21.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.406+0000 7f4dc8b97740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x560a378b91f0 con 0x560a378b6370 2024-09-12T02:14:21.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.406+0000 7f4dc8b97740 10 client.0 _ll_register_callbacks cb 0x560a3751f8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:14:21.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.406+0000 7f4dbffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a378acb40 0x560a378b3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:21.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.406+0000 7f4dbffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a378acb40 0x560a378b3010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55344/0 (socket says 172.21.15.16:55344) 2024-09-12T02:14:21.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.406+0000 7f4dbffff640 1 -- 172.21.15.16:0/3134641676 learned_addr learned my addr 172.21.15.16:0/3134641676 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:14:21.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.406+0000 7f4db7fff640 1 --2- 172.21.15.16:0/3134641676 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560a378ab5d0 0x560a378b5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:21.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.406+0000 7f4db7fff640 1 --2- 172.21.15.16:0/3134641676 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560a378b6370 0x560a378b3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:21.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.407+0000 7f4dbffff640 1 -- 172.21.15.16:0/3134641676 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560a378b6370 msgr2=0x560a378b3600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:14:21.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.407+0000 7f4dbffff640 1 --2- 172.21.15.16:0/3134641676 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560a378b6370 0x560a378b3600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:21.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.407+0000 7f4dbffff640 1 -- 172.21.15.16:0/3134641676 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560a378ab5d0 msgr2=0x560a378b5d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:14:21.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.407+0000 7f4dbffff640 1 --2- 172.21.15.16:0/3134641676 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560a378ab5d0 0x560a378b5d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:21.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.407+0000 7f4dbffff640 1 -- 172.21.15.16:0/3134641676 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a375823c0 con 0x560a378acb40 2024-09-12T02:14:21.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.407+0000 7f4dbffff640 1 --2- 172.21.15.16:0/3134641676 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a378acb40 0x560a378b3010 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7f4db0004180 tx=0x7f4db0003b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef76beed7c0a80eb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:14:21.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.407+0000 7f4d9f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:14:21.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.407+0000 7f4d9f7fe640 1 -- 172.21.15.16:0/3134641676 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4db002a060 con 0x560a378acb40 2024-09-12T02:14:21.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.407+0000 7f4d9f7fe640 1 -- 172.21.15.16:0/3134641676 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f4db000bce0 con 0x560a378acb40 2024-09-12T02:14:21.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.407+0000 7f4dc8b97740 1 -- 172.21.15.16:0/3134641676 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4db00041d0 con 0x560a378acb40 2024-09-12T02:14:21.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.407+0000 7f4d9f7fe640 1 -- 172.21.15.16:0/3134641676 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4db00035f0 con 0x560a378acb40 2024-09-12T02:14:21.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.407+0000 7f4dc8b97740 10 client.19638 fetch_fsmap learned FSMap version 25 2024-09-12T02:14:21.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.407+0000 7f4dc8b97740 1 -- 172.21.15.16:0/3134641676 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x560a378c4b10 con 0x560a378acb40 2024-09-12T02:14:21.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.407+0000 7f4d9f7fe640 1 -- 172.21.15.16:0/3134641676 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4db00041d0 con 0x560a378acb40 2024-09-12T02:14:21.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.408+0000 7f4d9f7fe640 1 -- 172.21.15.16:0/3134641676 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f4db002f070 con 0x560a378acb40 2024-09-12T02:14:21.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.408+0000 7f4dc8b97740 10 client.19638 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:14:21.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.408+0000 7f4dc8b97740 10 client.19638 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:14:21.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.408+0000 7f4dc8b97740 20 client.19638 populate_metadata read hostname 'smithi016' 2024-09-12T02:14:21.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.408+0000 7f4dc8b97740 1 --2- 172.21.15.16:0/3134641676 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x560a378c5f10 0x560a378c8300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:21.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.408+0000 7f4dc8b97740 4 client.19638 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:14:21.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.408+0000 7f4dc8b97740 1 -- 172.21.15.16:0/3134641676 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560a3781e860 con 0x560a378c5f10 2024-09-12T02:14:21.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.408+0000 7f4db7fff640 1 --2- 172.21.15.16:0/3134641676 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x560a378c5f10 0x560a378c8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:21.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.408+0000 7f4db7fff640 1 --2- 172.21.15.16:0/3134641676 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x560a378c5f10 0x560a378c8300 crc :-1 s=READY pgs=376 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf655d25b1fa3080 server_cookie=fbe6b64edffef45c in_seq=0 out_seq=0 2024-09-12T02:14:21.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.408+0000 7f4d9f7fe640 10 client.19638 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:14:21.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.409+0000 7f4d9f7fe640 1 -- 172.21.15.16:0/3134641676 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560a3781e860 con 0x560a378c5f10 2024-09-12T02:14:21.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.409+0000 7f4d9f7fe640 10 client.19638 handle_command_reply: tid=0 2024-09-12T02:14:21.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.409+0000 7f4dc8b97740 1 client.19638 shutdown 2024-09-12T02:14:21.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.409+0000 7f4dc8b97740 1 -- 172.21.15.16:0/3134641676 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a378acb40 msgr2=0x560a378b3010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:14:21.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.409+0000 7f4dc8b97740 1 --2- 172.21.15.16:0/3134641676 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a378acb40 0x560a378b3010 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7f4db0004180 tx=0x7f4db0003b70 comp rx=0 tx=0).stop 2024-09-12T02:14:21.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.409+0000 7f4dc8b97740 1 -- 172.21.15.16:0/3134641676 shutdown_connections 2024-09-12T02:14:21.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.409+0000 7f4dc8b97740 1 --2- 172.21.15.16:0/3134641676 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x560a378c5f10 0x560a378c8300 crc :-1 s=READY pgs=376 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:21.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.409+0000 7f4dc8b97740 1 --2- 172.21.15.16:0/3134641676 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560a378b6370 0x560a378b3600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:21.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.409+0000 7f4d9f7fe640 0 client.19638 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:14:21.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.409+0000 7f4dc8b97740 1 --2- 172.21.15.16:0/3134641676 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a378acb40 0x560a378b3010 unknown :-1 s=CLOSED pgs=1750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:21.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.409+0000 7f4dc8b97740 1 --2- 172.21.15.16:0/3134641676 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560a378ab5d0 0x560a378b5d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:21.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.409+0000 7f4dc8b97740 1 -- 172.21.15.16:0/3134641676 >> 172.21.15.16:0/3134641676 conn(0x560a37874310 msgr2=0x560a378ad960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:14:21.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.409+0000 7f4dc8b97740 1 -- 172.21.15.16:0/3134641676 shutdown_connections 2024-09-12T02:14:21.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.409+0000 7f4dc8b97740 1 -- 172.21.15.16:0/3134641676 wait complete. 2024-09-12T02:14:21.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.410+0000 7f4dc8b97740 20 client.19638 trim_cache size 0 max 16384 2024-09-12T02:14:21.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.423+0000 7f4dc8b97740 1 Processor -- start 2024-09-12T02:14:21.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.423+0000 7f4dc8b97740 1 -- start start 2024-09-12T02:14:21.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.423+0000 7f4dc8b97740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a378ab5d0 0x560a378a7e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:21.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.423+0000 7f4dc8b97740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560a378acb40 0x560a378c2310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:21.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.423+0000 7f4dc8b97740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560a378b6370 0x560a378cd900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:21.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.423+0000 7f4dc8b97740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4dc01afd50 con 0x560a378ab5d0 2024-09-12T02:14:21.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.423+0000 7f4dc8b97740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4dc01ad620 con 0x560a378acb40 2024-09-12T02:14:21.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.423+0000 7f4dc8b97740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f4dc01ab470 con 0x560a378b6370 2024-09-12T02:14:21.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.423+0000 7f4db7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a378ab5d0 0x560a378a7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:21.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.423+0000 7f4db7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a378ab5d0 0x560a378a7e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55360/0 (socket says 172.21.15.16:55360) 2024-09-12T02:14:21.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.423+0000 7f4db7fff640 1 -- 172.21.15.16:0/3871845605 learned_addr learned my addr 172.21.15.16:0/3871845605 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:14:21.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.424+0000 7f4dbffff640 1 --2- 172.21.15.16:0/3871845605 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560a378acb40 0x560a378c2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:21.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.424+0000 7f4db7fff640 1 --2- 172.21.15.16:0/3871845605 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560a378b6370 0x560a378cd900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:21.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.424+0000 7f4db7fff640 1 -- 172.21.15.16:0/3871845605 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560a378b6370 msgr2=0x560a378cd900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:14:21.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.424+0000 7f4db7fff640 1 --2- 172.21.15.16:0/3871845605 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560a378b6370 0x560a378cd900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:21.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.424+0000 7f4db7fff640 1 -- 172.21.15.16:0/3871845605 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560a378acb40 msgr2=0x560a378c2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:14:21.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.424+0000 7f4db7fff640 1 --2- 172.21.15.16:0/3871845605 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560a378acb40 0x560a378c2310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:21.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.424+0000 7f4db7fff640 1 -- 172.21.15.16:0/3871845605 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a378b3970 con 0x560a378ab5d0 2024-09-12T02:14:21.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.424+0000 7f4db7fff640 1 --2- 172.21.15.16:0/3871845605 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a378ab5d0 0x560a378a7e10 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7f4da8009230 tx=0x7f4da8005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b597e8b53a1ad2f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:14:21.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.424+0000 7f4db5ffb640 1 -- 172.21.15.16:0/3871845605 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4da8035040 con 0x560a378ab5d0 2024-09-12T02:14:21.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.424+0000 7f4db5ffb640 1 -- 172.21.15.16:0/3871845605 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4da8005040 con 0x560a378ab5d0 2024-09-12T02:14:21.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.424+0000 7f4db5ffb640 1 -- 172.21.15.16:0/3871845605 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4da8004860 con 0x560a378ab5d0 2024-09-12T02:14:21.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.425+0000 7f4dc8b97740 1 -- 172.21.15.16:0/3871845605 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a378ab5d0 msgr2=0x560a378a7e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:14:21.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.425+0000 7f4dc8b97740 1 --2- 172.21.15.16:0/3871845605 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a378ab5d0 0x560a378a7e10 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7f4da8009230 tx=0x7f4da8005e80 comp rx=0 tx=0).stop 2024-09-12T02:14:21.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.425+0000 7f4dc8b97740 1 -- 172.21.15.16:0/3871845605 shutdown_connections 2024-09-12T02:14:21.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.425+0000 7f4dc8b97740 1 --2- 172.21.15.16:0/3871845605 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560a378b6370 0x560a378cd900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:21.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.425+0000 7f4dc8b97740 1 --2- 172.21.15.16:0/3871845605 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560a378acb40 0x560a378c2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:21.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.425+0000 7f4dc8b97740 1 --2- 172.21.15.16:0/3871845605 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a378ab5d0 0x560a378a7e10 unknown :-1 s=CLOSED pgs=1751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:21.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.425+0000 7f4dc8b97740 1 -- 172.21.15.16:0/3871845605 >> 172.21.15.16:0/3871845605 conn(0x560a37874310 msgr2=0x560a378a7700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:14:21.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.425+0000 7f4dc8b97740 1 -- 172.21.15.16:0/3871845605 shutdown_connections 2024-09-12T02:14:21.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.425+0000 7f4dc8b97740 1 -- 172.21.15.16:0/3871845605 wait complete. 2024-09-12T02:14:21.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.425+0000 7f4dc8b97740 1 -- start start 2024-09-12T02:14:21.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.426+0000 7f4dc8b97740 1 Processor -- start 2024-09-12T02:14:21.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.426+0000 7f4dc8b97740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560a378ab5d0 0x560a3792c050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:21.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.426+0000 7f4dc8b97740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a378acb40 0x560a378be480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:21.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.426+0000 7f4dc8b97740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560a378b6370 0x560a378be9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:21.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.426+0000 7f4dc8b97740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4dc01abae0 con 0x560a378acb40 2024-09-12T02:14:21.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.426+0000 7f4dc8b97740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4dc01acf50 con 0x560a378ab5d0 2024-09-12T02:14:21.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.426+0000 7f4dc8b97740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f4dc01ac1b0 con 0x560a378b6370 2024-09-12T02:14:21.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.426+0000 7f4dc8b97740 10 client.0 _ll_register_callbacks cb 0x7f4da8005d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:14:21.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.426+0000 7f4dbffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a378acb40 0x560a378be480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:21.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.426+0000 7f4dbffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a378acb40 0x560a378be480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55372/0 (socket says 172.21.15.16:55372) 2024-09-12T02:14:21.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.426+0000 7f4dbffff640 1 -- 172.21.15.16:0/3840096955 learned_addr learned my addr 172.21.15.16:0/3840096955 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:14:21.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.426+0000 7f4db7fff640 1 --2- 172.21.15.16:0/3840096955 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560a378ab5d0 0x560a3792c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:21.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.426+0000 7f4db7fff640 1 --2- 172.21.15.16:0/3840096955 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560a378b6370 0x560a378be9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:21.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.426+0000 7f4dbffff640 1 -- 172.21.15.16:0/3840096955 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560a378b6370 msgr2=0x560a378be9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:14:21.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.426+0000 7f4dbffff640 1 --2- 172.21.15.16:0/3840096955 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560a378b6370 0x560a378be9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:21.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.426+0000 7f4dbffff640 1 -- 172.21.15.16:0/3840096955 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560a378ab5d0 msgr2=0x560a3792c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:14:21.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.426+0000 7f4dbffff640 1 --2- 172.21.15.16:0/3840096955 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560a378ab5d0 0x560a3792c050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:21.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.426+0000 7f4dbffff640 1 -- 172.21.15.16:0/3840096955 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a378bf0b0 con 0x560a378acb40 2024-09-12T02:14:21.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.427+0000 7f4dbffff640 1 --2- 172.21.15.16:0/3840096955 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a378acb40 0x560a378be480 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7f4db00107e0 tx=0x7f4db000bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7580e8776e3a19d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:14:21.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.427+0000 7f4d9f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:14:21.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.427+0000 7f4d9f7fe640 1 -- 172.21.15.16:0/3840096955 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4db0035030 con 0x560a378acb40 2024-09-12T02:14:21.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.427+0000 7f4d9f7fe640 1 -- 172.21.15.16:0/3840096955 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f4db0022070 con 0x560a378acb40 2024-09-12T02:14:21.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.427+0000 7f4dc8b97740 1 -- 172.21.15.16:0/3840096955 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4da8005040 con 0x560a378acb40 2024-09-12T02:14:21.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.427+0000 7f4d9f7fe640 1 -- 172.21.15.16:0/3840096955 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4db00041b0 con 0x560a378acb40 2024-09-12T02:14:21.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.427+0000 7f4dc8b97740 10 client.19644 fetch_fsmap learned FSMap version 25 2024-09-12T02:14:21.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.427+0000 7f4dc8b97740 1 -- 172.21.15.16:0/3840096955 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x560a37934d40 con 0x560a378acb40 2024-09-12T02:14:21.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.427+0000 7f4d9f7fe640 1 -- 172.21.15.16:0/3840096955 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4da8005040 con 0x560a378acb40 2024-09-12T02:14:21.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.427+0000 7f4d9f7fe640 1 -- 172.21.15.16:0/3840096955 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f4db002f040 con 0x560a378acb40 2024-09-12T02:14:21.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.427+0000 7f4dc8b97740 10 client.19644 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:14:21.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.427+0000 7f4dc8b97740 10 client.19644 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:14:21.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.427+0000 7f4dc8b97740 20 client.19644 populate_metadata read hostname 'smithi016' 2024-09-12T02:14:21.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.427+0000 7f4dc8b97740 1 --2- 172.21.15.16:0/3840096955 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x560a378c5f10 0x560a37937440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:21.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.428+0000 7f4dc8b97740 4 client.19644 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T02:14:21.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.428+0000 7f4dc8b97740 1 -- 172.21.15.16:0/3840096955 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560a37818d70 con 0x560a378c5f10 2024-09-12T02:14:21.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.428+0000 7f4db7fff640 1 --2- 172.21.15.16:0/3840096955 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x560a378c5f10 0x560a37937440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:21.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.428+0000 7f4db7fff640 1 --2- 172.21.15.16:0/3840096955 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x560a378c5f10 0x560a37937440 crc :-1 s=READY pgs=377 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a45e45b640eefe66 server_cookie=f5dfe6b525cb1da9 in_seq=0 out_seq=0 2024-09-12T02:14:21.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.428+0000 7f4d9f7fe640 10 client.19644 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:14:21.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.428+0000 7f4d9f7fe640 1 -- 172.21.15.16:0/3840096955 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560a37818d70 con 0x560a378c5f10 2024-09-12T02:14:21.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.428+0000 7f4d9f7fe640 10 client.19644 handle_command_reply: tid=0 2024-09-12T02:14:21.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.428+0000 7f4dc8b97740 1 client.19644 shutdown 2024-09-12T02:14:21.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.429+0000 7f4dc8b97740 1 -- 172.21.15.16:0/3840096955 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a378acb40 msgr2=0x560a378be480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:14:21.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.429+0000 7f4dc8b97740 1 --2- 172.21.15.16:0/3840096955 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a378acb40 0x560a378be480 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7f4db00107e0 tx=0x7f4db000bab0 comp rx=0 tx=0).stop 2024-09-12T02:14:21.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.429+0000 7f4dc8b97740 1 -- 172.21.15.16:0/3840096955 shutdown_connections 2024-09-12T02:14:21.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.429+0000 7f4dc8b97740 1 --2- 172.21.15.16:0/3840096955 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x560a378c5f10 0x560a37937440 crc :-1 s=READY pgs=377 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:21.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.429+0000 7f4d9f7fe640 0 client.19644 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:14:21.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.429+0000 7f4dc8b97740 1 --2- 172.21.15.16:0/3840096955 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560a378b6370 0x560a378be9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:21.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.429+0000 7f4dc8b97740 1 --2- 172.21.15.16:0/3840096955 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a378acb40 0x560a378be480 unknown :-1 s=CLOSED pgs=1752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:21.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.429+0000 7f4dc8b97740 1 --2- 172.21.15.16:0/3840096955 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560a378ab5d0 0x560a3792c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:21.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.429+0000 7f4dc8b97740 1 -- 172.21.15.16:0/3840096955 >> 172.21.15.16:0/3840096955 conn(0x560a37874310 msgr2=0x560a378a64e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:14:21.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.429+0000 7f4dc8b97740 1 -- 172.21.15.16:0/3840096955 shutdown_connections 2024-09-12T02:14:21.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.429+0000 7f4dc8b97740 1 -- 172.21.15.16:0/3840096955 wait complete. 2024-09-12T02:14:21.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.429+0000 7f4dc8b97740 20 client.19644 trim_cache size 0 max 16384 2024-09-12T02:14:21.436 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T02:14:21.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.429+0000 7f4dc67be640 1 -- 172.21.15.16:0/307025104 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f4dac0a2c00 msgr2=0x7f4dac0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:14:21.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.429+0000 7f4dc67be640 1 --2- 172.21.15.16:0/307025104 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f4dac0a2c00 0x7f4dac0a50c0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f4db8097330 tx=0x7f4db80c59b0 comp rx=0 tx=0).stop 2024-09-12T02:14:21.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.429+0000 7f4dc67be640 1 -- 172.21.15.16:0/307025104 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4dc0106a80 msgr2=0x7f4dc019cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:14:21.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.429+0000 7f4dc67be640 1 --2- 172.21.15.16:0/307025104 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4dc0106a80 0x7f4dc019cd60 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7f4db800b970 tx=0x7f4db800be30 comp rx=0 tx=0).stop 2024-09-12T02:14:21.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.429+0000 7f4dc67be640 1 -- 172.21.15.16:0/307025104 shutdown_connections 2024-09-12T02:14:21.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.429+0000 7f4dc67be640 1 --2- 172.21.15.16:0/307025104 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f4dac0a2c00 0x7f4dac0a50c0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:21.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.430+0000 7f4dc67be640 1 --2- 172.21.15.16:0/307025104 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4dc0106a80 0x7f4dc019cd60 unknown :-1 s=CLOSED pgs=1749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:21.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.430+0000 7f4dc67be640 1 --2- 172.21.15.16:0/307025104 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4dc0101420 0x7f4dc01989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:21.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.430+0000 7f4dc67be640 1 --2- 172.21.15.16:0/307025104 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4dc0100a80 0x7f4dc0198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:21.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.430+0000 7f4dc67be640 1 -- 172.21.15.16:0/307025104 >> 172.21.15.16:0/307025104 conn(0x7f4dc00fc7a0 msgr2=0x7f4dc0104830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:14:21.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.430+0000 7f4dc67be640 1 -- 172.21.15.16:0/307025104 shutdown_connections 2024-09-12T02:14:21.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:21.430+0000 7f4dc67be640 1 -- 172.21.15.16:0/307025104 wait complete. 2024-09-12T02:14:22.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:14:22 smithi016 ceph-mon[30947]: pgmap v708: 97 pgs: 97 active+clean; 2.4 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-12T02:14:22.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:14:22 smithi184 ceph-mon[35593]: pgmap v708: 97 pgs: 97 active+clean; 2.4 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-12T02:14:22.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:14:22 smithi033 ceph-mon[35332]: pgmap v708: 97 pgs: 97 active+clean; 2.4 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-12T02:14:23.142 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-12T02:14:23.142 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-12T02:14:23.415 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:14:23.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.832+0000 7f29825dc640 1 -- 172.21.15.16:0/1860679809 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f297c102d40 msgr2=0x7f297c103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:14:23.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.832+0000 7f29825dc640 1 --2- 172.21.15.16:0/1860679809 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f297c102d40 0x7f297c103120 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7f296c009900 tx=0x7f296c02a480 comp rx=0 tx=0).stop 2024-09-12T02:14:23.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.832+0000 7f29825dc640 1 -- 172.21.15.16:0/1860679809 shutdown_connections 2024-09-12T02:14:23.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.832+0000 7f29825dc640 1 --2- 172.21.15.16:0/1860679809 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f297c10c170 0x7f297c10e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:23.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.832+0000 7f29825dc640 1 --2- 172.21.15.16:0/1860679809 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f297c103660 0x7f297c10bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:23.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.832+0000 7f29825dc640 1 --2- 172.21.15.16:0/1860679809 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f297c102d40 0x7f297c103120 unknown :-1 s=CLOSED pgs=1753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:23.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.832+0000 7f29825dc640 1 -- 172.21.15.16:0/1860679809 >> 172.21.15.16:0/1860679809 conn(0x7f297c0fc7a0 msgr2=0x7f297c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:14:23.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.832+0000 7f29825dc640 1 -- 172.21.15.16:0/1860679809 shutdown_connections 2024-09-12T02:14:23.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.833+0000 7f29825dc640 1 -- 172.21.15.16:0/1860679809 wait complete. 2024-09-12T02:14:23.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.833+0000 7f29825dc640 1 Processor -- start 2024-09-12T02:14:23.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.833+0000 7f29825dc640 1 -- start start 2024-09-12T02:14:23.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.833+0000 7f29825dc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f297c102d40 0x7f297c198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:23.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.833+0000 7f29825dc640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f297c103660 0x7f297c198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:23.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.833+0000 7f29825dc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f297c10c170 0x7f297c19cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:23.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.833+0000 7f29825dc640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f297c111f70 con 0x7f297c102d40 2024-09-12T02:14:23.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.833+0000 7f29825dc640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f297c111df0 con 0x7f297c10c170 2024-09-12T02:14:23.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.833+0000 7f29825dc640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f297c1120f0 con 0x7f297c103660 2024-09-12T02:14:23.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.834+0000 7f297bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f297c102d40 0x7f297c198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:23.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.834+0000 7f297bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f297c102d40 0x7f297c198430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55390/0 (socket says 172.21.15.16:55390) 2024-09-12T02:14:23.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.834+0000 7f297bfff640 1 -- 172.21.15.16:0/54083660 learned_addr learned my addr 172.21.15.16:0/54083660 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:14:23.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.834+0000 7f2973fff640 1 --2- 172.21.15.16:0/54083660 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f297c103660 0x7f297c198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:23.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.834+0000 7f2980b79640 1 --2- 172.21.15.16:0/54083660 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f297c10c170 0x7f297c19cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:23.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.834+0000 7f297bfff640 1 -- 172.21.15.16:0/54083660 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f297c103660 msgr2=0x7f297c198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:14:23.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.834+0000 7f297bfff640 1 --2- 172.21.15.16:0/54083660 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f297c103660 0x7f297c198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:23.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.834+0000 7f297bfff640 1 -- 172.21.15.16:0/54083660 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f297c10c170 msgr2=0x7f297c19cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:14:23.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.834+0000 7f297bfff640 1 --2- 172.21.15.16:0/54083660 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f297c10c170 0x7f297c19cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:23.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.834+0000 7f297bfff640 1 -- 172.21.15.16:0/54083660 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f297c19d3e0 con 0x7f297c102d40 2024-09-12T02:14:23.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.834+0000 7f297bfff640 1 --2- 172.21.15.16:0/54083660 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f297c102d40 0x7f297c198430 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7f296c006010 tx=0x7f296c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:14:23.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.834+0000 7f2979ffb640 1 -- 172.21.15.16:0/54083660 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f296c03f030 con 0x7f297c102d40 2024-09-12T02:14:23.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.834+0000 7f29825dc640 1 -- 172.21.15.16:0/54083660 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f297c19d670 con 0x7f297c102d40 2024-09-12T02:14:23.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.834+0000 7f2979ffb640 1 -- 172.21.15.16:0/54083660 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f296c003ce0 con 0x7f297c102d40 2024-09-12T02:14:23.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.834+0000 7f2979ffb640 1 -- 172.21.15.16:0/54083660 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f296c035070 con 0x7f297c102d40 2024-09-12T02:14:23.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.835+0000 7f29825dc640 1 -- 172.21.15.16:0/54083660 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f297c1a4fb0 con 0x7f297c102d40 2024-09-12T02:14:23.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.837+0000 7f2979ffb640 1 -- 172.21.15.16:0/54083660 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f296c042030 con 0x7f297c102d40 2024-09-12T02:14:23.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.837+0000 7f29849b5740 1 Processor -- start 2024-09-12T02:14:23.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.837+0000 7f29849b5740 1 -- start start 2024-09-12T02:14:23.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.837+0000 7f29849b5740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56208e441f70 0x56208e442350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:23.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.837+0000 7f29849b5740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56208e442890 0x56208e44cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:23.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.837+0000 7f29849b5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56208e44d150 0x56208e44f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:23.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.838+0000 7f29849b5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56208e3a3280 con 0x56208e44d150 2024-09-12T02:14:23.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.838+0000 7f29849b5740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56208e34ab10 con 0x56208e441f70 2024-09-12T02:14:23.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.838+0000 7f29849b5740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x56208e39f240 con 0x56208e442890 2024-09-12T02:14:23.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.838+0000 7f2973fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56208e44d150 0x56208e44f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:23.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.838+0000 7f2973fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56208e44d150 0x56208e44f540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55404/0 (socket says 172.21.15.16:55404) 2024-09-12T02:14:23.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.838+0000 7f2973fff640 1 -- 172.21.15.16:0/2722763412 learned_addr learned my addr 172.21.15.16:0/2722763412 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:14:23.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.838+0000 7f297bfff640 1 --2- 172.21.15.16:0/2722763412 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56208e442890 0x56208e44cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:23.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.838+0000 7f2980b79640 1 --2- 172.21.15.16:0/2722763412 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56208e441f70 0x56208e442350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:23.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.838+0000 7f2979ffb640 1 --2- 172.21.15.16:0/54083660 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f29680a2b80 0x7f29680a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:23.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.838+0000 7f2980b79640 1 --2- 172.21.15.16:0/54083660 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f29680a2b80 0x7f29680a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:23.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.838+0000 7f2979ffb640 1 -- 172.21.15.16:0/54083660 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f296c0ef4a0 con 0x7f297c102d40 2024-09-12T02:14:23.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.838+0000 7f297bfff640 1 -- 172.21.15.16:0/2722763412 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56208e441f70 msgr2=0x56208e442350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:14:23.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.838+0000 7f297bfff640 1 --2- 172.21.15.16:0/2722763412 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56208e441f70 0x56208e442350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:23.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.838+0000 7f297bfff640 1 -- 172.21.15.16:0/2722763412 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56208e44d150 msgr2=0x56208e44f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:14:23.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.838+0000 7f297bfff640 1 --2- 172.21.15.16:0/2722763412 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56208e44d150 0x56208e44f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:23.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.838+0000 7f297bfff640 1 -- 172.21.15.16:0/2722763412 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56208e3571f0 con 0x56208e442890 2024-09-12T02:14:23.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.838+0000 7f2980b79640 1 --2- 172.21.15.16:0/2722763412 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56208e441f70 0x56208e442350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:14:23.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.838+0000 7f2973fff640 1 --2- 172.21.15.16:0/2722763412 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56208e44d150 0x56208e44f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:14:23.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.839+0000 7f297bfff640 1 --2- 172.21.15.16:0/2722763412 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56208e442890 0x56208e44cc10 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f296c08a800 tx=0x7f296c02a480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fcef0cb4f09e20e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:14:23.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.839+0000 7f2980b79640 1 --2- 172.21.15.16:0/54083660 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f29680a2b80 0x7f29680a5040 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f2974005fd0 tx=0x7f2974014040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:14:23.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.839+0000 7f29727fc640 1 -- 172.21.15.16:0/2722763412 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f296c049070 con 0x56208e442890 2024-09-12T02:14:23.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.839+0000 7f29727fc640 1 -- 172.21.15.16:0/2722763412 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f296c03b580 con 0x56208e442890 2024-09-12T02:14:23.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.839+0000 7f29727fc640 1 -- 172.21.15.16:0/2722763412 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f296c0c0170 con 0x56208e442890 2024-09-12T02:14:23.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.839+0000 7f29849b5740 1 -- 172.21.15.16:0/2722763412 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56208e442890 msgr2=0x56208e44cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:14:23.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.839+0000 7f29849b5740 1 --2- 172.21.15.16:0/2722763412 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56208e442890 0x56208e44cc10 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f296c08a800 tx=0x7f296c02a480 comp rx=0 tx=0).stop 2024-09-12T02:14:23.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.839+0000 7f29849b5740 1 -- 172.21.15.16:0/2722763412 shutdown_connections 2024-09-12T02:14:23.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.839+0000 7f29849b5740 1 --2- 172.21.15.16:0/2722763412 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56208e44d150 0x56208e44f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:23.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.839+0000 7f29849b5740 1 --2- 172.21.15.16:0/2722763412 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56208e442890 0x56208e44cc10 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:23.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.839+0000 7f29849b5740 1 --2- 172.21.15.16:0/2722763412 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56208e441f70 0x56208e442350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:23.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.839+0000 7f29849b5740 1 -- 172.21.15.16:0/2722763412 >> 172.21.15.16:0/2722763412 conn(0x56208e43e500 msgr2=0x56208e43e8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:14:23.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.839+0000 7f29849b5740 1 -- 172.21.15.16:0/2722763412 shutdown_connections 2024-09-12T02:14:23.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.840+0000 7f29849b5740 1 -- 172.21.15.16:0/2722763412 wait complete. 2024-09-12T02:14:23.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.840+0000 7f29849b5740 1 -- start start 2024-09-12T02:14:23.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.840+0000 7f29849b5740 1 Processor -- start 2024-09-12T02:14:23.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.840+0000 7f29849b5740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56208e441f70 0x56208e44a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:23.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.840+0000 7f29849b5740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56208e442890 0x56208e44a600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:23.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.840+0000 7f29849b5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56208e44d150 0x56208e44ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:23.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.840+0000 7f29849b5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56208e44fae0 con 0x56208e44d150 2024-09-12T02:14:23.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.840+0000 7f29849b5740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56208e35df10 con 0x56208e441f70 2024-09-12T02:14:23.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.840+0000 7f29849b5740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x56208e44fc60 con 0x56208e442890 2024-09-12T02:14:23.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.840+0000 7f29849b5740 10 client.0 _ll_register_callbacks cb 0x56208e326a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:14:23.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.841+0000 7f2980b79640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56208e44d150 0x56208e44ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:23.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.841+0000 7f2973fff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56208e442890 0x56208e44a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:23.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.841+0000 7f2980b79640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56208e44d150 0x56208e44ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55420/0 (socket says 172.21.15.16:55420) 2024-09-12T02:14:23.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.841+0000 7f2980b79640 1 -- 172.21.15.16:0/2448701358 learned_addr learned my addr 172.21.15.16:0/2448701358 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:14:23.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.841+0000 7f2973fff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56208e442890 0x56208e44a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:56854/0 (socket says 172.21.15.16:56854) 2024-09-12T02:14:23.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.841+0000 7f297bfff640 1 --2- 172.21.15.16:0/2448701358 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56208e441f70 0x56208e44a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:23.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.841+0000 7f2973fff640 1 -- 172.21.15.16:0/2448701358 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56208e441f70 msgr2=0x56208e44a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:14:23.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.841+0000 7f2973fff640 1 --2- 172.21.15.16:0/2448701358 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56208e441f70 0x56208e44a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:23.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.841+0000 7f2973fff640 1 -- 172.21.15.16:0/2448701358 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56208e44d150 msgr2=0x56208e44ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:14:23.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.841+0000 7f2973fff640 1 --2- 172.21.15.16:0/2448701358 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56208e44d150 0x56208e44ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:23.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.841+0000 7f2973fff640 1 -- 172.21.15.16:0/2448701358 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56208e1146a0 con 0x56208e442890 2024-09-12T02:14:23.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.841+0000 7f2980b79640 1 --2- 172.21.15.16:0/2448701358 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56208e44d150 0x56208e44ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:14:23.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.841+0000 7f297bfff640 1 --2- 172.21.15.16:0/2448701358 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56208e441f70 0x56208e44a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:14:23.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.841+0000 7f2973fff640 1 --2- 172.21.15.16:0/2448701358 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56208e442890 0x56208e44a600 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f296400bf30 tx=0x7f29640118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ecdce149ff25c27d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:14:23.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.841+0000 7f295b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-09-12T02:14:23.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.841+0000 7f295b7fe640 1 -- 172.21.15.16:0/2448701358 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2964004020 con 0x56208e442890 2024-09-12T02:14:23.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.841+0000 7f29849b5740 1 -- 172.21.15.16:0/2448701358 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56208e3b07f0 con 0x56208e442890 2024-09-12T02:14:23.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.841+0000 7f295b7fe640 1 -- 172.21.15.16:0/2448701358 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f29640041c0 con 0x56208e442890 2024-09-12T02:14:23.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.842+0000 7f295b7fe640 1 -- 172.21.15.16:0/2448701358 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f29640043c0 con 0x56208e442890 2024-09-12T02:14:23.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.842+0000 7f29849b5740 10 client.28234 fetch_fsmap learned FSMap version 25 2024-09-12T02:14:23.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.842+0000 7f29849b5740 1 -- 172.21.15.16:0/2448701358 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56208e45b630 con 0x56208e442890 2024-09-12T02:14:23.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.842+0000 7f295b7fe640 1 -- 172.21.15.16:0/2448701358 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x56208e3b07f0 con 0x56208e442890 2024-09-12T02:14:23.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.842+0000 7f295b7fe640 1 -- 172.21.15.16:0/2448701358 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f2964003b10 con 0x56208e442890 2024-09-12T02:14:23.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.842+0000 7f29849b5740 10 client.28234 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:14:23.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.842+0000 7f29849b5740 10 client.28234 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:14:23.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.842+0000 7f29849b5740 20 client.28234 populate_metadata read hostname 'smithi016' 2024-09-12T02:14:23.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.842+0000 7f29849b5740 1 --2- 172.21.15.16:0/2448701358 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x56208e45caf0 0x56208e45eee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:23.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.842+0000 7f29849b5740 4 client.28234 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:14:23.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.842+0000 7f29849b5740 1 -- 172.21.15.16:0/2448701358 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56208e3aad00 con 0x56208e45caf0 2024-09-12T02:14:23.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.843+0000 7f297bfff640 1 --2- 172.21.15.16:0/2448701358 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x56208e45caf0 0x56208e45eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:23.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.843+0000 7f295b7fe640 10 client.28234 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:14:23.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.843+0000 7f297bfff640 1 --2- 172.21.15.16:0/2448701358 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x56208e45caf0 0x56208e45eee0 crc :-1 s=READY pgs=378 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5559990185335d14 server_cookie=b2ea214006b42136 in_seq=0 out_seq=0 2024-09-12T02:14:23.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.844+0000 7f295b7fe640 1 -- 172.21.15.16:0/2448701358 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56208e3aad00 con 0x56208e45caf0 2024-09-12T02:14:23.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.844+0000 7f295b7fe640 10 client.28234 handle_command_reply: tid=0 2024-09-12T02:14:23.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.844+0000 7f29849b5740 1 client.28234 shutdown 2024-09-12T02:14:23.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.844+0000 7f29849b5740 1 -- 172.21.15.16:0/2448701358 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56208e442890 msgr2=0x56208e44a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:14:23.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.844+0000 7f29849b5740 1 --2- 172.21.15.16:0/2448701358 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56208e442890 0x56208e44a600 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f296400bf30 tx=0x7f29640118d0 comp rx=0 tx=0).stop 2024-09-12T02:14:23.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.844+0000 7f29849b5740 1 -- 172.21.15.16:0/2448701358 shutdown_connections 2024-09-12T02:14:23.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.844+0000 7f29849b5740 1 --2- 172.21.15.16:0/2448701358 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x56208e45caf0 0x56208e45eee0 crc :-1 s=READY pgs=378 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:23.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.844+0000 7f29849b5740 1 --2- 172.21.15.16:0/2448701358 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56208e44d150 0x56208e44ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:23.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.844+0000 7f29849b5740 1 --2- 172.21.15.16:0/2448701358 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56208e442890 0x56208e44a600 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:23.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.844+0000 7f295b7fe640 0 client.28234 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:14:23.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.844+0000 7f29849b5740 1 --2- 172.21.15.16:0/2448701358 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56208e441f70 0x56208e44a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:23.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.844+0000 7f29849b5740 1 -- 172.21.15.16:0/2448701358 >> 172.21.15.16:0/2448701358 conn(0x56208e43e500 msgr2=0x56208e439270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:14:23.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.844+0000 7f29849b5740 1 -- 172.21.15.16:0/2448701358 shutdown_connections 2024-09-12T02:14:23.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.844+0000 7f29849b5740 1 -- 172.21.15.16:0/2448701358 wait complete. 2024-09-12T02:14:23.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.844+0000 7f29849b5740 20 client.28234 trim_cache size 0 max 16384 2024-09-12T02:14:23.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.861+0000 7f29849b5740 1 Processor -- start 2024-09-12T02:14:23.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.862+0000 7f29849b5740 1 -- start start 2024-09-12T02:14:23.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.862+0000 7f29849b5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56208e441f70 0x56208e4647d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:23.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.862+0000 7f29849b5740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56208e442890 0x56208e464d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:23.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.862+0000 7f29849b5740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56208e44d150 0x56208e447320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:23.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.862+0000 7f29849b5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f297c1adb00 con 0x56208e441f70 2024-09-12T02:14:23.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.862+0000 7f29849b5740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f297c1ab3d0 con 0x56208e442890 2024-09-12T02:14:23.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.862+0000 7f29849b5740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f297c1a9220 con 0x56208e44d150 2024-09-12T02:14:23.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.862+0000 7f297bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56208e441f70 0x56208e4647d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:23.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.862+0000 7f297bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56208e441f70 0x56208e4647d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55434/0 (socket says 172.21.15.16:55434) 2024-09-12T02:14:23.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.862+0000 7f297bfff640 1 -- 172.21.15.16:0/1081776726 learned_addr learned my addr 172.21.15.16:0/1081776726 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:14:23.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.862+0000 7f2980b79640 1 --2- 172.21.15.16:0/1081776726 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56208e44d150 0x56208e447320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:23.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.862+0000 7f2973fff640 1 --2- 172.21.15.16:0/1081776726 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56208e442890 0x56208e464d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:23.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.862+0000 7f297bfff640 1 -- 172.21.15.16:0/1081776726 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56208e44d150 msgr2=0x56208e447320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:14:23.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.862+0000 7f297bfff640 1 --2- 172.21.15.16:0/1081776726 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56208e44d150 0x56208e447320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:23.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.862+0000 7f297bfff640 1 -- 172.21.15.16:0/1081776726 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56208e442890 msgr2=0x56208e464d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:14:23.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.862+0000 7f297bfff640 1 --2- 172.21.15.16:0/1081776726 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56208e442890 0x56208e464d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:23.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.862+0000 7f297bfff640 1 -- 172.21.15.16:0/1081776726 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56208e44c990 con 0x56208e441f70 2024-09-12T02:14:23.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.862+0000 7f297bfff640 1 --2- 172.21.15.16:0/1081776726 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56208e441f70 0x56208e4647d0 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7f296c0b82d0 tx=0x7f296c0bbf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=511b14984d62299 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:14:23.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.862+0000 7f2971ffb640 1 -- 172.21.15.16:0/1081776726 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f296c049040 con 0x56208e441f70 2024-09-12T02:14:23.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.863+0000 7f2971ffb640 1 -- 172.21.15.16:0/1081776726 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f296c0c4a30 con 0x56208e441f70 2024-09-12T02:14:23.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.863+0000 7f2971ffb640 1 -- 172.21.15.16:0/1081776726 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f296c0b84e0 con 0x56208e441f70 2024-09-12T02:14:23.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.863+0000 7f29849b5740 1 -- 172.21.15.16:0/1081776726 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56208e441f70 msgr2=0x56208e4647d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:14:23.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.863+0000 7f29849b5740 1 --2- 172.21.15.16:0/1081776726 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56208e441f70 0x56208e4647d0 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7f296c0b82d0 tx=0x7f296c0bbf90 comp rx=0 tx=0).stop 2024-09-12T02:14:23.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.863+0000 7f29849b5740 1 -- 172.21.15.16:0/1081776726 shutdown_connections 2024-09-12T02:14:23.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.863+0000 7f29849b5740 1 --2- 172.21.15.16:0/1081776726 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56208e44d150 0x56208e447320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:23.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.863+0000 7f29849b5740 1 --2- 172.21.15.16:0/1081776726 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56208e442890 0x56208e464d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:23.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.863+0000 7f29849b5740 1 --2- 172.21.15.16:0/1081776726 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56208e441f70 0x56208e4647d0 unknown :-1 s=CLOSED pgs=1755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:23.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.863+0000 7f29849b5740 1 -- 172.21.15.16:0/1081776726 >> 172.21.15.16:0/1081776726 conn(0x56208e43e500 msgr2=0x56208e439270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:14:23.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.863+0000 7f29849b5740 1 -- 172.21.15.16:0/1081776726 shutdown_connections 2024-09-12T02:14:23.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.863+0000 7f29849b5740 1 -- 172.21.15.16:0/1081776726 wait complete. 2024-09-12T02:14:23.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.863+0000 7f29849b5740 1 -- start start 2024-09-12T02:14:23.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.864+0000 7f29849b5740 1 Processor -- start 2024-09-12T02:14:23.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.864+0000 7f29849b5740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56208e441f70 0x56208e450dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:23.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.864+0000 7f29849b5740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56208e442890 0x56208e451300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:23.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.864+0000 7f29849b5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56208e44d150 0x56208e451860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:23.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.864+0000 7f29849b5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f297c1a9890 con 0x56208e44d150 2024-09-12T02:14:23.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.864+0000 7f29849b5740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f297c1aad00 con 0x56208e441f70 2024-09-12T02:14:23.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.864+0000 7f29849b5740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f297c1a9f60 con 0x56208e442890 2024-09-12T02:14:23.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.864+0000 7f29849b5740 10 client.0 _ll_register_callbacks cb 0x7f296400bb40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:14:23.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.864+0000 7f2980b79640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56208e44d150 0x56208e451860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:23.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.864+0000 7f2980b79640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56208e44d150 0x56208e451860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55440/0 (socket says 172.21.15.16:55440) 2024-09-12T02:14:23.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.864+0000 7f2980b79640 1 -- 172.21.15.16:0/2034443384 learned_addr learned my addr 172.21.15.16:0/2034443384 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:14:23.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.864+0000 7f297bfff640 1 --2- 172.21.15.16:0/2034443384 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56208e441f70 0x56208e450dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:23.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.864+0000 7f2973fff640 1 --2- 172.21.15.16:0/2034443384 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56208e442890 0x56208e451300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:23.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.865+0000 7f2980b79640 1 -- 172.21.15.16:0/2034443384 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56208e442890 msgr2=0x56208e451300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:14:23.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.865+0000 7f2980b79640 1 --2- 172.21.15.16:0/2034443384 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56208e442890 0x56208e451300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:23.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.865+0000 7f2980b79640 1 -- 172.21.15.16:0/2034443384 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56208e441f70 msgr2=0x56208e450dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:14:23.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.865+0000 7f2980b79640 1 --2- 172.21.15.16:0/2034443384 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56208e441f70 0x56208e450dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:23.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.865+0000 7f2980b79640 1 -- 172.21.15.16:0/2034443384 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56208e4c6a70 con 0x56208e44d150 2024-09-12T02:14:23.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.865+0000 7f2980b79640 1 --2- 172.21.15.16:0/2034443384 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56208e44d150 0x56208e451860 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7f297401fe80 tx=0x7f2974004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34e45b4f24bc8b63 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:14:23.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.865+0000 7f295b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:14:23.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.865+0000 7f295b7fe640 1 -- 172.21.15.16:0/2034443384 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f297402d060 con 0x56208e44d150 2024-09-12T02:14:23.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.865+0000 7f295b7fe640 1 -- 172.21.15.16:0/2034443384 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f2974028410 con 0x56208e44d150 2024-09-12T02:14:23.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.865+0000 7f29849b5740 1 -- 172.21.15.16:0/2034443384 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f296c0c4a30 con 0x56208e44d150 2024-09-12T02:14:23.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.865+0000 7f295b7fe640 1 -- 172.21.15.16:0/2034443384 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2974028610 con 0x56208e44d150 2024-09-12T02:14:23.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.865+0000 7f29849b5740 10 client.19662 fetch_fsmap learned FSMap version 25 2024-09-12T02:14:23.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.865+0000 7f29849b5740 1 -- 172.21.15.16:0/2034443384 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56208e4c7e10 con 0x56208e44d150 2024-09-12T02:14:23.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.865+0000 7f295b7fe640 1 -- 172.21.15.16:0/2034443384 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f296c0c4a30 con 0x56208e44d150 2024-09-12T02:14:23.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.865+0000 7f295b7fe640 1 -- 172.21.15.16:0/2034443384 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f2974032070 con 0x56208e44d150 2024-09-12T02:14:23.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.866+0000 7f29849b5740 10 client.19662 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:14:23.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.866+0000 7f29849b5740 10 client.19662 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:14:23.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.866+0000 7f29849b5740 20 client.19662 populate_metadata read hostname 'smithi016' 2024-09-12T02:14:23.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.866+0000 7f29849b5740 1 --2- 172.21.15.16:0/2034443384 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x56208e4c8530 0x56208e4ca920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:23.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.866+0000 7f29849b5740 4 client.19662 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-12T02:14:23.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.866+0000 7f29849b5740 1 -- 172.21.15.16:0/2034443384 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56208e327110 con 0x56208e4c8530 2024-09-12T02:14:23.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.866+0000 7f297bfff640 1 --2- 172.21.15.16:0/2034443384 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x56208e4c8530 0x56208e4ca920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:23.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.866+0000 7f297bfff640 1 --2- 172.21.15.16:0/2034443384 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x56208e4c8530 0x56208e4ca920 crc :-1 s=READY pgs=379 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=47a3aa13f558b5dd server_cookie=859c37e90fc6b8f2 in_seq=0 out_seq=0 2024-09-12T02:14:23.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.866+0000 7f295b7fe640 10 client.19662 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:14:23.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.867+0000 7f295b7fe640 1 -- 172.21.15.16:0/2034443384 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56208e327110 con 0x56208e4c8530 2024-09-12T02:14:23.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.867+0000 7f295b7fe640 10 client.19662 handle_command_reply: tid=0 2024-09-12T02:14:23.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.867+0000 7f29849b5740 1 client.19662 shutdown 2024-09-12T02:14:23.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.867+0000 7f29849b5740 1 -- 172.21.15.16:0/2034443384 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56208e44d150 msgr2=0x56208e451860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:14:23.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.867+0000 7f29849b5740 1 --2- 172.21.15.16:0/2034443384 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56208e44d150 0x56208e451860 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7f297401fe80 tx=0x7f2974004a40 comp rx=0 tx=0).stop 2024-09-12T02:14:23.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.868+0000 7f29849b5740 1 -- 172.21.15.16:0/2034443384 shutdown_connections 2024-09-12T02:14:23.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.868+0000 7f29849b5740 1 --2- 172.21.15.16:0/2034443384 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x56208e4c8530 0x56208e4ca920 crc :-1 s=READY pgs=379 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:23.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.868+0000 7f29849b5740 1 --2- 172.21.15.16:0/2034443384 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56208e44d150 0x56208e451860 unknown :-1 s=CLOSED pgs=1756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:23.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.868+0000 7f29849b5740 1 --2- 172.21.15.16:0/2034443384 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56208e442890 0x56208e451300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:23.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.868+0000 7f29849b5740 1 --2- 172.21.15.16:0/2034443384 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56208e441f70 0x56208e450dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:23.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.868+0000 7f29849b5740 1 -- 172.21.15.16:0/2034443384 >> 172.21.15.16:0/2034443384 conn(0x56208e43e500 msgr2=0x56208e44a060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:14:23.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.868+0000 7f295b7fe640 0 client.19662 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:14:23.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.868+0000 7f29849b5740 1 -- 172.21.15.16:0/2034443384 shutdown_connections 2024-09-12T02:14:23.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.868+0000 7f29849b5740 1 -- 172.21.15.16:0/2034443384 wait complete. 2024-09-12T02:14:23.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.868+0000 7f29849b5740 20 client.19662 trim_cache size 0 max 16384 2024-09-12T02:14:23.873 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T02:14:23.873 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-09-12T02:14:23.873 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "9a98f532-78d4-4c4f-92c7-ce4d4be9788a", 2024-09-12T02:14:23.873 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-09-12T02:14:23.873 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T02:14:23.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.868+0000 7f29825dc640 1 -- 172.21.15.16:0/54083660 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f29680a2b80 msgr2=0x7f29680a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:14:23.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.868+0000 7f29825dc640 1 --2- 172.21.15.16:0/54083660 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f29680a2b80 0x7f29680a5040 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f2974005fd0 tx=0x7f2974014040 comp rx=0 tx=0).stop 2024-09-12T02:14:23.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.868+0000 7f29825dc640 1 -- 172.21.15.16:0/54083660 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f297c102d40 msgr2=0x7f297c198430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:14:23.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.868+0000 7f29825dc640 1 --2- 172.21.15.16:0/54083660 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f297c102d40 0x7f297c198430 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7f296c006010 tx=0x7f296c0042f0 comp rx=0 tx=0).stop 2024-09-12T02:14:23.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.868+0000 7f29825dc640 1 -- 172.21.15.16:0/54083660 shutdown_connections 2024-09-12T02:14:23.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.868+0000 7f29825dc640 1 --2- 172.21.15.16:0/54083660 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f29680a2b80 0x7f29680a5040 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:23.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.868+0000 7f29825dc640 1 --2- 172.21.15.16:0/54083660 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f297c10c170 0x7f297c19cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:23.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.868+0000 7f29825dc640 1 --2- 172.21.15.16:0/54083660 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f297c103660 0x7f297c198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:23.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.868+0000 7f29825dc640 1 --2- 172.21.15.16:0/54083660 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f297c102d40 0x7f297c198430 unknown :-1 s=CLOSED pgs=1754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:23.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.868+0000 7f29825dc640 1 -- 172.21.15.16:0/54083660 >> 172.21.15.16:0/54083660 conn(0x7f297c0fc7a0 msgr2=0x7f297c0fd400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:14:23.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.869+0000 7f29825dc640 1 -- 172.21.15.16:0/54083660 shutdown_connections 2024-09-12T02:14:23.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:23.869+0000 7f29825dc640 1 -- 172.21.15.16:0/54083660 wait complete. 2024-09-12T02:14:24.429 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:14:24 smithi184 ceph-mon[35593]: pgmap v709: 97 pgs: 97 active+clean; 2.5 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 17 KiB/s wr, 14 op/s 2024-09-12T02:14:24.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:14:24 smithi016 ceph-mon[30947]: pgmap v709: 97 pgs: 97 active+clean; 2.5 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 17 KiB/s wr, 14 op/s 2024-09-12T02:14:24.471 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:14:24 smithi033 ceph-mon[35332]: pgmap v709: 97 pgs: 97 active+clean; 2.5 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 17 KiB/s wr, 14 op/s 2024-09-12T02:14:24.677 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T02:14:24.935 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:14:25.331 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:14:25 smithi016 ceph-mon[30947]: scrub queued for path: / 2024-09-12T02:14:25.331 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:14:25 smithi016 ceph-mon[30947]: scrub summary: idle+waiting paths [/] 2024-09-12T02:14:25.331 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:14:25 smithi016 ceph-mon[30947]: scrub summary: active paths [/] 2024-09-12T02:14:25.332 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:14:25 smithi016 ceph-mon[30947]: scrub summary: idle+waiting paths [/] 2024-09-12T02:14:25.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.417+0000 7f8c18b5f640 1 -- 172.21.15.16:0/2531583425 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c14102790 msgr2=0x7f8c141067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:14:25.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.417+0000 7f8c18b5f640 1 --2- 172.21.15.16:0/2531583425 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c14102790 0x7f8c141067e0 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7f8bfc009900 tx=0x7f8bfc02a410 comp rx=0 tx=0).stop 2024-09-12T02:14:25.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.417+0000 7f8c18b5f640 1 -- 172.21.15.16:0/2531583425 shutdown_connections 2024-09-12T02:14:25.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.417+0000 7f8c18b5f640 1 --2- 172.21.15.16:0/2531583425 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c14106f10 0x7f8c1410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:25.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.417+0000 7f8c18b5f640 1 --2- 172.21.15.16:0/2531583425 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c14102790 0x7f8c141067e0 unknown :-1 s=CLOSED pgs=1757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:25.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.417+0000 7f8c18b5f640 1 --2- 172.21.15.16:0/2531583425 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8c14101de0 0x7f8c141021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:25.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.417+0000 7f8c18b5f640 1 -- 172.21.15.16:0/2531583425 >> 172.21.15.16:0/2531583425 conn(0x7f8c140fd8c0 msgr2=0x7f8c140ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:14:25.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.417+0000 7f8c18b5f640 1 -- 172.21.15.16:0/2531583425 shutdown_connections 2024-09-12T02:14:25.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.417+0000 7f8c18b5f640 1 -- 172.21.15.16:0/2531583425 wait complete. 2024-09-12T02:14:25.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.418+0000 7f8c18b5f640 1 Processor -- start 2024-09-12T02:14:25.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.418+0000 7f8c18b5f640 1 -- start start 2024-09-12T02:14:25.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.418+0000 7f8c18b5f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c14101de0 0x7f8c14198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:25.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.418+0000 7f8c18b5f640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8c14102790 0x7f8c14198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:25.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.418+0000 7f8c18b5f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c14106f10 0x7f8c1419cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:25.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.418+0000 7f8c18b5f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8c14111f90 con 0x7f8c14101de0 2024-09-12T02:14:25.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.418+0000 7f8c18b5f640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8c14111e10 con 0x7f8c14106f10 2024-09-12T02:14:25.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.418+0000 7f8c18b5f640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8c14112110 con 0x7f8c14102790 2024-09-12T02:14:25.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.418+0000 7f8c1259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c14101de0 0x7f8c14198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:25.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.418+0000 7f8c1259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c14101de0 0x7f8c14198510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55458/0 (socket says 172.21.15.16:55458) 2024-09-12T02:14:25.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.418+0000 7f8c1259c640 1 -- 172.21.15.16:0/2146453064 learned_addr learned my addr 172.21.15.16:0/2146453064 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:14:25.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.418+0000 7f8c11d9b640 1 --2- 172.21.15.16:0/2146453064 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8c14102790 0x7f8c14198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:25.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.419+0000 7f8c12d9d640 1 --2- 172.21.15.16:0/2146453064 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c14106f10 0x7f8c1419cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:25.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.419+0000 7f8c1259c640 1 -- 172.21.15.16:0/2146453064 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8c14102790 msgr2=0x7f8c14198a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:14:25.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.419+0000 7f8c1259c640 1 --2- 172.21.15.16:0/2146453064 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8c14102790 0x7f8c14198a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:25.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.419+0000 7f8c1259c640 1 -- 172.21.15.16:0/2146453064 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c14106f10 msgr2=0x7f8c1419cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:14:25.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.419+0000 7f8c1259c640 1 --2- 172.21.15.16:0/2146453064 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c14106f10 0x7f8c1419cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:25.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.419+0000 7f8c1259c640 1 -- 172.21.15.16:0/2146453064 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c1419d4c0 con 0x7f8c14101de0 2024-09-12T02:14:25.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.419+0000 7f8c11d9b640 1 --2- 172.21.15.16:0/2146453064 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8c14102790 0x7f8c14198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:14:25.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.419+0000 7f8c1259c640 1 --2- 172.21.15.16:0/2146453064 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c14101de0 0x7f8c14198510 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7f8c0400ba50 tx=0x7f8c0400bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:14:25.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.419+0000 7f8c0b7fe640 1 -- 172.21.15.16:0/2146453064 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c04019070 con 0x7f8c14101de0 2024-09-12T02:14:25.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.419+0000 7f8c18b5f640 1 -- 172.21.15.16:0/2146453064 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c1419d7b0 con 0x7f8c14101de0 2024-09-12T02:14:25.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.419+0000 7f8c0b7fe640 1 -- 172.21.15.16:0/2146453064 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c0400cbe0 con 0x7f8c14101de0 2024-09-12T02:14:25.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.419+0000 7f8c0b7fe640 1 -- 172.21.15.16:0/2146453064 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c040076e0 con 0x7f8c14101de0 2024-09-12T02:14:25.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.419+0000 7f8c18b5f640 1 -- 172.21.15.16:0/2146453064 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c141a5090 con 0x7f8c14101de0 2024-09-12T02:14:25.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.421+0000 7f8c0b7fe640 1 -- 172.21.15.16:0/2146453064 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f8c0401c070 con 0x7f8c14101de0 2024-09-12T02:14:25.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.422+0000 7f8c1af38740 1 Processor -- start 2024-09-12T02:14:25.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.422+0000 7f8c1af38740 1 -- start start 2024-09-12T02:14:25.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.422+0000 7f8c1af38740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b161494a0 0x555b16149880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:25.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.422+0000 7f8c1af38740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b16149dc0 0x555b16154140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:25.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.422+0000 7f8c1af38740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555b16154680 0x555b16156a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:25.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.422+0000 7f8c1af38740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x555b160af2e0 con 0x555b161494a0 2024-09-12T02:14:25.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.422+0000 7f8c1af38740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x555b16056ce0 con 0x555b16149dc0 2024-09-12T02:14:25.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.422+0000 7f8c1af38740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x555b160ab2a0 con 0x555b16154680 2024-09-12T02:14:25.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.422+0000 7f8c12d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b161494a0 0x555b16149880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:25.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.422+0000 7f8c12d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b161494a0 0x555b16149880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55460/0 (socket says 172.21.15.16:55460) 2024-09-12T02:14:25.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.422+0000 7f8c12d9d640 1 -- 172.21.15.16:0/2845371687 learned_addr learned my addr 172.21.15.16:0/2845371687 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:14:25.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.423+0000 7f8c0b7fe640 1 --2- 172.21.15.16:0/2146453064 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f8c000a2b80 0x7f8c000a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:25.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.423+0000 7f8c1259c640 1 --2- 172.21.15.16:0/2845371687 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b16149dc0 0x555b16154140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:25.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.423+0000 7f8c11d9b640 1 --2- 172.21.15.16:0/2845371687 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555b16154680 0x555b16156a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:25.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.423+0000 7f8c12d9d640 1 -- 172.21.15.16:0/2845371687 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555b16154680 msgr2=0x555b16156a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:14:25.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.423+0000 7f8c12d9d640 1 --2- 172.21.15.16:0/2845371687 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555b16154680 0x555b16156a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:25.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.423+0000 7f8c12d9d640 1 -- 172.21.15.16:0/2845371687 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b16149dc0 msgr2=0x555b16154140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:14:25.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.423+0000 7f8c12d9d640 1 --2- 172.21.15.16:0/2845371687 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b16149dc0 0x555b16154140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:25.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.423+0000 7f8c12d9d640 1 -- 172.21.15.16:0/2845371687 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b1603b440 con 0x555b161494a0 2024-09-12T02:14:25.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.423+0000 7f8c0b7fe640 1 -- 172.21.15.16:0/2146453064 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f8c040ce710 con 0x7f8c14101de0 2024-09-12T02:14:25.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.423+0000 7f8c12d9d640 1 --2- 172.21.15.16:0/2146453064 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f8c000a2b80 0x7f8c000a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:25.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.423+0000 7f8c12d9d640 1 --2- 172.21.15.16:0/2845371687 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b161494a0 0x555b16149880 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c00bb90 tx=0x7f8c0c00d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=592d6250006ebd32 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:14:25.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.423+0000 7f8c08ff9640 1 -- 172.21.15.16:0/2845371687 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c0c011de0 con 0x555b161494a0 2024-09-12T02:14:25.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.423+0000 7f8c08ff9640 1 -- 172.21.15.16:0/2845371687 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c0c0198f0 con 0x555b161494a0 2024-09-12T02:14:25.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.423+0000 7f8c08ff9640 1 -- 172.21.15.16:0/2845371687 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c0c018780 con 0x555b161494a0 2024-09-12T02:14:25.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.424+0000 7f8c12d9d640 1 --2- 172.21.15.16:0/2146453064 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f8c000a2b80 0x7f8c000a5040 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c01f000 tx=0x7f8c0c020000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:14:25.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.424+0000 7f8c1af38740 1 -- 172.21.15.16:0/2845371687 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b161494a0 msgr2=0x555b16149880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:14:25.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.424+0000 7f8c1af38740 1 --2- 172.21.15.16:0/2845371687 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b161494a0 0x555b16149880 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c00bb90 tx=0x7f8c0c00d710 comp rx=0 tx=0).stop 2024-09-12T02:14:25.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.424+0000 7f8c1af38740 1 -- 172.21.15.16:0/2845371687 shutdown_connections 2024-09-12T02:14:25.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.424+0000 7f8c1af38740 1 --2- 172.21.15.16:0/2845371687 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555b16154680 0x555b16156a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:25.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.424+0000 7f8c1af38740 1 --2- 172.21.15.16:0/2845371687 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b16149dc0 0x555b16154140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:25.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.424+0000 7f8c1af38740 1 --2- 172.21.15.16:0/2845371687 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b161494a0 0x555b16149880 unknown :-1 s=CLOSED pgs=1759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:25.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.424+0000 7f8c1af38740 1 -- 172.21.15.16:0/2845371687 >> 172.21.15.16:0/2845371687 conn(0x555b16145a30 msgr2=0x555b16145e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:14:25.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.424+0000 7f8c1af38740 1 -- 172.21.15.16:0/2845371687 shutdown_connections 2024-09-12T02:14:25.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.424+0000 7f8c1af38740 1 -- 172.21.15.16:0/2845371687 wait complete. 2024-09-12T02:14:25.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.425+0000 7f8c1af38740 1 -- start start 2024-09-12T02:14:25.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.425+0000 7f8c1af38740 1 Processor -- start 2024-09-12T02:14:25.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.425+0000 7f8c1af38740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555b161494a0 0x555b16153fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:25.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.425+0000 7f8c1af38740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b16149dc0 0x555b16151260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:25.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.425+0000 7f8c1af38740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b16154680 0x555b161517a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:25.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.425+0000 7f8c1af38740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x555b16157250 con 0x555b16154680 2024-09-12T02:14:25.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.425+0000 7f8c1af38740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x555b1606a050 con 0x555b16149dc0 2024-09-12T02:14:25.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.425+0000 7f8c1af38740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x555b161573d0 con 0x555b161494a0 2024-09-12T02:14:25.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.425+0000 7f8c1af38740 10 client.0 _ll_register_callbacks cb 0x555b160aa160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:14:25.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.425+0000 7f8c12d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b16154680 0x555b161517a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:25.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.425+0000 7f8c12d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b16154680 0x555b161517a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55464/0 (socket says 172.21.15.16:55464) 2024-09-12T02:14:25.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.425+0000 7f8c12d9d640 1 -- 172.21.15.16:0/3788605729 learned_addr learned my addr 172.21.15.16:0/3788605729 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:14:25.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.425+0000 7f8c1259c640 1 --2- 172.21.15.16:0/3788605729 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555b161494a0 0x555b16153fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:25.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.425+0000 7f8c11d9b640 1 --2- 172.21.15.16:0/3788605729 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b16149dc0 0x555b16151260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:25.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.425+0000 7f8c12d9d640 1 -- 172.21.15.16:0/3788605729 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555b161494a0 msgr2=0x555b16153fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:14:25.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.425+0000 7f8c12d9d640 1 --2- 172.21.15.16:0/3788605729 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555b161494a0 0x555b16153fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:25.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.425+0000 7f8c12d9d640 1 -- 172.21.15.16:0/3788605729 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b16149dc0 msgr2=0x555b16151260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:14:25.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.425+0000 7f8c12d9d640 1 --2- 172.21.15.16:0/3788605729 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b16149dc0 0x555b16151260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:25.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.425+0000 7f8c12d9d640 1 -- 172.21.15.16:0/3788605729 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b15e203c0 con 0x555b16154680 2024-09-12T02:14:25.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.426+0000 7f8c12d9d640 1 --2- 172.21.15.16:0/3788605729 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b16154680 0x555b161517a0 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c0183e0 tx=0x7f8c0c018750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2aa1581de2bb81b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:14:25.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.426+0000 7f8bf9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:14:25.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.426+0000 7f8bf9ffb640 1 -- 172.21.15.16:0/3788605729 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c0c032060 con 0x555b16154680 2024-09-12T02:14:25.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.426+0000 7f8bf9ffb640 1 -- 172.21.15.16:0/3788605729 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f8c0c019620 con 0x555b16154680 2024-09-12T02:14:25.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.426+0000 7f8c1af38740 1 -- 172.21.15.16:0/3788605729 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c0c0198f0 con 0x555b16154680 2024-09-12T02:14:25.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.426+0000 7f8bf9ffb640 1 -- 172.21.15.16:0/3788605729 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f8c0c00b690 con 0x555b16154680 2024-09-12T02:14:25.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.426+0000 7f8c1af38740 10 client.19674 fetch_fsmap learned FSMap version 25 2024-09-12T02:14:25.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.426+0000 7f8c1af38740 1 -- 172.21.15.16:0/3788605729 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555b16167010 con 0x555b16154680 2024-09-12T02:14:25.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.426+0000 7f8bf9ffb640 1 -- 172.21.15.16:0/3788605729 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c0c0198f0 con 0x555b16154680 2024-09-12T02:14:25.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.426+0000 7f8bf9ffb640 1 -- 172.21.15.16:0/3788605729 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f8c0c037070 con 0x555b16154680 2024-09-12T02:14:25.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.426+0000 7f8c1af38740 10 client.19674 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:14:25.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.426+0000 7f8c1af38740 10 client.19674 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:14:25.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.427+0000 7f8c1af38740 20 client.19674 populate_metadata read hostname 'smithi016' 2024-09-12T02:14:25.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.427+0000 7f8c1af38740 1 --2- 172.21.15.16:0/3788605729 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x555b16168410 0x555b1616a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:25.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.427+0000 7f8c1af38740 4 client.19674 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:14:25.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.427+0000 7f8c1af38740 1 -- 172.21.15.16:0/3788605729 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555b160bc850 con 0x555b16168410 2024-09-12T02:14:25.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.427+0000 7f8c1259c640 1 --2- 172.21.15.16:0/3788605729 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x555b16168410 0x555b1616a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:25.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.427+0000 7f8c1259c640 1 --2- 172.21.15.16:0/3788605729 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x555b16168410 0x555b1616a800 crc :-1 s=READY pgs=380 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eda88e8cf3af21ac server_cookie=602b12e9d724ac2 in_seq=0 out_seq=0 2024-09-12T02:14:25.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.427+0000 7f8bf9ffb640 10 client.19674 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:14:25.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.428+0000 7f8bf9ffb640 1 -- 172.21.15.16:0/3788605729 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555b160bc850 con 0x555b16168410 2024-09-12T02:14:25.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.428+0000 7f8bf9ffb640 10 client.19674 handle_command_reply: tid=0 2024-09-12T02:14:25.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.428+0000 7f8c1af38740 1 client.19674 shutdown 2024-09-12T02:14:25.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.428+0000 7f8c1af38740 1 -- 172.21.15.16:0/3788605729 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b16154680 msgr2=0x555b161517a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:14:25.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.428+0000 7f8c1af38740 1 --2- 172.21.15.16:0/3788605729 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b16154680 0x555b161517a0 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c0183e0 tx=0x7f8c0c018750 comp rx=0 tx=0).stop 2024-09-12T02:14:25.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.428+0000 7f8c1af38740 1 -- 172.21.15.16:0/3788605729 shutdown_connections 2024-09-12T02:14:25.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.428+0000 7f8c1af38740 1 --2- 172.21.15.16:0/3788605729 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x555b16168410 0x555b1616a800 crc :-1 s=READY pgs=380 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:25.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.428+0000 7f8c1af38740 1 --2- 172.21.15.16:0/3788605729 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b16154680 0x555b161517a0 unknown :-1 s=CLOSED pgs=1760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:25.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.428+0000 7f8c1af38740 1 --2- 172.21.15.16:0/3788605729 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b16149dc0 0x555b16151260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:25.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.428+0000 7f8c1af38740 1 --2- 172.21.15.16:0/3788605729 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555b161494a0 0x555b16153fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:25.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.428+0000 7f8bf9ffb640 0 client.19674 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:14:25.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.428+0000 7f8c1af38740 1 -- 172.21.15.16:0/3788605729 >> 172.21.15.16:0/3788605729 conn(0x555b16145a30 msgr2=0x555b16144b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:14:25.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.428+0000 7f8c1af38740 1 -- 172.21.15.16:0/3788605729 shutdown_connections 2024-09-12T02:14:25.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.428+0000 7f8c1af38740 1 -- 172.21.15.16:0/3788605729 wait complete. 2024-09-12T02:14:25.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.428+0000 7f8c1af38740 20 client.19674 trim_cache size 0 max 16384 2024-09-12T02:14:25.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.446+0000 7f8c1af38740 1 Processor -- start 2024-09-12T02:14:25.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.446+0000 7f8c1af38740 1 -- start start 2024-09-12T02:14:25.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.446+0000 7f8c1af38740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b161494a0 0x555b16164940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:25.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.446+0000 7f8c1af38740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555b16149dc0 0x555b1616fe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:25.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.446+0000 7f8c1af38740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b16154680 0x555b16170340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:25.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.446+0000 7f8c1af38740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8c141adb10 con 0x555b16154680 2024-09-12T02:14:25.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.446+0000 7f8c1af38740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8c141ab3e0 con 0x555b161494a0 2024-09-12T02:14:25.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.446+0000 7f8c1af38740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8c141a9230 con 0x555b16149dc0 2024-09-12T02:14:25.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.446+0000 7f8c12d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b16154680 0x555b16170340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:25.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.446+0000 7f8c12d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b16154680 0x555b16170340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55474/0 (socket says 172.21.15.16:55474) 2024-09-12T02:14:25.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.446+0000 7f8c12d9d640 1 -- 172.21.15.16:0/422869405 learned_addr learned my addr 172.21.15.16:0/422869405 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:14:25.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.447+0000 7f8c1259c640 1 --2- 172.21.15.16:0/422869405 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b161494a0 0x555b16164940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:25.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.447+0000 7f8c11d9b640 1 --2- 172.21.15.16:0/422869405 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555b16149dc0 0x555b1616fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:25.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.447+0000 7f8c12d9d640 1 -- 172.21.15.16:0/422869405 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555b16149dc0 msgr2=0x555b1616fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:14:25.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.447+0000 7f8c12d9d640 1 --2- 172.21.15.16:0/422869405 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555b16149dc0 0x555b1616fe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:25.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.447+0000 7f8c12d9d640 1 -- 172.21.15.16:0/422869405 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b161494a0 msgr2=0x555b16164940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:14:25.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.447+0000 7f8c12d9d640 1 --2- 172.21.15.16:0/422869405 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b161494a0 0x555b16164940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:25.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.447+0000 7f8c12d9d640 1 -- 172.21.15.16:0/422869405 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b161492d0 con 0x555b16154680 2024-09-12T02:14:25.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.447+0000 7f8c11d9b640 1 --2- 172.21.15.16:0/422869405 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555b16149dc0 0x555b1616fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:14:25.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.447+0000 7f8c1259c640 1 --2- 172.21.15.16:0/422869405 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b161494a0 0x555b16164940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:14:25.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.447+0000 7f8c12d9d640 1 --2- 172.21.15.16:0/422869405 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b16154680 0x555b16170340 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c005f80 tx=0x7f8c0c00b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f126d1e512a4a4b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:14:25.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.447+0000 7f8bfbfff640 1 -- 172.21.15.16:0/422869405 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c0c046070 con 0x555b16154680 2024-09-12T02:14:25.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.447+0000 7f8bfbfff640 1 -- 172.21.15.16:0/422869405 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c0c00b690 con 0x555b16154680 2024-09-12T02:14:25.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.447+0000 7f8bfbfff640 1 -- 172.21.15.16:0/422869405 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c0c03d070 con 0x555b16154680 2024-09-12T02:14:25.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.448+0000 7f8c1af38740 1 -- 172.21.15.16:0/422869405 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b16154680 msgr2=0x555b16170340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:14:25.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.448+0000 7f8c1af38740 1 --2- 172.21.15.16:0/422869405 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b16154680 0x555b16170340 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c005f80 tx=0x7f8c0c00b8f0 comp rx=0 tx=0).stop 2024-09-12T02:14:25.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.448+0000 7f8c1af38740 1 -- 172.21.15.16:0/422869405 shutdown_connections 2024-09-12T02:14:25.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.448+0000 7f8c1af38740 1 --2- 172.21.15.16:0/422869405 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b16154680 0x555b16170340 unknown :-1 s=CLOSED pgs=1761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:25.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.448+0000 7f8c1af38740 1 --2- 172.21.15.16:0/422869405 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555b16149dc0 0x555b1616fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:25.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.448+0000 7f8c1af38740 1 --2- 172.21.15.16:0/422869405 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b161494a0 0x555b16164940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:25.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.448+0000 7f8c1af38740 1 -- 172.21.15.16:0/422869405 >> 172.21.15.16:0/422869405 conn(0x555b16145a30 msgr2=0x555b16150350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:14:25.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.448+0000 7f8c1af38740 1 -- 172.21.15.16:0/422869405 shutdown_connections 2024-09-12T02:14:25.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.448+0000 7f8c1af38740 1 -- 172.21.15.16:0/422869405 wait complete. 2024-09-12T02:14:25.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.448+0000 7f8c1af38740 1 -- start start 2024-09-12T02:14:25.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.448+0000 7f8c1af38740 1 Processor -- start 2024-09-12T02:14:25.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.449+0000 7f8c1af38740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b161494a0 0x555b16160ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:25.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.449+0000 7f8c1af38740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b16149dc0 0x555b1615e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:25.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.449+0000 7f8c1af38740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555b16154680 0x555b1615e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:25.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.449+0000 7f8c1af38740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8c141a98a0 con 0x555b16149dc0 2024-09-12T02:14:25.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.449+0000 7f8c1af38740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8c141aad10 con 0x555b161494a0 2024-09-12T02:14:25.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.449+0000 7f8c1af38740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8c141a9f70 con 0x555b16154680 2024-09-12T02:14:25.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.449+0000 7f8c1af38740 10 client.0 _ll_register_callbacks cb 0x7f8c0c00b9c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:14:25.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.449+0000 7f8c11d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b16149dc0 0x555b1615e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:25.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.449+0000 7f8c11d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b16149dc0 0x555b1615e210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55488/0 (socket says 172.21.15.16:55488) 2024-09-12T02:14:25.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.449+0000 7f8c11d9b640 1 -- 172.21.15.16:0/3125434329 learned_addr learned my addr 172.21.15.16:0/3125434329 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:14:25.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.449+0000 7f8c1259c640 1 --2- 172.21.15.16:0/3125434329 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b161494a0 0x555b16160ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:25.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.449+0000 7f8c12d9d640 1 --2- 172.21.15.16:0/3125434329 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555b16154680 0x555b1615e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:25.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.449+0000 7f8c11d9b640 1 -- 172.21.15.16:0/3125434329 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555b16154680 msgr2=0x555b1615e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:14:25.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.449+0000 7f8c11d9b640 1 --2- 172.21.15.16:0/3125434329 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555b16154680 0x555b1615e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:25.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.449+0000 7f8c11d9b640 1 -- 172.21.15.16:0/3125434329 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b161494a0 msgr2=0x555b16160ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:14:25.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.449+0000 7f8c11d9b640 1 --2- 172.21.15.16:0/3125434329 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b161494a0 0x555b16160ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:25.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.449+0000 7f8c11d9b640 1 -- 172.21.15.16:0/3125434329 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b1615ec90 con 0x555b16149dc0 2024-09-12T02:14:25.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.449+0000 7f8c11d9b640 1 --2- 172.21.15.16:0/3125434329 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b16149dc0 0x555b1615e210 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7f8bfc009900 tx=0x7f8bfc031af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32403529e1c492fc server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:14:25.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.449+0000 7f8bf9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:14:25.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.449+0000 7f8bf9ffb640 1 -- 172.21.15.16:0/3125434329 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bfc035040 con 0x555b16149dc0 2024-09-12T02:14:25.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.449+0000 7f8bf9ffb640 1 -- 172.21.15.16:0/3125434329 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f8bfc004400 con 0x555b16149dc0 2024-09-12T02:14:25.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.449+0000 7f8c1af38740 1 -- 172.21.15.16:0/3125434329 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c0c00b690 con 0x555b16149dc0 2024-09-12T02:14:25.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.450+0000 7f8bf9ffb640 1 -- 172.21.15.16:0/3125434329 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f8bfc005030 con 0x555b16149dc0 2024-09-12T02:14:25.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.450+0000 7f8c1af38740 10 client.19680 fetch_fsmap learned FSMap version 25 2024-09-12T02:14:25.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.450+0000 7f8c1af38740 1 -- 172.21.15.16:0/3125434329 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555b161d2a60 con 0x555b16149dc0 2024-09-12T02:14:25.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.450+0000 7f8bf9ffb640 1 -- 172.21.15.16:0/3125434329 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c0c00b690 con 0x555b16149dc0 2024-09-12T02:14:25.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.450+0000 7f8bf9ffb640 1 -- 172.21.15.16:0/3125434329 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f8bfc0035f0 con 0x555b16149dc0 2024-09-12T02:14:25.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.450+0000 7f8c1af38740 10 client.19680 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:14:25.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.450+0000 7f8c1af38740 10 client.19680 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:14:25.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.450+0000 7f8c1af38740 20 client.19680 populate_metadata read hostname 'smithi016' 2024-09-12T02:14:25.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.450+0000 7f8c1af38740 1 --2- 172.21.15.16:0/3125434329 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x555b161d3260 0x555b161d5680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:25.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.450+0000 7f8c1af38740 4 client.19680 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T02:14:25.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.450+0000 7f8c1af38740 1 -- 172.21.15.16:0/3125434329 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555b160b6d60 con 0x555b161d3260 2024-09-12T02:14:25.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.450+0000 7f8c1259c640 1 --2- 172.21.15.16:0/3125434329 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x555b161d3260 0x555b161d5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:25.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.451+0000 7f8c1259c640 1 --2- 172.21.15.16:0/3125434329 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x555b161d3260 0x555b161d5680 crc :-1 s=READY pgs=381 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81f83b0297f5a715 server_cookie=747769adc59a5602 in_seq=0 out_seq=0 2024-09-12T02:14:25.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.451+0000 7f8bf9ffb640 10 client.19680 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:14:25.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.451+0000 7f8bf9ffb640 1 -- 172.21.15.16:0/3125434329 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x555b160b6d60 con 0x555b161d3260 2024-09-12T02:14:25.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.451+0000 7f8bf9ffb640 10 client.19680 handle_command_reply: tid=0 2024-09-12T02:14:25.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.451+0000 7f8c1af38740 1 client.19680 shutdown 2024-09-12T02:14:25.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.451+0000 7f8c1af38740 1 -- 172.21.15.16:0/3125434329 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b16149dc0 msgr2=0x555b1615e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:14:25.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.451+0000 7f8c1af38740 1 --2- 172.21.15.16:0/3125434329 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b16149dc0 0x555b1615e210 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7f8bfc009900 tx=0x7f8bfc031af0 comp rx=0 tx=0).stop 2024-09-12T02:14:25.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.451+0000 7f8c1af38740 1 -- 172.21.15.16:0/3125434329 shutdown_connections 2024-09-12T02:14:25.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.451+0000 7f8c1af38740 1 --2- 172.21.15.16:0/3125434329 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x555b161d3260 0x555b161d5680 crc :-1 s=READY pgs=381 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:25.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.451+0000 7f8c1af38740 1 --2- 172.21.15.16:0/3125434329 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555b16154680 0x555b1615e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:25.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.451+0000 7f8bf9ffb640 0 client.19680 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:14:25.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.451+0000 7f8c1af38740 1 --2- 172.21.15.16:0/3125434329 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b16149dc0 0x555b1615e210 unknown :-1 s=CLOSED pgs=1762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:25.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.451+0000 7f8c1af38740 1 --2- 172.21.15.16:0/3125434329 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b161494a0 0x555b16160ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:25.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.451+0000 7f8c1af38740 1 -- 172.21.15.16:0/3125434329 >> 172.21.15.16:0/3125434329 conn(0x555b16145a30 msgr2=0x555b16144370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:14:25.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.451+0000 7f8c1af38740 1 -- 172.21.15.16:0/3125434329 shutdown_connections 2024-09-12T02:14:25.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.451+0000 7f8c1af38740 1 -- 172.21.15.16:0/3125434329 wait complete. 2024-09-12T02:14:25.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.452+0000 7f8c1af38740 20 client.19680 trim_cache size 0 max 16384 2024-09-12T02:14:25.461 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T02:14:25.461 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-12T02:14:25.461 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": { 2024-09-12T02:14:25.461 INFO:teuthology.orchestra.run.smithi016.stdout: "9a98f532-78d4-4c4f-92c7-ce4d4be9788a": { 2024-09-12T02:14:25.461 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-12T02:14:25.462 INFO:teuthology.orchestra.run.smithi016.stdout: "tag": "9a98f532-78d4-4c4f-92c7-ce4d4be9788a", 2024-09-12T02:14:25.462 INFO:teuthology.orchestra.run.smithi016.stdout: "options": "recursive,force" 2024-09-12T02:14:25.462 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:14:25.462 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:14:25.462 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T02:14:25.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.452+0000 7f8c18b5f640 1 -- 172.21.15.16:0/2146453064 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f8c000a2b80 msgr2=0x7f8c000a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:14:25.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.452+0000 7f8c18b5f640 1 --2- 172.21.15.16:0/2146453064 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f8c000a2b80 0x7f8c000a5040 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c01f000 tx=0x7f8c0c020000 comp rx=0 tx=0).stop 2024-09-12T02:14:25.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.452+0000 7f8c18b5f640 1 -- 172.21.15.16:0/2146453064 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c14101de0 msgr2=0x7f8c14198510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:14:25.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.452+0000 7f8c18b5f640 1 --2- 172.21.15.16:0/2146453064 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c14101de0 0x7f8c14198510 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7f8c0400ba50 tx=0x7f8c0400bf10 comp rx=0 tx=0).stop 2024-09-12T02:14:25.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.452+0000 7f8c18b5f640 1 -- 172.21.15.16:0/2146453064 shutdown_connections 2024-09-12T02:14:25.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.452+0000 7f8c18b5f640 1 --2- 172.21.15.16:0/2146453064 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f8c000a2b80 0x7f8c000a5040 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:25.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.452+0000 7f8c18b5f640 1 --2- 172.21.15.16:0/2146453064 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c14106f10 0x7f8c1419cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:25.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.452+0000 7f8c18b5f640 1 --2- 172.21.15.16:0/2146453064 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8c14102790 0x7f8c14198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:25.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.452+0000 7f8c18b5f640 1 --2- 172.21.15.16:0/2146453064 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c14101de0 0x7f8c14198510 unknown :-1 s=CLOSED pgs=1758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:25.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.452+0000 7f8c18b5f640 1 -- 172.21.15.16:0/2146453064 >> 172.21.15.16:0/2146453064 conn(0x7f8c140fd8c0 msgr2=0x7f8c140fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:14:25.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.452+0000 7f8c18b5f640 1 -- 172.21.15.16:0/2146453064 shutdown_connections 2024-09-12T02:14:25.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:25.452+0000 7f8c18b5f640 1 -- 172.21.15.16:0/2146453064 wait complete. 2024-09-12T02:14:25.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:14:25 smithi184 ceph-mon[35593]: scrub queued for path: / 2024-09-12T02:14:25.628 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:14:25 smithi184 ceph-mon[35593]: scrub summary: idle+waiting paths [/] 2024-09-12T02:14:25.628 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:14:25 smithi184 ceph-mon[35593]: scrub summary: active paths [/] 2024-09-12T02:14:25.628 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:14:25 smithi184 ceph-mon[35593]: scrub summary: idle+waiting paths [/] 2024-09-12T02:14:25.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:14:25 smithi033 ceph-mon[35332]: scrub queued for path: / 2024-09-12T02:14:25.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:14:25 smithi033 ceph-mon[35332]: scrub summary: idle+waiting paths [/] 2024-09-12T02:14:25.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:14:25 smithi033 ceph-mon[35332]: scrub summary: active paths [/] 2024-09-12T02:14:25.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:14:25 smithi033 ceph-mon[35332]: scrub summary: idle+waiting paths [/] 2024-09-12T02:14:26.171 INFO:tasks.cephfs.filesystem:scrub status for tag:9a98f532-78d4-4c4f-92c7-ce4d4be9788a - {'path': '/', 'tag': '9a98f532-78d4-4c4f-92c7-ce4d4be9788a', 'options': 'recursive,force'} 2024-09-12T02:14:26.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:14:26 smithi016 ceph-mon[30947]: pgmap v710: 97 pgs: 97 active+clean; 2.5 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-09-12T02:14:26.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:14:26 smithi184 ceph-mon[35593]: pgmap v710: 97 pgs: 97 active+clean; 2.5 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-09-12T02:14:26.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:14:26 smithi033 ceph-mon[35332]: pgmap v710: 97 pgs: 97 active+clean; 2.5 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-09-12T02:14:26.802 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.800+0000 7f4fb40c0640 1 -- 172.21.15.16:0/1706399534 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4fac1056d0 msgr2=0x7f4fac105b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:14:26.802 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.800+0000 7f4fb40c0640 1 --2- 172.21.15.16:0/1706399534 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4fac1056d0 0x7f4fac105b80 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7f4fa4009ca0 tx=0x7f4fa402a8c0 comp rx=0 tx=0).stop 2024-09-12T02:14:26.802 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.800+0000 7f4fb40c0640 1 -- 172.21.15.16:0/1706399534 shutdown_connections 2024-09-12T02:14:26.802 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.800+0000 7f4fb40c0640 1 --2- 172.21.15.16:0/1706399534 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4fac1056d0 0x7f4fac105b80 unknown :-1 s=CLOSED pgs=1763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:26.803 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.800+0000 7f4fb40c0640 1 --2- 172.21.15.16:0/1706399534 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4fac102d20 0x7f4fac105190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:26.803 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.800+0000 7f4fb40c0640 1 --2- 172.21.15.16:0/1706399534 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4fac1003d0 0x7f4fac1027e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:26.803 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.800+0000 7f4fb40c0640 1 -- 172.21.15.16:0/1706399534 >> 172.21.15.16:0/1706399534 conn(0x7f4fac0fc1e0 msgr2=0x7f4fac0fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:14:26.803 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.800+0000 7f4fb40c0640 1 -- 172.21.15.16:0/1706399534 shutdown_connections 2024-09-12T02:14:26.803 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.800+0000 7f4fb40c0640 1 -- 172.21.15.16:0/1706399534 wait complete. 2024-09-12T02:14:26.803 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.801+0000 7f4fb40c0640 1 Processor -- start 2024-09-12T02:14:26.803 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.801+0000 7f4fb40c0640 1 -- start start 2024-09-12T02:14:26.803 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.801+0000 7f4fb40c0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4fac1003d0 0x7f4fac19e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:26.803 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.801+0000 7f4fb40c0640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4fac102d20 0x7f4fac19e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:26.803 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.801+0000 7f4fb40c0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4fac1056d0 0x7f4fac1a5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:26.803 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.801+0000 7f4fb40c0640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4fac113e10 con 0x7f4fac1003d0 2024-09-12T02:14:26.803 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.801+0000 7f4fb40c0640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4fac113c90 con 0x7f4fac1056d0 2024-09-12T02:14:26.803 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.801+0000 7f4fb40c0640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f4fac113f90 con 0x7f4fac102d20 2024-09-12T02:14:26.804 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.801+0000 7f4fb1e5c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4fac1003d0 0x7f4fac19e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:26.804 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.801+0000 7f4fb1e5c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4fac1003d0 0x7f4fac19e350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55506/0 (socket says 172.21.15.16:55506) 2024-09-12T02:14:26.804 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.801+0000 7f4fb1e5c640 1 -- 172.21.15.16:0/2941764871 learned_addr learned my addr 172.21.15.16:0/2941764871 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:14:26.804 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.801+0000 7f4fb165b640 1 --2- 172.21.15.16:0/2941764871 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4fac102d20 0x7f4fac19e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:26.804 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.801+0000 7f4fb265d640 1 --2- 172.21.15.16:0/2941764871 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4fac1056d0 0x7f4fac1a5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:26.804 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.801+0000 7f4fb1e5c640 1 -- 172.21.15.16:0/2941764871 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4fac102d20 msgr2=0x7f4fac19e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:14:26.804 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.801+0000 7f4fb1e5c640 1 --2- 172.21.15.16:0/2941764871 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4fac102d20 0x7f4fac19e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:26.804 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.801+0000 7f4fb1e5c640 1 -- 172.21.15.16:0/2941764871 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4fac1056d0 msgr2=0x7f4fac1a5900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:14:26.804 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.801+0000 7f4fb1e5c640 1 --2- 172.21.15.16:0/2941764871 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4fac1056d0 0x7f4fac1a5900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:26.804 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.801+0000 7f4fb1e5c640 1 -- 172.21.15.16:0/2941764871 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fac1a5f70 con 0x7f4fac1003d0 2024-09-12T02:14:26.804 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.801+0000 7f4fb165b640 1 --2- 172.21.15.16:0/2941764871 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4fac102d20 0x7f4fac19e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:14:26.804 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.801+0000 7f4fb265d640 1 --2- 172.21.15.16:0/2941764871 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4fac1056d0 0x7f4fac1a5900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:14:26.804 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.802+0000 7f4fb1e5c640 1 --2- 172.21.15.16:0/2941764871 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4fac1003d0 0x7f4fac19e350 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7f4fa800e9f0 tx=0x7f4fa800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:14:26.804 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.802+0000 7f4fa2ffd640 1 -- 172.21.15.16:0/2941764871 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fa800cd80 con 0x7f4fac1003d0 2024-09-12T02:14:26.805 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.802+0000 7f4fa2ffd640 1 -- 172.21.15.16:0/2941764871 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4fa8010910 con 0x7f4fac1003d0 2024-09-12T02:14:26.805 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.802+0000 7f4fb40c0640 1 -- 172.21.15.16:0/2941764871 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fac1a6260 con 0x7f4fac1003d0 2024-09-12T02:14:26.805 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.802+0000 7f4fa2ffd640 1 -- 172.21.15.16:0/2941764871 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fa8018990 con 0x7f4fac1003d0 2024-09-12T02:14:26.805 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.802+0000 7f4fb40c0640 1 -- 172.21.15.16:0/2941764871 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fac1a6800 con 0x7f4fac1003d0 2024-09-12T02:14:26.807 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.805+0000 7f4fa2ffd640 1 -- 172.21.15.16:0/2941764871 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f4fa8010ac0 con 0x7f4fac1003d0 2024-09-12T02:14:26.807 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.805+0000 7f4fb6e54740 1 Processor -- start 2024-09-12T02:14:26.807 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.805+0000 7f4fb6e54740 1 -- start start 2024-09-12T02:14:26.807 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.805+0000 7f4fb6e54740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c53e8f9a0 0x564c53e8fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:26.807 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.805+0000 7f4fb6e54740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564c53e902e0 0x564c53e94da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:26.807 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.805+0000 7f4fb6e54740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564c53e95370 0x564c53e97830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:26.807 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.805+0000 7f4fb6e54740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x564c53df32f0 con 0x564c53e8f9a0 2024-09-12T02:14:26.807 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.805+0000 7f4fb6e54740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x564c53d9b270 con 0x564c53e902e0 2024-09-12T02:14:26.808 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.806+0000 7f4fb6e54740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x564c53def2b0 con 0x564c53e95370 2024-09-12T02:14:26.808 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.806+0000 7f4fb265d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c53e8f9a0 0x564c53e8fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:26.808 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.806+0000 7f4fb1e5c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564c53e902e0 0x564c53e94da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:26.808 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.806+0000 7f4fb265d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c53e8f9a0 0x564c53e8fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55516/0 (socket says 172.21.15.16:55516) 2024-09-12T02:14:26.808 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.806+0000 7f4fb265d640 1 -- 172.21.15.16:0/1155821124 learned_addr learned my addr 172.21.15.16:0/1155821124 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:14:26.808 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.806+0000 7f4fb165b640 1 --2- 172.21.15.16:0/1155821124 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564c53e95370 0x564c53e97830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:26.808 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.806+0000 7f4fa2ffd640 1 --2- 172.21.15.16:0/2941764871 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f4f980a2cb0 0x7f4f980a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:26.808 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.806+0000 7f4fb1e5c640 1 -- 172.21.15.16:0/1155821124 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564c53e95370 msgr2=0x564c53e97830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:14:26.808 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.806+0000 7f4fb1e5c640 1 --2- 172.21.15.16:0/1155821124 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564c53e95370 0x564c53e97830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:26.808 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.806+0000 7f4fb1e5c640 1 -- 172.21.15.16:0/1155821124 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c53e8f9a0 msgr2=0x564c53e8fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:14:26.808 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.806+0000 7f4fb1e5c640 1 --2- 172.21.15.16:0/1155821124 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c53e8f9a0 0x564c53e8fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:26.808 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.806+0000 7f4fb1e5c640 1 -- 172.21.15.16:0/1155821124 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c53d9a340 con 0x564c53e902e0 2024-09-12T02:14:26.809 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.806+0000 7f4fb265d640 1 --2- 172.21.15.16:0/1155821124 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c53e8f9a0 0x564c53e8fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:14:26.809 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.806+0000 7f4fb265d640 1 --2- 172.21.15.16:0/2941764871 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f4f980a2cb0 0x7f4f980a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:26.809 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.806+0000 7f4fa2ffd640 1 -- 172.21.15.16:0/2941764871 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f4fa80cf360 con 0x7f4fac1003d0 2024-09-12T02:14:26.809 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.806+0000 7f4fb1e5c640 1 --2- 172.21.15.16:0/1155821124 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564c53e902e0 0x564c53e94da0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f4fa80cced0 tx=0x7f4fa8024000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7bee884062ae2b83 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:14:26.809 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.807+0000 7f4fb265d640 1 --2- 172.21.15.16:0/2941764871 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f4f980a2cb0 0x7f4f980a5170 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f4fa402ade0 tx=0x7f4fa40095c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:14:26.809 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.807+0000 7f4f97fff640 1 -- 172.21.15.16:0/1155821124 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fa809b7a0 con 0x564c53e902e0 2024-09-12T02:14:26.809 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.807+0000 7f4f97fff640 1 -- 172.21.15.16:0/1155821124 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4fa809c360 con 0x564c53e902e0 2024-09-12T02:14:26.809 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.807+0000 7f4f97fff640 1 -- 172.21.15.16:0/1155821124 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fa809b5c0 con 0x564c53e902e0 2024-09-12T02:14:26.813 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.807+0000 7f4fb6e54740 1 -- 172.21.15.16:0/1155821124 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564c53e902e0 msgr2=0x564c53e94da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:14:26.814 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.807+0000 7f4fb6e54740 1 --2- 172.21.15.16:0/1155821124 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564c53e902e0 0x564c53e94da0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f4fa80cced0 tx=0x7f4fa8024000 comp rx=0 tx=0).stop 2024-09-12T02:14:26.814 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.807+0000 7f4fb6e54740 1 -- 172.21.15.16:0/1155821124 shutdown_connections 2024-09-12T02:14:26.814 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.807+0000 7f4fb6e54740 1 --2- 172.21.15.16:0/1155821124 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564c53e95370 0x564c53e97830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:26.814 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.807+0000 7f4fb6e54740 1 --2- 172.21.15.16:0/1155821124 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564c53e902e0 0x564c53e94da0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:26.814 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.807+0000 7f4fb6e54740 1 --2- 172.21.15.16:0/1155821124 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c53e8f9a0 0x564c53e8fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:26.814 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.807+0000 7f4fb6e54740 1 -- 172.21.15.16:0/1155821124 >> 172.21.15.16:0/1155821124 conn(0x564c53e8b010 msgr2=0x564c53e8b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:14:26.814 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.808+0000 7f4fb6e54740 1 -- 172.21.15.16:0/1155821124 shutdown_connections 2024-09-12T02:14:26.814 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.808+0000 7f4fb6e54740 1 -- 172.21.15.16:0/1155821124 wait complete. 2024-09-12T02:14:26.814 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.808+0000 7f4fb6e54740 1 -- start start 2024-09-12T02:14:26.814 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.808+0000 7f4fb6e54740 1 Processor -- start 2024-09-12T02:14:26.814 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.808+0000 7f4fb6e54740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564c53e8f9a0 0x564c53e99b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:26.814 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.808+0000 7f4fb6e54740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c53e902e0 0x564c53e9a070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:26.815 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.809+0000 7f4fb6e54740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564c53e95370 0x564c53ea10b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:26.815 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.809+0000 7f4fb6e54740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x564c53e982e0 con 0x564c53e902e0 2024-09-12T02:14:26.815 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.809+0000 7f4fb6e54740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x564c53d7c290 con 0x564c53e95370 2024-09-12T02:14:26.815 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.809+0000 7f4fb6e54740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x564c53e98460 con 0x564c53e8f9a0 2024-09-12T02:14:26.815 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.809+0000 7f4fb6e54740 10 client.0 _ll_register_callbacks cb 0x564c53dad140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:14:26.815 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.809+0000 7f4fb165b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c53e902e0 0x564c53e9a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:26.815 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.809+0000 7f4fb165b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c53e902e0 0x564c53e9a070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55526/0 (socket says 172.21.15.16:55526) 2024-09-12T02:14:26.815 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.809+0000 7f4fb165b640 1 -- 172.21.15.16:0/208492203 learned_addr learned my addr 172.21.15.16:0/208492203 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:14:26.815 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.809+0000 7f4fb265d640 1 --2- 172.21.15.16:0/208492203 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564c53e95370 0x564c53ea10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:26.815 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.809+0000 7f4fb1e5c640 1 --2- 172.21.15.16:0/208492203 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564c53e8f9a0 0x564c53e99b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:26.815 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.809+0000 7f4fb165b640 1 -- 172.21.15.16:0/208492203 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564c53e8f9a0 msgr2=0x564c53e99b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:14:26.815 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.809+0000 7f4fb165b640 1 --2- 172.21.15.16:0/208492203 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564c53e8f9a0 0x564c53e99b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:26.815 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.809+0000 7f4fb165b640 1 -- 172.21.15.16:0/208492203 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564c53e95370 msgr2=0x564c53ea10b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:14:26.815 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.809+0000 7f4fb165b640 1 --2- 172.21.15.16:0/208492203 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564c53e95370 0x564c53ea10b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:26.816 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.809+0000 7f4fb165b640 1 -- 172.21.15.16:0/208492203 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564c53b06210 con 0x564c53e902e0 2024-09-12T02:14:26.816 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.809+0000 7f4fb165b640 1 --2- 172.21.15.16:0/208492203 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c53e902e0 0x564c53e9a070 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7f4f9c009fa0 tx=0x7f4f9c011870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e867fdda651fe53 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:14:26.816 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.809+0000 7f4f957fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:14:26.816 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.809+0000 7f4f957fa640 1 -- 172.21.15.16:0/208492203 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f9c012920 con 0x564c53e902e0 2024-09-12T02:14:26.816 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.809+0000 7f4f957fa640 1 -- 172.21.15.16:0/208492203 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f4f9c012ac0 con 0x564c53e902e0 2024-09-12T02:14:26.816 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.809+0000 7f4fb6e54740 1 -- 172.21.15.16:0/208492203 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4fa809c360 con 0x564c53e902e0 2024-09-12T02:14:26.816 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.810+0000 7f4f957fa640 1 -- 172.21.15.16:0/208492203 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4f9c012cc0 con 0x564c53e902e0 2024-09-12T02:14:26.816 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.810+0000 7f4fb6e54740 10 client.19692 fetch_fsmap learned FSMap version 25 2024-09-12T02:14:26.816 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.810+0000 7f4fb6e54740 1 -- 172.21.15.16:0/208492203 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x564c53bd2620 con 0x564c53e902e0 2024-09-12T02:14:26.816 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.810+0000 7f4f957fa640 1 -- 172.21.15.16:0/208492203 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4fa809c360 con 0x564c53e902e0 2024-09-12T02:14:26.816 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.810+0000 7f4f957fa640 1 -- 172.21.15.16:0/208492203 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f4f9c00f040 con 0x564c53e902e0 2024-09-12T02:14:26.816 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.810+0000 7f4fb6e54740 10 client.19692 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:14:26.816 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.810+0000 7f4fb6e54740 10 client.19692 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:14:26.817 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.810+0000 7f4fb6e54740 20 client.19692 populate_metadata read hostname 'smithi016' 2024-09-12T02:14:26.817 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.810+0000 7f4fb6e54740 1 --2- 172.21.15.16:0/208492203 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x564c53ea5800 0x564c53ea7c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:26.817 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.810+0000 7f4fb6e54740 4 client.19692 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:14:26.817 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.810+0000 7f4fb6e54740 1 -- 172.21.15.16:0/208492203 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564c53e00a20 con 0x564c53ea5800 2024-09-12T02:14:26.817 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.810+0000 7f4fb1e5c640 1 --2- 172.21.15.16:0/208492203 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x564c53ea5800 0x564c53ea7c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:26.817 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.811+0000 7f4f957fa640 10 client.19692 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:14:26.817 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.811+0000 7f4fb1e5c640 1 --2- 172.21.15.16:0/208492203 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x564c53ea5800 0x564c53ea7c50 crc :-1 s=READY pgs=382 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6484159aa56ba2f server_cookie=89675bdc1be47a14 in_seq=0 out_seq=0 2024-09-12T02:14:26.817 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.811+0000 7f4f957fa640 1 -- 172.21.15.16:0/208492203 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564c53e00a20 con 0x564c53ea5800 2024-09-12T02:14:26.817 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.811+0000 7f4f957fa640 10 client.19692 handle_command_reply: tid=0 2024-09-12T02:14:26.817 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.812+0000 7f4fb6e54740 1 client.19692 shutdown 2024-09-12T02:14:26.817 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.812+0000 7f4fb6e54740 1 -- 172.21.15.16:0/208492203 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c53e902e0 msgr2=0x564c53e9a070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:14:26.817 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.812+0000 7f4fb6e54740 1 --2- 172.21.15.16:0/208492203 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c53e902e0 0x564c53e9a070 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7f4f9c009fa0 tx=0x7f4f9c011870 comp rx=0 tx=0).stop 2024-09-12T02:14:26.817 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.812+0000 7f4fb6e54740 1 -- 172.21.15.16:0/208492203 shutdown_connections 2024-09-12T02:14:26.818 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.812+0000 7f4fb6e54740 1 --2- 172.21.15.16:0/208492203 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x564c53ea5800 0x564c53ea7c50 crc :-1 s=READY pgs=382 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:26.818 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.812+0000 7f4fb6e54740 1 --2- 172.21.15.16:0/208492203 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564c53e95370 0x564c53ea10b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:26.818 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.812+0000 7f4fb6e54740 1 --2- 172.21.15.16:0/208492203 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c53e902e0 0x564c53e9a070 unknown :-1 s=CLOSED pgs=1765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:26.818 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.812+0000 7f4fb6e54740 1 --2- 172.21.15.16:0/208492203 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564c53e8f9a0 0x564c53e99b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:26.818 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.812+0000 7f4f957fa640 0 client.19692 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:14:26.818 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.812+0000 7f4fb6e54740 1 -- 172.21.15.16:0/208492203 >> 172.21.15.16:0/208492203 conn(0x564c53e8b010 msgr2=0x564c53e8c820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:14:26.818 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.812+0000 7f4fb6e54740 1 -- 172.21.15.16:0/208492203 shutdown_connections 2024-09-12T02:14:26.818 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.812+0000 7f4fb6e54740 1 -- 172.21.15.16:0/208492203 wait complete. 2024-09-12T02:14:26.818 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.812+0000 7f4fb6e54740 20 client.19692 trim_cache size 0 max 16384 2024-09-12T02:14:26.831 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.827+0000 7f4fb6e54740 1 Processor -- start 2024-09-12T02:14:26.832 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.827+0000 7f4fb6e54740 1 -- start start 2024-09-12T02:14:26.832 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.827+0000 7f4fb6e54740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564c53e8f9a0 0x564c53e8a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:26.832 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.827+0000 7f4fb6e54740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564c53e902e0 0x564c53ea1070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:26.832 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.827+0000 7f4fb6e54740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c53e95370 0x564c53e8c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:26.832 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.827+0000 7f4fb6e54740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4fac1b1d90 con 0x564c53e95370 2024-09-12T02:14:26.832 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.827+0000 7f4fb6e54740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4fac1af660 con 0x564c53e902e0 2024-09-12T02:14:26.832 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.827+0000 7f4fb6e54740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f4fac1ad4b0 con 0x564c53e8f9a0 2024-09-12T02:14:26.832 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.827+0000 7f4fb265d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c53e95370 0x564c53e8c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:26.832 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.827+0000 7f4fb265d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c53e95370 0x564c53e8c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55542/0 (socket says 172.21.15.16:55542) 2024-09-12T02:14:26.832 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.827+0000 7f4fb265d640 1 -- 172.21.15.16:0/1528485723 learned_addr learned my addr 172.21.15.16:0/1528485723 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:14:26.832 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.828+0000 7f4fb1e5c640 1 --2- 172.21.15.16:0/1528485723 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564c53e8f9a0 0x564c53e8a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:26.832 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.828+0000 7f4fb265d640 1 -- 172.21.15.16:0/1528485723 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564c53e8f9a0 msgr2=0x564c53e8a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:14:26.833 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.828+0000 7f4fb165b640 1 --2- 172.21.15.16:0/1528485723 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564c53e902e0 0x564c53ea1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:26.833 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.828+0000 7f4fb265d640 1 --2- 172.21.15.16:0/1528485723 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564c53e8f9a0 0x564c53e8a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:26.833 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.828+0000 7f4fb265d640 1 -- 172.21.15.16:0/1528485723 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564c53e902e0 msgr2=0x564c53ea1070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:14:26.833 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.828+0000 7f4fb265d640 1 --2- 172.21.15.16:0/1528485723 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564c53e902e0 0x564c53ea1070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:26.833 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.828+0000 7f4fb265d640 1 -- 172.21.15.16:0/1528485723 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c53e8ae30 con 0x564c53e95370 2024-09-12T02:14:26.833 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.828+0000 7f4fb1e5c640 1 --2- 172.21.15.16:0/1528485723 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564c53e8f9a0 0x564c53e8a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:14:26.833 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.828+0000 7f4fb265d640 1 --2- 172.21.15.16:0/1528485723 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c53e95370 0x564c53e8c400 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7f4fa4005cc0 tx=0x7f4fa4040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a59b70af699b184b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:14:26.833 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.828+0000 7f4f977fe640 1 -- 172.21.15.16:0/1528485723 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fa4003e40 con 0x564c53e95370 2024-09-12T02:14:26.833 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.828+0000 7f4f977fe640 1 -- 172.21.15.16:0/1528485723 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4fa4036040 con 0x564c53e95370 2024-09-12T02:14:26.833 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.828+0000 7f4f977fe640 1 -- 172.21.15.16:0/1528485723 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fa402fba0 con 0x564c53e95370 2024-09-12T02:14:26.833 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.829+0000 7f4fb6e54740 1 -- 172.21.15.16:0/1528485723 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c53e95370 msgr2=0x564c53e8c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:14:26.833 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.829+0000 7f4fb6e54740 1 --2- 172.21.15.16:0/1528485723 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c53e95370 0x564c53e8c400 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7f4fa4005cc0 tx=0x7f4fa4040000 comp rx=0 tx=0).stop 2024-09-12T02:14:26.834 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.829+0000 7f4fb6e54740 1 -- 172.21.15.16:0/1528485723 shutdown_connections 2024-09-12T02:14:26.834 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.829+0000 7f4fb6e54740 1 --2- 172.21.15.16:0/1528485723 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c53e95370 0x564c53e8c400 unknown :-1 s=CLOSED pgs=1766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:26.834 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.829+0000 7f4fb6e54740 1 --2- 172.21.15.16:0/1528485723 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564c53e902e0 0x564c53ea1070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:26.834 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.829+0000 7f4fb6e54740 1 --2- 172.21.15.16:0/1528485723 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564c53e8f9a0 0x564c53e8a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:26.834 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.829+0000 7f4fb6e54740 1 -- 172.21.15.16:0/1528485723 >> 172.21.15.16:0/1528485723 conn(0x564c53e8b010 msgr2=0x564c53e899a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:14:26.834 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.829+0000 7f4fb6e54740 1 -- 172.21.15.16:0/1528485723 shutdown_connections 2024-09-12T02:14:26.834 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.829+0000 7f4fb6e54740 1 -- 172.21.15.16:0/1528485723 wait complete. 2024-09-12T02:14:26.834 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.829+0000 7f4fb6e54740 1 -- start start 2024-09-12T02:14:26.834 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.829+0000 7f4fb6e54740 1 Processor -- start 2024-09-12T02:14:26.834 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.829+0000 7f4fb6e54740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564c53e8f9a0 0x564c53e9e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:26.834 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.829+0000 7f4fb6e54740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c53e902e0 0x564c53e9e870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:26.834 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.829+0000 7f4fb6e54740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564c53e95370 0x564c53e9f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:26.834 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.829+0000 7f4fb6e54740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4fac1adb20 con 0x564c53e902e0 2024-09-12T02:14:26.834 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.829+0000 7f4fb6e54740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4fac1aef90 con 0x564c53e95370 2024-09-12T02:14:26.835 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.829+0000 7f4fb6e54740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f4fac1ae1f0 con 0x564c53e8f9a0 2024-09-12T02:14:26.835 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.830+0000 7f4fb6e54740 10 client.0 _ll_register_callbacks cb 0x7f4f9c0097c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:14:26.835 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.830+0000 7f4fb165b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c53e902e0 0x564c53e9e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:26.835 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.830+0000 7f4fb165b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c53e902e0 0x564c53e9e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55544/0 (socket says 172.21.15.16:55544) 2024-09-12T02:14:26.835 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.830+0000 7f4fb165b640 1 -- 172.21.15.16:0/3622187880 learned_addr learned my addr 172.21.15.16:0/3622187880 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:14:26.835 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.830+0000 7f4fb1e5c640 1 --2- 172.21.15.16:0/3622187880 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564c53e8f9a0 0x564c53e9e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:26.835 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.830+0000 7f4fb265d640 1 --2- 172.21.15.16:0/3622187880 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564c53e95370 0x564c53e9f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:26.835 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.830+0000 7f4fb165b640 1 -- 172.21.15.16:0/3622187880 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564c53e8f9a0 msgr2=0x564c53e9e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:14:26.835 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.830+0000 7f4fb165b640 1 --2- 172.21.15.16:0/3622187880 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564c53e8f9a0 0x564c53e9e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:26.835 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.830+0000 7f4fb165b640 1 -- 172.21.15.16:0/3622187880 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564c53e95370 msgr2=0x564c53e9f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:14:26.835 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.830+0000 7f4fb165b640 1 --2- 172.21.15.16:0/3622187880 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564c53e95370 0x564c53e9f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:26.835 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.830+0000 7f4fb165b640 1 -- 172.21.15.16:0/3622187880 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564c53e9fe90 con 0x564c53e902e0 2024-09-12T02:14:26.835 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.830+0000 7f4fb165b640 1 --2- 172.21.15.16:0/3622187880 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c53e902e0 0x564c53e9e870 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7f4f9c006010 tx=0x7f4f9c005370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adf8d86320abf542 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:14:26.836 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.830+0000 7f4f957fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:14:26.836 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.830+0000 7f4f957fa640 1 -- 172.21.15.16:0/3622187880 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f9c033060 con 0x564c53e902e0 2024-09-12T02:14:26.836 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.830+0000 7f4f957fa640 1 -- 172.21.15.16:0/3622187880 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f4f9c0095b0 con 0x564c53e902e0 2024-09-12T02:14:26.836 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.830+0000 7f4fb6e54740 1 -- 172.21.15.16:0/3622187880 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4fa4036040 con 0x564c53e902e0 2024-09-12T02:14:26.836 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.830+0000 7f4f957fa640 1 -- 172.21.15.16:0/3622187880 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4f9c012e60 con 0x564c53e902e0 2024-09-12T02:14:26.836 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.830+0000 7f4fb6e54740 10 client.19698 fetch_fsmap learned FSMap version 25 2024-09-12T02:14:26.836 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.830+0000 7f4fb6e54740 1 -- 172.21.15.16:0/3622187880 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x564c53ea11e0 con 0x564c53e902e0 2024-09-12T02:14:26.836 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.831+0000 7f4f957fa640 1 -- 172.21.15.16:0/3622187880 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4fa4036040 con 0x564c53e902e0 2024-09-12T02:14:26.836 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.831+0000 7f4f957fa640 1 -- 172.21.15.16:0/3622187880 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f4f9c038070 con 0x564c53e902e0 2024-09-12T02:14:26.836 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.831+0000 7f4fb6e54740 10 client.19698 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:14:26.836 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.831+0000 7f4fb6e54740 10 client.19698 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:14:26.836 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.831+0000 7f4fb6e54740 20 client.19698 populate_metadata read hostname 'smithi016' 2024-09-12T02:14:26.837 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.831+0000 7f4fb6e54740 1 --2- 172.21.15.16:0/3622187880 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x564c53f10b30 0x564c53f12f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:26.837 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.831+0000 7f4fb6e54740 4 client.19698 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:14:26.837 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.831+0000 7f4fb6e54740 1 -- 172.21.15.16:0/3622187880 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564c53dfad70 con 0x564c53f10b30 2024-09-12T02:14:26.837 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.831+0000 7f4fb1e5c640 1 --2- 172.21.15.16:0/3622187880 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x564c53f10b30 0x564c53f12f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:26.837 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.831+0000 7f4fb1e5c640 1 --2- 172.21.15.16:0/3622187880 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x564c53f10b30 0x564c53f12f50 crc :-1 s=READY pgs=383 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5318532677961876 server_cookie=8eee3aa8b8e38c5c in_seq=0 out_seq=0 2024-09-12T02:14:26.837 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.831+0000 7f4f957fa640 10 client.19698 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:14:26.846 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.846+0000 7f4f957fa640 1 -- 172.21.15.16:0/3622187880 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564c53dfad70 con 0x564c53f10b30 2024-09-12T02:14:26.846 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.846+0000 7f4f957fa640 10 client.19698 handle_command_reply: tid=0 2024-09-12T02:14:26.846 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.846+0000 7f4fb6e54740 1 client.19698 shutdown 2024-09-12T02:14:26.846 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.846+0000 7f4fb6e54740 1 -- 172.21.15.16:0/3622187880 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c53e902e0 msgr2=0x564c53e9e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:14:26.846 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.846+0000 7f4fb6e54740 1 --2- 172.21.15.16:0/3622187880 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c53e902e0 0x564c53e9e870 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7f4f9c006010 tx=0x7f4f9c005370 comp rx=0 tx=0).stop 2024-09-12T02:14:26.846 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.846+0000 7f4fb6e54740 1 -- 172.21.15.16:0/3622187880 shutdown_connections 2024-09-12T02:14:26.846 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.846+0000 7f4fb6e54740 1 --2- 172.21.15.16:0/3622187880 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x564c53f10b30 0x564c53f12f50 crc :-1 s=READY pgs=383 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:26.846 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.846+0000 7f4fb6e54740 1 --2- 172.21.15.16:0/3622187880 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564c53e95370 0x564c53e9f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:26.846 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.846+0000 7f4fb6e54740 1 --2- 172.21.15.16:0/3622187880 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c53e902e0 0x564c53e9e870 unknown :-1 s=CLOSED pgs=1767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:26.846 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.846+0000 7f4fb6e54740 1 --2- 172.21.15.16:0/3622187880 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564c53e8f9a0 0x564c53e9e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:26.846 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.846+0000 7f4f957fa640 0 client.19698 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:14:26.846 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.846+0000 7f4fb6e54740 1 -- 172.21.15.16:0/3622187880 >> 172.21.15.16:0/3622187880 conn(0x564c53e8b010 msgr2=0x564c53e8a1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:14:26.847 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.846+0000 7f4fb6e54740 1 -- 172.21.15.16:0/3622187880 shutdown_connections 2024-09-12T02:14:26.847 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.846+0000 7f4fb6e54740 1 -- 172.21.15.16:0/3622187880 wait complete. 2024-09-12T02:14:26.847 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.846+0000 7f4fb6e54740 20 client.19698 trim_cache size 0 max 16384 2024-09-12T02:14:26.847 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-09-12T02:14:26.847 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-09-12T02:14:26.847 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-09-12T02:14:26.847 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-09-12T02:14:26.848 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.846+0000 7f4fb40c0640 1 -- 172.21.15.16:0/2941764871 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f4f980a2cb0 msgr2=0x7f4f980a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:14:26.848 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.846+0000 7f4fb40c0640 1 --2- 172.21.15.16:0/2941764871 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f4f980a2cb0 0x7f4f980a5170 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f4fa402ade0 tx=0x7f4fa40095c0 comp rx=0 tx=0).stop 2024-09-12T02:14:26.848 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.846+0000 7f4fb40c0640 1 -- 172.21.15.16:0/2941764871 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4fac1003d0 msgr2=0x7f4fac19e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:14:26.848 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.846+0000 7f4fb40c0640 1 --2- 172.21.15.16:0/2941764871 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4fac1003d0 0x7f4fac19e350 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7f4fa800e9f0 tx=0x7f4fa800eeb0 comp rx=0 tx=0).stop 2024-09-12T02:14:26.848 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.846+0000 7f4fb40c0640 1 -- 172.21.15.16:0/2941764871 shutdown_connections 2024-09-12T02:14:26.848 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.846+0000 7f4fb40c0640 1 --2- 172.21.15.16:0/2941764871 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f4f980a2cb0 0x7f4f980a5170 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:26.848 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.846+0000 7f4fb40c0640 1 --2- 172.21.15.16:0/2941764871 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4fac1056d0 0x7f4fac1a5900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:26.849 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.846+0000 7f4fb40c0640 1 --2- 172.21.15.16:0/2941764871 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4fac102d20 0x7f4fac19e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:26.849 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.846+0000 7f4fb40c0640 1 --2- 172.21.15.16:0/2941764871 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4fac1003d0 0x7f4fac19e350 unknown :-1 s=CLOSED pgs=1764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:26.849 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.846+0000 7f4fb40c0640 1 -- 172.21.15.16:0/2941764871 >> 172.21.15.16:0/2941764871 conn(0x7f4fac0fc1e0 msgr2=0x7f4fac103660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:14:26.849 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.846+0000 7f4fb40c0640 1 -- 172.21.15.16:0/2941764871 shutdown_connections 2024-09-12T02:14:26.849 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:26.846+0000 7f4fb40c0640 1 -- 172.21.15.16:0/2941764871 wait complete. 2024-09-12T02:14:27.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:14:27 smithi184 ceph-mon[35593]: scrub summary: idle 2024-09-12T02:14:27.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:14:27 smithi033 ceph-mon[35332]: scrub summary: idle 2024-09-12T02:14:27.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:14:27 smithi016 ceph-mon[30947]: scrub summary: idle 2024-09-12T02:14:28.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:14:28 smithi184 ceph-mon[35593]: pgmap v711: 97 pgs: 97 active+clean; 2.5 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 16 op/s 2024-09-12T02:14:28.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:14:28 smithi033 ceph-mon[35332]: pgmap v711: 97 pgs: 97 active+clean; 2.5 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 16 op/s 2024-09-12T02:14:28.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:14:28 smithi016 ceph-mon[30947]: pgmap v711: 97 pgs: 97 active+clean; 2.5 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 16 op/s 2024-09-12T02:14:30.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:14:30 smithi184 ceph-mon[35593]: pgmap v712: 97 pgs: 97 active+clean; 2.5 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-09-12T02:14:30.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:14:30 smithi033 ceph-mon[35332]: pgmap v712: 97 pgs: 97 active+clean; 2.5 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-09-12T02:14:30.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:14:30 smithi016 ceph-mon[30947]: pgmap v712: 97 pgs: 97 active+clean; 2.5 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-09-12T02:14:32.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:14:32 smithi184 ceph-mon[35593]: pgmap v713: 97 pgs: 97 active+clean; 2.5 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-12T02:14:32.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:14:32 smithi033 ceph-mon[35332]: pgmap v713: 97 pgs: 97 active+clean; 2.5 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-12T02:14:32.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:14:32 smithi016 ceph-mon[30947]: pgmap v713: 97 pgs: 97 active+clean; 2.5 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-12T02:14:34.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:14:34 smithi184 ceph-mon[35593]: pgmap v714: 97 pgs: 97 active+clean; 2.6 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-09-12T02:14:34.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:14:34 smithi033 ceph-mon[35332]: pgmap v714: 97 pgs: 97 active+clean; 2.6 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-09-12T02:14:34.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:14:34 smithi016 ceph-mon[30947]: pgmap v714: 97 pgs: 97 active+clean; 2.6 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-09-12T02:14:36.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:14:36 smithi184 ceph-mon[35593]: pgmap v715: 97 pgs: 97 active+clean; 2.6 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-09-12T02:14:36.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:14:36 smithi033 ceph-mon[35332]: pgmap v715: 97 pgs: 97 active+clean; 2.6 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-09-12T02:14:36.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:14:36 smithi016 ceph-mon[30947]: pgmap v715: 97 pgs: 97 active+clean; 2.6 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-09-12T02:14:38.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:14:38 smithi184 ceph-mon[35593]: pgmap v716: 97 pgs: 97 active+clean; 2.7 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 17 op/s 2024-09-12T02:14:38.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:14:38 smithi033 ceph-mon[35332]: pgmap v716: 97 pgs: 97 active+clean; 2.7 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 17 op/s 2024-09-12T02:14:38.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:14:38 smithi016 ceph-mon[30947]: pgmap v716: 97 pgs: 97 active+clean; 2.7 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 17 op/s 2024-09-12T02:14:39.947 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.945+0000 7f8a1528d640 1 -- 172.21.15.16:0/2476752184 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a10100d00 msgr2=0x7f8a10101180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:14:39.947 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.945+0000 7f8a1528d640 1 --2- 172.21.15.16:0/2476752184 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a10100d00 0x7f8a10101180 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7f8a00009900 tx=0x7f8a0002a410 comp rx=0 tx=0).stop 2024-09-12T02:14:39.947 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.945+0000 7f8a1528d640 1 -- 172.21.15.16:0/2476752184 shutdown_connections 2024-09-12T02:14:39.947 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.945+0000 7f8a1528d640 1 --2- 172.21.15.16:0/2476752184 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a101016c0 0x7f8a10108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:39.947 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.945+0000 7f8a1528d640 1 --2- 172.21.15.16:0/2476752184 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a10100d00 0x7f8a10101180 unknown :-1 s=CLOSED pgs=1768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:39.947 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.945+0000 7f8a1528d640 1 --2- 172.21.15.16:0/2476752184 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a10102060 0x7f8a10102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:39.947 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.945+0000 7f8a1528d640 1 -- 172.21.15.16:0/2476752184 >> 172.21.15.16:0/2476752184 conn(0x7f8a100fb980 msgr2=0x7f8a100fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:14:39.947 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.945+0000 7f8a1528d640 1 -- 172.21.15.16:0/2476752184 shutdown_connections 2024-09-12T02:14:39.948 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.945+0000 7f8a1528d640 1 -- 172.21.15.16:0/2476752184 wait complete. 2024-09-12T02:14:39.948 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.946+0000 7f8a1528d640 1 Processor -- start 2024-09-12T02:14:39.950 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.946+0000 7f8a1528d640 1 -- start start 2024-09-12T02:14:39.950 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.946+0000 7f8a1528d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a10100d00 0x7f8a10197dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:39.950 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.946+0000 7f8a1528d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a101016c0 0x7f8a10198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:39.950 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.947+0000 7f8a1528d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a10102060 0x7f8a1019f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:39.951 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.947+0000 7f8a1528d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8a1010c6f0 con 0x7f8a10102060 2024-09-12T02:14:39.951 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.947+0000 7f8a1528d640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8a1010c570 con 0x7f8a10100d00 2024-09-12T02:14:39.951 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.947+0000 7f8a1528d640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8a1010c870 con 0x7f8a101016c0 2024-09-12T02:14:39.951 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.947+0000 7f8a0f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a10102060 0x7f8a1019f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:39.951 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.947+0000 7f8a0effd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a10100d00 0x7f8a10197dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:39.951 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.947+0000 7f8a0f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a10102060 0x7f8a1019f380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42940/0 (socket says 172.21.15.16:42940) 2024-09-12T02:14:39.951 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.947+0000 7f8a0f7fe640 1 -- 172.21.15.16:0/3172609835 learned_addr learned my addr 172.21.15.16:0/3172609835 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:14:39.951 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.947+0000 7f8a0e7fc640 1 --2- 172.21.15.16:0/3172609835 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a101016c0 0x7f8a10198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:39.951 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.947+0000 7f8a0f7fe640 1 -- 172.21.15.16:0/3172609835 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a101016c0 msgr2=0x7f8a10198310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:14:39.951 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.947+0000 7f8a0f7fe640 1 --2- 172.21.15.16:0/3172609835 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a101016c0 0x7f8a10198310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:39.951 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.947+0000 7f8a0f7fe640 1 -- 172.21.15.16:0/3172609835 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a10100d00 msgr2=0x7f8a10197dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:14:39.951 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.947+0000 7f8a0f7fe640 1 --2- 172.21.15.16:0/3172609835 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a10100d00 0x7f8a10197dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:39.952 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.947+0000 7f8a0f7fe640 1 -- 172.21.15.16:0/3172609835 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a1019f9f0 con 0x7f8a10102060 2024-09-12T02:14:39.952 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.947+0000 7f8a0f7fe640 1 --2- 172.21.15.16:0/3172609835 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a10102060 0x7f8a1019f380 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7f8a0800eaa0 tx=0x7f8a0800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:14:39.952 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.947+0000 7f8a07fff640 1 -- 172.21.15.16:0/3172609835 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a0800cdb0 con 0x7f8a10102060 2024-09-12T02:14:39.952 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.947+0000 7f8a1528d640 1 -- 172.21.15.16:0/3172609835 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a1019fce0 con 0x7f8a10102060 2024-09-12T02:14:39.952 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.947+0000 7f8a07fff640 1 -- 172.21.15.16:0/3172609835 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a08005020 con 0x7f8a10102060 2024-09-12T02:14:39.952 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.947+0000 7f8a07fff640 1 -- 172.21.15.16:0/3172609835 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a080116d0 con 0x7f8a10102060 2024-09-12T02:14:39.952 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.948+0000 7f8a1528d640 1 -- 172.21.15.16:0/3172609835 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a101a01b0 con 0x7f8a10102060 2024-09-12T02:14:39.960 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.950+0000 7f8a07fff640 1 -- 172.21.15.16:0/3172609835 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f8a080054d0 con 0x7f8a10102060 2024-09-12T02:14:39.960 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.950+0000 7f8a1802d740 1 Processor -- start 2024-09-12T02:14:39.960 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.950+0000 7f8a1802d740 1 -- start start 2024-09-12T02:14:39.960 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.951+0000 7f8a1802d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a08ee269a0 0x55a08ee26da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:39.960 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.951+0000 7f8a1802d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a08ee272e0 0x55a08ee2bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:39.960 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.951+0000 7f8a1802d740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a08ee2c370 0x55a08ee2e830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:39.960 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.951+0000 7f8a1802d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55a08ed8a2f0 con 0x55a08ee269a0 2024-09-12T02:14:39.960 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.951+0000 7f8a1802d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55a08ed32270 con 0x55a08ee272e0 2024-09-12T02:14:39.960 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.951+0000 7f8a1802d740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55a08ed862b0 con 0x55a08ee2c370 2024-09-12T02:14:39.961 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.951+0000 7f8a0e7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a08ee269a0 0x55a08ee26da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:39.961 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.951+0000 7f8a0e7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a08ee269a0 0x55a08ee26da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42956/0 (socket says 172.21.15.16:42956) 2024-09-12T02:14:39.961 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.951+0000 7f8a0e7fc640 1 -- 172.21.15.16:0/550574833 learned_addr learned my addr 172.21.15.16:0/550574833 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:14:39.961 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.951+0000 7f8a07fff640 1 --2- 172.21.15.16:0/3172609835 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f89fc0a2c30 0x7f89fc0a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:39.961 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.951+0000 7f8a0effd640 1 --2- 172.21.15.16:0/550574833 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a08ee272e0 0x55a08ee2bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:39.961 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.951+0000 7f8a0e7fc640 1 --2- 172.21.15.16:0/550574833 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a08ee2c370 0x55a08ee2e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:39.961 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.951+0000 7f8a0f7fe640 1 --2- 172.21.15.16:0/3172609835 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f89fc0a2c30 0x7f89fc0a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:39.961 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.951+0000 7f8a0e7fc640 1 -- 172.21.15.16:0/550574833 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a08ee2c370 msgr2=0x55a08ee2e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:14:39.961 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.951+0000 7f8a0e7fc640 1 --2- 172.21.15.16:0/550574833 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a08ee2c370 0x55a08ee2e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:39.961 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.951+0000 7f8a0e7fc640 1 -- 172.21.15.16:0/550574833 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a08ee272e0 msgr2=0x55a08ee2bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:14:39.961 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.951+0000 7f8a0e7fc640 1 --2- 172.21.15.16:0/550574833 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a08ee272e0 0x55a08ee2bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:39.962 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.951+0000 7f8a0e7fc640 1 -- 172.21.15.16:0/550574833 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a08ed31340 con 0x55a08ee269a0 2024-09-12T02:14:39.962 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.951+0000 7f8a07fff640 1 -- 172.21.15.16:0/3172609835 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f8a080cf260 con 0x7f8a10102060 2024-09-12T02:14:39.962 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.952+0000 7f8a0e7fc640 1 --2- 172.21.15.16:0/550574833 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a08ee269a0 0x55a08ee26da0 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7f8a00031440 tx=0x7f8a000058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=630b10f6cae2b897 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:14:39.962 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.952+0000 7f8a0f7fe640 1 --2- 172.21.15.16:0/3172609835 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f89fc0a2c30 0x7f89fc0a50f0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f8a0800efd0 tx=0x7f8a08066be0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:14:39.962 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.952+0000 7f8a057fa640 1 -- 172.21.15.16:0/550574833 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a00035050 con 0x55a08ee269a0 2024-09-12T02:14:39.962 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.952+0000 7f8a057fa640 1 -- 172.21.15.16:0/550574833 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a000318f0 con 0x55a08ee269a0 2024-09-12T02:14:39.962 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.952+0000 7f8a057fa640 1 -- 172.21.15.16:0/550574833 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a00004920 con 0x55a08ee269a0 2024-09-12T02:14:39.962 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.953+0000 7f8a1802d740 1 -- 172.21.15.16:0/550574833 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a08ee269a0 msgr2=0x55a08ee26da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:14:39.962 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.953+0000 7f8a1802d740 1 --2- 172.21.15.16:0/550574833 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a08ee269a0 0x55a08ee26da0 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7f8a00031440 tx=0x7f8a000058b0 comp rx=0 tx=0).stop 2024-09-12T02:14:39.962 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.953+0000 7f8a1802d740 1 -- 172.21.15.16:0/550574833 shutdown_connections 2024-09-12T02:14:39.962 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.953+0000 7f8a1802d740 1 --2- 172.21.15.16:0/550574833 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a08ee2c370 0x55a08ee2e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:39.962 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.953+0000 7f8a1802d740 1 --2- 172.21.15.16:0/550574833 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a08ee272e0 0x55a08ee2bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:39.963 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.953+0000 7f8a1802d740 1 --2- 172.21.15.16:0/550574833 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a08ee269a0 0x55a08ee26da0 unknown :-1 s=CLOSED pgs=1770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:39.963 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.953+0000 7f8a1802d740 1 -- 172.21.15.16:0/550574833 >> 172.21.15.16:0/550574833 conn(0x55a08ee22010 msgr2=0x55a08ee22410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:14:39.963 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.953+0000 7f8a1802d740 1 -- 172.21.15.16:0/550574833 shutdown_connections 2024-09-12T02:14:39.963 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.953+0000 7f8a1802d740 1 -- 172.21.15.16:0/550574833 wait complete. 2024-09-12T02:14:39.963 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.953+0000 7f8a1802d740 1 -- start start 2024-09-12T02:14:39.963 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.953+0000 7f8a1802d740 1 Processor -- start 2024-09-12T02:14:39.963 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.954+0000 7f8a1802d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a08ee269a0 0x55a08ee30b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:39.963 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.954+0000 7f8a1802d740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a08ee272e0 0x55a08ee310d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:39.963 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.954+0000 7f8a1802d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a08ee2c370 0x55a08ee38110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:39.963 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.954+0000 7f8a1802d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55a08ee2f2e0 con 0x55a08ee269a0 2024-09-12T02:14:39.963 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.954+0000 7f8a1802d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55a08ed13290 con 0x55a08ee2c370 2024-09-12T02:14:39.963 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.954+0000 7f8a1802d740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55a08ee2f460 con 0x55a08ee272e0 2024-09-12T02:14:39.963 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.954+0000 7f8a1802d740 10 client.0 _ll_register_callbacks cb 0x55a08ed44140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:14:39.964 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.954+0000 7f8a0e7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a08ee269a0 0x55a08ee30b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:39.964 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.954+0000 7f8a0e7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a08ee269a0 0x55a08ee30b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42970/0 (socket says 172.21.15.16:42970) 2024-09-12T02:14:39.964 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.954+0000 7f8a0e7fc640 1 -- 172.21.15.16:0/1222933948 learned_addr learned my addr 172.21.15.16:0/1222933948 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:14:39.964 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.954+0000 7f8a0effd640 1 --2- 172.21.15.16:0/1222933948 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a08ee272e0 0x55a08ee310d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:39.964 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.954+0000 7f8a0e7fc640 1 --2- 172.21.15.16:0/1222933948 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a08ee2c370 0x55a08ee38110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:39.964 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.954+0000 7f8a0e7fc640 1 -- 172.21.15.16:0/1222933948 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a08ee272e0 msgr2=0x55a08ee310d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:14:39.964 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.954+0000 7f8a0e7fc640 1 --2- 172.21.15.16:0/1222933948 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a08ee272e0 0x55a08ee310d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:39.964 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.954+0000 7f8a0e7fc640 1 -- 172.21.15.16:0/1222933948 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a08ee2c370 msgr2=0x55a08ee38110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:14:39.964 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.954+0000 7f8a0e7fc640 1 --2- 172.21.15.16:0/1222933948 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a08ee2c370 0x55a08ee38110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:39.964 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.954+0000 7f8a0e7fc640 1 -- 172.21.15.16:0/1222933948 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a08ea9d210 con 0x55a08ee269a0 2024-09-12T02:14:39.965 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.954+0000 7f8a0e7fc640 1 --2- 172.21.15.16:0/1222933948 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a08ee269a0 0x55a08ee30b90 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7f8a00000c00 tx=0x7f8a00031aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=449f060ab6106b85 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:14:39.965 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.954+0000 7f89f67fc640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:14:39.965 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.955+0000 7f89f67fc640 1 -- 172.21.15.16:0/1222933948 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a00051060 con 0x55a08ee269a0 2024-09-12T02:14:39.965 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.955+0000 7f8a1802d740 1 -- 172.21.15.16:0/1222933948 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a000318f0 con 0x55a08ee269a0 2024-09-12T02:14:39.965 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.955+0000 7f89f67fc640 1 -- 172.21.15.16:0/1222933948 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f8a00003ab0 con 0x55a08ee269a0 2024-09-12T02:14:39.965 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.955+0000 7f89f67fc640 1 -- 172.21.15.16:0/1222933948 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f8a00035040 con 0x55a08ee269a0 2024-09-12T02:14:39.965 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.955+0000 7f8a1802d740 10 client.19710 fetch_fsmap learned FSMap version 25 2024-09-12T02:14:39.965 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.955+0000 7f8a1802d740 1 -- 172.21.15.16:0/1222933948 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a08eb69620 con 0x55a08ee269a0 2024-09-12T02:14:39.965 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.955+0000 7f89f67fc640 1 -- 172.21.15.16:0/1222933948 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a000318f0 con 0x55a08ee269a0 2024-09-12T02:14:39.965 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.955+0000 7f89f67fc640 1 -- 172.21.15.16:0/1222933948 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f8a00056070 con 0x55a08ee269a0 2024-09-12T02:14:39.966 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.955+0000 7f8a1802d740 10 client.19710 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:14:39.966 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.955+0000 7f8a1802d740 10 client.19710 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:14:39.966 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.955+0000 7f8a1802d740 20 client.19710 populate_metadata read hostname 'smithi016' 2024-09-12T02:14:39.966 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.955+0000 7f8a1802d740 1 --2- 172.21.15.16:0/1222933948 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55a08ee3c410 0x55a08ee3e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:39.966 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.955+0000 7f8a1802d740 4 client.19710 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:14:39.966 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.955+0000 7f8a1802d740 1 -- 172.21.15.16:0/1222933948 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a08ed97a20 con 0x55a08ee3c410 2024-09-12T02:14:39.966 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.956+0000 7f8a0effd640 1 --2- 172.21.15.16:0/1222933948 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55a08ee3c410 0x55a08ee3e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:39.966 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.956+0000 7f8a0effd640 1 --2- 172.21.15.16:0/1222933948 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55a08ee3c410 0x55a08ee3e800 crc :-1 s=READY pgs=384 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61484b641d7c67f server_cookie=944f116354c354ef in_seq=0 out_seq=0 2024-09-12T02:14:39.966 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.956+0000 7f89f67fc640 10 client.19710 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:14:39.966 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.957+0000 7f89f67fc640 1 -- 172.21.15.16:0/1222933948 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a08ed97a20 con 0x55a08ee3c410 2024-09-12T02:14:39.966 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.957+0000 7f89f67fc640 10 client.19710 handle_command_reply: tid=0 2024-09-12T02:14:39.967 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.957+0000 7f8a1802d740 1 client.19710 shutdown 2024-09-12T02:14:39.967 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.957+0000 7f8a1802d740 1 -- 172.21.15.16:0/1222933948 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a08ee269a0 msgr2=0x55a08ee30b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:14:39.967 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.957+0000 7f8a1802d740 1 --2- 172.21.15.16:0/1222933948 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a08ee269a0 0x55a08ee30b90 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7f8a00000c00 tx=0x7f8a00031aa0 comp rx=0 tx=0).stop 2024-09-12T02:14:39.967 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.957+0000 7f8a1802d740 1 -- 172.21.15.16:0/1222933948 shutdown_connections 2024-09-12T02:14:39.967 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.957+0000 7f8a1802d740 1 --2- 172.21.15.16:0/1222933948 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55a08ee3c410 0x55a08ee3e800 crc :-1 s=READY pgs=384 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:39.967 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.957+0000 7f8a1802d740 1 --2- 172.21.15.16:0/1222933948 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a08ee2c370 0x55a08ee38110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:39.967 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.957+0000 7f8a1802d740 1 --2- 172.21.15.16:0/1222933948 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a08ee272e0 0x55a08ee310d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:39.967 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.957+0000 7f89f67fc640 0 client.19710 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:14:39.967 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.957+0000 7f8a1802d740 1 --2- 172.21.15.16:0/1222933948 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a08ee269a0 0x55a08ee30b90 unknown :-1 s=CLOSED pgs=1771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:39.967 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.957+0000 7f8a1802d740 1 -- 172.21.15.16:0/1222933948 >> 172.21.15.16:0/1222933948 conn(0x55a08ee22010 msgr2=0x55a08ee23820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:14:39.967 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.957+0000 7f8a1802d740 1 -- 172.21.15.16:0/1222933948 shutdown_connections 2024-09-12T02:14:39.968 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.957+0000 7f8a1802d740 1 -- 172.21.15.16:0/1222933948 wait complete. 2024-09-12T02:14:39.968 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.957+0000 7f8a1802d740 20 client.19710 trim_cache size 0 max 16384 2024-09-12T02:14:39.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.972+0000 7f8a1802d740 1 Processor -- start 2024-09-12T02:14:39.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.972+0000 7f8a1802d740 1 -- start start 2024-09-12T02:14:39.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.972+0000 7f8a1802d740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a08ee269a0 0x55a08ee31160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:39.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.972+0000 7f8a1802d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a08ee272e0 0x55a08ee380a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:39.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.972+0000 7f8a1802d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a08ee2c370 0x55a08ee219b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:39.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.972+0000 7f8a1802d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8a101ab650 con 0x55a08ee272e0 2024-09-12T02:14:39.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.972+0000 7f8a1802d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8a101a8f20 con 0x55a08ee2c370 2024-09-12T02:14:39.976 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.972+0000 7f8a1802d740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8a101a6d70 con 0x55a08ee269a0 2024-09-12T02:14:39.976 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.972+0000 7f8a0effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a08ee272e0 0x55a08ee380a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:39.976 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.972+0000 7f8a0effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a08ee272e0 0x55a08ee380a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42984/0 (socket says 172.21.15.16:42984) 2024-09-12T02:14:39.976 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.972+0000 7f8a0effd640 1 -- 172.21.15.16:0/2246651628 learned_addr learned my addr 172.21.15.16:0/2246651628 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:14:39.976 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.973+0000 7f8a0e7fc640 1 --2- 172.21.15.16:0/2246651628 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a08ee269a0 0x55a08ee31160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:39.976 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.973+0000 7f8a0effd640 1 -- 172.21.15.16:0/2246651628 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a08ee269a0 msgr2=0x55a08ee31160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:14:39.976 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.973+0000 7f8a0effd640 1 --2- 172.21.15.16:0/2246651628 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a08ee269a0 0x55a08ee31160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:39.976 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.973+0000 7f8a0e7fc640 1 --2- 172.21.15.16:0/2246651628 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a08ee2c370 0x55a08ee219b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:39.976 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.973+0000 7f8a0effd640 1 -- 172.21.15.16:0/2246651628 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a08ee2c370 msgr2=0x55a08ee219b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:14:39.976 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.973+0000 7f8a0effd640 1 --2- 172.21.15.16:0/2246651628 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a08ee2c370 0x55a08ee219b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:39.976 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.973+0000 7f8a0effd640 1 -- 172.21.15.16:0/2246651628 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a08ee2b780 con 0x55a08ee272e0 2024-09-12T02:14:39.976 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.973+0000 7f8a0e7fc640 1 --2- 172.21.15.16:0/2246651628 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a08ee2c370 0x55a08ee219b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:14:39.977 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.973+0000 7f8a0effd640 1 --2- 172.21.15.16:0/2246651628 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a08ee272e0 0x55a08ee380a0 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7f89f800b740 tx=0x7f89f800b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b6741e822951f99 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:14:39.977 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.973+0000 7f8a04ff9640 1 -- 172.21.15.16:0/2246651628 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89f8003d60 con 0x55a08ee272e0 2024-09-12T02:14:39.977 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.973+0000 7f8a04ff9640 1 -- 172.21.15.16:0/2246651628 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89f8005500 con 0x55a08ee272e0 2024-09-12T02:14:39.977 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.973+0000 7f8a04ff9640 1 -- 172.21.15.16:0/2246651628 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89f80116a0 con 0x55a08ee272e0 2024-09-12T02:14:39.977 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.974+0000 7f8a1802d740 1 -- 172.21.15.16:0/2246651628 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a08ee272e0 msgr2=0x55a08ee380a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:14:39.977 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.974+0000 7f8a1802d740 1 --2- 172.21.15.16:0/2246651628 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a08ee272e0 0x55a08ee380a0 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7f89f800b740 tx=0x7f89f800b390 comp rx=0 tx=0).stop 2024-09-12T02:14:39.977 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.974+0000 7f8a1802d740 1 -- 172.21.15.16:0/2246651628 shutdown_connections 2024-09-12T02:14:39.977 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.974+0000 7f8a1802d740 1 --2- 172.21.15.16:0/2246651628 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a08ee2c370 0x55a08ee219b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:39.977 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.974+0000 7f8a1802d740 1 --2- 172.21.15.16:0/2246651628 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a08ee272e0 0x55a08ee380a0 unknown :-1 s=CLOSED pgs=1772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:39.977 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.974+0000 7f8a1802d740 1 --2- 172.21.15.16:0/2246651628 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a08ee269a0 0x55a08ee31160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:39.977 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.974+0000 7f8a1802d740 1 -- 172.21.15.16:0/2246651628 >> 172.21.15.16:0/2246651628 conn(0x55a08ee22010 msgr2=0x55a08ee22ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:14:39.977 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.974+0000 7f8a1802d740 1 -- 172.21.15.16:0/2246651628 shutdown_connections 2024-09-12T02:14:39.977 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.974+0000 7f8a1802d740 1 -- 172.21.15.16:0/2246651628 wait complete. 2024-09-12T02:14:39.978 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.974+0000 7f8a1802d740 1 -- start start 2024-09-12T02:14:39.978 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.974+0000 7f8a1802d740 1 Processor -- start 2024-09-12T02:14:39.980 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.975+0000 7f8a1802d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a08ee269a0 0x55a08ee35d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:39.981 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.975+0000 7f8a1802d740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a08ee272e0 0x55a08ee36250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:39.981 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.975+0000 7f8a1802d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a08ee2c370 0x55a08ee37210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:39.981 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.975+0000 7f8a1802d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8a101a73e0 con 0x55a08ee269a0 2024-09-12T02:14:39.981 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.975+0000 7f8a1802d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8a101a8850 con 0x55a08ee2c370 2024-09-12T02:14:39.981 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.975+0000 7f8a1802d740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8a101a7ab0 con 0x55a08ee272e0 2024-09-12T02:14:39.981 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.975+0000 7f8a1802d740 10 client.0 _ll_register_callbacks cb 0x7f89f800b770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:14:39.981 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.975+0000 7f8a0e7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a08ee269a0 0x55a08ee35d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:39.981 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.975+0000 7f8a0e7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a08ee269a0 0x55a08ee35d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43000/0 (socket says 172.21.15.16:43000) 2024-09-12T02:14:39.981 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.975+0000 7f8a0e7fc640 1 -- 172.21.15.16:0/4226505448 learned_addr learned my addr 172.21.15.16:0/4226505448 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:14:39.981 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.975+0000 7f8a0effd640 1 --2- 172.21.15.16:0/4226505448 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a08ee272e0 0x55a08ee36250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:39.981 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.975+0000 7f8a0e7fc640 1 --2- 172.21.15.16:0/4226505448 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a08ee2c370 0x55a08ee37210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:39.981 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.975+0000 7f8a0e7fc640 1 -- 172.21.15.16:0/4226505448 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a08ee272e0 msgr2=0x55a08ee36250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:14:39.982 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.975+0000 7f8a0e7fc640 1 --2- 172.21.15.16:0/4226505448 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a08ee272e0 0x55a08ee36250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:39.982 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.975+0000 7f8a0e7fc640 1 -- 172.21.15.16:0/4226505448 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a08ee2c370 msgr2=0x55a08ee37210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:14:39.982 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.975+0000 7f8a0e7fc640 1 --2- 172.21.15.16:0/4226505448 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a08ee2c370 0x55a08ee37210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:39.982 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.975+0000 7f8a0e7fc640 1 -- 172.21.15.16:0/4226505448 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a08ee37820 con 0x55a08ee269a0 2024-09-12T02:14:39.982 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.975+0000 7f8a0e7fc640 1 --2- 172.21.15.16:0/4226505448 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a08ee269a0 0x55a08ee35d10 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7f8a00031bf0 tx=0x7f8a00031aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98b4d4d02d2e9df4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:14:39.982 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.975+0000 7f89f67fc640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:14:39.982 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.975+0000 7f89f67fc640 1 -- 172.21.15.16:0/4226505448 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a0005c040 con 0x55a08ee269a0 2024-09-12T02:14:39.982 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.975+0000 7f89f67fc640 1 -- 172.21.15.16:0/4226505448 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f8a0004a8c0 con 0x55a08ee269a0 2024-09-12T02:14:39.982 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.975+0000 7f8a1802d740 1 -- 172.21.15.16:0/4226505448 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f89f8005500 con 0x55a08ee269a0 2024-09-12T02:14:39.982 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.976+0000 7f89f67fc640 1 -- 172.21.15.16:0/4226505448 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f8a000318f0 con 0x55a08ee269a0 2024-09-12T02:14:39.982 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.976+0000 7f8a1802d740 10 client.19716 fetch_fsmap learned FSMap version 25 2024-09-12T02:14:39.982 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.976+0000 7f8a1802d740 1 -- 172.21.15.16:0/4226505448 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a08eea7400 con 0x55a08ee269a0 2024-09-12T02:14:39.983 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.976+0000 7f89f67fc640 1 -- 172.21.15.16:0/4226505448 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89f8005500 con 0x55a08ee269a0 2024-09-12T02:14:39.983 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.976+0000 7f89f67fc640 1 -- 172.21.15.16:0/4226505448 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f8a00056040 con 0x55a08ee269a0 2024-09-12T02:14:39.983 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.976+0000 7f8a1802d740 10 client.19716 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:14:39.983 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.976+0000 7f8a1802d740 10 client.19716 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:14:39.983 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.976+0000 7f8a1802d740 20 client.19716 populate_metadata read hostname 'smithi016' 2024-09-12T02:14:39.983 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.976+0000 7f8a1802d740 1 --2- 172.21.15.16:0/4226505448 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55a08eea7d80 0x55a08eeaa1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:39.983 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.976+0000 7f8a1802d740 4 client.19716 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:14:39.983 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.976+0000 7f8a1802d740 1 -- 172.21.15.16:0/4226505448 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a08ed91d70 con 0x55a08eea7d80 2024-09-12T02:14:39.983 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.976+0000 7f8a0effd640 1 --2- 172.21.15.16:0/4226505448 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55a08eea7d80 0x55a08eeaa1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:39.983 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.977+0000 7f8a0effd640 1 --2- 172.21.15.16:0/4226505448 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55a08eea7d80 0x55a08eeaa1a0 crc :-1 s=READY pgs=385 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42d918826a4ce6be server_cookie=576812ab05a446af in_seq=0 out_seq=0 2024-09-12T02:14:39.983 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.977+0000 7f89f67fc640 10 client.19716 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:14:39.993 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.992+0000 7f89f67fc640 1 -- 172.21.15.16:0/4226505448 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a08ed91d70 con 0x55a08eea7d80 2024-09-12T02:14:39.993 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.992+0000 7f89f67fc640 10 client.19716 handle_command_reply: tid=0 2024-09-12T02:14:39.994 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.992+0000 7f8a1802d740 1 client.19716 shutdown 2024-09-12T02:14:39.994 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.992+0000 7f8a1802d740 1 -- 172.21.15.16:0/4226505448 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a08ee269a0 msgr2=0x55a08ee35d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:14:39.994 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.992+0000 7f8a1802d740 1 --2- 172.21.15.16:0/4226505448 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a08ee269a0 0x55a08ee35d10 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7f8a00031bf0 tx=0x7f8a00031aa0 comp rx=0 tx=0).stop 2024-09-12T02:14:39.994 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.992+0000 7f8a1802d740 1 -- 172.21.15.16:0/4226505448 shutdown_connections 2024-09-12T02:14:39.994 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.992+0000 7f8a1802d740 1 --2- 172.21.15.16:0/4226505448 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55a08eea7d80 0x55a08eeaa1a0 crc :-1 s=READY pgs=385 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:39.994 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.992+0000 7f8a1802d740 1 --2- 172.21.15.16:0/4226505448 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a08ee2c370 0x55a08ee37210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:39.994 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.992+0000 7f8a1802d740 1 --2- 172.21.15.16:0/4226505448 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a08ee272e0 0x55a08ee36250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:39.995 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.992+0000 7f8a1802d740 1 --2- 172.21.15.16:0/4226505448 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a08ee269a0 0x55a08ee35d10 unknown :-1 s=CLOSED pgs=1773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:39.995 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.992+0000 7f89f67fc640 0 client.19716 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:14:39.995 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.992+0000 7f8a1802d740 1 -- 172.21.15.16:0/4226505448 >> 172.21.15.16:0/4226505448 conn(0x55a08ee22010 msgr2=0x55a08ee236b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:14:39.995 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.992+0000 7f8a1802d740 1 -- 172.21.15.16:0/4226505448 shutdown_connections 2024-09-12T02:14:39.995 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.992+0000 7f8a1802d740 1 -- 172.21.15.16:0/4226505448 wait complete. 2024-09-12T02:14:39.995 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.992+0000 7f8a1802d740 20 client.19716 trim_cache size 0 max 16384 2024-09-12T02:14:39.996 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-09-12T02:14:39.996 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-09-12T02:14:39.996 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-09-12T02:14:39.996 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-09-12T02:14:39.997 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.992+0000 7f8a1528d640 1 -- 172.21.15.16:0/3172609835 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f89fc0a2c30 msgr2=0x7f89fc0a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:14:39.997 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.992+0000 7f8a1528d640 1 --2- 172.21.15.16:0/3172609835 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f89fc0a2c30 0x7f89fc0a50f0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f8a0800efd0 tx=0x7f8a08066be0 comp rx=0 tx=0).stop 2024-09-12T02:14:39.997 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.992+0000 7f8a1528d640 1 -- 172.21.15.16:0/3172609835 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a10102060 msgr2=0x7f8a1019f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:14:39.997 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.992+0000 7f8a1528d640 1 --2- 172.21.15.16:0/3172609835 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a10102060 0x7f8a1019f380 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7f8a0800eaa0 tx=0x7f8a0800ef60 comp rx=0 tx=0).stop 2024-09-12T02:14:39.998 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.992+0000 7f8a1528d640 1 -- 172.21.15.16:0/3172609835 shutdown_connections 2024-09-12T02:14:39.998 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.992+0000 7f8a1528d640 1 --2- 172.21.15.16:0/3172609835 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f89fc0a2c30 0x7f89fc0a50f0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:39.998 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.992+0000 7f8a1528d640 1 --2- 172.21.15.16:0/3172609835 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a10102060 0x7f8a1019f380 unknown :-1 s=CLOSED pgs=1769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:39.998 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.992+0000 7f8a1528d640 1 --2- 172.21.15.16:0/3172609835 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a101016c0 0x7f8a10198310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:39.998 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.992+0000 7f8a1528d640 1 --2- 172.21.15.16:0/3172609835 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a10100d00 0x7f8a10197dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:39.998 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.992+0000 7f8a1528d640 1 -- 172.21.15.16:0/3172609835 >> 172.21.15.16:0/3172609835 conn(0x7f8a100fb980 msgr2=0x7f8a100fd920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:14:39.999 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.992+0000 7f8a1528d640 1 -- 172.21.15.16:0/3172609835 shutdown_connections 2024-09-12T02:14:39.999 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:39.992+0000 7f8a1528d640 1 -- 172.21.15.16:0/3172609835 wait complete. 2024-09-12T02:14:40.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:14:40 smithi184 ceph-mon[35593]: pgmap v717: 97 pgs: 97 active+clean; 2.7 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-09-12T02:14:40.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:14:40 smithi033 ceph-mon[35332]: pgmap v717: 97 pgs: 97 active+clean; 2.7 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-09-12T02:14:40.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:14:40 smithi016 ceph-mon[30947]: pgmap v717: 97 pgs: 97 active+clean; 2.7 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-09-12T02:14:42.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:14:42 smithi184 ceph-mon[35593]: pgmap v718: 97 pgs: 97 active+clean; 2.7 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-09-12T02:14:42.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:14:42 smithi033 ceph-mon[35332]: pgmap v718: 97 pgs: 97 active+clean; 2.7 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-09-12T02:14:42.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:14:42 smithi016 ceph-mon[30947]: pgmap v718: 97 pgs: 97 active+clean; 2.7 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-09-12T02:14:44.628 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:14:44 smithi184 ceph-mon[35593]: pgmap v719: 97 pgs: 97 active+clean; 2.7 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-09-12T02:14:44.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:14:44 smithi033 ceph-mon[35332]: pgmap v719: 97 pgs: 97 active+clean; 2.7 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-09-12T02:14:44.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:14:44 smithi016 ceph-mon[30947]: pgmap v719: 97 pgs: 97 active+clean; 2.7 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-09-12T02:14:46.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:14:46 smithi184 ceph-mon[35593]: pgmap v720: 97 pgs: 97 active+clean; 2.7 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-09-12T02:14:46.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:14:46 smithi033 ceph-mon[35332]: pgmap v720: 97 pgs: 97 active+clean; 2.7 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-09-12T02:14:46.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:14:46 smithi016 ceph-mon[30947]: pgmap v720: 97 pgs: 97 active+clean; 2.7 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-09-12T02:14:48.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:14:48 smithi184 ceph-mon[35593]: pgmap v721: 97 pgs: 97 active+clean; 2.8 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-09-12T02:14:48.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:14:48 smithi033 ceph-mon[35332]: pgmap v721: 97 pgs: 97 active+clean; 2.8 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-09-12T02:14:48.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:14:48 smithi016 ceph-mon[30947]: pgmap v721: 97 pgs: 97 active+clean; 2.8 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-09-12T02:14:50.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:14:50 smithi184 ceph-mon[35593]: pgmap v722: 97 pgs: 97 active+clean; 2.8 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-09-12T02:14:50.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:14:50 smithi033 ceph-mon[35332]: pgmap v722: 97 pgs: 97 active+clean; 2.8 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-09-12T02:14:50.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:14:50 smithi016 ceph-mon[30947]: pgmap v722: 97 pgs: 97 active+clean; 2.8 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-09-12T02:14:52.629 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:14:52 smithi184 ceph-mon[35593]: pgmap v723: 97 pgs: 97 active+clean; 2.8 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-12T02:14:52.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:14:52 smithi033 ceph-mon[35332]: pgmap v723: 97 pgs: 97 active+clean; 2.8 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-12T02:14:52.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:14:52 smithi016 ceph-mon[30947]: pgmap v723: 97 pgs: 97 active+clean; 2.8 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-12T02:14:53.095 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.094+0000 7fefba18f640 1 -- 172.21.15.16:0/1199976977 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fefb4102ea0 msgr2=0x7fefb4103320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:14:53.096 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.094+0000 7fefba18f640 1 --2- 172.21.15.16:0/1199976977 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fefb4102ea0 0x7fefb4103320 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7fefa8009900 tx=0x7fefa802a3f0 comp rx=0 tx=0).stop 2024-09-12T02:14:53.096 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.095+0000 7fefba18f640 1 -- 172.21.15.16:0/1199976977 shutdown_connections 2024-09-12T02:14:53.096 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.095+0000 7fefba18f640 1 --2- 172.21.15.16:0/1199976977 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fefb4103860 0x7fefb410a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:53.096 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.095+0000 7fefba18f640 1 --2- 172.21.15.16:0/1199976977 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fefb4102ea0 0x7fefb4103320 unknown :-1 s=CLOSED pgs=1774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:53.096 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.095+0000 7fefba18f640 1 --2- 172.21.15.16:0/1199976977 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fefb40ffe30 0x7fefb4100230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:53.096 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.095+0000 7fefba18f640 1 -- 172.21.15.16:0/1199976977 >> 172.21.15.16:0/1199976977 conn(0x7fefb40fb970 msgr2=0x7fefb40fdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:14:53.096 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.095+0000 7fefba18f640 1 -- 172.21.15.16:0/1199976977 shutdown_connections 2024-09-12T02:14:53.096 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.095+0000 7fefba18f640 1 -- 172.21.15.16:0/1199976977 wait complete. 2024-09-12T02:14:53.096 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.095+0000 7fefba18f640 1 Processor -- start 2024-09-12T02:14:53.097 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.096+0000 7fefba18f640 1 -- start start 2024-09-12T02:14:53.099 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.096+0000 7fefba18f640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fefb40ffe30 0x7fefb4102810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:53.099 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.096+0000 7fefba18f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fefb4102ea0 0x7fefb4100e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:53.099 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.096+0000 7fefba18f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fefb4103860 0x7fefb4101390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:53.099 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.096+0000 7fefba18f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fefb410d7c0 con 0x7fefb4103860 2024-09-12T02:14:53.099 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.096+0000 7fefba18f640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fefb410d640 con 0x7fefb4102ea0 2024-09-12T02:14:53.099 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.096+0000 7fefba18f640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fefb410d940 con 0x7fefb40ffe30 2024-09-12T02:14:53.099 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.096+0000 7fefb3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fefb4103860 0x7fefb4101390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:53.099 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.096+0000 7fefb3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fefb4103860 0x7fefb4101390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38726/0 (socket says 172.21.15.16:38726) 2024-09-12T02:14:53.099 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.096+0000 7fefb3fff640 1 -- 172.21.15.16:0/2998304674 learned_addr learned my addr 172.21.15.16:0/2998304674 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:14:53.100 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.096+0000 7fefb37fe640 1 --2- 172.21.15.16:0/2998304674 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fefb40ffe30 0x7fefb4102810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:53.100 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.096+0000 7fefb2ffd640 1 --2- 172.21.15.16:0/2998304674 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fefb4102ea0 0x7fefb4100e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:53.100 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.096+0000 7fefb3fff640 1 -- 172.21.15.16:0/2998304674 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fefb40ffe30 msgr2=0x7fefb4102810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:14:53.100 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.096+0000 7fefb3fff640 1 --2- 172.21.15.16:0/2998304674 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fefb40ffe30 0x7fefb4102810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:53.100 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.096+0000 7fefb3fff640 1 -- 172.21.15.16:0/2998304674 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fefb4102ea0 msgr2=0x7fefb4100e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:14:53.100 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.096+0000 7fefb3fff640 1 --2- 172.21.15.16:0/2998304674 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fefb4102ea0 0x7fefb4100e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:53.100 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.096+0000 7fefb3fff640 1 -- 172.21.15.16:0/2998304674 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefb4101c20 con 0x7fefb4103860 2024-09-12T02:14:53.100 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.096+0000 7fefb2ffd640 1 --2- 172.21.15.16:0/2998304674 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fefb4102ea0 0x7fefb4100e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:14:53.100 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.097+0000 7fefb3fff640 1 --2- 172.21.15.16:0/2998304674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fefb4103860 0x7fefb4101390 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7fefac00cce0 tx=0x7fefac00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:14:53.100 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.097+0000 7fefb0ff9640 1 -- 172.21.15.16:0/2998304674 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefac00f450 con 0x7fefb4103860 2024-09-12T02:14:53.101 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.097+0000 7fefb0ff9640 1 -- 172.21.15.16:0/2998304674 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fefac00fad0 con 0x7fefb4103860 2024-09-12T02:14:53.101 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.097+0000 7fefba18f640 1 -- 172.21.15.16:0/2998304674 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefb41a1e70 con 0x7fefb4103860 2024-09-12T02:14:53.101 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.097+0000 7fefb0ff9640 1 -- 172.21.15.16:0/2998304674 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefac018c10 con 0x7fefb4103860 2024-09-12T02:14:53.101 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.097+0000 7fefba18f640 1 -- 172.21.15.16:0/2998304674 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefb41a2280 con 0x7fefb4103860 2024-09-12T02:14:53.108 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.099+0000 7fefb0ff9640 1 -- 172.21.15.16:0/2998304674 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fefac00f5f0 con 0x7fefb4103860 2024-09-12T02:14:53.109 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.099+0000 7fefbcfff740 1 Processor -- start 2024-09-12T02:14:53.109 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.099+0000 7fefbcfff740 1 -- start start 2024-09-12T02:14:53.109 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.100+0000 7fefbcfff740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5645b0beb9a0 0x5645b0bebda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:53.109 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.100+0000 7fefbcfff740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5645b0bec2e0 0x5645b0bf0da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:53.109 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.100+0000 7fefbcfff740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5645b0bf1370 0x5645b0bf3830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:53.109 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.100+0000 7fefbcfff740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5645b0b4f2f0 con 0x5645b0bec2e0 2024-09-12T02:14:53.109 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.100+0000 7fefbcfff740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5645b0af7270 con 0x5645b0bf1370 2024-09-12T02:14:53.109 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.100+0000 7fefbcfff740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5645b0b4b2b0 con 0x5645b0beb9a0 2024-09-12T02:14:53.109 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.100+0000 7fefb37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5645b0bec2e0 0x5645b0bf0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:53.109 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.100+0000 7fefb37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5645b0bec2e0 0x5645b0bf0da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38728/0 (socket says 172.21.15.16:38728) 2024-09-12T02:14:53.109 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.100+0000 7fefb37fe640 1 -- 172.21.15.16:0/1824713668 learned_addr learned my addr 172.21.15.16:0/1824713668 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:14:53.110 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.100+0000 7fefb2ffd640 1 --2- 172.21.15.16:0/1824713668 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5645b0beb9a0 0x5645b0bebda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:53.110 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.100+0000 7fefb2ffd640 1 --2- 172.21.15.16:0/1824713668 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5645b0bf1370 0x5645b0bf3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:53.110 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.100+0000 7fefb37fe640 1 -- 172.21.15.16:0/1824713668 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5645b0beb9a0 msgr2=0x5645b0bebda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:14:53.110 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.100+0000 7fefb37fe640 1 --2- 172.21.15.16:0/1824713668 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5645b0beb9a0 0x5645b0bebda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:53.110 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.100+0000 7fefb37fe640 1 -- 172.21.15.16:0/1824713668 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5645b0bf1370 msgr2=0x5645b0bf3830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:14:53.110 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.100+0000 7fefb37fe640 1 --2- 172.21.15.16:0/1824713668 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5645b0bf1370 0x5645b0bf3830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:53.110 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.100+0000 7fefb0ff9640 1 --2- 172.21.15.16:0/2998304674 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fef9c0a2c30 0x7fef9c0a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:53.110 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.100+0000 7fefb37fe640 1 -- 172.21.15.16:0/1824713668 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5645b0af6340 con 0x5645b0bec2e0 2024-09-12T02:14:53.110 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.100+0000 7fefb2ffd640 1 --2- 172.21.15.16:0/1824713668 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5645b0bf1370 0x5645b0bf3830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:14:53.110 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.100+0000 7fefb3fff640 1 --2- 172.21.15.16:0/2998304674 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fef9c0a2c30 0x7fef9c0a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:53.110 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.100+0000 7fefb37fe640 1 --2- 172.21.15.16:0/1824713668 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5645b0bec2e0 0x5645b0bf0da0 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7fefa400ac70 tx=0x7fefa400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=987a5fa832554828 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:14:53.110 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.100+0000 7fefb0ff9640 1 -- 172.21.15.16:0/2998304674 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fefac0cf5f0 con 0x7fefb4103860 2024-09-12T02:14:53.110 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.101+0000 7fef99ffb640 1 -- 172.21.15.16:0/1824713668 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefa4005220 con 0x5645b0bec2e0 2024-09-12T02:14:53.111 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.101+0000 7fef99ffb640 1 -- 172.21.15.16:0/1824713668 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fefa4004c80 con 0x5645b0bec2e0 2024-09-12T02:14:53.111 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.101+0000 7fef99ffb640 1 -- 172.21.15.16:0/1824713668 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefa4018940 con 0x5645b0bec2e0 2024-09-12T02:14:53.111 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.101+0000 7fefbcfff740 1 -- 172.21.15.16:0/1824713668 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5645b0bec2e0 msgr2=0x5645b0bf0da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:14:53.111 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.101+0000 7fefb3fff640 1 --2- 172.21.15.16:0/2998304674 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fef9c0a2c30 0x7fef9c0a50f0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fefac0988c0 tx=0x7fefac066f60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:14:53.111 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.101+0000 7fefbcfff740 1 --2- 172.21.15.16:0/1824713668 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5645b0bec2e0 0x5645b0bf0da0 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7fefa400ac70 tx=0x7fefa400f790 comp rx=0 tx=0).stop 2024-09-12T02:14:53.111 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.101+0000 7fefbcfff740 1 -- 172.21.15.16:0/1824713668 shutdown_connections 2024-09-12T02:14:53.111 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.101+0000 7fefbcfff740 1 --2- 172.21.15.16:0/1824713668 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5645b0bf1370 0x5645b0bf3830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:53.111 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.101+0000 7fefbcfff740 1 --2- 172.21.15.16:0/1824713668 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5645b0bec2e0 0x5645b0bf0da0 unknown :-1 s=CLOSED pgs=1776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:53.111 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.101+0000 7fefbcfff740 1 --2- 172.21.15.16:0/1824713668 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5645b0beb9a0 0x5645b0bebda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:53.111 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.101+0000 7fefbcfff740 1 -- 172.21.15.16:0/1824713668 >> 172.21.15.16:0/1824713668 conn(0x5645b0be7010 msgr2=0x5645b0be7410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:14:53.111 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.102+0000 7fefbcfff740 1 -- 172.21.15.16:0/1824713668 shutdown_connections 2024-09-12T02:14:53.111 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.102+0000 7fefbcfff740 1 -- 172.21.15.16:0/1824713668 wait complete. 2024-09-12T02:14:53.111 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.102+0000 7fefbcfff740 1 -- start start 2024-09-12T02:14:53.112 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.102+0000 7fefbcfff740 1 Processor -- start 2024-09-12T02:14:53.112 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.102+0000 7fefbcfff740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5645b0beb9a0 0x5645b0bf5b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:53.112 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.102+0000 7fefbcfff740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5645b0bec2e0 0x5645b0bf60b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:53.112 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.102+0000 7fefbcfff740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5645b0bf1370 0x5645b0bfd0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:53.112 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.102+0000 7fefbcfff740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5645b0bf4340 con 0x5645b0bf1370 2024-09-12T02:14:53.112 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.102+0000 7fefbcfff740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5645b0ad8290 con 0x5645b0beb9a0 2024-09-12T02:14:53.112 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.102+0000 7fefbcfff740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5645b0bf44c0 con 0x5645b0bec2e0 2024-09-12T02:14:53.112 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.103+0000 7fefbcfff740 10 client.0 _ll_register_callbacks cb 0x5645b0b09140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:14:53.112 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.103+0000 7fefb2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5645b0bf1370 0x5645b0bfd0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:53.112 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.103+0000 7fefb2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5645b0bf1370 0x5645b0bfd0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38734/0 (socket says 172.21.15.16:38734) 2024-09-12T02:14:53.112 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.103+0000 7fefb2ffd640 1 -- 172.21.15.16:0/1050525302 learned_addr learned my addr 172.21.15.16:0/1050525302 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:14:53.112 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.103+0000 7fefb2ffd640 1 --2- 172.21.15.16:0/1050525302 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5645b0beb9a0 0x5645b0bf5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:53.113 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.103+0000 7fefb37fe640 1 --2- 172.21.15.16:0/1050525302 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5645b0bec2e0 0x5645b0bf60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:53.113 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.103+0000 7fefb2ffd640 1 -- 172.21.15.16:0/1050525302 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5645b0bec2e0 msgr2=0x5645b0bf60b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:14:53.113 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.103+0000 7fefb2ffd640 1 --2- 172.21.15.16:0/1050525302 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5645b0bec2e0 0x5645b0bf60b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:53.113 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.103+0000 7fefb2ffd640 1 -- 172.21.15.16:0/1050525302 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5645b0beb9a0 msgr2=0x5645b0bf5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:14:53.113 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.103+0000 7fefb2ffd640 1 --2- 172.21.15.16:0/1050525302 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5645b0beb9a0 0x5645b0bf5b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:53.113 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.103+0000 7fefb2ffd640 1 -- 172.21.15.16:0/1050525302 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5645b0862210 con 0x5645b0bf1370 2024-09-12T02:14:53.113 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.103+0000 7fefb37fe640 1 --2- 172.21.15.16:0/1050525302 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5645b0bec2e0 0x5645b0bf60b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:14:53.113 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.103+0000 7fefb2ffd640 1 --2- 172.21.15.16:0/1050525302 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5645b0bf1370 0x5645b0bfd0f0 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7fefa80096c0 tx=0x7fefa8009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9680b28f5c98027 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:14:53.113 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.103+0000 7fef92ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:14:53.113 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.103+0000 7fef92ffd640 1 -- 172.21.15.16:0/1050525302 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefa8035040 con 0x5645b0bf1370 2024-09-12T02:14:53.113 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.103+0000 7fef92ffd640 1 -- 172.21.15.16:0/1050525302 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fefa8005020 con 0x5645b0bf1370 2024-09-12T02:14:53.113 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.103+0000 7fefbcfff740 1 -- 172.21.15.16:0/1050525302 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fefa4004c80 con 0x5645b0bf1370 2024-09-12T02:14:53.113 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.104+0000 7fef92ffd640 1 -- 172.21.15.16:0/1050525302 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fefa8005220 con 0x5645b0bf1370 2024-09-12T02:14:53.114 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.104+0000 7fefbcfff740 10 client.19728 fetch_fsmap learned FSMap version 25 2024-09-12T02:14:53.114 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.104+0000 7fefbcfff740 1 -- 172.21.15.16:0/1050525302 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5645b092e620 con 0x5645b0bf1370 2024-09-12T02:14:53.114 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.104+0000 7fef92ffd640 1 -- 172.21.15.16:0/1050525302 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fefa4004c80 con 0x5645b0bf1370 2024-09-12T02:14:53.114 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.104+0000 7fef92ffd640 1 -- 172.21.15.16:0/1050525302 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fefa803a040 con 0x5645b0bf1370 2024-09-12T02:14:53.114 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.104+0000 7fefbcfff740 10 client.19728 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:14:53.114 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.104+0000 7fefbcfff740 10 client.19728 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:14:53.114 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.104+0000 7fefbcfff740 20 client.19728 populate_metadata read hostname 'smithi016' 2024-09-12T02:14:53.114 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.104+0000 7fefbcfff740 1 --2- 172.21.15.16:0/1050525302 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5645b0c01750 0x5645b0c03bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:53.114 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.104+0000 7fefbcfff740 4 client.19728 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:14:53.114 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.104+0000 7fefbcfff740 1 -- 172.21.15.16:0/1050525302 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5645b0b5ca20 con 0x5645b0c01750 2024-09-12T02:14:53.114 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.104+0000 7fefb37fe640 1 --2- 172.21.15.16:0/1050525302 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5645b0c01750 0x5645b0c03bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:53.114 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.105+0000 7fefb37fe640 1 --2- 172.21.15.16:0/1050525302 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5645b0c01750 0x5645b0c03bc0 crc :-1 s=READY pgs=387 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f2d3d670980cc96 server_cookie=7f7a8b760ecbbec3 in_seq=0 out_seq=0 2024-09-12T02:14:53.114 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.105+0000 7fef92ffd640 10 client.19728 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:14:53.114 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.106+0000 7fef92ffd640 1 -- 172.21.15.16:0/1050525302 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5645b0b5ca20 con 0x5645b0c01750 2024-09-12T02:14:53.115 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.106+0000 7fef92ffd640 10 client.19728 handle_command_reply: tid=0 2024-09-12T02:14:53.115 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.106+0000 7fefbcfff740 1 client.19728 shutdown 2024-09-12T02:14:53.115 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.106+0000 7fefbcfff740 1 -- 172.21.15.16:0/1050525302 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5645b0bf1370 msgr2=0x5645b0bfd0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:14:53.115 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.106+0000 7fefbcfff740 1 --2- 172.21.15.16:0/1050525302 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5645b0bf1370 0x5645b0bfd0f0 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7fefa80096c0 tx=0x7fefa8009260 comp rx=0 tx=0).stop 2024-09-12T02:14:53.115 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.106+0000 7fefbcfff740 1 -- 172.21.15.16:0/1050525302 shutdown_connections 2024-09-12T02:14:53.115 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.106+0000 7fefbcfff740 1 --2- 172.21.15.16:0/1050525302 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5645b0c01750 0x5645b0c03bc0 crc :-1 s=READY pgs=387 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:53.115 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.106+0000 7fefbcfff740 1 --2- 172.21.15.16:0/1050525302 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5645b0bf1370 0x5645b0bfd0f0 unknown :-1 s=CLOSED pgs=1777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:53.115 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.106+0000 7fefbcfff740 1 --2- 172.21.15.16:0/1050525302 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5645b0bec2e0 0x5645b0bf60b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:53.115 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.106+0000 7fefbcfff740 1 --2- 172.21.15.16:0/1050525302 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5645b0beb9a0 0x5645b0bf5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:53.115 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.106+0000 7fefbcfff740 1 -- 172.21.15.16:0/1050525302 >> 172.21.15.16:0/1050525302 conn(0x5645b0be7010 msgr2=0x5645b0be8820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:14:53.115 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.106+0000 7fef92ffd640 0 client.19728 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:14:53.115 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.106+0000 7fefbcfff740 1 -- 172.21.15.16:0/1050525302 shutdown_connections 2024-09-12T02:14:53.115 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.106+0000 7fefbcfff740 1 -- 172.21.15.16:0/1050525302 wait complete. 2024-09-12T02:14:53.116 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.106+0000 7fefbcfff740 20 client.19728 trim_cache size 0 max 16384 2024-09-12T02:14:53.126 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.121+0000 7fefbcfff740 1 Processor -- start 2024-09-12T02:14:53.126 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.121+0000 7fefbcfff740 1 -- start start 2024-09-12T02:14:53.126 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.121+0000 7fefbcfff740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5645b0beb9a0 0x5645b0bfe270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:53.126 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.121+0000 7fefbcfff740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5645b0bec2e0 0x5645b0bfd0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:53.126 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.121+0000 7fefbcfff740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5645b0bf1370 0x5645b0be7ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:53.126 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.121+0000 7fefbcfff740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fefb41ad800 con 0x5645b0beb9a0 2024-09-12T02:14:53.127 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.121+0000 7fefbcfff740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fefb41ab0d0 con 0x5645b0bec2e0 2024-09-12T02:14:53.127 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.121+0000 7fefbcfff740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fefb41a8f20 con 0x5645b0bf1370 2024-09-12T02:14:53.127 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.121+0000 7fefb2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5645b0beb9a0 0x5645b0bfe270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:53.127 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.121+0000 7fefb2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5645b0beb9a0 0x5645b0bfe270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38748/0 (socket says 172.21.15.16:38748) 2024-09-12T02:14:53.127 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.121+0000 7fefb2ffd640 1 -- 172.21.15.16:0/3679219476 learned_addr learned my addr 172.21.15.16:0/3679219476 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:14:53.127 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.121+0000 7fefb2ffd640 1 -- 172.21.15.16:0/3679219476 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5645b0bf1370 msgr2=0x5645b0be7ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:14:53.127 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.121+0000 7fefb2ffd640 1 --2- 172.21.15.16:0/3679219476 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5645b0bf1370 0x5645b0be7ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:53.127 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.121+0000 7fefb2ffd640 1 -- 172.21.15.16:0/3679219476 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5645b0bec2e0 msgr2=0x5645b0bfd0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:14:53.127 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.121+0000 7fefb2ffd640 1 --2- 172.21.15.16:0/3679219476 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5645b0bec2e0 0x5645b0bfd0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:53.127 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.121+0000 7fefb2ffd640 1 -- 172.21.15.16:0/3679219476 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5645b0be22d0 con 0x5645b0beb9a0 2024-09-12T02:14:53.127 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.121+0000 7fefb2ffd640 1 --2- 172.21.15.16:0/3679219476 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5645b0beb9a0 0x5645b0bfe270 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7fefa8033000 tx=0x7fefa8005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41d252cf272d26f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:14:53.128 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.121+0000 7fef997fa640 1 -- 172.21.15.16:0/3679219476 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefa8035070 con 0x5645b0beb9a0 2024-09-12T02:14:53.128 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.122+0000 7fef997fa640 1 -- 172.21.15.16:0/3679219476 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fefa8032da0 con 0x5645b0beb9a0 2024-09-12T02:14:53.128 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.122+0000 7fef997fa640 1 -- 172.21.15.16:0/3679219476 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefa80056b0 con 0x5645b0beb9a0 2024-09-12T02:14:53.128 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.122+0000 7fefbcfff740 1 -- 172.21.15.16:0/3679219476 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5645b0beb9a0 msgr2=0x5645b0bfe270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:14:53.128 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.122+0000 7fefbcfff740 1 --2- 172.21.15.16:0/3679219476 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5645b0beb9a0 0x5645b0bfe270 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7fefa8033000 tx=0x7fefa8005d50 comp rx=0 tx=0).stop 2024-09-12T02:14:53.128 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.122+0000 7fefbcfff740 1 -- 172.21.15.16:0/3679219476 shutdown_connections 2024-09-12T02:14:53.128 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.122+0000 7fefbcfff740 1 --2- 172.21.15.16:0/3679219476 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5645b0bf1370 0x5645b0be7ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:53.128 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.122+0000 7fefbcfff740 1 --2- 172.21.15.16:0/3679219476 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5645b0bec2e0 0x5645b0bfd0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:53.128 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.122+0000 7fefbcfff740 1 --2- 172.21.15.16:0/3679219476 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5645b0beb9a0 0x5645b0bfe270 unknown :-1 s=CLOSED pgs=1778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:53.128 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.122+0000 7fefbcfff740 1 -- 172.21.15.16:0/3679219476 >> 172.21.15.16:0/3679219476 conn(0x5645b0be7010 msgr2=0x5645b0bfdb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:14:53.128 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.122+0000 7fefbcfff740 1 -- 172.21.15.16:0/3679219476 shutdown_connections 2024-09-12T02:14:53.128 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.122+0000 7fefbcfff740 1 -- 172.21.15.16:0/3679219476 wait complete. 2024-09-12T02:14:53.129 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.122+0000 7fefbcfff740 1 -- start start 2024-09-12T02:14:53.129 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.123+0000 7fefbcfff740 1 Processor -- start 2024-09-12T02:14:53.129 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.123+0000 7fefbcfff740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5645b0beb9a0 0x5645b0bfa4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:53.129 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.123+0000 7fefbcfff740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5645b0bec2e0 0x5645b0bfa9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:53.129 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.123+0000 7fefbcfff740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5645b0bf1370 0x5645b0bfb9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:53.129 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.123+0000 7fefbcfff740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fefb41a9590 con 0x5645b0beb9a0 2024-09-12T02:14:53.129 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.123+0000 7fefbcfff740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fefb41aaa00 con 0x5645b0bf1370 2024-09-12T02:14:53.129 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.123+0000 7fefbcfff740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fefb41a9c60 con 0x5645b0bec2e0 2024-09-12T02:14:53.129 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.123+0000 7fefbcfff740 10 client.0 _ll_register_callbacks cb 0x7fefa8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:14:53.129 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.123+0000 7fefb2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5645b0beb9a0 0x5645b0bfa4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:53.129 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.123+0000 7fefb2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5645b0beb9a0 0x5645b0bfa4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38764/0 (socket says 172.21.15.16:38764) 2024-09-12T02:14:53.129 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.123+0000 7fefb2ffd640 1 -- 172.21.15.16:0/2743545185 learned_addr learned my addr 172.21.15.16:0/2743545185 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:14:53.130 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.123+0000 7fefb37fe640 1 --2- 172.21.15.16:0/2743545185 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5645b0bec2e0 0x5645b0bfa9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:53.130 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.123+0000 7fefb2ffd640 1 -- 172.21.15.16:0/2743545185 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5645b0bec2e0 msgr2=0x5645b0bfa9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:14:53.130 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.123+0000 7fefb2ffd640 1 --2- 172.21.15.16:0/2743545185 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5645b0bec2e0 0x5645b0bfa9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:53.130 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.123+0000 7fefb2ffd640 1 -- 172.21.15.16:0/2743545185 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5645b0bf1370 msgr2=0x5645b0bfb9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:14:53.130 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.123+0000 7fefb2ffd640 1 --2- 172.21.15.16:0/2743545185 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5645b0bf1370 0x5645b0bfb9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:53.130 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.123+0000 7fefb2ffd640 1 -- 172.21.15.16:0/2743545185 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5645b0bf1110 con 0x5645b0beb9a0 2024-09-12T02:14:53.130 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.124+0000 7fefb2ffd640 1 --2- 172.21.15.16:0/2743545185 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5645b0beb9a0 0x5645b0bfa4a0 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7fefa8004910 tx=0x7fefa8005660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=222635c1ec8e34c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:14:53.130 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.124+0000 7fef92ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:14:53.130 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.124+0000 7fef92ffd640 1 -- 172.21.15.16:0/2743545185 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefa805d060 con 0x5645b0beb9a0 2024-09-12T02:14:53.130 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.124+0000 7fef92ffd640 1 -- 172.21.15.16:0/2743545185 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fefa8004e70 con 0x5645b0beb9a0 2024-09-12T02:14:53.130 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.124+0000 7fefbcfff740 1 -- 172.21.15.16:0/2743545185 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5645b0b56d70 con 0x5645b0beb9a0 2024-09-12T02:14:53.130 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.124+0000 7fef92ffd640 1 -- 172.21.15.16:0/2743545185 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fefa8035040 con 0x5645b0beb9a0 2024-09-12T02:14:53.130 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.124+0000 7fefbcfff740 10 client.19734 fetch_fsmap learned FSMap version 25 2024-09-12T02:14:53.130 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.124+0000 7fefbcfff740 1 -- 172.21.15.16:0/2743545185 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5645b0bfd0f0 con 0x5645b0beb9a0 2024-09-12T02:14:53.131 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.124+0000 7fef92ffd640 1 -- 172.21.15.16:0/2743545185 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5645b0b56d70 con 0x5645b0beb9a0 2024-09-12T02:14:53.131 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.124+0000 7fef92ffd640 1 -- 172.21.15.16:0/2743545185 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fefa80587e0 con 0x5645b0beb9a0 2024-09-12T02:14:53.131 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.124+0000 7fefbcfff740 10 client.19734 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:14:53.131 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.124+0000 7fefbcfff740 10 client.19734 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:14:53.131 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.124+0000 7fefbcfff740 20 client.19734 populate_metadata read hostname 'smithi016' 2024-09-12T02:14:53.131 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.124+0000 7fefbcfff740 1 --2- 172.21.15.16:0/2743545185 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5645b0c6c8b0 0x5645b0c6ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:53.131 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.125+0000 7fefbcfff740 4 client.19734 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:14:53.131 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.125+0000 7fefbcfff740 1 -- 172.21.15.16:0/2743545185 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5645b0ad3900 con 0x5645b0c6c8b0 2024-09-12T02:14:53.131 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.125+0000 7fefb37fe640 1 --2- 172.21.15.16:0/2743545185 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5645b0c6c8b0 0x5645b0c6ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:53.131 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.125+0000 7fefb37fe640 1 --2- 172.21.15.16:0/2743545185 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5645b0c6c8b0 0x5645b0c6ecd0 crc :-1 s=READY pgs=388 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=93029d945566ecc4 server_cookie=f001f02f5e7b848e in_seq=0 out_seq=0 2024-09-12T02:14:53.131 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.125+0000 7fef92ffd640 10 client.19734 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:14:53.141 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.140+0000 7fef92ffd640 1 -- 172.21.15.16:0/2743545185 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5645b0ad3900 con 0x5645b0c6c8b0 2024-09-12T02:14:53.141 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.140+0000 7fef92ffd640 10 client.19734 handle_command_reply: tid=0 2024-09-12T02:14:53.141 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.140+0000 7fefbcfff740 1 client.19734 shutdown 2024-09-12T02:14:53.141 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.140+0000 7fefbcfff740 1 -- 172.21.15.16:0/2743545185 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5645b0beb9a0 msgr2=0x5645b0bfa4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:14:53.141 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.140+0000 7fefbcfff740 1 --2- 172.21.15.16:0/2743545185 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5645b0beb9a0 0x5645b0bfa4a0 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7fefa8004910 tx=0x7fefa8005660 comp rx=0 tx=0).stop 2024-09-12T02:14:53.141 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.140+0000 7fefbcfff740 1 -- 172.21.15.16:0/2743545185 shutdown_connections 2024-09-12T02:14:53.141 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.140+0000 7fefbcfff740 1 --2- 172.21.15.16:0/2743545185 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5645b0c6c8b0 0x5645b0c6ecd0 crc :-1 s=READY pgs=388 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:53.141 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.140+0000 7fefbcfff740 1 --2- 172.21.15.16:0/2743545185 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5645b0bf1370 0x5645b0bfb9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:53.141 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.140+0000 7fefbcfff740 1 --2- 172.21.15.16:0/2743545185 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5645b0bec2e0 0x5645b0bfa9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:53.141 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.140+0000 7fefbcfff740 1 --2- 172.21.15.16:0/2743545185 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5645b0beb9a0 0x5645b0bfa4a0 unknown :-1 s=CLOSED pgs=1779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:53.142 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.140+0000 7fef92ffd640 0 client.19734 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:14:53.142 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.140+0000 7fefbcfff740 1 -- 172.21.15.16:0/2743545185 >> 172.21.15.16:0/2743545185 conn(0x5645b0be7010 msgr2=0x5645b0be59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:14:53.142 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.140+0000 7fefbcfff740 1 -- 172.21.15.16:0/2743545185 shutdown_connections 2024-09-12T02:14:53.142 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.140+0000 7fefbcfff740 1 -- 172.21.15.16:0/2743545185 wait complete. 2024-09-12T02:14:53.142 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.140+0000 7fefbcfff740 20 client.19734 trim_cache size 0 max 16384 2024-09-12T02:14:53.142 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-09-12T02:14:53.142 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-09-12T02:14:53.142 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-09-12T02:14:53.142 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-09-12T02:14:53.143 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.141+0000 7fefba18f640 1 -- 172.21.15.16:0/2998304674 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fef9c0a2c30 msgr2=0x7fef9c0a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:14:53.143 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.141+0000 7fefba18f640 1 --2- 172.21.15.16:0/2998304674 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fef9c0a2c30 0x7fef9c0a50f0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fefac0988c0 tx=0x7fefac066f60 comp rx=0 tx=0).stop 2024-09-12T02:14:53.143 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.141+0000 7fefba18f640 1 -- 172.21.15.16:0/2998304674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fefb4103860 msgr2=0x7fefb4101390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:14:53.143 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.141+0000 7fefba18f640 1 --2- 172.21.15.16:0/2998304674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fefb4103860 0x7fefb4101390 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7fefac00cce0 tx=0x7fefac00a620 comp rx=0 tx=0).stop 2024-09-12T02:14:53.143 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.141+0000 7fefba18f640 1 -- 172.21.15.16:0/2998304674 shutdown_connections 2024-09-12T02:14:53.143 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.141+0000 7fefba18f640 1 --2- 172.21.15.16:0/2998304674 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fef9c0a2c30 0x7fef9c0a50f0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:53.144 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.141+0000 7fefba18f640 1 --2- 172.21.15.16:0/2998304674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fefb4103860 0x7fefb4101390 unknown :-1 s=CLOSED pgs=1775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:53.144 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.141+0000 7fefba18f640 1 --2- 172.21.15.16:0/2998304674 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fefb4102ea0 0x7fefb4100e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:53.144 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.141+0000 7fefba18f640 1 --2- 172.21.15.16:0/2998304674 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fefb40ffe30 0x7fefb4102810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:53.144 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.141+0000 7fefba18f640 1 -- 172.21.15.16:0/2998304674 >> 172.21.15.16:0/2998304674 conn(0x7fefb40fb970 msgr2=0x7fefb41060d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:14:53.144 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.141+0000 7fefba18f640 1 -- 172.21.15.16:0/2998304674 shutdown_connections 2024-09-12T02:14:53.144 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:14:53.141+0000 7fefba18f640 1 -- 172.21.15.16:0/2998304674 wait complete. 2024-09-12T02:14:54.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:14:54 smithi184 ceph-mon[35593]: pgmap v724: 97 pgs: 97 active+clean; 2.8 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 14 op/s 2024-09-12T02:14:54.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:14:54 smithi033 ceph-mon[35332]: pgmap v724: 97 pgs: 97 active+clean; 2.8 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 14 op/s 2024-09-12T02:14:54.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:14:54 smithi016 ceph-mon[30947]: pgmap v724: 97 pgs: 97 active+clean; 2.8 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 14 op/s 2024-09-12T02:14:56.173 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T02:14:56.444 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:14:56.566 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:14:56 smithi016 ceph-mon[30947]: pgmap v725: 97 pgs: 97 active+clean; 2.8 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-09-12T02:14:56.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:14:56 smithi184 ceph-mon[35593]: pgmap v725: 97 pgs: 97 active+clean; 2.8 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-09-12T02:14:56.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:14:56 smithi033 ceph-mon[35332]: pgmap v725: 97 pgs: 97 active+clean; 2.8 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-09-12T02:14:57.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.021+0000 7f7c0c34e640 1 -- 172.21.15.16:0/999264482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c04102da0 msgr2=0x7f7c04103180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:14:57.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.021+0000 7f7c0c34e640 1 --2- 172.21.15.16:0/999264482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c04102da0 0x7f7c04103180 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7f7c00009900 tx=0x7f7c0002a480 comp rx=0 tx=0).stop 2024-09-12T02:14:57.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.021+0000 7f7c0c34e640 1 -- 172.21.15.16:0/999264482 shutdown_connections 2024-09-12T02:14:57.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.021+0000 7f7c0c34e640 1 --2- 172.21.15.16:0/999264482 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7c0410c180 0x7f7c0410e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:57.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.021+0000 7f7c0c34e640 1 --2- 172.21.15.16:0/999264482 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c041036c0 0x7f7c0410bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:57.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.021+0000 7f7c0c34e640 1 --2- 172.21.15.16:0/999264482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c04102da0 0x7f7c04103180 unknown :-1 s=CLOSED pgs=1780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:57.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.021+0000 7f7c0c34e640 1 -- 172.21.15.16:0/999264482 >> 172.21.15.16:0/999264482 conn(0x7f7c040fc7a0 msgr2=0x7f7c040febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:14:57.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.021+0000 7f7c0c34e640 1 -- 172.21.15.16:0/999264482 shutdown_connections 2024-09-12T02:14:57.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.021+0000 7f7c0c34e640 1 -- 172.21.15.16:0/999264482 wait complete. 2024-09-12T02:14:57.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.022+0000 7f7c0c34e640 1 Processor -- start 2024-09-12T02:14:57.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.022+0000 7f7c0c34e640 1 -- start start 2024-09-12T02:14:57.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.022+0000 7f7c0c34e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7c04102da0 0x7f7c04198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:57.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.022+0000 7f7c0c34e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c041036c0 0x7f7c041989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:57.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.022+0000 7f7c0c34e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c0410c180 0x7f7c0419cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:57.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.022+0000 7f7c0c34e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7c04111fa0 con 0x7f7c0410c180 2024-09-12T02:14:57.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.022+0000 7f7c0c34e640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7c04111e20 con 0x7f7c041036c0 2024-09-12T02:14:57.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.022+0000 7f7c0c34e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7c04112120 con 0x7f7c04102da0 2024-09-12T02:14:57.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.022+0000 7f7c0a8eb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c0410c180 0x7f7c0419cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:57.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.023+0000 7f7c0a8eb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c0410c180 0x7f7c0419cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38792/0 (socket says 172.21.15.16:38792) 2024-09-12T02:14:57.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.023+0000 7f7c0a8eb640 1 -- 172.21.15.16:0/2840345365 learned_addr learned my addr 172.21.15.16:0/2840345365 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:14:57.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.023+0000 7f7c0a0ea640 1 --2- 172.21.15.16:0/2840345365 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7c04102da0 0x7f7c04198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:57.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.023+0000 7f7c098e9640 1 --2- 172.21.15.16:0/2840345365 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c041036c0 0x7f7c041989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:57.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.023+0000 7f7c0a8eb640 1 -- 172.21.15.16:0/2840345365 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7c04102da0 msgr2=0x7f7c04198480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:14:57.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.023+0000 7f7c0a8eb640 1 --2- 172.21.15.16:0/2840345365 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7c04102da0 0x7f7c04198480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:57.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.023+0000 7f7c0a8eb640 1 -- 172.21.15.16:0/2840345365 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c041036c0 msgr2=0x7f7c041989c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:14:57.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.023+0000 7f7c0a8eb640 1 --2- 172.21.15.16:0/2840345365 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c041036c0 0x7f7c041989c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:57.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.023+0000 7f7c0a8eb640 1 -- 172.21.15.16:0/2840345365 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c0419d430 con 0x7f7c0410c180 2024-09-12T02:14:57.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.023+0000 7f7c0a8eb640 1 --2- 172.21.15.16:0/2840345365 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c0410c180 0x7f7c0419cd50 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7f7bfc00b970 tx=0x7f7bfc00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:14:57.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.023+0000 7f7bfb7fe640 1 -- 172.21.15.16:0/2840345365 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bfc019070 con 0x7f7c0410c180 2024-09-12T02:14:57.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.023+0000 7f7bfb7fe640 1 -- 172.21.15.16:0/2840345365 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7bfc00cb40 con 0x7f7c0410c180 2024-09-12T02:14:57.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.023+0000 7f7c0c34e640 1 -- 172.21.15.16:0/2840345365 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c0419d720 con 0x7f7c0410c180 2024-09-12T02:14:57.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.023+0000 7f7bfb7fe640 1 -- 172.21.15.16:0/2840345365 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bfc007650 con 0x7f7c0410c180 2024-09-12T02:14:57.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.023+0000 7f7c0c34e640 1 -- 172.21.15.16:0/2840345365 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c041a5000 con 0x7f7c0410c180 2024-09-12T02:14:57.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.026+0000 7f7bfb7fe640 1 -- 172.21.15.16:0/2840345365 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f7bfc01c070 con 0x7f7c0410c180 2024-09-12T02:14:57.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.026+0000 7f7c0e727740 1 Processor -- start 2024-09-12T02:14:57.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.026+0000 7f7c0e727740 1 -- start start 2024-09-12T02:14:57.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.027+0000 7f7c0e727740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5563bbb0f4a0 0x5563bbb0f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:57.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.027+0000 7f7c0e727740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563bbb0fdc0 0x5563bbb1a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:57.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.027+0000 7f7c0e727740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5563bbb1a680 0x5563bbb1ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:57.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.027+0000 7f7c0e727740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5563bba752e0 con 0x5563bbb0fdc0 2024-09-12T02:14:57.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.027+0000 7f7c0e727740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5563bba1cce0 con 0x5563bbb1a680 2024-09-12T02:14:57.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.027+0000 7f7c0e727740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5563bba712a0 con 0x5563bbb0f4a0 2024-09-12T02:14:57.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.027+0000 7f7c0a0ea640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563bbb0fdc0 0x5563bbb1a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:57.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.027+0000 7f7bfb7fe640 1 --2- 172.21.15.16:0/2840345365 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f7bf00a2c00 0x7f7bf00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:57.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.027+0000 7f7c0a0ea640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563bbb0fdc0 0x5563bbb1a140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38808/0 (socket says 172.21.15.16:38808) 2024-09-12T02:14:57.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.027+0000 7f7c0a0ea640 1 -- 172.21.15.16:0/3040435452 learned_addr learned my addr 172.21.15.16:0/3040435452 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:14:57.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.027+0000 7f7c098e9640 1 --2- 172.21.15.16:0/3040435452 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5563bbb1a680 0x5563bbb1ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:57.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.027+0000 7f7c098e9640 1 --2- 172.21.15.16:0/3040435452 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5563bbb0f4a0 0x5563bbb0f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:57.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.027+0000 7f7bfb7fe640 1 -- 172.21.15.16:0/2840345365 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f7bfc0ce5a0 con 0x7f7c0410c180 2024-09-12T02:14:57.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.027+0000 7f7c0a8eb640 1 --2- 172.21.15.16:0/2840345365 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f7bf00a2c00 0x7f7bf00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:57.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.027+0000 7f7c098e9640 1 -- 172.21.15.16:0/3040435452 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5563bbb0f4a0 msgr2=0x5563bbb0f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:14:57.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.027+0000 7f7c098e9640 1 --2- 172.21.15.16:0/3040435452 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5563bbb0f4a0 0x5563bbb0f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:57.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.027+0000 7f7c098e9640 1 -- 172.21.15.16:0/3040435452 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563bbb0fdc0 msgr2=0x5563bbb1a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:14:57.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.027+0000 7f7c098e9640 1 --2- 172.21.15.16:0/3040435452 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563bbb0fdc0 0x5563bbb1a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:57.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.027+0000 7f7c098e9640 1 -- 172.21.15.16:0/3040435452 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563bba01440 con 0x5563bbb1a680 2024-09-12T02:14:57.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.027+0000 7f7c0a0ea640 1 --2- 172.21.15.16:0/3040435452 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563bbb0fdc0 0x5563bbb1a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:14:57.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.027+0000 7f7c098e9640 1 --2- 172.21.15.16:0/3040435452 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5563bbb1a680 0x5563bbb1ca70 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f7bf4005d00 tx=0x7f7bf4005c90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6984142a96814843 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:14:57.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.028+0000 7f7c0a8eb640 1 --2- 172.21.15.16:0/2840345365 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f7bf00a2c00 0x7f7bf00a50c0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f7bfc097310 tx=0x7f7bfc009970 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:14:57.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.028+0000 7f7bf8dea640 1 -- 172.21.15.16:0/3040435452 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bf4012d20 con 0x5563bbb1a680 2024-09-12T02:14:57.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.028+0000 7f7bf8dea640 1 -- 172.21.15.16:0/3040435452 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7bf4005020 con 0x5563bbb1a680 2024-09-12T02:14:57.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.028+0000 7f7bf8dea640 1 -- 172.21.15.16:0/3040435452 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bf4004a00 con 0x5563bbb1a680 2024-09-12T02:14:57.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.028+0000 7f7c0e727740 1 -- 172.21.15.16:0/3040435452 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5563bbb1a680 msgr2=0x5563bbb1ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:14:57.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.028+0000 7f7c0e727740 1 --2- 172.21.15.16:0/3040435452 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5563bbb1a680 0x5563bbb1ca70 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f7bf4005d00 tx=0x7f7bf4005c90 comp rx=0 tx=0).stop 2024-09-12T02:14:57.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.028+0000 7f7c0e727740 1 -- 172.21.15.16:0/3040435452 shutdown_connections 2024-09-12T02:14:57.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.028+0000 7f7c0e727740 1 --2- 172.21.15.16:0/3040435452 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5563bbb1a680 0x5563bbb1ca70 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:57.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.028+0000 7f7c0e727740 1 --2- 172.21.15.16:0/3040435452 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563bbb0fdc0 0x5563bbb1a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:57.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.028+0000 7f7c0e727740 1 --2- 172.21.15.16:0/3040435452 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5563bbb0f4a0 0x5563bbb0f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:57.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.028+0000 7f7c0e727740 1 -- 172.21.15.16:0/3040435452 >> 172.21.15.16:0/3040435452 conn(0x5563bbb0ba30 msgr2=0x5563bbb0be10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:14:57.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.029+0000 7f7c0e727740 1 -- 172.21.15.16:0/3040435452 shutdown_connections 2024-09-12T02:14:57.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.029+0000 7f7c0e727740 1 -- 172.21.15.16:0/3040435452 wait complete. 2024-09-12T02:14:57.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.029+0000 7f7c0e727740 1 -- start start 2024-09-12T02:14:57.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.029+0000 7f7c0e727740 1 Processor -- start 2024-09-12T02:14:57.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.029+0000 7f7c0e727740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5563bbb0f4a0 0x5563bbb1a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:57.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.029+0000 7f7c0e727740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5563bbb0fdc0 0x5563bbb17310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:57.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.029+0000 7f7c0e727740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563bbb1a680 0x5563bbb178e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:57.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.029+0000 7f7c0e727740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5563bbb21660 con 0x5563bbb1a680 2024-09-12T02:14:57.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.029+0000 7f7c0e727740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5563bba30050 con 0x5563bbb0fdc0 2024-09-12T02:14:57.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.029+0000 7f7c0e727740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5563bbb217e0 con 0x5563bbb0f4a0 2024-09-12T02:14:57.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.029+0000 7f7c0e727740 10 client.0 _ll_register_callbacks cb 0x5563bba70160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:14:57.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.029+0000 7f7c098e9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563bbb1a680 0x5563bbb178e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:57.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.029+0000 7f7c098e9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563bbb1a680 0x5563bbb178e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38816/0 (socket says 172.21.15.16:38816) 2024-09-12T02:14:57.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.029+0000 7f7c098e9640 1 -- 172.21.15.16:0/1740429923 learned_addr learned my addr 172.21.15.16:0/1740429923 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:14:57.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.029+0000 7f7c0a0ea640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5563bbb0fdc0 0x5563bbb17310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:57.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.030+0000 7f7c098e9640 1 -- 172.21.15.16:0/1740429923 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5563bbb0f4a0 msgr2=0x5563bbb1a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:14:57.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.030+0000 7f7c098e9640 1 --2- 172.21.15.16:0/1740429923 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5563bbb0f4a0 0x5563bbb1a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:57.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.030+0000 7f7c098e9640 1 -- 172.21.15.16:0/1740429923 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5563bbb0fdc0 msgr2=0x5563bbb17310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:14:57.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.030+0000 7f7c098e9640 1 --2- 172.21.15.16:0/1740429923 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5563bbb0fdc0 0x5563bbb17310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:57.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.030+0000 7f7c098e9640 1 -- 172.21.15.16:0/1740429923 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563bb7e63c0 con 0x5563bbb1a680 2024-09-12T02:14:57.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.030+0000 7f7c098e9640 1 --2- 172.21.15.16:0/1740429923 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563bbb1a680 0x5563bbb178e0 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7f7bf400b610 tx=0x7f7bf4005730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a425bd34c2e91f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:14:57.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.030+0000 7f7bedffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:14:57.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.030+0000 7f7bedffb640 1 -- 172.21.15.16:0/1740429923 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bf40048c0 con 0x5563bbb1a680 2024-09-12T02:14:57.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.030+0000 7f7bedffb640 1 -- 172.21.15.16:0/1740429923 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f7bf40051d0 con 0x5563bbb1a680 2024-09-12T02:14:57.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.030+0000 7f7c0e727740 1 -- 172.21.15.16:0/1740429923 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7bf4005020 con 0x5563bbb1a680 2024-09-12T02:14:57.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.030+0000 7f7bedffb640 1 -- 172.21.15.16:0/1740429923 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7bf40053d0 con 0x5563bbb1a680 2024-09-12T02:14:57.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.030+0000 7f7c0e727740 10 client.19746 fetch_fsmap learned FSMap version 25 2024-09-12T02:14:57.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.030+0000 7f7c0e727740 1 -- 172.21.15.16:0/1740429923 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5563bbb2d0a0 con 0x5563bbb1a680 2024-09-12T02:14:57.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.031+0000 7f7bedffb640 1 -- 172.21.15.16:0/1740429923 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7bf4005020 con 0x5563bbb1a680 2024-09-12T02:14:57.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.031+0000 7f7bedffb640 1 -- 172.21.15.16:0/1740429923 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f7bf401e510 con 0x5563bbb1a680 2024-09-12T02:14:57.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.031+0000 7f7c0e727740 10 client.19746 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:14:57.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.031+0000 7f7c0e727740 10 client.19746 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:14:57.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.031+0000 7f7c0e727740 20 client.19746 populate_metadata read hostname 'smithi016' 2024-09-12T02:14:57.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.031+0000 7f7c0e727740 1 --2- 172.21.15.16:0/1740429923 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5563bbb2e3f0 0x5563bbb307e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:57.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.031+0000 7f7c0e727740 4 client.19746 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:14:57.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.031+0000 7f7c0e727740 1 -- 172.21.15.16:0/1740429923 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5563bba82850 con 0x5563bbb2e3f0 2024-09-12T02:14:57.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.031+0000 7f7c0a0ea640 1 --2- 172.21.15.16:0/1740429923 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5563bbb2e3f0 0x5563bbb307e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:57.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.031+0000 7f7c0a0ea640 1 --2- 172.21.15.16:0/1740429923 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5563bbb2e3f0 0x5563bbb307e0 crc :-1 s=READY pgs=389 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b298e0f9db59e24 server_cookie=8a4513f0d12e2e64 in_seq=0 out_seq=0 2024-09-12T02:14:57.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.031+0000 7f7bedffb640 10 client.19746 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:14:57.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.032+0000 7f7bedffb640 1 -- 172.21.15.16:0/1740429923 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5563bba82850 con 0x5563bbb2e3f0 2024-09-12T02:14:57.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.032+0000 7f7bedffb640 10 client.19746 handle_command_reply: tid=0 2024-09-12T02:14:57.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.032+0000 7f7c0e727740 1 client.19746 shutdown 2024-09-12T02:14:57.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.032+0000 7f7c0e727740 1 -- 172.21.15.16:0/1740429923 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563bbb1a680 msgr2=0x5563bbb178e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:14:57.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.032+0000 7f7c0e727740 1 --2- 172.21.15.16:0/1740429923 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563bbb1a680 0x5563bbb178e0 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7f7bf400b610 tx=0x7f7bf4005730 comp rx=0 tx=0).stop 2024-09-12T02:14:57.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.032+0000 7f7c0e727740 1 -- 172.21.15.16:0/1740429923 shutdown_connections 2024-09-12T02:14:57.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.032+0000 7f7c0e727740 1 --2- 172.21.15.16:0/1740429923 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5563bbb2e3f0 0x5563bbb307e0 crc :-1 s=READY pgs=389 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:57.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.032+0000 7f7c0e727740 1 --2- 172.21.15.16:0/1740429923 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563bbb1a680 0x5563bbb178e0 unknown :-1 s=CLOSED pgs=1782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:57.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.032+0000 7f7c0e727740 1 --2- 172.21.15.16:0/1740429923 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5563bbb0fdc0 0x5563bbb17310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:57.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.032+0000 7f7c0e727740 1 --2- 172.21.15.16:0/1740429923 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5563bbb0f4a0 0x5563bbb1a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:57.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.032+0000 7f7c0e727740 1 -- 172.21.15.16:0/1740429923 >> 172.21.15.16:0/1740429923 conn(0x5563bbb0ba30 msgr2=0x5563bbb11760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:14:57.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.032+0000 7f7bedffb640 0 client.19746 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:14:57.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.032+0000 7f7c0e727740 1 -- 172.21.15.16:0/1740429923 shutdown_connections 2024-09-12T02:14:57.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.032+0000 7f7c0e727740 1 -- 172.21.15.16:0/1740429923 wait complete. 2024-09-12T02:14:57.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.033+0000 7f7c0e727740 20 client.19746 trim_cache size 0 max 16384 2024-09-12T02:14:57.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.050+0000 7f7c0e727740 1 Processor -- start 2024-09-12T02:14:57.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.050+0000 7f7c0e727740 1 -- start start 2024-09-12T02:14:57.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.050+0000 7f7c0e727740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5563bbb0f4a0 0x5563bbb165b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:57.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.050+0000 7f7c0e727740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5563bbb0fdc0 0x5563bbb2a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:57.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.050+0000 7f7c0e727740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563bbb1a680 0x5563bbb18b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:57.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.050+0000 7f7c0e727740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7c041adaf0 con 0x5563bbb1a680 2024-09-12T02:14:57.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.050+0000 7f7c0e727740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7c041ab3c0 con 0x5563bbb0fdc0 2024-09-12T02:14:57.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.050+0000 7f7c0e727740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7c041a9210 con 0x5563bbb0f4a0 2024-09-12T02:14:57.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.050+0000 7f7c098e9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563bbb1a680 0x5563bbb18b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:57.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.050+0000 7f7c098e9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563bbb1a680 0x5563bbb18b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38828/0 (socket says 172.21.15.16:38828) 2024-09-12T02:14:57.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.050+0000 7f7c098e9640 1 -- 172.21.15.16:0/2609367069 learned_addr learned my addr 172.21.15.16:0/2609367069 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:14:57.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.050+0000 7f7c0a0ea640 1 --2- 172.21.15.16:0/2609367069 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5563bbb0fdc0 0x5563bbb2a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:57.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.050+0000 7f7c098e9640 1 --2- 172.21.15.16:0/2609367069 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5563bbb0f4a0 0x5563bbb165b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:57.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.050+0000 7f7c098e9640 1 -- 172.21.15.16:0/2609367069 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5563bbb0f4a0 msgr2=0x5563bbb165b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:14:57.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.050+0000 7f7c098e9640 1 --2- 172.21.15.16:0/2609367069 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5563bbb0f4a0 0x5563bbb165b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:57.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.050+0000 7f7c098e9640 1 -- 172.21.15.16:0/2609367069 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5563bbb0fdc0 msgr2=0x5563bbb2a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:14:57.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.050+0000 7f7c098e9640 1 --2- 172.21.15.16:0/2609367069 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5563bbb0fdc0 0x5563bbb2a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:57.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.050+0000 7f7c098e9640 1 -- 172.21.15.16:0/2609367069 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563bbb19a20 con 0x5563bbb1a680 2024-09-12T02:14:57.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.051+0000 7f7c098e9640 1 --2- 172.21.15.16:0/2609367069 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563bbb1a680 0x5563bbb18b70 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7f7bf4021040 tx=0x7f7bf4020f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e7e2c0de10446fa server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:14:57.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.051+0000 7f7beffff640 1 -- 172.21.15.16:0/2609367069 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bf401eba0 con 0x5563bbb1a680 2024-09-12T02:14:57.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.051+0000 7f7beffff640 1 -- 172.21.15.16:0/2609367069 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7bf401ed40 con 0x5563bbb1a680 2024-09-12T02:14:57.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.051+0000 7f7beffff640 1 -- 172.21.15.16:0/2609367069 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bf4010c70 con 0x5563bbb1a680 2024-09-12T02:14:57.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.051+0000 7f7c0e727740 1 -- 172.21.15.16:0/2609367069 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563bbb1a680 msgr2=0x5563bbb18b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:14:57.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.051+0000 7f7c0e727740 1 --2- 172.21.15.16:0/2609367069 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563bbb1a680 0x5563bbb18b70 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7f7bf4021040 tx=0x7f7bf4020f90 comp rx=0 tx=0).stop 2024-09-12T02:14:57.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.051+0000 7f7c0e727740 1 -- 172.21.15.16:0/2609367069 shutdown_connections 2024-09-12T02:14:57.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.051+0000 7f7c0e727740 1 --2- 172.21.15.16:0/2609367069 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563bbb1a680 0x5563bbb18b70 unknown :-1 s=CLOSED pgs=1783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:57.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.051+0000 7f7c0e727740 1 --2- 172.21.15.16:0/2609367069 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5563bbb0fdc0 0x5563bbb2a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:57.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.051+0000 7f7c0e727740 1 --2- 172.21.15.16:0/2609367069 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5563bbb0f4a0 0x5563bbb165b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:57.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.051+0000 7f7c0e727740 1 -- 172.21.15.16:0/2609367069 >> 172.21.15.16:0/2609367069 conn(0x5563bbb0ba30 msgr2=0x5563bbb128b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:14:57.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.051+0000 7f7c0e727740 1 -- 172.21.15.16:0/2609367069 shutdown_connections 2024-09-12T02:14:57.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.052+0000 7f7c0e727740 1 -- 172.21.15.16:0/2609367069 wait complete. 2024-09-12T02:14:57.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.052+0000 7f7c0e727740 1 -- start start 2024-09-12T02:14:57.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.052+0000 7f7c0e727740 1 Processor -- start 2024-09-12T02:14:57.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.052+0000 7f7c0e727740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5563bbb0f4a0 0x5563bbb26fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:57.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.052+0000 7f7c0e727740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5563bbb0fdc0 0x5563bbb222e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:57.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.052+0000 7f7c0e727740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563bbb1a680 0x5563bbb22820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:57.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.052+0000 7f7c0e727740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7c041a9880 con 0x5563bbb1a680 2024-09-12T02:14:57.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.052+0000 7f7c0e727740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7c041aacf0 con 0x5563bbb0fdc0 2024-09-12T02:14:57.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.052+0000 7f7c0e727740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7c041a9f50 con 0x5563bbb0f4a0 2024-09-12T02:14:57.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.052+0000 7f7c0e727740 10 client.0 _ll_register_callbacks cb 0x7f7c00009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:14:57.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.052+0000 7f7c098e9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563bbb1a680 0x5563bbb22820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:57.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.052+0000 7f7c098e9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563bbb1a680 0x5563bbb22820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38844/0 (socket says 172.21.15.16:38844) 2024-09-12T02:14:57.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.052+0000 7f7c098e9640 1 -- 172.21.15.16:0/2927498173 learned_addr learned my addr 172.21.15.16:0/2927498173 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:14:57.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.053+0000 7f7c0a0ea640 1 --2- 172.21.15.16:0/2927498173 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5563bbb0fdc0 0x5563bbb222e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:57.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.053+0000 7f7c098e9640 1 --2- 172.21.15.16:0/2927498173 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5563bbb0f4a0 0x5563bbb26fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:57.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.053+0000 7f7c098e9640 1 -- 172.21.15.16:0/2927498173 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5563bbb0f4a0 msgr2=0x5563bbb26fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:14:57.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.053+0000 7f7c098e9640 1 --2- 172.21.15.16:0/2927498173 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5563bbb0f4a0 0x5563bbb26fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:57.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.053+0000 7f7c098e9640 1 -- 172.21.15.16:0/2927498173 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5563bbb0fdc0 msgr2=0x5563bbb222e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:14:57.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.053+0000 7f7c098e9640 1 --2- 172.21.15.16:0/2927498173 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5563bbb0fdc0 0x5563bbb222e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:57.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.053+0000 7f7c098e9640 1 -- 172.21.15.16:0/2927498173 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563bbb1a3d0 con 0x5563bbb1a680 2024-09-12T02:14:57.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.053+0000 7f7c098e9640 1 --2- 172.21.15.16:0/2927498173 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563bbb1a680 0x5563bbb22820 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7f7bf4010950 tx=0x7f7bf4010850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=291afffeb2cde887 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:14:57.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.053+0000 7f7bedffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:14:57.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.053+0000 7f7bedffb640 1 -- 172.21.15.16:0/2927498173 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bf403e070 con 0x5563bbb1a680 2024-09-12T02:14:57.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.053+0000 7f7bedffb640 1 -- 172.21.15.16:0/2927498173 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f7bf401e3e0 con 0x5563bbb1a680 2024-09-12T02:14:57.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.053+0000 7f7c0e727740 1 -- 172.21.15.16:0/2927498173 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5563bba7cd60 con 0x5563bbb1a680 2024-09-12T02:14:57.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.053+0000 7f7bedffb640 1 -- 172.21.15.16:0/2927498173 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7bf4005630 con 0x5563bbb1a680 2024-09-12T02:14:57.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.053+0000 7f7c0e727740 10 client.19752 fetch_fsmap learned FSMap version 25 2024-09-12T02:14:57.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.053+0000 7f7c0e727740 1 -- 172.21.15.16:0/2927498173 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5563bbb98b40 con 0x5563bbb1a680 2024-09-12T02:14:57.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.054+0000 7f7bedffb640 1 -- 172.21.15.16:0/2927498173 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5563bba7cd60 con 0x5563bbb1a680 2024-09-12T02:14:57.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.054+0000 7f7bedffb640 1 -- 172.21.15.16:0/2927498173 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f7bf4041030 con 0x5563bbb1a680 2024-09-12T02:14:57.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.054+0000 7f7c0e727740 10 client.19752 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:14:57.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.054+0000 7f7c0e727740 10 client.19752 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:14:57.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.054+0000 7f7c0e727740 20 client.19752 populate_metadata read hostname 'smithi016' 2024-09-12T02:14:57.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.054+0000 7f7c0e727740 1 --2- 172.21.15.16:0/2927498173 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5563bbb990a0 0x5563bbb9b490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:57.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.054+0000 7f7c0e727740 4 client.19752 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T02:14:57.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.054+0000 7f7c0e727740 1 -- 172.21.15.16:0/2927498173 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5563bb9f9210 con 0x5563bbb990a0 2024-09-12T02:14:57.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.054+0000 7f7c0a0ea640 1 --2- 172.21.15.16:0/2927498173 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5563bbb990a0 0x5563bbb9b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:57.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.054+0000 7f7bedffb640 10 client.19752 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:14:57.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.054+0000 7f7c0a0ea640 1 --2- 172.21.15.16:0/2927498173 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5563bbb990a0 0x5563bbb9b490 crc :-1 s=READY pgs=390 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83c6955335d2f5b4 server_cookie=49dc1242f6f6e2ba in_seq=0 out_seq=0 2024-09-12T02:14:57.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.055+0000 7f7bedffb640 1 -- 172.21.15.16:0/2927498173 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5563bb9f9210 con 0x5563bbb990a0 2024-09-12T02:14:57.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.055+0000 7f7bedffb640 10 client.19752 handle_command_reply: tid=0 2024-09-12T02:14:57.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.055+0000 7f7c0e727740 1 client.19752 shutdown 2024-09-12T02:14:57.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.055+0000 7f7c0e727740 1 -- 172.21.15.16:0/2927498173 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563bbb1a680 msgr2=0x5563bbb22820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:14:57.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.055+0000 7f7c0e727740 1 --2- 172.21.15.16:0/2927498173 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563bbb1a680 0x5563bbb22820 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7f7bf4010950 tx=0x7f7bf4010850 comp rx=0 tx=0).stop 2024-09-12T02:14:57.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.055+0000 7f7c0e727740 1 -- 172.21.15.16:0/2927498173 shutdown_connections 2024-09-12T02:14:57.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.055+0000 7f7c0e727740 1 --2- 172.21.15.16:0/2927498173 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5563bbb990a0 0x5563bbb9b490 crc :-1 s=READY pgs=390 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:57.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.055+0000 7f7c0e727740 1 --2- 172.21.15.16:0/2927498173 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563bbb1a680 0x5563bbb22820 unknown :-1 s=CLOSED pgs=1784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:57.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.055+0000 7f7c0e727740 1 --2- 172.21.15.16:0/2927498173 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5563bbb0fdc0 0x5563bbb222e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:57.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.055+0000 7f7bedffb640 0 client.19752 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:14:57.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.055+0000 7f7c0e727740 1 --2- 172.21.15.16:0/2927498173 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5563bbb0f4a0 0x5563bbb26fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:57.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.055+0000 7f7c0e727740 1 -- 172.21.15.16:0/2927498173 >> 172.21.15.16:0/2927498173 conn(0x5563bbb0ba30 msgr2=0x5563bbb155a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:14:57.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.055+0000 7f7c0e727740 1 -- 172.21.15.16:0/2927498173 shutdown_connections 2024-09-12T02:14:57.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.055+0000 7f7c0e727740 1 -- 172.21.15.16:0/2927498173 wait complete. 2024-09-12T02:14:57.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.055+0000 7f7c0e727740 20 client.19752 trim_cache size 0 max 16384 2024-09-12T02:14:57.061 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T02:14:57.061 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-09-12T02:14:57.061 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {} 2024-09-12T02:14:57.061 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T02:14:57.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.055+0000 7f7c0c34e640 1 -- 172.21.15.16:0/2840345365 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f7bf00a2c00 msgr2=0x7f7bf00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:14:57.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.055+0000 7f7c0c34e640 1 --2- 172.21.15.16:0/2840345365 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f7bf00a2c00 0x7f7bf00a50c0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f7bfc097310 tx=0x7f7bfc009970 comp rx=0 tx=0).stop 2024-09-12T02:14:57.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.055+0000 7f7c0c34e640 1 -- 172.21.15.16:0/2840345365 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c0410c180 msgr2=0x7f7c0419cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:14:57.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.055+0000 7f7c0c34e640 1 --2- 172.21.15.16:0/2840345365 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c0410c180 0x7f7c0419cd50 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7f7bfc00b970 tx=0x7f7bfc00be30 comp rx=0 tx=0).stop 2024-09-12T02:14:57.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.056+0000 7f7c0c34e640 1 -- 172.21.15.16:0/2840345365 shutdown_connections 2024-09-12T02:14:57.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.056+0000 7f7c0c34e640 1 --2- 172.21.15.16:0/2840345365 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f7bf00a2c00 0x7f7bf00a50c0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:57.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.056+0000 7f7c0c34e640 1 --2- 172.21.15.16:0/2840345365 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c0410c180 0x7f7c0419cd50 unknown :-1 s=CLOSED pgs=1781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:57.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.056+0000 7f7c0c34e640 1 --2- 172.21.15.16:0/2840345365 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c041036c0 0x7f7c041989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:57.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.056+0000 7f7c0c34e640 1 --2- 172.21.15.16:0/2840345365 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7c04102da0 0x7f7c04198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:57.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.056+0000 7f7c0c34e640 1 -- 172.21.15.16:0/2840345365 >> 172.21.15.16:0/2840345365 conn(0x7f7c040fc7a0 msgr2=0x7f7c040fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:14:57.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.056+0000 7f7c0c34e640 1 -- 172.21.15.16:0/2840345365 shutdown_connections 2024-09-12T02:14:57.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:57.056+0000 7f7c0c34e640 1 -- 172.21.15.16:0/2840345365 wait complete. 2024-09-12T02:14:57.917 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-12T02:14:57.918 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:14:58.174 INFO:tasks.quiescer.fs.[cephfs]:Total duration (60) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-12T02:14:58.174 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-09-12T02:14:58.174 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:14:58.179 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:14:58.537 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:14:58 smithi016 ceph-mon[30947]: pgmap v726: 97 pgs: 97 active+clean; 2.9 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-09-12T02:14:58.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:58.619+0000 7effcb55d640 1 -- 172.21.15.16:0/2565266628 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7effc41008d0 msgr2=0x7effc4100cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:14:58.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:58.619+0000 7effcb55d640 1 --2- 172.21.15.16:0/2565266628 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7effc41008d0 0x7effc4100cb0 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7effc0009900 tx=0x7effc002a3f0 comp rx=0 tx=0).stop 2024-09-12T02:14:58.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:58.619+0000 7effcb55d640 1 -- 172.21.15.16:0/2565266628 shutdown_connections 2024-09-12T02:14:58.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:58.619+0000 7effcb55d640 1 --2- 172.21.15.16:0/2565266628 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7effc410e480 0x7effc4110870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:58.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:58.619+0000 7effcb55d640 1 --2- 172.21.15.16:0/2565266628 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7effc4101280 0x7effc410df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:58.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:58.619+0000 7effcb55d640 1 --2- 172.21.15.16:0/2565266628 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7effc41008d0 0x7effc4100cb0 unknown :-1 s=CLOSED pgs=1785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:58.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:58.619+0000 7effcb55d640 1 -- 172.21.15.16:0/2565266628 >> 172.21.15.16:0/2565266628 conn(0x7effc40fc7a0 msgr2=0x7effc40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:14:58.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:58.620+0000 7effcb55d640 1 -- 172.21.15.16:0/2565266628 shutdown_connections 2024-09-12T02:14:58.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:58.620+0000 7effcb55d640 1 -- 172.21.15.16:0/2565266628 wait complete. 2024-09-12T02:14:58.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:58.620+0000 7effcb55d640 1 Processor -- start 2024-09-12T02:14:58.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:58.620+0000 7effcb55d640 1 -- start start 2024-09-12T02:14:58.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:58.620+0000 7effcb55d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7effc41008d0 0x7effc419a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:58.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:58.620+0000 7effcb55d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7effc4101280 0x7effc419ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:58.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:58.620+0000 7effcb55d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7effc410e480 0x7effc419ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:58.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:58.620+0000 7effcb55d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7effc4114190 con 0x7effc41008d0 2024-09-12T02:14:58.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:58.620+0000 7effcb55d640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7effc4114010 con 0x7effc410e480 2024-09-12T02:14:58.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:58.620+0000 7effcb55d640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7effc4114310 con 0x7effc4101280 2024-09-12T02:14:58.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:58.621+0000 7effc92f9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7effc41008d0 0x7effc419a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:58.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:58.621+0000 7effc92f9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7effc41008d0 0x7effc419a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38854/0 (socket says 172.21.15.16:38854) 2024-09-12T02:14:58.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:58.621+0000 7effc92f9640 1 -- 172.21.15.16:0/498181714 learned_addr learned my addr 172.21.15.16:0/498181714 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:14:58.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:58.621+0000 7effc9afa640 1 --2- 172.21.15.16:0/498181714 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7effc410e480 0x7effc419ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:58.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:58.621+0000 7effc8af8640 1 --2- 172.21.15.16:0/498181714 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7effc4101280 0x7effc419ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:58.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:58.621+0000 7effc92f9640 1 -- 172.21.15.16:0/498181714 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7effc4101280 msgr2=0x7effc419ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:14:58.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:58.621+0000 7effc92f9640 1 --2- 172.21.15.16:0/498181714 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7effc4101280 0x7effc419ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:58.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:58.621+0000 7effc92f9640 1 -- 172.21.15.16:0/498181714 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7effc410e480 msgr2=0x7effc419ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:14:58.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:58.621+0000 7effc92f9640 1 --2- 172.21.15.16:0/498181714 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7effc410e480 0x7effc419ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:58.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:58.621+0000 7effc92f9640 1 -- 172.21.15.16:0/498181714 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effc419f5e0 con 0x7effc41008d0 2024-09-12T02:14:58.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:58.621+0000 7effc8af8640 1 --2- 172.21.15.16:0/498181714 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7effc4101280 0x7effc419ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:14:58.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:58.621+0000 7effc92f9640 1 --2- 172.21.15.16:0/498181714 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7effc41008d0 0x7effc419a630 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7effc0006010 tx=0x7effc00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:14:58.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:58.621+0000 7effba7fc640 1 -- 172.21.15.16:0/498181714 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effc003f030 con 0x7effc41008d0 2024-09-12T02:14:58.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:58.621+0000 7effcb55d640 1 -- 172.21.15.16:0/498181714 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effc419f870 con 0x7effc41008d0 2024-09-12T02:14:58.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:58.622+0000 7effba7fc640 1 -- 172.21.15.16:0/498181714 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7effc0003ce0 con 0x7effc41008d0 2024-09-12T02:14:58.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:58.622+0000 7effba7fc640 1 -- 172.21.15.16:0/498181714 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effc0035070 con 0x7effc41008d0 2024-09-12T02:14:58.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:58.622+0000 7effcb55d640 1 -- 172.21.15.16:0/498181714 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effc41a71b0 con 0x7effc41008d0 2024-09-12T02:14:58.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:58.623+0000 7effcb55d640 1 -- 172.21.15.16:0/498181714 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7effc4100cb0 con 0x7effc41008d0 2024-09-12T02:14:58.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:58.624+0000 7effba7fc640 1 -- 172.21.15.16:0/498181714 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7effc0042030 con 0x7effc41008d0 2024-09-12T02:14:58.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:58.625+0000 7effba7fc640 1 --2- 172.21.15.16:0/498181714 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7effb00a2c00 0x7effb00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:58.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:58.625+0000 7effba7fc640 1 -- 172.21.15.16:0/498181714 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7effc00ef870 con 0x7effc41008d0 2024-09-12T02:14:58.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:58.625+0000 7effc8af8640 1 --2- 172.21.15.16:0/498181714 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7effb00a2c00 0x7effb00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:58.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:58.625+0000 7effc8af8640 1 --2- 172.21.15.16:0/498181714 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7effb00a2c00 0x7effb00a50c0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7effb40058c0 tx=0x7effb4005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:14:58.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:14:58 smithi184 ceph-mon[35593]: pgmap v726: 97 pgs: 97 active+clean; 2.9 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-09-12T02:14:58.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:58.628+0000 7effba7fc640 1 -- 172.21.15.16:0/498181714 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7effc00bc930 con 0x7effc41008d0 2024-09-12T02:14:58.658 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:14:58.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:14:58 smithi033 ceph-mon[35332]: pgmap v726: 97 pgs: 97 active+clean; 2.9 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-09-12T02:14:58.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:58.845+0000 7effcb55d640 1 -- 172.21.15.16:0/498181714 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7effc4109610 con 0x7effc41008d0 2024-09-12T02:14:58.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:58.846+0000 7effba7fc640 1 -- 172.21.15.16:0/498181714 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11911 (secure 0 0 0) 0x7effc00bc230 con 0x7effc41008d0 2024-09-12T02:14:58.846 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T02:14:58.847 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":25,"btime":"2024-09-12T02:01:37:192021+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3279123279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3279123279},{"type":"v1","addr":"172.21.15.16:6835","nonce":3279123279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4119452382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4119452382},{"type":"v1","addr":"172.21.15.16:6837","nonce":4119452382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/3263823165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3263823165},{"type":"v1","addr":"172.21.15.16:6839","nonce":3263823165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/1510290732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":1510290732},{"type":"v1","addr":"172.21.15.16:6841","nonce":1510290732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/279674603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":279674603},{"type":"v1","addr":"172.21.15.33:6833","nonce":279674603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3555728028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3555728028},{"type":"v1","addr":"172.21.15.33:6835","nonce":3555728028}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T01:59:56.117608+0000","modified":"2024-09-12T02:01:36.294760+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24482,"mds_1":24481,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6833/1563374106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1563374106},{"type":"v1","addr":"172.21.15.184:6833","nonce":1563374106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6837/1892833868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1892833868},{"type":"v1","addr":"172.21.15.33:6837","nonce":1892833868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/1457679686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1457679686},{"type":"v1","addr":"172.21.15.184:6835","nonce":1457679686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.184:6837/1793393087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":1793393087},{"type":"v1","addr":"172.21.15.184:6837","nonce":1793393087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.184:6839/4203347941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":4203347941},{"type":"v1","addr":"172.21.15.184:6839","nonce":4203347941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.33:6839/4136670204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":4136670204},{"type":"v1","addr":"172.21.15.33:6839","nonce":4136670204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24481,24482,24469]},"id":1}]} 2024-09-12T02:14:58.847 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 25 2024-09-12T02:14:58.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:58.848+0000 7effcb55d640 1 -- 172.21.15.16:0/498181714 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7effb00a2c00 msgr2=0x7effb00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:14:58.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:58.848+0000 7effcb55d640 1 --2- 172.21.15.16:0/498181714 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7effb00a2c00 0x7effb00a50c0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7effb40058c0 tx=0x7effb4005850 comp rx=0 tx=0).stop 2024-09-12T02:14:58.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:58.848+0000 7effcb55d640 1 -- 172.21.15.16:0/498181714 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7effc41008d0 msgr2=0x7effc419a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:14:58.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:58.848+0000 7effcb55d640 1 --2- 172.21.15.16:0/498181714 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7effc41008d0 0x7effc419a630 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7effc0006010 tx=0x7effc00042f0 comp rx=0 tx=0).stop 2024-09-12T02:14:58.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:58.848+0000 7effcb55d640 1 -- 172.21.15.16:0/498181714 shutdown_connections 2024-09-12T02:14:58.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:58.848+0000 7effcb55d640 1 --2- 172.21.15.16:0/498181714 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7effb00a2c00 0x7effb00a50c0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:58.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:58.848+0000 7effcb55d640 1 --2- 172.21.15.16:0/498181714 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7effc410e480 0x7effc419ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:58.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:58.848+0000 7effcb55d640 1 --2- 172.21.15.16:0/498181714 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7effc4101280 0x7effc419ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:58.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:58.848+0000 7effcb55d640 1 --2- 172.21.15.16:0/498181714 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7effc41008d0 0x7effc419a630 unknown :-1 s=CLOSED pgs=1786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:58.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:58.848+0000 7effcb55d640 1 -- 172.21.15.16:0/498181714 >> 172.21.15.16:0/498181714 conn(0x7effc40fc7a0 msgr2=0x7effc40feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:14:58.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:58.848+0000 7effcb55d640 1 -- 172.21.15.16:0/498181714 shutdown_connections 2024-09-12T02:14:58.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:58.848+0000 7effcb55d640 1 -- 172.21.15.16:0/498181714 wait complete. 2024-09-12T02:14:59.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:59.292+0000 7f372d77c640 1 -- 172.21.15.16:0/3929860694 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f372810c170 msgr2=0x7f372810e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:14:59.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:59.292+0000 7f372d77c640 1 --2- 172.21.15.16:0/3929860694 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f372810c170 0x7f372810e610 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7f3720009900 tx=0x7f372002a480 comp rx=0 tx=0).stop 2024-09-12T02:14:59.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:59.292+0000 7f372d77c640 1 -- 172.21.15.16:0/3929860694 shutdown_connections 2024-09-12T02:14:59.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:59.292+0000 7f372d77c640 1 --2- 172.21.15.16:0/3929860694 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f372810c170 0x7f372810e610 unknown :-1 s=CLOSED pgs=1787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:59.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:59.292+0000 7f372d77c640 1 --2- 172.21.15.16:0/3929860694 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3728103660 0x7f372810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:59.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:59.292+0000 7f372d77c640 1 --2- 172.21.15.16:0/3929860694 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3728102d40 0x7f3728103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:59.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:59.292+0000 7f372d77c640 1 -- 172.21.15.16:0/3929860694 >> 172.21.15.16:0/3929860694 conn(0x7f37280fc7a0 msgr2=0x7f37280febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:14:59.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:59.292+0000 7f372d77c640 1 -- 172.21.15.16:0/3929860694 shutdown_connections 2024-09-12T02:14:59.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:59.292+0000 7f372d77c640 1 -- 172.21.15.16:0/3929860694 wait complete. 2024-09-12T02:14:59.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:59.292+0000 7f372d77c640 1 Processor -- start 2024-09-12T02:14:59.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:59.293+0000 7f372d77c640 1 -- start start 2024-09-12T02:14:59.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:59.293+0000 7f372d77c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3728102d40 0x7f3728198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:59.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:59.293+0000 7f372d77c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3728103660 0x7f37281988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:59.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:59.293+0000 7f372d77c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f372810c170 0x7f372819cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:59.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:59.293+0000 7f372d77c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3728111f70 con 0x7f3728102d40 2024-09-12T02:14:59.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:59.293+0000 7f372d77c640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3728111df0 con 0x7f372810c170 2024-09-12T02:14:59.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:59.293+0000 7f372d77c640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f37281120f0 con 0x7f3728103660 2024-09-12T02:14:59.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:59.293+0000 7f3726ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3728102d40 0x7f3728198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:59.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:59.293+0000 7f3726ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3728102d40 0x7f3728198370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38874/0 (socket says 172.21.15.16:38874) 2024-09-12T02:14:59.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:59.293+0000 7f3726ffd640 1 -- 172.21.15.16:0/4217321269 learned_addr learned my addr 172.21.15.16:0/4217321269 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:14:59.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:59.293+0000 7f37277fe640 1 --2- 172.21.15.16:0/4217321269 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f372810c170 0x7f372819cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:59.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:59.293+0000 7f37267fc640 1 --2- 172.21.15.16:0/4217321269 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3728103660 0x7f37281988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:59.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:59.293+0000 7f3726ffd640 1 -- 172.21.15.16:0/4217321269 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3728103660 msgr2=0x7f37281988b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:14:59.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:59.293+0000 7f3726ffd640 1 --2- 172.21.15.16:0/4217321269 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3728103660 0x7f37281988b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:59.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:59.293+0000 7f3726ffd640 1 -- 172.21.15.16:0/4217321269 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f372810c170 msgr2=0x7f372819cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:14:59.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:59.293+0000 7f3726ffd640 1 --2- 172.21.15.16:0/4217321269 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f372810c170 0x7f372819cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:59.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:59.293+0000 7f3726ffd640 1 -- 172.21.15.16:0/4217321269 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f372819d320 con 0x7f3728102d40 2024-09-12T02:14:59.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:59.293+0000 7f37267fc640 1 --2- 172.21.15.16:0/4217321269 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3728103660 0x7f37281988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:14:59.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:59.294+0000 7f3726ffd640 1 --2- 172.21.15.16:0/4217321269 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3728102d40 0x7f3728198370 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7f371800cce0 tx=0x7f371800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:14:59.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:59.294+0000 7f370ffff640 1 -- 172.21.15.16:0/4217321269 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3718004b20 con 0x7f3728102d40 2024-09-12T02:14:59.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:59.294+0000 7f372d77c640 1 -- 172.21.15.16:0/4217321269 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f372819d610 con 0x7f3728102d40 2024-09-12T02:14:59.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:59.294+0000 7f370ffff640 1 -- 172.21.15.16:0/4217321269 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3718004cc0 con 0x7f3728102d40 2024-09-12T02:14:59.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:59.294+0000 7f370ffff640 1 -- 172.21.15.16:0/4217321269 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f371800f7d0 con 0x7f3728102d40 2024-09-12T02:14:59.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:59.294+0000 7f372d77c640 1 -- 172.21.15.16:0/4217321269 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37281a4ef0 con 0x7f3728102d40 2024-09-12T02:14:59.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:59.295+0000 7f372d77c640 1 -- 172.21.15.16:0/4217321269 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3728103120 con 0x7f3728102d40 2024-09-12T02:14:59.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:59.297+0000 7f370ffff640 1 -- 172.21.15.16:0/4217321269 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f371800f970 con 0x7f3728102d40 2024-09-12T02:14:59.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:59.297+0000 7f370ffff640 1 --2- 172.21.15.16:0/4217321269 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f37100a2b80 0x7f37100a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:59.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:59.298+0000 7f37267fc640 1 --2- 172.21.15.16:0/4217321269 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f37100a2b80 0x7f37100a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:59.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:59.298+0000 7f370ffff640 1 -- 172.21.15.16:0/4217321269 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f37180cf430 con 0x7f3728102d40 2024-09-12T02:14:59.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:59.298+0000 7f37267fc640 1 --2- 172.21.15.16:0/4217321269 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f37100a2b80 0x7f37100a5040 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f372810d490 tx=0x7f371c00a5e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:14:59.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:59.301+0000 7f370ffff640 1 -- 172.21.15.16:0/4217321269 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3718014030 con 0x7f3728102d40 2024-09-12T02:14:59.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:59.513+0000 7f372d77c640 1 -- 172.21.15.16:0/4217321269 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f37281996d0 con 0x7f3728102d40 2024-09-12T02:14:59.514 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:14:59 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/498181714' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:14:59.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:59.514+0000 7f370ffff640 1 -- 172.21.15.16:0/4217321269 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11911 (secure 0 0 0) 0x7f371809c430 con 0x7f3728102d40 2024-09-12T02:14:59.515 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T02:14:59.515 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":25,"btime":"2024-09-12T02:01:37:192021+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3279123279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3279123279},{"type":"v1","addr":"172.21.15.16:6835","nonce":3279123279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4119452382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4119452382},{"type":"v1","addr":"172.21.15.16:6837","nonce":4119452382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/3263823165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3263823165},{"type":"v1","addr":"172.21.15.16:6839","nonce":3263823165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/1510290732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":1510290732},{"type":"v1","addr":"172.21.15.16:6841","nonce":1510290732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/279674603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":279674603},{"type":"v1","addr":"172.21.15.33:6833","nonce":279674603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3555728028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3555728028},{"type":"v1","addr":"172.21.15.33:6835","nonce":3555728028}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T01:59:56.117608+0000","modified":"2024-09-12T02:01:36.294760+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24482,"mds_1":24481,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6833/1563374106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1563374106},{"type":"v1","addr":"172.21.15.184:6833","nonce":1563374106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6837/1892833868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1892833868},{"type":"v1","addr":"172.21.15.33:6837","nonce":1892833868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/1457679686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1457679686},{"type":"v1","addr":"172.21.15.184:6835","nonce":1457679686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.184:6837/1793393087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":1793393087},{"type":"v1","addr":"172.21.15.184:6837","nonce":1793393087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.184:6839/4203347941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":4203347941},{"type":"v1","addr":"172.21.15.184:6839","nonce":4203347941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.33:6839/4136670204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":4136670204},{"type":"v1","addr":"172.21.15.33:6839","nonce":4136670204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24481,24482,24469]},"id":1}]} 2024-09-12T02:14:59.515 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 25 2024-09-12T02:14:59.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:59.516+0000 7f372d77c640 1 -- 172.21.15.16:0/4217321269 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f37100a2b80 msgr2=0x7f37100a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:14:59.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:59.516+0000 7f372d77c640 1 --2- 172.21.15.16:0/4217321269 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f37100a2b80 0x7f37100a5040 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f372810d490 tx=0x7f371c00a5e0 comp rx=0 tx=0).stop 2024-09-12T02:14:59.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:59.516+0000 7f372d77c640 1 -- 172.21.15.16:0/4217321269 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3728102d40 msgr2=0x7f3728198370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:14:59.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:59.516+0000 7f372d77c640 1 --2- 172.21.15.16:0/4217321269 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3728102d40 0x7f3728198370 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7f371800cce0 tx=0x7f371800a620 comp rx=0 tx=0).stop 2024-09-12T02:14:59.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:59.516+0000 7f372d77c640 1 -- 172.21.15.16:0/4217321269 shutdown_connections 2024-09-12T02:14:59.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:59.516+0000 7f372d77c640 1 --2- 172.21.15.16:0/4217321269 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f37100a2b80 0x7f37100a5040 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:59.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:59.516+0000 7f372d77c640 1 --2- 172.21.15.16:0/4217321269 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f372810c170 0x7f372819cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:59.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:59.516+0000 7f372d77c640 1 --2- 172.21.15.16:0/4217321269 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3728103660 0x7f37281988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:59.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:59.516+0000 7f372d77c640 1 --2- 172.21.15.16:0/4217321269 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3728102d40 0x7f3728198370 unknown :-1 s=CLOSED pgs=1788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:59.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:59.516+0000 7f372d77c640 1 -- 172.21.15.16:0/4217321269 >> 172.21.15.16:0/4217321269 conn(0x7f37280fc7a0 msgr2=0x7f37280fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:14:59.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:59.517+0000 7f372d77c640 1 -- 172.21.15.16:0/4217321269 shutdown_connections 2024-09-12T02:14:59.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:14:59.517+0000 7f372d77c640 1 -- 172.21.15.16:0/4217321269 wait complete. 2024-09-12T02:14:59.615 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-12T02:14:59.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:14:59 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/498181714' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:14:59.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:14:59 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/498181714' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:15:00.263 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24482 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0' 2024-09-12T02:15:00.263 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24482 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0 2024-09-12T02:15:00.341 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:15:00.447 INFO:tasks.workunit.client.0.smithi016.stdout:437/600: open fd = 3 2024-09-12T02:15:00.447 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:15:00.447 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:15:00.447 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:15:00.448 INFO:tasks.workunit.client.0.smithi016.stdout:438/600: open fd = 3 2024-09-12T02:15:00.448 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:15:00.448 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:15:00.448 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:15:00.448 INFO:tasks.workunit.client.0.smithi016.stdout:439/600: open fd = 3 2024-09-12T02:15:00.448 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:15:00.448 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:15:00.448 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:15:00.448 INFO:tasks.workunit.client.0.smithi016.stdout:440/600: open fd = 3 2024-09-12T02:15:00.448 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:15:00.448 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:15:00.448 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:15:00.448 INFO:tasks.workunit.client.0.smithi016.stdout:441/600: open fd = 3 2024-09-12T02:15:00.449 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:15:00.449 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:15:00.449 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:15:00.449 INFO:tasks.workunit.client.0.smithi016.stdout:442/600: open fd = 3 2024-09-12T02:15:00.449 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:15:00.449 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:15:00.449 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:15:00.449 INFO:tasks.workunit.client.0.smithi016.stdout:443/600: open fd = 3 2024-09-12T02:15:00.449 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:15:00.449 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:15:00.449 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:15:00.449 INFO:tasks.workunit.client.0.smithi016.stdout:444/600: open fd = 3 2024-09-12T02:15:00.449 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:15:00.450 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:15:00.450 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:15:00.450 INFO:tasks.workunit.client.0.smithi016.stdout:445/600: open fd = 3 2024-09-12T02:15:00.450 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:15:00.450 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:15:00.450 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:15:00.450 INFO:tasks.workunit.client.0.smithi016.stdout:446/600: open fd = 3 2024-09-12T02:15:00.450 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:15:00.450 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:15:00.450 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:15:00.450 INFO:tasks.workunit.client.0.smithi016.stdout:447/600: open fd = 3 2024-09-12T02:15:00.450 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:15:00.450 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:15:00.450 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:15:00.451 INFO:tasks.workunit.client.0.smithi016.stdout:448/600: open fd = 3 2024-09-12T02:15:00.451 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:15:00.451 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:15:00.451 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:15:00.451 INFO:tasks.workunit.client.0.smithi016.stdout:449/600: open fd = 3 2024-09-12T02:15:00.451 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:15:00.451 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:15:00.451 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:15:00.451 INFO:tasks.workunit.client.0.smithi016.stdout:450/600: open fd = 3 2024-09-12T02:15:00.451 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:15:00.451 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:15:00.451 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:15:00.451 INFO:tasks.workunit.client.0.smithi016.stdout:451/600: open fd = 3 2024-09-12T02:15:00.451 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:15:00.452 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:15:00.452 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:15:00.452 INFO:tasks.workunit.client.0.smithi016.stdout:452/600: open fd = 3 2024-09-12T02:15:00.452 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:15:00.452 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:15:00.452 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:15:00.452 INFO:tasks.workunit.client.0.smithi016.stdout:453/600: open fd = 3 2024-09-12T02:15:00.452 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:15:00.452 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:15:00.452 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:15:00.452 INFO:tasks.workunit.client.0.smithi016.stdout:454/600: open fd = 3 2024-09-12T02:15:00.452 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:15:00.452 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:15:00.453 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:15:00.453 INFO:tasks.workunit.client.0.smithi016.stdout:455/600: open fd = 3 2024-09-12T02:15:00.453 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:15:00.453 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:15:00.453 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:15:00.453 INFO:tasks.workunit.client.0.smithi016.stdout:456/600: open fd = 3 2024-09-12T02:15:00.453 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:15:00.453 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:15:00.453 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:15:00.453 INFO:tasks.workunit.client.0.smithi016.stdout:457/600: open fd = 3 2024-09-12T02:15:00.453 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:15:00.453 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:15:00.453 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:15:00.454 INFO:tasks.workunit.client.0.smithi016.stdout:458/600: open fd = 3 2024-09-12T02:15:00.454 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:15:00.454 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:15:00.454 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:15:00.454 INFO:tasks.workunit.client.0.smithi016.stdout:459/600: open fd = 3 2024-09-12T02:15:00.454 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:15:00.454 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:15:00.454 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:15:00.454 INFO:tasks.workunit.client.0.smithi016.stdout:460/600: open fd = 3 2024-09-12T02:15:00.454 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:15:00.454 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:15:00.454 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:15:00.454 INFO:tasks.workunit.client.0.smithi016.stdout:461/600: open fd = 3 2024-09-12T02:15:00.454 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:15:00.455 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:15:00.455 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:15:00.455 INFO:tasks.workunit.client.0.smithi016.stdout:462/600: open fd = 3 2024-09-12T02:15:00.455 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:15:00.455 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:15:00.455 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:15:00.455 INFO:tasks.workunit.client.0.smithi016.stdout:463/600: open fd = 3 2024-09-12T02:15:00.455 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:15:00.455 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:15:00.455 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:15:00.455 INFO:tasks.workunit.client.0.smithi016.stdout:464/600: open fd = 3 2024-09-12T02:15:00.455 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:15:00.455 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:15:00.456 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:15:00.456 INFO:tasks.workunit.client.0.smithi016.stdout:465/600: open fd = 3 2024-09-12T02:15:00.456 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:15:00.456 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:15:00.456 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:15:00.456 INFO:tasks.workunit.client.0.smithi016.stdout:466/600: open fd = 3 2024-09-12T02:15:00.456 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:15:00.456 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:15:00.456 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:15:00.456 INFO:tasks.workunit.client.0.smithi016.stdout:467/600: open fd = 3 2024-09-12T02:15:00.456 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:15:00.456 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:15:00.456 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:15:00.457 INFO:tasks.workunit.client.0.smithi016.stdout:468/600: open fd = 3 2024-09-12T02:15:00.457 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:15:00.457 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:15:00.457 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:15:00.457 INFO:tasks.workunit.client.0.smithi016.stdout:469/600: open fd = 3 2024-09-12T02:15:00.457 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:15:00.457 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:15:00.457 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:15:00.457 INFO:tasks.workunit.client.0.smithi016.stdout:470/600: open fd = 3 2024-09-12T02:15:00.457 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:15:00.457 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:15:00.457 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:15:00.457 INFO:tasks.workunit.client.0.smithi016.stdout:471/600: open fd = 3 2024-09-12T02:15:00.458 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:15:00.458 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:15:00.458 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:15:00.458 INFO:tasks.workunit.client.0.smithi016.stdout:472/600: open fd = 3 2024-09-12T02:15:00.458 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:15:00.458 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:15:00.458 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:15:00.458 INFO:tasks.workunit.client.0.smithi016.stdout:473/600: open fd = 3 2024-09-12T02:15:00.458 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:15:00.458 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:15:00.458 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:15:00.458 INFO:tasks.workunit.client.0.smithi016.stdout:474/600: open fd = 3 2024-09-12T02:15:00.458 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:15:00.458 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:15:00.459 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:15:00.459 INFO:tasks.workunit.client.0.smithi016.stdout:475/600: open fd = 3 2024-09-12T02:15:00.459 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:15:00.459 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:15:00.459 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:15:00.459 INFO:tasks.workunit.client.0.smithi016.stdout:476/600: open fd = 3 2024-09-12T02:15:00.459 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:15:00.459 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:15:00.459 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:15:00.459 INFO:tasks.workunit.client.0.smithi016.stdout:477/600: open fd = 3 2024-09-12T02:15:00.459 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:15:00.459 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:15:00.459 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:15:00.459 INFO:tasks.workunit.client.0.smithi016.stdout:478/600: open fd = 3 2024-09-12T02:15:00.460 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:15:00.460 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:15:00.460 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:15:00.460 INFO:tasks.workunit.client.0.smithi016.stdout:479/600: open fd = 3 2024-09-12T02:15:00.460 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:15:00.460 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:15:00.460 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:15:00.460 INFO:tasks.workunit.client.0.smithi016.stdout:480/600: open fd = 3 2024-09-12T02:15:00.460 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:15:00.460 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:15:00.460 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:15:00.460 INFO:tasks.workunit.client.0.smithi016.stdout:481/600: open fd = 3 2024-09-12T02:15:00.461 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:15:00.461 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:15:00.461 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:15:00.461 INFO:tasks.workunit.client.0.smithi016.stdout:482/600: open fd = 3 2024-09-12T02:15:00.461 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:15:00.461 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:15:00.461 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:15:00.461 INFO:tasks.workunit.client.0.smithi016.stdout:483/600: open fd = 3 2024-09-12T02:15:00.461 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:15:00.461 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:15:00.461 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:15:00.461 INFO:tasks.workunit.client.0.smithi016.stdout:484/600: open fd = 3 2024-09-12T02:15:00.461 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:15:00.461 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:15:00.462 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:15:00.462 INFO:tasks.workunit.client.0.smithi016.stdout:485/600: open fd = 3 2024-09-12T02:15:00.462 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:15:00.462 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:15:00.462 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:15:00.462 INFO:tasks.workunit.client.0.smithi016.stdout:486/600: open fd = 3 2024-09-12T02:15:00.462 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:15:00.462 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:15:00.462 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:15:00.462 INFO:tasks.workunit.client.0.smithi016.stdout:487/600: open fd = 3 2024-09-12T02:15:00.462 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:15:00.462 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:15:00.462 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:15:00.462 INFO:tasks.workunit.client.0.smithi016.stdout:488/600: open fd = 3 2024-09-12T02:15:00.463 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:15:00.463 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:15:00.463 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:15:00.463 INFO:tasks.workunit.client.0.smithi016.stdout:489/600: open fd = 3 2024-09-12T02:15:00.463 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:15:00.463 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:15:00.463 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:15:00.463 INFO:tasks.workunit.client.0.smithi016.stdout:490/600: open fd = 3 2024-09-12T02:15:00.463 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:15:00.463 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:15:00.463 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:15:00.463 INFO:tasks.workunit.client.0.smithi016.stdout:491/600: open fd = 3 2024-09-12T02:15:00.463 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:15:00.464 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:15:00.464 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:15:00.464 INFO:tasks.workunit.client.0.smithi016.stdout:492/600: open fd = 3 2024-09-12T02:15:00.464 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:15:00.464 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:15:00.464 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:15:00.464 INFO:tasks.workunit.client.0.smithi016.stdout:493/600: open fd = 3 2024-09-12T02:15:00.464 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:15:00.464 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:15:00.464 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:15:00.464 INFO:tasks.workunit.client.0.smithi016.stdout:494/600: open fd = 3 2024-09-12T02:15:00.464 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:15:00.464 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:15:00.464 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:15:00.465 INFO:tasks.workunit.client.0.smithi016.stdout:495/600: open fd = 3 2024-09-12T02:15:00.465 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:15:00.465 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:15:00.465 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:15:00.465 INFO:tasks.workunit.client.0.smithi016.stdout:496/600: open fd = 3 2024-09-12T02:15:00.465 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:15:00.465 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:15:00.465 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:15:00.465 INFO:tasks.workunit.client.0.smithi016.stdout:497/600: open fd = 3 2024-09-12T02:15:00.465 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:15:00.465 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:15:00.465 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:15:00.465 INFO:tasks.workunit.client.0.smithi016.stdout:498/600: open fd = 3 2024-09-12T02:15:00.465 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:15:00.466 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:15:00.466 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:15:00.572 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:15:00 smithi016 ceph-mon[30947]: pgmap v727: 97 pgs: 97 active+clean; 2.9 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-09-12T02:15:00.572 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:15:00 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/4217321269' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:15:00.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:15:00 smithi184 ceph-mon[35593]: pgmap v727: 97 pgs: 97 active+clean; 2.9 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-09-12T02:15:00.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:15:00 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/4217321269' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:15:00.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:15:00 smithi033 ceph-mon[35332]: pgmap v727: 97 pgs: 97 active+clean; 2.9 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-09-12T02:15:00.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:15:00 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/4217321269' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:15:00.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.822+0000 7fa2a82d3640 1 -- 172.21.15.16:0/1748166856 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2a010e450 msgr2=0x7fa2a0110840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:00.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.822+0000 7fa2a82d3640 1 --2- 172.21.15.16:0/1748166856 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2a010e450 0x7fa2a0110840 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7fa298009ca0 tx=0x7fa29802a950 comp rx=0 tx=0).stop 2024-09-12T02:15:00.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.822+0000 7fa2a82d3640 1 -- 172.21.15.16:0/1748166856 shutdown_connections 2024-09-12T02:15:00.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.822+0000 7fa2a82d3640 1 --2- 172.21.15.16:0/1748166856 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2a010e450 0x7fa2a0110840 unknown :-1 s=CLOSED pgs=1789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:00.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.822+0000 7fa2a82d3640 1 --2- 172.21.15.16:0/1748166856 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2a0074e30 0x7fa2a010df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:00.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.822+0000 7fa2a82d3640 1 --2- 172.21.15.16:0/1748166856 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2a0074510 0x7fa2a00748f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:00.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.822+0000 7fa2a82d3640 1 -- 172.21.15.16:0/1748166856 >> 172.21.15.16:0/1748166856 conn(0x7fa2a00fd8f0 msgr2=0x7fa2a00ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:15:00.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.822+0000 7fa2a82d3640 1 -- 172.21.15.16:0/1748166856 shutdown_connections 2024-09-12T02:15:00.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.822+0000 7fa2a82d3640 1 -- 172.21.15.16:0/1748166856 wait complete. 2024-09-12T02:15:00.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.823+0000 7fa2a82d3640 1 Processor -- start 2024-09-12T02:15:00.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.823+0000 7fa2a82d3640 1 -- start start 2024-09-12T02:15:00.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.823+0000 7fa2a82d3640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2a0074510 0x7fa2a019a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:00.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.823+0000 7fa2a82d3640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2a0074e30 0x7fa2a019abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:00.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.823+0000 7fa2a82d3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2a010e450 0x7fa2a019ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:00.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.823+0000 7fa2a82d3640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a01141c0 con 0x7fa2a010e450 2024-09-12T02:15:00.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.823+0000 7fa2a82d3640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a0114040 con 0x7fa2a0074e30 2024-09-12T02:15:00.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.823+0000 7fa2a82d3640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a0114340 con 0x7fa2a0074510 2024-09-12T02:15:00.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.824+0000 7fa2a6870640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2a010e450 0x7fa2a019ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:00.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.824+0000 7fa2a586e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2a0074e30 0x7fa2a019abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:00.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.824+0000 7fa2a6870640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2a010e450 0x7fa2a019ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38890/0 (socket says 172.21.15.16:38890) 2024-09-12T02:15:00.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.824+0000 7fa2a6870640 1 -- 172.21.15.16:0/4128548132 learned_addr learned my addr 172.21.15.16:0/4128548132 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:15:00.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.824+0000 7fa2a606f640 1 --2- 172.21.15.16:0/4128548132 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2a0074510 0x7fa2a019a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:00.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.824+0000 7fa2a6870640 1 -- 172.21.15.16:0/4128548132 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2a0074510 msgr2=0x7fa2a019a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:00.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.824+0000 7fa2a6870640 1 --2- 172.21.15.16:0/4128548132 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2a0074510 0x7fa2a019a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:00.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.824+0000 7fa2a6870640 1 -- 172.21.15.16:0/4128548132 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2a0074e30 msgr2=0x7fa2a019abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:00.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.824+0000 7fa2a6870640 1 --2- 172.21.15.16:0/4128548132 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2a0074e30 0x7fa2a019abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:00.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.824+0000 7fa2a6870640 1 -- 172.21.15.16:0/4128548132 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2a019f640 con 0x7fa2a010e450 2024-09-12T02:15:00.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.824+0000 7fa2a586e640 1 --2- 172.21.15.16:0/4128548132 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2a0074e30 0x7fa2a019abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:15:00.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.824+0000 7fa2a6870640 1 --2- 172.21.15.16:0/4128548132 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2a010e450 0x7fa2a019ef60 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7fa29802ae70 tx=0x7fa29802f5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:15:00.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.825+0000 7fa2977fe640 1 -- 172.21.15.16:0/4128548132 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa298034040 con 0x7fa2a010e450 2024-09-12T02:15:00.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.825+0000 7fa2a82d3640 1 -- 172.21.15.16:0/4128548132 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2a019f8d0 con 0x7fa2a010e450 2024-09-12T02:15:00.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.825+0000 7fa2977fe640 1 -- 172.21.15.16:0/4128548132 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa298030920 con 0x7fa2a010e450 2024-09-12T02:15:00.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.825+0000 7fa2977fe640 1 -- 172.21.15.16:0/4128548132 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa298004780 con 0x7fa2a010e450 2024-09-12T02:15:00.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.825+0000 7fa2a82d3640 1 -- 172.21.15.16:0/4128548132 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2a01a7210 con 0x7fa2a010e450 2024-09-12T02:15:00.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.827+0000 7fa2977fe640 1 -- 172.21.15.16:0/4128548132 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa298037020 con 0x7fa2a010e450 2024-09-12T02:15:00.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.828+0000 7fa2977fe640 1 --2- 172.21.15.16:0/4128548132 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fa28c0a2c00 0x7fa28c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:00.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.828+0000 7fa2aa6ac740 1 Processor -- start 2024-09-12T02:15:00.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.829+0000 7fa2977fe640 1 -- 172.21.15.16:0/4128548132 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fa2980efd80 con 0x7fa2a010e450 2024-09-12T02:15:00.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.829+0000 7fa2a606f640 1 --2- 172.21.15.16:0/4128548132 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fa28c0a2c00 0x7fa28c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:00.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.829+0000 7fa2aa6ac740 1 -- start start 2024-09-12T02:15:00.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.829+0000 7fa2aa6ac740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5607fac3fb40 0x5607fac3e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:00.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.829+0000 7fa2aa6ac740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5607fac3e5d0 0x5607fac3ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:00.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.829+0000 7fa2aa6ac740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5607fac51b80 0x5607fac53f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:00.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.829+0000 7fa2aa6ac740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5607faba42f0 con 0x5607fac3e5d0 2024-09-12T02:15:00.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.829+0000 7fa2aa6ac740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5607fab4bcf0 con 0x5607fac3fb40 2024-09-12T02:15:00.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.829+0000 7fa2aa6ac740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5607faba02b0 con 0x5607fac51b80 2024-09-12T02:15:00.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.829+0000 7fa2a586e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5607fac3e5d0 0x5607fac3ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:00.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.829+0000 7fa2a586e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5607fac3e5d0 0x5607fac3ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38900/0 (socket says 172.21.15.16:38900) 2024-09-12T02:15:00.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.829+0000 7fa2a586e640 1 -- 172.21.15.16:0/2710100411 learned_addr learned my addr 172.21.15.16:0/2710100411 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:15:00.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.830+0000 7fa2a6870640 1 --2- 172.21.15.16:0/2710100411 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5607fac51b80 0x5607fac53f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:00.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.830+0000 7fa2a606f640 1 --2- 172.21.15.16:0/4128548132 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fa28c0a2c00 0x7fa28c0a50c0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fa29c00bbe0 tx=0x7fa29c00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:15:00.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.830+0000 7fa2a586e640 1 -- 172.21.15.16:0/2710100411 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5607fac51b80 msgr2=0x5607fac53f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:15:00.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.830+0000 7fa2a586e640 1 --2- 172.21.15.16:0/2710100411 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5607fac51b80 0x5607fac53f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:00.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.830+0000 7fa2a586e640 1 -- 172.21.15.16:0/2710100411 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5607fac3fb40 msgr2=0x5607fac3e090 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T02:15:00.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.830+0000 7fa2a586e640 1 --2- 172.21.15.16:0/2710100411 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5607fac3fb40 0x5607fac3e090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:00.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.830+0000 7fa2a586e640 1 -- 172.21.15.16:0/2710100411 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607fab303a0 con 0x5607fac3e5d0 2024-09-12T02:15:00.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.830+0000 7fa2a586e640 1 --2- 172.21.15.16:0/2710100411 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5607fac3e5d0 0x5607fac3ea10 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7fa290009e10 tx=0x7fa29000e5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d9d2104d75270a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:15:00.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.830+0000 7fa294dea640 1 -- 172.21.15.16:0/2710100411 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa290017070 con 0x5607fac3e5d0 2024-09-12T02:15:00.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.830+0000 7fa294dea640 1 -- 172.21.15.16:0/2710100411 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa290013950 con 0x5607fac3e5d0 2024-09-12T02:15:00.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.830+0000 7fa294dea640 1 -- 172.21.15.16:0/2710100411 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2900129a0 con 0x5607fac3e5d0 2024-09-12T02:15:00.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.831+0000 7fa2aa6ac740 1 -- 172.21.15.16:0/2710100411 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5607fac3e5d0 msgr2=0x5607fac3ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:00.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.831+0000 7fa2aa6ac740 1 --2- 172.21.15.16:0/2710100411 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5607fac3e5d0 0x5607fac3ea10 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7fa290009e10 tx=0x7fa29000e5a0 comp rx=0 tx=0).stop 2024-09-12T02:15:00.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.831+0000 7fa2aa6ac740 1 -- 172.21.15.16:0/2710100411 shutdown_connections 2024-09-12T02:15:00.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.831+0000 7fa2aa6ac740 1 --2- 172.21.15.16:0/2710100411 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5607fac51b80 0x5607fac53f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:00.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.831+0000 7fa2aa6ac740 1 --2- 172.21.15.16:0/2710100411 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5607fac3e5d0 0x5607fac3ea10 unknown :-1 s=CLOSED pgs=1791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:00.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.831+0000 7fa2aa6ac740 1 --2- 172.21.15.16:0/2710100411 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5607fac3fb40 0x5607fac3e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:00.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.831+0000 7fa2aa6ac740 1 -- 172.21.15.16:0/2710100411 >> 172.21.15.16:0/2710100411 conn(0x5607fac07310 msgr2=0x5607fac3ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:15:00.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.832+0000 7fa2aa6ac740 1 -- 172.21.15.16:0/2710100411 shutdown_connections 2024-09-12T02:15:00.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.832+0000 7fa2aa6ac740 1 -- 172.21.15.16:0/2710100411 wait complete. 2024-09-12T02:15:00.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.832+0000 7fa2aa6ac740 1 -- start start 2024-09-12T02:15:00.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.832+0000 7fa2aa6ac740 1 Processor -- start 2024-09-12T02:15:00.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.833+0000 7fa2aa6ac740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5607fac3e5d0 0x5607fac51610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:00.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.833+0000 7fa2aa6ac740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5607fac3fb40 0x5607fac4e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:00.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.833+0000 7fa2aa6ac740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5607fac51b80 0x5607fac4f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:00.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.833+0000 7fa2aa6ac740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5607fac54880 con 0x5607fac51b80 2024-09-12T02:15:00.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.833+0000 7fa2aa6ac740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5607fab5f060 con 0x5607fac3e5d0 2024-09-12T02:15:00.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.833+0000 7fa2aa6ac740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5607fac54a00 con 0x5607fac3fb40 2024-09-12T02:15:00.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.833+0000 7fa2aa6ac740 10 client.0 _ll_register_callbacks cb 0x5607fa8b28c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:15:00.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.833+0000 7fa2a6870640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5607fac51b80 0x5607fac4f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:00.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.833+0000 7fa2a6870640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5607fac51b80 0x5607fac4f050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38908/0 (socket says 172.21.15.16:38908) 2024-09-12T02:15:00.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.833+0000 7fa2a6870640 1 -- 172.21.15.16:0/1505203588 learned_addr learned my addr 172.21.15.16:0/1505203588 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:15:00.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.833+0000 7fa2a606f640 1 --2- 172.21.15.16:0/1505203588 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5607fac3e5d0 0x5607fac51610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:00.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.833+0000 7fa2a606f640 1 -- 172.21.15.16:0/1505203588 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5607fac3fb40 msgr2=0x5607fac4e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:15:00.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.833+0000 7fa2a606f640 1 --2- 172.21.15.16:0/1505203588 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5607fac3fb40 0x5607fac4e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:00.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.833+0000 7fa2a606f640 1 -- 172.21.15.16:0/1505203588 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5607fac51b80 msgr2=0x5607fac4f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:15:00.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.833+0000 7fa2a606f640 1 --2- 172.21.15.16:0/1505203588 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5607fac51b80 0x5607fac4f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:00.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.833+0000 7fa2a606f640 1 -- 172.21.15.16:0/1505203588 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5607fa9153c0 con 0x5607fac3e5d0 2024-09-12T02:15:00.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.833+0000 7fa2a6870640 1 --2- 172.21.15.16:0/1505203588 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5607fac51b80 0x5607fac4f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:15:00.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.834+0000 7fa2a606f640 1 --2- 172.21.15.16:0/1505203588 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5607fac3e5d0 0x5607fac51610 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fa29c010860 tx=0x7fa29c004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d02b9f8dff452f11 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:15:00.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.834+0000 7fa289ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-12T02:15:00.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.834+0000 7fa289ffb640 1 -- 172.21.15.16:0/1505203588 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa29c005220 con 0x5607fac3e5d0 2024-09-12T02:15:00.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.834+0000 7fa2aa6ac740 1 -- 172.21.15.16:0/1505203588 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa290013950 con 0x5607fac3e5d0 2024-09-12T02:15:00.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.834+0000 7fa289ffb640 1 -- 172.21.15.16:0/1505203588 <== mon.2 v2:172.21.15.33:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fa29c0053c0 con 0x5607fac3e5d0 2024-09-12T02:15:00.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.834+0000 7fa289ffb640 1 -- 172.21.15.16:0/1505203588 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa29c0055c0 con 0x5607fac3e5d0 2024-09-12T02:15:00.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.834+0000 7fa2aa6ac740 10 client.28664 fetch_fsmap learned FSMap version 25 2024-09-12T02:15:00.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.834+0000 7fa2aa6ac740 1 -- 172.21.15.16:0/1505203588 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5607fac60480 con 0x5607fac3e5d0 2024-09-12T02:15:00.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.834+0000 7fa289ffb640 1 -- 172.21.15.16:0/1505203588 <== mon.2 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa290013950 con 0x5607fac3e5d0 2024-09-12T02:15:00.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.835+0000 7fa289ffb640 1 -- 172.21.15.16:0/1505203588 <== mon.2 v2:172.21.15.33:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fa29c01ec80 con 0x5607fac3e5d0 2024-09-12T02:15:00.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.835+0000 7fa2aa6ac740 10 client.28664 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:15:00.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.835+0000 7fa2aa6ac740 10 client.28664 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-12T02:15:00.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.835+0000 7fa2aa6ac740 20 client.28664 populate_metadata read hostname 'smithi016' 2024-09-12T02:15:00.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.835+0000 7fa2aa6ac740 1 --2- 172.21.15.16:0/1505203588 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x5607fac61930 0x5607fac63dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:00.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.835+0000 7fa2aa6ac740 4 client.28664 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:15:00.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.835+0000 7fa2aa6ac740 1 -- 172.21.15.16:0/1505203588 --> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5607fabb1860 con 0x5607fac61930 2024-09-12T02:15:00.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.835+0000 7fa2a586e640 1 --2- 172.21.15.16:0/1505203588 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x5607fac61930 0x5607fac63dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:00.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.835+0000 7fa2a586e640 1 --2- 172.21.15.16:0/1505203588 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x5607fac61930 0x5607fac63dd0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1307eb280d9fc6bc server_cookie=10f0f09eb0c6fc58 in_seq=0 out_seq=0 2024-09-12T02:15:00.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.835+0000 7fa289ffb640 10 client.28664 ms_handle_connect on v2:172.21.15.184:6836/1793393087 2024-09-12T02:15:00.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.836+0000 7fa289ffb640 1 -- 172.21.15.16:0/1505203588 <== mds.2 v2:172.21.15.184:6836/1793393087 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5607fabb1860 con 0x5607fac61930 2024-09-12T02:15:00.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.836+0000 7fa289ffb640 10 client.28664 handle_command_reply: tid=0 2024-09-12T02:15:00.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.836+0000 7fa2aa6ac740 1 client.28664 shutdown 2024-09-12T02:15:00.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.836+0000 7fa2aa6ac740 1 -- 172.21.15.16:0/1505203588 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5607fac3e5d0 msgr2=0x5607fac51610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:00.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.836+0000 7fa2aa6ac740 1 --2- 172.21.15.16:0/1505203588 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5607fac3e5d0 0x5607fac51610 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fa29c010860 tx=0x7fa29c004730 comp rx=0 tx=0).stop 2024-09-12T02:15:00.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.836+0000 7fa2aa6ac740 1 -- 172.21.15.16:0/1505203588 shutdown_connections 2024-09-12T02:15:00.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.836+0000 7fa2aa6ac740 1 --2- 172.21.15.16:0/1505203588 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x5607fac61930 0x5607fac63dd0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:00.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.836+0000 7fa2aa6ac740 1 --2- 172.21.15.16:0/1505203588 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5607fac51b80 0x5607fac4f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:00.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.836+0000 7fa2aa6ac740 1 --2- 172.21.15.16:0/1505203588 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5607fac3fb40 0x5607fac4e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:00.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.836+0000 7fa289ffb640 0 client.28664 ms_handle_reset on v2:172.21.15.184:6836/1793393087 2024-09-12T02:15:00.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.836+0000 7fa2aa6ac740 1 --2- 172.21.15.16:0/1505203588 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5607fac3e5d0 0x5607fac51610 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:00.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.836+0000 7fa2aa6ac740 1 -- 172.21.15.16:0/1505203588 >> 172.21.15.16:0/1505203588 conn(0x5607fac07310 msgr2=0x5607fac40960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:15:00.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.836+0000 7fa2aa6ac740 1 -- 172.21.15.16:0/1505203588 shutdown_connections 2024-09-12T02:15:00.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.836+0000 7fa2aa6ac740 1 -- 172.21.15.16:0/1505203588 wait complete. 2024-09-12T02:15:00.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.837+0000 7fa2aa6ac740 20 client.28664 trim_cache size 0 max 16384 2024-09-12T02:15:00.845 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:15:00.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.850+0000 7fa2aa6ac740 1 Processor -- start 2024-09-12T02:15:00.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.850+0000 7fa2aa6ac740 1 -- start start 2024-09-12T02:15:00.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.851+0000 7fa2aa6ac740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5607fac3e5d0 0x5607fac4d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:00.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.851+0000 7fa2aa6ac740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5607fac3fb40 0x5607fac696a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:00.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.851+0000 7fa2aa6ac740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5607fac51b80 0x5607fac69be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:00.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.851+0000 7fa2aa6ac740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a01adae0 con 0x5607fac3e5d0 2024-09-12T02:15:00.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.851+0000 7fa2aa6ac740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a01ab3b0 con 0x5607fac51b80 2024-09-12T02:15:00.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.851+0000 7fa2aa6ac740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a01a9200 con 0x5607fac3fb40 2024-09-12T02:15:00.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.851+0000 7fa2a606f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5607fac3e5d0 0x5607fac4d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:00.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.851+0000 7fa2a606f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5607fac3e5d0 0x5607fac4d8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38922/0 (socket says 172.21.15.16:38922) 2024-09-12T02:15:00.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.851+0000 7fa2a606f640 1 -- 172.21.15.16:0/1173015953 learned_addr learned my addr 172.21.15.16:0/1173015953 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:15:00.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.851+0000 7fa2a6870640 1 --2- 172.21.15.16:0/1173015953 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5607fac51b80 0x5607fac69be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:00.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.851+0000 7fa2a586e640 1 --2- 172.21.15.16:0/1173015953 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5607fac3fb40 0x5607fac696a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:00.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.851+0000 7fa2a606f640 1 -- 172.21.15.16:0/1173015953 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5607fac3fb40 msgr2=0x5607fac696a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:15:00.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.851+0000 7fa2a606f640 1 --2- 172.21.15.16:0/1173015953 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5607fac3fb40 0x5607fac696a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:00.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.851+0000 7fa2a606f640 1 -- 172.21.15.16:0/1173015953 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5607fac51b80 msgr2=0x5607fac69be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:15:00.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.851+0000 7fa2a606f640 1 --2- 172.21.15.16:0/1173015953 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5607fac51b80 0x5607fac69be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:00.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.851+0000 7fa2a606f640 1 -- 172.21.15.16:0/1173015953 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607fac4ce10 con 0x5607fac3e5d0 2024-09-12T02:15:00.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.851+0000 7fa2a6870640 1 --2- 172.21.15.16:0/1173015953 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5607fac51b80 0x5607fac69be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:15:00.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.851+0000 7fa2a586e640 1 --2- 172.21.15.16:0/1173015953 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5607fac3fb40 0x5607fac696a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:15:00.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.852+0000 7fa2a606f640 1 --2- 172.21.15.16:0/1173015953 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5607fac3e5d0 0x5607fac4d8a0 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7fa29c026480 tx=0x7fa29c0058f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6fe899545fe60ad server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:15:00.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.852+0000 7fa28bfff640 1 -- 172.21.15.16:0/1173015953 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa29c01e6f0 con 0x5607fac3e5d0 2024-09-12T02:15:00.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.852+0000 7fa28bfff640 1 -- 172.21.15.16:0/1173015953 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa29c01ed70 con 0x5607fac3e5d0 2024-09-12T02:15:00.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.852+0000 7fa28bfff640 1 -- 172.21.15.16:0/1173015953 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa29c03c420 con 0x5607fac3e5d0 2024-09-12T02:15:00.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.853+0000 7fa2aa6ac740 1 -- 172.21.15.16:0/1173015953 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5607fac3e5d0 msgr2=0x5607fac4d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:00.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.853+0000 7fa2aa6ac740 1 --2- 172.21.15.16:0/1173015953 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5607fac3e5d0 0x5607fac4d8a0 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7fa29c026480 tx=0x7fa29c0058f0 comp rx=0 tx=0).stop 2024-09-12T02:15:00.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.853+0000 7fa2aa6ac740 1 -- 172.21.15.16:0/1173015953 shutdown_connections 2024-09-12T02:15:00.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.853+0000 7fa2aa6ac740 1 --2- 172.21.15.16:0/1173015953 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5607fac51b80 0x5607fac69be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:00.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.853+0000 7fa2aa6ac740 1 --2- 172.21.15.16:0/1173015953 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5607fac3fb40 0x5607fac696a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:00.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.853+0000 7fa2aa6ac740 1 --2- 172.21.15.16:0/1173015953 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5607fac3e5d0 0x5607fac4d8a0 unknown :-1 s=CLOSED pgs=1792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:00.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.853+0000 7fa2aa6ac740 1 -- 172.21.15.16:0/1173015953 >> 172.21.15.16:0/1173015953 conn(0x5607fac07310 msgr2=0x5607fac4d190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:15:00.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.853+0000 7fa2aa6ac740 1 -- 172.21.15.16:0/1173015953 shutdown_connections 2024-09-12T02:15:00.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.853+0000 7fa2aa6ac740 1 -- 172.21.15.16:0/1173015953 wait complete. 2024-09-12T02:15:00.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.853+0000 7fa2aa6ac740 1 -- start start 2024-09-12T02:15:00.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.853+0000 7fa2aa6ac740 1 Processor -- start 2024-09-12T02:15:00.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.854+0000 7fa2aa6ac740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5607fac3e5d0 0x5607fac5a110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:00.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.854+0000 7fa2aa6ac740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5607fac3fb40 0x5607fac4b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:00.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.854+0000 7fa2aa6ac740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5607fac51b80 0x5607fac4b840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:00.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.854+0000 7fa2aa6ac740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a01a9870 con 0x5607fac3e5d0 2024-09-12T02:15:00.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.854+0000 7fa2aa6ac740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a01aace0 con 0x5607fac51b80 2024-09-12T02:15:00.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.854+0000 7fa2aa6ac740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a01a9f40 con 0x5607fac3fb40 2024-09-12T02:15:00.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.854+0000 7fa2aa6ac740 10 client.0 _ll_register_callbacks cb 0x7fa29c00b8f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:15:00.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.854+0000 7fa2a606f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5607fac3e5d0 0x5607fac5a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:00.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.854+0000 7fa2a606f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5607fac3e5d0 0x5607fac5a110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38928/0 (socket says 172.21.15.16:38928) 2024-09-12T02:15:00.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.854+0000 7fa2a606f640 1 -- 172.21.15.16:0/3377663269 learned_addr learned my addr 172.21.15.16:0/3377663269 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:15:00.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.854+0000 7fa2a6870640 1 --2- 172.21.15.16:0/3377663269 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5607fac51b80 0x5607fac4b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:00.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.854+0000 7fa2a586e640 1 --2- 172.21.15.16:0/3377663269 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5607fac3fb40 0x5607fac4b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:00.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.854+0000 7fa2a606f640 1 -- 172.21.15.16:0/3377663269 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5607fac3fb40 msgr2=0x5607fac4b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:15:00.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.854+0000 7fa2a606f640 1 --2- 172.21.15.16:0/3377663269 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5607fac3fb40 0x5607fac4b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:00.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.854+0000 7fa2a606f640 1 -- 172.21.15.16:0/3377663269 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5607fac51b80 msgr2=0x5607fac4b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:15:00.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.854+0000 7fa2a606f640 1 --2- 172.21.15.16:0/3377663269 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5607fac51b80 0x5607fac4b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:00.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.854+0000 7fa2a606f640 1 -- 172.21.15.16:0/3377663269 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5607fac4bd80 con 0x5607fac3e5d0 2024-09-12T02:15:00.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.854+0000 7fa2a6870640 1 --2- 172.21.15.16:0/3377663269 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5607fac51b80 0x5607fac4b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:15:00.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.854+0000 7fa2a606f640 1 --2- 172.21.15.16:0/3377663269 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5607fac3e5d0 0x5607fac5a110 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7fa29c031690 tx=0x7fa29c0316c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f536fa71841046f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:15:00.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.854+0000 7fa289ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:15:00.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.854+0000 7fa289ffb640 1 -- 172.21.15.16:0/3377663269 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa29c026880 con 0x5607fac3e5d0 2024-09-12T02:15:00.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.854+0000 7fa289ffb640 1 -- 172.21.15.16:0/3377663269 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fa29c026a20 con 0x5607fac3e5d0 2024-09-12T02:15:00.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.854+0000 7fa2aa6ac740 1 -- 172.21.15.16:0/3377663269 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa29c01ed70 con 0x5607fac3e5d0 2024-09-12T02:15:00.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.855+0000 7fa289ffb640 1 -- 172.21.15.16:0/3377663269 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa29c04b050 con 0x5607fac3e5d0 2024-09-12T02:15:00.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.855+0000 7fa2aa6ac740 10 client.19782 fetch_fsmap learned FSMap version 25 2024-09-12T02:15:00.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.855+0000 7fa2aa6ac740 1 -- 172.21.15.16:0/3377663269 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5607faccc380 con 0x5607fac3e5d0 2024-09-12T02:15:00.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.855+0000 7fa289ffb640 1 -- 172.21.15.16:0/3377663269 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa29c01ed70 con 0x5607fac3e5d0 2024-09-12T02:15:00.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.855+0000 7fa289ffb640 1 -- 172.21.15.16:0/3377663269 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fa29c03cac0 con 0x5607fac3e5d0 2024-09-12T02:15:00.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.855+0000 7fa2aa6ac740 10 client.19782 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:15:00.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.855+0000 7fa2aa6ac740 10 client.19782 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-12T02:15:00.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.855+0000 7fa2aa6ac740 20 client.19782 populate_metadata read hostname 'smithi016' 2024-09-12T02:15:00.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.855+0000 7fa2aa6ac740 1 --2- 172.21.15.16:0/3377663269 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x5607faccca60 0x5607faccee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:00.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.855+0000 7fa2aa6ac740 4 client.19782 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T02:15:00.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.855+0000 7fa2aa6ac740 1 -- 172.21.15.16:0/3377663269 --> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5607fababd70 con 0x5607faccca60 2024-09-12T02:15:00.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.855+0000 7fa2a586e640 1 --2- 172.21.15.16:0/3377663269 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x5607faccca60 0x5607faccee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:00.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.856+0000 7fa289ffb640 10 client.19782 ms_handle_connect on v2:172.21.15.184:6836/1793393087 2024-09-12T02:15:00.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.856+0000 7fa2a586e640 1 --2- 172.21.15.16:0/3377663269 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x5607faccca60 0x5607faccee80 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4c0a4ab5bf35cd8f server_cookie=ac5cce91508ae88d in_seq=0 out_seq=0 2024-09-12T02:15:00.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.856+0000 7fa289ffb640 1 -- 172.21.15.16:0/3377663269 <== mds.2 v2:172.21.15.184:6836/1793393087 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5607fababd70 con 0x5607faccca60 2024-09-12T02:15:00.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.856+0000 7fa289ffb640 10 client.19782 handle_command_reply: tid=0 2024-09-12T02:15:00.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.856+0000 7fa2aa6ac740 1 client.19782 shutdown 2024-09-12T02:15:00.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.856+0000 7fa2aa6ac740 1 -- 172.21.15.16:0/3377663269 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5607fac3e5d0 msgr2=0x5607fac5a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:00.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.856+0000 7fa2aa6ac740 1 --2- 172.21.15.16:0/3377663269 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5607fac3e5d0 0x5607fac5a110 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7fa29c031690 tx=0x7fa29c0316c0 comp rx=0 tx=0).stop 2024-09-12T02:15:00.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.856+0000 7fa2aa6ac740 1 -- 172.21.15.16:0/3377663269 shutdown_connections 2024-09-12T02:15:00.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.856+0000 7fa2aa6ac740 1 --2- 172.21.15.16:0/3377663269 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5607fac51b80 0x5607fac4b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:00.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.856+0000 7fa2aa6ac740 1 --2- 172.21.15.16:0/3377663269 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5607fac3fb40 0x5607fac4b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:00.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.856+0000 7fa2aa6ac740 1 --2- 172.21.15.16:0/3377663269 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x5607faccca60 0x5607faccee80 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:00.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.856+0000 7fa289ffb640 0 client.19782 ms_handle_reset on v2:172.21.15.184:6836/1793393087 2024-09-12T02:15:00.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.856+0000 7fa2aa6ac740 1 --2- 172.21.15.16:0/3377663269 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5607fac3e5d0 0x5607fac5a110 unknown :-1 s=CLOSED pgs=1793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:00.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.856+0000 7fa2aa6ac740 1 -- 172.21.15.16:0/3377663269 >> 172.21.15.16:0/3377663269 conn(0x5607fac07310 msgr2=0x5607fac4d9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:15:00.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.856+0000 7fa2aa6ac740 1 -- 172.21.15.16:0/3377663269 shutdown_connections 2024-09-12T02:15:00.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.856+0000 7fa2aa6ac740 1 -- 172.21.15.16:0/3377663269 wait complete. 2024-09-12T02:15:00.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.857+0000 7fa2aa6ac740 20 client.19782 trim_cache size 0 max 16384 2024-09-12T02:15:00.862 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T02:15:00.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.857+0000 7fa2a82d3640 1 -- 172.21.15.16:0/4128548132 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fa28c0a2c00 msgr2=0x7fa28c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:00.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.857+0000 7fa2a82d3640 1 --2- 172.21.15.16:0/4128548132 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fa28c0a2c00 0x7fa28c0a50c0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fa29c00bbe0 tx=0x7fa29c00b390 comp rx=0 tx=0).stop 2024-09-12T02:15:00.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.857+0000 7fa2a82d3640 1 -- 172.21.15.16:0/4128548132 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2a010e450 msgr2=0x7fa2a019ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:00.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.857+0000 7fa2a82d3640 1 --2- 172.21.15.16:0/4128548132 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2a010e450 0x7fa2a019ef60 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7fa29802ae70 tx=0x7fa29802f5b0 comp rx=0 tx=0).stop 2024-09-12T02:15:00.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.857+0000 7fa2a82d3640 1 -- 172.21.15.16:0/4128548132 shutdown_connections 2024-09-12T02:15:00.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.857+0000 7fa2a82d3640 1 --2- 172.21.15.16:0/4128548132 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fa28c0a2c00 0x7fa28c0a50c0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:00.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.857+0000 7fa2a82d3640 1 --2- 172.21.15.16:0/4128548132 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2a010e450 0x7fa2a019ef60 unknown :-1 s=CLOSED pgs=1790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:00.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.857+0000 7fa2a82d3640 1 --2- 172.21.15.16:0/4128548132 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2a0074e30 0x7fa2a019abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:00.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.857+0000 7fa2a82d3640 1 --2- 172.21.15.16:0/4128548132 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2a0074510 0x7fa2a019a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:00.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.857+0000 7fa2a82d3640 1 -- 172.21.15.16:0/4128548132 >> 172.21.15.16:0/4128548132 conn(0x7fa2a00fd8f0 msgr2=0x7fa2a00ff080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:15:00.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.857+0000 7fa2a82d3640 1 -- 172.21.15.16:0/4128548132 shutdown_connections 2024-09-12T02:15:00.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:00.857+0000 7fa2a82d3640 1 -- 172.21.15.16:0/4128548132 wait complete. 2024-09-12T02:15:01.878 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-12T02:15:02.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.033+0000 7f79b7b47640 1 -- 172.21.15.16:0/1145644672 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f79b0102780 msgr2=0x7f79b01067d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:02.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.033+0000 7f79b7b47640 1 --2- 172.21.15.16:0/1145644672 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f79b0102780 0x7f79b01067d0 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7f79a0009860 tx=0x7f79a002a340 comp rx=0 tx=0).stop 2024-09-12T02:15:02.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.033+0000 7f79b7b47640 1 -- 172.21.15.16:0/1145644672 shutdown_connections 2024-09-12T02:15:02.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.033+0000 7f79b7b47640 1 --2- 172.21.15.16:0/1145644672 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79b0106f00 0x7f79b010ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:02.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.033+0000 7f79b7b47640 1 --2- 172.21.15.16:0/1145644672 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f79b0102780 0x7f79b01067d0 unknown :-1 s=CLOSED pgs=1794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:02.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.033+0000 7f79b7b47640 1 --2- 172.21.15.16:0/1145644672 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f79b0101dd0 0x7f79b01021b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:02.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.033+0000 7f79b7b47640 1 -- 172.21.15.16:0/1145644672 >> 172.21.15.16:0/1145644672 conn(0x7f79b00fd8b0 msgr2=0x7f79b00ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:15:02.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.033+0000 7f79b7b47640 1 -- 172.21.15.16:0/1145644672 shutdown_connections 2024-09-12T02:15:02.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.033+0000 7f79b7b47640 1 -- 172.21.15.16:0/1145644672 wait complete. 2024-09-12T02:15:02.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.034+0000 7f79b7b47640 1 Processor -- start 2024-09-12T02:15:02.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.034+0000 7f79b7b47640 1 -- start start 2024-09-12T02:15:02.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.034+0000 7f79b7b47640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f79b0101dd0 0x7f79b0198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:02.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.034+0000 7f79b7b47640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79b0102780 0x7f79b01989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:02.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.034+0000 7f79b7b47640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f79b0106f00 0x7f79b019cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:02.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.034+0000 7f79b7b47640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f79b0111f60 con 0x7f79b0106f00 2024-09-12T02:15:02.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.034+0000 7f79b7b47640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f79b0111de0 con 0x7f79b0102780 2024-09-12T02:15:02.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.034+0000 7f79b7b47640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f79b01120e0 con 0x7f79b0101dd0 2024-09-12T02:15:02.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.035+0000 7f79b60e4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f79b0106f00 0x7f79b019cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:02.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.035+0000 7f79b60e4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f79b0106f00 0x7f79b019cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33296/0 (socket says 172.21.15.16:33296) 2024-09-12T02:15:02.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.035+0000 7f79b60e4640 1 -- 172.21.15.16:0/4142729292 learned_addr learned my addr 172.21.15.16:0/4142729292 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:15:02.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.035+0000 7f79b50e2640 1 --2- 172.21.15.16:0/4142729292 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79b0102780 0x7f79b01989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:02.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.035+0000 7f79b58e3640 1 --2- 172.21.15.16:0/4142729292 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f79b0101dd0 0x7f79b0198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:02.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.035+0000 7f79b60e4640 1 -- 172.21.15.16:0/4142729292 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f79b0101dd0 msgr2=0x7f79b0198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:02.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.035+0000 7f79b60e4640 1 --2- 172.21.15.16:0/4142729292 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f79b0101dd0 0x7f79b0198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:02.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.035+0000 7f79b60e4640 1 -- 172.21.15.16:0/4142729292 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79b0102780 msgr2=0x7f79b01989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:02.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.035+0000 7f79b60e4640 1 --2- 172.21.15.16:0/4142729292 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79b0102780 0x7f79b01989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:02.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.035+0000 7f79b60e4640 1 -- 172.21.15.16:0/4142729292 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79b019d410 con 0x7f79b0106f00 2024-09-12T02:15:02.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.035+0000 7f79b58e3640 1 --2- 172.21.15.16:0/4142729292 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f79b0101dd0 0x7f79b0198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:15:02.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.035+0000 7f79b60e4640 1 --2- 172.21.15.16:0/4142729292 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f79b0106f00 0x7f79b019cd30 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7f79a800efc0 tx=0x7f79a800c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:15:02.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.035+0000 7f79a6ffd640 1 -- 172.21.15.16:0/4142729292 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79a8003970 con 0x7f79b0106f00 2024-09-12T02:15:02.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.035+0000 7f79a6ffd640 1 -- 172.21.15.16:0/4142729292 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79a8004020 con 0x7f79b0106f00 2024-09-12T02:15:02.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.035+0000 7f79b7b47640 1 -- 172.21.15.16:0/4142729292 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79b019d700 con 0x7f79b0106f00 2024-09-12T02:15:02.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.035+0000 7f79a6ffd640 1 -- 172.21.15.16:0/4142729292 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79a800b9f0 con 0x7f79b0106f00 2024-09-12T02:15:02.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.035+0000 7f79b7b47640 1 -- 172.21.15.16:0/4142729292 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79b01a4fe0 con 0x7f79b0106f00 2024-09-12T02:15:02.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.038+0000 7f79a6ffd640 1 -- 172.21.15.16:0/4142729292 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f79a801d020 con 0x7f79b0106f00 2024-09-12T02:15:02.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.038+0000 7f79a6ffd640 1 --2- 172.21.15.16:0/4142729292 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f799c0a2c00 0x7f799c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:02.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.038+0000 7f79b58e3640 1 --2- 172.21.15.16:0/4142729292 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f799c0a2c00 0x7f799c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:02.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.038+0000 7f79a6ffd640 1 -- 172.21.15.16:0/4142729292 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f79a80cede0 con 0x7f79b0106f00 2024-09-12T02:15:02.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.039+0000 7f79b58e3640 1 --2- 172.21.15.16:0/4142729292 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f799c0a2c00 0x7f799c0a50c0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f79ac009b40 tx=0x7f79ac005e50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:15:02.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.039+0000 7f79b9f20740 1 Processor -- start 2024-09-12T02:15:02.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.039+0000 7f79b9f20740 1 -- start start 2024-09-12T02:15:02.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.040+0000 7f79b9f20740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f3cd0a6760 0x55f3cd0a6b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:02.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.040+0000 7f79b9f20740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f3cd0a7080 0x55f3cd0b1400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:02.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.040+0000 7f79b9f20740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f3cd0b1940 0x55f3cd0b3d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:02.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.040+0000 7f79b9f20740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55f3cd00c2a0 con 0x55f3cd0a7080 2024-09-12T02:15:02.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.040+0000 7f79b9f20740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55f3ccfb3b10 con 0x55f3cd0b1940 2024-09-12T02:15:02.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.040+0000 7f79b9f20740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55f3cd008260 con 0x55f3cd0a6760 2024-09-12T02:15:02.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.040+0000 7f79b50e2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f3cd0a7080 0x55f3cd0b1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:02.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.040+0000 7f79b50e2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f3cd0a7080 0x55f3cd0b1400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33302/0 (socket says 172.21.15.16:33302) 2024-09-12T02:15:02.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.040+0000 7f79b50e2640 1 -- 172.21.15.16:0/1205230367 learned_addr learned my addr 172.21.15.16:0/1205230367 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:15:02.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.040+0000 7f79b58e3640 1 --2- 172.21.15.16:0/1205230367 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f3cd0a6760 0x55f3cd0a6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:02.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.040+0000 7f79b60e4640 1 --2- 172.21.15.16:0/1205230367 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f3cd0b1940 0x55f3cd0b3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:02.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.040+0000 7f79b50e2640 1 -- 172.21.15.16:0/1205230367 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f3cd0a6760 msgr2=0x55f3cd0a6b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:15:02.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.040+0000 7f79b50e2640 1 --2- 172.21.15.16:0/1205230367 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f3cd0a6760 0x55f3cd0a6b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:02.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.040+0000 7f79b50e2640 1 -- 172.21.15.16:0/1205230367 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f3cd0b1940 msgr2=0x55f3cd0b3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:15:02.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.040+0000 7f79b50e2640 1 --2- 172.21.15.16:0/1205230367 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f3cd0b1940 0x55f3cd0b3d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:02.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.040+0000 7f79b50e2640 1 -- 172.21.15.16:0/1205230367 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3ccd7db90 con 0x55f3cd0a7080 2024-09-12T02:15:02.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.040+0000 7f79b60e4640 1 --2- 172.21.15.16:0/1205230367 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f3cd0b1940 0x55f3cd0b3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:15:02.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.040+0000 7f79b50e2640 1 --2- 172.21.15.16:0/1205230367 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f3cd0a7080 0x55f3cd0b1400 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7f79a0005e00 tx=0x7f79a0009510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=548c88265a4c6bf4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:15:02.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.040+0000 7f799bfff640 1 -- 172.21.15.16:0/1205230367 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79a0037040 con 0x55f3cd0a7080 2024-09-12T02:15:02.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.040+0000 7f799bfff640 1 -- 172.21.15.16:0/1205230367 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79a0030e50 con 0x55f3cd0a7080 2024-09-12T02:15:02.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.041+0000 7f799bfff640 1 -- 172.21.15.16:0/1205230367 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79a0003910 con 0x55f3cd0a7080 2024-09-12T02:15:02.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.041+0000 7f79b9f20740 1 -- 172.21.15.16:0/1205230367 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f3cd0a7080 msgr2=0x55f3cd0b1400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:02.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.041+0000 7f79b9f20740 1 --2- 172.21.15.16:0/1205230367 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f3cd0a7080 0x55f3cd0b1400 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7f79a0005e00 tx=0x7f79a0009510 comp rx=0 tx=0).stop 2024-09-12T02:15:02.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.041+0000 7f79b9f20740 1 -- 172.21.15.16:0/1205230367 shutdown_connections 2024-09-12T02:15:02.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.041+0000 7f79b9f20740 1 --2- 172.21.15.16:0/1205230367 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f3cd0b1940 0x55f3cd0b3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:02.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.041+0000 7f79b9f20740 1 --2- 172.21.15.16:0/1205230367 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f3cd0a7080 0x55f3cd0b1400 unknown :-1 s=CLOSED pgs=1796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:02.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.041+0000 7f79b9f20740 1 --2- 172.21.15.16:0/1205230367 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f3cd0a6760 0x55f3cd0a6b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:02.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.041+0000 7f79b9f20740 1 -- 172.21.15.16:0/1205230367 >> 172.21.15.16:0/1205230367 conn(0x55f3cd0a2cf0 msgr2=0x55f3cd0a30d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:15:02.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.041+0000 7f79b9f20740 1 -- 172.21.15.16:0/1205230367 shutdown_connections 2024-09-12T02:15:02.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.041+0000 7f79b9f20740 1 -- 172.21.15.16:0/1205230367 wait complete. 2024-09-12T02:15:02.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.042+0000 7f79b9f20740 1 -- start start 2024-09-12T02:15:02.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.042+0000 7f79b9f20740 1 Processor -- start 2024-09-12T02:15:02.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.043+0000 7f79b9f20740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f3cd0a6760 0x55f3cd0ae7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:02.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.043+0000 7f79b9f20740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f3cd0a7080 0x55f3cd0aed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:02.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.043+0000 7f79b9f20740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f3cd0b1940 0x55f3cd0af250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:02.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.043+0000 7f79b9f20740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55f3cd0b4510 con 0x55f3cd0a6760 2024-09-12T02:15:02.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.043+0000 7f79b9f20740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55f3ccf946e0 con 0x55f3cd0b1940 2024-09-12T02:15:02.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.043+0000 7f79b9f20740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55f3cd0b4690 con 0x55f3cd0a7080 2024-09-12T02:15:02.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.043+0000 7f79b58e3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f3cd0a6760 0x55f3cd0ae7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:02.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.043+0000 7f79b58e3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f3cd0a6760 0x55f3cd0ae7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33306/0 (socket says 172.21.15.16:33306) 2024-09-12T02:15:02.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.043+0000 7f79b58e3640 1 -- 172.21.15.16:0/3464113433 learned_addr learned my addr 172.21.15.16:0/3464113433 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:15:02.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.043+0000 7f79b9f20740 10 client.0 _ll_register_callbacks cb 0x55f3ccfb8a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:15:02.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.043+0000 7f79b60e4640 1 --2- 172.21.15.16:0/3464113433 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f3cd0b1940 0x55f3cd0af250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:02.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.043+0000 7f79b50e2640 1 --2- 172.21.15.16:0/3464113433 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f3cd0a7080 0x55f3cd0aed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:02.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.043+0000 7f79b58e3640 1 -- 172.21.15.16:0/3464113433 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f3cd0a7080 msgr2=0x55f3cd0aed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:15:02.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.043+0000 7f79b58e3640 1 --2- 172.21.15.16:0/3464113433 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f3cd0a7080 0x55f3cd0aed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:02.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.043+0000 7f79b58e3640 1 -- 172.21.15.16:0/3464113433 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f3cd0b1940 msgr2=0x55f3cd0af250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:15:02.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.043+0000 7f79b58e3640 1 --2- 172.21.15.16:0/3464113433 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f3cd0b1940 0x55f3cd0af250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:02.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.043+0000 7f79b58e3640 1 -- 172.21.15.16:0/3464113433 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3cd0bd9b0 con 0x55f3cd0a6760 2024-09-12T02:15:02.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.043+0000 7f79b58e3640 1 --2- 172.21.15.16:0/3464113433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f3cd0a6760 0x55f3cd0ae7d0 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7f79ac011c60 tx=0x7f79ac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc71dfd7fe79af0c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:15:02.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.043+0000 7f79997fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:15:02.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.044+0000 7f79997fa640 1 -- 172.21.15.16:0/3464113433 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79ac00ec20 con 0x55f3cd0a6760 2024-09-12T02:15:02.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.044+0000 7f79997fa640 1 -- 172.21.15.16:0/3464113433 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f79ac00edc0 con 0x55f3cd0a6760 2024-09-12T02:15:02.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.044+0000 7f79b9f20740 1 -- 172.21.15.16:0/3464113433 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f79a0030e50 con 0x55f3cd0a6760 2024-09-12T02:15:02.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.044+0000 7f79997fa640 1 -- 172.21.15.16:0/3464113433 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f79ac027400 con 0x55f3cd0a6760 2024-09-12T02:15:02.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.044+0000 7f79b9f20740 10 client.19794 fetch_fsmap learned FSMap version 25 2024-09-12T02:15:02.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.044+0000 7f79b9f20740 1 -- 172.21.15.16:0/3464113433 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f3cd0c03b0 con 0x55f3cd0a6760 2024-09-12T02:15:02.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.044+0000 7f79997fa640 1 -- 172.21.15.16:0/3464113433 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79a0030e50 con 0x55f3cd0a6760 2024-09-12T02:15:02.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.044+0000 7f79997fa640 1 -- 172.21.15.16:0/3464113433 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f79ac00f040 con 0x55f3cd0a6760 2024-09-12T02:15:02.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.044+0000 7f79b9f20740 10 client.19794 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:15:02.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.044+0000 7f79b9f20740 10 client.19794 resolve_mds: validated gid 24482 aka daemon mds.b 2024-09-12T02:15:02.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.045+0000 7f79b9f20740 20 client.19794 populate_metadata read hostname 'smithi016' 2024-09-12T02:15:02.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.045+0000 7f79b9f20740 1 --2- 172.21.15.16:0/3464113433 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55f3cd0c1870 0x55f3cd0c3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:02.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.045+0000 7f79b9f20740 4 client.19794 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:15:02.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.045+0000 7f79b9f20740 1 -- 172.21.15.16:0/3464113433 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f3cd0199d0 con 0x55f3cd0c1870 2024-09-12T02:15:02.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.045+0000 7f79b50e2640 1 --2- 172.21.15.16:0/3464113433 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55f3cd0c1870 0x55f3cd0c3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:02.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.045+0000 7f79b50e2640 1 --2- 172.21.15.16:0/3464113433 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55f3cd0c1870 0x55f3cd0c3cc0 crc :-1 s=READY pgs=391 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab332181a6efb0bb server_cookie=1ef3e28f3f6f4d62 in_seq=0 out_seq=0 2024-09-12T02:15:02.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.045+0000 7f79997fa640 10 client.19794 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:15:02.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.046+0000 7f79997fa640 1 -- 172.21.15.16:0/3464113433 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f3cd0199d0 con 0x55f3cd0c1870 2024-09-12T02:15:02.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.046+0000 7f79997fa640 10 client.19794 handle_command_reply: tid=0 2024-09-12T02:15:02.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.046+0000 7f79b9f20740 1 client.19794 shutdown 2024-09-12T02:15:02.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.046+0000 7f79b9f20740 1 -- 172.21.15.16:0/3464113433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f3cd0a6760 msgr2=0x55f3cd0ae7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:02.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.046+0000 7f79b9f20740 1 --2- 172.21.15.16:0/3464113433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f3cd0a6760 0x55f3cd0ae7d0 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7f79ac011c60 tx=0x7f79ac0046c0 comp rx=0 tx=0).stop 2024-09-12T02:15:02.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.046+0000 7f79b9f20740 1 -- 172.21.15.16:0/3464113433 shutdown_connections 2024-09-12T02:15:02.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.046+0000 7f79b9f20740 1 --2- 172.21.15.16:0/3464113433 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55f3cd0c1870 0x55f3cd0c3cc0 crc :-1 s=READY pgs=391 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:02.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.046+0000 7f79b9f20740 1 --2- 172.21.15.16:0/3464113433 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f3cd0b1940 0x55f3cd0af250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:02.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.046+0000 7f79b9f20740 1 --2- 172.21.15.16:0/3464113433 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f3cd0a7080 0x55f3cd0aed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:02.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.046+0000 7f79b9f20740 1 --2- 172.21.15.16:0/3464113433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f3cd0a6760 0x55f3cd0ae7d0 unknown :-1 s=CLOSED pgs=1797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:02.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.046+0000 7f79997fa640 0 client.19794 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:15:02.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.046+0000 7f79b9f20740 1 -- 172.21.15.16:0/3464113433 >> 172.21.15.16:0/3464113433 conn(0x55f3cd0a2cf0 msgr2=0x55f3cd0a1e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:15:02.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.046+0000 7f79b9f20740 1 -- 172.21.15.16:0/3464113433 shutdown_connections 2024-09-12T02:15:02.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.046+0000 7f79b9f20740 1 -- 172.21.15.16:0/3464113433 wait complete. 2024-09-12T02:15:02.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.047+0000 7f79b9f20740 20 client.19794 trim_cache size 0 max 16384 2024-09-12T02:15:02.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.064+0000 7f79b9f20740 1 Processor -- start 2024-09-12T02:15:02.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.064+0000 7f79b9f20740 1 -- start start 2024-09-12T02:15:02.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.064+0000 7f79b9f20740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f3cd0a6760 0x55f3cd0a91c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:02.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.064+0000 7f79b9f20740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f3cd0a7080 0x55f3cd0c9590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:02.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.064+0000 7f79b9f20740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f3cd0b1940 0x55f3cd0c9ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:02.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.064+0000 7f79b9f20740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f79b01afc80 con 0x55f3cd0b1940 2024-09-12T02:15:02.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.064+0000 7f79b9f20740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f79b01ad550 con 0x55f3cd0a7080 2024-09-12T02:15:02.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.064+0000 7f79b9f20740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f79b01ab3c0 con 0x55f3cd0a6760 2024-09-12T02:15:02.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.064+0000 7f79b50e2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f3cd0a7080 0x55f3cd0c9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:02.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.064+0000 7f79b58e3640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f3cd0a6760 0x55f3cd0a91c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:02.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.064+0000 7f79b50e2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f3cd0a7080 0x55f3cd0c9590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.16:55346/0 (socket says 172.21.15.16:55346) 2024-09-12T02:15:02.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.064+0000 7f79b50e2640 1 -- 172.21.15.16:0/4126080476 learned_addr learned my addr 172.21.15.16:0/4126080476 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:15:02.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.064+0000 7f79b60e4640 1 --2- 172.21.15.16:0/4126080476 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f3cd0b1940 0x55f3cd0c9ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:02.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.065+0000 7f79b50e2640 1 -- 172.21.15.16:0/4126080476 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f3cd0a6760 msgr2=0x55f3cd0a91c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:15:02.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.065+0000 7f79b50e2640 1 --2- 172.21.15.16:0/4126080476 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f3cd0a6760 0x55f3cd0a91c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:02.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.065+0000 7f79b50e2640 1 -- 172.21.15.16:0/4126080476 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f3cd0b1940 msgr2=0x55f3cd0c9ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:15:02.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.065+0000 7f79b50e2640 1 --2- 172.21.15.16:0/4126080476 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f3cd0b1940 0x55f3cd0c9ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:02.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.065+0000 7f79b50e2640 1 -- 172.21.15.16:0/4126080476 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3cd09e260 con 0x55f3cd0a7080 2024-09-12T02:15:02.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.065+0000 7f79b50e2640 1 --2- 172.21.15.16:0/4126080476 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f3cd0a7080 0x55f3cd0c9590 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f79a002a900 tx=0x7f79a00058d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=72f2139319ff9fa4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:15:02.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.065+0000 7f799b7fe640 1 -- 172.21.15.16:0/4126080476 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79a0037040 con 0x55f3cd0a7080 2024-09-12T02:15:02.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.065+0000 7f799b7fe640 1 -- 172.21.15.16:0/4126080476 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79a004f430 con 0x55f3cd0a7080 2024-09-12T02:15:02.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.065+0000 7f799b7fe640 1 -- 172.21.15.16:0/4126080476 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79a003d680 con 0x55f3cd0a7080 2024-09-12T02:15:02.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.066+0000 7f79b9f20740 1 -- 172.21.15.16:0/4126080476 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f3cd0a7080 msgr2=0x55f3cd0c9590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:02.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.066+0000 7f79b9f20740 1 --2- 172.21.15.16:0/4126080476 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f3cd0a7080 0x55f3cd0c9590 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f79a002a900 tx=0x7f79a00058d0 comp rx=0 tx=0).stop 2024-09-12T02:15:02.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.066+0000 7f79b9f20740 1 -- 172.21.15.16:0/4126080476 shutdown_connections 2024-09-12T02:15:02.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.066+0000 7f79b9f20740 1 --2- 172.21.15.16:0/4126080476 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f3cd0b1940 0x55f3cd0c9ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:02.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.066+0000 7f79b9f20740 1 --2- 172.21.15.16:0/4126080476 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f3cd0a7080 0x55f3cd0c9590 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:02.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.066+0000 7f79b9f20740 1 --2- 172.21.15.16:0/4126080476 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f3cd0a6760 0x55f3cd0a91c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:02.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.066+0000 7f79b9f20740 1 -- 172.21.15.16:0/4126080476 >> 172.21.15.16:0/4126080476 conn(0x55f3cd0a2cf0 msgr2=0x55f3cd0b0850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:15:02.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.066+0000 7f79b9f20740 1 -- 172.21.15.16:0/4126080476 shutdown_connections 2024-09-12T02:15:02.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.066+0000 7f79b9f20740 1 -- 172.21.15.16:0/4126080476 wait complete. 2024-09-12T02:15:02.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.067+0000 7f79b9f20740 1 -- start start 2024-09-12T02:15:02.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.067+0000 7f79b9f20740 1 Processor -- start 2024-09-12T02:15:02.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.067+0000 7f79b9f20740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f3cd0a6760 0x55f3cd0ba0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:02.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.067+0000 7f79b9f20740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f3cd0a7080 0x55f3cd0b5410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:02.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.067+0000 7f79b9f20740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f3cd0b1940 0x55f3cd0b5990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:02.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.067+0000 7f79b9f20740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f79b01aba10 con 0x55f3cd0a6760 2024-09-12T02:15:02.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.067+0000 7f79b9f20740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f79b01ace80 con 0x55f3cd0b1940 2024-09-12T02:15:02.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.067+0000 7f79b9f20740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f79b01ac0e0 con 0x55f3cd0a7080 2024-09-12T02:15:02.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.067+0000 7f79b9f20740 10 client.0 _ll_register_callbacks cb 0x7f79ac0118e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:15:02.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.067+0000 7f79b58e3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f3cd0a6760 0x55f3cd0ba0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:02.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.067+0000 7f79b58e3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f3cd0a6760 0x55f3cd0ba0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33314/0 (socket says 172.21.15.16:33314) 2024-09-12T02:15:02.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.067+0000 7f79b58e3640 1 -- 172.21.15.16:0/914341599 learned_addr learned my addr 172.21.15.16:0/914341599 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:15:02.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.068+0000 7f79b50e2640 1 --2- 172.21.15.16:0/914341599 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f3cd0a7080 0x55f3cd0b5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:02.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.068+0000 7f79b60e4640 1 --2- 172.21.15.16:0/914341599 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f3cd0b1940 0x55f3cd0b5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:02.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.068+0000 7f79b58e3640 1 -- 172.21.15.16:0/914341599 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f3cd0a7080 msgr2=0x55f3cd0b5410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:15:02.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.068+0000 7f79b58e3640 1 --2- 172.21.15.16:0/914341599 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f3cd0a7080 0x55f3cd0b5410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:02.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.068+0000 7f79b58e3640 1 -- 172.21.15.16:0/914341599 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f3cd0b1940 msgr2=0x55f3cd0b5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:15:02.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.068+0000 7f79b58e3640 1 --2- 172.21.15.16:0/914341599 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f3cd0b1940 0x55f3cd0b5990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:02.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.068+0000 7f79b58e3640 1 -- 172.21.15.16:0/914341599 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3cd0b61c0 con 0x55f3cd0a6760 2024-09-12T02:15:02.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.068+0000 7f79b58e3640 1 --2- 172.21.15.16:0/914341599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f3cd0a6760 0x55f3cd0ba0e0 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7f79ac011c30 tx=0x7f79ac010cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c6d7dd15f0a4a86 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:15:02.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.068+0000 7f79997fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:15:02.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.068+0000 7f79997fa640 1 -- 172.21.15.16:0/914341599 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79ac004930 con 0x55f3cd0a6760 2024-09-12T02:15:02.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.068+0000 7f79997fa640 1 -- 172.21.15.16:0/914341599 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f79ac004ad0 con 0x55f3cd0a6760 2024-09-12T02:15:02.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.068+0000 7f79b9f20740 1 -- 172.21.15.16:0/914341599 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f79a004f430 con 0x55f3cd0a6760 2024-09-12T02:15:02.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.069+0000 7f79997fa640 1 -- 172.21.15.16:0/914341599 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f79ac043050 con 0x55f3cd0a6760 2024-09-12T02:15:02.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.069+0000 7f79b9f20740 10 client.19800 fetch_fsmap learned FSMap version 25 2024-09-12T02:15:02.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.069+0000 7f79b9f20740 1 -- 172.21.15.16:0/914341599 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f3cd12c5e0 con 0x55f3cd0a6760 2024-09-12T02:15:02.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.069+0000 7f79997fa640 1 -- 172.21.15.16:0/914341599 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79a004f430 con 0x55f3cd0a6760 2024-09-12T02:15:02.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.069+0000 7f79997fa640 1 -- 172.21.15.16:0/914341599 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f79ac0275a0 con 0x55f3cd0a6760 2024-09-12T02:15:02.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.069+0000 7f79b9f20740 10 client.19800 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:15:02.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.069+0000 7f79b9f20740 10 client.19800 resolve_mds: validated gid 24482 aka daemon mds.b 2024-09-12T02:15:02.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.069+0000 7f79b9f20740 20 client.19800 populate_metadata read hostname 'smithi016' 2024-09-12T02:15:02.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.069+0000 7f79b9f20740 1 --2- 172.21.15.16:0/914341599 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55f3cd12cda0 0x55f3cd12f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:02.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.069+0000 7f79b9f20740 4 client.19800 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}] 2024-09-12T02:15:02.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.069+0000 7f79b9f20740 1 -- 172.21.15.16:0/914341599 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}) -- 0x55f3cd013d20 con 0x55f3cd12cda0 2024-09-12T02:15:02.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.069+0000 7f79b50e2640 1 --2- 172.21.15.16:0/914341599 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55f3cd12cda0 0x55f3cd12f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:02.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.070+0000 7f79997fa640 10 client.19800 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:15:02.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.070+0000 7f79b50e2640 1 --2- 172.21.15.16:0/914341599 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55f3cd12cda0 0x55f3cd12f1c0 crc :-1 s=READY pgs=392 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a78b7e0ecffb408 server_cookie=654720732c2bea5f in_seq=0 out_seq=0 2024-09-12T02:15:02.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.082+0000 7f79997fa640 1 -- 172.21.15.16:0/914341599 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55f3cd013d20 con 0x55f3cd12cda0 2024-09-12T02:15:02.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.082+0000 7f79997fa640 10 client.19800 handle_command_reply: tid=0 2024-09-12T02:15:02.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.082+0000 7f79b9f20740 1 client.19800 shutdown 2024-09-12T02:15:02.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.082+0000 7f79b9f20740 1 -- 172.21.15.16:0/914341599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f3cd0a6760 msgr2=0x55f3cd0ba0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:02.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.082+0000 7f79b9f20740 1 --2- 172.21.15.16:0/914341599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f3cd0a6760 0x55f3cd0ba0e0 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7f79ac011c30 tx=0x7f79ac010cb0 comp rx=0 tx=0).stop 2024-09-12T02:15:02.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.082+0000 7f79b9f20740 1 -- 172.21.15.16:0/914341599 shutdown_connections 2024-09-12T02:15:02.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.082+0000 7f79b9f20740 1 --2- 172.21.15.16:0/914341599 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55f3cd12cda0 0x55f3cd12f1c0 crc :-1 s=READY pgs=392 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:02.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.082+0000 7f79b9f20740 1 --2- 172.21.15.16:0/914341599 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f3cd0b1940 0x55f3cd0b5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:02.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.082+0000 7f79b9f20740 1 --2- 172.21.15.16:0/914341599 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f3cd0a7080 0x55f3cd0b5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:02.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.082+0000 7f79b9f20740 1 --2- 172.21.15.16:0/914341599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f3cd0a6760 0x55f3cd0ba0e0 unknown :-1 s=CLOSED pgs=1798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:02.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.082+0000 7f79b9f20740 1 -- 172.21.15.16:0/914341599 >> 172.21.15.16:0/914341599 conn(0x55f3cd0a2cf0 msgr2=0x55f3cd0b0030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:15:02.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.082+0000 7f79997fa640 0 client.19800 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:15:02.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.082+0000 7f79b9f20740 1 -- 172.21.15.16:0/914341599 shutdown_connections 2024-09-12T02:15:02.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.082+0000 7f79b9f20740 1 -- 172.21.15.16:0/914341599 wait complete. 2024-09-12T02:15:02.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.082+0000 7f79b9f20740 20 client.19800 trim_cache size 0 max 16384 2024-09-12T02:15:02.084 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T02:15:02.084 INFO:teuthology.orchestra.run.smithi016.stdout: "epoch": 25, 2024-09-12T02:15:02.084 INFO:teuthology.orchestra.run.smithi016.stdout: "leader": 24482, 2024-09-12T02:15:02.084 INFO:teuthology.orchestra.run.smithi016.stdout: "set_version": 50, 2024-09-12T02:15:02.084 INFO:teuthology.orchestra.run.smithi016.stdout: "sets": { 2024-09-12T02:15:02.084 INFO:teuthology.orchestra.run.smithi016.stdout: "901e6dc3": { 2024-09-12T02:15:02.084 INFO:teuthology.orchestra.run.smithi016.stdout: "version": 50, 2024-09-12T02:15:02.084 INFO:teuthology.orchestra.run.smithi016.stdout: "age_ref": 0.0, 2024-09-12T02:15:02.084 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-12T02:15:02.084 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "QUIESCED", 2024-09-12T02:15:02.084 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-12T02:15:02.085 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:15:02.085 INFO:teuthology.orchestra.run.smithi016.stdout: "timeout": 90.0, 2024-09-12T02:15:02.085 INFO:teuthology.orchestra.run.smithi016.stdout: "expiration": 150.0, 2024-09-12T02:15:02.085 INFO:teuthology.orchestra.run.smithi016.stdout: "members": { 2024-09-12T02:15:02.085 INFO:teuthology.orchestra.run.smithi016.stdout: "file:/": { 2024-09-12T02:15:02.085 INFO:teuthology.orchestra.run.smithi016.stdout: "excluded": false, 2024-09-12T02:15:02.085 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-12T02:15:02.085 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "QUIESCED", 2024-09-12T02:15:02.085 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-12T02:15:02.085 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:15:02.085 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:15:02.085 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:15:02.085 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:15:02.085 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:15:02.086 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T02:15:02.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.083+0000 7f79b7b47640 1 -- 172.21.15.16:0/4142729292 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f799c0a2c00 msgr2=0x7f799c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:02.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.083+0000 7f79b7b47640 1 --2- 172.21.15.16:0/4142729292 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f799c0a2c00 0x7f799c0a50c0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f79ac009b40 tx=0x7f79ac005e50 comp rx=0 tx=0).stop 2024-09-12T02:15:02.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.083+0000 7f79b7b47640 1 -- 172.21.15.16:0/4142729292 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f79b0106f00 msgr2=0x7f79b019cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:02.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.083+0000 7f79b7b47640 1 --2- 172.21.15.16:0/4142729292 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f79b0106f00 0x7f79b019cd30 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7f79a800efc0 tx=0x7f79a800c800 comp rx=0 tx=0).stop 2024-09-12T02:15:02.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.083+0000 7f79b7b47640 1 -- 172.21.15.16:0/4142729292 shutdown_connections 2024-09-12T02:15:02.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.083+0000 7f79b7b47640 1 --2- 172.21.15.16:0/4142729292 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f799c0a2c00 0x7f799c0a50c0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:02.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.083+0000 7f79b7b47640 1 --2- 172.21.15.16:0/4142729292 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f79b0106f00 0x7f79b019cd30 unknown :-1 s=CLOSED pgs=1795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:02.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.083+0000 7f79b7b47640 1 --2- 172.21.15.16:0/4142729292 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79b0102780 0x7f79b01989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:02.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.083+0000 7f79b7b47640 1 --2- 172.21.15.16:0/4142729292 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f79b0101dd0 0x7f79b0198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:02.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.083+0000 7f79b7b47640 1 -- 172.21.15.16:0/4142729292 >> 172.21.15.16:0/4142729292 conn(0x7f79b00fd8b0 msgr2=0x7f79b00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:15:02.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.083+0000 7f79b7b47640 1 -- 172.21.15.16:0/4142729292 shutdown_connections 2024-09-12T02:15:02.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:02.083+0000 7f79b7b47640 1 -- 172.21.15.16:0/4142729292 wait complete. 2024-09-12T02:15:02.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:15:02 smithi016 ceph-mon[30947]: pgmap v728: 97 pgs: 97 active+clean; 2.9 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-12T02:15:02.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:15:02 smithi184 ceph-mon[35593]: pgmap v728: 97 pgs: 97 active+clean; 2.9 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-12T02:15:02.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:15:02 smithi033 ceph-mon[35332]: pgmap v728: 97 pgs: 97 active+clean; 2.9 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-12T02:15:02.867 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '901e6dc3', quiesce took 4.7 seconds. Will release after: 25.3 2024-09-12T02:15:02.915 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:15:03.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.385+0000 7fab5cb95640 1 -- 172.21.15.16:0/2015115480 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab58103660 msgr2=0x7fab5810bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:03.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.385+0000 7fab5cb95640 1 --2- 172.21.15.16:0/2015115480 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab58103660 0x7fab5810bb50 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7fab40009900 tx=0x7fab4002a480 comp rx=0 tx=0).stop 2024-09-12T02:15:03.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.385+0000 7fab5cb95640 1 -- 172.21.15.16:0/2015115480 shutdown_connections 2024-09-12T02:15:03.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.385+0000 7fab5cb95640 1 --2- 172.21.15.16:0/2015115480 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab5810c170 0x7fab5810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:03.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.385+0000 7fab5cb95640 1 --2- 172.21.15.16:0/2015115480 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab58103660 0x7fab5810bb50 unknown :-1 s=CLOSED pgs=1799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:03.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.385+0000 7fab5cb95640 1 --2- 172.21.15.16:0/2015115480 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fab58102d40 0x7fab58103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:03.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.385+0000 7fab5cb95640 1 -- 172.21.15.16:0/2015115480 >> 172.21.15.16:0/2015115480 conn(0x7fab580fc7a0 msgr2=0x7fab580febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:15:03.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.385+0000 7fab5cb95640 1 -- 172.21.15.16:0/2015115480 shutdown_connections 2024-09-12T02:15:03.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.385+0000 7fab5cb95640 1 -- 172.21.15.16:0/2015115480 wait complete. 2024-09-12T02:15:03.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.386+0000 7fab5cb95640 1 Processor -- start 2024-09-12T02:15:03.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.386+0000 7fab5cb95640 1 -- start start 2024-09-12T02:15:03.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.386+0000 7fab5cb95640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab58102d40 0x7fab581984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:03.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.386+0000 7fab5cb95640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab58103660 0x7fab58198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:03.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.386+0000 7fab5cb95640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fab5810c170 0x7fab5819cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:03.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.386+0000 7fab5cb95640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fab58112000 con 0x7fab58102d40 2024-09-12T02:15:03.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.386+0000 7fab5cb95640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fab58111e80 con 0x7fab58103660 2024-09-12T02:15:03.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.386+0000 7fab5cb95640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fab58112180 con 0x7fab5810c170 2024-09-12T02:15:03.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.386+0000 7fab5659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab58102d40 0x7fab581984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:03.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.386+0000 7fab5659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab58102d40 0x7fab581984d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33334/0 (socket says 172.21.15.16:33334) 2024-09-12T02:15:03.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.387+0000 7fab5659c640 1 -- 172.21.15.16:0/2606917357 learned_addr learned my addr 172.21.15.16:0/2606917357 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:15:03.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.387+0000 7fab4ffff640 1 --2- 172.21.15.16:0/2606917357 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab58103660 0x7fab58198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:03.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.387+0000 7fab5659c640 1 -- 172.21.15.16:0/2606917357 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fab5810c170 msgr2=0x7fab5819cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:03.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.387+0000 7fab56d9d640 1 --2- 172.21.15.16:0/2606917357 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fab5810c170 0x7fab5819cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:03.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.387+0000 7fab5659c640 1 --2- 172.21.15.16:0/2606917357 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fab5810c170 0x7fab5819cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:03.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.387+0000 7fab5659c640 1 -- 172.21.15.16:0/2606917357 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab58103660 msgr2=0x7fab58198a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:03.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.387+0000 7fab5659c640 1 --2- 172.21.15.16:0/2606917357 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab58103660 0x7fab58198a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:03.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.387+0000 7fab5659c640 1 -- 172.21.15.16:0/2606917357 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab5819d480 con 0x7fab58102d40 2024-09-12T02:15:03.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.387+0000 7fab4ffff640 1 --2- 172.21.15.16:0/2606917357 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab58103660 0x7fab58198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:15:03.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.387+0000 7fab56d9d640 1 --2- 172.21.15.16:0/2606917357 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fab5810c170 0x7fab5819cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:15:03.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.387+0000 7fab5659c640 1 --2- 172.21.15.16:0/2606917357 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab58102d40 0x7fab581984d0 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7fab4800b4a0 tx=0x7fab4800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:15:03.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.387+0000 7fab4f7fe640 1 -- 172.21.15.16:0/2606917357 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab48014070 con 0x7fab58102d40 2024-09-12T02:15:03.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.387+0000 7fab4f7fe640 1 -- 172.21.15.16:0/2606917357 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab48010af0 con 0x7fab58102d40 2024-09-12T02:15:03.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.387+0000 7fab4f7fe640 1 -- 172.21.15.16:0/2606917357 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab48017d10 con 0x7fab58102d40 2024-09-12T02:15:03.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.387+0000 7fab5cb95640 1 -- 172.21.15.16:0/2606917357 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab5819d770 con 0x7fab58102d40 2024-09-12T02:15:03.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.387+0000 7fab5cb95640 1 -- 172.21.15.16:0/2606917357 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab581a5050 con 0x7fab58102d40 2024-09-12T02:15:03.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.390+0000 7fab4f7fe640 1 -- 172.21.15.16:0/2606917357 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fab48017070 con 0x7fab58102d40 2024-09-12T02:15:03.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.390+0000 7fab5ef6e740 1 Processor -- start 2024-09-12T02:15:03.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.390+0000 7fab5ef6e740 1 -- start start 2024-09-12T02:15:03.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.390+0000 7fab5ef6e740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c4337c4b40 0x55c4337c3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:03.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.390+0000 7fab5ef6e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4337c35d0 0x55c4337c3a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:03.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.390+0000 7fab5ef6e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c4337d2780 0x55c4337d4b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:03.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.390+0000 7fab5ef6e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55c4337292f0 con 0x55c4337c35d0 2024-09-12T02:15:03.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.390+0000 7fab5ef6e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55c4336d0cf0 con 0x55c4337d2780 2024-09-12T02:15:03.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.390+0000 7fab5ef6e740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55c4337252b0 con 0x55c4337c4b40 2024-09-12T02:15:03.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.391+0000 7fab4f7fe640 1 --2- 172.21.15.16:0/2606917357 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fab440a2c00 0x7fab440a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:03.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.391+0000 7fab5659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4337c35d0 0x55c4337c3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:03.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.391+0000 7fab5659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4337c35d0 0x55c4337c3a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33344/0 (socket says 172.21.15.16:33344) 2024-09-12T02:15:03.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.391+0000 7fab5659c640 1 -- 172.21.15.16:0/1752788659 learned_addr learned my addr 172.21.15.16:0/1752788659 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:15:03.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.391+0000 7fab56d9d640 1 --2- 172.21.15.16:0/1752788659 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c4337c4b40 0x55c4337c3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:03.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.391+0000 7fab4ffff640 1 --2- 172.21.15.16:0/1752788659 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c4337d2780 0x55c4337d4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:03.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.391+0000 7fab56d9d640 1 --2- 172.21.15.16:0/2606917357 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fab440a2c00 0x7fab440a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:03.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.391+0000 7fab4f7fe640 1 -- 172.21.15.16:0/2606917357 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fab480ce340 con 0x7fab58102d40 2024-09-12T02:15:03.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.391+0000 7fab5659c640 1 -- 172.21.15.16:0/1752788659 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c4337c4b40 msgr2=0x55c4337c3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:15:03.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.391+0000 7fab5659c640 1 --2- 172.21.15.16:0/1752788659 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c4337c4b40 0x55c4337c3090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:03.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.391+0000 7fab5659c640 1 -- 172.21.15.16:0/1752788659 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c4337d2780 msgr2=0x55c4337d4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:15:03.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.391+0000 7fab5659c640 1 --2- 172.21.15.16:0/1752788659 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c4337d2780 0x55c4337d4b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:03.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.391+0000 7fab5659c640 1 -- 172.21.15.16:0/1752788659 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c4336b53a0 con 0x55c4337c35d0 2024-09-12T02:15:03.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.391+0000 7fab5659c640 1 --2- 172.21.15.16:0/1752788659 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4337c35d0 0x55c4337c3a10 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7fab48000c00 tx=0x7fab48017730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d38430adc92b3dd server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:15:03.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.391+0000 7fab56d9d640 1 --2- 172.21.15.16:0/2606917357 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fab440a2c00 0x7fab440a50c0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fab5000afd0 tx=0x7fab50005e80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:15:03.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.391+0000 7fab4cff9640 1 -- 172.21.15.16:0/1752788659 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab4801e030 con 0x55c4337c35d0 2024-09-12T02:15:03.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.392+0000 7fab4cff9640 1 -- 172.21.15.16:0/1752788659 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab4809ac40 con 0x55c4337c35d0 2024-09-12T02:15:03.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.392+0000 7fab4cff9640 1 -- 172.21.15.16:0/1752788659 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab480972b0 con 0x55c4337c35d0 2024-09-12T02:15:03.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.392+0000 7fab5ef6e740 1 -- 172.21.15.16:0/1752788659 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4337c35d0 msgr2=0x55c4337c3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:03.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.392+0000 7fab5ef6e740 1 --2- 172.21.15.16:0/1752788659 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4337c35d0 0x55c4337c3a10 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7fab48000c00 tx=0x7fab48017730 comp rx=0 tx=0).stop 2024-09-12T02:15:03.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.392+0000 7fab5ef6e740 1 -- 172.21.15.16:0/1752788659 shutdown_connections 2024-09-12T02:15:03.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.392+0000 7fab5ef6e740 1 --2- 172.21.15.16:0/1752788659 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c4337d2780 0x55c4337d4b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:03.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.392+0000 7fab5ef6e740 1 --2- 172.21.15.16:0/1752788659 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4337c35d0 0x55c4337c3a10 unknown :-1 s=CLOSED pgs=1801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:03.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.392+0000 7fab5ef6e740 1 --2- 172.21.15.16:0/1752788659 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c4337c4b40 0x55c4337c3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:03.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.392+0000 7fab5ef6e740 1 -- 172.21.15.16:0/1752788659 >> 172.21.15.16:0/1752788659 conn(0x55c43378c310 msgr2=0x55c4337bfc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:15:03.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.392+0000 7fab5ef6e740 1 -- 172.21.15.16:0/1752788659 shutdown_connections 2024-09-12T02:15:03.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.392+0000 7fab5ef6e740 1 -- 172.21.15.16:0/1752788659 wait complete. 2024-09-12T02:15:03.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.393+0000 7fab5ef6e740 1 -- start start 2024-09-12T02:15:03.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.393+0000 7fab5ef6e740 1 Processor -- start 2024-09-12T02:15:03.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.393+0000 7fab5ef6e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4337c35d0 0x55c4337d2210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:03.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.393+0000 7fab5ef6e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c4337c4b40 0x55c4337cf4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:03.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.393+0000 7fab5ef6e740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c4337d2780 0x55c4337cfc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:03.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.393+0000 7fab5ef6e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55c4337d5420 con 0x55c4337c35d0 2024-09-12T02:15:03.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.393+0000 7fab5ef6e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55c4336e4060 con 0x55c4337c4b40 2024-09-12T02:15:03.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.393+0000 7fab5ef6e740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55c4337d55a0 con 0x55c4337d2780 2024-09-12T02:15:03.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.393+0000 7fab5659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4337c35d0 0x55c4337d2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:03.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.393+0000 7fab5659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4337c35d0 0x55c4337d2210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33350/0 (socket says 172.21.15.16:33350) 2024-09-12T02:15:03.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.393+0000 7fab5659c640 1 -- 172.21.15.16:0/4106599871 learned_addr learned my addr 172.21.15.16:0/4106599871 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:15:03.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.393+0000 7fab5ef6e740 10 client.0 _ll_register_callbacks cb 0x55c4334378c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:15:03.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.394+0000 7fab4ffff640 1 --2- 172.21.15.16:0/4106599871 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c4337c4b40 0x55c4337cf4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:03.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.394+0000 7fab56d9d640 1 --2- 172.21.15.16:0/4106599871 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c4337d2780 0x55c4337cfc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:03.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.394+0000 7fab5659c640 1 -- 172.21.15.16:0/4106599871 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c4337d2780 msgr2=0x55c4337cfc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:15:03.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.394+0000 7fab5659c640 1 --2- 172.21.15.16:0/4106599871 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c4337d2780 0x55c4337cfc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:03.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.394+0000 7fab5659c640 1 -- 172.21.15.16:0/4106599871 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c4337c4b40 msgr2=0x55c4337cf4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:15:03.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.394+0000 7fab5659c640 1 --2- 172.21.15.16:0/4106599871 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c4337c4b40 0x55c4337cf4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:03.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.394+0000 7fab5659c640 1 -- 172.21.15.16:0/4106599871 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c43349a3c0 con 0x55c4337c35d0 2024-09-12T02:15:03.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.394+0000 7fab56d9d640 1 --2- 172.21.15.16:0/4106599871 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c4337d2780 0x55c4337cfc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:15:03.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.394+0000 7fab4ffff640 1 --2- 172.21.15.16:0/4106599871 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c4337c4b40 0x55c4337cf4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:15:03.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.394+0000 7fab5659c640 1 --2- 172.21.15.16:0/4106599871 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4337c35d0 0x55c4337d2210 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7fab4809a940 tx=0x7fab480a3920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=293b53f7471e57fd server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:15:03.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.394+0000 7fab3dffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:15:03.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.394+0000 7fab3dffb640 1 -- 172.21.15.16:0/4106599871 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab4801e070 con 0x55c4337c35d0 2024-09-12T02:15:03.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.394+0000 7fab3dffb640 1 -- 172.21.15.16:0/4106599871 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fab480972b0 con 0x55c4337c35d0 2024-09-12T02:15:03.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.394+0000 7fab5ef6e740 1 -- 172.21.15.16:0/4106599871 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab4809ac40 con 0x55c4337c35d0 2024-09-12T02:15:03.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.394+0000 7fab3dffb640 1 -- 172.21.15.16:0/4106599871 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fab4809eff0 con 0x55c4337c35d0 2024-09-12T02:15:03.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.394+0000 7fab5ef6e740 10 client.19812 fetch_fsmap learned FSMap version 25 2024-09-12T02:15:03.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.394+0000 7fab5ef6e740 1 -- 172.21.15.16:0/4106599871 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c4337e0f70 con 0x55c4337c35d0 2024-09-12T02:15:03.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.394+0000 7fab3dffb640 1 -- 172.21.15.16:0/4106599871 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab4809ac40 con 0x55c4337c35d0 2024-09-12T02:15:03.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.395+0000 7fab3dffb640 1 -- 172.21.15.16:0/4106599871 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fab48096990 con 0x55c4337c35d0 2024-09-12T02:15:03.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.395+0000 7fab5ef6e740 10 client.19812 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:15:03.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.395+0000 7fab5ef6e740 10 client.19812 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T02:15:03.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.395+0000 7fab5ef6e740 20 client.19812 populate_metadata read hostname 'smithi016' 2024-09-12T02:15:03.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.395+0000 7fab5ef6e740 1 --2- 172.21.15.16:0/4106599871 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x55c4337e22c0 0x55c4337e46b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:03.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.395+0000 7fab5ef6e740 4 client.19812 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:15:03.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.395+0000 7fab5ef6e740 1 -- 172.21.15.16:0/4106599871 --> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c433736860 con 0x55c4337e22c0 2024-09-12T02:15:03.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.395+0000 7fab4ffff640 1 --2- 172.21.15.16:0/4106599871 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x55c4337e22c0 0x55c4337e46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:03.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.396+0000 7fab4ffff640 1 --2- 172.21.15.16:0/4106599871 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x55c4337e22c0 0x55c4337e46b0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=247a077ffd1d53b7 server_cookie=1de185188a4ca488 in_seq=0 out_seq=0 2024-09-12T02:15:03.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.396+0000 7fab3dffb640 10 client.19812 ms_handle_connect on v2:172.21.15.184:6838/4203347941 2024-09-12T02:15:03.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.396+0000 7fab3dffb640 1 -- 172.21.15.16:0/4106599871 <== mds.1 v2:172.21.15.184:6838/4203347941 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c433736860 con 0x55c4337e22c0 2024-09-12T02:15:03.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.396+0000 7fab3dffb640 10 client.19812 handle_command_reply: tid=0 2024-09-12T02:15:03.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.396+0000 7fab5ef6e740 1 client.19812 shutdown 2024-09-12T02:15:03.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.396+0000 7fab5ef6e740 1 -- 172.21.15.16:0/4106599871 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4337c35d0 msgr2=0x55c4337d2210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:03.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.396+0000 7fab5ef6e740 1 --2- 172.21.15.16:0/4106599871 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4337c35d0 0x55c4337d2210 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7fab4809a940 tx=0x7fab480a3920 comp rx=0 tx=0).stop 2024-09-12T02:15:03.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.396+0000 7fab5ef6e740 1 -- 172.21.15.16:0/4106599871 shutdown_connections 2024-09-12T02:15:03.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.396+0000 7fab5ef6e740 1 --2- 172.21.15.16:0/4106599871 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x55c4337e22c0 0x55c4337e46b0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:03.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.396+0000 7fab5ef6e740 1 --2- 172.21.15.16:0/4106599871 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c4337d2780 0x55c4337cfc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:03.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.396+0000 7fab5ef6e740 1 --2- 172.21.15.16:0/4106599871 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c4337c4b40 0x55c4337cf4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:03.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.396+0000 7fab5ef6e740 1 --2- 172.21.15.16:0/4106599871 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4337c35d0 0x55c4337d2210 unknown :-1 s=CLOSED pgs=1802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:03.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.397+0000 7fab5ef6e740 1 -- 172.21.15.16:0/4106599871 >> 172.21.15.16:0/4106599871 conn(0x55c43378c310 msgr2=0x55c4337c5990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:15:03.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.397+0000 7fab3dffb640 0 client.19812 ms_handle_reset on v2:172.21.15.184:6838/4203347941 2024-09-12T02:15:03.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.397+0000 7fab5ef6e740 1 -- 172.21.15.16:0/4106599871 shutdown_connections 2024-09-12T02:15:03.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.397+0000 7fab5ef6e740 1 -- 172.21.15.16:0/4106599871 wait complete. 2024-09-12T02:15:03.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.397+0000 7fab5ef6e740 20 client.19812 trim_cache size 0 max 16384 2024-09-12T02:15:03.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.410+0000 7fab5ef6e740 1 Processor -- start 2024-09-12T02:15:03.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.410+0000 7fab5ef6e740 1 -- start start 2024-09-12T02:15:03.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.410+0000 7fab5ef6e740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c4337c35d0 0x55c4337bfe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:03.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.410+0000 7fab5ef6e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c4337c4b40 0x55c4337e72f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:03.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.410+0000 7fab5ef6e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4337d2780 0x55c4337e7830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:03.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.410+0000 7fab5ef6e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fab581adcb0 con 0x55c4337d2780 2024-09-12T02:15:03.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.410+0000 7fab5ef6e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fab581ab580 con 0x55c4337c4b40 2024-09-12T02:15:03.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.410+0000 7fab5ef6e740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fab581a93d0 con 0x55c4337c35d0 2024-09-12T02:15:03.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.410+0000 7fab56d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4337d2780 0x55c4337e7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:03.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.410+0000 7fab56d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4337d2780 0x55c4337e7830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33364/0 (socket says 172.21.15.16:33364) 2024-09-12T02:15:03.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.410+0000 7fab4ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c4337c4b40 0x55c4337e72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:03.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.410+0000 7fab56d9d640 1 -- 172.21.15.16:0/304569553 learned_addr learned my addr 172.21.15.16:0/304569553 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:15:03.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.410+0000 7fab5659c640 1 --2- 172.21.15.16:0/304569553 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c4337c35d0 0x55c4337bfe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:03.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.410+0000 7fab4ffff640 1 -- 172.21.15.16:0/304569553 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c4337c35d0 msgr2=0x55c4337bfe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:15:03.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.410+0000 7fab4ffff640 1 --2- 172.21.15.16:0/304569553 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c4337c35d0 0x55c4337bfe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:03.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.410+0000 7fab4ffff640 1 -- 172.21.15.16:0/304569553 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4337d2780 msgr2=0x55c4337e7830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:15:03.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.410+0000 7fab4ffff640 1 --2- 172.21.15.16:0/304569553 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4337d2780 0x55c4337e7830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:03.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.410+0000 7fab4ffff640 1 -- 172.21.15.16:0/304569553 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c4337cff80 con 0x55c4337c4b40 2024-09-12T02:15:03.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.410+0000 7fab56d9d640 1 --2- 172.21.15.16:0/304569553 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4337d2780 0x55c4337e7830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:15:03.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.411+0000 7fab4ffff640 1 --2- 172.21.15.16:0/304569553 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c4337c4b40 0x55c4337e72f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fab40006010 tx=0x7fab40031440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3abc5c7bddc6ef64 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:15:03.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.411+0000 7fab3ffff640 1 -- 172.21.15.16:0/304569553 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab40035040 con 0x55c4337c4b40 2024-09-12T02:15:03.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.411+0000 7fab3ffff640 1 -- 172.21.15.16:0/304569553 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab40049070 con 0x55c4337c4b40 2024-09-12T02:15:03.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.411+0000 7fab3ffff640 1 -- 172.21.15.16:0/304569553 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab40004c90 con 0x55c4337c4b40 2024-09-12T02:15:03.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.411+0000 7fab5ef6e740 1 -- 172.21.15.16:0/304569553 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c4337c4b40 msgr2=0x55c4337e72f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:03.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.411+0000 7fab5ef6e740 1 --2- 172.21.15.16:0/304569553 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c4337c4b40 0x55c4337e72f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fab40006010 tx=0x7fab40031440 comp rx=0 tx=0).stop 2024-09-12T02:15:03.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.411+0000 7fab5ef6e740 1 -- 172.21.15.16:0/304569553 shutdown_connections 2024-09-12T02:15:03.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.411+0000 7fab5ef6e740 1 --2- 172.21.15.16:0/304569553 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4337d2780 0x55c4337e7830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:03.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.411+0000 7fab5ef6e740 1 --2- 172.21.15.16:0/304569553 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c4337c4b40 0x55c4337e72f0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:03.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.411+0000 7fab5ef6e740 1 --2- 172.21.15.16:0/304569553 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c4337c35d0 0x55c4337bfe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:03.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.411+0000 7fab5ef6e740 1 -- 172.21.15.16:0/304569553 >> 172.21.15.16:0/304569553 conn(0x55c43378c310 msgr2=0x55c4337bf700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:15:03.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.412+0000 7fab5ef6e740 1 -- 172.21.15.16:0/304569553 shutdown_connections 2024-09-12T02:15:03.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.412+0000 7fab5ef6e740 1 -- 172.21.15.16:0/304569553 wait complete. 2024-09-12T02:15:03.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.412+0000 7fab5ef6e740 1 -- start start 2024-09-12T02:15:03.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.412+0000 7fab5ef6e740 1 Processor -- start 2024-09-12T02:15:03.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.412+0000 7fab5ef6e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4337c35d0 0x55c4337e7670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:03.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.412+0000 7fab5ef6e740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c4337c4b40 0x55c4337e7bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:03.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.412+0000 7fab5ef6e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c4337d2780 0x55c4337e4c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:03.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.412+0000 7fab5ef6e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fab581a9a40 con 0x55c4337c35d0 2024-09-12T02:15:03.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.412+0000 7fab5ef6e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fab581aaeb0 con 0x55c4337d2780 2024-09-12T02:15:03.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.412+0000 7fab5ef6e740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fab581aa110 con 0x55c4337c4b40 2024-09-12T02:15:03.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.412+0000 7fab5ef6e740 10 client.0 _ll_register_callbacks cb 0x7fab5806b7f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:15:03.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.412+0000 7fab5659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4337c35d0 0x55c4337e7670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:03.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.412+0000 7fab5659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4337c35d0 0x55c4337e7670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33368/0 (socket says 172.21.15.16:33368) 2024-09-12T02:15:03.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.412+0000 7fab56d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c4337d2780 0x55c4337e4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:03.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.412+0000 7fab5659c640 1 -- 172.21.15.16:0/4061498076 learned_addr learned my addr 172.21.15.16:0/4061498076 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:15:03.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.413+0000 7fab4ffff640 1 --2- 172.21.15.16:0/4061498076 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c4337c4b40 0x55c4337e7bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:03.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.413+0000 7fab5659c640 1 -- 172.21.15.16:0/4061498076 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c4337c4b40 msgr2=0x55c4337e7bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:15:03.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.413+0000 7fab5659c640 1 --2- 172.21.15.16:0/4061498076 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c4337c4b40 0x55c4337e7bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:03.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.413+0000 7fab5659c640 1 -- 172.21.15.16:0/4061498076 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c4337d2780 msgr2=0x55c4337e4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:15:03.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.413+0000 7fab5659c640 1 --2- 172.21.15.16:0/4061498076 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c4337d2780 0x55c4337e4c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:03.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.413+0000 7fab5659c640 1 -- 172.21.15.16:0/4061498076 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c4337e5440 con 0x55c4337c35d0 2024-09-12T02:15:03.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.413+0000 7fab4ffff640 1 --2- 172.21.15.16:0/4061498076 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c4337c4b40 0x55c4337e7bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:15:03.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.413+0000 7fab56d9d640 1 --2- 172.21.15.16:0/4061498076 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c4337d2780 0x55c4337e4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:15:03.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.413+0000 7fab5659c640 1 --2- 172.21.15.16:0/4061498076 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4337c35d0 0x55c4337e7670 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7fab48097610 tx=0x7fab480047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=743518bb4302d825 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:15:03.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.413+0000 7fab3dffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:15:03.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.413+0000 7fab3dffb640 1 -- 172.21.15.16:0/4061498076 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab48078060 con 0x55c4337c35d0 2024-09-12T02:15:03.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.413+0000 7fab3dffb640 1 -- 172.21.15.16:0/4061498076 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fab480b9600 con 0x55c4337c35d0 2024-09-12T02:15:03.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.413+0000 7fab5ef6e740 1 -- 172.21.15.16:0/4061498076 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab40049070 con 0x55c4337c35d0 2024-09-12T02:15:03.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.413+0000 7fab3dffb640 1 -- 172.21.15.16:0/4061498076 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fab4801e070 con 0x55c4337c35d0 2024-09-12T02:15:03.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.413+0000 7fab5ef6e740 10 client.19818 fetch_fsmap learned FSMap version 25 2024-09-12T02:15:03.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.413+0000 7fab5ef6e740 1 -- 172.21.15.16:0/4061498076 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c43384c840 con 0x55c4337c35d0 2024-09-12T02:15:03.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.414+0000 7fab3dffb640 1 -- 172.21.15.16:0/4061498076 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab40049070 con 0x55c4337c35d0 2024-09-12T02:15:03.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.414+0000 7fab3dffb640 1 -- 172.21.15.16:0/4061498076 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fab4809a090 con 0x55c4337c35d0 2024-09-12T02:15:03.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.414+0000 7fab5ef6e740 10 client.19818 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:15:03.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.414+0000 7fab5ef6e740 10 client.19818 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T02:15:03.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.414+0000 7fab5ef6e740 20 client.19818 populate_metadata read hostname 'smithi016' 2024-09-12T02:15:03.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.414+0000 7fab5ef6e740 1 --2- 172.21.15.16:0/4061498076 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x55c43384cd20 0x55c43384f110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:03.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.414+0000 7fab5ef6e740 4 client.19818 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T02:15:03.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.414+0000 7fab5ef6e740 1 -- 172.21.15.16:0/4061498076 --> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c433730d70 con 0x55c43384cd20 2024-09-12T02:15:03.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.414+0000 7fab4ffff640 1 --2- 172.21.15.16:0/4061498076 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x55c43384cd20 0x55c43384f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:03.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.415+0000 7fab4ffff640 1 --2- 172.21.15.16:0/4061498076 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x55c43384cd20 0x55c43384f110 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=818aefeb8af12cff server_cookie=65657933bc5c13a3 in_seq=0 out_seq=0 2024-09-12T02:15:03.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.415+0000 7fab3dffb640 10 client.19818 ms_handle_connect on v2:172.21.15.184:6838/4203347941 2024-09-12T02:15:03.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.415+0000 7fab3dffb640 1 -- 172.21.15.16:0/4061498076 <== mds.1 v2:172.21.15.184:6838/4203347941 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c433730d70 con 0x55c43384cd20 2024-09-12T02:15:03.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.415+0000 7fab3dffb640 10 client.19818 handle_command_reply: tid=0 2024-09-12T02:15:03.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.415+0000 7fab5ef6e740 1 client.19818 shutdown 2024-09-12T02:15:03.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.415+0000 7fab5ef6e740 1 -- 172.21.15.16:0/4061498076 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4337c35d0 msgr2=0x55c4337e7670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:03.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.415+0000 7fab5ef6e740 1 --2- 172.21.15.16:0/4061498076 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4337c35d0 0x55c4337e7670 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7fab48097610 tx=0x7fab480047d0 comp rx=0 tx=0).stop 2024-09-12T02:15:03.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.415+0000 7fab5ef6e740 1 -- 172.21.15.16:0/4061498076 shutdown_connections 2024-09-12T02:15:03.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.415+0000 7fab5ef6e740 1 --2- 172.21.15.16:0/4061498076 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x55c43384cd20 0x55c43384f110 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:03.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.415+0000 7fab5ef6e740 1 --2- 172.21.15.16:0/4061498076 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c4337d2780 0x55c4337e4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:03.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.415+0000 7fab3dffb640 0 client.19818 ms_handle_reset on v2:172.21.15.184:6838/4203347941 2024-09-12T02:15:03.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.415+0000 7fab5ef6e740 1 --2- 172.21.15.16:0/4061498076 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c4337c4b40 0x55c4337e7bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:03.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.415+0000 7fab5ef6e740 1 --2- 172.21.15.16:0/4061498076 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4337c35d0 0x55c4337e7670 unknown :-1 s=CLOSED pgs=1803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:03.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.415+0000 7fab5ef6e740 1 -- 172.21.15.16:0/4061498076 >> 172.21.15.16:0/4061498076 conn(0x55c43378c310 msgr2=0x55c4337ce5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:15:03.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.415+0000 7fab5ef6e740 1 -- 172.21.15.16:0/4061498076 shutdown_connections 2024-09-12T02:15:03.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.415+0000 7fab5ef6e740 1 -- 172.21.15.16:0/4061498076 wait complete. 2024-09-12T02:15:03.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.415+0000 7fab5ef6e740 20 client.19818 trim_cache size 0 max 16384 2024-09-12T02:15:03.424 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T02:15:03.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.416+0000 7fab5cb95640 1 -- 172.21.15.16:0/2606917357 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fab440a2c00 msgr2=0x7fab440a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:03.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.416+0000 7fab5cb95640 1 --2- 172.21.15.16:0/2606917357 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fab440a2c00 0x7fab440a50c0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fab5000afd0 tx=0x7fab50005e80 comp rx=0 tx=0).stop 2024-09-12T02:15:03.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.416+0000 7fab5cb95640 1 -- 172.21.15.16:0/2606917357 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab58102d40 msgr2=0x7fab581984d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:03.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.416+0000 7fab5cb95640 1 --2- 172.21.15.16:0/2606917357 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab58102d40 0x7fab581984d0 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7fab4800b4a0 tx=0x7fab4800b960 comp rx=0 tx=0).stop 2024-09-12T02:15:03.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.416+0000 7fab5cb95640 1 -- 172.21.15.16:0/2606917357 shutdown_connections 2024-09-12T02:15:03.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.416+0000 7fab5cb95640 1 --2- 172.21.15.16:0/2606917357 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fab440a2c00 0x7fab440a50c0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:03.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.416+0000 7fab5cb95640 1 --2- 172.21.15.16:0/2606917357 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fab5810c170 0x7fab5819cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:03.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.416+0000 7fab5cb95640 1 --2- 172.21.15.16:0/2606917357 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab58103660 0x7fab58198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:03.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.416+0000 7fab5cb95640 1 --2- 172.21.15.16:0/2606917357 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab58102d40 0x7fab581984d0 unknown :-1 s=CLOSED pgs=1800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:03.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.416+0000 7fab5cb95640 1 -- 172.21.15.16:0/2606917357 >> 172.21.15.16:0/2606917357 conn(0x7fab580fc7a0 msgr2=0x7fab580fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:15:03.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.416+0000 7fab5cb95640 1 -- 172.21.15.16:0/2606917357 shutdown_connections 2024-09-12T02:15:03.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:03.416+0000 7fab5cb95640 1 -- 172.21.15.16:0/2606917357 wait complete. 2024-09-12T02:15:04.034 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-12T02:15:04.292 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:15:04.414 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:15:04 smithi016 ceph-mon[30947]: pgmap v729: 97 pgs: 97 active+clean; 3.0 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-09-12T02:15:04.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:15:04 smithi184 ceph-mon[35593]: pgmap v729: 97 pgs: 97 active+clean; 3.0 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-09-12T02:15:04.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:15:04 smithi033 ceph-mon[35332]: pgmap v729: 97 pgs: 97 active+clean; 3.0 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-09-12T02:15:04.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.775+0000 7ff311c82640 1 -- 172.21.15.16:0/949091492 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff30c101420 msgr2=0x7ff30c10d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:04.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.775+0000 7ff311c82640 1 --2- 172.21.15.16:0/949091492 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff30c101420 0x7ff30c10d950 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7ff304009cd0 tx=0x7ff30402a8e0 comp rx=0 tx=0).stop 2024-09-12T02:15:04.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.775+0000 7ff311c82640 1 -- 172.21.15.16:0/949091492 shutdown_connections 2024-09-12T02:15:04.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.775+0000 7ff311c82640 1 --2- 172.21.15.16:0/949091492 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff30c101420 0x7ff30c10d950 unknown :-1 s=CLOSED pgs=1804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:04.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.775+0000 7ff311c82640 1 --2- 172.21.15.16:0/949091492 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff30c100a80 0x7ff30c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:04.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.775+0000 7ff311c82640 1 --2- 172.21.15.16:0/949091492 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff30c106a80 0x7ff30c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:04.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.775+0000 7ff311c82640 1 -- 172.21.15.16:0/949091492 >> 172.21.15.16:0/949091492 conn(0x7ff30c0fc7a0 msgr2=0x7ff30c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:15:04.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.775+0000 7ff311c82640 1 -- 172.21.15.16:0/949091492 shutdown_connections 2024-09-12T02:15:04.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.775+0000 7ff311c82640 1 -- 172.21.15.16:0/949091492 wait complete. 2024-09-12T02:15:04.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.776+0000 7ff311c82640 1 Processor -- start 2024-09-12T02:15:04.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.776+0000 7ff311c82640 1 -- start start 2024-09-12T02:15:04.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.776+0000 7ff311c82640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff30c100a80 0x7ff30c198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:04.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.776+0000 7ff311c82640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff30c101420 0x7ff30c198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:04.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.777+0000 7ff311c82640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff30c106a80 0x7ff30c19cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:04.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.777+0000 7ff311c82640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff30c110e80 con 0x7ff30c101420 2024-09-12T02:15:04.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.777+0000 7ff311c82640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff30c110d00 con 0x7ff30c100a80 2024-09-12T02:15:04.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.777+0000 7ff311c82640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff30c111000 con 0x7ff30c106a80 2024-09-12T02:15:04.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.777+0000 7ff30affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff30c101420 0x7ff30c198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:04.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.777+0000 7ff30affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff30c101420 0x7ff30c198970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33394/0 (socket says 172.21.15.16:33394) 2024-09-12T02:15:04.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.777+0000 7ff30b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff30c100a80 0x7ff30c198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:04.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.777+0000 7ff30affd640 1 -- 172.21.15.16:0/57445823 learned_addr learned my addr 172.21.15.16:0/57445823 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:15:04.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.777+0000 7ff30bfff640 1 --2- 172.21.15.16:0/57445823 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff30c106a80 0x7ff30c19cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:04.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.777+0000 7ff30affd640 1 -- 172.21.15.16:0/57445823 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff30c106a80 msgr2=0x7ff30c19cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:04.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.777+0000 7ff30affd640 1 --2- 172.21.15.16:0/57445823 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff30c106a80 0x7ff30c19cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:04.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.777+0000 7ff30affd640 1 -- 172.21.15.16:0/57445823 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff30c100a80 msgr2=0x7ff30c198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:04.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.777+0000 7ff30affd640 1 --2- 172.21.15.16:0/57445823 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff30c100a80 0x7ff30c198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:04.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.777+0000 7ff30affd640 1 -- 172.21.15.16:0/57445823 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff30c19d3e0 con 0x7ff30c101420 2024-09-12T02:15:04.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.777+0000 7ff30b7fe640 1 --2- 172.21.15.16:0/57445823 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff30c100a80 0x7ff30c198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:15:04.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.777+0000 7ff30affd640 1 --2- 172.21.15.16:0/57445823 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff30c101420 0x7ff30c198970 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7ff2f400c890 tx=0x7ff2f400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:15:04.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.777+0000 7ff308ff9640 1 -- 172.21.15.16:0/57445823 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f4004220 con 0x7ff30c101420 2024-09-12T02:15:04.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.777+0000 7ff311c82640 1 -- 172.21.15.16:0/57445823 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff30c19d6d0 con 0x7ff30c101420 2024-09-12T02:15:04.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.777+0000 7ff308ff9640 1 -- 172.21.15.16:0/57445823 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2f40043c0 con 0x7ff30c101420 2024-09-12T02:15:04.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.777+0000 7ff308ff9640 1 -- 172.21.15.16:0/57445823 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f4004a00 con 0x7ff30c101420 2024-09-12T02:15:04.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.778+0000 7ff311c82640 1 -- 172.21.15.16:0/57445823 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff30c1a4fb0 con 0x7ff30c101420 2024-09-12T02:15:04.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.780+0000 7ff308ff9640 1 -- 172.21.15.16:0/57445823 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff2f40079b0 con 0x7ff30c101420 2024-09-12T02:15:04.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.780+0000 7ff31405b740 1 Processor -- start 2024-09-12T02:15:04.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.780+0000 7ff31405b740 1 -- start start 2024-09-12T02:15:04.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.780+0000 7ff31405b740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56342f275b40 0x56342f274090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:04.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.780+0000 7ff31405b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56342f2745d0 0x56342f274a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:04.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.780+0000 7ff31405b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56342f283780 0x56342f285b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:04.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.780+0000 7ff31405b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56342f1da2f0 con 0x56342f283780 2024-09-12T02:15:04.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.780+0000 7ff31405b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56342f181cf0 con 0x56342f2745d0 2024-09-12T02:15:04.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.780+0000 7ff31405b740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x56342f1d62b0 con 0x56342f275b40 2024-09-12T02:15:04.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.781+0000 7ff30affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56342f283780 0x56342f285b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:04.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.781+0000 7ff30affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56342f283780 0x56342f285b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33396/0 (socket says 172.21.15.16:33396) 2024-09-12T02:15:04.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.781+0000 7ff30bfff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56342f275b40 0x56342f274090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:04.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.781+0000 7ff30affd640 1 -- 172.21.15.16:0/1395897789 learned_addr learned my addr 172.21.15.16:0/1395897789 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:15:04.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.781+0000 7ff30b7fe640 1 --2- 172.21.15.16:0/1395897789 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56342f2745d0 0x56342f274a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:04.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.781+0000 7ff30affd640 1 -- 172.21.15.16:0/1395897789 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56342f275b40 msgr2=0x56342f274090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:15:04.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.781+0000 7ff30affd640 1 --2- 172.21.15.16:0/1395897789 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56342f275b40 0x56342f274090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:04.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.781+0000 7ff30affd640 1 -- 172.21.15.16:0/1395897789 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56342f2745d0 msgr2=0x56342f274a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:15:04.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.781+0000 7ff30affd640 1 --2- 172.21.15.16:0/1395897789 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56342f2745d0 0x56342f274a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:04.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.781+0000 7ff30affd640 1 -- 172.21.15.16:0/1395897789 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56342f1663a0 con 0x56342f283780 2024-09-12T02:15:04.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.781+0000 7ff30b7fe640 1 --2- 172.21.15.16:0/1395897789 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56342f2745d0 0x56342f274a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:15:04.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.781+0000 7ff308ff9640 1 --2- 172.21.15.16:0/57445823 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7ff2f80a2b80 0x7ff2f80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:04.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.781+0000 7ff30bfff640 1 --2- 172.21.15.16:0/57445823 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7ff2f80a2b80 0x7ff2f80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:04.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.781+0000 7ff30affd640 1 --2- 172.21.15.16:0/1395897789 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56342f283780 0x56342f285b70 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7ff2f40d2e80 tx=0x7ff2f40d77d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29b3ef0fa158b70e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:15:04.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.781+0000 7ff308ff9640 1 -- 172.21.15.16:0/57445823 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7ff2f40ceca0 con 0x7ff30c101420 2024-09-12T02:15:04.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.782+0000 7ff301ffb640 1 -- 172.21.15.16:0/1395897789 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f401d020 con 0x56342f283780 2024-09-12T02:15:04.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.782+0000 7ff301ffb640 1 -- 172.21.15.16:0/1395897789 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2f409bca0 con 0x56342f283780 2024-09-12T02:15:04.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.782+0000 7ff301ffb640 1 -- 172.21.15.16:0/1395897789 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f4097840 con 0x56342f283780 2024-09-12T02:15:04.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.782+0000 7ff31405b740 1 -- 172.21.15.16:0/1395897789 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56342f283780 msgr2=0x56342f285b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:04.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.782+0000 7ff30bfff640 1 --2- 172.21.15.16:0/57445823 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7ff2f80a2b80 0x7ff2f80a5040 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7ff304009430 tx=0x7ff304005c50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:15:04.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.782+0000 7ff31405b740 1 --2- 172.21.15.16:0/1395897789 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56342f283780 0x56342f285b70 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7ff2f40d2e80 tx=0x7ff2f40d77d0 comp rx=0 tx=0).stop 2024-09-12T02:15:04.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.782+0000 7ff31405b740 1 -- 172.21.15.16:0/1395897789 shutdown_connections 2024-09-12T02:15:04.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.782+0000 7ff31405b740 1 --2- 172.21.15.16:0/1395897789 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56342f283780 0x56342f285b70 unknown :-1 s=CLOSED pgs=1806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:04.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.782+0000 7ff31405b740 1 --2- 172.21.15.16:0/1395897789 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56342f2745d0 0x56342f274a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:04.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.782+0000 7ff31405b740 1 --2- 172.21.15.16:0/1395897789 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56342f275b40 0x56342f274090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:04.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.782+0000 7ff31405b740 1 -- 172.21.15.16:0/1395897789 >> 172.21.15.16:0/1395897789 conn(0x56342f23d310 msgr2=0x56342f270c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:15:04.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.782+0000 7ff31405b740 1 -- 172.21.15.16:0/1395897789 shutdown_connections 2024-09-12T02:15:04.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.783+0000 7ff31405b740 1 -- 172.21.15.16:0/1395897789 wait complete. 2024-09-12T02:15:04.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.783+0000 7ff31405b740 1 -- start start 2024-09-12T02:15:04.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.783+0000 7ff31405b740 1 Processor -- start 2024-09-12T02:15:04.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.783+0000 7ff31405b740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56342f2745d0 0x56342f283100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:04.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.784+0000 7ff31405b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56342f275b40 0x56342f2803b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:04.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.784+0000 7ff31405b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56342f283780 0x56342f280930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:04.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.784+0000 7ff31405b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56342f286480 con 0x56342f275b40 2024-09-12T02:15:04.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.784+0000 7ff31405b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56342f195060 con 0x56342f283780 2024-09-12T02:15:04.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.784+0000 7ff31405b740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x56342f286600 con 0x56342f2745d0 2024-09-12T02:15:04.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.784+0000 7ff31405b740 10 client.0 _ll_register_callbacks cb 0x56342eee88c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:15:04.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.784+0000 7ff30affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56342f275b40 0x56342f2803b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:04.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.784+0000 7ff30affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56342f275b40 0x56342f2803b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33400/0 (socket says 172.21.15.16:33400) 2024-09-12T02:15:04.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.784+0000 7ff30affd640 1 -- 172.21.15.16:0/337119264 learned_addr learned my addr 172.21.15.16:0/337119264 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:15:04.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.784+0000 7ff30bfff640 1 --2- 172.21.15.16:0/337119264 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56342f283780 0x56342f280930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:04.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.784+0000 7ff30b7fe640 1 --2- 172.21.15.16:0/337119264 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56342f2745d0 0x56342f283100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:04.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.784+0000 7ff30affd640 1 -- 172.21.15.16:0/337119264 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56342f2745d0 msgr2=0x56342f283100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:15:04.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.784+0000 7ff30affd640 1 --2- 172.21.15.16:0/337119264 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56342f2745d0 0x56342f283100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:04.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.784+0000 7ff30affd640 1 -- 172.21.15.16:0/337119264 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56342f283780 msgr2=0x56342f280930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:15:04.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.784+0000 7ff30affd640 1 --2- 172.21.15.16:0/337119264 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56342f283780 0x56342f280930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:04.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.784+0000 7ff30affd640 1 -- 172.21.15.16:0/337119264 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56342ef4b3c0 con 0x56342f275b40 2024-09-12T02:15:04.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.784+0000 7ff30affd640 1 --2- 172.21.15.16:0/337119264 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56342f275b40 0x56342f2803b0 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7ff2f409be50 tx=0x7ff2f40053a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fab69e49ebd8fce server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:15:04.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.784+0000 7ff2eaffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:15:04.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.784+0000 7ff2eaffd640 1 -- 172.21.15.16:0/337119264 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f401d040 con 0x56342f275b40 2024-09-12T02:15:04.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.785+0000 7ff2eaffd640 1 -- 172.21.15.16:0/337119264 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7ff2f4007ca0 con 0x56342f275b40 2024-09-12T02:15:04.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.785+0000 7ff31405b740 1 -- 172.21.15.16:0/337119264 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2f409bca0 con 0x56342f275b40 2024-09-12T02:15:04.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.785+0000 7ff2eaffd640 1 -- 172.21.15.16:0/337119264 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff2f409af00 con 0x56342f275b40 2024-09-12T02:15:04.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.785+0000 7ff31405b740 10 client.19830 fetch_fsmap learned FSMap version 25 2024-09-12T02:15:04.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.785+0000 7ff31405b740 1 -- 172.21.15.16:0/337119264 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56342f286480 con 0x56342f275b40 2024-09-12T02:15:04.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.785+0000 7ff2eaffd640 1 -- 172.21.15.16:0/337119264 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2f409bca0 con 0x56342f275b40 2024-09-12T02:15:04.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.785+0000 7ff2eaffd640 1 -- 172.21.15.16:0/337119264 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7ff2f40d3040 con 0x56342f275b40 2024-09-12T02:15:04.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.785+0000 7ff31405b740 10 client.19830 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:15:04.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.785+0000 7ff31405b740 10 client.19830 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:15:04.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.785+0000 7ff31405b740 20 client.19830 populate_metadata read hostname 'smithi016' 2024-09-12T02:15:04.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.785+0000 7ff31405b740 1 --2- 172.21.15.16:0/337119264 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x56342f293060 0x56342f295450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:04.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.785+0000 7ff31405b740 4 client.19830 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:15:04.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.785+0000 7ff31405b740 1 -- 172.21.15.16:0/337119264 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56342f1e7860 con 0x56342f293060 2024-09-12T02:15:04.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.785+0000 7ff30b7fe640 1 --2- 172.21.15.16:0/337119264 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x56342f293060 0x56342f295450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:04.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.786+0000 7ff30b7fe640 1 --2- 172.21.15.16:0/337119264 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x56342f293060 0x56342f295450 crc :-1 s=READY pgs=393 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58e911d4f780885e server_cookie=9013cb71c6d68449 in_seq=0 out_seq=0 2024-09-12T02:15:04.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.786+0000 7ff2eaffd640 10 client.19830 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:15:04.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.787+0000 7ff2eaffd640 1 -- 172.21.15.16:0/337119264 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56342f1e7860 con 0x56342f293060 2024-09-12T02:15:04.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.787+0000 7ff2eaffd640 10 client.19830 handle_command_reply: tid=0 2024-09-12T02:15:04.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.787+0000 7ff31405b740 1 client.19830 shutdown 2024-09-12T02:15:04.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.787+0000 7ff31405b740 1 -- 172.21.15.16:0/337119264 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56342f275b40 msgr2=0x56342f2803b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:04.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.787+0000 7ff31405b740 1 --2- 172.21.15.16:0/337119264 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56342f275b40 0x56342f2803b0 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7ff2f409be50 tx=0x7ff2f40053a0 comp rx=0 tx=0).stop 2024-09-12T02:15:04.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.787+0000 7ff31405b740 1 -- 172.21.15.16:0/337119264 shutdown_connections 2024-09-12T02:15:04.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.787+0000 7ff31405b740 1 --2- 172.21.15.16:0/337119264 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x56342f293060 0x56342f295450 crc :-1 s=READY pgs=393 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:04.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.787+0000 7ff31405b740 1 --2- 172.21.15.16:0/337119264 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56342f283780 0x56342f280930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:04.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.787+0000 7ff31405b740 1 --2- 172.21.15.16:0/337119264 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56342f275b40 0x56342f2803b0 unknown :-1 s=CLOSED pgs=1807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:04.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.787+0000 7ff31405b740 1 --2- 172.21.15.16:0/337119264 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56342f2745d0 0x56342f283100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:04.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.787+0000 7ff31405b740 1 -- 172.21.15.16:0/337119264 >> 172.21.15.16:0/337119264 conn(0x56342f23d310 msgr2=0x56342f276960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:15:04.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.787+0000 7ff2eaffd640 0 client.19830 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:15:04.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.787+0000 7ff31405b740 1 -- 172.21.15.16:0/337119264 shutdown_connections 2024-09-12T02:15:04.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.787+0000 7ff31405b740 1 -- 172.21.15.16:0/337119264 wait complete. 2024-09-12T02:15:04.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.787+0000 7ff31405b740 20 client.19830 trim_cache size 0 max 16384 2024-09-12T02:15:04.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.800+0000 7ff31405b740 1 Processor -- start 2024-09-12T02:15:04.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.800+0000 7ff31405b740 1 -- start start 2024-09-12T02:15:04.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.800+0000 7ff31405b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56342f2745d0 0x56342f28f750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:04.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.800+0000 7ff31405b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56342f275b40 0x56342f276820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:04.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.800+0000 7ff31405b740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56342f283780 0x56342f276d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:04.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.800+0000 7ff31405b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff30c1adb90 con 0x56342f2745d0 2024-09-12T02:15:04.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.800+0000 7ff31405b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff30c1ab460 con 0x56342f275b40 2024-09-12T02:15:04.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.800+0000 7ff31405b740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff30c1a92d0 con 0x56342f283780 2024-09-12T02:15:04.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.800+0000 7ff30b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56342f2745d0 0x56342f28f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:04.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.800+0000 7ff30b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56342f2745d0 0x56342f28f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33404/0 (socket says 172.21.15.16:33404) 2024-09-12T02:15:04.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.800+0000 7ff30b7fe640 1 -- 172.21.15.16:0/2727311276 learned_addr learned my addr 172.21.15.16:0/2727311276 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:15:04.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.801+0000 7ff30affd640 1 --2- 172.21.15.16:0/2727311276 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56342f275b40 0x56342f276820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:04.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.801+0000 7ff30bfff640 1 --2- 172.21.15.16:0/2727311276 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56342f283780 0x56342f276d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:04.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.801+0000 7ff30b7fe640 1 -- 172.21.15.16:0/2727311276 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56342f283780 msgr2=0x56342f276d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:15:04.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.801+0000 7ff30b7fe640 1 --2- 172.21.15.16:0/2727311276 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56342f283780 0x56342f276d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:04.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.801+0000 7ff30b7fe640 1 -- 172.21.15.16:0/2727311276 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56342f275b40 msgr2=0x56342f276820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:15:04.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.801+0000 7ff30b7fe640 1 --2- 172.21.15.16:0/2727311276 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56342f275b40 0x56342f276820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:04.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.801+0000 7ff30b7fe640 1 -- 172.21.15.16:0/2727311276 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56342f271d80 con 0x56342f2745d0 2024-09-12T02:15:04.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.801+0000 7ff30bfff640 1 --2- 172.21.15.16:0/2727311276 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56342f283780 0x56342f276d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:15:04.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.801+0000 7ff30affd640 1 --2- 172.21.15.16:0/2727311276 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56342f275b40 0x56342f276820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:15:04.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.801+0000 7ff30b7fe640 1 --2- 172.21.15.16:0/2727311276 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56342f2745d0 0x56342f28f750 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7ff2fc000c00 tx=0x7ff2fc00a2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45cb92bd58f06fd2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:15:04.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.801+0000 7ff3017fa640 1 -- 172.21.15.16:0/2727311276 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2fc004220 con 0x56342f2745d0 2024-09-12T02:15:04.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.801+0000 7ff3017fa640 1 -- 172.21.15.16:0/2727311276 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2fc0043c0 con 0x56342f2745d0 2024-09-12T02:15:04.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.801+0000 7ff3017fa640 1 -- 172.21.15.16:0/2727311276 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2fc00f680 con 0x56342f2745d0 2024-09-12T02:15:04.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.802+0000 7ff31405b740 1 -- 172.21.15.16:0/2727311276 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56342f2745d0 msgr2=0x56342f28f750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:04.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.802+0000 7ff31405b740 1 --2- 172.21.15.16:0/2727311276 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56342f2745d0 0x56342f28f750 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7ff2fc000c00 tx=0x7ff2fc00a2d0 comp rx=0 tx=0).stop 2024-09-12T02:15:04.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.802+0000 7ff31405b740 1 -- 172.21.15.16:0/2727311276 shutdown_connections 2024-09-12T02:15:04.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.802+0000 7ff31405b740 1 --2- 172.21.15.16:0/2727311276 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56342f283780 0x56342f276d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:04.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.802+0000 7ff31405b740 1 --2- 172.21.15.16:0/2727311276 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56342f275b40 0x56342f276820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:04.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.802+0000 7ff31405b740 1 --2- 172.21.15.16:0/2727311276 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56342f2745d0 0x56342f28f750 unknown :-1 s=CLOSED pgs=1808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:04.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.802+0000 7ff31405b740 1 -- 172.21.15.16:0/2727311276 >> 172.21.15.16:0/2727311276 conn(0x56342f23d310 msgr2=0x56342f276110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:15:04.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.802+0000 7ff31405b740 1 -- 172.21.15.16:0/2727311276 shutdown_connections 2024-09-12T02:15:04.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.802+0000 7ff31405b740 1 -- 172.21.15.16:0/2727311276 wait complete. 2024-09-12T02:15:04.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.802+0000 7ff31405b740 1 -- start start 2024-09-12T02:15:04.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.802+0000 7ff31405b740 1 Processor -- start 2024-09-12T02:15:04.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.803+0000 7ff31405b740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56342f2745d0 0x56342f28c010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:04.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.803+0000 7ff31405b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56342f275b40 0x56342f27d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:04.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.803+0000 7ff31405b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56342f283780 0x56342f27d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:04.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.803+0000 7ff31405b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff30c1a9920 con 0x56342f275b40 2024-09-12T02:15:04.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.803+0000 7ff31405b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff30c1aad90 con 0x56342f283780 2024-09-12T02:15:04.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.803+0000 7ff31405b740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff30c1a9ff0 con 0x56342f2745d0 2024-09-12T02:15:04.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.803+0000 7ff31405b740 10 client.0 _ll_register_callbacks cb 0x7ff2fc00a940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:15:04.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.803+0000 7ff30affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56342f275b40 0x56342f27d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:04.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.803+0000 7ff30affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56342f275b40 0x56342f27d1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33408/0 (socket says 172.21.15.16:33408) 2024-09-12T02:15:04.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.803+0000 7ff30affd640 1 -- 172.21.15.16:0/4068397598 learned_addr learned my addr 172.21.15.16:0/4068397598 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:15:04.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.803+0000 7ff30b7fe640 1 --2- 172.21.15.16:0/4068397598 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56342f2745d0 0x56342f28c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:04.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.803+0000 7ff30bfff640 1 --2- 172.21.15.16:0/4068397598 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56342f283780 0x56342f27d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:04.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.803+0000 7ff30affd640 1 -- 172.21.15.16:0/4068397598 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56342f2745d0 msgr2=0x56342f28c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:15:04.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.803+0000 7ff30affd640 1 --2- 172.21.15.16:0/4068397598 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56342f2745d0 0x56342f28c010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:04.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.803+0000 7ff30affd640 1 -- 172.21.15.16:0/4068397598 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56342f283780 msgr2=0x56342f27d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:15:04.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.803+0000 7ff30affd640 1 --2- 172.21.15.16:0/4068397598 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56342f283780 0x56342f27d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:04.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.803+0000 7ff30affd640 1 -- 172.21.15.16:0/4068397598 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56342f27df60 con 0x56342f275b40 2024-09-12T02:15:04.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.803+0000 7ff30affd640 1 --2- 172.21.15.16:0/4068397598 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56342f275b40 0x56342f27d1f0 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7ff2f4097f70 tx=0x7ff2f4007730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cc340fdbab32294 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:15:04.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.803+0000 7ff2eaffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:15:04.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.804+0000 7ff2eaffd640 1 -- 172.21.15.16:0/4068397598 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f409b0e0 con 0x56342f275b40 2024-09-12T02:15:04.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.804+0000 7ff2eaffd640 1 -- 172.21.15.16:0/4068397598 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7ff2f409c4c0 con 0x56342f275b40 2024-09-12T02:15:04.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.804+0000 7ff31405b740 1 -- 172.21.15.16:0/4068397598 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2fc0043c0 con 0x56342f275b40 2024-09-12T02:15:04.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.804+0000 7ff2eaffd640 1 -- 172.21.15.16:0/4068397598 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff2f409b370 con 0x56342f275b40 2024-09-12T02:15:04.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.804+0000 7ff31405b740 10 client.19836 fetch_fsmap learned FSMap version 25 2024-09-12T02:15:04.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.804+0000 7ff31405b740 1 -- 172.21.15.16:0/4068397598 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56342f27df60 con 0x56342f275b40 2024-09-12T02:15:04.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.804+0000 7ff2eaffd640 1 -- 172.21.15.16:0/4068397598 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2fc0043c0 con 0x56342f275b40 2024-09-12T02:15:04.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.804+0000 7ff2eaffd640 1 -- 172.21.15.16:0/4068397598 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7ff2f409b800 con 0x56342f275b40 2024-09-12T02:15:04.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.804+0000 7ff31405b740 10 client.19836 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:15:04.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.804+0000 7ff31405b740 10 client.19836 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:15:04.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.804+0000 7ff31405b740 20 client.19836 populate_metadata read hostname 'smithi016' 2024-09-12T02:15:04.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.804+0000 7ff31405b740 1 --2- 172.21.15.16:0/4068397598 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x56342f2fddb0 0x56342f3001a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:04.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.804+0000 7ff31405b740 4 client.19836 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T02:15:04.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.804+0000 7ff31405b740 1 -- 172.21.15.16:0/4068397598 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56342f1e1d70 con 0x56342f2fddb0 2024-09-12T02:15:04.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.804+0000 7ff30b7fe640 1 --2- 172.21.15.16:0/4068397598 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x56342f2fddb0 0x56342f3001a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:04.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.805+0000 7ff30b7fe640 1 --2- 172.21.15.16:0/4068397598 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x56342f2fddb0 0x56342f3001a0 crc :-1 s=READY pgs=394 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c660fabb7859c8ad server_cookie=9b5680092a884a21 in_seq=0 out_seq=0 2024-09-12T02:15:04.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.805+0000 7ff2eaffd640 10 client.19836 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:15:04.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.805+0000 7ff2eaffd640 1 -- 172.21.15.16:0/4068397598 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56342f1e1d70 con 0x56342f2fddb0 2024-09-12T02:15:04.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.805+0000 7ff2eaffd640 10 client.19836 handle_command_reply: tid=0 2024-09-12T02:15:04.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.805+0000 7ff31405b740 1 client.19836 shutdown 2024-09-12T02:15:04.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.805+0000 7ff31405b740 1 -- 172.21.15.16:0/4068397598 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56342f275b40 msgr2=0x56342f27d1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:04.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.805+0000 7ff31405b740 1 --2- 172.21.15.16:0/4068397598 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56342f275b40 0x56342f27d1f0 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7ff2f4097f70 tx=0x7ff2f4007730 comp rx=0 tx=0).stop 2024-09-12T02:15:04.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.805+0000 7ff31405b740 1 -- 172.21.15.16:0/4068397598 shutdown_connections 2024-09-12T02:15:04.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.805+0000 7ff31405b740 1 --2- 172.21.15.16:0/4068397598 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x56342f2fddb0 0x56342f3001a0 crc :-1 s=READY pgs=394 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:04.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.805+0000 7ff31405b740 1 --2- 172.21.15.16:0/4068397598 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56342f283780 0x56342f27d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:04.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.805+0000 7ff31405b740 1 --2- 172.21.15.16:0/4068397598 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56342f275b40 0x56342f27d1f0 unknown :-1 s=CLOSED pgs=1809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:04.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.805+0000 7ff31405b740 1 --2- 172.21.15.16:0/4068397598 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56342f2745d0 0x56342f28c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:04.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.805+0000 7ff31405b740 1 -- 172.21.15.16:0/4068397598 >> 172.21.15.16:0/4068397598 conn(0x56342f23d310 msgr2=0x56342f26fd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:15:04.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.805+0000 7ff2eaffd640 0 client.19836 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:15:04.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.805+0000 7ff31405b740 1 -- 172.21.15.16:0/4068397598 shutdown_connections 2024-09-12T02:15:04.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.805+0000 7ff31405b740 1 -- 172.21.15.16:0/4068397598 wait complete. 2024-09-12T02:15:04.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.806+0000 7ff31405b740 20 client.19836 trim_cache size 0 max 16384 2024-09-12T02:15:04.812 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T02:15:04.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.806+0000 7ff311c82640 1 -- 172.21.15.16:0/57445823 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7ff2f80a2b80 msgr2=0x7ff2f80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:04.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.806+0000 7ff311c82640 1 --2- 172.21.15.16:0/57445823 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7ff2f80a2b80 0x7ff2f80a5040 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7ff304009430 tx=0x7ff304005c50 comp rx=0 tx=0).stop 2024-09-12T02:15:04.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.806+0000 7ff311c82640 1 -- 172.21.15.16:0/57445823 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff30c101420 msgr2=0x7ff30c198970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:04.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.806+0000 7ff311c82640 1 --2- 172.21.15.16:0/57445823 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff30c101420 0x7ff30c198970 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7ff2f400c890 tx=0x7ff2f400cd50 comp rx=0 tx=0).stop 2024-09-12T02:15:04.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.806+0000 7ff311c82640 1 -- 172.21.15.16:0/57445823 shutdown_connections 2024-09-12T02:15:04.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.806+0000 7ff311c82640 1 --2- 172.21.15.16:0/57445823 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7ff2f80a2b80 0x7ff2f80a5040 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:04.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.806+0000 7ff311c82640 1 --2- 172.21.15.16:0/57445823 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff30c106a80 0x7ff30c19cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:04.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.806+0000 7ff311c82640 1 --2- 172.21.15.16:0/57445823 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff30c101420 0x7ff30c198970 unknown :-1 s=CLOSED pgs=1805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:04.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.806+0000 7ff311c82640 1 --2- 172.21.15.16:0/57445823 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff30c100a80 0x7ff30c198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:04.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.806+0000 7ff311c82640 1 -- 172.21.15.16:0/57445823 >> 172.21.15.16:0/57445823 conn(0x7ff30c0fc7a0 msgr2=0x7ff30c06a8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:15:04.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.806+0000 7ff311c82640 1 -- 172.21.15.16:0/57445823 shutdown_connections 2024-09-12T02:15:04.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:04.806+0000 7ff311c82640 1 -- 172.21.15.16:0/57445823 wait complete. 2024-09-12T02:15:06.244 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.242+0000 7feb6c18d640 1 -- 172.21.15.16:0/18323637 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb64100d10 msgr2=0x7feb64101170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:06.244 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.242+0000 7feb6c18d640 1 --2- 172.21.15.16:0/18323637 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb64100d10 0x7feb64101170 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7feb58009900 tx=0x7feb5802a410 comp rx=0 tx=0).stop 2024-09-12T02:15:06.245 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.242+0000 7feb6c18d640 1 -- 172.21.15.16:0/18323637 shutdown_connections 2024-09-12T02:15:06.245 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.242+0000 7feb6c18d640 1 --2- 172.21.15.16:0/18323637 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feb641016b0 0x7feb64101b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:06.245 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.242+0000 7feb6c18d640 1 --2- 172.21.15.16:0/18323637 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb64100d10 0x7feb64101170 unknown :-1 s=CLOSED pgs=1810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:06.245 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.242+0000 7feb6c18d640 1 --2- 172.21.15.16:0/18323637 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb641003d0 0x7feb641007d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:06.245 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.242+0000 7feb6c18d640 1 -- 172.21.15.16:0/18323637 >> 172.21.15.16:0/18323637 conn(0x7feb640fc1e0 msgr2=0x7feb640fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:15:06.245 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.242+0000 7feb6c18d640 1 -- 172.21.15.16:0/18323637 shutdown_connections 2024-09-12T02:15:06.245 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.242+0000 7feb6c18d640 1 -- 172.21.15.16:0/18323637 wait complete. 2024-09-12T02:15:06.245 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.243+0000 7feb6c18d640 1 Processor -- start 2024-09-12T02:15:06.245 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.243+0000 7feb6c18d640 1 -- start start 2024-09-12T02:15:06.245 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.243+0000 7feb6c18d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb641003d0 0x7feb6419a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:06.245 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.243+0000 7feb6c18d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb64100d10 0x7feb6419a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:06.245 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.243+0000 7feb6c18d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feb641016b0 0x7feb641a15b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:06.246 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.243+0000 7feb6c18d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7feb6410fa20 con 0x7feb64100d10 2024-09-12T02:15:06.246 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.243+0000 7feb6c18d640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7feb6410f8a0 con 0x7feb641003d0 2024-09-12T02:15:06.246 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.243+0000 7feb6c18d640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7feb6410fba0 con 0x7feb641016b0 2024-09-12T02:15:06.246 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.243+0000 7feb69728640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb64100d10 0x7feb6419a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:06.246 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.243+0000 7feb69728640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb64100d10 0x7feb6419a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33430/0 (socket says 172.21.15.16:33430) 2024-09-12T02:15:06.246 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.243+0000 7feb69728640 1 -- 172.21.15.16:0/3302527759 learned_addr learned my addr 172.21.15.16:0/3302527759 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:15:06.246 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.243+0000 7feb69f29640 1 --2- 172.21.15.16:0/3302527759 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb641003d0 0x7feb6419a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:06.246 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.243+0000 7feb6a72a640 1 --2- 172.21.15.16:0/3302527759 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feb641016b0 0x7feb641a15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:06.246 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.243+0000 7feb69728640 1 -- 172.21.15.16:0/3302527759 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feb641016b0 msgr2=0x7feb641a15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:06.246 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.243+0000 7feb69728640 1 --2- 172.21.15.16:0/3302527759 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feb641016b0 0x7feb641a15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:06.246 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.243+0000 7feb69728640 1 -- 172.21.15.16:0/3302527759 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb641003d0 msgr2=0x7feb6419a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:06.246 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.243+0000 7feb69728640 1 --2- 172.21.15.16:0/3302527759 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb641003d0 0x7feb6419a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:06.247 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.244+0000 7feb69728640 1 -- 172.21.15.16:0/3302527759 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb641a1c20 con 0x7feb64100d10 2024-09-12T02:15:06.247 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.244+0000 7feb69728640 1 --2- 172.21.15.16:0/3302527759 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb64100d10 0x7feb6419a540 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7feb58031440 tx=0x7feb58032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:15:06.247 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.244+0000 7feb56ffd640 1 -- 172.21.15.16:0/3302527759 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb5803f030 con 0x7feb64100d10 2024-09-12T02:15:06.247 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.244+0000 7feb56ffd640 1 -- 172.21.15.16:0/3302527759 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb58003ce0 con 0x7feb64100d10 2024-09-12T02:15:06.247 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.244+0000 7feb6c18d640 1 -- 172.21.15.16:0/3302527759 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb641a1eb0 con 0x7feb64100d10 2024-09-12T02:15:06.247 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.244+0000 7feb56ffd640 1 -- 172.21.15.16:0/3302527759 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb58035070 con 0x7feb64100d10 2024-09-12T02:15:06.247 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.244+0000 7feb6c18d640 1 -- 172.21.15.16:0/3302527759 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb641a2320 con 0x7feb64100d10 2024-09-12T02:15:06.249 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.246+0000 7feb56ffd640 1 -- 172.21.15.16:0/3302527759 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7feb58042030 con 0x7feb64100d10 2024-09-12T02:15:06.249 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.247+0000 7feb6f007740 1 Processor -- start 2024-09-12T02:15:06.249 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.247+0000 7feb6f007740 1 -- start start 2024-09-12T02:15:06.249 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.247+0000 7feb56ffd640 1 --2- 172.21.15.16:0/3302527759 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7feb500a2cb0 0x7feb500a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:06.250 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.247+0000 7feb6f007740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5627c3f609a0 0x5627c3f60da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:06.250 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.247+0000 7feb6f007740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5627c3f612e0 0x5627c3f65da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:06.250 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.247+0000 7feb6f007740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627c3f66370 0x5627c3f68830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:06.250 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.247+0000 7feb6f007740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5627c3ec42f0 con 0x5627c3f66370 2024-09-12T02:15:06.250 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.247+0000 7feb6f007740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5627c3e6c270 con 0x5627c3f612e0 2024-09-12T02:15:06.250 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.247+0000 7feb6f007740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5627c3ec02b0 con 0x5627c3f609a0 2024-09-12T02:15:06.250 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.248+0000 7feb6a72a640 1 --2- 172.21.15.16:0/3302527759 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7feb500a2cb0 0x7feb500a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:06.250 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.248+0000 7feb6a72a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627c3f66370 0x5627c3f68830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:06.250 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.248+0000 7feb69728640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5627c3f612e0 0x5627c3f65da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:06.250 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.248+0000 7feb69f29640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5627c3f609a0 0x5627c3f60da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:06.250 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.248+0000 7feb69f29640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5627c3f609a0 0x5627c3f60da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:37262/0 (socket says 172.21.15.16:37262) 2024-09-12T02:15:06.250 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.248+0000 7feb69f29640 1 -- 172.21.15.16:0/2266129674 learned_addr learned my addr 172.21.15.16:0/2266129674 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:15:06.251 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.248+0000 7feb56ffd640 1 -- 172.21.15.16:0/3302527759 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7feb580ef4d0 con 0x7feb64100d10 2024-09-12T02:15:06.251 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.248+0000 7feb69f29640 1 -- 172.21.15.16:0/2266129674 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5627c3f612e0 msgr2=0x5627c3f65da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:15:06.251 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.248+0000 7feb69f29640 1 --2- 172.21.15.16:0/2266129674 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5627c3f612e0 0x5627c3f65da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:06.251 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.248+0000 7feb69f29640 1 -- 172.21.15.16:0/2266129674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627c3f66370 msgr2=0x5627c3f68830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:15:06.251 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.248+0000 7feb69f29640 1 --2- 172.21.15.16:0/2266129674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627c3f66370 0x5627c3f68830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:06.251 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.248+0000 7feb69f29640 1 -- 172.21.15.16:0/2266129674 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5627c3e6b340 con 0x5627c3f609a0 2024-09-12T02:15:06.251 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.248+0000 7feb6a72a640 1 --2- 172.21.15.16:0/3302527759 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7feb500a2cb0 0x7feb500a5170 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7feb5c005c40 tx=0x7feb5c005bd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:15:06.251 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.248+0000 7feb69f29640 1 --2- 172.21.15.16:0/2266129674 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5627c3f609a0 0x5627c3f60da0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7feb6000bd50 tx=0x7feb600107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d10dbdac760b804a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:15:06.252 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.252+0000 7feb4ffff640 1 -- 172.21.15.16:0/2266129674 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb60005220 con 0x5627c3f609a0 2024-09-12T02:15:06.252 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.252+0000 7feb4ffff640 1 -- 172.21.15.16:0/2266129674 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb6000be40 con 0x5627c3f609a0 2024-09-12T02:15:06.252 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.252+0000 7feb4ffff640 1 -- 172.21.15.16:0/2266129674 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb600117c0 con 0x5627c3f609a0 2024-09-12T02:15:06.253 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.253+0000 7feb6f007740 1 -- 172.21.15.16:0/2266129674 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5627c3f609a0 msgr2=0x5627c3f60da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:06.253 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.253+0000 7feb6f007740 1 --2- 172.21.15.16:0/2266129674 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5627c3f609a0 0x5627c3f60da0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7feb6000bd50 tx=0x7feb600107a0 comp rx=0 tx=0).stop 2024-09-12T02:15:06.254 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.253+0000 7feb6f007740 1 -- 172.21.15.16:0/2266129674 shutdown_connections 2024-09-12T02:15:06.254 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.253+0000 7feb6f007740 1 --2- 172.21.15.16:0/2266129674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627c3f66370 0x5627c3f68830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:06.254 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.253+0000 7feb6f007740 1 --2- 172.21.15.16:0/2266129674 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5627c3f612e0 0x5627c3f65da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:06.254 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.253+0000 7feb6f007740 1 --2- 172.21.15.16:0/2266129674 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5627c3f609a0 0x5627c3f60da0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:06.254 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.253+0000 7feb6f007740 1 -- 172.21.15.16:0/2266129674 >> 172.21.15.16:0/2266129674 conn(0x5627c3f5c010 msgr2=0x5627c3f5c410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:15:06.254 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.253+0000 7feb6f007740 1 -- 172.21.15.16:0/2266129674 shutdown_connections 2024-09-12T02:15:06.254 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.253+0000 7feb6f007740 1 -- 172.21.15.16:0/2266129674 wait complete. 2024-09-12T02:15:06.254 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.253+0000 7feb6f007740 1 -- start start 2024-09-12T02:15:06.254 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.253+0000 7feb6f007740 1 Processor -- start 2024-09-12T02:15:06.258 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.254+0000 7feb6f007740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5627c3f609a0 0x5627c3f6ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:06.258 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.254+0000 7feb6f007740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627c3f612e0 0x5627c3f6b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:06.258 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.254+0000 7feb6f007740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5627c3f66370 0x5627c3f720e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:06.258 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.254+0000 7feb6f007740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5627c3f69340 con 0x5627c3f612e0 2024-09-12T02:15:06.258 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.254+0000 7feb6f007740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5627c3e4d290 con 0x5627c3f66370 2024-09-12T02:15:06.258 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.254+0000 7feb6f007740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5627c3f694c0 con 0x5627c3f609a0 2024-09-12T02:15:06.258 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.254+0000 7feb6f007740 10 client.0 _ll_register_callbacks cb 0x5627c3e7e140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:15:06.258 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.254+0000 7feb69728640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627c3f612e0 0x5627c3f6b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:06.259 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.254+0000 7feb69728640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627c3f612e0 0x5627c3f6b0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33456/0 (socket says 172.21.15.16:33456) 2024-09-12T02:15:06.259 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.254+0000 7feb69728640 1 -- 172.21.15.16:0/1937980280 learned_addr learned my addr 172.21.15.16:0/1937980280 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:15:06.259 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.254+0000 7feb69f29640 1 --2- 172.21.15.16:0/1937980280 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5627c3f609a0 0x5627c3f6ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:06.259 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.254+0000 7feb6a72a640 1 --2- 172.21.15.16:0/1937980280 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5627c3f66370 0x5627c3f720e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:06.259 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.254+0000 7feb69728640 1 -- 172.21.15.16:0/1937980280 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5627c3f609a0 msgr2=0x5627c3f6ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:15:06.259 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.254+0000 7feb69728640 1 --2- 172.21.15.16:0/1937980280 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5627c3f609a0 0x5627c3f6ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:06.259 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.254+0000 7feb69728640 1 -- 172.21.15.16:0/1937980280 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5627c3f66370 msgr2=0x5627c3f720e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:15:06.259 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.254+0000 7feb69728640 1 --2- 172.21.15.16:0/1937980280 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5627c3f66370 0x5627c3f720e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:06.259 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.254+0000 7feb69728640 1 -- 172.21.15.16:0/1937980280 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5627c3bd7210 con 0x5627c3f612e0 2024-09-12T02:15:06.259 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.255+0000 7feb69728640 1 --2- 172.21.15.16:0/1937980280 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627c3f612e0 0x5627c3f6b0a0 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7feb580b8240 tx=0x7feb580bb9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2757f5ef627d6d4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:15:06.259 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.255+0000 7feb4d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:15:06.259 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.255+0000 7feb4d7fa640 1 -- 172.21.15.16:0/1937980280 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb58049070 con 0x5627c3f612e0 2024-09-12T02:15:06.259 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.255+0000 7feb4d7fa640 1 -- 172.21.15.16:0/1937980280 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7feb5803bd40 con 0x5627c3f612e0 2024-09-12T02:15:06.259 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.255+0000 7feb6f007740 1 -- 172.21.15.16:0/1937980280 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5627c3ed1a20 con 0x5627c3f612e0 2024-09-12T02:15:06.260 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.255+0000 7feb4d7fa640 1 -- 172.21.15.16:0/1937980280 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7feb58005d50 con 0x5627c3f612e0 2024-09-12T02:15:06.260 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.255+0000 7feb6f007740 10 client.19848 fetch_fsmap learned FSMap version 25 2024-09-12T02:15:06.260 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.255+0000 7feb6f007740 1 -- 172.21.15.16:0/1937980280 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5627c3ca3620 con 0x5627c3f612e0 2024-09-12T02:15:06.260 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.255+0000 7feb4d7fa640 1 -- 172.21.15.16:0/1937980280 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5627c3ed1a20 con 0x5627c3f612e0 2024-09-12T02:15:06.260 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.255+0000 7feb4d7fa640 1 -- 172.21.15.16:0/1937980280 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7feb58084e30 con 0x5627c3f612e0 2024-09-12T02:15:06.260 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.255+0000 7feb6f007740 10 client.19848 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:15:06.260 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.255+0000 7feb6f007740 10 client.19848 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:15:06.260 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.255+0000 7feb6f007740 20 client.19848 populate_metadata read hostname 'smithi016' 2024-09-12T02:15:06.260 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.255+0000 7feb6f007740 1 --2- 172.21.15.16:0/1937980280 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5627c3f76860 0x5627c3f78ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:06.260 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.255+0000 7feb6f007740 4 client.19848 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:15:06.260 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.255+0000 7feb6f007740 1 -- 172.21.15.16:0/1937980280 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5627c3ecbd70 con 0x5627c3f76860 2024-09-12T02:15:06.261 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.256+0000 7feb69f29640 1 --2- 172.21.15.16:0/1937980280 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5627c3f76860 0x5627c3f78ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:06.261 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.256+0000 7feb69f29640 1 --2- 172.21.15.16:0/1937980280 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5627c3f76860 0x5627c3f78ce0 crc :-1 s=READY pgs=395 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90348d1d9369427d server_cookie=29060e7cf52ad483 in_seq=0 out_seq=0 2024-09-12T02:15:06.261 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.256+0000 7feb4d7fa640 10 client.19848 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:15:06.261 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.257+0000 7feb4d7fa640 1 -- 172.21.15.16:0/1937980280 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5627c3ecbd70 con 0x5627c3f76860 2024-09-12T02:15:06.261 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.257+0000 7feb4d7fa640 10 client.19848 handle_command_reply: tid=0 2024-09-12T02:15:06.261 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.257+0000 7feb6f007740 1 client.19848 shutdown 2024-09-12T02:15:06.261 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.257+0000 7feb6f007740 1 -- 172.21.15.16:0/1937980280 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627c3f612e0 msgr2=0x5627c3f6b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:06.261 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.257+0000 7feb6f007740 1 --2- 172.21.15.16:0/1937980280 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627c3f612e0 0x5627c3f6b0a0 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7feb580b8240 tx=0x7feb580bb9a0 comp rx=0 tx=0).stop 2024-09-12T02:15:06.261 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.257+0000 7feb6f007740 1 -- 172.21.15.16:0/1937980280 shutdown_connections 2024-09-12T02:15:06.261 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.257+0000 7feb6f007740 1 --2- 172.21.15.16:0/1937980280 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5627c3f76860 0x5627c3f78ce0 crc :-1 s=READY pgs=395 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:06.261 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.257+0000 7feb6f007740 1 --2- 172.21.15.16:0/1937980280 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5627c3f66370 0x5627c3f720e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:06.261 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.257+0000 7feb6f007740 1 --2- 172.21.15.16:0/1937980280 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627c3f612e0 0x5627c3f6b0a0 unknown :-1 s=CLOSED pgs=1812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:06.261 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.257+0000 7feb4d7fa640 0 client.19848 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:15:06.262 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.257+0000 7feb6f007740 1 --2- 172.21.15.16:0/1937980280 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5627c3f609a0 0x5627c3f6ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:06.262 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.257+0000 7feb6f007740 1 -- 172.21.15.16:0/1937980280 >> 172.21.15.16:0/1937980280 conn(0x5627c3f5c010 msgr2=0x5627c3f5d820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:15:06.262 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.257+0000 7feb6f007740 1 -- 172.21.15.16:0/1937980280 shutdown_connections 2024-09-12T02:15:06.262 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.257+0000 7feb6f007740 1 -- 172.21.15.16:0/1937980280 wait complete. 2024-09-12T02:15:06.262 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.257+0000 7feb6f007740 20 client.19848 trim_cache size 0 max 16384 2024-09-12T02:15:06.277 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.272+0000 7feb6f007740 1 Processor -- start 2024-09-12T02:15:06.277 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.272+0000 7feb6f007740 1 -- start start 2024-09-12T02:15:06.277 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.272+0000 7feb6f007740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5627c3f609a0 0x5627c3f5cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:06.277 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.272+0000 7feb6f007740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627c3f612e0 0x5627c3f720a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:06.277 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.272+0000 7feb6f007740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5627c3f66370 0x5627c3f5d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:06.277 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.272+0000 7feb6f007740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7feb641ad860 con 0x5627c3f612e0 2024-09-12T02:15:06.277 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.272+0000 7feb6f007740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7feb641ab130 con 0x5627c3f66370 2024-09-12T02:15:06.277 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.272+0000 7feb6f007740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7feb641a8fa0 con 0x5627c3f609a0 2024-09-12T02:15:06.277 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.272+0000 7feb69728640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627c3f612e0 0x5627c3f720a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:06.278 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.272+0000 7feb69728640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627c3f612e0 0x5627c3f720a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33468/0 (socket says 172.21.15.16:33468) 2024-09-12T02:15:06.278 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.272+0000 7feb69728640 1 -- 172.21.15.16:0/737421794 learned_addr learned my addr 172.21.15.16:0/737421794 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:15:06.278 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.272+0000 7feb69728640 1 -- 172.21.15.16:0/737421794 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5627c3f609a0 msgr2=0x5627c3f5cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:15:06.278 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.272+0000 7feb69728640 1 --2- 172.21.15.16:0/737421794 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5627c3f609a0 0x5627c3f5cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:06.278 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.272+0000 7feb69728640 1 -- 172.21.15.16:0/737421794 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5627c3f66370 msgr2=0x5627c3f5d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:15:06.278 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.272+0000 7feb6a72a640 1 --2- 172.21.15.16:0/737421794 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5627c3f66370 0x5627c3f5d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:06.278 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.273+0000 7feb69728640 1 --2- 172.21.15.16:0/737421794 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5627c3f66370 0x5627c3f5d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:06.278 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.273+0000 7feb69728640 1 -- 172.21.15.16:0/737421794 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5627c3f5be30 con 0x5627c3f612e0 2024-09-12T02:15:06.278 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.273+0000 7feb6a72a640 1 --2- 172.21.15.16:0/737421794 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5627c3f66370 0x5627c3f5d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:15:06.278 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.273+0000 7feb69728640 1 --2- 172.21.15.16:0/737421794 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627c3f612e0 0x5627c3f720a0 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7feb58005c20 tx=0x7feb580bbbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d62716fbe83354c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:15:06.278 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.273+0000 7feb4f7fe640 1 -- 172.21.15.16:0/737421794 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb58049050 con 0x5627c3f612e0 2024-09-12T02:15:06.278 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.273+0000 7feb4f7fe640 1 -- 172.21.15.16:0/737421794 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb580c00e0 con 0x5627c3f612e0 2024-09-12T02:15:06.279 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.273+0000 7feb4f7fe640 1 -- 172.21.15.16:0/737421794 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb580ed050 con 0x5627c3f612e0 2024-09-12T02:15:06.279 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.273+0000 7feb6f007740 1 -- 172.21.15.16:0/737421794 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627c3f612e0 msgr2=0x5627c3f720a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:06.279 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.273+0000 7feb6f007740 1 --2- 172.21.15.16:0/737421794 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627c3f612e0 0x5627c3f720a0 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7feb58005c20 tx=0x7feb580bbbb0 comp rx=0 tx=0).stop 2024-09-12T02:15:06.279 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.273+0000 7feb6f007740 1 -- 172.21.15.16:0/737421794 shutdown_connections 2024-09-12T02:15:06.279 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.273+0000 7feb6f007740 1 --2- 172.21.15.16:0/737421794 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5627c3f66370 0x5627c3f5d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:06.279 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.273+0000 7feb6f007740 1 --2- 172.21.15.16:0/737421794 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627c3f612e0 0x5627c3f720a0 unknown :-1 s=CLOSED pgs=1813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:06.279 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.273+0000 7feb6f007740 1 --2- 172.21.15.16:0/737421794 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5627c3f609a0 0x5627c3f5cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:06.279 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.273+0000 7feb6f007740 1 -- 172.21.15.16:0/737421794 >> 172.21.15.16:0/737421794 conn(0x5627c3f5c010 msgr2=0x5627c3f62420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:15:06.279 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.274+0000 7feb6f007740 1 -- 172.21.15.16:0/737421794 shutdown_connections 2024-09-12T02:15:06.279 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.274+0000 7feb6f007740 1 -- 172.21.15.16:0/737421794 wait complete. 2024-09-12T02:15:06.279 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.274+0000 7feb6f007740 1 -- start start 2024-09-12T02:15:06.279 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.274+0000 7feb6f007740 1 Processor -- start 2024-09-12T02:15:06.279 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.274+0000 7feb6f007740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5627c3f609a0 0x5627c3f6fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:06.280 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.274+0000 7feb6f007740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627c3f612e0 0x5627c3f70190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:06.280 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.274+0000 7feb6f007740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5627c3f66370 0x5627c3f711a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:06.280 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.274+0000 7feb6f007740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7feb641a95f0 con 0x5627c3f612e0 2024-09-12T02:15:06.280 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.274+0000 7feb6f007740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7feb641aaa60 con 0x5627c3f66370 2024-09-12T02:15:06.280 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.274+0000 7feb6f007740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7feb641a9cc0 con 0x5627c3f609a0 2024-09-12T02:15:06.280 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.274+0000 7feb6f007740 10 client.0 _ll_register_callbacks cb 0x7feb58005b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:15:06.280 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.274+0000 7feb69728640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627c3f612e0 0x5627c3f70190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:06.280 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.274+0000 7feb69728640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627c3f612e0 0x5627c3f70190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33478/0 (socket says 172.21.15.16:33478) 2024-09-12T02:15:06.280 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.275+0000 7feb69728640 1 -- 172.21.15.16:0/2159074024 learned_addr learned my addr 172.21.15.16:0/2159074024 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:15:06.280 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.275+0000 7feb6a72a640 1 --2- 172.21.15.16:0/2159074024 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5627c3f66370 0x5627c3f711a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:06.280 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.275+0000 7feb69f29640 1 --2- 172.21.15.16:0/2159074024 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5627c3f609a0 0x5627c3f6fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:06.280 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.275+0000 7feb69728640 1 -- 172.21.15.16:0/2159074024 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5627c3f609a0 msgr2=0x5627c3f6fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:15:06.280 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.275+0000 7feb69728640 1 --2- 172.21.15.16:0/2159074024 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5627c3f609a0 0x5627c3f6fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:06.281 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.275+0000 7feb69728640 1 -- 172.21.15.16:0/2159074024 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5627c3f66370 msgr2=0x5627c3f711a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:15:06.281 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.275+0000 7feb69728640 1 --2- 172.21.15.16:0/2159074024 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5627c3f66370 0x5627c3f711a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:06.281 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.275+0000 7feb69728640 1 -- 172.21.15.16:0/2159074024 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5627c3f717b0 con 0x5627c3f612e0 2024-09-12T02:15:06.281 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.275+0000 7feb69f29640 1 --2- 172.21.15.16:0/2159074024 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5627c3f609a0 0x5627c3f6fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:15:06.281 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.275+0000 7feb69728640 1 --2- 172.21.15.16:0/2159074024 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627c3f612e0 0x5627c3f70190 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7feb580ef0f0 tx=0x7feb58032f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b83bc789d882ceb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:15:06.281 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.275+0000 7feb4d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:15:06.281 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.275+0000 7feb4d7fa640 1 -- 172.21.15.16:0/2159074024 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb58049050 con 0x5627c3f612e0 2024-09-12T02:15:06.281 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.275+0000 7feb4d7fa640 1 -- 172.21.15.16:0/2159074024 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7feb5803b460 con 0x5627c3f612e0 2024-09-12T02:15:06.281 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.275+0000 7feb6f007740 1 -- 172.21.15.16:0/2159074024 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb580c00e0 con 0x5627c3f612e0 2024-09-12T02:15:06.281 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.275+0000 7feb4d7fa640 1 -- 172.21.15.16:0/2159074024 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7feb5803b6a0 con 0x5627c3f612e0 2024-09-12T02:15:06.281 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.275+0000 7feb6f007740 10 client.19854 fetch_fsmap learned FSMap version 25 2024-09-12T02:15:06.281 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.275+0000 7feb6f007740 1 -- 172.21.15.16:0/2159074024 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5627c3f717b0 con 0x5627c3f612e0 2024-09-12T02:15:06.282 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.276+0000 7feb4d7fa640 1 -- 172.21.15.16:0/2159074024 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb580c00e0 con 0x5627c3f612e0 2024-09-12T02:15:06.282 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.276+0000 7feb4d7fa640 1 -- 172.21.15.16:0/2159074024 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7feb580a2030 con 0x5627c3f612e0 2024-09-12T02:15:06.282 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.276+0000 7feb6f007740 10 client.19854 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:15:06.282 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.276+0000 7feb6f007740 10 client.19854 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:15:06.282 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.276+0000 7feb6f007740 20 client.19854 populate_metadata read hostname 'smithi016' 2024-09-12T02:15:06.282 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.276+0000 7feb6f007740 1 --2- 172.21.15.16:0/2159074024 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5627c3fe1f30 0x5627c3fe4350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:06.282 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.276+0000 7feb6f007740 4 client.19854 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:15:06.282 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.276+0000 7feb6f007740 1 -- 172.21.15.16:0/2159074024 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5627c3e48900 con 0x5627c3fe1f30 2024-09-12T02:15:06.282 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.276+0000 7feb69f29640 1 --2- 172.21.15.16:0/2159074024 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5627c3fe1f30 0x5627c3fe4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:06.282 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.276+0000 7feb4d7fa640 10 client.19854 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:15:06.282 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.276+0000 7feb69f29640 1 --2- 172.21.15.16:0/2159074024 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5627c3fe1f30 0x5627c3fe4350 crc :-1 s=READY pgs=396 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3aeb24addbbe6408 server_cookie=12f059ba7764b47d in_seq=0 out_seq=0 2024-09-12T02:15:06.292 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.292+0000 7feb4d7fa640 1 -- 172.21.15.16:0/2159074024 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5627c3e48900 con 0x5627c3fe1f30 2024-09-12T02:15:06.293 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.292+0000 7feb4d7fa640 10 client.19854 handle_command_reply: tid=0 2024-09-12T02:15:06.293 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.292+0000 7feb6f007740 1 client.19854 shutdown 2024-09-12T02:15:06.293 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.292+0000 7feb6f007740 1 -- 172.21.15.16:0/2159074024 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627c3f612e0 msgr2=0x5627c3f70190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:06.293 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.292+0000 7feb6f007740 1 --2- 172.21.15.16:0/2159074024 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627c3f612e0 0x5627c3f70190 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7feb580ef0f0 tx=0x7feb58032f90 comp rx=0 tx=0).stop 2024-09-12T02:15:06.293 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.292+0000 7feb6f007740 1 -- 172.21.15.16:0/2159074024 shutdown_connections 2024-09-12T02:15:06.293 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.292+0000 7feb6f007740 1 --2- 172.21.15.16:0/2159074024 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5627c3fe1f30 0x5627c3fe4350 crc :-1 s=READY pgs=396 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:06.293 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.292+0000 7feb6f007740 1 --2- 172.21.15.16:0/2159074024 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5627c3f66370 0x5627c3f711a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:06.293 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.292+0000 7feb6f007740 1 --2- 172.21.15.16:0/2159074024 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627c3f612e0 0x5627c3f70190 unknown :-1 s=CLOSED pgs=1814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:06.293 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.292+0000 7feb4d7fa640 0 client.19854 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:15:06.293 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.292+0000 7feb6f007740 1 --2- 172.21.15.16:0/2159074024 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5627c3f609a0 0x5627c3f6fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:06.293 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.292+0000 7feb6f007740 1 -- 172.21.15.16:0/2159074024 >> 172.21.15.16:0/2159074024 conn(0x5627c3f5c010 msgr2=0x5627c3f5a9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:15:06.294 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.292+0000 7feb6f007740 1 -- 172.21.15.16:0/2159074024 shutdown_connections 2024-09-12T02:15:06.294 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.292+0000 7feb6f007740 1 -- 172.21.15.16:0/2159074024 wait complete. 2024-09-12T02:15:06.294 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.292+0000 7feb6f007740 20 client.19854 trim_cache size 0 max 16384 2024-09-12T02:15:06.294 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-09-12T02:15:06.294 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-09-12T02:15:06.294 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-09-12T02:15:06.294 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-09-12T02:15:06.295 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.293+0000 7feb6c18d640 1 -- 172.21.15.16:0/3302527759 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7feb500a2cb0 msgr2=0x7feb500a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:06.295 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.293+0000 7feb6c18d640 1 --2- 172.21.15.16:0/3302527759 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7feb500a2cb0 0x7feb500a5170 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7feb5c005c40 tx=0x7feb5c005bd0 comp rx=0 tx=0).stop 2024-09-12T02:15:06.295 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.293+0000 7feb6c18d640 1 -- 172.21.15.16:0/3302527759 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb64100d10 msgr2=0x7feb6419a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:06.295 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.293+0000 7feb6c18d640 1 --2- 172.21.15.16:0/3302527759 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb64100d10 0x7feb6419a540 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7feb58031440 tx=0x7feb58032ce0 comp rx=0 tx=0).stop 2024-09-12T02:15:06.295 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.293+0000 7feb6c18d640 1 -- 172.21.15.16:0/3302527759 shutdown_connections 2024-09-12T02:15:06.295 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.293+0000 7feb6c18d640 1 --2- 172.21.15.16:0/3302527759 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7feb500a2cb0 0x7feb500a5170 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:06.295 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.293+0000 7feb6c18d640 1 --2- 172.21.15.16:0/3302527759 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feb641016b0 0x7feb641a15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:06.295 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.293+0000 7feb6c18d640 1 --2- 172.21.15.16:0/3302527759 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb64100d10 0x7feb6419a540 unknown :-1 s=CLOSED pgs=1811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:06.296 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.293+0000 7feb6c18d640 1 --2- 172.21.15.16:0/3302527759 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb641003d0 0x7feb6419a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:06.296 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.293+0000 7feb6c18d640 1 -- 172.21.15.16:0/3302527759 >> 172.21.15.16:0/3302527759 conn(0x7feb640fc1e0 msgr2=0x7feb640fdc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:15:06.296 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.293+0000 7feb6c18d640 1 -- 172.21.15.16:0/3302527759 shutdown_connections 2024-09-12T02:15:06.296 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:06.293+0000 7feb6c18d640 1 -- 172.21.15.16:0/3302527759 wait complete. 2024-09-12T02:15:06.450 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-12T02:15:06.451 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-12T02:15:06.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:15:06 smithi184 ceph-mon[35593]: pgmap v730: 97 pgs: 97 active+clean; 3.0 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-09-12T02:15:06.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:15:06 smithi033 ceph-mon[35332]: pgmap v730: 97 pgs: 97 active+clean; 3.0 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-09-12T02:15:06.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:15:06 smithi016 ceph-mon[30947]: pgmap v730: 97 pgs: 97 active+clean; 3.0 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-09-12T02:15:06.707 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:15:07.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.169+0000 7fad111f1640 1 -- 172.21.15.16:0/3806728810 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad0c1082a0 msgr2=0x7fad0c10ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:07.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.169+0000 7fad111f1640 1 --2- 172.21.15.16:0/3806728810 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad0c1082a0 0x7fad0c10ec40 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7fad04009cd0 tx=0x7fad0402a8e0 comp rx=0 tx=0).stop 2024-09-12T02:15:07.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.169+0000 7fad111f1640 1 -- 172.21.15.16:0/3806728810 shutdown_connections 2024-09-12T02:15:07.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.169+0000 7fad111f1640 1 --2- 172.21.15.16:0/3806728810 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad0c1082a0 0x7fad0c10ec40 unknown :-1 s=CLOSED pgs=1815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:07.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.169+0000 7fad111f1640 1 --2- 172.21.15.16:0/3806728810 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fad0c1036e0 0x7fad0c107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:07.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.169+0000 7fad111f1640 1 --2- 172.21.15.16:0/3806728810 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fad0c102d30 0x7fad0c103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:07.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.169+0000 7fad111f1640 1 -- 172.21.15.16:0/3806728810 >> 172.21.15.16:0/3806728810 conn(0x7fad0c0fc7d0 msgr2=0x7fad0c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:15:07.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.169+0000 7fad111f1640 1 -- 172.21.15.16:0/3806728810 shutdown_connections 2024-09-12T02:15:07.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.169+0000 7fad111f1640 1 -- 172.21.15.16:0/3806728810 wait complete. 2024-09-12T02:15:07.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.169+0000 7fad111f1640 1 Processor -- start 2024-09-12T02:15:07.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.170+0000 7fad111f1640 1 -- start start 2024-09-12T02:15:07.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.170+0000 7fad111f1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad0c102d30 0x7fad0c100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:07.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.170+0000 7fad111f1640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fad0c1036e0 0x7fad0c100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:07.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.170+0000 7fad111f1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fad0c1082a0 0x7fad0c101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:07.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.170+0000 7fad111f1640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fad0c1120a0 con 0x7fad0c102d30 2024-09-12T02:15:07.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.170+0000 7fad111f1640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fad0c111f20 con 0x7fad0c1082a0 2024-09-12T02:15:07.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.170+0000 7fad111f1640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fad0c112220 con 0x7fad0c1036e0 2024-09-12T02:15:07.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.170+0000 7fad0ad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad0c102d30 0x7fad0c100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:07.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.170+0000 7fad0ad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad0c102d30 0x7fad0c100990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33496/0 (socket says 172.21.15.16:33496) 2024-09-12T02:15:07.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.170+0000 7fad0ad9d640 1 -- 172.21.15.16:0/1979768523 learned_addr learned my addr 172.21.15.16:0/1979768523 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:15:07.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.170+0000 7fad0b59e640 1 --2- 172.21.15.16:0/1979768523 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fad0c1082a0 0x7fad0c101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:07.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.170+0000 7fad0a59c640 1 --2- 172.21.15.16:0/1979768523 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fad0c1036e0 0x7fad0c100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:07.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.170+0000 7fad0ad9d640 1 -- 172.21.15.16:0/1979768523 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fad0c1036e0 msgr2=0x7fad0c100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:07.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.170+0000 7fad0ad9d640 1 --2- 172.21.15.16:0/1979768523 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fad0c1036e0 0x7fad0c100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:07.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.170+0000 7fad0ad9d640 1 -- 172.21.15.16:0/1979768523 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fad0c1082a0 msgr2=0x7fad0c101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:07.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.170+0000 7fad0ad9d640 1 --2- 172.21.15.16:0/1979768523 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fad0c1082a0 0x7fad0c101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:07.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.170+0000 7fad0ad9d640 1 -- 172.21.15.16:0/1979768523 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad0c1a69b0 con 0x7fad0c102d30 2024-09-12T02:15:07.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.171+0000 7fad0ad9d640 1 --2- 172.21.15.16:0/1979768523 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad0c102d30 0x7fad0c100990 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7facfc00d770 tx=0x7facfc00db70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:15:07.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.171+0000 7facf3fff640 1 -- 172.21.15.16:0/1979768523 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facfc017070 con 0x7fad0c102d30 2024-09-12T02:15:07.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.171+0000 7fad111f1640 1 -- 172.21.15.16:0/1979768523 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad0c1a6ca0 con 0x7fad0c102d30 2024-09-12T02:15:07.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.171+0000 7facf3fff640 1 -- 172.21.15.16:0/1979768523 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7facfc009820 con 0x7fad0c102d30 2024-09-12T02:15:07.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.171+0000 7facf3fff640 1 -- 172.21.15.16:0/1979768523 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facfc004cd0 con 0x7fad0c102d30 2024-09-12T02:15:07.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.171+0000 7fad111f1640 1 -- 172.21.15.16:0/1979768523 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad0c1a7190 con 0x7fad0c102d30 2024-09-12T02:15:07.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.173+0000 7facf3fff640 1 -- 172.21.15.16:0/1979768523 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7facfc0099d0 con 0x7fad0c102d30 2024-09-12T02:15:07.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.174+0000 7fad135ca740 1 Processor -- start 2024-09-12T02:15:07.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.174+0000 7fad135ca740 1 -- start start 2024-09-12T02:15:07.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.174+0000 7fad135ca740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5601fa20bb60 0x5601fa20bf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:07.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.174+0000 7fad135ca740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5601fa20c480 0x5601fa216800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:07.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.174+0000 7fad135ca740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5601fa216d40 0x5601fa219130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:07.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.174+0000 7fad135ca740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5601fa171280 con 0x5601fa20bb60 2024-09-12T02:15:07.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.174+0000 7fad135ca740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5601fa118b10 con 0x5601fa20c480 2024-09-12T02:15:07.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.174+0000 7fad135ca740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5601fa16d240 con 0x5601fa216d40 2024-09-12T02:15:07.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.174+0000 7facf3fff640 1 --2- 172.21.15.16:0/1979768523 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7facf40a2b80 0x7facf40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:07.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.174+0000 7fad0b59e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5601fa20bb60 0x5601fa20bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:07.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.174+0000 7fad0b59e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5601fa20bb60 0x5601fa20bf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33506/0 (socket says 172.21.15.16:33506) 2024-09-12T02:15:07.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.174+0000 7fad0b59e640 1 -- 172.21.15.16:0/2491053805 learned_addr learned my addr 172.21.15.16:0/2491053805 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:15:07.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.174+0000 7fad0ad9d640 1 --2- 172.21.15.16:0/2491053805 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5601fa20c480 0x5601fa216800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:07.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.174+0000 7fad0b59e640 1 --2- 172.21.15.16:0/1979768523 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7facf40a2b80 0x7facf40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:07.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.174+0000 7fad0a59c640 1 --2- 172.21.15.16:0/2491053805 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5601fa216d40 0x5601fa219130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:07.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.174+0000 7facf3fff640 1 -- 172.21.15.16:0/1979768523 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7facfc0ce9d0 con 0x7fad0c102d30 2024-09-12T02:15:07.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.175+0000 7fad0b59e640 1 -- 172.21.15.16:0/2491053805 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5601fa216d40 msgr2=0x5601fa219130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:15:07.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.175+0000 7fad0b59e640 1 --2- 172.21.15.16:0/2491053805 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5601fa216d40 0x5601fa219130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:07.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.175+0000 7fad0b59e640 1 -- 172.21.15.16:0/2491053805 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5601fa20c480 msgr2=0x5601fa216800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:15:07.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.175+0000 7fad0b59e640 1 --2- 172.21.15.16:0/2491053805 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5601fa20c480 0x5601fa216800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:07.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.175+0000 7fad0b59e640 1 -- 172.21.15.16:0/2491053805 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5601fa1251f0 con 0x5601fa20bb60 2024-09-12T02:15:07.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.175+0000 7fad0b59e640 1 --2- 172.21.15.16:0/2491053805 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5601fa20bb60 0x5601fa20bf40 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7fad04009cd0 tx=0x7fad04031c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=230bb5b71f5fd325 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:15:07.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.175+0000 7fad0b59e640 1 --2- 172.21.15.16:0/1979768523 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7facf40a2b80 0x7facf40a5040 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fad04047000 tx=0x7fad0404a000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:15:07.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.175+0000 7facf17fa640 1 -- 172.21.15.16:0/2491053805 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad04004410 con 0x5601fa20bb60 2024-09-12T02:15:07.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.175+0000 7facf17fa640 1 -- 172.21.15.16:0/2491053805 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad04036070 con 0x5601fa20bb60 2024-09-12T02:15:07.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.175+0000 7facf17fa640 1 -- 172.21.15.16:0/2491053805 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad04030ed0 con 0x5601fa20bb60 2024-09-12T02:15:07.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.176+0000 7fad135ca740 1 -- 172.21.15.16:0/2491053805 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5601fa20bb60 msgr2=0x5601fa20bf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:07.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.176+0000 7fad135ca740 1 --2- 172.21.15.16:0/2491053805 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5601fa20bb60 0x5601fa20bf40 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7fad04009cd0 tx=0x7fad04031c90 comp rx=0 tx=0).stop 2024-09-12T02:15:07.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.176+0000 7fad135ca740 1 -- 172.21.15.16:0/2491053805 shutdown_connections 2024-09-12T02:15:07.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.176+0000 7fad135ca740 1 --2- 172.21.15.16:0/2491053805 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5601fa216d40 0x5601fa219130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:07.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.176+0000 7fad135ca740 1 --2- 172.21.15.16:0/2491053805 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5601fa20c480 0x5601fa216800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:07.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.176+0000 7fad135ca740 1 --2- 172.21.15.16:0/2491053805 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5601fa20bb60 0x5601fa20bf40 unknown :-1 s=CLOSED pgs=1817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:07.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.176+0000 7fad135ca740 1 -- 172.21.15.16:0/2491053805 >> 172.21.15.16:0/2491053805 conn(0x5601fa2080f0 msgr2=0x5601fa2084d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:15:07.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.176+0000 7fad135ca740 1 -- 172.21.15.16:0/2491053805 shutdown_connections 2024-09-12T02:15:07.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.176+0000 7fad135ca740 1 -- 172.21.15.16:0/2491053805 wait complete. 2024-09-12T02:15:07.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.176+0000 7fad135ca740 1 -- start start 2024-09-12T02:15:07.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.176+0000 7fad135ca740 1 Processor -- start 2024-09-12T02:15:07.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.177+0000 7fad135ca740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5601fa20bb60 0x5601fa213c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:07.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.177+0000 7fad135ca740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5601fa20c480 0x5601fa214180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:07.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.177+0000 7fad135ca740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5601fa216d40 0x5601fa2146c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:07.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.177+0000 7fad135ca740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5601fa2198b0 con 0x5601fa20bb60 2024-09-12T02:15:07.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.177+0000 7fad135ca740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5601fa12bf10 con 0x5601fa20c480 2024-09-12T02:15:07.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.177+0000 7fad135ca740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5601fa219a30 con 0x5601fa216d40 2024-09-12T02:15:07.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.177+0000 7fad135ca740 10 client.0 _ll_register_callbacks cb 0x5601fa0f4a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:15:07.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.177+0000 7fad0ad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5601fa20bb60 0x5601fa213c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:07.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.177+0000 7fad0ad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5601fa20bb60 0x5601fa213c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33510/0 (socket says 172.21.15.16:33510) 2024-09-12T02:15:07.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.177+0000 7fad0ad9d640 1 -- 172.21.15.16:0/4004895098 learned_addr learned my addr 172.21.15.16:0/4004895098 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:15:07.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.177+0000 7fad0a59c640 1 --2- 172.21.15.16:0/4004895098 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5601fa20c480 0x5601fa214180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:07.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.177+0000 7fad0b59e640 1 --2- 172.21.15.16:0/4004895098 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5601fa216d40 0x5601fa2146c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:07.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.177+0000 7fad0ad9d640 1 -- 172.21.15.16:0/4004895098 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5601fa216d40 msgr2=0x5601fa2146c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:15:07.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.177+0000 7fad0ad9d640 1 --2- 172.21.15.16:0/4004895098 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5601fa216d40 0x5601fa2146c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:07.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.177+0000 7fad0ad9d640 1 -- 172.21.15.16:0/4004895098 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5601fa20c480 msgr2=0x5601fa214180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:15:07.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.177+0000 7fad0ad9d640 1 --2- 172.21.15.16:0/4004895098 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5601fa20c480 0x5601fa214180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:07.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.177+0000 7fad0ad9d640 1 -- 172.21.15.16:0/4004895098 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5601f9ee26a0 con 0x5601fa20bb60 2024-09-12T02:15:07.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.177+0000 7fad0b59e640 1 --2- 172.21.15.16:0/4004895098 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5601fa216d40 0x5601fa2146c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:15:07.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.177+0000 7fad0ad9d640 1 --2- 172.21.15.16:0/4004895098 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5601fa20bb60 0x5601fa213c40 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7facfc00dfd0 tx=0x7facfc018a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bcda46ac495acb2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:15:07.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.177+0000 7faceeff5640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:15:07.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.178+0000 7faceeff5640 1 -- 172.21.15.16:0/4004895098 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facfc0b4060 con 0x5601fa20bb60 2024-09-12T02:15:07.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.178+0000 7faceeff5640 1 -- 172.21.15.16:0/4004895098 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7facfc017d00 con 0x5601fa20bb60 2024-09-12T02:15:07.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.178+0000 7fad135ca740 1 -- 172.21.15.16:0/4004895098 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad04036070 con 0x5601fa20bb60 2024-09-12T02:15:07.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.178+0000 7faceeff5640 1 -- 172.21.15.16:0/4004895098 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7facfc020040 con 0x5601fa20bb60 2024-09-12T02:15:07.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.178+0000 7fad135ca740 10 client.19866 fetch_fsmap learned FSMap version 25 2024-09-12T02:15:07.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.178+0000 7fad135ca740 1 -- 172.21.15.16:0/4004895098 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5601fa225590 con 0x5601fa20bb60 2024-09-12T02:15:07.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.178+0000 7faceeff5640 1 -- 172.21.15.16:0/4004895098 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad04036070 con 0x5601fa20bb60 2024-09-12T02:15:07.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.178+0000 7faceeff5640 1 -- 172.21.15.16:0/4004895098 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7facfc09f170 con 0x5601fa20bb60 2024-09-12T02:15:07.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.178+0000 7fad135ca740 10 client.19866 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:15:07.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.178+0000 7fad135ca740 10 client.19866 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:15:07.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.178+0000 7fad135ca740 20 client.19866 populate_metadata read hostname 'smithi016' 2024-09-12T02:15:07.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.178+0000 7fad135ca740 1 --2- 172.21.15.16:0/4004895098 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5601fa226a50 0x5601fa228e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:07.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.178+0000 7fad135ca740 4 client.19866 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:15:07.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.178+0000 7fad135ca740 1 -- 172.21.15.16:0/4004895098 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5601fa17e7f0 con 0x5601fa226a50 2024-09-12T02:15:07.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.179+0000 7fad0a59c640 1 --2- 172.21.15.16:0/4004895098 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5601fa226a50 0x5601fa228e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:07.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.179+0000 7fad0a59c640 1 --2- 172.21.15.16:0/4004895098 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5601fa226a50 0x5601fa228e40 crc :-1 s=READY pgs=397 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e913a7766e6d085 server_cookie=301c86818113da95 in_seq=0 out_seq=0 2024-09-12T02:15:07.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.179+0000 7faceeff5640 10 client.19866 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:15:07.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.180+0000 7faceeff5640 1 -- 172.21.15.16:0/4004895098 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5601fa17e7f0 con 0x5601fa226a50 2024-09-12T02:15:07.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.180+0000 7faceeff5640 10 client.19866 handle_command_reply: tid=0 2024-09-12T02:15:07.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.180+0000 7fad135ca740 1 client.19866 shutdown 2024-09-12T02:15:07.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.180+0000 7fad135ca740 1 -- 172.21.15.16:0/4004895098 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5601fa20bb60 msgr2=0x5601fa213c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:07.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.180+0000 7fad135ca740 1 --2- 172.21.15.16:0/4004895098 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5601fa20bb60 0x5601fa213c40 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7facfc00dfd0 tx=0x7facfc018a40 comp rx=0 tx=0).stop 2024-09-12T02:15:07.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.180+0000 7fad135ca740 1 -- 172.21.15.16:0/4004895098 shutdown_connections 2024-09-12T02:15:07.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.180+0000 7fad135ca740 1 --2- 172.21.15.16:0/4004895098 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5601fa226a50 0x5601fa228e40 crc :-1 s=READY pgs=397 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:07.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.180+0000 7faceeff5640 0 client.19866 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:15:07.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.180+0000 7fad135ca740 1 --2- 172.21.15.16:0/4004895098 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5601fa216d40 0x5601fa2146c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:07.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.180+0000 7fad135ca740 1 --2- 172.21.15.16:0/4004895098 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5601fa20c480 0x5601fa214180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:07.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.180+0000 7fad135ca740 1 --2- 172.21.15.16:0/4004895098 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5601fa20bb60 0x5601fa213c40 unknown :-1 s=CLOSED pgs=1818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:07.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.180+0000 7fad135ca740 1 -- 172.21.15.16:0/4004895098 >> 172.21.15.16:0/4004895098 conn(0x5601fa2080f0 msgr2=0x5601fa207270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:15:07.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.180+0000 7fad135ca740 1 -- 172.21.15.16:0/4004895098 shutdown_connections 2024-09-12T02:15:07.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.180+0000 7fad135ca740 1 -- 172.21.15.16:0/4004895098 wait complete. 2024-09-12T02:15:07.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.181+0000 7fad135ca740 20 client.19866 trim_cache size 0 max 16384 2024-09-12T02:15:07.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.198+0000 7fad135ca740 1 Processor -- start 2024-09-12T02:15:07.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.198+0000 7fad135ca740 1 -- start start 2024-09-12T02:15:07.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.198+0000 7fad135ca740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5601fa20bb60 0x5601fa20e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:07.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.198+0000 7fad135ca740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5601fa20c480 0x5601fa22e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:07.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.198+0000 7fad135ca740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5601fa216d40 0x5601fa22ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:07.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.198+0000 7fad135ca740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fad0c1b1e40 con 0x5601fa20c480 2024-09-12T02:15:07.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.198+0000 7fad135ca740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fad0c1af710 con 0x5601fa216d40 2024-09-12T02:15:07.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.198+0000 7fad135ca740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fad0c1ad5e0 con 0x5601fa20bb60 2024-09-12T02:15:07.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.198+0000 7fad0a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5601fa20c480 0x5601fa22e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:07.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.198+0000 7fad0a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5601fa20c480 0x5601fa22e6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33518/0 (socket says 172.21.15.16:33518) 2024-09-12T02:15:07.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.198+0000 7fad0a59c640 1 -- 172.21.15.16:0/1320236892 learned_addr learned my addr 172.21.15.16:0/1320236892 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:15:07.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.198+0000 7fad0ad9d640 1 --2- 172.21.15.16:0/1320236892 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5601fa20bb60 0x5601fa20e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:07.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.198+0000 7fad0a59c640 1 -- 172.21.15.16:0/1320236892 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5601fa20bb60 msgr2=0x5601fa20e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:15:07.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.198+0000 7fad0a59c640 1 --2- 172.21.15.16:0/1320236892 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5601fa20bb60 0x5601fa20e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:07.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.198+0000 7fad0a59c640 1 -- 172.21.15.16:0/1320236892 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5601fa216d40 msgr2=0x5601fa22ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:15:07.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.198+0000 7fad0b59e640 1 --2- 172.21.15.16:0/1320236892 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5601fa216d40 0x5601fa22ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:07.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.199+0000 7fad0a59c640 1 --2- 172.21.15.16:0/1320236892 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5601fa216d40 0x5601fa22ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:07.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.199+0000 7fad0a59c640 1 -- 172.21.15.16:0/1320236892 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5601fa216510 con 0x5601fa20c480 2024-09-12T02:15:07.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.199+0000 7fad0b59e640 1 --2- 172.21.15.16:0/1320236892 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5601fa216d40 0x5601fa22ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:15:07.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.199+0000 7fad0a59c640 1 --2- 172.21.15.16:0/1320236892 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5601fa20c480 0x5601fa22e6d0 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7fad0c06a370 tx=0x7fad00013830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=626a480a55979e15 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:15:07.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.199+0000 7facf0ff9640 1 -- 172.21.15.16:0/1320236892 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad0001d430 con 0x5601fa20c480 2024-09-12T02:15:07.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.199+0000 7facf0ff9640 1 -- 172.21.15.16:0/1320236892 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad0001dab0 con 0x5601fa20c480 2024-09-12T02:15:07.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.199+0000 7facf0ff9640 1 -- 172.21.15.16:0/1320236892 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad00011b20 con 0x5601fa20c480 2024-09-12T02:15:07.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.199+0000 7fad135ca740 1 -- 172.21.15.16:0/1320236892 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5601fa20c480 msgr2=0x5601fa22e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:07.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.199+0000 7fad135ca740 1 --2- 172.21.15.16:0/1320236892 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5601fa20c480 0x5601fa22e6d0 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7fad0c06a370 tx=0x7fad00013830 comp rx=0 tx=0).stop 2024-09-12T02:15:07.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.200+0000 7fad135ca740 1 -- 172.21.15.16:0/1320236892 shutdown_connections 2024-09-12T02:15:07.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.200+0000 7fad135ca740 1 --2- 172.21.15.16:0/1320236892 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5601fa216d40 0x5601fa22ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:07.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.200+0000 7fad135ca740 1 --2- 172.21.15.16:0/1320236892 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5601fa20c480 0x5601fa22e6d0 unknown :-1 s=CLOSED pgs=1819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:07.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.200+0000 7fad135ca740 1 --2- 172.21.15.16:0/1320236892 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5601fa20bb60 0x5601fa20e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:07.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.200+0000 7fad135ca740 1 -- 172.21.15.16:0/1320236892 >> 172.21.15.16:0/1320236892 conn(0x5601fa2080f0 msgr2=0x5601fa215bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:15:07.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.200+0000 7fad135ca740 1 -- 172.21.15.16:0/1320236892 shutdown_connections 2024-09-12T02:15:07.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.200+0000 7fad135ca740 1 -- 172.21.15.16:0/1320236892 wait complete. 2024-09-12T02:15:07.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.200+0000 7fad135ca740 1 -- start start 2024-09-12T02:15:07.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.200+0000 7fad135ca740 1 Processor -- start 2024-09-12T02:15:07.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.200+0000 7fad135ca740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5601fa20bb60 0x5601fa21b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:07.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.200+0000 7fad135ca740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5601fa20c480 0x5601fa21b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:07.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.200+0000 7fad135ca740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5601fa216d40 0x5601fa2906a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:07.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.200+0000 7fad135ca740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fad0c1adc30 con 0x5601fa216d40 2024-09-12T02:15:07.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.200+0000 7fad135ca740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fad0c1af040 con 0x5601fa20bb60 2024-09-12T02:15:07.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.200+0000 7fad135ca740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fad0c1ae2a0 con 0x5601fa20c480 2024-09-12T02:15:07.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.200+0000 7fad135ca740 10 client.0 _ll_register_callbacks cb 0x7facfc09b090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:15:07.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.201+0000 7fad0b59e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5601fa216d40 0x5601fa2906a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:07.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.201+0000 7fad0b59e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5601fa216d40 0x5601fa2906a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33524/0 (socket says 172.21.15.16:33524) 2024-09-12T02:15:07.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.201+0000 7fad0b59e640 1 -- 172.21.15.16:0/611530238 learned_addr learned my addr 172.21.15.16:0/611530238 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:15:07.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.201+0000 7fad0a59c640 1 --2- 172.21.15.16:0/611530238 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5601fa20c480 0x5601fa21b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:07.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.201+0000 7fad0ad9d640 1 --2- 172.21.15.16:0/611530238 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5601fa20bb60 0x5601fa21b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:07.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.201+0000 7fad0b59e640 1 -- 172.21.15.16:0/611530238 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5601fa20c480 msgr2=0x5601fa21b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:15:07.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.201+0000 7fad0b59e640 1 --2- 172.21.15.16:0/611530238 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5601fa20c480 0x5601fa21b680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:07.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.201+0000 7fad0b59e640 1 -- 172.21.15.16:0/611530238 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5601fa20bb60 msgr2=0x5601fa21b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:15:07.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.201+0000 7fad0b59e640 1 --2- 172.21.15.16:0/611530238 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5601fa20bb60 0x5601fa21b140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:07.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.201+0000 7fad0b59e640 1 -- 172.21.15.16:0/611530238 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5601fa290c10 con 0x5601fa216d40 2024-09-12T02:15:07.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.201+0000 7fad0b59e640 1 --2- 172.21.15.16:0/611530238 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5601fa216d40 0x5601fa2906a0 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7fad0404c040 tx=0x7fad04004640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83e117ef4e528ae7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:15:07.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.201+0000 7faceeff5640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:15:07.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.201+0000 7faceeff5640 1 -- 172.21.15.16:0/611530238 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad040305a0 con 0x5601fa216d40 2024-09-12T02:15:07.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.201+0000 7faceeff5640 1 -- 172.21.15.16:0/611530238 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fad04030740 con 0x5601fa216d40 2024-09-12T02:15:07.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.201+0000 7fad135ca740 1 -- 172.21.15.16:0/611530238 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad0001dab0 con 0x5601fa216d40 2024-09-12T02:15:07.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.201+0000 7faceeff5640 1 -- 172.21.15.16:0/611530238 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fad04030940 con 0x5601fa216d40 2024-09-12T02:15:07.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.201+0000 7fad135ca740 10 client.19872 fetch_fsmap learned FSMap version 25 2024-09-12T02:15:07.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.201+0000 7fad135ca740 1 -- 172.21.15.16:0/611530238 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5601fa291ea0 con 0x5601fa216d40 2024-09-12T02:15:07.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.202+0000 7faceeff5640 1 -- 172.21.15.16:0/611530238 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad0001dab0 con 0x5601fa216d40 2024-09-12T02:15:07.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.202+0000 7faceeff5640 1 -- 172.21.15.16:0/611530238 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fad04030070 con 0x5601fa216d40 2024-09-12T02:15:07.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.202+0000 7fad135ca740 10 client.19872 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:15:07.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.202+0000 7fad135ca740 10 client.19872 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:15:07.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.202+0000 7fad135ca740 20 client.19872 populate_metadata read hostname 'smithi016' 2024-09-12T02:15:07.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.202+0000 7fad135ca740 1 --2- 172.21.15.16:0/611530238 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5601fa226a50 0x5601fa2941f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:07.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.202+0000 7fad135ca740 4 client.19872 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-12T02:15:07.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.202+0000 7fad135ca740 1 -- 172.21.15.16:0/611530238 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5601fa178d00 con 0x5601fa226a50 2024-09-12T02:15:07.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.202+0000 7fad0ad9d640 1 --2- 172.21.15.16:0/611530238 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5601fa226a50 0x5601fa2941f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:07.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.202+0000 7fad0ad9d640 1 --2- 172.21.15.16:0/611530238 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5601fa226a50 0x5601fa2941f0 crc :-1 s=READY pgs=398 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42a468b065199dd4 server_cookie=ae31a13bbfdccd91 in_seq=0 out_seq=0 2024-09-12T02:15:07.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.202+0000 7faceeff5640 10 client.19872 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:15:07.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.204+0000 7faceeff5640 1 -- 172.21.15.16:0/611530238 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5601fa178d00 con 0x5601fa226a50 2024-09-12T02:15:07.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.204+0000 7faceeff5640 10 client.19872 handle_command_reply: tid=0 2024-09-12T02:15:07.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.204+0000 7fad135ca740 1 client.19872 shutdown 2024-09-12T02:15:07.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.204+0000 7fad135ca740 1 -- 172.21.15.16:0/611530238 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5601fa216d40 msgr2=0x5601fa2906a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:07.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.204+0000 7fad135ca740 1 --2- 172.21.15.16:0/611530238 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5601fa216d40 0x5601fa2906a0 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7fad0404c040 tx=0x7fad04004640 comp rx=0 tx=0).stop 2024-09-12T02:15:07.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.204+0000 7fad135ca740 1 -- 172.21.15.16:0/611530238 shutdown_connections 2024-09-12T02:15:07.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.204+0000 7fad135ca740 1 --2- 172.21.15.16:0/611530238 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5601fa226a50 0x5601fa2941f0 crc :-1 s=READY pgs=398 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:07.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.204+0000 7fad135ca740 1 --2- 172.21.15.16:0/611530238 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5601fa216d40 0x5601fa2906a0 unknown :-1 s=CLOSED pgs=1820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:07.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.204+0000 7fad135ca740 1 --2- 172.21.15.16:0/611530238 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5601fa20c480 0x5601fa21b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:07.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.204+0000 7fad135ca740 1 --2- 172.21.15.16:0/611530238 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5601fa20bb60 0x5601fa21b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:07.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.204+0000 7fad135ca740 1 -- 172.21.15.16:0/611530238 >> 172.21.15.16:0/611530238 conn(0x5601fa2080f0 msgr2=0x5601fa2151d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:15:07.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.204+0000 7faceeff5640 0 client.19872 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:15:07.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.204+0000 7fad135ca740 1 -- 172.21.15.16:0/611530238 shutdown_connections 2024-09-12T02:15:07.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.204+0000 7fad135ca740 1 -- 172.21.15.16:0/611530238 wait complete. 2024-09-12T02:15:07.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.204+0000 7fad135ca740 20 client.19872 trim_cache size 0 max 16384 2024-09-12T02:15:07.209 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T02:15:07.209 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-09-12T02:15:07.209 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "3ffc5951-9b9d-4d2b-9b10-ab40fafb587d", 2024-09-12T02:15:07.209 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-09-12T02:15:07.209 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T02:15:07.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.205+0000 7fad111f1640 1 -- 172.21.15.16:0/1979768523 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7facf40a2b80 msgr2=0x7facf40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:07.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.205+0000 7fad111f1640 1 --2- 172.21.15.16:0/1979768523 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7facf40a2b80 0x7facf40a5040 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fad04047000 tx=0x7fad0404a000 comp rx=0 tx=0).stop 2024-09-12T02:15:07.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.205+0000 7fad111f1640 1 -- 172.21.15.16:0/1979768523 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad0c102d30 msgr2=0x7fad0c100990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:07.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.205+0000 7fad111f1640 1 --2- 172.21.15.16:0/1979768523 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad0c102d30 0x7fad0c100990 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7facfc00d770 tx=0x7facfc00db70 comp rx=0 tx=0).stop 2024-09-12T02:15:07.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.205+0000 7fad111f1640 1 -- 172.21.15.16:0/1979768523 shutdown_connections 2024-09-12T02:15:07.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.205+0000 7fad111f1640 1 --2- 172.21.15.16:0/1979768523 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7facf40a2b80 0x7facf40a5040 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:07.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.205+0000 7fad111f1640 1 --2- 172.21.15.16:0/1979768523 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fad0c1082a0 0x7fad0c101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:07.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.205+0000 7fad111f1640 1 --2- 172.21.15.16:0/1979768523 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fad0c1036e0 0x7fad0c100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:07.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.205+0000 7fad111f1640 1 --2- 172.21.15.16:0/1979768523 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad0c102d30 0x7fad0c100990 unknown :-1 s=CLOSED pgs=1816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:07.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.205+0000 7fad111f1640 1 -- 172.21.15.16:0/1979768523 >> 172.21.15.16:0/1979768523 conn(0x7fad0c0fc7d0 msgr2=0x7fad0c106380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:15:07.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.205+0000 7fad111f1640 1 -- 172.21.15.16:0/1979768523 shutdown_connections 2024-09-12T02:15:07.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:07.205+0000 7fad111f1640 1 -- 172.21.15.16:0/1979768523 wait complete. 2024-09-12T02:15:07.984 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T02:15:08.243 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:15:08.531 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:15:08 smithi016 ceph-mon[30947]: pgmap v731: 97 pgs: 97 active+clean; 3.0 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-12T02:15:08.531 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:15:08 smithi016 ceph-mon[30947]: scrub queued for path: / 2024-09-12T02:15:08.531 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:15:08 smithi016 ceph-mon[30947]: scrub summary: idle+waiting paths [/] 2024-09-12T02:15:08.531 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:15:08 smithi016 ceph-mon[30947]: scrub summary: active paths [/] 2024-09-12T02:15:08.531 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:15:08 smithi016 ceph-mon[30947]: scrub summary: idle+waiting paths [/] 2024-09-12T02:15:08.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:15:08 smithi184 ceph-mon[35593]: pgmap v731: 97 pgs: 97 active+clean; 3.0 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-12T02:15:08.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:15:08 smithi184 ceph-mon[35593]: scrub queued for path: / 2024-09-12T02:15:08.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:15:08 smithi184 ceph-mon[35593]: scrub summary: idle+waiting paths [/] 2024-09-12T02:15:08.628 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:15:08 smithi184 ceph-mon[35593]: scrub summary: active paths [/] 2024-09-12T02:15:08.628 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:15:08 smithi184 ceph-mon[35593]: scrub summary: idle+waiting paths [/] 2024-09-12T02:15:08.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:15:08 smithi033 ceph-mon[35332]: pgmap v731: 97 pgs: 97 active+clean; 3.0 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-12T02:15:08.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:15:08 smithi033 ceph-mon[35332]: scrub queued for path: / 2024-09-12T02:15:08.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:15:08 smithi033 ceph-mon[35332]: scrub summary: idle+waiting paths [/] 2024-09-12T02:15:08.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:15:08 smithi033 ceph-mon[35332]: scrub summary: active paths [/] 2024-09-12T02:15:08.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:15:08 smithi033 ceph-mon[35332]: scrub summary: idle+waiting paths [/] 2024-09-12T02:15:08.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.751+0000 7f0717af2640 1 -- 172.21.15.16:0/2497766775 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0710106a80 msgr2=0x7f0710106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:08.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.751+0000 7f0717af2640 1 --2- 172.21.15.16:0/2497766775 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0710106a80 0x7f0710106e60 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7f070c009930 tx=0x7f070c02a420 comp rx=0 tx=0).stop 2024-09-12T02:15:08.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.752+0000 7f0717af2640 1 -- 172.21.15.16:0/2497766775 shutdown_connections 2024-09-12T02:15:08.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.752+0000 7f0717af2640 1 --2- 172.21.15.16:0/2497766775 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0710101420 0x7f071010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:08.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.752+0000 7f0717af2640 1 --2- 172.21.15.16:0/2497766775 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0710100a80 0x7f0710100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:08.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.752+0000 7f0717af2640 1 --2- 172.21.15.16:0/2497766775 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0710106a80 0x7f0710106e60 unknown :-1 s=CLOSED pgs=1821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:08.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.752+0000 7f0717af2640 1 -- 172.21.15.16:0/2497766775 >> 172.21.15.16:0/2497766775 conn(0x7f07100fc7a0 msgr2=0x7f07100febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:15:08.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.752+0000 7f0717af2640 1 -- 172.21.15.16:0/2497766775 shutdown_connections 2024-09-12T02:15:08.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.752+0000 7f0717af2640 1 -- 172.21.15.16:0/2497766775 wait complete. 2024-09-12T02:15:08.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.752+0000 7f0717af2640 1 Processor -- start 2024-09-12T02:15:08.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.752+0000 7f0717af2640 1 -- start start 2024-09-12T02:15:08.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.753+0000 7f0717af2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0710100a80 0x7f07101983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:08.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.753+0000 7f0717af2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0710101420 0x7f0710198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:08.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.753+0000 7f0717af2640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0710106a80 0x7f071019cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:08.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.753+0000 7f0717af2640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0710110e40 con 0x7f0710101420 2024-09-12T02:15:08.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.753+0000 7f0717af2640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0710110cc0 con 0x7f0710100a80 2024-09-12T02:15:08.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.753+0000 7f0717af2640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f0710110fc0 con 0x7f0710106a80 2024-09-12T02:15:08.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.753+0000 7f071508d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0710101420 0x7f0710198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:08.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.753+0000 7f071508d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0710101420 0x7f0710198900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33530/0 (socket says 172.21.15.16:33530) 2024-09-12T02:15:08.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.753+0000 7f071508d640 1 -- 172.21.15.16:0/2605863076 learned_addr learned my addr 172.21.15.16:0/2605863076 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:15:08.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.753+0000 7f071588e640 1 --2- 172.21.15.16:0/2605863076 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0710100a80 0x7f07101983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:08.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.753+0000 7f071608f640 1 --2- 172.21.15.16:0/2605863076 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0710106a80 0x7f071019cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:08.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.753+0000 7f071508d640 1 -- 172.21.15.16:0/2605863076 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0710106a80 msgr2=0x7f071019cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:08.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.753+0000 7f071508d640 1 --2- 172.21.15.16:0/2605863076 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0710106a80 0x7f071019cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:08.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.753+0000 7f071508d640 1 -- 172.21.15.16:0/2605863076 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0710100a80 msgr2=0x7f07101983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:08.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.753+0000 7f071508d640 1 --2- 172.21.15.16:0/2605863076 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0710100a80 0x7f07101983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:08.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.753+0000 7f071508d640 1 -- 172.21.15.16:0/2605863076 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f071019d370 con 0x7f0710101420 2024-09-12T02:15:08.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.753+0000 7f071508d640 1 --2- 172.21.15.16:0/2605863076 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0710101420 0x7f0710198900 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7f070000c920 tx=0x7f070000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:15:08.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.753+0000 7f0706ffd640 1 -- 172.21.15.16:0/2605863076 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0700007c70 con 0x7f0710101420 2024-09-12T02:15:08.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.753+0000 7f0706ffd640 1 -- 172.21.15.16:0/2605863076 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0700007e10 con 0x7f0710101420 2024-09-12T02:15:08.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.753+0000 7f0717af2640 1 -- 172.21.15.16:0/2605863076 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f071019d660 con 0x7f0710101420 2024-09-12T02:15:08.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.754+0000 7f0706ffd640 1 -- 172.21.15.16:0/2605863076 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0700018890 con 0x7f0710101420 2024-09-12T02:15:08.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.754+0000 7f0717af2640 1 -- 172.21.15.16:0/2605863076 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07101a4f40 con 0x7f0710101420 2024-09-12T02:15:08.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.756+0000 7f0706ffd640 1 -- 172.21.15.16:0/2605863076 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f0700018b30 con 0x7f0710101420 2024-09-12T02:15:08.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.756+0000 7f0719ecb740 1 Processor -- start 2024-09-12T02:15:08.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.757+0000 7f0719ecb740 1 -- start start 2024-09-12T02:15:08.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.757+0000 7f0706ffd640 1 --2- 172.21.15.16:0/2605863076 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f06fc0a2b80 0x7f06fc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:08.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.757+0000 7f0719ecb740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557fcd730360 0x557fcd730740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:08.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.757+0000 7f0719ecb740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557fcd730c80 0x557fcd73f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:08.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.757+0000 7f071608f640 1 --2- 172.21.15.16:0/2605863076 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f06fc0a2b80 0x7f06fc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:08.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.757+0000 7f0719ecb740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557fcd73fa90 0x557fcd741e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:08.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.757+0000 7f0706ffd640 1 -- 172.21.15.16:0/2605863076 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f07000cf040 con 0x7f0710101420 2024-09-12T02:15:08.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.757+0000 7f0719ecb740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x557fcd6962e0 con 0x557fcd730c80 2024-09-12T02:15:08.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.757+0000 7f0719ecb740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x557fcd63dce0 con 0x557fcd73fa90 2024-09-12T02:15:08.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.757+0000 7f0719ecb740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x557fcd6922a0 con 0x557fcd730360 2024-09-12T02:15:08.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.757+0000 7f071588e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557fcd730360 0x557fcd730740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:08.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.757+0000 7f071508d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557fcd730c80 0x557fcd73f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:08.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.757+0000 7f071588e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557fcd730360 0x557fcd730740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:37370/0 (socket says 172.21.15.16:37370) 2024-09-12T02:15:08.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.757+0000 7f071588e640 1 -- 172.21.15.16:0/3988613081 learned_addr learned my addr 172.21.15.16:0/3988613081 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:15:08.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.757+0000 7f071608f640 1 --2- 172.21.15.16:0/3988613081 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557fcd73fa90 0x557fcd741e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:08.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.757+0000 7f071508d640 1 -- 172.21.15.16:0/3988613081 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557fcd730360 msgr2=0x557fcd730740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:15:08.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.757+0000 7f071508d640 1 --2- 172.21.15.16:0/3988613081 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557fcd730360 0x557fcd730740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:08.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.757+0000 7f071508d640 1 -- 172.21.15.16:0/3988613081 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557fcd73fa90 msgr2=0x557fcd741e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:15:08.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.757+0000 7f071508d640 1 --2- 172.21.15.16:0/3988613081 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557fcd73fa90 0x557fcd741e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:08.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.757+0000 7f071508d640 1 -- 172.21.15.16:0/3988613081 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557fcd622440 con 0x557fcd730c80 2024-09-12T02:15:08.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.757+0000 7f071608f640 1 --2- 172.21.15.16:0/2605863076 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f06fc0a2b80 0x7f06fc0a5040 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f070800a780 tx=0x7f070800a8d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:15:08.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.758+0000 7f071508d640 1 --2- 172.21.15.16:0/3988613081 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557fcd730c80 0x557fcd73f550 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7f07000ccc30 tx=0x7f0700018ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ca328d90514af1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:15:08.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.758+0000 7f06fbfff640 1 -- 172.21.15.16:0/3988613081 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07000b4070 con 0x557fcd730c80 2024-09-12T02:15:08.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.758+0000 7f06fbfff640 1 -- 172.21.15.16:0/3988613081 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f070009fcf0 con 0x557fcd730c80 2024-09-12T02:15:08.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.758+0000 7f06fbfff640 1 -- 172.21.15.16:0/3988613081 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07000afe60 con 0x557fcd730c80 2024-09-12T02:15:08.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.758+0000 7f0719ecb740 1 -- 172.21.15.16:0/3988613081 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557fcd730c80 msgr2=0x557fcd73f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:08.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.758+0000 7f0719ecb740 1 --2- 172.21.15.16:0/3988613081 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557fcd730c80 0x557fcd73f550 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7f07000ccc30 tx=0x7f0700018ee0 comp rx=0 tx=0).stop 2024-09-12T02:15:08.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.759+0000 7f0719ecb740 1 -- 172.21.15.16:0/3988613081 shutdown_connections 2024-09-12T02:15:08.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.759+0000 7f0719ecb740 1 --2- 172.21.15.16:0/3988613081 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557fcd73fa90 0x557fcd741e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:08.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.759+0000 7f0719ecb740 1 --2- 172.21.15.16:0/3988613081 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557fcd730c80 0x557fcd73f550 unknown :-1 s=CLOSED pgs=1823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:08.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.759+0000 7f0719ecb740 1 --2- 172.21.15.16:0/3988613081 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557fcd730360 0x557fcd730740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:08.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.759+0000 7f0719ecb740 1 -- 172.21.15.16:0/3988613081 >> 172.21.15.16:0/3988613081 conn(0x557fcd72ca30 msgr2=0x557fcd72ce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:15:08.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.759+0000 7f0719ecb740 1 -- 172.21.15.16:0/3988613081 shutdown_connections 2024-09-12T02:15:08.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.759+0000 7f0719ecb740 1 -- 172.21.15.16:0/3988613081 wait complete. 2024-09-12T02:15:08.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.759+0000 7f0719ecb740 1 -- start start 2024-09-12T02:15:08.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.759+0000 7f0719ecb740 1 Processor -- start 2024-09-12T02:15:08.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.759+0000 7f0719ecb740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557fcd730360 0x557fcd73f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:08.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.759+0000 7f0719ecb740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557fcd730c80 0x557fcd73c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:08.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.759+0000 7f0719ecb740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557fcd73fa90 0x557fcd73cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:08.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.759+0000 7f0719ecb740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x557fcd742660 con 0x557fcd730360 2024-09-12T02:15:08.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.759+0000 7f0719ecb740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x557fcd651050 con 0x557fcd730c80 2024-09-12T02:15:08.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.759+0000 7f0719ecb740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x557fcd7427e0 con 0x557fcd73fa90 2024-09-12T02:15:08.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.760+0000 7f0719ecb740 10 client.0 _ll_register_callbacks cb 0x557fcd691160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:15:08.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.760+0000 7f071588e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557fcd730360 0x557fcd73f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:08.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.760+0000 7f071588e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557fcd730360 0x557fcd73f3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33550/0 (socket says 172.21.15.16:33550) 2024-09-12T02:15:08.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.760+0000 7f071588e640 1 -- 172.21.15.16:0/3027889952 learned_addr learned my addr 172.21.15.16:0/3027889952 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:15:08.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.760+0000 7f071508d640 1 --2- 172.21.15.16:0/3027889952 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557fcd730c80 0x557fcd73c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:08.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.760+0000 7f071608f640 1 --2- 172.21.15.16:0/3027889952 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557fcd73fa90 0x557fcd73cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:08.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.760+0000 7f071588e640 1 -- 172.21.15.16:0/3027889952 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557fcd73fa90 msgr2=0x557fcd73cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:15:08.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.760+0000 7f071588e640 1 --2- 172.21.15.16:0/3027889952 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557fcd73fa90 0x557fcd73cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:08.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.760+0000 7f071588e640 1 -- 172.21.15.16:0/3027889952 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557fcd730c80 msgr2=0x557fcd73c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:15:08.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.760+0000 7f071588e640 1 --2- 172.21.15.16:0/3027889952 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557fcd730c80 0x557fcd73c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:08.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.760+0000 7f071588e640 1 -- 172.21.15.16:0/3027889952 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557fcd4073c0 con 0x557fcd730360 2024-09-12T02:15:08.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.760+0000 7f071508d640 1 --2- 172.21.15.16:0/3027889952 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557fcd730c80 0x557fcd73c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:15:08.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.760+0000 7f071588e640 1 --2- 172.21.15.16:0/3027889952 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557fcd730360 0x557fcd73f3d0 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7f070c033000 tx=0x7f070c031990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77fe4dd55158838b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:15:08.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.760+0000 7f06f97fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:15:08.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.760+0000 7f06f97fa640 1 -- 172.21.15.16:0/3027889952 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f070c035040 con 0x557fcd730360 2024-09-12T02:15:08.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.760+0000 7f0719ecb740 1 -- 172.21.15.16:0/3027889952 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f070009fcf0 con 0x557fcd730360 2024-09-12T02:15:08.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.760+0000 7f06f97fa640 1 -- 172.21.15.16:0/3027889952 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f070c004300 con 0x557fcd730360 2024-09-12T02:15:08.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.761+0000 7f06f97fa640 1 -- 172.21.15.16:0/3027889952 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f070c005020 con 0x557fcd730360 2024-09-12T02:15:08.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.761+0000 7f0719ecb740 10 client.19884 fetch_fsmap learned FSMap version 25 2024-09-12T02:15:08.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.761+0000 7f0719ecb740 1 -- 172.21.15.16:0/3027889952 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x557fcd74e050 con 0x557fcd730360 2024-09-12T02:15:08.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.761+0000 7f06f97fa640 1 -- 172.21.15.16:0/3027889952 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f070009fcf0 con 0x557fcd730360 2024-09-12T02:15:08.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.761+0000 7f06f97fa640 1 -- 172.21.15.16:0/3027889952 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f070c0035f0 con 0x557fcd730360 2024-09-12T02:15:08.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.761+0000 7f0719ecb740 10 client.19884 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:15:08.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.761+0000 7f0719ecb740 10 client.19884 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:15:08.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.761+0000 7f0719ecb740 20 client.19884 populate_metadata read hostname 'smithi016' 2024-09-12T02:15:08.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.761+0000 7f0719ecb740 1 --2- 172.21.15.16:0/3027889952 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x557fcd74f510 0x557fcd751900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:08.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.761+0000 7f0719ecb740 4 client.19884 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:15:08.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.761+0000 7f0719ecb740 1 -- 172.21.15.16:0/3027889952 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557fcd6a3850 con 0x557fcd74f510 2024-09-12T02:15:08.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.761+0000 7f071508d640 1 --2- 172.21.15.16:0/3027889952 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x557fcd74f510 0x557fcd751900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:08.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.762+0000 7f06f97fa640 10 client.19884 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:15:08.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.762+0000 7f071508d640 1 --2- 172.21.15.16:0/3027889952 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x557fcd74f510 0x557fcd751900 crc :-1 s=READY pgs=399 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36bb5879306e7b10 server_cookie=586c07efd74b7cac in_seq=0 out_seq=0 2024-09-12T02:15:08.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.762+0000 7f06f97fa640 1 -- 172.21.15.16:0/3027889952 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557fcd6a3850 con 0x557fcd74f510 2024-09-12T02:15:08.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.762+0000 7f06f97fa640 10 client.19884 handle_command_reply: tid=0 2024-09-12T02:15:08.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.763+0000 7f0719ecb740 1 client.19884 shutdown 2024-09-12T02:15:08.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.763+0000 7f0719ecb740 1 -- 172.21.15.16:0/3027889952 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557fcd730360 msgr2=0x557fcd73f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:08.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.763+0000 7f0719ecb740 1 --2- 172.21.15.16:0/3027889952 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557fcd730360 0x557fcd73f3d0 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7f070c033000 tx=0x7f070c031990 comp rx=0 tx=0).stop 2024-09-12T02:15:08.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.763+0000 7f0719ecb740 1 -- 172.21.15.16:0/3027889952 shutdown_connections 2024-09-12T02:15:08.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.763+0000 7f0719ecb740 1 --2- 172.21.15.16:0/3027889952 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x557fcd74f510 0x557fcd751900 crc :-1 s=READY pgs=399 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:08.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.763+0000 7f0719ecb740 1 --2- 172.21.15.16:0/3027889952 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557fcd73fa90 0x557fcd73cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:08.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.763+0000 7f0719ecb740 1 --2- 172.21.15.16:0/3027889952 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557fcd730c80 0x557fcd73c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:08.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.763+0000 7f0719ecb740 1 --2- 172.21.15.16:0/3027889952 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557fcd730360 0x557fcd73f3d0 unknown :-1 s=CLOSED pgs=1824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:08.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.763+0000 7f06f97fa640 0 client.19884 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:15:08.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.763+0000 7f0719ecb740 1 -- 172.21.15.16:0/3027889952 >> 172.21.15.16:0/3027889952 conn(0x557fcd72ca30 msgr2=0x557fcd732790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:15:08.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.763+0000 7f0719ecb740 1 -- 172.21.15.16:0/3027889952 shutdown_connections 2024-09-12T02:15:08.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.763+0000 7f0719ecb740 1 -- 172.21.15.16:0/3027889952 wait complete. 2024-09-12T02:15:08.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.763+0000 7f0719ecb740 20 client.19884 trim_cache size 0 max 16384 2024-09-12T02:15:08.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.780+0000 7f0719ecb740 1 Processor -- start 2024-09-12T02:15:08.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.780+0000 7f0719ecb740 1 -- start start 2024-09-12T02:15:08.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.780+0000 7f0719ecb740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557fcd730360 0x557fcd74b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:08.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.780+0000 7f0719ecb740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557fcd730c80 0x557fcd757180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:08.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.780+0000 7f0719ecb740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557fcd73fa90 0x557fcd7576c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:08.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.780+0000 7f0719ecb740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f07101ada40 con 0x557fcd73fa90 2024-09-12T02:15:08.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.780+0000 7f0719ecb740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f07101ab310 con 0x557fcd730c80 2024-09-12T02:15:08.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.780+0000 7f0719ecb740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f07101a9180 con 0x557fcd730360 2024-09-12T02:15:08.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.781+0000 7f071608f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557fcd73fa90 0x557fcd7576c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:08.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.781+0000 7f071608f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557fcd73fa90 0x557fcd7576c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33560/0 (socket says 172.21.15.16:33560) 2024-09-12T02:15:08.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.781+0000 7f071608f640 1 -- 172.21.15.16:0/3141395860 learned_addr learned my addr 172.21.15.16:0/3141395860 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:15:08.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.781+0000 7f071508d640 1 --2- 172.21.15.16:0/3141395860 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557fcd730c80 0x557fcd757180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:08.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.781+0000 7f071588e640 1 --2- 172.21.15.16:0/3141395860 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557fcd730360 0x557fcd74b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:08.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.781+0000 7f071508d640 1 -- 172.21.15.16:0/3141395860 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557fcd730360 msgr2=0x557fcd74b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:15:08.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.781+0000 7f071508d640 1 --2- 172.21.15.16:0/3141395860 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557fcd730360 0x557fcd74b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:08.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.781+0000 7f071508d640 1 -- 172.21.15.16:0/3141395860 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557fcd73fa90 msgr2=0x557fcd7576c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:15:08.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.781+0000 7f071508d640 1 --2- 172.21.15.16:0/3141395860 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557fcd73fa90 0x557fcd7576c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:08.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.781+0000 7f071508d640 1 -- 172.21.15.16:0/3141395860 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557fcd73f740 con 0x557fcd730c80 2024-09-12T02:15:08.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.781+0000 7f071508d640 1 --2- 172.21.15.16:0/3141395860 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557fcd730c80 0x557fcd757180 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f07000ccc30 tx=0x7f070000ff30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=50905a6127fb1c46 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:15:08.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.781+0000 7f06fb7fe640 1 -- 172.21.15.16:0/3141395860 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07000afe60 con 0x557fcd730c80 2024-09-12T02:15:08.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.781+0000 7f06fb7fe640 1 -- 172.21.15.16:0/3141395860 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f070000fb30 con 0x557fcd730c80 2024-09-12T02:15:08.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.781+0000 7f06fb7fe640 1 -- 172.21.15.16:0/3141395860 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07000b4040 con 0x557fcd730c80 2024-09-12T02:15:08.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.782+0000 7f0719ecb740 1 -- 172.21.15.16:0/3141395860 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557fcd730c80 msgr2=0x557fcd757180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:08.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.782+0000 7f0719ecb740 1 --2- 172.21.15.16:0/3141395860 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557fcd730c80 0x557fcd757180 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f07000ccc30 tx=0x7f070000ff30 comp rx=0 tx=0).stop 2024-09-12T02:15:08.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.782+0000 7f0719ecb740 1 -- 172.21.15.16:0/3141395860 shutdown_connections 2024-09-12T02:15:08.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.782+0000 7f0719ecb740 1 --2- 172.21.15.16:0/3141395860 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557fcd73fa90 0x557fcd7576c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:08.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.782+0000 7f0719ecb740 1 --2- 172.21.15.16:0/3141395860 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557fcd730c80 0x557fcd757180 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:08.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.782+0000 7f0719ecb740 1 --2- 172.21.15.16:0/3141395860 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557fcd730360 0x557fcd74b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:08.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.782+0000 7f0719ecb740 1 -- 172.21.15.16:0/3141395860 >> 172.21.15.16:0/3141395860 conn(0x557fcd72ca30 msgr2=0x557fcd737d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:15:08.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.782+0000 7f0719ecb740 1 -- 172.21.15.16:0/3141395860 shutdown_connections 2024-09-12T02:15:08.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.782+0000 7f0719ecb740 1 -- 172.21.15.16:0/3141395860 wait complete. 2024-09-12T02:15:08.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.782+0000 7f0719ecb740 1 -- start start 2024-09-12T02:15:08.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.783+0000 7f0719ecb740 1 Processor -- start 2024-09-12T02:15:08.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.783+0000 7f0719ecb740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557fcd730360 0x557fcd748380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:08.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.783+0000 7f0719ecb740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557fcd730c80 0x557fcd7436b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:08.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.783+0000 7f0719ecb740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557fcd73fa90 0x557fcd743e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:08.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.783+0000 7f0719ecb740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f07101a97d0 con 0x557fcd730c80 2024-09-12T02:15:08.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.783+0000 7f0719ecb740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f07101aac40 con 0x557fcd730360 2024-09-12T02:15:08.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.783+0000 7f0719ecb740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f07101a9ea0 con 0x557fcd73fa90 2024-09-12T02:15:08.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.783+0000 7f0719ecb740 10 client.0 _ll_register_callbacks cb 0x7f070c031620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:15:08.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.783+0000 7f071588e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557fcd730360 0x557fcd748380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:08.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.783+0000 7f071588e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557fcd730360 0x557fcd748380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.16:55612/0 (socket says 172.21.15.16:55612) 2024-09-12T02:15:08.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.783+0000 7f071588e640 1 -- 172.21.15.16:0/474282165 learned_addr learned my addr 172.21.15.16:0/474282165 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:15:08.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.783+0000 7f071508d640 1 --2- 172.21.15.16:0/474282165 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557fcd730c80 0x557fcd7436b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:08.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.783+0000 7f071608f640 1 --2- 172.21.15.16:0/474282165 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557fcd73fa90 0x557fcd743e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:08.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.784+0000 7f071608f640 1 -- 172.21.15.16:0/474282165 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557fcd730360 msgr2=0x557fcd748380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:15:08.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.784+0000 7f071608f640 1 --2- 172.21.15.16:0/474282165 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557fcd730360 0x557fcd748380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:08.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.784+0000 7f071608f640 1 -- 172.21.15.16:0/474282165 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557fcd730c80 msgr2=0x557fcd7436b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:15:08.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.784+0000 7f071608f640 1 --2- 172.21.15.16:0/474282165 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557fcd730c80 0x557fcd7436b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:08.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.784+0000 7f071608f640 1 -- 172.21.15.16:0/474282165 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557fcd7b9150 con 0x557fcd73fa90 2024-09-12T02:15:08.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.784+0000 7f071588e640 1 --2- 172.21.15.16:0/474282165 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557fcd730360 0x557fcd748380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:15:08.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.784+0000 7f071508d640 1 --2- 172.21.15.16:0/474282165 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557fcd730c80 0x557fcd7436b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:15:08.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.784+0000 7f071608f640 1 --2- 172.21.15.16:0/474282165 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557fcd73fa90 0x557fcd743e40 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x557fcd6db820 tx=0x7f0708004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aecdb75e4c4f1d50 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:15:08.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.784+0000 7f06f97fa640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-09-12T02:15:08.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.784+0000 7f06f97fa640 1 -- 172.21.15.16:0/474282165 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0708018070 con 0x557fcd73fa90 2024-09-12T02:15:08.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.784+0000 7f06f97fa640 1 -- 172.21.15.16:0/474282165 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f070801de00 con 0x557fcd73fa90 2024-09-12T02:15:08.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.784+0000 7f0719ecb740 1 -- 172.21.15.16:0/474282165 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f070000fb30 con 0x557fcd73fa90 2024-09-12T02:15:08.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.784+0000 7f06f97fa640 1 -- 172.21.15.16:0/474282165 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f0708004b20 con 0x557fcd73fa90 2024-09-12T02:15:08.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.784+0000 7f0719ecb740 10 client.28387 fetch_fsmap learned FSMap version 25 2024-09-12T02:15:08.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.784+0000 7f0719ecb740 1 -- 172.21.15.16:0/474282165 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x557fcd7ba380 con 0x557fcd73fa90 2024-09-12T02:15:08.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.785+0000 7f06f97fa640 1 -- 172.21.15.16:0/474282165 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f070000fb30 con 0x557fcd73fa90 2024-09-12T02:15:08.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.785+0000 7f06f97fa640 1 -- 172.21.15.16:0/474282165 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f0708031030 con 0x557fcd73fa90 2024-09-12T02:15:08.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.785+0000 7f0719ecb740 10 client.28387 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:15:08.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.785+0000 7f0719ecb740 10 client.28387 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:15:08.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.785+0000 7f0719ecb740 20 client.28387 populate_metadata read hostname 'smithi016' 2024-09-12T02:15:08.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.785+0000 7f0719ecb740 1 --2- 172.21.15.16:0/474282165 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x557fcd7bab50 0x557fcd7bcf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:08.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.785+0000 7f0719ecb740 4 client.28387 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T02:15:08.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.785+0000 7f0719ecb740 1 -- 172.21.15.16:0/474282165 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557fcd69dd60 con 0x557fcd7bab50 2024-09-12T02:15:08.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.785+0000 7f071588e640 1 --2- 172.21.15.16:0/474282165 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x557fcd7bab50 0x557fcd7bcf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:08.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.786+0000 7f06f97fa640 10 client.28387 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:15:08.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.786+0000 7f071588e640 1 --2- 172.21.15.16:0/474282165 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x557fcd7bab50 0x557fcd7bcf70 crc :-1 s=READY pgs=400 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33cb00cd5a181c54 server_cookie=39b73bf0f2a4652a in_seq=0 out_seq=0 2024-09-12T02:15:08.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.786+0000 7f06f97fa640 1 -- 172.21.15.16:0/474282165 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x557fcd69dd60 con 0x557fcd7bab50 2024-09-12T02:15:08.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.786+0000 7f06f97fa640 10 client.28387 handle_command_reply: tid=0 2024-09-12T02:15:08.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.786+0000 7f0719ecb740 1 client.28387 shutdown 2024-09-12T02:15:08.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.786+0000 7f0719ecb740 1 -- 172.21.15.16:0/474282165 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557fcd73fa90 msgr2=0x557fcd743e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:08.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.786+0000 7f0719ecb740 1 --2- 172.21.15.16:0/474282165 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557fcd73fa90 0x557fcd743e40 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x557fcd6db820 tx=0x7f0708004ab0 comp rx=0 tx=0).stop 2024-09-12T02:15:08.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.786+0000 7f0719ecb740 1 -- 172.21.15.16:0/474282165 shutdown_connections 2024-09-12T02:15:08.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.786+0000 7f0719ecb740 1 --2- 172.21.15.16:0/474282165 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x557fcd7bab50 0x557fcd7bcf70 crc :-1 s=READY pgs=400 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:08.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.786+0000 7f0719ecb740 1 --2- 172.21.15.16:0/474282165 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557fcd73fa90 0x557fcd743e40 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:08.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.786+0000 7f0719ecb740 1 --2- 172.21.15.16:0/474282165 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557fcd730c80 0x557fcd7436b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:08.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.786+0000 7f0719ecb740 1 --2- 172.21.15.16:0/474282165 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557fcd730360 0x557fcd748380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:08.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.786+0000 7f06f97fa640 0 client.28387 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:15:08.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.786+0000 7f0719ecb740 1 -- 172.21.15.16:0/474282165 >> 172.21.15.16:0/474282165 conn(0x557fcd72ca30 msgr2=0x557fcd72b350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:15:08.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.786+0000 7f0719ecb740 1 -- 172.21.15.16:0/474282165 shutdown_connections 2024-09-12T02:15:08.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.786+0000 7f0719ecb740 1 -- 172.21.15.16:0/474282165 wait complete. 2024-09-12T02:15:08.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.787+0000 7f0719ecb740 20 client.28387 trim_cache size 0 max 16384 2024-09-12T02:15:08.791 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T02:15:08.792 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-12T02:15:08.792 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": { 2024-09-12T02:15:08.792 INFO:teuthology.orchestra.run.smithi016.stdout: "3ffc5951-9b9d-4d2b-9b10-ab40fafb587d": { 2024-09-12T02:15:08.792 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-12T02:15:08.792 INFO:teuthology.orchestra.run.smithi016.stdout: "tag": "3ffc5951-9b9d-4d2b-9b10-ab40fafb587d", 2024-09-12T02:15:08.792 INFO:teuthology.orchestra.run.smithi016.stdout: "options": "recursive,force" 2024-09-12T02:15:08.792 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:15:08.792 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:15:08.792 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T02:15:08.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.787+0000 7f0717af2640 1 -- 172.21.15.16:0/2605863076 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f06fc0a2b80 msgr2=0x7f06fc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:08.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.787+0000 7f0717af2640 1 --2- 172.21.15.16:0/2605863076 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f06fc0a2b80 0x7f06fc0a5040 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f070800a780 tx=0x7f070800a8d0 comp rx=0 tx=0).stop 2024-09-12T02:15:08.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.787+0000 7f0717af2640 1 -- 172.21.15.16:0/2605863076 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0710101420 msgr2=0x7f0710198900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:08.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.787+0000 7f0717af2640 1 --2- 172.21.15.16:0/2605863076 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0710101420 0x7f0710198900 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7f070000c920 tx=0x7f070000cde0 comp rx=0 tx=0).stop 2024-09-12T02:15:08.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.787+0000 7f0717af2640 1 -- 172.21.15.16:0/2605863076 shutdown_connections 2024-09-12T02:15:08.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.787+0000 7f0717af2640 1 --2- 172.21.15.16:0/2605863076 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f06fc0a2b80 0x7f06fc0a5040 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:08.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.787+0000 7f0717af2640 1 --2- 172.21.15.16:0/2605863076 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0710106a80 0x7f071019cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:08.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.787+0000 7f0717af2640 1 --2- 172.21.15.16:0/2605863076 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0710101420 0x7f0710198900 unknown :-1 s=CLOSED pgs=1822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:08.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.787+0000 7f0717af2640 1 --2- 172.21.15.16:0/2605863076 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0710100a80 0x7f07101983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:08.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.787+0000 7f0717af2640 1 -- 172.21.15.16:0/2605863076 >> 172.21.15.16:0/2605863076 conn(0x7f07100fc7a0 msgr2=0x7f071006aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:15:08.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.787+0000 7f0717af2640 1 -- 172.21.15.16:0/2605863076 shutdown_connections 2024-09-12T02:15:08.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:08.787+0000 7f0717af2640 1 -- 172.21.15.16:0/2605863076 wait complete. 2024-09-12T02:15:09.648 INFO:tasks.cephfs.filesystem:scrub status for tag:3ffc5951-9b9d-4d2b-9b10-ab40fafb587d - {'path': '/', 'tag': '3ffc5951-9b9d-4d2b-9b10-ab40fafb587d', 'options': 'recursive,force'} 2024-09-12T02:15:10.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:15:10 smithi184 ceph-mon[35593]: pgmap v732: 97 pgs: 97 active+clean; 3.0 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 7.4 KiB/s wr, 8 op/s 2024-09-12T02:15:10.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:15:10 smithi033 ceph-mon[35332]: pgmap v732: 97 pgs: 97 active+clean; 3.0 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 7.4 KiB/s wr, 8 op/s 2024-09-12T02:15:10.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:15:10 smithi016 ceph-mon[30947]: pgmap v732: 97 pgs: 97 active+clean; 3.0 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 7.4 KiB/s wr, 8 op/s 2024-09-12T02:15:12.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:15:12 smithi184 ceph-mon[35593]: pgmap v733: 97 pgs: 97 active+clean; 3.0 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.0 KiB/s wr, 12 op/s 2024-09-12T02:15:12.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:15:12 smithi184 ceph-mon[35593]: scrub summary: idle 2024-09-12T02:15:12.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:15:12 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:15:12.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:15:12 smithi033 ceph-mon[35332]: pgmap v733: 97 pgs: 97 active+clean; 3.0 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.0 KiB/s wr, 12 op/s 2024-09-12T02:15:12.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:15:12 smithi033 ceph-mon[35332]: scrub summary: idle 2024-09-12T02:15:12.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:15:12 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:15:12.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:15:12 smithi016 ceph-mon[30947]: pgmap v733: 97 pgs: 97 active+clean; 3.0 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.0 KiB/s wr, 12 op/s 2024-09-12T02:15:12.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:15:12 smithi016 ceph-mon[30947]: scrub summary: idle 2024-09-12T02:15:12.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:15:12 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:15:13.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:15:13 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:15:13.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:15:13 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:15:13.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:15:13 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T02:15:13.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:15:13 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:15:13.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:15:13 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:15:13.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:15:13 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T02:15:13.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:15:13 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:15:13.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:15:13 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:15:13.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:15:13 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T02:15:14.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:15:14 smithi184 ceph-mon[35593]: pgmap v734: 97 pgs: 97 active+clean; 3.0 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.7 KiB/s wr, 13 op/s 2024-09-12T02:15:14.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:15:14 smithi033 ceph-mon[35332]: pgmap v734: 97 pgs: 97 active+clean; 3.0 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.7 KiB/s wr, 13 op/s 2024-09-12T02:15:14.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:15:14 smithi016 ceph-mon[30947]: pgmap v734: 97 pgs: 97 active+clean; 3.0 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.7 KiB/s wr, 13 op/s 2024-09-12T02:15:16.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:15:16 smithi184 ceph-mon[35593]: pgmap v735: 97 pgs: 97 active+clean; 3.0 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-09-12T02:15:16.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:15:16 smithi033 ceph-mon[35332]: pgmap v735: 97 pgs: 97 active+clean; 3.0 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-09-12T02:15:16.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:15:16 smithi016 ceph-mon[30947]: pgmap v735: 97 pgs: 97 active+clean; 3.0 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-09-12T02:15:18.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:15:18 smithi184 ceph-mon[35593]: pgmap v736: 97 pgs: 97 active+clean; 3.0 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 4.3 KiB/s wr, 11 op/s 2024-09-12T02:15:18.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:15:18 smithi033 ceph-mon[35332]: pgmap v736: 97 pgs: 97 active+clean; 3.0 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 4.3 KiB/s wr, 11 op/s 2024-09-12T02:15:18.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:15:18 smithi016 ceph-mon[30947]: pgmap v736: 97 pgs: 97 active+clean; 3.0 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 4.3 KiB/s wr, 11 op/s 2024-09-12T02:15:19.394 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.394+0000 7f3b699fd640 1 -- 172.21.15.16:0/561727560 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b64104f10 msgr2=0x7f3b64107300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:19.394 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.394+0000 7f3b699fd640 1 --2- 172.21.15.16:0/561727560 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b64104f10 0x7f3b64107300 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7f3b58009900 tx=0x7f3b5802a410 comp rx=0 tx=0).stop 2024-09-12T02:15:19.394 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.394+0000 7f3b699fd640 1 -- 172.21.15.16:0/561727560 shutdown_connections 2024-09-12T02:15:19.394 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.394+0000 7f3b699fd640 1 --2- 172.21.15.16:0/561727560 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3b64107840 0x7f3b64109cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:19.394 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.394+0000 7f3b699fd640 1 --2- 172.21.15.16:0/561727560 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b64104f10 0x7f3b64107300 unknown :-1 s=CLOSED pgs=1825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:19.395 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.394+0000 7f3b699fd640 1 --2- 172.21.15.16:0/561727560 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3b641025e0 0x7f3b641049d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:19.395 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.394+0000 7f3b699fd640 1 -- 172.21.15.16:0/561727560 >> 172.21.15.16:0/561727560 conn(0x7f3b640fc1e0 msgr2=0x7f3b640fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:15:19.395 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.394+0000 7f3b699fd640 1 -- 172.21.15.16:0/561727560 shutdown_connections 2024-09-12T02:15:19.395 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.394+0000 7f3b699fd640 1 -- 172.21.15.16:0/561727560 wait complete. 2024-09-12T02:15:19.395 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.394+0000 7f3b699fd640 1 Processor -- start 2024-09-12T02:15:19.397 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.395+0000 7f3b699fd640 1 -- start start 2024-09-12T02:15:19.397 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.395+0000 7f3b699fd640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3b641025e0 0x7f3b64197e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:19.397 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.395+0000 7f3b699fd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b64104f10 0x7f3b641983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:19.397 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.395+0000 7f3b699fd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3b64107840 0x7f3b6419f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:19.397 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.395+0000 7f3b699fd640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3b6410d7f0 con 0x7f3b64104f10 2024-09-12T02:15:19.397 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.395+0000 7f3b699fd640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3b6410d670 con 0x7f3b64107840 2024-09-12T02:15:19.397 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.395+0000 7f3b699fd640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f3b6410d970 con 0x7f3b641025e0 2024-09-12T02:15:19.397 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.395+0000 7f3b627fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b64104f10 0x7f3b641983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:19.398 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.395+0000 7f3b627fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b64104f10 0x7f3b641983b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55626/0 (socket says 172.21.15.16:55626) 2024-09-12T02:15:19.398 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.395+0000 7f3b627fc640 1 -- 172.21.15.16:0/845044373 learned_addr learned my addr 172.21.15.16:0/845044373 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:15:19.398 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.395+0000 7f3b62ffd640 1 --2- 172.21.15.16:0/845044373 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3b641025e0 0x7f3b64197e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:19.398 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.395+0000 7f3b637fe640 1 --2- 172.21.15.16:0/845044373 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3b64107840 0x7f3b6419f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:19.398 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.395+0000 7f3b627fc640 1 -- 172.21.15.16:0/845044373 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3b641025e0 msgr2=0x7f3b64197e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:19.398 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.395+0000 7f3b627fc640 1 --2- 172.21.15.16:0/845044373 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3b641025e0 0x7f3b64197e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:19.398 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.395+0000 7f3b627fc640 1 -- 172.21.15.16:0/845044373 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3b64107840 msgr2=0x7f3b6419f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:19.398 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.395+0000 7f3b627fc640 1 --2- 172.21.15.16:0/845044373 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3b64107840 0x7f3b6419f420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:19.398 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.395+0000 7f3b627fc640 1 -- 172.21.15.16:0/845044373 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b6419fa90 con 0x7f3b64104f10 2024-09-12T02:15:19.398 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.396+0000 7f3b627fc640 1 --2- 172.21.15.16:0/845044373 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b64104f10 0x7f3b641983b0 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7f3b58031610 tx=0x7f3b580043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:15:19.398 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.396+0000 7f3b689fb640 1 -- 172.21.15.16:0/845044373 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b5803f030 con 0x7f3b64104f10 2024-09-12T02:15:19.398 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.396+0000 7f3b689fb640 1 -- 172.21.15.16:0/845044373 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b58003ce0 con 0x7f3b64104f10 2024-09-12T02:15:19.398 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.396+0000 7f3b689fb640 1 -- 172.21.15.16:0/845044373 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b58035070 con 0x7f3b64104f10 2024-09-12T02:15:19.398 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.396+0000 7f3b699fd640 1 -- 172.21.15.16:0/845044373 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b6419fd20 con 0x7f3b64104f10 2024-09-12T02:15:19.399 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.396+0000 7f3b699fd640 1 -- 172.21.15.16:0/845044373 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b641a0200 con 0x7f3b64104f10 2024-09-12T02:15:19.401 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.398+0000 7f3b689fb640 1 -- 172.21.15.16:0/845044373 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f3b58042030 con 0x7f3b64104f10 2024-09-12T02:15:19.402 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.399+0000 7f3b6c881740 1 Processor -- start 2024-09-12T02:15:19.402 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.399+0000 7f3b6c881740 1 -- start start 2024-09-12T02:15:19.402 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.399+0000 7f3b6c881740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5641a89a49a0 0x5641a89a4da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:19.402 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.399+0000 7f3b6c881740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5641a89a52e0 0x5641a89a9da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:19.402 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.399+0000 7f3b6c881740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5641a89aa370 0x5641a89ac830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:19.402 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.399+0000 7f3b6c881740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5641a89082f0 con 0x5641a89a52e0 2024-09-12T02:15:19.402 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.399+0000 7f3b6c881740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5641a88b0270 con 0x5641a89a49a0 2024-09-12T02:15:19.402 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.399+0000 7f3b6c881740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5641a89042b0 con 0x5641a89aa370 2024-09-12T02:15:19.402 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.399+0000 7f3b62ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5641a89a52e0 0x5641a89a9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:19.402 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.399+0000 7f3b62ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5641a89a52e0 0x5641a89a9da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55632/0 (socket says 172.21.15.16:55632) 2024-09-12T02:15:19.402 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.400+0000 7f3b62ffd640 1 -- 172.21.15.16:0/4193265951 learned_addr learned my addr 172.21.15.16:0/4193265951 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:15:19.402 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.400+0000 7f3b689fb640 1 --2- 172.21.15.16:0/845044373 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f3b4c0a2cb0 0x7f3b4c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:19.402 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.400+0000 7f3b637fe640 1 --2- 172.21.15.16:0/4193265951 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5641a89a49a0 0x5641a89a4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:19.403 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.400+0000 7f3b627fc640 1 --2- 172.21.15.16:0/4193265951 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5641a89aa370 0x5641a89ac830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:19.403 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.400+0000 7f3b62ffd640 1 -- 172.21.15.16:0/4193265951 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5641a89aa370 msgr2=0x5641a89ac830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:15:19.403 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.400+0000 7f3b62ffd640 1 --2- 172.21.15.16:0/4193265951 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5641a89aa370 0x5641a89ac830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:19.403 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.400+0000 7f3b62ffd640 1 -- 172.21.15.16:0/4193265951 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5641a89a49a0 msgr2=0x5641a89a4da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:15:19.403 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.400+0000 7f3b62ffd640 1 --2- 172.21.15.16:0/4193265951 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5641a89a49a0 0x5641a89a4da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:19.403 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.400+0000 7f3b62ffd640 1 -- 172.21.15.16:0/4193265951 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641a88af340 con 0x5641a89a52e0 2024-09-12T02:15:19.403 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.400+0000 7f3b637fe640 1 --2- 172.21.15.16:0/4193265951 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5641a89a49a0 0x5641a89a4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:15:19.403 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.400+0000 7f3b689fb640 1 -- 172.21.15.16:0/845044373 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f3b580ef180 con 0x7f3b64104f10 2024-09-12T02:15:19.403 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.400+0000 7f3b637fe640 1 --2- 172.21.15.16:0/845044373 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f3b4c0a2cb0 0x7f3b4c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:19.403 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.400+0000 7f3b62ffd640 1 --2- 172.21.15.16:0/4193265951 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5641a89a52e0 0x5641a89a9da0 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7f3b5400bd50 tx=0x7f3b540107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55554ea051e63a06 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:15:19.403 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.400+0000 7f3b49ffb640 1 -- 172.21.15.16:0/4193265951 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b54005220 con 0x5641a89a52e0 2024-09-12T02:15:19.403 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.400+0000 7f3b49ffb640 1 -- 172.21.15.16:0/4193265951 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b5400be40 con 0x5641a89a52e0 2024-09-12T02:15:19.403 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.400+0000 7f3b49ffb640 1 -- 172.21.15.16:0/4193265951 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b54011800 con 0x5641a89a52e0 2024-09-12T02:15:19.403 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.401+0000 7f3b6c881740 1 -- 172.21.15.16:0/4193265951 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5641a89a52e0 msgr2=0x5641a89a9da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:19.404 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.401+0000 7f3b637fe640 1 --2- 172.21.15.16:0/845044373 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f3b4c0a2cb0 0x7f3b4c0a5170 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f3b6406b3e0 tx=0x7f3b5c004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:15:19.404 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.401+0000 7f3b6c881740 1 --2- 172.21.15.16:0/4193265951 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5641a89a52e0 0x5641a89a9da0 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7f3b5400bd50 tx=0x7f3b540107a0 comp rx=0 tx=0).stop 2024-09-12T02:15:19.404 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.401+0000 7f3b6c881740 1 -- 172.21.15.16:0/4193265951 shutdown_connections 2024-09-12T02:15:19.404 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.401+0000 7f3b6c881740 1 --2- 172.21.15.16:0/4193265951 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5641a89aa370 0x5641a89ac830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:19.404 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.401+0000 7f3b6c881740 1 --2- 172.21.15.16:0/4193265951 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5641a89a52e0 0x5641a89a9da0 unknown :-1 s=CLOSED pgs=1827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:19.404 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.401+0000 7f3b6c881740 1 --2- 172.21.15.16:0/4193265951 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5641a89a49a0 0x5641a89a4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:19.404 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.401+0000 7f3b6c881740 1 -- 172.21.15.16:0/4193265951 >> 172.21.15.16:0/4193265951 conn(0x5641a89a0010 msgr2=0x5641a89a0410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:15:19.404 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.401+0000 7f3b6c881740 1 -- 172.21.15.16:0/4193265951 shutdown_connections 2024-09-12T02:15:19.404 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.401+0000 7f3b6c881740 1 -- 172.21.15.16:0/4193265951 wait complete. 2024-09-12T02:15:19.404 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.401+0000 7f3b6c881740 1 -- start start 2024-09-12T02:15:19.404 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.402+0000 7f3b6c881740 1 Processor -- start 2024-09-12T02:15:19.408 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.402+0000 7f3b6c881740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5641a89a49a0 0x5641a89aeb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:19.408 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.402+0000 7f3b6c881740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5641a89a52e0 0x5641a89af0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:19.408 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.402+0000 7f3b6c881740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5641a89aa370 0x5641a89b6110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:19.408 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.402+0000 7f3b6c881740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5641a89ad340 con 0x5641a89a49a0 2024-09-12T02:15:19.408 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.402+0000 7f3b6c881740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5641a8891290 con 0x5641a89aa370 2024-09-12T02:15:19.408 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.402+0000 7f3b6c881740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5641a89ad4c0 con 0x5641a89a52e0 2024-09-12T02:15:19.408 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.402+0000 7f3b6c881740 10 client.0 _ll_register_callbacks cb 0x5641a88c2140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:15:19.408 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.402+0000 7f3b62ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5641a89a49a0 0x5641a89aeb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:19.409 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.402+0000 7f3b62ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5641a89a49a0 0x5641a89aeb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55640/0 (socket says 172.21.15.16:55640) 2024-09-12T02:15:19.409 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.402+0000 7f3b62ffd640 1 -- 172.21.15.16:0/4132846779 learned_addr learned my addr 172.21.15.16:0/4132846779 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:15:19.409 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.402+0000 7f3b627fc640 1 --2- 172.21.15.16:0/4132846779 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5641a89a52e0 0x5641a89af0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:19.409 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.402+0000 7f3b637fe640 1 --2- 172.21.15.16:0/4132846779 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5641a89aa370 0x5641a89b6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:19.409 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.403+0000 7f3b62ffd640 1 -- 172.21.15.16:0/4132846779 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5641a89a52e0 msgr2=0x5641a89af0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:15:19.409 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.403+0000 7f3b62ffd640 1 --2- 172.21.15.16:0/4132846779 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5641a89a52e0 0x5641a89af0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:19.409 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.403+0000 7f3b62ffd640 1 -- 172.21.15.16:0/4132846779 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5641a89aa370 msgr2=0x5641a89b6110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:15:19.409 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.403+0000 7f3b62ffd640 1 --2- 172.21.15.16:0/4132846779 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5641a89aa370 0x5641a89b6110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:19.409 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.403+0000 7f3b62ffd640 1 -- 172.21.15.16:0/4132846779 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5641a861b210 con 0x5641a89a49a0 2024-09-12T02:15:19.409 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.403+0000 7f3b637fe640 1 --2- 172.21.15.16:0/4132846779 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5641a89aa370 0x5641a89b6110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:15:19.409 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.403+0000 7f3b62ffd640 1 --2- 172.21.15.16:0/4132846779 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5641a89a49a0 0x5641a89aeb90 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7f3b54000c00 tx=0x7f3b540117e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2b23dd21d2bbd50 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:15:19.409 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.403+0000 7f3b42ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:15:19.409 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.403+0000 7f3b42ffd640 1 -- 172.21.15.16:0/4132846779 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b54011be0 con 0x5641a89a49a0 2024-09-12T02:15:19.410 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.403+0000 7f3b42ffd640 1 -- 172.21.15.16:0/4132846779 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f3b54011d80 con 0x5641a89a49a0 2024-09-12T02:15:19.410 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.403+0000 7f3b6c881740 1 -- 172.21.15.16:0/4132846779 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5641a8915a20 con 0x5641a89a49a0 2024-09-12T02:15:19.410 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.403+0000 7f3b42ffd640 1 -- 172.21.15.16:0/4132846779 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f3b54004020 con 0x5641a89a49a0 2024-09-12T02:15:19.410 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.403+0000 7f3b6c881740 10 client.19902 fetch_fsmap learned FSMap version 25 2024-09-12T02:15:19.410 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.403+0000 7f3b6c881740 1 -- 172.21.15.16:0/4132846779 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5641a86e7620 con 0x5641a89a49a0 2024-09-12T02:15:19.410 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.403+0000 7f3b42ffd640 1 -- 172.21.15.16:0/4132846779 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5641a8915a20 con 0x5641a89a49a0 2024-09-12T02:15:19.410 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.404+0000 7f3b42ffd640 1 -- 172.21.15.16:0/4132846779 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f3b54026470 con 0x5641a89a49a0 2024-09-12T02:15:19.410 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.404+0000 7f3b6c881740 10 client.19902 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:15:19.410 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.404+0000 7f3b6c881740 10 client.19902 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:15:19.410 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.404+0000 7f3b6c881740 20 client.19902 populate_metadata read hostname 'smithi016' 2024-09-12T02:15:19.410 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.404+0000 7f3b6c881740 1 --2- 172.21.15.16:0/4132846779 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5641a89ba520 0x5641a89bc910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:19.410 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.404+0000 7f3b6c881740 4 client.19902 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:15:19.410 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.404+0000 7f3b6c881740 1 -- 172.21.15.16:0/4132846779 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5641a890fd70 con 0x5641a89ba520 2024-09-12T02:15:19.411 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.404+0000 7f3b627fc640 1 --2- 172.21.15.16:0/4132846779 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5641a89ba520 0x5641a89bc910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:19.411 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.404+0000 7f3b627fc640 1 --2- 172.21.15.16:0/4132846779 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5641a89ba520 0x5641a89bc910 crc :-1 s=READY pgs=401 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=991f8af9e73e82e0 server_cookie=8e32a6fb061912f5 in_seq=0 out_seq=0 2024-09-12T02:15:19.411 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.404+0000 7f3b42ffd640 10 client.19902 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:15:19.411 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.405+0000 7f3b42ffd640 1 -- 172.21.15.16:0/4132846779 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5641a890fd70 con 0x5641a89ba520 2024-09-12T02:15:19.411 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.405+0000 7f3b42ffd640 10 client.19902 handle_command_reply: tid=0 2024-09-12T02:15:19.411 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.405+0000 7f3b6c881740 1 client.19902 shutdown 2024-09-12T02:15:19.411 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.405+0000 7f3b6c881740 1 -- 172.21.15.16:0/4132846779 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5641a89a49a0 msgr2=0x5641a89aeb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:19.411 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.405+0000 7f3b6c881740 1 --2- 172.21.15.16:0/4132846779 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5641a89a49a0 0x5641a89aeb90 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7f3b54000c00 tx=0x7f3b540117e0 comp rx=0 tx=0).stop 2024-09-12T02:15:19.411 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.405+0000 7f3b6c881740 1 -- 172.21.15.16:0/4132846779 shutdown_connections 2024-09-12T02:15:19.411 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.405+0000 7f3b6c881740 1 --2- 172.21.15.16:0/4132846779 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5641a89ba520 0x5641a89bc910 crc :-1 s=READY pgs=401 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:19.411 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.405+0000 7f3b6c881740 1 --2- 172.21.15.16:0/4132846779 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5641a89aa370 0x5641a89b6110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:19.411 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.405+0000 7f3b6c881740 1 --2- 172.21.15.16:0/4132846779 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5641a89a52e0 0x5641a89af0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:19.411 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.405+0000 7f3b6c881740 1 --2- 172.21.15.16:0/4132846779 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5641a89a49a0 0x5641a89aeb90 unknown :-1 s=CLOSED pgs=1828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:19.412 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.405+0000 7f3b42ffd640 0 client.19902 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:15:19.412 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.405+0000 7f3b6c881740 1 -- 172.21.15.16:0/4132846779 >> 172.21.15.16:0/4132846779 conn(0x5641a89a0010 msgr2=0x5641a89a1820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:15:19.412 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.405+0000 7f3b6c881740 1 -- 172.21.15.16:0/4132846779 shutdown_connections 2024-09-12T02:15:19.412 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.405+0000 7f3b6c881740 1 -- 172.21.15.16:0/4132846779 wait complete. 2024-09-12T02:15:19.412 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.406+0000 7f3b6c881740 20 client.19902 trim_cache size 0 max 16384 2024-09-12T02:15:19.424 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.420+0000 7f3b6c881740 1 Processor -- start 2024-09-12T02:15:19.424 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.420+0000 7f3b6c881740 1 -- start start 2024-09-12T02:15:19.424 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.420+0000 7f3b6c881740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5641a89a49a0 0x5641a89af050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:19.425 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.420+0000 7f3b6c881740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5641a89a52e0 0x5641a89b60d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:19.425 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.420+0000 7f3b6c881740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5641a89aa370 0x5641a89a0e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:19.425 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.420+0000 7f3b6c881740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3b641ab7d0 con 0x5641a89aa370 2024-09-12T02:15:19.425 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.420+0000 7f3b6c881740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3b641a90a0 con 0x5641a89a49a0 2024-09-12T02:15:19.425 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.420+0000 7f3b6c881740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f3b641a6ef0 con 0x5641a89a52e0 2024-09-12T02:15:19.425 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.420+0000 7f3b637fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5641a89aa370 0x5641a89a0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:19.425 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.420+0000 7f3b637fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5641a89aa370 0x5641a89a0e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55644/0 (socket says 172.21.15.16:55644) 2024-09-12T02:15:19.425 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.420+0000 7f3b637fe640 1 -- 172.21.15.16:0/2652512205 learned_addr learned my addr 172.21.15.16:0/2652512205 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:15:19.425 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.421+0000 7f3b627fc640 1 --2- 172.21.15.16:0/2652512205 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5641a89a52e0 0x5641a89b60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:19.425 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.421+0000 7f3b62ffd640 1 --2- 172.21.15.16:0/2652512205 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5641a89a49a0 0x5641a89af050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:19.425 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.421+0000 7f3b637fe640 1 -- 172.21.15.16:0/2652512205 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5641a89a52e0 msgr2=0x5641a89b60d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:15:19.425 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.421+0000 7f3b637fe640 1 --2- 172.21.15.16:0/2652512205 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5641a89a52e0 0x5641a89b60d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:19.425 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.421+0000 7f3b637fe640 1 -- 172.21.15.16:0/2652512205 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5641a89a49a0 msgr2=0x5641a89af050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:15:19.426 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.421+0000 7f3b637fe640 1 --2- 172.21.15.16:0/2652512205 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5641a89a49a0 0x5641a89af050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:19.426 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.421+0000 7f3b637fe640 1 -- 172.21.15.16:0/2652512205 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641a89bcf40 con 0x5641a89aa370 2024-09-12T02:15:19.426 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.421+0000 7f3b62ffd640 1 --2- 172.21.15.16:0/2652512205 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5641a89a49a0 0x5641a89af050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:15:19.426 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.421+0000 7f3b637fe640 1 --2- 172.21.15.16:0/2652512205 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5641a89aa370 0x5641a89a0e90 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7f3b5c0137e0 tx=0x7f3b5c015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecf6cb82c97e606c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:15:19.426 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.421+0000 7f3b497fa640 1 -- 172.21.15.16:0/2652512205 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b5c01f9d0 con 0x5641a89aa370 2024-09-12T02:15:19.426 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.421+0000 7f3b497fa640 1 -- 172.21.15.16:0/2652512205 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b5c017070 con 0x5641a89aa370 2024-09-12T02:15:19.426 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.421+0000 7f3b497fa640 1 -- 172.21.15.16:0/2652512205 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b5c0273e0 con 0x5641a89aa370 2024-09-12T02:15:19.426 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.421+0000 7f3b6c881740 1 -- 172.21.15.16:0/2652512205 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5641a89aa370 msgr2=0x5641a89a0e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:19.426 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.421+0000 7f3b6c881740 1 --2- 172.21.15.16:0/2652512205 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5641a89aa370 0x5641a89a0e90 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7f3b5c0137e0 tx=0x7f3b5c015000 comp rx=0 tx=0).stop 2024-09-12T02:15:19.426 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.422+0000 7f3b6c881740 1 -- 172.21.15.16:0/2652512205 shutdown_connections 2024-09-12T02:15:19.426 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.422+0000 7f3b6c881740 1 --2- 172.21.15.16:0/2652512205 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5641a89aa370 0x5641a89a0e90 unknown :-1 s=CLOSED pgs=1829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:19.426 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.422+0000 7f3b6c881740 1 --2- 172.21.15.16:0/2652512205 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5641a89a52e0 0x5641a89b60d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:19.426 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.422+0000 7f3b6c881740 1 --2- 172.21.15.16:0/2652512205 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5641a89a49a0 0x5641a89af050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:19.427 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.422+0000 7f3b6c881740 1 -- 172.21.15.16:0/2652512205 >> 172.21.15.16:0/2652512205 conn(0x5641a89a0010 msgr2=0x5641a89ae940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:15:19.427 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.422+0000 7f3b6c881740 1 -- 172.21.15.16:0/2652512205 shutdown_connections 2024-09-12T02:15:19.427 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.422+0000 7f3b6c881740 1 -- 172.21.15.16:0/2652512205 wait complete. 2024-09-12T02:15:19.427 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.422+0000 7f3b6c881740 1 -- start start 2024-09-12T02:15:19.427 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.422+0000 7f3b6c881740 1 Processor -- start 2024-09-12T02:15:19.427 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.422+0000 7f3b6c881740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5641a89a49a0 0x5641a89b38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:19.427 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.422+0000 7f3b6c881740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5641a89a52e0 0x5641a89b3de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:19.427 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.422+0000 7f3b6c881740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5641a89aa370 0x5641a89b4df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:19.427 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.422+0000 7f3b6c881740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3b641a7560 con 0x5641a89a49a0 2024-09-12T02:15:19.427 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.422+0000 7f3b6c881740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3b641a89d0 con 0x5641a89aa370 2024-09-12T02:15:19.427 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.422+0000 7f3b6c881740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f3b641a7c30 con 0x5641a89a52e0 2024-09-12T02:15:19.427 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.422+0000 7f3b6c881740 10 client.0 _ll_register_callbacks cb 0x7f3b5400bba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:15:19.427 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.422+0000 7f3b62ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5641a89a49a0 0x5641a89b38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:19.428 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.423+0000 7f3b62ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5641a89a49a0 0x5641a89b38a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55658/0 (socket says 172.21.15.16:55658) 2024-09-12T02:15:19.428 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.423+0000 7f3b62ffd640 1 -- 172.21.15.16:0/3322869092 learned_addr learned my addr 172.21.15.16:0/3322869092 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:15:19.428 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.423+0000 7f3b627fc640 1 --2- 172.21.15.16:0/3322869092 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5641a89a52e0 0x5641a89b3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:19.428 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.423+0000 7f3b637fe640 1 --2- 172.21.15.16:0/3322869092 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5641a89aa370 0x5641a89b4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:19.428 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.423+0000 7f3b62ffd640 1 -- 172.21.15.16:0/3322869092 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5641a89a52e0 msgr2=0x5641a89b3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:15:19.428 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.423+0000 7f3b62ffd640 1 --2- 172.21.15.16:0/3322869092 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5641a89a52e0 0x5641a89b3de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:19.428 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.423+0000 7f3b62ffd640 1 -- 172.21.15.16:0/3322869092 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5641a89aa370 msgr2=0x5641a89b4df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:15:19.428 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.423+0000 7f3b62ffd640 1 --2- 172.21.15.16:0/3322869092 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5641a89aa370 0x5641a89b4df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:19.428 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.423+0000 7f3b62ffd640 1 -- 172.21.15.16:0/3322869092 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5641a89b5400 con 0x5641a89a49a0 2024-09-12T02:15:19.428 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.423+0000 7f3b637fe640 1 --2- 172.21.15.16:0/3322869092 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5641a89aa370 0x5641a89b4df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:15:19.428 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.423+0000 7f3b62ffd640 1 --2- 172.21.15.16:0/3322869092 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5641a89a49a0 0x5641a89b38a0 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7f3b54010ca0 tx=0x7f3b5400b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc3fa6987ca8ffde server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:15:19.428 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.423+0000 7f3b42ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:15:19.428 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.423+0000 7f3b42ffd640 1 -- 172.21.15.16:0/3322869092 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b54026cc0 con 0x5641a89a49a0 2024-09-12T02:15:19.429 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.423+0000 7f3b6c881740 1 -- 172.21.15.16:0/3322869092 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b5c017070 con 0x5641a89a49a0 2024-09-12T02:15:19.429 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.423+0000 7f3b42ffd640 1 -- 172.21.15.16:0/3322869092 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f3b54011be0 con 0x5641a89a49a0 2024-09-12T02:15:19.429 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.423+0000 7f3b42ffd640 1 -- 172.21.15.16:0/3322869092 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f3b54041050 con 0x5641a89a49a0 2024-09-12T02:15:19.429 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.423+0000 7f3b6c881740 10 client.19908 fetch_fsmap learned FSMap version 25 2024-09-12T02:15:19.429 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.423+0000 7f3b6c881740 1 -- 172.21.15.16:0/3322869092 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5641a8a25140 con 0x5641a89a49a0 2024-09-12T02:15:19.429 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.424+0000 7f3b42ffd640 1 -- 172.21.15.16:0/3322869092 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b5c017070 con 0x5641a89a49a0 2024-09-12T02:15:19.429 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.424+0000 7f3b42ffd640 1 -- 172.21.15.16:0/3322869092 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f3b54003b60 con 0x5641a89a49a0 2024-09-12T02:15:19.429 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.424+0000 7f3b6c881740 10 client.19908 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:15:19.429 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.424+0000 7f3b6c881740 10 client.19908 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:15:19.429 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.424+0000 7f3b6c881740 20 client.19908 populate_metadata read hostname 'smithi016' 2024-09-12T02:15:19.429 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.424+0000 7f3b6c881740 1 --2- 172.21.15.16:0/3322869092 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5641a8a25b10 0x5641a8a27f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:19.429 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.424+0000 7f3b6c881740 4 client.19908 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:15:19.429 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.424+0000 7f3b6c881740 1 -- 172.21.15.16:0/3322869092 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5641a888c900 con 0x5641a8a25b10 2024-09-12T02:15:19.430 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.424+0000 7f3b627fc640 1 --2- 172.21.15.16:0/3322869092 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5641a8a25b10 0x5641a8a27f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:19.430 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.424+0000 7f3b627fc640 1 --2- 172.21.15.16:0/3322869092 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5641a8a25b10 0x5641a8a27f30 crc :-1 s=READY pgs=402 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a788ea457d8cc73 server_cookie=cc821563ae8f7b86 in_seq=0 out_seq=0 2024-09-12T02:15:19.430 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.424+0000 7f3b42ffd640 10 client.19908 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:15:19.436 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.436+0000 7f3b42ffd640 1 -- 172.21.15.16:0/3322869092 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5641a888c900 con 0x5641a8a25b10 2024-09-12T02:15:19.436 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.436+0000 7f3b42ffd640 10 client.19908 handle_command_reply: tid=0 2024-09-12T02:15:19.436 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.436+0000 7f3b6c881740 1 client.19908 shutdown 2024-09-12T02:15:19.437 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.436+0000 7f3b6c881740 1 -- 172.21.15.16:0/3322869092 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5641a89a49a0 msgr2=0x5641a89b38a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:19.437 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.436+0000 7f3b6c881740 1 --2- 172.21.15.16:0/3322869092 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5641a89a49a0 0x5641a89b38a0 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7f3b54010ca0 tx=0x7f3b5400b980 comp rx=0 tx=0).stop 2024-09-12T02:15:19.437 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.436+0000 7f3b6c881740 1 -- 172.21.15.16:0/3322869092 shutdown_connections 2024-09-12T02:15:19.437 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.436+0000 7f3b6c881740 1 --2- 172.21.15.16:0/3322869092 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5641a8a25b10 0x5641a8a27f30 crc :-1 s=READY pgs=402 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:19.437 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.436+0000 7f3b6c881740 1 --2- 172.21.15.16:0/3322869092 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5641a89aa370 0x5641a89b4df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:19.437 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.436+0000 7f3b6c881740 1 --2- 172.21.15.16:0/3322869092 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5641a89a52e0 0x5641a89b3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:19.437 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.436+0000 7f3b42ffd640 0 client.19908 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:15:19.437 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.436+0000 7f3b6c881740 1 --2- 172.21.15.16:0/3322869092 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5641a89a49a0 0x5641a89b38a0 unknown :-1 s=CLOSED pgs=1830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:19.437 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.436+0000 7f3b6c881740 1 -- 172.21.15.16:0/3322869092 >> 172.21.15.16:0/3322869092 conn(0x5641a89a0010 msgr2=0x5641a899e9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:15:19.437 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.436+0000 7f3b6c881740 1 -- 172.21.15.16:0/3322869092 shutdown_connections 2024-09-12T02:15:19.437 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.436+0000 7f3b6c881740 1 -- 172.21.15.16:0/3322869092 wait complete. 2024-09-12T02:15:19.437 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.436+0000 7f3b6c881740 20 client.19908 trim_cache size 0 max 16384 2024-09-12T02:15:19.438 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-09-12T02:15:19.438 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-09-12T02:15:19.438 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-09-12T02:15:19.438 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-09-12T02:15:19.439 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.437+0000 7f3b699fd640 1 -- 172.21.15.16:0/845044373 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f3b4c0a2cb0 msgr2=0x7f3b4c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:19.439 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.437+0000 7f3b699fd640 1 --2- 172.21.15.16:0/845044373 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f3b4c0a2cb0 0x7f3b4c0a5170 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f3b6406b3e0 tx=0x7f3b5c004a40 comp rx=0 tx=0).stop 2024-09-12T02:15:19.439 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.437+0000 7f3b699fd640 1 -- 172.21.15.16:0/845044373 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b64104f10 msgr2=0x7f3b641983b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:19.439 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.437+0000 7f3b699fd640 1 --2- 172.21.15.16:0/845044373 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b64104f10 0x7f3b641983b0 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7f3b58031610 tx=0x7f3b580043b0 comp rx=0 tx=0).stop 2024-09-12T02:15:19.439 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.437+0000 7f3b699fd640 1 -- 172.21.15.16:0/845044373 shutdown_connections 2024-09-12T02:15:19.439 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.437+0000 7f3b699fd640 1 --2- 172.21.15.16:0/845044373 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f3b4c0a2cb0 0x7f3b4c0a5170 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:19.439 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.437+0000 7f3b699fd640 1 --2- 172.21.15.16:0/845044373 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3b64107840 0x7f3b6419f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:19.439 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.437+0000 7f3b699fd640 1 --2- 172.21.15.16:0/845044373 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b64104f10 0x7f3b641983b0 unknown :-1 s=CLOSED pgs=1826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:19.439 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.437+0000 7f3b699fd640 1 --2- 172.21.15.16:0/845044373 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3b641025e0 0x7f3b64197e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:19.439 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.437+0000 7f3b699fd640 1 -- 172.21.15.16:0/845044373 >> 172.21.15.16:0/845044373 conn(0x7f3b640fc1e0 msgr2=0x7f3b64103040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:15:19.440 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.437+0000 7f3b699fd640 1 -- 172.21.15.16:0/845044373 shutdown_connections 2024-09-12T02:15:19.440 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:19.437+0000 7f3b699fd640 1 -- 172.21.15.16:0/845044373 wait complete. 2024-09-12T02:15:20.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:15:20 smithi184 ceph-mon[35593]: pgmap v737: 97 pgs: 97 active+clean; 3.0 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.5 KiB/s wr, 9 op/s 2024-09-12T02:15:20.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:15:20 smithi033 ceph-mon[35332]: pgmap v737: 97 pgs: 97 active+clean; 3.0 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.5 KiB/s wr, 9 op/s 2024-09-12T02:15:20.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:15:20 smithi016 ceph-mon[30947]: pgmap v737: 97 pgs: 97 active+clean; 3.0 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.5 KiB/s wr, 9 op/s 2024-09-12T02:15:22.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:15:22 smithi184 ceph-mon[35593]: pgmap v738: 97 pgs: 97 active+clean; 3.0 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.6 KiB/s wr, 11 op/s 2024-09-12T02:15:22.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:15:22 smithi033 ceph-mon[35332]: pgmap v738: 97 pgs: 97 active+clean; 3.0 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.6 KiB/s wr, 11 op/s 2024-09-12T02:15:22.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:15:22 smithi016 ceph-mon[30947]: pgmap v738: 97 pgs: 97 active+clean; 3.0 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.6 KiB/s wr, 11 op/s 2024-09-12T02:15:24.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:15:24 smithi184 ceph-mon[35593]: pgmap v739: 97 pgs: 97 active+clean; 3.0 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 3.3 KiB/s wr, 9 op/s 2024-09-12T02:15:24.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:15:24 smithi033 ceph-mon[35332]: pgmap v739: 97 pgs: 97 active+clean; 3.0 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 3.3 KiB/s wr, 9 op/s 2024-09-12T02:15:24.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:15:24 smithi016 ceph-mon[30947]: pgmap v739: 97 pgs: 97 active+clean; 3.0 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 3.3 KiB/s wr, 9 op/s 2024-09-12T02:15:26.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:15:26 smithi184 ceph-mon[35593]: pgmap v740: 97 pgs: 97 active+clean; 3.0 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-09-12T02:15:26.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:15:26 smithi033 ceph-mon[35332]: pgmap v740: 97 pgs: 97 active+clean; 3.0 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-09-12T02:15:26.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:15:26 smithi016 ceph-mon[30947]: pgmap v740: 97 pgs: 97 active+clean; 3.0 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-09-12T02:15:28.168 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:15:28.441 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:15:28.486 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:15:28 smithi016 ceph-mon[30947]: pgmap v741: 97 pgs: 97 active+clean; 3.0 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-09-12T02:15:28.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:15:28 smithi184 ceph-mon[35593]: pgmap v741: 97 pgs: 97 active+clean; 3.0 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-09-12T02:15:28.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:15:28 smithi033 ceph-mon[35332]: pgmap v741: 97 pgs: 97 active+clean; 3.0 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-09-12T02:15:28.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:28.967+0000 7f4ab46bb640 1 -- 172.21.15.16:0/3896189470 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4aac102d90 msgr2=0x7f4aac103170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:28.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:28.967+0000 7f4ab46bb640 1 --2- 172.21.15.16:0/3896189470 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4aac102d90 0x7f4aac103170 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7f4aa8009930 tx=0x7f4aa802a420 comp rx=0 tx=0).stop 2024-09-12T02:15:28.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:28.967+0000 7f4ab46bb640 1 -- 172.21.15.16:0/3896189470 shutdown_connections 2024-09-12T02:15:28.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:28.967+0000 7f4ab46bb640 1 --2- 172.21.15.16:0/3896189470 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4aac108290 0x7f4aac10ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:28.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:28.967+0000 7f4ab46bb640 1 --2- 172.21.15.16:0/3896189470 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4aac103740 0x7f4aac107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:28.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:28.967+0000 7f4ab46bb640 1 --2- 172.21.15.16:0/3896189470 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4aac102d90 0x7f4aac103170 unknown :-1 s=CLOSED pgs=1831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:28.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:28.967+0000 7f4ab46bb640 1 -- 172.21.15.16:0/3896189470 >> 172.21.15.16:0/3896189470 conn(0x7f4aac0fc7d0 msgr2=0x7f4aac0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:15:28.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:28.967+0000 7f4ab46bb640 1 -- 172.21.15.16:0/3896189470 shutdown_connections 2024-09-12T02:15:28.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:28.967+0000 7f4ab46bb640 1 -- 172.21.15.16:0/3896189470 wait complete. 2024-09-12T02:15:28.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:28.968+0000 7f4ab46bb640 1 Processor -- start 2024-09-12T02:15:28.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:28.968+0000 7f4ab46bb640 1 -- start start 2024-09-12T02:15:28.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:28.968+0000 7f4ab46bb640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4aac102d90 0x7f4aac1009f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:28.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:28.968+0000 7f4ab46bb640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4aac103740 0x7f4aac100f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:28.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:28.968+0000 7f4ab46bb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4aac108290 0x7f4aac101470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:28.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:28.968+0000 7f4ab46bb640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4aac112090 con 0x7f4aac108290 2024-09-12T02:15:28.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:28.968+0000 7f4ab46bb640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4aac111f10 con 0x7f4aac102d90 2024-09-12T02:15:28.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:28.968+0000 7f4ab46bb640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f4aac112210 con 0x7f4aac103740 2024-09-12T02:15:28.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:28.968+0000 7f4ab2c58640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4aac108290 0x7f4aac101470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:28.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:28.969+0000 7f4ab2c58640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4aac108290 0x7f4aac101470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56398/0 (socket says 172.21.15.16:56398) 2024-09-12T02:15:28.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:28.969+0000 7f4ab2c58640 1 -- 172.21.15.16:0/3843618764 learned_addr learned my addr 172.21.15.16:0/3843618764 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:15:28.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:28.969+0000 7f4ab2457640 1 --2- 172.21.15.16:0/3843618764 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4aac102d90 0x7f4aac1009f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:28.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:28.969+0000 7f4ab1c56640 1 --2- 172.21.15.16:0/3843618764 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4aac103740 0x7f4aac100f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:28.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:28.969+0000 7f4ab2c58640 1 -- 172.21.15.16:0/3843618764 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4aac103740 msgr2=0x7f4aac100f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:28.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:28.969+0000 7f4ab2c58640 1 --2- 172.21.15.16:0/3843618764 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4aac103740 0x7f4aac100f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:28.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:28.969+0000 7f4ab2c58640 1 -- 172.21.15.16:0/3843618764 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4aac102d90 msgr2=0x7f4aac1009f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:28.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:28.969+0000 7f4ab2c58640 1 --2- 172.21.15.16:0/3843618764 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4aac102d90 0x7f4aac1009f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:28.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:28.969+0000 7f4ab2c58640 1 -- 172.21.15.16:0/3843618764 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4aac1a6a60 con 0x7f4aac108290 2024-09-12T02:15:28.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:28.969+0000 7f4ab2c58640 1 --2- 172.21.15.16:0/3843618764 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4aac108290 0x7f4aac101470 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7f4aa400eb10 tx=0x7f4aa400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:15:28.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:28.969+0000 7f4aa37fe640 1 -- 172.21.15.16:0/3843618764 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aa4004b20 con 0x7f4aac108290 2024-09-12T02:15:28.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:28.969+0000 7f4ab46bb640 1 -- 172.21.15.16:0/3843618764 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4aac1a6d50 con 0x7f4aac108290 2024-09-12T02:15:28.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:28.969+0000 7f4aa37fe640 1 -- 172.21.15.16:0/3843618764 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4aa4004cc0 con 0x7f4aac108290 2024-09-12T02:15:28.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:28.969+0000 7f4aa37fe640 1 -- 172.21.15.16:0/3843618764 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aa4010760 con 0x7f4aac108290 2024-09-12T02:15:28.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:28.969+0000 7f4ab46bb640 1 -- 172.21.15.16:0/3843618764 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4aac1a7240 con 0x7f4aac108290 2024-09-12T02:15:28.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:28.970+0000 7f4ab46bb640 1 -- 172.21.15.16:0/3843618764 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4aac06a7f0 con 0x7f4aac108290 2024-09-12T02:15:28.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:28.972+0000 7f4aa37fe640 1 -- 172.21.15.16:0/3843618764 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f4aa4010900 con 0x7f4aac108290 2024-09-12T02:15:28.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:28.972+0000 7f4aa37fe640 1 --2- 172.21.15.16:0/3843618764 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f4a980a2b80 0x7f4a980a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:28.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:28.973+0000 7f4ab2457640 1 --2- 172.21.15.16:0/3843618764 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f4a980a2b80 0x7f4a980a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:28.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:28.973+0000 7f4aa37fe640 1 -- 172.21.15.16:0/3843618764 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f4aa40cf770 con 0x7f4aac108290 2024-09-12T02:15:28.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:28.973+0000 7f4ab2457640 1 --2- 172.21.15.16:0/3843618764 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f4a980a2b80 0x7f4a980a5040 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f4aa8009900 tx=0x7f4aa80058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:15:28.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:28.976+0000 7f4aa37fe640 1 -- 172.21.15.16:0/3843618764 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4aa40541b0 con 0x7f4aac108290 2024-09-12T02:15:29.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:29.191+0000 7f4ab46bb640 1 -- 172.21.15.16:0/3843618764 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4aac103170 con 0x7f4aac108290 2024-09-12T02:15:29.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:29.192+0000 7f4aa37fe640 1 -- 172.21.15.16:0/3843618764 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11911 (secure 0 0 0) 0x7f4aa409c770 con 0x7f4aac108290 2024-09-12T02:15:29.192 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T02:15:29.193 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":25,"btime":"2024-09-12T02:01:37:192021+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3279123279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3279123279},{"type":"v1","addr":"172.21.15.16:6835","nonce":3279123279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4119452382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4119452382},{"type":"v1","addr":"172.21.15.16:6837","nonce":4119452382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/3263823165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3263823165},{"type":"v1","addr":"172.21.15.16:6839","nonce":3263823165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/1510290732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":1510290732},{"type":"v1","addr":"172.21.15.16:6841","nonce":1510290732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/279674603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":279674603},{"type":"v1","addr":"172.21.15.33:6833","nonce":279674603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3555728028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3555728028},{"type":"v1","addr":"172.21.15.33:6835","nonce":3555728028}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T01:59:56.117608+0000","modified":"2024-09-12T02:01:36.294760+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24482,"mds_1":24481,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6833/1563374106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1563374106},{"type":"v1","addr":"172.21.15.184:6833","nonce":1563374106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6837/1892833868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1892833868},{"type":"v1","addr":"172.21.15.33:6837","nonce":1892833868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/1457679686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1457679686},{"type":"v1","addr":"172.21.15.184:6835","nonce":1457679686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.184:6837/1793393087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":1793393087},{"type":"v1","addr":"172.21.15.184:6837","nonce":1793393087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.184:6839/4203347941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":4203347941},{"type":"v1","addr":"172.21.15.184:6839","nonce":4203347941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.33:6839/4136670204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":4136670204},{"type":"v1","addr":"172.21.15.33:6839","nonce":4136670204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24481,24482,24469]},"id":1}]} 2024-09-12T02:15:29.193 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 25 2024-09-12T02:15:29.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:29.194+0000 7f4ab46bb640 1 -- 172.21.15.16:0/3843618764 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f4a980a2b80 msgr2=0x7f4a980a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:29.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:29.194+0000 7f4ab46bb640 1 --2- 172.21.15.16:0/3843618764 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f4a980a2b80 0x7f4a980a5040 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f4aa8009900 tx=0x7f4aa80058d0 comp rx=0 tx=0).stop 2024-09-12T02:15:29.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:29.194+0000 7f4ab46bb640 1 -- 172.21.15.16:0/3843618764 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4aac108290 msgr2=0x7f4aac101470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:29.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:29.194+0000 7f4ab46bb640 1 --2- 172.21.15.16:0/3843618764 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4aac108290 0x7f4aac101470 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7f4aa400eb10 tx=0x7f4aa400c2d0 comp rx=0 tx=0).stop 2024-09-12T02:15:29.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:29.194+0000 7f4ab46bb640 1 -- 172.21.15.16:0/3843618764 shutdown_connections 2024-09-12T02:15:29.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:29.194+0000 7f4ab46bb640 1 --2- 172.21.15.16:0/3843618764 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f4a980a2b80 0x7f4a980a5040 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:29.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:29.194+0000 7f4ab46bb640 1 --2- 172.21.15.16:0/3843618764 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4aac108290 0x7f4aac101470 unknown :-1 s=CLOSED pgs=1832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:29.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:29.194+0000 7f4ab46bb640 1 --2- 172.21.15.16:0/3843618764 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4aac103740 0x7f4aac100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:29.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:29.194+0000 7f4ab46bb640 1 --2- 172.21.15.16:0/3843618764 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4aac102d90 0x7f4aac1009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:29.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:29.194+0000 7f4ab46bb640 1 -- 172.21.15.16:0/3843618764 >> 172.21.15.16:0/3843618764 conn(0x7f4aac0fc7d0 msgr2=0x7f4aac105db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:15:29.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:29.194+0000 7f4ab46bb640 1 -- 172.21.15.16:0/3843618764 shutdown_connections 2024-09-12T02:15:29.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:29.194+0000 7f4ab46bb640 1 -- 172.21.15.16:0/3843618764 wait complete. 2024-09-12T02:15:29.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:15:29 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/3843618764' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:15:29.631 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:15:29 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/3843618764' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:15:29.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:15:29 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/3843618764' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:15:29.872 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24482 quiesce db --set-id 901e6dc3 --release --await' 2024-09-12T02:15:29.872 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24482 quiesce db --set-id 901e6dc3 --release --await 2024-09-12T02:15:30.132 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:15:30.540 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:15:30 smithi016 ceph-mon[30947]: pgmap v742: 97 pgs: 97 active+clean; 3.0 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.9 KiB/s wr, 7 op/s 2024-09-12T02:15:30.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:15:30 smithi184 ceph-mon[35593]: pgmap v742: 97 pgs: 97 active+clean; 3.0 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.9 KiB/s wr, 7 op/s 2024-09-12T02:15:30.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.630+0000 7fea4138f640 1 -- 172.21.15.16:0/1414274356 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea3c101420 msgr2=0x7fea3c10d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:30.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.630+0000 7fea4138f640 1 --2- 172.21.15.16:0/1414274356 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea3c101420 0x7fea3c10d950 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7fea34009cd0 tx=0x7fea3402a8e0 comp rx=0 tx=0).stop 2024-09-12T02:15:30.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.630+0000 7fea4138f640 1 -- 172.21.15.16:0/1414274356 shutdown_connections 2024-09-12T02:15:30.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.630+0000 7fea4138f640 1 --2- 172.21.15.16:0/1414274356 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea3c101420 0x7fea3c10d950 unknown :-1 s=CLOSED pgs=1833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:30.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.630+0000 7fea4138f640 1 --2- 172.21.15.16:0/1414274356 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fea3c100a80 0x7fea3c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:30.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.630+0000 7fea4138f640 1 --2- 172.21.15.16:0/1414274356 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fea3c106a80 0x7fea3c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:30.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.630+0000 7fea4138f640 1 -- 172.21.15.16:0/1414274356 >> 172.21.15.16:0/1414274356 conn(0x7fea3c0fc7a0 msgr2=0x7fea3c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:15:30.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.631+0000 7fea4138f640 1 -- 172.21.15.16:0/1414274356 shutdown_connections 2024-09-12T02:15:30.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.631+0000 7fea4138f640 1 -- 172.21.15.16:0/1414274356 wait complete. 2024-09-12T02:15:30.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.631+0000 7fea4138f640 1 Processor -- start 2024-09-12T02:15:30.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.631+0000 7fea4138f640 1 -- start start 2024-09-12T02:15:30.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.632+0000 7fea4138f640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fea3c100a80 0x7fea3c198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:30.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.632+0000 7fea4138f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fea3c101420 0x7fea3c1988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:30.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.632+0000 7fea4138f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea3c106a80 0x7fea3c19cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:30.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.632+0000 7fea4138f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fea3c110ed0 con 0x7fea3c106a80 2024-09-12T02:15:30.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.632+0000 7fea4138f640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fea3c110d50 con 0x7fea3c101420 2024-09-12T02:15:30.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.632+0000 7fea4138f640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fea3c111050 con 0x7fea3c100a80 2024-09-12T02:15:30.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.632+0000 7fea3b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea3c106a80 0x7fea3c19cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:30.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.632+0000 7fea3b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea3c106a80 0x7fea3c19cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56416/0 (socket says 172.21.15.16:56416) 2024-09-12T02:15:30.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.632+0000 7fea3b7fe640 1 -- 172.21.15.16:0/3515656704 learned_addr learned my addr 172.21.15.16:0/3515656704 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:15:30.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.632+0000 7fea3affd640 1 --2- 172.21.15.16:0/3515656704 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fea3c100a80 0x7fea3c198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:30.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.632+0000 7fea327fc640 1 --2- 172.21.15.16:0/3515656704 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fea3c101420 0x7fea3c1988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:30.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.632+0000 7fea3b7fe640 1 -- 172.21.15.16:0/3515656704 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fea3c100a80 msgr2=0x7fea3c198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:30.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.632+0000 7fea3b7fe640 1 --2- 172.21.15.16:0/3515656704 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fea3c100a80 0x7fea3c198360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:30.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.632+0000 7fea3b7fe640 1 -- 172.21.15.16:0/3515656704 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fea3c101420 msgr2=0x7fea3c1988a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:30.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.632+0000 7fea3b7fe640 1 --2- 172.21.15.16:0/3515656704 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fea3c101420 0x7fea3c1988a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:30.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.632+0000 7fea3b7fe640 1 -- 172.21.15.16:0/3515656704 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea3c19d310 con 0x7fea3c106a80 2024-09-12T02:15:30.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.632+0000 7fea327fc640 1 --2- 172.21.15.16:0/3515656704 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fea3c101420 0x7fea3c1988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:15:30.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.632+0000 7fea3b7fe640 1 --2- 172.21.15.16:0/3515656704 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea3c106a80 0x7fea3c19cc30 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7fea34033040 tx=0x7fea34003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:15:30.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.632+0000 7fea38ff9640 1 -- 172.21.15.16:0/3515656704 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea34004040 con 0x7fea3c106a80 2024-09-12T02:15:30.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.633+0000 7fea38ff9640 1 -- 172.21.15.16:0/3515656704 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea340046c0 con 0x7fea3c106a80 2024-09-12T02:15:30.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.633+0000 7fea4138f640 1 -- 172.21.15.16:0/3515656704 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea3c19d5a0 con 0x7fea3c106a80 2024-09-12T02:15:30.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.633+0000 7fea38ff9640 1 -- 172.21.15.16:0/3515656704 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea34030eb0 con 0x7fea3c106a80 2024-09-12T02:15:30.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.633+0000 7fea4138f640 1 -- 172.21.15.16:0/3515656704 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea3c1a4ee0 con 0x7fea3c106a80 2024-09-12T02:15:30.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.635+0000 7fea38ff9640 1 -- 172.21.15.16:0/3515656704 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fea340041e0 con 0x7fea3c106a80 2024-09-12T02:15:30.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.635+0000 7fea43768740 1 Processor -- start 2024-09-12T02:15:30.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.635+0000 7fea43768740 1 -- start start 2024-09-12T02:15:30.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.636+0000 7fea43768740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55751850df60 0x55751850e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:30.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.636+0000 7fea43768740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55751850e880 0x55751851d110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:30.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.636+0000 7fea43768740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55751851d650 0x55751851fa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:30.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.636+0000 7fea43768740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x557518473620 con 0x55751850df60 2024-09-12T02:15:30.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.636+0000 7fea43768740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55751841ae50 con 0x55751851d650 2024-09-12T02:15:30.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.636+0000 7fea43768740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55751846f5e0 con 0x55751850e880 2024-09-12T02:15:30.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.636+0000 7fea38ff9640 1 --2- 172.21.15.16:0/3515656704 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fea280a2b80 0x7fea280a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:30.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.636+0000 7fea327fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55751850df60 0x55751850e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:30.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.636+0000 7fea327fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55751850df60 0x55751850e340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56420/0 (socket says 172.21.15.16:56420) 2024-09-12T02:15:30.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.636+0000 7fea327fc640 1 -- 172.21.15.16:0/3656046209 learned_addr learned my addr 172.21.15.16:0/3656046209 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:15:30.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.636+0000 7fea3b7fe640 1 --2- 172.21.15.16:0/3515656704 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fea280a2b80 0x7fea280a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:30.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.636+0000 7fea327fc640 1 --2- 172.21.15.16:0/3656046209 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55751851d650 0x55751851fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:30.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.636+0000 7fea3affd640 1 --2- 172.21.15.16:0/3656046209 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55751850e880 0x55751851d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:30.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.636+0000 7fea327fc640 1 -- 172.21.15.16:0/3656046209 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55751850e880 msgr2=0x55751851d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:15:30.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.636+0000 7fea327fc640 1 --2- 172.21.15.16:0/3656046209 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55751850e880 0x55751851d110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:30.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.636+0000 7fea327fc640 1 -- 172.21.15.16:0/3656046209 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55751851d650 msgr2=0x55751851fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:15:30.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.636+0000 7fea327fc640 1 --2- 172.21.15.16:0/3656046209 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55751851d650 0x55751851fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:30.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.636+0000 7fea327fc640 1 -- 172.21.15.16:0/3656046209 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55751840bd40 con 0x55751850df60 2024-09-12T02:15:30.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.636+0000 7fea3affd640 1 --2- 172.21.15.16:0/3656046209 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55751850e880 0x55751851d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:15:30.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.636+0000 7fea38ff9640 1 -- 172.21.15.16:0/3515656704 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fea3402e030 con 0x7fea3c106a80 2024-09-12T02:15:30.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.637+0000 7fea327fc640 1 --2- 172.21.15.16:0/3656046209 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55751850df60 0x55751850e340 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7fea2400be80 tx=0x7fea2400b730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a84933a81a9db43 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:15:30.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.637+0000 7fea3b7fe640 1 --2- 172.21.15.16:0/3515656704 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fea280a2b80 0x7fea280a5040 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fea340b8df0 tx=0x7fea340c0040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:15:30.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.637+0000 7fea317fa640 1 -- 172.21.15.16:0/3656046209 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea24019030 con 0x55751850df60 2024-09-12T02:15:30.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.637+0000 7fea317fa640 1 -- 172.21.15.16:0/3656046209 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea24011d60 con 0x55751850df60 2024-09-12T02:15:30.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.637+0000 7fea317fa640 1 -- 172.21.15.16:0/3656046209 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea2401d9e0 con 0x55751850df60 2024-09-12T02:15:30.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.638+0000 7fea43768740 1 -- 172.21.15.16:0/3656046209 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55751850df60 msgr2=0x55751850e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:30.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.638+0000 7fea43768740 1 --2- 172.21.15.16:0/3656046209 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55751850df60 0x55751850e340 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7fea2400be80 tx=0x7fea2400b730 comp rx=0 tx=0).stop 2024-09-12T02:15:30.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.638+0000 7fea43768740 1 -- 172.21.15.16:0/3656046209 shutdown_connections 2024-09-12T02:15:30.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.638+0000 7fea43768740 1 --2- 172.21.15.16:0/3656046209 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55751851d650 0x55751851fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:30.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.638+0000 7fea43768740 1 --2- 172.21.15.16:0/3656046209 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55751850e880 0x55751851d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:30.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.638+0000 7fea43768740 1 --2- 172.21.15.16:0/3656046209 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55751850df60 0x55751850e340 unknown :-1 s=CLOSED pgs=1835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:30.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.638+0000 7fea43768740 1 -- 172.21.15.16:0/3656046209 >> 172.21.15.16:0/3656046209 conn(0x55751850a5f0 msgr2=0x55751850a9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:15:30.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.638+0000 7fea43768740 1 -- 172.21.15.16:0/3656046209 shutdown_connections 2024-09-12T02:15:30.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.638+0000 7fea43768740 1 -- 172.21.15.16:0/3656046209 wait complete. 2024-09-12T02:15:30.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.638+0000 7fea43768740 1 -- start start 2024-09-12T02:15:30.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.638+0000 7fea43768740 1 Processor -- start 2024-09-12T02:15:30.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.639+0000 7fea43768740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55751850df60 0x55751851a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:30.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.639+0000 7fea43768740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55751850e880 0x55751851aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:30.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.639+0000 7fea43768740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55751851d650 0x55751851afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:30.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.639+0000 7fea43768740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x557518520280 con 0x55751850df60 2024-09-12T02:15:30.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.639+0000 7fea43768740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5575183fbf10 con 0x55751851d650 2024-09-12T02:15:30.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.639+0000 7fea43768740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x557518520400 con 0x55751850e880 2024-09-12T02:15:30.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.639+0000 7fea43768740 10 client.0 _ll_register_callbacks cb 0x55751841fdb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:15:30.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.639+0000 7fea327fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55751850df60 0x55751851a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:30.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.639+0000 7fea327fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55751850df60 0x55751851a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56432/0 (socket says 172.21.15.16:56432) 2024-09-12T02:15:30.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.639+0000 7fea327fc640 1 -- 172.21.15.16:0/3197807820 learned_addr learned my addr 172.21.15.16:0/3197807820 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:15:30.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.639+0000 7fea3affd640 1 --2- 172.21.15.16:0/3197807820 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55751850e880 0x55751851aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:30.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.639+0000 7fea327fc640 1 --2- 172.21.15.16:0/3197807820 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55751851d650 0x55751851afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:30.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.639+0000 7fea327fc640 1 -- 172.21.15.16:0/3197807820 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55751850e880 msgr2=0x55751851aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:15:30.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.639+0000 7fea327fc640 1 --2- 172.21.15.16:0/3197807820 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55751850e880 0x55751851aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:30.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.639+0000 7fea327fc640 1 -- 172.21.15.16:0/3197807820 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55751851d650 msgr2=0x55751851afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:15:30.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.639+0000 7fea327fc640 1 --2- 172.21.15.16:0/3197807820 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55751851d650 0x55751851afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:30.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.639+0000 7fea327fc640 1 -- 172.21.15.16:0/3197807820 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557518182580 con 0x55751850df60 2024-09-12T02:15:30.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.639+0000 7fea3affd640 1 --2- 172.21.15.16:0/3197807820 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55751850e880 0x55751851aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:15:30.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.639+0000 7fea327fc640 1 --2- 172.21.15.16:0/3197807820 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55751850df60 0x55751851a530 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7fea240057a0 tx=0x7fea2401d680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a23afd2b14be396 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:15:30.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.639+0000 7fea227fc640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:15:30.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.640+0000 7fea227fc640 1 -- 172.21.15.16:0/3197807820 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea24019040 con 0x55751850df60 2024-09-12T02:15:30.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.640+0000 7fea227fc640 1 -- 172.21.15.16:0/3197807820 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fea24011b00 con 0x55751850df60 2024-09-12T02:15:30.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.640+0000 7fea43768740 1 -- 172.21.15.16:0/3197807820 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea24011d60 con 0x55751850df60 2024-09-12T02:15:30.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.640+0000 7fea227fc640 1 -- 172.21.15.16:0/3197807820 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fea24013050 con 0x55751850df60 2024-09-12T02:15:30.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.640+0000 7fea43768740 10 client.19926 fetch_fsmap learned FSMap version 25 2024-09-12T02:15:30.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.640+0000 7fea43768740 1 -- 172.21.15.16:0/3197807820 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55751824c960 con 0x55751850df60 2024-09-12T02:15:30.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.640+0000 7fea227fc640 1 -- 172.21.15.16:0/3197807820 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea24011d60 con 0x55751850df60 2024-09-12T02:15:30.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.640+0000 7fea227fc640 1 -- 172.21.15.16:0/3197807820 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fea2401da00 con 0x55751850df60 2024-09-12T02:15:30.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.640+0000 7fea43768740 10 client.19926 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:15:30.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.640+0000 7fea43768740 10 client.19926 resolve_mds: validated gid 24482 aka daemon mds.b 2024-09-12T02:15:30.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.640+0000 7fea43768740 20 client.19926 populate_metadata read hostname 'smithi016' 2024-09-12T02:15:30.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.640+0000 7fea43768740 1 --2- 172.21.15.16:0/3197807820 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55751852d200 0x55751852f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:30.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.640+0000 7fea43768740 4 client.19926 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:15:30.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.640+0000 7fea43768740 1 -- 172.21.15.16:0/3197807820 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557518480b90 con 0x55751852d200 2024-09-12T02:15:30.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.640+0000 7fea3affd640 1 --2- 172.21.15.16:0/3197807820 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55751852d200 0x55751852f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:30.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.641+0000 7fea3affd640 1 --2- 172.21.15.16:0/3197807820 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55751852d200 0x55751852f5f0 crc :-1 s=READY pgs=403 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91b2167b3efe1331 server_cookie=18e0e1c7448ab557 in_seq=0 out_seq=0 2024-09-12T02:15:30.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.641+0000 7fea227fc640 10 client.19926 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:15:30.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.642+0000 7fea227fc640 1 -- 172.21.15.16:0/3197807820 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557518480b90 con 0x55751852d200 2024-09-12T02:15:30.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.642+0000 7fea227fc640 10 client.19926 handle_command_reply: tid=0 2024-09-12T02:15:30.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.642+0000 7fea43768740 1 client.19926 shutdown 2024-09-12T02:15:30.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.642+0000 7fea43768740 1 -- 172.21.15.16:0/3197807820 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55751850df60 msgr2=0x55751851a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:30.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.642+0000 7fea43768740 1 --2- 172.21.15.16:0/3197807820 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55751850df60 0x55751851a530 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7fea240057a0 tx=0x7fea2401d680 comp rx=0 tx=0).stop 2024-09-12T02:15:30.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.642+0000 7fea43768740 1 -- 172.21.15.16:0/3197807820 shutdown_connections 2024-09-12T02:15:30.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.642+0000 7fea43768740 1 --2- 172.21.15.16:0/3197807820 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55751852d200 0x55751852f5f0 crc :-1 s=READY pgs=403 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:30.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.642+0000 7fea43768740 1 --2- 172.21.15.16:0/3197807820 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55751851d650 0x55751851afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:30.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.642+0000 7fea43768740 1 --2- 172.21.15.16:0/3197807820 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55751850e880 0x55751851aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:30.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.642+0000 7fea227fc640 0 client.19926 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:15:30.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.642+0000 7fea43768740 1 --2- 172.21.15.16:0/3197807820 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55751850df60 0x55751851a530 unknown :-1 s=CLOSED pgs=1836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:30.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.642+0000 7fea43768740 1 -- 172.21.15.16:0/3197807820 >> 172.21.15.16:0/3197807820 conn(0x55751850a5f0 msgr2=0x557518510320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:15:30.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.642+0000 7fea43768740 1 -- 172.21.15.16:0/3197807820 shutdown_connections 2024-09-12T02:15:30.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.642+0000 7fea43768740 1 -- 172.21.15.16:0/3197807820 wait complete. 2024-09-12T02:15:30.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.642+0000 7fea43768740 20 client.19926 trim_cache size 0 max 16384 2024-09-12T02:15:30.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.660+0000 7fea43768740 1 Processor -- start 2024-09-12T02:15:30.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.660+0000 7fea43768740 1 -- start start 2024-09-12T02:15:30.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.660+0000 7fea43768740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55751850df60 0x557518534d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:30.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.660+0000 7fea43768740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55751850e880 0x557518535270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:30.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.660+0000 7fea43768740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55751851d650 0x557518517790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:30.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.660+0000 7fea43768740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fea3c1ad970 con 0x55751850df60 2024-09-12T02:15:30.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.660+0000 7fea43768740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fea3c1ab240 con 0x55751851d650 2024-09-12T02:15:30.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.660+0000 7fea43768740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fea3c1a9090 con 0x55751850e880 2024-09-12T02:15:30.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.660+0000 7fea327fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55751850df60 0x557518534d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:30.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.660+0000 7fea327fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55751850df60 0x557518534d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56436/0 (socket says 172.21.15.16:56436) 2024-09-12T02:15:30.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.660+0000 7fea327fc640 1 -- 172.21.15.16:0/3897289804 learned_addr learned my addr 172.21.15.16:0/3897289804 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:15:30.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.660+0000 7fea3affd640 1 --2- 172.21.15.16:0/3897289804 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55751850e880 0x557518535270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:30.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.660+0000 7fea327fc640 1 -- 172.21.15.16:0/3897289804 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55751850e880 msgr2=0x557518535270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:15:30.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.660+0000 7fea327fc640 1 --2- 172.21.15.16:0/3897289804 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55751850e880 0x557518535270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:30.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.660+0000 7fea327fc640 1 -- 172.21.15.16:0/3897289804 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55751851d650 msgr2=0x557518517790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:15:30.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.660+0000 7fea327fc640 1 --2- 172.21.15.16:0/3897289804 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55751851d650 0x557518517790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:30.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.660+0000 7fea327fc640 1 -- 172.21.15.16:0/3897289804 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55751850e6b0 con 0x55751850df60 2024-09-12T02:15:30.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.661+0000 7fea327fc640 1 --2- 172.21.15.16:0/3897289804 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55751850df60 0x557518534d30 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7fea2400c010 tx=0x7fea24015f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4e0cf6853223b07 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:15:30.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.661+0000 7fea30ff9640 1 -- 172.21.15.16:0/3897289804 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea24019050 con 0x55751850df60 2024-09-12T02:15:30.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.661+0000 7fea30ff9640 1 -- 172.21.15.16:0/3897289804 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea2400b450 con 0x55751850df60 2024-09-12T02:15:30.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.661+0000 7fea30ff9640 1 -- 172.21.15.16:0/3897289804 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea24005020 con 0x55751850df60 2024-09-12T02:15:30.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.661+0000 7fea43768740 1 -- 172.21.15.16:0/3897289804 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55751850df60 msgr2=0x557518534d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:30.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.661+0000 7fea43768740 1 --2- 172.21.15.16:0/3897289804 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55751850df60 0x557518534d30 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7fea2400c010 tx=0x7fea24015f60 comp rx=0 tx=0).stop 2024-09-12T02:15:30.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.662+0000 7fea43768740 1 -- 172.21.15.16:0/3897289804 shutdown_connections 2024-09-12T02:15:30.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.662+0000 7fea43768740 1 --2- 172.21.15.16:0/3897289804 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55751851d650 0x557518517790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:30.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.662+0000 7fea43768740 1 --2- 172.21.15.16:0/3897289804 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55751850e880 0x557518535270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:30.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.662+0000 7fea43768740 1 --2- 172.21.15.16:0/3897289804 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55751850df60 0x557518534d30 unknown :-1 s=CLOSED pgs=1837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:30.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.662+0000 7fea43768740 1 -- 172.21.15.16:0/3897289804 >> 172.21.15.16:0/3897289804 conn(0x55751850a5f0 msgr2=0x55751851bd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:15:30.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.662+0000 7fea43768740 1 -- 172.21.15.16:0/3897289804 shutdown_connections 2024-09-12T02:15:30.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.662+0000 7fea43768740 1 -- 172.21.15.16:0/3897289804 wait complete. 2024-09-12T02:15:30.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.662+0000 7fea43768740 1 -- start start 2024-09-12T02:15:30.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.662+0000 7fea43768740 1 Processor -- start 2024-09-12T02:15:30.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.662+0000 7fea43768740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55751850df60 0x5575185216a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:30.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.662+0000 7fea43768740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55751850e880 0x557518521be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:30.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.662+0000 7fea43768740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55751851d650 0x557518522120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:30.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.662+0000 7fea43768740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fea3c1a9700 con 0x55751850e880 2024-09-12T02:15:30.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.662+0000 7fea43768740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fea3c1aab70 con 0x55751850df60 2024-09-12T02:15:30.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.662+0000 7fea43768740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fea3c1a9dd0 con 0x55751851d650 2024-09-12T02:15:30.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.662+0000 7fea43768740 10 client.0 _ll_register_callbacks cb 0x7fea2400b3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:15:30.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.662+0000 7fea3affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55751850e880 0x557518521be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:30.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.662+0000 7fea3affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55751850e880 0x557518521be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56438/0 (socket says 172.21.15.16:56438) 2024-09-12T02:15:30.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.662+0000 7fea3affd640 1 -- 172.21.15.16:0/2182738044 learned_addr learned my addr 172.21.15.16:0/2182738044 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:15:30.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.663+0000 7fea327fc640 1 --2- 172.21.15.16:0/2182738044 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55751850df60 0x5575185216a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:30.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.663+0000 7fea3affd640 1 -- 172.21.15.16:0/2182738044 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55751851d650 msgr2=0x557518522120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:15:30.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.663+0000 7fea327fc640 1 --2- 172.21.15.16:0/2182738044 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55751851d650 0x557518522120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:30.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.663+0000 7fea3affd640 1 --2- 172.21.15.16:0/2182738044 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55751851d650 0x557518522120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:30.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.663+0000 7fea3affd640 1 -- 172.21.15.16:0/2182738044 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55751850df60 msgr2=0x5575185216a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:15:30.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.663+0000 7fea3affd640 1 --2- 172.21.15.16:0/2182738044 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55751850df60 0x5575185216a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:30.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.663+0000 7fea3affd640 1 -- 172.21.15.16:0/2182738044 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557518596f00 con 0x55751850e880 2024-09-12T02:15:30.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.663+0000 7fea327fc640 1 --2- 172.21.15.16:0/2182738044 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55751851d650 0x557518522120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:15:30.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.663+0000 7fea3affd640 1 --2- 172.21.15.16:0/2182738044 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55751850e880 0x557518521be0 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7fea2c00abc0 tx=0x7fea2c013b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74e2a84dc5a650b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:15:30.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.663+0000 7fea227fc640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:15:30.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.663+0000 7fea227fc640 1 -- 172.21.15.16:0/2182738044 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea2c013e50 con 0x55751850e880 2024-09-12T02:15:30.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.663+0000 7fea227fc640 1 -- 172.21.15.16:0/2182738044 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fea2c00f950 con 0x55751850e880 2024-09-12T02:15:30.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.663+0000 7fea43768740 1 -- 172.21.15.16:0/2182738044 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea2400b450 con 0x55751850e880 2024-09-12T02:15:30.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.663+0000 7fea227fc640 1 -- 172.21.15.16:0/2182738044 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fea2c02d050 con 0x55751850e880 2024-09-12T02:15:30.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.663+0000 7fea43768740 10 client.19932 fetch_fsmap learned FSMap version 25 2024-09-12T02:15:30.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.663+0000 7fea43768740 1 -- 172.21.15.16:0/2182738044 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5575185982b0 con 0x55751850e880 2024-09-12T02:15:30.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.664+0000 7fea227fc640 1 -- 172.21.15.16:0/2182738044 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea2400b450 con 0x55751850e880 2024-09-12T02:15:30.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.664+0000 7fea227fc640 1 -- 172.21.15.16:0/2182738044 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fea2c004850 con 0x55751850e880 2024-09-12T02:15:30.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.664+0000 7fea43768740 10 client.19932 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:15:30.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.664+0000 7fea43768740 10 client.19932 resolve_mds: validated gid 24482 aka daemon mds.b 2024-09-12T02:15:30.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.664+0000 7fea43768740 20 client.19932 populate_metadata read hostname 'smithi016' 2024-09-12T02:15:30.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.664+0000 7fea43768740 1 --2- 172.21.15.16:0/2182738044 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x557518598a10 0x55751859ae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:30.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.664+0000 7fea43768740 4 client.19932 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "901e6dc3", "release": true, "await": true}] 2024-09-12T02:15:30.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.664+0000 7fea43768740 1 -- 172.21.15.16:0/2182738044 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "quiesce db", "set_id": "901e6dc3", "release": true, "await": true}) -- 0x55751847b0a0 con 0x557518598a10 2024-09-12T02:15:30.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.664+0000 7fea327fc640 1 --2- 172.21.15.16:0/2182738044 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x557518598a10 0x55751859ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:30.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.665+0000 7fea227fc640 10 client.19932 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:15:30.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.665+0000 7fea327fc640 1 --2- 172.21.15.16:0/2182738044 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x557518598a10 0x55751859ae30 crc :-1 s=READY pgs=404 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e97bbe2fd03e6d6 server_cookie=a7161ef41e3e871c in_seq=0 out_seq=0 2024-09-12T02:15:30.670 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:15:30 smithi033 ceph-mon[35332]: pgmap v742: 97 pgs: 97 active+clean; 3.0 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.9 KiB/s wr, 7 op/s 2024-09-12T02:15:30.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.667+0000 7fea227fc640 1 -- 172.21.15.16:0/2182738044 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55751847b0a0 con 0x557518598a10 2024-09-12T02:15:30.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.667+0000 7fea227fc640 10 client.19932 handle_command_reply: tid=0 2024-09-12T02:15:30.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.667+0000 7fea43768740 1 client.19932 shutdown 2024-09-12T02:15:30.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.667+0000 7fea43768740 1 -- 172.21.15.16:0/2182738044 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55751850e880 msgr2=0x557518521be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:30.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.667+0000 7fea43768740 1 --2- 172.21.15.16:0/2182738044 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55751850e880 0x557518521be0 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7fea2c00abc0 tx=0x7fea2c013b20 comp rx=0 tx=0).stop 2024-09-12T02:15:30.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.667+0000 7fea43768740 1 -- 172.21.15.16:0/2182738044 shutdown_connections 2024-09-12T02:15:30.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.667+0000 7fea43768740 1 --2- 172.21.15.16:0/2182738044 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x557518598a10 0x55751859ae30 crc :-1 s=READY pgs=404 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:30.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.667+0000 7fea43768740 1 --2- 172.21.15.16:0/2182738044 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55751851d650 0x557518522120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:30.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.667+0000 7fea227fc640 0 client.19932 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:15:30.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.667+0000 7fea43768740 1 --2- 172.21.15.16:0/2182738044 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55751850e880 0x557518521be0 unknown :-1 s=CLOSED pgs=1838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:30.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.667+0000 7fea43768740 1 --2- 172.21.15.16:0/2182738044 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55751850df60 0x5575185216a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:30.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.667+0000 7fea43768740 1 -- 172.21.15.16:0/2182738044 >> 172.21.15.16:0/2182738044 conn(0x55751850a5f0 msgr2=0x55751850fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:15:30.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.667+0000 7fea43768740 1 -- 172.21.15.16:0/2182738044 shutdown_connections 2024-09-12T02:15:30.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.667+0000 7fea43768740 1 -- 172.21.15.16:0/2182738044 wait complete. 2024-09-12T02:15:30.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.667+0000 7fea43768740 20 client.19932 trim_cache size 0 max 16384 2024-09-12T02:15:30.672 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T02:15:30.672 INFO:teuthology.orchestra.run.smithi016.stdout: "epoch": 25, 2024-09-12T02:15:30.672 INFO:teuthology.orchestra.run.smithi016.stdout: "leader": 24482, 2024-09-12T02:15:30.673 INFO:teuthology.orchestra.run.smithi016.stdout: "set_version": 52, 2024-09-12T02:15:30.673 INFO:teuthology.orchestra.run.smithi016.stdout: "sets": { 2024-09-12T02:15:30.673 INFO:teuthology.orchestra.run.smithi016.stdout: "901e6dc3": { 2024-09-12T02:15:30.673 INFO:teuthology.orchestra.run.smithi016.stdout: "version": 52, 2024-09-12T02:15:30.673 INFO:teuthology.orchestra.run.smithi016.stdout: "age_ref": 0.0, 2024-09-12T02:15:30.673 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-12T02:15:30.673 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RELEASED", 2024-09-12T02:15:30.673 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-12T02:15:30.673 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:15:30.673 INFO:teuthology.orchestra.run.smithi016.stdout: "timeout": 90.0, 2024-09-12T02:15:30.673 INFO:teuthology.orchestra.run.smithi016.stdout: "expiration": 150.0, 2024-09-12T02:15:30.673 INFO:teuthology.orchestra.run.smithi016.stdout: "members": { 2024-09-12T02:15:30.673 INFO:teuthology.orchestra.run.smithi016.stdout: "file:/": { 2024-09-12T02:15:30.674 INFO:teuthology.orchestra.run.smithi016.stdout: "excluded": false, 2024-09-12T02:15:30.674 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-12T02:15:30.674 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RELEASED", 2024-09-12T02:15:30.674 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-12T02:15:30.674 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:15:30.674 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:15:30.674 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:15:30.674 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:15:30.674 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:15:30.674 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T02:15:30.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.668+0000 7fea4138f640 1 -- 172.21.15.16:0/3515656704 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fea280a2b80 msgr2=0x7fea280a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:30.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.668+0000 7fea4138f640 1 --2- 172.21.15.16:0/3515656704 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fea280a2b80 0x7fea280a5040 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fea340b8df0 tx=0x7fea340c0040 comp rx=0 tx=0).stop 2024-09-12T02:15:30.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.668+0000 7fea4138f640 1 -- 172.21.15.16:0/3515656704 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea3c106a80 msgr2=0x7fea3c19cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:30.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.668+0000 7fea4138f640 1 --2- 172.21.15.16:0/3515656704 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea3c106a80 0x7fea3c19cc30 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7fea34033040 tx=0x7fea34003970 comp rx=0 tx=0).stop 2024-09-12T02:15:30.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.668+0000 7fea4138f640 1 -- 172.21.15.16:0/3515656704 shutdown_connections 2024-09-12T02:15:30.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.668+0000 7fea4138f640 1 --2- 172.21.15.16:0/3515656704 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fea280a2b80 0x7fea280a5040 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:30.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.668+0000 7fea4138f640 1 --2- 172.21.15.16:0/3515656704 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea3c106a80 0x7fea3c19cc30 unknown :-1 s=CLOSED pgs=1834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:30.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.668+0000 7fea4138f640 1 --2- 172.21.15.16:0/3515656704 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fea3c101420 0x7fea3c1988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:30.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.668+0000 7fea4138f640 1 --2- 172.21.15.16:0/3515656704 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fea3c100a80 0x7fea3c198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:30.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.668+0000 7fea4138f640 1 -- 172.21.15.16:0/3515656704 >> 172.21.15.16:0/3515656704 conn(0x7fea3c0fc7a0 msgr2=0x7fea3c0fe480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:15:30.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.668+0000 7fea4138f640 1 -- 172.21.15.16:0/3515656704 shutdown_connections 2024-09-12T02:15:30.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:30.668+0000 7fea4138f640 1 -- 172.21.15.16:0/3515656704 wait complete. 2024-09-12T02:15:31.365 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '901e6dc3', total seconds elapsed: 33.2 2024-09-12T02:15:31.366 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-09-12T02:15:31.366 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:15:31.628 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:15:32.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:32.114+0000 7fa6a244a640 1 -- 172.21.15.16:0/360453295 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa69c103660 msgr2=0x7fa69c10bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:32.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:32.114+0000 7fa6a244a640 1 --2- 172.21.15.16:0/360453295 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa69c103660 0x7fa69c10bb50 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7fa690009900 tx=0x7fa69002a480 comp rx=0 tx=0).stop 2024-09-12T02:15:32.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:32.114+0000 7fa6a244a640 1 -- 172.21.15.16:0/360453295 shutdown_connections 2024-09-12T02:15:32.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:32.114+0000 7fa6a244a640 1 --2- 172.21.15.16:0/360453295 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa69c10c170 0x7fa69c10e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:32.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:32.114+0000 7fa6a244a640 1 --2- 172.21.15.16:0/360453295 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa69c103660 0x7fa69c10bb50 unknown :-1 s=CLOSED pgs=1839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:32.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:32.114+0000 7fa6a244a640 1 --2- 172.21.15.16:0/360453295 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa69c102d40 0x7fa69c103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:32.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:32.114+0000 7fa6a244a640 1 -- 172.21.15.16:0/360453295 >> 172.21.15.16:0/360453295 conn(0x7fa69c0fc7a0 msgr2=0x7fa69c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:15:32.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:32.114+0000 7fa6a244a640 1 -- 172.21.15.16:0/360453295 shutdown_connections 2024-09-12T02:15:32.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:32.114+0000 7fa6a244a640 1 -- 172.21.15.16:0/360453295 wait complete. 2024-09-12T02:15:32.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:32.115+0000 7fa6a244a640 1 Processor -- start 2024-09-12T02:15:32.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:32.115+0000 7fa6a244a640 1 -- start start 2024-09-12T02:15:32.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:32.115+0000 7fa6a244a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa69c102d40 0x7fa69c1984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:32.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:32.115+0000 7fa6a244a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa69c103660 0x7fa69c1989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:32.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:32.115+0000 7fa6a244a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa69c10c170 0x7fa69c19cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:32.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:32.115+0000 7fa6a244a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa69c111fd0 con 0x7fa69c102d40 2024-09-12T02:15:32.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:32.115+0000 7fa6a244a640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa69c111e50 con 0x7fa69c10c170 2024-09-12T02:15:32.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:32.115+0000 7fa6a244a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa69c112150 con 0x7fa69c103660 2024-09-12T02:15:32.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:32.115+0000 7fa69bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa69c102d40 0x7fa69c1984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:32.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:32.115+0000 7fa69bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa69c102d40 0x7fa69c1984a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59076/0 (socket says 172.21.15.16:59076) 2024-09-12T02:15:32.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:32.115+0000 7fa69bfff640 1 -- 172.21.15.16:0/749753194 learned_addr learned my addr 172.21.15.16:0/749753194 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:15:32.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:32.115+0000 7fa6a09e7640 1 --2- 172.21.15.16:0/749753194 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa69c10c170 0x7fa69c19cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:32.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:32.115+0000 7fa69b7fe640 1 --2- 172.21.15.16:0/749753194 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa69c103660 0x7fa69c1989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:32.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:32.116+0000 7fa69bfff640 1 -- 172.21.15.16:0/749753194 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa69c103660 msgr2=0x7fa69c1989e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:32.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:32.116+0000 7fa69bfff640 1 --2- 172.21.15.16:0/749753194 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa69c103660 0x7fa69c1989e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:32.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:32.116+0000 7fa69bfff640 1 -- 172.21.15.16:0/749753194 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa69c10c170 msgr2=0x7fa69c19cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:32.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:32.116+0000 7fa69bfff640 1 --2- 172.21.15.16:0/749753194 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa69c10c170 0x7fa69c19cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:32.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:32.116+0000 7fa69bfff640 1 -- 172.21.15.16:0/749753194 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa69c19d450 con 0x7fa69c102d40 2024-09-12T02:15:32.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:32.116+0000 7fa69b7fe640 1 --2- 172.21.15.16:0/749753194 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa69c103660 0x7fa69c1989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:15:32.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:32.116+0000 7fa69bfff640 1 --2- 172.21.15.16:0/749753194 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa69c102d40 0x7fa69c1984a0 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7fa68c00d770 tx=0x7fa68c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:15:32.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:32.116+0000 7fa6997fa640 1 -- 172.21.15.16:0/749753194 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa68c017070 con 0x7fa69c102d40 2024-09-12T02:15:32.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:32.116+0000 7fa6997fa640 1 -- 172.21.15.16:0/749753194 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa68c005500 con 0x7fa69c102d40 2024-09-12T02:15:32.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:32.116+0000 7fa6a244a640 1 -- 172.21.15.16:0/749753194 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa69c19d740 con 0x7fa69c102d40 2024-09-12T02:15:32.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:32.116+0000 7fa6997fa640 1 -- 172.21.15.16:0/749753194 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa68c004e00 con 0x7fa69c102d40 2024-09-12T02:15:32.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:32.116+0000 7fa6a244a640 1 -- 172.21.15.16:0/749753194 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa69c1a5020 con 0x7fa69c102d40 2024-09-12T02:15:32.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:32.117+0000 7fa6a244a640 1 -- 172.21.15.16:0/749753194 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa69c103120 con 0x7fa69c102d40 2024-09-12T02:15:32.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:32.118+0000 7fa6997fa640 1 -- 172.21.15.16:0/749753194 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa68c004fa0 con 0x7fa69c102d40 2024-09-12T02:15:32.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:32.119+0000 7fa6997fa640 1 --2- 172.21.15.16:0/749753194 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fa6840a2b80 0x7fa6840a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:32.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:32.119+0000 7fa69b7fe640 1 --2- 172.21.15.16:0/749753194 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fa6840a2b80 0x7fa6840a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:32.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:32.119+0000 7fa6997fa640 1 -- 172.21.15.16:0/749753194 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fa68c0cea30 con 0x7fa69c102d40 2024-09-12T02:15:32.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:32.120+0000 7fa69b7fe640 1 --2- 172.21.15.16:0/749753194 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fa6840a2b80 0x7fa6840a5040 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fa69c0feda0 tx=0x7fa690005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:15:32.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:32.123+0000 7fa6997fa640 1 -- 172.21.15.16:0/749753194 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa68c09ba30 con 0x7fa69c102d40 2024-09-12T02:15:32.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:32.334+0000 7fa6a244a640 1 -- 172.21.15.16:0/749753194 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa69c1071e0 con 0x7fa69c102d40 2024-09-12T02:15:32.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:32.335+0000 7fa6997fa640 1 -- 172.21.15.16:0/749753194 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11911 (secure 0 0 0) 0x7fa68c09b330 con 0x7fa69c102d40 2024-09-12T02:15:32.335 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T02:15:32.335 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":25,"btime":"2024-09-12T02:01:37:192021+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3279123279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3279123279},{"type":"v1","addr":"172.21.15.16:6835","nonce":3279123279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4119452382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4119452382},{"type":"v1","addr":"172.21.15.16:6837","nonce":4119452382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/3263823165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3263823165},{"type":"v1","addr":"172.21.15.16:6839","nonce":3263823165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/1510290732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":1510290732},{"type":"v1","addr":"172.21.15.16:6841","nonce":1510290732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/279674603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":279674603},{"type":"v1","addr":"172.21.15.33:6833","nonce":279674603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3555728028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3555728028},{"type":"v1","addr":"172.21.15.33:6835","nonce":3555728028}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T01:59:56.117608+0000","modified":"2024-09-12T02:01:36.294760+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24482,"mds_1":24481,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6833/1563374106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1563374106},{"type":"v1","addr":"172.21.15.184:6833","nonce":1563374106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6837/1892833868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1892833868},{"type":"v1","addr":"172.21.15.33:6837","nonce":1892833868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/1457679686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1457679686},{"type":"v1","addr":"172.21.15.184:6835","nonce":1457679686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.184:6837/1793393087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":1793393087},{"type":"v1","addr":"172.21.15.184:6837","nonce":1793393087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.184:6839/4203347941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":4203347941},{"type":"v1","addr":"172.21.15.184:6839","nonce":4203347941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.33:6839/4136670204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":4136670204},{"type":"v1","addr":"172.21.15.33:6839","nonce":4136670204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24481,24482,24469]},"id":1}]} 2024-09-12T02:15:32.336 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 25 2024-09-12T02:15:32.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:32.337+0000 7fa6a244a640 1 -- 172.21.15.16:0/749753194 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fa6840a2b80 msgr2=0x7fa6840a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:32.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:32.337+0000 7fa6a244a640 1 --2- 172.21.15.16:0/749753194 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fa6840a2b80 0x7fa6840a5040 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fa69c0feda0 tx=0x7fa690005a20 comp rx=0 tx=0).stop 2024-09-12T02:15:32.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:32.337+0000 7fa6a244a640 1 -- 172.21.15.16:0/749753194 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa69c102d40 msgr2=0x7fa69c1984a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:32.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:32.337+0000 7fa6a244a640 1 --2- 172.21.15.16:0/749753194 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa69c102d40 0x7fa69c1984a0 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7fa68c00d770 tx=0x7fa68c00dc30 comp rx=0 tx=0).stop 2024-09-12T02:15:32.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:32.337+0000 7fa6a244a640 1 -- 172.21.15.16:0/749753194 shutdown_connections 2024-09-12T02:15:32.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:32.337+0000 7fa6a244a640 1 --2- 172.21.15.16:0/749753194 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fa6840a2b80 0x7fa6840a5040 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:32.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:32.337+0000 7fa6a244a640 1 --2- 172.21.15.16:0/749753194 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa69c10c170 0x7fa69c19cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:32.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:32.337+0000 7fa6a244a640 1 --2- 172.21.15.16:0/749753194 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa69c103660 0x7fa69c1989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:32.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:32.337+0000 7fa6a244a640 1 --2- 172.21.15.16:0/749753194 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa69c102d40 0x7fa69c1984a0 unknown :-1 s=CLOSED pgs=1840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:32.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:32.337+0000 7fa6a244a640 1 -- 172.21.15.16:0/749753194 >> 172.21.15.16:0/749753194 conn(0x7fa69c0fc7a0 msgr2=0x7fa69c0fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:15:32.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:32.338+0000 7fa6a244a640 1 -- 172.21.15.16:0/749753194 shutdown_connections 2024-09-12T02:15:32.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:32.338+0000 7fa6a244a640 1 -- 172.21.15.16:0/749753194 wait complete. 2024-09-12T02:15:32.527 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.527+0000 7fcf67baf640 1 -- 172.21.15.16:0/3047997788 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf60104f10 msgr2=0x7fcf60107300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:32.527 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.527+0000 7fcf67baf640 1 --2- 172.21.15.16:0/3047997788 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf60104f10 0x7fcf60107300 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7fcf54009900 tx=0x7fcf5402a410 comp rx=0 tx=0).stop 2024-09-12T02:15:32.527 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.527+0000 7fcf67baf640 1 -- 172.21.15.16:0/3047997788 shutdown_connections 2024-09-12T02:15:32.527 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.527+0000 7fcf67baf640 1 --2- 172.21.15.16:0/3047997788 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcf60107840 0x7fcf60109cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:32.527 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.527+0000 7fcf67baf640 1 --2- 172.21.15.16:0/3047997788 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf60104f10 0x7fcf60107300 unknown :-1 s=CLOSED pgs=1841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:32.527 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.527+0000 7fcf67baf640 1 --2- 172.21.15.16:0/3047997788 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf601025e0 0x7fcf601049d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:32.527 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.527+0000 7fcf67baf640 1 -- 172.21.15.16:0/3047997788 >> 172.21.15.16:0/3047997788 conn(0x7fcf600fc1e0 msgr2=0x7fcf600fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:15:32.528 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.527+0000 7fcf67baf640 1 -- 172.21.15.16:0/3047997788 shutdown_connections 2024-09-12T02:15:32.528 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.527+0000 7fcf67baf640 1 -- 172.21.15.16:0/3047997788 wait complete. 2024-09-12T02:15:32.530 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.527+0000 7fcf67baf640 1 Processor -- start 2024-09-12T02:15:32.530 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.527+0000 7fcf67baf640 1 -- start start 2024-09-12T02:15:32.530 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.528+0000 7fcf67baf640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf601025e0 0x7fcf60101f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:32.530 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.528+0000 7fcf67baf640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf60104f10 0x7fcf60100590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:32.530 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.528+0000 7fcf67baf640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcf60107840 0x7fcf60100ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:32.530 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.528+0000 7fcf67baf640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fcf6010d7f0 con 0x7fcf601025e0 2024-09-12T02:15:32.530 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.528+0000 7fcf67baf640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fcf6010d670 con 0x7fcf60104f10 2024-09-12T02:15:32.530 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.528+0000 7fcf67baf640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fcf6010d970 con 0x7fcf60107840 2024-09-12T02:15:32.530 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.528+0000 7fcf6594b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf601025e0 0x7fcf60101f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:32.530 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.528+0000 7fcf6594b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf601025e0 0x7fcf60101f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59104/0 (socket says 172.21.15.16:59104) 2024-09-12T02:15:32.530 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.528+0000 7fcf6594b640 1 -- 172.21.15.16:0/253095230 learned_addr learned my addr 172.21.15.16:0/253095230 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:15:32.530 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.528+0000 7fcf6514a640 1 --2- 172.21.15.16:0/253095230 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf60104f10 0x7fcf60100590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:32.531 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.528+0000 7fcf6614c640 1 --2- 172.21.15.16:0/253095230 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcf60107840 0x7fcf60100ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:32.531 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.528+0000 7fcf6594b640 1 -- 172.21.15.16:0/253095230 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcf60107840 msgr2=0x7fcf60100ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:32.531 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.528+0000 7fcf6594b640 1 --2- 172.21.15.16:0/253095230 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcf60107840 0x7fcf60100ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:32.531 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.528+0000 7fcf6594b640 1 -- 172.21.15.16:0/253095230 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf60104f10 msgr2=0x7fcf60100590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:32.531 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.528+0000 7fcf6594b640 1 --2- 172.21.15.16:0/253095230 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf60104f10 0x7fcf60100590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:32.531 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.528+0000 7fcf6594b640 1 -- 172.21.15.16:0/253095230 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf60101360 con 0x7fcf601025e0 2024-09-12T02:15:32.531 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.528+0000 7fcf6594b640 1 --2- 172.21.15.16:0/253095230 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf601025e0 0x7fcf60101f50 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7fcf5c00e9f0 tx=0x7fcf5c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:15:32.531 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.528+0000 7fcf52ffd640 1 -- 172.21.15.16:0/253095230 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf5c00cd80 con 0x7fcf601025e0 2024-09-12T02:15:32.531 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.528+0000 7fcf52ffd640 1 -- 172.21.15.16:0/253095230 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf5c010910 con 0x7fcf601025e0 2024-09-12T02:15:32.531 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.528+0000 7fcf67baf640 1 -- 172.21.15.16:0/253095230 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf601a1f90 con 0x7fcf601025e0 2024-09-12T02:15:32.531 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.528+0000 7fcf52ffd640 1 -- 172.21.15.16:0/253095230 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf5c018990 con 0x7fcf601025e0 2024-09-12T02:15:32.531 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.529+0000 7fcf67baf640 1 -- 172.21.15.16:0/253095230 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf601a2390 con 0x7fcf601025e0 2024-09-12T02:15:32.542 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.531+0000 7fcf52ffd640 1 -- 172.21.15.16:0/253095230 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fcf5c010ac0 con 0x7fcf601025e0 2024-09-12T02:15:32.543 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.531+0000 7fcf6aa24740 1 Processor -- start 2024-09-12T02:15:32.543 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.531+0000 7fcf6aa24740 1 -- start start 2024-09-12T02:15:32.543 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.531+0000 7fcf6aa24740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e91109e9a0 0x55e91109eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:32.543 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.531+0000 7fcf6aa24740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e91109f2e0 0x55e9110a3da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:32.543 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.532+0000 7fcf6aa24740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9110a4370 0x55e9110a6830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:32.544 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.532+0000 7fcf6aa24740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e9110022f0 con 0x55e91109f2e0 2024-09-12T02:15:32.544 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.532+0000 7fcf6aa24740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55e910faa270 con 0x55e9110a4370 2024-09-12T02:15:32.544 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.532+0000 7fcf6aa24740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55e910ffe2b0 con 0x55e91109e9a0 2024-09-12T02:15:32.544 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.532+0000 7fcf6594b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e91109f2e0 0x55e9110a3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:32.544 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.532+0000 7fcf6594b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e91109f2e0 0x55e9110a3da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59120/0 (socket says 172.21.15.16:59120) 2024-09-12T02:15:32.544 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.532+0000 7fcf6594b640 1 -- 172.21.15.16:0/3549239711 learned_addr learned my addr 172.21.15.16:0/3549239711 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:15:32.545 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.532+0000 7fcf6614c640 1 --2- 172.21.15.16:0/3549239711 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e91109e9a0 0x55e91109eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:32.545 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.532+0000 7fcf6514a640 1 --2- 172.21.15.16:0/3549239711 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9110a4370 0x55e9110a6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:32.545 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.532+0000 7fcf52ffd640 1 --2- 172.21.15.16:0/253095230 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fcf4c0a2c30 0x7fcf4c0a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:32.545 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.532+0000 7fcf6594b640 1 -- 172.21.15.16:0/3549239711 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e91109e9a0 msgr2=0x55e91109eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:15:32.545 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.532+0000 7fcf6594b640 1 --2- 172.21.15.16:0/3549239711 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e91109e9a0 0x55e91109eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:32.545 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.532+0000 7fcf6594b640 1 -- 172.21.15.16:0/3549239711 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9110a4370 msgr2=0x55e9110a6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:15:32.546 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.532+0000 7fcf6594b640 1 --2- 172.21.15.16:0/3549239711 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9110a4370 0x55e9110a6830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:32.546 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.532+0000 7fcf6594b640 1 -- 172.21.15.16:0/3549239711 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e910fa9340 con 0x55e91109f2e0 2024-09-12T02:15:32.546 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.532+0000 7fcf6614c640 1 --2- 172.21.15.16:0/253095230 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fcf4c0a2c30 0x7fcf4c0a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:32.546 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.532+0000 7fcf52ffd640 1 -- 172.21.15.16:0/253095230 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fcf5c0cf300 con 0x7fcf601025e0 2024-09-12T02:15:32.546 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.532+0000 7fcf6594b640 1 --2- 172.21.15.16:0/3549239711 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e91109f2e0 0x55e9110a3da0 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7fcf5c0b95a0 tx=0x7fcf5c0c66f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4a26cb834984b40 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:15:32.546 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.532+0000 7fcf4bfff640 1 -- 172.21.15.16:0/3549239711 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf5c026030 con 0x55e91109f2e0 2024-09-12T02:15:32.546 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.533+0000 7fcf4bfff640 1 -- 172.21.15.16:0/3549239711 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf5c0d2e30 con 0x55e91109f2e0 2024-09-12T02:15:32.547 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.533+0000 7fcf4bfff640 1 -- 172.21.15.16:0/3549239711 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf5c0ba060 con 0x55e91109f2e0 2024-09-12T02:15:32.547 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.533+0000 7fcf6614c640 1 --2- 172.21.15.16:0/253095230 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fcf4c0a2c30 0x7fcf4c0a50f0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fcf6006b3e0 tx=0x7fcf58004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:15:32.547 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.533+0000 7fcf6aa24740 1 -- 172.21.15.16:0/3549239711 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e91109f2e0 msgr2=0x55e9110a3da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:32.547 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.533+0000 7fcf6aa24740 1 --2- 172.21.15.16:0/3549239711 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e91109f2e0 0x55e9110a3da0 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7fcf5c0b95a0 tx=0x7fcf5c0c66f0 comp rx=0 tx=0).stop 2024-09-12T02:15:32.547 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.533+0000 7fcf6aa24740 1 -- 172.21.15.16:0/3549239711 shutdown_connections 2024-09-12T02:15:32.547 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.533+0000 7fcf6aa24740 1 --2- 172.21.15.16:0/3549239711 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9110a4370 0x55e9110a6830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:32.547 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.533+0000 7fcf6aa24740 1 --2- 172.21.15.16:0/3549239711 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e91109f2e0 0x55e9110a3da0 unknown :-1 s=CLOSED pgs=1843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:32.548 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.533+0000 7fcf6aa24740 1 --2- 172.21.15.16:0/3549239711 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e91109e9a0 0x55e91109eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:32.548 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.533+0000 7fcf6aa24740 1 -- 172.21.15.16:0/3549239711 >> 172.21.15.16:0/3549239711 conn(0x55e91109a010 msgr2=0x55e91109a410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:15:32.548 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.533+0000 7fcf6aa24740 1 -- 172.21.15.16:0/3549239711 shutdown_connections 2024-09-12T02:15:32.548 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.533+0000 7fcf6aa24740 1 -- 172.21.15.16:0/3549239711 wait complete. 2024-09-12T02:15:32.548 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.534+0000 7fcf6aa24740 1 -- start start 2024-09-12T02:15:32.548 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.534+0000 7fcf6aa24740 1 Processor -- start 2024-09-12T02:15:32.549 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.534+0000 7fcf6aa24740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e91109e9a0 0x55e9110a8b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:32.549 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.534+0000 7fcf6aa24740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e91109f2e0 0x55e9110a9040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:32.549 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.534+0000 7fcf6aa24740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9110a4370 0x55e9110b0080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:32.549 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.534+0000 7fcf6aa24740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e9110a72e0 con 0x55e91109e9a0 2024-09-12T02:15:32.549 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.534+0000 7fcf6aa24740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55e910f8b290 con 0x55e9110a4370 2024-09-12T02:15:32.549 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.534+0000 7fcf6aa24740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55e9110a7460 con 0x55e91109f2e0 2024-09-12T02:15:32.549 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.534+0000 7fcf6aa24740 10 client.0 _ll_register_callbacks cb 0x55e910fbc140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:15:32.549 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.534+0000 7fcf6594b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e91109e9a0 0x55e9110a8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:32.550 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.534+0000 7fcf6594b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e91109e9a0 0x55e9110a8b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59128/0 (socket says 172.21.15.16:59128) 2024-09-12T02:15:32.550 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.534+0000 7fcf6594b640 1 -- 172.21.15.16:0/2910141697 learned_addr learned my addr 172.21.15.16:0/2910141697 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:15:32.550 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.534+0000 7fcf6514a640 1 --2- 172.21.15.16:0/2910141697 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e91109f2e0 0x55e9110a9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:32.550 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.535+0000 7fcf6614c640 1 --2- 172.21.15.16:0/2910141697 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9110a4370 0x55e9110b0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:32.550 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.535+0000 7fcf6594b640 1 -- 172.21.15.16:0/2910141697 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e91109f2e0 msgr2=0x55e9110a9040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:15:32.550 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.535+0000 7fcf6594b640 1 --2- 172.21.15.16:0/2910141697 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e91109f2e0 0x55e9110a9040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:32.550 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.535+0000 7fcf6594b640 1 -- 172.21.15.16:0/2910141697 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9110a4370 msgr2=0x55e9110b0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:15:32.550 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.535+0000 7fcf6594b640 1 --2- 172.21.15.16:0/2910141697 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9110a4370 0x55e9110b0080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:32.550 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.535+0000 7fcf6594b640 1 -- 172.21.15.16:0/2910141697 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e910d15210 con 0x55e91109e9a0 2024-09-12T02:15:32.550 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.535+0000 7fcf6614c640 1 --2- 172.21.15.16:0/2910141697 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9110a4370 0x55e9110b0080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:15:32.550 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.535+0000 7fcf6594b640 1 --2- 172.21.15.16:0/2910141697 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e91109e9a0 0x55e9110a8b00 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7fcf5c0a42a0 tx=0x7fcf5c09ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ff163a8e1c804ec server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:15:32.550 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.535+0000 7fcf497fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:15:32.550 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.535+0000 7fcf497fa640 1 -- 172.21.15.16:0/2910141697 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf5c06f290 con 0x55e91109e9a0 2024-09-12T02:15:32.550 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.535+0000 7fcf497fa640 1 -- 172.21.15.16:0/2910141697 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fcf5c099f10 con 0x55e91109e9a0 2024-09-12T02:15:32.551 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.535+0000 7fcf6aa24740 1 -- 172.21.15.16:0/2910141697 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf5c0d2e30 con 0x55e91109e9a0 2024-09-12T02:15:32.551 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.535+0000 7fcf497fa640 1 -- 172.21.15.16:0/2910141697 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fcf5c09bc10 con 0x55e91109e9a0 2024-09-12T02:15:32.551 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.535+0000 7fcf6aa24740 10 client.19950 fetch_fsmap learned FSMap version 25 2024-09-12T02:15:32.551 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.535+0000 7fcf6aa24740 1 -- 172.21.15.16:0/2910141697 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e910de1620 con 0x55e91109e9a0 2024-09-12T02:15:32.551 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.535+0000 7fcf497fa640 1 -- 172.21.15.16:0/2910141697 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf5c0d2e30 con 0x55e91109e9a0 2024-09-12T02:15:32.551 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.536+0000 7fcf497fa640 1 -- 172.21.15.16:0/2910141697 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fcf5c09be70 con 0x55e91109e9a0 2024-09-12T02:15:32.551 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.536+0000 7fcf6aa24740 10 client.19950 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:15:32.551 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.536+0000 7fcf6aa24740 10 client.19950 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:15:32.551 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.536+0000 7fcf6aa24740 20 client.19950 populate_metadata read hostname 'smithi016' 2024-09-12T02:15:32.551 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.536+0000 7fcf6aa24740 1 --2- 172.21.15.16:0/2910141697 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55e9110b43d0 0x55e9110b67c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:32.551 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.536+0000 7fcf6aa24740 4 client.19950 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:15:32.551 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.536+0000 7fcf6aa24740 1 -- 172.21.15.16:0/2910141697 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e91100fa20 con 0x55e9110b43d0 2024-09-12T02:15:32.551 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.536+0000 7fcf6514a640 1 --2- 172.21.15.16:0/2910141697 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55e9110b43d0 0x55e9110b67c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:32.552 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.536+0000 7fcf497fa640 10 client.19950 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:15:32.552 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.536+0000 7fcf6514a640 1 --2- 172.21.15.16:0/2910141697 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55e9110b43d0 0x55e9110b67c0 crc :-1 s=READY pgs=405 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0103bacd33ddadc server_cookie=20173ab329c134f5 in_seq=0 out_seq=0 2024-09-12T02:15:32.552 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.537+0000 7fcf497fa640 1 -- 172.21.15.16:0/2910141697 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e91100fa20 con 0x55e9110b43d0 2024-09-12T02:15:32.552 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.537+0000 7fcf497fa640 10 client.19950 handle_command_reply: tid=0 2024-09-12T02:15:32.552 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.537+0000 7fcf6aa24740 1 client.19950 shutdown 2024-09-12T02:15:32.552 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.537+0000 7fcf6aa24740 1 -- 172.21.15.16:0/2910141697 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e91109e9a0 msgr2=0x55e9110a8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:32.552 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.537+0000 7fcf6aa24740 1 --2- 172.21.15.16:0/2910141697 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e91109e9a0 0x55e9110a8b00 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7fcf5c0a42a0 tx=0x7fcf5c09ce90 comp rx=0 tx=0).stop 2024-09-12T02:15:32.552 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.537+0000 7fcf6aa24740 1 -- 172.21.15.16:0/2910141697 shutdown_connections 2024-09-12T02:15:32.552 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.537+0000 7fcf6aa24740 1 --2- 172.21.15.16:0/2910141697 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55e9110b43d0 0x55e9110b67c0 crc :-1 s=READY pgs=405 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:32.552 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.537+0000 7fcf6aa24740 1 --2- 172.21.15.16:0/2910141697 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9110a4370 0x55e9110b0080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:32.552 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.537+0000 7fcf497fa640 0 client.19950 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:15:32.552 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.537+0000 7fcf6aa24740 1 --2- 172.21.15.16:0/2910141697 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e91109f2e0 0x55e9110a9040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:32.552 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.537+0000 7fcf6aa24740 1 --2- 172.21.15.16:0/2910141697 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e91109e9a0 0x55e9110a8b00 unknown :-1 s=CLOSED pgs=1844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:32.553 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.537+0000 7fcf6aa24740 1 -- 172.21.15.16:0/2910141697 >> 172.21.15.16:0/2910141697 conn(0x55e91109a010 msgr2=0x55e91109b820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:15:32.553 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.537+0000 7fcf6aa24740 1 -- 172.21.15.16:0/2910141697 shutdown_connections 2024-09-12T02:15:32.553 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.537+0000 7fcf6aa24740 1 -- 172.21.15.16:0/2910141697 wait complete. 2024-09-12T02:15:32.553 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.538+0000 7fcf6aa24740 20 client.19950 trim_cache size 0 max 16384 2024-09-12T02:15:32.559 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.552+0000 7fcf6aa24740 1 Processor -- start 2024-09-12T02:15:32.559 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.552+0000 7fcf6aa24740 1 -- start start 2024-09-12T02:15:32.559 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.552+0000 7fcf6aa24740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e91109e9a0 0x55e91109b400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:32.559 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.552+0000 7fcf6aa24740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e91109f2e0 0x55e9110b0040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:32.559 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.552+0000 7fcf6aa24740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9110a4370 0x55e91109b940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:32.560 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.552+0000 7fcf6aa24740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fcf601ada20 con 0x55e9110a4370 2024-09-12T02:15:32.560 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.552+0000 7fcf6aa24740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fcf601ab2f0 con 0x55e91109f2e0 2024-09-12T02:15:32.560 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.552+0000 7fcf6aa24740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fcf601a9140 con 0x55e91109e9a0 2024-09-12T02:15:32.560 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.552+0000 7fcf6614c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9110a4370 0x55e91109b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:32.560 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.552+0000 7fcf6614c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9110a4370 0x55e91109b940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59130/0 (socket says 172.21.15.16:59130) 2024-09-12T02:15:32.560 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.552+0000 7fcf6614c640 1 -- 172.21.15.16:0/2852008181 learned_addr learned my addr 172.21.15.16:0/2852008181 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:15:32.560 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.553+0000 7fcf6614c640 1 -- 172.21.15.16:0/2852008181 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e91109e9a0 msgr2=0x55e91109b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:15:32.560 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.553+0000 7fcf6614c640 1 --2- 172.21.15.16:0/2852008181 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e91109e9a0 0x55e91109b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:32.560 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.553+0000 7fcf6614c640 1 -- 172.21.15.16:0/2852008181 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e91109f2e0 msgr2=0x55e9110b0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:15:32.560 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.553+0000 7fcf6614c640 1 --2- 172.21.15.16:0/2852008181 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e91109f2e0 0x55e9110b0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:32.560 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.553+0000 7fcf6614c640 1 -- 172.21.15.16:0/2852008181 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9110b6b30 con 0x55e9110a4370 2024-09-12T02:15:32.560 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.553+0000 7fcf6614c640 1 --2- 172.21.15.16:0/2852008181 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9110a4370 0x55e91109b940 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7fcf58010620 tx=0x7fcf58016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dbaf66cc4276fd8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:15:32.560 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.553+0000 7fcf4b7fe640 1 -- 172.21.15.16:0/2852008181 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf5801e510 con 0x55e9110a4370 2024-09-12T02:15:32.561 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.553+0000 7fcf4b7fe640 1 -- 172.21.15.16:0/2852008181 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf5801eb90 con 0x55e9110a4370 2024-09-12T02:15:32.561 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.553+0000 7fcf4b7fe640 1 -- 172.21.15.16:0/2852008181 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf5800fce0 con 0x55e9110a4370 2024-09-12T02:15:32.561 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.554+0000 7fcf6aa24740 1 -- 172.21.15.16:0/2852008181 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9110a4370 msgr2=0x55e91109b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:32.561 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.554+0000 7fcf6aa24740 1 --2- 172.21.15.16:0/2852008181 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9110a4370 0x55e91109b940 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7fcf58010620 tx=0x7fcf58016000 comp rx=0 tx=0).stop 2024-09-12T02:15:32.561 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.554+0000 7fcf6aa24740 1 -- 172.21.15.16:0/2852008181 shutdown_connections 2024-09-12T02:15:32.561 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.554+0000 7fcf6aa24740 1 --2- 172.21.15.16:0/2852008181 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9110a4370 0x55e91109b940 unknown :-1 s=CLOSED pgs=1845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:32.561 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.554+0000 7fcf6aa24740 1 --2- 172.21.15.16:0/2852008181 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e91109f2e0 0x55e9110b0040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:32.561 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.554+0000 7fcf6aa24740 1 --2- 172.21.15.16:0/2852008181 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e91109e9a0 0x55e91109b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:32.561 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.554+0000 7fcf6aa24740 1 -- 172.21.15.16:0/2852008181 >> 172.21.15.16:0/2852008181 conn(0x55e91109a010 msgr2=0x55e9110991c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:15:32.561 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.554+0000 7fcf6aa24740 1 -- 172.21.15.16:0/2852008181 shutdown_connections 2024-09-12T02:15:32.561 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.554+0000 7fcf6aa24740 1 -- 172.21.15.16:0/2852008181 wait complete. 2024-09-12T02:15:32.561 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.554+0000 7fcf6aa24740 1 -- start start 2024-09-12T02:15:32.561 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.554+0000 7fcf6aa24740 1 Processor -- start 2024-09-12T02:15:32.562 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.554+0000 7fcf6aa24740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e91109e9a0 0x55e9110ad690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:32.562 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.554+0000 7fcf6aa24740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e91109f2e0 0x55e9110adbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:32.562 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.554+0000 7fcf6aa24740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e9110a4370 0x55e9110aeb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:32.562 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.554+0000 7fcf6aa24740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fcf601a97b0 con 0x55e91109e9a0 2024-09-12T02:15:32.562 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.554+0000 7fcf6aa24740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fcf601aac20 con 0x55e91109f2e0 2024-09-12T02:15:32.562 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.554+0000 7fcf6aa24740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fcf601a9e80 con 0x55e9110a4370 2024-09-12T02:15:32.562 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.555+0000 7fcf6aa24740 10 client.0 _ll_register_callbacks cb 0x7fcf5800a8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:15:32.562 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.555+0000 7fcf6594b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e91109e9a0 0x55e9110ad690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:32.562 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.555+0000 7fcf6594b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e91109e9a0 0x55e9110ad690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59144/0 (socket says 172.21.15.16:59144) 2024-09-12T02:15:32.562 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.555+0000 7fcf6594b640 1 -- 172.21.15.16:0/3044467140 learned_addr learned my addr 172.21.15.16:0/3044467140 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:15:32.562 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.555+0000 7fcf6514a640 1 --2- 172.21.15.16:0/3044467140 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e91109f2e0 0x55e9110adbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:32.562 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.555+0000 7fcf6594b640 1 -- 172.21.15.16:0/3044467140 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e9110a4370 msgr2=0x55e9110aeb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:15:32.562 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.555+0000 7fcf6594b640 1 --2- 172.21.15.16:0/3044467140 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e9110a4370 0x55e9110aeb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:32.563 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.555+0000 7fcf6594b640 1 -- 172.21.15.16:0/3044467140 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e91109f2e0 msgr2=0x55e9110adbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:15:32.563 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.555+0000 7fcf6594b640 1 --2- 172.21.15.16:0/3044467140 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e91109f2e0 0x55e9110adbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:32.563 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.555+0000 7fcf6594b640 1 -- 172.21.15.16:0/3044467140 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9110af1a0 con 0x55e91109e9a0 2024-09-12T02:15:32.563 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.555+0000 7fcf6514a640 1 --2- 172.21.15.16:0/3044467140 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e91109f2e0 0x55e9110adbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:15:32.563 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.555+0000 7fcf6594b640 1 --2- 172.21.15.16:0/3044467140 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e91109e9a0 0x55e9110ad690 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7fcf5c097cb0 tx=0x7fcf5c0cf5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33fdb56edb2ef7b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:15:32.563 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.555+0000 7fcf497fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:15:32.563 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.555+0000 7fcf497fa640 1 -- 172.21.15.16:0/3044467140 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf5c06f290 con 0x55e91109e9a0 2024-09-12T02:15:32.563 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.555+0000 7fcf497fa640 1 -- 172.21.15.16:0/3044467140 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fcf5c099cb0 con 0x55e91109e9a0 2024-09-12T02:15:32.563 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.555+0000 7fcf6aa24740 1 -- 172.21.15.16:0/3044467140 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf5801eb90 con 0x55e91109e9a0 2024-09-12T02:15:32.563 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.555+0000 7fcf497fa640 1 -- 172.21.15.16:0/3044467140 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fcf5c09c970 con 0x55e91109e9a0 2024-09-12T02:15:32.563 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.555+0000 7fcf6aa24740 10 client.19956 fetch_fsmap learned FSMap version 25 2024-09-12T02:15:32.564 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.555+0000 7fcf6aa24740 1 -- 172.21.15.16:0/3044467140 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e9110af1a0 con 0x55e91109e9a0 2024-09-12T02:15:32.564 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.555+0000 7fcf497fa640 1 -- 172.21.15.16:0/3044467140 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf5801eb90 con 0x55e91109e9a0 2024-09-12T02:15:32.564 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.556+0000 7fcf497fa640 1 -- 172.21.15.16:0/3044467140 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fcf5c018c80 con 0x55e91109e9a0 2024-09-12T02:15:32.564 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.556+0000 7fcf6aa24740 10 client.19956 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:15:32.564 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.556+0000 7fcf6aa24740 10 client.19956 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:15:32.564 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.556+0000 7fcf6aa24740 20 client.19956 populate_metadata read hostname 'smithi016' 2024-09-12T02:15:32.564 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.556+0000 7fcf6aa24740 1 --2- 172.21.15.16:0/3044467140 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55e91111f5d0 0x55e9111219f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:32.564 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.556+0000 7fcf6aa24740 4 client.19956 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:15:32.564 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.556+0000 7fcf6aa24740 1 -- 172.21.15.16:0/3044467140 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e911009d70 con 0x55e91111f5d0 2024-09-12T02:15:32.564 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.556+0000 7fcf6514a640 1 --2- 172.21.15.16:0/3044467140 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55e91111f5d0 0x55e9111219f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:32.564 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.556+0000 7fcf497fa640 10 client.19956 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:15:32.564 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.556+0000 7fcf6514a640 1 --2- 172.21.15.16:0/3044467140 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55e91111f5d0 0x55e9111219f0 crc :-1 s=READY pgs=406 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42e022ca0698b796 server_cookie=5e0d2b23ece34b7a in_seq=0 out_seq=0 2024-09-12T02:15:32.568 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.567+0000 7fcf497fa640 1 -- 172.21.15.16:0/3044467140 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e911009d70 con 0x55e91111f5d0 2024-09-12T02:15:32.568 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.567+0000 7fcf497fa640 10 client.19956 handle_command_reply: tid=0 2024-09-12T02:15:32.568 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.567+0000 7fcf6aa24740 1 client.19956 shutdown 2024-09-12T02:15:32.568 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.567+0000 7fcf6aa24740 1 -- 172.21.15.16:0/3044467140 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e91109e9a0 msgr2=0x55e9110ad690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:32.568 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.567+0000 7fcf6aa24740 1 --2- 172.21.15.16:0/3044467140 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e91109e9a0 0x55e9110ad690 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7fcf5c097cb0 tx=0x7fcf5c0cf5a0 comp rx=0 tx=0).stop 2024-09-12T02:15:32.568 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.568+0000 7fcf6aa24740 1 -- 172.21.15.16:0/3044467140 shutdown_connections 2024-09-12T02:15:32.568 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.568+0000 7fcf6aa24740 1 --2- 172.21.15.16:0/3044467140 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55e91111f5d0 0x55e9111219f0 crc :-1 s=READY pgs=406 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:32.568 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.568+0000 7fcf6aa24740 1 --2- 172.21.15.16:0/3044467140 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e9110a4370 0x55e9110aeb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:32.568 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.568+0000 7fcf6aa24740 1 --2- 172.21.15.16:0/3044467140 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e91109f2e0 0x55e9110adbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:32.569 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.568+0000 7fcf497fa640 0 client.19956 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:15:32.569 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.568+0000 7fcf6aa24740 1 --2- 172.21.15.16:0/3044467140 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e91109e9a0 0x55e9110ad690 unknown :-1 s=CLOSED pgs=1846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:32.569 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.568+0000 7fcf6aa24740 1 -- 172.21.15.16:0/3044467140 >> 172.21.15.16:0/3044467140 conn(0x55e91109a010 msgr2=0x55e9110991c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:15:32.569 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.568+0000 7fcf6aa24740 1 -- 172.21.15.16:0/3044467140 shutdown_connections 2024-09-12T02:15:32.569 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.568+0000 7fcf6aa24740 1 -- 172.21.15.16:0/3044467140 wait complete. 2024-09-12T02:15:32.569 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.568+0000 7fcf6aa24740 20 client.19956 trim_cache size 0 max 16384 2024-09-12T02:15:32.569 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-09-12T02:15:32.569 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-09-12T02:15:32.569 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-09-12T02:15:32.570 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-09-12T02:15:32.570 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.568+0000 7fcf67baf640 1 -- 172.21.15.16:0/253095230 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fcf4c0a2c30 msgr2=0x7fcf4c0a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:32.570 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.568+0000 7fcf67baf640 1 --2- 172.21.15.16:0/253095230 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fcf4c0a2c30 0x7fcf4c0a50f0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fcf6006b3e0 tx=0x7fcf58004a40 comp rx=0 tx=0).stop 2024-09-12T02:15:32.570 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.568+0000 7fcf67baf640 1 -- 172.21.15.16:0/253095230 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf601025e0 msgr2=0x7fcf60101f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:32.570 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.568+0000 7fcf67baf640 1 --2- 172.21.15.16:0/253095230 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf601025e0 0x7fcf60101f50 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7fcf5c00e9f0 tx=0x7fcf5c00eeb0 comp rx=0 tx=0).stop 2024-09-12T02:15:32.570 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.568+0000 7fcf67baf640 1 -- 172.21.15.16:0/253095230 shutdown_connections 2024-09-12T02:15:32.571 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.568+0000 7fcf67baf640 1 --2- 172.21.15.16:0/253095230 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fcf4c0a2c30 0x7fcf4c0a50f0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:32.571 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.568+0000 7fcf67baf640 1 --2- 172.21.15.16:0/253095230 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcf60107840 0x7fcf60100ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:32.571 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.568+0000 7fcf67baf640 1 --2- 172.21.15.16:0/253095230 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf60104f10 0x7fcf60100590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:32.571 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.568+0000 7fcf67baf640 1 --2- 172.21.15.16:0/253095230 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf601025e0 0x7fcf60101f50 unknown :-1 s=CLOSED pgs=1842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:32.571 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.568+0000 7fcf67baf640 1 -- 172.21.15.16:0/253095230 >> 172.21.15.16:0/253095230 conn(0x7fcf600fc1e0 msgr2=0x7fcf60102e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:15:32.571 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.568+0000 7fcf67baf640 1 -- 172.21.15.16:0/253095230 shutdown_connections 2024-09-12T02:15:32.571 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:32.569+0000 7fcf67baf640 1 -- 172.21.15.16:0/253095230 wait complete. 2024-09-12T02:15:32.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:15:32 smithi184 ceph-mon[35593]: pgmap v743: 97 pgs: 97 active+clean; 3.0 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.9 KiB/s wr, 9 op/s 2024-09-12T02:15:32.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:15:32 smithi033 ceph-mon[35332]: pgmap v743: 97 pgs: 97 active+clean; 3.0 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.9 KiB/s wr, 9 op/s 2024-09-12T02:15:32.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:15:32 smithi016 ceph-mon[30947]: pgmap v743: 97 pgs: 97 active+clean; 3.0 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.9 KiB/s wr, 9 op/s 2024-09-12T02:15:33.013 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24482 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0' 2024-09-12T02:15:33.013 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24482 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0 2024-09-12T02:15:33.310 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:15:33.553 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:15:33 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/749753194' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:15:33.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:15:33 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/749753194' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:15:33.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:15:33 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/749753194' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:15:33.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.764+0000 7f7c583e7640 1 -- 172.21.15.16:0/2245358541 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c50106a80 msgr2=0x7f7c50106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:33.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.764+0000 7f7c583e7640 1 --2- 172.21.15.16:0/2245358541 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c50106a80 0x7f7c50106e60 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7f7c4c009930 tx=0x7f7c4c02a420 comp rx=0 tx=0).stop 2024-09-12T02:15:33.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.764+0000 7f7c583e7640 1 -- 172.21.15.16:0/2245358541 shutdown_connections 2024-09-12T02:15:33.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.764+0000 7f7c583e7640 1 --2- 172.21.15.16:0/2245358541 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c50101420 0x7f7c5010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:33.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.764+0000 7f7c583e7640 1 --2- 172.21.15.16:0/2245358541 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7c50100a80 0x7f7c50100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:33.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.764+0000 7f7c583e7640 1 --2- 172.21.15.16:0/2245358541 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c50106a80 0x7f7c50106e60 unknown :-1 s=CLOSED pgs=1847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:33.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.764+0000 7f7c583e7640 1 -- 172.21.15.16:0/2245358541 >> 172.21.15.16:0/2245358541 conn(0x7f7c500fc7a0 msgr2=0x7f7c500febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:15:33.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.764+0000 7f7c583e7640 1 -- 172.21.15.16:0/2245358541 shutdown_connections 2024-09-12T02:15:33.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.764+0000 7f7c583e7640 1 -- 172.21.15.16:0/2245358541 wait complete. 2024-09-12T02:15:33.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.765+0000 7f7c583e7640 1 Processor -- start 2024-09-12T02:15:33.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.765+0000 7f7c583e7640 1 -- start start 2024-09-12T02:15:33.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.765+0000 7f7c583e7640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c50100a80 0x7f7c501992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:33.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.765+0000 7f7c583e7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c50101420 0x7f7c50199830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:33.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.765+0000 7f7c583e7640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7c50106a80 0x7f7c501a4370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:33.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.765+0000 7f7c583e7640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7c50110ea0 con 0x7f7c50101420 2024-09-12T02:15:33.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.765+0000 7f7c583e7640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7c50110d20 con 0x7f7c50100a80 2024-09-12T02:15:33.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.765+0000 7f7c583e7640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7c50111020 con 0x7f7c50106a80 2024-09-12T02:15:33.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.765+0000 7f7c55982640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c50101420 0x7f7c50199830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:33.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.765+0000 7f7c55982640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c50101420 0x7f7c50199830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59172/0 (socket says 172.21.15.16:59172) 2024-09-12T02:15:33.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.765+0000 7f7c55982640 1 -- 172.21.15.16:0/509383587 learned_addr learned my addr 172.21.15.16:0/509383587 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:15:33.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.765+0000 7f7c56984640 1 --2- 172.21.15.16:0/509383587 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7c50106a80 0x7f7c501a4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:33.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.765+0000 7f7c56183640 1 --2- 172.21.15.16:0/509383587 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c50100a80 0x7f7c501992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:33.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.766+0000 7f7c55982640 1 -- 172.21.15.16:0/509383587 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7c50106a80 msgr2=0x7f7c501a4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:33.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.766+0000 7f7c55982640 1 --2- 172.21.15.16:0/509383587 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7c50106a80 0x7f7c501a4370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:33.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.766+0000 7f7c55982640 1 -- 172.21.15.16:0/509383587 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c50100a80 msgr2=0x7f7c501992f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:33.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.766+0000 7f7c55982640 1 --2- 172.21.15.16:0/509383587 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c50100a80 0x7f7c501992f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:33.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.766+0000 7f7c55982640 1 -- 172.21.15.16:0/509383587 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c501a48b0 con 0x7f7c50101420 2024-09-12T02:15:33.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.766+0000 7f7c55982640 1 --2- 172.21.15.16:0/509383587 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c50101420 0x7f7c50199830 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7f7c4000d950 tx=0x7f7c4000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:15:33.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.766+0000 7f7c477fe640 1 -- 172.21.15.16:0/509383587 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c40017070 con 0x7f7c50101420 2024-09-12T02:15:33.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.766+0000 7f7c477fe640 1 -- 172.21.15.16:0/509383587 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c40009c30 con 0x7f7c50101420 2024-09-12T02:15:33.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.766+0000 7f7c477fe640 1 -- 172.21.15.16:0/509383587 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c4000f6b0 con 0x7f7c50101420 2024-09-12T02:15:33.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.766+0000 7f7c583e7640 1 -- 172.21.15.16:0/509383587 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c501a4a80 con 0x7f7c50101420 2024-09-12T02:15:33.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.766+0000 7f7c583e7640 1 -- 172.21.15.16:0/509383587 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c501a4fc0 con 0x7f7c50101420 2024-09-12T02:15:33.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.768+0000 7f7c477fe640 1 -- 172.21.15.16:0/509383587 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f7c400109d0 con 0x7f7c50101420 2024-09-12T02:15:33.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.769+0000 7f7c5a7c0740 1 Processor -- start 2024-09-12T02:15:33.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.769+0000 7f7c5a7c0740 1 -- start start 2024-09-12T02:15:33.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.769+0000 7f7c5a7c0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638ad2bc660 0x5638ad2bca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:33.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.769+0000 7f7c5a7c0740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5638ad2bcf80 0x5638ad2cb810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:33.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.769+0000 7f7c5a7c0740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5638ad2cbd50 0x5638ad2ce140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:33.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.769+0000 7f7c5a7c0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5638ad2222a0 con 0x5638ad2bc660 2024-09-12T02:15:33.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.769+0000 7f7c5a7c0740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5638ad1c9b10 con 0x5638ad2cbd50 2024-09-12T02:15:33.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.769+0000 7f7c5a7c0740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5638ad21e260 con 0x5638ad2bcf80 2024-09-12T02:15:33.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.769+0000 7f7c56984640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638ad2bc660 0x5638ad2bca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:33.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.769+0000 7f7c477fe640 1 --2- 172.21.15.16:0/509383587 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f7c3c0a2c00 0x7f7c3c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:33.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.769+0000 7f7c56984640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638ad2bc660 0x5638ad2bca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59182/0 (socket says 172.21.15.16:59182) 2024-09-12T02:15:33.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.769+0000 7f7c56984640 1 -- 172.21.15.16:0/993868247 learned_addr learned my addr 172.21.15.16:0/993868247 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:15:33.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.769+0000 7f7c56183640 1 --2- 172.21.15.16:0/993868247 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5638ad2bcf80 0x5638ad2cb810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:33.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.769+0000 7f7c56984640 1 --2- 172.21.15.16:0/509383587 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f7c3c0a2c00 0x7f7c3c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:33.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.770+0000 7f7c477fe640 1 -- 172.21.15.16:0/509383587 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f7c400ceae0 con 0x7f7c50101420 2024-09-12T02:15:33.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.770+0000 7f7c55982640 1 --2- 172.21.15.16:0/993868247 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5638ad2cbd50 0x5638ad2ce140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:33.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.770+0000 7f7c56984640 1 -- 172.21.15.16:0/993868247 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5638ad2bcf80 msgr2=0x5638ad2cb810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:15:33.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.770+0000 7f7c56984640 1 --2- 172.21.15.16:0/993868247 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5638ad2bcf80 0x5638ad2cb810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:33.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.770+0000 7f7c56984640 1 -- 172.21.15.16:0/993868247 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5638ad2cbd50 msgr2=0x5638ad2ce140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:15:33.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.770+0000 7f7c56984640 1 --2- 172.21.15.16:0/993868247 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5638ad2cbd50 0x5638ad2ce140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:33.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.770+0000 7f7c56984640 1 -- 172.21.15.16:0/993868247 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638acf93b90 con 0x5638ad2bc660 2024-09-12T02:15:33.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.770+0000 7f7c56183640 1 --2- 172.21.15.16:0/993868247 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5638ad2bcf80 0x5638ad2cb810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:15:33.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.770+0000 7f7c56984640 1 --2- 172.21.15.16:0/993868247 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638ad2bc660 0x5638ad2bca40 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7f7c4800a990 tx=0x7f7c4800ae50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd9e6a24601f3256 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:15:33.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.770+0000 7f7c44dea640 1 -- 172.21.15.16:0/993868247 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c4801d030 con 0x5638ad2bc660 2024-09-12T02:15:33.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.770+0000 7f7c44dea640 1 -- 172.21.15.16:0/993868247 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c48015c60 con 0x5638ad2bc660 2024-09-12T02:15:33.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.770+0000 7f7c44dea640 1 -- 172.21.15.16:0/993868247 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c480213c0 con 0x5638ad2bc660 2024-09-12T02:15:33.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.770+0000 7f7c56984640 1 --2- 172.21.15.16:0/509383587 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f7c3c0a2c00 0x7f7c3c0a50c0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f7c48020000 tx=0x7f7c4801e000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:15:33.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.771+0000 7f7c5a7c0740 1 -- 172.21.15.16:0/993868247 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638ad2bc660 msgr2=0x5638ad2bca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:33.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.771+0000 7f7c5a7c0740 1 --2- 172.21.15.16:0/993868247 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638ad2bc660 0x5638ad2bca40 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7f7c4800a990 tx=0x7f7c4800ae50 comp rx=0 tx=0).stop 2024-09-12T02:15:33.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.771+0000 7f7c5a7c0740 1 -- 172.21.15.16:0/993868247 shutdown_connections 2024-09-12T02:15:33.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.771+0000 7f7c5a7c0740 1 --2- 172.21.15.16:0/993868247 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5638ad2cbd50 0x5638ad2ce140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:33.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.771+0000 7f7c5a7c0740 1 --2- 172.21.15.16:0/993868247 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5638ad2bcf80 0x5638ad2cb810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:33.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.771+0000 7f7c5a7c0740 1 --2- 172.21.15.16:0/993868247 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638ad2bc660 0x5638ad2bca40 unknown :-1 s=CLOSED pgs=1849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:33.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.771+0000 7f7c5a7c0740 1 -- 172.21.15.16:0/993868247 >> 172.21.15.16:0/993868247 conn(0x5638ad2b8cf0 msgr2=0x5638ad2b90d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:15:33.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.771+0000 7f7c5a7c0740 1 -- 172.21.15.16:0/993868247 shutdown_connections 2024-09-12T02:15:33.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.771+0000 7f7c5a7c0740 1 -- 172.21.15.16:0/993868247 wait complete. 2024-09-12T02:15:33.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.771+0000 7f7c5a7c0740 1 -- start start 2024-09-12T02:15:33.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.772+0000 7f7c5a7c0740 1 Processor -- start 2024-09-12T02:15:33.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.772+0000 7f7c5a7c0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638ad2bc660 0x5638ad2c8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:33.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.772+0000 7f7c5a7c0740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5638ad2bcf80 0x5638ad2c9200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:33.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.772+0000 7f7c5a7c0740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5638ad2cbd50 0x5638ad2c9740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:33.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.772+0000 7f7c5a7c0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5638ad2ce980 con 0x5638ad2bc660 2024-09-12T02:15:33.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.772+0000 7f7c5a7c0740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5638ad1aa6e0 con 0x5638ad2bcf80 2024-09-12T02:15:33.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.772+0000 7f7c5a7c0740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5638ad2ceb00 con 0x5638ad2cbd50 2024-09-12T02:15:33.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.772+0000 7f7c5a7c0740 10 client.0 _ll_register_callbacks cb 0x5638ad1cea80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:15:33.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.772+0000 7f7c56183640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638ad2bc660 0x5638ad2c8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:33.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.773+0000 7f7c56183640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638ad2bc660 0x5638ad2c8cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59184/0 (socket says 172.21.15.16:59184) 2024-09-12T02:15:33.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.773+0000 7f7c56183640 1 -- 172.21.15.16:0/2562806927 learned_addr learned my addr 172.21.15.16:0/2562806927 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:15:33.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.773+0000 7f7c55982640 1 --2- 172.21.15.16:0/2562806927 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5638ad2bcf80 0x5638ad2c9200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:33.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.773+0000 7f7c56984640 1 --2- 172.21.15.16:0/2562806927 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5638ad2cbd50 0x5638ad2c9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:33.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.773+0000 7f7c56183640 1 -- 172.21.15.16:0/2562806927 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5638ad2cbd50 msgr2=0x5638ad2c9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:15:33.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.773+0000 7f7c56183640 1 --2- 172.21.15.16:0/2562806927 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5638ad2cbd50 0x5638ad2c9740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:33.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.773+0000 7f7c56183640 1 -- 172.21.15.16:0/2562806927 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5638ad2bcf80 msgr2=0x5638ad2c9200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:15:33.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.773+0000 7f7c56183640 1 --2- 172.21.15.16:0/2562806927 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5638ad2bcf80 0x5638ad2c9200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:33.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.773+0000 7f7c56183640 1 -- 172.21.15.16:0/2562806927 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638ad2d7f30 con 0x5638ad2bc660 2024-09-12T02:15:33.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.773+0000 7f7c56984640 1 --2- 172.21.15.16:0/2562806927 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5638ad2cbd50 0x5638ad2c9740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:15:33.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.773+0000 7f7c56183640 1 --2- 172.21.15.16:0/2562806927 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638ad2bc660 0x5638ad2c8cc0 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7f7c4c009930 tx=0x7f7c4c0318d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b8b1cf9c60bc32d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:15:33.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.773+0000 7f7c39ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:15:33.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.773+0000 7f7c39ffb640 1 -- 172.21.15.16:0/2562806927 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c4c035040 con 0x5638ad2bc660 2024-09-12T02:15:33.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.773+0000 7f7c39ffb640 1 -- 172.21.15.16:0/2562806927 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f7c4c004240 con 0x5638ad2bc660 2024-09-12T02:15:33.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.773+0000 7f7c5a7c0740 1 -- 172.21.15.16:0/2562806927 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c48015c60 con 0x5638ad2bc660 2024-09-12T02:15:33.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.773+0000 7f7c39ffb640 1 -- 172.21.15.16:0/2562806927 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7c4c004440 con 0x5638ad2bc660 2024-09-12T02:15:33.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.773+0000 7f7c5a7c0740 10 client.19968 fetch_fsmap learned FSMap version 25 2024-09-12T02:15:33.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.774+0000 7f7c5a7c0740 1 -- 172.21.15.16:0/2562806927 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5638ad2da830 con 0x5638ad2bc660 2024-09-12T02:15:33.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.774+0000 7f7c39ffb640 1 -- 172.21.15.16:0/2562806927 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c48015c60 con 0x5638ad2bc660 2024-09-12T02:15:33.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.774+0000 7f7c39ffb640 1 -- 172.21.15.16:0/2562806927 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f7c4c0457c0 con 0x5638ad2bc660 2024-09-12T02:15:33.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.774+0000 7f7c5a7c0740 10 client.19968 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:15:33.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.774+0000 7f7c5a7c0740 10 client.19968 resolve_mds: validated gid 24482 aka daemon mds.b 2024-09-12T02:15:33.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.774+0000 7f7c5a7c0740 20 client.19968 populate_metadata read hostname 'smithi016' 2024-09-12T02:15:33.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.774+0000 7f7c5a7c0740 1 --2- 172.21.15.16:0/2562806927 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5638ad2dbcf0 0x5638ad2de0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:33.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.774+0000 7f7c5a7c0740 4 client.19968 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:15:33.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.774+0000 7f7c5a7c0740 1 -- 172.21.15.16:0/2562806927 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5638ad22f9d0 con 0x5638ad2dbcf0 2024-09-12T02:15:33.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.774+0000 7f7c55982640 1 --2- 172.21.15.16:0/2562806927 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5638ad2dbcf0 0x5638ad2de0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:33.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.775+0000 7f7c55982640 1 --2- 172.21.15.16:0/2562806927 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5638ad2dbcf0 0x5638ad2de0e0 crc :-1 s=READY pgs=407 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8e36982b0be083e server_cookie=3161f16664d7f115 in_seq=0 out_seq=0 2024-09-12T02:15:33.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.775+0000 7f7c39ffb640 10 client.19968 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:15:33.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.775+0000 7f7c39ffb640 1 -- 172.21.15.16:0/2562806927 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5638ad22f9d0 con 0x5638ad2dbcf0 2024-09-12T02:15:33.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.775+0000 7f7c39ffb640 10 client.19968 handle_command_reply: tid=0 2024-09-12T02:15:33.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.776+0000 7f7c5a7c0740 1 client.19968 shutdown 2024-09-12T02:15:33.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.776+0000 7f7c5a7c0740 1 -- 172.21.15.16:0/2562806927 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638ad2bc660 msgr2=0x5638ad2c8cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:33.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.776+0000 7f7c5a7c0740 1 --2- 172.21.15.16:0/2562806927 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638ad2bc660 0x5638ad2c8cc0 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7f7c4c009930 tx=0x7f7c4c0318d0 comp rx=0 tx=0).stop 2024-09-12T02:15:33.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.776+0000 7f7c5a7c0740 1 -- 172.21.15.16:0/2562806927 shutdown_connections 2024-09-12T02:15:33.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.776+0000 7f7c5a7c0740 1 --2- 172.21.15.16:0/2562806927 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5638ad2dbcf0 0x5638ad2de0e0 crc :-1 s=READY pgs=407 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:33.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.776+0000 7f7c5a7c0740 1 --2- 172.21.15.16:0/2562806927 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5638ad2cbd50 0x5638ad2c9740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:33.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.776+0000 7f7c5a7c0740 1 --2- 172.21.15.16:0/2562806927 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5638ad2bcf80 0x5638ad2c9200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:33.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.776+0000 7f7c39ffb640 0 client.19968 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:15:33.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.776+0000 7f7c5a7c0740 1 --2- 172.21.15.16:0/2562806927 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638ad2bc660 0x5638ad2c8cc0 unknown :-1 s=CLOSED pgs=1850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:33.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.776+0000 7f7c5a7c0740 1 -- 172.21.15.16:0/2562806927 >> 172.21.15.16:0/2562806927 conn(0x5638ad2b8cf0 msgr2=0x5638ad2be9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:15:33.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.776+0000 7f7c5a7c0740 1 -- 172.21.15.16:0/2562806927 shutdown_connections 2024-09-12T02:15:33.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.776+0000 7f7c5a7c0740 1 -- 172.21.15.16:0/2562806927 wait complete. 2024-09-12T02:15:33.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.776+0000 7f7c5a7c0740 20 client.19968 trim_cache size 0 max 16384 2024-09-12T02:15:33.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.793+0000 7f7c5a7c0740 1 Processor -- start 2024-09-12T02:15:33.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.793+0000 7f7c5a7c0740 1 -- start start 2024-09-12T02:15:33.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.793+0000 7f7c5a7c0740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5638ad2bc660 0x5638ad2e39b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:33.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.793+0000 7f7c5a7c0740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5638ad2bcf80 0x5638ad2e3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:33.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.793+0000 7f7c5a7c0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638ad2cbd50 0x5638ad2d3d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:33.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.793+0000 7f7c5a7c0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7c501adb80 con 0x5638ad2cbd50 2024-09-12T02:15:33.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.793+0000 7f7c5a7c0740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7c501ab450 con 0x5638ad2bcf80 2024-09-12T02:15:33.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.793+0000 7f7c5a7c0740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7c501a92c0 con 0x5638ad2bc660 2024-09-12T02:15:33.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.793+0000 7f7c56984640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638ad2cbd50 0x5638ad2d3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:33.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.793+0000 7f7c56984640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638ad2cbd50 0x5638ad2d3d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59200/0 (socket says 172.21.15.16:59200) 2024-09-12T02:15:33.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.793+0000 7f7c56984640 1 -- 172.21.15.16:0/1524311489 learned_addr learned my addr 172.21.15.16:0/1524311489 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:15:33.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.794+0000 7f7c55982640 1 --2- 172.21.15.16:0/1524311489 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5638ad2bcf80 0x5638ad2e3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:33.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.794+0000 7f7c56183640 1 --2- 172.21.15.16:0/1524311489 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5638ad2bc660 0x5638ad2e39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:33.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.794+0000 7f7c56984640 1 -- 172.21.15.16:0/1524311489 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5638ad2bc660 msgr2=0x5638ad2e39b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:15:33.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.794+0000 7f7c56984640 1 --2- 172.21.15.16:0/1524311489 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5638ad2bc660 0x5638ad2e39b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:33.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.794+0000 7f7c56984640 1 -- 172.21.15.16:0/1524311489 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5638ad2bcf80 msgr2=0x5638ad2e3ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:15:33.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.794+0000 7f7c56984640 1 --2- 172.21.15.16:0/1524311489 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5638ad2bcf80 0x5638ad2e3ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:33.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.794+0000 7f7c56984640 1 -- 172.21.15.16:0/1524311489 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638ad2b4260 con 0x5638ad2cbd50 2024-09-12T02:15:33.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.794+0000 7f7c55982640 1 --2- 172.21.15.16:0/1524311489 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5638ad2bcf80 0x5638ad2e3ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:15:33.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.794+0000 7f7c56984640 1 --2- 172.21.15.16:0/1524311489 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638ad2cbd50 0x5638ad2d3d80 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7f7c48009bc0 tx=0x7f7c4800a440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16aef0f413dedb73 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:15:33.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.794+0000 7f7c3bfff640 1 -- 172.21.15.16:0/1524311489 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c4802f070 con 0x5638ad2cbd50 2024-09-12T02:15:33.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.794+0000 7f7c3bfff640 1 -- 172.21.15.16:0/1524311489 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c48026070 con 0x5638ad2cbd50 2024-09-12T02:15:33.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.794+0000 7f7c3bfff640 1 -- 172.21.15.16:0/1524311489 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c4801d050 con 0x5638ad2cbd50 2024-09-12T02:15:33.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.795+0000 7f7c5a7c0740 1 -- 172.21.15.16:0/1524311489 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638ad2cbd50 msgr2=0x5638ad2d3d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:33.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.795+0000 7f7c5a7c0740 1 --2- 172.21.15.16:0/1524311489 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638ad2cbd50 0x5638ad2d3d80 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7f7c48009bc0 tx=0x7f7c4800a440 comp rx=0 tx=0).stop 2024-09-12T02:15:33.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.795+0000 7f7c5a7c0740 1 -- 172.21.15.16:0/1524311489 shutdown_connections 2024-09-12T02:15:33.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.795+0000 7f7c5a7c0740 1 --2- 172.21.15.16:0/1524311489 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638ad2cbd50 0x5638ad2d3d80 unknown :-1 s=CLOSED pgs=1851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:33.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.795+0000 7f7c5a7c0740 1 --2- 172.21.15.16:0/1524311489 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5638ad2bcf80 0x5638ad2e3ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:33.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.795+0000 7f7c5a7c0740 1 --2- 172.21.15.16:0/1524311489 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5638ad2bc660 0x5638ad2e39b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:33.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.795+0000 7f7c5a7c0740 1 -- 172.21.15.16:0/1524311489 >> 172.21.15.16:0/1524311489 conn(0x5638ad2b8cf0 msgr2=0x5638ad2b7e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:15:33.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.795+0000 7f7c5a7c0740 1 -- 172.21.15.16:0/1524311489 shutdown_connections 2024-09-12T02:15:33.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.795+0000 7f7c5a7c0740 1 -- 172.21.15.16:0/1524311489 wait complete. 2024-09-12T02:15:33.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.795+0000 7f7c5a7c0740 1 -- start start 2024-09-12T02:15:33.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.795+0000 7f7c5a7c0740 1 Processor -- start 2024-09-12T02:15:33.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.796+0000 7f7c5a7c0740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5638ad2bc660 0x5638ad2d45f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:33.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.796+0000 7f7c5a7c0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638ad2bcf80 0x5638ad2d0980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:33.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.796+0000 7f7c5a7c0740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5638ad2cbd50 0x5638ad2d0f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:33.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.796+0000 7f7c5a7c0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7c501a9910 con 0x5638ad2bcf80 2024-09-12T02:15:33.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.796+0000 7f7c5a7c0740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7c501aad80 con 0x5638ad2bc660 2024-09-12T02:15:33.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.796+0000 7f7c5a7c0740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7c501a9fe0 con 0x5638ad2cbd50 2024-09-12T02:15:33.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.796+0000 7f7c5a7c0740 10 client.0 _ll_register_callbacks cb 0x7f7c4c0095f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:15:33.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.796+0000 7f7c55982640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638ad2bcf80 0x5638ad2d0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:33.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.796+0000 7f7c55982640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638ad2bcf80 0x5638ad2d0980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59210/0 (socket says 172.21.15.16:59210) 2024-09-12T02:15:33.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.796+0000 7f7c55982640 1 -- 172.21.15.16:0/4184515042 learned_addr learned my addr 172.21.15.16:0/4184515042 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:15:33.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.796+0000 7f7c56183640 1 --2- 172.21.15.16:0/4184515042 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5638ad2bc660 0x5638ad2d45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:33.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.796+0000 7f7c56984640 1 --2- 172.21.15.16:0/4184515042 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5638ad2cbd50 0x5638ad2d0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:33.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.796+0000 7f7c55982640 1 -- 172.21.15.16:0/4184515042 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5638ad2cbd50 msgr2=0x5638ad2d0f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:15:33.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.796+0000 7f7c55982640 1 --2- 172.21.15.16:0/4184515042 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5638ad2cbd50 0x5638ad2d0f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:33.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.796+0000 7f7c55982640 1 -- 172.21.15.16:0/4184515042 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5638ad2bc660 msgr2=0x5638ad2d45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:15:33.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.796+0000 7f7c55982640 1 --2- 172.21.15.16:0/4184515042 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5638ad2bc660 0x5638ad2d45f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:33.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.796+0000 7f7c55982640 1 -- 172.21.15.16:0/4184515042 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638ad2d17a0 con 0x5638ad2bcf80 2024-09-12T02:15:33.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.796+0000 7f7c55982640 1 --2- 172.21.15.16:0/4184515042 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638ad2bcf80 0x5638ad2d0980 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7f7c40009610 tx=0x7f7c40000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5b67a09473498c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:15:33.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.796+0000 7f7c39ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:15:33.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.796+0000 7f7c39ffb640 1 -- 172.21.15.16:0/4184515042 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c40020040 con 0x5638ad2bcf80 2024-09-12T02:15:33.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.796+0000 7f7c39ffb640 1 -- 172.21.15.16:0/4184515042 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f7c40097b00 con 0x5638ad2bcf80 2024-09-12T02:15:33.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.796+0000 7f7c5a7c0740 1 -- 172.21.15.16:0/4184515042 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c48026070 con 0x5638ad2bcf80 2024-09-12T02:15:33.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.797+0000 7f7c39ffb640 1 -- 172.21.15.16:0/4184515042 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7c400a4050 con 0x5638ad2bcf80 2024-09-12T02:15:33.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.797+0000 7f7c5a7c0740 10 client.19974 fetch_fsmap learned FSMap version 25 2024-09-12T02:15:33.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.797+0000 7f7c5a7c0740 1 -- 172.21.15.16:0/4184515042 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5638ad347a80 con 0x5638ad2bcf80 2024-09-12T02:15:33.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.797+0000 7f7c39ffb640 1 -- 172.21.15.16:0/4184515042 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c48026070 con 0x5638ad2bcf80 2024-09-12T02:15:33.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.797+0000 7f7c39ffb640 1 -- 172.21.15.16:0/4184515042 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f7c4009a350 con 0x5638ad2bcf80 2024-09-12T02:15:33.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.797+0000 7f7c5a7c0740 10 client.19974 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:15:33.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.797+0000 7f7c5a7c0740 10 client.19974 resolve_mds: validated gid 24482 aka daemon mds.b 2024-09-12T02:15:33.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.797+0000 7f7c5a7c0740 20 client.19974 populate_metadata read hostname 'smithi016' 2024-09-12T02:15:33.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.797+0000 7f7c5a7c0740 1 --2- 172.21.15.16:0/4184515042 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5638ad348260 0x5638ad34a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:33.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.797+0000 7f7c5a7c0740 4 client.19974 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}] 2024-09-12T02:15:33.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.797+0000 7f7c5a7c0740 1 -- 172.21.15.16:0/4184515042 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}) -- 0x5638ad229d20 con 0x5638ad348260 2024-09-12T02:15:33.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.797+0000 7f7c56183640 1 --2- 172.21.15.16:0/4184515042 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5638ad348260 0x5638ad34a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:33.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.798+0000 7f7c39ffb640 10 client.19974 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:15:33.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.798+0000 7f7c56183640 1 --2- 172.21.15.16:0/4184515042 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5638ad348260 0x5638ad34a680 crc :-1 s=READY pgs=408 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10d16d8d079924a9 server_cookie=e46c499170f67500 in_seq=0 out_seq=0 2024-09-12T02:15:33.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.809+0000 7f7c39ffb640 1 -- 172.21.15.16:0/4184515042 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5638ad229d20 con 0x5638ad348260 2024-09-12T02:15:33.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.809+0000 7f7c39ffb640 10 client.19974 handle_command_reply: tid=0 2024-09-12T02:15:33.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.809+0000 7f7c5a7c0740 1 client.19974 shutdown 2024-09-12T02:15:33.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.809+0000 7f7c5a7c0740 1 -- 172.21.15.16:0/4184515042 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638ad2bcf80 msgr2=0x5638ad2d0980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:33.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.809+0000 7f7c5a7c0740 1 --2- 172.21.15.16:0/4184515042 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638ad2bcf80 0x5638ad2d0980 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7f7c40009610 tx=0x7f7c40000bc0 comp rx=0 tx=0).stop 2024-09-12T02:15:33.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.809+0000 7f7c5a7c0740 1 -- 172.21.15.16:0/4184515042 shutdown_connections 2024-09-12T02:15:33.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.809+0000 7f7c5a7c0740 1 --2- 172.21.15.16:0/4184515042 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5638ad348260 0x5638ad34a680 crc :-1 s=READY pgs=408 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:33.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.809+0000 7f7c5a7c0740 1 --2- 172.21.15.16:0/4184515042 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5638ad2cbd50 0x5638ad2d0f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:33.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.809+0000 7f7c39ffb640 0 client.19974 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:15:33.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.809+0000 7f7c5a7c0740 1 --2- 172.21.15.16:0/4184515042 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638ad2bcf80 0x5638ad2d0980 unknown :-1 s=CLOSED pgs=1852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:33.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.809+0000 7f7c5a7c0740 1 --2- 172.21.15.16:0/4184515042 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5638ad2bc660 0x5638ad2d45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:33.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.809+0000 7f7c5a7c0740 1 -- 172.21.15.16:0/4184515042 >> 172.21.15.16:0/4184515042 conn(0x5638ad2b8cf0 msgr2=0x5638ad2c8c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:15:33.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.809+0000 7f7c5a7c0740 1 -- 172.21.15.16:0/4184515042 shutdown_connections 2024-09-12T02:15:33.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.809+0000 7f7c5a7c0740 1 -- 172.21.15.16:0/4184515042 wait complete. 2024-09-12T02:15:33.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.810+0000 7f7c5a7c0740 20 client.19974 trim_cache size 0 max 16384 2024-09-12T02:15:33.811 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T02:15:33.811 INFO:teuthology.orchestra.run.smithi016.stdout: "epoch": 25, 2024-09-12T02:15:33.811 INFO:teuthology.orchestra.run.smithi016.stdout: "leader": 24482, 2024-09-12T02:15:33.811 INFO:teuthology.orchestra.run.smithi016.stdout: "set_version": 54, 2024-09-12T02:15:33.811 INFO:teuthology.orchestra.run.smithi016.stdout: "sets": { 2024-09-12T02:15:33.811 INFO:teuthology.orchestra.run.smithi016.stdout: "99f960cd": { 2024-09-12T02:15:33.811 INFO:teuthology.orchestra.run.smithi016.stdout: "version": 54, 2024-09-12T02:15:33.812 INFO:teuthology.orchestra.run.smithi016.stdout: "age_ref": 0.0, 2024-09-12T02:15:33.812 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-12T02:15:33.812 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "QUIESCED", 2024-09-12T02:15:33.812 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-12T02:15:33.812 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:15:33.812 INFO:teuthology.orchestra.run.smithi016.stdout: "timeout": 90.0, 2024-09-12T02:15:33.812 INFO:teuthology.orchestra.run.smithi016.stdout: "expiration": 150.0, 2024-09-12T02:15:33.812 INFO:teuthology.orchestra.run.smithi016.stdout: "members": { 2024-09-12T02:15:33.812 INFO:teuthology.orchestra.run.smithi016.stdout: "file:/": { 2024-09-12T02:15:33.812 INFO:teuthology.orchestra.run.smithi016.stdout: "excluded": false, 2024-09-12T02:15:33.812 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-12T02:15:33.812 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "QUIESCED", 2024-09-12T02:15:33.812 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-12T02:15:33.812 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:15:33.813 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:15:33.813 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:15:33.813 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:15:33.813 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:15:33.813 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T02:15:33.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.810+0000 7f7c583e7640 1 -- 172.21.15.16:0/509383587 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f7c3c0a2c00 msgr2=0x7f7c3c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:33.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.810+0000 7f7c583e7640 1 --2- 172.21.15.16:0/509383587 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f7c3c0a2c00 0x7f7c3c0a50c0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f7c48020000 tx=0x7f7c4801e000 comp rx=0 tx=0).stop 2024-09-12T02:15:33.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.810+0000 7f7c583e7640 1 -- 172.21.15.16:0/509383587 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c50101420 msgr2=0x7f7c50199830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:33.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.810+0000 7f7c583e7640 1 --2- 172.21.15.16:0/509383587 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c50101420 0x7f7c50199830 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7f7c4000d950 tx=0x7f7c4000de10 comp rx=0 tx=0).stop 2024-09-12T02:15:33.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.810+0000 7f7c583e7640 1 -- 172.21.15.16:0/509383587 shutdown_connections 2024-09-12T02:15:33.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.810+0000 7f7c583e7640 1 --2- 172.21.15.16:0/509383587 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f7c3c0a2c00 0x7f7c3c0a50c0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:33.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.810+0000 7f7c583e7640 1 --2- 172.21.15.16:0/509383587 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7c50106a80 0x7f7c501a4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:33.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.810+0000 7f7c583e7640 1 --2- 172.21.15.16:0/509383587 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c50101420 0x7f7c50199830 unknown :-1 s=CLOSED pgs=1848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:33.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.810+0000 7f7c583e7640 1 --2- 172.21.15.16:0/509383587 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c50100a80 0x7f7c501992f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:33.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.810+0000 7f7c583e7640 1 -- 172.21.15.16:0/509383587 >> 172.21.15.16:0/509383587 conn(0x7f7c500fc7a0 msgr2=0x7f7c500fe070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:15:33.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.810+0000 7f7c583e7640 1 -- 172.21.15.16:0/509383587 shutdown_connections 2024-09-12T02:15:33.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:33.810+0000 7f7c583e7640 1 -- 172.21.15.16:0/509383587 wait complete. 2024-09-12T02:15:34.566 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:15:34 smithi016 ceph-mon[30947]: pgmap v744: 97 pgs: 97 active+clean; 3.0 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 3.0 KiB/s wr, 7 op/s 2024-09-12T02:15:34.567 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '99f960cd', quiesce took 3.2 seconds. Will release after: 26.8 2024-09-12T02:15:34.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:15:34 smithi184 ceph-mon[35593]: pgmap v744: 97 pgs: 97 active+clean; 3.0 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 3.0 KiB/s wr, 7 op/s 2024-09-12T02:15:34.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:15:34 smithi033 ceph-mon[35332]: pgmap v744: 97 pgs: 97 active+clean; 3.0 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 3.0 KiB/s wr, 7 op/s 2024-09-12T02:15:36.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:15:36 smithi184 ceph-mon[35593]: pgmap v745: 97 pgs: 97 active+clean; 3.0 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-09-12T02:15:36.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:15:36 smithi033 ceph-mon[35332]: pgmap v745: 97 pgs: 97 active+clean; 3.0 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-09-12T02:15:36.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:15:36 smithi016 ceph-mon[30947]: pgmap v745: 97 pgs: 97 active+clean; 3.0 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-09-12T02:15:38.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:15:38 smithi184 ceph-mon[35593]: pgmap v746: 97 pgs: 97 active+clean; 3.0 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 6.2 KiB/s wr, 10 op/s 2024-09-12T02:15:38.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:15:38 smithi033 ceph-mon[35332]: pgmap v746: 97 pgs: 97 active+clean; 3.0 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 6.2 KiB/s wr, 10 op/s 2024-09-12T02:15:38.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:15:38 smithi016 ceph-mon[30947]: pgmap v746: 97 pgs: 97 active+clean; 3.0 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 6.2 KiB/s wr, 10 op/s 2024-09-12T02:15:39.649 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T02:15:39.926 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:15:40.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.413+0000 7f57076ff640 1 -- 172.21.15.16:0/3812525649 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5700102ef0 msgr2=0x7f570010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:40.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.413+0000 7f57076ff640 1 --2- 172.21.15.16:0/3812525649 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5700102ef0 0x7f570010da40 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7f56f8009ca0 tx=0x7f56f802a950 comp rx=0 tx=0).stop 2024-09-12T02:15:40.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.413+0000 7f57076ff640 1 -- 172.21.15.16:0/3812525649 shutdown_connections 2024-09-12T02:15:40.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.413+0000 7f57076ff640 1 --2- 172.21.15.16:0/3812525649 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5700102ef0 0x7f570010da40 unknown :-1 s=CLOSED pgs=1853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:40.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.413+0000 7f57076ff640 1 --2- 172.21.15.16:0/3812525649 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5700102570 0x7f57001029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:40.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.413+0000 7f57076ff640 1 --2- 172.21.15.16:0/3812525649 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5700100e20 0x7f5700101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:40.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.413+0000 7f57076ff640 1 -- 172.21.15.16:0/3812525649 >> 172.21.15.16:0/3812525649 conn(0x7f57000fc7d0 msgr2=0x7f57000febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:15:40.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.413+0000 7f57076ff640 1 -- 172.21.15.16:0/3812525649 shutdown_connections 2024-09-12T02:15:40.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.413+0000 7f57076ff640 1 -- 172.21.15.16:0/3812525649 wait complete. 2024-09-12T02:15:40.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.413+0000 7f57076ff640 1 Processor -- start 2024-09-12T02:15:40.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.414+0000 7f57076ff640 1 -- start start 2024-09-12T02:15:40.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.414+0000 7f57076ff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5700100e20 0x7f5700199370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:40.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.414+0000 7f57076ff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5700102570 0x7f57001998b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:40.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.414+0000 7f57076ff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5700102ef0 0x7f57001a43f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:40.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.414+0000 7f57076ff640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5700110f80 con 0x7f5700102570 2024-09-12T02:15:40.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.414+0000 7f57076ff640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5700110e00 con 0x7f5700100e20 2024-09-12T02:15:40.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.414+0000 7f57076ff640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5700111100 con 0x7f5700102ef0 2024-09-12T02:15:40.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.414+0000 7f5704c9a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5700102570 0x7f57001998b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:40.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.414+0000 7f5704c9a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5700102570 0x7f57001998b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59226/0 (socket says 172.21.15.16:59226) 2024-09-12T02:15:40.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.414+0000 7f5704c9a640 1 -- 172.21.15.16:0/2266439923 learned_addr learned my addr 172.21.15.16:0/2266439923 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:15:40.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.414+0000 7f570549b640 1 --2- 172.21.15.16:0/2266439923 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5700100e20 0x7f5700199370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:40.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.414+0000 7f5705c9c640 1 --2- 172.21.15.16:0/2266439923 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5700102ef0 0x7f57001a43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:40.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.414+0000 7f5704c9a640 1 -- 172.21.15.16:0/2266439923 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5700102ef0 msgr2=0x7f57001a43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:40.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.414+0000 7f5704c9a640 1 --2- 172.21.15.16:0/2266439923 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5700102ef0 0x7f57001a43f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:40.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.414+0000 7f5704c9a640 1 -- 172.21.15.16:0/2266439923 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5700100e20 msgr2=0x7f5700199370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:40.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.414+0000 7f5704c9a640 1 --2- 172.21.15.16:0/2266439923 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5700100e20 0x7f5700199370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:40.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.414+0000 7f5704c9a640 1 -- 172.21.15.16:0/2266439923 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57001a4930 con 0x7f5700102570 2024-09-12T02:15:40.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.415+0000 7f5704c9a640 1 --2- 172.21.15.16:0/2266439923 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5700102570 0x7f57001998b0 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7f56f000c970 tx=0x7f56f000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:15:40.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.415+0000 7f56f67fc640 1 -- 172.21.15.16:0/2266439923 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56f0007bb0 con 0x7f5700102570 2024-09-12T02:15:40.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.415+0000 7f56f67fc640 1 -- 172.21.15.16:0/2266439923 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56f0007d50 con 0x7f5700102570 2024-09-12T02:15:40.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.415+0000 7f57076ff640 1 -- 172.21.15.16:0/2266439923 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57001a4b00 con 0x7f5700102570 2024-09-12T02:15:40.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.415+0000 7f56f67fc640 1 -- 172.21.15.16:0/2266439923 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56f000f450 con 0x7f5700102570 2024-09-12T02:15:40.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.415+0000 7f57076ff640 1 -- 172.21.15.16:0/2266439923 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57001a4fc0 con 0x7f5700102570 2024-09-12T02:15:40.419 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:15:40 smithi016 ceph-mon[30947]: pgmap v747: 97 pgs: 97 active+clean; 3.0 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 6.2 KiB/s wr, 9 op/s 2024-09-12T02:15:40.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.417+0000 7f56f67fc640 1 -- 172.21.15.16:0/2266439923 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f56f0005020 con 0x7f5700102570 2024-09-12T02:15:40.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.418+0000 7f56f67fc640 1 --2- 172.21.15.16:0/2266439923 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f56ec0a2b80 0x7f56ec0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:40.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.418+0000 7f5709ad8740 1 Processor -- start 2024-09-12T02:15:40.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.418+0000 7f5709ad8740 1 -- start start 2024-09-12T02:15:40.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.418+0000 7f570549b640 1 --2- 172.21.15.16:0/2266439923 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f56ec0a2b80 0x7f56ec0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:40.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.418+0000 7f56f67fc640 1 -- 172.21.15.16:0/2266439923 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f56f00cec60 con 0x7f5700102570 2024-09-12T02:15:40.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.419+0000 7f5709ad8740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639f4e51380 0x5639f4e51760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:40.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.419+0000 7f5709ad8740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5639f4e51ca0 0x5639f4e60550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:40.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.419+0000 7f5709ad8740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5639f4e60a90 0x5639f4e62e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:40.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.419+0000 7f5709ad8740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5639f4db72e0 con 0x5639f4e51380 2024-09-12T02:15:40.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.419+0000 7f5709ad8740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5639f4d5ece0 con 0x5639f4e51ca0 2024-09-12T02:15:40.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.419+0000 7f5709ad8740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5639f4db32a0 con 0x5639f4e60a90 2024-09-12T02:15:40.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.419+0000 7f570549b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639f4e51380 0x5639f4e51760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:40.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.419+0000 7f570549b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639f4e51380 0x5639f4e51760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59236/0 (socket says 172.21.15.16:59236) 2024-09-12T02:15:40.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.419+0000 7f570549b640 1 -- 172.21.15.16:0/3298220545 learned_addr learned my addr 172.21.15.16:0/3298220545 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:15:40.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.419+0000 7f5704c9a640 1 --2- 172.21.15.16:0/3298220545 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5639f4e51ca0 0x5639f4e60550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:40.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.419+0000 7f5705c9c640 1 --2- 172.21.15.16:0/3298220545 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5639f4e60a90 0x5639f4e62e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:40.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.419+0000 7f570549b640 1 -- 172.21.15.16:0/3298220545 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5639f4e60a90 msgr2=0x5639f4e62e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:15:40.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.419+0000 7f570549b640 1 --2- 172.21.15.16:0/3298220545 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5639f4e60a90 0x5639f4e62e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:40.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.419+0000 7f570549b640 1 -- 172.21.15.16:0/3298220545 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5639f4e51ca0 msgr2=0x5639f4e60550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:15:40.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.419+0000 7f570549b640 1 --2- 172.21.15.16:0/3298220545 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5639f4e51ca0 0x5639f4e60550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:40.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.419+0000 7f570549b640 1 -- 172.21.15.16:0/3298220545 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639f4d43440 con 0x5639f4e51380 2024-09-12T02:15:40.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.419+0000 7f570549b640 1 --2- 172.21.15.16:0/2266439923 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f56ec0a2b80 0x7f56ec0a5040 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f56fc012500 tx=0x7f56fc012900 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:15:40.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.419+0000 7f570549b640 1 --2- 172.21.15.16:0/3298220545 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639f4e51380 0x5639f4e51760 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7f56fc012e00 tx=0x7f56fc01e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47631cd2da675c2b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:15:40.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.420+0000 7f56eb7fe640 1 -- 172.21.15.16:0/3298220545 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56fc024070 con 0x5639f4e51380 2024-09-12T02:15:40.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.420+0000 7f56eb7fe640 1 -- 172.21.15.16:0/3298220545 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56fc01f450 con 0x5639f4e51380 2024-09-12T02:15:40.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.420+0000 7f56eb7fe640 1 -- 172.21.15.16:0/3298220545 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56fc00f5e0 con 0x5639f4e51380 2024-09-12T02:15:40.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.420+0000 7f5709ad8740 1 -- 172.21.15.16:0/3298220545 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639f4e51380 msgr2=0x5639f4e51760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:40.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.420+0000 7f5709ad8740 1 --2- 172.21.15.16:0/3298220545 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639f4e51380 0x5639f4e51760 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7f56fc012e00 tx=0x7f56fc01e000 comp rx=0 tx=0).stop 2024-09-12T02:15:40.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.420+0000 7f5709ad8740 1 -- 172.21.15.16:0/3298220545 shutdown_connections 2024-09-12T02:15:40.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.420+0000 7f5709ad8740 1 --2- 172.21.15.16:0/3298220545 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5639f4e60a90 0x5639f4e62e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:40.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.420+0000 7f5709ad8740 1 --2- 172.21.15.16:0/3298220545 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5639f4e51ca0 0x5639f4e60550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:40.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.420+0000 7f5709ad8740 1 --2- 172.21.15.16:0/3298220545 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639f4e51380 0x5639f4e51760 unknown :-1 s=CLOSED pgs=1855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:40.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.420+0000 7f5709ad8740 1 -- 172.21.15.16:0/3298220545 >> 172.21.15.16:0/3298220545 conn(0x5639f4e4da30 msgr2=0x5639f4e4de10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:15:40.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.421+0000 7f5709ad8740 1 -- 172.21.15.16:0/3298220545 shutdown_connections 2024-09-12T02:15:40.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.421+0000 7f5709ad8740 1 -- 172.21.15.16:0/3298220545 wait complete. 2024-09-12T02:15:40.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.421+0000 7f5709ad8740 1 -- start start 2024-09-12T02:15:40.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.421+0000 7f5709ad8740 1 Processor -- start 2024-09-12T02:15:40.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.422+0000 7f5709ad8740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639f4e51380 0x5639f4e60440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:40.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.422+0000 7f5709ad8740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5639f4e51ca0 0x5639f4e5d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:40.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.422+0000 7f5709ad8740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5639f4e60a90 0x5639f4e5dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:40.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.422+0000 7f5709ad8740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5639f4e63660 con 0x5639f4e51380 2024-09-12T02:15:40.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.422+0000 7f5709ad8740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5639f4d72050 con 0x5639f4e60a90 2024-09-12T02:15:40.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.422+0000 7f5709ad8740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5639f4e637e0 con 0x5639f4e51ca0 2024-09-12T02:15:40.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.422+0000 7f570549b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639f4e51380 0x5639f4e60440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:40.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.422+0000 7f570549b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639f4e51380 0x5639f4e60440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59246/0 (socket says 172.21.15.16:59246) 2024-09-12T02:15:40.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.422+0000 7f570549b640 1 -- 172.21.15.16:0/1042212337 learned_addr learned my addr 172.21.15.16:0/1042212337 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:15:40.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.422+0000 7f5709ad8740 10 client.0 _ll_register_callbacks cb 0x5639f4db2160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:15:40.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.422+0000 7f5704c9a640 1 --2- 172.21.15.16:0/1042212337 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5639f4e51ca0 0x5639f4e5d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:40.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.422+0000 7f5705c9c640 1 --2- 172.21.15.16:0/1042212337 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5639f4e60a90 0x5639f4e5dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:40.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.422+0000 7f570549b640 1 -- 172.21.15.16:0/1042212337 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5639f4e51ca0 msgr2=0x5639f4e5d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:15:40.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.422+0000 7f570549b640 1 --2- 172.21.15.16:0/1042212337 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5639f4e51ca0 0x5639f4e5d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:40.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.422+0000 7f570549b640 1 -- 172.21.15.16:0/1042212337 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5639f4e60a90 msgr2=0x5639f4e5dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:15:40.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.422+0000 7f570549b640 1 --2- 172.21.15.16:0/1042212337 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5639f4e60a90 0x5639f4e5dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:40.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.422+0000 7f570549b640 1 -- 172.21.15.16:0/1042212337 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5639f4b283c0 con 0x5639f4e51380 2024-09-12T02:15:40.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.422+0000 7f5705c9c640 1 --2- 172.21.15.16:0/1042212337 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5639f4e60a90 0x5639f4e5dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:15:40.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.422+0000 7f570549b640 1 --2- 172.21.15.16:0/1042212337 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639f4e51380 0x5639f4e60440 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7f56fc01fe30 tx=0x7f56fc01f8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2de6e798ed42ef9c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:15:40.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.422+0000 7f56e8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:15:40.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.423+0000 7f56e8ff9640 1 -- 172.21.15.16:0/1042212337 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56fc024040 con 0x5639f4e51380 2024-09-12T02:15:40.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.423+0000 7f56e8ff9640 1 -- 172.21.15.16:0/1042212337 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f56fc01e780 con 0x5639f4e51380 2024-09-12T02:15:40.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.423+0000 7f5709ad8740 1 -- 172.21.15.16:0/1042212337 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f56fc01f450 con 0x5639f4e51380 2024-09-12T02:15:40.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.423+0000 7f56e8ff9640 1 -- 172.21.15.16:0/1042212337 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f56fc01e980 con 0x5639f4e51380 2024-09-12T02:15:40.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.423+0000 7f5709ad8740 10 client.19986 fetch_fsmap learned FSMap version 25 2024-09-12T02:15:40.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.423+0000 7f5709ad8740 1 -- 172.21.15.16:0/1042212337 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5639f4e6f090 con 0x5639f4e51380 2024-09-12T02:15:40.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.423+0000 7f56e8ff9640 1 -- 172.21.15.16:0/1042212337 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56fc01f450 con 0x5639f4e51380 2024-09-12T02:15:40.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.423+0000 7f56e8ff9640 1 -- 172.21.15.16:0/1042212337 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f56fc033020 con 0x5639f4e51380 2024-09-12T02:15:40.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.423+0000 7f5709ad8740 10 client.19986 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:15:40.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.423+0000 7f5709ad8740 10 client.19986 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:15:40.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.423+0000 7f5709ad8740 20 client.19986 populate_metadata read hostname 'smithi016' 2024-09-12T02:15:40.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.423+0000 7f5709ad8740 1 --2- 172.21.15.16:0/1042212337 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5639f4e703e0 0x5639f4e727d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:40.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.424+0000 7f5709ad8740 4 client.19986 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:15:40.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.424+0000 7f5709ad8740 1 -- 172.21.15.16:0/1042212337 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5639f4dc4850 con 0x5639f4e703e0 2024-09-12T02:15:40.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.424+0000 7f5704c9a640 1 --2- 172.21.15.16:0/1042212337 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5639f4e703e0 0x5639f4e727d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:40.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.424+0000 7f5704c9a640 1 --2- 172.21.15.16:0/1042212337 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5639f4e703e0 0x5639f4e727d0 crc :-1 s=READY pgs=409 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84cc0d27cdb957ad server_cookie=88c5d37344655014 in_seq=0 out_seq=0 2024-09-12T02:15:40.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.424+0000 7f56e8ff9640 10 client.19986 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:15:40.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.425+0000 7f56e8ff9640 1 -- 172.21.15.16:0/1042212337 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5639f4dc4850 con 0x5639f4e703e0 2024-09-12T02:15:40.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.425+0000 7f56e8ff9640 10 client.19986 handle_command_reply: tid=0 2024-09-12T02:15:40.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.425+0000 7f5709ad8740 1 client.19986 shutdown 2024-09-12T02:15:40.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.425+0000 7f5709ad8740 1 -- 172.21.15.16:0/1042212337 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639f4e51380 msgr2=0x5639f4e60440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:40.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.425+0000 7f5709ad8740 1 --2- 172.21.15.16:0/1042212337 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639f4e51380 0x5639f4e60440 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7f56fc01fe30 tx=0x7f56fc01f8b0 comp rx=0 tx=0).stop 2024-09-12T02:15:40.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.425+0000 7f5709ad8740 1 -- 172.21.15.16:0/1042212337 shutdown_connections 2024-09-12T02:15:40.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.425+0000 7f5709ad8740 1 --2- 172.21.15.16:0/1042212337 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5639f4e703e0 0x5639f4e727d0 crc :-1 s=READY pgs=409 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:40.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.425+0000 7f5709ad8740 1 --2- 172.21.15.16:0/1042212337 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5639f4e60a90 0x5639f4e5dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:40.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.425+0000 7f5709ad8740 1 --2- 172.21.15.16:0/1042212337 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5639f4e51ca0 0x5639f4e5d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:40.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.425+0000 7f5709ad8740 1 --2- 172.21.15.16:0/1042212337 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639f4e51380 0x5639f4e60440 unknown :-1 s=CLOSED pgs=1856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:40.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.425+0000 7f56e8ff9640 0 client.19986 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:15:40.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.425+0000 7f5709ad8740 1 -- 172.21.15.16:0/1042212337 >> 172.21.15.16:0/1042212337 conn(0x5639f4e4da30 msgr2=0x5639f4e53790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:15:40.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.425+0000 7f5709ad8740 1 -- 172.21.15.16:0/1042212337 shutdown_connections 2024-09-12T02:15:40.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.425+0000 7f5709ad8740 1 -- 172.21.15.16:0/1042212337 wait complete. 2024-09-12T02:15:40.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.426+0000 7f5709ad8740 20 client.19986 trim_cache size 0 max 16384 2024-09-12T02:15:40.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.443+0000 7f5709ad8740 1 Processor -- start 2024-09-12T02:15:40.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.443+0000 7f5709ad8740 1 -- start start 2024-09-12T02:15:40.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.443+0000 7f5709ad8740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639f4e51380 0x5639f4e6c900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:40.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.443+0000 7f5709ad8740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5639f4e51ca0 0x5639f4e77dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:40.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.443+0000 7f5709ad8740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5639f4e60a90 0x5639f4e78310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:40.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.443+0000 7f5709ad8740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f57001adb60 con 0x5639f4e51380 2024-09-12T02:15:40.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.443+0000 7f5709ad8740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f57001ab430 con 0x5639f4e60a90 2024-09-12T02:15:40.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.443+0000 7f5709ad8740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f57001a92c0 con 0x5639f4e51ca0 2024-09-12T02:15:40.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.443+0000 7f570549b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639f4e51380 0x5639f4e6c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:40.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.443+0000 7f570549b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639f4e51380 0x5639f4e6c900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59256/0 (socket says 172.21.15.16:59256) 2024-09-12T02:15:40.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.443+0000 7f570549b640 1 -- 172.21.15.16:0/774047000 learned_addr learned my addr 172.21.15.16:0/774047000 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:15:40.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.443+0000 7f570549b640 1 -- 172.21.15.16:0/774047000 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5639f4e51ca0 msgr2=0x5639f4e77dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:15:40.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.443+0000 7f570549b640 1 --2- 172.21.15.16:0/774047000 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5639f4e51ca0 0x5639f4e77dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:40.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.443+0000 7f570549b640 1 -- 172.21.15.16:0/774047000 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5639f4e60a90 msgr2=0x5639f4e78310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:15:40.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.443+0000 7f5705c9c640 1 --2- 172.21.15.16:0/774047000 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5639f4e60a90 0x5639f4e78310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:40.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.443+0000 7f570549b640 1 --2- 172.21.15.16:0/774047000 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5639f4e60a90 0x5639f4e78310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:40.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.443+0000 7f570549b640 1 -- 172.21.15.16:0/774047000 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639f4e51ad0 con 0x5639f4e51380 2024-09-12T02:15:40.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.443+0000 7f5705c9c640 1 --2- 172.21.15.16:0/774047000 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5639f4e60a90 0x5639f4e78310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:15:40.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.444+0000 7f570549b640 1 --2- 172.21.15.16:0/774047000 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639f4e51380 0x5639f4e6c900 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7f56fc014010 tx=0x7f56fc00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89a18e027460acf1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:15:40.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.444+0000 7f56eaffd640 1 -- 172.21.15.16:0/774047000 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56fc024070 con 0x5639f4e51380 2024-09-12T02:15:40.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.444+0000 7f56eaffd640 1 -- 172.21.15.16:0/774047000 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56fc01f450 con 0x5639f4e51380 2024-09-12T02:15:40.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.444+0000 7f56eaffd640 1 -- 172.21.15.16:0/774047000 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56fc01fe60 con 0x5639f4e51380 2024-09-12T02:15:40.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.444+0000 7f5709ad8740 1 -- 172.21.15.16:0/774047000 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639f4e51380 msgr2=0x5639f4e6c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:40.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.444+0000 7f5709ad8740 1 --2- 172.21.15.16:0/774047000 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639f4e51380 0x5639f4e6c900 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7f56fc014010 tx=0x7f56fc00ee80 comp rx=0 tx=0).stop 2024-09-12T02:15:40.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.444+0000 7f5709ad8740 1 -- 172.21.15.16:0/774047000 shutdown_connections 2024-09-12T02:15:40.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.444+0000 7f5709ad8740 1 --2- 172.21.15.16:0/774047000 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5639f4e60a90 0x5639f4e78310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:40.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.444+0000 7f5709ad8740 1 --2- 172.21.15.16:0/774047000 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5639f4e51ca0 0x5639f4e77dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:40.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.444+0000 7f5709ad8740 1 --2- 172.21.15.16:0/774047000 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639f4e51380 0x5639f4e6c900 unknown :-1 s=CLOSED pgs=1857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:40.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.444+0000 7f5709ad8740 1 -- 172.21.15.16:0/774047000 >> 172.21.15.16:0/774047000 conn(0x5639f4e4da30 msgr2=0x5639f4e5c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:15:40.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.445+0000 7f5709ad8740 1 -- 172.21.15.16:0/774047000 shutdown_connections 2024-09-12T02:15:40.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.445+0000 7f5709ad8740 1 -- 172.21.15.16:0/774047000 wait complete. 2024-09-12T02:15:40.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.445+0000 7f5709ad8740 1 -- start start 2024-09-12T02:15:40.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.445+0000 7f5709ad8740 1 Processor -- start 2024-09-12T02:15:40.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.445+0000 7f5709ad8740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639f4e51380 0x5639f4e63fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:40.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.445+0000 7f5709ad8740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5639f4e51ca0 0x5639f4e64510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:40.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.445+0000 7f5709ad8740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5639f4e60a90 0x5639f4ed8d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:40.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.445+0000 7f5709ad8740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f57001a9910 con 0x5639f4e51380 2024-09-12T02:15:40.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.445+0000 7f5709ad8740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f57001aad60 con 0x5639f4e60a90 2024-09-12T02:15:40.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.445+0000 7f5709ad8740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f57001a9fc0 con 0x5639f4e51ca0 2024-09-12T02:15:40.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.446+0000 7f570549b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639f4e51380 0x5639f4e63fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:40.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.446+0000 7f5709ad8740 10 client.0 _ll_register_callbacks cb 0x7f56fc022070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:15:40.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.446+0000 7f570549b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639f4e51380 0x5639f4e63fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59272/0 (socket says 172.21.15.16:59272) 2024-09-12T02:15:40.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.446+0000 7f570549b640 1 -- 172.21.15.16:0/2160861820 learned_addr learned my addr 172.21.15.16:0/2160861820 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:15:40.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.446+0000 7f5705c9c640 1 --2- 172.21.15.16:0/2160861820 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5639f4e60a90 0x5639f4ed8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:40.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.446+0000 7f5704c9a640 1 --2- 172.21.15.16:0/2160861820 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5639f4e51ca0 0x5639f4e64510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:40.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.446+0000 7f570549b640 1 -- 172.21.15.16:0/2160861820 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5639f4e51ca0 msgr2=0x5639f4e64510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:15:40.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.446+0000 7f570549b640 1 --2- 172.21.15.16:0/2160861820 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5639f4e51ca0 0x5639f4e64510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:40.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.446+0000 7f570549b640 1 -- 172.21.15.16:0/2160861820 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5639f4e60a90 msgr2=0x5639f4ed8d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:15:40.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.446+0000 7f570549b640 1 --2- 172.21.15.16:0/2160861820 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5639f4e60a90 0x5639f4ed8d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:40.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.446+0000 7f570549b640 1 -- 172.21.15.16:0/2160861820 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5639f4ed93a0 con 0x5639f4e51380 2024-09-12T02:15:40.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.446+0000 7f5705c9c640 1 --2- 172.21.15.16:0/2160861820 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5639f4e60a90 0x5639f4ed8d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:15:40.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.446+0000 7f570549b640 1 --2- 172.21.15.16:0/2160861820 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639f4e51380 0x5639f4e63fd0 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7f56fc005d00 tx=0x7f56fc01fe30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f36ec68b0da25104 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:15:40.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.446+0000 7f56e8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:15:40.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.446+0000 7f56e8ff9640 1 -- 172.21.15.16:0/2160861820 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56fc024040 con 0x5639f4e51380 2024-09-12T02:15:40.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.446+0000 7f5709ad8740 1 -- 172.21.15.16:0/2160861820 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f56fc01f450 con 0x5639f4e51380 2024-09-12T02:15:40.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.446+0000 7f56e8ff9640 1 -- 172.21.15.16:0/2160861820 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f56fc01e780 con 0x5639f4e51380 2024-09-12T02:15:40.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.446+0000 7f56e8ff9640 1 -- 172.21.15.16:0/2160861820 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f56fc035440 con 0x5639f4e51380 2024-09-12T02:15:40.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.446+0000 7f5709ad8740 10 client.19992 fetch_fsmap learned FSMap version 25 2024-09-12T02:15:40.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.447+0000 7f5709ad8740 1 -- 172.21.15.16:0/2160861820 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5639f4eda5e0 con 0x5639f4e51380 2024-09-12T02:15:40.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.447+0000 7f56e8ff9640 1 -- 172.21.15.16:0/2160861820 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56fc01f450 con 0x5639f4e51380 2024-09-12T02:15:40.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.447+0000 7f56e8ff9640 1 -- 172.21.15.16:0/2160861820 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f56fc04f020 con 0x5639f4e51380 2024-09-12T02:15:40.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.447+0000 7f5709ad8740 10 client.19992 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:15:40.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.447+0000 7f5709ad8740 10 client.19992 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:15:40.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.447+0000 7f5709ad8740 20 client.19992 populate_metadata read hostname 'smithi016' 2024-09-12T02:15:40.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.447+0000 7f5709ad8740 1 --2- 172.21.15.16:0/2160861820 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5639f4edabf0 0x5639f4edcfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:40.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.447+0000 7f5709ad8740 4 client.19992 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T02:15:40.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.447+0000 7f5709ad8740 1 -- 172.21.15.16:0/2160861820 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5639f4dbed60 con 0x5639f4edabf0 2024-09-12T02:15:40.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.447+0000 7f5704c9a640 1 --2- 172.21.15.16:0/2160861820 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5639f4edabf0 0x5639f4edcfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:40.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.448+0000 7f56e8ff9640 10 client.19992 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:15:40.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.448+0000 7f5704c9a640 1 --2- 172.21.15.16:0/2160861820 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5639f4edabf0 0x5639f4edcfe0 crc :-1 s=READY pgs=410 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17e4a661693c56d8 server_cookie=5ad1ff8f367bb1bd in_seq=0 out_seq=0 2024-09-12T02:15:40.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.448+0000 7f56e8ff9640 1 -- 172.21.15.16:0/2160861820 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5639f4dbed60 con 0x5639f4edabf0 2024-09-12T02:15:40.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.448+0000 7f56e8ff9640 10 client.19992 handle_command_reply: tid=0 2024-09-12T02:15:40.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.448+0000 7f5709ad8740 1 client.19992 shutdown 2024-09-12T02:15:40.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.448+0000 7f5709ad8740 1 -- 172.21.15.16:0/2160861820 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639f4e51380 msgr2=0x5639f4e63fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:40.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.448+0000 7f5709ad8740 1 --2- 172.21.15.16:0/2160861820 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639f4e51380 0x5639f4e63fd0 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7f56fc005d00 tx=0x7f56fc01fe30 comp rx=0 tx=0).stop 2024-09-12T02:15:40.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.448+0000 7f5709ad8740 1 -- 172.21.15.16:0/2160861820 shutdown_connections 2024-09-12T02:15:40.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.448+0000 7f5709ad8740 1 --2- 172.21.15.16:0/2160861820 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5639f4edabf0 0x5639f4edcfe0 crc :-1 s=READY pgs=410 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:40.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.448+0000 7f5709ad8740 1 --2- 172.21.15.16:0/2160861820 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5639f4e60a90 0x5639f4ed8d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:40.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.448+0000 7f5709ad8740 1 --2- 172.21.15.16:0/2160861820 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5639f4e51ca0 0x5639f4e64510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:40.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.448+0000 7f56e8ff9640 0 client.19992 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:15:40.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.448+0000 7f5709ad8740 1 --2- 172.21.15.16:0/2160861820 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639f4e51380 0x5639f4e63fd0 unknown :-1 s=CLOSED pgs=1858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:40.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.448+0000 7f5709ad8740 1 -- 172.21.15.16:0/2160861820 >> 172.21.15.16:0/2160861820 conn(0x5639f4e4da30 msgr2=0x5639f4e5bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:15:40.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.448+0000 7f5709ad8740 1 -- 172.21.15.16:0/2160861820 shutdown_connections 2024-09-12T02:15:40.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.448+0000 7f5709ad8740 1 -- 172.21.15.16:0/2160861820 wait complete. 2024-09-12T02:15:40.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.448+0000 7f5709ad8740 20 client.19992 trim_cache size 0 max 16384 2024-09-12T02:15:40.455 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T02:15:40.455 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-09-12T02:15:40.455 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {} 2024-09-12T02:15:40.455 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T02:15:40.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.449+0000 7f57076ff640 1 -- 172.21.15.16:0/2266439923 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f56ec0a2b80 msgr2=0x7f56ec0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:40.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.449+0000 7f57076ff640 1 --2- 172.21.15.16:0/2266439923 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f56ec0a2b80 0x7f56ec0a5040 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f56fc012500 tx=0x7f56fc012900 comp rx=0 tx=0).stop 2024-09-12T02:15:40.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.449+0000 7f57076ff640 1 -- 172.21.15.16:0/2266439923 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5700102570 msgr2=0x7f57001998b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:40.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.449+0000 7f57076ff640 1 --2- 172.21.15.16:0/2266439923 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5700102570 0x7f57001998b0 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7f56f000c970 tx=0x7f56f000ce30 comp rx=0 tx=0).stop 2024-09-12T02:15:40.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.449+0000 7f57076ff640 1 -- 172.21.15.16:0/2266439923 shutdown_connections 2024-09-12T02:15:40.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.449+0000 7f57076ff640 1 --2- 172.21.15.16:0/2266439923 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f56ec0a2b80 0x7f56ec0a5040 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:40.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.449+0000 7f57076ff640 1 --2- 172.21.15.16:0/2266439923 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5700102ef0 0x7f57001a43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:40.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.449+0000 7f57076ff640 1 --2- 172.21.15.16:0/2266439923 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5700102570 0x7f57001998b0 unknown :-1 s=CLOSED pgs=1854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:40.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.449+0000 7f57076ff640 1 --2- 172.21.15.16:0/2266439923 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5700100e20 0x7f5700199370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:40.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.449+0000 7f57076ff640 1 -- 172.21.15.16:0/2266439923 >> 172.21.15.16:0/2266439923 conn(0x7f57000fc7d0 msgr2=0x7f5700106880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:15:40.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.449+0000 7f57076ff640 1 -- 172.21.15.16:0/2266439923 shutdown_connections 2024-09-12T02:15:40.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:40.449+0000 7f57076ff640 1 -- 172.21.15.16:0/2266439923 wait complete. 2024-09-12T02:15:40.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:15:40 smithi184 ceph-mon[35593]: pgmap v747: 97 pgs: 97 active+clean; 3.0 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 6.2 KiB/s wr, 9 op/s 2024-09-12T02:15:40.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:15:40 smithi033 ceph-mon[35332]: pgmap v747: 97 pgs: 97 active+clean; 3.0 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 6.2 KiB/s wr, 9 op/s 2024-09-12T02:15:41.055 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-12T02:15:41.055 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:15:41.313 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:15:41.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:41.802+0000 7fc6c4b72640 1 -- 172.21.15.16:0/909979123 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6c010c170 msgr2=0x7fc6c010e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:41.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:41.802+0000 7fc6c4b72640 1 --2- 172.21.15.16:0/909979123 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6c010c170 0x7fc6c010e610 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7fc6b8009ca0 tx=0x7fc6b802a950 comp rx=0 tx=0).stop 2024-09-12T02:15:41.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:41.802+0000 7fc6c4b72640 1 -- 172.21.15.16:0/909979123 shutdown_connections 2024-09-12T02:15:41.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:41.802+0000 7fc6c4b72640 1 --2- 172.21.15.16:0/909979123 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6c010c170 0x7fc6c010e610 unknown :-1 s=CLOSED pgs=1859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:41.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:41.802+0000 7fc6c4b72640 1 --2- 172.21.15.16:0/909979123 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc6c0103660 0x7fc6c010bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:41.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:41.802+0000 7fc6c4b72640 1 --2- 172.21.15.16:0/909979123 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc6c0102d40 0x7fc6c0103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:41.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:41.802+0000 7fc6c4b72640 1 -- 172.21.15.16:0/909979123 >> 172.21.15.16:0/909979123 conn(0x7fc6c00fc7a0 msgr2=0x7fc6c00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:15:41.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:41.802+0000 7fc6c4b72640 1 -- 172.21.15.16:0/909979123 shutdown_connections 2024-09-12T02:15:41.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:41.802+0000 7fc6c4b72640 1 -- 172.21.15.16:0/909979123 wait complete. 2024-09-12T02:15:41.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:41.803+0000 7fc6c4b72640 1 Processor -- start 2024-09-12T02:15:41.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:41.803+0000 7fc6c4b72640 1 -- start start 2024-09-12T02:15:41.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:41.803+0000 7fc6c4b72640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6c0102d40 0x7fc6c01009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:41.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:41.803+0000 7fc6c4b72640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc6c0103660 0x7fc6c0100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:41.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:41.803+0000 7fc6c4b72640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc6c010c170 0x7fc6c0101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:41.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:41.803+0000 7fc6c4b72640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc6c0111f70 con 0x7fc6c0102d40 2024-09-12T02:15:41.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:41.803+0000 7fc6c4b72640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc6c0111df0 con 0x7fc6c0103660 2024-09-12T02:15:41.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:41.803+0000 7fc6c4b72640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc6c01120f0 con 0x7fc6c010c170 2024-09-12T02:15:41.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:41.803+0000 7fc6be59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6c0102d40 0x7fc6c01009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:41.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:41.803+0000 7fc6be59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6c0102d40 0x7fc6c01009a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53494/0 (socket says 172.21.15.16:53494) 2024-09-12T02:15:41.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:41.803+0000 7fc6be59c640 1 -- 172.21.15.16:0/2720550639 learned_addr learned my addr 172.21.15.16:0/2720550639 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:15:41.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:41.804+0000 7fc6bdd9b640 1 --2- 172.21.15.16:0/2720550639 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc6c0103660 0x7fc6c0100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:41.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:41.804+0000 7fc6bed9d640 1 --2- 172.21.15.16:0/2720550639 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc6c010c170 0x7fc6c0101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:41.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:41.804+0000 7fc6be59c640 1 -- 172.21.15.16:0/2720550639 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc6c010c170 msgr2=0x7fc6c0101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:41.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:41.804+0000 7fc6be59c640 1 --2- 172.21.15.16:0/2720550639 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc6c010c170 0x7fc6c0101420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:41.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:41.804+0000 7fc6be59c640 1 -- 172.21.15.16:0/2720550639 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc6c0103660 msgr2=0x7fc6c0100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:41.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:41.804+0000 7fc6be59c640 1 --2- 172.21.15.16:0/2720550639 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc6c0103660 0x7fc6c0100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:41.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:41.804+0000 7fc6be59c640 1 -- 172.21.15.16:0/2720550639 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6c01a6830 con 0x7fc6c0102d40 2024-09-12T02:15:41.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:41.804+0000 7fc6bed9d640 1 --2- 172.21.15.16:0/2720550639 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc6c010c170 0x7fc6c0101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:15:41.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:41.804+0000 7fc6be59c640 1 --2- 172.21.15.16:0/2720550639 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6c0102d40 0x7fc6c01009a0 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7fc6b000c920 tx=0x7fc6b000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:15:41.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:41.804+0000 7fc6b77fe640 1 -- 172.21.15.16:0/2720550639 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6b0007c70 con 0x7fc6c0102d40 2024-09-12T02:15:41.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:41.804+0000 7fc6c4b72640 1 -- 172.21.15.16:0/2720550639 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6c01a6b20 con 0x7fc6c0102d40 2024-09-12T02:15:41.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:41.804+0000 7fc6b77fe640 1 -- 172.21.15.16:0/2720550639 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc6b0007e10 con 0x7fc6c0102d40 2024-09-12T02:15:41.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:41.804+0000 7fc6b77fe640 1 -- 172.21.15.16:0/2720550639 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6b0018890 con 0x7fc6c0102d40 2024-09-12T02:15:41.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:41.804+0000 7fc6c4b72640 1 -- 172.21.15.16:0/2720550639 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6c01a70b0 con 0x7fc6c0102d40 2024-09-12T02:15:41.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:41.805+0000 7fc6c4b72640 1 -- 172.21.15.16:0/2720550639 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6c0103120 con 0x7fc6c0102d40 2024-09-12T02:15:41.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:41.807+0000 7fc6b77fe640 1 -- 172.21.15.16:0/2720550639 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fc6b0018a50 con 0x7fc6c0102d40 2024-09-12T02:15:41.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:41.807+0000 7fc6b77fe640 1 --2- 172.21.15.16:0/2720550639 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc6ac0a2b80 0x7fc6ac0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:41.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:41.808+0000 7fc6bdd9b640 1 --2- 172.21.15.16:0/2720550639 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc6ac0a2b80 0x7fc6ac0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:41.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:41.808+0000 7fc6b77fe640 1 -- 172.21.15.16:0/2720550639 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fc6b00cf380 con 0x7fc6c0102d40 2024-09-12T02:15:41.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:41.808+0000 7fc6bdd9b640 1 --2- 172.21.15.16:0/2720550639 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc6ac0a2b80 0x7fc6ac0a5040 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fc6c0102430 tx=0x7fc6a800a250 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:15:41.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:41.810+0000 7fc6b77fe640 1 -- 172.21.15.16:0/2720550639 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc6b0014030 con 0x7fc6c0102d40 2024-09-12T02:15:42.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:42.024+0000 7fc6c4b72640 1 -- 172.21.15.16:0/2720550639 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc6c01a72e0 con 0x7fc6c0102d40 2024-09-12T02:15:42.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:42.025+0000 7fc6b77fe640 1 -- 172.21.15.16:0/2720550639 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11911 (secure 0 0 0) 0x7fc6b009c410 con 0x7fc6c0102d40 2024-09-12T02:15:42.026 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T02:15:42.026 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":25,"btime":"2024-09-12T02:01:37:192021+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3279123279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3279123279},{"type":"v1","addr":"172.21.15.16:6835","nonce":3279123279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4119452382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4119452382},{"type":"v1","addr":"172.21.15.16:6837","nonce":4119452382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/3263823165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3263823165},{"type":"v1","addr":"172.21.15.16:6839","nonce":3263823165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/1510290732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":1510290732},{"type":"v1","addr":"172.21.15.16:6841","nonce":1510290732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/279674603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":279674603},{"type":"v1","addr":"172.21.15.33:6833","nonce":279674603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3555728028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3555728028},{"type":"v1","addr":"172.21.15.33:6835","nonce":3555728028}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T01:59:56.117608+0000","modified":"2024-09-12T02:01:36.294760+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24482,"mds_1":24481,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6833/1563374106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1563374106},{"type":"v1","addr":"172.21.15.184:6833","nonce":1563374106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6837/1892833868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1892833868},{"type":"v1","addr":"172.21.15.33:6837","nonce":1892833868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/1457679686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1457679686},{"type":"v1","addr":"172.21.15.184:6835","nonce":1457679686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.184:6837/1793393087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":1793393087},{"type":"v1","addr":"172.21.15.184:6837","nonce":1793393087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.184:6839/4203347941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":4203347941},{"type":"v1","addr":"172.21.15.184:6839","nonce":4203347941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.33:6839/4136670204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":4136670204},{"type":"v1","addr":"172.21.15.33:6839","nonce":4136670204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24481,24482,24469]},"id":1}]} 2024-09-12T02:15:42.027 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 25 2024-09-12T02:15:42.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:42.027+0000 7fc6c4b72640 1 -- 172.21.15.16:0/2720550639 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc6ac0a2b80 msgr2=0x7fc6ac0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:42.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:42.027+0000 7fc6c4b72640 1 --2- 172.21.15.16:0/2720550639 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc6ac0a2b80 0x7fc6ac0a5040 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fc6c0102430 tx=0x7fc6a800a250 comp rx=0 tx=0).stop 2024-09-12T02:15:42.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:42.027+0000 7fc6c4b72640 1 -- 172.21.15.16:0/2720550639 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6c0102d40 msgr2=0x7fc6c01009a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:42.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:42.027+0000 7fc6c4b72640 1 --2- 172.21.15.16:0/2720550639 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6c0102d40 0x7fc6c01009a0 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7fc6b000c920 tx=0x7fc6b000cde0 comp rx=0 tx=0).stop 2024-09-12T02:15:42.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:42.027+0000 7fc6c4b72640 1 -- 172.21.15.16:0/2720550639 shutdown_connections 2024-09-12T02:15:42.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:42.027+0000 7fc6c4b72640 1 --2- 172.21.15.16:0/2720550639 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc6ac0a2b80 0x7fc6ac0a5040 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:42.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:42.027+0000 7fc6c4b72640 1 --2- 172.21.15.16:0/2720550639 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc6c010c170 0x7fc6c0101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:42.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:42.027+0000 7fc6c4b72640 1 --2- 172.21.15.16:0/2720550639 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc6c0103660 0x7fc6c0100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:42.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:42.027+0000 7fc6c4b72640 1 --2- 172.21.15.16:0/2720550639 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6c0102d40 0x7fc6c01009a0 unknown :-1 s=CLOSED pgs=1860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:42.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:42.027+0000 7fc6c4b72640 1 -- 172.21.15.16:0/2720550639 >> 172.21.15.16:0/2720550639 conn(0x7fc6c00fc7a0 msgr2=0x7fc6c00fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:15:42.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:42.027+0000 7fc6c4b72640 1 -- 172.21.15.16:0/2720550639 shutdown_connections 2024-09-12T02:15:42.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:42.027+0000 7fc6c4b72640 1 -- 172.21.15.16:0/2720550639 wait complete. 2024-09-12T02:15:42.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:15:42 smithi184 ceph-mon[35593]: pgmap v748: 97 pgs: 97 active+clean; 3.0 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.2 KiB/s wr, 10 op/s 2024-09-12T02:15:42.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:15:42 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/2720550639' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:15:42.634 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:15:42 smithi016 ceph-mon[30947]: pgmap v748: 97 pgs: 97 active+clean; 3.0 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.2 KiB/s wr, 10 op/s 2024-09-12T02:15:42.635 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:15:42 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/2720550639' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:15:42.664 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-12T02:15:42.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:15:42 smithi033 ceph-mon[35332]: pgmap v748: 97 pgs: 97 active+clean; 3.0 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.2 KiB/s wr, 10 op/s 2024-09-12T02:15:42.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:15:42 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/2720550639' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:15:42.921 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:15:43.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.362+0000 7f6b9b171640 1 -- 172.21.15.16:0/548413749 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b94102db0 msgr2=0x7f6b94103190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:43.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.362+0000 7f6b9b171640 1 --2- 172.21.15.16:0/548413749 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b94102db0 0x7f6b94103190 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7f6b90009930 tx=0x7f6b9002a420 comp rx=0 tx=0).stop 2024-09-12T02:15:43.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.362+0000 7f6b9b171640 1 -- 172.21.15.16:0/548413749 shutdown_connections 2024-09-12T02:15:43.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.362+0000 7f6b9b171640 1 --2- 172.21.15.16:0/548413749 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6b941082e0 0x7f6b9410ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:43.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.362+0000 7f6b9b171640 1 --2- 172.21.15.16:0/548413749 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b94103760 0x7f6b94107ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:43.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.362+0000 7f6b9b171640 1 --2- 172.21.15.16:0/548413749 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b94102db0 0x7f6b94103190 unknown :-1 s=CLOSED pgs=1861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:43.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.362+0000 7f6b9b171640 1 -- 172.21.15.16:0/548413749 >> 172.21.15.16:0/548413749 conn(0x7f6b940fc7d0 msgr2=0x7f6b940febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:15:43.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.362+0000 7f6b9b171640 1 -- 172.21.15.16:0/548413749 shutdown_connections 2024-09-12T02:15:43.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.362+0000 7f6b9b171640 1 -- 172.21.15.16:0/548413749 wait complete. 2024-09-12T02:15:43.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.363+0000 7f6b9b171640 1 Processor -- start 2024-09-12T02:15:43.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.363+0000 7f6b9b171640 1 -- start start 2024-09-12T02:15:43.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.363+0000 7f6b9b171640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b94102db0 0x7f6b941985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:43.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.363+0000 7f6b9b171640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6b94103760 0x7f6b94198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:43.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.363+0000 7f6b9b171640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b941082e0 0x7f6b9419ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:43.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.363+0000 7f6b9b171640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6b94112090 con 0x7f6b94102db0 2024-09-12T02:15:43.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.363+0000 7f6b9b171640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6b94111f10 con 0x7f6b941082e0 2024-09-12T02:15:43.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.363+0000 7f6b9b171640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f6b94112210 con 0x7f6b94103760 2024-09-12T02:15:43.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.363+0000 7f6b98f0d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b94102db0 0x7f6b941985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:43.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.363+0000 7f6b98f0d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b94102db0 0x7f6b941985c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53512/0 (socket says 172.21.15.16:53512) 2024-09-12T02:15:43.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.363+0000 7f6b98f0d640 1 -- 172.21.15.16:0/1032539227 learned_addr learned my addr 172.21.15.16:0/1032539227 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:15:43.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.364+0000 7f6b8bfff640 1 --2- 172.21.15.16:0/1032539227 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6b94103760 0x7f6b94198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:43.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.364+0000 7f6b9970e640 1 --2- 172.21.15.16:0/1032539227 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b941082e0 0x7f6b9419ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:43.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.364+0000 7f6b98f0d640 1 -- 172.21.15.16:0/1032539227 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6b94103760 msgr2=0x7f6b94198b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:43.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.364+0000 7f6b98f0d640 1 --2- 172.21.15.16:0/1032539227 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6b94103760 0x7f6b94198b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:43.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.364+0000 7f6b98f0d640 1 -- 172.21.15.16:0/1032539227 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b941082e0 msgr2=0x7f6b9419ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:43.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.364+0000 7f6b98f0d640 1 --2- 172.21.15.16:0/1032539227 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b941082e0 0x7f6b9419ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:43.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.364+0000 7f6b98f0d640 1 -- 172.21.15.16:0/1032539227 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b9419d570 con 0x7f6b94102db0 2024-09-12T02:15:43.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.364+0000 7f6b9970e640 1 --2- 172.21.15.16:0/1032539227 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b941082e0 0x7f6b9419ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:15:43.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.364+0000 7f6b98f0d640 1 --2- 172.21.15.16:0/1032539227 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b94102db0 0x7f6b941985c0 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7f6b90009230 tx=0x7f6b90003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:15:43.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.364+0000 7f6b89ffb640 1 -- 172.21.15.16:0/1032539227 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b9003f070 con 0x7f6b94102db0 2024-09-12T02:15:43.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.364+0000 7f6b9b171640 1 -- 172.21.15.16:0/1032539227 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b9419d800 con 0x7f6b94102db0 2024-09-12T02:15:43.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.364+0000 7f6b89ffb640 1 -- 172.21.15.16:0/1032539227 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b90031e30 con 0x7f6b94102db0 2024-09-12T02:15:43.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.364+0000 7f6b89ffb640 1 -- 172.21.15.16:0/1032539227 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b90035030 con 0x7f6b94102db0 2024-09-12T02:15:43.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.364+0000 7f6b9b171640 1 -- 172.21.15.16:0/1032539227 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b941a5140 con 0x7f6b94102db0 2024-09-12T02:15:43.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.367+0000 7f6b89ffb640 1 -- 172.21.15.16:0/1032539227 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6b90042070 con 0x7f6b94102db0 2024-09-12T02:15:43.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.367+0000 7f6b9d54a740 1 Processor -- start 2024-09-12T02:15:43.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.367+0000 7f6b9d54a740 1 -- start start 2024-09-12T02:15:43.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.367+0000 7f6b9d54a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee3c7dcb40 0x55ee3c7db090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:43.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.367+0000 7f6b9d54a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ee3c7db5d0 0x55ee3c7dba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:43.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.367+0000 7f6b9d54a740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ee3c7e6370 0x55ee3c7e8760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:43.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.367+0000 7f6b9d54a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55ee3c7412f0 con 0x55ee3c7dcb40 2024-09-12T02:15:43.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.367+0000 7f6b9d54a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55ee3c6e8cf0 con 0x55ee3c7db5d0 2024-09-12T02:15:43.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.367+0000 7f6b9d54a740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55ee3c73d2b0 con 0x55ee3c7e6370 2024-09-12T02:15:43.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.368+0000 7f6b89ffb640 1 --2- 172.21.15.16:0/1032539227 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f6b7c0a2b80 0x7f6b7c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:43.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.368+0000 7f6b9970e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee3c7dcb40 0x55ee3c7db090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:43.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.368+0000 7f6b9970e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee3c7dcb40 0x55ee3c7db090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53516/0 (socket says 172.21.15.16:53516) 2024-09-12T02:15:43.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.368+0000 7f6b9970e640 1 -- 172.21.15.16:0/1791309004 learned_addr learned my addr 172.21.15.16:0/1791309004 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:15:43.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.368+0000 7f6b8bfff640 1 --2- 172.21.15.16:0/1791309004 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ee3c7e6370 0x55ee3c7e8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:43.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.368+0000 7f6b89ffb640 1 -- 172.21.15.16:0/1032539227 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f6b900efa40 con 0x7f6b94102db0 2024-09-12T02:15:43.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.368+0000 7f6b98f0d640 1 --2- 172.21.15.16:0/1791309004 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ee3c7db5d0 0x55ee3c7dba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:43.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.368+0000 7f6b9970e640 1 --2- 172.21.15.16:0/1032539227 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f6b7c0a2b80 0x7f6b7c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:43.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.368+0000 7f6b9970e640 1 -- 172.21.15.16:0/1791309004 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ee3c7e6370 msgr2=0x55ee3c7e8760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:15:43.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.368+0000 7f6b9970e640 1 --2- 172.21.15.16:0/1791309004 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ee3c7e6370 0x55ee3c7e8760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:43.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.368+0000 7f6b9970e640 1 -- 172.21.15.16:0/1791309004 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ee3c7db5d0 msgr2=0x55ee3c7dba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:15:43.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.368+0000 7f6b9970e640 1 --2- 172.21.15.16:0/1791309004 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ee3c7db5d0 0x55ee3c7dba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:43.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.368+0000 7f6b9970e640 1 -- 172.21.15.16:0/1791309004 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee3c6cd3a0 con 0x55ee3c7dcb40 2024-09-12T02:15:43.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.368+0000 7f6b8bfff640 1 --2- 172.21.15.16:0/1791309004 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ee3c7e6370 0x55ee3c7e8760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:15:43.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.368+0000 7f6b9970e640 1 --2- 172.21.15.16:0/1791309004 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee3c7dcb40 0x55ee3c7db090 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c014930 tx=0x7f6b8c014df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5242f7ca8c299b03 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:15:43.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.368+0000 7f6b7affd640 1 -- 172.21.15.16:0/1791309004 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b8c021070 con 0x55ee3c7dcb40 2024-09-12T02:15:43.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.369+0000 7f6b7affd640 1 -- 172.21.15.16:0/1791309004 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b8c008df0 con 0x55ee3c7dcb40 2024-09-12T02:15:43.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.369+0000 7f6b9970e640 1 --2- 172.21.15.16:0/1032539227 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f6b7c0a2b80 0x7f6b7c0a5040 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c01e000 tx=0x7f6b8c01a000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:15:43.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.369+0000 7f6b9d54a740 1 -- 172.21.15.16:0/1791309004 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee3c7dcb40 msgr2=0x55ee3c7db090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:43.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.369+0000 7f6b7affd640 1 -- 172.21.15.16:0/1791309004 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b8c013cc0 con 0x55ee3c7dcb40 2024-09-12T02:15:43.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.369+0000 7f6b9d54a740 1 --2- 172.21.15.16:0/1791309004 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee3c7dcb40 0x55ee3c7db090 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c014930 tx=0x7f6b8c014df0 comp rx=0 tx=0).stop 2024-09-12T02:15:43.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.369+0000 7f6b9d54a740 1 -- 172.21.15.16:0/1791309004 shutdown_connections 2024-09-12T02:15:43.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.369+0000 7f6b9d54a740 1 --2- 172.21.15.16:0/1791309004 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ee3c7e6370 0x55ee3c7e8760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:43.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.369+0000 7f6b9d54a740 1 --2- 172.21.15.16:0/1791309004 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ee3c7db5d0 0x55ee3c7dba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:43.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.369+0000 7f6b9d54a740 1 --2- 172.21.15.16:0/1791309004 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee3c7dcb40 0x55ee3c7db090 unknown :-1 s=CLOSED pgs=1863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:43.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.369+0000 7f6b9d54a740 1 -- 172.21.15.16:0/1791309004 >> 172.21.15.16:0/1791309004 conn(0x55ee3c7a4310 msgr2=0x55ee3c7d7c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:15:43.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.369+0000 7f6b9d54a740 1 -- 172.21.15.16:0/1791309004 shutdown_connections 2024-09-12T02:15:43.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.369+0000 7f6b9d54a740 1 -- 172.21.15.16:0/1791309004 wait complete. 2024-09-12T02:15:43.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.370+0000 7f6b9d54a740 1 -- start start 2024-09-12T02:15:43.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.370+0000 7f6b9d54a740 1 Processor -- start 2024-09-12T02:15:43.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.370+0000 7f6b9d54a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee3c7db5d0 0x55ee3c7e5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:43.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.370+0000 7f6b9d54a740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ee3c7dcb40 0x55ee3c7e2f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:43.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.370+0000 7f6b9d54a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ee3c7e6370 0x55ee3c7e3490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:43.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.370+0000 7f6b9d54a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55ee3c7e90c0 con 0x55ee3c7db5d0 2024-09-12T02:15:43.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.370+0000 7f6b9d54a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55ee3c6fc060 con 0x55ee3c7e6370 2024-09-12T02:15:43.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.370+0000 7f6b9d54a740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55ee3c7e9240 con 0x55ee3c7dcb40 2024-09-12T02:15:43.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.370+0000 7f6b9d54a740 10 client.0 _ll_register_callbacks cb 0x55ee3c44f8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:15:43.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.370+0000 7f6b98f0d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee3c7db5d0 0x55ee3c7e5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:43.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.370+0000 7f6b98f0d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee3c7db5d0 0x55ee3c7e5ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53532/0 (socket says 172.21.15.16:53532) 2024-09-12T02:15:43.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.370+0000 7f6b98f0d640 1 -- 172.21.15.16:0/3306715850 learned_addr learned my addr 172.21.15.16:0/3306715850 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:15:43.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.371+0000 7f6b8bfff640 1 --2- 172.21.15.16:0/3306715850 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ee3c7dcb40 0x55ee3c7e2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:43.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.371+0000 7f6b9970e640 1 --2- 172.21.15.16:0/3306715850 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ee3c7e6370 0x55ee3c7e3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:43.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.371+0000 7f6b98f0d640 1 -- 172.21.15.16:0/3306715850 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ee3c7dcb40 msgr2=0x55ee3c7e2f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:15:43.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.371+0000 7f6b98f0d640 1 --2- 172.21.15.16:0/3306715850 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ee3c7dcb40 0x55ee3c7e2f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:43.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.371+0000 7f6b98f0d640 1 -- 172.21.15.16:0/3306715850 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ee3c7e6370 msgr2=0x55ee3c7e3490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:15:43.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.371+0000 7f6b98f0d640 1 --2- 172.21.15.16:0/3306715850 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ee3c7e6370 0x55ee3c7e3490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:43.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.371+0000 7f6b98f0d640 1 -- 172.21.15.16:0/3306715850 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee3c4b23c0 con 0x55ee3c7db5d0 2024-09-12T02:15:43.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.371+0000 7f6b8bfff640 1 --2- 172.21.15.16:0/3306715850 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ee3c7dcb40 0x55ee3c7e2f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:15:43.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.371+0000 7f6b9970e640 1 --2- 172.21.15.16:0/3306715850 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ee3c7e6370 0x55ee3c7e3490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:15:43.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.371+0000 7f6b98f0d640 1 --2- 172.21.15.16:0/3306715850 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee3c7db5d0 0x55ee3c7e5ca0 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7f6b900b8c70 tx=0x7f6b90009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1133f75490960f39 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:15:43.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.371+0000 7f6b73fff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:15:43.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.371+0000 7f6b73fff640 1 -- 172.21.15.16:0/3306715850 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b90049040 con 0x55ee3c7db5d0 2024-09-12T02:15:43.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.371+0000 7f6b73fff640 1 -- 172.21.15.16:0/3306715850 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f6b9003b580 con 0x55ee3c7db5d0 2024-09-12T02:15:43.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.371+0000 7f6b9d54a740 1 -- 172.21.15.16:0/3306715850 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b8c008df0 con 0x55ee3c7db5d0 2024-09-12T02:15:43.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.371+0000 7f6b73fff640 1 -- 172.21.15.16:0/3306715850 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6b900c0650 con 0x55ee3c7db5d0 2024-09-12T02:15:43.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.371+0000 7f6b9d54a740 10 client.20010 fetch_fsmap learned FSMap version 25 2024-09-12T02:15:43.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.371+0000 7f6b9d54a740 1 -- 172.21.15.16:0/3306715850 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ee3c7f4b00 con 0x55ee3c7db5d0 2024-09-12T02:15:43.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.372+0000 7f6b73fff640 1 -- 172.21.15.16:0/3306715850 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b8c008df0 con 0x55ee3c7db5d0 2024-09-12T02:15:43.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.372+0000 7f6b73fff640 1 -- 172.21.15.16:0/3306715850 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f6b900c4b10 con 0x55ee3c7db5d0 2024-09-12T02:15:43.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.372+0000 7f6b9d54a740 10 client.20010 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:15:43.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.372+0000 7f6b9d54a740 10 client.20010 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-12T02:15:43.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.372+0000 7f6b9d54a740 20 client.20010 populate_metadata read hostname 'smithi016' 2024-09-12T02:15:43.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.372+0000 7f6b9d54a740 1 --2- 172.21.15.16:0/3306715850 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55ee3c7f5ef0 0x55ee3c7f8360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:43.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.372+0000 7f6b9d54a740 4 client.20010 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:15:43.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.372+0000 7f6b9d54a740 1 -- 172.21.15.16:0/3306715850 --> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ee3c74e860 con 0x55ee3c7f5ef0 2024-09-12T02:15:43.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.372+0000 7f6b8bfff640 1 --2- 172.21.15.16:0/3306715850 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55ee3c7f5ef0 0x55ee3c7f8360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:43.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.373+0000 7f6b73fff640 10 client.20010 ms_handle_connect on v2:172.21.15.184:6836/1793393087 2024-09-12T02:15:43.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.373+0000 7f6b8bfff640 1 --2- 172.21.15.16:0/3306715850 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55ee3c7f5ef0 0x55ee3c7f8360 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b0fac4e4fb40dbec server_cookie=f492ff136b088325 in_seq=0 out_seq=0 2024-09-12T02:15:43.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.373+0000 7f6b73fff640 1 -- 172.21.15.16:0/3306715850 <== mds.2 v2:172.21.15.184:6836/1793393087 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ee3c74e860 con 0x55ee3c7f5ef0 2024-09-12T02:15:43.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.373+0000 7f6b73fff640 10 client.20010 handle_command_reply: tid=0 2024-09-12T02:15:43.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.373+0000 7f6b9d54a740 1 client.20010 shutdown 2024-09-12T02:15:43.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.374+0000 7f6b9d54a740 1 -- 172.21.15.16:0/3306715850 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee3c7db5d0 msgr2=0x55ee3c7e5ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:43.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.374+0000 7f6b9d54a740 1 --2- 172.21.15.16:0/3306715850 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee3c7db5d0 0x55ee3c7e5ca0 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7f6b900b8c70 tx=0x7f6b90009260 comp rx=0 tx=0).stop 2024-09-12T02:15:43.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.374+0000 7f6b9d54a740 1 -- 172.21.15.16:0/3306715850 shutdown_connections 2024-09-12T02:15:43.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.374+0000 7f6b9d54a740 1 --2- 172.21.15.16:0/3306715850 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ee3c7e6370 0x55ee3c7e3490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:43.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.374+0000 7f6b9d54a740 1 --2- 172.21.15.16:0/3306715850 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ee3c7dcb40 0x55ee3c7e2f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:43.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.374+0000 7f6b9d54a740 1 --2- 172.21.15.16:0/3306715850 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55ee3c7f5ef0 0x55ee3c7f8360 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:43.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.374+0000 7f6b73fff640 0 client.20010 ms_handle_reset on v2:172.21.15.184:6836/1793393087 2024-09-12T02:15:43.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.374+0000 7f6b9d54a740 1 --2- 172.21.15.16:0/3306715850 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee3c7db5d0 0x55ee3c7e5ca0 unknown :-1 s=CLOSED pgs=1864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:43.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.374+0000 7f6b9d54a740 1 -- 172.21.15.16:0/3306715850 >> 172.21.15.16:0/3306715850 conn(0x55ee3c7a4310 msgr2=0x55ee3c7dd960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:15:43.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.374+0000 7f6b9d54a740 1 -- 172.21.15.16:0/3306715850 shutdown_connections 2024-09-12T02:15:43.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.374+0000 7f6b9d54a740 1 -- 172.21.15.16:0/3306715850 wait complete. 2024-09-12T02:15:43.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.374+0000 7f6b9d54a740 20 client.20010 trim_cache size 0 max 16384 2024-09-12T02:15:43.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.387+0000 7f6b9d54a740 1 Processor -- start 2024-09-12T02:15:43.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.387+0000 7f6b9d54a740 1 -- start start 2024-09-12T02:15:43.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.387+0000 7f6b9d54a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ee3c7db5d0 0x55ee3c7f23b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:43.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.387+0000 7f6b9d54a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee3c7dcb40 0x55ee3c7e2810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:43.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.387+0000 7f6b9d54a740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ee3c7e6370 0x55ee3c7fdc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:43.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.387+0000 7f6b9d54a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6b941afda0 con 0x55ee3c7dcb40 2024-09-12T02:15:43.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.387+0000 7f6b9d54a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6b941ad670 con 0x55ee3c7db5d0 2024-09-12T02:15:43.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.387+0000 7f6b9d54a740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f6b941ab4c0 con 0x55ee3c7e6370 2024-09-12T02:15:43.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.387+0000 7f6b8bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee3c7dcb40 0x55ee3c7e2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:43.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.387+0000 7f6b8bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee3c7dcb40 0x55ee3c7e2810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53548/0 (socket says 172.21.15.16:53548) 2024-09-12T02:15:43.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.387+0000 7f6b8bfff640 1 -- 172.21.15.16:0/4107262912 learned_addr learned my addr 172.21.15.16:0/4107262912 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:15:43.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.388+0000 7f6b98f0d640 1 --2- 172.21.15.16:0/4107262912 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ee3c7db5d0 0x55ee3c7f23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:43.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.388+0000 7f6b9970e640 1 --2- 172.21.15.16:0/4107262912 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ee3c7e6370 0x55ee3c7fdc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:43.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.388+0000 7f6b8bfff640 1 -- 172.21.15.16:0/4107262912 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ee3c7e6370 msgr2=0x55ee3c7fdc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:15:43.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.388+0000 7f6b8bfff640 1 --2- 172.21.15.16:0/4107262912 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ee3c7e6370 0x55ee3c7fdc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:43.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.388+0000 7f6b8bfff640 1 -- 172.21.15.16:0/4107262912 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ee3c7db5d0 msgr2=0x55ee3c7f23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:15:43.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.388+0000 7f6b8bfff640 1 --2- 172.21.15.16:0/4107262912 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ee3c7db5d0 0x55ee3c7f23b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:43.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.388+0000 7f6b8bfff640 1 -- 172.21.15.16:0/4107262912 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee3c7dc760 con 0x55ee3c7dcb40 2024-09-12T02:15:43.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.388+0000 7f6b98f0d640 1 --2- 172.21.15.16:0/4107262912 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ee3c7db5d0 0x55ee3c7f23b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:15:43.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.388+0000 7f6b8bfff640 1 --2- 172.21.15.16:0/4107262912 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee3c7dcb40 0x55ee3c7e2810 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7f6b8400cc40 tx=0x7f6b840108c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fce9559482550d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:15:43.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.388+0000 7f6b7a7fc640 1 -- 172.21.15.16:0/4107262912 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b84004220 con 0x55ee3c7dcb40 2024-09-12T02:15:43.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.388+0000 7f6b7a7fc640 1 -- 172.21.15.16:0/4107262912 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b840043c0 con 0x55ee3c7dcb40 2024-09-12T02:15:43.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.388+0000 7f6b7a7fc640 1 -- 172.21.15.16:0/4107262912 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b84003850 con 0x55ee3c7dcb40 2024-09-12T02:15:43.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.389+0000 7f6b9d54a740 1 -- 172.21.15.16:0/4107262912 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee3c7dcb40 msgr2=0x55ee3c7e2810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:43.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.389+0000 7f6b9d54a740 1 --2- 172.21.15.16:0/4107262912 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee3c7dcb40 0x55ee3c7e2810 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7f6b8400cc40 tx=0x7f6b840108c0 comp rx=0 tx=0).stop 2024-09-12T02:15:43.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.389+0000 7f6b9d54a740 1 -- 172.21.15.16:0/4107262912 shutdown_connections 2024-09-12T02:15:43.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.389+0000 7f6b9d54a740 1 --2- 172.21.15.16:0/4107262912 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ee3c7e6370 0x55ee3c7fdc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:43.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.389+0000 7f6b9d54a740 1 --2- 172.21.15.16:0/4107262912 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee3c7dcb40 0x55ee3c7e2810 unknown :-1 s=CLOSED pgs=1865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:43.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.389+0000 7f6b9d54a740 1 --2- 172.21.15.16:0/4107262912 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ee3c7db5d0 0x55ee3c7f23b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:43.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.389+0000 7f6b9d54a740 1 -- 172.21.15.16:0/4107262912 >> 172.21.15.16:0/4107262912 conn(0x55ee3c7a4310 msgr2=0x55ee3c7e1820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:15:43.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.389+0000 7f6b9d54a740 1 -- 172.21.15.16:0/4107262912 shutdown_connections 2024-09-12T02:15:43.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.389+0000 7f6b9d54a740 1 -- 172.21.15.16:0/4107262912 wait complete. 2024-09-12T02:15:43.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.389+0000 7f6b9d54a740 1 -- start start 2024-09-12T02:15:43.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.389+0000 7f6b9d54a740 1 Processor -- start 2024-09-12T02:15:43.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.390+0000 7f6b9d54a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ee3c7db5d0 0x55ee3c7ee520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:43.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.390+0000 7f6b9d54a740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ee3c7dcb40 0x55ee3c7eea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:43.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.390+0000 7f6b9d54a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee3c7e6370 0x55ee3c7eb4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:43.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.390+0000 7f6b9d54a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6b941abb30 con 0x55ee3c7e6370 2024-09-12T02:15:43.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.390+0000 7f6b9d54a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6b941acfa0 con 0x55ee3c7db5d0 2024-09-12T02:15:43.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.390+0000 7f6b9d54a740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f6b941ac200 con 0x55ee3c7dcb40 2024-09-12T02:15:43.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.390+0000 7f6b9d54a740 10 client.0 _ll_register_callbacks cb 0x7f6b900bbed0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:15:43.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.390+0000 7f6b9970e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee3c7e6370 0x55ee3c7eb4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:43.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.390+0000 7f6b9970e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee3c7e6370 0x55ee3c7eb4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53550/0 (socket says 172.21.15.16:53550) 2024-09-12T02:15:43.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.390+0000 7f6b9970e640 1 -- 172.21.15.16:0/3542943999 learned_addr learned my addr 172.21.15.16:0/3542943999 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:15:43.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.390+0000 7f6b98f0d640 1 --2- 172.21.15.16:0/3542943999 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ee3c7db5d0 0x55ee3c7ee520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:43.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.390+0000 7f6b8bfff640 1 --2- 172.21.15.16:0/3542943999 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ee3c7dcb40 0x55ee3c7eea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:43.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.390+0000 7f6b9970e640 1 -- 172.21.15.16:0/3542943999 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ee3c7dcb40 msgr2=0x55ee3c7eea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:15:43.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.390+0000 7f6b9970e640 1 --2- 172.21.15.16:0/3542943999 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ee3c7dcb40 0x55ee3c7eea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:43.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.390+0000 7f6b9970e640 1 -- 172.21.15.16:0/3542943999 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ee3c7db5d0 msgr2=0x55ee3c7ee520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:15:43.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.390+0000 7f6b9970e640 1 --2- 172.21.15.16:0/3542943999 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ee3c7db5d0 0x55ee3c7ee520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:43.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.390+0000 7f6b9970e640 1 -- 172.21.15.16:0/3542943999 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee3c7eba30 con 0x55ee3c7e6370 2024-09-12T02:15:43.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.390+0000 7f6b98f0d640 1 --2- 172.21.15.16:0/3542943999 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ee3c7db5d0 0x55ee3c7ee520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:15:43.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.390+0000 7f6b9970e640 1 --2- 172.21.15.16:0/3542943999 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee3c7e6370 0x55ee3c7eb4f0 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c014930 tx=0x7f6b8c011860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a23cab4bf144852 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:15:43.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.390+0000 7f6b73fff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:15:43.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.391+0000 7f6b73fff640 1 -- 172.21.15.16:0/3542943999 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b8c033060 con 0x55ee3c7e6370 2024-09-12T02:15:43.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.391+0000 7f6b73fff640 1 -- 172.21.15.16:0/3542943999 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f6b8c013870 con 0x55ee3c7e6370 2024-09-12T02:15:43.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.391+0000 7f6b9d54a740 1 -- 172.21.15.16:0/3542943999 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b840043c0 con 0x55ee3c7e6370 2024-09-12T02:15:43.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.391+0000 7f6b73fff640 1 -- 172.21.15.16:0/3542943999 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6b8c021040 con 0x55ee3c7e6370 2024-09-12T02:15:43.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.391+0000 7f6b9d54a740 10 client.20016 fetch_fsmap learned FSMap version 25 2024-09-12T02:15:43.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.391+0000 7f6b9d54a740 1 -- 172.21.15.16:0/3542943999 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ee3c8612b0 con 0x55ee3c7e6370 2024-09-12T02:15:43.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.391+0000 7f6b73fff640 1 -- 172.21.15.16:0/3542943999 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b840043c0 con 0x55ee3c7e6370 2024-09-12T02:15:43.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.391+0000 7f6b73fff640 1 -- 172.21.15.16:0/3542943999 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f6b8c038070 con 0x55ee3c7e6370 2024-09-12T02:15:43.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.391+0000 7f6b9d54a740 10 client.20016 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:15:43.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.391+0000 7f6b9d54a740 10 client.20016 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-12T02:15:43.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.391+0000 7f6b9d54a740 20 client.20016 populate_metadata read hostname 'smithi016' 2024-09-12T02:15:43.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.391+0000 7f6b9d54a740 1 --2- 172.21.15.16:0/3542943999 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55ee3c861860 0x55ee3c863c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:43.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.391+0000 7f6b9d54a740 4 client.20016 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T02:15:43.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.391+0000 7f6b9d54a740 1 -- 172.21.15.16:0/3542943999 --> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ee3c748d70 con 0x55ee3c861860 2024-09-12T02:15:43.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.392+0000 7f6b98f0d640 1 --2- 172.21.15.16:0/3542943999 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55ee3c861860 0x55ee3c863c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:43.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.392+0000 7f6b98f0d640 1 --2- 172.21.15.16:0/3542943999 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55ee3c861860 0x55ee3c863c80 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4cf9e657211c6fb4 server_cookie=c2859db49a564bb2 in_seq=0 out_seq=0 2024-09-12T02:15:43.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.392+0000 7f6b73fff640 10 client.20016 ms_handle_connect on v2:172.21.15.184:6836/1793393087 2024-09-12T02:15:43.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.392+0000 7f6b73fff640 1 -- 172.21.15.16:0/3542943999 <== mds.2 v2:172.21.15.184:6836/1793393087 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ee3c748d70 con 0x55ee3c861860 2024-09-12T02:15:43.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.392+0000 7f6b73fff640 10 client.20016 handle_command_reply: tid=0 2024-09-12T02:15:43.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.392+0000 7f6b9d54a740 1 client.20016 shutdown 2024-09-12T02:15:43.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.392+0000 7f6b9d54a740 1 -- 172.21.15.16:0/3542943999 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee3c7e6370 msgr2=0x55ee3c7eb4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:43.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.392+0000 7f6b9d54a740 1 --2- 172.21.15.16:0/3542943999 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee3c7e6370 0x55ee3c7eb4f0 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c014930 tx=0x7f6b8c011860 comp rx=0 tx=0).stop 2024-09-12T02:15:43.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.392+0000 7f6b9d54a740 1 -- 172.21.15.16:0/3542943999 shutdown_connections 2024-09-12T02:15:43.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.392+0000 7f6b9d54a740 1 --2- 172.21.15.16:0/3542943999 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55ee3c861860 0x55ee3c863c80 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:43.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.392+0000 7f6b9d54a740 1 --2- 172.21.15.16:0/3542943999 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee3c7e6370 0x55ee3c7eb4f0 unknown :-1 s=CLOSED pgs=1866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:43.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.392+0000 7f6b9d54a740 1 --2- 172.21.15.16:0/3542943999 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ee3c7dcb40 0x55ee3c7eea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:43.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.392+0000 7f6b9d54a740 1 --2- 172.21.15.16:0/3542943999 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ee3c7db5d0 0x55ee3c7ee520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:43.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.392+0000 7f6b9d54a740 1 -- 172.21.15.16:0/3542943999 >> 172.21.15.16:0/3542943999 conn(0x55ee3c7a4310 msgr2=0x55ee3c7e47b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:15:43.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.392+0000 7f6b73fff640 0 client.20016 ms_handle_reset on v2:172.21.15.184:6836/1793393087 2024-09-12T02:15:43.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.393+0000 7f6b9d54a740 1 -- 172.21.15.16:0/3542943999 shutdown_connections 2024-09-12T02:15:43.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.393+0000 7f6b9d54a740 1 -- 172.21.15.16:0/3542943999 wait complete. 2024-09-12T02:15:43.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.393+0000 7f6b9d54a740 20 client.20016 trim_cache size 0 max 16384 2024-09-12T02:15:43.403 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T02:15:43.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.393+0000 7f6b9b171640 1 -- 172.21.15.16:0/1032539227 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f6b7c0a2b80 msgr2=0x7f6b7c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:43.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.393+0000 7f6b9b171640 1 --2- 172.21.15.16:0/1032539227 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f6b7c0a2b80 0x7f6b7c0a5040 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c01e000 tx=0x7f6b8c01a000 comp rx=0 tx=0).stop 2024-09-12T02:15:43.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.393+0000 7f6b9b171640 1 -- 172.21.15.16:0/1032539227 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b94102db0 msgr2=0x7f6b941985c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:43.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.393+0000 7f6b9b171640 1 --2- 172.21.15.16:0/1032539227 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b94102db0 0x7f6b941985c0 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7f6b90009230 tx=0x7f6b90003c20 comp rx=0 tx=0).stop 2024-09-12T02:15:43.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.393+0000 7f6b9b171640 1 -- 172.21.15.16:0/1032539227 shutdown_connections 2024-09-12T02:15:43.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.393+0000 7f6b9b171640 1 --2- 172.21.15.16:0/1032539227 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f6b7c0a2b80 0x7f6b7c0a5040 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:43.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.393+0000 7f6b9b171640 1 --2- 172.21.15.16:0/1032539227 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b941082e0 0x7f6b9419ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:43.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.393+0000 7f6b9b171640 1 --2- 172.21.15.16:0/1032539227 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6b94103760 0x7f6b94198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:43.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.393+0000 7f6b9b171640 1 --2- 172.21.15.16:0/1032539227 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b94102db0 0x7f6b941985c0 unknown :-1 s=CLOSED pgs=1862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:43.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.393+0000 7f6b9b171640 1 -- 172.21.15.16:0/1032539227 >> 172.21.15.16:0/1032539227 conn(0x7f6b940fc7d0 msgr2=0x7f6b94105df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:15:43.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.393+0000 7f6b9b171640 1 -- 172.21.15.16:0/1032539227 shutdown_connections 2024-09-12T02:15:43.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:43.394+0000 7f6b9b171640 1 -- 172.21.15.16:0/1032539227 wait complete. 2024-09-12T02:15:44.063 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-12T02:15:44.320 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:15:44.577 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:15:44 smithi016 ceph-mon[30947]: pgmap v749: 97 pgs: 97 active+clean; 3.0 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 6.2 KiB/s wr, 10 op/s 2024-09-12T02:15:44.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:15:44 smithi184 ceph-mon[35593]: pgmap v749: 97 pgs: 97 active+clean; 3.0 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 6.2 KiB/s wr, 10 op/s 2024-09-12T02:15:44.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:15:44 smithi033 ceph-mon[35332]: pgmap v749: 97 pgs: 97 active+clean; 3.0 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 6.2 KiB/s wr, 10 op/s 2024-09-12T02:15:44.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.770+0000 7f52b86ec640 1 -- 172.21.15.16:0/3963701946 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f52b0106a80 msgr2=0x7f52b0106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:44.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.770+0000 7f52b86ec640 1 --2- 172.21.15.16:0/3963701946 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f52b0106a80 0x7f52b0106e60 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7f52ac009930 tx=0x7f52ac02a420 comp rx=0 tx=0).stop 2024-09-12T02:15:44.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.770+0000 7f52b86ec640 1 -- 172.21.15.16:0/3963701946 shutdown_connections 2024-09-12T02:15:44.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.770+0000 7f52b86ec640 1 --2- 172.21.15.16:0/3963701946 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f52b0101420 0x7f52b010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:44.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.770+0000 7f52b86ec640 1 --2- 172.21.15.16:0/3963701946 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52b0100a80 0x7f52b0100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:44.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.770+0000 7f52b86ec640 1 --2- 172.21.15.16:0/3963701946 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f52b0106a80 0x7f52b0106e60 unknown :-1 s=CLOSED pgs=1867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:44.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.770+0000 7f52b86ec640 1 -- 172.21.15.16:0/3963701946 >> 172.21.15.16:0/3963701946 conn(0x7f52b00fc7a0 msgr2=0x7f52b00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:15:44.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.771+0000 7f52b86ec640 1 -- 172.21.15.16:0/3963701946 shutdown_connections 2024-09-12T02:15:44.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.771+0000 7f52b86ec640 1 -- 172.21.15.16:0/3963701946 wait complete. 2024-09-12T02:15:44.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.771+0000 7f52b86ec640 1 Processor -- start 2024-09-12T02:15:44.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.771+0000 7f52b86ec640 1 -- start start 2024-09-12T02:15:44.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.771+0000 7f52b86ec640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f52b0100a80 0x7f52b0197360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:44.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.771+0000 7f52b86ec640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52b0101420 0x7f52b01978a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:44.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.771+0000 7f52b86ec640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f52b0106a80 0x7f52b019bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:44.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.771+0000 7f52b86ec640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f52b0110ea0 con 0x7f52b0100a80 2024-09-12T02:15:44.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.771+0000 7f52b86ec640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f52b0110d20 con 0x7f52b0101420 2024-09-12T02:15:44.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.771+0000 7f52b86ec640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f52b0111020 con 0x7f52b0106a80 2024-09-12T02:15:44.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.771+0000 7f52b6488640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f52b0100a80 0x7f52b0197360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:44.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.772+0000 7f52b6488640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f52b0100a80 0x7f52b0197360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53574/0 (socket says 172.21.15.16:53574) 2024-09-12T02:15:44.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.772+0000 7f52b6488640 1 -- 172.21.15.16:0/4134220260 learned_addr learned my addr 172.21.15.16:0/4134220260 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:15:44.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.772+0000 7f52b5c87640 1 --2- 172.21.15.16:0/4134220260 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52b0101420 0x7f52b01978a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:44.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.772+0000 7f52b6c89640 1 --2- 172.21.15.16:0/4134220260 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f52b0106a80 0x7f52b019bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:44.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.772+0000 7f52b6488640 1 -- 172.21.15.16:0/4134220260 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f52b0106a80 msgr2=0x7f52b019bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:44.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.772+0000 7f52b6488640 1 --2- 172.21.15.16:0/4134220260 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f52b0106a80 0x7f52b019bc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:44.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.772+0000 7f52b6488640 1 -- 172.21.15.16:0/4134220260 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52b0101420 msgr2=0x7f52b01978a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:44.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.772+0000 7f52b6488640 1 --2- 172.21.15.16:0/4134220260 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52b0101420 0x7f52b01978a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:44.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.772+0000 7f52b6488640 1 -- 172.21.15.16:0/4134220260 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52b019c310 con 0x7f52b0100a80 2024-09-12T02:15:44.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.772+0000 7f52b6c89640 1 --2- 172.21.15.16:0/4134220260 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f52b0106a80 0x7f52b019bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:15:44.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.772+0000 7f52b6488640 1 --2- 172.21.15.16:0/4134220260 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f52b0100a80 0x7f52b0197360 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7f52ac009230 tx=0x7f52ac004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:15:44.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.772+0000 7f52a37fe640 1 -- 172.21.15.16:0/4134220260 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52ac03f030 con 0x7f52b0100a80 2024-09-12T02:15:44.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.772+0000 7f52b86ec640 1 -- 172.21.15.16:0/4134220260 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52b019c5a0 con 0x7f52b0100a80 2024-09-12T02:15:44.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.772+0000 7f52a37fe640 1 -- 172.21.15.16:0/4134220260 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f52ac003ad0 con 0x7f52b0100a80 2024-09-12T02:15:44.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.772+0000 7f52a37fe640 1 -- 172.21.15.16:0/4134220260 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52ac035070 con 0x7f52b0100a80 2024-09-12T02:15:44.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.772+0000 7f52b86ec640 1 -- 172.21.15.16:0/4134220260 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52b01a3ee0 con 0x7f52b0100a80 2024-09-12T02:15:44.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.774+0000 7f52a37fe640 1 -- 172.21.15.16:0/4134220260 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f52ac042030 con 0x7f52b0100a80 2024-09-12T02:15:44.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.775+0000 7f52baac5740 1 Processor -- start 2024-09-12T02:15:44.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.775+0000 7f52baac5740 1 -- start start 2024-09-12T02:15:44.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.775+0000 7f52baac5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f6cbd48b40 0x55f6cbd47090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:44.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.775+0000 7f52baac5740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f6cbd475d0 0x55f6cbd47a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:44.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.775+0000 7f52baac5740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f6cbd54580 0x55f6cbd56970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:44.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.775+0000 7f52baac5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55f6cbcad2f0 con 0x55f6cbd48b40 2024-09-12T02:15:44.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.775+0000 7f52baac5740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55f6cbc54cf0 con 0x55f6cbd475d0 2024-09-12T02:15:44.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.775+0000 7f52a37fe640 1 --2- 172.21.15.16:0/4134220260 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f529c0a2b80 0x7f529c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:44.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.775+0000 7f52baac5740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55f6cbca92b0 con 0x55f6cbd54580 2024-09-12T02:15:44.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.776+0000 7f52b6c89640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f6cbd48b40 0x55f6cbd47090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:44.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.776+0000 7f52b6c89640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f6cbd48b40 0x55f6cbd47090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53584/0 (socket says 172.21.15.16:53584) 2024-09-12T02:15:44.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.776+0000 7f52b6c89640 1 -- 172.21.15.16:0/1299354799 learned_addr learned my addr 172.21.15.16:0/1299354799 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:15:44.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.776+0000 7f52a37fe640 1 -- 172.21.15.16:0/4134220260 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f52ac0ef510 con 0x7f52b0100a80 2024-09-12T02:15:44.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.776+0000 7f52b6488640 1 --2- 172.21.15.16:0/1299354799 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f6cbd475d0 0x55f6cbd47a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:44.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.776+0000 7f52b5c87640 1 --2- 172.21.15.16:0/1299354799 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f6cbd54580 0x55f6cbd56970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:44.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.776+0000 7f52b6c89640 1 --2- 172.21.15.16:0/4134220260 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f529c0a2b80 0x7f529c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:44.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.776+0000 7f52b6488640 1 -- 172.21.15.16:0/1299354799 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f6cbd54580 msgr2=0x55f6cbd56970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:15:44.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.776+0000 7f52b6488640 1 --2- 172.21.15.16:0/1299354799 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f6cbd54580 0x55f6cbd56970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:44.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.776+0000 7f52b6488640 1 -- 172.21.15.16:0/1299354799 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f6cbd48b40 msgr2=0x55f6cbd47090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:15:44.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.776+0000 7f52b6488640 1 --2- 172.21.15.16:0/1299354799 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f6cbd48b40 0x55f6cbd47090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:44.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.776+0000 7f52b6488640 1 -- 172.21.15.16:0/1299354799 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f6cbc393a0 con 0x55f6cbd475d0 2024-09-12T02:15:44.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.776+0000 7f52b6c89640 1 --2- 172.21.15.16:0/1299354799 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f6cbd48b40 0x55f6cbd47090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:15:44.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.776+0000 7f52b6488640 1 --2- 172.21.15.16:0/1299354799 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f6cbd475d0 0x55f6cbd47a10 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f52ac0b81d0 tx=0x7f52ac0045f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=335b341e040922a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:15:44.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.776+0000 7f52b6c89640 1 --2- 172.21.15.16:0/4134220260 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f529c0a2b80 0x7f529c0a5040 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f52a800eca0 tx=0x7f52a8009ad0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:15:44.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.776+0000 7f52a0ff9640 1 -- 172.21.15.16:0/1299354799 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52ac049040 con 0x55f6cbd475d0 2024-09-12T02:15:44.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.776+0000 7f52a0ff9640 1 -- 172.21.15.16:0/1299354799 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f52ac0bbd70 con 0x55f6cbd475d0 2024-09-12T02:15:44.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.776+0000 7f52a0ff9640 1 -- 172.21.15.16:0/1299354799 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52ac003c80 con 0x55f6cbd475d0 2024-09-12T02:15:44.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.777+0000 7f52baac5740 1 -- 172.21.15.16:0/1299354799 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f6cbd475d0 msgr2=0x55f6cbd47a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:44.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.777+0000 7f52baac5740 1 --2- 172.21.15.16:0/1299354799 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f6cbd475d0 0x55f6cbd47a10 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f52ac0b81d0 tx=0x7f52ac0045f0 comp rx=0 tx=0).stop 2024-09-12T02:15:44.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.777+0000 7f52baac5740 1 -- 172.21.15.16:0/1299354799 shutdown_connections 2024-09-12T02:15:44.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.777+0000 7f52baac5740 1 --2- 172.21.15.16:0/1299354799 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f6cbd54580 0x55f6cbd56970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:44.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.777+0000 7f52baac5740 1 --2- 172.21.15.16:0/1299354799 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f6cbd475d0 0x55f6cbd47a10 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:44.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.777+0000 7f52baac5740 1 --2- 172.21.15.16:0/1299354799 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f6cbd48b40 0x55f6cbd47090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:44.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.777+0000 7f52baac5740 1 -- 172.21.15.16:0/1299354799 >> 172.21.15.16:0/1299354799 conn(0x55f6cbd10310 msgr2=0x55f6cbd43c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:15:44.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.777+0000 7f52baac5740 1 -- 172.21.15.16:0/1299354799 shutdown_connections 2024-09-12T02:15:44.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.777+0000 7f52baac5740 1 -- 172.21.15.16:0/1299354799 wait complete. 2024-09-12T02:15:44.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.778+0000 7f52baac5740 1 -- start start 2024-09-12T02:15:44.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.778+0000 7f52baac5740 1 Processor -- start 2024-09-12T02:15:44.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.778+0000 7f52baac5740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f6cbd475d0 0x55f6cbd53fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:44.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.778+0000 7f52baac5740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f6cbd48b40 0x55f6cbd51290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:44.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.778+0000 7f52baac5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f6cbd54580 0x55f6cbd51a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:44.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.778+0000 7f52baac5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55f6cbd57040 con 0x55f6cbd54580 2024-09-12T02:15:44.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.778+0000 7f52baac5740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55f6cbc68060 con 0x55f6cbd475d0 2024-09-12T02:15:44.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.778+0000 7f52baac5740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55f6cbd571c0 con 0x55f6cbd48b40 2024-09-12T02:15:44.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.778+0000 7f52baac5740 10 client.0 _ll_register_callbacks cb 0x55f6cb9bb8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:15:44.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.778+0000 7f52b6c89640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f6cbd54580 0x55f6cbd51a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:44.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.778+0000 7f52b6c89640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f6cbd54580 0x55f6cbd51a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53596/0 (socket says 172.21.15.16:53596) 2024-09-12T02:15:44.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.778+0000 7f52b6c89640 1 -- 172.21.15.16:0/2694706261 learned_addr learned my addr 172.21.15.16:0/2694706261 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:15:44.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.779+0000 7f52b6488640 1 --2- 172.21.15.16:0/2694706261 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f6cbd475d0 0x55f6cbd53fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:44.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.779+0000 7f52b6c89640 1 -- 172.21.15.16:0/2694706261 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f6cbd48b40 msgr2=0x55f6cbd51290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:15:44.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.779+0000 7f52b6c89640 1 --2- 172.21.15.16:0/2694706261 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f6cbd48b40 0x55f6cbd51290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:44.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.779+0000 7f52b6c89640 1 -- 172.21.15.16:0/2694706261 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f6cbd475d0 msgr2=0x55f6cbd53fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:15:44.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.779+0000 7f52b6c89640 1 --2- 172.21.15.16:0/2694706261 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f6cbd475d0 0x55f6cbd53fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:44.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.779+0000 7f52b6c89640 1 -- 172.21.15.16:0/2694706261 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f6cba1e3c0 con 0x55f6cbd54580 2024-09-12T02:15:44.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.779+0000 7f52b6488640 1 --2- 172.21.15.16:0/2694706261 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f6cbd475d0 0x55f6cbd53fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:15:44.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.779+0000 7f52b6c89640 1 --2- 172.21.15.16:0/2694706261 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f6cbd54580 0x55f6cbd51a20 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7f52a800efd0 tx=0x7f52a80097b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3588327cab354958 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:15:44.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.779+0000 7f5299ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:15:44.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.779+0000 7f5299ffb640 1 -- 172.21.15.16:0/2694706261 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a801acd0 con 0x55f6cbd54580 2024-09-12T02:15:44.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.779+0000 7f5299ffb640 1 -- 172.21.15.16:0/2694706261 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f52a8028480 con 0x55f6cbd54580 2024-09-12T02:15:44.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.779+0000 7f52baac5740 1 -- 172.21.15.16:0/2694706261 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f52ac0bbd70 con 0x55f6cbd54580 2024-09-12T02:15:44.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.779+0000 7f5299ffb640 1 -- 172.21.15.16:0/2694706261 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f52a8028680 con 0x55f6cbd54580 2024-09-12T02:15:44.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.779+0000 7f52baac5740 10 client.20028 fetch_fsmap learned FSMap version 25 2024-09-12T02:15:44.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.779+0000 7f52baac5740 1 -- 172.21.15.16:0/2694706261 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f6cbd4bed0 con 0x55f6cbd54580 2024-09-12T02:15:44.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.779+0000 7f5299ffb640 1 -- 172.21.15.16:0/2694706261 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f52ac0bbd70 con 0x55f6cbd54580 2024-09-12T02:15:44.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.780+0000 7f5299ffb640 1 -- 172.21.15.16:0/2694706261 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f52a801b070 con 0x55f6cbd54580 2024-09-12T02:15:44.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.780+0000 7f52baac5740 10 client.20028 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:15:44.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.780+0000 7f52baac5740 10 client.20028 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T02:15:44.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.780+0000 7f52baac5740 20 client.20028 populate_metadata read hostname 'smithi016' 2024-09-12T02:15:44.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.780+0000 7f52baac5740 1 --2- 172.21.15.16:0/2694706261 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x55f6cbd66470 0x55f6cbd68860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:44.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.780+0000 7f52baac5740 4 client.20028 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:15:44.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.780+0000 7f52baac5740 1 -- 172.21.15.16:0/2694706261 --> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f6cbcba860 con 0x55f6cbd66470 2024-09-12T02:15:44.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.780+0000 7f52b6488640 1 --2- 172.21.15.16:0/2694706261 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x55f6cbd66470 0x55f6cbd68860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:44.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.780+0000 7f5299ffb640 10 client.20028 ms_handle_connect on v2:172.21.15.184:6838/4203347941 2024-09-12T02:15:44.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.780+0000 7f52b6488640 1 --2- 172.21.15.16:0/2694706261 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x55f6cbd66470 0x55f6cbd68860 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=184b1b1bf49584aa server_cookie=d2aba1a7b9591b26 in_seq=0 out_seq=0 2024-09-12T02:15:44.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.781+0000 7f5299ffb640 1 -- 172.21.15.16:0/2694706261 <== mds.1 v2:172.21.15.184:6838/4203347941 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f6cbcba860 con 0x55f6cbd66470 2024-09-12T02:15:44.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.781+0000 7f5299ffb640 10 client.20028 handle_command_reply: tid=0 2024-09-12T02:15:44.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.781+0000 7f52baac5740 1 client.20028 shutdown 2024-09-12T02:15:44.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.781+0000 7f52baac5740 1 -- 172.21.15.16:0/2694706261 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f6cbd54580 msgr2=0x55f6cbd51a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:44.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.781+0000 7f52baac5740 1 --2- 172.21.15.16:0/2694706261 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f6cbd54580 0x55f6cbd51a20 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7f52a800efd0 tx=0x7f52a80097b0 comp rx=0 tx=0).stop 2024-09-12T02:15:44.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.781+0000 7f52baac5740 1 -- 172.21.15.16:0/2694706261 shutdown_connections 2024-09-12T02:15:44.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.781+0000 7f52baac5740 1 --2- 172.21.15.16:0/2694706261 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x55f6cbd66470 0x55f6cbd68860 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:44.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.781+0000 7f52baac5740 1 --2- 172.21.15.16:0/2694706261 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f6cbd54580 0x55f6cbd51a20 unknown :-1 s=CLOSED pgs=1869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:44.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.781+0000 7f52baac5740 1 --2- 172.21.15.16:0/2694706261 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f6cbd48b40 0x55f6cbd51290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:44.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.781+0000 7f5299ffb640 0 client.20028 ms_handle_reset on v2:172.21.15.184:6838/4203347941 2024-09-12T02:15:44.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.781+0000 7f52baac5740 1 --2- 172.21.15.16:0/2694706261 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f6cbd475d0 0x55f6cbd53fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:44.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.781+0000 7f52baac5740 1 -- 172.21.15.16:0/2694706261 >> 172.21.15.16:0/2694706261 conn(0x55f6cbd10310 msgr2=0x55f6cbd499c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:15:44.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.781+0000 7f52baac5740 1 -- 172.21.15.16:0/2694706261 shutdown_connections 2024-09-12T02:15:44.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.781+0000 7f52baac5740 1 -- 172.21.15.16:0/2694706261 wait complete. 2024-09-12T02:15:44.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.782+0000 7f52baac5740 20 client.20028 trim_cache size 0 max 16384 2024-09-12T02:15:44.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.794+0000 7f52baac5740 1 Processor -- start 2024-09-12T02:15:44.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.794+0000 7f52baac5740 1 -- start start 2024-09-12T02:15:44.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.795+0000 7f52baac5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f6cbd475d0 0x55f6cbd50b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:44.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.795+0000 7f52baac5740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f6cbd48b40 0x55f6cbd43700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:44.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.795+0000 7f52baac5740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f6cbd54580 0x55f6cbd43c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:44.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.795+0000 7f52baac5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f52b01adae0 con 0x55f6cbd475d0 2024-09-12T02:15:44.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.795+0000 7f52baac5740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f52b01ab3b0 con 0x55f6cbd48b40 2024-09-12T02:15:44.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.795+0000 7f52baac5740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f52b01a9260 con 0x55f6cbd54580 2024-09-12T02:15:44.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.795+0000 7f52b6488640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f6cbd475d0 0x55f6cbd50b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:44.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.795+0000 7f52b6488640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f6cbd475d0 0x55f6cbd50b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53598/0 (socket says 172.21.15.16:53598) 2024-09-12T02:15:44.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.795+0000 7f52b6488640 1 -- 172.21.15.16:0/835525166 learned_addr learned my addr 172.21.15.16:0/835525166 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:15:44.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.795+0000 7f52b5c87640 1 --2- 172.21.15.16:0/835525166 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f6cbd48b40 0x55f6cbd43700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:44.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.795+0000 7f52b6c89640 1 --2- 172.21.15.16:0/835525166 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f6cbd54580 0x55f6cbd43c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:44.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.795+0000 7f52b6488640 1 -- 172.21.15.16:0/835525166 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f6cbd54580 msgr2=0x55f6cbd43c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:15:44.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.795+0000 7f52b6488640 1 --2- 172.21.15.16:0/835525166 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f6cbd54580 0x55f6cbd43c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:44.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.795+0000 7f52b6488640 1 -- 172.21.15.16:0/835525166 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f6cbd48b40 msgr2=0x55f6cbd43700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:15:44.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.795+0000 7f52b6488640 1 --2- 172.21.15.16:0/835525166 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f6cbd48b40 0x55f6cbd43700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:44.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.795+0000 7f52b6488640 1 -- 172.21.15.16:0/835525166 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f6cbd44180 con 0x55f6cbd475d0 2024-09-12T02:15:44.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.795+0000 7f52b6c89640 1 --2- 172.21.15.16:0/835525166 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f6cbd54580 0x55f6cbd43c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:15:44.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.795+0000 7f52b6488640 1 --2- 172.21.15.16:0/835525166 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f6cbd475d0 0x55f6cbd50b50 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7f52ac0c1ca0 tx=0x7f52ac005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75c02e39e136cbcb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:15:44.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.796+0000 7f529bfff640 1 -- 172.21.15.16:0/835525166 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52ac0da070 con 0x55f6cbd475d0 2024-09-12T02:15:44.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.796+0000 7f529bfff640 1 -- 172.21.15.16:0/835525166 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f52ac0d8070 con 0x55f6cbd475d0 2024-09-12T02:15:44.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.796+0000 7f529bfff640 1 -- 172.21.15.16:0/835525166 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52ac049040 con 0x55f6cbd475d0 2024-09-12T02:15:44.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.796+0000 7f52baac5740 1 -- 172.21.15.16:0/835525166 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f6cbd475d0 msgr2=0x55f6cbd50b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:44.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.796+0000 7f52baac5740 1 --2- 172.21.15.16:0/835525166 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f6cbd475d0 0x55f6cbd50b50 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7f52ac0c1ca0 tx=0x7f52ac005c00 comp rx=0 tx=0).stop 2024-09-12T02:15:44.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.796+0000 7f52baac5740 1 -- 172.21.15.16:0/835525166 shutdown_connections 2024-09-12T02:15:44.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.796+0000 7f52baac5740 1 --2- 172.21.15.16:0/835525166 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f6cbd54580 0x55f6cbd43c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:44.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.796+0000 7f52baac5740 1 --2- 172.21.15.16:0/835525166 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f6cbd48b40 0x55f6cbd43700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:44.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.796+0000 7f52baac5740 1 --2- 172.21.15.16:0/835525166 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f6cbd475d0 0x55f6cbd50b50 unknown :-1 s=CLOSED pgs=1870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:44.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.796+0000 7f52baac5740 1 -- 172.21.15.16:0/835525166 >> 172.21.15.16:0/835525166 conn(0x55f6cbd10310 msgr2=0x55f6cbd3ea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:15:44.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.796+0000 7f52baac5740 1 -- 172.21.15.16:0/835525166 shutdown_connections 2024-09-12T02:15:44.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.796+0000 7f52baac5740 1 -- 172.21.15.16:0/835525166 wait complete. 2024-09-12T02:15:44.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.797+0000 7f52baac5740 1 -- start start 2024-09-12T02:15:44.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.797+0000 7f52baac5740 1 Processor -- start 2024-09-12T02:15:44.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.797+0000 7f52baac5740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f6cbd475d0 0x55f6cbd5ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:44.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.797+0000 7f52baac5740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f6cbd48b40 0x55f6cbd57d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:44.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.797+0000 7f52baac5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f6cbd54580 0x55f6cbd58290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:44.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.797+0000 7f52baac5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f52b01a98b0 con 0x55f6cbd54580 2024-09-12T02:15:44.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.797+0000 7f52baac5740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f52b01aace0 con 0x55f6cbd475d0 2024-09-12T02:15:44.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.797+0000 7f52baac5740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f52b01a9f40 con 0x55f6cbd48b40 2024-09-12T02:15:44.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.797+0000 7f52baac5740 10 client.0 _ll_register_callbacks cb 0x7f52a800ecd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:15:44.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.797+0000 7f52b6c89640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f6cbd54580 0x55f6cbd58290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:44.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.797+0000 7f52b6c89640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f6cbd54580 0x55f6cbd58290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53600/0 (socket says 172.21.15.16:53600) 2024-09-12T02:15:44.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.797+0000 7f52b6c89640 1 -- 172.21.15.16:0/3128971138 learned_addr learned my addr 172.21.15.16:0/3128971138 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:15:44.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.797+0000 7f52b5c87640 1 --2- 172.21.15.16:0/3128971138 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f6cbd48b40 0x55f6cbd57d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:44.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.797+0000 7f52b6488640 1 --2- 172.21.15.16:0/3128971138 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f6cbd475d0 0x55f6cbd5ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:44.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.797+0000 7f52b6c89640 1 -- 172.21.15.16:0/3128971138 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f6cbd48b40 msgr2=0x55f6cbd57d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:15:44.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.797+0000 7f52b6c89640 1 --2- 172.21.15.16:0/3128971138 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f6cbd48b40 0x55f6cbd57d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:44.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.797+0000 7f52b6c89640 1 -- 172.21.15.16:0/3128971138 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f6cbd475d0 msgr2=0x55f6cbd5ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:15:44.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.797+0000 7f52b6c89640 1 --2- 172.21.15.16:0/3128971138 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f6cbd475d0 0x55f6cbd5ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:44.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.797+0000 7f52b6c89640 1 -- 172.21.15.16:0/3128971138 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f6cbd58980 con 0x55f6cbd54580 2024-09-12T02:15:44.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.798+0000 7f52b6c89640 1 --2- 172.21.15.16:0/3128971138 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f6cbd54580 0x55f6cbd58290 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x7f52a801c8f0 tx=0x7f52a80285e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1013675d1380ca06 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:15:44.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.798+0000 7f5299ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:15:44.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.798+0000 7f5299ffb640 1 -- 172.21.15.16:0/3128971138 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a801cd60 con 0x55f6cbd54580 2024-09-12T02:15:44.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.798+0000 7f5299ffb640 1 -- 172.21.15.16:0/3128971138 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f52a801c410 con 0x55f6cbd54580 2024-09-12T02:15:44.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.798+0000 7f52baac5740 1 -- 172.21.15.16:0/3128971138 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f52ac0d8070 con 0x55f6cbd54580 2024-09-12T02:15:44.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.798+0000 7f5299ffb640 1 -- 172.21.15.16:0/3128971138 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f52a800e9b0 con 0x55f6cbd54580 2024-09-12T02:15:44.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.798+0000 7f52baac5740 10 client.20034 fetch_fsmap learned FSMap version 25 2024-09-12T02:15:44.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.798+0000 7f52baac5740 1 -- 172.21.15.16:0/3128971138 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f6cbdd0de0 con 0x55f6cbd54580 2024-09-12T02:15:44.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.798+0000 7f5299ffb640 1 -- 172.21.15.16:0/3128971138 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f52ac0d8070 con 0x55f6cbd54580 2024-09-12T02:15:44.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.798+0000 7f5299ffb640 1 -- 172.21.15.16:0/3128971138 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f52a80336b0 con 0x55f6cbd54580 2024-09-12T02:15:44.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.798+0000 7f52baac5740 10 client.20034 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:15:44.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.798+0000 7f52baac5740 10 client.20034 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T02:15:44.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.798+0000 7f52baac5740 20 client.20034 populate_metadata read hostname 'smithi016' 2024-09-12T02:15:44.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.798+0000 7f52baac5740 1 --2- 172.21.15.16:0/3128971138 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x55f6cbdd14a0 0x55f6cbdd38c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:44.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.798+0000 7f52baac5740 4 client.20034 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T02:15:44.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.798+0000 7f52baac5740 1 -- 172.21.15.16:0/3128971138 --> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f6cbcb4d70 con 0x55f6cbdd14a0 2024-09-12T02:15:44.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.799+0000 7f52b6488640 1 --2- 172.21.15.16:0/3128971138 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x55f6cbdd14a0 0x55f6cbdd38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:44.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.799+0000 7f52b6488640 1 --2- 172.21.15.16:0/3128971138 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x55f6cbdd14a0 0x55f6cbdd38c0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=853071b64a3354c5 server_cookie=35f64ec3a38189a9 in_seq=0 out_seq=0 2024-09-12T02:15:44.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.799+0000 7f5299ffb640 10 client.20034 ms_handle_connect on v2:172.21.15.184:6838/4203347941 2024-09-12T02:15:44.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.800+0000 7f5299ffb640 1 -- 172.21.15.16:0/3128971138 <== mds.1 v2:172.21.15.184:6838/4203347941 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f6cbcb4d70 con 0x55f6cbdd14a0 2024-09-12T02:15:44.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.800+0000 7f5299ffb640 10 client.20034 handle_command_reply: tid=0 2024-09-12T02:15:44.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.800+0000 7f52baac5740 1 client.20034 shutdown 2024-09-12T02:15:44.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.800+0000 7f52baac5740 1 -- 172.21.15.16:0/3128971138 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f6cbd54580 msgr2=0x55f6cbd58290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:44.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.800+0000 7f52baac5740 1 --2- 172.21.15.16:0/3128971138 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f6cbd54580 0x55f6cbd58290 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x7f52a801c8f0 tx=0x7f52a80285e0 comp rx=0 tx=0).stop 2024-09-12T02:15:44.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.800+0000 7f52baac5740 1 -- 172.21.15.16:0/3128971138 shutdown_connections 2024-09-12T02:15:44.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.800+0000 7f52baac5740 1 --2- 172.21.15.16:0/3128971138 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x55f6cbdd14a0 0x55f6cbdd38c0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:44.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.800+0000 7f52baac5740 1 --2- 172.21.15.16:0/3128971138 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f6cbd54580 0x55f6cbd58290 unknown :-1 s=CLOSED pgs=1871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:44.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.800+0000 7f52baac5740 1 --2- 172.21.15.16:0/3128971138 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f6cbd48b40 0x55f6cbd57d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:44.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.800+0000 7f52baac5740 1 --2- 172.21.15.16:0/3128971138 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f6cbd475d0 0x55f6cbd5ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:44.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.800+0000 7f52baac5740 1 -- 172.21.15.16:0/3128971138 >> 172.21.15.16:0/3128971138 conn(0x55f6cbd10310 msgr2=0x55f6cbd43700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:15:44.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.800+0000 7f5299ffb640 0 client.20034 ms_handle_reset on v2:172.21.15.184:6838/4203347941 2024-09-12T02:15:44.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.800+0000 7f52baac5740 1 -- 172.21.15.16:0/3128971138 shutdown_connections 2024-09-12T02:15:44.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.800+0000 7f52baac5740 1 -- 172.21.15.16:0/3128971138 wait complete. 2024-09-12T02:15:44.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.800+0000 7f52baac5740 20 client.20034 trim_cache size 0 max 16384 2024-09-12T02:15:44.817 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T02:15:44.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.801+0000 7f52b86ec640 1 -- 172.21.15.16:0/4134220260 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f529c0a2b80 msgr2=0x7f529c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:44.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.801+0000 7f52b86ec640 1 --2- 172.21.15.16:0/4134220260 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f529c0a2b80 0x7f529c0a5040 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f52a800eca0 tx=0x7f52a8009ad0 comp rx=0 tx=0).stop 2024-09-12T02:15:44.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.801+0000 7f52b86ec640 1 -- 172.21.15.16:0/4134220260 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f52b0100a80 msgr2=0x7f52b0197360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:44.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.801+0000 7f52b86ec640 1 --2- 172.21.15.16:0/4134220260 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f52b0100a80 0x7f52b0197360 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7f52ac009230 tx=0x7f52ac004270 comp rx=0 tx=0).stop 2024-09-12T02:15:44.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.801+0000 7f52b86ec640 1 -- 172.21.15.16:0/4134220260 shutdown_connections 2024-09-12T02:15:44.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.801+0000 7f52b86ec640 1 --2- 172.21.15.16:0/4134220260 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f529c0a2b80 0x7f529c0a5040 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:44.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.801+0000 7f52b86ec640 1 --2- 172.21.15.16:0/4134220260 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f52b0106a80 0x7f52b019bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:44.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.801+0000 7f52b86ec640 1 --2- 172.21.15.16:0/4134220260 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52b0101420 0x7f52b01978a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:44.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.801+0000 7f52b86ec640 1 --2- 172.21.15.16:0/4134220260 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f52b0100a80 0x7f52b0197360 unknown :-1 s=CLOSED pgs=1868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:44.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.801+0000 7f52b86ec640 1 -- 172.21.15.16:0/4134220260 >> 172.21.15.16:0/4134220260 conn(0x7f52b00fc7a0 msgr2=0x7f52b00fdfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:15:44.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.801+0000 7f52b86ec640 1 -- 172.21.15.16:0/4134220260 shutdown_connections 2024-09-12T02:15:44.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:44.801+0000 7f52b86ec640 1 -- 172.21.15.16:0/4134220260 wait complete. 2024-09-12T02:15:45.504 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-12T02:15:45.659 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.657+0000 7f6da9b42640 1 -- 172.21.15.16:0/1065252536 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6da41025c0 msgr2=0x7f6da41049b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:45.659 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.657+0000 7f6da9b42640 1 --2- 172.21.15.16:0/1065252536 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6da41025c0 0x7f6da41049b0 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7f6d94009900 tx=0x7f6d9402a410 comp rx=0 tx=0).stop 2024-09-12T02:15:45.659 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.657+0000 7f6da9b42640 1 -- 172.21.15.16:0/1065252536 shutdown_connections 2024-09-12T02:15:45.659 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.657+0000 7f6da9b42640 1 --2- 172.21.15.16:0/1065252536 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6da4107820 0x7f6da4109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:45.659 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.657+0000 7f6da9b42640 1 --2- 172.21.15.16:0/1065252536 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6da4104ef0 0x7f6da41072e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:45.659 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.657+0000 7f6da9b42640 1 --2- 172.21.15.16:0/1065252536 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6da41025c0 0x7f6da41049b0 unknown :-1 s=CLOSED pgs=1872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:45.659 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.657+0000 7f6da9b42640 1 -- 172.21.15.16:0/1065252536 >> 172.21.15.16:0/1065252536 conn(0x7f6da40fc1e0 msgr2=0x7f6da40fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:15:45.659 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.657+0000 7f6da9b42640 1 -- 172.21.15.16:0/1065252536 shutdown_connections 2024-09-12T02:15:45.659 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.657+0000 7f6da9b42640 1 -- 172.21.15.16:0/1065252536 wait complete. 2024-09-12T02:15:45.659 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.658+0000 7f6da9b42640 1 Processor -- start 2024-09-12T02:15:45.659 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.658+0000 7f6da9b42640 1 -- start start 2024-09-12T02:15:45.661 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.658+0000 7f6da9b42640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6da41025c0 0x7f6da4197dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:45.662 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.658+0000 7f6da9b42640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6da4104ef0 0x7f6da4198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:45.662 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.658+0000 7f6da9b42640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6da4107820 0x7f6da419f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:45.662 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.658+0000 7f6da9b42640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6da410d800 con 0x7f6da41025c0 2024-09-12T02:15:45.662 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.658+0000 7f6da9b42640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6da410d680 con 0x7f6da4107820 2024-09-12T02:15:45.662 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.658+0000 7f6da9b42640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f6da410d980 con 0x7f6da4104ef0 2024-09-12T02:15:45.662 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.658+0000 7f6da37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6da41025c0 0x7f6da4197dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:45.662 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.658+0000 7f6da37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6da41025c0 0x7f6da4197dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53628/0 (socket says 172.21.15.16:53628) 2024-09-12T02:15:45.662 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.658+0000 7f6da37fe640 1 -- 172.21.15.16:0/2775163088 learned_addr learned my addr 172.21.15.16:0/2775163088 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:15:45.662 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.658+0000 7f6da2ffd640 1 --2- 172.21.15.16:0/2775163088 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6da4104ef0 0x7f6da4198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:45.662 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.658+0000 7f6da3fff640 1 --2- 172.21.15.16:0/2775163088 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6da4107820 0x7f6da419f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:45.662 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.659+0000 7f6da37fe640 1 -- 172.21.15.16:0/2775163088 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6da4104ef0 msgr2=0x7f6da4198300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:45.662 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.659+0000 7f6da37fe640 1 --2- 172.21.15.16:0/2775163088 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6da4104ef0 0x7f6da4198300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:45.662 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.659+0000 7f6da37fe640 1 -- 172.21.15.16:0/2775163088 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6da4107820 msgr2=0x7f6da419f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:45.662 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.659+0000 7f6da37fe640 1 --2- 172.21.15.16:0/2775163088 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6da4107820 0x7f6da419f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:45.663 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.659+0000 7f6da37fe640 1 -- 172.21.15.16:0/2775163088 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6da419f9d0 con 0x7f6da41025c0 2024-09-12T02:15:45.663 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.659+0000 7f6da3fff640 1 --2- 172.21.15.16:0/2775163088 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6da4107820 0x7f6da419f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:15:45.663 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.659+0000 7f6da37fe640 1 --2- 172.21.15.16:0/2775163088 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6da41025c0 0x7f6da4197dc0 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7f6d94009230 tx=0x7f6d94004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:15:45.663 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.659+0000 7f6da0ff9640 1 -- 172.21.15.16:0/2775163088 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d9403f030 con 0x7f6da41025c0 2024-09-12T02:15:45.663 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.659+0000 7f6da0ff9640 1 -- 172.21.15.16:0/2775163088 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d94003ce0 con 0x7f6da41025c0 2024-09-12T02:15:45.663 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.659+0000 7f6da9b42640 1 -- 172.21.15.16:0/2775163088 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6da419fc60 con 0x7f6da41025c0 2024-09-12T02:15:45.663 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.659+0000 7f6da0ff9640 1 -- 172.21.15.16:0/2775163088 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d94035070 con 0x7f6da41025c0 2024-09-12T02:15:45.664 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.659+0000 7f6da9b42640 1 -- 172.21.15.16:0/2775163088 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6da41a0150 con 0x7f6da41025c0 2024-09-12T02:15:45.670 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.661+0000 7f6da0ff9640 1 -- 172.21.15.16:0/2775163088 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6d94042030 con 0x7f6da41025c0 2024-09-12T02:15:45.670 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.662+0000 7f6dac8db740 1 Processor -- start 2024-09-12T02:15:45.670 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.662+0000 7f6dac8db740 1 -- start start 2024-09-12T02:15:45.670 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.662+0000 7f6dac8db740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555563a58db0 0x555563a591b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:45.670 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.662+0000 7f6dac8db740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555563a596f0 0x555563a5e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:45.670 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.662+0000 7f6dac8db740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555563a5e780 0x555563a60c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:45.670 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.662+0000 7f6dac8db740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5555639b82f0 con 0x555563a596f0 2024-09-12T02:15:45.671 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.662+0000 7f6dac8db740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x555563960270 con 0x555563a58db0 2024-09-12T02:15:45.671 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.662+0000 7f6dac8db740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5555639b42b0 con 0x555563a5e780 2024-09-12T02:15:45.671 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.662+0000 7f6da37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555563a596f0 0x555563a5e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:45.671 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.662+0000 7f6da37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555563a596f0 0x555563a5e1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53644/0 (socket says 172.21.15.16:53644) 2024-09-12T02:15:45.671 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.662+0000 7f6da37fe640 1 -- 172.21.15.16:0/3835039535 learned_addr learned my addr 172.21.15.16:0/3835039535 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:15:45.671 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.662+0000 7f6da2ffd640 1 --2- 172.21.15.16:0/3835039535 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555563a5e780 0x555563a60c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:45.671 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.662+0000 7f6da3fff640 1 --2- 172.21.15.16:0/3835039535 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555563a58db0 0x555563a591b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:45.671 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.662+0000 7f6da0ff9640 1 --2- 172.21.15.16:0/2775163088 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f6d8c0a2c30 0x7f6d8c0a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:45.671 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.663+0000 7f6da3fff640 1 --2- 172.21.15.16:0/2775163088 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f6d8c0a2c30 0x7f6d8c0a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:45.671 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.663+0000 7f6da2ffd640 1 -- 172.21.15.16:0/3835039535 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555563a58db0 msgr2=0x555563a591b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:15:45.671 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.663+0000 7f6da2ffd640 1 --2- 172.21.15.16:0/3835039535 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555563a58db0 0x555563a591b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:45.671 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.663+0000 7f6da2ffd640 1 -- 172.21.15.16:0/3835039535 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555563a596f0 msgr2=0x555563a5e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:15:45.671 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.663+0000 7f6da2ffd640 1 --2- 172.21.15.16:0/3835039535 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555563a596f0 0x555563a5e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:45.671 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.663+0000 7f6da2ffd640 1 -- 172.21.15.16:0/3835039535 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55556395f340 con 0x555563a5e780 2024-09-12T02:15:45.672 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.663+0000 7f6da37fe640 1 --2- 172.21.15.16:0/3835039535 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555563a596f0 0x555563a5e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:15:45.672 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.663+0000 7f6da0ff9640 1 -- 172.21.15.16:0/2775163088 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f6d940efb00 con 0x7f6da41025c0 2024-09-12T02:15:45.672 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.663+0000 7f6da2ffd640 1 --2- 172.21.15.16:0/3835039535 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555563a5e780 0x555563a60c40 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f6d9800cb70 tx=0x7f6d9800e720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=671f79a96a5c9985 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:15:45.672 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.663+0000 7f6da3fff640 1 --2- 172.21.15.16:0/2775163088 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f6d8c0a2c30 0x7f6d8c0a50f0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f6d9c009fd0 tx=0x7f6d9c004ab0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:15:45.672 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.663+0000 7f6d89ffb640 1 -- 172.21.15.16:0/3835039535 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d98004220 con 0x555563a5e780 2024-09-12T02:15:45.672 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.663+0000 7f6d89ffb640 1 -- 172.21.15.16:0/3835039535 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d980043c0 con 0x555563a5e780 2024-09-12T02:15:45.672 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.663+0000 7f6d89ffb640 1 -- 172.21.15.16:0/3835039535 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d98004dc0 con 0x555563a5e780 2024-09-12T02:15:45.672 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.664+0000 7f6dac8db740 1 -- 172.21.15.16:0/3835039535 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555563a5e780 msgr2=0x555563a60c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:45.672 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.664+0000 7f6dac8db740 1 --2- 172.21.15.16:0/3835039535 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555563a5e780 0x555563a60c40 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f6d9800cb70 tx=0x7f6d9800e720 comp rx=0 tx=0).stop 2024-09-12T02:15:45.672 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.664+0000 7f6dac8db740 1 -- 172.21.15.16:0/3835039535 shutdown_connections 2024-09-12T02:15:45.672 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.664+0000 7f6dac8db740 1 --2- 172.21.15.16:0/3835039535 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555563a5e780 0x555563a60c40 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:45.672 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.664+0000 7f6dac8db740 1 --2- 172.21.15.16:0/3835039535 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555563a596f0 0x555563a5e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:45.673 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.664+0000 7f6dac8db740 1 --2- 172.21.15.16:0/3835039535 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555563a58db0 0x555563a591b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:45.673 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.664+0000 7f6dac8db740 1 -- 172.21.15.16:0/3835039535 >> 172.21.15.16:0/3835039535 conn(0x555563a50010 msgr2=0x555563a50410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:15:45.673 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.664+0000 7f6dac8db740 1 -- 172.21.15.16:0/3835039535 shutdown_connections 2024-09-12T02:15:45.673 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.664+0000 7f6dac8db740 1 -- 172.21.15.16:0/3835039535 wait complete. 2024-09-12T02:15:45.673 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.664+0000 7f6dac8db740 1 -- start start 2024-09-12T02:15:45.673 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.665+0000 7f6dac8db740 1 Processor -- start 2024-09-12T02:15:45.673 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.665+0000 7f6dac8db740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555563a58db0 0x555563a62f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:45.676 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.665+0000 7f6dac8db740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555563a596f0 0x555563a63450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:45.676 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.665+0000 7f6dac8db740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555563a5e780 0x555563a6a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:45.676 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.665+0000 7f6dac8db740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x555563a61750 con 0x555563a596f0 2024-09-12T02:15:45.676 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.665+0000 7f6dac8db740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x555563941290 con 0x555563a5e780 2024-09-12T02:15:45.676 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.665+0000 7f6dac8db740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x555563a618d0 con 0x555563a58db0 2024-09-12T02:15:45.676 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.665+0000 7f6dac8db740 10 client.0 _ll_register_callbacks cb 0x555563972140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:15:45.676 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.665+0000 7f6da2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555563a596f0 0x555563a63450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:45.677 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.665+0000 7f6da37fe640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555563a58db0 0x555563a62f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:45.677 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.665+0000 7f6da2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555563a596f0 0x555563a63450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53646/0 (socket says 172.21.15.16:53646) 2024-09-12T02:15:45.677 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.665+0000 7f6da2ffd640 1 -- 172.21.15.16:0/3648637845 learned_addr learned my addr 172.21.15.16:0/3648637845 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:15:45.677 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.665+0000 7f6da3fff640 1 --2- 172.21.15.16:0/3648637845 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555563a5e780 0x555563a6a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:45.677 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.665+0000 7f6da2ffd640 1 -- 172.21.15.16:0/3648637845 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555563a58db0 msgr2=0x555563a62f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:15:45.677 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.665+0000 7f6da2ffd640 1 --2- 172.21.15.16:0/3648637845 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555563a58db0 0x555563a62f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:45.677 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.665+0000 7f6da2ffd640 1 -- 172.21.15.16:0/3648637845 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555563a5e780 msgr2=0x555563a6a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:15:45.678 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.665+0000 7f6da2ffd640 1 --2- 172.21.15.16:0/3648637845 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555563a5e780 0x555563a6a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:45.678 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.665+0000 7f6da2ffd640 1 -- 172.21.15.16:0/3648637845 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5555636cb210 con 0x555563a596f0 2024-09-12T02:15:45.678 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.665+0000 7f6da37fe640 1 --2- 172.21.15.16:0/3648637845 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555563a58db0 0x555563a62f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:15:45.678 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.665+0000 7f6da2ffd640 1 --2- 172.21.15.16:0/3648637845 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555563a596f0 0x555563a63450 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7f6d9801cae0 tx=0x7f6d9801cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=915f70bfdc4b3829 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:15:45.678 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.665+0000 7f6d82ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:15:45.678 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.666+0000 7f6d82ffd640 1 -- 172.21.15.16:0/3648637845 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d9800ce70 con 0x555563a596f0 2024-09-12T02:15:45.678 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.666+0000 7f6dac8db740 1 -- 172.21.15.16:0/3648637845 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d980043c0 con 0x555563a596f0 2024-09-12T02:15:45.678 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.666+0000 7f6d82ffd640 1 -- 172.21.15.16:0/3648637845 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f6d9800c6b0 con 0x555563a596f0 2024-09-12T02:15:45.678 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.666+0000 7f6d82ffd640 1 -- 172.21.15.16:0/3648637845 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6d9801cd30 con 0x555563a596f0 2024-09-12T02:15:45.678 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.666+0000 7f6dac8db740 10 client.20046 fetch_fsmap learned FSMap version 25 2024-09-12T02:15:45.679 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.666+0000 7f6dac8db740 1 -- 172.21.15.16:0/3648637845 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555563797620 con 0x555563a596f0 2024-09-12T02:15:45.679 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.666+0000 7f6d82ffd640 1 -- 172.21.15.16:0/3648637845 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d980043c0 con 0x555563a596f0 2024-09-12T02:15:45.679 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.666+0000 7f6d82ffd640 1 -- 172.21.15.16:0/3648637845 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f6d9801a460 con 0x555563a596f0 2024-09-12T02:15:45.679 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.666+0000 7f6dac8db740 10 client.20046 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:15:45.679 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.666+0000 7f6dac8db740 10 client.20046 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:15:45.679 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.666+0000 7f6dac8db740 20 client.20046 populate_metadata read hostname 'smithi016' 2024-09-12T02:15:45.679 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.666+0000 7f6dac8db740 1 --2- 172.21.15.16:0/3648637845 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x555563a6e910 0x555563a70d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:45.679 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.666+0000 7f6dac8db740 4 client.20046 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:15:45.679 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.666+0000 7f6dac8db740 1 -- 172.21.15.16:0/3648637845 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5555639c5a20 con 0x555563a6e910 2024-09-12T02:15:45.679 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.667+0000 7f6da37fe640 1 --2- 172.21.15.16:0/3648637845 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x555563a6e910 0x555563a70d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:45.679 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.667+0000 7f6d82ffd640 10 client.20046 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:15:45.679 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.667+0000 7f6da37fe640 1 --2- 172.21.15.16:0/3648637845 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x555563a6e910 0x555563a70d20 crc :-1 s=READY pgs=411 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c2e21e500aebc39 server_cookie=9959b6ddbc84df04 in_seq=0 out_seq=0 2024-09-12T02:15:45.680 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.668+0000 7f6d82ffd640 1 -- 172.21.15.16:0/3648637845 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5555639c5a20 con 0x555563a6e910 2024-09-12T02:15:45.680 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.668+0000 7f6d82ffd640 10 client.20046 handle_command_reply: tid=0 2024-09-12T02:15:45.680 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.668+0000 7f6dac8db740 1 client.20046 shutdown 2024-09-12T02:15:45.680 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.668+0000 7f6dac8db740 1 -- 172.21.15.16:0/3648637845 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555563a596f0 msgr2=0x555563a63450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:45.680 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.668+0000 7f6dac8db740 1 --2- 172.21.15.16:0/3648637845 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555563a596f0 0x555563a63450 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7f6d9801cae0 tx=0x7f6d9801cb10 comp rx=0 tx=0).stop 2024-09-12T02:15:45.680 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.668+0000 7f6dac8db740 1 -- 172.21.15.16:0/3648637845 shutdown_connections 2024-09-12T02:15:45.680 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.668+0000 7f6dac8db740 1 --2- 172.21.15.16:0/3648637845 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x555563a6e910 0x555563a70d20 crc :-1 s=READY pgs=411 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:45.680 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.668+0000 7f6dac8db740 1 --2- 172.21.15.16:0/3648637845 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555563a5e780 0x555563a6a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:45.680 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.668+0000 7f6dac8db740 1 --2- 172.21.15.16:0/3648637845 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555563a596f0 0x555563a63450 unknown :-1 s=CLOSED pgs=1874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:45.680 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.668+0000 7f6d82ffd640 0 client.20046 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:15:45.680 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.668+0000 7f6dac8db740 1 --2- 172.21.15.16:0/3648637845 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555563a58db0 0x555563a62f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:45.680 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.668+0000 7f6dac8db740 1 -- 172.21.15.16:0/3648637845 >> 172.21.15.16:0/3648637845 conn(0x555563a50010 msgr2=0x555563a51820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:15:45.681 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.668+0000 7f6dac8db740 1 -- 172.21.15.16:0/3648637845 shutdown_connections 2024-09-12T02:15:45.681 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.668+0000 7f6dac8db740 1 -- 172.21.15.16:0/3648637845 wait complete. 2024-09-12T02:15:45.681 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.668+0000 7f6dac8db740 20 client.20046 trim_cache size 0 max 16384 2024-09-12T02:15:45.685 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.683+0000 7f6dac8db740 1 Processor -- start 2024-09-12T02:15:45.685 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.683+0000 7f6dac8db740 1 -- start start 2024-09-12T02:15:45.685 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.683+0000 7f6dac8db740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555563a58db0 0x555563a70d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:45.685 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.683+0000 7f6dac8db740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555563a596f0 0x555563a6a450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:45.685 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.683+0000 7f6dac8db740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555563a5e780 0x555563a4e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:45.685 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.683+0000 7f6dac8db740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6da41a9510 con 0x555563a596f0 2024-09-12T02:15:45.685 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.683+0000 7f6dac8db740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6da41a6de0 con 0x555563a58db0 2024-09-12T02:15:45.685 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.683+0000 7f6dac8db740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f6da41a4c50 con 0x555563a5e780 2024-09-12T02:15:45.685 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.683+0000 7f6da2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555563a596f0 0x555563a6a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:45.686 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.683+0000 7f6da2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555563a596f0 0x555563a6a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53648/0 (socket says 172.21.15.16:53648) 2024-09-12T02:15:45.686 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.683+0000 7f6da2ffd640 1 -- 172.21.15.16:0/2908834354 learned_addr learned my addr 172.21.15.16:0/2908834354 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:15:45.686 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.683+0000 7f6da37fe640 1 --2- 172.21.15.16:0/2908834354 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555563a58db0 0x555563a70d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:45.686 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.683+0000 7f6da3fff640 1 --2- 172.21.15.16:0/2908834354 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555563a5e780 0x555563a4e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:45.686 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.683+0000 7f6da2ffd640 1 -- 172.21.15.16:0/2908834354 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555563a5e780 msgr2=0x555563a4e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:15:45.686 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.683+0000 7f6da2ffd640 1 --2- 172.21.15.16:0/2908834354 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555563a5e780 0x555563a4e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:45.686 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.683+0000 7f6da2ffd640 1 -- 172.21.15.16:0/2908834354 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555563a58db0 msgr2=0x555563a70d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:15:45.686 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.683+0000 7f6da2ffd640 1 --2- 172.21.15.16:0/2908834354 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555563a58db0 0x555563a70d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:45.686 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.683+0000 7f6da2ffd640 1 -- 172.21.15.16:0/2908834354 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555563a61ed0 con 0x555563a596f0 2024-09-12T02:15:45.686 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.683+0000 7f6da3fff640 1 --2- 172.21.15.16:0/2908834354 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555563a5e780 0x555563a4e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:15:45.686 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.683+0000 7f6da37fe640 1 --2- 172.21.15.16:0/2908834354 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555563a58db0 0x555563a70d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:15:45.687 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.684+0000 7f6da2ffd640 1 --2- 172.21.15.16:0/2908834354 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555563a596f0 0x555563a6a450 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7f6d9800cb70 tx=0x7f6d9800c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf003d545576c909 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:15:45.687 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.684+0000 7f6d897fa640 1 -- 172.21.15.16:0/2908834354 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d9800c6b0 con 0x555563a596f0 2024-09-12T02:15:45.687 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.684+0000 7f6d897fa640 1 -- 172.21.15.16:0/2908834354 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d9801a460 con 0x555563a596f0 2024-09-12T02:15:45.687 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.684+0000 7f6d897fa640 1 -- 172.21.15.16:0/2908834354 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d98010910 con 0x555563a596f0 2024-09-12T02:15:45.690 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.684+0000 7f6dac8db740 1 -- 172.21.15.16:0/2908834354 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555563a596f0 msgr2=0x555563a6a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:45.690 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.684+0000 7f6dac8db740 1 --2- 172.21.15.16:0/2908834354 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555563a596f0 0x555563a6a450 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7f6d9800cb70 tx=0x7f6d9800c470 comp rx=0 tx=0).stop 2024-09-12T02:15:45.690 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.685+0000 7f6dac8db740 1 -- 172.21.15.16:0/2908834354 shutdown_connections 2024-09-12T02:15:45.690 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.685+0000 7f6dac8db740 1 --2- 172.21.15.16:0/2908834354 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555563a5e780 0x555563a4e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:45.690 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.685+0000 7f6dac8db740 1 --2- 172.21.15.16:0/2908834354 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555563a596f0 0x555563a6a450 unknown :-1 s=CLOSED pgs=1875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:45.690 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.685+0000 7f6dac8db740 1 --2- 172.21.15.16:0/2908834354 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555563a58db0 0x555563a70d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:45.690 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.685+0000 7f6dac8db740 1 -- 172.21.15.16:0/2908834354 >> 172.21.15.16:0/2908834354 conn(0x555563a50010 msgr2=0x555563a62f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:15:45.690 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.685+0000 7f6dac8db740 1 -- 172.21.15.16:0/2908834354 shutdown_connections 2024-09-12T02:15:45.690 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.685+0000 7f6dac8db740 1 -- 172.21.15.16:0/2908834354 wait complete. 2024-09-12T02:15:45.690 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.685+0000 7f6dac8db740 1 -- start start 2024-09-12T02:15:45.691 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.685+0000 7f6dac8db740 1 Processor -- start 2024-09-12T02:15:45.691 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.686+0000 7f6dac8db740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555563a58db0 0x555563a67a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:45.691 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.686+0000 7f6dac8db740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555563a596f0 0x555563a67fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:45.691 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.686+0000 7f6dac8db740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555563a5e780 0x555563a68f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:45.691 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.686+0000 7f6dac8db740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6da41a52a0 con 0x555563a5e780 2024-09-12T02:15:45.691 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.686+0000 7f6dac8db740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6da41a6710 con 0x555563a596f0 2024-09-12T02:15:45.691 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.686+0000 7f6dac8db740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f6da41a5970 con 0x555563a58db0 2024-09-12T02:15:45.691 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.686+0000 7f6dac8db740 10 client.0 _ll_register_callbacks cb 0x7f6d98014070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:15:45.691 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.686+0000 7f6da3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555563a5e780 0x555563a68f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:45.691 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.686+0000 7f6da3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555563a5e780 0x555563a68f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53652/0 (socket says 172.21.15.16:53652) 2024-09-12T02:15:45.691 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.686+0000 7f6da3fff640 1 -- 172.21.15.16:0/523406313 learned_addr learned my addr 172.21.15.16:0/523406313 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:15:45.691 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.686+0000 7f6da2ffd640 1 --2- 172.21.15.16:0/523406313 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555563a596f0 0x555563a67fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:45.692 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.686+0000 7f6da37fe640 1 --2- 172.21.15.16:0/523406313 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555563a58db0 0x555563a67a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:45.692 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.686+0000 7f6da2ffd640 1 -- 172.21.15.16:0/523406313 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555563a58db0 msgr2=0x555563a67a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:15:45.692 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.686+0000 7f6da2ffd640 1 --2- 172.21.15.16:0/523406313 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555563a58db0 0x555563a67a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:45.692 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.686+0000 7f6da2ffd640 1 -- 172.21.15.16:0/523406313 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555563a5e780 msgr2=0x555563a68f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:15:45.692 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.686+0000 7f6da2ffd640 1 --2- 172.21.15.16:0/523406313 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555563a5e780 0x555563a68f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:45.692 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.686+0000 7f6da2ffd640 1 -- 172.21.15.16:0/523406313 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555563a695a0 con 0x555563a596f0 2024-09-12T02:15:45.692 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.686+0000 7f6da3fff640 1 --2- 172.21.15.16:0/523406313 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555563a5e780 0x555563a68f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:15:45.692 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.686+0000 7f6da2ffd640 1 --2- 172.21.15.16:0/523406313 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555563a596f0 0x555563a67fd0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f6d9800c3d0 tx=0x7f6d9801a610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=74529205e099074c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:15:45.692 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.686+0000 7f6d82ffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-12T02:15:45.692 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.687+0000 7f6d82ffd640 1 -- 172.21.15.16:0/523406313 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d9803f070 con 0x555563a596f0 2024-09-12T02:15:45.692 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.687+0000 7f6dac8db740 1 -- 172.21.15.16:0/523406313 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d9801a460 con 0x555563a596f0 2024-09-12T02:15:45.692 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.687+0000 7f6d82ffd640 1 -- 172.21.15.16:0/523406313 <== mon.2 v2:172.21.15.33:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f6d9801aaf0 con 0x555563a596f0 2024-09-12T02:15:45.692 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.687+0000 7f6d82ffd640 1 -- 172.21.15.16:0/523406313 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6d9804b050 con 0x555563a596f0 2024-09-12T02:15:45.693 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.687+0000 7f6dac8db740 10 client.28865 fetch_fsmap learned FSMap version 25 2024-09-12T02:15:45.693 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.687+0000 7f6dac8db740 1 -- 172.21.15.16:0/523406313 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555563ad9600 con 0x555563a596f0 2024-09-12T02:15:45.693 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.687+0000 7f6d82ffd640 1 -- 172.21.15.16:0/523406313 <== mon.2 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d9801a460 con 0x555563a596f0 2024-09-12T02:15:45.693 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.687+0000 7f6d82ffd640 1 -- 172.21.15.16:0/523406313 <== mon.2 v2:172.21.15.33:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f6d98026420 con 0x555563a596f0 2024-09-12T02:15:45.693 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.687+0000 7f6dac8db740 10 client.28865 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:15:45.693 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.687+0000 7f6dac8db740 10 client.28865 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:15:45.693 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.687+0000 7f6dac8db740 20 client.28865 populate_metadata read hostname 'smithi016' 2024-09-12T02:15:45.693 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.687+0000 7f6dac8db740 1 --2- 172.21.15.16:0/523406313 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x555563ad9e40 0x555563adc260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:45.693 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.687+0000 7f6dac8db740 4 client.28865 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:15:45.693 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.687+0000 7f6dac8db740 1 -- 172.21.15.16:0/523406313 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5555639bfd70 con 0x555563ad9e40 2024-09-12T02:15:45.693 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.688+0000 7f6da37fe640 1 --2- 172.21.15.16:0/523406313 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x555563ad9e40 0x555563adc260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:45.693 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.688+0000 7f6d82ffd640 10 client.28865 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:15:45.693 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.688+0000 7f6da37fe640 1 --2- 172.21.15.16:0/523406313 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x555563ad9e40 0x555563adc260 crc :-1 s=READY pgs=412 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d48f662f52c142c1 server_cookie=3a26ca5ba8c4518b in_seq=0 out_seq=0 2024-09-12T02:15:45.701 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.699+0000 7f6d82ffd640 1 -- 172.21.15.16:0/523406313 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5555639bfd70 con 0x555563ad9e40 2024-09-12T02:15:45.702 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.699+0000 7f6d82ffd640 10 client.28865 handle_command_reply: tid=0 2024-09-12T02:15:45.702 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.699+0000 7f6dac8db740 1 client.28865 shutdown 2024-09-12T02:15:45.702 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.700+0000 7f6dac8db740 1 -- 172.21.15.16:0/523406313 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555563a596f0 msgr2=0x555563a67fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:45.702 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.700+0000 7f6dac8db740 1 --2- 172.21.15.16:0/523406313 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555563a596f0 0x555563a67fd0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f6d9800c3d0 tx=0x7f6d9801a610 comp rx=0 tx=0).stop 2024-09-12T02:15:45.702 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.700+0000 7f6dac8db740 1 -- 172.21.15.16:0/523406313 shutdown_connections 2024-09-12T02:15:45.702 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.700+0000 7f6dac8db740 1 --2- 172.21.15.16:0/523406313 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x555563ad9e40 0x555563adc260 crc :-1 s=READY pgs=412 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:45.702 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.700+0000 7f6dac8db740 1 --2- 172.21.15.16:0/523406313 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555563a5e780 0x555563a68f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:45.702 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.700+0000 7f6d82ffd640 0 client.28865 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:15:45.702 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.700+0000 7f6dac8db740 1 --2- 172.21.15.16:0/523406313 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555563a596f0 0x555563a67fd0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:45.702 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.700+0000 7f6dac8db740 1 --2- 172.21.15.16:0/523406313 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555563a58db0 0x555563a67a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:45.702 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.700+0000 7f6dac8db740 1 -- 172.21.15.16:0/523406313 >> 172.21.15.16:0/523406313 conn(0x555563a50010 msgr2=0x555563a4f1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:15:45.702 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.700+0000 7f6dac8db740 1 -- 172.21.15.16:0/523406313 shutdown_connections 2024-09-12T02:15:45.702 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.700+0000 7f6dac8db740 1 -- 172.21.15.16:0/523406313 wait complete. 2024-09-12T02:15:45.703 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.700+0000 7f6dac8db740 20 client.28865 trim_cache size 0 max 16384 2024-09-12T02:15:45.703 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-09-12T02:15:45.703 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-09-12T02:15:45.703 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-09-12T02:15:45.703 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-09-12T02:15:45.704 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.701+0000 7f6da9b42640 1 -- 172.21.15.16:0/2775163088 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f6d8c0a2c30 msgr2=0x7f6d8c0a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:45.704 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.701+0000 7f6da9b42640 1 --2- 172.21.15.16:0/2775163088 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f6d8c0a2c30 0x7f6d8c0a50f0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f6d9c009fd0 tx=0x7f6d9c004ab0 comp rx=0 tx=0).stop 2024-09-12T02:15:45.704 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.701+0000 7f6da9b42640 1 -- 172.21.15.16:0/2775163088 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6da41025c0 msgr2=0x7f6da4197dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:45.704 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.701+0000 7f6da9b42640 1 --2- 172.21.15.16:0/2775163088 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6da41025c0 0x7f6da4197dc0 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7f6d94009230 tx=0x7f6d94004360 comp rx=0 tx=0).stop 2024-09-12T02:15:45.704 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.701+0000 7f6da9b42640 1 -- 172.21.15.16:0/2775163088 shutdown_connections 2024-09-12T02:15:45.704 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.701+0000 7f6da9b42640 1 --2- 172.21.15.16:0/2775163088 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f6d8c0a2c30 0x7f6d8c0a50f0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:45.704 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.701+0000 7f6da9b42640 1 --2- 172.21.15.16:0/2775163088 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6da4107820 0x7f6da419f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:45.705 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.701+0000 7f6da9b42640 1 --2- 172.21.15.16:0/2775163088 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6da4104ef0 0x7f6da4198300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:45.705 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.701+0000 7f6da9b42640 1 --2- 172.21.15.16:0/2775163088 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6da41025c0 0x7f6da4197dc0 unknown :-1 s=CLOSED pgs=1873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:45.705 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.701+0000 7f6da9b42640 1 -- 172.21.15.16:0/2775163088 >> 172.21.15.16:0/2775163088 conn(0x7f6da40fc1e0 msgr2=0x7f6da4102e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:15:45.705 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.701+0000 7f6da9b42640 1 -- 172.21.15.16:0/2775163088 shutdown_connections 2024-09-12T02:15:45.705 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:45.701+0000 7f6da9b42640 1 -- 172.21.15.16:0/2775163088 wait complete. 2024-09-12T02:15:45.766 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:15:46.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.237+0000 7f2aaec18640 1 -- 172.21.15.16:0/3570967487 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2aa8101dd0 msgr2=0x7f2aa8105900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:46.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.237+0000 7f2aaec18640 1 --2- 172.21.15.16:0/3570967487 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2aa8101dd0 0x7f2aa8105900 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7f2a98009ca0 tx=0x7f2a9802a8c0 comp rx=0 tx=0).stop 2024-09-12T02:15:46.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.237+0000 7f2aaec18640 1 -- 172.21.15.16:0/3570967487 shutdown_connections 2024-09-12T02:15:46.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.237+0000 7f2aaec18640 1 --2- 172.21.15.16:0/3570967487 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2aa8106030 0x7f2aa810dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:46.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.237+0000 7f2aaec18640 1 --2- 172.21.15.16:0/3570967487 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2aa8101dd0 0x7f2aa8105900 unknown :-1 s=CLOSED pgs=1876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:46.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.237+0000 7f2aaec18640 1 --2- 172.21.15.16:0/3570967487 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2aa8101420 0x7f2aa8101800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:46.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.237+0000 7f2aaec18640 1 -- 172.21.15.16:0/3570967487 >> 172.21.15.16:0/3570967487 conn(0x7f2aa80fbee0 msgr2=0x7f2aa80fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:15:46.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.237+0000 7f2aaec18640 1 -- 172.21.15.16:0/3570967487 shutdown_connections 2024-09-12T02:15:46.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.237+0000 7f2aaec18640 1 -- 172.21.15.16:0/3570967487 wait complete. 2024-09-12T02:15:46.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.238+0000 7f2aaec18640 1 Processor -- start 2024-09-12T02:15:46.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.238+0000 7f2aaec18640 1 -- start start 2024-09-12T02:15:46.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.238+0000 7f2aaec18640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2aa8101420 0x7f2aa8199520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:46.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.238+0000 7f2aaec18640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2aa8101dd0 0x7f2aa8199a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:46.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.238+0000 7f2aaec18640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2aa8106030 0x7f2aa81a45a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:46.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.238+0000 7f2aaec18640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2aa8111090 con 0x7f2aa8101dd0 2024-09-12T02:15:46.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.238+0000 7f2aaec18640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2aa8110f10 con 0x7f2aa8106030 2024-09-12T02:15:46.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.238+0000 7f2aaec18640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f2aa8111210 con 0x7f2aa8101420 2024-09-12T02:15:46.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.238+0000 7f2aa7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2aa8101dd0 0x7f2aa8199a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:46.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.238+0000 7f2aa7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2aa8101dd0 0x7f2aa8199a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53672/0 (socket says 172.21.15.16:53672) 2024-09-12T02:15:46.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.238+0000 7f2aa7fff640 1 -- 172.21.15.16:0/1160074624 learned_addr learned my addr 172.21.15.16:0/1160074624 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:15:46.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.238+0000 7f2aad1b5640 1 --2- 172.21.15.16:0/1160074624 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2aa8106030 0x7f2aa81a45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:46.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.238+0000 7f2aac9b4640 1 --2- 172.21.15.16:0/1160074624 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2aa8101420 0x7f2aa8199520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:46.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.238+0000 7f2aa7fff640 1 -- 172.21.15.16:0/1160074624 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2aa8101420 msgr2=0x7f2aa8199520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:46.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.238+0000 7f2aa7fff640 1 --2- 172.21.15.16:0/1160074624 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2aa8101420 0x7f2aa8199520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:46.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.238+0000 7f2aa7fff640 1 -- 172.21.15.16:0/1160074624 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2aa8106030 msgr2=0x7f2aa81a45a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:46.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.238+0000 7f2aa7fff640 1 --2- 172.21.15.16:0/1160074624 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2aa8106030 0x7f2aa81a45a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:46.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.238+0000 7f2aa7fff640 1 -- 172.21.15.16:0/1160074624 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2aa81a4ae0 con 0x7f2aa8101dd0 2024-09-12T02:15:46.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.238+0000 7f2aad1b5640 1 --2- 172.21.15.16:0/1160074624 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2aa8106030 0x7f2aa81a45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:15:46.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.238+0000 7f2aac9b4640 1 --2- 172.21.15.16:0/1160074624 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2aa8101420 0x7f2aa8199520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:15:46.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.239+0000 7f2aa7fff640 1 --2- 172.21.15.16:0/1160074624 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2aa8101dd0 0x7f2aa8199a60 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7f2a9802ade0 tx=0x7f2a980319f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:15:46.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.239+0000 7f2aa5ffb640 1 -- 172.21.15.16:0/1160074624 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a9802fa20 con 0x7f2aa8101dd0 2024-09-12T02:15:46.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.239+0000 7f2aa5ffb640 1 -- 172.21.15.16:0/1160074624 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a9802fbc0 con 0x7f2aa8101dd0 2024-09-12T02:15:46.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.239+0000 7f2aaec18640 1 -- 172.21.15.16:0/1160074624 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2aa81a4cb0 con 0x7f2aa8101dd0 2024-09-12T02:15:46.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.239+0000 7f2aa5ffb640 1 -- 172.21.15.16:0/1160074624 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a980467b0 con 0x7f2aa8101dd0 2024-09-12T02:15:46.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.239+0000 7f2aaec18640 1 -- 172.21.15.16:0/1160074624 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2aa81a5190 con 0x7f2aa8101dd0 2024-09-12T02:15:46.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.241+0000 7f2aa5ffb640 1 -- 172.21.15.16:0/1160074624 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f2a98046950 con 0x7f2aa8101dd0 2024-09-12T02:15:46.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.242+0000 7f2ab0ff1740 1 Processor -- start 2024-09-12T02:15:46.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.242+0000 7f2ab0ff1740 1 -- start start 2024-09-12T02:15:46.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.242+0000 7f2ab0ff1740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55905fc9db40 0x55905fc9c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:46.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.242+0000 7f2ab0ff1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55905fc9c5d0 0x55905fc9ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:46.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.242+0000 7f2ab0ff1740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55905fca7370 0x55905fca9760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:46.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.242+0000 7f2ab0ff1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55905fc022f0 con 0x55905fc9c5d0 2024-09-12T02:15:46.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.242+0000 7f2ab0ff1740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55905fba9cf0 con 0x55905fca7370 2024-09-12T02:15:46.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.242+0000 7f2ab0ff1740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55905fbfe2b0 con 0x55905fc9db40 2024-09-12T02:15:46.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.242+0000 7f2aac9b4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55905fc9c5d0 0x55905fc9ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:46.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.242+0000 7f2aa5ffb640 1 --2- 172.21.15.16:0/1160074624 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f2a940a2c00 0x7f2a940a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:46.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.242+0000 7f2aac9b4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55905fc9c5d0 0x55905fc9ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53682/0 (socket says 172.21.15.16:53682) 2024-09-12T02:15:46.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.242+0000 7f2aac9b4640 1 -- 172.21.15.16:0/3252128428 learned_addr learned my addr 172.21.15.16:0/3252128428 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:15:46.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.242+0000 7f2aa7fff640 1 --2- 172.21.15.16:0/3252128428 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55905fca7370 0x55905fca9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:46.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.242+0000 7f2aad1b5640 1 --2- 172.21.15.16:0/3252128428 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55905fc9db40 0x55905fc9c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:46.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.242+0000 7f2aad1b5640 1 --2- 172.21.15.16:0/1160074624 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f2a940a2c00 0x7f2a940a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:46.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.242+0000 7f2aa5ffb640 1 -- 172.21.15.16:0/1160074624 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f2a980f0490 con 0x7f2aa8101dd0 2024-09-12T02:15:46.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.242+0000 7f2aac9b4640 1 -- 172.21.15.16:0/3252128428 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55905fc9db40 msgr2=0x55905fc9c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:15:46.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.242+0000 7f2aac9b4640 1 --2- 172.21.15.16:0/3252128428 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55905fc9db40 0x55905fc9c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:46.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.242+0000 7f2aac9b4640 1 -- 172.21.15.16:0/3252128428 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55905fca7370 msgr2=0x55905fca9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:15:46.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.242+0000 7f2aac9b4640 1 --2- 172.21.15.16:0/3252128428 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55905fca7370 0x55905fca9760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:46.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.242+0000 7f2aac9b4640 1 -- 172.21.15.16:0/3252128428 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55905fb8e3a0 con 0x55905fc9c5d0 2024-09-12T02:15:46.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.243+0000 7f2aac9b4640 1 --2- 172.21.15.16:0/3252128428 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55905fc9c5d0 0x55905fc9ca10 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7f2aa0009c50 tx=0x7f2aa000e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=832a7140a23b103b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:15:46.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.243+0000 7f2aad1b5640 1 --2- 172.21.15.16:0/1160074624 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f2a940a2c00 0x7f2a940a50c0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f2a90005fd0 tx=0x7f2a90005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:15:46.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.243+0000 7f2a9effd640 1 -- 172.21.15.16:0/3252128428 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aa0016070 con 0x55905fc9c5d0 2024-09-12T02:15:46.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.243+0000 7f2a9effd640 1 -- 172.21.15.16:0/3252128428 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2aa0011950 con 0x55905fc9c5d0 2024-09-12T02:15:46.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.243+0000 7f2a9effd640 1 -- 172.21.15.16:0/3252128428 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aa00108f0 con 0x55905fc9c5d0 2024-09-12T02:15:46.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.244+0000 7f2ab0ff1740 1 -- 172.21.15.16:0/3252128428 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55905fc9c5d0 msgr2=0x55905fc9ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:46.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.244+0000 7f2ab0ff1740 1 --2- 172.21.15.16:0/3252128428 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55905fc9c5d0 0x55905fc9ca10 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7f2aa0009c50 tx=0x7f2aa000e6c0 comp rx=0 tx=0).stop 2024-09-12T02:15:46.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.244+0000 7f2ab0ff1740 1 -- 172.21.15.16:0/3252128428 shutdown_connections 2024-09-12T02:15:46.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.244+0000 7f2ab0ff1740 1 --2- 172.21.15.16:0/3252128428 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55905fca7370 0x55905fca9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:46.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.244+0000 7f2ab0ff1740 1 --2- 172.21.15.16:0/3252128428 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55905fc9c5d0 0x55905fc9ca10 unknown :-1 s=CLOSED pgs=1878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:46.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.244+0000 7f2ab0ff1740 1 --2- 172.21.15.16:0/3252128428 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55905fc9db40 0x55905fc9c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:46.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.244+0000 7f2ab0ff1740 1 -- 172.21.15.16:0/3252128428 >> 172.21.15.16:0/3252128428 conn(0x55905fc65310 msgr2=0x55905fc98c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:15:46.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.244+0000 7f2ab0ff1740 1 -- 172.21.15.16:0/3252128428 shutdown_connections 2024-09-12T02:15:46.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.244+0000 7f2ab0ff1740 1 -- 172.21.15.16:0/3252128428 wait complete. 2024-09-12T02:15:46.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.244+0000 7f2ab0ff1740 1 -- start start 2024-09-12T02:15:46.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.244+0000 7f2ab0ff1740 1 Processor -- start 2024-09-12T02:15:46.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.245+0000 7f2ab0ff1740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55905fc9c5d0 0x55905fca6d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:46.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.245+0000 7f2ab0ff1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55905fc9db40 0x55905fca3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:46.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.245+0000 7f2ab0ff1740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55905fca7370 0x55905fca45d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:46.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.245+0000 7f2ab0ff1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55905fcaa070 con 0x55905fc9db40 2024-09-12T02:15:46.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.245+0000 7f2ab0ff1740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55905fbbd060 con 0x55905fca7370 2024-09-12T02:15:46.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.245+0000 7f2ab0ff1740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55905fcaa1f0 con 0x55905fc9c5d0 2024-09-12T02:15:46.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.245+0000 7f2ab0ff1740 10 client.0 _ll_register_callbacks cb 0x55905f9108c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:15:46.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.245+0000 7f2aa7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55905fc9db40 0x55905fca3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:46.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.245+0000 7f2aa7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55905fc9db40 0x55905fca3fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53690/0 (socket says 172.21.15.16:53690) 2024-09-12T02:15:46.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.245+0000 7f2aa7fff640 1 -- 172.21.15.16:0/1936888154 learned_addr learned my addr 172.21.15.16:0/1936888154 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:15:46.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.245+0000 7f2aad1b5640 1 --2- 172.21.15.16:0/1936888154 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55905fca7370 0x55905fca45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:46.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.245+0000 7f2aac9b4640 1 --2- 172.21.15.16:0/1936888154 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55905fc9c5d0 0x55905fca6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:46.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.245+0000 7f2aa7fff640 1 -- 172.21.15.16:0/1936888154 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55905fc9c5d0 msgr2=0x55905fca6d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:15:46.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.245+0000 7f2aa7fff640 1 --2- 172.21.15.16:0/1936888154 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55905fc9c5d0 0x55905fca6d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:46.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.245+0000 7f2aa7fff640 1 -- 172.21.15.16:0/1936888154 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55905fca7370 msgr2=0x55905fca45d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:15:46.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.245+0000 7f2aa7fff640 1 --2- 172.21.15.16:0/1936888154 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55905fca7370 0x55905fca45d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:46.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.245+0000 7f2aa7fff640 1 -- 172.21.15.16:0/1936888154 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55905f9733c0 con 0x55905fc9db40 2024-09-12T02:15:46.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.245+0000 7f2aac9b4640 1 --2- 172.21.15.16:0/1936888154 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55905fc9c5d0 0x55905fca6d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:15:46.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.245+0000 7f2aa7fff640 1 --2- 172.21.15.16:0/1936888154 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55905fc9db40 0x55905fca3fe0 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7f2a98000f80 tx=0x7f2a98087e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0b1239db871ad0d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:15:46.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.245+0000 7f2a87fff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:15:46.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.245+0000 7f2a87fff640 1 -- 172.21.15.16:0/1936888154 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a98087f00 con 0x55905fc9db40 2024-09-12T02:15:46.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.245+0000 7f2a87fff640 1 -- 172.21.15.16:0/1936888154 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f2a980f0110 con 0x55905fc9db40 2024-09-12T02:15:46.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.245+0000 7f2ab0ff1740 1 -- 172.21.15.16:0/1936888154 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2aa0011950 con 0x55905fc9db40 2024-09-12T02:15:46.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.246+0000 7f2a87fff640 1 -- 172.21.15.16:0/1936888154 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2a980bd520 con 0x55905fc9db40 2024-09-12T02:15:46.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.246+0000 7f2ab0ff1740 10 client.20064 fetch_fsmap learned FSMap version 25 2024-09-12T02:15:46.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.246+0000 7f2ab0ff1740 1 -- 172.21.15.16:0/1936888154 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55905fcaa070 con 0x55905fc9db40 2024-09-12T02:15:46.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.246+0000 7f2a87fff640 1 -- 172.21.15.16:0/1936888154 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2aa0011950 con 0x55905fc9db40 2024-09-12T02:15:46.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.246+0000 7f2a87fff640 1 -- 172.21.15.16:0/1936888154 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f2a9804e030 con 0x55905fc9db40 2024-09-12T02:15:46.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.246+0000 7f2ab0ff1740 10 client.20064 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:15:46.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.246+0000 7f2ab0ff1740 10 client.20064 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:15:46.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.246+0000 7f2ab0ff1740 20 client.20064 populate_metadata read hostname 'smithi016' 2024-09-12T02:15:46.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.246+0000 7f2ab0ff1740 1 --2- 172.21.15.16:0/1936888154 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55905fcb6ea0 0x55905fcb92f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:46.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.246+0000 7f2ab0ff1740 4 client.20064 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:15:46.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.246+0000 7f2ab0ff1740 1 -- 172.21.15.16:0/1936888154 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55905fc0f860 con 0x55905fcb6ea0 2024-09-12T02:15:46.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.246+0000 7f2aac9b4640 1 --2- 172.21.15.16:0/1936888154 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55905fcb6ea0 0x55905fcb92f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:46.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.247+0000 7f2a87fff640 10 client.20064 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:15:46.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.247+0000 7f2aac9b4640 1 --2- 172.21.15.16:0/1936888154 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55905fcb6ea0 0x55905fcb92f0 crc :-1 s=READY pgs=413 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4dcbbf62bfd6105e server_cookie=f438e967b7e8c50 in_seq=0 out_seq=0 2024-09-12T02:15:46.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.247+0000 7f2a87fff640 1 -- 172.21.15.16:0/1936888154 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55905fc0f860 con 0x55905fcb6ea0 2024-09-12T02:15:46.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.247+0000 7f2a87fff640 10 client.20064 handle_command_reply: tid=0 2024-09-12T02:15:46.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.248+0000 7f2ab0ff1740 1 client.20064 shutdown 2024-09-12T02:15:46.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.248+0000 7f2ab0ff1740 1 -- 172.21.15.16:0/1936888154 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55905fc9db40 msgr2=0x55905fca3fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:46.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.248+0000 7f2ab0ff1740 1 --2- 172.21.15.16:0/1936888154 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55905fc9db40 0x55905fca3fe0 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7f2a98000f80 tx=0x7f2a98087e90 comp rx=0 tx=0).stop 2024-09-12T02:15:46.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.248+0000 7f2ab0ff1740 1 -- 172.21.15.16:0/1936888154 shutdown_connections 2024-09-12T02:15:46.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.248+0000 7f2ab0ff1740 1 --2- 172.21.15.16:0/1936888154 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55905fcb6ea0 0x55905fcb92f0 crc :-1 s=READY pgs=413 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:46.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.248+0000 7f2ab0ff1740 1 --2- 172.21.15.16:0/1936888154 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55905fca7370 0x55905fca45d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:46.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.248+0000 7f2ab0ff1740 1 --2- 172.21.15.16:0/1936888154 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55905fc9db40 0x55905fca3fe0 unknown :-1 s=CLOSED pgs=1879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:46.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.248+0000 7f2ab0ff1740 1 --2- 172.21.15.16:0/1936888154 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55905fc9c5d0 0x55905fca6d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:46.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.248+0000 7f2a87fff640 0 client.20064 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:15:46.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.248+0000 7f2ab0ff1740 1 -- 172.21.15.16:0/1936888154 >> 172.21.15.16:0/1936888154 conn(0x55905fc65310 msgr2=0x55905fc9e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:15:46.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.248+0000 7f2ab0ff1740 1 -- 172.21.15.16:0/1936888154 shutdown_connections 2024-09-12T02:15:46.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.248+0000 7f2ab0ff1740 1 -- 172.21.15.16:0/1936888154 wait complete. 2024-09-12T02:15:46.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.248+0000 7f2ab0ff1740 20 client.20064 trim_cache size 0 max 16384 2024-09-12T02:15:46.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.261+0000 7f2ab0ff1740 1 Processor -- start 2024-09-12T02:15:46.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.261+0000 7f2ab0ff1740 1 -- start start 2024-09-12T02:15:46.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.261+0000 7f2ab0ff1740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55905fc9c5d0 0x55905fc98e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:46.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.261+0000 7f2ab0ff1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55905fc9db40 0x55905fcb3310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:46.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.261+0000 7f2ab0ff1740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55905fca7370 0x55905fcbebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:46.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.261+0000 7f2ab0ff1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2aa81afbd0 con 0x55905fc9db40 2024-09-12T02:15:46.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.261+0000 7f2ab0ff1740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2aa81ad4a0 con 0x55905fc9c5d0 2024-09-12T02:15:46.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.261+0000 7f2ab0ff1740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f2aa81ab310 con 0x55905fca7370 2024-09-12T02:15:46.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.261+0000 7f2aa7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55905fc9db40 0x55905fcb3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:46.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.261+0000 7f2aa7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55905fc9db40 0x55905fcb3310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53698/0 (socket says 172.21.15.16:53698) 2024-09-12T02:15:46.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.261+0000 7f2aa7fff640 1 -- 172.21.15.16:0/745723563 learned_addr learned my addr 172.21.15.16:0/745723563 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:15:46.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.262+0000 7f2aad1b5640 1 --2- 172.21.15.16:0/745723563 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55905fca7370 0x55905fcbebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:46.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.262+0000 7f2aac9b4640 1 --2- 172.21.15.16:0/745723563 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55905fc9c5d0 0x55905fc98e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:46.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.262+0000 7f2aa7fff640 1 -- 172.21.15.16:0/745723563 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55905fca7370 msgr2=0x55905fcbebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:15:46.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.262+0000 7f2aa7fff640 1 --2- 172.21.15.16:0/745723563 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55905fca7370 0x55905fcbebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:46.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.262+0000 7f2aa7fff640 1 -- 172.21.15.16:0/745723563 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55905fc9c5d0 msgr2=0x55905fc98e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:15:46.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.262+0000 7f2aa7fff640 1 --2- 172.21.15.16:0/745723563 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55905fc9c5d0 0x55905fc98e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:46.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.262+0000 7f2aa7fff640 1 -- 172.21.15.16:0/745723563 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55905fca2530 con 0x55905fc9db40 2024-09-12T02:15:46.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.262+0000 7f2aac9b4640 1 --2- 172.21.15.16:0/745723563 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55905fc9c5d0 0x55905fc98e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:15:46.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.262+0000 7f2aa7fff640 1 --2- 172.21.15.16:0/745723563 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55905fc9db40 0x55905fcb3310 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7f2a980bcb20 tx=0x7f2a980c1100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d036ca5a3965f214 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:15:46.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.262+0000 7f2a9e7fc640 1 -- 172.21.15.16:0/745723563 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a980453e0 con 0x55905fc9db40 2024-09-12T02:15:46.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.263+0000 7f2a9e7fc640 1 -- 172.21.15.16:0/745723563 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a98036040 con 0x55905fc9db40 2024-09-12T02:15:46.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.263+0000 7f2a9e7fc640 1 -- 172.21.15.16:0/745723563 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a980f0110 con 0x55905fc9db40 2024-09-12T02:15:46.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.263+0000 7f2ab0ff1740 1 -- 172.21.15.16:0/745723563 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55905fc9db40 msgr2=0x55905fcb3310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:46.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.263+0000 7f2ab0ff1740 1 --2- 172.21.15.16:0/745723563 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55905fc9db40 0x55905fcb3310 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7f2a980bcb20 tx=0x7f2a980c1100 comp rx=0 tx=0).stop 2024-09-12T02:15:46.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.263+0000 7f2ab0ff1740 1 -- 172.21.15.16:0/745723563 shutdown_connections 2024-09-12T02:15:46.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.263+0000 7f2ab0ff1740 1 --2- 172.21.15.16:0/745723563 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55905fca7370 0x55905fcbebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:46.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.263+0000 7f2ab0ff1740 1 --2- 172.21.15.16:0/745723563 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55905fc9db40 0x55905fcb3310 unknown :-1 s=CLOSED pgs=1880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:46.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.263+0000 7f2ab0ff1740 1 --2- 172.21.15.16:0/745723563 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55905fc9c5d0 0x55905fc98e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:46.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.263+0000 7f2ab0ff1740 1 -- 172.21.15.16:0/745723563 >> 172.21.15.16:0/745723563 conn(0x55905fc65310 msgr2=0x55905fc98700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:15:46.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.263+0000 7f2ab0ff1740 1 -- 172.21.15.16:0/745723563 shutdown_connections 2024-09-12T02:15:46.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.263+0000 7f2ab0ff1740 1 -- 172.21.15.16:0/745723563 wait complete. 2024-09-12T02:15:46.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.264+0000 7f2ab0ff1740 1 -- start start 2024-09-12T02:15:46.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.264+0000 7f2ab0ff1740 1 Processor -- start 2024-09-12T02:15:46.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.264+0000 7f2ab0ff1740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55905fc9c5d0 0x55905fcab490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:46.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.264+0000 7f2ab0ff1740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55905fc9db40 0x55905fcab9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:46.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.264+0000 7f2ab0ff1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55905fca7370 0x55905fcabf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:46.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.264+0000 7f2ab0ff1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2aa81ab960 con 0x55905fca7370 2024-09-12T02:15:46.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.264+0000 7f2ab0ff1740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2aa81acdd0 con 0x55905fc9c5d0 2024-09-12T02:15:46.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.264+0000 7f2ab0ff1740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f2aa81ac030 con 0x55905fc9db40 2024-09-12T02:15:46.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.264+0000 7f2ab0ff1740 10 client.0 _ll_register_callbacks cb 0x7f2a980099e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:15:46.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.264+0000 7f2aac9b4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55905fc9c5d0 0x55905fcab490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:46.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.264+0000 7f2aad1b5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55905fca7370 0x55905fcabf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:46.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.264+0000 7f2aac9b4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55905fc9c5d0 0x55905fcab490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.16:44518/0 (socket says 172.21.15.16:44518) 2024-09-12T02:15:46.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.264+0000 7f2aac9b4640 1 -- 172.21.15.16:0/3872555277 learned_addr learned my addr 172.21.15.16:0/3872555277 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:15:46.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.265+0000 7f2aa7fff640 1 --2- 172.21.15.16:0/3872555277 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55905fc9db40 0x55905fcab9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:46.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.265+0000 7f2aac9b4640 1 -- 172.21.15.16:0/3872555277 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55905fc9db40 msgr2=0x55905fcab9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:15:46.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.265+0000 7f2aac9b4640 1 --2- 172.21.15.16:0/3872555277 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55905fc9db40 0x55905fcab9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:46.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.265+0000 7f2aac9b4640 1 -- 172.21.15.16:0/3872555277 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55905fca7370 msgr2=0x55905fcabf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:15:46.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.265+0000 7f2aac9b4640 1 --2- 172.21.15.16:0/3872555277 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55905fca7370 0x55905fcabf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:46.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.265+0000 7f2aac9b4640 1 -- 172.21.15.16:0/3872555277 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55905fd20de0 con 0x55905fc9c5d0 2024-09-12T02:15:46.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.265+0000 7f2aa7fff640 1 --2- 172.21.15.16:0/3872555277 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55905fc9db40 0x55905fcab9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:15:46.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.265+0000 7f2aad1b5640 1 --2- 172.21.15.16:0/3872555277 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55905fca7370 0x55905fcabf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:15:46.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.265+0000 7f2aac9b4640 1 --2- 172.21.15.16:0/3872555277 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55905fc9c5d0 0x55905fcab490 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f2aa00099e0 tx=0x7f2aa0009520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6990765b7055c42 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:15:46.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.265+0000 7f2a87fff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-12T02:15:46.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.265+0000 7f2a87fff640 1 -- 172.21.15.16:0/3872555277 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aa0016040 con 0x55905fc9c5d0 2024-09-12T02:15:46.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.265+0000 7f2ab0ff1740 1 -- 172.21.15.16:0/3872555277 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a98036040 con 0x55905fc9c5d0 2024-09-12T02:15:46.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.265+0000 7f2a87fff640 1 -- 172.21.15.16:0/3872555277 <== mon.2 v2:172.21.15.33:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f2aa0005030 con 0x55905fc9c5d0 2024-09-12T02:15:46.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.265+0000 7f2a87fff640 1 -- 172.21.15.16:0/3872555277 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2aa0005230 con 0x55905fc9c5d0 2024-09-12T02:15:46.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.265+0000 7f2ab0ff1740 10 client.28877 fetch_fsmap learned FSMap version 25 2024-09-12T02:15:46.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.265+0000 7f2ab0ff1740 1 -- 172.21.15.16:0/3872555277 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55905fd220b0 con 0x55905fc9c5d0 2024-09-12T02:15:46.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.266+0000 7f2a87fff640 1 -- 172.21.15.16:0/3872555277 <== mon.2 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a98036040 con 0x55905fc9c5d0 2024-09-12T02:15:46.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.266+0000 7f2a87fff640 1 -- 172.21.15.16:0/3872555277 <== mon.2 v2:172.21.15.33:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f2aa0031020 con 0x55905fc9c5d0 2024-09-12T02:15:46.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.266+0000 7f2ab0ff1740 10 client.28877 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:15:46.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.266+0000 7f2ab0ff1740 10 client.28877 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:15:46.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.266+0000 7f2ab0ff1740 20 client.28877 populate_metadata read hostname 'smithi016' 2024-09-12T02:15:46.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.266+0000 7f2ab0ff1740 1 --2- 172.21.15.16:0/3872555277 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55905fd22550 0x55905fd24990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:46.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.266+0000 7f2ab0ff1740 4 client.28877 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T02:15:46.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.266+0000 7f2ab0ff1740 1 -- 172.21.15.16:0/3872555277 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55905fc09d70 con 0x55905fd22550 2024-09-12T02:15:46.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.266+0000 7f2aa7fff640 1 --2- 172.21.15.16:0/3872555277 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55905fd22550 0x55905fd24990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:46.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.266+0000 7f2a87fff640 10 client.28877 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:15:46.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.266+0000 7f2aa7fff640 1 --2- 172.21.15.16:0/3872555277 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55905fd22550 0x55905fd24990 crc :-1 s=READY pgs=414 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b5a63dca13af01e8 server_cookie=6d8ae41fbce3c836 in_seq=0 out_seq=0 2024-09-12T02:15:46.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.267+0000 7f2a87fff640 1 -- 172.21.15.16:0/3872555277 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55905fc09d70 con 0x55905fd22550 2024-09-12T02:15:46.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.267+0000 7f2a87fff640 10 client.28877 handle_command_reply: tid=0 2024-09-12T02:15:46.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.267+0000 7f2ab0ff1740 1 client.28877 shutdown 2024-09-12T02:15:46.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.267+0000 7f2ab0ff1740 1 -- 172.21.15.16:0/3872555277 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55905fc9c5d0 msgr2=0x55905fcab490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:46.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.267+0000 7f2ab0ff1740 1 --2- 172.21.15.16:0/3872555277 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55905fc9c5d0 0x55905fcab490 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f2aa00099e0 tx=0x7f2aa0009520 comp rx=0 tx=0).stop 2024-09-12T02:15:46.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.267+0000 7f2ab0ff1740 1 -- 172.21.15.16:0/3872555277 shutdown_connections 2024-09-12T02:15:46.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.267+0000 7f2ab0ff1740 1 --2- 172.21.15.16:0/3872555277 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55905fd22550 0x55905fd24990 crc :-1 s=READY pgs=414 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:46.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.267+0000 7f2ab0ff1740 1 --2- 172.21.15.16:0/3872555277 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55905fca7370 0x55905fcabf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:46.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.267+0000 7f2ab0ff1740 1 --2- 172.21.15.16:0/3872555277 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55905fc9db40 0x55905fcab9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:46.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.267+0000 7f2a87fff640 0 client.28877 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:15:46.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.267+0000 7f2ab0ff1740 1 --2- 172.21.15.16:0/3872555277 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55905fc9c5d0 0x55905fcab490 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:46.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.267+0000 7f2ab0ff1740 1 -- 172.21.15.16:0/3872555277 >> 172.21.15.16:0/3872555277 conn(0x55905fc65310 msgr2=0x55905fca6060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:15:46.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.267+0000 7f2ab0ff1740 1 -- 172.21.15.16:0/3872555277 shutdown_connections 2024-09-12T02:15:46.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.267+0000 7f2ab0ff1740 1 -- 172.21.15.16:0/3872555277 wait complete. 2024-09-12T02:15:46.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.267+0000 7f2ab0ff1740 20 client.28877 trim_cache size 0 max 16384 2024-09-12T02:15:46.275 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T02:15:46.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.268+0000 7f2aaec18640 1 -- 172.21.15.16:0/1160074624 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f2a940a2c00 msgr2=0x7f2a940a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:46.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.268+0000 7f2aaec18640 1 --2- 172.21.15.16:0/1160074624 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f2a940a2c00 0x7f2a940a50c0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f2a90005fd0 tx=0x7f2a90005850 comp rx=0 tx=0).stop 2024-09-12T02:15:46.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.268+0000 7f2aaec18640 1 -- 172.21.15.16:0/1160074624 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2aa8101dd0 msgr2=0x7f2aa8199a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:46.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.268+0000 7f2aaec18640 1 --2- 172.21.15.16:0/1160074624 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2aa8101dd0 0x7f2aa8199a60 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7f2a9802ade0 tx=0x7f2a980319f0 comp rx=0 tx=0).stop 2024-09-12T02:15:46.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.268+0000 7f2aaec18640 1 -- 172.21.15.16:0/1160074624 shutdown_connections 2024-09-12T02:15:46.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.268+0000 7f2aaec18640 1 --2- 172.21.15.16:0/1160074624 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f2a940a2c00 0x7f2a940a50c0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:46.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.268+0000 7f2aaec18640 1 --2- 172.21.15.16:0/1160074624 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2aa8106030 0x7f2aa81a45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:46.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.268+0000 7f2aaec18640 1 --2- 172.21.15.16:0/1160074624 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2aa8101dd0 0x7f2aa8199a60 unknown :-1 s=CLOSED pgs=1877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:46.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.268+0000 7f2aaec18640 1 --2- 172.21.15.16:0/1160074624 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2aa8101420 0x7f2aa8199520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:46.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.268+0000 7f2aaec18640 1 -- 172.21.15.16:0/1160074624 >> 172.21.15.16:0/1160074624 conn(0x7f2aa80fbee0 msgr2=0x7f2aa80fd230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:15:46.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.268+0000 7f2aaec18640 1 -- 172.21.15.16:0/1160074624 shutdown_connections 2024-09-12T02:15:46.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:46.268+0000 7f2aaec18640 1 -- 172.21.15.16:0/1160074624 wait complete. 2024-09-12T02:15:46.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:15:46 smithi016 ceph-mon[30947]: pgmap v750: 97 pgs: 97 active+clean; 3.0 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 4.8 KiB/s wr, 9 op/s 2024-09-12T02:15:46.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:15:46 smithi184 ceph-mon[35593]: pgmap v750: 97 pgs: 97 active+clean; 3.0 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 4.8 KiB/s wr, 9 op/s 2024-09-12T02:15:46.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:15:46 smithi033 ceph-mon[35332]: pgmap v750: 97 pgs: 97 active+clean; 3.0 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 4.8 KiB/s wr, 9 op/s 2024-09-12T02:15:47.921 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-12T02:15:47.921 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-12T02:15:48.192 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:15:48.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:15:48 smithi184 ceph-mon[35593]: pgmap v751: 97 pgs: 97 active+clean; 3.1 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 6.2 KiB/s wr, 10 op/s 2024-09-12T02:15:48.628 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:15:48 smithi016 ceph-mon[30947]: pgmap v751: 97 pgs: 97 active+clean; 3.1 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 6.2 KiB/s wr, 10 op/s 2024-09-12T02:15:48.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.629+0000 7eff69212640 1 -- 172.21.15.16:0/833097962 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff6410e4e0 msgr2=0x7eff641108d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:48.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.629+0000 7eff69212640 1 --2- 172.21.15.16:0/833097962 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff6410e4e0 0x7eff641108d0 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7eff5c009ca0 tx=0x7eff5c02a8c0 comp rx=0 tx=0).stop 2024-09-12T02:15:48.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.629+0000 7eff69212640 1 -- 172.21.15.16:0/833097962 shutdown_connections 2024-09-12T02:15:48.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.629+0000 7eff69212640 1 --2- 172.21.15.16:0/833097962 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff6410e4e0 0x7eff641108d0 unknown :-1 s=CLOSED pgs=1881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:48.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.629+0000 7eff69212640 1 --2- 172.21.15.16:0/833097962 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7eff641012e0 0x7eff6410dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:48.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.629+0000 7eff69212640 1 --2- 172.21.15.16:0/833097962 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7eff64100930 0x7eff64100d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:48.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.629+0000 7eff69212640 1 -- 172.21.15.16:0/833097962 >> 172.21.15.16:0/833097962 conn(0x7eff640fc7a0 msgr2=0x7eff640febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:15:48.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.629+0000 7eff69212640 1 -- 172.21.15.16:0/833097962 shutdown_connections 2024-09-12T02:15:48.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.629+0000 7eff69212640 1 -- 172.21.15.16:0/833097962 wait complete. 2024-09-12T02:15:48.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.630+0000 7eff69212640 1 Processor -- start 2024-09-12T02:15:48.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.630+0000 7eff69212640 1 -- start start 2024-09-12T02:15:48.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.630+0000 7eff69212640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7eff64100930 0x7eff6419a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:48.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.630+0000 7eff69212640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff641012e0 0x7eff6419abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:48.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.630+0000 7eff69212640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7eff6410e4e0 0x7eff6419ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:48.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.630+0000 7eff69212640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7eff641141e0 con 0x7eff641012e0 2024-09-12T02:15:48.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.630+0000 7eff69212640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7eff64114060 con 0x7eff64100930 2024-09-12T02:15:48.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.630+0000 7eff69212640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7eff64114360 con 0x7eff6410e4e0 2024-09-12T02:15:48.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.630+0000 7eff6259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff641012e0 0x7eff6419abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:48.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.630+0000 7eff6259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff641012e0 0x7eff6419abc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53730/0 (socket says 172.21.15.16:53730) 2024-09-12T02:15:48.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.630+0000 7eff6259c640 1 -- 172.21.15.16:0/3968568522 learned_addr learned my addr 172.21.15.16:0/3968568522 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:15:48.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.631+0000 7eff6359e640 1 --2- 172.21.15.16:0/3968568522 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7eff6410e4e0 0x7eff6419ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:48.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.631+0000 7eff62d9d640 1 --2- 172.21.15.16:0/3968568522 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7eff64100930 0x7eff6419a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:48.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.631+0000 7eff6259c640 1 -- 172.21.15.16:0/3968568522 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7eff6410e4e0 msgr2=0x7eff6419ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:48.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.631+0000 7eff6259c640 1 --2- 172.21.15.16:0/3968568522 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7eff6410e4e0 0x7eff6419ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:48.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.631+0000 7eff6259c640 1 -- 172.21.15.16:0/3968568522 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7eff64100930 msgr2=0x7eff6419a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:48.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.631+0000 7eff6259c640 1 --2- 172.21.15.16:0/3968568522 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7eff64100930 0x7eff6419a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:48.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.631+0000 7eff6259c640 1 -- 172.21.15.16:0/3968568522 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff6419f630 con 0x7eff641012e0 2024-09-12T02:15:48.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.631+0000 7eff62d9d640 1 --2- 172.21.15.16:0/3968568522 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7eff64100930 0x7eff6419a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:15:48.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.631+0000 7eff6259c640 1 --2- 172.21.15.16:0/3968568522 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff641012e0 0x7eff6419abc0 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7eff5800d770 tx=0x7eff5800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:15:48.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.631+0000 7eff4bfff640 1 -- 172.21.15.16:0/3968568522 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff58017070 con 0x7eff641012e0 2024-09-12T02:15:48.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.631+0000 7eff69212640 1 -- 172.21.15.16:0/3968568522 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff6419f920 con 0x7eff641012e0 2024-09-12T02:15:48.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.631+0000 7eff4bfff640 1 -- 172.21.15.16:0/3968568522 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff58005500 con 0x7eff641012e0 2024-09-12T02:15:48.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.631+0000 7eff4bfff640 1 -- 172.21.15.16:0/3968568522 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff58004e00 con 0x7eff641012e0 2024-09-12T02:15:48.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.631+0000 7eff69212640 1 -- 172.21.15.16:0/3968568522 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff641a7200 con 0x7eff641012e0 2024-09-12T02:15:48.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.634+0000 7eff4bfff640 1 -- 172.21.15.16:0/3968568522 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7eff58004fa0 con 0x7eff641012e0 2024-09-12T02:15:48.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.634+0000 7eff6b5eb740 1 Processor -- start 2024-09-12T02:15:48.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.634+0000 7eff6b5eb740 1 -- start start 2024-09-12T02:15:48.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.634+0000 7eff4bfff640 1 --2- 172.21.15.16:0/3968568522 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7eff4c0a2b80 0x7eff4c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:48.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.634+0000 7eff6b5eb740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620891d4b60 0x5620891d4f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:48.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.634+0000 7eff6b5eb740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5620891d5480 0x5620891df800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:48.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.634+0000 7eff6b5eb740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620891dfd40 0x5620891e2130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:48.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.634+0000 7eff6b5eb740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56208913a280 con 0x5620891d4b60 2024-09-12T02:15:48.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.634+0000 7eff6b5eb740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5620890e1b10 con 0x5620891dfd40 2024-09-12T02:15:48.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.634+0000 7eff6b5eb740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x562089136240 con 0x5620891d5480 2024-09-12T02:15:48.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.635+0000 7eff6359e640 1 --2- 172.21.15.16:0/3968568522 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7eff4c0a2b80 0x7eff4c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:48.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.635+0000 7eff4bfff640 1 -- 172.21.15.16:0/3968568522 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7eff580cea30 con 0x7eff641012e0 2024-09-12T02:15:48.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.635+0000 7eff62d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620891d4b60 0x5620891d4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:48.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.635+0000 7eff62d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620891d4b60 0x5620891d4f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53732/0 (socket says 172.21.15.16:53732) 2024-09-12T02:15:48.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.635+0000 7eff62d9d640 1 -- 172.21.15.16:0/2319540471 learned_addr learned my addr 172.21.15.16:0/2319540471 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:15:48.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.635+0000 7eff6359e640 1 --2- 172.21.15.16:0/2319540471 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620891dfd40 0x5620891e2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:48.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.635+0000 7eff6259c640 1 --2- 172.21.15.16:0/2319540471 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5620891d5480 0x5620891df800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:48.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.635+0000 7eff62d9d640 1 -- 172.21.15.16:0/2319540471 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5620891d5480 msgr2=0x5620891df800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:15:48.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.635+0000 7eff62d9d640 1 --2- 172.21.15.16:0/2319540471 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5620891d5480 0x5620891df800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:48.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.635+0000 7eff62d9d640 1 -- 172.21.15.16:0/2319540471 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620891dfd40 msgr2=0x5620891e2130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:15:48.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.635+0000 7eff62d9d640 1 --2- 172.21.15.16:0/2319540471 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620891dfd40 0x5620891e2130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:48.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.635+0000 7eff62d9d640 1 -- 172.21.15.16:0/2319540471 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5620890ee1f0 con 0x5620891d4b60 2024-09-12T02:15:48.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.635+0000 7eff6359e640 1 --2- 172.21.15.16:0/2319540471 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620891dfd40 0x5620891e2130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:15:48.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.635+0000 7eff6259c640 1 --2- 172.21.15.16:0/2319540471 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5620891d5480 0x5620891df800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:15:48.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.635+0000 7eff6359e640 1 --2- 172.21.15.16:0/3968568522 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7eff4c0a2b80 0x7eff4c0a5040 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7eff5c005e30 tx=0x7eff5c009930 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:15:48.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.635+0000 7eff62d9d640 1 --2- 172.21.15.16:0/2319540471 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620891d4b60 0x5620891d4f40 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 crypto rx=0x7eff54007850 tx=0x7eff54007d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36fb1c86e4300fb8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:15:48.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.635+0000 7eff497fa640 1 -- 172.21.15.16:0/2319540471 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff5400d950 con 0x5620891d4b60 2024-09-12T02:15:48.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.635+0000 7eff497fa640 1 -- 172.21.15.16:0/2319540471 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff54018070 con 0x5620891d4b60 2024-09-12T02:15:48.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.635+0000 7eff497fa640 1 -- 172.21.15.16:0/2319540471 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff540123e0 con 0x5620891d4b60 2024-09-12T02:15:48.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.636+0000 7eff6b5eb740 1 -- 172.21.15.16:0/2319540471 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620891d4b60 msgr2=0x5620891d4f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:48.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.636+0000 7eff6b5eb740 1 --2- 172.21.15.16:0/2319540471 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620891d4b60 0x5620891d4f40 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 crypto rx=0x7eff54007850 tx=0x7eff54007d10 comp rx=0 tx=0).stop 2024-09-12T02:15:48.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.636+0000 7eff6b5eb740 1 -- 172.21.15.16:0/2319540471 shutdown_connections 2024-09-12T02:15:48.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.636+0000 7eff6b5eb740 1 --2- 172.21.15.16:0/2319540471 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620891dfd40 0x5620891e2130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:48.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.636+0000 7eff6b5eb740 1 --2- 172.21.15.16:0/2319540471 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5620891d5480 0x5620891df800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:48.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.636+0000 7eff6b5eb740 1 --2- 172.21.15.16:0/2319540471 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620891d4b60 0x5620891d4f40 unknown :-1 s=CLOSED pgs=1883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:48.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.636+0000 7eff6b5eb740 1 -- 172.21.15.16:0/2319540471 >> 172.21.15.16:0/2319540471 conn(0x5620891d10f0 msgr2=0x5620891d14d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:15:48.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.636+0000 7eff6b5eb740 1 -- 172.21.15.16:0/2319540471 shutdown_connections 2024-09-12T02:15:48.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.636+0000 7eff6b5eb740 1 -- 172.21.15.16:0/2319540471 wait complete. 2024-09-12T02:15:48.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.637+0000 7eff6b5eb740 1 -- start start 2024-09-12T02:15:48.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.637+0000 7eff6b5eb740 1 Processor -- start 2024-09-12T02:15:48.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.637+0000 7eff6b5eb740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5620891d4b60 0x5620891dcba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:48.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.637+0000 7eff6b5eb740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620891d5480 0x5620891dd0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:48.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.637+0000 7eff6b5eb740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620891dfd40 0x5620891dd620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:48.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.637+0000 7eff6b5eb740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5620891e2910 con 0x5620891dfd40 2024-09-12T02:15:48.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.637+0000 7eff6b5eb740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5620890f4f10 con 0x5620891d5480 2024-09-12T02:15:48.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.637+0000 7eff6b5eb740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5620891e2a90 con 0x5620891d4b60 2024-09-12T02:15:48.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.637+0000 7eff6b5eb740 10 client.0 _ll_register_callbacks cb 0x5620890bda40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:15:48.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.637+0000 7eff6359e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620891dfd40 0x5620891dd620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:48.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.637+0000 7eff6359e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620891dfd40 0x5620891dd620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53742/0 (socket says 172.21.15.16:53742) 2024-09-12T02:15:48.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.637+0000 7eff6359e640 1 -- 172.21.15.16:0/1337271531 learned_addr learned my addr 172.21.15.16:0/1337271531 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:15:48.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.637+0000 7eff62d9d640 1 --2- 172.21.15.16:0/1337271531 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5620891d4b60 0x5620891dcba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:48.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.637+0000 7eff6259c640 1 --2- 172.21.15.16:0/1337271531 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620891d5480 0x5620891dd0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:48.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.638+0000 7eff6359e640 1 -- 172.21.15.16:0/1337271531 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5620891d4b60 msgr2=0x5620891dcba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:15:48.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.638+0000 7eff6359e640 1 --2- 172.21.15.16:0/1337271531 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5620891d4b60 0x5620891dcba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:48.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.638+0000 7eff6359e640 1 -- 172.21.15.16:0/1337271531 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620891d5480 msgr2=0x5620891dd0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:15:48.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.638+0000 7eff6359e640 1 --2- 172.21.15.16:0/1337271531 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620891d5480 0x5620891dd0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:48.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.638+0000 7eff6359e640 1 -- 172.21.15.16:0/1337271531 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562088eab6a0 con 0x5620891dfd40 2024-09-12T02:15:48.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.638+0000 7eff6259c640 1 --2- 172.21.15.16:0/1337271531 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620891d5480 0x5620891dd0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:15:48.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.638+0000 7eff6359e640 1 --2- 172.21.15.16:0/1337271531 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620891dfd40 0x5620891dd620 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7eff5c009a60 tx=0x7eff5c02a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26d869e4f8bf7657 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:15:48.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.638+0000 7eff46ff5640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:15:48.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.638+0000 7eff46ff5640 1 -- 172.21.15.16:0/1337271531 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff5c04f070 con 0x5620891dfd40 2024-09-12T02:15:48.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.638+0000 7eff46ff5640 1 -- 172.21.15.16:0/1337271531 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7eff5c004590 con 0x5620891dfd40 2024-09-12T02:15:48.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.638+0000 7eff6b5eb740 1 -- 172.21.15.16:0/1337271531 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff54018070 con 0x5620891dfd40 2024-09-12T02:15:48.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.638+0000 7eff46ff5640 1 -- 172.21.15.16:0/1337271531 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7eff5c05a050 con 0x5620891dfd40 2024-09-12T02:15:48.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.638+0000 7eff6b5eb740 10 client.20082 fetch_fsmap learned FSMap version 25 2024-09-12T02:15:48.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.638+0000 7eff6b5eb740 1 -- 172.21.15.16:0/1337271531 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5620891ee460 con 0x5620891dfd40 2024-09-12T02:15:48.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.638+0000 7eff46ff5640 1 -- 172.21.15.16:0/1337271531 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff54018070 con 0x5620891dfd40 2024-09-12T02:15:48.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.639+0000 7eff46ff5640 1 -- 172.21.15.16:0/1337271531 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7eff5c030650 con 0x5620891dfd40 2024-09-12T02:15:48.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.639+0000 7eff6b5eb740 10 client.20082 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:15:48.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.639+0000 7eff6b5eb740 10 client.20082 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:15:48.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.639+0000 7eff6b5eb740 20 client.20082 populate_metadata read hostname 'smithi016' 2024-09-12T02:15:48.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.639+0000 7eff6b5eb740 1 --2- 172.21.15.16:0/1337271531 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5620891ef870 0x5620891f1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:48.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.639+0000 7eff6b5eb740 4 client.20082 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:15:48.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.639+0000 7eff6b5eb740 1 -- 172.21.15.16:0/1337271531 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5620891477f0 con 0x5620891ef870 2024-09-12T02:15:48.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.639+0000 7eff62d9d640 1 --2- 172.21.15.16:0/1337271531 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5620891ef870 0x5620891f1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:48.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.639+0000 7eff62d9d640 1 --2- 172.21.15.16:0/1337271531 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5620891ef870 0x5620891f1ce0 crc :-1 s=READY pgs=415 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1b3fa37d7136d99 server_cookie=d67a7ca6ad34e9c4 in_seq=0 out_seq=0 2024-09-12T02:15:48.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.639+0000 7eff46ff5640 10 client.20082 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:15:48.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.640+0000 7eff46ff5640 1 -- 172.21.15.16:0/1337271531 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5620891477f0 con 0x5620891ef870 2024-09-12T02:15:48.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.640+0000 7eff46ff5640 10 client.20082 handle_command_reply: tid=0 2024-09-12T02:15:48.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.640+0000 7eff6b5eb740 1 client.20082 shutdown 2024-09-12T02:15:48.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.640+0000 7eff6b5eb740 1 -- 172.21.15.16:0/1337271531 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620891dfd40 msgr2=0x5620891dd620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:48.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.640+0000 7eff6b5eb740 1 --2- 172.21.15.16:0/1337271531 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620891dfd40 0x5620891dd620 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7eff5c009a60 tx=0x7eff5c02a8c0 comp rx=0 tx=0).stop 2024-09-12T02:15:48.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.640+0000 7eff6b5eb740 1 -- 172.21.15.16:0/1337271531 shutdown_connections 2024-09-12T02:15:48.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.640+0000 7eff6b5eb740 1 --2- 172.21.15.16:0/1337271531 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5620891ef870 0x5620891f1ce0 crc :-1 s=READY pgs=415 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:48.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.640+0000 7eff6b5eb740 1 --2- 172.21.15.16:0/1337271531 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620891dfd40 0x5620891dd620 unknown :-1 s=CLOSED pgs=1884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:48.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.640+0000 7eff6b5eb740 1 --2- 172.21.15.16:0/1337271531 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620891d5480 0x5620891dd0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:48.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.640+0000 7eff46ff5640 0 client.20082 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:15:48.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.640+0000 7eff6b5eb740 1 --2- 172.21.15.16:0/1337271531 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5620891d4b60 0x5620891dcba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:48.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.640+0000 7eff6b5eb740 1 -- 172.21.15.16:0/1337271531 >> 172.21.15.16:0/1337271531 conn(0x5620891d10f0 msgr2=0x5620891d0270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:15:48.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.640+0000 7eff6b5eb740 1 -- 172.21.15.16:0/1337271531 shutdown_connections 2024-09-12T02:15:48.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.640+0000 7eff6b5eb740 1 -- 172.21.15.16:0/1337271531 wait complete. 2024-09-12T02:15:48.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.640+0000 7eff6b5eb740 20 client.20082 trim_cache size 0 max 16384 2024-09-12T02:15:48.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.658+0000 7eff6b5eb740 1 Processor -- start 2024-09-12T02:15:48.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.658+0000 7eff6b5eb740 1 -- start start 2024-09-12T02:15:48.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.658+0000 7eff6b5eb740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620891d4b60 0x5620891d6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:48.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.658+0000 7eff6b5eb740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620891d5480 0x5620891f7600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:48.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.658+0000 7eff6b5eb740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5620891dfd40 0x5620891f7b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:48.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.658+0000 7eff6b5eb740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7eff641b1e80 con 0x5620891d5480 2024-09-12T02:15:48.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.658+0000 7eff6b5eb740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7eff641af750 con 0x5620891d4b60 2024-09-12T02:15:48.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.658+0000 7eff6b5eb740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7eff641ad5e0 con 0x5620891dfd40 2024-09-12T02:15:48.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.658+0000 7eff6259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620891d5480 0x5620891f7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:48.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.658+0000 7eff6259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620891d5480 0x5620891f7600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53752/0 (socket says 172.21.15.16:53752) 2024-09-12T02:15:48.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.658+0000 7eff6259c640 1 -- 172.21.15.16:0/1667917053 learned_addr learned my addr 172.21.15.16:0/1667917053 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:15:48.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.658+0000 7eff6359e640 1 --2- 172.21.15.16:0/1667917053 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5620891dfd40 0x5620891f7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:48.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.658+0000 7eff6259c640 1 -- 172.21.15.16:0/1667917053 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5620891dfd40 msgr2=0x5620891f7b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:15:48.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.658+0000 7eff62d9d640 1 --2- 172.21.15.16:0/1667917053 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620891d4b60 0x5620891d6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:48.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.658+0000 7eff6259c640 1 --2- 172.21.15.16:0/1667917053 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5620891dfd40 0x5620891f7b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:48.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.658+0000 7eff6259c640 1 -- 172.21.15.16:0/1667917053 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620891d4b60 msgr2=0x5620891d6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:15:48.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.658+0000 7eff6259c640 1 --2- 172.21.15.16:0/1667917053 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620891d4b60 0x5620891d6cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:48.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.658+0000 7eff6259c640 1 -- 172.21.15.16:0/1667917053 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5620891d52b0 con 0x5620891d5480 2024-09-12T02:15:48.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.658+0000 7eff62d9d640 1 --2- 172.21.15.16:0/1667917053 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620891d4b60 0x5620891d6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:15:48.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.659+0000 7eff6259c640 1 --2- 172.21.15.16:0/1667917053 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620891d5480 0x5620891f7600 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7eff58009610 tx=0x7eff58009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb48464ebee1df32 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:15:48.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.659+0000 7eff48ff9640 1 -- 172.21.15.16:0/1667917053 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff58020040 con 0x5620891d5480 2024-09-12T02:15:48.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.659+0000 7eff48ff9640 1 -- 172.21.15.16:0/1667917053 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff5809b330 con 0x5620891d5480 2024-09-12T02:15:48.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.659+0000 7eff48ff9640 1 -- 172.21.15.16:0/1667917053 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff580979a0 con 0x5620891d5480 2024-09-12T02:15:48.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.659+0000 7eff6b5eb740 1 -- 172.21.15.16:0/1667917053 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620891d5480 msgr2=0x5620891f7600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:48.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.659+0000 7eff6b5eb740 1 --2- 172.21.15.16:0/1667917053 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620891d5480 0x5620891f7600 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7eff58009610 tx=0x7eff58009c30 comp rx=0 tx=0).stop 2024-09-12T02:15:48.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.659+0000 7eff6b5eb740 1 -- 172.21.15.16:0/1667917053 shutdown_connections 2024-09-12T02:15:48.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.659+0000 7eff6b5eb740 1 --2- 172.21.15.16:0/1667917053 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5620891dfd40 0x5620891f7b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:48.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.659+0000 7eff6b5eb740 1 --2- 172.21.15.16:0/1667917053 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620891d5480 0x5620891f7600 unknown :-1 s=CLOSED pgs=1885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:48.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.659+0000 7eff6b5eb740 1 --2- 172.21.15.16:0/1667917053 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620891d4b60 0x5620891d6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:48.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.659+0000 7eff6b5eb740 1 -- 172.21.15.16:0/1667917053 >> 172.21.15.16:0/1667917053 conn(0x5620891d10f0 msgr2=0x5620891d65b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:15:48.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.659+0000 7eff6b5eb740 1 -- 172.21.15.16:0/1667917053 shutdown_connections 2024-09-12T02:15:48.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.659+0000 7eff6b5eb740 1 -- 172.21.15.16:0/1667917053 wait complete. 2024-09-12T02:15:48.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.660+0000 7eff6b5eb740 1 -- start start 2024-09-12T02:15:48.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.660+0000 7eff6b5eb740 1 Processor -- start 2024-09-12T02:15:48.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.660+0000 7eff6b5eb740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620891d4b60 0x5620891d9b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:48.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.660+0000 7eff6b5eb740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620891d5480 0x5620891da0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:48.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.660+0000 7eff6b5eb740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5620891dfd40 0x5620891da600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:48.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.660+0000 7eff6b5eb740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7eff641adc30 con 0x5620891d4b60 2024-09-12T02:15:48.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.660+0000 7eff6b5eb740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7eff641af080 con 0x5620891d5480 2024-09-12T02:15:48.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.660+0000 7eff6b5eb740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7eff641ae2e0 con 0x5620891dfd40 2024-09-12T02:15:48.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.660+0000 7eff6b5eb740 10 client.0 _ll_register_callbacks cb 0x7eff5c009360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:15:48.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.660+0000 7eff62d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620891d4b60 0x5620891d9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:48.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.660+0000 7eff62d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620891d4b60 0x5620891d9b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53768/0 (socket says 172.21.15.16:53768) 2024-09-12T02:15:48.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.660+0000 7eff62d9d640 1 -- 172.21.15.16:0/3587590683 learned_addr learned my addr 172.21.15.16:0/3587590683 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:15:48.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.661+0000 7eff6359e640 1 --2- 172.21.15.16:0/3587590683 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5620891dfd40 0x5620891da600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:48.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.661+0000 7eff6259c640 1 --2- 172.21.15.16:0/3587590683 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620891d5480 0x5620891da0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:48.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.661+0000 7eff62d9d640 1 -- 172.21.15.16:0/3587590683 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5620891dfd40 msgr2=0x5620891da600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:15:48.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.661+0000 7eff62d9d640 1 --2- 172.21.15.16:0/3587590683 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5620891dfd40 0x5620891da600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:48.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.661+0000 7eff62d9d640 1 -- 172.21.15.16:0/3587590683 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620891d5480 msgr2=0x5620891da0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:15:48.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.661+0000 7eff62d9d640 1 --2- 172.21.15.16:0/3587590683 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620891d5480 0x5620891da0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:48.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.661+0000 7eff62d9d640 1 -- 172.21.15.16:0/3587590683 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5620892595d0 con 0x5620891d4b60 2024-09-12T02:15:48.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.661+0000 7eff62d9d640 1 --2- 172.21.15.16:0/3587590683 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620891d4b60 0x5620891d9b80 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 crypto rx=0x7eff54006f40 tx=0x7eff5400d670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=465e234483438e14 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:15:48.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.661+0000 7eff46ff5640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:15:48.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.661+0000 7eff46ff5640 1 -- 172.21.15.16:0/3587590683 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff5402a030 con 0x5620891d4b60 2024-09-12T02:15:48.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.661+0000 7eff46ff5640 1 -- 172.21.15.16:0/3587590683 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7eff54013480 con 0x5620891d4b60 2024-09-12T02:15:48.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.661+0000 7eff6b5eb740 1 -- 172.21.15.16:0/3587590683 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff5809b330 con 0x5620891d4b60 2024-09-12T02:15:48.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.661+0000 7eff46ff5640 1 -- 172.21.15.16:0/3587590683 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7eff540136a0 con 0x5620891d4b60 2024-09-12T02:15:48.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.661+0000 7eff6b5eb740 10 client.20088 fetch_fsmap learned FSMap version 25 2024-09-12T02:15:48.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.661+0000 7eff6b5eb740 1 -- 172.21.15.16:0/3587590683 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56208925a820 con 0x5620891d4b60 2024-09-12T02:15:48.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.662+0000 7eff46ff5640 1 -- 172.21.15.16:0/3587590683 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff5809b330 con 0x5620891d4b60 2024-09-12T02:15:48.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.662+0000 7eff46ff5640 1 -- 172.21.15.16:0/3587590683 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7eff54026b30 con 0x5620891d4b60 2024-09-12T02:15:48.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.662+0000 7eff6b5eb740 10 client.20088 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:15:48.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.662+0000 7eff6b5eb740 10 client.20088 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:15:48.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.662+0000 7eff6b5eb740 20 client.20088 populate_metadata read hostname 'smithi016' 2024-09-12T02:15:48.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.662+0000 7eff6b5eb740 1 --2- 172.21.15.16:0/3587590683 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x56208925ae10 0x56208925d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:48.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.662+0000 7eff6b5eb740 4 client.20088 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-12T02:15:48.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.662+0000 7eff6b5eb740 1 -- 172.21.15.16:0/3587590683 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x562089141d00 con 0x56208925ae10 2024-09-12T02:15:48.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.662+0000 7eff6259c640 1 --2- 172.21.15.16:0/3587590683 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x56208925ae10 0x56208925d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:48.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.662+0000 7eff6259c640 1 --2- 172.21.15.16:0/3587590683 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x56208925ae10 0x56208925d230 crc :-1 s=READY pgs=416 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ece1f5941a1f8c0 server_cookie=f882a3ed2fa9bc33 in_seq=0 out_seq=0 2024-09-12T02:15:48.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.662+0000 7eff46ff5640 10 client.20088 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:15:48.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.663+0000 7eff46ff5640 1 -- 172.21.15.16:0/3587590683 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x562089141d00 con 0x56208925ae10 2024-09-12T02:15:48.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.663+0000 7eff46ff5640 10 client.20088 handle_command_reply: tid=0 2024-09-12T02:15:48.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.663+0000 7eff6b5eb740 1 client.20088 shutdown 2024-09-12T02:15:48.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.664+0000 7eff6b5eb740 1 -- 172.21.15.16:0/3587590683 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620891d4b60 msgr2=0x5620891d9b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:48.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.664+0000 7eff6b5eb740 1 --2- 172.21.15.16:0/3587590683 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620891d4b60 0x5620891d9b80 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 crypto rx=0x7eff54006f40 tx=0x7eff5400d670 comp rx=0 tx=0).stop 2024-09-12T02:15:48.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.664+0000 7eff6b5eb740 1 -- 172.21.15.16:0/3587590683 shutdown_connections 2024-09-12T02:15:48.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.664+0000 7eff6b5eb740 1 --2- 172.21.15.16:0/3587590683 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x56208925ae10 0x56208925d230 crc :-1 s=READY pgs=416 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:48.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.664+0000 7eff6b5eb740 1 --2- 172.21.15.16:0/3587590683 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5620891dfd40 0x5620891da600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:48.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.664+0000 7eff6b5eb740 1 --2- 172.21.15.16:0/3587590683 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620891d5480 0x5620891da0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:48.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.664+0000 7eff6b5eb740 1 --2- 172.21.15.16:0/3587590683 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620891d4b60 0x5620891d9b80 unknown :-1 s=CLOSED pgs=1886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:48.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.664+0000 7eff6b5eb740 1 -- 172.21.15.16:0/3587590683 >> 172.21.15.16:0/3587590683 conn(0x5620891d10f0 msgr2=0x5620891dd380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:15:48.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.664+0000 7eff46ff5640 0 client.20088 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:15:48.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.664+0000 7eff6b5eb740 1 -- 172.21.15.16:0/3587590683 shutdown_connections 2024-09-12T02:15:48.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.664+0000 7eff6b5eb740 1 -- 172.21.15.16:0/3587590683 wait complete. 2024-09-12T02:15:48.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.664+0000 7eff6b5eb740 20 client.20088 trim_cache size 0 max 16384 2024-09-12T02:15:48.671 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T02:15:48.671 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-09-12T02:15:48.671 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "44ea79e1-6891-4db1-900e-a672d238e67c", 2024-09-12T02:15:48.671 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-09-12T02:15:48.671 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T02:15:48.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.664+0000 7eff69212640 1 -- 172.21.15.16:0/3968568522 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7eff4c0a2b80 msgr2=0x7eff4c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:48.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.664+0000 7eff69212640 1 --2- 172.21.15.16:0/3968568522 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7eff4c0a2b80 0x7eff4c0a5040 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7eff5c005e30 tx=0x7eff5c009930 comp rx=0 tx=0).stop 2024-09-12T02:15:48.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.664+0000 7eff69212640 1 -- 172.21.15.16:0/3968568522 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff641012e0 msgr2=0x7eff6419abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:48.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.664+0000 7eff69212640 1 --2- 172.21.15.16:0/3968568522 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff641012e0 0x7eff6419abc0 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7eff5800d770 tx=0x7eff5800dc30 comp rx=0 tx=0).stop 2024-09-12T02:15:48.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.665+0000 7eff69212640 1 -- 172.21.15.16:0/3968568522 shutdown_connections 2024-09-12T02:15:48.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.665+0000 7eff69212640 1 --2- 172.21.15.16:0/3968568522 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7eff4c0a2b80 0x7eff4c0a5040 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:48.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.665+0000 7eff69212640 1 --2- 172.21.15.16:0/3968568522 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7eff6410e4e0 0x7eff6419ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:48.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.665+0000 7eff69212640 1 --2- 172.21.15.16:0/3968568522 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff641012e0 0x7eff6419abc0 unknown :-1 s=CLOSED pgs=1882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:48.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.665+0000 7eff69212640 1 --2- 172.21.15.16:0/3968568522 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7eff64100930 0x7eff6419a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:48.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.665+0000 7eff69212640 1 -- 172.21.15.16:0/3968568522 >> 172.21.15.16:0/3968568522 conn(0x7eff640fc7a0 msgr2=0x7eff640feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:15:48.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.665+0000 7eff69212640 1 -- 172.21.15.16:0/3968568522 shutdown_connections 2024-09-12T02:15:48.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:48.665+0000 7eff69212640 1 -- 172.21.15.16:0/3968568522 wait complete. 2024-09-12T02:15:48.673 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:15:48 smithi033 ceph-mon[35332]: pgmap v751: 97 pgs: 97 active+clean; 3.1 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 6.2 KiB/s wr, 10 op/s 2024-09-12T02:15:49.339 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T02:15:49.595 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:15:50.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.052+0000 7fa481041640 1 -- 172.21.15.16:0/3331144293 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa47c101de0 msgr2=0x7fa47c1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:50.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.052+0000 7fa481041640 1 --2- 172.21.15.16:0/3331144293 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa47c101de0 0x7fa47c1021c0 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7fa46c009900 tx=0x7fa46c02a410 comp rx=0 tx=0).stop 2024-09-12T02:15:50.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.052+0000 7fa481041640 1 -- 172.21.15.16:0/3331144293 shutdown_connections 2024-09-12T02:15:50.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.052+0000 7fa481041640 1 --2- 172.21.15.16:0/3331144293 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa47c106f10 0x7fa47c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:50.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.052+0000 7fa481041640 1 --2- 172.21.15.16:0/3331144293 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa47c102790 0x7fa47c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:50.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.052+0000 7fa481041640 1 --2- 172.21.15.16:0/3331144293 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa47c101de0 0x7fa47c1021c0 unknown :-1 s=CLOSED pgs=1887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:50.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.052+0000 7fa481041640 1 -- 172.21.15.16:0/3331144293 >> 172.21.15.16:0/3331144293 conn(0x7fa47c0fd8c0 msgr2=0x7fa47c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:15:50.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.052+0000 7fa481041640 1 -- 172.21.15.16:0/3331144293 shutdown_connections 2024-09-12T02:15:50.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.052+0000 7fa481041640 1 -- 172.21.15.16:0/3331144293 wait complete. 2024-09-12T02:15:50.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.053+0000 7fa481041640 1 Processor -- start 2024-09-12T02:15:50.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.053+0000 7fa481041640 1 -- start start 2024-09-12T02:15:50.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.054+0000 7fa481041640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa47c101de0 0x7fa47c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:50.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.054+0000 7fa481041640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa47c102790 0x7fa47c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:50.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.054+0000 7fa481041640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa47c106f10 0x7fa47c074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:50.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.054+0000 7fa481041640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa47c111f90 con 0x7fa47c106f10 2024-09-12T02:15:50.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.054+0000 7fa481041640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa47c111e10 con 0x7fa47c102790 2024-09-12T02:15:50.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.054+0000 7fa481041640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa47c112110 con 0x7fa47c101de0 2024-09-12T02:15:50.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.054+0000 7fa47b59e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa47c106f10 0x7fa47c074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:50.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.054+0000 7fa47b59e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa47c106f10 0x7fa47c074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53774/0 (socket says 172.21.15.16:53774) 2024-09-12T02:15:50.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.054+0000 7fa47b59e640 1 -- 172.21.15.16:0/3223746431 learned_addr learned my addr 172.21.15.16:0/3223746431 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:15:50.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.054+0000 7fa47ad9d640 1 --2- 172.21.15.16:0/3223746431 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa47c101de0 0x7fa47c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:50.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.054+0000 7fa47a59c640 1 --2- 172.21.15.16:0/3223746431 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa47c102790 0x7fa47c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:50.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.054+0000 7fa47b59e640 1 -- 172.21.15.16:0/3223746431 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa47c101de0 msgr2=0x7fa47c0744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:50.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.054+0000 7fa47b59e640 1 --2- 172.21.15.16:0/3223746431 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa47c101de0 0x7fa47c0744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:50.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.054+0000 7fa47b59e640 1 -- 172.21.15.16:0/3223746431 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa47c102790 msgr2=0x7fa47c074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:50.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.054+0000 7fa47b59e640 1 --2- 172.21.15.16:0/3223746431 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa47c102790 0x7fa47c074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:50.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.054+0000 7fa47b59e640 1 -- 172.21.15.16:0/3223746431 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa47c1a6890 con 0x7fa47c106f10 2024-09-12T02:15:50.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.054+0000 7fa47b59e640 1 --2- 172.21.15.16:0/3223746431 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa47c106f10 0x7fa47c074f70 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7fa47400b970 tx=0x7fa47400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:15:50.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.055+0000 7fa463fff640 1 -- 172.21.15.16:0/3223746431 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa474019070 con 0x7fa47c106f10 2024-09-12T02:15:50.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.055+0000 7fa463fff640 1 -- 172.21.15.16:0/3223746431 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa47400cb40 con 0x7fa47c106f10 2024-09-12T02:15:50.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.055+0000 7fa481041640 1 -- 172.21.15.16:0/3223746431 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa47c1a6b80 con 0x7fa47c106f10 2024-09-12T02:15:50.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.055+0000 7fa463fff640 1 -- 172.21.15.16:0/3223746431 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa474007650 con 0x7fa47c106f10 2024-09-12T02:15:50.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.055+0000 7fa481041640 1 -- 172.21.15.16:0/3223746431 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa47c1a7070 con 0x7fa47c106f10 2024-09-12T02:15:50.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.057+0000 7fa463fff640 1 -- 172.21.15.16:0/3223746431 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa47401c070 con 0x7fa47c106f10 2024-09-12T02:15:50.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.057+0000 7fa48341a740 1 Processor -- start 2024-09-12T02:15:50.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.057+0000 7fa48341a740 1 -- start start 2024-09-12T02:15:50.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.057+0000 7fa48341a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d803e664a0 0x55d803e66880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:50.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.057+0000 7fa48341a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d803e66dc0 0x55d803e71140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:50.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.057+0000 7fa48341a740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d803e71680 0x55d803e73a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:50.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.057+0000 7fa48341a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55d803dcc2e0 con 0x55d803e664a0 2024-09-12T02:15:50.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.057+0000 7fa48341a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55d803d73ce0 con 0x55d803e66dc0 2024-09-12T02:15:50.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.057+0000 7fa48341a740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55d803dc82a0 con 0x55d803e71680 2024-09-12T02:15:50.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.057+0000 7fa47a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d803e664a0 0x55d803e66880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:50.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.058+0000 7fa47a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d803e664a0 0x55d803e66880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53790/0 (socket says 172.21.15.16:53790) 2024-09-12T02:15:50.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.058+0000 7fa47a59c640 1 -- 172.21.15.16:0/16042239 learned_addr learned my addr 172.21.15.16:0/16042239 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:15:50.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.058+0000 7fa463fff640 1 --2- 172.21.15.16:0/3223746431 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fa4640a2b80 0x7fa4640a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:50.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.058+0000 7fa47ad9d640 1 --2- 172.21.15.16:0/16042239 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d803e66dc0 0x55d803e71140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:50.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.058+0000 7fa47a59c640 1 -- 172.21.15.16:0/16042239 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d803e71680 msgr2=0x55d803e73a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:15:50.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.058+0000 7fa47a59c640 1 --2- 172.21.15.16:0/16042239 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d803e71680 0x55d803e73a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:50.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.058+0000 7fa47a59c640 1 -- 172.21.15.16:0/16042239 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d803e66dc0 msgr2=0x55d803e71140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:15:50.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.058+0000 7fa47a59c640 1 --2- 172.21.15.16:0/16042239 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d803e66dc0 0x55d803e71140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:50.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.058+0000 7fa47a59c640 1 -- 172.21.15.16:0/16042239 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d803d58440 con 0x55d803e664a0 2024-09-12T02:15:50.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.058+0000 7fa47b59e640 1 --2- 172.21.15.16:0/3223746431 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fa4640a2b80 0x7fa4640a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:50.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.058+0000 7fa47a59c640 1 --2- 172.21.15.16:0/16042239 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d803e664a0 0x55d803e66880 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x7fa47000cca0 tx=0x7fa470010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc70a25e1ba8194d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:15:50.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.058+0000 7fa463fff640 1 -- 172.21.15.16:0/3223746431 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fa4740ce540 con 0x7fa47c106f10 2024-09-12T02:15:50.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.058+0000 7fa4617fa640 1 -- 172.21.15.16:0/16042239 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa470016070 con 0x55d803e664a0 2024-09-12T02:15:50.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.058+0000 7fa4617fa640 1 -- 172.21.15.16:0/16042239 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa470012b60 con 0x55d803e664a0 2024-09-12T02:15:50.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.058+0000 7fa4617fa640 1 -- 172.21.15.16:0/16042239 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa470011cc0 con 0x55d803e664a0 2024-09-12T02:15:50.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.059+0000 7fa48341a740 1 -- 172.21.15.16:0/16042239 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d803e664a0 msgr2=0x55d803e66880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:50.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.059+0000 7fa48341a740 1 --2- 172.21.15.16:0/16042239 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d803e664a0 0x55d803e66880 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x7fa47000cca0 tx=0x7fa470010740 comp rx=0 tx=0).stop 2024-09-12T02:15:50.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.059+0000 7fa47b59e640 1 --2- 172.21.15.16:0/3223746431 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fa4640a2b80 0x7fa4640a5040 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fa474097350 tx=0x7fa474096b10 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:15:50.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.059+0000 7fa48341a740 1 -- 172.21.15.16:0/16042239 shutdown_connections 2024-09-12T02:15:50.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.059+0000 7fa48341a740 1 --2- 172.21.15.16:0/16042239 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d803e71680 0x55d803e73a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:50.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.059+0000 7fa48341a740 1 --2- 172.21.15.16:0/16042239 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d803e66dc0 0x55d803e71140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:50.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.059+0000 7fa48341a740 1 --2- 172.21.15.16:0/16042239 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d803e664a0 0x55d803e66880 unknown :-1 s=CLOSED pgs=1889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:50.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.059+0000 7fa48341a740 1 -- 172.21.15.16:0/16042239 >> 172.21.15.16:0/16042239 conn(0x55d803e62a30 msgr2=0x55d803e62e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:15:50.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.059+0000 7fa48341a740 1 -- 172.21.15.16:0/16042239 shutdown_connections 2024-09-12T02:15:50.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.059+0000 7fa48341a740 1 -- 172.21.15.16:0/16042239 wait complete. 2024-09-12T02:15:50.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.059+0000 7fa48341a740 1 -- start start 2024-09-12T02:15:50.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.059+0000 7fa48341a740 1 Processor -- start 2024-09-12T02:15:50.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.060+0000 7fa48341a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d803e664a0 0x55d803e71030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:50.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.060+0000 7fa48341a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d803e66dc0 0x55d803e6e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:50.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.060+0000 7fa48341a740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d803e71680 0x55d803e6e880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:50.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.060+0000 7fa48341a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55d803e74250 con 0x55d803e664a0 2024-09-12T02:15:50.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.060+0000 7fa48341a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55d803d87050 con 0x55d803e66dc0 2024-09-12T02:15:50.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.060+0000 7fa48341a740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55d803e743d0 con 0x55d803e71680 2024-09-12T02:15:50.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.060+0000 7fa48341a740 10 client.0 _ll_register_callbacks cb 0x55d803dc7160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:15:50.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.060+0000 7fa47a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d803e664a0 0x55d803e71030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:50.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.060+0000 7fa47a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d803e664a0 0x55d803e71030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53792/0 (socket says 172.21.15.16:53792) 2024-09-12T02:15:50.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.060+0000 7fa47a59c640 1 -- 172.21.15.16:0/1266693701 learned_addr learned my addr 172.21.15.16:0/1266693701 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:15:50.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.060+0000 7fa47ad9d640 1 --2- 172.21.15.16:0/1266693701 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d803e66dc0 0x55d803e6e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:50.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.060+0000 7fa47a59c640 1 -- 172.21.15.16:0/1266693701 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d803e71680 msgr2=0x55d803e6e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:15:50.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.060+0000 7fa47a59c640 1 --2- 172.21.15.16:0/1266693701 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d803e71680 0x55d803e6e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:50.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.060+0000 7fa47a59c640 1 -- 172.21.15.16:0/1266693701 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d803e66dc0 msgr2=0x55d803e6e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:15:50.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.060+0000 7fa47a59c640 1 --2- 172.21.15.16:0/1266693701 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d803e66dc0 0x55d803e6e2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:50.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.060+0000 7fa47a59c640 1 -- 172.21.15.16:0/1266693701 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d803b3d3c0 con 0x55d803e664a0 2024-09-12T02:15:50.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.060+0000 7fa47a59c640 1 --2- 172.21.15.16:0/1266693701 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d803e664a0 0x55d803e71030 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7fa470011800 tx=0x7fa470005da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fd878d4fa69557b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:15:50.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.060+0000 7fa45eff5640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:15:50.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.060+0000 7fa45eff5640 1 -- 172.21.15.16:0/1266693701 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa470016070 con 0x55d803e664a0 2024-09-12T02:15:50.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.060+0000 7fa45eff5640 1 -- 172.21.15.16:0/1266693701 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fa470004230 con 0x55d803e664a0 2024-09-12T02:15:50.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.060+0000 7fa48341a740 1 -- 172.21.15.16:0/1266693701 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa470012b60 con 0x55d803e664a0 2024-09-12T02:15:50.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.061+0000 7fa45eff5640 1 -- 172.21.15.16:0/1266693701 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa470004430 con 0x55d803e664a0 2024-09-12T02:15:50.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.061+0000 7fa48341a740 10 client.20100 fetch_fsmap learned FSMap version 25 2024-09-12T02:15:50.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.061+0000 7fa48341a740 1 -- 172.21.15.16:0/1266693701 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d803e7fc60 con 0x55d803e664a0 2024-09-12T02:15:50.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.061+0000 7fa45eff5640 1 -- 172.21.15.16:0/1266693701 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa470012b60 con 0x55d803e664a0 2024-09-12T02:15:50.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.061+0000 7fa45eff5640 1 -- 172.21.15.16:0/1266693701 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fa4700046c0 con 0x55d803e664a0 2024-09-12T02:15:50.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.061+0000 7fa48341a740 10 client.20100 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:15:50.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.061+0000 7fa48341a740 10 client.20100 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:15:50.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.061+0000 7fa48341a740 20 client.20100 populate_metadata read hostname 'smithi016' 2024-09-12T02:15:50.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.061+0000 7fa48341a740 1 --2- 172.21.15.16:0/1266693701 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55d803e81070 0x55d803e83460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:50.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.061+0000 7fa48341a740 4 client.20100 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:15:50.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.061+0000 7fa48341a740 1 -- 172.21.15.16:0/1266693701 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d803dd9850 con 0x55d803e81070 2024-09-12T02:15:50.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.062+0000 7fa47ad9d640 1 --2- 172.21.15.16:0/1266693701 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55d803e81070 0x55d803e83460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:50.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.062+0000 7fa45eff5640 10 client.20100 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:15:50.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.062+0000 7fa47ad9d640 1 --2- 172.21.15.16:0/1266693701 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55d803e81070 0x55d803e83460 crc :-1 s=READY pgs=417 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11613ddd48c48e02 server_cookie=fb5fd2463250afdd in_seq=0 out_seq=0 2024-09-12T02:15:50.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.064+0000 7fa45eff5640 1 -- 172.21.15.16:0/1266693701 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d803dd9850 con 0x55d803e81070 2024-09-12T02:15:50.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.064+0000 7fa45eff5640 10 client.20100 handle_command_reply: tid=0 2024-09-12T02:15:50.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.064+0000 7fa48341a740 1 client.20100 shutdown 2024-09-12T02:15:50.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.064+0000 7fa48341a740 1 -- 172.21.15.16:0/1266693701 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d803e664a0 msgr2=0x55d803e71030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:50.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.064+0000 7fa48341a740 1 --2- 172.21.15.16:0/1266693701 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d803e664a0 0x55d803e71030 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7fa470011800 tx=0x7fa470005da0 comp rx=0 tx=0).stop 2024-09-12T02:15:50.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.064+0000 7fa48341a740 1 -- 172.21.15.16:0/1266693701 shutdown_connections 2024-09-12T02:15:50.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.064+0000 7fa48341a740 1 --2- 172.21.15.16:0/1266693701 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55d803e81070 0x55d803e83460 crc :-1 s=READY pgs=417 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:50.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.064+0000 7fa48341a740 1 --2- 172.21.15.16:0/1266693701 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d803e71680 0x55d803e6e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:50.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.064+0000 7fa48341a740 1 --2- 172.21.15.16:0/1266693701 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d803e66dc0 0x55d803e6e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:50.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.064+0000 7fa48341a740 1 --2- 172.21.15.16:0/1266693701 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d803e664a0 0x55d803e71030 unknown :-1 s=CLOSED pgs=1890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:50.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.064+0000 7fa45eff5640 0 client.20100 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:15:50.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.064+0000 7fa48341a740 1 -- 172.21.15.16:0/1266693701 >> 172.21.15.16:0/1266693701 conn(0x55d803e62a30 msgr2=0x55d803e68760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:15:50.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.064+0000 7fa48341a740 1 -- 172.21.15.16:0/1266693701 shutdown_connections 2024-09-12T02:15:50.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.064+0000 7fa48341a740 1 -- 172.21.15.16:0/1266693701 wait complete. 2024-09-12T02:15:50.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.064+0000 7fa48341a740 20 client.20100 trim_cache size 0 max 16384 2024-09-12T02:15:50.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.080+0000 7fa48341a740 1 Processor -- start 2024-09-12T02:15:50.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.080+0000 7fa48341a740 1 -- start start 2024-09-12T02:15:50.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.081+0000 7fa48341a740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d803e664a0 0x55d803e88a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:50.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.081+0000 7fa48341a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d803e66dc0 0x55d803e88fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:50.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.081+0000 7fa48341a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d803e71680 0x55d803e6b560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:50.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.081+0000 7fa48341a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa47c1b1eb0 con 0x55d803e66dc0 2024-09-12T02:15:50.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.081+0000 7fa48341a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa47c1af780 con 0x55d803e71680 2024-09-12T02:15:50.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.081+0000 7fa48341a740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa47c1ad5f0 con 0x55d803e664a0 2024-09-12T02:15:50.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.081+0000 7fa47ad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d803e66dc0 0x55d803e88fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:50.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.081+0000 7fa47ad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d803e66dc0 0x55d803e88fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53802/0 (socket says 172.21.15.16:53802) 2024-09-12T02:15:50.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.081+0000 7fa47ad9d640 1 -- 172.21.15.16:0/3291860835 learned_addr learned my addr 172.21.15.16:0/3291860835 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:15:50.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.081+0000 7fa47a59c640 1 --2- 172.21.15.16:0/3291860835 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d803e71680 0x55d803e6b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:50.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.081+0000 7fa47a59c640 1 --2- 172.21.15.16:0/3291860835 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d803e664a0 0x55d803e88a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:50.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.081+0000 7fa47ad9d640 1 -- 172.21.15.16:0/3291860835 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d803e664a0 msgr2=0x55d803e88a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:15:50.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.081+0000 7fa47ad9d640 1 --2- 172.21.15.16:0/3291860835 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d803e664a0 0x55d803e88a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:50.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.081+0000 7fa47ad9d640 1 -- 172.21.15.16:0/3291860835 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d803e71680 msgr2=0x55d803e6b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:15:50.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.081+0000 7fa47ad9d640 1 --2- 172.21.15.16:0/3291860835 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d803e71680 0x55d803e6b560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:50.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.081+0000 7fa47ad9d640 1 -- 172.21.15.16:0/3291860835 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d803e713a0 con 0x55d803e66dc0 2024-09-12T02:15:50.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.081+0000 7fa47a59c640 1 --2- 172.21.15.16:0/3291860835 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d803e664a0 0x55d803e88a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:15:50.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.081+0000 7fa47ad9d640 1 --2- 172.21.15.16:0/3291860835 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d803e66dc0 0x55d803e88fd0 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7fa46c02a930 tx=0x7fa46c031ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23adc07639b6ac01 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:15:50.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.081+0000 7fa460ff9640 1 -- 172.21.15.16:0/3291860835 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa46c035040 con 0x55d803e66dc0 2024-09-12T02:15:50.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.081+0000 7fa460ff9640 1 -- 172.21.15.16:0/3291860835 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa46c004240 con 0x55d803e66dc0 2024-09-12T02:15:50.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.082+0000 7fa460ff9640 1 -- 172.21.15.16:0/3291860835 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa46c047730 con 0x55d803e66dc0 2024-09-12T02:15:50.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.082+0000 7fa48341a740 1 -- 172.21.15.16:0/3291860835 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d803e66dc0 msgr2=0x55d803e88fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:50.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.082+0000 7fa48341a740 1 --2- 172.21.15.16:0/3291860835 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d803e66dc0 0x55d803e88fd0 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7fa46c02a930 tx=0x7fa46c031ad0 comp rx=0 tx=0).stop 2024-09-12T02:15:50.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.082+0000 7fa48341a740 1 -- 172.21.15.16:0/3291860835 shutdown_connections 2024-09-12T02:15:50.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.082+0000 7fa48341a740 1 --2- 172.21.15.16:0/3291860835 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d803e71680 0x55d803e6b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:50.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.082+0000 7fa48341a740 1 --2- 172.21.15.16:0/3291860835 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d803e66dc0 0x55d803e88fd0 unknown :-1 s=CLOSED pgs=1891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:50.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.082+0000 7fa48341a740 1 --2- 172.21.15.16:0/3291860835 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d803e664a0 0x55d803e88a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:50.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.082+0000 7fa48341a740 1 -- 172.21.15.16:0/3291860835 >> 172.21.15.16:0/3291860835 conn(0x55d803e62a30 msgr2=0x55d803e61b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:15:50.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.082+0000 7fa48341a740 1 -- 172.21.15.16:0/3291860835 shutdown_connections 2024-09-12T02:15:50.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.082+0000 7fa48341a740 1 -- 172.21.15.16:0/3291860835 wait complete. 2024-09-12T02:15:50.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.083+0000 7fa48341a740 1 -- start start 2024-09-12T02:15:50.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.083+0000 7fa48341a740 1 Processor -- start 2024-09-12T02:15:50.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.083+0000 7fa48341a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d803e664a0 0x55d803e79e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:50.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.083+0000 7fa48341a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d803e66dc0 0x55d803e771e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:50.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.083+0000 7fa48341a740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d803e71680 0x55d803e777d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:50.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.083+0000 7fa48341a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa47c1adc40 con 0x55d803e66dc0 2024-09-12T02:15:50.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.083+0000 7fa48341a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa47c1af0b0 con 0x55d803e664a0 2024-09-12T02:15:50.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.083+0000 7fa48341a740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa47c1ae310 con 0x55d803e71680 2024-09-12T02:15:50.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.083+0000 7fa48341a740 10 client.0 _ll_register_callbacks cb 0x7fa47000cba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:15:50.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.083+0000 7fa47ad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d803e66dc0 0x55d803e771e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:50.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.083+0000 7fa47ad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d803e66dc0 0x55d803e771e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53810/0 (socket says 172.21.15.16:53810) 2024-09-12T02:15:50.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.083+0000 7fa47ad9d640 1 -- 172.21.15.16:0/1683889711 learned_addr learned my addr 172.21.15.16:0/1683889711 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:15:50.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.083+0000 7fa47a59c640 1 --2- 172.21.15.16:0/1683889711 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d803e664a0 0x55d803e79e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:50.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.083+0000 7fa47a59c640 1 --2- 172.21.15.16:0/1683889711 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d803e71680 0x55d803e777d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:50.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.083+0000 7fa47ad9d640 1 -- 172.21.15.16:0/1683889711 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d803e71680 msgr2=0x55d803e777d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:15:50.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.083+0000 7fa47ad9d640 1 --2- 172.21.15.16:0/1683889711 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d803e71680 0x55d803e777d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:50.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.083+0000 7fa47ad9d640 1 -- 172.21.15.16:0/1683889711 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d803e664a0 msgr2=0x55d803e79e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:15:50.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.083+0000 7fa47ad9d640 1 --2- 172.21.15.16:0/1683889711 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d803e664a0 0x55d803e79e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:50.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.083+0000 7fa47ad9d640 1 -- 172.21.15.16:0/1683889711 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d803e78000 con 0x55d803e66dc0 2024-09-12T02:15:50.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.083+0000 7fa47a59c640 1 --2- 172.21.15.16:0/1683889711 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d803e71680 0x55d803e777d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:15:50.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.084+0000 7fa47ad9d640 1 --2- 172.21.15.16:0/1683889711 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d803e66dc0 0x55d803e771e0 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7fa46c009230 tx=0x7fa46c02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=643d9e2f81c3b74c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:15:50.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.084+0000 7fa45eff5640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:15:50.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.084+0000 7fa45eff5640 1 -- 172.21.15.16:0/1683889711 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa46c035040 con 0x55d803e66dc0 2024-09-12T02:15:50.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.084+0000 7fa45eff5640 1 -- 172.21.15.16:0/1683889711 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fa46c0043f0 con 0x55d803e66dc0 2024-09-12T02:15:50.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.084+0000 7fa48341a740 1 -- 172.21.15.16:0/1683889711 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa46c004240 con 0x55d803e66dc0 2024-09-12T02:15:50.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.084+0000 7fa45eff5640 1 -- 172.21.15.16:0/1683889711 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa46c0035f0 con 0x55d803e66dc0 2024-09-12T02:15:50.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.084+0000 7fa48341a740 10 client.20106 fetch_fsmap learned FSMap version 25 2024-09-12T02:15:50.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.084+0000 7fa48341a740 1 -- 172.21.15.16:0/1683889711 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d803eebb00 con 0x55d803e66dc0 2024-09-12T02:15:50.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.084+0000 7fa45eff5640 1 -- 172.21.15.16:0/1683889711 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa46c004240 con 0x55d803e66dc0 2024-09-12T02:15:50.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.084+0000 7fa45eff5640 1 -- 172.21.15.16:0/1683889711 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fa46c0479c0 con 0x55d803e66dc0 2024-09-12T02:15:50.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.084+0000 7fa48341a740 10 client.20106 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:15:50.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.084+0000 7fa48341a740 10 client.20106 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:15:50.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.084+0000 7fa48341a740 20 client.20106 populate_metadata read hostname 'smithi016' 2024-09-12T02:15:50.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.085+0000 7fa48341a740 1 --2- 172.21.15.16:0/1683889711 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55d803eec250 0x55d803eee670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:50.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.085+0000 7fa48341a740 4 client.20106 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T02:15:50.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.085+0000 7fa48341a740 1 -- 172.21.15.16:0/1683889711 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d803dd3d60 con 0x55d803eec250 2024-09-12T02:15:50.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.085+0000 7fa47a59c640 1 --2- 172.21.15.16:0/1683889711 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55d803eec250 0x55d803eee670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:50.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.085+0000 7fa47a59c640 1 --2- 172.21.15.16:0/1683889711 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55d803eec250 0x55d803eee670 crc :-1 s=READY pgs=418 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39cbc5c97b1e0754 server_cookie=7c8e6978e1277ee in_seq=0 out_seq=0 2024-09-12T02:15:50.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.085+0000 7fa45eff5640 10 client.20106 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:15:50.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.085+0000 7fa45eff5640 1 -- 172.21.15.16:0/1683889711 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55d803dd3d60 con 0x55d803eec250 2024-09-12T02:15:50.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.085+0000 7fa45eff5640 10 client.20106 handle_command_reply: tid=0 2024-09-12T02:15:50.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.085+0000 7fa48341a740 1 client.20106 shutdown 2024-09-12T02:15:50.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.085+0000 7fa48341a740 1 -- 172.21.15.16:0/1683889711 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d803e66dc0 msgr2=0x55d803e771e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:50.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.085+0000 7fa48341a740 1 --2- 172.21.15.16:0/1683889711 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d803e66dc0 0x55d803e771e0 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7fa46c009230 tx=0x7fa46c02a410 comp rx=0 tx=0).stop 2024-09-12T02:15:50.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.086+0000 7fa48341a740 1 -- 172.21.15.16:0/1683889711 shutdown_connections 2024-09-12T02:15:50.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.086+0000 7fa48341a740 1 --2- 172.21.15.16:0/1683889711 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55d803eec250 0x55d803eee670 crc :-1 s=READY pgs=418 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:50.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.086+0000 7fa48341a740 1 --2- 172.21.15.16:0/1683889711 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d803e71680 0x55d803e777d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:50.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.086+0000 7fa48341a740 1 --2- 172.21.15.16:0/1683889711 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d803e66dc0 0x55d803e771e0 unknown :-1 s=CLOSED pgs=1892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:50.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.086+0000 7fa48341a740 1 --2- 172.21.15.16:0/1683889711 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d803e664a0 0x55d803e79e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:50.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.086+0000 7fa48341a740 1 -- 172.21.15.16:0/1683889711 >> 172.21.15.16:0/1683889711 conn(0x55d803e62a30 msgr2=0x55d803e6fb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:15:50.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.086+0000 7fa45eff5640 0 client.20106 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:15:50.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.086+0000 7fa48341a740 1 -- 172.21.15.16:0/1683889711 shutdown_connections 2024-09-12T02:15:50.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.086+0000 7fa48341a740 1 -- 172.21.15.16:0/1683889711 wait complete. 2024-09-12T02:15:50.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.086+0000 7fa48341a740 20 client.20106 trim_cache size 0 max 16384 2024-09-12T02:15:50.100 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T02:15:50.100 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-12T02:15:50.100 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": { 2024-09-12T02:15:50.100 INFO:teuthology.orchestra.run.smithi016.stdout: "44ea79e1-6891-4db1-900e-a672d238e67c": { 2024-09-12T02:15:50.100 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-12T02:15:50.100 INFO:teuthology.orchestra.run.smithi016.stdout: "tag": "44ea79e1-6891-4db1-900e-a672d238e67c", 2024-09-12T02:15:50.100 INFO:teuthology.orchestra.run.smithi016.stdout: "options": "recursive,force" 2024-09-12T02:15:50.100 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:15:50.100 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:15:50.101 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T02:15:50.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.086+0000 7fa481041640 1 -- 172.21.15.16:0/3223746431 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fa4640a2b80 msgr2=0x7fa4640a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:50.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.086+0000 7fa481041640 1 --2- 172.21.15.16:0/3223746431 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fa4640a2b80 0x7fa4640a5040 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fa474097350 tx=0x7fa474096b10 comp rx=0 tx=0).stop 2024-09-12T02:15:50.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.086+0000 7fa481041640 1 -- 172.21.15.16:0/3223746431 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa47c106f10 msgr2=0x7fa47c074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:50.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.086+0000 7fa481041640 1 --2- 172.21.15.16:0/3223746431 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa47c106f10 0x7fa47c074f70 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7fa47400b970 tx=0x7fa47400be30 comp rx=0 tx=0).stop 2024-09-12T02:15:50.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.086+0000 7fa481041640 1 -- 172.21.15.16:0/3223746431 shutdown_connections 2024-09-12T02:15:50.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.086+0000 7fa481041640 1 --2- 172.21.15.16:0/3223746431 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fa4640a2b80 0x7fa4640a5040 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:50.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.086+0000 7fa481041640 1 --2- 172.21.15.16:0/3223746431 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa47c106f10 0x7fa47c074f70 unknown :-1 s=CLOSED pgs=1888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:50.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.086+0000 7fa481041640 1 --2- 172.21.15.16:0/3223746431 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa47c102790 0x7fa47c074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:50.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.086+0000 7fa481041640 1 --2- 172.21.15.16:0/3223746431 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa47c101de0 0x7fa47c0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:50.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.086+0000 7fa481041640 1 -- 172.21.15.16:0/3223746431 >> 172.21.15.16:0/3223746431 conn(0x7fa47c0fd8c0 msgr2=0x7fa47c0fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:15:50.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.087+0000 7fa481041640 1 -- 172.21.15.16:0/3223746431 shutdown_connections 2024-09-12T02:15:50.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:15:50.087+0000 7fa481041640 1 -- 172.21.15.16:0/3223746431 wait complete. 2024-09-12T02:15:50.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:15:50 smithi184 ceph-mon[35593]: pgmap v752: 97 pgs: 97 active+clean; 3.1 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 2.1 KiB/s wr, 6 op/s 2024-09-12T02:15:50.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:15:50 smithi184 ceph-mon[35593]: scrub queued for path: / 2024-09-12T02:15:50.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:15:50 smithi184 ceph-mon[35593]: scrub summary: idle+waiting paths [/] 2024-09-12T02:15:50.628 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:15:50 smithi184 ceph-mon[35593]: scrub summary: active paths [/] 2024-09-12T02:15:50.628 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:15:50 smithi184 ceph-mon[35593]: scrub summary: idle+waiting paths [/] 2024-09-12T02:15:50.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:15:50 smithi033 ceph-mon[35332]: pgmap v752: 97 pgs: 97 active+clean; 3.1 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 2.1 KiB/s wr, 6 op/s 2024-09-12T02:15:50.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:15:50 smithi033 ceph-mon[35332]: scrub queued for path: / 2024-09-12T02:15:50.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:15:50 smithi033 ceph-mon[35332]: scrub summary: idle+waiting paths [/] 2024-09-12T02:15:50.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:15:50 smithi033 ceph-mon[35332]: scrub summary: active paths [/] 2024-09-12T02:15:50.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:15:50 smithi033 ceph-mon[35332]: scrub summary: idle+waiting paths [/] 2024-09-12T02:15:50.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:15:50 smithi016 ceph-mon[30947]: pgmap v752: 97 pgs: 97 active+clean; 3.1 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 2.1 KiB/s wr, 6 op/s 2024-09-12T02:15:50.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:15:50 smithi016 ceph-mon[30947]: scrub queued for path: / 2024-09-12T02:15:50.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:15:50 smithi016 ceph-mon[30947]: scrub summary: idle+waiting paths [/] 2024-09-12T02:15:50.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:15:50 smithi016 ceph-mon[30947]: scrub summary: active paths [/] 2024-09-12T02:15:50.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:15:50 smithi016 ceph-mon[30947]: scrub summary: idle+waiting paths [/] 2024-09-12T02:15:50.779 INFO:tasks.cephfs.filesystem:scrub status for tag:44ea79e1-6891-4db1-900e-a672d238e67c - {'path': '/', 'tag': '44ea79e1-6891-4db1-900e-a672d238e67c', 'options': 'recursive,force'} 2024-09-12T02:15:52.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:15:52 smithi184 ceph-mon[35593]: pgmap v753: 97 pgs: 97 active+clean; 3.1 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.7 KiB/s wr, 10 op/s 2024-09-12T02:15:52.628 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:15:52 smithi184 ceph-mon[35593]: scrub summary: idle 2024-09-12T02:15:52.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:15:52 smithi033 ceph-mon[35332]: pgmap v753: 97 pgs: 97 active+clean; 3.1 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.7 KiB/s wr, 10 op/s 2024-09-12T02:15:52.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:15:52 smithi033 ceph-mon[35332]: scrub summary: idle 2024-09-12T02:15:52.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:15:52 smithi016 ceph-mon[30947]: pgmap v753: 97 pgs: 97 active+clean; 3.1 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.7 KiB/s wr, 10 op/s 2024-09-12T02:15:52.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:15:52 smithi016 ceph-mon[30947]: scrub summary: idle 2024-09-12T02:15:54.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:15:54 smithi184 ceph-mon[35593]: pgmap v754: 97 pgs: 97 active+clean; 3.1 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 2.8 KiB/s wr, 10 op/s 2024-09-12T02:15:54.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:15:54 smithi033 ceph-mon[35332]: pgmap v754: 97 pgs: 97 active+clean; 3.1 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 2.8 KiB/s wr, 10 op/s 2024-09-12T02:15:54.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:15:54 smithi016 ceph-mon[30947]: pgmap v754: 97 pgs: 97 active+clean; 3.1 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 2.8 KiB/s wr, 10 op/s 2024-09-12T02:15:56.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:15:56 smithi184 ceph-mon[35593]: pgmap v755: 97 pgs: 97 active+clean; 3.1 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 2.8 KiB/s wr, 10 op/s 2024-09-12T02:15:56.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:15:56 smithi033 ceph-mon[35332]: pgmap v755: 97 pgs: 97 active+clean; 3.1 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 2.8 KiB/s wr, 10 op/s 2024-09-12T02:15:56.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:15:56 smithi016 ceph-mon[30947]: pgmap v755: 97 pgs: 97 active+clean; 3.1 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 2.8 KiB/s wr, 10 op/s 2024-09-12T02:15:58.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:15:58 smithi184 ceph-mon[35593]: pgmap v756: 97 pgs: 97 active+clean; 3.1 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 2.8 KiB/s wr, 11 op/s 2024-09-12T02:15:58.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:15:58 smithi033 ceph-mon[35332]: pgmap v756: 97 pgs: 97 active+clean; 3.1 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 2.8 KiB/s wr, 11 op/s 2024-09-12T02:15:58.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:15:58 smithi016 ceph-mon[30947]: pgmap v756: 97 pgs: 97 active+clean; 3.1 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 2.8 KiB/s wr, 11 op/s 2024-09-12T02:15:58.793 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.793+0000 7f5ad6df8640 1 -- 172.21.15.16:0/1241300721 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ad0100a40 msgr2=0x7f5ad0100e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:58.794 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.793+0000 7f5ad6df8640 1 --2- 172.21.15.16:0/1241300721 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ad0100a40 0x7f5ad0100e40 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7f5ac4009900 tx=0x7f5ac402a410 comp rx=0 tx=0).stop 2024-09-12T02:15:58.794 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.793+0000 7f5ad6df8640 1 -- 172.21.15.16:0/1241300721 shutdown_connections 2024-09-12T02:15:58.794 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.793+0000 7f5ad6df8640 1 --2- 172.21.15.16:0/1241300721 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ad01025f0 0x7f5ad0108e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:58.794 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.793+0000 7f5ad6df8640 1 --2- 172.21.15.16:0/1241300721 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5ad0101c30 0x7f5ad01020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:58.794 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.793+0000 7f5ad6df8640 1 --2- 172.21.15.16:0/1241300721 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ad0100a40 0x7f5ad0100e40 unknown :-1 s=CLOSED pgs=1893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:58.794 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.793+0000 7f5ad6df8640 1 -- 172.21.15.16:0/1241300721 >> 172.21.15.16:0/1241300721 conn(0x7f5ad00fc1e0 msgr2=0x7f5ad00fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:15:58.794 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.793+0000 7f5ad6df8640 1 -- 172.21.15.16:0/1241300721 shutdown_connections 2024-09-12T02:15:58.794 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.793+0000 7f5ad6df8640 1 -- 172.21.15.16:0/1241300721 wait complete. 2024-09-12T02:15:58.796 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.794+0000 7f5ad6df8640 1 Processor -- start 2024-09-12T02:15:58.796 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.794+0000 7f5ad6df8640 1 -- start start 2024-09-12T02:15:58.797 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.794+0000 7f5ad6df8640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5ad0100a40 0x7f5ad0197e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:58.797 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.794+0000 7f5ad6df8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ad0101c30 0x7f5ad0198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:58.797 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.794+0000 7f5ad6df8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ad01025f0 0x7f5ad019f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:58.797 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.794+0000 7f5ad6df8640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5ad010c6c0 con 0x7f5ad0101c30 2024-09-12T02:15:58.797 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.794+0000 7f5ad6df8640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5ad010c540 con 0x7f5ad01025f0 2024-09-12T02:15:58.797 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.794+0000 7f5ad6df8640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5ad010c840 con 0x7f5ad0100a40 2024-09-12T02:15:58.798 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.794+0000 7f5acffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ad0101c30 0x7f5ad0198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:58.798 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.794+0000 7f5acffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ad0101c30 0x7f5ad0198360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37314/0 (socket says 172.21.15.16:37314) 2024-09-12T02:15:58.798 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.794+0000 7f5acffff640 1 -- 172.21.15.16:0/927034657 learned_addr learned my addr 172.21.15.16:0/927034657 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:15:58.798 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.794+0000 7f5ad5395640 1 --2- 172.21.15.16:0/927034657 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ad01025f0 0x7f5ad019f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:58.798 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.794+0000 7f5ad4b94640 1 --2- 172.21.15.16:0/927034657 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5ad0100a40 0x7f5ad0197e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:58.798 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.795+0000 7f5acffff640 1 -- 172.21.15.16:0/927034657 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5ad0100a40 msgr2=0x7f5ad0197e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:58.799 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.795+0000 7f5acffff640 1 --2- 172.21.15.16:0/927034657 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5ad0100a40 0x7f5ad0197e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:58.799 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.795+0000 7f5acffff640 1 -- 172.21.15.16:0/927034657 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ad01025f0 msgr2=0x7f5ad019f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:58.799 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.795+0000 7f5acffff640 1 --2- 172.21.15.16:0/927034657 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ad01025f0 0x7f5ad019f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:58.799 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.795+0000 7f5acffff640 1 -- 172.21.15.16:0/927034657 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ad019fa40 con 0x7f5ad0101c30 2024-09-12T02:15:58.799 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.795+0000 7f5acffff640 1 --2- 172.21.15.16:0/927034657 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ad0101c30 0x7f5ad0198360 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x7f5abc00d770 tx=0x7f5abc00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:15:58.800 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.795+0000 7f5acdffb640 1 -- 172.21.15.16:0/927034657 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5abc017070 con 0x7f5ad0101c30 2024-09-12T02:15:58.800 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.795+0000 7f5acdffb640 1 -- 172.21.15.16:0/927034657 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5abc005500 con 0x7f5ad0101c30 2024-09-12T02:15:58.800 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.795+0000 7f5ad6df8640 1 -- 172.21.15.16:0/927034657 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ad019fd30 con 0x7f5ad0101c30 2024-09-12T02:15:58.800 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.795+0000 7f5acdffb640 1 -- 172.21.15.16:0/927034657 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5abc004e00 con 0x7f5ad0101c30 2024-09-12T02:15:58.800 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.795+0000 7f5ad6df8640 1 -- 172.21.15.16:0/927034657 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ad01a0220 con 0x7f5ad0101c30 2024-09-12T02:15:58.810 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.798+0000 7f5acdffb640 1 -- 172.21.15.16:0/927034657 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5abc004fa0 con 0x7f5ad0101c30 2024-09-12T02:15:58.810 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.798+0000 7f5ad9c67740 1 Processor -- start 2024-09-12T02:15:58.810 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.798+0000 7f5ad9c67740 1 -- start start 2024-09-12T02:15:58.810 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.798+0000 7f5ad9c67740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556da18649a0 0x556da1864da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:58.811 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.798+0000 7f5ad9c67740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556da18652e0 0x556da1869da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:58.811 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.798+0000 7f5ad9c67740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556da186a370 0x556da186c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:58.811 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.798+0000 7f5ad9c67740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x556da17c82f0 con 0x556da186a370 2024-09-12T02:15:58.811 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.798+0000 7f5ad9c67740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x556da1770270 con 0x556da18649a0 2024-09-12T02:15:58.811 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.798+0000 7f5ad9c67740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x556da17c42b0 con 0x556da18652e0 2024-09-12T02:15:58.811 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.798+0000 7f5acdffb640 1 --2- 172.21.15.16:0/927034657 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f5ac00a2cb0 0x7f5ac00a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:58.811 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.798+0000 7f5acffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556da186a370 0x556da186c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:58.811 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.798+0000 7f5ad5395640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556da18649a0 0x556da1864da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:58.811 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.798+0000 7f5ad5395640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556da18649a0 0x556da1864da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.16:38810/0 (socket says 172.21.15.16:38810) 2024-09-12T02:15:58.811 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.798+0000 7f5acffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556da186a370 0x556da186c830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37320/0 (socket says 172.21.15.16:37320) 2024-09-12T02:15:58.811 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.798+0000 7f5ad5395640 1 -- 172.21.15.16:0/2171662029 learned_addr learned my addr 172.21.15.16:0/2171662029 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:15:58.812 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.798+0000 7f5ad4b94640 1 --2- 172.21.15.16:0/2171662029 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556da18652e0 0x556da1869da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:58.812 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.798+0000 7f5ad5395640 1 --2- 172.21.15.16:0/927034657 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f5ac00a2cb0 0x7f5ac00a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:58.812 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.799+0000 7f5acdffb640 1 -- 172.21.15.16:0/927034657 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f5abc0ce900 con 0x7f5ad0101c30 2024-09-12T02:15:58.812 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.799+0000 7f5ad5395640 1 -- 172.21.15.16:0/2171662029 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556da18652e0 msgr2=0x556da1869da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:15:58.812 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.799+0000 7f5ad5395640 1 --2- 172.21.15.16:0/2171662029 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556da18652e0 0x556da1869da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:58.812 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.799+0000 7f5ad5395640 1 -- 172.21.15.16:0/2171662029 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556da186a370 msgr2=0x556da186c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:15:58.812 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.799+0000 7f5ad5395640 1 --2- 172.21.15.16:0/2171662029 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556da186a370 0x556da186c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:58.812 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.799+0000 7f5ad5395640 1 -- 172.21.15.16:0/2171662029 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556da176f340 con 0x556da18649a0 2024-09-12T02:15:58.813 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.799+0000 7f5ad5395640 1 --2- 172.21.15.16:0/927034657 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f5ac00a2cb0 0x7f5ac00a5170 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f5ac8010c40 tx=0x7f5ac800c000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:15:58.813 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.799+0000 7f5ad5395640 1 --2- 172.21.15.16:0/2171662029 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556da18649a0 0x556da1864da0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f5ac8012d80 tx=0x7f5ac8010740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=237e868081674a5a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:15:58.813 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.799+0000 7f5abaffd640 1 -- 172.21.15.16:0/2171662029 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ac8024070 con 0x556da18649a0 2024-09-12T02:15:58.813 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.799+0000 7f5abaffd640 1 -- 172.21.15.16:0/2171662029 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ac80125a0 con 0x556da18649a0 2024-09-12T02:15:58.813 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.799+0000 7f5abaffd640 1 -- 172.21.15.16:0/2171662029 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ac8005260 con 0x556da18649a0 2024-09-12T02:15:58.813 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.800+0000 7f5ad9c67740 1 -- 172.21.15.16:0/2171662029 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556da18649a0 msgr2=0x556da1864da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:58.813 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.800+0000 7f5ad9c67740 1 --2- 172.21.15.16:0/2171662029 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556da18649a0 0x556da1864da0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f5ac8012d80 tx=0x7f5ac8010740 comp rx=0 tx=0).stop 2024-09-12T02:15:58.813 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.800+0000 7f5ad9c67740 1 -- 172.21.15.16:0/2171662029 shutdown_connections 2024-09-12T02:15:58.813 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.800+0000 7f5ad9c67740 1 --2- 172.21.15.16:0/2171662029 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556da186a370 0x556da186c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:58.813 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.800+0000 7f5ad9c67740 1 --2- 172.21.15.16:0/2171662029 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556da18652e0 0x556da1869da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:58.813 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.800+0000 7f5ad9c67740 1 --2- 172.21.15.16:0/2171662029 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556da18649a0 0x556da1864da0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:58.813 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.800+0000 7f5ad9c67740 1 -- 172.21.15.16:0/2171662029 >> 172.21.15.16:0/2171662029 conn(0x556da1860010 msgr2=0x556da1860410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:15:58.814 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.800+0000 7f5ad9c67740 1 -- 172.21.15.16:0/2171662029 shutdown_connections 2024-09-12T02:15:58.814 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.800+0000 7f5ad9c67740 1 -- 172.21.15.16:0/2171662029 wait complete. 2024-09-12T02:15:58.814 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.801+0000 7f5ad9c67740 1 -- start start 2024-09-12T02:15:58.814 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.801+0000 7f5ad9c67740 1 Processor -- start 2024-09-12T02:15:58.814 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.801+0000 7f5ad9c67740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556da18649a0 0x556da186eba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:58.814 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.801+0000 7f5ad9c67740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556da18652e0 0x556da186f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:58.814 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.801+0000 7f5ad9c67740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556da186a370 0x556da18760b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:58.814 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.801+0000 7f5ad9c67740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x556da186d2e0 con 0x556da186a370 2024-09-12T02:15:58.814 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.801+0000 7f5ad9c67740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x556da1751290 con 0x556da18652e0 2024-09-12T02:15:58.814 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.801+0000 7f5ad9c67740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x556da186d460 con 0x556da18649a0 2024-09-12T02:15:58.814 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.801+0000 7f5ad9c67740 10 client.0 _ll_register_callbacks cb 0x556da1782140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:15:58.814 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.801+0000 7f5ad5395640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556da186a370 0x556da18760b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:58.814 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.801+0000 7f5ad5395640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556da186a370 0x556da18760b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37322/0 (socket says 172.21.15.16:37322) 2024-09-12T02:15:58.815 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.801+0000 7f5ad5395640 1 -- 172.21.15.16:0/3361265827 learned_addr learned my addr 172.21.15.16:0/3361265827 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:15:58.815 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.801+0000 7f5acffff640 1 --2- 172.21.15.16:0/3361265827 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556da18652e0 0x556da186f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:58.815 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.801+0000 7f5ad4b94640 1 --2- 172.21.15.16:0/3361265827 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556da18649a0 0x556da186eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:58.815 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.801+0000 7f5ad5395640 1 -- 172.21.15.16:0/3361265827 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556da18649a0 msgr2=0x556da186eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:15:58.815 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.801+0000 7f5ad5395640 1 --2- 172.21.15.16:0/3361265827 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556da18649a0 0x556da186eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:58.815 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.801+0000 7f5ad5395640 1 -- 172.21.15.16:0/3361265827 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556da18652e0 msgr2=0x556da186f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:15:58.815 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.801+0000 7f5ad5395640 1 --2- 172.21.15.16:0/3361265827 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556da18652e0 0x556da186f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:58.815 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.801+0000 7f5ad5395640 1 -- 172.21.15.16:0/3361265827 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556da14db210 con 0x556da186a370 2024-09-12T02:15:58.815 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.802+0000 7f5ad5395640 1 --2- 172.21.15.16:0/3361265827 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556da186a370 0x556da18760b0 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7f5ac8012fd0 tx=0x7f5ac800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d231f02ad058d587 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:15:58.815 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.802+0000 7f5aaffff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:15:58.815 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.802+0000 7f5aaffff640 1 -- 172.21.15.16:0/3361265827 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ac8024050 con 0x556da186a370 2024-09-12T02:15:58.815 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.802+0000 7f5aaffff640 1 -- 172.21.15.16:0/3361265827 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f5ac8004a00 con 0x556da186a370 2024-09-12T02:15:58.815 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.802+0000 7f5ad9c67740 1 -- 172.21.15.16:0/3361265827 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ac80125a0 con 0x556da186a370 2024-09-12T02:15:58.816 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.802+0000 7f5aaffff640 1 -- 172.21.15.16:0/3361265827 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5ac801b040 con 0x556da186a370 2024-09-12T02:15:58.816 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.802+0000 7f5ad9c67740 10 client.20118 fetch_fsmap learned FSMap version 25 2024-09-12T02:15:58.816 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.802+0000 7f5ad9c67740 1 -- 172.21.15.16:0/3361265827 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x556da15a7620 con 0x556da186a370 2024-09-12T02:15:58.816 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.802+0000 7f5aaffff640 1 -- 172.21.15.16:0/3361265827 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ac80125a0 con 0x556da186a370 2024-09-12T02:15:58.816 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.802+0000 7f5aaffff640 1 -- 172.21.15.16:0/3361265827 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f5ac800f040 con 0x556da186a370 2024-09-12T02:15:58.816 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.802+0000 7f5ad9c67740 10 client.20118 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:15:58.816 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.802+0000 7f5ad9c67740 10 client.20118 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:15:58.816 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.803+0000 7f5ad9c67740 20 client.20118 populate_metadata read hostname 'smithi016' 2024-09-12T02:15:58.816 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.803+0000 7f5ad9c67740 1 --2- 172.21.15.16:0/3361265827 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x556da187a640 0x556da187ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:58.816 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.803+0000 7f5ad9c67740 4 client.20118 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:15:58.816 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.803+0000 7f5ad9c67740 1 -- 172.21.15.16:0/3361265827 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556da17d5a20 con 0x556da187a640 2024-09-12T02:15:58.816 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.803+0000 7f5ad4b94640 1 --2- 172.21.15.16:0/3361265827 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x556da187a640 0x556da187ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:58.816 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.803+0000 7f5aaffff640 10 client.20118 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:15:58.817 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.803+0000 7f5ad4b94640 1 --2- 172.21.15.16:0/3361265827 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x556da187a640 0x556da187ca30 crc :-1 s=READY pgs=419 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=288000dd357203b9 server_cookie=b0c36f819a904270 in_seq=0 out_seq=0 2024-09-12T02:15:58.817 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.804+0000 7f5aaffff640 1 -- 172.21.15.16:0/3361265827 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556da17d5a20 con 0x556da187a640 2024-09-12T02:15:58.817 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.804+0000 7f5aaffff640 10 client.20118 handle_command_reply: tid=0 2024-09-12T02:15:58.817 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.804+0000 7f5ad9c67740 1 client.20118 shutdown 2024-09-12T02:15:58.817 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.804+0000 7f5ad9c67740 1 -- 172.21.15.16:0/3361265827 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556da186a370 msgr2=0x556da18760b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:58.817 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.804+0000 7f5ad9c67740 1 --2- 172.21.15.16:0/3361265827 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556da186a370 0x556da18760b0 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7f5ac8012fd0 tx=0x7f5ac800cce0 comp rx=0 tx=0).stop 2024-09-12T02:15:58.817 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.804+0000 7f5ad9c67740 1 -- 172.21.15.16:0/3361265827 shutdown_connections 2024-09-12T02:15:58.817 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.804+0000 7f5ad9c67740 1 --2- 172.21.15.16:0/3361265827 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x556da187a640 0x556da187ca30 crc :-1 s=READY pgs=419 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:58.817 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.804+0000 7f5ad9c67740 1 --2- 172.21.15.16:0/3361265827 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556da186a370 0x556da18760b0 unknown :-1 s=CLOSED pgs=1895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:58.817 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.804+0000 7f5ad9c67740 1 --2- 172.21.15.16:0/3361265827 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556da18652e0 0x556da186f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:58.817 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.804+0000 7f5ad9c67740 1 --2- 172.21.15.16:0/3361265827 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556da18649a0 0x556da186eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:58.817 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.804+0000 7f5aaffff640 0 client.20118 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:15:58.817 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.804+0000 7f5ad9c67740 1 -- 172.21.15.16:0/3361265827 >> 172.21.15.16:0/3361265827 conn(0x556da1860010 msgr2=0x556da1861820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:15:58.818 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.804+0000 7f5ad9c67740 1 -- 172.21.15.16:0/3361265827 shutdown_connections 2024-09-12T02:15:58.818 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.804+0000 7f5ad9c67740 1 -- 172.21.15.16:0/3361265827 wait complete. 2024-09-12T02:15:58.818 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.804+0000 7f5ad9c67740 20 client.20118 trim_cache size 0 max 16384 2024-09-12T02:15:58.820 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.819+0000 7f5ad9c67740 1 Processor -- start 2024-09-12T02:15:58.820 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.819+0000 7f5ad9c67740 1 -- start start 2024-09-12T02:15:58.820 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.819+0000 7f5ad9c67740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556da18649a0 0x556da18603f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:58.820 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.819+0000 7f5ad9c67740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556da18652e0 0x556da1876070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:58.820 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.819+0000 7f5ad9c67740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556da186a370 0x556da1860930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:58.820 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.819+0000 7f5ad9c67740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5ad01a9440 con 0x556da18649a0 2024-09-12T02:15:58.820 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.819+0000 7f5ad9c67740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5ad01a6d10 con 0x556da186a370 2024-09-12T02:15:58.820 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.819+0000 7f5ad9c67740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5ad01a4b80 con 0x556da18652e0 2024-09-12T02:15:58.820 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.819+0000 7f5ad4b94640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556da18649a0 0x556da18603f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:58.821 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.819+0000 7f5ad4b94640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556da18649a0 0x556da18603f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37338/0 (socket says 172.21.15.16:37338) 2024-09-12T02:15:58.821 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.819+0000 7f5ad4b94640 1 -- 172.21.15.16:0/1417034027 learned_addr learned my addr 172.21.15.16:0/1417034027 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:15:58.821 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.819+0000 7f5acffff640 1 --2- 172.21.15.16:0/1417034027 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556da18652e0 0x556da1876070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:58.821 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.819+0000 7f5ad5395640 1 --2- 172.21.15.16:0/1417034027 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556da186a370 0x556da1860930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:58.821 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.819+0000 7f5ad4b94640 1 -- 172.21.15.16:0/1417034027 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556da18652e0 msgr2=0x556da1876070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:15:58.821 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.819+0000 7f5ad4b94640 1 --2- 172.21.15.16:0/1417034027 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556da18652e0 0x556da1876070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:58.821 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.819+0000 7f5ad4b94640 1 -- 172.21.15.16:0/1417034027 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556da186a370 msgr2=0x556da1860930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:15:58.821 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.819+0000 7f5ad4b94640 1 --2- 172.21.15.16:0/1417034027 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556da186a370 0x556da1860930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:58.821 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.819+0000 7f5ad4b94640 1 -- 172.21.15.16:0/1417034027 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556da185fe30 con 0x556da18649a0 2024-09-12T02:15:58.821 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.819+0000 7f5ad5395640 1 --2- 172.21.15.16:0/1417034027 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556da186a370 0x556da1860930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:15:58.821 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.820+0000 7f5ad4b94640 1 --2- 172.21.15.16:0/1417034027 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556da18649a0 0x556da18603f0 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7f5ad0101a90 tx=0x7f5ac4005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e51d22fd23575148 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:15:58.821 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.820+0000 7f5aba7fc640 1 -- 172.21.15.16:0/1417034027 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ac404b070 con 0x556da18649a0 2024-09-12T02:15:58.821 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.820+0000 7f5aba7fc640 1 -- 172.21.15.16:0/1417034027 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ac4005520 con 0x556da18649a0 2024-09-12T02:15:58.822 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.820+0000 7f5aba7fc640 1 -- 172.21.15.16:0/1417034027 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ac4035040 con 0x556da18649a0 2024-09-12T02:15:58.825 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.820+0000 7f5ad9c67740 1 -- 172.21.15.16:0/1417034027 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556da18649a0 msgr2=0x556da18603f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:58.825 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.820+0000 7f5ad9c67740 1 --2- 172.21.15.16:0/1417034027 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556da18649a0 0x556da18603f0 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7f5ad0101a90 tx=0x7f5ac4005fd0 comp rx=0 tx=0).stop 2024-09-12T02:15:58.825 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.821+0000 7f5ad9c67740 1 -- 172.21.15.16:0/1417034027 shutdown_connections 2024-09-12T02:15:58.825 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.821+0000 7f5ad9c67740 1 --2- 172.21.15.16:0/1417034027 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556da186a370 0x556da1860930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:58.825 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.821+0000 7f5ad9c67740 1 --2- 172.21.15.16:0/1417034027 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556da18652e0 0x556da1876070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:58.825 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.821+0000 7f5ad9c67740 1 --2- 172.21.15.16:0/1417034027 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556da18649a0 0x556da18603f0 unknown :-1 s=CLOSED pgs=1896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:58.825 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.821+0000 7f5ad9c67740 1 -- 172.21.15.16:0/1417034027 >> 172.21.15.16:0/1417034027 conn(0x556da1860010 msgr2=0x556da1869b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:15:58.825 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.821+0000 7f5ad9c67740 1 -- 172.21.15.16:0/1417034027 shutdown_connections 2024-09-12T02:15:58.826 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.821+0000 7f5ad9c67740 1 -- 172.21.15.16:0/1417034027 wait complete. 2024-09-12T02:15:58.826 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.821+0000 7f5ad9c67740 1 -- start start 2024-09-12T02:15:58.826 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.821+0000 7f5ad9c67740 1 Processor -- start 2024-09-12T02:15:58.826 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.821+0000 7f5ad9c67740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556da18649a0 0x556da1873150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:58.826 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.821+0000 7f5ad9c67740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556da18652e0 0x556da1873690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:58.826 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.821+0000 7f5ad9c67740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556da186a370 0x556da18746a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:58.826 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.821+0000 7f5ad9c67740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5ad01a51d0 con 0x556da18652e0 2024-09-12T02:15:58.826 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.821+0000 7f5ad9c67740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5ad01a6640 con 0x556da186a370 2024-09-12T02:15:58.826 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.821+0000 7f5ad9c67740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5ad01a58a0 con 0x556da18649a0 2024-09-12T02:15:58.826 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.821+0000 7f5ad9c67740 10 client.0 _ll_register_callbacks cb 0x7f5abc0056b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:15:58.826 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.822+0000 7f5acffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556da18652e0 0x556da1873690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:58.826 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.822+0000 7f5acffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556da18652e0 0x556da1873690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37346/0 (socket says 172.21.15.16:37346) 2024-09-12T02:15:58.826 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.822+0000 7f5acffff640 1 -- 172.21.15.16:0/552791350 learned_addr learned my addr 172.21.15.16:0/552791350 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:15:58.827 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.822+0000 7f5ad5395640 1 --2- 172.21.15.16:0/552791350 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556da186a370 0x556da18746a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:58.827 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.822+0000 7f5ad4b94640 1 --2- 172.21.15.16:0/552791350 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556da18649a0 0x556da1873150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:58.827 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.822+0000 7f5acffff640 1 -- 172.21.15.16:0/552791350 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556da18649a0 msgr2=0x556da1873150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:15:58.827 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.822+0000 7f5acffff640 1 --2- 172.21.15.16:0/552791350 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556da18649a0 0x556da1873150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:58.827 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.822+0000 7f5acffff640 1 -- 172.21.15.16:0/552791350 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556da186a370 msgr2=0x556da18746a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:15:58.827 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.822+0000 7f5acffff640 1 --2- 172.21.15.16:0/552791350 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556da186a370 0x556da18746a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:58.827 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.822+0000 7f5acffff640 1 -- 172.21.15.16:0/552791350 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556da1874cb0 con 0x556da18652e0 2024-09-12T02:15:58.827 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.822+0000 7f5ad5395640 1 --2- 172.21.15.16:0/552791350 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556da186a370 0x556da18746a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:15:58.827 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.822+0000 7f5acffff640 1 --2- 172.21.15.16:0/552791350 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556da18652e0 0x556da1873690 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7f5abc009610 tx=0x7f5abc0048a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6d1698ee18f26fe server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:15:58.827 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.822+0000 7f5aaffff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:15:58.827 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.822+0000 7f5aaffff640 1 -- 172.21.15.16:0/552791350 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5abc020040 con 0x556da18652e0 2024-09-12T02:15:58.827 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.822+0000 7f5aaffff640 1 -- 172.21.15.16:0/552791350 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f5abc0995b0 con 0x556da18652e0 2024-09-12T02:15:58.827 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.822+0000 7f5ad9c67740 1 -- 172.21.15.16:0/552791350 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ac4005520 con 0x556da18652e0 2024-09-12T02:15:58.828 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.822+0000 7f5aaffff640 1 -- 172.21.15.16:0/552791350 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5abc009cc0 con 0x556da18652e0 2024-09-12T02:15:58.828 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.822+0000 7f5ad9c67740 10 client.20124 fetch_fsmap learned FSMap version 25 2024-09-12T02:15:58.828 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.822+0000 7f5ad9c67740 1 -- 172.21.15.16:0/552791350 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x556da1876020 con 0x556da18652e0 2024-09-12T02:15:58.828 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.823+0000 7f5aaffff640 1 -- 172.21.15.16:0/552791350 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ac4005520 con 0x556da18652e0 2024-09-12T02:15:58.828 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.823+0000 7f5aaffff640 1 -- 172.21.15.16:0/552791350 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f5abc0b3020 con 0x556da18652e0 2024-09-12T02:15:58.828 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.823+0000 7f5ad9c67740 10 client.20124 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:15:58.828 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.823+0000 7f5ad9c67740 10 client.20124 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:15:58.828 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.823+0000 7f5ad9c67740 20 client.20124 populate_metadata read hostname 'smithi016' 2024-09-12T02:15:58.828 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.823+0000 7f5ad9c67740 1 --2- 172.21.15.16:0/552791350 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x556da18e57a0 0x556da18e7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:58.828 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.823+0000 7f5ad9c67740 4 client.20124 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:15:58.828 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.823+0000 7f5ad9c67740 1 -- 172.21.15.16:0/552791350 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556da17cfd70 con 0x556da18e57a0 2024-09-12T02:15:58.828 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.823+0000 7f5ad4b94640 1 --2- 172.21.15.16:0/552791350 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x556da18e57a0 0x556da18e7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:58.829 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.824+0000 7f5ad4b94640 1 --2- 172.21.15.16:0/552791350 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x556da18e57a0 0x556da18e7bc0 crc :-1 s=READY pgs=420 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=18cd1f952274fccf server_cookie=7295f3e6db29738e in_seq=0 out_seq=0 2024-09-12T02:15:58.829 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.824+0000 7f5aaffff640 10 client.20124 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:15:58.834 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.833+0000 7f5aaffff640 1 -- 172.21.15.16:0/552791350 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556da17cfd70 con 0x556da18e57a0 2024-09-12T02:15:58.834 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.833+0000 7f5aaffff640 10 client.20124 handle_command_reply: tid=0 2024-09-12T02:15:58.834 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.833+0000 7f5ad9c67740 1 client.20124 shutdown 2024-09-12T02:15:58.834 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.834+0000 7f5ad9c67740 1 -- 172.21.15.16:0/552791350 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556da18652e0 msgr2=0x556da1873690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:58.834 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.834+0000 7f5ad9c67740 1 --2- 172.21.15.16:0/552791350 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556da18652e0 0x556da1873690 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7f5abc009610 tx=0x7f5abc0048a0 comp rx=0 tx=0).stop 2024-09-12T02:15:58.834 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.834+0000 7f5ad9c67740 1 -- 172.21.15.16:0/552791350 shutdown_connections 2024-09-12T02:15:58.834 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.834+0000 7f5ad9c67740 1 --2- 172.21.15.16:0/552791350 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x556da18e57a0 0x556da18e7bc0 crc :-1 s=READY pgs=420 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:58.835 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.834+0000 7f5ad9c67740 1 --2- 172.21.15.16:0/552791350 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556da186a370 0x556da18746a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:58.835 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.834+0000 7f5ad9c67740 1 --2- 172.21.15.16:0/552791350 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556da18652e0 0x556da1873690 unknown :-1 s=CLOSED pgs=1897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:58.835 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.834+0000 7f5aaffff640 0 client.20124 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:15:58.835 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.834+0000 7f5ad9c67740 1 --2- 172.21.15.16:0/552791350 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556da18649a0 0x556da1873150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:58.835 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.834+0000 7f5ad9c67740 1 -- 172.21.15.16:0/552791350 >> 172.21.15.16:0/552791350 conn(0x556da1860010 msgr2=0x556da185e9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:15:58.835 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.834+0000 7f5ad9c67740 1 -- 172.21.15.16:0/552791350 shutdown_connections 2024-09-12T02:15:58.835 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.834+0000 7f5ad9c67740 1 -- 172.21.15.16:0/552791350 wait complete. 2024-09-12T02:15:58.835 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.834+0000 7f5ad9c67740 20 client.20124 trim_cache size 0 max 16384 2024-09-12T02:15:58.835 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-09-12T02:15:58.835 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-09-12T02:15:58.836 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-09-12T02:15:58.836 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-09-12T02:15:58.836 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.835+0000 7f5ad6df8640 1 -- 172.21.15.16:0/927034657 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f5ac00a2cb0 msgr2=0x7f5ac00a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:58.837 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.835+0000 7f5ad6df8640 1 --2- 172.21.15.16:0/927034657 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f5ac00a2cb0 0x7f5ac00a5170 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f5ac8010c40 tx=0x7f5ac800c000 comp rx=0 tx=0).stop 2024-09-12T02:15:58.837 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.835+0000 7f5ad6df8640 1 -- 172.21.15.16:0/927034657 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ad0101c30 msgr2=0x7f5ad0198360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:58.837 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.835+0000 7f5ad6df8640 1 --2- 172.21.15.16:0/927034657 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ad0101c30 0x7f5ad0198360 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x7f5abc00d770 tx=0x7f5abc00dc30 comp rx=0 tx=0).stop 2024-09-12T02:15:58.837 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.835+0000 7f5ad6df8640 1 -- 172.21.15.16:0/927034657 shutdown_connections 2024-09-12T02:15:58.837 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.835+0000 7f5ad6df8640 1 --2- 172.21.15.16:0/927034657 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f5ac00a2cb0 0x7f5ac00a5170 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:58.837 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.835+0000 7f5ad6df8640 1 --2- 172.21.15.16:0/927034657 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ad01025f0 0x7f5ad019f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:58.837 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.835+0000 7f5ad6df8640 1 --2- 172.21.15.16:0/927034657 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ad0101c30 0x7f5ad0198360 unknown :-1 s=CLOSED pgs=1894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:58.837 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.835+0000 7f5ad6df8640 1 --2- 172.21.15.16:0/927034657 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5ad0100a40 0x7f5ad0197e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:58.837 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.835+0000 7f5ad6df8640 1 -- 172.21.15.16:0/927034657 >> 172.21.15.16:0/927034657 conn(0x7f5ad00fc1e0 msgr2=0x7f5ad00fdcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:15:58.837 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.835+0000 7f5ad6df8640 1 -- 172.21.15.16:0/927034657 shutdown_connections 2024-09-12T02:15:58.837 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:15:58.835+0000 7f5ad6df8640 1 -- 172.21.15.16:0/927034657 wait complete. 2024-09-12T02:16:00.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:16:00 smithi184 ceph-mon[35593]: pgmap v757: 97 pgs: 97 active+clean; 3.1 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 938 B/s wr, 8 op/s 2024-09-12T02:16:00.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:16:00 smithi033 ceph-mon[35332]: pgmap v757: 97 pgs: 97 active+clean; 3.1 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 938 B/s wr, 8 op/s 2024-09-12T02:16:00.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:16:00 smithi016 ceph-mon[30947]: pgmap v757: 97 pgs: 97 active+clean; 3.1 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 938 B/s wr, 8 op/s 2024-09-12T02:16:01.368 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:16:01.644 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:16:02.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:02.081+0000 7ffa3a4bd640 1 -- 172.21.15.16:0/454424160 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa34103740 msgr2=0x7ffa34107b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:02.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:02.081+0000 7ffa3a4bd640 1 --2- 172.21.15.16:0/454424160 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa34103740 0x7ffa34107b80 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7ffa28009930 tx=0x7ffa2802a420 comp rx=0 tx=0).stop 2024-09-12T02:16:02.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:02.081+0000 7ffa3a4bd640 1 -- 172.21.15.16:0/454424160 shutdown_connections 2024-09-12T02:16:02.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:02.081+0000 7ffa3a4bd640 1 --2- 172.21.15.16:0/454424160 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa34108290 0x7ffa3410ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:02.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:02.081+0000 7ffa3a4bd640 1 --2- 172.21.15.16:0/454424160 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa34103740 0x7ffa34107b80 unknown :-1 s=CLOSED pgs=1898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:02.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:02.081+0000 7ffa3a4bd640 1 --2- 172.21.15.16:0/454424160 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffa34102d90 0x7ffa34103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:02.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:02.081+0000 7ffa3a4bd640 1 -- 172.21.15.16:0/454424160 >> 172.21.15.16:0/454424160 conn(0x7ffa340fc7d0 msgr2=0x7ffa340febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:02.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:02.081+0000 7ffa3a4bd640 1 -- 172.21.15.16:0/454424160 shutdown_connections 2024-09-12T02:16:02.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:02.081+0000 7ffa3a4bd640 1 -- 172.21.15.16:0/454424160 wait complete. 2024-09-12T02:16:02.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:02.081+0000 7ffa3a4bd640 1 Processor -- start 2024-09-12T02:16:02.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:02.081+0000 7ffa3a4bd640 1 -- start start 2024-09-12T02:16:02.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:02.082+0000 7ffa3a4bd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa34102d90 0x7ffa341009f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:02.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:02.082+0000 7ffa3a4bd640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffa34103740 0x7ffa34100f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:02.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:02.082+0000 7ffa3a4bd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa34108290 0x7ffa34101470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:02.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:02.082+0000 7ffa3a4bd640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ffa34112090 con 0x7ffa34102d90 2024-09-12T02:16:02.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:02.082+0000 7ffa3a4bd640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ffa34111f10 con 0x7ffa34108290 2024-09-12T02:16:02.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:02.082+0000 7ffa3a4bd640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ffa34112210 con 0x7ffa34103740 2024-09-12T02:16:02.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:02.082+0000 7ffa33fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa34102d90 0x7ffa341009f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:02.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:02.082+0000 7ffa33fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa34102d90 0x7ffa341009f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57844/0 (socket says 172.21.15.16:57844) 2024-09-12T02:16:02.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:02.082+0000 7ffa33fff640 1 -- 172.21.15.16:0/2259111402 learned_addr learned my addr 172.21.15.16:0/2259111402 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:16:02.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:02.082+0000 7ffa337fe640 1 --2- 172.21.15.16:0/2259111402 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffa34103740 0x7ffa34100f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:02.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:02.082+0000 7ffa38a5a640 1 --2- 172.21.15.16:0/2259111402 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa34108290 0x7ffa34101470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:02.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:02.082+0000 7ffa33fff640 1 -- 172.21.15.16:0/2259111402 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffa34103740 msgr2=0x7ffa34100f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:02.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:02.082+0000 7ffa33fff640 1 --2- 172.21.15.16:0/2259111402 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffa34103740 0x7ffa34100f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:02.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:02.082+0000 7ffa33fff640 1 -- 172.21.15.16:0/2259111402 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa34108290 msgr2=0x7ffa34101470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:02.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:02.082+0000 7ffa33fff640 1 --2- 172.21.15.16:0/2259111402 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa34108290 0x7ffa34101470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:02.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:02.082+0000 7ffa33fff640 1 -- 172.21.15.16:0/2259111402 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa341a6a10 con 0x7ffa34102d90 2024-09-12T02:16:02.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:02.082+0000 7ffa38a5a640 1 --2- 172.21.15.16:0/2259111402 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa34108290 0x7ffa34101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:16:02.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:02.083+0000 7ffa33fff640 1 --2- 172.21.15.16:0/2259111402 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa34102d90 0x7ffa341009f0 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 crypto rx=0x7ffa2400d950 tx=0x7ffa2400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:02.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:02.083+0000 7ffa317fa640 1 -- 172.21.15.16:0/2259111402 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa24017070 con 0x7ffa34102d90 2024-09-12T02:16:02.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:02.083+0000 7ffa3a4bd640 1 -- 172.21.15.16:0/2259111402 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa341a6d00 con 0x7ffa34102d90 2024-09-12T02:16:02.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:02.083+0000 7ffa317fa640 1 -- 172.21.15.16:0/2259111402 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa24009c30 con 0x7ffa34102d90 2024-09-12T02:16:02.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:02.083+0000 7ffa317fa640 1 -- 172.21.15.16:0/2259111402 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa2400f6b0 con 0x7ffa34102d90 2024-09-12T02:16:02.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:02.083+0000 7ffa3a4bd640 1 -- 172.21.15.16:0/2259111402 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa341a71c0 con 0x7ffa34102d90 2024-09-12T02:16:02.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:02.084+0000 7ffa3a4bd640 1 -- 172.21.15.16:0/2259111402 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa34103170 con 0x7ffa34102d90 2024-09-12T02:16:02.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:02.085+0000 7ffa317fa640 1 -- 172.21.15.16:0/2259111402 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ffa24010460 con 0x7ffa34102d90 2024-09-12T02:16:02.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:02.086+0000 7ffa317fa640 1 --2- 172.21.15.16:0/2259111402 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7ffa1c0a2b80 0x7ffa1c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:02.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:02.086+0000 7ffa337fe640 1 --2- 172.21.15.16:0/2259111402 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7ffa1c0a2b80 0x7ffa1c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:02.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:02.086+0000 7ffa317fa640 1 -- 172.21.15.16:0/2259111402 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7ffa240cec10 con 0x7ffa34102d90 2024-09-12T02:16:02.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:02.087+0000 7ffa337fe640 1 --2- 172.21.15.16:0/2259111402 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7ffa1c0a2b80 0x7ffa1c0a5040 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7ffa28004770 tx=0x7ffa280058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:02.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:02.090+0000 7ffa317fa640 1 -- 172.21.15.16:0/2259111402 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ffa2409bc10 con 0x7ffa34102d90 2024-09-12T02:16:02.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:02.304+0000 7ffa3a4bd640 1 -- 172.21.15.16:0/2259111402 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffa3410a2a0 con 0x7ffa34102d90 2024-09-12T02:16:02.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:02.305+0000 7ffa317fa640 1 -- 172.21.15.16:0/2259111402 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11911 (secure 0 0 0) 0x7ffa24009de0 con 0x7ffa34102d90 2024-09-12T02:16:02.305 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T02:16:02.306 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":25,"btime":"2024-09-12T02:01:37:192021+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3279123279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3279123279},{"type":"v1","addr":"172.21.15.16:6835","nonce":3279123279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4119452382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4119452382},{"type":"v1","addr":"172.21.15.16:6837","nonce":4119452382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/3263823165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3263823165},{"type":"v1","addr":"172.21.15.16:6839","nonce":3263823165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/1510290732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":1510290732},{"type":"v1","addr":"172.21.15.16:6841","nonce":1510290732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/279674603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":279674603},{"type":"v1","addr":"172.21.15.33:6833","nonce":279674603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3555728028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3555728028},{"type":"v1","addr":"172.21.15.33:6835","nonce":3555728028}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T01:59:56.117608+0000","modified":"2024-09-12T02:01:36.294760+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24482,"mds_1":24481,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6833/1563374106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1563374106},{"type":"v1","addr":"172.21.15.184:6833","nonce":1563374106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6837/1892833868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1892833868},{"type":"v1","addr":"172.21.15.33:6837","nonce":1892833868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/1457679686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1457679686},{"type":"v1","addr":"172.21.15.184:6835","nonce":1457679686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.184:6837/1793393087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":1793393087},{"type":"v1","addr":"172.21.15.184:6837","nonce":1793393087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.184:6839/4203347941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":4203347941},{"type":"v1","addr":"172.21.15.184:6839","nonce":4203347941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.33:6839/4136670204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":4136670204},{"type":"v1","addr":"172.21.15.33:6839","nonce":4136670204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24481,24482,24469]},"id":1}]} 2024-09-12T02:16:02.306 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 25 2024-09-12T02:16:02.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:02.307+0000 7ffa3a4bd640 1 -- 172.21.15.16:0/2259111402 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7ffa1c0a2b80 msgr2=0x7ffa1c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:02.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:02.307+0000 7ffa3a4bd640 1 --2- 172.21.15.16:0/2259111402 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7ffa1c0a2b80 0x7ffa1c0a5040 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7ffa28004770 tx=0x7ffa280058d0 comp rx=0 tx=0).stop 2024-09-12T02:16:02.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:02.307+0000 7ffa3a4bd640 1 -- 172.21.15.16:0/2259111402 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa34102d90 msgr2=0x7ffa341009f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:02.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:02.307+0000 7ffa3a4bd640 1 --2- 172.21.15.16:0/2259111402 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa34102d90 0x7ffa341009f0 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 crypto rx=0x7ffa2400d950 tx=0x7ffa2400de10 comp rx=0 tx=0).stop 2024-09-12T02:16:02.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:02.307+0000 7ffa3a4bd640 1 -- 172.21.15.16:0/2259111402 shutdown_connections 2024-09-12T02:16:02.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:02.307+0000 7ffa3a4bd640 1 --2- 172.21.15.16:0/2259111402 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7ffa1c0a2b80 0x7ffa1c0a5040 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:02.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:02.307+0000 7ffa3a4bd640 1 --2- 172.21.15.16:0/2259111402 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa34108290 0x7ffa34101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:02.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:02.307+0000 7ffa3a4bd640 1 --2- 172.21.15.16:0/2259111402 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffa34103740 0x7ffa34100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:02.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:02.307+0000 7ffa3a4bd640 1 --2- 172.21.15.16:0/2259111402 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa34102d90 0x7ffa341009f0 unknown :-1 s=CLOSED pgs=1899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:02.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:02.307+0000 7ffa3a4bd640 1 -- 172.21.15.16:0/2259111402 >> 172.21.15.16:0/2259111402 conn(0x7ffa340fc7d0 msgr2=0x7ffa34105d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:02.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:02.308+0000 7ffa3a4bd640 1 -- 172.21.15.16:0/2259111402 shutdown_connections 2024-09-12T02:16:02.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:02.308+0000 7ffa3a4bd640 1 -- 172.21.15.16:0/2259111402 wait complete. 2024-09-12T02:16:02.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:16:02 smithi184 ceph-mon[35593]: pgmap v758: 97 pgs: 97 active+clean; 3.1 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-09-12T02:16:02.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:16:02 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/2259111402' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:16:02.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:16:02 smithi033 ceph-mon[35332]: pgmap v758: 97 pgs: 97 active+clean; 3.1 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-09-12T02:16:02.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:16:02 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/2259111402' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:16:02.685 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:16:02 smithi016 ceph-mon[30947]: pgmap v758: 97 pgs: 97 active+clean; 3.1 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-09-12T02:16:02.685 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:16:02 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/2259111402' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:16:03.076 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24482 quiesce db --set-id 99f960cd --release --await' 2024-09-12T02:16:03.076 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24482 quiesce db --set-id 99f960cd --release --await 2024-09-12T02:16:03.341 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:16:03.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.808+0000 7f7a09dd1640 1 -- 172.21.15.16:0/863096942 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a041082a0 msgr2=0x7f7a0410ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:03.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.808+0000 7f7a09dd1640 1 --2- 172.21.15.16:0/863096942 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a041082a0 0x7f7a0410ec40 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 crypto rx=0x7f79fc009cd0 tx=0x7f79fc02a8e0 comp rx=0 tx=0).stop 2024-09-12T02:16:03.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.808+0000 7f7a09dd1640 1 -- 172.21.15.16:0/863096942 shutdown_connections 2024-09-12T02:16:03.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.808+0000 7f7a09dd1640 1 --2- 172.21.15.16:0/863096942 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a041082a0 0x7f7a0410ec40 unknown :-1 s=CLOSED pgs=1900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:03.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.808+0000 7f7a09dd1640 1 --2- 172.21.15.16:0/863096942 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a041036e0 0x7f7a04107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:03.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.808+0000 7f7a09dd1640 1 --2- 172.21.15.16:0/863096942 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7a04102d30 0x7f7a04103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:03.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.808+0000 7f7a09dd1640 1 -- 172.21.15.16:0/863096942 >> 172.21.15.16:0/863096942 conn(0x7f7a040fc7d0 msgr2=0x7f7a040febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:03.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.808+0000 7f7a09dd1640 1 -- 172.21.15.16:0/863096942 shutdown_connections 2024-09-12T02:16:03.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.808+0000 7f7a09dd1640 1 -- 172.21.15.16:0/863096942 wait complete. 2024-09-12T02:16:03.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.809+0000 7f7a09dd1640 1 Processor -- start 2024-09-12T02:16:03.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.809+0000 7f7a09dd1640 1 -- start start 2024-09-12T02:16:03.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.809+0000 7f7a09dd1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a04102d30 0x7f7a0419a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:03.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.809+0000 7f7a09dd1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a041036e0 0x7f7a0419ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:03.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.809+0000 7f7a09dd1640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7a041082a0 0x7f7a0419efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:03.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.809+0000 7f7a09dd1640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7a041120a0 con 0x7f7a04102d30 2024-09-12T02:16:03.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.809+0000 7f7a09dd1640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7a04111f20 con 0x7f7a041036e0 2024-09-12T02:16:03.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.809+0000 7f7a09dd1640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7a04112220 con 0x7f7a041082a0 2024-09-12T02:16:03.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.809+0000 7f7a037fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a04102d30 0x7f7a0419a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:03.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.809+0000 7f7a037fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a04102d30 0x7f7a0419a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57858/0 (socket says 172.21.15.16:57858) 2024-09-12T02:16:03.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.809+0000 7f7a037fe640 1 -- 172.21.15.16:0/884389545 learned_addr learned my addr 172.21.15.16:0/884389545 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:16:03.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.809+0000 7f7a03fff640 1 --2- 172.21.15.16:0/884389545 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7a041082a0 0x7f7a0419efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:03.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.809+0000 7f7a02ffd640 1 --2- 172.21.15.16:0/884389545 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a041036e0 0x7f7a0419ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:03.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.809+0000 7f7a037fe640 1 -- 172.21.15.16:0/884389545 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7a041082a0 msgr2=0x7f7a0419efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:03.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.809+0000 7f7a037fe640 1 --2- 172.21.15.16:0/884389545 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7a041082a0 0x7f7a0419efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:03.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.809+0000 7f7a037fe640 1 -- 172.21.15.16:0/884389545 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a041036e0 msgr2=0x7f7a0419ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:03.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.809+0000 7f7a037fe640 1 --2- 172.21.15.16:0/884389545 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a041036e0 0x7f7a0419ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:03.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.809+0000 7f7a037fe640 1 -- 172.21.15.16:0/884389545 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a0419f6c0 con 0x7f7a04102d30 2024-09-12T02:16:03.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.810+0000 7f7a037fe640 1 --2- 172.21.15.16:0/884389545 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a04102d30 0x7f7a0419a710 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7f79f400d770 tx=0x7f79f400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:03.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.810+0000 7f7a00ff9640 1 -- 172.21.15.16:0/884389545 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79f4017070 con 0x7f7a04102d30 2024-09-12T02:16:03.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.810+0000 7f7a00ff9640 1 -- 172.21.15.16:0/884389545 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79f4005500 con 0x7f7a04102d30 2024-09-12T02:16:03.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.810+0000 7f7a00ff9640 1 -- 172.21.15.16:0/884389545 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79f4004e00 con 0x7f7a04102d30 2024-09-12T02:16:03.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.810+0000 7f7a09dd1640 1 -- 172.21.15.16:0/884389545 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a0419f9b0 con 0x7f7a04102d30 2024-09-12T02:16:03.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.810+0000 7f7a09dd1640 1 -- 172.21.15.16:0/884389545 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a041a7290 con 0x7f7a04102d30 2024-09-12T02:16:03.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.812+0000 7f7a00ff9640 1 -- 172.21.15.16:0/884389545 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f79f4004fa0 con 0x7f7a04102d30 2024-09-12T02:16:03.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.812+0000 7f7a0c1aa740 1 Processor -- start 2024-09-12T02:16:03.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.812+0000 7f7a0c1aa740 1 -- start start 2024-09-12T02:16:03.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.813+0000 7f7a0c1aa740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562f9d0bb060 0x562f9d0bb440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:03.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.813+0000 7f7a0c1aa740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f9d0bb980 0x562f9d0c5d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:03.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.813+0000 7f7a0c1aa740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562f9d0c6240 0x562f9d0c8630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:03.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.813+0000 7f7a0c1aa740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x562f9d020620 con 0x562f9d0bb980 2024-09-12T02:16:03.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.813+0000 7f7a0c1aa740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x562f9cfc7e50 con 0x562f9d0c6240 2024-09-12T02:16:03.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.813+0000 7f7a0c1aa740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x562f9d01c5e0 con 0x562f9d0bb060 2024-09-12T02:16:03.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.813+0000 7f7a00ff9640 1 --2- 172.21.15.16:0/884389545 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f79ec0a2b80 0x7f79ec0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:03.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.813+0000 7f7a037fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f9d0bb980 0x562f9d0c5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:03.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.813+0000 7f7a037fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f9d0bb980 0x562f9d0c5d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57864/0 (socket says 172.21.15.16:57864) 2024-09-12T02:16:03.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.813+0000 7f7a037fe640 1 -- 172.21.15.16:0/2425167555 learned_addr learned my addr 172.21.15.16:0/2425167555 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:16:03.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.813+0000 7f7a02ffd640 1 --2- 172.21.15.16:0/2425167555 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562f9d0c6240 0x562f9d0c8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:03.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.813+0000 7f7a03fff640 1 --2- 172.21.15.16:0/2425167555 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562f9d0bb060 0x562f9d0bb440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:03.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.813+0000 7f7a03fff640 1 --2- 172.21.15.16:0/884389545 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f79ec0a2b80 0x7f79ec0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:03.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.813+0000 7f7a00ff9640 1 -- 172.21.15.16:0/884389545 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f79f40cea20 con 0x7f7a04102d30 2024-09-12T02:16:03.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.813+0000 7f7a037fe640 1 -- 172.21.15.16:0/2425167555 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562f9d0bb060 msgr2=0x562f9d0bb440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:03.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.813+0000 7f7a037fe640 1 --2- 172.21.15.16:0/2425167555 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562f9d0bb060 0x562f9d0bb440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:03.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.813+0000 7f7a037fe640 1 -- 172.21.15.16:0/2425167555 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562f9d0c6240 msgr2=0x562f9d0c8630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:03.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.813+0000 7f7a037fe640 1 --2- 172.21.15.16:0/2425167555 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562f9d0c6240 0x562f9d0c8630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:03.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.813+0000 7f7a037fe640 1 -- 172.21.15.16:0/2425167555 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f9cfb8d40 con 0x562f9d0bb980 2024-09-12T02:16:03.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.814+0000 7f7a03fff640 1 --2- 172.21.15.16:0/884389545 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f79ec0a2b80 0x7f79ec0a5040 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f79fc004ab0 tx=0x7f79fc004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:03.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.814+0000 7f7a037fe640 1 --2- 172.21.15.16:0/2425167555 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f9d0bb980 0x562f9d0c5d00 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7f79f40cc500 tx=0x7f79f40048a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=627dbd4a503742a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:03.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.814+0000 7f79e9ffb640 1 -- 172.21.15.16:0/2425167555 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79f4020070 con 0x562f9d0bb980 2024-09-12T02:16:03.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.814+0000 7f79e9ffb640 1 -- 172.21.15.16:0/2425167555 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79f409ba20 con 0x562f9d0bb980 2024-09-12T02:16:03.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.814+0000 7f79e9ffb640 1 -- 172.21.15.16:0/2425167555 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79f4009cc0 con 0x562f9d0bb980 2024-09-12T02:16:03.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.814+0000 7f7a0c1aa740 1 -- 172.21.15.16:0/2425167555 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f9d0bb980 msgr2=0x562f9d0c5d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:03.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.814+0000 7f7a0c1aa740 1 --2- 172.21.15.16:0/2425167555 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f9d0bb980 0x562f9d0c5d00 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7f79f40cc500 tx=0x7f79f40048a0 comp rx=0 tx=0).stop 2024-09-12T02:16:03.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.815+0000 7f7a0c1aa740 1 -- 172.21.15.16:0/2425167555 shutdown_connections 2024-09-12T02:16:03.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.815+0000 7f7a0c1aa740 1 --2- 172.21.15.16:0/2425167555 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562f9d0c6240 0x562f9d0c8630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:03.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.815+0000 7f7a0c1aa740 1 --2- 172.21.15.16:0/2425167555 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f9d0bb980 0x562f9d0c5d00 unknown :-1 s=CLOSED pgs=1902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:03.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.815+0000 7f7a0c1aa740 1 --2- 172.21.15.16:0/2425167555 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562f9d0bb060 0x562f9d0bb440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:03.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.815+0000 7f7a0c1aa740 1 -- 172.21.15.16:0/2425167555 >> 172.21.15.16:0/2425167555 conn(0x562f9d0b75f0 msgr2=0x562f9d0b79d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:03.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.815+0000 7f7a0c1aa740 1 -- 172.21.15.16:0/2425167555 shutdown_connections 2024-09-12T02:16:03.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.815+0000 7f7a0c1aa740 1 -- 172.21.15.16:0/2425167555 wait complete. 2024-09-12T02:16:03.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.815+0000 7f7a0c1aa740 1 -- start start 2024-09-12T02:16:03.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.815+0000 7f7a0c1aa740 1 Processor -- start 2024-09-12T02:16:03.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.815+0000 7f7a0c1aa740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f9d0bb060 0x562f9d0c3150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:03.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.815+0000 7f7a0c1aa740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562f9d0bb980 0x562f9d0c3690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:03.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.815+0000 7f7a0c1aa740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562f9d0c6240 0x562f9d0c3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:03.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.815+0000 7f7a0c1aa740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x562f9d0c8db0 con 0x562f9d0bb060 2024-09-12T02:16:03.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.815+0000 7f7a0c1aa740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x562f9cfa8f10 con 0x562f9d0c6240 2024-09-12T02:16:03.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.815+0000 7f7a0c1aa740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x562f9d0c8f30 con 0x562f9d0bb980 2024-09-12T02:16:03.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.815+0000 7f7a0c1aa740 10 client.0 _ll_register_callbacks cb 0x562f9cfccdb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:16:03.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.815+0000 7f7a037fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f9d0bb060 0x562f9d0c3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:03.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.816+0000 7f7a037fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f9d0bb060 0x562f9d0c3150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57872/0 (socket says 172.21.15.16:57872) 2024-09-12T02:16:03.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.816+0000 7f7a037fe640 1 -- 172.21.15.16:0/1563096202 learned_addr learned my addr 172.21.15.16:0/1563096202 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:16:03.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.816+0000 7f7a02ffd640 1 --2- 172.21.15.16:0/1563096202 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562f9d0bb980 0x562f9d0c3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:03.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.816+0000 7f7a03fff640 1 --2- 172.21.15.16:0/1563096202 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562f9d0c6240 0x562f9d0c3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:03.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.816+0000 7f7a037fe640 1 -- 172.21.15.16:0/1563096202 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562f9d0bb980 msgr2=0x562f9d0c3690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:03.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.816+0000 7f7a037fe640 1 --2- 172.21.15.16:0/1563096202 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562f9d0bb980 0x562f9d0c3690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:03.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.816+0000 7f7a037fe640 1 -- 172.21.15.16:0/1563096202 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562f9d0c6240 msgr2=0x562f9d0c3bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:03.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.816+0000 7f7a037fe640 1 --2- 172.21.15.16:0/1563096202 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562f9d0c6240 0x562f9d0c3bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:03.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.816+0000 7f7a037fe640 1 -- 172.21.15.16:0/1563096202 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f9cd2f580 con 0x562f9d0bb060 2024-09-12T02:16:03.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.816+0000 7f7a03fff640 1 --2- 172.21.15.16:0/1563096202 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562f9d0c6240 0x562f9d0c3bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:16:03.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.816+0000 7f7a037fe640 1 --2- 172.21.15.16:0/1563096202 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f9d0bb060 0x562f9d0c3150 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7f79f409b2b0 tx=0x7f79f40047f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8cad062934664e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:03.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.816+0000 7f79e2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:16:03.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.816+0000 7f79e2ffd640 1 -- 172.21.15.16:0/1563096202 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79f4020040 con 0x562f9d0bb060 2024-09-12T02:16:03.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.816+0000 7f79e2ffd640 1 -- 172.21.15.16:0/1563096202 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f79f4009cc0 con 0x562f9d0bb060 2024-09-12T02:16:03.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.816+0000 7f7a0c1aa740 1 -- 172.21.15.16:0/1563096202 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f79f409ba20 con 0x562f9d0bb060 2024-09-12T02:16:03.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.816+0000 7f79e2ffd640 1 -- 172.21.15.16:0/1563096202 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f79f409f6d0 con 0x562f9d0bb060 2024-09-12T02:16:03.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.816+0000 7f7a0c1aa740 10 client.20142 fetch_fsmap learned FSMap version 25 2024-09-12T02:16:03.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.816+0000 7f7a0c1aa740 1 -- 172.21.15.16:0/1563096202 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x562f9cdf9960 con 0x562f9d0bb060 2024-09-12T02:16:03.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.817+0000 7f79e2ffd640 1 -- 172.21.15.16:0/1563096202 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79f409ba20 con 0x562f9d0bb060 2024-09-12T02:16:03.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.817+0000 7f79e2ffd640 1 -- 172.21.15.16:0/1563096202 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f79f409a870 con 0x562f9d0bb060 2024-09-12T02:16:03.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.817+0000 7f7a0c1aa740 10 client.20142 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:16:03.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.817+0000 7f7a0c1aa740 10 client.20142 resolve_mds: validated gid 24482 aka daemon mds.b 2024-09-12T02:16:03.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.817+0000 7f7a0c1aa740 20 client.20142 populate_metadata read hostname 'smithi016' 2024-09-12T02:16:03.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.817+0000 7f7a0c1aa740 1 --2- 172.21.15.16:0/1563096202 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x562f9d0d5c10 0x562f9d0d8000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:03.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.817+0000 7f7a0c1aa740 4 client.20142 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:16:03.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.817+0000 7f7a0c1aa740 1 -- 172.21.15.16:0/1563096202 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562f9d02db90 con 0x562f9d0d5c10 2024-09-12T02:16:03.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.817+0000 7f7a02ffd640 1 --2- 172.21.15.16:0/1563096202 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x562f9d0d5c10 0x562f9d0d8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:03.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.818+0000 7f79e2ffd640 10 client.20142 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:16:03.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.818+0000 7f7a02ffd640 1 --2- 172.21.15.16:0/1563096202 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x562f9d0d5c10 0x562f9d0d8000 crc :-1 s=READY pgs=421 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8080c02f0791e9f1 server_cookie=a37c06f6b96cdba3 in_seq=0 out_seq=0 2024-09-12T02:16:03.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.818+0000 7f79e2ffd640 1 -- 172.21.15.16:0/1563096202 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562f9d02db90 con 0x562f9d0d5c10 2024-09-12T02:16:03.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.818+0000 7f79e2ffd640 10 client.20142 handle_command_reply: tid=0 2024-09-12T02:16:03.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.818+0000 7f7a0c1aa740 1 client.20142 shutdown 2024-09-12T02:16:03.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.819+0000 7f7a0c1aa740 1 -- 172.21.15.16:0/1563096202 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f9d0bb060 msgr2=0x562f9d0c3150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:03.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.819+0000 7f7a0c1aa740 1 --2- 172.21.15.16:0/1563096202 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f9d0bb060 0x562f9d0c3150 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7f79f409b2b0 tx=0x7f79f40047f0 comp rx=0 tx=0).stop 2024-09-12T02:16:03.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.819+0000 7f7a0c1aa740 1 -- 172.21.15.16:0/1563096202 shutdown_connections 2024-09-12T02:16:03.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.819+0000 7f7a0c1aa740 1 --2- 172.21.15.16:0/1563096202 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x562f9d0d5c10 0x562f9d0d8000 crc :-1 s=READY pgs=421 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:03.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.819+0000 7f7a0c1aa740 1 --2- 172.21.15.16:0/1563096202 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562f9d0c6240 0x562f9d0c3bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:03.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.819+0000 7f7a0c1aa740 1 --2- 172.21.15.16:0/1563096202 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562f9d0bb980 0x562f9d0c3690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:03.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.819+0000 7f79e2ffd640 0 client.20142 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:16:03.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.819+0000 7f7a0c1aa740 1 --2- 172.21.15.16:0/1563096202 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f9d0bb060 0x562f9d0c3150 unknown :-1 s=CLOSED pgs=1903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:03.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.819+0000 7f7a0c1aa740 1 -- 172.21.15.16:0/1563096202 >> 172.21.15.16:0/1563096202 conn(0x562f9d0b75f0 msgr2=0x562f9d0bd2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:03.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.819+0000 7f7a0c1aa740 1 -- 172.21.15.16:0/1563096202 shutdown_connections 2024-09-12T02:16:03.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.819+0000 7f7a0c1aa740 1 -- 172.21.15.16:0/1563096202 wait complete. 2024-09-12T02:16:03.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.819+0000 7f7a0c1aa740 20 client.20142 trim_cache size 0 max 16384 2024-09-12T02:16:03.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.836+0000 7f7a0c1aa740 1 Processor -- start 2024-09-12T02:16:03.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.836+0000 7f7a0c1aa740 1 -- start start 2024-09-12T02:16:03.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.836+0000 7f7a0c1aa740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562f9d0bb060 0x562f9d0bdac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:03.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.836+0000 7f7a0c1aa740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562f9d0bb980 0x562f9d0b6640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:03.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.836+0000 7f7a0c1aa740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f9d0c6240 0x562f9d0c03b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:03.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.836+0000 7f7a0c1aa740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7a041b1ed0 con 0x562f9d0c6240 2024-09-12T02:16:03.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.836+0000 7f7a0c1aa740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7a041af7a0 con 0x562f9d0bb060 2024-09-12T02:16:03.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.836+0000 7f7a0c1aa740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7a041ad610 con 0x562f9d0bb980 2024-09-12T02:16:03.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.836+0000 7f7a03fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f9d0c6240 0x562f9d0c03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:03.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.836+0000 7f7a03fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f9d0c6240 0x562f9d0c03b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57878/0 (socket says 172.21.15.16:57878) 2024-09-12T02:16:03.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.836+0000 7f7a03fff640 1 -- 172.21.15.16:0/3866352775 learned_addr learned my addr 172.21.15.16:0/3866352775 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:16:03.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.836+0000 7f7a02ffd640 1 --2- 172.21.15.16:0/3866352775 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562f9d0bb980 0x562f9d0b6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:03.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.836+0000 7f7a037fe640 1 --2- 172.21.15.16:0/3866352775 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562f9d0bb060 0x562f9d0bdac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:03.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.836+0000 7f7a03fff640 1 -- 172.21.15.16:0/3866352775 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562f9d0bb980 msgr2=0x562f9d0b6640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:03.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.836+0000 7f7a03fff640 1 --2- 172.21.15.16:0/3866352775 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562f9d0bb980 0x562f9d0b6640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:03.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.836+0000 7f7a03fff640 1 -- 172.21.15.16:0/3866352775 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562f9d0bb060 msgr2=0x562f9d0bdac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:03.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.836+0000 7f7a03fff640 1 --2- 172.21.15.16:0/3866352775 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562f9d0bb060 0x562f9d0bdac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:03.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.836+0000 7f7a03fff640 1 -- 172.21.15.16:0/3866352775 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f9d0bb7b0 con 0x562f9d0c6240 2024-09-12T02:16:03.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.837+0000 7f7a02ffd640 1 --2- 172.21.15.16:0/3866352775 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562f9d0bb980 0x562f9d0b6640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:16:03.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.837+0000 7f7a03fff640 1 --2- 172.21.15.16:0/3866352775 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f9d0c6240 0x562f9d0c03b0 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7f79fc035000 tx=0x7f79fc005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=871e9b7ab86af70d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:03.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.837+0000 7f79e97fa640 1 -- 172.21.15.16:0/3866352775 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79fc050070 con 0x562f9d0c6240 2024-09-12T02:16:03.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.837+0000 7f79e97fa640 1 -- 172.21.15.16:0/3866352775 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79fc036070 con 0x562f9d0c6240 2024-09-12T02:16:03.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.837+0000 7f79e97fa640 1 -- 172.21.15.16:0/3866352775 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79fc030e20 con 0x562f9d0c6240 2024-09-12T02:16:03.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.837+0000 7f7a0c1aa740 1 -- 172.21.15.16:0/3866352775 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f9d0c6240 msgr2=0x562f9d0c03b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:03.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.837+0000 7f7a0c1aa740 1 --2- 172.21.15.16:0/3866352775 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f9d0c6240 0x562f9d0c03b0 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7f79fc035000 tx=0x7f79fc005f10 comp rx=0 tx=0).stop 2024-09-12T02:16:03.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.838+0000 7f7a0c1aa740 1 -- 172.21.15.16:0/3866352775 shutdown_connections 2024-09-12T02:16:03.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.838+0000 7f7a0c1aa740 1 --2- 172.21.15.16:0/3866352775 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f9d0c6240 0x562f9d0c03b0 unknown :-1 s=CLOSED pgs=1904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:03.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.838+0000 7f7a0c1aa740 1 --2- 172.21.15.16:0/3866352775 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562f9d0bb980 0x562f9d0b6640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:03.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.838+0000 7f7a0c1aa740 1 --2- 172.21.15.16:0/3866352775 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562f9d0bb060 0x562f9d0bdac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:03.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.838+0000 7f7a0c1aa740 1 -- 172.21.15.16:0/3866352775 >> 172.21.15.16:0/3866352775 conn(0x562f9d0b75f0 msgr2=0x562f9d0b5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:03.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.838+0000 7f7a0c1aa740 1 -- 172.21.15.16:0/3866352775 shutdown_connections 2024-09-12T02:16:03.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.838+0000 7f7a0c1aa740 1 -- 172.21.15.16:0/3866352775 wait complete. 2024-09-12T02:16:03.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.838+0000 7f7a0c1aa740 1 -- start start 2024-09-12T02:16:03.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.838+0000 7f7a0c1aa740 1 Processor -- start 2024-09-12T02:16:03.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.838+0000 7f7a0c1aa740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562f9d0bb060 0x562f9d0d9020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:03.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.838+0000 7f7a0c1aa740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562f9d0bb980 0x562f9d0d9560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:03.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.838+0000 7f7a0c1aa740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f9d0c6240 0x562f9d0d9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:03.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.838+0000 7f7a0c1aa740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7a041adc60 con 0x562f9d0c6240 2024-09-12T02:16:03.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.838+0000 7f7a0c1aa740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7a041af0d0 con 0x562f9d0bb980 2024-09-12T02:16:03.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.838+0000 7f7a0c1aa740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7a041ae330 con 0x562f9d0bb060 2024-09-12T02:16:03.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.839+0000 7f7a0c1aa740 10 client.0 _ll_register_callbacks cb 0x7f79f40056b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:16:03.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.839+0000 7f7a03fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f9d0c6240 0x562f9d0d9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:03.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.839+0000 7f7a03fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f9d0c6240 0x562f9d0d9aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57886/0 (socket says 172.21.15.16:57886) 2024-09-12T02:16:03.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.839+0000 7f7a03fff640 1 -- 172.21.15.16:0/2638598624 learned_addr learned my addr 172.21.15.16:0/2638598624 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:16:03.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.839+0000 7f7a02ffd640 1 --2- 172.21.15.16:0/2638598624 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562f9d0bb980 0x562f9d0d9560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:03.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.839+0000 7f7a037fe640 1 --2- 172.21.15.16:0/2638598624 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562f9d0bb060 0x562f9d0d9020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:03.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.839+0000 7f7a03fff640 1 -- 172.21.15.16:0/2638598624 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562f9d0bb060 msgr2=0x562f9d0d9020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:03.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.839+0000 7f7a03fff640 1 --2- 172.21.15.16:0/2638598624 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562f9d0bb060 0x562f9d0d9020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:03.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.839+0000 7f7a03fff640 1 -- 172.21.15.16:0/2638598624 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562f9d0bb980 msgr2=0x562f9d0d9560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:03.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.839+0000 7f7a03fff640 1 --2- 172.21.15.16:0/2638598624 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562f9d0bb980 0x562f9d0d9560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:03.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.839+0000 7f7a03fff640 1 -- 172.21.15.16:0/2638598624 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f9d13f7a0 con 0x562f9d0c6240 2024-09-12T02:16:03.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.839+0000 7f7a03fff640 1 --2- 172.21.15.16:0/2638598624 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f9d0c6240 0x562f9d0d9aa0 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x7f79fc004410 tx=0x7f79fc030f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2adcb4d55455f3d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:03.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.839+0000 7f79e2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:16:03.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.839+0000 7f79e2ffd640 1 -- 172.21.15.16:0/2638598624 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79fc05a070 con 0x562f9d0c6240 2024-09-12T02:16:03.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.839+0000 7f79e2ffd640 1 -- 172.21.15.16:0/2638598624 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f79fc0305a0 con 0x562f9d0c6240 2024-09-12T02:16:03.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.839+0000 7f7a0c1aa740 1 -- 172.21.15.16:0/2638598624 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f79fc036070 con 0x562f9d0c6240 2024-09-12T02:16:03.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.839+0000 7f79e2ffd640 1 -- 172.21.15.16:0/2638598624 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f79fc050050 con 0x562f9d0c6240 2024-09-12T02:16:03.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.840+0000 7f7a0c1aa740 10 client.20148 fetch_fsmap learned FSMap version 25 2024-09-12T02:16:03.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.840+0000 7f7a0c1aa740 1 -- 172.21.15.16:0/2638598624 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x562f9d140ae0 con 0x562f9d0c6240 2024-09-12T02:16:03.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.840+0000 7f79e2ffd640 1 -- 172.21.15.16:0/2638598624 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79fc036070 con 0x562f9d0c6240 2024-09-12T02:16:03.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.840+0000 7f79e2ffd640 1 -- 172.21.15.16:0/2638598624 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f79fc067020 con 0x562f9d0c6240 2024-09-12T02:16:03.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.840+0000 7f7a0c1aa740 10 client.20148 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:16:03.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.840+0000 7f7a0c1aa740 10 client.20148 resolve_mds: validated gid 24482 aka daemon mds.b 2024-09-12T02:16:03.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.840+0000 7f7a0c1aa740 20 client.20148 populate_metadata read hostname 'smithi016' 2024-09-12T02:16:03.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.840+0000 7f7a0c1aa740 1 --2- 172.21.15.16:0/2638598624 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x562f9d1411d0 0x562f9d1435f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:03.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.840+0000 7f7a0c1aa740 4 client.20148 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "99f960cd", "release": true, "await": true}] 2024-09-12T02:16:03.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.840+0000 7f7a0c1aa740 1 -- 172.21.15.16:0/2638598624 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "quiesce db", "set_id": "99f960cd", "release": true, "await": true}) -- 0x562f9d0280a0 con 0x562f9d1411d0 2024-09-12T02:16:03.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.840+0000 7f7a037fe640 1 --2- 172.21.15.16:0/2638598624 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x562f9d1411d0 0x562f9d1435f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:03.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.841+0000 7f7a037fe640 1 --2- 172.21.15.16:0/2638598624 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x562f9d1411d0 0x562f9d1435f0 crc :-1 s=READY pgs=422 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c22eb0c205f5b5c0 server_cookie=516b55a675dc4fbe in_seq=0 out_seq=0 2024-09-12T02:16:03.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.841+0000 7f79e2ffd640 10 client.20148 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:16:03.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.844+0000 7f79e2ffd640 1 -- 172.21.15.16:0/2638598624 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x562f9d0280a0 con 0x562f9d1411d0 2024-09-12T02:16:03.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.844+0000 7f79e2ffd640 10 client.20148 handle_command_reply: tid=0 2024-09-12T02:16:03.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.844+0000 7f7a0c1aa740 1 client.20148 shutdown 2024-09-12T02:16:03.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.844+0000 7f7a0c1aa740 1 -- 172.21.15.16:0/2638598624 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f9d0c6240 msgr2=0x562f9d0d9aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:03.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.844+0000 7f7a0c1aa740 1 --2- 172.21.15.16:0/2638598624 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f9d0c6240 0x562f9d0d9aa0 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x7f79fc004410 tx=0x7f79fc030f80 comp rx=0 tx=0).stop 2024-09-12T02:16:03.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.844+0000 7f7a0c1aa740 1 -- 172.21.15.16:0/2638598624 shutdown_connections 2024-09-12T02:16:03.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.844+0000 7f7a0c1aa740 1 --2- 172.21.15.16:0/2638598624 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x562f9d1411d0 0x562f9d1435f0 crc :-1 s=READY pgs=422 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:03.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.844+0000 7f7a0c1aa740 1 --2- 172.21.15.16:0/2638598624 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f9d0c6240 0x562f9d0d9aa0 unknown :-1 s=CLOSED pgs=1905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:03.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.844+0000 7f7a0c1aa740 1 --2- 172.21.15.16:0/2638598624 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562f9d0bb980 0x562f9d0d9560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:03.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.844+0000 7f7a0c1aa740 1 --2- 172.21.15.16:0/2638598624 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562f9d0bb060 0x562f9d0d9020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:03.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.844+0000 7f7a0c1aa740 1 -- 172.21.15.16:0/2638598624 >> 172.21.15.16:0/2638598624 conn(0x562f9d0b75f0 msgr2=0x562f9d0b6750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:03.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.844+0000 7f79e2ffd640 0 client.20148 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:16:03.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.844+0000 7f7a0c1aa740 1 -- 172.21.15.16:0/2638598624 shutdown_connections 2024-09-12T02:16:03.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.844+0000 7f7a0c1aa740 1 -- 172.21.15.16:0/2638598624 wait complete. 2024-09-12T02:16:03.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.844+0000 7f7a0c1aa740 20 client.20148 trim_cache size 0 max 16384 2024-09-12T02:16:03.848 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T02:16:03.848 INFO:teuthology.orchestra.run.smithi016.stdout: "epoch": 25, 2024-09-12T02:16:03.848 INFO:teuthology.orchestra.run.smithi016.stdout: "leader": 24482, 2024-09-12T02:16:03.848 INFO:teuthology.orchestra.run.smithi016.stdout: "set_version": 56, 2024-09-12T02:16:03.848 INFO:teuthology.orchestra.run.smithi016.stdout: "sets": { 2024-09-12T02:16:03.848 INFO:teuthology.orchestra.run.smithi016.stdout: "99f960cd": { 2024-09-12T02:16:03.849 INFO:teuthology.orchestra.run.smithi016.stdout: "version": 56, 2024-09-12T02:16:03.849 INFO:teuthology.orchestra.run.smithi016.stdout: "age_ref": 0.0, 2024-09-12T02:16:03.849 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-12T02:16:03.849 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RELEASED", 2024-09-12T02:16:03.849 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-12T02:16:03.849 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:16:03.849 INFO:teuthology.orchestra.run.smithi016.stdout: "timeout": 90.0, 2024-09-12T02:16:03.849 INFO:teuthology.orchestra.run.smithi016.stdout: "expiration": 150.0, 2024-09-12T02:16:03.849 INFO:teuthology.orchestra.run.smithi016.stdout: "members": { 2024-09-12T02:16:03.849 INFO:teuthology.orchestra.run.smithi016.stdout: "file:/": { 2024-09-12T02:16:03.849 INFO:teuthology.orchestra.run.smithi016.stdout: "excluded": false, 2024-09-12T02:16:03.849 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-12T02:16:03.849 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RELEASED", 2024-09-12T02:16:03.849 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-12T02:16:03.850 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:16:03.850 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:16:03.850 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:16:03.850 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:16:03.850 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:16:03.850 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T02:16:03.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.845+0000 7f7a09dd1640 1 -- 172.21.15.16:0/884389545 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f79ec0a2b80 msgr2=0x7f79ec0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:03.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.845+0000 7f7a09dd1640 1 --2- 172.21.15.16:0/884389545 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f79ec0a2b80 0x7f79ec0a5040 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f79fc004ab0 tx=0x7f79fc004a40 comp rx=0 tx=0).stop 2024-09-12T02:16:03.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.845+0000 7f7a09dd1640 1 -- 172.21.15.16:0/884389545 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a04102d30 msgr2=0x7f7a0419a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:03.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.845+0000 7f7a09dd1640 1 --2- 172.21.15.16:0/884389545 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a04102d30 0x7f7a0419a710 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7f79f400d770 tx=0x7f79f400dc30 comp rx=0 tx=0).stop 2024-09-12T02:16:03.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.845+0000 7f7a09dd1640 1 -- 172.21.15.16:0/884389545 shutdown_connections 2024-09-12T02:16:03.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.845+0000 7f7a09dd1640 1 --2- 172.21.15.16:0/884389545 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f79ec0a2b80 0x7f79ec0a5040 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:03.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.845+0000 7f7a09dd1640 1 --2- 172.21.15.16:0/884389545 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7a041082a0 0x7f7a0419efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:03.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.845+0000 7f7a09dd1640 1 --2- 172.21.15.16:0/884389545 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a041036e0 0x7f7a0419ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:03.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.845+0000 7f7a09dd1640 1 --2- 172.21.15.16:0/884389545 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a04102d30 0x7f7a0419a710 unknown :-1 s=CLOSED pgs=1901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:03.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.845+0000 7f7a09dd1640 1 -- 172.21.15.16:0/884389545 >> 172.21.15.16:0/884389545 conn(0x7f7a040fc7d0 msgr2=0x7f7a04101170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:03.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.845+0000 7f7a09dd1640 1 -- 172.21.15.16:0/884389545 shutdown_connections 2024-09-12T02:16:03.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:03.845+0000 7f7a09dd1640 1 -- 172.21.15.16:0/884389545 wait complete. 2024-09-12T02:16:04.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:16:04 smithi016 ceph-mon[30947]: pgmap v759: 97 pgs: 97 active+clean; 3.1 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-09-12T02:16:04.613 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '99f960cd', total seconds elapsed: 33.2 2024-09-12T02:16:04.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:16:04 smithi184 ceph-mon[35593]: pgmap v759: 97 pgs: 97 active+clean; 3.1 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-09-12T02:16:04.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:16:04 smithi033 ceph-mon[35332]: pgmap v759: 97 pgs: 97 active+clean; 3.1 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-09-12T02:16:06.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:16:06 smithi184 ceph-mon[35593]: pgmap v760: 97 pgs: 97 active+clean; 3.1 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1023 B/s wr, 7 op/s 2024-09-12T02:16:06.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:16:06 smithi033 ceph-mon[35332]: pgmap v760: 97 pgs: 97 active+clean; 3.1 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1023 B/s wr, 7 op/s 2024-09-12T02:16:06.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:16:06 smithi016 ceph-mon[30947]: pgmap v760: 97 pgs: 97 active+clean; 3.1 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1023 B/s wr, 7 op/s 2024-09-12T02:16:08.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:16:08 smithi184 ceph-mon[35593]: pgmap v761: 97 pgs: 97 active+clean; 3.1 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 4.3 KiB/s wr, 9 op/s 2024-09-12T02:16:08.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:16:08 smithi033 ceph-mon[35332]: pgmap v761: 97 pgs: 97 active+clean; 3.1 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 4.3 KiB/s wr, 9 op/s 2024-09-12T02:16:08.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:16:08 smithi016 ceph-mon[30947]: pgmap v761: 97 pgs: 97 active+clean; 3.1 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 4.3 KiB/s wr, 9 op/s 2024-09-12T02:16:10.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:16:10 smithi184 ceph-mon[35593]: pgmap v762: 97 pgs: 97 active+clean; 3.1 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 4.3 KiB/s wr, 7 op/s 2024-09-12T02:16:10.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:16:10 smithi033 ceph-mon[35332]: pgmap v762: 97 pgs: 97 active+clean; 3.1 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 4.3 KiB/s wr, 7 op/s 2024-09-12T02:16:10.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:16:10 smithi016 ceph-mon[30947]: pgmap v762: 97 pgs: 97 active+clean; 3.1 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 4.3 KiB/s wr, 7 op/s 2024-09-12T02:16:11.938 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.936+0000 7f7d3f40e640 1 -- 172.21.15.16:0/3404045410 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d381037b0 msgr2=0x7f7d3810a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:11.938 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.936+0000 7f7d3f40e640 1 --2- 172.21.15.16:0/3404045410 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d381037b0 0x7f7d3810a040 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7f7d30009ca0 tx=0x7f7d3002a8c0 comp rx=0 tx=0).stop 2024-09-12T02:16:11.938 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.936+0000 7f7d3f40e640 1 -- 172.21.15.16:0/3404045410 shutdown_connections 2024-09-12T02:16:11.938 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.936+0000 7f7d3f40e640 1 --2- 172.21.15.16:0/3404045410 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d381037b0 0x7f7d3810a040 unknown :-1 s=CLOSED pgs=1906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:11.939 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.936+0000 7f7d3f40e640 1 --2- 172.21.15.16:0/3404045410 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7d38102df0 0x7f7d38103270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:11.939 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.936+0000 7f7d3f40e640 1 --2- 172.21.15.16:0/3404045410 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d380ffe20 0x7f7d38100220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:11.939 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.936+0000 7f7d3f40e640 1 -- 172.21.15.16:0/3404045410 >> 172.21.15.16:0/3404045410 conn(0x7f7d380fb980 msgr2=0x7f7d380fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:11.939 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.936+0000 7f7d3f40e640 1 -- 172.21.15.16:0/3404045410 shutdown_connections 2024-09-12T02:16:11.939 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.936+0000 7f7d3f40e640 1 -- 172.21.15.16:0/3404045410 wait complete. 2024-09-12T02:16:11.939 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.937+0000 7f7d3f40e640 1 Processor -- start 2024-09-12T02:16:11.939 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.937+0000 7f7d3f40e640 1 -- start start 2024-09-12T02:16:11.939 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.937+0000 7f7d3f40e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d380ffe20 0x7f7d38197da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:11.939 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.937+0000 7f7d3f40e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7d38102df0 0x7f7d381982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:11.939 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.937+0000 7f7d3f40e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d381037b0 0x7f7d3819f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:11.939 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.937+0000 7f7d3f40e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7d3810d7f0 con 0x7f7d380ffe20 2024-09-12T02:16:11.940 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.937+0000 7f7d3f40e640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7d3810d670 con 0x7f7d381037b0 2024-09-12T02:16:11.940 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.937+0000 7f7d3f40e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7d3810d970 con 0x7f7d38102df0 2024-09-12T02:16:11.940 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.937+0000 7f7d3d1aa640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d380ffe20 0x7f7d38197da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:11.940 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.937+0000 7f7d3d1aa640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d380ffe20 0x7f7d38197da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53006/0 (socket says 172.21.15.16:53006) 2024-09-12T02:16:11.940 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.937+0000 7f7d3d1aa640 1 -- 172.21.15.16:0/3221948587 learned_addr learned my addr 172.21.15.16:0/3221948587 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:16:11.940 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.937+0000 7f7d3c9a9640 1 --2- 172.21.15.16:0/3221948587 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7d38102df0 0x7f7d381982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:11.940 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.937+0000 7f7d3d9ab640 1 --2- 172.21.15.16:0/3221948587 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d381037b0 0x7f7d3819f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:11.940 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.937+0000 7f7d3d1aa640 1 -- 172.21.15.16:0/3221948587 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7d38102df0 msgr2=0x7f7d381982e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:11.940 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.937+0000 7f7d3d1aa640 1 --2- 172.21.15.16:0/3221948587 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7d38102df0 0x7f7d381982e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:11.940 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.937+0000 7f7d3d1aa640 1 -- 172.21.15.16:0/3221948587 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d381037b0 msgr2=0x7f7d3819f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:11.940 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.937+0000 7f7d3d1aa640 1 --2- 172.21.15.16:0/3221948587 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d381037b0 0x7f7d3819f310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:11.940 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.937+0000 7f7d3d1aa640 1 -- 172.21.15.16:0/3221948587 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d3819f9b0 con 0x7f7d380ffe20 2024-09-12T02:16:11.940 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.938+0000 7f7d3d1aa640 1 --2- 172.21.15.16:0/3221948587 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d380ffe20 0x7f7d38197da0 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7f7d3400d770 tx=0x7f7d3400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:11.941 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.938+0000 7f7d2e7fc640 1 -- 172.21.15.16:0/3221948587 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d34017070 con 0x7f7d380ffe20 2024-09-12T02:16:11.941 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.938+0000 7f7d2e7fc640 1 -- 172.21.15.16:0/3221948587 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d34009c30 con 0x7f7d380ffe20 2024-09-12T02:16:11.941 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.938+0000 7f7d2e7fc640 1 -- 172.21.15.16:0/3221948587 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d34003dd0 con 0x7f7d380ffe20 2024-09-12T02:16:11.941 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.938+0000 7f7d3f40e640 1 -- 172.21.15.16:0/3221948587 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d3819fca0 con 0x7f7d380ffe20 2024-09-12T02:16:11.941 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.938+0000 7f7d3f40e640 1 -- 172.21.15.16:0/3221948587 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d381a01a0 con 0x7f7d380ffe20 2024-09-12T02:16:11.946 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.940+0000 7f7d2e7fc640 1 -- 172.21.15.16:0/3221948587 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f7d34003f70 con 0x7f7d380ffe20 2024-09-12T02:16:11.946 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.941+0000 7f7d42287740 1 Processor -- start 2024-09-12T02:16:11.946 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.941+0000 7f7d42287740 1 -- start start 2024-09-12T02:16:11.946 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.941+0000 7f7d42287740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55933f9de9a0 0x55933f9deda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:11.947 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.941+0000 7f7d42287740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55933f9df2e0 0x55933f9e3da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:11.947 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.941+0000 7f7d42287740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55933f9e4370 0x55933f9e6830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:11.947 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.941+0000 7f7d42287740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55933f9422f0 con 0x55933f9de9a0 2024-09-12T02:16:11.947 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.941+0000 7f7d42287740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55933f8ea270 con 0x55933f9df2e0 2024-09-12T02:16:11.947 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.941+0000 7f7d42287740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55933f93e2b0 con 0x55933f9e4370 2024-09-12T02:16:11.947 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.941+0000 7f7d3d9ab640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55933f9de9a0 0x55933f9deda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:11.947 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.941+0000 7f7d2e7fc640 1 --2- 172.21.15.16:0/3221948587 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f7d240a2c30 0x7f7d240a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:11.947 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.941+0000 7f7d3d9ab640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55933f9de9a0 0x55933f9deda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53010/0 (socket says 172.21.15.16:53010) 2024-09-12T02:16:11.948 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.941+0000 7f7d3d9ab640 1 -- 172.21.15.16:0/2971374853 learned_addr learned my addr 172.21.15.16:0/2971374853 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:16:11.948 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.941+0000 7f7d3d1aa640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55933f9df2e0 0x55933f9e3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:11.948 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.941+0000 7f7d3c9a9640 1 --2- 172.21.15.16:0/2971374853 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55933f9e4370 0x55933f9e6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:11.948 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.941+0000 7f7d3d9ab640 1 --2- 172.21.15.16:0/3221948587 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f7d240a2c30 0x7f7d240a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:11.948 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.941+0000 7f7d2e7fc640 1 -- 172.21.15.16:0/3221948587 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f7d340cea90 con 0x7f7d380ffe20 2024-09-12T02:16:11.948 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.942+0000 7f7d3d9ab640 1 -- 172.21.15.16:0/2971374853 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55933f9e4370 msgr2=0x55933f9e6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:11.948 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.942+0000 7f7d3d9ab640 1 --2- 172.21.15.16:0/2971374853 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55933f9e4370 0x55933f9e6830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:11.948 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.942+0000 7f7d3d9ab640 1 -- 172.21.15.16:0/2971374853 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55933f9df2e0 msgr2=0x55933f9e3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:11.948 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.942+0000 7f7d3d9ab640 1 --2- 172.21.15.16:0/2971374853 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55933f9df2e0 0x55933f9e3da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:11.948 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.942+0000 7f7d3d9ab640 1 -- 172.21.15.16:0/2971374853 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55933f8e9340 con 0x55933f9de9a0 2024-09-12T02:16:11.948 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.942+0000 7f7d3d9ab640 1 --2- 172.21.15.16:0/2971374853 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55933f9de9a0 0x55933f9deda0 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 crypto rx=0x7f7d3002ade0 tx=0x7f7d30031870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3400d34a745cebad server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:11.949 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.942+0000 7f7d3d9ab640 1 --2- 172.21.15.16:0/3221948587 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f7d240a2c30 0x7f7d240a50f0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f7d30031c10 tx=0x7f7d30047000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:11.949 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.942+0000 7f7d237fe640 1 -- 172.21.15.16:0/2971374853 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d3002af40 con 0x55933f9de9a0 2024-09-12T02:16:11.949 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.942+0000 7f7d237fe640 1 -- 172.21.15.16:0/2971374853 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d30036070 con 0x55933f9de9a0 2024-09-12T02:16:11.949 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.942+0000 7f7d237fe640 1 -- 172.21.15.16:0/2971374853 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d3002fde0 con 0x55933f9de9a0 2024-09-12T02:16:11.949 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.943+0000 7f7d42287740 1 -- 172.21.15.16:0/2971374853 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55933f9de9a0 msgr2=0x55933f9deda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:11.949 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.943+0000 7f7d42287740 1 --2- 172.21.15.16:0/2971374853 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55933f9de9a0 0x55933f9deda0 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 crypto rx=0x7f7d3002ade0 tx=0x7f7d30031870 comp rx=0 tx=0).stop 2024-09-12T02:16:11.949 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.943+0000 7f7d42287740 1 -- 172.21.15.16:0/2971374853 shutdown_connections 2024-09-12T02:16:11.949 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.943+0000 7f7d42287740 1 --2- 172.21.15.16:0/2971374853 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55933f9e4370 0x55933f9e6830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:11.949 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.943+0000 7f7d42287740 1 --2- 172.21.15.16:0/2971374853 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55933f9df2e0 0x55933f9e3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:11.949 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.943+0000 7f7d42287740 1 --2- 172.21.15.16:0/2971374853 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55933f9de9a0 0x55933f9deda0 unknown :-1 s=CLOSED pgs=1908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:11.949 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.943+0000 7f7d42287740 1 -- 172.21.15.16:0/2971374853 >> 172.21.15.16:0/2971374853 conn(0x55933f9da010 msgr2=0x55933f9da410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:11.950 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.943+0000 7f7d42287740 1 -- 172.21.15.16:0/2971374853 shutdown_connections 2024-09-12T02:16:11.950 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.943+0000 7f7d42287740 1 -- 172.21.15.16:0/2971374853 wait complete. 2024-09-12T02:16:11.950 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.943+0000 7f7d42287740 1 -- start start 2024-09-12T02:16:11.950 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.943+0000 7f7d42287740 1 Processor -- start 2024-09-12T02:16:11.950 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.944+0000 7f7d42287740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55933f9de9a0 0x55933f9e8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:11.950 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.944+0000 7f7d42287740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55933f9df2e0 0x55933f9e9130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:11.950 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.944+0000 7f7d42287740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55933f9e4370 0x55933f9f0170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:11.950 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.944+0000 7f7d42287740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55933f9e72e0 con 0x55933f9de9a0 2024-09-12T02:16:11.950 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.944+0000 7f7d42287740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55933f8cb290 con 0x55933f9e4370 2024-09-12T02:16:11.951 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.944+0000 7f7d42287740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55933f9e7460 con 0x55933f9df2e0 2024-09-12T02:16:11.951 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.944+0000 7f7d42287740 10 client.0 _ll_register_callbacks cb 0x55933f8fc140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:16:11.951 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.944+0000 7f7d3d1aa640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55933f9de9a0 0x55933f9e8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:11.951 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.944+0000 7f7d3d1aa640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55933f9de9a0 0x55933f9e8bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53018/0 (socket says 172.21.15.16:53018) 2024-09-12T02:16:11.951 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.944+0000 7f7d3d1aa640 1 -- 172.21.15.16:0/3499652230 learned_addr learned my addr 172.21.15.16:0/3499652230 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:16:11.951 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.944+0000 7f7d3c9a9640 1 --2- 172.21.15.16:0/3499652230 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55933f9df2e0 0x55933f9e9130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:11.951 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.944+0000 7f7d3d9ab640 1 --2- 172.21.15.16:0/3499652230 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55933f9e4370 0x55933f9f0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:11.951 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.944+0000 7f7d3d1aa640 1 -- 172.21.15.16:0/3499652230 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55933f9df2e0 msgr2=0x55933f9e9130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:11.951 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.944+0000 7f7d3d1aa640 1 --2- 172.21.15.16:0/3499652230 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55933f9df2e0 0x55933f9e9130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:11.951 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.944+0000 7f7d3d1aa640 1 -- 172.21.15.16:0/3499652230 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55933f9e4370 msgr2=0x55933f9f0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:11.951 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.944+0000 7f7d3d1aa640 1 --2- 172.21.15.16:0/3499652230 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55933f9e4370 0x55933f9f0170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:11.952 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.944+0000 7f7d3d1aa640 1 -- 172.21.15.16:0/3499652230 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55933f655210 con 0x55933f9de9a0 2024-09-12T02:16:11.952 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.945+0000 7f7d3d1aa640 1 --2- 172.21.15.16:0/3499652230 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55933f9de9a0 0x55933f9e8bf0 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7f7d34003820 tx=0x7f7d3406a280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f50c0797fd68374 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:11.952 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.945+0000 7f7d20ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:16:11.952 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.945+0000 7f7d20ff9640 1 -- 172.21.15.16:0/3499652230 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d34020070 con 0x55933f9de9a0 2024-09-12T02:16:11.952 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.945+0000 7f7d20ff9640 1 -- 172.21.15.16:0/3499652230 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f7d3409a7e0 con 0x55933f9de9a0 2024-09-12T02:16:11.952 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.945+0000 7f7d42287740 1 -- 172.21.15.16:0/3499652230 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d30036070 con 0x55933f9de9a0 2024-09-12T02:16:11.952 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.945+0000 7f7d20ff9640 1 -- 172.21.15.16:0/3499652230 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7d340994d0 con 0x55933f9de9a0 2024-09-12T02:16:11.952 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.945+0000 7f7d42287740 10 client.20160 fetch_fsmap learned FSMap version 25 2024-09-12T02:16:11.952 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.945+0000 7f7d42287740 1 -- 172.21.15.16:0/3499652230 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55933f721620 con 0x55933f9de9a0 2024-09-12T02:16:11.952 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.945+0000 7f7d20ff9640 1 -- 172.21.15.16:0/3499652230 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d30036070 con 0x55933f9de9a0 2024-09-12T02:16:11.952 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.945+0000 7f7d20ff9640 1 -- 172.21.15.16:0/3499652230 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f7d340bb0e0 con 0x55933f9de9a0 2024-09-12T02:16:11.952 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.945+0000 7f7d42287740 10 client.20160 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:16:11.953 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.945+0000 7f7d42287740 10 client.20160 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:16:11.953 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.945+0000 7f7d42287740 20 client.20160 populate_metadata read hostname 'smithi016' 2024-09-12T02:16:11.953 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.945+0000 7f7d42287740 1 --2- 172.21.15.16:0/3499652230 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55933f9f48b0 0x55933f9f6ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:11.953 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.945+0000 7f7d42287740 4 client.20160 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:16:11.953 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.945+0000 7f7d42287740 1 -- 172.21.15.16:0/3499652230 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55933f94fa20 con 0x55933f9f48b0 2024-09-12T02:16:11.953 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.946+0000 7f7d3c9a9640 1 --2- 172.21.15.16:0/3499652230 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55933f9f48b0 0x55933f9f6ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:11.953 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.946+0000 7f7d20ff9640 10 client.20160 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:16:11.953 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.946+0000 7f7d3c9a9640 1 --2- 172.21.15.16:0/3499652230 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55933f9f48b0 0x55933f9f6ce0 crc :-1 s=READY pgs=423 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1404e93cccc5f8d2 server_cookie=8a81a775622de215 in_seq=0 out_seq=0 2024-09-12T02:16:11.954 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.947+0000 7f7d20ff9640 1 -- 172.21.15.16:0/3499652230 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55933f94fa20 con 0x55933f9f48b0 2024-09-12T02:16:11.954 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.947+0000 7f7d20ff9640 10 client.20160 handle_command_reply: tid=0 2024-09-12T02:16:11.955 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.947+0000 7f7d42287740 1 client.20160 shutdown 2024-09-12T02:16:11.955 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.947+0000 7f7d42287740 1 -- 172.21.15.16:0/3499652230 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55933f9de9a0 msgr2=0x55933f9e8bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:11.955 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.947+0000 7f7d42287740 1 --2- 172.21.15.16:0/3499652230 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55933f9de9a0 0x55933f9e8bf0 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7f7d34003820 tx=0x7f7d3406a280 comp rx=0 tx=0).stop 2024-09-12T02:16:11.955 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.947+0000 7f7d42287740 1 -- 172.21.15.16:0/3499652230 shutdown_connections 2024-09-12T02:16:11.955 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.947+0000 7f7d42287740 1 --2- 172.21.15.16:0/3499652230 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55933f9f48b0 0x55933f9f6ce0 crc :-1 s=READY pgs=423 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:11.955 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.947+0000 7f7d42287740 1 --2- 172.21.15.16:0/3499652230 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55933f9e4370 0x55933f9f0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:11.955 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.947+0000 7f7d42287740 1 --2- 172.21.15.16:0/3499652230 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55933f9df2e0 0x55933f9e9130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:11.955 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.947+0000 7f7d42287740 1 --2- 172.21.15.16:0/3499652230 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55933f9de9a0 0x55933f9e8bf0 unknown :-1 s=CLOSED pgs=1909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:11.955 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.947+0000 7f7d20ff9640 0 client.20160 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:16:11.955 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.947+0000 7f7d42287740 1 -- 172.21.15.16:0/3499652230 >> 172.21.15.16:0/3499652230 conn(0x55933f9da010 msgr2=0x55933f9db820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:11.955 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.947+0000 7f7d42287740 1 -- 172.21.15.16:0/3499652230 shutdown_connections 2024-09-12T02:16:11.955 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.947+0000 7f7d42287740 1 -- 172.21.15.16:0/3499652230 wait complete. 2024-09-12T02:16:11.956 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.947+0000 7f7d42287740 20 client.20160 trim_cache size 0 max 16384 2024-09-12T02:16:11.963 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.962+0000 7f7d42287740 1 Processor -- start 2024-09-12T02:16:11.963 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.962+0000 7f7d42287740 1 -- start start 2024-09-12T02:16:11.963 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.962+0000 7f7d42287740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55933f9de9a0 0x55933f9e91c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:11.963 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.962+0000 7f7d42287740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55933f9df2e0 0x55933f9f0130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:11.963 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.962+0000 7f7d42287740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55933f9e4370 0x55933f9d90b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:11.964 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.962+0000 7f7d42287740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7d381ab5a0 con 0x55933f9e4370 2024-09-12T02:16:11.964 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.962+0000 7f7d42287740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7d381a8e70 con 0x55933f9df2e0 2024-09-12T02:16:11.964 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.962+0000 7f7d42287740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7d381a6cc0 con 0x55933f9de9a0 2024-09-12T02:16:11.964 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.962+0000 7f7d3d9ab640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55933f9e4370 0x55933f9d90b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:11.964 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.962+0000 7f7d3d9ab640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55933f9e4370 0x55933f9d90b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53028/0 (socket says 172.21.15.16:53028) 2024-09-12T02:16:11.965 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.962+0000 7f7d3d9ab640 1 -- 172.21.15.16:0/3811512796 learned_addr learned my addr 172.21.15.16:0/3811512796 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:16:11.965 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.963+0000 7f7d3d1aa640 1 --2- 172.21.15.16:0/3811512796 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55933f9de9a0 0x55933f9e91c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:11.965 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.963+0000 7f7d3c9a9640 1 --2- 172.21.15.16:0/3811512796 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55933f9df2e0 0x55933f9f0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:11.965 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.963+0000 7f7d3d9ab640 1 -- 172.21.15.16:0/3811512796 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55933f9de9a0 msgr2=0x55933f9e91c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:11.965 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.963+0000 7f7d3d9ab640 1 --2- 172.21.15.16:0/3811512796 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55933f9de9a0 0x55933f9e91c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:11.965 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.963+0000 7f7d3d9ab640 1 -- 172.21.15.16:0/3811512796 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55933f9df2e0 msgr2=0x55933f9f0130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:11.966 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.963+0000 7f7d3d9ab640 1 --2- 172.21.15.16:0/3811512796 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55933f9df2e0 0x55933f9f0130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:11.966 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.963+0000 7f7d3d9ab640 1 -- 172.21.15.16:0/3811512796 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55933f9e0080 con 0x55933f9e4370 2024-09-12T02:16:11.966 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.963+0000 7f7d3c9a9640 1 --2- 172.21.15.16:0/3811512796 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55933f9df2e0 0x55933f9f0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:16:11.966 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.963+0000 7f7d3d9ab640 1 --2- 172.21.15.16:0/3811512796 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55933f9e4370 0x55933f9d90b0 secure :-1 s=READY pgs=1910 cs=0 l=1 rev1=1 crypto rx=0x7f7d30031740 tx=0x7f7d3002a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95bb551ca4b694de server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:11.966 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.963+0000 7f7d22ffd640 1 -- 172.21.15.16:0/3811512796 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d30004610 con 0x55933f9e4370 2024-09-12T02:16:11.967 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.963+0000 7f7d22ffd640 1 -- 172.21.15.16:0/3811512796 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d30036040 con 0x55933f9e4370 2024-09-12T02:16:11.967 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.963+0000 7f7d22ffd640 1 -- 172.21.15.16:0/3811512796 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d300305c0 con 0x55933f9e4370 2024-09-12T02:16:11.971 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.964+0000 7f7d42287740 1 -- 172.21.15.16:0/3811512796 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55933f9e4370 msgr2=0x55933f9d90b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:11.972 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.964+0000 7f7d42287740 1 --2- 172.21.15.16:0/3811512796 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55933f9e4370 0x55933f9d90b0 secure :-1 s=READY pgs=1910 cs=0 l=1 rev1=1 crypto rx=0x7f7d30031740 tx=0x7f7d3002a8c0 comp rx=0 tx=0).stop 2024-09-12T02:16:11.972 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.964+0000 7f7d42287740 1 -- 172.21.15.16:0/3811512796 shutdown_connections 2024-09-12T02:16:11.972 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.964+0000 7f7d42287740 1 --2- 172.21.15.16:0/3811512796 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55933f9e4370 0x55933f9d90b0 unknown :-1 s=CLOSED pgs=1910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:11.972 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.964+0000 7f7d42287740 1 --2- 172.21.15.16:0/3811512796 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55933f9df2e0 0x55933f9f0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:11.972 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.964+0000 7f7d42287740 1 --2- 172.21.15.16:0/3811512796 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55933f9de9a0 0x55933f9e91c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:11.972 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.964+0000 7f7d42287740 1 -- 172.21.15.16:0/3811512796 >> 172.21.15.16:0/3811512796 conn(0x55933f9da010 msgr2=0x55933f9d89a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:11.972 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.964+0000 7f7d42287740 1 -- 172.21.15.16:0/3811512796 shutdown_connections 2024-09-12T02:16:11.972 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.964+0000 7f7d42287740 1 -- 172.21.15.16:0/3811512796 wait complete. 2024-09-12T02:16:11.972 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.964+0000 7f7d42287740 1 -- start start 2024-09-12T02:16:11.973 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.964+0000 7f7d42287740 1 Processor -- start 2024-09-12T02:16:11.973 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.964+0000 7f7d42287740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55933f9de9a0 0x55933f9edf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:11.973 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.964+0000 7f7d42287740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55933f9df2e0 0x55933f9ee450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:11.973 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.964+0000 7f7d42287740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55933f9e4370 0x55933f9ef460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:11.973 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.964+0000 7f7d42287740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7d381a7330 con 0x55933f9de9a0 2024-09-12T02:16:11.973 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.964+0000 7f7d42287740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7d381a87a0 con 0x55933f9e4370 2024-09-12T02:16:11.973 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.964+0000 7f7d42287740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7d381a7a00 con 0x55933f9df2e0 2024-09-12T02:16:11.973 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.965+0000 7f7d42287740 10 client.0 _ll_register_callbacks cb 0x7f7d3400d6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:16:11.973 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.965+0000 7f7d3d1aa640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55933f9de9a0 0x55933f9edf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:11.973 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.965+0000 7f7d3d1aa640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55933f9de9a0 0x55933f9edf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53044/0 (socket says 172.21.15.16:53044) 2024-09-12T02:16:11.974 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.965+0000 7f7d3d1aa640 1 -- 172.21.15.16:0/4209955748 learned_addr learned my addr 172.21.15.16:0/4209955748 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:16:11.974 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.965+0000 7f7d3c9a9640 1 --2- 172.21.15.16:0/4209955748 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55933f9df2e0 0x55933f9ee450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:11.974 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.965+0000 7f7d3d9ab640 1 --2- 172.21.15.16:0/4209955748 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55933f9e4370 0x55933f9ef460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:11.974 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.965+0000 7f7d3d1aa640 1 -- 172.21.15.16:0/4209955748 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55933f9df2e0 msgr2=0x55933f9ee450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:11.974 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.965+0000 7f7d3d1aa640 1 --2- 172.21.15.16:0/4209955748 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55933f9df2e0 0x55933f9ee450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:11.974 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.965+0000 7f7d3d1aa640 1 -- 172.21.15.16:0/4209955748 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55933f9e4370 msgr2=0x55933f9ef460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:11.974 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.965+0000 7f7d3d1aa640 1 --2- 172.21.15.16:0/4209955748 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55933f9e4370 0x55933f9ef460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:11.974 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.965+0000 7f7d3d1aa640 1 -- 172.21.15.16:0/4209955748 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55933f9efa70 con 0x55933f9de9a0 2024-09-12T02:16:11.974 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.965+0000 7f7d3c9a9640 1 --2- 172.21.15.16:0/4209955748 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55933f9df2e0 0x55933f9ee450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:16:11.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.965+0000 7f7d3d9ab640 1 --2- 172.21.15.16:0/4209955748 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55933f9e4370 0x55933f9ef460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:16:11.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.965+0000 7f7d3d1aa640 1 --2- 172.21.15.16:0/4209955748 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55933f9de9a0 0x55933f9edf10 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7f7d340bb500 tx=0x7f7d340bb2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd96b381eb6e393d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:11.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.965+0000 7f7d20ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:16:11.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.965+0000 7f7d20ff9640 1 -- 172.21.15.16:0/4209955748 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d34020070 con 0x55933f9de9a0 2024-09-12T02:16:11.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.965+0000 7f7d20ff9640 1 -- 172.21.15.16:0/4209955748 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f7d34097050 con 0x55933f9de9a0 2024-09-12T02:16:11.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.965+0000 7f7d42287740 1 -- 172.21.15.16:0/4209955748 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d30036040 con 0x55933f9de9a0 2024-09-12T02:16:11.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.965+0000 7f7d20ff9640 1 -- 172.21.15.16:0/4209955748 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7d340975a0 con 0x55933f9de9a0 2024-09-12T02:16:11.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.965+0000 7f7d42287740 10 client.20166 fetch_fsmap learned FSMap version 25 2024-09-12T02:16:11.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.965+0000 7f7d42287740 1 -- 172.21.15.16:0/4209955748 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55933fa5fba0 con 0x55933f9de9a0 2024-09-12T02:16:11.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.966+0000 7f7d20ff9640 1 -- 172.21.15.16:0/4209955748 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d30036040 con 0x55933f9de9a0 2024-09-12T02:16:11.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.966+0000 7f7d20ff9640 1 -- 172.21.15.16:0/4209955748 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f7d3406b070 con 0x55933f9de9a0 2024-09-12T02:16:11.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.966+0000 7f7d42287740 10 client.20166 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:16:11.976 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.966+0000 7f7d42287740 10 client.20166 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:16:11.976 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.966+0000 7f7d42287740 20 client.20166 populate_metadata read hostname 'smithi016' 2024-09-12T02:16:11.976 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.966+0000 7f7d42287740 1 --2- 172.21.15.16:0/4209955748 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55933fa60520 0x55933fa62940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:11.976 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.966+0000 7f7d42287740 4 client.20166 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:16:11.976 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.966+0000 7f7d42287740 1 -- 172.21.15.16:0/4209955748 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55933f949d70 con 0x55933fa60520 2024-09-12T02:16:11.976 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.966+0000 7f7d3c9a9640 1 --2- 172.21.15.16:0/4209955748 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55933fa60520 0x55933fa62940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:11.976 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.966+0000 7f7d20ff9640 10 client.20166 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:16:11.976 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.966+0000 7f7d3c9a9640 1 --2- 172.21.15.16:0/4209955748 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55933fa60520 0x55933fa62940 crc :-1 s=READY pgs=424 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ef5f01cccf6a0cb server_cookie=84933f8eac189e39 in_seq=0 out_seq=0 2024-09-12T02:16:11.978 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.977+0000 7f7d20ff9640 1 -- 172.21.15.16:0/4209955748 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55933f949d70 con 0x55933fa60520 2024-09-12T02:16:11.978 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.977+0000 7f7d20ff9640 10 client.20166 handle_command_reply: tid=0 2024-09-12T02:16:11.978 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.977+0000 7f7d42287740 1 client.20166 shutdown 2024-09-12T02:16:11.978 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.977+0000 7f7d42287740 1 -- 172.21.15.16:0/4209955748 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55933f9de9a0 msgr2=0x55933f9edf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:11.978 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.977+0000 7f7d42287740 1 --2- 172.21.15.16:0/4209955748 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55933f9de9a0 0x55933f9edf10 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7f7d340bb500 tx=0x7f7d340bb2e0 comp rx=0 tx=0).stop 2024-09-12T02:16:11.978 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.977+0000 7f7d42287740 1 -- 172.21.15.16:0/4209955748 shutdown_connections 2024-09-12T02:16:11.979 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.977+0000 7f7d42287740 1 --2- 172.21.15.16:0/4209955748 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55933fa60520 0x55933fa62940 crc :-1 s=READY pgs=424 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:11.979 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.977+0000 7f7d42287740 1 --2- 172.21.15.16:0/4209955748 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55933f9e4370 0x55933f9ef460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:11.979 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.978+0000 7f7d42287740 1 --2- 172.21.15.16:0/4209955748 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55933f9df2e0 0x55933f9ee450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:11.979 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.978+0000 7f7d42287740 1 --2- 172.21.15.16:0/4209955748 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55933f9de9a0 0x55933f9edf10 unknown :-1 s=CLOSED pgs=1911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:11.979 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.978+0000 7f7d20ff9640 0 client.20166 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:16:11.979 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.978+0000 7f7d42287740 1 -- 172.21.15.16:0/4209955748 >> 172.21.15.16:0/4209955748 conn(0x55933f9da010 msgr2=0x55933f9d91c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:11.979 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.978+0000 7f7d42287740 1 -- 172.21.15.16:0/4209955748 shutdown_connections 2024-09-12T02:16:11.979 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.978+0000 7f7d42287740 1 -- 172.21.15.16:0/4209955748 wait complete. 2024-09-12T02:16:11.979 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.978+0000 7f7d42287740 20 client.20166 trim_cache size 0 max 16384 2024-09-12T02:16:11.980 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-09-12T02:16:11.980 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-09-12T02:16:11.980 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-09-12T02:16:11.980 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-09-12T02:16:11.981 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.978+0000 7f7d3f40e640 1 -- 172.21.15.16:0/3221948587 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f7d240a2c30 msgr2=0x7f7d240a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:11.981 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.978+0000 7f7d3f40e640 1 --2- 172.21.15.16:0/3221948587 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f7d240a2c30 0x7f7d240a50f0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f7d30031c10 tx=0x7f7d30047000 comp rx=0 tx=0).stop 2024-09-12T02:16:11.981 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.978+0000 7f7d3f40e640 1 -- 172.21.15.16:0/3221948587 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d380ffe20 msgr2=0x7f7d38197da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:11.981 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.978+0000 7f7d3f40e640 1 --2- 172.21.15.16:0/3221948587 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d380ffe20 0x7f7d38197da0 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7f7d3400d770 tx=0x7f7d3400dc30 comp rx=0 tx=0).stop 2024-09-12T02:16:11.981 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.978+0000 7f7d3f40e640 1 -- 172.21.15.16:0/3221948587 shutdown_connections 2024-09-12T02:16:11.981 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.978+0000 7f7d3f40e640 1 --2- 172.21.15.16:0/3221948587 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f7d240a2c30 0x7f7d240a50f0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:11.981 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.978+0000 7f7d3f40e640 1 --2- 172.21.15.16:0/3221948587 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d381037b0 0x7f7d3819f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:11.981 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.978+0000 7f7d3f40e640 1 --2- 172.21.15.16:0/3221948587 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7d38102df0 0x7f7d381982e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:11.982 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.978+0000 7f7d3f40e640 1 --2- 172.21.15.16:0/3221948587 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d380ffe20 0x7f7d38197da0 unknown :-1 s=CLOSED pgs=1907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:11.982 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.978+0000 7f7d3f40e640 1 -- 172.21.15.16:0/3221948587 >> 172.21.15.16:0/3221948587 conn(0x7f7d380fb980 msgr2=0x7f7d381060f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:11.982 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.978+0000 7f7d3f40e640 1 -- 172.21.15.16:0/3221948587 shutdown_connections 2024-09-12T02:16:11.982 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:11.979+0000 7f7d3f40e640 1 -- 172.21.15.16:0/3221948587 wait complete. 2024-09-12T02:16:12.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:16:12 smithi016 ceph-mon[30947]: pgmap v763: 97 pgs: 97 active+clean; 3.1 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 4.3 KiB/s wr, 9 op/s 2024-09-12T02:16:12.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:16:12 smithi184 ceph-mon[35593]: pgmap v763: 97 pgs: 97 active+clean; 3.1 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 4.3 KiB/s wr, 9 op/s 2024-09-12T02:16:12.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:16:12 smithi033 ceph-mon[35332]: pgmap v763: 97 pgs: 97 active+clean; 3.1 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 4.3 KiB/s wr, 9 op/s 2024-09-12T02:16:13.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:16:13 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:16:13.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:16:13 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:16:13.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:16:13 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:16:13.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:16:13 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T02:16:13.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:16:13 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:16:13.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:16:13 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:16:13.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:16:13 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:16:13.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:16:13 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T02:16:13.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:16:13 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:16:13.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:16:13 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:16:13.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:16:13 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:16:13.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:16:13 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T02:16:14.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:16:14 smithi184 ceph-mon[35593]: pgmap v764: 97 pgs: 97 active+clean; 3.2 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 9.2 KiB/s wr, 9 op/s 2024-09-12T02:16:14.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:16:14 smithi033 ceph-mon[35332]: pgmap v764: 97 pgs: 97 active+clean; 3.2 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 9.2 KiB/s wr, 9 op/s 2024-09-12T02:16:14.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:16:14 smithi016 ceph-mon[30947]: pgmap v764: 97 pgs: 97 active+clean; 3.2 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 9.2 KiB/s wr, 9 op/s 2024-09-12T02:16:16.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:16:16 smithi184 ceph-mon[35593]: pgmap v765: 97 pgs: 97 active+clean; 3.2 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 9.2 KiB/s wr, 9 op/s 2024-09-12T02:16:16.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:16:16 smithi033 ceph-mon[35332]: pgmap v765: 97 pgs: 97 active+clean; 3.2 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 9.2 KiB/s wr, 9 op/s 2024-09-12T02:16:16.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:16:16 smithi016 ceph-mon[30947]: pgmap v765: 97 pgs: 97 active+clean; 3.2 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 9.2 KiB/s wr, 9 op/s 2024-09-12T02:16:18.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:16:18 smithi184 ceph-mon[35593]: pgmap v766: 97 pgs: 97 active+clean; 3.2 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 16 KiB/s wr, 14 op/s 2024-09-12T02:16:18.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:16:18 smithi033 ceph-mon[35332]: pgmap v766: 97 pgs: 97 active+clean; 3.2 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 16 KiB/s wr, 14 op/s 2024-09-12T02:16:18.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:16:18 smithi016 ceph-mon[30947]: pgmap v766: 97 pgs: 97 active+clean; 3.2 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 16 KiB/s wr, 14 op/s 2024-09-12T02:16:20.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:16:20 smithi184 ceph-mon[35593]: pgmap v767: 97 pgs: 97 active+clean; 3.2 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T02:16:20.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:16:20 smithi033 ceph-mon[35332]: pgmap v767: 97 pgs: 97 active+clean; 3.2 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T02:16:20.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:16:20 smithi016 ceph-mon[30947]: pgmap v767: 97 pgs: 97 active+clean; 3.2 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T02:16:20.779 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T02:16:21.050 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:16:21.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.520+0000 7fb87ad67640 1 -- 172.21.15.16:0/3472659493 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb874101de0 msgr2=0x7fb8741021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:21.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.520+0000 7fb87ad67640 1 --2- 172.21.15.16:0/3472659493 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb874101de0 0x7fb8741021c0 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x7fb868009900 tx=0x7fb86802a410 comp rx=0 tx=0).stop 2024-09-12T02:16:21.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.520+0000 7fb87ad67640 1 -- 172.21.15.16:0/3472659493 shutdown_connections 2024-09-12T02:16:21.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.520+0000 7fb87ad67640 1 --2- 172.21.15.16:0/3472659493 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb874106f10 0x7fb87410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:21.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.520+0000 7fb87ad67640 1 --2- 172.21.15.16:0/3472659493 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb874102790 0x7fb8741067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:21.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.520+0000 7fb87ad67640 1 --2- 172.21.15.16:0/3472659493 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb874101de0 0x7fb8741021c0 unknown :-1 s=CLOSED pgs=1912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:21.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.520+0000 7fb87ad67640 1 -- 172.21.15.16:0/3472659493 >> 172.21.15.16:0/3472659493 conn(0x7fb8740fd8c0 msgr2=0x7fb8740ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:21.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.520+0000 7fb87ad67640 1 -- 172.21.15.16:0/3472659493 shutdown_connections 2024-09-12T02:16:21.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.520+0000 7fb87ad67640 1 -- 172.21.15.16:0/3472659493 wait complete. 2024-09-12T02:16:21.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.521+0000 7fb87ad67640 1 Processor -- start 2024-09-12T02:16:21.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.521+0000 7fb87ad67640 1 -- start start 2024-09-12T02:16:21.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.521+0000 7fb87ad67640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb874101de0 0x7fb8740744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:21.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.521+0000 7fb87ad67640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb874102790 0x7fb874074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:21.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.521+0000 7fb87ad67640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb874106f10 0x7fb874074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:21.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.521+0000 7fb87ad67640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb874111f70 con 0x7fb874101de0 2024-09-12T02:16:21.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.521+0000 7fb87ad67640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb874111df0 con 0x7fb874106f10 2024-09-12T02:16:21.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.521+0000 7fb87ad67640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb8741120f0 con 0x7fb874102790 2024-09-12T02:16:21.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.521+0000 7fb878b03640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb874101de0 0x7fb8740744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:21.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.521+0000 7fb878b03640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb874101de0 0x7fb8740744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40102/0 (socket says 172.21.15.16:40102) 2024-09-12T02:16:21.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.521+0000 7fb878b03640 1 -- 172.21.15.16:0/1334297762 learned_addr learned my addr 172.21.15.16:0/1334297762 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:16:21.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.521+0000 7fb873fff640 1 --2- 172.21.15.16:0/1334297762 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb874102790 0x7fb874074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:21.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.521+0000 7fb879304640 1 --2- 172.21.15.16:0/1334297762 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb874106f10 0x7fb874074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:21.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.522+0000 7fb878b03640 1 -- 172.21.15.16:0/1334297762 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb874102790 msgr2=0x7fb874074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:21.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.522+0000 7fb878b03640 1 --2- 172.21.15.16:0/1334297762 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb874102790 0x7fb874074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:21.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.522+0000 7fb878b03640 1 -- 172.21.15.16:0/1334297762 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb874106f10 msgr2=0x7fb874074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:21.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.522+0000 7fb878b03640 1 --2- 172.21.15.16:0/1334297762 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb874106f10 0x7fb874074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:21.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.522+0000 7fb878b03640 1 -- 172.21.15.16:0/1334297762 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8741a6890 con 0x7fb874101de0 2024-09-12T02:16:21.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.522+0000 7fb878b03640 1 --2- 172.21.15.16:0/1334297762 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb874101de0 0x7fb8740744f0 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7fb868006010 tx=0x7fb8680042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:21.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.522+0000 7fb871ffb640 1 -- 172.21.15.16:0/1334297762 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb86803f030 con 0x7fb874101de0 2024-09-12T02:16:21.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.522+0000 7fb871ffb640 1 -- 172.21.15.16:0/1334297762 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb868003ce0 con 0x7fb874101de0 2024-09-12T02:16:21.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.522+0000 7fb87ad67640 1 -- 172.21.15.16:0/1334297762 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8741a6b20 con 0x7fb874101de0 2024-09-12T02:16:21.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.522+0000 7fb871ffb640 1 -- 172.21.15.16:0/1334297762 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb868035070 con 0x7fb874101de0 2024-09-12T02:16:21.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.522+0000 7fb87ad67640 1 -- 172.21.15.16:0/1334297762 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8741a7050 con 0x7fb874101de0 2024-09-12T02:16:21.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.524+0000 7fb871ffb640 1 -- 172.21.15.16:0/1334297762 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fb868042030 con 0x7fb874101de0 2024-09-12T02:16:21.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.525+0000 7fb87d140740 1 Processor -- start 2024-09-12T02:16:21.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.525+0000 7fb87d140740 1 -- start start 2024-09-12T02:16:21.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.525+0000 7fb87d140740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5616a14694a0 0x5616a1469880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:21.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.525+0000 7fb87d140740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616a1469dc0 0x5616a1474140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:21.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.525+0000 7fb87d140740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5616a1474680 0x5616a1476a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:21.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.525+0000 7fb87d140740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5616a13cf2e0 con 0x5616a1469dc0 2024-09-12T02:16:21.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.525+0000 7fb87d140740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5616a1376ce0 con 0x5616a14694a0 2024-09-12T02:16:21.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.525+0000 7fb87d140740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5616a13cb2a0 con 0x5616a1474680 2024-09-12T02:16:21.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.525+0000 7fb878b03640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616a1469dc0 0x5616a1474140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:21.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.525+0000 7fb878b03640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616a1469dc0 0x5616a1474140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40106/0 (socket says 172.21.15.16:40106) 2024-09-12T02:16:21.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.525+0000 7fb878b03640 1 -- 172.21.15.16:0/3451787890 learned_addr learned my addr 172.21.15.16:0/3451787890 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:16:21.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.525+0000 7fb871ffb640 1 --2- 172.21.15.16:0/1334297762 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fb8640a2c00 0x7fb8640a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:21.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.525+0000 7fb879304640 1 --2- 172.21.15.16:0/3451787890 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5616a14694a0 0x5616a1469880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:21.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.525+0000 7fb879304640 1 --2- 172.21.15.16:0/1334297762 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fb8640a2c00 0x7fb8640a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:21.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.526+0000 7fb871ffb640 1 -- 172.21.15.16:0/1334297762 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fb8680ef4a0 con 0x7fb874101de0 2024-09-12T02:16:21.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.526+0000 7fb878b03640 1 -- 172.21.15.16:0/3451787890 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5616a1474680 msgr2=0x5616a1476a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:21.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.525+0000 7fb873fff640 1 --2- 172.21.15.16:0/3451787890 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5616a1474680 0x5616a1476a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:21.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.526+0000 7fb878b03640 1 --2- 172.21.15.16:0/3451787890 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5616a1474680 0x5616a1476a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:21.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.526+0000 7fb878b03640 1 -- 172.21.15.16:0/3451787890 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5616a14694a0 msgr2=0x5616a1469880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:21.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.526+0000 7fb878b03640 1 --2- 172.21.15.16:0/3451787890 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5616a14694a0 0x5616a1469880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:21.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.526+0000 7fb878b03640 1 -- 172.21.15.16:0/3451787890 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5616a135b440 con 0x5616a1469dc0 2024-09-12T02:16:21.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.526+0000 7fb873fff640 1 --2- 172.21.15.16:0/3451787890 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5616a1474680 0x5616a1476a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:16:21.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.526+0000 7fb878b03640 1 --2- 172.21.15.16:0/3451787890 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616a1469dc0 0x5616a1474140 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x7fb8680bbea0 tx=0x7fb86802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49bd0da20ea12910 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:21.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.526+0000 7fb879304640 1 --2- 172.21.15.16:0/1334297762 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fb8640a2c00 0x7fb8640a50c0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fb86c007fd0 tx=0x7fb86c007520 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:21.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.526+0000 7fb85effd640 1 -- 172.21.15.16:0/3451787890 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb868049030 con 0x5616a1469dc0 2024-09-12T02:16:21.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.526+0000 7fb85effd640 1 -- 172.21.15.16:0/3451787890 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb86803b580 con 0x5616a1469dc0 2024-09-12T02:16:21.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.526+0000 7fb85effd640 1 -- 172.21.15.16:0/3451787890 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8680c0210 con 0x5616a1469dc0 2024-09-12T02:16:21.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.527+0000 7fb87d140740 1 -- 172.21.15.16:0/3451787890 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616a1469dc0 msgr2=0x5616a1474140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:21.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.527+0000 7fb87d140740 1 --2- 172.21.15.16:0/3451787890 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616a1469dc0 0x5616a1474140 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x7fb8680bbea0 tx=0x7fb86802a410 comp rx=0 tx=0).stop 2024-09-12T02:16:21.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.527+0000 7fb87d140740 1 -- 172.21.15.16:0/3451787890 shutdown_connections 2024-09-12T02:16:21.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.527+0000 7fb87d140740 1 --2- 172.21.15.16:0/3451787890 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5616a1474680 0x5616a1476a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:21.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.527+0000 7fb87d140740 1 --2- 172.21.15.16:0/3451787890 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616a1469dc0 0x5616a1474140 unknown :-1 s=CLOSED pgs=1914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:21.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.527+0000 7fb87d140740 1 --2- 172.21.15.16:0/3451787890 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5616a14694a0 0x5616a1469880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:21.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.527+0000 7fb87d140740 1 -- 172.21.15.16:0/3451787890 >> 172.21.15.16:0/3451787890 conn(0x5616a1465a30 msgr2=0x5616a1465e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:21.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.527+0000 7fb87d140740 1 -- 172.21.15.16:0/3451787890 shutdown_connections 2024-09-12T02:16:21.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.527+0000 7fb87d140740 1 -- 172.21.15.16:0/3451787890 wait complete. 2024-09-12T02:16:21.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.527+0000 7fb87d140740 1 -- start start 2024-09-12T02:16:21.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.527+0000 7fb87d140740 1 Processor -- start 2024-09-12T02:16:21.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.528+0000 7fb87d140740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616a14694a0 0x5616a1473ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:21.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.528+0000 7fb87d140740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5616a1469dc0 0x5616a1471150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:21.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.528+0000 7fb87d140740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5616a1474680 0x5616a1471690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:21.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.528+0000 7fb87d140740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5616a1477010 con 0x5616a14694a0 2024-09-12T02:16:21.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.528+0000 7fb87d140740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5616a138a050 con 0x5616a1474680 2024-09-12T02:16:21.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.528+0000 7fb87d140740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5616a1477190 con 0x5616a1469dc0 2024-09-12T02:16:21.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.528+0000 7fb87d140740 10 client.0 _ll_register_callbacks cb 0x5616a13ca160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:16:21.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.528+0000 7fb878b03640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616a14694a0 0x5616a1473ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:21.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.528+0000 7fb878b03640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616a14694a0 0x5616a1473ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40116/0 (socket says 172.21.15.16:40116) 2024-09-12T02:16:21.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.528+0000 7fb878b03640 1 -- 172.21.15.16:0/3088691555 learned_addr learned my addr 172.21.15.16:0/3088691555 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:16:21.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.528+0000 7fb873fff640 1 --2- 172.21.15.16:0/3088691555 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5616a1469dc0 0x5616a1471150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:21.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.528+0000 7fb879304640 1 --2- 172.21.15.16:0/3088691555 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5616a1474680 0x5616a1471690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:21.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.528+0000 7fb878b03640 1 -- 172.21.15.16:0/3088691555 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5616a1469dc0 msgr2=0x5616a1471150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:21.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.528+0000 7fb878b03640 1 --2- 172.21.15.16:0/3088691555 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5616a1469dc0 0x5616a1471150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:21.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.528+0000 7fb878b03640 1 -- 172.21.15.16:0/3088691555 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5616a1474680 msgr2=0x5616a1471690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:21.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.528+0000 7fb878b03640 1 --2- 172.21.15.16:0/3088691555 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5616a1474680 0x5616a1471690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:21.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.528+0000 7fb878b03640 1 -- 172.21.15.16:0/3088691555 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5616a11403c0 con 0x5616a14694a0 2024-09-12T02:16:21.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.528+0000 7fb879304640 1 --2- 172.21.15.16:0/3088691555 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5616a1474680 0x5616a1471690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:16:21.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.528+0000 7fb873fff640 1 --2- 172.21.15.16:0/3088691555 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5616a1469dc0 0x5616a1471150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:16:21.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.528+0000 7fb878b03640 1 --2- 172.21.15.16:0/3088691555 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616a14694a0 0x5616a1473ea0 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x7fb868005ab0 tx=0x7fb8680095e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb4d9dfabc4a9d68 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:21.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.528+0000 7fb853fff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:16:21.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.528+0000 7fb853fff640 1 -- 172.21.15.16:0/3088691555 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8680d9060 con 0x5616a14694a0 2024-09-12T02:16:21.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.529+0000 7fb853fff640 1 -- 172.21.15.16:0/3088691555 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fb8680b8580 con 0x5616a14694a0 2024-09-12T02:16:21.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.529+0000 7fb87d140740 1 -- 172.21.15.16:0/3088691555 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5616a13dc850 con 0x5616a14694a0 2024-09-12T02:16:21.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.529+0000 7fb853fff640 1 -- 172.21.15.16:0/3088691555 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fb868049070 con 0x5616a14694a0 2024-09-12T02:16:21.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.529+0000 7fb87d140740 10 client.20178 fetch_fsmap learned FSMap version 25 2024-09-12T02:16:21.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.529+0000 7fb87d140740 1 -- 172.21.15.16:0/3088691555 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5616a1477010 con 0x5616a14694a0 2024-09-12T02:16:21.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.529+0000 7fb853fff640 1 -- 172.21.15.16:0/3088691555 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5616a13dc850 con 0x5616a14694a0 2024-09-12T02:16:21.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.529+0000 7fb853fff640 1 -- 172.21.15.16:0/3088691555 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fb8680bc170 con 0x5616a14694a0 2024-09-12T02:16:21.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.529+0000 7fb87d140740 10 client.20178 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:16:21.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.529+0000 7fb87d140740 10 client.20178 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:16:21.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.529+0000 7fb87d140740 20 client.20178 populate_metadata read hostname 'smithi016' 2024-09-12T02:16:21.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.529+0000 7fb87d140740 1 --2- 172.21.15.16:0/3088691555 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5616a1483a30 0x5616a1485e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:21.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.529+0000 7fb87d140740 4 client.20178 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:16:21.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.529+0000 7fb87d140740 1 -- 172.21.15.16:0/3088691555 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5616a13d6d60 con 0x5616a1483a30 2024-09-12T02:16:21.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.530+0000 7fb873fff640 1 --2- 172.21.15.16:0/3088691555 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5616a1483a30 0x5616a1485e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:21.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.530+0000 7fb873fff640 1 --2- 172.21.15.16:0/3088691555 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5616a1483a30 0x5616a1485e50 crc :-1 s=READY pgs=425 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8dfb1d9d0ac04390 server_cookie=2511a7fac010673b in_seq=0 out_seq=0 2024-09-12T02:16:21.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.530+0000 7fb853fff640 10 client.20178 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:16:21.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.531+0000 7fb853fff640 1 -- 172.21.15.16:0/3088691555 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5616a13d6d60 con 0x5616a1483a30 2024-09-12T02:16:21.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.531+0000 7fb853fff640 10 client.20178 handle_command_reply: tid=0 2024-09-12T02:16:21.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.531+0000 7fb87d140740 1 client.20178 shutdown 2024-09-12T02:16:21.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.531+0000 7fb87d140740 1 -- 172.21.15.16:0/3088691555 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616a14694a0 msgr2=0x5616a1473ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:21.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.531+0000 7fb87d140740 1 --2- 172.21.15.16:0/3088691555 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616a14694a0 0x5616a1473ea0 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x7fb868005ab0 tx=0x7fb8680095e0 comp rx=0 tx=0).stop 2024-09-12T02:16:21.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.531+0000 7fb87d140740 1 -- 172.21.15.16:0/3088691555 shutdown_connections 2024-09-12T02:16:21.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.531+0000 7fb87d140740 1 --2- 172.21.15.16:0/3088691555 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5616a1483a30 0x5616a1485e50 crc :-1 s=READY pgs=425 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:21.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.531+0000 7fb87d140740 1 --2- 172.21.15.16:0/3088691555 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5616a1474680 0x5616a1471690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:21.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.531+0000 7fb87d140740 1 --2- 172.21.15.16:0/3088691555 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5616a1469dc0 0x5616a1471150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:21.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.531+0000 7fb87d140740 1 --2- 172.21.15.16:0/3088691555 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616a14694a0 0x5616a1473ea0 unknown :-1 s=CLOSED pgs=1915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:21.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.531+0000 7fb853fff640 0 client.20178 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:16:21.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.531+0000 7fb87d140740 1 -- 172.21.15.16:0/3088691555 >> 172.21.15.16:0/3088691555 conn(0x5616a1465a30 msgr2=0x5616a146b760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:21.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.531+0000 7fb87d140740 1 -- 172.21.15.16:0/3088691555 shutdown_connections 2024-09-12T02:16:21.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.531+0000 7fb87d140740 1 -- 172.21.15.16:0/3088691555 wait complete. 2024-09-12T02:16:21.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.531+0000 7fb87d140740 20 client.20178 trim_cache size 0 max 16384 2024-09-12T02:16:21.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.548+0000 7fb87d140740 1 Processor -- start 2024-09-12T02:16:21.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.548+0000 7fb87d140740 1 -- start start 2024-09-12T02:16:21.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.548+0000 7fb87d140740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5616a14694a0 0x5616a146fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:21.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.548+0000 7fb87d140740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5616a1469dc0 0x5616a148b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:21.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.548+0000 7fb87d140740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616a1474680 0x5616a148bc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:21.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.548+0000 7fb87d140740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb8741b1e30 con 0x5616a1474680 2024-09-12T02:16:21.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.548+0000 7fb87d140740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb8741af700 con 0x5616a1469dc0 2024-09-12T02:16:21.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.548+0000 7fb87d140740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb8741ad570 con 0x5616a14694a0 2024-09-12T02:16:21.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.549+0000 7fb879304640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616a1474680 0x5616a148bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:21.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.549+0000 7fb879304640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616a1474680 0x5616a148bc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40128/0 (socket says 172.21.15.16:40128) 2024-09-12T02:16:21.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.549+0000 7fb879304640 1 -- 172.21.15.16:0/3193669047 learned_addr learned my addr 172.21.15.16:0/3193669047 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:16:21.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.549+0000 7fb878b03640 1 --2- 172.21.15.16:0/3193669047 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5616a14694a0 0x5616a146fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:21.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.549+0000 7fb873fff640 1 --2- 172.21.15.16:0/3193669047 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5616a1469dc0 0x5616a148b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:21.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.549+0000 7fb879304640 1 -- 172.21.15.16:0/3193669047 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5616a14694a0 msgr2=0x5616a146fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:21.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.549+0000 7fb879304640 1 --2- 172.21.15.16:0/3193669047 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5616a14694a0 0x5616a146fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:21.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.549+0000 7fb879304640 1 -- 172.21.15.16:0/3193669047 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5616a1469dc0 msgr2=0x5616a148b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:21.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.549+0000 7fb879304640 1 --2- 172.21.15.16:0/3193669047 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5616a1469dc0 0x5616a148b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:21.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.549+0000 7fb879304640 1 -- 172.21.15.16:0/3193669047 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5616a1465860 con 0x5616a1474680 2024-09-12T02:16:21.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.549+0000 7fb873fff640 1 --2- 172.21.15.16:0/3193669047 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5616a1469dc0 0x5616a148b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:16:21.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.549+0000 7fb879304640 1 --2- 172.21.15.16:0/3193669047 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616a1474680 0x5616a148bc60 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7fb86c011c60 tx=0x7fb86c0078c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2667bbb0af083d36 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:21.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.549+0000 7fb85e7fc640 1 -- 172.21.15.16:0/3193669047 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb86c011c90 con 0x5616a1474680 2024-09-12T02:16:21.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.549+0000 7fb85e7fc640 1 -- 172.21.15.16:0/3193669047 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb86c011e30 con 0x5616a1474680 2024-09-12T02:16:21.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.549+0000 7fb85e7fc640 1 -- 172.21.15.16:0/3193669047 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb86c005580 con 0x5616a1474680 2024-09-12T02:16:21.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.550+0000 7fb87d140740 1 -- 172.21.15.16:0/3193669047 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616a1474680 msgr2=0x5616a148bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:21.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.550+0000 7fb87d140740 1 --2- 172.21.15.16:0/3193669047 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616a1474680 0x5616a148bc60 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7fb86c011c60 tx=0x7fb86c0078c0 comp rx=0 tx=0).stop 2024-09-12T02:16:21.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.550+0000 7fb87d140740 1 -- 172.21.15.16:0/3193669047 shutdown_connections 2024-09-12T02:16:21.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.550+0000 7fb87d140740 1 --2- 172.21.15.16:0/3193669047 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616a1474680 0x5616a148bc60 unknown :-1 s=CLOSED pgs=1916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:21.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.550+0000 7fb87d140740 1 --2- 172.21.15.16:0/3193669047 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5616a1469dc0 0x5616a148b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:21.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.550+0000 7fb87d140740 1 --2- 172.21.15.16:0/3193669047 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5616a14694a0 0x5616a146fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:21.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.550+0000 7fb87d140740 1 -- 172.21.15.16:0/3193669047 >> 172.21.15.16:0/3193669047 conn(0x5616a1465a30 msgr2=0x5616a1464a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:21.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.550+0000 7fb87d140740 1 -- 172.21.15.16:0/3193669047 shutdown_connections 2024-09-12T02:16:21.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.550+0000 7fb87d140740 1 -- 172.21.15.16:0/3193669047 wait complete. 2024-09-12T02:16:21.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.550+0000 7fb87d140740 1 -- start start 2024-09-12T02:16:21.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.550+0000 7fb87d140740 1 Processor -- start 2024-09-12T02:16:21.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.551+0000 7fb87d140740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5616a14694a0 0x5616a147cc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:21.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.551+0000 7fb87d140740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616a1469dc0 0x5616a147a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:21.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.551+0000 7fb87d140740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5616a1474680 0x5616a147a830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:21.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.551+0000 7fb87d140740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb8741adbc0 con 0x5616a1469dc0 2024-09-12T02:16:21.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.551+0000 7fb87d140740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb8741af030 con 0x5616a14694a0 2024-09-12T02:16:21.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.551+0000 7fb87d140740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb8741ae290 con 0x5616a1474680 2024-09-12T02:16:21.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.551+0000 7fb87d140740 10 client.0 _ll_register_callbacks cb 0x7fb86c007e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:16:21.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.551+0000 7fb873fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616a1469dc0 0x5616a147a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:21.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.551+0000 7fb879304640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5616a1474680 0x5616a147a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:21.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.551+0000 7fb873fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616a1469dc0 0x5616a147a0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40140/0 (socket says 172.21.15.16:40140) 2024-09-12T02:16:21.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.551+0000 7fb873fff640 1 -- 172.21.15.16:0/2572464527 learned_addr learned my addr 172.21.15.16:0/2572464527 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:16:21.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.551+0000 7fb878b03640 1 --2- 172.21.15.16:0/2572464527 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5616a14694a0 0x5616a147cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:21.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.551+0000 7fb873fff640 1 -- 172.21.15.16:0/2572464527 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5616a1474680 msgr2=0x5616a147a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:21.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.551+0000 7fb873fff640 1 --2- 172.21.15.16:0/2572464527 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5616a1474680 0x5616a147a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:21.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.551+0000 7fb873fff640 1 -- 172.21.15.16:0/2572464527 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5616a14694a0 msgr2=0x5616a147cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:21.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.551+0000 7fb873fff640 1 --2- 172.21.15.16:0/2572464527 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5616a14694a0 0x5616a147cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:21.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.551+0000 7fb873fff640 1 -- 172.21.15.16:0/2572464527 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5616a14ed700 con 0x5616a1469dc0 2024-09-12T02:16:21.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.551+0000 7fb873fff640 1 --2- 172.21.15.16:0/2572464527 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616a1469dc0 0x5616a147a0c0 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7fb86000bd10 tx=0x7fb8600054d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc05cf4e52c6ecd7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:21.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.551+0000 7fb853fff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:16:21.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.552+0000 7fb853fff640 1 -- 172.21.15.16:0/2572464527 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb860012650 con 0x5616a1469dc0 2024-09-12T02:16:21.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.552+0000 7fb87d140740 1 -- 172.21.15.16:0/2572464527 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb86c011e30 con 0x5616a1469dc0 2024-09-12T02:16:21.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.552+0000 7fb853fff640 1 -- 172.21.15.16:0/2572464527 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fb8600127f0 con 0x5616a1469dc0 2024-09-12T02:16:21.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.552+0000 7fb853fff640 1 -- 172.21.15.16:0/2572464527 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fb8600129f0 con 0x5616a1469dc0 2024-09-12T02:16:21.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.552+0000 7fb87d140740 10 client.20184 fetch_fsmap learned FSMap version 25 2024-09-12T02:16:21.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.552+0000 7fb87d140740 1 -- 172.21.15.16:0/2572464527 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5616a14ee910 con 0x5616a1469dc0 2024-09-12T02:16:21.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.552+0000 7fb853fff640 1 -- 172.21.15.16:0/2572464527 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb86c011e30 con 0x5616a1469dc0 2024-09-12T02:16:21.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.552+0000 7fb853fff640 1 -- 172.21.15.16:0/2572464527 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fb86001bae0 con 0x5616a1469dc0 2024-09-12T02:16:21.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.552+0000 7fb87d140740 10 client.20184 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:16:21.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.552+0000 7fb87d140740 10 client.20184 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:16:21.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.552+0000 7fb87d140740 20 client.20184 populate_metadata read hostname 'smithi016' 2024-09-12T02:16:21.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.552+0000 7fb87d140740 1 --2- 172.21.15.16:0/2572464527 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5616a14ef0c0 0x5616a14f14e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:21.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.552+0000 7fb87d140740 4 client.20184 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T02:16:21.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.552+0000 7fb87d140740 1 -- 172.21.15.16:0/2572464527 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5616a1353210 con 0x5616a14ef0c0 2024-09-12T02:16:21.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.553+0000 7fb878b03640 1 --2- 172.21.15.16:0/2572464527 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5616a14ef0c0 0x5616a14f14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:21.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.553+0000 7fb878b03640 1 --2- 172.21.15.16:0/2572464527 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5616a14ef0c0 0x5616a14f14e0 crc :-1 s=READY pgs=426 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0c03239b14557bc server_cookie=499f4d76ccb159de in_seq=0 out_seq=0 2024-09-12T02:16:21.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.553+0000 7fb853fff640 10 client.20184 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:16:21.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.553+0000 7fb853fff640 1 -- 172.21.15.16:0/2572464527 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5616a1353210 con 0x5616a14ef0c0 2024-09-12T02:16:21.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.553+0000 7fb853fff640 10 client.20184 handle_command_reply: tid=0 2024-09-12T02:16:21.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.553+0000 7fb87d140740 1 client.20184 shutdown 2024-09-12T02:16:21.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.553+0000 7fb87d140740 1 -- 172.21.15.16:0/2572464527 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616a1469dc0 msgr2=0x5616a147a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:21.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.553+0000 7fb87d140740 1 --2- 172.21.15.16:0/2572464527 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616a1469dc0 0x5616a147a0c0 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7fb86000bd10 tx=0x7fb8600054d0 comp rx=0 tx=0).stop 2024-09-12T02:16:21.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.553+0000 7fb87d140740 1 -- 172.21.15.16:0/2572464527 shutdown_connections 2024-09-12T02:16:21.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.553+0000 7fb87d140740 1 --2- 172.21.15.16:0/2572464527 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5616a14ef0c0 0x5616a14f14e0 crc :-1 s=READY pgs=426 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:21.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.553+0000 7fb87d140740 1 --2- 172.21.15.16:0/2572464527 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5616a1474680 0x5616a147a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:21.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.553+0000 7fb87d140740 1 --2- 172.21.15.16:0/2572464527 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616a1469dc0 0x5616a147a0c0 unknown :-1 s=CLOSED pgs=1917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:21.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.553+0000 7fb853fff640 0 client.20184 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:16:21.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.553+0000 7fb87d140740 1 --2- 172.21.15.16:0/2572464527 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5616a14694a0 0x5616a147cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:21.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.553+0000 7fb87d140740 1 -- 172.21.15.16:0/2572464527 >> 172.21.15.16:0/2572464527 conn(0x5616a1465a30 msgr2=0x5616a1464220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:21.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.553+0000 7fb87d140740 1 -- 172.21.15.16:0/2572464527 shutdown_connections 2024-09-12T02:16:21.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.553+0000 7fb87d140740 1 -- 172.21.15.16:0/2572464527 wait complete. 2024-09-12T02:16:21.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.554+0000 7fb87d140740 20 client.20184 trim_cache size 0 max 16384 2024-09-12T02:16:21.569 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T02:16:21.569 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-09-12T02:16:21.569 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {} 2024-09-12T02:16:21.569 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T02:16:21.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.554+0000 7fb87ad67640 1 -- 172.21.15.16:0/1334297762 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fb8640a2c00 msgr2=0x7fb8640a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:21.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.554+0000 7fb87ad67640 1 --2- 172.21.15.16:0/1334297762 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fb8640a2c00 0x7fb8640a50c0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fb86c007fd0 tx=0x7fb86c007520 comp rx=0 tx=0).stop 2024-09-12T02:16:21.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.554+0000 7fb87ad67640 1 -- 172.21.15.16:0/1334297762 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb874101de0 msgr2=0x7fb8740744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:21.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.554+0000 7fb87ad67640 1 --2- 172.21.15.16:0/1334297762 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb874101de0 0x7fb8740744f0 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7fb868006010 tx=0x7fb8680042f0 comp rx=0 tx=0).stop 2024-09-12T02:16:21.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.554+0000 7fb87ad67640 1 -- 172.21.15.16:0/1334297762 shutdown_connections 2024-09-12T02:16:21.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.554+0000 7fb87ad67640 1 --2- 172.21.15.16:0/1334297762 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fb8640a2c00 0x7fb8640a50c0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:21.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.554+0000 7fb87ad67640 1 --2- 172.21.15.16:0/1334297762 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb874106f10 0x7fb874074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:21.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.554+0000 7fb87ad67640 1 --2- 172.21.15.16:0/1334297762 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb874102790 0x7fb874074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:21.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.554+0000 7fb87ad67640 1 --2- 172.21.15.16:0/1334297762 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb874101de0 0x7fb8740744f0 unknown :-1 s=CLOSED pgs=1913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:21.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.554+0000 7fb87ad67640 1 -- 172.21.15.16:0/1334297762 >> 172.21.15.16:0/1334297762 conn(0x7fb8740fd8c0 msgr2=0x7fb8740febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:21.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.554+0000 7fb87ad67640 1 -- 172.21.15.16:0/1334297762 shutdown_connections 2024-09-12T02:16:21.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:21.554+0000 7fb87ad67640 1 -- 172.21.15.16:0/1334297762 wait complete. 2024-09-12T02:16:22.345 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-12T02:16:22.346 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:16:22.605 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:16:22.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:16:22 smithi184 ceph-mon[35593]: pgmap v768: 97 pgs: 97 active+clean; 3.2 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-12T02:16:22.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:16:22 smithi033 ceph-mon[35332]: pgmap v768: 97 pgs: 97 active+clean; 3.2 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-12T02:16:22.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:16:22 smithi016 ceph-mon[30947]: pgmap v768: 97 pgs: 97 active+clean; 3.2 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-12T02:16:23.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:23.108+0000 7f7edf419640 1 -- 172.21.15.16:0/1289090975 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ed8101de0 msgr2=0x7f7ed81021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:23.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:23.108+0000 7f7edf419640 1 --2- 172.21.15.16:0/1289090975 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ed8101de0 0x7f7ed81021c0 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 crypto rx=0x7f7ecc009900 tx=0x7f7ecc02a410 comp rx=0 tx=0).stop 2024-09-12T02:16:23.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:23.108+0000 7f7edf419640 1 -- 172.21.15.16:0/1289090975 shutdown_connections 2024-09-12T02:16:23.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:23.108+0000 7f7edf419640 1 --2- 172.21.15.16:0/1289090975 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7ed8106f10 0x7f7ed810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:23.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:23.108+0000 7f7edf419640 1 --2- 172.21.15.16:0/1289090975 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7ed8102790 0x7f7ed81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:23.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:23.108+0000 7f7edf419640 1 --2- 172.21.15.16:0/1289090975 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ed8101de0 0x7f7ed81021c0 unknown :-1 s=CLOSED pgs=1918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:23.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:23.108+0000 7f7edf419640 1 -- 172.21.15.16:0/1289090975 >> 172.21.15.16:0/1289090975 conn(0x7f7ed80fd8c0 msgr2=0x7f7ed80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:23.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:23.108+0000 7f7edf419640 1 -- 172.21.15.16:0/1289090975 shutdown_connections 2024-09-12T02:16:23.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:23.108+0000 7f7edf419640 1 -- 172.21.15.16:0/1289090975 wait complete. 2024-09-12T02:16:23.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:23.109+0000 7f7edf419640 1 Processor -- start 2024-09-12T02:16:23.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:23.109+0000 7f7edf419640 1 -- start start 2024-09-12T02:16:23.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:23.109+0000 7f7edf419640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7ed8101de0 0x7f7ed8198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:23.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:23.109+0000 7f7edf419640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ed8102790 0x7f7ed8198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:23.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:23.109+0000 7f7edf419640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7ed8106f10 0x7f7ed819cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:23.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:23.109+0000 7f7edf419640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7ed8111f30 con 0x7f7ed8102790 2024-09-12T02:16:23.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:23.109+0000 7f7edf419640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7ed8111db0 con 0x7f7ed8101de0 2024-09-12T02:16:23.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:23.109+0000 7f7edf419640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7ed81120b0 con 0x7f7ed8106f10 2024-09-12T02:16:23.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:23.109+0000 7f7edc9b4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ed8102790 0x7f7ed8198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:23.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:23.109+0000 7f7edc9b4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ed8102790 0x7f7ed8198980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40162/0 (socket says 172.21.15.16:40162) 2024-09-12T02:16:23.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:23.109+0000 7f7edc9b4640 1 -- 172.21.15.16:0/2559641550 learned_addr learned my addr 172.21.15.16:0/2559641550 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:16:23.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:23.109+0000 7f7edd1b5640 1 --2- 172.21.15.16:0/2559641550 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7ed8101de0 0x7f7ed8198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:23.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:23.110+0000 7f7edc9b4640 1 -- 172.21.15.16:0/2559641550 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7ed8106f10 msgr2=0x7f7ed819cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:23.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:23.110+0000 7f7edd9b6640 1 --2- 172.21.15.16:0/2559641550 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7ed8106f10 0x7f7ed819cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:23.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:23.110+0000 7f7edc9b4640 1 --2- 172.21.15.16:0/2559641550 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7ed8106f10 0x7f7ed819cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:23.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:23.110+0000 7f7edc9b4640 1 -- 172.21.15.16:0/2559641550 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7ed8101de0 msgr2=0x7f7ed8198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:23.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:23.110+0000 7f7edc9b4640 1 --2- 172.21.15.16:0/2559641550 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7ed8101de0 0x7f7ed8198440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:23.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:23.110+0000 7f7edc9b4640 1 -- 172.21.15.16:0/2559641550 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ed819d3f0 con 0x7f7ed8102790 2024-09-12T02:16:23.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:23.110+0000 7f7edd1b5640 1 --2- 172.21.15.16:0/2559641550 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7ed8101de0 0x7f7ed8198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:16:23.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:23.110+0000 7f7edd9b6640 1 --2- 172.21.15.16:0/2559641550 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7ed8106f10 0x7f7ed819cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:16:23.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:23.110+0000 7f7edc9b4640 1 --2- 172.21.15.16:0/2559641550 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ed8102790 0x7f7ed8198980 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x7f7ed400c890 tx=0x7f7ed400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:23.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:23.110+0000 7f7eca7fc640 1 -- 172.21.15.16:0/2559641550 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ed4004220 con 0x7f7ed8102790 2024-09-12T02:16:23.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:23.110+0000 7f7eca7fc640 1 -- 172.21.15.16:0/2559641550 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ed40043c0 con 0x7f7ed8102790 2024-09-12T02:16:23.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:23.110+0000 7f7edf419640 1 -- 172.21.15.16:0/2559641550 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ed819d6e0 con 0x7f7ed8102790 2024-09-12T02:16:23.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:23.110+0000 7f7eca7fc640 1 -- 172.21.15.16:0/2559641550 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ed40049e0 con 0x7f7ed8102790 2024-09-12T02:16:23.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:23.110+0000 7f7edf419640 1 -- 172.21.15.16:0/2559641550 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ed81a4fc0 con 0x7f7ed8102790 2024-09-12T02:16:23.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:23.111+0000 7f7edf419640 1 -- 172.21.15.16:0/2559641550 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ed81021c0 con 0x7f7ed8102790 2024-09-12T02:16:23.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:23.112+0000 7f7eca7fc640 1 -- 172.21.15.16:0/2559641550 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f7ed40079b0 con 0x7f7ed8102790 2024-09-12T02:16:23.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:23.113+0000 7f7eca7fc640 1 --2- 172.21.15.16:0/2559641550 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f7ec40a2c00 0x7f7ec40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:23.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:23.113+0000 7f7edd1b5640 1 --2- 172.21.15.16:0/2559641550 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f7ec40a2c00 0x7f7ec40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:23.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:23.113+0000 7f7eca7fc640 1 -- 172.21.15.16:0/2559641550 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f7ed40cf070 con 0x7f7ed8102790 2024-09-12T02:16:23.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:23.113+0000 7f7edd1b5640 1 --2- 172.21.15.16:0/2559641550 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f7ec40a2c00 0x7f7ec40a50c0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f7ecc005ab0 tx=0x7f7ecc005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:23.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:23.117+0000 7f7eca7fc640 1 -- 172.21.15.16:0/2559641550 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7ed4014030 con 0x7f7ed8102790 2024-09-12T02:16:23.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:23.329+0000 7f7edf419640 1 -- 172.21.15.16:0/2559641550 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7ed8109b60 con 0x7f7ed8102790 2024-09-12T02:16:23.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:23.330+0000 7f7eca7fc640 1 -- 172.21.15.16:0/2559641550 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11911 (secure 0 0 0) 0x7f7ed409c070 con 0x7f7ed8102790 2024-09-12T02:16:23.330 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T02:16:23.330 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":25,"btime":"2024-09-12T02:01:37:192021+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3279123279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3279123279},{"type":"v1","addr":"172.21.15.16:6835","nonce":3279123279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4119452382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4119452382},{"type":"v1","addr":"172.21.15.16:6837","nonce":4119452382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/3263823165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3263823165},{"type":"v1","addr":"172.21.15.16:6839","nonce":3263823165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/1510290732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":1510290732},{"type":"v1","addr":"172.21.15.16:6841","nonce":1510290732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/279674603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":279674603},{"type":"v1","addr":"172.21.15.33:6833","nonce":279674603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3555728028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3555728028},{"type":"v1","addr":"172.21.15.33:6835","nonce":3555728028}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T01:59:56.117608+0000","modified":"2024-09-12T02:01:36.294760+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24482,"mds_1":24481,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6833/1563374106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1563374106},{"type":"v1","addr":"172.21.15.184:6833","nonce":1563374106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6837/1892833868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1892833868},{"type":"v1","addr":"172.21.15.33:6837","nonce":1892833868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/1457679686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1457679686},{"type":"v1","addr":"172.21.15.184:6835","nonce":1457679686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.184:6837/1793393087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":1793393087},{"type":"v1","addr":"172.21.15.184:6837","nonce":1793393087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.184:6839/4203347941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":4203347941},{"type":"v1","addr":"172.21.15.184:6839","nonce":4203347941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.33:6839/4136670204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":4136670204},{"type":"v1","addr":"172.21.15.33:6839","nonce":4136670204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24481,24482,24469]},"id":1}]} 2024-09-12T02:16:23.331 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 25 2024-09-12T02:16:23.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:23.332+0000 7f7edf419640 1 -- 172.21.15.16:0/2559641550 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f7ec40a2c00 msgr2=0x7f7ec40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:23.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:23.332+0000 7f7edf419640 1 --2- 172.21.15.16:0/2559641550 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f7ec40a2c00 0x7f7ec40a50c0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f7ecc005ab0 tx=0x7f7ecc005a20 comp rx=0 tx=0).stop 2024-09-12T02:16:23.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:23.332+0000 7f7edf419640 1 -- 172.21.15.16:0/2559641550 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ed8102790 msgr2=0x7f7ed8198980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:23.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:23.332+0000 7f7edf419640 1 --2- 172.21.15.16:0/2559641550 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ed8102790 0x7f7ed8198980 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x7f7ed400c890 tx=0x7f7ed400cd50 comp rx=0 tx=0).stop 2024-09-12T02:16:23.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:23.332+0000 7f7edf419640 1 -- 172.21.15.16:0/2559641550 shutdown_connections 2024-09-12T02:16:23.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:23.332+0000 7f7edf419640 1 --2- 172.21.15.16:0/2559641550 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f7ec40a2c00 0x7f7ec40a50c0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:23.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:23.332+0000 7f7edf419640 1 --2- 172.21.15.16:0/2559641550 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7ed8106f10 0x7f7ed819cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:23.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:23.332+0000 7f7edf419640 1 --2- 172.21.15.16:0/2559641550 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ed8102790 0x7f7ed8198980 unknown :-1 s=CLOSED pgs=1919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:23.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:23.332+0000 7f7edf419640 1 --2- 172.21.15.16:0/2559641550 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7ed8101de0 0x7f7ed8198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:23.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:23.332+0000 7f7edf419640 1 -- 172.21.15.16:0/2559641550 >> 172.21.15.16:0/2559641550 conn(0x7f7ed80fd8c0 msgr2=0x7f7ed80fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:23.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:23.332+0000 7f7edf419640 1 -- 172.21.15.16:0/2559641550 shutdown_connections 2024-09-12T02:16:23.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:23.332+0000 7f7edf419640 1 -- 172.21.15.16:0/2559641550 wait complete. 2024-09-12T02:16:23.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:16:23 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/2559641550' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:16:23.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:16:23 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/2559641550' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:16:23.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:16:23 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/2559641550' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:16:24.134 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-12T02:16:24.398 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:16:24.645 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:16:24 smithi016 ceph-mon[30947]: pgmap v769: 97 pgs: 97 active+clean; 3.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 14 op/s 2024-09-12T02:16:24.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:16:24 smithi033 ceph-mon[35332]: pgmap v769: 97 pgs: 97 active+clean; 3.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 14 op/s 2024-09-12T02:16:24.778 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:16:24 smithi184 ceph-mon[35593]: pgmap v769: 97 pgs: 97 active+clean; 3.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 14 op/s 2024-09-12T02:16:24.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.852+0000 7fbafe774640 1 -- 172.21.15.16:0/1966423946 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbaf8106f10 msgr2=0x7fbaf810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:24.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.852+0000 7fbafe774640 1 --2- 172.21.15.16:0/1966423946 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbaf8106f10 0x7fbaf810ea90 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7fbaf0009ca0 tx=0x7fbaf002a8c0 comp rx=0 tx=0).stop 2024-09-12T02:16:24.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.852+0000 7fbafe774640 1 -- 172.21.15.16:0/1966423946 shutdown_connections 2024-09-12T02:16:24.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.852+0000 7fbafe774640 1 --2- 172.21.15.16:0/1966423946 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbaf8106f10 0x7fbaf810ea90 unknown :-1 s=CLOSED pgs=1920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:24.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.852+0000 7fbafe774640 1 --2- 172.21.15.16:0/1966423946 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbaf8102790 0x7fbaf81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:24.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.852+0000 7fbafe774640 1 --2- 172.21.15.16:0/1966423946 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbaf8101de0 0x7fbaf81021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:24.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.852+0000 7fbafe774640 1 -- 172.21.15.16:0/1966423946 >> 172.21.15.16:0/1966423946 conn(0x7fbaf80fd8c0 msgr2=0x7fbaf80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:24.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.852+0000 7fbafe774640 1 -- 172.21.15.16:0/1966423946 shutdown_connections 2024-09-12T02:16:24.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.852+0000 7fbafe774640 1 -- 172.21.15.16:0/1966423946 wait complete. 2024-09-12T02:16:24.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.852+0000 7fbafe774640 1 Processor -- start 2024-09-12T02:16:24.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.853+0000 7fbafe774640 1 -- start start 2024-09-12T02:16:24.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.853+0000 7fbafe774640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbaf8101de0 0x7fbaf81983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:24.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.853+0000 7fbafe774640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbaf8102790 0x7fbaf8198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:24.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.853+0000 7fbafe774640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbaf8106f10 0x7fbaf819ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:24.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.853+0000 7fbafe774640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbaf8111f70 con 0x7fbaf8102790 2024-09-12T02:16:24.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.853+0000 7fbafe774640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbaf8111df0 con 0x7fbaf8106f10 2024-09-12T02:16:24.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.853+0000 7fbafe774640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fbaf81120f0 con 0x7fbaf8101de0 2024-09-12T02:16:24.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.853+0000 7fbaf77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbaf8102790 0x7fbaf8198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:24.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.853+0000 7fbaf77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbaf8102790 0x7fbaf8198930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40178/0 (socket says 172.21.15.16:40178) 2024-09-12T02:16:24.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.853+0000 7fbaf77fe640 1 -- 172.21.15.16:0/2099594934 learned_addr learned my addr 172.21.15.16:0/2099594934 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:16:24.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.853+0000 7fbafcd11640 1 --2- 172.21.15.16:0/2099594934 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbaf8106f10 0x7fbaf819ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:24.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.853+0000 7fbaf7fff640 1 --2- 172.21.15.16:0/2099594934 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbaf8101de0 0x7fbaf81983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:24.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.853+0000 7fbaf77fe640 1 -- 172.21.15.16:0/2099594934 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbaf8101de0 msgr2=0x7fbaf81983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:24.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.853+0000 7fbaf77fe640 1 --2- 172.21.15.16:0/2099594934 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbaf8101de0 0x7fbaf81983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:24.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.853+0000 7fbaf77fe640 1 -- 172.21.15.16:0/2099594934 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbaf8106f10 msgr2=0x7fbaf819ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:24.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.853+0000 7fbaf77fe640 1 --2- 172.21.15.16:0/2099594934 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbaf8106f10 0x7fbaf819ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:24.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.853+0000 7fbaf77fe640 1 -- 172.21.15.16:0/2099594934 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbaf819d3a0 con 0x7fbaf8102790 2024-09-12T02:16:24.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.853+0000 7fbaf7fff640 1 --2- 172.21.15.16:0/2099594934 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbaf8101de0 0x7fbaf81983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:16:24.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.854+0000 7fbaf77fe640 1 --2- 172.21.15.16:0/2099594934 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbaf8102790 0x7fbaf8198930 secure :-1 s=READY pgs=1921 cs=0 l=1 rev1=1 crypto rx=0x7fbaec00d770 tx=0x7fbaec00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:24.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.854+0000 7fbaf57fa640 1 -- 172.21.15.16:0/2099594934 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaec017070 con 0x7fbaf8102790 2024-09-12T02:16:24.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.854+0000 7fbaf57fa640 1 -- 172.21.15.16:0/2099594934 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbaec005500 con 0x7fbaf8102790 2024-09-12T02:16:24.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.854+0000 7fbaf57fa640 1 -- 172.21.15.16:0/2099594934 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaec004e00 con 0x7fbaf8102790 2024-09-12T02:16:24.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.854+0000 7fbafe774640 1 -- 172.21.15.16:0/2099594934 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbaf819d690 con 0x7fbaf8102790 2024-09-12T02:16:24.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.854+0000 7fbafe774640 1 -- 172.21.15.16:0/2099594934 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbaf81a4f70 con 0x7fbaf8102790 2024-09-12T02:16:24.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.856+0000 7fbaf57fa640 1 -- 172.21.15.16:0/2099594934 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fbaec004fa0 con 0x7fbaf8102790 2024-09-12T02:16:24.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.857+0000 7fbb00b4d740 1 Processor -- start 2024-09-12T02:16:24.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.857+0000 7fbb00b4d740 1 -- start start 2024-09-12T02:16:24.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.857+0000 7fbb00b4d740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55db886b6b40 0x55db886b5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:24.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.857+0000 7fbb00b4d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db886b55d0 0x55db886b5a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:24.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.857+0000 7fbb00b4d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55db886c0370 0x55db886c2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:24.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.857+0000 7fbb00b4d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55db8861b2f0 con 0x55db886b55d0 2024-09-12T02:16:24.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.857+0000 7fbb00b4d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55db885c2cf0 con 0x55db886c0370 2024-09-12T02:16:24.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.857+0000 7fbb00b4d740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55db886172b0 con 0x55db886b6b40 2024-09-12T02:16:24.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.857+0000 7fbaf7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db886b55d0 0x55db886b5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:24.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.857+0000 7fbaf7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db886b55d0 0x55db886b5a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40188/0 (socket says 172.21.15.16:40188) 2024-09-12T02:16:24.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.857+0000 7fbaf7fff640 1 -- 172.21.15.16:0/3295563058 learned_addr learned my addr 172.21.15.16:0/3295563058 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:16:24.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.857+0000 7fbafcd11640 1 --2- 172.21.15.16:0/3295563058 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55db886b6b40 0x55db886b5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:24.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.857+0000 7fbaf77fe640 1 --2- 172.21.15.16:0/3295563058 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55db886c0370 0x55db886c2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:24.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.857+0000 7fbaf57fa640 1 --2- 172.21.15.16:0/2099594934 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fbae00a2c00 0x7fbae00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:24.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.857+0000 7fbaf7fff640 1 -- 172.21.15.16:0/3295563058 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55db886b6b40 msgr2=0x55db886b5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:24.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.857+0000 7fbaf7fff640 1 --2- 172.21.15.16:0/3295563058 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55db886b6b40 0x55db886b5090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:24.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.858+0000 7fbaf7fff640 1 -- 172.21.15.16:0/3295563058 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55db886c0370 msgr2=0x55db886c2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:24.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.858+0000 7fbaf7fff640 1 --2- 172.21.15.16:0/3295563058 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55db886c0370 0x55db886c2760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:24.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.858+0000 7fbaf7fff640 1 -- 172.21.15.16:0/3295563058 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db885a73a0 con 0x55db886b55d0 2024-09-12T02:16:24.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.858+0000 7fbaf77fe640 1 --2- 172.21.15.16:0/3295563058 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55db886c0370 0x55db886c2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:16:24.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.858+0000 7fbafcd11640 1 --2- 172.21.15.16:0/3295563058 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55db886b6b40 0x55db886b5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:16:24.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.858+0000 7fbaf57fa640 1 -- 172.21.15.16:0/2099594934 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fbaec0ce9e0 con 0x7fbaf8102790 2024-09-12T02:16:24.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.858+0000 7fbafcd11640 1 --2- 172.21.15.16:0/2099594934 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fbae00a2c00 0x7fbae00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:24.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.858+0000 7fbaf7fff640 1 --2- 172.21.15.16:0/3295563058 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db886b55d0 0x55db886b5a10 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7fbae800cca0 tx=0x7fbae800e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73fb355a92ec7adf server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:24.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.858+0000 7fbade7fc640 1 -- 172.21.15.16:0/3295563058 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbae8005220 con 0x55db886b55d0 2024-09-12T02:16:24.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.858+0000 7fbade7fc640 1 -- 172.21.15.16:0/3295563058 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbae8004c80 con 0x55db886b55d0 2024-09-12T02:16:24.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.858+0000 7fbade7fc640 1 -- 172.21.15.16:0/3295563058 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbae8010800 con 0x55db886b55d0 2024-09-12T02:16:24.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.859+0000 7fbb00b4d740 1 -- 172.21.15.16:0/3295563058 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db886b55d0 msgr2=0x55db886b5a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:24.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.859+0000 7fbafcd11640 1 --2- 172.21.15.16:0/2099594934 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fbae00a2c00 0x7fbae00a50c0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fbaf0005e30 tx=0x7fbaf003f040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:24.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.859+0000 7fbb00b4d740 1 --2- 172.21.15.16:0/3295563058 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db886b55d0 0x55db886b5a10 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7fbae800cca0 tx=0x7fbae800e720 comp rx=0 tx=0).stop 2024-09-12T02:16:24.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.859+0000 7fbb00b4d740 1 -- 172.21.15.16:0/3295563058 shutdown_connections 2024-09-12T02:16:24.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.859+0000 7fbb00b4d740 1 --2- 172.21.15.16:0/3295563058 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55db886c0370 0x55db886c2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:24.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.859+0000 7fbb00b4d740 1 --2- 172.21.15.16:0/3295563058 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db886b55d0 0x55db886b5a10 unknown :-1 s=CLOSED pgs=1922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:24.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.859+0000 7fbb00b4d740 1 --2- 172.21.15.16:0/3295563058 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55db886b6b40 0x55db886b5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:24.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.859+0000 7fbb00b4d740 1 -- 172.21.15.16:0/3295563058 >> 172.21.15.16:0/3295563058 conn(0x55db8867e310 msgr2=0x55db886b1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:24.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.859+0000 7fbb00b4d740 1 -- 172.21.15.16:0/3295563058 shutdown_connections 2024-09-12T02:16:24.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.859+0000 7fbb00b4d740 1 -- 172.21.15.16:0/3295563058 wait complete. 2024-09-12T02:16:24.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.859+0000 7fbb00b4d740 1 -- start start 2024-09-12T02:16:24.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.859+0000 7fbb00b4d740 1 Processor -- start 2024-09-12T02:16:24.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.860+0000 7fbb00b4d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db886b55d0 0x55db886bfcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:24.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.860+0000 7fbb00b4d740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55db886b6b40 0x55db886bcf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:24.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.860+0000 7fbb00b4d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55db886c0370 0x55db886bd4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:24.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.860+0000 7fbb00b4d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55db886c3070 con 0x55db886b55d0 2024-09-12T02:16:24.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.860+0000 7fbb00b4d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55db885d6060 con 0x55db886c0370 2024-09-12T02:16:24.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.860+0000 7fbb00b4d740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55db886c31f0 con 0x55db886b6b40 2024-09-12T02:16:24.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.860+0000 7fbb00b4d740 10 client.0 _ll_register_callbacks cb 0x55db883298c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:16:24.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.860+0000 7fbaf7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db886b55d0 0x55db886bfcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:24.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.860+0000 7fbaf7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db886b55d0 0x55db886bfcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40200/0 (socket says 172.21.15.16:40200) 2024-09-12T02:16:24.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.860+0000 7fbaf7fff640 1 -- 172.21.15.16:0/3742910094 learned_addr learned my addr 172.21.15.16:0/3742910094 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:16:24.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.860+0000 7fbafcd11640 1 --2- 172.21.15.16:0/3742910094 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55db886c0370 0x55db886bd4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:24.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.860+0000 7fbaf77fe640 1 --2- 172.21.15.16:0/3742910094 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55db886b6b40 0x55db886bcf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:24.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.860+0000 7fbaf7fff640 1 -- 172.21.15.16:0/3742910094 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55db886b6b40 msgr2=0x55db886bcf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:24.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.860+0000 7fbaf7fff640 1 --2- 172.21.15.16:0/3742910094 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55db886b6b40 0x55db886bcf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:24.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.860+0000 7fbaf7fff640 1 -- 172.21.15.16:0/3742910094 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55db886c0370 msgr2=0x55db886bd4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:24.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.860+0000 7fbaf7fff640 1 --2- 172.21.15.16:0/3742910094 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55db886c0370 0x55db886bd4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:24.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.860+0000 7fbaf7fff640 1 -- 172.21.15.16:0/3742910094 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db8838c3c0 con 0x55db886b55d0 2024-09-12T02:16:24.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.861+0000 7fbaf7fff640 1 --2- 172.21.15.16:0/3742910094 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db886b55d0 0x55db886bfcd0 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7fbae800ec20 tx=0x7fbae800c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d460c53ae90a717 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:24.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.861+0000 7fbad77fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:16:24.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.861+0000 7fbad77fe640 1 -- 172.21.15.16:0/3742910094 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbae802a060 con 0x55db886b55d0 2024-09-12T02:16:24.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.861+0000 7fbad77fe640 1 -- 172.21.15.16:0/3742910094 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fbae80048d0 con 0x55db886b55d0 2024-09-12T02:16:24.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.861+0000 7fbb00b4d740 1 -- 172.21.15.16:0/3742910094 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbae8004c80 con 0x55db886b55d0 2024-09-12T02:16:24.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.861+0000 7fbad77fe640 1 -- 172.21.15.16:0/3742910094 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fbae8004ad0 con 0x55db886b55d0 2024-09-12T02:16:24.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.861+0000 7fbb00b4d740 10 client.20202 fetch_fsmap learned FSMap version 25 2024-09-12T02:16:24.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.861+0000 7fbb00b4d740 1 -- 172.21.15.16:0/3742910094 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55db886cea20 con 0x55db886b55d0 2024-09-12T02:16:24.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.861+0000 7fbad77fe640 1 -- 172.21.15.16:0/3742910094 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbae8004c80 con 0x55db886b55d0 2024-09-12T02:16:24.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.861+0000 7fbad77fe640 1 -- 172.21.15.16:0/3742910094 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fbae802f070 con 0x55db886b55d0 2024-09-12T02:16:24.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.861+0000 7fbb00b4d740 10 client.20202 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:16:24.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.861+0000 7fbb00b4d740 10 client.20202 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-12T02:16:24.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.861+0000 7fbb00b4d740 20 client.20202 populate_metadata read hostname 'smithi016' 2024-09-12T02:16:24.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.861+0000 7fbb00b4d740 1 --2- 172.21.15.16:0/3742910094 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55db886cfe20 0x55db886d2210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:24.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.861+0000 7fbb00b4d740 4 client.20202 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:16:24.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.861+0000 7fbb00b4d740 1 -- 172.21.15.16:0/3742910094 --> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55db88628860 con 0x55db886cfe20 2024-09-12T02:16:24.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.862+0000 7fbaf77fe640 1 --2- 172.21.15.16:0/3742910094 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55db886cfe20 0x55db886d2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:24.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.862+0000 7fbad77fe640 10 client.20202 ms_handle_connect on v2:172.21.15.184:6836/1793393087 2024-09-12T02:16:24.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.862+0000 7fbaf77fe640 1 --2- 172.21.15.16:0/3742910094 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55db886cfe20 0x55db886d2210 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2dc303ee8562e0ae server_cookie=cbd12c37f41ad1be in_seq=0 out_seq=0 2024-09-12T02:16:24.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.863+0000 7fbad77fe640 1 -- 172.21.15.16:0/3742910094 <== mds.2 v2:172.21.15.184:6836/1793393087 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55db88628860 con 0x55db886cfe20 2024-09-12T02:16:24.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.863+0000 7fbad77fe640 10 client.20202 handle_command_reply: tid=0 2024-09-12T02:16:24.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.863+0000 7fbb00b4d740 1 client.20202 shutdown 2024-09-12T02:16:24.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.863+0000 7fbb00b4d740 1 -- 172.21.15.16:0/3742910094 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db886b55d0 msgr2=0x55db886bfcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:24.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.863+0000 7fbb00b4d740 1 --2- 172.21.15.16:0/3742910094 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db886b55d0 0x55db886bfcd0 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7fbae800ec20 tx=0x7fbae800c9e0 comp rx=0 tx=0).stop 2024-09-12T02:16:24.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.863+0000 7fbb00b4d740 1 -- 172.21.15.16:0/3742910094 shutdown_connections 2024-09-12T02:16:24.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.863+0000 7fbb00b4d740 1 --2- 172.21.15.16:0/3742910094 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55db886c0370 0x55db886bd4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:24.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.863+0000 7fbb00b4d740 1 --2- 172.21.15.16:0/3742910094 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55db886b6b40 0x55db886bcf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:24.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.863+0000 7fbb00b4d740 1 --2- 172.21.15.16:0/3742910094 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55db886cfe20 0x55db886d2210 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:24.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.863+0000 7fbb00b4d740 1 --2- 172.21.15.16:0/3742910094 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db886b55d0 0x55db886bfcd0 unknown :-1 s=CLOSED pgs=1923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:24.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.863+0000 7fbad77fe640 0 client.20202 ms_handle_reset on v2:172.21.15.184:6836/1793393087 2024-09-12T02:16:24.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.863+0000 7fbb00b4d740 1 -- 172.21.15.16:0/3742910094 >> 172.21.15.16:0/3742910094 conn(0x55db8867e310 msgr2=0x55db886b7930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:24.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.863+0000 7fbb00b4d740 1 -- 172.21.15.16:0/3742910094 shutdown_connections 2024-09-12T02:16:24.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.863+0000 7fbb00b4d740 1 -- 172.21.15.16:0/3742910094 wait complete. 2024-09-12T02:16:24.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.863+0000 7fbb00b4d740 20 client.20202 trim_cache size 0 max 16384 2024-09-12T02:16:24.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.876+0000 7fbb00b4d740 1 Processor -- start 2024-09-12T02:16:24.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.876+0000 7fbb00b4d740 1 -- start start 2024-09-12T02:16:24.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.876+0000 7fbb00b4d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db886b55d0 0x55db886cc340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:24.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.876+0000 7fbb00b4d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55db886b6b40 0x55db886b1700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:24.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.876+0000 7fbb00b4d740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55db886c0370 0x55db886bbf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:24.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.876+0000 7fbb00b4d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbaf81afd20 con 0x55db886b55d0 2024-09-12T02:16:24.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.876+0000 7fbb00b4d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbaf81ad5f0 con 0x55db886b6b40 2024-09-12T02:16:24.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.876+0000 7fbb00b4d740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fbaf81ab440 con 0x55db886c0370 2024-09-12T02:16:24.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.877+0000 7fbaf7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db886b55d0 0x55db886cc340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:24.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.877+0000 7fbaf7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db886b55d0 0x55db886cc340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40210/0 (socket says 172.21.15.16:40210) 2024-09-12T02:16:24.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.877+0000 7fbaf7fff640 1 -- 172.21.15.16:0/2651444051 learned_addr learned my addr 172.21.15.16:0/2651444051 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:16:24.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.877+0000 7fbaf77fe640 1 --2- 172.21.15.16:0/2651444051 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55db886b6b40 0x55db886b1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:24.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.877+0000 7fbaf7fff640 1 -- 172.21.15.16:0/2651444051 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55db886c0370 msgr2=0x55db886bbf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:24.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.877+0000 7fbaf7fff640 1 --2- 172.21.15.16:0/2651444051 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55db886c0370 0x55db886bbf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:24.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.877+0000 7fbaf7fff640 1 -- 172.21.15.16:0/2651444051 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55db886b6b40 msgr2=0x55db886b1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:24.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.877+0000 7fbaf7fff640 1 --2- 172.21.15.16:0/2651444051 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55db886b6b40 0x55db886b1700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:24.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.877+0000 7fbaf7fff640 1 -- 172.21.15.16:0/2651444051 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db886b1ed0 con 0x55db886b55d0 2024-09-12T02:16:24.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.877+0000 7fbaf7fff640 1 --2- 172.21.15.16:0/2651444051 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db886b55d0 0x55db886cc340 secure :-1 s=READY pgs=1924 cs=0 l=1 rev1=1 crypto rx=0x7fbae800ed50 tx=0x7fbae800c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72349d686fc3b719 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:24.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.877+0000 7fbaddffb640 1 -- 172.21.15.16:0/2651444051 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbae8035020 con 0x55db886b55d0 2024-09-12T02:16:24.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.877+0000 7fbaddffb640 1 -- 172.21.15.16:0/2651444051 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbae8025e50 con 0x55db886b55d0 2024-09-12T02:16:24.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.877+0000 7fbaddffb640 1 -- 172.21.15.16:0/2651444051 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbae8004280 con 0x55db886b55d0 2024-09-12T02:16:24.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.878+0000 7fbb00b4d740 1 -- 172.21.15.16:0/2651444051 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db886b55d0 msgr2=0x55db886cc340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:24.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.878+0000 7fbb00b4d740 1 --2- 172.21.15.16:0/2651444051 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db886b55d0 0x55db886cc340 secure :-1 s=READY pgs=1924 cs=0 l=1 rev1=1 crypto rx=0x7fbae800ed50 tx=0x7fbae800c510 comp rx=0 tx=0).stop 2024-09-12T02:16:24.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.878+0000 7fbb00b4d740 1 -- 172.21.15.16:0/2651444051 shutdown_connections 2024-09-12T02:16:24.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.878+0000 7fbb00b4d740 1 --2- 172.21.15.16:0/2651444051 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55db886c0370 0x55db886bbf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:24.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.878+0000 7fbb00b4d740 1 --2- 172.21.15.16:0/2651444051 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55db886b6b40 0x55db886b1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:24.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.878+0000 7fbb00b4d740 1 --2- 172.21.15.16:0/2651444051 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db886b55d0 0x55db886cc340 unknown :-1 s=CLOSED pgs=1924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:24.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.878+0000 7fbb00b4d740 1 -- 172.21.15.16:0/2651444051 >> 172.21.15.16:0/2651444051 conn(0x55db8867e310 msgr2=0x55db886bb850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:24.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.878+0000 7fbb00b4d740 1 -- 172.21.15.16:0/2651444051 shutdown_connections 2024-09-12T02:16:24.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.878+0000 7fbb00b4d740 1 -- 172.21.15.16:0/2651444051 wait complete. 2024-09-12T02:16:24.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.878+0000 7fbb00b4d740 1 -- start start 2024-09-12T02:16:24.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.878+0000 7fbb00b4d740 1 Processor -- start 2024-09-12T02:16:24.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.879+0000 7fbb00b4d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55db886b55d0 0x55db886c8a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:24.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.879+0000 7fbb00b4d740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55db886b6b40 0x55db886c17d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:24.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.879+0000 7fbb00b4d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db886c0370 0x55db886c1d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:24.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.879+0000 7fbb00b4d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbaf81abab0 con 0x55db886c0370 2024-09-12T02:16:24.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.879+0000 7fbb00b4d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbaf81acf20 con 0x55db886b55d0 2024-09-12T02:16:24.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.879+0000 7fbb00b4d740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fbaf81ac180 con 0x55db886b6b40 2024-09-12T02:16:24.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.879+0000 7fbb00b4d740 10 client.0 _ll_register_callbacks cb 0x7fbae8014070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:16:24.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.879+0000 7fbafcd11640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db886c0370 0x55db886c1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:24.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.879+0000 7fbafcd11640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db886c0370 0x55db886c1d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40220/0 (socket says 172.21.15.16:40220) 2024-09-12T02:16:24.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.879+0000 7fbafcd11640 1 -- 172.21.15.16:0/1019689779 learned_addr learned my addr 172.21.15.16:0/1019689779 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:16:24.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.879+0000 7fbaf7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55db886b55d0 0x55db886c8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:24.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.879+0000 7fbaf77fe640 1 --2- 172.21.15.16:0/1019689779 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55db886b6b40 0x55db886c17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:24.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.879+0000 7fbafcd11640 1 -- 172.21.15.16:0/1019689779 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55db886b6b40 msgr2=0x55db886c17d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:24.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.879+0000 7fbafcd11640 1 --2- 172.21.15.16:0/1019689779 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55db886b6b40 0x55db886c17d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:24.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.879+0000 7fbafcd11640 1 -- 172.21.15.16:0/1019689779 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55db886b55d0 msgr2=0x55db886c8a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:24.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.879+0000 7fbafcd11640 1 --2- 172.21.15.16:0/1019689779 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55db886b55d0 0x55db886c8a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:24.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.879+0000 7fbafcd11640 1 -- 172.21.15.16:0/1019689779 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db886bd850 con 0x55db886c0370 2024-09-12T02:16:24.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.879+0000 7fbaf7fff640 1 --2- 172.21.15.16:0/1019689779 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55db886b55d0 0x55db886c8a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:16:24.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.879+0000 7fbafcd11640 1 --2- 172.21.15.16:0/1019689779 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db886c0370 0x55db886c1d10 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7fbaf002ade0 tx=0x7fbaf00095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d045e165768d7df7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:24.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.879+0000 7fbad77fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:16:24.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.880+0000 7fbad77fe640 1 -- 172.21.15.16:0/1019689779 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaf002f610 con 0x55db886c0370 2024-09-12T02:16:24.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.880+0000 7fbad77fe640 1 -- 172.21.15.16:0/1019689779 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fbaf002f7b0 con 0x55db886c0370 2024-09-12T02:16:24.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.880+0000 7fbb00b4d740 1 -- 172.21.15.16:0/1019689779 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbae8025e50 con 0x55db886c0370 2024-09-12T02:16:24.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.880+0000 7fbad77fe640 1 -- 172.21.15.16:0/1019689779 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fbaf002f9b0 con 0x55db886c0370 2024-09-12T02:16:24.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.880+0000 7fbb00b4d740 10 client.20208 fetch_fsmap learned FSMap version 25 2024-09-12T02:16:24.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.880+0000 7fbb00b4d740 1 -- 172.21.15.16:0/1019689779 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55db886c7390 con 0x55db886c0370 2024-09-12T02:16:24.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.880+0000 7fbad77fe640 1 -- 172.21.15.16:0/1019689779 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbae8025e50 con 0x55db886c0370 2024-09-12T02:16:24.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.880+0000 7fbad77fe640 1 -- 172.21.15.16:0/1019689779 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fbaf00305c0 con 0x55db886c0370 2024-09-12T02:16:24.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.880+0000 7fbb00b4d740 10 client.20208 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:16:24.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.880+0000 7fbb00b4d740 10 client.20208 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-12T02:16:24.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.880+0000 7fbb00b4d740 20 client.20208 populate_metadata read hostname 'smithi016' 2024-09-12T02:16:24.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.880+0000 7fbb00b4d740 1 --2- 172.21.15.16:0/1019689779 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55db8873aca0 0x55db8873d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:24.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.880+0000 7fbb00b4d740 4 client.20208 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T02:16:24.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.880+0000 7fbb00b4d740 1 -- 172.21.15.16:0/1019689779 --> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55db88622d70 con 0x55db8873aca0 2024-09-12T02:16:24.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.881+0000 7fbaf7fff640 1 --2- 172.21.15.16:0/1019689779 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55db8873aca0 0x55db8873d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:24.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.881+0000 7fbad77fe640 10 client.20208 ms_handle_connect on v2:172.21.15.184:6836/1793393087 2024-09-12T02:16:24.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.881+0000 7fbaf7fff640 1 --2- 172.21.15.16:0/1019689779 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55db8873aca0 0x55db8873d0c0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=84abb9aeb9aa4b92 server_cookie=e7a3d524cb7109ca in_seq=0 out_seq=0 2024-09-12T02:16:24.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.881+0000 7fbad77fe640 1 -- 172.21.15.16:0/1019689779 <== mds.2 v2:172.21.15.184:6836/1793393087 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55db88622d70 con 0x55db8873aca0 2024-09-12T02:16:24.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.881+0000 7fbad77fe640 10 client.20208 handle_command_reply: tid=0 2024-09-12T02:16:24.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.881+0000 7fbb00b4d740 1 client.20208 shutdown 2024-09-12T02:16:24.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.881+0000 7fbb00b4d740 1 -- 172.21.15.16:0/1019689779 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db886c0370 msgr2=0x55db886c1d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:24.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.881+0000 7fbb00b4d740 1 --2- 172.21.15.16:0/1019689779 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db886c0370 0x55db886c1d10 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7fbaf002ade0 tx=0x7fbaf00095c0 comp rx=0 tx=0).stop 2024-09-12T02:16:24.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.882+0000 7fbb00b4d740 1 -- 172.21.15.16:0/1019689779 shutdown_connections 2024-09-12T02:16:24.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.882+0000 7fbb00b4d740 1 --2- 172.21.15.16:0/1019689779 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x55db8873aca0 0x55db8873d0c0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:24.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.882+0000 7fbb00b4d740 1 --2- 172.21.15.16:0/1019689779 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db886c0370 0x55db886c1d10 unknown :-1 s=CLOSED pgs=1925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:24.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.882+0000 7fbb00b4d740 1 --2- 172.21.15.16:0/1019689779 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55db886b6b40 0x55db886c17d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:24.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.882+0000 7fbb00b4d740 1 --2- 172.21.15.16:0/1019689779 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55db886b55d0 0x55db886c8a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:24.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.882+0000 7fbb00b4d740 1 -- 172.21.15.16:0/1019689779 >> 172.21.15.16:0/1019689779 conn(0x55db8867e310 msgr2=0x55db886b7110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:24.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.882+0000 7fbad77fe640 0 client.20208 ms_handle_reset on v2:172.21.15.184:6836/1793393087 2024-09-12T02:16:24.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.882+0000 7fbb00b4d740 1 -- 172.21.15.16:0/1019689779 shutdown_connections 2024-09-12T02:16:24.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.882+0000 7fbb00b4d740 1 -- 172.21.15.16:0/1019689779 wait complete. 2024-09-12T02:16:24.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.882+0000 7fbb00b4d740 20 client.20208 trim_cache size 0 max 16384 2024-09-12T02:16:24.890 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T02:16:24.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.882+0000 7fbafe774640 1 -- 172.21.15.16:0/2099594934 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fbae00a2c00 msgr2=0x7fbae00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:24.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.882+0000 7fbafe774640 1 --2- 172.21.15.16:0/2099594934 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fbae00a2c00 0x7fbae00a50c0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fbaf0005e30 tx=0x7fbaf003f040 comp rx=0 tx=0).stop 2024-09-12T02:16:24.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.882+0000 7fbafe774640 1 -- 172.21.15.16:0/2099594934 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbaf8102790 msgr2=0x7fbaf8198930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:24.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.882+0000 7fbafe774640 1 --2- 172.21.15.16:0/2099594934 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbaf8102790 0x7fbaf8198930 secure :-1 s=READY pgs=1921 cs=0 l=1 rev1=1 crypto rx=0x7fbaec00d770 tx=0x7fbaec00dc30 comp rx=0 tx=0).stop 2024-09-12T02:16:24.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.883+0000 7fbafe774640 1 -- 172.21.15.16:0/2099594934 shutdown_connections 2024-09-12T02:16:24.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.883+0000 7fbafe774640 1 --2- 172.21.15.16:0/2099594934 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fbae00a2c00 0x7fbae00a50c0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:24.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.883+0000 7fbafe774640 1 --2- 172.21.15.16:0/2099594934 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbaf8106f10 0x7fbaf819ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:24.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.883+0000 7fbafe774640 1 --2- 172.21.15.16:0/2099594934 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbaf8102790 0x7fbaf8198930 unknown :-1 s=CLOSED pgs=1921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:24.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.883+0000 7fbafe774640 1 --2- 172.21.15.16:0/2099594934 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbaf8101de0 0x7fbaf81983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:24.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.883+0000 7fbafe774640 1 -- 172.21.15.16:0/2099594934 >> 172.21.15.16:0/2099594934 conn(0x7fbaf80fd8c0 msgr2=0x7fbaf80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:24.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.883+0000 7fbafe774640 1 -- 172.21.15.16:0/2099594934 shutdown_connections 2024-09-12T02:16:24.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:24.883+0000 7fbafe774640 1 -- 172.21.15.16:0/2099594934 wait complete. 2024-09-12T02:16:25.071 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.068+0000 7fd80cbe8640 1 -- 172.21.15.16:0/1265389553 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd808101b60 msgr2=0x7fd808101f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:25.071 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.068+0000 7fd80cbe8640 1 --2- 172.21.15.16:0/1265389553 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd808101b60 0x7fd808101f60 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7fd7f8009900 tx=0x7fd7f802a410 comp rx=0 tx=0).stop 2024-09-12T02:16:25.071 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.068+0000 7fd80cbe8640 1 -- 172.21.15.16:0/1265389553 shutdown_connections 2024-09-12T02:16:25.071 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.068+0000 7fd80cbe8640 1 --2- 172.21.15.16:0/1265389553 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd808103710 0x7fd808109fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:25.071 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.068+0000 7fd80cbe8640 1 --2- 172.21.15.16:0/1265389553 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd808102d50 0x7fd8081031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:25.071 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.068+0000 7fd80cbe8640 1 --2- 172.21.15.16:0/1265389553 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd808101b60 0x7fd808101f60 unknown :-1 s=CLOSED pgs=1926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:25.071 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.068+0000 7fd80cbe8640 1 -- 172.21.15.16:0/1265389553 >> 172.21.15.16:0/1265389553 conn(0x7fd8080fd2e0 msgr2=0x7fd8080ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:25.071 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.069+0000 7fd80cbe8640 1 -- 172.21.15.16:0/1265389553 shutdown_connections 2024-09-12T02:16:25.071 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.069+0000 7fd80cbe8640 1 -- 172.21.15.16:0/1265389553 wait complete. 2024-09-12T02:16:25.072 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.069+0000 7fd80cbe8640 1 Processor -- start 2024-09-12T02:16:25.072 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.069+0000 7fd80cbe8640 1 -- start start 2024-09-12T02:16:25.072 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.069+0000 7fd80cbe8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd808101b60 0x7fd808197de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:25.072 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.069+0000 7fd80cbe8640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd808102d50 0x7fd808198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:25.072 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.070+0000 7fd80cbe8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd808103710 0x7fd80819f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:25.072 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.070+0000 7fd80cbe8640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd80810d7c0 con 0x7fd808103710 2024-09-12T02:16:25.072 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.070+0000 7fd80cbe8640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd80810d640 con 0x7fd808101b60 2024-09-12T02:16:25.072 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.070+0000 7fd80cbe8640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd80810d940 con 0x7fd808102d50 2024-09-12T02:16:25.072 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.070+0000 7fd806d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd808103710 0x7fd80819f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:25.072 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.070+0000 7fd806d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd808103710 0x7fd80819f390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40226/0 (socket says 172.21.15.16:40226) 2024-09-12T02:16:25.072 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.070+0000 7fd806d9d640 1 -- 172.21.15.16:0/2588633197 learned_addr learned my addr 172.21.15.16:0/2588633197 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:16:25.072 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.070+0000 7fd80659c640 1 --2- 172.21.15.16:0/2588633197 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd808101b60 0x7fd808197de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:25.073 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.070+0000 7fd805d9b640 1 --2- 172.21.15.16:0/2588633197 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd808102d50 0x7fd808198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:25.073 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.070+0000 7fd806d9d640 1 -- 172.21.15.16:0/2588633197 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd808102d50 msgr2=0x7fd808198320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:25.073 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.070+0000 7fd806d9d640 1 --2- 172.21.15.16:0/2588633197 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd808102d50 0x7fd808198320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:25.073 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.070+0000 7fd806d9d640 1 -- 172.21.15.16:0/2588633197 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd808101b60 msgr2=0x7fd808197de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:25.073 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.070+0000 7fd806d9d640 1 --2- 172.21.15.16:0/2588633197 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd808101b60 0x7fd808197de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:25.073 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.070+0000 7fd806d9d640 1 -- 172.21.15.16:0/2588633197 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd80819fa00 con 0x7fd808103710 2024-09-12T02:16:25.073 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.070+0000 7fd805d9b640 1 --2- 172.21.15.16:0/2588633197 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd808102d50 0x7fd808198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:16:25.073 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.070+0000 7fd806d9d640 1 --2- 172.21.15.16:0/2588633197 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd808103710 0x7fd80819f390 secure :-1 s=READY pgs=1927 cs=0 l=1 rev1=1 crypto rx=0x7fd80000eaa0 tx=0x7fd80000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:25.073 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.070+0000 7fd7f77fe640 1 -- 172.21.15.16:0/2588633197 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd80000cdb0 con 0x7fd808103710 2024-09-12T02:16:25.073 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.070+0000 7fd7f77fe640 1 -- 172.21.15.16:0/2588633197 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd800005020 con 0x7fd808103710 2024-09-12T02:16:25.073 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.070+0000 7fd80cbe8640 1 -- 172.21.15.16:0/2588633197 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd80819fcf0 con 0x7fd808103710 2024-09-12T02:16:25.073 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.070+0000 7fd7f77fe640 1 -- 172.21.15.16:0/2588633197 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8000116b0 con 0x7fd808103710 2024-09-12T02:16:25.073 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.070+0000 7fd80cbe8640 1 -- 172.21.15.16:0/2588633197 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8081a0190 con 0x7fd808103710 2024-09-12T02:16:25.080 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.073+0000 7fd7f77fe640 1 -- 172.21.15.16:0/2588633197 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fd800005560 con 0x7fd808103710 2024-09-12T02:16:25.080 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.073+0000 7fd80fa5f740 1 Processor -- start 2024-09-12T02:16:25.080 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.073+0000 7fd80fa5f740 1 -- start start 2024-09-12T02:16:25.080 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.073+0000 7fd80fa5f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e01ceafdb0 0x55e01ceb01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:25.080 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.073+0000 7fd80fa5f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e01ceb06f0 0x55e01ceb51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:25.080 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.073+0000 7fd80fa5f740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e01ceb5780 0x55e01ceb7c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:25.080 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.073+0000 7fd80fa5f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e01ce0f2f0 con 0x55e01ceb06f0 2024-09-12T02:16:25.080 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.073+0000 7fd80fa5f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55e01cdb7270 con 0x55e01ceafdb0 2024-09-12T02:16:25.080 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.073+0000 7fd80fa5f740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55e01ce0b2b0 con 0x55e01ceb5780 2024-09-12T02:16:25.080 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.073+0000 7fd80659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e01ceb06f0 0x55e01ceb51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:25.080 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.073+0000 7fd80659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e01ceb06f0 0x55e01ceb51b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40230/0 (socket says 172.21.15.16:40230) 2024-09-12T02:16:25.081 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.073+0000 7fd80659c640 1 -- 172.21.15.16:0/1395124435 learned_addr learned my addr 172.21.15.16:0/1395124435 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:16:25.081 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.073+0000 7fd805d9b640 1 --2- 172.21.15.16:0/1395124435 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e01ceafdb0 0x55e01ceb01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:25.081 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.074+0000 7fd805d9b640 1 --2- 172.21.15.16:0/1395124435 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e01ceb5780 0x55e01ceb7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:25.081 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.074+0000 7fd80659c640 1 -- 172.21.15.16:0/1395124435 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e01ceb5780 msgr2=0x55e01ceb7c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:25.081 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.074+0000 7fd80659c640 1 --2- 172.21.15.16:0/1395124435 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e01ceb5780 0x55e01ceb7c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:25.081 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.074+0000 7fd80659c640 1 -- 172.21.15.16:0/1395124435 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e01ceafdb0 msgr2=0x55e01ceb01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:25.081 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.074+0000 7fd80659c640 1 --2- 172.21.15.16:0/1395124435 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e01ceafdb0 0x55e01ceb01b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:25.081 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.074+0000 7fd80659c640 1 -- 172.21.15.16:0/1395124435 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e01cdb6340 con 0x55e01ceb06f0 2024-09-12T02:16:25.081 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.074+0000 7fd7f77fe640 1 --2- 172.21.15.16:0/2588633197 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fd7f00a2c30 0x7fd7f00a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:25.081 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.074+0000 7fd80659c640 1 --2- 172.21.15.16:0/1395124435 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e01ceb06f0 0x55e01ceb51b0 secure :-1 s=READY pgs=1928 cs=0 l=1 rev1=1 crypto rx=0x7fd7f8031610 tx=0x7fd7f8005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d232e094cbb80f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:25.081 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.074+0000 7fd805d9b640 1 --2- 172.21.15.16:0/2588633197 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fd7f00a2c30 0x7fd7f00a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:25.081 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.074+0000 7fd7f77fe640 1 -- 172.21.15.16:0/2588633197 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fd8000cf1f0 con 0x7fd808103710 2024-09-12T02:16:25.081 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.074+0000 7fd7f4ff9640 1 -- 172.21.15.16:0/1395124435 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7f8040070 con 0x55e01ceb06f0 2024-09-12T02:16:25.082 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.074+0000 7fd7f4ff9640 1 -- 172.21.15.16:0/1395124435 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd7f80318b0 con 0x55e01ceb06f0 2024-09-12T02:16:25.082 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.074+0000 7fd7f4ff9640 1 -- 172.21.15.16:0/1395124435 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7f8035040 con 0x55e01ceb06f0 2024-09-12T02:16:25.082 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.075+0000 7fd80fa5f740 1 -- 172.21.15.16:0/1395124435 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e01ceb06f0 msgr2=0x55e01ceb51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:25.082 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.075+0000 7fd80fa5f740 1 --2- 172.21.15.16:0/1395124435 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e01ceb06f0 0x55e01ceb51b0 secure :-1 s=READY pgs=1928 cs=0 l=1 rev1=1 crypto rx=0x7fd7f8031610 tx=0x7fd7f8005c00 comp rx=0 tx=0).stop 2024-09-12T02:16:25.082 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.075+0000 7fd805d9b640 1 --2- 172.21.15.16:0/2588633197 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fd7f00a2c30 0x7fd7f00a50f0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fd7fc0058c0 tx=0x7fd7fc00b580 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:25.082 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.075+0000 7fd80fa5f740 1 -- 172.21.15.16:0/1395124435 shutdown_connections 2024-09-12T02:16:25.082 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.075+0000 7fd80fa5f740 1 --2- 172.21.15.16:0/1395124435 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e01ceb5780 0x55e01ceb7c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:25.082 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.075+0000 7fd80fa5f740 1 --2- 172.21.15.16:0/1395124435 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e01ceb06f0 0x55e01ceb51b0 unknown :-1 s=CLOSED pgs=1928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:25.082 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.075+0000 7fd80fa5f740 1 --2- 172.21.15.16:0/1395124435 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e01ceafdb0 0x55e01ceb01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:25.082 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.075+0000 7fd80fa5f740 1 -- 172.21.15.16:0/1395124435 >> 172.21.15.16:0/1395124435 conn(0x55e01ceab420 msgr2=0x55e01ceab820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:25.082 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.075+0000 7fd80fa5f740 1 -- 172.21.15.16:0/1395124435 shutdown_connections 2024-09-12T02:16:25.082 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.075+0000 7fd80fa5f740 1 -- 172.21.15.16:0/1395124435 wait complete. 2024-09-12T02:16:25.082 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.075+0000 7fd80fa5f740 1 -- start start 2024-09-12T02:16:25.082 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.075+0000 7fd80fa5f740 1 Processor -- start 2024-09-12T02:16:25.083 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.076+0000 7fd80fa5f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e01ceafdb0 0x55e01ceb9fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:25.083 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.076+0000 7fd80fa5f740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e01ceb06f0 0x55e01ceba4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:25.083 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.076+0000 7fd80fa5f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e01ceb5780 0x55e01cec14b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:25.083 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.076+0000 7fd80fa5f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e01ceb86f0 con 0x55e01ceb5780 2024-09-12T02:16:25.083 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.076+0000 7fd80fa5f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55e01cd98290 con 0x55e01ceafdb0 2024-09-12T02:16:25.083 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.076+0000 7fd80fa5f740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55e01ceb8870 con 0x55e01ceb06f0 2024-09-12T02:16:25.083 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.076+0000 7fd80fa5f740 10 client.0 _ll_register_callbacks cb 0x55e01cdc9140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:16:25.083 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.076+0000 7fd806d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e01ceb5780 0x55e01cec14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:25.083 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.076+0000 7fd806d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e01ceb5780 0x55e01cec14b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40244/0 (socket says 172.21.15.16:40244) 2024-09-12T02:16:25.083 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.076+0000 7fd80659c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e01ceafdb0 0x55e01ceb9fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:25.083 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.076+0000 7fd806d9d640 1 -- 172.21.15.16:0/1405067347 learned_addr learned my addr 172.21.15.16:0/1405067347 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:16:25.083 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.076+0000 7fd805d9b640 1 --2- 172.21.15.16:0/1405067347 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e01ceb06f0 0x55e01ceba4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:25.084 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.076+0000 7fd80659c640 1 -- 172.21.15.16:0/1405067347 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e01ceb06f0 msgr2=0x55e01ceba4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:25.084 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.076+0000 7fd80659c640 1 --2- 172.21.15.16:0/1405067347 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e01ceb06f0 0x55e01ceba4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:25.084 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.076+0000 7fd80659c640 1 -- 172.21.15.16:0/1405067347 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e01ceb5780 msgr2=0x55e01cec14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:25.084 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.076+0000 7fd80659c640 1 --2- 172.21.15.16:0/1405067347 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e01ceb5780 0x55e01cec14b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:25.084 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.076+0000 7fd80659c640 1 -- 172.21.15.16:0/1405067347 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e01cb22210 con 0x55e01ceafdb0 2024-09-12T02:16:25.084 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.076+0000 7fd806d9d640 1 --2- 172.21.15.16:0/1405067347 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e01ceb5780 0x55e01cec14b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:16:25.084 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.076+0000 7fd80659c640 1 --2- 172.21.15.16:0/1405067347 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e01ceafdb0 0x55e01ceb9fa0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fd7f8005ab0 tx=0x7fd7f8032e60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5fda978a10bd1675 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:25.084 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.076+0000 7fd7edffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-12T02:16:25.084 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.076+0000 7fd7edffb640 1 -- 172.21.15.16:0/1405067347 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7f804b020 con 0x55e01ceafdb0 2024-09-12T02:16:25.084 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.077+0000 7fd7edffb640 1 -- 172.21.15.16:0/1405067347 <== mon.2 v2:172.21.15.33:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fd7f8004450 con 0x55e01ceafdb0 2024-09-12T02:16:25.084 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.077+0000 7fd80fa5f740 1 -- 172.21.15.16:0/1405067347 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd7f80318b0 con 0x55e01ceafdb0 2024-09-12T02:16:25.084 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.077+0000 7fd7edffb640 1 -- 172.21.15.16:0/1405067347 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd7f8035070 con 0x55e01ceafdb0 2024-09-12T02:16:25.084 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.077+0000 7fd80fa5f740 10 client.28982 fetch_fsmap learned FSMap version 25 2024-09-12T02:16:25.085 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.077+0000 7fd80fa5f740 1 -- 172.21.15.16:0/1405067347 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e01cbee620 con 0x55e01ceafdb0 2024-09-12T02:16:25.085 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.077+0000 7fd7edffb640 1 -- 172.21.15.16:0/1405067347 <== mon.2 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd7f80318b0 con 0x55e01ceafdb0 2024-09-12T02:16:25.085 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.077+0000 7fd7edffb640 1 -- 172.21.15.16:0/1405067347 <== mon.2 v2:172.21.15.33:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fd7f8004bd0 con 0x55e01ceafdb0 2024-09-12T02:16:25.085 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.077+0000 7fd80fa5f740 10 client.28982 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:16:25.085 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.077+0000 7fd80fa5f740 10 client.28982 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:16:25.085 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.077+0000 7fd80fa5f740 20 client.28982 populate_metadata read hostname 'smithi016' 2024-09-12T02:16:25.085 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.077+0000 7fd80fa5f740 1 --2- 172.21.15.16:0/1405067347 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55e01cec57b0 0x55e01cec7ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:25.085 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.077+0000 7fd80fa5f740 4 client.28982 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:16:25.085 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.077+0000 7fd80fa5f740 1 -- 172.21.15.16:0/1405067347 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e01ce1ca20 con 0x55e01cec57b0 2024-09-12T02:16:25.085 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.077+0000 7fd805d9b640 1 --2- 172.21.15.16:0/1405067347 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55e01cec57b0 0x55e01cec7ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:25.085 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.078+0000 7fd805d9b640 1 --2- 172.21.15.16:0/1405067347 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55e01cec57b0 0x55e01cec7ba0 crc :-1 s=READY pgs=427 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba9873ccf3562cd9 server_cookie=903461747d0944 in_seq=0 out_seq=0 2024-09-12T02:16:25.085 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.078+0000 7fd7edffb640 10 client.28982 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:16:25.085 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.079+0000 7fd7edffb640 1 -- 172.21.15.16:0/1405067347 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e01ce1ca20 con 0x55e01cec57b0 2024-09-12T02:16:25.085 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.079+0000 7fd7edffb640 10 client.28982 handle_command_reply: tid=0 2024-09-12T02:16:25.086 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.079+0000 7fd80fa5f740 1 client.28982 shutdown 2024-09-12T02:16:25.086 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.079+0000 7fd80fa5f740 1 -- 172.21.15.16:0/1405067347 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e01ceafdb0 msgr2=0x55e01ceb9fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:25.086 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.079+0000 7fd80fa5f740 1 --2- 172.21.15.16:0/1405067347 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e01ceafdb0 0x55e01ceb9fa0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fd7f8005ab0 tx=0x7fd7f8032e60 comp rx=0 tx=0).stop 2024-09-12T02:16:25.086 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.079+0000 7fd80fa5f740 1 -- 172.21.15.16:0/1405067347 shutdown_connections 2024-09-12T02:16:25.086 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.079+0000 7fd80fa5f740 1 --2- 172.21.15.16:0/1405067347 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55e01cec57b0 0x55e01cec7ba0 crc :-1 s=READY pgs=427 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:25.086 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.079+0000 7fd80fa5f740 1 --2- 172.21.15.16:0/1405067347 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e01ceb5780 0x55e01cec14b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:25.086 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.079+0000 7fd7edffb640 0 client.28982 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:16:25.086 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.079+0000 7fd80fa5f740 1 --2- 172.21.15.16:0/1405067347 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e01ceb06f0 0x55e01ceba4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:25.086 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.079+0000 7fd80fa5f740 1 --2- 172.21.15.16:0/1405067347 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e01ceafdb0 0x55e01ceb9fa0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:25.086 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.079+0000 7fd80fa5f740 1 -- 172.21.15.16:0/1405067347 >> 172.21.15.16:0/1405067347 conn(0x55e01ceab420 msgr2=0x55e01cea61c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:25.086 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.079+0000 7fd80fa5f740 1 -- 172.21.15.16:0/1405067347 shutdown_connections 2024-09-12T02:16:25.086 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.079+0000 7fd80fa5f740 1 -- 172.21.15.16:0/1405067347 wait complete. 2024-09-12T02:16:25.086 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.079+0000 7fd80fa5f740 20 client.28982 trim_cache size 0 max 16384 2024-09-12T02:16:25.097 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.093+0000 7fd80fa5f740 1 Processor -- start 2024-09-12T02:16:25.098 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.093+0000 7fd80fa5f740 1 -- start start 2024-09-12T02:16:25.098 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.094+0000 7fd80fa5f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e01ceafdb0 0x55e01ceba570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:25.098 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.094+0000 7fd80fa5f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e01ceb06f0 0x55e01cec1470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:25.098 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.094+0000 7fd80fa5f740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e01ceb5780 0x55e01cea60b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:25.098 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.094+0000 7fd80fa5f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd8081a94e0 con 0x55e01ceb06f0 2024-09-12T02:16:25.098 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.094+0000 7fd80fa5f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd8081a6db0 con 0x55e01ceafdb0 2024-09-12T02:16:25.098 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.094+0000 7fd80fa5f740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd8081a4c00 con 0x55e01ceb5780 2024-09-12T02:16:25.098 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.094+0000 7fd805d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e01ceb06f0 0x55e01cec1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:25.098 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.094+0000 7fd805d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e01ceb06f0 0x55e01cec1470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40246/0 (socket says 172.21.15.16:40246) 2024-09-12T02:16:25.098 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.094+0000 7fd805d9b640 1 -- 172.21.15.16:0/4132713986 learned_addr learned my addr 172.21.15.16:0/4132713986 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:16:25.098 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.094+0000 7fd80659c640 1 --2- 172.21.15.16:0/4132713986 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e01ceafdb0 0x55e01ceba570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:25.099 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.094+0000 7fd806d9d640 1 --2- 172.21.15.16:0/4132713986 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e01ceb5780 0x55e01cea60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:25.099 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.094+0000 7fd805d9b640 1 -- 172.21.15.16:0/4132713986 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e01ceb5780 msgr2=0x55e01cea60b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:25.099 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.094+0000 7fd805d9b640 1 --2- 172.21.15.16:0/4132713986 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e01ceb5780 0x55e01cea60b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:25.099 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.094+0000 7fd805d9b640 1 -- 172.21.15.16:0/4132713986 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e01ceafdb0 msgr2=0x55e01ceba570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:25.099 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.094+0000 7fd805d9b640 1 --2- 172.21.15.16:0/4132713986 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e01ceafdb0 0x55e01ceba570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:25.099 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.094+0000 7fd805d9b640 1 -- 172.21.15.16:0/4132713986 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e01cec2600 con 0x55e01ceb06f0 2024-09-12T02:16:25.099 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.094+0000 7fd806d9d640 1 --2- 172.21.15.16:0/4132713986 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e01ceb5780 0x55e01cea60b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:16:25.099 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.094+0000 7fd805d9b640 1 --2- 172.21.15.16:0/4132713986 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e01ceb06f0 0x55e01cec1470 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7fd7fc005d00 tx=0x7fd7fc014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=557bccb00f889928 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:25.100 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.094+0000 7fd7effff640 1 -- 172.21.15.16:0/4132713986 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7fc02d070 con 0x55e01ceb06f0 2024-09-12T02:16:25.100 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.095+0000 7fd7effff640 1 -- 172.21.15.16:0/4132713986 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd7fc020d00 con 0x55e01ceb06f0 2024-09-12T02:16:25.100 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.095+0000 7fd7effff640 1 -- 172.21.15.16:0/4132713986 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7fc01c710 con 0x55e01ceb06f0 2024-09-12T02:16:25.100 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.095+0000 7fd80fa5f740 1 -- 172.21.15.16:0/4132713986 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e01ceb06f0 msgr2=0x55e01cec1470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:25.100 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.095+0000 7fd80fa5f740 1 --2- 172.21.15.16:0/4132713986 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e01ceb06f0 0x55e01cec1470 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7fd7fc005d00 tx=0x7fd7fc014040 comp rx=0 tx=0).stop 2024-09-12T02:16:25.100 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.095+0000 7fd80fa5f740 1 -- 172.21.15.16:0/4132713986 shutdown_connections 2024-09-12T02:16:25.100 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.095+0000 7fd80fa5f740 1 --2- 172.21.15.16:0/4132713986 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e01ceb5780 0x55e01cea60b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:25.100 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.095+0000 7fd80fa5f740 1 --2- 172.21.15.16:0/4132713986 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e01ceb06f0 0x55e01cec1470 unknown :-1 s=CLOSED pgs=1929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:25.100 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.095+0000 7fd80fa5f740 1 --2- 172.21.15.16:0/4132713986 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e01ceafdb0 0x55e01ceba570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:25.100 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.095+0000 7fd80fa5f740 1 -- 172.21.15.16:0/4132713986 >> 172.21.15.16:0/4132713986 conn(0x55e01ceab420 msgr2=0x55e01cea59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:25.100 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.095+0000 7fd80fa5f740 1 -- 172.21.15.16:0/4132713986 shutdown_connections 2024-09-12T02:16:25.100 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.095+0000 7fd80fa5f740 1 -- 172.21.15.16:0/4132713986 wait complete. 2024-09-12T02:16:25.101 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.095+0000 7fd80fa5f740 1 -- start start 2024-09-12T02:16:25.101 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.096+0000 7fd80fa5f740 1 Processor -- start 2024-09-12T02:16:25.101 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.096+0000 7fd80fa5f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e01ceafdb0 0x55e01cebe5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:25.101 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.096+0000 7fd80fa5f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e01ceb06f0 0x55e01cebeb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:25.101 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.096+0000 7fd80fa5f740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e01ceb5780 0x55e01cebfb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:25.101 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.096+0000 7fd80fa5f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd8081a5270 con 0x55e01ceafdb0 2024-09-12T02:16:25.101 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.096+0000 7fd80fa5f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd8081a66e0 con 0x55e01ceb06f0 2024-09-12T02:16:25.101 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.096+0000 7fd80fa5f740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd8081a5940 con 0x55e01ceb5780 2024-09-12T02:16:25.101 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.096+0000 7fd80fa5f740 10 client.0 _ll_register_callbacks cb 0x7fd7fc00b420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:16:25.101 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.096+0000 7fd80659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e01ceafdb0 0x55e01cebe5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:25.101 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.096+0000 7fd80659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e01ceafdb0 0x55e01cebe5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40258/0 (socket says 172.21.15.16:40258) 2024-09-12T02:16:25.101 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.096+0000 7fd80659c640 1 -- 172.21.15.16:0/1282842756 learned_addr learned my addr 172.21.15.16:0/1282842756 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:16:25.101 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.096+0000 7fd805d9b640 1 --2- 172.21.15.16:0/1282842756 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e01ceb06f0 0x55e01cebeb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:25.102 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.096+0000 7fd806d9d640 1 --2- 172.21.15.16:0/1282842756 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e01ceb5780 0x55e01cebfb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:25.102 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.096+0000 7fd80659c640 1 -- 172.21.15.16:0/1282842756 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e01ceb5780 msgr2=0x55e01cebfb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:25.102 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.096+0000 7fd80659c640 1 --2- 172.21.15.16:0/1282842756 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e01ceb5780 0x55e01cebfb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:25.102 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.096+0000 7fd80659c640 1 -- 172.21.15.16:0/1282842756 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e01ceb06f0 msgr2=0x55e01cebeb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:25.102 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.096+0000 7fd80659c640 1 --2- 172.21.15.16:0/1282842756 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e01ceb06f0 0x55e01cebeb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:25.102 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.096+0000 7fd80659c640 1 -- 172.21.15.16:0/1282842756 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e01cec7f10 con 0x55e01ceafdb0 2024-09-12T02:16:25.102 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.096+0000 7fd80659c640 1 --2- 172.21.15.16:0/1282842756 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e01ceafdb0 0x55e01cebe5f0 secure :-1 s=READY pgs=1930 cs=0 l=1 rev1=1 crypto rx=0x7fd7f8001eb0 tx=0x7fd7f8004480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94e3fa07b2dac295 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:25.102 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.096+0000 7fd7edffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:16:25.102 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.096+0000 7fd7edffb640 1 -- 172.21.15.16:0/1282842756 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7f804b070 con 0x55e01ceafdb0 2024-09-12T02:16:25.102 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.097+0000 7fd7edffb640 1 -- 172.21.15.16:0/1282842756 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fd7f8004c50 con 0x55e01ceafdb0 2024-09-12T02:16:25.102 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.097+0000 7fd80fa5f740 1 -- 172.21.15.16:0/1282842756 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd7fc020d00 con 0x55e01ceafdb0 2024-09-12T02:16:25.102 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.097+0000 7fd7edffb640 1 -- 172.21.15.16:0/1282842756 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd7f8040040 con 0x55e01ceafdb0 2024-09-12T02:16:25.103 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.097+0000 7fd80fa5f740 10 client.20226 fetch_fsmap learned FSMap version 25 2024-09-12T02:16:25.103 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.097+0000 7fd80fa5f740 1 -- 172.21.15.16:0/1282842756 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e01cec1290 con 0x55e01ceafdb0 2024-09-12T02:16:25.103 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.097+0000 7fd7edffb640 1 -- 172.21.15.16:0/1282842756 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd7fc020d00 con 0x55e01ceafdb0 2024-09-12T02:16:25.103 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.097+0000 7fd7edffb640 1 -- 172.21.15.16:0/1282842756 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fd7f8059030 con 0x55e01ceafdb0 2024-09-12T02:16:25.103 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.097+0000 7fd80fa5f740 10 client.20226 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:16:25.103 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.097+0000 7fd80fa5f740 10 client.20226 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:16:25.103 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.097+0000 7fd80fa5f740 20 client.20226 populate_metadata read hostname 'smithi016' 2024-09-12T02:16:25.103 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.097+0000 7fd80fa5f740 1 --2- 172.21.15.16:0/1282842756 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55e01cf30560 0x55e01cf32950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:25.103 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.097+0000 7fd80fa5f740 4 client.20226 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:16:25.103 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.097+0000 7fd80fa5f740 1 -- 172.21.15.16:0/1282842756 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e01ce16d70 con 0x55e01cf30560 2024-09-12T02:16:25.103 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.097+0000 7fd805d9b640 1 --2- 172.21.15.16:0/1282842756 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55e01cf30560 0x55e01cf32950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:25.103 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.098+0000 7fd805d9b640 1 --2- 172.21.15.16:0/1282842756 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55e01cf30560 0x55e01cf32950 crc :-1 s=READY pgs=428 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=beb27c0c7514d05 server_cookie=5f8d5ed976681ffb in_seq=0 out_seq=0 2024-09-12T02:16:25.103 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.098+0000 7fd7edffb640 10 client.20226 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:16:25.111 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.110+0000 7fd7edffb640 1 -- 172.21.15.16:0/1282842756 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e01ce16d70 con 0x55e01cf30560 2024-09-12T02:16:25.111 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.110+0000 7fd7edffb640 10 client.20226 handle_command_reply: tid=0 2024-09-12T02:16:25.111 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.110+0000 7fd80fa5f740 1 client.20226 shutdown 2024-09-12T02:16:25.111 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.110+0000 7fd80fa5f740 1 -- 172.21.15.16:0/1282842756 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e01ceafdb0 msgr2=0x55e01cebe5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:25.111 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.110+0000 7fd80fa5f740 1 --2- 172.21.15.16:0/1282842756 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e01ceafdb0 0x55e01cebe5f0 secure :-1 s=READY pgs=1930 cs=0 l=1 rev1=1 crypto rx=0x7fd7f8001eb0 tx=0x7fd7f8004480 comp rx=0 tx=0).stop 2024-09-12T02:16:25.111 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.110+0000 7fd80fa5f740 1 -- 172.21.15.16:0/1282842756 shutdown_connections 2024-09-12T02:16:25.111 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.110+0000 7fd80fa5f740 1 --2- 172.21.15.16:0/1282842756 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55e01cf30560 0x55e01cf32950 crc :-1 s=READY pgs=428 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:25.111 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.110+0000 7fd80fa5f740 1 --2- 172.21.15.16:0/1282842756 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e01ceb5780 0x55e01cebfb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:25.111 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.110+0000 7fd80fa5f740 1 --2- 172.21.15.16:0/1282842756 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e01ceb06f0 0x55e01cebeb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:25.112 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.110+0000 7fd80fa5f740 1 --2- 172.21.15.16:0/1282842756 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e01ceafdb0 0x55e01cebe5f0 unknown :-1 s=CLOSED pgs=1930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:25.112 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.110+0000 7fd7edffb640 0 client.20226 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:16:25.112 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.110+0000 7fd80fa5f740 1 -- 172.21.15.16:0/1282842756 >> 172.21.15.16:0/1282842756 conn(0x55e01ceab420 msgr2=0x55e01cea61c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:25.112 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.110+0000 7fd80fa5f740 1 -- 172.21.15.16:0/1282842756 shutdown_connections 2024-09-12T02:16:25.112 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.110+0000 7fd80fa5f740 1 -- 172.21.15.16:0/1282842756 wait complete. 2024-09-12T02:16:25.112 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.111+0000 7fd80fa5f740 20 client.20226 trim_cache size 0 max 16384 2024-09-12T02:16:25.112 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-09-12T02:16:25.112 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-09-12T02:16:25.112 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-09-12T02:16:25.112 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-09-12T02:16:25.113 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.111+0000 7fd80cbe8640 1 -- 172.21.15.16:0/2588633197 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fd7f00a2c30 msgr2=0x7fd7f00a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:25.113 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.111+0000 7fd80cbe8640 1 --2- 172.21.15.16:0/2588633197 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fd7f00a2c30 0x7fd7f00a50f0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fd7fc0058c0 tx=0x7fd7fc00b580 comp rx=0 tx=0).stop 2024-09-12T02:16:25.113 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.111+0000 7fd80cbe8640 1 -- 172.21.15.16:0/2588633197 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd808103710 msgr2=0x7fd80819f390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:25.113 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.111+0000 7fd80cbe8640 1 --2- 172.21.15.16:0/2588633197 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd808103710 0x7fd80819f390 secure :-1 s=READY pgs=1927 cs=0 l=1 rev1=1 crypto rx=0x7fd80000eaa0 tx=0x7fd80000ef60 comp rx=0 tx=0).stop 2024-09-12T02:16:25.114 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.111+0000 7fd80cbe8640 1 -- 172.21.15.16:0/2588633197 shutdown_connections 2024-09-12T02:16:25.114 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.111+0000 7fd80cbe8640 1 --2- 172.21.15.16:0/2588633197 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fd7f00a2c30 0x7fd7f00a50f0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:25.114 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.111+0000 7fd80cbe8640 1 --2- 172.21.15.16:0/2588633197 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd808103710 0x7fd80819f390 unknown :-1 s=CLOSED pgs=1927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:25.114 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.111+0000 7fd80cbe8640 1 --2- 172.21.15.16:0/2588633197 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd808102d50 0x7fd808198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:25.114 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.111+0000 7fd80cbe8640 1 --2- 172.21.15.16:0/2588633197 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd808101b60 0x7fd808197de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:25.114 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.111+0000 7fd80cbe8640 1 -- 172.21.15.16:0/2588633197 >> 172.21.15.16:0/2588633197 conn(0x7fd8080fd2e0 msgr2=0x7fd8081060d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:25.114 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.111+0000 7fd80cbe8640 1 -- 172.21.15.16:0/2588633197 shutdown_connections 2024-09-12T02:16:25.114 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:25.111+0000 7fd80cbe8640 1 -- 172.21.15.16:0/2588633197 wait complete. 2024-09-12T02:16:25.595 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-12T02:16:25.852 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:16:26.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.336+0000 7f818efd1640 1 -- 172.21.15.16:0/448494864 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8188101de0 msgr2=0x7f81881021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:26.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.336+0000 7f818efd1640 1 --2- 172.21.15.16:0/448494864 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8188101de0 0x7f81881021c0 secure :-1 s=READY pgs=1931 cs=0 l=1 rev1=1 crypto rx=0x7f817c009900 tx=0x7f817c02a410 comp rx=0 tx=0).stop 2024-09-12T02:16:26.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.336+0000 7f818efd1640 1 -- 172.21.15.16:0/448494864 shutdown_connections 2024-09-12T02:16:26.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.336+0000 7f818efd1640 1 --2- 172.21.15.16:0/448494864 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8188106f10 0x7f818810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:26.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.336+0000 7f818efd1640 1 --2- 172.21.15.16:0/448494864 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8188102790 0x7f81881067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:26.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.336+0000 7f818efd1640 1 --2- 172.21.15.16:0/448494864 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8188101de0 0x7f81881021c0 unknown :-1 s=CLOSED pgs=1931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:26.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.336+0000 7f818efd1640 1 -- 172.21.15.16:0/448494864 >> 172.21.15.16:0/448494864 conn(0x7f81880fd8c0 msgr2=0x7f81880ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:26.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.336+0000 7f818efd1640 1 -- 172.21.15.16:0/448494864 shutdown_connections 2024-09-12T02:16:26.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.336+0000 7f818efd1640 1 -- 172.21.15.16:0/448494864 wait complete. 2024-09-12T02:16:26.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.337+0000 7f818efd1640 1 Processor -- start 2024-09-12T02:16:26.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.337+0000 7f818efd1640 1 -- start start 2024-09-12T02:16:26.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.337+0000 7f818efd1640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8188101de0 0x7f8188198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:26.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.337+0000 7f818efd1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8188102790 0x7f81881989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:26.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.337+0000 7f818efd1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8188106f10 0x7f818819cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:26.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.337+0000 7f818efd1640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8188111f90 con 0x7f8188102790 2024-09-12T02:16:26.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.337+0000 7f818efd1640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8188111e10 con 0x7f8188106f10 2024-09-12T02:16:26.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.337+0000 7f818efd1640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8188112110 con 0x7f8188101de0 2024-09-12T02:16:26.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.338+0000 7f8187fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8188102790 0x7f81881989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:26.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.338+0000 7f8187fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8188102790 0x7f81881989d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40280/0 (socket says 172.21.15.16:40280) 2024-09-12T02:16:26.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.338+0000 7f8187fff640 1 -- 172.21.15.16:0/1892916166 learned_addr learned my addr 172.21.15.16:0/1892916166 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:16:26.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.338+0000 7f818d56e640 1 --2- 172.21.15.16:0/1892916166 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8188106f10 0x7f818819cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:26.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.338+0000 7f818cd6d640 1 --2- 172.21.15.16:0/1892916166 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8188101de0 0x7f8188198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:26.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.338+0000 7f8187fff640 1 -- 172.21.15.16:0/1892916166 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8188101de0 msgr2=0x7f8188198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:26.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.338+0000 7f8187fff640 1 --2- 172.21.15.16:0/1892916166 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8188101de0 0x7f8188198490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:26.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.338+0000 7f8187fff640 1 -- 172.21.15.16:0/1892916166 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8188106f10 msgr2=0x7f818819cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:26.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.338+0000 7f8187fff640 1 --2- 172.21.15.16:0/1892916166 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8188106f10 0x7f818819cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:26.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.338+0000 7f8187fff640 1 -- 172.21.15.16:0/1892916166 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f818819d440 con 0x7f8188102790 2024-09-12T02:16:26.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.338+0000 7f818cd6d640 1 --2- 172.21.15.16:0/1892916166 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8188101de0 0x7f8188198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:16:26.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.338+0000 7f8187fff640 1 --2- 172.21.15.16:0/1892916166 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8188102790 0x7f81881989d0 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7f817400c890 tx=0x7f817400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:26.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.338+0000 7f8185ffb640 1 -- 172.21.15.16:0/1892916166 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8174004220 con 0x7f8188102790 2024-09-12T02:16:26.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.338+0000 7f8185ffb640 1 -- 172.21.15.16:0/1892916166 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f81740043c0 con 0x7f8188102790 2024-09-12T02:16:26.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.338+0000 7f818efd1640 1 -- 172.21.15.16:0/1892916166 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f818819d730 con 0x7f8188102790 2024-09-12T02:16:26.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.338+0000 7f8185ffb640 1 -- 172.21.15.16:0/1892916166 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81740049e0 con 0x7f8188102790 2024-09-12T02:16:26.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.339+0000 7f818efd1640 1 -- 172.21.15.16:0/1892916166 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81881a5010 con 0x7f8188102790 2024-09-12T02:16:26.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.341+0000 7f8185ffb640 1 -- 172.21.15.16:0/1892916166 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f81740079b0 con 0x7f8188102790 2024-09-12T02:16:26.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.341+0000 7f81913aa740 1 Processor -- start 2024-09-12T02:16:26.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.341+0000 7f81913aa740 1 -- start start 2024-09-12T02:16:26.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.342+0000 7f81913aa740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fdfb78fb40 0x55fdfb78e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:26.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.342+0000 7f81913aa740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fdfb78e5d0 0x55fdfb78ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:26.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.342+0000 7f81913aa740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fdfb799370 0x55fdfb79b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:26.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.342+0000 7f81913aa740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55fdfb6f42f0 con 0x55fdfb799370 2024-09-12T02:16:26.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.342+0000 7f81913aa740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55fdfb69bcf0 con 0x55fdfb78e5d0 2024-09-12T02:16:26.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.342+0000 7f81913aa740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55fdfb6f02b0 con 0x55fdfb78fb40 2024-09-12T02:16:26.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.342+0000 7f8185ffb640 1 --2- 172.21.15.16:0/1892916166 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f81780a2b80 0x7f81780a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:26.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.342+0000 7f8187fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fdfb799370 0x55fdfb79b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:26.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.342+0000 7f8187fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fdfb799370 0x55fdfb79b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40290/0 (socket says 172.21.15.16:40290) 2024-09-12T02:16:26.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.342+0000 7f818d56e640 1 --2- 172.21.15.16:0/1892916166 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f81780a2b80 0x7f81780a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:26.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.342+0000 7f8187fff640 1 -- 172.21.15.16:0/1722589862 learned_addr learned my addr 172.21.15.16:0/1722589862 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:16:26.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.342+0000 7f818cd6d640 1 --2- 172.21.15.16:0/1722589862 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fdfb78e5d0 0x55fdfb78ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:26.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.342+0000 7f8185ffb640 1 -- 172.21.15.16:0/1892916166 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f81740cf150 con 0x7f8188102790 2024-09-12T02:16:26.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.342+0000 7f818d56e640 1 --2- 172.21.15.16:0/1722589862 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fdfb78fb40 0x55fdfb78e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:26.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.342+0000 7f8187fff640 1 -- 172.21.15.16:0/1722589862 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fdfb78fb40 msgr2=0x55fdfb78e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:26.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.342+0000 7f8187fff640 1 --2- 172.21.15.16:0/1722589862 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fdfb78fb40 0x55fdfb78e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:26.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.342+0000 7f8187fff640 1 -- 172.21.15.16:0/1722589862 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fdfb78e5d0 msgr2=0x55fdfb78ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:26.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.342+0000 7f8187fff640 1 --2- 172.21.15.16:0/1722589862 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fdfb78e5d0 0x55fdfb78ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:26.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.342+0000 7f8187fff640 1 -- 172.21.15.16:0/1722589862 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fdfb6803a0 con 0x55fdfb799370 2024-09-12T02:16:26.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.342+0000 7f8187fff640 1 --2- 172.21.15.16:0/1722589862 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fdfb799370 0x55fdfb79b760 secure :-1 s=READY pgs=1933 cs=0 l=1 rev1=1 crypto rx=0x7f8174005fd0 tx=0x7f81740c6540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c270fe1c86b561ad server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:26.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.342+0000 7f818d56e640 1 --2- 172.21.15.16:0/1892916166 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f81780a2b80 0x7f81780a5040 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f8180009fd0 tx=0x7f8180009d80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:26.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.343+0000 7f8172ffd640 1 -- 172.21.15.16:0/1722589862 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f817401d070 con 0x55fdfb799370 2024-09-12T02:16:26.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.343+0000 7f8172ffd640 1 -- 172.21.15.16:0/1722589862 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f817409c0b0 con 0x55fdfb799370 2024-09-12T02:16:26.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.343+0000 7f8172ffd640 1 -- 172.21.15.16:0/1722589862 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81740a4620 con 0x55fdfb799370 2024-09-12T02:16:26.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.343+0000 7f81913aa740 1 -- 172.21.15.16:0/1722589862 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fdfb799370 msgr2=0x55fdfb79b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:26.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.343+0000 7f81913aa740 1 --2- 172.21.15.16:0/1722589862 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fdfb799370 0x55fdfb79b760 secure :-1 s=READY pgs=1933 cs=0 l=1 rev1=1 crypto rx=0x7f8174005fd0 tx=0x7f81740c6540 comp rx=0 tx=0).stop 2024-09-12T02:16:26.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.344+0000 7f81913aa740 1 -- 172.21.15.16:0/1722589862 shutdown_connections 2024-09-12T02:16:26.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.344+0000 7f81913aa740 1 --2- 172.21.15.16:0/1722589862 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fdfb799370 0x55fdfb79b760 unknown :-1 s=CLOSED pgs=1933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:26.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.344+0000 7f81913aa740 1 --2- 172.21.15.16:0/1722589862 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fdfb78e5d0 0x55fdfb78ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:26.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.344+0000 7f81913aa740 1 --2- 172.21.15.16:0/1722589862 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fdfb78fb40 0x55fdfb78e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:26.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.344+0000 7f81913aa740 1 -- 172.21.15.16:0/1722589862 >> 172.21.15.16:0/1722589862 conn(0x55fdfb757310 msgr2=0x55fdfb78ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:26.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.344+0000 7f81913aa740 1 -- 172.21.15.16:0/1722589862 shutdown_connections 2024-09-12T02:16:26.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.344+0000 7f81913aa740 1 -- 172.21.15.16:0/1722589862 wait complete. 2024-09-12T02:16:26.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.344+0000 7f81913aa740 1 -- start start 2024-09-12T02:16:26.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.344+0000 7f81913aa740 1 Processor -- start 2024-09-12T02:16:26.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.344+0000 7f81913aa740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fdfb78e5d0 0x55fdfb798d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:26.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.344+0000 7f81913aa740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fdfb78fb40 0x55fdfb796010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:26.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.344+0000 7f81913aa740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fdfb799370 0x55fdfb796600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:26.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.344+0000 7f81913aa740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55fdfb79c010 con 0x55fdfb78fb40 2024-09-12T02:16:26.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.344+0000 7f81913aa740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55fdfb6af060 con 0x55fdfb78e5d0 2024-09-12T02:16:26.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.344+0000 7f81913aa740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55fdfb79c190 con 0x55fdfb799370 2024-09-12T02:16:26.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.345+0000 7f81913aa740 10 client.0 _ll_register_callbacks cb 0x55fdfb4028c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:16:26.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.345+0000 7f8187fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fdfb78fb40 0x55fdfb796010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:26.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.345+0000 7f818cd6d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fdfb78e5d0 0x55fdfb798d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:26.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.345+0000 7f8187fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fdfb78fb40 0x55fdfb796010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40298/0 (socket says 172.21.15.16:40298) 2024-09-12T02:16:26.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.345+0000 7f8187fff640 1 -- 172.21.15.16:0/4093995771 learned_addr learned my addr 172.21.15.16:0/4093995771 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:16:26.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.345+0000 7f818d56e640 1 --2- 172.21.15.16:0/4093995771 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fdfb799370 0x55fdfb796600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:26.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.345+0000 7f8187fff640 1 -- 172.21.15.16:0/4093995771 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fdfb799370 msgr2=0x55fdfb796600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:26.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.345+0000 7f8187fff640 1 --2- 172.21.15.16:0/4093995771 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fdfb799370 0x55fdfb796600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:26.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.345+0000 7f8187fff640 1 -- 172.21.15.16:0/4093995771 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fdfb78e5d0 msgr2=0x55fdfb798d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:26.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.345+0000 7f8187fff640 1 --2- 172.21.15.16:0/4093995771 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fdfb78e5d0 0x55fdfb798d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:26.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.345+0000 7f8187fff640 1 -- 172.21.15.16:0/4093995771 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fdfb4653c0 con 0x55fdfb78fb40 2024-09-12T02:16:26.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.345+0000 7f818d56e640 1 --2- 172.21.15.16:0/4093995771 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fdfb799370 0x55fdfb796600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:16:26.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.345+0000 7f818cd6d640 1 --2- 172.21.15.16:0/4093995771 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fdfb78e5d0 0x55fdfb798d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:16:26.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.345+0000 7f8187fff640 1 --2- 172.21.15.16:0/4093995771 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fdfb78fb40 0x55fdfb796010 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 crypto rx=0x7f817409b940 tx=0x7f817409baf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e02aacf4a2e5163 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:26.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.345+0000 7f8167fff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:16:26.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.345+0000 7f8167fff640 1 -- 172.21.15.16:0/4093995771 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f817401d040 con 0x55fdfb78fb40 2024-09-12T02:16:26.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.345+0000 7f8167fff640 1 -- 172.21.15.16:0/4093995771 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f817409bc10 con 0x55fdfb78fb40 2024-09-12T02:16:26.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.345+0000 7f81913aa740 1 -- 172.21.15.16:0/4093995771 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f817409c0b0 con 0x55fdfb78fb40 2024-09-12T02:16:26.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.346+0000 7f8167fff640 1 -- 172.21.15.16:0/4093995771 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f81740b9050 con 0x55fdfb78fb40 2024-09-12T02:16:26.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.346+0000 7f81913aa740 10 client.20238 fetch_fsmap learned FSMap version 25 2024-09-12T02:16:26.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.346+0000 7f81913aa740 1 -- 172.21.15.16:0/4093995771 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55fdfb7a7ab0 con 0x55fdfb78fb40 2024-09-12T02:16:26.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.346+0000 7f8167fff640 1 -- 172.21.15.16:0/4093995771 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f817409c0b0 con 0x55fdfb78fb40 2024-09-12T02:16:26.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.346+0000 7f8167fff640 1 -- 172.21.15.16:0/4093995771 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f817409cce0 con 0x55fdfb78fb40 2024-09-12T02:16:26.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.346+0000 7f81913aa740 10 client.20238 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:16:26.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.346+0000 7f81913aa740 10 client.20238 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T02:16:26.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.346+0000 7f81913aa740 20 client.20238 populate_metadata read hostname 'smithi016' 2024-09-12T02:16:26.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.346+0000 7f81913aa740 1 --2- 172.21.15.16:0/4093995771 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x55fdfb7a8e00 0x55fdfb7ab1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:26.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.346+0000 7f81913aa740 4 client.20238 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:16:26.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.346+0000 7f81913aa740 1 -- 172.21.15.16:0/4093995771 --> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fdfb701860 con 0x55fdfb7a8e00 2024-09-12T02:16:26.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.346+0000 7f818cd6d640 1 --2- 172.21.15.16:0/4093995771 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x55fdfb7a8e00 0x55fdfb7ab1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:26.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.347+0000 7f818cd6d640 1 --2- 172.21.15.16:0/4093995771 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x55fdfb7a8e00 0x55fdfb7ab1f0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=15b06cbd0ca47d88 server_cookie=21f274a661c7501a in_seq=0 out_seq=0 2024-09-12T02:16:26.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.347+0000 7f8167fff640 10 client.20238 ms_handle_connect on v2:172.21.15.184:6838/4203347941 2024-09-12T02:16:26.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.347+0000 7f8167fff640 1 -- 172.21.15.16:0/4093995771 <== mds.1 v2:172.21.15.184:6838/4203347941 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fdfb701860 con 0x55fdfb7a8e00 2024-09-12T02:16:26.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.347+0000 7f8167fff640 10 client.20238 handle_command_reply: tid=0 2024-09-12T02:16:26.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.347+0000 7f81913aa740 1 client.20238 shutdown 2024-09-12T02:16:26.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.348+0000 7f81913aa740 1 -- 172.21.15.16:0/4093995771 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fdfb78fb40 msgr2=0x55fdfb796010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:26.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.348+0000 7f81913aa740 1 --2- 172.21.15.16:0/4093995771 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fdfb78fb40 0x55fdfb796010 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 crypto rx=0x7f817409b940 tx=0x7f817409baf0 comp rx=0 tx=0).stop 2024-09-12T02:16:26.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.348+0000 7f81913aa740 1 -- 172.21.15.16:0/4093995771 shutdown_connections 2024-09-12T02:16:26.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.348+0000 7f81913aa740 1 --2- 172.21.15.16:0/4093995771 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x55fdfb7a8e00 0x55fdfb7ab1f0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:26.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.348+0000 7f81913aa740 1 --2- 172.21.15.16:0/4093995771 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fdfb799370 0x55fdfb796600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:26.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.348+0000 7f81913aa740 1 --2- 172.21.15.16:0/4093995771 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fdfb78fb40 0x55fdfb796010 unknown :-1 s=CLOSED pgs=1934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:26.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.348+0000 7f8167fff640 0 client.20238 ms_handle_reset on v2:172.21.15.184:6838/4203347941 2024-09-12T02:16:26.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.348+0000 7f81913aa740 1 --2- 172.21.15.16:0/4093995771 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fdfb78e5d0 0x55fdfb798d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:26.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.348+0000 7f81913aa740 1 -- 172.21.15.16:0/4093995771 >> 172.21.15.16:0/4093995771 conn(0x55fdfb757310 msgr2=0x55fdfb790990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:26.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.348+0000 7f81913aa740 1 -- 172.21.15.16:0/4093995771 shutdown_connections 2024-09-12T02:16:26.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.348+0000 7f81913aa740 1 -- 172.21.15.16:0/4093995771 wait complete. 2024-09-12T02:16:26.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.348+0000 7f81913aa740 20 client.20238 trim_cache size 0 max 16384 2024-09-12T02:16:26.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.361+0000 7f81913aa740 1 Processor -- start 2024-09-12T02:16:26.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.361+0000 7f81913aa740 1 -- start start 2024-09-12T02:16:26.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.361+0000 7f81913aa740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fdfb78e5d0 0x55fdfb78dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:26.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.361+0000 7f81913aa740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fdfb78fb40 0x55fdfb797f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:26.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.361+0000 7f81913aa740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fdfb799370 0x55fdfb79a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:26.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.361+0000 7f81913aa740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f81881afde0 con 0x55fdfb78e5d0 2024-09-12T02:16:26.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.361+0000 7f81913aa740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f81881ad6b0 con 0x55fdfb799370 2024-09-12T02:16:26.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.361+0000 7f81913aa740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f81881ab520 con 0x55fdfb78fb40 2024-09-12T02:16:26.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.361+0000 7f818cd6d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fdfb78e5d0 0x55fdfb78dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:26.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.361+0000 7f818cd6d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fdfb78e5d0 0x55fdfb78dd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40310/0 (socket says 172.21.15.16:40310) 2024-09-12T02:16:26.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.361+0000 7f818cd6d640 1 -- 172.21.15.16:0/3412646115 learned_addr learned my addr 172.21.15.16:0/3412646115 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:16:26.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.361+0000 7f8187fff640 1 --2- 172.21.15.16:0/3412646115 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fdfb78fb40 0x55fdfb797f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:26.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.361+0000 7f818d56e640 1 --2- 172.21.15.16:0/3412646115 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fdfb799370 0x55fdfb79a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:26.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.362+0000 7f818cd6d640 1 -- 172.21.15.16:0/3412646115 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fdfb78fb40 msgr2=0x55fdfb797f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:26.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.362+0000 7f818cd6d640 1 --2- 172.21.15.16:0/3412646115 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fdfb78fb40 0x55fdfb797f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:26.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.362+0000 7f818cd6d640 1 -- 172.21.15.16:0/3412646115 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fdfb799370 msgr2=0x55fdfb79a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:26.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.362+0000 7f818cd6d640 1 --2- 172.21.15.16:0/3412646115 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fdfb799370 0x55fdfb79a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:26.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.362+0000 7f818cd6d640 1 -- 172.21.15.16:0/3412646115 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fdfb798980 con 0x55fdfb78e5d0 2024-09-12T02:16:26.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.362+0000 7f818d56e640 1 --2- 172.21.15.16:0/3412646115 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fdfb799370 0x55fdfb79a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:16:26.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.362+0000 7f818cd6d640 1 --2- 172.21.15.16:0/3412646115 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fdfb78e5d0 0x55fdfb78dd90 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 crypto rx=0x7f817c009230 tx=0x7f817c005a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=799f14298f4ed57e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:26.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.362+0000 7f81727fc640 1 -- 172.21.15.16:0/3412646115 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f817c035040 con 0x55fdfb78e5d0 2024-09-12T02:16:26.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.362+0000 7f81727fc640 1 -- 172.21.15.16:0/3412646115 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f817c049070 con 0x55fdfb78e5d0 2024-09-12T02:16:26.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.362+0000 7f81727fc640 1 -- 172.21.15.16:0/3412646115 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f817c005630 con 0x55fdfb78e5d0 2024-09-12T02:16:26.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.363+0000 7f81913aa740 1 -- 172.21.15.16:0/3412646115 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fdfb78e5d0 msgr2=0x55fdfb78dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:26.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.363+0000 7f81913aa740 1 --2- 172.21.15.16:0/3412646115 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fdfb78e5d0 0x55fdfb78dd90 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 crypto rx=0x7f817c009230 tx=0x7f817c005a40 comp rx=0 tx=0).stop 2024-09-12T02:16:26.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.363+0000 7f81913aa740 1 -- 172.21.15.16:0/3412646115 shutdown_connections 2024-09-12T02:16:26.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.363+0000 7f81913aa740 1 --2- 172.21.15.16:0/3412646115 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fdfb799370 0x55fdfb79a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:26.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.363+0000 7f81913aa740 1 --2- 172.21.15.16:0/3412646115 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fdfb78fb40 0x55fdfb797f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:26.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.363+0000 7f81913aa740 1 --2- 172.21.15.16:0/3412646115 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fdfb78e5d0 0x55fdfb78dd90 unknown :-1 s=CLOSED pgs=1935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:26.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.363+0000 7f81913aa740 1 -- 172.21.15.16:0/3412646115 >> 172.21.15.16:0/3412646115 conn(0x55fdfb757310 msgr2=0x55fdfb797870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:26.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.363+0000 7f81913aa740 1 -- 172.21.15.16:0/3412646115 shutdown_connections 2024-09-12T02:16:26.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.363+0000 7f81913aa740 1 -- 172.21.15.16:0/3412646115 wait complete. 2024-09-12T02:16:26.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.363+0000 7f81913aa740 1 -- start start 2024-09-12T02:16:26.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.363+0000 7f81913aa740 1 Processor -- start 2024-09-12T02:16:26.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.364+0000 7f81913aa740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fdfb78e5d0 0x55fdfb7ae8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:26.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.364+0000 7f81913aa740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fdfb78fb40 0x55fdfb792cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:26.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.364+0000 7f81913aa740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fdfb799370 0x55fdfb793230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:26.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.364+0000 7f81913aa740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f81881abb70 con 0x55fdfb799370 2024-09-12T02:16:26.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.364+0000 7f81913aa740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f81881acfe0 con 0x55fdfb78e5d0 2024-09-12T02:16:26.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.364+0000 7f81913aa740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f81881ac240 con 0x55fdfb78fb40 2024-09-12T02:16:26.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.364+0000 7f81913aa740 10 client.0 _ll_register_callbacks cb 0x7f817c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:16:26.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.364+0000 7f818d56e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fdfb799370 0x55fdfb793230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:26.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.364+0000 7f818d56e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fdfb799370 0x55fdfb793230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40322/0 (socket says 172.21.15.16:40322) 2024-09-12T02:16:26.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.364+0000 7f818d56e640 1 -- 172.21.15.16:0/916373741 learned_addr learned my addr 172.21.15.16:0/916373741 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:16:26.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.364+0000 7f818cd6d640 1 --2- 172.21.15.16:0/916373741 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fdfb78e5d0 0x55fdfb7ae8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:26.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.364+0000 7f8187fff640 1 --2- 172.21.15.16:0/916373741 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fdfb78fb40 0x55fdfb792cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:26.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.364+0000 7f818d56e640 1 -- 172.21.15.16:0/916373741 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fdfb78fb40 msgr2=0x55fdfb792cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:26.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.364+0000 7f818d56e640 1 --2- 172.21.15.16:0/916373741 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fdfb78fb40 0x55fdfb792cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:26.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.364+0000 7f818d56e640 1 -- 172.21.15.16:0/916373741 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fdfb78e5d0 msgr2=0x55fdfb7ae8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:26.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.364+0000 7f818d56e640 1 --2- 172.21.15.16:0/916373741 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fdfb78e5d0 0x55fdfb7ae8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:26.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.364+0000 7f818d56e640 1 -- 172.21.15.16:0/916373741 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fdfb793a00 con 0x55fdfb799370 2024-09-12T02:16:26.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.364+0000 7f818d56e640 1 --2- 172.21.15.16:0/916373741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fdfb799370 0x55fdfb793230 secure :-1 s=READY pgs=1936 cs=0 l=1 rev1=1 crypto rx=0x7f8180016e00 tx=0x7f8180019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50049cb47d87025a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:26.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.364+0000 7f8167fff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:16:26.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.364+0000 7f8167fff640 1 -- 172.21.15.16:0/916373741 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8180021040 con 0x55fdfb799370 2024-09-12T02:16:26.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.365+0000 7f8167fff640 1 -- 172.21.15.16:0/916373741 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f81800253f0 con 0x55fdfb799370 2024-09-12T02:16:26.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.365+0000 7f81913aa740 1 -- 172.21.15.16:0/916373741 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f817c049070 con 0x55fdfb799370 2024-09-12T02:16:26.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.365+0000 7f8167fff640 1 -- 172.21.15.16:0/916373741 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f8180013040 con 0x55fdfb799370 2024-09-12T02:16:26.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.365+0000 7f81913aa740 10 client.20244 fetch_fsmap learned FSMap version 25 2024-09-12T02:16:26.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.365+0000 7f81913aa740 1 -- 172.21.15.16:0/916373741 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55fdfb817ae0 con 0x55fdfb799370 2024-09-12T02:16:26.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.365+0000 7f8167fff640 1 -- 172.21.15.16:0/916373741 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f817c049070 con 0x55fdfb799370 2024-09-12T02:16:26.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.365+0000 7f8167fff640 1 -- 172.21.15.16:0/916373741 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f8180024030 con 0x55fdfb799370 2024-09-12T02:16:26.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.365+0000 7f81913aa740 10 client.20244 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:16:26.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.365+0000 7f81913aa740 10 client.20244 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T02:16:26.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.365+0000 7f81913aa740 20 client.20244 populate_metadata read hostname 'smithi016' 2024-09-12T02:16:26.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.365+0000 7f81913aa740 1 --2- 172.21.15.16:0/916373741 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x55fdfb818310 0x55fdfb81a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:26.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.365+0000 7f81913aa740 4 client.20244 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T02:16:26.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.365+0000 7f81913aa740 1 -- 172.21.15.16:0/916373741 --> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fdfb6fbd70 con 0x55fdfb818310 2024-09-12T02:16:26.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.365+0000 7f818cd6d640 1 --2- 172.21.15.16:0/916373741 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x55fdfb818310 0x55fdfb81a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:26.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.366+0000 7f818cd6d640 1 --2- 172.21.15.16:0/916373741 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x55fdfb818310 0x55fdfb81a730 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6978114b6d9056c1 server_cookie=89eca381f6152299 in_seq=0 out_seq=0 2024-09-12T02:16:26.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.366+0000 7f8167fff640 10 client.20244 ms_handle_connect on v2:172.21.15.184:6838/4203347941 2024-09-12T02:16:26.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.366+0000 7f8167fff640 1 -- 172.21.15.16:0/916373741 <== mds.1 v2:172.21.15.184:6838/4203347941 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fdfb6fbd70 con 0x55fdfb818310 2024-09-12T02:16:26.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.366+0000 7f8167fff640 10 client.20244 handle_command_reply: tid=0 2024-09-12T02:16:26.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.366+0000 7f81913aa740 1 client.20244 shutdown 2024-09-12T02:16:26.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.366+0000 7f81913aa740 1 -- 172.21.15.16:0/916373741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fdfb799370 msgr2=0x55fdfb793230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:26.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.366+0000 7f81913aa740 1 --2- 172.21.15.16:0/916373741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fdfb799370 0x55fdfb793230 secure :-1 s=READY pgs=1936 cs=0 l=1 rev1=1 crypto rx=0x7f8180016e00 tx=0x7f8180019000 comp rx=0 tx=0).stop 2024-09-12T02:16:26.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.366+0000 7f81913aa740 1 -- 172.21.15.16:0/916373741 shutdown_connections 2024-09-12T02:16:26.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.366+0000 7f81913aa740 1 --2- 172.21.15.16:0/916373741 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x55fdfb818310 0x55fdfb81a730 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:26.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.366+0000 7f81913aa740 1 --2- 172.21.15.16:0/916373741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fdfb799370 0x55fdfb793230 unknown :-1 s=CLOSED pgs=1936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:26.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.366+0000 7f81913aa740 1 --2- 172.21.15.16:0/916373741 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fdfb78fb40 0x55fdfb792cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:26.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.366+0000 7f8167fff640 0 client.20244 ms_handle_reset on v2:172.21.15.184:6838/4203347941 2024-09-12T02:16:26.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.366+0000 7f81913aa740 1 --2- 172.21.15.16:0/916373741 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fdfb78e5d0 0x55fdfb7ae8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:26.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.366+0000 7f81913aa740 1 -- 172.21.15.16:0/916373741 >> 172.21.15.16:0/916373741 conn(0x55fdfb757310 msgr2=0x55fdfb789d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:26.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.367+0000 7f81913aa740 1 -- 172.21.15.16:0/916373741 shutdown_connections 2024-09-12T02:16:26.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.367+0000 7f81913aa740 1 -- 172.21.15.16:0/916373741 wait complete. 2024-09-12T02:16:26.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.367+0000 7f81913aa740 20 client.20244 trim_cache size 0 max 16384 2024-09-12T02:16:26.372 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T02:16:26.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.367+0000 7f818efd1640 1 -- 172.21.15.16:0/1892916166 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f81780a2b80 msgr2=0x7f81780a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:26.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.367+0000 7f818efd1640 1 --2- 172.21.15.16:0/1892916166 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f81780a2b80 0x7f81780a5040 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f8180009fd0 tx=0x7f8180009d80 comp rx=0 tx=0).stop 2024-09-12T02:16:26.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.367+0000 7f818efd1640 1 -- 172.21.15.16:0/1892916166 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8188102790 msgr2=0x7f81881989d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:26.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.367+0000 7f818efd1640 1 --2- 172.21.15.16:0/1892916166 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8188102790 0x7f81881989d0 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7f817400c890 tx=0x7f817400cd50 comp rx=0 tx=0).stop 2024-09-12T02:16:26.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.367+0000 7f818efd1640 1 -- 172.21.15.16:0/1892916166 shutdown_connections 2024-09-12T02:16:26.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.367+0000 7f818efd1640 1 --2- 172.21.15.16:0/1892916166 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f81780a2b80 0x7f81780a5040 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:26.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.367+0000 7f818efd1640 1 --2- 172.21.15.16:0/1892916166 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8188106f10 0x7f818819cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:26.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.367+0000 7f818efd1640 1 --2- 172.21.15.16:0/1892916166 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8188102790 0x7f81881989d0 unknown :-1 s=CLOSED pgs=1932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:26.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.367+0000 7f818efd1640 1 --2- 172.21.15.16:0/1892916166 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8188101de0 0x7f8188198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:26.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.367+0000 7f818efd1640 1 -- 172.21.15.16:0/1892916166 >> 172.21.15.16:0/1892916166 conn(0x7f81880fd8c0 msgr2=0x7f81880fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:26.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.367+0000 7f818efd1640 1 -- 172.21.15.16:0/1892916166 shutdown_connections 2024-09-12T02:16:26.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:26.367+0000 7f818efd1640 1 -- 172.21.15.16:0/1892916166 wait complete. 2024-09-12T02:16:26.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:16:26 smithi033 ceph-mon[35332]: pgmap v770: 97 pgs: 97 active+clean; 3.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-09-12T02:16:26.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:16:26 smithi016 ceph-mon[30947]: pgmap v770: 97 pgs: 97 active+clean; 3.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-09-12T02:16:26.878 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:16:26 smithi184 ceph-mon[35593]: pgmap v770: 97 pgs: 97 active+clean; 3.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-09-12T02:16:27.097 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-12T02:16:27.361 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:16:27.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.802+0000 7fc50f7e1640 1 -- 172.21.15.16:0/1089684574 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc508100e20 msgr2=0x7fc508101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:27.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.802+0000 7fc50f7e1640 1 --2- 172.21.15.16:0/1089684574 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc508100e20 0x7fc508101200 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7fc504009900 tx=0x7fc50402a480 comp rx=0 tx=0).stop 2024-09-12T02:16:27.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.803+0000 7fc50f7e1640 1 -- 172.21.15.16:0/1089684574 shutdown_connections 2024-09-12T02:16:27.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.803+0000 7fc50f7e1640 1 --2- 172.21.15.16:0/1089684574 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc508102ef0 0x7fc50810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:27.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.803+0000 7fc50f7e1640 1 --2- 172.21.15.16:0/1089684574 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc508102570 0x7fc5081029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:27.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.803+0000 7fc50f7e1640 1 --2- 172.21.15.16:0/1089684574 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc508100e20 0x7fc508101200 unknown :-1 s=CLOSED pgs=1937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:27.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.803+0000 7fc50f7e1640 1 -- 172.21.15.16:0/1089684574 >> 172.21.15.16:0/1089684574 conn(0x7fc5080fc7d0 msgr2=0x7fc5080febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:27.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.803+0000 7fc50f7e1640 1 -- 172.21.15.16:0/1089684574 shutdown_connections 2024-09-12T02:16:27.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.803+0000 7fc50f7e1640 1 -- 172.21.15.16:0/1089684574 wait complete. 2024-09-12T02:16:27.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.803+0000 7fc50f7e1640 1 Processor -- start 2024-09-12T02:16:27.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.803+0000 7fc50f7e1640 1 -- start start 2024-09-12T02:16:27.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.804+0000 7fc50f7e1640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc508100e20 0x7fc508199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:27.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.804+0000 7fc50f7e1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc508102570 0x7fc5081997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:27.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.804+0000 7fc50f7e1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc508102ef0 0x7fc5081a4310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:27.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.804+0000 7fc50f7e1640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc508110f80 con 0x7fc508102570 2024-09-12T02:16:27.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.804+0000 7fc50f7e1640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc508110e00 con 0x7fc508102ef0 2024-09-12T02:16:27.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.804+0000 7fc50f7e1640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc508111100 con 0x7fc508100e20 2024-09-12T02:16:27.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.804+0000 7fc50cd7c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc508102570 0x7fc5081997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:27.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.804+0000 7fc50cd7c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc508102570 0x7fc5081997d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40336/0 (socket says 172.21.15.16:40336) 2024-09-12T02:16:27.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.804+0000 7fc50cd7c640 1 -- 172.21.15.16:0/2350626140 learned_addr learned my addr 172.21.15.16:0/2350626140 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:16:27.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.804+0000 7fc50d57d640 1 --2- 172.21.15.16:0/2350626140 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc508100e20 0x7fc508199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:27.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.804+0000 7fc50dd7e640 1 --2- 172.21.15.16:0/2350626140 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc508102ef0 0x7fc5081a4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:27.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.804+0000 7fc50cd7c640 1 -- 172.21.15.16:0/2350626140 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc508100e20 msgr2=0x7fc508199290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:27.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.804+0000 7fc50cd7c640 1 --2- 172.21.15.16:0/2350626140 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc508100e20 0x7fc508199290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:27.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.804+0000 7fc50cd7c640 1 -- 172.21.15.16:0/2350626140 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc508102ef0 msgr2=0x7fc5081a4310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:27.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.804+0000 7fc50cd7c640 1 --2- 172.21.15.16:0/2350626140 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc508102ef0 0x7fc5081a4310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:27.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.804+0000 7fc50cd7c640 1 -- 172.21.15.16:0/2350626140 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5081a4850 con 0x7fc508102570 2024-09-12T02:16:27.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.804+0000 7fc50dd7e640 1 --2- 172.21.15.16:0/2350626140 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc508102ef0 0x7fc5081a4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:16:27.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.804+0000 7fc50cd7c640 1 --2- 172.21.15.16:0/2350626140 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc508102570 0x7fc5081997d0 secure :-1 s=READY pgs=1938 cs=0 l=1 rev1=1 crypto rx=0x7fc4f800d950 tx=0x7fc4f800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:27.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.804+0000 7fc4fe7fc640 1 -- 172.21.15.16:0/2350626140 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4f8017070 con 0x7fc508102570 2024-09-12T02:16:27.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.804+0000 7fc50f7e1640 1 -- 172.21.15.16:0/2350626140 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5081a4a20 con 0x7fc508102570 2024-09-12T02:16:27.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.805+0000 7fc4fe7fc640 1 -- 172.21.15.16:0/2350626140 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4f8009c30 con 0x7fc508102570 2024-09-12T02:16:27.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.805+0000 7fc4fe7fc640 1 -- 172.21.15.16:0/2350626140 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4f800f6b0 con 0x7fc508102570 2024-09-12T02:16:27.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.805+0000 7fc50f7e1640 1 -- 172.21.15.16:0/2350626140 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5081a4fb0 con 0x7fc508102570 2024-09-12T02:16:27.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.807+0000 7fc4fe7fc640 1 -- 172.21.15.16:0/2350626140 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fc4f80109d0 con 0x7fc508102570 2024-09-12T02:16:27.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.807+0000 7fc511bba740 1 Processor -- start 2024-09-12T02:16:27.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.807+0000 7fc511bba740 1 -- start start 2024-09-12T02:16:27.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.808+0000 7fc511bba740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5604271b6b40 0x5604271b5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:27.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.808+0000 7fc511bba740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5604271b55d0 0x5604271b5a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:27.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.808+0000 7fc511bba740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5604271c4780 0x5604271c6b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:27.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.808+0000 7fc511bba740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56042711b2f0 con 0x5604271b6b40 2024-09-12T02:16:27.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.808+0000 7fc511bba740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5604270c2cf0 con 0x5604271c4780 2024-09-12T02:16:27.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.808+0000 7fc511bba740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5604271172b0 con 0x5604271b55d0 2024-09-12T02:16:27.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.808+0000 7fc50dd7e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5604271b6b40 0x5604271b5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:27.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.808+0000 7fc4fe7fc640 1 --2- 172.21.15.16:0/2350626140 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc4f40a2b80 0x7fc4f40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:27.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.808+0000 7fc50dd7e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5604271b6b40 0x5604271b5090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40342/0 (socket says 172.21.15.16:40342) 2024-09-12T02:16:27.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.808+0000 7fc50dd7e640 1 -- 172.21.15.16:0/768719841 learned_addr learned my addr 172.21.15.16:0/768719841 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:16:27.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.808+0000 7fc50d57d640 1 --2- 172.21.15.16:0/768719841 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5604271b55d0 0x5604271b5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:27.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.808+0000 7fc50cd7c640 1 --2- 172.21.15.16:0/768719841 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5604271c4780 0x5604271c6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:27.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.808+0000 7fc50dd7e640 1 --2- 172.21.15.16:0/2350626140 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc4f40a2b80 0x7fc4f40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:27.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.808+0000 7fc4fe7fc640 1 -- 172.21.15.16:0/2350626140 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fc4f80cec10 con 0x7fc508102570 2024-09-12T02:16:27.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.808+0000 7fc50dd7e640 1 -- 172.21.15.16:0/768719841 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5604271b55d0 msgr2=0x5604271b5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:27.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.808+0000 7fc50dd7e640 1 --2- 172.21.15.16:0/768719841 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5604271b55d0 0x5604271b5a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:27.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.808+0000 7fc50dd7e640 1 -- 172.21.15.16:0/768719841 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5604271c4780 msgr2=0x5604271c6b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:27.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.808+0000 7fc50dd7e640 1 --2- 172.21.15.16:0/768719841 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5604271c4780 0x5604271c6b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:27.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.808+0000 7fc50dd7e640 1 -- 172.21.15.16:0/768719841 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604270a73a0 con 0x5604271b6b40 2024-09-12T02:16:27.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.808+0000 7fc50cd7c640 1 --2- 172.21.15.16:0/768719841 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5604271c4780 0x5604271c6b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:16:27.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.808+0000 7fc50dd7e640 1 --2- 172.21.15.16:0/768719841 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5604271b6b40 0x5604271b5090 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7fc500012940 tx=0x7fc500012e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99abc79ad2ce6cfa server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:27.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.809+0000 7fc4f37fe640 1 -- 172.21.15.16:0/768719841 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc500022070 con 0x5604271b6b40 2024-09-12T02:16:27.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.809+0000 7fc4f37fe640 1 -- 172.21.15.16:0/768719841 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc500012250 con 0x5604271b6b40 2024-09-12T02:16:27.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.809+0000 7fc4f37fe640 1 -- 172.21.15.16:0/768719841 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc50001dd70 con 0x5604271b6b40 2024-09-12T02:16:27.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.809+0000 7fc50dd7e640 1 --2- 172.21.15.16:0/2350626140 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc4f40a2b80 0x7fc4f40a5040 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fc50001f000 tx=0x7fc50001d000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:27.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.809+0000 7fc511bba740 1 -- 172.21.15.16:0/768719841 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5604271b6b40 msgr2=0x5604271b5090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:27.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.809+0000 7fc511bba740 1 --2- 172.21.15.16:0/768719841 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5604271b6b40 0x5604271b5090 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7fc500012940 tx=0x7fc500012e00 comp rx=0 tx=0).stop 2024-09-12T02:16:27.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.809+0000 7fc511bba740 1 -- 172.21.15.16:0/768719841 shutdown_connections 2024-09-12T02:16:27.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.809+0000 7fc511bba740 1 --2- 172.21.15.16:0/768719841 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5604271c4780 0x5604271c6b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:27.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.809+0000 7fc511bba740 1 --2- 172.21.15.16:0/768719841 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5604271b55d0 0x5604271b5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:27.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.809+0000 7fc511bba740 1 --2- 172.21.15.16:0/768719841 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5604271b6b40 0x5604271b5090 unknown :-1 s=CLOSED pgs=1939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:27.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.809+0000 7fc511bba740 1 -- 172.21.15.16:0/768719841 >> 172.21.15.16:0/768719841 conn(0x56042717e310 msgr2=0x5604271b1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:27.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.810+0000 7fc511bba740 1 -- 172.21.15.16:0/768719841 shutdown_connections 2024-09-12T02:16:27.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.810+0000 7fc511bba740 1 -- 172.21.15.16:0/768719841 wait complete. 2024-09-12T02:16:27.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.810+0000 7fc511bba740 1 -- start start 2024-09-12T02:16:27.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.810+0000 7fc511bba740 1 Processor -- start 2024-09-12T02:16:27.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.810+0000 7fc511bba740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5604271b55d0 0x5604271c4100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:27.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.810+0000 7fc511bba740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5604271b6b40 0x5604271c1400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:27.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.810+0000 7fc511bba740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5604271c4780 0x5604271c1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:27.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.810+0000 7fc511bba740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5604271c7480 con 0x5604271b55d0 2024-09-12T02:16:27.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.810+0000 7fc511bba740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5604270d6060 con 0x5604271b6b40 2024-09-12T02:16:27.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.810+0000 7fc511bba740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5604271c7600 con 0x5604271c4780 2024-09-12T02:16:27.859 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T02:16:27.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.810+0000 7fc511bba740 10 client.0 _ll_register_callbacks cb 0x560426e298c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:16:27.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.811+0000 7fc50d57d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5604271b55d0 0x5604271c4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:27.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.811+0000 7fc50d57d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5604271b55d0 0x5604271c4100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40346/0 (socket says 172.21.15.16:40346) 2024-09-12T02:16:27.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.811+0000 7fc50d57d640 1 -- 172.21.15.16:0/2917341736 learned_addr learned my addr 172.21.15.16:0/2917341736 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:16:27.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.811+0000 7fc50dd7e640 1 --2- 172.21.15.16:0/2917341736 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5604271c4780 0x5604271c1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:27.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.811+0000 7fc50cd7c640 1 --2- 172.21.15.16:0/2917341736 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5604271b6b40 0x5604271c1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:27.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.811+0000 7fc50d57d640 1 -- 172.21.15.16:0/2917341736 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5604271c4780 msgr2=0x5604271c1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:27.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.811+0000 7fc50d57d640 1 --2- 172.21.15.16:0/2917341736 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5604271c4780 0x5604271c1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:27.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.811+0000 7fc50d57d640 1 -- 172.21.15.16:0/2917341736 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5604271b6b40 msgr2=0x5604271c1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:27.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.811+0000 7fc50d57d640 1 --2- 172.21.15.16:0/2917341736 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5604271b6b40 0x5604271c1400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:27.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.811+0000 7fc50d57d640 1 -- 172.21.15.16:0/2917341736 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560426e8c3c0 con 0x5604271b55d0 2024-09-12T02:16:27.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.811+0000 7fc50cd7c640 1 --2- 172.21.15.16:0/2917341736 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5604271b6b40 0x5604271c1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:16:27.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.811+0000 7fc50d57d640 1 --2- 172.21.15.16:0/2917341736 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5604271b55d0 0x5604271c4100 secure :-1 s=READY pgs=1940 cs=0 l=1 rev1=1 crypto rx=0x7fc50402a9a0 tx=0x7fc5040044f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef240c6f12cc2c85 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:27.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.811+0000 7fc4f0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:16:27.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.811+0000 7fc4f0ff9640 1 -- 172.21.15.16:0/2917341736 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc504035040 con 0x5604271b55d0 2024-09-12T02:16:27.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.811+0000 7fc511bba740 1 -- 172.21.15.16:0/2917341736 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc500012250 con 0x5604271b55d0 2024-09-12T02:16:27.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.811+0000 7fc4f0ff9640 1 -- 172.21.15.16:0/2917341736 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fc504003b40 con 0x5604271b55d0 2024-09-12T02:16:27.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.811+0000 7fc4f0ff9640 1 -- 172.21.15.16:0/2917341736 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc504003d40 con 0x5604271b55d0 2024-09-12T02:16:27.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.811+0000 7fc511bba740 10 client.20256 fetch_fsmap learned FSMap version 25 2024-09-12T02:16:27.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.811+0000 7fc511bba740 1 -- 172.21.15.16:0/2917341736 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5604271d2eb0 con 0x5604271b55d0 2024-09-12T02:16:27.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.812+0000 7fc4f0ff9640 1 -- 172.21.15.16:0/2917341736 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc500012250 con 0x5604271b55d0 2024-09-12T02:16:27.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.812+0000 7fc4f0ff9640 1 -- 172.21.15.16:0/2917341736 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fc50403c740 con 0x5604271b55d0 2024-09-12T02:16:27.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.812+0000 7fc511bba740 10 client.20256 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:16:27.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.812+0000 7fc511bba740 10 client.20256 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:16:27.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.812+0000 7fc511bba740 20 client.20256 populate_metadata read hostname 'smithi016' 2024-09-12T02:16:27.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.812+0000 7fc511bba740 1 --2- 172.21.15.16:0/2917341736 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5604271d43c0 0x5604271d67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:27.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.812+0000 7fc511bba740 4 client.20256 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:16:27.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.812+0000 7fc511bba740 1 -- 172.21.15.16:0/2917341736 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560427128860 con 0x5604271d43c0 2024-09-12T02:16:27.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.812+0000 7fc50cd7c640 1 --2- 172.21.15.16:0/2917341736 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5604271d43c0 0x5604271d67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:27.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.812+0000 7fc4f0ff9640 10 client.20256 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:16:27.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.812+0000 7fc50cd7c640 1 --2- 172.21.15.16:0/2917341736 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5604271d43c0 0x5604271d67f0 crc :-1 s=READY pgs=429 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3acc8fc3acec884 server_cookie=21ac8659049f0691 in_seq=0 out_seq=0 2024-09-12T02:16:27.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.813+0000 7fc4f0ff9640 1 -- 172.21.15.16:0/2917341736 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560427128860 con 0x5604271d43c0 2024-09-12T02:16:27.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.813+0000 7fc4f0ff9640 10 client.20256 handle_command_reply: tid=0 2024-09-12T02:16:27.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.813+0000 7fc511bba740 1 client.20256 shutdown 2024-09-12T02:16:27.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.813+0000 7fc511bba740 1 -- 172.21.15.16:0/2917341736 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5604271b55d0 msgr2=0x5604271c4100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:27.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.813+0000 7fc511bba740 1 --2- 172.21.15.16:0/2917341736 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5604271b55d0 0x5604271c4100 secure :-1 s=READY pgs=1940 cs=0 l=1 rev1=1 crypto rx=0x7fc50402a9a0 tx=0x7fc5040044f0 comp rx=0 tx=0).stop 2024-09-12T02:16:27.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.813+0000 7fc511bba740 1 -- 172.21.15.16:0/2917341736 shutdown_connections 2024-09-12T02:16:27.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.813+0000 7fc511bba740 1 --2- 172.21.15.16:0/2917341736 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5604271d43c0 0x5604271d67f0 crc :-1 s=READY pgs=429 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:27.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.813+0000 7fc511bba740 1 --2- 172.21.15.16:0/2917341736 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5604271c4780 0x5604271c1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:27.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.813+0000 7fc511bba740 1 --2- 172.21.15.16:0/2917341736 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5604271b6b40 0x5604271c1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:27.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.813+0000 7fc4f0ff9640 0 client.20256 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:16:27.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.813+0000 7fc511bba740 1 --2- 172.21.15.16:0/2917341736 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5604271b55d0 0x5604271c4100 unknown :-1 s=CLOSED pgs=1940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:27.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.813+0000 7fc511bba740 1 -- 172.21.15.16:0/2917341736 >> 172.21.15.16:0/2917341736 conn(0x56042717e310 msgr2=0x5604271b7960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:27.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.814+0000 7fc511bba740 1 -- 172.21.15.16:0/2917341736 shutdown_connections 2024-09-12T02:16:27.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.814+0000 7fc511bba740 1 -- 172.21.15.16:0/2917341736 wait complete. 2024-09-12T02:16:27.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.814+0000 7fc511bba740 20 client.20256 trim_cache size 0 max 16384 2024-09-12T02:16:27.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.827+0000 7fc511bba740 1 Processor -- start 2024-09-12T02:16:27.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.827+0000 7fc511bba740 1 -- start start 2024-09-12T02:16:27.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.827+0000 7fc511bba740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5604271b55d0 0x5604271d0720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:27.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.827+0000 7fc511bba740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5604271b6b40 0x5604271dc0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:27.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.827+0000 7fc511bba740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5604271c4780 0x5604271dc600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:27.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.827+0000 7fc511bba740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc5081ad9b0 con 0x5604271b55d0 2024-09-12T02:16:27.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.827+0000 7fc511bba740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc5081ab280 con 0x5604271b6b40 2024-09-12T02:16:27.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.827+0000 7fc511bba740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc5081a9110 con 0x5604271c4780 2024-09-12T02:16:27.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.827+0000 7fc50d57d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5604271b55d0 0x5604271d0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:27.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.827+0000 7fc50d57d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5604271b55d0 0x5604271d0720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40348/0 (socket says 172.21.15.16:40348) 2024-09-12T02:16:27.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.827+0000 7fc50d57d640 1 -- 172.21.15.16:0/3902408536 learned_addr learned my addr 172.21.15.16:0/3902408536 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:16:27.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.827+0000 7fc50d57d640 1 -- 172.21.15.16:0/3902408536 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5604271c4780 msgr2=0x5604271dc600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:27.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.827+0000 7fc50d57d640 1 --2- 172.21.15.16:0/3902408536 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5604271c4780 0x5604271dc600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:27.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.827+0000 7fc50d57d640 1 -- 172.21.15.16:0/3902408536 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5604271b6b40 msgr2=0x5604271dc0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:27.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.827+0000 7fc50d57d640 1 --2- 172.21.15.16:0/3902408536 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5604271b6b40 0x5604271dc0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:27.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.827+0000 7fc50d57d640 1 -- 172.21.15.16:0/3902408536 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604271c2010 con 0x5604271b55d0 2024-09-12T02:16:27.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.827+0000 7fc50d57d640 1 --2- 172.21.15.16:0/3902408536 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5604271b55d0 0x5604271d0720 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7fc5040314e0 tx=0x7fc50402a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dca12d0440e39f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:27.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.827+0000 7fc4f2ffd640 1 -- 172.21.15.16:0/3902408536 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc504035070 con 0x5604271b55d0 2024-09-12T02:16:27.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.828+0000 7fc4f2ffd640 1 -- 172.21.15.16:0/3902408536 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc50403c920 con 0x5604271b55d0 2024-09-12T02:16:27.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.828+0000 7fc4f2ffd640 1 -- 172.21.15.16:0/3902408536 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5040046c0 con 0x5604271b55d0 2024-09-12T02:16:27.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.828+0000 7fc511bba740 1 -- 172.21.15.16:0/3902408536 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5604271b55d0 msgr2=0x5604271d0720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:27.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.828+0000 7fc511bba740 1 --2- 172.21.15.16:0/3902408536 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5604271b55d0 0x5604271d0720 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7fc5040314e0 tx=0x7fc50402a480 comp rx=0 tx=0).stop 2024-09-12T02:16:27.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.828+0000 7fc511bba740 1 -- 172.21.15.16:0/3902408536 shutdown_connections 2024-09-12T02:16:27.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.828+0000 7fc511bba740 1 --2- 172.21.15.16:0/3902408536 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5604271c4780 0x5604271dc600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:27.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.828+0000 7fc511bba740 1 --2- 172.21.15.16:0/3902408536 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5604271b6b40 0x5604271dc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:27.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.828+0000 7fc511bba740 1 --2- 172.21.15.16:0/3902408536 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5604271b55d0 0x5604271d0720 unknown :-1 s=CLOSED pgs=1941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:27.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.828+0000 7fc511bba740 1 -- 172.21.15.16:0/3902408536 >> 172.21.15.16:0/3902408536 conn(0x56042717e310 msgr2=0x5604271c04f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:27.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.828+0000 7fc511bba740 1 -- 172.21.15.16:0/3902408536 shutdown_connections 2024-09-12T02:16:27.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.828+0000 7fc511bba740 1 -- 172.21.15.16:0/3902408536 wait complete. 2024-09-12T02:16:27.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.828+0000 7fc511bba740 1 -- start start 2024-09-12T02:16:27.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.829+0000 7fc511bba740 1 Processor -- start 2024-09-12T02:16:27.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.829+0000 7fc511bba740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5604271b55d0 0x5604271cc890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:27.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.829+0000 7fc511bba740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5604271b6b40 0x5604271ccdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:27.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.829+0000 7fc511bba740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5604271c4780 0x5604271c9860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:27.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.829+0000 7fc511bba740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc5081a9760 con 0x5604271c4780 2024-09-12T02:16:27.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.829+0000 7fc511bba740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc5081aabb0 con 0x5604271b55d0 2024-09-12T02:16:27.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.829+0000 7fc511bba740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc5081a9e10 con 0x5604271b6b40 2024-09-12T02:16:27.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.829+0000 7fc511bba740 10 client.0 _ll_register_callbacks cb 0x7fc504005d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:16:27.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.829+0000 7fc50dd7e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5604271c4780 0x5604271c9860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:27.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.829+0000 7fc50dd7e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5604271c4780 0x5604271c9860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40352/0 (socket says 172.21.15.16:40352) 2024-09-12T02:16:27.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.829+0000 7fc50dd7e640 1 -- 172.21.15.16:0/244091715 learned_addr learned my addr 172.21.15.16:0/244091715 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:16:27.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.829+0000 7fc50d57d640 1 --2- 172.21.15.16:0/244091715 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5604271b55d0 0x5604271cc890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:27.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.829+0000 7fc50dd7e640 1 -- 172.21.15.16:0/244091715 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5604271b6b40 msgr2=0x5604271ccdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:27.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.829+0000 7fc50cd7c640 1 --2- 172.21.15.16:0/244091715 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5604271b6b40 0x5604271ccdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:27.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.829+0000 7fc50dd7e640 1 --2- 172.21.15.16:0/244091715 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5604271b6b40 0x5604271ccdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:27.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.829+0000 7fc50dd7e640 1 -- 172.21.15.16:0/244091715 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5604271b55d0 msgr2=0x5604271cc890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:27.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.829+0000 7fc50dd7e640 1 --2- 172.21.15.16:0/244091715 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5604271b55d0 0x5604271cc890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:27.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.829+0000 7fc50dd7e640 1 -- 172.21.15.16:0/244091715 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5604271c23a0 con 0x5604271c4780 2024-09-12T02:16:27.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.829+0000 7fc50d57d640 1 --2- 172.21.15.16:0/244091715 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5604271b55d0 0x5604271cc890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:16:27.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.830+0000 7fc50dd7e640 1 --2- 172.21.15.16:0/244091715 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5604271c4780 0x5604271c9860 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7fc50001d9e0 tx=0x7fc50001d6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3592853e768a39d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:27.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.830+0000 7fc4f0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:16:27.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.830+0000 7fc4f0ff9640 1 -- 172.21.15.16:0/244091715 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc500022040 con 0x5604271c4780 2024-09-12T02:16:27.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.830+0000 7fc4f0ff9640 1 -- 172.21.15.16:0/244091715 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fc50001dc90 con 0x5604271c4780 2024-09-12T02:16:27.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.830+0000 7fc511bba740 1 -- 172.21.15.16:0/244091715 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc50403c920 con 0x5604271c4780 2024-09-12T02:16:27.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.830+0000 7fc4f0ff9640 1 -- 172.21.15.16:0/244091715 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc500028450 con 0x5604271c4780 2024-09-12T02:16:27.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.830+0000 7fc511bba740 10 client.20262 fetch_fsmap learned FSMap version 25 2024-09-12T02:16:27.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.830+0000 7fc511bba740 1 -- 172.21.15.16:0/244091715 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56042723f500 con 0x5604271c4780 2024-09-12T02:16:27.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.830+0000 7fc4f0ff9640 1 -- 172.21.15.16:0/244091715 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc50403c920 con 0x5604271c4780 2024-09-12T02:16:27.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.830+0000 7fc4f0ff9640 1 -- 172.21.15.16:0/244091715 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fc500029430 con 0x5604271c4780 2024-09-12T02:16:27.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.830+0000 7fc511bba740 10 client.20262 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:16:27.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.830+0000 7fc511bba740 10 client.20262 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:16:27.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.830+0000 7fc511bba740 20 client.20262 populate_metadata read hostname 'smithi016' 2024-09-12T02:16:27.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.830+0000 7fc511bba740 1 --2- 172.21.15.16:0/244091715 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x56042723fb30 0x560427241f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:27.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.830+0000 7fc511bba740 4 client.20262 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T02:16:27.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.830+0000 7fc511bba740 1 -- 172.21.15.16:0/244091715 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560427122d70 con 0x56042723fb30 2024-09-12T02:16:27.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.831+0000 7fc50d57d640 1 --2- 172.21.15.16:0/244091715 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x56042723fb30 0x560427241f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:27.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.831+0000 7fc4f0ff9640 10 client.20262 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:16:27.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.831+0000 7fc50d57d640 1 --2- 172.21.15.16:0/244091715 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x56042723fb30 0x560427241f50 crc :-1 s=READY pgs=430 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=daa13dd8b5455c7a server_cookie=5ed14b6382151d1e in_seq=0 out_seq=0 2024-09-12T02:16:27.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.831+0000 7fc4f0ff9640 1 -- 172.21.15.16:0/244091715 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560427122d70 con 0x56042723fb30 2024-09-12T02:16:27.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.831+0000 7fc4f0ff9640 10 client.20262 handle_command_reply: tid=0 2024-09-12T02:16:27.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.831+0000 7fc511bba740 1 client.20262 shutdown 2024-09-12T02:16:27.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.832+0000 7fc511bba740 1 -- 172.21.15.16:0/244091715 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5604271c4780 msgr2=0x5604271c9860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:27.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.832+0000 7fc511bba740 1 --2- 172.21.15.16:0/244091715 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5604271c4780 0x5604271c9860 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7fc50001d9e0 tx=0x7fc50001d6b0 comp rx=0 tx=0).stop 2024-09-12T02:16:27.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.832+0000 7fc511bba740 1 -- 172.21.15.16:0/244091715 shutdown_connections 2024-09-12T02:16:27.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.832+0000 7fc511bba740 1 --2- 172.21.15.16:0/244091715 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x56042723fb30 0x560427241f50 crc :-1 s=READY pgs=430 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:27.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.832+0000 7fc511bba740 1 --2- 172.21.15.16:0/244091715 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5604271c4780 0x5604271c9860 unknown :-1 s=CLOSED pgs=1942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:27.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.832+0000 7fc511bba740 1 --2- 172.21.15.16:0/244091715 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5604271b6b40 0x5604271ccdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:27.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.832+0000 7fc4f0ff9640 0 client.20262 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:16:27.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.832+0000 7fc511bba740 1 --2- 172.21.15.16:0/244091715 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5604271b55d0 0x5604271cc890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:27.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.832+0000 7fc511bba740 1 -- 172.21.15.16:0/244091715 >> 172.21.15.16:0/244091715 conn(0x56042717e310 msgr2=0x5604271b0500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:27.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.832+0000 7fc511bba740 1 -- 172.21.15.16:0/244091715 shutdown_connections 2024-09-12T02:16:27.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.832+0000 7fc511bba740 1 -- 172.21.15.16:0/244091715 wait complete. 2024-09-12T02:16:27.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.832+0000 7fc511bba740 20 client.20262 trim_cache size 0 max 16384 2024-09-12T02:16:27.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.832+0000 7fc50f7e1640 1 -- 172.21.15.16:0/2350626140 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc4f40a2b80 msgr2=0x7fc4f40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:27.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.832+0000 7fc50f7e1640 1 --2- 172.21.15.16:0/2350626140 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc4f40a2b80 0x7fc4f40a5040 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fc50001f000 tx=0x7fc50001d000 comp rx=0 tx=0).stop 2024-09-12T02:16:27.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.832+0000 7fc50f7e1640 1 -- 172.21.15.16:0/2350626140 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc508102570 msgr2=0x7fc5081997d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:27.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.832+0000 7fc50f7e1640 1 --2- 172.21.15.16:0/2350626140 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc508102570 0x7fc5081997d0 secure :-1 s=READY pgs=1938 cs=0 l=1 rev1=1 crypto rx=0x7fc4f800d950 tx=0x7fc4f800de10 comp rx=0 tx=0).stop 2024-09-12T02:16:27.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.832+0000 7fc50f7e1640 1 -- 172.21.15.16:0/2350626140 shutdown_connections 2024-09-12T02:16:27.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.832+0000 7fc50f7e1640 1 --2- 172.21.15.16:0/2350626140 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc4f40a2b80 0x7fc4f40a5040 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:27.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.832+0000 7fc50f7e1640 1 --2- 172.21.15.16:0/2350626140 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc508102ef0 0x7fc5081a4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:27.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.832+0000 7fc50f7e1640 1 --2- 172.21.15.16:0/2350626140 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc508102570 0x7fc5081997d0 unknown :-1 s=CLOSED pgs=1938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:27.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.832+0000 7fc50f7e1640 1 --2- 172.21.15.16:0/2350626140 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc508100e20 0x7fc508199290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:27.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.832+0000 7fc50f7e1640 1 -- 172.21.15.16:0/2350626140 >> 172.21.15.16:0/2350626140 conn(0x7fc5080fc7d0 msgr2=0x7fc5080fe060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:27.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.833+0000 7fc50f7e1640 1 -- 172.21.15.16:0/2350626140 shutdown_connections 2024-09-12T02:16:27.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:27.833+0000 7fc50f7e1640 1 -- 172.21.15.16:0/2350626140 wait complete. 2024-09-12T02:16:28.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:16:28 smithi016 ceph-mon[30947]: pgmap v771: 97 pgs: 97 active+clean; 3.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-09-12T02:16:28.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:16:28 smithi033 ceph-mon[35332]: pgmap v771: 97 pgs: 97 active+clean; 3.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-09-12T02:16:28.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:16:28 smithi184 ceph-mon[35593]: pgmap v771: 97 pgs: 97 active+clean; 3.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-09-12T02:16:29.570 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-12T02:16:29.571 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-12T02:16:29.849 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:16:30.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.299+0000 7f16726e4640 1 -- 172.21.15.16:0/3825566724 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f166c102ef0 msgr2=0x7f166c10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:30.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.299+0000 7f16726e4640 1 --2- 172.21.15.16:0/3825566724 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f166c102ef0 0x7f166c10da40 secure :-1 s=READY pgs=1943 cs=0 l=1 rev1=1 crypto rx=0x7f1664009ca0 tx=0x7f166402a950 comp rx=0 tx=0).stop 2024-09-12T02:16:30.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.299+0000 7f16726e4640 1 -- 172.21.15.16:0/3825566724 shutdown_connections 2024-09-12T02:16:30.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.299+0000 7f16726e4640 1 --2- 172.21.15.16:0/3825566724 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f166c102ef0 0x7f166c10da40 unknown :-1 s=CLOSED pgs=1943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:30.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.299+0000 7f16726e4640 1 --2- 172.21.15.16:0/3825566724 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f166c102570 0x7f166c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:30.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.299+0000 7f16726e4640 1 --2- 172.21.15.16:0/3825566724 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f166c100e20 0x7f166c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:30.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.299+0000 7f16726e4640 1 -- 172.21.15.16:0/3825566724 >> 172.21.15.16:0/3825566724 conn(0x7f166c0fc7d0 msgr2=0x7f166c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:30.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.299+0000 7f16726e4640 1 -- 172.21.15.16:0/3825566724 shutdown_connections 2024-09-12T02:16:30.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.299+0000 7f16726e4640 1 -- 172.21.15.16:0/3825566724 wait complete. 2024-09-12T02:16:30.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.300+0000 7f16726e4640 1 Processor -- start 2024-09-12T02:16:30.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.300+0000 7f16726e4640 1 -- start start 2024-09-12T02:16:30.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.300+0000 7f16726e4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f166c100e20 0x7f166c197330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:30.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.300+0000 7f16726e4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f166c102570 0x7f166c197870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:30.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.300+0000 7f16726e4640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f166c102ef0 0x7f166c19bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:30.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.300+0000 7f16726e4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f166c110fd0 con 0x7f166c100e20 2024-09-12T02:16:30.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.300+0000 7f16726e4640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f166c110e50 con 0x7f166c102570 2024-09-12T02:16:30.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.300+0000 7f16726e4640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f166c111150 con 0x7f166c102ef0 2024-09-12T02:16:30.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.300+0000 7f166bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f166c100e20 0x7f166c197330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:30.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.300+0000 7f166bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f166c100e20 0x7f166c197330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40382/0 (socket says 172.21.15.16:40382) 2024-09-12T02:16:30.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.300+0000 7f166bfff640 1 -- 172.21.15.16:0/2903466150 learned_addr learned my addr 172.21.15.16:0/2903466150 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:16:30.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.301+0000 7f1663fff640 1 --2- 172.21.15.16:0/2903466150 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f166c102570 0x7f166c197870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:30.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.301+0000 7f1670c81640 1 --2- 172.21.15.16:0/2903466150 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f166c102ef0 0x7f166c19bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:30.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.301+0000 7f166bfff640 1 -- 172.21.15.16:0/2903466150 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f166c102ef0 msgr2=0x7f166c19bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:30.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.301+0000 7f166bfff640 1 --2- 172.21.15.16:0/2903466150 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f166c102ef0 0x7f166c19bc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:30.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.301+0000 7f166bfff640 1 -- 172.21.15.16:0/2903466150 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f166c102570 msgr2=0x7f166c197870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:30.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.301+0000 7f166bfff640 1 --2- 172.21.15.16:0/2903466150 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f166c102570 0x7f166c197870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:30.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.301+0000 7f166bfff640 1 -- 172.21.15.16:0/2903466150 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f166c19c2e0 con 0x7f166c100e20 2024-09-12T02:16:30.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.301+0000 7f1670c81640 1 --2- 172.21.15.16:0/2903466150 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f166c102ef0 0x7f166c19bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:16:30.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.301+0000 7f166bfff640 1 --2- 172.21.15.16:0/2903466150 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f166c100e20 0x7f166c197330 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 crypto rx=0x7f165c00c970 tx=0x7f165c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:30.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.301+0000 7f1669ffb640 1 -- 172.21.15.16:0/2903466150 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f165c007bb0 con 0x7f166c100e20 2024-09-12T02:16:30.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.301+0000 7f1669ffb640 1 -- 172.21.15.16:0/2903466150 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f165c007d50 con 0x7f166c100e20 2024-09-12T02:16:30.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.301+0000 7f16726e4640 1 -- 172.21.15.16:0/2903466150 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f166c19c5d0 con 0x7f166c100e20 2024-09-12T02:16:30.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.301+0000 7f1669ffb640 1 -- 172.21.15.16:0/2903466150 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f165c00f450 con 0x7f166c100e20 2024-09-12T02:16:30.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.301+0000 7f16726e4640 1 -- 172.21.15.16:0/2903466150 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f166c1a3eb0 con 0x7f166c100e20 2024-09-12T02:16:30.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.304+0000 7f1669ffb640 1 -- 172.21.15.16:0/2903466150 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f165c005020 con 0x7f166c100e20 2024-09-12T02:16:30.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.304+0000 7f1674abd740 1 Processor -- start 2024-09-12T02:16:30.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.304+0000 7f1674abd740 1 -- start start 2024-09-12T02:16:30.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.304+0000 7f1674abd740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b2c2a68900 0x55b2c2a68ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:30.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.304+0000 7f1674abd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b2c2a64100 0x55b2c2a64560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:30.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.304+0000 7f1674abd740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b2c2a64aa0 0x55b2c2a70780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:30.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.304+0000 7f1674abd740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55b2c29c4280 con 0x55b2c2a68900 2024-09-12T02:16:30.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.304+0000 7f1674abd740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55b2c296bb10 con 0x55b2c2a64100 2024-09-12T02:16:30.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.304+0000 7f1674abd740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55b2c29c0240 con 0x55b2c2a64aa0 2024-09-12T02:16:30.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.305+0000 7f1670c81640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b2c2a68900 0x55b2c2a68ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:30.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.305+0000 7f1670c81640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b2c2a68900 0x55b2c2a68ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40398/0 (socket says 172.21.15.16:40398) 2024-09-12T02:16:30.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.305+0000 7f1670c81640 1 -- 172.21.15.16:0/3298348919 learned_addr learned my addr 172.21.15.16:0/3298348919 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:16:30.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.305+0000 7f166bfff640 1 --2- 172.21.15.16:0/3298348919 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b2c2a64100 0x55b2c2a64560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:30.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.305+0000 7f1663fff640 1 --2- 172.21.15.16:0/3298348919 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b2c2a64aa0 0x55b2c2a70780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:30.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.305+0000 7f1670c81640 1 -- 172.21.15.16:0/3298348919 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b2c2a64aa0 msgr2=0x55b2c2a70780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:30.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.305+0000 7f1670c81640 1 --2- 172.21.15.16:0/3298348919 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b2c2a64aa0 0x55b2c2a70780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:30.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.305+0000 7f1670c81640 1 -- 172.21.15.16:0/3298348919 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b2c2a64100 msgr2=0x55b2c2a64560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:30.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.305+0000 7f1669ffb640 1 --2- 172.21.15.16:0/2903466150 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f16580a2c00 0x7f16580a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:30.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.305+0000 7f1670c81640 1 --2- 172.21.15.16:0/3298348919 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b2c2a64100 0x55b2c2a64560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:30.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.305+0000 7f1670c81640 1 -- 172.21.15.16:0/3298348919 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2c29781f0 con 0x55b2c2a68900 2024-09-12T02:16:30.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.305+0000 7f1669ffb640 1 -- 172.21.15.16:0/2903466150 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f165c0ced10 con 0x7f166c100e20 2024-09-12T02:16:30.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.305+0000 7f1670c81640 1 --2- 172.21.15.16:0/3298348919 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b2c2a68900 0x55b2c2a68ce0 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7f1664035000 tx=0x7f166402f6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42b81d1e00200fb5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:30.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.305+0000 7f1670c81640 1 --2- 172.21.15.16:0/2903466150 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f16580a2c00 0x7f16580a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:30.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.305+0000 7f16627fc640 1 -- 172.21.15.16:0/3298348919 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1664049040 con 0x55b2c2a68900 2024-09-12T02:16:30.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.306+0000 7f16627fc640 1 -- 172.21.15.16:0/3298348919 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f166402fe10 con 0x55b2c2a68900 2024-09-12T02:16:30.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.306+0000 7f16627fc640 1 -- 172.21.15.16:0/3298348919 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1664034040 con 0x55b2c2a68900 2024-09-12T02:16:30.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.306+0000 7f1674abd740 1 -- 172.21.15.16:0/3298348919 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b2c2a68900 msgr2=0x55b2c2a68ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:30.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.306+0000 7f1670c81640 1 --2- 172.21.15.16:0/2903466150 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f16580a2c00 0x7f16580a50c0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f166402f8d0 tx=0x7f1664044830 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:30.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.306+0000 7f1674abd740 1 --2- 172.21.15.16:0/3298348919 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b2c2a68900 0x55b2c2a68ce0 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7f1664035000 tx=0x7f166402f6f0 comp rx=0 tx=0).stop 2024-09-12T02:16:30.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.306+0000 7f1674abd740 1 -- 172.21.15.16:0/3298348919 shutdown_connections 2024-09-12T02:16:30.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.306+0000 7f1674abd740 1 --2- 172.21.15.16:0/3298348919 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b2c2a64aa0 0x55b2c2a70780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:30.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.306+0000 7f1674abd740 1 --2- 172.21.15.16:0/3298348919 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b2c2a64100 0x55b2c2a64560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:30.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.306+0000 7f1674abd740 1 --2- 172.21.15.16:0/3298348919 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b2c2a68900 0x55b2c2a68ce0 unknown :-1 s=CLOSED pgs=1945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:30.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.306+0000 7f1674abd740 1 -- 172.21.15.16:0/3298348919 >> 172.21.15.16:0/3298348919 conn(0x55b2c2a5abe0 msgr2=0x55b2c2a5afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:30.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.306+0000 7f1674abd740 1 -- 172.21.15.16:0/3298348919 shutdown_connections 2024-09-12T02:16:30.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.307+0000 7f1674abd740 1 -- 172.21.15.16:0/3298348919 wait complete. 2024-09-12T02:16:30.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.307+0000 7f1674abd740 1 -- start start 2024-09-12T02:16:30.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.307+0000 7f1674abd740 1 Processor -- start 2024-09-12T02:16:30.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.307+0000 7f1674abd740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b2c2a64100 0x55b2c2a6db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:30.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.307+0000 7f1674abd740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b2c2a64aa0 0x55b2c2a6e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:30.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.307+0000 7f1674abd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b2c2a68900 0x55b2c2a6e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:30.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.307+0000 7f1674abd740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55b2c2a70cc0 con 0x55b2c2a64100 2024-09-12T02:16:30.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.307+0000 7f1674abd740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55b2c297ef10 con 0x55b2c2a68900 2024-09-12T02:16:30.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.307+0000 7f1674abd740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55b2c2a70e40 con 0x55b2c2a64aa0 2024-09-12T02:16:30.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.307+0000 7f1674abd740 10 client.0 _ll_register_callbacks cb 0x55b2c2947a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:16:30.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.307+0000 7f166bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b2c2a64100 0x55b2c2a6db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:30.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.307+0000 7f166bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b2c2a64100 0x55b2c2a6db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40410/0 (socket says 172.21.15.16:40410) 2024-09-12T02:16:30.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.308+0000 7f166bfff640 1 -- 172.21.15.16:0/3500856336 learned_addr learned my addr 172.21.15.16:0/3500856336 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:16:30.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.308+0000 7f1663fff640 1 --2- 172.21.15.16:0/3500856336 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b2c2a64aa0 0x55b2c2a6e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:30.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.308+0000 7f1670c81640 1 --2- 172.21.15.16:0/3500856336 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b2c2a68900 0x55b2c2a6e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:30.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.308+0000 7f166bfff640 1 -- 172.21.15.16:0/3500856336 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b2c2a64aa0 msgr2=0x55b2c2a6e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:30.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.308+0000 7f166bfff640 1 --2- 172.21.15.16:0/3500856336 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b2c2a64aa0 0x55b2c2a6e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:30.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.308+0000 7f166bfff640 1 -- 172.21.15.16:0/3500856336 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b2c2a68900 msgr2=0x55b2c2a6e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:30.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.308+0000 7f166bfff640 1 --2- 172.21.15.16:0/3500856336 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b2c2a68900 0x55b2c2a6e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:30.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.308+0000 7f166bfff640 1 -- 172.21.15.16:0/3500856336 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2c27356a0 con 0x55b2c2a64100 2024-09-12T02:16:30.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.308+0000 7f166bfff640 1 --2- 172.21.15.16:0/3500856336 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b2c2a64100 0x55b2c2a6db60 secure :-1 s=READY pgs=1946 cs=0 l=1 rev1=1 crypto rx=0x7f165c0058c0 tx=0x7f165c0971f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=780d8f56244675e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:30.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.308+0000 7f164b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:16:30.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.308+0000 7f164b7fe640 1 -- 172.21.15.16:0/3500856336 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f165c01d040 con 0x55b2c2a64100 2024-09-12T02:16:30.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.308+0000 7f164b7fe640 1 -- 172.21.15.16:0/3500856336 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f165c005310 con 0x55b2c2a64100 2024-09-12T02:16:30.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.308+0000 7f1674abd740 1 -- 172.21.15.16:0/3500856336 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f166402fe10 con 0x55b2c2a64100 2024-09-12T02:16:30.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.308+0000 7f164b7fe640 1 -- 172.21.15.16:0/3500856336 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f165c09b3e0 con 0x55b2c2a64100 2024-09-12T02:16:30.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.308+0000 7f1674abd740 10 client.20274 fetch_fsmap learned FSMap version 25 2024-09-12T02:16:30.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.308+0000 7f1674abd740 1 -- 172.21.15.16:0/3500856336 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b2c2a7c8f0 con 0x55b2c2a64100 2024-09-12T02:16:30.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.308+0000 7f164b7fe640 1 -- 172.21.15.16:0/3500856336 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f166402fe10 con 0x55b2c2a64100 2024-09-12T02:16:30.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.309+0000 7f164b7fe640 1 -- 172.21.15.16:0/3500856336 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f165c084ba0 con 0x55b2c2a64100 2024-09-12T02:16:30.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.309+0000 7f1674abd740 10 client.20274 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:16:30.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.309+0000 7f1674abd740 10 client.20274 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:16:30.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.309+0000 7f1674abd740 20 client.20274 populate_metadata read hostname 'smithi016' 2024-09-12T02:16:30.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.309+0000 7f1674abd740 1 --2- 172.21.15.16:0/3500856336 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55b2c2a7de00 0x55b2c2a80230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:30.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.309+0000 7f1674abd740 4 client.20274 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:16:30.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.309+0000 7f1674abd740 1 -- 172.21.15.16:0/3500856336 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b2c29d17f0 con 0x55b2c2a7de00 2024-09-12T02:16:30.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.309+0000 7f1663fff640 1 --2- 172.21.15.16:0/3500856336 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55b2c2a7de00 0x55b2c2a80230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:30.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.310+0000 7f164b7fe640 10 client.20274 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:16:30.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.310+0000 7f1663fff640 1 --2- 172.21.15.16:0/3500856336 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55b2c2a7de00 0x55b2c2a80230 crc :-1 s=READY pgs=431 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=795c07bf2c930096 server_cookie=7b44a2d6412c9c31 in_seq=0 out_seq=0 2024-09-12T02:16:30.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.310+0000 7f164b7fe640 1 -- 172.21.15.16:0/3500856336 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b2c29d17f0 con 0x55b2c2a7de00 2024-09-12T02:16:30.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.310+0000 7f164b7fe640 10 client.20274 handle_command_reply: tid=0 2024-09-12T02:16:30.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.310+0000 7f1674abd740 1 client.20274 shutdown 2024-09-12T02:16:30.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.310+0000 7f1674abd740 1 -- 172.21.15.16:0/3500856336 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b2c2a64100 msgr2=0x55b2c2a6db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:30.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.310+0000 7f1674abd740 1 --2- 172.21.15.16:0/3500856336 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b2c2a64100 0x55b2c2a6db60 secure :-1 s=READY pgs=1946 cs=0 l=1 rev1=1 crypto rx=0x7f165c0058c0 tx=0x7f165c0971f0 comp rx=0 tx=0).stop 2024-09-12T02:16:30.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.311+0000 7f1674abd740 1 -- 172.21.15.16:0/3500856336 shutdown_connections 2024-09-12T02:16:30.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.311+0000 7f1674abd740 1 --2- 172.21.15.16:0/3500856336 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55b2c2a7de00 0x55b2c2a80230 crc :-1 s=READY pgs=431 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:30.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.311+0000 7f1674abd740 1 --2- 172.21.15.16:0/3500856336 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b2c2a68900 0x55b2c2a6e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:30.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.311+0000 7f1674abd740 1 --2- 172.21.15.16:0/3500856336 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b2c2a64aa0 0x55b2c2a6e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:30.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.311+0000 7f1674abd740 1 --2- 172.21.15.16:0/3500856336 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b2c2a64100 0x55b2c2a6db60 unknown :-1 s=CLOSED pgs=1946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:30.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.311+0000 7f1674abd740 1 -- 172.21.15.16:0/3500856336 >> 172.21.15.16:0/3500856336 conn(0x55b2c2a5abe0 msgr2=0x55b2c2a5a270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:30.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.311+0000 7f164b7fe640 0 client.20274 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:16:30.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.311+0000 7f1674abd740 1 -- 172.21.15.16:0/3500856336 shutdown_connections 2024-09-12T02:16:30.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.311+0000 7f1674abd740 1 -- 172.21.15.16:0/3500856336 wait complete. 2024-09-12T02:16:30.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.311+0000 7f1674abd740 20 client.20274 trim_cache size 0 max 16384 2024-09-12T02:16:30.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.346+0000 7f1674abd740 1 Processor -- start 2024-09-12T02:16:30.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.346+0000 7f1674abd740 1 -- start start 2024-09-12T02:16:30.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.346+0000 7f1674abd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b2c2a64100 0x55b2c2a662b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:30.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.346+0000 7f1674abd740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b2c2a64aa0 0x55b2c2a6cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:30.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.346+0000 7f1674abd740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b2c2a68900 0x55b2c2a630f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:30.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.346+0000 7f1674abd740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f166c1adaa0 con 0x55b2c2a64aa0 2024-09-12T02:16:30.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.346+0000 7f1674abd740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f166c1ab370 con 0x55b2c2a64100 2024-09-12T02:16:30.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.346+0000 7f1674abd740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f166c1a9200 con 0x55b2c2a68900 2024-09-12T02:16:30.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.346+0000 7f1663fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b2c2a64aa0 0x55b2c2a6cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:30.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.346+0000 7f1663fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b2c2a64aa0 0x55b2c2a6cb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40426/0 (socket says 172.21.15.16:40426) 2024-09-12T02:16:30.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.346+0000 7f1663fff640 1 -- 172.21.15.16:0/3553096815 learned_addr learned my addr 172.21.15.16:0/3553096815 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:16:30.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.346+0000 7f166bfff640 1 --2- 172.21.15.16:0/3553096815 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b2c2a64100 0x55b2c2a662b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:30.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.346+0000 7f1670c81640 1 --2- 172.21.15.16:0/3553096815 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b2c2a68900 0x55b2c2a630f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:30.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.347+0000 7f1663fff640 1 -- 172.21.15.16:0/3553096815 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b2c2a68900 msgr2=0x55b2c2a630f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:30.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.347+0000 7f1663fff640 1 --2- 172.21.15.16:0/3553096815 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b2c2a68900 0x55b2c2a630f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:30.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.347+0000 7f1663fff640 1 -- 172.21.15.16:0/3553096815 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b2c2a64100 msgr2=0x55b2c2a662b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:30.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.347+0000 7f1663fff640 1 --2- 172.21.15.16:0/3553096815 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b2c2a64100 0x55b2c2a662b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:30.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.347+0000 7f1663fff640 1 -- 172.21.15.16:0/3553096815 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2c2a56830 con 0x55b2c2a64aa0 2024-09-12T02:16:30.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.347+0000 7f1663fff640 1 --2- 172.21.15.16:0/3553096815 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b2c2a64aa0 0x55b2c2a6cb40 secure :-1 s=READY pgs=1947 cs=0 l=1 rev1=1 crypto rx=0x7f166c0feca0 tx=0x7f1654013830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef9fe4fa1667e632 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:30.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.347+0000 7f1661ffb640 1 -- 172.21.15.16:0/3553096815 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1654004020 con 0x55b2c2a64aa0 2024-09-12T02:16:30.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.347+0000 7f1661ffb640 1 -- 172.21.15.16:0/3553096815 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16540041c0 con 0x55b2c2a64aa0 2024-09-12T02:16:30.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.347+0000 7f1661ffb640 1 -- 172.21.15.16:0/3553096815 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f165401c6c0 con 0x55b2c2a64aa0 2024-09-12T02:16:30.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.348+0000 7f1674abd740 1 -- 172.21.15.16:0/3553096815 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b2c2a64aa0 msgr2=0x55b2c2a6cb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:30.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.348+0000 7f1674abd740 1 --2- 172.21.15.16:0/3553096815 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b2c2a64aa0 0x55b2c2a6cb40 secure :-1 s=READY pgs=1947 cs=0 l=1 rev1=1 crypto rx=0x7f166c0feca0 tx=0x7f1654013830 comp rx=0 tx=0).stop 2024-09-12T02:16:30.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.348+0000 7f1674abd740 1 -- 172.21.15.16:0/3553096815 shutdown_connections 2024-09-12T02:16:30.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.348+0000 7f1674abd740 1 --2- 172.21.15.16:0/3553096815 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b2c2a68900 0x55b2c2a630f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:30.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.348+0000 7f1674abd740 1 --2- 172.21.15.16:0/3553096815 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b2c2a64aa0 0x55b2c2a6cb40 unknown :-1 s=CLOSED pgs=1947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:30.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.348+0000 7f1674abd740 1 --2- 172.21.15.16:0/3553096815 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b2c2a64100 0x55b2c2a662b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:30.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.348+0000 7f1674abd740 1 -- 172.21.15.16:0/3553096815 >> 172.21.15.16:0/3553096815 conn(0x55b2c2a5abe0 msgr2=0x55b2c2a6c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:30.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.348+0000 7f1674abd740 1 -- 172.21.15.16:0/3553096815 shutdown_connections 2024-09-12T02:16:30.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.348+0000 7f1674abd740 1 -- 172.21.15.16:0/3553096815 wait complete. 2024-09-12T02:16:30.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.349+0000 7f1674abd740 1 -- start start 2024-09-12T02:16:30.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.349+0000 7f1674abd740 1 Processor -- start 2024-09-12T02:16:30.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.349+0000 7f1674abd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b2c2a64100 0x55b2c2a760d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:30.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.349+0000 7f1674abd740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b2c2a64aa0 0x55b2c2a76610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:30.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.349+0000 7f1674abd740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b2c2a68900 0x55b2c2a84af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:30.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.349+0000 7f1674abd740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f166c1a9850 con 0x55b2c2a68900 2024-09-12T02:16:30.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.349+0000 7f1674abd740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f166c1aaca0 con 0x55b2c2a64100 2024-09-12T02:16:30.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.349+0000 7f1674abd740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f166c1a9f00 con 0x55b2c2a64aa0 2024-09-12T02:16:30.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.349+0000 7f1674abd740 10 client.0 _ll_register_callbacks cb 0x7f165c0d2520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:16:30.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.349+0000 7f1670c81640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b2c2a68900 0x55b2c2a84af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:30.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.349+0000 7f1670c81640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b2c2a68900 0x55b2c2a84af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40440/0 (socket says 172.21.15.16:40440) 2024-09-12T02:16:30.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.349+0000 7f1670c81640 1 -- 172.21.15.16:0/2479461946 learned_addr learned my addr 172.21.15.16:0/2479461946 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:16:30.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.350+0000 7f166bfff640 1 --2- 172.21.15.16:0/2479461946 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b2c2a64100 0x55b2c2a760d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:30.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.350+0000 7f1663fff640 1 --2- 172.21.15.16:0/2479461946 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b2c2a64aa0 0x55b2c2a76610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:30.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.350+0000 7f1670c81640 1 -- 172.21.15.16:0/2479461946 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b2c2a64aa0 msgr2=0x55b2c2a76610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:30.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.350+0000 7f1670c81640 1 --2- 172.21.15.16:0/2479461946 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b2c2a64aa0 0x55b2c2a76610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:30.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.350+0000 7f1670c81640 1 -- 172.21.15.16:0/2479461946 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b2c2a64100 msgr2=0x55b2c2a760d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:30.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.350+0000 7f1670c81640 1 --2- 172.21.15.16:0/2479461946 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b2c2a64100 0x55b2c2a760d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:30.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.350+0000 7f1670c81640 1 -- 172.21.15.16:0/2479461946 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2c2a85160 con 0x55b2c2a68900 2024-09-12T02:16:30.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.350+0000 7f1663fff640 1 --2- 172.21.15.16:0/2479461946 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b2c2a64aa0 0x55b2c2a76610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:16:30.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.350+0000 7f1670c81640 1 --2- 172.21.15.16:0/2479461946 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b2c2a68900 0x55b2c2a84af0 secure :-1 s=READY pgs=1948 cs=0 l=1 rev1=1 crypto rx=0x7f1664009ca0 tx=0x7f1664042000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3acd290d7b508604 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:30.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.350+0000 7f164b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:16:30.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.350+0000 7f164b7fe640 1 -- 172.21.15.16:0/2479461946 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1664056040 con 0x55b2c2a68900 2024-09-12T02:16:30.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.350+0000 7f164b7fe640 1 -- 172.21.15.16:0/2479461946 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f166402f930 con 0x55b2c2a68900 2024-09-12T02:16:30.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.350+0000 7f1674abd740 1 -- 172.21.15.16:0/2479461946 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16540041c0 con 0x55b2c2a68900 2024-09-12T02:16:30.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.351+0000 7f164b7fe640 1 -- 172.21.15.16:0/2479461946 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1664034040 con 0x55b2c2a68900 2024-09-12T02:16:30.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.351+0000 7f1674abd740 10 client.20280 fetch_fsmap learned FSMap version 25 2024-09-12T02:16:30.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.351+0000 7f1674abd740 1 -- 172.21.15.16:0/2479461946 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b2c2a864a0 con 0x55b2c2a68900 2024-09-12T02:16:30.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.351+0000 7f164b7fe640 1 -- 172.21.15.16:0/2479461946 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16540041c0 con 0x55b2c2a68900 2024-09-12T02:16:30.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.351+0000 7f164b7fe640 1 -- 172.21.15.16:0/2479461946 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f1664042070 con 0x55b2c2a68900 2024-09-12T02:16:30.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.351+0000 7f1674abd740 10 client.20280 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:16:30.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.351+0000 7f1674abd740 10 client.20280 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:16:30.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.351+0000 7f1674abd740 20 client.20280 populate_metadata read hostname 'smithi016' 2024-09-12T02:16:30.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.351+0000 7f1674abd740 1 --2- 172.21.15.16:0/2479461946 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55b2c2ae9ae0 0x55b2c2aebed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:30.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.351+0000 7f1674abd740 4 client.20280 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-12T02:16:30.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.351+0000 7f1674abd740 1 -- 172.21.15.16:0/2479461946 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55b2c29cbd00 con 0x55b2c2ae9ae0 2024-09-12T02:16:30.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.351+0000 7f166bfff640 1 --2- 172.21.15.16:0/2479461946 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55b2c2ae9ae0 0x55b2c2aebed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:30.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.352+0000 7f166bfff640 1 --2- 172.21.15.16:0/2479461946 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55b2c2ae9ae0 0x55b2c2aebed0 crc :-1 s=READY pgs=432 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec962c9abc3fb8f9 server_cookie=80db32a66a49e5b3 in_seq=0 out_seq=0 2024-09-12T02:16:30.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.352+0000 7f164b7fe640 10 client.20280 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:16:30.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.353+0000 7f164b7fe640 1 -- 172.21.15.16:0/2479461946 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55b2c29cbd00 con 0x55b2c2ae9ae0 2024-09-12T02:16:30.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.353+0000 7f164b7fe640 10 client.20280 handle_command_reply: tid=0 2024-09-12T02:16:30.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.353+0000 7f1674abd740 1 client.20280 shutdown 2024-09-12T02:16:30.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.353+0000 7f1674abd740 1 -- 172.21.15.16:0/2479461946 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b2c2a68900 msgr2=0x55b2c2a84af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:30.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.353+0000 7f1674abd740 1 --2- 172.21.15.16:0/2479461946 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b2c2a68900 0x55b2c2a84af0 secure :-1 s=READY pgs=1948 cs=0 l=1 rev1=1 crypto rx=0x7f1664009ca0 tx=0x7f1664042000 comp rx=0 tx=0).stop 2024-09-12T02:16:30.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.353+0000 7f1674abd740 1 -- 172.21.15.16:0/2479461946 shutdown_connections 2024-09-12T02:16:30.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.353+0000 7f1674abd740 1 --2- 172.21.15.16:0/2479461946 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55b2c2ae9ae0 0x55b2c2aebed0 crc :-1 s=READY pgs=432 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:30.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.353+0000 7f1674abd740 1 --2- 172.21.15.16:0/2479461946 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b2c2a68900 0x55b2c2a84af0 unknown :-1 s=CLOSED pgs=1948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:30.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.353+0000 7f1674abd740 1 --2- 172.21.15.16:0/2479461946 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b2c2a64aa0 0x55b2c2a76610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:30.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.353+0000 7f1674abd740 1 --2- 172.21.15.16:0/2479461946 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b2c2a64100 0x55b2c2a760d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:30.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.353+0000 7f1674abd740 1 -- 172.21.15.16:0/2479461946 >> 172.21.15.16:0/2479461946 conn(0x55b2c2a5abe0 msgr2=0x55b2c2a6cc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:30.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.353+0000 7f164b7fe640 0 client.20280 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:16:30.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.353+0000 7f1674abd740 1 -- 172.21.15.16:0/2479461946 shutdown_connections 2024-09-12T02:16:30.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.353+0000 7f1674abd740 1 -- 172.21.15.16:0/2479461946 wait complete. 2024-09-12T02:16:30.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.354+0000 7f1674abd740 20 client.20280 trim_cache size 0 max 16384 2024-09-12T02:16:30.360 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T02:16:30.360 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-09-12T02:16:30.360 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "ed655ed4-415b-4bf0-a8cd-4a3ec3ada3a3", 2024-09-12T02:16:30.360 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-09-12T02:16:30.360 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T02:16:30.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.354+0000 7f16726e4640 1 -- 172.21.15.16:0/2903466150 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f16580a2c00 msgr2=0x7f16580a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:30.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.354+0000 7f16726e4640 1 --2- 172.21.15.16:0/2903466150 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f16580a2c00 0x7f16580a50c0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f166402f8d0 tx=0x7f1664044830 comp rx=0 tx=0).stop 2024-09-12T02:16:30.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.354+0000 7f16726e4640 1 -- 172.21.15.16:0/2903466150 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f166c100e20 msgr2=0x7f166c197330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:30.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.354+0000 7f16726e4640 1 --2- 172.21.15.16:0/2903466150 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f166c100e20 0x7f166c197330 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 crypto rx=0x7f165c00c970 tx=0x7f165c00ce30 comp rx=0 tx=0).stop 2024-09-12T02:16:30.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.354+0000 7f16726e4640 1 -- 172.21.15.16:0/2903466150 shutdown_connections 2024-09-12T02:16:30.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.354+0000 7f16726e4640 1 --2- 172.21.15.16:0/2903466150 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f16580a2c00 0x7f16580a50c0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:30.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.354+0000 7f16726e4640 1 --2- 172.21.15.16:0/2903466150 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f166c102ef0 0x7f166c19bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:30.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.354+0000 7f16726e4640 1 --2- 172.21.15.16:0/2903466150 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f166c102570 0x7f166c197870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:30.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.354+0000 7f16726e4640 1 --2- 172.21.15.16:0/2903466150 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f166c100e20 0x7f166c197330 unknown :-1 s=CLOSED pgs=1944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:30.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.354+0000 7f16726e4640 1 -- 172.21.15.16:0/2903466150 >> 172.21.15.16:0/2903466150 conn(0x7f166c0fc7d0 msgr2=0x7f166c0ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:30.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.354+0000 7f16726e4640 1 -- 172.21.15.16:0/2903466150 shutdown_connections 2024-09-12T02:16:30.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:30.354+0000 7f16726e4640 1 -- 172.21.15.16:0/2903466150 wait complete. 2024-09-12T02:16:30.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:16:30 smithi033 ceph-mon[35332]: pgmap v772: 97 pgs: 97 active+clean; 3.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 11 op/s 2024-09-12T02:16:30.672 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:16:30 smithi016 ceph-mon[30947]: pgmap v772: 97 pgs: 97 active+clean; 3.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 11 op/s 2024-09-12T02:16:30.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:16:30 smithi184 ceph-mon[35593]: pgmap v772: 97 pgs: 97 active+clean; 3.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 11 op/s 2024-09-12T02:16:31.084 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T02:16:31.343 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:16:31.647 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:16:31 smithi016 ceph-mon[30947]: scrub queued for path: / 2024-09-12T02:16:31.647 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:16:31 smithi016 ceph-mon[30947]: scrub summary: idle+waiting paths [/] 2024-09-12T02:16:31.647 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:16:31 smithi016 ceph-mon[30947]: scrub summary: active paths [/] 2024-09-12T02:16:31.647 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:16:31 smithi016 ceph-mon[30947]: scrub summary: idle+waiting paths [/] 2024-09-12T02:16:31.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:16:31 smithi033 ceph-mon[35332]: scrub queued for path: / 2024-09-12T02:16:31.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:16:31 smithi033 ceph-mon[35332]: scrub summary: idle+waiting paths [/] 2024-09-12T02:16:31.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:16:31 smithi033 ceph-mon[35332]: scrub summary: active paths [/] 2024-09-12T02:16:31.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:16:31 smithi033 ceph-mon[35332]: scrub summary: idle+waiting paths [/] 2024-09-12T02:16:31.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.828+0000 7f0703c49640 1 -- 172.21.15.16:0/641016338 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06fc106a80 msgr2=0x7f06fc106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:31.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.828+0000 7f0703c49640 1 --2- 172.21.15.16:0/641016338 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06fc106a80 0x7f06fc106e60 secure :-1 s=READY pgs=1949 cs=0 l=1 rev1=1 crypto rx=0x7f06f8009cd0 tx=0x7f06f802a8e0 comp rx=0 tx=0).stop 2024-09-12T02:16:31.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.828+0000 7f0703c49640 1 -- 172.21.15.16:0/641016338 shutdown_connections 2024-09-12T02:16:31.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.828+0000 7f0703c49640 1 --2- 172.21.15.16:0/641016338 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f06fc101420 0x7f06fc10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:31.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.828+0000 7f0703c49640 1 --2- 172.21.15.16:0/641016338 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f06fc100a80 0x7f06fc100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:31.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.828+0000 7f0703c49640 1 --2- 172.21.15.16:0/641016338 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06fc106a80 0x7f06fc106e60 unknown :-1 s=CLOSED pgs=1949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:31.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.828+0000 7f0703c49640 1 -- 172.21.15.16:0/641016338 >> 172.21.15.16:0/641016338 conn(0x7f06fc0fc7a0 msgr2=0x7f06fc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:31.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.828+0000 7f0703c49640 1 -- 172.21.15.16:0/641016338 shutdown_connections 2024-09-12T02:16:31.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.828+0000 7f0703c49640 1 -- 172.21.15.16:0/641016338 wait complete. 2024-09-12T02:16:31.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.829+0000 7f0703c49640 1 Processor -- start 2024-09-12T02:16:31.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.829+0000 7f0703c49640 1 -- start start 2024-09-12T02:16:31.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.829+0000 7f0703c49640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f06fc100a80 0x7f06fc0fce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:31.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.829+0000 7f0703c49640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06fc101420 0x7f06fc0fd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:31.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.829+0000 7f0703c49640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f06fc106a80 0x7f06fc0fd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:31.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.829+0000 7f0703c49640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f06fc110e80 con 0x7f06fc101420 2024-09-12T02:16:31.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.829+0000 7f0703c49640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f06fc110d00 con 0x7f06fc106a80 2024-09-12T02:16:31.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.829+0000 7f0703c49640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f06fc111000 con 0x7f06fc100a80 2024-09-12T02:16:31.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.829+0000 7f07011e4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06fc101420 0x7f06fc0fd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:31.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.829+0000 7f07011e4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06fc101420 0x7f06fc0fd3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53586/0 (socket says 172.21.15.16:53586) 2024-09-12T02:16:31.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.829+0000 7f07011e4640 1 -- 172.21.15.16:0/2256397790 learned_addr learned my addr 172.21.15.16:0/2256397790 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:16:31.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.829+0000 7f07019e5640 1 --2- 172.21.15.16:0/2256397790 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f06fc100a80 0x7f06fc0fce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:31.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.829+0000 7f07021e6640 1 --2- 172.21.15.16:0/2256397790 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f06fc106a80 0x7f06fc0fd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:31.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.830+0000 7f07011e4640 1 -- 172.21.15.16:0/2256397790 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f06fc100a80 msgr2=0x7f06fc0fce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:31.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.830+0000 7f07011e4640 1 --2- 172.21.15.16:0/2256397790 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f06fc100a80 0x7f06fc0fce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:31.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.830+0000 7f07011e4640 1 -- 172.21.15.16:0/2256397790 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f06fc106a80 msgr2=0x7f06fc0fd8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:31.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.830+0000 7f07011e4640 1 --2- 172.21.15.16:0/2256397790 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f06fc106a80 0x7f06fc0fd8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:31.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.830+0000 7f07011e4640 1 -- 172.21.15.16:0/2256397790 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06fc1a69c0 con 0x7f06fc101420 2024-09-12T02:16:31.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.830+0000 7f07019e5640 1 --2- 172.21.15.16:0/2256397790 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f06fc100a80 0x7f06fc0fce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:16:31.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.830+0000 7f07021e6640 1 --2- 172.21.15.16:0/2256397790 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f06fc106a80 0x7f06fc0fd8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:16:31.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.830+0000 7f07011e4640 1 --2- 172.21.15.16:0/2256397790 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06fc101420 0x7f06fc0fd3b0 secure :-1 s=READY pgs=1950 cs=0 l=1 rev1=1 crypto rx=0x7f06f000d770 tx=0x7f06f000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:31.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.830+0000 7f06eeffd640 1 -- 172.21.15.16:0/2256397790 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06f0017070 con 0x7f06fc101420 2024-09-12T02:16:31.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.830+0000 7f0703c49640 1 -- 172.21.15.16:0/2256397790 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06fc1a6cb0 con 0x7f06fc101420 2024-09-12T02:16:31.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.830+0000 7f06eeffd640 1 -- 172.21.15.16:0/2256397790 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06f0005500 con 0x7f06fc101420 2024-09-12T02:16:31.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.830+0000 7f06eeffd640 1 -- 172.21.15.16:0/2256397790 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06f0004e00 con 0x7f06fc101420 2024-09-12T02:16:31.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.830+0000 7f0703c49640 1 -- 172.21.15.16:0/2256397790 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06fc1a71a0 con 0x7f06fc101420 2024-09-12T02:16:31.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.832+0000 7f06eeffd640 1 -- 172.21.15.16:0/2256397790 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f06f0004fa0 con 0x7f06fc101420 2024-09-12T02:16:31.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.833+0000 7f0706022740 1 Processor -- start 2024-09-12T02:16:31.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.833+0000 7f0706022740 1 -- start start 2024-09-12T02:16:31.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.833+0000 7f0706022740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fcde89e4a0 0x55fcde89e880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:31.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.833+0000 7f0706022740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fcde89edc0 0x55fcde8a9140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:31.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.833+0000 7f0706022740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fcde8a9680 0x55fcde8aba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:31.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.833+0000 7f0706022740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55fcde8042e0 con 0x55fcde89edc0 2024-09-12T02:16:31.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.833+0000 7f0706022740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55fcde7abce0 con 0x55fcde8a9680 2024-09-12T02:16:31.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.833+0000 7f0706022740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55fcde8002a0 con 0x55fcde89e4a0 2024-09-12T02:16:31.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.833+0000 7f07019e5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fcde89edc0 0x55fcde8a9140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:31.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.833+0000 7f07019e5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fcde89edc0 0x55fcde8a9140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53590/0 (socket says 172.21.15.16:53590) 2024-09-12T02:16:31.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.833+0000 7f07019e5640 1 -- 172.21.15.16:0/2348939980 learned_addr learned my addr 172.21.15.16:0/2348939980 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:16:31.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.833+0000 7f06eeffd640 1 --2- 172.21.15.16:0/2256397790 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f06e80a2b80 0x7f06e80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:31.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.833+0000 7f07021e6640 1 --2- 172.21.15.16:0/2348939980 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fcde89e4a0 0x55fcde89e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:31.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.833+0000 7f07011e4640 1 --2- 172.21.15.16:0/2348939980 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fcde8a9680 0x55fcde8aba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:31.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.833+0000 7f07021e6640 1 --2- 172.21.15.16:0/2256397790 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f06e80a2b80 0x7f06e80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:31.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.834+0000 7f07019e5640 1 -- 172.21.15.16:0/2348939980 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fcde89e4a0 msgr2=0x55fcde89e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:31.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.834+0000 7f07019e5640 1 --2- 172.21.15.16:0/2348939980 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fcde89e4a0 0x55fcde89e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:31.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.834+0000 7f07019e5640 1 -- 172.21.15.16:0/2348939980 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fcde8a9680 msgr2=0x55fcde8aba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:31.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.834+0000 7f06eeffd640 1 -- 172.21.15.16:0/2256397790 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f06f00ceb70 con 0x7f06fc101420 2024-09-12T02:16:31.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.834+0000 7f07019e5640 1 --2- 172.21.15.16:0/2348939980 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fcde8a9680 0x55fcde8aba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:31.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.834+0000 7f07019e5640 1 -- 172.21.15.16:0/2348939980 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fcde790440 con 0x55fcde89edc0 2024-09-12T02:16:31.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.834+0000 7f07011e4640 1 --2- 172.21.15.16:0/2348939980 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fcde8a9680 0x55fcde8aba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:16:31.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.834+0000 7f07019e5640 1 --2- 172.21.15.16:0/2348939980 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fcde89edc0 0x55fcde8a9140 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7f06f8000f80 tx=0x7f06f8009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=689927091b409723 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:31.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.834+0000 7f07021e6640 1 --2- 172.21.15.16:0/2256397790 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f06e80a2b80 0x7f06e80a5040 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f06f40058c0 tx=0x7f06f4005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:31.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.834+0000 7f06e7fff640 1 -- 172.21.15.16:0/2348939980 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06f80316f0 con 0x55fcde89edc0 2024-09-12T02:16:31.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.834+0000 7f06e7fff640 1 -- 172.21.15.16:0/2348939980 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06f8036040 con 0x55fcde89edc0 2024-09-12T02:16:31.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.834+0000 7f06e7fff640 1 -- 172.21.15.16:0/2348939980 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06f8004680 con 0x55fcde89edc0 2024-09-12T02:16:31.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.835+0000 7f0706022740 1 -- 172.21.15.16:0/2348939980 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fcde89edc0 msgr2=0x55fcde8a9140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:31.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.835+0000 7f0706022740 1 --2- 172.21.15.16:0/2348939980 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fcde89edc0 0x55fcde8a9140 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7f06f8000f80 tx=0x7f06f8009a20 comp rx=0 tx=0).stop 2024-09-12T02:16:31.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.835+0000 7f0706022740 1 -- 172.21.15.16:0/2348939980 shutdown_connections 2024-09-12T02:16:31.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.835+0000 7f0706022740 1 --2- 172.21.15.16:0/2348939980 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fcde8a9680 0x55fcde8aba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:31.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.835+0000 7f0706022740 1 --2- 172.21.15.16:0/2348939980 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fcde89edc0 0x55fcde8a9140 unknown :-1 s=CLOSED pgs=1951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:31.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.835+0000 7f0706022740 1 --2- 172.21.15.16:0/2348939980 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fcde89e4a0 0x55fcde89e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:31.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.835+0000 7f0706022740 1 -- 172.21.15.16:0/2348939980 >> 172.21.15.16:0/2348939980 conn(0x55fcde89aa30 msgr2=0x55fcde89ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:31.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.835+0000 7f0706022740 1 -- 172.21.15.16:0/2348939980 shutdown_connections 2024-09-12T02:16:31.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.835+0000 7f0706022740 1 -- 172.21.15.16:0/2348939980 wait complete. 2024-09-12T02:16:31.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.835+0000 7f0706022740 1 -- start start 2024-09-12T02:16:31.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.835+0000 7f0706022740 1 Processor -- start 2024-09-12T02:16:31.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.836+0000 7f0706022740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fcde89e4a0 0x55fcde8a8f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:31.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.836+0000 7f0706022740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fcde89edc0 0x55fcde8a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:31.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.836+0000 7f0706022740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fcde8a9680 0x55fcde8a6720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:31.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.836+0000 7f0706022740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55fcde8ac250 con 0x55fcde8a9680 2024-09-12T02:16:31.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.836+0000 7f0706022740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55fcde7bf050 con 0x55fcde89edc0 2024-09-12T02:16:31.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.836+0000 7f0706022740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55fcde8ac3d0 con 0x55fcde89e4a0 2024-09-12T02:16:31.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.836+0000 7f0706022740 10 client.0 _ll_register_callbacks cb 0x55fcde7ff160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:16:31.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.836+0000 7f07021e6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fcde8a9680 0x55fcde8a6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:31.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.836+0000 7f07021e6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fcde8a9680 0x55fcde8a6720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53602/0 (socket says 172.21.15.16:53602) 2024-09-12T02:16:31.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.836+0000 7f07021e6640 1 -- 172.21.15.16:0/2299128373 learned_addr learned my addr 172.21.15.16:0/2299128373 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:16:31.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.836+0000 7f07019e5640 1 --2- 172.21.15.16:0/2299128373 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fcde89e4a0 0x55fcde8a8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:31.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.836+0000 7f07011e4640 1 --2- 172.21.15.16:0/2299128373 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fcde89edc0 0x55fcde8a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:31.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.836+0000 7f07021e6640 1 -- 172.21.15.16:0/2299128373 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fcde89e4a0 msgr2=0x55fcde8a8f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:31.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.836+0000 7f07021e6640 1 --2- 172.21.15.16:0/2299128373 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fcde89e4a0 0x55fcde8a8f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:31.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.836+0000 7f07021e6640 1 -- 172.21.15.16:0/2299128373 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fcde89edc0 msgr2=0x55fcde8a61e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:31.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.836+0000 7f07021e6640 1 --2- 172.21.15.16:0/2299128373 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fcde89edc0 0x55fcde8a61e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:31.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.836+0000 7f07021e6640 1 -- 172.21.15.16:0/2299128373 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fcde5753c0 con 0x55fcde8a9680 2024-09-12T02:16:31.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.837+0000 7f07021e6640 1 --2- 172.21.15.16:0/2299128373 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fcde8a9680 0x55fcde8a6720 secure :-1 s=READY pgs=1952 cs=0 l=1 rev1=1 crypto rx=0x7f06f4010c70 tx=0x7f06f40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9fc45e05ab1e6f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:31.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.837+0000 7f06e57fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:16:31.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.837+0000 7f06e57fa640 1 -- 172.21.15.16:0/2299128373 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06f4024070 con 0x55fcde8a9680 2024-09-12T02:16:31.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.837+0000 7f06e57fa640 1 -- 172.21.15.16:0/2299128373 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f06f40205f0 con 0x55fcde8a9680 2024-09-12T02:16:31.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.837+0000 7f0706022740 1 -- 172.21.15.16:0/2299128373 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f06f8036040 con 0x55fcde8a9680 2024-09-12T02:16:31.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.837+0000 7f06e57fa640 1 -- 172.21.15.16:0/2299128373 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f06f4030050 con 0x55fcde8a9680 2024-09-12T02:16:31.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.837+0000 7f0706022740 10 client.20292 fetch_fsmap learned FSMap version 25 2024-09-12T02:16:31.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.837+0000 7f0706022740 1 -- 172.21.15.16:0/2299128373 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55fcde8b7bf0 con 0x55fcde8a9680 2024-09-12T02:16:31.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.837+0000 7f06e57fa640 1 -- 172.21.15.16:0/2299128373 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06f8036040 con 0x55fcde8a9680 2024-09-12T02:16:31.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.837+0000 7f06e57fa640 1 -- 172.21.15.16:0/2299128373 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f06f401f070 con 0x55fcde8a9680 2024-09-12T02:16:31.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.837+0000 7f0706022740 10 client.20292 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:16:31.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.837+0000 7f0706022740 10 client.20292 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:16:31.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.837+0000 7f0706022740 20 client.20292 populate_metadata read hostname 'smithi016' 2024-09-12T02:16:31.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.837+0000 7f0706022740 1 --2- 172.21.15.16:0/2299128373 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55fcde8b9000 0x55fcde8bb4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:31.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.837+0000 7f0706022740 4 client.20292 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:16:31.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.837+0000 7f0706022740 1 -- 172.21.15.16:0/2299128373 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fcde811850 con 0x55fcde8b9000 2024-09-12T02:16:31.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.838+0000 7f07019e5640 1 --2- 172.21.15.16:0/2299128373 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55fcde8b9000 0x55fcde8bb4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:31.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.838+0000 7f07019e5640 1 --2- 172.21.15.16:0/2299128373 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55fcde8b9000 0x55fcde8bb4a0 crc :-1 s=READY pgs=433 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=661266796bc37c91 server_cookie=9d9ee41e5eabf98a in_seq=0 out_seq=0 2024-09-12T02:16:31.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.838+0000 7f06e57fa640 10 client.20292 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:16:31.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.839+0000 7f06e57fa640 1 -- 172.21.15.16:0/2299128373 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fcde811850 con 0x55fcde8b9000 2024-09-12T02:16:31.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.839+0000 7f06e57fa640 10 client.20292 handle_command_reply: tid=0 2024-09-12T02:16:31.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.839+0000 7f0706022740 1 client.20292 shutdown 2024-09-12T02:16:31.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.839+0000 7f0706022740 1 -- 172.21.15.16:0/2299128373 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fcde8a9680 msgr2=0x55fcde8a6720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:31.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.839+0000 7f0706022740 1 --2- 172.21.15.16:0/2299128373 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fcde8a9680 0x55fcde8a6720 secure :-1 s=READY pgs=1952 cs=0 l=1 rev1=1 crypto rx=0x7f06f4010c70 tx=0x7f06f40046c0 comp rx=0 tx=0).stop 2024-09-12T02:16:31.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.839+0000 7f0706022740 1 -- 172.21.15.16:0/2299128373 shutdown_connections 2024-09-12T02:16:31.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.839+0000 7f0706022740 1 --2- 172.21.15.16:0/2299128373 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55fcde8b9000 0x55fcde8bb4a0 crc :-1 s=READY pgs=433 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:31.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.839+0000 7f0706022740 1 --2- 172.21.15.16:0/2299128373 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fcde8a9680 0x55fcde8a6720 unknown :-1 s=CLOSED pgs=1952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:31.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.839+0000 7f0706022740 1 --2- 172.21.15.16:0/2299128373 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fcde89edc0 0x55fcde8a61e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:31.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.839+0000 7f0706022740 1 --2- 172.21.15.16:0/2299128373 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fcde89e4a0 0x55fcde8a8f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:31.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.839+0000 7f06e57fa640 0 client.20292 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:16:31.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.839+0000 7f0706022740 1 -- 172.21.15.16:0/2299128373 >> 172.21.15.16:0/2299128373 conn(0x55fcde89aa30 msgr2=0x55fcde8a0710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:31.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.839+0000 7f0706022740 1 -- 172.21.15.16:0/2299128373 shutdown_connections 2024-09-12T02:16:31.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.839+0000 7f0706022740 1 -- 172.21.15.16:0/2299128373 wait complete. 2024-09-12T02:16:31.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.839+0000 7f0706022740 20 client.20292 trim_cache size 0 max 16384 2024-09-12T02:16:31.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.856+0000 7f0706022740 1 Processor -- start 2024-09-12T02:16:31.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.856+0000 7f0706022740 1 -- start start 2024-09-12T02:16:31.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.856+0000 7f0706022740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fcde89aa30 0x55fcde8a61a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:31.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.856+0000 7f0706022740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fcde89e4a0 0x55fcde899a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:31.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.856+0000 7f0706022740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fcde89edc0 0x55fcde8c0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:31.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.856+0000 7f0706022740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f06fc1b1f20 con 0x55fcde89aa30 2024-09-12T02:16:31.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.856+0000 7f0706022740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f06fc1af7f0 con 0x55fcde89edc0 2024-09-12T02:16:31.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.856+0000 7f0706022740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f06fc1ad660 con 0x55fcde89e4a0 2024-09-12T02:16:31.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.857+0000 7f07019e5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fcde89aa30 0x55fcde8a61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:31.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.857+0000 7f07019e5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fcde89aa30 0x55fcde8a61a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53604/0 (socket says 172.21.15.16:53604) 2024-09-12T02:16:31.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.857+0000 7f07019e5640 1 -- 172.21.15.16:0/1009173481 learned_addr learned my addr 172.21.15.16:0/1009173481 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:16:31.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.857+0000 7f07021e6640 1 --2- 172.21.15.16:0/1009173481 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fcde89edc0 0x55fcde8c0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:31.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.857+0000 7f07011e4640 1 --2- 172.21.15.16:0/1009173481 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fcde89e4a0 0x55fcde899a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:31.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.857+0000 7f07019e5640 1 -- 172.21.15.16:0/1009173481 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fcde89e4a0 msgr2=0x55fcde899a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:31.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.857+0000 7f07019e5640 1 --2- 172.21.15.16:0/1009173481 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fcde89e4a0 0x55fcde899a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:31.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.857+0000 7f07019e5640 1 -- 172.21.15.16:0/1009173481 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fcde89edc0 msgr2=0x55fcde8c0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:31.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.857+0000 7f07019e5640 1 --2- 172.21.15.16:0/1009173481 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fcde89edc0 0x55fcde8c0dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:31.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.857+0000 7f07019e5640 1 -- 172.21.15.16:0/1009173481 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fcde89ebf0 con 0x55fcde89aa30 2024-09-12T02:16:31.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.857+0000 7f07021e6640 1 --2- 172.21.15.16:0/1009173481 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fcde89edc0 0x55fcde8c0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:16:31.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.857+0000 7f07019e5640 1 --2- 172.21.15.16:0/1009173481 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fcde89aa30 0x55fcde8a61a0 secure :-1 s=READY pgs=1953 cs=0 l=1 rev1=1 crypto rx=0x7f06f8000f80 tx=0x7f06f80041c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6130cc6810b8cc3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:31.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.857+0000 7f06e77fe640 1 -- 172.21.15.16:0/1009173481 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06f8031a50 con 0x55fcde89aa30 2024-09-12T02:16:31.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.857+0000 7f06e77fe640 1 -- 172.21.15.16:0/1009173481 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06f8036040 con 0x55fcde89aa30 2024-09-12T02:16:31.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.857+0000 7f06e77fe640 1 -- 172.21.15.16:0/1009173481 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06f8003f90 con 0x55fcde89aa30 2024-09-12T02:16:31.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.858+0000 7f0706022740 1 -- 172.21.15.16:0/1009173481 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fcde89aa30 msgr2=0x55fcde8a61a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:31.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.858+0000 7f0706022740 1 --2- 172.21.15.16:0/1009173481 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fcde89aa30 0x55fcde8a61a0 secure :-1 s=READY pgs=1953 cs=0 l=1 rev1=1 crypto rx=0x7f06f8000f80 tx=0x7f06f80041c0 comp rx=0 tx=0).stop 2024-09-12T02:16:31.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.858+0000 7f0706022740 1 -- 172.21.15.16:0/1009173481 shutdown_connections 2024-09-12T02:16:31.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.858+0000 7f0706022740 1 --2- 172.21.15.16:0/1009173481 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fcde89edc0 0x55fcde8c0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:31.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.858+0000 7f0706022740 1 --2- 172.21.15.16:0/1009173481 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fcde89e4a0 0x55fcde899a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:31.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.858+0000 7f0706022740 1 --2- 172.21.15.16:0/1009173481 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fcde89aa30 0x55fcde8a61a0 unknown :-1 s=CLOSED pgs=1953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:31.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.858+0000 7f0706022740 1 -- 172.21.15.16:0/1009173481 >> 172.21.15.16:0/1009173481 conn(0x55fcde8a6880 msgr2=0x55fcde899350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:31.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.858+0000 7f0706022740 1 -- 172.21.15.16:0/1009173481 shutdown_connections 2024-09-12T02:16:31.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.858+0000 7f0706022740 1 -- 172.21.15.16:0/1009173481 wait complete. 2024-09-12T02:16:31.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.858+0000 7f0706022740 1 -- start start 2024-09-12T02:16:31.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.858+0000 7f0706022740 1 Processor -- start 2024-09-12T02:16:31.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.859+0000 7f0706022740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fcde89aa30 0x55fcde8b1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:31.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.859+0000 7f0706022740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fcde89e4a0 0x55fcde8af230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:31.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.859+0000 7f0706022740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fcde89edc0 0x55fcde8af820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:31.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.859+0000 7f0706022740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f06fc1adcb0 con 0x55fcde89edc0 2024-09-12T02:16:31.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.859+0000 7f0706022740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f06fc1af120 con 0x55fcde89aa30 2024-09-12T02:16:31.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.859+0000 7f0706022740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f06fc1ae380 con 0x55fcde89e4a0 2024-09-12T02:16:31.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.859+0000 7f0706022740 10 client.0 _ll_register_callbacks cb 0x7f06f4010920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:16:31.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.859+0000 7f07021e6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fcde89edc0 0x55fcde8af820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:31.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.859+0000 7f07021e6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fcde89edc0 0x55fcde8af820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53608/0 (socket says 172.21.15.16:53608) 2024-09-12T02:16:31.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.859+0000 7f07011e4640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fcde89e4a0 0x55fcde8af230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:31.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.859+0000 7f07021e6640 1 -- 172.21.15.16:0/3759122021 learned_addr learned my addr 172.21.15.16:0/3759122021 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:16:31.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.859+0000 7f07019e5640 1 --2- 172.21.15.16:0/3759122021 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fcde89aa30 0x55fcde8b1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:31.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.859+0000 7f07021e6640 1 -- 172.21.15.16:0/3759122021 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fcde89e4a0 msgr2=0x55fcde8af230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:31.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.859+0000 7f07021e6640 1 --2- 172.21.15.16:0/3759122021 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fcde89e4a0 0x55fcde8af230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:31.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.859+0000 7f07021e6640 1 -- 172.21.15.16:0/3759122021 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fcde89aa30 msgr2=0x55fcde8b1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:31.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.859+0000 7f07021e6640 1 --2- 172.21.15.16:0/3759122021 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fcde89aa30 0x55fcde8b1ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:31.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.859+0000 7f07021e6640 1 -- 172.21.15.16:0/3759122021 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fcde8b0050 con 0x55fcde89edc0 2024-09-12T02:16:31.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.859+0000 7f07019e5640 1 --2- 172.21.15.16:0/3759122021 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fcde89aa30 0x55fcde8b1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:16:31.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.859+0000 7f07021e6640 1 --2- 172.21.15.16:0/3759122021 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fcde89edc0 0x55fcde8af820 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7f06f4016040 tx=0x7f06f400fd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22bd0a1546ce5000 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:31.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.859+0000 7f06e57fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:16:31.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.860+0000 7f06e57fa640 1 -- 172.21.15.16:0/3759122021 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06f4039060 con 0x55fcde89edc0 2024-09-12T02:16:31.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.860+0000 7f06e57fa640 1 -- 172.21.15.16:0/3759122021 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f06f4005040 con 0x55fcde89edc0 2024-09-12T02:16:31.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.860+0000 7f0706022740 1 -- 172.21.15.16:0/3759122021 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f06f8036040 con 0x55fcde89edc0 2024-09-12T02:16:31.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.860+0000 7f06e57fa640 1 -- 172.21.15.16:0/3759122021 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f06f4024070 con 0x55fcde89edc0 2024-09-12T02:16:31.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.860+0000 7f0706022740 10 client.20298 fetch_fsmap learned FSMap version 25 2024-09-12T02:16:31.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.860+0000 7f0706022740 1 -- 172.21.15.16:0/3759122021 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55fcde923ec0 con 0x55fcde89edc0 2024-09-12T02:16:31.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.860+0000 7f06e57fa640 1 -- 172.21.15.16:0/3759122021 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06f8036040 con 0x55fcde89edc0 2024-09-12T02:16:31.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.860+0000 7f06e57fa640 1 -- 172.21.15.16:0/3759122021 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f06f403e070 con 0x55fcde89edc0 2024-09-12T02:16:31.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.860+0000 7f0706022740 10 client.20298 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:16:31.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.860+0000 7f0706022740 10 client.20298 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:16:31.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.860+0000 7f0706022740 20 client.20298 populate_metadata read hostname 'smithi016' 2024-09-12T02:16:31.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.860+0000 7f0706022740 1 --2- 172.21.15.16:0/3759122021 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55fcde8a9680 0x55fcde9265f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:31.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.860+0000 7f0706022740 4 client.20298 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T02:16:31.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.860+0000 7f0706022740 1 -- 172.21.15.16:0/3759122021 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55fcde80bd60 con 0x55fcde8a9680 2024-09-12T02:16:31.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.861+0000 7f07019e5640 1 --2- 172.21.15.16:0/3759122021 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55fcde8a9680 0x55fcde9265f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:31.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.861+0000 7f07019e5640 1 --2- 172.21.15.16:0/3759122021 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55fcde8a9680 0x55fcde9265f0 crc :-1 s=READY pgs=434 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f99389aca41d2526 server_cookie=285d91ee0cb9a580 in_seq=0 out_seq=0 2024-09-12T02:16:31.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.861+0000 7f06e57fa640 10 client.20298 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:16:31.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.861+0000 7f06e57fa640 1 -- 172.21.15.16:0/3759122021 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55fcde80bd60 con 0x55fcde8a9680 2024-09-12T02:16:31.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.861+0000 7f06e57fa640 10 client.20298 handle_command_reply: tid=0 2024-09-12T02:16:31.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.861+0000 7f0706022740 1 client.20298 shutdown 2024-09-12T02:16:31.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.861+0000 7f0706022740 1 -- 172.21.15.16:0/3759122021 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fcde89edc0 msgr2=0x55fcde8af820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:31.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.861+0000 7f0706022740 1 --2- 172.21.15.16:0/3759122021 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fcde89edc0 0x55fcde8af820 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7f06f4016040 tx=0x7f06f400fd60 comp rx=0 tx=0).stop 2024-09-12T02:16:31.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.861+0000 7f0706022740 1 -- 172.21.15.16:0/3759122021 shutdown_connections 2024-09-12T02:16:31.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.861+0000 7f0706022740 1 --2- 172.21.15.16:0/3759122021 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55fcde8a9680 0x55fcde9265f0 crc :-1 s=READY pgs=434 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:31.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.861+0000 7f06e57fa640 0 client.20298 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:16:31.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.861+0000 7f0706022740 1 --2- 172.21.15.16:0/3759122021 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fcde89edc0 0x55fcde8af820 unknown :-1 s=CLOSED pgs=1954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:31.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.861+0000 7f0706022740 1 --2- 172.21.15.16:0/3759122021 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fcde89e4a0 0x55fcde8af230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:31.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.861+0000 7f0706022740 1 --2- 172.21.15.16:0/3759122021 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fcde89aa30 0x55fcde8b1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:31.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.861+0000 7f0706022740 1 -- 172.21.15.16:0/3759122021 >> 172.21.15.16:0/3759122021 conn(0x55fcde8a6880 msgr2=0x55fcde8a8260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:31.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.861+0000 7f0706022740 1 -- 172.21.15.16:0/3759122021 shutdown_connections 2024-09-12T02:16:31.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.862+0000 7f0706022740 1 -- 172.21.15.16:0/3759122021 wait complete. 2024-09-12T02:16:31.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.862+0000 7f0706022740 20 client.20298 trim_cache size 0 max 16384 2024-09-12T02:16:31.868 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T02:16:31.868 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-09-12T02:16:31.868 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {} 2024-09-12T02:16:31.868 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T02:16:31.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.862+0000 7f0703c49640 1 -- 172.21.15.16:0/2256397790 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f06e80a2b80 msgr2=0x7f06e80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:31.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.862+0000 7f0703c49640 1 --2- 172.21.15.16:0/2256397790 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f06e80a2b80 0x7f06e80a5040 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f06f40058c0 tx=0x7f06f4005850 comp rx=0 tx=0).stop 2024-09-12T02:16:31.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.862+0000 7f0703c49640 1 -- 172.21.15.16:0/2256397790 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06fc101420 msgr2=0x7f06fc0fd3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:31.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.862+0000 7f0703c49640 1 --2- 172.21.15.16:0/2256397790 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06fc101420 0x7f06fc0fd3b0 secure :-1 s=READY pgs=1950 cs=0 l=1 rev1=1 crypto rx=0x7f06f000d770 tx=0x7f06f000dc30 comp rx=0 tx=0).stop 2024-09-12T02:16:31.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.862+0000 7f0703c49640 1 -- 172.21.15.16:0/2256397790 shutdown_connections 2024-09-12T02:16:31.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.862+0000 7f0703c49640 1 --2- 172.21.15.16:0/2256397790 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f06e80a2b80 0x7f06e80a5040 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:31.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.862+0000 7f0703c49640 1 --2- 172.21.15.16:0/2256397790 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f06fc106a80 0x7f06fc0fd8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:31.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.862+0000 7f0703c49640 1 --2- 172.21.15.16:0/2256397790 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06fc101420 0x7f06fc0fd3b0 unknown :-1 s=CLOSED pgs=1950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:31.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.862+0000 7f0703c49640 1 --2- 172.21.15.16:0/2256397790 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f06fc100a80 0x7f06fc0fce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:31.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.862+0000 7f0703c49640 1 -- 172.21.15.16:0/2256397790 >> 172.21.15.16:0/2256397790 conn(0x7f06fc0fc7a0 msgr2=0x7f06fc06aad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:31.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.862+0000 7f0703c49640 1 -- 172.21.15.16:0/2256397790 shutdown_connections 2024-09-12T02:16:31.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:31.862+0000 7f0703c49640 1 -- 172.21.15.16:0/2256397790 wait complete. 2024-09-12T02:16:31.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:16:31 smithi184 ceph-mon[35593]: scrub queued for path: / 2024-09-12T02:16:31.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:16:31 smithi184 ceph-mon[35593]: scrub summary: idle+waiting paths [/] 2024-09-12T02:16:31.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:16:31 smithi184 ceph-mon[35593]: scrub summary: active paths [/] 2024-09-12T02:16:31.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:16:31 smithi184 ceph-mon[35593]: scrub summary: idle+waiting paths [/] 2024-09-12T02:16:32.561 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-12T02:16:32.562 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:16:32.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:16:32 smithi033 ceph-mon[35332]: pgmap v773: 97 pgs: 97 active+clean; 3.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-09-12T02:16:32.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:16:32 smithi033 ceph-mon[35332]: scrub summary: idle 2024-09-12T02:16:32.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:16:32 smithi016 ceph-mon[30947]: pgmap v773: 97 pgs: 97 active+clean; 3.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-09-12T02:16:32.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:16:32 smithi016 ceph-mon[30947]: scrub summary: idle 2024-09-12T02:16:32.820 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:16:32.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:16:32 smithi184 ceph-mon[35593]: pgmap v773: 97 pgs: 97 active+clean; 3.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-09-12T02:16:32.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:16:32 smithi184 ceph-mon[35593]: scrub summary: idle 2024-09-12T02:16:33.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:33.277+0000 7fc99775c640 1 -- 172.21.15.16:0/4030687859 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc990100e20 msgr2=0x7fc990101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:33.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:33.277+0000 7fc99775c640 1 --2- 172.21.15.16:0/4030687859 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc990100e20 0x7fc990101200 secure :-1 s=READY pgs=1955 cs=0 l=1 rev1=1 crypto rx=0x7fc98c009900 tx=0x7fc98c02a480 comp rx=0 tx=0).stop 2024-09-12T02:16:33.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:33.277+0000 7fc99775c640 1 -- 172.21.15.16:0/4030687859 shutdown_connections 2024-09-12T02:16:33.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:33.277+0000 7fc99775c640 1 --2- 172.21.15.16:0/4030687859 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc990102ef0 0x7fc99010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:33.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:33.277+0000 7fc99775c640 1 --2- 172.21.15.16:0/4030687859 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc990102570 0x7fc9901029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:33.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:33.277+0000 7fc99775c640 1 --2- 172.21.15.16:0/4030687859 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc990100e20 0x7fc990101200 unknown :-1 s=CLOSED pgs=1955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:33.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:33.277+0000 7fc99775c640 1 -- 172.21.15.16:0/4030687859 >> 172.21.15.16:0/4030687859 conn(0x7fc9900fc7d0 msgr2=0x7fc9900febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:33.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:33.278+0000 7fc99775c640 1 -- 172.21.15.16:0/4030687859 shutdown_connections 2024-09-12T02:16:33.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:33.278+0000 7fc99775c640 1 -- 172.21.15.16:0/4030687859 wait complete. 2024-09-12T02:16:33.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:33.278+0000 7fc99775c640 1 Processor -- start 2024-09-12T02:16:33.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:33.278+0000 7fc99775c640 1 -- start start 2024-09-12T02:16:33.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:33.278+0000 7fc99775c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc990100e20 0x7fc9900fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:33.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:33.278+0000 7fc99775c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc990102570 0x7fc9900fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:33.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:33.278+0000 7fc99775c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc990102ef0 0x7fc9900fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:33.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:33.279+0000 7fc99775c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc990110fa0 con 0x7fc990102ef0 2024-09-12T02:16:33.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:33.279+0000 7fc99775c640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc990110e20 con 0x7fc990102570 2024-09-12T02:16:33.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:33.279+0000 7fc99775c640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc990111120 con 0x7fc990100e20 2024-09-12T02:16:33.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:33.279+0000 7fc995cf9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc990102ef0 0x7fc9900fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:33.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:33.279+0000 7fc995cf9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc990102ef0 0x7fc9900fd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53612/0 (socket says 172.21.15.16:53612) 2024-09-12T02:16:33.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:33.279+0000 7fc995cf9640 1 -- 172.21.15.16:0/1108987538 learned_addr learned my addr 172.21.15.16:0/1108987538 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:16:33.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:33.279+0000 7fc9954f8640 1 --2- 172.21.15.16:0/1108987538 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc990100e20 0x7fc9900fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:33.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:33.279+0000 7fc994cf7640 1 --2- 172.21.15.16:0/1108987538 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc990102570 0x7fc9900fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:33.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:33.279+0000 7fc995cf9640 1 -- 172.21.15.16:0/1108987538 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc990100e20 msgr2=0x7fc9900fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:33.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:33.279+0000 7fc995cf9640 1 --2- 172.21.15.16:0/1108987538 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc990100e20 0x7fc9900fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:33.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:33.279+0000 7fc995cf9640 1 -- 172.21.15.16:0/1108987538 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc990102570 msgr2=0x7fc9900fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:33.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:33.279+0000 7fc995cf9640 1 --2- 172.21.15.16:0/1108987538 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc990102570 0x7fc9900fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:33.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:33.279+0000 7fc995cf9640 1 -- 172.21.15.16:0/1108987538 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9901a6a50 con 0x7fc990102ef0 2024-09-12T02:16:33.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:33.279+0000 7fc994cf7640 1 --2- 172.21.15.16:0/1108987538 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc990102570 0x7fc9900fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:16:33.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:33.279+0000 7fc995cf9640 1 --2- 172.21.15.16:0/1108987538 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc990102ef0 0x7fc9900fd700 secure :-1 s=READY pgs=1956 cs=0 l=1 rev1=1 crypto rx=0x7fc98800b970 tx=0x7fc98800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:33.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:33.279+0000 7fc9867fc640 1 -- 172.21.15.16:0/1108987538 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc988019070 con 0x7fc990102ef0 2024-09-12T02:16:33.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:33.279+0000 7fc9867fc640 1 -- 172.21.15.16:0/1108987538 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc98800cb40 con 0x7fc990102ef0 2024-09-12T02:16:33.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:33.279+0000 7fc99775c640 1 -- 172.21.15.16:0/1108987538 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9901a6d40 con 0x7fc990102ef0 2024-09-12T02:16:33.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:33.279+0000 7fc9867fc640 1 -- 172.21.15.16:0/1108987538 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc988007650 con 0x7fc990102ef0 2024-09-12T02:16:33.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:33.279+0000 7fc99775c640 1 -- 172.21.15.16:0/1108987538 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9901a7280 con 0x7fc990102ef0 2024-09-12T02:16:33.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:33.280+0000 7fc99775c640 1 -- 172.21.15.16:0/1108987538 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc990101200 con 0x7fc990102ef0 2024-09-12T02:16:33.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:33.282+0000 7fc9867fc640 1 -- 172.21.15.16:0/1108987538 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fc98801c070 con 0x7fc990102ef0 2024-09-12T02:16:33.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:33.282+0000 7fc9867fc640 1 --2- 172.21.15.16:0/1108987538 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc97c0a2c00 0x7fc97c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:33.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:33.282+0000 7fc9954f8640 1 --2- 172.21.15.16:0/1108987538 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc97c0a2c00 0x7fc97c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:33.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:33.282+0000 7fc9867fc640 1 -- 172.21.15.16:0/1108987538 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fc9880ce4a0 con 0x7fc990102ef0 2024-09-12T02:16:33.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:33.283+0000 7fc9954f8640 1 --2- 172.21.15.16:0/1108987538 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc97c0a2c00 0x7fc97c0a50c0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fc98c000c00 tx=0x7fc98c0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:33.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:33.286+0000 7fc9867fc640 1 -- 172.21.15.16:0/1108987538 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc98809ae50 con 0x7fc990102ef0 2024-09-12T02:16:33.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:33.498+0000 7fc99775c640 1 -- 172.21.15.16:0/1108987538 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc99010a0a0 con 0x7fc990102ef0 2024-09-12T02:16:33.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:33.499+0000 7fc9867fc640 1 -- 172.21.15.16:0/1108987538 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11911 (secure 0 0 0) 0x7fc98809f150 con 0x7fc990102ef0 2024-09-12T02:16:33.499 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T02:16:33.500 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":25,"btime":"2024-09-12T02:01:37:192021+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3279123279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3279123279},{"type":"v1","addr":"172.21.15.16:6835","nonce":3279123279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4119452382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4119452382},{"type":"v1","addr":"172.21.15.16:6837","nonce":4119452382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/3263823165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3263823165},{"type":"v1","addr":"172.21.15.16:6839","nonce":3263823165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/1510290732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":1510290732},{"type":"v1","addr":"172.21.15.16:6841","nonce":1510290732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/279674603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":279674603},{"type":"v1","addr":"172.21.15.33:6833","nonce":279674603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3555728028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3555728028},{"type":"v1","addr":"172.21.15.33:6835","nonce":3555728028}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T01:59:56.117608+0000","modified":"2024-09-12T02:01:36.294760+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24482,"mds_1":24481,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6833/1563374106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1563374106},{"type":"v1","addr":"172.21.15.184:6833","nonce":1563374106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6837/1892833868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1892833868},{"type":"v1","addr":"172.21.15.33:6837","nonce":1892833868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/1457679686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1457679686},{"type":"v1","addr":"172.21.15.184:6835","nonce":1457679686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.184:6837/1793393087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":1793393087},{"type":"v1","addr":"172.21.15.184:6837","nonce":1793393087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.184:6839/4203347941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":4203347941},{"type":"v1","addr":"172.21.15.184:6839","nonce":4203347941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.33:6839/4136670204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":4136670204},{"type":"v1","addr":"172.21.15.33:6839","nonce":4136670204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24481,24482,24469]},"id":1}]} 2024-09-12T02:16:33.500 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 25 2024-09-12T02:16:33.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:33.501+0000 7fc99775c640 1 -- 172.21.15.16:0/1108987538 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc97c0a2c00 msgr2=0x7fc97c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:33.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:33.501+0000 7fc99775c640 1 --2- 172.21.15.16:0/1108987538 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc97c0a2c00 0x7fc97c0a50c0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fc98c000c00 tx=0x7fc98c0058d0 comp rx=0 tx=0).stop 2024-09-12T02:16:33.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:33.501+0000 7fc99775c640 1 -- 172.21.15.16:0/1108987538 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc990102ef0 msgr2=0x7fc9900fd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:33.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:33.501+0000 7fc99775c640 1 --2- 172.21.15.16:0/1108987538 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc990102ef0 0x7fc9900fd700 secure :-1 s=READY pgs=1956 cs=0 l=1 rev1=1 crypto rx=0x7fc98800b970 tx=0x7fc98800be30 comp rx=0 tx=0).stop 2024-09-12T02:16:33.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:33.502+0000 7fc99775c640 1 -- 172.21.15.16:0/1108987538 shutdown_connections 2024-09-12T02:16:33.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:33.502+0000 7fc99775c640 1 --2- 172.21.15.16:0/1108987538 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fc97c0a2c00 0x7fc97c0a50c0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:33.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:33.502+0000 7fc99775c640 1 --2- 172.21.15.16:0/1108987538 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc990102ef0 0x7fc9900fd700 unknown :-1 s=CLOSED pgs=1956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:33.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:33.502+0000 7fc99775c640 1 --2- 172.21.15.16:0/1108987538 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc990102570 0x7fc9900fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:33.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:33.502+0000 7fc99775c640 1 --2- 172.21.15.16:0/1108987538 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc990100e20 0x7fc9900fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:33.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:33.502+0000 7fc99775c640 1 -- 172.21.15.16:0/1108987538 >> 172.21.15.16:0/1108987538 conn(0x7fc9900fc7d0 msgr2=0x7fc99006a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:33.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:33.502+0000 7fc99775c640 1 -- 172.21.15.16:0/1108987538 shutdown_connections 2024-09-12T02:16:33.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:33.502+0000 7fc99775c640 1 -- 172.21.15.16:0/1108987538 wait complete. 2024-09-12T02:16:34.134 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-12T02:16:34.397 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:16:34.659 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:16:34 smithi016 ceph-mon[30947]: pgmap v774: 97 pgs: 97 active+clean; 3.4 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-09-12T02:16:34.659 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:16:34 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/1108987538' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:16:34.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:16:34 smithi033 ceph-mon[35332]: pgmap v774: 97 pgs: 97 active+clean; 3.4 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-09-12T02:16:34.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:16:34 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/1108987538' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:16:34.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:16:34 smithi184 ceph-mon[35593]: pgmap v774: 97 pgs: 97 active+clean; 3.4 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-09-12T02:16:34.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:16:34 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/1108987538' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:16:34.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.885+0000 7f687d0f5640 1 -- 172.21.15.16:0/4257052005 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6878106a80 msgr2=0x7f6878106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:34.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.885+0000 7f687d0f5640 1 --2- 172.21.15.16:0/4257052005 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6878106a80 0x7f6878106e60 secure :-1 s=READY pgs=1957 cs=0 l=1 rev1=1 crypto rx=0x7f6868009930 tx=0x7f686802a420 comp rx=0 tx=0).stop 2024-09-12T02:16:34.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.885+0000 7f687d0f5640 1 -- 172.21.15.16:0/4257052005 shutdown_connections 2024-09-12T02:16:34.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.885+0000 7f687d0f5640 1 --2- 172.21.15.16:0/4257052005 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6878101420 0x7f687810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:34.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.886+0000 7f687d0f5640 1 --2- 172.21.15.16:0/4257052005 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6878100a80 0x7f6878100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:34.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.886+0000 7f687d0f5640 1 --2- 172.21.15.16:0/4257052005 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6878106a80 0x7f6878106e60 unknown :-1 s=CLOSED pgs=1957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:34.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.886+0000 7f687d0f5640 1 -- 172.21.15.16:0/4257052005 >> 172.21.15.16:0/4257052005 conn(0x7f68780fc7a0 msgr2=0x7f68780febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:34.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.886+0000 7f687d0f5640 1 -- 172.21.15.16:0/4257052005 shutdown_connections 2024-09-12T02:16:34.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.886+0000 7f687d0f5640 1 -- 172.21.15.16:0/4257052005 wait complete. 2024-09-12T02:16:34.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.886+0000 7f687d0f5640 1 Processor -- start 2024-09-12T02:16:34.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.887+0000 7f687d0f5640 1 -- start start 2024-09-12T02:16:34.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.887+0000 7f687d0f5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6878100a80 0x7f6878198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:34.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.887+0000 7f687d0f5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6878101420 0x7f6878198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:34.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.887+0000 7f687d0f5640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6878106a80 0x7f687819cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:34.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.887+0000 7f687d0f5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6878110e80 con 0x7f6878100a80 2024-09-12T02:16:34.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.887+0000 7f687d0f5640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6878110d00 con 0x7f6878101420 2024-09-12T02:16:34.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.887+0000 7f687d0f5640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f6878111000 con 0x7f6878106a80 2024-09-12T02:16:34.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.887+0000 7f6876d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6878100a80 0x7f6878198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:34.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.887+0000 7f6876d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6878100a80 0x7f6878198440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53632/0 (socket says 172.21.15.16:53632) 2024-09-12T02:16:34.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.887+0000 7f6876d9d640 1 -- 172.21.15.16:0/3756704926 learned_addr learned my addr 172.21.15.16:0/3756704926 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:16:34.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.887+0000 7f687659c640 1 --2- 172.21.15.16:0/3756704926 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6878101420 0x7f6878198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:34.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.887+0000 7f687759e640 1 --2- 172.21.15.16:0/3756704926 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6878106a80 0x7f687819cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:34.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.887+0000 7f6876d9d640 1 -- 172.21.15.16:0/3756704926 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6878106a80 msgr2=0x7f687819cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:34.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.887+0000 7f6876d9d640 1 --2- 172.21.15.16:0/3756704926 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6878106a80 0x7f687819cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:34.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.887+0000 7f6876d9d640 1 -- 172.21.15.16:0/3756704926 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6878101420 msgr2=0x7f6878198980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:34.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.887+0000 7f6876d9d640 1 --2- 172.21.15.16:0/3756704926 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6878101420 0x7f6878198980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:34.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.887+0000 7f6876d9d640 1 -- 172.21.15.16:0/3756704926 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f687819d3f0 con 0x7f6878100a80 2024-09-12T02:16:34.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.888+0000 7f6876d9d640 1 --2- 172.21.15.16:0/3756704926 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6878100a80 0x7f6878198440 secure :-1 s=READY pgs=1958 cs=0 l=1 rev1=1 crypto rx=0x7f6868009230 tx=0x7f6868004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:34.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.888+0000 7f685ffff640 1 -- 172.21.15.16:0/3756704926 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f686803f030 con 0x7f6878100a80 2024-09-12T02:16:34.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.888+0000 7f687d0f5640 1 -- 172.21.15.16:0/3756704926 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f687819d680 con 0x7f6878100a80 2024-09-12T02:16:34.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.888+0000 7f685ffff640 1 -- 172.21.15.16:0/3756704926 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6868003ad0 con 0x7f6878100a80 2024-09-12T02:16:34.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.888+0000 7f685ffff640 1 -- 172.21.15.16:0/3756704926 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6868035070 con 0x7f6878100a80 2024-09-12T02:16:34.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.888+0000 7f687d0f5640 1 -- 172.21.15.16:0/3756704926 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68781a4fc0 con 0x7f6878100a80 2024-09-12T02:16:34.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.890+0000 7f685ffff640 1 -- 172.21.15.16:0/3756704926 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6868042030 con 0x7f6878100a80 2024-09-12T02:16:34.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.891+0000 7f687f4ce740 1 Processor -- start 2024-09-12T02:16:34.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.891+0000 7f687f4ce740 1 -- start start 2024-09-12T02:16:34.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.891+0000 7f687f4ce740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562a9a04bf50 0x562a9a04a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:34.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.891+0000 7f687f4ce740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562a9a04a9e0 0x562a9a04ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:34.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.891+0000 7f685ffff640 1 --2- 172.21.15.16:0/3756704926 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f68600a2b80 0x7f68600a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:34.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.891+0000 7f687f4ce740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a9a055780 0x562a9a057b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:34.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.891+0000 7f687f4ce740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x562a99fac2f0 con 0x562a9a055780 2024-09-12T02:16:34.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.891+0000 7f687f4ce740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x562a99f53cf0 con 0x562a9a04a9e0 2024-09-12T02:16:34.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.891+0000 7f687f4ce740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x562a99fa82b0 con 0x562a9a04bf50 2024-09-12T02:16:34.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.891+0000 7f685ffff640 1 -- 172.21.15.16:0/3756704926 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f68680ef120 con 0x7f6878100a80 2024-09-12T02:16:34.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.891+0000 7f687759e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a9a055780 0x562a9a057b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:34.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.891+0000 7f687759e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a9a055780 0x562a9a057b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53640/0 (socket says 172.21.15.16:53640) 2024-09-12T02:16:34.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.891+0000 7f687759e640 1 -- 172.21.15.16:0/1863802283 learned_addr learned my addr 172.21.15.16:0/1863802283 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:16:34.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.891+0000 7f6876d9d640 1 --2- 172.21.15.16:0/3756704926 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f68600a2b80 0x7f68600a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:34.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.891+0000 7f687759e640 1 --2- 172.21.15.16:0/1863802283 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562a9a04bf50 0x562a9a04a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:34.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.891+0000 7f687659c640 1 --2- 172.21.15.16:0/1863802283 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562a9a04a9e0 0x562a9a04ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:34.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.892+0000 7f687759e640 1 -- 172.21.15.16:0/1863802283 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562a9a04bf50 msgr2=0x562a9a04a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:34.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.892+0000 7f687759e640 1 --2- 172.21.15.16:0/1863802283 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562a9a04bf50 0x562a9a04a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:34.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.892+0000 7f687759e640 1 -- 172.21.15.16:0/1863802283 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562a9a04a9e0 msgr2=0x562a9a04ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:34.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.892+0000 7f687759e640 1 --2- 172.21.15.16:0/1863802283 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562a9a04a9e0 0x562a9a04ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:34.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.892+0000 7f687759e640 1 -- 172.21.15.16:0/1863802283 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a99f383a0 con 0x562a9a055780 2024-09-12T02:16:34.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.892+0000 7f6876d9d640 1 --2- 172.21.15.16:0/3756704926 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f68600a2b80 0x7f68600a5040 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f68680bb9c0 tx=0x7f68680a6040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:34.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.892+0000 7f687759e640 1 --2- 172.21.15.16:0/1863802283 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a9a055780 0x562a9a057b70 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 crypto rx=0x7f6870012a30 tx=0x7f6870009aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21f2ac58fe5b16b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:34.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.892+0000 7f685d7fa640 1 -- 172.21.15.16:0/1863802283 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6870003970 con 0x562a9a055780 2024-09-12T02:16:34.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.892+0000 7f685d7fa640 1 -- 172.21.15.16:0/1863802283 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6870010770 con 0x562a9a055780 2024-09-12T02:16:34.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.892+0000 7f685d7fa640 1 -- 172.21.15.16:0/1863802283 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6870004390 con 0x562a9a055780 2024-09-12T02:16:34.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.893+0000 7f687f4ce740 1 -- 172.21.15.16:0/1863802283 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a9a055780 msgr2=0x562a9a057b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:34.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.893+0000 7f687f4ce740 1 --2- 172.21.15.16:0/1863802283 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a9a055780 0x562a9a057b70 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 crypto rx=0x7f6870012a30 tx=0x7f6870009aa0 comp rx=0 tx=0).stop 2024-09-12T02:16:34.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.893+0000 7f687f4ce740 1 -- 172.21.15.16:0/1863802283 shutdown_connections 2024-09-12T02:16:34.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.893+0000 7f687f4ce740 1 --2- 172.21.15.16:0/1863802283 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a9a055780 0x562a9a057b70 unknown :-1 s=CLOSED pgs=1959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:34.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.893+0000 7f687f4ce740 1 --2- 172.21.15.16:0/1863802283 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562a9a04a9e0 0x562a9a04ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:34.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.893+0000 7f687f4ce740 1 --2- 172.21.15.16:0/1863802283 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562a9a04bf50 0x562a9a04a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:34.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.893+0000 7f687f4ce740 1 -- 172.21.15.16:0/1863802283 >> 172.21.15.16:0/1863802283 conn(0x562a9a00f310 msgr2=0x562a9a047070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:34.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.893+0000 7f687f4ce740 1 -- 172.21.15.16:0/1863802283 shutdown_connections 2024-09-12T02:16:34.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.893+0000 7f687f4ce740 1 -- 172.21.15.16:0/1863802283 wait complete. 2024-09-12T02:16:34.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.893+0000 7f687f4ce740 1 -- start start 2024-09-12T02:16:34.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.893+0000 7f687f4ce740 1 Processor -- start 2024-09-12T02:16:34.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.894+0000 7f687f4ce740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562a9a04a9e0 0x562a9a0551f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:34.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.894+0000 7f687f4ce740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a9a04bf50 0x562a9a0524a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:34.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.894+0000 7f687f4ce740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562a9a055780 0x562a9a052be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:34.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.894+0000 7f687f4ce740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x562a9a058480 con 0x562a9a04bf50 2024-09-12T02:16:34.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.894+0000 7f687f4ce740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x562a99f67060 con 0x562a9a055780 2024-09-12T02:16:34.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.894+0000 7f687f4ce740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x562a9a058600 con 0x562a9a04a9e0 2024-09-12T02:16:34.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.894+0000 7f687f4ce740 10 client.0 _ll_register_callbacks cb 0x562a99cba8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:16:34.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.894+0000 7f687659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a9a04bf50 0x562a9a0524a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:34.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.894+0000 7f687659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a9a04bf50 0x562a9a0524a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53650/0 (socket says 172.21.15.16:53650) 2024-09-12T02:16:34.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.894+0000 7f687659c640 1 -- 172.21.15.16:0/616837916 learned_addr learned my addr 172.21.15.16:0/616837916 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:16:34.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.894+0000 7f687759e640 1 --2- 172.21.15.16:0/616837916 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562a9a04a9e0 0x562a9a0551f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:34.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.894+0000 7f687759e640 1 --2- 172.21.15.16:0/616837916 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562a9a055780 0x562a9a052be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:34.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.894+0000 7f687659c640 1 -- 172.21.15.16:0/616837916 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562a9a04a9e0 msgr2=0x562a9a0551f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:34.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.894+0000 7f687659c640 1 --2- 172.21.15.16:0/616837916 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562a9a04a9e0 0x562a9a0551f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:34.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.894+0000 7f687659c640 1 -- 172.21.15.16:0/616837916 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562a9a055780 msgr2=0x562a9a052be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:34.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.894+0000 7f687659c640 1 --2- 172.21.15.16:0/616837916 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562a9a055780 0x562a9a052be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:34.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.894+0000 7f687659c640 1 -- 172.21.15.16:0/616837916 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a99d1d3c0 con 0x562a9a04bf50 2024-09-12T02:16:34.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.894+0000 7f687659c640 1 --2- 172.21.15.16:0/616837916 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a9a04bf50 0x562a9a0524a0 secure :-1 s=READY pgs=1960 cs=0 l=1 rev1=1 crypto rx=0x7f686c00bad0 tx=0x7f686c00bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf40de2c7a73cfbb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:34.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.894+0000 7f685aff5640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:16:34.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.895+0000 7f685aff5640 1 -- 172.21.15.16:0/616837916 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f686c003e20 con 0x562a9a04bf50 2024-09-12T02:16:34.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.895+0000 7f685aff5640 1 -- 172.21.15.16:0/616837916 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f686c011440 con 0x562a9a04bf50 2024-09-12T02:16:34.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.895+0000 7f687f4ce740 1 -- 172.21.15.16:0/616837916 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6870010770 con 0x562a9a04bf50 2024-09-12T02:16:34.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.895+0000 7f685aff5640 1 -- 172.21.15.16:0/616837916 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f686c005020 con 0x562a9a04bf50 2024-09-12T02:16:34.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.895+0000 7f687f4ce740 10 client.20316 fetch_fsmap learned FSMap version 25 2024-09-12T02:16:34.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.895+0000 7f687f4ce740 1 -- 172.21.15.16:0/616837916 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x562a9a064010 con 0x562a9a04bf50 2024-09-12T02:16:34.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.895+0000 7f685aff5640 1 -- 172.21.15.16:0/616837916 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6870010770 con 0x562a9a04bf50 2024-09-12T02:16:34.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.895+0000 7f685aff5640 1 -- 172.21.15.16:0/616837916 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f686c0116d0 con 0x562a9a04bf50 2024-09-12T02:16:34.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.895+0000 7f687f4ce740 10 client.20316 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:16:34.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.895+0000 7f687f4ce740 10 client.20316 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-12T02:16:34.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.895+0000 7f687f4ce740 20 client.20316 populate_metadata read hostname 'smithi016' 2024-09-12T02:16:34.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.895+0000 7f687f4ce740 1 --2- 172.21.15.16:0/616837916 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x562a9a065570 0x562a9a067960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:34.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.895+0000 7f687f4ce740 4 client.20316 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:16:34.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.895+0000 7f687f4ce740 1 -- 172.21.15.16:0/616837916 --> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562a99fb9860 con 0x562a9a065570 2024-09-12T02:16:34.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.896+0000 7f687759e640 1 --2- 172.21.15.16:0/616837916 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x562a9a065570 0x562a9a067960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:34.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.896+0000 7f687759e640 1 --2- 172.21.15.16:0/616837916 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x562a9a065570 0x562a9a067960 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=57bb43cfafd66ab7 server_cookie=bd252d7cb308ec57 in_seq=0 out_seq=0 2024-09-12T02:16:34.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.896+0000 7f685aff5640 10 client.20316 ms_handle_connect on v2:172.21.15.184:6836/1793393087 2024-09-12T02:16:34.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.897+0000 7f685aff5640 1 -- 172.21.15.16:0/616837916 <== mds.2 v2:172.21.15.184:6836/1793393087 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562a99fb9860 con 0x562a9a065570 2024-09-12T02:16:34.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.897+0000 7f685aff5640 10 client.20316 handle_command_reply: tid=0 2024-09-12T02:16:34.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.897+0000 7f687f4ce740 1 client.20316 shutdown 2024-09-12T02:16:34.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.897+0000 7f687f4ce740 1 -- 172.21.15.16:0/616837916 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a9a04bf50 msgr2=0x562a9a0524a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:34.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.897+0000 7f687f4ce740 1 --2- 172.21.15.16:0/616837916 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a9a04bf50 0x562a9a0524a0 secure :-1 s=READY pgs=1960 cs=0 l=1 rev1=1 crypto rx=0x7f686c00bad0 tx=0x7f686c00bf90 comp rx=0 tx=0).stop 2024-09-12T02:16:34.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.897+0000 7f687f4ce740 1 -- 172.21.15.16:0/616837916 shutdown_connections 2024-09-12T02:16:34.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.897+0000 7f687f4ce740 1 --2- 172.21.15.16:0/616837916 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562a9a055780 0x562a9a052be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:34.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.897+0000 7f687f4ce740 1 --2- 172.21.15.16:0/616837916 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x562a9a065570 0x562a9a067960 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:34.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.897+0000 7f687f4ce740 1 --2- 172.21.15.16:0/616837916 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a9a04bf50 0x562a9a0524a0 unknown :-1 s=CLOSED pgs=1960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:34.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.897+0000 7f687f4ce740 1 --2- 172.21.15.16:0/616837916 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562a9a04a9e0 0x562a9a0551f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:34.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.897+0000 7f685aff5640 0 client.20316 ms_handle_reset on v2:172.21.15.184:6836/1793393087 2024-09-12T02:16:34.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.897+0000 7f687f4ce740 1 -- 172.21.15.16:0/616837916 >> 172.21.15.16:0/616837916 conn(0x562a9a00f310 msgr2=0x562a9a04cd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:34.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.897+0000 7f687f4ce740 1 -- 172.21.15.16:0/616837916 shutdown_connections 2024-09-12T02:16:34.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.897+0000 7f687f4ce740 1 -- 172.21.15.16:0/616837916 wait complete. 2024-09-12T02:16:34.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.897+0000 7f687f4ce740 20 client.20316 trim_cache size 0 max 16384 2024-09-12T02:16:34.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.911+0000 7f687f4ce740 1 Processor -- start 2024-09-12T02:16:34.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.911+0000 7f687f4ce740 1 -- start start 2024-09-12T02:16:34.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.911+0000 7f687f4ce740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562a9a04a9e0 0x562a9a04b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:34.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.911+0000 7f687f4ce740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a9a04bf50 0x562a9a061720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:34.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.911+0000 7f687f4ce740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562a9a055780 0x562a9a051480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:34.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.911+0000 7f687f4ce740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f68781adb30 con 0x562a9a04bf50 2024-09-12T02:16:34.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.911+0000 7f687f4ce740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f68781ab400 con 0x562a9a055780 2024-09-12T02:16:34.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.911+0000 7f687f4ce740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f68781a9250 con 0x562a9a04a9e0 2024-09-12T02:16:34.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.911+0000 7f687659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a9a04bf50 0x562a9a061720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:34.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.911+0000 7f687659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a9a04bf50 0x562a9a061720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53662/0 (socket says 172.21.15.16:53662) 2024-09-12T02:16:34.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.911+0000 7f687659c640 1 -- 172.21.15.16:0/1514490563 learned_addr learned my addr 172.21.15.16:0/1514490563 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:16:34.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.911+0000 7f687759e640 1 --2- 172.21.15.16:0/1514490563 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562a9a04a9e0 0x562a9a04b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:34.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.912+0000 7f687759e640 1 --2- 172.21.15.16:0/1514490563 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562a9a055780 0x562a9a051480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:34.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.912+0000 7f687659c640 1 -- 172.21.15.16:0/1514490563 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562a9a04a9e0 msgr2=0x562a9a04b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:34.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.912+0000 7f687659c640 1 --2- 172.21.15.16:0/1514490563 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562a9a04a9e0 0x562a9a04b7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:34.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.912+0000 7f687659c640 1 -- 172.21.15.16:0/1514490563 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562a9a055780 msgr2=0x562a9a051480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:34.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.912+0000 7f687659c640 1 --2- 172.21.15.16:0/1514490563 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562a9a055780 0x562a9a051480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:34.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.912+0000 7f687659c640 1 -- 172.21.15.16:0/1514490563 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a9a057ee0 con 0x562a9a04bf50 2024-09-12T02:16:34.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.912+0000 7f687759e640 1 --2- 172.21.15.16:0/1514490563 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562a9a04a9e0 0x562a9a04b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:16:34.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.912+0000 7f687659c640 1 --2- 172.21.15.16:0/1514490563 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a9a04bf50 0x562a9a061720 secure :-1 s=READY pgs=1961 cs=0 l=1 rev1=1 crypto rx=0x7f686c011b90 tx=0x7f686c003fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54e6516be63f2ae7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:34.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.912+0000 7f685cff9640 1 -- 172.21.15.16:0/1514490563 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f686c00b400 con 0x562a9a04bf50 2024-09-12T02:16:34.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.912+0000 7f685cff9640 1 -- 172.21.15.16:0/1514490563 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f686c0051c0 con 0x562a9a04bf50 2024-09-12T02:16:34.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.912+0000 7f685cff9640 1 -- 172.21.15.16:0/1514490563 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f686c004bc0 con 0x562a9a04bf50 2024-09-12T02:16:34.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.913+0000 7f687f4ce740 1 -- 172.21.15.16:0/1514490563 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a9a04bf50 msgr2=0x562a9a061720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:34.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.913+0000 7f687f4ce740 1 --2- 172.21.15.16:0/1514490563 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a9a04bf50 0x562a9a061720 secure :-1 s=READY pgs=1961 cs=0 l=1 rev1=1 crypto rx=0x7f686c011b90 tx=0x7f686c003fd0 comp rx=0 tx=0).stop 2024-09-12T02:16:34.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.913+0000 7f687f4ce740 1 -- 172.21.15.16:0/1514490563 shutdown_connections 2024-09-12T02:16:34.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.913+0000 7f687f4ce740 1 --2- 172.21.15.16:0/1514490563 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562a9a055780 0x562a9a051480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:34.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.913+0000 7f687f4ce740 1 --2- 172.21.15.16:0/1514490563 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a9a04bf50 0x562a9a061720 unknown :-1 s=CLOSED pgs=1961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:34.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.913+0000 7f687f4ce740 1 --2- 172.21.15.16:0/1514490563 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562a9a04a9e0 0x562a9a04b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:34.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.913+0000 7f687f4ce740 1 -- 172.21.15.16:0/1514490563 >> 172.21.15.16:0/1514490563 conn(0x562a9a00f310 msgr2=0x562a9a050d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:34.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.913+0000 7f687f4ce740 1 -- 172.21.15.16:0/1514490563 shutdown_connections 2024-09-12T02:16:34.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.913+0000 7f687f4ce740 1 -- 172.21.15.16:0/1514490563 wait complete. 2024-09-12T02:16:34.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.913+0000 7f687f4ce740 1 -- start start 2024-09-12T02:16:34.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.913+0000 7f687f4ce740 1 Processor -- start 2024-09-12T02:16:34.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.914+0000 7f687f4ce740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562a9a04a9e0 0x562a9a05ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:34.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.914+0000 7f687f4ce740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a9a04bf50 0x562a9a0590f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:34.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.914+0000 7f687f4ce740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562a9a055780 0x562a9a059630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:34.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.914+0000 7f687f4ce740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f68781a98c0 con 0x562a9a04bf50 2024-09-12T02:16:34.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.914+0000 7f687f4ce740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f68781aad30 con 0x562a9a04a9e0 2024-09-12T02:16:34.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.914+0000 7f687f4ce740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f68781a9f90 con 0x562a9a055780 2024-09-12T02:16:34.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.914+0000 7f687f4ce740 10 client.0 _ll_register_callbacks cb 0x7f686c004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:16:34.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.914+0000 7f687659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a9a04bf50 0x562a9a0590f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:34.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.914+0000 7f687659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a9a04bf50 0x562a9a0590f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53674/0 (socket says 172.21.15.16:53674) 2024-09-12T02:16:34.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.914+0000 7f687659c640 1 -- 172.21.15.16:0/4282260820 learned_addr learned my addr 172.21.15.16:0/4282260820 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:16:34.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.914+0000 7f687759e640 1 --2- 172.21.15.16:0/4282260820 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562a9a055780 0x562a9a059630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:34.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.914+0000 7f687759e640 1 --2- 172.21.15.16:0/4282260820 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562a9a04a9e0 0x562a9a05ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:34.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.914+0000 7f687659c640 1 -- 172.21.15.16:0/4282260820 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562a9a055780 msgr2=0x562a9a059630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:34.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.914+0000 7f687659c640 1 --2- 172.21.15.16:0/4282260820 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562a9a055780 0x562a9a059630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:34.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.914+0000 7f687659c640 1 -- 172.21.15.16:0/4282260820 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562a9a04a9e0 msgr2=0x562a9a05ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:34.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.914+0000 7f687659c640 1 --2- 172.21.15.16:0/4282260820 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562a9a04a9e0 0x562a9a05ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:34.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.914+0000 7f687659c640 1 -- 172.21.15.16:0/4282260820 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a9a059c20 con 0x562a9a04bf50 2024-09-12T02:16:34.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.914+0000 7f687759e640 1 --2- 172.21.15.16:0/4282260820 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562a9a04a9e0 0x562a9a05ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:16:34.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.914+0000 7f687659c640 1 --2- 172.21.15.16:0/4282260820 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a9a04bf50 0x562a9a0590f0 secure :-1 s=READY pgs=1962 cs=0 l=1 rev1=1 crypto rx=0x7f686c011d50 tx=0x7f686c0058f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a6a52860af7d013 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:34.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.914+0000 7f685aff5640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:16:34.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.915+0000 7f685aff5640 1 -- 172.21.15.16:0/4282260820 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f686c011650 con 0x562a9a04bf50 2024-09-12T02:16:34.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.915+0000 7f685aff5640 1 -- 172.21.15.16:0/4282260820 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f686c017070 con 0x562a9a04bf50 2024-09-12T02:16:34.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.915+0000 7f687f4ce740 1 -- 172.21.15.16:0/4282260820 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f686c0051c0 con 0x562a9a04bf50 2024-09-12T02:16:34.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.915+0000 7f685aff5640 1 -- 172.21.15.16:0/4282260820 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f686c0117f0 con 0x562a9a04bf50 2024-09-12T02:16:34.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.915+0000 7f687f4ce740 10 client.20322 fetch_fsmap learned FSMap version 25 2024-09-12T02:16:34.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.915+0000 7f687f4ce740 1 -- 172.21.15.16:0/4282260820 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x562a9a0cffe0 con 0x562a9a04bf50 2024-09-12T02:16:34.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.915+0000 7f685aff5640 1 -- 172.21.15.16:0/4282260820 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f686c0051c0 con 0x562a9a04bf50 2024-09-12T02:16:34.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.915+0000 7f685aff5640 1 -- 172.21.15.16:0/4282260820 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f686c02c030 con 0x562a9a04bf50 2024-09-12T02:16:34.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.915+0000 7f687f4ce740 10 client.20322 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:16:34.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.915+0000 7f687f4ce740 10 client.20322 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-12T02:16:34.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.915+0000 7f687f4ce740 20 client.20322 populate_metadata read hostname 'smithi016' 2024-09-12T02:16:34.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.915+0000 7f687f4ce740 1 --2- 172.21.15.16:0/4282260820 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x562a9a0d06f0 0x562a9a0d2b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:34.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.916+0000 7f687f4ce740 4 client.20322 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T02:16:34.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.916+0000 7f687f4ce740 1 -- 172.21.15.16:0/4282260820 --> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562a99fb3d70 con 0x562a9a0d06f0 2024-09-12T02:16:34.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.916+0000 7f687759e640 1 --2- 172.21.15.16:0/4282260820 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x562a9a0d06f0 0x562a9a0d2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:34.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.916+0000 7f685aff5640 10 client.20322 ms_handle_connect on v2:172.21.15.184:6836/1793393087 2024-09-12T02:16:34.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.916+0000 7f687759e640 1 --2- 172.21.15.16:0/4282260820 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x562a9a0d06f0 0x562a9a0d2b10 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a42f799c40f6b0b2 server_cookie=c5388b05f17f9c86 in_seq=0 out_seq=0 2024-09-12T02:16:34.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.916+0000 7f685aff5640 1 -- 172.21.15.16:0/4282260820 <== mds.2 v2:172.21.15.184:6836/1793393087 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562a99fb3d70 con 0x562a9a0d06f0 2024-09-12T02:16:34.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.916+0000 7f685aff5640 10 client.20322 handle_command_reply: tid=0 2024-09-12T02:16:34.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.916+0000 7f687f4ce740 1 client.20322 shutdown 2024-09-12T02:16:34.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.917+0000 7f687f4ce740 1 -- 172.21.15.16:0/4282260820 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a9a04bf50 msgr2=0x562a9a0590f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:34.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.917+0000 7f687f4ce740 1 --2- 172.21.15.16:0/4282260820 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a9a04bf50 0x562a9a0590f0 secure :-1 s=READY pgs=1962 cs=0 l=1 rev1=1 crypto rx=0x7f686c011d50 tx=0x7f686c0058f0 comp rx=0 tx=0).stop 2024-09-12T02:16:34.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.917+0000 7f687f4ce740 1 -- 172.21.15.16:0/4282260820 shutdown_connections 2024-09-12T02:16:34.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.917+0000 7f687f4ce740 1 --2- 172.21.15.16:0/4282260820 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562a9a055780 0x562a9a059630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:34.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.917+0000 7f687f4ce740 1 --2- 172.21.15.16:0/4282260820 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x562a9a0d06f0 0x562a9a0d2b10 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:34.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.917+0000 7f687f4ce740 1 --2- 172.21.15.16:0/4282260820 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a9a04bf50 0x562a9a0590f0 unknown :-1 s=CLOSED pgs=1962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:34.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.917+0000 7f687f4ce740 1 --2- 172.21.15.16:0/4282260820 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562a9a04a9e0 0x562a9a05ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:34.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.917+0000 7f685aff5640 0 client.20322 ms_handle_reset on v2:172.21.15.184:6836/1793393087 2024-09-12T02:16:34.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.917+0000 7f687f4ce740 1 -- 172.21.15.16:0/4282260820 >> 172.21.15.16:0/4282260820 conn(0x562a9a00f310 msgr2=0x562a9a0414e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:34.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.917+0000 7f687f4ce740 1 -- 172.21.15.16:0/4282260820 shutdown_connections 2024-09-12T02:16:34.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.917+0000 7f687f4ce740 1 -- 172.21.15.16:0/4282260820 wait complete. 2024-09-12T02:16:34.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.917+0000 7f687f4ce740 20 client.20322 trim_cache size 0 max 16384 2024-09-12T02:16:34.922 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T02:16:34.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.917+0000 7f687d0f5640 1 -- 172.21.15.16:0/3756704926 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f68600a2b80 msgr2=0x7f68600a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:34.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.917+0000 7f687d0f5640 1 --2- 172.21.15.16:0/3756704926 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f68600a2b80 0x7f68600a5040 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f68680bb9c0 tx=0x7f68680a6040 comp rx=0 tx=0).stop 2024-09-12T02:16:34.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.917+0000 7f687d0f5640 1 -- 172.21.15.16:0/3756704926 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6878100a80 msgr2=0x7f6878198440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:34.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.917+0000 7f687d0f5640 1 --2- 172.21.15.16:0/3756704926 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6878100a80 0x7f6878198440 secure :-1 s=READY pgs=1958 cs=0 l=1 rev1=1 crypto rx=0x7f6868009230 tx=0x7f6868004270 comp rx=0 tx=0).stop 2024-09-12T02:16:34.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.918+0000 7f687d0f5640 1 -- 172.21.15.16:0/3756704926 shutdown_connections 2024-09-12T02:16:34.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.918+0000 7f687d0f5640 1 --2- 172.21.15.16:0/3756704926 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f68600a2b80 0x7f68600a5040 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:34.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.918+0000 7f687d0f5640 1 --2- 172.21.15.16:0/3756704926 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6878106a80 0x7f687819cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:34.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.918+0000 7f687d0f5640 1 --2- 172.21.15.16:0/3756704926 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6878101420 0x7f6878198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:34.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.918+0000 7f687d0f5640 1 --2- 172.21.15.16:0/3756704926 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6878100a80 0x7f6878198440 unknown :-1 s=CLOSED pgs=1958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:34.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.918+0000 7f687d0f5640 1 -- 172.21.15.16:0/3756704926 >> 172.21.15.16:0/3756704926 conn(0x7f68780fc7a0 msgr2=0x7f6878104c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:34.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.918+0000 7f687d0f5640 1 -- 172.21.15.16:0/3756704926 shutdown_connections 2024-09-12T02:16:34.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:34.918+0000 7f687d0f5640 1 -- 172.21.15.16:0/3756704926 wait complete. 2024-09-12T02:16:35.586 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-12T02:16:35.845 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:16:36.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.369+0000 7f8326352640 1 -- 172.21.15.16:0/3312870140 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8320106a80 msgr2=0x7f8320106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:36.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.369+0000 7f8326352640 1 --2- 172.21.15.16:0/3312870140 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8320106a80 0x7f8320106e60 secure :-1 s=READY pgs=1963 cs=0 l=1 rev1=1 crypto rx=0x7f8310009930 tx=0x7f831002a420 comp rx=0 tx=0).stop 2024-09-12T02:16:36.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.369+0000 7f8326352640 1 -- 172.21.15.16:0/3312870140 shutdown_connections 2024-09-12T02:16:36.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.369+0000 7f8326352640 1 --2- 172.21.15.16:0/3312870140 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8320101420 0x7f832010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:36.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.369+0000 7f8326352640 1 --2- 172.21.15.16:0/3312870140 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8320100a80 0x7f8320100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:36.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.369+0000 7f8326352640 1 --2- 172.21.15.16:0/3312870140 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8320106a80 0x7f8320106e60 unknown :-1 s=CLOSED pgs=1963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:36.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.369+0000 7f8326352640 1 -- 172.21.15.16:0/3312870140 >> 172.21.15.16:0/3312870140 conn(0x7f83200fc7a0 msgr2=0x7f83200febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:36.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.369+0000 7f8326352640 1 -- 172.21.15.16:0/3312870140 shutdown_connections 2024-09-12T02:16:36.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.369+0000 7f8326352640 1 -- 172.21.15.16:0/3312870140 wait complete. 2024-09-12T02:16:36.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.370+0000 7f8326352640 1 Processor -- start 2024-09-12T02:16:36.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.370+0000 7f8326352640 1 -- start start 2024-09-12T02:16:36.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.370+0000 7f8326352640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8320100a80 0x7f83201983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:36.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.370+0000 7f8326352640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8320101420 0x7f8320198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:36.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.370+0000 7f8326352640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8320106a80 0x7f832019ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:36.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.370+0000 7f8326352640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8320110ed0 con 0x7f8320101420 2024-09-12T02:16:36.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.370+0000 7f8326352640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8320110d50 con 0x7f8320106a80 2024-09-12T02:16:36.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.370+0000 7f8326352640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8320111050 con 0x7f8320100a80 2024-09-12T02:16:36.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.370+0000 7f831f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8320101420 0x7f8320198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:36.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.370+0000 7f831f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8320101420 0x7f8320198920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53684/0 (socket says 172.21.15.16:53684) 2024-09-12T02:16:36.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.370+0000 7f831f7fe640 1 -- 172.21.15.16:0/2428650221 learned_addr learned my addr 172.21.15.16:0/2428650221 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:16:36.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.371+0000 7f831ffff640 1 --2- 172.21.15.16:0/2428650221 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8320100a80 0x7f83201983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:36.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.371+0000 7f83248ef640 1 --2- 172.21.15.16:0/2428650221 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8320106a80 0x7f832019ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:36.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.371+0000 7f831f7fe640 1 -- 172.21.15.16:0/2428650221 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8320100a80 msgr2=0x7f83201983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:36.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.371+0000 7f831f7fe640 1 --2- 172.21.15.16:0/2428650221 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8320100a80 0x7f83201983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:36.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.371+0000 7f831f7fe640 1 -- 172.21.15.16:0/2428650221 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8320106a80 msgr2=0x7f832019ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:36.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.371+0000 7f831f7fe640 1 --2- 172.21.15.16:0/2428650221 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8320106a80 0x7f832019ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:36.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.371+0000 7f831f7fe640 1 -- 172.21.15.16:0/2428650221 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f832019d390 con 0x7f8320101420 2024-09-12T02:16:36.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.371+0000 7f831ffff640 1 --2- 172.21.15.16:0/2428650221 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8320100a80 0x7f83201983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:16:36.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.371+0000 7f831f7fe640 1 --2- 172.21.15.16:0/2428650221 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8320101420 0x7f8320198920 secure :-1 s=READY pgs=1964 cs=0 l=1 rev1=1 crypto rx=0x7f831400e9f0 tx=0x7f831400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:36.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.371+0000 7f831d7fa640 1 -- 172.21.15.16:0/2428650221 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f831400cd80 con 0x7f8320101420 2024-09-12T02:16:36.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.371+0000 7f8326352640 1 -- 172.21.15.16:0/2428650221 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f832019d680 con 0x7f8320101420 2024-09-12T02:16:36.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.371+0000 7f831d7fa640 1 -- 172.21.15.16:0/2428650221 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8314010910 con 0x7f8320101420 2024-09-12T02:16:36.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.371+0000 7f831d7fa640 1 -- 172.21.15.16:0/2428650221 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83140188f0 con 0x7f8320101420 2024-09-12T02:16:36.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.371+0000 7f8326352640 1 -- 172.21.15.16:0/2428650221 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83201a4f60 con 0x7f8320101420 2024-09-12T02:16:36.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.374+0000 7f831d7fa640 1 -- 172.21.15.16:0/2428650221 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f8314010ac0 con 0x7f8320101420 2024-09-12T02:16:36.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.374+0000 7f832872b740 1 Processor -- start 2024-09-12T02:16:36.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.374+0000 7f832872b740 1 -- start start 2024-09-12T02:16:36.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.374+0000 7f832872b740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5628db4beb40 0x5628db4bd090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:36.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.374+0000 7f832872b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5628db4bd5d0 0x5628db4bda10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:36.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.374+0000 7f832872b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5628db4cc780 0x5628db4ceb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:36.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.374+0000 7f832872b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5628db4232f0 con 0x5628db4cc780 2024-09-12T02:16:36.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.374+0000 7f832872b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5628db3cacf0 con 0x5628db4bd5d0 2024-09-12T02:16:36.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.374+0000 7f832872b740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5628db41f2b0 con 0x5628db4beb40 2024-09-12T02:16:36.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.374+0000 7f831d7fa640 1 --2- 172.21.15.16:0/2428650221 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f83080a2b80 0x7f83080a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:36.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.375+0000 7f831f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5628db4cc780 0x5628db4ceb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:36.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.375+0000 7f831ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5628db4bd5d0 0x5628db4bda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:36.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.375+0000 7f831ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5628db4bd5d0 0x5628db4bda10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.16:51608/0 (socket says 172.21.15.16:51608) 2024-09-12T02:16:36.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.375+0000 7f831ffff640 1 -- 172.21.15.16:0/1653977197 learned_addr learned my addr 172.21.15.16:0/1653977197 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:16:36.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.375+0000 7f83248ef640 1 --2- 172.21.15.16:0/1653977197 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5628db4beb40 0x5628db4bd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:36.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.375+0000 7f83248ef640 1 --2- 172.21.15.16:0/2428650221 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f83080a2b80 0x7f83080a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:36.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.375+0000 7f831d7fa640 1 -- 172.21.15.16:0/2428650221 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f83140cf490 con 0x7f8320101420 2024-09-12T02:16:36.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.375+0000 7f831f7fe640 1 -- 172.21.15.16:0/1653977197 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5628db4beb40 msgr2=0x5628db4bd090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:36.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.375+0000 7f831f7fe640 1 --2- 172.21.15.16:0/1653977197 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5628db4beb40 0x5628db4bd090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:36.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.375+0000 7f831f7fe640 1 -- 172.21.15.16:0/1653977197 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5628db4bd5d0 msgr2=0x5628db4bda10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:36.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.375+0000 7f831f7fe640 1 --2- 172.21.15.16:0/1653977197 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5628db4bd5d0 0x5628db4bda10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:36.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.375+0000 7f831f7fe640 1 -- 172.21.15.16:0/1653977197 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5628db3af3a0 con 0x5628db4cc780 2024-09-12T02:16:36.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.375+0000 7f831f7fe640 1 --2- 172.21.15.16:0/1653977197 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5628db4cc780 0x5628db4ceb70 secure :-1 s=READY pgs=1965 cs=0 l=1 rev1=1 crypto rx=0x7f83140cd080 tx=0x7f831406ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b007baf1d6e8a1b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:36.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.375+0000 7f83248ef640 1 --2- 172.21.15.16:0/2428650221 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f83080a2b80 0x7f83080a5040 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f8318007fd0 tx=0x7f8318007850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:36.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.375+0000 7f83067fc640 1 -- 172.21.15.16:0/1653977197 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83140a4a00 con 0x5628db4cc780 2024-09-12T02:16:36.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.376+0000 7f83067fc640 1 -- 172.21.15.16:0/1653977197 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f831409bd90 con 0x5628db4cc780 2024-09-12T02:16:36.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.376+0000 7f83067fc640 1 -- 172.21.15.16:0/1653977197 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8314098590 con 0x5628db4cc780 2024-09-12T02:16:36.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.376+0000 7f832872b740 1 -- 172.21.15.16:0/1653977197 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5628db4cc780 msgr2=0x5628db4ceb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:36.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.376+0000 7f832872b740 1 --2- 172.21.15.16:0/1653977197 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5628db4cc780 0x5628db4ceb70 secure :-1 s=READY pgs=1965 cs=0 l=1 rev1=1 crypto rx=0x7f83140cd080 tx=0x7f831406ef10 comp rx=0 tx=0).stop 2024-09-12T02:16:36.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.376+0000 7f832872b740 1 -- 172.21.15.16:0/1653977197 shutdown_connections 2024-09-12T02:16:36.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.376+0000 7f832872b740 1 --2- 172.21.15.16:0/1653977197 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5628db4cc780 0x5628db4ceb70 unknown :-1 s=CLOSED pgs=1965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:36.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.376+0000 7f832872b740 1 --2- 172.21.15.16:0/1653977197 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5628db4bd5d0 0x5628db4bda10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:36.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.376+0000 7f832872b740 1 --2- 172.21.15.16:0/1653977197 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5628db4beb40 0x5628db4bd090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:36.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.376+0000 7f832872b740 1 -- 172.21.15.16:0/1653977197 >> 172.21.15.16:0/1653977197 conn(0x5628db486310 msgr2=0x5628db4b9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:36.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.376+0000 7f832872b740 1 -- 172.21.15.16:0/1653977197 shutdown_connections 2024-09-12T02:16:36.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.376+0000 7f832872b740 1 -- 172.21.15.16:0/1653977197 wait complete. 2024-09-12T02:16:36.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.377+0000 7f832872b740 1 -- start start 2024-09-12T02:16:36.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.377+0000 7f832872b740 1 Processor -- start 2024-09-12T02:16:36.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.377+0000 7f832872b740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5628db4bd5d0 0x5628db4cc1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:36.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.377+0000 7f832872b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5628db4beb40 0x5628db4c9460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:36.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.377+0000 7f832872b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5628db4cc780 0x5628db4c9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:36.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.377+0000 7f832872b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5628db4cf3c0 con 0x5628db4beb40 2024-09-12T02:16:36.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.377+0000 7f832872b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5628db3de060 con 0x5628db4cc780 2024-09-12T02:16:36.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.377+0000 7f832872b740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5628db4cf540 con 0x5628db4bd5d0 2024-09-12T02:16:36.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.377+0000 7f832872b740 10 client.0 _ll_register_callbacks cb 0x5628db1318c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:16:36.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.377+0000 7f831f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5628db4beb40 0x5628db4c9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:36.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.377+0000 7f831f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5628db4beb40 0x5628db4c9460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53708/0 (socket says 172.21.15.16:53708) 2024-09-12T02:16:36.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.377+0000 7f831f7fe640 1 -- 172.21.15.16:0/41991380 learned_addr learned my addr 172.21.15.16:0/41991380 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:16:36.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.378+0000 7f831f7fe640 1 -- 172.21.15.16:0/41991380 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5628db4bd5d0 msgr2=0x5628db4cc1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:36.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.378+0000 7f831ffff640 1 --2- 172.21.15.16:0/41991380 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5628db4bd5d0 0x5628db4cc1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:36.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.378+0000 7f83248ef640 1 --2- 172.21.15.16:0/41991380 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5628db4cc780 0x5628db4c9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:36.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.378+0000 7f831f7fe640 1 --2- 172.21.15.16:0/41991380 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5628db4bd5d0 0x5628db4cc1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:36.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.378+0000 7f831f7fe640 1 -- 172.21.15.16:0/41991380 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5628db4cc780 msgr2=0x5628db4c9bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:36.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.378+0000 7f831f7fe640 1 --2- 172.21.15.16:0/41991380 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5628db4cc780 0x5628db4c9bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:36.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.378+0000 7f831f7fe640 1 -- 172.21.15.16:0/41991380 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5628db1943c0 con 0x5628db4beb40 2024-09-12T02:16:36.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.378+0000 7f831ffff640 1 --2- 172.21.15.16:0/41991380 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5628db4bd5d0 0x5628db4cc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:16:36.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.378+0000 7f83248ef640 1 --2- 172.21.15.16:0/41991380 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5628db4cc780 0x5628db4c9bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:16:36.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.378+0000 7f831f7fe640 1 --2- 172.21.15.16:0/41991380 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5628db4beb40 0x5628db4c9460 secure :-1 s=READY pgs=1966 cs=0 l=1 rev1=1 crypto rx=0x7f831409ba90 tx=0x7f8314098130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=555fd39eb6fee722 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:36.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.378+0000 7f82ff7fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:16:36.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.378+0000 7f82ff7fe640 1 -- 172.21.15.16:0/41991380 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83140a4770 con 0x5628db4beb40 2024-09-12T02:16:36.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.378+0000 7f82ff7fe640 1 -- 172.21.15.16:0/41991380 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f831409b1e0 con 0x5628db4beb40 2024-09-12T02:16:36.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.378+0000 7f832872b740 1 -- 172.21.15.16:0/41991380 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f831409bd90 con 0x5628db4beb40 2024-09-12T02:16:36.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.378+0000 7f82ff7fe640 1 -- 172.21.15.16:0/41991380 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f83140a0140 con 0x5628db4beb40 2024-09-12T02:16:36.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.378+0000 7f832872b740 10 client.20334 fetch_fsmap learned FSMap version 25 2024-09-12T02:16:36.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.378+0000 7f832872b740 1 -- 172.21.15.16:0/41991380 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5628db4daea0 con 0x5628db4beb40 2024-09-12T02:16:36.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.379+0000 7f82ff7fe640 1 -- 172.21.15.16:0/41991380 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f831409bd90 con 0x5628db4beb40 2024-09-12T02:16:36.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.379+0000 7f82ff7fe640 1 -- 172.21.15.16:0/41991380 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f831409ae80 con 0x5628db4beb40 2024-09-12T02:16:36.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.379+0000 7f832872b740 10 client.20334 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:16:36.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.379+0000 7f832872b740 10 client.20334 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T02:16:36.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.379+0000 7f832872b740 20 client.20334 populate_metadata read hostname 'smithi016' 2024-09-12T02:16:36.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.379+0000 7f832872b740 1 --2- 172.21.15.16:0/41991380 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x5628db4dc1f0 0x5628db4de5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:36.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.379+0000 7f832872b740 4 client.20334 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:16:36.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.379+0000 7f832872b740 1 -- 172.21.15.16:0/41991380 --> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5628db430860 con 0x5628db4dc1f0 2024-09-12T02:16:36.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.379+0000 7f831ffff640 1 --2- 172.21.15.16:0/41991380 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x5628db4dc1f0 0x5628db4de5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:36.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.380+0000 7f831ffff640 1 --2- 172.21.15.16:0/41991380 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x5628db4dc1f0 0x5628db4de5e0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5adf5f93127b3d4d server_cookie=441a335d97dcfdfb in_seq=0 out_seq=0 2024-09-12T02:16:36.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.380+0000 7f82ff7fe640 10 client.20334 ms_handle_connect on v2:172.21.15.184:6838/4203347941 2024-09-12T02:16:36.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.380+0000 7f82ff7fe640 1 -- 172.21.15.16:0/41991380 <== mds.1 v2:172.21.15.184:6838/4203347941 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5628db430860 con 0x5628db4dc1f0 2024-09-12T02:16:36.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.380+0000 7f82ff7fe640 10 client.20334 handle_command_reply: tid=0 2024-09-12T02:16:36.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.381+0000 7f832872b740 1 client.20334 shutdown 2024-09-12T02:16:36.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.381+0000 7f832872b740 1 -- 172.21.15.16:0/41991380 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5628db4beb40 msgr2=0x5628db4c9460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:36.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.381+0000 7f832872b740 1 --2- 172.21.15.16:0/41991380 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5628db4beb40 0x5628db4c9460 secure :-1 s=READY pgs=1966 cs=0 l=1 rev1=1 crypto rx=0x7f831409ba90 tx=0x7f8314098130 comp rx=0 tx=0).stop 2024-09-12T02:16:36.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.381+0000 7f832872b740 1 -- 172.21.15.16:0/41991380 shutdown_connections 2024-09-12T02:16:36.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.381+0000 7f832872b740 1 --2- 172.21.15.16:0/41991380 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x5628db4dc1f0 0x5628db4de5e0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:36.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.381+0000 7f82ff7fe640 0 client.20334 ms_handle_reset on v2:172.21.15.184:6838/4203347941 2024-09-12T02:16:36.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.381+0000 7f832872b740 1 --2- 172.21.15.16:0/41991380 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5628db4cc780 0x5628db4c9bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:36.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.381+0000 7f832872b740 1 --2- 172.21.15.16:0/41991380 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5628db4beb40 0x5628db4c9460 unknown :-1 s=CLOSED pgs=1966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:36.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.381+0000 7f832872b740 1 --2- 172.21.15.16:0/41991380 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5628db4bd5d0 0x5628db4cc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:36.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.381+0000 7f832872b740 1 -- 172.21.15.16:0/41991380 >> 172.21.15.16:0/41991380 conn(0x5628db486310 msgr2=0x5628db4bf990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:36.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.381+0000 7f832872b740 1 -- 172.21.15.16:0/41991380 shutdown_connections 2024-09-12T02:16:36.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.381+0000 7f832872b740 1 -- 172.21.15.16:0/41991380 wait complete. 2024-09-12T02:16:36.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.381+0000 7f832872b740 20 client.20334 trim_cache size 0 max 16384 2024-09-12T02:16:36.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.394+0000 7f832872b740 1 Processor -- start 2024-09-12T02:16:36.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.394+0000 7f832872b740 1 -- start start 2024-09-12T02:16:36.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.394+0000 7f832872b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5628db4bd5d0 0x5628db4b9d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:36.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.394+0000 7f832872b740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5628db4beb40 0x5628db4cb3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:36.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.394+0000 7f832872b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5628db4cc780 0x5628db4e3c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:36.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.394+0000 7f832872b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f83201adae0 con 0x5628db4bd5d0 2024-09-12T02:16:36.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.394+0000 7f832872b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f83201ab3b0 con 0x5628db4cc780 2024-09-12T02:16:36.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.394+0000 7f832872b740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f83201a9220 con 0x5628db4beb40 2024-09-12T02:16:36.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.394+0000 7f831ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5628db4bd5d0 0x5628db4b9d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:36.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.394+0000 7f831ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5628db4bd5d0 0x5628db4b9d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53714/0 (socket says 172.21.15.16:53714) 2024-09-12T02:16:36.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.394+0000 7f831ffff640 1 -- 172.21.15.16:0/3036254496 learned_addr learned my addr 172.21.15.16:0/3036254496 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:16:36.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.394+0000 7f831f7fe640 1 --2- 172.21.15.16:0/3036254496 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5628db4beb40 0x5628db4cb3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:36.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.394+0000 7f831ffff640 1 -- 172.21.15.16:0/3036254496 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5628db4beb40 msgr2=0x5628db4cb3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:36.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.394+0000 7f831ffff640 1 --2- 172.21.15.16:0/3036254496 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5628db4beb40 0x5628db4cb3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:36.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.394+0000 7f831ffff640 1 -- 172.21.15.16:0/3036254496 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5628db4cc780 msgr2=0x5628db4e3c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:36.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.394+0000 7f83248ef640 1 --2- 172.21.15.16:0/3036254496 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5628db4cc780 0x5628db4e3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:36.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.395+0000 7f831ffff640 1 --2- 172.21.15.16:0/3036254496 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5628db4cc780 0x5628db4e3c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:36.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.395+0000 7f831ffff640 1 -- 172.21.15.16:0/3036254496 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5628db4cbdd0 con 0x5628db4bd5d0 2024-09-12T02:16:36.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.395+0000 7f831f7fe640 1 --2- 172.21.15.16:0/3036254496 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5628db4beb40 0x5628db4cb3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:16:36.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.395+0000 7f83248ef640 1 --2- 172.21.15.16:0/3036254496 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5628db4cc780 0x5628db4e3c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:16:36.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.395+0000 7f831ffff640 1 --2- 172.21.15.16:0/3036254496 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5628db4bd5d0 0x5628db4b9d70 secure :-1 s=READY pgs=1967 cs=0 l=1 rev1=1 crypto rx=0x7f8310006010 tx=0x7f8310031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=934b49171a2c4847 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:36.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.395+0000 7f8305ffb640 1 -- 172.21.15.16:0/3036254496 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8310035040 con 0x5628db4bd5d0 2024-09-12T02:16:36.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.395+0000 7f8305ffb640 1 -- 172.21.15.16:0/3036254496 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8310049070 con 0x5628db4bd5d0 2024-09-12T02:16:36.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.395+0000 7f8305ffb640 1 -- 172.21.15.16:0/3036254496 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8310004d90 con 0x5628db4bd5d0 2024-09-12T02:16:36.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.395+0000 7f832872b740 1 -- 172.21.15.16:0/3036254496 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5628db4bd5d0 msgr2=0x5628db4b9d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:36.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.395+0000 7f832872b740 1 --2- 172.21.15.16:0/3036254496 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5628db4bd5d0 0x5628db4b9d70 secure :-1 s=READY pgs=1967 cs=0 l=1 rev1=1 crypto rx=0x7f8310006010 tx=0x7f8310031400 comp rx=0 tx=0).stop 2024-09-12T02:16:36.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.396+0000 7f832872b740 1 -- 172.21.15.16:0/3036254496 shutdown_connections 2024-09-12T02:16:36.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.396+0000 7f832872b740 1 --2- 172.21.15.16:0/3036254496 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5628db4cc780 0x5628db4e3c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:36.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.396+0000 7f832872b740 1 --2- 172.21.15.16:0/3036254496 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5628db4beb40 0x5628db4cb3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:36.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.396+0000 7f832872b740 1 --2- 172.21.15.16:0/3036254496 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5628db4bd5d0 0x5628db4b9d70 unknown :-1 s=CLOSED pgs=1967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:36.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.396+0000 7f832872b740 1 -- 172.21.15.16:0/3036254496 >> 172.21.15.16:0/3036254496 conn(0x5628db486310 msgr2=0x5628db4cacc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:36.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.396+0000 7f832872b740 1 -- 172.21.15.16:0/3036254496 shutdown_connections 2024-09-12T02:16:36.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.396+0000 7f832872b740 1 -- 172.21.15.16:0/3036254496 wait complete. 2024-09-12T02:16:36.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.396+0000 7f832872b740 1 -- start start 2024-09-12T02:16:36.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.396+0000 7f832872b740 1 Processor -- start 2024-09-12T02:16:36.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.396+0000 7f832872b740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5628db4bd5d0 0x5628db4d4df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:36.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.396+0000 7f832872b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5628db4beb40 0x5628db4c6090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:36.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.396+0000 7f832872b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5628db4cc780 0x5628db4c65d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:36.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.396+0000 7f832872b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f83201a9870 con 0x5628db4cc780 2024-09-12T02:16:36.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.396+0000 7f832872b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f83201aace0 con 0x5628db4beb40 2024-09-12T02:16:36.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.396+0000 7f832872b740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f83201a9f40 con 0x5628db4bd5d0 2024-09-12T02:16:36.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.397+0000 7f832872b740 10 client.0 _ll_register_callbacks cb 0x7f8314005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:16:36.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.397+0000 7f83248ef640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5628db4cc780 0x5628db4c65d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:36.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.397+0000 7f83248ef640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5628db4cc780 0x5628db4c65d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53716/0 (socket says 172.21.15.16:53716) 2024-09-12T02:16:36.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.397+0000 7f831f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5628db4beb40 0x5628db4c6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:36.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.397+0000 7f83248ef640 1 -- 172.21.15.16:0/396914128 learned_addr learned my addr 172.21.15.16:0/396914128 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:16:36.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.397+0000 7f831ffff640 1 --2- 172.21.15.16:0/396914128 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5628db4bd5d0 0x5628db4d4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:36.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.397+0000 7f83248ef640 1 -- 172.21.15.16:0/396914128 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5628db4bd5d0 msgr2=0x5628db4d4df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:36.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.397+0000 7f83248ef640 1 --2- 172.21.15.16:0/396914128 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5628db4bd5d0 0x5628db4d4df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:36.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.397+0000 7f83248ef640 1 -- 172.21.15.16:0/396914128 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5628db4beb40 msgr2=0x5628db4c6090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:36.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.397+0000 7f83248ef640 1 --2- 172.21.15.16:0/396914128 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5628db4beb40 0x5628db4c6090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:36.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.397+0000 7f83248ef640 1 -- 172.21.15.16:0/396914128 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5628db4c6cc0 con 0x5628db4cc780 2024-09-12T02:16:36.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.397+0000 7f831f7fe640 1 --2- 172.21.15.16:0/396914128 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5628db4beb40 0x5628db4c6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:16:36.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.397+0000 7f831ffff640 1 --2- 172.21.15.16:0/396914128 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5628db4bd5d0 0x5628db4d4df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:16:36.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.397+0000 7f83248ef640 1 --2- 172.21.15.16:0/396914128 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5628db4cc780 0x5628db4c65d0 secure :-1 s=READY pgs=1968 cs=0 l=1 rev1=1 crypto rx=0x7f8318007a90 tx=0x7f8318007550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9501af0aef12639 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:36.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.397+0000 7f82ff7fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:16:36.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.397+0000 7f82ff7fe640 1 -- 172.21.15.16:0/396914128 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8318004030 con 0x5628db4cc780 2024-09-12T02:16:36.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.397+0000 7f82ff7fe640 1 -- 172.21.15.16:0/396914128 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f83180041d0 con 0x5628db4cc780 2024-09-12T02:16:36.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.397+0000 7f832872b740 1 -- 172.21.15.16:0/396914128 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8310049070 con 0x5628db4cc780 2024-09-12T02:16:36.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.398+0000 7f82ff7fe640 1 -- 172.21.15.16:0/396914128 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f83180043d0 con 0x5628db4cc780 2024-09-12T02:16:36.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.398+0000 7f832872b740 10 client.20340 fetch_fsmap learned FSMap version 25 2024-09-12T02:16:36.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.398+0000 7f832872b740 1 -- 172.21.15.16:0/396914128 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5628db546a10 con 0x5628db4cc780 2024-09-12T02:16:36.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.398+0000 7f82ff7fe640 1 -- 172.21.15.16:0/396914128 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8310049070 con 0x5628db4cc780 2024-09-12T02:16:36.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.398+0000 7f82ff7fe640 1 -- 172.21.15.16:0/396914128 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f831800e470 con 0x5628db4cc780 2024-09-12T02:16:36.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.398+0000 7f832872b740 10 client.20340 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:16:36.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.398+0000 7f832872b740 10 client.20340 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T02:16:36.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.398+0000 7f832872b740 20 client.20340 populate_metadata read hostname 'smithi016' 2024-09-12T02:16:36.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.398+0000 7f832872b740 1 --2- 172.21.15.16:0/396914128 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x5628db4dc1f0 0x5628db549200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:36.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.398+0000 7f832872b740 4 client.20340 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T02:16:36.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.398+0000 7f832872b740 1 -- 172.21.15.16:0/396914128 --> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5628db42ad70 con 0x5628db4dc1f0 2024-09-12T02:16:36.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.398+0000 7f831ffff640 1 --2- 172.21.15.16:0/396914128 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x5628db4dc1f0 0x5628db549200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:36.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.399+0000 7f831ffff640 1 --2- 172.21.15.16:0/396914128 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x5628db4dc1f0 0x5628db549200 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=512c18d20773fc90 server_cookie=c9e242092335112c in_seq=0 out_seq=0 2024-09-12T02:16:36.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.399+0000 7f82ff7fe640 10 client.20340 ms_handle_connect on v2:172.21.15.184:6838/4203347941 2024-09-12T02:16:36.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.399+0000 7f82ff7fe640 1 -- 172.21.15.16:0/396914128 <== mds.1 v2:172.21.15.184:6838/4203347941 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5628db42ad70 con 0x5628db4dc1f0 2024-09-12T02:16:36.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.399+0000 7f82ff7fe640 10 client.20340 handle_command_reply: tid=0 2024-09-12T02:16:36.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.399+0000 7f832872b740 1 client.20340 shutdown 2024-09-12T02:16:36.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.399+0000 7f832872b740 1 -- 172.21.15.16:0/396914128 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5628db4cc780 msgr2=0x5628db4c65d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:36.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.399+0000 7f832872b740 1 --2- 172.21.15.16:0/396914128 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5628db4cc780 0x5628db4c65d0 secure :-1 s=READY pgs=1968 cs=0 l=1 rev1=1 crypto rx=0x7f8318007a90 tx=0x7f8318007550 comp rx=0 tx=0).stop 2024-09-12T02:16:36.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.399+0000 7f832872b740 1 -- 172.21.15.16:0/396914128 shutdown_connections 2024-09-12T02:16:36.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.399+0000 7f832872b740 1 --2- 172.21.15.16:0/396914128 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x5628db4dc1f0 0x5628db549200 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:36.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.399+0000 7f832872b740 1 --2- 172.21.15.16:0/396914128 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5628db4cc780 0x5628db4c65d0 unknown :-1 s=CLOSED pgs=1968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:36.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.399+0000 7f82ff7fe640 0 client.20340 ms_handle_reset on v2:172.21.15.184:6838/4203347941 2024-09-12T02:16:36.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.399+0000 7f832872b740 1 --2- 172.21.15.16:0/396914128 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5628db4beb40 0x5628db4c6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:36.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.399+0000 7f832872b740 1 --2- 172.21.15.16:0/396914128 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5628db4bd5d0 0x5628db4d4df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:36.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.399+0000 7f832872b740 1 -- 172.21.15.16:0/396914128 >> 172.21.15.16:0/396914128 conn(0x5628db486310 msgr2=0x5628db4cb4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:36.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.399+0000 7f832872b740 1 -- 172.21.15.16:0/396914128 shutdown_connections 2024-09-12T02:16:36.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.399+0000 7f832872b740 1 -- 172.21.15.16:0/396914128 wait complete. 2024-09-12T02:16:36.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.400+0000 7f832872b740 20 client.20340 trim_cache size 0 max 16384 2024-09-12T02:16:36.405 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T02:16:36.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.400+0000 7f8326352640 1 -- 172.21.15.16:0/2428650221 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f83080a2b80 msgr2=0x7f83080a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:36.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.400+0000 7f8326352640 1 --2- 172.21.15.16:0/2428650221 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f83080a2b80 0x7f83080a5040 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f8318007fd0 tx=0x7f8318007850 comp rx=0 tx=0).stop 2024-09-12T02:16:36.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.400+0000 7f8326352640 1 -- 172.21.15.16:0/2428650221 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8320101420 msgr2=0x7f8320198920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:36.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.400+0000 7f8326352640 1 --2- 172.21.15.16:0/2428650221 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8320101420 0x7f8320198920 secure :-1 s=READY pgs=1964 cs=0 l=1 rev1=1 crypto rx=0x7f831400e9f0 tx=0x7f831400eeb0 comp rx=0 tx=0).stop 2024-09-12T02:16:36.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.400+0000 7f8326352640 1 -- 172.21.15.16:0/2428650221 shutdown_connections 2024-09-12T02:16:36.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.400+0000 7f8326352640 1 --2- 172.21.15.16:0/2428650221 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f83080a2b80 0x7f83080a5040 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:36.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.400+0000 7f8326352640 1 --2- 172.21.15.16:0/2428650221 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8320106a80 0x7f832019ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:36.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.400+0000 7f8326352640 1 --2- 172.21.15.16:0/2428650221 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8320101420 0x7f8320198920 unknown :-1 s=CLOSED pgs=1964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:36.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.400+0000 7f8326352640 1 --2- 172.21.15.16:0/2428650221 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8320100a80 0x7f83201983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:36.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.400+0000 7f8326352640 1 -- 172.21.15.16:0/2428650221 >> 172.21.15.16:0/2428650221 conn(0x7f83200fc7a0 msgr2=0x7f832006a820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:36.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.400+0000 7f8326352640 1 -- 172.21.15.16:0/2428650221 shutdown_connections 2024-09-12T02:16:36.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:36.400+0000 7f8326352640 1 -- 172.21.15.16:0/2428650221 wait complete. 2024-09-12T02:16:36.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:16:36 smithi033 ceph-mon[35332]: pgmap v775: 97 pgs: 97 active+clean; 3.4 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-09-12T02:16:36.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:16:36 smithi016 ceph-mon[30947]: pgmap v775: 97 pgs: 97 active+clean; 3.4 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-09-12T02:16:36.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:16:36 smithi184 ceph-mon[35593]: pgmap v775: 97 pgs: 97 active+clean; 3.4 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-09-12T02:16:37.137 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-12T02:16:37.397 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:16:37.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.877+0000 7f7ed04a1640 1 -- 172.21.15.16:0/2951873007 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ec81036e0 msgr2=0x7f7ec8107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:37.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.877+0000 7f7ed04a1640 1 --2- 172.21.15.16:0/2951873007 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ec81036e0 0x7f7ec8107b90 secure :-1 s=READY pgs=1969 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc009930 tx=0x7f7ebc02a420 comp rx=0 tx=0).stop 2024-09-12T02:16:37.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.877+0000 7f7ed04a1640 1 -- 172.21.15.16:0/2951873007 shutdown_connections 2024-09-12T02:16:37.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.877+0000 7f7ed04a1640 1 --2- 172.21.15.16:0/2951873007 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7ec81082a0 0x7f7ec810ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:37.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.877+0000 7f7ed04a1640 1 --2- 172.21.15.16:0/2951873007 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ec81036e0 0x7f7ec8107b90 unknown :-1 s=CLOSED pgs=1969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:37.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.877+0000 7f7ed04a1640 1 --2- 172.21.15.16:0/2951873007 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7ec8102d30 0x7f7ec8103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:37.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.877+0000 7f7ed04a1640 1 -- 172.21.15.16:0/2951873007 >> 172.21.15.16:0/2951873007 conn(0x7f7ec80fc7d0 msgr2=0x7f7ec80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:37.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.877+0000 7f7ed04a1640 1 -- 172.21.15.16:0/2951873007 shutdown_connections 2024-09-12T02:16:37.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.877+0000 7f7ed04a1640 1 -- 172.21.15.16:0/2951873007 wait complete. 2024-09-12T02:16:37.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.877+0000 7f7ed04a1640 1 Processor -- start 2024-09-12T02:16:37.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.877+0000 7f7ed04a1640 1 -- start start 2024-09-12T02:16:37.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.878+0000 7f7ed04a1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ec8102d30 0x7f7ec81985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:37.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.878+0000 7f7ed04a1640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7ec81036e0 0x7f7ec8198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:37.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.878+0000 7f7ed04a1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7ec81082a0 0x7f7ec819ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:37.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.878+0000 7f7ed04a1640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7ec81120b0 con 0x7f7ec8102d30 2024-09-12T02:16:37.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.878+0000 7f7ed04a1640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7ec8111f30 con 0x7f7ec81082a0 2024-09-12T02:16:37.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.878+0000 7f7ed04a1640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7ec8112230 con 0x7f7ec81036e0 2024-09-12T02:16:37.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.878+0000 7f7ece23d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ec8102d30 0x7f7ec81985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:37.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.878+0000 7f7ece23d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ec8102d30 0x7f7ec81985b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53740/0 (socket says 172.21.15.16:53740) 2024-09-12T02:16:37.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.878+0000 7f7ece23d640 1 -- 172.21.15.16:0/4197433031 learned_addr learned my addr 172.21.15.16:0/4197433031 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:16:37.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.878+0000 7f7ecda3c640 1 --2- 172.21.15.16:0/4197433031 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7ec81036e0 0x7f7ec8198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:37.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.878+0000 7f7ecea3e640 1 --2- 172.21.15.16:0/4197433031 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7ec81082a0 0x7f7ec819ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:37.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.878+0000 7f7ece23d640 1 -- 172.21.15.16:0/4197433031 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7ec81036e0 msgr2=0x7f7ec8198af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:37.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.878+0000 7f7ece23d640 1 --2- 172.21.15.16:0/4197433031 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7ec81036e0 0x7f7ec8198af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:37.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.878+0000 7f7ece23d640 1 -- 172.21.15.16:0/4197433031 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7ec81082a0 msgr2=0x7f7ec819ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:37.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.878+0000 7f7ece23d640 1 --2- 172.21.15.16:0/4197433031 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7ec81082a0 0x7f7ec819ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:37.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.878+0000 7f7ece23d640 1 -- 172.21.15.16:0/4197433031 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ec819d560 con 0x7f7ec8102d30 2024-09-12T02:16:37.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.878+0000 7f7ecea3e640 1 --2- 172.21.15.16:0/4197433031 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7ec81082a0 0x7f7ec819ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:16:37.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.878+0000 7f7ece23d640 1 --2- 172.21.15.16:0/4197433031 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ec8102d30 0x7f7ec81985b0 secure :-1 s=READY pgs=1970 cs=0 l=1 rev1=1 crypto rx=0x7f7eb800eb90 tx=0x7f7eb800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:37.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.879+0000 7f7ec77fe640 1 -- 172.21.15.16:0/4197433031 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eb8004d20 con 0x7f7ec8102d30 2024-09-12T02:16:37.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.879+0000 7f7ed04a1640 1 -- 172.21.15.16:0/4197433031 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ec819d850 con 0x7f7ec8102d30 2024-09-12T02:16:37.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.879+0000 7f7ec77fe640 1 -- 172.21.15.16:0/4197433031 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7eb800ee90 con 0x7f7ec8102d30 2024-09-12T02:16:37.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.879+0000 7f7ed04a1640 1 -- 172.21.15.16:0/4197433031 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ec81a5130 con 0x7f7ec8102d30 2024-09-12T02:16:37.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.879+0000 7f7ec77fe640 1 -- 172.21.15.16:0/4197433031 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eb8010830 con 0x7f7ec8102d30 2024-09-12T02:16:37.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.881+0000 7f7ec77fe640 1 -- 172.21.15.16:0/4197433031 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f7eb8005020 con 0x7f7ec8102d30 2024-09-12T02:16:37.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.881+0000 7f7ed287a740 1 Processor -- start 2024-09-12T02:16:37.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.881+0000 7f7ed287a740 1 -- start start 2024-09-12T02:16:37.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.882+0000 7f7ed287a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56419a762b40 0x56419a761090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:37.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.882+0000 7f7ed287a740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56419a7615d0 0x56419a761a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:37.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.882+0000 7f7ed287a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56419a76c370 0x56419a76e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:37.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.882+0000 7f7ed287a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56419a6c72f0 con 0x56419a762b40 2024-09-12T02:16:37.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.882+0000 7f7ed287a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56419a66ecf0 con 0x56419a76c370 2024-09-12T02:16:37.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.882+0000 7f7ed287a740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x56419a6c32b0 con 0x56419a7615d0 2024-09-12T02:16:37.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.882+0000 7f7ecea3e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56419a762b40 0x56419a761090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:37.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.882+0000 7f7ec77fe640 1 --2- 172.21.15.16:0/4197433031 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f7eb00a2b80 0x7f7eb00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:37.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.882+0000 7f7ecda3c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56419a76c370 0x56419a76e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:37.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.882+0000 7f7ecea3e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56419a762b40 0x56419a761090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53748/0 (socket says 172.21.15.16:53748) 2024-09-12T02:16:37.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.882+0000 7f7ecea3e640 1 -- 172.21.15.16:0/454331116 learned_addr learned my addr 172.21.15.16:0/454331116 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:16:37.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.882+0000 7f7ecea3e640 1 --2- 172.21.15.16:0/4197433031 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f7eb00a2b80 0x7f7eb00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:37.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.882+0000 7f7ec77fe640 1 -- 172.21.15.16:0/4197433031 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f7eb80cf760 con 0x7f7ec8102d30 2024-09-12T02:16:37.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.882+0000 7f7ece23d640 1 --2- 172.21.15.16:0/454331116 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56419a7615d0 0x56419a761a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:37.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.882+0000 7f7ecea3e640 1 -- 172.21.15.16:0/454331116 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56419a7615d0 msgr2=0x56419a761a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:37.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.882+0000 7f7ecea3e640 1 --2- 172.21.15.16:0/454331116 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56419a7615d0 0x56419a761a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:37.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.882+0000 7f7ecea3e640 1 -- 172.21.15.16:0/454331116 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56419a76c370 msgr2=0x56419a76e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:37.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.882+0000 7f7ecea3e640 1 --2- 172.21.15.16:0/454331116 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56419a76c370 0x56419a76e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:37.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.882+0000 7f7ecea3e640 1 -- 172.21.15.16:0/454331116 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56419a6533a0 con 0x56419a762b40 2024-09-12T02:16:37.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.882+0000 7f7ecda3c640 1 --2- 172.21.15.16:0/454331116 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56419a76c370 0x56419a76e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:16:37.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.883+0000 7f7ecea3e640 1 --2- 172.21.15.16:0/454331116 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56419a762b40 0x56419a761090 secure :-1 s=READY pgs=1971 cs=0 l=1 rev1=1 crypto rx=0x7f7ec0013a00 tx=0x7f7ec0013e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d96d4f13c9288e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:37.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.883+0000 7f7ecea3e640 1 --2- 172.21.15.16:0/4197433031 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f7eb00a2b80 0x7f7eb00a5040 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f7ec0011750 tx=0x7f7ec001e000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:37.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.883+0000 7f7ec4ff9640 1 -- 172.21.15.16:0/454331116 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ec0023070 con 0x56419a762b40 2024-09-12T02:16:37.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.883+0000 7f7ec4ff9640 1 -- 172.21.15.16:0/454331116 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ec00099b0 con 0x56419a762b40 2024-09-12T02:16:37.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.883+0000 7f7ec4ff9640 1 -- 172.21.15.16:0/454331116 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ec0004c60 con 0x56419a762b40 2024-09-12T02:16:37.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.883+0000 7f7ed287a740 1 -- 172.21.15.16:0/454331116 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56419a762b40 msgr2=0x56419a761090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:37.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.883+0000 7f7ed287a740 1 --2- 172.21.15.16:0/454331116 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56419a762b40 0x56419a761090 secure :-1 s=READY pgs=1971 cs=0 l=1 rev1=1 crypto rx=0x7f7ec0013a00 tx=0x7f7ec0013e00 comp rx=0 tx=0).stop 2024-09-12T02:16:37.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.884+0000 7f7ed287a740 1 -- 172.21.15.16:0/454331116 shutdown_connections 2024-09-12T02:16:37.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.884+0000 7f7ed287a740 1 --2- 172.21.15.16:0/454331116 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56419a76c370 0x56419a76e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:37.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.884+0000 7f7ed287a740 1 --2- 172.21.15.16:0/454331116 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56419a7615d0 0x56419a761a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:37.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.884+0000 7f7ed287a740 1 --2- 172.21.15.16:0/454331116 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56419a762b40 0x56419a761090 unknown :-1 s=CLOSED pgs=1971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:37.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.884+0000 7f7ed287a740 1 -- 172.21.15.16:0/454331116 >> 172.21.15.16:0/454331116 conn(0x56419a72a310 msgr2=0x56419a75dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:37.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.884+0000 7f7ed287a740 1 -- 172.21.15.16:0/454331116 shutdown_connections 2024-09-12T02:16:37.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.884+0000 7f7ed287a740 1 -- 172.21.15.16:0/454331116 wait complete. 2024-09-12T02:16:37.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.884+0000 7f7ed287a740 1 -- start start 2024-09-12T02:16:37.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.884+0000 7f7ed287a740 1 Processor -- start 2024-09-12T02:16:37.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.884+0000 7f7ed287a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56419a7615d0 0x56419a76bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:37.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.884+0000 7f7ed287a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56419a762b40 0x56419a769050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:37.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.884+0000 7f7ed287a740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56419a76c370 0x56419a7697e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:37.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.884+0000 7f7ed287a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56419a76f010 con 0x56419a762b40 2024-09-12T02:16:37.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.884+0000 7f7ed287a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56419a682060 con 0x56419a7615d0 2024-09-12T02:16:37.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.884+0000 7f7ed287a740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x56419a76f190 con 0x56419a76c370 2024-09-12T02:16:37.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.884+0000 7f7ed287a740 10 client.0 _ll_register_callbacks cb 0x56419a3d58c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:16:37.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.885+0000 7f7ecda3c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56419a762b40 0x56419a769050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:37.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.885+0000 7f7ecda3c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56419a762b40 0x56419a769050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53762/0 (socket says 172.21.15.16:53762) 2024-09-12T02:16:37.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.885+0000 7f7ecda3c640 1 -- 172.21.15.16:0/1144187230 learned_addr learned my addr 172.21.15.16:0/1144187230 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:16:37.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.885+0000 7f7ece23d640 1 --2- 172.21.15.16:0/1144187230 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56419a7615d0 0x56419a76bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:37.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.885+0000 7f7ecea3e640 1 --2- 172.21.15.16:0/1144187230 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56419a76c370 0x56419a7697e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:37.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.885+0000 7f7ecda3c640 1 -- 172.21.15.16:0/1144187230 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56419a76c370 msgr2=0x56419a7697e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:37.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.885+0000 7f7ecda3c640 1 --2- 172.21.15.16:0/1144187230 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56419a76c370 0x56419a7697e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:37.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.885+0000 7f7ecda3c640 1 -- 172.21.15.16:0/1144187230 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56419a7615d0 msgr2=0x56419a76bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:37.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.885+0000 7f7ecda3c640 1 --2- 172.21.15.16:0/1144187230 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56419a7615d0 0x56419a76bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:37.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.885+0000 7f7ecda3c640 1 -- 172.21.15.16:0/1144187230 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56419a4383c0 con 0x56419a762b40 2024-09-12T02:16:37.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.885+0000 7f7ecda3c640 1 --2- 172.21.15.16:0/1144187230 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56419a762b40 0x56419a769050 secure :-1 s=READY pgs=1972 cs=0 l=1 rev1=1 crypto rx=0x7f7ec80697d0 tx=0x7f7ebc0052f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c26a4bc09780bf0e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:37.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.885+0000 7f7eadffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:16:37.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.885+0000 7f7eadffb640 1 -- 172.21.15.16:0/1144187230 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ebc04a060 con 0x56419a762b40 2024-09-12T02:16:37.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.885+0000 7f7eadffb640 1 -- 172.21.15.16:0/1144187230 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f7ebc03b8e0 con 0x56419a762b40 2024-09-12T02:16:37.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.885+0000 7f7ed287a740 1 -- 172.21.15.16:0/1144187230 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ec00099b0 con 0x56419a762b40 2024-09-12T02:16:37.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.885+0000 7f7eadffb640 1 -- 172.21.15.16:0/1144187230 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7ebc035070 con 0x56419a762b40 2024-09-12T02:16:37.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.886+0000 7f7ed287a740 10 client.20352 fetch_fsmap learned FSMap version 25 2024-09-12T02:16:37.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.886+0000 7f7ed287a740 1 -- 172.21.15.16:0/1144187230 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56419a77aaf0 con 0x56419a762b40 2024-09-12T02:16:37.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.886+0000 7f7eadffb640 1 -- 172.21.15.16:0/1144187230 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ec00099b0 con 0x56419a762b40 2024-09-12T02:16:37.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.886+0000 7f7eadffb640 1 -- 172.21.15.16:0/1144187230 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f7ebc04f070 con 0x56419a762b40 2024-09-12T02:16:37.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.886+0000 7f7ed287a740 10 client.20352 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:16:37.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.886+0000 7f7ed287a740 10 client.20352 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:16:37.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.886+0000 7f7ed287a740 20 client.20352 populate_metadata read hostname 'smithi016' 2024-09-12T02:16:37.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.886+0000 7f7ed287a740 1 --2- 172.21.15.16:0/1144187230 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x56419a77bfb0 0x56419a77e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:37.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.886+0000 7f7ed287a740 4 client.20352 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:16:37.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.886+0000 7f7ed287a740 1 -- 172.21.15.16:0/1144187230 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56419a6d4860 con 0x56419a77bfb0 2024-09-12T02:16:37.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.886+0000 7f7ece23d640 1 --2- 172.21.15.16:0/1144187230 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x56419a77bfb0 0x56419a77e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:37.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.887+0000 7f7ece23d640 1 --2- 172.21.15.16:0/1144187230 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x56419a77bfb0 0x56419a77e3f0 crc :-1 s=READY pgs=435 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6cd246228eada557 server_cookie=a5b6614e3a85c1b7 in_seq=0 out_seq=0 2024-09-12T02:16:37.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.887+0000 7f7eadffb640 10 client.20352 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:16:37.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.887+0000 7f7eadffb640 1 -- 172.21.15.16:0/1144187230 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56419a6d4860 con 0x56419a77bfb0 2024-09-12T02:16:37.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.887+0000 7f7eadffb640 10 client.20352 handle_command_reply: tid=0 2024-09-12T02:16:37.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.888+0000 7f7ed287a740 1 client.20352 shutdown 2024-09-12T02:16:37.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.888+0000 7f7ed287a740 1 -- 172.21.15.16:0/1144187230 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56419a762b40 msgr2=0x56419a769050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:37.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.888+0000 7f7ed287a740 1 --2- 172.21.15.16:0/1144187230 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56419a762b40 0x56419a769050 secure :-1 s=READY pgs=1972 cs=0 l=1 rev1=1 crypto rx=0x7f7ec80697d0 tx=0x7f7ebc0052f0 comp rx=0 tx=0).stop 2024-09-12T02:16:37.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.888+0000 7f7ed287a740 1 -- 172.21.15.16:0/1144187230 shutdown_connections 2024-09-12T02:16:37.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.888+0000 7f7ed287a740 1 --2- 172.21.15.16:0/1144187230 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x56419a77bfb0 0x56419a77e3f0 crc :-1 s=READY pgs=435 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:37.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.888+0000 7f7ed287a740 1 --2- 172.21.15.16:0/1144187230 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56419a76c370 0x56419a7697e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:37.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.888+0000 7f7ed287a740 1 --2- 172.21.15.16:0/1144187230 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56419a762b40 0x56419a769050 unknown :-1 s=CLOSED pgs=1972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:37.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.888+0000 7f7ed287a740 1 --2- 172.21.15.16:0/1144187230 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56419a7615d0 0x56419a76bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:37.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.888+0000 7f7ed287a740 1 -- 172.21.15.16:0/1144187230 >> 172.21.15.16:0/1144187230 conn(0x56419a72a310 msgr2=0x56419a763960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:37.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.888+0000 7f7eadffb640 0 client.20352 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:16:37.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.888+0000 7f7ed287a740 1 -- 172.21.15.16:0/1144187230 shutdown_connections 2024-09-12T02:16:37.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.888+0000 7f7ed287a740 1 -- 172.21.15.16:0/1144187230 wait complete. 2024-09-12T02:16:37.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.888+0000 7f7ed287a740 20 client.20352 trim_cache size 0 max 16384 2024-09-12T02:16:37.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.901+0000 7f7ed287a740 1 Processor -- start 2024-09-12T02:16:37.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.901+0000 7f7ed287a740 1 -- start start 2024-09-12T02:16:37.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.901+0000 7f7ed287a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56419a7615d0 0x56419a75de10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:37.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.901+0000 7f7ed287a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56419a762b40 0x56419a783cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:37.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.901+0000 7f7ed287a740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56419a76c370 0x56419a784200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:37.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.901+0000 7f7ed287a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7ec81adae0 con 0x56419a7615d0 2024-09-12T02:16:37.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.901+0000 7f7ed287a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7ec81ab3b0 con 0x56419a762b40 2024-09-12T02:16:37.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.901+0000 7f7ed287a740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7ec81a9220 con 0x56419a76c370 2024-09-12T02:16:37.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.901+0000 7f7ece23d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56419a7615d0 0x56419a75de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:37.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.901+0000 7f7ece23d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56419a7615d0 0x56419a75de10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53766/0 (socket says 172.21.15.16:53766) 2024-09-12T02:16:37.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.901+0000 7f7ece23d640 1 -- 172.21.15.16:0/3426042138 learned_addr learned my addr 172.21.15.16:0/3426042138 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:16:37.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.902+0000 7f7ecda3c640 1 --2- 172.21.15.16:0/3426042138 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56419a762b40 0x56419a783cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:37.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.902+0000 7f7ece23d640 1 -- 172.21.15.16:0/3426042138 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56419a76c370 msgr2=0x56419a784200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:37.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.902+0000 7f7ece23d640 1 --2- 172.21.15.16:0/3426042138 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56419a76c370 0x56419a784200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:37.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.902+0000 7f7ece23d640 1 -- 172.21.15.16:0/3426042138 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56419a762b40 msgr2=0x56419a783cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:37.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.902+0000 7f7ece23d640 1 --2- 172.21.15.16:0/3426042138 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56419a762b40 0x56419a783cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:37.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.902+0000 7f7ece23d640 1 -- 172.21.15.16:0/3426042138 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56419a75e960 con 0x56419a7615d0 2024-09-12T02:16:37.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.902+0000 7f7ece23d640 1 --2- 172.21.15.16:0/3426042138 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56419a7615d0 0x56419a75de10 secure :-1 s=READY pgs=1973 cs=0 l=1 rev1=1 crypto rx=0x7f7eb8009ae0 tx=0x7f7eb80cf040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cb42218f9860bea server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:37.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.902+0000 7f7eaffff640 1 -- 172.21.15.16:0/3426042138 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eb809c060 con 0x56419a7615d0 2024-09-12T02:16:37.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.902+0000 7f7eaffff640 1 -- 172.21.15.16:0/3426042138 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7eb80a0410 con 0x56419a7615d0 2024-09-12T02:16:37.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.902+0000 7f7eaffff640 1 -- 172.21.15.16:0/3426042138 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eb8005590 con 0x56419a7615d0 2024-09-12T02:16:37.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.903+0000 7f7ed287a740 1 -- 172.21.15.16:0/3426042138 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56419a7615d0 msgr2=0x56419a75de10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:37.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.903+0000 7f7ed287a740 1 --2- 172.21.15.16:0/3426042138 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56419a7615d0 0x56419a75de10 secure :-1 s=READY pgs=1973 cs=0 l=1 rev1=1 crypto rx=0x7f7eb8009ae0 tx=0x7f7eb80cf040 comp rx=0 tx=0).stop 2024-09-12T02:16:37.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.903+0000 7f7ed287a740 1 -- 172.21.15.16:0/3426042138 shutdown_connections 2024-09-12T02:16:37.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.903+0000 7f7ed287a740 1 --2- 172.21.15.16:0/3426042138 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56419a76c370 0x56419a784200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:37.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.903+0000 7f7ed287a740 1 --2- 172.21.15.16:0/3426042138 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56419a762b40 0x56419a783cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:37.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.903+0000 7f7ed287a740 1 --2- 172.21.15.16:0/3426042138 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56419a7615d0 0x56419a75de10 unknown :-1 s=CLOSED pgs=1973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:37.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.903+0000 7f7ed287a740 1 -- 172.21.15.16:0/3426042138 >> 172.21.15.16:0/3426042138 conn(0x56419a72a310 msgr2=0x56419a75d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:37.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.903+0000 7f7ed287a740 1 -- 172.21.15.16:0/3426042138 shutdown_connections 2024-09-12T02:16:37.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.903+0000 7f7ed287a740 1 -- 172.21.15.16:0/3426042138 wait complete. 2024-09-12T02:16:37.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.903+0000 7f7ed287a740 1 -- start start 2024-09-12T02:16:37.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.903+0000 7f7ed287a740 1 Processor -- start 2024-09-12T02:16:37.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.903+0000 7f7ed287a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56419a762b40 0x56419a774790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:37.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.903+0000 7f7ed287a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56419a76c370 0x56419a774cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:37.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.903+0000 7f7ed287a740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56419a765b50 0x56419a765fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:37.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.903+0000 7f7ed287a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7ec81a9870 con 0x56419a76c370 2024-09-12T02:16:37.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.903+0000 7f7ed287a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7ec81aace0 con 0x56419a762b40 2024-09-12T02:16:37.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.903+0000 7f7ed287a740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7ec81a9f40 con 0x56419a765b50 2024-09-12T02:16:37.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.903+0000 7f7ed287a740 10 client.0 _ll_register_callbacks cb 0x7f7ec0005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:16:37.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.904+0000 7f7ecda3c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56419a76c370 0x56419a774cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:37.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.904+0000 7f7ecda3c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56419a76c370 0x56419a774cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53772/0 (socket says 172.21.15.16:53772) 2024-09-12T02:16:37.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.904+0000 7f7ecda3c640 1 -- 172.21.15.16:0/484455021 learned_addr learned my addr 172.21.15.16:0/484455021 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:16:37.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.904+0000 7f7ecea3e640 1 --2- 172.21.15.16:0/484455021 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56419a765b50 0x56419a765fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:37.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.904+0000 7f7ece23d640 1 --2- 172.21.15.16:0/484455021 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56419a762b40 0x56419a774790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:37.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.904+0000 7f7ecda3c640 1 -- 172.21.15.16:0/484455021 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56419a765b50 msgr2=0x56419a765fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:37.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.904+0000 7f7ecda3c640 1 --2- 172.21.15.16:0/484455021 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56419a765b50 0x56419a765fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:37.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.904+0000 7f7ecda3c640 1 -- 172.21.15.16:0/484455021 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56419a762b40 msgr2=0x56419a774790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:37.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.904+0000 7f7ecda3c640 1 --2- 172.21.15.16:0/484455021 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56419a762b40 0x56419a774790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:37.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.904+0000 7f7ecda3c640 1 -- 172.21.15.16:0/484455021 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56419a764120 con 0x56419a76c370 2024-09-12T02:16:37.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.904+0000 7f7ece23d640 1 --2- 172.21.15.16:0/484455021 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56419a762b40 0x56419a774790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:16:37.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.904+0000 7f7ecda3c640 1 --2- 172.21.15.16:0/484455021 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56419a76c370 0x56419a774cd0 secure :-1 s=READY pgs=1974 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc02a940 tx=0x7f7ebc0097f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72c40b247e60a404 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:37.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.904+0000 7f7eadffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:16:37.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.904+0000 7f7eadffb640 1 -- 172.21.15.16:0/484455021 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ebc055040 con 0x56419a76c370 2024-09-12T02:16:37.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.904+0000 7f7eadffb640 1 -- 172.21.15.16:0/484455021 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f7ebc004c80 con 0x56419a76c370 2024-09-12T02:16:37.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.904+0000 7f7ed287a740 1 -- 172.21.15.16:0/484455021 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7eb80a0410 con 0x56419a76c370 2024-09-12T02:16:37.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.904+0000 7f7eadffb640 1 -- 172.21.15.16:0/484455021 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7ebc005d50 con 0x56419a76c370 2024-09-12T02:16:37.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.904+0000 7f7ed287a740 10 client.20358 fetch_fsmap learned FSMap version 25 2024-09-12T02:16:37.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.904+0000 7f7ed287a740 1 -- 172.21.15.16:0/484455021 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56419a7e6b90 con 0x56419a76c370 2024-09-12T02:16:37.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.905+0000 7f7eadffb640 1 -- 172.21.15.16:0/484455021 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7eb80a0410 con 0x56419a76c370 2024-09-12T02:16:37.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.905+0000 7f7eadffb640 1 -- 172.21.15.16:0/484455021 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f7ebc058070 con 0x56419a76c370 2024-09-12T02:16:37.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.905+0000 7f7ed287a740 10 client.20358 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:16:37.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.905+0000 7f7ed287a740 10 client.20358 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:16:37.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.905+0000 7f7ed287a740 20 client.20358 populate_metadata read hostname 'smithi016' 2024-09-12T02:16:37.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.905+0000 7f7ed287a740 1 --2- 172.21.15.16:0/484455021 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x56419a7e7310 0x56419a7e9730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:37.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.905+0000 7f7ed287a740 4 client.20358 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T02:16:37.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.905+0000 7f7ed287a740 1 -- 172.21.15.16:0/484455021 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56419a6ced70 con 0x56419a7e7310 2024-09-12T02:16:37.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.905+0000 7f7ece23d640 1 --2- 172.21.15.16:0/484455021 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x56419a7e7310 0x56419a7e9730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:37.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.905+0000 7f7ece23d640 1 --2- 172.21.15.16:0/484455021 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x56419a7e7310 0x56419a7e9730 crc :-1 s=READY pgs=436 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62f42d2c8db4821c server_cookie=8fd02a0117ad0659 in_seq=0 out_seq=0 2024-09-12T02:16:37.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.906+0000 7f7eadffb640 10 client.20358 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:16:37.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.906+0000 7f7eadffb640 1 -- 172.21.15.16:0/484455021 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56419a6ced70 con 0x56419a7e7310 2024-09-12T02:16:37.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.906+0000 7f7eadffb640 10 client.20358 handle_command_reply: tid=0 2024-09-12T02:16:37.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.906+0000 7f7ed287a740 1 client.20358 shutdown 2024-09-12T02:16:37.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.906+0000 7f7ed287a740 1 -- 172.21.15.16:0/484455021 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56419a76c370 msgr2=0x56419a774cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:37.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.906+0000 7f7ed287a740 1 --2- 172.21.15.16:0/484455021 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56419a76c370 0x56419a774cd0 secure :-1 s=READY pgs=1974 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc02a940 tx=0x7f7ebc0097f0 comp rx=0 tx=0).stop 2024-09-12T02:16:37.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.906+0000 7f7ed287a740 1 -- 172.21.15.16:0/484455021 shutdown_connections 2024-09-12T02:16:37.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.906+0000 7f7ed287a740 1 --2- 172.21.15.16:0/484455021 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x56419a7e7310 0x56419a7e9730 crc :-1 s=READY pgs=436 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:37.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.906+0000 7f7ed287a740 1 --2- 172.21.15.16:0/484455021 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56419a765b50 0x56419a765fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:37.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.906+0000 7f7ed287a740 1 --2- 172.21.15.16:0/484455021 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56419a76c370 0x56419a774cd0 unknown :-1 s=CLOSED pgs=1974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:37.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.906+0000 7f7ed287a740 1 --2- 172.21.15.16:0/484455021 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56419a762b40 0x56419a774790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:37.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.906+0000 7f7ed287a740 1 -- 172.21.15.16:0/484455021 >> 172.21.15.16:0/484455021 conn(0x56419a72a310 msgr2=0x56419a767980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:37.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.906+0000 7f7eadffb640 0 client.20358 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:16:37.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.906+0000 7f7ed287a740 1 -- 172.21.15.16:0/484455021 shutdown_connections 2024-09-12T02:16:37.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.906+0000 7f7ed287a740 1 -- 172.21.15.16:0/484455021 wait complete. 2024-09-12T02:16:37.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.906+0000 7f7ed287a740 20 client.20358 trim_cache size 0 max 16384 2024-09-12T02:16:37.913 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T02:16:37.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.907+0000 7f7ed04a1640 1 -- 172.21.15.16:0/4197433031 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f7eb00a2b80 msgr2=0x7f7eb00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:37.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.907+0000 7f7ed04a1640 1 --2- 172.21.15.16:0/4197433031 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f7eb00a2b80 0x7f7eb00a5040 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f7ec0011750 tx=0x7f7ec001e000 comp rx=0 tx=0).stop 2024-09-12T02:16:37.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.907+0000 7f7ed04a1640 1 -- 172.21.15.16:0/4197433031 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ec8102d30 msgr2=0x7f7ec81985b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:37.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.907+0000 7f7ed04a1640 1 --2- 172.21.15.16:0/4197433031 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ec8102d30 0x7f7ec81985b0 secure :-1 s=READY pgs=1970 cs=0 l=1 rev1=1 crypto rx=0x7f7eb800eb90 tx=0x7f7eb800c6a0 comp rx=0 tx=0).stop 2024-09-12T02:16:37.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.907+0000 7f7ed04a1640 1 -- 172.21.15.16:0/4197433031 shutdown_connections 2024-09-12T02:16:37.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.907+0000 7f7ed04a1640 1 --2- 172.21.15.16:0/4197433031 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f7eb00a2b80 0x7f7eb00a5040 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:37.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.907+0000 7f7ed04a1640 1 --2- 172.21.15.16:0/4197433031 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7ec81082a0 0x7f7ec819ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:37.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.907+0000 7f7ed04a1640 1 --2- 172.21.15.16:0/4197433031 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7ec81036e0 0x7f7ec8198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:37.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.907+0000 7f7ed04a1640 1 --2- 172.21.15.16:0/4197433031 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ec8102d30 0x7f7ec81985b0 unknown :-1 s=CLOSED pgs=1970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:37.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.907+0000 7f7ed04a1640 1 -- 172.21.15.16:0/4197433031 >> 172.21.15.16:0/4197433031 conn(0x7f7ec80fc7d0 msgr2=0x7f7ec8106380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:37.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.907+0000 7f7ed04a1640 1 -- 172.21.15.16:0/4197433031 shutdown_connections 2024-09-12T02:16:37.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:37.907+0000 7f7ed04a1640 1 -- 172.21.15.16:0/4197433031 wait complete. 2024-09-12T02:16:38.203 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.202+0000 7f13653d6640 1 -- 172.21.15.16:0/312412349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13601037a0 msgr2=0x7f136010a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:38.203 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.202+0000 7f13653d6640 1 --2- 172.21.15.16:0/312412349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13601037a0 0x7f136010a030 secure :-1 s=READY pgs=1975 cs=0 l=1 rev1=1 crypto rx=0x7f1358009ca0 tx=0x7f135802a8c0 comp rx=0 tx=0).stop 2024-09-12T02:16:38.203 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.202+0000 7f13653d6640 1 -- 172.21.15.16:0/312412349 shutdown_connections 2024-09-12T02:16:38.203 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.202+0000 7f13653d6640 1 --2- 172.21.15.16:0/312412349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13601037a0 0x7f136010a030 unknown :-1 s=CLOSED pgs=1975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:38.203 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.202+0000 7f13653d6640 1 --2- 172.21.15.16:0/312412349 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1360102de0 0x7f1360103260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:38.203 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.202+0000 7f13653d6640 1 --2- 172.21.15.16:0/312412349 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f13600ffe10 0x7f1360100210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:38.203 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.202+0000 7f13653d6640 1 -- 172.21.15.16:0/312412349 >> 172.21.15.16:0/312412349 conn(0x7f13600fb930 msgr2=0x7f13600fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:38.203 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.202+0000 7f13653d6640 1 -- 172.21.15.16:0/312412349 shutdown_connections 2024-09-12T02:16:38.203 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.202+0000 7f13653d6640 1 -- 172.21.15.16:0/312412349 wait complete. 2024-09-12T02:16:38.204 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.203+0000 7f13653d6640 1 Processor -- start 2024-09-12T02:16:38.206 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.203+0000 7f13653d6640 1 -- start start 2024-09-12T02:16:38.206 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.203+0000 7f13653d6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13600ffe10 0x7f1360102750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:38.206 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.203+0000 7f13653d6640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1360102de0 0x7f1360100d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:38.206 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.203+0000 7f13653d6640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f13601037a0 0x7f13601012d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:38.207 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.203+0000 7f13653d6640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f136010d840 con 0x7f13600ffe10 2024-09-12T02:16:38.207 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.203+0000 7f13653d6640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f136010d6c0 con 0x7f1360102de0 2024-09-12T02:16:38.207 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.203+0000 7f13653d6640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f136010d9c0 con 0x7f13601037a0 2024-09-12T02:16:38.207 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.203+0000 7f135effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13600ffe10 0x7f1360102750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:38.207 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.203+0000 7f135effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13600ffe10 0x7f1360102750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53786/0 (socket says 172.21.15.16:53786) 2024-09-12T02:16:38.207 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.203+0000 7f135effd640 1 -- 172.21.15.16:0/1720097857 learned_addr learned my addr 172.21.15.16:0/1720097857 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:16:38.207 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.204+0000 7f135e7fc640 1 --2- 172.21.15.16:0/1720097857 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1360102de0 0x7f1360100d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:38.207 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.204+0000 7f135f7fe640 1 --2- 172.21.15.16:0/1720097857 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f13601037a0 0x7f13601012d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:38.207 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.204+0000 7f135effd640 1 -- 172.21.15.16:0/1720097857 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f13601037a0 msgr2=0x7f13601012d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:38.207 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.204+0000 7f135effd640 1 --2- 172.21.15.16:0/1720097857 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f13601037a0 0x7f13601012d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:38.207 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.204+0000 7f135effd640 1 -- 172.21.15.16:0/1720097857 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1360102de0 msgr2=0x7f1360100d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:38.207 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.204+0000 7f135effd640 1 --2- 172.21.15.16:0/1720097857 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1360102de0 0x7f1360100d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:38.208 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.204+0000 7f135effd640 1 -- 172.21.15.16:0/1720097857 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1360101b60 con 0x7f13600ffe10 2024-09-12T02:16:38.208 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.204+0000 7f135e7fc640 1 --2- 172.21.15.16:0/1720097857 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1360102de0 0x7f1360100d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:16:38.208 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.204+0000 7f135effd640 1 --2- 172.21.15.16:0/1720097857 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13600ffe10 0x7f1360102750 secure :-1 s=READY pgs=1976 cs=0 l=1 rev1=1 crypto rx=0x7f135000c970 tx=0x7f135000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:38.208 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.204+0000 7f1347fff640 1 -- 172.21.15.16:0/1720097857 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1350004220 con 0x7f13600ffe10 2024-09-12T02:16:38.208 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.204+0000 7f1347fff640 1 -- 172.21.15.16:0/1720097857 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13500043c0 con 0x7f13600ffe10 2024-09-12T02:16:38.208 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.204+0000 7f13653d6640 1 -- 172.21.15.16:0/1720097857 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13601a1ef0 con 0x7f13600ffe10 2024-09-12T02:16:38.208 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.204+0000 7f1347fff640 1 -- 172.21.15.16:0/1720097857 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f135000f450 con 0x7f13600ffe10 2024-09-12T02:16:38.208 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.204+0000 7f13653d6640 1 -- 172.21.15.16:0/1720097857 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13601a23d0 con 0x7f13600ffe10 2024-09-12T02:16:38.208 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.206+0000 7f1347fff640 1 -- 172.21.15.16:0/1720097857 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f13500079b0 con 0x7f13600ffe10 2024-09-12T02:16:38.215 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.207+0000 7f136824a740 1 Processor -- start 2024-09-12T02:16:38.215 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.207+0000 7f136824a740 1 -- start start 2024-09-12T02:16:38.215 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.207+0000 7f136824a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d670cd9a0 0x559d670cdda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:38.215 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.207+0000 7f136824a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559d670ce2e0 0x559d670d2da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:38.216 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.207+0000 7f136824a740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559d670d3370 0x559d670d5830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:38.216 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.207+0000 7f136824a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x559d670312f0 con 0x559d670cd9a0 2024-09-12T02:16:38.216 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.207+0000 7f136824a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x559d66fd9270 con 0x559d670ce2e0 2024-09-12T02:16:38.216 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.207+0000 7f136824a740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x559d6702d2b0 con 0x559d670d3370 2024-09-12T02:16:38.216 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.207+0000 7f135f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d670cd9a0 0x559d670cdda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:38.216 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.207+0000 7f135f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d670cd9a0 0x559d670cdda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53800/0 (socket says 172.21.15.16:53800) 2024-09-12T02:16:38.216 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.207+0000 7f135f7fe640 1 -- 172.21.15.16:0/1342010748 learned_addr learned my addr 172.21.15.16:0/1342010748 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:16:38.216 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.207+0000 7f1347fff640 1 --2- 172.21.15.16:0/1720097857 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f13480a2cb0 0x7f13480a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:38.216 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.207+0000 7f135e7fc640 1 --2- 172.21.15.16:0/1342010748 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559d670d3370 0x559d670d5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:38.216 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.207+0000 7f135effd640 1 --2- 172.21.15.16:0/1342010748 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559d670ce2e0 0x559d670d2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:38.216 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.208+0000 7f135f7fe640 1 -- 172.21.15.16:0/1342010748 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559d670d3370 msgr2=0x559d670d5830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:38.216 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.208+0000 7f135f7fe640 1 --2- 172.21.15.16:0/1342010748 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559d670d3370 0x559d670d5830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:38.217 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.208+0000 7f135f7fe640 1 -- 172.21.15.16:0/1342010748 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559d670ce2e0 msgr2=0x559d670d2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:38.217 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.208+0000 7f135f7fe640 1 --2- 172.21.15.16:0/1342010748 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559d670ce2e0 0x559d670d2da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:38.217 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.208+0000 7f135f7fe640 1 -- 172.21.15.16:0/1342010748 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d66fd8340 con 0x559d670cd9a0 2024-09-12T02:16:38.217 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.208+0000 7f135f7fe640 1 --2- 172.21.15.16:0/1720097857 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f13480a2cb0 0x7f13480a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:38.217 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.208+0000 7f1347fff640 1 -- 172.21.15.16:0/1720097857 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f13500ced60 con 0x7f13600ffe10 2024-09-12T02:16:38.217 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.208+0000 7f135f7fe640 1 --2- 172.21.15.16:0/1342010748 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d670cd9a0 0x559d670cdda0 secure :-1 s=READY pgs=1977 cs=0 l=1 rev1=1 crypto rx=0x7f13580097b0 tx=0x7f13580099c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=389419d8a40a4110 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:38.217 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.208+0000 7f13457fa640 1 -- 172.21.15.16:0/1342010748 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13580317d0 con 0x559d670cd9a0 2024-09-12T02:16:38.217 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.208+0000 7f13457fa640 1 -- 172.21.15.16:0/1342010748 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1358036070 con 0x559d670cd9a0 2024-09-12T02:16:38.217 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.208+0000 7f13457fa640 1 -- 172.21.15.16:0/1342010748 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f135802fcf0 con 0x559d670cd9a0 2024-09-12T02:16:38.217 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.209+0000 7f136824a740 1 -- 172.21.15.16:0/1342010748 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d670cd9a0 msgr2=0x559d670cdda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:38.217 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.209+0000 7f135f7fe640 1 --2- 172.21.15.16:0/1720097857 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f13480a2cb0 0x7f13480a5170 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f135804b000 tx=0x7f1358048000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:38.217 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.209+0000 7f136824a740 1 --2- 172.21.15.16:0/1342010748 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d670cd9a0 0x559d670cdda0 secure :-1 s=READY pgs=1977 cs=0 l=1 rev1=1 crypto rx=0x7f13580097b0 tx=0x7f13580099c0 comp rx=0 tx=0).stop 2024-09-12T02:16:38.217 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.209+0000 7f136824a740 1 -- 172.21.15.16:0/1342010748 shutdown_connections 2024-09-12T02:16:38.218 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.209+0000 7f136824a740 1 --2- 172.21.15.16:0/1342010748 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559d670d3370 0x559d670d5830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:38.218 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.209+0000 7f136824a740 1 --2- 172.21.15.16:0/1342010748 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559d670ce2e0 0x559d670d2da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:38.218 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.209+0000 7f136824a740 1 --2- 172.21.15.16:0/1342010748 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d670cd9a0 0x559d670cdda0 unknown :-1 s=CLOSED pgs=1977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:38.218 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.209+0000 7f136824a740 1 -- 172.21.15.16:0/1342010748 >> 172.21.15.16:0/1342010748 conn(0x559d670c9010 msgr2=0x559d670c9410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:38.218 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.209+0000 7f136824a740 1 -- 172.21.15.16:0/1342010748 shutdown_connections 2024-09-12T02:16:38.218 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.209+0000 7f136824a740 1 -- 172.21.15.16:0/1342010748 wait complete. 2024-09-12T02:16:38.218 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.209+0000 7f136824a740 1 -- start start 2024-09-12T02:16:38.218 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.209+0000 7f136824a740 1 Processor -- start 2024-09-12T02:16:38.218 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.209+0000 7f136824a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d670cd9a0 0x559d670d7a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:38.218 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.209+0000 7f136824a740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559d670ce2e0 0x559d670d7f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:38.218 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.209+0000 7f136824a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559d670d3370 0x559d670defa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:38.218 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.209+0000 7f136824a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x559d670d61c0 con 0x559d670cd9a0 2024-09-12T02:16:38.218 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.209+0000 7f136824a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x559d66fba290 con 0x559d670d3370 2024-09-12T02:16:38.219 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.209+0000 7f136824a740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x559d670d6340 con 0x559d670ce2e0 2024-09-12T02:16:38.219 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.209+0000 7f136824a740 10 client.0 _ll_register_callbacks cb 0x559d66feb140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:16:38.219 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.209+0000 7f135effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d670cd9a0 0x559d670d7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:38.219 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.210+0000 7f135effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d670cd9a0 0x559d670d7a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53812/0 (socket says 172.21.15.16:53812) 2024-09-12T02:16:38.219 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.210+0000 7f135effd640 1 -- 172.21.15.16:0/783663120 learned_addr learned my addr 172.21.15.16:0/783663120 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:16:38.219 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.210+0000 7f135e7fc640 1 --2- 172.21.15.16:0/783663120 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559d670ce2e0 0x559d670d7f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:38.219 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.210+0000 7f135f7fe640 1 --2- 172.21.15.16:0/783663120 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559d670d3370 0x559d670defa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:38.219 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.210+0000 7f135effd640 1 -- 172.21.15.16:0/783663120 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559d670ce2e0 msgr2=0x559d670d7f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:38.219 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.210+0000 7f135effd640 1 --2- 172.21.15.16:0/783663120 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559d670ce2e0 0x559d670d7f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:38.219 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.210+0000 7f135effd640 1 -- 172.21.15.16:0/783663120 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559d670d3370 msgr2=0x559d670defa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:38.219 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.210+0000 7f135effd640 1 --2- 172.21.15.16:0/783663120 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559d670d3370 0x559d670defa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:38.219 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.210+0000 7f135effd640 1 -- 172.21.15.16:0/783663120 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d66d44210 con 0x559d670cd9a0 2024-09-12T02:16:38.219 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.210+0000 7f135effd640 1 --2- 172.21.15.16:0/783663120 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d670cd9a0 0x559d670d7a20 secure :-1 s=READY pgs=1978 cs=0 l=1 rev1=1 crypto rx=0x7f1350007590 tx=0x7f135009bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a752a07fa6297e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:38.219 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.210+0000 7f1342ff5640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:16:38.220 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.210+0000 7f1342ff5640 1 -- 172.21.15.16:0/783663120 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f135001d040 con 0x559d670cd9a0 2024-09-12T02:16:38.220 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.210+0000 7f1342ff5640 1 -- 172.21.15.16:0/783663120 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f1350003610 con 0x559d670cd9a0 2024-09-12T02:16:38.220 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.210+0000 7f136824a740 1 -- 172.21.15.16:0/783663120 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1358036070 con 0x559d670cd9a0 2024-09-12T02:16:38.220 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.210+0000 7f1342ff5640 1 -- 172.21.15.16:0/783663120 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f135009fa10 con 0x559d670cd9a0 2024-09-12T02:16:38.220 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.210+0000 7f136824a740 10 client.20370 fetch_fsmap learned FSMap version 25 2024-09-12T02:16:38.220 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.210+0000 7f136824a740 1 -- 172.21.15.16:0/783663120 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x559d66e10620 con 0x559d670cd9a0 2024-09-12T02:16:38.220 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.211+0000 7f1342ff5640 1 -- 172.21.15.16:0/783663120 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1358036070 con 0x559d670cd9a0 2024-09-12T02:16:38.220 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.211+0000 7f1342ff5640 1 -- 172.21.15.16:0/783663120 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f135009ac40 con 0x559d670cd9a0 2024-09-12T02:16:38.220 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.211+0000 7f136824a740 10 client.20370 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:16:38.220 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.211+0000 7f136824a740 10 client.20370 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:16:38.220 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.211+0000 7f136824a740 20 client.20370 populate_metadata read hostname 'smithi016' 2024-09-12T02:16:38.220 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.211+0000 7f136824a740 1 --2- 172.21.15.16:0/783663120 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x559d670e3610 0x559d670e5a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:38.220 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.211+0000 7f136824a740 4 client.20370 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:16:38.221 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.211+0000 7f136824a740 1 -- 172.21.15.16:0/783663120 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559d6703ea20 con 0x559d670e3610 2024-09-12T02:16:38.221 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.211+0000 7f135e7fc640 1 --2- 172.21.15.16:0/783663120 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x559d670e3610 0x559d670e5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:38.221 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.211+0000 7f1342ff5640 10 client.20370 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:16:38.221 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.211+0000 7f135e7fc640 1 --2- 172.21.15.16:0/783663120 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x559d670e3610 0x559d670e5a40 crc :-1 s=READY pgs=437 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=12abf743653d58d1 server_cookie=2990a12306442b0d in_seq=0 out_seq=0 2024-09-12T02:16:38.221 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.212+0000 7f1342ff5640 1 -- 172.21.15.16:0/783663120 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559d6703ea20 con 0x559d670e3610 2024-09-12T02:16:38.221 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.212+0000 7f1342ff5640 10 client.20370 handle_command_reply: tid=0 2024-09-12T02:16:38.221 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.212+0000 7f136824a740 1 client.20370 shutdown 2024-09-12T02:16:38.221 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.212+0000 7f136824a740 1 -- 172.21.15.16:0/783663120 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d670cd9a0 msgr2=0x559d670d7a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:38.221 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.212+0000 7f136824a740 1 --2- 172.21.15.16:0/783663120 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d670cd9a0 0x559d670d7a20 secure :-1 s=READY pgs=1978 cs=0 l=1 rev1=1 crypto rx=0x7f1350007590 tx=0x7f135009bd60 comp rx=0 tx=0).stop 2024-09-12T02:16:38.221 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.212+0000 7f136824a740 1 -- 172.21.15.16:0/783663120 shutdown_connections 2024-09-12T02:16:38.221 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.212+0000 7f136824a740 1 --2- 172.21.15.16:0/783663120 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x559d670e3610 0x559d670e5a40 crc :-1 s=READY pgs=437 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:38.221 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.212+0000 7f136824a740 1 --2- 172.21.15.16:0/783663120 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559d670d3370 0x559d670defa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:38.221 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.212+0000 7f1342ff5640 0 client.20370 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:16:38.222 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.212+0000 7f136824a740 1 --2- 172.21.15.16:0/783663120 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559d670ce2e0 0x559d670d7f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:38.222 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.212+0000 7f136824a740 1 --2- 172.21.15.16:0/783663120 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d670cd9a0 0x559d670d7a20 unknown :-1 s=CLOSED pgs=1978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:38.222 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.212+0000 7f136824a740 1 -- 172.21.15.16:0/783663120 >> 172.21.15.16:0/783663120 conn(0x559d670c9010 msgr2=0x559d670ca820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:38.222 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.212+0000 7f136824a740 1 -- 172.21.15.16:0/783663120 shutdown_connections 2024-09-12T02:16:38.222 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.212+0000 7f136824a740 1 -- 172.21.15.16:0/783663120 wait complete. 2024-09-12T02:16:38.222 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.213+0000 7f136824a740 20 client.20370 trim_cache size 0 max 16384 2024-09-12T02:16:38.232 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.227+0000 7f136824a740 1 Processor -- start 2024-09-12T02:16:38.232 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.227+0000 7f136824a740 1 -- start start 2024-09-12T02:16:38.232 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.227+0000 7f136824a740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559d670cd9a0 0x559d670d7ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:38.232 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.227+0000 7f136824a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d670ce2e0 0x559d670def30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:38.232 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.227+0000 7f136824a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559d670d3370 0x559d670c9ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:38.232 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.227+0000 7f136824a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f13601ada00 con 0x559d670ce2e0 2024-09-12T02:16:38.233 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.227+0000 7f136824a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f13601ab2d0 con 0x559d670d3370 2024-09-12T02:16:38.233 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.227+0000 7f136824a740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f13601a9120 con 0x559d670cd9a0 2024-09-12T02:16:38.233 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.227+0000 7f135e7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d670ce2e0 0x559d670def30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:38.233 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.227+0000 7f135e7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d670ce2e0 0x559d670def30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53820/0 (socket says 172.21.15.16:53820) 2024-09-12T02:16:38.233 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.227+0000 7f135e7fc640 1 -- 172.21.15.16:0/1650887390 learned_addr learned my addr 172.21.15.16:0/1650887390 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:16:38.233 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.227+0000 7f135f7fe640 1 --2- 172.21.15.16:0/1650887390 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559d670d3370 0x559d670c9ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:38.233 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.227+0000 7f135effd640 1 --2- 172.21.15.16:0/1650887390 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559d670cd9a0 0x559d670d7ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:38.233 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.227+0000 7f135e7fc640 1 -- 172.21.15.16:0/1650887390 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559d670cd9a0 msgr2=0x559d670d7ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:38.233 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.227+0000 7f135e7fc640 1 --2- 172.21.15.16:0/1650887390 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559d670cd9a0 0x559d670d7ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:38.233 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.227+0000 7f135e7fc640 1 -- 172.21.15.16:0/1650887390 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559d670d3370 msgr2=0x559d670c9ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:38.233 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.227+0000 7f135e7fc640 1 --2- 172.21.15.16:0/1650887390 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559d670d3370 0x559d670c9ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:38.233 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.228+0000 7f135e7fc640 1 -- 172.21.15.16:0/1650887390 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d670e60c0 con 0x559d670ce2e0 2024-09-12T02:16:38.234 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.228+0000 7f135f7fe640 1 --2- 172.21.15.16:0/1650887390 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559d670d3370 0x559d670c9ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:16:38.234 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.228+0000 7f135effd640 1 --2- 172.21.15.16:0/1650887390 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559d670cd9a0 0x559d670d7ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:16:38.234 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.228+0000 7f135e7fc640 1 --2- 172.21.15.16:0/1650887390 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d670ce2e0 0x559d670def30 secure :-1 s=READY pgs=1979 cs=0 l=1 rev1=1 crypto rx=0x7f1354000c00 tx=0x7f1354008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66d1acb403fdb9be server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:38.234 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.228+0000 7f1344ff9640 1 -- 172.21.15.16:0/1650887390 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1354015930 con 0x559d670ce2e0 2024-09-12T02:16:38.234 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.228+0000 7f1344ff9640 1 -- 172.21.15.16:0/1650887390 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f135401f030 con 0x559d670ce2e0 2024-09-12T02:16:38.234 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.228+0000 7f1344ff9640 1 -- 172.21.15.16:0/1650887390 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1354023ce0 con 0x559d670ce2e0 2024-09-12T02:16:38.234 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.228+0000 7f136824a740 1 -- 172.21.15.16:0/1650887390 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d670ce2e0 msgr2=0x559d670def30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:38.234 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.228+0000 7f136824a740 1 --2- 172.21.15.16:0/1650887390 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d670ce2e0 0x559d670def30 secure :-1 s=READY pgs=1979 cs=0 l=1 rev1=1 crypto rx=0x7f1354000c00 tx=0x7f1354008040 comp rx=0 tx=0).stop 2024-09-12T02:16:38.234 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.228+0000 7f136824a740 1 -- 172.21.15.16:0/1650887390 shutdown_connections 2024-09-12T02:16:38.234 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.228+0000 7f136824a740 1 --2- 172.21.15.16:0/1650887390 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559d670d3370 0x559d670c9ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:38.234 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.229+0000 7f136824a740 1 --2- 172.21.15.16:0/1650887390 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d670ce2e0 0x559d670def30 unknown :-1 s=CLOSED pgs=1979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:38.235 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.229+0000 7f136824a740 1 --2- 172.21.15.16:0/1650887390 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559d670cd9a0 0x559d670d7ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:38.235 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.229+0000 7f136824a740 1 -- 172.21.15.16:0/1650887390 >> 172.21.15.16:0/1650887390 conn(0x559d670c9010 msgr2=0x559d670d77d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:38.235 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.229+0000 7f136824a740 1 -- 172.21.15.16:0/1650887390 shutdown_connections 2024-09-12T02:16:38.235 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.229+0000 7f136824a740 1 -- 172.21.15.16:0/1650887390 wait complete. 2024-09-12T02:16:38.235 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.229+0000 7f136824a740 1 -- start start 2024-09-12T02:16:38.235 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.229+0000 7f136824a740 1 Processor -- start 2024-09-12T02:16:38.235 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.229+0000 7f136824a740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559d670cd9a0 0x559d670dc870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:38.235 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.229+0000 7f136824a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d670ce2e0 0x559d670dcdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:38.235 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.229+0000 7f136824a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559d670d3370 0x559d670dddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:38.235 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.229+0000 7f136824a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f13601a9790 con 0x559d670ce2e0 2024-09-12T02:16:38.235 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.229+0000 7f136824a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f13601aac00 con 0x559d670d3370 2024-09-12T02:16:38.236 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.229+0000 7f136824a740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f13601a9e60 con 0x559d670cd9a0 2024-09-12T02:16:38.236 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.229+0000 7f136824a740 10 client.0 _ll_register_callbacks cb 0x7f135009b1a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:16:38.236 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.229+0000 7f135e7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d670ce2e0 0x559d670dcdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:38.236 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.229+0000 7f135e7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d670ce2e0 0x559d670dcdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53834/0 (socket says 172.21.15.16:53834) 2024-09-12T02:16:38.236 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.229+0000 7f135e7fc640 1 -- 172.21.15.16:0/1444428112 learned_addr learned my addr 172.21.15.16:0/1444428112 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:16:38.236 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.229+0000 7f135effd640 1 --2- 172.21.15.16:0/1444428112 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559d670cd9a0 0x559d670dc870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:38.236 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.230+0000 7f135f7fe640 1 --2- 172.21.15.16:0/1444428112 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559d670d3370 0x559d670dddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:38.236 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.230+0000 7f135e7fc640 1 -- 172.21.15.16:0/1444428112 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559d670cd9a0 msgr2=0x559d670dc870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:38.236 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.230+0000 7f135e7fc640 1 --2- 172.21.15.16:0/1444428112 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559d670cd9a0 0x559d670dc870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:38.237 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.230+0000 7f135e7fc640 1 -- 172.21.15.16:0/1444428112 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559d670d3370 msgr2=0x559d670dddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:38.237 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.230+0000 7f135e7fc640 1 --2- 172.21.15.16:0/1444428112 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559d670d3370 0x559d670dddc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:38.237 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.230+0000 7f135e7fc640 1 -- 172.21.15.16:0/1444428112 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d670de3d0 con 0x559d670ce2e0 2024-09-12T02:16:38.237 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.230+0000 7f135e7fc640 1 --2- 172.21.15.16:0/1444428112 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d670ce2e0 0x559d670dcdb0 secure :-1 s=READY pgs=1980 cs=0 l=1 rev1=1 crypto rx=0x7f1354006a70 tx=0x7f1354015930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79acb6ea8d88a456 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:38.237 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.230+0000 7f1342ff5640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:16:38.237 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.230+0000 7f1342ff5640 1 -- 172.21.15.16:0/1444428112 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f135402a070 con 0x559d670ce2e0 2024-09-12T02:16:38.237 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.230+0000 7f1342ff5640 1 -- 172.21.15.16:0/1444428112 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f13540233d0 con 0x559d670ce2e0 2024-09-12T02:16:38.237 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.230+0000 7f136824a740 1 -- 172.21.15.16:0/1444428112 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f135401f030 con 0x559d670ce2e0 2024-09-12T02:16:38.237 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.230+0000 7f1342ff5640 1 -- 172.21.15.16:0/1444428112 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f13540235d0 con 0x559d670ce2e0 2024-09-12T02:16:38.237 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.230+0000 7f136824a740 10 client.20376 fetch_fsmap learned FSMap version 25 2024-09-12T02:16:38.237 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.230+0000 7f136824a740 1 -- 172.21.15.16:0/1444428112 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x559d6714e470 con 0x559d670ce2e0 2024-09-12T02:16:38.238 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.230+0000 7f1342ff5640 1 -- 172.21.15.16:0/1444428112 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f135401f030 con 0x559d670ce2e0 2024-09-12T02:16:38.238 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.230+0000 7f1342ff5640 1 -- 172.21.15.16:0/1444428112 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f135402d360 con 0x559d670ce2e0 2024-09-12T02:16:38.238 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.230+0000 7f136824a740 10 client.20376 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:16:38.238 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.230+0000 7f136824a740 10 client.20376 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:16:38.238 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.231+0000 7f136824a740 20 client.20376 populate_metadata read hostname 'smithi016' 2024-09-12T02:16:38.238 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.231+0000 7f136824a740 1 --2- 172.21.15.16:0/1444428112 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x559d6714edf0 0x559d67151210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:38.238 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.231+0000 7f136824a740 4 client.20376 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:16:38.238 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.231+0000 7f136824a740 1 -- 172.21.15.16:0/1444428112 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559d67038d70 con 0x559d6714edf0 2024-09-12T02:16:38.238 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.231+0000 7f135effd640 1 --2- 172.21.15.16:0/1444428112 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x559d6714edf0 0x559d67151210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:38.238 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.231+0000 7f135effd640 1 --2- 172.21.15.16:0/1444428112 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x559d6714edf0 0x559d67151210 crc :-1 s=READY pgs=438 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d602178eeeac6858 server_cookie=b800e540a59551b9 in_seq=0 out_seq=0 2024-09-12T02:16:38.239 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.231+0000 7f1342ff5640 10 client.20376 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:16:38.246 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.245+0000 7f1342ff5640 1 -- 172.21.15.16:0/1444428112 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559d67038d70 con 0x559d6714edf0 2024-09-12T02:16:38.246 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.245+0000 7f1342ff5640 10 client.20376 handle_command_reply: tid=0 2024-09-12T02:16:38.246 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.245+0000 7f136824a740 1 client.20376 shutdown 2024-09-12T02:16:38.246 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.245+0000 7f136824a740 1 -- 172.21.15.16:0/1444428112 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d670ce2e0 msgr2=0x559d670dcdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:38.246 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.245+0000 7f136824a740 1 --2- 172.21.15.16:0/1444428112 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d670ce2e0 0x559d670dcdb0 secure :-1 s=READY pgs=1980 cs=0 l=1 rev1=1 crypto rx=0x7f1354006a70 tx=0x7f1354015930 comp rx=0 tx=0).stop 2024-09-12T02:16:38.246 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.245+0000 7f136824a740 1 -- 172.21.15.16:0/1444428112 shutdown_connections 2024-09-12T02:16:38.246 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.245+0000 7f136824a740 1 --2- 172.21.15.16:0/1444428112 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x559d6714edf0 0x559d67151210 crc :-1 s=READY pgs=438 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:38.246 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.245+0000 7f136824a740 1 --2- 172.21.15.16:0/1444428112 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559d670d3370 0x559d670dddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:38.246 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.245+0000 7f136824a740 1 --2- 172.21.15.16:0/1444428112 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d670ce2e0 0x559d670dcdb0 unknown :-1 s=CLOSED pgs=1980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:38.246 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.245+0000 7f1342ff5640 0 client.20376 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:16:38.247 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.245+0000 7f136824a740 1 --2- 172.21.15.16:0/1444428112 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559d670cd9a0 0x559d670dc870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:38.247 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.245+0000 7f136824a740 1 -- 172.21.15.16:0/1444428112 >> 172.21.15.16:0/1444428112 conn(0x559d670c9010 msgr2=0x559d670c79a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:38.247 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.245+0000 7f136824a740 1 -- 172.21.15.16:0/1444428112 shutdown_connections 2024-09-12T02:16:38.247 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.245+0000 7f136824a740 1 -- 172.21.15.16:0/1444428112 wait complete. 2024-09-12T02:16:38.247 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.246+0000 7f136824a740 20 client.20376 trim_cache size 0 max 16384 2024-09-12T02:16:38.247 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-09-12T02:16:38.247 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-09-12T02:16:38.247 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-09-12T02:16:38.247 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-09-12T02:16:38.248 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.246+0000 7f13653d6640 1 -- 172.21.15.16:0/1720097857 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f13480a2cb0 msgr2=0x7f13480a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:38.248 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.246+0000 7f13653d6640 1 --2- 172.21.15.16:0/1720097857 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f13480a2cb0 0x7f13480a5170 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f135804b000 tx=0x7f1358048000 comp rx=0 tx=0).stop 2024-09-12T02:16:38.248 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.246+0000 7f13653d6640 1 -- 172.21.15.16:0/1720097857 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13600ffe10 msgr2=0x7f1360102750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:38.248 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.246+0000 7f13653d6640 1 --2- 172.21.15.16:0/1720097857 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13600ffe10 0x7f1360102750 secure :-1 s=READY pgs=1976 cs=0 l=1 rev1=1 crypto rx=0x7f135000c970 tx=0x7f135000ce30 comp rx=0 tx=0).stop 2024-09-12T02:16:38.249 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.246+0000 7f13653d6640 1 -- 172.21.15.16:0/1720097857 shutdown_connections 2024-09-12T02:16:38.249 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.246+0000 7f13653d6640 1 --2- 172.21.15.16:0/1720097857 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f13480a2cb0 0x7f13480a5170 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:38.249 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.246+0000 7f13653d6640 1 --2- 172.21.15.16:0/1720097857 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f13601037a0 0x7f13601012d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:38.249 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.246+0000 7f13653d6640 1 --2- 172.21.15.16:0/1720097857 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1360102de0 0x7f1360100d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:38.249 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.246+0000 7f13653d6640 1 --2- 172.21.15.16:0/1720097857 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13600ffe10 0x7f1360102750 unknown :-1 s=CLOSED pgs=1976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:38.249 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.246+0000 7f13653d6640 1 -- 172.21.15.16:0/1720097857 >> 172.21.15.16:0/1720097857 conn(0x7f13600fb930 msgr2=0x7f13600fd960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:38.249 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.246+0000 7f13653d6640 1 -- 172.21.15.16:0/1720097857 shutdown_connections 2024-09-12T02:16:38.249 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:38.246+0000 7f13653d6640 1 -- 172.21.15.16:0/1720097857 wait complete. 2024-09-12T02:16:38.664 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:16:38 smithi016 ceph-mon[30947]: pgmap v776: 97 pgs: 97 active+clean; 3.5 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 17 op/s 2024-09-12T02:16:38.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:16:38 smithi033 ceph-mon[35332]: pgmap v776: 97 pgs: 97 active+clean; 3.5 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 17 op/s 2024-09-12T02:16:38.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:16:38 smithi184 ceph-mon[35593]: pgmap v776: 97 pgs: 97 active+clean; 3.5 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 17 op/s 2024-09-12T02:16:39.693 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-12T02:16:39.693 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-12T02:16:39.965 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:16:40.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.361+0000 7fe260f24640 1 -- 172.21.15.16:0/1830113199 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe25c101420 msgr2=0x7fe25c10d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:40.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.361+0000 7fe260f24640 1 --2- 172.21.15.16:0/1830113199 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe25c101420 0x7fe25c10d950 secure :-1 s=READY pgs=1981 cs=0 l=1 rev1=1 crypto rx=0x7fe254009cd0 tx=0x7fe25402a8e0 comp rx=0 tx=0).stop 2024-09-12T02:16:40.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.361+0000 7fe260f24640 1 -- 172.21.15.16:0/1830113199 shutdown_connections 2024-09-12T02:16:40.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.361+0000 7fe260f24640 1 --2- 172.21.15.16:0/1830113199 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe25c101420 0x7fe25c10d950 unknown :-1 s=CLOSED pgs=1981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:40.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.361+0000 7fe260f24640 1 --2- 172.21.15.16:0/1830113199 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe25c100a80 0x7fe25c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:40.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.361+0000 7fe260f24640 1 --2- 172.21.15.16:0/1830113199 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe25c106a80 0x7fe25c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:40.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.361+0000 7fe260f24640 1 -- 172.21.15.16:0/1830113199 >> 172.21.15.16:0/1830113199 conn(0x7fe25c0fc7a0 msgr2=0x7fe25c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:40.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.361+0000 7fe260f24640 1 -- 172.21.15.16:0/1830113199 shutdown_connections 2024-09-12T02:16:40.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.361+0000 7fe260f24640 1 -- 172.21.15.16:0/1830113199 wait complete. 2024-09-12T02:16:40.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.362+0000 7fe260f24640 1 Processor -- start 2024-09-12T02:16:40.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.362+0000 7fe260f24640 1 -- start start 2024-09-12T02:16:40.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.362+0000 7fe260f24640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe25c100a80 0x7fe25c198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:40.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.362+0000 7fe260f24640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe25c101420 0x7fe25c1988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:40.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.362+0000 7fe260f24640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe25c106a80 0x7fe25c19cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:40.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.362+0000 7fe260f24640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe25c111fe0 con 0x7fe25c106a80 2024-09-12T02:16:40.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.362+0000 7fe260f24640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe25c111e60 con 0x7fe25c101420 2024-09-12T02:16:40.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.362+0000 7fe260f24640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe25c112160 con 0x7fe25c100a80 2024-09-12T02:16:40.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.362+0000 7fe25ad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe25c106a80 0x7fe25c19cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:40.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.362+0000 7fe25ad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe25c106a80 0x7fe25c19cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53864/0 (socket says 172.21.15.16:53864) 2024-09-12T02:16:40.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.362+0000 7fe25ad9d640 1 -- 172.21.15.16:0/3702339494 learned_addr learned my addr 172.21.15.16:0/3702339494 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:16:40.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.363+0000 7fe259d9b640 1 --2- 172.21.15.16:0/3702339494 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe25c101420 0x7fe25c1988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:40.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.363+0000 7fe25a59c640 1 --2- 172.21.15.16:0/3702339494 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe25c100a80 0x7fe25c198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:40.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.363+0000 7fe25ad9d640 1 -- 172.21.15.16:0/3702339494 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe25c100a80 msgr2=0x7fe25c198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:40.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.363+0000 7fe25ad9d640 1 --2- 172.21.15.16:0/3702339494 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe25c100a80 0x7fe25c198360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:40.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.363+0000 7fe25ad9d640 1 -- 172.21.15.16:0/3702339494 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe25c101420 msgr2=0x7fe25c1988a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:40.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.363+0000 7fe25ad9d640 1 --2- 172.21.15.16:0/3702339494 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe25c101420 0x7fe25c1988a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:40.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.363+0000 7fe25ad9d640 1 -- 172.21.15.16:0/3702339494 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe25c19d310 con 0x7fe25c106a80 2024-09-12T02:16:40.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.363+0000 7fe25ad9d640 1 --2- 172.21.15.16:0/3702339494 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe25c106a80 0x7fe25c19cc30 secure :-1 s=READY pgs=1982 cs=0 l=1 rev1=1 crypto rx=0x7fe254033040 tx=0x7fe254003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:40.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.363+0000 7fe24b7fe640 1 -- 172.21.15.16:0/3702339494 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe254004040 con 0x7fe25c106a80 2024-09-12T02:16:40.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.363+0000 7fe24b7fe640 1 -- 172.21.15.16:0/3702339494 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe2540046c0 con 0x7fe25c106a80 2024-09-12T02:16:40.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.363+0000 7fe24b7fe640 1 -- 172.21.15.16:0/3702339494 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe254030eb0 con 0x7fe25c106a80 2024-09-12T02:16:40.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.363+0000 7fe260f24640 1 -- 172.21.15.16:0/3702339494 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe25c19d5a0 con 0x7fe25c106a80 2024-09-12T02:16:40.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.363+0000 7fe260f24640 1 -- 172.21.15.16:0/3702339494 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe25c1a4ee0 con 0x7fe25c106a80 2024-09-12T02:16:40.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.366+0000 7fe24b7fe640 1 -- 172.21.15.16:0/3702339494 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fe2540041e0 con 0x7fe25c106a80 2024-09-12T02:16:40.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.366+0000 7fe2632fd740 1 Processor -- start 2024-09-12T02:16:40.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.366+0000 7fe2632fd740 1 -- start start 2024-09-12T02:16:40.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.366+0000 7fe2632fd740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c8a6a8b60 0x563c8a6a8f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:40.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.366+0000 7fe2632fd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c8a6a9480 0x563c8a6b3800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:40.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.366+0000 7fe2632fd740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563c8a6b3d40 0x563c8a6b6130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:40.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.366+0000 7fe2632fd740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x563c8a60e280 con 0x563c8a6a8b60 2024-09-12T02:16:40.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.366+0000 7fe2632fd740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x563c8a5b5b10 con 0x563c8a6a9480 2024-09-12T02:16:40.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.366+0000 7fe2632fd740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x563c8a60a240 con 0x563c8a6b3d40 2024-09-12T02:16:40.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.366+0000 7fe24b7fe640 1 --2- 172.21.15.16:0/3702339494 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fe2440a2b80 0x7fe2440a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:40.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.366+0000 7fe259d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c8a6a8b60 0x563c8a6a8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:40.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.366+0000 7fe259d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c8a6a8b60 0x563c8a6a8f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53876/0 (socket says 172.21.15.16:53876) 2024-09-12T02:16:40.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.367+0000 7fe259d9b640 1 -- 172.21.15.16:0/2873327987 learned_addr learned my addr 172.21.15.16:0/2873327987 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:16:40.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.367+0000 7fe25ad9d640 1 --2- 172.21.15.16:0/3702339494 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fe2440a2b80 0x7fe2440a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:40.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.367+0000 7fe25a59c640 1 --2- 172.21.15.16:0/2873327987 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c8a6a9480 0x563c8a6b3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:40.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.367+0000 7fe259d9b640 1 --2- 172.21.15.16:0/2873327987 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563c8a6b3d40 0x563c8a6b6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:40.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.367+0000 7fe259d9b640 1 -- 172.21.15.16:0/2873327987 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563c8a6b3d40 msgr2=0x563c8a6b6130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:40.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.367+0000 7fe259d9b640 1 --2- 172.21.15.16:0/2873327987 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563c8a6b3d40 0x563c8a6b6130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:40.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.367+0000 7fe259d9b640 1 -- 172.21.15.16:0/2873327987 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c8a6a9480 msgr2=0x563c8a6b3800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:40.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.367+0000 7fe259d9b640 1 --2- 172.21.15.16:0/2873327987 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c8a6a9480 0x563c8a6b3800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:40.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.367+0000 7fe259d9b640 1 -- 172.21.15.16:0/2873327987 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c8a5c21f0 con 0x563c8a6a8b60 2024-09-12T02:16:40.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.367+0000 7fe24b7fe640 1 -- 172.21.15.16:0/3702339494 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fe2540f0140 con 0x7fe25c106a80 2024-09-12T02:16:40.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.367+0000 7fe259d9b640 1 --2- 172.21.15.16:0/2873327987 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c8a6a8b60 0x563c8a6a8f40 secure :-1 s=READY pgs=1983 cs=0 l=1 rev1=1 crypto rx=0x7fe250013560 tx=0x7fe25000a800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5879aa0e5531f50b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:40.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.367+0000 7fe25ad9d640 1 --2- 172.21.15.16:0/3702339494 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fe2440a2b80 0x7fe2440a5040 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fe2540b8720 tx=0x7fe2540b74a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:40.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.367+0000 7fe248ff9640 1 -- 172.21.15.16:0/2873327987 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe250019030 con 0x563c8a6a8b60 2024-09-12T02:16:40.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.367+0000 7fe248ff9640 1 -- 172.21.15.16:0/2873327987 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe2500110d0 con 0x563c8a6a8b60 2024-09-12T02:16:40.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.367+0000 7fe248ff9640 1 -- 172.21.15.16:0/2873327987 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe25000f8e0 con 0x563c8a6a8b60 2024-09-12T02:16:40.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.368+0000 7fe2632fd740 1 -- 172.21.15.16:0/2873327987 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c8a6a8b60 msgr2=0x563c8a6a8f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:40.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.368+0000 7fe2632fd740 1 --2- 172.21.15.16:0/2873327987 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c8a6a8b60 0x563c8a6a8f40 secure :-1 s=READY pgs=1983 cs=0 l=1 rev1=1 crypto rx=0x7fe250013560 tx=0x7fe25000a800 comp rx=0 tx=0).stop 2024-09-12T02:16:40.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.368+0000 7fe2632fd740 1 -- 172.21.15.16:0/2873327987 shutdown_connections 2024-09-12T02:16:40.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.368+0000 7fe2632fd740 1 --2- 172.21.15.16:0/2873327987 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563c8a6b3d40 0x563c8a6b6130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:40.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.368+0000 7fe2632fd740 1 --2- 172.21.15.16:0/2873327987 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c8a6a9480 0x563c8a6b3800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:40.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.368+0000 7fe2632fd740 1 --2- 172.21.15.16:0/2873327987 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c8a6a8b60 0x563c8a6a8f40 unknown :-1 s=CLOSED pgs=1983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:40.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.368+0000 7fe2632fd740 1 -- 172.21.15.16:0/2873327987 >> 172.21.15.16:0/2873327987 conn(0x563c8a6a50f0 msgr2=0x563c8a6a54d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:40.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.368+0000 7fe2632fd740 1 -- 172.21.15.16:0/2873327987 shutdown_connections 2024-09-12T02:16:40.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.368+0000 7fe2632fd740 1 -- 172.21.15.16:0/2873327987 wait complete. 2024-09-12T02:16:40.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.369+0000 7fe2632fd740 1 -- start start 2024-09-12T02:16:40.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.369+0000 7fe2632fd740 1 Processor -- start 2024-09-12T02:16:40.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.369+0000 7fe2632fd740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c8a6a8b60 0x563c8a6b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:40.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.369+0000 7fe2632fd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c8a6a9480 0x563c8a6b1100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:40.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.369+0000 7fe2632fd740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563c8a6b3d40 0x563c8a6b1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:40.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.369+0000 7fe2632fd740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x563c8a6b6910 con 0x563c8a6a8b60 2024-09-12T02:16:40.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.369+0000 7fe2632fd740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x563c8a5c8f10 con 0x563c8a6a9480 2024-09-12T02:16:40.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.369+0000 7fe2632fd740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x563c8a6b6a90 con 0x563c8a6b3d40 2024-09-12T02:16:40.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.369+0000 7fe2632fd740 10 client.0 _ll_register_callbacks cb 0x563c8a591a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:16:40.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.369+0000 7fe259d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c8a6a8b60 0x563c8a6b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:40.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.369+0000 7fe259d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c8a6a8b60 0x563c8a6b0bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53882/0 (socket says 172.21.15.16:53882) 2024-09-12T02:16:40.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.369+0000 7fe259d9b640 1 -- 172.21.15.16:0/262691543 learned_addr learned my addr 172.21.15.16:0/262691543 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:16:40.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.369+0000 7fe25a59c640 1 --2- 172.21.15.16:0/262691543 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c8a6a9480 0x563c8a6b1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:40.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.370+0000 7fe259d9b640 1 --2- 172.21.15.16:0/262691543 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563c8a6b3d40 0x563c8a6b1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:40.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.370+0000 7fe259d9b640 1 -- 172.21.15.16:0/262691543 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563c8a6b3d40 msgr2=0x563c8a6b1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:40.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.370+0000 7fe259d9b640 1 --2- 172.21.15.16:0/262691543 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563c8a6b3d40 0x563c8a6b1640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:40.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.370+0000 7fe259d9b640 1 -- 172.21.15.16:0/262691543 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c8a6a9480 msgr2=0x563c8a6b1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:40.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.370+0000 7fe259d9b640 1 --2- 172.21.15.16:0/262691543 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c8a6a9480 0x563c8a6b1100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:40.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.370+0000 7fe259d9b640 1 -- 172.21.15.16:0/262691543 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c8a37f6a0 con 0x563c8a6a8b60 2024-09-12T02:16:40.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.370+0000 7fe259d9b640 1 --2- 172.21.15.16:0/262691543 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c8a6a8b60 0x563c8a6b0bc0 secure :-1 s=READY pgs=1984 cs=0 l=1 rev1=1 crypto rx=0x7fe250011330 tx=0x7fe250011280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17a144045a6c50b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:40.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.370+0000 7fe241ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:16:40.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.370+0000 7fe241ffb640 1 -- 172.21.15.16:0/262691543 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe250019040 con 0x563c8a6a8b60 2024-09-12T02:16:40.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.370+0000 7fe2632fd740 1 -- 172.21.15.16:0/262691543 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe2500110d0 con 0x563c8a6a8b60 2024-09-12T02:16:40.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.370+0000 7fe241ffb640 1 -- 172.21.15.16:0/262691543 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fe250005240 con 0x563c8a6a8b60 2024-09-12T02:16:40.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.370+0000 7fe241ffb640 1 -- 172.21.15.16:0/262691543 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe250005440 con 0x563c8a6a8b60 2024-09-12T02:16:40.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.370+0000 7fe2632fd740 10 client.20388 fetch_fsmap learned FSMap version 25 2024-09-12T02:16:40.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.370+0000 7fe2632fd740 1 -- 172.21.15.16:0/262691543 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x563c8a6c24c0 con 0x563c8a6a8b60 2024-09-12T02:16:40.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.370+0000 7fe241ffb640 1 -- 172.21.15.16:0/262691543 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe2500110d0 con 0x563c8a6a8b60 2024-09-12T02:16:40.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.371+0000 7fe241ffb640 1 -- 172.21.15.16:0/262691543 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fe25000f780 con 0x563c8a6a8b60 2024-09-12T02:16:40.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.371+0000 7fe2632fd740 10 client.20388 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:16:40.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.371+0000 7fe2632fd740 10 client.20388 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:16:40.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.371+0000 7fe2632fd740 20 client.20388 populate_metadata read hostname 'smithi016' 2024-09-12T02:16:40.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.371+0000 7fe2632fd740 1 --2- 172.21.15.16:0/262691543 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x563c8a6c3810 0x563c8a6c5c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:40.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.371+0000 7fe2632fd740 4 client.20388 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:16:40.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.371+0000 7fe2632fd740 1 -- 172.21.15.16:0/262691543 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563c8a61b7f0 con 0x563c8a6c3810 2024-09-12T02:16:40.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.371+0000 7fe25a59c640 1 --2- 172.21.15.16:0/262691543 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x563c8a6c3810 0x563c8a6c5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:40.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.371+0000 7fe25a59c640 1 --2- 172.21.15.16:0/262691543 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x563c8a6c3810 0x563c8a6c5c00 crc :-1 s=READY pgs=439 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e050946e134b607 server_cookie=2c95b1f9fd59114c in_seq=0 out_seq=0 2024-09-12T02:16:40.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.371+0000 7fe241ffb640 10 client.20388 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:16:40.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.372+0000 7fe241ffb640 1 -- 172.21.15.16:0/262691543 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563c8a61b7f0 con 0x563c8a6c3810 2024-09-12T02:16:40.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.372+0000 7fe241ffb640 10 client.20388 handle_command_reply: tid=0 2024-09-12T02:16:40.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.372+0000 7fe2632fd740 1 client.20388 shutdown 2024-09-12T02:16:40.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.372+0000 7fe2632fd740 1 -- 172.21.15.16:0/262691543 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c8a6a8b60 msgr2=0x563c8a6b0bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:40.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.372+0000 7fe2632fd740 1 --2- 172.21.15.16:0/262691543 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c8a6a8b60 0x563c8a6b0bc0 secure :-1 s=READY pgs=1984 cs=0 l=1 rev1=1 crypto rx=0x7fe250011330 tx=0x7fe250011280 comp rx=0 tx=0).stop 2024-09-12T02:16:40.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.372+0000 7fe2632fd740 1 -- 172.21.15.16:0/262691543 shutdown_connections 2024-09-12T02:16:40.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.372+0000 7fe2632fd740 1 --2- 172.21.15.16:0/262691543 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x563c8a6c3810 0x563c8a6c5c00 crc :-1 s=READY pgs=439 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:40.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.372+0000 7fe241ffb640 0 client.20388 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:16:40.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.372+0000 7fe2632fd740 1 --2- 172.21.15.16:0/262691543 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563c8a6b3d40 0x563c8a6b1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:40.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.372+0000 7fe2632fd740 1 --2- 172.21.15.16:0/262691543 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c8a6a9480 0x563c8a6b1100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:40.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.372+0000 7fe2632fd740 1 --2- 172.21.15.16:0/262691543 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c8a6a8b60 0x563c8a6b0bc0 unknown :-1 s=CLOSED pgs=1984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:40.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.372+0000 7fe2632fd740 1 -- 172.21.15.16:0/262691543 >> 172.21.15.16:0/262691543 conn(0x563c8a6a50f0 msgr2=0x563c8a6aae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:40.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.372+0000 7fe2632fd740 1 -- 172.21.15.16:0/262691543 shutdown_connections 2024-09-12T02:16:40.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.373+0000 7fe2632fd740 1 -- 172.21.15.16:0/262691543 wait complete. 2024-09-12T02:16:40.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.373+0000 7fe2632fd740 20 client.20388 trim_cache size 0 max 16384 2024-09-12T02:16:40.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.395+0000 7fe2632fd740 1 Processor -- start 2024-09-12T02:16:40.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.395+0000 7fe2632fd740 1 -- start start 2024-09-12T02:16:40.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.395+0000 7fe2632fd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c8a6a8b60 0x563c8a6ab5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:40.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.395+0000 7fe2632fd740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563c8a6a9480 0x563c8a6cb340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:40.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.395+0000 7fe2632fd740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c8a6b3d40 0x563c8a6cb880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:40.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.395+0000 7fe2632fd740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe25c1afc20 con 0x563c8a6b3d40 2024-09-12T02:16:40.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.395+0000 7fe2632fd740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe25c1ad4f0 con 0x563c8a6a8b60 2024-09-12T02:16:40.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.395+0000 7fe2632fd740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe25c1ab340 con 0x563c8a6a9480 2024-09-12T02:16:40.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.395+0000 7fe259d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c8a6b3d40 0x563c8a6cb880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:40.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.395+0000 7fe259d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c8a6b3d40 0x563c8a6cb880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53894/0 (socket says 172.21.15.16:53894) 2024-09-12T02:16:40.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.395+0000 7fe259d9b640 1 -- 172.21.15.16:0/3071700924 learned_addr learned my addr 172.21.15.16:0/3071700924 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:16:40.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.395+0000 7fe259d9b640 1 --2- 172.21.15.16:0/3071700924 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c8a6a8b60 0x563c8a6ab5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:40.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.395+0000 7fe25a59c640 1 --2- 172.21.15.16:0/3071700924 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563c8a6a9480 0x563c8a6cb340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:40.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.395+0000 7fe259d9b640 1 -- 172.21.15.16:0/3071700924 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563c8a6a9480 msgr2=0x563c8a6cb340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:40.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.395+0000 7fe259d9b640 1 --2- 172.21.15.16:0/3071700924 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563c8a6a9480 0x563c8a6cb340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:40.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.395+0000 7fe259d9b640 1 -- 172.21.15.16:0/3071700924 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c8a6a8b60 msgr2=0x563c8a6ab5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:40.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.395+0000 7fe259d9b640 1 --2- 172.21.15.16:0/3071700924 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c8a6a8b60 0x563c8a6ab5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:40.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.395+0000 7fe259d9b640 1 -- 172.21.15.16:0/3071700924 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c8a6a92b0 con 0x563c8a6b3d40 2024-09-12T02:16:40.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.396+0000 7fe259d9b640 1 --2- 172.21.15.16:0/3071700924 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c8a6b3d40 0x563c8a6cb880 secure :-1 s=READY pgs=1985 cs=0 l=1 rev1=1 crypto rx=0x7fe250001eb0 tx=0x7fe25000af50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25062f72de11f7df server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:40.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.396+0000 7fe243fff640 1 -- 172.21.15.16:0/3071700924 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe250019030 con 0x563c8a6b3d40 2024-09-12T02:16:40.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.396+0000 7fe243fff640 1 -- 172.21.15.16:0/3071700924 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe25000f9e0 con 0x563c8a6b3d40 2024-09-12T02:16:40.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.396+0000 7fe243fff640 1 -- 172.21.15.16:0/3071700924 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe25001ea10 con 0x563c8a6b3d40 2024-09-12T02:16:40.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.396+0000 7fe2632fd740 1 -- 172.21.15.16:0/3071700924 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c8a6b3d40 msgr2=0x563c8a6cb880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:40.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.396+0000 7fe2632fd740 1 --2- 172.21.15.16:0/3071700924 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c8a6b3d40 0x563c8a6cb880 secure :-1 s=READY pgs=1985 cs=0 l=1 rev1=1 crypto rx=0x7fe250001eb0 tx=0x7fe25000af50 comp rx=0 tx=0).stop 2024-09-12T02:16:40.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.396+0000 7fe2632fd740 1 -- 172.21.15.16:0/3071700924 shutdown_connections 2024-09-12T02:16:40.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.396+0000 7fe2632fd740 1 --2- 172.21.15.16:0/3071700924 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c8a6b3d40 0x563c8a6cb880 unknown :-1 s=CLOSED pgs=1985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:40.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.396+0000 7fe2632fd740 1 --2- 172.21.15.16:0/3071700924 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563c8a6a9480 0x563c8a6cb340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:40.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.396+0000 7fe2632fd740 1 --2- 172.21.15.16:0/3071700924 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c8a6a8b60 0x563c8a6ab5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:40.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.396+0000 7fe2632fd740 1 -- 172.21.15.16:0/3071700924 >> 172.21.15.16:0/3071700924 conn(0x563c8a6a50f0 msgr2=0x563c8a6b13a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:40.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.396+0000 7fe2632fd740 1 -- 172.21.15.16:0/3071700924 shutdown_connections 2024-09-12T02:16:40.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.396+0000 7fe2632fd740 1 -- 172.21.15.16:0/3071700924 wait complete. 2024-09-12T02:16:40.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.397+0000 7fe2632fd740 1 -- start start 2024-09-12T02:16:40.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.397+0000 7fe2632fd740 1 Processor -- start 2024-09-12T02:16:40.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.397+0000 7fe2632fd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c8a6a8b60 0x563c8a6adc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:40.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.397+0000 7fe2632fd740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563c8a6a9480 0x563c8a6ae150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:40.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.397+0000 7fe2632fd740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c8a6b3d40 0x563c8a6ae690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:40.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.397+0000 7fe2632fd740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe25c1ab9b0 con 0x563c8a6b3d40 2024-09-12T02:16:40.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.397+0000 7fe2632fd740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe25c1ace20 con 0x563c8a6a8b60 2024-09-12T02:16:40.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.397+0000 7fe2632fd740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe25c1ac080 con 0x563c8a6a9480 2024-09-12T02:16:40.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.397+0000 7fe2632fd740 10 client.0 _ll_register_callbacks cb 0x7fe24c004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:16:40.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.397+0000 7fe259d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c8a6b3d40 0x563c8a6ae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:40.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.397+0000 7fe259d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c8a6b3d40 0x563c8a6ae690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53908/0 (socket says 172.21.15.16:53908) 2024-09-12T02:16:40.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.397+0000 7fe259d9b640 1 -- 172.21.15.16:0/689253550 learned_addr learned my addr 172.21.15.16:0/689253550 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:16:40.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.397+0000 7fe259d9b640 1 --2- 172.21.15.16:0/689253550 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c8a6a8b60 0x563c8a6adc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:40.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.397+0000 7fe25a59c640 1 --2- 172.21.15.16:0/689253550 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563c8a6a9480 0x563c8a6ae150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:40.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.398+0000 7fe259d9b640 1 -- 172.21.15.16:0/689253550 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563c8a6a9480 msgr2=0x563c8a6ae150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:40.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.398+0000 7fe259d9b640 1 --2- 172.21.15.16:0/689253550 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563c8a6a9480 0x563c8a6ae150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:40.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.398+0000 7fe259d9b640 1 -- 172.21.15.16:0/689253550 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c8a6a8b60 msgr2=0x563c8a6adc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:40.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.398+0000 7fe259d9b640 1 --2- 172.21.15.16:0/689253550 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c8a6a8b60 0x563c8a6adc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:40.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.398+0000 7fe259d9b640 1 -- 172.21.15.16:0/689253550 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c8a731770 con 0x563c8a6b3d40 2024-09-12T02:16:40.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.398+0000 7fe25a59c640 1 --2- 172.21.15.16:0/689253550 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563c8a6a9480 0x563c8a6ae150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:16:40.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.398+0000 7fe259d9b640 1 --2- 172.21.15.16:0/689253550 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c8a6b3d40 0x563c8a6ae690 secure :-1 s=READY pgs=1986 cs=0 l=1 rev1=1 crypto rx=0x7fe25000a350 tx=0x7fe25001e5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=780dc32c3edbf3c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:40.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.398+0000 7fe241ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:16:40.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.398+0000 7fe241ffb640 1 -- 172.21.15.16:0/689253550 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe25003d060 con 0x563c8a6b3d40 2024-09-12T02:16:40.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.398+0000 7fe241ffb640 1 -- 172.21.15.16:0/689253550 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fe25001e890 con 0x563c8a6b3d40 2024-09-12T02:16:40.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.398+0000 7fe2632fd740 1 -- 172.21.15.16:0/689253550 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x563c8a615d00 con 0x563c8a6b3d40 2024-09-12T02:16:40.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.398+0000 7fe241ffb640 1 -- 172.21.15.16:0/689253550 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe250019040 con 0x563c8a6b3d40 2024-09-12T02:16:40.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.398+0000 7fe2632fd740 10 client.20394 fetch_fsmap learned FSMap version 25 2024-09-12T02:16:40.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.398+0000 7fe2632fd740 1 -- 172.21.15.16:0/689253550 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x563c8a732a90 con 0x563c8a6b3d40 2024-09-12T02:16:40.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.398+0000 7fe241ffb640 1 -- 172.21.15.16:0/689253550 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x563c8a615d00 con 0x563c8a6b3d40 2024-09-12T02:16:40.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.399+0000 7fe241ffb640 1 -- 172.21.15.16:0/689253550 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fe25002b870 con 0x563c8a6b3d40 2024-09-12T02:16:40.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.399+0000 7fe2632fd740 10 client.20394 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:16:40.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.399+0000 7fe2632fd740 10 client.20394 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:16:40.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.399+0000 7fe2632fd740 20 client.20394 populate_metadata read hostname 'smithi016' 2024-09-12T02:16:40.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.399+0000 7fe2632fd740 1 --2- 172.21.15.16:0/689253550 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x563c8a6c3810 0x563c8a735100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:40.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.399+0000 7fe2632fd740 4 client.20394 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-12T02:16:40.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.399+0000 7fe2632fd740 1 -- 172.21.15.16:0/689253550 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x563c8a592110 con 0x563c8a6c3810 2024-09-12T02:16:40.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.399+0000 7fe25a59c640 1 --2- 172.21.15.16:0/689253550 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x563c8a6c3810 0x563c8a735100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:40.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.399+0000 7fe241ffb640 10 client.20394 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:16:40.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.399+0000 7fe25a59c640 1 --2- 172.21.15.16:0/689253550 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x563c8a6c3810 0x563c8a735100 crc :-1 s=READY pgs=440 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bff800b76610eda5 server_cookie=9a95c173d14ab1a4 in_seq=0 out_seq=0 2024-09-12T02:16:40.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.400+0000 7fe241ffb640 1 -- 172.21.15.16:0/689253550 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x563c8a592110 con 0x563c8a6c3810 2024-09-12T02:16:40.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.400+0000 7fe241ffb640 10 client.20394 handle_command_reply: tid=0 2024-09-12T02:16:40.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.400+0000 7fe2632fd740 1 client.20394 shutdown 2024-09-12T02:16:40.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.400+0000 7fe2632fd740 1 -- 172.21.15.16:0/689253550 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c8a6b3d40 msgr2=0x563c8a6ae690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:40.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.400+0000 7fe2632fd740 1 --2- 172.21.15.16:0/689253550 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c8a6b3d40 0x563c8a6ae690 secure :-1 s=READY pgs=1986 cs=0 l=1 rev1=1 crypto rx=0x7fe25000a350 tx=0x7fe25001e5f0 comp rx=0 tx=0).stop 2024-09-12T02:16:40.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.400+0000 7fe2632fd740 1 -- 172.21.15.16:0/689253550 shutdown_connections 2024-09-12T02:16:40.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.401+0000 7fe2632fd740 1 --2- 172.21.15.16:0/689253550 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x563c8a6c3810 0x563c8a735100 crc :-1 s=READY pgs=440 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:40.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.401+0000 7fe2632fd740 1 --2- 172.21.15.16:0/689253550 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c8a6b3d40 0x563c8a6ae690 unknown :-1 s=CLOSED pgs=1986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:40.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.401+0000 7fe2632fd740 1 --2- 172.21.15.16:0/689253550 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563c8a6a9480 0x563c8a6ae150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:40.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.401+0000 7fe241ffb640 0 client.20394 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:16:40.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.401+0000 7fe2632fd740 1 --2- 172.21.15.16:0/689253550 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c8a6a8b60 0x563c8a6adc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:40.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.401+0000 7fe2632fd740 1 -- 172.21.15.16:0/689253550 >> 172.21.15.16:0/689253550 conn(0x563c8a6a50f0 msgr2=0x563c8a6b0b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:40.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.401+0000 7fe2632fd740 1 -- 172.21.15.16:0/689253550 shutdown_connections 2024-09-12T02:16:40.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.401+0000 7fe2632fd740 1 -- 172.21.15.16:0/689253550 wait complete. 2024-09-12T02:16:40.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.401+0000 7fe2632fd740 20 client.20394 trim_cache size 0 max 16384 2024-09-12T02:16:40.419 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T02:16:40.419 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-09-12T02:16:40.419 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "3cadb7ac-e61b-428a-ad8b-697c51865ad4", 2024-09-12T02:16:40.419 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-09-12T02:16:40.419 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T02:16:40.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.401+0000 7fe260f24640 1 -- 172.21.15.16:0/3702339494 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fe2440a2b80 msgr2=0x7fe2440a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:40.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.401+0000 7fe260f24640 1 --2- 172.21.15.16:0/3702339494 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fe2440a2b80 0x7fe2440a5040 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fe2540b8720 tx=0x7fe2540b74a0 comp rx=0 tx=0).stop 2024-09-12T02:16:40.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.401+0000 7fe260f24640 1 -- 172.21.15.16:0/3702339494 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe25c106a80 msgr2=0x7fe25c19cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:40.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.401+0000 7fe260f24640 1 --2- 172.21.15.16:0/3702339494 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe25c106a80 0x7fe25c19cc30 secure :-1 s=READY pgs=1982 cs=0 l=1 rev1=1 crypto rx=0x7fe254033040 tx=0x7fe254003970 comp rx=0 tx=0).stop 2024-09-12T02:16:40.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.402+0000 7fe260f24640 1 -- 172.21.15.16:0/3702339494 shutdown_connections 2024-09-12T02:16:40.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.402+0000 7fe260f24640 1 --2- 172.21.15.16:0/3702339494 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fe2440a2b80 0x7fe2440a5040 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:40.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.402+0000 7fe260f24640 1 --2- 172.21.15.16:0/3702339494 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe25c106a80 0x7fe25c19cc30 unknown :-1 s=CLOSED pgs=1982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:40.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.402+0000 7fe260f24640 1 --2- 172.21.15.16:0/3702339494 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe25c101420 0x7fe25c1988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:40.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.402+0000 7fe260f24640 1 --2- 172.21.15.16:0/3702339494 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe25c100a80 0x7fe25c198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:40.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.402+0000 7fe260f24640 1 -- 172.21.15.16:0/3702339494 >> 172.21.15.16:0/3702339494 conn(0x7fe25c0fc7a0 msgr2=0x7fe25c0fccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:40.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.402+0000 7fe260f24640 1 -- 172.21.15.16:0/3702339494 shutdown_connections 2024-09-12T02:16:40.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:40.402+0000 7fe260f24640 1 -- 172.21.15.16:0/3702339494 wait complete. 2024-09-12T02:16:40.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:16:40 smithi033 ceph-mon[35332]: pgmap v777: 97 pgs: 97 active+clean; 3.5 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-09-12T02:16:40.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:16:40 smithi016 ceph-mon[30947]: pgmap v777: 97 pgs: 97 active+clean; 3.5 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-09-12T02:16:40.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:16:40 smithi184 ceph-mon[35593]: pgmap v777: 97 pgs: 97 active+clean; 3.5 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-09-12T02:16:41.214 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T02:16:41.476 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:16:41.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:16:41 smithi033 ceph-mon[35332]: scrub queued for path: / 2024-09-12T02:16:41.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:16:41 smithi033 ceph-mon[35332]: scrub summary: idle+waiting paths [/] 2024-09-12T02:16:41.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:16:41 smithi033 ceph-mon[35332]: scrub summary: active paths [/] 2024-09-12T02:16:41.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:16:41 smithi033 ceph-mon[35332]: scrub summary: idle+waiting paths [/] 2024-09-12T02:16:41.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:16:41 smithi033 ceph-mon[35332]: scrub summary: idle 2024-09-12T02:16:41.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:16:41 smithi016 ceph-mon[30947]: scrub queued for path: / 2024-09-12T02:16:41.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:16:41 smithi016 ceph-mon[30947]: scrub summary: idle+waiting paths [/] 2024-09-12T02:16:41.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:16:41 smithi016 ceph-mon[30947]: scrub summary: active paths [/] 2024-09-12T02:16:41.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:16:41 smithi016 ceph-mon[30947]: scrub summary: idle+waiting paths [/] 2024-09-12T02:16:41.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:16:41 smithi016 ceph-mon[30947]: scrub summary: idle 2024-09-12T02:16:41.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:16:41 smithi184 ceph-mon[35593]: scrub queued for path: / 2024-09-12T02:16:41.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:16:41 smithi184 ceph-mon[35593]: scrub summary: idle+waiting paths [/] 2024-09-12T02:16:41.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:16:41 smithi184 ceph-mon[35593]: scrub summary: active paths [/] 2024-09-12T02:16:41.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:16:41 smithi184 ceph-mon[35593]: scrub summary: idle+waiting paths [/] 2024-09-12T02:16:41.878 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:16:41 smithi184 ceph-mon[35593]: scrub summary: idle 2024-09-12T02:16:41.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:41.976+0000 7f0138821640 1 -- 172.21.15.16:0/859800741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0134106a80 msgr2=0x7f0134106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:41.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:41.976+0000 7f0138821640 1 --2- 172.21.15.16:0/859800741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0134106a80 0x7f0134106e60 secure :-1 s=READY pgs=1987 cs=0 l=1 rev1=1 crypto rx=0x7f0124009930 tx=0x7f012402a420 comp rx=0 tx=0).stop 2024-09-12T02:16:41.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:41.976+0000 7f0138821640 1 -- 172.21.15.16:0/859800741 shutdown_connections 2024-09-12T02:16:41.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:41.976+0000 7f0138821640 1 --2- 172.21.15.16:0/859800741 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0134101420 0x7f013410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:41.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:41.976+0000 7f0138821640 1 --2- 172.21.15.16:0/859800741 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0134100a80 0x7f0134100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:41.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:41.976+0000 7f0138821640 1 --2- 172.21.15.16:0/859800741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0134106a80 0x7f0134106e60 unknown :-1 s=CLOSED pgs=1987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:41.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:41.976+0000 7f0138821640 1 -- 172.21.15.16:0/859800741 >> 172.21.15.16:0/859800741 conn(0x7f01340fc7a0 msgr2=0x7f01340febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:41.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:41.976+0000 7f0138821640 1 -- 172.21.15.16:0/859800741 shutdown_connections 2024-09-12T02:16:41.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:41.976+0000 7f0138821640 1 -- 172.21.15.16:0/859800741 wait complete. 2024-09-12T02:16:41.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:41.977+0000 7f0138821640 1 Processor -- start 2024-09-12T02:16:41.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:41.977+0000 7f0138821640 1 -- start start 2024-09-12T02:16:41.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:41.977+0000 7f0138821640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0134100a80 0x7f013419a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:41.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:41.977+0000 7f0138821640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0134101420 0x7f013419ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:41.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:41.977+0000 7f0138821640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0134106a80 0x7f013419ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:41.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:41.977+0000 7f0138821640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0134110e80 con 0x7f0134100a80 2024-09-12T02:16:41.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:41.977+0000 7f0138821640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0134110d00 con 0x7f0134101420 2024-09-12T02:16:41.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:41.977+0000 7f0138821640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f0134111000 con 0x7f0134106a80 2024-09-12T02:16:41.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:41.977+0000 7f013259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0134100a80 0x7f013419a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:41.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:41.977+0000 7f013259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0134100a80 0x7f013419a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51324/0 (socket says 172.21.15.16:51324) 2024-09-12T02:16:41.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:41.977+0000 7f013259c640 1 -- 172.21.15.16:0/3096610324 learned_addr learned my addr 172.21.15.16:0/3096610324 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:16:41.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:41.977+0000 7f0132d9d640 1 --2- 172.21.15.16:0/3096610324 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0134106a80 0x7f013419ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:41.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:41.978+0000 7f012bfff640 1 --2- 172.21.15.16:0/3096610324 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0134101420 0x7f013419ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:41.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:41.978+0000 7f013259c640 1 -- 172.21.15.16:0/3096610324 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0134106a80 msgr2=0x7f013419ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:41.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:41.978+0000 7f013259c640 1 --2- 172.21.15.16:0/3096610324 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0134106a80 0x7f013419ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:41.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:41.978+0000 7f013259c640 1 -- 172.21.15.16:0/3096610324 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0134101420 msgr2=0x7f013419ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:41.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:41.978+0000 7f013259c640 1 --2- 172.21.15.16:0/3096610324 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0134101420 0x7f013419ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:41.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:41.978+0000 7f013259c640 1 -- 172.21.15.16:0/3096610324 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f013419f600 con 0x7f0134100a80 2024-09-12T02:16:41.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:41.978+0000 7f012bfff640 1 --2- 172.21.15.16:0/3096610324 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0134101420 0x7f013419ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:16:41.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:41.978+0000 7f013259c640 1 --2- 172.21.15.16:0/3096610324 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0134100a80 0x7f013419a650 secure :-1 s=READY pgs=1988 cs=0 l=1 rev1=1 crypto rx=0x7f0124009230 tx=0x7f0124004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:41.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:41.978+0000 7f012b7fe640 1 -- 172.21.15.16:0/3096610324 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f012403f030 con 0x7f0134100a80 2024-09-12T02:16:41.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:41.978+0000 7f012b7fe640 1 -- 172.21.15.16:0/3096610324 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0124004440 con 0x7f0134100a80 2024-09-12T02:16:41.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:41.978+0000 7f0138821640 1 -- 172.21.15.16:0/3096610324 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f013419f890 con 0x7f0134100a80 2024-09-12T02:16:41.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:41.978+0000 7f012b7fe640 1 -- 172.21.15.16:0/3096610324 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0124035070 con 0x7f0134100a80 2024-09-12T02:16:41.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:41.978+0000 7f0138821640 1 -- 172.21.15.16:0/3096610324 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01341a71d0 con 0x7f0134100a80 2024-09-12T02:16:41.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:41.980+0000 7f012b7fe640 1 -- 172.21.15.16:0/3096610324 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f0124042030 con 0x7f0134100a80 2024-09-12T02:16:41.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:41.981+0000 7f013abfa740 1 Processor -- start 2024-09-12T02:16:41.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:41.981+0000 7f013abfa740 1 -- start start 2024-09-12T02:16:41.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:41.981+0000 7f013abfa740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9428ac4a0 0x55f9428ac880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:41.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:41.981+0000 7f013abfa740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f9428acdc0 0x55f9428b7140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:41.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:41.981+0000 7f013abfa740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f9428b7680 0x55f9428b9a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:41.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:41.981+0000 7f013abfa740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55f9428122e0 con 0x55f9428ac4a0 2024-09-12T02:16:41.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:41.981+0000 7f013abfa740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55f9427b9ce0 con 0x55f9428b7680 2024-09-12T02:16:41.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:41.981+0000 7f013abfa740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55f94280e2a0 con 0x55f9428acdc0 2024-09-12T02:16:41.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:41.981+0000 7f0132d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9428ac4a0 0x55f9428ac880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:41.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:41.981+0000 7f0132d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9428ac4a0 0x55f9428ac880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51328/0 (socket says 172.21.15.16:51328) 2024-09-12T02:16:41.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:41.981+0000 7f0132d9d640 1 -- 172.21.15.16:0/3527698169 learned_addr learned my addr 172.21.15.16:0/3527698169 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:16:41.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:41.982+0000 7f012b7fe640 1 --2- 172.21.15.16:0/3096610324 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f01200a2c00 0x7f01200a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:41.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:41.982+0000 7f013259c640 1 --2- 172.21.15.16:0/3527698169 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f9428acdc0 0x55f9428b7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:41.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:41.982+0000 7f012bfff640 1 --2- 172.21.15.16:0/3527698169 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f9428b7680 0x55f9428b9a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:41.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:41.982+0000 7f0132d9d640 1 -- 172.21.15.16:0/3527698169 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f9428acdc0 msgr2=0x55f9428b7140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:41.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:41.982+0000 7f0132d9d640 1 --2- 172.21.15.16:0/3527698169 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f9428acdc0 0x55f9428b7140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:41.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:41.982+0000 7f0132d9d640 1 -- 172.21.15.16:0/3527698169 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f9428b7680 msgr2=0x55f9428b9a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:41.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:41.982+0000 7f0132d9d640 1 --2- 172.21.15.16:0/3527698169 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f9428b7680 0x55f9428b9a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:41.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:41.982+0000 7f0132d9d640 1 -- 172.21.15.16:0/3527698169 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f94279e440 con 0x55f9428ac4a0 2024-09-12T02:16:41.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:41.982+0000 7f012bfff640 1 --2- 172.21.15.16:0/3527698169 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f9428b7680 0x55f9428b9a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:16:41.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:41.982+0000 7f013259c640 1 --2- 172.21.15.16:0/3527698169 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f9428acdc0 0x55f9428b7140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:16:41.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:41.982+0000 7f012b7fe640 1 -- 172.21.15.16:0/3096610324 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f01240ef2f0 con 0x7f0134100a80 2024-09-12T02:16:41.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:41.982+0000 7f0132d9d640 1 --2- 172.21.15.16:0/3096610324 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f01200a2c00 0x7f01200a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:41.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:41.982+0000 7f0132d9d640 1 --2- 172.21.15.16:0/3527698169 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9428ac4a0 0x55f9428ac880 secure :-1 s=READY pgs=1989 cs=0 l=1 rev1=1 crypto rx=0x7f012c011590 tx=0x7f012c011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df08752bcc6ad7fc server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:41.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:41.982+0000 7f0128ff9640 1 -- 172.21.15.16:0/3527698169 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f012c017050 con 0x55f9428ac4a0 2024-09-12T02:16:41.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:41.982+0000 7f0128ff9640 1 -- 172.21.15.16:0/3527698169 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f012c01bc50 con 0x55f9428ac4a0 2024-09-12T02:16:41.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:41.982+0000 7f0128ff9640 1 -- 172.21.15.16:0/3527698169 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f012c00e3b0 con 0x55f9428ac4a0 2024-09-12T02:16:41.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:41.983+0000 7f013abfa740 1 -- 172.21.15.16:0/3527698169 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9428ac4a0 msgr2=0x55f9428ac880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:41.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:41.983+0000 7f0132d9d640 1 --2- 172.21.15.16:0/3096610324 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f01200a2c00 0x7f01200a50c0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f012c022000 tx=0x7f012c023000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:41.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:41.983+0000 7f013abfa740 1 --2- 172.21.15.16:0/3527698169 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9428ac4a0 0x55f9428ac880 secure :-1 s=READY pgs=1989 cs=0 l=1 rev1=1 crypto rx=0x7f012c011590 tx=0x7f012c011a50 comp rx=0 tx=0).stop 2024-09-12T02:16:41.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:41.983+0000 7f013abfa740 1 -- 172.21.15.16:0/3527698169 shutdown_connections 2024-09-12T02:16:41.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:41.983+0000 7f013abfa740 1 --2- 172.21.15.16:0/3527698169 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f9428b7680 0x55f9428b9a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:41.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:41.983+0000 7f013abfa740 1 --2- 172.21.15.16:0/3527698169 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f9428acdc0 0x55f9428b7140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:41.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:41.983+0000 7f013abfa740 1 --2- 172.21.15.16:0/3527698169 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9428ac4a0 0x55f9428ac880 unknown :-1 s=CLOSED pgs=1989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:41.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:41.983+0000 7f013abfa740 1 -- 172.21.15.16:0/3527698169 >> 172.21.15.16:0/3527698169 conn(0x55f9428a8a30 msgr2=0x55f9428a8e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:41.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:41.983+0000 7f013abfa740 1 -- 172.21.15.16:0/3527698169 shutdown_connections 2024-09-12T02:16:41.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:41.983+0000 7f013abfa740 1 -- 172.21.15.16:0/3527698169 wait complete. 2024-09-12T02:16:41.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:41.984+0000 7f013abfa740 1 -- start start 2024-09-12T02:16:41.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:41.984+0000 7f013abfa740 1 Processor -- start 2024-09-12T02:16:41.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:41.984+0000 7f013abfa740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f9428ac4a0 0x55f9428b6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:41.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:41.984+0000 7f013abfa740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9428acdc0 0x55f9428b4180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:41.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:41.984+0000 7f013abfa740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f9428b7680 0x55f9428b46c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:41.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:41.984+0000 7f013abfa740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55f9428ba250 con 0x55f9428acdc0 2024-09-12T02:16:41.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:41.984+0000 7f013abfa740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55f9427cd050 con 0x55f9428ac4a0 2024-09-12T02:16:41.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:41.984+0000 7f013abfa740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55f9428ba3d0 con 0x55f9428b7680 2024-09-12T02:16:41.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:41.984+0000 7f013abfa740 10 client.0 _ll_register_callbacks cb 0x55f94280d160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:16:41.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:41.984+0000 7f012bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9428acdc0 0x55f9428b4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:41.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:41.984+0000 7f012bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9428acdc0 0x55f9428b4180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51338/0 (socket says 172.21.15.16:51338) 2024-09-12T02:16:41.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:41.984+0000 7f012bfff640 1 -- 172.21.15.16:0/1131315673 learned_addr learned my addr 172.21.15.16:0/1131315673 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:16:41.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:41.984+0000 7f0132d9d640 1 --2- 172.21.15.16:0/1131315673 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f9428b7680 0x55f9428b46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:41.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:41.984+0000 7f013259c640 1 --2- 172.21.15.16:0/1131315673 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f9428ac4a0 0x55f9428b6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:41.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:41.984+0000 7f012bfff640 1 -- 172.21.15.16:0/1131315673 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f9428b7680 msgr2=0x55f9428b46c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:41.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:41.984+0000 7f012bfff640 1 --2- 172.21.15.16:0/1131315673 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f9428b7680 0x55f9428b46c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:41.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:41.984+0000 7f012bfff640 1 -- 172.21.15.16:0/1131315673 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f9428ac4a0 msgr2=0x55f9428b6ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:41.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:41.984+0000 7f012bfff640 1 --2- 172.21.15.16:0/1131315673 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f9428ac4a0 0x55f9428b6ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:41.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:41.984+0000 7f012bfff640 1 -- 172.21.15.16:0/1131315673 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9425833c0 con 0x55f9428acdc0 2024-09-12T02:16:41.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:41.984+0000 7f013259c640 1 --2- 172.21.15.16:0/1131315673 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f9428ac4a0 0x55f9428b6ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:16:41.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:41.985+0000 7f012bfff640 1 --2- 172.21.15.16:0/1131315673 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9428acdc0 0x55f9428b4180 secure :-1 s=READY pgs=1990 cs=0 l=1 rev1=1 crypto rx=0x7f011c012780 tx=0x7f011c012c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5316c6a12c3b295d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:41.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:41.985+0000 7f0119ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:16:41.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:41.985+0000 7f0119ffb640 1 -- 172.21.15.16:0/1131315673 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f011c010dc0 con 0x55f9428acdc0 2024-09-12T02:16:41.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:41.985+0000 7f0119ffb640 1 -- 172.21.15.16:0/1131315673 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f011c004030 con 0x55f9428acdc0 2024-09-12T02:16:41.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:41.985+0000 7f013abfa740 1 -- 172.21.15.16:0/1131315673 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f012c01bc50 con 0x55f9428acdc0 2024-09-12T02:16:41.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:41.985+0000 7f0119ffb640 1 -- 172.21.15.16:0/1131315673 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f011c004230 con 0x55f9428acdc0 2024-09-12T02:16:41.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:41.985+0000 7f013abfa740 10 client.20406 fetch_fsmap learned FSMap version 25 2024-09-12T02:16:41.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:41.985+0000 7f013abfa740 1 -- 172.21.15.16:0/1131315673 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f9428c5af0 con 0x55f9428acdc0 2024-09-12T02:16:41.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:41.985+0000 7f0119ffb640 1 -- 172.21.15.16:0/1131315673 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f012c01bc50 con 0x55f9428acdc0 2024-09-12T02:16:41.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:41.985+0000 7f0119ffb640 1 -- 172.21.15.16:0/1131315673 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f011c01a4f0 con 0x55f9428acdc0 2024-09-12T02:16:41.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:41.986+0000 7f013abfa740 10 client.20406 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:16:41.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:41.986+0000 7f013abfa740 10 client.20406 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:16:41.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:41.986+0000 7f013abfa740 20 client.20406 populate_metadata read hostname 'smithi016' 2024-09-12T02:16:41.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:41.986+0000 7f013abfa740 1 --2- 172.21.15.16:0/1131315673 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55f9428c6fb0 0x55f9428c9470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:41.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:41.986+0000 7f013abfa740 4 client.20406 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:16:41.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:41.986+0000 7f013abfa740 1 -- 172.21.15.16:0/1131315673 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f94281f850 con 0x55f9428c6fb0 2024-09-12T02:16:41.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:41.986+0000 7f013259c640 1 --2- 172.21.15.16:0/1131315673 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55f9428c6fb0 0x55f9428c9470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:41.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:41.986+0000 7f0119ffb640 10 client.20406 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:16:41.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:41.986+0000 7f013259c640 1 --2- 172.21.15.16:0/1131315673 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55f9428c6fb0 0x55f9428c9470 crc :-1 s=READY pgs=441 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45cdee238e644ef6 server_cookie=2593c4e8fa7ad032 in_seq=0 out_seq=0 2024-09-12T02:16:41.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:41.987+0000 7f0119ffb640 1 -- 172.21.15.16:0/1131315673 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f94281f850 con 0x55f9428c6fb0 2024-09-12T02:16:41.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:41.987+0000 7f0119ffb640 10 client.20406 handle_command_reply: tid=0 2024-09-12T02:16:41.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:41.987+0000 7f013abfa740 1 client.20406 shutdown 2024-09-12T02:16:41.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:41.987+0000 7f013abfa740 1 -- 172.21.15.16:0/1131315673 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9428acdc0 msgr2=0x55f9428b4180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:41.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:41.987+0000 7f013abfa740 1 --2- 172.21.15.16:0/1131315673 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9428acdc0 0x55f9428b4180 secure :-1 s=READY pgs=1990 cs=0 l=1 rev1=1 crypto rx=0x7f011c012780 tx=0x7f011c012c40 comp rx=0 tx=0).stop 2024-09-12T02:16:41.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:41.987+0000 7f013abfa740 1 -- 172.21.15.16:0/1131315673 shutdown_connections 2024-09-12T02:16:41.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:41.987+0000 7f013abfa740 1 --2- 172.21.15.16:0/1131315673 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55f9428c6fb0 0x55f9428c9470 crc :-1 s=READY pgs=441 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:41.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:41.987+0000 7f013abfa740 1 --2- 172.21.15.16:0/1131315673 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f9428b7680 0x55f9428b46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:41.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:41.987+0000 7f013abfa740 1 --2- 172.21.15.16:0/1131315673 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9428acdc0 0x55f9428b4180 unknown :-1 s=CLOSED pgs=1990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:41.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:41.987+0000 7f013abfa740 1 --2- 172.21.15.16:0/1131315673 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f9428ac4a0 0x55f9428b6ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:41.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:41.987+0000 7f0119ffb640 0 client.20406 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:16:41.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:41.987+0000 7f013abfa740 1 -- 172.21.15.16:0/1131315673 >> 172.21.15.16:0/1131315673 conn(0x55f9428a8a30 msgr2=0x55f9428ae710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:41.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:41.987+0000 7f013abfa740 1 -- 172.21.15.16:0/1131315673 shutdown_connections 2024-09-12T02:16:41.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:41.987+0000 7f013abfa740 1 -- 172.21.15.16:0/1131315673 wait complete. 2024-09-12T02:16:41.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:41.987+0000 7f013abfa740 20 client.20406 trim_cache size 0 max 16384 2024-09-12T02:16:42.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:42.005+0000 7f013abfa740 1 Processor -- start 2024-09-12T02:16:42.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:42.005+0000 7f013abfa740 1 -- start start 2024-09-12T02:16:42.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:42.005+0000 7f013abfa740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9428a8a30 0x55f9428c34b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:42.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:42.005+0000 7f013abfa740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f9428ac4a0 0x55f9428af7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:42.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:42.005+0000 7f013abfa740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f9428acdc0 0x55f9428ced80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:42.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:42.005+0000 7f013abfa740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f01341b1f60 con 0x55f9428a8a30 2024-09-12T02:16:42.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:42.005+0000 7f013abfa740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f01341af830 con 0x55f9428ac4a0 2024-09-12T02:16:42.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:42.005+0000 7f013abfa740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f01341ad680 con 0x55f9428acdc0 2024-09-12T02:16:42.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:42.005+0000 7f013259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9428a8a30 0x55f9428c34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:42.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:42.005+0000 7f013259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9428a8a30 0x55f9428c34b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51350/0 (socket says 172.21.15.16:51350) 2024-09-12T02:16:42.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:42.005+0000 7f013259c640 1 -- 172.21.15.16:0/2293134408 learned_addr learned my addr 172.21.15.16:0/2293134408 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:16:42.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:42.005+0000 7f012bfff640 1 --2- 172.21.15.16:0/2293134408 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f9428ac4a0 0x55f9428af7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:42.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:42.006+0000 7f013259c640 1 -- 172.21.15.16:0/2293134408 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f9428acdc0 msgr2=0x55f9428ced80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:42.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:42.006+0000 7f013259c640 1 --2- 172.21.15.16:0/2293134408 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f9428acdc0 0x55f9428ced80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:42.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:42.006+0000 7f013259c640 1 -- 172.21.15.16:0/2293134408 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f9428ac4a0 msgr2=0x55f9428af7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:42.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:42.006+0000 7f013259c640 1 --2- 172.21.15.16:0/2293134408 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f9428ac4a0 0x55f9428af7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:42.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:42.006+0000 7f013259c640 1 -- 172.21.15.16:0/2293134408 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9428b7240 con 0x55f9428a8a30 2024-09-12T02:16:42.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:42.006+0000 7f013259c640 1 --2- 172.21.15.16:0/2293134408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9428a8a30 0x55f9428c34b0 secure :-1 s=READY pgs=1991 cs=0 l=1 rev1=1 crypto rx=0x7f01240ece70 tx=0x7f01240b7e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63d7c558465cc218 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:42.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:42.006+0000 7f011bfff640 1 -- 172.21.15.16:0/2293134408 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0124049040 con 0x55f9428a8a30 2024-09-12T02:16:42.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:42.006+0000 7f011bfff640 1 -- 172.21.15.16:0/2293134408 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f01240bff80 con 0x55f9428a8a30 2024-09-12T02:16:42.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:42.006+0000 7f011bfff640 1 -- 172.21.15.16:0/2293134408 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01240c43b0 con 0x55f9428a8a30 2024-09-12T02:16:42.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:42.007+0000 7f013abfa740 1 -- 172.21.15.16:0/2293134408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9428a8a30 msgr2=0x55f9428c34b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:42.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:42.007+0000 7f013abfa740 1 --2- 172.21.15.16:0/2293134408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9428a8a30 0x55f9428c34b0 secure :-1 s=READY pgs=1991 cs=0 l=1 rev1=1 crypto rx=0x7f01240ece70 tx=0x7f01240b7e70 comp rx=0 tx=0).stop 2024-09-12T02:16:42.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:42.007+0000 7f013abfa740 1 -- 172.21.15.16:0/2293134408 shutdown_connections 2024-09-12T02:16:42.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:42.007+0000 7f013abfa740 1 --2- 172.21.15.16:0/2293134408 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f9428acdc0 0x55f9428ced80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:42.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:42.007+0000 7f013abfa740 1 --2- 172.21.15.16:0/2293134408 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f9428ac4a0 0x55f9428af7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:42.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:42.007+0000 7f013abfa740 1 --2- 172.21.15.16:0/2293134408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9428a8a30 0x55f9428c34b0 unknown :-1 s=CLOSED pgs=1991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:42.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:42.007+0000 7f013abfa740 1 -- 172.21.15.16:0/2293134408 >> 172.21.15.16:0/2293134408 conn(0x55f9428b4820 msgr2=0x55f9428af0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:42.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:42.007+0000 7f013abfa740 1 -- 172.21.15.16:0/2293134408 shutdown_connections 2024-09-12T02:16:42.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:42.007+0000 7f013abfa740 1 -- 172.21.15.16:0/2293134408 wait complete. 2024-09-12T02:16:42.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:42.007+0000 7f013abfa740 1 -- start start 2024-09-12T02:16:42.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:42.007+0000 7f013abfa740 1 Processor -- start 2024-09-12T02:16:42.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:42.008+0000 7f013abfa740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f9428a8a30 0x55f9428bfef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:42.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:42.008+0000 7f013abfa740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9428ac4a0 0x55f9428bb220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:42.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:42.008+0000 7f013abfa740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f9428acdc0 0x55f9428bb9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:42.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:42.008+0000 7f013abfa740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f01341adcf0 con 0x55f9428ac4a0 2024-09-12T02:16:42.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:42.008+0000 7f013abfa740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f01341af160 con 0x55f9428a8a30 2024-09-12T02:16:42.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:42.008+0000 7f013abfa740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f01341ae3c0 con 0x55f9428acdc0 2024-09-12T02:16:42.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:42.008+0000 7f013abfa740 10 client.0 _ll_register_callbacks cb 0x7f011c0095b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:16:42.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:42.008+0000 7f012bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9428ac4a0 0x55f9428bb220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:42.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:42.008+0000 7f012bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9428ac4a0 0x55f9428bb220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51358/0 (socket says 172.21.15.16:51358) 2024-09-12T02:16:42.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:42.008+0000 7f013259c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f9428a8a30 0x55f9428bfef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:42.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:42.008+0000 7f012bfff640 1 -- 172.21.15.16:0/1989731260 learned_addr learned my addr 172.21.15.16:0/1989731260 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:16:42.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:42.008+0000 7f0132d9d640 1 --2- 172.21.15.16:0/1989731260 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f9428acdc0 0x55f9428bb9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:42.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:42.008+0000 7f012bfff640 1 -- 172.21.15.16:0/1989731260 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f9428acdc0 msgr2=0x55f9428bb9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:42.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:42.008+0000 7f012bfff640 1 --2- 172.21.15.16:0/1989731260 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f9428acdc0 0x55f9428bb9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:42.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:42.008+0000 7f012bfff640 1 -- 172.21.15.16:0/1989731260 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f9428a8a30 msgr2=0x55f9428bfef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:42.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:42.008+0000 7f012bfff640 1 --2- 172.21.15.16:0/1989731260 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f9428a8a30 0x55f9428bfef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:42.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:42.008+0000 7f012bfff640 1 -- 172.21.15.16:0/1989731260 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f942930d50 con 0x55f9428ac4a0 2024-09-12T02:16:42.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:42.008+0000 7f013259c640 1 --2- 172.21.15.16:0/1989731260 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f9428a8a30 0x55f9428bfef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:16:42.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:42.008+0000 7f012bfff640 1 --2- 172.21.15.16:0/1989731260 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9428ac4a0 0x55f9428bb220 secure :-1 s=READY pgs=1992 cs=0 l=1 rev1=1 crypto rx=0x7f011c009700 tx=0x7f011c009d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dec20b9530084eb3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:42.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:42.008+0000 7f0119ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:16:42.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:42.008+0000 7f0119ffb640 1 -- 172.21.15.16:0/1989731260 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f011c029040 con 0x55f9428ac4a0 2024-09-12T02:16:42.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:42.008+0000 7f0119ffb640 1 -- 172.21.15.16:0/1989731260 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f011c010dc0 con 0x55f9428ac4a0 2024-09-12T02:16:42.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:42.009+0000 7f013abfa740 1 -- 172.21.15.16:0/1989731260 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f01240bff80 con 0x55f9428ac4a0 2024-09-12T02:16:42.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:42.009+0000 7f0119ffb640 1 -- 172.21.15.16:0/1989731260 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f011c01a420 con 0x55f9428ac4a0 2024-09-12T02:16:42.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:42.009+0000 7f013abfa740 10 client.20412 fetch_fsmap learned FSMap version 25 2024-09-12T02:16:42.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:42.009+0000 7f013abfa740 1 -- 172.21.15.16:0/1989731260 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f942931ef0 con 0x55f9428ac4a0 2024-09-12T02:16:42.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:42.009+0000 7f0119ffb640 1 -- 172.21.15.16:0/1989731260 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f01240bff80 con 0x55f9428ac4a0 2024-09-12T02:16:42.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:42.009+0000 7f0119ffb640 1 -- 172.21.15.16:0/1989731260 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f011c025410 con 0x55f9428ac4a0 2024-09-12T02:16:42.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:42.009+0000 7f013abfa740 10 client.20412 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:16:42.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:42.009+0000 7f013abfa740 10 client.20412 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:16:42.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:42.009+0000 7f013abfa740 20 client.20412 populate_metadata read hostname 'smithi016' 2024-09-12T02:16:42.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:42.009+0000 7f013abfa740 1 --2- 172.21.15.16:0/1989731260 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55f9428b7680 0x55f942934730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:42.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:42.009+0000 7f013abfa740 4 client.20412 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T02:16:42.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:42.009+0000 7f013abfa740 1 -- 172.21.15.16:0/1989731260 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f942819d60 con 0x55f9428b7680 2024-09-12T02:16:42.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:42.009+0000 7f013259c640 1 --2- 172.21.15.16:0/1989731260 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55f9428b7680 0x55f942934730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:42.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:42.010+0000 7f013259c640 1 --2- 172.21.15.16:0/1989731260 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55f9428b7680 0x55f942934730 crc :-1 s=READY pgs=442 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dac21eb011d45fc1 server_cookie=8239e20743673c25 in_seq=0 out_seq=0 2024-09-12T02:16:42.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:42.010+0000 7f0119ffb640 10 client.20412 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:16:42.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:42.010+0000 7f0119ffb640 1 -- 172.21.15.16:0/1989731260 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55f942819d60 con 0x55f9428b7680 2024-09-12T02:16:42.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:42.010+0000 7f0119ffb640 10 client.20412 handle_command_reply: tid=0 2024-09-12T02:16:42.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:42.010+0000 7f013abfa740 1 client.20412 shutdown 2024-09-12T02:16:42.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:42.010+0000 7f013abfa740 1 -- 172.21.15.16:0/1989731260 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9428ac4a0 msgr2=0x55f9428bb220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:42.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:42.010+0000 7f013abfa740 1 --2- 172.21.15.16:0/1989731260 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9428ac4a0 0x55f9428bb220 secure :-1 s=READY pgs=1992 cs=0 l=1 rev1=1 crypto rx=0x7f011c009700 tx=0x7f011c009d30 comp rx=0 tx=0).stop 2024-09-12T02:16:42.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:42.010+0000 7f013abfa740 1 -- 172.21.15.16:0/1989731260 shutdown_connections 2024-09-12T02:16:42.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:42.010+0000 7f013abfa740 1 --2- 172.21.15.16:0/1989731260 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55f9428b7680 0x55f942934730 crc :-1 s=READY pgs=442 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:42.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:42.010+0000 7f013abfa740 1 --2- 172.21.15.16:0/1989731260 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f9428acdc0 0x55f9428bb9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:42.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:42.010+0000 7f013abfa740 1 --2- 172.21.15.16:0/1989731260 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9428ac4a0 0x55f9428bb220 unknown :-1 s=CLOSED pgs=1992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:42.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:42.010+0000 7f013abfa740 1 --2- 172.21.15.16:0/1989731260 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f9428a8a30 0x55f9428bfef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:42.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:42.010+0000 7f0119ffb640 0 client.20412 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:16:42.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:42.010+0000 7f013abfa740 1 -- 172.21.15.16:0/1989731260 >> 172.21.15.16:0/1989731260 conn(0x55f9428b4820 msgr2=0x55f9428adef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:42.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:42.010+0000 7f013abfa740 1 -- 172.21.15.16:0/1989731260 shutdown_connections 2024-09-12T02:16:42.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:42.010+0000 7f013abfa740 1 -- 172.21.15.16:0/1989731260 wait complete. 2024-09-12T02:16:42.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:42.010+0000 7f013abfa740 20 client.20412 trim_cache size 0 max 16384 2024-09-12T02:16:42.017 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T02:16:42.017 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-09-12T02:16:42.018 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {} 2024-09-12T02:16:42.018 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T02:16:42.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:42.011+0000 7f0138821640 1 -- 172.21.15.16:0/3096610324 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f01200a2c00 msgr2=0x7f01200a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:42.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:42.011+0000 7f0138821640 1 --2- 172.21.15.16:0/3096610324 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f01200a2c00 0x7f01200a50c0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f012c022000 tx=0x7f012c023000 comp rx=0 tx=0).stop 2024-09-12T02:16:42.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:42.011+0000 7f0138821640 1 -- 172.21.15.16:0/3096610324 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0134100a80 msgr2=0x7f013419a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:42.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:42.011+0000 7f0138821640 1 --2- 172.21.15.16:0/3096610324 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0134100a80 0x7f013419a650 secure :-1 s=READY pgs=1988 cs=0 l=1 rev1=1 crypto rx=0x7f0124009230 tx=0x7f0124004040 comp rx=0 tx=0).stop 2024-09-12T02:16:42.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:42.011+0000 7f0138821640 1 -- 172.21.15.16:0/3096610324 shutdown_connections 2024-09-12T02:16:42.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:42.011+0000 7f0138821640 1 --2- 172.21.15.16:0/3096610324 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f01200a2c00 0x7f01200a50c0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:42.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:42.011+0000 7f0138821640 1 --2- 172.21.15.16:0/3096610324 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0134106a80 0x7f013419ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:42.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:42.011+0000 7f0138821640 1 --2- 172.21.15.16:0/3096610324 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0134101420 0x7f013419ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:42.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:42.011+0000 7f0138821640 1 --2- 172.21.15.16:0/3096610324 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0134100a80 0x7f013419a650 unknown :-1 s=CLOSED pgs=1988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:42.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:42.011+0000 7f0138821640 1 -- 172.21.15.16:0/3096610324 >> 172.21.15.16:0/3096610324 conn(0x7f01340fc7a0 msgr2=0x7f013406a840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:42.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:42.011+0000 7f0138821640 1 -- 172.21.15.16:0/3096610324 shutdown_connections 2024-09-12T02:16:42.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:42.011+0000 7f0138821640 1 -- 172.21.15.16:0/3096610324 wait complete. 2024-09-12T02:16:42.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:16:42 smithi033 ceph-mon[35332]: pgmap v778: 97 pgs: 97 active+clean; 3.5 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-09-12T02:16:42.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:16:42 smithi016 ceph-mon[30947]: pgmap v778: 97 pgs: 97 active+clean; 3.5 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-09-12T02:16:42.735 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-12T02:16:42.736 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:16:42.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:16:42 smithi184 ceph-mon[35593]: pgmap v778: 97 pgs: 97 active+clean; 3.5 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-09-12T02:16:42.994 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:16:43.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:43.468+0000 7fbe1e080640 1 -- 172.21.15.16:0/1652155471 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe18102570 msgr2=0x7fbe181029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:43.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:43.468+0000 7fbe1e080640 1 --2- 172.21.15.16:0/1652155471 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe18102570 0x7fbe181029b0 secure :-1 s=READY pgs=1993 cs=0 l=1 rev1=1 crypto rx=0x7fbe0c009900 tx=0x7fbe0c02a480 comp rx=0 tx=0).stop 2024-09-12T02:16:43.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:43.468+0000 7fbe1e080640 1 -- 172.21.15.16:0/1652155471 shutdown_connections 2024-09-12T02:16:43.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:43.468+0000 7fbe1e080640 1 --2- 172.21.15.16:0/1652155471 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbe18102ef0 0x7fbe1810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:43.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:43.468+0000 7fbe1e080640 1 --2- 172.21.15.16:0/1652155471 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe18102570 0x7fbe181029b0 unknown :-1 s=CLOSED pgs=1993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:43.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:43.468+0000 7fbe1e080640 1 --2- 172.21.15.16:0/1652155471 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe18100e20 0x7fbe18101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:43.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:43.468+0000 7fbe1e080640 1 -- 172.21.15.16:0/1652155471 >> 172.21.15.16:0/1652155471 conn(0x7fbe180fc7d0 msgr2=0x7fbe180febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:43.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:43.468+0000 7fbe1e080640 1 -- 172.21.15.16:0/1652155471 shutdown_connections 2024-09-12T02:16:43.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:43.468+0000 7fbe1e080640 1 -- 172.21.15.16:0/1652155471 wait complete. 2024-09-12T02:16:43.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:43.469+0000 7fbe1e080640 1 Processor -- start 2024-09-12T02:16:43.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:43.469+0000 7fbe1e080640 1 -- start start 2024-09-12T02:16:43.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:43.469+0000 7fbe1e080640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbe18100e20 0x7fbe18197450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:43.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:43.469+0000 7fbe1e080640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe18102570 0x7fbe18197990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:43.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:43.469+0000 7fbe1e080640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe18102ef0 0x7fbe1819bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:43.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:43.469+0000 7fbe1e080640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbe18110f50 con 0x7fbe18102570 2024-09-12T02:16:43.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:43.469+0000 7fbe1e080640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbe18110dd0 con 0x7fbe18102ef0 2024-09-12T02:16:43.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:43.469+0000 7fbe1e080640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fbe181110d0 con 0x7fbe18100e20 2024-09-12T02:16:43.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:43.469+0000 7fbe16ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe18102570 0x7fbe18197990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:43.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:43.469+0000 7fbe16ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe18102570 0x7fbe18197990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51380/0 (socket says 172.21.15.16:51380) 2024-09-12T02:16:43.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:43.469+0000 7fbe177fe640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbe18100e20 0x7fbe18197450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:43.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:43.469+0000 7fbe16ffd640 1 -- 172.21.15.16:0/2857657776 learned_addr learned my addr 172.21.15.16:0/2857657776 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:16:43.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:43.470+0000 7fbe17fff640 1 --2- 172.21.15.16:0/2857657776 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe18102ef0 0x7fbe1819bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:43.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:43.470+0000 7fbe16ffd640 1 -- 172.21.15.16:0/2857657776 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbe18100e20 msgr2=0x7fbe18197450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:43.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:43.470+0000 7fbe16ffd640 1 --2- 172.21.15.16:0/2857657776 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbe18100e20 0x7fbe18197450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:43.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:43.470+0000 7fbe16ffd640 1 -- 172.21.15.16:0/2857657776 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe18102ef0 msgr2=0x7fbe1819bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:43.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:43.470+0000 7fbe16ffd640 1 --2- 172.21.15.16:0/2857657776 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe18102ef0 0x7fbe1819bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:43.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:43.470+0000 7fbe16ffd640 1 -- 172.21.15.16:0/2857657776 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe1819c400 con 0x7fbe18102570 2024-09-12T02:16:43.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:43.470+0000 7fbe16ffd640 1 --2- 172.21.15.16:0/2857657776 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe18102570 0x7fbe18197990 secure :-1 s=READY pgs=1994 cs=0 l=1 rev1=1 crypto rx=0x7fbe0c006010 tx=0x7fbe0c003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:43.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:43.470+0000 7fbe14ff9640 1 -- 172.21.15.16:0/2857657776 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe0c03f070 con 0x7fbe18102570 2024-09-12T02:16:43.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:43.470+0000 7fbe14ff9640 1 -- 172.21.15.16:0/2857657776 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe0c031c30 con 0x7fbe18102570 2024-09-12T02:16:43.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:43.470+0000 7fbe14ff9640 1 -- 172.21.15.16:0/2857657776 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe0c035030 con 0x7fbe18102570 2024-09-12T02:16:43.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:43.470+0000 7fbe1e080640 1 -- 172.21.15.16:0/2857657776 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe1819c690 con 0x7fbe18102570 2024-09-12T02:16:43.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:43.470+0000 7fbe1e080640 1 -- 172.21.15.16:0/2857657776 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe181a3fd0 con 0x7fbe18102570 2024-09-12T02:16:43.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:43.471+0000 7fbe1e080640 1 -- 172.21.15.16:0/2857657776 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe181021b0 con 0x7fbe18102570 2024-09-12T02:16:43.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:43.473+0000 7fbe14ff9640 1 -- 172.21.15.16:0/2857657776 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fbe0c042070 con 0x7fbe18102570 2024-09-12T02:16:43.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:43.473+0000 7fbe14ff9640 1 --2- 172.21.15.16:0/2857657776 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fbe000a2b80 0x7fbe000a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:43.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:43.473+0000 7fbe177fe640 1 --2- 172.21.15.16:0/2857657776 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fbe000a2b80 0x7fbe000a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:43.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:43.473+0000 7fbe14ff9640 1 -- 172.21.15.16:0/2857657776 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fbe0c0efad0 con 0x7fbe18102570 2024-09-12T02:16:43.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:43.474+0000 7fbe177fe640 1 --2- 172.21.15.16:0/2857657776 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fbe000a2b80 0x7fbe000a5040 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fbe08009bd0 tx=0x7fbe08005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:43.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:43.477+0000 7fbe14ff9640 1 -- 172.21.15.16:0/2857657776 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbe0c0bc480 con 0x7fbe18102570 2024-09-12T02:16:43.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:43.692+0000 7fbe1e080640 1 -- 172.21.15.16:0/2857657776 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbe18109090 con 0x7fbe18102570 2024-09-12T02:16:43.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:43.692+0000 7fbe14ff9640 1 -- 172.21.15.16:0/2857657776 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11911 (secure 0 0 0) 0x7fbe0c0c0780 con 0x7fbe18102570 2024-09-12T02:16:43.693 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T02:16:43.693 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":25,"btime":"2024-09-12T02:01:37:192021+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3279123279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3279123279},{"type":"v1","addr":"172.21.15.16:6835","nonce":3279123279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4119452382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4119452382},{"type":"v1","addr":"172.21.15.16:6837","nonce":4119452382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/3263823165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3263823165},{"type":"v1","addr":"172.21.15.16:6839","nonce":3263823165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/1510290732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":1510290732},{"type":"v1","addr":"172.21.15.16:6841","nonce":1510290732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/279674603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":279674603},{"type":"v1","addr":"172.21.15.33:6833","nonce":279674603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3555728028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3555728028},{"type":"v1","addr":"172.21.15.33:6835","nonce":3555728028}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T01:59:56.117608+0000","modified":"2024-09-12T02:01:36.294760+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24482,"mds_1":24481,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6833/1563374106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1563374106},{"type":"v1","addr":"172.21.15.184:6833","nonce":1563374106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6837/1892833868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1892833868},{"type":"v1","addr":"172.21.15.33:6837","nonce":1892833868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/1457679686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1457679686},{"type":"v1","addr":"172.21.15.184:6835","nonce":1457679686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.184:6837/1793393087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":1793393087},{"type":"v1","addr":"172.21.15.184:6837","nonce":1793393087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.184:6839/4203347941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":4203347941},{"type":"v1","addr":"172.21.15.184:6839","nonce":4203347941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.33:6839/4136670204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":4136670204},{"type":"v1","addr":"172.21.15.33:6839","nonce":4136670204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24481,24482,24469]},"id":1}]} 2024-09-12T02:16:43.693 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 25 2024-09-12T02:16:43.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:43.695+0000 7fbe1e080640 1 -- 172.21.15.16:0/2857657776 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fbe000a2b80 msgr2=0x7fbe000a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:43.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:43.695+0000 7fbe1e080640 1 --2- 172.21.15.16:0/2857657776 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fbe000a2b80 0x7fbe000a5040 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fbe08009bd0 tx=0x7fbe08005f90 comp rx=0 tx=0).stop 2024-09-12T02:16:43.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:43.695+0000 7fbe1e080640 1 -- 172.21.15.16:0/2857657776 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe18102570 msgr2=0x7fbe18197990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:43.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:43.695+0000 7fbe1e080640 1 --2- 172.21.15.16:0/2857657776 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe18102570 0x7fbe18197990 secure :-1 s=READY pgs=1994 cs=0 l=1 rev1=1 crypto rx=0x7fbe0c006010 tx=0x7fbe0c003c10 comp rx=0 tx=0).stop 2024-09-12T02:16:43.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:43.695+0000 7fbe1e080640 1 -- 172.21.15.16:0/2857657776 shutdown_connections 2024-09-12T02:16:43.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:43.695+0000 7fbe1e080640 1 --2- 172.21.15.16:0/2857657776 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fbe000a2b80 0x7fbe000a5040 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:43.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:43.695+0000 7fbe1e080640 1 --2- 172.21.15.16:0/2857657776 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe18102ef0 0x7fbe1819bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:43.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:43.695+0000 7fbe1e080640 1 --2- 172.21.15.16:0/2857657776 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe18102570 0x7fbe18197990 unknown :-1 s=CLOSED pgs=1994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:43.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:43.695+0000 7fbe1e080640 1 --2- 172.21.15.16:0/2857657776 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbe18100e20 0x7fbe18197450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:43.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:43.695+0000 7fbe1e080640 1 -- 172.21.15.16:0/2857657776 >> 172.21.15.16:0/2857657776 conn(0x7fbe180fc7d0 msgr2=0x7fbe180febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:43.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:43.695+0000 7fbe1e080640 1 -- 172.21.15.16:0/2857657776 shutdown_connections 2024-09-12T02:16:43.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:43.695+0000 7fbe1e080640 1 -- 172.21.15.16:0/2857657776 wait complete. 2024-09-12T02:16:44.404 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-12T02:16:44.664 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:16:44.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:16:44 smithi033 ceph-mon[35332]: pgmap v779: 97 pgs: 97 active+clean; 3.5 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-09-12T02:16:44.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:16:44 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/2857657776' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:16:44.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:16:44 smithi016 ceph-mon[30947]: pgmap v779: 97 pgs: 97 active+clean; 3.5 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-09-12T02:16:44.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:16:44 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/2857657776' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:16:44.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:16:44 smithi184 ceph-mon[35593]: pgmap v779: 97 pgs: 97 active+clean; 3.5 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-09-12T02:16:44.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:16:44 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/2857657776' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:16:45.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.098+0000 7f04dfea3640 1 -- 172.21.15.16:0/3438460834 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04d8102570 msgr2=0x7f04d81029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:45.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.098+0000 7f04dfea3640 1 --2- 172.21.15.16:0/3438460834 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04d8102570 0x7f04d81029b0 secure :-1 s=READY pgs=1995 cs=0 l=1 rev1=1 crypto rx=0x7f04c8009900 tx=0x7f04c802a480 comp rx=0 tx=0).stop 2024-09-12T02:16:45.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.098+0000 7f04dfea3640 1 -- 172.21.15.16:0/3438460834 shutdown_connections 2024-09-12T02:16:45.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.098+0000 7f04dfea3640 1 --2- 172.21.15.16:0/3438460834 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04d8102ef0 0x7f04d810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:45.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.098+0000 7f04dfea3640 1 --2- 172.21.15.16:0/3438460834 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04d8102570 0x7f04d81029b0 unknown :-1 s=CLOSED pgs=1995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:45.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.098+0000 7f04dfea3640 1 --2- 172.21.15.16:0/3438460834 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f04d8100e20 0x7f04d8101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:45.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.098+0000 7f04dfea3640 1 -- 172.21.15.16:0/3438460834 >> 172.21.15.16:0/3438460834 conn(0x7f04d80fc7d0 msgr2=0x7f04d80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:45.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.098+0000 7f04dfea3640 1 -- 172.21.15.16:0/3438460834 shutdown_connections 2024-09-12T02:16:45.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.098+0000 7f04dfea3640 1 -- 172.21.15.16:0/3438460834 wait complete. 2024-09-12T02:16:45.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.100+0000 7f04dfea3640 1 Processor -- start 2024-09-12T02:16:45.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.100+0000 7f04dfea3640 1 -- start start 2024-09-12T02:16:45.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.100+0000 7f04dfea3640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04d8100e20 0x7f04d819a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:45.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.100+0000 7f04dfea3640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f04d8102570 0x7f04d819ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:45.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.100+0000 7f04dfea3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04d8102ef0 0x7f04d819efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:45.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.100+0000 7f04dfea3640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f04d8110fa0 con 0x7f04d8102ef0 2024-09-12T02:16:45.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.100+0000 7f04dfea3640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f04d8110e20 con 0x7f04d8100e20 2024-09-12T02:16:45.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.100+0000 7f04dfea3640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f04d8111120 con 0x7f04d8102570 2024-09-12T02:16:45.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.100+0000 7f04de440640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04d8102ef0 0x7f04d819efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:45.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.100+0000 7f04de440640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04d8102ef0 0x7f04d819efe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51388/0 (socket says 172.21.15.16:51388) 2024-09-12T02:16:45.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.100+0000 7f04de440640 1 -- 172.21.15.16:0/1893697617 learned_addr learned my addr 172.21.15.16:0/1893697617 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:16:45.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.100+0000 7f04dd43e640 1 --2- 172.21.15.16:0/1893697617 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f04d8102570 0x7f04d819ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:45.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.100+0000 7f04ddc3f640 1 --2- 172.21.15.16:0/1893697617 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04d8100e20 0x7f04d819a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:45.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.100+0000 7f04de440640 1 -- 172.21.15.16:0/1893697617 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f04d8102570 msgr2=0x7f04d819ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:45.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.100+0000 7f04de440640 1 --2- 172.21.15.16:0/1893697617 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f04d8102570 0x7f04d819ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:45.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.100+0000 7f04de440640 1 -- 172.21.15.16:0/1893697617 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04d8100e20 msgr2=0x7f04d819a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:45.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.100+0000 7f04de440640 1 --2- 172.21.15.16:0/1893697617 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04d8100e20 0x7f04d819a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:45.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.100+0000 7f04de440640 1 -- 172.21.15.16:0/1893697617 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04d819f6c0 con 0x7f04d8102ef0 2024-09-12T02:16:45.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.101+0000 7f04de440640 1 --2- 172.21.15.16:0/1893697617 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04d8102ef0 0x7f04d819efe0 secure :-1 s=READY pgs=1996 cs=0 l=1 rev1=1 crypto rx=0x7f04d000cce0 tx=0x7f04d000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:45.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.101+0000 7f04ceffd640 1 -- 172.21.15.16:0/1893697617 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d000f450 con 0x7f04d8102ef0 2024-09-12T02:16:45.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.101+0000 7f04ceffd640 1 -- 172.21.15.16:0/1893697617 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04d000fad0 con 0x7f04d8102ef0 2024-09-12T02:16:45.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.101+0000 7f04dfea3640 1 -- 172.21.15.16:0/1893697617 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04d819f9b0 con 0x7f04d8102ef0 2024-09-12T02:16:45.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.101+0000 7f04ceffd640 1 -- 172.21.15.16:0/1893697617 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d0018b60 con 0x7f04d8102ef0 2024-09-12T02:16:45.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.101+0000 7f04dfea3640 1 -- 172.21.15.16:0/1893697617 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04d81a7290 con 0x7f04d8102ef0 2024-09-12T02:16:45.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.103+0000 7f04ceffd640 1 -- 172.21.15.16:0/1893697617 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f04d0018d00 con 0x7f04d8102ef0 2024-09-12T02:16:45.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.104+0000 7f04e227c740 1 Processor -- start 2024-09-12T02:16:45.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.104+0000 7f04e227c740 1 -- start start 2024-09-12T02:16:45.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.104+0000 7f04e227c740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5654821f5b40 0x5654821f4090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:45.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.104+0000 7f04e227c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5654821f45d0 0x5654821f4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:45.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.104+0000 7f04e227c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5654821ff370 0x565482201760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:45.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.104+0000 7f04e227c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56548215a2f0 con 0x5654821f45d0 2024-09-12T02:16:45.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.104+0000 7f04e227c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x565482101cf0 con 0x5654821ff370 2024-09-12T02:16:45.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.104+0000 7f04e227c740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5654821562b0 con 0x5654821f5b40 2024-09-12T02:16:45.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.104+0000 7f04ddc3f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5654821f45d0 0x5654821f4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:45.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.104+0000 7f04ceffd640 1 --2- 172.21.15.16:0/1893697617 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f04c40a2c00 0x7f04c40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:45.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.104+0000 7f04ddc3f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5654821f45d0 0x5654821f4a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51392/0 (socket says 172.21.15.16:51392) 2024-09-12T02:16:45.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.104+0000 7f04ddc3f640 1 -- 172.21.15.16:0/1042322917 learned_addr learned my addr 172.21.15.16:0/1042322917 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:16:45.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.104+0000 7f04dd43e640 1 --2- 172.21.15.16:0/1042322917 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5654821ff370 0x565482201760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:45.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.105+0000 7f04ceffd640 1 -- 172.21.15.16:0/1893697617 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f04d00cf420 con 0x7f04d8102ef0 2024-09-12T02:16:45.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.105+0000 7f04dd43e640 1 --2- 172.21.15.16:0/1042322917 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5654821f5b40 0x5654821f4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:45.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.105+0000 7f04de440640 1 --2- 172.21.15.16:0/1893697617 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f04c40a2c00 0x7f04c40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:45.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.105+0000 7f04ddc3f640 1 -- 172.21.15.16:0/1042322917 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5654821f5b40 msgr2=0x5654821f4090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:45.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.105+0000 7f04ddc3f640 1 --2- 172.21.15.16:0/1042322917 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5654821f5b40 0x5654821f4090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:45.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.105+0000 7f04ddc3f640 1 -- 172.21.15.16:0/1042322917 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5654821ff370 msgr2=0x565482201760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:45.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.105+0000 7f04ddc3f640 1 --2- 172.21.15.16:0/1042322917 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5654821ff370 0x565482201760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:45.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.105+0000 7f04ddc3f640 1 -- 172.21.15.16:0/1042322917 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654820e63a0 con 0x5654821f45d0 2024-09-12T02:16:45.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.105+0000 7f04ddc3f640 1 --2- 172.21.15.16:0/1042322917 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5654821f45d0 0x5654821f4a10 secure :-1 s=READY pgs=1997 cs=0 l=1 rev1=1 crypto rx=0x7f04d4009a70 tx=0x7f04d4009f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9520c04ddde1e50f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:45.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.105+0000 7f04de440640 1 --2- 172.21.15.16:0/1893697617 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f04c40a2c00 0x7f04c40a50c0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f04d009bcb0 tx=0x7f04d000cf10 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:45.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.105+0000 7f04c3fff640 1 -- 172.21.15.16:0/1042322917 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d4017070 con 0x5654821f45d0 2024-09-12T02:16:45.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.105+0000 7f04c3fff640 1 -- 172.21.15.16:0/1042322917 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04d4013470 con 0x5654821f45d0 2024-09-12T02:16:45.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.105+0000 7f04c3fff640 1 -- 172.21.15.16:0/1042322917 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d4012630 con 0x5654821f45d0 2024-09-12T02:16:45.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.106+0000 7f04e227c740 1 -- 172.21.15.16:0/1042322917 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5654821f45d0 msgr2=0x5654821f4a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:45.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.106+0000 7f04e227c740 1 --2- 172.21.15.16:0/1042322917 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5654821f45d0 0x5654821f4a10 secure :-1 s=READY pgs=1997 cs=0 l=1 rev1=1 crypto rx=0x7f04d4009a70 tx=0x7f04d4009f30 comp rx=0 tx=0).stop 2024-09-12T02:16:45.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.106+0000 7f04e227c740 1 -- 172.21.15.16:0/1042322917 shutdown_connections 2024-09-12T02:16:45.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.106+0000 7f04e227c740 1 --2- 172.21.15.16:0/1042322917 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5654821ff370 0x565482201760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:45.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.106+0000 7f04e227c740 1 --2- 172.21.15.16:0/1042322917 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5654821f45d0 0x5654821f4a10 unknown :-1 s=CLOSED pgs=1997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:45.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.106+0000 7f04e227c740 1 --2- 172.21.15.16:0/1042322917 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5654821f5b40 0x5654821f4090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:45.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.106+0000 7f04e227c740 1 -- 172.21.15.16:0/1042322917 >> 172.21.15.16:0/1042322917 conn(0x5654821bd310 msgr2=0x5654821f0c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:45.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.106+0000 7f04e227c740 1 -- 172.21.15.16:0/1042322917 shutdown_connections 2024-09-12T02:16:45.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.106+0000 7f04e227c740 1 -- 172.21.15.16:0/1042322917 wait complete. 2024-09-12T02:16:45.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.106+0000 7f04e227c740 1 -- start start 2024-09-12T02:16:45.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.107+0000 7f04e227c740 1 Processor -- start 2024-09-12T02:16:45.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.107+0000 7f04e227c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5654821f45d0 0x5654821fed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:45.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.107+0000 7f04e227c740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5654821f5b40 0x5654821fbfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:45.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.107+0000 7f04e227c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5654821ff370 0x5654821fc570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:45.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.107+0000 7f04e227c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x565482202070 con 0x5654821f45d0 2024-09-12T02:16:45.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.107+0000 7f04e227c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x565482115060 con 0x5654821ff370 2024-09-12T02:16:45.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.107+0000 7f04e227c740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5654822021f0 con 0x5654821f5b40 2024-09-12T02:16:45.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.107+0000 7f04e227c740 10 client.0 _ll_register_callbacks cb 0x565481e688c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:16:45.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.107+0000 7f04dd43e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5654821f45d0 0x5654821fed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:45.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.107+0000 7f04dd43e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5654821f45d0 0x5654821fed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51408/0 (socket says 172.21.15.16:51408) 2024-09-12T02:16:45.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.107+0000 7f04dd43e640 1 -- 172.21.15.16:0/1665258808 learned_addr learned my addr 172.21.15.16:0/1665258808 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:16:45.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.107+0000 7f04ddc3f640 1 --2- 172.21.15.16:0/1665258808 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5654821f5b40 0x5654821fbfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:45.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.107+0000 7f04dd43e640 1 -- 172.21.15.16:0/1665258808 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5654821f5b40 msgr2=0x5654821fbfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:45.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.107+0000 7f04dd43e640 1 --2- 172.21.15.16:0/1665258808 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5654821f5b40 0x5654821fbfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:45.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.107+0000 7f04dd43e640 1 -- 172.21.15.16:0/1665258808 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5654821ff370 msgr2=0x5654821fc570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:45.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.107+0000 7f04dd43e640 1 --2- 172.21.15.16:0/1665258808 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5654821ff370 0x5654821fc570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:45.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.108+0000 7f04dd43e640 1 -- 172.21.15.16:0/1665258808 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565481ecb3c0 con 0x5654821f45d0 2024-09-12T02:16:45.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.108+0000 7f04ddc3f640 1 --2- 172.21.15.16:0/1665258808 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5654821f5b40 0x5654821fbfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:16:45.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.108+0000 7f04dd43e640 1 --2- 172.21.15.16:0/1665258808 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5654821f45d0 0x5654821fed20 secure :-1 s=READY pgs=1998 cs=0 l=1 rev1=1 crypto rx=0x7f04c8009900 tx=0x7f04c80092d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aecd894134a8b048 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:45.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.108+0000 7f04c17fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:16:45.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.108+0000 7f04c17fa640 1 -- 172.21.15.16:0/1665258808 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04c8035040 con 0x5654821f45d0 2024-09-12T02:16:45.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.108+0000 7f04c17fa640 1 -- 172.21.15.16:0/1665258808 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f04c8003cf0 con 0x5654821f45d0 2024-09-12T02:16:45.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.108+0000 7f04e227c740 1 -- 172.21.15.16:0/1665258808 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04d4013470 con 0x5654821f45d0 2024-09-12T02:16:45.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.108+0000 7f04c17fa640 1 -- 172.21.15.16:0/1665258808 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f04c8031540 con 0x5654821f45d0 2024-09-12T02:16:45.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.108+0000 7f04e227c740 10 client.20430 fetch_fsmap learned FSMap version 25 2024-09-12T02:16:45.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.108+0000 7f04e227c740 1 -- 172.21.15.16:0/1665258808 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56548220dac0 con 0x5654821f45d0 2024-09-12T02:16:45.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.108+0000 7f04c17fa640 1 -- 172.21.15.16:0/1665258808 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04d4013470 con 0x5654821f45d0 2024-09-12T02:16:45.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.109+0000 7f04c17fa640 1 -- 172.21.15.16:0/1665258808 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f04c8047020 con 0x5654821f45d0 2024-09-12T02:16:45.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.109+0000 7f04e227c740 10 client.20430 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:16:45.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.109+0000 7f04e227c740 10 client.20430 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-12T02:16:45.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.109+0000 7f04e227c740 20 client.20430 populate_metadata read hostname 'smithi016' 2024-09-12T02:16:45.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.109+0000 7f04e227c740 1 --2- 172.21.15.16:0/1665258808 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x56548220ef80 0x5654822113d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:45.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.109+0000 7f04e227c740 4 client.20430 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:16:45.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.109+0000 7f04e227c740 1 -- 172.21.15.16:0/1665258808 --> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565482167860 con 0x56548220ef80 2024-09-12T02:16:45.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.109+0000 7f04ddc3f640 1 --2- 172.21.15.16:0/1665258808 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x56548220ef80 0x5654822113d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:45.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.109+0000 7f04ddc3f640 1 --2- 172.21.15.16:0/1665258808 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x56548220ef80 0x5654822113d0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=efeafc7419201dc3 server_cookie=c4491cd2a6e5662d in_seq=0 out_seq=0 2024-09-12T02:16:45.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.109+0000 7f04c17fa640 10 client.20430 ms_handle_connect on v2:172.21.15.184:6836/1793393087 2024-09-12T02:16:45.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.110+0000 7f04c17fa640 1 -- 172.21.15.16:0/1665258808 <== mds.2 v2:172.21.15.184:6836/1793393087 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x565482167860 con 0x56548220ef80 2024-09-12T02:16:45.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.110+0000 7f04c17fa640 10 client.20430 handle_command_reply: tid=0 2024-09-12T02:16:45.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.110+0000 7f04e227c740 1 client.20430 shutdown 2024-09-12T02:16:45.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.110+0000 7f04e227c740 1 -- 172.21.15.16:0/1665258808 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5654821f45d0 msgr2=0x5654821fed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:45.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.110+0000 7f04e227c740 1 --2- 172.21.15.16:0/1665258808 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5654821f45d0 0x5654821fed20 secure :-1 s=READY pgs=1998 cs=0 l=1 rev1=1 crypto rx=0x7f04c8009900 tx=0x7f04c80092d0 comp rx=0 tx=0).stop 2024-09-12T02:16:45.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.110+0000 7f04e227c740 1 -- 172.21.15.16:0/1665258808 shutdown_connections 2024-09-12T02:16:45.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.110+0000 7f04e227c740 1 --2- 172.21.15.16:0/1665258808 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5654821ff370 0x5654821fc570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:45.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.110+0000 7f04e227c740 1 --2- 172.21.15.16:0/1665258808 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5654821f5b40 0x5654821fbfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:45.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.110+0000 7f04e227c740 1 --2- 172.21.15.16:0/1665258808 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x56548220ef80 0x5654822113d0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:45.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.110+0000 7f04e227c740 1 --2- 172.21.15.16:0/1665258808 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5654821f45d0 0x5654821fed20 unknown :-1 s=CLOSED pgs=1998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:45.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.110+0000 7f04c17fa640 0 client.20430 ms_handle_reset on v2:172.21.15.184:6836/1793393087 2024-09-12T02:16:45.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.110+0000 7f04e227c740 1 -- 172.21.15.16:0/1665258808 >> 172.21.15.16:0/1665258808 conn(0x5654821bd310 msgr2=0x5654821f6960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:45.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.110+0000 7f04e227c740 1 -- 172.21.15.16:0/1665258808 shutdown_connections 2024-09-12T02:16:45.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.111+0000 7f04e227c740 1 -- 172.21.15.16:0/1665258808 wait complete. 2024-09-12T02:16:45.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.111+0000 7f04e227c740 20 client.20430 trim_cache size 0 max 16384 2024-09-12T02:16:45.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.123+0000 7f04e227c740 1 Processor -- start 2024-09-12T02:16:45.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.123+0000 7f04e227c740 1 -- start start 2024-09-12T02:16:45.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.124+0000 7f04e227c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5654821f45d0 0x5654821fdf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:45.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.124+0000 7f04e227c740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5654821f5b40 0x5654821fe480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:45.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.124+0000 7f04e227c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5654821ff370 0x5654821f7120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:45.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.124+0000 7f04e227c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f04d81b1ee0 con 0x5654821ff370 2024-09-12T02:16:45.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.124+0000 7f04e227c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f04d81af7b0 con 0x5654821f45d0 2024-09-12T02:16:45.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.124+0000 7f04e227c740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f04d81ad600 con 0x5654821f5b40 2024-09-12T02:16:45.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.124+0000 7f04dd43e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5654821ff370 0x5654821f7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:45.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.124+0000 7f04dd43e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5654821ff370 0x5654821f7120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51420/0 (socket says 172.21.15.16:51420) 2024-09-12T02:16:45.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.124+0000 7f04dd43e640 1 -- 172.21.15.16:0/1242281635 learned_addr learned my addr 172.21.15.16:0/1242281635 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:16:45.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.124+0000 7f04ddc3f640 1 --2- 172.21.15.16:0/1242281635 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5654821f5b40 0x5654821fe480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:45.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.124+0000 7f04dd43e640 1 --2- 172.21.15.16:0/1242281635 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5654821f45d0 0x5654821fdf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:45.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.124+0000 7f04dd43e640 1 -- 172.21.15.16:0/1242281635 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5654821f5b40 msgr2=0x5654821fe480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:45.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.124+0000 7f04dd43e640 1 --2- 172.21.15.16:0/1242281635 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5654821f5b40 0x5654821fe480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:45.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.124+0000 7f04dd43e640 1 -- 172.21.15.16:0/1242281635 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5654821f45d0 msgr2=0x5654821fdf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:45.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.124+0000 7f04dd43e640 1 --2- 172.21.15.16:0/1242281635 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5654821f45d0 0x5654821fdf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:45.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.124+0000 7f04dd43e640 1 -- 172.21.15.16:0/1242281635 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654821f1730 con 0x5654821ff370 2024-09-12T02:16:45.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.124+0000 7f04ddc3f640 1 --2- 172.21.15.16:0/1242281635 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5654821f5b40 0x5654821fe480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:16:45.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.124+0000 7f04dd43e640 1 --2- 172.21.15.16:0/1242281635 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5654821ff370 0x5654821f7120 secure :-1 s=READY pgs=1999 cs=0 l=1 rev1=1 crypto rx=0x7f04c803d960 tx=0x7f04c802a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c15c5252a2d35a2d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:45.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.125+0000 7f04c37fe640 1 -- 172.21.15.16:0/1242281635 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04c8035040 con 0x5654821ff370 2024-09-12T02:16:45.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.125+0000 7f04c37fe640 1 -- 172.21.15.16:0/1242281635 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04c8004400 con 0x5654821ff370 2024-09-12T02:16:45.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.125+0000 7f04c37fe640 1 -- 172.21.15.16:0/1242281635 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04c8005280 con 0x5654821ff370 2024-09-12T02:16:45.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.125+0000 7f04e227c740 1 -- 172.21.15.16:0/1242281635 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5654821ff370 msgr2=0x5654821f7120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:45.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.125+0000 7f04e227c740 1 --2- 172.21.15.16:0/1242281635 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5654821ff370 0x5654821f7120 secure :-1 s=READY pgs=1999 cs=0 l=1 rev1=1 crypto rx=0x7f04c803d960 tx=0x7f04c802a480 comp rx=0 tx=0).stop 2024-09-12T02:16:45.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.125+0000 7f04e227c740 1 -- 172.21.15.16:0/1242281635 shutdown_connections 2024-09-12T02:16:45.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.125+0000 7f04e227c740 1 --2- 172.21.15.16:0/1242281635 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5654821ff370 0x5654821f7120 unknown :-1 s=CLOSED pgs=1999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:45.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.125+0000 7f04e227c740 1 --2- 172.21.15.16:0/1242281635 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5654821f5b40 0x5654821fe480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:45.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.125+0000 7f04e227c740 1 --2- 172.21.15.16:0/1242281635 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5654821f45d0 0x5654821fdf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:45.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.125+0000 7f04e227c740 1 -- 172.21.15.16:0/1242281635 >> 172.21.15.16:0/1242281635 conn(0x5654821bd310 msgr2=0x5654821fd830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:45.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.125+0000 7f04e227c740 1 -- 172.21.15.16:0/1242281635 shutdown_connections 2024-09-12T02:16:45.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.125+0000 7f04e227c740 1 -- 172.21.15.16:0/1242281635 wait complete. 2024-09-12T02:16:45.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.126+0000 7f04e227c740 1 -- start start 2024-09-12T02:16:45.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.126+0000 7f04e227c740 1 Processor -- start 2024-09-12T02:16:45.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.126+0000 7f04e227c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5654821f45d0 0x565482207c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:45.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.126+0000 7f04e227c740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5654821f5b40 0x5654821f8d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:45.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.126+0000 7f04e227c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5654821ff370 0x5654821f9260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:45.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.126+0000 7f04e227c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f04d81adc70 con 0x5654821ff370 2024-09-12T02:16:45.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.126+0000 7f04e227c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f04d81af0e0 con 0x5654821f45d0 2024-09-12T02:16:45.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.126+0000 7f04e227c740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f04d81ae340 con 0x5654821f5b40 2024-09-12T02:16:45.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.126+0000 7f04e227c740 10 client.0 _ll_register_callbacks cb 0x7f04c8005bc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:16:45.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.126+0000 7f04dd43e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5654821ff370 0x5654821f9260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:45.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.126+0000 7f04dd43e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5654821ff370 0x5654821f9260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51432/0 (socket says 172.21.15.16:51432) 2024-09-12T02:16:45.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.126+0000 7f04dd43e640 1 -- 172.21.15.16:0/1202455612 learned_addr learned my addr 172.21.15.16:0/1202455612 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:16:45.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.127+0000 7f04ddc3f640 1 --2- 172.21.15.16:0/1202455612 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5654821f5b40 0x5654821f8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:45.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.127+0000 7f04dd43e640 1 --2- 172.21.15.16:0/1202455612 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5654821f45d0 0x565482207c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:45.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.127+0000 7f04dd43e640 1 -- 172.21.15.16:0/1202455612 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5654821f5b40 msgr2=0x5654821f8d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:45.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.127+0000 7f04dd43e640 1 --2- 172.21.15.16:0/1202455612 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5654821f5b40 0x5654821f8d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:45.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.127+0000 7f04dd43e640 1 -- 172.21.15.16:0/1202455612 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5654821f45d0 msgr2=0x565482207c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:45.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.127+0000 7f04dd43e640 1 --2- 172.21.15.16:0/1202455612 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5654821f45d0 0x565482207c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:45.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.127+0000 7f04dd43e640 1 -- 172.21.15.16:0/1202455612 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654821f9a90 con 0x5654821ff370 2024-09-12T02:16:45.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.127+0000 7f04dd43e640 1 --2- 172.21.15.16:0/1202455612 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5654821ff370 0x5654821f9260 secure :-1 s=READY pgs=2000 cs=0 l=1 rev1=1 crypto rx=0x7f04c8031f20 tx=0x7f04c8041040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58a2642fb836e764 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:45.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.127+0000 7f04c17fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:16:45.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.127+0000 7f04c17fa640 1 -- 172.21.15.16:0/1202455612 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04c8035070 con 0x5654821ff370 2024-09-12T02:16:45.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.127+0000 7f04e227c740 1 -- 172.21.15.16:0/1202455612 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04c8004400 con 0x5654821ff370 2024-09-12T02:16:45.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.127+0000 7f04c17fa640 1 -- 172.21.15.16:0/1202455612 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f04c8003af0 con 0x5654821ff370 2024-09-12T02:16:45.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.127+0000 7f04c17fa640 1 -- 172.21.15.16:0/1202455612 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f04c80318e0 con 0x5654821ff370 2024-09-12T02:16:45.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.127+0000 7f04e227c740 10 client.20436 fetch_fsmap learned FSMap version 25 2024-09-12T02:16:45.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.127+0000 7f04e227c740 1 -- 172.21.15.16:0/1202455612 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x565482279c80 con 0x5654821ff370 2024-09-12T02:16:45.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.128+0000 7f04c17fa640 1 -- 172.21.15.16:0/1202455612 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04c8004400 con 0x5654821ff370 2024-09-12T02:16:45.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.128+0000 7f04c17fa640 1 -- 172.21.15.16:0/1202455612 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f04c8047070 con 0x5654821ff370 2024-09-12T02:16:45.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.128+0000 7f04e227c740 10 client.20436 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:16:45.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.128+0000 7f04e227c740 10 client.20436 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-12T02:16:45.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.128+0000 7f04e227c740 20 client.20436 populate_metadata read hostname 'smithi016' 2024-09-12T02:16:45.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.128+0000 7f04e227c740 1 --2- 172.21.15.16:0/1202455612 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x56548227a3d0 0x56548227c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:45.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.128+0000 7f04e227c740 4 client.20436 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T02:16:45.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.128+0000 7f04e227c740 1 -- 172.21.15.16:0/1202455612 --> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] -- command(tid 0: {"prefix": "damage ls"}) -- 0x565482161d70 con 0x56548227a3d0 2024-09-12T02:16:45.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.128+0000 7f04ddc3f640 1 --2- 172.21.15.16:0/1202455612 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x56548227a3d0 0x56548227c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:45.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.128+0000 7f04ddc3f640 1 --2- 172.21.15.16:0/1202455612 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x56548227a3d0 0x56548227c7f0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bff4a843aed641bf server_cookie=3b73be4569d415e in_seq=0 out_seq=0 2024-09-12T02:16:45.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.128+0000 7f04c17fa640 10 client.20436 ms_handle_connect on v2:172.21.15.184:6836/1793393087 2024-09-12T02:16:45.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.129+0000 7f04c17fa640 1 -- 172.21.15.16:0/1202455612 <== mds.2 v2:172.21.15.184:6836/1793393087 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x565482161d70 con 0x56548227a3d0 2024-09-12T02:16:45.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.129+0000 7f04c17fa640 10 client.20436 handle_command_reply: tid=0 2024-09-12T02:16:45.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.129+0000 7f04e227c740 1 client.20436 shutdown 2024-09-12T02:16:45.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.129+0000 7f04e227c740 1 -- 172.21.15.16:0/1202455612 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5654821ff370 msgr2=0x5654821f9260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:45.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.129+0000 7f04e227c740 1 --2- 172.21.15.16:0/1202455612 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5654821ff370 0x5654821f9260 secure :-1 s=READY pgs=2000 cs=0 l=1 rev1=1 crypto rx=0x7f04c8031f20 tx=0x7f04c8041040 comp rx=0 tx=0).stop 2024-09-12T02:16:45.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.129+0000 7f04e227c740 1 -- 172.21.15.16:0/1202455612 shutdown_connections 2024-09-12T02:16:45.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.129+0000 7f04e227c740 1 --2- 172.21.15.16:0/1202455612 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x56548227a3d0 0x56548227c7f0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:45.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.129+0000 7f04e227c740 1 --2- 172.21.15.16:0/1202455612 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5654821ff370 0x5654821f9260 unknown :-1 s=CLOSED pgs=2000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:45.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.129+0000 7f04e227c740 1 --2- 172.21.15.16:0/1202455612 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5654821f5b40 0x5654821f8d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:45.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.129+0000 7f04c17fa640 0 client.20436 ms_handle_reset on v2:172.21.15.184:6836/1793393087 2024-09-12T02:16:45.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.129+0000 7f04e227c740 1 --2- 172.21.15.16:0/1202455612 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5654821f45d0 0x565482207c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:45.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.129+0000 7f04e227c740 1 -- 172.21.15.16:0/1202455612 >> 172.21.15.16:0/1202455612 conn(0x5654821bd310 msgr2=0x5654821fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:45.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.129+0000 7f04e227c740 1 -- 172.21.15.16:0/1202455612 shutdown_connections 2024-09-12T02:16:45.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.129+0000 7f04e227c740 1 -- 172.21.15.16:0/1202455612 wait complete. 2024-09-12T02:16:45.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.129+0000 7f04e227c740 20 client.20436 trim_cache size 0 max 16384 2024-09-12T02:16:45.136 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T02:16:45.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.130+0000 7f04dfea3640 1 -- 172.21.15.16:0/1893697617 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f04c40a2c00 msgr2=0x7f04c40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:45.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.130+0000 7f04dfea3640 1 --2- 172.21.15.16:0/1893697617 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f04c40a2c00 0x7f04c40a50c0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f04d009bcb0 tx=0x7f04d000cf10 comp rx=0 tx=0).stop 2024-09-12T02:16:45.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.130+0000 7f04dfea3640 1 -- 172.21.15.16:0/1893697617 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04d8102ef0 msgr2=0x7f04d819efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:45.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.130+0000 7f04dfea3640 1 --2- 172.21.15.16:0/1893697617 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04d8102ef0 0x7f04d819efe0 secure :-1 s=READY pgs=1996 cs=0 l=1 rev1=1 crypto rx=0x7f04d000cce0 tx=0x7f04d000a620 comp rx=0 tx=0).stop 2024-09-12T02:16:45.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.130+0000 7f04dfea3640 1 -- 172.21.15.16:0/1893697617 shutdown_connections 2024-09-12T02:16:45.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.130+0000 7f04dfea3640 1 --2- 172.21.15.16:0/1893697617 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f04c40a2c00 0x7f04c40a50c0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:45.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.130+0000 7f04dfea3640 1 --2- 172.21.15.16:0/1893697617 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04d8102ef0 0x7f04d819efe0 unknown :-1 s=CLOSED pgs=1996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:45.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.130+0000 7f04dfea3640 1 --2- 172.21.15.16:0/1893697617 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f04d8102570 0x7f04d819ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:45.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.130+0000 7f04dfea3640 1 --2- 172.21.15.16:0/1893697617 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04d8100e20 0x7f04d819a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:45.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.130+0000 7f04dfea3640 1 -- 172.21.15.16:0/1893697617 >> 172.21.15.16:0/1893697617 conn(0x7f04d80fc7d0 msgr2=0x7f04d806a470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:45.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.130+0000 7f04dfea3640 1 -- 172.21.15.16:0/1893697617 shutdown_connections 2024-09-12T02:16:45.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:45.130+0000 7f04dfea3640 1 -- 172.21.15.16:0/1893697617 wait complete. 2024-09-12T02:16:45.987 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-12T02:16:46.247 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:16:46.616 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:16:46 smithi016 ceph-mon[30947]: pgmap v780: 97 pgs: 97 active+clean; 3.5 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-09-12T02:16:46.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:16:46 smithi033 ceph-mon[35332]: pgmap v780: 97 pgs: 97 active+clean; 3.5 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-09-12T02:16:46.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.700+0000 7f137fb4e640 1 -- 172.21.15.16:0/1057941700 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1378106a80 msgr2=0x7f1378106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:46.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.700+0000 7f137fb4e640 1 --2- 172.21.15.16:0/1057941700 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1378106a80 0x7f1378106e60 secure :-1 s=READY pgs=2001 cs=0 l=1 rev1=1 crypto rx=0x7f1374009930 tx=0x7f137402a420 comp rx=0 tx=0).stop 2024-09-12T02:16:46.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.700+0000 7f137fb4e640 1 -- 172.21.15.16:0/1057941700 shutdown_connections 2024-09-12T02:16:46.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.700+0000 7f137fb4e640 1 --2- 172.21.15.16:0/1057941700 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1378101420 0x7f137810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:46.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.700+0000 7f137fb4e640 1 --2- 172.21.15.16:0/1057941700 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1378100a80 0x7f1378100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:46.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.700+0000 7f137fb4e640 1 --2- 172.21.15.16:0/1057941700 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1378106a80 0x7f1378106e60 unknown :-1 s=CLOSED pgs=2001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:46.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.700+0000 7f137fb4e640 1 -- 172.21.15.16:0/1057941700 >> 172.21.15.16:0/1057941700 conn(0x7f13780fc7a0 msgr2=0x7f13780febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:46.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.700+0000 7f137fb4e640 1 -- 172.21.15.16:0/1057941700 shutdown_connections 2024-09-12T02:16:46.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.700+0000 7f137fb4e640 1 -- 172.21.15.16:0/1057941700 wait complete. 2024-09-12T02:16:46.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.701+0000 7f137fb4e640 1 Processor -- start 2024-09-12T02:16:46.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.701+0000 7f137fb4e640 1 -- start start 2024-09-12T02:16:46.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.701+0000 7f137fb4e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1378100a80 0x7f13781983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:46.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.701+0000 7f137fb4e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1378101420 0x7f1378198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:46.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.701+0000 7f137fb4e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1378106a80 0x7f137819ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:46.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.701+0000 7f137fb4e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1378111f60 con 0x7f1378100a80 2024-09-12T02:16:46.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.701+0000 7f137fb4e640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1378111de0 con 0x7f1378106a80 2024-09-12T02:16:46.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.701+0000 7f137fb4e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f13781120e0 con 0x7f1378101420 2024-09-12T02:16:46.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.701+0000 7f137d8ea640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1378100a80 0x7f13781983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:46.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.701+0000 7f137d8ea640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1378100a80 0x7f13781983e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51436/0 (socket says 172.21.15.16:51436) 2024-09-12T02:16:46.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.701+0000 7f137d8ea640 1 -- 172.21.15.16:0/4136396725 learned_addr learned my addr 172.21.15.16:0/4136396725 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:16:46.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.701+0000 7f137e0eb640 1 --2- 172.21.15.16:0/4136396725 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1378106a80 0x7f137819ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:46.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.701+0000 7f137d0e9640 1 --2- 172.21.15.16:0/4136396725 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1378101420 0x7f1378198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:46.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.701+0000 7f137d8ea640 1 -- 172.21.15.16:0/4136396725 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1378101420 msgr2=0x7f1378198920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:46.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.701+0000 7f137d8ea640 1 --2- 172.21.15.16:0/4136396725 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1378101420 0x7f1378198920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:46.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.701+0000 7f137d8ea640 1 -- 172.21.15.16:0/4136396725 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1378106a80 msgr2=0x7f137819ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:46.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.701+0000 7f137d8ea640 1 --2- 172.21.15.16:0/4136396725 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1378106a80 0x7f137819ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:46.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.701+0000 7f137d8ea640 1 -- 172.21.15.16:0/4136396725 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f137819d390 con 0x7f1378100a80 2024-09-12T02:16:46.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.701+0000 7f137e0eb640 1 --2- 172.21.15.16:0/4136396725 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1378106a80 0x7f137819ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:16:46.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.702+0000 7f137d8ea640 1 --2- 172.21.15.16:0/4136396725 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1378100a80 0x7f13781983e0 secure :-1 s=READY pgs=2002 cs=0 l=1 rev1=1 crypto rx=0x7f1374009230 tx=0x7f1374004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:46.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.702+0000 7f136effd640 1 -- 172.21.15.16:0/4136396725 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f137403f030 con 0x7f1378100a80 2024-09-12T02:16:46.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.702+0000 7f136effd640 1 -- 172.21.15.16:0/4136396725 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1374003ad0 con 0x7f1378100a80 2024-09-12T02:16:46.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.702+0000 7f137fb4e640 1 -- 172.21.15.16:0/4136396725 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f137819d620 con 0x7f1378100a80 2024-09-12T02:16:46.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.702+0000 7f136effd640 1 -- 172.21.15.16:0/4136396725 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1374035070 con 0x7f1378100a80 2024-09-12T02:16:46.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.702+0000 7f137fb4e640 1 -- 172.21.15.16:0/4136396725 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13781a4f60 con 0x7f1378100a80 2024-09-12T02:16:46.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.704+0000 7f136effd640 1 -- 172.21.15.16:0/4136396725 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f1374042030 con 0x7f1378100a80 2024-09-12T02:16:46.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.704+0000 7f1381f27740 1 Processor -- start 2024-09-12T02:16:46.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.705+0000 7f1381f27740 1 -- start start 2024-09-12T02:16:46.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.705+0000 7f1381f27740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55edba701b40 0x55edba700090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:46.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.705+0000 7f1381f27740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55edba7005d0 0x55edba700a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:46.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.705+0000 7f1381f27740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55edba70f780 0x55edba711b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:46.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.705+0000 7f1381f27740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55edba6662f0 con 0x55edba7005d0 2024-09-12T02:16:46.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.705+0000 7f1381f27740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55edba60dcf0 con 0x55edba70f780 2024-09-12T02:16:46.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.705+0000 7f1381f27740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55edba6622b0 con 0x55edba701b40 2024-09-12T02:16:46.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.705+0000 7f137d8ea640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55edba7005d0 0x55edba700a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:46.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.705+0000 7f136effd640 1 --2- 172.21.15.16:0/4136396725 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f13640a2b80 0x7f13640a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:46.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.705+0000 7f137d8ea640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55edba7005d0 0x55edba700a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51438/0 (socket says 172.21.15.16:51438) 2024-09-12T02:16:46.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.705+0000 7f137d8ea640 1 -- 172.21.15.16:0/1586142408 learned_addr learned my addr 172.21.15.16:0/1586142408 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:16:46.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.705+0000 7f137e0eb640 1 --2- 172.21.15.16:0/1586142408 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55edba701b40 0x55edba700090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:46.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.705+0000 7f137e0eb640 1 --2- 172.21.15.16:0/4136396725 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f13640a2b80 0x7f13640a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:46.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.705+0000 7f137d0e9640 1 --2- 172.21.15.16:0/1586142408 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55edba70f780 0x55edba711b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:46.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.706+0000 7f136effd640 1 -- 172.21.15.16:0/4136396725 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f13740ef550 con 0x7f1378100a80 2024-09-12T02:16:46.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.706+0000 7f137e0eb640 1 -- 172.21.15.16:0/1586142408 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55edba70f780 msgr2=0x55edba711b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:46.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.706+0000 7f137e0eb640 1 --2- 172.21.15.16:0/1586142408 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55edba70f780 0x55edba711b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:46.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.706+0000 7f137e0eb640 1 -- 172.21.15.16:0/1586142408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55edba7005d0 msgr2=0x55edba700a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:46.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.706+0000 7f137e0eb640 1 --2- 172.21.15.16:0/1586142408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55edba7005d0 0x55edba700a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:46.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.706+0000 7f137e0eb640 1 -- 172.21.15.16:0/1586142408 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55edba5f23a0 con 0x55edba701b40 2024-09-12T02:16:46.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.706+0000 7f137d8ea640 1 --2- 172.21.15.16:0/1586142408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55edba7005d0 0x55edba700a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:16:46.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.706+0000 7f137e0eb640 1 --2- 172.21.15.16:0/4136396725 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f13640a2b80 0x7f13640a5040 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f1370005fd0 tx=0x7f1370017000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:46.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.706+0000 7f137e0eb640 1 --2- 172.21.15.16:0/1586142408 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55edba701b40 0x55edba700090 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f13700092f0 tx=0x7f1370010740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c23a6859884b4e17 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:46.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.706+0000 7f1363fff640 1 -- 172.21.15.16:0/1586142408 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1370023070 con 0x55edba701b40 2024-09-12T02:16:46.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.706+0000 7f1363fff640 1 -- 172.21.15.16:0/1586142408 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1370009c70 con 0x55edba701b40 2024-09-12T02:16:46.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.706+0000 7f1363fff640 1 -- 172.21.15.16:0/1586142408 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1370005020 con 0x55edba701b40 2024-09-12T02:16:46.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.707+0000 7f1381f27740 1 -- 172.21.15.16:0/1586142408 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55edba701b40 msgr2=0x55edba700090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:46.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.707+0000 7f1381f27740 1 --2- 172.21.15.16:0/1586142408 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55edba701b40 0x55edba700090 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f13700092f0 tx=0x7f1370010740 comp rx=0 tx=0).stop 2024-09-12T02:16:46.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.707+0000 7f1381f27740 1 -- 172.21.15.16:0/1586142408 shutdown_connections 2024-09-12T02:16:46.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.707+0000 7f1381f27740 1 --2- 172.21.15.16:0/1586142408 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55edba70f780 0x55edba711b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:46.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.707+0000 7f1381f27740 1 --2- 172.21.15.16:0/1586142408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55edba7005d0 0x55edba700a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:46.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.707+0000 7f1381f27740 1 --2- 172.21.15.16:0/1586142408 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55edba701b40 0x55edba700090 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:46.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.707+0000 7f1381f27740 1 -- 172.21.15.16:0/1586142408 >> 172.21.15.16:0/1586142408 conn(0x55edba6c9310 msgr2=0x55edba6fcc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:46.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.707+0000 7f1381f27740 1 -- 172.21.15.16:0/1586142408 shutdown_connections 2024-09-12T02:16:46.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.707+0000 7f1381f27740 1 -- 172.21.15.16:0/1586142408 wait complete. 2024-09-12T02:16:46.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.708+0000 7f1381f27740 1 -- start start 2024-09-12T02:16:46.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.708+0000 7f1381f27740 1 Processor -- start 2024-09-12T02:16:46.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.708+0000 7f1381f27740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55edba7005d0 0x55edba70f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:46.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.708+0000 7f1381f27740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55edba701b40 0x55edba70c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:46.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.708+0000 7f1381f27740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55edba70f780 0x55edba70cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:46.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.708+0000 7f1381f27740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55edba712420 con 0x55edba701b40 2024-09-12T02:16:46.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.708+0000 7f1381f27740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55edba621060 con 0x55edba7005d0 2024-09-12T02:16:46.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.708+0000 7f1381f27740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55edba7125a0 con 0x55edba70f780 2024-09-12T02:16:46.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.708+0000 7f1381f27740 10 client.0 _ll_register_callbacks cb 0x55edba3748c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:16:46.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.708+0000 7f137d0e9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55edba701b40 0x55edba70c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:46.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.708+0000 7f137d0e9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55edba701b40 0x55edba70c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51450/0 (socket says 172.21.15.16:51450) 2024-09-12T02:16:46.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.708+0000 7f137d0e9640 1 -- 172.21.15.16:0/2002286492 learned_addr learned my addr 172.21.15.16:0/2002286492 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:16:46.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.708+0000 7f137d8ea640 1 --2- 172.21.15.16:0/2002286492 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55edba7005d0 0x55edba70f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:46.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.708+0000 7f137e0eb640 1 --2- 172.21.15.16:0/2002286492 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55edba70f780 0x55edba70cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:46.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.709+0000 7f137d0e9640 1 -- 172.21.15.16:0/2002286492 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55edba70f780 msgr2=0x55edba70cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:46.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.709+0000 7f137d0e9640 1 --2- 172.21.15.16:0/2002286492 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55edba70f780 0x55edba70cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:46.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.709+0000 7f137d0e9640 1 -- 172.21.15.16:0/2002286492 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55edba7005d0 msgr2=0x55edba70f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:46.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.709+0000 7f137d0e9640 1 --2- 172.21.15.16:0/2002286492 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55edba7005d0 0x55edba70f1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:46.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.709+0000 7f137d0e9640 1 -- 172.21.15.16:0/2002286492 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55edba3d73c0 con 0x55edba701b40 2024-09-12T02:16:46.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.709+0000 7f137d0e9640 1 --2- 172.21.15.16:0/2002286492 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55edba701b40 0x55edba70c480 secure :-1 s=READY pgs=2003 cs=0 l=1 rev1=1 crypto rx=0x7f136800cd50 tx=0x7f13680039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de27d5169991d204 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:46.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.709+0000 7f13617fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:16:46.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.709+0000 7f13617fa640 1 -- 172.21.15.16:0/2002286492 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1368010840 con 0x55edba701b40 2024-09-12T02:16:46.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.709+0000 7f13617fa640 1 -- 172.21.15.16:0/2002286492 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f13680109e0 con 0x55edba701b40 2024-09-12T02:16:46.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.709+0000 7f1381f27740 1 -- 172.21.15.16:0/2002286492 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55edba673860 con 0x55edba701b40 2024-09-12T02:16:46.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.709+0000 7f13617fa640 1 -- 172.21.15.16:0/2002286492 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1368010be0 con 0x55edba701b40 2024-09-12T02:16:46.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.709+0000 7f1381f27740 10 client.20448 fetch_fsmap learned FSMap version 25 2024-09-12T02:16:46.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.709+0000 7f1381f27740 1 -- 172.21.15.16:0/2002286492 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55edba71df90 con 0x55edba701b40 2024-09-12T02:16:46.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.710+0000 7f13617fa640 1 -- 172.21.15.16:0/2002286492 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55edba673860 con 0x55edba701b40 2024-09-12T02:16:46.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.710+0000 7f13617fa640 1 -- 172.21.15.16:0/2002286492 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f1368018020 con 0x55edba701b40 2024-09-12T02:16:46.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.710+0000 7f1381f27740 10 client.20448 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:16:46.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.710+0000 7f1381f27740 10 client.20448 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T02:16:46.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.710+0000 7f1381f27740 20 client.20448 populate_metadata read hostname 'smithi016' 2024-09-12T02:16:46.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.710+0000 7f1381f27740 1 --2- 172.21.15.16:0/2002286492 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x55edba71f450 0x55edba721880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:46.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.710+0000 7f1381f27740 4 client.20448 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:16:46.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.710+0000 7f1381f27740 1 -- 172.21.15.16:0/2002286492 --> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55edba66dd70 con 0x55edba71f450 2024-09-12T02:16:46.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.710+0000 7f137d8ea640 1 --2- 172.21.15.16:0/2002286492 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x55edba71f450 0x55edba721880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:46.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.711+0000 7f13617fa640 10 client.20448 ms_handle_connect on v2:172.21.15.184:6838/4203347941 2024-09-12T02:16:46.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.711+0000 7f137d8ea640 1 --2- 172.21.15.16:0/2002286492 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x55edba71f450 0x55edba721880 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f678d46e13a580c0 server_cookie=35b8c9811687fcb1 in_seq=0 out_seq=0 2024-09-12T02:16:46.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.711+0000 7f13617fa640 1 -- 172.21.15.16:0/2002286492 <== mds.1 v2:172.21.15.184:6838/4203347941 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55edba66dd70 con 0x55edba71f450 2024-09-12T02:16:46.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.711+0000 7f13617fa640 10 client.20448 handle_command_reply: tid=0 2024-09-12T02:16:46.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.712+0000 7f1381f27740 1 client.20448 shutdown 2024-09-12T02:16:46.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.712+0000 7f1381f27740 1 -- 172.21.15.16:0/2002286492 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55edba701b40 msgr2=0x55edba70c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:46.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.712+0000 7f1381f27740 1 --2- 172.21.15.16:0/2002286492 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55edba701b40 0x55edba70c480 secure :-1 s=READY pgs=2003 cs=0 l=1 rev1=1 crypto rx=0x7f136800cd50 tx=0x7f13680039c0 comp rx=0 tx=0).stop 2024-09-12T02:16:46.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.712+0000 7f1381f27740 1 -- 172.21.15.16:0/2002286492 shutdown_connections 2024-09-12T02:16:46.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.712+0000 7f1381f27740 1 --2- 172.21.15.16:0/2002286492 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x55edba71f450 0x55edba721880 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:46.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.712+0000 7f1381f27740 1 --2- 172.21.15.16:0/2002286492 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55edba70f780 0x55edba70cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:46.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.712+0000 7f1381f27740 1 --2- 172.21.15.16:0/2002286492 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55edba701b40 0x55edba70c480 unknown :-1 s=CLOSED pgs=2003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:46.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.712+0000 7f1381f27740 1 --2- 172.21.15.16:0/2002286492 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55edba7005d0 0x55edba70f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:46.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.712+0000 7f13617fa640 0 client.20448 ms_handle_reset on v2:172.21.15.184:6838/4203347941 2024-09-12T02:16:46.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.712+0000 7f1381f27740 1 -- 172.21.15.16:0/2002286492 >> 172.21.15.16:0/2002286492 conn(0x55edba6c9310 msgr2=0x55edba702960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:46.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.712+0000 7f1381f27740 1 -- 172.21.15.16:0/2002286492 shutdown_connections 2024-09-12T02:16:46.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.712+0000 7f1381f27740 1 -- 172.21.15.16:0/2002286492 wait complete. 2024-09-12T02:16:46.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.712+0000 7f1381f27740 20 client.20448 trim_cache size 0 max 16384 2024-09-12T02:16:46.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.725+0000 7f1381f27740 1 Processor -- start 2024-09-12T02:16:46.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.725+0000 7f1381f27740 1 -- start start 2024-09-12T02:16:46.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.725+0000 7f1381f27740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55edba7005d0 0x55edba70bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:46.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.725+0000 7f1381f27740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55edba701b40 0x55edba70ad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:46.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.725+0000 7f1381f27740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55edba70f780 0x55edba70b290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:46.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.725+0000 7f1381f27740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f13781ad9f0 con 0x55edba70f780 2024-09-12T02:16:46.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.725+0000 7f1381f27740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f13781ab2c0 con 0x55edba701b40 2024-09-12T02:16:46.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.725+0000 7f1381f27740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f13781a9110 con 0x55edba7005d0 2024-09-12T02:16:46.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.725+0000 7f137e0eb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55edba70f780 0x55edba70b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:46.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.725+0000 7f137e0eb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55edba70f780 0x55edba70b290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51464/0 (socket says 172.21.15.16:51464) 2024-09-12T02:16:46.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.725+0000 7f137e0eb640 1 -- 172.21.15.16:0/1390725129 learned_addr learned my addr 172.21.15.16:0/1390725129 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:16:46.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.725+0000 7f137d8ea640 1 --2- 172.21.15.16:0/1390725129 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55edba7005d0 0x55edba70bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:46.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.725+0000 7f137d0e9640 1 --2- 172.21.15.16:0/1390725129 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55edba701b40 0x55edba70ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:46.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.726+0000 7f137e0eb640 1 -- 172.21.15.16:0/1390725129 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55edba7005d0 msgr2=0x55edba70bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:46.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.726+0000 7f137e0eb640 1 --2- 172.21.15.16:0/1390725129 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55edba7005d0 0x55edba70bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:46.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.726+0000 7f137e0eb640 1 -- 172.21.15.16:0/1390725129 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55edba701b40 msgr2=0x55edba70ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:46.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.726+0000 7f137e0eb640 1 --2- 172.21.15.16:0/1390725129 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55edba701b40 0x55edba70ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:46.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.726+0000 7f137e0eb640 1 -- 172.21.15.16:0/1390725129 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55edba70a9d0 con 0x55edba70f780 2024-09-12T02:16:46.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.726+0000 7f137d8ea640 1 --2- 172.21.15.16:0/1390725129 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55edba7005d0 0x55edba70bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:16:46.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.726+0000 7f137e0eb640 1 --2- 172.21.15.16:0/1390725129 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55edba70f780 0x55edba70b290 secure :-1 s=READY pgs=2004 cs=0 l=1 rev1=1 crypto rx=0x7f1370013010 tx=0x7f1370009eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3166af01d366dda server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:46.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.726+0000 7f13637fe640 1 -- 172.21.15.16:0/1390725129 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1370023040 con 0x55edba70f780 2024-09-12T02:16:46.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.726+0000 7f13637fe640 1 -- 172.21.15.16:0/1390725129 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1370012e10 con 0x55edba70f780 2024-09-12T02:16:46.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.726+0000 7f13637fe640 1 -- 172.21.15.16:0/1390725129 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1370005290 con 0x55edba70f780 2024-09-12T02:16:46.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.727+0000 7f1381f27740 1 -- 172.21.15.16:0/1390725129 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55edba70f780 msgr2=0x55edba70b290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:46.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.727+0000 7f1381f27740 1 --2- 172.21.15.16:0/1390725129 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55edba70f780 0x55edba70b290 secure :-1 s=READY pgs=2004 cs=0 l=1 rev1=1 crypto rx=0x7f1370013010 tx=0x7f1370009eb0 comp rx=0 tx=0).stop 2024-09-12T02:16:46.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.727+0000 7f1381f27740 1 -- 172.21.15.16:0/1390725129 shutdown_connections 2024-09-12T02:16:46.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.727+0000 7f1381f27740 1 --2- 172.21.15.16:0/1390725129 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55edba70f780 0x55edba70b290 unknown :-1 s=CLOSED pgs=2004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:46.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.727+0000 7f1381f27740 1 --2- 172.21.15.16:0/1390725129 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55edba701b40 0x55edba70ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:46.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.727+0000 7f1381f27740 1 --2- 172.21.15.16:0/1390725129 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55edba7005d0 0x55edba70bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:46.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.727+0000 7f1381f27740 1 -- 172.21.15.16:0/1390725129 >> 172.21.15.16:0/1390725129 conn(0x55edba6c9310 msgr2=0x55edba6ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:46.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.727+0000 7f1381f27740 1 -- 172.21.15.16:0/1390725129 shutdown_connections 2024-09-12T02:16:46.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.727+0000 7f1381f27740 1 -- 172.21.15.16:0/1390725129 wait complete. 2024-09-12T02:16:46.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.727+0000 7f1381f27740 1 -- start start 2024-09-12T02:16:46.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.727+0000 7f1381f27740 1 Processor -- start 2024-09-12T02:16:46.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.728+0000 7f1381f27740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55edba7005d0 0x55edba717cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:46.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.728+0000 7f1381f27740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55edba701b40 0x55edba712fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:46.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.728+0000 7f1381f27740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55edba70f780 0x55edba713520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:46.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.728+0000 7f1381f27740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f13781a9780 con 0x55edba70f780 2024-09-12T02:16:46.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.728+0000 7f1381f27740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f13781aabf0 con 0x55edba7005d0 2024-09-12T02:16:46.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.728+0000 7f1381f27740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f13781a9e50 con 0x55edba701b40 2024-09-12T02:16:46.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.728+0000 7f1381f27740 10 client.0 _ll_register_callbacks cb 0x7f1368004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:16:46.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.728+0000 7f137e0eb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55edba70f780 0x55edba713520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:46.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.728+0000 7f137e0eb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55edba70f780 0x55edba713520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51472/0 (socket says 172.21.15.16:51472) 2024-09-12T02:16:46.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.728+0000 7f137d8ea640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55edba7005d0 0x55edba717cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:46.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.728+0000 7f137e0eb640 1 -- 172.21.15.16:0/954009795 learned_addr learned my addr 172.21.15.16:0/954009795 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:16:46.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.728+0000 7f137d0e9640 1 --2- 172.21.15.16:0/954009795 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55edba701b40 0x55edba712fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:46.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.728+0000 7f137e0eb640 1 -- 172.21.15.16:0/954009795 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55edba701b40 msgr2=0x55edba712fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:46.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.728+0000 7f137e0eb640 1 --2- 172.21.15.16:0/954009795 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55edba701b40 0x55edba712fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:46.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.728+0000 7f137e0eb640 1 -- 172.21.15.16:0/954009795 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55edba7005d0 msgr2=0x55edba717cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:46.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.728+0000 7f137e0eb640 1 --2- 172.21.15.16:0/954009795 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55edba7005d0 0x55edba717cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:46.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.728+0000 7f137e0eb640 1 -- 172.21.15.16:0/954009795 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55edba713a60 con 0x55edba70f780 2024-09-12T02:16:46.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.728+0000 7f137d8ea640 1 --2- 172.21.15.16:0/954009795 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55edba7005d0 0x55edba717cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:16:46.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.728+0000 7f137e0eb640 1 --2- 172.21.15.16:0/954009795 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55edba70f780 0x55edba713520 secure :-1 s=READY pgs=2005 cs=0 l=1 rev1=1 crypto rx=0x7f137000ccc0 tx=0x7f1370009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acfb94c63ddd11dc server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:46.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.728+0000 7f13617fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:16:46.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.728+0000 7f13617fa640 1 -- 172.21.15.16:0/954009795 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1370023040 con 0x55edba70f780 2024-09-12T02:16:46.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.729+0000 7f13617fa640 1 -- 172.21.15.16:0/954009795 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f1370005460 con 0x55edba70f780 2024-09-12T02:16:46.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.729+0000 7f1381f27740 1 -- 172.21.15.16:0/954009795 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1370012e10 con 0x55edba70f780 2024-09-12T02:16:46.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.729+0000 7f13617fa640 1 -- 172.21.15.16:0/954009795 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f13700127a0 con 0x55edba70f780 2024-09-12T02:16:46.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.729+0000 7f1381f27740 10 client.20454 fetch_fsmap learned FSMap version 25 2024-09-12T02:16:46.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.729+0000 7f1381f27740 1 -- 172.21.15.16:0/954009795 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55edba789e90 con 0x55edba70f780 2024-09-12T02:16:46.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.729+0000 7f13617fa640 1 -- 172.21.15.16:0/954009795 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1370012e10 con 0x55edba70f780 2024-09-12T02:16:46.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.729+0000 7f13617fa640 1 -- 172.21.15.16:0/954009795 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f137001f6a0 con 0x55edba70f780 2024-09-12T02:16:46.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.729+0000 7f1381f27740 10 client.20454 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:16:46.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.729+0000 7f1381f27740 10 client.20454 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T02:16:46.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.729+0000 7f1381f27740 20 client.20454 populate_metadata read hostname 'smithi016' 2024-09-12T02:16:46.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.729+0000 7f1381f27740 1 --2- 172.21.15.16:0/954009795 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x55edba78a360 0x55edba78c750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:46.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.729+0000 7f1381f27740 4 client.20454 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T02:16:46.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.729+0000 7f1381f27740 1 -- 172.21.15.16:0/954009795 --> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55edba5ea170 con 0x55edba78a360 2024-09-12T02:16:46.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.729+0000 7f137d8ea640 1 --2- 172.21.15.16:0/954009795 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x55edba78a360 0x55edba78c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:46.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.730+0000 7f13617fa640 10 client.20454 ms_handle_connect on v2:172.21.15.184:6838/4203347941 2024-09-12T02:16:46.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.730+0000 7f137d8ea640 1 --2- 172.21.15.16:0/954009795 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x55edba78a360 0x55edba78c750 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=85fcf6a22d5685cf server_cookie=258bd50e687269db in_seq=0 out_seq=0 2024-09-12T02:16:46.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.730+0000 7f13617fa640 1 -- 172.21.15.16:0/954009795 <== mds.1 v2:172.21.15.184:6838/4203347941 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55edba5ea170 con 0x55edba78a360 2024-09-12T02:16:46.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.730+0000 7f13617fa640 10 client.20454 handle_command_reply: tid=0 2024-09-12T02:16:46.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.730+0000 7f1381f27740 1 client.20454 shutdown 2024-09-12T02:16:46.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.730+0000 7f1381f27740 1 -- 172.21.15.16:0/954009795 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55edba70f780 msgr2=0x55edba713520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:46.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.730+0000 7f1381f27740 1 --2- 172.21.15.16:0/954009795 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55edba70f780 0x55edba713520 secure :-1 s=READY pgs=2005 cs=0 l=1 rev1=1 crypto rx=0x7f137000ccc0 tx=0x7f1370009a20 comp rx=0 tx=0).stop 2024-09-12T02:16:46.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.730+0000 7f1381f27740 1 -- 172.21.15.16:0/954009795 shutdown_connections 2024-09-12T02:16:46.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.730+0000 7f1381f27740 1 --2- 172.21.15.16:0/954009795 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x55edba78a360 0x55edba78c750 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:46.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.730+0000 7f1381f27740 1 --2- 172.21.15.16:0/954009795 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55edba70f780 0x55edba713520 unknown :-1 s=CLOSED pgs=2005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:46.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.730+0000 7f1381f27740 1 --2- 172.21.15.16:0/954009795 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55edba701b40 0x55edba712fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:46.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.730+0000 7f13617fa640 0 client.20454 ms_handle_reset on v2:172.21.15.184:6838/4203347941 2024-09-12T02:16:46.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.730+0000 7f1381f27740 1 --2- 172.21.15.16:0/954009795 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55edba7005d0 0x55edba717cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:46.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.730+0000 7f1381f27740 1 -- 172.21.15.16:0/954009795 >> 172.21.15.16:0/954009795 conn(0x55edba6c9310 msgr2=0x55edba70ac80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:46.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.730+0000 7f1381f27740 1 -- 172.21.15.16:0/954009795 shutdown_connections 2024-09-12T02:16:46.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.730+0000 7f1381f27740 1 -- 172.21.15.16:0/954009795 wait complete. 2024-09-12T02:16:46.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.730+0000 7f1381f27740 20 client.20454 trim_cache size 0 max 16384 2024-09-12T02:16:46.737 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T02:16:46.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.731+0000 7f137fb4e640 1 -- 172.21.15.16:0/4136396725 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f13640a2b80 msgr2=0x7f13640a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:46.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.731+0000 7f137fb4e640 1 --2- 172.21.15.16:0/4136396725 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f13640a2b80 0x7f13640a5040 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f1370005fd0 tx=0x7f1370017000 comp rx=0 tx=0).stop 2024-09-12T02:16:46.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.731+0000 7f137fb4e640 1 -- 172.21.15.16:0/4136396725 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1378100a80 msgr2=0x7f13781983e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:46.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.731+0000 7f137fb4e640 1 --2- 172.21.15.16:0/4136396725 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1378100a80 0x7f13781983e0 secure :-1 s=READY pgs=2002 cs=0 l=1 rev1=1 crypto rx=0x7f1374009230 tx=0x7f1374004270 comp rx=0 tx=0).stop 2024-09-12T02:16:46.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.731+0000 7f137fb4e640 1 -- 172.21.15.16:0/4136396725 shutdown_connections 2024-09-12T02:16:46.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.731+0000 7f137fb4e640 1 --2- 172.21.15.16:0/4136396725 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f13640a2b80 0x7f13640a5040 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:46.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.731+0000 7f137fb4e640 1 --2- 172.21.15.16:0/4136396725 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1378106a80 0x7f137819ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:46.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.731+0000 7f137fb4e640 1 --2- 172.21.15.16:0/4136396725 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1378101420 0x7f1378198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:46.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.731+0000 7f137fb4e640 1 --2- 172.21.15.16:0/4136396725 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1378100a80 0x7f13781983e0 unknown :-1 s=CLOSED pgs=2002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:46.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.731+0000 7f137fb4e640 1 -- 172.21.15.16:0/4136396725 >> 172.21.15.16:0/4136396725 conn(0x7f13780fc7a0 msgr2=0x7f13780fcf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:46.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.731+0000 7f137fb4e640 1 -- 172.21.15.16:0/4136396725 shutdown_connections 2024-09-12T02:16:46.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:46.731+0000 7f137fb4e640 1 -- 172.21.15.16:0/4136396725 wait complete. 2024-09-12T02:16:46.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:16:46 smithi184 ceph-mon[35593]: pgmap v780: 97 pgs: 97 active+clean; 3.5 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-09-12T02:16:47.524 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-12T02:16:47.787 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:16:48.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.241+0000 7f87a44ef640 1 -- 172.21.15.16:0/195449542 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f879c102570 msgr2=0x7f879c1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:48.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.241+0000 7f87a44ef640 1 --2- 172.21.15.16:0/195449542 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f879c102570 0x7f879c1029b0 secure :-1 s=READY pgs=2006 cs=0 l=1 rev1=1 crypto rx=0x7f878c009900 tx=0x7f878c02a480 comp rx=0 tx=0).stop 2024-09-12T02:16:48.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.242+0000 7f87a44ef640 1 -- 172.21.15.16:0/195449542 shutdown_connections 2024-09-12T02:16:48.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.242+0000 7f87a44ef640 1 --2- 172.21.15.16:0/195449542 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f879c102ef0 0x7f879c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:48.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.242+0000 7f87a44ef640 1 --2- 172.21.15.16:0/195449542 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f879c102570 0x7f879c1029b0 unknown :-1 s=CLOSED pgs=2006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:48.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.242+0000 7f87a44ef640 1 --2- 172.21.15.16:0/195449542 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f879c100e20 0x7f879c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:48.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.242+0000 7f87a44ef640 1 -- 172.21.15.16:0/195449542 >> 172.21.15.16:0/195449542 conn(0x7f879c0fc7d0 msgr2=0x7f879c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:48.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.242+0000 7f87a44ef640 1 -- 172.21.15.16:0/195449542 shutdown_connections 2024-09-12T02:16:48.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.242+0000 7f87a44ef640 1 -- 172.21.15.16:0/195449542 wait complete. 2024-09-12T02:16:48.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.242+0000 7f87a44ef640 1 Processor -- start 2024-09-12T02:16:48.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.242+0000 7f87a44ef640 1 -- start start 2024-09-12T02:16:48.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.243+0000 7f87a44ef640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f879c100e20 0x7f879c0fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:48.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.243+0000 7f87a44ef640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f879c102570 0x7f879c0fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:48.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.243+0000 7f87a44ef640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f879c102ef0 0x7f879c0fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:48.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.243+0000 7f87a44ef640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f879c110f80 con 0x7f879c102ef0 2024-09-12T02:16:48.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.243+0000 7f87a44ef640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f879c110e00 con 0x7f879c100e20 2024-09-12T02:16:48.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.243+0000 7f87a44ef640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f879c111100 con 0x7f879c102570 2024-09-12T02:16:48.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.243+0000 7f87a2a8c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f879c102ef0 0x7f879c0fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:48.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.243+0000 7f87a2a8c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f879c102ef0 0x7f879c0fd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51484/0 (socket says 172.21.15.16:51484) 2024-09-12T02:16:48.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.243+0000 7f87a2a8c640 1 -- 172.21.15.16:0/2862407782 learned_addr learned my addr 172.21.15.16:0/2862407782 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:16:48.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.243+0000 7f87a1a8a640 1 --2- 172.21.15.16:0/2862407782 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f879c102570 0x7f879c0fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:48.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.243+0000 7f87a228b640 1 --2- 172.21.15.16:0/2862407782 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f879c100e20 0x7f879c0fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:48.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.243+0000 7f87a2a8c640 1 -- 172.21.15.16:0/2862407782 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f879c102570 msgr2=0x7f879c0fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:48.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.243+0000 7f87a2a8c640 1 --2- 172.21.15.16:0/2862407782 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f879c102570 0x7f879c0fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:48.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.243+0000 7f87a2a8c640 1 -- 172.21.15.16:0/2862407782 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f879c100e20 msgr2=0x7f879c0fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:48.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.243+0000 7f87a2a8c640 1 --2- 172.21.15.16:0/2862407782 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f879c100e20 0x7f879c0fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:48.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.243+0000 7f87a2a8c640 1 -- 172.21.15.16:0/2862407782 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f879c1a6910 con 0x7f879c102ef0 2024-09-12T02:16:48.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.243+0000 7f87a2a8c640 1 --2- 172.21.15.16:0/2862407782 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f879c102ef0 0x7f879c0fd700 secure :-1 s=READY pgs=2007 cs=0 l=1 rev1=1 crypto rx=0x7f879400cce0 tx=0x7f879400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:48.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.243+0000 7f87937fe640 1 -- 172.21.15.16:0/2862407782 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f879400f450 con 0x7f879c102ef0 2024-09-12T02:16:48.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.243+0000 7f87a44ef640 1 -- 172.21.15.16:0/2862407782 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f879c1a6c00 con 0x7f879c102ef0 2024-09-12T02:16:48.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.243+0000 7f87937fe640 1 -- 172.21.15.16:0/2862407782 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f879400fad0 con 0x7f879c102ef0 2024-09-12T02:16:48.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.243+0000 7f87937fe640 1 -- 172.21.15.16:0/2862407782 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8794018c10 con 0x7f879c102ef0 2024-09-12T02:16:48.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.244+0000 7f87a44ef640 1 -- 172.21.15.16:0/2862407782 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f879c1a7110 con 0x7f879c102ef0 2024-09-12T02:16:48.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.246+0000 7f87937fe640 1 -- 172.21.15.16:0/2862407782 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f879400f5f0 con 0x7f879c102ef0 2024-09-12T02:16:48.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.246+0000 7f87a68c8740 1 Processor -- start 2024-09-12T02:16:48.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.246+0000 7f87a68c8740 1 -- start start 2024-09-12T02:16:48.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.246+0000 7f87a68c8740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d098632b40 0x55d098631090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:48.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.246+0000 7f87a68c8740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d0986315d0 0x55d098631a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:48.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.246+0000 7f87a68c8740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d09863c370 0x55d09863e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:48.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.246+0000 7f87a68c8740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55d0985972f0 con 0x55d09863c370 2024-09-12T02:16:48.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.246+0000 7f87a68c8740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55d09853ecf0 con 0x55d098632b40 2024-09-12T02:16:48.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.246+0000 7f87a68c8740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55d0985932b0 con 0x55d0986315d0 2024-09-12T02:16:48.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.247+0000 7f87a1a8a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d09863c370 0x55d09863e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:48.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.247+0000 7f87a1a8a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d09863c370 0x55d09863e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51490/0 (socket says 172.21.15.16:51490) 2024-09-12T02:16:48.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.247+0000 7f87a1a8a640 1 -- 172.21.15.16:0/3754962874 learned_addr learned my addr 172.21.15.16:0/3754962874 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:16:48.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.247+0000 7f87937fe640 1 --2- 172.21.15.16:0/2862407782 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f87880a2b80 0x7f87880a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:48.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.247+0000 7f87a228b640 1 --2- 172.21.15.16:0/3754962874 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d0986315d0 0x55d098631a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:48.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.247+0000 7f87a1a8a640 1 --2- 172.21.15.16:0/3754962874 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d098632b40 0x55d098631090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:48.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.247+0000 7f87a2a8c640 1 --2- 172.21.15.16:0/2862407782 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f87880a2b80 0x7f87880a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:48.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.247+0000 7f87a1a8a640 1 -- 172.21.15.16:0/3754962874 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d0986315d0 msgr2=0x55d098631a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:48.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.247+0000 7f87a1a8a640 1 --2- 172.21.15.16:0/3754962874 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d0986315d0 0x55d098631a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:48.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.247+0000 7f87a1a8a640 1 -- 172.21.15.16:0/3754962874 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d098632b40 msgr2=0x55d098631090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:48.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.247+0000 7f87a1a8a640 1 --2- 172.21.15.16:0/3754962874 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d098632b40 0x55d098631090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:48.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.247+0000 7f87a1a8a640 1 -- 172.21.15.16:0/3754962874 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0985233a0 con 0x55d09863c370 2024-09-12T02:16:48.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.247+0000 7f87937fe640 1 -- 172.21.15.16:0/2862407782 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f87940cf620 con 0x7f879c102ef0 2024-09-12T02:16:48.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.247+0000 7f87a1a8a640 1 --2- 172.21.15.16:0/3754962874 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d09863c370 0x55d09863e760 secure :-1 s=READY pgs=2008 cs=0 l=1 rev1=1 crypto rx=0x7f878c009900 tx=0x7f878c005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4409404be69484b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:48.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.247+0000 7f87a2a8c640 1 --2- 172.21.15.16:0/2862407782 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f87880a2b80 0x7f87880a5040 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f87940137f0 tx=0x7f8794066f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:48.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.247+0000 7f8790ff9640 1 -- 172.21.15.16:0/3754962874 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f878c035040 con 0x55d09863c370 2024-09-12T02:16:48.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.248+0000 7f8790ff9640 1 -- 172.21.15.16:0/3754962874 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f878c003d50 con 0x55d09863c370 2024-09-12T02:16:48.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.248+0000 7f8790ff9640 1 -- 172.21.15.16:0/3754962874 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f878c03d480 con 0x55d09863c370 2024-09-12T02:16:48.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.248+0000 7f87a68c8740 1 -- 172.21.15.16:0/3754962874 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d09863c370 msgr2=0x55d09863e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:48.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.248+0000 7f87a68c8740 1 --2- 172.21.15.16:0/3754962874 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d09863c370 0x55d09863e760 secure :-1 s=READY pgs=2008 cs=0 l=1 rev1=1 crypto rx=0x7f878c009900 tx=0x7f878c005c00 comp rx=0 tx=0).stop 2024-09-12T02:16:48.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.248+0000 7f87a68c8740 1 -- 172.21.15.16:0/3754962874 shutdown_connections 2024-09-12T02:16:48.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.248+0000 7f87a68c8740 1 --2- 172.21.15.16:0/3754962874 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d09863c370 0x55d09863e760 unknown :-1 s=CLOSED pgs=2008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:48.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.248+0000 7f87a68c8740 1 --2- 172.21.15.16:0/3754962874 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d0986315d0 0x55d098631a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:48.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.248+0000 7f87a68c8740 1 --2- 172.21.15.16:0/3754962874 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d098632b40 0x55d098631090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:48.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.248+0000 7f87a68c8740 1 -- 172.21.15.16:0/3754962874 >> 172.21.15.16:0/3754962874 conn(0x55d0985fa310 msgr2=0x55d09862dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:48.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.248+0000 7f87a68c8740 1 -- 172.21.15.16:0/3754962874 shutdown_connections 2024-09-12T02:16:48.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.248+0000 7f87a68c8740 1 -- 172.21.15.16:0/3754962874 wait complete. 2024-09-12T02:16:48.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.249+0000 7f87a68c8740 1 -- start start 2024-09-12T02:16:48.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.249+0000 7f87a68c8740 1 Processor -- start 2024-09-12T02:16:48.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.249+0000 7f87a68c8740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d0986315d0 0x55d09863bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:48.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.249+0000 7f87a68c8740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d098632b40 0x55d098638fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:48.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.249+0000 7f87a68c8740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d09863c370 0x55d0986395d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:48.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.249+0000 7f87a68c8740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55d09863f010 con 0x55d098632b40 2024-09-12T02:16:48.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.249+0000 7f87a68c8740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55d098552060 con 0x55d09863c370 2024-09-12T02:16:48.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.249+0000 7f87a68c8740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55d09863f190 con 0x55d0986315d0 2024-09-12T02:16:48.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.249+0000 7f87a68c8740 10 client.0 _ll_register_callbacks cb 0x55d0982a58c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:16:48.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.249+0000 7f87a228b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d098632b40 0x55d098638fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:48.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.249+0000 7f87a228b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d098632b40 0x55d098638fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51500/0 (socket says 172.21.15.16:51500) 2024-09-12T02:16:48.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.249+0000 7f87a228b640 1 -- 172.21.15.16:0/1563254957 learned_addr learned my addr 172.21.15.16:0/1563254957 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:16:48.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.249+0000 7f87a1a8a640 1 --2- 172.21.15.16:0/1563254957 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d0986315d0 0x55d09863bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:48.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.249+0000 7f87a1a8a640 1 --2- 172.21.15.16:0/1563254957 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d09863c370 0x55d0986395d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:48.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.249+0000 7f87a228b640 1 -- 172.21.15.16:0/1563254957 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d0986315d0 msgr2=0x55d09863bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:48.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.249+0000 7f87a228b640 1 --2- 172.21.15.16:0/1563254957 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d0986315d0 0x55d09863bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:48.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.249+0000 7f87a228b640 1 -- 172.21.15.16:0/1563254957 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d09863c370 msgr2=0x55d0986395d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:48.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.249+0000 7f87a228b640 1 --2- 172.21.15.16:0/1563254957 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d09863c370 0x55d0986395d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:48.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.249+0000 7f87a228b640 1 -- 172.21.15.16:0/1563254957 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0983083c0 con 0x55d098632b40 2024-09-12T02:16:48.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.250+0000 7f87a1a8a640 1 --2- 172.21.15.16:0/1563254957 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d0986315d0 0x55d09863bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:16:48.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.250+0000 7f87a228b640 1 --2- 172.21.15.16:0/1563254957 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d098632b40 0x55d098638fe0 secure :-1 s=READY pgs=2009 cs=0 l=1 rev1=1 crypto rx=0x7f8798010810 tx=0x7f8798010cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bac6524e7974c2f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:48.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.250+0000 7f8785ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:16:48.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.250+0000 7f8785ffb640 1 -- 172.21.15.16:0/1563254957 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f879800ed40 con 0x55d098632b40 2024-09-12T02:16:48.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.250+0000 7f8785ffb640 1 -- 172.21.15.16:0/1563254957 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f879801c480 con 0x55d098632b40 2024-09-12T02:16:48.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.250+0000 7f87a68c8740 1 -- 172.21.15.16:0/1563254957 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f878c003d50 con 0x55d098632b40 2024-09-12T02:16:48.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.250+0000 7f8785ffb640 1 -- 172.21.15.16:0/1563254957 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f879801c680 con 0x55d098632b40 2024-09-12T02:16:48.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.250+0000 7f87a68c8740 10 client.20466 fetch_fsmap learned FSMap version 25 2024-09-12T02:16:48.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.250+0000 7f87a68c8740 1 -- 172.21.15.16:0/1563254957 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d09864ab00 con 0x55d098632b40 2024-09-12T02:16:48.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.250+0000 7f8785ffb640 1 -- 172.21.15.16:0/1563254957 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f878c003d50 con 0x55d098632b40 2024-09-12T02:16:48.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.250+0000 7f8785ffb640 1 -- 172.21.15.16:0/1563254957 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f8798011890 con 0x55d098632b40 2024-09-12T02:16:48.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.251+0000 7f87a68c8740 10 client.20466 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:16:48.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.251+0000 7f87a68c8740 10 client.20466 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:16:48.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.251+0000 7f87a68c8740 20 client.20466 populate_metadata read hostname 'smithi016' 2024-09-12T02:16:48.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.251+0000 7f87a68c8740 1 --2- 172.21.15.16:0/1563254957 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55d09864bfc0 0x55d09864e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:48.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.251+0000 7f87a68c8740 4 client.20466 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:16:48.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.251+0000 7f87a68c8740 1 -- 172.21.15.16:0/1563254957 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d0985a4860 con 0x55d09864bfc0 2024-09-12T02:16:48.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.251+0000 7f87a1a8a640 1 --2- 172.21.15.16:0/1563254957 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55d09864bfc0 0x55d09864e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:48.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.252+0000 7f87a1a8a640 1 --2- 172.21.15.16:0/1563254957 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55d09864bfc0 0x55d09864e3b0 crc :-1 s=READY pgs=443 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ddaa50503a5a2b8 server_cookie=aed77105491f3ca0 in_seq=0 out_seq=0 2024-09-12T02:16:48.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.252+0000 7f8785ffb640 10 client.20466 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:16:48.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.252+0000 7f8785ffb640 1 -- 172.21.15.16:0/1563254957 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d0985a4860 con 0x55d09864bfc0 2024-09-12T02:16:48.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.252+0000 7f8785ffb640 10 client.20466 handle_command_reply: tid=0 2024-09-12T02:16:48.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.252+0000 7f87a68c8740 1 client.20466 shutdown 2024-09-12T02:16:48.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.253+0000 7f87a68c8740 1 -- 172.21.15.16:0/1563254957 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d098632b40 msgr2=0x55d098638fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:48.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.253+0000 7f87a68c8740 1 --2- 172.21.15.16:0/1563254957 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d098632b40 0x55d098638fe0 secure :-1 s=READY pgs=2009 cs=0 l=1 rev1=1 crypto rx=0x7f8798010810 tx=0x7f8798010cd0 comp rx=0 tx=0).stop 2024-09-12T02:16:48.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.253+0000 7f87a68c8740 1 -- 172.21.15.16:0/1563254957 shutdown_connections 2024-09-12T02:16:48.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.253+0000 7f87a68c8740 1 --2- 172.21.15.16:0/1563254957 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55d09864bfc0 0x55d09864e3b0 crc :-1 s=READY pgs=443 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:48.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.253+0000 7f87a68c8740 1 --2- 172.21.15.16:0/1563254957 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d09863c370 0x55d0986395d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:48.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.253+0000 7f87a68c8740 1 --2- 172.21.15.16:0/1563254957 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d098632b40 0x55d098638fe0 unknown :-1 s=CLOSED pgs=2009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:48.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.253+0000 7f8785ffb640 0 client.20466 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:16:48.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.253+0000 7f87a68c8740 1 --2- 172.21.15.16:0/1563254957 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d0986315d0 0x55d09863bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:48.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.253+0000 7f87a68c8740 1 -- 172.21.15.16:0/1563254957 >> 172.21.15.16:0/1563254957 conn(0x55d0985fa310 msgr2=0x55d098633960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:48.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.253+0000 7f87a68c8740 1 -- 172.21.15.16:0/1563254957 shutdown_connections 2024-09-12T02:16:48.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.253+0000 7f87a68c8740 1 -- 172.21.15.16:0/1563254957 wait complete. 2024-09-12T02:16:48.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.253+0000 7f87a68c8740 20 client.20466 trim_cache size 0 max 16384 2024-09-12T02:16:48.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.266+0000 7f87a68c8740 1 Processor -- start 2024-09-12T02:16:48.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.266+0000 7f87a68c8740 1 -- start start 2024-09-12T02:16:48.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.266+0000 7f87a68c8740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d0986315d0 0x55d0986482e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:48.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.266+0000 7f87a68c8740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d098632b40 0x55d098628ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:48.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.266+0000 7f87a68c8740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d09863c370 0x55d098637fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:48.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.266+0000 7f87a68c8740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f879c1b1da0 con 0x55d0986315d0 2024-09-12T02:16:48.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.266+0000 7f87a68c8740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f879c1af670 con 0x55d09863c370 2024-09-12T02:16:48.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.266+0000 7f87a68c8740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f879c1ad500 con 0x55d098632b40 2024-09-12T02:16:48.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.266+0000 7f87a1a8a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d0986315d0 0x55d0986482e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:48.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.266+0000 7f87a1a8a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d0986315d0 0x55d0986482e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51514/0 (socket says 172.21.15.16:51514) 2024-09-12T02:16:48.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.266+0000 7f87a1a8a640 1 -- 172.21.15.16:0/312049292 learned_addr learned my addr 172.21.15.16:0/312049292 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:16:48.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.266+0000 7f87a228b640 1 --2- 172.21.15.16:0/312049292 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d098632b40 0x55d098628ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:48.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.267+0000 7f87a1a8a640 1 -- 172.21.15.16:0/312049292 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d098632b40 msgr2=0x55d098628ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:48.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.267+0000 7f87a1a8a640 1 --2- 172.21.15.16:0/312049292 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d098632b40 0x55d098628ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:48.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.267+0000 7f87a1a8a640 1 -- 172.21.15.16:0/312049292 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d09863c370 msgr2=0x55d098637fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:48.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.267+0000 7f87a1a8a640 1 --2- 172.21.15.16:0/312049292 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d09863c370 0x55d098637fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:48.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.267+0000 7f87a1a8a640 1 -- 172.21.15.16:0/312049292 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d09862dde0 con 0x55d0986315d0 2024-09-12T02:16:48.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.267+0000 7f87a1a8a640 1 --2- 172.21.15.16:0/312049292 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d0986315d0 0x55d0986482e0 secure :-1 s=READY pgs=2010 cs=0 l=1 rev1=1 crypto rx=0x7f878c009900 tx=0x7f878c031b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=420208a99f0687cd server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:48.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.267+0000 7f8787fff640 1 -- 172.21.15.16:0/312049292 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f878c035040 con 0x55d0986315d0 2024-09-12T02:16:48.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.267+0000 7f8787fff640 1 -- 172.21.15.16:0/312049292 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f878c005500 con 0x55d0986315d0 2024-09-12T02:16:48.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.267+0000 7f8787fff640 1 -- 172.21.15.16:0/312049292 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f878c0049e0 con 0x55d0986315d0 2024-09-12T02:16:48.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.268+0000 7f87a68c8740 1 -- 172.21.15.16:0/312049292 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d0986315d0 msgr2=0x55d0986482e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:48.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.268+0000 7f87a68c8740 1 --2- 172.21.15.16:0/312049292 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d0986315d0 0x55d0986482e0 secure :-1 s=READY pgs=2010 cs=0 l=1 rev1=1 crypto rx=0x7f878c009900 tx=0x7f878c031b70 comp rx=0 tx=0).stop 2024-09-12T02:16:48.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.268+0000 7f87a68c8740 1 -- 172.21.15.16:0/312049292 shutdown_connections 2024-09-12T02:16:48.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.268+0000 7f87a68c8740 1 --2- 172.21.15.16:0/312049292 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d09863c370 0x55d098637fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:48.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.268+0000 7f87a68c8740 1 --2- 172.21.15.16:0/312049292 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d098632b40 0x55d098628ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:48.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.268+0000 7f87a68c8740 1 --2- 172.21.15.16:0/312049292 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d0986315d0 0x55d0986482e0 unknown :-1 s=CLOSED pgs=2010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:48.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.268+0000 7f87a68c8740 1 -- 172.21.15.16:0/312049292 >> 172.21.15.16:0/312049292 conn(0x55d0985fa310 msgr2=0x55d0986378d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:48.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.268+0000 7f87a68c8740 1 -- 172.21.15.16:0/312049292 shutdown_connections 2024-09-12T02:16:48.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.268+0000 7f87a68c8740 1 -- 172.21.15.16:0/312049292 wait complete. 2024-09-12T02:16:48.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.268+0000 7f87a68c8740 1 -- start start 2024-09-12T02:16:48.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.268+0000 7f87a68c8740 1 Processor -- start 2024-09-12T02:16:48.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.268+0000 7f87a68c8740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d0986315d0 0x55d098644450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:48.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.268+0000 7f87a68c8740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d098632b40 0x55d098644990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:48.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.268+0000 7f87a68c8740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d09863c370 0x55d098635630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:48.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.268+0000 7f87a68c8740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f879c1adb50 con 0x55d0986315d0 2024-09-12T02:16:48.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.268+0000 7f87a68c8740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f879c1aefa0 con 0x55d098632b40 2024-09-12T02:16:48.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.268+0000 7f87a68c8740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f879c1ae200 con 0x55d09863c370 2024-09-12T02:16:48.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.268+0000 7f87a68c8740 10 client.0 _ll_register_callbacks cb 0x7f879800c450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:16:48.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.268+0000 7f87a1a8a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d0986315d0 0x55d098644450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:48.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.268+0000 7f87a1a8a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d0986315d0 0x55d098644450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51516/0 (socket says 172.21.15.16:51516) 2024-09-12T02:16:48.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.268+0000 7f87a1a8a640 1 -- 172.21.15.16:0/1435820932 learned_addr learned my addr 172.21.15.16:0/1435820932 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:16:48.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.269+0000 7f87a228b640 1 --2- 172.21.15.16:0/1435820932 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d098632b40 0x55d098644990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:48.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.269+0000 7f87a1a8a640 1 --2- 172.21.15.16:0/1435820932 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d09863c370 0x55d098635630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:48.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.269+0000 7f87a1a8a640 1 -- 172.21.15.16:0/1435820932 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d09863c370 msgr2=0x55d098635630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:48.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.269+0000 7f87a1a8a640 1 --2- 172.21.15.16:0/1435820932 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d09863c370 0x55d098635630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:48.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.269+0000 7f87a1a8a640 1 -- 172.21.15.16:0/1435820932 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d098632b40 msgr2=0x55d098644990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:48.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.269+0000 7f87a1a8a640 1 --2- 172.21.15.16:0/1435820932 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d098632b40 0x55d098644990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:48.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.269+0000 7f87a1a8a640 1 -- 172.21.15.16:0/1435820932 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d098635c20 con 0x55d0986315d0 2024-09-12T02:16:48.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.269+0000 7f87a1a8a640 1 --2- 172.21.15.16:0/1435820932 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d0986315d0 0x55d098644450 secure :-1 s=READY pgs=2011 cs=0 l=1 rev1=1 crypto rx=0x7f878c031480 tx=0x7f878c004540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5171cbc9c62ea088 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:48.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.269+0000 7f8785ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:16:48.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.269+0000 7f8785ffb640 1 -- 172.21.15.16:0/1435820932 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f878c035070 con 0x55d0986315d0 2024-09-12T02:16:48.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.269+0000 7f87a68c8740 1 -- 172.21.15.16:0/1435820932 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f878c005500 con 0x55d0986315d0 2024-09-12T02:16:48.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.269+0000 7f8785ffb640 1 -- 172.21.15.16:0/1435820932 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f878c031540 con 0x55d0986315d0 2024-09-12T02:16:48.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.269+0000 7f8785ffb640 1 -- 172.21.15.16:0/1435820932 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f878c003670 con 0x55d0986315d0 2024-09-12T02:16:48.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.269+0000 7f87a68c8740 10 client.20472 fetch_fsmap learned FSMap version 25 2024-09-12T02:16:48.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.269+0000 7f87a68c8740 1 -- 172.21.15.16:0/1435820932 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d0986b6510 con 0x55d0986315d0 2024-09-12T02:16:48.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.269+0000 7f8785ffb640 1 -- 172.21.15.16:0/1435820932 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f878c005500 con 0x55d0986315d0 2024-09-12T02:16:48.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.270+0000 7f8785ffb640 1 -- 172.21.15.16:0/1435820932 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f878c03d9e0 con 0x55d0986315d0 2024-09-12T02:16:48.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.270+0000 7f87a68c8740 10 client.20472 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:16:48.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.270+0000 7f87a68c8740 10 client.20472 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:16:48.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.270+0000 7f87a68c8740 20 client.20472 populate_metadata read hostname 'smithi016' 2024-09-12T02:16:48.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.270+0000 7f87a68c8740 1 --2- 172.21.15.16:0/1435820932 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55d0986b66e0 0x55d0986b8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:48.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.270+0000 7f87a68c8740 4 client.20472 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T02:16:48.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.270+0000 7f87a68c8740 1 -- 172.21.15.16:0/1435820932 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d09859ed70 con 0x55d0986b66e0 2024-09-12T02:16:48.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.270+0000 7f87a228b640 1 --2- 172.21.15.16:0/1435820932 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55d0986b66e0 0x55d0986b8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:48.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.270+0000 7f8785ffb640 10 client.20472 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:16:48.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.270+0000 7f87a228b640 1 --2- 172.21.15.16:0/1435820932 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55d0986b66e0 0x55d0986b8ad0 crc :-1 s=READY pgs=444 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a53449d982ea5c05 server_cookie=56782bf07d3a2450 in_seq=0 out_seq=0 2024-09-12T02:16:48.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.271+0000 7f8785ffb640 1 -- 172.21.15.16:0/1435820932 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d09859ed70 con 0x55d0986b66e0 2024-09-12T02:16:48.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.271+0000 7f8785ffb640 10 client.20472 handle_command_reply: tid=0 2024-09-12T02:16:48.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.271+0000 7f87a68c8740 1 client.20472 shutdown 2024-09-12T02:16:48.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.271+0000 7f87a68c8740 1 -- 172.21.15.16:0/1435820932 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d0986315d0 msgr2=0x55d098644450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:48.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.271+0000 7f87a68c8740 1 --2- 172.21.15.16:0/1435820932 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d0986315d0 0x55d098644450 secure :-1 s=READY pgs=2011 cs=0 l=1 rev1=1 crypto rx=0x7f878c031480 tx=0x7f878c004540 comp rx=0 tx=0).stop 2024-09-12T02:16:48.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.271+0000 7f87a68c8740 1 -- 172.21.15.16:0/1435820932 shutdown_connections 2024-09-12T02:16:48.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.271+0000 7f87a68c8740 1 --2- 172.21.15.16:0/1435820932 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55d0986b66e0 0x55d0986b8ad0 crc :-1 s=READY pgs=444 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:48.313 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T02:16:48.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.271+0000 7f87a68c8740 1 --2- 172.21.15.16:0/1435820932 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d09863c370 0x55d098635630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:48.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.271+0000 7f87a68c8740 1 --2- 172.21.15.16:0/1435820932 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d098632b40 0x55d098644990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:48.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.271+0000 7f8785ffb640 0 client.20472 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:16:48.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.271+0000 7f87a68c8740 1 --2- 172.21.15.16:0/1435820932 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d0986315d0 0x55d098644450 unknown :-1 s=CLOSED pgs=2011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:48.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.271+0000 7f87a68c8740 1 -- 172.21.15.16:0/1435820932 >> 172.21.15.16:0/1435820932 conn(0x55d0985fa310 msgr2=0x55d0986380f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:48.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.271+0000 7f87a68c8740 1 -- 172.21.15.16:0/1435820932 shutdown_connections 2024-09-12T02:16:48.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.271+0000 7f87a68c8740 1 -- 172.21.15.16:0/1435820932 wait complete. 2024-09-12T02:16:48.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.271+0000 7f87a68c8740 20 client.20472 trim_cache size 0 max 16384 2024-09-12T02:16:48.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.271+0000 7f87a44ef640 1 -- 172.21.15.16:0/2862407782 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f87880a2b80 msgr2=0x7f87880a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:48.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.271+0000 7f87a44ef640 1 --2- 172.21.15.16:0/2862407782 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f87880a2b80 0x7f87880a5040 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f87940137f0 tx=0x7f8794066f90 comp rx=0 tx=0).stop 2024-09-12T02:16:48.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.271+0000 7f87a44ef640 1 -- 172.21.15.16:0/2862407782 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f879c102ef0 msgr2=0x7f879c0fd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:48.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.271+0000 7f87a44ef640 1 --2- 172.21.15.16:0/2862407782 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f879c102ef0 0x7f879c0fd700 secure :-1 s=READY pgs=2007 cs=0 l=1 rev1=1 crypto rx=0x7f879400cce0 tx=0x7f879400a620 comp rx=0 tx=0).stop 2024-09-12T02:16:48.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.271+0000 7f87a44ef640 1 -- 172.21.15.16:0/2862407782 shutdown_connections 2024-09-12T02:16:48.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.271+0000 7f87a44ef640 1 --2- 172.21.15.16:0/2862407782 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f87880a2b80 0x7f87880a5040 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:48.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.271+0000 7f87a44ef640 1 --2- 172.21.15.16:0/2862407782 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f879c102ef0 0x7f879c0fd700 unknown :-1 s=CLOSED pgs=2007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:48.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.271+0000 7f87a44ef640 1 --2- 172.21.15.16:0/2862407782 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f879c102570 0x7f879c0fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:48.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.271+0000 7f87a44ef640 1 --2- 172.21.15.16:0/2862407782 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f879c100e20 0x7f879c0fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:48.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.271+0000 7f87a44ef640 1 -- 172.21.15.16:0/2862407782 >> 172.21.15.16:0/2862407782 conn(0x7f879c0fc7d0 msgr2=0x7f879c06a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:48.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.272+0000 7f87a44ef640 1 -- 172.21.15.16:0/2862407782 shutdown_connections 2024-09-12T02:16:48.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:48.272+0000 7f87a44ef640 1 -- 172.21.15.16:0/2862407782 wait complete. 2024-09-12T02:16:48.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:16:48 smithi016 ceph-mon[30947]: pgmap v781: 97 pgs: 97 active+clean; 3.6 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-12T02:16:48.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:16:48 smithi184 ceph-mon[35593]: pgmap v781: 97 pgs: 97 active+clean; 3.6 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-12T02:16:48.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:16:48 smithi033 ceph-mon[35332]: pgmap v781: 97 pgs: 97 active+clean; 3.6 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-12T02:16:50.095 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-12T02:16:50.096 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-12T02:16:50.368 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:16:50.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:16:50 smithi016 ceph-mon[30947]: pgmap v782: 97 pgs: 97 active+clean; 3.6 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-09-12T02:16:50.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.852+0000 7ff28f749640 1 -- 172.21.15.16:0/1181724633 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff28810c170 msgr2=0x7ff28810e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:50.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.852+0000 7ff28f749640 1 --2- 172.21.15.16:0/1181724633 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff28810c170 0x7ff28810e610 secure :-1 s=READY pgs=2012 cs=0 l=1 rev1=1 crypto rx=0x7ff280009ca0 tx=0x7ff28002a950 comp rx=0 tx=0).stop 2024-09-12T02:16:50.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.852+0000 7ff28f749640 1 -- 172.21.15.16:0/1181724633 shutdown_connections 2024-09-12T02:16:50.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.852+0000 7ff28f749640 1 --2- 172.21.15.16:0/1181724633 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff28810c170 0x7ff28810e610 unknown :-1 s=CLOSED pgs=2012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:50.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.852+0000 7ff28f749640 1 --2- 172.21.15.16:0/1181724633 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff288103660 0x7ff28810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:50.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.852+0000 7ff28f749640 1 --2- 172.21.15.16:0/1181724633 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff288102d40 0x7ff288103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:50.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.852+0000 7ff28f749640 1 -- 172.21.15.16:0/1181724633 >> 172.21.15.16:0/1181724633 conn(0x7ff2880fc7a0 msgr2=0x7ff2880febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:50.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.852+0000 7ff28f749640 1 -- 172.21.15.16:0/1181724633 shutdown_connections 2024-09-12T02:16:50.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.852+0000 7ff28f749640 1 -- 172.21.15.16:0/1181724633 wait complete. 2024-09-12T02:16:50.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.853+0000 7ff28f749640 1 Processor -- start 2024-09-12T02:16:50.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.853+0000 7ff28f749640 1 -- start start 2024-09-12T02:16:50.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.853+0000 7ff28f749640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff288102d40 0x7ff2881009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:50.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.853+0000 7ff28f749640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff288103660 0x7ff288100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:50.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.853+0000 7ff28f749640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff28810c170 0x7ff288101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:50.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.853+0000 7ff28f749640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff288111f70 con 0x7ff288103660 2024-09-12T02:16:50.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.853+0000 7ff28f749640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff288111df0 con 0x7ff28810c170 2024-09-12T02:16:50.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.853+0000 7ff28f749640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff2881120f0 con 0x7ff288102d40 2024-09-12T02:16:50.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.853+0000 7ff28cce4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff288103660 0x7ff288100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:50.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.853+0000 7ff28cce4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff288103660 0x7ff288100ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51548/0 (socket says 172.21.15.16:51548) 2024-09-12T02:16:50.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.853+0000 7ff28cce4640 1 -- 172.21.15.16:0/1229546726 learned_addr learned my addr 172.21.15.16:0/1229546726 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:16:50.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.853+0000 7ff28dce6640 1 --2- 172.21.15.16:0/1229546726 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff28810c170 0x7ff288101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:50.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.853+0000 7ff28d4e5640 1 --2- 172.21.15.16:0/1229546726 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff288102d40 0x7ff2881009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:50.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.853+0000 7ff28cce4640 1 -- 172.21.15.16:0/1229546726 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff288102d40 msgr2=0x7ff2881009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:50.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.853+0000 7ff28cce4640 1 --2- 172.21.15.16:0/1229546726 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff288102d40 0x7ff2881009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:50.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.853+0000 7ff28cce4640 1 -- 172.21.15.16:0/1229546726 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff28810c170 msgr2=0x7ff288101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:50.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.853+0000 7ff28cce4640 1 --2- 172.21.15.16:0/1229546726 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff28810c170 0x7ff288101420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:50.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.853+0000 7ff28cce4640 1 -- 172.21.15.16:0/1229546726 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2881a68d0 con 0x7ff288103660 2024-09-12T02:16:50.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.853+0000 7ff28d4e5640 1 --2- 172.21.15.16:0/1229546726 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff288102d40 0x7ff2881009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:16:50.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.854+0000 7ff28cce4640 1 --2- 172.21.15.16:0/1229546726 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff288103660 0x7ff288100ee0 secure :-1 s=READY pgs=2013 cs=0 l=1 rev1=1 crypto rx=0x7ff27800c940 tx=0x7ff27800ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:50.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.854+0000 7ff27e7fc640 1 -- 172.21.15.16:0/1229546726 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff278004220 con 0x7ff288103660 2024-09-12T02:16:50.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.854+0000 7ff27e7fc640 1 -- 172.21.15.16:0/1229546726 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2780043c0 con 0x7ff288103660 2024-09-12T02:16:50.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.854+0000 7ff28f749640 1 -- 172.21.15.16:0/1229546726 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2881a6bc0 con 0x7ff288103660 2024-09-12T02:16:50.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.854+0000 7ff27e7fc640 1 -- 172.21.15.16:0/1229546726 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff278004a00 con 0x7ff288103660 2024-09-12T02:16:50.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.854+0000 7ff28f749640 1 -- 172.21.15.16:0/1229546726 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2881a7100 con 0x7ff288103660 2024-09-12T02:16:50.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.857+0000 7ff27e7fc640 1 -- 172.21.15.16:0/1229546726 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff2780079b0 con 0x7ff288103660 2024-09-12T02:16:50.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.857+0000 7ff291b22740 1 Processor -- start 2024-09-12T02:16:50.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.857+0000 7ff291b22740 1 -- start start 2024-09-12T02:16:50.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.857+0000 7ff291b22740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559ce1d00b60 0x559ce1d00f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:50.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.857+0000 7ff291b22740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559ce1d01480 0x559ce1d0b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:50.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.857+0000 7ff291b22740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559ce1d0bd40 0x559ce1d0e130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:50.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.857+0000 7ff291b22740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x559ce1c66280 con 0x559ce1d01480 2024-09-12T02:16:50.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.857+0000 7ff291b22740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x559ce1c0db10 con 0x559ce1d00b60 2024-09-12T02:16:50.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.857+0000 7ff291b22740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x559ce1c62240 con 0x559ce1d0bd40 2024-09-12T02:16:50.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.857+0000 7ff28d4e5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559ce1d01480 0x559ce1d0b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:50.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.857+0000 7ff28d4e5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559ce1d01480 0x559ce1d0b800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51554/0 (socket says 172.21.15.16:51554) 2024-09-12T02:16:50.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.857+0000 7ff28d4e5640 1 -- 172.21.15.16:0/1850051091 learned_addr learned my addr 172.21.15.16:0/1850051091 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:16:50.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.857+0000 7ff28dce6640 1 --2- 172.21.15.16:0/1850051091 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559ce1d00b60 0x559ce1d00f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:50.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.857+0000 7ff28cce4640 1 --2- 172.21.15.16:0/1850051091 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559ce1d0bd40 0x559ce1d0e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:50.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.857+0000 7ff28d4e5640 1 -- 172.21.15.16:0/1850051091 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559ce1d0bd40 msgr2=0x559ce1d0e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:50.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.858+0000 7ff27e7fc640 1 --2- 172.21.15.16:0/1229546726 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7ff2740a2b80 0x7ff2740a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:50.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.858+0000 7ff28d4e5640 1 --2- 172.21.15.16:0/1850051091 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559ce1d0bd40 0x559ce1d0e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:50.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.858+0000 7ff28d4e5640 1 -- 172.21.15.16:0/1850051091 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559ce1d00b60 msgr2=0x559ce1d00f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:50.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.858+0000 7ff28d4e5640 1 --2- 172.21.15.16:0/1850051091 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559ce1d00b60 0x559ce1d00f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:50.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.858+0000 7ff28d4e5640 1 -- 172.21.15.16:0/1850051091 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ce1c1a1f0 con 0x559ce1d01480 2024-09-12T02:16:50.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.858+0000 7ff28cce4640 1 --2- 172.21.15.16:0/1850051091 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559ce1d0bd40 0x559ce1d0e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:16:50.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.858+0000 7ff28dce6640 1 --2- 172.21.15.16:0/1850051091 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559ce1d00b60 0x559ce1d00f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:16:50.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.858+0000 7ff27e7fc640 1 -- 172.21.15.16:0/1229546726 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7ff2780ceee0 con 0x7ff288103660 2024-09-12T02:16:50.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.858+0000 7ff28dce6640 1 --2- 172.21.15.16:0/1229546726 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7ff2740a2b80 0x7ff2740a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:50.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.858+0000 7ff28d4e5640 1 --2- 172.21.15.16:0/1850051091 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559ce1d01480 0x559ce1d0b800 secure :-1 s=READY pgs=2014 cs=0 l=1 rev1=1 crypto rx=0x7ff28400cca0 tx=0x7ff28400e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e2c9720ca3b5c35 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:50.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.858+0000 7ff2737fe640 1 -- 172.21.15.16:0/1850051091 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff284005220 con 0x559ce1d01480 2024-09-12T02:16:50.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.858+0000 7ff2737fe640 1 -- 172.21.15.16:0/1850051091 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff284004c80 con 0x559ce1d01480 2024-09-12T02:16:50.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.858+0000 7ff2737fe640 1 -- 172.21.15.16:0/1850051091 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff284010890 con 0x559ce1d01480 2024-09-12T02:16:50.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.859+0000 7ff291b22740 1 -- 172.21.15.16:0/1850051091 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559ce1d01480 msgr2=0x559ce1d0b800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:50.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.859+0000 7ff28dce6640 1 --2- 172.21.15.16:0/1229546726 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7ff2740a2b80 0x7ff2740a5040 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7ff2800098d0 tx=0x7ff2800095c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:50.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.859+0000 7ff291b22740 1 --2- 172.21.15.16:0/1850051091 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559ce1d01480 0x559ce1d0b800 secure :-1 s=READY pgs=2014 cs=0 l=1 rev1=1 crypto rx=0x7ff28400cca0 tx=0x7ff28400e720 comp rx=0 tx=0).stop 2024-09-12T02:16:50.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.859+0000 7ff291b22740 1 -- 172.21.15.16:0/1850051091 shutdown_connections 2024-09-12T02:16:50.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.859+0000 7ff291b22740 1 --2- 172.21.15.16:0/1850051091 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559ce1d0bd40 0x559ce1d0e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:50.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.859+0000 7ff291b22740 1 --2- 172.21.15.16:0/1850051091 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559ce1d01480 0x559ce1d0b800 unknown :-1 s=CLOSED pgs=2014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:50.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.859+0000 7ff291b22740 1 --2- 172.21.15.16:0/1850051091 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559ce1d00b60 0x559ce1d00f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:50.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.859+0000 7ff291b22740 1 -- 172.21.15.16:0/1850051091 >> 172.21.15.16:0/1850051091 conn(0x559ce1cfd0f0 msgr2=0x559ce1cfd4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:50.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.859+0000 7ff291b22740 1 -- 172.21.15.16:0/1850051091 shutdown_connections 2024-09-12T02:16:50.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.859+0000 7ff291b22740 1 -- 172.21.15.16:0/1850051091 wait complete. 2024-09-12T02:16:50.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.859+0000 7ff291b22740 1 -- start start 2024-09-12T02:16:50.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.859+0000 7ff291b22740 1 Processor -- start 2024-09-12T02:16:50.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.860+0000 7ff291b22740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559ce1d00b60 0x559ce1d08bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:50.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.860+0000 7ff291b22740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559ce1d01480 0x559ce1d09110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:50.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.860+0000 7ff291b22740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559ce1d0bd40 0x559ce1d09650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:50.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.860+0000 7ff291b22740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x559ce1d0e910 con 0x559ce1d00b60 2024-09-12T02:16:50.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.860+0000 7ff291b22740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x559ce1c20f10 con 0x559ce1d0bd40 2024-09-12T02:16:50.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.860+0000 7ff291b22740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x559ce1d0ea90 con 0x559ce1d01480 2024-09-12T02:16:50.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.860+0000 7ff291b22740 10 client.0 _ll_register_callbacks cb 0x559ce1be9a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:16:50.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.860+0000 7ff28d4e5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559ce1d00b60 0x559ce1d08bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:50.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.860+0000 7ff28d4e5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559ce1d00b60 0x559ce1d08bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51568/0 (socket says 172.21.15.16:51568) 2024-09-12T02:16:50.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.860+0000 7ff28d4e5640 1 -- 172.21.15.16:0/292647301 learned_addr learned my addr 172.21.15.16:0/292647301 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:16:50.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.860+0000 7ff28dce6640 1 --2- 172.21.15.16:0/292647301 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559ce1d0bd40 0x559ce1d09650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:50.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.860+0000 7ff28cce4640 1 --2- 172.21.15.16:0/292647301 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559ce1d01480 0x559ce1d09110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:50.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.860+0000 7ff28d4e5640 1 -- 172.21.15.16:0/292647301 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559ce1d01480 msgr2=0x559ce1d09110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:50.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.860+0000 7ff28d4e5640 1 --2- 172.21.15.16:0/292647301 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559ce1d01480 0x559ce1d09110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:50.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.860+0000 7ff28d4e5640 1 -- 172.21.15.16:0/292647301 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559ce1d0bd40 msgr2=0x559ce1d09650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:50.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.860+0000 7ff28d4e5640 1 --2- 172.21.15.16:0/292647301 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559ce1d0bd40 0x559ce1d09650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:50.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.860+0000 7ff28d4e5640 1 -- 172.21.15.16:0/292647301 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ce19d76a0 con 0x559ce1d00b60 2024-09-12T02:16:50.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.860+0000 7ff28d4e5640 1 --2- 172.21.15.16:0/292647301 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559ce1d00b60 0x559ce1d08bd0 secure :-1 s=READY pgs=2015 cs=0 l=1 rev1=1 crypto rx=0x7ff28400cc70 tx=0x7ff28400c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53ffd189302d4b02 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:50.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.860+0000 7ff270ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:16:50.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.861+0000 7ff270ff9640 1 -- 172.21.15.16:0/292647301 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff284029060 con 0x559ce1d00b60 2024-09-12T02:16:50.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.861+0000 7ff270ff9640 1 -- 172.21.15.16:0/292647301 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7ff284005210 con 0x559ce1d00b60 2024-09-12T02:16:50.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.861+0000 7ff291b22740 1 -- 172.21.15.16:0/292647301 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff284004c80 con 0x559ce1d00b60 2024-09-12T02:16:50.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.861+0000 7ff270ff9640 1 -- 172.21.15.16:0/292647301 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff284005410 con 0x559ce1d00b60 2024-09-12T02:16:50.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.861+0000 7ff291b22740 10 client.20484 fetch_fsmap learned FSMap version 25 2024-09-12T02:16:50.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.861+0000 7ff291b22740 1 -- 172.21.15.16:0/292647301 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x559ce1d1a460 con 0x559ce1d00b60 2024-09-12T02:16:50.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.861+0000 7ff270ff9640 1 -- 172.21.15.16:0/292647301 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff284004c80 con 0x559ce1d00b60 2024-09-12T02:16:50.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.861+0000 7ff270ff9640 1 -- 172.21.15.16:0/292647301 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7ff28402e070 con 0x559ce1d00b60 2024-09-12T02:16:50.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.861+0000 7ff291b22740 10 client.20484 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:16:50.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.861+0000 7ff291b22740 10 client.20484 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:16:50.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.861+0000 7ff291b22740 20 client.20484 populate_metadata read hostname 'smithi016' 2024-09-12T02:16:50.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.861+0000 7ff291b22740 1 --2- 172.21.15.16:0/292647301 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x559ce1d1b920 0x559ce1d1dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:50.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.861+0000 7ff291b22740 4 client.20484 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:16:50.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.861+0000 7ff291b22740 1 -- 172.21.15.16:0/292647301 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559ce1c737f0 con 0x559ce1d1b920 2024-09-12T02:16:50.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.862+0000 7ff28cce4640 1 --2- 172.21.15.16:0/292647301 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x559ce1d1b920 0x559ce1d1dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:50.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.862+0000 7ff28cce4640 1 --2- 172.21.15.16:0/292647301 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x559ce1d1b920 0x559ce1d1dd10 crc :-1 s=READY pgs=445 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71cd81fd0eee08d server_cookie=6513e414e5781b59 in_seq=0 out_seq=0 2024-09-12T02:16:50.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.862+0000 7ff270ff9640 10 client.20484 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:16:50.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.863+0000 7ff270ff9640 1 -- 172.21.15.16:0/292647301 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559ce1c737f0 con 0x559ce1d1b920 2024-09-12T02:16:50.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.863+0000 7ff270ff9640 10 client.20484 handle_command_reply: tid=0 2024-09-12T02:16:50.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.863+0000 7ff291b22740 1 client.20484 shutdown 2024-09-12T02:16:50.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.863+0000 7ff291b22740 1 -- 172.21.15.16:0/292647301 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559ce1d00b60 msgr2=0x559ce1d08bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:50.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.863+0000 7ff291b22740 1 --2- 172.21.15.16:0/292647301 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559ce1d00b60 0x559ce1d08bd0 secure :-1 s=READY pgs=2015 cs=0 l=1 rev1=1 crypto rx=0x7ff28400cc70 tx=0x7ff28400c9e0 comp rx=0 tx=0).stop 2024-09-12T02:16:50.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.863+0000 7ff291b22740 1 -- 172.21.15.16:0/292647301 shutdown_connections 2024-09-12T02:16:50.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.863+0000 7ff291b22740 1 --2- 172.21.15.16:0/292647301 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x559ce1d1b920 0x559ce1d1dd10 crc :-1 s=READY pgs=445 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:50.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.863+0000 7ff291b22740 1 --2- 172.21.15.16:0/292647301 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559ce1d0bd40 0x559ce1d09650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:50.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.863+0000 7ff291b22740 1 --2- 172.21.15.16:0/292647301 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559ce1d01480 0x559ce1d09110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:50.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.863+0000 7ff270ff9640 0 client.20484 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:16:50.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.863+0000 7ff291b22740 1 --2- 172.21.15.16:0/292647301 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559ce1d00b60 0x559ce1d08bd0 unknown :-1 s=CLOSED pgs=2015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:50.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.863+0000 7ff291b22740 1 -- 172.21.15.16:0/292647301 >> 172.21.15.16:0/292647301 conn(0x559ce1cfd0f0 msgr2=0x559ce1d02e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:50.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.863+0000 7ff291b22740 1 -- 172.21.15.16:0/292647301 shutdown_connections 2024-09-12T02:16:50.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.863+0000 7ff291b22740 1 -- 172.21.15.16:0/292647301 wait complete. 2024-09-12T02:16:50.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.863+0000 7ff291b22740 20 client.20484 trim_cache size 0 max 16384 2024-09-12T02:16:50.892 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:16:50 smithi184 ceph-mon[35593]: pgmap v782: 97 pgs: 97 active+clean; 3.6 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-09-12T02:16:50.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.881+0000 7ff291b22740 1 Processor -- start 2024-09-12T02:16:50.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.881+0000 7ff291b22740 1 -- start start 2024-09-12T02:16:50.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.881+0000 7ff291b22740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559ce1d00b60 0x559ce1d035c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:50.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.881+0000 7ff291b22740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559ce1d01480 0x559ce1d20d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:50.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.881+0000 7ff291b22740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559ce1d0bd40 0x559ce1d21270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:50.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.881+0000 7ff291b22740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff2881b1ee0 con 0x559ce1d01480 2024-09-12T02:16:50.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.881+0000 7ff291b22740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff2881af7b0 con 0x559ce1d00b60 2024-09-12T02:16:50.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.881+0000 7ff291b22740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff2881ad600 con 0x559ce1d0bd40 2024-09-12T02:16:50.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.881+0000 7ff28cce4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559ce1d01480 0x559ce1d20d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:50.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.881+0000 7ff28cce4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559ce1d01480 0x559ce1d20d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51572/0 (socket says 172.21.15.16:51572) 2024-09-12T02:16:50.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.881+0000 7ff28cce4640 1 -- 172.21.15.16:0/2923077372 learned_addr learned my addr 172.21.15.16:0/2923077372 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:16:50.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.881+0000 7ff28d4e5640 1 --2- 172.21.15.16:0/2923077372 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559ce1d00b60 0x559ce1d035c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:50.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.882+0000 7ff28cce4640 1 -- 172.21.15.16:0/2923077372 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559ce1d0bd40 msgr2=0x559ce1d21270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:50.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.882+0000 7ff28dce6640 1 --2- 172.21.15.16:0/2923077372 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559ce1d0bd40 0x559ce1d21270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:50.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.882+0000 7ff28cce4640 1 --2- 172.21.15.16:0/2923077372 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559ce1d0bd40 0x559ce1d21270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:50.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.882+0000 7ff28cce4640 1 -- 172.21.15.16:0/2923077372 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559ce1d00b60 msgr2=0x559ce1d035c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:50.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.882+0000 7ff28cce4640 1 --2- 172.21.15.16:0/2923077372 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559ce1d00b60 0x559ce1d035c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:50.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.882+0000 7ff28cce4640 1 -- 172.21.15.16:0/2923077372 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ce1cfcf20 con 0x559ce1d01480 2024-09-12T02:16:50.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.882+0000 7ff28cce4640 1 --2- 172.21.15.16:0/2923077372 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559ce1d01480 0x559ce1d20d30 secure :-1 s=READY pgs=2016 cs=0 l=1 rev1=1 crypto rx=0x7ff278005fd0 tx=0x7ff2780d1410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0e2c9c42ed06149 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:50.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.882+0000 7ff272ffd640 1 -- 172.21.15.16:0/2923077372 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff27801d040 con 0x559ce1d01480 2024-09-12T02:16:50.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.882+0000 7ff272ffd640 1 -- 172.21.15.16:0/2923077372 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff27809bee0 con 0x559ce1d01480 2024-09-12T02:16:50.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.882+0000 7ff272ffd640 1 -- 172.21.15.16:0/2923077372 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff278007ca0 con 0x559ce1d01480 2024-09-12T02:16:50.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.883+0000 7ff291b22740 1 -- 172.21.15.16:0/2923077372 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559ce1d01480 msgr2=0x559ce1d20d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:50.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.883+0000 7ff291b22740 1 --2- 172.21.15.16:0/2923077372 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559ce1d01480 0x559ce1d20d30 secure :-1 s=READY pgs=2016 cs=0 l=1 rev1=1 crypto rx=0x7ff278005fd0 tx=0x7ff2780d1410 comp rx=0 tx=0).stop 2024-09-12T02:16:50.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.883+0000 7ff291b22740 1 -- 172.21.15.16:0/2923077372 shutdown_connections 2024-09-12T02:16:50.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.883+0000 7ff291b22740 1 --2- 172.21.15.16:0/2923077372 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559ce1d0bd40 0x559ce1d21270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:50.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.883+0000 7ff291b22740 1 --2- 172.21.15.16:0/2923077372 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559ce1d01480 0x559ce1d20d30 unknown :-1 s=CLOSED pgs=2016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:50.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.883+0000 7ff291b22740 1 --2- 172.21.15.16:0/2923077372 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559ce1d00b60 0x559ce1d035c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:50.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.883+0000 7ff291b22740 1 -- 172.21.15.16:0/2923077372 >> 172.21.15.16:0/2923077372 conn(0x559ce1cfd0f0 msgr2=0x559ce1d17530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:50.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.883+0000 7ff291b22740 1 -- 172.21.15.16:0/2923077372 shutdown_connections 2024-09-12T02:16:50.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.883+0000 7ff291b22740 1 -- 172.21.15.16:0/2923077372 wait complete. 2024-09-12T02:16:50.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.883+0000 7ff291b22740 1 -- start start 2024-09-12T02:16:50.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.883+0000 7ff291b22740 1 Processor -- start 2024-09-12T02:16:50.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.884+0000 7ff291b22740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559ce1d00b60 0x559ce1d1f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:50.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.884+0000 7ff291b22740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559ce1d01480 0x559ce1d1f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:50.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.884+0000 7ff291b22740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559ce1d0bd40 0x559ce1d85260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:50.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.884+0000 7ff291b22740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff2881adc70 con 0x559ce1d00b60 2024-09-12T02:16:50.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.884+0000 7ff291b22740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff2881af0e0 con 0x559ce1d0bd40 2024-09-12T02:16:50.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.884+0000 7ff291b22740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff2881ae340 con 0x559ce1d01480 2024-09-12T02:16:50.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.884+0000 7ff291b22740 10 client.0 _ll_register_callbacks cb 0x7ff284014070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:16:50.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.884+0000 7ff28d4e5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559ce1d00b60 0x559ce1d1f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:50.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.884+0000 7ff28d4e5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559ce1d00b60 0x559ce1d1f1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51580/0 (socket says 172.21.15.16:51580) 2024-09-12T02:16:50.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.884+0000 7ff28d4e5640 1 -- 172.21.15.16:0/4160618710 learned_addr learned my addr 172.21.15.16:0/4160618710 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:16:50.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.884+0000 7ff28dce6640 1 --2- 172.21.15.16:0/4160618710 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559ce1d0bd40 0x559ce1d85260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:50.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.884+0000 7ff28cce4640 1 --2- 172.21.15.16:0/4160618710 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559ce1d01480 0x559ce1d1f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:50.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.884+0000 7ff28d4e5640 1 -- 172.21.15.16:0/4160618710 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559ce1d01480 msgr2=0x559ce1d1f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:50.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.884+0000 7ff28d4e5640 1 --2- 172.21.15.16:0/4160618710 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559ce1d01480 0x559ce1d1f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:50.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.884+0000 7ff28d4e5640 1 -- 172.21.15.16:0/4160618710 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559ce1d0bd40 msgr2=0x559ce1d85260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:50.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.884+0000 7ff28d4e5640 1 --2- 172.21.15.16:0/4160618710 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559ce1d0bd40 0x559ce1d85260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:50.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.884+0000 7ff28d4e5640 1 -- 172.21.15.16:0/4160618710 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ce1d857a0 con 0x559ce1d00b60 2024-09-12T02:16:50.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.884+0000 7ff28dce6640 1 --2- 172.21.15.16:0/4160618710 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559ce1d0bd40 0x559ce1d85260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:16:50.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.884+0000 7ff28cce4640 1 --2- 172.21.15.16:0/4160618710 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559ce1d01480 0x559ce1d1f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:16:50.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.884+0000 7ff28d4e5640 1 --2- 172.21.15.16:0/4160618710 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559ce1d00b60 0x559ce1d1f1b0 secure :-1 s=READY pgs=2017 cs=0 l=1 rev1=1 crypto rx=0x7ff28400cc70 tx=0x7ff28400cb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da0301208e756502 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:50.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.884+0000 7ff270ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:16:50.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.884+0000 7ff270ff9640 1 -- 172.21.15.16:0/4160618710 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff284034070 con 0x559ce1d00b60 2024-09-12T02:16:50.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.885+0000 7ff270ff9640 1 -- 172.21.15.16:0/4160618710 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7ff284005210 con 0x559ce1d00b60 2024-09-12T02:16:50.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.885+0000 7ff291b22740 1 -- 172.21.15.16:0/4160618710 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff27809bee0 con 0x559ce1d00b60 2024-09-12T02:16:50.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.885+0000 7ff270ff9640 1 -- 172.21.15.16:0/4160618710 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff28400c330 con 0x559ce1d00b60 2024-09-12T02:16:50.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.885+0000 7ff291b22740 10 client.20490 fetch_fsmap learned FSMap version 25 2024-09-12T02:16:50.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.885+0000 7ff291b22740 1 -- 172.21.15.16:0/4160618710 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x559ce1d86a80 con 0x559ce1d00b60 2024-09-12T02:16:50.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.885+0000 7ff270ff9640 1 -- 172.21.15.16:0/4160618710 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff27809bee0 con 0x559ce1d00b60 2024-09-12T02:16:50.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.885+0000 7ff270ff9640 1 -- 172.21.15.16:0/4160618710 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7ff28402e050 con 0x559ce1d00b60 2024-09-12T02:16:50.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.885+0000 7ff291b22740 10 client.20490 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:16:50.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.885+0000 7ff291b22740 10 client.20490 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:16:50.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.885+0000 7ff291b22740 20 client.20490 populate_metadata read hostname 'smithi016' 2024-09-12T02:16:50.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.885+0000 7ff291b22740 1 --2- 172.21.15.16:0/4160618710 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x559ce1d87120 0x559ce1d89540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:50.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.885+0000 7ff291b22740 4 client.20490 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-12T02:16:50.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.885+0000 7ff291b22740 1 -- 172.21.15.16:0/4160618710 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x559ce1c6dd00 con 0x559ce1d87120 2024-09-12T02:16:50.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.885+0000 7ff28cce4640 1 --2- 172.21.15.16:0/4160618710 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x559ce1d87120 0x559ce1d89540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:50.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.886+0000 7ff28cce4640 1 --2- 172.21.15.16:0/4160618710 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x559ce1d87120 0x559ce1d89540 crc :-1 s=READY pgs=446 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d36f54f196a98bad server_cookie=43ecc17cc973efc2 in_seq=0 out_seq=0 2024-09-12T02:16:50.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.886+0000 7ff270ff9640 10 client.20490 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:16:50.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.887+0000 7ff270ff9640 1 -- 172.21.15.16:0/4160618710 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x559ce1c6dd00 con 0x559ce1d87120 2024-09-12T02:16:50.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.887+0000 7ff270ff9640 10 client.20490 handle_command_reply: tid=0 2024-09-12T02:16:50.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.887+0000 7ff291b22740 1 client.20490 shutdown 2024-09-12T02:16:50.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.887+0000 7ff291b22740 1 -- 172.21.15.16:0/4160618710 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559ce1d00b60 msgr2=0x559ce1d1f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:50.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.887+0000 7ff291b22740 1 --2- 172.21.15.16:0/4160618710 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559ce1d00b60 0x559ce1d1f1b0 secure :-1 s=READY pgs=2017 cs=0 l=1 rev1=1 crypto rx=0x7ff28400cc70 tx=0x7ff28400cb60 comp rx=0 tx=0).stop 2024-09-12T02:16:50.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.887+0000 7ff291b22740 1 -- 172.21.15.16:0/4160618710 shutdown_connections 2024-09-12T02:16:50.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.887+0000 7ff291b22740 1 --2- 172.21.15.16:0/4160618710 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x559ce1d87120 0x559ce1d89540 crc :-1 s=READY pgs=446 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:50.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.887+0000 7ff291b22740 1 --2- 172.21.15.16:0/4160618710 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559ce1d0bd40 0x559ce1d85260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:50.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.887+0000 7ff291b22740 1 --2- 172.21.15.16:0/4160618710 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559ce1d01480 0x559ce1d1f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:50.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.887+0000 7ff270ff9640 0 client.20490 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:16:50.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.887+0000 7ff291b22740 1 --2- 172.21.15.16:0/4160618710 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559ce1d00b60 0x559ce1d1f1b0 unknown :-1 s=CLOSED pgs=2017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:50.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.887+0000 7ff291b22740 1 -- 172.21.15.16:0/4160618710 >> 172.21.15.16:0/4160618710 conn(0x559ce1cfd0f0 msgr2=0x559ce1cfba50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:50.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.887+0000 7ff291b22740 1 -- 172.21.15.16:0/4160618710 shutdown_connections 2024-09-12T02:16:50.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.887+0000 7ff291b22740 1 -- 172.21.15.16:0/4160618710 wait complete. 2024-09-12T02:16:50.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.887+0000 7ff291b22740 20 client.20490 trim_cache size 0 max 16384 2024-09-12T02:16:50.919 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T02:16:50.919 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-09-12T02:16:50.919 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "478ebbfe-8549-4944-837a-95ceb7bc4cb9", 2024-09-12T02:16:50.919 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-09-12T02:16:50.919 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T02:16:50.920 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:16:50 smithi033 ceph-mon[35332]: pgmap v782: 97 pgs: 97 active+clean; 3.6 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-09-12T02:16:50.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.888+0000 7ff28f749640 1 -- 172.21.15.16:0/1229546726 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7ff2740a2b80 msgr2=0x7ff2740a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:50.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.888+0000 7ff28f749640 1 --2- 172.21.15.16:0/1229546726 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7ff2740a2b80 0x7ff2740a5040 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7ff2800098d0 tx=0x7ff2800095c0 comp rx=0 tx=0).stop 2024-09-12T02:16:50.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.888+0000 7ff28f749640 1 -- 172.21.15.16:0/1229546726 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff288103660 msgr2=0x7ff288100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:50.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.888+0000 7ff28f749640 1 --2- 172.21.15.16:0/1229546726 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff288103660 0x7ff288100ee0 secure :-1 s=READY pgs=2013 cs=0 l=1 rev1=1 crypto rx=0x7ff27800c940 tx=0x7ff27800ce00 comp rx=0 tx=0).stop 2024-09-12T02:16:50.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.888+0000 7ff28f749640 1 -- 172.21.15.16:0/1229546726 shutdown_connections 2024-09-12T02:16:50.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.888+0000 7ff28f749640 1 --2- 172.21.15.16:0/1229546726 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7ff2740a2b80 0x7ff2740a5040 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:50.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.888+0000 7ff28f749640 1 --2- 172.21.15.16:0/1229546726 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff28810c170 0x7ff288101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:50.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.888+0000 7ff28f749640 1 --2- 172.21.15.16:0/1229546726 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff288103660 0x7ff288100ee0 unknown :-1 s=CLOSED pgs=2013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:50.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.888+0000 7ff28f749640 1 --2- 172.21.15.16:0/1229546726 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff288102d40 0x7ff2881009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:50.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.888+0000 7ff28f749640 1 -- 172.21.15.16:0/1229546726 >> 172.21.15.16:0/1229546726 conn(0x7ff2880fc7a0 msgr2=0x7ff2880fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:50.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.888+0000 7ff28f749640 1 -- 172.21.15.16:0/1229546726 shutdown_connections 2024-09-12T02:16:50.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:50.888+0000 7ff28f749640 1 -- 172.21.15.16:0/1229546726 wait complete. 2024-09-12T02:16:51.355 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.336+0000 7ff9d1ee4640 1 -- 172.21.15.16:0/2257919357 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9cc102610 msgr2=0x7ff9cc108ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:51.355 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.336+0000 7ff9d1ee4640 1 --2- 172.21.15.16:0/2257919357 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9cc102610 0x7ff9cc108ea0 secure :-1 s=READY pgs=2018 cs=0 l=1 rev1=1 crypto rx=0x7ff9c4009ca0 tx=0x7ff9c402a8c0 comp rx=0 tx=0).stop 2024-09-12T02:16:51.355 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.337+0000 7ff9d1ee4640 1 -- 172.21.15.16:0/2257919357 shutdown_connections 2024-09-12T02:16:51.356 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.337+0000 7ff9d1ee4640 1 --2- 172.21.15.16:0/2257919357 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9cc102610 0x7ff9cc108ea0 unknown :-1 s=CLOSED pgs=2018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:51.356 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.337+0000 7ff9d1ee4640 1 --2- 172.21.15.16:0/2257919357 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff9cc101c50 0x7ff9cc1020d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:51.356 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.337+0000 7ff9d1ee4640 1 --2- 172.21.15.16:0/2257919357 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff9cc100a60 0x7ff9cc100e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:51.356 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.337+0000 7ff9d1ee4640 1 -- 172.21.15.16:0/2257919357 >> 172.21.15.16:0/2257919357 conn(0x7ff9cc0fc1e0 msgr2=0x7ff9cc0fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:51.356 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.337+0000 7ff9d1ee4640 1 -- 172.21.15.16:0/2257919357 shutdown_connections 2024-09-12T02:16:51.356 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.337+0000 7ff9d1ee4640 1 -- 172.21.15.16:0/2257919357 wait complete. 2024-09-12T02:16:51.357 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.337+0000 7ff9d1ee4640 1 Processor -- start 2024-09-12T02:16:51.357 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.337+0000 7ff9d1ee4640 1 -- start start 2024-09-12T02:16:51.357 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.337+0000 7ff9d1ee4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff9cc100a60 0x7ff9cc06ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:51.357 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.337+0000 7ff9d1ee4640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff9cc101c50 0x7ff9cc19cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:51.357 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.337+0000 7ff9d1ee4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9cc102610 0x7ff9cc19f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:51.358 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.337+0000 7ff9d1ee4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff9cc10c6c0 con 0x7ff9cc102610 2024-09-12T02:16:51.358 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.337+0000 7ff9d1ee4640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff9cc10c540 con 0x7ff9cc100a60 2024-09-12T02:16:51.358 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.337+0000 7ff9d1ee4640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff9cc10c840 con 0x7ff9cc101c50 2024-09-12T02:16:51.358 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.337+0000 7ff9cbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9cc102610 0x7ff9cc19f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:51.358 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.338+0000 7ff9cbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9cc102610 0x7ff9cc19f0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58486/0 (socket says 172.21.15.16:58486) 2024-09-12T02:16:51.358 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.338+0000 7ff9cbfff640 1 -- 172.21.15.16:0/1072099128 learned_addr learned my addr 172.21.15.16:0/1072099128 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:16:51.359 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.338+0000 7ff9cb7fe640 1 --2- 172.21.15.16:0/1072099128 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff9cc100a60 0x7ff9cc06ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:51.359 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.338+0000 7ff9bbfff640 1 --2- 172.21.15.16:0/1072099128 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff9cc101c50 0x7ff9cc19cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:51.359 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.338+0000 7ff9cbfff640 1 -- 172.21.15.16:0/1072099128 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff9cc101c50 msgr2=0x7ff9cc19cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:51.359 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.338+0000 7ff9cbfff640 1 --2- 172.21.15.16:0/1072099128 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff9cc101c50 0x7ff9cc19cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:51.359 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.338+0000 7ff9cbfff640 1 -- 172.21.15.16:0/1072099128 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff9cc100a60 msgr2=0x7ff9cc06ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:51.359 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.338+0000 7ff9cbfff640 1 --2- 172.21.15.16:0/1072099128 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff9cc100a60 0x7ff9cc06ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:51.360 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.338+0000 7ff9cbfff640 1 -- 172.21.15.16:0/1072099128 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9cc19fa00 con 0x7ff9cc102610 2024-09-12T02:16:51.360 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.338+0000 7ff9cbfff640 1 --2- 172.21.15.16:0/1072099128 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9cc102610 0x7ff9cc19f0f0 secure :-1 s=READY pgs=2019 cs=0 l=1 rev1=1 crypto rx=0x7ff9c4005e30 tx=0x7ff9c4031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:51.360 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.338+0000 7ff9c97fa640 1 -- 172.21.15.16:0/1072099128 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9c4003c30 con 0x7ff9cc102610 2024-09-12T02:16:51.360 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.338+0000 7ff9c97fa640 1 -- 172.21.15.16:0/1072099128 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9c4003dd0 con 0x7ff9cc102610 2024-09-12T02:16:51.360 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.338+0000 7ff9c97fa640 1 -- 172.21.15.16:0/1072099128 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9c402fde0 con 0x7ff9cc102610 2024-09-12T02:16:51.361 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.338+0000 7ff9d1ee4640 1 -- 172.21.15.16:0/1072099128 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9cc19fc30 con 0x7ff9cc102610 2024-09-12T02:16:51.361 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.338+0000 7ff9d1ee4640 1 -- 172.21.15.16:0/1072099128 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9cc1a01d0 con 0x7ff9cc102610 2024-09-12T02:16:51.361 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.341+0000 7ff9c97fa640 1 -- 172.21.15.16:0/1072099128 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff9c4004050 con 0x7ff9cc102610 2024-09-12T02:16:51.361 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.341+0000 7ff9d4c79740 1 Processor -- start 2024-09-12T02:16:51.361 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.341+0000 7ff9d4c79740 1 -- start start 2024-09-12T02:16:51.361 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.342+0000 7ff9d4c79740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558d445549a0 0x558d44554da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:51.362 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.342+0000 7ff9d4c79740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d445552e0 0x558d44559da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:51.362 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.342+0000 7ff9d4c79740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558d4455a370 0x558d4455c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:51.362 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.342+0000 7ff9d4c79740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x558d444b82f0 con 0x558d445552e0 2024-09-12T02:16:51.362 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.342+0000 7ff9d4c79740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x558d44460270 con 0x558d4455a370 2024-09-12T02:16:51.362 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.342+0000 7ff9d4c79740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x558d444b42b0 con 0x558d445549a0 2024-09-12T02:16:51.363 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.342+0000 7ff9cb7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d445552e0 0x558d44559da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:51.363 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.342+0000 7ff9cb7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d445552e0 0x558d44559da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58502/0 (socket says 172.21.15.16:58502) 2024-09-12T02:16:51.363 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.342+0000 7ff9cb7fe640 1 -- 172.21.15.16:0/3622064181 learned_addr learned my addr 172.21.15.16:0/3622064181 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:16:51.363 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.342+0000 7ff9bbfff640 1 --2- 172.21.15.16:0/3622064181 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558d4455a370 0x558d4455c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:51.363 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.342+0000 7ff9bbfff640 1 --2- 172.21.15.16:0/3622064181 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558d445549a0 0x558d44554da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:51.364 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.342+0000 7ff9cb7fe640 1 -- 172.21.15.16:0/3622064181 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558d445549a0 msgr2=0x558d44554da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:51.364 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.342+0000 7ff9cb7fe640 1 --2- 172.21.15.16:0/3622064181 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558d445549a0 0x558d44554da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:51.364 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.342+0000 7ff9cb7fe640 1 -- 172.21.15.16:0/3622064181 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558d4455a370 msgr2=0x558d4455c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:51.364 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.342+0000 7ff9cb7fe640 1 --2- 172.21.15.16:0/3622064181 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558d4455a370 0x558d4455c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:51.364 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.342+0000 7ff9cb7fe640 1 -- 172.21.15.16:0/3622064181 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d4445f340 con 0x558d445552e0 2024-09-12T02:16:51.364 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.342+0000 7ff9bbfff640 1 --2- 172.21.15.16:0/3622064181 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558d445549a0 0x558d44554da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:16:51.365 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.342+0000 7ff9c97fa640 1 --2- 172.21.15.16:0/1072099128 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7ff9b40a2cb0 0x7ff9b40a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:51.365 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.342+0000 7ff9cb7fe640 1 --2- 172.21.15.16:0/3622064181 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d445552e0 0x558d44559da0 secure :-1 s=READY pgs=2020 cs=0 l=1 rev1=1 crypto rx=0x7ff9bc00abf0 tx=0x7ff9bc00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88deb6d107f0235a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:51.365 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.342+0000 7ff9bbfff640 1 --2- 172.21.15.16:0/1072099128 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7ff9b40a2cb0 0x7ff9b40a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:51.365 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.342+0000 7ff9c97fa640 1 -- 172.21.15.16:0/1072099128 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7ff9c40f01b0 con 0x7ff9cc102610 2024-09-12T02:16:51.365 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.342+0000 7ff9b9ffb640 1 -- 172.21.15.16:0/3622064181 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9bc004220 con 0x558d445552e0 2024-09-12T02:16:51.366 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.343+0000 7ff9b9ffb640 1 -- 172.21.15.16:0/3622064181 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9bc0043c0 con 0x558d445552e0 2024-09-12T02:16:51.366 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.343+0000 7ff9b9ffb640 1 -- 172.21.15.16:0/3622064181 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9bc018900 con 0x558d445552e0 2024-09-12T02:16:51.366 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.343+0000 7ff9d4c79740 1 -- 172.21.15.16:0/3622064181 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d445552e0 msgr2=0x558d44559da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:51.366 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.343+0000 7ff9bbfff640 1 --2- 172.21.15.16:0/1072099128 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7ff9b40a2cb0 0x7ff9b40a5170 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7ff9cc06abe0 tx=0x7ff9c0004730 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:51.366 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.343+0000 7ff9d4c79740 1 --2- 172.21.15.16:0/3622064181 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d445552e0 0x558d44559da0 secure :-1 s=READY pgs=2020 cs=0 l=1 rev1=1 crypto rx=0x7ff9bc00abf0 tx=0x7ff9bc00f790 comp rx=0 tx=0).stop 2024-09-12T02:16:51.367 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.343+0000 7ff9d4c79740 1 -- 172.21.15.16:0/3622064181 shutdown_connections 2024-09-12T02:16:51.367 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.343+0000 7ff9d4c79740 1 --2- 172.21.15.16:0/3622064181 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558d4455a370 0x558d4455c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:51.367 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.343+0000 7ff9d4c79740 1 --2- 172.21.15.16:0/3622064181 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d445552e0 0x558d44559da0 unknown :-1 s=CLOSED pgs=2020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:51.367 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.343+0000 7ff9d4c79740 1 --2- 172.21.15.16:0/3622064181 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558d445549a0 0x558d44554da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:51.367 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.343+0000 7ff9d4c79740 1 -- 172.21.15.16:0/3622064181 >> 172.21.15.16:0/3622064181 conn(0x558d44550010 msgr2=0x558d44550410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:51.368 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.343+0000 7ff9d4c79740 1 -- 172.21.15.16:0/3622064181 shutdown_connections 2024-09-12T02:16:51.368 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.343+0000 7ff9d4c79740 1 -- 172.21.15.16:0/3622064181 wait complete. 2024-09-12T02:16:51.368 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.344+0000 7ff9d4c79740 1 -- start start 2024-09-12T02:16:51.368 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.344+0000 7ff9d4c79740 1 Processor -- start 2024-09-12T02:16:51.368 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.344+0000 7ff9d4c79740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558d445549a0 0x558d4455eb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:51.369 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.344+0000 7ff9d4c79740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558d445552e0 0x558d4455f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:51.369 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.344+0000 7ff9d4c79740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d4455a370 0x558d445660c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:51.369 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.344+0000 7ff9d4c79740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x558d4455d340 con 0x558d4455a370 2024-09-12T02:16:51.369 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.344+0000 7ff9d4c79740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x558d44441290 con 0x558d445552e0 2024-09-12T02:16:51.369 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.344+0000 7ff9d4c79740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x558d4455d4c0 con 0x558d445549a0 2024-09-12T02:16:51.369 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.344+0000 7ff9d4c79740 10 client.0 _ll_register_callbacks cb 0x558d44472140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:16:51.370 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.344+0000 7ff9cbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d4455a370 0x558d445660c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:51.370 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.344+0000 7ff9cbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d4455a370 0x558d445660c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58514/0 (socket says 172.21.15.16:58514) 2024-09-12T02:16:51.370 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.344+0000 7ff9cbfff640 1 -- 172.21.15.16:0/339835353 learned_addr learned my addr 172.21.15.16:0/339835353 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:16:51.370 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.344+0000 7ff9bbfff640 1 --2- 172.21.15.16:0/339835353 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558d445552e0 0x558d4455f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:51.370 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.345+0000 7ff9cb7fe640 1 --2- 172.21.15.16:0/339835353 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558d445549a0 0x558d4455eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:51.371 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.345+0000 7ff9cbfff640 1 -- 172.21.15.16:0/339835353 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558d445549a0 msgr2=0x558d4455eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:51.371 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.345+0000 7ff9cbfff640 1 --2- 172.21.15.16:0/339835353 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558d445549a0 0x558d4455eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:51.371 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.345+0000 7ff9cbfff640 1 -- 172.21.15.16:0/339835353 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558d445552e0 msgr2=0x558d4455f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:51.371 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.345+0000 7ff9cbfff640 1 --2- 172.21.15.16:0/339835353 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558d445552e0 0x558d4455f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:51.371 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.345+0000 7ff9cbfff640 1 -- 172.21.15.16:0/339835353 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d441cb210 con 0x558d4455a370 2024-09-12T02:16:51.372 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.345+0000 7ff9cbfff640 1 --2- 172.21.15.16:0/339835353 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d4455a370 0x558d445660c0 secure :-1 s=READY pgs=2021 cs=0 l=1 rev1=1 crypto rx=0x7ff9c4031a40 tx=0x7ff9c40bc4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=258831878ab75177 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:51.372 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.345+0000 7ff9aaffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:16:51.372 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.345+0000 7ff9aaffd640 1 -- 172.21.15.16:0/339835353 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9c40d5060 con 0x558d4455a370 2024-09-12T02:16:51.372 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.345+0000 7ff9aaffd640 1 -- 172.21.15.16:0/339835353 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7ff9c40b9120 con 0x558d4455a370 2024-09-12T02:16:51.372 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.345+0000 7ff9d4c79740 1 -- 172.21.15.16:0/339835353 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9bc0043c0 con 0x558d4455a370 2024-09-12T02:16:51.373 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.345+0000 7ff9aaffd640 1 -- 172.21.15.16:0/339835353 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff9c40ef030 con 0x558d4455a370 2024-09-12T02:16:51.373 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.345+0000 7ff9d4c79740 10 client.20502 fetch_fsmap learned FSMap version 25 2024-09-12T02:16:51.373 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.345+0000 7ff9d4c79740 1 -- 172.21.15.16:0/339835353 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x558d44297620 con 0x558d4455a370 2024-09-12T02:16:51.373 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.345+0000 7ff9aaffd640 1 -- 172.21.15.16:0/339835353 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9bc0043c0 con 0x558d4455a370 2024-09-12T02:16:51.373 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.345+0000 7ff9aaffd640 1 -- 172.21.15.16:0/339835353 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7ff9c40c5290 con 0x558d4455a370 2024-09-12T02:16:51.374 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.346+0000 7ff9d4c79740 10 client.20502 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:16:51.374 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.346+0000 7ff9d4c79740 10 client.20502 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:16:51.374 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.346+0000 7ff9d4c79740 20 client.20502 populate_metadata read hostname 'smithi016' 2024-09-12T02:16:51.374 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.346+0000 7ff9d4c79740 1 --2- 172.21.15.16:0/339835353 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x558d4456a710 0x558d4456cb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:51.374 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.346+0000 7ff9d4c79740 4 client.20502 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:16:51.375 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.346+0000 7ff9d4c79740 1 -- 172.21.15.16:0/339835353 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558d444c5a20 con 0x558d4456a710 2024-09-12T02:16:51.375 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.346+0000 7ff9cb7fe640 1 --2- 172.21.15.16:0/339835353 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x558d4456a710 0x558d4456cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:51.375 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.346+0000 7ff9cb7fe640 1 --2- 172.21.15.16:0/339835353 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x558d4456a710 0x558d4456cb20 crc :-1 s=READY pgs=447 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f62ba02b7bac6b8 server_cookie=1a718b8ebf3ce11d in_seq=0 out_seq=0 2024-09-12T02:16:51.375 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.346+0000 7ff9aaffd640 10 client.20502 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:16:51.375 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.347+0000 7ff9aaffd640 1 -- 172.21.15.16:0/339835353 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558d444c5a20 con 0x558d4456a710 2024-09-12T02:16:51.375 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.347+0000 7ff9aaffd640 10 client.20502 handle_command_reply: tid=0 2024-09-12T02:16:51.376 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.347+0000 7ff9d4c79740 1 client.20502 shutdown 2024-09-12T02:16:51.376 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.347+0000 7ff9d4c79740 1 -- 172.21.15.16:0/339835353 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d4455a370 msgr2=0x558d445660c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:51.376 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.347+0000 7ff9d4c79740 1 --2- 172.21.15.16:0/339835353 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d4455a370 0x558d445660c0 secure :-1 s=READY pgs=2021 cs=0 l=1 rev1=1 crypto rx=0x7ff9c4031a40 tx=0x7ff9c40bc4d0 comp rx=0 tx=0).stop 2024-09-12T02:16:51.376 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.347+0000 7ff9d4c79740 1 -- 172.21.15.16:0/339835353 shutdown_connections 2024-09-12T02:16:51.376 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.347+0000 7ff9d4c79740 1 --2- 172.21.15.16:0/339835353 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x558d4456a710 0x558d4456cb20 crc :-1 s=READY pgs=447 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:51.377 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.347+0000 7ff9d4c79740 1 --2- 172.21.15.16:0/339835353 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d4455a370 0x558d445660c0 unknown :-1 s=CLOSED pgs=2021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:51.377 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.347+0000 7ff9d4c79740 1 --2- 172.21.15.16:0/339835353 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558d445552e0 0x558d4455f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:51.377 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.347+0000 7ff9aaffd640 0 client.20502 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:16:51.377 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.347+0000 7ff9d4c79740 1 --2- 172.21.15.16:0/339835353 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558d445549a0 0x558d4455eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:51.377 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.347+0000 7ff9d4c79740 1 -- 172.21.15.16:0/339835353 >> 172.21.15.16:0/339835353 conn(0x558d44550010 msgr2=0x558d44551820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:51.378 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.347+0000 7ff9d4c79740 1 -- 172.21.15.16:0/339835353 shutdown_connections 2024-09-12T02:16:51.378 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.347+0000 7ff9d4c79740 1 -- 172.21.15.16:0/339835353 wait complete. 2024-09-12T02:16:51.378 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.347+0000 7ff9d4c79740 20 client.20502 trim_cache size 0 max 16384 2024-09-12T02:16:51.390 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.362+0000 7ff9d4c79740 1 Processor -- start 2024-09-12T02:16:51.390 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.362+0000 7ff9d4c79740 1 -- start start 2024-09-12T02:16:51.391 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.362+0000 7ff9d4c79740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d445549a0 0x558d4455eb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:51.391 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.362+0000 7ff9d4c79740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558d445552e0 0x558d44566080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:51.391 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.362+0000 7ff9d4c79740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558d4455a370 0x558d4455f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:51.391 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.362+0000 7ff9d4c79740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff9cc1ab730 con 0x558d445549a0 2024-09-12T02:16:51.391 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.362+0000 7ff9d4c79740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff9cc1a9000 con 0x558d4455a370 2024-09-12T02:16:51.392 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.362+0000 7ff9d4c79740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff9cc1a6e50 con 0x558d445552e0 2024-09-12T02:16:51.392 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.362+0000 7ff9cb7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d445549a0 0x558d4455eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:51.392 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.362+0000 7ff9cb7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d445549a0 0x558d4455eb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58518/0 (socket says 172.21.15.16:58518) 2024-09-12T02:16:51.392 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.362+0000 7ff9cb7fe640 1 -- 172.21.15.16:0/1576377586 learned_addr learned my addr 172.21.15.16:0/1576377586 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:16:51.392 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.362+0000 7ff9cb7fe640 1 -- 172.21.15.16:0/1576377586 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558d445552e0 msgr2=0x558d44566080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:51.393 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.362+0000 7ff9cb7fe640 1 --2- 172.21.15.16:0/1576377586 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558d445552e0 0x558d44566080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:51.393 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.362+0000 7ff9cb7fe640 1 -- 172.21.15.16:0/1576377586 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558d4455a370 msgr2=0x558d4455f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:51.393 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.362+0000 7ff9cb7fe640 1 --2- 172.21.15.16:0/1576377586 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558d4455a370 0x558d4455f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:51.393 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.362+0000 7ff9cb7fe640 1 -- 172.21.15.16:0/1576377586 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d44555f70 con 0x558d445549a0 2024-09-12T02:16:51.393 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.363+0000 7ff9cb7fe640 1 --2- 172.21.15.16:0/1576377586 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d445549a0 0x558d4455eb40 secure :-1 s=READY pgs=2022 cs=0 l=1 rev1=1 crypto rx=0x7ff9bc00a8e0 tx=0x7ff9bc000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=580b25992777cbc8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:51.394 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.363+0000 7ff9b97fa640 1 -- 172.21.15.16:0/1576377586 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9bc004080 con 0x558d445549a0 2024-09-12T02:16:51.394 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.363+0000 7ff9b97fa640 1 -- 172.21.15.16:0/1576377586 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9bc02f030 con 0x558d445549a0 2024-09-12T02:16:51.394 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.363+0000 7ff9b97fa640 1 -- 172.21.15.16:0/1576377586 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9bc004770 con 0x558d445549a0 2024-09-12T02:16:51.394 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.363+0000 7ff9d4c79740 1 -- 172.21.15.16:0/1576377586 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d445549a0 msgr2=0x558d4455eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:51.394 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.363+0000 7ff9d4c79740 1 --2- 172.21.15.16:0/1576377586 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d445549a0 0x558d4455eb40 secure :-1 s=READY pgs=2022 cs=0 l=1 rev1=1 crypto rx=0x7ff9bc00a8e0 tx=0x7ff9bc000bc0 comp rx=0 tx=0).stop 2024-09-12T02:16:51.394 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.363+0000 7ff9d4c79740 1 -- 172.21.15.16:0/1576377586 shutdown_connections 2024-09-12T02:16:51.395 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.363+0000 7ff9d4c79740 1 --2- 172.21.15.16:0/1576377586 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558d4455a370 0x558d4455f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:51.395 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.363+0000 7ff9d4c79740 1 --2- 172.21.15.16:0/1576377586 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558d445552e0 0x558d44566080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:51.395 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.363+0000 7ff9d4c79740 1 --2- 172.21.15.16:0/1576377586 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d445549a0 0x558d4455eb40 unknown :-1 s=CLOSED pgs=2022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:51.395 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.363+0000 7ff9d4c79740 1 -- 172.21.15.16:0/1576377586 >> 172.21.15.16:0/1576377586 conn(0x558d44550010 msgr2=0x558d44556480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:51.395 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.363+0000 7ff9d4c79740 1 -- 172.21.15.16:0/1576377586 shutdown_connections 2024-09-12T02:16:51.396 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.363+0000 7ff9d4c79740 1 -- 172.21.15.16:0/1576377586 wait complete. 2024-09-12T02:16:51.396 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.364+0000 7ff9d4c79740 1 -- start start 2024-09-12T02:16:51.396 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.364+0000 7ff9d4c79740 1 Processor -- start 2024-09-12T02:16:51.396 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.364+0000 7ff9d4c79740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d445549a0 0x558d44563210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:51.396 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.364+0000 7ff9d4c79740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558d445552e0 0x558d44563750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:51.397 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.364+0000 7ff9d4c79740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558d4455a370 0x558d44564760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:51.397 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.364+0000 7ff9d4c79740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff9cc1a74c0 con 0x558d445549a0 2024-09-12T02:16:51.397 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.364+0000 7ff9d4c79740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff9cc1a8930 con 0x558d4455a370 2024-09-12T02:16:51.397 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.364+0000 7ff9d4c79740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff9cc1a7b90 con 0x558d445552e0 2024-09-12T02:16:51.397 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.364+0000 7ff9d4c79740 10 client.0 _ll_register_callbacks cb 0x7ff9c40095e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:16:51.397 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.364+0000 7ff9cb7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d445549a0 0x558d44563210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:51.398 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.364+0000 7ff9cb7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d445549a0 0x558d44563210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58528/0 (socket says 172.21.15.16:58528) 2024-09-12T02:16:51.398 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.364+0000 7ff9cb7fe640 1 -- 172.21.15.16:0/2559981312 learned_addr learned my addr 172.21.15.16:0/2559981312 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:16:51.398 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.364+0000 7ff9bbfff640 1 --2- 172.21.15.16:0/2559981312 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558d445552e0 0x558d44563750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:51.398 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.364+0000 7ff9cbfff640 1 --2- 172.21.15.16:0/2559981312 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558d4455a370 0x558d44564760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:51.399 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.365+0000 7ff9cb7fe640 1 -- 172.21.15.16:0/2559981312 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558d445552e0 msgr2=0x558d44563750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:51.399 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.365+0000 7ff9cb7fe640 1 --2- 172.21.15.16:0/2559981312 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558d445552e0 0x558d44563750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:51.399 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.365+0000 7ff9cb7fe640 1 -- 172.21.15.16:0/2559981312 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558d4455a370 msgr2=0x558d44564760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:51.399 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.365+0000 7ff9cb7fe640 1 --2- 172.21.15.16:0/2559981312 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558d4455a370 0x558d44564760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:51.399 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.365+0000 7ff9cb7fe640 1 -- 172.21.15.16:0/2559981312 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d44564d70 con 0x558d445549a0 2024-09-12T02:16:51.399 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.365+0000 7ff9cbfff640 1 --2- 172.21.15.16:0/2559981312 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558d4455a370 0x558d44564760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:16:51.400 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.365+0000 7ff9bbfff640 1 --2- 172.21.15.16:0/2559981312 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558d445552e0 0x558d44563750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:16:51.400 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.365+0000 7ff9cb7fe640 1 --2- 172.21.15.16:0/2559981312 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d445549a0 0x558d44563210 secure :-1 s=READY pgs=2023 cs=0 l=1 rev1=1 crypto rx=0x7ff9bc02e040 tx=0x7ff9bc0042d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bda7f3b2c443da79 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:51.400 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.365+0000 7ff9aaffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:16:51.400 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.365+0000 7ff9aaffd640 1 -- 172.21.15.16:0/2559981312 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9bc038060 con 0x558d445549a0 2024-09-12T02:16:51.400 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.365+0000 7ff9aaffd640 1 -- 172.21.15.16:0/2559981312 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7ff9bc005030 con 0x558d445549a0 2024-09-12T02:16:51.401 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.365+0000 7ff9d4c79740 1 -- 172.21.15.16:0/2559981312 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9bc02f030 con 0x558d445549a0 2024-09-12T02:16:51.401 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.365+0000 7ff9aaffd640 1 -- 172.21.15.16:0/2559981312 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff9bc005230 con 0x558d445549a0 2024-09-12T02:16:51.401 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.365+0000 7ff9d4c79740 10 client.20508 fetch_fsmap learned FSMap version 25 2024-09-12T02:16:51.401 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.365+0000 7ff9d4c79740 1 -- 172.21.15.16:0/2559981312 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x558d44566080 con 0x558d445549a0 2024-09-12T02:16:51.401 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.365+0000 7ff9aaffd640 1 -- 172.21.15.16:0/2559981312 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9bc02f030 con 0x558d445549a0 2024-09-12T02:16:51.402 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.365+0000 7ff9aaffd640 1 -- 172.21.15.16:0/2559981312 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7ff9bc003680 con 0x558d445549a0 2024-09-12T02:16:51.402 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.365+0000 7ff9d4c79740 10 client.20508 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:16:51.402 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.365+0000 7ff9d4c79740 10 client.20508 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:16:51.402 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.365+0000 7ff9d4c79740 20 client.20508 populate_metadata read hostname 'smithi016' 2024-09-12T02:16:51.402 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.365+0000 7ff9d4c79740 1 --2- 172.21.15.16:0/2559981312 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x558d445d56f0 0x558d445d7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:51.402 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.366+0000 7ff9d4c79740 4 client.20508 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:16:51.403 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.366+0000 7ff9d4c79740 1 -- 172.21.15.16:0/2559981312 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558d444bfd70 con 0x558d445d56f0 2024-09-12T02:16:51.403 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.366+0000 7ff9bbfff640 1 --2- 172.21.15.16:0/2559981312 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x558d445d56f0 0x558d445d7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:51.403 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.366+0000 7ff9bbfff640 1 --2- 172.21.15.16:0/2559981312 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x558d445d56f0 0x558d445d7ae0 crc :-1 s=READY pgs=448 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62bff29f44a52c7f server_cookie=1b9351595b36ae4f in_seq=0 out_seq=0 2024-09-12T02:16:51.403 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.366+0000 7ff9aaffd640 10 client.20508 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:16:51.404 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.379+0000 7ff9aaffd640 1 -- 172.21.15.16:0/2559981312 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558d444bfd70 con 0x558d445d56f0 2024-09-12T02:16:51.404 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.379+0000 7ff9aaffd640 10 client.20508 handle_command_reply: tid=0 2024-09-12T02:16:51.404 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.379+0000 7ff9d4c79740 1 client.20508 shutdown 2024-09-12T02:16:51.404 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.379+0000 7ff9d4c79740 1 -- 172.21.15.16:0/2559981312 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d445549a0 msgr2=0x558d44563210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:51.404 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.379+0000 7ff9d4c79740 1 --2- 172.21.15.16:0/2559981312 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d445549a0 0x558d44563210 secure :-1 s=READY pgs=2023 cs=0 l=1 rev1=1 crypto rx=0x7ff9bc02e040 tx=0x7ff9bc0042d0 comp rx=0 tx=0).stop 2024-09-12T02:16:51.404 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.380+0000 7ff9d4c79740 1 -- 172.21.15.16:0/2559981312 shutdown_connections 2024-09-12T02:16:51.405 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.380+0000 7ff9d4c79740 1 --2- 172.21.15.16:0/2559981312 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x558d445d56f0 0x558d445d7ae0 crc :-1 s=READY pgs=448 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:51.405 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.380+0000 7ff9d4c79740 1 --2- 172.21.15.16:0/2559981312 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558d4455a370 0x558d44564760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:51.405 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.380+0000 7ff9d4c79740 1 --2- 172.21.15.16:0/2559981312 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558d445552e0 0x558d44563750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:51.405 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.380+0000 7ff9aaffd640 0 client.20508 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:16:51.405 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.380+0000 7ff9d4c79740 1 --2- 172.21.15.16:0/2559981312 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d445549a0 0x558d44563210 unknown :-1 s=CLOSED pgs=2023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:51.406 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.380+0000 7ff9d4c79740 1 -- 172.21.15.16:0/2559981312 >> 172.21.15.16:0/2559981312 conn(0x558d44550010 msgr2=0x558d44559620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:51.406 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.380+0000 7ff9d4c79740 1 -- 172.21.15.16:0/2559981312 shutdown_connections 2024-09-12T02:16:51.406 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.380+0000 7ff9d4c79740 1 -- 172.21.15.16:0/2559981312 wait complete. 2024-09-12T02:16:51.406 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.380+0000 7ff9d4c79740 20 client.20508 trim_cache size 0 max 16384 2024-09-12T02:16:51.407 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-09-12T02:16:51.407 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-09-12T02:16:51.407 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-09-12T02:16:51.407 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-09-12T02:16:51.409 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.380+0000 7ff9d1ee4640 1 -- 172.21.15.16:0/1072099128 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7ff9b40a2cb0 msgr2=0x7ff9b40a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:51.409 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.380+0000 7ff9d1ee4640 1 --2- 172.21.15.16:0/1072099128 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7ff9b40a2cb0 0x7ff9b40a5170 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7ff9cc06abe0 tx=0x7ff9c0004730 comp rx=0 tx=0).stop 2024-09-12T02:16:51.409 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.380+0000 7ff9d1ee4640 1 -- 172.21.15.16:0/1072099128 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9cc102610 msgr2=0x7ff9cc19f0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:51.409 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.380+0000 7ff9d1ee4640 1 --2- 172.21.15.16:0/1072099128 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9cc102610 0x7ff9cc19f0f0 secure :-1 s=READY pgs=2019 cs=0 l=1 rev1=1 crypto rx=0x7ff9c4005e30 tx=0x7ff9c4031ee0 comp rx=0 tx=0).stop 2024-09-12T02:16:51.409 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.380+0000 7ff9d1ee4640 1 -- 172.21.15.16:0/1072099128 shutdown_connections 2024-09-12T02:16:51.410 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.380+0000 7ff9d1ee4640 1 --2- 172.21.15.16:0/1072099128 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7ff9b40a2cb0 0x7ff9b40a5170 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:51.410 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.380+0000 7ff9d1ee4640 1 --2- 172.21.15.16:0/1072099128 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9cc102610 0x7ff9cc19f0f0 unknown :-1 s=CLOSED pgs=2019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:51.410 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.380+0000 7ff9d1ee4640 1 --2- 172.21.15.16:0/1072099128 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff9cc101c50 0x7ff9cc19cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:51.410 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.380+0000 7ff9d1ee4640 1 --2- 172.21.15.16:0/1072099128 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff9cc100a60 0x7ff9cc06ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:51.410 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.380+0000 7ff9d1ee4640 1 -- 172.21.15.16:0/1072099128 >> 172.21.15.16:0/1072099128 conn(0x7ff9cc0fc1e0 msgr2=0x7ff9cc104f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:51.411 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.380+0000 7ff9d1ee4640 1 -- 172.21.15.16:0/1072099128 shutdown_connections 2024-09-12T02:16:51.411 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:16:51.380+0000 7ff9d1ee4640 1 -- 172.21.15.16:0/1072099128 wait complete. 2024-09-12T02:16:51.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:16:51 smithi016 ceph-mon[30947]: scrub queued for path: / 2024-09-12T02:16:51.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:16:51 smithi016 ceph-mon[30947]: scrub summary: idle+waiting paths [/] 2024-09-12T02:16:51.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:16:51 smithi016 ceph-mon[30947]: scrub summary: active paths [/] 2024-09-12T02:16:51.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:16:51 smithi016 ceph-mon[30947]: scrub summary: idle+waiting paths [/] 2024-09-12T02:16:51.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:16:51 smithi016 ceph-mon[30947]: scrub summary: idle 2024-09-12T02:16:51.767 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T02:16:51.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:16:51 smithi184 ceph-mon[35593]: scrub queued for path: / 2024-09-12T02:16:51.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:16:51 smithi184 ceph-mon[35593]: scrub summary: idle+waiting paths [/] 2024-09-12T02:16:51.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:16:51 smithi184 ceph-mon[35593]: scrub summary: active paths [/] 2024-09-12T02:16:51.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:16:51 smithi184 ceph-mon[35593]: scrub summary: idle+waiting paths [/] 2024-09-12T02:16:51.878 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:16:51 smithi184 ceph-mon[35593]: scrub summary: idle 2024-09-12T02:16:51.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:16:51 smithi033 ceph-mon[35332]: scrub queued for path: / 2024-09-12T02:16:51.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:16:51 smithi033 ceph-mon[35332]: scrub summary: idle+waiting paths [/] 2024-09-12T02:16:51.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:16:51 smithi033 ceph-mon[35332]: scrub summary: active paths [/] 2024-09-12T02:16:51.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:16:51 smithi033 ceph-mon[35332]: scrub summary: idle+waiting paths [/] 2024-09-12T02:16:51.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:16:51 smithi033 ceph-mon[35332]: scrub summary: idle 2024-09-12T02:16:52.021 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:16:52.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.476+0000 7f1f72d66640 1 -- 172.21.15.16:0/2248028954 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f6c102790 msgr2=0x7f1f6c1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:52.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.476+0000 7f1f72d66640 1 --2- 172.21.15.16:0/2248028954 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f6c102790 0x7f1f6c1067e0 secure :-1 s=READY pgs=2024 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c009900 tx=0x7f1f5c02a410 comp rx=0 tx=0).stop 2024-09-12T02:16:52.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.476+0000 7f1f72d66640 1 -- 172.21.15.16:0/2248028954 shutdown_connections 2024-09-12T02:16:52.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.476+0000 7f1f72d66640 1 --2- 172.21.15.16:0/2248028954 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f6c106f10 0x7f1f6c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:52.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.476+0000 7f1f72d66640 1 --2- 172.21.15.16:0/2248028954 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f6c102790 0x7f1f6c1067e0 unknown :-1 s=CLOSED pgs=2024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:52.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.476+0000 7f1f72d66640 1 --2- 172.21.15.16:0/2248028954 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1f6c101de0 0x7f1f6c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:52.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.476+0000 7f1f72d66640 1 -- 172.21.15.16:0/2248028954 >> 172.21.15.16:0/2248028954 conn(0x7f1f6c0fd8c0 msgr2=0x7f1f6c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:52.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.476+0000 7f1f72d66640 1 -- 172.21.15.16:0/2248028954 shutdown_connections 2024-09-12T02:16:52.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.476+0000 7f1f72d66640 1 -- 172.21.15.16:0/2248028954 wait complete. 2024-09-12T02:16:52.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.477+0000 7f1f72d66640 1 Processor -- start 2024-09-12T02:16:52.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.477+0000 7f1f72d66640 1 -- start start 2024-09-12T02:16:52.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.477+0000 7f1f72d66640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f6c101de0 0x7f1f6c1984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:52.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.477+0000 7f1f72d66640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f6c102790 0x7f1f6c1989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:52.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.477+0000 7f1f72d66640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1f6c106f10 0x7f1f6c19cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:52.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.477+0000 7f1f72d66640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1f6c111fb0 con 0x7f1f6c101de0 2024-09-12T02:16:52.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.477+0000 7f1f72d66640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1f6c111e30 con 0x7f1f6c102790 2024-09-12T02:16:52.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.477+0000 7f1f72d66640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1f6c112130 con 0x7f1f6c106f10 2024-09-12T02:16:52.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.477+0000 7f1f70b02640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f6c101de0 0x7f1f6c1984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:52.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.477+0000 7f1f70b02640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f6c101de0 0x7f1f6c1984b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58540/0 (socket says 172.21.15.16:58540) 2024-09-12T02:16:52.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.477+0000 7f1f70b02640 1 -- 172.21.15.16:0/393012148 learned_addr learned my addr 172.21.15.16:0/393012148 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:16:52.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.478+0000 7f1f71303640 1 --2- 172.21.15.16:0/393012148 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1f6c106f10 0x7f1f6c19cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:52.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.478+0000 7f1f63fff640 1 --2- 172.21.15.16:0/393012148 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f6c102790 0x7f1f6c1989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:52.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.478+0000 7f1f70b02640 1 -- 172.21.15.16:0/393012148 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1f6c106f10 msgr2=0x7f1f6c19cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:52.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.478+0000 7f1f70b02640 1 --2- 172.21.15.16:0/393012148 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1f6c106f10 0x7f1f6c19cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:52.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.478+0000 7f1f70b02640 1 -- 172.21.15.16:0/393012148 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f6c102790 msgr2=0x7f1f6c1989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:52.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.478+0000 7f1f70b02640 1 --2- 172.21.15.16:0/393012148 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f6c102790 0x7f1f6c1989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:52.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.478+0000 7f1f70b02640 1 -- 172.21.15.16:0/393012148 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f6c19d460 con 0x7f1f6c101de0 2024-09-12T02:16:52.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.478+0000 7f1f71303640 1 --2- 172.21.15.16:0/393012148 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1f6c106f10 0x7f1f6c19cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:16:52.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.478+0000 7f1f70b02640 1 --2- 172.21.15.16:0/393012148 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f6c101de0 0x7f1f6c1984b0 secure :-1 s=READY pgs=2025 cs=0 l=1 rev1=1 crypto rx=0x7f1f6800d770 tx=0x7f1f6800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:52.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.478+0000 7f1f61ffb640 1 -- 172.21.15.16:0/393012148 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f68017070 con 0x7f1f6c101de0 2024-09-12T02:16:52.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.478+0000 7f1f61ffb640 1 -- 172.21.15.16:0/393012148 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f68005500 con 0x7f1f6c101de0 2024-09-12T02:16:52.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.478+0000 7f1f61ffb640 1 -- 172.21.15.16:0/393012148 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f68004e00 con 0x7f1f6c101de0 2024-09-12T02:16:52.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.478+0000 7f1f72d66640 1 -- 172.21.15.16:0/393012148 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f6c19d750 con 0x7f1f6c101de0 2024-09-12T02:16:52.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.478+0000 7f1f72d66640 1 -- 172.21.15.16:0/393012148 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f6c1a5030 con 0x7f1f6c101de0 2024-09-12T02:16:52.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.480+0000 7f1f61ffb640 1 -- 172.21.15.16:0/393012148 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f1f68004fa0 con 0x7f1f6c101de0 2024-09-12T02:16:52.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.481+0000 7f1f7513f740 1 Processor -- start 2024-09-12T02:16:52.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.481+0000 7f1f7513f740 1 -- start start 2024-09-12T02:16:52.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.481+0000 7f1f7513f740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562072c854a0 0x562072c85880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:52.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.481+0000 7f1f7513f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562072c85dc0 0x562072c90140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:52.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.481+0000 7f1f7513f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562072c90680 0x562072c92a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:52.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.481+0000 7f1f7513f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x562072beb2e0 con 0x562072c85dc0 2024-09-12T02:16:52.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.481+0000 7f1f7513f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x562072b92ce0 con 0x562072c90680 2024-09-12T02:16:52.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.481+0000 7f1f7513f740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x562072be72a0 con 0x562072c854a0 2024-09-12T02:16:52.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.481+0000 7f1f61ffb640 1 --2- 172.21.15.16:0/393012148 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f1f540a2c00 0x7f1f540a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:52.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.481+0000 7f1f70b02640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562072c85dc0 0x562072c90140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:52.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.481+0000 7f1f71303640 1 --2- 172.21.15.16:0/393012148 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f1f540a2c00 0x7f1f540a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:52.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.481+0000 7f1f70b02640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562072c85dc0 0x562072c90140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58546/0 (socket says 172.21.15.16:58546) 2024-09-12T02:16:52.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.481+0000 7f1f70b02640 1 -- 172.21.15.16:0/3012998187 learned_addr learned my addr 172.21.15.16:0/3012998187 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:16:52.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.482+0000 7f1f63fff640 1 --2- 172.21.15.16:0/3012998187 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562072c90680 0x562072c92a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:52.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.482+0000 7f1f71303640 1 --2- 172.21.15.16:0/3012998187 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562072c854a0 0x562072c85880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:52.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.482+0000 7f1f61ffb640 1 -- 172.21.15.16:0/393012148 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f1f680cea20 con 0x7f1f6c101de0 2024-09-12T02:16:52.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.482+0000 7f1f70b02640 1 -- 172.21.15.16:0/3012998187 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562072c854a0 msgr2=0x562072c85880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:52.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.482+0000 7f1f70b02640 1 --2- 172.21.15.16:0/3012998187 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562072c854a0 0x562072c85880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:52.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.482+0000 7f1f70b02640 1 -- 172.21.15.16:0/3012998187 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562072c90680 msgr2=0x562072c92a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:52.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.482+0000 7f1f70b02640 1 --2- 172.21.15.16:0/3012998187 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562072c90680 0x562072c92a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:52.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.482+0000 7f1f70b02640 1 -- 172.21.15.16:0/3012998187 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562072b77440 con 0x562072c85dc0 2024-09-12T02:16:52.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.482+0000 7f1f71303640 1 --2- 172.21.15.16:0/3012998187 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562072c854a0 0x562072c85880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:16:52.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.482+0000 7f1f70b02640 1 --2- 172.21.15.16:0/3012998187 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562072c85dc0 0x562072c90140 secure :-1 s=READY pgs=2026 cs=0 l=1 rev1=1 crypto rx=0x7f1f6806e3b0 tx=0x7f1f680c5eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb976ce956851e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:52.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.482+0000 7f1f71303640 1 --2- 172.21.15.16:0/393012148 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f1f540a2c00 0x7f1f540a50c0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f1f64009440 tx=0x7f1f64009600 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:52.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.482+0000 7f1f52ffd640 1 -- 172.21.15.16:0/3012998187 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f68020070 con 0x562072c85dc0 2024-09-12T02:16:52.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.482+0000 7f1f52ffd640 1 -- 172.21.15.16:0/3012998187 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f680046c0 con 0x562072c85dc0 2024-09-12T02:16:52.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.482+0000 7f1f52ffd640 1 -- 172.21.15.16:0/3012998187 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f6809b320 con 0x562072c85dc0 2024-09-12T02:16:52.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.483+0000 7f1f7513f740 1 -- 172.21.15.16:0/3012998187 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562072c85dc0 msgr2=0x562072c90140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:52.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.483+0000 7f1f7513f740 1 --2- 172.21.15.16:0/3012998187 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562072c85dc0 0x562072c90140 secure :-1 s=READY pgs=2026 cs=0 l=1 rev1=1 crypto rx=0x7f1f6806e3b0 tx=0x7f1f680c5eb0 comp rx=0 tx=0).stop 2024-09-12T02:16:52.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.483+0000 7f1f7513f740 1 -- 172.21.15.16:0/3012998187 shutdown_connections 2024-09-12T02:16:52.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.483+0000 7f1f7513f740 1 --2- 172.21.15.16:0/3012998187 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562072c90680 0x562072c92a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:52.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.483+0000 7f1f7513f740 1 --2- 172.21.15.16:0/3012998187 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562072c85dc0 0x562072c90140 unknown :-1 s=CLOSED pgs=2026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:52.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.483+0000 7f1f7513f740 1 --2- 172.21.15.16:0/3012998187 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562072c854a0 0x562072c85880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:52.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.483+0000 7f1f7513f740 1 -- 172.21.15.16:0/3012998187 >> 172.21.15.16:0/3012998187 conn(0x562072c81a30 msgr2=0x562072c81e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:52.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.483+0000 7f1f7513f740 1 -- 172.21.15.16:0/3012998187 shutdown_connections 2024-09-12T02:16:52.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.483+0000 7f1f7513f740 1 -- 172.21.15.16:0/3012998187 wait complete. 2024-09-12T02:16:52.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.483+0000 7f1f7513f740 1 -- start start 2024-09-12T02:16:52.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.484+0000 7f1f7513f740 1 Processor -- start 2024-09-12T02:16:52.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.484+0000 7f1f7513f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562072c854a0 0x562072c90030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:52.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.484+0000 7f1f7513f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562072c85dc0 0x562072c8d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:52.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.484+0000 7f1f7513f740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562072c90680 0x562072c8d880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:52.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.484+0000 7f1f7513f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x562072c93250 con 0x562072c85dc0 2024-09-12T02:16:52.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.484+0000 7f1f7513f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x562072ba6050 con 0x562072c854a0 2024-09-12T02:16:52.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.484+0000 7f1f7513f740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x562072c933d0 con 0x562072c90680 2024-09-12T02:16:52.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.484+0000 7f1f7513f740 10 client.0 _ll_register_callbacks cb 0x562072be6160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:16:52.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.484+0000 7f1f63fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562072c85dc0 0x562072c8d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:52.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.484+0000 7f1f63fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562072c85dc0 0x562072c8d2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58560/0 (socket says 172.21.15.16:58560) 2024-09-12T02:16:52.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.484+0000 7f1f63fff640 1 -- 172.21.15.16:0/2850900609 learned_addr learned my addr 172.21.15.16:0/2850900609 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:16:52.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.484+0000 7f1f70b02640 1 --2- 172.21.15.16:0/2850900609 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562072c854a0 0x562072c90030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:52.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.484+0000 7f1f71303640 1 --2- 172.21.15.16:0/2850900609 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562072c90680 0x562072c8d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:52.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.484+0000 7f1f63fff640 1 -- 172.21.15.16:0/2850900609 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562072c90680 msgr2=0x562072c8d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:52.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.484+0000 7f1f63fff640 1 --2- 172.21.15.16:0/2850900609 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562072c90680 0x562072c8d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:52.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.484+0000 7f1f63fff640 1 -- 172.21.15.16:0/2850900609 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562072c854a0 msgr2=0x562072c90030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:52.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.484+0000 7f1f63fff640 1 --2- 172.21.15.16:0/2850900609 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562072c854a0 0x562072c90030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:52.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.484+0000 7f1f63fff640 1 -- 172.21.15.16:0/2850900609 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56207295c3c0 con 0x562072c85dc0 2024-09-12T02:16:52.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.485+0000 7f1f63fff640 1 --2- 172.21.15.16:0/2850900609 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562072c85dc0 0x562072c8d2e0 secure :-1 s=READY pgs=2027 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c009900 tx=0x7f1f5c031a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db5bb0e15407f72e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:52.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.485+0000 7f1f4bfff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:16:52.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.485+0000 7f1f4bfff640 1 -- 172.21.15.16:0/2850900609 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f5c035040 con 0x562072c85dc0 2024-09-12T02:16:52.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.485+0000 7f1f4bfff640 1 -- 172.21.15.16:0/2850900609 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f1f5c004240 con 0x562072c85dc0 2024-09-12T02:16:52.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.485+0000 7f1f7513f740 1 -- 172.21.15.16:0/2850900609 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x562072bf8850 con 0x562072c85dc0 2024-09-12T02:16:52.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.485+0000 7f1f4bfff640 1 -- 172.21.15.16:0/2850900609 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1f5c005d50 con 0x562072c85dc0 2024-09-12T02:16:52.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.485+0000 7f1f7513f740 10 client.20520 fetch_fsmap learned FSMap version 25 2024-09-12T02:16:52.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.485+0000 7f1f7513f740 1 -- 172.21.15.16:0/2850900609 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x562072c9ecb0 con 0x562072c85dc0 2024-09-12T02:16:52.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.485+0000 7f1f4bfff640 1 -- 172.21.15.16:0/2850900609 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x562072bf8850 con 0x562072c85dc0 2024-09-12T02:16:52.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.485+0000 7f1f4bfff640 1 -- 172.21.15.16:0/2850900609 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f1f5c004be0 con 0x562072c85dc0 2024-09-12T02:16:52.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.486+0000 7f1f7513f740 10 client.20520 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:16:52.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.486+0000 7f1f7513f740 10 client.20520 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:16:52.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.486+0000 7f1f7513f740 20 client.20520 populate_metadata read hostname 'smithi016' 2024-09-12T02:16:52.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.486+0000 7f1f7513f740 1 --2- 172.21.15.16:0/2850900609 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x562072ca01c0 0x562072ca25f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:52.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.486+0000 7f1f7513f740 4 client.20520 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:16:52.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.486+0000 7f1f7513f740 1 -- 172.21.15.16:0/2850900609 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562072bf2d60 con 0x562072ca01c0 2024-09-12T02:16:52.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.486+0000 7f1f70b02640 1 --2- 172.21.15.16:0/2850900609 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x562072ca01c0 0x562072ca25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:52.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.486+0000 7f1f70b02640 1 --2- 172.21.15.16:0/2850900609 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x562072ca01c0 0x562072ca25f0 crc :-1 s=READY pgs=449 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb45825d33d3ebf server_cookie=98e19ed4a58150ef in_seq=0 out_seq=0 2024-09-12T02:16:52.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.486+0000 7f1f4bfff640 10 client.20520 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:16:52.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.487+0000 7f1f4bfff640 1 -- 172.21.15.16:0/2850900609 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562072bf2d60 con 0x562072ca01c0 2024-09-12T02:16:52.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.487+0000 7f1f4bfff640 10 client.20520 handle_command_reply: tid=0 2024-09-12T02:16:52.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.487+0000 7f1f7513f740 1 client.20520 shutdown 2024-09-12T02:16:52.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.487+0000 7f1f7513f740 1 -- 172.21.15.16:0/2850900609 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562072c85dc0 msgr2=0x562072c8d2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:52.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.487+0000 7f1f7513f740 1 --2- 172.21.15.16:0/2850900609 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562072c85dc0 0x562072c8d2e0 secure :-1 s=READY pgs=2027 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c009900 tx=0x7f1f5c031a20 comp rx=0 tx=0).stop 2024-09-12T02:16:52.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.487+0000 7f1f7513f740 1 -- 172.21.15.16:0/2850900609 shutdown_connections 2024-09-12T02:16:52.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.487+0000 7f1f7513f740 1 --2- 172.21.15.16:0/2850900609 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x562072ca01c0 0x562072ca25f0 crc :-1 s=READY pgs=449 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:52.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.487+0000 7f1f7513f740 1 --2- 172.21.15.16:0/2850900609 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562072c90680 0x562072c8d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:52.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.487+0000 7f1f7513f740 1 --2- 172.21.15.16:0/2850900609 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562072c85dc0 0x562072c8d2e0 unknown :-1 s=CLOSED pgs=2027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:52.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.487+0000 7f1f7513f740 1 --2- 172.21.15.16:0/2850900609 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562072c854a0 0x562072c90030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:52.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.487+0000 7f1f4bfff640 0 client.20520 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:16:52.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.487+0000 7f1f7513f740 1 -- 172.21.15.16:0/2850900609 >> 172.21.15.16:0/2850900609 conn(0x562072c81a30 msgr2=0x562072c87760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:52.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.487+0000 7f1f7513f740 1 -- 172.21.15.16:0/2850900609 shutdown_connections 2024-09-12T02:16:52.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.487+0000 7f1f7513f740 1 -- 172.21.15.16:0/2850900609 wait complete. 2024-09-12T02:16:52.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.487+0000 7f1f7513f740 20 client.20520 trim_cache size 0 max 16384 2024-09-12T02:16:52.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.505+0000 7f1f7513f740 1 Processor -- start 2024-09-12T02:16:52.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.505+0000 7f1f7513f740 1 -- start start 2024-09-12T02:16:52.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.505+0000 7f1f7513f740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562072c854a0 0x562072c8bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:52.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.505+0000 7f1f7513f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562072c85dc0 0x562072ca7f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:52.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.505+0000 7f1f7513f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562072c90680 0x562072ca8440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:52.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.505+0000 7f1f7513f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1f6c1afcc0 con 0x562072c90680 2024-09-12T02:16:52.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.505+0000 7f1f7513f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1f6c1ad590 con 0x562072c85dc0 2024-09-12T02:16:52.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.505+0000 7f1f7513f740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1f6c1ab400 con 0x562072c854a0 2024-09-12T02:16:52.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.505+0000 7f1f71303640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562072c90680 0x562072ca8440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:52.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.505+0000 7f1f71303640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562072c90680 0x562072ca8440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58566/0 (socket says 172.21.15.16:58566) 2024-09-12T02:16:52.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.505+0000 7f1f71303640 1 -- 172.21.15.16:0/2404701276 learned_addr learned my addr 172.21.15.16:0/2404701276 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:16:52.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.505+0000 7f1f63fff640 1 --2- 172.21.15.16:0/2404701276 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562072c85dc0 0x562072ca7f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:52.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.505+0000 7f1f70b02640 1 --2- 172.21.15.16:0/2404701276 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562072c854a0 0x562072c8bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:52.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.505+0000 7f1f71303640 1 -- 172.21.15.16:0/2404701276 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562072c854a0 msgr2=0x562072c8bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:52.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.505+0000 7f1f71303640 1 --2- 172.21.15.16:0/2404701276 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562072c854a0 0x562072c8bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:52.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.506+0000 7f1f71303640 1 -- 172.21.15.16:0/2404701276 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562072c85dc0 msgr2=0x562072ca7f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:52.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.506+0000 7f1f71303640 1 --2- 172.21.15.16:0/2404701276 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562072c85dc0 0x562072ca7f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:52.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.506+0000 7f1f71303640 1 -- 172.21.15.16:0/2404701276 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562072c8c290 con 0x562072c90680 2024-09-12T02:16:52.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.506+0000 7f1f63fff640 1 --2- 172.21.15.16:0/2404701276 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562072c85dc0 0x562072ca7f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:16:52.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.506+0000 7f1f71303640 1 --2- 172.21.15.16:0/2404701276 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562072c90680 0x562072ca8440 secure :-1 s=READY pgs=2028 cs=0 l=1 rev1=1 crypto rx=0x7f1f64014840 tx=0x7f1f64011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41d61d3b98dc782 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:52.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.506+0000 7f1f527fc640 1 -- 172.21.15.16:0/2404701276 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f64013040 con 0x562072c90680 2024-09-12T02:16:52.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.506+0000 7f1f527fc640 1 -- 172.21.15.16:0/2404701276 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f64029070 con 0x562072c90680 2024-09-12T02:16:52.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.506+0000 7f1f527fc640 1 -- 172.21.15.16:0/2404701276 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f64024aa0 con 0x562072c90680 2024-09-12T02:16:52.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.507+0000 7f1f7513f740 1 -- 172.21.15.16:0/2404701276 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562072c90680 msgr2=0x562072ca8440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:52.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.507+0000 7f1f7513f740 1 --2- 172.21.15.16:0/2404701276 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562072c90680 0x562072ca8440 secure :-1 s=READY pgs=2028 cs=0 l=1 rev1=1 crypto rx=0x7f1f64014840 tx=0x7f1f64011000 comp rx=0 tx=0).stop 2024-09-12T02:16:52.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.507+0000 7f1f7513f740 1 -- 172.21.15.16:0/2404701276 shutdown_connections 2024-09-12T02:16:52.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.507+0000 7f1f7513f740 1 --2- 172.21.15.16:0/2404701276 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562072c90680 0x562072ca8440 unknown :-1 s=CLOSED pgs=2028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:52.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.507+0000 7f1f7513f740 1 --2- 172.21.15.16:0/2404701276 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562072c85dc0 0x562072ca7f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:52.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.507+0000 7f1f7513f740 1 --2- 172.21.15.16:0/2404701276 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562072c854a0 0x562072c8bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:52.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.507+0000 7f1f7513f740 1 -- 172.21.15.16:0/2404701276 >> 172.21.15.16:0/2404701276 conn(0x562072c81a30 msgr2=0x562072c8f360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:52.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.507+0000 7f1f7513f740 1 -- 172.21.15.16:0/2404701276 shutdown_connections 2024-09-12T02:16:52.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.507+0000 7f1f7513f740 1 -- 172.21.15.16:0/2404701276 wait complete. 2024-09-12T02:16:52.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.507+0000 7f1f7513f740 1 -- start start 2024-09-12T02:16:52.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.507+0000 7f1f7513f740 1 Processor -- start 2024-09-12T02:16:52.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.508+0000 7f1f7513f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562072c854a0 0x562072d05ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:52.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.508+0000 7f1f7513f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562072c85dc0 0x562072d06400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:52.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.508+0000 7f1f7513f740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562072c90680 0x562072d06940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:52.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.508+0000 7f1f7513f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1f6c1aba50 con 0x562072c85dc0 2024-09-12T02:16:52.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.508+0000 7f1f7513f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1f6c1acec0 con 0x562072c854a0 2024-09-12T02:16:52.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.508+0000 7f1f7513f740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1f6c1ac120 con 0x562072c90680 2024-09-12T02:16:52.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.508+0000 7f1f7513f740 10 client.0 _ll_register_callbacks cb 0x7f1f5c0314d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:16:52.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.508+0000 7f1f63fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562072c85dc0 0x562072d06400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:52.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.508+0000 7f1f63fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562072c85dc0 0x562072d06400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58574/0 (socket says 172.21.15.16:58574) 2024-09-12T02:16:52.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.508+0000 7f1f63fff640 1 -- 172.21.15.16:0/2460050475 learned_addr learned my addr 172.21.15.16:0/2460050475 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:16:52.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.508+0000 7f1f70b02640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562072c854a0 0x562072d05ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:52.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.508+0000 7f1f71303640 1 --2- 172.21.15.16:0/2460050475 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562072c90680 0x562072d06940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:52.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.508+0000 7f1f63fff640 1 -- 172.21.15.16:0/2460050475 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562072c90680 msgr2=0x562072d06940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:52.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.508+0000 7f1f63fff640 1 --2- 172.21.15.16:0/2460050475 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562072c90680 0x562072d06940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:52.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.508+0000 7f1f63fff640 1 -- 172.21.15.16:0/2460050475 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562072c854a0 msgr2=0x562072d05ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:52.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.508+0000 7f1f63fff640 1 --2- 172.21.15.16:0/2460050475 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562072c854a0 0x562072d05ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:52.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.508+0000 7f1f63fff640 1 -- 172.21.15.16:0/2460050475 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562072c8dbf0 con 0x562072c85dc0 2024-09-12T02:16:52.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.508+0000 7f1f70b02640 1 --2- 172.21.15.16:0/2460050475 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562072c854a0 0x562072d05ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:16:52.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.508+0000 7f1f63fff640 1 --2- 172.21.15.16:0/2460050475 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562072c85dc0 0x562072d06400 secure :-1 s=READY pgs=2029 cs=0 l=1 rev1=1 crypto rx=0x562072c8e920 tx=0x7f1f5c005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5e8b7d1eb59049c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:52.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.508+0000 7f1f4bfff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:16:52.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.508+0000 7f1f4bfff640 1 -- 172.21.15.16:0/2460050475 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f5c035040 con 0x562072c85dc0 2024-09-12T02:16:52.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.508+0000 7f1f4bfff640 1 -- 172.21.15.16:0/2460050475 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f1f5c0046c0 con 0x562072c85dc0 2024-09-12T02:16:52.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.509+0000 7f1f7513f740 1 -- 172.21.15.16:0/2460050475 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f64029070 con 0x562072c85dc0 2024-09-12T02:16:52.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.509+0000 7f1f4bfff640 1 -- 172.21.15.16:0/2460050475 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1f5c009520 con 0x562072c85dc0 2024-09-12T02:16:52.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.509+0000 7f1f7513f740 10 client.20526 fetch_fsmap learned FSMap version 25 2024-09-12T02:16:52.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.509+0000 7f1f7513f740 1 -- 172.21.15.16:0/2460050475 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x562072d0f540 con 0x562072c85dc0 2024-09-12T02:16:52.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.509+0000 7f1f4bfff640 1 -- 172.21.15.16:0/2460050475 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f64029070 con 0x562072c85dc0 2024-09-12T02:16:52.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.509+0000 7f1f4bfff640 1 -- 172.21.15.16:0/2460050475 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f1f5c004110 con 0x562072c85dc0 2024-09-12T02:16:52.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.509+0000 7f1f7513f740 10 client.20526 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:16:52.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.509+0000 7f1f7513f740 10 client.20526 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:16:52.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.509+0000 7f1f7513f740 20 client.20526 populate_metadata read hostname 'smithi016' 2024-09-12T02:16:52.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.509+0000 7f1f7513f740 1 --2- 172.21.15.16:0/2460050475 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x562072d0fcf0 0x562072d12110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:52.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.509+0000 7f1f7513f740 4 client.20526 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T02:16:52.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.509+0000 7f1f7513f740 1 -- 172.21.15.16:0/2460050475 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562072b6f210 con 0x562072d0fcf0 2024-09-12T02:16:52.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.509+0000 7f1f70b02640 1 --2- 172.21.15.16:0/2460050475 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x562072d0fcf0 0x562072d12110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:52.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.510+0000 7f1f4bfff640 10 client.20526 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:16:52.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.510+0000 7f1f70b02640 1 --2- 172.21.15.16:0/2460050475 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x562072d0fcf0 0x562072d12110 crc :-1 s=READY pgs=450 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88dc2e583b5ec55d server_cookie=de4133a41c498f85 in_seq=0 out_seq=0 2024-09-12T02:16:52.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.510+0000 7f1f4bfff640 1 -- 172.21.15.16:0/2460050475 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x562072b6f210 con 0x562072d0fcf0 2024-09-12T02:16:52.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.510+0000 7f1f4bfff640 10 client.20526 handle_command_reply: tid=0 2024-09-12T02:16:52.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.510+0000 7f1f7513f740 1 client.20526 shutdown 2024-09-12T02:16:52.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.510+0000 7f1f7513f740 1 -- 172.21.15.16:0/2460050475 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562072c85dc0 msgr2=0x562072d06400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:52.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.510+0000 7f1f7513f740 1 --2- 172.21.15.16:0/2460050475 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562072c85dc0 0x562072d06400 secure :-1 s=READY pgs=2029 cs=0 l=1 rev1=1 crypto rx=0x562072c8e920 tx=0x7f1f5c005fd0 comp rx=0 tx=0).stop 2024-09-12T02:16:52.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.510+0000 7f1f7513f740 1 -- 172.21.15.16:0/2460050475 shutdown_connections 2024-09-12T02:16:52.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.510+0000 7f1f7513f740 1 --2- 172.21.15.16:0/2460050475 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x562072d0fcf0 0x562072d12110 crc :-1 s=READY pgs=450 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:52.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.510+0000 7f1f7513f740 1 --2- 172.21.15.16:0/2460050475 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562072c90680 0x562072d06940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:52.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.510+0000 7f1f7513f740 1 --2- 172.21.15.16:0/2460050475 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562072c85dc0 0x562072d06400 unknown :-1 s=CLOSED pgs=2029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:52.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.510+0000 7f1f7513f740 1 --2- 172.21.15.16:0/2460050475 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562072c854a0 0x562072d05ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:52.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.510+0000 7f1f7513f740 1 -- 172.21.15.16:0/2460050475 >> 172.21.15.16:0/2460050475 conn(0x562072c81a30 msgr2=0x562072c8f360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:52.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.510+0000 7f1f4bfff640 0 client.20526 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:16:52.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.510+0000 7f1f7513f740 1 -- 172.21.15.16:0/2460050475 shutdown_connections 2024-09-12T02:16:52.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.510+0000 7f1f7513f740 1 -- 172.21.15.16:0/2460050475 wait complete. 2024-09-12T02:16:52.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.510+0000 7f1f7513f740 20 client.20526 trim_cache size 0 max 16384 2024-09-12T02:16:52.518 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T02:16:52.519 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-09-12T02:16:52.519 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {} 2024-09-12T02:16:52.519 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T02:16:52.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.511+0000 7f1f72d66640 1 -- 172.21.15.16:0/393012148 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f1f540a2c00 msgr2=0x7f1f540a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:52.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.511+0000 7f1f72d66640 1 --2- 172.21.15.16:0/393012148 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f1f540a2c00 0x7f1f540a50c0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f1f64009440 tx=0x7f1f64009600 comp rx=0 tx=0).stop 2024-09-12T02:16:52.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.511+0000 7f1f72d66640 1 -- 172.21.15.16:0/393012148 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f6c101de0 msgr2=0x7f1f6c1984b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:52.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.511+0000 7f1f72d66640 1 --2- 172.21.15.16:0/393012148 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f6c101de0 0x7f1f6c1984b0 secure :-1 s=READY pgs=2025 cs=0 l=1 rev1=1 crypto rx=0x7f1f6800d770 tx=0x7f1f6800dc30 comp rx=0 tx=0).stop 2024-09-12T02:16:52.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.511+0000 7f1f72d66640 1 -- 172.21.15.16:0/393012148 shutdown_connections 2024-09-12T02:16:52.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.511+0000 7f1f72d66640 1 --2- 172.21.15.16:0/393012148 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f1f540a2c00 0x7f1f540a50c0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:52.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.511+0000 7f1f72d66640 1 --2- 172.21.15.16:0/393012148 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1f6c106f10 0x7f1f6c19cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:52.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.511+0000 7f1f72d66640 1 --2- 172.21.15.16:0/393012148 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f6c102790 0x7f1f6c1989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:52.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.511+0000 7f1f72d66640 1 --2- 172.21.15.16:0/393012148 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f6c101de0 0x7f1f6c1984b0 unknown :-1 s=CLOSED pgs=2025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:52.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.511+0000 7f1f72d66640 1 -- 172.21.15.16:0/393012148 >> 172.21.15.16:0/393012148 conn(0x7f1f6c0fd8c0 msgr2=0x7f1f6c0feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:52.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.511+0000 7f1f72d66640 1 -- 172.21.15.16:0/393012148 shutdown_connections 2024-09-12T02:16:52.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:52.511+0000 7f1f72d66640 1 -- 172.21.15.16:0/393012148 wait complete. 2024-09-12T02:16:52.685 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:16:52 smithi016 ceph-mon[30947]: pgmap v783: 97 pgs: 97 active+clean; 3.6 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-12T02:16:52.878 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:16:52 smithi184 ceph-mon[35593]: pgmap v783: 97 pgs: 97 active+clean; 3.6 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-12T02:16:52.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:16:52 smithi033 ceph-mon[35332]: pgmap v783: 97 pgs: 97 active+clean; 3.6 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-12T02:16:53.299 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-12T02:16:53.300 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:16:53.557 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:16:54.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:54.022+0000 7f313f01f640 1 -- 172.21.15.16:0/1982334702 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3138074620 msgr2=0x7f3138074a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:54.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:54.022+0000 7f313f01f640 1 --2- 172.21.15.16:0/1982334702 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3138074620 0x7f3138074a00 secure :-1 s=READY pgs=2030 cs=0 l=1 rev1=1 crypto rx=0x7f3128009900 tx=0x7f312802a480 comp rx=0 tx=0).stop 2024-09-12T02:16:54.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:54.022+0000 7f313f01f640 1 -- 172.21.15.16:0/1982334702 shutdown_connections 2024-09-12T02:16:54.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:54.022+0000 7f313f01f640 1 --2- 172.21.15.16:0/1982334702 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f313810e3c0 0x7f3138110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:54.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:54.022+0000 7f313f01f640 1 --2- 172.21.15.16:0/1982334702 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3138074f40 0x7f313810de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:54.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:54.022+0000 7f313f01f640 1 --2- 172.21.15.16:0/1982334702 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3138074620 0x7f3138074a00 unknown :-1 s=CLOSED pgs=2030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:54.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:54.022+0000 7f313f01f640 1 -- 172.21.15.16:0/1982334702 >> 172.21.15.16:0/1982334702 conn(0x7f31380fd8c0 msgr2=0x7f31380ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:54.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:54.022+0000 7f313f01f640 1 -- 172.21.15.16:0/1982334702 shutdown_connections 2024-09-12T02:16:54.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:54.022+0000 7f313f01f640 1 -- 172.21.15.16:0/1982334702 wait complete. 2024-09-12T02:16:54.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:54.022+0000 7f313f01f640 1 Processor -- start 2024-09-12T02:16:54.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:54.022+0000 7f313f01f640 1 -- start start 2024-09-12T02:16:54.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:54.023+0000 7f313f01f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3138074620 0x7f313819a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:54.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:54.023+0000 7f313f01f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3138074f40 0x7f313819abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:54.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:54.023+0000 7f313f01f640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f313810e3c0 0x7f313819ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:54.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:54.023+0000 7f313f01f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3138114150 con 0x7f3138074f40 2024-09-12T02:16:54.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:54.023+0000 7f313f01f640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3138113fd0 con 0x7f3138074620 2024-09-12T02:16:54.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:54.023+0000 7f313f01f640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f31381142d0 con 0x7f313810e3c0 2024-09-12T02:16:54.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:54.023+0000 7f3137fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3138074f40 0x7f313819abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:54.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:54.023+0000 7f3137fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3138074f40 0x7f313819abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58588/0 (socket says 172.21.15.16:58588) 2024-09-12T02:16:54.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:54.023+0000 7f3137fff640 1 -- 172.21.15.16:0/4158149425 learned_addr learned my addr 172.21.15.16:0/4158149425 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:16:54.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:54.023+0000 7f313d5bc640 1 --2- 172.21.15.16:0/4158149425 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f313810e3c0 0x7f313819ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:54.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:54.023+0000 7f313cdbb640 1 --2- 172.21.15.16:0/4158149425 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3138074620 0x7f313819a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:54.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:54.023+0000 7f3137fff640 1 -- 172.21.15.16:0/4158149425 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f313810e3c0 msgr2=0x7f313819ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:54.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:54.023+0000 7f3137fff640 1 --2- 172.21.15.16:0/4158149425 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f313810e3c0 0x7f313819ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:54.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:54.023+0000 7f3137fff640 1 -- 172.21.15.16:0/4158149425 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3138074620 msgr2=0x7f313819a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:54.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:54.023+0000 7f3137fff640 1 --2- 172.21.15.16:0/4158149425 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3138074620 0x7f313819a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:54.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:54.023+0000 7f3137fff640 1 -- 172.21.15.16:0/4158149425 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f313819f640 con 0x7f3138074f40 2024-09-12T02:16:54.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:54.023+0000 7f313cdbb640 1 --2- 172.21.15.16:0/4158149425 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3138074620 0x7f313819a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:16:54.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:54.023+0000 7f3137fff640 1 --2- 172.21.15.16:0/4158149425 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3138074f40 0x7f313819abd0 secure :-1 s=READY pgs=2031 cs=0 l=1 rev1=1 crypto rx=0x7f312000d950 tx=0x7f312000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:54.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:54.024+0000 7f3135ffb640 1 -- 172.21.15.16:0/4158149425 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3120017030 con 0x7f3138074f40 2024-09-12T02:16:54.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:54.024+0000 7f3135ffb640 1 -- 172.21.15.16:0/4158149425 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3120004020 con 0x7f3138074f40 2024-09-12T02:16:54.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:54.024+0000 7f313f01f640 1 -- 172.21.15.16:0/4158149425 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f313819f930 con 0x7f3138074f40 2024-09-12T02:16:54.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:54.024+0000 7f3135ffb640 1 -- 172.21.15.16:0/4158149425 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31200047a0 con 0x7f3138074f40 2024-09-12T02:16:54.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:54.024+0000 7f313f01f640 1 -- 172.21.15.16:0/4158149425 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31381a7210 con 0x7f3138074f40 2024-09-12T02:16:54.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:54.025+0000 7f313f01f640 1 -- 172.21.15.16:0/4158149425 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3138074a00 con 0x7f3138074f40 2024-09-12T02:16:54.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:54.026+0000 7f3135ffb640 1 -- 172.21.15.16:0/4158149425 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f31200041d0 con 0x7f3138074f40 2024-09-12T02:16:54.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:54.027+0000 7f3135ffb640 1 --2- 172.21.15.16:0/4158149425 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f31240a2b80 0x7f31240a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:54.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:54.027+0000 7f3135ffb640 1 -- 172.21.15.16:0/4158149425 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f31200cf0c0 con 0x7f3138074f40 2024-09-12T02:16:54.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:54.027+0000 7f313cdbb640 1 --2- 172.21.15.16:0/4158149425 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f31240a2b80 0x7f31240a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:54.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:54.027+0000 7f313cdbb640 1 --2- 172.21.15.16:0/4158149425 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f31240a2b80 0x7f31240a5040 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f3128005ab0 tx=0x7f3128005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:54.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:54.030+0000 7f3135ffb640 1 -- 172.21.15.16:0/4158149425 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f312009c020 con 0x7f3138074f40 2024-09-12T02:16:54.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:54.240+0000 7f313f01f640 1 -- 172.21.15.16:0/4158149425 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f313819b9f0 con 0x7f3138074f40 2024-09-12T02:16:54.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:54.240+0000 7f3135ffb640 1 -- 172.21.15.16:0/4158149425 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11911 (secure 0 0 0) 0x7f313819b9f0 con 0x7f3138074f40 2024-09-12T02:16:54.242 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T02:16:54.242 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":25,"btime":"2024-09-12T02:01:37:192021+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3279123279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3279123279},{"type":"v1","addr":"172.21.15.16:6835","nonce":3279123279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4119452382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4119452382},{"type":"v1","addr":"172.21.15.16:6837","nonce":4119452382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/3263823165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3263823165},{"type":"v1","addr":"172.21.15.16:6839","nonce":3263823165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/1510290732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":1510290732},{"type":"v1","addr":"172.21.15.16:6841","nonce":1510290732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/279674603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":279674603},{"type":"v1","addr":"172.21.15.33:6833","nonce":279674603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3555728028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3555728028},{"type":"v1","addr":"172.21.15.33:6835","nonce":3555728028}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T01:59:56.117608+0000","modified":"2024-09-12T02:01:36.294760+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24482,"mds_1":24481,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6833/1563374106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1563374106},{"type":"v1","addr":"172.21.15.184:6833","nonce":1563374106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6837/1892833868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1892833868},{"type":"v1","addr":"172.21.15.33:6837","nonce":1892833868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/1457679686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1457679686},{"type":"v1","addr":"172.21.15.184:6835","nonce":1457679686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.184:6837/1793393087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":1793393087},{"type":"v1","addr":"172.21.15.184:6837","nonce":1793393087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.184:6839/4203347941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":4203347941},{"type":"v1","addr":"172.21.15.184:6839","nonce":4203347941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.33:6839/4136670204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":4136670204},{"type":"v1","addr":"172.21.15.33:6839","nonce":4136670204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24481,24482,24469]},"id":1}]} 2024-09-12T02:16:54.242 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 25 2024-09-12T02:16:54.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:54.243+0000 7f313f01f640 1 -- 172.21.15.16:0/4158149425 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f31240a2b80 msgr2=0x7f31240a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:54.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:54.243+0000 7f313f01f640 1 --2- 172.21.15.16:0/4158149425 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f31240a2b80 0x7f31240a5040 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f3128005ab0 tx=0x7f3128005a20 comp rx=0 tx=0).stop 2024-09-12T02:16:54.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:54.243+0000 7f313f01f640 1 -- 172.21.15.16:0/4158149425 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3138074f40 msgr2=0x7f313819abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:54.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:54.243+0000 7f313f01f640 1 --2- 172.21.15.16:0/4158149425 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3138074f40 0x7f313819abd0 secure :-1 s=READY pgs=2031 cs=0 l=1 rev1=1 crypto rx=0x7f312000d950 tx=0x7f312000de10 comp rx=0 tx=0).stop 2024-09-12T02:16:54.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:54.243+0000 7f313f01f640 1 -- 172.21.15.16:0/4158149425 shutdown_connections 2024-09-12T02:16:54.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:54.243+0000 7f313f01f640 1 --2- 172.21.15.16:0/4158149425 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f31240a2b80 0x7f31240a5040 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:54.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:54.243+0000 7f313f01f640 1 --2- 172.21.15.16:0/4158149425 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f313810e3c0 0x7f313819ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:54.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:54.244+0000 7f313f01f640 1 --2- 172.21.15.16:0/4158149425 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3138074f40 0x7f313819abd0 unknown :-1 s=CLOSED pgs=2031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:54.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:54.244+0000 7f313f01f640 1 --2- 172.21.15.16:0/4158149425 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3138074620 0x7f313819a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:54.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:54.244+0000 7f313f01f640 1 -- 172.21.15.16:0/4158149425 >> 172.21.15.16:0/4158149425 conn(0x7f31380fd8c0 msgr2=0x7f31380fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:54.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:54.244+0000 7f313f01f640 1 -- 172.21.15.16:0/4158149425 shutdown_connections 2024-09-12T02:16:54.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:54.244+0000 7f313f01f640 1 -- 172.21.15.16:0/4158149425 wait complete. 2024-09-12T02:16:54.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:16:54 smithi016 ceph-mon[30947]: pgmap v784: 97 pgs: 97 active+clean; 3.6 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 14 op/s 2024-09-12T02:16:54.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:16:54 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/4158149425' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:16:54.721 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:16:54 smithi033 ceph-mon[35332]: pgmap v784: 97 pgs: 97 active+clean; 3.6 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 14 op/s 2024-09-12T02:16:54.721 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:16:54 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/4158149425' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:16:54.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:16:54 smithi184 ceph-mon[35593]: pgmap v784: 97 pgs: 97 active+clean; 3.6 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 14 op/s 2024-09-12T02:16:54.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:16:54 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/4158149425' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:16:55.134 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-12T02:16:55.400 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:16:55.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.850+0000 7f23758e4640 1 -- 172.21.15.16:0/425756410 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2370074620 msgr2=0x7f2370074a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:55.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.850+0000 7f23758e4640 1 --2- 172.21.15.16:0/425756410 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2370074620 0x7f2370074a00 secure :-1 s=READY pgs=2032 cs=0 l=1 rev1=1 crypto rx=0x7f2360009ca0 tx=0x7f236002a950 comp rx=0 tx=0).stop 2024-09-12T02:16:55.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.850+0000 7f23758e4640 1 -- 172.21.15.16:0/425756410 shutdown_connections 2024-09-12T02:16:55.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.850+0000 7f23758e4640 1 --2- 172.21.15.16:0/425756410 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f237010e3c0 0x7f2370110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:55.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.850+0000 7f23758e4640 1 --2- 172.21.15.16:0/425756410 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2370074f40 0x7f237010de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:55.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.850+0000 7f23758e4640 1 --2- 172.21.15.16:0/425756410 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2370074620 0x7f2370074a00 unknown :-1 s=CLOSED pgs=2032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:55.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.850+0000 7f23758e4640 1 -- 172.21.15.16:0/425756410 >> 172.21.15.16:0/425756410 conn(0x7f23700fd8c0 msgr2=0x7f23700ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:55.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.850+0000 7f23758e4640 1 -- 172.21.15.16:0/425756410 shutdown_connections 2024-09-12T02:16:55.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.850+0000 7f23758e4640 1 -- 172.21.15.16:0/425756410 wait complete. 2024-09-12T02:16:55.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.851+0000 7f23758e4640 1 Processor -- start 2024-09-12T02:16:55.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.851+0000 7f23758e4640 1 -- start start 2024-09-12T02:16:55.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.851+0000 7f23758e4640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2370074620 0x7f237019a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:55.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.851+0000 7f23758e4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2370074f40 0x7f237019ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:55.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.851+0000 7f23758e4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f237010e3c0 0x7f237019efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:55.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.851+0000 7f23758e4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2370114170 con 0x7f237010e3c0 2024-09-12T02:16:55.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.851+0000 7f23758e4640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2370113ff0 con 0x7f2370074f40 2024-09-12T02:16:55.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.851+0000 7f23758e4640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f23701142f0 con 0x7f2370074620 2024-09-12T02:16:55.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.851+0000 7f236f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f237010e3c0 0x7f237019efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:55.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.851+0000 7f236f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f237010e3c0 0x7f237019efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58594/0 (socket says 172.21.15.16:58594) 2024-09-12T02:16:55.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.851+0000 7f236f7fe640 1 -- 172.21.15.16:0/1190141388 learned_addr learned my addr 172.21.15.16:0/1190141388 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:16:55.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.851+0000 7f236effd640 1 --2- 172.21.15.16:0/1190141388 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2370074620 0x7f237019a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:55.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.851+0000 7f236e7fc640 1 --2- 172.21.15.16:0/1190141388 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2370074f40 0x7f237019ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:55.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.852+0000 7f236f7fe640 1 -- 172.21.15.16:0/1190141388 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2370074620 msgr2=0x7f237019a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:55.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.852+0000 7f236f7fe640 1 --2- 172.21.15.16:0/1190141388 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2370074620 0x7f237019a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:55.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.852+0000 7f236f7fe640 1 -- 172.21.15.16:0/1190141388 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2370074f40 msgr2=0x7f237019ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:55.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.852+0000 7f236f7fe640 1 --2- 172.21.15.16:0/1190141388 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2370074f40 0x7f237019ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:55.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.852+0000 7f236f7fe640 1 -- 172.21.15.16:0/1190141388 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f237019f680 con 0x7f237010e3c0 2024-09-12T02:16:55.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.852+0000 7f236f7fe640 1 --2- 172.21.15.16:0/1190141388 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f237010e3c0 0x7f237019efa0 secure :-1 s=READY pgs=2033 cs=0 l=1 rev1=1 crypto rx=0x7f236800d750 tx=0x7f236800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:55.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.852+0000 7f23748e2640 1 -- 172.21.15.16:0/1190141388 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2368017070 con 0x7f237010e3c0 2024-09-12T02:16:55.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.852+0000 7f23748e2640 1 -- 172.21.15.16:0/1190141388 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2368005500 con 0x7f237010e3c0 2024-09-12T02:16:55.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.852+0000 7f23758e4640 1 -- 172.21.15.16:0/1190141388 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f237019f970 con 0x7f237010e3c0 2024-09-12T02:16:55.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.852+0000 7f23748e2640 1 -- 172.21.15.16:0/1190141388 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2368004d90 con 0x7f237010e3c0 2024-09-12T02:16:55.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.852+0000 7f23758e4640 1 -- 172.21.15.16:0/1190141388 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23701a7250 con 0x7f237010e3c0 2024-09-12T02:16:55.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.856+0000 7f23748e2640 1 -- 172.21.15.16:0/1190141388 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f2368004f30 con 0x7f237010e3c0 2024-09-12T02:16:55.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.856+0000 7f2377cbd740 1 Processor -- start 2024-09-12T02:16:55.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.856+0000 7f2377cbd740 1 -- start start 2024-09-12T02:16:55.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.856+0000 7f2377cbd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56113fca4b40 0x56113fca3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:55.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.856+0000 7f2377cbd740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56113fca35d0 0x56113fca3a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:55.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.856+0000 7f2377cbd740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56113fcae370 0x56113fcb0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:55.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.856+0000 7f2377cbd740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56113fc092f0 con 0x56113fcae370 2024-09-12T02:16:55.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.856+0000 7f2377cbd740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56113fbb0cf0 con 0x56113fca4b40 2024-09-12T02:16:55.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.856+0000 7f2377cbd740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x56113fc052b0 con 0x56113fca35d0 2024-09-12T02:16:55.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.856+0000 7f236e7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56113fcae370 0x56113fcb0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:55.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.856+0000 7f236e7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56113fcae370 0x56113fcb0760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58602/0 (socket says 172.21.15.16:58602) 2024-09-12T02:16:55.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.856+0000 7f236e7fc640 1 -- 172.21.15.16:0/1274904924 learned_addr learned my addr 172.21.15.16:0/1274904924 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:16:55.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.856+0000 7f236effd640 1 --2- 172.21.15.16:0/1274904924 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56113fca35d0 0x56113fca3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:55.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.856+0000 7f236e7fc640 1 --2- 172.21.15.16:0/1274904924 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56113fca4b40 0x56113fca3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:55.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.856+0000 7f23748e2640 1 --2- 172.21.15.16:0/1190141388 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f23580a2c00 0x7f23580a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:55.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.856+0000 7f236e7fc640 1 -- 172.21.15.16:0/1274904924 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56113fca35d0 msgr2=0x56113fca3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:55.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.856+0000 7f236e7fc640 1 --2- 172.21.15.16:0/1274904924 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56113fca35d0 0x56113fca3a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:55.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.856+0000 7f236e7fc640 1 -- 172.21.15.16:0/1274904924 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56113fca4b40 msgr2=0x56113fca3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:55.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.856+0000 7f236e7fc640 1 --2- 172.21.15.16:0/1274904924 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56113fca4b40 0x56113fca3090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:55.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.856+0000 7f236f7fe640 1 --2- 172.21.15.16:0/1190141388 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f23580a2c00 0x7f23580a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:55.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.856+0000 7f236e7fc640 1 -- 172.21.15.16:0/1274904924 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56113fb953a0 con 0x56113fcae370 2024-09-12T02:16:55.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.856+0000 7f23748e2640 1 -- 172.21.15.16:0/1190141388 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f23680ce7a0 con 0x7f237010e3c0 2024-09-12T02:16:55.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.856+0000 7f236e7fc640 1 --2- 172.21.15.16:0/1274904924 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56113fcae370 0x56113fcb0760 secure :-1 s=READY pgs=2034 cs=0 l=1 rev1=1 crypto rx=0x7f2364011730 tx=0x7f236400a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26065412bada6f26 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:55.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.857+0000 7f2355ffb640 1 -- 172.21.15.16:0/1274904924 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2364004020 con 0x56113fcae370 2024-09-12T02:16:55.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.857+0000 7f236f7fe640 1 --2- 172.21.15.16:0/1190141388 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f23580a2c00 0x7f23580a50c0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f236806e1e0 tx=0x7f236806e210 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:55.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.857+0000 7f2355ffb640 1 -- 172.21.15.16:0/1274904924 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2364005500 con 0x56113fcae370 2024-09-12T02:16:55.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.857+0000 7f2355ffb640 1 -- 172.21.15.16:0/1274904924 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2364004c60 con 0x56113fcae370 2024-09-12T02:16:55.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.857+0000 7f2377cbd740 1 -- 172.21.15.16:0/1274904924 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56113fcae370 msgr2=0x56113fcb0760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:55.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.857+0000 7f2377cbd740 1 --2- 172.21.15.16:0/1274904924 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56113fcae370 0x56113fcb0760 secure :-1 s=READY pgs=2034 cs=0 l=1 rev1=1 crypto rx=0x7f2364011730 tx=0x7f236400a6a0 comp rx=0 tx=0).stop 2024-09-12T02:16:55.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.857+0000 7f2377cbd740 1 -- 172.21.15.16:0/1274904924 shutdown_connections 2024-09-12T02:16:55.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.857+0000 7f2377cbd740 1 --2- 172.21.15.16:0/1274904924 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56113fcae370 0x56113fcb0760 unknown :-1 s=CLOSED pgs=2034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:55.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.858+0000 7f2377cbd740 1 --2- 172.21.15.16:0/1274904924 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56113fca35d0 0x56113fca3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:55.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.858+0000 7f2377cbd740 1 --2- 172.21.15.16:0/1274904924 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56113fca4b40 0x56113fca3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:55.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.858+0000 7f2377cbd740 1 -- 172.21.15.16:0/1274904924 >> 172.21.15.16:0/1274904924 conn(0x56113fc6c310 msgr2=0x56113fc9fc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:55.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.858+0000 7f2377cbd740 1 -- 172.21.15.16:0/1274904924 shutdown_connections 2024-09-12T02:16:55.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.858+0000 7f2377cbd740 1 -- 172.21.15.16:0/1274904924 wait complete. 2024-09-12T02:16:55.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.858+0000 7f2377cbd740 1 -- start start 2024-09-12T02:16:55.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.858+0000 7f2377cbd740 1 Processor -- start 2024-09-12T02:16:55.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.858+0000 7f2377cbd740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56113fca35d0 0x56113fcaddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:55.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.858+0000 7f2377cbd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56113fca4b40 0x56113fcab060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:55.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.858+0000 7f2377cbd740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56113fcae370 0x56113fcab7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:55.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.858+0000 7f2377cbd740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56113fcb1070 con 0x56113fca35d0 2024-09-12T02:16:55.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.858+0000 7f2377cbd740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56113fbc4060 con 0x56113fca4b40 2024-09-12T02:16:55.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.858+0000 7f2377cbd740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x56113fcb11f0 con 0x56113fcae370 2024-09-12T02:16:55.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.858+0000 7f2377cbd740 10 client.0 _ll_register_callbacks cb 0x56113f9178c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:16:55.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.859+0000 7f236e7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56113fca35d0 0x56113fcaddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:55.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.859+0000 7f236e7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56113fca35d0 0x56113fcaddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58618/0 (socket says 172.21.15.16:58618) 2024-09-12T02:16:55.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.859+0000 7f236e7fc640 1 -- 172.21.15.16:0/667869851 learned_addr learned my addr 172.21.15.16:0/667869851 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:16:55.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.859+0000 7f236effd640 1 --2- 172.21.15.16:0/667869851 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56113fca4b40 0x56113fcab060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:55.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.859+0000 7f236e7fc640 1 --2- 172.21.15.16:0/667869851 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56113fcae370 0x56113fcab7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:55.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.859+0000 7f236e7fc640 1 -- 172.21.15.16:0/667869851 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56113fcae370 msgr2=0x56113fcab7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:55.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.859+0000 7f236e7fc640 1 --2- 172.21.15.16:0/667869851 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56113fcae370 0x56113fcab7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:55.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.859+0000 7f236e7fc640 1 -- 172.21.15.16:0/667869851 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56113fca4b40 msgr2=0x56113fcab060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:55.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.859+0000 7f236e7fc640 1 --2- 172.21.15.16:0/667869851 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56113fca4b40 0x56113fcab060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:55.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.859+0000 7f236e7fc640 1 -- 172.21.15.16:0/667869851 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56113f97a3c0 con 0x56113fca35d0 2024-09-12T02:16:55.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.859+0000 7f236effd640 1 --2- 172.21.15.16:0/667869851 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56113fca4b40 0x56113fcab060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:16:55.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.859+0000 7f236e7fc640 1 --2- 172.21.15.16:0/667869851 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56113fca35d0 0x56113fcaddb0 secure :-1 s=READY pgs=2035 cs=0 l=1 rev1=1 crypto rx=0x7f23640058c0 tx=0x7f2364004840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb52b5e2f8ff908c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:55.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.859+0000 7f234effd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:16:55.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.859+0000 7f234effd640 1 -- 172.21.15.16:0/667869851 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f236402b060 con 0x56113fca35d0 2024-09-12T02:16:55.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.859+0000 7f234effd640 1 -- 172.21.15.16:0/667869851 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f236400f7c0 con 0x56113fca35d0 2024-09-12T02:16:55.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.859+0000 7f2377cbd740 1 -- 172.21.15.16:0/667869851 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2364005500 con 0x56113fca35d0 2024-09-12T02:16:55.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.860+0000 7f234effd640 1 -- 172.21.15.16:0/667869851 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2364004660 con 0x56113fca35d0 2024-09-12T02:16:55.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.860+0000 7f2377cbd740 10 client.20544 fetch_fsmap learned FSMap version 25 2024-09-12T02:16:55.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.860+0000 7f2377cbd740 1 -- 172.21.15.16:0/667869851 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56113fcbcb60 con 0x56113fca35d0 2024-09-12T02:16:55.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.860+0000 7f234effd640 1 -- 172.21.15.16:0/667869851 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2364005500 con 0x56113fca35d0 2024-09-12T02:16:55.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.860+0000 7f234effd640 1 -- 172.21.15.16:0/667869851 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f2364030070 con 0x56113fca35d0 2024-09-12T02:16:55.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.860+0000 7f2377cbd740 10 client.20544 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:16:55.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.860+0000 7f2377cbd740 10 client.20544 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-12T02:16:55.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.860+0000 7f2377cbd740 20 client.20544 populate_metadata read hostname 'smithi016' 2024-09-12T02:16:55.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.860+0000 7f2377cbd740 1 --2- 172.21.15.16:0/667869851 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x56113fcbdf00 0x56113fcc02f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:55.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.860+0000 7f2377cbd740 4 client.20544 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:16:55.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.860+0000 7f2377cbd740 1 -- 172.21.15.16:0/667869851 --> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56113fc16860 con 0x56113fcbdf00 2024-09-12T02:16:55.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.861+0000 7f236effd640 1 --2- 172.21.15.16:0/667869851 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x56113fcbdf00 0x56113fcc02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:55.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.861+0000 7f234effd640 10 client.20544 ms_handle_connect on v2:172.21.15.184:6836/1793393087 2024-09-12T02:16:55.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.861+0000 7f236effd640 1 --2- 172.21.15.16:0/667869851 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x56113fcbdf00 0x56113fcc02f0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c4417742baf6da85 server_cookie=85b58c938d6c71d5 in_seq=0 out_seq=0 2024-09-12T02:16:55.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.862+0000 7f234effd640 1 -- 172.21.15.16:0/667869851 <== mds.2 v2:172.21.15.184:6836/1793393087 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56113fc16860 con 0x56113fcbdf00 2024-09-12T02:16:55.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.862+0000 7f234effd640 10 client.20544 handle_command_reply: tid=0 2024-09-12T02:16:55.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.862+0000 7f2377cbd740 1 client.20544 shutdown 2024-09-12T02:16:55.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.862+0000 7f2377cbd740 1 -- 172.21.15.16:0/667869851 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56113fca35d0 msgr2=0x56113fcaddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:55.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.862+0000 7f2377cbd740 1 --2- 172.21.15.16:0/667869851 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56113fca35d0 0x56113fcaddb0 secure :-1 s=READY pgs=2035 cs=0 l=1 rev1=1 crypto rx=0x7f23640058c0 tx=0x7f2364004840 comp rx=0 tx=0).stop 2024-09-12T02:16:55.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.862+0000 7f2377cbd740 1 -- 172.21.15.16:0/667869851 shutdown_connections 2024-09-12T02:16:55.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.862+0000 7f2377cbd740 1 --2- 172.21.15.16:0/667869851 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56113fcae370 0x56113fcab7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:55.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.862+0000 7f2377cbd740 1 --2- 172.21.15.16:0/667869851 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56113fca4b40 0x56113fcab060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:55.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.862+0000 7f2377cbd740 1 --2- 172.21.15.16:0/667869851 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x56113fcbdf00 0x56113fcc02f0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:55.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.862+0000 7f2377cbd740 1 --2- 172.21.15.16:0/667869851 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56113fca35d0 0x56113fcaddb0 unknown :-1 s=CLOSED pgs=2035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:55.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.862+0000 7f2377cbd740 1 -- 172.21.15.16:0/667869851 >> 172.21.15.16:0/667869851 conn(0x56113fc6c310 msgr2=0x56113fca5960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:55.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.862+0000 7f234effd640 0 client.20544 ms_handle_reset on v2:172.21.15.184:6836/1793393087 2024-09-12T02:16:55.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.862+0000 7f2377cbd740 1 -- 172.21.15.16:0/667869851 shutdown_connections 2024-09-12T02:16:55.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.862+0000 7f2377cbd740 1 -- 172.21.15.16:0/667869851 wait complete. 2024-09-12T02:16:55.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.862+0000 7f2377cbd740 20 client.20544 trim_cache size 0 max 16384 2024-09-12T02:16:55.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.875+0000 7f2377cbd740 1 Processor -- start 2024-09-12T02:16:55.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.875+0000 7f2377cbd740 1 -- start start 2024-09-12T02:16:55.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.875+0000 7f2377cbd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56113fca35d0 0x56113fc9fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:55.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.875+0000 7f2377cbd740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56113fca4b40 0x56113fcc5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:55.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.875+0000 7f2377cbd740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56113fcae370 0x56113fcc5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:55.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.875+0000 7f2377cbd740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f23701b1f60 con 0x56113fcae370 2024-09-12T02:16:55.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.875+0000 7f2377cbd740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f23701af830 con 0x56113fca35d0 2024-09-12T02:16:55.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.875+0000 7f2377cbd740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f23701ad680 con 0x56113fca4b40 2024-09-12T02:16:55.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.875+0000 7f236e7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56113fcae370 0x56113fcc5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:55.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.875+0000 7f236e7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56113fcae370 0x56113fcc5ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58632/0 (socket says 172.21.15.16:58632) 2024-09-12T02:16:55.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.875+0000 7f236e7fc640 1 -- 172.21.15.16:0/3468452384 learned_addr learned my addr 172.21.15.16:0/3468452384 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:16:55.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.876+0000 7f236e7fc640 1 -- 172.21.15.16:0/3468452384 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56113fca4b40 msgr2=0x56113fcc5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:55.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.876+0000 7f236e7fc640 1 --2- 172.21.15.16:0/3468452384 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56113fca4b40 0x56113fcc5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:55.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.876+0000 7f236e7fc640 1 -- 172.21.15.16:0/3468452384 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56113fca35d0 msgr2=0x56113fc9fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:55.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.876+0000 7f236e7fc640 1 --2- 172.21.15.16:0/3468452384 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56113fca35d0 0x56113fc9fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:55.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.876+0000 7f236e7fc640 1 -- 172.21.15.16:0/3468452384 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56113fca0960 con 0x56113fcae370 2024-09-12T02:16:55.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.876+0000 7f236e7fc640 1 --2- 172.21.15.16:0/3468452384 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56113fcae370 0x56113fcc5ec0 secure :-1 s=READY pgs=2036 cs=0 l=1 rev1=1 crypto rx=0x7f2364005e60 tx=0x7f2364004930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7000a7de3618db4d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:55.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.876+0000 7f23557fa640 1 -- 172.21.15.16:0/3468452384 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2364004660 con 0x56113fcae370 2024-09-12T02:16:55.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.876+0000 7f23557fa640 1 -- 172.21.15.16:0/3468452384 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2364005470 con 0x56113fcae370 2024-09-12T02:16:55.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.876+0000 7f23557fa640 1 -- 172.21.15.16:0/3468452384 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2364004020 con 0x56113fcae370 2024-09-12T02:16:55.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.877+0000 7f2377cbd740 1 -- 172.21.15.16:0/3468452384 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56113fcae370 msgr2=0x56113fcc5ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:55.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.877+0000 7f2377cbd740 1 --2- 172.21.15.16:0/3468452384 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56113fcae370 0x56113fcc5ec0 secure :-1 s=READY pgs=2036 cs=0 l=1 rev1=1 crypto rx=0x7f2364005e60 tx=0x7f2364004930 comp rx=0 tx=0).stop 2024-09-12T02:16:55.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.877+0000 7f2377cbd740 1 -- 172.21.15.16:0/3468452384 shutdown_connections 2024-09-12T02:16:55.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.877+0000 7f2377cbd740 1 --2- 172.21.15.16:0/3468452384 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56113fcae370 0x56113fcc5ec0 unknown :-1 s=CLOSED pgs=2036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:55.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.877+0000 7f2377cbd740 1 --2- 172.21.15.16:0/3468452384 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56113fca4b40 0x56113fcc5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:55.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.877+0000 7f2377cbd740 1 --2- 172.21.15.16:0/3468452384 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56113fca35d0 0x56113fc9fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:55.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.877+0000 7f2377cbd740 1 -- 172.21.15.16:0/3468452384 >> 172.21.15.16:0/3468452384 conn(0x56113fc6c310 msgr2=0x56113fc9f6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:55.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.877+0000 7f2377cbd740 1 -- 172.21.15.16:0/3468452384 shutdown_connections 2024-09-12T02:16:55.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.877+0000 7f2377cbd740 1 -- 172.21.15.16:0/3468452384 wait complete. 2024-09-12T02:16:55.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.877+0000 7f2377cbd740 1 -- start start 2024-09-12T02:16:55.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.877+0000 7f2377cbd740 1 Processor -- start 2024-09-12T02:16:55.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.877+0000 7f2377cbd740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56113fca35d0 0x56113fcb1440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:55.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.878+0000 7f2377cbd740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56113fca4b40 0x56113fcb1980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:55.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.878+0000 7f2377cbd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56113fcae370 0x56113fcb1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:55.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.878+0000 7f2377cbd740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f23701adcf0 con 0x56113fca35d0 2024-09-12T02:16:55.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.878+0000 7f2377cbd740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f23701af160 con 0x56113fcae370 2024-09-12T02:16:55.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.878+0000 7f2377cbd740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f23701ae3c0 con 0x56113fca4b40 2024-09-12T02:16:55.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.878+0000 7f2377cbd740 10 client.0 _ll_register_callbacks cb 0x7f236400adb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:16:55.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.878+0000 7f236e7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56113fca35d0 0x56113fcb1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:55.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.878+0000 7f236e7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56113fca35d0 0x56113fcb1440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58636/0 (socket says 172.21.15.16:58636) 2024-09-12T02:16:55.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.878+0000 7f236e7fc640 1 -- 172.21.15.16:0/259911268 learned_addr learned my addr 172.21.15.16:0/259911268 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:16:55.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.878+0000 7f236effd640 1 --2- 172.21.15.16:0/259911268 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56113fca4b40 0x56113fcb1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:55.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.878+0000 7f236e7fc640 1 --2- 172.21.15.16:0/259911268 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56113fcae370 0x56113fcb1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:55.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.878+0000 7f236e7fc640 1 -- 172.21.15.16:0/259911268 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56113fca4b40 msgr2=0x56113fcb1980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:55.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.878+0000 7f236e7fc640 1 --2- 172.21.15.16:0/259911268 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56113fca4b40 0x56113fcb1980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:55.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.878+0000 7f236e7fc640 1 -- 172.21.15.16:0/259911268 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56113fcae370 msgr2=0x56113fcb1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:55.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.878+0000 7f236e7fc640 1 --2- 172.21.15.16:0/259911268 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56113fcae370 0x56113fcb1ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:55.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.878+0000 7f236e7fc640 1 -- 172.21.15.16:0/259911268 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56113fca6120 con 0x56113fca35d0 2024-09-12T02:16:55.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.878+0000 7f236e7fc640 1 --2- 172.21.15.16:0/259911268 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56113fca35d0 0x56113fcb1440 secure :-1 s=READY pgs=2037 cs=0 l=1 rev1=1 crypto rx=0x7f2364005680 tx=0x7f23640116c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ce2a604f4b7d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:55.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.878+0000 7f234effd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:16:55.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.878+0000 7f234effd640 1 -- 172.21.15.16:0/259911268 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2364036070 con 0x56113fca35d0 2024-09-12T02:16:55.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.878+0000 7f234effd640 1 -- 172.21.15.16:0/259911268 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f236401e6c0 con 0x56113fca35d0 2024-09-12T02:16:55.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.878+0000 7f2377cbd740 1 -- 172.21.15.16:0/259911268 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2364005470 con 0x56113fca35d0 2024-09-12T02:16:55.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.879+0000 7f234effd640 1 -- 172.21.15.16:0/259911268 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2364011020 con 0x56113fca35d0 2024-09-12T02:16:55.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.879+0000 7f2377cbd740 10 client.20550 fetch_fsmap learned FSMap version 25 2024-09-12T02:16:55.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.879+0000 7f2377cbd740 1 -- 172.21.15.16:0/259911268 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56113fd27b20 con 0x56113fca35d0 2024-09-12T02:16:55.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.879+0000 7f234effd640 1 -- 172.21.15.16:0/259911268 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2364005470 con 0x56113fca35d0 2024-09-12T02:16:55.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.879+0000 7f234effd640 1 -- 172.21.15.16:0/259911268 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f2364030070 con 0x56113fca35d0 2024-09-12T02:16:55.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.879+0000 7f2377cbd740 10 client.20550 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:16:55.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.879+0000 7f2377cbd740 10 client.20550 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-12T02:16:55.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.879+0000 7f2377cbd740 20 client.20550 populate_metadata read hostname 'smithi016' 2024-09-12T02:16:55.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.879+0000 7f2377cbd740 1 --2- 172.21.15.16:0/259911268 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x56113fd28060 0x56113fd2a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:55.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.879+0000 7f2377cbd740 4 client.20550 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T02:16:55.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.879+0000 7f2377cbd740 1 -- 172.21.15.16:0/259911268 --> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56113fc10d70 con 0x56113fd28060 2024-09-12T02:16:55.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.879+0000 7f236effd640 1 --2- 172.21.15.16:0/259911268 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x56113fd28060 0x56113fd2a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:55.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.880+0000 7f234effd640 10 client.20550 ms_handle_connect on v2:172.21.15.184:6836/1793393087 2024-09-12T02:16:55.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.880+0000 7f236effd640 1 --2- 172.21.15.16:0/259911268 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x56113fd28060 0x56113fd2a480 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=db5de770f6972b2b server_cookie=1652a78dbfa27199 in_seq=0 out_seq=0 2024-09-12T02:16:55.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.880+0000 7f234effd640 1 -- 172.21.15.16:0/259911268 <== mds.2 v2:172.21.15.184:6836/1793393087 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56113fc10d70 con 0x56113fd28060 2024-09-12T02:16:55.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.880+0000 7f234effd640 10 client.20550 handle_command_reply: tid=0 2024-09-12T02:16:55.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.880+0000 7f2377cbd740 1 client.20550 shutdown 2024-09-12T02:16:55.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.880+0000 7f2377cbd740 1 -- 172.21.15.16:0/259911268 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56113fca35d0 msgr2=0x56113fcb1440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:55.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.880+0000 7f2377cbd740 1 --2- 172.21.15.16:0/259911268 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56113fca35d0 0x56113fcb1440 secure :-1 s=READY pgs=2037 cs=0 l=1 rev1=1 crypto rx=0x7f2364005680 tx=0x7f23640116c0 comp rx=0 tx=0).stop 2024-09-12T02:16:55.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.880+0000 7f2377cbd740 1 -- 172.21.15.16:0/259911268 shutdown_connections 2024-09-12T02:16:55.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.880+0000 7f2377cbd740 1 --2- 172.21.15.16:0/259911268 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56113fcae370 0x56113fcb1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:55.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.880+0000 7f2377cbd740 1 --2- 172.21.15.16:0/259911268 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56113fca4b40 0x56113fcb1980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:55.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.880+0000 7f2377cbd740 1 --2- 172.21.15.16:0/259911268 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x56113fd28060 0x56113fd2a480 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:55.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.880+0000 7f2377cbd740 1 --2- 172.21.15.16:0/259911268 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56113fca35d0 0x56113fcb1440 unknown :-1 s=CLOSED pgs=2037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:55.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.880+0000 7f234effd640 0 client.20550 ms_handle_reset on v2:172.21.15.184:6836/1793393087 2024-09-12T02:16:55.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.880+0000 7f2377cbd740 1 -- 172.21.15.16:0/259911268 >> 172.21.15.16:0/259911268 conn(0x56113fc6c310 msgr2=0x56113fc9e4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:55.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.880+0000 7f2377cbd740 1 -- 172.21.15.16:0/259911268 shutdown_connections 2024-09-12T02:16:55.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.880+0000 7f2377cbd740 1 -- 172.21.15.16:0/259911268 wait complete. 2024-09-12T02:16:55.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.881+0000 7f2377cbd740 20 client.20550 trim_cache size 0 max 16384 2024-09-12T02:16:55.889 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T02:16:55.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.881+0000 7f23758e4640 1 -- 172.21.15.16:0/1190141388 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f23580a2c00 msgr2=0x7f23580a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:55.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.881+0000 7f23758e4640 1 --2- 172.21.15.16:0/1190141388 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f23580a2c00 0x7f23580a50c0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f236806e1e0 tx=0x7f236806e210 comp rx=0 tx=0).stop 2024-09-12T02:16:55.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.881+0000 7f23758e4640 1 -- 172.21.15.16:0/1190141388 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f237010e3c0 msgr2=0x7f237019efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:55.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.881+0000 7f23758e4640 1 --2- 172.21.15.16:0/1190141388 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f237010e3c0 0x7f237019efa0 secure :-1 s=READY pgs=2033 cs=0 l=1 rev1=1 crypto rx=0x7f236800d750 tx=0x7f236800dc10 comp rx=0 tx=0).stop 2024-09-12T02:16:55.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.881+0000 7f23758e4640 1 -- 172.21.15.16:0/1190141388 shutdown_connections 2024-09-12T02:16:55.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.881+0000 7f23758e4640 1 --2- 172.21.15.16:0/1190141388 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f23580a2c00 0x7f23580a50c0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:55.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.881+0000 7f23758e4640 1 --2- 172.21.15.16:0/1190141388 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f237010e3c0 0x7f237019efa0 unknown :-1 s=CLOSED pgs=2033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:55.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.881+0000 7f23758e4640 1 --2- 172.21.15.16:0/1190141388 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2370074f40 0x7f237019ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:55.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.881+0000 7f23758e4640 1 --2- 172.21.15.16:0/1190141388 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2370074620 0x7f237019a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:55.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.881+0000 7f23758e4640 1 -- 172.21.15.16:0/1190141388 >> 172.21.15.16:0/1190141388 conn(0x7f23700fd8c0 msgr2=0x7f23700fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:55.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.881+0000 7f23758e4640 1 -- 172.21.15.16:0/1190141388 shutdown_connections 2024-09-12T02:16:55.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:55.881+0000 7f23758e4640 1 -- 172.21.15.16:0/1190141388 wait complete. 2024-09-12T02:16:56.689 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:16:56 smithi016 ceph-mon[30947]: pgmap v785: 97 pgs: 97 active+clean; 3.6 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 12 op/s 2024-09-12T02:16:56.723 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-12T02:16:56.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:16:56 smithi184 ceph-mon[35593]: pgmap v785: 97 pgs: 97 active+clean; 3.6 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 12 op/s 2024-09-12T02:16:56.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:16:56 smithi033 ceph-mon[35332]: pgmap v785: 97 pgs: 97 active+clean; 3.6 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 12 op/s 2024-09-12T02:16:56.985 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:16:57.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.384+0000 7fdb1b6af640 1 -- 172.21.15.16:0/2834962224 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb141011e0 msgr2=0x7fdb14105980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:57.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.384+0000 7fdb1b6af640 1 --2- 172.21.15.16:0/2834962224 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb141011e0 0x7fdb14105980 secure :-1 s=READY pgs=2038 cs=0 l=1 rev1=1 crypto rx=0x7fdb04009900 tx=0x7fdb0402a3f0 comp rx=0 tx=0).stop 2024-09-12T02:16:57.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.384+0000 7fdb1b6af640 1 -- 172.21.15.16:0/2834962224 shutdown_connections 2024-09-12T02:16:57.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.384+0000 7fdb1b6af640 1 --2- 172.21.15.16:0/2834962224 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb14105ec0 0x7fdb14106340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:57.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.384+0000 7fdb1b6af640 1 --2- 172.21.15.16:0/2834962224 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb141011e0 0x7fdb14105980 unknown :-1 s=CLOSED pgs=2038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:57.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.384+0000 7fdb1b6af640 1 --2- 172.21.15.16:0/2834962224 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdb141008c0 0x7fdb14100ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:57.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.384+0000 7fdb1b6af640 1 -- 172.21.15.16:0/2834962224 >> 172.21.15.16:0/2834962224 conn(0x7fdb140fc7d0 msgr2=0x7fdb140febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:57.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.384+0000 7fdb1b6af640 1 -- 172.21.15.16:0/2834962224 shutdown_connections 2024-09-12T02:16:57.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.384+0000 7fdb1b6af640 1 -- 172.21.15.16:0/2834962224 wait complete. 2024-09-12T02:16:57.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.385+0000 7fdb1b6af640 1 Processor -- start 2024-09-12T02:16:57.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.385+0000 7fdb1b6af640 1 -- start start 2024-09-12T02:16:57.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.385+0000 7fdb1b6af640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb141008c0 0x7fdb141a4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:57.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.385+0000 7fdb1b6af640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdb141011e0 0x7fdb141a4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:57.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.385+0000 7fdb1b6af640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb14105ec0 0x7fdb1419e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:57.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.385+0000 7fdb1b6af640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fdb14118660 con 0x7fdb141008c0 2024-09-12T02:16:57.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.385+0000 7fdb1b6af640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fdb141184e0 con 0x7fdb14105ec0 2024-09-12T02:16:57.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.385+0000 7fdb1b6af640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fdb141187e0 con 0x7fdb141011e0 2024-09-12T02:16:57.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.385+0000 7fdb1944b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb141008c0 0x7fdb141a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:57.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.385+0000 7fdb1944b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb141008c0 0x7fdb141a4770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58652/0 (socket says 172.21.15.16:58652) 2024-09-12T02:16:57.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.385+0000 7fdb1944b640 1 -- 172.21.15.16:0/2405642837 learned_addr learned my addr 172.21.15.16:0/2405642837 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:16:57.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.386+0000 7fdb19c4c640 1 --2- 172.21.15.16:0/2405642837 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb14105ec0 0x7fdb1419e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:57.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.386+0000 7fdb18c4a640 1 --2- 172.21.15.16:0/2405642837 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdb141011e0 0x7fdb141a4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:57.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.386+0000 7fdb1944b640 1 -- 172.21.15.16:0/2405642837 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdb141011e0 msgr2=0x7fdb141a4cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:57.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.386+0000 7fdb1944b640 1 --2- 172.21.15.16:0/2405642837 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdb141011e0 0x7fdb141a4cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:57.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.386+0000 7fdb1944b640 1 -- 172.21.15.16:0/2405642837 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb14105ec0 msgr2=0x7fdb1419e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:57.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.386+0000 7fdb1944b640 1 --2- 172.21.15.16:0/2405642837 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb14105ec0 0x7fdb1419e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:57.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.386+0000 7fdb1944b640 1 -- 172.21.15.16:0/2405642837 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb1419f060 con 0x7fdb141008c0 2024-09-12T02:16:57.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.386+0000 7fdb19c4c640 1 --2- 172.21.15.16:0/2405642837 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb14105ec0 0x7fdb1419e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:16:57.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.386+0000 7fdb1944b640 1 --2- 172.21.15.16:0/2405642837 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb141008c0 0x7fdb141a4770 secure :-1 s=READY pgs=2039 cs=0 l=1 rev1=1 crypto rx=0x7fdb1000c920 tx=0x7fdb1000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:57.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.386+0000 7fdb0a7fc640 1 -- 172.21.15.16:0/2405642837 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb10007c70 con 0x7fdb141008c0 2024-09-12T02:16:57.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.386+0000 7fdb0a7fc640 1 -- 172.21.15.16:0/2405642837 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb10007e10 con 0x7fdb141008c0 2024-09-12T02:16:57.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.386+0000 7fdb1b6af640 1 -- 172.21.15.16:0/2405642837 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb1419f350 con 0x7fdb141008c0 2024-09-12T02:16:57.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.386+0000 7fdb0a7fc640 1 -- 172.21.15.16:0/2405642837 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb10018890 con 0x7fdb141008c0 2024-09-12T02:16:57.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.386+0000 7fdb1b6af640 1 -- 172.21.15.16:0/2405642837 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb141ab640 con 0x7fdb141008c0 2024-09-12T02:16:57.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.388+0000 7fdb0a7fc640 1 -- 172.21.15.16:0/2405642837 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fdb10018a50 con 0x7fdb141008c0 2024-09-12T02:16:57.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.389+0000 7fdb1da88740 1 Processor -- start 2024-09-12T02:16:57.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.389+0000 7fdb1da88740 1 -- start start 2024-09-12T02:16:57.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.389+0000 7fdb0a7fc640 1 --2- 172.21.15.16:0/2405642837 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fdb000a2c00 0x7fdb000a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:57.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.389+0000 7fdb1da88740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ce840d1b40 0x55ce840d0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:57.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.389+0000 7fdb1da88740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce840d05d0 0x55ce840d0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:57.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.389+0000 7fdb1da88740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ce840db370 0x55ce840dd760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:57.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.389+0000 7fdb1da88740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55ce840362f0 con 0x55ce840d05d0 2024-09-12T02:16:57.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.389+0000 7fdb1da88740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55ce83fddcf0 con 0x55ce840d1b40 2024-09-12T02:16:57.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.389+0000 7fdb1da88740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55ce840322b0 con 0x55ce840db370 2024-09-12T02:16:57.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.390+0000 7fdb19c4c640 1 --2- 172.21.15.16:0/2405642837 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fdb000a2c00 0x7fdb000a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:57.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.390+0000 7fdb0a7fc640 1 -- 172.21.15.16:0/2405642837 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fdb100cf120 con 0x7fdb141008c0 2024-09-12T02:16:57.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.390+0000 7fdb18c4a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce840d05d0 0x55ce840d0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:57.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.390+0000 7fdb18c4a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce840d05d0 0x55ce840d0a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58658/0 (socket says 172.21.15.16:58658) 2024-09-12T02:16:57.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.390+0000 7fdb18c4a640 1 -- 172.21.15.16:0/303991365 learned_addr learned my addr 172.21.15.16:0/303991365 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:16:57.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.390+0000 7fdb19c4c640 1 --2- 172.21.15.16:0/303991365 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ce840db370 0x55ce840dd760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:57.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.390+0000 7fdb1944b640 1 --2- 172.21.15.16:0/303991365 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ce840d1b40 0x55ce840d0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:57.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.390+0000 7fdb18c4a640 1 -- 172.21.15.16:0/303991365 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ce840db370 msgr2=0x55ce840dd760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:57.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.390+0000 7fdb18c4a640 1 --2- 172.21.15.16:0/303991365 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ce840db370 0x55ce840dd760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:57.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.390+0000 7fdb18c4a640 1 -- 172.21.15.16:0/303991365 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ce840d1b40 msgr2=0x55ce840d0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:57.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.390+0000 7fdb18c4a640 1 --2- 172.21.15.16:0/303991365 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ce840d1b40 0x55ce840d0090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:57.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.390+0000 7fdb18c4a640 1 -- 172.21.15.16:0/303991365 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce83fc23a0 con 0x55ce840d05d0 2024-09-12T02:16:57.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.390+0000 7fdb19c4c640 1 --2- 172.21.15.16:0/2405642837 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fdb000a2c00 0x7fdb000a50c0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fdb0c009370 tx=0x7fdb0c009e20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:57.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.390+0000 7fdb18c4a640 1 --2- 172.21.15.16:0/303991365 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce840d05d0 0x55ce840d0a10 secure :-1 s=READY pgs=2040 cs=0 l=1 rev1=1 crypto rx=0x7fdb04009230 tx=0x7fdb04004200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a46fb50e31f335e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:57.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.390+0000 7fdaff7fe640 1 -- 172.21.15.16:0/303991365 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb04035030 con 0x55ce840d05d0 2024-09-12T02:16:57.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.390+0000 7fdaff7fe640 1 -- 172.21.15.16:0/303991365 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb04005520 con 0x55ce840d05d0 2024-09-12T02:16:57.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.390+0000 7fdaff7fe640 1 -- 172.21.15.16:0/303991365 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb040035f0 con 0x55ce840d05d0 2024-09-12T02:16:57.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.391+0000 7fdb1da88740 1 -- 172.21.15.16:0/303991365 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce840d05d0 msgr2=0x55ce840d0a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:57.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.391+0000 7fdb1da88740 1 --2- 172.21.15.16:0/303991365 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce840d05d0 0x55ce840d0a10 secure :-1 s=READY pgs=2040 cs=0 l=1 rev1=1 crypto rx=0x7fdb04009230 tx=0x7fdb04004200 comp rx=0 tx=0).stop 2024-09-12T02:16:57.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.391+0000 7fdb1da88740 1 -- 172.21.15.16:0/303991365 shutdown_connections 2024-09-12T02:16:57.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.391+0000 7fdb1da88740 1 --2- 172.21.15.16:0/303991365 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ce840db370 0x55ce840dd760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:57.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.391+0000 7fdb1da88740 1 --2- 172.21.15.16:0/303991365 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce840d05d0 0x55ce840d0a10 unknown :-1 s=CLOSED pgs=2040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:57.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.391+0000 7fdb1da88740 1 --2- 172.21.15.16:0/303991365 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ce840d1b40 0x55ce840d0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:57.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.391+0000 7fdb1da88740 1 -- 172.21.15.16:0/303991365 >> 172.21.15.16:0/303991365 conn(0x55ce84099310 msgr2=0x55ce840ccc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:57.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.391+0000 7fdb1da88740 1 -- 172.21.15.16:0/303991365 shutdown_connections 2024-09-12T02:16:57.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.391+0000 7fdb1da88740 1 -- 172.21.15.16:0/303991365 wait complete. 2024-09-12T02:16:57.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.392+0000 7fdb1da88740 1 -- start start 2024-09-12T02:16:57.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.392+0000 7fdb1da88740 1 Processor -- start 2024-09-12T02:16:57.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.392+0000 7fdb1da88740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce840d05d0 0x55ce840dad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:57.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.392+0000 7fdb1da88740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ce840d1b40 0x55ce840d8020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:57.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.392+0000 7fdb1da88740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ce840db370 0x55ce840d8610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:57.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.392+0000 7fdb1da88740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55ce840de010 con 0x55ce840d05d0 2024-09-12T02:16:57.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.392+0000 7fdb1da88740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55ce83ff1060 con 0x55ce840d1b40 2024-09-12T02:16:57.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.392+0000 7fdb1da88740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55ce840de190 con 0x55ce840db370 2024-09-12T02:16:57.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.392+0000 7fdb1da88740 10 client.0 _ll_register_callbacks cb 0x55ce83d448c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:16:57.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.392+0000 7fdb1944b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce840d05d0 0x55ce840dad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:57.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.392+0000 7fdb1944b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce840d05d0 0x55ce840dad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58668/0 (socket says 172.21.15.16:58668) 2024-09-12T02:16:57.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.392+0000 7fdb1944b640 1 -- 172.21.15.16:0/2157906543 learned_addr learned my addr 172.21.15.16:0/2157906543 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:16:57.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.392+0000 7fdb19c4c640 1 --2- 172.21.15.16:0/2157906543 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ce840db370 0x55ce840d8610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:57.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.392+0000 7fdb18c4a640 1 --2- 172.21.15.16:0/2157906543 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ce840d1b40 0x55ce840d8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:57.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.393+0000 7fdb1944b640 1 -- 172.21.15.16:0/2157906543 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ce840db370 msgr2=0x55ce840d8610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:57.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.393+0000 7fdb1944b640 1 --2- 172.21.15.16:0/2157906543 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ce840db370 0x55ce840d8610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:57.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.393+0000 7fdb1944b640 1 -- 172.21.15.16:0/2157906543 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ce840d1b40 msgr2=0x55ce840d8020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:57.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.393+0000 7fdb1944b640 1 --2- 172.21.15.16:0/2157906543 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ce840d1b40 0x55ce840d8020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:57.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.393+0000 7fdb1944b640 1 -- 172.21.15.16:0/2157906543 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ce83da73c0 con 0x55ce840d05d0 2024-09-12T02:16:57.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.393+0000 7fdb1944b640 1 --2- 172.21.15.16:0/2157906543 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce840d05d0 0x55ce840dad70 secure :-1 s=READY pgs=2041 cs=0 l=1 rev1=1 crypto rx=0x7fdb10000c00 tx=0x7fdb1001f300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b7ac23ee2e3131c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:57.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.393+0000 7fdafcff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:16:57.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.393+0000 7fdafcff9640 1 -- 172.21.15.16:0/2157906543 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb1006fea0 con 0x55ce840d05d0 2024-09-12T02:16:57.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.393+0000 7fdafcff9640 1 -- 172.21.15.16:0/2157906543 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fdb1000fb30 con 0x55ce840d05d0 2024-09-12T02:16:57.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.393+0000 7fdb1da88740 1 -- 172.21.15.16:0/2157906543 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb04005520 con 0x55ce840d05d0 2024-09-12T02:16:57.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.393+0000 7fdafcff9640 1 -- 172.21.15.16:0/2157906543 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fdb1009ba20 con 0x55ce840d05d0 2024-09-12T02:16:57.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.393+0000 7fdb1da88740 10 client.20562 fetch_fsmap learned FSMap version 25 2024-09-12T02:16:57.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.393+0000 7fdb1da88740 1 -- 172.21.15.16:0/2157906543 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ce840e9ab0 con 0x55ce840d05d0 2024-09-12T02:16:57.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.393+0000 7fdafcff9640 1 -- 172.21.15.16:0/2157906543 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb04005520 con 0x55ce840d05d0 2024-09-12T02:16:57.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.393+0000 7fdafcff9640 1 -- 172.21.15.16:0/2157906543 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fdb100a4200 con 0x55ce840d05d0 2024-09-12T02:16:57.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.394+0000 7fdb1da88740 10 client.20562 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:16:57.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.394+0000 7fdb1da88740 10 client.20562 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T02:16:57.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.394+0000 7fdb1da88740 20 client.20562 populate_metadata read hostname 'smithi016' 2024-09-12T02:16:57.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.394+0000 7fdb1da88740 1 --2- 172.21.15.16:0/2157906543 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x55ce840eaec0 0x55ce840ed2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:57.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.394+0000 7fdb1da88740 4 client.20562 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:16:57.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.394+0000 7fdb1da88740 1 -- 172.21.15.16:0/2157906543 --> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ce84043860 con 0x55ce840eaec0 2024-09-12T02:16:57.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.394+0000 7fdb18c4a640 1 --2- 172.21.15.16:0/2157906543 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x55ce840eaec0 0x55ce840ed2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:57.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.394+0000 7fdb18c4a640 1 --2- 172.21.15.16:0/2157906543 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x55ce840eaec0 0x55ce840ed2b0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=dc7651cb461239f4 server_cookie=50c56e1de06c881f in_seq=0 out_seq=0 2024-09-12T02:16:57.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.394+0000 7fdafcff9640 10 client.20562 ms_handle_connect on v2:172.21.15.184:6838/4203347941 2024-09-12T02:16:57.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.395+0000 7fdafcff9640 1 -- 172.21.15.16:0/2157906543 <== mds.1 v2:172.21.15.184:6838/4203347941 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ce84043860 con 0x55ce840eaec0 2024-09-12T02:16:57.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.395+0000 7fdafcff9640 10 client.20562 handle_command_reply: tid=0 2024-09-12T02:16:57.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.395+0000 7fdb1da88740 1 client.20562 shutdown 2024-09-12T02:16:57.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.395+0000 7fdb1da88740 1 -- 172.21.15.16:0/2157906543 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce840d05d0 msgr2=0x55ce840dad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:57.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.395+0000 7fdb1da88740 1 --2- 172.21.15.16:0/2157906543 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce840d05d0 0x55ce840dad70 secure :-1 s=READY pgs=2041 cs=0 l=1 rev1=1 crypto rx=0x7fdb10000c00 tx=0x7fdb1001f300 comp rx=0 tx=0).stop 2024-09-12T02:16:57.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.395+0000 7fdb1da88740 1 -- 172.21.15.16:0/2157906543 shutdown_connections 2024-09-12T02:16:57.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.395+0000 7fdb1da88740 1 --2- 172.21.15.16:0/2157906543 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x55ce840eaec0 0x55ce840ed2b0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:57.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.395+0000 7fdb1da88740 1 --2- 172.21.15.16:0/2157906543 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ce840db370 0x55ce840d8610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:57.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.395+0000 7fdb1da88740 1 --2- 172.21.15.16:0/2157906543 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ce840d1b40 0x55ce840d8020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:57.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.395+0000 7fdb1da88740 1 --2- 172.21.15.16:0/2157906543 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce840d05d0 0x55ce840dad70 unknown :-1 s=CLOSED pgs=2041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:57.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.395+0000 7fdb1da88740 1 -- 172.21.15.16:0/2157906543 >> 172.21.15.16:0/2157906543 conn(0x55ce84099310 msgr2=0x55ce840d2930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:57.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.395+0000 7fdafcff9640 0 client.20562 ms_handle_reset on v2:172.21.15.184:6838/4203347941 2024-09-12T02:16:57.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.395+0000 7fdb1da88740 1 -- 172.21.15.16:0/2157906543 shutdown_connections 2024-09-12T02:16:57.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.396+0000 7fdb1da88740 1 -- 172.21.15.16:0/2157906543 wait complete. 2024-09-12T02:16:57.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.396+0000 7fdb1da88740 20 client.20562 trim_cache size 0 max 16384 2024-09-12T02:16:57.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.408+0000 7fdb1da88740 1 Processor -- start 2024-09-12T02:16:57.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.408+0000 7fdb1da88740 1 -- start start 2024-09-12T02:16:57.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.409+0000 7fdb1da88740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ce840d05d0 0x55ce840e72b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:57.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.409+0000 7fdb1da88740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce840d1b40 0x55ce840d78e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:57.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.409+0000 7fdb1da88740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ce840db370 0x55ce840d68f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:57.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.409+0000 7fdb1da88740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fdb141b6120 con 0x55ce840d1b40 2024-09-12T02:16:57.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.409+0000 7fdb1da88740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fdb141b39f0 con 0x55ce840d05d0 2024-09-12T02:16:57.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.409+0000 7fdb1da88740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fdb141b1840 con 0x55ce840db370 2024-09-12T02:16:57.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.409+0000 7fdb18c4a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce840d1b40 0x55ce840d78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:57.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.409+0000 7fdb18c4a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce840d1b40 0x55ce840d78e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58680/0 (socket says 172.21.15.16:58680) 2024-09-12T02:16:57.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.409+0000 7fdb18c4a640 1 -- 172.21.15.16:0/909393528 learned_addr learned my addr 172.21.15.16:0/909393528 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:16:57.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.409+0000 7fdb19c4c640 1 --2- 172.21.15.16:0/909393528 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ce840db370 0x55ce840d68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:57.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.409+0000 7fdb1944b640 1 --2- 172.21.15.16:0/909393528 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ce840d05d0 0x55ce840e72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:57.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.409+0000 7fdb18c4a640 1 -- 172.21.15.16:0/909393528 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ce840db370 msgr2=0x55ce840d68f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:57.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.409+0000 7fdb18c4a640 1 --2- 172.21.15.16:0/909393528 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ce840db370 0x55ce840d68f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:57.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.409+0000 7fdb18c4a640 1 -- 172.21.15.16:0/909393528 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ce840d05d0 msgr2=0x55ce840e72b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:57.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.409+0000 7fdb18c4a640 1 --2- 172.21.15.16:0/909393528 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ce840d05d0 0x55ce840e72b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:57.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.409+0000 7fdb18c4a640 1 -- 172.21.15.16:0/909393528 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce840d1760 con 0x55ce840d1b40 2024-09-12T02:16:57.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.409+0000 7fdb1944b640 1 --2- 172.21.15.16:0/909393528 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ce840d05d0 0x55ce840e72b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:16:57.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.409+0000 7fdb18c4a640 1 --2- 172.21.15.16:0/909393528 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce840d1b40 0x55ce840d78e0 secure :-1 s=READY pgs=2042 cs=0 l=1 rev1=1 crypto rx=0x7fdb040092a0 tx=0x7fdb0402a3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27442826ba3ffa34 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:57.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.410+0000 7fdafeffd640 1 -- 172.21.15.16:0/909393528 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb04035040 con 0x55ce840d1b40 2024-09-12T02:16:57.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.410+0000 7fdafeffd640 1 -- 172.21.15.16:0/909393528 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb04031540 con 0x55ce840d1b40 2024-09-12T02:16:57.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.410+0000 7fdafeffd640 1 -- 172.21.15.16:0/909393528 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb040506d0 con 0x55ce840d1b40 2024-09-12T02:16:57.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.410+0000 7fdb1da88740 1 -- 172.21.15.16:0/909393528 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce840d1b40 msgr2=0x55ce840d78e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:57.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.410+0000 7fdb1da88740 1 --2- 172.21.15.16:0/909393528 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce840d1b40 0x55ce840d78e0 secure :-1 s=READY pgs=2042 cs=0 l=1 rev1=1 crypto rx=0x7fdb040092a0 tx=0x7fdb0402a3f0 comp rx=0 tx=0).stop 2024-09-12T02:16:57.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.410+0000 7fdb1da88740 1 -- 172.21.15.16:0/909393528 shutdown_connections 2024-09-12T02:16:57.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.410+0000 7fdb1da88740 1 --2- 172.21.15.16:0/909393528 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ce840db370 0x55ce840d68f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:57.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.410+0000 7fdb1da88740 1 --2- 172.21.15.16:0/909393528 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce840d1b40 0x55ce840d78e0 unknown :-1 s=CLOSED pgs=2042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:57.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.410+0000 7fdb1da88740 1 --2- 172.21.15.16:0/909393528 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ce840d05d0 0x55ce840e72b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:57.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.410+0000 7fdb1da88740 1 -- 172.21.15.16:0/909393528 >> 172.21.15.16:0/909393528 conn(0x55ce84099310 msgr2=0x55ce840d32c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:57.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.410+0000 7fdb1da88740 1 -- 172.21.15.16:0/909393528 shutdown_connections 2024-09-12T02:16:57.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.410+0000 7fdb1da88740 1 -- 172.21.15.16:0/909393528 wait complete. 2024-09-12T02:16:57.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.411+0000 7fdb1da88740 1 -- start start 2024-09-12T02:16:57.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.411+0000 7fdb1da88740 1 Processor -- start 2024-09-12T02:16:57.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.411+0000 7fdb1da88740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce840d05d0 0x55ce840e3810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:57.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.411+0000 7fdb1da88740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ce840d1b40 0x55ce840e3d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:57.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.411+0000 7fdb1da88740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ce840db370 0x55ce840f2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:57.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.411+0000 7fdb1da88740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fdb141b1eb0 con 0x55ce840d05d0 2024-09-12T02:16:57.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.411+0000 7fdb1da88740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fdb141b3320 con 0x55ce840d1b40 2024-09-12T02:16:57.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.411+0000 7fdb1da88740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fdb141b2580 con 0x55ce840db370 2024-09-12T02:16:57.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.411+0000 7fdb1da88740 10 client.0 _ll_register_callbacks cb 0x7fdb100739f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:16:57.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.411+0000 7fdb1944b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce840d05d0 0x55ce840e3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:57.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.411+0000 7fdb1944b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce840d05d0 0x55ce840e3810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58682/0 (socket says 172.21.15.16:58682) 2024-09-12T02:16:57.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.411+0000 7fdb1944b640 1 -- 172.21.15.16:0/366070270 learned_addr learned my addr 172.21.15.16:0/366070270 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:16:57.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.411+0000 7fdb19c4c640 1 --2- 172.21.15.16:0/366070270 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ce840db370 0x55ce840f2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:57.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.412+0000 7fdb18c4a640 1 --2- 172.21.15.16:0/366070270 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ce840d1b40 0x55ce840e3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:57.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.412+0000 7fdb1944b640 1 -- 172.21.15.16:0/366070270 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ce840db370 msgr2=0x55ce840f2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:57.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.412+0000 7fdb1944b640 1 --2- 172.21.15.16:0/366070270 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ce840db370 0x55ce840f2370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:57.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.412+0000 7fdb1944b640 1 -- 172.21.15.16:0/366070270 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ce840d1b40 msgr2=0x55ce840e3d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:57.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.412+0000 7fdb1944b640 1 --2- 172.21.15.16:0/366070270 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ce840d1b40 0x55ce840e3d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:57.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.412+0000 7fdb1944b640 1 -- 172.21.15.16:0/366070270 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ce840f2ba0 con 0x55ce840d05d0 2024-09-12T02:16:57.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.412+0000 7fdb18c4a640 1 --2- 172.21.15.16:0/366070270 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ce840d1b40 0x55ce840e3d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:16:57.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.412+0000 7fdb1944b640 1 --2- 172.21.15.16:0/366070270 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce840d05d0 0x55ce840e3810 secure :-1 s=READY pgs=2043 cs=0 l=1 rev1=1 crypto rx=0x7fdb10097e30 tx=0x7fdb10097cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9857708b47e052a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:57.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.412+0000 7fdafcff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:16:57.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.412+0000 7fdafcff9640 1 -- 172.21.15.16:0/366070270 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb100b8030 con 0x55ce840d05d0 2024-09-12T02:16:57.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.412+0000 7fdafcff9640 1 -- 172.21.15.16:0/366070270 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fdb1000fb30 con 0x55ce840d05d0 2024-09-12T02:16:57.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.412+0000 7fdb1da88740 1 -- 172.21.15.16:0/366070270 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb04031540 con 0x55ce840d05d0 2024-09-12T02:16:57.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.412+0000 7fdafcff9640 1 -- 172.21.15.16:0/366070270 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fdb10078070 con 0x55ce840d05d0 2024-09-12T02:16:57.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.412+0000 7fdb1da88740 10 client.20568 fetch_fsmap learned FSMap version 25 2024-09-12T02:16:57.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.412+0000 7fdb1da88740 1 -- 172.21.15.16:0/366070270 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ce840e20a0 con 0x55ce840d05d0 2024-09-12T02:16:57.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.413+0000 7fdafcff9640 1 -- 172.21.15.16:0/366070270 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb04031540 con 0x55ce840d05d0 2024-09-12T02:16:57.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.413+0000 7fdafcff9640 1 -- 172.21.15.16:0/366070270 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fdb10097770 con 0x55ce840d05d0 2024-09-12T02:16:57.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.413+0000 7fdb1da88740 10 client.20568 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:16:57.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.413+0000 7fdb1da88740 10 client.20568 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T02:16:57.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.413+0000 7fdb1da88740 20 client.20568 populate_metadata read hostname 'smithi016' 2024-09-12T02:16:57.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.413+0000 7fdb1da88740 1 --2- 172.21.15.16:0/366070270 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x55ce84155d80 0x55ce841581c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:57.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.413+0000 7fdb1da88740 4 client.20568 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T02:16:57.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.413+0000 7fdb1da88740 1 -- 172.21.15.16:0/366070270 --> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ce8403dd70 con 0x55ce84155d80 2024-09-12T02:16:57.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.413+0000 7fdb18c4a640 1 --2- 172.21.15.16:0/366070270 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x55ce84155d80 0x55ce841581c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:57.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.413+0000 7fdb18c4a640 1 --2- 172.21.15.16:0/366070270 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x55ce84155d80 0x55ce841581c0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2bcb964556bfb0d0 server_cookie=933fd30939733fd5 in_seq=0 out_seq=0 2024-09-12T02:16:57.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.413+0000 7fdafcff9640 10 client.20568 ms_handle_connect on v2:172.21.15.184:6838/4203347941 2024-09-12T02:16:57.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.414+0000 7fdafcff9640 1 -- 172.21.15.16:0/366070270 <== mds.1 v2:172.21.15.184:6838/4203347941 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ce8403dd70 con 0x55ce84155d80 2024-09-12T02:16:57.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.414+0000 7fdafcff9640 10 client.20568 handle_command_reply: tid=0 2024-09-12T02:16:57.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.414+0000 7fdb1da88740 1 client.20568 shutdown 2024-09-12T02:16:57.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.414+0000 7fdb1da88740 1 -- 172.21.15.16:0/366070270 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce840d05d0 msgr2=0x55ce840e3810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:57.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.414+0000 7fdb1da88740 1 --2- 172.21.15.16:0/366070270 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce840d05d0 0x55ce840e3810 secure :-1 s=READY pgs=2043 cs=0 l=1 rev1=1 crypto rx=0x7fdb10097e30 tx=0x7fdb10097cc0 comp rx=0 tx=0).stop 2024-09-12T02:16:57.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.414+0000 7fdb1da88740 1 -- 172.21.15.16:0/366070270 shutdown_connections 2024-09-12T02:16:57.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.414+0000 7fdb1da88740 1 --2- 172.21.15.16:0/366070270 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x55ce84155d80 0x55ce841581c0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:57.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.414+0000 7fdb1da88740 1 --2- 172.21.15.16:0/366070270 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ce840db370 0x55ce840f2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:57.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.414+0000 7fdafcff9640 0 client.20568 ms_handle_reset on v2:172.21.15.184:6838/4203347941 2024-09-12T02:16:57.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.414+0000 7fdb1da88740 1 --2- 172.21.15.16:0/366070270 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ce840d1b40 0x55ce840e3d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:57.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.414+0000 7fdb1da88740 1 --2- 172.21.15.16:0/366070270 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce840d05d0 0x55ce840e3810 unknown :-1 s=CLOSED pgs=2043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:57.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.414+0000 7fdb1da88740 1 -- 172.21.15.16:0/366070270 >> 172.21.15.16:0/366070270 conn(0x55ce84099310 msgr2=0x55ce840d2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:57.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.414+0000 7fdb1da88740 1 -- 172.21.15.16:0/366070270 shutdown_connections 2024-09-12T02:16:57.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.414+0000 7fdb1da88740 1 -- 172.21.15.16:0/366070270 wait complete. 2024-09-12T02:16:57.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.414+0000 7fdb1da88740 20 client.20568 trim_cache size 0 max 16384 2024-09-12T02:16:57.421 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T02:16:57.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.414+0000 7fdb1b6af640 1 -- 172.21.15.16:0/2405642837 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fdb000a2c00 msgr2=0x7fdb000a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:57.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.414+0000 7fdb1b6af640 1 --2- 172.21.15.16:0/2405642837 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fdb000a2c00 0x7fdb000a50c0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fdb0c009370 tx=0x7fdb0c009e20 comp rx=0 tx=0).stop 2024-09-12T02:16:57.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.415+0000 7fdb1b6af640 1 -- 172.21.15.16:0/2405642837 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb141008c0 msgr2=0x7fdb141a4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:57.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.415+0000 7fdb1b6af640 1 --2- 172.21.15.16:0/2405642837 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb141008c0 0x7fdb141a4770 secure :-1 s=READY pgs=2039 cs=0 l=1 rev1=1 crypto rx=0x7fdb1000c920 tx=0x7fdb1000cde0 comp rx=0 tx=0).stop 2024-09-12T02:16:57.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.415+0000 7fdb1b6af640 1 -- 172.21.15.16:0/2405642837 shutdown_connections 2024-09-12T02:16:57.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.415+0000 7fdb1b6af640 1 --2- 172.21.15.16:0/2405642837 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fdb000a2c00 0x7fdb000a50c0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:57.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.415+0000 7fdb1b6af640 1 --2- 172.21.15.16:0/2405642837 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb14105ec0 0x7fdb1419e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:57.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.415+0000 7fdb1b6af640 1 --2- 172.21.15.16:0/2405642837 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdb141011e0 0x7fdb141a4cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:57.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.415+0000 7fdb1b6af640 1 --2- 172.21.15.16:0/2405642837 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb141008c0 0x7fdb141a4770 unknown :-1 s=CLOSED pgs=2039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:57.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.415+0000 7fdb1b6af640 1 -- 172.21.15.16:0/2405642837 >> 172.21.15.16:0/2405642837 conn(0x7fdb140fc7d0 msgr2=0x7fdb14103950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:57.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.415+0000 7fdb1b6af640 1 -- 172.21.15.16:0/2405642837 shutdown_connections 2024-09-12T02:16:57.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:57.415+0000 7fdb1b6af640 1 -- 172.21.15.16:0/2405642837 wait complete. 2024-09-12T02:16:58.142 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-12T02:16:58.398 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:16:58.693 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:16:58 smithi016 ceph-mon[30947]: pgmap v786: 97 pgs: 97 active+clean; 3.7 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-12T02:16:58.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.866+0000 7f0517bc4640 1 -- 172.21.15.16:0/1014319719 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0510101de0 msgr2=0x7f05101021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:58.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.866+0000 7f0517bc4640 1 --2- 172.21.15.16:0/1014319719 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0510101de0 0x7f05101021c0 secure :-1 s=READY pgs=2044 cs=0 l=1 rev1=1 crypto rx=0x7f050c009900 tx=0x7f050c02a410 comp rx=0 tx=0).stop 2024-09-12T02:16:58.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.866+0000 7f0517bc4640 1 -- 172.21.15.16:0/1014319719 shutdown_connections 2024-09-12T02:16:58.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.866+0000 7f0517bc4640 1 --2- 172.21.15.16:0/1014319719 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0510106f10 0x7f051010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:58.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.866+0000 7f0517bc4640 1 --2- 172.21.15.16:0/1014319719 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0510102790 0x7f05101067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:58.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.866+0000 7f0517bc4640 1 --2- 172.21.15.16:0/1014319719 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0510101de0 0x7f05101021c0 unknown :-1 s=CLOSED pgs=2044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:58.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.866+0000 7f0517bc4640 1 -- 172.21.15.16:0/1014319719 >> 172.21.15.16:0/1014319719 conn(0x7f05100fd8c0 msgr2=0x7f05100ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:58.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.866+0000 7f0517bc4640 1 -- 172.21.15.16:0/1014319719 shutdown_connections 2024-09-12T02:16:58.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.866+0000 7f0517bc4640 1 -- 172.21.15.16:0/1014319719 wait complete. 2024-09-12T02:16:58.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.867+0000 7f0517bc4640 1 Processor -- start 2024-09-12T02:16:58.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.867+0000 7f0517bc4640 1 -- start start 2024-09-12T02:16:58.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.867+0000 7f0517bc4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0510101de0 0x7f05100744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:58.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.867+0000 7f0517bc4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0510102790 0x7f0510074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:58.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.867+0000 7f0517bc4640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0510106f10 0x7f0510074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:58.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.867+0000 7f0517bc4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0510112010 con 0x7f0510102790 2024-09-12T02:16:58.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.867+0000 7f0517bc4640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0510111e90 con 0x7f0510101de0 2024-09-12T02:16:58.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.867+0000 7f0517bc4640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f0510112190 con 0x7f0510106f10 2024-09-12T02:16:58.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.867+0000 7f051515f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0510102790 0x7f0510074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:58.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.867+0000 7f051515f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0510102790 0x7f0510074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58692/0 (socket says 172.21.15.16:58692) 2024-09-12T02:16:58.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.868+0000 7f051515f640 1 -- 172.21.15.16:0/1754396649 learned_addr learned my addr 172.21.15.16:0/1754396649 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:16:58.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.868+0000 7f0515960640 1 --2- 172.21.15.16:0/1754396649 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0510101de0 0x7f05100744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:58.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.868+0000 7f0516161640 1 --2- 172.21.15.16:0/1754396649 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0510106f10 0x7f0510074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:58.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.868+0000 7f051515f640 1 -- 172.21.15.16:0/1754396649 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0510106f10 msgr2=0x7f0510074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:58.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.868+0000 7f051515f640 1 --2- 172.21.15.16:0/1754396649 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0510106f10 0x7f0510074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:58.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.868+0000 7f051515f640 1 -- 172.21.15.16:0/1754396649 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0510101de0 msgr2=0x7f05100744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:58.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.868+0000 7f051515f640 1 --2- 172.21.15.16:0/1754396649 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0510101de0 0x7f05100744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:58.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.868+0000 7f051515f640 1 -- 172.21.15.16:0/1754396649 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05101a6a70 con 0x7f0510102790 2024-09-12T02:16:58.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.868+0000 7f0515960640 1 --2- 172.21.15.16:0/1754396649 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0510101de0 0x7f05100744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:16:58.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.868+0000 7f051515f640 1 --2- 172.21.15.16:0/1754396649 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0510102790 0x7f0510074a30 secure :-1 s=READY pgs=2045 cs=0 l=1 rev1=1 crypto rx=0x7f050000b4a0 tx=0x7f050000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:58.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.868+0000 7f0506ffd640 1 -- 172.21.15.16:0/1754396649 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0500014070 con 0x7f0510102790 2024-09-12T02:16:58.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.868+0000 7f0506ffd640 1 -- 172.21.15.16:0/1754396649 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0500010af0 con 0x7f0510102790 2024-09-12T02:16:58.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.868+0000 7f0517bc4640 1 -- 172.21.15.16:0/1754396649 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05101a6d60 con 0x7f0510102790 2024-09-12T02:16:58.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.868+0000 7f0506ffd640 1 -- 172.21.15.16:0/1754396649 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0500017d10 con 0x7f0510102790 2024-09-12T02:16:58.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.868+0000 7f0517bc4640 1 -- 172.21.15.16:0/1754396649 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05101a7240 con 0x7f0510102790 2024-09-12T02:16:58.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.871+0000 7f0519f9d740 1 Processor -- start 2024-09-12T02:16:58.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.871+0000 7f0506ffd640 1 -- 172.21.15.16:0/1754396649 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f0500017070 con 0x7f0510102790 2024-09-12T02:16:58.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.871+0000 7f0519f9d740 1 -- start start 2024-09-12T02:16:58.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.871+0000 7f0519f9d740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55745cdf2b40 0x55745cdf1090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:58.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.871+0000 7f0519f9d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55745cdf15d0 0x55745cdf1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:58.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.871+0000 7f0519f9d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55745cdfc370 0x55745cdfe760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:58.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.871+0000 7f0519f9d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55745cd572f0 con 0x55745cdf15d0 2024-09-12T02:16:58.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.871+0000 7f0519f9d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55745ccfecf0 con 0x55745cdfc370 2024-09-12T02:16:58.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.871+0000 7f0519f9d740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55745cd532b0 con 0x55745cdf2b40 2024-09-12T02:16:58.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.871+0000 7f0515960640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55745cdf15d0 0x55745cdf1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:58.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.872+0000 7f0515960640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55745cdf15d0 0x55745cdf1a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58706/0 (socket says 172.21.15.16:58706) 2024-09-12T02:16:58.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.872+0000 7f0515960640 1 -- 172.21.15.16:0/1159736852 learned_addr learned my addr 172.21.15.16:0/1159736852 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:16:58.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.872+0000 7f0516161640 1 --2- 172.21.15.16:0/1159736852 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55745cdf2b40 0x55745cdf1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:58.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.872+0000 7f051515f640 1 --2- 172.21.15.16:0/1159736852 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55745cdfc370 0x55745cdfe760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:58.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.872+0000 7f0506ffd640 1 --2- 172.21.15.16:0/1754396649 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f04fc0a2b80 0x7f04fc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:58.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.872+0000 7f0515960640 1 -- 172.21.15.16:0/1159736852 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55745cdf2b40 msgr2=0x55745cdf1090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:58.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.872+0000 7f0515960640 1 --2- 172.21.15.16:0/1159736852 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55745cdf2b40 0x55745cdf1090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:58.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.872+0000 7f0515960640 1 -- 172.21.15.16:0/1159736852 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55745cdfc370 msgr2=0x55745cdfe760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:58.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.872+0000 7f0515960640 1 --2- 172.21.15.16:0/1159736852 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55745cdfc370 0x55745cdfe760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:58.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.872+0000 7f0515960640 1 -- 172.21.15.16:0/1159736852 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55745cce33a0 con 0x55745cdf15d0 2024-09-12T02:16:58.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.872+0000 7f051515f640 1 --2- 172.21.15.16:0/1159736852 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55745cdfc370 0x55745cdfe760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:16:58.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.872+0000 7f0506ffd640 1 -- 172.21.15.16:0/1754396649 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f05000ce3c0 con 0x7f0510102790 2024-09-12T02:16:58.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.872+0000 7f0516161640 1 --2- 172.21.15.16:0/1754396649 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f04fc0a2b80 0x7f04fc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:58.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.872+0000 7f0515960640 1 --2- 172.21.15.16:0/1159736852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55745cdf15d0 0x55745cdf1a10 secure :-1 s=READY pgs=2046 cs=0 l=1 rev1=1 crypto rx=0x7f050c009900 tx=0x7f050c0316e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d05166e5df45d0b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:58.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.872+0000 7f04fbfff640 1 -- 172.21.15.16:0/1159736852 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f050c03f070 con 0x55745cdf15d0 2024-09-12T02:16:58.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.873+0000 7f04fbfff640 1 -- 172.21.15.16:0/1159736852 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f050c0035f0 con 0x55745cdf15d0 2024-09-12T02:16:58.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.873+0000 7f04fbfff640 1 -- 172.21.15.16:0/1159736852 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f050c035070 con 0x55745cdf15d0 2024-09-12T02:16:58.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.873+0000 7f0519f9d740 1 -- 172.21.15.16:0/1159736852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55745cdf15d0 msgr2=0x55745cdf1a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:58.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.873+0000 7f0519f9d740 1 --2- 172.21.15.16:0/1159736852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55745cdf15d0 0x55745cdf1a10 secure :-1 s=READY pgs=2046 cs=0 l=1 rev1=1 crypto rx=0x7f050c009900 tx=0x7f050c0316e0 comp rx=0 tx=0).stop 2024-09-12T02:16:58.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.873+0000 7f0516161640 1 --2- 172.21.15.16:0/1754396649 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f04fc0a2b80 0x7f04fc0a5040 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f050800a4e0 tx=0x7f050800c040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:58.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.873+0000 7f0519f9d740 1 -- 172.21.15.16:0/1159736852 shutdown_connections 2024-09-12T02:16:58.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.873+0000 7f0519f9d740 1 --2- 172.21.15.16:0/1159736852 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55745cdfc370 0x55745cdfe760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:58.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.873+0000 7f0519f9d740 1 --2- 172.21.15.16:0/1159736852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55745cdf15d0 0x55745cdf1a10 unknown :-1 s=CLOSED pgs=2046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:58.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.873+0000 7f0519f9d740 1 --2- 172.21.15.16:0/1159736852 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55745cdf2b40 0x55745cdf1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:58.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.873+0000 7f0519f9d740 1 -- 172.21.15.16:0/1159736852 >> 172.21.15.16:0/1159736852 conn(0x55745cdba310 msgr2=0x55745cdedc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:58.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.873+0000 7f0519f9d740 1 -- 172.21.15.16:0/1159736852 shutdown_connections 2024-09-12T02:16:58.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.873+0000 7f0519f9d740 1 -- 172.21.15.16:0/1159736852 wait complete. 2024-09-12T02:16:58.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.874+0000 7f0519f9d740 1 -- start start 2024-09-12T02:16:58.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.874+0000 7f0519f9d740 1 Processor -- start 2024-09-12T02:16:58.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.874+0000 7f0519f9d740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55745cdf15d0 0x55745cdfbd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:58.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.874+0000 7f0519f9d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55745cdf2b40 0x55745cdf9000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:58.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.874+0000 7f0519f9d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55745cdfc370 0x55745cdf95d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:58.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.874+0000 7f0519f9d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55745cdff010 con 0x55745cdf2b40 2024-09-12T02:16:58.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.874+0000 7f0519f9d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55745cd12060 con 0x55745cdfc370 2024-09-12T02:16:58.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.874+0000 7f0519f9d740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55745cdff190 con 0x55745cdf15d0 2024-09-12T02:16:58.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.874+0000 7f0519f9d740 10 client.0 _ll_register_callbacks cb 0x55745ca658c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:16:58.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.874+0000 7f051515f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55745cdf2b40 0x55745cdf9000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:58.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.874+0000 7f051515f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55745cdf2b40 0x55745cdf9000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58708/0 (socket says 172.21.15.16:58708) 2024-09-12T02:16:58.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.874+0000 7f051515f640 1 -- 172.21.15.16:0/3017762179 learned_addr learned my addr 172.21.15.16:0/3017762179 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:16:58.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.875+0000 7f0515960640 1 --2- 172.21.15.16:0/3017762179 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55745cdf15d0 0x55745cdfbd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:58.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.875+0000 7f0516161640 1 --2- 172.21.15.16:0/3017762179 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55745cdfc370 0x55745cdf95d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:58.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.875+0000 7f051515f640 1 -- 172.21.15.16:0/3017762179 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55745cdf15d0 msgr2=0x55745cdfbd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:58.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.875+0000 7f051515f640 1 --2- 172.21.15.16:0/3017762179 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55745cdf15d0 0x55745cdfbd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:58.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.875+0000 7f051515f640 1 -- 172.21.15.16:0/3017762179 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55745cdfc370 msgr2=0x55745cdf95d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:58.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.875+0000 7f051515f640 1 --2- 172.21.15.16:0/3017762179 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55745cdfc370 0x55745cdf95d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:58.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.875+0000 7f051515f640 1 -- 172.21.15.16:0/3017762179 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55745cac83c0 con 0x55745cdf2b40 2024-09-12T02:16:58.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.875+0000 7f051515f640 1 --2- 172.21.15.16:0/3017762179 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55745cdf2b40 0x55745cdf9000 secure :-1 s=READY pgs=2047 cs=0 l=1 rev1=1 crypto rx=0x7f0500097120 tx=0x7f050009a800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=759d4235785844da server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:58.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.875+0000 7f04f97fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:16:58.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.875+0000 7f04f97fa640 1 -- 172.21.15.16:0/3017762179 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f050001e070 con 0x55745cdf2b40 2024-09-12T02:16:58.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.875+0000 7f04f97fa640 1 -- 172.21.15.16:0/3017762179 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f0500010ca0 con 0x55745cdf2b40 2024-09-12T02:16:58.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.875+0000 7f0519f9d740 1 -- 172.21.15.16:0/3017762179 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f050c0035f0 con 0x55745cdf2b40 2024-09-12T02:16:58.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.875+0000 7f04f97fa640 1 -- 172.21.15.16:0/3017762179 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f050009acc0 con 0x55745cdf2b40 2024-09-12T02:16:58.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.875+0000 7f0519f9d740 10 client.20580 fetch_fsmap learned FSMap version 25 2024-09-12T02:16:58.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.875+0000 7f0519f9d740 1 -- 172.21.15.16:0/3017762179 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55745ce0aac0 con 0x55745cdf2b40 2024-09-12T02:16:58.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.876+0000 7f04f97fa640 1 -- 172.21.15.16:0/3017762179 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f050c0035f0 con 0x55745cdf2b40 2024-09-12T02:16:58.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.876+0000 7f04f97fa640 1 -- 172.21.15.16:0/3017762179 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f05000a34a0 con 0x55745cdf2b40 2024-09-12T02:16:58.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.876+0000 7f0519f9d740 10 client.20580 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:16:58.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.876+0000 7f0519f9d740 10 client.20580 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:16:58.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.876+0000 7f0519f9d740 20 client.20580 populate_metadata read hostname 'smithi016' 2024-09-12T02:16:58.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.876+0000 7f0519f9d740 1 --2- 172.21.15.16:0/3017762179 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55745ce0bfd0 0x55745ce0e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:58.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.876+0000 7f0519f9d740 4 client.20580 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:16:58.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.876+0000 7f0519f9d740 1 -- 172.21.15.16:0/3017762179 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55745cd64860 con 0x55745ce0bfd0 2024-09-12T02:16:58.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.876+0000 7f0515960640 1 --2- 172.21.15.16:0/3017762179 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55745ce0bfd0 0x55745ce0e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:58.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.876+0000 7f0515960640 1 --2- 172.21.15.16:0/3017762179 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55745ce0bfd0 0x55745ce0e3c0 crc :-1 s=READY pgs=451 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aac1f44b1c15bb06 server_cookie=9a1500c14977a607 in_seq=0 out_seq=0 2024-09-12T02:16:58.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.876+0000 7f04f97fa640 10 client.20580 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:16:58.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.877+0000 7f04f97fa640 1 -- 172.21.15.16:0/3017762179 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55745cd64860 con 0x55745ce0bfd0 2024-09-12T02:16:58.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.877+0000 7f04f97fa640 10 client.20580 handle_command_reply: tid=0 2024-09-12T02:16:58.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.878+0000 7f0519f9d740 1 client.20580 shutdown 2024-09-12T02:16:58.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.878+0000 7f0519f9d740 1 -- 172.21.15.16:0/3017762179 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55745cdf2b40 msgr2=0x55745cdf9000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:58.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.878+0000 7f0519f9d740 1 --2- 172.21.15.16:0/3017762179 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55745cdf2b40 0x55745cdf9000 secure :-1 s=READY pgs=2047 cs=0 l=1 rev1=1 crypto rx=0x7f0500097120 tx=0x7f050009a800 comp rx=0 tx=0).stop 2024-09-12T02:16:58.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.878+0000 7f0519f9d740 1 -- 172.21.15.16:0/3017762179 shutdown_connections 2024-09-12T02:16:58.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.878+0000 7f0519f9d740 1 --2- 172.21.15.16:0/3017762179 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55745ce0bfd0 0x55745ce0e3c0 crc :-1 s=READY pgs=451 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:58.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.878+0000 7f0519f9d740 1 --2- 172.21.15.16:0/3017762179 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55745cdfc370 0x55745cdf95d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:58.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.878+0000 7f0519f9d740 1 --2- 172.21.15.16:0/3017762179 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55745cdf2b40 0x55745cdf9000 unknown :-1 s=CLOSED pgs=2047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:58.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.878+0000 7f0519f9d740 1 --2- 172.21.15.16:0/3017762179 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55745cdf15d0 0x55745cdfbd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:58.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.878+0000 7f04f97fa640 0 client.20580 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:16:58.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.878+0000 7f0519f9d740 1 -- 172.21.15.16:0/3017762179 >> 172.21.15.16:0/3017762179 conn(0x55745cdba310 msgr2=0x55745cdf3930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:58.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.878+0000 7f0519f9d740 1 -- 172.21.15.16:0/3017762179 shutdown_connections 2024-09-12T02:16:58.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.878+0000 7f0519f9d740 1 -- 172.21.15.16:0/3017762179 wait complete. 2024-09-12T02:16:58.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.878+0000 7f0519f9d740 20 client.20580 trim_cache size 0 max 16384 2024-09-12T02:16:58.887 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:16:58 smithi184 ceph-mon[35593]: pgmap v786: 97 pgs: 97 active+clean; 3.7 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-12T02:16:58.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.891+0000 7f0519f9d740 1 Processor -- start 2024-09-12T02:16:58.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.891+0000 7f0519f9d740 1 -- start start 2024-09-12T02:16:58.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.891+0000 7f0519f9d740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55745cdf15d0 0x55745cdede10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:58.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.891+0000 7f0519f9d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55745cdf2b40 0x55745ce082e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:58.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.891+0000 7f0519f9d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55745cdfc370 0x55745ce13c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:58.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.891+0000 7f0519f9d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f05101b1fe0 con 0x55745cdfc370 2024-09-12T02:16:58.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.891+0000 7f0519f9d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f05101af8b0 con 0x55745cdf2b40 2024-09-12T02:16:58.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.891+0000 7f0519f9d740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f05101ad700 con 0x55745cdf15d0 2024-09-12T02:16:58.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.891+0000 7f0516161640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55745cdfc370 0x55745ce13c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:58.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.891+0000 7f0516161640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55745cdfc370 0x55745ce13c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58714/0 (socket says 172.21.15.16:58714) 2024-09-12T02:16:58.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.891+0000 7f0516161640 1 -- 172.21.15.16:0/128999109 learned_addr learned my addr 172.21.15.16:0/128999109 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:16:58.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.891+0000 7f051515f640 1 --2- 172.21.15.16:0/128999109 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55745cdf2b40 0x55745ce082e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:58.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.891+0000 7f0515960640 1 --2- 172.21.15.16:0/128999109 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55745cdf15d0 0x55745cdede10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:58.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.892+0000 7f0516161640 1 -- 172.21.15.16:0/128999109 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55745cdf15d0 msgr2=0x55745cdede10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:58.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.892+0000 7f0516161640 1 --2- 172.21.15.16:0/128999109 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55745cdf15d0 0x55745cdede10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:58.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.892+0000 7f0516161640 1 -- 172.21.15.16:0/128999109 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55745cdf2b40 msgr2=0x55745ce082e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:58.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.892+0000 7f0516161640 1 --2- 172.21.15.16:0/128999109 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55745cdf2b40 0x55745ce082e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:58.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.892+0000 7f0516161640 1 -- 172.21.15.16:0/128999109 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55745cdfb970 con 0x55745cdfc370 2024-09-12T02:16:58.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.892+0000 7f051515f640 1 --2- 172.21.15.16:0/128999109 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55745cdf2b40 0x55745ce082e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:16:58.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.892+0000 7f0515960640 1 --2- 172.21.15.16:0/128999109 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55745cdf15d0 0x55745cdede10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:16:58.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.892+0000 7f0516161640 1 --2- 172.21.15.16:0/128999109 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55745cdfc370 0x55745ce13c90 secure :-1 s=READY pgs=2048 cs=0 l=1 rev1=1 crypto rx=0x7f0508009c50 tx=0x7f0508004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c907bb9c9140af6f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:58.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.892+0000 7f04fb7fe640 1 -- 172.21.15.16:0/128999109 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0508028070 con 0x55745cdfc370 2024-09-12T02:16:58.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.892+0000 7f04fb7fe640 1 -- 172.21.15.16:0/128999109 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f05080233f0 con 0x55745cdfc370 2024-09-12T02:16:58.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.892+0000 7f04fb7fe640 1 -- 172.21.15.16:0/128999109 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05080225f0 con 0x55745cdfc370 2024-09-12T02:16:58.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.892+0000 7f0519f9d740 1 -- 172.21.15.16:0/128999109 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55745cdfc370 msgr2=0x55745ce13c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:58.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.893+0000 7f0519f9d740 1 --2- 172.21.15.16:0/128999109 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55745cdfc370 0x55745ce13c90 secure :-1 s=READY pgs=2048 cs=0 l=1 rev1=1 crypto rx=0x7f0508009c50 tx=0x7f0508004a40 comp rx=0 tx=0).stop 2024-09-12T02:16:58.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.893+0000 7f0519f9d740 1 -- 172.21.15.16:0/128999109 shutdown_connections 2024-09-12T02:16:58.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.893+0000 7f0519f9d740 1 --2- 172.21.15.16:0/128999109 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55745cdfc370 0x55745ce13c90 unknown :-1 s=CLOSED pgs=2048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:58.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.893+0000 7f0519f9d740 1 --2- 172.21.15.16:0/128999109 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55745cdf2b40 0x55745ce082e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:58.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.893+0000 7f0519f9d740 1 --2- 172.21.15.16:0/128999109 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55745cdf15d0 0x55745cdede10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:58.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.893+0000 7f0519f9d740 1 -- 172.21.15.16:0/128999109 >> 172.21.15.16:0/128999109 conn(0x55745cdba310 msgr2=0x55745cded700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:58.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.893+0000 7f0519f9d740 1 -- 172.21.15.16:0/128999109 shutdown_connections 2024-09-12T02:16:58.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.893+0000 7f0519f9d740 1 -- 172.21.15.16:0/128999109 wait complete. 2024-09-12T02:16:58.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.893+0000 7f0519f9d740 1 -- start start 2024-09-12T02:16:58.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.893+0000 7f0519f9d740 1 Processor -- start 2024-09-12T02:16:58.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.893+0000 7f0519f9d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55745cdf15d0 0x55745ce04630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:58.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.893+0000 7f0519f9d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55745cdf2b40 0x55745ce04b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:58.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.893+0000 7f0519f9d740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55745cdfc370 0x55745ce01420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:58.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.894+0000 7f0519f9d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f05101add70 con 0x55745cdf15d0 2024-09-12T02:16:58.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.894+0000 7f0519f9d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f05101af1e0 con 0x55745cdf2b40 2024-09-12T02:16:58.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.894+0000 7f0519f9d740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f05101ae440 con 0x55745cdfc370 2024-09-12T02:16:58.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.894+0000 7f0519f9d740 10 client.0 _ll_register_callbacks cb 0x7f050000b3f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:16:58.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.894+0000 7f0515960640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55745cdf15d0 0x55745ce04630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:58.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.894+0000 7f0515960640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55745cdf15d0 0x55745ce04630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58718/0 (socket says 172.21.15.16:58718) 2024-09-12T02:16:58.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.894+0000 7f0515960640 1 -- 172.21.15.16:0/3881805549 learned_addr learned my addr 172.21.15.16:0/3881805549 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:16:58.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.894+0000 7f051515f640 1 --2- 172.21.15.16:0/3881805549 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55745cdf2b40 0x55745ce04b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:58.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.894+0000 7f0516161640 1 --2- 172.21.15.16:0/3881805549 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55745cdfc370 0x55745ce01420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:58.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.894+0000 7f0515960640 1 -- 172.21.15.16:0/3881805549 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55745cdfc370 msgr2=0x55745ce01420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:58.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.894+0000 7f0515960640 1 --2- 172.21.15.16:0/3881805549 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55745cdfc370 0x55745ce01420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:58.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.894+0000 7f0515960640 1 -- 172.21.15.16:0/3881805549 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55745cdf2b40 msgr2=0x55745ce04b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:16:58.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.894+0000 7f0515960640 1 --2- 172.21.15.16:0/3881805549 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55745cdf2b40 0x55745ce04b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:58.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.894+0000 7f0515960640 1 -- 172.21.15.16:0/3881805549 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55745ce01b60 con 0x55745cdf15d0 2024-09-12T02:16:58.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.894+0000 7f0516161640 1 --2- 172.21.15.16:0/3881805549 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55745cdfc370 0x55745ce01420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:16:58.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.894+0000 7f051515f640 1 --2- 172.21.15.16:0/3881805549 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55745cdf2b40 0x55745ce04b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:16:58.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.894+0000 7f0515960640 1 --2- 172.21.15.16:0/3881805549 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55745cdf15d0 0x55745ce04630 secure :-1 s=READY pgs=2049 cs=0 l=1 rev1=1 crypto rx=0x7f050c009900 tx=0x7f050c0058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=469e09d948ba32c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:58.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.894+0000 7f04f97fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:16:58.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.894+0000 7f04f97fa640 1 -- 172.21.15.16:0/3881805549 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f050c035040 con 0x55745cdf15d0 2024-09-12T02:16:58.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.894+0000 7f0519f9d740 1 -- 172.21.15.16:0/3881805549 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f05080233f0 con 0x55745cdf15d0 2024-09-12T02:16:58.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.894+0000 7f04f97fa640 1 -- 172.21.15.16:0/3881805549 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f050c003af0 con 0x55745cdf15d0 2024-09-12T02:16:58.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.894+0000 7f04f97fa640 1 -- 172.21.15.16:0/3881805549 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f050c03f040 con 0x55745cdf15d0 2024-09-12T02:16:58.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.895+0000 7f0519f9d740 10 client.20586 fetch_fsmap learned FSMap version 25 2024-09-12T02:16:58.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.895+0000 7f0519f9d740 1 -- 172.21.15.16:0/3881805549 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55745ce775e0 con 0x55745cdf15d0 2024-09-12T02:16:58.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.895+0000 7f04f97fa640 1 -- 172.21.15.16:0/3881805549 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f05080233f0 con 0x55745cdf15d0 2024-09-12T02:16:58.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.895+0000 7f04f97fa640 1 -- 172.21.15.16:0/3881805549 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f050c057020 con 0x55745cdf15d0 2024-09-12T02:16:58.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.895+0000 7f0519f9d740 10 client.20586 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:16:58.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.895+0000 7f0519f9d740 10 client.20586 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:16:58.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.895+0000 7f0519f9d740 20 client.20586 populate_metadata read hostname 'smithi016' 2024-09-12T02:16:58.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.895+0000 7f0519f9d740 1 --2- 172.21.15.16:0/3881805549 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55745ce77940 0x55745ce79d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:58.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.895+0000 7f0519f9d740 4 client.20586 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T02:16:58.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.895+0000 7f0519f9d740 1 -- 172.21.15.16:0/3881805549 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55745cd5ed70 con 0x55745ce77940 2024-09-12T02:16:58.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.895+0000 7f051515f640 1 --2- 172.21.15.16:0/3881805549 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55745ce77940 0x55745ce79d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:58.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.896+0000 7f051515f640 1 --2- 172.21.15.16:0/3881805549 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55745ce77940 0x55745ce79d60 crc :-1 s=READY pgs=452 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b178c99e1c5257 server_cookie=5991dda788982b64 in_seq=0 out_seq=0 2024-09-12T02:16:58.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.896+0000 7f04f97fa640 10 client.20586 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:16:58.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.896+0000 7f04f97fa640 1 -- 172.21.15.16:0/3881805549 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55745cd5ed70 con 0x55745ce77940 2024-09-12T02:16:58.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.896+0000 7f04f97fa640 10 client.20586 handle_command_reply: tid=0 2024-09-12T02:16:58.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.896+0000 7f0519f9d740 1 client.20586 shutdown 2024-09-12T02:16:58.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.896+0000 7f0519f9d740 1 -- 172.21.15.16:0/3881805549 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55745cdf15d0 msgr2=0x55745ce04630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:58.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.896+0000 7f0519f9d740 1 --2- 172.21.15.16:0/3881805549 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55745cdf15d0 0x55745ce04630 secure :-1 s=READY pgs=2049 cs=0 l=1 rev1=1 crypto rx=0x7f050c009900 tx=0x7f050c0058b0 comp rx=0 tx=0).stop 2024-09-12T02:16:58.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.896+0000 7f0519f9d740 1 -- 172.21.15.16:0/3881805549 shutdown_connections 2024-09-12T02:16:58.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.896+0000 7f0519f9d740 1 --2- 172.21.15.16:0/3881805549 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55745ce77940 0x55745ce79d60 crc :-1 s=READY pgs=452 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:58.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.896+0000 7f0519f9d740 1 --2- 172.21.15.16:0/3881805549 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55745cdfc370 0x55745ce01420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:58.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.896+0000 7f0519f9d740 1 --2- 172.21.15.16:0/3881805549 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55745cdf2b40 0x55745ce04b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:58.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.896+0000 7f04f97fa640 0 client.20586 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:16:58.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.896+0000 7f0519f9d740 1 --2- 172.21.15.16:0/3881805549 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55745cdf15d0 0x55745ce04630 unknown :-1 s=CLOSED pgs=2049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:58.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.896+0000 7f0519f9d740 1 -- 172.21.15.16:0/3881805549 >> 172.21.15.16:0/3881805549 conn(0x55745cdba310 msgr2=0x55745cdfa860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:58.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.896+0000 7f0519f9d740 1 -- 172.21.15.16:0/3881805549 shutdown_connections 2024-09-12T02:16:58.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.896+0000 7f0519f9d740 1 -- 172.21.15.16:0/3881805549 wait complete. 2024-09-12T02:16:58.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.896+0000 7f0519f9d740 20 client.20586 trim_cache size 0 max 16384 2024-09-12T02:16:58.906 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T02:16:58.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.897+0000 7f0517bc4640 1 -- 172.21.15.16:0/1754396649 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f04fc0a2b80 msgr2=0x7f04fc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:58.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.897+0000 7f0517bc4640 1 --2- 172.21.15.16:0/1754396649 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f04fc0a2b80 0x7f04fc0a5040 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f050800a4e0 tx=0x7f050800c040 comp rx=0 tx=0).stop 2024-09-12T02:16:58.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.897+0000 7f0517bc4640 1 -- 172.21.15.16:0/1754396649 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0510102790 msgr2=0x7f0510074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:58.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.897+0000 7f0517bc4640 1 --2- 172.21.15.16:0/1754396649 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0510102790 0x7f0510074a30 secure :-1 s=READY pgs=2045 cs=0 l=1 rev1=1 crypto rx=0x7f050000b4a0 tx=0x7f050000b960 comp rx=0 tx=0).stop 2024-09-12T02:16:58.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.897+0000 7f0517bc4640 1 -- 172.21.15.16:0/1754396649 shutdown_connections 2024-09-12T02:16:58.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.897+0000 7f0517bc4640 1 --2- 172.21.15.16:0/1754396649 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f04fc0a2b80 0x7f04fc0a5040 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:58.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.897+0000 7f0517bc4640 1 --2- 172.21.15.16:0/1754396649 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0510106f10 0x7f0510074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:58.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.897+0000 7f0517bc4640 1 --2- 172.21.15.16:0/1754396649 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0510102790 0x7f0510074a30 unknown :-1 s=CLOSED pgs=2045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:58.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.897+0000 7f0517bc4640 1 --2- 172.21.15.16:0/1754396649 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0510101de0 0x7f05100744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:58.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.897+0000 7f0517bc4640 1 -- 172.21.15.16:0/1754396649 >> 172.21.15.16:0/1754396649 conn(0x7f05100fd8c0 msgr2=0x7f05100fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:58.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.897+0000 7f0517bc4640 1 -- 172.21.15.16:0/1754396649 shutdown_connections 2024-09-12T02:16:58.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:16:58.897+0000 7f0517bc4640 1 -- 172.21.15.16:0/1754396649 wait complete. 2024-09-12T02:16:58.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:16:58 smithi033 ceph-mon[35332]: pgmap v786: 97 pgs: 97 active+clean; 3.7 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-12T02:17:00.287 INFO:tasks.workunit.client.0.smithi016.stdout:499/600: open fd = 3 2024-09-12T02:17:00.287 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:00.287 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:00.287 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:17:00.287 INFO:tasks.workunit.client.0.smithi016.stdout:500/600: open fd = 3 2024-09-12T02:17:00.287 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:00.287 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:00.287 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:17:00.287 INFO:tasks.workunit.client.0.smithi016.stdout:501/600: open fd = 3 2024-09-12T02:17:00.287 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:00.288 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:00.288 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:17:00.288 INFO:tasks.workunit.client.0.smithi016.stdout:502/600: open fd = 3 2024-09-12T02:17:00.288 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:00.288 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:00.288 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:17:00.288 INFO:tasks.workunit.client.0.smithi016.stdout:503/600: open fd = 3 2024-09-12T02:17:00.288 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:00.288 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:00.288 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:17:00.289 INFO:tasks.workunit.client.0.smithi016.stdout:504/600: open fd = 3 2024-09-12T02:17:00.289 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:00.289 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:00.289 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:17:00.289 INFO:tasks.workunit.client.0.smithi016.stdout:505/600: open fd = 3 2024-09-12T02:17:00.289 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:00.289 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:00.289 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:17:00.289 INFO:tasks.workunit.client.0.smithi016.stdout:506/600: open fd = 3 2024-09-12T02:17:00.289 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:00.289 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:00.290 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:17:00.290 INFO:tasks.workunit.client.0.smithi016.stdout:507/600: open fd = 3 2024-09-12T02:17:00.290 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:00.290 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:00.290 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:17:00.290 INFO:tasks.workunit.client.0.smithi016.stdout:508/600: open fd = 3 2024-09-12T02:17:00.290 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:00.290 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:00.291 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:17:00.291 INFO:tasks.workunit.client.0.smithi016.stdout:509/600: open fd = 3 2024-09-12T02:17:00.291 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:00.291 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:00.291 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:17:00.291 INFO:tasks.workunit.client.0.smithi016.stdout:510/600: open fd = 3 2024-09-12T02:17:00.291 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:00.291 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:00.291 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:17:00.291 INFO:tasks.workunit.client.0.smithi016.stdout:511/600: open fd = 3 2024-09-12T02:17:00.292 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:00.292 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:00.292 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:17:00.292 INFO:tasks.workunit.client.0.smithi016.stdout:512/600: open fd = 3 2024-09-12T02:17:00.292 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:00.292 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:00.292 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:17:00.292 INFO:tasks.workunit.client.0.smithi016.stdout:513/600: open fd = 3 2024-09-12T02:17:00.292 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:00.292 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:00.292 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:17:00.292 INFO:tasks.workunit.client.0.smithi016.stdout:514/600: open fd = 3 2024-09-12T02:17:00.292 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:00.292 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:00.293 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:17:00.293 INFO:tasks.workunit.client.0.smithi016.stdout:515/600: open fd = 3 2024-09-12T02:17:00.293 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:00.293 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:00.293 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:17:00.293 INFO:tasks.workunit.client.0.smithi016.stdout:516/600: open fd = 3 2024-09-12T02:17:00.293 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:00.293 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:00.293 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:17:00.293 INFO:tasks.workunit.client.0.smithi016.stdout:517/600: open fd = 3 2024-09-12T02:17:00.293 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:00.293 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:00.293 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:17:00.293 INFO:tasks.workunit.client.0.smithi016.stdout:518/600: open fd = 3 2024-09-12T02:17:00.294 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:00.294 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:00.294 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:17:00.294 INFO:tasks.workunit.client.0.smithi016.stdout:519/600: open fd = 3 2024-09-12T02:17:00.294 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:00.294 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:00.294 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:17:00.294 INFO:tasks.workunit.client.0.smithi016.stdout:520/600: open fd = 3 2024-09-12T02:17:00.294 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:00.295 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:00.295 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:17:00.295 INFO:tasks.workunit.client.0.smithi016.stdout:521/600: open fd = 3 2024-09-12T02:17:00.295 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:00.295 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:00.295 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:17:00.295 INFO:tasks.workunit.client.0.smithi016.stdout:522/600: open fd = 3 2024-09-12T02:17:00.295 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:00.295 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:00.295 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:17:00.295 INFO:tasks.workunit.client.0.smithi016.stdout:523/600: open fd = 3 2024-09-12T02:17:00.295 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:00.295 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:00.296 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:17:00.296 INFO:tasks.workunit.client.0.smithi016.stdout:524/600: open fd = 3 2024-09-12T02:17:00.296 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:00.296 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:00.296 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:17:00.296 INFO:tasks.workunit.client.0.smithi016.stdout:525/600: open fd = 3 2024-09-12T02:17:00.296 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:00.296 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:00.296 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:17:00.296 INFO:tasks.workunit.client.0.smithi016.stdout:526/600: open fd = 3 2024-09-12T02:17:00.296 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:00.296 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:00.297 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:17:00.297 INFO:tasks.workunit.client.0.smithi016.stdout:527/600: open fd = 3 2024-09-12T02:17:00.297 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:00.297 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:00.297 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:17:00.297 INFO:tasks.workunit.client.0.smithi016.stdout:528/600: open fd = 3 2024-09-12T02:17:00.297 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:00.297 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:00.297 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:17:00.297 INFO:tasks.workunit.client.0.smithi016.stdout:529/600: open fd = 3 2024-09-12T02:17:00.297 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:00.297 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:00.297 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:17:00.297 INFO:tasks.workunit.client.0.smithi016.stdout:530/600: open fd = 3 2024-09-12T02:17:00.298 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:00.298 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:00.298 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:17:00.298 INFO:tasks.workunit.client.0.smithi016.stdout:531/600: open fd = 3 2024-09-12T02:17:00.298 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:00.298 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:00.298 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:17:00.298 INFO:tasks.workunit.client.0.smithi016.stdout:532/600: open fd = 3 2024-09-12T02:17:00.298 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:00.298 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:00.298 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:17:00.298 INFO:tasks.workunit.client.0.smithi016.stdout:533/600: open fd = 3 2024-09-12T02:17:00.298 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:00.298 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:00.299 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:17:00.299 INFO:tasks.workunit.client.0.smithi016.stdout:534/600: open fd = 3 2024-09-12T02:17:00.299 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:00.299 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:00.299 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:17:00.299 INFO:tasks.workunit.client.0.smithi016.stdout:535/600: open fd = 3 2024-09-12T02:17:00.299 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:00.299 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:00.299 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:17:00.299 INFO:tasks.workunit.client.0.smithi016.stdout:536/600: open fd = 3 2024-09-12T02:17:00.299 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:00.299 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:00.299 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:17:00.299 INFO:tasks.workunit.client.0.smithi016.stdout:537/600: open fd = 3 2024-09-12T02:17:00.299 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:00.300 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:00.300 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:17:00.300 INFO:tasks.workunit.client.0.smithi016.stdout:538/600: open fd = 3 2024-09-12T02:17:00.301 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:00.301 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:00.301 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:17:00.301 INFO:tasks.workunit.client.0.smithi016.stdout:539/600: open fd = 3 2024-09-12T02:17:00.301 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:00.301 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:00.301 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:17:00.301 INFO:tasks.workunit.client.0.smithi016.stdout:540/600: open fd = 3 2024-09-12T02:17:00.301 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:00.301 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:00.301 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:17:00.301 INFO:tasks.workunit.client.0.smithi016.stdout:541/600: open fd = 3 2024-09-12T02:17:00.301 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:00.302 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:00.302 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:17:00.302 INFO:tasks.workunit.client.0.smithi016.stdout:542/600: open fd = 3 2024-09-12T02:17:00.302 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:00.302 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:00.302 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:17:00.302 INFO:tasks.workunit.client.0.smithi016.stdout:543/600: open fd = 3 2024-09-12T02:17:00.302 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:00.302 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:00.302 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:17:00.302 INFO:tasks.workunit.client.0.smithi016.stdout:544/600: open fd = 3 2024-09-12T02:17:00.302 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:00.302 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:00.302 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:17:00.302 INFO:tasks.workunit.client.0.smithi016.stdout:545/600: open fd = 3 2024-09-12T02:17:00.303 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:00.303 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:00.303 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:17:00.303 INFO:tasks.workunit.client.0.smithi016.stdout:546/600: open fd = 3 2024-09-12T02:17:00.303 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:00.303 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:00.303 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:17:00.303 INFO:tasks.workunit.client.0.smithi016.stdout:547/600: open fd = 3 2024-09-12T02:17:00.303 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:00.303 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:00.303 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:17:00.303 INFO:tasks.workunit.client.0.smithi016.stdout:548/600: open fd = 3 2024-09-12T02:17:00.303 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:00.303 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:00.304 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:17:00.304 INFO:tasks.workunit.client.0.smithi016.stdout:549/600: open fd = 3 2024-09-12T02:17:00.304 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:00.304 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:00.304 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:17:00.304 INFO:tasks.workunit.client.0.smithi016.stdout:550/600: open fd = 3 2024-09-12T02:17:00.304 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:00.304 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:00.304 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:17:00.304 INFO:tasks.workunit.client.0.smithi016.stdout:551/600: open fd = 3 2024-09-12T02:17:00.304 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:00.304 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:00.304 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:17:00.305 INFO:tasks.workunit.client.0.smithi016.stdout:552/600: open fd = 3 2024-09-12T02:17:00.305 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:00.305 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:00.305 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:17:00.305 INFO:tasks.workunit.client.0.smithi016.stdout:553/600: open fd = 3 2024-09-12T02:17:00.305 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:00.305 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:00.305 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:17:00.305 INFO:tasks.workunit.client.0.smithi016.stdout:554/600: open fd = 3 2024-09-12T02:17:00.305 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:00.305 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:00.305 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:17:00.305 INFO:tasks.workunit.client.0.smithi016.stdout:555/600: open fd = 3 2024-09-12T02:17:00.305 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:00.306 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:00.306 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:17:00.306 INFO:tasks.workunit.client.0.smithi016.stdout:556/600: open fd = 3 2024-09-12T02:17:00.306 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:00.306 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:00.306 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:17:00.306 INFO:tasks.workunit.client.0.smithi016.stdout:557/600: open fd = 3 2024-09-12T02:17:00.306 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:00.306 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:00.306 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:17:00.306 INFO:tasks.workunit.client.0.smithi016.stdout:558/600: open fd = 3 2024-09-12T02:17:00.306 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:00.306 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:00.306 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:17:00.307 INFO:tasks.workunit.client.0.smithi016.stdout:559/600: open fd = 3 2024-09-12T02:17:00.307 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:00.307 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:00.307 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:17:00.307 INFO:tasks.workunit.client.0.smithi016.stdout:560/600: open fd = 3 2024-09-12T02:17:00.307 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:00.307 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:00.307 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:17:00.704 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-12T02:17:00.704 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-12T02:17:00.729 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:17:00 smithi016 ceph-mon[30947]: pgmap v787: 97 pgs: 97 active+clean; 3.7 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-09-12T02:17:00.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:17:00 smithi184 ceph-mon[35593]: pgmap v787: 97 pgs: 97 active+clean; 3.7 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-09-12T02:17:00.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:17:00 smithi033 ceph-mon[35332]: pgmap v787: 97 pgs: 97 active+clean; 3.7 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-09-12T02:17:00.965 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:17:01.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.446+0000 7fa6580d0640 1 -- 172.21.15.16:0/813316817 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa650074590 msgr2=0x7fa650074970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:01.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.446+0000 7fa6580d0640 1 --2- 172.21.15.16:0/813316817 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa650074590 0x7fa650074970 secure :-1 s=READY pgs=2050 cs=0 l=1 rev1=1 crypto rx=0x7fa64c009900 tx=0x7fa64c02a480 comp rx=0 tx=0).stop 2024-09-12T02:17:01.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.446+0000 7fa6580d0640 1 -- 172.21.15.16:0/813316817 shutdown_connections 2024-09-12T02:17:01.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.447+0000 7fa6580d0640 1 --2- 172.21.15.16:0/813316817 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa650112ba0 0x7fa650114f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:01.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.447+0000 7fa6580d0640 1 --2- 172.21.15.16:0/813316817 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa650074eb0 0x7fa650112660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:01.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.447+0000 7fa6580d0640 1 --2- 172.21.15.16:0/813316817 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa650074590 0x7fa650074970 unknown :-1 s=CLOSED pgs=2050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:01.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.447+0000 7fa6580d0640 1 -- 172.21.15.16:0/813316817 >> 172.21.15.16:0/813316817 conn(0x7fa6500fd8f0 msgr2=0x7fa6500ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:01.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.447+0000 7fa6580d0640 1 -- 172.21.15.16:0/813316817 shutdown_connections 2024-09-12T02:17:01.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.447+0000 7fa6580d0640 1 -- 172.21.15.16:0/813316817 wait complete. 2024-09-12T02:17:01.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.447+0000 7fa6580d0640 1 Processor -- start 2024-09-12T02:17:01.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.447+0000 7fa6580d0640 1 -- start start 2024-09-12T02:17:01.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.447+0000 7fa6580d0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa650074590 0x7fa6501a4730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:01.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.447+0000 7fa6580d0640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa650074eb0 0x7fa6501a4c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:01.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.447+0000 7fa6580d0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa650112ba0 0x7fa65019e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:01.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.447+0000 7fa6580d0640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa6501185b0 con 0x7fa650074590 2024-09-12T02:17:01.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.447+0000 7fa6580d0640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa650118430 con 0x7fa650112ba0 2024-09-12T02:17:01.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.447+0000 7fa6580d0640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa650118730 con 0x7fa650074eb0 2024-09-12T02:17:01.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.448+0000 7fa655e6c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa650074590 0x7fa6501a4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:01.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.448+0000 7fa655e6c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa650074590 0x7fa6501a4730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46152/0 (socket says 172.21.15.16:46152) 2024-09-12T02:17:01.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.448+0000 7fa655e6c640 1 -- 172.21.15.16:0/2147205893 learned_addr learned my addr 172.21.15.16:0/2147205893 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:17:01.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.448+0000 7fa65666d640 1 --2- 172.21.15.16:0/2147205893 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa650112ba0 0x7fa65019e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:01.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.448+0000 7fa65566b640 1 --2- 172.21.15.16:0/2147205893 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa650074eb0 0x7fa6501a4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:01.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.448+0000 7fa655e6c640 1 -- 172.21.15.16:0/2147205893 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa650074eb0 msgr2=0x7fa6501a4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:01.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.448+0000 7fa655e6c640 1 --2- 172.21.15.16:0/2147205893 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa650074eb0 0x7fa6501a4c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:01.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.448+0000 7fa655e6c640 1 -- 172.21.15.16:0/2147205893 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa650112ba0 msgr2=0x7fa65019e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:01.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.448+0000 7fa655e6c640 1 --2- 172.21.15.16:0/2147205893 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa650112ba0 0x7fa65019e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:01.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.448+0000 7fa655e6c640 1 -- 172.21.15.16:0/2147205893 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa65019f020 con 0x7fa650074590 2024-09-12T02:17:01.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.448+0000 7fa65666d640 1 --2- 172.21.15.16:0/2147205893 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa650112ba0 0x7fa65019e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:17:01.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.448+0000 7fa655e6c640 1 --2- 172.21.15.16:0/2147205893 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa650074590 0x7fa6501a4730 secure :-1 s=READY pgs=2051 cs=0 l=1 rev1=1 crypto rx=0x7fa64c004770 tx=0x7fa64c003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:01.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.448+0000 7fa646ffd640 1 -- 172.21.15.16:0/2147205893 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa64c03f070 con 0x7fa650074590 2024-09-12T02:17:01.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.448+0000 7fa6580d0640 1 -- 172.21.15.16:0/2147205893 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa65019f2b0 con 0x7fa650074590 2024-09-12T02:17:01.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.448+0000 7fa646ffd640 1 -- 172.21.15.16:0/2147205893 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa64c031ca0 con 0x7fa650074590 2024-09-12T02:17:01.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.449+0000 7fa646ffd640 1 -- 172.21.15.16:0/2147205893 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa64c035030 con 0x7fa650074590 2024-09-12T02:17:01.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.449+0000 7fa6580d0640 1 -- 172.21.15.16:0/2147205893 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6501ab600 con 0x7fa650074590 2024-09-12T02:17:01.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.452+0000 7fa646ffd640 1 -- 172.21.15.16:0/2147205893 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa64c042070 con 0x7fa650074590 2024-09-12T02:17:01.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.452+0000 7fa65a4a9740 1 Processor -- start 2024-09-12T02:17:01.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.452+0000 7fa65a4a9740 1 -- start start 2024-09-12T02:17:01.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.452+0000 7fa65a4a9740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5570dd498b60 0x5570dd498f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:01.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.452+0000 7fa65a4a9740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5570dd499480 0x5570dd4a3800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:01.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.452+0000 7fa65a4a9740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570dd4a3d40 0x5570dd4a6130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:01.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.452+0000 7fa65a4a9740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5570dd3fe280 con 0x5570dd4a3d40 2024-09-12T02:17:01.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.452+0000 7fa65a4a9740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5570dd3a5b10 con 0x5570dd498b60 2024-09-12T02:17:01.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.452+0000 7fa65a4a9740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5570dd3fa240 con 0x5570dd499480 2024-09-12T02:17:01.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.452+0000 7fa655e6c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5570dd499480 0x5570dd4a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:01.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.452+0000 7fa655e6c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5570dd499480 0x5570dd4a3800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:35658/0 (socket says 172.21.15.16:35658) 2024-09-12T02:17:01.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.452+0000 7fa655e6c640 1 -- 172.21.15.16:0/1197335418 learned_addr learned my addr 172.21.15.16:0/1197335418 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:17:01.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.452+0000 7fa65566b640 1 --2- 172.21.15.16:0/1197335418 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570dd4a3d40 0x5570dd4a6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:01.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.453+0000 7fa65666d640 1 --2- 172.21.15.16:0/1197335418 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5570dd498b60 0x5570dd498f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:01.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.453+0000 7fa646ffd640 1 --2- 172.21.15.16:0/2147205893 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fa63c0a2b80 0x7fa63c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:01.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.453+0000 7fa65666d640 1 --2- 172.21.15.16:0/2147205893 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fa63c0a2b80 0x7fa63c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:01.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.453+0000 7fa646ffd640 1 -- 172.21.15.16:0/2147205893 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fa64c0ef780 con 0x7fa650074590 2024-09-12T02:17:01.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.453+0000 7fa65666d640 1 -- 172.21.15.16:0/1197335418 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5570dd499480 msgr2=0x5570dd4a3800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:01.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.453+0000 7fa65666d640 1 --2- 172.21.15.16:0/1197335418 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5570dd499480 0x5570dd4a3800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:01.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.453+0000 7fa65666d640 1 -- 172.21.15.16:0/1197335418 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570dd4a3d40 msgr2=0x5570dd4a6130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:01.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.453+0000 7fa65666d640 1 --2- 172.21.15.16:0/1197335418 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570dd4a3d40 0x5570dd4a6130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:01.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.453+0000 7fa65666d640 1 -- 172.21.15.16:0/1197335418 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570dd3b21f0 con 0x5570dd498b60 2024-09-12T02:17:01.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.453+0000 7fa655e6c640 1 --2- 172.21.15.16:0/1197335418 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5570dd499480 0x5570dd4a3800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:17:01.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.453+0000 7fa65666d640 1 --2- 172.21.15.16:0/1197335418 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5570dd498b60 0x5570dd498f40 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fa648005fd0 tx=0x7fa648014dd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=822ed4c7986de93f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:01.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.453+0000 7fa65666d640 1 --2- 172.21.15.16:0/2147205893 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fa63c0a2b80 0x7fa63c0a5040 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fa64801c000 tx=0x7fa64801e000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:01.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.453+0000 7fa63bfff640 1 -- 172.21.15.16:0/1197335418 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa648023070 con 0x5570dd498b60 2024-09-12T02:17:01.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.454+0000 7fa63bfff640 1 -- 172.21.15.16:0/1197335418 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa64801e3f0 con 0x5570dd498b60 2024-09-12T02:17:01.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.454+0000 7fa63bfff640 1 -- 172.21.15.16:0/1197335418 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa64800f5c0 con 0x5570dd498b60 2024-09-12T02:17:01.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.454+0000 7fa65a4a9740 1 -- 172.21.15.16:0/1197335418 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5570dd498b60 msgr2=0x5570dd498f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:01.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.454+0000 7fa65a4a9740 1 --2- 172.21.15.16:0/1197335418 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5570dd498b60 0x5570dd498f40 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fa648005fd0 tx=0x7fa648014dd0 comp rx=0 tx=0).stop 2024-09-12T02:17:01.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.454+0000 7fa65a4a9740 1 -- 172.21.15.16:0/1197335418 shutdown_connections 2024-09-12T02:17:01.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.454+0000 7fa65a4a9740 1 --2- 172.21.15.16:0/1197335418 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570dd4a3d40 0x5570dd4a6130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:01.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.454+0000 7fa65a4a9740 1 --2- 172.21.15.16:0/1197335418 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5570dd499480 0x5570dd4a3800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:01.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.454+0000 7fa65a4a9740 1 --2- 172.21.15.16:0/1197335418 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5570dd498b60 0x5570dd498f40 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:01.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.454+0000 7fa65a4a9740 1 -- 172.21.15.16:0/1197335418 >> 172.21.15.16:0/1197335418 conn(0x5570dd4950f0 msgr2=0x5570dd4954d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:01.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.454+0000 7fa65a4a9740 1 -- 172.21.15.16:0/1197335418 shutdown_connections 2024-09-12T02:17:01.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.454+0000 7fa65a4a9740 1 -- 172.21.15.16:0/1197335418 wait complete. 2024-09-12T02:17:01.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.455+0000 7fa65a4a9740 1 -- start start 2024-09-12T02:17:01.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.455+0000 7fa65a4a9740 1 Processor -- start 2024-09-12T02:17:01.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.455+0000 7fa65a4a9740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5570dd498b60 0x5570dd4a0c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:01.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.455+0000 7fa65a4a9740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570dd499480 0x5570dd4a11d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:01.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.455+0000 7fa65a4a9740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5570dd4a3d40 0x5570dd4a1710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:01.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.455+0000 7fa65a4a9740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5570dd4a6910 con 0x5570dd499480 2024-09-12T02:17:01.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.455+0000 7fa65a4a9740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5570dd3b8f10 con 0x5570dd498b60 2024-09-12T02:17:01.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.455+0000 7fa65a4a9740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5570dd4a6a90 con 0x5570dd4a3d40 2024-09-12T02:17:01.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.455+0000 7fa65a4a9740 10 client.0 _ll_register_callbacks cb 0x5570dd381a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:17:01.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.455+0000 7fa65566b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570dd499480 0x5570dd4a11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:01.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.455+0000 7fa65566b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570dd499480 0x5570dd4a11d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46158/0 (socket says 172.21.15.16:46158) 2024-09-12T02:17:01.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.455+0000 7fa65566b640 1 -- 172.21.15.16:0/1583471912 learned_addr learned my addr 172.21.15.16:0/1583471912 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:17:01.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.456+0000 7fa65666d640 1 --2- 172.21.15.16:0/1583471912 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5570dd4a3d40 0x5570dd4a1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:01.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.456+0000 7fa655e6c640 1 --2- 172.21.15.16:0/1583471912 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5570dd498b60 0x5570dd4a0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:01.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.456+0000 7fa65566b640 1 -- 172.21.15.16:0/1583471912 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5570dd4a3d40 msgr2=0x5570dd4a1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:01.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.456+0000 7fa65566b640 1 --2- 172.21.15.16:0/1583471912 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5570dd4a3d40 0x5570dd4a1710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:01.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.456+0000 7fa65566b640 1 -- 172.21.15.16:0/1583471912 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5570dd498b60 msgr2=0x5570dd4a0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:01.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.456+0000 7fa65566b640 1 --2- 172.21.15.16:0/1583471912 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5570dd498b60 0x5570dd4a0c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:01.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.456+0000 7fa65566b640 1 -- 172.21.15.16:0/1583471912 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570dd16f6a0 con 0x5570dd499480 2024-09-12T02:17:01.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.456+0000 7fa655e6c640 1 --2- 172.21.15.16:0/1583471912 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5570dd498b60 0x5570dd4a0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:17:01.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.456+0000 7fa65566b640 1 --2- 172.21.15.16:0/1583471912 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570dd499480 0x5570dd4a11d0 secure :-1 s=READY pgs=2052 cs=0 l=1 rev1=1 crypto rx=0x7fa640013740 tx=0x7fa640013c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85962a82752b65cb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:01.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.456+0000 7fa6397fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:17:01.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.456+0000 7fa6397fa640 1 -- 172.21.15.16:0/1583471912 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa640020060 con 0x5570dd499480 2024-09-12T02:17:01.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.456+0000 7fa6397fa640 1 -- 172.21.15.16:0/1583471912 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fa640011dd0 con 0x5570dd499480 2024-09-12T02:17:01.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.456+0000 7fa65a4a9740 1 -- 172.21.15.16:0/1583471912 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa64801e3f0 con 0x5570dd499480 2024-09-12T02:17:01.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.456+0000 7fa6397fa640 1 -- 172.21.15.16:0/1583471912 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa64000f450 con 0x5570dd499480 2024-09-12T02:17:01.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.456+0000 7fa65a4a9740 10 client.20598 fetch_fsmap learned FSMap version 25 2024-09-12T02:17:01.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.456+0000 7fa65a4a9740 1 -- 172.21.15.16:0/1583471912 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5570dd4b25f0 con 0x5570dd499480 2024-09-12T02:17:01.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.457+0000 7fa6397fa640 1 -- 172.21.15.16:0/1583471912 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa64801e3f0 con 0x5570dd499480 2024-09-12T02:17:01.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.457+0000 7fa6397fa640 1 -- 172.21.15.16:0/1583471912 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fa640025070 con 0x5570dd499480 2024-09-12T02:17:01.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.457+0000 7fa65a4a9740 10 client.20598 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:17:01.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.457+0000 7fa65a4a9740 10 client.20598 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:17:01.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.457+0000 7fa65a4a9740 20 client.20598 populate_metadata read hostname 'smithi016' 2024-09-12T02:17:01.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.457+0000 7fa65a4a9740 1 --2- 172.21.15.16:0/1583471912 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5570dd4b3ab0 0x5570dd4b5f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:01.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.457+0000 7fa65a4a9740 4 client.20598 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:17:01.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.457+0000 7fa65a4a9740 1 -- 172.21.15.16:0/1583471912 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5570dd40b7f0 con 0x5570dd4b3ab0 2024-09-12T02:17:01.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.457+0000 7fa655e6c640 1 --2- 172.21.15.16:0/1583471912 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5570dd4b3ab0 0x5570dd4b5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:01.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.458+0000 7fa655e6c640 1 --2- 172.21.15.16:0/1583471912 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5570dd4b3ab0 0x5570dd4b5f00 crc :-1 s=READY pgs=453 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81582fa583e1435 server_cookie=756b34684cd06065 in_seq=0 out_seq=0 2024-09-12T02:17:01.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.458+0000 7fa6397fa640 10 client.20598 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:17:01.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.458+0000 7fa6397fa640 1 -- 172.21.15.16:0/1583471912 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5570dd40b7f0 con 0x5570dd4b3ab0 2024-09-12T02:17:01.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.458+0000 7fa6397fa640 10 client.20598 handle_command_reply: tid=0 2024-09-12T02:17:01.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.458+0000 7fa65a4a9740 1 client.20598 shutdown 2024-09-12T02:17:01.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.458+0000 7fa65a4a9740 1 -- 172.21.15.16:0/1583471912 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570dd499480 msgr2=0x5570dd4a11d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:01.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.458+0000 7fa65a4a9740 1 --2- 172.21.15.16:0/1583471912 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570dd499480 0x5570dd4a11d0 secure :-1 s=READY pgs=2052 cs=0 l=1 rev1=1 crypto rx=0x7fa640013740 tx=0x7fa640013c00 comp rx=0 tx=0).stop 2024-09-12T02:17:01.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.459+0000 7fa65a4a9740 1 -- 172.21.15.16:0/1583471912 shutdown_connections 2024-09-12T02:17:01.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.459+0000 7fa65a4a9740 1 --2- 172.21.15.16:0/1583471912 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5570dd4b3ab0 0x5570dd4b5f00 crc :-1 s=READY pgs=453 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:01.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.459+0000 7fa65a4a9740 1 --2- 172.21.15.16:0/1583471912 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5570dd4a3d40 0x5570dd4a1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:01.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.459+0000 7fa65a4a9740 1 --2- 172.21.15.16:0/1583471912 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570dd499480 0x5570dd4a11d0 unknown :-1 s=CLOSED pgs=2052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:01.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.459+0000 7fa65a4a9740 1 --2- 172.21.15.16:0/1583471912 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5570dd498b60 0x5570dd4a0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:01.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.459+0000 7fa65a4a9740 1 -- 172.21.15.16:0/1583471912 >> 172.21.15.16:0/1583471912 conn(0x5570dd4950f0 msgr2=0x5570dd49ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:01.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.459+0000 7fa6397fa640 0 client.20598 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:17:01.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.459+0000 7fa65a4a9740 1 -- 172.21.15.16:0/1583471912 shutdown_connections 2024-09-12T02:17:01.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.459+0000 7fa65a4a9740 1 -- 172.21.15.16:0/1583471912 wait complete. 2024-09-12T02:17:01.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.459+0000 7fa65a4a9740 20 client.20598 trim_cache size 0 max 16384 2024-09-12T02:17:01.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.476+0000 7fa65a4a9740 1 Processor -- start 2024-09-12T02:17:01.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.476+0000 7fa65a4a9740 1 -- start start 2024-09-12T02:17:01.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.476+0000 7fa65a4a9740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5570dd498b60 0x5570dd49b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:01.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.476+0000 7fa65a4a9740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5570dd499480 0x5570dd4bb810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:01.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.476+0000 7fa65a4a9740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570dd4a3d40 0x5570dd4bbd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:01.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.476+0000 7fa65a4a9740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa6501b6310 con 0x5570dd4a3d40 2024-09-12T02:17:01.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.476+0000 7fa65a4a9740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa6501b3be0 con 0x5570dd499480 2024-09-12T02:17:01.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.476+0000 7fa65a4a9740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa6501b1a30 con 0x5570dd498b60 2024-09-12T02:17:01.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.476+0000 7fa65666d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570dd4a3d40 0x5570dd4bbd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:01.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.476+0000 7fa655e6c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5570dd498b60 0x5570dd49b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:01.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.476+0000 7fa65666d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570dd4a3d40 0x5570dd4bbd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46164/0 (socket says 172.21.15.16:46164) 2024-09-12T02:17:01.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.476+0000 7fa65666d640 1 -- 172.21.15.16:0/1607033221 learned_addr learned my addr 172.21.15.16:0/1607033221 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:17:01.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.477+0000 7fa65566b640 1 --2- 172.21.15.16:0/1607033221 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5570dd499480 0x5570dd4bb810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:01.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.477+0000 7fa65666d640 1 -- 172.21.15.16:0/1607033221 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5570dd498b60 msgr2=0x5570dd49b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:01.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.477+0000 7fa65666d640 1 --2- 172.21.15.16:0/1607033221 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5570dd498b60 0x5570dd49b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:01.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.477+0000 7fa65666d640 1 -- 172.21.15.16:0/1607033221 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5570dd499480 msgr2=0x5570dd4bb810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:01.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.477+0000 7fa65666d640 1 --2- 172.21.15.16:0/1607033221 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5570dd499480 0x5570dd4bb810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:01.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.477+0000 7fa65666d640 1 -- 172.21.15.16:0/1607033221 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570dd4961a0 con 0x5570dd4a3d40 2024-09-12T02:17:01.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.477+0000 7fa65666d640 1 --2- 172.21.15.16:0/1607033221 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570dd4a3d40 0x5570dd4bbd50 secure :-1 s=READY pgs=2053 cs=0 l=1 rev1=1 crypto rx=0x7fa648009700 tx=0x7fa648009730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=230e53d8770cad57 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:01.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.477+0000 7fa63b7fe640 1 -- 172.21.15.16:0/1607033221 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa648023040 con 0x5570dd4a3d40 2024-09-12T02:17:01.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.477+0000 7fa63b7fe640 1 -- 172.21.15.16:0/1607033221 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa648014470 con 0x5570dd4a3d40 2024-09-12T02:17:01.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.477+0000 7fa63b7fe640 1 -- 172.21.15.16:0/1607033221 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa648029a60 con 0x5570dd4a3d40 2024-09-12T02:17:01.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.478+0000 7fa65a4a9740 1 -- 172.21.15.16:0/1607033221 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570dd4a3d40 msgr2=0x5570dd4bbd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:01.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.478+0000 7fa65a4a9740 1 --2- 172.21.15.16:0/1607033221 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570dd4a3d40 0x5570dd4bbd50 secure :-1 s=READY pgs=2053 cs=0 l=1 rev1=1 crypto rx=0x7fa648009700 tx=0x7fa648009730 comp rx=0 tx=0).stop 2024-09-12T02:17:01.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.478+0000 7fa65a4a9740 1 -- 172.21.15.16:0/1607033221 shutdown_connections 2024-09-12T02:17:01.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.478+0000 7fa65a4a9740 1 --2- 172.21.15.16:0/1607033221 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570dd4a3d40 0x5570dd4bbd50 unknown :-1 s=CLOSED pgs=2053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:01.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.478+0000 7fa65a4a9740 1 --2- 172.21.15.16:0/1607033221 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5570dd499480 0x5570dd4bb810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:01.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.478+0000 7fa65a4a9740 1 --2- 172.21.15.16:0/1607033221 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5570dd498b60 0x5570dd49b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:01.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.478+0000 7fa65a4a9740 1 -- 172.21.15.16:0/1607033221 >> 172.21.15.16:0/1607033221 conn(0x5570dd4950f0 msgr2=0x5570dd4a2c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:01.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.478+0000 7fa65a4a9740 1 -- 172.21.15.16:0/1607033221 shutdown_connections 2024-09-12T02:17:01.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.478+0000 7fa65a4a9740 1 -- 172.21.15.16:0/1607033221 wait complete. 2024-09-12T02:17:01.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.478+0000 7fa65a4a9740 1 -- start start 2024-09-12T02:17:01.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.478+0000 7fa65a4a9740 1 Processor -- start 2024-09-12T02:17:01.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.479+0000 7fa65a4a9740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5570dd498b60 0x5570dd4a7e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:01.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.479+0000 7fa65a4a9740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570dd499480 0x5570dd4a8350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:01.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.479+0000 7fa65a4a9740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5570dd4a3d40 0x5570dd51d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:01.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.479+0000 7fa65a4a9740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa6501b20a0 con 0x5570dd499480 2024-09-12T02:17:01.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.479+0000 7fa65a4a9740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa6501b3510 con 0x5570dd4a3d40 2024-09-12T02:17:01.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.479+0000 7fa65a4a9740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa6501b2770 con 0x5570dd498b60 2024-09-12T02:17:01.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.479+0000 7fa65a4a9740 10 client.0 _ll_register_callbacks cb 0x7fa640009610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:17:01.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.479+0000 7fa65566b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570dd499480 0x5570dd4a8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:01.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.479+0000 7fa65566b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570dd499480 0x5570dd4a8350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46168/0 (socket says 172.21.15.16:46168) 2024-09-12T02:17:01.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.479+0000 7fa65566b640 1 -- 172.21.15.16:0/3049788627 learned_addr learned my addr 172.21.15.16:0/3049788627 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:17:01.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.479+0000 7fa655e6c640 1 --2- 172.21.15.16:0/3049788627 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5570dd498b60 0x5570dd4a7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:01.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.479+0000 7fa65666d640 1 --2- 172.21.15.16:0/3049788627 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5570dd4a3d40 0x5570dd51d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:01.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.479+0000 7fa65566b640 1 -- 172.21.15.16:0/3049788627 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5570dd498b60 msgr2=0x5570dd4a7e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:01.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.479+0000 7fa65566b640 1 --2- 172.21.15.16:0/3049788627 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5570dd498b60 0x5570dd4a7e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:01.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.479+0000 7fa65566b640 1 -- 172.21.15.16:0/3049788627 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5570dd4a3d40 msgr2=0x5570dd51d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:01.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.479+0000 7fa65566b640 1 --2- 172.21.15.16:0/3049788627 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5570dd4a3d40 0x5570dd51d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:01.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.479+0000 7fa65566b640 1 -- 172.21.15.16:0/3049788627 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570dd51dd20 con 0x5570dd499480 2024-09-12T02:17:01.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.480+0000 7fa65566b640 1 --2- 172.21.15.16:0/3049788627 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570dd499480 0x5570dd4a8350 secure :-1 s=READY pgs=2054 cs=0 l=1 rev1=1 crypto rx=0x7fa640000c00 tx=0x7fa640011e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b27e0e6819db61b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:01.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.480+0000 7fa6397fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:17:01.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.480+0000 7fa6397fa640 1 -- 172.21.15.16:0/3049788627 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa64002b030 con 0x5570dd499480 2024-09-12T02:17:01.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.480+0000 7fa6397fa640 1 -- 172.21.15.16:0/3049788627 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fa640026c80 con 0x5570dd499480 2024-09-12T02:17:01.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.480+0000 7fa65a4a9740 1 -- 172.21.15.16:0/3049788627 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa648014470 con 0x5570dd499480 2024-09-12T02:17:01.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.480+0000 7fa6397fa640 1 -- 172.21.15.16:0/3049788627 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa640020020 con 0x5570dd499480 2024-09-12T02:17:01.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.480+0000 7fa65a4a9740 10 client.20604 fetch_fsmap learned FSMap version 25 2024-09-12T02:17:01.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.480+0000 7fa65a4a9740 1 -- 172.21.15.16:0/3049788627 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5570dd51ef70 con 0x5570dd499480 2024-09-12T02:17:01.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.480+0000 7fa6397fa640 1 -- 172.21.15.16:0/3049788627 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa648014470 con 0x5570dd499480 2024-09-12T02:17:01.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.480+0000 7fa6397fa640 1 -- 172.21.15.16:0/3049788627 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fa640025070 con 0x5570dd499480 2024-09-12T02:17:01.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.480+0000 7fa65a4a9740 10 client.20604 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:17:01.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.480+0000 7fa65a4a9740 10 client.20604 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:17:01.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.480+0000 7fa65a4a9740 20 client.20604 populate_metadata read hostname 'smithi016' 2024-09-12T02:17:01.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.480+0000 7fa65a4a9740 1 --2- 172.21.15.16:0/3049788627 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5570dd51f720 0x5570dd521b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:01.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.480+0000 7fa65a4a9740 4 client.20604 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-12T02:17:01.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.480+0000 7fa65a4a9740 1 -- 172.21.15.16:0/3049788627 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5570dd4b6290 con 0x5570dd51f720 2024-09-12T02:17:01.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.481+0000 7fa655e6c640 1 --2- 172.21.15.16:0/3049788627 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5570dd51f720 0x5570dd521b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:01.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.481+0000 7fa655e6c640 1 --2- 172.21.15.16:0/3049788627 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5570dd51f720 0x5570dd521b40 crc :-1 s=READY pgs=454 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90f294de084f0265 server_cookie=804d37d399e5acc9 in_seq=0 out_seq=0 2024-09-12T02:17:01.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.481+0000 7fa6397fa640 10 client.20604 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:17:01.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.482+0000 7fa6397fa640 1 -- 172.21.15.16:0/3049788627 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5570dd4b6290 con 0x5570dd51f720 2024-09-12T02:17:01.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.482+0000 7fa6397fa640 10 client.20604 handle_command_reply: tid=0 2024-09-12T02:17:01.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.482+0000 7fa65a4a9740 1 client.20604 shutdown 2024-09-12T02:17:01.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.482+0000 7fa65a4a9740 1 -- 172.21.15.16:0/3049788627 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570dd499480 msgr2=0x5570dd4a8350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:01.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.482+0000 7fa65a4a9740 1 --2- 172.21.15.16:0/3049788627 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570dd499480 0x5570dd4a8350 secure :-1 s=READY pgs=2054 cs=0 l=1 rev1=1 crypto rx=0x7fa640000c00 tx=0x7fa640011e00 comp rx=0 tx=0).stop 2024-09-12T02:17:01.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.482+0000 7fa65a4a9740 1 -- 172.21.15.16:0/3049788627 shutdown_connections 2024-09-12T02:17:01.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.482+0000 7fa65a4a9740 1 --2- 172.21.15.16:0/3049788627 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5570dd51f720 0x5570dd521b40 crc :-1 s=READY pgs=454 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:01.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.482+0000 7fa65a4a9740 1 --2- 172.21.15.16:0/3049788627 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5570dd4a3d40 0x5570dd51d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:01.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.482+0000 7fa65a4a9740 1 --2- 172.21.15.16:0/3049788627 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570dd499480 0x5570dd4a8350 unknown :-1 s=CLOSED pgs=2054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:01.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.482+0000 7fa6397fa640 0 client.20604 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:17:01.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.482+0000 7fa65a4a9740 1 --2- 172.21.15.16:0/3049788627 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5570dd498b60 0x5570dd4a7e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:01.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.482+0000 7fa65a4a9740 1 -- 172.21.15.16:0/3049788627 >> 172.21.15.16:0/3049788627 conn(0x5570dd4950f0 msgr2=0x5570dd4a2300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:01.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.482+0000 7fa65a4a9740 1 -- 172.21.15.16:0/3049788627 shutdown_connections 2024-09-12T02:17:01.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.483+0000 7fa65a4a9740 1 -- 172.21.15.16:0/3049788627 wait complete. 2024-09-12T02:17:01.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.483+0000 7fa65a4a9740 20 client.20604 trim_cache size 0 max 16384 2024-09-12T02:17:01.487 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T02:17:01.487 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-09-12T02:17:01.488 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "2a2b33f6-80fc-435b-8450-0975f2267ee0", 2024-09-12T02:17:01.488 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-09-12T02:17:01.488 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T02:17:01.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.483+0000 7fa6580d0640 1 -- 172.21.15.16:0/2147205893 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fa63c0a2b80 msgr2=0x7fa63c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:01.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.483+0000 7fa6580d0640 1 --2- 172.21.15.16:0/2147205893 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fa63c0a2b80 0x7fa63c0a5040 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fa64801c000 tx=0x7fa64801e000 comp rx=0 tx=0).stop 2024-09-12T02:17:01.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.483+0000 7fa6580d0640 1 -- 172.21.15.16:0/2147205893 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa650074590 msgr2=0x7fa6501a4730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:01.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.483+0000 7fa6580d0640 1 --2- 172.21.15.16:0/2147205893 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa650074590 0x7fa6501a4730 secure :-1 s=READY pgs=2051 cs=0 l=1 rev1=1 crypto rx=0x7fa64c004770 tx=0x7fa64c003c10 comp rx=0 tx=0).stop 2024-09-12T02:17:01.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.483+0000 7fa6580d0640 1 -- 172.21.15.16:0/2147205893 shutdown_connections 2024-09-12T02:17:01.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.483+0000 7fa6580d0640 1 --2- 172.21.15.16:0/2147205893 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fa63c0a2b80 0x7fa63c0a5040 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:01.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.483+0000 7fa6580d0640 1 --2- 172.21.15.16:0/2147205893 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa650112ba0 0x7fa65019e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:01.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.483+0000 7fa6580d0640 1 --2- 172.21.15.16:0/2147205893 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa650074eb0 0x7fa6501a4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:01.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.483+0000 7fa6580d0640 1 --2- 172.21.15.16:0/2147205893 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa650074590 0x7fa6501a4730 unknown :-1 s=CLOSED pgs=2051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:01.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.483+0000 7fa6580d0640 1 -- 172.21.15.16:0/2147205893 >> 172.21.15.16:0/2147205893 conn(0x7fa6500fd8f0 msgr2=0x7fa6500ff080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:01.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.483+0000 7fa6580d0640 1 -- 172.21.15.16:0/2147205893 shutdown_connections 2024-09-12T02:17:01.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:01.483+0000 7fa6580d0640 1 -- 172.21.15.16:0/2147205893 wait complete. 2024-09-12T02:17:02.150 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T02:17:02.450 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:17:02.699 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:17:02 smithi016 ceph-mon[30947]: pgmap v788: 97 pgs: 97 active+clean; 3.7 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-09-12T02:17:02.699 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:17:02 smithi016 ceph-mon[30947]: scrub queued for path: / 2024-09-12T02:17:02.699 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:17:02 smithi016 ceph-mon[30947]: scrub summary: idle+waiting paths [/] 2024-09-12T02:17:02.699 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:17:02 smithi016 ceph-mon[30947]: scrub summary: active paths [/] 2024-09-12T02:17:02.699 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:17:02 smithi016 ceph-mon[30947]: scrub summary: idle+waiting paths [/] 2024-09-12T02:17:02.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:17:02 smithi184 ceph-mon[35593]: pgmap v788: 97 pgs: 97 active+clean; 3.7 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-09-12T02:17:02.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:17:02 smithi184 ceph-mon[35593]: scrub queued for path: / 2024-09-12T02:17:02.878 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:17:02 smithi184 ceph-mon[35593]: scrub summary: idle+waiting paths [/] 2024-09-12T02:17:02.878 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:17:02 smithi184 ceph-mon[35593]: scrub summary: active paths [/] 2024-09-12T02:17:02.878 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:17:02 smithi184 ceph-mon[35593]: scrub summary: idle+waiting paths [/] 2024-09-12T02:17:02.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.917+0000 7fd08984f640 1 -- 172.21.15.16:0/2099326573 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd084101420 msgr2=0x7fd08410d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:02.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.917+0000 7fd08984f640 1 --2- 172.21.15.16:0/2099326573 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd084101420 0x7fd08410d950 secure :-1 s=READY pgs=2055 cs=0 l=1 rev1=1 crypto rx=0x7fd07c009cd0 tx=0x7fd07c02a8e0 comp rx=0 tx=0).stop 2024-09-12T02:17:02.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.917+0000 7fd08984f640 1 -- 172.21.15.16:0/2099326573 shutdown_connections 2024-09-12T02:17:02.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.917+0000 7fd08984f640 1 --2- 172.21.15.16:0/2099326573 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd084101420 0x7fd08410d950 unknown :-1 s=CLOSED pgs=2055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:02.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.917+0000 7fd08984f640 1 --2- 172.21.15.16:0/2099326573 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd084100a80 0x7fd084100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:02.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.917+0000 7fd08984f640 1 --2- 172.21.15.16:0/2099326573 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd084106a80 0x7fd084106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:02.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.917+0000 7fd08984f640 1 -- 172.21.15.16:0/2099326573 >> 172.21.15.16:0/2099326573 conn(0x7fd0840fc7a0 msgr2=0x7fd0840febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:02.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.917+0000 7fd08984f640 1 -- 172.21.15.16:0/2099326573 shutdown_connections 2024-09-12T02:17:02.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.917+0000 7fd08984f640 1 -- 172.21.15.16:0/2099326573 wait complete. 2024-09-12T02:17:02.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.918+0000 7fd08984f640 1 Processor -- start 2024-09-12T02:17:02.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.918+0000 7fd08984f640 1 -- start start 2024-09-12T02:17:02.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.918+0000 7fd08984f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd084100a80 0x7fd084198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:02.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.918+0000 7fd08984f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd084101420 0x7fd0841989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:02.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.918+0000 7fd08984f640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd084106a80 0x7fd08419cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:02.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.918+0000 7fd08984f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd084110ea0 con 0x7fd084101420 2024-09-12T02:17:02.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.918+0000 7fd08984f640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd084110d20 con 0x7fd084100a80 2024-09-12T02:17:02.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.918+0000 7fd08984f640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd084111020 con 0x7fd084106a80 2024-09-12T02:17:02.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.918+0000 7fd07bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd084101420 0x7fd0841989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:02.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.918+0000 7fd07bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd084101420 0x7fd0841989c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46198/0 (socket says 172.21.15.16:46198) 2024-09-12T02:17:02.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.918+0000 7fd07bfff640 1 -- 172.21.15.16:0/2304355827 learned_addr learned my addr 172.21.15.16:0/2304355827 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:17:02.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.919+0000 7fd082ffd640 1 --2- 172.21.15.16:0/2304355827 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd084100a80 0x7fd084198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:02.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.919+0000 7fd0837fe640 1 --2- 172.21.15.16:0/2304355827 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd084106a80 0x7fd08419cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:02.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.919+0000 7fd07bfff640 1 -- 172.21.15.16:0/2304355827 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd084106a80 msgr2=0x7fd08419cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:02.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.919+0000 7fd07bfff640 1 --2- 172.21.15.16:0/2304355827 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd084106a80 0x7fd08419cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:02.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.919+0000 7fd07bfff640 1 -- 172.21.15.16:0/2304355827 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd084100a80 msgr2=0x7fd084198480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:02.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.919+0000 7fd07bfff640 1 --2- 172.21.15.16:0/2304355827 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd084100a80 0x7fd084198480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:02.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.919+0000 7fd07bfff640 1 -- 172.21.15.16:0/2304355827 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd08419d430 con 0x7fd084101420 2024-09-12T02:17:02.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.919+0000 7fd07bfff640 1 --2- 172.21.15.16:0/2304355827 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd084101420 0x7fd0841989c0 secure :-1 s=READY pgs=2056 cs=0 l=1 rev1=1 crypto rx=0x7fd06c00b4a0 tx=0x7fd06c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:02.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.919+0000 7fd08884d640 1 -- 172.21.15.16:0/2304355827 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd06c014070 con 0x7fd084101420 2024-09-12T02:17:02.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.919+0000 7fd08884d640 1 -- 172.21.15.16:0/2304355827 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd06c010af0 con 0x7fd084101420 2024-09-12T02:17:02.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.919+0000 7fd08984f640 1 -- 172.21.15.16:0/2304355827 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd08419d720 con 0x7fd084101420 2024-09-12T02:17:02.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.919+0000 7fd08884d640 1 -- 172.21.15.16:0/2304355827 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd06c017d10 con 0x7fd084101420 2024-09-12T02:17:02.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.919+0000 7fd08984f640 1 -- 172.21.15.16:0/2304355827 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0841a5000 con 0x7fd084101420 2024-09-12T02:17:02.926 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:17:02 smithi033 ceph-mon[35332]: pgmap v788: 97 pgs: 97 active+clean; 3.7 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-09-12T02:17:02.926 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:17:02 smithi033 ceph-mon[35332]: scrub queued for path: / 2024-09-12T02:17:02.926 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:17:02 smithi033 ceph-mon[35332]: scrub summary: idle+waiting paths [/] 2024-09-12T02:17:02.926 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:17:02 smithi033 ceph-mon[35332]: scrub summary: active paths [/] 2024-09-12T02:17:02.926 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:17:02 smithi033 ceph-mon[35332]: scrub summary: idle+waiting paths [/] 2024-09-12T02:17:02.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.922+0000 7fd08884d640 1 -- 172.21.15.16:0/2304355827 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fd06c017070 con 0x7fd084101420 2024-09-12T02:17:02.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.922+0000 7fd08bc28740 1 Processor -- start 2024-09-12T02:17:02.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.922+0000 7fd08bc28740 1 -- start start 2024-09-12T02:17:02.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.922+0000 7fd08bc28740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f62da424a0 0x55f62da42880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:02.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.922+0000 7fd08bc28740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f62da42dc0 0x55f62da4d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:02.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.922+0000 7fd08bc28740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f62da4d680 0x55f62da4fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:02.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.922+0000 7fd08bc28740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55f62d9a82e0 con 0x55f62da42dc0 2024-09-12T02:17:02.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.922+0000 7fd08bc28740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55f62d94fce0 con 0x55f62da424a0 2024-09-12T02:17:02.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.922+0000 7fd08bc28740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55f62d9a42a0 con 0x55f62da4d680 2024-09-12T02:17:02.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.922+0000 7fd082ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f62da42dc0 0x55f62da4d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:02.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.922+0000 7fd082ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f62da42dc0 0x55f62da4d140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46214/0 (socket says 172.21.15.16:46214) 2024-09-12T02:17:02.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.922+0000 7fd082ffd640 1 -- 172.21.15.16:0/3641779551 learned_addr learned my addr 172.21.15.16:0/3641779551 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:17:02.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.922+0000 7fd0837fe640 1 --2- 172.21.15.16:0/3641779551 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f62da424a0 0x55f62da42880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:02.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.922+0000 7fd07bfff640 1 --2- 172.21.15.16:0/3641779551 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f62da4d680 0x55f62da4fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:02.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.923+0000 7fd082ffd640 1 -- 172.21.15.16:0/3641779551 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f62da4d680 msgr2=0x55f62da4fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:02.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.923+0000 7fd082ffd640 1 --2- 172.21.15.16:0/3641779551 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f62da4d680 0x55f62da4fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:02.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.923+0000 7fd082ffd640 1 -- 172.21.15.16:0/3641779551 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f62da424a0 msgr2=0x55f62da42880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:02.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.923+0000 7fd082ffd640 1 --2- 172.21.15.16:0/3641779551 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f62da424a0 0x55f62da42880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:02.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.923+0000 7fd082ffd640 1 -- 172.21.15.16:0/3641779551 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f62d934440 con 0x55f62da42dc0 2024-09-12T02:17:02.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.923+0000 7fd08884d640 1 --2- 172.21.15.16:0/2304355827 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fd0700a2bc0 0x7fd0700a5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:02.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.923+0000 7fd082ffd640 1 --2- 172.21.15.16:0/3641779551 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f62da42dc0 0x55f62da4d140 secure :-1 s=READY pgs=2057 cs=0 l=1 rev1=1 crypto rx=0x7fd07400dcb0 tx=0x7fd07400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbfb635f1a87b82f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:02.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.923+0000 7fd0837fe640 1 --2- 172.21.15.16:0/2304355827 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fd0700a2bc0 0x7fd0700a5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:02.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.923+0000 7fd08884d640 1 -- 172.21.15.16:0/2304355827 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fd06c0ce420 con 0x7fd084101420 2024-09-12T02:17:02.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.923+0000 7fd079ffb640 1 -- 172.21.15.16:0/3641779551 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd074011440 con 0x55f62da42dc0 2024-09-12T02:17:02.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.923+0000 7fd079ffb640 1 -- 172.21.15.16:0/3641779551 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd074011ac0 con 0x55f62da42dc0 2024-09-12T02:17:02.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.923+0000 7fd079ffb640 1 -- 172.21.15.16:0/3641779551 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd07401ba20 con 0x55f62da42dc0 2024-09-12T02:17:02.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.924+0000 7fd08bc28740 1 -- 172.21.15.16:0/3641779551 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f62da42dc0 msgr2=0x55f62da4d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:02.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.924+0000 7fd08bc28740 1 --2- 172.21.15.16:0/3641779551 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f62da42dc0 0x55f62da4d140 secure :-1 s=READY pgs=2057 cs=0 l=1 rev1=1 crypto rx=0x7fd07400dcb0 tx=0x7fd07400f730 comp rx=0 tx=0).stop 2024-09-12T02:17:02.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.924+0000 7fd0837fe640 1 --2- 172.21.15.16:0/2304355827 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fd0700a2bc0 0x7fd0700a5080 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fd07c02aed0 tx=0x7fd07c009800 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:02.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.924+0000 7fd08bc28740 1 -- 172.21.15.16:0/3641779551 shutdown_connections 2024-09-12T02:17:02.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.924+0000 7fd08bc28740 1 --2- 172.21.15.16:0/3641779551 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f62da4d680 0x55f62da4fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:02.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.924+0000 7fd08bc28740 1 --2- 172.21.15.16:0/3641779551 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f62da42dc0 0x55f62da4d140 unknown :-1 s=CLOSED pgs=2057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:02.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.924+0000 7fd08bc28740 1 --2- 172.21.15.16:0/3641779551 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f62da424a0 0x55f62da42880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:02.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.924+0000 7fd08bc28740 1 -- 172.21.15.16:0/3641779551 >> 172.21.15.16:0/3641779551 conn(0x55f62da3ea30 msgr2=0x55f62da3ee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:02.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.924+0000 7fd08bc28740 1 -- 172.21.15.16:0/3641779551 shutdown_connections 2024-09-12T02:17:02.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.924+0000 7fd08bc28740 1 -- 172.21.15.16:0/3641779551 wait complete. 2024-09-12T02:17:02.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.924+0000 7fd08bc28740 1 -- start start 2024-09-12T02:17:02.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.925+0000 7fd08bc28740 1 Processor -- start 2024-09-12T02:17:02.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.925+0000 7fd08bc28740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f62da424a0 0x55f62da4d000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:02.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.925+0000 7fd08bc28740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f62da42dc0 0x55f62da4a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:02.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.925+0000 7fd08bc28740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f62da4d680 0x55f62da4a830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:02.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.925+0000 7fd08bc28740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55f62da50250 con 0x55f62da42dc0 2024-09-12T02:17:02.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.925+0000 7fd08bc28740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55f62d963050 con 0x55f62da4d680 2024-09-12T02:17:02.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.925+0000 7fd08bc28740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55f62da503d0 con 0x55f62da424a0 2024-09-12T02:17:02.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.925+0000 7fd08bc28740 10 client.0 _ll_register_callbacks cb 0x55f62d9a3160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:17:02.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.925+0000 7fd07bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f62da42dc0 0x55f62da4a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:02.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.925+0000 7fd07bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f62da42dc0 0x55f62da4a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46218/0 (socket says 172.21.15.16:46218) 2024-09-12T02:17:02.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.925+0000 7fd07bfff640 1 -- 172.21.15.16:0/3093633213 learned_addr learned my addr 172.21.15.16:0/3093633213 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:17:02.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.925+0000 7fd082ffd640 1 --2- 172.21.15.16:0/3093633213 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f62da424a0 0x55f62da4d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:02.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.925+0000 7fd0837fe640 1 --2- 172.21.15.16:0/3093633213 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f62da4d680 0x55f62da4a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:02.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.925+0000 7fd07bfff640 1 -- 172.21.15.16:0/3093633213 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f62da424a0 msgr2=0x55f62da4d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:02.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.925+0000 7fd07bfff640 1 --2- 172.21.15.16:0/3093633213 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f62da424a0 0x55f62da4d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:02.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.925+0000 7fd07bfff640 1 -- 172.21.15.16:0/3093633213 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f62da4d680 msgr2=0x55f62da4a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:02.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.925+0000 7fd07bfff640 1 --2- 172.21.15.16:0/3093633213 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f62da4d680 0x55f62da4a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:02.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.925+0000 7fd07bfff640 1 -- 172.21.15.16:0/3093633213 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f62d7193c0 con 0x55f62da42dc0 2024-09-12T02:17:02.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.926+0000 7fd07bfff640 1 --2- 172.21.15.16:0/3093633213 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f62da42dc0 0x55f62da4a2b0 secure :-1 s=READY pgs=2058 cs=0 l=1 rev1=1 crypto rx=0x7fd06c097180 tx=0x7fd06c09a860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=626d1cfa4961c71 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:02.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.926+0000 7fd062ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:17:02.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.926+0000 7fd062ffd640 1 -- 172.21.15.16:0/3093633213 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd06c01e070 con 0x55f62da42dc0 2024-09-12T02:17:02.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.926+0000 7fd062ffd640 1 -- 172.21.15.16:0/3093633213 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fd06c017360 con 0x55f62da42dc0 2024-09-12T02:17:02.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.926+0000 7fd08bc28740 1 -- 172.21.15.16:0/3093633213 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd074011ac0 con 0x55f62da42dc0 2024-09-12T02:17:02.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.926+0000 7fd062ffd640 1 -- 172.21.15.16:0/3093633213 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd06c09ad20 con 0x55f62da42dc0 2024-09-12T02:17:02.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.926+0000 7fd08bc28740 10 client.20616 fetch_fsmap learned FSMap version 25 2024-09-12T02:17:02.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.926+0000 7fd08bc28740 1 -- 172.21.15.16:0/3093633213 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f62da60050 con 0x55f62da42dc0 2024-09-12T02:17:02.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.926+0000 7fd062ffd640 1 -- 172.21.15.16:0/3093633213 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd074011ac0 con 0x55f62da42dc0 2024-09-12T02:17:02.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.926+0000 7fd062ffd640 1 -- 172.21.15.16:0/3093633213 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fd06c09a170 con 0x55f62da42dc0 2024-09-12T02:17:02.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.926+0000 7fd08bc28740 10 client.20616 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:17:02.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.926+0000 7fd08bc28740 10 client.20616 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:17:02.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.926+0000 7fd08bc28740 20 client.20616 populate_metadata read hostname 'smithi016' 2024-09-12T02:17:02.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.926+0000 7fd08bc28740 1 --2- 172.21.15.16:0/3093633213 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55f62da61510 0x55f62da63920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:02.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.927+0000 7fd08bc28740 4 client.20616 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:17:02.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.927+0000 7fd08bc28740 1 -- 172.21.15.16:0/3093633213 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f62d9b5850 con 0x55f62da61510 2024-09-12T02:17:02.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.927+0000 7fd082ffd640 1 --2- 172.21.15.16:0/3093633213 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55f62da61510 0x55f62da63920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:02.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.927+0000 7fd062ffd640 10 client.20616 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:17:02.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.927+0000 7fd082ffd640 1 --2- 172.21.15.16:0/3093633213 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55f62da61510 0x55f62da63920 crc :-1 s=READY pgs=455 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=200ab84cbf690fc7 server_cookie=6aa2e755af7954d7 in_seq=0 out_seq=0 2024-09-12T02:17:02.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.928+0000 7fd062ffd640 1 -- 172.21.15.16:0/3093633213 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f62d9b5850 con 0x55f62da61510 2024-09-12T02:17:02.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.928+0000 7fd062ffd640 10 client.20616 handle_command_reply: tid=0 2024-09-12T02:17:02.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.928+0000 7fd08bc28740 1 client.20616 shutdown 2024-09-12T02:17:02.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.928+0000 7fd08bc28740 1 -- 172.21.15.16:0/3093633213 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f62da42dc0 msgr2=0x55f62da4a2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:02.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.928+0000 7fd08bc28740 1 --2- 172.21.15.16:0/3093633213 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f62da42dc0 0x55f62da4a2b0 secure :-1 s=READY pgs=2058 cs=0 l=1 rev1=1 crypto rx=0x7fd06c097180 tx=0x7fd06c09a860 comp rx=0 tx=0).stop 2024-09-12T02:17:02.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.928+0000 7fd08bc28740 1 -- 172.21.15.16:0/3093633213 shutdown_connections 2024-09-12T02:17:02.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.928+0000 7fd08bc28740 1 --2- 172.21.15.16:0/3093633213 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55f62da61510 0x55f62da63920 crc :-1 s=READY pgs=455 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:02.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.928+0000 7fd08bc28740 1 --2- 172.21.15.16:0/3093633213 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f62da4d680 0x55f62da4a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:02.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.928+0000 7fd08bc28740 1 --2- 172.21.15.16:0/3093633213 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f62da42dc0 0x55f62da4a2b0 unknown :-1 s=CLOSED pgs=2058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:02.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.928+0000 7fd062ffd640 0 client.20616 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:17:02.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.928+0000 7fd08bc28740 1 --2- 172.21.15.16:0/3093633213 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f62da424a0 0x55f62da4d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:02.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.928+0000 7fd08bc28740 1 -- 172.21.15.16:0/3093633213 >> 172.21.15.16:0/3093633213 conn(0x55f62da3ea30 msgr2=0x55f62da44760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:02.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.928+0000 7fd08bc28740 1 -- 172.21.15.16:0/3093633213 shutdown_connections 2024-09-12T02:17:02.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.928+0000 7fd08bc28740 1 -- 172.21.15.16:0/3093633213 wait complete. 2024-09-12T02:17:02.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.928+0000 7fd08bc28740 20 client.20616 trim_cache size 0 max 16384 2024-09-12T02:17:02.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.946+0000 7fd08bc28740 1 Processor -- start 2024-09-12T02:17:02.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.946+0000 7fd08bc28740 1 -- start start 2024-09-12T02:17:02.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.946+0000 7fd08bc28740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f62da424a0 0x55f62da48b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:02.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.946+0000 7fd08bc28740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f62da42dc0 0x55f62da691f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:02.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.946+0000 7fd08bc28740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f62da4d680 0x55f62da69730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:02.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.946+0000 7fd08bc28740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd0841adb90 con 0x55f62da4d680 2024-09-12T02:17:02.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.946+0000 7fd08bc28740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd0841ab460 con 0x55f62da424a0 2024-09-12T02:17:02.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.946+0000 7fd08bc28740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd0841a92d0 con 0x55f62da42dc0 2024-09-12T02:17:02.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.946+0000 7fd0837fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f62da4d680 0x55f62da69730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:02.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.946+0000 7fd0837fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f62da4d680 0x55f62da69730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46224/0 (socket says 172.21.15.16:46224) 2024-09-12T02:17:02.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.946+0000 7fd0837fe640 1 -- 172.21.15.16:0/2327112814 learned_addr learned my addr 172.21.15.16:0/2327112814 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:17:02.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.946+0000 7fd0837fe640 1 -- 172.21.15.16:0/2327112814 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f62da42dc0 msgr2=0x55f62da691f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:02.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.946+0000 7fd082ffd640 1 --2- 172.21.15.16:0/2327112814 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f62da424a0 0x55f62da48b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:02.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.946+0000 7fd0837fe640 1 --2- 172.21.15.16:0/2327112814 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f62da42dc0 0x55f62da691f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:02.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.946+0000 7fd0837fe640 1 -- 172.21.15.16:0/2327112814 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f62da424a0 msgr2=0x55f62da48b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:02.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.946+0000 7fd0837fe640 1 --2- 172.21.15.16:0/2327112814 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f62da424a0 0x55f62da48b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:02.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.946+0000 7fd0837fe640 1 -- 172.21.15.16:0/2327112814 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f62da49260 con 0x55f62da4d680 2024-09-12T02:17:02.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.947+0000 7fd0837fe640 1 --2- 172.21.15.16:0/2327112814 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f62da4d680 0x55f62da69730 secure :-1 s=READY pgs=2059 cs=0 l=1 rev1=1 crypto rx=0x7fd07c009cd0 tx=0x7fd07c03f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2044882fe58050a0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:02.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.947+0000 7fd0797fa640 1 -- 172.21.15.16:0/2327112814 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd07c003c30 con 0x55f62da4d680 2024-09-12T02:17:02.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.947+0000 7fd0797fa640 1 -- 172.21.15.16:0/2327112814 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd07c036040 con 0x55f62da4d680 2024-09-12T02:17:02.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.947+0000 7fd0797fa640 1 -- 172.21.15.16:0/2327112814 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd07c030a00 con 0x55f62da4d680 2024-09-12T02:17:02.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.947+0000 7fd08bc28740 1 -- 172.21.15.16:0/2327112814 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f62da4d680 msgr2=0x55f62da69730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:02.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.947+0000 7fd08bc28740 1 --2- 172.21.15.16:0/2327112814 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f62da4d680 0x55f62da69730 secure :-1 s=READY pgs=2059 cs=0 l=1 rev1=1 crypto rx=0x7fd07c009cd0 tx=0x7fd07c03f000 comp rx=0 tx=0).stop 2024-09-12T02:17:02.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.947+0000 7fd08bc28740 1 -- 172.21.15.16:0/2327112814 shutdown_connections 2024-09-12T02:17:02.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.947+0000 7fd08bc28740 1 --2- 172.21.15.16:0/2327112814 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f62da4d680 0x55f62da69730 unknown :-1 s=CLOSED pgs=2059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:02.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.947+0000 7fd08bc28740 1 --2- 172.21.15.16:0/2327112814 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f62da42dc0 0x55f62da691f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:02.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.947+0000 7fd08bc28740 1 --2- 172.21.15.16:0/2327112814 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f62da424a0 0x55f62da48b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:02.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.947+0000 7fd08bc28740 1 -- 172.21.15.16:0/2327112814 >> 172.21.15.16:0/2327112814 conn(0x55f62da3ea30 msgr2=0x55f62da4c330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:02.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.947+0000 7fd08bc28740 1 -- 172.21.15.16:0/2327112814 shutdown_connections 2024-09-12T02:17:02.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.947+0000 7fd08bc28740 1 -- 172.21.15.16:0/2327112814 wait complete. 2024-09-12T02:17:02.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.948+0000 7fd08bc28740 1 -- start start 2024-09-12T02:17:02.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.948+0000 7fd08bc28740 1 Processor -- start 2024-09-12T02:17:02.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.948+0000 7fd08bc28740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f62da424a0 0x55f62da53300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:02.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.948+0000 7fd08bc28740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f62da42dc0 0x55f62da53840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:02.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.948+0000 7fd08bc28740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f62da4d680 0x55f62da53d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:02.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.948+0000 7fd08bc28740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd0841a9920 con 0x55f62da42dc0 2024-09-12T02:17:02.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.948+0000 7fd08bc28740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd0841aad90 con 0x55f62da424a0 2024-09-12T02:17:02.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.948+0000 7fd08bc28740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd0841a9ff0 con 0x55f62da4d680 2024-09-12T02:17:02.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.948+0000 7fd08bc28740 10 client.0 _ll_register_callbacks cb 0x7fd06c0d1a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:17:02.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.948+0000 7fd07bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f62da42dc0 0x55f62da53840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:02.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.948+0000 7fd07bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f62da42dc0 0x55f62da53840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46226/0 (socket says 172.21.15.16:46226) 2024-09-12T02:17:02.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.948+0000 7fd07bfff640 1 -- 172.21.15.16:0/3722806405 learned_addr learned my addr 172.21.15.16:0/3722806405 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:17:02.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.949+0000 7fd082ffd640 1 --2- 172.21.15.16:0/3722806405 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f62da424a0 0x55f62da53300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:02.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.949+0000 7fd07bfff640 1 -- 172.21.15.16:0/3722806405 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f62da4d680 msgr2=0x55f62da53d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:02.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.949+0000 7fd07bfff640 1 --2- 172.21.15.16:0/3722806405 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f62da4d680 0x55f62da53d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:02.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.949+0000 7fd07bfff640 1 -- 172.21.15.16:0/3722806405 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f62da424a0 msgr2=0x55f62da53300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:02.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.949+0000 7fd07bfff640 1 --2- 172.21.15.16:0/3722806405 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f62da424a0 0x55f62da53300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:02.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.949+0000 7fd07bfff640 1 -- 172.21.15.16:0/3722806405 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f62dacb1d0 con 0x55f62da42dc0 2024-09-12T02:17:02.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.949+0000 7fd082ffd640 1 --2- 172.21.15.16:0/3722806405 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f62da424a0 0x55f62da53300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:17:02.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.949+0000 7fd07bfff640 1 --2- 172.21.15.16:0/3722806405 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f62da42dc0 0x55f62da53840 secure :-1 s=READY pgs=2060 cs=0 l=1 rev1=1 crypto rx=0x7fd06c09acb0 tx=0x7fd06c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb21933f66116fb5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:02.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.949+0000 7fd062ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:17:02.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.949+0000 7fd062ffd640 1 -- 172.21.15.16:0/3722806405 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd06c0b8060 con 0x55f62da42dc0 2024-09-12T02:17:02.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.949+0000 7fd062ffd640 1 -- 172.21.15.16:0/3722806405 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fd06c096c10 con 0x55f62da42dc0 2024-09-12T02:17:02.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.949+0000 7fd08bc28740 1 -- 172.21.15.16:0/3722806405 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd07c036040 con 0x55f62da42dc0 2024-09-12T02:17:02.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.950+0000 7fd062ffd640 1 -- 172.21.15.16:0/3722806405 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd06c01e070 con 0x55f62da42dc0 2024-09-12T02:17:02.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.950+0000 7fd08bc28740 10 client.20622 fetch_fsmap learned FSMap version 25 2024-09-12T02:17:02.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.950+0000 7fd08bc28740 1 -- 172.21.15.16:0/3722806405 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f62dacc420 con 0x55f62da42dc0 2024-09-12T02:17:02.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.950+0000 7fd062ffd640 1 -- 172.21.15.16:0/3722806405 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd07c036040 con 0x55f62da42dc0 2024-09-12T02:17:02.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.950+0000 7fd062ffd640 1 -- 172.21.15.16:0/3722806405 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fd06c099be0 con 0x55f62da42dc0 2024-09-12T02:17:02.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.950+0000 7fd08bc28740 10 client.20622 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:17:02.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.950+0000 7fd08bc28740 10 client.20622 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:17:02.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.950+0000 7fd08bc28740 20 client.20622 populate_metadata read hostname 'smithi016' 2024-09-12T02:17:02.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.950+0000 7fd08bc28740 1 --2- 172.21.15.16:0/3722806405 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55f62da61510 0x55f62daceb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:02.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.950+0000 7fd08bc28740 4 client.20622 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T02:17:02.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.950+0000 7fd08bc28740 1 -- 172.21.15.16:0/3722806405 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f62d9afd60 con 0x55f62da61510 2024-09-12T02:17:02.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.950+0000 7fd082ffd640 1 --2- 172.21.15.16:0/3722806405 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55f62da61510 0x55f62daceb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:02.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.951+0000 7fd082ffd640 1 --2- 172.21.15.16:0/3722806405 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55f62da61510 0x55f62daceb50 crc :-1 s=READY pgs=456 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=486d083cbabd6ce server_cookie=7ac158d16c58a738 in_seq=0 out_seq=0 2024-09-12T02:17:02.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.951+0000 7fd062ffd640 10 client.20622 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:17:02.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.951+0000 7fd062ffd640 1 -- 172.21.15.16:0/3722806405 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55f62d9afd60 con 0x55f62da61510 2024-09-12T02:17:02.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.951+0000 7fd062ffd640 10 client.20622 handle_command_reply: tid=0 2024-09-12T02:17:02.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.951+0000 7fd08bc28740 1 client.20622 shutdown 2024-09-12T02:17:02.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.951+0000 7fd08bc28740 1 -- 172.21.15.16:0/3722806405 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f62da42dc0 msgr2=0x55f62da53840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:02.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.951+0000 7fd08bc28740 1 --2- 172.21.15.16:0/3722806405 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f62da42dc0 0x55f62da53840 secure :-1 s=READY pgs=2060 cs=0 l=1 rev1=1 crypto rx=0x7fd06c09acb0 tx=0x7fd06c0046c0 comp rx=0 tx=0).stop 2024-09-12T02:17:02.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.951+0000 7fd08bc28740 1 -- 172.21.15.16:0/3722806405 shutdown_connections 2024-09-12T02:17:02.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.951+0000 7fd08bc28740 1 --2- 172.21.15.16:0/3722806405 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55f62da61510 0x55f62daceb50 crc :-1 s=READY pgs=456 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:02.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.951+0000 7fd08bc28740 1 --2- 172.21.15.16:0/3722806405 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f62da4d680 0x55f62da53d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:02.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.951+0000 7fd08bc28740 1 --2- 172.21.15.16:0/3722806405 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f62da42dc0 0x55f62da53840 unknown :-1 s=CLOSED pgs=2060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:02.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.951+0000 7fd08bc28740 1 --2- 172.21.15.16:0/3722806405 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f62da424a0 0x55f62da53300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:02.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.951+0000 7fd08bc28740 1 -- 172.21.15.16:0/3722806405 >> 172.21.15.16:0/3722806405 conn(0x55f62da3ea30 msgr2=0x55f62da4c330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:02.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.951+0000 7fd062ffd640 0 client.20622 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:17:02.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.952+0000 7fd08bc28740 1 -- 172.21.15.16:0/3722806405 shutdown_connections 2024-09-12T02:17:02.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.952+0000 7fd08bc28740 1 -- 172.21.15.16:0/3722806405 wait complete. 2024-09-12T02:17:02.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.952+0000 7fd08bc28740 20 client.20622 trim_cache size 0 max 16384 2024-09-12T02:17:02.957 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T02:17:02.957 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-12T02:17:02.957 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": { 2024-09-12T02:17:02.957 INFO:teuthology.orchestra.run.smithi016.stdout: "2a2b33f6-80fc-435b-8450-0975f2267ee0": { 2024-09-12T02:17:02.957 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-12T02:17:02.957 INFO:teuthology.orchestra.run.smithi016.stdout: "tag": "2a2b33f6-80fc-435b-8450-0975f2267ee0", 2024-09-12T02:17:02.957 INFO:teuthology.orchestra.run.smithi016.stdout: "options": "recursive,force" 2024-09-12T02:17:02.957 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:02.957 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:02.957 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T02:17:02.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.952+0000 7fd08984f640 1 -- 172.21.15.16:0/2304355827 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fd0700a2bc0 msgr2=0x7fd0700a5080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:02.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.952+0000 7fd08984f640 1 --2- 172.21.15.16:0/2304355827 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fd0700a2bc0 0x7fd0700a5080 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fd07c02aed0 tx=0x7fd07c009800 comp rx=0 tx=0).stop 2024-09-12T02:17:02.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.952+0000 7fd08984f640 1 -- 172.21.15.16:0/2304355827 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd084101420 msgr2=0x7fd0841989c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:02.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.952+0000 7fd08984f640 1 --2- 172.21.15.16:0/2304355827 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd084101420 0x7fd0841989c0 secure :-1 s=READY pgs=2056 cs=0 l=1 rev1=1 crypto rx=0x7fd06c00b4a0 tx=0x7fd06c00b960 comp rx=0 tx=0).stop 2024-09-12T02:17:02.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.952+0000 7fd08984f640 1 -- 172.21.15.16:0/2304355827 shutdown_connections 2024-09-12T02:17:02.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.952+0000 7fd08984f640 1 --2- 172.21.15.16:0/2304355827 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fd0700a2bc0 0x7fd0700a5080 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:02.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.952+0000 7fd08984f640 1 --2- 172.21.15.16:0/2304355827 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd084106a80 0x7fd08419cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:02.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.952+0000 7fd08984f640 1 --2- 172.21.15.16:0/2304355827 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd084101420 0x7fd0841989c0 unknown :-1 s=CLOSED pgs=2056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:02.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.952+0000 7fd08984f640 1 --2- 172.21.15.16:0/2304355827 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd084100a80 0x7fd084198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:02.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.952+0000 7fd08984f640 1 -- 172.21.15.16:0/2304355827 >> 172.21.15.16:0/2304355827 conn(0x7fd0840fc7a0 msgr2=0x7fd084104830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:02.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.952+0000 7fd08984f640 1 -- 172.21.15.16:0/2304355827 shutdown_connections 2024-09-12T02:17:02.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:02.952+0000 7fd08984f640 1 -- 172.21.15.16:0/2304355827 wait complete. 2024-09-12T02:17:03.619 INFO:tasks.cephfs.filesystem:scrub status for tag:2a2b33f6-80fc-435b-8450-0975f2267ee0 - {'path': '/', 'tag': '2a2b33f6-80fc-435b-8450-0975f2267ee0', 'options': 'recursive,force'} 2024-09-12T02:17:04.482 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.482+0000 7ff82287b640 1 -- 172.21.15.16:0/2592368570 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff81c104ef0 msgr2=0x7ff81c1072e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:04.482 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.482+0000 7ff82287b640 1 --2- 172.21.15.16:0/2592368570 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff81c104ef0 0x7ff81c1072e0 secure :-1 s=READY pgs=2061 cs=0 l=1 rev1=1 crypto rx=0x7ff810009900 tx=0x7ff81002a410 comp rx=0 tx=0).stop 2024-09-12T02:17:04.483 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.482+0000 7ff82287b640 1 -- 172.21.15.16:0/2592368570 shutdown_connections 2024-09-12T02:17:04.483 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.482+0000 7ff82287b640 1 --2- 172.21.15.16:0/2592368570 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff81c107820 0x7ff81c109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:04.483 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.482+0000 7ff82287b640 1 --2- 172.21.15.16:0/2592368570 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff81c104ef0 0x7ff81c1072e0 unknown :-1 s=CLOSED pgs=2061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:04.483 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.482+0000 7ff82287b640 1 --2- 172.21.15.16:0/2592368570 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff81c1025c0 0x7ff81c1049b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:04.483 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.482+0000 7ff82287b640 1 -- 172.21.15.16:0/2592368570 >> 172.21.15.16:0/2592368570 conn(0x7ff81c0fc1e0 msgr2=0x7ff81c0fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:04.483 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.482+0000 7ff82287b640 1 -- 172.21.15.16:0/2592368570 shutdown_connections 2024-09-12T02:17:04.483 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.482+0000 7ff82287b640 1 -- 172.21.15.16:0/2592368570 wait complete. 2024-09-12T02:17:04.486 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.483+0000 7ff82287b640 1 Processor -- start 2024-09-12T02:17:04.486 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.483+0000 7ff82287b640 1 -- start start 2024-09-12T02:17:04.486 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.483+0000 7ff82287b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff81c1025c0 0x7ff81c197e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:04.486 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.484+0000 7ff82287b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff81c104ef0 0x7ff81c198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:04.486 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.484+0000 7ff82287b640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff81c107820 0x7ff81c19f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:04.486 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.484+0000 7ff82287b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff81c10d820 con 0x7ff81c1025c0 2024-09-12T02:17:04.486 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.484+0000 7ff82287b640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff81c10d6a0 con 0x7ff81c104ef0 2024-09-12T02:17:04.486 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.484+0000 7ff82287b640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff81c10d9a0 con 0x7ff81c107820 2024-09-12T02:17:04.486 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.484+0000 7ff81bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff81c1025c0 0x7ff81c197e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:04.486 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.484+0000 7ff81bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff81c1025c0 0x7ff81c197e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46242/0 (socket says 172.21.15.16:46242) 2024-09-12T02:17:04.487 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.484+0000 7ff81bfff640 1 -- 172.21.15.16:0/2778608251 learned_addr learned my addr 172.21.15.16:0/2778608251 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:17:04.487 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.484+0000 7ff81b7fe640 1 --2- 172.21.15.16:0/2778608251 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff81c104ef0 0x7ff81c198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:04.487 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.484+0000 7ff820e18640 1 --2- 172.21.15.16:0/2778608251 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff81c107820 0x7ff81c19f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:04.487 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.484+0000 7ff81bfff640 1 -- 172.21.15.16:0/2778608251 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff81c107820 msgr2=0x7ff81c19f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:04.487 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.484+0000 7ff81bfff640 1 --2- 172.21.15.16:0/2778608251 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff81c107820 0x7ff81c19f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:04.487 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.484+0000 7ff81bfff640 1 -- 172.21.15.16:0/2778608251 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff81c104ef0 msgr2=0x7ff81c198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:04.487 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.484+0000 7ff81bfff640 1 --2- 172.21.15.16:0/2778608251 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff81c104ef0 0x7ff81c198380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:04.487 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.484+0000 7ff81bfff640 1 -- 172.21.15.16:0/2778608251 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff81c19fa60 con 0x7ff81c1025c0 2024-09-12T02:17:04.487 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.484+0000 7ff820e18640 1 --2- 172.21.15.16:0/2778608251 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff81c107820 0x7ff81c19f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:17:04.487 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.484+0000 7ff81bfff640 1 --2- 172.21.15.16:0/2778608251 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff81c1025c0 0x7ff81c197e40 secure :-1 s=READY pgs=2062 cs=0 l=1 rev1=1 crypto rx=0x7ff80c00d950 tx=0x7ff80c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:04.487 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.484+0000 7ff8197fa640 1 -- 172.21.15.16:0/2778608251 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff80c017030 con 0x7ff81c1025c0 2024-09-12T02:17:04.487 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.485+0000 7ff82287b640 1 -- 172.21.15.16:0/2778608251 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff81c19fd50 con 0x7ff81c1025c0 2024-09-12T02:17:04.488 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.485+0000 7ff8197fa640 1 -- 172.21.15.16:0/2778608251 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff80c004020 con 0x7ff81c1025c0 2024-09-12T02:17:04.488 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.485+0000 7ff8197fa640 1 -- 172.21.15.16:0/2778608251 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff80c0047a0 con 0x7ff81c1025c0 2024-09-12T02:17:04.488 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.485+0000 7ff82287b640 1 -- 172.21.15.16:0/2778608251 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff81c1a01c0 con 0x7ff81c1025c0 2024-09-12T02:17:04.495 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.488+0000 7ff82561a740 1 Processor -- start 2024-09-12T02:17:04.495 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.488+0000 7ff82561a740 1 -- start start 2024-09-12T02:17:04.495 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.488+0000 7ff82561a740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560b879bb9a0 0x560b879bbda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:04.496 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.488+0000 7ff82561a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b879bc2e0 0x560b879c0da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:04.496 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.488+0000 7ff82561a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b879c1370 0x560b879c3830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:04.496 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.488+0000 7ff82561a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x560b8791f2f0 con 0x560b879bc2e0 2024-09-12T02:17:04.496 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.488+0000 7ff82561a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x560b878c7270 con 0x560b879c1370 2024-09-12T02:17:04.496 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.488+0000 7ff82561a740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x560b8791b2b0 con 0x560b879bb9a0 2024-09-12T02:17:04.496 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.488+0000 7ff8197fa640 1 -- 172.21.15.16:0/2778608251 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff80c0041d0 con 0x7ff81c1025c0 2024-09-12T02:17:04.496 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.488+0000 7ff81b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b879c1370 0x560b879c3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:04.496 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.488+0000 7ff81b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b879c1370 0x560b879c3830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.16:43328/0 (socket says 172.21.15.16:43328) 2024-09-12T02:17:04.496 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.488+0000 7ff820e18640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560b879bb9a0 0x560b879bbda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:04.496 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.488+0000 7ff81b7fe640 1 -- 172.21.15.16:0/1275762180 learned_addr learned my addr 172.21.15.16:0/1275762180 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:17:04.496 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.488+0000 7ff81bfff640 1 --2- 172.21.15.16:0/1275762180 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b879bc2e0 0x560b879c0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:04.497 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.489+0000 7ff81b7fe640 1 -- 172.21.15.16:0/1275762180 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560b879bb9a0 msgr2=0x560b879bbda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:04.497 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.489+0000 7ff81b7fe640 1 --2- 172.21.15.16:0/1275762180 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560b879bb9a0 0x560b879bbda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:04.497 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.489+0000 7ff81b7fe640 1 -- 172.21.15.16:0/1275762180 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b879bc2e0 msgr2=0x560b879c0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:04.497 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.489+0000 7ff81b7fe640 1 --2- 172.21.15.16:0/1275762180 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b879bc2e0 0x560b879c0da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:04.497 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.489+0000 7ff81b7fe640 1 -- 172.21.15.16:0/1275762180 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b878c6340 con 0x560b879c1370 2024-09-12T02:17:04.497 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.489+0000 7ff820e18640 1 --2- 172.21.15.16:0/1275762180 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560b879bb9a0 0x560b879bbda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:17:04.497 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.489+0000 7ff81b7fe640 1 --2- 172.21.15.16:0/1275762180 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b879c1370 0x560b879c3830 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7ff81c06a400 tx=0x7ff810004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aac9f45d366ef642 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:04.497 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.489+0000 7ff8027fc640 1 -- 172.21.15.16:0/1275762180 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff810035030 con 0x560b879c1370 2024-09-12T02:17:04.497 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.489+0000 7ff8027fc640 1 -- 172.21.15.16:0/1275762180 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff810004430 con 0x560b879c1370 2024-09-12T02:17:04.497 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.489+0000 7ff8027fc640 1 -- 172.21.15.16:0/1275762180 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8100316b0 con 0x560b879c1370 2024-09-12T02:17:04.497 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.490+0000 7ff8197fa640 1 --2- 172.21.15.16:0/2778608251 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7ff8040a2c30 0x7ff8040a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:04.497 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.490+0000 7ff820e18640 1 --2- 172.21.15.16:0/2778608251 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7ff8040a2c30 0x7ff8040a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:04.497 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.490+0000 7ff8197fa640 1 -- 172.21.15.16:0/2778608251 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7ff80c0ceac0 con 0x7ff81c1025c0 2024-09-12T02:17:04.498 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.490+0000 7ff82561a740 1 -- 172.21.15.16:0/1275762180 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b879c1370 msgr2=0x560b879c3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:04.498 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.490+0000 7ff82561a740 1 --2- 172.21.15.16:0/1275762180 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b879c1370 0x560b879c3830 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7ff81c06a400 tx=0x7ff810004040 comp rx=0 tx=0).stop 2024-09-12T02:17:04.498 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.490+0000 7ff82561a740 1 -- 172.21.15.16:0/1275762180 shutdown_connections 2024-09-12T02:17:04.498 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.490+0000 7ff82561a740 1 --2- 172.21.15.16:0/1275762180 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b879c1370 0x560b879c3830 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:04.498 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.490+0000 7ff82561a740 1 --2- 172.21.15.16:0/1275762180 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b879bc2e0 0x560b879c0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:04.498 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.490+0000 7ff82561a740 1 --2- 172.21.15.16:0/1275762180 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560b879bb9a0 0x560b879bbda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:04.498 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.490+0000 7ff82561a740 1 -- 172.21.15.16:0/1275762180 >> 172.21.15.16:0/1275762180 conn(0x560b879b7010 msgr2=0x560b879b7410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:04.498 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.490+0000 7ff82561a740 1 -- 172.21.15.16:0/1275762180 shutdown_connections 2024-09-12T02:17:04.498 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.490+0000 7ff820e18640 1 --2- 172.21.15.16:0/2778608251 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7ff8040a2c30 0x7ff8040a50f0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7ff81c06a980 tx=0x7ff814009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:04.498 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.490+0000 7ff82561a740 1 -- 172.21.15.16:0/1275762180 wait complete. 2024-09-12T02:17:04.499 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.491+0000 7ff82561a740 1 -- start start 2024-09-12T02:17:04.499 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.491+0000 7ff82561a740 1 Processor -- start 2024-09-12T02:17:04.499 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.491+0000 7ff82561a740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560b879bb9a0 0x560b879c5b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:04.499 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.491+0000 7ff82561a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b879bc2e0 0x560b879c6080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:04.499 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.491+0000 7ff82561a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b879c1370 0x560b879cd0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:04.499 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.491+0000 7ff82561a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x560b879c42e0 con 0x560b879bc2e0 2024-09-12T02:17:04.499 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.491+0000 7ff82561a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x560b878a8290 con 0x560b879c1370 2024-09-12T02:17:04.499 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.491+0000 7ff82561a740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x560b879c4460 con 0x560b879bb9a0 2024-09-12T02:17:04.499 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.491+0000 7ff82561a740 10 client.0 _ll_register_callbacks cb 0x560b878d9140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:17:04.499 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.491+0000 7ff81b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b879bc2e0 0x560b879c6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:04.499 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.491+0000 7ff81b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b879bc2e0 0x560b879c6080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46252/0 (socket says 172.21.15.16:46252) 2024-09-12T02:17:04.499 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.491+0000 7ff81b7fe640 1 -- 172.21.15.16:0/3284589224 learned_addr learned my addr 172.21.15.16:0/3284589224 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:17:04.500 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.491+0000 7ff81bfff640 1 --2- 172.21.15.16:0/3284589224 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560b879bb9a0 0x560b879c5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:04.500 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.492+0000 7ff820e18640 1 --2- 172.21.15.16:0/3284589224 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b879c1370 0x560b879cd0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:04.500 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.492+0000 7ff81b7fe640 1 -- 172.21.15.16:0/3284589224 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560b879bb9a0 msgr2=0x560b879c5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:04.500 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.492+0000 7ff81b7fe640 1 --2- 172.21.15.16:0/3284589224 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560b879bb9a0 0x560b879c5b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:04.500 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.492+0000 7ff81b7fe640 1 -- 172.21.15.16:0/3284589224 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b879c1370 msgr2=0x560b879cd0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:04.500 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.492+0000 7ff81b7fe640 1 --2- 172.21.15.16:0/3284589224 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b879c1370 0x560b879cd0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:04.500 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.492+0000 7ff81b7fe640 1 -- 172.21.15.16:0/3284589224 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b87632210 con 0x560b879bc2e0 2024-09-12T02:17:04.500 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.492+0000 7ff81b7fe640 1 --2- 172.21.15.16:0/3284589224 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b879bc2e0 0x560b879c6080 secure :-1 s=READY pgs=2063 cs=0 l=1 rev1=1 crypto rx=0x7ff81002a930 tx=0x7ff810031960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b569ede1a55d374 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:04.500 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.492+0000 7ff7fb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:17:04.500 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.492+0000 7ff7fb7fe640 1 -- 172.21.15.16:0/3284589224 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff810051060 con 0x560b879bc2e0 2024-09-12T02:17:04.500 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.492+0000 7ff7fb7fe640 1 -- 172.21.15.16:0/3284589224 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7ff810031d60 con 0x560b879bc2e0 2024-09-12T02:17:04.500 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.492+0000 7ff82561a740 1 -- 172.21.15.16:0/3284589224 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff810004430 con 0x560b879bc2e0 2024-09-12T02:17:04.500 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.492+0000 7ff7fb7fe640 1 -- 172.21.15.16:0/3284589224 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff810035030 con 0x560b879bc2e0 2024-09-12T02:17:04.501 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.492+0000 7ff82561a740 10 client.20634 fetch_fsmap learned FSMap version 25 2024-09-12T02:17:04.501 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.492+0000 7ff82561a740 1 -- 172.21.15.16:0/3284589224 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x560b876fe620 con 0x560b879bc2e0 2024-09-12T02:17:04.501 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.493+0000 7ff7fb7fe640 1 -- 172.21.15.16:0/3284589224 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff810004430 con 0x560b879bc2e0 2024-09-12T02:17:04.501 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.493+0000 7ff7fb7fe640 1 -- 172.21.15.16:0/3284589224 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7ff810056070 con 0x560b879bc2e0 2024-09-12T02:17:04.501 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.493+0000 7ff82561a740 10 client.20634 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:17:04.501 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.493+0000 7ff82561a740 10 client.20634 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:17:04.501 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.493+0000 7ff82561a740 20 client.20634 populate_metadata read hostname 'smithi016' 2024-09-12T02:17:04.501 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.493+0000 7ff82561a740 1 --2- 172.21.15.16:0/3284589224 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x560b879d13c0 0x560b879d37b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:04.501 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.493+0000 7ff82561a740 4 client.20634 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:17:04.501 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.493+0000 7ff82561a740 1 -- 172.21.15.16:0/3284589224 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560b8792ca20 con 0x560b879d13c0 2024-09-12T02:17:04.501 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.493+0000 7ff81bfff640 1 --2- 172.21.15.16:0/3284589224 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x560b879d13c0 0x560b879d37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:04.501 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.494+0000 7ff81bfff640 1 --2- 172.21.15.16:0/3284589224 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x560b879d13c0 0x560b879d37b0 crc :-1 s=READY pgs=457 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d0e95771cb41cf9 server_cookie=d1dfe47dbca7ee31 in_seq=0 out_seq=0 2024-09-12T02:17:04.501 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.494+0000 7ff7fb7fe640 10 client.20634 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:17:04.502 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.494+0000 7ff7fb7fe640 1 -- 172.21.15.16:0/3284589224 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560b8792ca20 con 0x560b879d13c0 2024-09-12T02:17:04.502 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.494+0000 7ff7fb7fe640 10 client.20634 handle_command_reply: tid=0 2024-09-12T02:17:04.502 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.494+0000 7ff82561a740 1 client.20634 shutdown 2024-09-12T02:17:04.502 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.495+0000 7ff82561a740 1 -- 172.21.15.16:0/3284589224 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b879bc2e0 msgr2=0x560b879c6080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:04.502 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.495+0000 7ff82561a740 1 --2- 172.21.15.16:0/3284589224 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b879bc2e0 0x560b879c6080 secure :-1 s=READY pgs=2063 cs=0 l=1 rev1=1 crypto rx=0x7ff81002a930 tx=0x7ff810031960 comp rx=0 tx=0).stop 2024-09-12T02:17:04.502 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.495+0000 7ff82561a740 1 -- 172.21.15.16:0/3284589224 shutdown_connections 2024-09-12T02:17:04.502 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.495+0000 7ff82561a740 1 --2- 172.21.15.16:0/3284589224 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x560b879d13c0 0x560b879d37b0 crc :-1 s=READY pgs=457 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:04.502 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.495+0000 7ff82561a740 1 --2- 172.21.15.16:0/3284589224 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b879c1370 0x560b879cd0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:04.502 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.495+0000 7ff82561a740 1 --2- 172.21.15.16:0/3284589224 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b879bc2e0 0x560b879c6080 unknown :-1 s=CLOSED pgs=2063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:04.502 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.495+0000 7ff7fb7fe640 0 client.20634 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:17:04.502 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.495+0000 7ff82561a740 1 --2- 172.21.15.16:0/3284589224 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560b879bb9a0 0x560b879c5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:04.502 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.495+0000 7ff82561a740 1 -- 172.21.15.16:0/3284589224 >> 172.21.15.16:0/3284589224 conn(0x560b879b7010 msgr2=0x560b879b8820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:04.502 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.495+0000 7ff82561a740 1 -- 172.21.15.16:0/3284589224 shutdown_connections 2024-09-12T02:17:04.503 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.495+0000 7ff82561a740 1 -- 172.21.15.16:0/3284589224 wait complete. 2024-09-12T02:17:04.503 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.495+0000 7ff82561a740 20 client.20634 trim_cache size 0 max 16384 2024-09-12T02:17:04.512 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.509+0000 7ff82561a740 1 Processor -- start 2024-09-12T02:17:04.512 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.510+0000 7ff82561a740 1 -- start start 2024-09-12T02:17:04.512 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.510+0000 7ff82561a740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560b879bb9a0 0x560b879b60b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:04.512 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.510+0000 7ff82561a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b879bc2e0 0x560b879cd080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:04.512 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.510+0000 7ff82561a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b879c1370 0x560b879b8400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:04.512 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.510+0000 7ff82561a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff81c1ab6e0 con 0x560b879c1370 2024-09-12T02:17:04.512 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.510+0000 7ff82561a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff81c1a8fb0 con 0x560b879bc2e0 2024-09-12T02:17:04.512 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.510+0000 7ff82561a740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff81c1a6e70 con 0x560b879bb9a0 2024-09-12T02:17:04.513 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.510+0000 7ff820e18640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b879c1370 0x560b879b8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:04.513 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.510+0000 7ff820e18640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b879c1370 0x560b879b8400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46266/0 (socket says 172.21.15.16:46266) 2024-09-12T02:17:04.513 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.510+0000 7ff820e18640 1 -- 172.21.15.16:0/94105013 learned_addr learned my addr 172.21.15.16:0/94105013 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:17:04.513 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.510+0000 7ff81b7fe640 1 --2- 172.21.15.16:0/94105013 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b879bc2e0 0x560b879cd080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:04.513 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.510+0000 7ff81bfff640 1 --2- 172.21.15.16:0/94105013 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560b879bb9a0 0x560b879b60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:04.513 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.510+0000 7ff820e18640 1 -- 172.21.15.16:0/94105013 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560b879bb9a0 msgr2=0x560b879b60b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:04.513 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.510+0000 7ff820e18640 1 --2- 172.21.15.16:0/94105013 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560b879bb9a0 0x560b879b60b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:04.514 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.510+0000 7ff820e18640 1 -- 172.21.15.16:0/94105013 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b879bc2e0 msgr2=0x560b879cd080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:04.514 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.510+0000 7ff820e18640 1 --2- 172.21.15.16:0/94105013 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b879bc2e0 0x560b879cd080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:04.514 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.510+0000 7ff820e18640 1 -- 172.21.15.16:0/94105013 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b879bcf70 con 0x560b879c1370 2024-09-12T02:17:04.514 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.510+0000 7ff820e18640 1 --2- 172.21.15.16:0/94105013 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b879c1370 0x560b879b8400 secure :-1 s=READY pgs=2064 cs=0 l=1 rev1=1 crypto rx=0x7ff814010680 tx=0x7ff814015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8565f0393966efda server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:04.514 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.511+0000 7ff801ffb640 1 -- 172.21.15.16:0/94105013 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff81402c070 con 0x560b879c1370 2024-09-12T02:17:04.514 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.511+0000 7ff801ffb640 1 -- 172.21.15.16:0/94105013 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff814010df0 con 0x560b879c1370 2024-09-12T02:17:04.514 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.511+0000 7ff801ffb640 1 -- 172.21.15.16:0/94105013 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff81401e810 con 0x560b879c1370 2024-09-12T02:17:04.514 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.511+0000 7ff82561a740 1 -- 172.21.15.16:0/94105013 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b879c1370 msgr2=0x560b879b8400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:04.514 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.511+0000 7ff82561a740 1 --2- 172.21.15.16:0/94105013 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b879c1370 0x560b879b8400 secure :-1 s=READY pgs=2064 cs=0 l=1 rev1=1 crypto rx=0x7ff814010680 tx=0x7ff814015000 comp rx=0 tx=0).stop 2024-09-12T02:17:04.514 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.511+0000 7ff82561a740 1 -- 172.21.15.16:0/94105013 shutdown_connections 2024-09-12T02:17:04.514 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.511+0000 7ff82561a740 1 --2- 172.21.15.16:0/94105013 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b879c1370 0x560b879b8400 unknown :-1 s=CLOSED pgs=2064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:04.514 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.511+0000 7ff82561a740 1 --2- 172.21.15.16:0/94105013 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b879bc2e0 0x560b879cd080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:04.515 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.511+0000 7ff82561a740 1 --2- 172.21.15.16:0/94105013 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560b879bb9a0 0x560b879b60b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:04.515 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.511+0000 7ff82561a740 1 -- 172.21.15.16:0/94105013 >> 172.21.15.16:0/94105013 conn(0x560b879b7010 msgr2=0x560b879b59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:04.515 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.511+0000 7ff82561a740 1 -- 172.21.15.16:0/94105013 shutdown_connections 2024-09-12T02:17:04.515 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.511+0000 7ff82561a740 1 -- 172.21.15.16:0/94105013 wait complete. 2024-09-12T02:17:04.515 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.512+0000 7ff82561a740 1 -- start start 2024-09-12T02:17:04.515 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.512+0000 7ff82561a740 1 Processor -- start 2024-09-12T02:17:04.518 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.512+0000 7ff82561a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b879bb9a0 0x560b879cad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:04.518 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.512+0000 7ff82561a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b879bc2e0 0x560b879cb240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:04.518 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.512+0000 7ff82561a740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560b879c1370 0x560b879cc250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:04.518 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.512+0000 7ff82561a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff81c1a7470 con 0x560b879bc2e0 2024-09-12T02:17:04.518 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.512+0000 7ff82561a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff81c1a88e0 con 0x560b879bb9a0 2024-09-12T02:17:04.518 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.512+0000 7ff82561a740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff81c1a7b40 con 0x560b879c1370 2024-09-12T02:17:04.518 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.512+0000 7ff82561a740 10 client.0 _ll_register_callbacks cb 0x7ff80c0d24b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:17:04.519 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.512+0000 7ff81b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b879bc2e0 0x560b879cb240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:04.519 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.512+0000 7ff81b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b879bc2e0 0x560b879cb240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46274/0 (socket says 172.21.15.16:46274) 2024-09-12T02:17:04.519 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.512+0000 7ff81b7fe640 1 -- 172.21.15.16:0/240474306 learned_addr learned my addr 172.21.15.16:0/240474306 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:17:04.519 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.512+0000 7ff81bfff640 1 --2- 172.21.15.16:0/240474306 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b879bb9a0 0x560b879cad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:04.519 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.512+0000 7ff820e18640 1 --2- 172.21.15.16:0/240474306 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560b879c1370 0x560b879cc250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:04.519 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.513+0000 7ff81b7fe640 1 -- 172.21.15.16:0/240474306 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560b879c1370 msgr2=0x560b879cc250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:04.519 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.513+0000 7ff81b7fe640 1 --2- 172.21.15.16:0/240474306 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560b879c1370 0x560b879cc250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:04.519 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.513+0000 7ff81b7fe640 1 -- 172.21.15.16:0/240474306 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b879bb9a0 msgr2=0x560b879cad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:04.519 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.513+0000 7ff81b7fe640 1 --2- 172.21.15.16:0/240474306 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b879bb9a0 0x560b879cad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:04.519 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.513+0000 7ff81b7fe640 1 -- 172.21.15.16:0/240474306 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b879cc860 con 0x560b879bc2e0 2024-09-12T02:17:04.519 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.513+0000 7ff81b7fe640 1 --2- 172.21.15.16:0/240474306 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b879bc2e0 0x560b879cb240 secure :-1 s=READY pgs=2065 cs=0 l=1 rev1=1 crypto rx=0x7ff8100314e0 tx=0x7ff8100045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bba9953a066bcca server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:04.519 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.513+0000 7ff7fb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:17:04.519 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.513+0000 7ff7fb7fe640 1 -- 172.21.15.16:0/240474306 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff81005c040 con 0x560b879bc2e0 2024-09-12T02:17:04.520 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.513+0000 7ff7fb7fe640 1 -- 172.21.15.16:0/240474306 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7ff8100049d0 con 0x560b879bc2e0 2024-09-12T02:17:04.520 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.513+0000 7ff82561a740 1 -- 172.21.15.16:0/240474306 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff814010df0 con 0x560b879bc2e0 2024-09-12T02:17:04.520 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.513+0000 7ff7fb7fe640 1 -- 172.21.15.16:0/240474306 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff810048070 con 0x560b879bc2e0 2024-09-12T02:17:04.520 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.513+0000 7ff82561a740 10 client.20640 fetch_fsmap learned FSMap version 25 2024-09-12T02:17:04.520 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.513+0000 7ff82561a740 1 -- 172.21.15.16:0/240474306 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x560b87a3c560 con 0x560b879bc2e0 2024-09-12T02:17:04.520 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.513+0000 7ff7fb7fe640 1 -- 172.21.15.16:0/240474306 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff814010df0 con 0x560b879bc2e0 2024-09-12T02:17:04.520 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.513+0000 7ff7fb7fe640 1 -- 172.21.15.16:0/240474306 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7ff81005f020 con 0x560b879bc2e0 2024-09-12T02:17:04.520 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.513+0000 7ff82561a740 10 client.20640 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:17:04.520 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.513+0000 7ff82561a740 10 client.20640 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:17:04.520 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.514+0000 7ff82561a740 20 client.20640 populate_metadata read hostname 'smithi016' 2024-09-12T02:17:04.520 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.514+0000 7ff82561a740 1 --2- 172.21.15.16:0/240474306 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x560b87a3ce80 0x560b87a3f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:04.520 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.514+0000 7ff82561a740 4 client.20640 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:17:04.521 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.514+0000 7ff82561a740 1 -- 172.21.15.16:0/240474306 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560b87926d70 con 0x560b87a3ce80 2024-09-12T02:17:04.521 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.514+0000 7ff81bfff640 1 --2- 172.21.15.16:0/240474306 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x560b87a3ce80 0x560b87a3f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:04.521 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.514+0000 7ff7fb7fe640 10 client.20640 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:17:04.521 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.514+0000 7ff81bfff640 1 --2- 172.21.15.16:0/240474306 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x560b87a3ce80 0x560b87a3f2a0 crc :-1 s=READY pgs=458 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=daded8dacd66314d server_cookie=ac8acbea667aaf76 in_seq=0 out_seq=0 2024-09-12T02:17:04.529 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.528+0000 7ff7fb7fe640 1 -- 172.21.15.16:0/240474306 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560b87926d70 con 0x560b87a3ce80 2024-09-12T02:17:04.529 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.528+0000 7ff7fb7fe640 10 client.20640 handle_command_reply: tid=0 2024-09-12T02:17:04.529 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.528+0000 7ff82561a740 1 client.20640 shutdown 2024-09-12T02:17:04.529 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.528+0000 7ff82561a740 1 -- 172.21.15.16:0/240474306 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b879bc2e0 msgr2=0x560b879cb240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:04.529 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.528+0000 7ff82561a740 1 --2- 172.21.15.16:0/240474306 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b879bc2e0 0x560b879cb240 secure :-1 s=READY pgs=2065 cs=0 l=1 rev1=1 crypto rx=0x7ff8100314e0 tx=0x7ff8100045e0 comp rx=0 tx=0).stop 2024-09-12T02:17:04.529 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.528+0000 7ff82561a740 1 -- 172.21.15.16:0/240474306 shutdown_connections 2024-09-12T02:17:04.530 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.528+0000 7ff82561a740 1 --2- 172.21.15.16:0/240474306 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x560b87a3ce80 0x560b87a3f2a0 crc :-1 s=READY pgs=458 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:04.530 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.528+0000 7ff82561a740 1 --2- 172.21.15.16:0/240474306 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560b879c1370 0x560b879cc250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:04.530 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.528+0000 7ff82561a740 1 --2- 172.21.15.16:0/240474306 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b879bc2e0 0x560b879cb240 unknown :-1 s=CLOSED pgs=2065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:04.530 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.528+0000 7ff82561a740 1 --2- 172.21.15.16:0/240474306 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b879bb9a0 0x560b879cad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:04.530 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.528+0000 7ff7fb7fe640 0 client.20640 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:17:04.530 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.528+0000 7ff82561a740 1 -- 172.21.15.16:0/240474306 >> 172.21.15.16:0/240474306 conn(0x560b879b7010 msgr2=0x560b879b61c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:04.530 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.528+0000 7ff82561a740 1 -- 172.21.15.16:0/240474306 shutdown_connections 2024-09-12T02:17:04.530 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.528+0000 7ff82561a740 1 -- 172.21.15.16:0/240474306 wait complete. 2024-09-12T02:17:04.530 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.528+0000 7ff82561a740 20 client.20640 trim_cache size 0 max 16384 2024-09-12T02:17:04.530 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-09-12T02:17:04.531 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-09-12T02:17:04.531 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-09-12T02:17:04.531 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-09-12T02:17:04.531 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.529+0000 7ff82287b640 1 -- 172.21.15.16:0/2778608251 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7ff8040a2c30 msgr2=0x7ff8040a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:04.532 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.529+0000 7ff82287b640 1 --2- 172.21.15.16:0/2778608251 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7ff8040a2c30 0x7ff8040a50f0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7ff81c06a980 tx=0x7ff814009500 comp rx=0 tx=0).stop 2024-09-12T02:17:04.532 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.529+0000 7ff82287b640 1 -- 172.21.15.16:0/2778608251 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff81c1025c0 msgr2=0x7ff81c197e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:04.532 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.529+0000 7ff82287b640 1 --2- 172.21.15.16:0/2778608251 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff81c1025c0 0x7ff81c197e40 secure :-1 s=READY pgs=2062 cs=0 l=1 rev1=1 crypto rx=0x7ff80c00d950 tx=0x7ff80c00de10 comp rx=0 tx=0).stop 2024-09-12T02:17:04.532 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.529+0000 7ff82287b640 1 -- 172.21.15.16:0/2778608251 shutdown_connections 2024-09-12T02:17:04.532 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.529+0000 7ff82287b640 1 --2- 172.21.15.16:0/2778608251 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7ff8040a2c30 0x7ff8040a50f0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:04.532 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.529+0000 7ff82287b640 1 --2- 172.21.15.16:0/2778608251 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff81c107820 0x7ff81c19f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:04.532 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.529+0000 7ff82287b640 1 --2- 172.21.15.16:0/2778608251 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff81c104ef0 0x7ff81c198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:04.532 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.529+0000 7ff82287b640 1 --2- 172.21.15.16:0/2778608251 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff81c1025c0 0x7ff81c197e40 unknown :-1 s=CLOSED pgs=2062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:04.532 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.529+0000 7ff82287b640 1 -- 172.21.15.16:0/2778608251 >> 172.21.15.16:0/2778608251 conn(0x7ff81c0fc1e0 msgr2=0x7ff81c102f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:04.532 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.529+0000 7ff82287b640 1 -- 172.21.15.16:0/2778608251 shutdown_connections 2024-09-12T02:17:04.532 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:04.529+0000 7ff82287b640 1 -- 172.21.15.16:0/2778608251 wait complete. 2024-09-12T02:17:04.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:17:04 smithi184 ceph-mon[35593]: pgmap v789: 97 pgs: 97 active+clean; 3.7 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-09-12T02:17:04.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:17:04 smithi033 ceph-mon[35332]: pgmap v789: 97 pgs: 97 active+clean; 3.7 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-09-12T02:17:04.933 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:17:04 smithi016 ceph-mon[30947]: pgmap v789: 97 pgs: 97 active+clean; 3.7 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-09-12T02:17:06.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:17:06 smithi184 ceph-mon[35593]: pgmap v790: 97 pgs: 97 active+clean; 3.7 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-09-12T02:17:06.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:17:06 smithi184 ceph-mon[35593]: scrub summary: idle 2024-09-12T02:17:06.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:17:06 smithi033 ceph-mon[35332]: pgmap v790: 97 pgs: 97 active+clean; 3.7 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-09-12T02:17:06.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:17:06 smithi033 ceph-mon[35332]: scrub summary: idle 2024-09-12T02:17:06.933 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:17:06 smithi016 ceph-mon[30947]: pgmap v790: 97 pgs: 97 active+clean; 3.7 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-09-12T02:17:06.933 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:17:06 smithi016 ceph-mon[30947]: scrub summary: idle 2024-09-12T02:17:08.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:17:08 smithi184 ceph-mon[35593]: pgmap v791: 97 pgs: 97 active+clean; 3.8 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 18 op/s 2024-09-12T02:17:08.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:17:08 smithi033 ceph-mon[35332]: pgmap v791: 97 pgs: 97 active+clean; 3.8 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 18 op/s 2024-09-12T02:17:08.933 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:17:08 smithi016 ceph-mon[30947]: pgmap v791: 97 pgs: 97 active+clean; 3.8 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 18 op/s 2024-09-12T02:17:10.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:17:10 smithi184 ceph-mon[35593]: pgmap v792: 97 pgs: 97 active+clean; 3.8 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-12T02:17:10.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:17:10 smithi033 ceph-mon[35332]: pgmap v792: 97 pgs: 97 active+clean; 3.8 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-12T02:17:10.933 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:17:10 smithi016 ceph-mon[30947]: pgmap v792: 97 pgs: 97 active+clean; 3.8 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-12T02:17:12.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:17:12 smithi184 ceph-mon[35593]: pgmap v793: 97 pgs: 97 active+clean; 3.8 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-09-12T02:17:12.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:17:12 smithi033 ceph-mon[35332]: pgmap v793: 97 pgs: 97 active+clean; 3.8 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-09-12T02:17:12.933 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:17:12 smithi016 ceph-mon[30947]: pgmap v793: 97 pgs: 97 active+clean; 3.8 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-09-12T02:17:13.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:17:13 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:17:13.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:17:13 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:17:13.933 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:17:13 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:17:14.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:17:14 smithi184 ceph-mon[35593]: pgmap v794: 97 pgs: 97 active+clean; 3.9 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-09-12T02:17:14.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:17:14 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:17:14.878 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:17:14 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:17:14.878 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:17:14 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T02:17:14.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:17:14 smithi033 ceph-mon[35332]: pgmap v794: 97 pgs: 97 active+clean; 3.9 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-09-12T02:17:14.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:17:14 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:17:14.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:17:14 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:17:14.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:17:14 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T02:17:14.933 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:17:14 smithi016 ceph-mon[30947]: pgmap v794: 97 pgs: 97 active+clean; 3.9 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-09-12T02:17:14.934 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:17:14 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:17:14.934 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:17:14 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:17:14.934 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:17:14 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T02:17:16.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:17:16 smithi184 ceph-mon[35593]: pgmap v795: 97 pgs: 97 active+clean; 3.9 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-09-12T02:17:16.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:17:16 smithi033 ceph-mon[35332]: pgmap v795: 97 pgs: 97 active+clean; 3.9 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-09-12T02:17:16.933 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:17:16 smithi016 ceph-mon[30947]: pgmap v795: 97 pgs: 97 active+clean; 3.9 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-09-12T02:17:17.629 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.629+0000 7f7d2590a640 1 -- 172.21.15.16:0/3197562373 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d20100a60 msgr2=0x7f7d20100e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:17.630 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.629+0000 7f7d2590a640 1 --2- 172.21.15.16:0/3197562373 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d20100a60 0x7f7d20100e60 secure :-1 s=READY pgs=2066 cs=0 l=1 rev1=1 crypto rx=0x7f7d10009ca0 tx=0x7f7d1002a8c0 comp rx=0 tx=0).stop 2024-09-12T02:17:17.630 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.629+0000 7f7d2590a640 1 -- 172.21.15.16:0/3197562373 shutdown_connections 2024-09-12T02:17:17.630 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.629+0000 7f7d2590a640 1 --2- 172.21.15.16:0/3197562373 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d20102610 0x7f7d20108ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:17.630 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.629+0000 7f7d2590a640 1 --2- 172.21.15.16:0/3197562373 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7d20101c50 0x7f7d201020d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:17.630 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.629+0000 7f7d2590a640 1 --2- 172.21.15.16:0/3197562373 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d20100a60 0x7f7d20100e60 unknown :-1 s=CLOSED pgs=2066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:17.630 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.629+0000 7f7d2590a640 1 -- 172.21.15.16:0/3197562373 >> 172.21.15.16:0/3197562373 conn(0x7f7d200fc1e0 msgr2=0x7f7d200fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:17.630 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.629+0000 7f7d2590a640 1 -- 172.21.15.16:0/3197562373 shutdown_connections 2024-09-12T02:17:17.630 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.629+0000 7f7d2590a640 1 -- 172.21.15.16:0/3197562373 wait complete. 2024-09-12T02:17:17.630 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.630+0000 7f7d2590a640 1 Processor -- start 2024-09-12T02:17:17.632 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.630+0000 7f7d2590a640 1 -- start start 2024-09-12T02:17:17.632 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.630+0000 7f7d2590a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7d20100a60 0x7f7d20197e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:17.632 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.630+0000 7f7d2590a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d20101c50 0x7f7d20198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:17.632 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.630+0000 7f7d2590a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d20102610 0x7f7d2019f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:17.633 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.630+0000 7f7d2590a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7d2010c6e0 con 0x7f7d20102610 2024-09-12T02:17:17.633 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.630+0000 7f7d2590a640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7d2010c560 con 0x7f7d20101c50 2024-09-12T02:17:17.633 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.630+0000 7f7d2590a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7d2010c860 con 0x7f7d20100a60 2024-09-12T02:17:17.633 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.630+0000 7f7d1f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d20102610 0x7f7d2019f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:17.633 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.630+0000 7f7d1f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d20102610 0x7f7d2019f3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45550/0 (socket says 172.21.15.16:45550) 2024-09-12T02:17:17.633 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.630+0000 7f7d1f7fe640 1 -- 172.21.15.16:0/143792771 learned_addr learned my addr 172.21.15.16:0/143792771 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:17:17.633 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.631+0000 7f7d1e7fc640 1 --2- 172.21.15.16:0/143792771 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d20101c50 0x7f7d20198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:17.633 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.631+0000 7f7d1effd640 1 --2- 172.21.15.16:0/143792771 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7d20100a60 0x7f7d20197e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:17.633 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.631+0000 7f7d1f7fe640 1 -- 172.21.15.16:0/143792771 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7d20100a60 msgr2=0x7f7d20197e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:17.633 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.631+0000 7f7d1f7fe640 1 --2- 172.21.15.16:0/143792771 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7d20100a60 0x7f7d20197e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:17.633 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.631+0000 7f7d1f7fe640 1 -- 172.21.15.16:0/143792771 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d20101c50 msgr2=0x7f7d20198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:17.633 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.631+0000 7f7d1f7fe640 1 --2- 172.21.15.16:0/143792771 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d20101c50 0x7f7d20198370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:17.633 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.631+0000 7f7d1f7fe640 1 -- 172.21.15.16:0/143792771 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d2019fa50 con 0x7f7d20102610 2024-09-12T02:17:17.634 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.631+0000 7f7d1f7fe640 1 --2- 172.21.15.16:0/143792771 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d20102610 0x7f7d2019f3e0 secure :-1 s=READY pgs=2067 cs=0 l=1 rev1=1 crypto rx=0x7f7d1800e970 tx=0x7f7d1800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:17.634 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.631+0000 7f7d24908640 1 -- 172.21.15.16:0/143792771 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d1800ccc0 con 0x7f7d20102610 2024-09-12T02:17:17.634 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.631+0000 7f7d24908640 1 -- 172.21.15.16:0/143792771 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d1800ce60 con 0x7f7d20102610 2024-09-12T02:17:17.634 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.631+0000 7f7d24908640 1 -- 172.21.15.16:0/143792771 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d18010490 con 0x7f7d20102610 2024-09-12T02:17:17.634 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.631+0000 7f7d2590a640 1 -- 172.21.15.16:0/143792771 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d2019fd40 con 0x7f7d20102610 2024-09-12T02:17:17.634 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.631+0000 7f7d2590a640 1 -- 172.21.15.16:0/143792771 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d201a01e0 con 0x7f7d20102610 2024-09-12T02:17:17.641 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.634+0000 7f7d24908640 1 -- 172.21.15.16:0/143792771 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f7d18005020 con 0x7f7d20102610 2024-09-12T02:17:17.641 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.634+0000 7f7d2869f740 1 Processor -- start 2024-09-12T02:17:17.641 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.634+0000 7f7d2869f740 1 -- start start 2024-09-12T02:17:17.641 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.634+0000 7f7d2869f740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5587069b19a0 0x5587069b1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:17.641 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.634+0000 7f7d2869f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5587069b22e0 0x5587069b6da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:17.641 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.634+0000 7f7d2869f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587069b7370 0x5587069b9830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:17.641 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.634+0000 7f7d2869f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5587069152f0 con 0x5587069b7370 2024-09-12T02:17:17.641 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.634+0000 7f7d2869f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5587068bd270 con 0x5587069b22e0 2024-09-12T02:17:17.641 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.634+0000 7f7d2869f740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5587069112b0 con 0x5587069b19a0 2024-09-12T02:17:17.641 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.634+0000 7f7d1e7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587069b7370 0x5587069b9830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:17.641 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.634+0000 7f7d1e7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587069b7370 0x5587069b9830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45560/0 (socket says 172.21.15.16:45560) 2024-09-12T02:17:17.642 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.634+0000 7f7d1e7fc640 1 -- 172.21.15.16:0/1725027118 learned_addr learned my addr 172.21.15.16:0/1725027118 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:17:17.642 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.634+0000 7f7d1effd640 1 --2- 172.21.15.16:0/1725027118 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5587069b22e0 0x5587069b6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:17.642 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.634+0000 7f7d1e7fc640 1 --2- 172.21.15.16:0/1725027118 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5587069b19a0 0x5587069b1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:17.642 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.635+0000 7f7d1e7fc640 1 -- 172.21.15.16:0/1725027118 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5587069b19a0 msgr2=0x5587069b1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:17.642 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.635+0000 7f7d1e7fc640 1 --2- 172.21.15.16:0/1725027118 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5587069b19a0 0x5587069b1da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:17.642 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.635+0000 7f7d1e7fc640 1 -- 172.21.15.16:0/1725027118 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5587069b22e0 msgr2=0x5587069b6da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:17.642 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.635+0000 7f7d1e7fc640 1 --2- 172.21.15.16:0/1725027118 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5587069b22e0 0x5587069b6da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:17.642 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.635+0000 7f7d1e7fc640 1 -- 172.21.15.16:0/1725027118 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587068bc340 con 0x5587069b7370 2024-09-12T02:17:17.642 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.635+0000 7f7d1effd640 1 --2- 172.21.15.16:0/1725027118 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5587069b22e0 0x5587069b6da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:17:17.642 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.635+0000 7f7d1e7fc640 1 --2- 172.21.15.16:0/1725027118 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587069b7370 0x5587069b9830 secure :-1 s=READY pgs=2068 cs=0 l=1 rev1=1 crypto rx=0x7f7d08009fa0 tx=0x7f7d080097b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12824931851dd172 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:17.642 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.635+0000 7f7d24908640 1 --2- 172.21.15.16:0/143792771 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f7d0c0a2c30 0x7f7d0c0a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:17.642 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.635+0000 7f7d15ffb640 1 -- 172.21.15.16:0/1725027118 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d08004230 con 0x5587069b7370 2024-09-12T02:17:17.643 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.635+0000 7f7d15ffb640 1 -- 172.21.15.16:0/1725027118 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d080043d0 con 0x5587069b7370 2024-09-12T02:17:17.643 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.635+0000 7f7d15ffb640 1 -- 172.21.15.16:0/1725027118 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d08010940 con 0x5587069b7370 2024-09-12T02:17:17.643 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.635+0000 7f7d1effd640 1 --2- 172.21.15.16:0/143792771 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f7d0c0a2c30 0x7f7d0c0a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:17.643 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.635+0000 7f7d24908640 1 -- 172.21.15.16:0/143792771 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f7d180cf590 con 0x7f7d20102610 2024-09-12T02:17:17.643 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.636+0000 7f7d2869f740 1 -- 172.21.15.16:0/1725027118 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587069b7370 msgr2=0x5587069b9830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:17.643 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.636+0000 7f7d2869f740 1 --2- 172.21.15.16:0/1725027118 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587069b7370 0x5587069b9830 secure :-1 s=READY pgs=2068 cs=0 l=1 rev1=1 crypto rx=0x7f7d08009fa0 tx=0x7f7d080097b0 comp rx=0 tx=0).stop 2024-09-12T02:17:17.643 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.636+0000 7f7d1effd640 1 --2- 172.21.15.16:0/143792771 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f7d0c0a2c30 0x7f7d0c0a50f0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f7d10006010 tx=0x7f7d10031bb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:17.643 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.636+0000 7f7d2869f740 1 -- 172.21.15.16:0/1725027118 shutdown_connections 2024-09-12T02:17:17.643 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.636+0000 7f7d2869f740 1 --2- 172.21.15.16:0/1725027118 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587069b7370 0x5587069b9830 unknown :-1 s=CLOSED pgs=2068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:17.643 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.636+0000 7f7d2869f740 1 --2- 172.21.15.16:0/1725027118 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5587069b22e0 0x5587069b6da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:17.643 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.636+0000 7f7d2869f740 1 --2- 172.21.15.16:0/1725027118 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5587069b19a0 0x5587069b1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:17.643 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.636+0000 7f7d2869f740 1 -- 172.21.15.16:0/1725027118 >> 172.21.15.16:0/1725027118 conn(0x5587069ad010 msgr2=0x5587069ad410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:17.643 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.636+0000 7f7d2869f740 1 -- 172.21.15.16:0/1725027118 shutdown_connections 2024-09-12T02:17:17.643 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.636+0000 7f7d2869f740 1 -- 172.21.15.16:0/1725027118 wait complete. 2024-09-12T02:17:17.644 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.636+0000 7f7d2869f740 1 -- start start 2024-09-12T02:17:17.644 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.637+0000 7f7d2869f740 1 Processor -- start 2024-09-12T02:17:17.644 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.637+0000 7f7d2869f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587069b19a0 0x5587069bbb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:17.644 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.637+0000 7f7d2869f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5587069b22e0 0x5587069bc0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:17.644 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.637+0000 7f7d2869f740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5587069b7370 0x5587069c3110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:17.644 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.637+0000 7f7d2869f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5587069ba340 con 0x5587069b19a0 2024-09-12T02:17:17.644 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.637+0000 7f7d2869f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55870689e290 con 0x5587069b22e0 2024-09-12T02:17:17.644 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.637+0000 7f7d2869f740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5587069ba4c0 con 0x5587069b7370 2024-09-12T02:17:17.644 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.637+0000 7f7d2869f740 10 client.0 _ll_register_callbacks cb 0x5587068cf140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:17:17.644 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.637+0000 7f7d1effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587069b19a0 0x5587069bbb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:17.644 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.637+0000 7f7d1effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587069b19a0 0x5587069bbb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45566/0 (socket says 172.21.15.16:45566) 2024-09-12T02:17:17.645 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.637+0000 7f7d1effd640 1 -- 172.21.15.16:0/3726971581 learned_addr learned my addr 172.21.15.16:0/3726971581 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:17:17.645 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.637+0000 7f7d1e7fc640 1 --2- 172.21.15.16:0/3726971581 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5587069b22e0 0x5587069bc0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:17.645 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.637+0000 7f7d1f7fe640 1 --2- 172.21.15.16:0/3726971581 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5587069b7370 0x5587069c3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:17.645 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.637+0000 7f7d1effd640 1 -- 172.21.15.16:0/3726971581 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5587069b7370 msgr2=0x5587069c3110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:17.645 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.637+0000 7f7d1effd640 1 --2- 172.21.15.16:0/3726971581 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5587069b7370 0x5587069c3110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:17.645 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.637+0000 7f7d1effd640 1 -- 172.21.15.16:0/3726971581 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5587069b22e0 msgr2=0x5587069bc0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:17.645 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.637+0000 7f7d1effd640 1 --2- 172.21.15.16:0/3726971581 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5587069b22e0 0x5587069bc0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:17.645 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.637+0000 7f7d1effd640 1 -- 172.21.15.16:0/3726971581 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558706628210 con 0x5587069b19a0 2024-09-12T02:17:17.645 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.638+0000 7f7d1effd640 1 --2- 172.21.15.16:0/3726971581 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587069b19a0 0x5587069bbb90 secure :-1 s=READY pgs=2069 cs=0 l=1 rev1=1 crypto rx=0x7f7d10009ca0 tx=0x7f7d1002a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33ff149c8f48c35f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:17.645 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.638+0000 7f7cfeffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:17:17.645 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.638+0000 7f7cfeffd640 1 -- 172.21.15.16:0/3726971581 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d100318c0 con 0x5587069b19a0 2024-09-12T02:17:17.645 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.638+0000 7f7cfeffd640 1 -- 172.21.15.16:0/3726971581 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f7d10003e00 con 0x5587069b19a0 2024-09-12T02:17:17.645 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.638+0000 7f7d2869f740 1 -- 172.21.15.16:0/3726971581 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d080043d0 con 0x5587069b19a0 2024-09-12T02:17:17.646 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.638+0000 7f7cfeffd640 1 -- 172.21.15.16:0/3726971581 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7d10004510 con 0x5587069b19a0 2024-09-12T02:17:17.646 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.638+0000 7f7d2869f740 10 client.20652 fetch_fsmap learned FSMap version 25 2024-09-12T02:17:17.646 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.638+0000 7f7d2869f740 1 -- 172.21.15.16:0/3726971581 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5587066f4620 con 0x5587069b19a0 2024-09-12T02:17:17.646 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.638+0000 7f7cfeffd640 1 -- 172.21.15.16:0/3726971581 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d080043d0 con 0x5587069b19a0 2024-09-12T02:17:17.646 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.638+0000 7f7cfeffd640 1 -- 172.21.15.16:0/3726971581 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f7d100305c0 con 0x5587069b19a0 2024-09-12T02:17:17.646 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.639+0000 7f7d2869f740 10 client.20652 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:17:17.646 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.639+0000 7f7d2869f740 10 client.20652 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:17:17.646 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.639+0000 7f7d2869f740 20 client.20652 populate_metadata read hostname 'smithi016' 2024-09-12T02:17:17.646 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.639+0000 7f7d2869f740 1 --2- 172.21.15.16:0/3726971581 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5587069c7850 0x5587069c9ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:17.646 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.639+0000 7f7d2869f740 4 client.20652 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:17:17.646 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.639+0000 7f7d2869f740 1 -- 172.21.15.16:0/3726971581 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558706922a20 con 0x5587069c7850 2024-09-12T02:17:17.646 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.639+0000 7f7d1e7fc640 1 --2- 172.21.15.16:0/3726971581 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5587069c7850 0x5587069c9ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:17.646 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.639+0000 7f7cfeffd640 10 client.20652 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:17:17.647 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.639+0000 7f7d1e7fc640 1 --2- 172.21.15.16:0/3726971581 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5587069c7850 0x5587069c9ca0 crc :-1 s=READY pgs=459 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b5740cc8d63cb7f server_cookie=7c884b893e0efa3b in_seq=0 out_seq=0 2024-09-12T02:17:17.647 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.640+0000 7f7cfeffd640 1 -- 172.21.15.16:0/3726971581 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558706922a20 con 0x5587069c7850 2024-09-12T02:17:17.647 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.640+0000 7f7cfeffd640 10 client.20652 handle_command_reply: tid=0 2024-09-12T02:17:17.647 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.640+0000 7f7d2869f740 1 client.20652 shutdown 2024-09-12T02:17:17.647 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.640+0000 7f7d2869f740 1 -- 172.21.15.16:0/3726971581 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587069b19a0 msgr2=0x5587069bbb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:17.647 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.640+0000 7f7d2869f740 1 --2- 172.21.15.16:0/3726971581 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587069b19a0 0x5587069bbb90 secure :-1 s=READY pgs=2069 cs=0 l=1 rev1=1 crypto rx=0x7f7d10009ca0 tx=0x7f7d1002a8c0 comp rx=0 tx=0).stop 2024-09-12T02:17:17.647 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.640+0000 7f7d2869f740 1 -- 172.21.15.16:0/3726971581 shutdown_connections 2024-09-12T02:17:17.647 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.640+0000 7f7d2869f740 1 --2- 172.21.15.16:0/3726971581 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5587069c7850 0x5587069c9ca0 crc :-1 s=READY pgs=459 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:17.647 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.640+0000 7f7d2869f740 1 --2- 172.21.15.16:0/3726971581 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5587069b7370 0x5587069c3110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:17.647 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.640+0000 7f7d2869f740 1 --2- 172.21.15.16:0/3726971581 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5587069b22e0 0x5587069bc0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:17.647 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.640+0000 7f7cfeffd640 0 client.20652 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:17:17.648 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.640+0000 7f7d2869f740 1 --2- 172.21.15.16:0/3726971581 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587069b19a0 0x5587069bbb90 unknown :-1 s=CLOSED pgs=2069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:17.648 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.640+0000 7f7d2869f740 1 -- 172.21.15.16:0/3726971581 >> 172.21.15.16:0/3726971581 conn(0x5587069ad010 msgr2=0x5587069ae820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:17.648 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.640+0000 7f7d2869f740 1 -- 172.21.15.16:0/3726971581 shutdown_connections 2024-09-12T02:17:17.648 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.640+0000 7f7d2869f740 1 -- 172.21.15.16:0/3726971581 wait complete. 2024-09-12T02:17:17.648 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.640+0000 7f7d2869f740 20 client.20652 trim_cache size 0 max 16384 2024-09-12T02:17:17.659 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.655+0000 7f7d2869f740 1 Processor -- start 2024-09-12T02:17:17.660 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.655+0000 7f7d2869f740 1 -- start start 2024-09-12T02:17:17.660 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.655+0000 7f7d2869f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5587069b19a0 0x5587069ac0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:17.660 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.655+0000 7f7d2869f740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5587069b22e0 0x5587069c30d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:17.660 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.655+0000 7f7d2869f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587069b7370 0x5587069ae400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:17.660 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.655+0000 7f7d2869f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7d201ab6e0 con 0x5587069b7370 2024-09-12T02:17:17.660 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.655+0000 7f7d2869f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7d201a8fb0 con 0x5587069b19a0 2024-09-12T02:17:17.660 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.655+0000 7f7d2869f740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7d201a6e00 con 0x5587069b22e0 2024-09-12T02:17:17.660 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.655+0000 7f7d1f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587069b7370 0x5587069ae400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:17.660 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.655+0000 7f7d1f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587069b7370 0x5587069ae400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45580/0 (socket says 172.21.15.16:45580) 2024-09-12T02:17:17.660 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.655+0000 7f7d1f7fe640 1 -- 172.21.15.16:0/2064974830 learned_addr learned my addr 172.21.15.16:0/2064974830 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:17:17.660 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.655+0000 7f7d1effd640 1 --2- 172.21.15.16:0/2064974830 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5587069b19a0 0x5587069ac0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:17.660 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.655+0000 7f7d1f7fe640 1 -- 172.21.15.16:0/2064974830 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5587069b22e0 msgr2=0x5587069c30d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:17.661 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.655+0000 7f7d1f7fe640 1 --2- 172.21.15.16:0/2064974830 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5587069b22e0 0x5587069c30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:17.661 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.656+0000 7f7d1f7fe640 1 -- 172.21.15.16:0/2064974830 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5587069b19a0 msgr2=0x5587069ac0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:17.661 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.656+0000 7f7d1f7fe640 1 --2- 172.21.15.16:0/2064974830 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5587069b19a0 0x5587069ac0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:17.661 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.656+0000 7f7d1f7fe640 1 -- 172.21.15.16:0/2064974830 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587069ace30 con 0x5587069b7370 2024-09-12T02:17:17.661 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.656+0000 7f7d1effd640 1 --2- 172.21.15.16:0/2064974830 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5587069b19a0 0x5587069ac0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:17:17.661 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.656+0000 7f7d1f7fe640 1 --2- 172.21.15.16:0/2064974830 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587069b7370 0x5587069ae400 secure :-1 s=READY pgs=2070 cs=0 l=1 rev1=1 crypto rx=0x7f7d1800d010 tx=0x7f7d1809b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db96bea59d257fb8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:17.661 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.656+0000 7f7d157fa640 1 -- 172.21.15.16:0/2064974830 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d180a4b00 con 0x5587069b7370 2024-09-12T02:17:17.661 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.656+0000 7f7d157fa640 1 -- 172.21.15.16:0/2064974830 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d180b8030 con 0x5587069b7370 2024-09-12T02:17:17.661 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.656+0000 7f7d157fa640 1 -- 172.21.15.16:0/2064974830 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d18005310 con 0x5587069b7370 2024-09-12T02:17:17.661 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.656+0000 7f7d2869f740 1 -- 172.21.15.16:0/2064974830 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587069b7370 msgr2=0x5587069ae400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:17.661 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.656+0000 7f7d2869f740 1 --2- 172.21.15.16:0/2064974830 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587069b7370 0x5587069ae400 secure :-1 s=READY pgs=2070 cs=0 l=1 rev1=1 crypto rx=0x7f7d1800d010 tx=0x7f7d1809b8b0 comp rx=0 tx=0).stop 2024-09-12T02:17:17.661 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.656+0000 7f7d2869f740 1 -- 172.21.15.16:0/2064974830 shutdown_connections 2024-09-12T02:17:17.661 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.656+0000 7f7d2869f740 1 --2- 172.21.15.16:0/2064974830 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587069b7370 0x5587069ae400 unknown :-1 s=CLOSED pgs=2070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:17.662 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.656+0000 7f7d2869f740 1 --2- 172.21.15.16:0/2064974830 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5587069b22e0 0x5587069c30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:17.662 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.656+0000 7f7d2869f740 1 --2- 172.21.15.16:0/2064974830 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5587069b19a0 0x5587069ac0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:17.662 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.656+0000 7f7d2869f740 1 -- 172.21.15.16:0/2064974830 >> 172.21.15.16:0/2064974830 conn(0x5587069ad010 msgr2=0x5587069ab9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:17.662 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.657+0000 7f7d2869f740 1 -- 172.21.15.16:0/2064974830 shutdown_connections 2024-09-12T02:17:17.662 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.657+0000 7f7d2869f740 1 -- 172.21.15.16:0/2064974830 wait complete. 2024-09-12T02:17:17.662 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.657+0000 7f7d2869f740 1 -- start start 2024-09-12T02:17:17.662 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.657+0000 7f7d2869f740 1 Processor -- start 2024-09-12T02:17:17.662 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.657+0000 7f7d2869f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587069b19a0 0x5587069c07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:17.662 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.657+0000 7f7d2869f740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5587069b22e0 0x5587069c0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:17.662 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.657+0000 7f7d2869f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5587069b7370 0x5587069c1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:17.662 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.657+0000 7f7d2869f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7d201a7470 con 0x5587069b19a0 2024-09-12T02:17:17.662 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.657+0000 7f7d2869f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7d201a88e0 con 0x5587069b7370 2024-09-12T02:17:17.662 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.657+0000 7f7d2869f740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7d201a7b40 con 0x5587069b22e0 2024-09-12T02:17:17.662 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.657+0000 7f7d2869f740 10 client.0 _ll_register_callbacks cb 0x7f7d18005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:17:17.663 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.657+0000 7f7d1effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587069b19a0 0x5587069c07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:17.663 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.658+0000 7f7d1effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587069b19a0 0x5587069c07a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45586/0 (socket says 172.21.15.16:45586) 2024-09-12T02:17:17.663 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.658+0000 7f7d1effd640 1 -- 172.21.15.16:0/3531159261 learned_addr learned my addr 172.21.15.16:0/3531159261 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:17:17.663 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.658+0000 7f7d1e7fc640 1 --2- 172.21.15.16:0/3531159261 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5587069b22e0 0x5587069c0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:17.663 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.658+0000 7f7d1f7fe640 1 --2- 172.21.15.16:0/3531159261 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5587069b7370 0x5587069c1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:17.663 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.658+0000 7f7d1effd640 1 -- 172.21.15.16:0/3531159261 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5587069b22e0 msgr2=0x5587069c0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:17.663 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.658+0000 7f7d1effd640 1 --2- 172.21.15.16:0/3531159261 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5587069b22e0 0x5587069c0ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:17.663 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.658+0000 7f7d1effd640 1 -- 172.21.15.16:0/3531159261 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5587069b7370 msgr2=0x5587069c1cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:17.663 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.658+0000 7f7d1effd640 1 --2- 172.21.15.16:0/3531159261 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5587069b7370 0x5587069c1cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:17.663 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.658+0000 7f7d1effd640 1 -- 172.21.15.16:0/3531159261 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5587069c2300 con 0x5587069b19a0 2024-09-12T02:17:17.663 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.658+0000 7f7d1f7fe640 1 --2- 172.21.15.16:0/3531159261 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5587069b7370 0x5587069c1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:17:17.663 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.658+0000 7f7d1effd640 1 --2- 172.21.15.16:0/3531159261 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587069b19a0 0x5587069c07a0 secure :-1 s=READY pgs=2071 cs=0 l=1 rev1=1 crypto rx=0x7f7d10009a60 tx=0x7f7d100093e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97e9cf5e8eb5b42f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:17.664 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.658+0000 7f7cfeffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:17:17.664 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.658+0000 7f7cfeffd640 1 -- 172.21.15.16:0/3531159261 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d10060020 con 0x5587069b19a0 2024-09-12T02:17:17.664 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.658+0000 7f7d2869f740 1 -- 172.21.15.16:0/3531159261 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d180b8030 con 0x5587069b19a0 2024-09-12T02:17:17.664 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.658+0000 7f7cfeffd640 1 -- 172.21.15.16:0/3531159261 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f7d100305c0 con 0x5587069b19a0 2024-09-12T02:17:17.664 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.658+0000 7f7cfeffd640 1 -- 172.21.15.16:0/3531159261 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7d100318c0 con 0x5587069b19a0 2024-09-12T02:17:17.664 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.658+0000 7f7d2869f740 10 client.20658 fetch_fsmap learned FSMap version 25 2024-09-12T02:17:17.664 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.658+0000 7f7d2869f740 1 -- 172.21.15.16:0/3531159261 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x558706a32580 con 0x5587069b19a0 2024-09-12T02:17:17.664 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.659+0000 7f7cfeffd640 1 -- 172.21.15.16:0/3531159261 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d180b8030 con 0x5587069b19a0 2024-09-12T02:17:17.664 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.659+0000 7f7cfeffd640 1 -- 172.21.15.16:0/3531159261 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f7d10065440 con 0x5587069b19a0 2024-09-12T02:17:17.664 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.659+0000 7f7d2869f740 10 client.20658 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:17:17.664 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.659+0000 7f7d2869f740 10 client.20658 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:17:17.664 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.659+0000 7f7d2869f740 20 client.20658 populate_metadata read hostname 'smithi016' 2024-09-12T02:17:17.664 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.659+0000 7f7d2869f740 1 --2- 172.21.15.16:0/3531159261 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x558706a32f30 0x558706a35350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:17.664 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.659+0000 7f7d2869f740 4 client.20658 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:17:17.665 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.659+0000 7f7d2869f740 1 -- 172.21.15.16:0/3531159261 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55870691cd70 con 0x558706a32f30 2024-09-12T02:17:17.665 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.659+0000 7f7d1e7fc640 1 --2- 172.21.15.16:0/3531159261 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x558706a32f30 0x558706a35350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:17.665 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.659+0000 7f7d1e7fc640 1 --2- 172.21.15.16:0/3531159261 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x558706a32f30 0x558706a35350 crc :-1 s=READY pgs=460 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1692096f410b2189 server_cookie=41770559d425a21e in_seq=0 out_seq=0 2024-09-12T02:17:17.665 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.659+0000 7f7cfeffd640 10 client.20658 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:17:17.677 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.674+0000 7f7cfeffd640 1 -- 172.21.15.16:0/3531159261 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55870691cd70 con 0x558706a32f30 2024-09-12T02:17:17.677 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.674+0000 7f7cfeffd640 10 client.20658 handle_command_reply: tid=0 2024-09-12T02:17:17.677 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.674+0000 7f7d2869f740 1 client.20658 shutdown 2024-09-12T02:17:17.677 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.675+0000 7f7d2869f740 1 -- 172.21.15.16:0/3531159261 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587069b19a0 msgr2=0x5587069c07a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:17.677 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.675+0000 7f7d2869f740 1 --2- 172.21.15.16:0/3531159261 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587069b19a0 0x5587069c07a0 secure :-1 s=READY pgs=2071 cs=0 l=1 rev1=1 crypto rx=0x7f7d10009a60 tx=0x7f7d100093e0 comp rx=0 tx=0).stop 2024-09-12T02:17:17.677 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.675+0000 7f7d2869f740 1 -- 172.21.15.16:0/3531159261 shutdown_connections 2024-09-12T02:17:17.677 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.675+0000 7f7d2869f740 1 --2- 172.21.15.16:0/3531159261 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x558706a32f30 0x558706a35350 crc :-1 s=READY pgs=460 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:17.677 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.675+0000 7f7d2869f740 1 --2- 172.21.15.16:0/3531159261 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5587069b7370 0x5587069c1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:17.677 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.675+0000 7f7d2869f740 1 --2- 172.21.15.16:0/3531159261 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5587069b22e0 0x5587069c0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:17.677 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.675+0000 7f7d2869f740 1 --2- 172.21.15.16:0/3531159261 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587069b19a0 0x5587069c07a0 unknown :-1 s=CLOSED pgs=2071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:17.678 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.675+0000 7f7cfeffd640 0 client.20658 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:17:17.678 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.675+0000 7f7d2869f740 1 -- 172.21.15.16:0/3531159261 >> 172.21.15.16:0/3531159261 conn(0x5587069ad010 msgr2=0x5587069ac1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:17.678 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.675+0000 7f7d2869f740 1 -- 172.21.15.16:0/3531159261 shutdown_connections 2024-09-12T02:17:17.678 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.675+0000 7f7d2869f740 1 -- 172.21.15.16:0/3531159261 wait complete. 2024-09-12T02:17:17.678 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.675+0000 7f7d2869f740 20 client.20658 trim_cache size 0 max 16384 2024-09-12T02:17:17.678 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-09-12T02:17:17.678 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-09-12T02:17:17.678 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-09-12T02:17:17.678 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-09-12T02:17:17.679 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.676+0000 7f7d2590a640 1 -- 172.21.15.16:0/143792771 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f7d0c0a2c30 msgr2=0x7f7d0c0a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:17.679 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.676+0000 7f7d2590a640 1 --2- 172.21.15.16:0/143792771 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f7d0c0a2c30 0x7f7d0c0a50f0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f7d10006010 tx=0x7f7d10031bb0 comp rx=0 tx=0).stop 2024-09-12T02:17:17.679 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.676+0000 7f7d2590a640 1 -- 172.21.15.16:0/143792771 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d20102610 msgr2=0x7f7d2019f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:17.679 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.676+0000 7f7d2590a640 1 --2- 172.21.15.16:0/143792771 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d20102610 0x7f7d2019f3e0 secure :-1 s=READY pgs=2067 cs=0 l=1 rev1=1 crypto rx=0x7f7d1800e970 tx=0x7f7d1800ed70 comp rx=0 tx=0).stop 2024-09-12T02:17:17.679 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.676+0000 7f7d2590a640 1 -- 172.21.15.16:0/143792771 shutdown_connections 2024-09-12T02:17:17.679 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.676+0000 7f7d2590a640 1 --2- 172.21.15.16:0/143792771 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f7d0c0a2c30 0x7f7d0c0a50f0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:17.680 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.676+0000 7f7d2590a640 1 --2- 172.21.15.16:0/143792771 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d20102610 0x7f7d2019f3e0 unknown :-1 s=CLOSED pgs=2067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:17.680 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.676+0000 7f7d2590a640 1 --2- 172.21.15.16:0/143792771 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d20101c50 0x7f7d20198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:17.680 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.676+0000 7f7d2590a640 1 --2- 172.21.15.16:0/143792771 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7d20100a60 0x7f7d20197e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:17.680 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.676+0000 7f7d2590a640 1 -- 172.21.15.16:0/143792771 >> 172.21.15.16:0/143792771 conn(0x7f7d200fc1e0 msgr2=0x7f7d200fdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:17.680 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.676+0000 7f7d2590a640 1 -- 172.21.15.16:0/143792771 shutdown_connections 2024-09-12T02:17:17.680 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:17.676+0000 7f7d2590a640 1 -- 172.21.15.16:0/143792771 wait complete. 2024-09-12T02:17:18.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:17:18 smithi184 ceph-mon[35593]: pgmap v796: 97 pgs: 97 active+clean; 3.9 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 17 op/s 2024-09-12T02:17:18.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:17:18 smithi033 ceph-mon[35332]: pgmap v796: 97 pgs: 97 active+clean; 3.9 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 17 op/s 2024-09-12T02:17:18.933 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:17:18 smithi016 ceph-mon[30947]: pgmap v796: 97 pgs: 97 active+clean; 3.9 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 17 op/s 2024-09-12T02:17:20.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:17:20 smithi184 ceph-mon[35593]: pgmap v797: 97 pgs: 97 active+clean; 3.9 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-12T02:17:20.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:17:20 smithi033 ceph-mon[35332]: pgmap v797: 97 pgs: 97 active+clean; 3.9 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-12T02:17:20.933 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:17:20 smithi016 ceph-mon[30947]: pgmap v797: 97 pgs: 97 active+clean; 3.9 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-12T02:17:22.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:17:22 smithi184 ceph-mon[35593]: pgmap v798: 97 pgs: 97 active+clean; 3.9 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-12T02:17:22.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:17:22 smithi033 ceph-mon[35332]: pgmap v798: 97 pgs: 97 active+clean; 3.9 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-12T02:17:22.933 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:17:22 smithi016 ceph-mon[30947]: pgmap v798: 97 pgs: 97 active+clean; 3.9 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-12T02:17:23.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:17:23 smithi184 ceph-mon[35593]: pgmap v799: 97 pgs: 97 active+clean; 4.0 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-09-12T02:17:23.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:17:23 smithi033 ceph-mon[35332]: pgmap v799: 97 pgs: 97 active+clean; 4.0 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-09-12T02:17:23.933 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:17:23 smithi016 ceph-mon[30947]: pgmap v799: 97 pgs: 97 active+clean; 4.0 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-09-12T02:17:25.933 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:17:25 smithi016 ceph-mon[30947]: pgmap v800: 97 pgs: 97 active+clean; 4.0 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-12T02:17:26.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:17:25 smithi184 ceph-mon[35593]: pgmap v800: 97 pgs: 97 active+clean; 4.0 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-12T02:17:26.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:17:25 smithi033 ceph-mon[35332]: pgmap v800: 97 pgs: 97 active+clean; 4.0 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-12T02:17:27.933 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:17:27 smithi016 ceph-mon[30947]: pgmap v801: 97 pgs: 97 active+clean; 4.0 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-09-12T02:17:28.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:17:27 smithi184 ceph-mon[35593]: pgmap v801: 97 pgs: 97 active+clean; 4.0 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-09-12T02:17:28.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:17:27 smithi033 ceph-mon[35332]: pgmap v801: 97 pgs: 97 active+clean; 4.0 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-09-12T02:17:29.933 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:17:29 smithi016 ceph-mon[30947]: pgmap v802: 97 pgs: 97 active+clean; 4.0 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 11 op/s 2024-09-12T02:17:30.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:17:29 smithi184 ceph-mon[35593]: pgmap v802: 97 pgs: 97 active+clean; 4.0 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 11 op/s 2024-09-12T02:17:30.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:17:29 smithi033 ceph-mon[35332]: pgmap v802: 97 pgs: 97 active+clean; 4.0 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 11 op/s 2024-09-12T02:17:30.770 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.767+0000 7f5e6f56d640 1 -- 172.21.15.16:0/3026576474 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e68100a40 msgr2=0x7f5e68100e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:30.771 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.767+0000 7f5e6f56d640 1 --2- 172.21.15.16:0/3026576474 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e68100a40 0x7f5e68100e40 secure :-1 s=READY pgs=2072 cs=0 l=1 rev1=1 crypto rx=0x7f5e64009900 tx=0x7f5e6402a410 comp rx=0 tx=0).stop 2024-09-12T02:17:30.771 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.767+0000 7f5e6f56d640 1 -- 172.21.15.16:0/3026576474 shutdown_connections 2024-09-12T02:17:30.771 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.767+0000 7f5e6f56d640 1 --2- 172.21.15.16:0/3026576474 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5e681025f0 0x7f5e68108e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:30.771 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.767+0000 7f5e6f56d640 1 --2- 172.21.15.16:0/3026576474 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5e68101c30 0x7f5e681020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:30.771 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.767+0000 7f5e6f56d640 1 --2- 172.21.15.16:0/3026576474 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e68100a40 0x7f5e68100e40 unknown :-1 s=CLOSED pgs=2072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:30.771 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.767+0000 7f5e6f56d640 1 -- 172.21.15.16:0/3026576474 >> 172.21.15.16:0/3026576474 conn(0x7f5e680fc1e0 msgr2=0x7f5e680fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:30.772 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.767+0000 7f5e6f56d640 1 -- 172.21.15.16:0/3026576474 shutdown_connections 2024-09-12T02:17:30.772 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.767+0000 7f5e6f56d640 1 -- 172.21.15.16:0/3026576474 wait complete. 2024-09-12T02:17:30.772 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.767+0000 7f5e6f56d640 1 Processor -- start 2024-09-12T02:17:30.772 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.767+0000 7f5e6f56d640 1 -- start start 2024-09-12T02:17:30.772 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.768+0000 7f5e6f56d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5e68100a40 0x7f5e68197df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:30.773 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.768+0000 7f5e6f56d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e68101c30 0x7f5e68198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:30.773 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.768+0000 7f5e6f56d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5e681025f0 0x7f5e6819f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:30.773 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.768+0000 7f5e6f56d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5e6810c6c0 con 0x7f5e68101c30 2024-09-12T02:17:30.773 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.768+0000 7f5e6f56d640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5e6810c540 con 0x7f5e681025f0 2024-09-12T02:17:30.773 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.768+0000 7f5e6f56d640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5e6810c840 con 0x7f5e68100a40 2024-09-12T02:17:30.774 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.768+0000 7f5e6cb08640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e68101c30 0x7f5e68198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:30.774 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.768+0000 7f5e6cb08640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e68101c30 0x7f5e68198330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50700/0 (socket says 172.21.15.16:50700) 2024-09-12T02:17:30.774 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.768+0000 7f5e6cb08640 1 -- 172.21.15.16:0/2686600137 learned_addr learned my addr 172.21.15.16:0/2686600137 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:17:30.774 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.768+0000 7f5e6db0a640 1 --2- 172.21.15.16:0/2686600137 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5e681025f0 0x7f5e6819f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:30.774 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.768+0000 7f5e6d309640 1 --2- 172.21.15.16:0/2686600137 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5e68100a40 0x7f5e68197df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:30.774 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.768+0000 7f5e6cb08640 1 -- 172.21.15.16:0/2686600137 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5e68100a40 msgr2=0x7f5e68197df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:30.775 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.768+0000 7f5e6cb08640 1 --2- 172.21.15.16:0/2686600137 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5e68100a40 0x7f5e68197df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:30.775 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.768+0000 7f5e6cb08640 1 -- 172.21.15.16:0/2686600137 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5e681025f0 msgr2=0x7f5e6819f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:30.775 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.768+0000 7f5e6cb08640 1 --2- 172.21.15.16:0/2686600137 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5e681025f0 0x7f5e6819f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:30.775 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.768+0000 7f5e6cb08640 1 -- 172.21.15.16:0/2686600137 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e6819fa10 con 0x7f5e68101c30 2024-09-12T02:17:30.775 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.768+0000 7f5e6cb08640 1 --2- 172.21.15.16:0/2686600137 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e68101c30 0x7f5e68198330 secure :-1 s=READY pgs=2073 cs=0 l=1 rev1=1 crypto rx=0x7f5e5800e9f0 tx=0x7f5e5800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:30.775 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.768+0000 7f5e5e7fc640 1 -- 172.21.15.16:0/2686600137 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e5800cd80 con 0x7f5e68101c30 2024-09-12T02:17:30.776 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.769+0000 7f5e5e7fc640 1 -- 172.21.15.16:0/2686600137 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e58010910 con 0x7f5e68101c30 2024-09-12T02:17:30.776 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.769+0000 7f5e6f56d640 1 -- 172.21.15.16:0/2686600137 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e6819fd00 con 0x7f5e68101c30 2024-09-12T02:17:30.776 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.769+0000 7f5e5e7fc640 1 -- 172.21.15.16:0/2686600137 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e58018990 con 0x7f5e68101c30 2024-09-12T02:17:30.776 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.769+0000 7f5e6f56d640 1 -- 172.21.15.16:0/2686600137 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e681a0240 con 0x7f5e68101c30 2024-09-12T02:17:30.782 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.771+0000 7f5e723e8740 1 Processor -- start 2024-09-12T02:17:30.783 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.771+0000 7f5e723e8740 1 -- start start 2024-09-12T02:17:30.783 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.772+0000 7f5e723e8740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563d18a649a0 0x563d18a64da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:30.783 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.772+0000 7f5e723e8740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d18a652e0 0x563d18a69da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:30.783 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.772+0000 7f5e723e8740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d18a6a370 0x563d18a6c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:30.783 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.772+0000 7f5e723e8740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x563d189c82f0 con 0x563d18a6a370 2024-09-12T02:17:30.783 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.772+0000 7f5e723e8740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x563d18970270 con 0x563d18a652e0 2024-09-12T02:17:30.783 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.772+0000 7f5e723e8740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x563d189c42b0 con 0x563d18a649a0 2024-09-12T02:17:30.783 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.772+0000 7f5e6db0a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563d18a649a0 0x563d18a64da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:30.783 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.772+0000 7f5e6db0a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563d18a649a0 0x563d18a64da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:49198/0 (socket says 172.21.15.16:49198) 2024-09-12T02:17:30.784 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.772+0000 7f5e6db0a640 1 -- 172.21.15.16:0/732830094 learned_addr learned my addr 172.21.15.16:0/732830094 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:17:30.784 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.772+0000 7f5e6d309640 1 --2- 172.21.15.16:0/732830094 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d18a652e0 0x563d18a69da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:30.784 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.772+0000 7f5e5e7fc640 1 -- 172.21.15.16:0/2686600137 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5e58010ac0 con 0x7f5e68101c30 2024-09-12T02:17:30.784 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.772+0000 7f5e6db0a640 1 -- 172.21.15.16:0/732830094 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d18a652e0 msgr2=0x563d18a69da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:30.784 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.772+0000 7f5e6db0a640 1 --2- 172.21.15.16:0/732830094 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d18a652e0 0x563d18a69da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:30.784 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.772+0000 7f5e6db0a640 1 -- 172.21.15.16:0/732830094 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d18a6a370 msgr2=0x563d18a6c830 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-12T02:17:30.784 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.772+0000 7f5e6db0a640 1 --2- 172.21.15.16:0/732830094 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d18a6a370 0x563d18a6c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:30.784 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.772+0000 7f5e6db0a640 1 -- 172.21.15.16:0/732830094 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d1896f340 con 0x563d18a649a0 2024-09-12T02:17:30.784 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.773+0000 7f5e6db0a640 1 --2- 172.21.15.16:0/732830094 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563d18a649a0 0x563d18a64da0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f5e60010370 tx=0x7f5e60010830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=97277b56e58992bd server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:30.784 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.773+0000 7f5e5e7fc640 1 --2- 172.21.15.16:0/2686600137 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f5e540a2cb0 0x7f5e540a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:30.784 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.773+0000 7f5e6d309640 1 --2- 172.21.15.16:0/2686600137 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f5e540a2cb0 0x7f5e540a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:30.784 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.773+0000 7f5e537fe640 1 -- 172.21.15.16:0/732830094 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e60016070 con 0x563d18a649a0 2024-09-12T02:17:30.784 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.773+0000 7f5e5e7fc640 1 -- 172.21.15.16:0/2686600137 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f5e580cf210 con 0x7f5e68101c30 2024-09-12T02:17:30.785 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.773+0000 7f5e537fe640 1 -- 172.21.15.16:0/732830094 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e60004d20 con 0x563d18a649a0 2024-09-12T02:17:30.785 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.773+0000 7f5e537fe640 1 -- 172.21.15.16:0/732830094 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e60005480 con 0x563d18a649a0 2024-09-12T02:17:30.785 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.774+0000 7f5e6d309640 1 --2- 172.21.15.16:0/2686600137 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f5e540a2cb0 0x7f5e540a5170 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f5e64005ab0 tx=0x7f5e64005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:30.785 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.774+0000 7f5e723e8740 1 -- 172.21.15.16:0/732830094 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563d18a649a0 msgr2=0x563d18a64da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:30.785 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.774+0000 7f5e723e8740 1 --2- 172.21.15.16:0/732830094 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563d18a649a0 0x563d18a64da0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f5e60010370 tx=0x7f5e60010830 comp rx=0 tx=0).stop 2024-09-12T02:17:30.785 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.774+0000 7f5e723e8740 1 -- 172.21.15.16:0/732830094 shutdown_connections 2024-09-12T02:17:30.785 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.774+0000 7f5e723e8740 1 --2- 172.21.15.16:0/732830094 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d18a6a370 0x563d18a6c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:30.785 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.774+0000 7f5e723e8740 1 --2- 172.21.15.16:0/732830094 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d18a652e0 0x563d18a69da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:30.785 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.774+0000 7f5e723e8740 1 --2- 172.21.15.16:0/732830094 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563d18a649a0 0x563d18a64da0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:30.785 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.774+0000 7f5e723e8740 1 -- 172.21.15.16:0/732830094 >> 172.21.15.16:0/732830094 conn(0x563d18a60010 msgr2=0x563d18a60410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:30.785 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.774+0000 7f5e723e8740 1 -- 172.21.15.16:0/732830094 shutdown_connections 2024-09-12T02:17:30.785 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.774+0000 7f5e723e8740 1 -- 172.21.15.16:0/732830094 wait complete. 2024-09-12T02:17:30.786 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.774+0000 7f5e723e8740 1 -- start start 2024-09-12T02:17:30.786 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.774+0000 7f5e723e8740 1 Processor -- start 2024-09-12T02:17:30.786 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.775+0000 7f5e723e8740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d18a649a0 0x563d18a6ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:30.786 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.775+0000 7f5e723e8740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d18a652e0 0x563d18a6f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:30.786 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.775+0000 7f5e723e8740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563d18a6a370 0x563d18a76200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:30.786 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.775+0000 7f5e723e8740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x563d18a6d340 con 0x563d18a649a0 2024-09-12T02:17:30.786 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.775+0000 7f5e723e8740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x563d18951290 con 0x563d18a652e0 2024-09-12T02:17:30.787 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.775+0000 7f5e723e8740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x563d18a6d4c0 con 0x563d18a6a370 2024-09-12T02:17:30.787 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.775+0000 7f5e723e8740 10 client.0 _ll_register_callbacks cb 0x563d18982140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:17:30.787 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.775+0000 7f5e6d309640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d18a649a0 0x563d18a6ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:30.787 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.775+0000 7f5e6d309640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d18a649a0 0x563d18a6ec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50726/0 (socket says 172.21.15.16:50726) 2024-09-12T02:17:30.787 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.775+0000 7f5e6d309640 1 -- 172.21.15.16:0/3725955573 learned_addr learned my addr 172.21.15.16:0/3725955573 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:17:30.787 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.775+0000 7f5e6cb08640 1 --2- 172.21.15.16:0/3725955573 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d18a652e0 0x563d18a6f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:30.787 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.775+0000 7f5e6db0a640 1 --2- 172.21.15.16:0/3725955573 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563d18a6a370 0x563d18a76200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:30.787 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.775+0000 7f5e6d309640 1 -- 172.21.15.16:0/3725955573 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563d18a6a370 msgr2=0x563d18a76200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:30.787 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.775+0000 7f5e6d309640 1 --2- 172.21.15.16:0/3725955573 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563d18a6a370 0x563d18a76200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:30.787 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.775+0000 7f5e6d309640 1 -- 172.21.15.16:0/3725955573 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d18a652e0 msgr2=0x563d18a6f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:30.787 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.775+0000 7f5e6d309640 1 --2- 172.21.15.16:0/3725955573 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d18a652e0 0x563d18a6f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:30.787 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.775+0000 7f5e6d309640 1 -- 172.21.15.16:0/3725955573 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d186db210 con 0x563d18a649a0 2024-09-12T02:17:30.787 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.776+0000 7f5e6d309640 1 --2- 172.21.15.16:0/3725955573 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d18a649a0 0x563d18a6ec80 secure :-1 s=READY pgs=2074 cs=0 l=1 rev1=1 crypto rx=0x7f5e64000c00 tx=0x7f5e64009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5638d60581f40d21 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:30.788 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.776+0000 7f5e50ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:17:30.788 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.776+0000 7f5e50ff9640 1 -- 172.21.15.16:0/3725955573 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e6404d030 con 0x563d18a649a0 2024-09-12T02:17:30.788 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.776+0000 7f5e50ff9640 1 -- 172.21.15.16:0/3725955573 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f5e640053d0 con 0x563d18a649a0 2024-09-12T02:17:30.788 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.776+0000 7f5e723e8740 1 -- 172.21.15.16:0/3725955573 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5e60004d20 con 0x563d18a649a0 2024-09-12T02:17:30.788 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.776+0000 7f5e50ff9640 1 -- 172.21.15.16:0/3725955573 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5e64035040 con 0x563d18a649a0 2024-09-12T02:17:30.788 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.776+0000 7f5e723e8740 10 client.20667 fetch_fsmap learned FSMap version 25 2024-09-12T02:17:30.788 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.776+0000 7f5e723e8740 1 -- 172.21.15.16:0/3725955573 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x563d187a7620 con 0x563d18a649a0 2024-09-12T02:17:30.788 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.776+0000 7f5e50ff9640 1 -- 172.21.15.16:0/3725955573 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e60004d20 con 0x563d18a649a0 2024-09-12T02:17:30.788 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.776+0000 7f5e50ff9640 1 -- 172.21.15.16:0/3725955573 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f5e6403d680 con 0x563d18a649a0 2024-09-12T02:17:30.788 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.776+0000 7f5e723e8740 10 client.20667 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:17:30.788 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.776+0000 7f5e723e8740 10 client.20667 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:17:30.789 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.776+0000 7f5e723e8740 20 client.20667 populate_metadata read hostname 'smithi016' 2024-09-12T02:17:30.789 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.776+0000 7f5e723e8740 1 --2- 172.21.15.16:0/3725955573 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x563d18a7a8f0 0x563d18a7cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:30.789 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.776+0000 7f5e723e8740 4 client.20667 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:17:30.789 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.776+0000 7f5e723e8740 1 -- 172.21.15.16:0/3725955573 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563d189d5a20 con 0x563d18a7a8f0 2024-09-12T02:17:30.789 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.777+0000 7f5e6cb08640 1 --2- 172.21.15.16:0/3725955573 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x563d18a7a8f0 0x563d18a7cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:30.789 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.777+0000 7f5e6cb08640 1 --2- 172.21.15.16:0/3725955573 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x563d18a7a8f0 0x563d18a7cd00 crc :-1 s=READY pgs=461 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bddcb85654245c72 server_cookie=4b5cb922bb8b7f5f in_seq=0 out_seq=0 2024-09-12T02:17:30.789 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.777+0000 7f5e50ff9640 10 client.20667 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:17:30.789 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.778+0000 7f5e50ff9640 1 -- 172.21.15.16:0/3725955573 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563d189d5a20 con 0x563d18a7a8f0 2024-09-12T02:17:30.789 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.778+0000 7f5e50ff9640 10 client.20667 handle_command_reply: tid=0 2024-09-12T02:17:30.789 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.778+0000 7f5e723e8740 1 client.20667 shutdown 2024-09-12T02:17:30.789 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.778+0000 7f5e723e8740 1 -- 172.21.15.16:0/3725955573 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d18a649a0 msgr2=0x563d18a6ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:30.789 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.778+0000 7f5e723e8740 1 --2- 172.21.15.16:0/3725955573 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d18a649a0 0x563d18a6ec80 secure :-1 s=READY pgs=2074 cs=0 l=1 rev1=1 crypto rx=0x7f5e64000c00 tx=0x7f5e64009260 comp rx=0 tx=0).stop 2024-09-12T02:17:30.789 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.778+0000 7f5e723e8740 1 -- 172.21.15.16:0/3725955573 shutdown_connections 2024-09-12T02:17:30.790 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.778+0000 7f5e723e8740 1 --2- 172.21.15.16:0/3725955573 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x563d18a7a8f0 0x563d18a7cd00 crc :-1 s=READY pgs=461 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:30.790 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.778+0000 7f5e723e8740 1 --2- 172.21.15.16:0/3725955573 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563d18a6a370 0x563d18a76200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:30.790 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.778+0000 7f5e50ff9640 0 client.20667 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:17:30.790 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.778+0000 7f5e723e8740 1 --2- 172.21.15.16:0/3725955573 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d18a652e0 0x563d18a6f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:30.790 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.778+0000 7f5e723e8740 1 --2- 172.21.15.16:0/3725955573 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d18a649a0 0x563d18a6ec80 unknown :-1 s=CLOSED pgs=2074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:30.790 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.778+0000 7f5e723e8740 1 -- 172.21.15.16:0/3725955573 >> 172.21.15.16:0/3725955573 conn(0x563d18a60010 msgr2=0x563d18a61820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:30.790 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.778+0000 7f5e723e8740 1 -- 172.21.15.16:0/3725955573 shutdown_connections 2024-09-12T02:17:30.790 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.778+0000 7f5e723e8740 1 -- 172.21.15.16:0/3725955573 wait complete. 2024-09-12T02:17:30.790 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.778+0000 7f5e723e8740 20 client.20667 trim_cache size 0 max 16384 2024-09-12T02:17:30.798 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.792+0000 7f5e723e8740 1 Processor -- start 2024-09-12T02:17:30.798 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.792+0000 7f5e723e8740 1 -- start start 2024-09-12T02:17:30.798 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.793+0000 7f5e723e8740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d18a649a0 0x563d18a64d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:30.798 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.793+0000 7f5e723e8740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d18a652e0 0x563d18a6f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:30.798 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.793+0000 7f5e723e8740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563d18a6a370 0x563d18a761c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:30.798 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.793+0000 7f5e723e8740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5e681ab730 con 0x563d18a652e0 2024-09-12T02:17:30.798 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.793+0000 7f5e723e8740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5e681a9000 con 0x563d18a649a0 2024-09-12T02:17:30.798 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.793+0000 7f5e723e8740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5e681a6e50 con 0x563d18a6a370 2024-09-12T02:17:30.798 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.793+0000 7f5e6cb08640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d18a652e0 0x563d18a6f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:30.798 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.793+0000 7f5e6cb08640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d18a652e0 0x563d18a6f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50742/0 (socket says 172.21.15.16:50742) 2024-09-12T02:17:30.799 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.793+0000 7f5e6cb08640 1 -- 172.21.15.16:0/3193837322 learned_addr learned my addr 172.21.15.16:0/3193837322 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:17:30.799 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.793+0000 7f5e6d309640 1 --2- 172.21.15.16:0/3193837322 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d18a649a0 0x563d18a64d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:30.799 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.793+0000 7f5e6db0a640 1 --2- 172.21.15.16:0/3193837322 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563d18a6a370 0x563d18a761c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:30.799 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.793+0000 7f5e6cb08640 1 -- 172.21.15.16:0/3193837322 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563d18a6a370 msgr2=0x563d18a761c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:30.799 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.793+0000 7f5e6cb08640 1 --2- 172.21.15.16:0/3193837322 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563d18a6a370 0x563d18a761c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:30.799 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.793+0000 7f5e6cb08640 1 -- 172.21.15.16:0/3193837322 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d18a649a0 msgr2=0x563d18a64d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:30.799 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.793+0000 7f5e6cb08640 1 --2- 172.21.15.16:0/3193837322 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d18a649a0 0x563d18a64d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:30.799 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.793+0000 7f5e6cb08640 1 -- 172.21.15.16:0/3193837322 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d18a5fe30 con 0x563d18a652e0 2024-09-12T02:17:30.799 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.793+0000 7f5e6cb08640 1 --2- 172.21.15.16:0/3193837322 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d18a652e0 0x563d18a6f0a0 secure :-1 s=READY pgs=2075 cs=0 l=1 rev1=1 crypto rx=0x7f5e5809b8c0 tx=0x7f5e5801a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb1e8ade17e7817b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:30.799 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.793+0000 7f5e52ffd640 1 -- 172.21.15.16:0/3193837322 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e580b3070 con 0x563d18a652e0 2024-09-12T02:17:30.799 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.793+0000 7f5e52ffd640 1 -- 172.21.15.16:0/3193837322 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e580ccec0 con 0x563d18a652e0 2024-09-12T02:17:30.799 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.793+0000 7f5e52ffd640 1 -- 172.21.15.16:0/3193837322 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e58018e20 con 0x563d18a652e0 2024-09-12T02:17:30.799 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.794+0000 7f5e723e8740 1 -- 172.21.15.16:0/3193837322 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d18a652e0 msgr2=0x563d18a6f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:30.800 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.794+0000 7f5e723e8740 1 --2- 172.21.15.16:0/3193837322 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d18a652e0 0x563d18a6f0a0 secure :-1 s=READY pgs=2075 cs=0 l=1 rev1=1 crypto rx=0x7f5e5809b8c0 tx=0x7f5e5801a040 comp rx=0 tx=0).stop 2024-09-12T02:17:30.800 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.794+0000 7f5e723e8740 1 -- 172.21.15.16:0/3193837322 shutdown_connections 2024-09-12T02:17:30.800 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.794+0000 7f5e723e8740 1 --2- 172.21.15.16:0/3193837322 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563d18a6a370 0x563d18a761c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:30.800 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.794+0000 7f5e723e8740 1 --2- 172.21.15.16:0/3193837322 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d18a652e0 0x563d18a6f0a0 unknown :-1 s=CLOSED pgs=2075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:30.800 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.794+0000 7f5e723e8740 1 --2- 172.21.15.16:0/3193837322 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d18a649a0 0x563d18a64d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:30.800 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.794+0000 7f5e723e8740 1 -- 172.21.15.16:0/3193837322 >> 172.21.15.16:0/3193837322 conn(0x563d18a60010 msgr2=0x563d18a5f1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:30.800 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.794+0000 7f5e723e8740 1 -- 172.21.15.16:0/3193837322 shutdown_connections 2024-09-12T02:17:30.800 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.794+0000 7f5e723e8740 1 -- 172.21.15.16:0/3193837322 wait complete. 2024-09-12T02:17:30.800 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.795+0000 7f5e723e8740 1 -- start start 2024-09-12T02:17:30.800 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.795+0000 7f5e723e8740 1 Processor -- start 2024-09-12T02:17:30.800 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.795+0000 7f5e723e8740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d18a649a0 0x563d18a73ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:30.800 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.795+0000 7f5e723e8740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d18a652e0 0x563d18a74010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:30.800 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.795+0000 7f5e723e8740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563d18a6a370 0x563d18a74fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:30.801 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.795+0000 7f5e723e8740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5e681a74c0 con 0x563d18a649a0 2024-09-12T02:17:30.801 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.795+0000 7f5e723e8740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5e681a8930 con 0x563d18a652e0 2024-09-12T02:17:30.801 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.795+0000 7f5e723e8740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5e681a7b90 con 0x563d18a6a370 2024-09-12T02:17:30.801 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.795+0000 7f5e723e8740 10 client.0 _ll_register_callbacks cb 0x7f5e64005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:17:30.801 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.795+0000 7f5e6d309640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d18a649a0 0x563d18a73ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:30.801 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.795+0000 7f5e6d309640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d18a649a0 0x563d18a73ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50756/0 (socket says 172.21.15.16:50756) 2024-09-12T02:17:30.801 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.795+0000 7f5e6d309640 1 -- 172.21.15.16:0/843131950 learned_addr learned my addr 172.21.15.16:0/843131950 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:17:30.801 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.795+0000 7f5e6cb08640 1 --2- 172.21.15.16:0/843131950 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d18a652e0 0x563d18a74010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:30.801 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.795+0000 7f5e6db0a640 1 --2- 172.21.15.16:0/843131950 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563d18a6a370 0x563d18a74fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:30.802 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.796+0000 7f5e6d309640 1 -- 172.21.15.16:0/843131950 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563d18a6a370 msgr2=0x563d18a74fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:30.802 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.796+0000 7f5e6d309640 1 --2- 172.21.15.16:0/843131950 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563d18a6a370 0x563d18a74fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:30.802 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.796+0000 7f5e6d309640 1 -- 172.21.15.16:0/843131950 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d18a652e0 msgr2=0x563d18a74010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:30.802 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.796+0000 7f5e6d309640 1 --2- 172.21.15.16:0/843131950 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d18a652e0 0x563d18a74010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:30.802 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.796+0000 7f5e6d309640 1 -- 172.21.15.16:0/843131950 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d18a755e0 con 0x563d18a649a0 2024-09-12T02:17:30.802 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.796+0000 7f5e6cb08640 1 --2- 172.21.15.16:0/843131950 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d18a652e0 0x563d18a74010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:17:30.802 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.796+0000 7f5e6d309640 1 --2- 172.21.15.16:0/843131950 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d18a649a0 0x563d18a73ad0 secure :-1 s=READY pgs=2076 cs=0 l=1 rev1=1 crypto rx=0x7f5e6402a930 tx=0x7f5e64049440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc55d0ed5abbc21f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:30.802 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.796+0000 7f5e50ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:17:30.802 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.796+0000 7f5e50ff9640 1 -- 172.21.15.16:0/843131950 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e6404d040 con 0x563d18a649a0 2024-09-12T02:17:30.802 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.796+0000 7f5e50ff9640 1 -- 172.21.15.16:0/843131950 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f5e64004040 con 0x563d18a649a0 2024-09-12T02:17:30.802 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.796+0000 7f5e723e8740 1 -- 172.21.15.16:0/843131950 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5e580ccec0 con 0x563d18a649a0 2024-09-12T02:17:30.803 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.796+0000 7f5e50ff9640 1 -- 172.21.15.16:0/843131950 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5e64004240 con 0x563d18a649a0 2024-09-12T02:17:30.803 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.796+0000 7f5e723e8740 10 client.20673 fetch_fsmap learned FSMap version 25 2024-09-12T02:17:30.803 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.796+0000 7f5e723e8740 1 -- 172.21.15.16:0/843131950 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x563d18ae5860 con 0x563d18a649a0 2024-09-12T02:17:30.803 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.796+0000 7f5e50ff9640 1 -- 172.21.15.16:0/843131950 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e580ccec0 con 0x563d18a649a0 2024-09-12T02:17:30.803 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.797+0000 7f5e50ff9640 1 -- 172.21.15.16:0/843131950 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f5e64066030 con 0x563d18a649a0 2024-09-12T02:17:30.803 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.797+0000 7f5e723e8740 10 client.20673 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:17:30.803 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.797+0000 7f5e723e8740 10 client.20673 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:17:30.803 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.797+0000 7f5e723e8740 20 client.20673 populate_metadata read hostname 'smithi016' 2024-09-12T02:17:30.803 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.797+0000 7f5e723e8740 1 --2- 172.21.15.16:0/843131950 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x563d18ae6210 0x563d18ae8630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:30.803 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.797+0000 7f5e723e8740 4 client.20673 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:17:30.803 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.797+0000 7f5e723e8740 1 -- 172.21.15.16:0/843131950 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563d189cfd70 con 0x563d18ae6210 2024-09-12T02:17:30.803 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.797+0000 7f5e6cb08640 1 --2- 172.21.15.16:0/843131950 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x563d18ae6210 0x563d18ae8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:30.803 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.798+0000 7f5e6cb08640 1 --2- 172.21.15.16:0/843131950 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x563d18ae6210 0x563d18ae8630 crc :-1 s=READY pgs=462 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e54e8982f9158637 server_cookie=3006a51fc86bcfd0 in_seq=0 out_seq=0 2024-09-12T02:17:30.804 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.798+0000 7f5e50ff9640 10 client.20673 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:17:30.813 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.812+0000 7f5e50ff9640 1 -- 172.21.15.16:0/843131950 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563d189cfd70 con 0x563d18ae6210 2024-09-12T02:17:30.813 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.813+0000 7f5e50ff9640 10 client.20673 handle_command_reply: tid=0 2024-09-12T02:17:30.814 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.813+0000 7f5e723e8740 1 client.20673 shutdown 2024-09-12T02:17:30.814 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.813+0000 7f5e723e8740 1 -- 172.21.15.16:0/843131950 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d18a649a0 msgr2=0x563d18a73ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:30.814 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.813+0000 7f5e723e8740 1 --2- 172.21.15.16:0/843131950 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d18a649a0 0x563d18a73ad0 secure :-1 s=READY pgs=2076 cs=0 l=1 rev1=1 crypto rx=0x7f5e6402a930 tx=0x7f5e64049440 comp rx=0 tx=0).stop 2024-09-12T02:17:30.814 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.813+0000 7f5e723e8740 1 -- 172.21.15.16:0/843131950 shutdown_connections 2024-09-12T02:17:30.814 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.813+0000 7f5e723e8740 1 --2- 172.21.15.16:0/843131950 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x563d18ae6210 0x563d18ae8630 crc :-1 s=READY pgs=462 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:30.814 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.813+0000 7f5e723e8740 1 --2- 172.21.15.16:0/843131950 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563d18a6a370 0x563d18a74fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:30.814 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.813+0000 7f5e723e8740 1 --2- 172.21.15.16:0/843131950 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d18a652e0 0x563d18a74010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:30.814 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.813+0000 7f5e50ff9640 0 client.20673 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:17:30.814 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.813+0000 7f5e723e8740 1 --2- 172.21.15.16:0/843131950 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d18a649a0 0x563d18a73ad0 unknown :-1 s=CLOSED pgs=2076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:30.814 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.813+0000 7f5e723e8740 1 -- 172.21.15.16:0/843131950 >> 172.21.15.16:0/843131950 conn(0x563d18a60010 msgr2=0x563d18a603f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:30.814 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.813+0000 7f5e723e8740 1 -- 172.21.15.16:0/843131950 shutdown_connections 2024-09-12T02:17:30.814 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.813+0000 7f5e723e8740 1 -- 172.21.15.16:0/843131950 wait complete. 2024-09-12T02:17:30.815 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.813+0000 7f5e723e8740 20 client.20673 trim_cache size 0 max 16384 2024-09-12T02:17:30.815 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-09-12T02:17:30.815 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-09-12T02:17:30.815 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-09-12T02:17:30.815 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-09-12T02:17:30.816 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.813+0000 7f5e6f56d640 1 -- 172.21.15.16:0/2686600137 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f5e540a2cb0 msgr2=0x7f5e540a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:30.816 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.813+0000 7f5e6f56d640 1 --2- 172.21.15.16:0/2686600137 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f5e540a2cb0 0x7f5e540a5170 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f5e64005ab0 tx=0x7f5e64005fd0 comp rx=0 tx=0).stop 2024-09-12T02:17:30.816 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.813+0000 7f5e6f56d640 1 -- 172.21.15.16:0/2686600137 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e68101c30 msgr2=0x7f5e68198330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:30.816 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.814+0000 7f5e6f56d640 1 --2- 172.21.15.16:0/2686600137 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e68101c30 0x7f5e68198330 secure :-1 s=READY pgs=2073 cs=0 l=1 rev1=1 crypto rx=0x7f5e5800e9f0 tx=0x7f5e5800eeb0 comp rx=0 tx=0).stop 2024-09-12T02:17:30.816 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.814+0000 7f5e6f56d640 1 -- 172.21.15.16:0/2686600137 shutdown_connections 2024-09-12T02:17:30.816 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.814+0000 7f5e6f56d640 1 --2- 172.21.15.16:0/2686600137 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f5e540a2cb0 0x7f5e540a5170 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:30.816 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.814+0000 7f5e6f56d640 1 --2- 172.21.15.16:0/2686600137 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5e681025f0 0x7f5e6819f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:30.816 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.814+0000 7f5e6f56d640 1 --2- 172.21.15.16:0/2686600137 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e68101c30 0x7f5e68198330 unknown :-1 s=CLOSED pgs=2073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:30.816 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.814+0000 7f5e6f56d640 1 --2- 172.21.15.16:0/2686600137 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5e68100a40 0x7f5e68197df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:30.817 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.814+0000 7f5e6f56d640 1 -- 172.21.15.16:0/2686600137 >> 172.21.15.16:0/2686600137 conn(0x7f5e680fc1e0 msgr2=0x7f5e680698a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:30.817 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.814+0000 7f5e6f56d640 1 -- 172.21.15.16:0/2686600137 shutdown_connections 2024-09-12T02:17:30.817 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:30.814+0000 7f5e6f56d640 1 -- 172.21.15.16:0/2686600137 wait complete. 2024-09-12T02:17:31.933 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:17:31 smithi016 ceph-mon[30947]: pgmap v803: 97 pgs: 97 active+clean; 4.0 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-09-12T02:17:32.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:17:31 smithi184 ceph-mon[35593]: pgmap v803: 97 pgs: 97 active+clean; 4.0 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-09-12T02:17:32.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:17:31 smithi033 ceph-mon[35332]: pgmap v803: 97 pgs: 97 active+clean; 4.0 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-09-12T02:17:33.620 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T02:17:33.894 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:17:33.933 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:17:33 smithi016 ceph-mon[30947]: pgmap v804: 97 pgs: 97 active+clean; 4.1 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 20 KiB/s wr, 14 op/s 2024-09-12T02:17:34.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:17:33 smithi184 ceph-mon[35593]: pgmap v804: 97 pgs: 97 active+clean; 4.1 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 20 KiB/s wr, 14 op/s 2024-09-12T02:17:34.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:17:33 smithi033 ceph-mon[35332]: pgmap v804: 97 pgs: 97 active+clean; 4.1 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 20 KiB/s wr, 14 op/s 2024-09-12T02:17:34.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.468+0000 7f85b74a3640 1 -- 172.21.15.16:0/2912547577 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85b0102dd0 msgr2=0x7f85b01031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:34.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.468+0000 7f85b74a3640 1 --2- 172.21.15.16:0/2912547577 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85b0102dd0 0x7f85b01031b0 secure :-1 s=READY pgs=2077 cs=0 l=1 rev1=1 crypto rx=0x7f85ac009900 tx=0x7f85ac02a480 comp rx=0 tx=0).stop 2024-09-12T02:17:34.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.468+0000 7f85b74a3640 1 -- 172.21.15.16:0/2912547577 shutdown_connections 2024-09-12T02:17:34.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.468+0000 7f85b74a3640 1 --2- 172.21.15.16:0/2912547577 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f85b010c170 0x7f85b010e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:34.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.468+0000 7f85b74a3640 1 --2- 172.21.15.16:0/2912547577 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85b01036f0 0x7f85b010bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:34.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.468+0000 7f85b74a3640 1 --2- 172.21.15.16:0/2912547577 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85b0102dd0 0x7f85b01031b0 unknown :-1 s=CLOSED pgs=2077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:34.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.468+0000 7f85b74a3640 1 -- 172.21.15.16:0/2912547577 >> 172.21.15.16:0/2912547577 conn(0x7f85b00fc790 msgr2=0x7f85b00febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:34.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.468+0000 7f85b74a3640 1 -- 172.21.15.16:0/2912547577 shutdown_connections 2024-09-12T02:17:34.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.468+0000 7f85b74a3640 1 -- 172.21.15.16:0/2912547577 wait complete. 2024-09-12T02:17:34.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.469+0000 7f85b74a3640 1 Processor -- start 2024-09-12T02:17:34.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.469+0000 7f85b74a3640 1 -- start start 2024-09-12T02:17:34.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.469+0000 7f85b74a3640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85b0102dd0 0x7f85b019a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:34.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.469+0000 7f85b74a3640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f85b01036f0 0x7f85b019ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:34.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.469+0000 7f85b74a3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85b010c170 0x7f85b019efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:34.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.469+0000 7f85b74a3640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f85b0111fa0 con 0x7f85b010c170 2024-09-12T02:17:34.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.469+0000 7f85b74a3640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f85b0111e20 con 0x7f85b0102dd0 2024-09-12T02:17:34.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.469+0000 7f85b74a3640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f85b0112120 con 0x7f85b01036f0 2024-09-12T02:17:34.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.469+0000 7f85b5a40640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85b010c170 0x7f85b019efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:34.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.469+0000 7f85b5a40640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85b010c170 0x7f85b019efe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53730/0 (socket says 172.21.15.16:53730) 2024-09-12T02:17:34.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.469+0000 7f85b5a40640 1 -- 172.21.15.16:0/500779859 learned_addr learned my addr 172.21.15.16:0/500779859 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:17:34.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.469+0000 7f85b4a3e640 1 --2- 172.21.15.16:0/500779859 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f85b01036f0 0x7f85b019ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:34.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.469+0000 7f85b523f640 1 --2- 172.21.15.16:0/500779859 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85b0102dd0 0x7f85b019a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:34.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.470+0000 7f85b5a40640 1 -- 172.21.15.16:0/500779859 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f85b01036f0 msgr2=0x7f85b019ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:34.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.470+0000 7f85b5a40640 1 --2- 172.21.15.16:0/500779859 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f85b01036f0 0x7f85b019ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:34.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.470+0000 7f85b5a40640 1 -- 172.21.15.16:0/500779859 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85b0102dd0 msgr2=0x7f85b019a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:34.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.470+0000 7f85b5a40640 1 --2- 172.21.15.16:0/500779859 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85b0102dd0 0x7f85b019a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:34.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.470+0000 7f85b5a40640 1 -- 172.21.15.16:0/500779859 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85b019f6c0 con 0x7f85b010c170 2024-09-12T02:17:34.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.470+0000 7f85b5a40640 1 --2- 172.21.15.16:0/500779859 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85b010c170 0x7f85b019efe0 secure :-1 s=READY pgs=2078 cs=0 l=1 rev1=1 crypto rx=0x7f85a800b970 tx=0x7f85a800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:34.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.470+0000 7f85a67fc640 1 -- 172.21.15.16:0/500779859 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a8019070 con 0x7f85b010c170 2024-09-12T02:17:34.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.470+0000 7f85a67fc640 1 -- 172.21.15.16:0/500779859 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85a800cb40 con 0x7f85b010c170 2024-09-12T02:17:34.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.470+0000 7f85b74a3640 1 -- 172.21.15.16:0/500779859 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85b019f9b0 con 0x7f85b010c170 2024-09-12T02:17:34.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.470+0000 7f85a67fc640 1 -- 172.21.15.16:0/500779859 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a8007650 con 0x7f85b010c170 2024-09-12T02:17:34.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.470+0000 7f85b74a3640 1 -- 172.21.15.16:0/500779859 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85b01a7290 con 0x7f85b010c170 2024-09-12T02:17:34.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.472+0000 7f85a67fc640 1 -- 172.21.15.16:0/500779859 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f85a801c070 con 0x7f85b010c170 2024-09-12T02:17:34.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.473+0000 7f85b987c740 1 Processor -- start 2024-09-12T02:17:34.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.473+0000 7f85b987c740 1 -- start start 2024-09-12T02:17:34.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.473+0000 7f85b987c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582c4bfa4a0 0x5582c4bfa880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:34.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.473+0000 7f85b987c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5582c4bfadc0 0x5582c4c05140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:34.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.473+0000 7f85b987c740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5582c4c05680 0x5582c4c07a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:34.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.473+0000 7f85b987c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5582c4b602e0 con 0x5582c4bfa4a0 2024-09-12T02:17:34.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.473+0000 7f85b987c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5582c4b07ce0 con 0x5582c4bfadc0 2024-09-12T02:17:34.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.473+0000 7f85b987c740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5582c4b5c2a0 con 0x5582c4c05680 2024-09-12T02:17:34.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.473+0000 7f85a67fc640 1 --2- 172.21.15.16:0/500779859 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f859c0a2b80 0x7f859c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:34.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.473+0000 7f85b4a3e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582c4bfa4a0 0x5582c4bfa880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:34.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.473+0000 7f85b4a3e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582c4bfa4a0 0x5582c4bfa880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53732/0 (socket says 172.21.15.16:53732) 2024-09-12T02:17:34.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.473+0000 7f85b4a3e640 1 -- 172.21.15.16:0/4237595154 learned_addr learned my addr 172.21.15.16:0/4237595154 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:17:34.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.473+0000 7f85b523f640 1 --2- 172.21.15.16:0/4237595154 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5582c4bfadc0 0x5582c4c05140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:34.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.473+0000 7f85b5a40640 1 --2- 172.21.15.16:0/500779859 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f859c0a2b80 0x7f859c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:34.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.474+0000 7f85a67fc640 1 -- 172.21.15.16:0/500779859 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f85a80ce5b0 con 0x7f85b010c170 2024-09-12T02:17:34.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.474+0000 7f85b4a3e640 1 --2- 172.21.15.16:0/4237595154 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5582c4c05680 0x5582c4c07a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:34.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.474+0000 7f85b4a3e640 1 -- 172.21.15.16:0/4237595154 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5582c4c05680 msgr2=0x5582c4c07a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:34.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.474+0000 7f85b4a3e640 1 --2- 172.21.15.16:0/4237595154 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5582c4c05680 0x5582c4c07a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:34.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.474+0000 7f85b4a3e640 1 -- 172.21.15.16:0/4237595154 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5582c4bfadc0 msgr2=0x5582c4c05140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:34.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.474+0000 7f85b4a3e640 1 --2- 172.21.15.16:0/4237595154 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5582c4bfadc0 0x5582c4c05140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:34.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.474+0000 7f85b4a3e640 1 -- 172.21.15.16:0/4237595154 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5582c4aec440 con 0x5582c4bfa4a0 2024-09-12T02:17:34.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.474+0000 7f85b523f640 1 --2- 172.21.15.16:0/4237595154 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5582c4bfadc0 0x5582c4c05140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:17:34.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.474+0000 7f85b4a3e640 1 --2- 172.21.15.16:0/4237595154 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582c4bfa4a0 0x5582c4bfa880 secure :-1 s=READY pgs=2079 cs=0 l=1 rev1=1 crypto rx=0x7f85a00058a0 tx=0x7f85a000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f96cd6bccb70dde server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:34.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.474+0000 7f85b5a40640 1 --2- 172.21.15.16:0/500779859 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f859c0a2b80 0x7f859c0a5040 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f85a80973c0 tx=0x7f85a8014d00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:34.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.474+0000 7f859b7fe640 1 -- 172.21.15.16:0/4237595154 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a0004030 con 0x5582c4bfa4a0 2024-09-12T02:17:34.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.474+0000 7f859b7fe640 1 -- 172.21.15.16:0/4237595154 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85a00041d0 con 0x5582c4bfa4a0 2024-09-12T02:17:34.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.474+0000 7f859b7fe640 1 -- 172.21.15.16:0/4237595154 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a00106c0 con 0x5582c4bfa4a0 2024-09-12T02:17:34.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.475+0000 7f85b987c740 1 -- 172.21.15.16:0/4237595154 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582c4bfa4a0 msgr2=0x5582c4bfa880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:34.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.475+0000 7f85b987c740 1 --2- 172.21.15.16:0/4237595154 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582c4bfa4a0 0x5582c4bfa880 secure :-1 s=READY pgs=2079 cs=0 l=1 rev1=1 crypto rx=0x7f85a00058a0 tx=0x7f85a000b840 comp rx=0 tx=0).stop 2024-09-12T02:17:34.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.475+0000 7f85b987c740 1 -- 172.21.15.16:0/4237595154 shutdown_connections 2024-09-12T02:17:34.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.475+0000 7f85b987c740 1 --2- 172.21.15.16:0/4237595154 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5582c4c05680 0x5582c4c07a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:34.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.475+0000 7f85b987c740 1 --2- 172.21.15.16:0/4237595154 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5582c4bfadc0 0x5582c4c05140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:34.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.475+0000 7f85b987c740 1 --2- 172.21.15.16:0/4237595154 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582c4bfa4a0 0x5582c4bfa880 unknown :-1 s=CLOSED pgs=2079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:34.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.475+0000 7f85b987c740 1 -- 172.21.15.16:0/4237595154 >> 172.21.15.16:0/4237595154 conn(0x5582c4bf6a30 msgr2=0x5582c4bf6e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:34.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.475+0000 7f85b987c740 1 -- 172.21.15.16:0/4237595154 shutdown_connections 2024-09-12T02:17:34.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.475+0000 7f85b987c740 1 -- 172.21.15.16:0/4237595154 wait complete. 2024-09-12T02:17:34.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.475+0000 7f85b987c740 1 -- start start 2024-09-12T02:17:34.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.475+0000 7f85b987c740 1 Processor -- start 2024-09-12T02:17:34.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.476+0000 7f85b987c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5582c4bfa4a0 0x5582c4c04f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:34.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.476+0000 7f85b987c740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5582c4bfadc0 0x5582c4c02220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:34.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.476+0000 7f85b987c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582c4c05680 0x5582c4c02760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:34.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.476+0000 7f85b987c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5582c4c08250 con 0x5582c4c05680 2024-09-12T02:17:34.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.476+0000 7f85b987c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5582c4b1b050 con 0x5582c4bfa4a0 2024-09-12T02:17:34.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.476+0000 7f85b987c740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5582c4c083d0 con 0x5582c4bfadc0 2024-09-12T02:17:34.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.476+0000 7f85b987c740 10 client.0 _ll_register_callbacks cb 0x5582c4b5b160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:17:34.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.476+0000 7f85b4a3e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582c4c05680 0x5582c4c02760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:34.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.476+0000 7f85b4a3e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582c4c05680 0x5582c4c02760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53734/0 (socket says 172.21.15.16:53734) 2024-09-12T02:17:34.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.476+0000 7f85b4a3e640 1 -- 172.21.15.16:0/1867181374 learned_addr learned my addr 172.21.15.16:0/1867181374 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:17:34.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.476+0000 7f85b4a3e640 1 --2- 172.21.15.16:0/1867181374 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5582c4bfa4a0 0x5582c4c04f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:34.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.476+0000 7f85b523f640 1 --2- 172.21.15.16:0/1867181374 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5582c4bfadc0 0x5582c4c02220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:34.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.477+0000 7f85b4a3e640 1 -- 172.21.15.16:0/1867181374 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5582c4bfadc0 msgr2=0x5582c4c02220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:34.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.477+0000 7f85b4a3e640 1 --2- 172.21.15.16:0/1867181374 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5582c4bfadc0 0x5582c4c02220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:34.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.477+0000 7f85b4a3e640 1 -- 172.21.15.16:0/1867181374 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5582c4bfa4a0 msgr2=0x5582c4c04f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:34.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.477+0000 7f85b4a3e640 1 --2- 172.21.15.16:0/1867181374 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5582c4bfa4a0 0x5582c4c04f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:34.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.477+0000 7f85b4a3e640 1 -- 172.21.15.16:0/1867181374 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5582c48d13c0 con 0x5582c4c05680 2024-09-12T02:17:34.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.477+0000 7f85b4a3e640 1 --2- 172.21.15.16:0/1867181374 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582c4c05680 0x5582c4c02760 secure :-1 s=READY pgs=2080 cs=0 l=1 rev1=1 crypto rx=0x7f85a0003d30 tx=0x7f85a0003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a887e360c0a4d574 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:34.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.477+0000 7f8598ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:17:34.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.477+0000 7f8598ff9640 1 -- 172.21.15.16:0/1867181374 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a0028040 con 0x5582c4c05680 2024-09-12T02:17:34.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.477+0000 7f8598ff9640 1 -- 172.21.15.16:0/1867181374 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f85a0005020 con 0x5582c4c05680 2024-09-12T02:17:34.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.477+0000 7f85b987c740 1 -- 172.21.15.16:0/1867181374 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85a00041d0 con 0x5582c4c05680 2024-09-12T02:17:34.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.477+0000 7f8598ff9640 1 -- 172.21.15.16:0/1867181374 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f85a0005220 con 0x5582c4c05680 2024-09-12T02:17:34.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.477+0000 7f85b987c740 10 client.20685 fetch_fsmap learned FSMap version 25 2024-09-12T02:17:34.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.477+0000 7f85b987c740 1 -- 172.21.15.16:0/1867181374 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5582c4c13bc0 con 0x5582c4c05680 2024-09-12T02:17:34.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.477+0000 7f8598ff9640 1 -- 172.21.15.16:0/1867181374 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85a00041d0 con 0x5582c4c05680 2024-09-12T02:17:34.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.477+0000 7f8598ff9640 1 -- 172.21.15.16:0/1867181374 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f85a0004ba0 con 0x5582c4c05680 2024-09-12T02:17:34.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.478+0000 7f85b987c740 10 client.20685 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:17:34.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.478+0000 7f85b987c740 10 client.20685 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:17:34.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.478+0000 7f85b987c740 20 client.20685 populate_metadata read hostname 'smithi016' 2024-09-12T02:17:34.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.478+0000 7f85b987c740 1 --2- 172.21.15.16:0/1867181374 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5582c4c14f10 0x5582c4c17300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:34.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.478+0000 7f85b987c740 4 client.20685 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:17:34.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.478+0000 7f85b987c740 1 -- 172.21.15.16:0/1867181374 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5582c4b6d850 con 0x5582c4c14f10 2024-09-12T02:17:34.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.478+0000 7f85b523f640 1 --2- 172.21.15.16:0/1867181374 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5582c4c14f10 0x5582c4c17300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:34.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.478+0000 7f85b523f640 1 --2- 172.21.15.16:0/1867181374 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5582c4c14f10 0x5582c4c17300 crc :-1 s=READY pgs=463 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a378f312a55b6fb8 server_cookie=7b750b28948cfbbc in_seq=0 out_seq=0 2024-09-12T02:17:34.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.478+0000 7f8598ff9640 10 client.20685 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:17:34.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.479+0000 7f8598ff9640 1 -- 172.21.15.16:0/1867181374 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5582c4b6d850 con 0x5582c4c14f10 2024-09-12T02:17:34.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.479+0000 7f8598ff9640 10 client.20685 handle_command_reply: tid=0 2024-09-12T02:17:34.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.479+0000 7f85b987c740 1 client.20685 shutdown 2024-09-12T02:17:34.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.479+0000 7f85b987c740 1 -- 172.21.15.16:0/1867181374 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582c4c05680 msgr2=0x5582c4c02760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:34.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.479+0000 7f85b987c740 1 --2- 172.21.15.16:0/1867181374 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582c4c05680 0x5582c4c02760 secure :-1 s=READY pgs=2080 cs=0 l=1 rev1=1 crypto rx=0x7f85a0003d30 tx=0x7f85a0003b90 comp rx=0 tx=0).stop 2024-09-12T02:17:34.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.479+0000 7f85b987c740 1 -- 172.21.15.16:0/1867181374 shutdown_connections 2024-09-12T02:17:34.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.479+0000 7f85b987c740 1 --2- 172.21.15.16:0/1867181374 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5582c4c14f10 0x5582c4c17300 crc :-1 s=READY pgs=463 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:34.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.479+0000 7f85b987c740 1 --2- 172.21.15.16:0/1867181374 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582c4c05680 0x5582c4c02760 unknown :-1 s=CLOSED pgs=2080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:34.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.479+0000 7f85b987c740 1 --2- 172.21.15.16:0/1867181374 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5582c4bfadc0 0x5582c4c02220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:34.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.479+0000 7f8598ff9640 0 client.20685 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:17:34.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.479+0000 7f85b987c740 1 --2- 172.21.15.16:0/1867181374 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5582c4bfa4a0 0x5582c4c04f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:34.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.479+0000 7f85b987c740 1 -- 172.21.15.16:0/1867181374 >> 172.21.15.16:0/1867181374 conn(0x5582c4bf6a30 msgr2=0x5582c4bfc760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:34.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.479+0000 7f85b987c740 1 -- 172.21.15.16:0/1867181374 shutdown_connections 2024-09-12T02:17:34.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.479+0000 7f85b987c740 1 -- 172.21.15.16:0/1867181374 wait complete. 2024-09-12T02:17:34.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.479+0000 7f85b987c740 20 client.20685 trim_cache size 0 max 16384 2024-09-12T02:17:34.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.497+0000 7f85b987c740 1 Processor -- start 2024-09-12T02:17:34.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.497+0000 7f85b987c740 1 -- start start 2024-09-12T02:17:34.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.497+0000 7f85b987c740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5582c4bfa4a0 0x5582c4c114b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:34.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.497+0000 7f85b987c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582c4bfadc0 0x5582c4c1ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:34.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.497+0000 7f85b987c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5582c4c05680 0x5582c4c1cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:34.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.497+0000 7f85b987c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f85b01b1fb0 con 0x5582c4bfadc0 2024-09-12T02:17:34.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.497+0000 7f85b987c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f85b01af880 con 0x5582c4c05680 2024-09-12T02:17:34.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.497+0000 7f85b987c740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f85b01ad6d0 con 0x5582c4bfa4a0 2024-09-12T02:17:34.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.497+0000 7f85b523f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582c4bfadc0 0x5582c4c1ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:34.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.497+0000 7f85b523f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582c4bfadc0 0x5582c4c1ca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53744/0 (socket says 172.21.15.16:53744) 2024-09-12T02:17:34.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.497+0000 7f85b523f640 1 -- 172.21.15.16:0/829307284 learned_addr learned my addr 172.21.15.16:0/829307284 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:17:34.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.497+0000 7f85b523f640 1 -- 172.21.15.16:0/829307284 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5582c4bfa4a0 msgr2=0x5582c4c114b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:34.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.497+0000 7f85b523f640 1 --2- 172.21.15.16:0/829307284 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5582c4bfa4a0 0x5582c4c114b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:34.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.497+0000 7f85b523f640 1 -- 172.21.15.16:0/829307284 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5582c4c05680 msgr2=0x5582c4c1cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:34.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.497+0000 7f85b523f640 1 --2- 172.21.15.16:0/829307284 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5582c4c05680 0x5582c4c1cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:34.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.497+0000 7f85b523f640 1 -- 172.21.15.16:0/829307284 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5582c4bf6860 con 0x5582c4bfadc0 2024-09-12T02:17:34.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.498+0000 7f85b523f640 1 --2- 172.21.15.16:0/829307284 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582c4bfadc0 0x5582c4c1ca90 secure :-1 s=READY pgs=2081 cs=0 l=1 rev1=1 crypto rx=0x7f85ac005940 tx=0x7f85ac005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63a74149e4c32a35 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:34.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.498+0000 7f859affd640 1 -- 172.21.15.16:0/829307284 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85ac049070 con 0x5582c4bfadc0 2024-09-12T02:17:34.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.498+0000 7f859affd640 1 -- 172.21.15.16:0/829307284 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85ac004400 con 0x5582c4bfadc0 2024-09-12T02:17:34.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.498+0000 7f859affd640 1 -- 172.21.15.16:0/829307284 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85ac035040 con 0x5582c4bfadc0 2024-09-12T02:17:34.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.498+0000 7f85b987c740 1 -- 172.21.15.16:0/829307284 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582c4bfadc0 msgr2=0x5582c4c1ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:34.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.498+0000 7f85b987c740 1 --2- 172.21.15.16:0/829307284 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582c4bfadc0 0x5582c4c1ca90 secure :-1 s=READY pgs=2081 cs=0 l=1 rev1=1 crypto rx=0x7f85ac005940 tx=0x7f85ac005d50 comp rx=0 tx=0).stop 2024-09-12T02:17:34.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.498+0000 7f85b987c740 1 -- 172.21.15.16:0/829307284 shutdown_connections 2024-09-12T02:17:34.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.498+0000 7f85b987c740 1 --2- 172.21.15.16:0/829307284 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5582c4c05680 0x5582c4c1cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:34.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.499+0000 7f85b987c740 1 --2- 172.21.15.16:0/829307284 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582c4bfadc0 0x5582c4c1ca90 unknown :-1 s=CLOSED pgs=2081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:34.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.499+0000 7f85b987c740 1 --2- 172.21.15.16:0/829307284 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5582c4bfa4a0 0x5582c4c114b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:34.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.499+0000 7f85b987c740 1 -- 172.21.15.16:0/829307284 >> 172.21.15.16:0/829307284 conn(0x5582c4bf6a30 msgr2=0x5582c4bf5b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:34.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.499+0000 7f85b987c740 1 -- 172.21.15.16:0/829307284 shutdown_connections 2024-09-12T02:17:34.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.499+0000 7f85b987c740 1 -- 172.21.15.16:0/829307284 wait complete. 2024-09-12T02:17:34.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.499+0000 7f85b987c740 1 -- start start 2024-09-12T02:17:34.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.499+0000 7f85b987c740 1 Processor -- start 2024-09-12T02:17:34.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.499+0000 7f85b987c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582c4bfa4a0 0x5582c4c0acb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:34.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.499+0000 7f85b987c740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5582c4bfadc0 0x5582c4c0b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:34.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.499+0000 7f85b987c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5582c4c05680 0x5582c4c77aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:34.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.499+0000 7f85b987c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f85b01add40 con 0x5582c4bfa4a0 2024-09-12T02:17:34.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.499+0000 7f85b987c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f85b01af1b0 con 0x5582c4c05680 2024-09-12T02:17:34.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.499+0000 7f85b987c740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f85b01ae410 con 0x5582c4bfadc0 2024-09-12T02:17:34.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.500+0000 7f85b987c740 10 client.0 _ll_register_callbacks cb 0x7f85a0012480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:17:34.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.500+0000 7f85b4a3e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582c4bfa4a0 0x5582c4c0acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:34.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.500+0000 7f85b4a3e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582c4bfa4a0 0x5582c4c0acb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53746/0 (socket says 172.21.15.16:53746) 2024-09-12T02:17:34.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.500+0000 7f85b4a3e640 1 -- 172.21.15.16:0/2866454463 learned_addr learned my addr 172.21.15.16:0/2866454463 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:17:34.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.500+0000 7f85b523f640 1 --2- 172.21.15.16:0/2866454463 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5582c4bfadc0 0x5582c4c0b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:34.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.500+0000 7f85b4a3e640 1 -- 172.21.15.16:0/2866454463 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5582c4bfadc0 msgr2=0x5582c4c0b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:34.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.500+0000 7f85b4a3e640 1 --2- 172.21.15.16:0/2866454463 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5582c4bfadc0 0x5582c4c0b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:34.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.500+0000 7f85b4a3e640 1 -- 172.21.15.16:0/2866454463 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5582c4c05680 msgr2=0x5582c4c77aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:34.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.500+0000 7f85b4a3e640 1 --2- 172.21.15.16:0/2866454463 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5582c4c05680 0x5582c4c77aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:34.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.500+0000 7f85b4a3e640 1 -- 172.21.15.16:0/2866454463 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5582c4c780e0 con 0x5582c4bfa4a0 2024-09-12T02:17:34.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.500+0000 7f85b523f640 1 --2- 172.21.15.16:0/2866454463 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5582c4bfadc0 0x5582c4c0b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:17:34.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.500+0000 7f85b4a3e640 1 --2- 172.21.15.16:0/2866454463 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582c4bfa4a0 0x5582c4c0acb0 secure :-1 s=READY pgs=2082 cs=0 l=1 rev1=1 crypto rx=0x7f85a0004f40 tx=0x7f85a000be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=190efbaee91fa8f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:34.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.500+0000 7f8598ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:17:34.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.500+0000 7f8598ff9640 1 -- 172.21.15.16:0/2866454463 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a000b770 con 0x5582c4bfa4a0 2024-09-12T02:17:34.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.500+0000 7f8598ff9640 1 -- 172.21.15.16:0/2866454463 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f85a0003800 con 0x5582c4bfa4a0 2024-09-12T02:17:34.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.500+0000 7f85b987c740 1 -- 172.21.15.16:0/2866454463 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85ac004400 con 0x5582c4bfa4a0 2024-09-12T02:17:34.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.500+0000 7f8598ff9640 1 -- 172.21.15.16:0/2866454463 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f85a0003a20 con 0x5582c4bfa4a0 2024-09-12T02:17:34.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.500+0000 7f85b987c740 10 client.20691 fetch_fsmap learned FSMap version 25 2024-09-12T02:17:34.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.500+0000 7f85b987c740 1 -- 172.21.15.16:0/2866454463 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5582c4c7f4a0 con 0x5582c4bfa4a0 2024-09-12T02:17:34.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.501+0000 7f8598ff9640 1 -- 172.21.15.16:0/2866454463 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85ac004400 con 0x5582c4bfa4a0 2024-09-12T02:17:34.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.501+0000 7f8598ff9640 1 -- 172.21.15.16:0/2866454463 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f85a000e490 con 0x5582c4bfa4a0 2024-09-12T02:17:34.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.501+0000 7f85b987c740 10 client.20691 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:17:34.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.501+0000 7f85b987c740 10 client.20691 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:17:34.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.501+0000 7f85b987c740 20 client.20691 populate_metadata read hostname 'smithi016' 2024-09-12T02:17:34.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.501+0000 7f85b987c740 1 --2- 172.21.15.16:0/2866454463 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5582c4c7fb10 0x5582c4c81f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:34.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.501+0000 7f85b987c740 4 client.20691 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T02:17:34.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.501+0000 7f85b987c740 1 -- 172.21.15.16:0/2866454463 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5582c4b67d60 con 0x5582c4c7fb10 2024-09-12T02:17:34.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.501+0000 7f85b523f640 1 --2- 172.21.15.16:0/2866454463 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5582c4c7fb10 0x5582c4c81f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:34.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.502+0000 7f8598ff9640 10 client.20691 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:17:34.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.502+0000 7f85b523f640 1 --2- 172.21.15.16:0/2866454463 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5582c4c7fb10 0x5582c4c81f30 crc :-1 s=READY pgs=464 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5145da13bcca9fbf server_cookie=b319224821d185b6 in_seq=0 out_seq=0 2024-09-12T02:17:34.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.502+0000 7f8598ff9640 1 -- 172.21.15.16:0/2866454463 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5582c4b67d60 con 0x5582c4c7fb10 2024-09-12T02:17:34.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.502+0000 7f8598ff9640 10 client.20691 handle_command_reply: tid=0 2024-09-12T02:17:34.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.502+0000 7f85b987c740 1 client.20691 shutdown 2024-09-12T02:17:34.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.502+0000 7f85b987c740 1 -- 172.21.15.16:0/2866454463 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582c4bfa4a0 msgr2=0x5582c4c0acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:34.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.502+0000 7f85b987c740 1 --2- 172.21.15.16:0/2866454463 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582c4bfa4a0 0x5582c4c0acb0 secure :-1 s=READY pgs=2082 cs=0 l=1 rev1=1 crypto rx=0x7f85a0004f40 tx=0x7f85a000be10 comp rx=0 tx=0).stop 2024-09-12T02:17:34.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.502+0000 7f85b987c740 1 -- 172.21.15.16:0/2866454463 shutdown_connections 2024-09-12T02:17:34.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.502+0000 7f85b987c740 1 --2- 172.21.15.16:0/2866454463 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5582c4c7fb10 0x5582c4c81f30 crc :-1 s=READY pgs=464 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:34.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.502+0000 7f85b987c740 1 --2- 172.21.15.16:0/2866454463 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5582c4c05680 0x5582c4c77aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:34.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.502+0000 7f85b987c740 1 --2- 172.21.15.16:0/2866454463 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5582c4bfadc0 0x5582c4c0b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:34.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.502+0000 7f8598ff9640 0 client.20691 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:17:34.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.502+0000 7f85b987c740 1 --2- 172.21.15.16:0/2866454463 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582c4bfa4a0 0x5582c4c0acb0 unknown :-1 s=CLOSED pgs=2082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:34.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.502+0000 7f85b987c740 1 -- 172.21.15.16:0/2866454463 >> 172.21.15.16:0/2866454463 conn(0x5582c4bf6a30 msgr2=0x5582c4c03a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:34.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.502+0000 7f85b987c740 1 -- 172.21.15.16:0/2866454463 shutdown_connections 2024-09-12T02:17:34.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.502+0000 7f85b987c740 1 -- 172.21.15.16:0/2866454463 wait complete. 2024-09-12T02:17:34.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.502+0000 7f85b987c740 20 client.20691 trim_cache size 0 max 16384 2024-09-12T02:17:34.508 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T02:17:34.509 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-09-12T02:17:34.509 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {} 2024-09-12T02:17:34.509 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T02:17:34.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.503+0000 7f85b74a3640 1 -- 172.21.15.16:0/500779859 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f859c0a2b80 msgr2=0x7f859c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:34.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.503+0000 7f85b74a3640 1 --2- 172.21.15.16:0/500779859 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f859c0a2b80 0x7f859c0a5040 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f85a80973c0 tx=0x7f85a8014d00 comp rx=0 tx=0).stop 2024-09-12T02:17:34.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.503+0000 7f85b74a3640 1 -- 172.21.15.16:0/500779859 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85b010c170 msgr2=0x7f85b019efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:34.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.503+0000 7f85b74a3640 1 --2- 172.21.15.16:0/500779859 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85b010c170 0x7f85b019efe0 secure :-1 s=READY pgs=2078 cs=0 l=1 rev1=1 crypto rx=0x7f85a800b970 tx=0x7f85a800be30 comp rx=0 tx=0).stop 2024-09-12T02:17:34.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.503+0000 7f85b74a3640 1 -- 172.21.15.16:0/500779859 shutdown_connections 2024-09-12T02:17:34.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.503+0000 7f85b74a3640 1 --2- 172.21.15.16:0/500779859 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f859c0a2b80 0x7f859c0a5040 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:34.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.503+0000 7f85b74a3640 1 --2- 172.21.15.16:0/500779859 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85b010c170 0x7f85b019efe0 unknown :-1 s=CLOSED pgs=2078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:34.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.503+0000 7f85b74a3640 1 --2- 172.21.15.16:0/500779859 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f85b01036f0 0x7f85b019ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:34.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.503+0000 7f85b74a3640 1 --2- 172.21.15.16:0/500779859 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85b0102dd0 0x7f85b019a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:34.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.503+0000 7f85b74a3640 1 -- 172.21.15.16:0/500779859 >> 172.21.15.16:0/500779859 conn(0x7f85b00fc790 msgr2=0x7f85b01009c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:34.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.503+0000 7f85b74a3640 1 -- 172.21.15.16:0/500779859 shutdown_connections 2024-09-12T02:17:34.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:34.503+0000 7f85b74a3640 1 -- 172.21.15.16:0/500779859 wait complete. 2024-09-12T02:17:35.335 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-12T02:17:35.335 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:17:35.597 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:17:36.017 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:17:35 smithi016 ceph-mon[30947]: pgmap v805: 97 pgs: 97 active+clean; 4.1 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-09-12T02:17:36.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:17:35 smithi184 ceph-mon[35593]: pgmap v805: 97 pgs: 97 active+clean; 4.1 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-09-12T02:17:36.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:36.099+0000 7f71093b5640 1 -- 172.21.15.16:0/1338413772 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7104100370 msgr2=0x7f7104100750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:36.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:36.099+0000 7f71093b5640 1 --2- 172.21.15.16:0/1338413772 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7104100370 0x7f7104100750 secure :-1 s=READY pgs=2083 cs=0 l=1 rev1=1 crypto rx=0x7f70f4009930 tx=0x7f70f402a420 comp rx=0 tx=0).stop 2024-09-12T02:17:36.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:36.099+0000 7f71093b5640 1 -- 172.21.15.16:0/1338413772 shutdown_connections 2024-09-12T02:17:36.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:36.099+0000 7f71093b5640 1 --2- 172.21.15.16:0/1338413772 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7104106fc0 0x7f710410eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:36.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:36.099+0000 7f71093b5640 1 --2- 172.21.15.16:0/1338413772 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7104100d20 0x7f7104106860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:36.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:36.099+0000 7f71093b5640 1 --2- 172.21.15.16:0/1338413772 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7104100370 0x7f7104100750 unknown :-1 s=CLOSED pgs=2083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:36.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:36.099+0000 7f71093b5640 1 -- 172.21.15.16:0/1338413772 >> 172.21.15.16:0/1338413772 conn(0x7f7104077210 msgr2=0x7f71040fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:36.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:36.099+0000 7f71093b5640 1 -- 172.21.15.16:0/1338413772 shutdown_connections 2024-09-12T02:17:36.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:36.099+0000 7f71093b5640 1 -- 172.21.15.16:0/1338413772 wait complete. 2024-09-12T02:17:36.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:36.100+0000 7f71093b5640 1 Processor -- start 2024-09-12T02:17:36.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:36.100+0000 7f71093b5640 1 -- start start 2024-09-12T02:17:36.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:36.100+0000 7f71093b5640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7104100370 0x7f7104198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:36.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:36.100+0000 7f71093b5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7104100d20 0x7f7104198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:36.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:36.100+0000 7f71093b5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7104106fc0 0x7f710419ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:36.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:36.100+0000 7f71093b5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7104111ff0 con 0x7f7104106fc0 2024-09-12T02:17:36.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:36.100+0000 7f71093b5640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7104111e70 con 0x7f7104100d20 2024-09-12T02:17:36.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:36.100+0000 7f71093b5640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7104112170 con 0x7f7104100370 2024-09-12T02:17:36.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:36.101+0000 7f71037fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7104106fc0 0x7f710419ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:36.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:36.101+0000 7f71037fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7104106fc0 0x7f710419ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53754/0 (socket says 172.21.15.16:53754) 2024-09-12T02:17:36.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:36.101+0000 7f71027fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7104100d20 0x7f7104198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:36.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:36.101+0000 7f71037fe640 1 -- 172.21.15.16:0/3579263442 learned_addr learned my addr 172.21.15.16:0/3579263442 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:17:36.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:36.101+0000 7f7102ffd640 1 --2- 172.21.15.16:0/3579263442 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7104100370 0x7f7104198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:36.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:36.101+0000 7f71037fe640 1 -- 172.21.15.16:0/3579263442 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7104100370 msgr2=0x7f7104198570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:36.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:36.101+0000 7f71037fe640 1 --2- 172.21.15.16:0/3579263442 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7104100370 0x7f7104198570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:36.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:36.101+0000 7f71037fe640 1 -- 172.21.15.16:0/3579263442 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7104100d20 msgr2=0x7f7104198ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:36.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:36.101+0000 7f71037fe640 1 --2- 172.21.15.16:0/3579263442 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7104100d20 0x7f7104198ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:36.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:36.101+0000 7f71037fe640 1 -- 172.21.15.16:0/3579263442 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f710419d520 con 0x7f7104106fc0 2024-09-12T02:17:36.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:36.101+0000 7f71037fe640 1 --2- 172.21.15.16:0/3579263442 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7104106fc0 0x7f710419ce40 secure :-1 s=READY pgs=2084 cs=0 l=1 rev1=1 crypto rx=0x7f70fc00cad0 tx=0x7f70fc00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:36.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:36.101+0000 7f70fbfff640 1 -- 172.21.15.16:0/3579263442 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70fc00ae20 con 0x7f7104106fc0 2024-09-12T02:17:36.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:36.101+0000 7f71093b5640 1 -- 172.21.15.16:0/3579263442 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f710419d810 con 0x7f7104106fc0 2024-09-12T02:17:36.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:36.101+0000 7f70fbfff640 1 -- 172.21.15.16:0/3579263442 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70fc005020 con 0x7f7104106fc0 2024-09-12T02:17:36.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:36.101+0000 7f70fbfff640 1 -- 172.21.15.16:0/3579263442 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70fc00f6f0 con 0x7f7104106fc0 2024-09-12T02:17:36.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:36.102+0000 7f71093b5640 1 -- 172.21.15.16:0/3579263442 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71041a50f0 con 0x7f7104106fc0 2024-09-12T02:17:36.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:36.103+0000 7f71093b5640 1 -- 172.21.15.16:0/3579263442 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7104100750 con 0x7f7104106fc0 2024-09-12T02:17:36.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:36.105+0000 7f70fbfff640 1 -- 172.21.15.16:0/3579263442 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f70fc0051d0 con 0x7f7104106fc0 2024-09-12T02:17:36.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:36.106+0000 7f70fbfff640 1 --2- 172.21.15.16:0/3579263442 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f70f00a2b80 0x7f70f00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:36.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:36.106+0000 7f70fbfff640 1 -- 172.21.15.16:0/3579263442 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f70fc0cf330 con 0x7f7104106fc0 2024-09-12T02:17:36.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:36.106+0000 7f7102ffd640 1 --2- 172.21.15.16:0/3579263442 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f70f00a2b80 0x7f70f00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:36.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:36.106+0000 7f7102ffd640 1 --2- 172.21.15.16:0/3579263442 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f70f00a2b80 0x7f70f00a5040 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f70f4006010 tx=0x7f70f4000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:36.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:36.109+0000 7f70fbfff640 1 -- 172.21.15.16:0/3579263442 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f70fc014030 con 0x7f7104106fc0 2024-09-12T02:17:36.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:17:35 smithi033 ceph-mon[35332]: pgmap v805: 97 pgs: 97 active+clean; 4.1 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-09-12T02:17:36.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:36.323+0000 7f71093b5640 1 -- 172.21.15.16:0/3579263442 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f710410a160 con 0x7f7104106fc0 2024-09-12T02:17:36.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:36.324+0000 7f70fbfff640 1 -- 172.21.15.16:0/3579263442 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11911 (secure 0 0 0) 0x7f70fc09c330 con 0x7f7104106fc0 2024-09-12T02:17:36.324 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T02:17:36.325 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":25,"btime":"2024-09-12T02:01:37:192021+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3279123279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3279123279},{"type":"v1","addr":"172.21.15.16:6835","nonce":3279123279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4119452382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4119452382},{"type":"v1","addr":"172.21.15.16:6837","nonce":4119452382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/3263823165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3263823165},{"type":"v1","addr":"172.21.15.16:6839","nonce":3263823165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/1510290732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":1510290732},{"type":"v1","addr":"172.21.15.16:6841","nonce":1510290732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/279674603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":279674603},{"type":"v1","addr":"172.21.15.33:6833","nonce":279674603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3555728028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3555728028},{"type":"v1","addr":"172.21.15.33:6835","nonce":3555728028}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T01:59:56.117608+0000","modified":"2024-09-12T02:01:36.294760+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24482,"mds_1":24481,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6833/1563374106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1563374106},{"type":"v1","addr":"172.21.15.184:6833","nonce":1563374106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6837/1892833868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1892833868},{"type":"v1","addr":"172.21.15.33:6837","nonce":1892833868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/1457679686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1457679686},{"type":"v1","addr":"172.21.15.184:6835","nonce":1457679686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.184:6837/1793393087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":1793393087},{"type":"v1","addr":"172.21.15.184:6837","nonce":1793393087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.184:6839/4203347941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":4203347941},{"type":"v1","addr":"172.21.15.184:6839","nonce":4203347941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.33:6839/4136670204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":4136670204},{"type":"v1","addr":"172.21.15.33:6839","nonce":4136670204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24481,24482,24469]},"id":1}]} 2024-09-12T02:17:36.325 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 25 2024-09-12T02:17:36.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:36.326+0000 7f71093b5640 1 -- 172.21.15.16:0/3579263442 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f70f00a2b80 msgr2=0x7f70f00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:36.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:36.326+0000 7f71093b5640 1 --2- 172.21.15.16:0/3579263442 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f70f00a2b80 0x7f70f00a5040 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f70f4006010 tx=0x7f70f4000bc0 comp rx=0 tx=0).stop 2024-09-12T02:17:36.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:36.326+0000 7f71093b5640 1 -- 172.21.15.16:0/3579263442 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7104106fc0 msgr2=0x7f710419ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:36.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:36.326+0000 7f71093b5640 1 --2- 172.21.15.16:0/3579263442 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7104106fc0 0x7f710419ce40 secure :-1 s=READY pgs=2084 cs=0 l=1 rev1=1 crypto rx=0x7f70fc00cad0 tx=0x7f70fc00cf90 comp rx=0 tx=0).stop 2024-09-12T02:17:36.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:36.326+0000 7f71093b5640 1 -- 172.21.15.16:0/3579263442 shutdown_connections 2024-09-12T02:17:36.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:36.326+0000 7f71093b5640 1 --2- 172.21.15.16:0/3579263442 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f70f00a2b80 0x7f70f00a5040 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:36.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:36.326+0000 7f71093b5640 1 --2- 172.21.15.16:0/3579263442 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7104106fc0 0x7f710419ce40 unknown :-1 s=CLOSED pgs=2084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:36.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:36.326+0000 7f71093b5640 1 --2- 172.21.15.16:0/3579263442 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7104100d20 0x7f7104198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:36.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:36.326+0000 7f71093b5640 1 --2- 172.21.15.16:0/3579263442 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7104100370 0x7f7104198570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:36.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:36.326+0000 7f71093b5640 1 -- 172.21.15.16:0/3579263442 >> 172.21.15.16:0/3579263442 conn(0x7f7104077210 msgr2=0x7f71040fc840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:36.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:36.326+0000 7f71093b5640 1 -- 172.21.15.16:0/3579263442 shutdown_connections 2024-09-12T02:17:36.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:36.326+0000 7f71093b5640 1 -- 172.21.15.16:0/3579263442 wait complete. 2024-09-12T02:17:36.933 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:17:36 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/3579263442' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:17:37.044 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-12T02:17:37.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:17:36 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/3579263442' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:17:37.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:17:36 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/3579263442' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:17:37.306 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:17:37.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.799+0000 7fe9d4d06640 1 -- 172.21.15.16:0/184970226 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9d010e480 msgr2=0x7fe9d0110870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:37.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.799+0000 7fe9d4d06640 1 --2- 172.21.15.16:0/184970226 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9d010e480 0x7fe9d0110870 secure :-1 s=READY pgs=2085 cs=0 l=1 rev1=1 crypto rx=0x7fe9c8009ca0 tx=0x7fe9c802a8c0 comp rx=0 tx=0).stop 2024-09-12T02:17:37.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.799+0000 7fe9d4d06640 1 -- 172.21.15.16:0/184970226 shutdown_connections 2024-09-12T02:17:37.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.799+0000 7fe9d4d06640 1 --2- 172.21.15.16:0/184970226 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9d010e480 0x7fe9d0110870 unknown :-1 s=CLOSED pgs=2085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:37.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.799+0000 7fe9d4d06640 1 --2- 172.21.15.16:0/184970226 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe9d0101280 0x7fe9d010df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:37.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.799+0000 7fe9d4d06640 1 --2- 172.21.15.16:0/184970226 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe9d01008d0 0x7fe9d0100cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:37.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.799+0000 7fe9d4d06640 1 -- 172.21.15.16:0/184970226 >> 172.21.15.16:0/184970226 conn(0x7fe9d00fc7a0 msgr2=0x7fe9d00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:37.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.799+0000 7fe9d4d06640 1 -- 172.21.15.16:0/184970226 shutdown_connections 2024-09-12T02:17:37.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.799+0000 7fe9d4d06640 1 -- 172.21.15.16:0/184970226 wait complete. 2024-09-12T02:17:37.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.800+0000 7fe9d4d06640 1 Processor -- start 2024-09-12T02:17:37.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.800+0000 7fe9d4d06640 1 -- start start 2024-09-12T02:17:37.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.800+0000 7fe9d4d06640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9d01008d0 0x7fe9d019a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:37.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.800+0000 7fe9d4d06640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe9d0101280 0x7fe9d019abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:37.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.800+0000 7fe9d4d06640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe9d010e480 0x7fe9d019ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:37.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.800+0000 7fe9d4d06640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe9d0114170 con 0x7fe9d01008d0 2024-09-12T02:17:37.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.800+0000 7fe9d4d06640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe9d0113ff0 con 0x7fe9d0101280 2024-09-12T02:17:37.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.800+0000 7fe9d4d06640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe9d01142f0 con 0x7fe9d010e480 2024-09-12T02:17:37.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.801+0000 7fe9ce59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9d01008d0 0x7fe9d019a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:37.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.801+0000 7fe9ce59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9d01008d0 0x7fe9d019a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53770/0 (socket says 172.21.15.16:53770) 2024-09-12T02:17:37.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.801+0000 7fe9ce59c640 1 -- 172.21.15.16:0/523964805 learned_addr learned my addr 172.21.15.16:0/523964805 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:17:37.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.801+0000 7fe9c7fff640 1 --2- 172.21.15.16:0/523964805 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe9d0101280 0x7fe9d019abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:37.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.801+0000 7fe9ced9d640 1 --2- 172.21.15.16:0/523964805 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe9d010e480 0x7fe9d019ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:37.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.801+0000 7fe9ce59c640 1 -- 172.21.15.16:0/523964805 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe9d010e480 msgr2=0x7fe9d019ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:37.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.801+0000 7fe9ce59c640 1 --2- 172.21.15.16:0/523964805 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe9d010e480 0x7fe9d019ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:37.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.801+0000 7fe9ce59c640 1 -- 172.21.15.16:0/523964805 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe9d0101280 msgr2=0x7fe9d019abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:37.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.801+0000 7fe9ce59c640 1 --2- 172.21.15.16:0/523964805 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe9d0101280 0x7fe9d019abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:37.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.801+0000 7fe9ce59c640 1 -- 172.21.15.16:0/523964805 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9d019f620 con 0x7fe9d01008d0 2024-09-12T02:17:37.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.801+0000 7fe9ce59c640 1 --2- 172.21.15.16:0/523964805 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9d01008d0 0x7fe9d019a670 secure :-1 s=READY pgs=2086 cs=0 l=1 rev1=1 crypto rx=0x7fe9c000c970 tx=0x7fe9c000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:37.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.801+0000 7fe9c77fe640 1 -- 172.21.15.16:0/523964805 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9c0007bb0 con 0x7fe9d01008d0 2024-09-12T02:17:37.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.801+0000 7fe9c77fe640 1 -- 172.21.15.16:0/523964805 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe9c0007d50 con 0x7fe9d01008d0 2024-09-12T02:17:37.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.801+0000 7fe9c77fe640 1 -- 172.21.15.16:0/523964805 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9c000f450 con 0x7fe9d01008d0 2024-09-12T02:17:37.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.801+0000 7fe9d4d06640 1 -- 172.21.15.16:0/523964805 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9d019f910 con 0x7fe9d01008d0 2024-09-12T02:17:37.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.801+0000 7fe9d4d06640 1 -- 172.21.15.16:0/523964805 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9d01a71f0 con 0x7fe9d01008d0 2024-09-12T02:17:37.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.804+0000 7fe9c77fe640 1 -- 172.21.15.16:0/523964805 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fe9c0005020 con 0x7fe9d01008d0 2024-09-12T02:17:37.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.804+0000 7fe9d70df740 1 Processor -- start 2024-09-12T02:17:37.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.804+0000 7fe9d70df740 1 -- start start 2024-09-12T02:17:37.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.805+0000 7fe9d70df740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56128b6e5b40 0x56128b6e4090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:37.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.805+0000 7fe9d70df740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56128b6e45d0 0x56128b6e4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:37.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.805+0000 7fe9d70df740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56128b6ef370 0x56128b6f1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:37.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.805+0000 7fe9d70df740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56128b64a2f0 con 0x56128b6e5b40 2024-09-12T02:17:37.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.805+0000 7fe9d70df740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56128b5f1cf0 con 0x56128b6ef370 2024-09-12T02:17:37.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.805+0000 7fe9d70df740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x56128b6462b0 con 0x56128b6e45d0 2024-09-12T02:17:37.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.805+0000 7fe9ced9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56128b6e5b40 0x56128b6e4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:37.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.805+0000 7fe9ced9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56128b6e5b40 0x56128b6e4090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53788/0 (socket says 172.21.15.16:53788) 2024-09-12T02:17:37.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.805+0000 7fe9ced9d640 1 -- 172.21.15.16:0/3443984425 learned_addr learned my addr 172.21.15.16:0/3443984425 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:17:37.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.805+0000 7fe9ce59c640 1 --2- 172.21.15.16:0/3443984425 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56128b6e45d0 0x56128b6e4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:37.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.805+0000 7fe9c77fe640 1 --2- 172.21.15.16:0/523964805 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fe9bc0a2c00 0x7fe9bc0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:37.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.805+0000 7fe9ced9d640 1 -- 172.21.15.16:0/3443984425 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56128b6e45d0 msgr2=0x56128b6e4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:37.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.805+0000 7fe9ced9d640 1 --2- 172.21.15.16:0/3443984425 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56128b6e45d0 0x56128b6e4a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:37.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.805+0000 7fe9ced9d640 1 -- 172.21.15.16:0/3443984425 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56128b6ef370 msgr2=0x56128b6f1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:37.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.805+0000 7fe9c7fff640 1 --2- 172.21.15.16:0/3443984425 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56128b6ef370 0x56128b6f1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:37.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.805+0000 7fe9ced9d640 1 --2- 172.21.15.16:0/3443984425 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56128b6ef370 0x56128b6f1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:37.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.805+0000 7fe9ced9d640 1 -- 172.21.15.16:0/3443984425 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56128b5d63a0 con 0x56128b6e5b40 2024-09-12T02:17:37.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.805+0000 7fe9c7fff640 1 --2- 172.21.15.16:0/3443984425 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56128b6ef370 0x56128b6f1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:17:37.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.805+0000 7fe9c77fe640 1 -- 172.21.15.16:0/523964805 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fe9c00ceda0 con 0x7fe9d01008d0 2024-09-12T02:17:37.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.805+0000 7fe9ced9d640 1 --2- 172.21.15.16:0/523964805 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fe9bc0a2c00 0x7fe9bc0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:37.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.806+0000 7fe9ced9d640 1 --2- 172.21.15.16:0/3443984425 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56128b6e5b40 0x56128b6e4090 secure :-1 s=READY pgs=2087 cs=0 l=1 rev1=1 crypto rx=0x7fe9c8005cc0 tx=0x7fe9c8031bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b6d8e835d99d07b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:37.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.806+0000 7fe9c4ff9640 1 -- 172.21.15.16:0/3443984425 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9c8041040 con 0x56128b6e5b40 2024-09-12T02:17:37.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.806+0000 7fe9c4ff9640 1 -- 172.21.15.16:0/3443984425 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe9c8036070 con 0x56128b6e5b40 2024-09-12T02:17:37.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.806+0000 7fe9c4ff9640 1 -- 172.21.15.16:0/3443984425 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9c802fa30 con 0x56128b6e5b40 2024-09-12T02:17:37.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.806+0000 7fe9ced9d640 1 --2- 172.21.15.16:0/523964805 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fe9bc0a2c00 0x7fe9bc0a50c0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fe9c8009430 tx=0x7fe9c804d000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:37.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.807+0000 7fe9d70df740 1 -- 172.21.15.16:0/3443984425 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56128b6e5b40 msgr2=0x56128b6e4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:37.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.807+0000 7fe9d70df740 1 --2- 172.21.15.16:0/3443984425 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56128b6e5b40 0x56128b6e4090 secure :-1 s=READY pgs=2087 cs=0 l=1 rev1=1 crypto rx=0x7fe9c8005cc0 tx=0x7fe9c8031bb0 comp rx=0 tx=0).stop 2024-09-12T02:17:37.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.807+0000 7fe9d70df740 1 -- 172.21.15.16:0/3443984425 shutdown_connections 2024-09-12T02:17:37.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.807+0000 7fe9d70df740 1 --2- 172.21.15.16:0/3443984425 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56128b6ef370 0x56128b6f1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:37.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.807+0000 7fe9d70df740 1 --2- 172.21.15.16:0/3443984425 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56128b6e45d0 0x56128b6e4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:37.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.807+0000 7fe9d70df740 1 --2- 172.21.15.16:0/3443984425 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56128b6e5b40 0x56128b6e4090 unknown :-1 s=CLOSED pgs=2087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:37.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.807+0000 7fe9d70df740 1 -- 172.21.15.16:0/3443984425 >> 172.21.15.16:0/3443984425 conn(0x56128b6ad310 msgr2=0x56128b6e0c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:37.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.807+0000 7fe9d70df740 1 -- 172.21.15.16:0/3443984425 shutdown_connections 2024-09-12T02:17:37.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.807+0000 7fe9d70df740 1 -- 172.21.15.16:0/3443984425 wait complete. 2024-09-12T02:17:37.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.807+0000 7fe9d70df740 1 -- start start 2024-09-12T02:17:37.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.807+0000 7fe9d70df740 1 Processor -- start 2024-09-12T02:17:37.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.807+0000 7fe9d70df740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56128b6e45d0 0x56128b6eed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:37.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.807+0000 7fe9d70df740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56128b6e5b40 0x56128b6ec040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:37.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.807+0000 7fe9d70df740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56128b6ef370 0x56128b6ec650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:37.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.807+0000 7fe9d70df740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56128b6f2070 con 0x56128b6e5b40 2024-09-12T02:17:37.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.807+0000 7fe9d70df740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56128b605060 con 0x56128b6ef370 2024-09-12T02:17:37.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.807+0000 7fe9d70df740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x56128b6f21f0 con 0x56128b6e45d0 2024-09-12T02:17:37.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.807+0000 7fe9d70df740 10 client.0 _ll_register_callbacks cb 0x56128b3588c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:17:37.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.808+0000 7fe9c7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56128b6e5b40 0x56128b6ec040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:37.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.808+0000 7fe9c7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56128b6e5b40 0x56128b6ec040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53794/0 (socket says 172.21.15.16:53794) 2024-09-12T02:17:37.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.808+0000 7fe9c7fff640 1 -- 172.21.15.16:0/2575877408 learned_addr learned my addr 172.21.15.16:0/2575877408 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:17:37.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.808+0000 7fe9ced9d640 1 --2- 172.21.15.16:0/2575877408 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56128b6ef370 0x56128b6ec650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:37.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.808+0000 7fe9ce59c640 1 --2- 172.21.15.16:0/2575877408 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56128b6e45d0 0x56128b6eed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:37.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.808+0000 7fe9c7fff640 1 -- 172.21.15.16:0/2575877408 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56128b6e45d0 msgr2=0x56128b6eed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:37.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.808+0000 7fe9c7fff640 1 --2- 172.21.15.16:0/2575877408 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56128b6e45d0 0x56128b6eed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:37.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.808+0000 7fe9c7fff640 1 -- 172.21.15.16:0/2575877408 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56128b6ef370 msgr2=0x56128b6ec650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:37.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.808+0000 7fe9c7fff640 1 --2- 172.21.15.16:0/2575877408 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56128b6ef370 0x56128b6ec650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:37.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.808+0000 7fe9c7fff640 1 -- 172.21.15.16:0/2575877408 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56128b3bb3c0 con 0x56128b6e5b40 2024-09-12T02:17:37.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.808+0000 7fe9ce59c640 1 --2- 172.21.15.16:0/2575877408 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56128b6e45d0 0x56128b6eed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:17:37.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.808+0000 7fe9c7fff640 1 --2- 172.21.15.16:0/2575877408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56128b6e5b40 0x56128b6ec040 secure :-1 s=READY pgs=2088 cs=0 l=1 rev1=1 crypto rx=0x7fe9b800bed0 tx=0x7fe9b80117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5a54cb0f80a7bdc server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:37.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.808+0000 7fe9b5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:17:37.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.808+0000 7fe9b5ffb640 1 -- 172.21.15.16:0/2575877408 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9b8012920 con 0x56128b6e5b40 2024-09-12T02:17:37.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.808+0000 7fe9b5ffb640 1 -- 172.21.15.16:0/2575877408 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fe9b8012ac0 con 0x56128b6e5b40 2024-09-12T02:17:37.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.808+0000 7fe9d70df740 1 -- 172.21.15.16:0/2575877408 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe9c8036070 con 0x56128b6e5b40 2024-09-12T02:17:37.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.809+0000 7fe9b5ffb640 1 -- 172.21.15.16:0/2575877408 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe9b8012cc0 con 0x56128b6e5b40 2024-09-12T02:17:37.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.809+0000 7fe9d70df740 10 client.20709 fetch_fsmap learned FSMap version 25 2024-09-12T02:17:37.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.809+0000 7fe9d70df740 1 -- 172.21.15.16:0/2575877408 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56128b6fdb60 con 0x56128b6e5b40 2024-09-12T02:17:37.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.809+0000 7fe9b5ffb640 1 -- 172.21.15.16:0/2575877408 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe9c8036070 con 0x56128b6e5b40 2024-09-12T02:17:37.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.809+0000 7fe9b5ffb640 1 -- 172.21.15.16:0/2575877408 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fe9b8028520 con 0x56128b6e5b40 2024-09-12T02:17:37.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.809+0000 7fe9d70df740 10 client.20709 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:17:37.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.809+0000 7fe9d70df740 10 client.20709 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-12T02:17:37.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.809+0000 7fe9d70df740 20 client.20709 populate_metadata read hostname 'smithi016' 2024-09-12T02:17:37.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.809+0000 7fe9d70df740 1 --2- 172.21.15.16:0/2575877408 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x56128b6ff020 0x56128b701470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:37.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.809+0000 7fe9d70df740 4 client.20709 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:17:37.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.809+0000 7fe9d70df740 1 -- 172.21.15.16:0/2575877408 --> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56128b657860 con 0x56128b6ff020 2024-09-12T02:17:37.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.809+0000 7fe9ce59c640 1 --2- 172.21.15.16:0/2575877408 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x56128b6ff020 0x56128b701470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:37.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.810+0000 7fe9ce59c640 1 --2- 172.21.15.16:0/2575877408 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x56128b6ff020 0x56128b701470 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5e0bc11ab33809df server_cookie=63eb0882a74cebf5 in_seq=0 out_seq=0 2024-09-12T02:17:37.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.810+0000 7fe9b5ffb640 10 client.20709 ms_handle_connect on v2:172.21.15.184:6836/1793393087 2024-09-12T02:17:37.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.811+0000 7fe9b5ffb640 1 -- 172.21.15.16:0/2575877408 <== mds.2 v2:172.21.15.184:6836/1793393087 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56128b657860 con 0x56128b6ff020 2024-09-12T02:17:37.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.811+0000 7fe9b5ffb640 10 client.20709 handle_command_reply: tid=0 2024-09-12T02:17:37.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.811+0000 7fe9d70df740 1 client.20709 shutdown 2024-09-12T02:17:37.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.811+0000 7fe9d70df740 1 -- 172.21.15.16:0/2575877408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56128b6e5b40 msgr2=0x56128b6ec040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:37.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.811+0000 7fe9d70df740 1 --2- 172.21.15.16:0/2575877408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56128b6e5b40 0x56128b6ec040 secure :-1 s=READY pgs=2088 cs=0 l=1 rev1=1 crypto rx=0x7fe9b800bed0 tx=0x7fe9b80117b0 comp rx=0 tx=0).stop 2024-09-12T02:17:37.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.811+0000 7fe9d70df740 1 -- 172.21.15.16:0/2575877408 shutdown_connections 2024-09-12T02:17:37.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.811+0000 7fe9d70df740 1 --2- 172.21.15.16:0/2575877408 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56128b6ef370 0x56128b6ec650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:37.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.811+0000 7fe9d70df740 1 --2- 172.21.15.16:0/2575877408 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x56128b6ff020 0x56128b701470 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:37.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.811+0000 7fe9d70df740 1 --2- 172.21.15.16:0/2575877408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56128b6e5b40 0x56128b6ec040 unknown :-1 s=CLOSED pgs=2088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:37.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.811+0000 7fe9d70df740 1 --2- 172.21.15.16:0/2575877408 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56128b6e45d0 0x56128b6eed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:37.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.811+0000 7fe9d70df740 1 -- 172.21.15.16:0/2575877408 >> 172.21.15.16:0/2575877408 conn(0x56128b6ad310 msgr2=0x56128b6e6960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:37.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.811+0000 7fe9b5ffb640 0 client.20709 ms_handle_reset on v2:172.21.15.184:6836/1793393087 2024-09-12T02:17:37.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.811+0000 7fe9d70df740 1 -- 172.21.15.16:0/2575877408 shutdown_connections 2024-09-12T02:17:37.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.811+0000 7fe9d70df740 1 -- 172.21.15.16:0/2575877408 wait complete. 2024-09-12T02:17:37.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.811+0000 7fe9d70df740 20 client.20709 trim_cache size 0 max 16384 2024-09-12T02:17:37.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.824+0000 7fe9d70df740 1 Processor -- start 2024-09-12T02:17:37.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.824+0000 7fe9d70df740 1 -- start start 2024-09-12T02:17:37.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.824+0000 7fe9d70df740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56128b6e45d0 0x56128b6e0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:37.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.824+0000 7fe9d70df740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56128b6e5b40 0x56128b6fb310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:37.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.824+0000 7fe9d70df740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56128b6ef370 0x56128b6dbab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:37.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.824+0000 7fe9d70df740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe9d01b1eb0 con 0x56128b6e5b40 2024-09-12T02:17:37.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.824+0000 7fe9d70df740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe9d01af780 con 0x56128b6e45d0 2024-09-12T02:17:37.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.824+0000 7fe9d70df740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe9d01ad5d0 con 0x56128b6ef370 2024-09-12T02:17:37.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.824+0000 7fe9c7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56128b6e5b40 0x56128b6fb310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:37.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.824+0000 7fe9c7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56128b6e5b40 0x56128b6fb310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53796/0 (socket says 172.21.15.16:53796) 2024-09-12T02:17:37.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.824+0000 7fe9c7fff640 1 -- 172.21.15.16:0/3306976846 learned_addr learned my addr 172.21.15.16:0/3306976846 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:17:37.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.824+0000 7fe9ce59c640 1 --2- 172.21.15.16:0/3306976846 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56128b6e45d0 0x56128b6e0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:37.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.824+0000 7fe9ced9d640 1 --2- 172.21.15.16:0/3306976846 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56128b6ef370 0x56128b6dbab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:37.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.825+0000 7fe9c7fff640 1 -- 172.21.15.16:0/3306976846 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56128b6ef370 msgr2=0x56128b6dbab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:37.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.825+0000 7fe9c7fff640 1 --2- 172.21.15.16:0/3306976846 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56128b6ef370 0x56128b6dbab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:37.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.825+0000 7fe9c7fff640 1 -- 172.21.15.16:0/3306976846 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56128b6e45d0 msgr2=0x56128b6e0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:37.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.825+0000 7fe9c7fff640 1 --2- 172.21.15.16:0/3306976846 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56128b6e45d0 0x56128b6e0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:37.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.825+0000 7fe9c7fff640 1 -- 172.21.15.16:0/3306976846 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56128b6ec980 con 0x56128b6e5b40 2024-09-12T02:17:37.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.825+0000 7fe9ce59c640 1 --2- 172.21.15.16:0/3306976846 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56128b6e45d0 0x56128b6e0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:17:37.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.825+0000 7fe9c7fff640 1 --2- 172.21.15.16:0/3306976846 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56128b6e5b40 0x56128b6fb310 secure :-1 s=READY pgs=2089 cs=0 l=1 rev1=1 crypto rx=0x7fe9b8028b00 tx=0x7fe9b8005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e106cc9906ad282f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:37.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.825+0000 7fe9b7fff640 1 -- 172.21.15.16:0/3306976846 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9b802d070 con 0x56128b6e5b40 2024-09-12T02:17:37.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.825+0000 7fe9b7fff640 1 -- 172.21.15.16:0/3306976846 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe9b8028d80 con 0x56128b6e5b40 2024-09-12T02:17:37.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.825+0000 7fe9b7fff640 1 -- 172.21.15.16:0/3306976846 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9b80053b0 con 0x56128b6e5b40 2024-09-12T02:17:37.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.826+0000 7fe9d70df740 1 -- 172.21.15.16:0/3306976846 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56128b6e5b40 msgr2=0x56128b6fb310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:37.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.826+0000 7fe9d70df740 1 --2- 172.21.15.16:0/3306976846 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56128b6e5b40 0x56128b6fb310 secure :-1 s=READY pgs=2089 cs=0 l=1 rev1=1 crypto rx=0x7fe9b8028b00 tx=0x7fe9b8005c50 comp rx=0 tx=0).stop 2024-09-12T02:17:37.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.826+0000 7fe9d70df740 1 -- 172.21.15.16:0/3306976846 shutdown_connections 2024-09-12T02:17:37.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.826+0000 7fe9d70df740 1 --2- 172.21.15.16:0/3306976846 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56128b6ef370 0x56128b6dbab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:37.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.826+0000 7fe9d70df740 1 --2- 172.21.15.16:0/3306976846 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56128b6e5b40 0x56128b6fb310 unknown :-1 s=CLOSED pgs=2089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:37.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.826+0000 7fe9d70df740 1 --2- 172.21.15.16:0/3306976846 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56128b6e45d0 0x56128b6e0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:37.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.826+0000 7fe9d70df740 1 -- 172.21.15.16:0/3306976846 >> 172.21.15.16:0/3306976846 conn(0x56128b6ad310 msgr2=0x56128b6e0700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:37.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.826+0000 7fe9d70df740 1 -- 172.21.15.16:0/3306976846 shutdown_connections 2024-09-12T02:17:37.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.826+0000 7fe9d70df740 1 -- 172.21.15.16:0/3306976846 wait complete. 2024-09-12T02:17:37.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.826+0000 7fe9d70df740 1 -- start start 2024-09-12T02:17:37.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.826+0000 7fe9d70df740 1 Processor -- start 2024-09-12T02:17:37.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.826+0000 7fe9d70df740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56128b6e45d0 0x56128b6f7480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:37.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.826+0000 7fe9d70df740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56128b6e5b40 0x56128b6f79c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:37.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.826+0000 7fe9d70df740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56128b6ef370 0x56128b6e82b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:37.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.826+0000 7fe9d70df740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe9d01adc40 con 0x56128b6e45d0 2024-09-12T02:17:37.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.826+0000 7fe9d70df740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe9d01af0b0 con 0x56128b6ef370 2024-09-12T02:17:37.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.827+0000 7fe9d70df740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe9d01ae310 con 0x56128b6e5b40 2024-09-12T02:17:37.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.827+0000 7fe9d70df740 10 client.0 _ll_register_callbacks cb 0x7fe9b800b730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:17:37.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.827+0000 7fe9ce59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56128b6e45d0 0x56128b6f7480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:37.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.827+0000 7fe9ce59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56128b6e45d0 0x56128b6f7480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53810/0 (socket says 172.21.15.16:53810) 2024-09-12T02:17:37.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.827+0000 7fe9ce59c640 1 -- 172.21.15.16:0/3607155874 learned_addr learned my addr 172.21.15.16:0/3607155874 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:17:37.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.827+0000 7fe9ced9d640 1 --2- 172.21.15.16:0/3607155874 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56128b6ef370 0x56128b6e82b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:37.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.827+0000 7fe9c7fff640 1 --2- 172.21.15.16:0/3607155874 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56128b6e5b40 0x56128b6f79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:37.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.827+0000 7fe9ce59c640 1 -- 172.21.15.16:0/3607155874 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56128b6e5b40 msgr2=0x56128b6f79c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:37.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.827+0000 7fe9ce59c640 1 --2- 172.21.15.16:0/3607155874 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56128b6e5b40 0x56128b6f79c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:37.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.827+0000 7fe9ce59c640 1 -- 172.21.15.16:0/3607155874 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56128b6ef370 msgr2=0x56128b6e82b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:37.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.827+0000 7fe9ce59c640 1 --2- 172.21.15.16:0/3607155874 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56128b6ef370 0x56128b6e82b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:37.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.827+0000 7fe9ce59c640 1 -- 172.21.15.16:0/3607155874 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56128b6e9f30 con 0x56128b6e45d0 2024-09-12T02:17:37.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.827+0000 7fe9ced9d640 1 --2- 172.21.15.16:0/3607155874 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56128b6ef370 0x56128b6e82b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:17:37.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.827+0000 7fe9ce59c640 1 --2- 172.21.15.16:0/3607155874 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56128b6e45d0 0x56128b6f7480 secure :-1 s=READY pgs=2090 cs=0 l=1 rev1=1 crypto rx=0x7fe9c009b590 tx=0x7fe9c00979d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=763ccde14aefa936 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:37.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.827+0000 7fe9b5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:17:37.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.827+0000 7fe9b5ffb640 1 -- 172.21.15.16:0/3607155874 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9c001d040 con 0x56128b6e45d0 2024-09-12T02:17:37.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.827+0000 7fe9b5ffb640 1 -- 172.21.15.16:0/3607155874 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fe9c00cc9b0 con 0x56128b6e45d0 2024-09-12T02:17:37.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.827+0000 7fe9d70df740 1 -- 172.21.15.16:0/3607155874 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe9b8028d80 con 0x56128b6e45d0 2024-09-12T02:17:37.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.827+0000 7fe9b5ffb640 1 -- 172.21.15.16:0/3607155874 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe9c009bd00 con 0x56128b6e45d0 2024-09-12T02:17:37.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.828+0000 7fe9d70df740 10 client.20715 fetch_fsmap learned FSMap version 25 2024-09-12T02:17:37.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.828+0000 7fe9d70df740 1 -- 172.21.15.16:0/3607155874 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56128b769230 con 0x56128b6e45d0 2024-09-12T02:17:37.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.828+0000 7fe9b5ffb640 1 -- 172.21.15.16:0/3607155874 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe9b8028d80 con 0x56128b6e45d0 2024-09-12T02:17:37.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.828+0000 7fe9b5ffb640 1 -- 172.21.15.16:0/3607155874 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fe9c0005310 con 0x56128b6e45d0 2024-09-12T02:17:37.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.828+0000 7fe9d70df740 10 client.20715 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:17:37.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.828+0000 7fe9d70df740 10 client.20715 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-12T02:17:37.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.828+0000 7fe9d70df740 20 client.20715 populate_metadata read hostname 'smithi016' 2024-09-12T02:17:37.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.828+0000 7fe9d70df740 1 --2- 172.21.15.16:0/3607155874 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x56128b769980 0x56128b76bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:37.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.828+0000 7fe9d70df740 4 client.20715 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T02:17:37.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.828+0000 7fe9d70df740 1 -- 172.21.15.16:0/3607155874 --> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56128b651d70 con 0x56128b769980 2024-09-12T02:17:37.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.828+0000 7fe9c7fff640 1 --2- 172.21.15.16:0/3607155874 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x56128b769980 0x56128b76bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:37.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.829+0000 7fe9c7fff640 1 --2- 172.21.15.16:0/3607155874 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x56128b769980 0x56128b76bda0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e3eb27afb74c0327 server_cookie=1e863c14038bfabe in_seq=0 out_seq=0 2024-09-12T02:17:37.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.829+0000 7fe9b5ffb640 10 client.20715 ms_handle_connect on v2:172.21.15.184:6836/1793393087 2024-09-12T02:17:37.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.829+0000 7fe9b5ffb640 1 -- 172.21.15.16:0/3607155874 <== mds.2 v2:172.21.15.184:6836/1793393087 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56128b651d70 con 0x56128b769980 2024-09-12T02:17:37.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.829+0000 7fe9b5ffb640 10 client.20715 handle_command_reply: tid=0 2024-09-12T02:17:37.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.829+0000 7fe9d70df740 1 client.20715 shutdown 2024-09-12T02:17:37.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.829+0000 7fe9d70df740 1 -- 172.21.15.16:0/3607155874 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56128b6e45d0 msgr2=0x56128b6f7480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:37.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.829+0000 7fe9d70df740 1 --2- 172.21.15.16:0/3607155874 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56128b6e45d0 0x56128b6f7480 secure :-1 s=READY pgs=2090 cs=0 l=1 rev1=1 crypto rx=0x7fe9c009b590 tx=0x7fe9c00979d0 comp rx=0 tx=0).stop 2024-09-12T02:17:37.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.829+0000 7fe9d70df740 1 -- 172.21.15.16:0/3607155874 shutdown_connections 2024-09-12T02:17:37.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.829+0000 7fe9d70df740 1 --2- 172.21.15.16:0/3607155874 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56128b6ef370 0x56128b6e82b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:37.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.829+0000 7fe9d70df740 1 --2- 172.21.15.16:0/3607155874 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56128b6e5b40 0x56128b6f79c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:37.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.829+0000 7fe9d70df740 1 --2- 172.21.15.16:0/3607155874 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x56128b769980 0x56128b76bda0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:37.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.829+0000 7fe9d70df740 1 --2- 172.21.15.16:0/3607155874 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56128b6e45d0 0x56128b6f7480 unknown :-1 s=CLOSED pgs=2090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:37.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.829+0000 7fe9b5ffb640 0 client.20715 ms_handle_reset on v2:172.21.15.184:6836/1793393087 2024-09-12T02:17:37.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.829+0000 7fe9d70df740 1 -- 172.21.15.16:0/3607155874 >> 172.21.15.16:0/3607155874 conn(0x56128b6ad310 msgr2=0x56128b6ea7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:37.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.829+0000 7fe9d70df740 1 -- 172.21.15.16:0/3607155874 shutdown_connections 2024-09-12T02:17:37.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.829+0000 7fe9d70df740 1 -- 172.21.15.16:0/3607155874 wait complete. 2024-09-12T02:17:37.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.829+0000 7fe9d70df740 20 client.20715 trim_cache size 0 max 16384 2024-09-12T02:17:37.835 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T02:17:37.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.830+0000 7fe9d4d06640 1 -- 172.21.15.16:0/523964805 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fe9bc0a2c00 msgr2=0x7fe9bc0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:37.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.830+0000 7fe9d4d06640 1 --2- 172.21.15.16:0/523964805 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fe9bc0a2c00 0x7fe9bc0a50c0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fe9c8009430 tx=0x7fe9c804d000 comp rx=0 tx=0).stop 2024-09-12T02:17:37.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.830+0000 7fe9d4d06640 1 -- 172.21.15.16:0/523964805 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9d01008d0 msgr2=0x7fe9d019a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:37.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.830+0000 7fe9d4d06640 1 --2- 172.21.15.16:0/523964805 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9d01008d0 0x7fe9d019a670 secure :-1 s=READY pgs=2086 cs=0 l=1 rev1=1 crypto rx=0x7fe9c000c970 tx=0x7fe9c000ce30 comp rx=0 tx=0).stop 2024-09-12T02:17:37.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.830+0000 7fe9d4d06640 1 -- 172.21.15.16:0/523964805 shutdown_connections 2024-09-12T02:17:37.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.830+0000 7fe9d4d06640 1 --2- 172.21.15.16:0/523964805 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fe9bc0a2c00 0x7fe9bc0a50c0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:37.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.830+0000 7fe9d4d06640 1 --2- 172.21.15.16:0/523964805 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe9d010e480 0x7fe9d019ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:37.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.830+0000 7fe9d4d06640 1 --2- 172.21.15.16:0/523964805 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe9d0101280 0x7fe9d019abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:37.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.830+0000 7fe9d4d06640 1 --2- 172.21.15.16:0/523964805 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9d01008d0 0x7fe9d019a670 unknown :-1 s=CLOSED pgs=2086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:37.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.830+0000 7fe9d4d06640 1 -- 172.21.15.16:0/523964805 >> 172.21.15.16:0/523964805 conn(0x7fe9d00fc7a0 msgr2=0x7fe9d00feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:37.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.830+0000 7fe9d4d06640 1 -- 172.21.15.16:0/523964805 shutdown_connections 2024-09-12T02:17:37.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:37.830+0000 7fe9d4d06640 1 -- 172.21.15.16:0/523964805 wait complete. 2024-09-12T02:17:38.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:17:37 smithi184 ceph-mon[35593]: pgmap v806: 97 pgs: 97 active+clean; 4.2 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-09-12T02:17:38.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:17:37 smithi033 ceph-mon[35332]: pgmap v806: 97 pgs: 97 active+clean; 4.2 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-09-12T02:17:38.183 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:17:37 smithi016 ceph-mon[30947]: pgmap v806: 97 pgs: 97 active+clean; 4.2 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-09-12T02:17:38.730 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-12T02:17:38.992 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:17:39.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.511+0000 7f5f5fcb0640 1 -- 172.21.15.16:0/3849118303 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f58102790 msgr2=0x7f5f581067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:39.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.511+0000 7f5f5fcb0640 1 --2- 172.21.15.16:0/3849118303 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f58102790 0x7f5f581067e0 secure :-1 s=READY pgs=2091 cs=0 l=1 rev1=1 crypto rx=0x7f5f48009900 tx=0x7f5f4802a410 comp rx=0 tx=0).stop 2024-09-12T02:17:39.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.511+0000 7f5f5fcb0640 1 -- 172.21.15.16:0/3849118303 shutdown_connections 2024-09-12T02:17:39.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.511+0000 7f5f5fcb0640 1 --2- 172.21.15.16:0/3849118303 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5f58106f10 0x7f5f5810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:39.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.511+0000 7f5f5fcb0640 1 --2- 172.21.15.16:0/3849118303 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f58102790 0x7f5f581067e0 unknown :-1 s=CLOSED pgs=2091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:39.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.511+0000 7f5f5fcb0640 1 --2- 172.21.15.16:0/3849118303 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f58101de0 0x7f5f581021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:39.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.511+0000 7f5f5fcb0640 1 -- 172.21.15.16:0/3849118303 >> 172.21.15.16:0/3849118303 conn(0x7f5f580fd8c0 msgr2=0x7f5f580ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:39.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.511+0000 7f5f5fcb0640 1 -- 172.21.15.16:0/3849118303 shutdown_connections 2024-09-12T02:17:39.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.511+0000 7f5f5fcb0640 1 -- 172.21.15.16:0/3849118303 wait complete. 2024-09-12T02:17:39.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.512+0000 7f5f5fcb0640 1 Processor -- start 2024-09-12T02:17:39.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.512+0000 7f5f5fcb0640 1 -- start start 2024-09-12T02:17:39.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.512+0000 7f5f5fcb0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f58101de0 0x7f5f58198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:39.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.512+0000 7f5f5fcb0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f58102790 0x7f5f58198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:39.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.512+0000 7f5f5fcb0640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5f58106f10 0x7f5f5819ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:39.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.512+0000 7f5f5fcb0640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5f58111f90 con 0x7f5f58102790 2024-09-12T02:17:39.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.512+0000 7f5f5fcb0640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5f58111e10 con 0x7f5f58101de0 2024-09-12T02:17:39.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.512+0000 7f5f5fcb0640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5f58112110 con 0x7f5f58106f10 2024-09-12T02:17:39.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.512+0000 7f5f5d24b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f58102790 0x7f5f58198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:39.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.512+0000 7f5f5d24b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f58102790 0x7f5f58198960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53818/0 (socket says 172.21.15.16:53818) 2024-09-12T02:17:39.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.512+0000 7f5f5d24b640 1 -- 172.21.15.16:0/1354757146 learned_addr learned my addr 172.21.15.16:0/1354757146 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:17:39.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.512+0000 7f5f5e24d640 1 --2- 172.21.15.16:0/1354757146 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5f58106f10 0x7f5f5819ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:39.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.512+0000 7f5f5da4c640 1 --2- 172.21.15.16:0/1354757146 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f58101de0 0x7f5f58198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:39.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.512+0000 7f5f5d24b640 1 -- 172.21.15.16:0/1354757146 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5f58106f10 msgr2=0x7f5f5819ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:39.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.512+0000 7f5f5d24b640 1 --2- 172.21.15.16:0/1354757146 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5f58106f10 0x7f5f5819ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:39.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.512+0000 7f5f5d24b640 1 -- 172.21.15.16:0/1354757146 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f58101de0 msgr2=0x7f5f58198420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:39.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.512+0000 7f5f5d24b640 1 --2- 172.21.15.16:0/1354757146 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f58101de0 0x7f5f58198420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:39.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.512+0000 7f5f5d24b640 1 -- 172.21.15.16:0/1354757146 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f5819d3d0 con 0x7f5f58102790 2024-09-12T02:17:39.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.512+0000 7f5f5da4c640 1 --2- 172.21.15.16:0/1354757146 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f58101de0 0x7f5f58198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:17:39.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.512+0000 7f5f5e24d640 1 --2- 172.21.15.16:0/1354757146 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5f58106f10 0x7f5f5819ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:17:39.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.513+0000 7f5f5d24b640 1 --2- 172.21.15.16:0/1354757146 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f58102790 0x7f5f58198960 secure :-1 s=READY pgs=2092 cs=0 l=1 rev1=1 crypto rx=0x7f5f48005ab0 tx=0x7f5f480042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:39.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.513+0000 7f5f4effd640 1 -- 172.21.15.16:0/1354757146 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f4803f030 con 0x7f5f58102790 2024-09-12T02:17:39.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.513+0000 7f5f4effd640 1 -- 172.21.15.16:0/1354757146 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f48003ce0 con 0x7f5f58102790 2024-09-12T02:17:39.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.513+0000 7f5f5fcb0640 1 -- 172.21.15.16:0/1354757146 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f5819d660 con 0x7f5f58102790 2024-09-12T02:17:39.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.513+0000 7f5f4effd640 1 -- 172.21.15.16:0/1354757146 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f48035070 con 0x7f5f58102790 2024-09-12T02:17:39.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.513+0000 7f5f5fcb0640 1 -- 172.21.15.16:0/1354757146 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f581a4fa0 con 0x7f5f58102790 2024-09-12T02:17:39.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.515+0000 7f5f4effd640 1 -- 172.21.15.16:0/1354757146 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5f48042030 con 0x7f5f58102790 2024-09-12T02:17:39.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.515+0000 7f5f62089740 1 Processor -- start 2024-09-12T02:17:39.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.516+0000 7f5f62089740 1 -- start start 2024-09-12T02:17:39.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.516+0000 7f5f62089740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c96b767b40 0x55c96b766090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:39.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.516+0000 7f5f62089740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c96b7665d0 0x55c96b766a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:39.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.516+0000 7f5f62089740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c96b771370 0x55c96b773760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:39.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.516+0000 7f5f62089740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55c96b6cc2f0 con 0x55c96b7665d0 2024-09-12T02:17:39.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.516+0000 7f5f62089740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55c96b673cf0 con 0x55c96b771370 2024-09-12T02:17:39.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.516+0000 7f5f62089740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55c96b6c82b0 con 0x55c96b767b40 2024-09-12T02:17:39.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.516+0000 7f5f5da4c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c96b7665d0 0x55c96b766a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:39.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.516+0000 7f5f5da4c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c96b7665d0 0x55c96b766a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53824/0 (socket says 172.21.15.16:53824) 2024-09-12T02:17:39.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.516+0000 7f5f5da4c640 1 -- 172.21.15.16:0/4119900569 learned_addr learned my addr 172.21.15.16:0/4119900569 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:17:39.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.516+0000 7f5f5e24d640 1 --2- 172.21.15.16:0/4119900569 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c96b767b40 0x55c96b766090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:39.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.516+0000 7f5f4effd640 1 --2- 172.21.15.16:0/1354757146 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f5f440a2c00 0x7f5f440a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:39.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.516+0000 7f5f5d24b640 1 --2- 172.21.15.16:0/4119900569 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c96b771370 0x55c96b773760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:39.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.516+0000 7f5f5da4c640 1 -- 172.21.15.16:0/4119900569 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c96b767b40 msgr2=0x55c96b766090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:39.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.516+0000 7f5f5da4c640 1 --2- 172.21.15.16:0/4119900569 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c96b767b40 0x55c96b766090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:39.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.516+0000 7f5f5da4c640 1 -- 172.21.15.16:0/4119900569 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c96b771370 msgr2=0x55c96b773760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:39.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.516+0000 7f5f5da4c640 1 --2- 172.21.15.16:0/4119900569 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c96b771370 0x55c96b773760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:39.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.516+0000 7f5f5da4c640 1 -- 172.21.15.16:0/4119900569 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c96b6583a0 con 0x55c96b7665d0 2024-09-12T02:17:39.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.516+0000 7f5f5e24d640 1 --2- 172.21.15.16:0/4119900569 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c96b767b40 0x55c96b766090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:17:39.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.516+0000 7f5f5e24d640 1 --2- 172.21.15.16:0/1354757146 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f5f440a2c00 0x7f5f440a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:39.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.516+0000 7f5f4effd640 1 -- 172.21.15.16:0/1354757146 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f5f480ef8e0 con 0x7f5f58102790 2024-09-12T02:17:39.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.517+0000 7f5f5da4c640 1 --2- 172.21.15.16:0/4119900569 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c96b7665d0 0x55c96b766a10 secure :-1 s=READY pgs=2093 cs=0 l=1 rev1=1 crypto rx=0x7f5f5400bc20 tx=0x7f5f5400d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b42eca8e073a3f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:39.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.517+0000 7f5f43fff640 1 -- 172.21.15.16:0/4119900569 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f54017620 con 0x55c96b7665d0 2024-09-12T02:17:39.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.517+0000 7f5f43fff640 1 -- 172.21.15.16:0/4119900569 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f54017ca0 con 0x55c96b7665d0 2024-09-12T02:17:39.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.517+0000 7f5f43fff640 1 -- 172.21.15.16:0/4119900569 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f5400fbf0 con 0x55c96b7665d0 2024-09-12T02:17:39.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.518+0000 7f5f5e24d640 1 --2- 172.21.15.16:0/1354757146 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f5f440a2c00 0x7f5f440a50c0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f5f5000c4e0 tx=0x7f5f5000e040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:39.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.518+0000 7f5f62089740 1 -- 172.21.15.16:0/4119900569 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c96b7665d0 msgr2=0x55c96b766a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:39.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.518+0000 7f5f62089740 1 --2- 172.21.15.16:0/4119900569 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c96b7665d0 0x55c96b766a10 secure :-1 s=READY pgs=2093 cs=0 l=1 rev1=1 crypto rx=0x7f5f5400bc20 tx=0x7f5f5400d710 comp rx=0 tx=0).stop 2024-09-12T02:17:39.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.518+0000 7f5f62089740 1 -- 172.21.15.16:0/4119900569 shutdown_connections 2024-09-12T02:17:39.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.518+0000 7f5f62089740 1 --2- 172.21.15.16:0/4119900569 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c96b771370 0x55c96b773760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:39.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.518+0000 7f5f62089740 1 --2- 172.21.15.16:0/4119900569 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c96b7665d0 0x55c96b766a10 unknown :-1 s=CLOSED pgs=2093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:39.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.518+0000 7f5f62089740 1 --2- 172.21.15.16:0/4119900569 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c96b767b40 0x55c96b766090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:39.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.518+0000 7f5f62089740 1 -- 172.21.15.16:0/4119900569 >> 172.21.15.16:0/4119900569 conn(0x55c96b72f310 msgr2=0x55c96b762c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:39.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.518+0000 7f5f62089740 1 -- 172.21.15.16:0/4119900569 shutdown_connections 2024-09-12T02:17:39.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.518+0000 7f5f62089740 1 -- 172.21.15.16:0/4119900569 wait complete. 2024-09-12T02:17:39.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.518+0000 7f5f62089740 1 -- start start 2024-09-12T02:17:39.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.518+0000 7f5f62089740 1 Processor -- start 2024-09-12T02:17:39.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.519+0000 7f5f62089740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c96b7665d0 0x55c96b770d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:39.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.519+0000 7f5f62089740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c96b767b40 0x55c96b76dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:39.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.519+0000 7f5f62089740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c96b771370 0x55c96b76e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:39.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.519+0000 7f5f62089740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55c96b774070 con 0x55c96b771370 2024-09-12T02:17:39.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.519+0000 7f5f62089740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55c96b687060 con 0x55c96b767b40 2024-09-12T02:17:39.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.519+0000 7f5f62089740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55c96b7741f0 con 0x55c96b7665d0 2024-09-12T02:17:39.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.519+0000 7f5f62089740 10 client.0 _ll_register_callbacks cb 0x55c96b3da8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:17:39.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.519+0000 7f5f5e24d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c96b771370 0x55c96b76e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:39.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.519+0000 7f5f5e24d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c96b771370 0x55c96b76e530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53828/0 (socket says 172.21.15.16:53828) 2024-09-12T02:17:39.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.519+0000 7f5f5e24d640 1 -- 172.21.15.16:0/1634365128 learned_addr learned my addr 172.21.15.16:0/1634365128 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:17:39.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.519+0000 7f5f5d24b640 1 --2- 172.21.15.16:0/1634365128 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c96b767b40 0x55c96b76dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:39.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.519+0000 7f5f5da4c640 1 --2- 172.21.15.16:0/1634365128 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c96b7665d0 0x55c96b770d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:39.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.519+0000 7f5f5e24d640 1 -- 172.21.15.16:0/1634365128 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c96b7665d0 msgr2=0x55c96b770d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:39.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.519+0000 7f5f5e24d640 1 --2- 172.21.15.16:0/1634365128 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c96b7665d0 0x55c96b770d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:39.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.519+0000 7f5f5e24d640 1 -- 172.21.15.16:0/1634365128 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c96b767b40 msgr2=0x55c96b76dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:39.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.519+0000 7f5f5e24d640 1 --2- 172.21.15.16:0/1634365128 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c96b767b40 0x55c96b76dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:39.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.519+0000 7f5f5e24d640 1 -- 172.21.15.16:0/1634365128 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c96b43d3c0 con 0x55c96b771370 2024-09-12T02:17:39.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.519+0000 7f5f5e24d640 1 --2- 172.21.15.16:0/1634365128 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c96b771370 0x55c96b76e530 secure :-1 s=READY pgs=2094 cs=0 l=1 rev1=1 crypto rx=0x7f5f50016a60 tx=0x7f5f50004b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bb101a755080be server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:39.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.519+0000 7f5f417fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:17:39.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.519+0000 7f5f417fa640 1 -- 172.21.15.16:0/1634365128 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f50026060 con 0x55c96b771370 2024-09-12T02:17:39.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.520+0000 7f5f417fa640 1 -- 172.21.15.16:0/1634365128 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f5f5000ad90 con 0x55c96b771370 2024-09-12T02:17:39.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.520+0000 7f5f62089740 1 -- 172.21.15.16:0/1634365128 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f54017ca0 con 0x55c96b771370 2024-09-12T02:17:39.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.520+0000 7f5f417fa640 1 -- 172.21.15.16:0/1634365128 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5f50021400 con 0x55c96b771370 2024-09-12T02:17:39.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.520+0000 7f5f62089740 10 client.20727 fetch_fsmap learned FSMap version 25 2024-09-12T02:17:39.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.520+0000 7f5f62089740 1 -- 172.21.15.16:0/1634365128 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c96b77fa70 con 0x55c96b771370 2024-09-12T02:17:39.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.520+0000 7f5f417fa640 1 -- 172.21.15.16:0/1634365128 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f54017ca0 con 0x55c96b771370 2024-09-12T02:17:39.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.520+0000 7f5f417fa640 1 -- 172.21.15.16:0/1634365128 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f5f50007730 con 0x55c96b771370 2024-09-12T02:17:39.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.520+0000 7f5f62089740 10 client.20727 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:17:39.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.520+0000 7f5f62089740 10 client.20727 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T02:17:39.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.520+0000 7f5f62089740 20 client.20727 populate_metadata read hostname 'smithi016' 2024-09-12T02:17:39.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.520+0000 7f5f62089740 1 --2- 172.21.15.16:0/1634365128 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x55c96b780f30 0x55c96b783380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:39.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.520+0000 7f5f62089740 4 client.20727 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:17:39.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.520+0000 7f5f62089740 1 -- 172.21.15.16:0/1634365128 --> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c96b6d9860 con 0x55c96b780f30 2024-09-12T02:17:39.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.520+0000 7f5f5da4c640 1 --2- 172.21.15.16:0/1634365128 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x55c96b780f30 0x55c96b783380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:39.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.521+0000 7f5f417fa640 10 client.20727 ms_handle_connect on v2:172.21.15.184:6838/4203347941 2024-09-12T02:17:39.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.521+0000 7f5f5da4c640 1 --2- 172.21.15.16:0/1634365128 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x55c96b780f30 0x55c96b783380 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3ac7790190957e86 server_cookie=a074cc61a65baa76 in_seq=0 out_seq=0 2024-09-12T02:17:39.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.522+0000 7f5f417fa640 1 -- 172.21.15.16:0/1634365128 <== mds.1 v2:172.21.15.184:6838/4203347941 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c96b6d9860 con 0x55c96b780f30 2024-09-12T02:17:39.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.522+0000 7f5f417fa640 10 client.20727 handle_command_reply: tid=0 2024-09-12T02:17:39.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.522+0000 7f5f62089740 1 client.20727 shutdown 2024-09-12T02:17:39.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.522+0000 7f5f62089740 1 -- 172.21.15.16:0/1634365128 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c96b771370 msgr2=0x55c96b76e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:39.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.522+0000 7f5f62089740 1 --2- 172.21.15.16:0/1634365128 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c96b771370 0x55c96b76e530 secure :-1 s=READY pgs=2094 cs=0 l=1 rev1=1 crypto rx=0x7f5f50016a60 tx=0x7f5f50004b50 comp rx=0 tx=0).stop 2024-09-12T02:17:39.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.522+0000 7f5f62089740 1 -- 172.21.15.16:0/1634365128 shutdown_connections 2024-09-12T02:17:39.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.522+0000 7f5f62089740 1 --2- 172.21.15.16:0/1634365128 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x55c96b780f30 0x55c96b783380 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:39.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.522+0000 7f5f62089740 1 --2- 172.21.15.16:0/1634365128 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c96b771370 0x55c96b76e530 unknown :-1 s=CLOSED pgs=2094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:39.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.522+0000 7f5f62089740 1 --2- 172.21.15.16:0/1634365128 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c96b767b40 0x55c96b76dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:39.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.522+0000 7f5f417fa640 0 client.20727 ms_handle_reset on v2:172.21.15.184:6838/4203347941 2024-09-12T02:17:39.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.522+0000 7f5f62089740 1 --2- 172.21.15.16:0/1634365128 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c96b7665d0 0x55c96b770d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:39.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.522+0000 7f5f62089740 1 -- 172.21.15.16:0/1634365128 >> 172.21.15.16:0/1634365128 conn(0x55c96b72f310 msgr2=0x55c96b768960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:39.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.522+0000 7f5f62089740 1 -- 172.21.15.16:0/1634365128 shutdown_connections 2024-09-12T02:17:39.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.522+0000 7f5f62089740 1 -- 172.21.15.16:0/1634365128 wait complete. 2024-09-12T02:17:39.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.522+0000 7f5f62089740 20 client.20727 trim_cache size 0 max 16384 2024-09-12T02:17:39.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.538+0000 7f5f62089740 1 Processor -- start 2024-09-12T02:17:39.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.538+0000 7f5f62089740 1 -- start start 2024-09-12T02:17:39.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.538+0000 7f5f62089740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c96b7665d0 0x55c96b77d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:39.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.538+0000 7f5f62089740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c96b767b40 0x55c96b76d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:39.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.538+0000 7f5f62089740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c96b771370 0x55c96b772760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:39.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.538+0000 7f5f62089740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5f581afcd0 con 0x55c96b767b40 2024-09-12T02:17:39.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.538+0000 7f5f62089740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5f581ad5a0 con 0x55c96b7665d0 2024-09-12T02:17:39.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.538+0000 7f5f62089740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5f581ab410 con 0x55c96b771370 2024-09-12T02:17:39.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.538+0000 7f5f5d24b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c96b767b40 0x55c96b76d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:39.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.538+0000 7f5f5d24b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c96b767b40 0x55c96b76d870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53842/0 (socket says 172.21.15.16:53842) 2024-09-12T02:17:39.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.538+0000 7f5f5e24d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c96b771370 0x55c96b772760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:39.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.538+0000 7f5f5d24b640 1 -- 172.21.15.16:0/3088029255 learned_addr learned my addr 172.21.15.16:0/3088029255 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:17:39.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.538+0000 7f5f5da4c640 1 --2- 172.21.15.16:0/3088029255 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c96b7665d0 0x55c96b77d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:39.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.538+0000 7f5f5e24d640 1 -- 172.21.15.16:0/3088029255 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c96b7665d0 msgr2=0x55c96b77d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:39.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.538+0000 7f5f5e24d640 1 --2- 172.21.15.16:0/3088029255 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c96b7665d0 0x55c96b77d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:39.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.538+0000 7f5f5e24d640 1 -- 172.21.15.16:0/3088029255 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c96b767b40 msgr2=0x55c96b76d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:39.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.538+0000 7f5f5e24d640 1 --2- 172.21.15.16:0/3088029255 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c96b767b40 0x55c96b76d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:39.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.538+0000 7f5f5e24d640 1 -- 172.21.15.16:0/3088029255 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c96b783a00 con 0x55c96b771370 2024-09-12T02:17:39.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.539+0000 7f5f5e24d640 1 --2- 172.21.15.16:0/3088029255 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c96b771370 0x55c96b772760 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5f5000c040 tx=0x7f5f50016770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ec2ee8924695b49a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:39.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.539+0000 7f5f437fe640 1 -- 172.21.15.16:0/3088029255 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f50030070 con 0x55c96b771370 2024-09-12T02:17:39.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.539+0000 7f5f437fe640 1 -- 172.21.15.16:0/3088029255 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f500078b0 con 0x55c96b771370 2024-09-12T02:17:39.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.539+0000 7f5f437fe640 1 -- 172.21.15.16:0/3088029255 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f500215a0 con 0x55c96b771370 2024-09-12T02:17:39.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.540+0000 7f5f62089740 1 -- 172.21.15.16:0/3088029255 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c96b771370 msgr2=0x55c96b772760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:39.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.540+0000 7f5f62089740 1 --2- 172.21.15.16:0/3088029255 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c96b771370 0x55c96b772760 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5f5000c040 tx=0x7f5f50016770 comp rx=0 tx=0).stop 2024-09-12T02:17:39.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.540+0000 7f5f62089740 1 -- 172.21.15.16:0/3088029255 shutdown_connections 2024-09-12T02:17:39.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.540+0000 7f5f62089740 1 --2- 172.21.15.16:0/3088029255 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c96b771370 0x55c96b772760 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:39.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.540+0000 7f5f62089740 1 --2- 172.21.15.16:0/3088029255 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c96b767b40 0x55c96b76d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:39.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.540+0000 7f5f62089740 1 --2- 172.21.15.16:0/3088029255 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c96b7665d0 0x55c96b77d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:39.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.540+0000 7f5f62089740 1 -- 172.21.15.16:0/3088029255 >> 172.21.15.16:0/3088029255 conn(0x55c96b72f310 msgr2=0x55c96b76c880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:39.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.540+0000 7f5f62089740 1 -- 172.21.15.16:0/3088029255 shutdown_connections 2024-09-12T02:17:39.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.540+0000 7f5f62089740 1 -- 172.21.15.16:0/3088029255 wait complete. 2024-09-12T02:17:39.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.540+0000 7f5f62089740 1 -- start start 2024-09-12T02:17:39.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.540+0000 7f5f62089740 1 Processor -- start 2024-09-12T02:17:39.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.541+0000 7f5f62089740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c96b7665d0 0x55c96b779be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:39.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.541+0000 7f5f62089740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c96b767b40 0x55c96b788570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:39.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.541+0000 7f5f62089740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c96b771370 0x55c96b788ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:39.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.541+0000 7f5f62089740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5f581aba60 con 0x55c96b771370 2024-09-12T02:17:39.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.541+0000 7f5f62089740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5f581aced0 con 0x55c96b767b40 2024-09-12T02:17:39.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.541+0000 7f5f62089740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5f581ac130 con 0x55c96b7665d0 2024-09-12T02:17:39.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.541+0000 7f5f5e24d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c96b771370 0x55c96b788ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:39.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.541+0000 7f5f5e24d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c96b771370 0x55c96b788ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53852/0 (socket says 172.21.15.16:53852) 2024-09-12T02:17:39.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.541+0000 7f5f5e24d640 1 -- 172.21.15.16:0/272989939 learned_addr learned my addr 172.21.15.16:0/272989939 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:17:39.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.541+0000 7f5f5da4c640 1 --2- 172.21.15.16:0/272989939 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c96b7665d0 0x55c96b779be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:39.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.541+0000 7f5f62089740 10 client.0 _ll_register_callbacks cb 0x7f5f50009c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:17:39.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.541+0000 7f5f5d24b640 1 --2- 172.21.15.16:0/272989939 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c96b767b40 0x55c96b788570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:39.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.542+0000 7f5f5da4c640 1 -- 172.21.15.16:0/272989939 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c96b767b40 msgr2=0x55c96b788570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:39.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.542+0000 7f5f5da4c640 1 --2- 172.21.15.16:0/272989939 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c96b767b40 0x55c96b788570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:39.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.542+0000 7f5f5da4c640 1 -- 172.21.15.16:0/272989939 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c96b771370 msgr2=0x55c96b788ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:39.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.542+0000 7f5f5da4c640 1 --2- 172.21.15.16:0/272989939 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c96b771370 0x55c96b788ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:39.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.542+0000 7f5f5da4c640 1 -- 172.21.15.16:0/272989939 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c96b7892e0 con 0x55c96b7665d0 2024-09-12T02:17:39.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.542+0000 7f5f5e24d640 1 --2- 172.21.15.16:0/272989939 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c96b771370 0x55c96b788ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:17:39.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.542+0000 7f5f5da4c640 1 --2- 172.21.15.16:0/272989939 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c96b7665d0 0x55c96b779be0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f5f54000c00 tx=0x7f5f5400f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b1d073389139fb6a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:39.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.542+0000 7f5f417fa640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-09-12T02:17:39.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.542+0000 7f5f417fa640 1 -- 172.21.15.16:0/272989939 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f5402d070 con 0x55c96b7665d0 2024-09-12T02:17:39.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.542+0000 7f5f62089740 1 -- 172.21.15.16:0/272989939 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f500078b0 con 0x55c96b7665d0 2024-09-12T02:17:39.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.542+0000 7f5f417fa640 1 -- 172.21.15.16:0/272989939 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f5f54005040 con 0x55c96b7665d0 2024-09-12T02:17:39.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.542+0000 7f5f417fa640 1 -- 172.21.15.16:0/272989939 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5f54005260 con 0x55c96b7665d0 2024-09-12T02:17:39.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.542+0000 7f5f62089740 10 client.28999 fetch_fsmap learned FSMap version 25 2024-09-12T02:17:39.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.542+0000 7f5f62089740 1 -- 172.21.15.16:0/272989939 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c96b7ebc20 con 0x55c96b7665d0 2024-09-12T02:17:39.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.542+0000 7f5f417fa640 1 -- 172.21.15.16:0/272989939 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f500078b0 con 0x55c96b7665d0 2024-09-12T02:17:39.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.543+0000 7f5f417fa640 1 -- 172.21.15.16:0/272989939 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f5f54004660 con 0x55c96b7665d0 2024-09-12T02:17:39.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.543+0000 7f5f62089740 10 client.28999 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:17:39.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.543+0000 7f5f62089740 10 client.28999 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T02:17:39.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.543+0000 7f5f62089740 20 client.28999 populate_metadata read hostname 'smithi016' 2024-09-12T02:17:39.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.543+0000 7f5f62089740 1 --2- 172.21.15.16:0/272989939 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x55c96b7ec070 0x55c96b7ee460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:39.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.543+0000 7f5f62089740 4 client.28999 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T02:17:39.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.543+0000 7f5f62089740 1 -- 172.21.15.16:0/272989939 --> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c96b6d3d70 con 0x55c96b7ec070 2024-09-12T02:17:39.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.543+0000 7f5f5d24b640 1 --2- 172.21.15.16:0/272989939 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x55c96b7ec070 0x55c96b7ee460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:39.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.543+0000 7f5f5d24b640 1 --2- 172.21.15.16:0/272989939 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x55c96b7ec070 0x55c96b7ee460 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=52346d1f16ffc2e1 server_cookie=6221ccfca9ac2364 in_seq=0 out_seq=0 2024-09-12T02:17:39.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.543+0000 7f5f417fa640 10 client.28999 ms_handle_connect on v2:172.21.15.184:6838/4203347941 2024-09-12T02:17:39.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.543+0000 7f5f417fa640 1 -- 172.21.15.16:0/272989939 <== mds.1 v2:172.21.15.184:6838/4203347941 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c96b6d3d70 con 0x55c96b7ec070 2024-09-12T02:17:39.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.543+0000 7f5f417fa640 10 client.28999 handle_command_reply: tid=0 2024-09-12T02:17:39.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.544+0000 7f5f62089740 1 client.28999 shutdown 2024-09-12T02:17:39.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.544+0000 7f5f62089740 1 -- 172.21.15.16:0/272989939 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c96b7665d0 msgr2=0x55c96b779be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:39.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.544+0000 7f5f62089740 1 --2- 172.21.15.16:0/272989939 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c96b7665d0 0x55c96b779be0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f5f54000c00 tx=0x7f5f5400f790 comp rx=0 tx=0).stop 2024-09-12T02:17:39.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.544+0000 7f5f62089740 1 -- 172.21.15.16:0/272989939 shutdown_connections 2024-09-12T02:17:39.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.544+0000 7f5f62089740 1 --2- 172.21.15.16:0/272989939 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x55c96b7ec070 0x55c96b7ee460 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:39.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.544+0000 7f5f62089740 1 --2- 172.21.15.16:0/272989939 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c96b771370 0x55c96b788ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:39.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.544+0000 7f5f62089740 1 --2- 172.21.15.16:0/272989939 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c96b767b40 0x55c96b788570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:39.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.544+0000 7f5f417fa640 0 client.28999 ms_handle_reset on v2:172.21.15.184:6838/4203347941 2024-09-12T02:17:39.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.544+0000 7f5f62089740 1 --2- 172.21.15.16:0/272989939 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c96b7665d0 0x55c96b779be0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:39.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.544+0000 7f5f62089740 1 -- 172.21.15.16:0/272989939 >> 172.21.15.16:0/272989939 conn(0x55c96b72f310 msgr2=0x55c96b76f810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:39.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.544+0000 7f5f62089740 1 -- 172.21.15.16:0/272989939 shutdown_connections 2024-09-12T02:17:39.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.544+0000 7f5f62089740 1 -- 172.21.15.16:0/272989939 wait complete. 2024-09-12T02:17:39.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.544+0000 7f5f62089740 20 client.28999 trim_cache size 0 max 16384 2024-09-12T02:17:39.554 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T02:17:39.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.544+0000 7f5f5fcb0640 1 -- 172.21.15.16:0/1354757146 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f5f440a2c00 msgr2=0x7f5f440a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:39.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.544+0000 7f5f5fcb0640 1 --2- 172.21.15.16:0/1354757146 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f5f440a2c00 0x7f5f440a50c0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f5f5000c4e0 tx=0x7f5f5000e040 comp rx=0 tx=0).stop 2024-09-12T02:17:39.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.544+0000 7f5f5fcb0640 1 -- 172.21.15.16:0/1354757146 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f58102790 msgr2=0x7f5f58198960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:39.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.544+0000 7f5f5fcb0640 1 --2- 172.21.15.16:0/1354757146 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f58102790 0x7f5f58198960 secure :-1 s=READY pgs=2092 cs=0 l=1 rev1=1 crypto rx=0x7f5f48005ab0 tx=0x7f5f480042f0 comp rx=0 tx=0).stop 2024-09-12T02:17:39.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.545+0000 7f5f5fcb0640 1 -- 172.21.15.16:0/1354757146 shutdown_connections 2024-09-12T02:17:39.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.545+0000 7f5f5fcb0640 1 --2- 172.21.15.16:0/1354757146 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f5f440a2c00 0x7f5f440a50c0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:39.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.545+0000 7f5f5fcb0640 1 --2- 172.21.15.16:0/1354757146 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5f58106f10 0x7f5f5819ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:39.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.545+0000 7f5f5fcb0640 1 --2- 172.21.15.16:0/1354757146 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f58102790 0x7f5f58198960 unknown :-1 s=CLOSED pgs=2092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:39.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.545+0000 7f5f5fcb0640 1 --2- 172.21.15.16:0/1354757146 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f58101de0 0x7f5f58198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:39.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.545+0000 7f5f5fcb0640 1 -- 172.21.15.16:0/1354757146 >> 172.21.15.16:0/1354757146 conn(0x7f5f580fd8c0 msgr2=0x7f5f580fec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:39.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.545+0000 7f5f5fcb0640 1 -- 172.21.15.16:0/1354757146 shutdown_connections 2024-09-12T02:17:39.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:39.545+0000 7f5f5fcb0640 1 -- 172.21.15.16:0/1354757146 wait complete. 2024-09-12T02:17:40.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:17:39 smithi184 ceph-mon[35593]: pgmap v807: 97 pgs: 97 active+clean; 4.2 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-09-12T02:17:40.138 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:17:39 smithi016 ceph-mon[30947]: pgmap v807: 97 pgs: 97 active+clean; 4.2 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-09-12T02:17:40.167 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-12T02:17:40.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:17:39 smithi033 ceph-mon[35332]: pgmap v807: 97 pgs: 97 active+clean; 4.2 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-09-12T02:17:40.423 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:17:40.590 INFO:tasks.workunit.client.0.smithi016.stdout:561/600: open fd = 3 2024-09-12T02:17:40.590 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:40.590 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:40.591 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:17:40.591 INFO:tasks.workunit.client.0.smithi016.stdout:562/600: open fd = 3 2024-09-12T02:17:40.591 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:40.591 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:40.591 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:17:40.591 INFO:tasks.workunit.client.0.smithi016.stdout:563/600: open fd = 3 2024-09-12T02:17:40.591 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:40.591 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:40.591 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:17:40.591 INFO:tasks.workunit.client.0.smithi016.stdout:564/600: open fd = 3 2024-09-12T02:17:40.591 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:40.591 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:40.591 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:17:40.591 INFO:tasks.workunit.client.0.smithi016.stdout:565/600: open fd = 3 2024-09-12T02:17:40.592 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:40.592 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:40.592 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:17:40.592 INFO:tasks.workunit.client.0.smithi016.stdout:566/600: open fd = 3 2024-09-12T02:17:40.592 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:40.592 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:40.592 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:17:40.592 INFO:tasks.workunit.client.0.smithi016.stdout:567/600: open fd = 3 2024-09-12T02:17:40.592 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:40.592 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:40.592 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:17:40.592 INFO:tasks.workunit.client.0.smithi016.stdout:568/600: open fd = 3 2024-09-12T02:17:40.592 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:40.593 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:40.593 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:17:40.593 INFO:tasks.workunit.client.0.smithi016.stdout:569/600: open fd = 3 2024-09-12T02:17:40.593 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:40.593 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:40.593 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:17:40.593 INFO:tasks.workunit.client.0.smithi016.stdout:570/600: open fd = 3 2024-09-12T02:17:40.593 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:40.593 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:40.593 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:17:40.593 INFO:tasks.workunit.client.0.smithi016.stdout:571/600: open fd = 3 2024-09-12T02:17:40.593 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:40.593 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:40.593 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:17:40.594 INFO:tasks.workunit.client.0.smithi016.stdout:572/600: open fd = 3 2024-09-12T02:17:40.594 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:40.594 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:40.594 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:17:40.594 INFO:tasks.workunit.client.0.smithi016.stdout:573/600: open fd = 3 2024-09-12T02:17:40.594 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:40.594 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:40.594 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:17:40.594 INFO:tasks.workunit.client.0.smithi016.stdout:574/600: open fd = 3 2024-09-12T02:17:40.594 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:40.594 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:40.594 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:17:40.594 INFO:tasks.workunit.client.0.smithi016.stdout:575/600: open fd = 3 2024-09-12T02:17:40.595 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:40.595 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:40.595 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:17:40.595 INFO:tasks.workunit.client.0.smithi016.stdout:576/600: open fd = 3 2024-09-12T02:17:40.595 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:40.595 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:40.595 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:17:40.595 INFO:tasks.workunit.client.0.smithi016.stdout:577/600: open fd = 3 2024-09-12T02:17:40.595 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:40.595 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:40.595 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:17:40.595 INFO:tasks.workunit.client.0.smithi016.stdout:578/600: open fd = 3 2024-09-12T02:17:40.595 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:40.595 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:40.596 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:17:40.596 INFO:tasks.workunit.client.0.smithi016.stdout:579/600: open fd = 3 2024-09-12T02:17:40.596 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:40.596 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:40.596 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:17:40.596 INFO:tasks.workunit.client.0.smithi016.stdout:580/600: open fd = 3 2024-09-12T02:17:40.596 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:40.596 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:40.596 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:17:40.596 INFO:tasks.workunit.client.0.smithi016.stdout:581/600: open fd = 3 2024-09-12T02:17:40.596 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:40.596 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:40.596 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:17:40.597 INFO:tasks.workunit.client.0.smithi016.stdout:582/600: open fd = 3 2024-09-12T02:17:40.597 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:40.597 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:40.597 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:17:40.597 INFO:tasks.workunit.client.0.smithi016.stdout:583/600: open fd = 3 2024-09-12T02:17:40.597 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:40.597 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:40.597 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:17:40.597 INFO:tasks.workunit.client.0.smithi016.stdout:584/600: open fd = 3 2024-09-12T02:17:40.597 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:40.597 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:40.597 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:17:40.597 INFO:tasks.workunit.client.0.smithi016.stdout:585/600: open fd = 3 2024-09-12T02:17:40.598 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:40.598 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:40.598 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:17:40.598 INFO:tasks.workunit.client.0.smithi016.stdout:586/600: open fd = 3 2024-09-12T02:17:40.598 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:40.598 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:40.598 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:17:40.598 INFO:tasks.workunit.client.0.smithi016.stdout:587/600: open fd = 3 2024-09-12T02:17:40.598 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:40.598 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:40.599 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:17:40.599 INFO:tasks.workunit.client.0.smithi016.stdout:588/600: open fd = 3 2024-09-12T02:17:40.599 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:40.599 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:40.599 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:17:40.599 INFO:tasks.workunit.client.0.smithi016.stdout:589/600: open fd = 3 2024-09-12T02:17:40.599 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:40.599 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:40.599 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:17:40.599 INFO:tasks.workunit.client.0.smithi016.stdout:590/600: open fd = 3 2024-09-12T02:17:40.599 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:40.599 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:40.599 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:17:40.600 INFO:tasks.workunit.client.0.smithi016.stdout:591/600: open fd = 3 2024-09-12T02:17:40.600 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:40.600 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:40.600 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:17:40.600 INFO:tasks.workunit.client.0.smithi016.stdout:592/600: open fd = 3 2024-09-12T02:17:40.600 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:40.600 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:40.600 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:17:40.600 INFO:tasks.workunit.client.0.smithi016.stdout:593/600: open fd = 3 2024-09-12T02:17:40.600 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:40.600 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:40.600 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:17:40.600 INFO:tasks.workunit.client.0.smithi016.stdout:594/600: open fd = 3 2024-09-12T02:17:40.600 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:40.601 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:40.601 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:17:40.601 INFO:tasks.workunit.client.0.smithi016.stdout:595/600: open fd = 3 2024-09-12T02:17:40.601 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:40.601 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:40.601 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:17:40.601 INFO:tasks.workunit.client.0.smithi016.stdout:596/600: open fd = 3 2024-09-12T02:17:40.601 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:40.601 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:40.601 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:17:40.601 INFO:tasks.workunit.client.0.smithi016.stdout:597/600: open fd = 3 2024-09-12T02:17:40.601 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:40.601 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:40.601 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:17:40.602 INFO:tasks.workunit.client.0.smithi016.stdout:598/600: open fd = 3 2024-09-12T02:17:40.602 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:40.602 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:40.602 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:17:40.602 INFO:tasks.workunit.client.0.smithi016.stdout:599/600: open fd = 3 2024-09-12T02:17:40.602 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:40.602 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:40.602 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:17:40.602 INFO:tasks.workunit.client.0.smithi016.stdout:600/600: open fd = 3 2024-09-12T02:17:40.602 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:40.602 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T02:17:40.604 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T02:17:40.605 INFO:tasks.workunit.client.0.smithi016.stderr:+ echo OK 2024-09-12T02:17:40.606 INFO:tasks.workunit.client.0.smithi016.stdout:OK 2024-09-12T02:17:40.606 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-12T02:17:40.606 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-12T02:17:40.651 INFO:tasks.workunit:Stopping ['fs/test_o_trunc.sh'] on client.0... 2024-09-12T02:17:40.651 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-09-12T02:17:40.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.903+0000 7f3aa7057640 1 -- 172.21.15.16:0/3269475152 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3aa0102570 msgr2=0x7f3aa01029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:40.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.903+0000 7f3aa7057640 1 --2- 172.21.15.16:0/3269475152 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3aa0102570 0x7f3aa01029b0 secure :-1 s=READY pgs=2095 cs=0 l=1 rev1=1 crypto rx=0x7f3a88009900 tx=0x7f3a8802a480 comp rx=0 tx=0).stop 2024-09-12T02:17:40.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.903+0000 7f3aa7057640 1 -- 172.21.15.16:0/3269475152 shutdown_connections 2024-09-12T02:17:40.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.903+0000 7f3aa7057640 1 --2- 172.21.15.16:0/3269475152 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3aa0102ef0 0x7f3aa010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:40.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.903+0000 7f3aa7057640 1 --2- 172.21.15.16:0/3269475152 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3aa0102570 0x7f3aa01029b0 unknown :-1 s=CLOSED pgs=2095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:40.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.903+0000 7f3aa7057640 1 --2- 172.21.15.16:0/3269475152 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3aa0100e20 0x7f3aa0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:40.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.903+0000 7f3aa7057640 1 -- 172.21.15.16:0/3269475152 >> 172.21.15.16:0/3269475152 conn(0x7f3aa00fc7d0 msgr2=0x7f3aa00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:40.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.903+0000 7f3aa7057640 1 -- 172.21.15.16:0/3269475152 shutdown_connections 2024-09-12T02:17:40.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.903+0000 7f3aa7057640 1 -- 172.21.15.16:0/3269475152 wait complete. 2024-09-12T02:17:40.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.904+0000 7f3aa7057640 1 Processor -- start 2024-09-12T02:17:40.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.904+0000 7f3aa7057640 1 -- start start 2024-09-12T02:17:40.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.904+0000 7f3aa7057640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3aa0100e20 0x7f3aa01984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:40.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.904+0000 7f3aa7057640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3aa0102570 0x7f3aa01989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:40.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.904+0000 7f3aa7057640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3aa0102ef0 0x7f3aa019cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:40.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.904+0000 7f3aa7057640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3aa0110f80 con 0x7f3aa0100e20 2024-09-12T02:17:40.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.904+0000 7f3aa7057640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3aa0110e00 con 0x7f3aa0102570 2024-09-12T02:17:40.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.904+0000 7f3aa7057640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f3aa0111100 con 0x7f3aa0102ef0 2024-09-12T02:17:40.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.904+0000 7f3aa4df3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3aa0100e20 0x7f3aa01984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:40.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.904+0000 7f3aa4df3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3aa0100e20 0x7f3aa01984b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53866/0 (socket says 172.21.15.16:53866) 2024-09-12T02:17:40.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.904+0000 7f3aa4df3640 1 -- 172.21.15.16:0/267156835 learned_addr learned my addr 172.21.15.16:0/267156835 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:17:40.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.904+0000 7f3aa55f4640 1 --2- 172.21.15.16:0/267156835 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3aa0102ef0 0x7f3aa019cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:40.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.905+0000 7f3a9ffff640 1 --2- 172.21.15.16:0/267156835 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3aa0102570 0x7f3aa01989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:40.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.905+0000 7f3aa4df3640 1 -- 172.21.15.16:0/267156835 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3aa0102ef0 msgr2=0x7f3aa019cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:40.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.905+0000 7f3aa4df3640 1 --2- 172.21.15.16:0/267156835 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3aa0102ef0 0x7f3aa019cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:40.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.905+0000 7f3aa4df3640 1 -- 172.21.15.16:0/267156835 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3aa0102570 msgr2=0x7f3aa01989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:40.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.905+0000 7f3aa4df3640 1 --2- 172.21.15.16:0/267156835 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3aa0102570 0x7f3aa01989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:40.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.905+0000 7f3aa4df3640 1 -- 172.21.15.16:0/267156835 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3aa019d460 con 0x7f3aa0100e20 2024-09-12T02:17:40.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.905+0000 7f3a9ffff640 1 --2- 172.21.15.16:0/267156835 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3aa0102570 0x7f3aa01989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:17:40.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.905+0000 7f3aa4df3640 1 --2- 172.21.15.16:0/267156835 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3aa0100e20 0x7f3aa01984b0 secure :-1 s=READY pgs=2096 cs=0 l=1 rev1=1 crypto rx=0x7f3a9000d950 tx=0x7f3a9000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:40.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.905+0000 7f3a9dffb640 1 -- 172.21.15.16:0/267156835 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a90017030 con 0x7f3aa0100e20 2024-09-12T02:17:40.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.905+0000 7f3a9dffb640 1 -- 172.21.15.16:0/267156835 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a90004020 con 0x7f3aa0100e20 2024-09-12T02:17:40.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.905+0000 7f3aa7057640 1 -- 172.21.15.16:0/267156835 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3aa019d750 con 0x7f3aa0100e20 2024-09-12T02:17:40.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.905+0000 7f3a9dffb640 1 -- 172.21.15.16:0/267156835 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a900047a0 con 0x7f3aa0100e20 2024-09-12T02:17:40.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.905+0000 7f3aa7057640 1 -- 172.21.15.16:0/267156835 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3aa01a5030 con 0x7f3aa0100e20 2024-09-12T02:17:40.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.908+0000 7f3a9dffb640 1 -- 172.21.15.16:0/267156835 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f3a900041d0 con 0x7f3aa0100e20 2024-09-12T02:17:40.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.908+0000 7f3aa9430740 1 Processor -- start 2024-09-12T02:17:40.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.908+0000 7f3aa9430740 1 -- start start 2024-09-12T02:17:40.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.908+0000 7f3aa9430740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629889ceb40 0x5629889cd090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:40.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.908+0000 7f3aa9430740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629889cd5d0 0x5629889cda10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:40.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.908+0000 7f3aa9430740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5629889dc780 0x5629889deb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:40.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.908+0000 7f3aa9430740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5629889332f0 con 0x5629889ceb40 2024-09-12T02:17:40.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.908+0000 7f3aa9430740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5629888dacf0 con 0x5629889cd5d0 2024-09-12T02:17:40.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.908+0000 7f3aa9430740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x56298892f2b0 con 0x5629889dc780 2024-09-12T02:17:40.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.909+0000 7f3aa55f4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629889ceb40 0x5629889cd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:40.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.909+0000 7f3aa55f4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629889ceb40 0x5629889cd090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53878/0 (socket says 172.21.15.16:53878) 2024-09-12T02:17:40.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.909+0000 7f3aa55f4640 1 -- 172.21.15.16:0/2301532246 learned_addr learned my addr 172.21.15.16:0/2301532246 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:17:40.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.909+0000 7f3a9ffff640 1 --2- 172.21.15.16:0/2301532246 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5629889dc780 0x5629889deb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:40.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.909+0000 7f3aa4df3640 1 --2- 172.21.15.16:0/2301532246 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629889cd5d0 0x5629889cda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:40.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.909+0000 7f3aa55f4640 1 -- 172.21.15.16:0/2301532246 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5629889dc780 msgr2=0x5629889deb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:40.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.909+0000 7f3aa55f4640 1 --2- 172.21.15.16:0/2301532246 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5629889dc780 0x5629889deb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:40.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.909+0000 7f3aa55f4640 1 -- 172.21.15.16:0/2301532246 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629889cd5d0 msgr2=0x5629889cda10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:40.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.909+0000 7f3aa55f4640 1 --2- 172.21.15.16:0/2301532246 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629889cd5d0 0x5629889cda10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:40.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.909+0000 7f3aa55f4640 1 -- 172.21.15.16:0/2301532246 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629888bf3a0 con 0x5629889ceb40 2024-09-12T02:17:40.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.909+0000 7f3a9dffb640 1 --2- 172.21.15.16:0/267156835 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f3a8c0a2b80 0x7f3a8c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:40.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.909+0000 7f3aa55f4640 1 --2- 172.21.15.16:0/2301532246 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629889ceb40 0x5629889cd090 secure :-1 s=READY pgs=2097 cs=0 l=1 rev1=1 crypto rx=0x7f3a98007ec0 tx=0x7f3a9800f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f993c44042278e49 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:40.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.909+0000 7f3aa4df3640 1 --2- 172.21.15.16:0/267156835 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f3a8c0a2b80 0x7f3a8c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:40.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.909+0000 7f3a96ffd640 1 -- 172.21.15.16:0/2301532246 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a98010840 con 0x5629889ceb40 2024-09-12T02:17:40.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.910+0000 7f3a96ffd640 1 -- 172.21.15.16:0/2301532246 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a980183f0 con 0x5629889ceb40 2024-09-12T02:17:40.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.910+0000 7f3a9dffb640 1 -- 172.21.15.16:0/267156835 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f3a900cea50 con 0x7f3aa0100e20 2024-09-12T02:17:40.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.910+0000 7f3aa9430740 1 -- 172.21.15.16:0/2301532246 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629889ceb40 msgr2=0x5629889cd090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:40.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.910+0000 7f3aa9430740 1 --2- 172.21.15.16:0/2301532246 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629889ceb40 0x5629889cd090 secure :-1 s=READY pgs=2097 cs=0 l=1 rev1=1 crypto rx=0x7f3a98007ec0 tx=0x7f3a9800f790 comp rx=0 tx=0).stop 2024-09-12T02:17:40.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.910+0000 7f3a96ffd640 1 -- 172.21.15.16:0/2301532246 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a98017600 con 0x5629889ceb40 2024-09-12T02:17:40.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.910+0000 7f3aa4df3640 1 --2- 172.21.15.16:0/267156835 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f3a8c0a2b80 0x7f3a8c0a5040 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f3a9009b240 tx=0x7f3a90004e20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:40.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.910+0000 7f3aa9430740 1 -- 172.21.15.16:0/2301532246 shutdown_connections 2024-09-12T02:17:40.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.910+0000 7f3aa9430740 1 --2- 172.21.15.16:0/2301532246 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5629889dc780 0x5629889deb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:40.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.910+0000 7f3aa9430740 1 --2- 172.21.15.16:0/2301532246 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629889cd5d0 0x5629889cda10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:40.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.910+0000 7f3aa9430740 1 --2- 172.21.15.16:0/2301532246 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629889ceb40 0x5629889cd090 unknown :-1 s=CLOSED pgs=2097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:40.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.910+0000 7f3aa9430740 1 -- 172.21.15.16:0/2301532246 >> 172.21.15.16:0/2301532246 conn(0x562988996310 msgr2=0x5629889c9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:40.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.910+0000 7f3aa9430740 1 -- 172.21.15.16:0/2301532246 shutdown_connections 2024-09-12T02:17:40.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.911+0000 7f3aa9430740 1 -- 172.21.15.16:0/2301532246 wait complete. 2024-09-12T02:17:40.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.911+0000 7f3aa9430740 1 -- start start 2024-09-12T02:17:40.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.911+0000 7f3aa9430740 1 Processor -- start 2024-09-12T02:17:40.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.911+0000 7f3aa9430740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5629889cd5d0 0x5629889dbe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:40.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.911+0000 7f3aa9430740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629889dc780 0x5629889d90d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:40.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.911+0000 7f3aa9430740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629889d9610 0x5629889d9a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:40.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.911+0000 7f3aa9430740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5629889df4d0 con 0x5629889d9610 2024-09-12T02:17:40.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.911+0000 7f3aa9430740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5629888ee060 con 0x5629889dc780 2024-09-12T02:17:40.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.911+0000 7f3aa9430740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5629889df650 con 0x5629889cd5d0 2024-09-12T02:17:40.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.911+0000 7f3aa9430740 10 client.0 _ll_register_callbacks cb 0x5629886418c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:17:40.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.912+0000 7f3aa55f4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629889d9610 0x5629889d9a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:40.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.912+0000 7f3aa55f4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629889d9610 0x5629889d9a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53890/0 (socket says 172.21.15.16:53890) 2024-09-12T02:17:40.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.912+0000 7f3aa55f4640 1 -- 172.21.15.16:0/2081834931 learned_addr learned my addr 172.21.15.16:0/2081834931 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:17:40.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.912+0000 7f3a9ffff640 1 --2- 172.21.15.16:0/2081834931 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629889dc780 0x5629889d90d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:40.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.912+0000 7f3aa55f4640 1 --2- 172.21.15.16:0/2081834931 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5629889cd5d0 0x5629889dbe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:40.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.912+0000 7f3aa55f4640 1 -- 172.21.15.16:0/2081834931 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5629889cd5d0 msgr2=0x5629889dbe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:40.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.912+0000 7f3aa55f4640 1 --2- 172.21.15.16:0/2081834931 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5629889cd5d0 0x5629889dbe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:40.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.912+0000 7f3aa55f4640 1 -- 172.21.15.16:0/2081834931 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629889dc780 msgr2=0x5629889d90d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:40.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.912+0000 7f3aa55f4640 1 --2- 172.21.15.16:0/2081834931 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629889dc780 0x5629889d90d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:40.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.912+0000 7f3aa55f4640 1 -- 172.21.15.16:0/2081834931 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629886a43c0 con 0x5629889d9610 2024-09-12T02:17:40.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.912+0000 7f3aa55f4640 1 --2- 172.21.15.16:0/2081834931 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629889d9610 0x5629889d9a90 secure :-1 s=READY pgs=2098 cs=0 l=1 rev1=1 crypto rx=0x7f3a9801a040 tx=0x7f3a98012040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dea321fdd5d894 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:40.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.912+0000 7f3a7ffff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:17:40.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.912+0000 7f3a7ffff640 1 -- 172.21.15.16:0/2081834931 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a9802c060 con 0x5629889d9610 2024-09-12T02:17:40.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.913+0000 7f3a7ffff640 1 -- 172.21.15.16:0/2081834931 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f3a980187a0 con 0x5629889d9610 2024-09-12T02:17:40.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.913+0000 7f3aa9430740 1 -- 172.21.15.16:0/2081834931 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3a980183f0 con 0x5629889d9610 2024-09-12T02:17:40.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.913+0000 7f3a7ffff640 1 -- 172.21.15.16:0/2081834931 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f3a98004a40 con 0x5629889d9610 2024-09-12T02:17:40.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.913+0000 7f3aa9430740 10 client.20745 fetch_fsmap learned FSMap version 25 2024-09-12T02:17:40.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.913+0000 7f3aa9430740 1 -- 172.21.15.16:0/2081834931 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5629889eb050 con 0x5629889d9610 2024-09-12T02:17:40.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.913+0000 7f3a7ffff640 1 -- 172.21.15.16:0/2081834931 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a980183f0 con 0x5629889d9610 2024-09-12T02:17:40.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.913+0000 7f3a7ffff640 1 -- 172.21.15.16:0/2081834931 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f3a98031070 con 0x5629889d9610 2024-09-12T02:17:40.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.913+0000 7f3aa9430740 10 client.20745 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:17:40.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.913+0000 7f3aa9430740 10 client.20745 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:17:40.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.913+0000 7f3aa9430740 20 client.20745 populate_metadata read hostname 'smithi016' 2024-09-12T02:17:40.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.913+0000 7f3aa9430740 1 --2- 172.21.15.16:0/2081834931 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5629889ec510 0x5629889ee900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:40.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.913+0000 7f3aa9430740 4 client.20745 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:17:40.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.913+0000 7f3aa9430740 1 -- 172.21.15.16:0/2081834931 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562988940860 con 0x5629889ec510 2024-09-12T02:17:40.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.914+0000 7f3a9ffff640 1 --2- 172.21.15.16:0/2081834931 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5629889ec510 0x5629889ee900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:40.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.914+0000 7f3a9ffff640 1 --2- 172.21.15.16:0/2081834931 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5629889ec510 0x5629889ee900 crc :-1 s=READY pgs=465 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=845c0061d6cc53fe server_cookie=7f00fbeab733dd73 in_seq=0 out_seq=0 2024-09-12T02:17:40.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.914+0000 7f3a7ffff640 10 client.20745 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:17:40.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.915+0000 7f3a7ffff640 1 -- 172.21.15.16:0/2081834931 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562988940860 con 0x5629889ec510 2024-09-12T02:17:40.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.915+0000 7f3a7ffff640 10 client.20745 handle_command_reply: tid=0 2024-09-12T02:17:40.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.915+0000 7f3aa9430740 1 client.20745 shutdown 2024-09-12T02:17:40.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.915+0000 7f3aa9430740 1 -- 172.21.15.16:0/2081834931 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629889d9610 msgr2=0x5629889d9a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:40.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.915+0000 7f3aa9430740 1 --2- 172.21.15.16:0/2081834931 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629889d9610 0x5629889d9a90 secure :-1 s=READY pgs=2098 cs=0 l=1 rev1=1 crypto rx=0x7f3a9801a040 tx=0x7f3a98012040 comp rx=0 tx=0).stop 2024-09-12T02:17:40.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.915+0000 7f3aa9430740 1 -- 172.21.15.16:0/2081834931 shutdown_connections 2024-09-12T02:17:40.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.915+0000 7f3aa9430740 1 --2- 172.21.15.16:0/2081834931 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5629889ec510 0x5629889ee900 crc :-1 s=READY pgs=465 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:40.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.915+0000 7f3aa9430740 1 --2- 172.21.15.16:0/2081834931 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629889d9610 0x5629889d9a90 unknown :-1 s=CLOSED pgs=2098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:40.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.915+0000 7f3aa9430740 1 --2- 172.21.15.16:0/2081834931 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629889dc780 0x5629889d90d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:40.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.915+0000 7f3aa9430740 1 --2- 172.21.15.16:0/2081834931 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5629889cd5d0 0x5629889dbe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:40.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.915+0000 7f3aa9430740 1 -- 172.21.15.16:0/2081834931 >> 172.21.15.16:0/2081834931 conn(0x562988996310 msgr2=0x5629889cf960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:40.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.915+0000 7f3a7ffff640 0 client.20745 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:17:40.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.915+0000 7f3aa9430740 1 -- 172.21.15.16:0/2081834931 shutdown_connections 2024-09-12T02:17:40.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.915+0000 7f3aa9430740 1 -- 172.21.15.16:0/2081834931 wait complete. 2024-09-12T02:17:40.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.915+0000 7f3aa9430740 20 client.20745 trim_cache size 0 max 16384 2024-09-12T02:17:40.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.928+0000 7f3aa9430740 1 Processor -- start 2024-09-12T02:17:40.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.928+0000 7f3aa9430740 1 -- start start 2024-09-12T02:17:40.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.929+0000 7f3aa9430740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5629889cd5d0 0x5629889db400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:40.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.929+0000 7f3aa9430740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629889d9610 0x5629889c94f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:40.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.929+0000 7f3aa9430740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629889dc780 0x5629889c9a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:40.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.929+0000 7f3aa9430740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3aa01ada60 con 0x5629889d9610 2024-09-12T02:17:40.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.929+0000 7f3aa9430740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3aa01ab330 con 0x5629889dc780 2024-09-12T02:17:40.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.929+0000 7f3aa9430740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f3aa01a91e0 con 0x5629889cd5d0 2024-09-12T02:17:40.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.929+0000 7f3a9ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629889d9610 0x5629889c94f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:40.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.929+0000 7f3a9ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629889d9610 0x5629889c94f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53892/0 (socket says 172.21.15.16:53892) 2024-09-12T02:17:40.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.929+0000 7f3a9ffff640 1 -- 172.21.15.16:0/3185730203 learned_addr learned my addr 172.21.15.16:0/3185730203 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:17:40.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.929+0000 7f3aa55f4640 1 --2- 172.21.15.16:0/3185730203 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5629889cd5d0 0x5629889db400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:40.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.929+0000 7f3aa55f4640 1 --2- 172.21.15.16:0/3185730203 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629889dc780 0x5629889c9a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:40.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.929+0000 7f3aa55f4640 1 -- 172.21.15.16:0/3185730203 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629889dc780 msgr2=0x5629889c9a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:40.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.929+0000 7f3aa55f4640 1 --2- 172.21.15.16:0/3185730203 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629889dc780 0x5629889c9a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:40.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.929+0000 7f3aa55f4640 1 -- 172.21.15.16:0/3185730203 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629889d9610 msgr2=0x5629889c94f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:40.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.929+0000 7f3aa55f4640 1 --2- 172.21.15.16:0/3185730203 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629889d9610 0x5629889c94f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:40.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.929+0000 7f3aa55f4640 1 -- 172.21.15.16:0/3185730203 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629889d0120 con 0x5629889cd5d0 2024-09-12T02:17:40.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.930+0000 7f3aa55f4640 1 --2- 172.21.15.16:0/3185730203 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5629889cd5d0 0x5629889db400 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f3a98004db0 tx=0x7f3a98004be0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e8fdf70bced80cf3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:40.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.930+0000 7f3a967fc640 1 -- 172.21.15.16:0/3185730203 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a98037040 con 0x5629889cd5d0 2024-09-12T02:17:40.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.930+0000 7f3a967fc640 1 -- 172.21.15.16:0/3185730203 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a98005e90 con 0x5629889cd5d0 2024-09-12T02:17:40.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.930+0000 7f3a967fc640 1 -- 172.21.15.16:0/3185730203 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a98018650 con 0x5629889cd5d0 2024-09-12T02:17:40.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.930+0000 7f3aa9430740 1 -- 172.21.15.16:0/3185730203 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5629889cd5d0 msgr2=0x5629889db400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:40.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.931+0000 7f3aa9430740 1 --2- 172.21.15.16:0/3185730203 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5629889cd5d0 0x5629889db400 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f3a98004db0 tx=0x7f3a98004be0 comp rx=0 tx=0).stop 2024-09-12T02:17:40.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.931+0000 7f3aa9430740 1 -- 172.21.15.16:0/3185730203 shutdown_connections 2024-09-12T02:17:40.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.931+0000 7f3aa9430740 1 --2- 172.21.15.16:0/3185730203 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629889dc780 0x5629889c9a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:40.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.931+0000 7f3aa9430740 1 --2- 172.21.15.16:0/3185730203 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629889d9610 0x5629889c94f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:40.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.931+0000 7f3aa9430740 1 --2- 172.21.15.16:0/3185730203 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5629889cd5d0 0x5629889db400 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:40.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.931+0000 7f3aa9430740 1 -- 172.21.15.16:0/3185730203 >> 172.21.15.16:0/3185730203 conn(0x562988996310 msgr2=0x5629889d8090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:40.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.931+0000 7f3aa9430740 1 -- 172.21.15.16:0/3185730203 shutdown_connections 2024-09-12T02:17:40.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.931+0000 7f3aa9430740 1 -- 172.21.15.16:0/3185730203 wait complete. 2024-09-12T02:17:40.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.931+0000 7f3aa9430740 1 -- start start 2024-09-12T02:17:40.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.931+0000 7f3aa9430740 1 Processor -- start 2024-09-12T02:17:40.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.932+0000 7f3aa9430740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5629889cd5d0 0x5629889f2220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:40.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.932+0000 7f3aa9430740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629889d9610 0x5629889ef570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:40.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.932+0000 7f3aa9430740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629889dc780 0x5629889efcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:40.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.932+0000 7f3aa9430740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3aa01a9830 con 0x5629889dc780 2024-09-12T02:17:40.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.932+0000 7f3aa9430740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3aa01aac60 con 0x5629889d9610 2024-09-12T02:17:40.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.932+0000 7f3aa9430740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f3aa01a9ec0 con 0x5629889cd5d0 2024-09-12T02:17:40.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.932+0000 7f3aa9430740 10 client.0 _ll_register_callbacks cb 0x7f3a88009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:17:40.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.932+0000 7f3aa55f4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629889dc780 0x5629889efcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:40.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.932+0000 7f3aa55f4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629889dc780 0x5629889efcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53904/0 (socket says 172.21.15.16:53904) 2024-09-12T02:17:40.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.932+0000 7f3aa55f4640 1 -- 172.21.15.16:0/2475200534 learned_addr learned my addr 172.21.15.16:0/2475200534 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:17:40.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.932+0000 7f3aa55f4640 1 --2- 172.21.15.16:0/2475200534 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5629889cd5d0 0x5629889f2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:40.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.932+0000 7f3a9ffff640 1 --2- 172.21.15.16:0/2475200534 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629889d9610 0x5629889ef570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:40.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.932+0000 7f3aa55f4640 1 -- 172.21.15.16:0/2475200534 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5629889cd5d0 msgr2=0x5629889f2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:40.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.932+0000 7f3aa55f4640 1 --2- 172.21.15.16:0/2475200534 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5629889cd5d0 0x5629889f2220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:40.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.932+0000 7f3aa55f4640 1 -- 172.21.15.16:0/2475200534 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629889d9610 msgr2=0x5629889ef570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:40.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.932+0000 7f3aa55f4640 1 --2- 172.21.15.16:0/2475200534 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629889d9610 0x5629889ef570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:40.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.932+0000 7f3aa55f4640 1 -- 172.21.15.16:0/2475200534 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562988a55e50 con 0x5629889dc780 2024-09-12T02:17:40.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.932+0000 7f3a9ffff640 1 --2- 172.21.15.16:0/2475200534 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629889d9610 0x5629889ef570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:17:40.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.932+0000 7f3aa55f4640 1 --2- 172.21.15.16:0/2475200534 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629889dc780 0x5629889efcb0 secure :-1 s=READY pgs=2099 cs=0 l=1 rev1=1 crypto rx=0x7f3a98010b90 tx=0x7f3a98010a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f43c129656e42faa server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:40.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.932+0000 7f3a7ffff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:17:40.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.932+0000 7f3a7ffff640 1 -- 172.21.15.16:0/2475200534 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a98003b70 con 0x5629889dc780 2024-09-12T02:17:40.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.933+0000 7f3a7ffff640 1 -- 172.21.15.16:0/2475200534 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f3a98003d10 con 0x5629889dc780 2024-09-12T02:17:40.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.933+0000 7f3aa9430740 1 -- 172.21.15.16:0/2475200534 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3a98005e90 con 0x5629889dc780 2024-09-12T02:17:40.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.933+0000 7f3a7ffff640 1 -- 172.21.15.16:0/2475200534 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f3a980183f0 con 0x5629889dc780 2024-09-12T02:17:40.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.933+0000 7f3aa9430740 10 client.20751 fetch_fsmap learned FSMap version 25 2024-09-12T02:17:40.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.933+0000 7f3aa9430740 1 -- 172.21.15.16:0/2475200534 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x562988a56ff0 con 0x5629889dc780 2024-09-12T02:17:40.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.933+0000 7f3a7ffff640 1 -- 172.21.15.16:0/2475200534 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a98005e90 con 0x5629889dc780 2024-09-12T02:17:40.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.933+0000 7f3a7ffff640 1 -- 172.21.15.16:0/2475200534 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f3a9803d070 con 0x5629889dc780 2024-09-12T02:17:40.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.933+0000 7f3aa9430740 10 client.20751 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:17:40.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.933+0000 7f3aa9430740 10 client.20751 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:17:40.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.933+0000 7f3aa9430740 20 client.20751 populate_metadata read hostname 'smithi016' 2024-09-12T02:17:40.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.933+0000 7f3aa9430740 1 --2- 172.21.15.16:0/2475200534 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x562988a575a0 0x562988a599c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:40.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.933+0000 7f3aa9430740 4 client.20751 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T02:17:40.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.933+0000 7f3aa9430740 1 -- 172.21.15.16:0/2475200534 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56298893ad70 con 0x562988a575a0 2024-09-12T02:17:40.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.934+0000 7f3a9ffff640 1 --2- 172.21.15.16:0/2475200534 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x562988a575a0 0x562988a599c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:40.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.934+0000 7f3a9ffff640 1 --2- 172.21.15.16:0/2475200534 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x562988a575a0 0x562988a599c0 crc :-1 s=READY pgs=466 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb8f878289d8b6fc server_cookie=17d69a2c34edb078 in_seq=0 out_seq=0 2024-09-12T02:17:40.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.934+0000 7f3a7ffff640 10 client.20751 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:17:40.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.934+0000 7f3a7ffff640 1 -- 172.21.15.16:0/2475200534 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56298893ad70 con 0x562988a575a0 2024-09-12T02:17:40.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.934+0000 7f3a7ffff640 10 client.20751 handle_command_reply: tid=0 2024-09-12T02:17:40.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.934+0000 7f3aa9430740 1 client.20751 shutdown 2024-09-12T02:17:40.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.934+0000 7f3aa9430740 1 -- 172.21.15.16:0/2475200534 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629889dc780 msgr2=0x5629889efcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:40.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.934+0000 7f3aa9430740 1 --2- 172.21.15.16:0/2475200534 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629889dc780 0x5629889efcb0 secure :-1 s=READY pgs=2099 cs=0 l=1 rev1=1 crypto rx=0x7f3a98010b90 tx=0x7f3a98010a90 comp rx=0 tx=0).stop 2024-09-12T02:17:40.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.935+0000 7f3aa9430740 1 -- 172.21.15.16:0/2475200534 shutdown_connections 2024-09-12T02:17:40.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.935+0000 7f3aa9430740 1 --2- 172.21.15.16:0/2475200534 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x562988a575a0 0x562988a599c0 crc :-1 s=READY pgs=466 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:40.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.935+0000 7f3aa9430740 1 --2- 172.21.15.16:0/2475200534 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629889dc780 0x5629889efcb0 unknown :-1 s=CLOSED pgs=2099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:40.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.935+0000 7f3aa9430740 1 --2- 172.21.15.16:0/2475200534 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629889d9610 0x5629889ef570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:40.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.935+0000 7f3aa9430740 1 --2- 172.21.15.16:0/2475200534 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5629889cd5d0 0x5629889f2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:40.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.935+0000 7f3aa9430740 1 -- 172.21.15.16:0/2475200534 >> 172.21.15.16:0/2475200534 conn(0x562988996310 msgr2=0x5629889d7870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:40.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.935+0000 7f3a7ffff640 0 client.20751 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:17:40.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.935+0000 7f3aa9430740 1 -- 172.21.15.16:0/2475200534 shutdown_connections 2024-09-12T02:17:40.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.935+0000 7f3aa9430740 1 -- 172.21.15.16:0/2475200534 wait complete. 2024-09-12T02:17:40.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.935+0000 7f3aa9430740 20 client.20751 trim_cache size 0 max 16384 2024-09-12T02:17:40.940 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T02:17:40.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.936+0000 7f3aa7057640 1 -- 172.21.15.16:0/267156835 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f3a8c0a2b80 msgr2=0x7f3a8c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:40.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.936+0000 7f3aa7057640 1 --2- 172.21.15.16:0/267156835 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f3a8c0a2b80 0x7f3a8c0a5040 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f3a9009b240 tx=0x7f3a90004e20 comp rx=0 tx=0).stop 2024-09-12T02:17:40.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.936+0000 7f3aa7057640 1 -- 172.21.15.16:0/267156835 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3aa0100e20 msgr2=0x7f3aa01984b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:40.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.936+0000 7f3aa7057640 1 --2- 172.21.15.16:0/267156835 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3aa0100e20 0x7f3aa01984b0 secure :-1 s=READY pgs=2096 cs=0 l=1 rev1=1 crypto rx=0x7f3a9000d950 tx=0x7f3a9000de10 comp rx=0 tx=0).stop 2024-09-12T02:17:40.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.936+0000 7f3aa7057640 1 -- 172.21.15.16:0/267156835 shutdown_connections 2024-09-12T02:17:40.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.936+0000 7f3aa7057640 1 --2- 172.21.15.16:0/267156835 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f3a8c0a2b80 0x7f3a8c0a5040 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:40.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.936+0000 7f3aa7057640 1 --2- 172.21.15.16:0/267156835 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3aa0102ef0 0x7f3aa019cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:40.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.936+0000 7f3aa7057640 1 --2- 172.21.15.16:0/267156835 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3aa0102570 0x7f3aa01989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:40.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.936+0000 7f3aa7057640 1 --2- 172.21.15.16:0/267156835 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3aa0100e20 0x7f3aa01984b0 unknown :-1 s=CLOSED pgs=2096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:40.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.936+0000 7f3aa7057640 1 -- 172.21.15.16:0/267156835 >> 172.21.15.16:0/267156835 conn(0x7f3aa00fc7d0 msgr2=0x7f3aa0069f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:40.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.936+0000 7f3aa7057640 1 -- 172.21.15.16:0/267156835 shutdown_connections 2024-09-12T02:17:40.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:40.936+0000 7f3aa7057640 1 -- 172.21.15.16:0/267156835 wait complete. 2024-09-12T02:17:41.035 DEBUG:teuthology.parallel:result is None 2024-09-12T02:17:41.036 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-09-12T02:17:41.072 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-09-12T02:17:41.072 DEBUG:teuthology.run_tasks:Unwinding manager quiescer 2024-09-12T02:17:41.208 INFO:tasks.quiescer:joining Quiescers 2024-09-12T02:17:41.209 WARNING:tasks.quiescer:The quiescer is requested to stop, running cancel all 2024-09-12T02:17:41.209 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:17:41.905 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:17:42.026 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:17:41 smithi016 ceph-mon[30947]: pgmap v808: 97 pgs: 97 active+clean; 4.2 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-09-12T02:17:42.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:17:41 smithi184 ceph-mon[35593]: pgmap v808: 97 pgs: 97 active+clean; 4.2 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-09-12T02:17:42.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:17:41 smithi033 ceph-mon[35332]: pgmap v808: 97 pgs: 97 active+clean; 4.2 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-09-12T02:17:42.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:42.376+0000 7fad38350640 1 -- 172.21.15.16:0/2902963044 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad30103660 msgr2=0x7fad3010bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:42.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:42.376+0000 7fad38350640 1 --2- 172.21.15.16:0/2902963044 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad30103660 0x7fad3010bb50 secure :-1 s=READY pgs=2100 cs=0 l=1 rev1=1 crypto rx=0x7fad20009900 tx=0x7fad2002a480 comp rx=0 tx=0).stop 2024-09-12T02:17:42.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:42.377+0000 7fad38350640 1 -- 172.21.15.16:0/2902963044 shutdown_connections 2024-09-12T02:17:42.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:42.377+0000 7fad38350640 1 --2- 172.21.15.16:0/2902963044 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fad3010c170 0x7fad3010e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:42.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:42.377+0000 7fad38350640 1 --2- 172.21.15.16:0/2902963044 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad30103660 0x7fad3010bb50 unknown :-1 s=CLOSED pgs=2100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:42.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:42.377+0000 7fad38350640 1 --2- 172.21.15.16:0/2902963044 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fad30102d40 0x7fad30103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:42.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:42.377+0000 7fad38350640 1 -- 172.21.15.16:0/2902963044 >> 172.21.15.16:0/2902963044 conn(0x7fad300fc7a0 msgr2=0x7fad300febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:42.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:42.377+0000 7fad38350640 1 -- 172.21.15.16:0/2902963044 shutdown_connections 2024-09-12T02:17:42.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:42.377+0000 7fad38350640 1 -- 172.21.15.16:0/2902963044 wait complete. 2024-09-12T02:17:42.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:42.377+0000 7fad38350640 1 Processor -- start 2024-09-12T02:17:42.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:42.377+0000 7fad38350640 1 -- start start 2024-09-12T02:17:42.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:42.377+0000 7fad38350640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad30102d40 0x7fad301983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:42.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:42.377+0000 7fad38350640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fad30103660 0x7fad30198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:42.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:42.377+0000 7fad38350640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fad3010c170 0x7fad3019ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:42.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:42.377+0000 7fad38350640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fad30111f70 con 0x7fad30102d40 2024-09-12T02:17:42.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:42.377+0000 7fad38350640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fad30111df0 con 0x7fad3010c170 2024-09-12T02:17:42.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:42.377+0000 7fad38350640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fad301120f0 con 0x7fad30103660 2024-09-12T02:17:42.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:42.378+0000 7fad360ec640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad30102d40 0x7fad301983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:42.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:42.378+0000 7fad360ec640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad30102d40 0x7fad301983e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47106/0 (socket says 172.21.15.16:47106) 2024-09-12T02:17:42.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:42.378+0000 7fad360ec640 1 -- 172.21.15.16:0/1142742157 learned_addr learned my addr 172.21.15.16:0/1142742157 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:17:42.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:42.378+0000 7fad368ed640 1 --2- 172.21.15.16:0/1142742157 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fad3010c170 0x7fad3019ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:42.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:42.378+0000 7fad358eb640 1 --2- 172.21.15.16:0/1142742157 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fad30103660 0x7fad30198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:42.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:42.378+0000 7fad360ec640 1 -- 172.21.15.16:0/1142742157 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fad30103660 msgr2=0x7fad30198920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:42.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:42.378+0000 7fad360ec640 1 --2- 172.21.15.16:0/1142742157 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fad30103660 0x7fad30198920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:42.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:42.378+0000 7fad360ec640 1 -- 172.21.15.16:0/1142742157 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fad3010c170 msgr2=0x7fad3019ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:42.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:42.378+0000 7fad360ec640 1 --2- 172.21.15.16:0/1142742157 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fad3010c170 0x7fad3019ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:42.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:42.378+0000 7fad360ec640 1 -- 172.21.15.16:0/1142742157 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad3019d390 con 0x7fad30102d40 2024-09-12T02:17:42.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:42.378+0000 7fad368ed640 1 --2- 172.21.15.16:0/1142742157 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fad3010c170 0x7fad3019ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:17:42.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:42.378+0000 7fad360ec640 1 --2- 172.21.15.16:0/1142742157 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad30102d40 0x7fad301983e0 secure :-1 s=READY pgs=2101 cs=0 l=1 rev1=1 crypto rx=0x7fad2c00d950 tx=0x7fad2c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:42.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:42.378+0000 7fad277fe640 1 -- 172.21.15.16:0/1142742157 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad2c017070 con 0x7fad30102d40 2024-09-12T02:17:42.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:42.378+0000 7fad277fe640 1 -- 172.21.15.16:0/1142742157 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad2c009c30 con 0x7fad30102d40 2024-09-12T02:17:42.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:42.378+0000 7fad38350640 1 -- 172.21.15.16:0/1142742157 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad3019d680 con 0x7fad30102d40 2024-09-12T02:17:42.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:42.378+0000 7fad277fe640 1 -- 172.21.15.16:0/1142742157 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad2c00f6b0 con 0x7fad30102d40 2024-09-12T02:17:42.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:42.378+0000 7fad38350640 1 -- 172.21.15.16:0/1142742157 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad301a4f60 con 0x7fad30102d40 2024-09-12T02:17:42.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:42.379+0000 7fad38350640 1 -- 172.21.15.16:0/1142742157 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad30103120 con 0x7fad30102d40 2024-09-12T02:17:42.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:42.380+0000 7fad277fe640 1 -- 172.21.15.16:0/1142742157 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fad2c0109d0 con 0x7fad30102d40 2024-09-12T02:17:42.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:42.381+0000 7fad277fe640 1 --2- 172.21.15.16:0/1142742157 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fad1c0a2c00 0x7fad1c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:42.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:42.381+0000 7fad358eb640 1 --2- 172.21.15.16:0/1142742157 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fad1c0a2c00 0x7fad1c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:42.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:42.381+0000 7fad277fe640 1 -- 172.21.15.16:0/1142742157 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fad2c0cebe0 con 0x7fad30102d40 2024-09-12T02:17:42.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:42.382+0000 7fad358eb640 1 --2- 172.21.15.16:0/1142742157 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fad1c0a2c00 0x7fad1c0a50c0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fad200096c0 tx=0x7fad20000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:42.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:42.384+0000 7fad277fe640 1 -- 172.21.15.16:0/1142742157 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fad2c009de0 con 0x7fad30102d40 2024-09-12T02:17:42.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:42.595+0000 7fad38350640 1 -- 172.21.15.16:0/1142742157 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fad30199740 con 0x7fad30102d40 2024-09-12T02:17:42.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:42.596+0000 7fad277fe640 1 -- 172.21.15.16:0/1142742157 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11911 (secure 0 0 0) 0x7fad2c09bc60 con 0x7fad30102d40 2024-09-12T02:17:42.596 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T02:17:42.597 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":25,"btime":"2024-09-12T02:01:37:192021+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3279123279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3279123279},{"type":"v1","addr":"172.21.15.16:6835","nonce":3279123279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4119452382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4119452382},{"type":"v1","addr":"172.21.15.16:6837","nonce":4119452382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/3263823165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3263823165},{"type":"v1","addr":"172.21.15.16:6839","nonce":3263823165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/1510290732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":1510290732},{"type":"v1","addr":"172.21.15.16:6841","nonce":1510290732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/279674603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":279674603},{"type":"v1","addr":"172.21.15.33:6833","nonce":279674603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3555728028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3555728028},{"type":"v1","addr":"172.21.15.33:6835","nonce":3555728028}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T01:59:56.117608+0000","modified":"2024-09-12T02:01:36.294760+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24482,"mds_1":24481,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6833/1563374106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1563374106},{"type":"v1","addr":"172.21.15.184:6833","nonce":1563374106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6837/1892833868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1892833868},{"type":"v1","addr":"172.21.15.33:6837","nonce":1892833868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/1457679686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1457679686},{"type":"v1","addr":"172.21.15.184:6835","nonce":1457679686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.184:6837/1793393087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":1793393087},{"type":"v1","addr":"172.21.15.184:6837","nonce":1793393087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.184:6839/4203347941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":4203347941},{"type":"v1","addr":"172.21.15.184:6839","nonce":4203347941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.33:6839/4136670204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":4136670204},{"type":"v1","addr":"172.21.15.33:6839","nonce":4136670204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24481,24482,24469]},"id":1}]} 2024-09-12T02:17:42.597 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 25 2024-09-12T02:17:42.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:42.599+0000 7fad38350640 1 -- 172.21.15.16:0/1142742157 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fad1c0a2c00 msgr2=0x7fad1c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:42.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:42.599+0000 7fad38350640 1 --2- 172.21.15.16:0/1142742157 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fad1c0a2c00 0x7fad1c0a50c0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fad200096c0 tx=0x7fad20000bc0 comp rx=0 tx=0).stop 2024-09-12T02:17:42.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:42.599+0000 7fad38350640 1 -- 172.21.15.16:0/1142742157 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad30102d40 msgr2=0x7fad301983e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:42.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:42.599+0000 7fad38350640 1 --2- 172.21.15.16:0/1142742157 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad30102d40 0x7fad301983e0 secure :-1 s=READY pgs=2101 cs=0 l=1 rev1=1 crypto rx=0x7fad2c00d950 tx=0x7fad2c00de10 comp rx=0 tx=0).stop 2024-09-12T02:17:42.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:42.599+0000 7fad38350640 1 -- 172.21.15.16:0/1142742157 shutdown_connections 2024-09-12T02:17:42.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:42.599+0000 7fad38350640 1 --2- 172.21.15.16:0/1142742157 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fad1c0a2c00 0x7fad1c0a50c0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:42.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:42.599+0000 7fad38350640 1 --2- 172.21.15.16:0/1142742157 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fad3010c170 0x7fad3019ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:42.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:42.599+0000 7fad38350640 1 --2- 172.21.15.16:0/1142742157 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fad30103660 0x7fad30198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:42.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:42.599+0000 7fad38350640 1 --2- 172.21.15.16:0/1142742157 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad30102d40 0x7fad301983e0 unknown :-1 s=CLOSED pgs=2101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:42.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:42.599+0000 7fad38350640 1 -- 172.21.15.16:0/1142742157 >> 172.21.15.16:0/1142742157 conn(0x7fad300fc7a0 msgr2=0x7fad300fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:42.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:42.599+0000 7fad38350640 1 -- 172.21.15.16:0/1142742157 shutdown_connections 2024-09-12T02:17:42.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:42.599+0000 7fad38350640 1 -- 172.21.15.16:0/1142742157 wait complete. 2024-09-12T02:17:42.864 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-12T02:17:42.865 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-12T02:17:42.889 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:17:42 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/1142742157' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:17:43.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:17:42 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/1142742157' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:17:43.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:17:42 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/1142742157' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:17:43.195 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24482 quiesce db --cancel --all' 2024-09-12T02:17:43.195 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24482 quiesce db --cancel --all 2024-09-12T02:17:43.255 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:17:43.833 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:17:43 smithi016 ceph-mon[30947]: pgmap v809: 97 pgs: 97 active+clean; 4.2 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-09-12T02:17:43.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.833+0000 7fd295c3e640 1 -- 172.21.15.16:0/2686371547 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2901036e0 msgr2=0x7fd290107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:43.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.833+0000 7fd295c3e640 1 --2- 172.21.15.16:0/2686371547 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2901036e0 0x7fd290107b90 secure :-1 s=READY pgs=2102 cs=0 l=1 rev1=1 crypto rx=0x7fd284009930 tx=0x7fd28402a420 comp rx=0 tx=0).stop 2024-09-12T02:17:43.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.833+0000 7fd295c3e640 1 -- 172.21.15.16:0/2686371547 shutdown_connections 2024-09-12T02:17:43.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.833+0000 7fd295c3e640 1 --2- 172.21.15.16:0/2686371547 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2901082a0 0x7fd29010ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:43.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.833+0000 7fd295c3e640 1 --2- 172.21.15.16:0/2686371547 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2901036e0 0x7fd290107b90 unknown :-1 s=CLOSED pgs=2102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:43.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.833+0000 7fd295c3e640 1 --2- 172.21.15.16:0/2686371547 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd290102d30 0x7fd290103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:43.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.833+0000 7fd295c3e640 1 -- 172.21.15.16:0/2686371547 >> 172.21.15.16:0/2686371547 conn(0x7fd2900fc7d0 msgr2=0x7fd2900febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:43.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.833+0000 7fd295c3e640 1 -- 172.21.15.16:0/2686371547 shutdown_connections 2024-09-12T02:17:43.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.833+0000 7fd295c3e640 1 -- 172.21.15.16:0/2686371547 wait complete. 2024-09-12T02:17:43.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.833+0000 7fd295c3e640 1 Processor -- start 2024-09-12T02:17:43.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.834+0000 7fd295c3e640 1 -- start start 2024-09-12T02:17:43.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.834+0000 7fd295c3e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd290102d30 0x7fd29019a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:43.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.834+0000 7fd295c3e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd2901036e0 0x7fd29019ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:43.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.834+0000 7fd295c3e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2901082a0 0x7fd29019efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:43.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.834+0000 7fd295c3e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd2901120a0 con 0x7fd2901082a0 2024-09-12T02:17:43.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.834+0000 7fd295c3e640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd290111f20 con 0x7fd290102d30 2024-09-12T02:17:43.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.834+0000 7fd295c3e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd290112220 con 0x7fd2901036e0 2024-09-12T02:17:43.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.834+0000 7fd28ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2901082a0 0x7fd29019efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:43.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.834+0000 7fd28ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2901082a0 0x7fd29019efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47120/0 (socket says 172.21.15.16:47120) 2024-09-12T02:17:43.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.834+0000 7fd28ffff640 1 -- 172.21.15.16:0/85231231 learned_addr learned my addr 172.21.15.16:0/85231231 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:17:43.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.834+0000 7fd28effd640 1 --2- 172.21.15.16:0/85231231 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd2901036e0 0x7fd29019ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:43.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.834+0000 7fd28f7fe640 1 --2- 172.21.15.16:0/85231231 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd290102d30 0x7fd29019a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:43.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.834+0000 7fd28ffff640 1 -- 172.21.15.16:0/85231231 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd2901036e0 msgr2=0x7fd29019ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:43.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.834+0000 7fd28ffff640 1 --2- 172.21.15.16:0/85231231 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd2901036e0 0x7fd29019ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:43.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.834+0000 7fd28ffff640 1 -- 172.21.15.16:0/85231231 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd290102d30 msgr2=0x7fd29019a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:43.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.834+0000 7fd28ffff640 1 --2- 172.21.15.16:0/85231231 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd290102d30 0x7fd29019a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:43.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.834+0000 7fd28ffff640 1 -- 172.21.15.16:0/85231231 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd29019f690 con 0x7fd2901082a0 2024-09-12T02:17:43.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.835+0000 7fd28ffff640 1 --2- 172.21.15.16:0/85231231 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2901082a0 0x7fd29019efb0 secure :-1 s=READY pgs=2103 cs=0 l=1 rev1=1 crypto rx=0x7fd28800cce0 tx=0x7fd28800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:43.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.835+0000 7fd28cff9640 1 -- 172.21.15.16:0/85231231 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd28800f450 con 0x7fd2901082a0 2024-09-12T02:17:43.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.835+0000 7fd295c3e640 1 -- 172.21.15.16:0/85231231 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd29019f980 con 0x7fd2901082a0 2024-09-12T02:17:43.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.835+0000 7fd28cff9640 1 -- 172.21.15.16:0/85231231 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd28800fad0 con 0x7fd2901082a0 2024-09-12T02:17:43.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.835+0000 7fd28cff9640 1 -- 172.21.15.16:0/85231231 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd288018b60 con 0x7fd2901082a0 2024-09-12T02:17:43.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.835+0000 7fd295c3e640 1 -- 172.21.15.16:0/85231231 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2901a7260 con 0x7fd2901082a0 2024-09-12T02:17:43.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.838+0000 7fd298017740 1 Processor -- start 2024-09-12T02:17:43.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.838+0000 7fd298017740 1 -- start start 2024-09-12T02:17:43.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.839+0000 7fd298017740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b6b171aa20 0x55b6b171ae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:43.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.839+0000 7fd298017740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b6b171b340 0x55b6b1736810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:43.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.839+0000 7fd298017740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b6b1736d50 0x55b6b1739140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:43.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.839+0000 7fd298017740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55b6b1680280 con 0x55b6b171b340 2024-09-12T02:17:43.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.839+0000 7fd298017740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55b6b1627b10 con 0x55b6b171aa20 2024-09-12T02:17:43.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.839+0000 7fd298017740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55b6b167c240 con 0x55b6b1736d50 2024-09-12T02:17:43.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.839+0000 7fd28cff9640 1 -- 172.21.15.16:0/85231231 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fd28800fc80 con 0x7fd2901082a0 2024-09-12T02:17:43.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.839+0000 7fd28f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b6b171b340 0x55b6b1736810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:43.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.839+0000 7fd28f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b6b171b340 0x55b6b1736810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47136/0 (socket says 172.21.15.16:47136) 2024-09-12T02:17:43.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.839+0000 7fd28f7fe640 1 -- 172.21.15.16:0/710130014 learned_addr learned my addr 172.21.15.16:0/710130014 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:17:43.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.840+0000 7fd28effd640 1 --2- 172.21.15.16:0/710130014 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b6b171aa20 0x55b6b171ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:43.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.840+0000 7fd28effd640 1 --2- 172.21.15.16:0/710130014 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b6b1736d50 0x55b6b1739140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:43.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.840+0000 7fd28f7fe640 1 -- 172.21.15.16:0/710130014 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b6b1736d50 msgr2=0x55b6b1739140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:43.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.840+0000 7fd28f7fe640 1 --2- 172.21.15.16:0/710130014 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b6b1736d50 0x55b6b1739140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:43.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.840+0000 7fd28f7fe640 1 -- 172.21.15.16:0/710130014 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b6b171aa20 msgr2=0x55b6b171ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:43.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.840+0000 7fd28f7fe640 1 --2- 172.21.15.16:0/710130014 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b6b171aa20 0x55b6b171ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:43.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.840+0000 7fd28f7fe640 1 -- 172.21.15.16:0/710130014 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6b16341f0 con 0x55b6b171b340 2024-09-12T02:17:43.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.840+0000 7fd28effd640 1 --2- 172.21.15.16:0/710130014 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b6b171aa20 0x55b6b171ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:17:43.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.841+0000 7fd28f7fe640 1 --2- 172.21.15.16:0/710130014 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b6b171b340 0x55b6b1736810 secure :-1 s=READY pgs=2104 cs=0 l=1 rev1=1 crypto rx=0x7fd28000ad00 tx=0x7fd28000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca7d098f840c922c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:43.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.841+0000 7fd28cff9640 1 --2- 172.21.15.16:0/85231231 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fd2780a2c00 0x7fd2780a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:43.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.841+0000 7fd28cff9640 1 -- 172.21.15.16:0/85231231 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fd2880cf460 con 0x7fd2901082a0 2024-09-12T02:17:43.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.841+0000 7fd28effd640 1 --2- 172.21.15.16:0/85231231 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fd2780a2c00 0x7fd2780a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:43.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.841+0000 7fd275ffb640 1 -- 172.21.15.16:0/710130014 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd28001d070 con 0x55b6b171b340 2024-09-12T02:17:43.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.841+0000 7fd275ffb640 1 -- 172.21.15.16:0/710130014 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd280018b20 con 0x55b6b171b340 2024-09-12T02:17:43.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.842+0000 7fd275ffb640 1 -- 172.21.15.16:0/710130014 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd280010be0 con 0x55b6b171b340 2024-09-12T02:17:43.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.842+0000 7fd28effd640 1 --2- 172.21.15.16:0/85231231 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fd2780a2c00 0x7fd2780a50c0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fd284005940 tx=0x7fd284000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:43.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.842+0000 7fd298017740 1 -- 172.21.15.16:0/710130014 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b6b171b340 msgr2=0x55b6b1736810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:43.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.842+0000 7fd298017740 1 --2- 172.21.15.16:0/710130014 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b6b171b340 0x55b6b1736810 secure :-1 s=READY pgs=2104 cs=0 l=1 rev1=1 crypto rx=0x7fd28000ad00 tx=0x7fd28000f790 comp rx=0 tx=0).stop 2024-09-12T02:17:43.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.842+0000 7fd298017740 1 -- 172.21.15.16:0/710130014 shutdown_connections 2024-09-12T02:17:43.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.842+0000 7fd298017740 1 --2- 172.21.15.16:0/710130014 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b6b1736d50 0x55b6b1739140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:43.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.842+0000 7fd298017740 1 --2- 172.21.15.16:0/710130014 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b6b171b340 0x55b6b1736810 unknown :-1 s=CLOSED pgs=2104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:43.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.842+0000 7fd298017740 1 --2- 172.21.15.16:0/710130014 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b6b171aa20 0x55b6b171ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:43.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.842+0000 7fd298017740 1 -- 172.21.15.16:0/710130014 >> 172.21.15.16:0/710130014 conn(0x55b6b17170f0 msgr2=0x55b6b17174d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:43.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.843+0000 7fd298017740 1 -- 172.21.15.16:0/710130014 shutdown_connections 2024-09-12T02:17:43.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.843+0000 7fd298017740 1 -- 172.21.15.16:0/710130014 wait complete. 2024-09-12T02:17:43.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.843+0000 7fd298017740 1 -- start start 2024-09-12T02:17:43.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.843+0000 7fd298017740 1 Processor -- start 2024-09-12T02:17:43.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.844+0000 7fd298017740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b6b171aa20 0x55b6b1733cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:43.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.844+0000 7fd298017740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b6b171b340 0x55b6b17341f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:43.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.844+0000 7fd298017740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b6b1736d50 0x55b6b1734730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:43.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.844+0000 7fd298017740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55b6b1742180 con 0x55b6b171aa20 2024-09-12T02:17:43.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.844+0000 7fd298017740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55b6b163af10 con 0x55b6b171b340 2024-09-12T02:17:43.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.844+0000 7fd298017740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55b6b1742300 con 0x55b6b1736d50 2024-09-12T02:17:43.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.844+0000 7fd298017740 10 client.0 _ll_register_callbacks cb 0x55b6b1603a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:17:43.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.844+0000 7fd28f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b6b171aa20 0x55b6b1733cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:43.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.844+0000 7fd28f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b6b171aa20 0x55b6b1733cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47146/0 (socket says 172.21.15.16:47146) 2024-09-12T02:17:43.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.844+0000 7fd28f7fe640 1 -- 172.21.15.16:0/3114622124 learned_addr learned my addr 172.21.15.16:0/3114622124 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:17:43.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.844+0000 7fd28effd640 1 --2- 172.21.15.16:0/3114622124 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b6b171b340 0x55b6b17341f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:43.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.844+0000 7fd28ffff640 1 --2- 172.21.15.16:0/3114622124 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b6b1736d50 0x55b6b1734730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:43.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.844+0000 7fd28f7fe640 1 -- 172.21.15.16:0/3114622124 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b6b1736d50 msgr2=0x55b6b1734730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:43.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.844+0000 7fd28f7fe640 1 --2- 172.21.15.16:0/3114622124 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b6b1736d50 0x55b6b1734730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:43.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.844+0000 7fd28f7fe640 1 -- 172.21.15.16:0/3114622124 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b6b171b340 msgr2=0x55b6b17341f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:43.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.844+0000 7fd28f7fe640 1 --2- 172.21.15.16:0/3114622124 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b6b171b340 0x55b6b17341f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:43.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.844+0000 7fd28f7fe640 1 -- 172.21.15.16:0/3114622124 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6b13f16a0 con 0x55b6b171aa20 2024-09-12T02:17:43.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.845+0000 7fd28f7fe640 1 --2- 172.21.15.16:0/3114622124 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b6b171aa20 0x55b6b1733cb0 secure :-1 s=READY pgs=2105 cs=0 l=1 rev1=1 crypto rx=0x7fd28001b040 tx=0x7fd280019f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfa9230c13d6aaf7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:43.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.845+0000 7fd26effd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:17:43.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.845+0000 7fd26effd640 1 -- 172.21.15.16:0/3114622124 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd28001d040 con 0x55b6b171aa20 2024-09-12T02:17:43.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.845+0000 7fd26effd640 1 -- 172.21.15.16:0/3114622124 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fd2800186c0 con 0x55b6b171aa20 2024-09-12T02:17:43.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.845+0000 7fd298017740 1 -- 172.21.15.16:0/3114622124 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd280018b20 con 0x55b6b171aa20 2024-09-12T02:17:43.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.845+0000 7fd26effd640 1 -- 172.21.15.16:0/3114622124 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd2800188c0 con 0x55b6b171aa20 2024-09-12T02:17:43.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.845+0000 7fd298017740 10 client.20769 fetch_fsmap learned FSMap version 25 2024-09-12T02:17:43.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.845+0000 7fd298017740 1 -- 172.21.15.16:0/3114622124 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b6b174de90 con 0x55b6b171aa20 2024-09-12T02:17:43.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.846+0000 7fd26effd640 1 -- 172.21.15.16:0/3114622124 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd280018b20 con 0x55b6b171aa20 2024-09-12T02:17:43.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.846+0000 7fd26effd640 1 -- 172.21.15.16:0/3114622124 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fd280020030 con 0x55b6b171aa20 2024-09-12T02:17:43.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.846+0000 7fd298017740 10 client.20769 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:17:43.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.846+0000 7fd298017740 10 client.20769 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:17:43.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.846+0000 7fd298017740 20 client.20769 populate_metadata read hostname 'smithi016' 2024-09-12T02:17:43.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.846+0000 7fd298017740 1 --2- 172.21.15.16:0/3114622124 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55b6b174f1e0 0x55b6b17515d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:43.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.846+0000 7fd298017740 4 client.20769 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:17:43.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.846+0000 7fd298017740 1 -- 172.21.15.16:0/3114622124 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b6b168d7f0 con 0x55b6b174f1e0 2024-09-12T02:17:43.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.846+0000 7fd28effd640 1 --2- 172.21.15.16:0/3114622124 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55b6b174f1e0 0x55b6b17515d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:43.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.847+0000 7fd26effd640 10 client.20769 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:17:43.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.847+0000 7fd28effd640 1 --2- 172.21.15.16:0/3114622124 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55b6b174f1e0 0x55b6b17515d0 crc :-1 s=READY pgs=467 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a069773956815335 server_cookie=cad14c7bbaa66d2 in_seq=0 out_seq=0 2024-09-12T02:17:43.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.847+0000 7fd26effd640 1 -- 172.21.15.16:0/3114622124 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b6b168d7f0 con 0x55b6b174f1e0 2024-09-12T02:17:43.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.847+0000 7fd26effd640 10 client.20769 handle_command_reply: tid=0 2024-09-12T02:17:43.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.848+0000 7fd298017740 1 client.20769 shutdown 2024-09-12T02:17:43.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.848+0000 7fd298017740 1 -- 172.21.15.16:0/3114622124 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b6b171aa20 msgr2=0x55b6b1733cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:43.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.848+0000 7fd298017740 1 --2- 172.21.15.16:0/3114622124 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b6b171aa20 0x55b6b1733cb0 secure :-1 s=READY pgs=2105 cs=0 l=1 rev1=1 crypto rx=0x7fd28001b040 tx=0x7fd280019f10 comp rx=0 tx=0).stop 2024-09-12T02:17:43.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.848+0000 7fd298017740 1 -- 172.21.15.16:0/3114622124 shutdown_connections 2024-09-12T02:17:43.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.848+0000 7fd298017740 1 --2- 172.21.15.16:0/3114622124 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55b6b174f1e0 0x55b6b17515d0 crc :-1 s=READY pgs=467 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:43.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.848+0000 7fd26effd640 0 client.20769 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:17:43.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.848+0000 7fd298017740 1 --2- 172.21.15.16:0/3114622124 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b6b1736d50 0x55b6b1734730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:43.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.848+0000 7fd298017740 1 --2- 172.21.15.16:0/3114622124 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b6b171b340 0x55b6b17341f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:43.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.848+0000 7fd298017740 1 --2- 172.21.15.16:0/3114622124 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b6b171aa20 0x55b6b1733cb0 unknown :-1 s=CLOSED pgs=2105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:43.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.848+0000 7fd298017740 1 -- 172.21.15.16:0/3114622124 >> 172.21.15.16:0/3114622124 conn(0x55b6b17170f0 msgr2=0x55b6b171ce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:43.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.848+0000 7fd298017740 1 -- 172.21.15.16:0/3114622124 shutdown_connections 2024-09-12T02:17:43.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.848+0000 7fd298017740 1 -- 172.21.15.16:0/3114622124 wait complete. 2024-09-12T02:17:43.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.848+0000 7fd298017740 20 client.20769 trim_cache size 0 max 16384 2024-09-12T02:17:43.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.873+0000 7fd298017740 1 Processor -- start 2024-09-12T02:17:43.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.873+0000 7fd298017740 1 -- start start 2024-09-12T02:17:43.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.873+0000 7fd298017740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b6b171aa20 0x55b6b1734580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:43.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.873+0000 7fd298017740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b6b171b340 0x55b6b1756c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:43.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.873+0000 7fd298017740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b6b1736d50 0x55b6b1757140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:43.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.873+0000 7fd298017740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd290072290 con 0x55b6b171b340 2024-09-12T02:17:43.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.873+0000 7fd298017740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd29006fb60 con 0x55b6b1736d50 2024-09-12T02:17:43.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.873+0000 7fd298017740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd29006d9b0 con 0x55b6b171aa20 2024-09-12T02:17:43.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.873+0000 7fd28effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b6b171b340 0x55b6b1756c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:43.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.873+0000 7fd28effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b6b171b340 0x55b6b1756c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47162/0 (socket says 172.21.15.16:47162) 2024-09-12T02:17:43.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.873+0000 7fd28effd640 1 -- 172.21.15.16:0/4267408216 learned_addr learned my addr 172.21.15.16:0/4267408216 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:17:43.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.873+0000 7fd28ffff640 1 --2- 172.21.15.16:0/4267408216 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b6b1736d50 0x55b6b1757140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:43.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.874+0000 7fd28ffff640 1 -- 172.21.15.16:0/4267408216 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b6b171aa20 msgr2=0x55b6b1734580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:43.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.874+0000 7fd28ffff640 1 --2- 172.21.15.16:0/4267408216 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b6b171aa20 0x55b6b1734580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:43.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.874+0000 7fd28ffff640 1 -- 172.21.15.16:0/4267408216 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b6b171b340 msgr2=0x55b6b1756c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:43.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.874+0000 7fd28ffff640 1 --2- 172.21.15.16:0/4267408216 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b6b171b340 0x55b6b1756c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:43.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.874+0000 7fd28ffff640 1 -- 172.21.15.16:0/4267408216 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6b1718230 con 0x55b6b1736d50 2024-09-12T02:17:43.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.874+0000 7fd28ffff640 1 --2- 172.21.15.16:0/4267408216 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b6b1736d50 0x55b6b1757140 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fd28809b9c0 tx=0x7fd28809b730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=83dffc0f3cbcdc26 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:43.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.874+0000 7fd2757fa640 1 -- 172.21.15.16:0/4267408216 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2880b8020 con 0x55b6b1736d50 2024-09-12T02:17:43.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.875+0000 7fd2757fa640 1 -- 172.21.15.16:0/4267408216 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd2880a0070 con 0x55b6b1736d50 2024-09-12T02:17:43.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.875+0000 7fd2757fa640 1 -- 172.21.15.16:0/4267408216 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd28809b370 con 0x55b6b1736d50 2024-09-12T02:17:43.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.875+0000 7fd298017740 1 -- 172.21.15.16:0/4267408216 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b6b1736d50 msgr2=0x55b6b1757140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:43.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.875+0000 7fd298017740 1 --2- 172.21.15.16:0/4267408216 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b6b1736d50 0x55b6b1757140 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fd28809b9c0 tx=0x7fd28809b730 comp rx=0 tx=0).stop 2024-09-12T02:17:43.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.875+0000 7fd298017740 1 -- 172.21.15.16:0/4267408216 shutdown_connections 2024-09-12T02:17:43.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.875+0000 7fd298017740 1 --2- 172.21.15.16:0/4267408216 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b6b1736d50 0x55b6b1757140 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:43.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.875+0000 7fd298017740 1 --2- 172.21.15.16:0/4267408216 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b6b171b340 0x55b6b1756c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:43.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.875+0000 7fd298017740 1 --2- 172.21.15.16:0/4267408216 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b6b171aa20 0x55b6b1734580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:43.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.875+0000 7fd298017740 1 -- 172.21.15.16:0/4267408216 >> 172.21.15.16:0/4267408216 conn(0x55b6b17170f0 msgr2=0x55b6b1735c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:43.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.876+0000 7fd298017740 1 -- 172.21.15.16:0/4267408216 shutdown_connections 2024-09-12T02:17:43.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.876+0000 7fd298017740 1 -- 172.21.15.16:0/4267408216 wait complete. 2024-09-12T02:17:43.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.876+0000 7fd298017740 1 -- start start 2024-09-12T02:17:43.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.876+0000 7fd298017740 1 Processor -- start 2024-09-12T02:17:43.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.877+0000 7fd298017740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b6b171aa20 0x55b6b1730cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:43.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.877+0000 7fd298017740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b6b171b340 0x55b6b1731200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:43.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.877+0000 7fd298017740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b6b1736d50 0x55b6b1731740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:43.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.877+0000 7fd298017740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd29006e020 con 0x55b6b171b340 2024-09-12T02:17:43.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.877+0000 7fd298017740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd29006f490 con 0x55b6b1736d50 2024-09-12T02:17:43.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.877+0000 7fd298017740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd29006e6f0 con 0x55b6b171aa20 2024-09-12T02:17:43.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.877+0000 7fd28effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b6b171b340 0x55b6b1731200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:43.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.877+0000 7fd28effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b6b171b340 0x55b6b1731200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47172/0 (socket says 172.21.15.16:47172) 2024-09-12T02:17:43.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.877+0000 7fd28effd640 1 -- 172.21.15.16:0/1453316875 learned_addr learned my addr 172.21.15.16:0/1453316875 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:17:43.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.877+0000 7fd28ffff640 1 --2- 172.21.15.16:0/1453316875 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b6b1736d50 0x55b6b1731740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:43.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.877+0000 7fd28f7fe640 1 --2- 172.21.15.16:0/1453316875 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b6b171aa20 0x55b6b1730cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:43.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.877+0000 7fd298017740 10 client.0 _ll_register_callbacks cb 0x7fd28001b070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:17:43.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.877+0000 7fd28effd640 1 -- 172.21.15.16:0/1453316875 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b6b171aa20 msgr2=0x55b6b1730cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:43.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.877+0000 7fd28effd640 1 --2- 172.21.15.16:0/1453316875 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b6b171aa20 0x55b6b1730cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:43.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.877+0000 7fd28effd640 1 -- 172.21.15.16:0/1453316875 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b6b1736d50 msgr2=0x55b6b1731740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:43.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.877+0000 7fd28effd640 1 --2- 172.21.15.16:0/1453316875 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b6b1736d50 0x55b6b1731740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:43.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.877+0000 7fd28effd640 1 -- 172.21.15.16:0/1453316875 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6b17b8bd0 con 0x55b6b171b340 2024-09-12T02:17:43.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.878+0000 7fd28effd640 1 --2- 172.21.15.16:0/1453316875 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b6b171b340 0x55b6b1731200 secure :-1 s=READY pgs=2106 cs=0 l=1 rev1=1 crypto rx=0x7fd284009930 tx=0x7fd284009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c8479f8c8b66e5a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:43.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.878+0000 7fd26effd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:17:43.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.878+0000 7fd26effd640 1 -- 172.21.15.16:0/1453316875 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd284035030 con 0x55b6b171b340 2024-09-12T02:17:43.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.878+0000 7fd26effd640 1 -- 172.21.15.16:0/1453316875 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fd28403bdb0 con 0x55b6b171b340 2024-09-12T02:17:43.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.878+0000 7fd298017740 1 -- 172.21.15.16:0/1453316875 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd2880a0070 con 0x55b6b171b340 2024-09-12T02:17:43.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.878+0000 7fd26effd640 1 -- 172.21.15.16:0/1453316875 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd28403d450 con 0x55b6b171b340 2024-09-12T02:17:43.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.878+0000 7fd298017740 10 client.20775 fetch_fsmap learned FSMap version 25 2024-09-12T02:17:43.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.878+0000 7fd298017740 1 -- 172.21.15.16:0/1453316875 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b6b17b9ee0 con 0x55b6b171b340 2024-09-12T02:17:43.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.878+0000 7fd26effd640 1 -- 172.21.15.16:0/1453316875 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd2880a0070 con 0x55b6b171b340 2024-09-12T02:17:43.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.879+0000 7fd26effd640 1 -- 172.21.15.16:0/1453316875 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fd284003820 con 0x55b6b171b340 2024-09-12T02:17:43.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.879+0000 7fd298017740 10 client.20775 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:17:43.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.879+0000 7fd298017740 10 client.20775 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:17:43.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.879+0000 7fd298017740 20 client.20775 populate_metadata read hostname 'smithi016' 2024-09-12T02:17:43.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.879+0000 7fd298017740 1 --2- 172.21.15.16:0/1453316875 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55b6b174f1e0 0x55b6b17bc6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:43.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.879+0000 7fd298017740 4 client.20775 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-12T02:17:43.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.879+0000 7fd298017740 1 -- 172.21.15.16:0/1453316875 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55b6b1687d00 con 0x55b6b174f1e0 2024-09-12T02:17:43.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.879+0000 7fd28f7fe640 1 --2- 172.21.15.16:0/1453316875 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55b6b174f1e0 0x55b6b17bc6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:43.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.880+0000 7fd28f7fe640 1 --2- 172.21.15.16:0/1453316875 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55b6b174f1e0 0x55b6b17bc6d0 crc :-1 s=READY pgs=468 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1fddfdd14f0df566 server_cookie=2d38fdaebc4a51a0 in_seq=0 out_seq=0 2024-09-12T02:17:43.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.880+0000 7fd26effd640 10 client.20775 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:17:43.886 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:17:43.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.881+0000 7fd26effd640 1 -- 172.21.15.16:0/1453316875 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55b6b1687d00 con 0x55b6b174f1e0 2024-09-12T02:17:43.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.881+0000 7fd26effd640 10 client.20775 handle_command_reply: tid=0 2024-09-12T02:17:43.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.881+0000 7fd298017740 1 client.20775 shutdown 2024-09-12T02:17:43.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.881+0000 7fd298017740 1 -- 172.21.15.16:0/1453316875 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b6b171b340 msgr2=0x55b6b1731200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:43.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.881+0000 7fd298017740 1 --2- 172.21.15.16:0/1453316875 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b6b171b340 0x55b6b1731200 secure :-1 s=READY pgs=2106 cs=0 l=1 rev1=1 crypto rx=0x7fd284009930 tx=0x7fd284009260 comp rx=0 tx=0).stop 2024-09-12T02:17:43.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.881+0000 7fd298017740 1 -- 172.21.15.16:0/1453316875 shutdown_connections 2024-09-12T02:17:43.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.881+0000 7fd298017740 1 --2- 172.21.15.16:0/1453316875 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55b6b174f1e0 0x55b6b17bc6d0 crc :-1 s=READY pgs=468 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:43.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.881+0000 7fd298017740 1 --2- 172.21.15.16:0/1453316875 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b6b1736d50 0x55b6b1731740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:43.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.881+0000 7fd298017740 1 --2- 172.21.15.16:0/1453316875 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b6b171b340 0x55b6b1731200 unknown :-1 s=CLOSED pgs=2106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:43.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.881+0000 7fd298017740 1 --2- 172.21.15.16:0/1453316875 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b6b171aa20 0x55b6b1730cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:43.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.881+0000 7fd298017740 1 -- 172.21.15.16:0/1453316875 >> 172.21.15.16:0/1453316875 conn(0x55b6b17170f0 msgr2=0x55b6b1715a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:43.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.881+0000 7fd26effd640 0 client.20775 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:17:43.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.881+0000 7fd298017740 1 -- 172.21.15.16:0/1453316875 shutdown_connections 2024-09-12T02:17:43.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.882+0000 7fd298017740 1 -- 172.21.15.16:0/1453316875 wait complete. 2024-09-12T02:17:43.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.882+0000 7fd298017740 20 client.20775 trim_cache size 0 max 16384 2024-09-12T02:17:43.891 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T02:17:43.891 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-09-12T02:17:43.891 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "5cb8d932-ef38-4b29-a2d4-b0c5fc070310", 2024-09-12T02:17:43.892 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-09-12T02:17:43.892 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T02:17:43.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.882+0000 7fd295c3e640 1 -- 172.21.15.16:0/85231231 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fd2780a2c00 msgr2=0x7fd2780a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:43.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.882+0000 7fd295c3e640 1 --2- 172.21.15.16:0/85231231 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fd2780a2c00 0x7fd2780a50c0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fd284005940 tx=0x7fd284000bc0 comp rx=0 tx=0).stop 2024-09-12T02:17:43.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.882+0000 7fd295c3e640 1 -- 172.21.15.16:0/85231231 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2901082a0 msgr2=0x7fd29019efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:43.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.882+0000 7fd295c3e640 1 --2- 172.21.15.16:0/85231231 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2901082a0 0x7fd29019efb0 secure :-1 s=READY pgs=2103 cs=0 l=1 rev1=1 crypto rx=0x7fd28800cce0 tx=0x7fd28800a620 comp rx=0 tx=0).stop 2024-09-12T02:17:43.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.883+0000 7fd295c3e640 1 -- 172.21.15.16:0/85231231 shutdown_connections 2024-09-12T02:17:43.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.883+0000 7fd295c3e640 1 --2- 172.21.15.16:0/85231231 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fd2780a2c00 0x7fd2780a50c0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:43.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.883+0000 7fd295c3e640 1 --2- 172.21.15.16:0/85231231 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2901082a0 0x7fd29019efb0 unknown :-1 s=CLOSED pgs=2103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:43.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.883+0000 7fd295c3e640 1 --2- 172.21.15.16:0/85231231 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd2901036e0 0x7fd29019ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:43.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.883+0000 7fd295c3e640 1 --2- 172.21.15.16:0/85231231 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd290102d30 0x7fd29019a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:43.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.883+0000 7fd295c3e640 1 -- 172.21.15.16:0/85231231 >> 172.21.15.16:0/85231231 conn(0x7fd2900fc7d0 msgr2=0x7fd2900fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:43.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.883+0000 7fd295c3e640 1 -- 172.21.15.16:0/85231231 shutdown_connections 2024-09-12T02:17:43.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:43.883+0000 7fd295c3e640 1 -- 172.21.15.16:0/85231231 wait complete. 2024-09-12T02:17:43.927 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.924+0000 7f48954a5640 1 -- 172.21.15.16:0/2314395826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4890102d50 msgr2=0x7f48901031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:43.927 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.924+0000 7f48954a5640 1 --2- 172.21.15.16:0/2314395826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4890102d50 0x7f48901031d0 secure :-1 s=READY pgs=2107 cs=0 l=1 rev1=1 crypto rx=0x7f4880009900 tx=0x7f488002a410 comp rx=0 tx=0).stop 2024-09-12T02:17:43.927 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.924+0000 7f48954a5640 1 -- 172.21.15.16:0/2314395826 shutdown_connections 2024-09-12T02:17:43.927 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.924+0000 7f48954a5640 1 --2- 172.21.15.16:0/2314395826 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4890103710 0x7f4890109fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:43.927 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.924+0000 7f48954a5640 1 --2- 172.21.15.16:0/2314395826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4890102d50 0x7f48901031d0 unknown :-1 s=CLOSED pgs=2107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:43.927 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.924+0000 7f48954a5640 1 --2- 172.21.15.16:0/2314395826 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4890101b60 0x7f4890101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:43.928 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.924+0000 7f48954a5640 1 -- 172.21.15.16:0/2314395826 >> 172.21.15.16:0/2314395826 conn(0x7f48900fd2e0 msgr2=0x7f48900ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:43.928 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.924+0000 7f48954a5640 1 -- 172.21.15.16:0/2314395826 shutdown_connections 2024-09-12T02:17:43.928 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.924+0000 7f48954a5640 1 -- 172.21.15.16:0/2314395826 wait complete. 2024-09-12T02:17:43.928 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.925+0000 7f48954a5640 1 Processor -- start 2024-09-12T02:17:43.928 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.925+0000 7f48954a5640 1 -- start start 2024-09-12T02:17:43.928 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.925+0000 7f48954a5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4890101b60 0x7f4890197dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:43.928 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.925+0000 7f48954a5640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4890102d50 0x7f4890198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:43.928 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.925+0000 7f48954a5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4890103710 0x7f489019f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:43.928 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.925+0000 7f48954a5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f489010d7e0 con 0x7f4890103710 2024-09-12T02:17:43.928 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.925+0000 7f48954a5640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f489010d660 con 0x7f4890101b60 2024-09-12T02:17:43.928 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.925+0000 7f48954a5640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f489010d960 con 0x7f4890102d50 2024-09-12T02:17:43.928 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.925+0000 7f488f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4890103710 0x7f489019f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:43.929 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.925+0000 7f488f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4890103710 0x7f489019f380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47190/0 (socket says 172.21.15.16:47190) 2024-09-12T02:17:43.929 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.925+0000 7f488f7fe640 1 -- 172.21.15.16:0/602067268 learned_addr learned my addr 172.21.15.16:0/602067268 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:17:43.929 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.925+0000 7f488effd640 1 --2- 172.21.15.16:0/602067268 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4890101b60 0x7f4890197dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:43.929 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.925+0000 7f488f7fe640 1 -- 172.21.15.16:0/602067268 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4890102d50 msgr2=0x7f4890198310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:43.929 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.925+0000 7f488f7fe640 1 --2- 172.21.15.16:0/602067268 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4890102d50 0x7f4890198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:43.929 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.925+0000 7f488f7fe640 1 -- 172.21.15.16:0/602067268 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4890101b60 msgr2=0x7f4890197dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:43.929 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.925+0000 7f488f7fe640 1 --2- 172.21.15.16:0/602067268 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4890101b60 0x7f4890197dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:43.929 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.925+0000 7f488f7fe640 1 -- 172.21.15.16:0/602067268 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f489019f9f0 con 0x7f4890103710 2024-09-12T02:17:43.929 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.925+0000 7f488effd640 1 --2- 172.21.15.16:0/602067268 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4890101b60 0x7f4890197dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:17:43.929 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.926+0000 7f488f7fe640 1 --2- 172.21.15.16:0/602067268 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4890103710 0x7f489019f380 secure :-1 s=READY pgs=2108 cs=0 l=1 rev1=1 crypto rx=0x7f488800eaa0 tx=0x7f488800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:43.929 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.926+0000 7f4887fff640 1 -- 172.21.15.16:0/602067268 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f488800cdb0 con 0x7f4890103710 2024-09-12T02:17:43.929 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.926+0000 7f4887fff640 1 -- 172.21.15.16:0/602067268 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4888005020 con 0x7f4890103710 2024-09-12T02:17:43.929 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.926+0000 7f48954a5640 1 -- 172.21.15.16:0/602067268 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f489019fce0 con 0x7f4890103710 2024-09-12T02:17:43.930 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.926+0000 7f4887fff640 1 -- 172.21.15.16:0/602067268 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48880116b0 con 0x7f4890103710 2024-09-12T02:17:43.930 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.926+0000 7f48954a5640 1 -- 172.21.15.16:0/602067268 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48901a0230 con 0x7f4890103710 2024-09-12T02:17:43.935 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.928+0000 7f4887fff640 1 -- 172.21.15.16:0/602067268 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f4888005560 con 0x7f4890103710 2024-09-12T02:17:43.935 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.928+0000 7f4898243740 1 Processor -- start 2024-09-12T02:17:43.935 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.929+0000 7f4898243740 1 -- start start 2024-09-12T02:17:43.935 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.929+0000 7f4898243740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55af940029a0 0x55af94002da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:43.935 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.929+0000 7f4898243740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55af940032e0 0x55af94007da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:43.935 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.929+0000 7f4898243740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55af94008370 0x55af9400a830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:43.935 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.929+0000 7f4898243740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55af93f662f0 con 0x55af940029a0 2024-09-12T02:17:43.935 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.929+0000 7f4898243740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55af93f0e270 con 0x55af940032e0 2024-09-12T02:17:43.935 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.929+0000 7f4898243740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55af93f622b0 con 0x55af94008370 2024-09-12T02:17:43.935 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.929+0000 7f488e7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55af940029a0 0x55af94002da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:43.936 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.929+0000 7f488e7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55af940029a0 0x55af94002da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47202/0 (socket says 172.21.15.16:47202) 2024-09-12T02:17:43.936 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.929+0000 7f488e7fc640 1 -- 172.21.15.16:0/1573679965 learned_addr learned my addr 172.21.15.16:0/1573679965 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:17:43.936 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.929+0000 7f488effd640 1 --2- 172.21.15.16:0/1573679965 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55af940032e0 0x55af94007da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:43.936 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.929+0000 7f4887fff640 1 --2- 172.21.15.16:0/602067268 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f487c0a2cb0 0x7f487c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:43.936 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.929+0000 7f488e7fc640 1 --2- 172.21.15.16:0/1573679965 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55af94008370 0x55af9400a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:43.936 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.929+0000 7f488e7fc640 1 -- 172.21.15.16:0/1573679965 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55af94008370 msgr2=0x55af9400a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:43.936 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.929+0000 7f488e7fc640 1 --2- 172.21.15.16:0/1573679965 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55af94008370 0x55af9400a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:43.936 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.929+0000 7f488e7fc640 1 -- 172.21.15.16:0/1573679965 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55af940032e0 msgr2=0x55af94007da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:43.936 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.929+0000 7f488e7fc640 1 --2- 172.21.15.16:0/1573679965 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55af940032e0 0x55af94007da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:43.936 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.929+0000 7f488e7fc640 1 -- 172.21.15.16:0/1573679965 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af93f0d340 con 0x55af940029a0 2024-09-12T02:17:43.936 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.929+0000 7f488f7fe640 1 --2- 172.21.15.16:0/602067268 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f487c0a2cb0 0x7f487c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:43.936 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.930+0000 7f488effd640 1 --2- 172.21.15.16:0/1573679965 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55af940032e0 0x55af94007da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:17:43.937 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.930+0000 7f4887fff640 1 -- 172.21.15.16:0/602067268 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f48880cf2f0 con 0x7f4890103710 2024-09-12T02:17:43.937 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.930+0000 7f488e7fc640 1 --2- 172.21.15.16:0/1573679965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55af940029a0 0x55af94002da0 secure :-1 s=READY pgs=2109 cs=0 l=1 rev1=1 crypto rx=0x7f4880005dd0 tx=0x7f4880000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0f30b1d84f411b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:43.937 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.930+0000 7f48857fa640 1 -- 172.21.15.16:0/1573679965 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4880044070 con 0x55af940029a0 2024-09-12T02:17:43.937 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.930+0000 7f48857fa640 1 -- 172.21.15.16:0/1573679965 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48800319b0 con 0x55af940029a0 2024-09-12T02:17:43.937 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.930+0000 7f48857fa640 1 -- 172.21.15.16:0/1573679965 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4880035040 con 0x55af940029a0 2024-09-12T02:17:43.937 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.931+0000 7f4898243740 1 -- 172.21.15.16:0/1573679965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55af940029a0 msgr2=0x55af94002da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:43.937 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.931+0000 7f488f7fe640 1 --2- 172.21.15.16:0/602067268 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f487c0a2cb0 0x7f487c0a5170 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f4888097fb0 tx=0x7f488801eb40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:43.937 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.931+0000 7f4898243740 1 --2- 172.21.15.16:0/1573679965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55af940029a0 0x55af94002da0 secure :-1 s=READY pgs=2109 cs=0 l=1 rev1=1 crypto rx=0x7f4880005dd0 tx=0x7f4880000bc0 comp rx=0 tx=0).stop 2024-09-12T02:17:43.937 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.931+0000 7f4898243740 1 -- 172.21.15.16:0/1573679965 shutdown_connections 2024-09-12T02:17:43.937 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.931+0000 7f4898243740 1 --2- 172.21.15.16:0/1573679965 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55af94008370 0x55af9400a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:43.937 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.931+0000 7f4898243740 1 --2- 172.21.15.16:0/1573679965 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55af940032e0 0x55af94007da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:43.937 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.931+0000 7f4898243740 1 --2- 172.21.15.16:0/1573679965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55af940029a0 0x55af94002da0 unknown :-1 s=CLOSED pgs=2109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:43.937 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.931+0000 7f4898243740 1 -- 172.21.15.16:0/1573679965 >> 172.21.15.16:0/1573679965 conn(0x55af93ffe010 msgr2=0x55af93ffe410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:43.938 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.931+0000 7f4898243740 1 -- 172.21.15.16:0/1573679965 shutdown_connections 2024-09-12T02:17:43.938 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.931+0000 7f4898243740 1 -- 172.21.15.16:0/1573679965 wait complete. 2024-09-12T02:17:43.938 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.931+0000 7f4898243740 1 -- start start 2024-09-12T02:17:43.938 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.931+0000 7f4898243740 1 Processor -- start 2024-09-12T02:17:43.938 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.932+0000 7f4898243740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55af940029a0 0x55af9400cc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:43.938 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.932+0000 7f4898243740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55af940032e0 0x55af9400d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:43.938 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.932+0000 7f4898243740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55af94008370 0x55af94014190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:43.938 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.932+0000 7f4898243740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55af9400b2e0 con 0x55af94008370 2024-09-12T02:17:43.938 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.932+0000 7f4898243740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55af93eef290 con 0x55af940032e0 2024-09-12T02:17:43.938 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.932+0000 7f4898243740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55af9400b460 con 0x55af940029a0 2024-09-12T02:17:43.938 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.932+0000 7f4898243740 10 client.0 _ll_register_callbacks cb 0x55af93f20140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:17:43.938 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.932+0000 7f488e7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55af94008370 0x55af94014190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:43.938 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.932+0000 7f488e7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55af94008370 0x55af94014190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47206/0 (socket says 172.21.15.16:47206) 2024-09-12T02:17:43.939 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.932+0000 7f488e7fc640 1 -- 172.21.15.16:0/3070129365 learned_addr learned my addr 172.21.15.16:0/3070129365 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:17:43.939 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.932+0000 7f488e7fc640 1 --2- 172.21.15.16:0/3070129365 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55af940029a0 0x55af9400cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:43.939 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.932+0000 7f488effd640 1 --2- 172.21.15.16:0/3070129365 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55af940032e0 0x55af9400d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:43.939 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.932+0000 7f488e7fc640 1 -- 172.21.15.16:0/3070129365 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55af940029a0 msgr2=0x55af9400cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:43.939 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.932+0000 7f488e7fc640 1 --2- 172.21.15.16:0/3070129365 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55af940029a0 0x55af9400cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:43.939 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.932+0000 7f488e7fc640 1 -- 172.21.15.16:0/3070129365 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55af940032e0 msgr2=0x55af9400d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:43.939 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.932+0000 7f488e7fc640 1 --2- 172.21.15.16:0/3070129365 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55af940032e0 0x55af9400d1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:43.940 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.932+0000 7f488e7fc640 1 -- 172.21.15.16:0/3070129365 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af93c79210 con 0x55af94008370 2024-09-12T02:17:43.940 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.932+0000 7f488e7fc640 1 --2- 172.21.15.16:0/3070129365 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55af94008370 0x55af94014190 secure :-1 s=READY pgs=2110 cs=0 l=1 rev1=1 crypto rx=0x7f4880031610 tx=0x7f4880031b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8aa0a4d646cb8199 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:43.940 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.932+0000 7f48767fc640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:17:43.940 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.932+0000 7f48767fc640 1 -- 172.21.15.16:0/3070129365 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4880035050 con 0x55af94008370 2024-09-12T02:17:43.940 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.932+0000 7f48767fc640 1 -- 172.21.15.16:0/3070129365 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f4880004030 con 0x55af94008370 2024-09-12T02:17:43.940 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.932+0000 7f4898243740 1 -- 172.21.15.16:0/3070129365 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f48800319b0 con 0x55af94008370 2024-09-12T02:17:43.940 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.932+0000 7f48767fc640 1 -- 172.21.15.16:0/3070129365 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4880044070 con 0x55af94008370 2024-09-12T02:17:43.940 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.932+0000 7f4898243740 10 client.20787 fetch_fsmap learned FSMap version 25 2024-09-12T02:17:43.941 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.932+0000 7f4898243740 1 -- 172.21.15.16:0/3070129365 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55af93d45620 con 0x55af94008370 2024-09-12T02:17:43.941 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.933+0000 7f48767fc640 1 -- 172.21.15.16:0/3070129365 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48800319b0 con 0x55af94008370 2024-09-12T02:17:43.941 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.933+0000 7f48767fc640 1 -- 172.21.15.16:0/3070129365 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f488004b070 con 0x55af94008370 2024-09-12T02:17:43.941 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.933+0000 7f4898243740 10 client.20787 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:17:43.941 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.933+0000 7f4898243740 10 client.20787 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:17:43.941 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.933+0000 7f4898243740 20 client.20787 populate_metadata read hostname 'smithi016' 2024-09-12T02:17:43.941 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.933+0000 7f4898243740 1 --2- 172.21.15.16:0/3070129365 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55af94018490 0x55af9401a880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:43.941 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.933+0000 7f4898243740 4 client.20787 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:17:43.941 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.933+0000 7f4898243740 1 -- 172.21.15.16:0/3070129365 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55af93f73a20 con 0x55af94018490 2024-09-12T02:17:43.942 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.933+0000 7f488effd640 1 --2- 172.21.15.16:0/3070129365 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55af94018490 0x55af9401a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:43.942 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.933+0000 7f488effd640 1 --2- 172.21.15.16:0/3070129365 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55af94018490 0x55af9401a880 crc :-1 s=READY pgs=469 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b2866c9e6496ead server_cookie=2a611691ee9deaf9 in_seq=0 out_seq=0 2024-09-12T02:17:43.942 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.933+0000 7f48767fc640 10 client.20787 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:17:43.942 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.934+0000 7f48767fc640 1 -- 172.21.15.16:0/3070129365 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55af93f73a20 con 0x55af94018490 2024-09-12T02:17:43.942 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.934+0000 7f48767fc640 10 client.20787 handle_command_reply: tid=0 2024-09-12T02:17:43.942 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.934+0000 7f4898243740 1 client.20787 shutdown 2024-09-12T02:17:43.942 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.934+0000 7f4898243740 1 -- 172.21.15.16:0/3070129365 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55af94008370 msgr2=0x55af94014190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:43.942 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.934+0000 7f4898243740 1 --2- 172.21.15.16:0/3070129365 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55af94008370 0x55af94014190 secure :-1 s=READY pgs=2110 cs=0 l=1 rev1=1 crypto rx=0x7f4880031610 tx=0x7f4880031b60 comp rx=0 tx=0).stop 2024-09-12T02:17:43.942 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.934+0000 7f4898243740 1 -- 172.21.15.16:0/3070129365 shutdown_connections 2024-09-12T02:17:43.942 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.934+0000 7f4898243740 1 --2- 172.21.15.16:0/3070129365 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55af94018490 0x55af9401a880 crc :-1 s=READY pgs=469 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:43.942 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.934+0000 7f4898243740 1 --2- 172.21.15.16:0/3070129365 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55af94008370 0x55af94014190 unknown :-1 s=CLOSED pgs=2110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:43.942 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.934+0000 7f4898243740 1 --2- 172.21.15.16:0/3070129365 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55af940032e0 0x55af9400d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:43.943 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.934+0000 7f4898243740 1 --2- 172.21.15.16:0/3070129365 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55af940029a0 0x55af9400cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:43.943 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.934+0000 7f4898243740 1 -- 172.21.15.16:0/3070129365 >> 172.21.15.16:0/3070129365 conn(0x55af93ffe010 msgr2=0x55af93fff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:43.943 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.934+0000 7f48767fc640 0 client.20787 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:17:43.943 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.934+0000 7f4898243740 1 -- 172.21.15.16:0/3070129365 shutdown_connections 2024-09-12T02:17:43.943 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.934+0000 7f4898243740 1 -- 172.21.15.16:0/3070129365 wait complete. 2024-09-12T02:17:43.943 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.935+0000 7f4898243740 20 client.20787 trim_cache size 0 max 16384 2024-09-12T02:17:43.953 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.949+0000 7f4898243740 1 Processor -- start 2024-09-12T02:17:43.953 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.949+0000 7f4898243740 1 -- start start 2024-09-12T02:17:43.953 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.949+0000 7f4898243740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55af940029a0 0x55af93ffeec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:43.953 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.949+0000 7f4898243740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55af940032e0 0x55af94014150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:43.953 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.949+0000 7f4898243740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55af94008370 0x55af93fff400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:43.953 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.949+0000 7f4898243740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f48901ab630 con 0x55af94008370 2024-09-12T02:17:43.954 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.949+0000 7f4898243740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f48901a8f00 con 0x55af940032e0 2024-09-12T02:17:43.954 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.949+0000 7f4898243740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f48901a6d50 con 0x55af940029a0 2024-09-12T02:17:43.954 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.949+0000 7f488e7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55af94008370 0x55af93fff400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:43.954 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.949+0000 7f488e7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55af94008370 0x55af93fff400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47214/0 (socket says 172.21.15.16:47214) 2024-09-12T02:17:43.954 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.949+0000 7f488e7fc640 1 -- 172.21.15.16:0/1047168508 learned_addr learned my addr 172.21.15.16:0/1047168508 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:17:43.954 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.949+0000 7f488e7fc640 1 -- 172.21.15.16:0/1047168508 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55af940029a0 msgr2=0x55af93ffeec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:43.954 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.949+0000 7f488e7fc640 1 --2- 172.21.15.16:0/1047168508 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55af940029a0 0x55af93ffeec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:43.954 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.949+0000 7f488e7fc640 1 -- 172.21.15.16:0/1047168508 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55af940032e0 msgr2=0x55af94014150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:43.954 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.949+0000 7f488e7fc640 1 --2- 172.21.15.16:0/1047168508 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55af940032e0 0x55af94014150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:43.954 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.949+0000 7f488e7fc640 1 -- 172.21.15.16:0/1047168508 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af93ffde30 con 0x55af94008370 2024-09-12T02:17:43.955 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.950+0000 7f488e7fc640 1 --2- 172.21.15.16:0/1047168508 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55af94008370 0x55af93fff400 secure :-1 s=READY pgs=2111 cs=0 l=1 rev1=1 crypto rx=0x7f4880005c40 tx=0x7f4880031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f559172d82b4ca1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:43.955 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.950+0000 7f4884ff9640 1 -- 172.21.15.16:0/1047168508 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4880035040 con 0x55af94008370 2024-09-12T02:17:43.955 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.950+0000 7f4884ff9640 1 -- 172.21.15.16:0/1047168508 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4880004470 con 0x55af94008370 2024-09-12T02:17:43.955 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.950+0000 7f4884ff9640 1 -- 172.21.15.16:0/1047168508 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4880031970 con 0x55af94008370 2024-09-12T02:17:43.955 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.950+0000 7f4898243740 1 -- 172.21.15.16:0/1047168508 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55af94008370 msgr2=0x55af93fff400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:43.955 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.950+0000 7f4898243740 1 --2- 172.21.15.16:0/1047168508 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55af94008370 0x55af93fff400 secure :-1 s=READY pgs=2111 cs=0 l=1 rev1=1 crypto rx=0x7f4880005c40 tx=0x7f4880031400 comp rx=0 tx=0).stop 2024-09-12T02:17:43.955 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.950+0000 7f4898243740 1 -- 172.21.15.16:0/1047168508 shutdown_connections 2024-09-12T02:17:43.956 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.950+0000 7f4898243740 1 --2- 172.21.15.16:0/1047168508 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55af94008370 0x55af93fff400 unknown :-1 s=CLOSED pgs=2111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:43.956 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.950+0000 7f4898243740 1 --2- 172.21.15.16:0/1047168508 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55af940032e0 0x55af94014150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:43.956 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.950+0000 7f4898243740 1 --2- 172.21.15.16:0/1047168508 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55af940029a0 0x55af93ffeec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:43.956 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.951+0000 7f4898243740 1 -- 172.21.15.16:0/1047168508 >> 172.21.15.16:0/1047168508 conn(0x55af93ffe010 msgr2=0x55af9400cc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:43.956 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.951+0000 7f4898243740 1 -- 172.21.15.16:0/1047168508 shutdown_connections 2024-09-12T02:17:43.956 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.951+0000 7f4898243740 1 -- 172.21.15.16:0/1047168508 wait complete. 2024-09-12T02:17:43.957 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.951+0000 7f4898243740 1 -- start start 2024-09-12T02:17:43.957 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.951+0000 7f4898243740 1 Processor -- start 2024-09-12T02:17:43.957 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.951+0000 7f4898243740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55af940029a0 0x55af94011590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:43.957 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.951+0000 7f4898243740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55af940032e0 0x55af94011ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:43.957 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.951+0000 7f4898243740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55af94008370 0x55af94012ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:43.957 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.951+0000 7f4898243740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f48901a73c0 con 0x55af94008370 2024-09-12T02:17:43.957 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.951+0000 7f4898243740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f48901a8830 con 0x55af940029a0 2024-09-12T02:17:43.958 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.951+0000 7f4898243740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f48901a7a90 con 0x55af940032e0 2024-09-12T02:17:43.958 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.951+0000 7f4898243740 10 client.0 _ll_register_callbacks cb 0x7f4880042070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:17:43.958 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.951+0000 7f488e7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55af94008370 0x55af94012ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:43.958 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.951+0000 7f488e7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55af94008370 0x55af94012ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47228/0 (socket says 172.21.15.16:47228) 2024-09-12T02:17:43.958 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.951+0000 7f488e7fc640 1 -- 172.21.15.16:0/791608840 learned_addr learned my addr 172.21.15.16:0/791608840 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:17:43.958 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.951+0000 7f488e7fc640 1 --2- 172.21.15.16:0/791608840 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55af940029a0 0x55af94011590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:43.958 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.951+0000 7f488e7fc640 1 -- 172.21.15.16:0/791608840 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55af940032e0 msgr2=0x55af94011ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:43.958 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.951+0000 7f488e7fc640 1 --2- 172.21.15.16:0/791608840 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55af940032e0 0x55af94011ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:43.958 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.951+0000 7f488e7fc640 1 -- 172.21.15.16:0/791608840 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55af940029a0 msgr2=0x55af94011590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:43.958 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.951+0000 7f488e7fc640 1 --2- 172.21.15.16:0/791608840 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55af940029a0 0x55af94011590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:43.959 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.952+0000 7f488e7fc640 1 -- 172.21.15.16:0/791608840 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af940130f0 con 0x55af94008370 2024-09-12T02:17:43.959 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.952+0000 7f488e7fc640 1 --2- 172.21.15.16:0/791608840 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55af94008370 0x55af94012ae0 secure :-1 s=READY pgs=2112 cs=0 l=1 rev1=1 crypto rx=0x7f488003ea00 tx=0x7f4880005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2551827f7f92ddc server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:43.959 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.952+0000 7f48767fc640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:17:43.959 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.952+0000 7f48767fc640 1 -- 172.21.15.16:0/791608840 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4880035070 con 0x55af94008370 2024-09-12T02:17:43.959 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.952+0000 7f4898243740 1 -- 172.21.15.16:0/791608840 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4880004470 con 0x55af94008370 2024-09-12T02:17:43.959 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.952+0000 7f48767fc640 1 -- 172.21.15.16:0/791608840 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f488003f860 con 0x55af94008370 2024-09-12T02:17:43.959 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.952+0000 7f48767fc640 1 -- 172.21.15.16:0/791608840 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4880005d50 con 0x55af94008370 2024-09-12T02:17:43.959 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.952+0000 7f4898243740 10 client.20793 fetch_fsmap learned FSMap version 25 2024-09-12T02:17:43.959 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.952+0000 7f4898243740 1 -- 172.21.15.16:0/791608840 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55af940130f0 con 0x55af94008370 2024-09-12T02:17:43.959 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.952+0000 7f48767fc640 1 -- 172.21.15.16:0/791608840 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4880004470 con 0x55af94008370 2024-09-12T02:17:43.959 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.952+0000 7f48767fc640 1 -- 172.21.15.16:0/791608840 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f488006e030 con 0x55af94008370 2024-09-12T02:17:43.959 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.952+0000 7f4898243740 10 client.20793 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:17:43.960 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.952+0000 7f4898243740 10 client.20793 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:17:43.960 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.952+0000 7f4898243740 20 client.20793 populate_metadata read hostname 'smithi016' 2024-09-12T02:17:43.960 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.952+0000 7f4898243740 1 --2- 172.21.15.16:0/791608840 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55af940832b0 0x55af940856d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:43.960 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.952+0000 7f4898243740 4 client.20793 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:17:43.960 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.952+0000 7f4898243740 1 -- 172.21.15.16:0/791608840 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55af93f6dd70 con 0x55af940832b0 2024-09-12T02:17:43.960 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.953+0000 7f488effd640 1 --2- 172.21.15.16:0/791608840 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55af940832b0 0x55af940856d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:43.960 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.953+0000 7f488effd640 1 --2- 172.21.15.16:0/791608840 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55af940832b0 0x55af940856d0 crc :-1 s=READY pgs=470 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=105db2f92dab796a server_cookie=6127958dd6fa4f4d in_seq=0 out_seq=0 2024-09-12T02:17:43.960 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.953+0000 7f48767fc640 10 client.20793 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:17:43.969 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.968+0000 7f48767fc640 1 -- 172.21.15.16:0/791608840 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55af93f6dd70 con 0x55af940832b0 2024-09-12T02:17:43.969 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.968+0000 7f48767fc640 10 client.20793 handle_command_reply: tid=0 2024-09-12T02:17:43.969 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.968+0000 7f4898243740 1 client.20793 shutdown 2024-09-12T02:17:43.969 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.969+0000 7f4898243740 1 -- 172.21.15.16:0/791608840 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55af94008370 msgr2=0x55af94012ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:43.969 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.969+0000 7f4898243740 1 --2- 172.21.15.16:0/791608840 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55af94008370 0x55af94012ae0 secure :-1 s=READY pgs=2112 cs=0 l=1 rev1=1 crypto rx=0x7f488003ea00 tx=0x7f4880005c90 comp rx=0 tx=0).stop 2024-09-12T02:17:43.970 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.969+0000 7f4898243740 1 -- 172.21.15.16:0/791608840 shutdown_connections 2024-09-12T02:17:43.970 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.969+0000 7f4898243740 1 --2- 172.21.15.16:0/791608840 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55af940832b0 0x55af940856d0 crc :-1 s=READY pgs=470 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:43.970 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.969+0000 7f4898243740 1 --2- 172.21.15.16:0/791608840 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55af94008370 0x55af94012ae0 unknown :-1 s=CLOSED pgs=2112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:43.970 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.969+0000 7f4898243740 1 --2- 172.21.15.16:0/791608840 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55af940032e0 0x55af94011ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:43.970 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.969+0000 7f48767fc640 0 client.20793 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:17:43.970 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.969+0000 7f4898243740 1 --2- 172.21.15.16:0/791608840 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55af940029a0 0x55af94011590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:43.970 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.969+0000 7f4898243740 1 -- 172.21.15.16:0/791608840 >> 172.21.15.16:0/791608840 conn(0x55af93ffe010 msgr2=0x55af93ffc9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:43.970 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.969+0000 7f4898243740 1 -- 172.21.15.16:0/791608840 shutdown_connections 2024-09-12T02:17:43.970 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.969+0000 7f4898243740 1 -- 172.21.15.16:0/791608840 wait complete. 2024-09-12T02:17:43.970 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.969+0000 7f4898243740 20 client.20793 trim_cache size 0 max 16384 2024-09-12T02:17:43.971 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-09-12T02:17:43.971 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-09-12T02:17:43.971 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-09-12T02:17:43.971 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-09-12T02:17:43.972 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.969+0000 7f48954a5640 1 -- 172.21.15.16:0/602067268 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f487c0a2cb0 msgr2=0x7f487c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:43.972 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.969+0000 7f48954a5640 1 --2- 172.21.15.16:0/602067268 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f487c0a2cb0 0x7f487c0a5170 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f4888097fb0 tx=0x7f488801eb40 comp rx=0 tx=0).stop 2024-09-12T02:17:43.972 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.969+0000 7f48954a5640 1 -- 172.21.15.16:0/602067268 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4890103710 msgr2=0x7f489019f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:43.972 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.969+0000 7f48954a5640 1 --2- 172.21.15.16:0/602067268 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4890103710 0x7f489019f380 secure :-1 s=READY pgs=2108 cs=0 l=1 rev1=1 crypto rx=0x7f488800eaa0 tx=0x7f488800ef60 comp rx=0 tx=0).stop 2024-09-12T02:17:43.972 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.969+0000 7f48954a5640 1 -- 172.21.15.16:0/602067268 shutdown_connections 2024-09-12T02:17:43.972 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.969+0000 7f48954a5640 1 --2- 172.21.15.16:0/602067268 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f487c0a2cb0 0x7f487c0a5170 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:43.972 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.969+0000 7f48954a5640 1 --2- 172.21.15.16:0/602067268 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4890103710 0x7f489019f380 unknown :-1 s=CLOSED pgs=2108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:43.972 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.969+0000 7f48954a5640 1 --2- 172.21.15.16:0/602067268 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4890102d50 0x7f4890198310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:43.972 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.969+0000 7f48954a5640 1 --2- 172.21.15.16:0/602067268 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4890101b60 0x7f4890197dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:43.973 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.969+0000 7f48954a5640 1 -- 172.21.15.16:0/602067268 >> 172.21.15.16:0/602067268 conn(0x7f48900fd2e0 msgr2=0x7f4890106070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:43.973 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.970+0000 7f48954a5640 1 -- 172.21.15.16:0/602067268 shutdown_connections 2024-09-12T02:17:43.973 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-12T02:17:43.970+0000 7f48954a5640 1 -- 172.21.15.16:0/602067268 wait complete. 2024-09-12T02:17:44.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:17:43 smithi184 ceph-mon[35593]: pgmap v809: 97 pgs: 97 active+clean; 4.2 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-09-12T02:17:44.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:17:43 smithi033 ceph-mon[35332]: pgmap v809: 97 pgs: 97 active+clean; 4.2 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-09-12T02:17:44.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.655+0000 7f9da1792640 1 -- 172.21.15.16:0/2892704824 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d9c106f10 msgr2=0x7f9d9c10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:44.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.655+0000 7f9da1792640 1 --2- 172.21.15.16:0/2892704824 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d9c106f10 0x7f9d9c10ea90 secure :-1 s=READY pgs=2113 cs=0 l=1 rev1=1 crypto rx=0x7f9d94009ca0 tx=0x7f9d9402a8c0 comp rx=0 tx=0).stop 2024-09-12T02:17:44.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.655+0000 7f9da1792640 1 -- 172.21.15.16:0/2892704824 shutdown_connections 2024-09-12T02:17:44.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.655+0000 7f9da1792640 1 --2- 172.21.15.16:0/2892704824 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d9c106f10 0x7f9d9c10ea90 unknown :-1 s=CLOSED pgs=2113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:44.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.655+0000 7f9da1792640 1 --2- 172.21.15.16:0/2892704824 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9d9c102790 0x7f9d9c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:44.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.655+0000 7f9da1792640 1 --2- 172.21.15.16:0/2892704824 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9d9c101de0 0x7f9d9c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:44.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.655+0000 7f9da1792640 1 -- 172.21.15.16:0/2892704824 >> 172.21.15.16:0/2892704824 conn(0x7f9d9c0fd8c0 msgr2=0x7f9d9c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:44.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.655+0000 7f9da1792640 1 -- 172.21.15.16:0/2892704824 shutdown_connections 2024-09-12T02:17:44.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.655+0000 7f9da1792640 1 -- 172.21.15.16:0/2892704824 wait complete. 2024-09-12T02:17:44.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.656+0000 7f9da1792640 1 Processor -- start 2024-09-12T02:17:44.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.656+0000 7f9da1792640 1 -- start start 2024-09-12T02:17:44.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.657+0000 7f9da1792640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9d9c101de0 0x7f9d9c1983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:44.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.657+0000 7f9da1792640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d9c102790 0x7f9d9c198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:44.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.657+0000 7f9da1792640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9d9c106f10 0x7f9d9c19ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:44.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.657+0000 7f9da1792640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9d9c111f70 con 0x7f9d9c102790 2024-09-12T02:17:44.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.657+0000 7f9da1792640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9d9c111df0 con 0x7f9d9c101de0 2024-09-12T02:17:44.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.657+0000 7f9da1792640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f9d9c1120f0 con 0x7f9d9c106f10 2024-09-12T02:17:44.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.657+0000 7f9d93fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d9c102790 0x7f9d9c198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:44.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.657+0000 7f9d93fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d9c102790 0x7f9d9c198930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47248/0 (socket says 172.21.15.16:47248) 2024-09-12T02:17:44.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.657+0000 7f9d93fff640 1 -- 172.21.15.16:0/567815358 learned_addr learned my addr 172.21.15.16:0/567815358 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:17:44.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.657+0000 7f9d9b7fe640 1 --2- 172.21.15.16:0/567815358 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9d9c106f10 0x7f9d9c19ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:44.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.657+0000 7f9d9affd640 1 --2- 172.21.15.16:0/567815358 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9d9c101de0 0x7f9d9c1983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:44.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.657+0000 7f9d93fff640 1 -- 172.21.15.16:0/567815358 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9d9c106f10 msgr2=0x7f9d9c19ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:44.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.657+0000 7f9d93fff640 1 --2- 172.21.15.16:0/567815358 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9d9c106f10 0x7f9d9c19ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:44.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.657+0000 7f9d93fff640 1 -- 172.21.15.16:0/567815358 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9d9c101de0 msgr2=0x7f9d9c1983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:44.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.657+0000 7f9d93fff640 1 --2- 172.21.15.16:0/567815358 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9d9c101de0 0x7f9d9c1983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:44.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.657+0000 7f9d93fff640 1 -- 172.21.15.16:0/567815358 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d9c19d3a0 con 0x7f9d9c102790 2024-09-12T02:17:44.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.657+0000 7f9d93fff640 1 --2- 172.21.15.16:0/567815358 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d9c102790 0x7f9d9c198930 secure :-1 s=READY pgs=2114 cs=0 l=1 rev1=1 crypto rx=0x7f9d8400c890 tx=0x7f9d8400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:44.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.658+0000 7f9d98ff9640 1 -- 172.21.15.16:0/567815358 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d84004220 con 0x7f9d9c102790 2024-09-12T02:17:44.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.658+0000 7f9d98ff9640 1 -- 172.21.15.16:0/567815358 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d840043c0 con 0x7f9d9c102790 2024-09-12T02:17:44.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.658+0000 7f9da1792640 1 -- 172.21.15.16:0/567815358 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d9c19d690 con 0x7f9d9c102790 2024-09-12T02:17:44.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.658+0000 7f9d98ff9640 1 -- 172.21.15.16:0/567815358 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d840049e0 con 0x7f9d9c102790 2024-09-12T02:17:44.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.658+0000 7f9da1792640 1 -- 172.21.15.16:0/567815358 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d9c1a4f70 con 0x7f9d9c102790 2024-09-12T02:17:44.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.660+0000 7f9d98ff9640 1 -- 172.21.15.16:0/567815358 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f9d840079b0 con 0x7f9d9c102790 2024-09-12T02:17:44.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.660+0000 7f9da3b6b740 1 Processor -- start 2024-09-12T02:17:44.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.660+0000 7f9da3b6b740 1 -- start start 2024-09-12T02:17:44.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.661+0000 7f9da3b6b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560bdbb30a00 0x560bdbb30de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:44.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.661+0000 7f9da3b6b740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560bdbb31320 0x560bdbb3b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:44.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.661+0000 7f9da3b6b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560bdbb3bbe0 0x560bdbb3dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:44.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.661+0000 7f9da3b6b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x560bdba95200 con 0x560bdbb3bbe0 2024-09-12T02:17:44.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.661+0000 7f9da3b6b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x560bdba3bdc0 con 0x560bdbb30a00 2024-09-12T02:17:44.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.661+0000 7f9da3b6b740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x560bdba911c0 con 0x560bdbb31320 2024-09-12T02:17:44.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.661+0000 7f9d93fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560bdbb3bbe0 0x560bdbb3dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:44.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.661+0000 7f9d98ff9640 1 --2- 172.21.15.16:0/567815358 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f9d880a2b80 0x7f9d880a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:44.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.661+0000 7f9d93fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560bdbb3bbe0 0x560bdbb3dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47252/0 (socket says 172.21.15.16:47252) 2024-09-12T02:17:44.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.661+0000 7f9d93fff640 1 -- 172.21.15.16:0/3736051569 learned_addr learned my addr 172.21.15.16:0/3736051569 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:17:44.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.661+0000 7f9d9affd640 1 --2- 172.21.15.16:0/3736051569 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560bdbb31320 0x560bdbb3b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:44.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.661+0000 7f9d9b7fe640 1 --2- 172.21.15.16:0/3736051569 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560bdbb30a00 0x560bdbb30de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:44.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.661+0000 7f9d9b7fe640 1 --2- 172.21.15.16:0/567815358 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f9d880a2b80 0x7f9d880a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:44.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.662+0000 7f9d98ff9640 1 -- 172.21.15.16:0/567815358 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f9d840cef90 con 0x7f9d9c102790 2024-09-12T02:17:44.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.662+0000 7f9d9affd640 1 -- 172.21.15.16:0/3736051569 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560bdbb30a00 msgr2=0x560bdbb30de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:44.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.662+0000 7f9d9affd640 1 --2- 172.21.15.16:0/3736051569 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560bdbb30a00 0x560bdbb30de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:44.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.662+0000 7f9d9affd640 1 -- 172.21.15.16:0/3736051569 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560bdbb3bbe0 msgr2=0x560bdbb3dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:44.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.662+0000 7f9d9affd640 1 --2- 172.21.15.16:0/3736051569 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560bdbb3bbe0 0x560bdbb3dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:44.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.662+0000 7f9d9affd640 1 -- 172.21.15.16:0/3736051569 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560bdb806620 con 0x560bdbb31320 2024-09-12T02:17:44.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.662+0000 7f9d9affd640 1 --2- 172.21.15.16:0/3736051569 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560bdbb31320 0x560bdbb3b6a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c004630 tx=0x7f9d8c00bb20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fafb378f1a11db8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:44.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.662+0000 7f9d9b7fe640 1 --2- 172.21.15.16:0/567815358 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f9d880a2b80 0x7f9d880a5040 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f9d94004af0 tx=0x7f9d9402f310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:44.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.662+0000 7f9d917fa640 1 -- 172.21.15.16:0/3736051569 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d8c003bd0 con 0x560bdbb31320 2024-09-12T02:17:44.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.662+0000 7f9d917fa640 1 -- 172.21.15.16:0/3736051569 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d8c015070 con 0x560bdbb31320 2024-09-12T02:17:44.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.662+0000 7f9d917fa640 1 -- 172.21.15.16:0/3736051569 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d8c004cd0 con 0x560bdbb31320 2024-09-12T02:17:44.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.663+0000 7f9da3b6b740 1 -- 172.21.15.16:0/3736051569 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560bdbb31320 msgr2=0x560bdbb3b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:44.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.663+0000 7f9da3b6b740 1 --2- 172.21.15.16:0/3736051569 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560bdbb31320 0x560bdbb3b6a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c004630 tx=0x7f9d8c00bb20 comp rx=0 tx=0).stop 2024-09-12T02:17:44.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.663+0000 7f9da3b6b740 1 -- 172.21.15.16:0/3736051569 shutdown_connections 2024-09-12T02:17:44.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.663+0000 7f9da3b6b740 1 --2- 172.21.15.16:0/3736051569 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560bdbb3bbe0 0x560bdbb3dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:44.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.663+0000 7f9da3b6b740 1 --2- 172.21.15.16:0/3736051569 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560bdbb31320 0x560bdbb3b6a0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:44.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.663+0000 7f9da3b6b740 1 --2- 172.21.15.16:0/3736051569 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560bdbb30a00 0x560bdbb30de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:44.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.663+0000 7f9da3b6b740 1 -- 172.21.15.16:0/3736051569 >> 172.21.15.16:0/3736051569 conn(0x560bdbb2cf90 msgr2=0x560bdbb2d370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:44.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.663+0000 7f9da3b6b740 1 -- 172.21.15.16:0/3736051569 shutdown_connections 2024-09-12T02:17:44.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.663+0000 7f9da3b6b740 1 -- 172.21.15.16:0/3736051569 wait complete. 2024-09-12T02:17:44.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.664+0000 7f9da3b6b740 1 -- start start 2024-09-12T02:17:44.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.664+0000 7f9da3b6b740 1 Processor -- start 2024-09-12T02:17:44.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.664+0000 7f9da3b6b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560bdbb30a00 0x560bdbb3b490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:44.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.664+0000 7f9da3b6b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560bdbb31320 0x560bdbb38740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:44.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.664+0000 7f9da3b6b740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560bdbb3bbe0 0x560bdbb38c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:44.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.664+0000 7f9da3b6b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x560bdbb3e7b0 con 0x560bdbb31320 2024-09-12T02:17:44.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.664+0000 7f9da3b6b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x560bdba2fc50 con 0x560bdbb30a00 2024-09-12T02:17:44.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.664+0000 7f9da3b6b740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x560bdbb3e930 con 0x560bdbb3bbe0 2024-09-12T02:17:44.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.664+0000 7f9da3b6b740 10 client.0 _ll_register_callbacks cb 0x560bdba41ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:17:44.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.664+0000 7f9d93fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560bdbb31320 0x560bdbb38740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:44.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.664+0000 7f9d93fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560bdbb31320 0x560bdbb38740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47266/0 (socket says 172.21.15.16:47266) 2024-09-12T02:17:44.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.664+0000 7f9d93fff640 1 -- 172.21.15.16:0/2046616140 learned_addr learned my addr 172.21.15.16:0/2046616140 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:17:44.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.664+0000 7f9d9affd640 1 --2- 172.21.15.16:0/2046616140 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560bdbb30a00 0x560bdbb3b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:44.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.664+0000 7f9d9b7fe640 1 --2- 172.21.15.16:0/2046616140 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560bdbb3bbe0 0x560bdbb38c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:44.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.664+0000 7f9d93fff640 1 -- 172.21.15.16:0/2046616140 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560bdbb3bbe0 msgr2=0x560bdbb38c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:44.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.664+0000 7f9d93fff640 1 --2- 172.21.15.16:0/2046616140 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560bdbb3bbe0 0x560bdbb38c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:44.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.664+0000 7f9d93fff640 1 -- 172.21.15.16:0/2046616140 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560bdbb30a00 msgr2=0x560bdbb3b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:44.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.664+0000 7f9d93fff640 1 --2- 172.21.15.16:0/2046616140 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560bdbb30a00 0x560bdbb3b490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:44.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.664+0000 7f9d93fff640 1 -- 172.21.15.16:0/2046616140 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560bdbb39480 con 0x560bdbb31320 2024-09-12T02:17:44.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.664+0000 7f9d9affd640 1 --2- 172.21.15.16:0/2046616140 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560bdbb30a00 0x560bdbb3b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:17:44.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.665+0000 7f9d93fff640 1 --2- 172.21.15.16:0/2046616140 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560bdbb31320 0x560bdbb38740 secure :-1 s=READY pgs=2115 cs=0 l=1 rev1=1 crypto rx=0x7f9d840a3f30 tx=0x7f9d84020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85ba2af6ebcf251d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:44.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.665+0000 7f9d827fc640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:17:44.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.665+0000 7f9d827fc640 1 -- 172.21.15.16:0/2046616140 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d8401d040 con 0x560bdbb31320 2024-09-12T02:17:44.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.665+0000 7f9d827fc640 1 -- 172.21.15.16:0/2046616140 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f9d840860c0 con 0x560bdbb31320 2024-09-12T02:17:44.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.665+0000 7f9da3b6b740 1 -- 172.21.15.16:0/2046616140 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d8c015070 con 0x560bdbb31320 2024-09-12T02:17:44.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.665+0000 7f9d827fc640 1 -- 172.21.15.16:0/2046616140 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9d840862c0 con 0x560bdbb31320 2024-09-12T02:17:44.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.665+0000 7f9da3b6b740 10 client.20805 fetch_fsmap learned FSMap version 25 2024-09-12T02:17:44.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.665+0000 7f9da3b6b740 1 -- 172.21.15.16:0/2046616140 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x560bdbb4a2a0 con 0x560bdbb31320 2024-09-12T02:17:44.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.665+0000 7f9d827fc640 1 -- 172.21.15.16:0/2046616140 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d8c015070 con 0x560bdbb31320 2024-09-12T02:17:44.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.665+0000 7f9d827fc640 1 -- 172.21.15.16:0/2046616140 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f9d8409acf0 con 0x560bdbb31320 2024-09-12T02:17:44.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.665+0000 7f9da3b6b740 10 client.20805 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:17:44.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.665+0000 7f9da3b6b740 10 client.20805 resolve_mds: validated gid 24482 aka daemon mds.b 2024-09-12T02:17:44.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.665+0000 7f9da3b6b740 20 client.20805 populate_metadata read hostname 'smithi016' 2024-09-12T02:17:44.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.665+0000 7f9da3b6b740 1 --2- 172.21.15.16:0/2046616140 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x560bdbb4b700 0x560bdbb4db50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:44.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.665+0000 7f9da3b6b740 4 client.20805 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:17:44.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.665+0000 7f9da3b6b740 1 -- 172.21.15.16:0/2046616140 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560bdbaa2770 con 0x560bdbb4b700 2024-09-12T02:17:44.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.666+0000 7f9d9affd640 1 --2- 172.21.15.16:0/2046616140 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x560bdbb4b700 0x560bdbb4db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:44.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.666+0000 7f9d9affd640 1 --2- 172.21.15.16:0/2046616140 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x560bdbb4b700 0x560bdbb4db50 crc :-1 s=READY pgs=471 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3921ed49e03c334b server_cookie=2663a1051432c51f in_seq=0 out_seq=0 2024-09-12T02:17:44.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.666+0000 7f9d827fc640 10 client.20805 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:17:44.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.667+0000 7f9d827fc640 1 -- 172.21.15.16:0/2046616140 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560bdbaa2770 con 0x560bdbb4b700 2024-09-12T02:17:44.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.667+0000 7f9d827fc640 10 client.20805 handle_command_reply: tid=0 2024-09-12T02:17:44.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.667+0000 7f9da3b6b740 1 client.20805 shutdown 2024-09-12T02:17:44.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.667+0000 7f9da3b6b740 1 -- 172.21.15.16:0/2046616140 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560bdbb31320 msgr2=0x560bdbb38740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:44.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.667+0000 7f9da3b6b740 1 --2- 172.21.15.16:0/2046616140 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560bdbb31320 0x560bdbb38740 secure :-1 s=READY pgs=2115 cs=0 l=1 rev1=1 crypto rx=0x7f9d840a3f30 tx=0x7f9d84020000 comp rx=0 tx=0).stop 2024-09-12T02:17:44.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.667+0000 7f9da3b6b740 1 -- 172.21.15.16:0/2046616140 shutdown_connections 2024-09-12T02:17:44.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.667+0000 7f9da3b6b740 1 --2- 172.21.15.16:0/2046616140 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x560bdbb4b700 0x560bdbb4db50 crc :-1 s=READY pgs=471 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:44.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.667+0000 7f9da3b6b740 1 --2- 172.21.15.16:0/2046616140 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560bdbb3bbe0 0x560bdbb38c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:44.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.667+0000 7f9da3b6b740 1 --2- 172.21.15.16:0/2046616140 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560bdbb31320 0x560bdbb38740 unknown :-1 s=CLOSED pgs=2115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:44.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.667+0000 7f9da3b6b740 1 --2- 172.21.15.16:0/2046616140 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560bdbb30a00 0x560bdbb3b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:44.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.667+0000 7f9da3b6b740 1 -- 172.21.15.16:0/2046616140 >> 172.21.15.16:0/2046616140 conn(0x560bdbb2cf90 msgr2=0x560bdbb2c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:44.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.667+0000 7f9d827fc640 0 client.20805 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:17:44.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.667+0000 7f9da3b6b740 1 -- 172.21.15.16:0/2046616140 shutdown_connections 2024-09-12T02:17:44.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.667+0000 7f9da3b6b740 1 -- 172.21.15.16:0/2046616140 wait complete. 2024-09-12T02:17:44.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.667+0000 7f9da3b6b740 20 client.20805 trim_cache size 0 max 16384 2024-09-12T02:17:44.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.684+0000 7f9da3b6b740 1 Processor -- start 2024-09-12T02:17:44.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.684+0000 7f9da3b6b740 1 -- start start 2024-09-12T02:17:44.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.685+0000 7f9da3b6b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560bdbb4dcb0 0x560bdbb2c020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:44.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.685+0000 7f9da3b6b740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560bdbb2cf90 0x560bdbb37010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:44.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.685+0000 7f9da3b6b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560bdbb30a00 0x560bdbb47a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:44.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.685+0000 7f9da3b6b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9d9c1afd40 con 0x560bdbb4dcb0 2024-09-12T02:17:44.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.685+0000 7f9da3b6b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9d9c1ad610 con 0x560bdbb30a00 2024-09-12T02:17:44.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.685+0000 7f9da3b6b740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f9d9c1ab4a0 con 0x560bdbb2cf90 2024-09-12T02:17:44.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.685+0000 7f9d9affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560bdbb4dcb0 0x560bdbb2c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:44.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.685+0000 7f9d9affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560bdbb4dcb0 0x560bdbb2c020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47276/0 (socket says 172.21.15.16:47276) 2024-09-12T02:17:44.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.685+0000 7f9d9affd640 1 -- 172.21.15.16:0/3355650843 learned_addr learned my addr 172.21.15.16:0/3355650843 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:17:44.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.685+0000 7f9d93fff640 1 --2- 172.21.15.16:0/3355650843 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560bdbb2cf90 0x560bdbb37010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:44.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.685+0000 7f9d9b7fe640 1 --2- 172.21.15.16:0/3355650843 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560bdbb30a00 0x560bdbb47a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:44.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.685+0000 7f9d9affd640 1 -- 172.21.15.16:0/3355650843 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560bdbb2cf90 msgr2=0x560bdbb37010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:44.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.685+0000 7f9d9affd640 1 --2- 172.21.15.16:0/3355650843 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560bdbb2cf90 0x560bdbb37010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:44.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.685+0000 7f9d9affd640 1 -- 172.21.15.16:0/3355650843 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560bdbb30a00 msgr2=0x560bdbb47a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:44.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.685+0000 7f9d9affd640 1 --2- 172.21.15.16:0/3355650843 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560bdbb30a00 0x560bdbb47a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:44.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.686+0000 7f9d9affd640 1 -- 172.21.15.16:0/3355650843 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560bdbb2cdc0 con 0x560bdbb4dcb0 2024-09-12T02:17:44.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.686+0000 7f9d9affd640 1 --2- 172.21.15.16:0/3355650843 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560bdbb4dcb0 0x560bdbb2c020 secure :-1 s=READY pgs=2116 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c00b620 tx=0x7f9d8c00b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fde119f91360052d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:44.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.686+0000 7f9d90ff9640 1 -- 172.21.15.16:0/3355650843 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d8c003bd0 con 0x560bdbb4dcb0 2024-09-12T02:17:44.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.686+0000 7f9d90ff9640 1 -- 172.21.15.16:0/3355650843 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d8c015040 con 0x560bdbb4dcb0 2024-09-12T02:17:44.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.686+0000 7f9d90ff9640 1 -- 172.21.15.16:0/3355650843 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d8c0048c0 con 0x560bdbb4dcb0 2024-09-12T02:17:44.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.687+0000 7f9da3b6b740 1 -- 172.21.15.16:0/3355650843 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560bdbb4dcb0 msgr2=0x560bdbb2c020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:44.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.687+0000 7f9da3b6b740 1 --2- 172.21.15.16:0/3355650843 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560bdbb4dcb0 0x560bdbb2c020 secure :-1 s=READY pgs=2116 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c00b620 tx=0x7f9d8c00b390 comp rx=0 tx=0).stop 2024-09-12T02:17:44.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.687+0000 7f9da3b6b740 1 -- 172.21.15.16:0/3355650843 shutdown_connections 2024-09-12T02:17:44.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.687+0000 7f9da3b6b740 1 --2- 172.21.15.16:0/3355650843 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560bdbb30a00 0x560bdbb47a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:44.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.687+0000 7f9da3b6b740 1 --2- 172.21.15.16:0/3355650843 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560bdbb2cf90 0x560bdbb37010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:44.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.687+0000 7f9da3b6b740 1 --2- 172.21.15.16:0/3355650843 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560bdbb4dcb0 0x560bdbb2c020 unknown :-1 s=CLOSED pgs=2116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:44.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.687+0000 7f9da3b6b740 1 -- 172.21.15.16:0/3355650843 >> 172.21.15.16:0/3355650843 conn(0x560bdbb38de0 msgr2=0x560bdbb3a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:44.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.687+0000 7f9da3b6b740 1 -- 172.21.15.16:0/3355650843 shutdown_connections 2024-09-12T02:17:44.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.687+0000 7f9da3b6b740 1 -- 172.21.15.16:0/3355650843 wait complete. 2024-09-12T02:17:44.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.687+0000 7f9da3b6b740 1 -- start start 2024-09-12T02:17:44.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.687+0000 7f9da3b6b740 1 Processor -- start 2024-09-12T02:17:44.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.688+0000 7f9da3b6b740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560bdbb2cf90 0x560bdbb43f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:44.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.688+0000 7f9da3b6b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560bdbb30a00 0x560bdbb42340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:44.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.688+0000 7f9da3b6b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560bdbb4dcb0 0x560bdbb42880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:44.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.688+0000 7f9da3b6b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9d9c1abaf0 con 0x560bdbb4dcb0 2024-09-12T02:17:44.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.688+0000 7f9da3b6b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9d9c1acf40 con 0x560bdbb30a00 2024-09-12T02:17:44.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.688+0000 7f9da3b6b740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f9d9c1ac1a0 con 0x560bdbb2cf90 2024-09-12T02:17:44.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.688+0000 7f9da3b6b740 10 client.0 _ll_register_callbacks cb 0x7f9d8409b720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:17:44.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.688+0000 7f9d9b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560bdbb4dcb0 0x560bdbb42880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:44.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.688+0000 7f9d9b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560bdbb4dcb0 0x560bdbb42880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47292/0 (socket says 172.21.15.16:47292) 2024-09-12T02:17:44.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.688+0000 7f9d9b7fe640 1 -- 172.21.15.16:0/712505008 learned_addr learned my addr 172.21.15.16:0/712505008 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:17:44.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.688+0000 7f9d93fff640 1 --2- 172.21.15.16:0/712505008 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560bdbb30a00 0x560bdbb42340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:44.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.688+0000 7f9d9affd640 1 --2- 172.21.15.16:0/712505008 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560bdbb2cf90 0x560bdbb43f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:44.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.688+0000 7f9d9b7fe640 1 -- 172.21.15.16:0/712505008 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560bdbb2cf90 msgr2=0x560bdbb43f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:44.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.688+0000 7f9d9b7fe640 1 --2- 172.21.15.16:0/712505008 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560bdbb2cf90 0x560bdbb43f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:44.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.688+0000 7f9d9b7fe640 1 -- 172.21.15.16:0/712505008 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560bdbb30a00 msgr2=0x560bdbb42340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:44.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.688+0000 7f9d9b7fe640 1 --2- 172.21.15.16:0/712505008 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560bdbb30a00 0x560bdbb42340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:44.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.688+0000 7f9d9b7fe640 1 -- 172.21.15.16:0/712505008 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560bdbb42dc0 con 0x560bdbb4dcb0 2024-09-12T02:17:44.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.688+0000 7f9d93fff640 1 --2- 172.21.15.16:0/712505008 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560bdbb30a00 0x560bdbb42340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:17:44.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.688+0000 7f9d9b7fe640 1 --2- 172.21.15.16:0/712505008 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560bdbb4dcb0 0x560bdbb42880 secure :-1 s=READY pgs=2117 cs=0 l=1 rev1=1 crypto rx=0x7f9d94035000 tx=0x7f9d9403e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=964f0efde788b525 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:44.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.688+0000 7f9d827fc640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:17:44.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.689+0000 7f9d827fc640 1 -- 172.21.15.16:0/712505008 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d94004440 con 0x560bdbb4dcb0 2024-09-12T02:17:44.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.689+0000 7f9d827fc640 1 -- 172.21.15.16:0/712505008 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f9d940045e0 con 0x560bdbb4dcb0 2024-09-12T02:17:44.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.689+0000 7f9da3b6b740 1 -- 172.21.15.16:0/712505008 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d8c015040 con 0x560bdbb4dcb0 2024-09-12T02:17:44.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.689+0000 7f9d827fc640 1 -- 172.21.15.16:0/712505008 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9d940047e0 con 0x560bdbb4dcb0 2024-09-12T02:17:44.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.689+0000 7f9da3b6b740 10 client.20811 fetch_fsmap learned FSMap version 25 2024-09-12T02:17:44.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.689+0000 7f9da3b6b740 1 -- 172.21.15.16:0/712505008 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x560bdbbb7160 con 0x560bdbb4dcb0 2024-09-12T02:17:44.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.689+0000 7f9d827fc640 1 -- 172.21.15.16:0/712505008 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d8c015040 con 0x560bdbb4dcb0 2024-09-12T02:17:44.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.689+0000 7f9d827fc640 1 -- 172.21.15.16:0/712505008 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f9d94030800 con 0x560bdbb4dcb0 2024-09-12T02:17:44.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.689+0000 7f9da3b6b740 10 client.20811 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:17:44.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.689+0000 7f9da3b6b740 10 client.20811 resolve_mds: validated gid 24482 aka daemon mds.b 2024-09-12T02:17:44.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.689+0000 7f9da3b6b740 20 client.20811 populate_metadata read hostname 'smithi016' 2024-09-12T02:17:44.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.689+0000 7f9da3b6b740 1 --2- 172.21.15.16:0/712505008 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x560bdbb31320 0x560bdbbb9950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:44.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.689+0000 7f9da3b6b740 4 client.20811 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "quiesce db", "cancel": true, "all": true}] 2024-09-12T02:17:44.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.689+0000 7f9da3b6b740 1 -- 172.21.15.16:0/712505008 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "quiesce db", "cancel": true, "all": true}) -- 0x560bdba9cc80 con 0x560bdbb31320 2024-09-12T02:17:44.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.690+0000 7f9d9affd640 1 --2- 172.21.15.16:0/712505008 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x560bdbb31320 0x560bdbbb9950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:44.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.690+0000 7f9d9affd640 1 --2- 172.21.15.16:0/712505008 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x560bdbb31320 0x560bdbbb9950 crc :-1 s=READY pgs=472 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f904b61b64abb6c server_cookie=324d8756b6ab1966 in_seq=0 out_seq=0 2024-09-12T02:17:44.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.690+0000 7f9d827fc640 10 client.20811 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:17:44.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.690+0000 7f9d827fc640 1 -- 172.21.15.16:0/712505008 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+80 (crc 0 0 0) 0x560bdba9cc80 con 0x560bdbb31320 2024-09-12T02:17:44.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.690+0000 7f9d827fc640 10 client.20811 handle_command_reply: tid=0 2024-09-12T02:17:44.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.690+0000 7f9da3b6b740 1 client.20811 shutdown 2024-09-12T02:17:44.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.690+0000 7f9da3b6b740 1 -- 172.21.15.16:0/712505008 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560bdbb4dcb0 msgr2=0x560bdbb42880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:44.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.690+0000 7f9da3b6b740 1 --2- 172.21.15.16:0/712505008 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560bdbb4dcb0 0x560bdbb42880 secure :-1 s=READY pgs=2117 cs=0 l=1 rev1=1 crypto rx=0x7f9d94035000 tx=0x7f9d9403e070 comp rx=0 tx=0).stop 2024-09-12T02:17:44.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.691+0000 7f9da3b6b740 1 -- 172.21.15.16:0/712505008 shutdown_connections 2024-09-12T02:17:44.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.691+0000 7f9da3b6b740 1 --2- 172.21.15.16:0/712505008 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x560bdbb31320 0x560bdbbb9950 crc :-1 s=READY pgs=472 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:44.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.691+0000 7f9da3b6b740 1 --2- 172.21.15.16:0/712505008 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560bdbb4dcb0 0x560bdbb42880 unknown :-1 s=CLOSED pgs=2117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:44.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.691+0000 7f9da3b6b740 1 --2- 172.21.15.16:0/712505008 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560bdbb30a00 0x560bdbb42340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:44.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.691+0000 7f9da3b6b740 1 --2- 172.21.15.16:0/712505008 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560bdbb2cf90 0x560bdbb43f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:44.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.691+0000 7f9da3b6b740 1 -- 172.21.15.16:0/712505008 >> 172.21.15.16:0/712505008 conn(0x560bdbb38de0 msgr2=0x560bdbb3a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:44.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.691+0000 7f9d827fc640 0 client.20811 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:17:44.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.691+0000 7f9da3b6b740 1 -- 172.21.15.16:0/712505008 shutdown_connections 2024-09-12T02:17:44.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.691+0000 7f9da3b6b740 1 -- 172.21.15.16:0/712505008 wait complete. 2024-09-12T02:17:44.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.691+0000 7f9da3b6b740 20 client.20811 trim_cache size 0 max 16384 2024-09-12T02:17:44.696 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T02:17:44.696 INFO:teuthology.orchestra.run.smithi016.stdout: "epoch": 25, 2024-09-12T02:17:44.696 INFO:teuthology.orchestra.run.smithi016.stdout: "leader": 24482, 2024-09-12T02:17:44.696 INFO:teuthology.orchestra.run.smithi016.stdout: "set_version": 56, 2024-09-12T02:17:44.696 INFO:teuthology.orchestra.run.smithi016.stdout: "sets": {} 2024-09-12T02:17:44.696 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T02:17:44.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.691+0000 7f9da1792640 1 -- 172.21.15.16:0/567815358 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f9d880a2b80 msgr2=0x7f9d880a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:44.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.691+0000 7f9da1792640 1 --2- 172.21.15.16:0/567815358 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f9d880a2b80 0x7f9d880a5040 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f9d94004af0 tx=0x7f9d9402f310 comp rx=0 tx=0).stop 2024-09-12T02:17:44.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.691+0000 7f9da1792640 1 -- 172.21.15.16:0/567815358 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d9c102790 msgr2=0x7f9d9c198930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:44.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.691+0000 7f9da1792640 1 --2- 172.21.15.16:0/567815358 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d9c102790 0x7f9d9c198930 secure :-1 s=READY pgs=2114 cs=0 l=1 rev1=1 crypto rx=0x7f9d8400c890 tx=0x7f9d8400cd50 comp rx=0 tx=0).stop 2024-09-12T02:17:44.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.691+0000 7f9da1792640 1 -- 172.21.15.16:0/567815358 shutdown_connections 2024-09-12T02:17:44.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.691+0000 7f9da1792640 1 --2- 172.21.15.16:0/567815358 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f9d880a2b80 0x7f9d880a5040 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:44.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.691+0000 7f9da1792640 1 --2- 172.21.15.16:0/567815358 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9d9c106f10 0x7f9d9c19ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:44.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.691+0000 7f9da1792640 1 --2- 172.21.15.16:0/567815358 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d9c102790 0x7f9d9c198930 unknown :-1 s=CLOSED pgs=2114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:44.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.691+0000 7f9da1792640 1 --2- 172.21.15.16:0/567815358 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9d9c101de0 0x7f9d9c1983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:44.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.691+0000 7f9da1792640 1 -- 172.21.15.16:0/567815358 >> 172.21.15.16:0/567815358 conn(0x7f9d9c0fd8c0 msgr2=0x7f9d9c0fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:44.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.692+0000 7f9da1792640 1 -- 172.21.15.16:0/567815358 shutdown_connections 2024-09-12T02:17:44.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:44.692+0000 7f9da1792640 1 -- 172.21.15.16:0/567815358 wait complete. 2024-09-12T02:17:44.933 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:17:44 smithi016 ceph-mon[30947]: scrub queued for path: / 2024-09-12T02:17:44.934 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:17:44 smithi016 ceph-mon[30947]: scrub summary: idle+waiting paths [/] 2024-09-12T02:17:44.934 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:17:44 smithi016 ceph-mon[30947]: scrub summary: active paths [/] 2024-09-12T02:17:44.934 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:17:44 smithi016 ceph-mon[30947]: scrub summary: idle+waiting paths [/] 2024-09-12T02:17:45.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:17:44 smithi184 ceph-mon[35593]: scrub queued for path: / 2024-09-12T02:17:45.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:17:44 smithi184 ceph-mon[35593]: scrub summary: idle+waiting paths [/] 2024-09-12T02:17:45.128 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:17:44 smithi184 ceph-mon[35593]: scrub summary: active paths [/] 2024-09-12T02:17:45.128 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:17:44 smithi184 ceph-mon[35593]: scrub summary: idle+waiting paths [/] 2024-09-12T02:17:45.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:17:44 smithi033 ceph-mon[35332]: scrub queued for path: / 2024-09-12T02:17:45.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:17:44 smithi033 ceph-mon[35332]: scrub summary: idle+waiting paths [/] 2024-09-12T02:17:45.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:17:44 smithi033 ceph-mon[35332]: scrub summary: active paths [/] 2024-09-12T02:17:45.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:17:44 smithi033 ceph-mon[35332]: scrub summary: idle+waiting paths [/] 2024-09-12T02:17:45.838 INFO:tasks.quiescer:done joining Quiescers 2024-09-12T02:17:45.838 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-09-12T02:17:45.850 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi016.front.sepia.ceph.com 2024-09-12T02:17:45.889 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-09-12T02:17:45.901 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-09-12T02:17:45.905 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T02:17:45.974 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:17:45 smithi016 ceph-mon[30947]: pgmap v810: 97 pgs: 97 active+clean; 4.2 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-09-12T02:17:46.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:17:45 smithi184 ceph-mon[35593]: pgmap v810: 97 pgs: 97 active+clean; 4.2 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-09-12T02:17:46.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:17:45 smithi033 ceph-mon[35332]: pgmap v810: 97 pgs: 97 active+clean; 4.2 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-09-12T02:17:46.213 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:17:46.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.741+0000 7f5d2c979640 1 -- 172.21.15.16:0/1722899919 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d281082a0 msgr2=0x7f5d2810ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:46.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.741+0000 7f5d2c979640 1 --2- 172.21.15.16:0/1722899919 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d281082a0 0x7f5d2810ec40 secure :-1 s=READY pgs=2118 cs=0 l=1 rev1=1 crypto rx=0x7f5d20009cd0 tx=0x7f5d2002a8e0 comp rx=0 tx=0).stop 2024-09-12T02:17:46.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.741+0000 7f5d2c979640 1 -- 172.21.15.16:0/1722899919 shutdown_connections 2024-09-12T02:17:46.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.741+0000 7f5d2c979640 1 --2- 172.21.15.16:0/1722899919 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d281082a0 0x7f5d2810ec40 unknown :-1 s=CLOSED pgs=2118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:46.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.741+0000 7f5d2c979640 1 --2- 172.21.15.16:0/1722899919 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5d281036e0 0x7f5d28107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:46.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.741+0000 7f5d2c979640 1 --2- 172.21.15.16:0/1722899919 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d28102d30 0x7f5d28103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:46.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.741+0000 7f5d2c979640 1 -- 172.21.15.16:0/1722899919 >> 172.21.15.16:0/1722899919 conn(0x7f5d280fc7d0 msgr2=0x7f5d280febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:46.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.741+0000 7f5d2c979640 1 -- 172.21.15.16:0/1722899919 shutdown_connections 2024-09-12T02:17:46.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.741+0000 7f5d2c979640 1 -- 172.21.15.16:0/1722899919 wait complete. 2024-09-12T02:17:46.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.742+0000 7f5d2c979640 1 Processor -- start 2024-09-12T02:17:46.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.742+0000 7f5d2c979640 1 -- start start 2024-09-12T02:17:46.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.742+0000 7f5d2c979640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d28102d30 0x7f5d281984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:46.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.742+0000 7f5d2c979640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5d281036e0 0x7f5d28198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:46.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.742+0000 7f5d2c979640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d281082a0 0x7f5d2819cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:46.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.742+0000 7f5d2c979640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5d281120a0 con 0x7f5d281082a0 2024-09-12T02:17:46.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.742+0000 7f5d2c979640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5d28111f20 con 0x7f5d28102d30 2024-09-12T02:17:46.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.742+0000 7f5d2c979640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5d28112220 con 0x7f5d281036e0 2024-09-12T02:17:46.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.742+0000 7f5d26d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d281082a0 0x7f5d2819cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:46.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.742+0000 7f5d26d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d281082a0 0x7f5d2819cd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47308/0 (socket says 172.21.15.16:47308) 2024-09-12T02:17:46.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.742+0000 7f5d26d9d640 1 -- 172.21.15.16:0/90028287 learned_addr learned my addr 172.21.15.16:0/90028287 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:17:46.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.742+0000 7f5d25d9b640 1 --2- 172.21.15.16:0/90028287 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5d281036e0 0x7f5d28198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:46.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.742+0000 7f5d2659c640 1 --2- 172.21.15.16:0/90028287 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d28102d30 0x7f5d281984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:46.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.742+0000 7f5d26d9d640 1 -- 172.21.15.16:0/90028287 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5d281036e0 msgr2=0x7f5d28198a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:46.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.742+0000 7f5d26d9d640 1 --2- 172.21.15.16:0/90028287 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5d281036e0 0x7f5d28198a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:46.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.742+0000 7f5d26d9d640 1 -- 172.21.15.16:0/90028287 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d28102d30 msgr2=0x7f5d281984c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:46.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.742+0000 7f5d26d9d640 1 --2- 172.21.15.16:0/90028287 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d28102d30 0x7f5d281984c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:46.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.742+0000 7f5d26d9d640 1 -- 172.21.15.16:0/90028287 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d2819d470 con 0x7f5d281082a0 2024-09-12T02:17:46.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.743+0000 7f5d26d9d640 1 --2- 172.21.15.16:0/90028287 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d281082a0 0x7f5d2819cd90 secure :-1 s=READY pgs=2119 cs=0 l=1 rev1=1 crypto rx=0x7f5d20005cc0 tx=0x7f5d20003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:46.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.743+0000 7f5d177fe640 1 -- 172.21.15.16:0/90028287 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d20004040 con 0x7f5d281082a0 2024-09-12T02:17:46.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.743+0000 7f5d177fe640 1 -- 172.21.15.16:0/90028287 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d200046c0 con 0x7f5d281082a0 2024-09-12T02:17:46.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.743+0000 7f5d2c979640 1 -- 172.21.15.16:0/90028287 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d2819d700 con 0x7f5d281082a0 2024-09-12T02:17:46.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.743+0000 7f5d177fe640 1 -- 172.21.15.16:0/90028287 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d20030e70 con 0x7f5d281082a0 2024-09-12T02:17:46.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.743+0000 7f5d2c979640 1 -- 172.21.15.16:0/90028287 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d281a5040 con 0x7f5d281082a0 2024-09-12T02:17:46.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.745+0000 7f5d177fe640 1 -- 172.21.15.16:0/90028287 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5d200305a0 con 0x7f5d281082a0 2024-09-12T02:17:46.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.745+0000 7f5d2ed52740 1 Processor -- start 2024-09-12T02:17:46.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.745+0000 7f5d2ed52740 1 -- start start 2024-09-12T02:17:46.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.746+0000 7f5d2ed52740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620696c54a0 0x5620696c5880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:46.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.746+0000 7f5d2ed52740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5620696c5dc0 0x5620696d0140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:46.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.746+0000 7f5d2ed52740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620696d0680 0x5620696d2a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:46.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.746+0000 7f5d2ed52740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56206962b2e0 con 0x5620696c54a0 2024-09-12T02:17:46.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.746+0000 7f5d2ed52740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5620695d2ce0 con 0x5620696d0680 2024-09-12T02:17:46.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.746+0000 7f5d2ed52740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5620696272a0 con 0x5620696c5dc0 2024-09-12T02:17:46.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.746+0000 7f5d25d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620696c54a0 0x5620696c5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:46.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.746+0000 7f5d25d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620696c54a0 0x5620696c5880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47322/0 (socket says 172.21.15.16:47322) 2024-09-12T02:17:46.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.746+0000 7f5d25d9b640 1 -- 172.21.15.16:0/660972442 learned_addr learned my addr 172.21.15.16:0/660972442 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:17:46.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.746+0000 7f5d177fe640 1 --2- 172.21.15.16:0/90028287 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f5d100a2c00 0x7f5d100a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:46.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.746+0000 7f5d2659c640 1 --2- 172.21.15.16:0/660972442 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5620696c5dc0 0x5620696d0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:46.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.746+0000 7f5d25d9b640 1 --2- 172.21.15.16:0/660972442 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620696d0680 0x5620696d2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:46.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.746+0000 7f5d26d9d640 1 --2- 172.21.15.16:0/90028287 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f5d100a2c00 0x7f5d100a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:46.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.746+0000 7f5d25d9b640 1 -- 172.21.15.16:0/660972442 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5620696c5dc0 msgr2=0x5620696d0140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:46.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.746+0000 7f5d25d9b640 1 --2- 172.21.15.16:0/660972442 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5620696c5dc0 0x5620696d0140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:46.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.746+0000 7f5d25d9b640 1 -- 172.21.15.16:0/660972442 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620696d0680 msgr2=0x5620696d2a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:46.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.746+0000 7f5d25d9b640 1 --2- 172.21.15.16:0/660972442 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620696d0680 0x5620696d2a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:46.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.746+0000 7f5d177fe640 1 -- 172.21.15.16:0/90028287 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f5d2002e030 con 0x7f5d281082a0 2024-09-12T02:17:46.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.746+0000 7f5d25d9b640 1 -- 172.21.15.16:0/660972442 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5620695b7440 con 0x5620696c54a0 2024-09-12T02:17:46.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.747+0000 7f5d25d9b640 1 --2- 172.21.15.16:0/660972442 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620696c54a0 0x5620696c5880 secure :-1 s=READY pgs=2120 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c00afd0 tx=0x7f5d1c00a740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6157ec65a163f1ef server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:46.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.747+0000 7f5d26d9d640 1 --2- 172.21.15.16:0/90028287 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f5d100a2c00 0x7f5d100a50c0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f5d20006010 tx=0x7f5d20087470 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:46.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.747+0000 7f5d14ff9640 1 -- 172.21.15.16:0/660972442 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d1c0116b0 con 0x5620696c54a0 2024-09-12T02:17:46.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.747+0000 7f5d14ff9640 1 -- 172.21.15.16:0/660972442 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d1c020070 con 0x5620696c54a0 2024-09-12T02:17:46.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.747+0000 7f5d14ff9640 1 -- 172.21.15.16:0/660972442 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d1c01bb40 con 0x5620696c54a0 2024-09-12T02:17:46.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.747+0000 7f5d2ed52740 1 -- 172.21.15.16:0/660972442 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620696c54a0 msgr2=0x5620696c5880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:46.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.747+0000 7f5d2ed52740 1 --2- 172.21.15.16:0/660972442 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620696c54a0 0x5620696c5880 secure :-1 s=READY pgs=2120 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c00afd0 tx=0x7f5d1c00a740 comp rx=0 tx=0).stop 2024-09-12T02:17:46.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.748+0000 7f5d2ed52740 1 -- 172.21.15.16:0/660972442 shutdown_connections 2024-09-12T02:17:46.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.748+0000 7f5d2ed52740 1 --2- 172.21.15.16:0/660972442 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620696d0680 0x5620696d2a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:46.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.748+0000 7f5d2ed52740 1 --2- 172.21.15.16:0/660972442 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5620696c5dc0 0x5620696d0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:46.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.748+0000 7f5d2ed52740 1 --2- 172.21.15.16:0/660972442 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620696c54a0 0x5620696c5880 unknown :-1 s=CLOSED pgs=2120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:46.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.748+0000 7f5d2ed52740 1 -- 172.21.15.16:0/660972442 >> 172.21.15.16:0/660972442 conn(0x5620696c1a30 msgr2=0x5620696c1e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:46.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.748+0000 7f5d2ed52740 1 -- 172.21.15.16:0/660972442 shutdown_connections 2024-09-12T02:17:46.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.748+0000 7f5d2ed52740 1 -- 172.21.15.16:0/660972442 wait complete. 2024-09-12T02:17:46.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.748+0000 7f5d2ed52740 1 -- start start 2024-09-12T02:17:46.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.748+0000 7f5d2ed52740 1 Processor -- start 2024-09-12T02:17:46.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.748+0000 7f5d2ed52740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620696c54a0 0x5620696d0040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:46.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.748+0000 7f5d2ed52740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5620696c5dc0 0x5620696cd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:46.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.748+0000 7f5d2ed52740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620696d0680 0x5620696cd8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:46.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.748+0000 7f5d2ed52740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5620696d3250 con 0x5620696c54a0 2024-09-12T02:17:46.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.748+0000 7f5d2ed52740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5620695e6050 con 0x5620696d0680 2024-09-12T02:17:46.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.748+0000 7f5d2ed52740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5620696d33d0 con 0x5620696c5dc0 2024-09-12T02:17:46.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.748+0000 7f5d2ed52740 10 client.0 _ll_register_callbacks cb 0x562069626160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:17:46.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.748+0000 7f5d25d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620696c54a0 0x5620696d0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:46.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.748+0000 7f5d25d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620696c54a0 0x5620696d0040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47328/0 (socket says 172.21.15.16:47328) 2024-09-12T02:17:46.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.748+0000 7f5d25d9b640 1 -- 172.21.15.16:0/3722258808 learned_addr learned my addr 172.21.15.16:0/3722258808 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:17:46.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.749+0000 7f5d2659c640 1 --2- 172.21.15.16:0/3722258808 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5620696c5dc0 0x5620696cd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:46.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.749+0000 7f5d25d9b640 1 -- 172.21.15.16:0/3722258808 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5620696c5dc0 msgr2=0x5620696cd2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:46.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.749+0000 7f5d25d9b640 1 --2- 172.21.15.16:0/3722258808 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5620696c5dc0 0x5620696cd2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:46.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.749+0000 7f5d25d9b640 1 -- 172.21.15.16:0/3722258808 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620696d0680 msgr2=0x5620696cd8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:46.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.749+0000 7f5d25d9b640 1 --2- 172.21.15.16:0/3722258808 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620696d0680 0x5620696cd8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:46.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.749+0000 7f5d25d9b640 1 -- 172.21.15.16:0/3722258808 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56206939c3c0 con 0x5620696c54a0 2024-09-12T02:17:46.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.749+0000 7f5d25d9b640 1 --2- 172.21.15.16:0/3722258808 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620696c54a0 0x5620696d0040 secure :-1 s=READY pgs=2121 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c00a310 tx=0x7f5d1c01cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b289af779099a35e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:46.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.749+0000 7f5d0dffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:17:46.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.749+0000 7f5d0dffb640 1 -- 172.21.15.16:0/3722258808 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d1c02a070 con 0x5620696c54a0 2024-09-12T02:17:46.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.749+0000 7f5d0dffb640 1 -- 172.21.15.16:0/3722258808 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f5d1c005240 con 0x5620696c54a0 2024-09-12T02:17:46.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.749+0000 7f5d2ed52740 1 -- 172.21.15.16:0/3722258808 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d1c020070 con 0x5620696c54a0 2024-09-12T02:17:46.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.749+0000 7f5d0dffb640 1 -- 172.21.15.16:0/3722258808 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5d1c00a7b0 con 0x5620696c54a0 2024-09-12T02:17:46.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.749+0000 7f5d2ed52740 10 client.20823 fetch_fsmap learned FSMap version 25 2024-09-12T02:17:46.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.749+0000 7f5d2ed52740 1 -- 172.21.15.16:0/3722258808 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5620696dec70 con 0x5620696c54a0 2024-09-12T02:17:46.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.749+0000 7f5d0dffb640 1 -- 172.21.15.16:0/3722258808 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d1c020070 con 0x5620696c54a0 2024-09-12T02:17:46.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.750+0000 7f5d0dffb640 1 -- 172.21.15.16:0/3722258808 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f5d1c0046c0 con 0x5620696c54a0 2024-09-12T02:17:46.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.750+0000 7f5d2ed52740 10 client.20823 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:17:46.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.750+0000 7f5d2ed52740 10 client.20823 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:17:46.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.750+0000 7f5d2ed52740 20 client.20823 populate_metadata read hostname 'smithi016' 2024-09-12T02:17:46.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.750+0000 7f5d2ed52740 1 --2- 172.21.15.16:0/3722258808 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5620696e0020 0x5620696e2410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:46.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.750+0000 7f5d2ed52740 4 client.20823 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:17:46.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.750+0000 7f5d2ed52740 1 -- 172.21.15.16:0/3722258808 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562069638850 con 0x5620696e0020 2024-09-12T02:17:46.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.750+0000 7f5d2659c640 1 --2- 172.21.15.16:0/3722258808 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5620696e0020 0x5620696e2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:46.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.750+0000 7f5d0dffb640 10 client.20823 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:17:46.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.750+0000 7f5d2659c640 1 --2- 172.21.15.16:0/3722258808 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5620696e0020 0x5620696e2410 crc :-1 s=READY pgs=473 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c80a9e554224e33 server_cookie=7035576f3b2ace6d in_seq=0 out_seq=0 2024-09-12T02:17:46.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.751+0000 7f5d0dffb640 1 -- 172.21.15.16:0/3722258808 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562069638850 con 0x5620696e0020 2024-09-12T02:17:46.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.751+0000 7f5d0dffb640 10 client.20823 handle_command_reply: tid=0 2024-09-12T02:17:46.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.751+0000 7f5d2ed52740 1 client.20823 shutdown 2024-09-12T02:17:46.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.751+0000 7f5d2ed52740 1 -- 172.21.15.16:0/3722258808 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620696c54a0 msgr2=0x5620696d0040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:46.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.751+0000 7f5d2ed52740 1 --2- 172.21.15.16:0/3722258808 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620696c54a0 0x5620696d0040 secure :-1 s=READY pgs=2121 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c00a310 tx=0x7f5d1c01cf50 comp rx=0 tx=0).stop 2024-09-12T02:17:46.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.751+0000 7f5d2ed52740 1 -- 172.21.15.16:0/3722258808 shutdown_connections 2024-09-12T02:17:46.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.751+0000 7f5d2ed52740 1 --2- 172.21.15.16:0/3722258808 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x5620696e0020 0x5620696e2410 crc :-1 s=READY pgs=473 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:46.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.751+0000 7f5d2ed52740 1 --2- 172.21.15.16:0/3722258808 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620696d0680 0x5620696cd8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:46.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.751+0000 7f5d2ed52740 1 --2- 172.21.15.16:0/3722258808 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5620696c5dc0 0x5620696cd2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:46.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.751+0000 7f5d0dffb640 0 client.20823 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:17:46.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.751+0000 7f5d2ed52740 1 --2- 172.21.15.16:0/3722258808 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620696c54a0 0x5620696d0040 unknown :-1 s=CLOSED pgs=2121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:46.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.751+0000 7f5d2ed52740 1 -- 172.21.15.16:0/3722258808 >> 172.21.15.16:0/3722258808 conn(0x5620696c1a30 msgr2=0x5620696c7760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:46.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.751+0000 7f5d2ed52740 1 -- 172.21.15.16:0/3722258808 shutdown_connections 2024-09-12T02:17:46.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.751+0000 7f5d2ed52740 1 -- 172.21.15.16:0/3722258808 wait complete. 2024-09-12T02:17:46.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.751+0000 7f5d2ed52740 20 client.20823 trim_cache size 0 max 16384 2024-09-12T02:17:46.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.769+0000 7f5d2ed52740 1 Processor -- start 2024-09-12T02:17:46.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.769+0000 7f5d2ed52740 1 -- start start 2024-09-12T02:17:46.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.770+0000 7f5d2ed52740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5620696c54a0 0x5620696c1380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:46.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.770+0000 7f5d2ed52740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620696c5dc0 0x5620696e79e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:46.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.770+0000 7f5d2ed52740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620696d0680 0x5620696e7f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:46.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.770+0000 7f5d2ed52740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5d281ada30 con 0x5620696c5dc0 2024-09-12T02:17:46.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.770+0000 7f5d2ed52740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5d281ab300 con 0x5620696d0680 2024-09-12T02:17:46.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.770+0000 7f5d2ed52740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5d281a9170 con 0x5620696c54a0 2024-09-12T02:17:46.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.770+0000 7f5d2659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620696c5dc0 0x5620696e79e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:46.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.770+0000 7f5d2659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620696c5dc0 0x5620696e79e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47336/0 (socket says 172.21.15.16:47336) 2024-09-12T02:17:46.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.770+0000 7f5d2659c640 1 -- 172.21.15.16:0/1337674168 learned_addr learned my addr 172.21.15.16:0/1337674168 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:17:46.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.770+0000 7f5d25d9b640 1 --2- 172.21.15.16:0/1337674168 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5620696c54a0 0x5620696c1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:46.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.770+0000 7f5d25d9b640 1 --2- 172.21.15.16:0/1337674168 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620696d0680 0x5620696e7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:46.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.770+0000 7f5d2659c640 1 -- 172.21.15.16:0/1337674168 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5620696c54a0 msgr2=0x5620696c1380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:46.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.770+0000 7f5d2659c640 1 --2- 172.21.15.16:0/1337674168 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5620696c54a0 0x5620696c1380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:46.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.770+0000 7f5d2659c640 1 -- 172.21.15.16:0/1337674168 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620696d0680 msgr2=0x5620696e7f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:46.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.770+0000 7f5d2659c640 1 --2- 172.21.15.16:0/1337674168 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620696d0680 0x5620696e7f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:46.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.770+0000 7f5d2659c640 1 -- 172.21.15.16:0/1337674168 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5620696d2de0 con 0x5620696c5dc0 2024-09-12T02:17:46.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.770+0000 7f5d25d9b640 1 --2- 172.21.15.16:0/1337674168 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620696d0680 0x5620696e7f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:17:46.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.770+0000 7f5d2659c640 1 --2- 172.21.15.16:0/1337674168 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620696c5dc0 0x5620696e79e0 secure :-1 s=READY pgs=2122 cs=0 l=1 rev1=1 crypto rx=0x5620696c2890 tx=0x7f5d18011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53d967ad7ce71d42 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:46.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.771+0000 7f5d0ffff640 1 -- 172.21.15.16:0/1337674168 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d18004220 con 0x5620696c5dc0 2024-09-12T02:17:46.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.771+0000 7f5d0ffff640 1 -- 172.21.15.16:0/1337674168 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d180043c0 con 0x5620696c5dc0 2024-09-12T02:17:46.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.771+0000 7f5d0ffff640 1 -- 172.21.15.16:0/1337674168 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d180035f0 con 0x5620696c5dc0 2024-09-12T02:17:46.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.771+0000 7f5d2ed52740 1 -- 172.21.15.16:0/1337674168 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620696c5dc0 msgr2=0x5620696e79e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:46.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.771+0000 7f5d2ed52740 1 --2- 172.21.15.16:0/1337674168 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620696c5dc0 0x5620696e79e0 secure :-1 s=READY pgs=2122 cs=0 l=1 rev1=1 crypto rx=0x5620696c2890 tx=0x7f5d18011810 comp rx=0 tx=0).stop 2024-09-12T02:17:46.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.771+0000 7f5d2ed52740 1 -- 172.21.15.16:0/1337674168 shutdown_connections 2024-09-12T02:17:46.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.771+0000 7f5d2ed52740 1 --2- 172.21.15.16:0/1337674168 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620696d0680 0x5620696e7f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:46.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.771+0000 7f5d2ed52740 1 --2- 172.21.15.16:0/1337674168 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620696c5dc0 0x5620696e79e0 unknown :-1 s=CLOSED pgs=2122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:46.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.771+0000 7f5d2ed52740 1 --2- 172.21.15.16:0/1337674168 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5620696c54a0 0x5620696c1380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:46.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.771+0000 7f5d2ed52740 1 -- 172.21.15.16:0/1337674168 >> 172.21.15.16:0/1337674168 conn(0x5620696c1a30 msgr2=0x5620696cf370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:46.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.772+0000 7f5d2ed52740 1 -- 172.21.15.16:0/1337674168 shutdown_connections 2024-09-12T02:17:46.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.772+0000 7f5d2ed52740 1 -- 172.21.15.16:0/1337674168 wait complete. 2024-09-12T02:17:46.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.772+0000 7f5d2ed52740 1 -- start start 2024-09-12T02:17:46.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.772+0000 7f5d2ed52740 1 Processor -- start 2024-09-12T02:17:46.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.772+0000 7f5d2ed52740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620696c54a0 0x5620696d6580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:46.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.772+0000 7f5d2ed52740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5620696c5dc0 0x5620696d6ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:46.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.772+0000 7f5d2ed52740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620696d0680 0x5620696d7000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:46.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.772+0000 7f5d2ed52740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5d281a97c0 con 0x5620696d0680 2024-09-12T02:17:46.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.772+0000 7f5d2ed52740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5d281aac30 con 0x5620696c54a0 2024-09-12T02:17:46.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.772+0000 7f5d2ed52740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5d281a9e90 con 0x5620696c5dc0 2024-09-12T02:17:46.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.772+0000 7f5d2ed52740 10 client.0 _ll_register_callbacks cb 0x7f5d1c013f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:17:46.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.772+0000 7f5d25d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620696d0680 0x5620696d7000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:46.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.772+0000 7f5d25d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620696d0680 0x5620696d7000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47344/0 (socket says 172.21.15.16:47344) 2024-09-12T02:17:46.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.772+0000 7f5d25d9b640 1 -- 172.21.15.16:0/2941050101 learned_addr learned my addr 172.21.15.16:0/2941050101 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:17:46.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.773+0000 7f5d25d9b640 1 --2- 172.21.15.16:0/2941050101 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620696c54a0 0x5620696d6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:46.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.773+0000 7f5d2659c640 1 --2- 172.21.15.16:0/2941050101 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5620696c5dc0 0x5620696d6ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:46.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.773+0000 7f5d25d9b640 1 -- 172.21.15.16:0/2941050101 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5620696c5dc0 msgr2=0x5620696d6ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:46.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.773+0000 7f5d25d9b640 1 --2- 172.21.15.16:0/2941050101 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5620696c5dc0 0x5620696d6ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:46.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.773+0000 7f5d25d9b640 1 -- 172.21.15.16:0/2941050101 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620696c54a0 msgr2=0x5620696d6580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:46.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.773+0000 7f5d25d9b640 1 --2- 172.21.15.16:0/2941050101 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620696c54a0 0x5620696d6580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:46.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.773+0000 7f5d25d9b640 1 -- 172.21.15.16:0/2941050101 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562069749bc0 con 0x5620696d0680 2024-09-12T02:17:46.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.773+0000 7f5d2659c640 1 --2- 172.21.15.16:0/2941050101 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5620696c5dc0 0x5620696d6ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:17:46.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.773+0000 7f5d25d9b640 1 --2- 172.21.15.16:0/2941050101 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620696d0680 0x5620696d7000 secure :-1 s=READY pgs=2123 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c004ac0 tx=0x7f5d1c01cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80fcf178b8b0c62f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:46.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.773+0000 7f5d0dffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:17:46.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.773+0000 7f5d0dffb640 1 -- 172.21.15.16:0/2941050101 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d1c02a040 con 0x5620696d0680 2024-09-12T02:17:46.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.773+0000 7f5d0dffb640 1 -- 172.21.15.16:0/2941050101 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f5d1c013270 con 0x5620696d0680 2024-09-12T02:17:46.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.773+0000 7f5d2ed52740 1 -- 172.21.15.16:0/2941050101 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d180043c0 con 0x5620696d0680 2024-09-12T02:17:46.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.773+0000 7f5d0dffb640 1 -- 172.21.15.16:0/2941050101 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5d1c01c020 con 0x5620696d0680 2024-09-12T02:17:46.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.773+0000 7f5d2ed52740 10 client.20829 fetch_fsmap learned FSMap version 25 2024-09-12T02:17:46.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.773+0000 7f5d2ed52740 1 -- 172.21.15.16:0/2941050101 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56206974ae00 con 0x5620696d0680 2024-09-12T02:17:46.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.774+0000 7f5d0dffb640 1 -- 172.21.15.16:0/2941050101 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d180043c0 con 0x5620696d0680 2024-09-12T02:17:46.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.774+0000 7f5d0dffb640 1 -- 172.21.15.16:0/2941050101 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f5d1c01b720 con 0x5620696d0680 2024-09-12T02:17:46.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.774+0000 7f5d2ed52740 10 client.20829 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:17:46.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.774+0000 7f5d2ed52740 10 client.20829 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:17:46.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.774+0000 7f5d2ed52740 20 client.20829 populate_metadata read hostname 'smithi016' 2024-09-12T02:17:46.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.774+0000 7f5d2ed52740 1 --2- 172.21.15.16:0/2941050101 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x56206974b500 0x56206974d920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:46.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.774+0000 7f5d2ed52740 4 client.20829 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T02:17:46.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.774+0000 7f5d2ed52740 1 -- 172.21.15.16:0/2941050101 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562069632d60 con 0x56206974b500 2024-09-12T02:17:46.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.774+0000 7f5d2659c640 1 --2- 172.21.15.16:0/2941050101 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x56206974b500 0x56206974d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:46.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.774+0000 7f5d2659c640 1 --2- 172.21.15.16:0/2941050101 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x56206974b500 0x56206974d920 crc :-1 s=READY pgs=474 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92c2ad8f8b18fc10 server_cookie=fb19f5d9de5372b8 in_seq=0 out_seq=0 2024-09-12T02:17:46.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.774+0000 7f5d0dffb640 10 client.20829 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:17:46.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.775+0000 7f5d0dffb640 1 -- 172.21.15.16:0/2941050101 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x562069632d60 con 0x56206974b500 2024-09-12T02:17:46.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.775+0000 7f5d0dffb640 10 client.20829 handle_command_reply: tid=0 2024-09-12T02:17:46.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.775+0000 7f5d2ed52740 1 client.20829 shutdown 2024-09-12T02:17:46.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.775+0000 7f5d2ed52740 1 -- 172.21.15.16:0/2941050101 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620696d0680 msgr2=0x5620696d7000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:46.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.775+0000 7f5d2ed52740 1 --2- 172.21.15.16:0/2941050101 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620696d0680 0x5620696d7000 secure :-1 s=READY pgs=2123 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c004ac0 tx=0x7f5d1c01cf50 comp rx=0 tx=0).stop 2024-09-12T02:17:46.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.775+0000 7f5d2ed52740 1 -- 172.21.15.16:0/2941050101 shutdown_connections 2024-09-12T02:17:46.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.775+0000 7f5d2ed52740 1 --2- 172.21.15.16:0/2941050101 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x56206974b500 0x56206974d920 crc :-1 s=READY pgs=474 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:46.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.775+0000 7f5d2ed52740 1 --2- 172.21.15.16:0/2941050101 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620696d0680 0x5620696d7000 unknown :-1 s=CLOSED pgs=2123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:46.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.775+0000 7f5d2ed52740 1 --2- 172.21.15.16:0/2941050101 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5620696c5dc0 0x5620696d6ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:46.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.775+0000 7f5d0dffb640 0 client.20829 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:17:46.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.775+0000 7f5d2ed52740 1 --2- 172.21.15.16:0/2941050101 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620696c54a0 0x5620696d6580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:46.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.775+0000 7f5d2ed52740 1 -- 172.21.15.16:0/2941050101 >> 172.21.15.16:0/2941050101 conn(0x5620696c1a30 msgr2=0x5620696c0350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:46.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.775+0000 7f5d2ed52740 1 -- 172.21.15.16:0/2941050101 shutdown_connections 2024-09-12T02:17:46.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.775+0000 7f5d2ed52740 1 -- 172.21.15.16:0/2941050101 wait complete. 2024-09-12T02:17:46.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.775+0000 7f5d2ed52740 20 client.20829 trim_cache size 0 max 16384 2024-09-12T02:17:46.782 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T02:17:46.782 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-09-12T02:17:46.782 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {} 2024-09-12T02:17:46.783 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T02:17:46.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.776+0000 7f5d2c979640 1 -- 172.21.15.16:0/90028287 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f5d100a2c00 msgr2=0x7f5d100a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:46.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.776+0000 7f5d2c979640 1 --2- 172.21.15.16:0/90028287 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f5d100a2c00 0x7f5d100a50c0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f5d20006010 tx=0x7f5d20087470 comp rx=0 tx=0).stop 2024-09-12T02:17:46.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.776+0000 7f5d2c979640 1 -- 172.21.15.16:0/90028287 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d281082a0 msgr2=0x7f5d2819cd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:46.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.776+0000 7f5d2c979640 1 --2- 172.21.15.16:0/90028287 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d281082a0 0x7f5d2819cd90 secure :-1 s=READY pgs=2119 cs=0 l=1 rev1=1 crypto rx=0x7f5d20005cc0 tx=0x7f5d20003970 comp rx=0 tx=0).stop 2024-09-12T02:17:46.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.776+0000 7f5d2c979640 1 -- 172.21.15.16:0/90028287 shutdown_connections 2024-09-12T02:17:46.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.776+0000 7f5d2c979640 1 --2- 172.21.15.16:0/90028287 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f5d100a2c00 0x7f5d100a50c0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:46.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.776+0000 7f5d2c979640 1 --2- 172.21.15.16:0/90028287 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d281082a0 0x7f5d2819cd90 unknown :-1 s=CLOSED pgs=2119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:46.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.776+0000 7f5d2c979640 1 --2- 172.21.15.16:0/90028287 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5d281036e0 0x7f5d28198a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:46.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.776+0000 7f5d2c979640 1 --2- 172.21.15.16:0/90028287 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d28102d30 0x7f5d281984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:46.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.776+0000 7f5d2c979640 1 -- 172.21.15.16:0/90028287 >> 172.21.15.16:0/90028287 conn(0x7f5d280fc7d0 msgr2=0x7f5d28106380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:46.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.776+0000 7f5d2c979640 1 -- 172.21.15.16:0/90028287 shutdown_connections 2024-09-12T02:17:46.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:46.776+0000 7f5d2c979640 1 -- 172.21.15.16:0/90028287 wait complete. 2024-09-12T02:17:46.934 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:17:46 smithi016 ceph-mon[30947]: scrub summary: idle 2024-09-12T02:17:47.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:17:46 smithi184 ceph-mon[35593]: scrub summary: idle 2024-09-12T02:17:47.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:17:46 smithi033 ceph-mon[35332]: scrub summary: idle 2024-09-12T02:17:47.575 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-12T02:17:47.576 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:17:47.878 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:17:47.934 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:17:47 smithi016 ceph-mon[30947]: pgmap v811: 97 pgs: 97 active+clean; 4.3 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 KiB/s wr, 17 op/s 2024-09-12T02:17:48.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:17:47 smithi184 ceph-mon[35593]: pgmap v811: 97 pgs: 97 active+clean; 4.3 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 KiB/s wr, 17 op/s 2024-09-12T02:17:48.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:17:47 smithi033 ceph-mon[35332]: pgmap v811: 97 pgs: 97 active+clean; 4.3 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 KiB/s wr, 17 op/s 2024-09-12T02:17:48.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:48.319+0000 7f8faddc4640 1 -- 172.21.15.16:0/3299287286 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8fa8103700 msgr2=0x7f8fa810bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:48.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:48.319+0000 7f8faddc4640 1 --2- 172.21.15.16:0/3299287286 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8fa8103700 0x7f8fa810bbf0 secure :-1 s=READY pgs=2124 cs=0 l=1 rev1=1 crypto rx=0x7f8f9c009900 tx=0x7f8f9c02a480 comp rx=0 tx=0).stop 2024-09-12T02:17:48.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:48.320+0000 7f8faddc4640 1 -- 172.21.15.16:0/3299287286 shutdown_connections 2024-09-12T02:17:48.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:48.320+0000 7f8faddc4640 1 --2- 172.21.15.16:0/3299287286 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8fa810c180 0x7f8fa810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:48.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:48.320+0000 7f8faddc4640 1 --2- 172.21.15.16:0/3299287286 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8fa8103700 0x7f8fa810bbf0 unknown :-1 s=CLOSED pgs=2124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:48.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:48.320+0000 7f8faddc4640 1 --2- 172.21.15.16:0/3299287286 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8fa8102de0 0x7f8fa81031c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:48.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:48.320+0000 7f8faddc4640 1 -- 172.21.15.16:0/3299287286 >> 172.21.15.16:0/3299287286 conn(0x7f8fa80fc7a0 msgr2=0x7f8fa80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:48.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:48.320+0000 7f8faddc4640 1 -- 172.21.15.16:0/3299287286 shutdown_connections 2024-09-12T02:17:48.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:48.320+0000 7f8faddc4640 1 -- 172.21.15.16:0/3299287286 wait complete. 2024-09-12T02:17:48.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:48.320+0000 7f8faddc4640 1 Processor -- start 2024-09-12T02:17:48.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:48.321+0000 7f8faddc4640 1 -- start start 2024-09-12T02:17:48.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:48.321+0000 7f8faddc4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8fa8102de0 0x7f8fa8198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:48.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:48.321+0000 7f8faddc4640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8fa8103700 0x7f8fa8198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:48.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:48.321+0000 7f8faddc4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8fa810c180 0x7f8fa819cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:48.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:48.321+0000 7f8faddc4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8fa8111f70 con 0x7f8fa810c180 2024-09-12T02:17:48.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:48.321+0000 7f8faddc4640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8fa8111df0 con 0x7f8fa8102de0 2024-09-12T02:17:48.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:48.321+0000 7f8faddc4640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8fa81120f0 con 0x7f8fa8103700 2024-09-12T02:17:48.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:48.321+0000 7f8fa7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8fa810c180 0x7f8fa819cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:48.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:48.321+0000 7f8fa7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8fa810c180 0x7f8fa819cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47374/0 (socket says 172.21.15.16:47374) 2024-09-12T02:17:48.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:48.321+0000 7f8fa7fff640 1 -- 172.21.15.16:0/3714256476 learned_addr learned my addr 172.21.15.16:0/3714256476 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:17:48.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:48.321+0000 7f8fa6ffd640 1 --2- 172.21.15.16:0/3714256476 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8fa8103700 0x7f8fa8198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:48.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:48.321+0000 7f8fa77fe640 1 --2- 172.21.15.16:0/3714256476 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8fa8102de0 0x7f8fa8198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:48.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:48.321+0000 7f8fa7fff640 1 -- 172.21.15.16:0/3714256476 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8fa8103700 msgr2=0x7f8fa8198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:48.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:48.321+0000 7f8fa7fff640 1 --2- 172.21.15.16:0/3714256476 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8fa8103700 0x7f8fa8198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:48.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:48.321+0000 7f8fa7fff640 1 -- 172.21.15.16:0/3714256476 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8fa8102de0 msgr2=0x7f8fa8198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:48.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:48.322+0000 7f8fa7fff640 1 --2- 172.21.15.16:0/3714256476 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8fa8102de0 0x7f8fa8198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:48.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:48.322+0000 7f8fa7fff640 1 -- 172.21.15.16:0/3714256476 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fa819d3c0 con 0x7f8fa810c180 2024-09-12T02:17:48.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:48.322+0000 7f8fa7fff640 1 --2- 172.21.15.16:0/3714256476 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8fa810c180 0x7f8fa819cce0 secure :-1 s=READY pgs=2125 cs=0 l=1 rev1=1 crypto rx=0x7f8fa000cce0 tx=0x7f8fa000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:48.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:48.322+0000 7f8fa4ff9640 1 -- 172.21.15.16:0/3714256476 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fa000f450 con 0x7f8fa810c180 2024-09-12T02:17:48.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:48.322+0000 7f8fa4ff9640 1 -- 172.21.15.16:0/3714256476 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8fa000fad0 con 0x7f8fa810c180 2024-09-12T02:17:48.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:48.322+0000 7f8faddc4640 1 -- 172.21.15.16:0/3714256476 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fa819d6b0 con 0x7f8fa810c180 2024-09-12T02:17:48.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:48.322+0000 7f8fa4ff9640 1 -- 172.21.15.16:0/3714256476 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fa0018c10 con 0x7f8fa810c180 2024-09-12T02:17:48.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:48.322+0000 7f8faddc4640 1 -- 172.21.15.16:0/3714256476 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fa81a4f90 con 0x7f8fa810c180 2024-09-12T02:17:48.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:48.323+0000 7f8faddc4640 1 -- 172.21.15.16:0/3714256476 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fa81031c0 con 0x7f8fa810c180 2024-09-12T02:17:48.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:48.324+0000 7f8fa4ff9640 1 -- 172.21.15.16:0/3714256476 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f8fa000f5f0 con 0x7f8fa810c180 2024-09-12T02:17:48.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:48.325+0000 7f8fa4ff9640 1 --2- 172.21.15.16:0/3714256476 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f8f900a2c00 0x7f8f900a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:48.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:48.325+0000 7f8fa77fe640 1 --2- 172.21.15.16:0/3714256476 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f8f900a2c00 0x7f8f900a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:48.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:48.325+0000 7f8fa4ff9640 1 -- 172.21.15.16:0/3714256476 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f8fa00cf4a0 con 0x7f8fa810c180 2024-09-12T02:17:48.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:48.326+0000 7f8fa77fe640 1 --2- 172.21.15.16:0/3714256476 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f8f900a2c00 0x7f8f900a50c0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f8f9800bbe0 tx=0x7f8f9800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:48.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:48.329+0000 7f8fa4ff9640 1 -- 172.21.15.16:0/3714256476 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8fa0014030 con 0x7f8fa810c180 2024-09-12T02:17:48.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:48.540+0000 7f8faddc4640 1 -- 172.21.15.16:0/3714256476 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8fa802d7c0 con 0x7f8fa810c180 2024-09-12T02:17:48.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:48.541+0000 7f8fa4ff9640 1 -- 172.21.15.16:0/3714256476 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11911 (secure 0 0 0) 0x7f8fa009c4a0 con 0x7f8fa810c180 2024-09-12T02:17:48.541 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T02:17:48.542 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":25,"btime":"2024-09-12T02:01:37:192021+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3279123279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3279123279},{"type":"v1","addr":"172.21.15.16:6835","nonce":3279123279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4119452382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4119452382},{"type":"v1","addr":"172.21.15.16:6837","nonce":4119452382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/3263823165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3263823165},{"type":"v1","addr":"172.21.15.16:6839","nonce":3263823165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/1510290732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":1510290732},{"type":"v1","addr":"172.21.15.16:6841","nonce":1510290732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/279674603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":279674603},{"type":"v1","addr":"172.21.15.33:6833","nonce":279674603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3555728028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3555728028},{"type":"v1","addr":"172.21.15.33:6835","nonce":3555728028}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T01:59:56.117608+0000","modified":"2024-09-12T02:01:36.294760+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24482,"mds_1":24481,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6833/1563374106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1563374106},{"type":"v1","addr":"172.21.15.184:6833","nonce":1563374106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6837/1892833868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1892833868},{"type":"v1","addr":"172.21.15.33:6837","nonce":1892833868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/1457679686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1457679686},{"type":"v1","addr":"172.21.15.184:6835","nonce":1457679686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.184:6837/1793393087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":1793393087},{"type":"v1","addr":"172.21.15.184:6837","nonce":1793393087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.184:6839/4203347941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":4203347941},{"type":"v1","addr":"172.21.15.184:6839","nonce":4203347941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.33:6839/4136670204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":4136670204},{"type":"v1","addr":"172.21.15.33:6839","nonce":4136670204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24481,24482,24469]},"id":1}]} 2024-09-12T02:17:48.542 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 25 2024-09-12T02:17:48.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:48.543+0000 7f8faddc4640 1 -- 172.21.15.16:0/3714256476 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f8f900a2c00 msgr2=0x7f8f900a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:48.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:48.543+0000 7f8faddc4640 1 --2- 172.21.15.16:0/3714256476 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f8f900a2c00 0x7f8f900a50c0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f8f9800bbe0 tx=0x7f8f9800b390 comp rx=0 tx=0).stop 2024-09-12T02:17:48.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:48.543+0000 7f8faddc4640 1 -- 172.21.15.16:0/3714256476 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8fa810c180 msgr2=0x7f8fa819cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:48.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:48.543+0000 7f8faddc4640 1 --2- 172.21.15.16:0/3714256476 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8fa810c180 0x7f8fa819cce0 secure :-1 s=READY pgs=2125 cs=0 l=1 rev1=1 crypto rx=0x7f8fa000cce0 tx=0x7f8fa000a620 comp rx=0 tx=0).stop 2024-09-12T02:17:48.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:48.543+0000 7f8faddc4640 1 -- 172.21.15.16:0/3714256476 shutdown_connections 2024-09-12T02:17:48.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:48.543+0000 7f8faddc4640 1 --2- 172.21.15.16:0/3714256476 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f8f900a2c00 0x7f8f900a50c0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:48.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:48.543+0000 7f8faddc4640 1 --2- 172.21.15.16:0/3714256476 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8fa810c180 0x7f8fa819cce0 unknown :-1 s=CLOSED pgs=2125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:48.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:48.543+0000 7f8faddc4640 1 --2- 172.21.15.16:0/3714256476 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8fa8103700 0x7f8fa8198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:48.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:48.543+0000 7f8faddc4640 1 --2- 172.21.15.16:0/3714256476 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8fa8102de0 0x7f8fa8198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:48.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:48.543+0000 7f8faddc4640 1 -- 172.21.15.16:0/3714256476 >> 172.21.15.16:0/3714256476 conn(0x7f8fa80fc7a0 msgr2=0x7f8fa80fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:48.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:48.543+0000 7f8faddc4640 1 -- 172.21.15.16:0/3714256476 shutdown_connections 2024-09-12T02:17:48.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:48.543+0000 7f8faddc4640 1 -- 172.21.15.16:0/3714256476 wait complete. 2024-09-12T02:17:49.030 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:17:48 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/3714256476' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:17:49.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:17:48 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/3714256476' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:17:49.138 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-12T02:17:49.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:17:48 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/3714256476' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:17:49.394 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:17:49.805 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:17:49 smithi016 ceph-mon[30947]: pgmap v812: 97 pgs: 97 active+clean; 4.3 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.8 KiB/s wr, 13 op/s 2024-09-12T02:17:49.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.888+0000 7f7d2704c640 1 -- 172.21.15.16:0/2753277638 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d20101df0 msgr2=0x7f7d201021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:49.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.888+0000 7f7d2704c640 1 --2- 172.21.15.16:0/2753277638 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d20101df0 0x7f7d201021d0 secure :-1 s=READY pgs=2126 cs=0 l=1 rev1=1 crypto rx=0x7f7d14009900 tx=0x7f7d1402a410 comp rx=0 tx=0).stop 2024-09-12T02:17:49.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.888+0000 7f7d2704c640 1 -- 172.21.15.16:0/2753277638 shutdown_connections 2024-09-12T02:17:49.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.888+0000 7f7d2704c640 1 --2- 172.21.15.16:0/2753277638 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d20106f20 0x7f7d2010eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:49.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.888+0000 7f7d2704c640 1 --2- 172.21.15.16:0/2753277638 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7d201027a0 0x7f7d201067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:49.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.888+0000 7f7d2704c640 1 --2- 172.21.15.16:0/2753277638 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d20101df0 0x7f7d201021d0 unknown :-1 s=CLOSED pgs=2126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:49.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.888+0000 7f7d2704c640 1 -- 172.21.15.16:0/2753277638 >> 172.21.15.16:0/2753277638 conn(0x7f7d200fd8b0 msgr2=0x7f7d200ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:49.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.888+0000 7f7d2704c640 1 -- 172.21.15.16:0/2753277638 shutdown_connections 2024-09-12T02:17:49.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.888+0000 7f7d2704c640 1 -- 172.21.15.16:0/2753277638 wait complete. 2024-09-12T02:17:49.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.889+0000 7f7d2704c640 1 Processor -- start 2024-09-12T02:17:49.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.889+0000 7f7d2704c640 1 -- start start 2024-09-12T02:17:49.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.889+0000 7f7d2704c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d20101df0 0x7f7d2019a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:49.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.889+0000 7f7d2704c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d201027a0 0x7f7d2019abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:49.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.889+0000 7f7d2704c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7d20106f20 0x7f7d2019ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:49.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.889+0000 7f7d2704c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7d20111f80 con 0x7f7d201027a0 2024-09-12T02:17:49.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.889+0000 7f7d2704c640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7d20111e00 con 0x7f7d20101df0 2024-09-12T02:17:49.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.889+0000 7f7d2704c640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7d20112100 con 0x7f7d20106f20 2024-09-12T02:17:49.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.889+0000 7f7d1ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d201027a0 0x7f7d2019abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:49.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.889+0000 7f7d24de8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d20101df0 0x7f7d2019a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:49.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.889+0000 7f7d1ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d201027a0 0x7f7d2019abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47386/0 (socket says 172.21.15.16:47386) 2024-09-12T02:17:49.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.889+0000 7f7d1ffff640 1 -- 172.21.15.16:0/996717621 learned_addr learned my addr 172.21.15.16:0/996717621 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:17:49.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.889+0000 7f7d24de8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d20101df0 0x7f7d2019a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.16:37648/0 (socket says 172.21.15.16:37648) 2024-09-12T02:17:49.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.890+0000 7f7d255e9640 1 --2- 172.21.15.16:0/996717621 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7d20106f20 0x7f7d2019ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:49.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.890+0000 7f7d1ffff640 1 -- 172.21.15.16:0/996717621 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7d20106f20 msgr2=0x7f7d2019ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:49.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.890+0000 7f7d1ffff640 1 --2- 172.21.15.16:0/996717621 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7d20106f20 0x7f7d2019ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:49.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.890+0000 7f7d1ffff640 1 -- 172.21.15.16:0/996717621 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d20101df0 msgr2=0x7f7d2019a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:49.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.890+0000 7f7d1ffff640 1 --2- 172.21.15.16:0/996717621 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d20101df0 0x7f7d2019a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:49.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.890+0000 7f7d1ffff640 1 -- 172.21.15.16:0/996717621 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d2019f650 con 0x7f7d201027a0 2024-09-12T02:17:49.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.890+0000 7f7d1ffff640 1 --2- 172.21.15.16:0/996717621 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d201027a0 0x7f7d2019abe0 secure :-1 s=READY pgs=2127 cs=0 l=1 rev1=1 crypto rx=0x7f7d0c00d770 tx=0x7f7d0c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:49.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.890+0000 7f7d1dffb640 1 -- 172.21.15.16:0/996717621 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d0c017070 con 0x7f7d201027a0 2024-09-12T02:17:49.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.890+0000 7f7d1dffb640 1 -- 172.21.15.16:0/996717621 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d0c005500 con 0x7f7d201027a0 2024-09-12T02:17:49.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.890+0000 7f7d2704c640 1 -- 172.21.15.16:0/996717621 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d2019f940 con 0x7f7d201027a0 2024-09-12T02:17:49.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.890+0000 7f7d1dffb640 1 -- 172.21.15.16:0/996717621 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d0c004e00 con 0x7f7d201027a0 2024-09-12T02:17:49.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.890+0000 7f7d2704c640 1 -- 172.21.15.16:0/996717621 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d201a7220 con 0x7f7d201027a0 2024-09-12T02:17:49.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.892+0000 7f7d1dffb640 1 -- 172.21.15.16:0/996717621 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f7d0c004fa0 con 0x7f7d201027a0 2024-09-12T02:17:49.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.893+0000 7f7d29425740 1 Processor -- start 2024-09-12T02:17:49.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.893+0000 7f7d29425740 1 -- start start 2024-09-12T02:17:49.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.893+0000 7f7d29425740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55863790eb40 0x55863790d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:49.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.893+0000 7f7d29425740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55863790d5d0 0x55863790da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:49.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.893+0000 7f7d29425740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558637918370 0x55863791a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:49.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.893+0000 7f7d29425740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5586378732f0 con 0x558637918370 2024-09-12T02:17:49.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.893+0000 7f7d29425740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55863781acf0 con 0x55863790eb40 2024-09-12T02:17:49.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.893+0000 7f7d29425740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55863786f2b0 con 0x55863790d5d0 2024-09-12T02:17:49.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.893+0000 7f7d1ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558637918370 0x55863791a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:49.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.893+0000 7f7d1dffb640 1 --2- 172.21.15.16:0/996717621 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f7d100a2c00 0x7f7d100a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:49.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.893+0000 7f7d1ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558637918370 0x55863791a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47394/0 (socket says 172.21.15.16:47394) 2024-09-12T02:17:49.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.893+0000 7f7d1ffff640 1 -- 172.21.15.16:0/363821661 learned_addr learned my addr 172.21.15.16:0/363821661 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:17:49.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.893+0000 7f7d24de8640 1 --2- 172.21.15.16:0/363821661 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55863790d5d0 0x55863790da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:49.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.893+0000 7f7d255e9640 1 --2- 172.21.15.16:0/996717621 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f7d100a2c00 0x7f7d100a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:49.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.894+0000 7f7d255e9640 1 --2- 172.21.15.16:0/363821661 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55863790eb40 0x55863790d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:49.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.894+0000 7f7d1dffb640 1 -- 172.21.15.16:0/996717621 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f7d0c0ce960 con 0x7f7d201027a0 2024-09-12T02:17:49.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.894+0000 7f7d1ffff640 1 -- 172.21.15.16:0/363821661 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55863790d5d0 msgr2=0x55863790da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:49.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.894+0000 7f7d1ffff640 1 --2- 172.21.15.16:0/363821661 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55863790d5d0 0x55863790da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:49.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.894+0000 7f7d1ffff640 1 -- 172.21.15.16:0/363821661 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55863790eb40 msgr2=0x55863790d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:49.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.894+0000 7f7d1ffff640 1 --2- 172.21.15.16:0/363821661 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55863790eb40 0x55863790d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:49.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.894+0000 7f7d1ffff640 1 -- 172.21.15.16:0/363821661 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586377ff3a0 con 0x558637918370 2024-09-12T02:17:49.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.894+0000 7f7d1ffff640 1 --2- 172.21.15.16:0/363821661 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558637918370 0x55863791a760 secure :-1 s=READY pgs=2128 cs=0 l=1 rev1=1 crypto rx=0x7f7d0c021000 tx=0x7f7d0c009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce676b6395dc5978 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:49.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.894+0000 7f7d255e9640 1 --2- 172.21.15.16:0/996717621 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f7d100a2c00 0x7f7d100a50c0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f7d18009fd0 tx=0x7f7d18009920 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:49.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.894+0000 7f7d0affd640 1 -- 172.21.15.16:0/363821661 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d0c020070 con 0x558637918370 2024-09-12T02:17:49.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.894+0000 7f7d0affd640 1 -- 172.21.15.16:0/363821661 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d0c09b270 con 0x558637918370 2024-09-12T02:17:49.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.894+0000 7f7d0affd640 1 -- 172.21.15.16:0/363821661 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d0c0978e0 con 0x558637918370 2024-09-12T02:17:49.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.895+0000 7f7d29425740 1 -- 172.21.15.16:0/363821661 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558637918370 msgr2=0x55863791a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:49.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.895+0000 7f7d29425740 1 --2- 172.21.15.16:0/363821661 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558637918370 0x55863791a760 secure :-1 s=READY pgs=2128 cs=0 l=1 rev1=1 crypto rx=0x7f7d0c021000 tx=0x7f7d0c009c30 comp rx=0 tx=0).stop 2024-09-12T02:17:49.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.895+0000 7f7d29425740 1 -- 172.21.15.16:0/363821661 shutdown_connections 2024-09-12T02:17:49.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.895+0000 7f7d29425740 1 --2- 172.21.15.16:0/363821661 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558637918370 0x55863791a760 unknown :-1 s=CLOSED pgs=2128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:49.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.895+0000 7f7d29425740 1 --2- 172.21.15.16:0/363821661 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55863790d5d0 0x55863790da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:49.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.895+0000 7f7d29425740 1 --2- 172.21.15.16:0/363821661 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55863790eb40 0x55863790d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:49.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.895+0000 7f7d29425740 1 -- 172.21.15.16:0/363821661 >> 172.21.15.16:0/363821661 conn(0x5586378d6310 msgr2=0x558637909c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:49.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.895+0000 7f7d29425740 1 -- 172.21.15.16:0/363821661 shutdown_connections 2024-09-12T02:17:49.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.895+0000 7f7d29425740 1 -- 172.21.15.16:0/363821661 wait complete. 2024-09-12T02:17:49.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.895+0000 7f7d29425740 1 -- start start 2024-09-12T02:17:49.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.896+0000 7f7d29425740 1 Processor -- start 2024-09-12T02:17:49.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.896+0000 7f7d29425740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55863790d5d0 0x558637917d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:49.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.896+0000 7f7d29425740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55863790eb40 0x558637915040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:49.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.896+0000 7f7d29425740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558637918370 0x558637915650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:49.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.896+0000 7f7d29425740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55863791b010 con 0x558637918370 2024-09-12T02:17:49.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.896+0000 7f7d29425740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55863782e060 con 0x55863790d5d0 2024-09-12T02:17:49.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.896+0000 7f7d29425740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55863791b190 con 0x55863790eb40 2024-09-12T02:17:49.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.896+0000 7f7d29425740 10 client.0 _ll_register_callbacks cb 0x5586375818c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:17:49.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.896+0000 7f7d24de8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55863790d5d0 0x558637917d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:49.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.896+0000 7f7d24de8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55863790d5d0 0x558637917d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.16:37666/0 (socket says 172.21.15.16:37666) 2024-09-12T02:17:49.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.896+0000 7f7d24de8640 1 -- 172.21.15.16:0/1820647860 learned_addr learned my addr 172.21.15.16:0/1820647860 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:17:49.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.896+0000 7f7d1ffff640 1 --2- 172.21.15.16:0/1820647860 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55863790eb40 0x558637915040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:49.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.896+0000 7f7d255e9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558637918370 0x558637915650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:49.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.897+0000 7f7d24de8640 1 -- 172.21.15.16:0/1820647860 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55863790eb40 msgr2=0x558637915040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:49.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.897+0000 7f7d24de8640 1 --2- 172.21.15.16:0/1820647860 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55863790eb40 0x558637915040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:49.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.897+0000 7f7d24de8640 1 -- 172.21.15.16:0/1820647860 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558637918370 msgr2=0x558637915650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:49.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.897+0000 7f7d24de8640 1 --2- 172.21.15.16:0/1820647860 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558637918370 0x558637915650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:49.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.897+0000 7f7d24de8640 1 -- 172.21.15.16:0/1820647860 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586375e43c0 con 0x55863790d5d0 2024-09-12T02:17:49.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.897+0000 7f7d24de8640 1 --2- 172.21.15.16:0/1820647860 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55863790d5d0 0x558637917d90 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f7d14000c00 tx=0x7f7d140319b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dceda8f17a29dd6a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:49.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.897+0000 7f7cfffff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-12T02:17:49.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.897+0000 7f7cfffff640 1 -- 172.21.15.16:0/1820647860 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d14035040 con 0x55863790d5d0 2024-09-12T02:17:49.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.897+0000 7f7d29425740 1 -- 172.21.15.16:0/1820647860 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d0c09b270 con 0x55863790d5d0 2024-09-12T02:17:49.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.897+0000 7f7cfffff640 1 -- 172.21.15.16:0/1820647860 <== mon.2 v2:172.21.15.33:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f7d14004240 con 0x55863790d5d0 2024-09-12T02:17:49.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.897+0000 7f7cfffff640 1 -- 172.21.15.16:0/1820647860 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7d14004440 con 0x55863790d5d0 2024-09-12T02:17:49.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.897+0000 7f7d29425740 10 client.29465 fetch_fsmap learned FSMap version 25 2024-09-12T02:17:49.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.897+0000 7f7d29425740 1 -- 172.21.15.16:0/1820647860 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x558637926b30 con 0x55863790d5d0 2024-09-12T02:17:49.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.897+0000 7f7cfffff640 1 -- 172.21.15.16:0/1820647860 <== mon.2 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d0c09b270 con 0x55863790d5d0 2024-09-12T02:17:49.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.898+0000 7f7cfffff640 1 -- 172.21.15.16:0/1820647860 <== mon.2 v2:172.21.15.33:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f7d140467d0 con 0x55863790d5d0 2024-09-12T02:17:49.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.898+0000 7f7d29425740 10 client.29465 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:17:49.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.898+0000 7f7d29425740 10 client.29465 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-12T02:17:49.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.898+0000 7f7d29425740 20 client.29465 populate_metadata read hostname 'smithi016' 2024-09-12T02:17:49.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.898+0000 7f7d29425740 1 --2- 172.21.15.16:0/1820647860 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x558637927ff0 0x55863792a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:49.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.898+0000 7f7d29425740 4 client.29465 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:17:49.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.898+0000 7f7d29425740 1 -- 172.21.15.16:0/1820647860 --> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558637880860 con 0x558637927ff0 2024-09-12T02:17:49.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.898+0000 7f7d1ffff640 1 --2- 172.21.15.16:0/1820647860 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x558637927ff0 0x55863792a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:49.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.898+0000 7f7d1ffff640 1 --2- 172.21.15.16:0/1820647860 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x558637927ff0 0x55863792a3e0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a8fc8f78166e9b37 server_cookie=c9dead8bcd2fec61 in_seq=0 out_seq=0 2024-09-12T02:17:49.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.898+0000 7f7cfffff640 10 client.29465 ms_handle_connect on v2:172.21.15.184:6836/1793393087 2024-09-12T02:17:49.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.899+0000 7f7cfffff640 1 -- 172.21.15.16:0/1820647860 <== mds.2 v2:172.21.15.184:6836/1793393087 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558637880860 con 0x558637927ff0 2024-09-12T02:17:49.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.899+0000 7f7cfffff640 10 client.29465 handle_command_reply: tid=0 2024-09-12T02:17:49.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.899+0000 7f7d29425740 1 client.29465 shutdown 2024-09-12T02:17:49.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.899+0000 7f7d29425740 1 -- 172.21.15.16:0/1820647860 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55863790d5d0 msgr2=0x558637917d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:49.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.899+0000 7f7d29425740 1 --2- 172.21.15.16:0/1820647860 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55863790d5d0 0x558637917d90 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f7d14000c00 tx=0x7f7d140319b0 comp rx=0 tx=0).stop 2024-09-12T02:17:49.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.899+0000 7f7d29425740 1 -- 172.21.15.16:0/1820647860 shutdown_connections 2024-09-12T02:17:49.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.899+0000 7f7d29425740 1 --2- 172.21.15.16:0/1820647860 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x558637927ff0 0x55863792a3e0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:49.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.899+0000 7f7d29425740 1 --2- 172.21.15.16:0/1820647860 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558637918370 0x558637915650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:49.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.899+0000 7f7d29425740 1 --2- 172.21.15.16:0/1820647860 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55863790eb40 0x558637915040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:49.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.899+0000 7f7cfffff640 0 client.29465 ms_handle_reset on v2:172.21.15.184:6836/1793393087 2024-09-12T02:17:49.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.899+0000 7f7d29425740 1 --2- 172.21.15.16:0/1820647860 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55863790d5d0 0x558637917d90 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:49.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.899+0000 7f7d29425740 1 -- 172.21.15.16:0/1820647860 >> 172.21.15.16:0/1820647860 conn(0x5586378d6310 msgr2=0x55863790f990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:49.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.900+0000 7f7d29425740 1 -- 172.21.15.16:0/1820647860 shutdown_connections 2024-09-12T02:17:49.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.900+0000 7f7d29425740 1 -- 172.21.15.16:0/1820647860 wait complete. 2024-09-12T02:17:49.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.900+0000 7f7d29425740 20 client.29465 trim_cache size 0 max 16384 2024-09-12T02:17:49.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.913+0000 7f7d29425740 1 Processor -- start 2024-09-12T02:17:49.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.913+0000 7f7d29425740 1 -- start start 2024-09-12T02:17:49.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.913+0000 7f7d29425740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55863790d5d0 0x558637910180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:49.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.913+0000 7f7d29425740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55863790eb40 0x5586379106c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:49.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.913+0000 7f7d29425740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558637918370 0x55863792fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:49.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.913+0000 7f7d29425740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7d201b1dd0 con 0x55863790eb40 2024-09-12T02:17:49.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.913+0000 7f7d29425740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7d201af6a0 con 0x55863790d5d0 2024-09-12T02:17:49.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.913+0000 7f7d29425740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7d201ad4f0 con 0x558637918370 2024-09-12T02:17:49.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.913+0000 7f7d1ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55863790eb40 0x5586379106c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:49.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.913+0000 7f7d1ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55863790eb40 0x5586379106c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47404/0 (socket says 172.21.15.16:47404) 2024-09-12T02:17:49.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.913+0000 7f7d1ffff640 1 -- 172.21.15.16:0/1744053551 learned_addr learned my addr 172.21.15.16:0/1744053551 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:17:49.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.913+0000 7f7d24de8640 1 --2- 172.21.15.16:0/1744053551 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55863790d5d0 0x558637910180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:49.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.913+0000 7f7d1ffff640 1 -- 172.21.15.16:0/1744053551 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558637918370 msgr2=0x55863792fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:49.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.913+0000 7f7d1ffff640 1 --2- 172.21.15.16:0/1744053551 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558637918370 0x55863792fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:49.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.913+0000 7f7d1ffff640 1 -- 172.21.15.16:0/1744053551 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55863790d5d0 msgr2=0x558637910180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:49.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.913+0000 7f7d1ffff640 1 --2- 172.21.15.16:0/1744053551 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55863790d5d0 0x558637910180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:49.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.913+0000 7f7d1ffff640 1 -- 172.21.15.16:0/1744053551 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586379159c0 con 0x55863790eb40 2024-09-12T02:17:49.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.913+0000 7f7d24de8640 1 --2- 172.21.15.16:0/1744053551 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55863790d5d0 0x558637910180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:17:49.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.914+0000 7f7d1ffff640 1 --2- 172.21.15.16:0/1744053551 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55863790eb40 0x5586379106c0 secure :-1 s=READY pgs=2129 cs=0 l=1 rev1=1 crypto rx=0x7f7d0c099520 tx=0x7f7d0c09b170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a967fceb2e7e17e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:49.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.914+0000 7f7d0a7fc640 1 -- 172.21.15.16:0/1744053551 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d0c020040 con 0x55863790eb40 2024-09-12T02:17:49.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.914+0000 7f7d0a7fc640 1 -- 172.21.15.16:0/1744053551 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d0c005290 con 0x55863790eb40 2024-09-12T02:17:49.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.914+0000 7f7d0a7fc640 1 -- 172.21.15.16:0/1744053551 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d0c0bb110 con 0x55863790eb40 2024-09-12T02:17:49.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.914+0000 7f7d29425740 1 -- 172.21.15.16:0/1744053551 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55863790eb40 msgr2=0x5586379106c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:49.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.914+0000 7f7d29425740 1 --2- 172.21.15.16:0/1744053551 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55863790eb40 0x5586379106c0 secure :-1 s=READY pgs=2129 cs=0 l=1 rev1=1 crypto rx=0x7f7d0c099520 tx=0x7f7d0c09b170 comp rx=0 tx=0).stop 2024-09-12T02:17:49.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.914+0000 7f7d29425740 1 -- 172.21.15.16:0/1744053551 shutdown_connections 2024-09-12T02:17:49.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.914+0000 7f7d29425740 1 --2- 172.21.15.16:0/1744053551 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558637918370 0x55863792fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:49.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.914+0000 7f7d29425740 1 --2- 172.21.15.16:0/1744053551 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55863790eb40 0x5586379106c0 unknown :-1 s=CLOSED pgs=2129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:49.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.914+0000 7f7d29425740 1 --2- 172.21.15.16:0/1744053551 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55863790d5d0 0x558637910180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:49.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.914+0000 7f7d29425740 1 -- 172.21.15.16:0/1744053551 >> 172.21.15.16:0/1744053551 conn(0x5586378d6310 msgr2=0x558637908d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:49.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.914+0000 7f7d29425740 1 -- 172.21.15.16:0/1744053551 shutdown_connections 2024-09-12T02:17:49.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.914+0000 7f7d29425740 1 -- 172.21.15.16:0/1744053551 wait complete. 2024-09-12T02:17:49.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.915+0000 7f7d29425740 1 -- start start 2024-09-12T02:17:49.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.915+0000 7f7d29425740 1 Processor -- start 2024-09-12T02:17:49.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.915+0000 7f7d29425740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55863790d5d0 0x558637920420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:49.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.915+0000 7f7d29425740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55863790eb40 0x558637920960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:49.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.915+0000 7f7d29425740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558637918370 0x55863791d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:49.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.915+0000 7f7d29425740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7d201adb60 con 0x558637918370 2024-09-12T02:17:49.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.915+0000 7f7d29425740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7d201aefd0 con 0x55863790eb40 2024-09-12T02:17:49.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.915+0000 7f7d29425740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7d201ae230 con 0x55863790d5d0 2024-09-12T02:17:49.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.915+0000 7f7d29425740 10 client.0 _ll_register_callbacks cb 0x7f7d14009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:17:49.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.915+0000 7f7d255e9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558637918370 0x55863791d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:49.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.915+0000 7f7d255e9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558637918370 0x55863791d3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47414/0 (socket says 172.21.15.16:47414) 2024-09-12T02:17:49.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.915+0000 7f7d255e9640 1 -- 172.21.15.16:0/1134513290 learned_addr learned my addr 172.21.15.16:0/1134513290 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:17:49.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.916+0000 7f7d1ffff640 1 --2- 172.21.15.16:0/1134513290 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55863790eb40 0x558637920960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:49.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.916+0000 7f7d255e9640 1 -- 172.21.15.16:0/1134513290 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55863790d5d0 msgr2=0x558637920420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:49.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.916+0000 7f7d255e9640 1 --2- 172.21.15.16:0/1134513290 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55863790d5d0 0x558637920420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:49.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.916+0000 7f7d255e9640 1 -- 172.21.15.16:0/1134513290 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55863790eb40 msgr2=0x558637920960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:49.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.916+0000 7f7d255e9640 1 --2- 172.21.15.16:0/1134513290 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55863790eb40 0x558637920960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:49.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.916+0000 7f7d255e9640 1 -- 172.21.15.16:0/1134513290 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55863791d960 con 0x558637918370 2024-09-12T02:17:49.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.916+0000 7f7d1ffff640 1 --2- 172.21.15.16:0/1134513290 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55863790eb40 0x558637920960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:17:49.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.916+0000 7f7d255e9640 1 --2- 172.21.15.16:0/1134513290 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558637918370 0x55863791d3f0 secure :-1 s=READY pgs=2130 cs=0 l=1 rev1=1 crypto rx=0x7f7d18010cb0 tx=0x7f7d18009d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3e2c02387e0fdd server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:49.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.916+0000 7f7cfffff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:17:49.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.916+0000 7f7cfffff640 1 -- 172.21.15.16:0/1134513290 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d18011cf0 con 0x558637918370 2024-09-12T02:17:49.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.916+0000 7f7cfffff640 1 -- 172.21.15.16:0/1134513290 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f7d1802c480 con 0x558637918370 2024-09-12T02:17:49.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.916+0000 7f7d29425740 1 -- 172.21.15.16:0/1134513290 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d0c005290 con 0x558637918370 2024-09-12T02:17:49.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.916+0000 7f7cfffff640 1 -- 172.21.15.16:0/1134513290 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7d18010e60 con 0x558637918370 2024-09-12T02:17:49.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.916+0000 7f7d29425740 10 client.20853 fetch_fsmap learned FSMap version 25 2024-09-12T02:17:49.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.916+0000 7f7d29425740 1 -- 172.21.15.16:0/1134513290 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5586379934d0 con 0x558637918370 2024-09-12T02:17:49.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.917+0000 7f7cfffff640 1 -- 172.21.15.16:0/1134513290 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d0c005290 con 0x558637918370 2024-09-12T02:17:49.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.917+0000 7f7cfffff640 1 -- 172.21.15.16:0/1134513290 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f7d180217a0 con 0x558637918370 2024-09-12T02:17:49.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.917+0000 7f7d29425740 10 client.20853 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:17:49.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.917+0000 7f7d29425740 10 client.20853 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-12T02:17:49.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.917+0000 7f7d29425740 20 client.20853 populate_metadata read hostname 'smithi016' 2024-09-12T02:17:49.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.917+0000 7f7d29425740 1 --2- 172.21.15.16:0/1134513290 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x558637993d00 0x558637996120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:49.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.917+0000 7f7d29425740 4 client.20853 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T02:17:49.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.917+0000 7f7d29425740 1 -- 172.21.15.16:0/1134513290 --> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55863787ad70 con 0x558637993d00 2024-09-12T02:17:49.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.917+0000 7f7d24de8640 1 --2- 172.21.15.16:0/1134513290 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x558637993d00 0x558637996120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:49.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.918+0000 7f7d24de8640 1 --2- 172.21.15.16:0/1134513290 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x558637993d00 0x558637996120 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=521e9bc766917900 server_cookie=b75ba3b8e547c7bf in_seq=0 out_seq=0 2024-09-12T02:17:49.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.918+0000 7f7cfffff640 10 client.20853 ms_handle_connect on v2:172.21.15.184:6836/1793393087 2024-09-12T02:17:49.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.918+0000 7f7cfffff640 1 -- 172.21.15.16:0/1134513290 <== mds.2 v2:172.21.15.184:6836/1793393087 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55863787ad70 con 0x558637993d00 2024-09-12T02:17:49.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.918+0000 7f7cfffff640 10 client.20853 handle_command_reply: tid=0 2024-09-12T02:17:49.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.918+0000 7f7d29425740 1 client.20853 shutdown 2024-09-12T02:17:49.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.918+0000 7f7d29425740 1 -- 172.21.15.16:0/1134513290 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558637918370 msgr2=0x55863791d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:49.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.918+0000 7f7d29425740 1 --2- 172.21.15.16:0/1134513290 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558637918370 0x55863791d3f0 secure :-1 s=READY pgs=2130 cs=0 l=1 rev1=1 crypto rx=0x7f7d18010cb0 tx=0x7f7d18009d80 comp rx=0 tx=0).stop 2024-09-12T02:17:49.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.918+0000 7f7d29425740 1 -- 172.21.15.16:0/1134513290 shutdown_connections 2024-09-12T02:17:49.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.918+0000 7f7d29425740 1 --2- 172.21.15.16:0/1134513290 >> [v2:172.21.15.184:6836/1793393087,v1:172.21.15.184:6837/1793393087] conn(0x558637993d00 0x558637996120 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:49.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.918+0000 7f7d29425740 1 --2- 172.21.15.16:0/1134513290 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558637918370 0x55863791d3f0 unknown :-1 s=CLOSED pgs=2130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:49.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.918+0000 7f7d29425740 1 --2- 172.21.15.16:0/1134513290 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55863790eb40 0x558637920960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:49.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.918+0000 7f7cfffff640 0 client.20853 ms_handle_reset on v2:172.21.15.184:6836/1793393087 2024-09-12T02:17:49.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.918+0000 7f7d29425740 1 --2- 172.21.15.16:0/1134513290 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55863790d5d0 0x558637920420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:49.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.918+0000 7f7d29425740 1 -- 172.21.15.16:0/1134513290 >> 172.21.15.16:0/1134513290 conn(0x5586378d6310 msgr2=0x5586379168a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:49.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.918+0000 7f7d29425740 1 -- 172.21.15.16:0/1134513290 shutdown_connections 2024-09-12T02:17:49.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.918+0000 7f7d29425740 1 -- 172.21.15.16:0/1134513290 wait complete. 2024-09-12T02:17:49.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.918+0000 7f7d29425740 20 client.20853 trim_cache size 0 max 16384 2024-09-12T02:17:49.925 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T02:17:49.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.919+0000 7f7d2704c640 1 -- 172.21.15.16:0/996717621 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f7d100a2c00 msgr2=0x7f7d100a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:49.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.919+0000 7f7d2704c640 1 --2- 172.21.15.16:0/996717621 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f7d100a2c00 0x7f7d100a50c0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f7d18009fd0 tx=0x7f7d18009920 comp rx=0 tx=0).stop 2024-09-12T02:17:49.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.919+0000 7f7d2704c640 1 -- 172.21.15.16:0/996717621 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d201027a0 msgr2=0x7f7d2019abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:49.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.919+0000 7f7d2704c640 1 --2- 172.21.15.16:0/996717621 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d201027a0 0x7f7d2019abe0 secure :-1 s=READY pgs=2127 cs=0 l=1 rev1=1 crypto rx=0x7f7d0c00d770 tx=0x7f7d0c00dc30 comp rx=0 tx=0).stop 2024-09-12T02:17:49.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.919+0000 7f7d2704c640 1 -- 172.21.15.16:0/996717621 shutdown_connections 2024-09-12T02:17:49.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.919+0000 7f7d2704c640 1 --2- 172.21.15.16:0/996717621 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f7d100a2c00 0x7f7d100a50c0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:49.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.919+0000 7f7d2704c640 1 --2- 172.21.15.16:0/996717621 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7d20106f20 0x7f7d2019ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:49.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.919+0000 7f7d2704c640 1 --2- 172.21.15.16:0/996717621 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d201027a0 0x7f7d2019abe0 unknown :-1 s=CLOSED pgs=2127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:49.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.919+0000 7f7d2704c640 1 --2- 172.21.15.16:0/996717621 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d20101df0 0x7f7d2019a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:49.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.919+0000 7f7d2704c640 1 -- 172.21.15.16:0/996717621 >> 172.21.15.16:0/996717621 conn(0x7f7d200fd8b0 msgr2=0x7f7d200febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:49.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.919+0000 7f7d2704c640 1 -- 172.21.15.16:0/996717621 shutdown_connections 2024-09-12T02:17:49.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:49.919+0000 7f7d2704c640 1 -- 172.21.15.16:0/996717621 wait complete. 2024-09-12T02:17:50.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:17:49 smithi184 ceph-mon[35593]: pgmap v812: 97 pgs: 97 active+clean; 4.3 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.8 KiB/s wr, 13 op/s 2024-09-12T02:17:50.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:17:49 smithi033 ceph-mon[35332]: pgmap v812: 97 pgs: 97 active+clean; 4.3 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.8 KiB/s wr, 13 op/s 2024-09-12T02:17:50.781 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-12T02:17:51.040 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:17:51.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.471+0000 7f370d209640 1 -- 172.21.15.16:0/3286133555 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3708102570 msgr2=0x7f37081029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:51.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.471+0000 7f370d209640 1 --2- 172.21.15.16:0/3286133555 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3708102570 0x7f37081029b0 secure :-1 s=READY pgs=2131 cs=0 l=1 rev1=1 crypto rx=0x7f36f0009900 tx=0x7f36f002a480 comp rx=0 tx=0).stop 2024-09-12T02:17:51.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.471+0000 7f370d209640 1 -- 172.21.15.16:0/3286133555 shutdown_connections 2024-09-12T02:17:51.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.471+0000 7f370d209640 1 --2- 172.21.15.16:0/3286133555 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3708102ef0 0x7f370810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:51.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.471+0000 7f370d209640 1 --2- 172.21.15.16:0/3286133555 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3708102570 0x7f37081029b0 unknown :-1 s=CLOSED pgs=2131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:51.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.471+0000 7f370d209640 1 --2- 172.21.15.16:0/3286133555 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3708100e20 0x7f3708101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:51.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.471+0000 7f370d209640 1 -- 172.21.15.16:0/3286133555 >> 172.21.15.16:0/3286133555 conn(0x7f37080fc7d0 msgr2=0x7f37080febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:51.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.471+0000 7f370d209640 1 -- 172.21.15.16:0/3286133555 shutdown_connections 2024-09-12T02:17:51.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.471+0000 7f370d209640 1 -- 172.21.15.16:0/3286133555 wait complete. 2024-09-12T02:17:51.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.472+0000 7f370d209640 1 Processor -- start 2024-09-12T02:17:51.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.473+0000 7f370d209640 1 -- start start 2024-09-12T02:17:51.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.473+0000 7f370d209640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3708100e20 0x7f3708196280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:51.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.473+0000 7f370d209640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3708102570 0x7f37081967c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:51.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.473+0000 7f370d209640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3708102ef0 0x7f370819ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:51.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.473+0000 7f370d209640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f370810fe80 con 0x7f3708100e20 2024-09-12T02:17:51.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.473+0000 7f370d209640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f370810fd00 con 0x7f3708102570 2024-09-12T02:17:51.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.473+0000 7f370d209640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f3708110000 con 0x7f3708102ef0 2024-09-12T02:17:51.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.473+0000 7f3706d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3708100e20 0x7f3708196280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:51.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.473+0000 7f3706d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3708100e20 0x7f3708196280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54472/0 (socket says 172.21.15.16:54472) 2024-09-12T02:17:51.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.473+0000 7f3706d9d640 1 -- 172.21.15.16:0/670976303 learned_addr learned my addr 172.21.15.16:0/670976303 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:17:51.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.473+0000 7f370759e640 1 --2- 172.21.15.16:0/670976303 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3708102ef0 0x7f370819ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:51.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.473+0000 7f36fffff640 1 --2- 172.21.15.16:0/670976303 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3708102570 0x7f37081967c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:51.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.473+0000 7f3706d9d640 1 -- 172.21.15.16:0/670976303 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3708102ef0 msgr2=0x7f370819ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:51.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.473+0000 7f3706d9d640 1 --2- 172.21.15.16:0/670976303 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3708102ef0 0x7f370819ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:51.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.473+0000 7f3706d9d640 1 -- 172.21.15.16:0/670976303 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3708102570 msgr2=0x7f37081967c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:51.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.473+0000 7f3706d9d640 1 --2- 172.21.15.16:0/670976303 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3708102570 0x7f37081967c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:51.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.473+0000 7f3706d9d640 1 -- 172.21.15.16:0/670976303 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f370819b230 con 0x7f3708100e20 2024-09-12T02:17:51.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.474+0000 7f3706d9d640 1 --2- 172.21.15.16:0/670976303 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3708100e20 0x7f3708196280 secure :-1 s=READY pgs=2132 cs=0 l=1 rev1=1 crypto rx=0x7f36f800c920 tx=0x7f36f800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:51.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.474+0000 7f3704d99640 1 -- 172.21.15.16:0/670976303 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36f8007c70 con 0x7f3708100e20 2024-09-12T02:17:51.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.474+0000 7f3704d99640 1 -- 172.21.15.16:0/670976303 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36f8007e10 con 0x7f3708100e20 2024-09-12T02:17:51.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.474+0000 7f370d209640 1 -- 172.21.15.16:0/670976303 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f370819b520 con 0x7f3708100e20 2024-09-12T02:17:51.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.474+0000 7f3704d99640 1 -- 172.21.15.16:0/670976303 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36f8018890 con 0x7f3708100e20 2024-09-12T02:17:51.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.474+0000 7f370d209640 1 -- 172.21.15.16:0/670976303 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37081a2e00 con 0x7f3708100e20 2024-09-12T02:17:51.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.476+0000 7f3704d99640 1 -- 172.21.15.16:0/670976303 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f36f8018a70 con 0x7f3708100e20 2024-09-12T02:17:51.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.477+0000 7f370f5e2740 1 Processor -- start 2024-09-12T02:17:51.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.477+0000 7f370f5e2740 1 -- start start 2024-09-12T02:17:51.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.477+0000 7f370f5e2740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d85d8a7b40 0x55d85d8a6090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:51.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.477+0000 7f370f5e2740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d85d8a65d0 0x55d85d8a6a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:51.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.477+0000 7f370f5e2740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d85d8b1370 0x55d85d8b3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:51.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.477+0000 7f370f5e2740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55d85d80c2f0 con 0x55d85d8a7b40 2024-09-12T02:17:51.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.477+0000 7f370f5e2740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55d85d7b3cf0 con 0x55d85d8b1370 2024-09-12T02:17:51.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.477+0000 7f370f5e2740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55d85d8082b0 con 0x55d85d8a65d0 2024-09-12T02:17:51.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.477+0000 7f370759e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d85d8a7b40 0x55d85d8a6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:51.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.477+0000 7f370759e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d85d8a7b40 0x55d85d8a6090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54488/0 (socket says 172.21.15.16:54488) 2024-09-12T02:17:51.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.477+0000 7f370759e640 1 -- 172.21.15.16:0/3313860187 learned_addr learned my addr 172.21.15.16:0/3313860187 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:17:51.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.478+0000 7f3706d9d640 1 --2- 172.21.15.16:0/3313860187 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d85d8a65d0 0x55d85d8a6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:51.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.478+0000 7f36fffff640 1 --2- 172.21.15.16:0/3313860187 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d85d8b1370 0x55d85d8b3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:51.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.478+0000 7f370759e640 1 -- 172.21.15.16:0/3313860187 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d85d8a65d0 msgr2=0x55d85d8a6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:51.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.478+0000 7f370759e640 1 --2- 172.21.15.16:0/3313860187 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d85d8a65d0 0x55d85d8a6a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:51.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.478+0000 7f370759e640 1 -- 172.21.15.16:0/3313860187 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d85d8b1370 msgr2=0x55d85d8b3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:51.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.478+0000 7f370759e640 1 --2- 172.21.15.16:0/3313860187 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d85d8b1370 0x55d85d8b3760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:51.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.478+0000 7f3704d99640 1 --2- 172.21.15.16:0/670976303 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f36f40a6f90 0x7f36f40a9450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:51.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.478+0000 7f370759e640 1 -- 172.21.15.16:0/3313860187 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d85d7983a0 con 0x55d85d8a7b40 2024-09-12T02:17:51.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.478+0000 7f36fffff640 1 --2- 172.21.15.16:0/3313860187 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d85d8b1370 0x55d85d8b3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:17:51.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.478+0000 7f3706d9d640 1 --2- 172.21.15.16:0/670976303 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f36f40a6f90 0x7f36f40a9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:51.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.478+0000 7f3704d99640 1 -- 172.21.15.16:0/670976303 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f36f80cf040 con 0x7f3708100e20 2024-09-12T02:17:51.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.478+0000 7f370759e640 1 --2- 172.21.15.16:0/3313860187 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d85d8a7b40 0x55d85d8a6090 secure :-1 s=READY pgs=2133 cs=0 l=1 rev1=1 crypto rx=0x7f370000ae90 tx=0x7f370000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9768472d209300db server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:51.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.478+0000 7f36fd7fa640 1 -- 172.21.15.16:0/3313860187 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3700003b70 con 0x55d85d8a7b40 2024-09-12T02:17:51.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.478+0000 7f36fd7fa640 1 -- 172.21.15.16:0/3313860187 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3700003d10 con 0x55d85d8a7b40 2024-09-12T02:17:51.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.478+0000 7f36fd7fa640 1 -- 172.21.15.16:0/3313860187 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3700010ca0 con 0x55d85d8a7b40 2024-09-12T02:17:51.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.479+0000 7f370f5e2740 1 -- 172.21.15.16:0/3313860187 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d85d8a7b40 msgr2=0x55d85d8a6090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:51.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.479+0000 7f3706d9d640 1 --2- 172.21.15.16:0/670976303 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f36f40a6f90 0x7f36f40a9450 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f36f8097da0 tx=0x7f36f800ff30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:51.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.479+0000 7f370f5e2740 1 --2- 172.21.15.16:0/3313860187 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d85d8a7b40 0x55d85d8a6090 secure :-1 s=READY pgs=2133 cs=0 l=1 rev1=1 crypto rx=0x7f370000ae90 tx=0x7f370000f790 comp rx=0 tx=0).stop 2024-09-12T02:17:51.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.479+0000 7f370f5e2740 1 -- 172.21.15.16:0/3313860187 shutdown_connections 2024-09-12T02:17:51.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.479+0000 7f370f5e2740 1 --2- 172.21.15.16:0/3313860187 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d85d8b1370 0x55d85d8b3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:51.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.479+0000 7f370f5e2740 1 --2- 172.21.15.16:0/3313860187 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d85d8a65d0 0x55d85d8a6a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:51.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.479+0000 7f370f5e2740 1 --2- 172.21.15.16:0/3313860187 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d85d8a7b40 0x55d85d8a6090 unknown :-1 s=CLOSED pgs=2133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:51.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.479+0000 7f370f5e2740 1 -- 172.21.15.16:0/3313860187 >> 172.21.15.16:0/3313860187 conn(0x55d85d86f310 msgr2=0x55d85d8a2c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:51.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.479+0000 7f370f5e2740 1 -- 172.21.15.16:0/3313860187 shutdown_connections 2024-09-12T02:17:51.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.479+0000 7f370f5e2740 1 -- 172.21.15.16:0/3313860187 wait complete. 2024-09-12T02:17:51.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.480+0000 7f370f5e2740 1 -- start start 2024-09-12T02:17:51.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.480+0000 7f370f5e2740 1 Processor -- start 2024-09-12T02:17:51.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.480+0000 7f370f5e2740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d85d8a65d0 0x55d85d8b0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:51.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.480+0000 7f370f5e2740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d85d8a7b40 0x55d85d8adf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:51.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.480+0000 7f370f5e2740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d85d8b1370 0x55d85d8ae4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:51.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.480+0000 7f370f5e2740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55d85d8b4070 con 0x55d85d8a7b40 2024-09-12T02:17:51.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.480+0000 7f370f5e2740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55d85d7c7060 con 0x55d85d8a65d0 2024-09-12T02:17:51.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.480+0000 7f370f5e2740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55d85d8b41f0 con 0x55d85d8b1370 2024-09-12T02:17:51.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.480+0000 7f370f5e2740 10 client.0 _ll_register_callbacks cb 0x55d85d51a8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:17:51.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.480+0000 7f36fffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d85d8a7b40 0x55d85d8adf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:51.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.480+0000 7f36fffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d85d8a7b40 0x55d85d8adf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54496/0 (socket says 172.21.15.16:54496) 2024-09-12T02:17:51.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.480+0000 7f36fffff640 1 -- 172.21.15.16:0/3712899479 learned_addr learned my addr 172.21.15.16:0/3712899479 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:17:51.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.480+0000 7f370759e640 1 --2- 172.21.15.16:0/3712899479 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d85d8a65d0 0x55d85d8b0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:51.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.480+0000 7f370759e640 1 --2- 172.21.15.16:0/3712899479 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d85d8b1370 0x55d85d8ae4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:51.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.480+0000 7f36fffff640 1 -- 172.21.15.16:0/3712899479 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d85d8b1370 msgr2=0x55d85d8ae4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:51.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.480+0000 7f36fffff640 1 --2- 172.21.15.16:0/3712899479 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d85d8b1370 0x55d85d8ae4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:51.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.480+0000 7f36fffff640 1 -- 172.21.15.16:0/3712899479 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d85d8a65d0 msgr2=0x55d85d8b0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:51.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.480+0000 7f36fffff640 1 --2- 172.21.15.16:0/3712899479 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d85d8a65d0 0x55d85d8b0cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:51.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.481+0000 7f36fffff640 1 -- 172.21.15.16:0/3712899479 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d85d57d3c0 con 0x55d85d8a7b40 2024-09-12T02:17:51.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.481+0000 7f36fffff640 1 --2- 172.21.15.16:0/3712899479 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d85d8a7b40 0x55d85d8adf80 secure :-1 s=READY pgs=2134 cs=0 l=1 rev1=1 crypto rx=0x7f36f0009900 tx=0x7f36f00044f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e8b05a5b8155e08 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:51.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.481+0000 7f36ee7fc640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:17:51.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.481+0000 7f36ee7fc640 1 -- 172.21.15.16:0/3712899479 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36f0035070 con 0x55d85d8a7b40 2024-09-12T02:17:51.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.481+0000 7f370f5e2740 1 -- 172.21.15.16:0/3712899479 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3700003d10 con 0x55d85d8a7b40 2024-09-12T02:17:51.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.481+0000 7f36ee7fc640 1 -- 172.21.15.16:0/3712899479 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f36f0003b40 con 0x55d85d8a7b40 2024-09-12T02:17:51.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.481+0000 7f36ee7fc640 1 -- 172.21.15.16:0/3712899479 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f36f0003d40 con 0x55d85d8a7b40 2024-09-12T02:17:51.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.481+0000 7f370f5e2740 10 client.20865 fetch_fsmap learned FSMap version 25 2024-09-12T02:17:51.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.481+0000 7f370f5e2740 1 -- 172.21.15.16:0/3712899479 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d85d8bfad0 con 0x55d85d8a7b40 2024-09-12T02:17:51.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.481+0000 7f36ee7fc640 1 -- 172.21.15.16:0/3712899479 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3700003d10 con 0x55d85d8a7b40 2024-09-12T02:17:51.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.482+0000 7f36ee7fc640 1 -- 172.21.15.16:0/3712899479 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f36f0045710 con 0x55d85d8a7b40 2024-09-12T02:17:51.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.482+0000 7f370f5e2740 10 client.20865 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:17:51.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.482+0000 7f370f5e2740 10 client.20865 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T02:17:51.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.482+0000 7f370f5e2740 20 client.20865 populate_metadata read hostname 'smithi016' 2024-09-12T02:17:51.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.482+0000 7f370f5e2740 1 --2- 172.21.15.16:0/3712899479 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x55d85d8c13a0 0x55d85d8c37e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:51.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.482+0000 7f370f5e2740 4 client.20865 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:17:51.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.482+0000 7f370f5e2740 1 -- 172.21.15.16:0/3712899479 --> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d85d819860 con 0x55d85d8c13a0 2024-09-12T02:17:51.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.482+0000 7f370759e640 1 --2- 172.21.15.16:0/3712899479 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x55d85d8c13a0 0x55d85d8c37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:51.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.482+0000 7f36ee7fc640 10 client.20865 ms_handle_connect on v2:172.21.15.184:6838/4203347941 2024-09-12T02:17:51.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.482+0000 7f370759e640 1 --2- 172.21.15.16:0/3712899479 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x55d85d8c13a0 0x55d85d8c37e0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3060d6230992840a server_cookie=98d142b8c1a45cc6 in_seq=0 out_seq=0 2024-09-12T02:17:51.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.483+0000 7f36ee7fc640 1 -- 172.21.15.16:0/3712899479 <== mds.1 v2:172.21.15.184:6838/4203347941 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d85d819860 con 0x55d85d8c13a0 2024-09-12T02:17:51.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.483+0000 7f36ee7fc640 10 client.20865 handle_command_reply: tid=0 2024-09-12T02:17:51.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.483+0000 7f370f5e2740 1 client.20865 shutdown 2024-09-12T02:17:51.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.483+0000 7f370f5e2740 1 -- 172.21.15.16:0/3712899479 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d85d8a7b40 msgr2=0x55d85d8adf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:51.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.483+0000 7f370f5e2740 1 --2- 172.21.15.16:0/3712899479 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d85d8a7b40 0x55d85d8adf80 secure :-1 s=READY pgs=2134 cs=0 l=1 rev1=1 crypto rx=0x7f36f0009900 tx=0x7f36f00044f0 comp rx=0 tx=0).stop 2024-09-12T02:17:51.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.483+0000 7f370f5e2740 1 -- 172.21.15.16:0/3712899479 shutdown_connections 2024-09-12T02:17:51.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.483+0000 7f370f5e2740 1 --2- 172.21.15.16:0/3712899479 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x55d85d8c13a0 0x55d85d8c37e0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:51.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.483+0000 7f370f5e2740 1 --2- 172.21.15.16:0/3712899479 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d85d8b1370 0x55d85d8ae4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:51.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.483+0000 7f370f5e2740 1 --2- 172.21.15.16:0/3712899479 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d85d8a7b40 0x55d85d8adf80 unknown :-1 s=CLOSED pgs=2134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:51.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.483+0000 7f370f5e2740 1 --2- 172.21.15.16:0/3712899479 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d85d8a65d0 0x55d85d8b0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:51.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.483+0000 7f36ee7fc640 0 client.20865 ms_handle_reset on v2:172.21.15.184:6838/4203347941 2024-09-12T02:17:51.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.483+0000 7f370f5e2740 1 -- 172.21.15.16:0/3712899479 >> 172.21.15.16:0/3712899479 conn(0x55d85d86f310 msgr2=0x55d85d8a8960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:51.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.483+0000 7f370f5e2740 1 -- 172.21.15.16:0/3712899479 shutdown_connections 2024-09-12T02:17:51.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.483+0000 7f370f5e2740 1 -- 172.21.15.16:0/3712899479 wait complete. 2024-09-12T02:17:51.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.484+0000 7f370f5e2740 20 client.20865 trim_cache size 0 max 16384 2024-09-12T02:17:51.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.497+0000 7f370f5e2740 1 Processor -- start 2024-09-12T02:17:51.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.497+0000 7f370f5e2740 1 -- start start 2024-09-12T02:17:51.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.497+0000 7f370f5e2740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d85d8a65d0 0x55d85d8bd310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:51.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.497+0000 7f370f5e2740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d85d8a7b40 0x55d85d8ad840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:51.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.497+0000 7f370f5e2740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d85d8b1370 0x55d85d8add80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:51.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.497+0000 7f370f5e2740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f37081ada10 con 0x55d85d8b1370 2024-09-12T02:17:51.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.497+0000 7f370f5e2740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f37081ab2e0 con 0x55d85d8a65d0 2024-09-12T02:17:51.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.497+0000 7f370f5e2740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f37081a9130 con 0x55d85d8a7b40 2024-09-12T02:17:51.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.497+0000 7f370759e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d85d8b1370 0x55d85d8add80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:51.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.497+0000 7f370759e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d85d8b1370 0x55d85d8add80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54508/0 (socket says 172.21.15.16:54508) 2024-09-12T02:17:51.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.497+0000 7f370759e640 1 -- 172.21.15.16:0/182846901 learned_addr learned my addr 172.21.15.16:0/182846901 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:17:51.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.497+0000 7f36fffff640 1 --2- 172.21.15.16:0/182846901 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d85d8a7b40 0x55d85d8ad840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:51.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.497+0000 7f370759e640 1 --2- 172.21.15.16:0/182846901 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d85d8a65d0 0x55d85d8bd310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:51.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.497+0000 7f370759e640 1 -- 172.21.15.16:0/182846901 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d85d8a7b40 msgr2=0x55d85d8ad840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:51.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.497+0000 7f370759e640 1 --2- 172.21.15.16:0/182846901 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d85d8a7b40 0x55d85d8ad840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:51.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.497+0000 7f370759e640 1 -- 172.21.15.16:0/182846901 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d85d8a65d0 msgr2=0x55d85d8bd310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:51.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.497+0000 7f370759e640 1 --2- 172.21.15.16:0/182846901 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d85d8a65d0 0x55d85d8bd310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:51.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.497+0000 7f370759e640 1 -- 172.21.15.16:0/182846901 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d85d8ae850 con 0x55d85d8b1370 2024-09-12T02:17:51.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.498+0000 7f370759e640 1 --2- 172.21.15.16:0/182846901 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d85d8b1370 0x55d85d8add80 secure :-1 s=READY pgs=2135 cs=0 l=1 rev1=1 crypto rx=0x7f370000fc90 tx=0x7f370001a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c171ba2c1d27aba2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:51.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.498+0000 7f36fcff9640 1 -- 172.21.15.16:0/182846901 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37000109e0 con 0x55d85d8b1370 2024-09-12T02:17:51.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.498+0000 7f36fcff9640 1 -- 172.21.15.16:0/182846901 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3700010b80 con 0x55d85d8b1370 2024-09-12T02:17:51.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.498+0000 7f36fcff9640 1 -- 172.21.15.16:0/182846901 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3700005830 con 0x55d85d8b1370 2024-09-12T02:17:51.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.498+0000 7f370f5e2740 1 -- 172.21.15.16:0/182846901 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d85d8b1370 msgr2=0x55d85d8add80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:51.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.498+0000 7f370f5e2740 1 --2- 172.21.15.16:0/182846901 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d85d8b1370 0x55d85d8add80 secure :-1 s=READY pgs=2135 cs=0 l=1 rev1=1 crypto rx=0x7f370000fc90 tx=0x7f370001a040 comp rx=0 tx=0).stop 2024-09-12T02:17:51.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.498+0000 7f370f5e2740 1 -- 172.21.15.16:0/182846901 shutdown_connections 2024-09-12T02:17:51.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.498+0000 7f370f5e2740 1 --2- 172.21.15.16:0/182846901 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d85d8b1370 0x55d85d8add80 unknown :-1 s=CLOSED pgs=2135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:51.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.499+0000 7f370f5e2740 1 --2- 172.21.15.16:0/182846901 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d85d8a7b40 0x55d85d8ad840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:51.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.499+0000 7f370f5e2740 1 --2- 172.21.15.16:0/182846901 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d85d8a65d0 0x55d85d8bd310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:51.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.499+0000 7f370f5e2740 1 -- 172.21.15.16:0/182846901 >> 172.21.15.16:0/182846901 conn(0x55d85d86f310 msgr2=0x55d85d8a9310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:51.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.499+0000 7f370f5e2740 1 -- 172.21.15.16:0/182846901 shutdown_connections 2024-09-12T02:17:51.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.499+0000 7f370f5e2740 1 -- 172.21.15.16:0/182846901 wait complete. 2024-09-12T02:17:51.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.499+0000 7f370f5e2740 1 -- start start 2024-09-12T02:17:51.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.499+0000 7f370f5e2740 1 Processor -- start 2024-09-12T02:17:51.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.499+0000 7f370f5e2740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d85d8a65d0 0x55d85d8b9460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:51.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.499+0000 7f370f5e2740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d85d8a7b40 0x55d85d8b99a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:51.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.499+0000 7f370f5e2740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d85d8b1370 0x55d85d8b49e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:51.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.499+0000 7f370f5e2740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f37081a97a0 con 0x55d85d8b1370 2024-09-12T02:17:51.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.499+0000 7f370f5e2740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f37081aac10 con 0x55d85d8a65d0 2024-09-12T02:17:51.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.499+0000 7f370f5e2740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f37081a9e70 con 0x55d85d8a7b40 2024-09-12T02:17:51.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.499+0000 7f370f5e2740 10 client.0 _ll_register_callbacks cb 0x7f36f0009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:17:51.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.499+0000 7f370759e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d85d8b1370 0x55d85d8b49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:51.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.499+0000 7f370759e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d85d8b1370 0x55d85d8b49e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54516/0 (socket says 172.21.15.16:54516) 2024-09-12T02:17:51.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.499+0000 7f370759e640 1 -- 172.21.15.16:0/3986599554 learned_addr learned my addr 172.21.15.16:0/3986599554 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:17:51.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.500+0000 7f370759e640 1 --2- 172.21.15.16:0/3986599554 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d85d8a65d0 0x55d85d8b9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:51.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.500+0000 7f36fffff640 1 --2- 172.21.15.16:0/3986599554 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d85d8a7b40 0x55d85d8b99a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:51.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.500+0000 7f370759e640 1 -- 172.21.15.16:0/3986599554 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d85d8a7b40 msgr2=0x55d85d8b99a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:51.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.500+0000 7f370759e640 1 --2- 172.21.15.16:0/3986599554 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d85d8a7b40 0x55d85d8b99a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:51.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.500+0000 7f370759e640 1 -- 172.21.15.16:0/3986599554 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d85d8a65d0 msgr2=0x55d85d8b9460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:51.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.500+0000 7f370759e640 1 --2- 172.21.15.16:0/3986599554 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d85d8a65d0 0x55d85d8b9460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:51.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.500+0000 7f370759e640 1 -- 172.21.15.16:0/3986599554 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d85d8b51c0 con 0x55d85d8b1370 2024-09-12T02:17:51.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.500+0000 7f370759e640 1 --2- 172.21.15.16:0/3986599554 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d85d8b1370 0x55d85d8b49e0 secure :-1 s=READY pgs=2136 cs=0 l=1 rev1=1 crypto rx=0x7f370000a750 tx=0x7f3700028040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57d51ba3f0c04a19 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:51.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.500+0000 7f36ee7fc640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:17:51.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.500+0000 7f36ee7fc640 1 -- 172.21.15.16:0/3986599554 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37000055b0 con 0x55d85d8b1370 2024-09-12T02:17:51.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.500+0000 7f36ee7fc640 1 -- 172.21.15.16:0/3986599554 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f3700005750 con 0x55d85d8b1370 2024-09-12T02:17:51.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.500+0000 7f370f5e2740 1 -- 172.21.15.16:0/3986599554 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3700010b80 con 0x55d85d8b1370 2024-09-12T02:17:51.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.500+0000 7f36ee7fc640 1 -- 172.21.15.16:0/3986599554 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f3700005950 con 0x55d85d8b1370 2024-09-12T02:17:51.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.500+0000 7f370f5e2740 10 client.20871 fetch_fsmap learned FSMap version 25 2024-09-12T02:17:51.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.500+0000 7f370f5e2740 1 -- 172.21.15.16:0/3986599554 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d85d92bb00 con 0x55d85d8b1370 2024-09-12T02:17:51.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.501+0000 7f36ee7fc640 1 -- 172.21.15.16:0/3986599554 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3700010b80 con 0x55d85d8b1370 2024-09-12T02:17:51.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.501+0000 7f36ee7fc640 1 -- 172.21.15.16:0/3986599554 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f37000263f0 con 0x55d85d8b1370 2024-09-12T02:17:51.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.501+0000 7f370f5e2740 10 client.20871 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:17:51.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.501+0000 7f370f5e2740 10 client.20871 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T02:17:51.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.501+0000 7f370f5e2740 20 client.20871 populate_metadata read hostname 'smithi016' 2024-09-12T02:17:51.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.501+0000 7f370f5e2740 1 --2- 172.21.15.16:0/3986599554 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x55d85d92bf70 0x55d85d92e360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:51.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.501+0000 7f370f5e2740 4 client.20871 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T02:17:51.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.501+0000 7f370f5e2740 1 -- 172.21.15.16:0/3986599554 --> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d85d813d70 con 0x55d85d92bf70 2024-09-12T02:17:51.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.501+0000 7f36fffff640 1 --2- 172.21.15.16:0/3986599554 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x55d85d92bf70 0x55d85d92e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:51.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.501+0000 7f36fffff640 1 --2- 172.21.15.16:0/3986599554 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x55d85d92bf70 0x55d85d92e360 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fe4fd4aedcecde95 server_cookie=fa6ff358ade6b5c0 in_seq=0 out_seq=0 2024-09-12T02:17:51.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.501+0000 7f36ee7fc640 10 client.20871 ms_handle_connect on v2:172.21.15.184:6838/4203347941 2024-09-12T02:17:51.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.502+0000 7f36ee7fc640 1 -- 172.21.15.16:0/3986599554 <== mds.1 v2:172.21.15.184:6838/4203347941 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d85d813d70 con 0x55d85d92bf70 2024-09-12T02:17:51.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.502+0000 7f36ee7fc640 10 client.20871 handle_command_reply: tid=0 2024-09-12T02:17:51.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.502+0000 7f370f5e2740 1 client.20871 shutdown 2024-09-12T02:17:51.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.502+0000 7f370f5e2740 1 -- 172.21.15.16:0/3986599554 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d85d8b1370 msgr2=0x55d85d8b49e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:51.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.502+0000 7f370f5e2740 1 --2- 172.21.15.16:0/3986599554 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d85d8b1370 0x55d85d8b49e0 secure :-1 s=READY pgs=2136 cs=0 l=1 rev1=1 crypto rx=0x7f370000a750 tx=0x7f3700028040 comp rx=0 tx=0).stop 2024-09-12T02:17:51.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.502+0000 7f370f5e2740 1 -- 172.21.15.16:0/3986599554 shutdown_connections 2024-09-12T02:17:51.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.502+0000 7f370f5e2740 1 --2- 172.21.15.16:0/3986599554 >> [v2:172.21.15.184:6838/4203347941,v1:172.21.15.184:6839/4203347941] conn(0x55d85d92bf70 0x55d85d92e360 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:51.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.502+0000 7f370f5e2740 1 --2- 172.21.15.16:0/3986599554 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d85d8b1370 0x55d85d8b49e0 unknown :-1 s=CLOSED pgs=2136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:51.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.502+0000 7f370f5e2740 1 --2- 172.21.15.16:0/3986599554 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d85d8a7b40 0x55d85d8b99a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:51.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.502+0000 7f370f5e2740 1 --2- 172.21.15.16:0/3986599554 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d85d8a65d0 0x55d85d8b9460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:51.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.502+0000 7f370f5e2740 1 -- 172.21.15.16:0/3986599554 >> 172.21.15.16:0/3986599554 conn(0x55d85d86f310 msgr2=0x55d85d8af7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:51.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.502+0000 7f36ee7fc640 0 client.20871 ms_handle_reset on v2:172.21.15.184:6838/4203347941 2024-09-12T02:17:51.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.502+0000 7f370f5e2740 1 -- 172.21.15.16:0/3986599554 shutdown_connections 2024-09-12T02:17:51.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.502+0000 7f370f5e2740 1 -- 172.21.15.16:0/3986599554 wait complete. 2024-09-12T02:17:51.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.502+0000 7f370f5e2740 20 client.20871 trim_cache size 0 max 16384 2024-09-12T02:17:51.511 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T02:17:51.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.502+0000 7f370d209640 1 -- 172.21.15.16:0/670976303 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f36f40a6f90 msgr2=0x7f36f40a9450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:51.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.503+0000 7f370d209640 1 --2- 172.21.15.16:0/670976303 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f36f40a6f90 0x7f36f40a9450 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f36f8097da0 tx=0x7f36f800ff30 comp rx=0 tx=0).stop 2024-09-12T02:17:51.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.503+0000 7f370d209640 1 -- 172.21.15.16:0/670976303 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3708100e20 msgr2=0x7f3708196280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:51.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.503+0000 7f370d209640 1 --2- 172.21.15.16:0/670976303 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3708100e20 0x7f3708196280 secure :-1 s=READY pgs=2132 cs=0 l=1 rev1=1 crypto rx=0x7f36f800c920 tx=0x7f36f800cde0 comp rx=0 tx=0).stop 2024-09-12T02:17:51.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.503+0000 7f370d209640 1 -- 172.21.15.16:0/670976303 shutdown_connections 2024-09-12T02:17:51.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.503+0000 7f370d209640 1 --2- 172.21.15.16:0/670976303 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f36f40a6f90 0x7f36f40a9450 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:51.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.503+0000 7f370d209640 1 --2- 172.21.15.16:0/670976303 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3708102ef0 0x7f370819ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:51.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.503+0000 7f370d209640 1 --2- 172.21.15.16:0/670976303 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3708102570 0x7f37081967c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:51.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.503+0000 7f370d209640 1 --2- 172.21.15.16:0/670976303 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3708100e20 0x7f3708196280 unknown :-1 s=CLOSED pgs=2132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:51.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.503+0000 7f370d209640 1 -- 172.21.15.16:0/670976303 >> 172.21.15.16:0/670976303 conn(0x7f37080fc7d0 msgr2=0x7f3708106830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:51.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.503+0000 7f370d209640 1 -- 172.21.15.16:0/670976303 shutdown_connections 2024-09-12T02:17:51.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:51.503+0000 7f370d209640 1 -- 172.21.15.16:0/670976303 wait complete. 2024-09-12T02:17:52.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:17:51 smithi184 ceph-mon[35593]: pgmap v813: 97 pgs: 97 active+clean; 4.3 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.8 KiB/s wr, 14 op/s 2024-09-12T02:17:52.168 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:17:51 smithi016 ceph-mon[30947]: pgmap v813: 97 pgs: 97 active+clean; 4.3 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.8 KiB/s wr, 14 op/s 2024-09-12T02:17:52.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:17:51 smithi033 ceph-mon[35332]: pgmap v813: 97 pgs: 97 active+clean; 4.3 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.8 KiB/s wr, 14 op/s 2024-09-12T02:17:52.199 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-12T02:17:52.457 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:17:52.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.953+0000 7f663c5e9640 1 -- 172.21.15.16:0/3286684788 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6634101de0 msgr2=0x7f66341021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:52.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.953+0000 7f663c5e9640 1 --2- 172.21.15.16:0/3286684788 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6634101de0 0x7f66341021c0 secure :-1 s=READY pgs=2137 cs=0 l=1 rev1=1 crypto rx=0x7f6630009900 tx=0x7f663002a410 comp rx=0 tx=0).stop 2024-09-12T02:17:52.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.953+0000 7f663c5e9640 1 -- 172.21.15.16:0/3286684788 shutdown_connections 2024-09-12T02:17:52.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.953+0000 7f663c5e9640 1 --2- 172.21.15.16:0/3286684788 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6634106f10 0x7f663410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:52.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.953+0000 7f663c5e9640 1 --2- 172.21.15.16:0/3286684788 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6634102790 0x7f66341067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:52.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.953+0000 7f663c5e9640 1 --2- 172.21.15.16:0/3286684788 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6634101de0 0x7f66341021c0 unknown :-1 s=CLOSED pgs=2137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:52.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.953+0000 7f663c5e9640 1 -- 172.21.15.16:0/3286684788 >> 172.21.15.16:0/3286684788 conn(0x7f66340fd8c0 msgr2=0x7f66340ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:52.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.953+0000 7f663c5e9640 1 -- 172.21.15.16:0/3286684788 shutdown_connections 2024-09-12T02:17:52.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.953+0000 7f663c5e9640 1 -- 172.21.15.16:0/3286684788 wait complete. 2024-09-12T02:17:52.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.954+0000 7f663c5e9640 1 Processor -- start 2024-09-12T02:17:52.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.954+0000 7f663c5e9640 1 -- start start 2024-09-12T02:17:52.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.954+0000 7f663c5e9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6634101de0 0x7f6634198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:52.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.954+0000 7f663c5e9640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6634102790 0x7f6634198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:52.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.954+0000 7f663c5e9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6634106f10 0x7f663419cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:52.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.954+0000 7f663c5e9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6634111f30 con 0x7f6634101de0 2024-09-12T02:17:52.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.954+0000 7f663c5e9640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6634111db0 con 0x7f6634106f10 2024-09-12T02:17:52.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.954+0000 7f663c5e9640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f66341120b0 con 0x7f6634102790 2024-09-12T02:17:52.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.954+0000 7f663a385640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6634101de0 0x7f6634198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:52.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.954+0000 7f663a385640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6634101de0 0x7f6634198510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54540/0 (socket says 172.21.15.16:54540) 2024-09-12T02:17:52.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.954+0000 7f663a385640 1 -- 172.21.15.16:0/1934919752 learned_addr learned my addr 172.21.15.16:0/1934919752 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:17:52.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.954+0000 7f6639b84640 1 --2- 172.21.15.16:0/1934919752 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6634102790 0x7f6634198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:52.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.955+0000 7f663ab86640 1 --2- 172.21.15.16:0/1934919752 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6634106f10 0x7f663419cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:52.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.955+0000 7f663a385640 1 -- 172.21.15.16:0/1934919752 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6634102790 msgr2=0x7f6634198a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:52.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.955+0000 7f663a385640 1 --2- 172.21.15.16:0/1934919752 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6634102790 0x7f6634198a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:52.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.955+0000 7f663a385640 1 -- 172.21.15.16:0/1934919752 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6634106f10 msgr2=0x7f663419cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:52.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.955+0000 7f663a385640 1 --2- 172.21.15.16:0/1934919752 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6634106f10 0x7f663419cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:52.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.955+0000 7f663a385640 1 -- 172.21.15.16:0/1934919752 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f663419d4c0 con 0x7f6634101de0 2024-09-12T02:17:52.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.955+0000 7f663a385640 1 --2- 172.21.15.16:0/1934919752 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6634101de0 0x7f6634198510 secure :-1 s=READY pgs=2138 cs=0 l=1 rev1=1 crypto rx=0x7f6630006010 tx=0x7f66300042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:52.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.955+0000 7f662b7fe640 1 -- 172.21.15.16:0/1934919752 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f663003f030 con 0x7f6634101de0 2024-09-12T02:17:52.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.955+0000 7f662b7fe640 1 -- 172.21.15.16:0/1934919752 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6630003ce0 con 0x7f6634101de0 2024-09-12T02:17:52.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.955+0000 7f663c5e9640 1 -- 172.21.15.16:0/1934919752 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f663419d750 con 0x7f6634101de0 2024-09-12T02:17:52.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.955+0000 7f662b7fe640 1 -- 172.21.15.16:0/1934919752 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6630035070 con 0x7f6634101de0 2024-09-12T02:17:52.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.955+0000 7f663c5e9640 1 -- 172.21.15.16:0/1934919752 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66341a5090 con 0x7f6634101de0 2024-09-12T02:17:52.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.957+0000 7f662b7fe640 1 -- 172.21.15.16:0/1934919752 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6630042030 con 0x7f6634101de0 2024-09-12T02:17:52.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.958+0000 7f663e9c2740 1 Processor -- start 2024-09-12T02:17:52.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.958+0000 7f663e9c2740 1 -- start start 2024-09-12T02:17:52.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.958+0000 7f663e9c2740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a8e2bdfb40 0x55a8e2bde090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:52.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.958+0000 7f663e9c2740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a8e2bde5d0 0x55a8e2bdea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:52.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.958+0000 7f663e9c2740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a8e2be9370 0x55a8e2beb760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:52.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.958+0000 7f663e9c2740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55a8e2b442f0 con 0x55a8e2be9370 2024-09-12T02:17:52.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.958+0000 7f663e9c2740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55a8e2aebcf0 con 0x55a8e2bde5d0 2024-09-12T02:17:52.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.958+0000 7f663e9c2740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55a8e2b402b0 con 0x55a8e2bdfb40 2024-09-12T02:17:52.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.958+0000 7f6639b84640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a8e2be9370 0x55a8e2beb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:52.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.958+0000 7f6639b84640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a8e2be9370 0x55a8e2beb760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54546/0 (socket says 172.21.15.16:54546) 2024-09-12T02:17:52.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.958+0000 7f6639b84640 1 -- 172.21.15.16:0/711892972 learned_addr learned my addr 172.21.15.16:0/711892972 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:17:52.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.958+0000 7f663a385640 1 --2- 172.21.15.16:0/711892972 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a8e2bde5d0 0x55a8e2bdea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:52.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.958+0000 7f663ab86640 1 --2- 172.21.15.16:0/711892972 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a8e2bdfb40 0x55a8e2bde090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:52.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.959+0000 7f6639b84640 1 -- 172.21.15.16:0/711892972 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a8e2bdfb40 msgr2=0x55a8e2bde090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:52.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.959+0000 7f6639b84640 1 --2- 172.21.15.16:0/711892972 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a8e2bdfb40 0x55a8e2bde090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:52.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.959+0000 7f6639b84640 1 -- 172.21.15.16:0/711892972 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a8e2bde5d0 msgr2=0x55a8e2bdea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:52.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.959+0000 7f6639b84640 1 --2- 172.21.15.16:0/711892972 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a8e2bde5d0 0x55a8e2bdea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:52.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.959+0000 7f662b7fe640 1 --2- 172.21.15.16:0/1934919752 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f66200a2b80 0x7f66200a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:52.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.959+0000 7f6639b84640 1 -- 172.21.15.16:0/711892972 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a8e2ad03a0 con 0x55a8e2be9370 2024-09-12T02:17:52.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.959+0000 7f663ab86640 1 --2- 172.21.15.16:0/1934919752 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f66200a2b80 0x7f66200a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:52.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.959+0000 7f662b7fe640 1 -- 172.21.15.16:0/1934919752 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f66300ef660 con 0x7f6634101de0 2024-09-12T02:17:52.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.959+0000 7f6639b84640 1 --2- 172.21.15.16:0/711892972 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a8e2be9370 0x55a8e2beb760 secure :-1 s=READY pgs=2139 cs=0 l=1 rev1=1 crypto rx=0x7f662400bcb0 tx=0x7f66240107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c6bb4f554be5b75 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:52.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.959+0000 7f6628ff9640 1 -- 172.21.15.16:0/711892972 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6624004030 con 0x55a8e2be9370 2024-09-12T02:17:52.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.959+0000 7f6628ff9640 1 -- 172.21.15.16:0/711892972 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f66240041d0 con 0x55a8e2be9370 2024-09-12T02:17:52.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.959+0000 7f6628ff9640 1 -- 172.21.15.16:0/711892972 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66240117a0 con 0x55a8e2be9370 2024-09-12T02:17:52.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.960+0000 7f663e9c2740 1 -- 172.21.15.16:0/711892972 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a8e2be9370 msgr2=0x55a8e2beb760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:52.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.960+0000 7f663ab86640 1 --2- 172.21.15.16:0/1934919752 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f66200a2b80 0x7f66200a5040 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f662c0092f0 tx=0x7f662c009280 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:52.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.960+0000 7f663e9c2740 1 --2- 172.21.15.16:0/711892972 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a8e2be9370 0x55a8e2beb760 secure :-1 s=READY pgs=2139 cs=0 l=1 rev1=1 crypto rx=0x7f662400bcb0 tx=0x7f66240107a0 comp rx=0 tx=0).stop 2024-09-12T02:17:52.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.960+0000 7f663e9c2740 1 -- 172.21.15.16:0/711892972 shutdown_connections 2024-09-12T02:17:52.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.960+0000 7f663e9c2740 1 --2- 172.21.15.16:0/711892972 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a8e2be9370 0x55a8e2beb760 unknown :-1 s=CLOSED pgs=2139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:52.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.960+0000 7f663e9c2740 1 --2- 172.21.15.16:0/711892972 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a8e2bde5d0 0x55a8e2bdea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:52.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.960+0000 7f663e9c2740 1 --2- 172.21.15.16:0/711892972 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a8e2bdfb40 0x55a8e2bde090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:52.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.960+0000 7f663e9c2740 1 -- 172.21.15.16:0/711892972 >> 172.21.15.16:0/711892972 conn(0x55a8e2ba7310 msgr2=0x55a8e2bdac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:52.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.960+0000 7f663e9c2740 1 -- 172.21.15.16:0/711892972 shutdown_connections 2024-09-12T02:17:52.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.960+0000 7f663e9c2740 1 -- 172.21.15.16:0/711892972 wait complete. 2024-09-12T02:17:52.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.960+0000 7f663e9c2740 1 -- start start 2024-09-12T02:17:52.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.960+0000 7f663e9c2740 1 Processor -- start 2024-09-12T02:17:52.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.961+0000 7f663e9c2740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a8e2bde5d0 0x55a8e2be8d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:52.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.961+0000 7f663e9c2740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a8e2bdfb40 0x55a8e2be5fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:52.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.961+0000 7f663e9c2740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a8e2be9370 0x55a8e2be65d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:52.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.961+0000 7f663e9c2740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55a8e2bec070 con 0x55a8e2be9370 2024-09-12T02:17:52.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.961+0000 7f663e9c2740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55a8e2aff060 con 0x55a8e2bdfb40 2024-09-12T02:17:52.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.961+0000 7f663e9c2740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55a8e2bec1f0 con 0x55a8e2bde5d0 2024-09-12T02:17:52.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.961+0000 7f663e9c2740 10 client.0 _ll_register_callbacks cb 0x55a8e28528c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:17:52.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.961+0000 7f663ab86640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a8e2be9370 0x55a8e2be65d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:52.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.961+0000 7f663ab86640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a8e2be9370 0x55a8e2be65d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54558/0 (socket says 172.21.15.16:54558) 2024-09-12T02:17:52.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.961+0000 7f663ab86640 1 -- 172.21.15.16:0/393169381 learned_addr learned my addr 172.21.15.16:0/393169381 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:17:52.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.961+0000 7f6639b84640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a8e2bdfb40 0x55a8e2be5fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:52.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.961+0000 7f663a385640 1 --2- 172.21.15.16:0/393169381 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a8e2bde5d0 0x55a8e2be8d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:52.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.961+0000 7f663ab86640 1 -- 172.21.15.16:0/393169381 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a8e2bde5d0 msgr2=0x55a8e2be8d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:52.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.961+0000 7f663ab86640 1 --2- 172.21.15.16:0/393169381 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a8e2bde5d0 0x55a8e2be8d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:52.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.961+0000 7f663ab86640 1 -- 172.21.15.16:0/393169381 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a8e2bdfb40 msgr2=0x55a8e2be5fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:52.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.961+0000 7f663ab86640 1 --2- 172.21.15.16:0/393169381 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a8e2bdfb40 0x55a8e2be5fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:52.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.961+0000 7f663ab86640 1 -- 172.21.15.16:0/393169381 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a8e28b53c0 con 0x55a8e2be9370 2024-09-12T02:17:52.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.962+0000 7f663ab86640 1 --2- 172.21.15.16:0/393169381 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a8e2be9370 0x55a8e2be65d0 secure :-1 s=READY pgs=2140 cs=0 l=1 rev1=1 crypto rx=0x7f662c012870 tx=0x7f662c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e35efc5c549bc1f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:52.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.962+0000 7f661dffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:17:52.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.962+0000 7f661dffb640 1 -- 172.21.15.16:0/393169381 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f662c00ead0 con 0x55a8e2be9370 2024-09-12T02:17:52.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.962+0000 7f661dffb640 1 -- 172.21.15.16:0/393169381 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f662c00ec70 con 0x55a8e2be9370 2024-09-12T02:17:52.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.962+0000 7f663e9c2740 1 -- 172.21.15.16:0/393169381 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f66240041d0 con 0x55a8e2be9370 2024-09-12T02:17:52.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.962+0000 7f661dffb640 1 -- 172.21.15.16:0/393169381 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f662c01f420 con 0x55a8e2be9370 2024-09-12T02:17:52.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.962+0000 7f663e9c2740 10 client.20883 fetch_fsmap learned FSMap version 25 2024-09-12T02:17:52.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.962+0000 7f663e9c2740 1 -- 172.21.15.16:0/393169381 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a8e2bf7b30 con 0x55a8e2be9370 2024-09-12T02:17:52.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.962+0000 7f661dffb640 1 -- 172.21.15.16:0/393169381 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f66240041d0 con 0x55a8e2be9370 2024-09-12T02:17:52.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.962+0000 7f661dffb640 1 -- 172.21.15.16:0/393169381 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f662c00f040 con 0x55a8e2be9370 2024-09-12T02:17:52.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.962+0000 7f663e9c2740 10 client.20883 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:17:52.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.962+0000 7f663e9c2740 10 client.20883 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:17:52.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.963+0000 7f663e9c2740 20 client.20883 populate_metadata read hostname 'smithi016' 2024-09-12T02:17:52.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.963+0000 7f663e9c2740 1 --2- 172.21.15.16:0/393169381 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55a8e2bf8ff0 0x55a8e2bfb440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:52.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.963+0000 7f663e9c2740 4 client.20883 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:17:52.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.963+0000 7f663e9c2740 1 -- 172.21.15.16:0/393169381 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a8e2b51860 con 0x55a8e2bf8ff0 2024-09-12T02:17:52.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.963+0000 7f663a385640 1 --2- 172.21.15.16:0/393169381 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55a8e2bf8ff0 0x55a8e2bfb440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:52.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.963+0000 7f663a385640 1 --2- 172.21.15.16:0/393169381 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55a8e2bf8ff0 0x55a8e2bfb440 crc :-1 s=READY pgs=475 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51954a2b7a1d0d5b server_cookie=dab3df07c3230123 in_seq=0 out_seq=0 2024-09-12T02:17:52.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.963+0000 7f661dffb640 10 client.20883 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:17:52.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.964+0000 7f661dffb640 1 -- 172.21.15.16:0/393169381 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a8e2b51860 con 0x55a8e2bf8ff0 2024-09-12T02:17:52.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.964+0000 7f661dffb640 10 client.20883 handle_command_reply: tid=0 2024-09-12T02:17:52.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.964+0000 7f663e9c2740 1 client.20883 shutdown 2024-09-12T02:17:52.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.964+0000 7f663e9c2740 1 -- 172.21.15.16:0/393169381 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a8e2be9370 msgr2=0x55a8e2be65d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:52.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.964+0000 7f663e9c2740 1 --2- 172.21.15.16:0/393169381 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a8e2be9370 0x55a8e2be65d0 secure :-1 s=READY pgs=2140 cs=0 l=1 rev1=1 crypto rx=0x7f662c012870 tx=0x7f662c004a40 comp rx=0 tx=0).stop 2024-09-12T02:17:52.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.964+0000 7f663e9c2740 1 -- 172.21.15.16:0/393169381 shutdown_connections 2024-09-12T02:17:52.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.964+0000 7f663e9c2740 1 --2- 172.21.15.16:0/393169381 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55a8e2bf8ff0 0x55a8e2bfb440 crc :-1 s=READY pgs=475 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:52.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.964+0000 7f661dffb640 0 client.20883 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:17:52.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.964+0000 7f663e9c2740 1 --2- 172.21.15.16:0/393169381 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a8e2be9370 0x55a8e2be65d0 unknown :-1 s=CLOSED pgs=2140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:52.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.964+0000 7f663e9c2740 1 --2- 172.21.15.16:0/393169381 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a8e2bdfb40 0x55a8e2be5fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:52.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.964+0000 7f663e9c2740 1 --2- 172.21.15.16:0/393169381 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a8e2bde5d0 0x55a8e2be8d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:52.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.964+0000 7f663e9c2740 1 -- 172.21.15.16:0/393169381 >> 172.21.15.16:0/393169381 conn(0x55a8e2ba7310 msgr2=0x55a8e2be0960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:52.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.964+0000 7f663e9c2740 1 -- 172.21.15.16:0/393169381 shutdown_connections 2024-09-12T02:17:52.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.964+0000 7f663e9c2740 1 -- 172.21.15.16:0/393169381 wait complete. 2024-09-12T02:17:52.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.964+0000 7f663e9c2740 20 client.20883 trim_cache size 0 max 16384 2024-09-12T02:17:52.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.977+0000 7f663e9c2740 1 Processor -- start 2024-09-12T02:17:52.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.977+0000 7f663e9c2740 1 -- start start 2024-09-12T02:17:52.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.977+0000 7f663e9c2740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a8e2bde5d0 0x55a8e2be7f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:52.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.978+0000 7f663e9c2740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a8e2bdfb40 0x55a8e2be1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:52.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.978+0000 7f663e9c2740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a8e2be9370 0x55a8e2be1660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:52.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.978+0000 7f663e9c2740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f66341afcf0 con 0x55a8e2bdfb40 2024-09-12T02:17:52.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.978+0000 7f663e9c2740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f66341ad5c0 con 0x55a8e2be9370 2024-09-12T02:17:52.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.978+0000 7f663e9c2740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f66341ab430 con 0x55a8e2bde5d0 2024-09-12T02:17:52.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.978+0000 7f6639b84640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a8e2bdfb40 0x55a8e2be1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:52.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.978+0000 7f6639b84640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a8e2bdfb40 0x55a8e2be1120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54568/0 (socket says 172.21.15.16:54568) 2024-09-12T02:17:52.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.978+0000 7f6639b84640 1 -- 172.21.15.16:0/2687948502 learned_addr learned my addr 172.21.15.16:0/2687948502 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:17:52.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.978+0000 7f663ab86640 1 --2- 172.21.15.16:0/2687948502 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a8e2be9370 0x55a8e2be1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:52.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.978+0000 7f663a385640 1 --2- 172.21.15.16:0/2687948502 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a8e2bde5d0 0x55a8e2be7f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:52.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.978+0000 7f6639b84640 1 -- 172.21.15.16:0/2687948502 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a8e2bde5d0 msgr2=0x55a8e2be7f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:52.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.978+0000 7f6639b84640 1 --2- 172.21.15.16:0/2687948502 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a8e2bde5d0 0x55a8e2be7f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:52.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.978+0000 7f6639b84640 1 -- 172.21.15.16:0/2687948502 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a8e2be9370 msgr2=0x55a8e2be1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:52.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.978+0000 7f6639b84640 1 --2- 172.21.15.16:0/2687948502 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a8e2be9370 0x55a8e2be1660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:52.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.978+0000 7f6639b84640 1 -- 172.21.15.16:0/2687948502 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a8e2be4530 con 0x55a8e2bdfb40 2024-09-12T02:17:52.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.978+0000 7f663ab86640 1 --2- 172.21.15.16:0/2687948502 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a8e2be9370 0x55a8e2be1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:17:52.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.978+0000 7f6639b84640 1 --2- 172.21.15.16:0/2687948502 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a8e2bdfb40 0x55a8e2be1120 secure :-1 s=READY pgs=2141 cs=0 l=1 rev1=1 crypto rx=0x7f662400bcb0 tx=0x7f6624003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf069ff7bbc948e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:52.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.978+0000 7f661ffff640 1 -- 172.21.15.16:0/2687948502 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f662402b070 con 0x55a8e2bdfb40 2024-09-12T02:17:52.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.978+0000 7f661ffff640 1 -- 172.21.15.16:0/2687948502 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f66240035f0 con 0x55a8e2bdfb40 2024-09-12T02:17:52.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.978+0000 7f661ffff640 1 -- 172.21.15.16:0/2687948502 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f662400bea0 con 0x55a8e2bdfb40 2024-09-12T02:17:52.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.979+0000 7f663e9c2740 1 -- 172.21.15.16:0/2687948502 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a8e2bdfb40 msgr2=0x55a8e2be1120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:52.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.979+0000 7f663e9c2740 1 --2- 172.21.15.16:0/2687948502 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a8e2bdfb40 0x55a8e2be1120 secure :-1 s=READY pgs=2141 cs=0 l=1 rev1=1 crypto rx=0x7f662400bcb0 tx=0x7f6624003f90 comp rx=0 tx=0).stop 2024-09-12T02:17:52.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.979+0000 7f663e9c2740 1 -- 172.21.15.16:0/2687948502 shutdown_connections 2024-09-12T02:17:52.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.979+0000 7f663e9c2740 1 --2- 172.21.15.16:0/2687948502 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a8e2be9370 0x55a8e2be1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:52.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.979+0000 7f663e9c2740 1 --2- 172.21.15.16:0/2687948502 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a8e2bdfb40 0x55a8e2be1120 unknown :-1 s=CLOSED pgs=2141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:52.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.979+0000 7f663e9c2740 1 --2- 172.21.15.16:0/2687948502 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a8e2bde5d0 0x55a8e2be7f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:52.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.979+0000 7f663e9c2740 1 -- 172.21.15.16:0/2687948502 >> 172.21.15.16:0/2687948502 conn(0x55a8e2ba7310 msgr2=0x55a8e2be7840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:52.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.979+0000 7f663e9c2740 1 -- 172.21.15.16:0/2687948502 shutdown_connections 2024-09-12T02:17:52.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.979+0000 7f663e9c2740 1 -- 172.21.15.16:0/2687948502 wait complete. 2024-09-12T02:17:52.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.979+0000 7f663e9c2740 1 -- start start 2024-09-12T02:17:52.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.979+0000 7f663e9c2740 1 Processor -- start 2024-09-12T02:17:52.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.980+0000 7f663e9c2740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a8e2bde5d0 0x55a8e2bed900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:52.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.980+0000 7f663e9c2740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a8e2bdfb40 0x55a8e2bede40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:52.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.980+0000 7f663e9c2740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a8e2be9370 0x55a8e2c62ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:52.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.980+0000 7f663e9c2740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f66341aba80 con 0x55a8e2bdfb40 2024-09-12T02:17:52.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.980+0000 7f663e9c2740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f66341acef0 con 0x55a8e2bde5d0 2024-09-12T02:17:52.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.980+0000 7f663e9c2740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f66341ac150 con 0x55a8e2be9370 2024-09-12T02:17:52.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.980+0000 7f663e9c2740 10 client.0 _ll_register_callbacks cb 0x7f662c009ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:17:52.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.980+0000 7f6639b84640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a8e2bdfb40 0x55a8e2bede40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:52.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.980+0000 7f6639b84640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a8e2bdfb40 0x55a8e2bede40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54574/0 (socket says 172.21.15.16:54574) 2024-09-12T02:17:52.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.980+0000 7f6639b84640 1 -- 172.21.15.16:0/1557376729 learned_addr learned my addr 172.21.15.16:0/1557376729 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:17:52.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.980+0000 7f663a385640 1 --2- 172.21.15.16:0/1557376729 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a8e2bde5d0 0x55a8e2bed900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:52.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.980+0000 7f663ab86640 1 --2- 172.21.15.16:0/1557376729 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a8e2be9370 0x55a8e2c62ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:52.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.980+0000 7f6639b84640 1 -- 172.21.15.16:0/1557376729 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a8e2be9370 msgr2=0x55a8e2c62ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:52.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.980+0000 7f6639b84640 1 --2- 172.21.15.16:0/1557376729 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a8e2be9370 0x55a8e2c62ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:52.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.980+0000 7f6639b84640 1 -- 172.21.15.16:0/1557376729 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a8e2bde5d0 msgr2=0x55a8e2bed900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:52.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.980+0000 7f6639b84640 1 --2- 172.21.15.16:0/1557376729 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a8e2bde5d0 0x55a8e2bed900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:52.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.980+0000 7f6639b84640 1 -- 172.21.15.16:0/1557376729 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a8e2c63220 con 0x55a8e2bdfb40 2024-09-12T02:17:52.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.980+0000 7f6639b84640 1 --2- 172.21.15.16:0/1557376729 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a8e2bdfb40 0x55a8e2bede40 secure :-1 s=READY pgs=2142 cs=0 l=1 rev1=1 crypto rx=0x7f6624000c00 tx=0x7f6624005280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e09c02824264f81b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:52.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.980+0000 7f661dffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T02:17:52.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.981+0000 7f661dffb640 1 -- 172.21.15.16:0/1557376729 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6624033020 con 0x55a8e2bdfb40 2024-09-12T02:17:52.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.981+0000 7f663e9c2740 1 -- 172.21.15.16:0/1557376729 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f66240035f0 con 0x55a8e2bdfb40 2024-09-12T02:17:52.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.981+0000 7f661dffb640 1 -- 172.21.15.16:0/1557376729 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f66240037a0 con 0x55a8e2bdfb40 2024-09-12T02:17:52.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.981+0000 7f661dffb640 1 -- 172.21.15.16:0/1557376729 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f662402b040 con 0x55a8e2bdfb40 2024-09-12T02:17:52.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.981+0000 7f663e9c2740 10 client.20889 fetch_fsmap learned FSMap version 25 2024-09-12T02:17:52.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.981+0000 7f663e9c2740 1 -- 172.21.15.16:0/1557376729 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a8e2c644f0 con 0x55a8e2bdfb40 2024-09-12T02:17:52.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.981+0000 7f661dffb640 1 -- 172.21.15.16:0/1557376729 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f66240035f0 con 0x55a8e2bdfb40 2024-09-12T02:17:52.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.981+0000 7f661dffb640 1 -- 172.21.15.16:0/1557376729 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f6624025070 con 0x55a8e2bdfb40 2024-09-12T02:17:52.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.981+0000 7f663e9c2740 10 client.20889 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:17:52.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.981+0000 7f663e9c2740 10 client.20889 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T02:17:52.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.981+0000 7f663e9c2740 20 client.20889 populate_metadata read hostname 'smithi016' 2024-09-12T02:17:52.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.981+0000 7f663e9c2740 1 --2- 172.21.15.16:0/1557376729 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55a8e2bf8ff0 0x55a8e2c66a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:52.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.981+0000 7f663e9c2740 4 client.20889 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T02:17:52.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.981+0000 7f663e9c2740 1 -- 172.21.15.16:0/1557376729 --> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a8e2b4bd70 con 0x55a8e2bf8ff0 2024-09-12T02:17:52.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.982+0000 7f663a385640 1 --2- 172.21.15.16:0/1557376729 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55a8e2bf8ff0 0x55a8e2c66a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:52.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.982+0000 7f663a385640 1 --2- 172.21.15.16:0/1557376729 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55a8e2bf8ff0 0x55a8e2c66a50 crc :-1 s=READY pgs=476 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a6bab9323ec0d6aa server_cookie=33e089cbe56bf17b in_seq=0 out_seq=0 2024-09-12T02:17:52.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.982+0000 7f661dffb640 10 client.20889 ms_handle_connect on v2:172.21.15.33:6838/4136670204 2024-09-12T02:17:52.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.982+0000 7f661dffb640 1 -- 172.21.15.16:0/1557376729 <== mds.0 v2:172.21.15.33:6838/4136670204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a8e2b4bd70 con 0x55a8e2bf8ff0 2024-09-12T02:17:52.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.982+0000 7f661dffb640 10 client.20889 handle_command_reply: tid=0 2024-09-12T02:17:52.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.982+0000 7f663e9c2740 1 client.20889 shutdown 2024-09-12T02:17:52.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.982+0000 7f663e9c2740 1 -- 172.21.15.16:0/1557376729 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a8e2bdfb40 msgr2=0x55a8e2bede40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:52.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.982+0000 7f663e9c2740 1 --2- 172.21.15.16:0/1557376729 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a8e2bdfb40 0x55a8e2bede40 secure :-1 s=READY pgs=2142 cs=0 l=1 rev1=1 crypto rx=0x7f6624000c00 tx=0x7f6624005280 comp rx=0 tx=0).stop 2024-09-12T02:17:52.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.982+0000 7f663e9c2740 1 -- 172.21.15.16:0/1557376729 shutdown_connections 2024-09-12T02:17:52.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.982+0000 7f663e9c2740 1 --2- 172.21.15.16:0/1557376729 >> [v2:172.21.15.33:6838/4136670204,v1:172.21.15.33:6839/4136670204] conn(0x55a8e2bf8ff0 0x55a8e2c66a50 crc :-1 s=READY pgs=476 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:52.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.982+0000 7f663e9c2740 1 --2- 172.21.15.16:0/1557376729 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a8e2be9370 0x55a8e2c62ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:52.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.982+0000 7f663e9c2740 1 --2- 172.21.15.16:0/1557376729 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a8e2bdfb40 0x55a8e2bede40 unknown :-1 s=CLOSED pgs=2142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:52.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.982+0000 7f663e9c2740 1 --2- 172.21.15.16:0/1557376729 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a8e2bde5d0 0x55a8e2bed900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:52.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.982+0000 7f663e9c2740 1 -- 172.21.15.16:0/1557376729 >> 172.21.15.16:0/1557376729 conn(0x55a8e2ba7310 msgr2=0x55a8e2be8060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:52.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.982+0000 7f661dffb640 0 client.20889 ms_handle_reset on v2:172.21.15.33:6838/4136670204 2024-09-12T02:17:52.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.983+0000 7f663e9c2740 1 -- 172.21.15.16:0/1557376729 shutdown_connections 2024-09-12T02:17:52.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.983+0000 7f663e9c2740 1 -- 172.21.15.16:0/1557376729 wait complete. 2024-09-12T02:17:52.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.983+0000 7f663e9c2740 20 client.20889 trim_cache size 0 max 16384 2024-09-12T02:17:52.990 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T02:17:52.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.983+0000 7f663c5e9640 1 -- 172.21.15.16:0/1934919752 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f66200a2b80 msgr2=0x7f66200a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:52.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.983+0000 7f663c5e9640 1 --2- 172.21.15.16:0/1934919752 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f66200a2b80 0x7f66200a5040 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f662c0092f0 tx=0x7f662c009280 comp rx=0 tx=0).stop 2024-09-12T02:17:52.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.983+0000 7f663c5e9640 1 -- 172.21.15.16:0/1934919752 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6634101de0 msgr2=0x7f6634198510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:52.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.983+0000 7f663c5e9640 1 --2- 172.21.15.16:0/1934919752 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6634101de0 0x7f6634198510 secure :-1 s=READY pgs=2138 cs=0 l=1 rev1=1 crypto rx=0x7f6630006010 tx=0x7f66300042f0 comp rx=0 tx=0).stop 2024-09-12T02:17:52.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.983+0000 7f663c5e9640 1 -- 172.21.15.16:0/1934919752 shutdown_connections 2024-09-12T02:17:52.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.983+0000 7f663c5e9640 1 --2- 172.21.15.16:0/1934919752 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f66200a2b80 0x7f66200a5040 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:52.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.983+0000 7f663c5e9640 1 --2- 172.21.15.16:0/1934919752 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6634106f10 0x7f663419cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:52.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.983+0000 7f663c5e9640 1 --2- 172.21.15.16:0/1934919752 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6634102790 0x7f6634198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:52.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.983+0000 7f663c5e9640 1 --2- 172.21.15.16:0/1934919752 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6634101de0 0x7f6634198510 unknown :-1 s=CLOSED pgs=2138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:52.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.983+0000 7f663c5e9640 1 -- 172.21.15.16:0/1934919752 >> 172.21.15.16:0/1934919752 conn(0x7f66340fd8c0 msgr2=0x7f66340fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:52.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.983+0000 7f663c5e9640 1 -- 172.21.15.16:0/1934919752 shutdown_connections 2024-09-12T02:17:52.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:52.983+0000 7f663c5e9640 1 -- 172.21.15.16:0/1934919752 wait complete. 2024-09-12T02:17:53.722 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph_086c737fa75c22d521a3932c59b31c5e1d1c9ef4/qa/tasks/fwd_scrub.py", line 35, in _run self.do_scrub() File "/home/teuthworker/src/git.ceph.com_ceph_086c737fa75c22d521a3932c59b31c5e1d1c9ef4/qa/tasks/fwd_scrub.py", line 49, in do_scrub self.sleep_unless_stopped(self.sleep_between_iterations) File "/home/teuthworker/src/git.ceph.com_ceph_086c737fa75c22d521a3932c59b31c5e1d1c9ef4/qa/tasks/thrasher.py", line 47, in sleep_unless_stopped raise self.Stopped() tasks.thrasher.ThrasherGreenlet.Stopped 2024-09-12T02:17:53.723 INFO:tasks.fwd_scrub:done joining 2024-09-12T02:17:53.724 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-09-12T02:17:53.734 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T02:17:53.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:53.840+0000 7f0b362ba640 1 -- 172.21.15.16:0/2463189150 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b30107810 msgr2=0x7f0b30109c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:53.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:53.840+0000 7f0b362ba640 1 --2- 172.21.15.16:0/2463189150 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b30107810 0x7f0b30109c80 secure :-1 s=READY pgs=2143 cs=0 l=1 rev1=1 crypto rx=0x7f0b28009ca0 tx=0x7f0b2802a8c0 comp rx=0 tx=0).stop 2024-09-12T02:17:53.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:53.841+0000 7f0b362ba640 1 -- 172.21.15.16:0/2463189150 shutdown_connections 2024-09-12T02:17:53.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:53.841+0000 7f0b362ba640 1 --2- 172.21.15.16:0/2463189150 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b30107810 0x7f0b30109c80 unknown :-1 s=CLOSED pgs=2143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:53.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:53.841+0000 7f0b362ba640 1 --2- 172.21.15.16:0/2463189150 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b30104ee0 0x7f0b301072d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:53.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:53.841+0000 7f0b362ba640 1 --2- 172.21.15.16:0/2463189150 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0b301025b0 0x7f0b301049a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:53.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:53.841+0000 7f0b362ba640 1 -- 172.21.15.16:0/2463189150 >> 172.21.15.16:0/2463189150 conn(0x7f0b300fc1b0 msgr2=0x7f0b300fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:53.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:53.841+0000 7f0b362ba640 1 -- 172.21.15.16:0/2463189150 shutdown_connections 2024-09-12T02:17:53.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:53.841+0000 7f0b362ba640 1 -- 172.21.15.16:0/2463189150 wait complete. 2024-09-12T02:17:53.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:53.841+0000 7f0b362ba640 1 Processor -- start 2024-09-12T02:17:53.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:53.841+0000 7f0b362ba640 1 -- start start 2024-09-12T02:17:53.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:53.841+0000 7f0b362ba640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0b301025b0 0x7f0b30197ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:53.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:53.841+0000 7f0b362ba640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b30104ee0 0x7f0b30198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:53.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:53.841+0000 7f0b362ba640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b30107810 0x7f0b3019f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:53.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:53.841+0000 7f0b362ba640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0b3010d850 con 0x7f0b30107810 2024-09-12T02:17:53.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:53.841+0000 7f0b362ba640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0b3010d6d0 con 0x7f0b30104ee0 2024-09-12T02:17:53.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:53.842+0000 7f0b362ba640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f0b3010d9d0 con 0x7f0b301025b0 2024-09-12T02:17:53.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:53.842+0000 7f0b34857640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b30107810 0x7f0b3019f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:53.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:53.842+0000 7f0b34857640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b30107810 0x7f0b3019f470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54598/0 (socket says 172.21.15.16:54598) 2024-09-12T02:17:53.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:53.842+0000 7f0b34857640 1 -- 172.21.15.16:0/1673809096 learned_addr learned my addr 172.21.15.16:0/1673809096 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:17:53.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:53.842+0000 7f0b2ffff640 1 --2- 172.21.15.16:0/1673809096 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0b301025b0 0x7f0b30197ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:53.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:53.842+0000 7f0b2f7fe640 1 --2- 172.21.15.16:0/1673809096 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b30104ee0 0x7f0b30198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:53.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:53.842+0000 7f0b34857640 1 -- 172.21.15.16:0/1673809096 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0b301025b0 msgr2=0x7f0b30197ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:53.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:53.842+0000 7f0b34857640 1 --2- 172.21.15.16:0/1673809096 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0b301025b0 0x7f0b30197ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:53.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:53.842+0000 7f0b34857640 1 -- 172.21.15.16:0/1673809096 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b30104ee0 msgr2=0x7f0b30198400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:53.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:53.842+0000 7f0b34857640 1 --2- 172.21.15.16:0/1673809096 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b30104ee0 0x7f0b30198400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:53.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:53.842+0000 7f0b34857640 1 -- 172.21.15.16:0/1673809096 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b3019fae0 con 0x7f0b30107810 2024-09-12T02:17:53.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:53.842+0000 7f0b2f7fe640 1 --2- 172.21.15.16:0/1673809096 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b30104ee0 0x7f0b30198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:17:53.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:53.842+0000 7f0b34857640 1 --2- 172.21.15.16:0/1673809096 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b30107810 0x7f0b3019f470 secure :-1 s=READY pgs=2144 cs=0 l=1 rev1=1 crypto rx=0x7f0b28031a40 tx=0x7f0b28031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:53.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:53.842+0000 7f0b2d7fa640 1 -- 172.21.15.16:0/1673809096 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b28003c30 con 0x7f0b30107810 2024-09-12T02:17:53.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:53.842+0000 7f0b2d7fa640 1 -- 172.21.15.16:0/1673809096 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b28003dd0 con 0x7f0b30107810 2024-09-12T02:17:53.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:53.842+0000 7f0b362ba640 1 -- 172.21.15.16:0/1673809096 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b3019fd70 con 0x7f0b30107810 2024-09-12T02:17:53.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:53.842+0000 7f0b2d7fa640 1 -- 172.21.15.16:0/1673809096 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b2802fdb0 con 0x7f0b30107810 2024-09-12T02:17:53.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:53.842+0000 7f0b362ba640 1 -- 172.21.15.16:0/1673809096 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b301a0280 con 0x7f0b30107810 2024-09-12T02:17:53.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:53.843+0000 7f0b362ba640 1 -- 172.21.15.16:0/1673809096 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b300fdca0 con 0x7f0b30107810 2024-09-12T02:17:53.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:53.845+0000 7f0b2d7fa640 1 -- 172.21.15.16:0/1673809096 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f0b28004050 con 0x7f0b30107810 2024-09-12T02:17:53.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:53.846+0000 7f0b2d7fa640 1 --2- 172.21.15.16:0/1673809096 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f0b180a2c10 0x7f0b180a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:53.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:53.846+0000 7f0b2d7fa640 1 -- 172.21.15.16:0/1673809096 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f0b280f0470 con 0x7f0b30107810 2024-09-12T02:17:53.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:53.846+0000 7f0b2ffff640 1 --2- 172.21.15.16:0/1673809096 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f0b180a2c10 0x7f0b180a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:53.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:53.846+0000 7f0b2ffff640 1 --2- 172.21.15.16:0/1673809096 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f0b180a2c10 0x7f0b180a50d0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f0b20009890 tx=0x7f0b20008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:53.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:53.849+0000 7f0b2d7fa640 1 -- 172.21.15.16:0/1673809096 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0b280bd500 con 0x7f0b30107810 2024-09-12T02:17:53.933 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:17:53 smithi016 ceph-mon[30947]: pgmap v814: 97 pgs: 97 active+clean; 4.3 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.9 KiB/s wr, 14 op/s 2024-09-12T02:17:54.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.027+0000 7f0b362ba640 1 -- 172.21.15.16:0/1673809096 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0b30062e90 con 0x7f0b30107810 2024-09-12T02:17:54.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.029+0000 7f0b2d7fa640 1 -- 172.21.15.16:0/1673809096 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+22584 (secure 0 0 0) 0x7f0b280bce00 con 0x7f0b30107810 2024-09-12T02:17:54.030 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T02:17:54.030 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":82,"fsid":"879f3b56-70a9-11ef-bcea-c7b262605968","created":"2024-09-12T01:52:12.841045+0000","modified":"2024-09-12T02:00:09.559511+0000","last_up_change":"2024-09-12T01:59:08.975443+0000","last_in_change":"2024-09-12T01:58:44.530220+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T01:55:26.727564+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T01:59:52.402097+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T01:59:54.463627+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3ea41eb6-e384-4ddc-810f-0f103c127849","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4000187867},{"type":"v1","addr":"172.21.15.16:6803","nonce":4000187867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4000187867},{"type":"v1","addr":"172.21.15.16:6805","nonce":4000187867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":4000187867},{"type":"v1","addr":"172.21.15.16:6809","nonce":4000187867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4000187867},{"type":"v1","addr":"172.21.15.16:6807","nonce":4000187867}]},"public_addr":"172.21.15.16:6803/4000187867","cluster_addr":"172.21.15.16:6805/4000187867","heartbeat_back_addr":"172.21.15.16:6809/4000187867","heartbeat_front_addr":"172.21.15.16:6807/4000187867","state":["exists","up"]},{"osd":1,"uuid":"07bf63e7-70fa-41b2-af53-12a5fec3f33c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":3862661148},{"type":"v1","addr":"172.21.15.33:6801","nonce":3862661148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":3862661148},{"type":"v1","addr":"172.21.15.33:6803","nonce":3862661148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":3862661148},{"type":"v1","addr":"172.21.15.33:6807","nonce":3862661148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":3862661148},{"type":"v1","addr":"172.21.15.33:6805","nonce":3862661148}]},"public_addr":"172.21.15.33:6801/3862661148","cluster_addr":"172.21.15.33:6803/3862661148","heartbeat_back_addr":"172.21.15.33:6807/3862661148","heartbeat_front_addr":"172.21.15.33:6805/3862661148","state":["exists","up"]},{"osd":2,"uuid":"22e40cac-3bc0-4cce-b47b-77499b767f7e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":3958753963},{"type":"v1","addr":"172.21.15.184:6801","nonce":3958753963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":3958753963},{"type":"v1","addr":"172.21.15.184:6803","nonce":3958753963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":3958753963},{"type":"v1","addr":"172.21.15.184:6807","nonce":3958753963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":3958753963},{"type":"v1","addr":"172.21.15.184:6805","nonce":3958753963}]},"public_addr":"172.21.15.184:6801/3958753963","cluster_addr":"172.21.15.184:6803/3958753963","heartbeat_back_addr":"172.21.15.184:6807/3958753963","heartbeat_front_addr":"172.21.15.184:6805/3958753963","state":["exists","up"]},{"osd":3,"uuid":"834a07f5-2c24-49e1-8e4a-12c96cfbdc65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3395182524},{"type":"v1","addr":"172.21.15.16:6811","nonce":3395182524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3395182524},{"type":"v1","addr":"172.21.15.16:6813","nonce":3395182524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3395182524},{"type":"v1","addr":"172.21.15.16:6817","nonce":3395182524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3395182524},{"type":"v1","addr":"172.21.15.16:6815","nonce":3395182524}]},"public_addr":"172.21.15.16:6811/3395182524","cluster_addr":"172.21.15.16:6813/3395182524","heartbeat_back_addr":"172.21.15.16:6817/3395182524","heartbeat_front_addr":"172.21.15.16:6815/3395182524","state":["exists","up"]},{"osd":4,"uuid":"3ca56e09-c194-4404-b9bd-2524d38367eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":1355184322},{"type":"v1","addr":"172.21.15.33:6809","nonce":1355184322}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1355184322},{"type":"v1","addr":"172.21.15.33:6811","nonce":1355184322}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1355184322},{"type":"v1","addr":"172.21.15.33:6815","nonce":1355184322}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1355184322},{"type":"v1","addr":"172.21.15.33:6813","nonce":1355184322}]},"public_addr":"172.21.15.33:6809/1355184322","cluster_addr":"172.21.15.33:6811/1355184322","heartbeat_back_addr":"172.21.15.33:6815/1355184322","heartbeat_front_addr":"172.21.15.33:6813/1355184322","state":["exists","up"]},{"osd":5,"uuid":"0b05ecdb-b987-461f-951c-6d76739bfb4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":1533397276},{"type":"v1","addr":"172.21.15.184:6809","nonce":1533397276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":1533397276},{"type":"v1","addr":"172.21.15.184:6811","nonce":1533397276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":1533397276},{"type":"v1","addr":"172.21.15.184:6815","nonce":1533397276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":1533397276},{"type":"v1","addr":"172.21.15.184:6813","nonce":1533397276}]},"public_addr":"172.21.15.184:6809/1533397276","cluster_addr":"172.21.15.184:6811/1533397276","heartbeat_back_addr":"172.21.15.184:6815/1533397276","heartbeat_front_addr":"172.21.15.184:6813/1533397276","state":["exists","up"]},{"osd":6,"uuid":"b3936e21-e197-49fa-8441-d909b258b612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":902466142},{"type":"v1","addr":"172.21.15.16:6819","nonce":902466142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":902466142},{"type":"v1","addr":"172.21.15.16:6821","nonce":902466142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":902466142},{"type":"v1","addr":"172.21.15.16:6825","nonce":902466142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":902466142},{"type":"v1","addr":"172.21.15.16:6823","nonce":902466142}]},"public_addr":"172.21.15.16:6819/902466142","cluster_addr":"172.21.15.16:6821/902466142","heartbeat_back_addr":"172.21.15.16:6825/902466142","heartbeat_front_addr":"172.21.15.16:6823/902466142","state":["exists","up"]},{"osd":7,"uuid":"4ddbe011-34c1-4b3f-a8eb-577422fb5426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":4186032630},{"type":"v1","addr":"172.21.15.33:6817","nonce":4186032630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":4186032630},{"type":"v1","addr":"172.21.15.33:6819","nonce":4186032630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":4186032630},{"type":"v1","addr":"172.21.15.33:6823","nonce":4186032630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":4186032630},{"type":"v1","addr":"172.21.15.33:6821","nonce":4186032630}]},"public_addr":"172.21.15.33:6817/4186032630","cluster_addr":"172.21.15.33:6819/4186032630","heartbeat_back_addr":"172.21.15.33:6823/4186032630","heartbeat_front_addr":"172.21.15.33:6821/4186032630","state":["exists","up"]},{"osd":8,"uuid":"dca5bfb8-ac6a-47ce-905c-be7fcaf0b5ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":781851781},{"type":"v1","addr":"172.21.15.184:6817","nonce":781851781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":781851781},{"type":"v1","addr":"172.21.15.184:6819","nonce":781851781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":781851781},{"type":"v1","addr":"172.21.15.184:6823","nonce":781851781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":781851781},{"type":"v1","addr":"172.21.15.184:6821","nonce":781851781}]},"public_addr":"172.21.15.184:6817/781851781","cluster_addr":"172.21.15.184:6819/781851781","heartbeat_back_addr":"172.21.15.184:6823/781851781","heartbeat_front_addr":"172.21.15.184:6821/781851781","state":["exists","up"]},{"osd":9,"uuid":"23a05cb5-01d5-40f3-b66a-15fcccedbd6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":1569578746},{"type":"v1","addr":"172.21.15.16:6827","nonce":1569578746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":1569578746},{"type":"v1","addr":"172.21.15.16:6829","nonce":1569578746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1569578746},{"type":"v1","addr":"172.21.15.16:6833","nonce":1569578746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":1569578746},{"type":"v1","addr":"172.21.15.16:6831","nonce":1569578746}]},"public_addr":"172.21.15.16:6827/1569578746","cluster_addr":"172.21.15.16:6829/1569578746","heartbeat_back_addr":"172.21.15.16:6833/1569578746","heartbeat_front_addr":"172.21.15.16:6831/1569578746","state":["exists","up"]},{"osd":10,"uuid":"e4bd6bce-3c69-44db-bc09-d264a0ad094a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":469977847},{"type":"v1","addr":"172.21.15.33:6825","nonce":469977847}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":469977847},{"type":"v1","addr":"172.21.15.33:6827","nonce":469977847}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":469977847},{"type":"v1","addr":"172.21.15.33:6831","nonce":469977847}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":469977847},{"type":"v1","addr":"172.21.15.33:6829","nonce":469977847}]},"public_addr":"172.21.15.33:6825/469977847","cluster_addr":"172.21.15.33:6827/469977847","heartbeat_back_addr":"172.21.15.33:6831/469977847","heartbeat_front_addr":"172.21.15.33:6829/469977847","state":["exists","up"]},{"osd":11,"uuid":"d60212f8-5a1d-45f7-9a1b-12294bb4c80d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":1619871346},{"type":"v1","addr":"172.21.15.184:6825","nonce":1619871346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":1619871346},{"type":"v1","addr":"172.21.15.184:6827","nonce":1619871346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":1619871346},{"type":"v1","addr":"172.21.15.184:6831","nonce":1619871346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":1619871346},{"type":"v1","addr":"172.21.15.184:6829","nonce":1619871346}]},"public_addr":"172.21.15.184:6825/1619871346","cluster_addr":"172.21.15.184:6827/1619871346","heartbeat_back_addr":"172.21.15.184:6831/1619871346","heartbeat_front_addr":"172.21.15.184:6829/1619871346","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:54:32.057979+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:54:57.501562+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:55:23.137225+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:55:48.646155+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:56:13.311907+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:56:38.214188+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:57:02.570921+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:57:27.506097+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:57:52.038529+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:58:17.423300+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:58:41.579992+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:59:06.245817+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:0/3532565698":"2024-09-13T01:53:08.218475+0000","172.21.15.16:0/1822157179":"2024-09-13T01:52:44.194152+0000","172.21.15.16:0/3919664061":"2024-09-13T01:52:44.194152+0000","172.21.15.16:0/2269394774":"2024-09-13T01:52:44.194152+0000","172.21.15.16:0/2595308634":"2024-09-13T01:53:08.218475+0000","172.21.15.16:6801/2754331153":"2024-09-13T01:52:44.194152+0000","172.21.15.16:6801/2615861334":"2024-09-13T01:53:08.218475+0000","172.21.15.16:6800/2615861334":"2024-09-13T01:53:08.218475+0000","172.21.15.16:0/4199114771":"2024-09-13T01:53:08.218475+0000","172.21.15.16:6800/2754331153":"2024-09-13T01:52:44.194152+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T02:17:54.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.031+0000 7f0b362ba640 1 -- 172.21.15.16:0/1673809096 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f0b180a2c10 msgr2=0x7f0b180a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:54.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.031+0000 7f0b362ba640 1 --2- 172.21.15.16:0/1673809096 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f0b180a2c10 0x7f0b180a50d0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f0b20009890 tx=0x7f0b20008040 comp rx=0 tx=0).stop 2024-09-12T02:17:54.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.031+0000 7f0b362ba640 1 -- 172.21.15.16:0/1673809096 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b30107810 msgr2=0x7f0b3019f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:54.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.031+0000 7f0b362ba640 1 --2- 172.21.15.16:0/1673809096 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b30107810 0x7f0b3019f470 secure :-1 s=READY pgs=2144 cs=0 l=1 rev1=1 crypto rx=0x7f0b28031a40 tx=0x7f0b28031ee0 comp rx=0 tx=0).stop 2024-09-12T02:17:54.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.031+0000 7f0b362ba640 1 -- 172.21.15.16:0/1673809096 shutdown_connections 2024-09-12T02:17:54.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.031+0000 7f0b362ba640 1 --2- 172.21.15.16:0/1673809096 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f0b180a2c10 0x7f0b180a50d0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:54.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.031+0000 7f0b362ba640 1 --2- 172.21.15.16:0/1673809096 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b30107810 0x7f0b3019f470 unknown :-1 s=CLOSED pgs=2144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:54.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.031+0000 7f0b362ba640 1 --2- 172.21.15.16:0/1673809096 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b30104ee0 0x7f0b30198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:54.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.031+0000 7f0b362ba640 1 --2- 172.21.15.16:0/1673809096 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0b301025b0 0x7f0b30197ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:54.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.031+0000 7f0b362ba640 1 -- 172.21.15.16:0/1673809096 >> 172.21.15.16:0/1673809096 conn(0x7f0b300fc1b0 msgr2=0x7f0b30102e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:54.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.031+0000 7f0b362ba640 1 -- 172.21.15.16:0/1673809096 shutdown_connections 2024-09-12T02:17:54.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.031+0000 7f0b362ba640 1 -- 172.21.15.16:0/1673809096 wait complete. 2024-09-12T02:17:54.043 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-12T01:55:26.727564+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-12T01:59:52.402097+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-12T01:59:54.463627+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-12T02:17:54.044 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-12T02:17:54.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:17:53 smithi184 ceph-mon[35593]: pgmap v814: 97 pgs: 97 active+clean; 4.3 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.9 KiB/s wr, 14 op/s 2024-09-12T02:17:54.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.149+0000 7f76c652a640 1 -- 172.21.15.16:0/713836092 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76c0101b10 msgr2=0x7f76c0101f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:54.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.149+0000 7f76c652a640 1 --2- 172.21.15.16:0/713836092 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76c0101b10 0x7f76c0101f10 secure :-1 s=READY pgs=2145 cs=0 l=1 rev1=1 crypto rx=0x7f76b0009900 tx=0x7f76b002a410 comp rx=0 tx=0).stop 2024-09-12T02:17:54.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.149+0000 7f76c652a640 1 -- 172.21.15.16:0/713836092 shutdown_connections 2024-09-12T02:17:54.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.149+0000 7f76c652a640 1 --2- 172.21.15.16:0/713836092 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76c01036c0 0x7f76c0109f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:54.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.149+0000 7f76c652a640 1 --2- 172.21.15.16:0/713836092 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f76c0102d00 0x7f76c0103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:54.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.149+0000 7f76c652a640 1 --2- 172.21.15.16:0/713836092 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76c0101b10 0x7f76c0101f10 unknown :-1 s=CLOSED pgs=2145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:54.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.149+0000 7f76c652a640 1 -- 172.21.15.16:0/713836092 >> 172.21.15.16:0/713836092 conn(0x7f76c00fd2b0 msgr2=0x7f76c00ff6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:54.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.149+0000 7f76c652a640 1 -- 172.21.15.16:0/713836092 shutdown_connections 2024-09-12T02:17:54.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.149+0000 7f76c652a640 1 -- 172.21.15.16:0/713836092 wait complete. 2024-09-12T02:17:54.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.150+0000 7f76c652a640 1 Processor -- start 2024-09-12T02:17:54.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.150+0000 7f76c652a640 1 -- start start 2024-09-12T02:17:54.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.150+0000 7f76c652a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f76c0101b10 0x7f76c0197e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:54.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.150+0000 7f76c652a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76c0102d00 0x7f76c01983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:54.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.150+0000 7f76c652a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76c01036c0 0x7f76c019f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:54.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.150+0000 7f76c652a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f76c010d770 con 0x7f76c0102d00 2024-09-12T02:17:54.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.150+0000 7f76c652a640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f76c010d5f0 con 0x7f76c01036c0 2024-09-12T02:17:54.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.150+0000 7f76c652a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f76c010d8f0 con 0x7f76c0101b10 2024-09-12T02:17:54.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.150+0000 7f76bf7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76c0102d00 0x7f76c01983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:54.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.150+0000 7f76bf7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76c0102d00 0x7f76c01983d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54614/0 (socket says 172.21.15.16:54614) 2024-09-12T02:17:54.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.150+0000 7f76bf7fe640 1 -- 172.21.15.16:0/3675695022 learned_addr learned my addr 172.21.15.16:0/3675695022 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:17:54.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.150+0000 7f76bffff640 1 --2- 172.21.15.16:0/3675695022 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f76c0101b10 0x7f76c0197e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:54.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.150+0000 7f76c4ac7640 1 --2- 172.21.15.16:0/3675695022 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76c01036c0 0x7f76c019f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:54.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.150+0000 7f76bf7fe640 1 -- 172.21.15.16:0/3675695022 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f76c0101b10 msgr2=0x7f76c0197e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:54.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.150+0000 7f76bf7fe640 1 --2- 172.21.15.16:0/3675695022 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f76c0101b10 0x7f76c0197e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:54.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.150+0000 7f76bf7fe640 1 -- 172.21.15.16:0/3675695022 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76c01036c0 msgr2=0x7f76c019f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:54.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.150+0000 7f76bf7fe640 1 --2- 172.21.15.16:0/3675695022 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76c01036c0 0x7f76c019f440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:54.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.150+0000 7f76bf7fe640 1 -- 172.21.15.16:0/3675695022 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76c019fab0 con 0x7f76c0102d00 2024-09-12T02:17:54.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.151+0000 7f76bf7fe640 1 --2- 172.21.15.16:0/3675695022 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76c0102d00 0x7f76c01983d0 secure :-1 s=READY pgs=2146 cs=0 l=1 rev1=1 crypto rx=0x7f76b400d950 tx=0x7f76b400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:54.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.151+0000 7f76bd7fa640 1 -- 172.21.15.16:0/3675695022 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76b4017030 con 0x7f76c0102d00 2024-09-12T02:17:54.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.151+0000 7f76bd7fa640 1 -- 172.21.15.16:0/3675695022 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76b4004020 con 0x7f76c0102d00 2024-09-12T02:17:54.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.151+0000 7f76c652a640 1 -- 172.21.15.16:0/3675695022 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76c019fda0 con 0x7f76c0102d00 2024-09-12T02:17:54.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.151+0000 7f76bd7fa640 1 -- 172.21.15.16:0/3675695022 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76b40047a0 con 0x7f76c0102d00 2024-09-12T02:17:54.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.151+0000 7f76c652a640 1 -- 172.21.15.16:0/3675695022 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76c01a0260 con 0x7f76c0102d00 2024-09-12T02:17:54.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.152+0000 7f76c652a640 1 -- 172.21.15.16:0/3675695022 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76c0108fa0 con 0x7f76c0102d00 2024-09-12T02:17:54.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.153+0000 7f76bd7fa640 1 -- 172.21.15.16:0/3675695022 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f76b40041d0 con 0x7f76c0102d00 2024-09-12T02:17:54.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.154+0000 7f76bd7fa640 1 --2- 172.21.15.16:0/3675695022 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f76a80a2b90 0x7f76a80a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:54.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.154+0000 7f76bffff640 1 --2- 172.21.15.16:0/3675695022 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f76a80a2b90 0x7f76a80a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:54.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.154+0000 7f76bd7fa640 1 -- 172.21.15.16:0/3675695022 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f76b40cf0e0 con 0x7f76c0102d00 2024-09-12T02:17:54.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.154+0000 7f76bffff640 1 --2- 172.21.15.16:0/3675695022 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f76a80a2b90 0x7f76a80a5050 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f76b0006010 tx=0x7f76b0000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:54.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.157+0000 7f76bd7fa640 1 -- 172.21.15.16:0/3675695022 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f76b409c0e0 con 0x7f76c0102d00 2024-09-12T02:17:54.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:17:53 smithi033 ceph-mon[35332]: pgmap v814: 97 pgs: 97 active+clean; 4.3 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.9 KiB/s wr, 14 op/s 2024-09-12T02:17:54.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.332+0000 7f76c652a640 1 -- 172.21.15.16:0/3675695022 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f76c002da10 con 0x7f76c0102d00 2024-09-12T02:17:54.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.332+0000 7f76bd7fa640 1 -- 172.21.15.16:0/3675695022 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7f76b409b9e0 con 0x7f76c0102d00 2024-09-12T02:17:54.334 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-09-12T02:17:54.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.335+0000 7f76c652a640 1 -- 172.21.15.16:0/3675695022 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f76a80a2b90 msgr2=0x7f76a80a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:54.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.335+0000 7f76c652a640 1 --2- 172.21.15.16:0/3675695022 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f76a80a2b90 0x7f76a80a5050 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f76b0006010 tx=0x7f76b0000bc0 comp rx=0 tx=0).stop 2024-09-12T02:17:54.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.335+0000 7f76c652a640 1 -- 172.21.15.16:0/3675695022 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76c0102d00 msgr2=0x7f76c01983d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:54.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.335+0000 7f76c652a640 1 --2- 172.21.15.16:0/3675695022 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76c0102d00 0x7f76c01983d0 secure :-1 s=READY pgs=2146 cs=0 l=1 rev1=1 crypto rx=0x7f76b400d950 tx=0x7f76b400de10 comp rx=0 tx=0).stop 2024-09-12T02:17:54.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.335+0000 7f76c652a640 1 -- 172.21.15.16:0/3675695022 shutdown_connections 2024-09-12T02:17:54.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.335+0000 7f76c652a640 1 --2- 172.21.15.16:0/3675695022 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f76a80a2b90 0x7f76a80a5050 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:54.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.335+0000 7f76c652a640 1 --2- 172.21.15.16:0/3675695022 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76c01036c0 0x7f76c019f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:54.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.335+0000 7f76c652a640 1 --2- 172.21.15.16:0/3675695022 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76c0102d00 0x7f76c01983d0 unknown :-1 s=CLOSED pgs=2146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:54.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.335+0000 7f76c652a640 1 --2- 172.21.15.16:0/3675695022 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f76c0101b10 0x7f76c0197e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:54.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.335+0000 7f76c652a640 1 -- 172.21.15.16:0/3675695022 >> 172.21.15.16:0/3675695022 conn(0x7f76c00fd2b0 msgr2=0x7f76c0106170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:54.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.335+0000 7f76c652a640 1 -- 172.21.15.16:0/3675695022 shutdown_connections 2024-09-12T02:17:54.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.335+0000 7f76c652a640 1 -- 172.21.15.16:0/3675695022 wait complete. 2024-09-12T02:17:54.347 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-12T02:17:54.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.452+0000 7f0cd74c9640 1 -- 172.21.15.16:0/3137123633 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0cd0107840 msgr2=0x7f0cd0109c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:54.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.452+0000 7f0cd74c9640 1 --2- 172.21.15.16:0/3137123633 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0cd0107840 0x7f0cd0109c50 secure :-1 s=READY pgs=2147 cs=0 l=1 rev1=1 crypto rx=0x7f0cc8009ca0 tx=0x7f0cc802a8c0 comp rx=0 tx=0).stop 2024-09-12T02:17:54.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.452+0000 7f0cd74c9640 1 -- 172.21.15.16:0/3137123633 shutdown_connections 2024-09-12T02:17:54.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.452+0000 7f0cd74c9640 1 --2- 172.21.15.16:0/3137123633 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0cd0107840 0x7f0cd0109c50 unknown :-1 s=CLOSED pgs=2147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:54.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.452+0000 7f0cd74c9640 1 --2- 172.21.15.16:0/3137123633 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0cd0104f10 0x7f0cd0107300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:54.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.452+0000 7f0cd74c9640 1 --2- 172.21.15.16:0/3137123633 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0cd01025e0 0x7f0cd01049d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:54.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.452+0000 7f0cd74c9640 1 -- 172.21.15.16:0/3137123633 >> 172.21.15.16:0/3137123633 conn(0x7f0cd00fc140 msgr2=0x7f0cd00fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:54.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.452+0000 7f0cd74c9640 1 -- 172.21.15.16:0/3137123633 shutdown_connections 2024-09-12T02:17:54.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.452+0000 7f0cd74c9640 1 -- 172.21.15.16:0/3137123633 wait complete. 2024-09-12T02:17:54.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.453+0000 7f0cd74c9640 1 Processor -- start 2024-09-12T02:17:54.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.453+0000 7f0cd74c9640 1 -- start start 2024-09-12T02:17:54.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.453+0000 7f0cd74c9640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0cd01025e0 0x7f0cd0197ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:54.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.453+0000 7f0cd74c9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0cd0104f10 0x7f0cd01983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:54.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.453+0000 7f0cd74c9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0cd0107840 0x7f0cd019f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:54.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.453+0000 7f0cd74c9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0cd010d7c0 con 0x7f0cd0107840 2024-09-12T02:17:54.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.453+0000 7f0cd74c9640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0cd010d640 con 0x7f0cd0104f10 2024-09-12T02:17:54.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.453+0000 7f0cd74c9640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f0cd010d940 con 0x7f0cd01025e0 2024-09-12T02:17:54.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.453+0000 7f0cd5a66640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0cd0107840 0x7f0cd019f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:54.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.453+0000 7f0cd5a66640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0cd0107840 0x7f0cd019f450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54630/0 (socket says 172.21.15.16:54630) 2024-09-12T02:17:54.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.453+0000 7f0cd5a66640 1 -- 172.21.15.16:0/1933043804 learned_addr learned my addr 172.21.15.16:0/1933043804 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:17:54.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.453+0000 7f0cd5265640 1 --2- 172.21.15.16:0/1933043804 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0cd01025e0 0x7f0cd0197ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:54.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.453+0000 7f0cd4a64640 1 --2- 172.21.15.16:0/1933043804 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0cd0104f10 0x7f0cd01983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:54.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.454+0000 7f0cd5a66640 1 -- 172.21.15.16:0/1933043804 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0cd01025e0 msgr2=0x7f0cd0197ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:54.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.454+0000 7f0cd5a66640 1 --2- 172.21.15.16:0/1933043804 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0cd01025e0 0x7f0cd0197ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:54.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.454+0000 7f0cd5a66640 1 -- 172.21.15.16:0/1933043804 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0cd0104f10 msgr2=0x7f0cd01983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:54.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.454+0000 7f0cd5a66640 1 --2- 172.21.15.16:0/1933043804 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0cd0104f10 0x7f0cd01983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:54.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.454+0000 7f0cd5a66640 1 -- 172.21.15.16:0/1933043804 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cd019fac0 con 0x7f0cd0107840 2024-09-12T02:17:54.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.454+0000 7f0cd4a64640 1 --2- 172.21.15.16:0/1933043804 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0cd0104f10 0x7f0cd01983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:17:54.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.454+0000 7f0cd5265640 1 --2- 172.21.15.16:0/1933043804 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0cd01025e0 0x7f0cd0197ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:17:54.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.454+0000 7f0cd5a66640 1 --2- 172.21.15.16:0/1933043804 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0cd0107840 0x7f0cd019f450 secure :-1 s=READY pgs=2148 cs=0 l=1 rev1=1 crypto rx=0x7f0cc802f540 tx=0x7f0cc8031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:54.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.454+0000 7f0cc67fc640 1 -- 172.21.15.16:0/1933043804 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cc8003c30 con 0x7f0cd0107840 2024-09-12T02:17:54.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.454+0000 7f0cc67fc640 1 -- 172.21.15.16:0/1933043804 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0cc8003dd0 con 0x7f0cd0107840 2024-09-12T02:17:54.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.454+0000 7f0cd74c9640 1 -- 172.21.15.16:0/1933043804 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cd019fd50 con 0x7f0cd0107840 2024-09-12T02:17:54.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.454+0000 7f0cc67fc640 1 -- 172.21.15.16:0/1933043804 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cc802fd90 con 0x7f0cd0107840 2024-09-12T02:17:54.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.454+0000 7f0cd74c9640 1 -- 172.21.15.16:0/1933043804 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cd01a0240 con 0x7f0cd0107840 2024-09-12T02:17:54.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.455+0000 7f0cd74c9640 1 -- 172.21.15.16:0/1933043804 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0cd00fdc30 con 0x7f0cd0107840 2024-09-12T02:17:54.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.456+0000 7f0cc67fc640 1 -- 172.21.15.16:0/1933043804 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f0cc8004050 con 0x7f0cd0107840 2024-09-12T02:17:54.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.457+0000 7f0cc67fc640 1 --2- 172.21.15.16:0/1933043804 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f0cbc0a2b90 0x7f0cbc0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:54.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.457+0000 7f0cc67fc640 1 -- 172.21.15.16:0/1933043804 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f0cc80f0450 con 0x7f0cd0107840 2024-09-12T02:17:54.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.457+0000 7f0cd5265640 1 --2- 172.21.15.16:0/1933043804 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f0cbc0a2b90 0x7f0cbc0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:54.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.457+0000 7f0cd5265640 1 --2- 172.21.15.16:0/1933043804 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f0cbc0a2b90 0x7f0cbc0a5050 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f0ccc0058c0 tx=0x7f0ccc005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:54.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.461+0000 7f0cc67fc640 1 -- 172.21.15.16:0/1933043804 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0cc80bd3b0 con 0x7f0cd0107840 2024-09-12T02:17:54.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.636+0000 7f0cd74c9640 1 -- 172.21.15.16:0/1933043804 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f0cd0062e90 con 0x7f0cd0107840 2024-09-12T02:17:54.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.637+0000 7f0cc67fc640 1 -- 172.21.15.16:0/1933043804 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v82) ==== 104+0+11 (secure 0 0 0) 0x7f0cc80bccb0 con 0x7f0cd0107840 2024-09-12T02:17:54.638 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2024-09-12T02:17:54.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.638+0000 7f0cd74c9640 1 -- 172.21.15.16:0/1933043804 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f0cbc0a2b90 msgr2=0x7f0cbc0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:54.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.638+0000 7f0cd74c9640 1 --2- 172.21.15.16:0/1933043804 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f0cbc0a2b90 0x7f0cbc0a5050 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f0ccc0058c0 tx=0x7f0ccc005850 comp rx=0 tx=0).stop 2024-09-12T02:17:54.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.638+0000 7f0cd74c9640 1 -- 172.21.15.16:0/1933043804 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0cd0107840 msgr2=0x7f0cd019f450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:54.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.638+0000 7f0cd74c9640 1 --2- 172.21.15.16:0/1933043804 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0cd0107840 0x7f0cd019f450 secure :-1 s=READY pgs=2148 cs=0 l=1 rev1=1 crypto rx=0x7f0cc802f540 tx=0x7f0cc8031ee0 comp rx=0 tx=0).stop 2024-09-12T02:17:54.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.639+0000 7f0cd74c9640 1 -- 172.21.15.16:0/1933043804 shutdown_connections 2024-09-12T02:17:54.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.639+0000 7f0cd74c9640 1 --2- 172.21.15.16:0/1933043804 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f0cbc0a2b90 0x7f0cbc0a5050 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:54.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.639+0000 7f0cd74c9640 1 --2- 172.21.15.16:0/1933043804 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0cd0107840 0x7f0cd019f450 unknown :-1 s=CLOSED pgs=2148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:54.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.639+0000 7f0cd74c9640 1 --2- 172.21.15.16:0/1933043804 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0cd0104f10 0x7f0cd01983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:54.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.639+0000 7f0cd74c9640 1 --2- 172.21.15.16:0/1933043804 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0cd01025e0 0x7f0cd0197ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:54.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.639+0000 7f0cd74c9640 1 -- 172.21.15.16:0/1933043804 >> 172.21.15.16:0/1933043804 conn(0x7f0cd00fc140 msgr2=0x7f0cd0102f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:54.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.639+0000 7f0cd74c9640 1 -- 172.21.15.16:0/1933043804 shutdown_connections 2024-09-12T02:17:54.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.639+0000 7f0cd74c9640 1 -- 172.21.15.16:0/1933043804 wait complete. 2024-09-12T02:17:54.651 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-12T02:17:54.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.757+0000 7f509b48b640 1 -- 172.21.15.16:0/3280138843 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50941056a0 msgr2=0x7f5094105b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:54.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.757+0000 7f509b48b640 1 --2- 172.21.15.16:0/3280138843 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50941056a0 0x7f5094105b50 secure :-1 s=READY pgs=2149 cs=0 l=1 rev1=1 crypto rx=0x7f508c009ca0 tx=0x7f508c02a8c0 comp rx=0 tx=0).stop 2024-09-12T02:17:54.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.757+0000 7f509b48b640 1 -- 172.21.15.16:0/3280138843 shutdown_connections 2024-09-12T02:17:54.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.757+0000 7f509b48b640 1 --2- 172.21.15.16:0/3280138843 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50941056a0 0x7f5094105b50 unknown :-1 s=CLOSED pgs=2149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:54.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.757+0000 7f509b48b640 1 --2- 172.21.15.16:0/3280138843 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5094102cf0 0x7f5094105160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:54.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.757+0000 7f509b48b640 1 --2- 172.21.15.16:0/3280138843 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f50941003a0 0x7f50941027b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:54.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.757+0000 7f509b48b640 1 -- 172.21.15.16:0/3280138843 >> 172.21.15.16:0/3280138843 conn(0x7f50940fc1b0 msgr2=0x7f50940fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:54.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.757+0000 7f509b48b640 1 -- 172.21.15.16:0/3280138843 shutdown_connections 2024-09-12T02:17:54.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.757+0000 7f509b48b640 1 -- 172.21.15.16:0/3280138843 wait complete. 2024-09-12T02:17:54.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.758+0000 7f509b48b640 1 Processor -- start 2024-09-12T02:17:54.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.758+0000 7f509b48b640 1 -- start start 2024-09-12T02:17:54.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.758+0000 7f509b48b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50941003a0 0x7f509419e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:54.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.758+0000 7f509b48b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5094102cf0 0x7f509419e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:54.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.758+0000 7f509b48b640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f50941056a0 0x7f50941a5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:54.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.758+0000 7f509b48b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5094113de0 con 0x7f50941003a0 2024-09-12T02:17:54.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.758+0000 7f509b48b640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5094113c60 con 0x7f5094102cf0 2024-09-12T02:17:54.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.758+0000 7f509b48b640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5094113f60 con 0x7f50941056a0 2024-09-12T02:17:54.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.758+0000 7f5099227640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50941003a0 0x7f509419e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:54.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.758+0000 7f5099227640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50941003a0 0x7f509419e490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54650/0 (socket says 172.21.15.16:54650) 2024-09-12T02:17:54.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.758+0000 7f5099227640 1 -- 172.21.15.16:0/1569776192 learned_addr learned my addr 172.21.15.16:0/1569776192 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:17:54.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.758+0000 7f5098a26640 1 --2- 172.21.15.16:0/1569776192 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5094102cf0 0x7f509419e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:54.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.758+0000 7f5099a28640 1 --2- 172.21.15.16:0/1569776192 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f50941056a0 0x7f50941a5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:54.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.758+0000 7f5099227640 1 -- 172.21.15.16:0/1569776192 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f50941056a0 msgr2=0x7f50941a5a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:54.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.758+0000 7f5099227640 1 --2- 172.21.15.16:0/1569776192 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f50941056a0 0x7f50941a5a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:54.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.758+0000 7f5099227640 1 -- 172.21.15.16:0/1569776192 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5094102cf0 msgr2=0x7f509419e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:54.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.758+0000 7f5099227640 1 --2- 172.21.15.16:0/1569776192 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5094102cf0 0x7f509419e9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:54.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.758+0000 7f5099227640 1 -- 172.21.15.16:0/1569776192 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50941a60b0 con 0x7f50941003a0 2024-09-12T02:17:54.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.759+0000 7f5099227640 1 --2- 172.21.15.16:0/1569776192 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50941003a0 0x7f509419e490 secure :-1 s=READY pgs=2150 cs=0 l=1 rev1=1 crypto rx=0x7f509000e9f0 tx=0x7f509000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:54.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.759+0000 7f508a7fc640 1 -- 172.21.15.16:0/1569776192 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f509000cd80 con 0x7f50941003a0 2024-09-12T02:17:54.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.759+0000 7f508a7fc640 1 -- 172.21.15.16:0/1569776192 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5090010910 con 0x7f50941003a0 2024-09-12T02:17:54.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.759+0000 7f508a7fc640 1 -- 172.21.15.16:0/1569776192 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5090018990 con 0x7f50941003a0 2024-09-12T02:17:54.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.759+0000 7f509b48b640 1 -- 172.21.15.16:0/1569776192 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50941a63a0 con 0x7f50941003a0 2024-09-12T02:17:54.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.759+0000 7f509b48b640 1 -- 172.21.15.16:0/1569776192 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50941a68a0 con 0x7f50941003a0 2024-09-12T02:17:54.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.760+0000 7f509b48b640 1 -- 172.21.15.16:0/1569776192 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50940fdca0 con 0x7f50941003a0 2024-09-12T02:17:54.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.762+0000 7f508a7fc640 1 -- 172.21.15.16:0/1569776192 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5090010ac0 con 0x7f50941003a0 2024-09-12T02:17:54.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.762+0000 7f508a7fc640 1 --2- 172.21.15.16:0/1569776192 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f50800a2c10 0x7f50800a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:54.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.762+0000 7f5098a26640 1 --2- 172.21.15.16:0/1569776192 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f50800a2c10 0x7f50800a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:54.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.762+0000 7f508a7fc640 1 -- 172.21.15.16:0/1569776192 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f50900cf630 con 0x7f50941003a0 2024-09-12T02:17:54.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.763+0000 7f5098a26640 1 --2- 172.21.15.16:0/1569776192 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f50800a2c10 0x7f50800a50d0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f509419f9a0 tx=0x7f508400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:54.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.766+0000 7f508a7fc640 1 -- 172.21.15.16:0/1569776192 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f509009bfe0 con 0x7f50941003a0 2024-09-12T02:17:54.933 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:17:54 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/1673809096' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T02:17:54.933 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:17:54 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/3675695022' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-12T02:17:54.934 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:17:54 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/1933043804' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-12T02:17:54.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.941+0000 7f509b48b640 1 -- 172.21.15.16:0/1569776192 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f509402da50 con 0x7f50941003a0 2024-09-12T02:17:54.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.941+0000 7f508a7fc640 1 -- 172.21.15.16:0/1569776192 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v82) ==== 100+0+11 (secure 0 0 0) 0x7f50900a02e0 con 0x7f50941003a0 2024-09-12T02:17:54.943 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2024-09-12T02:17:54.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.943+0000 7f509b48b640 1 -- 172.21.15.16:0/1569776192 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f50800a2c10 msgr2=0x7f50800a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:54.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.943+0000 7f509b48b640 1 --2- 172.21.15.16:0/1569776192 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f50800a2c10 0x7f50800a50d0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f509419f9a0 tx=0x7f508400b390 comp rx=0 tx=0).stop 2024-09-12T02:17:54.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.943+0000 7f509b48b640 1 -- 172.21.15.16:0/1569776192 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50941003a0 msgr2=0x7f509419e490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:54.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.943+0000 7f509b48b640 1 --2- 172.21.15.16:0/1569776192 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50941003a0 0x7f509419e490 secure :-1 s=READY pgs=2150 cs=0 l=1 rev1=1 crypto rx=0x7f509000e9f0 tx=0x7f509000eeb0 comp rx=0 tx=0).stop 2024-09-12T02:17:54.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.943+0000 7f509b48b640 1 -- 172.21.15.16:0/1569776192 shutdown_connections 2024-09-12T02:17:54.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.943+0000 7f509b48b640 1 --2- 172.21.15.16:0/1569776192 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f50800a2c10 0x7f50800a50d0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:54.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.943+0000 7f509b48b640 1 --2- 172.21.15.16:0/1569776192 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f50941056a0 0x7f50941a5a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:54.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.943+0000 7f509b48b640 1 --2- 172.21.15.16:0/1569776192 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5094102cf0 0x7f509419e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:54.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.943+0000 7f509b48b640 1 --2- 172.21.15.16:0/1569776192 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50941003a0 0x7f509419e490 unknown :-1 s=CLOSED pgs=2150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:54.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.943+0000 7f509b48b640 1 -- 172.21.15.16:0/1569776192 >> 172.21.15.16:0/1569776192 conn(0x7f50940fc1b0 msgr2=0x7f5094100c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:54.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.943+0000 7f509b48b640 1 -- 172.21.15.16:0/1569776192 shutdown_connections 2024-09-12T02:17:54.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:54.943+0000 7f509b48b640 1 -- 172.21.15.16:0/1569776192 wait complete. 2024-09-12T02:17:54.955 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-09-12T02:17:55.028 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:17:54 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/1673809096' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T02:17:55.029 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:17:54 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/3675695022' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-12T02:17:55.029 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:17:54 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/1933043804' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-12T02:17:55.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:55.057+0000 7f69b40ba640 1 -- 172.21.15.16:0/2592341407 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69ac104ee0 msgr2=0x7f69ac1072d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:55.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:55.057+0000 7f69b40ba640 1 --2- 172.21.15.16:0/2592341407 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69ac104ee0 0x7f69ac1072d0 secure :-1 s=READY pgs=2151 cs=0 l=1 rev1=1 crypto rx=0x7f699c009900 tx=0x7f699c02a410 comp rx=0 tx=0).stop 2024-09-12T02:17:55.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:55.057+0000 7f69b40ba640 1 -- 172.21.15.16:0/2592341407 shutdown_connections 2024-09-12T02:17:55.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:55.057+0000 7f69b40ba640 1 --2- 172.21.15.16:0/2592341407 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f69ac107810 0x7f69ac109c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:55.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:55.057+0000 7f69b40ba640 1 --2- 172.21.15.16:0/2592341407 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69ac104ee0 0x7f69ac1072d0 unknown :-1 s=CLOSED pgs=2151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:55.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:55.057+0000 7f69b40ba640 1 --2- 172.21.15.16:0/2592341407 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f69ac1025b0 0x7f69ac1049a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:55.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:55.057+0000 7f69b40ba640 1 -- 172.21.15.16:0/2592341407 >> 172.21.15.16:0/2592341407 conn(0x7f69ac0fc1b0 msgr2=0x7f69ac0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:55.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:55.057+0000 7f69b40ba640 1 -- 172.21.15.16:0/2592341407 shutdown_connections 2024-09-12T02:17:55.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:55.057+0000 7f69b40ba640 1 -- 172.21.15.16:0/2592341407 wait complete. 2024-09-12T02:17:55.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:55.058+0000 7f69b40ba640 1 Processor -- start 2024-09-12T02:17:55.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:55.058+0000 7f69b40ba640 1 -- start start 2024-09-12T02:17:55.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:55.058+0000 7f69b40ba640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f69ac1025b0 0x7f69ac101f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:55.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:55.058+0000 7f69b40ba640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f69ac104ee0 0x7f69ac100560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:55.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:55.058+0000 7f69b40ba640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69ac107810 0x7f69ac100aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:55.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:55.058+0000 7f69b40ba640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f69ac10d840 con 0x7f69ac107810 2024-09-12T02:17:55.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:55.058+0000 7f69b40ba640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f69ac10d6c0 con 0x7f69ac1025b0 2024-09-12T02:17:55.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:55.058+0000 7f69b40ba640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f69ac10d9c0 con 0x7f69ac104ee0 2024-09-12T02:17:55.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:55.058+0000 7f69b2657640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69ac107810 0x7f69ac100aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:55.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:55.058+0000 7f69b2657640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69ac107810 0x7f69ac100aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54670/0 (socket says 172.21.15.16:54670) 2024-09-12T02:17:55.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:55.058+0000 7f69b2657640 1 -- 172.21.15.16:0/3263547107 learned_addr learned my addr 172.21.15.16:0/3263547107 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:17:55.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:55.058+0000 7f69b1e56640 1 --2- 172.21.15.16:0/3263547107 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f69ac1025b0 0x7f69ac101f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:55.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:55.058+0000 7f69b1655640 1 --2- 172.21.15.16:0/3263547107 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f69ac104ee0 0x7f69ac100560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:55.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:55.058+0000 7f69b2657640 1 -- 172.21.15.16:0/3263547107 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f69ac104ee0 msgr2=0x7f69ac100560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:55.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:55.058+0000 7f69b2657640 1 --2- 172.21.15.16:0/3263547107 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f69ac104ee0 0x7f69ac100560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:55.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:55.058+0000 7f69b2657640 1 -- 172.21.15.16:0/3263547107 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f69ac1025b0 msgr2=0x7f69ac101f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:55.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:55.058+0000 7f69b2657640 1 --2- 172.21.15.16:0/3263547107 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f69ac1025b0 0x7f69ac101f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:55.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:55.058+0000 7f69b2657640 1 -- 172.21.15.16:0/3263547107 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69ac101330 con 0x7f69ac107810 2024-09-12T02:17:55.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:55.059+0000 7f69b2657640 1 --2- 172.21.15.16:0/3263547107 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69ac107810 0x7f69ac100aa0 secure :-1 s=READY pgs=2152 cs=0 l=1 rev1=1 crypto rx=0x7f69a400e9d0 tx=0x7f69a400ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:55.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:55.059+0000 7f69a2ffd640 1 -- 172.21.15.16:0/3263547107 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69a400cd80 con 0x7f69ac107810 2024-09-12T02:17:55.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:55.059+0000 7f69a2ffd640 1 -- 172.21.15.16:0/3263547107 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69a4005020 con 0x7f69ac107810 2024-09-12T02:17:55.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:55.059+0000 7f69b40ba640 1 -- 172.21.15.16:0/3263547107 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69ac1a1fe0 con 0x7f69ac107810 2024-09-12T02:17:55.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:55.059+0000 7f69a2ffd640 1 -- 172.21.15.16:0/3263547107 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69a40116d0 con 0x7f69ac107810 2024-09-12T02:17:55.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:55.059+0000 7f69b40ba640 1 -- 172.21.15.16:0/3263547107 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69ac1a24a0 con 0x7f69ac107810 2024-09-12T02:17:55.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:55.060+0000 7f69b40ba640 1 -- 172.21.15.16:0/3263547107 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69ac0fdca0 con 0x7f69ac107810 2024-09-12T02:17:55.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:55.061+0000 7f69a2ffd640 1 -- 172.21.15.16:0/3263547107 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f69a40051d0 con 0x7f69ac107810 2024-09-12T02:17:55.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:55.062+0000 7f69a2ffd640 1 --2- 172.21.15.16:0/3263547107 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f69980a2c10 0x7f69980a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:55.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:55.062+0000 7f69b1e56640 1 --2- 172.21.15.16:0/3263547107 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f69980a2c10 0x7f69980a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:55.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:55.062+0000 7f69a2ffd640 1 -- 172.21.15.16:0/3263547107 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f69a40cf3c0 con 0x7f69ac107810 2024-09-12T02:17:55.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:55.062+0000 7f69b1e56640 1 --2- 172.21.15.16:0/3263547107 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f69980a2c10 0x7f69980a50d0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f69a800bbe0 tx=0x7f69a800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:55.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:55.066+0000 7f69a2ffd640 1 -- 172.21.15.16:0/3263547107 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f69a4016030 con 0x7f69ac107810 2024-09-12T02:17:55.070 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:17:54 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/1673809096' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T02:17:55.070 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:17:54 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/3675695022' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-12T02:17:55.070 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:17:54 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/1933043804' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-12T02:17:55.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:55.291+0000 7f69b40ba640 1 -- 172.21.15.16:0/3263547107 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f69ac02d9c0 con 0x7f69ac107810 2024-09-12T02:17:55.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:55.299+0000 7f69a2ffd640 1 -- 172.21.15.16:0/3263547107 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v18) ==== 81+0+740370 (secure 0 0 0) 0x7f69a409bd70 con 0x7f69ac107810 2024-09-12T02:17:55.310 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T02:17:55.310 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T02:17:55.310 INFO:teuthology.orchestra.run.smithi016.stdout: "epoch": 18, 2024-09-12T02:17:55.310 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.310 INFO:teuthology.orchestra.run.smithi016.stdout: "active_gid": 14150, 2024-09-12T02:17:55.310 INFO:teuthology.orchestra.run.smithi016.stdout: "active_name": "x", 2024-09-12T02:17:55.310 INFO:teuthology.orchestra.run.smithi016.stdout: "active_addrs": { 2024-09-12T02:17:55.310 INFO:teuthology.orchestra.run.smithi016.stdout: "addrvec": [ 2024-09-12T02:17:55.311 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T02:17:55.311 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v2", 2024-09-12T02:17:55.311 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "172.21.15.16:6800", 2024-09-12T02:17:55.311 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 3887444820 2024-09-12T02:17:55.311 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.311 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T02:17:55.311 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2024-09-12T02:17:55.311 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "172.21.15.16:6801", 2024-09-12T02:17:55.311 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 3887444820 2024-09-12T02:17:55.311 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:55.311 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-09-12T02:17:55.311 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.311 INFO:teuthology.orchestra.run.smithi016.stdout: "active_addr": "172.21.15.16:6801/3887444820", 2024-09-12T02:17:55.312 INFO:teuthology.orchestra.run.smithi016.stdout: "active_change": "2024-09-12T01:53:08.218622+0000", 2024-09-12T02:17:55.312 INFO:teuthology.orchestra.run.smithi016.stdout: "active_mgr_features": 4540701547738038271, 2024-09-12T02:17:55.312 INFO:teuthology.orchestra.run.smithi016.stdout: "available": true, 2024-09-12T02:17:55.312 INFO:teuthology.orchestra.run.smithi016.stdout: "standbys": [ 2024-09-12T02:17:55.312 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T02:17:55.312 INFO:teuthology.orchestra.run.smithi016.stdout: "gid": 24109, 2024-09-12T02:17:55.312 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "z", 2024-09-12T02:17:55.312 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr_features": 4540701547738038271, 2024-09-12T02:17:55.312 INFO:teuthology.orchestra.run.smithi016.stdout: "available_modules": [ 2024-09-12T02:17:55.312 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T02:17:55.312 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "alerts", 2024-09-12T02:17:55.312 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T02:17:55.313 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T02:17:55.313 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T02:17:55.313 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-09-12T02:17:55.313 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-09-12T02:17:55.313 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-12T02:17:55.313 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.313 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.313 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-09-12T02:17:55.313 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.313 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.313 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.313 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "How frequently to reexamine health status", 2024-09-12T02:17:55.313 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.314 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.314 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.314 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.314 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T02:17:55.314 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T02:17:55.314 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.314 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.314 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.314 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:55.314 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.314 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.314 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:55.314 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:55.314 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:55.315 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:55.315 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:55.315 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:55.315 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:55.315 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:55.315 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.315 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.315 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.315 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.315 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.315 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T02:17:55.315 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T02:17:55.316 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.316 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.316 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.316 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:55.316 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.316 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.316 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.316 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.316 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.316 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.316 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.316 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.316 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T02:17:55.317 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T02:17:55.317 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.317 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.317 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.317 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T02:17:55.317 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.317 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.317 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:55.317 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:55.317 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:55.317 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:55.317 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:55.317 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:55.318 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:55.318 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:55.318 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.318 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.318 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.318 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.318 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.318 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T02:17:55.318 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T02:17:55.318 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.318 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.318 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.318 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:55.319 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.319 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.319 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.319 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.319 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.319 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.319 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.319 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.319 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_destination": { 2024-09-12T02:17:55.319 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_destination", 2024-09-12T02:17:55.319 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.319 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.319 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.319 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:55.320 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.320 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.320 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.320 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Email address to send alerts to", 2024-09-12T02:17:55.320 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.320 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.320 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.320 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.320 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_from_name": { 2024-09-12T02:17:55.320 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_from_name", 2024-09-12T02:17:55.320 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.320 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.320 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.320 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "Ceph", 2024-09-12T02:17:55.321 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.321 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.321 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.321 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Email From: name", 2024-09-12T02:17:55.321 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.321 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.321 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.321 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.321 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_host": { 2024-09-12T02:17:55.321 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_host", 2024-09-12T02:17:55.321 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.321 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.321 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.322 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:55.322 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.322 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.322 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.322 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "SMTP server", 2024-09-12T02:17:55.322 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.322 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.322 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.322 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.322 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_password": { 2024-09-12T02:17:55.322 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_password", 2024-09-12T02:17:55.322 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.322 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.323 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.323 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:55.323 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.323 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.323 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.323 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Password to authenticate with", 2024-09-12T02:17:55.323 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.323 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.323 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.323 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.323 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_port": { 2024-09-12T02:17:55.323 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_port", 2024-09-12T02:17:55.323 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T02:17:55.324 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.324 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.324 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "465", 2024-09-12T02:17:55.324 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.324 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.324 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.324 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "SMTP port", 2024-09-12T02:17:55.324 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.324 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.324 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.324 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.324 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_sender": { 2024-09-12T02:17:55.325 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_sender", 2024-09-12T02:17:55.325 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.325 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.325 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.325 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:55.325 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.325 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.325 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.325 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "SMTP envelope sender", 2024-09-12T02:17:55.325 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.325 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.326 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.326 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.326 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_ssl": { 2024-09-12T02:17:55.326 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_ssl", 2024-09-12T02:17:55.326 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.326 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.326 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.326 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T02:17:55.327 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.327 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.327 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.327 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-12T02:17:55.327 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.327 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.327 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.327 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.327 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_user": { 2024-09-12T02:17:55.327 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_user", 2024-09-12T02:17:55.327 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.327 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.327 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.328 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:55.328 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.328 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.328 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.328 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "User to authenticate as", 2024-09-12T02:17:55.328 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.328 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.328 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.328 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:55.328 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:55.328 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.329 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T02:17:55.329 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "balancer", 2024-09-12T02:17:55.329 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T02:17:55.329 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T02:17:55.329 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T02:17:55.329 INFO:teuthology.orchestra.run.smithi016.stdout: "active": { 2024-09-12T02:17:55.329 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "active", 2024-09-12T02:17:55.329 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.329 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.329 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.329 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T02:17:55.329 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.330 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.330 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.330 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "automatically balance PGs across cluster", 2024-09-12T02:17:55.330 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.330 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.330 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.330 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.330 INFO:teuthology.orchestra.run.smithi016.stdout: "begin_time": { 2024-09-12T02:17:55.330 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "begin_time", 2024-09-12T02:17:55.330 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.330 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.330 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.330 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0000", 2024-09-12T02:17:55.331 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.331 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.331 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.331 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "beginning time of day to automatically balance", 2024-09-12T02:17:55.331 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-12T02:17:55.331 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.331 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.331 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.331 INFO:teuthology.orchestra.run.smithi016.stdout: "begin_weekday": { 2024-09-12T02:17:55.331 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "begin_weekday", 2024-09-12T02:17:55.331 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-09-12T02:17:55.331 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.331 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.331 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-12T02:17:55.332 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "0", 2024-09-12T02:17:55.332 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "6", 2024-09-12T02:17:55.332 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.332 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-12T02:17:55.332 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-12T02:17:55.332 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.332 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.332 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.332 INFO:teuthology.orchestra.run.smithi016.stdout: "crush_compat_max_iterations": { 2024-09-12T02:17:55.332 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crush_compat_max_iterations", 2024-09-12T02:17:55.332 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-09-12T02:17:55.332 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.332 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.333 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "25", 2024-09-12T02:17:55.333 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1", 2024-09-12T02:17:55.333 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "250", 2024-09-12T02:17:55.333 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.333 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-12T02:17:55.333 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.333 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.333 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.333 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.333 INFO:teuthology.orchestra.run.smithi016.stdout: "crush_compat_metrics": { 2024-09-12T02:17:55.333 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crush_compat_metrics", 2024-09-12T02:17:55.333 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.333 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.334 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.334 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "pgs,objects,bytes", 2024-09-12T02:17:55.334 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.334 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.334 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.334 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-12T02:17:55.334 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-12T02:17:55.334 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.334 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.334 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.335 INFO:teuthology.orchestra.run.smithi016.stdout: "crush_compat_step": { 2024-09-12T02:17:55.335 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crush_compat_step", 2024-09-12T02:17:55.335 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-09-12T02:17:55.335 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.335 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.335 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0.5", 2024-09-12T02:17:55.335 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "0.001", 2024-09-12T02:17:55.335 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "0.999", 2024-09-12T02:17:55.335 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.335 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "aggressiveness of optimization", 2024-09-12T02:17:55.335 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-12T02:17:55.336 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.336 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.336 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.336 INFO:teuthology.orchestra.run.smithi016.stdout: "end_time": { 2024-09-12T02:17:55.336 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "end_time", 2024-09-12T02:17:55.336 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.336 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.336 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.336 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "2359", 2024-09-12T02:17:55.336 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.336 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.336 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.337 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "ending time of day to automatically balance", 2024-09-12T02:17:55.337 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-12T02:17:55.337 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.337 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.337 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.337 INFO:teuthology.orchestra.run.smithi016.stdout: "end_weekday": { 2024-09-12T02:17:55.337 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "end_weekday", 2024-09-12T02:17:55.337 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-09-12T02:17:55.337 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.337 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.337 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-12T02:17:55.337 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "0", 2024-09-12T02:17:55.337 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "6", 2024-09-12T02:17:55.338 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.338 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-12T02:17:55.338 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-12T02:17:55.338 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.338 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.338 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.338 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T02:17:55.338 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T02:17:55.338 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.338 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.338 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.338 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:55.338 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.339 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.339 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:55.339 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:55.339 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:55.339 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:55.339 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:55.339 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:55.339 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:55.339 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:55.339 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.339 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.339 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.339 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.339 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.340 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T02:17:55.340 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T02:17:55.340 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.340 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.340 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.340 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:55.340 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.340 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.340 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.340 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.340 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.341 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.341 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.341 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.341 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T02:17:55.341 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T02:17:55.341 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.341 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.341 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.341 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T02:17:55.341 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.341 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.341 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:55.341 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:55.341 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:55.342 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:55.342 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:55.342 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:55.342 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:55.342 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:55.342 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.342 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.342 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.342 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.342 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.342 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T02:17:55.342 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T02:17:55.342 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.343 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.343 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.343 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:55.343 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.343 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.343 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.343 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.343 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.343 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.343 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.343 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.343 INFO:teuthology.orchestra.run.smithi016.stdout: "min_score": { 2024-09-12T02:17:55.343 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "min_score", 2024-09-12T02:17:55.343 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-09-12T02:17:55.344 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.344 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.344 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-12T02:17:55.344 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.344 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.344 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.344 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-12T02:17:55.344 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.344 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.344 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.344 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.344 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": { 2024-09-12T02:17:55.344 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mode", 2024-09-12T02:17:55.345 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.345 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.345 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.345 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "upmap", 2024-09-12T02:17:55.345 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.345 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.345 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:55.345 INFO:teuthology.orchestra.run.smithi016.stdout: "crush-compat", 2024-09-12T02:17:55.345 INFO:teuthology.orchestra.run.smithi016.stdout: "none", 2024-09-12T02:17:55.345 INFO:teuthology.orchestra.run.smithi016.stdout: "read", 2024-09-12T02:17:55.345 INFO:teuthology.orchestra.run.smithi016.stdout: "upmap", 2024-09-12T02:17:55.345 INFO:teuthology.orchestra.run.smithi016.stdout: "upmap-read" 2024-09-12T02:17:55.346 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:55.346 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Balancer mode", 2024-09-12T02:17:55.346 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.346 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.346 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.346 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.346 INFO:teuthology.orchestra.run.smithi016.stdout: "pool_ids": { 2024-09-12T02:17:55.346 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "pool_ids", 2024-09-12T02:17:55.346 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.346 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.346 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.346 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:55.346 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.347 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.347 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.347 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-12T02:17:55.347 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.347 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.347 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.347 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.347 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-09-12T02:17:55.347 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-09-12T02:17:55.347 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-12T02:17:55.347 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.347 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.347 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-09-12T02:17:55.348 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.348 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.348 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.348 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-12T02:17:55.348 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.348 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.348 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.348 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.348 INFO:teuthology.orchestra.run.smithi016.stdout: "upmap_max_deviation": { 2024-09-12T02:17:55.348 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "upmap_max_deviation", 2024-09-12T02:17:55.348 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T02:17:55.348 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.348 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.349 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5", 2024-09-12T02:17:55.349 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1", 2024-09-12T02:17:55.349 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.349 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.349 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-12T02:17:55.349 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-12T02:17:55.349 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.349 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.349 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.349 INFO:teuthology.orchestra.run.smithi016.stdout: "upmap_max_optimizations": { 2024-09-12T02:17:55.349 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "upmap_max_optimizations", 2024-09-12T02:17:55.349 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-09-12T02:17:55.349 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.350 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.350 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-12T02:17:55.350 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.350 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.350 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.350 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-12T02:17:55.350 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.350 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.350 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.350 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:55.350 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:55.350 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.351 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T02:17:55.351 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cephadm", 2024-09-12T02:17:55.351 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T02:17:55.351 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T02:17:55.351 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T02:17:55.351 INFO:teuthology.orchestra.run.smithi016.stdout: "agent_down_multiplier": { 2024-09-12T02:17:55.351 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "agent_down_multiplier", 2024-09-12T02:17:55.351 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-09-12T02:17:55.351 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.351 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.351 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "3.0", 2024-09-12T02:17:55.351 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.351 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.352 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.352 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-12T02:17:55.352 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.352 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.352 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.352 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.352 INFO:teuthology.orchestra.run.smithi016.stdout: "agent_refresh_rate": { 2024-09-12T02:17:55.352 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "agent_refresh_rate", 2024-09-12T02:17:55.352 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-12T02:17:55.352 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.352 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.352 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "20", 2024-09-12T02:17:55.353 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.353 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.353 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.353 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-12T02:17:55.353 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.353 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.353 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.353 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.353 INFO:teuthology.orchestra.run.smithi016.stdout: "agent_starting_port": { 2024-09-12T02:17:55.353 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "agent_starting_port", 2024-09-12T02:17:55.353 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T02:17:55.353 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.354 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.354 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "4721", 2024-09-12T02:17:55.354 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.354 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.354 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.354 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-12T02:17:55.354 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.354 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.354 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.354 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.354 INFO:teuthology.orchestra.run.smithi016.stdout: "allow_ptrace": { 2024-09-12T02:17:55.355 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "allow_ptrace", 2024-09-12T02:17:55.355 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.355 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.355 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.355 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:55.355 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.355 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.356 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.356 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-12T02:17:55.356 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-12T02:17:55.356 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.356 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.356 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.356 INFO:teuthology.orchestra.run.smithi016.stdout: "autotune_interval": { 2024-09-12T02:17:55.356 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "autotune_interval", 2024-09-12T02:17:55.356 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-12T02:17:55.356 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.356 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.357 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-09-12T02:17:55.357 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.357 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.357 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.357 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-12T02:17:55.357 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.357 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.357 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.357 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.357 INFO:teuthology.orchestra.run.smithi016.stdout: "autotune_memory_target_ratio": { 2024-09-12T02:17:55.357 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "autotune_memory_target_ratio", 2024-09-12T02:17:55.357 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-09-12T02:17:55.357 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.358 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.358 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0.7", 2024-09-12T02:17:55.358 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.358 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.358 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.358 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-12T02:17:55.358 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.358 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.358 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.358 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.358 INFO:teuthology.orchestra.run.smithi016.stdout: "cephadm_log_destination": { 2024-09-12T02:17:55.358 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cephadm_log_destination", 2024-09-12T02:17:55.358 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.359 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.359 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.359 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:55.359 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.359 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.359 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:55.359 INFO:teuthology.orchestra.run.smithi016.stdout: "file", 2024-09-12T02:17:55.359 INFO:teuthology.orchestra.run.smithi016.stdout: "file,syslog", 2024-09-12T02:17:55.359 INFO:teuthology.orchestra.run.smithi016.stdout: "syslog" 2024-09-12T02:17:55.359 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:55.359 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-12T02:17:55.359 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.359 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.359 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.360 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.360 INFO:teuthology.orchestra.run.smithi016.stdout: "cgroups_split": { 2024-09-12T02:17:55.360 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cgroups_split", 2024-09-12T02:17:55.360 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.360 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.360 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.360 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T02:17:55.360 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.360 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.360 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.360 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-12T02:17:55.360 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.361 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.361 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.361 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.361 INFO:teuthology.orchestra.run.smithi016.stdout: "config_checks_enabled": { 2024-09-12T02:17:55.361 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "config_checks_enabled", 2024-09-12T02:17:55.361 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.361 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.361 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.361 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:55.361 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.361 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.361 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.361 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-12T02:17:55.362 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.362 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.362 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.362 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.362 INFO:teuthology.orchestra.run.smithi016.stdout: "config_dashboard": { 2024-09-12T02:17:55.362 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "config_dashboard", 2024-09-12T02:17:55.362 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.362 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.362 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.362 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T02:17:55.362 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.362 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.363 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.363 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-12T02:17:55.363 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.363 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.363 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.363 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.363 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_alertmanager": { 2024-09-12T02:17:55.363 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_alertmanager", 2024-09-12T02:17:55.363 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.363 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.363 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.363 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-12T02:17:55.363 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.364 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.364 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.364 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Prometheus container image", 2024-09-12T02:17:55.364 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.364 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.364 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.364 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.364 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_base": { 2024-09-12T02:17:55.364 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_base", 2024-09-12T02:17:55.364 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.364 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.364 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.365 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-12T02:17:55.365 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.365 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.365 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.365 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Container image name, without the tag", 2024-09-12T02:17:55.365 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.365 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.365 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.365 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.369 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_elasticsearch": { 2024-09-12T02:17:55.369 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_elasticsearch", 2024-09-12T02:17:55.369 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.369 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.369 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.369 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-12T02:17:55.369 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.369 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.369 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.370 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "elasticsearch container image", 2024-09-12T02:17:55.370 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.370 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.370 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.370 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.370 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_grafana": { 2024-09-12T02:17:55.370 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_grafana", 2024-09-12T02:17:55.370 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.370 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.370 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.370 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-12T02:17:55.370 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.370 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.371 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.371 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Prometheus container image", 2024-09-12T02:17:55.371 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.371 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.371 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.371 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.371 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_haproxy": { 2024-09-12T02:17:55.371 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_haproxy", 2024-09-12T02:17:55.371 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.371 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.371 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.371 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-12T02:17:55.372 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.372 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.372 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.372 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "HAproxy container image", 2024-09-12T02:17:55.372 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.372 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.372 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.372 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.372 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_jaeger_agent": { 2024-09-12T02:17:55.372 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_jaeger_agent", 2024-09-12T02:17:55.373 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.373 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.373 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.373 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-12T02:17:55.373 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.373 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.373 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.373 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Jaeger agent container image", 2024-09-12T02:17:55.373 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.373 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.373 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.373 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.374 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_jaeger_collector": { 2024-09-12T02:17:55.374 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_jaeger_collector", 2024-09-12T02:17:55.374 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.374 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.374 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.374 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-12T02:17:55.374 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.374 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.374 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.374 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Jaeger collector container image", 2024-09-12T02:17:55.374 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.374 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.374 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.375 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.375 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_jaeger_query": { 2024-09-12T02:17:55.375 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_jaeger_query", 2024-09-12T02:17:55.375 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.375 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.375 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.375 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-12T02:17:55.375 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.375 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.375 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.375 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Jaeger query container image", 2024-09-12T02:17:55.375 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.375 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.375 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.376 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.376 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_keepalived": { 2024-09-12T02:17:55.376 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_keepalived", 2024-09-12T02:17:55.376 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.376 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.376 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.376 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-12T02:17:55.376 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.376 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.376 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.376 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Keepalived container image", 2024-09-12T02:17:55.376 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.377 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.377 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.377 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.377 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_loki": { 2024-09-12T02:17:55.377 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_loki", 2024-09-12T02:17:55.377 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.377 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.377 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.377 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-12T02:17:55.377 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.377 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.377 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.377 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Loki container image", 2024-09-12T02:17:55.378 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.378 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.378 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.378 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.378 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_node_exporter": { 2024-09-12T02:17:55.378 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_node_exporter", 2024-09-12T02:17:55.378 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.378 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.378 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.378 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-12T02:17:55.378 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.378 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.378 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.379 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Prometheus container image", 2024-09-12T02:17:55.379 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.379 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.379 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.379 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.379 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_nvmeof": { 2024-09-12T02:17:55.379 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_nvmeof", 2024-09-12T02:17:55.379 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.379 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.379 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.379 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-12T02:17:55.379 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.379 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.380 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.380 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Nvme-of container image", 2024-09-12T02:17:55.380 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.380 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.380 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.380 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.380 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_prometheus": { 2024-09-12T02:17:55.380 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_prometheus", 2024-09-12T02:17:55.380 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.380 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.380 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.380 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-12T02:17:55.381 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.381 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.381 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.381 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Prometheus container image", 2024-09-12T02:17:55.381 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.381 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.381 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.381 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.381 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_promtail": { 2024-09-12T02:17:55.381 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_promtail", 2024-09-12T02:17:55.381 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.381 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.382 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.382 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-12T02:17:55.382 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.382 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.382 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.382 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Promtail container image", 2024-09-12T02:17:55.382 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.382 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.382 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.382 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.382 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_samba": { 2024-09-12T02:17:55.382 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_samba", 2024-09-12T02:17:55.382 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.383 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.383 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.383 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-12T02:17:55.383 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.383 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.383 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.383 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Samba/SMB container image", 2024-09-12T02:17:55.383 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.383 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.383 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.383 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.383 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_snmp_gateway": { 2024-09-12T02:17:55.384 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_snmp_gateway", 2024-09-12T02:17:55.384 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.384 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.384 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.384 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-12T02:17:55.384 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.384 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.384 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.384 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "SNMP Gateway container image", 2024-09-12T02:17:55.384 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.384 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.384 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.384 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.384 INFO:teuthology.orchestra.run.smithi016.stdout: "container_init": { 2024-09-12T02:17:55.385 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_init", 2024-09-12T02:17:55.385 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.385 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.385 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.385 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T02:17:55.385 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.385 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.385 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.385 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Run podman/docker with `--init`", 2024-09-12T02:17:55.385 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.385 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.386 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.386 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.386 INFO:teuthology.orchestra.run.smithi016.stdout: "daemon_cache_timeout": { 2024-09-12T02:17:55.386 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "daemon_cache_timeout", 2024-09-12T02:17:55.386 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-12T02:17:55.386 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.386 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.386 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-09-12T02:17:55.386 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.386 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.386 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.386 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-12T02:17:55.386 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.387 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.387 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.387 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.387 INFO:teuthology.orchestra.run.smithi016.stdout: "default_cephadm_command_timeout": { 2024-09-12T02:17:55.387 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "default_cephadm_command_timeout", 2024-09-12T02:17:55.387 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T02:17:55.387 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.387 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.387 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "900", 2024-09-12T02:17:55.387 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.387 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.387 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.387 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-12T02:17:55.388 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.388 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.388 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.388 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.388 INFO:teuthology.orchestra.run.smithi016.stdout: "default_registry": { 2024-09-12T02:17:55.388 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "default_registry", 2024-09-12T02:17:55.388 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.388 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.388 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.388 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "docker.io", 2024-09-12T02:17:55.388 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.388 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.388 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.389 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-12T02:17:55.389 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.389 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.389 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.389 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.389 INFO:teuthology.orchestra.run.smithi016.stdout: "device_cache_timeout": { 2024-09-12T02:17:55.389 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "device_cache_timeout", 2024-09-12T02:17:55.389 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-12T02:17:55.389 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.389 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.389 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "1800", 2024-09-12T02:17:55.389 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.389 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.390 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.390 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "seconds to cache device inventory", 2024-09-12T02:17:55.390 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.390 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.390 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.390 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.390 INFO:teuthology.orchestra.run.smithi016.stdout: "device_enhanced_scan": { 2024-09-12T02:17:55.390 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "device_enhanced_scan", 2024-09-12T02:17:55.390 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.390 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.390 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.390 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:55.390 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.391 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.391 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.391 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-12T02:17:55.391 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.391 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.391 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.391 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.391 INFO:teuthology.orchestra.run.smithi016.stdout: "facts_cache_timeout": { 2024-09-12T02:17:55.391 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "facts_cache_timeout", 2024-09-12T02:17:55.391 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-12T02:17:55.391 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.391 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.391 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-09-12T02:17:55.391 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.392 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.392 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.392 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "seconds to cache host facts data", 2024-09-12T02:17:55.392 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.392 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.392 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.392 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.392 INFO:teuthology.orchestra.run.smithi016.stdout: "grafana_dashboards_path": { 2024-09-12T02:17:55.392 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "grafana_dashboards_path", 2024-09-12T02:17:55.392 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.392 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.392 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.392 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-12T02:17:55.393 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.393 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.393 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.393 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-12T02:17:55.393 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.393 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.393 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.393 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.393 INFO:teuthology.orchestra.run.smithi016.stdout: "host_check_interval": { 2024-09-12T02:17:55.393 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "host_check_interval", 2024-09-12T02:17:55.393 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-12T02:17:55.393 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.393 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.394 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-09-12T02:17:55.394 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.394 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.394 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.394 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to perform a host check", 2024-09-12T02:17:55.394 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.394 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.394 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.394 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.394 INFO:teuthology.orchestra.run.smithi016.stdout: "hw_monitoring": { 2024-09-12T02:17:55.394 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "hw_monitoring", 2024-09-12T02:17:55.394 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.394 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.394 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.395 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:55.395 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.395 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.395 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.395 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-12T02:17:55.395 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.395 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.395 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.395 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.395 INFO:teuthology.orchestra.run.smithi016.stdout: "inventory_list_all": { 2024-09-12T02:17:55.395 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "inventory_list_all", 2024-09-12T02:17:55.395 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.395 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.396 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.396 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:55.396 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.396 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.396 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.396 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-12T02:17:55.396 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.396 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.396 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.396 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.396 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T02:17:55.396 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T02:17:55.397 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.397 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.397 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.397 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:55.397 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.397 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.397 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:55.397 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:55.397 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:55.397 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:55.397 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:55.397 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:55.397 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:55.398 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:55.398 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.398 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.398 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.398 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.398 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.398 INFO:teuthology.orchestra.run.smithi016.stdout: "log_refresh_metadata": { 2024-09-12T02:17:55.398 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_refresh_metadata", 2024-09-12T02:17:55.398 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.398 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.398 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.398 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:55.399 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.399 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.399 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.399 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-12T02:17:55.399 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.399 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.399 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.399 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.399 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T02:17:55.399 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T02:17:55.399 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.399 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.400 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.400 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T02:17:55.400 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.400 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.400 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.400 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-12T02:17:55.400 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.400 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.400 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.400 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.400 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T02:17:55.400 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T02:17:55.401 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.401 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.401 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.401 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T02:17:55.401 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.401 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.401 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:55.401 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:55.401 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:55.401 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:55.401 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:55.401 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:55.401 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:55.402 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:55.402 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.402 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.402 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.402 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.402 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.402 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T02:17:55.402 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T02:17:55.402 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.402 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.402 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.402 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:55.402 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.403 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.403 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.403 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.403 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.403 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.403 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.403 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.403 INFO:teuthology.orchestra.run.smithi016.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-12T02:17:55.403 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-12T02:17:55.403 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.403 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.403 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.403 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:55.404 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.404 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.404 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.404 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-12T02:17:55.404 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.404 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.404 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.404 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.404 INFO:teuthology.orchestra.run.smithi016.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-12T02:17:55.404 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-12T02:17:55.404 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.404 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.405 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.405 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "*", 2024-09-12T02:17:55.405 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.405 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.405 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.405 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-12T02:17:55.405 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.405 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.405 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.405 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.405 INFO:teuthology.orchestra.run.smithi016.stdout: "max_count_per_host": { 2024-09-12T02:17:55.405 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_count_per_host", 2024-09-12T02:17:55.406 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T02:17:55.406 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.406 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.406 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-12T02:17:55.406 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.406 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.406 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.406 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "max number of daemons per service per host", 2024-09-12T02:17:55.406 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.406 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.406 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.406 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.406 INFO:teuthology.orchestra.run.smithi016.stdout: "max_osd_draining_count": { 2024-09-12T02:17:55.407 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_osd_draining_count", 2024-09-12T02:17:55.407 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T02:17:55.407 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.407 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.407 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-12T02:17:55.407 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.407 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.407 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.407 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-12T02:17:55.407 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.407 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.407 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.407 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.408 INFO:teuthology.orchestra.run.smithi016.stdout: "migration_current": { 2024-09-12T02:17:55.408 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "migration_current", 2024-09-12T02:17:55.408 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T02:17:55.408 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.408 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.408 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:55.408 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.408 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.408 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.408 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "internal - do not modify", 2024-09-12T02:17:55.408 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.408 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.408 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.409 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.409 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": { 2024-09-12T02:17:55.409 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mode", 2024-09-12T02:17:55.409 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.409 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.409 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.409 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "root", 2024-09-12T02:17:55.409 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.409 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.409 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:55.409 INFO:teuthology.orchestra.run.smithi016.stdout: "cephadm-package", 2024-09-12T02:17:55.409 INFO:teuthology.orchestra.run.smithi016.stdout: "root" 2024-09-12T02:17:55.409 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:55.410 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "mode for remote execution of cephadm", 2024-09-12T02:17:55.410 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.410 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.410 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.410 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.410 INFO:teuthology.orchestra.run.smithi016.stdout: "oob_default_addr": { 2024-09-12T02:17:55.410 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "oob_default_addr", 2024-09-12T02:17:55.410 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.410 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.410 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.410 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "169.254.1.1", 2024-09-12T02:17:55.410 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.410 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.411 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.411 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-12T02:17:55.411 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.411 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.411 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.411 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.411 INFO:teuthology.orchestra.run.smithi016.stdout: "prometheus_alerts_path": { 2024-09-12T02:17:55.411 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "prometheus_alerts_path", 2024-09-12T02:17:55.411 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.411 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.411 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.411 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-12T02:17:55.411 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.412 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.412 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.412 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-12T02:17:55.412 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.412 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.412 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.412 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.412 INFO:teuthology.orchestra.run.smithi016.stdout: "registry_insecure": { 2024-09-12T02:17:55.412 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "registry_insecure", 2024-09-12T02:17:55.412 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.412 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.413 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.413 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:55.413 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.413 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.413 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.413 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-12T02:17:55.413 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.413 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.413 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.413 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.413 INFO:teuthology.orchestra.run.smithi016.stdout: "registry_password": { 2024-09-12T02:17:55.413 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "registry_password", 2024-09-12T02:17:55.414 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.414 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.414 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.414 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:55.414 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.414 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.414 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.414 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-12T02:17:55.414 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.414 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.414 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.414 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.414 INFO:teuthology.orchestra.run.smithi016.stdout: "registry_url": { 2024-09-12T02:17:55.415 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "registry_url", 2024-09-12T02:17:55.415 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.415 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.415 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.415 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:55.415 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.415 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.415 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.415 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-12T02:17:55.415 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.415 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.415 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.415 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.416 INFO:teuthology.orchestra.run.smithi016.stdout: "registry_username": { 2024-09-12T02:17:55.416 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "registry_username", 2024-09-12T02:17:55.416 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.416 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.416 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.416 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:55.416 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.416 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.416 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.416 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-12T02:17:55.416 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.416 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.416 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.417 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.417 INFO:teuthology.orchestra.run.smithi016.stdout: "secure_monitoring_stack": { 2024-09-12T02:17:55.417 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "secure_monitoring_stack", 2024-09-12T02:17:55.417 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.417 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.417 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.417 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:55.417 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.417 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.417 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.417 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-12T02:17:55.417 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.418 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.418 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.418 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.418 INFO:teuthology.orchestra.run.smithi016.stdout: "service_discovery_port": { 2024-09-12T02:17:55.418 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "service_discovery_port", 2024-09-12T02:17:55.418 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T02:17:55.418 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.418 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.418 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8765", 2024-09-12T02:17:55.418 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.418 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.418 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.418 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "cephadm service discovery port", 2024-09-12T02:17:55.419 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.419 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.419 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.419 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.419 INFO:teuthology.orchestra.run.smithi016.stdout: "ssh_config_file": { 2024-09-12T02:17:55.419 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ssh_config_file", 2024-09-12T02:17:55.419 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.419 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.419 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.419 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:55.419 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.419 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.419 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.420 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-12T02:17:55.420 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.420 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.420 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.420 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.420 INFO:teuthology.orchestra.run.smithi016.stdout: "use_agent": { 2024-09-12T02:17:55.420 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "use_agent", 2024-09-12T02:17:55.420 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.420 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.420 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.420 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:55.420 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.420 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.421 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.421 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-12T02:17:55.421 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.421 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.421 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.421 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.421 INFO:teuthology.orchestra.run.smithi016.stdout: "use_repo_digest": { 2024-09-12T02:17:55.421 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "use_repo_digest", 2024-09-12T02:17:55.421 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.421 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.421 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.421 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T02:17:55.422 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.422 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.422 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.422 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-12T02:17:55.422 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.422 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.422 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.422 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.422 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_on_failed_host_check": { 2024-09-12T02:17:55.422 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_on_failed_host_check", 2024-09-12T02:17:55.422 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.422 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.422 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.423 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T02:17:55.423 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.423 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.423 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.423 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "raise a health warning if the host check fails", 2024-09-12T02:17:55.423 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.423 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.423 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.423 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.423 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_on_stray_daemons": { 2024-09-12T02:17:55.423 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_on_stray_daemons", 2024-09-12T02:17:55.423 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.423 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.424 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.424 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T02:17:55.424 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.424 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.424 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.424 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-12T02:17:55.424 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.424 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.424 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.424 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.424 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_on_stray_hosts": { 2024-09-12T02:17:55.424 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_on_stray_hosts", 2024-09-12T02:17:55.424 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.425 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.425 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.425 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T02:17:55.425 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.425 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.425 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.425 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-12T02:17:55.425 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.425 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.425 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.425 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:55.425 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:55.426 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.426 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T02:17:55.426 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crash", 2024-09-12T02:17:55.426 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T02:17:55.426 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T02:17:55.426 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T02:17:55.426 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T02:17:55.426 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T02:17:55.426 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.426 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.426 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.426 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:55.427 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.427 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.427 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:55.427 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:55.427 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:55.427 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:55.427 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:55.427 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:55.427 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:55.427 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:55.427 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.427 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.427 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.428 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.428 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.428 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T02:17:55.428 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T02:17:55.428 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.428 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.428 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.428 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:55.428 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.428 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.428 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.428 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.429 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.429 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.429 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.429 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.429 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T02:17:55.429 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T02:17:55.429 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.429 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.429 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.429 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T02:17:55.429 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.429 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.430 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:55.430 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:55.430 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:55.430 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:55.430 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:55.430 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:55.430 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:55.430 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:55.430 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.430 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.430 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.430 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.430 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.431 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T02:17:55.431 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T02:17:55.431 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.431 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.431 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.431 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:55.431 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.431 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.431 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.431 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.431 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.431 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.431 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.432 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.432 INFO:teuthology.orchestra.run.smithi016.stdout: "retain_interval": { 2024-09-12T02:17:55.432 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "retain_interval", 2024-09-12T02:17:55.432 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-12T02:17:55.432 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.432 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.432 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "31536000", 2024-09-12T02:17:55.432 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.432 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.432 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.432 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-12T02:17:55.432 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.433 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.433 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.433 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.433 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_recent_interval": { 2024-09-12T02:17:55.433 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_recent_interval", 2024-09-12T02:17:55.433 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-12T02:17:55.433 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.433 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.433 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "1209600", 2024-09-12T02:17:55.433 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.433 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.433 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.433 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-12T02:17:55.434 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.434 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.434 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.434 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:55.434 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:55.434 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.434 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T02:17:55.434 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "dashboard", 2024-09-12T02:17:55.434 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T02:17:55.434 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T02:17:55.434 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T02:17:55.434 INFO:teuthology.orchestra.run.smithi016.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-12T02:17:55.434 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-12T02:17:55.435 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T02:17:55.435 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.435 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.435 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-12T02:17:55.435 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.435 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.435 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.435 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.435 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.435 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.435 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.436 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.436 INFO:teuthology.orchestra.run.smithi016.stdout: "ALERTMANAGER_API_HOST": { 2024-09-12T02:17:55.436 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-12T02:17:55.436 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.436 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.436 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.436 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:55.436 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.436 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.436 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.436 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.436 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.437 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.437 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.437 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.437 INFO:teuthology.orchestra.run.smithi016.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-12T02:17:55.437 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-12T02:17:55.437 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.437 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.437 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.437 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T02:17:55.437 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.437 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.437 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.437 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.437 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.438 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.438 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.438 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.438 INFO:teuthology.orchestra.run.smithi016.stdout: "AUDIT_API_ENABLED": { 2024-09-12T02:17:55.438 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "AUDIT_API_ENABLED", 2024-09-12T02:17:55.438 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.438 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.438 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.438 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:55.438 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.438 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.438 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.438 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.439 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.439 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.439 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.439 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.439 INFO:teuthology.orchestra.run.smithi016.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-12T02:17:55.439 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-12T02:17:55.439 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.439 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.439 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.439 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T02:17:55.439 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.439 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.439 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.439 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.440 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.440 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.440 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.440 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.440 INFO:teuthology.orchestra.run.smithi016.stdout: "ENABLE_BROWSABLE_API": { 2024-09-12T02:17:55.440 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-12T02:17:55.440 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.440 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.440 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.440 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T02:17:55.440 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.440 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.440 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.441 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.441 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.441 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.441 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.441 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.441 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-12T02:17:55.441 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-12T02:17:55.441 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.441 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.441 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.441 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T02:17:55.441 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.442 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.442 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.442 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.442 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.442 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.442 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.442 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.442 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-12T02:17:55.442 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-12T02:17:55.442 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.442 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.442 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.442 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T02:17:55.443 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.443 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.443 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.443 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.443 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.443 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.443 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.443 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.443 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-12T02:17:55.443 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-12T02:17:55.443 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.443 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.443 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.444 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T02:17:55.444 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.444 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.444 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.444 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.444 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.444 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.444 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.444 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.444 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-12T02:17:55.444 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-12T02:17:55.444 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.445 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.445 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.445 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T02:17:55.445 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.445 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.445 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.445 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.445 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.445 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.445 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.445 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.445 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-12T02:17:55.445 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-12T02:17:55.446 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.446 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.446 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.446 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T02:17:55.446 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.446 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.446 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.446 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.446 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.446 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.446 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.446 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.446 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-12T02:17:55.447 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-12T02:17:55.447 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.447 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.447 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.447 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T02:17:55.447 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.447 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.447 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.447 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.447 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.447 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.447 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.447 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.448 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-12T02:17:55.448 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-12T02:17:55.448 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.448 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.448 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.448 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T02:17:55.448 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.448 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.448 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.448 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.448 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.448 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.448 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.449 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.449 INFO:teuthology.orchestra.run.smithi016.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-12T02:17:55.449 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-12T02:17:55.449 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.449 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.449 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.449 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:55.449 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.449 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.449 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.449 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.449 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.449 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.449 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.450 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.450 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_API_PASSWORD": { 2024-09-12T02:17:55.450 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-12T02:17:55.450 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.450 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.450 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.450 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "admin", 2024-09-12T02:17:55.450 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.450 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.450 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.450 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.450 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.451 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.451 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.451 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.451 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-12T02:17:55.451 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-12T02:17:55.451 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.451 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.451 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.451 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T02:17:55.451 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.451 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.451 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.451 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.452 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.452 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.452 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.452 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.452 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_API_URL": { 2024-09-12T02:17:55.452 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_API_URL", 2024-09-12T02:17:55.452 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.452 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.452 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.452 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:55.452 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.452 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.452 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.453 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.453 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.453 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.453 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.453 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.453 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_API_USERNAME": { 2024-09-12T02:17:55.453 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-12T02:17:55.453 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.453 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.453 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.453 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "admin", 2024-09-12T02:17:55.453 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.453 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.454 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.454 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.454 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.454 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.454 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.454 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.454 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-12T02:17:55.454 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-12T02:17:55.454 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.454 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.454 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.454 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:55.454 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.455 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.455 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.455 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.455 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.455 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.455 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.455 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.455 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-12T02:17:55.455 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-12T02:17:55.455 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.455 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.456 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.456 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:55.456 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.456 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.456 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.456 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.456 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.456 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.456 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.456 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.456 INFO:teuthology.orchestra.run.smithi016.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-12T02:17:55.456 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-12T02:17:55.456 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.457 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.457 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.457 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T02:17:55.457 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.457 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.457 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.457 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.457 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.457 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.457 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.457 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.457 INFO:teuthology.orchestra.run.smithi016.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-12T02:17:55.458 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-12T02:17:55.458 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.458 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.458 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.458 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:55.458 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.458 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.458 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.458 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.458 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.458 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.458 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.458 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.458 INFO:teuthology.orchestra.run.smithi016.stdout: "PROMETHEUS_API_HOST": { 2024-09-12T02:17:55.459 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-12T02:17:55.459 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.459 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.459 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.459 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:55.459 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.459 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.459 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.459 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.459 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.459 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.459 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.459 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.460 INFO:teuthology.orchestra.run.smithi016.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-12T02:17:55.460 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-12T02:17:55.460 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.460 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.460 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.460 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T02:17:55.460 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.460 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.460 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.460 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.460 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.460 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.460 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.461 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.461 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-12T02:17:55.461 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-12T02:17:55.461 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.461 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.461 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.461 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:55.461 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.461 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.461 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.461 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.461 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.461 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.461 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.462 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.462 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-12T02:17:55.462 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-12T02:17:55.462 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.462 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.462 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.462 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:55.462 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.462 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.462 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.462 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.462 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.462 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.463 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.463 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.463 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-12T02:17:55.463 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-12T02:17:55.463 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.463 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.463 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.463 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T02:17:55.463 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.463 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.463 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.463 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.463 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.464 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.464 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.464 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.464 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-12T02:17:55.464 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-12T02:17:55.464 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.464 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.464 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.464 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T02:17:55.464 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.464 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.464 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.464 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.464 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.465 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.465 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.465 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.465 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-12T02:17:55.465 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-12T02:17:55.465 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.465 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.465 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.465 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:55.465 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.465 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.465 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.465 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.466 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.466 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.466 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.466 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.466 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-12T02:17:55.466 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-12T02:17:55.466 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.466 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.466 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.466 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:55.466 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.466 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.466 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.466 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.467 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.467 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.467 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.467 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.467 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-12T02:17:55.467 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-12T02:17:55.467 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.467 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.467 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.467 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:55.467 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.467 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.467 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.467 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.468 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.468 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.468 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.468 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.468 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_ENABLED": { 2024-09-12T02:17:55.468 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-12T02:17:55.468 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.468 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.468 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.468 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T02:17:55.468 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.468 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.468 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.469 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.469 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.469 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.469 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.469 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.469 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-12T02:17:55.469 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-12T02:17:55.469 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.469 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.469 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.469 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-12T02:17:55.469 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.470 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.470 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.470 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.470 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.470 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.470 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.470 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.470 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-12T02:17:55.470 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-12T02:17:55.470 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T02:17:55.470 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.470 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.470 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-12T02:17:55.471 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.471 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.471 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.471 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.471 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.471 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.471 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.471 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.471 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-12T02:17:55.471 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-12T02:17:55.471 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T02:17:55.471 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.471 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.471 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8", 2024-09-12T02:17:55.472 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.472 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.472 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.472 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.472 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.472 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.472 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.472 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.472 INFO:teuthology.orchestra.run.smithi016.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-12T02:17:55.472 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-12T02:17:55.472 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T02:17:55.472 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.472 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.473 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "45", 2024-09-12T02:17:55.473 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.473 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.473 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.473 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.473 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.473 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.473 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.473 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.473 INFO:teuthology.orchestra.run.smithi016.stdout: "RGW_API_ACCESS_KEY": { 2024-09-12T02:17:55.473 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-12T02:17:55.473 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.473 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.473 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.474 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:55.474 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.474 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.474 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.474 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.474 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.474 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.474 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.474 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.474 INFO:teuthology.orchestra.run.smithi016.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-12T02:17:55.474 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-12T02:17:55.474 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.474 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.475 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.475 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "admin", 2024-09-12T02:17:55.475 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.475 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.475 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.475 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.475 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.475 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.475 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.475 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.475 INFO:teuthology.orchestra.run.smithi016.stdout: "RGW_API_SECRET_KEY": { 2024-09-12T02:17:55.475 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-12T02:17:55.475 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.475 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.476 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.476 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:55.476 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.476 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.476 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.476 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.476 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.476 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.476 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.476 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.476 INFO:teuthology.orchestra.run.smithi016.stdout: "RGW_API_SSL_VERIFY": { 2024-09-12T02:17:55.476 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-12T02:17:55.476 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.476 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.477 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.477 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T02:17:55.477 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.477 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.477 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.477 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.477 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.477 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.477 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.477 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.477 INFO:teuthology.orchestra.run.smithi016.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-12T02:17:55.477 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-12T02:17:55.477 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T02:17:55.478 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.478 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.478 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-12T02:17:55.478 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.478 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.478 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.478 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.478 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.478 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.478 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.478 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.478 INFO:teuthology.orchestra.run.smithi016.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-12T02:17:55.478 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-12T02:17:55.479 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T02:17:55.479 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.479 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.479 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-12T02:17:55.479 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.479 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.479 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.479 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.479 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.479 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.479 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.479 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.479 INFO:teuthology.orchestra.run.smithi016.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-12T02:17:55.479 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-12T02:17:55.480 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T02:17:55.480 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.480 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.480 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5", 2024-09-12T02:17:55.480 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.480 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.480 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.480 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.480 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.480 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.480 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.480 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.480 INFO:teuthology.orchestra.run.smithi016.stdout: "cross_origin_url": { 2024-09-12T02:17:55.481 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cross_origin_url", 2024-09-12T02:17:55.481 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.481 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.481 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.481 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:55.481 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.481 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.481 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.481 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.481 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.481 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.481 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.481 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.482 INFO:teuthology.orchestra.run.smithi016.stdout: "crt_file": { 2024-09-12T02:17:55.482 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crt_file", 2024-09-12T02:17:55.482 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.482 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.482 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.482 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:55.482 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.482 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.482 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.482 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.482 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.482 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.482 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.482 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.483 INFO:teuthology.orchestra.run.smithi016.stdout: "debug": { 2024-09-12T02:17:55.483 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "debug", 2024-09-12T02:17:55.483 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.483 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.483 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.483 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:55.483 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.483 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.483 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.483 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Enable/disable debug options", 2024-09-12T02:17:55.483 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.483 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.484 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.484 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.484 INFO:teuthology.orchestra.run.smithi016.stdout: "jwt_token_ttl": { 2024-09-12T02:17:55.484 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "jwt_token_ttl", 2024-09-12T02:17:55.484 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T02:17:55.484 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.484 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.484 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "28800", 2024-09-12T02:17:55.484 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.484 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.484 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.484 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.484 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.485 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.485 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.485 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.485 INFO:teuthology.orchestra.run.smithi016.stdout: "key_file": { 2024-09-12T02:17:55.485 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "key_file", 2024-09-12T02:17:55.485 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.485 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.485 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.485 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:55.485 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.485 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.485 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.485 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.486 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.486 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.486 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.486 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.486 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T02:17:55.486 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T02:17:55.486 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.486 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.486 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.486 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:55.486 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.486 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.486 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:55.487 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:55.487 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:55.487 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:55.487 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:55.487 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:55.487 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:55.487 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:55.487 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.487 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.487 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.487 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.487 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.487 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T02:17:55.488 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T02:17:55.488 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.488 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.488 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.488 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:55.488 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.488 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.488 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.488 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.488 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.488 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.488 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.488 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.488 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T02:17:55.489 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T02:17:55.489 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.489 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.489 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.489 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T02:17:55.489 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.489 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.489 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:55.489 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:55.489 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:55.489 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:55.489 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:55.489 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:55.490 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:55.490 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:55.490 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.490 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.490 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.490 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.490 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.490 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T02:17:55.490 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T02:17:55.490 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.490 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.490 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.490 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:55.490 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.491 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.491 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.491 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.491 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.491 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.491 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.491 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.491 INFO:teuthology.orchestra.run.smithi016.stdout: "motd": { 2024-09-12T02:17:55.491 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "motd", 2024-09-12T02:17:55.491 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.491 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.491 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.491 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:55.491 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.492 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.492 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.492 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "The message of the day", 2024-09-12T02:17:55.492 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.492 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.492 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.492 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.492 INFO:teuthology.orchestra.run.smithi016.stdout: "redirect_resolve_ip_addr": { 2024-09-12T02:17:55.492 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "redirect_resolve_ip_addr", 2024-09-12T02:17:55.492 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.492 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.492 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.492 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:55.492 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.493 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.493 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.493 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.493 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.493 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.493 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.493 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.493 INFO:teuthology.orchestra.run.smithi016.stdout: "server_addr": { 2024-09-12T02:17:55.493 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_addr", 2024-09-12T02:17:55.493 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.493 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.493 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.493 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "::", 2024-09-12T02:17:55.494 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.494 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.494 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.494 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.494 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.494 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.494 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.494 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.494 INFO:teuthology.orchestra.run.smithi016.stdout: "server_port": { 2024-09-12T02:17:55.494 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_port", 2024-09-12T02:17:55.494 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T02:17:55.494 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.494 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.494 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8080", 2024-09-12T02:17:55.495 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.495 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.495 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.495 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.495 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.495 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.495 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.495 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.495 INFO:teuthology.orchestra.run.smithi016.stdout: "ssl": { 2024-09-12T02:17:55.495 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ssl", 2024-09-12T02:17:55.495 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.495 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.495 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.495 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T02:17:55.496 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.496 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.496 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.496 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.496 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.496 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.496 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.496 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.496 INFO:teuthology.orchestra.run.smithi016.stdout: "ssl_server_port": { 2024-09-12T02:17:55.496 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ssl_server_port", 2024-09-12T02:17:55.496 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T02:17:55.496 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.497 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.497 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8443", 2024-09-12T02:17:55.497 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.497 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.497 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.497 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.497 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.497 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.497 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.497 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.497 INFO:teuthology.orchestra.run.smithi016.stdout: "standby_behaviour": { 2024-09-12T02:17:55.497 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "standby_behaviour", 2024-09-12T02:17:55.497 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.498 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.498 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.498 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "redirect", 2024-09-12T02:17:55.498 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.498 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.498 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:55.498 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:55.498 INFO:teuthology.orchestra.run.smithi016.stdout: "redirect" 2024-09-12T02:17:55.498 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:55.498 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.498 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.498 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.499 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.499 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.499 INFO:teuthology.orchestra.run.smithi016.stdout: "standby_error_status_code": { 2024-09-12T02:17:55.499 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "standby_error_status_code", 2024-09-12T02:17:55.499 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T02:17:55.499 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.499 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.499 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "500", 2024-09-12T02:17:55.499 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "400", 2024-09-12T02:17:55.499 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "599", 2024-09-12T02:17:55.499 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.499 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.499 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.500 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.500 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.500 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.500 INFO:teuthology.orchestra.run.smithi016.stdout: "url_prefix": { 2024-09-12T02:17:55.500 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "url_prefix", 2024-09-12T02:17:55.500 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.500 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.500 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.500 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:55.500 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.500 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.500 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.500 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.501 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.501 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.501 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.501 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:55.501 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:55.501 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.501 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T02:17:55.501 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "devicehealth", 2024-09-12T02:17:55.501 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T02:17:55.501 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T02:17:55.501 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T02:17:55.501 INFO:teuthology.orchestra.run.smithi016.stdout: "enable_monitoring": { 2024-09-12T02:17:55.501 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "enable_monitoring", 2024-09-12T02:17:55.502 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.502 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.502 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.502 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T02:17:55.502 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.502 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.502 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.502 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "monitor device health metrics", 2024-09-12T02:17:55.502 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.502 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.502 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.502 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.502 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T02:17:55.503 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T02:17:55.503 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.503 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.503 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.503 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:55.503 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.503 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.503 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:55.503 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:55.503 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:55.503 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:55.503 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:55.503 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:55.504 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:55.504 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:55.504 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.504 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.504 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.504 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.504 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.504 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T02:17:55.504 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T02:17:55.504 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.504 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.504 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.504 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:55.505 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.505 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.505 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.505 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.505 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.505 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.505 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.505 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.505 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T02:17:55.505 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T02:17:55.505 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.505 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.505 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.506 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T02:17:55.506 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.506 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.506 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:55.506 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:55.506 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:55.506 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:55.506 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:55.506 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:55.506 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:55.506 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:55.506 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.506 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.506 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.507 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.507 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.507 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T02:17:55.507 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T02:17:55.507 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.507 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.507 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.507 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:55.507 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.507 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.507 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.507 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.507 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.508 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.508 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.508 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.508 INFO:teuthology.orchestra.run.smithi016.stdout: "mark_out_threshold": { 2024-09-12T02:17:55.508 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mark_out_threshold", 2024-09-12T02:17:55.508 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-12T02:17:55.508 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.508 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.508 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "2419200", 2024-09-12T02:17:55.508 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.508 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.508 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.508 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-12T02:17:55.509 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.509 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.509 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.509 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.509 INFO:teuthology.orchestra.run.smithi016.stdout: "pool_name": { 2024-09-12T02:17:55.509 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "pool_name", 2024-09-12T02:17:55.509 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.509 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.509 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.509 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "device_health_metrics", 2024-09-12T02:17:55.509 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.509 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.509 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.509 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-12T02:17:55.510 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.510 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.510 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.510 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.510 INFO:teuthology.orchestra.run.smithi016.stdout: "retention_period": { 2024-09-12T02:17:55.510 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "retention_period", 2024-09-12T02:17:55.510 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-12T02:17:55.510 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.510 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.510 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "15552000", 2024-09-12T02:17:55.510 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.510 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.510 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.511 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how long to retain device health metrics", 2024-09-12T02:17:55.511 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.511 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.511 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.511 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.511 INFO:teuthology.orchestra.run.smithi016.stdout: "scrape_frequency": { 2024-09-12T02:17:55.511 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "scrape_frequency", 2024-09-12T02:17:55.511 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-12T02:17:55.511 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.511 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.511 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "86400", 2024-09-12T02:17:55.511 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.511 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.512 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.512 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-12T02:17:55.512 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.512 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.512 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.512 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.512 INFO:teuthology.orchestra.run.smithi016.stdout: "self_heal": { 2024-09-12T02:17:55.512 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "self_heal", 2024-09-12T02:17:55.512 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.512 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.513 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.513 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T02:17:55.513 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.513 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.513 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.513 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-12T02:17:55.513 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.513 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.513 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.513 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.513 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-09-12T02:17:55.513 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-09-12T02:17:55.513 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-12T02:17:55.513 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.514 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.514 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-09-12T02:17:55.514 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.514 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.514 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.514 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to wake up and check device health", 2024-09-12T02:17:55.514 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.514 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.514 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.514 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.514 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_threshold": { 2024-09-12T02:17:55.514 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_threshold", 2024-09-12T02:17:55.514 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-12T02:17:55.515 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.515 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.515 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "7257600", 2024-09-12T02:17:55.515 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.515 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.515 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.515 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-12T02:17:55.515 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.515 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.515 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.515 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:55.515 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:55.515 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.515 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T02:17:55.516 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "diskprediction_local", 2024-09-12T02:17:55.516 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T02:17:55.516 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T02:17:55.516 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T02:17:55.516 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T02:17:55.516 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T02:17:55.516 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.516 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.516 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.516 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:55.516 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.516 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.516 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:55.517 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:55.517 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:55.517 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:55.517 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:55.517 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:55.517 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:55.517 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:55.517 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.517 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.517 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.517 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.517 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.518 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T02:17:55.518 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T02:17:55.518 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.518 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.518 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.518 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:55.518 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.518 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.518 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.518 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.518 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.518 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.518 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.518 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.519 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T02:17:55.519 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T02:17:55.519 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.519 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.519 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.519 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T02:17:55.519 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.519 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.519 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:55.519 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:55.519 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:55.519 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:55.520 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:55.520 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:55.520 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:55.520 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:55.520 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.520 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.520 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.520 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.520 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.520 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T02:17:55.520 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T02:17:55.520 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.521 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.521 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.521 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:55.521 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.521 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.521 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.521 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.521 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.521 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.521 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.521 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.521 INFO:teuthology.orchestra.run.smithi016.stdout: "predict_interval": { 2024-09-12T02:17:55.521 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "predict_interval", 2024-09-12T02:17:55.522 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.522 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.522 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.522 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "86400", 2024-09-12T02:17:55.522 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.522 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.522 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.522 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.522 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.522 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.522 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.522 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.522 INFO:teuthology.orchestra.run.smithi016.stdout: "predictor_model": { 2024-09-12T02:17:55.523 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "predictor_model", 2024-09-12T02:17:55.523 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.523 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.523 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.523 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "prophetstor", 2024-09-12T02:17:55.523 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.523 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.523 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.523 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.523 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.523 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.523 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.523 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.523 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-09-12T02:17:55.524 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-09-12T02:17:55.524 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.524 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.524 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.524 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-09-12T02:17:55.524 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.524 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.524 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.524 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.524 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.524 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.524 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.525 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:55.525 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:55.525 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.525 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T02:17:55.525 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "influx", 2024-09-12T02:17:55.525 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": false, 2024-09-12T02:17:55.525 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "influxdb python module not found", 2024-09-12T02:17:55.525 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T02:17:55.525 INFO:teuthology.orchestra.run.smithi016.stdout: "batch_size": { 2024-09-12T02:17:55.525 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "batch_size", 2024-09-12T02:17:55.525 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T02:17:55.525 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.525 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.526 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5000", 2024-09-12T02:17:55.526 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.526 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.526 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.526 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-12T02:17:55.526 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.526 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.526 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.526 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.526 INFO:teuthology.orchestra.run.smithi016.stdout: "database": { 2024-09-12T02:17:55.526 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "database", 2024-09-12T02:17:55.526 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.527 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.527 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.527 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "ceph", 2024-09-12T02:17:55.527 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.527 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.527 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.527 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-12T02:17:55.527 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.527 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.527 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.527 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.527 INFO:teuthology.orchestra.run.smithi016.stdout: "hostname": { 2024-09-12T02:17:55.527 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "hostname", 2024-09-12T02:17:55.528 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.528 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.528 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.528 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:55.528 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.528 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.528 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.528 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "InfluxDB server hostname", 2024-09-12T02:17:55.528 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.528 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.528 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.528 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.529 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-09-12T02:17:55.529 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-09-12T02:17:55.529 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-12T02:17:55.529 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.529 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.529 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "30", 2024-09-12T02:17:55.529 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "5", 2024-09-12T02:17:55.529 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.529 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.529 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-12T02:17:55.529 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.529 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.529 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.530 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.530 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T02:17:55.530 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T02:17:55.530 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.530 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.530 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.530 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:55.530 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.530 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.530 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:55.530 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:55.530 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:55.531 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:55.531 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:55.531 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:55.531 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:55.531 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:55.531 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.531 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.531 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.531 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.531 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.531 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T02:17:55.531 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T02:17:55.531 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.532 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.532 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.532 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:55.532 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.532 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.532 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.532 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.532 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.532 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.532 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.532 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.533 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T02:17:55.533 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T02:17:55.533 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.533 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.533 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.533 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T02:17:55.533 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.533 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.533 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:55.533 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:55.533 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:55.533 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:55.533 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:55.534 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:55.534 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:55.534 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:55.534 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.534 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.534 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.534 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.534 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.534 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T02:17:55.534 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T02:17:55.534 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.534 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.534 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.534 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:55.535 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.535 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.535 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.535 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.535 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.535 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.535 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.535 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.535 INFO:teuthology.orchestra.run.smithi016.stdout: "password": { 2024-09-12T02:17:55.535 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "password", 2024-09-12T02:17:55.535 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.535 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.535 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.536 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:55.536 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.536 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.536 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.536 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "password of InfluxDB server user", 2024-09-12T02:17:55.536 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.536 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.536 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.536 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.537 INFO:teuthology.orchestra.run.smithi016.stdout: "port": { 2024-09-12T02:17:55.537 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "port", 2024-09-12T02:17:55.537 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T02:17:55.537 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.537 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.537 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8086", 2024-09-12T02:17:55.537 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.537 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.537 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.537 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "InfluxDB server port", 2024-09-12T02:17:55.537 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.537 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.538 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.538 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.538 INFO:teuthology.orchestra.run.smithi016.stdout: "ssl": { 2024-09-12T02:17:55.538 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ssl", 2024-09-12T02:17:55.538 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.538 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.538 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.538 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "false", 2024-09-12T02:17:55.538 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.538 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.538 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.538 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-12T02:17:55.538 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.539 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.539 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.539 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.539 INFO:teuthology.orchestra.run.smithi016.stdout: "threads": { 2024-09-12T02:17:55.539 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "threads", 2024-09-12T02:17:55.539 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T02:17:55.539 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.539 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.539 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5", 2024-09-12T02:17:55.539 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1", 2024-09-12T02:17:55.539 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "32", 2024-09-12T02:17:55.539 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.539 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-12T02:17:55.540 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.540 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.540 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.540 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.540 INFO:teuthology.orchestra.run.smithi016.stdout: "username": { 2024-09-12T02:17:55.540 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "username", 2024-09-12T02:17:55.540 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.540 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.540 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.540 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:55.540 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.540 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.541 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.541 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "username of InfluxDB server user", 2024-09-12T02:17:55.541 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.541 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.541 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.541 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.541 INFO:teuthology.orchestra.run.smithi016.stdout: "verify_ssl": { 2024-09-12T02:17:55.541 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "verify_ssl", 2024-09-12T02:17:55.541 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.541 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.541 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.541 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "true", 2024-09-12T02:17:55.541 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.542 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.542 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.542 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-12T02:17:55.542 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.542 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.542 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.542 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:55.542 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:55.542 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.542 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T02:17:55.542 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "insights", 2024-09-12T02:17:55.542 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T02:17:55.542 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T02:17:55.543 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T02:17:55.543 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T02:17:55.543 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T02:17:55.543 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.543 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.543 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.543 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:55.543 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.543 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.543 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:55.543 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:55.543 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:55.543 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:55.544 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:55.544 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:55.544 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:55.544 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:55.544 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.544 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.544 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.544 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.544 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.544 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T02:17:55.544 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T02:17:55.544 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.545 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.545 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.545 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:55.545 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.545 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.545 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.545 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.545 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.545 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.545 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.545 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.545 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T02:17:55.545 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T02:17:55.546 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.546 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.546 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.546 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T02:17:55.546 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.546 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.546 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:55.546 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:55.546 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:55.546 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:55.546 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:55.546 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:55.546 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:55.547 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:55.547 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.547 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.547 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.547 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.547 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.547 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T02:17:55.547 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T02:17:55.547 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.547 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.547 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.547 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:55.547 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.548 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.548 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.548 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.548 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.548 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.548 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.548 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:55.548 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:55.548 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.548 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T02:17:55.548 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "iostat", 2024-09-12T02:17:55.548 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T02:17:55.548 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T02:17:55.549 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T02:17:55.549 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T02:17:55.549 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T02:17:55.549 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.549 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.549 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.549 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:55.549 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.549 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.549 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:55.549 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:55.549 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:55.549 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:55.549 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:55.550 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:55.550 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:55.550 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:55.550 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.550 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.550 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.550 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.550 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.550 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T02:17:55.550 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T02:17:55.550 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.551 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.551 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.551 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:55.551 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.551 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.551 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.551 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.551 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.551 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.551 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.551 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.551 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T02:17:55.551 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T02:17:55.551 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.552 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.552 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.552 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T02:17:55.552 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.552 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.552 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:55.552 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:55.552 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:55.552 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:55.552 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:55.552 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:55.552 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:55.552 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:55.553 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.553 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.553 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.553 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.553 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.553 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T02:17:55.553 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T02:17:55.553 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.553 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.553 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.553 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:55.553 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.553 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.554 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.554 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.554 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.554 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.554 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.554 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:55.554 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:55.554 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.554 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T02:17:55.554 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "k8sevents", 2024-09-12T02:17:55.554 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T02:17:55.554 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T02:17:55.554 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T02:17:55.554 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_event_retention_days": { 2024-09-12T02:17:55.555 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ceph_event_retention_days", 2024-09-12T02:17:55.555 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T02:17:55.555 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.555 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.555 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "7", 2024-09-12T02:17:55.555 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.555 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.555 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.555 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-12T02:17:55.555 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.555 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.555 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.555 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.556 INFO:teuthology.orchestra.run.smithi016.stdout: "config_check_secs": { 2024-09-12T02:17:55.556 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "config_check_secs", 2024-09-12T02:17:55.556 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T02:17:55.556 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.556 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.556 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-12T02:17:55.556 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "10", 2024-09-12T02:17:55.556 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.556 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.556 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-12T02:17:55.556 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.557 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.557 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.557 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.557 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T02:17:55.557 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T02:17:55.557 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.557 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.557 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.557 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:55.557 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.557 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.557 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:55.558 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:55.558 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:55.558 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:55.558 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:55.558 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:55.558 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:55.558 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:55.558 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.558 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.558 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.558 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.558 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.558 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T02:17:55.559 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T02:17:55.559 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.559 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.559 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.559 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:55.559 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.559 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.559 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.559 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.559 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.559 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.559 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.559 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.559 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T02:17:55.560 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T02:17:55.560 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.560 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.560 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.560 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T02:17:55.560 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.560 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.560 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:55.560 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:55.560 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:55.560 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:55.561 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:55.561 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:55.561 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:55.561 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:55.561 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.561 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.561 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.561 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.561 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.561 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T02:17:55.561 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T02:17:55.561 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.561 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.562 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.562 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:55.562 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.562 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.562 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.562 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.562 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.562 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.562 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.562 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:55.562 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:55.562 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.562 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T02:17:55.563 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "localpool", 2024-09-12T02:17:55.563 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T02:17:55.563 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T02:17:55.563 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T02:17:55.563 INFO:teuthology.orchestra.run.smithi016.stdout: "failure_domain": { 2024-09-12T02:17:55.563 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "failure_domain", 2024-09-12T02:17:55.563 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.563 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.563 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.563 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "host", 2024-09-12T02:17:55.563 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.563 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.563 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.564 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "failure domain for any created local pool", 2024-09-12T02:17:55.564 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-12T02:17:55.564 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.564 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.564 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.564 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T02:17:55.564 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T02:17:55.564 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.564 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.564 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.564 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:55.564 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.564 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.565 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:55.565 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:55.565 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:55.565 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:55.565 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:55.565 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:55.565 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:55.565 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:55.565 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.565 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.565 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.565 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.566 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.566 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T02:17:55.566 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T02:17:55.566 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.566 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.566 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.566 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:55.566 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.566 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.566 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.566 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.566 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.566 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.567 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.567 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.567 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T02:17:55.567 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T02:17:55.567 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.567 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.567 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.567 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T02:17:55.567 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.567 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.567 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:55.567 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:55.568 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:55.568 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:55.568 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:55.568 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:55.568 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:55.568 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:55.568 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.568 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.568 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.568 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.568 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.568 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T02:17:55.569 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T02:17:55.569 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.569 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.569 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.569 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:55.569 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.569 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.569 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.569 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.569 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.569 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.573 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.573 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.573 INFO:teuthology.orchestra.run.smithi016.stdout: "min_size": { 2024-09-12T02:17:55.573 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "min_size", 2024-09-12T02:17:55.573 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T02:17:55.573 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.573 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.573 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:55.573 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.573 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.573 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.573 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "default min_size for any created local pool", 2024-09-12T02:17:55.574 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-12T02:17:55.574 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.574 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.574 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.574 INFO:teuthology.orchestra.run.smithi016.stdout: "num_rep": { 2024-09-12T02:17:55.574 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "num_rep", 2024-09-12T02:17:55.574 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T02:17:55.574 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.574 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.574 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "3", 2024-09-12T02:17:55.574 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.575 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.575 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.575 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "default replica count for any created local pool", 2024-09-12T02:17:55.575 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.575 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.575 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.575 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.575 INFO:teuthology.orchestra.run.smithi016.stdout: "pg_num": { 2024-09-12T02:17:55.575 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "pg_num", 2024-09-12T02:17:55.575 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T02:17:55.575 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.575 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.575 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "128", 2024-09-12T02:17:55.576 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.576 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.576 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.576 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "default pg_num for any created local pool", 2024-09-12T02:17:55.576 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.576 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.576 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.576 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.576 INFO:teuthology.orchestra.run.smithi016.stdout: "prefix": { 2024-09-12T02:17:55.576 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "prefix", 2024-09-12T02:17:55.576 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.576 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.577 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.577 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:55.577 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.577 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.577 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.577 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "name prefix for any created local pool", 2024-09-12T02:17:55.577 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.577 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.577 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.577 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.577 INFO:teuthology.orchestra.run.smithi016.stdout: "subtree": { 2024-09-12T02:17:55.577 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "subtree", 2024-09-12T02:17:55.578 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.578 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.578 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.578 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "rack", 2024-09-12T02:17:55.578 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.578 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.578 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.578 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-12T02:17:55.578 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-12T02:17:55.578 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.578 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.578 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:55.578 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:55.578 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.579 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T02:17:55.579 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mds_autoscaler", 2024-09-12T02:17:55.579 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T02:17:55.579 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T02:17:55.579 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T02:17:55.579 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T02:17:55.579 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T02:17:55.579 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.579 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.579 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.579 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:55.579 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.579 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.579 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:55.580 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:55.580 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:55.580 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:55.580 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:55.580 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:55.580 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:55.580 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:55.580 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.580 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.580 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.580 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.580 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.580 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T02:17:55.580 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T02:17:55.581 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.581 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.581 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.581 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:55.581 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.581 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.581 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.581 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.581 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.581 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.581 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.581 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.581 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T02:17:55.582 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T02:17:55.582 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.582 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.582 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.582 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T02:17:55.582 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.582 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.582 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:55.582 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:55.582 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:55.582 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:55.582 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:55.582 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:55.583 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:55.583 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:55.583 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.583 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.583 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.583 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.583 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.583 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T02:17:55.583 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T02:17:55.583 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.583 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.583 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.583 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:55.584 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.584 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.584 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.584 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.584 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.584 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.584 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.584 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:55.584 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:55.584 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.584 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T02:17:55.584 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mirroring", 2024-09-12T02:17:55.584 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T02:17:55.585 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T02:17:55.585 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T02:17:55.585 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T02:17:55.585 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T02:17:55.585 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.585 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.585 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.585 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:55.585 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.585 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.585 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:55.585 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:55.585 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:55.586 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:55.586 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:55.586 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:55.586 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:55.586 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:55.586 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.586 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.586 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.586 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.586 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.586 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T02:17:55.586 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T02:17:55.586 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.587 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.587 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.587 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:55.587 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.587 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.587 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.587 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.587 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.587 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.587 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.587 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.587 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T02:17:55.587 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T02:17:55.588 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.588 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.588 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.588 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T02:17:55.588 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.588 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.588 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:55.588 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:55.588 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:55.588 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:55.588 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:55.588 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:55.588 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:55.588 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:55.589 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.589 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.589 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.589 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.589 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.589 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T02:17:55.589 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T02:17:55.589 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.589 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.589 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.589 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:55.589 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.590 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.590 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.590 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.590 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.590 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.590 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.590 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:55.590 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:55.590 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.590 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T02:17:55.590 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "nfs", 2024-09-12T02:17:55.590 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T02:17:55.591 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T02:17:55.591 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T02:17:55.591 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T02:17:55.591 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T02:17:55.591 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.591 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.591 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.591 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:55.591 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.591 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.591 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:55.591 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:55.591 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:55.592 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:55.592 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:55.592 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:55.592 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:55.592 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:55.592 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.592 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.592 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.592 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.592 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.592 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T02:17:55.592 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T02:17:55.592 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.593 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.593 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.593 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:55.593 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.593 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.593 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.593 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.593 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.593 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.593 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.593 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.593 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T02:17:55.593 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T02:17:55.594 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.594 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.594 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.594 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T02:17:55.594 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.594 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.594 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:55.594 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:55.594 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:55.594 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:55.594 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:55.594 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:55.594 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:55.595 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:55.595 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.595 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.595 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.595 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.595 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.595 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T02:17:55.595 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T02:17:55.595 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.595 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.595 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.595 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:55.595 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.596 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.596 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.596 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.596 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.596 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.596 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.596 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:55.596 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:55.596 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.596 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T02:17:55.596 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "orchestrator", 2024-09-12T02:17:55.596 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T02:17:55.596 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T02:17:55.597 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T02:17:55.597 INFO:teuthology.orchestra.run.smithi016.stdout: "fail_fs": { 2024-09-12T02:17:55.597 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "fail_fs", 2024-09-12T02:17:55.597 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.597 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.597 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.597 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:55.597 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.597 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.597 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.597 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-12T02:17:55.597 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.597 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.598 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.598 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.598 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T02:17:55.598 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T02:17:55.598 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.598 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.598 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.598 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:55.598 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.598 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.598 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:55.598 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:55.598 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:55.599 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:55.599 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:55.599 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:55.599 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:55.599 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:55.599 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.599 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.599 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.599 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.599 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.599 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T02:17:55.599 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T02:17:55.599 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.600 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.600 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.600 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:55.600 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.600 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.600 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.600 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.600 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.600 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.600 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.600 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.600 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T02:17:55.600 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T02:17:55.601 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.601 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.601 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.601 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T02:17:55.601 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.601 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.601 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:55.601 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:55.601 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:55.601 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:55.601 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:55.601 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:55.601 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:55.601 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:55.602 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.602 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.602 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.602 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.602 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.602 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T02:17:55.602 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T02:17:55.602 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.602 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.602 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.602 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:55.602 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.602 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.603 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.603 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.603 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.603 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.603 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.603 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.603 INFO:teuthology.orchestra.run.smithi016.stdout: "orchestrator": { 2024-09-12T02:17:55.603 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "orchestrator", 2024-09-12T02:17:55.603 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.603 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.603 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.603 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:55.603 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.604 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.604 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:55.604 INFO:teuthology.orchestra.run.smithi016.stdout: "cephadm", 2024-09-12T02:17:55.604 INFO:teuthology.orchestra.run.smithi016.stdout: "rook", 2024-09-12T02:17:55.604 INFO:teuthology.orchestra.run.smithi016.stdout: "test_orchestrator" 2024-09-12T02:17:55.604 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:55.604 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Orchestrator backend", 2024-09-12T02:17:55.604 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.604 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.604 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.604 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:55.604 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:55.605 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.605 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T02:17:55.605 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "osd_perf_query", 2024-09-12T02:17:55.605 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T02:17:55.605 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T02:17:55.605 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T02:17:55.605 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T02:17:55.605 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T02:17:55.605 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.605 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.605 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.606 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:55.606 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.606 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.606 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:55.606 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:55.606 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:55.606 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:55.606 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:55.606 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:55.606 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:55.606 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:55.606 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.607 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.607 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.607 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.607 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.607 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T02:17:55.607 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T02:17:55.607 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.607 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.607 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.607 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:55.607 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.608 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.608 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.608 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.608 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.608 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.608 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.608 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.608 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T02:17:55.608 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T02:17:55.608 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.608 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.608 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.608 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T02:17:55.608 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.609 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.609 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:55.609 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:55.609 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:55.609 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:55.609 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:55.609 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:55.609 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:55.609 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:55.609 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.609 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.609 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.609 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.610 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.610 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T02:17:55.610 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T02:17:55.610 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.610 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.610 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.610 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:55.610 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.610 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.610 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.610 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.610 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.610 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.611 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.611 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:55.611 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:55.611 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.611 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T02:17:55.611 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "osd_support", 2024-09-12T02:17:55.611 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T02:17:55.611 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T02:17:55.611 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T02:17:55.611 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T02:17:55.611 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T02:17:55.611 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.611 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.612 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.612 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:55.612 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.612 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.612 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:55.612 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:55.612 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:55.612 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:55.612 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:55.612 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:55.612 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:55.612 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:55.612 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.612 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.613 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.613 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.613 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.613 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T02:17:55.613 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T02:17:55.613 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.613 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.613 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.613 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:55.613 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.613 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.613 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.613 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.613 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.614 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.614 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.614 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.614 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T02:17:55.614 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T02:17:55.614 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.614 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.614 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.614 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T02:17:55.614 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.614 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.614 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:55.614 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:55.614 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:55.615 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:55.615 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:55.615 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:55.615 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:55.615 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:55.615 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.615 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.615 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.615 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.615 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.615 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T02:17:55.615 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T02:17:55.615 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.616 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.616 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.616 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:55.616 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.616 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.616 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.616 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.616 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.616 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.616 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.616 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:55.616 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:55.616 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.617 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T02:17:55.617 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "pg_autoscaler", 2024-09-12T02:17:55.617 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T02:17:55.617 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T02:17:55.617 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T02:17:55.617 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T02:17:55.617 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T02:17:55.617 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.617 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.617 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.617 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:55.617 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.617 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.617 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:55.618 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:55.618 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:55.618 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:55.618 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:55.618 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:55.618 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:55.618 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:55.618 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.618 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.618 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.618 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.618 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.618 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T02:17:55.619 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T02:17:55.619 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.619 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.619 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.619 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:55.619 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.619 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.619 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.619 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.619 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.619 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.619 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.619 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.620 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T02:17:55.620 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T02:17:55.620 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.620 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.620 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.620 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T02:17:55.620 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.620 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.620 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:55.620 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:55.620 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:55.620 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:55.621 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:55.621 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:55.621 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:55.621 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:55.621 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.621 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.621 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.621 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.621 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.621 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T02:17:55.621 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T02:17:55.621 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.621 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.622 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.622 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:55.622 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.622 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.622 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.622 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.622 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.622 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.622 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.622 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.622 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-09-12T02:17:55.622 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-09-12T02:17:55.622 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-12T02:17:55.623 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.623 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.623 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-09-12T02:17:55.623 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.623 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.623 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.623 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.623 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.623 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.623 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.623 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.623 INFO:teuthology.orchestra.run.smithi016.stdout: "threshold": { 2024-09-12T02:17:55.623 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "threshold", 2024-09-12T02:17:55.623 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-09-12T02:17:55.624 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.624 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.624 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "3.0", 2024-09-12T02:17:55.624 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1.0", 2024-09-12T02:17:55.624 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.624 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.624 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "scaling threshold", 2024-09-12T02:17:55.624 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-12T02:17:55.624 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.624 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.624 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:55.624 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:55.624 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.625 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T02:17:55.625 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "progress", 2024-09-12T02:17:55.625 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T02:17:55.625 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T02:17:55.625 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T02:17:55.625 INFO:teuthology.orchestra.run.smithi016.stdout: "allow_pg_recovery_event": { 2024-09-12T02:17:55.625 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "allow_pg_recovery_event", 2024-09-12T02:17:55.625 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.625 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.625 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.625 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:55.625 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.625 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.626 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.626 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-12T02:17:55.626 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.626 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.626 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.626 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.626 INFO:teuthology.orchestra.run.smithi016.stdout: "enabled": { 2024-09-12T02:17:55.626 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "enabled", 2024-09-12T02:17:55.626 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.626 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.626 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.626 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T02:17:55.626 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.626 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.627 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.627 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.627 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.627 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.627 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.627 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.627 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T02:17:55.627 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T02:17:55.627 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.627 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.627 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.627 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:55.627 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.628 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.628 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:55.628 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:55.628 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:55.628 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:55.628 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:55.628 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:55.628 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:55.628 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:55.628 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.628 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.628 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.628 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.628 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.629 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T02:17:55.629 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T02:17:55.629 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.629 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.629 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.629 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:55.629 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.629 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.629 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.629 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.629 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.629 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.629 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.630 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.630 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T02:17:55.630 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T02:17:55.630 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.630 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.630 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.630 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T02:17:55.630 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.630 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.630 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:55.630 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:55.630 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:55.630 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:55.630 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:55.631 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:55.631 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:55.631 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:55.631 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.631 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.631 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.631 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.631 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.631 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T02:17:55.631 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T02:17:55.631 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.632 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.632 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.632 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:55.632 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.632 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.632 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.632 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.632 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.632 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.632 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.632 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.632 INFO:teuthology.orchestra.run.smithi016.stdout: "max_completed_events": { 2024-09-12T02:17:55.632 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_completed_events", 2024-09-12T02:17:55.633 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T02:17:55.633 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.633 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.633 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "50", 2024-09-12T02:17:55.633 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.633 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.633 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.633 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "number of past completed events to remember", 2024-09-12T02:17:55.633 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.633 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.633 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.633 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.633 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-09-12T02:17:55.633 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-09-12T02:17:55.634 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-12T02:17:55.634 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.634 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.634 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5", 2024-09-12T02:17:55.634 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.634 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.634 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.634 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how long the module is going to sleep", 2024-09-12T02:17:55.634 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.634 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.634 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.634 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:55.635 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:55.635 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.635 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T02:17:55.635 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "prometheus", 2024-09-12T02:17:55.635 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T02:17:55.635 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T02:17:55.635 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T02:17:55.635 INFO:teuthology.orchestra.run.smithi016.stdout: "cache": { 2024-09-12T02:17:55.635 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cache", 2024-09-12T02:17:55.635 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.635 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.635 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.635 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T02:17:55.636 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.636 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.636 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.636 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.636 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.636 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.636 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.636 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.636 INFO:teuthology.orchestra.run.smithi016.stdout: "exclude_perf_counters": { 2024-09-12T02:17:55.636 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "exclude_perf_counters", 2024-09-12T02:17:55.636 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.636 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.637 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.637 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T02:17:55.637 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.637 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.637 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.637 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-12T02:17:55.637 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-12T02:17:55.637 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.637 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.637 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.637 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T02:17:55.637 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T02:17:55.637 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.638 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.638 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.638 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:55.638 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.638 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.638 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:55.638 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:55.638 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:55.638 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:55.638 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:55.638 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:55.638 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:55.638 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:55.639 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.639 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.639 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.639 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.639 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.639 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T02:17:55.639 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T02:17:55.639 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.639 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.639 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.639 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:55.639 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.640 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.640 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.640 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.640 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.640 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.640 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.640 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.640 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T02:17:55.640 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T02:17:55.640 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.640 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.640 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.641 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T02:17:55.641 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.641 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.641 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:55.641 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:55.641 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:55.641 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:55.641 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:55.641 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:55.641 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:55.641 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:55.641 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.641 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.641 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.642 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.642 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.642 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T02:17:55.642 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T02:17:55.642 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.642 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.642 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.642 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:55.642 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.642 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.642 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.642 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.642 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.642 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.643 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.643 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.643 INFO:teuthology.orchestra.run.smithi016.stdout: "rbd_stats_pools": { 2024-09-12T02:17:55.643 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rbd_stats_pools", 2024-09-12T02:17:55.643 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.643 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.643 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.643 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:55.643 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.643 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.643 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.643 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.643 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.644 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.644 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.644 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.644 INFO:teuthology.orchestra.run.smithi016.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-12T02:17:55.644 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-12T02:17:55.644 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T02:17:55.644 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.644 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.644 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "300", 2024-09-12T02:17:55.644 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.644 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.644 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.645 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.645 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.645 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.645 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.645 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.645 INFO:teuthology.orchestra.run.smithi016.stdout: "scrape_interval": { 2024-09-12T02:17:55.645 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "scrape_interval", 2024-09-12T02:17:55.645 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-09-12T02:17:55.645 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.645 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.645 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "15.0", 2024-09-12T02:17:55.645 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.645 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.646 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.646 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.646 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.646 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.646 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.646 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.646 INFO:teuthology.orchestra.run.smithi016.stdout: "server_addr": { 2024-09-12T02:17:55.646 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_addr", 2024-09-12T02:17:55.646 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.646 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.646 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.646 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "::", 2024-09-12T02:17:55.646 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.647 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.647 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.647 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-12T02:17:55.647 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.647 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.647 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.647 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.647 INFO:teuthology.orchestra.run.smithi016.stdout: "server_port": { 2024-09-12T02:17:55.647 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_port", 2024-09-12T02:17:55.647 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T02:17:55.647 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.647 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.647 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "9283", 2024-09-12T02:17:55.648 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.648 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.648 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.648 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-12T02:17:55.648 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.648 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.648 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.648 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.648 INFO:teuthology.orchestra.run.smithi016.stdout: "stale_cache_strategy": { 2024-09-12T02:17:55.648 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "stale_cache_strategy", 2024-09-12T02:17:55.648 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.649 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.649 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.649 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "log", 2024-09-12T02:17:55.649 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.649 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.649 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.649 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.649 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.649 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.649 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.649 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.649 INFO:teuthology.orchestra.run.smithi016.stdout: "standby_behaviour": { 2024-09-12T02:17:55.650 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "standby_behaviour", 2024-09-12T02:17:55.650 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.650 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.650 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.650 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "default", 2024-09-12T02:17:55.650 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.650 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.650 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:55.650 INFO:teuthology.orchestra.run.smithi016.stdout: "default", 2024-09-12T02:17:55.650 INFO:teuthology.orchestra.run.smithi016.stdout: "error" 2024-09-12T02:17:55.650 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:55.650 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.650 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.651 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.651 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.651 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.651 INFO:teuthology.orchestra.run.smithi016.stdout: "standby_error_status_code": { 2024-09-12T02:17:55.651 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "standby_error_status_code", 2024-09-12T02:17:55.651 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T02:17:55.651 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.651 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.651 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "500", 2024-09-12T02:17:55.651 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "400", 2024-09-12T02:17:55.651 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "599", 2024-09-12T02:17:55.651 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.651 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.652 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.652 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.652 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.652 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:55.652 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:55.652 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.652 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T02:17:55.652 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rbd_support", 2024-09-12T02:17:55.652 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T02:17:55.652 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T02:17:55.652 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T02:17:55.652 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T02:17:55.652 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T02:17:55.652 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.652 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.653 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.653 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:55.653 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.653 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.653 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:55.653 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:55.653 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:55.653 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:55.653 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:55.653 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:55.653 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:55.653 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:55.653 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.654 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.654 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.654 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.654 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.654 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T02:17:55.654 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T02:17:55.654 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.654 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.654 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.654 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:55.654 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.655 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.655 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.655 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.655 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.655 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.655 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.655 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.655 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T02:17:55.655 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T02:17:55.655 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.655 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.655 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.655 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T02:17:55.656 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.656 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.656 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:55.656 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:55.656 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:55.656 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:55.656 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:55.656 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:55.656 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:55.656 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:55.656 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.656 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.657 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.657 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.657 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.657 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T02:17:55.657 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T02:17:55.657 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.657 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.657 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.657 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:55.657 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.657 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.657 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.657 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.658 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.658 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.658 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.658 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.658 INFO:teuthology.orchestra.run.smithi016.stdout: "max_concurrent_snap_create": { 2024-09-12T02:17:55.658 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_concurrent_snap_create", 2024-09-12T02:17:55.658 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T02:17:55.658 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.658 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.658 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-12T02:17:55.658 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.658 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.658 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.659 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.659 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.659 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.659 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.659 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.659 INFO:teuthology.orchestra.run.smithi016.stdout: "mirror_snapshot_schedule": { 2024-09-12T02:17:55.659 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mirror_snapshot_schedule", 2024-09-12T02:17:55.659 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.659 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.659 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.659 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:55.659 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.659 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.660 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.660 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.660 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.660 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.660 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.660 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.660 INFO:teuthology.orchestra.run.smithi016.stdout: "trash_purge_schedule": { 2024-09-12T02:17:55.660 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "trash_purge_schedule", 2024-09-12T02:17:55.660 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.660 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.660 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.660 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:55.660 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.661 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.661 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.661 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.661 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.661 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.661 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.661 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:55.661 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:55.661 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.661 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T02:17:55.661 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "restful", 2024-09-12T02:17:55.661 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T02:17:55.661 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T02:17:55.661 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T02:17:55.662 INFO:teuthology.orchestra.run.smithi016.stdout: "enable_auth": { 2024-09-12T02:17:55.662 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "enable_auth", 2024-09-12T02:17:55.662 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.662 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.662 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.662 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T02:17:55.662 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.662 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.662 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.662 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.662 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.662 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.663 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.663 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.663 INFO:teuthology.orchestra.run.smithi016.stdout: "key_file": { 2024-09-12T02:17:55.663 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "key_file", 2024-09-12T02:17:55.663 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.663 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.663 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.663 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:55.663 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.663 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.663 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.663 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.663 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.664 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.664 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.664 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.664 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T02:17:55.664 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T02:17:55.664 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.664 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.664 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.664 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:55.665 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.665 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.665 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:55.665 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:55.665 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:55.665 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:55.665 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:55.665 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:55.665 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:55.665 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:55.665 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.665 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.666 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.666 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.666 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.666 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T02:17:55.666 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T02:17:55.666 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.666 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.666 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.666 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:55.666 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.666 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.666 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.666 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.666 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.667 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.667 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.667 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.667 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T02:17:55.667 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T02:17:55.667 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.667 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.667 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.667 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T02:17:55.667 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.667 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.667 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:55.668 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:55.668 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:55.668 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:55.668 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:55.668 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:55.668 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:55.668 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:55.668 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.668 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.668 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.668 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.668 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.669 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T02:17:55.669 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T02:17:55.669 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.669 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.669 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.669 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:55.669 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.669 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.669 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.669 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.669 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.669 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.669 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.670 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.670 INFO:teuthology.orchestra.run.smithi016.stdout: "server_addr": { 2024-09-12T02:17:55.670 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_addr", 2024-09-12T02:17:55.670 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.670 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.670 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.670 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:55.670 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.670 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.670 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.670 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.670 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.670 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.670 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.671 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.671 INFO:teuthology.orchestra.run.smithi016.stdout: "server_port": { 2024-09-12T02:17:55.671 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_port", 2024-09-12T02:17:55.671 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.671 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.671 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.671 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:55.671 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.671 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.671 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.671 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.671 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.671 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.672 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.672 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:55.672 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:55.672 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.672 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T02:17:55.672 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rgw", 2024-09-12T02:17:55.672 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T02:17:55.672 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T02:17:55.672 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T02:17:55.672 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T02:17:55.672 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T02:17:55.672 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.673 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.673 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.673 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:55.673 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.673 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.673 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:55.673 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:55.673 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:55.673 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:55.673 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:55.673 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:55.673 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:55.673 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:55.673 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.674 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.674 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.674 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.674 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.674 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T02:17:55.674 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T02:17:55.674 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.674 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.674 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.674 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:55.674 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.674 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.674 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.674 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.675 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.675 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.675 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.675 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.675 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T02:17:55.675 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T02:17:55.675 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.675 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.675 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.675 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T02:17:55.675 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.675 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.675 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:55.676 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:55.676 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:55.676 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:55.676 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:55.676 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:55.676 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:55.676 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:55.676 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.676 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.676 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.676 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.676 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.676 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T02:17:55.677 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T02:17:55.677 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.677 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.677 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.677 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:55.677 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.677 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.677 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.677 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.677 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.677 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.677 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.677 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:55.678 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:55.678 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.678 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T02:17:55.678 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rook", 2024-09-12T02:17:55.678 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T02:17:55.678 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T02:17:55.678 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T02:17:55.678 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T02:17:55.678 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T02:17:55.678 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.678 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.678 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.678 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:55.678 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.679 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.679 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:55.679 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:55.679 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:55.679 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:55.679 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:55.679 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:55.679 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:55.679 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:55.679 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.679 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.679 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.680 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.680 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.680 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T02:17:55.680 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T02:17:55.680 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.680 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.680 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.680 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:55.680 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.681 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.681 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.681 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.681 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.681 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.681 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.681 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.681 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T02:17:55.681 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T02:17:55.681 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.681 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.682 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.682 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T02:17:55.682 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.682 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.682 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:55.682 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:55.682 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:55.682 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:55.682 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:55.682 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:55.682 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:55.682 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:55.682 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.683 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.683 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.683 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.683 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.683 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T02:17:55.683 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T02:17:55.683 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.683 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.683 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.683 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:55.683 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.683 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.684 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.684 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.684 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.684 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.684 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.684 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.684 INFO:teuthology.orchestra.run.smithi016.stdout: "storage_class": { 2024-09-12T02:17:55.684 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "storage_class", 2024-09-12T02:17:55.684 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.684 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.684 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.685 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "local", 2024-09-12T02:17:55.685 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.685 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.685 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.685 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-12T02:17:55.685 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.685 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.685 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.685 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:55.685 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:55.685 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.685 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T02:17:55.685 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "selftest", 2024-09-12T02:17:55.686 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T02:17:55.686 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T02:17:55.686 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T02:17:55.686 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T02:17:55.686 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T02:17:55.686 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.686 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.686 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.686 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:55.686 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.686 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.686 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:55.687 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:55.687 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:55.687 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:55.687 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:55.687 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:55.687 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:55.687 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:55.687 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.687 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.687 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.687 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.687 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.688 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T02:17:55.688 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T02:17:55.688 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.688 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.688 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.688 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:55.688 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.688 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.688 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.688 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.688 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.688 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.688 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.689 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.689 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T02:17:55.689 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T02:17:55.689 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.689 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.689 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.689 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T02:17:55.689 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.689 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.689 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:55.689 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:55.689 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:55.689 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:55.690 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:55.690 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:55.690 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:55.690 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:55.690 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.690 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.690 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.690 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.690 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.690 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T02:17:55.690 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T02:17:55.690 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.690 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.690 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.691 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:55.691 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.691 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.691 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.691 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.691 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.691 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.691 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.691 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.691 INFO:teuthology.orchestra.run.smithi016.stdout: "roption1": { 2024-09-12T02:17:55.691 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "roption1", 2024-09-12T02:17:55.691 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.691 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.692 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.692 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:55.692 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.692 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.692 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.692 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.692 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.692 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.692 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.692 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.692 INFO:teuthology.orchestra.run.smithi016.stdout: "roption2": { 2024-09-12T02:17:55.692 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "roption2", 2024-09-12T02:17:55.692 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.692 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.693 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.693 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "xyz", 2024-09-12T02:17:55.693 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.693 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.693 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.693 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.693 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.693 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.693 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.693 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.693 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption1": { 2024-09-12T02:17:55.693 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption1", 2024-09-12T02:17:55.693 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.693 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.694 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.694 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:55.694 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.694 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.694 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.694 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.694 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.694 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.694 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.694 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.694 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption2": { 2024-09-12T02:17:55.694 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption2", 2024-09-12T02:17:55.694 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T02:17:55.695 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.695 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.695 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:55.695 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.695 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.695 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.695 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.695 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.695 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.695 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.695 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.695 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption3": { 2024-09-12T02:17:55.696 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption3", 2024-09-12T02:17:55.696 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-09-12T02:17:55.696 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.696 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.696 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:55.696 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.696 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.696 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.696 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.696 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.696 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.696 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.696 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.696 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption4": { 2024-09-12T02:17:55.697 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption4", 2024-09-12T02:17:55.697 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.697 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.697 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.697 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:55.697 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.697 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.697 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.697 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.697 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.697 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.697 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.697 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.698 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption5": { 2024-09-12T02:17:55.698 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption5", 2024-09-12T02:17:55.698 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.698 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.698 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.698 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:55.698 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.698 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.698 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.698 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.698 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.698 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.698 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.698 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.699 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption6": { 2024-09-12T02:17:55.699 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption6", 2024-09-12T02:17:55.699 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.699 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.699 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.699 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T02:17:55.699 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.699 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.699 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.699 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.699 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.699 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.699 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.700 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.700 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption7": { 2024-09-12T02:17:55.700 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption7", 2024-09-12T02:17:55.700 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T02:17:55.700 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.700 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.700 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:55.700 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1", 2024-09-12T02:17:55.700 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "42", 2024-09-12T02:17:55.700 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.700 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.700 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.701 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.701 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.701 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.701 INFO:teuthology.orchestra.run.smithi016.stdout: "testkey": { 2024-09-12T02:17:55.701 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "testkey", 2024-09-12T02:17:55.701 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.701 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.701 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.701 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:55.701 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.701 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.701 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.701 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.702 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.702 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.702 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.702 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.702 INFO:teuthology.orchestra.run.smithi016.stdout: "testlkey": { 2024-09-12T02:17:55.702 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "testlkey", 2024-09-12T02:17:55.702 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.702 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.702 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.702 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:55.702 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.702 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.702 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.702 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.703 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.703 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.703 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.703 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.703 INFO:teuthology.orchestra.run.smithi016.stdout: "testnewline": { 2024-09-12T02:17:55.703 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "testnewline", 2024-09-12T02:17:55.703 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.703 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.703 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.703 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:55.703 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.703 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.703 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.703 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.704 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.704 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.704 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.704 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:55.704 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:55.704 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.704 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T02:17:55.704 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "snap_schedule", 2024-09-12T02:17:55.704 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T02:17:55.704 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T02:17:55.704 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T02:17:55.704 INFO:teuthology.orchestra.run.smithi016.stdout: "allow_m_granularity": { 2024-09-12T02:17:55.705 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "allow_m_granularity", 2024-09-12T02:17:55.705 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.705 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.705 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.705 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:55.705 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.705 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.705 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.705 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "allow minute scheduled snapshots", 2024-09-12T02:17:55.705 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.705 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.705 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.705 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.706 INFO:teuthology.orchestra.run.smithi016.stdout: "dump_on_update": { 2024-09-12T02:17:55.706 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "dump_on_update", 2024-09-12T02:17:55.706 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.706 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.706 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.706 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:55.706 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.706 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.706 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.706 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "dump database to debug log on update", 2024-09-12T02:17:55.706 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.706 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.706 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.707 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.707 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T02:17:55.707 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T02:17:55.707 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.707 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.707 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.707 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:55.707 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.707 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.707 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:55.707 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:55.707 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:55.707 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:55.708 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:55.708 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:55.708 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:55.708 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:55.708 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.708 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.708 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.708 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.708 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.708 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T02:17:55.708 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T02:17:55.708 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.708 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.709 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.709 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:55.709 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.709 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.709 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.709 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.709 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.709 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.709 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.709 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.709 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T02:17:55.709 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T02:17:55.709 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.710 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.710 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.710 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T02:17:55.710 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.710 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.710 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:55.710 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:55.710 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:55.710 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:55.710 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:55.710 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:55.710 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:55.710 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:55.711 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.711 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.711 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.711 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.711 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.711 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T02:17:55.711 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T02:17:55.711 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.711 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.711 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.711 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:55.711 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.711 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.711 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.712 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.712 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.712 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.712 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.712 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:55.712 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:55.712 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.712 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T02:17:55.712 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "stats", 2024-09-12T02:17:55.712 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T02:17:55.712 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T02:17:55.712 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T02:17:55.712 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T02:17:55.713 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T02:17:55.713 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.713 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.713 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.713 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:55.713 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.713 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.713 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:55.713 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:55.713 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:55.713 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:55.713 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:55.713 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:55.714 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:55.714 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:55.714 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.714 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.714 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.714 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.714 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.714 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T02:17:55.714 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T02:17:55.714 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.714 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.714 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.714 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:55.714 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.715 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.715 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.715 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.715 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.715 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.715 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.715 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.715 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T02:17:55.715 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T02:17:55.715 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.715 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.715 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.715 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T02:17:55.716 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.716 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.716 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:55.716 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:55.716 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:55.716 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:55.716 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:55.716 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:55.716 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:55.716 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:55.716 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.716 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.717 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.717 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.717 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.717 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T02:17:55.717 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T02:17:55.717 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.717 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.717 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.717 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:55.717 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.717 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.717 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.717 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.717 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.718 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.718 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.718 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:55.718 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:55.718 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.718 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T02:17:55.718 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "status", 2024-09-12T02:17:55.718 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T02:17:55.718 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T02:17:55.718 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T02:17:55.718 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T02:17:55.718 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T02:17:55.718 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.718 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.719 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.719 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:55.719 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.719 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.719 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:55.719 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:55.719 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:55.719 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:55.719 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:55.719 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:55.719 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:55.719 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:55.720 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.720 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.720 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.720 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.720 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.720 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T02:17:55.720 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T02:17:55.720 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.720 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.720 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.720 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:55.720 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.720 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.721 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.721 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.721 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.721 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.721 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.721 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.721 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T02:17:55.721 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T02:17:55.721 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.721 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.721 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.721 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T02:17:55.721 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.722 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.722 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:55.722 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:55.722 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:55.722 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:55.722 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:55.722 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:55.722 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:55.722 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:55.722 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.722 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.722 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.722 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.722 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.723 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T02:17:55.723 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T02:17:55.723 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.723 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.723 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.723 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:55.723 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.723 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.723 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.723 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.723 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.723 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.723 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.723 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:55.724 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:55.724 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.724 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T02:17:55.724 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "telegraf", 2024-09-12T02:17:55.724 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T02:17:55.724 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T02:17:55.724 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T02:17:55.724 INFO:teuthology.orchestra.run.smithi016.stdout: "address": { 2024-09-12T02:17:55.724 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "address", 2024-09-12T02:17:55.724 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.724 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.724 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.725 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-12T02:17:55.725 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.725 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.725 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.725 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.725 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.725 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.725 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.725 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.725 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-09-12T02:17:55.725 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-09-12T02:17:55.725 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-12T02:17:55.726 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.726 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.726 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "15", 2024-09-12T02:17:55.726 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.726 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.726 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.726 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.726 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.726 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.726 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.726 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.726 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T02:17:55.726 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T02:17:55.726 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.727 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.727 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.727 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:55.727 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.727 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.727 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:55.727 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:55.727 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:55.727 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:55.727 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:55.727 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:55.727 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:55.727 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:55.728 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.728 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.728 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.728 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.728 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.728 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T02:17:55.728 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T02:17:55.728 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.728 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.728 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.728 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:55.728 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.729 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.729 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.729 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.729 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.729 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.729 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.729 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.729 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T02:17:55.729 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T02:17:55.729 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.729 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.729 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.729 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T02:17:55.730 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.730 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.730 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:55.730 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:55.730 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:55.730 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:55.730 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:55.730 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:55.730 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:55.730 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:55.730 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.730 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.730 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.731 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.731 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.731 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T02:17:55.731 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T02:17:55.731 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.731 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.731 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.731 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:55.731 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.731 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.731 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.731 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.731 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.731 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.732 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.732 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:55.732 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:55.732 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.732 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T02:17:55.732 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "telemetry", 2024-09-12T02:17:55.732 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T02:17:55.732 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T02:17:55.732 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T02:17:55.732 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_basic": { 2024-09-12T02:17:55.732 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_basic", 2024-09-12T02:17:55.732 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.732 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.733 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.733 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T02:17:55.733 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.733 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.733 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.733 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-12T02:17:55.733 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.733 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.733 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.733 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.733 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_crash": { 2024-09-12T02:17:55.733 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_crash", 2024-09-12T02:17:55.733 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.734 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.734 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.734 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T02:17:55.734 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.734 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.734 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.734 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-12T02:17:55.734 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.734 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.734 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.734 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.734 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_device": { 2024-09-12T02:17:55.734 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_device", 2024-09-12T02:17:55.734 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.735 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.735 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.735 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T02:17:55.735 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.735 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.735 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.735 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-12T02:17:55.735 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.735 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.735 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.735 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.735 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_ident": { 2024-09-12T02:17:55.735 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_ident", 2024-09-12T02:17:55.736 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.736 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.736 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.736 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:55.736 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.736 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.736 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.736 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-12T02:17:55.736 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.736 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.736 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.736 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.737 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_perf": { 2024-09-12T02:17:55.737 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_perf", 2024-09-12T02:17:55.737 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.737 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.737 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.737 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:55.737 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.737 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.737 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.737 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-12T02:17:55.737 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.737 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.737 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.738 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.738 INFO:teuthology.orchestra.run.smithi016.stdout: "contact": { 2024-09-12T02:17:55.738 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "contact", 2024-09-12T02:17:55.738 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.738 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.738 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.738 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:55.738 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.738 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.738 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.738 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.738 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.738 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.739 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.739 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.739 INFO:teuthology.orchestra.run.smithi016.stdout: "description": { 2024-09-12T02:17:55.739 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "description", 2024-09-12T02:17:55.739 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.739 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.739 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.739 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:55.739 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.739 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.739 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.740 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.740 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.740 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.740 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.740 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.740 INFO:teuthology.orchestra.run.smithi016.stdout: "device_url": { 2024-09-12T02:17:55.740 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "device_url", 2024-09-12T02:17:55.740 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.740 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.740 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.740 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-12T02:17:55.740 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.740 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.741 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.741 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.741 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.741 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.741 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.741 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.741 INFO:teuthology.orchestra.run.smithi016.stdout: "enabled": { 2024-09-12T02:17:55.741 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "enabled", 2024-09-12T02:17:55.741 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.741 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.741 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.741 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:55.742 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.742 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.742 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.742 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.742 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.742 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.742 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.742 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.742 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-09-12T02:17:55.742 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-09-12T02:17:55.742 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T02:17:55.742 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.742 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.743 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "24", 2024-09-12T02:17:55.743 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "8", 2024-09-12T02:17:55.743 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.743 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.743 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.743 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.743 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.743 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.743 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.743 INFO:teuthology.orchestra.run.smithi016.stdout: "last_opt_revision": { 2024-09-12T02:17:55.743 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "last_opt_revision", 2024-09-12T02:17:55.743 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T02:17:55.743 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.743 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.744 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "1", 2024-09-12T02:17:55.744 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.744 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.744 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.744 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.744 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.744 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.744 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.744 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.744 INFO:teuthology.orchestra.run.smithi016.stdout: "leaderboard": { 2024-09-12T02:17:55.744 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "leaderboard", 2024-09-12T02:17:55.744 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.745 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.745 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.745 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:55.745 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.745 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.745 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.745 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.745 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.745 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.745 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.745 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.745 INFO:teuthology.orchestra.run.smithi016.stdout: "leaderboard_description": { 2024-09-12T02:17:55.745 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "leaderboard_description", 2024-09-12T02:17:55.745 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.746 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.746 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.746 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:55.746 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.746 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.746 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.746 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.746 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.746 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.746 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.746 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.746 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T02:17:55.746 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T02:17:55.747 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.747 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.747 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.747 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:55.747 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.747 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.747 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:55.747 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:55.747 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:55.747 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:55.747 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:55.747 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:55.747 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:55.747 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:55.748 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.748 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.748 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.748 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.748 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.748 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T02:17:55.748 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T02:17:55.748 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.748 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.748 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.748 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:55.748 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.748 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.749 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.749 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.749 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.749 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.749 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.749 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.749 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T02:17:55.749 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T02:17:55.749 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.749 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.749 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.749 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T02:17:55.749 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.749 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.750 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:55.750 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:55.750 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:55.750 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:55.750 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:55.750 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:55.750 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:55.750 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:55.750 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.750 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.750 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.750 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.750 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.751 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T02:17:55.751 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T02:17:55.751 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.751 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.751 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.751 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:55.751 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.751 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.751 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.751 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.751 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.751 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.751 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.751 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.752 INFO:teuthology.orchestra.run.smithi016.stdout: "organization": { 2024-09-12T02:17:55.752 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "organization", 2024-09-12T02:17:55.752 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.752 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.752 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.752 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:55.752 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.752 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.752 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.752 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.752 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.752 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.752 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.753 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.753 INFO:teuthology.orchestra.run.smithi016.stdout: "proxy": { 2024-09-12T02:17:55.753 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "proxy", 2024-09-12T02:17:55.753 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.753 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.753 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.753 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:55.753 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.753 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.753 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.753 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.753 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.753 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.754 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.754 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.754 INFO:teuthology.orchestra.run.smithi016.stdout: "url": { 2024-09-12T02:17:55.754 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "url", 2024-09-12T02:17:55.754 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.754 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.754 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.754 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-12T02:17:55.754 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.754 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.754 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.754 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.755 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.755 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.755 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.755 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:55.755 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:55.755 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.755 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T02:17:55.755 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "test_orchestrator", 2024-09-12T02:17:55.755 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T02:17:55.755 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T02:17:55.755 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T02:17:55.755 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T02:17:55.755 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T02:17:55.756 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.756 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.756 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.756 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:55.756 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.756 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.756 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:55.756 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:55.756 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:55.756 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:55.756 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:55.756 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:55.756 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:55.757 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:55.757 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.757 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.757 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.757 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.757 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.757 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T02:17:55.757 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T02:17:55.757 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.757 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.757 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.757 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:55.757 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.757 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.758 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.758 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.758 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.758 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.758 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.758 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.758 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T02:17:55.758 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T02:17:55.758 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.758 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.758 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.758 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T02:17:55.759 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.759 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.759 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:55.759 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:55.759 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:55.759 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:55.759 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:55.759 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:55.759 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:55.759 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:55.759 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.759 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.759 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.759 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.760 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.760 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T02:17:55.760 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T02:17:55.760 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.760 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.760 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.760 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:55.760 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.760 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.760 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.760 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.760 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.760 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.761 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.761 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:55.761 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:55.761 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.761 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T02:17:55.761 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "volumes", 2024-09-12T02:17:55.761 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T02:17:55.761 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T02:17:55.761 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T02:17:55.761 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T02:17:55.761 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T02:17:55.761 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.761 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.762 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.762 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:55.762 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.762 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.762 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:55.762 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:55.762 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:55.762 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:55.762 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:55.762 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:55.762 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:55.762 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:55.762 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.763 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.763 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.763 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.763 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.763 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T02:17:55.763 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T02:17:55.763 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.763 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.763 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.763 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:55.763 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.763 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.763 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.763 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.764 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.764 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.764 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.764 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.764 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T02:17:55.764 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T02:17:55.764 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.764 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.764 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.764 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T02:17:55.764 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.764 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.765 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:55.765 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:55.765 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:55.765 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:55.765 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:55.765 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:55.765 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:55.765 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:55.765 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.765 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.765 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.765 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.765 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.765 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T02:17:55.766 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T02:17:55.766 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.766 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.766 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.766 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:55.766 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.766 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.766 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.766 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.766 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.766 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.766 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.766 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.767 INFO:teuthology.orchestra.run.smithi016.stdout: "max_concurrent_clones": { 2024-09-12T02:17:55.767 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_concurrent_clones", 2024-09-12T02:17:55.767 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T02:17:55.767 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.767 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.767 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "4", 2024-09-12T02:17:55.767 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.767 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.767 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.767 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-12T02:17:55.767 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.767 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.767 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.768 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.768 INFO:teuthology.orchestra.run.smithi016.stdout: "periodic_async_work": { 2024-09-12T02:17:55.768 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "periodic_async_work", 2024-09-12T02:17:55.768 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.768 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.768 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.768 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:55.768 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.768 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.768 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.768 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Periodically check for async work", 2024-09-12T02:17:55.768 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.769 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.769 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.769 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.769 INFO:teuthology.orchestra.run.smithi016.stdout: "snapshot_clone_delay": { 2024-09-12T02:17:55.769 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "snapshot_clone_delay", 2024-09-12T02:17:55.769 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T02:17:55.769 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.769 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.769 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-12T02:17:55.769 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.769 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.769 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.770 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-12T02:17:55.770 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.770 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.770 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.770 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.770 INFO:teuthology.orchestra.run.smithi016.stdout: "snapshot_clone_no_wait": { 2024-09-12T02:17:55.770 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "snapshot_clone_no_wait", 2024-09-12T02:17:55.770 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.770 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.770 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.770 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T02:17:55.770 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.770 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.771 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.771 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-12T02:17:55.771 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.771 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.771 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.771 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:55.771 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:55.771 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.771 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T02:17:55.771 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "zabbix", 2024-09-12T02:17:55.771 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T02:17:55.771 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T02:17:55.771 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T02:17:55.771 INFO:teuthology.orchestra.run.smithi016.stdout: "discovery_interval": { 2024-09-12T02:17:55.772 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "discovery_interval", 2024-09-12T02:17:55.772 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-09-12T02:17:55.772 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.772 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.772 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "100", 2024-09-12T02:17:55.772 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.772 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.772 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.772 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.772 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.772 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.772 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.773 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.773 INFO:teuthology.orchestra.run.smithi016.stdout: "identifier": { 2024-09-12T02:17:55.773 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "identifier", 2024-09-12T02:17:55.773 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.773 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.773 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.773 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:55.773 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.773 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.773 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.773 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.773 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.774 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.774 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.774 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.774 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-09-12T02:17:55.774 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-09-12T02:17:55.774 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-12T02:17:55.774 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.774 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.774 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-09-12T02:17:55.774 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.774 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.774 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.775 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.775 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.775 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.775 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.775 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.775 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T02:17:55.775 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T02:17:55.775 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.775 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.775 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.775 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:55.775 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.775 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.776 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:55.776 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:55.776 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:55.776 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:55.776 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:55.776 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:55.776 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:55.776 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:55.776 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.776 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.776 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.776 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.777 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.777 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T02:17:55.781 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T02:17:55.781 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.781 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.781 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.781 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:55.782 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.782 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.782 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.782 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.782 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.782 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.782 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.782 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.782 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T02:17:55.782 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T02:17:55.782 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.782 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.782 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.783 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T02:17:55.783 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.783 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.783 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:55.783 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:55.783 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:55.783 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:55.783 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:55.783 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:55.783 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:55.783 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:55.783 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.783 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.784 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.784 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.784 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.784 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T02:17:55.784 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T02:17:55.784 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.784 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.784 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.784 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:55.784 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.784 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.784 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.784 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.784 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.785 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.785 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.785 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.785 INFO:teuthology.orchestra.run.smithi016.stdout: "zabbix_host": { 2024-09-12T02:17:55.785 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "zabbix_host", 2024-09-12T02:17:55.785 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.785 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.785 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.785 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:55.785 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.785 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.785 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.785 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.786 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.786 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.786 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.786 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.786 INFO:teuthology.orchestra.run.smithi016.stdout: "zabbix_port": { 2024-09-12T02:17:55.786 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "zabbix_port", 2024-09-12T02:17:55.786 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T02:17:55.786 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.786 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.786 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10051", 2024-09-12T02:17:55.786 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.786 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.786 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.787 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.787 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.787 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.787 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.787 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.787 INFO:teuthology.orchestra.run.smithi016.stdout: "zabbix_sender": { 2024-09-12T02:17:55.787 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "zabbix_sender", 2024-09-12T02:17:55.787 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.787 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.787 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.787 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-12T02:17:55.787 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.787 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.788 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.788 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.788 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.788 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.788 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.788 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:55.788 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:55.788 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:55.788 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-09-12T02:17:55.788 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.788 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T02:17:55.789 INFO:teuthology.orchestra.run.smithi016.stdout: "gid": 24110, 2024-09-12T02:17:55.789 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "y", 2024-09-12T02:17:55.789 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr_features": 4540701547738038271, 2024-09-12T02:17:55.789 INFO:teuthology.orchestra.run.smithi016.stdout: "available_modules": [ 2024-09-12T02:17:55.789 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T02:17:55.789 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "alerts", 2024-09-12T02:17:55.789 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T02:17:55.789 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T02:17:55.789 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T02:17:55.789 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-09-12T02:17:55.789 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-09-12T02:17:55.789 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-12T02:17:55.789 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.790 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.790 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-09-12T02:17:55.790 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.790 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.790 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.790 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "How frequently to reexamine health status", 2024-09-12T02:17:55.790 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.790 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.790 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.790 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.790 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T02:17:55.790 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T02:17:55.790 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.791 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.791 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.791 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:55.791 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.791 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.791 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:55.791 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:55.791 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:55.791 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:55.791 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:55.791 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:55.791 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:55.791 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:55.792 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.792 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.792 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.792 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.792 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.792 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T02:17:55.792 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T02:17:55.792 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.792 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.792 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.792 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:55.792 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.792 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.793 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.793 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.793 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.793 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.793 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.793 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.793 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T02:17:55.793 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T02:17:55.793 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.793 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.793 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.793 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T02:17:55.793 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.794 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.794 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:55.794 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:55.794 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:55.794 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:55.794 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:55.794 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:55.794 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:55.794 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:55.794 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.794 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.794 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.794 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.795 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.795 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T02:17:55.795 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T02:17:55.795 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.795 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.795 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.795 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:55.795 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.795 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.795 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.795 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.795 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.795 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.796 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.796 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.796 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_destination": { 2024-09-12T02:17:55.796 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_destination", 2024-09-12T02:17:55.796 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.796 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.796 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.796 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:55.796 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.796 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.796 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.796 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Email address to send alerts to", 2024-09-12T02:17:55.796 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.797 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.797 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.797 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.797 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_from_name": { 2024-09-12T02:17:55.797 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_from_name", 2024-09-12T02:17:55.797 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.797 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.797 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.797 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "Ceph", 2024-09-12T02:17:55.797 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.797 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.797 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.797 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Email From: name", 2024-09-12T02:17:55.798 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.798 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.798 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.798 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.798 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_host": { 2024-09-12T02:17:55.798 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_host", 2024-09-12T02:17:55.798 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.798 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.798 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.798 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:55.798 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.798 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.798 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.799 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "SMTP server", 2024-09-12T02:17:55.799 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.799 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.799 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.799 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.799 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_password": { 2024-09-12T02:17:55.799 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_password", 2024-09-12T02:17:55.799 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.799 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.799 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.799 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:55.799 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.799 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.799 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.800 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Password to authenticate with", 2024-09-12T02:17:55.800 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.800 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.800 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.800 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.800 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_port": { 2024-09-12T02:17:55.800 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_port", 2024-09-12T02:17:55.800 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T02:17:55.800 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.800 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.800 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "465", 2024-09-12T02:17:55.800 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.800 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.800 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.801 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "SMTP port", 2024-09-12T02:17:55.801 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.801 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.801 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.801 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.801 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_sender": { 2024-09-12T02:17:55.801 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_sender", 2024-09-12T02:17:55.801 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.801 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.801 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.801 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:55.801 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.801 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.802 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.802 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "SMTP envelope sender", 2024-09-12T02:17:55.802 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.802 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.802 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.802 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.802 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_ssl": { 2024-09-12T02:17:55.802 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_ssl", 2024-09-12T02:17:55.802 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.802 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.802 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.802 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T02:17:55.802 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.803 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.803 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.803 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-12T02:17:55.803 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.803 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.803 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.803 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.803 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_user": { 2024-09-12T02:17:55.803 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_user", 2024-09-12T02:17:55.803 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.803 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.803 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.804 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:55.804 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.804 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.804 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.804 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "User to authenticate as", 2024-09-12T02:17:55.804 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.804 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.804 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.804 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:55.804 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:55.804 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.804 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T02:17:55.804 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "balancer", 2024-09-12T02:17:55.805 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T02:17:55.805 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T02:17:55.805 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T02:17:55.805 INFO:teuthology.orchestra.run.smithi016.stdout: "active": { 2024-09-12T02:17:55.805 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "active", 2024-09-12T02:17:55.805 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.805 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.805 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.805 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T02:17:55.805 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.805 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.805 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.805 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "automatically balance PGs across cluster", 2024-09-12T02:17:55.805 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.806 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.806 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.806 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.806 INFO:teuthology.orchestra.run.smithi016.stdout: "begin_time": { 2024-09-12T02:17:55.806 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "begin_time", 2024-09-12T02:17:55.806 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.806 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.806 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.806 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0000", 2024-09-12T02:17:55.806 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.806 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.806 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.806 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "beginning time of day to automatically balance", 2024-09-12T02:17:55.807 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-12T02:17:55.807 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.807 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.807 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.807 INFO:teuthology.orchestra.run.smithi016.stdout: "begin_weekday": { 2024-09-12T02:17:55.807 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "begin_weekday", 2024-09-12T02:17:55.807 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-09-12T02:17:55.807 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.807 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.807 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-12T02:17:55.807 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "0", 2024-09-12T02:17:55.807 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "6", 2024-09-12T02:17:55.807 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.807 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-12T02:17:55.808 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-12T02:17:55.808 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.808 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.808 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.808 INFO:teuthology.orchestra.run.smithi016.stdout: "crush_compat_max_iterations": { 2024-09-12T02:17:55.808 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crush_compat_max_iterations", 2024-09-12T02:17:55.808 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-09-12T02:17:55.808 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.808 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.808 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "25", 2024-09-12T02:17:55.808 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1", 2024-09-12T02:17:55.808 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "250", 2024-09-12T02:17:55.808 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.809 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-12T02:17:55.809 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.809 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.809 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.809 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.809 INFO:teuthology.orchestra.run.smithi016.stdout: "crush_compat_metrics": { 2024-09-12T02:17:55.809 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crush_compat_metrics", 2024-09-12T02:17:55.809 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.809 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.809 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.809 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "pgs,objects,bytes", 2024-09-12T02:17:55.809 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.809 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.810 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.810 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-12T02:17:55.810 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-12T02:17:55.810 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.810 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.810 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.810 INFO:teuthology.orchestra.run.smithi016.stdout: "crush_compat_step": { 2024-09-12T02:17:55.810 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crush_compat_step", 2024-09-12T02:17:55.810 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-09-12T02:17:55.810 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.810 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.810 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0.5", 2024-09-12T02:17:55.810 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "0.001", 2024-09-12T02:17:55.811 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "0.999", 2024-09-12T02:17:55.811 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.811 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "aggressiveness of optimization", 2024-09-12T02:17:55.811 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-12T02:17:55.811 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.811 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.811 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.811 INFO:teuthology.orchestra.run.smithi016.stdout: "end_time": { 2024-09-12T02:17:55.811 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "end_time", 2024-09-12T02:17:55.811 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.811 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.811 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.811 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "2359", 2024-09-12T02:17:55.812 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.812 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.812 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.812 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "ending time of day to automatically balance", 2024-09-12T02:17:55.812 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-12T02:17:55.812 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.812 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.812 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.812 INFO:teuthology.orchestra.run.smithi016.stdout: "end_weekday": { 2024-09-12T02:17:55.812 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "end_weekday", 2024-09-12T02:17:55.812 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-09-12T02:17:55.812 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.812 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.812 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-12T02:17:55.813 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "0", 2024-09-12T02:17:55.813 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "6", 2024-09-12T02:17:55.813 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.813 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-12T02:17:55.813 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-12T02:17:55.813 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.813 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.813 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.813 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T02:17:55.813 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T02:17:55.813 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.813 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.813 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.814 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:55.814 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.814 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.814 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:55.814 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:55.814 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:55.814 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:55.814 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:55.814 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:55.814 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:55.814 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:55.814 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.814 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.814 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.815 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.815 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.815 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T02:17:55.815 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T02:17:55.815 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.815 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.815 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.815 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:55.815 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.815 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.815 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.815 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.815 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.815 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.816 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.816 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.816 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T02:17:55.816 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T02:17:55.816 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.816 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.816 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.816 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T02:17:55.816 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.816 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.816 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:55.817 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:55.817 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:55.817 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:55.817 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:55.817 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:55.817 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:55.817 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:55.817 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.817 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.817 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.817 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.817 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.817 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T02:17:55.818 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T02:17:55.818 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.818 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.818 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.818 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:55.818 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.818 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.818 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.818 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.818 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.818 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.818 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.818 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.819 INFO:teuthology.orchestra.run.smithi016.stdout: "min_score": { 2024-09-12T02:17:55.819 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "min_score", 2024-09-12T02:17:55.819 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-09-12T02:17:55.819 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.819 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.819 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-12T02:17:55.819 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.819 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.819 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.819 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-12T02:17:55.819 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.819 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.819 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.819 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.820 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": { 2024-09-12T02:17:55.820 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mode", 2024-09-12T02:17:55.820 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.820 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.820 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.820 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "upmap", 2024-09-12T02:17:55.820 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.820 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.820 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:55.820 INFO:teuthology.orchestra.run.smithi016.stdout: "crush-compat", 2024-09-12T02:17:55.820 INFO:teuthology.orchestra.run.smithi016.stdout: "none", 2024-09-12T02:17:55.820 INFO:teuthology.orchestra.run.smithi016.stdout: "read", 2024-09-12T02:17:55.820 INFO:teuthology.orchestra.run.smithi016.stdout: "upmap", 2024-09-12T02:17:55.820 INFO:teuthology.orchestra.run.smithi016.stdout: "upmap-read" 2024-09-12T02:17:55.821 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:55.821 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Balancer mode", 2024-09-12T02:17:55.821 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.821 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.821 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.821 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.821 INFO:teuthology.orchestra.run.smithi016.stdout: "pool_ids": { 2024-09-12T02:17:55.821 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "pool_ids", 2024-09-12T02:17:55.821 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.821 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.821 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.821 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:55.821 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.822 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.822 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.822 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-12T02:17:55.822 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.822 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.822 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.822 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.822 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-09-12T02:17:55.822 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-09-12T02:17:55.822 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-12T02:17:55.822 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.822 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.822 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-09-12T02:17:55.823 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.823 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.823 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.823 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-12T02:17:55.823 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.823 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.823 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.823 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.823 INFO:teuthology.orchestra.run.smithi016.stdout: "upmap_max_deviation": { 2024-09-12T02:17:55.823 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "upmap_max_deviation", 2024-09-12T02:17:55.823 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T02:17:55.823 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.824 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.824 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5", 2024-09-12T02:17:55.824 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1", 2024-09-12T02:17:55.824 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.824 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.824 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-12T02:17:55.824 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-12T02:17:55.824 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.824 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.824 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.824 INFO:teuthology.orchestra.run.smithi016.stdout: "upmap_max_optimizations": { 2024-09-12T02:17:55.824 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "upmap_max_optimizations", 2024-09-12T02:17:55.824 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-09-12T02:17:55.825 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.825 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.825 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-12T02:17:55.825 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.825 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.825 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.825 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-12T02:17:55.825 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.825 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.825 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.825 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:55.825 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:55.825 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.826 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T02:17:55.826 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cephadm", 2024-09-12T02:17:55.826 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T02:17:55.826 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T02:17:55.826 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T02:17:55.826 INFO:teuthology.orchestra.run.smithi016.stdout: "agent_down_multiplier": { 2024-09-12T02:17:55.826 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "agent_down_multiplier", 2024-09-12T02:17:55.826 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-09-12T02:17:55.826 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.826 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.826 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "3.0", 2024-09-12T02:17:55.827 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.827 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.827 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.827 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-12T02:17:55.827 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.827 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.827 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.827 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.827 INFO:teuthology.orchestra.run.smithi016.stdout: "agent_refresh_rate": { 2024-09-12T02:17:55.827 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "agent_refresh_rate", 2024-09-12T02:17:55.827 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-12T02:17:55.827 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.827 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.828 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "20", 2024-09-12T02:17:55.828 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.828 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.828 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.828 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-12T02:17:55.828 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.828 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.828 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.828 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.828 INFO:teuthology.orchestra.run.smithi016.stdout: "agent_starting_port": { 2024-09-12T02:17:55.828 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "agent_starting_port", 2024-09-12T02:17:55.828 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T02:17:55.828 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.828 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.829 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "4721", 2024-09-12T02:17:55.829 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.829 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.829 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.829 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-12T02:17:55.829 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.829 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.829 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.829 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.829 INFO:teuthology.orchestra.run.smithi016.stdout: "allow_ptrace": { 2024-09-12T02:17:55.829 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "allow_ptrace", 2024-09-12T02:17:55.829 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.829 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.830 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.830 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:55.830 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.830 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.830 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.830 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-12T02:17:55.830 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-12T02:17:55.830 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.830 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.830 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.830 INFO:teuthology.orchestra.run.smithi016.stdout: "autotune_interval": { 2024-09-12T02:17:55.830 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "autotune_interval", 2024-09-12T02:17:55.831 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-12T02:17:55.831 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.831 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.831 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-09-12T02:17:55.831 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.831 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.831 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.831 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-12T02:17:55.831 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.831 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.831 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.831 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.831 INFO:teuthology.orchestra.run.smithi016.stdout: "autotune_memory_target_ratio": { 2024-09-12T02:17:55.832 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "autotune_memory_target_ratio", 2024-09-12T02:17:55.832 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-09-12T02:17:55.832 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.832 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.832 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0.7", 2024-09-12T02:17:55.832 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.832 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.832 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.832 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-12T02:17:55.832 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.832 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.832 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.832 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.832 INFO:teuthology.orchestra.run.smithi016.stdout: "cephadm_log_destination": { 2024-09-12T02:17:55.833 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cephadm_log_destination", 2024-09-12T02:17:55.833 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.833 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.833 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.833 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:55.833 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.833 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.833 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:55.833 INFO:teuthology.orchestra.run.smithi016.stdout: "file", 2024-09-12T02:17:55.833 INFO:teuthology.orchestra.run.smithi016.stdout: "file,syslog", 2024-09-12T02:17:55.833 INFO:teuthology.orchestra.run.smithi016.stdout: "syslog" 2024-09-12T02:17:55.833 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:55.833 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-12T02:17:55.834 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.834 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.834 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.834 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.834 INFO:teuthology.orchestra.run.smithi016.stdout: "cgroups_split": { 2024-09-12T02:17:55.834 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cgroups_split", 2024-09-12T02:17:55.834 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.834 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.834 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.834 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T02:17:55.834 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.834 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.834 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.835 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-12T02:17:55.835 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.835 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.835 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.835 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.835 INFO:teuthology.orchestra.run.smithi016.stdout: "config_checks_enabled": { 2024-09-12T02:17:55.835 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "config_checks_enabled", 2024-09-12T02:17:55.835 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.835 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.835 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.835 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:55.835 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.835 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.835 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.836 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-12T02:17:55.836 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.836 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.836 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.836 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.836 INFO:teuthology.orchestra.run.smithi016.stdout: "config_dashboard": { 2024-09-12T02:17:55.836 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "config_dashboard", 2024-09-12T02:17:55.836 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.836 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.836 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.836 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T02:17:55.836 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.837 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.837 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.837 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-12T02:17:55.837 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.837 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.837 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.837 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.837 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_alertmanager": { 2024-09-12T02:17:55.837 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_alertmanager", 2024-09-12T02:17:55.837 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.837 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.837 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.837 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-12T02:17:55.838 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.838 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.838 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.838 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Prometheus container image", 2024-09-12T02:17:55.838 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.838 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.838 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.838 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.838 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_base": { 2024-09-12T02:17:55.838 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_base", 2024-09-12T02:17:55.838 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.838 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.838 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.839 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-12T02:17:55.839 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.839 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.839 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.839 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Container image name, without the tag", 2024-09-12T02:17:55.839 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.839 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.839 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.839 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.839 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_elasticsearch": { 2024-09-12T02:17:55.839 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_elasticsearch", 2024-09-12T02:17:55.839 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.839 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.839 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.840 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-12T02:17:55.840 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.840 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.840 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.840 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "elasticsearch container image", 2024-09-12T02:17:55.840 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.840 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.840 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.840 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.840 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_grafana": { 2024-09-12T02:17:55.840 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_grafana", 2024-09-12T02:17:55.840 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.840 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.841 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.841 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-12T02:17:55.841 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.841 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.841 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.841 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Prometheus container image", 2024-09-12T02:17:55.841 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.841 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.841 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.841 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.841 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_haproxy": { 2024-09-12T02:17:55.842 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_haproxy", 2024-09-12T02:17:55.842 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.842 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.842 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.842 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-12T02:17:55.842 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.842 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.842 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.842 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "HAproxy container image", 2024-09-12T02:17:55.842 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.842 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.842 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.842 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.842 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_jaeger_agent": { 2024-09-12T02:17:55.843 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_jaeger_agent", 2024-09-12T02:17:55.843 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.843 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.843 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.843 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-12T02:17:55.843 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.843 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.843 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.843 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Jaeger agent container image", 2024-09-12T02:17:55.843 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.843 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.843 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.844 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.844 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_jaeger_collector": { 2024-09-12T02:17:55.844 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_jaeger_collector", 2024-09-12T02:17:55.844 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.844 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.844 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.844 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-12T02:17:55.844 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.844 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.844 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.844 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Jaeger collector container image", 2024-09-12T02:17:55.845 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.845 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.845 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.845 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.845 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_jaeger_query": { 2024-09-12T02:17:55.845 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_jaeger_query", 2024-09-12T02:17:55.845 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.845 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.845 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.845 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-12T02:17:55.845 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.845 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.845 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.846 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Jaeger query container image", 2024-09-12T02:17:55.846 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.846 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.846 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.846 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.846 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_keepalived": { 2024-09-12T02:17:55.846 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_keepalived", 2024-09-12T02:17:55.846 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.846 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.846 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.846 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-12T02:17:55.846 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.846 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.847 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.847 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Keepalived container image", 2024-09-12T02:17:55.847 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.847 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.847 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.847 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.847 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_loki": { 2024-09-12T02:17:55.847 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_loki", 2024-09-12T02:17:55.847 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.847 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.847 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.847 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-12T02:17:55.848 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.848 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.848 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.848 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Loki container image", 2024-09-12T02:17:55.848 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.848 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.848 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.848 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.848 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_node_exporter": { 2024-09-12T02:17:55.848 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_node_exporter", 2024-09-12T02:17:55.848 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.848 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.849 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.849 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-12T02:17:55.849 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.849 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.849 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.849 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Prometheus container image", 2024-09-12T02:17:55.849 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.849 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.849 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.849 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.849 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_nvmeof": { 2024-09-12T02:17:55.849 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_nvmeof", 2024-09-12T02:17:55.849 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.850 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.850 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.850 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-12T02:17:55.850 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.850 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.850 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.850 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Nvme-of container image", 2024-09-12T02:17:55.850 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.850 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.850 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.850 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.850 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_prometheus": { 2024-09-12T02:17:55.850 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_prometheus", 2024-09-12T02:17:55.851 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.851 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.851 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.851 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-12T02:17:55.851 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.851 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.851 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.851 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Prometheus container image", 2024-09-12T02:17:55.851 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.851 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.851 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.851 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.851 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_promtail": { 2024-09-12T02:17:55.852 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_promtail", 2024-09-12T02:17:55.852 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.852 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.852 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.852 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-12T02:17:55.852 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.852 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.852 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.852 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Promtail container image", 2024-09-12T02:17:55.852 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.852 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.852 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.852 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.853 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_samba": { 2024-09-12T02:17:55.853 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_samba", 2024-09-12T02:17:55.853 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.853 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.853 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.853 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-12T02:17:55.853 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.853 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.853 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.853 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Samba/SMB container image", 2024-09-12T02:17:55.853 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.853 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.853 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.854 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.854 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_snmp_gateway": { 2024-09-12T02:17:55.854 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_snmp_gateway", 2024-09-12T02:17:55.854 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.854 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.854 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.854 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-12T02:17:55.854 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.854 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.854 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.854 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "SNMP Gateway container image", 2024-09-12T02:17:55.854 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.854 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.854 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.854 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.855 INFO:teuthology.orchestra.run.smithi016.stdout: "container_init": { 2024-09-12T02:17:55.855 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_init", 2024-09-12T02:17:55.855 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.855 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.855 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.855 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T02:17:55.855 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.855 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.855 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.855 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Run podman/docker with `--init`", 2024-09-12T02:17:55.855 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.855 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.855 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.855 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.855 INFO:teuthology.orchestra.run.smithi016.stdout: "daemon_cache_timeout": { 2024-09-12T02:17:55.856 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "daemon_cache_timeout", 2024-09-12T02:17:55.856 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-12T02:17:55.856 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.856 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.856 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-09-12T02:17:55.856 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.856 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.856 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.856 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-12T02:17:55.856 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.856 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.856 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.856 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.857 INFO:teuthology.orchestra.run.smithi016.stdout: "default_cephadm_command_timeout": { 2024-09-12T02:17:55.857 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "default_cephadm_command_timeout", 2024-09-12T02:17:55.857 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T02:17:55.857 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.857 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.857 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "900", 2024-09-12T02:17:55.857 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.857 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.857 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.857 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-12T02:17:55.857 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.857 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.858 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.858 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.858 INFO:teuthology.orchestra.run.smithi016.stdout: "default_registry": { 2024-09-12T02:17:55.858 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "default_registry", 2024-09-12T02:17:55.858 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.858 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.858 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.858 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "docker.io", 2024-09-12T02:17:55.858 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.858 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.858 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.858 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-12T02:17:55.858 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.859 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.859 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.859 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.859 INFO:teuthology.orchestra.run.smithi016.stdout: "device_cache_timeout": { 2024-09-12T02:17:55.859 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "device_cache_timeout", 2024-09-12T02:17:55.859 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-12T02:17:55.859 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.859 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.859 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "1800", 2024-09-12T02:17:55.859 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.859 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.859 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.859 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "seconds to cache device inventory", 2024-09-12T02:17:55.860 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.860 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.860 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.860 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.860 INFO:teuthology.orchestra.run.smithi016.stdout: "device_enhanced_scan": { 2024-09-12T02:17:55.860 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "device_enhanced_scan", 2024-09-12T02:17:55.860 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.860 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.860 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.860 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:55.860 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.860 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.861 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.861 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-12T02:17:55.861 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.861 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.861 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.861 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.861 INFO:teuthology.orchestra.run.smithi016.stdout: "facts_cache_timeout": { 2024-09-12T02:17:55.861 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "facts_cache_timeout", 2024-09-12T02:17:55.861 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-12T02:17:55.861 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.861 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.861 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-09-12T02:17:55.861 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.862 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.862 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.862 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "seconds to cache host facts data", 2024-09-12T02:17:55.862 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.862 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.862 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.862 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.862 INFO:teuthology.orchestra.run.smithi016.stdout: "grafana_dashboards_path": { 2024-09-12T02:17:55.862 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "grafana_dashboards_path", 2024-09-12T02:17:55.862 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.862 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.862 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.862 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-12T02:17:55.862 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.862 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.863 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.863 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-12T02:17:55.863 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.863 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.863 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.863 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.863 INFO:teuthology.orchestra.run.smithi016.stdout: "host_check_interval": { 2024-09-12T02:17:55.863 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "host_check_interval", 2024-09-12T02:17:55.863 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-12T02:17:55.863 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.863 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.863 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-09-12T02:17:55.863 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.864 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.864 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.864 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to perform a host check", 2024-09-12T02:17:55.864 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.864 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.864 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.864 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.864 INFO:teuthology.orchestra.run.smithi016.stdout: "hw_monitoring": { 2024-09-12T02:17:55.864 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "hw_monitoring", 2024-09-12T02:17:55.864 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.864 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.864 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.864 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:55.865 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.865 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.865 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.865 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-12T02:17:55.865 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.865 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.865 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.865 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.865 INFO:teuthology.orchestra.run.smithi016.stdout: "inventory_list_all": { 2024-09-12T02:17:55.865 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "inventory_list_all", 2024-09-12T02:17:55.865 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.865 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.866 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.866 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:55.866 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.866 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.866 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.866 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-12T02:17:55.866 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.866 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.866 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.866 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.866 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T02:17:55.866 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T02:17:55.866 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.867 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.867 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.867 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:55.867 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.867 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.867 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:55.867 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:55.867 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:55.867 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:55.867 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:55.867 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:55.867 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:55.867 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:55.867 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.868 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.868 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.868 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.868 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.868 INFO:teuthology.orchestra.run.smithi016.stdout: "log_refresh_metadata": { 2024-09-12T02:17:55.868 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_refresh_metadata", 2024-09-12T02:17:55.868 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.868 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.868 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.868 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:55.868 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.869 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.869 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.869 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-12T02:17:55.869 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.869 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.869 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.869 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.869 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T02:17:55.869 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T02:17:55.869 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.869 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.869 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.869 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T02:17:55.870 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.870 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.870 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.870 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-12T02:17:55.870 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.870 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.870 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.870 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.870 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T02:17:55.870 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T02:17:55.870 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.870 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.871 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.871 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T02:17:55.871 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.871 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.871 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:55.871 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:55.871 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:55.871 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:55.871 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:55.871 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:55.871 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:55.871 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:55.872 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.872 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.872 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.872 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.872 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.872 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T02:17:55.872 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T02:17:55.872 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.872 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.872 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.872 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:55.872 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.872 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.873 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.873 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.873 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.873 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.873 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.873 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.873 INFO:teuthology.orchestra.run.smithi016.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-12T02:17:55.873 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-12T02:17:55.873 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.873 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.873 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.873 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:55.873 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.874 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.874 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.874 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-12T02:17:55.874 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.874 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.874 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.874 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.874 INFO:teuthology.orchestra.run.smithi016.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-12T02:17:55.874 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-12T02:17:55.874 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.874 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.874 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.874 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "*", 2024-09-12T02:17:55.875 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.875 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.875 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.875 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-12T02:17:55.875 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.875 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.875 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.875 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.875 INFO:teuthology.orchestra.run.smithi016.stdout: "max_count_per_host": { 2024-09-12T02:17:55.875 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_count_per_host", 2024-09-12T02:17:55.875 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T02:17:55.875 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.876 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.876 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-12T02:17:55.876 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.876 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.876 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.876 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "max number of daemons per service per host", 2024-09-12T02:17:55.876 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.876 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.876 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.876 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.876 INFO:teuthology.orchestra.run.smithi016.stdout: "max_osd_draining_count": { 2024-09-12T02:17:55.876 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_osd_draining_count", 2024-09-12T02:17:55.877 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T02:17:55.877 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.877 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.877 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-12T02:17:55.877 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.877 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.877 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.877 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-12T02:17:55.877 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.877 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.877 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.877 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.877 INFO:teuthology.orchestra.run.smithi016.stdout: "migration_current": { 2024-09-12T02:17:55.878 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "migration_current", 2024-09-12T02:17:55.878 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T02:17:55.878 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.878 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.878 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:55.878 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.878 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.878 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.878 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "internal - do not modify", 2024-09-12T02:17:55.878 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.878 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.878 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.878 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.878 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": { 2024-09-12T02:17:55.878 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mode", 2024-09-12T02:17:55.879 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.879 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.879 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.879 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "root", 2024-09-12T02:17:55.879 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.879 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.879 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:55.879 INFO:teuthology.orchestra.run.smithi016.stdout: "cephadm-package", 2024-09-12T02:17:55.879 INFO:teuthology.orchestra.run.smithi016.stdout: "root" 2024-09-12T02:17:55.879 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:55.879 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "mode for remote execution of cephadm", 2024-09-12T02:17:55.879 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.879 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.880 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.880 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.880 INFO:teuthology.orchestra.run.smithi016.stdout: "oob_default_addr": { 2024-09-12T02:17:55.880 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "oob_default_addr", 2024-09-12T02:17:55.880 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.880 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.880 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.880 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "169.254.1.1", 2024-09-12T02:17:55.880 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.880 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.880 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.880 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-12T02:17:55.880 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.881 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.881 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.881 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.881 INFO:teuthology.orchestra.run.smithi016.stdout: "prometheus_alerts_path": { 2024-09-12T02:17:55.881 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "prometheus_alerts_path", 2024-09-12T02:17:55.881 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.881 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.881 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.881 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-12T02:17:55.881 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.881 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.881 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.882 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-12T02:17:55.882 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.882 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.882 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.882 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.882 INFO:teuthology.orchestra.run.smithi016.stdout: "registry_insecure": { 2024-09-12T02:17:55.882 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "registry_insecure", 2024-09-12T02:17:55.882 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.882 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.882 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.882 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:55.882 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.882 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.882 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.883 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-12T02:17:55.883 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.883 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.883 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.883 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.883 INFO:teuthology.orchestra.run.smithi016.stdout: "registry_password": { 2024-09-12T02:17:55.883 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "registry_password", 2024-09-12T02:17:55.883 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.883 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.883 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.883 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:55.883 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.884 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.884 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.884 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-12T02:17:55.884 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.884 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.884 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.884 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.884 INFO:teuthology.orchestra.run.smithi016.stdout: "registry_url": { 2024-09-12T02:17:55.884 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "registry_url", 2024-09-12T02:17:55.884 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.884 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.884 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.884 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:55.885 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.885 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.885 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.885 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-12T02:17:55.885 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.885 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.885 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.885 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.885 INFO:teuthology.orchestra.run.smithi016.stdout: "registry_username": { 2024-09-12T02:17:55.885 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "registry_username", 2024-09-12T02:17:55.885 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.885 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.885 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.886 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:55.886 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.886 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.886 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.886 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-12T02:17:55.886 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.886 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.886 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.886 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.886 INFO:teuthology.orchestra.run.smithi016.stdout: "secure_monitoring_stack": { 2024-09-12T02:17:55.886 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "secure_monitoring_stack", 2024-09-12T02:17:55.886 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.886 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.887 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.887 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:55.887 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.887 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.887 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.887 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-12T02:17:55.887 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.887 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.887 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.887 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.887 INFO:teuthology.orchestra.run.smithi016.stdout: "service_discovery_port": { 2024-09-12T02:17:55.887 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "service_discovery_port", 2024-09-12T02:17:55.887 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T02:17:55.887 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.888 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.888 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8765", 2024-09-12T02:17:55.888 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.888 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.888 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.888 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "cephadm service discovery port", 2024-09-12T02:17:55.888 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.888 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.888 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.888 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.888 INFO:teuthology.orchestra.run.smithi016.stdout: "ssh_config_file": { 2024-09-12T02:17:55.888 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ssh_config_file", 2024-09-12T02:17:55.888 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.889 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.889 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.889 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:55.889 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.889 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.889 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.889 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-12T02:17:55.889 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.889 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.889 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.889 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.889 INFO:teuthology.orchestra.run.smithi016.stdout: "use_agent": { 2024-09-12T02:17:55.889 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "use_agent", 2024-09-12T02:17:55.890 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.890 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.890 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.890 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:55.890 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.890 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.890 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.890 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-12T02:17:55.890 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.890 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.890 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.890 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.890 INFO:teuthology.orchestra.run.smithi016.stdout: "use_repo_digest": { 2024-09-12T02:17:55.890 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "use_repo_digest", 2024-09-12T02:17:55.891 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.891 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.891 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.891 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T02:17:55.891 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.891 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.891 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.891 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-12T02:17:55.891 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.891 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.891 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.891 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.891 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_on_failed_host_check": { 2024-09-12T02:17:55.892 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_on_failed_host_check", 2024-09-12T02:17:55.892 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.892 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.892 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.892 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T02:17:55.892 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.892 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.892 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.892 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "raise a health warning if the host check fails", 2024-09-12T02:17:55.892 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.892 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.892 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.892 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.893 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_on_stray_daemons": { 2024-09-12T02:17:55.893 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_on_stray_daemons", 2024-09-12T02:17:55.893 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.893 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.893 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.893 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T02:17:55.893 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.893 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.893 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.893 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-12T02:17:55.893 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.893 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.893 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.894 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.894 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_on_stray_hosts": { 2024-09-12T02:17:55.894 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_on_stray_hosts", 2024-09-12T02:17:55.894 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.894 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.894 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.894 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T02:17:55.894 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.894 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.894 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.894 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-12T02:17:55.894 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.894 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.894 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.895 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:55.895 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:55.895 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.895 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T02:17:55.895 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crash", 2024-09-12T02:17:55.895 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T02:17:55.895 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T02:17:55.895 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T02:17:55.895 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T02:17:55.895 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T02:17:55.895 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.895 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.896 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.896 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:55.896 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.896 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.896 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:55.896 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:55.896 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:55.896 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:55.896 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:55.896 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:55.896 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:55.896 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:55.897 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.897 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.897 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.897 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.897 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.897 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T02:17:55.897 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T02:17:55.897 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.897 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.897 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.897 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:55.897 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.898 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.898 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.898 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.898 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.898 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.898 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.898 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.898 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T02:17:55.898 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T02:17:55.898 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.898 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.898 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.898 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T02:17:55.899 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.899 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.899 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:55.899 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:55.899 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:55.899 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:55.899 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:55.899 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:55.899 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:55.899 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:55.899 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.899 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.899 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.899 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.900 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.900 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T02:17:55.900 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T02:17:55.900 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.900 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.900 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.900 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:55.900 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.900 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.900 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.900 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.900 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.900 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.901 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.901 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.901 INFO:teuthology.orchestra.run.smithi016.stdout: "retain_interval": { 2024-09-12T02:17:55.901 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "retain_interval", 2024-09-12T02:17:55.901 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-12T02:17:55.901 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.901 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.901 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "31536000", 2024-09-12T02:17:55.901 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.901 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.901 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.901 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-12T02:17:55.902 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.902 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.902 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.902 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.902 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_recent_interval": { 2024-09-12T02:17:55.902 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_recent_interval", 2024-09-12T02:17:55.902 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-12T02:17:55.902 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.902 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.902 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "1209600", 2024-09-12T02:17:55.902 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.902 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.902 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.903 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-12T02:17:55.903 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.903 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.903 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.903 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:55.903 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:55.903 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.903 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T02:17:55.903 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "dashboard", 2024-09-12T02:17:55.903 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T02:17:55.903 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T02:17:55.903 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T02:17:55.903 INFO:teuthology.orchestra.run.smithi016.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-12T02:17:55.904 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-12T02:17:55.904 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T02:17:55.904 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.904 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.904 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-12T02:17:55.904 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.904 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.904 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.904 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.904 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.904 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.904 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.904 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.905 INFO:teuthology.orchestra.run.smithi016.stdout: "ALERTMANAGER_API_HOST": { 2024-09-12T02:17:55.905 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-12T02:17:55.905 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.905 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.905 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.905 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:55.905 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.905 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.905 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.905 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.905 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.905 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.905 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.905 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.906 INFO:teuthology.orchestra.run.smithi016.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-12T02:17:55.906 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-12T02:17:55.906 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.906 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.906 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.906 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T02:17:55.906 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.906 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.906 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.906 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.906 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.906 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.906 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.907 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.907 INFO:teuthology.orchestra.run.smithi016.stdout: "AUDIT_API_ENABLED": { 2024-09-12T02:17:55.907 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "AUDIT_API_ENABLED", 2024-09-12T02:17:55.907 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.907 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.907 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.907 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:55.907 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.907 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.907 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.907 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.907 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.907 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.908 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.908 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.908 INFO:teuthology.orchestra.run.smithi016.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-12T02:17:55.908 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-12T02:17:55.908 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.908 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.908 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.908 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T02:17:55.908 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.908 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.908 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.908 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.908 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.908 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.909 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.909 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.909 INFO:teuthology.orchestra.run.smithi016.stdout: "ENABLE_BROWSABLE_API": { 2024-09-12T02:17:55.909 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-12T02:17:55.909 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.909 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.909 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.909 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T02:17:55.909 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.909 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.909 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.909 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.909 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.910 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.910 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.910 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.910 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-12T02:17:55.910 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-12T02:17:55.910 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.910 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.910 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.910 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T02:17:55.910 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.910 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.910 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.910 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.911 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.911 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.911 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.911 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.911 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-12T02:17:55.911 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-12T02:17:55.911 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.911 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.911 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.911 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T02:17:55.911 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.911 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.911 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.912 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.912 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.912 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.912 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.912 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.912 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-12T02:17:55.912 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-12T02:17:55.912 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.912 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.912 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.912 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T02:17:55.912 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.912 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.913 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.913 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.913 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.913 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.913 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.913 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.913 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-12T02:17:55.913 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-12T02:17:55.913 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.913 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.913 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.913 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T02:17:55.913 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.913 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.914 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.914 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.914 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.914 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.914 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.914 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.914 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-12T02:17:55.914 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-12T02:17:55.914 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.914 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.914 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.914 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T02:17:55.914 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.914 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.915 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.915 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.915 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.915 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.915 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.915 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.915 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-12T02:17:55.915 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-12T02:17:55.915 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.915 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.915 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.915 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T02:17:55.916 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.916 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.916 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.916 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.916 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.916 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.916 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.916 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.916 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-12T02:17:55.916 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-12T02:17:55.916 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.916 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.917 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.917 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T02:17:55.917 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.917 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.917 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.917 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.917 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.917 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.917 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.917 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.917 INFO:teuthology.orchestra.run.smithi016.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-12T02:17:55.917 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-12T02:17:55.918 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.918 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.918 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.918 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:55.918 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.918 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.918 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.918 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.918 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.918 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.918 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.918 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.918 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_API_PASSWORD": { 2024-09-12T02:17:55.918 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-12T02:17:55.919 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.919 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.919 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.919 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "admin", 2024-09-12T02:17:55.919 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.919 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.919 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.919 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.919 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.919 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.919 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.919 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.919 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-12T02:17:55.920 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-12T02:17:55.920 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.920 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.920 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.920 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T02:17:55.920 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.920 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.920 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.920 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.920 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.920 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.920 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.920 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.920 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_API_URL": { 2024-09-12T02:17:55.920 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_API_URL", 2024-09-12T02:17:55.921 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.921 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.921 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.921 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:55.921 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.921 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.921 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.921 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.921 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.921 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.921 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.921 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.921 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_API_USERNAME": { 2024-09-12T02:17:55.922 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-12T02:17:55.922 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.922 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.922 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.922 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "admin", 2024-09-12T02:17:55.922 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.922 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.922 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.922 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.922 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.922 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.922 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.922 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.923 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-12T02:17:55.923 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-12T02:17:55.923 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.923 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.923 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.923 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:55.923 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.923 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.923 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.923 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.923 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.923 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.923 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.924 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.924 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-12T02:17:55.924 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-12T02:17:55.924 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.924 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.924 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.924 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:55.924 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.924 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.924 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.924 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.924 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.925 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.925 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.925 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.925 INFO:teuthology.orchestra.run.smithi016.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-12T02:17:55.925 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-12T02:17:55.925 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.925 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.925 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.925 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T02:17:55.925 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.925 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.925 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.926 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.926 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.926 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.926 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.926 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.926 INFO:teuthology.orchestra.run.smithi016.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-12T02:17:55.926 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-12T02:17:55.926 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.926 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.926 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.926 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:55.926 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.926 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.927 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.927 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.927 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.927 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.927 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.927 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.927 INFO:teuthology.orchestra.run.smithi016.stdout: "PROMETHEUS_API_HOST": { 2024-09-12T02:17:55.927 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-12T02:17:55.927 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.927 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.927 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.927 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:55.927 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.928 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.928 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.928 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.928 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.928 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.928 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.928 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.928 INFO:teuthology.orchestra.run.smithi016.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-12T02:17:55.928 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-12T02:17:55.928 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.928 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.928 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.929 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T02:17:55.929 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.929 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.929 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.929 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.929 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.929 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.929 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.929 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.929 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-12T02:17:55.929 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-12T02:17:55.929 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.929 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.930 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.930 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:55.930 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.930 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.930 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.930 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.930 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.930 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.930 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.930 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.930 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-12T02:17:55.930 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-12T02:17:55.930 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.931 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.931 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.931 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:55.931 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.931 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.931 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.931 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.931 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.931 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.931 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.931 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.931 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-12T02:17:55.931 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-12T02:17:55.932 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.932 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.932 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.932 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T02:17:55.932 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.932 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.932 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.932 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.932 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.932 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.932 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.932 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.932 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-12T02:17:55.933 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-12T02:17:55.933 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.933 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.933 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.933 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T02:17:55.933 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.933 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.933 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.933 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.933 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.933 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.933 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.933 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.934 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-12T02:17:55.934 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-12T02:17:55.934 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.934 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.934 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.934 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:55.934 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.934 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.934 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.934 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.934 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.934 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.934 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.935 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.935 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-12T02:17:55.935 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-12T02:17:55.935 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.935 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.935 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.935 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:55.935 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.935 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.935 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.935 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.935 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.935 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.935 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.936 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.936 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-12T02:17:55.936 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-12T02:17:55.936 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.936 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.936 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.936 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:55.936 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.936 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.936 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.936 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.936 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.936 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.937 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.937 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.937 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_ENABLED": { 2024-09-12T02:17:55.937 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-12T02:17:55.937 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.937 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.937 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.937 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T02:17:55.937 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.937 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.937 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.937 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.937 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.937 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.938 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.938 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.938 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-12T02:17:55.938 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-12T02:17:55.938 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.938 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.938 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.938 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-12T02:17:55.938 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.938 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.938 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.938 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.939 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.939 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.939 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.939 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.939 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-12T02:17:55.939 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-12T02:17:55.939 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T02:17:55.939 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.939 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.939 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-12T02:17:55.939 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.939 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.940 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.940 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.940 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.940 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.940 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.940 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.940 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-12T02:17:55.940 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-12T02:17:55.940 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T02:17:55.940 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.940 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.940 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8", 2024-09-12T02:17:55.940 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.941 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.941 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.941 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.941 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.941 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.941 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.941 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.941 INFO:teuthology.orchestra.run.smithi016.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-12T02:17:55.941 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-12T02:17:55.941 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T02:17:55.941 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.941 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.941 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "45", 2024-09-12T02:17:55.942 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.942 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.942 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.942 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.942 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.942 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.942 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.942 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.942 INFO:teuthology.orchestra.run.smithi016.stdout: "RGW_API_ACCESS_KEY": { 2024-09-12T02:17:55.942 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-12T02:17:55.942 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.942 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.943 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.943 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:55.943 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.943 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.943 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.943 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.943 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.943 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.943 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.943 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.943 INFO:teuthology.orchestra.run.smithi016.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-12T02:17:55.943 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-12T02:17:55.943 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.944 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.944 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.944 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "admin", 2024-09-12T02:17:55.944 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.944 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.944 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.944 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.944 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.944 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.944 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.944 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.944 INFO:teuthology.orchestra.run.smithi016.stdout: "RGW_API_SECRET_KEY": { 2024-09-12T02:17:55.944 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-12T02:17:55.945 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.945 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.945 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.945 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:55.945 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.945 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.945 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.945 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.945 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.945 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.945 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.945 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.945 INFO:teuthology.orchestra.run.smithi016.stdout: "RGW_API_SSL_VERIFY": { 2024-09-12T02:17:55.945 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-12T02:17:55.946 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.946 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.946 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.946 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T02:17:55.946 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.946 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.946 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.946 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.946 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.946 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.946 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.946 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.946 INFO:teuthology.orchestra.run.smithi016.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-12T02:17:55.947 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-12T02:17:55.947 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T02:17:55.947 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.947 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.947 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-12T02:17:55.947 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.947 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.947 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.947 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.947 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.947 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.947 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.947 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.948 INFO:teuthology.orchestra.run.smithi016.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-12T02:17:55.948 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-12T02:17:55.948 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T02:17:55.948 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.948 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.948 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-12T02:17:55.948 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.948 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.948 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.948 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.948 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.948 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.948 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.949 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.949 INFO:teuthology.orchestra.run.smithi016.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-12T02:17:55.949 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-12T02:17:55.949 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T02:17:55.949 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.949 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.949 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5", 2024-09-12T02:17:55.949 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.949 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.949 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.949 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.949 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.950 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.950 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.950 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.950 INFO:teuthology.orchestra.run.smithi016.stdout: "cross_origin_url": { 2024-09-12T02:17:55.950 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cross_origin_url", 2024-09-12T02:17:55.950 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.950 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.950 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.950 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:55.950 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.950 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.950 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.950 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.950 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.951 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.951 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.951 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.951 INFO:teuthology.orchestra.run.smithi016.stdout: "crt_file": { 2024-09-12T02:17:55.951 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crt_file", 2024-09-12T02:17:55.951 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.951 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.951 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.951 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:55.951 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.951 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.951 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.951 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.952 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.952 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.952 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.952 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.952 INFO:teuthology.orchestra.run.smithi016.stdout: "debug": { 2024-09-12T02:17:55.952 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "debug", 2024-09-12T02:17:55.952 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.952 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.952 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.952 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:55.952 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.952 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.953 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.953 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Enable/disable debug options", 2024-09-12T02:17:55.953 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.953 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.953 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.953 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.953 INFO:teuthology.orchestra.run.smithi016.stdout: "jwt_token_ttl": { 2024-09-12T02:17:55.953 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "jwt_token_ttl", 2024-09-12T02:17:55.953 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T02:17:55.953 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.953 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.953 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "28800", 2024-09-12T02:17:55.953 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.954 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.954 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.954 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.954 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.954 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.954 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.954 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.954 INFO:teuthology.orchestra.run.smithi016.stdout: "key_file": { 2024-09-12T02:17:55.954 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "key_file", 2024-09-12T02:17:55.954 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.954 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.954 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.954 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:55.954 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.955 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.955 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.955 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.955 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.955 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.955 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.955 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.955 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T02:17:55.955 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T02:17:55.955 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.955 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.955 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.955 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:55.956 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.956 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.956 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:55.956 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:55.956 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:55.956 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:55.956 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:55.956 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:55.956 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:55.956 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:55.956 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.956 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.956 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.957 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.957 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.957 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T02:17:55.957 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T02:17:55.957 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.957 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.957 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.957 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:55.957 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.957 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.957 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.957 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.957 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.957 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.958 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.958 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.958 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T02:17:55.958 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T02:17:55.958 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.958 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.958 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.958 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T02:17:55.958 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.958 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.958 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:55.958 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:55.959 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:55.959 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:55.959 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:55.959 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:55.959 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:55.959 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:55.959 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.959 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.959 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.959 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.959 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.959 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T02:17:55.959 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T02:17:55.960 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.960 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.960 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.960 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:55.960 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.960 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.960 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.960 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.960 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.960 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.960 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.960 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.961 INFO:teuthology.orchestra.run.smithi016.stdout: "motd": { 2024-09-12T02:17:55.961 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "motd", 2024-09-12T02:17:55.961 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.961 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.961 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.961 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:55.961 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.961 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.961 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.961 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "The message of the day", 2024-09-12T02:17:55.961 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.961 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.961 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.962 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.962 INFO:teuthology.orchestra.run.smithi016.stdout: "redirect_resolve_ip_addr": { 2024-09-12T02:17:55.962 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "redirect_resolve_ip_addr", 2024-09-12T02:17:55.962 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.962 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.962 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.962 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:55.962 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.962 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.962 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.962 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.962 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.962 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.962 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.963 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.963 INFO:teuthology.orchestra.run.smithi016.stdout: "server_addr": { 2024-09-12T02:17:55.963 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_addr", 2024-09-12T02:17:55.963 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.963 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.963 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.963 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "::", 2024-09-12T02:17:55.963 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.963 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.963 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.963 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.963 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.963 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.964 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.964 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.964 INFO:teuthology.orchestra.run.smithi016.stdout: "server_port": { 2024-09-12T02:17:55.964 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_port", 2024-09-12T02:17:55.964 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T02:17:55.964 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.964 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.964 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8080", 2024-09-12T02:17:55.964 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.964 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.964 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.964 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.964 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.965 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.965 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.965 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.965 INFO:teuthology.orchestra.run.smithi016.stdout: "ssl": { 2024-09-12T02:17:55.965 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ssl", 2024-09-12T02:17:55.965 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.965 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.965 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.965 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T02:17:55.965 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.965 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.965 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.965 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.966 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.966 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.966 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.966 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.966 INFO:teuthology.orchestra.run.smithi016.stdout: "ssl_server_port": { 2024-09-12T02:17:55.966 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ssl_server_port", 2024-09-12T02:17:55.966 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T02:17:55.966 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.966 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.966 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8443", 2024-09-12T02:17:55.966 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.966 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.967 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.967 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.967 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.967 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.967 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.967 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.967 INFO:teuthology.orchestra.run.smithi016.stdout: "standby_behaviour": { 2024-09-12T02:17:55.967 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "standby_behaviour", 2024-09-12T02:17:55.967 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.967 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.967 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.967 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "redirect", 2024-09-12T02:17:55.967 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.967 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.968 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:55.968 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:55.968 INFO:teuthology.orchestra.run.smithi016.stdout: "redirect" 2024-09-12T02:17:55.968 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:55.968 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.968 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.968 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.968 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.968 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.968 INFO:teuthology.orchestra.run.smithi016.stdout: "standby_error_status_code": { 2024-09-12T02:17:55.968 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "standby_error_status_code", 2024-09-12T02:17:55.968 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T02:17:55.968 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.969 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.969 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "500", 2024-09-12T02:17:55.969 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "400", 2024-09-12T02:17:55.969 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "599", 2024-09-12T02:17:55.969 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.969 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.969 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.969 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.969 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.969 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.969 INFO:teuthology.orchestra.run.smithi016.stdout: "url_prefix": { 2024-09-12T02:17:55.969 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "url_prefix", 2024-09-12T02:17:55.970 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.970 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.970 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.970 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:55.970 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.970 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.970 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.970 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.970 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.970 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.970 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.970 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:55.970 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:55.970 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.971 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T02:17:55.971 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "devicehealth", 2024-09-12T02:17:55.971 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T02:17:55.971 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T02:17:55.971 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T02:17:55.971 INFO:teuthology.orchestra.run.smithi016.stdout: "enable_monitoring": { 2024-09-12T02:17:55.971 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "enable_monitoring", 2024-09-12T02:17:55.971 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.971 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.971 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.971 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T02:17:55.971 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.972 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.972 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.972 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "monitor device health metrics", 2024-09-12T02:17:55.972 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.972 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.972 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.972 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.972 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T02:17:55.972 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T02:17:55.972 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.972 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.972 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.973 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:55.973 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.973 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.973 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:55.973 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:55.973 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:55.973 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:55.973 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:55.973 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:55.973 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:55.973 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:55.973 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.973 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.974 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.974 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.974 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.974 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T02:17:55.974 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T02:17:55.974 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.974 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.974 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.974 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:55.974 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.974 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.975 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.975 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.975 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.975 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.975 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.975 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.975 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T02:17:55.975 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T02:17:55.975 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.975 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.975 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.975 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T02:17:55.975 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.976 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.976 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:55.976 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:55.976 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:55.976 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:55.976 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:55.976 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:55.976 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:55.976 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:55.976 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.976 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.976 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.977 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.977 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.977 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T02:17:55.977 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T02:17:55.977 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.977 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.977 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.977 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:55.977 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.977 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.977 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.977 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.978 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.978 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.978 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.978 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.978 INFO:teuthology.orchestra.run.smithi016.stdout: "mark_out_threshold": { 2024-09-12T02:17:55.978 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mark_out_threshold", 2024-09-12T02:17:55.978 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-12T02:17:55.978 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.978 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.978 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "2419200", 2024-09-12T02:17:55.978 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.978 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.978 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.979 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-12T02:17:55.979 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.979 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.979 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.979 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.979 INFO:teuthology.orchestra.run.smithi016.stdout: "pool_name": { 2024-09-12T02:17:55.979 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "pool_name", 2024-09-12T02:17:55.979 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.979 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.979 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.979 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "device_health_metrics", 2024-09-12T02:17:55.979 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.979 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.980 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.980 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-12T02:17:55.980 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.980 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.980 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.980 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.980 INFO:teuthology.orchestra.run.smithi016.stdout: "retention_period": { 2024-09-12T02:17:55.980 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "retention_period", 2024-09-12T02:17:55.980 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-12T02:17:55.980 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.980 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.980 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "15552000", 2024-09-12T02:17:55.981 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.981 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.981 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.981 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how long to retain device health metrics", 2024-09-12T02:17:55.981 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.981 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.981 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.981 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.981 INFO:teuthology.orchestra.run.smithi016.stdout: "scrape_frequency": { 2024-09-12T02:17:55.981 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "scrape_frequency", 2024-09-12T02:17:55.981 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-12T02:17:55.981 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.981 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.985 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "86400", 2024-09-12T02:17:55.985 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.985 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.985 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.986 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-12T02:17:55.986 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.986 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.986 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.986 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.986 INFO:teuthology.orchestra.run.smithi016.stdout: "self_heal": { 2024-09-12T02:17:55.986 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "self_heal", 2024-09-12T02:17:55.986 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.986 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.986 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.986 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T02:17:55.986 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.986 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.987 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.987 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-12T02:17:55.987 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.987 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.987 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.987 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.987 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-09-12T02:17:55.987 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-09-12T02:17:55.987 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-12T02:17:55.987 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.987 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.987 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-09-12T02:17:55.988 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.988 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.988 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.988 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to wake up and check device health", 2024-09-12T02:17:55.988 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.988 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.988 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.988 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.988 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_threshold": { 2024-09-12T02:17:55.988 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_threshold", 2024-09-12T02:17:55.988 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-12T02:17:55.988 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.988 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.989 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "7257600", 2024-09-12T02:17:55.989 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.989 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.989 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.989 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-12T02:17:55.989 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.989 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.989 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.989 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:55.989 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:55.989 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.989 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T02:17:55.989 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "diskprediction_local", 2024-09-12T02:17:55.990 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T02:17:55.990 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T02:17:55.990 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T02:17:55.990 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T02:17:55.990 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T02:17:55.990 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.990 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.990 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.990 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:55.990 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.990 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.990 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:55.990 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:55.990 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:55.991 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:55.991 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:55.991 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:55.991 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:55.991 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:55.991 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.991 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.991 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.991 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.991 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.991 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T02:17:55.991 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T02:17:55.991 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.992 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.992 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.992 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:55.992 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.992 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.992 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.992 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.992 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.992 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.992 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.992 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.992 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T02:17:55.993 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T02:17:55.993 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.993 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.993 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.993 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T02:17:55.993 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.993 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.993 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:55.993 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:55.993 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:55.993 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:55.993 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:55.993 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:55.993 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:55.994 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:55.994 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.994 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.994 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.994 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.994 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.994 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T02:17:55.994 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T02:17:55.994 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:55.994 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.994 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:55.994 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:55.994 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.995 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.995 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.995 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.995 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.995 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.995 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.995 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.995 INFO:teuthology.orchestra.run.smithi016.stdout: "predict_interval": { 2024-09-12T02:17:55.995 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "predict_interval", 2024-09-12T02:17:55.995 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.995 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.995 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.995 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "86400", 2024-09-12T02:17:55.996 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.996 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.996 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.996 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.996 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.996 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.996 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.996 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.996 INFO:teuthology.orchestra.run.smithi016.stdout: "predictor_model": { 2024-09-12T02:17:55.996 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "predictor_model", 2024-09-12T02:17:55.996 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.996 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.997 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.997 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "prophetstor", 2024-09-12T02:17:55.997 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.997 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.997 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.997 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.997 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.997 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.997 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.997 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.997 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-09-12T02:17:55.997 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-09-12T02:17:55.997 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:55.998 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.998 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.998 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-09-12T02:17:55.998 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.998 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.998 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:55.998 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:55.998 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:55.998 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:55.998 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:55.998 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:55.998 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:55.998 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:55.999 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T02:17:55.999 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "influx", 2024-09-12T02:17:55.999 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": false, 2024-09-12T02:17:55.999 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "influxdb python module not found", 2024-09-12T02:17:55.999 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T02:17:55.999 INFO:teuthology.orchestra.run.smithi016.stdout: "batch_size": { 2024-09-12T02:17:55.999 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "batch_size", 2024-09-12T02:17:55.999 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T02:17:55.999 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:55.999 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:55.999 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5000", 2024-09-12T02:17:55.999 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:55.999 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:55.999 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.000 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-12T02:17:56.000 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.000 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.000 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.000 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.000 INFO:teuthology.orchestra.run.smithi016.stdout: "database": { 2024-09-12T02:17:56.000 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "database", 2024-09-12T02:17:56.000 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.000 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.000 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.000 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "ceph", 2024-09-12T02:17:56.000 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.000 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.001 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.001 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-12T02:17:56.001 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.001 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.001 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.001 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.001 INFO:teuthology.orchestra.run.smithi016.stdout: "hostname": { 2024-09-12T02:17:56.001 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "hostname", 2024-09-12T02:17:56.001 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.001 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.001 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.001 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:56.002 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.002 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.002 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.002 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "InfluxDB server hostname", 2024-09-12T02:17:56.002 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.002 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.002 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.002 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.002 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-09-12T02:17:56.002 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-09-12T02:17:56.002 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-12T02:17:56.002 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.002 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.003 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "30", 2024-09-12T02:17:56.003 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "5", 2024-09-12T02:17:56.003 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.003 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.003 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-12T02:17:56.003 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.003 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.003 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.003 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.003 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T02:17:56.003 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T02:17:56.003 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.003 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.004 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.004 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:56.004 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.004 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.004 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:56.004 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:56.004 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:56.004 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:56.004 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:56.004 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:56.004 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:56.004 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:56.004 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.005 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.005 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.005 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.005 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.005 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T02:17:56.005 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T02:17:56.005 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.005 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.005 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.005 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.005 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.005 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.005 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.006 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.006 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.006 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.006 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.006 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.006 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T02:17:56.006 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T02:17:56.006 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.006 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.006 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.006 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T02:17:56.006 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.006 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.007 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:56.007 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:56.007 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:56.007 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:56.007 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:56.007 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:56.007 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:56.007 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:56.007 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.007 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.007 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.007 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.007 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.007 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T02:17:56.008 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T02:17:56.008 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.008 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.008 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.008 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.008 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.008 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.008 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.008 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.008 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.008 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.008 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.008 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.008 INFO:teuthology.orchestra.run.smithi016.stdout: "password": { 2024-09-12T02:17:56.009 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "password", 2024-09-12T02:17:56.009 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.009 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.009 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.009 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:56.009 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.009 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.009 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.009 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "password of InfluxDB server user", 2024-09-12T02:17:56.009 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.009 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.009 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.009 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.010 INFO:teuthology.orchestra.run.smithi016.stdout: "port": { 2024-09-12T02:17:56.010 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "port", 2024-09-12T02:17:56.010 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T02:17:56.010 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.010 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.010 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8086", 2024-09-12T02:17:56.010 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.010 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.010 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.010 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "InfluxDB server port", 2024-09-12T02:17:56.010 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.010 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.010 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.010 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.011 INFO:teuthology.orchestra.run.smithi016.stdout: "ssl": { 2024-09-12T02:17:56.011 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ssl", 2024-09-12T02:17:56.011 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.011 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.011 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.011 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "false", 2024-09-12T02:17:56.011 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.011 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.011 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.011 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-12T02:17:56.011 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.011 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.011 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.012 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.012 INFO:teuthology.orchestra.run.smithi016.stdout: "threads": { 2024-09-12T02:17:56.012 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "threads", 2024-09-12T02:17:56.012 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T02:17:56.012 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.012 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.012 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5", 2024-09-12T02:17:56.012 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1", 2024-09-12T02:17:56.012 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "32", 2024-09-12T02:17:56.012 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.012 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-12T02:17:56.012 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.012 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.012 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.013 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.013 INFO:teuthology.orchestra.run.smithi016.stdout: "username": { 2024-09-12T02:17:56.013 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "username", 2024-09-12T02:17:56.013 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.013 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.013 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.013 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:56.013 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.013 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.013 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.013 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "username of InfluxDB server user", 2024-09-12T02:17:56.013 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.013 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.014 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.014 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.014 INFO:teuthology.orchestra.run.smithi016.stdout: "verify_ssl": { 2024-09-12T02:17:56.014 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "verify_ssl", 2024-09-12T02:17:56.014 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.014 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.014 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.014 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "true", 2024-09-12T02:17:56.014 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.014 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.014 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.014 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-12T02:17:56.014 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.015 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.015 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.015 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:56.015 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:56.015 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.015 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T02:17:56.015 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "insights", 2024-09-12T02:17:56.015 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T02:17:56.015 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T02:17:56.015 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T02:17:56.015 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T02:17:56.015 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T02:17:56.016 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.016 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.016 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.016 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:56.016 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.016 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.016 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:56.016 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:56.016 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:56.016 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:56.016 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:56.016 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:56.017 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:56.017 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:56.017 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.017 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.017 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.017 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.017 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.017 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T02:17:56.017 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T02:17:56.017 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.017 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.017 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.017 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.017 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.018 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.018 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.018 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.018 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.018 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.018 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.018 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.018 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T02:17:56.018 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T02:17:56.018 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.018 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.018 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.019 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T02:17:56.019 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.019 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.019 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:56.019 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:56.019 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:56.019 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:56.019 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:56.019 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:56.019 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:56.019 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:56.019 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.019 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.020 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.020 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.020 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.020 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T02:17:56.020 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T02:17:56.020 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.020 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.020 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.020 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.020 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.020 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.020 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.021 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.021 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.021 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.021 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.021 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:56.021 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:56.021 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.021 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T02:17:56.021 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "iostat", 2024-09-12T02:17:56.021 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T02:17:56.021 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T02:17:56.021 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T02:17:56.021 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T02:17:56.022 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T02:17:56.022 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.022 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.022 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.022 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:56.022 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.022 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.022 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:56.022 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:56.022 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:56.022 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:56.022 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:56.022 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:56.022 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:56.023 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:56.023 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.023 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.023 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.023 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.023 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.023 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T02:17:56.023 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T02:17:56.023 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.023 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.023 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.023 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.023 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.024 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.024 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.024 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.024 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.024 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.024 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.024 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.024 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T02:17:56.024 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T02:17:56.024 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.024 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.024 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.024 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T02:17:56.025 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.025 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.025 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:56.025 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:56.025 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:56.025 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:56.025 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:56.025 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:56.025 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:56.025 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:56.025 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.025 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.026 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.026 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.026 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.026 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T02:17:56.026 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T02:17:56.026 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.026 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.026 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.026 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.026 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.026 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.026 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.027 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.027 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.027 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.027 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.027 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:56.027 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:56.027 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.027 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T02:17:56.027 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "k8sevents", 2024-09-12T02:17:56.027 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T02:17:56.027 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T02:17:56.027 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T02:17:56.027 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_event_retention_days": { 2024-09-12T02:17:56.028 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ceph_event_retention_days", 2024-09-12T02:17:56.028 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T02:17:56.028 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.028 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.028 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "7", 2024-09-12T02:17:56.028 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.028 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.028 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.028 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-12T02:17:56.028 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.028 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.028 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.028 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.029 INFO:teuthology.orchestra.run.smithi016.stdout: "config_check_secs": { 2024-09-12T02:17:56.029 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "config_check_secs", 2024-09-12T02:17:56.029 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T02:17:56.029 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.029 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.029 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-12T02:17:56.029 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "10", 2024-09-12T02:17:56.029 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.029 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.029 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-12T02:17:56.029 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.029 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.029 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.030 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.030 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T02:17:56.030 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T02:17:56.030 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.030 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.030 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.030 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:56.030 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.030 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.030 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:56.030 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:56.030 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:56.030 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:56.030 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:56.031 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:56.031 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:56.031 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:56.031 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.031 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.031 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.031 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.031 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.031 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T02:17:56.031 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T02:17:56.031 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.031 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.031 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.032 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.032 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.032 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.032 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.032 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.032 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.032 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.032 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.032 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.032 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T02:17:56.032 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T02:17:56.032 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.033 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.033 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.033 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T02:17:56.033 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.033 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.033 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:56.033 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:56.033 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:56.033 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:56.033 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:56.033 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:56.033 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:56.033 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:56.034 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.034 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.034 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.034 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.034 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.034 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T02:17:56.034 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T02:17:56.034 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.034 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.034 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.034 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.034 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.034 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.034 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.035 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.035 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.035 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.035 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.035 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:56.035 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:56.035 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.035 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T02:17:56.035 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "localpool", 2024-09-12T02:17:56.035 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T02:17:56.035 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T02:17:56.035 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T02:17:56.035 INFO:teuthology.orchestra.run.smithi016.stdout: "failure_domain": { 2024-09-12T02:17:56.036 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "failure_domain", 2024-09-12T02:17:56.036 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.036 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.036 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.036 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "host", 2024-09-12T02:17:56.036 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.036 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.036 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.036 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "failure domain for any created local pool", 2024-09-12T02:17:56.036 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-12T02:17:56.036 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.036 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.036 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.037 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T02:17:56.037 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T02:17:56.037 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.037 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.037 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.037 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:56.037 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.037 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.037 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:56.037 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:56.037 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:56.037 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:56.038 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:56.038 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:56.038 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:56.038 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:56.038 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.038 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.038 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.038 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.038 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.038 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T02:17:56.038 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T02:17:56.038 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.038 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.039 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.039 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.039 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.039 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.039 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.039 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.039 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.039 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.039 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.039 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.039 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T02:17:56.039 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T02:17:56.039 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.039 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.040 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.040 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T02:17:56.040 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.040 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.040 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:56.040 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:56.040 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:56.040 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:56.040 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:56.040 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:56.040 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:56.040 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:56.040 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.041 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.041 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.041 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.041 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.041 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T02:17:56.041 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T02:17:56.041 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.041 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.041 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.041 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.041 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.041 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.041 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.042 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.042 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.042 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.042 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.042 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.042 INFO:teuthology.orchestra.run.smithi016.stdout: "min_size": { 2024-09-12T02:17:56.042 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "min_size", 2024-09-12T02:17:56.042 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T02:17:56.042 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.042 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.042 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:56.042 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.042 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.042 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.043 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "default min_size for any created local pool", 2024-09-12T02:17:56.043 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-12T02:17:56.043 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.043 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.043 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.043 INFO:teuthology.orchestra.run.smithi016.stdout: "num_rep": { 2024-09-12T02:17:56.043 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "num_rep", 2024-09-12T02:17:56.043 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T02:17:56.043 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.043 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.043 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "3", 2024-09-12T02:17:56.043 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.043 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.044 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.044 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "default replica count for any created local pool", 2024-09-12T02:17:56.044 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.044 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.044 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.044 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.044 INFO:teuthology.orchestra.run.smithi016.stdout: "pg_num": { 2024-09-12T02:17:56.044 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "pg_num", 2024-09-12T02:17:56.044 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T02:17:56.044 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.044 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.045 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "128", 2024-09-12T02:17:56.045 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.045 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.045 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.045 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "default pg_num for any created local pool", 2024-09-12T02:17:56.045 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.045 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.045 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.045 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.045 INFO:teuthology.orchestra.run.smithi016.stdout: "prefix": { 2024-09-12T02:17:56.045 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "prefix", 2024-09-12T02:17:56.045 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.046 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.046 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.046 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:56.046 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.046 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.046 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.046 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "name prefix for any created local pool", 2024-09-12T02:17:56.046 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.046 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.046 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.046 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.046 INFO:teuthology.orchestra.run.smithi016.stdout: "subtree": { 2024-09-12T02:17:56.047 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "subtree", 2024-09-12T02:17:56.047 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.047 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.047 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.047 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "rack", 2024-09-12T02:17:56.047 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.047 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.047 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.047 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-12T02:17:56.047 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-12T02:17:56.047 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.047 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.047 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:56.048 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:56.048 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.048 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T02:17:56.048 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mds_autoscaler", 2024-09-12T02:17:56.048 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T02:17:56.048 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T02:17:56.048 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T02:17:56.048 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T02:17:56.048 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T02:17:56.048 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.048 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.048 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.048 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:56.049 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.049 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.049 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:56.049 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:56.049 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:56.049 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:56.049 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:56.049 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:56.049 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:56.049 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:56.049 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.049 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.049 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.050 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.050 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.050 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T02:17:56.050 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T02:17:56.050 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.050 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.050 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.050 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.050 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.050 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.050 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.050 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.050 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.051 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.051 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.051 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.051 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T02:17:56.051 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T02:17:56.051 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.051 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.051 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.051 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T02:17:56.051 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.051 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.051 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:56.051 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:56.052 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:56.052 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:56.052 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:56.052 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:56.052 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:56.052 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:56.052 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.052 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.052 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.052 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.052 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.052 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T02:17:56.052 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T02:17:56.053 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.053 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.053 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.053 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.053 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.053 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.053 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.053 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.053 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.053 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.054 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.054 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:56.054 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:56.054 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.054 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T02:17:56.054 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mirroring", 2024-09-12T02:17:56.054 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T02:17:56.054 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T02:17:56.054 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T02:17:56.054 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T02:17:56.054 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T02:17:56.055 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.055 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.055 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.055 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:56.055 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.055 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.055 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:56.055 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:56.055 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:56.055 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:56.055 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:56.055 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:56.055 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:56.055 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:56.056 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.056 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.056 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.056 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.056 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.056 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T02:17:56.056 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T02:17:56.056 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.056 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.056 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.056 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.056 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.056 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.057 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.057 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.057 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.057 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.057 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.057 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.057 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T02:17:56.057 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T02:17:56.057 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.057 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.057 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.057 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T02:17:56.057 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.058 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.058 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:56.058 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:56.058 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:56.058 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:56.058 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:56.058 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:56.058 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:56.058 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:56.058 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.058 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.058 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.058 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.058 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.059 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T02:17:56.059 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T02:17:56.059 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.059 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.059 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.059 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.059 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.059 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.059 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.059 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.059 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.059 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.059 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.060 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:56.060 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:56.060 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.060 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T02:17:56.060 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "nfs", 2024-09-12T02:17:56.060 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T02:17:56.060 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T02:17:56.060 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T02:17:56.060 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T02:17:56.060 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T02:17:56.060 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.060 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.061 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.061 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:56.061 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.061 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.061 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:56.061 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:56.061 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:56.061 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:56.061 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:56.061 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:56.061 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:56.061 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:56.062 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.062 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.062 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.062 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.062 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.062 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T02:17:56.062 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T02:17:56.062 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.062 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.062 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.062 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.062 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.062 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.063 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.063 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.063 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.063 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.063 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.063 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.063 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T02:17:56.063 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T02:17:56.063 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.063 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.063 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.063 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T02:17:56.063 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.064 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.064 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:56.064 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:56.064 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:56.064 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:56.064 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:56.064 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:56.064 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:56.064 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:56.064 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.064 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.064 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.064 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.065 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.065 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T02:17:56.065 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T02:17:56.065 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.065 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.065 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.065 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.065 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.065 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.065 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.065 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.065 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.065 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.066 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.066 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:56.066 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:56.066 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.066 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T02:17:56.066 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "orchestrator", 2024-09-12T02:17:56.066 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T02:17:56.066 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T02:17:56.066 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T02:17:56.066 INFO:teuthology.orchestra.run.smithi016.stdout: "fail_fs": { 2024-09-12T02:17:56.066 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "fail_fs", 2024-09-12T02:17:56.066 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.067 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.067 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.067 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.067 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.067 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.067 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.067 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-12T02:17:56.067 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.067 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.067 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.067 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.067 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T02:17:56.067 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T02:17:56.067 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.068 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.068 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.068 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:56.068 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.068 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.068 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:56.068 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:56.068 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:56.068 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:56.068 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:56.068 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:56.068 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:56.069 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:56.069 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.069 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.069 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.069 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.069 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.069 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T02:17:56.069 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T02:17:56.069 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.070 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.070 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.070 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.070 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.070 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.070 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.070 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.070 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.070 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.070 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.070 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.070 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T02:17:56.070 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T02:17:56.070 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.071 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.071 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.071 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T02:17:56.071 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.071 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.071 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:56.071 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:56.071 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:56.071 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:56.071 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:56.071 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:56.072 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:56.072 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:56.072 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.072 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.072 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.072 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.072 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.072 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T02:17:56.072 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T02:17:56.072 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.073 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.073 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.073 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.073 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.073 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.073 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.073 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.073 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.073 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.073 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.073 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.074 INFO:teuthology.orchestra.run.smithi016.stdout: "orchestrator": { 2024-09-12T02:17:56.074 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "orchestrator", 2024-09-12T02:17:56.074 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.074 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.074 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.074 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:56.074 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.074 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.074 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:56.074 INFO:teuthology.orchestra.run.smithi016.stdout: "cephadm", 2024-09-12T02:17:56.074 INFO:teuthology.orchestra.run.smithi016.stdout: "rook", 2024-09-12T02:17:56.075 INFO:teuthology.orchestra.run.smithi016.stdout: "test_orchestrator" 2024-09-12T02:17:56.075 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:56.075 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Orchestrator backend", 2024-09-12T02:17:56.075 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.075 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.075 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.075 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:56.075 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:56.075 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.075 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T02:17:56.075 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "osd_perf_query", 2024-09-12T02:17:56.075 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T02:17:56.076 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T02:17:56.076 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T02:17:56.076 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T02:17:56.076 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T02:17:56.076 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.076 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.076 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.076 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:56.076 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.076 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.077 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:56.077 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:56.077 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:56.077 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:56.077 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:56.077 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:56.077 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:56.077 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:56.077 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.077 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.078 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.078 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.078 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.078 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T02:17:56.078 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T02:17:56.078 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.078 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.078 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.078 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.078 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.079 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.079 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.079 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.079 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.079 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.079 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.079 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.079 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T02:17:56.079 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T02:17:56.079 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.080 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.080 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.080 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T02:17:56.080 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.080 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.080 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:56.080 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:56.080 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:56.080 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:56.081 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:56.081 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:56.081 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:56.081 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:56.081 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.081 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.081 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.081 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.081 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.081 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T02:17:56.081 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T02:17:56.082 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.082 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.082 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.082 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.082 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.082 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.082 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.082 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.082 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.082 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.082 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.082 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:56.082 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:56.083 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.083 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T02:17:56.083 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "osd_support", 2024-09-12T02:17:56.083 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T02:17:56.083 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T02:17:56.083 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T02:17:56.083 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T02:17:56.083 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T02:17:56.083 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.083 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.083 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.083 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:56.083 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.083 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.084 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:56.084 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:56.084 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:56.084 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:56.084 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:56.084 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:56.084 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:56.084 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:56.084 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.084 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.084 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.084 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.084 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.085 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T02:17:56.085 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T02:17:56.085 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.085 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.085 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.085 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.085 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.085 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.085 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.085 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.085 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.085 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.085 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.085 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.086 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T02:17:56.086 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T02:17:56.086 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.086 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.086 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.086 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T02:17:56.086 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.086 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.086 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:56.086 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:56.086 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:56.086 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:56.086 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:56.087 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:56.087 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:56.087 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:56.087 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.087 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.087 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.087 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.087 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.087 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T02:17:56.087 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T02:17:56.087 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.087 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.087 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.088 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.088 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.088 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.088 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.088 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.088 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.088 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.088 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.088 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:56.088 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:56.088 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.088 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T02:17:56.088 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "pg_autoscaler", 2024-09-12T02:17:56.088 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T02:17:56.089 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T02:17:56.089 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T02:17:56.089 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T02:17:56.089 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T02:17:56.089 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.089 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.089 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.089 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:56.089 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.089 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.089 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:56.089 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:56.089 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:56.090 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:56.090 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:56.090 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:56.090 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:56.090 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:56.090 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.090 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.090 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.090 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.090 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.090 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T02:17:56.090 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T02:17:56.090 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.090 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.090 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.091 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.091 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.091 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.091 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.091 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.091 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.091 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.091 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.091 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.091 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T02:17:56.091 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T02:17:56.091 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.091 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.091 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.092 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T02:17:56.092 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.092 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.092 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:56.092 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:56.092 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:56.092 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:56.092 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:56.092 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:56.092 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:56.092 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:56.093 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.093 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.093 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.093 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.093 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.093 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T02:17:56.093 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T02:17:56.093 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.093 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.093 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.093 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.093 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.093 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.094 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.094 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.094 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.094 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.094 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.094 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.094 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-09-12T02:17:56.094 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-09-12T02:17:56.094 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-12T02:17:56.094 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.094 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.094 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-09-12T02:17:56.094 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.095 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.095 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.095 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.095 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.095 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.095 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.095 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.095 INFO:teuthology.orchestra.run.smithi016.stdout: "threshold": { 2024-09-12T02:17:56.095 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "threshold", 2024-09-12T02:17:56.095 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-09-12T02:17:56.095 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.095 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.096 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "3.0", 2024-09-12T02:17:56.096 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1.0", 2024-09-12T02:17:56.096 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.096 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.096 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "scaling threshold", 2024-09-12T02:17:56.096 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-12T02:17:56.096 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.096 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.096 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:56.096 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:56.096 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.096 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T02:17:56.096 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "progress", 2024-09-12T02:17:56.097 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T02:17:56.097 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T02:17:56.097 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T02:17:56.097 INFO:teuthology.orchestra.run.smithi016.stdout: "allow_pg_recovery_event": { 2024-09-12T02:17:56.097 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "allow_pg_recovery_event", 2024-09-12T02:17:56.097 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.097 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.097 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.097 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.097 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.097 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.097 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.098 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-12T02:17:56.098 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.098 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.098 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.098 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.098 INFO:teuthology.orchestra.run.smithi016.stdout: "enabled": { 2024-09-12T02:17:56.098 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "enabled", 2024-09-12T02:17:56.098 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.098 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.098 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.098 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T02:17:56.098 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.098 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.099 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.099 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.099 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.099 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.099 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.099 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.099 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T02:17:56.099 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T02:17:56.099 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.099 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.099 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.099 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:56.099 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.099 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.100 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:56.100 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:56.100 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:56.100 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:56.100 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:56.100 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:56.100 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:56.100 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:56.100 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.100 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.100 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.100 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.100 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.101 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T02:17:56.101 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T02:17:56.101 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.101 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.101 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.101 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.101 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.101 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.101 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.101 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.101 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.101 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.101 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.102 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.102 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T02:17:56.102 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T02:17:56.102 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.102 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.102 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.102 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T02:17:56.102 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.102 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.102 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:56.102 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:56.102 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:56.102 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:56.102 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:56.103 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:56.103 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:56.103 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:56.103 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.103 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.103 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.103 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.103 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.103 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T02:17:56.103 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T02:17:56.103 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.103 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.103 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.104 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.104 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.104 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.104 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.104 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.104 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.104 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.104 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.104 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.104 INFO:teuthology.orchestra.run.smithi016.stdout: "max_completed_events": { 2024-09-12T02:17:56.104 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_completed_events", 2024-09-12T02:17:56.104 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T02:17:56.104 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.105 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.105 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "50", 2024-09-12T02:17:56.105 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.105 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.105 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.105 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "number of past completed events to remember", 2024-09-12T02:17:56.105 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.105 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.105 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.105 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.105 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-09-12T02:17:56.105 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-09-12T02:17:56.105 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-12T02:17:56.105 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.106 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.106 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5", 2024-09-12T02:17:56.106 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.106 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.106 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.106 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how long the module is going to sleep", 2024-09-12T02:17:56.106 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.106 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.106 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.106 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:56.106 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:56.106 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.106 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T02:17:56.107 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "prometheus", 2024-09-12T02:17:56.107 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T02:17:56.107 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T02:17:56.107 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T02:17:56.107 INFO:teuthology.orchestra.run.smithi016.stdout: "cache": { 2024-09-12T02:17:56.107 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cache", 2024-09-12T02:17:56.107 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.107 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.107 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.107 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T02:17:56.107 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.107 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.107 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.108 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.108 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.108 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.108 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.108 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.108 INFO:teuthology.orchestra.run.smithi016.stdout: "exclude_perf_counters": { 2024-09-12T02:17:56.108 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "exclude_perf_counters", 2024-09-12T02:17:56.108 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.108 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.108 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.108 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T02:17:56.108 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.108 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.109 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.109 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-12T02:17:56.109 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-12T02:17:56.109 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.109 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.109 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.109 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T02:17:56.109 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T02:17:56.109 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.109 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.109 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.109 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:56.109 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.109 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.110 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:56.110 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:56.110 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:56.110 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:56.110 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:56.110 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:56.110 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:56.110 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:56.110 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.110 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.110 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.110 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.110 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.111 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T02:17:56.111 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T02:17:56.111 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.111 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.111 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.111 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.111 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.111 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.111 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.111 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.111 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.111 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.111 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.111 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.112 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T02:17:56.112 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T02:17:56.112 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.112 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.112 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.112 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T02:17:56.112 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.112 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.112 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:56.112 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:56.112 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:56.112 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:56.113 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:56.113 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:56.113 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:56.113 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:56.113 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.113 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.113 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.113 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.113 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.113 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T02:17:56.113 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T02:17:56.113 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.113 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.113 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.114 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.114 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.114 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.114 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.114 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.114 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.114 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.114 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.114 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.114 INFO:teuthology.orchestra.run.smithi016.stdout: "rbd_stats_pools": { 2024-09-12T02:17:56.114 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rbd_stats_pools", 2024-09-12T02:17:56.114 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.114 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.114 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.115 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:56.115 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.115 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.115 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.115 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.115 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.115 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.115 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.115 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.115 INFO:teuthology.orchestra.run.smithi016.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-12T02:17:56.115 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-12T02:17:56.115 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T02:17:56.115 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.116 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.116 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "300", 2024-09-12T02:17:56.116 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.116 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.116 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.116 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.116 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.116 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.116 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.116 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.116 INFO:teuthology.orchestra.run.smithi016.stdout: "scrape_interval": { 2024-09-12T02:17:56.116 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "scrape_interval", 2024-09-12T02:17:56.116 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-09-12T02:17:56.116 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.117 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.117 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "15.0", 2024-09-12T02:17:56.117 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.117 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.117 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.117 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.117 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.117 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.117 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.117 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.117 INFO:teuthology.orchestra.run.smithi016.stdout: "server_addr": { 2024-09-12T02:17:56.117 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_addr", 2024-09-12T02:17:56.117 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.117 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.118 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.118 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "::", 2024-09-12T02:17:56.118 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.118 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.118 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.118 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-12T02:17:56.118 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.118 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.118 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.118 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.118 INFO:teuthology.orchestra.run.smithi016.stdout: "server_port": { 2024-09-12T02:17:56.118 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_port", 2024-09-12T02:17:56.118 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T02:17:56.118 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.119 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.119 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "9283", 2024-09-12T02:17:56.119 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.119 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.119 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.119 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-12T02:17:56.119 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.119 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.119 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.119 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.119 INFO:teuthology.orchestra.run.smithi016.stdout: "stale_cache_strategy": { 2024-09-12T02:17:56.119 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "stale_cache_strategy", 2024-09-12T02:17:56.119 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.120 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.120 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.120 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "log", 2024-09-12T02:17:56.120 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.120 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.120 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.120 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.120 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.120 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.120 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.120 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.120 INFO:teuthology.orchestra.run.smithi016.stdout: "standby_behaviour": { 2024-09-12T02:17:56.120 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "standby_behaviour", 2024-09-12T02:17:56.121 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.121 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.121 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.121 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "default", 2024-09-12T02:17:56.121 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.121 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.121 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:56.121 INFO:teuthology.orchestra.run.smithi016.stdout: "default", 2024-09-12T02:17:56.121 INFO:teuthology.orchestra.run.smithi016.stdout: "error" 2024-09-12T02:17:56.121 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:56.121 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.121 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.121 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.122 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.122 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.122 INFO:teuthology.orchestra.run.smithi016.stdout: "standby_error_status_code": { 2024-09-12T02:17:56.122 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "standby_error_status_code", 2024-09-12T02:17:56.122 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T02:17:56.122 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.122 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.122 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "500", 2024-09-12T02:17:56.122 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "400", 2024-09-12T02:17:56.122 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "599", 2024-09-12T02:17:56.122 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.122 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.122 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.122 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.123 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.123 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:56.123 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:56.123 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.123 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T02:17:56.123 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rbd_support", 2024-09-12T02:17:56.123 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T02:17:56.123 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T02:17:56.123 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T02:17:56.123 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T02:17:56.123 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T02:17:56.123 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.123 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.124 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.124 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:56.124 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.124 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.124 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:56.124 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:56.124 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:56.124 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:56.124 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:56.124 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:56.124 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:56.124 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:56.124 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.125 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.125 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.125 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.125 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.125 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T02:17:56.125 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T02:17:56.125 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.125 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.125 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.125 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.125 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.125 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.125 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.126 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.126 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.126 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.126 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.126 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.126 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T02:17:56.126 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T02:17:56.126 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.126 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.126 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.126 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T02:17:56.126 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.126 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.127 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:56.127 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:56.127 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:56.127 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:56.127 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:56.127 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:56.127 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:56.127 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:56.127 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.127 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.127 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.127 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.127 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.127 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T02:17:56.128 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T02:17:56.128 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.128 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.128 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.128 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.128 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.128 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.128 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.128 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.128 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.128 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.128 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.129 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.129 INFO:teuthology.orchestra.run.smithi016.stdout: "max_concurrent_snap_create": { 2024-09-12T02:17:56.129 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_concurrent_snap_create", 2024-09-12T02:17:56.129 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T02:17:56.129 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.129 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.129 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-12T02:17:56.129 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.129 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.129 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.129 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.129 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.129 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.129 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.130 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.130 INFO:teuthology.orchestra.run.smithi016.stdout: "mirror_snapshot_schedule": { 2024-09-12T02:17:56.130 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mirror_snapshot_schedule", 2024-09-12T02:17:56.130 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.130 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.130 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.130 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:56.130 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.130 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.130 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.130 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.130 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.130 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.130 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.131 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.131 INFO:teuthology.orchestra.run.smithi016.stdout: "trash_purge_schedule": { 2024-09-12T02:17:56.131 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "trash_purge_schedule", 2024-09-12T02:17:56.131 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.131 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.131 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.131 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:56.131 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.131 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.131 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.131 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.131 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.131 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.132 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.132 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:56.132 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:56.132 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.132 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T02:17:56.132 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "restful", 2024-09-12T02:17:56.132 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T02:17:56.132 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T02:17:56.132 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T02:17:56.132 INFO:teuthology.orchestra.run.smithi016.stdout: "enable_auth": { 2024-09-12T02:17:56.132 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "enable_auth", 2024-09-12T02:17:56.132 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.132 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.133 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.133 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T02:17:56.133 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.133 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.133 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.133 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.133 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.133 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.133 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.133 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.133 INFO:teuthology.orchestra.run.smithi016.stdout: "key_file": { 2024-09-12T02:17:56.133 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "key_file", 2024-09-12T02:17:56.133 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.133 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.134 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.134 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:56.134 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.134 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.134 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.134 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.134 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.134 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.134 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.134 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.134 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T02:17:56.134 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T02:17:56.134 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.135 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.135 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.135 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:56.135 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.135 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.135 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:56.135 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:56.135 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:56.135 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:56.135 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:56.135 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:56.135 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:56.135 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:56.136 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.136 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.136 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.136 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.136 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.136 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T02:17:56.136 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T02:17:56.136 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.136 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.136 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.136 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.136 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.137 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.137 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.137 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.137 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.137 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.137 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.137 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.137 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T02:17:56.137 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T02:17:56.137 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.138 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.138 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.138 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T02:17:56.138 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.138 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.138 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:56.138 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:56.138 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:56.138 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:56.138 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:56.138 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:56.139 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:56.139 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:56.139 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.139 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.139 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.139 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.139 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.139 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T02:17:56.140 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T02:17:56.140 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.140 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.140 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.140 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.140 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.140 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.140 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.140 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.140 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.141 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.141 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.141 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.141 INFO:teuthology.orchestra.run.smithi016.stdout: "server_addr": { 2024-09-12T02:17:56.141 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_addr", 2024-09-12T02:17:56.141 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.141 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.141 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.141 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:56.142 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.142 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.142 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.142 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.142 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.142 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.142 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.142 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.142 INFO:teuthology.orchestra.run.smithi016.stdout: "server_port": { 2024-09-12T02:17:56.142 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_port", 2024-09-12T02:17:56.142 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.142 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.143 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.143 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:56.143 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.143 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.143 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.143 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.143 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.143 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.143 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.143 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:56.143 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:56.143 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.143 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T02:17:56.143 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rgw", 2024-09-12T02:17:56.144 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T02:17:56.144 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T02:17:56.144 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T02:17:56.144 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T02:17:56.144 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T02:17:56.144 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.144 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.144 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.144 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:56.144 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.144 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.144 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:56.144 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:56.145 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:56.145 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:56.145 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:56.145 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:56.145 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:56.145 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:56.145 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.145 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.145 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.145 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.145 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.145 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T02:17:56.145 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T02:17:56.146 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.146 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.146 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.146 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.146 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.146 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.146 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.146 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.146 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.146 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.146 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.146 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.146 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T02:17:56.146 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T02:17:56.147 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.147 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.147 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.147 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T02:17:56.147 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.147 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.147 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:56.147 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:56.147 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:56.147 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:56.147 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:56.147 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:56.147 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:56.147 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:56.148 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.148 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.148 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.148 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.148 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.148 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T02:17:56.148 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T02:17:56.148 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.148 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.148 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.148 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.148 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.148 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.149 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.149 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.149 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.149 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.149 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.149 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:56.149 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:56.149 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.149 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T02:17:56.149 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rook", 2024-09-12T02:17:56.149 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T02:17:56.149 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T02:17:56.149 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T02:17:56.149 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T02:17:56.150 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T02:17:56.150 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.150 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.150 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.150 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:56.150 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.150 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.150 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:56.150 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:56.150 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:56.150 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:56.150 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:56.150 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:56.151 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:56.151 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:56.151 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.151 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.151 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.151 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.151 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.151 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T02:17:56.151 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T02:17:56.151 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.151 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.151 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.151 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.152 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.152 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.152 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.152 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.152 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.152 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.152 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.152 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.152 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T02:17:56.152 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T02:17:56.152 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.152 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.152 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.153 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T02:17:56.153 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.153 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.153 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:56.153 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:56.153 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:56.153 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:56.153 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:56.153 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:56.153 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:56.153 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:56.153 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.154 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.154 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.154 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.154 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.154 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T02:17:56.154 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T02:17:56.154 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.154 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.154 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.154 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.154 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.154 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.155 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.155 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.155 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.155 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.155 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.155 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.155 INFO:teuthology.orchestra.run.smithi016.stdout: "storage_class": { 2024-09-12T02:17:56.155 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "storage_class", 2024-09-12T02:17:56.155 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.155 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.155 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.155 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "local", 2024-09-12T02:17:56.155 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.156 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.156 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.156 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-12T02:17:56.156 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.156 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.156 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.156 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:56.156 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:56.156 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.156 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T02:17:56.156 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "selftest", 2024-09-12T02:17:56.156 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T02:17:56.156 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T02:17:56.157 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T02:17:56.157 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T02:17:56.157 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T02:17:56.157 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.157 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.157 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.157 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:56.157 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.157 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.157 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:56.158 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:56.158 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:56.158 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:56.158 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:56.158 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:56.158 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:56.158 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:56.158 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.158 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.158 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.158 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.158 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.159 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T02:17:56.159 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T02:17:56.159 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.159 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.159 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.159 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.159 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.159 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.159 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.159 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.160 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.160 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.160 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.160 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.160 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T02:17:56.160 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T02:17:56.160 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.160 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.160 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.160 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T02:17:56.160 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.160 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.161 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:56.161 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:56.161 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:56.161 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:56.161 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:56.161 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:56.161 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:56.161 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:56.161 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.161 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.161 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.162 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.162 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.162 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T02:17:56.162 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T02:17:56.162 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.162 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.162 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.162 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.162 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.162 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.162 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.162 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.163 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.163 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.163 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.163 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.163 INFO:teuthology.orchestra.run.smithi016.stdout: "roption1": { 2024-09-12T02:17:56.163 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "roption1", 2024-09-12T02:17:56.163 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.163 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.163 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.163 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:56.164 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.164 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.164 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.164 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.164 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.164 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.164 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.164 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.164 INFO:teuthology.orchestra.run.smithi016.stdout: "roption2": { 2024-09-12T02:17:56.164 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "roption2", 2024-09-12T02:17:56.164 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.164 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.165 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.165 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "xyz", 2024-09-12T02:17:56.165 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.165 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.165 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.165 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.165 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.165 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.165 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.165 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.165 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption1": { 2024-09-12T02:17:56.165 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption1", 2024-09-12T02:17:56.165 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.166 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.166 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.166 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:56.166 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.166 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.166 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.166 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.166 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.166 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.166 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.166 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.166 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption2": { 2024-09-12T02:17:56.166 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption2", 2024-09-12T02:17:56.167 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T02:17:56.167 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.167 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.167 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:56.167 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.167 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.167 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.167 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.167 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.167 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.167 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.167 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.167 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption3": { 2024-09-12T02:17:56.167 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption3", 2024-09-12T02:17:56.168 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-09-12T02:17:56.168 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.168 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.168 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:56.168 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.168 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.168 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.168 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.168 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.168 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.168 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.168 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.169 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption4": { 2024-09-12T02:17:56.169 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption4", 2024-09-12T02:17:56.169 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.169 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.169 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.169 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:56.169 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.169 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.169 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.169 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.169 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.169 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.169 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.170 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.170 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption5": { 2024-09-12T02:17:56.170 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption5", 2024-09-12T02:17:56.170 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.170 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.170 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.170 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:56.170 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.170 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.170 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.170 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.170 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.170 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.170 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.171 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.171 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption6": { 2024-09-12T02:17:56.171 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption6", 2024-09-12T02:17:56.171 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.171 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.171 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.171 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T02:17:56.171 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.171 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.171 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.171 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.171 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.171 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.171 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.172 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.172 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption7": { 2024-09-12T02:17:56.172 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption7", 2024-09-12T02:17:56.172 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T02:17:56.172 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.172 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.172 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:56.172 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1", 2024-09-12T02:17:56.172 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "42", 2024-09-12T02:17:56.172 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.172 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.172 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.173 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.173 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.173 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.173 INFO:teuthology.orchestra.run.smithi016.stdout: "testkey": { 2024-09-12T02:17:56.173 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "testkey", 2024-09-12T02:17:56.173 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.173 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.173 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.173 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:56.173 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.173 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.173 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.173 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.173 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.174 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.174 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.174 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.174 INFO:teuthology.orchestra.run.smithi016.stdout: "testlkey": { 2024-09-12T02:17:56.174 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "testlkey", 2024-09-12T02:17:56.174 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.174 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.174 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.174 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:56.174 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.174 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.174 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.174 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.175 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.175 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.175 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.175 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.175 INFO:teuthology.orchestra.run.smithi016.stdout: "testnewline": { 2024-09-12T02:17:56.175 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "testnewline", 2024-09-12T02:17:56.175 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.175 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.175 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.175 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:56.175 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.175 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.175 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.176 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.176 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.176 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.176 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.176 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:56.176 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:56.176 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.176 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T02:17:56.176 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "snap_schedule", 2024-09-12T02:17:56.176 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T02:17:56.176 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T02:17:56.176 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T02:17:56.176 INFO:teuthology.orchestra.run.smithi016.stdout: "allow_m_granularity": { 2024-09-12T02:17:56.177 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "allow_m_granularity", 2024-09-12T02:17:56.177 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.177 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.177 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.177 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.177 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.177 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.177 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.177 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "allow minute scheduled snapshots", 2024-09-12T02:17:56.177 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.177 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.177 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.177 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.177 INFO:teuthology.orchestra.run.smithi016.stdout: "dump_on_update": { 2024-09-12T02:17:56.178 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "dump_on_update", 2024-09-12T02:17:56.178 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.178 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.178 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.178 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.178 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.178 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.178 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.178 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "dump database to debug log on update", 2024-09-12T02:17:56.178 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.178 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.178 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.178 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.178 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T02:17:56.179 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T02:17:56.179 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.179 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.179 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.179 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:56.179 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.179 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.179 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:56.179 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:56.179 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:56.179 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:56.179 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:56.179 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:56.180 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:56.180 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:56.180 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.180 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.180 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.180 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.180 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.180 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T02:17:56.180 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T02:17:56.180 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.180 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.180 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.181 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.181 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.181 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.181 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.181 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.181 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.181 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.181 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.181 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.181 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T02:17:56.181 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T02:17:56.181 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.181 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.181 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.182 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T02:17:56.182 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.182 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.182 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:56.182 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:56.182 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:56.182 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:56.182 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:56.182 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:56.182 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:56.182 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:56.182 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.182 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.183 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.183 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.183 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.183 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T02:17:56.183 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T02:17:56.183 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.183 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.183 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.183 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.183 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.183 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.183 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.184 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.184 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.184 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.184 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.184 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:56.184 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:56.184 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.184 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T02:17:56.184 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "stats", 2024-09-12T02:17:56.184 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T02:17:56.184 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T02:17:56.184 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T02:17:56.184 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T02:17:56.185 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T02:17:56.185 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.185 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.185 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.185 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:56.185 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.189 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.189 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:56.189 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:56.189 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:56.189 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:56.189 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:56.189 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:56.189 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:56.189 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:56.189 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.189 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.189 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.189 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.190 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.190 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T02:17:56.190 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T02:17:56.190 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.190 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.190 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.190 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.190 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.190 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.190 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.190 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.190 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.190 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.191 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.191 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.191 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T02:17:56.191 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T02:17:56.191 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.191 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.191 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.191 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T02:17:56.191 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.191 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.191 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:56.192 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:56.192 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:56.192 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:56.192 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:56.192 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:56.192 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:56.192 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:56.192 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.192 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.192 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.192 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.192 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.192 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T02:17:56.193 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T02:17:56.193 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.193 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.193 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.193 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.193 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.193 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.193 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.193 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.193 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.193 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.193 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.194 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:56.194 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:56.194 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.194 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T02:17:56.194 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "status", 2024-09-12T02:17:56.194 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T02:17:56.194 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T02:17:56.194 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T02:17:56.194 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T02:17:56.194 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T02:17:56.194 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.194 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.194 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.194 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:56.195 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.195 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.195 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:56.195 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:56.195 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:56.195 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:56.195 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:56.195 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:56.195 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:56.195 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:56.195 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.195 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.195 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.195 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.196 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.196 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T02:17:56.196 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T02:17:56.196 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.196 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.196 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.196 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.196 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.196 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.196 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.196 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.196 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.196 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.197 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.197 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.197 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T02:17:56.197 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T02:17:56.197 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.197 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.197 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.197 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T02:17:56.197 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.197 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.197 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:56.197 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:56.197 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:56.198 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:56.198 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:56.198 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:56.198 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:56.198 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:56.198 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.198 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.198 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.198 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.198 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.198 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T02:17:56.198 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T02:17:56.198 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.198 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.199 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.199 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.199 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.199 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.199 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.199 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.199 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.199 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.199 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.199 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:56.199 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:56.199 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.199 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T02:17:56.199 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "telegraf", 2024-09-12T02:17:56.200 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T02:17:56.200 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T02:17:56.200 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T02:17:56.200 INFO:teuthology.orchestra.run.smithi016.stdout: "address": { 2024-09-12T02:17:56.200 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "address", 2024-09-12T02:17:56.200 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.200 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.200 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.200 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-12T02:17:56.200 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.200 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.200 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.200 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.201 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.201 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.201 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.201 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.201 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-09-12T02:17:56.201 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-09-12T02:17:56.201 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-12T02:17:56.201 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.201 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.201 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "15", 2024-09-12T02:17:56.201 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.201 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.201 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.202 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.202 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.202 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.202 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.202 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.202 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T02:17:56.202 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T02:17:56.202 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.202 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.202 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.202 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:56.202 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.202 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.203 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:56.203 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:56.203 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:56.203 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:56.203 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:56.203 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:56.203 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:56.203 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:56.203 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.203 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.203 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.203 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.203 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.203 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T02:17:56.204 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T02:17:56.204 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.204 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.204 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.204 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.204 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.204 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.204 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.204 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.204 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.204 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.204 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.204 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.205 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T02:17:56.205 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T02:17:56.205 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.205 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.205 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.205 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T02:17:56.205 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.205 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.205 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:56.205 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:56.205 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:56.205 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:56.205 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:56.205 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:56.206 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:56.206 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:56.206 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.206 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.206 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.206 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.206 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.206 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T02:17:56.206 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T02:17:56.206 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.206 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.206 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.206 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.207 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.207 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.207 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.207 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.207 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.207 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.207 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.207 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:56.207 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:56.207 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.207 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T02:17:56.207 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "telemetry", 2024-09-12T02:17:56.207 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T02:17:56.207 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T02:17:56.208 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T02:17:56.208 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_basic": { 2024-09-12T02:17:56.208 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_basic", 2024-09-12T02:17:56.208 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.208 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.208 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.208 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T02:17:56.208 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.208 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.208 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.208 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-12T02:17:56.208 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.208 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.209 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.209 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.209 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_crash": { 2024-09-12T02:17:56.209 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_crash", 2024-09-12T02:17:56.209 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.209 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.209 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.209 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T02:17:56.209 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.209 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.209 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.209 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-12T02:17:56.209 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.210 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.210 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.210 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.210 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_device": { 2024-09-12T02:17:56.210 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_device", 2024-09-12T02:17:56.210 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.210 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.210 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.210 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T02:17:56.210 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.210 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.210 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.210 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-12T02:17:56.210 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.211 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.211 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.211 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.211 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_ident": { 2024-09-12T02:17:56.211 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_ident", 2024-09-12T02:17:56.211 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.211 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.211 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.211 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.211 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.211 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.211 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.211 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-12T02:17:56.211 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.212 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.212 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.212 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.212 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_perf": { 2024-09-12T02:17:56.212 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_perf", 2024-09-12T02:17:56.212 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.212 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.212 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.212 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.212 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.212 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.212 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.212 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-12T02:17:56.212 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.213 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.213 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.213 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.213 INFO:teuthology.orchestra.run.smithi016.stdout: "contact": { 2024-09-12T02:17:56.213 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "contact", 2024-09-12T02:17:56.213 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.213 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.213 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.213 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:56.213 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.213 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.213 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.213 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.214 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.214 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.214 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.214 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.214 INFO:teuthology.orchestra.run.smithi016.stdout: "description": { 2024-09-12T02:17:56.214 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "description", 2024-09-12T02:17:56.214 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.214 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.214 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.214 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:56.214 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.214 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.214 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.215 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.215 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.215 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.215 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.215 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.215 INFO:teuthology.orchestra.run.smithi016.stdout: "device_url": { 2024-09-12T02:17:56.215 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "device_url", 2024-09-12T02:17:56.215 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.215 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.215 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.215 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-12T02:17:56.215 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.215 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.216 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.216 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.216 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.216 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.216 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.216 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.216 INFO:teuthology.orchestra.run.smithi016.stdout: "enabled": { 2024-09-12T02:17:56.216 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "enabled", 2024-09-12T02:17:56.216 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.216 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.216 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.216 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.217 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.217 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.217 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.217 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.217 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.217 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.217 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.217 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.217 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-09-12T02:17:56.217 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-09-12T02:17:56.217 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T02:17:56.217 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.217 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.217 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "24", 2024-09-12T02:17:56.218 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "8", 2024-09-12T02:17:56.218 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.218 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.218 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.218 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.218 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.218 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.218 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.218 INFO:teuthology.orchestra.run.smithi016.stdout: "last_opt_revision": { 2024-09-12T02:17:56.218 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "last_opt_revision", 2024-09-12T02:17:56.218 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T02:17:56.218 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.218 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.219 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "1", 2024-09-12T02:17:56.219 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.219 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.219 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.219 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.219 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.219 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.219 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.219 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.219 INFO:teuthology.orchestra.run.smithi016.stdout: "leaderboard": { 2024-09-12T02:17:56.219 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "leaderboard", 2024-09-12T02:17:56.219 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.219 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.220 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.220 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.220 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.220 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.220 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.220 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.220 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.220 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.220 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.220 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.220 INFO:teuthology.orchestra.run.smithi016.stdout: "leaderboard_description": { 2024-09-12T02:17:56.220 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "leaderboard_description", 2024-09-12T02:17:56.220 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.221 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.221 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.221 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:56.221 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.221 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.221 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.221 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.221 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.221 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.221 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.221 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.221 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T02:17:56.221 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T02:17:56.221 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.222 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.222 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.222 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:56.222 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.222 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.222 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:56.222 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:56.222 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:56.222 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:56.222 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:56.222 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:56.222 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:56.222 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:56.223 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.223 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.223 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.223 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.223 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.223 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T02:17:56.223 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T02:17:56.223 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.223 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.223 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.223 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.223 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.223 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.224 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.224 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.224 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.224 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.224 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.224 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.224 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T02:17:56.224 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T02:17:56.224 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.224 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.224 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.224 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T02:17:56.224 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.225 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.225 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:56.225 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:56.225 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:56.225 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:56.225 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:56.225 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:56.225 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:56.225 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:56.225 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.225 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.225 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.225 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.226 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.226 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T02:17:56.226 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T02:17:56.226 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.226 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.226 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.226 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.226 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.226 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.226 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.226 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.226 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.226 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.227 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.227 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.227 INFO:teuthology.orchestra.run.smithi016.stdout: "organization": { 2024-09-12T02:17:56.227 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "organization", 2024-09-12T02:17:56.227 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.227 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.227 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.227 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:56.227 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.227 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.227 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.227 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.227 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.228 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.228 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.228 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.228 INFO:teuthology.orchestra.run.smithi016.stdout: "proxy": { 2024-09-12T02:17:56.228 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "proxy", 2024-09-12T02:17:56.228 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.228 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.228 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.228 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:56.228 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.228 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.228 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.228 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.229 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.229 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.229 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.229 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.229 INFO:teuthology.orchestra.run.smithi016.stdout: "url": { 2024-09-12T02:17:56.229 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "url", 2024-09-12T02:17:56.229 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.229 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.229 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.229 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-12T02:17:56.229 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.229 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.230 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.230 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.230 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.230 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.230 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.230 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:56.230 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:56.230 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.230 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T02:17:56.230 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "test_orchestrator", 2024-09-12T02:17:56.230 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T02:17:56.230 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T02:17:56.230 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T02:17:56.231 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T02:17:56.231 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T02:17:56.231 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.231 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.231 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.231 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:56.231 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.231 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.231 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:56.231 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:56.231 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:56.231 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:56.231 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:56.232 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:56.232 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:56.232 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:56.232 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.232 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.232 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.232 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.232 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.232 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T02:17:56.232 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T02:17:56.232 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.232 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.232 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.233 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.233 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.233 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.233 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.233 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.233 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.233 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.233 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.233 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.233 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T02:17:56.233 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T02:17:56.233 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.233 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.234 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.234 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T02:17:56.234 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.234 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.234 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:56.234 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:56.234 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:56.234 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:56.234 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:56.234 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:56.234 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:56.234 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:56.234 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.234 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.235 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.235 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.235 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.235 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T02:17:56.235 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T02:17:56.235 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.235 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.235 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.235 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.235 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.235 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.235 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.235 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.235 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.236 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.236 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.236 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:56.236 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:56.236 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.236 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T02:17:56.236 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "volumes", 2024-09-12T02:17:56.236 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T02:17:56.236 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T02:17:56.236 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T02:17:56.236 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T02:17:56.236 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T02:17:56.237 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.237 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.237 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.237 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:56.237 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.237 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.237 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:56.237 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:56.237 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:56.237 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:56.237 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:56.237 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:56.237 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:56.238 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:56.238 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.238 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.238 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.238 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.238 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.238 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T02:17:56.238 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T02:17:56.238 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.238 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.238 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.238 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.238 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.239 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.239 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.239 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.239 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.239 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.239 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.239 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.239 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T02:17:56.239 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T02:17:56.239 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.239 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.239 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.239 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T02:17:56.239 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.240 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.240 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:56.240 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:56.240 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:56.240 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:56.240 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:56.240 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:56.240 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:56.240 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:56.240 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.240 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.240 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.240 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.240 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.241 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T02:17:56.241 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T02:17:56.241 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.241 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.241 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.241 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.241 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.241 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.241 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.241 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.241 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.241 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.241 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.242 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.242 INFO:teuthology.orchestra.run.smithi016.stdout: "max_concurrent_clones": { 2024-09-12T02:17:56.242 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_concurrent_clones", 2024-09-12T02:17:56.242 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T02:17:56.242 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.242 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.242 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "4", 2024-09-12T02:17:56.242 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.242 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.242 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.242 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-12T02:17:56.242 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.242 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.243 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.243 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.243 INFO:teuthology.orchestra.run.smithi016.stdout: "periodic_async_work": { 2024-09-12T02:17:56.243 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "periodic_async_work", 2024-09-12T02:17:56.243 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.243 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.243 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.243 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.243 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.243 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.243 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.243 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Periodically check for async work", 2024-09-12T02:17:56.243 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.244 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.244 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.244 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.244 INFO:teuthology.orchestra.run.smithi016.stdout: "snapshot_clone_delay": { 2024-09-12T02:17:56.244 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "snapshot_clone_delay", 2024-09-12T02:17:56.244 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T02:17:56.244 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.244 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.244 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-12T02:17:56.244 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.244 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.244 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.244 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-12T02:17:56.245 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.245 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.245 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.245 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.245 INFO:teuthology.orchestra.run.smithi016.stdout: "snapshot_clone_no_wait": { 2024-09-12T02:17:56.245 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "snapshot_clone_no_wait", 2024-09-12T02:17:56.245 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.245 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.245 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.245 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T02:17:56.245 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.245 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.245 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.246 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-12T02:17:56.246 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.246 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.246 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.246 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:56.246 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:56.246 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.246 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T02:17:56.246 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "zabbix", 2024-09-12T02:17:56.246 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T02:17:56.246 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T02:17:56.246 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T02:17:56.246 INFO:teuthology.orchestra.run.smithi016.stdout: "discovery_interval": { 2024-09-12T02:17:56.247 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "discovery_interval", 2024-09-12T02:17:56.247 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-09-12T02:17:56.247 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.247 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.247 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "100", 2024-09-12T02:17:56.247 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.247 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.247 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.247 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.247 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.247 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.248 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.248 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.248 INFO:teuthology.orchestra.run.smithi016.stdout: "identifier": { 2024-09-12T02:17:56.248 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "identifier", 2024-09-12T02:17:56.248 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.248 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.248 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.248 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:56.248 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.248 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.248 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.248 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.248 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.249 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.249 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.249 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.249 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-09-12T02:17:56.249 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-09-12T02:17:56.249 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-12T02:17:56.249 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.249 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.249 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-09-12T02:17:56.249 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.249 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.249 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.249 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.250 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.250 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.250 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.250 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.250 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T02:17:56.250 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T02:17:56.250 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.250 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.250 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.250 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:56.250 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.250 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.251 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:56.251 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:56.251 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:56.251 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:56.251 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:56.251 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:56.251 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:56.251 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:56.251 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.251 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.251 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.251 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.251 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.252 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T02:17:56.252 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T02:17:56.252 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.252 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.252 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.252 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.252 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.252 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.252 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.252 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.252 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.252 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.252 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.253 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.253 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T02:17:56.253 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T02:17:56.253 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.253 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.253 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.253 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T02:17:56.253 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.253 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.253 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:56.253 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:56.253 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:56.253 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:56.254 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:56.254 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:56.254 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:56.254 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:56.254 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.254 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.254 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.254 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.254 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.254 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T02:17:56.254 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T02:17:56.254 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.254 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.254 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.255 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.255 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.255 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.255 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.255 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.255 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.255 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.255 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.255 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.255 INFO:teuthology.orchestra.run.smithi016.stdout: "zabbix_host": { 2024-09-12T02:17:56.255 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "zabbix_host", 2024-09-12T02:17:56.255 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.255 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.256 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.256 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:56.256 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.256 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.256 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.256 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.256 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.256 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.256 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.256 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.256 INFO:teuthology.orchestra.run.smithi016.stdout: "zabbix_port": { 2024-09-12T02:17:56.256 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "zabbix_port", 2024-09-12T02:17:56.256 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T02:17:56.257 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.257 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.257 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10051", 2024-09-12T02:17:56.257 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.257 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.257 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.257 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.257 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.257 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.257 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.257 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.257 INFO:teuthology.orchestra.run.smithi016.stdout: "zabbix_sender": { 2024-09-12T02:17:56.257 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "zabbix_sender", 2024-09-12T02:17:56.258 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.258 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.258 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.258 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-12T02:17:56.258 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.258 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.258 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.258 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.258 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.258 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.258 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.258 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:56.258 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:56.258 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:56.259 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-09-12T02:17:56.259 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:56.259 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:56.259 INFO:teuthology.orchestra.run.smithi016.stdout: "modules": [ 2024-09-12T02:17:56.259 INFO:teuthology.orchestra.run.smithi016.stdout: "cephadm", 2024-09-12T02:17:56.259 INFO:teuthology.orchestra.run.smithi016.stdout: "dashboard", 2024-09-12T02:17:56.259 INFO:teuthology.orchestra.run.smithi016.stdout: "iostat", 2024-09-12T02:17:56.259 INFO:teuthology.orchestra.run.smithi016.stdout: "nfs", 2024-09-12T02:17:56.259 INFO:teuthology.orchestra.run.smithi016.stdout: "restful" 2024-09-12T02:17:56.259 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:56.259 INFO:teuthology.orchestra.run.smithi016.stdout: "available_modules": [ 2024-09-12T02:17:56.259 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T02:17:56.259 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "alerts", 2024-09-12T02:17:56.260 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T02:17:56.260 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T02:17:56.260 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T02:17:56.260 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-09-12T02:17:56.260 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-09-12T02:17:56.260 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-12T02:17:56.260 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.260 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.260 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-09-12T02:17:56.260 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.260 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.260 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.261 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "How frequently to reexamine health status", 2024-09-12T02:17:56.261 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.261 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.261 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.261 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.261 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T02:17:56.261 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T02:17:56.261 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.261 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.261 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.261 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:56.261 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.261 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.262 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:56.262 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:56.262 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:56.262 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:56.262 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:56.262 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:56.262 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:56.262 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:56.262 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.262 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.262 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.262 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.262 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.262 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T02:17:56.263 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T02:17:56.263 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.263 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.263 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.263 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.263 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.263 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.263 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.263 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.263 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.263 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.263 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.263 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.264 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T02:17:56.264 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T02:17:56.264 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.264 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.264 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.264 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T02:17:56.264 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.264 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.264 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:56.264 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:56.264 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:56.264 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:56.264 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:56.265 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:56.265 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:56.265 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:56.265 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.265 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.265 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.265 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.265 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.265 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T02:17:56.265 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T02:17:56.265 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.265 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.265 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.266 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.266 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.266 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.266 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.266 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.266 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.266 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.266 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.266 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.266 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_destination": { 2024-09-12T02:17:56.266 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_destination", 2024-09-12T02:17:56.266 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.266 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.266 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.267 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:56.267 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.267 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.267 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.267 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Email address to send alerts to", 2024-09-12T02:17:56.267 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.267 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.267 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.267 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.267 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_from_name": { 2024-09-12T02:17:56.267 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_from_name", 2024-09-12T02:17:56.267 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.267 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.267 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.268 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "Ceph", 2024-09-12T02:17:56.268 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.268 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.268 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.268 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Email From: name", 2024-09-12T02:17:56.268 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.268 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.268 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.268 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.268 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_host": { 2024-09-12T02:17:56.268 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_host", 2024-09-12T02:17:56.268 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.268 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.269 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.269 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:56.269 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.269 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.269 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.269 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "SMTP server", 2024-09-12T02:17:56.269 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.269 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.269 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.269 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.269 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_password": { 2024-09-12T02:17:56.269 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_password", 2024-09-12T02:17:56.269 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.270 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.270 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.270 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:56.270 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.270 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.270 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.270 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Password to authenticate with", 2024-09-12T02:17:56.270 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.270 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.270 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.270 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.270 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_port": { 2024-09-12T02:17:56.270 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_port", 2024-09-12T02:17:56.270 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T02:17:56.271 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.271 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.271 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "465", 2024-09-12T02:17:56.271 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.271 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.271 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.271 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "SMTP port", 2024-09-12T02:17:56.271 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.271 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.271 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.271 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.271 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_sender": { 2024-09-12T02:17:56.271 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_sender", 2024-09-12T02:17:56.272 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.272 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.272 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.272 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:56.272 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.272 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.272 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.272 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "SMTP envelope sender", 2024-09-12T02:17:56.272 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.272 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.272 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.272 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.272 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_ssl": { 2024-09-12T02:17:56.273 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_ssl", 2024-09-12T02:17:56.273 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.273 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.273 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.273 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T02:17:56.273 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.273 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.273 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.273 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-12T02:17:56.273 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.273 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.273 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.273 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.273 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_user": { 2024-09-12T02:17:56.274 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_user", 2024-09-12T02:17:56.274 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.274 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.274 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.274 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:56.274 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.274 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.274 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.274 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "User to authenticate as", 2024-09-12T02:17:56.274 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.274 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.274 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.274 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:56.274 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:56.275 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.275 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T02:17:56.275 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "balancer", 2024-09-12T02:17:56.275 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T02:17:56.275 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T02:17:56.275 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T02:17:56.275 INFO:teuthology.orchestra.run.smithi016.stdout: "active": { 2024-09-12T02:17:56.275 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "active", 2024-09-12T02:17:56.275 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.275 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.275 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.275 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T02:17:56.275 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.276 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.276 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.276 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "automatically balance PGs across cluster", 2024-09-12T02:17:56.276 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.276 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.276 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.276 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.276 INFO:teuthology.orchestra.run.smithi016.stdout: "begin_time": { 2024-09-12T02:17:56.276 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "begin_time", 2024-09-12T02:17:56.276 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.276 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.276 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.277 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0000", 2024-09-12T02:17:56.277 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.277 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.277 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.277 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "beginning time of day to automatically balance", 2024-09-12T02:17:56.277 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-12T02:17:56.277 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.277 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.277 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.277 INFO:teuthology.orchestra.run.smithi016.stdout: "begin_weekday": { 2024-09-12T02:17:56.277 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "begin_weekday", 2024-09-12T02:17:56.277 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-09-12T02:17:56.277 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.278 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.278 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-12T02:17:56.278 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "0", 2024-09-12T02:17:56.278 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "6", 2024-09-12T02:17:56.278 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.278 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-12T02:17:56.278 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-12T02:17:56.278 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.278 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.278 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.278 INFO:teuthology.orchestra.run.smithi016.stdout: "crush_compat_max_iterations": { 2024-09-12T02:17:56.278 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crush_compat_max_iterations", 2024-09-12T02:17:56.279 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-09-12T02:17:56.279 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.279 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.279 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "25", 2024-09-12T02:17:56.279 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1", 2024-09-12T02:17:56.279 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "250", 2024-09-12T02:17:56.279 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.279 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-12T02:17:56.279 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.279 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.279 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.279 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.279 INFO:teuthology.orchestra.run.smithi016.stdout: "crush_compat_metrics": { 2024-09-12T02:17:56.280 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crush_compat_metrics", 2024-09-12T02:17:56.280 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.280 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.280 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.280 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "pgs,objects,bytes", 2024-09-12T02:17:56.280 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.280 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.280 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.280 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-12T02:17:56.280 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-12T02:17:56.280 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.280 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.280 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.281 INFO:teuthology.orchestra.run.smithi016.stdout: "crush_compat_step": { 2024-09-12T02:17:56.281 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crush_compat_step", 2024-09-12T02:17:56.281 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-09-12T02:17:56.281 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.281 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.281 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0.5", 2024-09-12T02:17:56.281 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "0.001", 2024-09-12T02:17:56.281 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "0.999", 2024-09-12T02:17:56.281 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.281 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "aggressiveness of optimization", 2024-09-12T02:17:56.281 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-12T02:17:56.281 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.282 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.282 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.282 INFO:teuthology.orchestra.run.smithi016.stdout: "end_time": { 2024-09-12T02:17:56.282 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "end_time", 2024-09-12T02:17:56.282 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.282 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.282 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.282 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "2359", 2024-09-12T02:17:56.282 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.282 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.282 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.282 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "ending time of day to automatically balance", 2024-09-12T02:17:56.282 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-12T02:17:56.283 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.283 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.283 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.283 INFO:teuthology.orchestra.run.smithi016.stdout: "end_weekday": { 2024-09-12T02:17:56.283 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "end_weekday", 2024-09-12T02:17:56.283 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-09-12T02:17:56.283 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.283 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.283 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-12T02:17:56.283 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "0", 2024-09-12T02:17:56.283 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "6", 2024-09-12T02:17:56.283 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.283 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-12T02:17:56.284 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-12T02:17:56.284 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.284 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.284 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.284 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T02:17:56.284 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T02:17:56.284 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.284 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.284 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.284 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:56.284 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.284 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.284 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:56.284 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:56.285 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:56.285 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:56.285 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:56.285 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:56.285 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:56.285 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:56.285 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.285 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.285 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.285 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.285 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.285 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T02:17:56.286 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T02:17:56.286 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.286 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.286 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.286 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.286 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.286 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.286 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.286 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.286 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.286 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.286 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.286 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.286 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T02:17:56.287 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T02:17:56.287 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.287 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.287 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.287 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T02:17:56.287 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.287 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.287 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:56.287 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:56.287 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:56.287 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:56.287 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:56.287 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:56.287 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:56.288 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:56.288 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.288 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.288 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.288 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.288 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.288 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T02:17:56.288 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T02:17:56.288 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.288 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.288 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.288 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.289 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.289 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.289 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.289 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.289 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.289 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.289 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.289 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.289 INFO:teuthology.orchestra.run.smithi016.stdout: "min_score": { 2024-09-12T02:17:56.289 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "min_score", 2024-09-12T02:17:56.289 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-09-12T02:17:56.289 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.289 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.290 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-12T02:17:56.290 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.290 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.290 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.290 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-12T02:17:56.290 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.290 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.290 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.290 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.290 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": { 2024-09-12T02:17:56.290 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mode", 2024-09-12T02:17:56.290 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.290 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.290 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.291 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "upmap", 2024-09-12T02:17:56.291 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.291 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.291 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:56.291 INFO:teuthology.orchestra.run.smithi016.stdout: "crush-compat", 2024-09-12T02:17:56.291 INFO:teuthology.orchestra.run.smithi016.stdout: "none", 2024-09-12T02:17:56.291 INFO:teuthology.orchestra.run.smithi016.stdout: "read", 2024-09-12T02:17:56.291 INFO:teuthology.orchestra.run.smithi016.stdout: "upmap", 2024-09-12T02:17:56.291 INFO:teuthology.orchestra.run.smithi016.stdout: "upmap-read" 2024-09-12T02:17:56.291 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:56.291 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Balancer mode", 2024-09-12T02:17:56.291 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.291 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.292 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.292 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.292 INFO:teuthology.orchestra.run.smithi016.stdout: "pool_ids": { 2024-09-12T02:17:56.292 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "pool_ids", 2024-09-12T02:17:56.292 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.292 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.292 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.292 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:56.292 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.292 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.292 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.292 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-12T02:17:56.292 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.293 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.293 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.293 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.293 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-09-12T02:17:56.293 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-09-12T02:17:56.293 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-12T02:17:56.293 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.293 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.293 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-09-12T02:17:56.293 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.293 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.293 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.293 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-12T02:17:56.294 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.294 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.294 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.294 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.294 INFO:teuthology.orchestra.run.smithi016.stdout: "upmap_max_deviation": { 2024-09-12T02:17:56.294 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "upmap_max_deviation", 2024-09-12T02:17:56.294 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T02:17:56.294 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.294 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.294 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5", 2024-09-12T02:17:56.294 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1", 2024-09-12T02:17:56.294 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.294 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.295 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-12T02:17:56.295 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-12T02:17:56.295 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.295 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.295 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.295 INFO:teuthology.orchestra.run.smithi016.stdout: "upmap_max_optimizations": { 2024-09-12T02:17:56.295 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "upmap_max_optimizations", 2024-09-12T02:17:56.295 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-09-12T02:17:56.295 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.295 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.295 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-12T02:17:56.295 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.295 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.296 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.296 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-12T02:17:56.296 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.296 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.296 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.296 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:56.296 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:56.296 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.296 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T02:17:56.296 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cephadm", 2024-09-12T02:17:56.296 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T02:17:56.296 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T02:17:56.297 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T02:17:56.297 INFO:teuthology.orchestra.run.smithi016.stdout: "agent_down_multiplier": { 2024-09-12T02:17:56.297 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "agent_down_multiplier", 2024-09-12T02:17:56.297 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-09-12T02:17:56.297 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.297 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.297 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "3.0", 2024-09-12T02:17:56.297 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.297 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.297 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.297 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-12T02:17:56.297 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.297 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.297 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.298 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.298 INFO:teuthology.orchestra.run.smithi016.stdout: "agent_refresh_rate": { 2024-09-12T02:17:56.298 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "agent_refresh_rate", 2024-09-12T02:17:56.298 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-12T02:17:56.298 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.298 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.298 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "20", 2024-09-12T02:17:56.298 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.298 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.298 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.298 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-12T02:17:56.298 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.298 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.299 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.299 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.299 INFO:teuthology.orchestra.run.smithi016.stdout: "agent_starting_port": { 2024-09-12T02:17:56.299 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "agent_starting_port", 2024-09-12T02:17:56.299 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T02:17:56.299 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.299 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.299 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "4721", 2024-09-12T02:17:56.299 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.299 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.299 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.299 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-12T02:17:56.299 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.300 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.300 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.300 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.300 INFO:teuthology.orchestra.run.smithi016.stdout: "allow_ptrace": { 2024-09-12T02:17:56.300 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "allow_ptrace", 2024-09-12T02:17:56.300 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.300 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.300 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.300 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.300 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.300 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.300 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.300 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-12T02:17:56.301 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-12T02:17:56.301 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.301 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.301 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.301 INFO:teuthology.orchestra.run.smithi016.stdout: "autotune_interval": { 2024-09-12T02:17:56.301 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "autotune_interval", 2024-09-12T02:17:56.301 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-12T02:17:56.301 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.301 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.301 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-09-12T02:17:56.301 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.301 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.301 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.302 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-12T02:17:56.302 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.302 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.302 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.302 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.302 INFO:teuthology.orchestra.run.smithi016.stdout: "autotune_memory_target_ratio": { 2024-09-12T02:17:56.302 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "autotune_memory_target_ratio", 2024-09-12T02:17:56.302 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-09-12T02:17:56.302 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.302 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.302 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0.7", 2024-09-12T02:17:56.302 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.302 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.303 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.303 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-12T02:17:56.303 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.303 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.303 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.303 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.303 INFO:teuthology.orchestra.run.smithi016.stdout: "cephadm_log_destination": { 2024-09-12T02:17:56.303 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cephadm_log_destination", 2024-09-12T02:17:56.303 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.303 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.303 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.303 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:56.303 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.303 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.304 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:56.304 INFO:teuthology.orchestra.run.smithi016.stdout: "file", 2024-09-12T02:17:56.304 INFO:teuthology.orchestra.run.smithi016.stdout: "file,syslog", 2024-09-12T02:17:56.304 INFO:teuthology.orchestra.run.smithi016.stdout: "syslog" 2024-09-12T02:17:56.304 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:56.304 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-12T02:17:56.304 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.304 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.304 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.304 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.304 INFO:teuthology.orchestra.run.smithi016.stdout: "cgroups_split": { 2024-09-12T02:17:56.304 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cgroups_split", 2024-09-12T02:17:56.304 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.305 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.305 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.305 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T02:17:56.305 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.305 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.305 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.305 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-12T02:17:56.305 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.305 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.305 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.305 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.305 INFO:teuthology.orchestra.run.smithi016.stdout: "config_checks_enabled": { 2024-09-12T02:17:56.305 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "config_checks_enabled", 2024-09-12T02:17:56.306 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.306 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.306 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.306 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.306 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.306 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.306 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.306 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-12T02:17:56.306 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.306 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.306 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.306 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.306 INFO:teuthology.orchestra.run.smithi016.stdout: "config_dashboard": { 2024-09-12T02:17:56.307 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "config_dashboard", 2024-09-12T02:17:56.307 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.307 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.307 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.307 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T02:17:56.307 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.307 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.307 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.307 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-12T02:17:56.307 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.307 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.307 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.307 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.307 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_alertmanager": { 2024-09-12T02:17:56.308 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_alertmanager", 2024-09-12T02:17:56.308 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.308 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.308 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.308 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-12T02:17:56.308 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.308 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.308 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.308 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Prometheus container image", 2024-09-12T02:17:56.308 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.308 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.308 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.308 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.309 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_base": { 2024-09-12T02:17:56.309 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_base", 2024-09-12T02:17:56.309 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.309 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.309 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.309 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-12T02:17:56.309 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.309 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.309 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.309 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Container image name, without the tag", 2024-09-12T02:17:56.309 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.309 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.309 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.310 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.310 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_elasticsearch": { 2024-09-12T02:17:56.310 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_elasticsearch", 2024-09-12T02:17:56.310 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.310 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.310 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.310 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-12T02:17:56.310 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.310 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.310 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.310 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "elasticsearch container image", 2024-09-12T02:17:56.310 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.310 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.310 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.311 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.311 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_grafana": { 2024-09-12T02:17:56.311 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_grafana", 2024-09-12T02:17:56.311 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.311 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.311 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.311 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-12T02:17:56.311 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.311 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.311 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.311 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Prometheus container image", 2024-09-12T02:17:56.311 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.312 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.312 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.312 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.312 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_haproxy": { 2024-09-12T02:17:56.312 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_haproxy", 2024-09-12T02:17:56.312 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.312 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.312 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.312 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-12T02:17:56.312 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.312 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.312 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.312 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "HAproxy container image", 2024-09-12T02:17:56.313 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.313 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.313 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.313 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.313 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_jaeger_agent": { 2024-09-12T02:17:56.313 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_jaeger_agent", 2024-09-12T02:17:56.313 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.313 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.313 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.313 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-12T02:17:56.313 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.313 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.313 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.314 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Jaeger agent container image", 2024-09-12T02:17:56.314 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.314 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.314 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.314 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.314 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_jaeger_collector": { 2024-09-12T02:17:56.314 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_jaeger_collector", 2024-09-12T02:17:56.314 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.314 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.314 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.314 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-12T02:17:56.314 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.314 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.314 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.315 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Jaeger collector container image", 2024-09-12T02:17:56.315 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.315 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.315 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.315 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.315 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_jaeger_query": { 2024-09-12T02:17:56.315 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_jaeger_query", 2024-09-12T02:17:56.315 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.315 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.315 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.315 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-12T02:17:56.315 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.315 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.316 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.316 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Jaeger query container image", 2024-09-12T02:17:56.316 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.316 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.316 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.316 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.316 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_keepalived": { 2024-09-12T02:17:56.316 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_keepalived", 2024-09-12T02:17:56.316 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.316 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.316 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.316 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-12T02:17:56.316 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.317 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.317 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.317 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Keepalived container image", 2024-09-12T02:17:56.317 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.317 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.317 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.317 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.317 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_loki": { 2024-09-12T02:17:56.317 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_loki", 2024-09-12T02:17:56.317 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.317 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.317 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.317 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-12T02:17:56.318 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.318 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.318 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.318 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Loki container image", 2024-09-12T02:17:56.318 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.318 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.318 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.318 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.318 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_node_exporter": { 2024-09-12T02:17:56.318 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_node_exporter", 2024-09-12T02:17:56.318 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.318 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.318 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.318 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-12T02:17:56.319 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.319 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.319 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.319 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Prometheus container image", 2024-09-12T02:17:56.319 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.319 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.319 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.319 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.319 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_nvmeof": { 2024-09-12T02:17:56.319 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_nvmeof", 2024-09-12T02:17:56.319 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.319 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.319 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.320 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-12T02:17:56.320 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.320 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.320 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.320 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Nvme-of container image", 2024-09-12T02:17:56.320 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.320 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.320 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.320 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.320 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_prometheus": { 2024-09-12T02:17:56.320 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_prometheus", 2024-09-12T02:17:56.320 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.320 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.321 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.321 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-12T02:17:56.321 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.321 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.321 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.321 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Prometheus container image", 2024-09-12T02:17:56.321 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.321 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.321 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.321 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.321 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_promtail": { 2024-09-12T02:17:56.321 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_promtail", 2024-09-12T02:17:56.321 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.322 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.322 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.322 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-12T02:17:56.322 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.322 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.322 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.322 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Promtail container image", 2024-09-12T02:17:56.322 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.322 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.322 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.322 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.322 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_samba": { 2024-09-12T02:17:56.322 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_samba", 2024-09-12T02:17:56.323 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.323 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.323 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.323 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-12T02:17:56.323 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.323 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.323 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.323 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Samba/SMB container image", 2024-09-12T02:17:56.323 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.323 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.323 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.323 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.323 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_snmp_gateway": { 2024-09-12T02:17:56.323 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_snmp_gateway", 2024-09-12T02:17:56.324 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.324 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.324 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.324 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-12T02:17:56.324 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.324 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.324 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.324 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "SNMP Gateway container image", 2024-09-12T02:17:56.324 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.324 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.324 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.324 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.324 INFO:teuthology.orchestra.run.smithi016.stdout: "container_init": { 2024-09-12T02:17:56.325 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_init", 2024-09-12T02:17:56.325 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.325 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.325 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.325 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T02:17:56.325 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.325 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.325 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.325 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Run podman/docker with `--init`", 2024-09-12T02:17:56.325 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.325 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.325 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.325 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.326 INFO:teuthology.orchestra.run.smithi016.stdout: "daemon_cache_timeout": { 2024-09-12T02:17:56.326 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "daemon_cache_timeout", 2024-09-12T02:17:56.326 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-12T02:17:56.326 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.326 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.326 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-09-12T02:17:56.326 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.326 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.326 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.326 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-12T02:17:56.326 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.326 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.327 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.327 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.327 INFO:teuthology.orchestra.run.smithi016.stdout: "default_cephadm_command_timeout": { 2024-09-12T02:17:56.327 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "default_cephadm_command_timeout", 2024-09-12T02:17:56.327 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T02:17:56.327 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.327 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.327 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "900", 2024-09-12T02:17:56.327 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.327 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.327 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.327 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-12T02:17:56.327 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.327 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.328 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.328 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.328 INFO:teuthology.orchestra.run.smithi016.stdout: "default_registry": { 2024-09-12T02:17:56.328 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "default_registry", 2024-09-12T02:17:56.328 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.328 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.328 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.328 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "docker.io", 2024-09-12T02:17:56.328 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.328 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.328 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.328 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-12T02:17:56.328 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.329 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.329 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.329 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.329 INFO:teuthology.orchestra.run.smithi016.stdout: "device_cache_timeout": { 2024-09-12T02:17:56.329 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "device_cache_timeout", 2024-09-12T02:17:56.329 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-12T02:17:56.329 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.329 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.329 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "1800", 2024-09-12T02:17:56.329 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.329 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.329 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.329 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "seconds to cache device inventory", 2024-09-12T02:17:56.330 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.330 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.330 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.330 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.330 INFO:teuthology.orchestra.run.smithi016.stdout: "device_enhanced_scan": { 2024-09-12T02:17:56.330 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "device_enhanced_scan", 2024-09-12T02:17:56.330 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.330 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.330 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.330 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.330 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.330 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.330 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.331 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-12T02:17:56.331 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.331 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.331 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.331 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.331 INFO:teuthology.orchestra.run.smithi016.stdout: "facts_cache_timeout": { 2024-09-12T02:17:56.331 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "facts_cache_timeout", 2024-09-12T02:17:56.331 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-12T02:17:56.331 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.331 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.331 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-09-12T02:17:56.331 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.331 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.332 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.332 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "seconds to cache host facts data", 2024-09-12T02:17:56.332 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.332 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.332 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.332 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.332 INFO:teuthology.orchestra.run.smithi016.stdout: "grafana_dashboards_path": { 2024-09-12T02:17:56.332 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "grafana_dashboards_path", 2024-09-12T02:17:56.332 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.332 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.332 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.332 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-12T02:17:56.332 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.332 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.333 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.333 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-12T02:17:56.333 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.333 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.333 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.333 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.333 INFO:teuthology.orchestra.run.smithi016.stdout: "host_check_interval": { 2024-09-12T02:17:56.333 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "host_check_interval", 2024-09-12T02:17:56.333 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-12T02:17:56.333 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.333 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.333 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-09-12T02:17:56.333 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.334 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.334 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.334 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to perform a host check", 2024-09-12T02:17:56.334 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.334 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.334 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.334 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.334 INFO:teuthology.orchestra.run.smithi016.stdout: "hw_monitoring": { 2024-09-12T02:17:56.334 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "hw_monitoring", 2024-09-12T02:17:56.334 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.334 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.334 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.335 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.335 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.335 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.335 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.335 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-12T02:17:56.335 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.335 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.335 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.335 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.335 INFO:teuthology.orchestra.run.smithi016.stdout: "inventory_list_all": { 2024-09-12T02:17:56.335 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "inventory_list_all", 2024-09-12T02:17:56.335 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.335 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.335 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.336 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.336 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.336 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.336 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.336 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-12T02:17:56.336 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.336 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.336 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.336 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.336 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T02:17:56.336 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T02:17:56.336 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.337 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.337 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.337 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:56.337 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.337 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.337 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:56.337 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:56.337 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:56.337 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:56.337 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:56.337 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:56.337 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:56.338 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:56.338 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.338 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.338 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.338 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.338 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.338 INFO:teuthology.orchestra.run.smithi016.stdout: "log_refresh_metadata": { 2024-09-12T02:17:56.338 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_refresh_metadata", 2024-09-12T02:17:56.338 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.338 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.338 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.338 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.338 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.339 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.339 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.339 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-12T02:17:56.339 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.339 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.339 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.339 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.339 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T02:17:56.339 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T02:17:56.339 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.339 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.339 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.339 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T02:17:56.340 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.340 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.340 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.340 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-12T02:17:56.340 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.340 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.340 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.340 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.340 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T02:17:56.340 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T02:17:56.340 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.341 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.341 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.341 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T02:17:56.341 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.341 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.341 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:56.341 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:56.341 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:56.341 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:56.341 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:56.341 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:56.341 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:56.342 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:56.342 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.342 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.342 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.342 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.342 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.342 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T02:17:56.342 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T02:17:56.342 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.342 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.342 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.342 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.343 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.343 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.343 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.343 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.343 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.343 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.343 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.343 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.343 INFO:teuthology.orchestra.run.smithi016.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-12T02:17:56.343 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-12T02:17:56.343 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.343 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.344 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.344 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.344 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.344 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.344 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.344 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-12T02:17:56.344 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.344 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.344 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.344 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.344 INFO:teuthology.orchestra.run.smithi016.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-12T02:17:56.344 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-12T02:17:56.345 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.345 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.345 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.345 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "*", 2024-09-12T02:17:56.345 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.345 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.345 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.345 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-12T02:17:56.345 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.345 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.345 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.345 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.345 INFO:teuthology.orchestra.run.smithi016.stdout: "max_count_per_host": { 2024-09-12T02:17:56.346 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_count_per_host", 2024-09-12T02:17:56.346 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T02:17:56.346 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.346 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.346 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-12T02:17:56.346 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.346 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.346 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.346 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "max number of daemons per service per host", 2024-09-12T02:17:56.346 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.346 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.346 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.346 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.347 INFO:teuthology.orchestra.run.smithi016.stdout: "max_osd_draining_count": { 2024-09-12T02:17:56.347 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_osd_draining_count", 2024-09-12T02:17:56.347 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T02:17:56.347 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.347 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.347 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-12T02:17:56.347 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.347 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.347 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.347 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-12T02:17:56.347 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.347 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.347 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.348 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.348 INFO:teuthology.orchestra.run.smithi016.stdout: "migration_current": { 2024-09-12T02:17:56.348 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "migration_current", 2024-09-12T02:17:56.348 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T02:17:56.348 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.348 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.348 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:56.348 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.348 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.348 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.348 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "internal - do not modify", 2024-09-12T02:17:56.348 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.348 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.349 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.349 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.349 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": { 2024-09-12T02:17:56.349 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mode", 2024-09-12T02:17:56.349 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.349 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.349 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.349 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "root", 2024-09-12T02:17:56.349 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.349 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.349 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:56.349 INFO:teuthology.orchestra.run.smithi016.stdout: "cephadm-package", 2024-09-12T02:17:56.349 INFO:teuthology.orchestra.run.smithi016.stdout: "root" 2024-09-12T02:17:56.349 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:56.350 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "mode for remote execution of cephadm", 2024-09-12T02:17:56.350 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.350 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.350 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.350 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.350 INFO:teuthology.orchestra.run.smithi016.stdout: "oob_default_addr": { 2024-09-12T02:17:56.350 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "oob_default_addr", 2024-09-12T02:17:56.350 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.350 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.350 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.350 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "169.254.1.1", 2024-09-12T02:17:56.350 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.350 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.351 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.351 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-12T02:17:56.351 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.351 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.351 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.351 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.351 INFO:teuthology.orchestra.run.smithi016.stdout: "prometheus_alerts_path": { 2024-09-12T02:17:56.351 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "prometheus_alerts_path", 2024-09-12T02:17:56.351 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.351 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.351 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.351 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-12T02:17:56.351 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.351 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.352 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.352 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-12T02:17:56.352 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.352 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.352 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.352 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.352 INFO:teuthology.orchestra.run.smithi016.stdout: "registry_insecure": { 2024-09-12T02:17:56.352 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "registry_insecure", 2024-09-12T02:17:56.352 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.352 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.352 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.352 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.352 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.353 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.353 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.353 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-12T02:17:56.353 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.353 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.353 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.353 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.353 INFO:teuthology.orchestra.run.smithi016.stdout: "registry_password": { 2024-09-12T02:17:56.353 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "registry_password", 2024-09-12T02:17:56.353 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.353 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.353 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.353 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:56.353 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.354 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.354 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.354 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-12T02:17:56.354 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.354 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.354 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.354 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.354 INFO:teuthology.orchestra.run.smithi016.stdout: "registry_url": { 2024-09-12T02:17:56.354 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "registry_url", 2024-09-12T02:17:56.354 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.354 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.354 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.354 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:56.355 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.355 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.355 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.355 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-12T02:17:56.355 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.355 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.355 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.355 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.355 INFO:teuthology.orchestra.run.smithi016.stdout: "registry_username": { 2024-09-12T02:17:56.355 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "registry_username", 2024-09-12T02:17:56.355 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.355 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.355 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.355 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:56.356 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.356 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.356 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.356 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-12T02:17:56.356 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.356 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.356 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.356 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.356 INFO:teuthology.orchestra.run.smithi016.stdout: "secure_monitoring_stack": { 2024-09-12T02:17:56.356 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "secure_monitoring_stack", 2024-09-12T02:17:56.356 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.356 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.356 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.356 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.357 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.357 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.357 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.357 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-12T02:17:56.357 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.357 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.357 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.357 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.357 INFO:teuthology.orchestra.run.smithi016.stdout: "service_discovery_port": { 2024-09-12T02:17:56.357 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "service_discovery_port", 2024-09-12T02:17:56.357 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T02:17:56.357 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.358 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.358 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8765", 2024-09-12T02:17:56.358 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.358 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.358 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.358 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "cephadm service discovery port", 2024-09-12T02:17:56.358 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.358 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.358 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.358 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.358 INFO:teuthology.orchestra.run.smithi016.stdout: "ssh_config_file": { 2024-09-12T02:17:56.358 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ssh_config_file", 2024-09-12T02:17:56.358 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.358 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.359 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.359 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:56.359 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.359 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.359 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.359 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-12T02:17:56.359 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.359 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.359 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.359 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.359 INFO:teuthology.orchestra.run.smithi016.stdout: "use_agent": { 2024-09-12T02:17:56.359 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "use_agent", 2024-09-12T02:17:56.359 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.360 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.360 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.360 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.360 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.360 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.360 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.360 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-12T02:17:56.360 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.360 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.360 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.360 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.360 INFO:teuthology.orchestra.run.smithi016.stdout: "use_repo_digest": { 2024-09-12T02:17:56.360 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "use_repo_digest", 2024-09-12T02:17:56.360 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.361 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.361 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.361 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T02:17:56.361 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.361 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.361 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.361 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-12T02:17:56.361 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.361 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.361 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.361 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.361 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_on_failed_host_check": { 2024-09-12T02:17:56.361 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_on_failed_host_check", 2024-09-12T02:17:56.362 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.362 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.362 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.362 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T02:17:56.362 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.362 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.362 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.362 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "raise a health warning if the host check fails", 2024-09-12T02:17:56.362 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.362 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.362 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.362 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.363 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_on_stray_daemons": { 2024-09-12T02:17:56.363 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_on_stray_daemons", 2024-09-12T02:17:56.363 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.363 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.363 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.363 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T02:17:56.363 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.363 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.363 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.363 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-12T02:17:56.363 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.363 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.364 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.364 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.364 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_on_stray_hosts": { 2024-09-12T02:17:56.364 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_on_stray_hosts", 2024-09-12T02:17:56.364 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.364 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.364 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.364 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T02:17:56.364 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.364 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.364 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.364 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-12T02:17:56.364 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.365 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.365 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.365 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:56.365 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:56.365 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.365 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T02:17:56.365 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crash", 2024-09-12T02:17:56.365 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T02:17:56.365 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T02:17:56.365 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T02:17:56.365 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T02:17:56.365 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T02:17:56.365 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.365 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.366 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.366 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:56.366 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.366 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.366 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:56.366 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:56.366 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:56.366 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:56.366 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:56.366 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:56.366 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:56.366 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:56.366 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.366 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.367 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.367 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.367 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.367 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T02:17:56.367 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T02:17:56.367 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.367 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.367 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.367 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.367 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.367 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.367 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.368 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.368 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.368 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.368 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.368 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.368 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T02:17:56.368 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T02:17:56.368 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.368 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.368 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.368 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T02:17:56.368 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.368 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.369 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:56.369 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:56.369 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:56.369 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:56.369 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:56.369 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:56.369 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:56.369 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:56.369 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.369 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.369 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.369 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.369 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.369 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T02:17:56.370 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T02:17:56.370 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.370 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.370 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.370 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.370 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.370 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.370 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.370 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.370 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.370 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.370 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.370 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.370 INFO:teuthology.orchestra.run.smithi016.stdout: "retain_interval": { 2024-09-12T02:17:56.371 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "retain_interval", 2024-09-12T02:17:56.371 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-12T02:17:56.371 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.371 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.371 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "31536000", 2024-09-12T02:17:56.371 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.371 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.371 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.371 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-12T02:17:56.371 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.371 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.371 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.371 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.371 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_recent_interval": { 2024-09-12T02:17:56.372 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_recent_interval", 2024-09-12T02:17:56.372 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-12T02:17:56.372 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.372 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.372 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "1209600", 2024-09-12T02:17:56.372 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.372 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.372 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.372 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-12T02:17:56.372 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.372 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.372 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.372 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:56.372 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:56.373 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.373 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T02:17:56.373 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "dashboard", 2024-09-12T02:17:56.373 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T02:17:56.373 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T02:17:56.373 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T02:17:56.373 INFO:teuthology.orchestra.run.smithi016.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-12T02:17:56.373 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-12T02:17:56.373 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T02:17:56.373 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.373 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.374 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-12T02:17:56.374 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.374 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.374 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.374 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.374 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.374 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.374 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.374 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.374 INFO:teuthology.orchestra.run.smithi016.stdout: "ALERTMANAGER_API_HOST": { 2024-09-12T02:17:56.374 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-12T02:17:56.374 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.374 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.375 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.375 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:56.375 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.375 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.375 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.375 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.375 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.375 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.375 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.375 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.375 INFO:teuthology.orchestra.run.smithi016.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-12T02:17:56.375 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-12T02:17:56.376 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.376 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.376 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.376 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T02:17:56.376 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.376 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.376 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.376 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.376 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.376 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.376 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.376 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.377 INFO:teuthology.orchestra.run.smithi016.stdout: "AUDIT_API_ENABLED": { 2024-09-12T02:17:56.377 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "AUDIT_API_ENABLED", 2024-09-12T02:17:56.377 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.377 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.377 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.377 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.377 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.377 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.377 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.377 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.377 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.377 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.378 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.378 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.378 INFO:teuthology.orchestra.run.smithi016.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-12T02:17:56.378 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-12T02:17:56.378 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.378 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.378 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.378 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T02:17:56.378 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.378 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.378 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.378 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.378 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.379 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.379 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.379 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.379 INFO:teuthology.orchestra.run.smithi016.stdout: "ENABLE_BROWSABLE_API": { 2024-09-12T02:17:56.379 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-12T02:17:56.379 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.379 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.379 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.379 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T02:17:56.379 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.379 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.379 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.379 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.380 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.380 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.380 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.380 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.380 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-12T02:17:56.380 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-12T02:17:56.380 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.380 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.380 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.380 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T02:17:56.380 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.380 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.380 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.381 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.381 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.381 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.381 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.381 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.381 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-12T02:17:56.381 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-12T02:17:56.381 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.381 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.381 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.381 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T02:17:56.381 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.382 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.382 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.382 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.382 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.382 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.382 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.382 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.382 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-12T02:17:56.382 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-12T02:17:56.382 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.382 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.382 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.382 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T02:17:56.382 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.383 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.383 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.383 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.383 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.383 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.383 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.383 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.383 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-12T02:17:56.383 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-12T02:17:56.383 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.383 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.383 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.383 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T02:17:56.383 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.384 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.384 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.384 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.384 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.384 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.384 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.384 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.384 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-12T02:17:56.384 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-12T02:17:56.384 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.384 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.384 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.384 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T02:17:56.385 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.385 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.385 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.385 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.385 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.385 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.385 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.385 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.385 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-12T02:17:56.385 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-12T02:17:56.385 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.385 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.386 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.386 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T02:17:56.386 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.386 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.386 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.386 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.386 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.386 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.386 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.386 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.386 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-12T02:17:56.386 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-12T02:17:56.386 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.387 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.387 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.387 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T02:17:56.387 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.387 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.387 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.387 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.387 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.387 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.387 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.387 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.387 INFO:teuthology.orchestra.run.smithi016.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-12T02:17:56.387 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-12T02:17:56.387 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.388 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.388 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.388 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:56.388 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.388 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.388 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.388 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.388 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.388 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.388 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.388 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.388 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_API_PASSWORD": { 2024-09-12T02:17:56.388 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-12T02:17:56.389 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.389 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.389 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.389 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "admin", 2024-09-12T02:17:56.389 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.389 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.389 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.389 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.389 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.389 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.389 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.392 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.392 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-12T02:17:56.392 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-12T02:17:56.392 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.392 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.392 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.392 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T02:17:56.392 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.392 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.393 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.393 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.393 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.393 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.393 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.393 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.393 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_API_URL": { 2024-09-12T02:17:56.393 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_API_URL", 2024-09-12T02:17:56.393 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.393 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.393 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.393 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:56.393 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.394 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.394 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.394 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.394 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.394 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.394 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.394 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.394 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_API_USERNAME": { 2024-09-12T02:17:56.394 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-12T02:17:56.394 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.394 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.394 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.394 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "admin", 2024-09-12T02:17:56.395 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.395 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.395 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.395 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.395 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.395 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.395 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.395 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.395 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-12T02:17:56.395 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-12T02:17:56.395 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.395 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.395 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.396 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:56.396 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.396 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.396 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.396 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.396 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.396 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.396 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.396 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.396 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-12T02:17:56.396 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-12T02:17:56.396 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.396 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.397 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.397 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.397 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.397 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.397 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.397 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.397 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.397 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.397 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.397 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.397 INFO:teuthology.orchestra.run.smithi016.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-12T02:17:56.397 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-12T02:17:56.397 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.398 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.398 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.398 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T02:17:56.398 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.398 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.398 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.398 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.398 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.398 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.398 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.398 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.398 INFO:teuthology.orchestra.run.smithi016.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-12T02:17:56.398 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-12T02:17:56.398 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.399 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.399 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.399 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:56.399 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.399 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.399 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.399 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.399 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.399 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.399 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.399 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.399 INFO:teuthology.orchestra.run.smithi016.stdout: "PROMETHEUS_API_HOST": { 2024-09-12T02:17:56.399 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-12T02:17:56.400 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.400 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.400 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.400 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:56.400 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.400 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.400 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.400 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.400 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.400 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.400 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.400 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.400 INFO:teuthology.orchestra.run.smithi016.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-12T02:17:56.401 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-12T02:17:56.401 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.401 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.401 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.401 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T02:17:56.401 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.401 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.401 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.401 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.401 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.401 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.401 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.401 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.402 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-12T02:17:56.402 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-12T02:17:56.402 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.402 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.402 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.402 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.402 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.402 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.402 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.402 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.402 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.402 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.402 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.403 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.403 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-12T02:17:56.403 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-12T02:17:56.403 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.403 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.403 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.403 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.403 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.403 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.403 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.403 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.403 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.403 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.404 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.404 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.404 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-12T02:17:56.404 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-12T02:17:56.404 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.404 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.404 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.404 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T02:17:56.404 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.404 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.404 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.404 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.405 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.405 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.405 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.405 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.405 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-12T02:17:56.405 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-12T02:17:56.405 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.405 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.405 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.405 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T02:17:56.405 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.405 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.406 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.406 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.406 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.406 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.406 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.406 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.406 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-12T02:17:56.406 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-12T02:17:56.406 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.406 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.406 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.406 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.406 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.406 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.407 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.407 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.407 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.407 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.407 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.407 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.407 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-12T02:17:56.407 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-12T02:17:56.407 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.407 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.407 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.407 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.407 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.407 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.408 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.408 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.408 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.408 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.408 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.408 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.408 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-12T02:17:56.408 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-12T02:17:56.408 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.408 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.408 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.408 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.408 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.409 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.409 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.409 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.409 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.409 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.409 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.409 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.409 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_ENABLED": { 2024-09-12T02:17:56.409 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-12T02:17:56.409 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.409 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.409 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.409 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T02:17:56.410 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.410 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.410 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.410 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.410 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.410 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.410 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.410 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.410 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-12T02:17:56.410 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-12T02:17:56.410 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.410 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.410 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.411 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-12T02:17:56.411 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.411 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.411 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.411 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.411 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.411 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.411 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.411 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.411 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-12T02:17:56.411 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-12T02:17:56.411 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T02:17:56.411 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.412 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.412 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-12T02:17:56.412 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.412 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.412 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.412 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.412 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.412 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.412 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.412 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.412 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-12T02:17:56.412 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-12T02:17:56.412 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T02:17:56.413 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.413 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.413 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8", 2024-09-12T02:17:56.413 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.413 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.413 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.413 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.413 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.413 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.413 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.413 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.413 INFO:teuthology.orchestra.run.smithi016.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-12T02:17:56.413 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-12T02:17:56.414 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T02:17:56.414 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.414 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.414 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "45", 2024-09-12T02:17:56.414 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.414 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.414 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.414 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.414 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.414 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.414 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.414 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.414 INFO:teuthology.orchestra.run.smithi016.stdout: "RGW_API_ACCESS_KEY": { 2024-09-12T02:17:56.414 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-12T02:17:56.415 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.415 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.415 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.415 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:56.415 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.415 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.415 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.415 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.415 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.415 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.415 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.415 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.415 INFO:teuthology.orchestra.run.smithi016.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-12T02:17:56.416 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-12T02:17:56.416 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.416 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.416 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.416 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "admin", 2024-09-12T02:17:56.416 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.416 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.416 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.416 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.416 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.416 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.416 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.416 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.417 INFO:teuthology.orchestra.run.smithi016.stdout: "RGW_API_SECRET_KEY": { 2024-09-12T02:17:56.417 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-12T02:17:56.417 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.417 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.417 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.417 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:56.417 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.417 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.417 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.417 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.417 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.417 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.417 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.418 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.418 INFO:teuthology.orchestra.run.smithi016.stdout: "RGW_API_SSL_VERIFY": { 2024-09-12T02:17:56.418 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-12T02:17:56.418 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.418 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.418 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.418 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T02:17:56.418 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.418 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.418 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.418 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.418 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.418 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.419 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.419 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.419 INFO:teuthology.orchestra.run.smithi016.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-12T02:17:56.419 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-12T02:17:56.419 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T02:17:56.419 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.419 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.419 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-12T02:17:56.419 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.419 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.419 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.419 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.419 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.419 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.420 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.420 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.420 INFO:teuthology.orchestra.run.smithi016.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-12T02:17:56.420 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-12T02:17:56.420 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T02:17:56.420 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.420 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.420 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-12T02:17:56.420 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.420 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.420 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.420 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.420 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.421 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.421 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.421 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.421 INFO:teuthology.orchestra.run.smithi016.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-12T02:17:56.421 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-12T02:17:56.421 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T02:17:56.421 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.421 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.421 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5", 2024-09-12T02:17:56.421 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.421 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.421 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.422 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.422 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.422 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.422 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.422 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.422 INFO:teuthology.orchestra.run.smithi016.stdout: "cross_origin_url": { 2024-09-12T02:17:56.422 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cross_origin_url", 2024-09-12T02:17:56.422 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.422 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.422 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.422 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:56.422 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.423 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.423 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.423 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.423 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.423 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.423 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.423 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.423 INFO:teuthology.orchestra.run.smithi016.stdout: "crt_file": { 2024-09-12T02:17:56.423 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crt_file", 2024-09-12T02:17:56.423 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.423 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.423 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.423 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:56.424 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.424 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.424 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.424 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.424 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.424 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.424 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.424 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.424 INFO:teuthology.orchestra.run.smithi016.stdout: "debug": { 2024-09-12T02:17:56.424 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "debug", 2024-09-12T02:17:56.424 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.425 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.425 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.425 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.425 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.425 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.425 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.425 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Enable/disable debug options", 2024-09-12T02:17:56.425 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.425 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.425 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.425 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.425 INFO:teuthology.orchestra.run.smithi016.stdout: "jwt_token_ttl": { 2024-09-12T02:17:56.425 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "jwt_token_ttl", 2024-09-12T02:17:56.426 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T02:17:56.426 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.426 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.426 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "28800", 2024-09-12T02:17:56.426 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.426 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.426 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.426 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.426 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.426 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.426 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.426 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.427 INFO:teuthology.orchestra.run.smithi016.stdout: "key_file": { 2024-09-12T02:17:56.427 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "key_file", 2024-09-12T02:17:56.427 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.427 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.427 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.427 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:56.427 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.427 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.427 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.427 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.427 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.427 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.428 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.428 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.428 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T02:17:56.428 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T02:17:56.428 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.428 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.428 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.428 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:56.428 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.428 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.428 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:56.428 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:56.429 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:56.429 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:56.429 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:56.429 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:56.429 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:56.429 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:56.429 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.429 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.429 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.429 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.429 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.429 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T02:17:56.429 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T02:17:56.430 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.430 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.430 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.430 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.430 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.430 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.430 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.430 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.430 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.430 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.430 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.430 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.430 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T02:17:56.430 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T02:17:56.431 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.431 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.431 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.431 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T02:17:56.431 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.431 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.431 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:56.431 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:56.431 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:56.431 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:56.431 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:56.431 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:56.431 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:56.432 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:56.432 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.432 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.432 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.432 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.432 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.432 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T02:17:56.432 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T02:17:56.432 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.432 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.432 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.432 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.432 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.433 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.433 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.433 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.433 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.433 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.433 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.433 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.433 INFO:teuthology.orchestra.run.smithi016.stdout: "motd": { 2024-09-12T02:17:56.433 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "motd", 2024-09-12T02:17:56.433 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.433 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.433 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.434 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:56.434 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.434 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.434 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.434 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "The message of the day", 2024-09-12T02:17:56.434 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.434 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.434 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.434 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.434 INFO:teuthology.orchestra.run.smithi016.stdout: "redirect_resolve_ip_addr": { 2024-09-12T02:17:56.434 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "redirect_resolve_ip_addr", 2024-09-12T02:17:56.434 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.434 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.435 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.435 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.435 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.435 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.435 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.435 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.435 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.435 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.435 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.435 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.435 INFO:teuthology.orchestra.run.smithi016.stdout: "server_addr": { 2024-09-12T02:17:56.435 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_addr", 2024-09-12T02:17:56.435 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.436 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.436 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.436 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "::", 2024-09-12T02:17:56.436 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.436 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.436 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.436 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.436 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.436 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.436 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.436 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.436 INFO:teuthology.orchestra.run.smithi016.stdout: "server_port": { 2024-09-12T02:17:56.436 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_port", 2024-09-12T02:17:56.437 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T02:17:56.437 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.437 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.437 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8080", 2024-09-12T02:17:56.437 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.437 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.437 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.437 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.437 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.437 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.437 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.437 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.437 INFO:teuthology.orchestra.run.smithi016.stdout: "ssl": { 2024-09-12T02:17:56.438 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ssl", 2024-09-12T02:17:56.438 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.438 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.438 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.438 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T02:17:56.438 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.438 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.438 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.438 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.438 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.438 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.438 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.438 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.438 INFO:teuthology.orchestra.run.smithi016.stdout: "ssl_server_port": { 2024-09-12T02:17:56.439 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ssl_server_port", 2024-09-12T02:17:56.439 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T02:17:56.439 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.439 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.439 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8443", 2024-09-12T02:17:56.439 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.439 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.439 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.439 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.439 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.439 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.439 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.439 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.439 INFO:teuthology.orchestra.run.smithi016.stdout: "standby_behaviour": { 2024-09-12T02:17:56.440 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "standby_behaviour", 2024-09-12T02:17:56.440 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.440 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.440 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.440 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "redirect", 2024-09-12T02:17:56.440 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.440 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.440 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:56.440 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:56.440 INFO:teuthology.orchestra.run.smithi016.stdout: "redirect" 2024-09-12T02:17:56.440 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:56.440 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.440 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.441 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.441 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.441 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.441 INFO:teuthology.orchestra.run.smithi016.stdout: "standby_error_status_code": { 2024-09-12T02:17:56.441 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "standby_error_status_code", 2024-09-12T02:17:56.441 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T02:17:56.441 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.441 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.441 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "500", 2024-09-12T02:17:56.441 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "400", 2024-09-12T02:17:56.441 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "599", 2024-09-12T02:17:56.441 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.441 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.442 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.442 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.442 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.442 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.442 INFO:teuthology.orchestra.run.smithi016.stdout: "url_prefix": { 2024-09-12T02:17:56.442 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "url_prefix", 2024-09-12T02:17:56.442 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.442 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.442 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.442 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:56.442 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.442 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.442 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.442 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.443 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.443 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.443 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.443 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:56.443 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:56.443 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.443 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T02:17:56.443 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "devicehealth", 2024-09-12T02:17:56.443 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T02:17:56.443 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T02:17:56.443 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T02:17:56.443 INFO:teuthology.orchestra.run.smithi016.stdout: "enable_monitoring": { 2024-09-12T02:17:56.443 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "enable_monitoring", 2024-09-12T02:17:56.443 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.444 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.444 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.444 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T02:17:56.444 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.444 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.444 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.444 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "monitor device health metrics", 2024-09-12T02:17:56.444 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.444 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.444 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.444 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.444 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T02:17:56.444 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T02:17:56.445 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.445 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.445 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.445 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:56.445 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.445 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.445 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:56.445 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:56.445 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:56.445 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:56.445 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:56.445 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:56.445 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:56.446 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:56.446 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.446 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.446 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.446 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.446 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.446 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T02:17:56.446 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T02:17:56.446 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.446 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.446 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.446 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.446 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.447 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.447 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.447 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.447 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.447 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.447 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.447 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.447 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T02:17:56.447 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T02:17:56.447 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.447 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.447 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.447 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T02:17:56.448 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.448 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.448 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:56.448 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:56.448 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:56.448 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:56.448 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:56.448 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:56.448 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:56.448 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:56.448 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.448 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.448 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.448 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.449 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.449 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T02:17:56.449 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T02:17:56.449 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.449 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.449 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.449 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.449 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.449 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.449 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.449 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.449 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.449 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.450 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.450 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.450 INFO:teuthology.orchestra.run.smithi016.stdout: "mark_out_threshold": { 2024-09-12T02:17:56.450 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mark_out_threshold", 2024-09-12T02:17:56.450 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-12T02:17:56.450 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.450 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.450 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "2419200", 2024-09-12T02:17:56.450 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.450 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.450 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.450 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-12T02:17:56.450 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.451 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.451 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.451 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.451 INFO:teuthology.orchestra.run.smithi016.stdout: "pool_name": { 2024-09-12T02:17:56.451 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "pool_name", 2024-09-12T02:17:56.451 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.451 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.451 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.451 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "device_health_metrics", 2024-09-12T02:17:56.451 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.451 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.451 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.451 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-12T02:17:56.451 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.452 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.452 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.452 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.452 INFO:teuthology.orchestra.run.smithi016.stdout: "retention_period": { 2024-09-12T02:17:56.452 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "retention_period", 2024-09-12T02:17:56.452 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-12T02:17:56.452 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.452 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.452 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "15552000", 2024-09-12T02:17:56.452 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.452 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.452 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.453 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how long to retain device health metrics", 2024-09-12T02:17:56.453 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.453 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.453 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.453 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.453 INFO:teuthology.orchestra.run.smithi016.stdout: "scrape_frequency": { 2024-09-12T02:17:56.453 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "scrape_frequency", 2024-09-12T02:17:56.453 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-12T02:17:56.453 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.453 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.453 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "86400", 2024-09-12T02:17:56.453 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.453 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.454 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.454 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-12T02:17:56.454 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.454 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.454 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.454 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.454 INFO:teuthology.orchestra.run.smithi016.stdout: "self_heal": { 2024-09-12T02:17:56.454 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "self_heal", 2024-09-12T02:17:56.454 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.454 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.454 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.454 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T02:17:56.455 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.455 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.455 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.455 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-12T02:17:56.455 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.455 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.455 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.455 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.455 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-09-12T02:17:56.455 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-09-12T02:17:56.455 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-12T02:17:56.456 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.456 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.456 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-09-12T02:17:56.456 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.456 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.456 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.456 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to wake up and check device health", 2024-09-12T02:17:56.456 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.456 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.456 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.456 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.456 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_threshold": { 2024-09-12T02:17:56.456 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_threshold", 2024-09-12T02:17:56.457 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-12T02:17:56.457 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.457 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.457 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "7257600", 2024-09-12T02:17:56.457 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.457 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.457 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.457 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-12T02:17:56.457 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.457 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.457 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.457 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:56.457 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:56.458 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.458 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T02:17:56.458 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "diskprediction_local", 2024-09-12T02:17:56.458 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T02:17:56.458 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T02:17:56.458 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T02:17:56.458 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T02:17:56.458 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T02:17:56.458 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.458 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.458 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.458 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:56.458 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.459 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.459 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:56.459 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:56.459 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:56.459 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:56.459 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:56.459 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:56.459 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:56.459 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:56.459 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.459 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.460 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.460 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.460 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.460 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T02:17:56.460 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T02:17:56.460 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.460 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.460 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.460 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.460 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.460 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.460 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.460 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.461 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.461 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.461 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.461 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.461 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T02:17:56.461 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T02:17:56.461 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.461 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.461 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.461 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T02:17:56.461 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.462 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.462 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:56.462 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:56.462 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:56.462 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:56.462 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:56.462 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:56.462 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:56.462 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:56.462 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.462 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.462 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.462 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.463 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.463 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T02:17:56.463 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T02:17:56.463 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.463 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.463 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.463 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.463 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.463 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.463 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.463 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.463 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.464 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.464 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.464 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.464 INFO:teuthology.orchestra.run.smithi016.stdout: "predict_interval": { 2024-09-12T02:17:56.464 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "predict_interval", 2024-09-12T02:17:56.464 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.464 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.464 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.464 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "86400", 2024-09-12T02:17:56.465 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.465 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.465 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.465 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.465 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.465 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.465 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.465 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.465 INFO:teuthology.orchestra.run.smithi016.stdout: "predictor_model": { 2024-09-12T02:17:56.465 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "predictor_model", 2024-09-12T02:17:56.465 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.465 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.465 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.466 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "prophetstor", 2024-09-12T02:17:56.466 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.466 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.466 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.466 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.466 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.466 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.466 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.466 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.466 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-09-12T02:17:56.466 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-09-12T02:17:56.466 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.466 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.467 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.467 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-09-12T02:17:56.467 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.467 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.467 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.467 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.467 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.467 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.467 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.467 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:56.467 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:56.467 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.467 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T02:17:56.467 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "influx", 2024-09-12T02:17:56.468 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": false, 2024-09-12T02:17:56.468 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "influxdb python module not found", 2024-09-12T02:17:56.468 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T02:17:56.468 INFO:teuthology.orchestra.run.smithi016.stdout: "batch_size": { 2024-09-12T02:17:56.468 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "batch_size", 2024-09-12T02:17:56.468 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T02:17:56.468 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.468 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.468 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5000", 2024-09-12T02:17:56.468 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.468 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.468 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.468 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-12T02:17:56.469 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.469 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.469 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.469 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.469 INFO:teuthology.orchestra.run.smithi016.stdout: "database": { 2024-09-12T02:17:56.469 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "database", 2024-09-12T02:17:56.469 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.469 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.469 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.469 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "ceph", 2024-09-12T02:17:56.469 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.469 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.470 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.470 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-12T02:17:56.470 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.470 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.470 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.470 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.470 INFO:teuthology.orchestra.run.smithi016.stdout: "hostname": { 2024-09-12T02:17:56.470 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "hostname", 2024-09-12T02:17:56.470 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.470 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.470 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.470 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:56.470 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.470 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.471 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.471 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "InfluxDB server hostname", 2024-09-12T02:17:56.471 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.471 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.471 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.471 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.471 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-09-12T02:17:56.471 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-09-12T02:17:56.471 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-12T02:17:56.471 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.471 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.471 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "30", 2024-09-12T02:17:56.471 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "5", 2024-09-12T02:17:56.472 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.472 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.472 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-12T02:17:56.472 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.472 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.472 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.472 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.472 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T02:17:56.472 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T02:17:56.472 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.472 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.472 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.473 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:56.473 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.473 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.473 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:56.473 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:56.473 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:56.473 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:56.473 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:56.473 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:56.473 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:56.473 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:56.473 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.473 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.474 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.474 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.474 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.474 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T02:17:56.474 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T02:17:56.474 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.474 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.474 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.474 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.474 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.474 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.474 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.474 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.474 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.475 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.475 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.475 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.475 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T02:17:56.475 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T02:17:56.475 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.475 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.475 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.475 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T02:17:56.475 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.475 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.475 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:56.475 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:56.476 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:56.476 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:56.476 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:56.476 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:56.476 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:56.476 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:56.476 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.476 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.476 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.476 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.476 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.476 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T02:17:56.476 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T02:17:56.477 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.477 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.477 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.477 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.477 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.477 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.477 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.477 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.477 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.477 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.477 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.477 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.477 INFO:teuthology.orchestra.run.smithi016.stdout: "password": { 2024-09-12T02:17:56.478 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "password", 2024-09-12T02:17:56.478 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.478 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.478 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.478 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:56.478 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.478 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.478 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.478 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "password of InfluxDB server user", 2024-09-12T02:17:56.478 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.478 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.478 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.478 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.478 INFO:teuthology.orchestra.run.smithi016.stdout: "port": { 2024-09-12T02:17:56.479 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "port", 2024-09-12T02:17:56.479 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T02:17:56.479 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.479 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.479 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8086", 2024-09-12T02:17:56.479 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.479 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.479 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.479 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "InfluxDB server port", 2024-09-12T02:17:56.479 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.479 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.479 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.479 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.480 INFO:teuthology.orchestra.run.smithi016.stdout: "ssl": { 2024-09-12T02:17:56.480 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ssl", 2024-09-12T02:17:56.480 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.480 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.480 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.480 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "false", 2024-09-12T02:17:56.480 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.480 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.480 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.480 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-12T02:17:56.480 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.480 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.480 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.481 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.481 INFO:teuthology.orchestra.run.smithi016.stdout: "threads": { 2024-09-12T02:17:56.481 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "threads", 2024-09-12T02:17:56.481 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T02:17:56.481 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.481 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.481 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5", 2024-09-12T02:17:56.481 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1", 2024-09-12T02:17:56.481 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "32", 2024-09-12T02:17:56.481 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.481 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-12T02:17:56.481 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.481 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.482 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.482 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.482 INFO:teuthology.orchestra.run.smithi016.stdout: "username": { 2024-09-12T02:17:56.482 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "username", 2024-09-12T02:17:56.482 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.482 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.482 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.482 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:56.482 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.482 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.482 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.482 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "username of InfluxDB server user", 2024-09-12T02:17:56.482 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.483 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.483 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.483 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.483 INFO:teuthology.orchestra.run.smithi016.stdout: "verify_ssl": { 2024-09-12T02:17:56.483 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "verify_ssl", 2024-09-12T02:17:56.483 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.483 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.483 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.483 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "true", 2024-09-12T02:17:56.483 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.483 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.483 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.484 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-12T02:17:56.484 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.484 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.484 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.484 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:56.484 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:56.484 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.484 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T02:17:56.484 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "insights", 2024-09-12T02:17:56.484 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T02:17:56.484 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T02:17:56.484 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T02:17:56.484 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T02:17:56.485 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T02:17:56.485 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.485 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.485 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.485 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:56.485 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.485 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.485 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:56.485 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:56.485 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:56.485 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:56.485 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:56.485 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:56.486 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:56.486 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:56.486 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.486 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.486 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.486 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.486 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.486 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T02:17:56.486 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T02:17:56.486 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.486 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.486 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.486 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.487 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.487 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.487 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.487 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.487 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.487 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.487 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.487 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.487 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T02:17:56.487 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T02:17:56.487 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.487 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.487 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.488 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T02:17:56.488 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.488 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.488 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:56.488 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:56.488 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:56.488 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:56.488 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:56.488 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:56.488 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:56.488 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:56.488 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.488 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.489 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.489 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.489 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.489 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T02:17:56.489 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T02:17:56.489 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.489 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.489 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.489 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.489 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.489 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.489 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.489 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.490 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.490 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.490 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.490 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:56.490 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:56.490 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.490 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T02:17:56.490 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "iostat", 2024-09-12T02:17:56.490 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T02:17:56.490 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T02:17:56.490 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T02:17:56.490 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T02:17:56.490 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T02:17:56.490 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.491 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.491 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.491 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:56.491 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.491 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.491 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:56.491 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:56.491 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:56.491 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:56.491 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:56.491 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:56.491 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:56.491 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:56.492 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.492 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.492 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.492 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.492 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.492 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T02:17:56.492 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T02:17:56.492 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.492 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.492 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.492 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.492 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.492 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.493 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.493 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.493 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.493 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.493 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.493 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.493 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T02:17:56.493 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T02:17:56.493 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.493 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.493 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.493 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T02:17:56.493 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.494 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.494 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:56.494 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:56.494 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:56.494 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:56.494 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:56.494 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:56.494 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:56.494 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:56.494 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.494 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.494 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.494 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.494 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.495 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T02:17:56.495 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T02:17:56.495 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.495 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.495 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.495 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.495 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.495 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.495 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.495 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.495 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.495 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.495 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.495 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:56.496 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:56.496 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.496 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T02:17:56.496 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "k8sevents", 2024-09-12T02:17:56.496 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T02:17:56.496 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T02:17:56.496 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T02:17:56.496 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_event_retention_days": { 2024-09-12T02:17:56.496 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ceph_event_retention_days", 2024-09-12T02:17:56.496 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T02:17:56.496 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.496 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.496 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "7", 2024-09-12T02:17:56.497 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.497 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.497 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.497 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-12T02:17:56.497 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.497 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.497 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.497 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.497 INFO:teuthology.orchestra.run.smithi016.stdout: "config_check_secs": { 2024-09-12T02:17:56.497 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "config_check_secs", 2024-09-12T02:17:56.497 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T02:17:56.497 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.497 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.497 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-12T02:17:56.498 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "10", 2024-09-12T02:17:56.498 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.498 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.498 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-12T02:17:56.498 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.498 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.498 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.498 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.498 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T02:17:56.498 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T02:17:56.498 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.498 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.498 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.499 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:56.499 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.499 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.499 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:56.499 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:56.499 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:56.499 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:56.499 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:56.499 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:56.499 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:56.499 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:56.499 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.499 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.499 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.500 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.500 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.500 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T02:17:56.500 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T02:17:56.500 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.500 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.500 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.500 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.500 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.500 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.500 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.500 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.501 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.501 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.501 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.501 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.501 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T02:17:56.501 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T02:17:56.501 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.501 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.501 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.501 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T02:17:56.501 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.501 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.501 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:56.501 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:56.502 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:56.502 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:56.502 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:56.502 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:56.502 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:56.502 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:56.502 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.502 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.502 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.502 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.502 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.502 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T02:17:56.502 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T02:17:56.503 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.503 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.503 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.503 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.503 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.503 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.503 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.503 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.503 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.503 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.503 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.503 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:56.503 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:56.504 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.504 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T02:17:56.504 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "localpool", 2024-09-12T02:17:56.504 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T02:17:56.504 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T02:17:56.504 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T02:17:56.504 INFO:teuthology.orchestra.run.smithi016.stdout: "failure_domain": { 2024-09-12T02:17:56.504 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "failure_domain", 2024-09-12T02:17:56.504 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.504 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.504 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.504 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "host", 2024-09-12T02:17:56.504 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.505 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.505 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.505 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "failure domain for any created local pool", 2024-09-12T02:17:56.505 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-12T02:17:56.505 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.505 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.505 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.505 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T02:17:56.505 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T02:17:56.505 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.505 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.505 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.505 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:56.506 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.506 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.506 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:56.506 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:56.506 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:56.506 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:56.506 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:56.506 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:56.506 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:56.506 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:56.506 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.506 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.506 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.506 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.507 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.507 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T02:17:56.507 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T02:17:56.507 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.507 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.507 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.507 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.507 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.507 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.507 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.507 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.507 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.507 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.508 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.508 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.508 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T02:17:56.508 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T02:17:56.508 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.508 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.508 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.508 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T02:17:56.508 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.508 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.508 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:56.508 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:56.508 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:56.509 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:56.509 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:56.509 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:56.509 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:56.509 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:56.509 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.509 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.509 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.509 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.509 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.509 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T02:17:56.509 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T02:17:56.509 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.509 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.510 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.510 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.510 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.510 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.510 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.510 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.510 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.510 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.510 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.510 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.510 INFO:teuthology.orchestra.run.smithi016.stdout: "min_size": { 2024-09-12T02:17:56.510 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "min_size", 2024-09-12T02:17:56.510 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T02:17:56.511 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.511 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.511 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:56.511 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.511 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.511 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.511 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "default min_size for any created local pool", 2024-09-12T02:17:56.511 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-12T02:17:56.511 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.511 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.511 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.511 INFO:teuthology.orchestra.run.smithi016.stdout: "num_rep": { 2024-09-12T02:17:56.511 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "num_rep", 2024-09-12T02:17:56.511 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T02:17:56.512 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.512 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.512 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "3", 2024-09-12T02:17:56.512 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.512 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.512 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.512 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "default replica count for any created local pool", 2024-09-12T02:17:56.512 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.512 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.512 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.512 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.512 INFO:teuthology.orchestra.run.smithi016.stdout: "pg_num": { 2024-09-12T02:17:56.512 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "pg_num", 2024-09-12T02:17:56.512 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T02:17:56.513 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.513 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.513 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "128", 2024-09-12T02:17:56.513 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.513 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.513 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.513 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "default pg_num for any created local pool", 2024-09-12T02:17:56.513 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.513 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.513 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.513 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.513 INFO:teuthology.orchestra.run.smithi016.stdout: "prefix": { 2024-09-12T02:17:56.513 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "prefix", 2024-09-12T02:17:56.514 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.514 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.514 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.514 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:56.514 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.514 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.514 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.514 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "name prefix for any created local pool", 2024-09-12T02:17:56.514 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.514 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.514 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.514 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.514 INFO:teuthology.orchestra.run.smithi016.stdout: "subtree": { 2024-09-12T02:17:56.514 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "subtree", 2024-09-12T02:17:56.515 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.515 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.515 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.515 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "rack", 2024-09-12T02:17:56.515 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.515 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.515 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.515 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-12T02:17:56.515 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-12T02:17:56.515 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.515 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.515 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:56.515 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:56.515 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.516 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T02:17:56.516 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mds_autoscaler", 2024-09-12T02:17:56.516 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T02:17:56.516 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T02:17:56.516 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T02:17:56.516 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T02:17:56.516 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T02:17:56.516 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.516 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.516 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.516 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:56.516 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.516 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.517 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:56.517 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:56.517 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:56.517 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:56.517 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:56.517 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:56.517 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:56.517 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:56.517 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.517 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.517 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.517 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.517 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.518 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T02:17:56.518 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T02:17:56.518 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.518 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.518 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.518 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.518 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.518 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.518 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.518 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.518 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.518 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.518 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.519 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.519 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T02:17:56.519 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T02:17:56.519 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.519 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.519 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.519 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T02:17:56.519 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.519 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.519 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:56.519 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:56.519 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:56.519 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:56.520 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:56.520 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:56.520 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:56.520 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:56.520 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.520 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.520 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.520 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.520 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.520 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T02:17:56.520 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T02:17:56.520 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.520 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.521 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.521 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.521 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.521 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.521 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.521 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.521 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.521 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.521 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.521 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:56.521 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:56.521 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.521 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T02:17:56.521 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mirroring", 2024-09-12T02:17:56.522 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T02:17:56.522 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T02:17:56.522 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T02:17:56.522 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T02:17:56.522 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T02:17:56.522 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.522 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.522 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.522 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:56.522 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.522 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.522 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:56.522 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:56.522 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:56.523 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:56.523 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:56.523 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:56.523 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:56.523 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:56.523 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.523 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.523 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.523 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.523 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.523 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T02:17:56.523 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T02:17:56.523 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.524 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.524 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.524 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.524 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.524 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.524 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.524 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.524 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.524 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.524 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.524 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.524 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T02:17:56.524 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T02:17:56.525 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.525 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.525 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.525 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T02:17:56.525 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.525 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.525 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:56.525 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:56.525 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:56.525 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:56.525 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:56.525 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:56.525 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:56.526 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:56.526 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.526 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.526 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.526 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.526 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.526 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T02:17:56.526 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T02:17:56.526 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.526 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.526 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.526 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.526 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.526 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.527 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.527 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.527 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.527 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.527 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.527 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:56.527 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:56.527 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.527 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T02:17:56.527 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "nfs", 2024-09-12T02:17:56.527 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T02:17:56.527 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T02:17:56.527 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T02:17:56.528 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T02:17:56.528 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T02:17:56.528 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.528 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.528 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.528 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:56.528 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.528 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.528 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:56.528 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:56.528 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:56.528 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:56.528 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:56.529 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:56.529 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:56.529 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:56.529 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.529 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.529 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.529 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.529 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.529 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T02:17:56.529 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T02:17:56.529 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.529 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.529 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.529 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.530 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.530 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.530 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.530 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.530 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.530 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.530 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.530 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.530 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T02:17:56.530 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T02:17:56.530 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.530 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.530 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.531 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T02:17:56.531 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.531 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.531 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:56.531 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:56.531 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:56.531 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:56.531 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:56.531 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:56.531 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:56.531 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:56.531 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.531 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.532 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.532 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.532 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.532 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T02:17:56.532 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T02:17:56.532 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.532 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.532 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.532 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.532 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.532 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.532 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.532 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.533 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.533 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.533 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.533 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:56.533 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:56.533 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.533 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T02:17:56.533 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "orchestrator", 2024-09-12T02:17:56.533 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T02:17:56.533 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T02:17:56.533 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T02:17:56.533 INFO:teuthology.orchestra.run.smithi016.stdout: "fail_fs": { 2024-09-12T02:17:56.534 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "fail_fs", 2024-09-12T02:17:56.534 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.534 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.534 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.534 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.534 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.534 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.534 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.534 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-12T02:17:56.534 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.534 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.534 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.534 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.535 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T02:17:56.535 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T02:17:56.535 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.535 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.535 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.535 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:56.535 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.535 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.535 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:56.535 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:56.535 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:56.535 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:56.535 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:56.536 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:56.536 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:56.536 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:56.536 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.536 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.536 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.536 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.536 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.536 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T02:17:56.536 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T02:17:56.536 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.537 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.537 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.537 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.537 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.537 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.537 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.537 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.537 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.537 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.537 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.537 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.537 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T02:17:56.538 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T02:17:56.538 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.538 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.538 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.538 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T02:17:56.538 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.538 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.538 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:56.538 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:56.538 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:56.538 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:56.538 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:56.538 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:56.539 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:56.539 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:56.539 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.539 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.539 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.539 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.539 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.539 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T02:17:56.539 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T02:17:56.539 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.539 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.539 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.539 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.539 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.540 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.540 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.540 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.540 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.540 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.540 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.540 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.540 INFO:teuthology.orchestra.run.smithi016.stdout: "orchestrator": { 2024-09-12T02:17:56.540 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "orchestrator", 2024-09-12T02:17:56.540 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.540 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.540 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.541 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:56.541 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.541 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.541 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:56.541 INFO:teuthology.orchestra.run.smithi016.stdout: "cephadm", 2024-09-12T02:17:56.541 INFO:teuthology.orchestra.run.smithi016.stdout: "rook", 2024-09-12T02:17:56.541 INFO:teuthology.orchestra.run.smithi016.stdout: "test_orchestrator" 2024-09-12T02:17:56.541 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:56.541 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Orchestrator backend", 2024-09-12T02:17:56.541 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.541 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.541 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.541 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:56.541 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:56.542 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.542 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T02:17:56.542 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "osd_perf_query", 2024-09-12T02:17:56.542 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T02:17:56.542 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T02:17:56.542 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T02:17:56.542 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T02:17:56.542 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T02:17:56.542 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.542 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.542 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.542 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:56.542 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.542 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.543 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:56.543 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:56.543 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:56.543 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:56.543 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:56.543 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:56.543 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:56.543 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:56.543 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.543 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.543 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.543 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.543 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.543 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T02:17:56.543 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T02:17:56.544 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.544 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.544 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.544 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.544 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.544 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.544 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.544 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.544 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.544 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.544 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.544 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.544 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T02:17:56.544 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T02:17:56.545 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.545 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.545 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.545 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T02:17:56.545 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.545 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.545 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:56.545 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:56.545 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:56.545 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:56.545 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:56.545 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:56.545 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:56.546 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:56.546 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.546 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.546 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.546 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.546 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.546 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T02:17:56.546 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T02:17:56.546 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.546 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.546 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.547 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.547 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.547 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.547 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.547 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.547 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.547 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.547 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.547 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:56.547 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:56.547 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.547 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T02:17:56.547 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "osd_support", 2024-09-12T02:17:56.548 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T02:17:56.548 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T02:17:56.548 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T02:17:56.548 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T02:17:56.548 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T02:17:56.548 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.548 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.548 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.548 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:56.548 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.548 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.548 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:56.548 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:56.548 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:56.549 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:56.549 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:56.549 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:56.549 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:56.549 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:56.549 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.549 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.549 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.549 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.549 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.549 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T02:17:56.549 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T02:17:56.550 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.550 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.550 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.550 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.550 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.550 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.550 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.550 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.550 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.550 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.550 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.550 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.550 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T02:17:56.550 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T02:17:56.551 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.551 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.551 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.551 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T02:17:56.551 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.551 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.551 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:56.551 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:56.551 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:56.551 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:56.551 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:56.551 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:56.551 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:56.551 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:56.552 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.552 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.552 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.552 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.552 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.552 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T02:17:56.552 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T02:17:56.552 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.552 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.552 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.552 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.552 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.552 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.553 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.553 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.553 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.553 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.553 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.553 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:56.553 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:56.553 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.553 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T02:17:56.553 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "pg_autoscaler", 2024-09-12T02:17:56.553 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T02:17:56.553 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T02:17:56.553 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T02:17:56.554 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T02:17:56.554 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T02:17:56.554 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.554 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.554 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.554 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:56.554 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.554 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.554 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:56.554 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:56.554 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:56.554 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:56.555 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:56.555 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:56.555 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:56.555 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:56.555 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.555 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.555 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.555 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.555 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.555 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T02:17:56.555 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T02:17:56.555 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.555 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.556 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.556 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.556 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.556 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.556 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.556 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.556 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.556 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.556 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.556 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.556 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T02:17:56.556 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T02:17:56.556 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.556 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.557 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.557 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T02:17:56.557 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.557 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.557 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:56.557 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:56.557 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:56.557 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:56.557 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:56.557 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:56.557 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:56.557 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:56.557 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.558 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.558 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.558 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.558 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.558 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T02:17:56.558 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T02:17:56.558 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.558 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.558 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.558 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.558 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.558 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.558 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.558 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.559 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.559 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.559 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.559 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.559 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-09-12T02:17:56.559 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-09-12T02:17:56.559 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-12T02:17:56.559 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.559 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.559 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-09-12T02:17:56.559 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.559 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.559 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.560 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.560 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.560 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.560 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.560 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.560 INFO:teuthology.orchestra.run.smithi016.stdout: "threshold": { 2024-09-12T02:17:56.560 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "threshold", 2024-09-12T02:17:56.560 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-09-12T02:17:56.560 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.560 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.560 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "3.0", 2024-09-12T02:17:56.560 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1.0", 2024-09-12T02:17:56.560 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.561 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.561 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "scaling threshold", 2024-09-12T02:17:56.561 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-12T02:17:56.561 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.561 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.561 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:56.561 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:56.561 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.561 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T02:17:56.561 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "progress", 2024-09-12T02:17:56.561 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T02:17:56.561 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T02:17:56.561 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T02:17:56.561 INFO:teuthology.orchestra.run.smithi016.stdout: "allow_pg_recovery_event": { 2024-09-12T02:17:56.562 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "allow_pg_recovery_event", 2024-09-12T02:17:56.562 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.562 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.562 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.562 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.562 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.562 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.562 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.562 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-12T02:17:56.562 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.562 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.562 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.562 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.562 INFO:teuthology.orchestra.run.smithi016.stdout: "enabled": { 2024-09-12T02:17:56.563 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "enabled", 2024-09-12T02:17:56.563 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.563 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.563 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.563 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T02:17:56.563 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.563 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.563 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.563 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.563 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.563 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.563 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.563 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.563 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T02:17:56.564 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T02:17:56.564 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.564 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.564 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.564 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:56.564 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.564 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.564 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:56.564 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:56.564 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:56.564 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:56.564 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:56.564 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:56.565 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:56.565 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:56.565 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.565 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.565 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.565 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.565 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.565 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T02:17:56.565 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T02:17:56.565 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.565 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.565 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.565 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.565 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.566 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.566 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.566 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.566 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.566 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.566 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.566 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.566 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T02:17:56.566 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T02:17:56.566 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.566 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.566 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.566 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T02:17:56.567 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.567 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.567 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:56.567 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:56.567 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:56.567 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:56.567 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:56.567 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:56.567 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:56.567 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:56.567 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.567 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.567 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.568 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.568 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.568 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T02:17:56.568 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T02:17:56.568 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.568 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.568 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.568 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.568 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.568 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.568 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.568 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.568 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.569 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.569 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.569 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.569 INFO:teuthology.orchestra.run.smithi016.stdout: "max_completed_events": { 2024-09-12T02:17:56.569 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_completed_events", 2024-09-12T02:17:56.569 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T02:17:56.569 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.569 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.569 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "50", 2024-09-12T02:17:56.569 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.569 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.569 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.569 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "number of past completed events to remember", 2024-09-12T02:17:56.570 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.570 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.570 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.570 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.570 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-09-12T02:17:56.570 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-09-12T02:17:56.570 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-12T02:17:56.570 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.570 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.570 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5", 2024-09-12T02:17:56.570 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.570 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.570 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.571 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how long the module is going to sleep", 2024-09-12T02:17:56.571 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.571 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.571 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.571 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:56.571 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:56.571 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.571 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T02:17:56.571 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "prometheus", 2024-09-12T02:17:56.571 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T02:17:56.571 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T02:17:56.571 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T02:17:56.571 INFO:teuthology.orchestra.run.smithi016.stdout: "cache": { 2024-09-12T02:17:56.572 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cache", 2024-09-12T02:17:56.572 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.572 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.572 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.572 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T02:17:56.572 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.572 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.572 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.572 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.572 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.572 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.572 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.573 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.573 INFO:teuthology.orchestra.run.smithi016.stdout: "exclude_perf_counters": { 2024-09-12T02:17:56.573 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "exclude_perf_counters", 2024-09-12T02:17:56.576 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.576 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.576 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.577 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T02:17:56.577 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.577 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.577 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.577 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-12T02:17:56.577 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-12T02:17:56.577 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.577 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.577 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.577 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T02:17:56.577 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T02:17:56.577 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.578 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.578 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.578 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:56.578 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.578 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.578 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:56.578 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:56.578 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:56.578 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:56.578 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:56.578 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:56.578 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:56.578 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:56.578 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.579 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.579 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.579 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.579 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.579 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T02:17:56.579 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T02:17:56.579 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.579 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.579 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.579 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.579 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.579 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.579 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.580 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.580 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.580 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.580 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.580 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.580 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T02:17:56.580 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T02:17:56.580 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.580 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.580 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.580 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T02:17:56.580 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.580 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.581 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:56.581 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:56.581 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:56.581 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:56.581 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:56.581 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:56.581 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:56.581 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:56.581 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.581 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.581 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.581 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.581 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.582 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T02:17:56.582 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T02:17:56.582 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.582 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.582 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.582 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.582 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.582 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.582 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.582 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.582 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.582 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.582 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.582 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.583 INFO:teuthology.orchestra.run.smithi016.stdout: "rbd_stats_pools": { 2024-09-12T02:17:56.583 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rbd_stats_pools", 2024-09-12T02:17:56.583 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.583 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.583 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.583 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:56.583 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.583 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.583 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.583 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.583 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.583 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.583 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.584 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.584 INFO:teuthology.orchestra.run.smithi016.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-12T02:17:56.584 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-12T02:17:56.584 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T02:17:56.584 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.584 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.584 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "300", 2024-09-12T02:17:56.584 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.584 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.584 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.584 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.584 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.585 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.585 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.585 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.585 INFO:teuthology.orchestra.run.smithi016.stdout: "scrape_interval": { 2024-09-12T02:17:56.585 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "scrape_interval", 2024-09-12T02:17:56.585 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-09-12T02:17:56.585 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.585 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.585 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "15.0", 2024-09-12T02:17:56.585 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.585 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.585 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.585 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.585 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.586 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.586 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.586 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.586 INFO:teuthology.orchestra.run.smithi016.stdout: "server_addr": { 2024-09-12T02:17:56.586 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_addr", 2024-09-12T02:17:56.586 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.586 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.586 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.586 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "::", 2024-09-12T02:17:56.586 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.586 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.586 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.587 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-12T02:17:56.587 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.587 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.587 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.587 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.587 INFO:teuthology.orchestra.run.smithi016.stdout: "server_port": { 2024-09-12T02:17:56.587 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_port", 2024-09-12T02:17:56.587 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T02:17:56.587 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.587 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.587 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "9283", 2024-09-12T02:17:56.587 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.587 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.587 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.588 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-12T02:17:56.588 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.588 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.588 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.588 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.588 INFO:teuthology.orchestra.run.smithi016.stdout: "stale_cache_strategy": { 2024-09-12T02:17:56.588 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "stale_cache_strategy", 2024-09-12T02:17:56.588 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.588 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.588 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.588 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "log", 2024-09-12T02:17:56.588 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.589 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.589 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.589 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.589 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.589 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.589 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.589 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.589 INFO:teuthology.orchestra.run.smithi016.stdout: "standby_behaviour": { 2024-09-12T02:17:56.589 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "standby_behaviour", 2024-09-12T02:17:56.589 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.589 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.589 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.589 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "default", 2024-09-12T02:17:56.590 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.590 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.590 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:56.590 INFO:teuthology.orchestra.run.smithi016.stdout: "default", 2024-09-12T02:17:56.590 INFO:teuthology.orchestra.run.smithi016.stdout: "error" 2024-09-12T02:17:56.590 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:56.590 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.590 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.590 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.590 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.590 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.590 INFO:teuthology.orchestra.run.smithi016.stdout: "standby_error_status_code": { 2024-09-12T02:17:56.591 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "standby_error_status_code", 2024-09-12T02:17:56.591 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T02:17:56.591 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.591 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.591 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "500", 2024-09-12T02:17:56.591 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "400", 2024-09-12T02:17:56.591 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "599", 2024-09-12T02:17:56.591 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.591 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.591 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.591 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.591 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.591 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:56.592 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:56.592 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.592 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T02:17:56.592 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rbd_support", 2024-09-12T02:17:56.592 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T02:17:56.592 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T02:17:56.592 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T02:17:56.592 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T02:17:56.592 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T02:17:56.592 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.592 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.592 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.592 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:56.592 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.593 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.593 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:56.593 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:56.593 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:56.593 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:56.593 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:56.593 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:56.593 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:56.593 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:56.593 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.593 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.593 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.593 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.594 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.594 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T02:17:56.594 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T02:17:56.594 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.594 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.594 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.594 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.594 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.594 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.594 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.594 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.594 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.594 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.595 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.595 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.595 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T02:17:56.595 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T02:17:56.595 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.595 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.595 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.595 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T02:17:56.595 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.595 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.595 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:56.595 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:56.595 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:56.596 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:56.596 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:56.596 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:56.596 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:56.596 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:56.596 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.596 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.596 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.596 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.596 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.596 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T02:17:56.596 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T02:17:56.597 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.597 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.597 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.597 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.597 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.597 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.597 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.597 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.597 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.597 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.597 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.597 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.597 INFO:teuthology.orchestra.run.smithi016.stdout: "max_concurrent_snap_create": { 2024-09-12T02:17:56.598 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_concurrent_snap_create", 2024-09-12T02:17:56.598 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T02:17:56.598 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.598 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.598 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-12T02:17:56.598 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.598 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.598 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.598 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.598 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.598 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.598 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.598 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.599 INFO:teuthology.orchestra.run.smithi016.stdout: "mirror_snapshot_schedule": { 2024-09-12T02:17:56.599 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mirror_snapshot_schedule", 2024-09-12T02:17:56.599 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.599 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.599 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.599 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:56.599 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.599 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.599 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.599 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.599 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.599 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.600 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.600 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.600 INFO:teuthology.orchestra.run.smithi016.stdout: "trash_purge_schedule": { 2024-09-12T02:17:56.600 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "trash_purge_schedule", 2024-09-12T02:17:56.600 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.600 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.600 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.600 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:56.600 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.600 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.600 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.600 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.600 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.601 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.601 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.601 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:56.601 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:56.601 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.601 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T02:17:56.601 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "restful", 2024-09-12T02:17:56.601 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T02:17:56.601 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T02:17:56.601 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T02:17:56.601 INFO:teuthology.orchestra.run.smithi016.stdout: "enable_auth": { 2024-09-12T02:17:56.601 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "enable_auth", 2024-09-12T02:17:56.601 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.602 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.602 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.602 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T02:17:56.602 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.602 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.602 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.602 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.602 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.602 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.602 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.602 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.602 INFO:teuthology.orchestra.run.smithi016.stdout: "key_file": { 2024-09-12T02:17:56.602 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "key_file", 2024-09-12T02:17:56.603 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.603 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.603 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.603 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:56.603 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.603 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.603 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.603 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.603 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.603 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.603 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.603 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.603 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T02:17:56.603 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T02:17:56.604 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.604 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.604 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.604 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:56.604 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.604 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.604 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:56.604 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:56.604 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:56.604 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:56.604 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:56.604 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:56.604 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:56.605 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:56.605 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.605 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.605 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.605 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.605 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.605 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T02:17:56.605 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T02:17:56.605 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.605 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.605 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.605 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.606 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.606 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.606 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.606 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.606 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.606 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.606 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.606 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.606 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T02:17:56.606 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T02:17:56.606 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.606 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.606 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.607 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T02:17:56.607 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.607 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.607 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:56.607 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:56.607 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:56.607 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:56.607 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:56.607 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:56.607 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:56.607 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:56.607 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.607 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.607 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.608 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.608 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.608 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T02:17:56.608 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T02:17:56.608 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.608 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.608 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.608 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.608 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.608 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.608 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.608 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.609 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.609 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.609 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.609 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.609 INFO:teuthology.orchestra.run.smithi016.stdout: "server_addr": { 2024-09-12T02:17:56.609 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_addr", 2024-09-12T02:17:56.609 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.609 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.609 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.609 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:56.609 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.609 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.609 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.609 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.610 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.610 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.610 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.610 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.610 INFO:teuthology.orchestra.run.smithi016.stdout: "server_port": { 2024-09-12T02:17:56.610 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_port", 2024-09-12T02:17:56.610 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.610 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.610 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.610 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:56.610 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.610 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.610 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.610 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.611 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.611 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.611 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.611 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:56.611 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:56.611 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.611 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T02:17:56.611 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rgw", 2024-09-12T02:17:56.611 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T02:17:56.611 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T02:17:56.611 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T02:17:56.611 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T02:17:56.612 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T02:17:56.612 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.612 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.612 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.612 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:56.612 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.612 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.612 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:56.612 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:56.612 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:56.612 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:56.612 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:56.613 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:56.613 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:56.613 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:56.613 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.613 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.613 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.613 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.613 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.613 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T02:17:56.613 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T02:17:56.613 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.613 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.613 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.614 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.614 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.614 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.614 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.614 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.614 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.614 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.614 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.614 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.614 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T02:17:56.614 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T02:17:56.614 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.614 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.615 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.615 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T02:17:56.615 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.615 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.615 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:56.615 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:56.615 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:56.615 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:56.615 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:56.615 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:56.615 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:56.615 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:56.615 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.616 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.616 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.616 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.616 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.616 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T02:17:56.616 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T02:17:56.616 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.616 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.616 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.616 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.616 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.616 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.616 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.617 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.617 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.617 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.617 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.617 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:56.617 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:56.617 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.617 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T02:17:56.617 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rook", 2024-09-12T02:17:56.617 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T02:17:56.617 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T02:17:56.617 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T02:17:56.617 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T02:17:56.618 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T02:17:56.618 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.618 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.618 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.618 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:56.618 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.618 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.618 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:56.618 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:56.618 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:56.618 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:56.618 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:56.619 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:56.619 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:56.619 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:56.619 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.619 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.619 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.619 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.619 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.619 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T02:17:56.619 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T02:17:56.619 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.619 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.619 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.620 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.620 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.620 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.620 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.620 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.620 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.620 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.620 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.620 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.620 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T02:17:56.620 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T02:17:56.620 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.621 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.621 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.621 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T02:17:56.621 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.621 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.621 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:56.621 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:56.621 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:56.621 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:56.621 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:56.621 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:56.621 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:56.621 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:56.622 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.622 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.622 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.622 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.622 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.622 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T02:17:56.622 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T02:17:56.622 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.622 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.622 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.622 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.622 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.622 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.623 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.623 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.623 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.623 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.623 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.623 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.623 INFO:teuthology.orchestra.run.smithi016.stdout: "storage_class": { 2024-09-12T02:17:56.623 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "storage_class", 2024-09-12T02:17:56.623 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.623 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.623 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.623 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "local", 2024-09-12T02:17:56.623 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.623 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.624 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.624 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-12T02:17:56.624 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.624 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.624 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.624 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:56.624 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:56.624 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.624 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T02:17:56.624 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "selftest", 2024-09-12T02:17:56.624 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T02:17:56.624 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T02:17:56.624 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T02:17:56.624 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T02:17:56.625 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T02:17:56.625 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.625 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.625 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.625 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:56.625 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.625 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.625 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:56.625 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:56.625 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:56.625 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:56.625 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:56.625 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:56.626 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:56.626 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:56.626 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.626 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.626 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.626 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.626 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.626 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T02:17:56.626 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T02:17:56.626 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.626 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.626 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.627 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.627 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.627 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.627 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.627 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.627 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.627 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.627 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.627 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.627 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T02:17:56.627 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T02:17:56.627 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.627 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.627 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.628 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T02:17:56.628 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.628 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.628 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:56.628 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:56.628 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:56.628 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:56.628 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:56.628 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:56.628 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:56.628 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:56.628 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.628 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.629 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.629 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.629 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.629 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T02:17:56.629 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T02:17:56.629 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.629 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.629 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.629 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.629 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.629 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.629 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.629 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.630 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.630 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.630 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.630 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.630 INFO:teuthology.orchestra.run.smithi016.stdout: "roption1": { 2024-09-12T02:17:56.630 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "roption1", 2024-09-12T02:17:56.630 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.630 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.630 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.630 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:56.630 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.630 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.630 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.631 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.631 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.631 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.631 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.631 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.631 INFO:teuthology.orchestra.run.smithi016.stdout: "roption2": { 2024-09-12T02:17:56.631 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "roption2", 2024-09-12T02:17:56.631 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.631 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.631 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.631 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "xyz", 2024-09-12T02:17:56.631 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.632 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.632 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.632 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.632 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.632 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.632 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.632 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.632 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption1": { 2024-09-12T02:17:56.632 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption1", 2024-09-12T02:17:56.632 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.632 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.632 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.632 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:56.633 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.633 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.633 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.633 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.633 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.633 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.633 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.633 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.633 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption2": { 2024-09-12T02:17:56.633 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption2", 2024-09-12T02:17:56.633 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T02:17:56.633 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.633 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.634 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:56.634 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.634 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.634 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.634 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.634 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.634 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.634 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.634 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.634 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption3": { 2024-09-12T02:17:56.634 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption3", 2024-09-12T02:17:56.634 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-09-12T02:17:56.634 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.635 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.635 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:56.635 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.635 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.635 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.635 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.635 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.635 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.635 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.635 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.635 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption4": { 2024-09-12T02:17:56.635 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption4", 2024-09-12T02:17:56.635 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.635 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.636 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.636 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:56.636 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.636 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.636 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.636 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.636 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.636 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.636 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.636 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.636 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption5": { 2024-09-12T02:17:56.636 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption5", 2024-09-12T02:17:56.636 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.637 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.637 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.637 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:56.637 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.637 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.637 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.637 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.637 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.637 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.637 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.637 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.637 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption6": { 2024-09-12T02:17:56.638 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption6", 2024-09-12T02:17:56.638 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.638 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.638 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.638 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T02:17:56.638 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.638 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.638 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.638 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.638 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.638 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.638 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.638 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.638 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption7": { 2024-09-12T02:17:56.639 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption7", 2024-09-12T02:17:56.639 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T02:17:56.639 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.639 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.639 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:56.639 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1", 2024-09-12T02:17:56.639 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "42", 2024-09-12T02:17:56.639 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.639 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.639 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.639 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.639 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.639 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.640 INFO:teuthology.orchestra.run.smithi016.stdout: "testkey": { 2024-09-12T02:17:56.640 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "testkey", 2024-09-12T02:17:56.640 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.640 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.640 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.640 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:56.640 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.640 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.640 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.640 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.640 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.640 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.640 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.641 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.641 INFO:teuthology.orchestra.run.smithi016.stdout: "testlkey": { 2024-09-12T02:17:56.641 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "testlkey", 2024-09-12T02:17:56.641 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.641 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.641 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.641 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:56.641 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.641 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.641 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.641 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.641 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.641 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.642 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.642 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.642 INFO:teuthology.orchestra.run.smithi016.stdout: "testnewline": { 2024-09-12T02:17:56.642 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "testnewline", 2024-09-12T02:17:56.642 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.642 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.642 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.642 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:56.642 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.642 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.642 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.642 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.642 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.643 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.643 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.643 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:56.643 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:56.643 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.643 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T02:17:56.643 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "snap_schedule", 2024-09-12T02:17:56.643 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T02:17:56.643 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T02:17:56.643 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T02:17:56.643 INFO:teuthology.orchestra.run.smithi016.stdout: "allow_m_granularity": { 2024-09-12T02:17:56.643 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "allow_m_granularity", 2024-09-12T02:17:56.643 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.643 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.644 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.644 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.644 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.644 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.644 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.644 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "allow minute scheduled snapshots", 2024-09-12T02:17:56.644 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.644 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.644 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.644 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.644 INFO:teuthology.orchestra.run.smithi016.stdout: "dump_on_update": { 2024-09-12T02:17:56.645 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "dump_on_update", 2024-09-12T02:17:56.645 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.645 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.645 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.645 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.645 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.645 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.645 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.645 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "dump database to debug log on update", 2024-09-12T02:17:56.645 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.645 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.645 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.645 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.645 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T02:17:56.646 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T02:17:56.646 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.646 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.646 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.646 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:56.646 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.646 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.646 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:56.646 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:56.646 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:56.646 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:56.646 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:56.646 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:56.647 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:56.647 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:56.647 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.647 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.647 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.647 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.647 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.647 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T02:17:56.647 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T02:17:56.647 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.647 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.647 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.647 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.648 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.648 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.648 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.648 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.648 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.648 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.648 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.648 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.648 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T02:17:56.648 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T02:17:56.648 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.648 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.648 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.649 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T02:17:56.649 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.649 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.649 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:56.649 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:56.649 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:56.649 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:56.649 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:56.649 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:56.649 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:56.649 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:56.649 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.649 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.650 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.650 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.650 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.650 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T02:17:56.650 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T02:17:56.650 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.650 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.650 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.650 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.650 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.650 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.650 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.650 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.650 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.651 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.651 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.651 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:56.651 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:56.651 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.651 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T02:17:56.651 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "stats", 2024-09-12T02:17:56.651 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T02:17:56.651 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T02:17:56.651 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T02:17:56.651 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T02:17:56.651 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T02:17:56.651 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.652 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.652 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.652 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:56.652 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.652 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.652 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:56.652 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:56.652 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:56.652 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:56.652 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:56.652 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:56.652 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:56.652 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:56.653 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.653 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.653 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.653 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.653 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.653 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T02:17:56.653 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T02:17:56.653 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.653 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.653 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.653 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.653 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.654 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.654 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.654 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.654 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.654 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.654 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.654 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.654 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T02:17:56.654 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T02:17:56.654 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.654 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.654 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.654 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T02:17:56.655 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.655 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.655 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:56.655 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:56.655 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:56.655 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:56.655 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:56.655 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:56.655 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:56.655 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:56.655 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.655 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.655 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.655 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.656 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.656 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T02:17:56.656 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T02:17:56.656 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.656 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.656 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.656 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.656 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.656 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.656 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.656 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.656 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.657 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.657 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.657 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:56.657 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:56.657 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.657 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T02:17:56.657 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "status", 2024-09-12T02:17:56.657 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T02:17:56.657 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T02:17:56.657 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T02:17:56.657 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T02:17:56.657 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T02:17:56.657 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.658 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.658 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.658 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:56.658 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.658 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.658 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:56.658 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:56.658 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:56.658 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:56.658 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:56.658 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:56.658 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:56.658 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:56.659 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.659 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.659 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.659 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.659 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.659 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T02:17:56.659 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T02:17:56.659 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.659 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.659 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.659 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.659 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.660 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.660 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.660 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.660 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.660 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.660 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.660 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.660 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T02:17:56.660 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T02:17:56.660 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.660 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.660 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.660 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T02:17:56.661 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.661 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.661 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:56.661 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:56.661 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:56.661 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:56.661 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:56.661 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:56.661 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:56.661 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:56.661 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.661 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.661 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.662 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.662 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.662 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T02:17:56.662 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T02:17:56.662 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.662 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.662 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.662 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.662 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.662 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.662 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.662 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.663 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.663 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.663 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.663 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:56.663 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:56.663 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.663 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T02:17:56.663 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "telegraf", 2024-09-12T02:17:56.663 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T02:17:56.663 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T02:17:56.663 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T02:17:56.663 INFO:teuthology.orchestra.run.smithi016.stdout: "address": { 2024-09-12T02:17:56.663 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "address", 2024-09-12T02:17:56.663 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.664 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.664 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.664 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-12T02:17:56.664 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.664 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.664 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.664 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.664 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.664 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.664 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.664 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.664 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-09-12T02:17:56.665 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-09-12T02:17:56.665 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-12T02:17:56.665 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.665 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.665 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "15", 2024-09-12T02:17:56.665 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.665 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.665 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.665 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.665 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.665 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.665 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.665 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.666 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T02:17:56.666 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T02:17:56.666 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.666 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.666 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.666 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:56.666 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.666 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.666 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:56.666 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:56.666 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:56.666 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:56.666 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:56.667 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:56.667 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:56.667 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:56.667 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.667 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.667 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.667 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.667 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.667 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T02:17:56.667 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T02:17:56.667 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.667 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.667 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.668 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.668 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.668 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.668 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.668 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.668 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.668 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.668 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.668 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.668 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T02:17:56.668 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T02:17:56.668 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.668 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.669 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.669 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T02:17:56.669 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.669 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.669 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:56.669 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:56.669 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:56.669 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:56.669 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:56.669 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:56.669 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:56.669 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:56.669 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.670 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.670 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.670 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.670 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.670 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T02:17:56.670 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T02:17:56.670 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.670 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.670 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.670 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.670 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.670 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.670 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.671 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.671 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.671 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.671 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.671 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:56.671 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:56.671 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.671 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T02:17:56.671 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "telemetry", 2024-09-12T02:17:56.671 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T02:17:56.671 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T02:17:56.671 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T02:17:56.671 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_basic": { 2024-09-12T02:17:56.672 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_basic", 2024-09-12T02:17:56.672 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.672 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.672 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.672 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T02:17:56.672 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.672 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.672 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.672 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-12T02:17:56.672 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.672 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.673 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.673 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.673 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_crash": { 2024-09-12T02:17:56.673 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_crash", 2024-09-12T02:17:56.673 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.673 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.673 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.673 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T02:17:56.673 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.673 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.673 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.673 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-12T02:17:56.674 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.674 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.674 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.674 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.674 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_device": { 2024-09-12T02:17:56.674 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_device", 2024-09-12T02:17:56.674 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.674 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.674 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.674 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T02:17:56.674 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.674 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.674 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.675 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-12T02:17:56.675 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.675 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.675 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.675 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.675 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_ident": { 2024-09-12T02:17:56.675 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_ident", 2024-09-12T02:17:56.675 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.675 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.675 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.675 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.675 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.676 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.676 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.676 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-12T02:17:56.676 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.676 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.676 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.676 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.676 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_perf": { 2024-09-12T02:17:56.676 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_perf", 2024-09-12T02:17:56.676 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.676 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.676 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.676 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.677 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.677 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.677 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.677 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-12T02:17:56.677 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.677 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.677 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.677 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.677 INFO:teuthology.orchestra.run.smithi016.stdout: "contact": { 2024-09-12T02:17:56.677 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "contact", 2024-09-12T02:17:56.677 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.677 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.677 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.678 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:56.678 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.678 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.678 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.678 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.678 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.678 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.678 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.678 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.678 INFO:teuthology.orchestra.run.smithi016.stdout: "description": { 2024-09-12T02:17:56.678 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "description", 2024-09-12T02:17:56.678 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.679 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.679 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.679 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:56.679 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.679 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.679 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.679 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.679 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.679 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.679 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.679 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.679 INFO:teuthology.orchestra.run.smithi016.stdout: "device_url": { 2024-09-12T02:17:56.679 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "device_url", 2024-09-12T02:17:56.680 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.680 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.680 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.680 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-12T02:17:56.680 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.680 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.680 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.680 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.680 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.680 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.680 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.680 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.681 INFO:teuthology.orchestra.run.smithi016.stdout: "enabled": { 2024-09-12T02:17:56.681 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "enabled", 2024-09-12T02:17:56.681 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.681 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.681 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.681 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.681 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.681 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.681 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.681 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.681 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.681 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.681 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.681 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.682 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-09-12T02:17:56.682 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-09-12T02:17:56.682 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T02:17:56.682 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.682 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.682 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "24", 2024-09-12T02:17:56.682 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "8", 2024-09-12T02:17:56.682 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.682 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.682 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.682 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.682 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.682 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.683 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.683 INFO:teuthology.orchestra.run.smithi016.stdout: "last_opt_revision": { 2024-09-12T02:17:56.683 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "last_opt_revision", 2024-09-12T02:17:56.683 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T02:17:56.683 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.683 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.683 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "1", 2024-09-12T02:17:56.683 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.683 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.683 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.683 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.683 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.683 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.683 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.684 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.684 INFO:teuthology.orchestra.run.smithi016.stdout: "leaderboard": { 2024-09-12T02:17:56.684 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "leaderboard", 2024-09-12T02:17:56.684 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.684 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.684 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.684 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.684 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.684 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.684 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.684 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.684 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.684 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.685 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.685 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.685 INFO:teuthology.orchestra.run.smithi016.stdout: "leaderboard_description": { 2024-09-12T02:17:56.685 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "leaderboard_description", 2024-09-12T02:17:56.685 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.685 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.685 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.685 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:56.685 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.685 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.685 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.685 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.685 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.686 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.686 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.686 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.686 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T02:17:56.686 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T02:17:56.686 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.686 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.686 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.686 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:56.686 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.686 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.686 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:56.687 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:56.687 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:56.687 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:56.687 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:56.687 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:56.687 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:56.687 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:56.687 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.687 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.687 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.687 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.687 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.687 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T02:17:56.688 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T02:17:56.688 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.688 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.688 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.688 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.688 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.688 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.688 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.688 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.688 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.688 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.688 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.689 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.689 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T02:17:56.689 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T02:17:56.689 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.689 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.689 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.689 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T02:17:56.689 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.689 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.689 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:56.689 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:56.689 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:56.689 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:56.690 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:56.690 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:56.690 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:56.690 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:56.690 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.690 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.690 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.690 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.690 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.690 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T02:17:56.690 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T02:17:56.690 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.691 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.691 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.691 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.691 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.691 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.691 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.691 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.691 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.691 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.691 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.691 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.691 INFO:teuthology.orchestra.run.smithi016.stdout: "organization": { 2024-09-12T02:17:56.691 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "organization", 2024-09-12T02:17:56.692 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.692 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.692 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.692 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:56.692 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.692 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.692 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.692 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.692 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.692 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.692 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.692 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.693 INFO:teuthology.orchestra.run.smithi016.stdout: "proxy": { 2024-09-12T02:17:56.693 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "proxy", 2024-09-12T02:17:56.693 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.693 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.693 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.693 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:56.693 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.693 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.693 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.693 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.693 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.693 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.693 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.694 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.694 INFO:teuthology.orchestra.run.smithi016.stdout: "url": { 2024-09-12T02:17:56.694 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "url", 2024-09-12T02:17:56.694 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.694 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.694 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.694 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-12T02:17:56.694 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.694 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.694 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.694 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.694 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.694 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.695 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.695 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:56.695 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:56.695 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.695 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T02:17:56.695 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "test_orchestrator", 2024-09-12T02:17:56.695 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T02:17:56.695 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T02:17:56.695 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T02:17:56.695 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T02:17:56.695 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T02:17:56.695 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.696 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.696 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.696 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:56.696 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.696 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.696 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:56.696 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:56.696 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:56.696 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:56.696 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:56.696 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:56.696 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:56.696 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:56.696 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.697 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.697 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.697 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.697 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.697 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T02:17:56.697 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T02:17:56.697 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.697 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.697 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.697 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.697 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.697 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.697 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.697 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.698 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.698 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.698 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.698 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.698 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T02:17:56.698 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T02:17:56.698 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.698 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.698 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.698 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T02:17:56.698 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.698 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.698 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:56.699 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:56.699 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:56.699 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:56.699 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:56.699 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:56.699 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:56.699 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:56.699 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.699 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.699 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.699 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.699 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.699 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T02:17:56.700 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T02:17:56.700 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.700 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.700 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.700 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.700 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.700 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.700 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.700 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.700 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.700 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.700 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.700 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:56.700 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:56.701 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.701 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T02:17:56.701 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "volumes", 2024-09-12T02:17:56.701 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T02:17:56.701 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T02:17:56.701 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T02:17:56.701 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T02:17:56.701 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T02:17:56.701 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.701 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.701 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.701 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:56.701 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.702 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.702 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:56.702 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:56.702 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:56.702 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:56.702 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:56.702 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:56.702 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:56.702 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:56.702 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.702 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.702 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.702 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.703 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.703 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T02:17:56.703 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T02:17:56.703 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.703 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.703 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.703 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.703 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.703 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.703 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.703 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.703 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.703 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.704 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.704 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.704 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T02:17:56.704 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T02:17:56.704 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.704 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.704 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.704 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T02:17:56.704 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.704 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.704 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:56.704 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:56.704 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:56.705 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:56.705 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:56.705 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:56.705 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:56.705 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:56.705 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.705 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.705 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.705 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.705 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.705 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T02:17:56.705 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T02:17:56.705 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.706 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.706 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.706 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.706 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.706 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.706 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.706 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.706 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.706 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.706 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.706 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.706 INFO:teuthology.orchestra.run.smithi016.stdout: "max_concurrent_clones": { 2024-09-12T02:17:56.706 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_concurrent_clones", 2024-09-12T02:17:56.707 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T02:17:56.707 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.707 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.707 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "4", 2024-09-12T02:17:56.707 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.707 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.707 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.707 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-12T02:17:56.707 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.707 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.707 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.707 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.707 INFO:teuthology.orchestra.run.smithi016.stdout: "periodic_async_work": { 2024-09-12T02:17:56.708 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "periodic_async_work", 2024-09-12T02:17:56.708 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.708 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.708 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.708 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.708 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.708 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.708 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.708 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Periodically check for async work", 2024-09-12T02:17:56.708 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.708 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.708 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.708 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.709 INFO:teuthology.orchestra.run.smithi016.stdout: "snapshot_clone_delay": { 2024-09-12T02:17:56.709 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "snapshot_clone_delay", 2024-09-12T02:17:56.709 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T02:17:56.709 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.709 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.709 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-12T02:17:56.709 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.709 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.709 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.709 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-12T02:17:56.709 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.709 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.709 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.710 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.710 INFO:teuthology.orchestra.run.smithi016.stdout: "snapshot_clone_no_wait": { 2024-09-12T02:17:56.710 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "snapshot_clone_no_wait", 2024-09-12T02:17:56.710 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.710 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.710 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.710 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T02:17:56.710 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.710 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.710 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.710 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-12T02:17:56.710 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.710 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.711 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.711 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:56.711 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:56.711 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.711 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T02:17:56.711 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "zabbix", 2024-09-12T02:17:56.711 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T02:17:56.711 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T02:17:56.711 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T02:17:56.711 INFO:teuthology.orchestra.run.smithi016.stdout: "discovery_interval": { 2024-09-12T02:17:56.711 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "discovery_interval", 2024-09-12T02:17:56.711 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-09-12T02:17:56.712 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.712 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.712 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "100", 2024-09-12T02:17:56.712 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.712 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.712 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.712 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.712 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.712 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.712 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.712 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.712 INFO:teuthology.orchestra.run.smithi016.stdout: "identifier": { 2024-09-12T02:17:56.712 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "identifier", 2024-09-12T02:17:56.712 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.713 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.713 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.713 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:56.713 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.713 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.713 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.713 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.713 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.713 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.713 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.713 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.713 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-09-12T02:17:56.713 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-09-12T02:17:56.714 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-12T02:17:56.714 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.714 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.714 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-09-12T02:17:56.714 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.714 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.714 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.714 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.714 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.714 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.714 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.714 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.714 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T02:17:56.714 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T02:17:56.715 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.715 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.715 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.715 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:56.715 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.715 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.715 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:56.715 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:56.715 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:56.715 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:56.715 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:56.715 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:56.715 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:56.716 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:56.716 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.716 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.716 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.716 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.716 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.716 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T02:17:56.716 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T02:17:56.716 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.716 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.716 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.716 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.716 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.717 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.717 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.717 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.717 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.717 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.717 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.717 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.717 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T02:17:56.717 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T02:17:56.717 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.717 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.717 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.717 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T02:17:56.718 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.718 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.718 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T02:17:56.718 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T02:17:56.718 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T02:17:56.718 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T02:17:56.718 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T02:17:56.718 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T02:17:56.718 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T02:17:56.718 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:56.718 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.718 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.718 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.718 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.719 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.719 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T02:17:56.719 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T02:17:56.719 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T02:17:56.719 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.719 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T02:17:56.719 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T02:17:56.719 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.719 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.719 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.719 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.719 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.719 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.719 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.720 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.720 INFO:teuthology.orchestra.run.smithi016.stdout: "zabbix_host": { 2024-09-12T02:17:56.720 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "zabbix_host", 2024-09-12T02:17:56.720 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.720 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.720 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.720 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T02:17:56.720 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.720 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.720 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.720 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.720 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.720 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.720 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.721 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.721 INFO:teuthology.orchestra.run.smithi016.stdout: "zabbix_port": { 2024-09-12T02:17:56.721 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "zabbix_port", 2024-09-12T02:17:56.721 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T02:17:56.721 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.721 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.721 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10051", 2024-09-12T02:17:56.721 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.721 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.721 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.721 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.721 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.722 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.722 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.722 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.722 INFO:teuthology.orchestra.run.smithi016.stdout: "zabbix_sender": { 2024-09-12T02:17:56.722 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "zabbix_sender", 2024-09-12T02:17:56.722 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T02:17:56.722 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T02:17:56.722 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T02:17:56.722 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-12T02:17:56.722 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T02:17:56.722 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T02:17:56.722 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T02:17:56.722 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T02:17:56.723 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T02:17:56.723 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T02:17:56.723 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T02:17:56.723 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:56.723 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:56.723 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:56.723 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:56.723 INFO:teuthology.orchestra.run.smithi016.stdout: "services": { 2024-09-12T02:17:56.723 INFO:teuthology.orchestra.run.smithi016.stdout: "dashboard": "https://172.21.15.16:8443/" 2024-09-12T02:17:56.723 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.723 INFO:teuthology.orchestra.run.smithi016.stdout: "always_on_modules": { 2024-09-12T02:17:56.723 INFO:teuthology.orchestra.run.smithi016.stdout: "octopus": [ 2024-09-12T02:17:56.723 INFO:teuthology.orchestra.run.smithi016.stdout: "balancer", 2024-09-12T02:17:56.724 INFO:teuthology.orchestra.run.smithi016.stdout: "crash", 2024-09-12T02:17:56.724 INFO:teuthology.orchestra.run.smithi016.stdout: "devicehealth", 2024-09-12T02:17:56.724 INFO:teuthology.orchestra.run.smithi016.stdout: "orchestrator", 2024-09-12T02:17:56.724 INFO:teuthology.orchestra.run.smithi016.stdout: "pg_autoscaler", 2024-09-12T02:17:56.724 INFO:teuthology.orchestra.run.smithi016.stdout: "progress", 2024-09-12T02:17:56.724 INFO:teuthology.orchestra.run.smithi016.stdout: "rbd_support", 2024-09-12T02:17:56.724 INFO:teuthology.orchestra.run.smithi016.stdout: "status", 2024-09-12T02:17:56.724 INFO:teuthology.orchestra.run.smithi016.stdout: "telemetry", 2024-09-12T02:17:56.724 INFO:teuthology.orchestra.run.smithi016.stdout: "volumes" 2024-09-12T02:17:56.724 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:56.724 INFO:teuthology.orchestra.run.smithi016.stdout: "pacific": [ 2024-09-12T02:17:56.724 INFO:teuthology.orchestra.run.smithi016.stdout: "balancer", 2024-09-12T02:17:56.725 INFO:teuthology.orchestra.run.smithi016.stdout: "crash", 2024-09-12T02:17:56.725 INFO:teuthology.orchestra.run.smithi016.stdout: "devicehealth", 2024-09-12T02:17:56.725 INFO:teuthology.orchestra.run.smithi016.stdout: "orchestrator", 2024-09-12T02:17:56.725 INFO:teuthology.orchestra.run.smithi016.stdout: "pg_autoscaler", 2024-09-12T02:17:56.725 INFO:teuthology.orchestra.run.smithi016.stdout: "progress", 2024-09-12T02:17:56.725 INFO:teuthology.orchestra.run.smithi016.stdout: "rbd_support", 2024-09-12T02:17:56.725 INFO:teuthology.orchestra.run.smithi016.stdout: "status", 2024-09-12T02:17:56.725 INFO:teuthology.orchestra.run.smithi016.stdout: "telemetry", 2024-09-12T02:17:56.725 INFO:teuthology.orchestra.run.smithi016.stdout: "volumes" 2024-09-12T02:17:56.725 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:56.725 INFO:teuthology.orchestra.run.smithi016.stdout: "quincy": [ 2024-09-12T02:17:56.725 INFO:teuthology.orchestra.run.smithi016.stdout: "balancer", 2024-09-12T02:17:56.725 INFO:teuthology.orchestra.run.smithi016.stdout: "crash", 2024-09-12T02:17:56.725 INFO:teuthology.orchestra.run.smithi016.stdout: "devicehealth", 2024-09-12T02:17:56.726 INFO:teuthology.orchestra.run.smithi016.stdout: "orchestrator", 2024-09-12T02:17:56.726 INFO:teuthology.orchestra.run.smithi016.stdout: "pg_autoscaler", 2024-09-12T02:17:56.726 INFO:teuthology.orchestra.run.smithi016.stdout: "progress", 2024-09-12T02:17:56.726 INFO:teuthology.orchestra.run.smithi016.stdout: "rbd_support", 2024-09-12T02:17:56.726 INFO:teuthology.orchestra.run.smithi016.stdout: "status", 2024-09-12T02:17:56.726 INFO:teuthology.orchestra.run.smithi016.stdout: "telemetry", 2024-09-12T02:17:56.726 INFO:teuthology.orchestra.run.smithi016.stdout: "volumes" 2024-09-12T02:17:56.726 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:56.726 INFO:teuthology.orchestra.run.smithi016.stdout: "reef": [ 2024-09-12T02:17:56.726 INFO:teuthology.orchestra.run.smithi016.stdout: "balancer", 2024-09-12T02:17:56.726 INFO:teuthology.orchestra.run.smithi016.stdout: "crash", 2024-09-12T02:17:56.727 INFO:teuthology.orchestra.run.smithi016.stdout: "devicehealth", 2024-09-12T02:17:56.727 INFO:teuthology.orchestra.run.smithi016.stdout: "orchestrator", 2024-09-12T02:17:56.727 INFO:teuthology.orchestra.run.smithi016.stdout: "pg_autoscaler", 2024-09-12T02:17:56.727 INFO:teuthology.orchestra.run.smithi016.stdout: "progress", 2024-09-12T02:17:56.727 INFO:teuthology.orchestra.run.smithi016.stdout: "rbd_support", 2024-09-12T02:17:56.727 INFO:teuthology.orchestra.run.smithi016.stdout: "status", 2024-09-12T02:17:56.727 INFO:teuthology.orchestra.run.smithi016.stdout: "telemetry", 2024-09-12T02:17:56.727 INFO:teuthology.orchestra.run.smithi016.stdout: "volumes" 2024-09-12T02:17:56.727 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T02:17:56.727 INFO:teuthology.orchestra.run.smithi016.stdout: "squid": [ 2024-09-12T02:17:56.727 INFO:teuthology.orchestra.run.smithi016.stdout: "balancer", 2024-09-12T02:17:56.727 INFO:teuthology.orchestra.run.smithi016.stdout: "crash", 2024-09-12T02:17:56.727 INFO:teuthology.orchestra.run.smithi016.stdout: "devicehealth", 2024-09-12T02:17:56.727 INFO:teuthology.orchestra.run.smithi016.stdout: "orchestrator", 2024-09-12T02:17:56.728 INFO:teuthology.orchestra.run.smithi016.stdout: "pg_autoscaler", 2024-09-12T02:17:56.728 INFO:teuthology.orchestra.run.smithi016.stdout: "progress", 2024-09-12T02:17:56.728 INFO:teuthology.orchestra.run.smithi016.stdout: "rbd_support", 2024-09-12T02:17:56.728 INFO:teuthology.orchestra.run.smithi016.stdout: "status", 2024-09-12T02:17:56.728 INFO:teuthology.orchestra.run.smithi016.stdout: "telemetry", 2024-09-12T02:17:56.728 INFO:teuthology.orchestra.run.smithi016.stdout: "volumes" 2024-09-12T02:17:56.728 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-09-12T02:17:56.728 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.728 INFO:teuthology.orchestra.run.smithi016.stdout: "last_failure_osd_epoch": 3, 2024-09-12T02:17:56.728 INFO:teuthology.orchestra.run.smithi016.stdout: "active_clients": [ 2024-09-12T02:17:56.728 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T02:17:56.728 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "devicehealth", 2024-09-12T02:17:56.728 INFO:teuthology.orchestra.run.smithi016.stdout: "addrvec": [ 2024-09-12T02:17:56.729 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T02:17:56.729 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v2", 2024-09-12T02:17:56.729 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "172.21.15.16:0", 2024-09-12T02:17:56.729 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 890192749 2024-09-12T02:17:56.729 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:56.729 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-09-12T02:17:56.729 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.729 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T02:17:56.729 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "libcephsqlite", 2024-09-12T02:17:56.729 INFO:teuthology.orchestra.run.smithi016.stdout: "addrvec": [ 2024-09-12T02:17:56.729 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T02:17:56.729 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v2", 2024-09-12T02:17:56.730 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "172.21.15.16:0", 2024-09-12T02:17:56.730 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 2939463879 2024-09-12T02:17:56.730 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:56.730 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-09-12T02:17:56.730 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.730 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T02:17:56.730 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rbd_support", 2024-09-12T02:17:56.730 INFO:teuthology.orchestra.run.smithi016.stdout: "addrvec": [ 2024-09-12T02:17:56.730 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T02:17:56.730 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v2", 2024-09-12T02:17:56.730 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "172.21.15.16:0", 2024-09-12T02:17:56.730 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 3820405214 2024-09-12T02:17:56.730 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:56.731 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-09-12T02:17:56.731 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:56.731 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T02:17:56.731 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "volumes", 2024-09-12T02:17:56.731 INFO:teuthology.orchestra.run.smithi016.stdout: "addrvec": [ 2024-09-12T02:17:56.731 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T02:17:56.731 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v2", 2024-09-12T02:17:56.731 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "172.21.15.16:0", 2024-09-12T02:17:56.731 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 3431006499 2024-09-12T02:17:56.731 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:56.731 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-09-12T02:17:56.731 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:56.732 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-09-12T02:17:56.732 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T02:17:56.733 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:17:55 smithi033 ceph-mon[35332]: pgmap v815: 97 pgs: 97 active+clean; 4.3 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 3.5 KiB/s wr, 11 op/s 2024-09-12T02:17:56.733 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:17:55 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/1569776192' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-12T02:17:56.733 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:17:55 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/3263547107' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-12T02:17:56.733 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:17:55 smithi184 ceph-mon[35593]: pgmap v815: 97 pgs: 97 active+clean; 4.3 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 3.5 KiB/s wr, 11 op/s 2024-09-12T02:17:56.733 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:17:55 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/1569776192' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-12T02:17:56.733 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:17:55 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/3263547107' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-12T02:17:56.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:55.305+0000 7f69b40ba640 1 -- 172.21.15.16:0/3263547107 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f69980a2c10 msgr2=0x7f69980a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:56.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:55.305+0000 7f69b40ba640 1 --2- 172.21.15.16:0/3263547107 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f69980a2c10 0x7f69980a50d0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f69a800bbe0 tx=0x7f69a800b390 comp rx=0 tx=0).stop 2024-09-12T02:17:56.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:55.305+0000 7f69b40ba640 1 -- 172.21.15.16:0/3263547107 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69ac107810 msgr2=0x7f69ac100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:56.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:55.305+0000 7f69b40ba640 1 --2- 172.21.15.16:0/3263547107 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69ac107810 0x7f69ac100aa0 secure :-1 s=READY pgs=2152 cs=0 l=1 rev1=1 crypto rx=0x7f69a400e9d0 tx=0x7f69a400ee90 comp rx=0 tx=0).stop 2024-09-12T02:17:56.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:55.305+0000 7f69b40ba640 1 -- 172.21.15.16:0/3263547107 shutdown_connections 2024-09-12T02:17:56.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:55.305+0000 7f69b40ba640 1 --2- 172.21.15.16:0/3263547107 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f69980a2c10 0x7f69980a50d0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:56.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:55.305+0000 7f69b40ba640 1 --2- 172.21.15.16:0/3263547107 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69ac107810 0x7f69ac100aa0 unknown :-1 s=CLOSED pgs=2152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:56.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:55.305+0000 7f69b40ba640 1 --2- 172.21.15.16:0/3263547107 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f69ac104ee0 0x7f69ac100560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:56.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:55.305+0000 7f69b40ba640 1 --2- 172.21.15.16:0/3263547107 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f69ac1025b0 0x7f69ac101f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:56.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:55.305+0000 7f69b40ba640 1 -- 172.21.15.16:0/3263547107 >> 172.21.15.16:0/3263547107 conn(0x7f69ac0fc1b0 msgr2=0x7f69ac102f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:56.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:55.305+0000 7f69b40ba640 1 -- 172.21.15.16:0/3263547107 shutdown_connections 2024-09-12T02:17:56.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:17:55.305+0000 7f69b40ba640 1 -- 172.21.15.16:0/3263547107 wait complete. 2024-09-12T02:17:56.799 DEBUG:tasks.check_counter:Getting stats from a 2024-09-12T02:17:56.800 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph daemon mds.a perf dump 2024-09-12T02:17:56.800 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:17:55 smithi016 ceph-mon[30947]: pgmap v815: 97 pgs: 97 active+clean; 4.3 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 3.5 KiB/s wr, 11 op/s 2024-09-12T02:17:56.800 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:17:55 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/1569776192' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-12T02:17:56.800 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:17:55 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/3263547107' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-12T02:17:57.070 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:17:57.534 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T02:17:57.534 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T02:17:57.534 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 296, 2024-09-12T02:17:57.534 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 293, 2024-09-12T02:17:57.534 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 500624, 2024-09-12T02:17:57.534 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 180400, 2024-09-12T02:17:57.534 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 4, 2024-09-12T02:17:57.535 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 0, 2024-09-12T02:17:57.535 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.042398772, 2024-09-12T02:17:57.535 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.021700501, 2024-09-12T02:17:57.535 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.022077041, 2024-09-12T02:17:57.535 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T02:17:57.535 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T02:17:57.535 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 292, 2024-09-12T02:17:57.535 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.016115122, 2024-09-12T02:17:57.535 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000055188 2024-09-12T02:17:57.535 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:57.536 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-09-12T02:17:57.536 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-12T02:17:57.536 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-12T02:17:57.536 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-12T02:17:57.536 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:57.536 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 500624, 2024-09-12T02:17:57.536 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 180400 2024-09-12T02:17:57.536 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:57.536 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T02:17:57.536 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 2, 2024-09-12T02:17:57.536 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 223, 2024-09-12T02:17:57.536 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 320, 2024-09-12T02:17:57.536 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 196656, 2024-09-12T02:17:57.537 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 4, 2024-09-12T02:17:57.537 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 2, 2024-09-12T02:17:57.537 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.019426030, 2024-09-12T02:17:57.537 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.014916372, 2024-09-12T02:17:57.537 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.001931091, 2024-09-12T02:17:57.537 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T02:17:57.537 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T02:17:57.537 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 221, 2024-09-12T02:17:57.537 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.012819909, 2024-09-12T02:17:57.537 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000058008 2024-09-12T02:17:57.537 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:57.538 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-09-12T02:17:57.538 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-12T02:17:57.538 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-12T02:17:57.538 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-12T02:17:57.538 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:57.538 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-12T02:17:57.538 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 196656 2024-09-12T02:17:57.538 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:57.538 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T02:17:57.538 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 4, 2024-09-12T02:17:57.538 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 4, 2024-09-12T02:17:57.538 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 3024, 2024-09-12T02:17:57.538 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 448, 2024-09-12T02:17:57.538 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 2, 2024-09-12T02:17:57.539 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 1, 2024-09-12T02:17:57.539 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.002590249, 2024-09-12T02:17:57.539 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.000050939, 2024-09-12T02:17:57.539 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.002555830, 2024-09-12T02:17:57.539 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T02:17:57.539 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T02:17:57.539 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 3, 2024-09-12T02:17:57.539 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.004290341, 2024-09-12T02:17:57.539 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.001430113 2024-09-12T02:17:57.539 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:57.539 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-09-12T02:17:57.539 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-12T02:17:57.539 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-12T02:17:57.539 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-12T02:17:57.540 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:57.540 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-09-12T02:17:57.540 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 448 2024-09-12T02:17:57.540 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:57.540 INFO:teuthology.orchestra.run.smithi016.stdout: "cct": { 2024-09-12T02:17:57.540 INFO:teuthology.orchestra.run.smithi016.stdout: "total_workers": 0, 2024-09-12T02:17:57.540 INFO:teuthology.orchestra.run.smithi016.stdout: "unhealthy_workers": 0 2024-09-12T02:17:57.540 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:57.540 INFO:teuthology.orchestra.run.smithi016.stdout: "mempool": { 2024-09-12T02:17:57.540 INFO:teuthology.orchestra.run.smithi016.stdout: "bloom_filter_bytes": 0, 2024-09-12T02:17:57.540 INFO:teuthology.orchestra.run.smithi016.stdout: "bloom_filter_items": 0, 2024-09-12T02:17:57.540 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T02:17:57.540 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_alloc_items": 0, 2024-09-12T02:17:57.541 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T02:17:57.541 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_data_items": 0, 2024-09-12T02:17:57.541 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T02:17:57.541 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T02:17:57.541 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T02:17:57.541 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T02:17:57.541 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T02:17:57.541 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_other_items": 0, 2024-09-12T02:17:57.541 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T02:17:57.541 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T02:17:57.541 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_extent_bytes": 0, 2024-09-12T02:17:57.542 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_extent_items": 0, 2024-09-12T02:17:57.542 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_blob_bytes": 0, 2024-09-12T02:17:57.542 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_blob_items": 0, 2024-09-12T02:17:57.542 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T02:17:57.542 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T02:17:57.542 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T02:17:57.542 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T02:17:57.542 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T02:17:57.542 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_fsck_items": 0, 2024-09-12T02:17:57.542 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_txc_bytes": 0, 2024-09-12T02:17:57.542 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_txc_items": 0, 2024-09-12T02:17:57.542 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T02:17:57.542 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T02:17:57.542 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_bytes": 0, 2024-09-12T02:17:57.543 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_items": 0, 2024-09-12T02:17:57.543 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_bytes": 0, 2024-09-12T02:17:57.543 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_items": 0, 2024-09-12T02:17:57.543 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T02:17:57.543 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_reader_items": 0, 2024-09-12T02:17:57.543 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T02:17:57.543 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_writer_items": 0, 2024-09-12T02:17:57.543 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_anon_bytes": 148704, 2024-09-12T02:17:57.543 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_anon_items": 29, 2024-09-12T02:17:57.543 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_meta_bytes": 0, 2024-09-12T02:17:57.543 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_meta_items": 0, 2024-09-12T02:17:57.543 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_bytes": 0, 2024-09-12T02:17:57.543 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_items": 0, 2024-09-12T02:17:57.543 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_mapbl_bytes": 0, 2024-09-12T02:17:57.544 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_mapbl_items": 0, 2024-09-12T02:17:57.544 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_pglog_bytes": 0, 2024-09-12T02:17:57.544 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_pglog_items": 0, 2024-09-12T02:17:57.544 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_bytes": 0, 2024-09-12T02:17:57.544 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_items": 0, 2024-09-12T02:17:57.544 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T02:17:57.544 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_mapping_items": 0, 2024-09-12T02:17:57.544 INFO:teuthology.orchestra.run.smithi016.stdout: "pgmap_bytes": 0, 2024-09-12T02:17:57.544 INFO:teuthology.orchestra.run.smithi016.stdout: "pgmap_items": 0, 2024-09-12T02:17:57.544 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_co_bytes": 928, 2024-09-12T02:17:57.544 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_co_items": 2, 2024-09-12T02:17:57.544 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_1_bytes": 0, 2024-09-12T02:17:57.544 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_1_items": 0, 2024-09-12T02:17:57.544 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_2_bytes": 0, 2024-09-12T02:17:57.545 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_2_items": 0 2024-09-12T02:17:57.545 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:57.545 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T02:17:57.545 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-09-12T02:17:57.545 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 104857600, 2024-09-12T02:17:57.545 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-09-12T02:17:57.545 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 298, 2024-09-12T02:17:57.545 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 458809, 2024-09-12T02:17:57.545 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-09-12T02:17:57.546 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 298, 2024-09-12T02:17:57.546 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 0, 2024-09-12T02:17:57.546 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 0, 2024-09-12T02:17:57.546 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 298, 2024-09-12T02:17:57.546 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 458809, 2024-09-12T02:17:57.546 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-09-12T02:17:57.546 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-12T02:17:57.546 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-12T02:17:57.546 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-12T02:17:57.546 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:57.546 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:57.546 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T02:17:57.933 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:17:57 smithi016 ceph-mon[30947]: pgmap v816: 97 pgs: 97 active+clean; 4.3 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 3.5 KiB/s wr, 11 op/s 2024-09-12T02:17:58.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:17:57 smithi184 ceph-mon[35593]: pgmap v816: 97 pgs: 97 active+clean; 4.3 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 3.5 KiB/s wr, 11 op/s 2024-09-12T02:17:58.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:17:57 smithi033 ceph-mon[35332]: pgmap v816: 97 pgs: 97 active+clean; 4.3 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 3.5 KiB/s wr, 11 op/s 2024-09-12T02:17:58.327 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.a 2024-09-12T02:17:58.327 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.a 2024-09-12T02:17:58.327 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.a 2024-09-12T02:17:58.327 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.a 2024-09-12T02:17:58.328 DEBUG:tasks.check_counter:Getting stats from d 2024-09-12T02:17:58.328 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph daemon mds.d perf dump 2024-09-12T02:17:58.591 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:17:59.088 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T02:17:59.088 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T02:17:59.088 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 4, 2024-09-12T02:17:59.088 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 4, 2024-09-12T02:17:59.088 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 3024, 2024-09-12T02:17:59.088 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 448, 2024-09-12T02:17:59.088 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 4, 2024-09-12T02:17:59.089 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 0, 2024-09-12T02:17:59.089 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.004074088, 2024-09-12T02:17:59.089 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.000055749, 2024-09-12T02:17:59.089 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.002687541, 2024-09-12T02:17:59.089 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T02:17:59.089 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T02:17:59.089 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 3, 2024-09-12T02:17:59.089 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.004762168, 2024-09-12T02:17:59.089 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.001587389 2024-09-12T02:17:59.089 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:59.089 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-09-12T02:17:59.089 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-12T02:17:59.089 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-12T02:17:59.090 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-12T02:17:59.090 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:59.090 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-09-12T02:17:59.090 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 448 2024-09-12T02:17:59.090 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:59.090 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T02:17:59.090 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 301, 2024-09-12T02:17:59.090 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 298, 2024-09-12T02:17:59.090 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 501984, 2024-09-12T02:17:59.090 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 183600, 2024-09-12T02:17:59.090 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 2, 2024-09-12T02:17:59.090 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 1, 2024-09-12T02:17:59.091 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.041680842, 2024-09-12T02:17:59.091 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.021741353, 2024-09-12T02:17:59.091 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.023064909, 2024-09-12T02:17:59.091 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T02:17:59.091 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T02:17:59.091 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 297, 2024-09-12T02:17:59.091 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.017042947, 2024-09-12T02:17:59.091 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000057383 2024-09-12T02:17:59.091 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:59.091 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-09-12T02:17:59.091 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-12T02:17:59.091 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-12T02:17:59.091 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-12T02:17:59.091 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:59.092 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 501984, 2024-09-12T02:17:59.092 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 183600 2024-09-12T02:17:59.092 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:59.092 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T02:17:59.092 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 2, 2024-09-12T02:17:59.092 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 227, 2024-09-12T02:17:59.092 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 320, 2024-09-12T02:17:59.092 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 197360, 2024-09-12T02:17:59.092 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 4, 2024-09-12T02:17:59.092 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 2, 2024-09-12T02:17:59.092 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.019274016, 2024-09-12T02:17:59.092 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.014336645, 2024-09-12T02:17:59.092 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.002121951, 2024-09-12T02:17:59.092 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T02:17:59.093 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T02:17:59.093 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 225, 2024-09-12T02:17:59.093 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.013356236, 2024-09-12T02:17:59.093 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000059361 2024-09-12T02:17:59.093 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:59.093 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-09-12T02:17:59.093 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-12T02:17:59.093 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-12T02:17:59.093 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-12T02:17:59.093 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:59.093 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-12T02:17:59.093 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 197360 2024-09-12T02:17:59.093 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:59.093 INFO:teuthology.orchestra.run.smithi016.stdout: "cct": { 2024-09-12T02:17:59.094 INFO:teuthology.orchestra.run.smithi016.stdout: "total_workers": 0, 2024-09-12T02:17:59.094 INFO:teuthology.orchestra.run.smithi016.stdout: "unhealthy_workers": 0 2024-09-12T02:17:59.094 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:59.094 INFO:teuthology.orchestra.run.smithi016.stdout: "mempool": { 2024-09-12T02:17:59.094 INFO:teuthology.orchestra.run.smithi016.stdout: "bloom_filter_bytes": 0, 2024-09-12T02:17:59.094 INFO:teuthology.orchestra.run.smithi016.stdout: "bloom_filter_items": 0, 2024-09-12T02:17:59.094 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T02:17:59.094 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_alloc_items": 0, 2024-09-12T02:17:59.094 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T02:17:59.094 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_data_items": 0, 2024-09-12T02:17:59.094 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T02:17:59.094 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T02:17:59.094 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T02:17:59.095 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T02:17:59.095 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T02:17:59.095 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_other_items": 0, 2024-09-12T02:17:59.095 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T02:17:59.095 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T02:17:59.095 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_extent_bytes": 0, 2024-09-12T02:17:59.095 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_extent_items": 0, 2024-09-12T02:17:59.095 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_blob_bytes": 0, 2024-09-12T02:17:59.095 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_blob_items": 0, 2024-09-12T02:17:59.095 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T02:17:59.095 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T02:17:59.095 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T02:17:59.095 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T02:17:59.096 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T02:17:59.096 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_fsck_items": 0, 2024-09-12T02:17:59.096 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_txc_bytes": 0, 2024-09-12T02:17:59.096 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_txc_items": 0, 2024-09-12T02:17:59.096 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T02:17:59.096 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T02:17:59.096 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_bytes": 0, 2024-09-12T02:17:59.096 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_items": 0, 2024-09-12T02:17:59.096 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_bytes": 0, 2024-09-12T02:17:59.096 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_items": 0, 2024-09-12T02:17:59.096 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T02:17:59.096 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_reader_items": 0, 2024-09-12T02:17:59.097 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T02:17:59.097 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_writer_items": 0, 2024-09-12T02:17:59.097 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_anon_bytes": 148704, 2024-09-12T02:17:59.097 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_anon_items": 29, 2024-09-12T02:17:59.097 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_meta_bytes": 0, 2024-09-12T02:17:59.097 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_meta_items": 0, 2024-09-12T02:17:59.097 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_bytes": 0, 2024-09-12T02:17:59.097 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_items": 0, 2024-09-12T02:17:59.097 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_mapbl_bytes": 0, 2024-09-12T02:17:59.097 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_mapbl_items": 0, 2024-09-12T02:17:59.097 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_pglog_bytes": 0, 2024-09-12T02:17:59.097 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_pglog_items": 0, 2024-09-12T02:17:59.098 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_bytes": 0, 2024-09-12T02:17:59.098 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_items": 0, 2024-09-12T02:17:59.098 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T02:17:59.098 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_mapping_items": 0, 2024-09-12T02:17:59.098 INFO:teuthology.orchestra.run.smithi016.stdout: "pgmap_bytes": 0, 2024-09-12T02:17:59.098 INFO:teuthology.orchestra.run.smithi016.stdout: "pgmap_items": 0, 2024-09-12T02:17:59.098 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_co_bytes": 928, 2024-09-12T02:17:59.098 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_co_items": 2, 2024-09-12T02:17:59.098 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_1_bytes": 0, 2024-09-12T02:17:59.098 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_1_items": 0, 2024-09-12T02:17:59.098 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_2_bytes": 0, 2024-09-12T02:17:59.098 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_2_items": 0 2024-09-12T02:17:59.098 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:17:59.099 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T02:17:59.099 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-09-12T02:17:59.099 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 104857600, 2024-09-12T02:17:59.099 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-09-12T02:17:59.099 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 303, 2024-09-12T02:17:59.099 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 459459, 2024-09-12T02:17:59.099 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-09-12T02:17:59.099 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 303, 2024-09-12T02:17:59.099 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 0, 2024-09-12T02:17:59.099 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 0, 2024-09-12T02:17:59.099 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 303, 2024-09-12T02:17:59.099 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 459459, 2024-09-12T02:17:59.099 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-09-12T02:17:59.100 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-12T02:17:59.100 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-12T02:17:59.100 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-12T02:17:59.100 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:59.100 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:17:59.100 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T02:17:59.890 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-09-12T02:17:59.890 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-09-12T02:17:59.890 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.d 2024-09-12T02:17:59.890 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.d 2024-09-12T02:17:59.890 DEBUG:tasks.check_counter:Getting stats from g 2024-09-12T02:17:59.891 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph daemon mds.g perf dump 2024-09-12T02:18:00.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:17:59 smithi184 ceph-mon[35593]: pgmap v817: 97 pgs: 97 active+clean; 4.3 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 682 B/s wr, 5 op/s 2024-09-12T02:18:00.150 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:18:00.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:17:59 smithi033 ceph-mon[35332]: pgmap v817: 97 pgs: 97 active+clean; 4.3 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 682 B/s wr, 5 op/s 2024-09-12T02:18:00.183 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:17:59 smithi016 ceph-mon[30947]: pgmap v817: 97 pgs: 97 active+clean; 4.3 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 682 B/s wr, 5 op/s 2024-09-12T02:18:00.672 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T02:18:00.672 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T02:18:00.672 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 297, 2024-09-12T02:18:00.672 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 294, 2024-09-12T02:18:00.672 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 500896, 2024-09-12T02:18:00.673 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 181040, 2024-09-12T02:18:00.673 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 4, 2024-09-12T02:18:00.673 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 0, 2024-09-12T02:18:00.673 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.041621248, 2024-09-12T02:18:00.673 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.021650727, 2024-09-12T02:18:00.673 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.021523964, 2024-09-12T02:18:00.673 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T02:18:00.673 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T02:18:00.673 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 293, 2024-09-12T02:18:00.673 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.016515669, 2024-09-12T02:18:00.673 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000056367 2024-09-12T02:18:00.673 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:18:00.674 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-09-12T02:18:00.674 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-12T02:18:00.674 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-12T02:18:00.674 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-12T02:18:00.674 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:18:00.674 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 500896, 2024-09-12T02:18:00.674 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 181040 2024-09-12T02:18:00.674 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:18:00.674 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T02:18:00.674 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 2, 2024-09-12T02:18:00.674 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 224, 2024-09-12T02:18:00.674 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 320, 2024-09-12T02:18:00.674 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 196832, 2024-09-12T02:18:00.674 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 4, 2024-09-12T02:18:00.675 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 2, 2024-09-12T02:18:00.675 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.018750470, 2024-09-12T02:18:00.675 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.014132791, 2024-09-12T02:18:00.675 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.001943977, 2024-09-12T02:18:00.675 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T02:18:00.675 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T02:18:00.675 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 222, 2024-09-12T02:18:00.675 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.013007705, 2024-09-12T02:18:00.675 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000058593 2024-09-12T02:18:00.675 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:18:00.676 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-09-12T02:18:00.676 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-12T02:18:00.676 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-12T02:18:00.676 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-12T02:18:00.676 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:18:00.676 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-12T02:18:00.676 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 196832 2024-09-12T02:18:00.676 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:18:00.676 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T02:18:00.676 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 4, 2024-09-12T02:18:00.676 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 4, 2024-09-12T02:18:00.677 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 3024, 2024-09-12T02:18:00.677 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 448, 2024-09-12T02:18:00.677 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 2, 2024-09-12T02:18:00.677 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 1, 2024-09-12T02:18:00.677 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.002512643, 2024-09-12T02:18:00.677 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.000036527, 2024-09-12T02:18:00.677 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.002418990, 2024-09-12T02:18:00.677 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T02:18:00.677 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T02:18:00.677 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 3, 2024-09-12T02:18:00.678 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.004318032, 2024-09-12T02:18:00.678 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.001439344 2024-09-12T02:18:00.678 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:18:00.678 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-09-12T02:18:00.678 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-12T02:18:00.678 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-12T02:18:00.678 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-12T02:18:00.678 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:18:00.678 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-09-12T02:18:00.678 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 448 2024-09-12T02:18:00.678 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:18:00.678 INFO:teuthology.orchestra.run.smithi016.stdout: "cct": { 2024-09-12T02:18:00.678 INFO:teuthology.orchestra.run.smithi016.stdout: "total_workers": 0, 2024-09-12T02:18:00.678 INFO:teuthology.orchestra.run.smithi016.stdout: "unhealthy_workers": 0 2024-09-12T02:18:00.679 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:18:00.679 INFO:teuthology.orchestra.run.smithi016.stdout: "mempool": { 2024-09-12T02:18:00.679 INFO:teuthology.orchestra.run.smithi016.stdout: "bloom_filter_bytes": 0, 2024-09-12T02:18:00.679 INFO:teuthology.orchestra.run.smithi016.stdout: "bloom_filter_items": 0, 2024-09-12T02:18:00.679 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T02:18:00.679 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_alloc_items": 0, 2024-09-12T02:18:00.679 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T02:18:00.679 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_data_items": 0, 2024-09-12T02:18:00.679 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T02:18:00.679 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T02:18:00.679 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T02:18:00.679 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T02:18:00.679 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T02:18:00.680 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_other_items": 0, 2024-09-12T02:18:00.680 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T02:18:00.680 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T02:18:00.680 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_extent_bytes": 0, 2024-09-12T02:18:00.680 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_extent_items": 0, 2024-09-12T02:18:00.680 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_blob_bytes": 0, 2024-09-12T02:18:00.680 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_blob_items": 0, 2024-09-12T02:18:00.680 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T02:18:00.680 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T02:18:00.680 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T02:18:00.680 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T02:18:00.680 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T02:18:00.680 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_fsck_items": 0, 2024-09-12T02:18:00.681 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_txc_bytes": 0, 2024-09-12T02:18:00.681 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_txc_items": 0, 2024-09-12T02:18:00.681 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T02:18:00.681 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T02:18:00.681 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_bytes": 0, 2024-09-12T02:18:00.681 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_items": 0, 2024-09-12T02:18:00.681 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_bytes": 0, 2024-09-12T02:18:00.681 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_items": 0, 2024-09-12T02:18:00.681 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T02:18:00.681 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_reader_items": 0, 2024-09-12T02:18:00.681 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T02:18:00.681 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_writer_items": 0, 2024-09-12T02:18:00.682 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_anon_bytes": 149752, 2024-09-12T02:18:00.682 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_anon_items": 31, 2024-09-12T02:18:00.682 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_meta_bytes": 0, 2024-09-12T02:18:00.682 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_meta_items": 0, 2024-09-12T02:18:00.682 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_bytes": 0, 2024-09-12T02:18:00.682 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_items": 0, 2024-09-12T02:18:00.682 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_mapbl_bytes": 0, 2024-09-12T02:18:00.682 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_mapbl_items": 0, 2024-09-12T02:18:00.682 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_pglog_bytes": 0, 2024-09-12T02:18:00.682 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_pglog_items": 0, 2024-09-12T02:18:00.682 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_bytes": 0, 2024-09-12T02:18:00.682 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_items": 0, 2024-09-12T02:18:00.682 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T02:18:00.683 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_mapping_items": 0, 2024-09-12T02:18:00.683 INFO:teuthology.orchestra.run.smithi016.stdout: "pgmap_bytes": 0, 2024-09-12T02:18:00.683 INFO:teuthology.orchestra.run.smithi016.stdout: "pgmap_items": 0, 2024-09-12T02:18:00.683 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_co_bytes": 928, 2024-09-12T02:18:00.683 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_co_items": 2, 2024-09-12T02:18:00.683 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_1_bytes": 0, 2024-09-12T02:18:00.683 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_1_items": 0, 2024-09-12T02:18:00.683 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_2_bytes": 0, 2024-09-12T02:18:00.683 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_2_items": 0 2024-09-12T02:18:00.683 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:18:00.683 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T02:18:00.683 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-09-12T02:18:00.683 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 104857600, 2024-09-12T02:18:00.683 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-09-12T02:18:00.683 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 299, 2024-09-12T02:18:00.683 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 458939, 2024-09-12T02:18:00.684 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-09-12T02:18:00.684 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 299, 2024-09-12T02:18:00.684 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 0, 2024-09-12T02:18:00.684 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 0, 2024-09-12T02:18:00.684 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 299, 2024-09-12T02:18:00.684 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 458939, 2024-09-12T02:18:00.684 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-09-12T02:18:00.684 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-12T02:18:00.684 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-12T02:18:00.684 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-12T02:18:00.684 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:18:00.684 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:18:00.684 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T02:18:01.482 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2024-09-12T02:18:01.482 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2024-09-12T02:18:01.482 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.g 2024-09-12T02:18:01.482 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.g 2024-09-12T02:18:01.482 DEBUG:tasks.check_counter:Getting stats from j 2024-09-12T02:18:01.483 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph daemon mds.j perf dump 2024-09-12T02:18:01.750 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:18:02.009 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:18:01 smithi016 ceph-mon[30947]: pgmap v818: 97 pgs: 97 active+clean; 4.3 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-12T02:18:02.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:18:01 smithi184 ceph-mon[35593]: pgmap v818: 97 pgs: 97 active+clean; 4.3 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-12T02:18:02.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:18:01 smithi033 ceph-mon[35332]: pgmap v818: 97 pgs: 97 active+clean; 4.3 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-12T02:18:02.223 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T02:18:02.224 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T02:18:02.224 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 0, 2024-09-12T02:18:02.224 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 2, 2024-09-12T02:18:02.224 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 0, 2024-09-12T02:18:02.224 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 0, 2024-09-12T02:18:02.224 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 4, 2024-09-12T02:18:02.224 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 0, 2024-09-12T02:18:02.224 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.001265637, 2024-09-12T02:18:02.224 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.000000000, 2024-09-12T02:18:02.224 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.000256823, 2024-09-12T02:18:02.224 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T02:18:02.225 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T02:18:02.225 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-12T02:18:02.225 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-12T02:18:02.225 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-12T02:18:02.225 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:18:02.225 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-09-12T02:18:02.225 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-12T02:18:02.225 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-12T02:18:02.225 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-12T02:18:02.225 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:18:02.225 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-12T02:18:02.225 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 0 2024-09-12T02:18:02.225 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:18:02.226 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T02:18:02.226 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 302, 2024-09-12T02:18:02.226 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 299, 2024-09-12T02:18:02.226 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 502256, 2024-09-12T02:18:02.226 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 184240, 2024-09-12T02:18:02.226 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 2, 2024-09-12T02:18:02.226 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 1, 2024-09-12T02:18:02.226 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.042060534, 2024-09-12T02:18:02.226 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.021718022, 2024-09-12T02:18:02.226 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.023349142, 2024-09-12T02:18:02.226 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T02:18:02.226 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T02:18:02.226 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 298, 2024-09-12T02:18:02.226 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.016515813, 2024-09-12T02:18:02.227 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000055422 2024-09-12T02:18:02.227 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:18:02.227 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-09-12T02:18:02.227 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-12T02:18:02.227 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-12T02:18:02.227 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-12T02:18:02.227 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:18:02.227 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 502256, 2024-09-12T02:18:02.227 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 184240 2024-09-12T02:18:02.227 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:18:02.227 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T02:18:02.228 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 6, 2024-09-12T02:18:02.228 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 230, 2024-09-12T02:18:02.228 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 3344, 2024-09-12T02:18:02.228 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 197984, 2024-09-12T02:18:02.228 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 4, 2024-09-12T02:18:02.228 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 2, 2024-09-12T02:18:02.228 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.021022463, 2024-09-12T02:18:02.228 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.014975653, 2024-09-12T02:18:02.228 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.003590070, 2024-09-12T02:18:02.228 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T02:18:02.228 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T02:18:02.228 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 229, 2024-09-12T02:18:02.228 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.016720762, 2024-09-12T02:18:02.229 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000073016 2024-09-12T02:18:02.229 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:18:02.229 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-09-12T02:18:02.229 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-12T02:18:02.229 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-12T02:18:02.229 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-12T02:18:02.229 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:18:02.229 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 3344, 2024-09-12T02:18:02.229 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 197984 2024-09-12T02:18:02.229 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:18:02.229 INFO:teuthology.orchestra.run.smithi016.stdout: "cct": { 2024-09-12T02:18:02.229 INFO:teuthology.orchestra.run.smithi016.stdout: "total_workers": 0, 2024-09-12T02:18:02.229 INFO:teuthology.orchestra.run.smithi016.stdout: "unhealthy_workers": 0 2024-09-12T02:18:02.230 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:18:02.230 INFO:teuthology.orchestra.run.smithi016.stdout: "mempool": { 2024-09-12T02:18:02.230 INFO:teuthology.orchestra.run.smithi016.stdout: "bloom_filter_bytes": 0, 2024-09-12T02:18:02.230 INFO:teuthology.orchestra.run.smithi016.stdout: "bloom_filter_items": 0, 2024-09-12T02:18:02.230 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T02:18:02.230 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_alloc_items": 0, 2024-09-12T02:18:02.230 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T02:18:02.230 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_data_items": 0, 2024-09-12T02:18:02.230 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T02:18:02.230 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T02:18:02.230 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T02:18:02.230 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T02:18:02.230 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T02:18:02.231 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_other_items": 0, 2024-09-12T02:18:02.231 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T02:18:02.231 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T02:18:02.231 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_extent_bytes": 0, 2024-09-12T02:18:02.231 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_extent_items": 0, 2024-09-12T02:18:02.231 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_blob_bytes": 0, 2024-09-12T02:18:02.231 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_blob_items": 0, 2024-09-12T02:18:02.231 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T02:18:02.231 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T02:18:02.231 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T02:18:02.231 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T02:18:02.231 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T02:18:02.231 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_fsck_items": 0, 2024-09-12T02:18:02.232 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_txc_bytes": 0, 2024-09-12T02:18:02.232 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_txc_items": 0, 2024-09-12T02:18:02.232 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T02:18:02.232 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T02:18:02.232 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_bytes": 0, 2024-09-12T02:18:02.232 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_items": 0, 2024-09-12T02:18:02.232 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_bytes": 0, 2024-09-12T02:18:02.232 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_items": 0, 2024-09-12T02:18:02.232 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T02:18:02.232 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_reader_items": 0, 2024-09-12T02:18:02.232 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T02:18:02.232 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_writer_items": 0, 2024-09-12T02:18:02.233 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_anon_bytes": 148721, 2024-09-12T02:18:02.233 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_anon_items": 30, 2024-09-12T02:18:02.233 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_meta_bytes": 0, 2024-09-12T02:18:02.233 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_meta_items": 0, 2024-09-12T02:18:02.233 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_bytes": 0, 2024-09-12T02:18:02.233 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_items": 0, 2024-09-12T02:18:02.233 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_mapbl_bytes": 0, 2024-09-12T02:18:02.233 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_mapbl_items": 0, 2024-09-12T02:18:02.233 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_pglog_bytes": 0, 2024-09-12T02:18:02.233 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_pglog_items": 0, 2024-09-12T02:18:02.233 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_bytes": 0, 2024-09-12T02:18:02.233 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_items": 0, 2024-09-12T02:18:02.233 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T02:18:02.234 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_mapping_items": 0, 2024-09-12T02:18:02.234 INFO:teuthology.orchestra.run.smithi016.stdout: "pgmap_bytes": 0, 2024-09-12T02:18:02.234 INFO:teuthology.orchestra.run.smithi016.stdout: "pgmap_items": 0, 2024-09-12T02:18:02.234 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_co_bytes": 928, 2024-09-12T02:18:02.234 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_co_items": 2, 2024-09-12T02:18:02.234 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_1_bytes": 0, 2024-09-12T02:18:02.234 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_1_items": 0, 2024-09-12T02:18:02.234 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_2_bytes": 0, 2024-09-12T02:18:02.234 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_2_items": 0 2024-09-12T02:18:02.234 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T02:18:02.234 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T02:18:02.234 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-09-12T02:18:02.234 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 104857600, 2024-09-12T02:18:02.235 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-09-12T02:18:02.235 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 304, 2024-09-12T02:18:02.235 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 459589, 2024-09-12T02:18:02.235 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-09-12T02:18:02.235 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 304, 2024-09-12T02:18:02.235 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 0, 2024-09-12T02:18:02.235 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 0, 2024-09-12T02:18:02.235 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 304, 2024-09-12T02:18:02.235 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 459589, 2024-09-12T02:18:02.235 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-09-12T02:18:02.235 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-12T02:18:02.235 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-12T02:18:02.235 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-12T02:18:02.236 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:18:02.236 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T02:18:02.236 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T02:18:02.971 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-09-12T02:18:02.971 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-09-12T02:18:02.971 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2024-09-12T02:18:02.971 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2024-09-12T02:18:02.971 DEBUG:tasks.check_counter:Getting stats from b 2024-09-12T02:18:02.972 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph daemon mds.b perf dump 2024-09-12T02:18:03.236 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T02:18:03.820 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-12T02:18:03.820 INFO:teuthology.orchestra.run.smithi033.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T02:18:03.820 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_messages": 5349, 2024-09-12T02:18:03.820 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages": 6020, 2024-09-12T02:18:03.820 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_bytes": 1325976, 2024-09-12T02:18:03.820 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_bytes": 10020626, 2024-09-12T02:18:03.820 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_created_connections": 10, 2024-09-12T02:18:03.820 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_active_connections": 4, 2024-09-12T02:18:03.820 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_total_time": 0.487247600, 2024-09-12T02:18:03.821 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_send_time": 0.178645739, 2024-09-12T02:18:03.821 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_recv_time": 0.163654074, 2024-09-12T02:18:03.821 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_fast_dispatch_time": 0.100966615, 2024-09-12T02:18:03.821 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T02:18:03.821 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 6022, 2024-09-12T02:18:03.821 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.327198533, 2024-09-12T02:18:03.821 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000054333 2024-09-12T02:18:03.821 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-12T02:18:03.821 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_handle_ack_lat": { 2024-09-12T02:18:03.821 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 3209, 2024-09-12T02:18:03.821 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000873934, 2024-09-12T02:18:03.821 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000272 2024-09-12T02:18:03.821 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-12T02:18:03.822 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-09-12T02:18:03.822 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_encrypted_bytes": 448 2024-09-12T02:18:03.822 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-12T02:18:03.822 INFO:teuthology.orchestra.run.smithi033.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T02:18:03.822 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_messages": 1885, 2024-09-12T02:18:03.822 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages": 2774, 2024-09-12T02:18:03.822 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_bytes": 415875, 2024-09-12T02:18:03.822 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_bytes": 1376032, 2024-09-12T02:18:03.822 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_created_connections": 9, 2024-09-12T02:18:03.822 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_active_connections": 5, 2024-09-12T02:18:03.822 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_total_time": 0.178969762, 2024-09-12T02:18:03.822 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_send_time": 0.075969602, 2024-09-12T02:18:03.823 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_recv_time": 0.096061432, 2024-09-12T02:18:03.823 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_fast_dispatch_time": 0.015630330, 2024-09-12T02:18:03.823 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T02:18:03.823 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 2775, 2024-09-12T02:18:03.823 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.179851012, 2024-09-12T02:18:03.823 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000064811 2024-09-12T02:18:03.823 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-12T02:18:03.823 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_handle_ack_lat": { 2024-09-12T02:18:03.823 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 3186, 2024-09-12T02:18:03.823 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000744810, 2024-09-12T02:18:03.823 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000233 2024-09-12T02:18:03.823 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-12T02:18:03.823 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-12T02:18:03.823 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_encrypted_bytes": 571840 2024-09-12T02:18:03.824 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-12T02:18:03.824 INFO:teuthology.orchestra.run.smithi033.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T02:18:03.824 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_messages": 3968, 2024-09-12T02:18:03.824 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages": 5375, 2024-09-12T02:18:03.824 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_bytes": 1838287, 2024-09-12T02:18:03.824 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_bytes": 7738168, 2024-09-12T02:18:03.824 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_created_connections": 461, 2024-09-12T02:18:03.824 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_active_connections": 9, 2024-09-12T02:18:03.824 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_total_time": 0.645693479, 2024-09-12T02:18:03.824 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_send_time": 0.194803360, 2024-09-12T02:18:03.824 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_recv_time": 0.371518092, 2024-09-12T02:18:03.824 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_fast_dispatch_time": 0.083388093, 2024-09-12T02:18:03.824 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T02:18:03.825 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 5368, 2024-09-12T02:18:03.825 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.196221661, 2024-09-12T02:18:03.825 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000036553 2024-09-12T02:18:03.825 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-12T02:18:03.825 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_handle_ack_lat": { 2024-09-12T02:18:03.825 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 5762, 2024-09-12T02:18:03.825 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.001107980, 2024-09-12T02:18:03.825 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000192 2024-09-12T02:18:03.825 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-12T02:18:03.825 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_encrypted_bytes": 1604464, 2024-09-12T02:18:03.825 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_encrypted_bytes": 1817456 2024-09-12T02:18:03.825 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-12T02:18:03.825 INFO:teuthology.orchestra.run.smithi033.stdout: "cct": { 2024-09-12T02:18:03.826 INFO:teuthology.orchestra.run.smithi033.stdout: "total_workers": 1, 2024-09-12T02:18:03.826 INFO:teuthology.orchestra.run.smithi033.stdout: "unhealthy_workers": 0 2024-09-12T02:18:03.826 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-12T02:18:03.826 INFO:teuthology.orchestra.run.smithi033.stdout: "finisher-MDSRank": { 2024-09-12T02:18:03.826 INFO:teuthology.orchestra.run.smithi033.stdout: "queue_len": 0, 2024-09-12T02:18:03.826 INFO:teuthology.orchestra.run.smithi033.stdout: "complete_latency": { 2024-09-12T02:18:03.826 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 6889, 2024-09-12T02:18:03.826 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 1.704572734, 2024-09-12T02:18:03.826 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000247433 2024-09-12T02:18:03.826 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-12T02:18:03.826 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-12T02:18:03.826 INFO:teuthology.orchestra.run.smithi033.stdout: "finisher-PurgeQueue": { 2024-09-12T02:18:03.826 INFO:teuthology.orchestra.run.smithi033.stdout: "queue_len": 0, 2024-09-12T02:18:03.827 INFO:teuthology.orchestra.run.smithi033.stdout: "complete_latency": { 2024-09-12T02:18:03.827 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 22, 2024-09-12T02:18:03.827 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000576488, 2024-09-12T02:18:03.827 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000026204 2024-09-12T02:18:03.827 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-12T02:18:03.827 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-12T02:18:03.827 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": { 2024-09-12T02:18:03.827 INFO:teuthology.orchestra.run.smithi033.stdout: "request": 734, 2024-09-12T02:18:03.827 INFO:teuthology.orchestra.run.smithi033.stdout: "reply": 734, 2024-09-12T02:18:03.827 INFO:teuthology.orchestra.run.smithi033.stdout: "reply_latency": { 2024-09-12T02:18:03.827 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 734, 2024-09-12T02:18:03.827 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 3.109727526, 2024-09-12T02:18:03.827 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.004236686 2024-09-12T02:18:03.827 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-12T02:18:03.828 INFO:teuthology.orchestra.run.smithi033.stdout: "slow_reply": 0, 2024-09-12T02:18:03.828 INFO:teuthology.orchestra.run.smithi033.stdout: "forward": 0, 2024-09-12T02:18:03.828 INFO:teuthology.orchestra.run.smithi033.stdout: "dir_fetch_complete": 0, 2024-09-12T02:18:03.828 INFO:teuthology.orchestra.run.smithi033.stdout: "dir_fetch_keys": 0, 2024-09-12T02:18:03.828 INFO:teuthology.orchestra.run.smithi033.stdout: "dir_commit": 499, 2024-09-12T02:18:03.828 INFO:teuthology.orchestra.run.smithi033.stdout: "dir_split": 0, 2024-09-12T02:18:03.828 INFO:teuthology.orchestra.run.smithi033.stdout: "dir_merge": 0, 2024-09-12T02:18:03.828 INFO:teuthology.orchestra.run.smithi033.stdout: "inodes": 26, 2024-09-12T02:18:03.828 INFO:teuthology.orchestra.run.smithi033.stdout: "inodes_top": 6, 2024-09-12T02:18:03.828 INFO:teuthology.orchestra.run.smithi033.stdout: "inodes_bottom": 20, 2024-09-12T02:18:03.828 INFO:teuthology.orchestra.run.smithi033.stdout: "inodes_pin_tail": 0, 2024-09-12T02:18:03.828 INFO:teuthology.orchestra.run.smithi033.stdout: "inodes_pinned": 17, 2024-09-12T02:18:03.828 INFO:teuthology.orchestra.run.smithi033.stdout: "inodes_expired": 0, 2024-09-12T02:18:03.829 INFO:teuthology.orchestra.run.smithi033.stdout: "inodes_with_caps": 4, 2024-09-12T02:18:03.829 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": 4, 2024-09-12T02:18:03.829 INFO:teuthology.orchestra.run.smithi033.stdout: "subtrees": 2, 2024-09-12T02:18:03.829 INFO:teuthology.orchestra.run.smithi033.stdout: "traverse": 887, 2024-09-12T02:18:03.829 INFO:teuthology.orchestra.run.smithi033.stdout: "traverse_hit": 878, 2024-09-12T02:18:03.829 INFO:teuthology.orchestra.run.smithi033.stdout: "traverse_forward": 0, 2024-09-12T02:18:03.829 INFO:teuthology.orchestra.run.smithi033.stdout: "traverse_discover": 0, 2024-09-12T02:18:03.830 INFO:teuthology.orchestra.run.smithi033.stdout: "traverse_dir_fetch": 0, 2024-09-12T02:18:03.830 INFO:teuthology.orchestra.run.smithi033.stdout: "traverse_remote_ino": 0, 2024-09-12T02:18:03.830 INFO:teuthology.orchestra.run.smithi033.stdout: "traverse_lock": 0, 2024-09-12T02:18:03.830 INFO:teuthology.orchestra.run.smithi033.stdout: "load_cent": 469, 2024-09-12T02:18:03.830 INFO:teuthology.orchestra.run.smithi033.stdout: "q": 0, 2024-09-12T02:18:03.830 INFO:teuthology.orchestra.run.smithi033.stdout: "exported": 0, 2024-09-12T02:18:03.830 INFO:teuthology.orchestra.run.smithi033.stdout: "exported_inodes": 0, 2024-09-12T02:18:03.830 INFO:teuthology.orchestra.run.smithi033.stdout: "imported": 0, 2024-09-12T02:18:03.830 INFO:teuthology.orchestra.run.smithi033.stdout: "imported_inodes": 0, 2024-09-12T02:18:03.830 INFO:teuthology.orchestra.run.smithi033.stdout: "openino_dir_fetch": 0, 2024-09-12T02:18:03.830 INFO:teuthology.orchestra.run.smithi033.stdout: "openino_backtrace_fetch": 468, 2024-09-12T02:18:03.830 INFO:teuthology.orchestra.run.smithi033.stdout: "openino_peer_discover": 0, 2024-09-12T02:18:03.830 INFO:teuthology.orchestra.run.smithi033.stdout: "root_rfiles": 4, 2024-09-12T02:18:03.831 INFO:teuthology.orchestra.run.smithi033.stdout: "root_rbytes": 234, 2024-09-12T02:18:03.831 INFO:teuthology.orchestra.run.smithi033.stdout: "root_rsnaps": 0, 2024-09-12T02:18:03.831 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_backtrace_fetch": 468, 2024-09-12T02:18:03.831 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_set_tag": 0, 2024-09-12T02:18:03.831 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_backtrace_repaired": 0, 2024-09-12T02:18:03.831 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_inotable_repaired": 0, 2024-09-12T02:18:03.831 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_dir_inodes": 301, 2024-09-12T02:18:03.831 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_dir_base_inodes": 34, 2024-09-12T02:18:03.831 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_dirfrag_rstats": 267, 2024-09-12T02:18:03.831 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_file_inodes": 167, 2024-09-12T02:18:03.831 INFO:teuthology.orchestra.run.smithi033.stdout: "handle_inode_file_caps": 0, 2024-09-12T02:18:03.831 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph_cap_op_revoke": 647, 2024-09-12T02:18:03.831 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph_cap_op_grant": 668, 2024-09-12T02:18:03.831 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph_cap_op_trunc": 599, 2024-09-12T02:18:03.832 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-12T02:18:03.832 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-12T02:18:03.832 INFO:teuthology.orchestra.run.smithi033.stdout: "handle_client_caps": 730, 2024-09-12T02:18:03.832 INFO:teuthology.orchestra.run.smithi033.stdout: "handle_client_caps_dirty": 692, 2024-09-12T02:18:03.832 INFO:teuthology.orchestra.run.smithi033.stdout: "handle_client_cap_release": 2, 2024-09-12T02:18:03.832 INFO:teuthology.orchestra.run.smithi033.stdout: "process_request_cap_release": 627 2024-09-12T02:18:03.832 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-12T02:18:03.832 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_cache": { 2024-09-12T02:18:03.832 INFO:teuthology.orchestra.run.smithi033.stdout: "dir_update": 11, 2024-09-12T02:18:03.832 INFO:teuthology.orchestra.run.smithi033.stdout: "dir_update_receipt": 0, 2024-09-12T02:18:03.832 INFO:teuthology.orchestra.run.smithi033.stdout: "dir_try_discover": 0, 2024-09-12T02:18:03.832 INFO:teuthology.orchestra.run.smithi033.stdout: "dir_send_discover": 0, 2024-09-12T02:18:03.833 INFO:teuthology.orchestra.run.smithi033.stdout: "dir_handle_discover": 24, 2024-09-12T02:18:03.833 INFO:teuthology.orchestra.run.smithi033.stdout: "num_strays": 1, 2024-09-12T02:18:03.833 INFO:teuthology.orchestra.run.smithi033.stdout: "num_strays_delayed": 0, 2024-09-12T02:18:03.833 INFO:teuthology.orchestra.run.smithi033.stdout: "num_strays_enqueuing": 0, 2024-09-12T02:18:03.833 INFO:teuthology.orchestra.run.smithi033.stdout: "strays_created": 3, 2024-09-12T02:18:03.833 INFO:teuthology.orchestra.run.smithi033.stdout: "strays_enqueued": 2, 2024-09-12T02:18:03.833 INFO:teuthology.orchestra.run.smithi033.stdout: "strays_reintegrated": 0, 2024-09-12T02:18:03.833 INFO:teuthology.orchestra.run.smithi033.stdout: "strays_migrated": 0, 2024-09-12T02:18:03.833 INFO:teuthology.orchestra.run.smithi033.stdout: "num_recovering_processing": 0, 2024-09-12T02:18:03.833 INFO:teuthology.orchestra.run.smithi033.stdout: "num_recovering_enqueued": 0, 2024-09-12T02:18:03.833 INFO:teuthology.orchestra.run.smithi033.stdout: "num_recovering_prioritized": 0, 2024-09-12T02:18:03.833 INFO:teuthology.orchestra.run.smithi033.stdout: "recovery_started": 0, 2024-09-12T02:18:03.834 INFO:teuthology.orchestra.run.smithi033.stdout: "recovery_completed": 0, 2024-09-12T02:18:03.834 INFO:teuthology.orchestra.run.smithi033.stdout: "ireq_quiesce_path": 14, 2024-09-12T02:18:03.834 INFO:teuthology.orchestra.run.smithi033.stdout: "ireq_quiesce_inode": 194, 2024-09-12T02:18:03.834 INFO:teuthology.orchestra.run.smithi033.stdout: "ireq_enqueue_scrub": 34, 2024-09-12T02:18:03.834 INFO:teuthology.orchestra.run.smithi033.stdout: "ireq_exportdir": 0, 2024-09-12T02:18:03.834 INFO:teuthology.orchestra.run.smithi033.stdout: "ireq_flush": 0, 2024-09-12T02:18:03.834 INFO:teuthology.orchestra.run.smithi033.stdout: "ireq_fragmentdir": 0, 2024-09-12T02:18:03.834 INFO:teuthology.orchestra.run.smithi033.stdout: "ireq_fragstats": 0, 2024-09-12T02:18:03.834 INFO:teuthology.orchestra.run.smithi033.stdout: "ireq_inodestats": 0 2024-09-12T02:18:03.834 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-12T02:18:03.834 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_log": { 2024-09-12T02:18:03.834 INFO:teuthology.orchestra.run.smithi033.stdout: "evlrg": 0, 2024-09-12T02:18:03.834 INFO:teuthology.orchestra.run.smithi033.stdout: "evadd": 2518, 2024-09-12T02:18:03.835 INFO:teuthology.orchestra.run.smithi033.stdout: "evex": 2510, 2024-09-12T02:18:03.835 INFO:teuthology.orchestra.run.smithi033.stdout: "evtrm": 2510, 2024-09-12T02:18:03.835 INFO:teuthology.orchestra.run.smithi033.stdout: "ev": 8, 2024-09-12T02:18:03.835 INFO:teuthology.orchestra.run.smithi033.stdout: "evexg": 0, 2024-09-12T02:18:03.835 INFO:teuthology.orchestra.run.smithi033.stdout: "evexd": 45, 2024-09-12T02:18:03.835 INFO:teuthology.orchestra.run.smithi033.stdout: "segadd": 76, 2024-09-12T02:18:03.835 INFO:teuthology.orchestra.run.smithi033.stdout: "segex": 75, 2024-09-12T02:18:03.835 INFO:teuthology.orchestra.run.smithi033.stdout: "segtrm": 75, 2024-09-12T02:18:03.835 INFO:teuthology.orchestra.run.smithi033.stdout: "seg": 1, 2024-09-12T02:18:03.835 INFO:teuthology.orchestra.run.smithi033.stdout: "segmjr": 1, 2024-09-12T02:18:03.835 INFO:teuthology.orchestra.run.smithi033.stdout: "segexg": 0, 2024-09-12T02:18:03.836 INFO:teuthology.orchestra.run.smithi033.stdout: "segexd": 1, 2024-09-12T02:18:03.836 INFO:teuthology.orchestra.run.smithi033.stdout: "expos": 12238854, 2024-09-12T02:18:03.836 INFO:teuthology.orchestra.run.smithi033.stdout: "wrpos": 12254541, 2024-09-12T02:18:03.836 INFO:teuthology.orchestra.run.smithi033.stdout: "rdpos": 0, 2024-09-12T02:18:03.836 INFO:teuthology.orchestra.run.smithi033.stdout: "jlat": { 2024-09-12T02:18:03.836 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 2505, 2024-09-12T02:18:03.836 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 5.556033585, 2024-09-12T02:18:03.836 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.002217977 2024-09-12T02:18:03.836 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-12T02:18:03.836 INFO:teuthology.orchestra.run.smithi033.stdout: "replayed": 0 2024-09-12T02:18:03.836 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-12T02:18:03.837 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_mem": { 2024-09-12T02:18:03.837 INFO:teuthology.orchestra.run.smithi033.stdout: "ino": 24, 2024-09-12T02:18:03.837 INFO:teuthology.orchestra.run.smithi033.stdout: "ino+": 60, 2024-09-12T02:18:03.837 INFO:teuthology.orchestra.run.smithi033.stdout: "ino-": 36, 2024-09-12T02:18:03.837 INFO:teuthology.orchestra.run.smithi033.stdout: "dir": 18, 2024-09-12T02:18:03.837 INFO:teuthology.orchestra.run.smithi033.stdout: "dir+": 20, 2024-09-12T02:18:03.837 INFO:teuthology.orchestra.run.smithi033.stdout: "dir-": 2, 2024-09-12T02:18:03.837 INFO:teuthology.orchestra.run.smithi033.stdout: "dn": 26, 2024-09-12T02:18:03.837 INFO:teuthology.orchestra.run.smithi033.stdout: "dn+": 29, 2024-09-12T02:18:03.837 INFO:teuthology.orchestra.run.smithi033.stdout: "dn-": 3, 2024-09-12T02:18:03.837 INFO:teuthology.orchestra.run.smithi033.stdout: "cap": 4, 2024-09-12T02:18:03.837 INFO:teuthology.orchestra.run.smithi033.stdout: "cap+": 18, 2024-09-12T02:18:03.837 INFO:teuthology.orchestra.run.smithi033.stdout: "cap-": 14, 2024-09-12T02:18:03.837 INFO:teuthology.orchestra.run.smithi033.stdout: "rss": 58396, 2024-09-12T02:18:03.838 INFO:teuthology.orchestra.run.smithi033.stdout: "heap": 231712 2024-09-12T02:18:03.838 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-12T02:18:03.838 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_server": { 2024-09-12T02:18:03.838 INFO:teuthology.orchestra.run.smithi033.stdout: "dispatch_client_request": 1956, 2024-09-12T02:18:03.838 INFO:teuthology.orchestra.run.smithi033.stdout: "dispatch_server_request": 0, 2024-09-12T02:18:03.838 INFO:teuthology.orchestra.run.smithi033.stdout: "handle_client_request": 734, 2024-09-12T02:18:03.838 INFO:teuthology.orchestra.run.smithi033.stdout: "handle_client_session": 19, 2024-09-12T02:18:03.838 INFO:teuthology.orchestra.run.smithi033.stdout: "handle_peer_request": 0, 2024-09-12T02:18:03.838 INFO:teuthology.orchestra.run.smithi033.stdout: "req_create_latency": { 2024-09-12T02:18:03.838 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 18, 2024-09-12T02:18:03.838 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.011515941, 2024-09-12T02:18:03.838 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000639774 2024-09-12T02:18:03.839 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-12T02:18:03.839 INFO:teuthology.orchestra.run.smithi033.stdout: "req_getattr_latency": { 2024-09-12T02:18:03.839 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 88, 2024-09-12T02:18:03.839 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.020691719, 2024-09-12T02:18:03.839 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000235133 2024-09-12T02:18:03.839 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-12T02:18:03.839 INFO:teuthology.orchestra.run.smithi033.stdout: "req_getfilelock_latency": { 2024-09-12T02:18:03.839 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-12T02:18:03.839 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-12T02:18:03.839 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-12T02:18:03.839 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-12T02:18:03.839 INFO:teuthology.orchestra.run.smithi033.stdout: "req_link_latency": { 2024-09-12T02:18:03.839 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-12T02:18:03.839 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-12T02:18:03.840 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-12T02:18:03.840 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-12T02:18:03.840 INFO:teuthology.orchestra.run.smithi033.stdout: "req_lookup_latency": { 2024-09-12T02:18:03.840 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 3, 2024-09-12T02:18:03.840 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.001150220, 2024-09-12T02:18:03.840 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000383406 2024-09-12T02:18:03.840 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-12T02:18:03.840 INFO:teuthology.orchestra.run.smithi033.stdout: "req_lookuphash_latency": { 2024-09-12T02:18:03.840 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-12T02:18:03.840 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-12T02:18:03.840 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-12T02:18:03.840 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-12T02:18:03.840 INFO:teuthology.orchestra.run.smithi033.stdout: "req_lookupino_latency": { 2024-09-12T02:18:03.841 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 2, 2024-09-12T02:18:03.841 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000349336, 2024-09-12T02:18:03.841 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000174668 2024-09-12T02:18:03.841 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-12T02:18:03.841 INFO:teuthology.orchestra.run.smithi033.stdout: "req_lookupname_latency": { 2024-09-12T02:18:03.841 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-12T02:18:03.841 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-12T02:18:03.841 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-12T02:18:03.841 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-12T02:18:03.841 INFO:teuthology.orchestra.run.smithi033.stdout: "req_lookupparent_latency": { 2024-09-12T02:18:03.841 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-12T02:18:03.841 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-12T02:18:03.842 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-12T02:18:03.842 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-12T02:18:03.842 INFO:teuthology.orchestra.run.smithi033.stdout: "req_lookupsnap_latency": { 2024-09-12T02:18:03.842 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-12T02:18:03.842 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-12T02:18:03.842 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-12T02:18:03.842 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-12T02:18:03.842 INFO:teuthology.orchestra.run.smithi033.stdout: "req_lssnap_latency": { 2024-09-12T02:18:03.842 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-12T02:18:03.842 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-12T02:18:03.842 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-12T02:18:03.842 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-12T02:18:03.843 INFO:teuthology.orchestra.run.smithi033.stdout: "req_mkdir_latency": { 2024-09-12T02:18:03.843 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 8, 2024-09-12T02:18:03.843 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.012135904, 2024-09-12T02:18:03.843 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.001516988 2024-09-12T02:18:03.843 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-12T02:18:03.843 INFO:teuthology.orchestra.run.smithi033.stdout: "req_mknod_latency": { 2024-09-12T02:18:03.843 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-12T02:18:03.843 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-12T02:18:03.843 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-12T02:18:03.843 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-12T02:18:03.844 INFO:teuthology.orchestra.run.smithi033.stdout: "req_mksnap_latency": { 2024-09-12T02:18:03.844 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-12T02:18:03.844 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-12T02:18:03.844 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-12T02:18:03.844 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-12T02:18:03.844 INFO:teuthology.orchestra.run.smithi033.stdout: "req_open_latency": { 2024-09-12T02:18:03.844 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 1, 2024-09-12T02:18:03.844 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000136748, 2024-09-12T02:18:03.844 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000136748 2024-09-12T02:18:03.844 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-12T02:18:03.844 INFO:teuthology.orchestra.run.smithi033.stdout: "req_readdir_latency": { 2024-09-12T02:18:03.844 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 2, 2024-09-12T02:18:03.845 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.003724231, 2024-09-12T02:18:03.845 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.001862115 2024-09-12T02:18:03.845 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-12T02:18:03.845 INFO:teuthology.orchestra.run.smithi033.stdout: "req_rename_latency": { 2024-09-12T02:18:03.845 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 2, 2024-09-12T02:18:03.845 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.002060797, 2024-09-12T02:18:03.845 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.001030398 2024-09-12T02:18:03.845 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-12T02:18:03.845 INFO:teuthology.orchestra.run.smithi033.stdout: "req_renamesnap_latency": { 2024-09-12T02:18:03.845 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-12T02:18:03.845 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-12T02:18:03.845 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-12T02:18:03.845 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-12T02:18:03.846 INFO:teuthology.orchestra.run.smithi033.stdout: "req_snapdiff_latency": { 2024-09-12T02:18:03.846 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-12T02:18:03.846 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-12T02:18:03.846 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-12T02:18:03.846 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-12T02:18:03.846 INFO:teuthology.orchestra.run.smithi033.stdout: "req_rmdir_latency": { 2024-09-12T02:18:03.846 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 2, 2024-09-12T02:18:03.846 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.017490082, 2024-09-12T02:18:03.846 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.008745041 2024-09-12T02:18:03.846 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-12T02:18:03.846 INFO:teuthology.orchestra.run.smithi033.stdout: "req_rmsnap_latency": { 2024-09-12T02:18:03.846 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-12T02:18:03.846 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-12T02:18:03.847 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-12T02:18:03.847 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-12T02:18:03.847 INFO:teuthology.orchestra.run.smithi033.stdout: "req_rmxattr_latency": { 2024-09-12T02:18:03.847 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-12T02:18:03.847 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-12T02:18:03.847 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-12T02:18:03.847 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-12T02:18:03.847 INFO:teuthology.orchestra.run.smithi033.stdout: "req_setattr_latency": { 2024-09-12T02:18:03.847 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 600, 2024-09-12T02:18:03.847 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 3.018821531, 2024-09-12T02:18:03.847 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.005031369 2024-09-12T02:18:03.847 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-12T02:18:03.847 INFO:teuthology.orchestra.run.smithi033.stdout: "req_setdirlayout_latency": { 2024-09-12T02:18:03.847 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-12T02:18:03.848 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-12T02:18:03.848 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-12T02:18:03.848 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-12T02:18:03.848 INFO:teuthology.orchestra.run.smithi033.stdout: "req_setfilelock_latency": { 2024-09-12T02:18:03.848 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-12T02:18:03.848 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-12T02:18:03.848 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-12T02:18:03.848 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-12T02:18:03.848 INFO:teuthology.orchestra.run.smithi033.stdout: "req_setlayout_latency": { 2024-09-12T02:18:03.848 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-12T02:18:03.848 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-12T02:18:03.848 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-12T02:18:03.848 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-12T02:18:03.849 INFO:teuthology.orchestra.run.smithi033.stdout: "req_setxattr_latency": { 2024-09-12T02:18:03.849 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 7, 2024-09-12T02:18:03.849 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.019500344, 2024-09-12T02:18:03.849 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.002785763 2024-09-12T02:18:03.849 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-12T02:18:03.849 INFO:teuthology.orchestra.run.smithi033.stdout: "req_symlink_latency": { 2024-09-12T02:18:03.849 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-12T02:18:03.849 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-12T02:18:03.849 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-12T02:18:03.849 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-12T02:18:03.849 INFO:teuthology.orchestra.run.smithi033.stdout: "req_unlink_latency": { 2024-09-12T02:18:03.849 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 1, 2024-09-12T02:18:03.849 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.002150673, 2024-09-12T02:18:03.850 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.002150673 2024-09-12T02:18:03.850 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-12T02:18:03.850 INFO:teuthology.orchestra.run.smithi033.stdout: "cap_revoke_eviction": 0, 2024-09-12T02:18:03.850 INFO:teuthology.orchestra.run.smithi033.stdout: "cap_acquisition_throttle": 0, 2024-09-12T02:18:03.850 INFO:teuthology.orchestra.run.smithi033.stdout: "req_getvxattr_latency": { 2024-09-12T02:18:03.850 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-12T02:18:03.850 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-12T02:18:03.850 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-12T02:18:03.850 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-12T02:18:03.850 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-12T02:18:03.850 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_sessions": { 2024-09-12T02:18:03.850 INFO:teuthology.orchestra.run.smithi033.stdout: "session_count": 1, 2024-09-12T02:18:03.850 INFO:teuthology.orchestra.run.smithi033.stdout: "session_add": 2, 2024-09-12T02:18:03.851 INFO:teuthology.orchestra.run.smithi033.stdout: "session_remove": 1, 2024-09-12T02:18:03.851 INFO:teuthology.orchestra.run.smithi033.stdout: "sessions_open": 1, 2024-09-12T02:18:03.851 INFO:teuthology.orchestra.run.smithi033.stdout: "sessions_stale": 0, 2024-09-12T02:18:03.851 INFO:teuthology.orchestra.run.smithi033.stdout: "total_load": 76, 2024-09-12T02:18:03.851 INFO:teuthology.orchestra.run.smithi033.stdout: "average_load": 76, 2024-09-12T02:18:03.851 INFO:teuthology.orchestra.run.smithi033.stdout: "avg_session_uptime": 985, 2024-09-12T02:18:03.851 INFO:teuthology.orchestra.run.smithi033.stdout: "mdthresh_evicted": 0 2024-09-12T02:18:03.851 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-12T02:18:03.851 INFO:teuthology.orchestra.run.smithi033.stdout: "mempool": { 2024-09-12T02:18:03.851 INFO:teuthology.orchestra.run.smithi033.stdout: "bloom_filter_bytes": 0, 2024-09-12T02:18:03.851 INFO:teuthology.orchestra.run.smithi033.stdout: "bloom_filter_items": 0, 2024-09-12T02:18:03.851 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T02:18:03.851 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_alloc_items": 0, 2024-09-12T02:18:03.852 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T02:18:03.852 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_data_items": 0, 2024-09-12T02:18:03.852 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T02:18:03.852 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T02:18:03.852 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T02:18:03.852 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T02:18:03.852 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T02:18:03.852 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_other_items": 0, 2024-09-12T02:18:03.852 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T02:18:03.852 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T02:18:03.852 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_extent_bytes": 0, 2024-09-12T02:18:03.852 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_extent_items": 0, 2024-09-12T02:18:03.852 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_blob_bytes": 0, 2024-09-12T02:18:03.853 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_blob_items": 0, 2024-09-12T02:18:03.853 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T02:18:03.853 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T02:18:03.853 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T02:18:03.853 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T02:18:03.853 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T02:18:03.853 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_fsck_items": 0, 2024-09-12T02:18:03.853 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_txc_bytes": 0, 2024-09-12T02:18:03.853 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_txc_items": 0, 2024-09-12T02:18:03.853 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T02:18:03.853 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T02:18:03.853 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_writing_bytes": 0, 2024-09-12T02:18:03.854 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_writing_items": 0, 2024-09-12T02:18:03.854 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_bytes": 0, 2024-09-12T02:18:03.854 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_items": 0, 2024-09-12T02:18:03.854 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T02:18:03.854 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_file_reader_items": 0, 2024-09-12T02:18:03.854 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T02:18:03.854 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_file_writer_items": 0, 2024-09-12T02:18:03.854 INFO:teuthology.orchestra.run.smithi033.stdout: "buffer_anon_bytes": 287340, 2024-09-12T02:18:03.854 INFO:teuthology.orchestra.run.smithi033.stdout: "buffer_anon_items": 97, 2024-09-12T02:18:03.854 INFO:teuthology.orchestra.run.smithi033.stdout: "buffer_meta_bytes": 0, 2024-09-12T02:18:03.854 INFO:teuthology.orchestra.run.smithi033.stdout: "buffer_meta_items": 0, 2024-09-12T02:18:03.854 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_bytes": 0, 2024-09-12T02:18:03.854 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_items": 0, 2024-09-12T02:18:03.855 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_mapbl_bytes": 0, 2024-09-12T02:18:03.855 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_mapbl_items": 0, 2024-09-12T02:18:03.855 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_pglog_bytes": 0, 2024-09-12T02:18:03.855 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_pglog_items": 0, 2024-09-12T02:18:03.855 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_bytes": 7304, 2024-09-12T02:18:03.855 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_items": 150, 2024-09-12T02:18:03.855 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T02:18:03.855 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_mapping_items": 0, 2024-09-12T02:18:03.855 INFO:teuthology.orchestra.run.smithi033.stdout: "pgmap_bytes": 0, 2024-09-12T02:18:03.855 INFO:teuthology.orchestra.run.smithi033.stdout: "pgmap_items": 0, 2024-09-12T02:18:03.855 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_co_bytes": 100416, 2024-09-12T02:18:03.855 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_co_items": 792, 2024-09-12T02:18:03.855 INFO:teuthology.orchestra.run.smithi033.stdout: "unittest_1_bytes": 0, 2024-09-12T02:18:03.855 INFO:teuthology.orchestra.run.smithi033.stdout: "unittest_1_items": 0, 2024-09-12T02:18:03.856 INFO:teuthology.orchestra.run.smithi033.stdout: "unittest_2_bytes": 0, 2024-09-12T02:18:03.856 INFO:teuthology.orchestra.run.smithi033.stdout: "unittest_2_items": 0 2024-09-12T02:18:03.856 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-12T02:18:03.856 INFO:teuthology.orchestra.run.smithi033.stdout: "objecter": { 2024-09-12T02:18:03.856 INFO:teuthology.orchestra.run.smithi033.stdout: "op_active": 0, 2024-09-12T02:18:03.856 INFO:teuthology.orchestra.run.smithi033.stdout: "op_laggy": 0, 2024-09-12T02:18:03.856 INFO:teuthology.orchestra.run.smithi033.stdout: "op_send": 5077, 2024-09-12T02:18:03.856 INFO:teuthology.orchestra.run.smithi033.stdout: "op_send_bytes": 8569438, 2024-09-12T02:18:03.856 INFO:teuthology.orchestra.run.smithi033.stdout: "op_resend": 37, 2024-09-12T02:18:03.856 INFO:teuthology.orchestra.run.smithi033.stdout: "op_reply": 5077, 2024-09-12T02:18:03.856 INFO:teuthology.orchestra.run.smithi033.stdout: "op_latency": { 2024-09-12T02:18:03.856 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 5077, 2024-09-12T02:18:03.857 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 27.095546946, 2024-09-12T02:18:03.857 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.005336920 2024-09-12T02:18:03.857 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-12T02:18:03.857 INFO:teuthology.orchestra.run.smithi033.stdout: "op_inflight": 0, 2024-09-12T02:18:03.857 INFO:teuthology.orchestra.run.smithi033.stdout: "oplen_avg": { 2024-09-12T02:18:03.857 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 5077, 2024-09-12T02:18:03.857 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 6114 2024-09-12T02:18:03.857 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-12T02:18:03.857 INFO:teuthology.orchestra.run.smithi033.stdout: "op": 5077, 2024-09-12T02:18:03.857 INFO:teuthology.orchestra.run.smithi033.stdout: "op_r": 538, 2024-09-12T02:18:03.857 INFO:teuthology.orchestra.run.smithi033.stdout: "op_w": 4539, 2024-09-12T02:18:03.857 INFO:teuthology.orchestra.run.smithi033.stdout: "op_rmw": 0, 2024-09-12T02:18:03.857 INFO:teuthology.orchestra.run.smithi033.stdout: "op_pg": 0, 2024-09-12T02:18:03.858 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_stat": 479, 2024-09-12T02:18:03.858 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_create": 28, 2024-09-12T02:18:03.858 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_read": 36, 2024-09-12T02:18:03.858 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_write": 2507, 2024-09-12T02:18:03.858 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_writefull": 235, 2024-09-12T02:18:03.858 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_writesame": 0, 2024-09-12T02:18:03.858 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_append": 0, 2024-09-12T02:18:03.858 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_zero": 0, 2024-09-12T02:18:03.858 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_truncate": 0, 2024-09-12T02:18:03.858 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_delete": 516, 2024-09-12T02:18:03.858 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_mapext": 0, 2024-09-12T02:18:03.858 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_sparse_read": 0, 2024-09-12T02:18:03.858 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_clonerange": 0, 2024-09-12T02:18:03.859 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_getxattr": 502, 2024-09-12T02:18:03.859 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_setxattr": 56, 2024-09-12T02:18:03.859 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_cmpxattr": 0, 2024-09-12T02:18:03.859 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_rmxattr": 0, 2024-09-12T02:18:03.859 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_resetxattrs": 0, 2024-09-12T02:18:03.859 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_call": 0, 2024-09-12T02:18:03.859 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_watch": 0, 2024-09-12T02:18:03.859 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_notify": 0, 2024-09-12T02:18:03.859 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_src_cmpxattr": 0, 2024-09-12T02:18:03.859 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_pgls": 0, 2024-09-12T02:18:03.859 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_pgls_filter": 0, 2024-09-12T02:18:03.859 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_other": 599, 2024-09-12T02:18:03.859 INFO:teuthology.orchestra.run.smithi033.stdout: "linger_active": 0, 2024-09-12T02:18:03.859 INFO:teuthology.orchestra.run.smithi033.stdout: "linger_send": 0, 2024-09-12T02:18:03.860 INFO:teuthology.orchestra.run.smithi033.stdout: "linger_resend": 0, 2024-09-12T02:18:03.860 INFO:teuthology.orchestra.run.smithi033.stdout: "linger_ping": 0, 2024-09-12T02:18:03.860 INFO:teuthology.orchestra.run.smithi033.stdout: "poolop_active": 0, 2024-09-12T02:18:03.860 INFO:teuthology.orchestra.run.smithi033.stdout: "poolop_send": 0, 2024-09-12T02:18:03.860 INFO:teuthology.orchestra.run.smithi033.stdout: "poolop_resend": 0, 2024-09-12T02:18:03.860 INFO:teuthology.orchestra.run.smithi033.stdout: "poolstat_active": 0, 2024-09-12T02:18:03.860 INFO:teuthology.orchestra.run.smithi033.stdout: "poolstat_send": 0, 2024-09-12T02:18:03.860 INFO:teuthology.orchestra.run.smithi033.stdout: "poolstat_resend": 0, 2024-09-12T02:18:03.860 INFO:teuthology.orchestra.run.smithi033.stdout: "statfs_active": 0, 2024-09-12T02:18:03.860 INFO:teuthology.orchestra.run.smithi033.stdout: "statfs_send": 0, 2024-09-12T02:18:03.860 INFO:teuthology.orchestra.run.smithi033.stdout: "statfs_resend": 0, 2024-09-12T02:18:03.860 INFO:teuthology.orchestra.run.smithi033.stdout: "command_active": 0, 2024-09-12T02:18:03.860 INFO:teuthology.orchestra.run.smithi033.stdout: "command_send": 0, 2024-09-12T02:18:03.861 INFO:teuthology.orchestra.run.smithi033.stdout: "command_resend": 0, 2024-09-12T02:18:03.861 INFO:teuthology.orchestra.run.smithi033.stdout: "map_epoch": 82, 2024-09-12T02:18:03.861 INFO:teuthology.orchestra.run.smithi033.stdout: "map_full": 0, 2024-09-12T02:18:03.861 INFO:teuthology.orchestra.run.smithi033.stdout: "map_inc": 2, 2024-09-12T02:18:03.861 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_sessions": 12, 2024-09-12T02:18:03.861 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_session_open": 12, 2024-09-12T02:18:03.861 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_session_close": 0, 2024-09-12T02:18:03.861 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_laggy": 0, 2024-09-12T02:18:03.861 INFO:teuthology.orchestra.run.smithi033.stdout: "omap_wr": 1145, 2024-09-12T02:18:03.861 INFO:teuthology.orchestra.run.smithi033.stdout: "omap_rd": 0, 2024-09-12T02:18:03.861 INFO:teuthology.orchestra.run.smithi033.stdout: "omap_del": 11 2024-09-12T02:18:03.861 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-12T02:18:03.861 INFO:teuthology.orchestra.run.smithi033.stdout: "oft": { 2024-09-12T02:18:03.862 INFO:teuthology.orchestra.run.smithi033.stdout: "omap_total_objs": 1, 2024-09-12T02:18:03.862 INFO:teuthology.orchestra.run.smithi033.stdout: "omap_total_kv_pairs": 0, 2024-09-12T02:18:03.862 INFO:teuthology.orchestra.run.smithi033.stdout: "omap_total_updates": 0, 2024-09-12T02:18:03.862 INFO:teuthology.orchestra.run.smithi033.stdout: "omap_total_removes": 0 2024-09-12T02:18:03.862 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-12T02:18:03.862 INFO:teuthology.orchestra.run.smithi033.stdout: "osd-slow-ops": { 2024-09-12T02:18:03.862 INFO:teuthology.orchestra.run.smithi033.stdout: "slow_ops_count": 208 2024-09-12T02:18:03.862 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-12T02:18:03.862 INFO:teuthology.orchestra.run.smithi033.stdout: "purge_queue": { 2024-09-12T02:18:03.862 INFO:teuthology.orchestra.run.smithi033.stdout: "pq_executing_ops": 0, 2024-09-12T02:18:03.862 INFO:teuthology.orchestra.run.smithi033.stdout: "pq_executing_ops_high_water": 1, 2024-09-12T02:18:03.862 INFO:teuthology.orchestra.run.smithi033.stdout: "pq_executing": 0, 2024-09-12T02:18:03.863 INFO:teuthology.orchestra.run.smithi033.stdout: "pq_executing_high_water": 1, 2024-09-12T02:18:03.863 INFO:teuthology.orchestra.run.smithi033.stdout: "pq_executed_ops": 2, 2024-09-12T02:18:03.863 INFO:teuthology.orchestra.run.smithi033.stdout: "pq_executed": 2, 2024-09-12T02:18:03.863 INFO:teuthology.orchestra.run.smithi033.stdout: "pq_item_in_journal": 0 2024-09-12T02:18:03.863 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-12T02:18:03.863 INFO:teuthology.orchestra.run.smithi033.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T02:18:03.863 INFO:teuthology.orchestra.run.smithi033.stdout: "val": 0, 2024-09-12T02:18:03.863 INFO:teuthology.orchestra.run.smithi033.stdout: "max": 104857600, 2024-09-12T02:18:03.863 INFO:teuthology.orchestra.run.smithi033.stdout: "get_started": 0, 2024-09-12T02:18:03.863 INFO:teuthology.orchestra.run.smithi033.stdout: "get": 11198, 2024-09-12T02:18:03.863 INFO:teuthology.orchestra.run.smithi033.stdout: "get_sum": 2432738, 2024-09-12T02:18:03.863 INFO:teuthology.orchestra.run.smithi033.stdout: "get_or_fail_fail": 0, 2024-09-12T02:18:03.863 INFO:teuthology.orchestra.run.smithi033.stdout: "get_or_fail_success": 11198, 2024-09-12T02:18:03.864 INFO:teuthology.orchestra.run.smithi033.stdout: "take": 0, 2024-09-12T02:18:03.864 INFO:teuthology.orchestra.run.smithi033.stdout: "take_sum": 0, 2024-09-12T02:18:03.864 INFO:teuthology.orchestra.run.smithi033.stdout: "put": 11198, 2024-09-12T02:18:03.864 INFO:teuthology.orchestra.run.smithi033.stdout: "put_sum": 2432738, 2024-09-12T02:18:03.864 INFO:teuthology.orchestra.run.smithi033.stdout: "wait": { 2024-09-12T02:18:03.864 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-12T02:18:03.864 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-12T02:18:03.864 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-12T02:18:03.864 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-12T02:18:03.864 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-12T02:18:03.864 INFO:teuthology.orchestra.run.smithi033.stdout: "throttle-objecter_bytes": { 2024-09-12T02:18:03.864 INFO:teuthology.orchestra.run.smithi033.stdout: "val": 0, 2024-09-12T02:18:03.864 INFO:teuthology.orchestra.run.smithi033.stdout: "max": 104857600, 2024-09-12T02:18:03.864 INFO:teuthology.orchestra.run.smithi033.stdout: "get_started": 0, 2024-09-12T02:18:03.865 INFO:teuthology.orchestra.run.smithi033.stdout: "get": 0, 2024-09-12T02:18:03.865 INFO:teuthology.orchestra.run.smithi033.stdout: "get_sum": 0, 2024-09-12T02:18:03.865 INFO:teuthology.orchestra.run.smithi033.stdout: "get_or_fail_fail": 0, 2024-09-12T02:18:03.865 INFO:teuthology.orchestra.run.smithi033.stdout: "get_or_fail_success": 0, 2024-09-12T02:18:03.865 INFO:teuthology.orchestra.run.smithi033.stdout: "take": 5077, 2024-09-12T02:18:03.865 INFO:teuthology.orchestra.run.smithi033.stdout: "take_sum": 8569640, 2024-09-12T02:18:03.865 INFO:teuthology.orchestra.run.smithi033.stdout: "put": 5077, 2024-09-12T02:18:03.865 INFO:teuthology.orchestra.run.smithi033.stdout: "put_sum": 8569640, 2024-09-12T02:18:03.865 INFO:teuthology.orchestra.run.smithi033.stdout: "wait": { 2024-09-12T02:18:03.865 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-12T02:18:03.865 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-12T02:18:03.865 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-12T02:18:03.866 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-12T02:18:03.866 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-12T02:18:03.866 INFO:teuthology.orchestra.run.smithi033.stdout: "throttle-objecter_ops": { 2024-09-12T02:18:03.866 INFO:teuthology.orchestra.run.smithi033.stdout: "val": 0, 2024-09-12T02:18:03.866 INFO:teuthology.orchestra.run.smithi033.stdout: "max": 1024, 2024-09-12T02:18:03.866 INFO:teuthology.orchestra.run.smithi033.stdout: "get_started": 0, 2024-09-12T02:18:03.866 INFO:teuthology.orchestra.run.smithi033.stdout: "get": 0, 2024-09-12T02:18:03.866 INFO:teuthology.orchestra.run.smithi033.stdout: "get_sum": 0, 2024-09-12T02:18:03.866 INFO:teuthology.orchestra.run.smithi033.stdout: "get_or_fail_fail": 0, 2024-09-12T02:18:03.866 INFO:teuthology.orchestra.run.smithi033.stdout: "get_or_fail_success": 0, 2024-09-12T02:18:03.866 INFO:teuthology.orchestra.run.smithi033.stdout: "take": 5077, 2024-09-12T02:18:03.866 INFO:teuthology.orchestra.run.smithi033.stdout: "take_sum": 5077, 2024-09-12T02:18:03.866 INFO:teuthology.orchestra.run.smithi033.stdout: "put": 5077, 2024-09-12T02:18:03.866 INFO:teuthology.orchestra.run.smithi033.stdout: "put_sum": 5077, 2024-09-12T02:18:03.867 INFO:teuthology.orchestra.run.smithi033.stdout: "wait": { 2024-09-12T02:18:03.867 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-12T02:18:03.867 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-12T02:18:03.867 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-12T02:18:03.867 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-12T02:18:03.867 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-12T02:18:03.867 INFO:teuthology.orchestra.run.smithi033.stdout: "throttle-write_buf_throttle": { 2024-09-12T02:18:03.867 INFO:teuthology.orchestra.run.smithi033.stdout: "val": 0, 2024-09-12T02:18:03.867 INFO:teuthology.orchestra.run.smithi033.stdout: "max": 3758096384, 2024-09-12T02:18:03.867 INFO:teuthology.orchestra.run.smithi033.stdout: "get_started": 0, 2024-09-12T02:18:03.867 INFO:teuthology.orchestra.run.smithi033.stdout: "get": 2, 2024-09-12T02:18:03.867 INFO:teuthology.orchestra.run.smithi033.stdout: "get_sum": 202, 2024-09-12T02:18:03.867 INFO:teuthology.orchestra.run.smithi033.stdout: "get_or_fail_fail": 0, 2024-09-12T02:18:03.868 INFO:teuthology.orchestra.run.smithi033.stdout: "get_or_fail_success": 2, 2024-09-12T02:18:03.868 INFO:teuthology.orchestra.run.smithi033.stdout: "take": 0, 2024-09-12T02:18:03.868 INFO:teuthology.orchestra.run.smithi033.stdout: "take_sum": 0, 2024-09-12T02:18:03.868 INFO:teuthology.orchestra.run.smithi033.stdout: "put": 2, 2024-09-12T02:18:03.868 INFO:teuthology.orchestra.run.smithi033.stdout: "put_sum": 202, 2024-09-12T02:18:03.868 INFO:teuthology.orchestra.run.smithi033.stdout: "wait": { 2024-09-12T02:18:03.868 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-12T02:18:03.868 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-12T02:18:03.868 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-12T02:18:03.868 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-12T02:18:03.868 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-12T02:18:03.868 INFO:teuthology.orchestra.run.smithi033.stdout: "throttle-write_buf_throttle-0x558aa2f5c960": { 2024-09-12T02:18:03.868 INFO:teuthology.orchestra.run.smithi033.stdout: "val": 0, 2024-09-12T02:18:03.869 INFO:teuthology.orchestra.run.smithi033.stdout: "max": 3758096384, 2024-09-12T02:18:03.869 INFO:teuthology.orchestra.run.smithi033.stdout: "get_started": 0, 2024-09-12T02:18:03.869 INFO:teuthology.orchestra.run.smithi033.stdout: "get": 2518, 2024-09-12T02:18:03.869 INFO:teuthology.orchestra.run.smithi033.stdout: "get_sum": 8060237, 2024-09-12T02:18:03.869 INFO:teuthology.orchestra.run.smithi033.stdout: "get_or_fail_fail": 0, 2024-09-12T02:18:03.869 INFO:teuthology.orchestra.run.smithi033.stdout: "get_or_fail_success": 2518, 2024-09-12T02:18:03.869 INFO:teuthology.orchestra.run.smithi033.stdout: "take": 0, 2024-09-12T02:18:03.869 INFO:teuthology.orchestra.run.smithi033.stdout: "take_sum": 0, 2024-09-12T02:18:03.870 INFO:teuthology.orchestra.run.smithi033.stdout: "put": 2505, 2024-09-12T02:18:03.870 INFO:teuthology.orchestra.run.smithi033.stdout: "put_sum": 8060237, 2024-09-12T02:18:03.870 INFO:teuthology.orchestra.run.smithi033.stdout: "wait": { 2024-09-12T02:18:03.870 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-12T02:18:03.870 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-12T02:18:03.870 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-12T02:18:03.870 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-12T02:18:03.870 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-12T02:18:03.870 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-12T02:18:03.919 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:18:03 smithi033 ceph-mon[35332]: pgmap v819: 97 pgs: 97 active+clean; 4.3 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-12T02:18:04.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:18:03 smithi184 ceph-mon[35593]: pgmap v819: 97 pgs: 97 active+clean; 4.3 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-12T02:18:04.183 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:18:03 smithi016 ceph-mon[30947]: pgmap v819: 97 pgs: 97 active+clean; 4.3 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-12T02:18:04.527 INFO:tasks.check_counter:Daemon mds.b mds.exported=0 2024-09-12T02:18:04.527 INFO:tasks.check_counter:Daemon mds.b mds.imported=0 2024-09-12T02:18:04.527 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=11 2024-09-12T02:18:04.527 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=0 2024-09-12T02:18:04.527 DEBUG:tasks.check_counter:Getting stats from e 2024-09-12T02:18:04.527 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph daemon mds.e perf dump 2024-09-12T02:18:04.783 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T02:18:05.254 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-12T02:18:05.255 INFO:teuthology.orchestra.run.smithi033.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T02:18:05.255 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_messages": 0, 2024-09-12T02:18:05.255 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages": 2, 2024-09-12T02:18:05.255 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_bytes": 0, 2024-09-12T02:18:05.255 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_bytes": 0, 2024-09-12T02:18:05.255 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_created_connections": 4, 2024-09-12T02:18:05.255 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_active_connections": 0, 2024-09-12T02:18:05.255 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_total_time": 0.001016741, 2024-09-12T02:18:05.255 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_send_time": 0.000000000, 2024-09-12T02:18:05.255 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_recv_time": 0.000102138, 2024-09-12T02:18:05.255 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T02:18:05.255 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T02:18:05.256 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-12T02:18:05.256 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-12T02:18:05.256 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-12T02:18:05.256 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-12T02:18:05.256 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_handle_ack_lat": { 2024-09-12T02:18:05.256 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-12T02:18:05.256 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-12T02:18:05.256 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-12T02:18:05.256 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-12T02:18:05.256 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-12T02:18:05.256 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_encrypted_bytes": 0 2024-09-12T02:18:05.256 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-12T02:18:05.256 INFO:teuthology.orchestra.run.smithi033.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T02:18:05.257 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_messages": 2, 2024-09-12T02:18:05.257 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages": 228, 2024-09-12T02:18:05.257 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_bytes": 320, 2024-09-12T02:18:05.257 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_bytes": 197536, 2024-09-12T02:18:05.257 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_created_connections": 4, 2024-09-12T02:18:05.257 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_active_connections": 2, 2024-09-12T02:18:05.257 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_total_time": 0.015355721, 2024-09-12T02:18:05.257 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_send_time": 0.011595589, 2024-09-12T02:18:05.257 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_recv_time": 0.001128564, 2024-09-12T02:18:05.257 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T02:18:05.257 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T02:18:05.257 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 226, 2024-09-12T02:18:05.257 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.013653238, 2024-09-12T02:18:05.258 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000060412 2024-09-12T02:18:05.258 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-12T02:18:05.258 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_handle_ack_lat": { 2024-09-12T02:18:05.258 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-12T02:18:05.258 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-12T02:18:05.258 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-12T02:18:05.258 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-12T02:18:05.258 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-12T02:18:05.258 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_encrypted_bytes": 197536 2024-09-12T02:18:05.258 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-12T02:18:05.258 INFO:teuthology.orchestra.run.smithi033.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T02:18:05.258 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_messages": 306, 2024-09-12T02:18:05.258 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages": 301, 2024-09-12T02:18:05.259 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_bytes": 505280, 2024-09-12T02:18:05.259 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_bytes": 184688, 2024-09-12T02:18:05.259 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_created_connections": 2, 2024-09-12T02:18:05.259 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_active_connections": 1, 2024-09-12T02:18:05.259 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_total_time": 0.045694127, 2024-09-12T02:18:05.259 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_send_time": 0.020963555, 2024-09-12T02:18:05.259 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_recv_time": 0.030052748, 2024-09-12T02:18:05.259 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T02:18:05.259 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T02:18:05.259 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 301, 2024-09-12T02:18:05.259 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.022369202, 2024-09-12T02:18:05.259 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000074316 2024-09-12T02:18:05.260 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-12T02:18:05.260 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_handle_ack_lat": { 2024-09-12T02:18:05.260 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-12T02:18:05.260 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-12T02:18:05.260 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-12T02:18:05.260 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-12T02:18:05.260 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_encrypted_bytes": 505280, 2024-09-12T02:18:05.260 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_encrypted_bytes": 184688 2024-09-12T02:18:05.260 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-12T02:18:05.260 INFO:teuthology.orchestra.run.smithi033.stdout: "cct": { 2024-09-12T02:18:05.260 INFO:teuthology.orchestra.run.smithi033.stdout: "total_workers": 0, 2024-09-12T02:18:05.260 INFO:teuthology.orchestra.run.smithi033.stdout: "unhealthy_workers": 0 2024-09-12T02:18:05.260 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-12T02:18:05.261 INFO:teuthology.orchestra.run.smithi033.stdout: "mempool": { 2024-09-12T02:18:05.261 INFO:teuthology.orchestra.run.smithi033.stdout: "bloom_filter_bytes": 0, 2024-09-12T02:18:05.261 INFO:teuthology.orchestra.run.smithi033.stdout: "bloom_filter_items": 0, 2024-09-12T02:18:05.261 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T02:18:05.261 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_alloc_items": 0, 2024-09-12T02:18:05.261 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T02:18:05.261 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_data_items": 0, 2024-09-12T02:18:05.261 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T02:18:05.261 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T02:18:05.261 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T02:18:05.261 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T02:18:05.261 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T02:18:05.261 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_other_items": 0, 2024-09-12T02:18:05.262 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T02:18:05.262 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T02:18:05.262 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_extent_bytes": 0, 2024-09-12T02:18:05.262 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_extent_items": 0, 2024-09-12T02:18:05.262 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_blob_bytes": 0, 2024-09-12T02:18:05.262 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_blob_items": 0, 2024-09-12T02:18:05.262 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T02:18:05.262 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T02:18:05.262 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T02:18:05.262 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T02:18:05.262 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T02:18:05.262 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_fsck_items": 0, 2024-09-12T02:18:05.262 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_txc_bytes": 0, 2024-09-12T02:18:05.262 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_txc_items": 0, 2024-09-12T02:18:05.263 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T02:18:05.263 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T02:18:05.263 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_writing_bytes": 0, 2024-09-12T02:18:05.263 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_writing_items": 0, 2024-09-12T02:18:05.263 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_bytes": 0, 2024-09-12T02:18:05.263 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_items": 0, 2024-09-12T02:18:05.263 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T02:18:05.263 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_file_reader_items": 0, 2024-09-12T02:18:05.263 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T02:18:05.263 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_file_writer_items": 0, 2024-09-12T02:18:05.263 INFO:teuthology.orchestra.run.smithi033.stdout: "buffer_anon_bytes": 148660, 2024-09-12T02:18:05.263 INFO:teuthology.orchestra.run.smithi033.stdout: "buffer_anon_items": 29, 2024-09-12T02:18:05.263 INFO:teuthology.orchestra.run.smithi033.stdout: "buffer_meta_bytes": 0, 2024-09-12T02:18:05.264 INFO:teuthology.orchestra.run.smithi033.stdout: "buffer_meta_items": 0, 2024-09-12T02:18:05.264 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_bytes": 0, 2024-09-12T02:18:05.264 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_items": 0, 2024-09-12T02:18:05.264 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_mapbl_bytes": 0, 2024-09-12T02:18:05.264 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_mapbl_items": 0, 2024-09-12T02:18:05.264 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_pglog_bytes": 0, 2024-09-12T02:18:05.264 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_pglog_items": 0, 2024-09-12T02:18:05.264 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_bytes": 0, 2024-09-12T02:18:05.264 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_items": 0, 2024-09-12T02:18:05.264 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T02:18:05.264 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_mapping_items": 0, 2024-09-12T02:18:05.264 INFO:teuthology.orchestra.run.smithi033.stdout: "pgmap_bytes": 0, 2024-09-12T02:18:05.264 INFO:teuthology.orchestra.run.smithi033.stdout: "pgmap_items": 0, 2024-09-12T02:18:05.265 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_co_bytes": 928, 2024-09-12T02:18:05.265 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_co_items": 2, 2024-09-12T02:18:05.265 INFO:teuthology.orchestra.run.smithi033.stdout: "unittest_1_bytes": 0, 2024-09-12T02:18:05.265 INFO:teuthology.orchestra.run.smithi033.stdout: "unittest_1_items": 0, 2024-09-12T02:18:05.265 INFO:teuthology.orchestra.run.smithi033.stdout: "unittest_2_bytes": 0, 2024-09-12T02:18:05.265 INFO:teuthology.orchestra.run.smithi033.stdout: "unittest_2_items": 0 2024-09-12T02:18:05.265 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-12T02:18:05.265 INFO:teuthology.orchestra.run.smithi033.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T02:18:05.265 INFO:teuthology.orchestra.run.smithi033.stdout: "val": 0, 2024-09-12T02:18:05.265 INFO:teuthology.orchestra.run.smithi033.stdout: "max": 104857600, 2024-09-12T02:18:05.265 INFO:teuthology.orchestra.run.smithi033.stdout: "get_started": 0, 2024-09-12T02:18:05.265 INFO:teuthology.orchestra.run.smithi033.stdout: "get": 304, 2024-09-12T02:18:05.265 INFO:teuthology.orchestra.run.smithi033.stdout: "get_sum": 459589, 2024-09-12T02:18:05.266 INFO:teuthology.orchestra.run.smithi033.stdout: "get_or_fail_fail": 0, 2024-09-12T02:18:05.266 INFO:teuthology.orchestra.run.smithi033.stdout: "get_or_fail_success": 304, 2024-09-12T02:18:05.266 INFO:teuthology.orchestra.run.smithi033.stdout: "take": 0, 2024-09-12T02:18:05.266 INFO:teuthology.orchestra.run.smithi033.stdout: "take_sum": 0, 2024-09-12T02:18:05.266 INFO:teuthology.orchestra.run.smithi033.stdout: "put": 304, 2024-09-12T02:18:05.266 INFO:teuthology.orchestra.run.smithi033.stdout: "put_sum": 459589, 2024-09-12T02:18:05.266 INFO:teuthology.orchestra.run.smithi033.stdout: "wait": { 2024-09-12T02:18:05.266 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-12T02:18:05.266 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-12T02:18:05.266 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-12T02:18:05.266 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-12T02:18:05.266 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-12T02:18:05.266 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-12T02:18:05.799 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:18:05 smithi033 ceph-mon[35332]: pgmap v820: 97 pgs: 97 active+clean; 4.3 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 341 B/s wr, 5 op/s 2024-09-12T02:18:05.824 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.e 2024-09-12T02:18:05.824 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.e 2024-09-12T02:18:05.824 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.e 2024-09-12T02:18:05.824 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.e 2024-09-12T02:18:05.824 DEBUG:tasks.check_counter:Getting stats from h 2024-09-12T02:18:05.825 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph daemon mds.h perf dump 2024-09-12T02:18:06.073 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T02:18:06.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:18:05 smithi184 ceph-mon[35593]: pgmap v820: 97 pgs: 97 active+clean; 4.3 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 341 B/s wr, 5 op/s 2024-09-12T02:18:06.183 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:18:05 smithi016 ceph-mon[30947]: pgmap v820: 97 pgs: 97 active+clean; 4.3 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 341 B/s wr, 5 op/s 2024-09-12T02:18:06.556 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-12T02:18:06.556 INFO:teuthology.orchestra.run.smithi033.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T02:18:06.556 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_messages": 1389, 2024-09-12T02:18:06.556 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages": 1384, 2024-09-12T02:18:06.556 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_bytes": 887280, 2024-09-12T02:18:06.556 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_bytes": 527584, 2024-09-12T02:18:06.556 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_created_connections": 5, 2024-09-12T02:18:06.556 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_active_connections": 1, 2024-09-12T02:18:06.556 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_total_time": 0.166472790, 2024-09-12T02:18:06.556 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_send_time": 0.072492878, 2024-09-12T02:18:06.557 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_recv_time": 0.051896282, 2024-09-12T02:18:06.557 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_fast_dispatch_time": 0.032167083, 2024-09-12T02:18:06.557 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T02:18:06.557 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 1383, 2024-09-12T02:18:06.557 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.081767762, 2024-09-12T02:18:06.557 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000059123 2024-09-12T02:18:06.557 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-12T02:18:06.557 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_handle_ack_lat": { 2024-09-12T02:18:06.557 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-12T02:18:06.557 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-12T02:18:06.557 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-12T02:18:06.557 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-12T02:18:06.557 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_encrypted_bytes": 524400, 2024-09-12T02:18:06.558 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_encrypted_bytes": 184144 2024-09-12T02:18:06.558 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-12T02:18:06.558 INFO:teuthology.orchestra.run.smithi033.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T02:18:06.558 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_messages": 4, 2024-09-12T02:18:06.558 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages": 229, 2024-09-12T02:18:06.558 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_bytes": 876, 2024-09-12T02:18:06.558 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_bytes": 567290, 2024-09-12T02:18:06.558 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_created_connections": 6, 2024-09-12T02:18:06.558 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_active_connections": 3, 2024-09-12T02:18:06.558 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_total_time": 0.013728764, 2024-09-12T02:18:06.558 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_send_time": 0.009207892, 2024-09-12T02:18:06.558 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_recv_time": 0.002070778, 2024-09-12T02:18:06.558 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_fast_dispatch_time": 0.000059779, 2024-09-12T02:18:06.558 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T02:18:06.559 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 227, 2024-09-12T02:18:06.559 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.013056476, 2024-09-12T02:18:06.559 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000057517 2024-09-12T02:18:06.559 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-12T02:18:06.559 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_handle_ack_lat": { 2024-09-12T02:18:06.559 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-12T02:18:06.559 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-12T02:18:06.559 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-12T02:18:06.559 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-12T02:18:06.559 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-12T02:18:06.559 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_encrypted_bytes": 566592 2024-09-12T02:18:06.559 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-12T02:18:06.559 INFO:teuthology.orchestra.run.smithi033.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T02:18:06.560 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_messages": 2077, 2024-09-12T02:18:06.560 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages": 2077, 2024-09-12T02:18:06.560 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_bytes": 532029, 2024-09-12T02:18:06.560 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_bytes": 659669, 2024-09-12T02:18:06.560 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_created_connections": 5, 2024-09-12T02:18:06.560 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_active_connections": 3, 2024-09-12T02:18:06.560 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_total_time": 0.131737979, 2024-09-12T02:18:06.561 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_send_time": 0.030496090, 2024-09-12T02:18:06.561 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_recv_time": 0.038685447, 2024-09-12T02:18:06.561 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_fast_dispatch_time": 0.054294220, 2024-09-12T02:18:06.561 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T02:18:06.561 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 2076, 2024-09-12T02:18:06.561 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.062609685, 2024-09-12T02:18:06.561 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000030158 2024-09-12T02:18:06.561 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-12T02:18:06.561 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_handle_ack_lat": { 2024-09-12T02:18:06.561 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-12T02:18:06.561 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-12T02:18:06.561 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-12T02:18:06.562 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-12T02:18:06.562 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-09-12T02:18:06.562 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_encrypted_bytes": 448 2024-09-12T02:18:06.562 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-12T02:18:06.562 INFO:teuthology.orchestra.run.smithi033.stdout: "cct": { 2024-09-12T02:18:06.562 INFO:teuthology.orchestra.run.smithi033.stdout: "total_workers": 1, 2024-09-12T02:18:06.562 INFO:teuthology.orchestra.run.smithi033.stdout: "unhealthy_workers": 0 2024-09-12T02:18:06.562 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-12T02:18:06.562 INFO:teuthology.orchestra.run.smithi033.stdout: "finisher-MDSRank": { 2024-09-12T02:18:06.562 INFO:teuthology.orchestra.run.smithi033.stdout: "queue_len": 0, 2024-09-12T02:18:06.562 INFO:teuthology.orchestra.run.smithi033.stdout: "complete_latency": { 2024-09-12T02:18:06.562 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 5311, 2024-09-12T02:18:06.562 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.129990653, 2024-09-12T02:18:06.563 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000024475 2024-09-12T02:18:06.563 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-12T02:18:06.563 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-12T02:18:06.563 INFO:teuthology.orchestra.run.smithi033.stdout: "finisher-PurgeQueue": { 2024-09-12T02:18:06.563 INFO:teuthology.orchestra.run.smithi033.stdout: "queue_len": 0, 2024-09-12T02:18:06.563 INFO:teuthology.orchestra.run.smithi033.stdout: "complete_latency": { 2024-09-12T02:18:06.563 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-12T02:18:06.563 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-12T02:18:06.563 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-12T02:18:06.563 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-12T02:18:06.563 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-12T02:18:06.563 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": { 2024-09-12T02:18:06.563 INFO:teuthology.orchestra.run.smithi033.stdout: "request": 0, 2024-09-12T02:18:06.564 INFO:teuthology.orchestra.run.smithi033.stdout: "reply": 0, 2024-09-12T02:18:06.564 INFO:teuthology.orchestra.run.smithi033.stdout: "reply_latency": { 2024-09-12T02:18:06.564 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-12T02:18:06.564 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-12T02:18:06.564 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-12T02:18:06.564 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-12T02:18:06.564 INFO:teuthology.orchestra.run.smithi033.stdout: "slow_reply": 0, 2024-09-12T02:18:06.564 INFO:teuthology.orchestra.run.smithi033.stdout: "forward": 0, 2024-09-12T02:18:06.564 INFO:teuthology.orchestra.run.smithi033.stdout: "dir_fetch_complete": 0, 2024-09-12T02:18:06.564 INFO:teuthology.orchestra.run.smithi033.stdout: "dir_fetch_keys": 0, 2024-09-12T02:18:06.564 INFO:teuthology.orchestra.run.smithi033.stdout: "dir_commit": 0, 2024-09-12T02:18:06.564 INFO:teuthology.orchestra.run.smithi033.stdout: "dir_split": 0, 2024-09-12T02:18:06.564 INFO:teuthology.orchestra.run.smithi033.stdout: "dir_merge": 0, 2024-09-12T02:18:06.565 INFO:teuthology.orchestra.run.smithi033.stdout: "inodes": 0, 2024-09-12T02:18:06.565 INFO:teuthology.orchestra.run.smithi033.stdout: "inodes_top": 0, 2024-09-12T02:18:06.565 INFO:teuthology.orchestra.run.smithi033.stdout: "inodes_bottom": 0, 2024-09-12T02:18:06.565 INFO:teuthology.orchestra.run.smithi033.stdout: "inodes_pin_tail": 0, 2024-09-12T02:18:06.565 INFO:teuthology.orchestra.run.smithi033.stdout: "inodes_pinned": 0, 2024-09-12T02:18:06.565 INFO:teuthology.orchestra.run.smithi033.stdout: "inodes_expired": 0, 2024-09-12T02:18:06.565 INFO:teuthology.orchestra.run.smithi033.stdout: "inodes_with_caps": 0, 2024-09-12T02:18:06.565 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": 0, 2024-09-12T02:18:06.565 INFO:teuthology.orchestra.run.smithi033.stdout: "subtrees": 1, 2024-09-12T02:18:06.565 INFO:teuthology.orchestra.run.smithi033.stdout: "traverse": 0, 2024-09-12T02:18:06.565 INFO:teuthology.orchestra.run.smithi033.stdout: "traverse_hit": 0, 2024-09-12T02:18:06.565 INFO:teuthology.orchestra.run.smithi033.stdout: "traverse_forward": 0, 2024-09-12T02:18:06.565 INFO:teuthology.orchestra.run.smithi033.stdout: "traverse_discover": 0, 2024-09-12T02:18:06.566 INFO:teuthology.orchestra.run.smithi033.stdout: "traverse_dir_fetch": 0, 2024-09-12T02:18:06.566 INFO:teuthology.orchestra.run.smithi033.stdout: "traverse_remote_ino": 0, 2024-09-12T02:18:06.566 INFO:teuthology.orchestra.run.smithi033.stdout: "traverse_lock": 0, 2024-09-12T02:18:06.566 INFO:teuthology.orchestra.run.smithi033.stdout: "load_cent": 0, 2024-09-12T02:18:06.566 INFO:teuthology.orchestra.run.smithi033.stdout: "q": 0, 2024-09-12T02:18:06.566 INFO:teuthology.orchestra.run.smithi033.stdout: "exported": 0, 2024-09-12T02:18:06.566 INFO:teuthology.orchestra.run.smithi033.stdout: "exported_inodes": 0, 2024-09-12T02:18:06.566 INFO:teuthology.orchestra.run.smithi033.stdout: "imported": 0, 2024-09-12T02:18:06.566 INFO:teuthology.orchestra.run.smithi033.stdout: "imported_inodes": 0, 2024-09-12T02:18:06.566 INFO:teuthology.orchestra.run.smithi033.stdout: "openino_dir_fetch": 0, 2024-09-12T02:18:06.566 INFO:teuthology.orchestra.run.smithi033.stdout: "openino_backtrace_fetch": 0, 2024-09-12T02:18:06.566 INFO:teuthology.orchestra.run.smithi033.stdout: "openino_peer_discover": 0, 2024-09-12T02:18:06.566 INFO:teuthology.orchestra.run.smithi033.stdout: "root_rfiles": 0, 2024-09-12T02:18:06.566 INFO:teuthology.orchestra.run.smithi033.stdout: "root_rbytes": 0, 2024-09-12T02:18:06.567 INFO:teuthology.orchestra.run.smithi033.stdout: "root_rsnaps": 0, 2024-09-12T02:18:06.567 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_backtrace_fetch": 0, 2024-09-12T02:18:06.567 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_set_tag": 0, 2024-09-12T02:18:06.567 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_backtrace_repaired": 0, 2024-09-12T02:18:06.567 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_inotable_repaired": 0, 2024-09-12T02:18:06.567 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_dir_inodes": 0, 2024-09-12T02:18:06.567 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_dir_base_inodes": 0, 2024-09-12T02:18:06.567 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_dirfrag_rstats": 0, 2024-09-12T02:18:06.567 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_file_inodes": 0, 2024-09-12T02:18:06.567 INFO:teuthology.orchestra.run.smithi033.stdout: "handle_inode_file_caps": 0, 2024-09-12T02:18:06.567 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph_cap_op_revoke": 0, 2024-09-12T02:18:06.567 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph_cap_op_grant": 0, 2024-09-12T02:18:06.567 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph_cap_op_trunc": 0, 2024-09-12T02:18:06.567 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-12T02:18:06.568 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-12T02:18:06.568 INFO:teuthology.orchestra.run.smithi033.stdout: "handle_client_caps": 0, 2024-09-12T02:18:06.568 INFO:teuthology.orchestra.run.smithi033.stdout: "handle_client_caps_dirty": 0, 2024-09-12T02:18:06.568 INFO:teuthology.orchestra.run.smithi033.stdout: "handle_client_cap_release": 0, 2024-09-12T02:18:06.568 INFO:teuthology.orchestra.run.smithi033.stdout: "process_request_cap_release": 0 2024-09-12T02:18:06.568 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-12T02:18:06.568 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_cache": { 2024-09-12T02:18:06.568 INFO:teuthology.orchestra.run.smithi033.stdout: "dir_update": 0, 2024-09-12T02:18:06.568 INFO:teuthology.orchestra.run.smithi033.stdout: "dir_update_receipt": 0, 2024-09-12T02:18:06.568 INFO:teuthology.orchestra.run.smithi033.stdout: "dir_try_discover": 0, 2024-09-12T02:18:06.568 INFO:teuthology.orchestra.run.smithi033.stdout: "dir_send_discover": 0, 2024-09-12T02:18:06.568 INFO:teuthology.orchestra.run.smithi033.stdout: "dir_handle_discover": 0, 2024-09-12T02:18:06.568 INFO:teuthology.orchestra.run.smithi033.stdout: "num_strays": 0, 2024-09-12T02:18:06.569 INFO:teuthology.orchestra.run.smithi033.stdout: "num_strays_delayed": 0, 2024-09-12T02:18:06.569 INFO:teuthology.orchestra.run.smithi033.stdout: "num_strays_enqueuing": 0, 2024-09-12T02:18:06.569 INFO:teuthology.orchestra.run.smithi033.stdout: "strays_created": 0, 2024-09-12T02:18:06.569 INFO:teuthology.orchestra.run.smithi033.stdout: "strays_enqueued": 0, 2024-09-12T02:18:06.569 INFO:teuthology.orchestra.run.smithi033.stdout: "strays_reintegrated": 0, 2024-09-12T02:18:06.569 INFO:teuthology.orchestra.run.smithi033.stdout: "strays_migrated": 0, 2024-09-12T02:18:06.569 INFO:teuthology.orchestra.run.smithi033.stdout: "num_recovering_processing": 0, 2024-09-12T02:18:06.569 INFO:teuthology.orchestra.run.smithi033.stdout: "num_recovering_enqueued": 0, 2024-09-12T02:18:06.569 INFO:teuthology.orchestra.run.smithi033.stdout: "num_recovering_prioritized": 0, 2024-09-12T02:18:06.569 INFO:teuthology.orchestra.run.smithi033.stdout: "recovery_started": 0, 2024-09-12T02:18:06.569 INFO:teuthology.orchestra.run.smithi033.stdout: "recovery_completed": 0, 2024-09-12T02:18:06.569 INFO:teuthology.orchestra.run.smithi033.stdout: "ireq_quiesce_path": 0, 2024-09-12T02:18:06.569 INFO:teuthology.orchestra.run.smithi033.stdout: "ireq_quiesce_inode": 0, 2024-09-12T02:18:06.570 INFO:teuthology.orchestra.run.smithi033.stdout: "ireq_enqueue_scrub": 0, 2024-09-12T02:18:06.570 INFO:teuthology.orchestra.run.smithi033.stdout: "ireq_exportdir": 0, 2024-09-12T02:18:06.570 INFO:teuthology.orchestra.run.smithi033.stdout: "ireq_flush": 0, 2024-09-12T02:18:06.570 INFO:teuthology.orchestra.run.smithi033.stdout: "ireq_fragmentdir": 0, 2024-09-12T02:18:06.570 INFO:teuthology.orchestra.run.smithi033.stdout: "ireq_fragstats": 0, 2024-09-12T02:18:06.570 INFO:teuthology.orchestra.run.smithi033.stdout: "ireq_inodestats": 0 2024-09-12T02:18:06.570 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-12T02:18:06.570 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_log": { 2024-09-12T02:18:06.570 INFO:teuthology.orchestra.run.smithi033.stdout: "evlrg": 0, 2024-09-12T02:18:06.570 INFO:teuthology.orchestra.run.smithi033.stdout: "evadd": 0, 2024-09-12T02:18:06.570 INFO:teuthology.orchestra.run.smithi033.stdout: "evex": 0, 2024-09-12T02:18:06.570 INFO:teuthology.orchestra.run.smithi033.stdout: "evtrm": 0, 2024-09-12T02:18:06.570 INFO:teuthology.orchestra.run.smithi033.stdout: "ev": 4, 2024-09-12T02:18:06.570 INFO:teuthology.orchestra.run.smithi033.stdout: "evexg": 0, 2024-09-12T02:18:06.571 INFO:teuthology.orchestra.run.smithi033.stdout: "evexd": 0, 2024-09-12T02:18:06.571 INFO:teuthology.orchestra.run.smithi033.stdout: "segadd": 0, 2024-09-12T02:18:06.571 INFO:teuthology.orchestra.run.smithi033.stdout: "segex": 0, 2024-09-12T02:18:06.571 INFO:teuthology.orchestra.run.smithi033.stdout: "segtrm": 0, 2024-09-12T02:18:06.571 INFO:teuthology.orchestra.run.smithi033.stdout: "seg": 2, 2024-09-12T02:18:06.571 INFO:teuthology.orchestra.run.smithi033.stdout: "segmjr": 2, 2024-09-12T02:18:06.571 INFO:teuthology.orchestra.run.smithi033.stdout: "segexg": 0, 2024-09-12T02:18:06.571 INFO:teuthology.orchestra.run.smithi033.stdout: "segexd": 0, 2024-09-12T02:18:06.571 INFO:teuthology.orchestra.run.smithi033.stdout: "expos": 4194304, 2024-09-12T02:18:06.571 INFO:teuthology.orchestra.run.smithi033.stdout: "wrpos": 4195426, 2024-09-12T02:18:06.571 INFO:teuthology.orchestra.run.smithi033.stdout: "rdpos": 4195285, 2024-09-12T02:18:06.571 INFO:teuthology.orchestra.run.smithi033.stdout: "jlat": { 2024-09-12T02:18:06.571 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-12T02:18:06.571 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-12T02:18:06.572 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-12T02:18:06.572 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-12T02:18:06.572 INFO:teuthology.orchestra.run.smithi033.stdout: "replayed": 4 2024-09-12T02:18:06.572 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-12T02:18:06.572 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_mem": { 2024-09-12T02:18:06.572 INFO:teuthology.orchestra.run.smithi033.stdout: "ino": 3, 2024-09-12T02:18:06.572 INFO:teuthology.orchestra.run.smithi033.stdout: "ino+": 3, 2024-09-12T02:18:06.572 INFO:teuthology.orchestra.run.smithi033.stdout: "ino-": 0, 2024-09-12T02:18:06.572 INFO:teuthology.orchestra.run.smithi033.stdout: "dir": 1, 2024-09-12T02:18:06.572 INFO:teuthology.orchestra.run.smithi033.stdout: "dir+": 1, 2024-09-12T02:18:06.572 INFO:teuthology.orchestra.run.smithi033.stdout: "dir-": 0, 2024-09-12T02:18:06.572 INFO:teuthology.orchestra.run.smithi033.stdout: "dn": 0, 2024-09-12T02:18:06.572 INFO:teuthology.orchestra.run.smithi033.stdout: "dn+": 0, 2024-09-12T02:18:06.573 INFO:teuthology.orchestra.run.smithi033.stdout: "dn-": 0, 2024-09-12T02:18:06.573 INFO:teuthology.orchestra.run.smithi033.stdout: "cap": 0, 2024-09-12T02:18:06.573 INFO:teuthology.orchestra.run.smithi033.stdout: "cap+": 0, 2024-09-12T02:18:06.573 INFO:teuthology.orchestra.run.smithi033.stdout: "cap-": 0, 2024-09-12T02:18:06.573 INFO:teuthology.orchestra.run.smithi033.stdout: "rss": 48396, 2024-09-12T02:18:06.573 INFO:teuthology.orchestra.run.smithi033.stdout: "heap": 215328 2024-09-12T02:18:06.573 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-12T02:18:06.573 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_server": { 2024-09-12T02:18:06.573 INFO:teuthology.orchestra.run.smithi033.stdout: "dispatch_client_request": 0, 2024-09-12T02:18:06.573 INFO:teuthology.orchestra.run.smithi033.stdout: "dispatch_server_request": 0, 2024-09-12T02:18:06.573 INFO:teuthology.orchestra.run.smithi033.stdout: "handle_client_request": 0, 2024-09-12T02:18:06.573 INFO:teuthology.orchestra.run.smithi033.stdout: "handle_client_session": 0, 2024-09-12T02:18:06.573 INFO:teuthology.orchestra.run.smithi033.stdout: "handle_peer_request": 0, 2024-09-12T02:18:06.574 INFO:teuthology.orchestra.run.smithi033.stdout: "req_create_latency": { 2024-09-12T02:18:06.574 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-12T02:18:06.574 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-12T02:18:06.574 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-12T02:18:06.574 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-12T02:18:06.574 INFO:teuthology.orchestra.run.smithi033.stdout: "req_getattr_latency": { 2024-09-12T02:18:06.574 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-12T02:18:06.574 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-12T02:18:06.574 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-12T02:18:06.574 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-12T02:18:06.574 INFO:teuthology.orchestra.run.smithi033.stdout: "req_getfilelock_latency": { 2024-09-12T02:18:06.574 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-12T02:18:06.574 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-12T02:18:06.574 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-12T02:18:06.575 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-12T02:18:06.575 INFO:teuthology.orchestra.run.smithi033.stdout: "req_link_latency": { 2024-09-12T02:18:06.575 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-12T02:18:06.575 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-12T02:18:06.575 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-12T02:18:06.575 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-12T02:18:06.576 INFO:teuthology.orchestra.run.smithi033.stdout: "req_lookup_latency": { 2024-09-12T02:18:06.576 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-12T02:18:06.576 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-12T02:18:06.576 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-12T02:18:06.576 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-12T02:18:06.576 INFO:teuthology.orchestra.run.smithi033.stdout: "req_lookuphash_latency": { 2024-09-12T02:18:06.576 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-12T02:18:06.576 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-12T02:18:06.576 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-12T02:18:06.576 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-12T02:18:06.576 INFO:teuthology.orchestra.run.smithi033.stdout: "req_lookupino_latency": { 2024-09-12T02:18:06.576 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-12T02:18:06.577 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-12T02:18:06.577 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-12T02:18:06.577 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-12T02:18:06.577 INFO:teuthology.orchestra.run.smithi033.stdout: "req_lookupname_latency": { 2024-09-12T02:18:06.577 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-12T02:18:06.577 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-12T02:18:06.577 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-12T02:18:06.577 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-12T02:18:06.577 INFO:teuthology.orchestra.run.smithi033.stdout: "req_lookupparent_latency": { 2024-09-12T02:18:06.577 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-12T02:18:06.577 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-12T02:18:06.577 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-12T02:18:06.577 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-12T02:18:06.577 INFO:teuthology.orchestra.run.smithi033.stdout: "req_lookupsnap_latency": { 2024-09-12T02:18:06.578 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-12T02:18:06.578 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-12T02:18:06.578 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-12T02:18:06.578 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-12T02:18:06.578 INFO:teuthology.orchestra.run.smithi033.stdout: "req_lssnap_latency": { 2024-09-12T02:18:06.578 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-12T02:18:06.578 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-12T02:18:06.578 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-12T02:18:06.578 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-12T02:18:06.578 INFO:teuthology.orchestra.run.smithi033.stdout: "req_mkdir_latency": { 2024-09-12T02:18:06.578 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-12T02:18:06.579 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-12T02:18:06.579 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-12T02:18:06.579 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-12T02:18:06.579 INFO:teuthology.orchestra.run.smithi033.stdout: "req_mknod_latency": { 2024-09-12T02:18:06.579 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-12T02:18:06.579 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-12T02:18:06.580 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-12T02:18:06.580 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-12T02:18:06.580 INFO:teuthology.orchestra.run.smithi033.stdout: "req_mksnap_latency": { 2024-09-12T02:18:06.580 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-12T02:18:06.580 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-12T02:18:06.580 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-12T02:18:06.580 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-12T02:18:06.580 INFO:teuthology.orchestra.run.smithi033.stdout: "req_open_latency": { 2024-09-12T02:18:06.580 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-12T02:18:06.580 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-12T02:18:06.580 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-12T02:18:06.580 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-12T02:18:06.580 INFO:teuthology.orchestra.run.smithi033.stdout: "req_readdir_latency": { 2024-09-12T02:18:06.581 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-12T02:18:06.581 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-12T02:18:06.581 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-12T02:18:06.581 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-12T02:18:06.581 INFO:teuthology.orchestra.run.smithi033.stdout: "req_rename_latency": { 2024-09-12T02:18:06.581 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-12T02:18:06.581 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-12T02:18:06.581 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-12T02:18:06.581 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-12T02:18:06.581 INFO:teuthology.orchestra.run.smithi033.stdout: "req_renamesnap_latency": { 2024-09-12T02:18:06.581 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-12T02:18:06.581 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-12T02:18:06.581 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-12T02:18:06.582 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-12T02:18:06.582 INFO:teuthology.orchestra.run.smithi033.stdout: "req_snapdiff_latency": { 2024-09-12T02:18:06.582 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-12T02:18:06.582 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-12T02:18:06.582 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-12T02:18:06.582 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-12T02:18:06.582 INFO:teuthology.orchestra.run.smithi033.stdout: "req_rmdir_latency": { 2024-09-12T02:18:06.582 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-12T02:18:06.582 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-12T02:18:06.582 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-12T02:18:06.582 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-12T02:18:06.582 INFO:teuthology.orchestra.run.smithi033.stdout: "req_rmsnap_latency": { 2024-09-12T02:18:06.582 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-12T02:18:06.583 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-12T02:18:06.583 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-12T02:18:06.583 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-12T02:18:06.583 INFO:teuthology.orchestra.run.smithi033.stdout: "req_rmxattr_latency": { 2024-09-12T02:18:06.583 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-12T02:18:06.583 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-12T02:18:06.583 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-12T02:18:06.583 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-12T02:18:06.583 INFO:teuthology.orchestra.run.smithi033.stdout: "req_setattr_latency": { 2024-09-12T02:18:06.583 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-12T02:18:06.583 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-12T02:18:06.583 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-12T02:18:06.583 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-12T02:18:06.583 INFO:teuthology.orchestra.run.smithi033.stdout: "req_setdirlayout_latency": { 2024-09-12T02:18:06.584 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-12T02:18:06.584 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-12T02:18:06.584 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-12T02:18:06.584 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-12T02:18:06.584 INFO:teuthology.orchestra.run.smithi033.stdout: "req_setfilelock_latency": { 2024-09-12T02:18:06.584 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-12T02:18:06.584 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-12T02:18:06.584 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-12T02:18:06.584 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-12T02:18:06.584 INFO:teuthology.orchestra.run.smithi033.stdout: "req_setlayout_latency": { 2024-09-12T02:18:06.584 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-12T02:18:06.584 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-12T02:18:06.584 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-12T02:18:06.585 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-12T02:18:06.585 INFO:teuthology.orchestra.run.smithi033.stdout: "req_setxattr_latency": { 2024-09-12T02:18:06.585 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-12T02:18:06.585 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-12T02:18:06.585 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-12T02:18:06.585 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-12T02:18:06.585 INFO:teuthology.orchestra.run.smithi033.stdout: "req_symlink_latency": { 2024-09-12T02:18:06.585 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-12T02:18:06.585 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-12T02:18:06.585 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-12T02:18:06.585 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-12T02:18:06.585 INFO:teuthology.orchestra.run.smithi033.stdout: "req_unlink_latency": { 2024-09-12T02:18:06.585 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-12T02:18:06.585 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-12T02:18:06.586 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-12T02:18:06.586 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-12T02:18:06.586 INFO:teuthology.orchestra.run.smithi033.stdout: "cap_revoke_eviction": 0, 2024-09-12T02:18:06.586 INFO:teuthology.orchestra.run.smithi033.stdout: "cap_acquisition_throttle": 0, 2024-09-12T02:18:06.586 INFO:teuthology.orchestra.run.smithi033.stdout: "req_getvxattr_latency": { 2024-09-12T02:18:06.586 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-12T02:18:06.586 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-12T02:18:06.586 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-12T02:18:06.586 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-12T02:18:06.586 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-12T02:18:06.586 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_sessions": { 2024-09-12T02:18:06.586 INFO:teuthology.orchestra.run.smithi033.stdout: "session_count": 0, 2024-09-12T02:18:06.586 INFO:teuthology.orchestra.run.smithi033.stdout: "session_add": 1, 2024-09-12T02:18:06.587 INFO:teuthology.orchestra.run.smithi033.stdout: "session_remove": 1, 2024-09-12T02:18:06.587 INFO:teuthology.orchestra.run.smithi033.stdout: "sessions_open": 1, 2024-09-12T02:18:06.587 INFO:teuthology.orchestra.run.smithi033.stdout: "sessions_stale": 0, 2024-09-12T02:18:06.587 INFO:teuthology.orchestra.run.smithi033.stdout: "total_load": 0, 2024-09-12T02:18:06.587 INFO:teuthology.orchestra.run.smithi033.stdout: "average_load": 0, 2024-09-12T02:18:06.587 INFO:teuthology.orchestra.run.smithi033.stdout: "avg_session_uptime": 1236, 2024-09-12T02:18:06.587 INFO:teuthology.orchestra.run.smithi033.stdout: "mdthresh_evicted": 0 2024-09-12T02:18:06.587 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-12T02:18:06.587 INFO:teuthology.orchestra.run.smithi033.stdout: "mempool": { 2024-09-12T02:18:06.587 INFO:teuthology.orchestra.run.smithi033.stdout: "bloom_filter_bytes": 0, 2024-09-12T02:18:06.587 INFO:teuthology.orchestra.run.smithi033.stdout: "bloom_filter_items": 0, 2024-09-12T02:18:06.587 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T02:18:06.587 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_alloc_items": 0, 2024-09-12T02:18:06.588 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T02:18:06.588 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_data_items": 0, 2024-09-12T02:18:06.588 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T02:18:06.588 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T02:18:06.588 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T02:18:06.588 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T02:18:06.588 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T02:18:06.588 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_other_items": 0, 2024-09-12T02:18:06.588 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T02:18:06.588 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T02:18:06.588 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_extent_bytes": 0, 2024-09-12T02:18:06.588 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_extent_items": 0, 2024-09-12T02:18:06.588 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_blob_bytes": 0, 2024-09-12T02:18:06.589 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_blob_items": 0, 2024-09-12T02:18:06.589 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T02:18:06.589 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T02:18:06.589 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T02:18:06.589 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T02:18:06.589 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T02:18:06.589 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_fsck_items": 0, 2024-09-12T02:18:06.589 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_txc_bytes": 0, 2024-09-12T02:18:06.589 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_txc_items": 0, 2024-09-12T02:18:06.589 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T02:18:06.589 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T02:18:06.589 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_writing_bytes": 0, 2024-09-12T02:18:06.589 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_writing_items": 0, 2024-09-12T02:18:06.589 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_bytes": 0, 2024-09-12T02:18:06.590 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_items": 0, 2024-09-12T02:18:06.590 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T02:18:06.590 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_file_reader_items": 0, 2024-09-12T02:18:06.590 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T02:18:06.590 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_file_writer_items": 0, 2024-09-12T02:18:06.590 INFO:teuthology.orchestra.run.smithi033.stdout: "buffer_anon_bytes": 182087, 2024-09-12T02:18:06.590 INFO:teuthology.orchestra.run.smithi033.stdout: "buffer_anon_items": 48, 2024-09-12T02:18:06.590 INFO:teuthology.orchestra.run.smithi033.stdout: "buffer_meta_bytes": 0, 2024-09-12T02:18:06.590 INFO:teuthology.orchestra.run.smithi033.stdout: "buffer_meta_items": 0, 2024-09-12T02:18:06.590 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_bytes": 0, 2024-09-12T02:18:06.590 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_items": 0, 2024-09-12T02:18:06.590 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_mapbl_bytes": 0, 2024-09-12T02:18:06.590 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_mapbl_items": 0, 2024-09-12T02:18:06.591 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_pglog_bytes": 0, 2024-09-12T02:18:06.591 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_pglog_items": 0, 2024-09-12T02:18:06.591 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_bytes": 7304, 2024-09-12T02:18:06.591 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_items": 150, 2024-09-12T02:18:06.591 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T02:18:06.591 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_mapping_items": 0, 2024-09-12T02:18:06.591 INFO:teuthology.orchestra.run.smithi033.stdout: "pgmap_bytes": 0, 2024-09-12T02:18:06.591 INFO:teuthology.orchestra.run.smithi033.stdout: "pgmap_items": 0, 2024-09-12T02:18:06.591 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_co_bytes": 8512, 2024-09-12T02:18:06.591 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_co_items": 13, 2024-09-12T02:18:06.591 INFO:teuthology.orchestra.run.smithi033.stdout: "unittest_1_bytes": 0, 2024-09-12T02:18:06.591 INFO:teuthology.orchestra.run.smithi033.stdout: "unittest_1_items": 0, 2024-09-12T02:18:06.591 INFO:teuthology.orchestra.run.smithi033.stdout: "unittest_2_bytes": 0, 2024-09-12T02:18:06.591 INFO:teuthology.orchestra.run.smithi033.stdout: "unittest_2_items": 0 2024-09-12T02:18:06.592 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-12T02:18:06.592 INFO:teuthology.orchestra.run.smithi033.stdout: "objecter": { 2024-09-12T02:18:06.592 INFO:teuthology.orchestra.run.smithi033.stdout: "op_active": 0, 2024-09-12T02:18:06.592 INFO:teuthology.orchestra.run.smithi033.stdout: "op_laggy": 0, 2024-09-12T02:18:06.592 INFO:teuthology.orchestra.run.smithi033.stdout: "op_send": 3155, 2024-09-12T02:18:06.592 INFO:teuthology.orchestra.run.smithi033.stdout: "op_send_bytes": 21, 2024-09-12T02:18:06.592 INFO:teuthology.orchestra.run.smithi033.stdout: "op_resend": 3, 2024-09-12T02:18:06.592 INFO:teuthology.orchestra.run.smithi033.stdout: "op_reply": 3155, 2024-09-12T02:18:06.592 INFO:teuthology.orchestra.run.smithi033.stdout: "op_latency": { 2024-09-12T02:18:06.592 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 3155, 2024-09-12T02:18:06.592 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 1.656971998, 2024-09-12T02:18:06.592 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000525189 2024-09-12T02:18:06.592 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-12T02:18:06.593 INFO:teuthology.orchestra.run.smithi033.stdout: "op_inflight": 0, 2024-09-12T02:18:06.593 INFO:teuthology.orchestra.run.smithi033.stdout: "oplen_avg": { 2024-09-12T02:18:06.593 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 3155, 2024-09-12T02:18:06.593 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 3156 2024-09-12T02:18:06.593 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-12T02:18:06.593 INFO:teuthology.orchestra.run.smithi033.stdout: "op": 3155, 2024-09-12T02:18:06.593 INFO:teuthology.orchestra.run.smithi033.stdout: "op_r": 3155, 2024-09-12T02:18:06.593 INFO:teuthology.orchestra.run.smithi033.stdout: "op_w": 0, 2024-09-12T02:18:06.593 INFO:teuthology.orchestra.run.smithi033.stdout: "op_rmw": 0, 2024-09-12T02:18:06.593 INFO:teuthology.orchestra.run.smithi033.stdout: "op_pg": 0, 2024-09-12T02:18:06.593 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_stat": 2067, 2024-09-12T02:18:06.593 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_create": 0, 2024-09-12T02:18:06.593 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_read": 1086, 2024-09-12T02:18:06.594 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_write": 0, 2024-09-12T02:18:06.594 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_writefull": 0, 2024-09-12T02:18:06.594 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_writesame": 0, 2024-09-12T02:18:06.594 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_append": 0, 2024-09-12T02:18:06.594 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_zero": 0, 2024-09-12T02:18:06.594 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_truncate": 0, 2024-09-12T02:18:06.594 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_delete": 0, 2024-09-12T02:18:06.594 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_mapext": 0, 2024-09-12T02:18:06.594 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_sparse_read": 0, 2024-09-12T02:18:06.594 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_clonerange": 0, 2024-09-12T02:18:06.594 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_getxattr": 1, 2024-09-12T02:18:06.594 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_setxattr": 0, 2024-09-12T02:18:06.594 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_cmpxattr": 0, 2024-09-12T02:18:06.594 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_rmxattr": 0, 2024-09-12T02:18:06.595 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_resetxattrs": 0, 2024-09-12T02:18:06.595 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_call": 0, 2024-09-12T02:18:06.595 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_watch": 0, 2024-09-12T02:18:06.595 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_notify": 0, 2024-09-12T02:18:06.595 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_src_cmpxattr": 0, 2024-09-12T02:18:06.595 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_pgls": 0, 2024-09-12T02:18:06.595 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_pgls_filter": 0, 2024-09-12T02:18:06.595 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_other": 0, 2024-09-12T02:18:06.595 INFO:teuthology.orchestra.run.smithi033.stdout: "linger_active": 0, 2024-09-12T02:18:06.595 INFO:teuthology.orchestra.run.smithi033.stdout: "linger_send": 0, 2024-09-12T02:18:06.595 INFO:teuthology.orchestra.run.smithi033.stdout: "linger_resend": 0, 2024-09-12T02:18:06.595 INFO:teuthology.orchestra.run.smithi033.stdout: "linger_ping": 0, 2024-09-12T02:18:06.595 INFO:teuthology.orchestra.run.smithi033.stdout: "poolop_active": 0, 2024-09-12T02:18:06.596 INFO:teuthology.orchestra.run.smithi033.stdout: "poolop_send": 0, 2024-09-12T02:18:06.596 INFO:teuthology.orchestra.run.smithi033.stdout: "poolop_resend": 0, 2024-09-12T02:18:06.596 INFO:teuthology.orchestra.run.smithi033.stdout: "poolstat_active": 0, 2024-09-12T02:18:06.596 INFO:teuthology.orchestra.run.smithi033.stdout: "poolstat_send": 0, 2024-09-12T02:18:06.596 INFO:teuthology.orchestra.run.smithi033.stdout: "poolstat_resend": 0, 2024-09-12T02:18:06.596 INFO:teuthology.orchestra.run.smithi033.stdout: "statfs_active": 0, 2024-09-12T02:18:06.596 INFO:teuthology.orchestra.run.smithi033.stdout: "statfs_send": 0, 2024-09-12T02:18:06.596 INFO:teuthology.orchestra.run.smithi033.stdout: "statfs_resend": 0, 2024-09-12T02:18:06.596 INFO:teuthology.orchestra.run.smithi033.stdout: "command_active": 0, 2024-09-12T02:18:06.596 INFO:teuthology.orchestra.run.smithi033.stdout: "command_send": 0, 2024-09-12T02:18:06.596 INFO:teuthology.orchestra.run.smithi033.stdout: "command_resend": 0, 2024-09-12T02:18:06.596 INFO:teuthology.orchestra.run.smithi033.stdout: "map_epoch": 82, 2024-09-12T02:18:06.596 INFO:teuthology.orchestra.run.smithi033.stdout: "map_full": 0, 2024-09-12T02:18:06.597 INFO:teuthology.orchestra.run.smithi033.stdout: "map_inc": 2, 2024-09-12T02:18:06.597 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_sessions": 4, 2024-09-12T02:18:06.597 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_session_open": 6, 2024-09-12T02:18:06.597 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_session_close": 2, 2024-09-12T02:18:06.597 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_laggy": 0, 2024-09-12T02:18:06.597 INFO:teuthology.orchestra.run.smithi033.stdout: "omap_wr": 0, 2024-09-12T02:18:06.597 INFO:teuthology.orchestra.run.smithi033.stdout: "omap_rd": 2, 2024-09-12T02:18:06.597 INFO:teuthology.orchestra.run.smithi033.stdout: "omap_del": 0 2024-09-12T02:18:06.597 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-12T02:18:06.597 INFO:teuthology.orchestra.run.smithi033.stdout: "oft": { 2024-09-12T02:18:06.597 INFO:teuthology.orchestra.run.smithi033.stdout: "omap_total_objs": 0, 2024-09-12T02:18:06.597 INFO:teuthology.orchestra.run.smithi033.stdout: "omap_total_kv_pairs": 0, 2024-09-12T02:18:06.597 INFO:teuthology.orchestra.run.smithi033.stdout: "omap_total_updates": 0, 2024-09-12T02:18:06.597 INFO:teuthology.orchestra.run.smithi033.stdout: "omap_total_removes": 0 2024-09-12T02:18:06.598 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-12T02:18:06.598 INFO:teuthology.orchestra.run.smithi033.stdout: "osd-slow-ops": { 2024-09-12T02:18:06.598 INFO:teuthology.orchestra.run.smithi033.stdout: "slow_ops_count": 0 2024-09-12T02:18:06.598 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-12T02:18:06.598 INFO:teuthology.orchestra.run.smithi033.stdout: "purge_queue": { 2024-09-12T02:18:06.598 INFO:teuthology.orchestra.run.smithi033.stdout: "pq_executing_ops": 0, 2024-09-12T02:18:06.598 INFO:teuthology.orchestra.run.smithi033.stdout: "pq_executing_ops_high_water": 0, 2024-09-12T02:18:06.598 INFO:teuthology.orchestra.run.smithi033.stdout: "pq_executing": 0, 2024-09-12T02:18:06.598 INFO:teuthology.orchestra.run.smithi033.stdout: "pq_executing_high_water": 0, 2024-09-12T02:18:06.598 INFO:teuthology.orchestra.run.smithi033.stdout: "pq_executed_ops": 0, 2024-09-12T02:18:06.598 INFO:teuthology.orchestra.run.smithi033.stdout: "pq_executed": 0, 2024-09-12T02:18:06.598 INFO:teuthology.orchestra.run.smithi033.stdout: "pq_item_in_journal": 0 2024-09-12T02:18:06.598 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-12T02:18:06.599 INFO:teuthology.orchestra.run.smithi033.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T02:18:06.599 INFO:teuthology.orchestra.run.smithi033.stdout: "val": 0, 2024-09-12T02:18:06.599 INFO:teuthology.orchestra.run.smithi033.stdout: "max": 104857600, 2024-09-12T02:18:06.599 INFO:teuthology.orchestra.run.smithi033.stdout: "get_started": 0, 2024-09-12T02:18:06.599 INFO:teuthology.orchestra.run.smithi033.stdout: "get": 3466, 2024-09-12T02:18:06.599 INFO:teuthology.orchestra.run.smithi033.stdout: "get_sum": 1089280, 2024-09-12T02:18:06.599 INFO:teuthology.orchestra.run.smithi033.stdout: "get_or_fail_fail": 0, 2024-09-12T02:18:06.599 INFO:teuthology.orchestra.run.smithi033.stdout: "get_or_fail_success": 3466, 2024-09-12T02:18:06.599 INFO:teuthology.orchestra.run.smithi033.stdout: "take": 0, 2024-09-12T02:18:06.599 INFO:teuthology.orchestra.run.smithi033.stdout: "take_sum": 0, 2024-09-12T02:18:06.599 INFO:teuthology.orchestra.run.smithi033.stdout: "put": 3466, 2024-09-12T02:18:06.599 INFO:teuthology.orchestra.run.smithi033.stdout: "put_sum": 1089280, 2024-09-12T02:18:06.599 INFO:teuthology.orchestra.run.smithi033.stdout: "wait": { 2024-09-12T02:18:06.599 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-12T02:18:06.600 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-12T02:18:06.600 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-12T02:18:06.600 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-12T02:18:06.600 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-12T02:18:06.600 INFO:teuthology.orchestra.run.smithi033.stdout: "throttle-objecter_bytes": { 2024-09-12T02:18:06.600 INFO:teuthology.orchestra.run.smithi033.stdout: "val": 0, 2024-09-12T02:18:06.600 INFO:teuthology.orchestra.run.smithi033.stdout: "max": 104857600, 2024-09-12T02:18:06.600 INFO:teuthology.orchestra.run.smithi033.stdout: "get_started": 0, 2024-09-12T02:18:06.600 INFO:teuthology.orchestra.run.smithi033.stdout: "get": 0, 2024-09-12T02:18:06.600 INFO:teuthology.orchestra.run.smithi033.stdout: "get_sum": 0, 2024-09-12T02:18:06.600 INFO:teuthology.orchestra.run.smithi033.stdout: "get_or_fail_fail": 0, 2024-09-12T02:18:06.600 INFO:teuthology.orchestra.run.smithi033.stdout: "get_or_fail_success": 0, 2024-09-12T02:18:06.600 INFO:teuthology.orchestra.run.smithi033.stdout: "take": 3155, 2024-09-12T02:18:06.601 INFO:teuthology.orchestra.run.smithi033.stdout: "take_sum": 1127, 2024-09-12T02:18:06.601 INFO:teuthology.orchestra.run.smithi033.stdout: "put": 3155, 2024-09-12T02:18:06.601 INFO:teuthology.orchestra.run.smithi033.stdout: "put_sum": 1127, 2024-09-12T02:18:06.601 INFO:teuthology.orchestra.run.smithi033.stdout: "wait": { 2024-09-12T02:18:06.601 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-12T02:18:06.601 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-12T02:18:06.601 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-12T02:18:06.601 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-12T02:18:06.601 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-12T02:18:06.601 INFO:teuthology.orchestra.run.smithi033.stdout: "throttle-objecter_ops": { 2024-09-12T02:18:06.601 INFO:teuthology.orchestra.run.smithi033.stdout: "val": 0, 2024-09-12T02:18:06.601 INFO:teuthology.orchestra.run.smithi033.stdout: "max": 1024, 2024-09-12T02:18:06.601 INFO:teuthology.orchestra.run.smithi033.stdout: "get_started": 0, 2024-09-12T02:18:06.601 INFO:teuthology.orchestra.run.smithi033.stdout: "get": 0, 2024-09-12T02:18:06.602 INFO:teuthology.orchestra.run.smithi033.stdout: "get_sum": 0, 2024-09-12T02:18:06.602 INFO:teuthology.orchestra.run.smithi033.stdout: "get_or_fail_fail": 0, 2024-09-12T02:18:06.602 INFO:teuthology.orchestra.run.smithi033.stdout: "get_or_fail_success": 0, 2024-09-12T02:18:06.602 INFO:teuthology.orchestra.run.smithi033.stdout: "take": 3155, 2024-09-12T02:18:06.602 INFO:teuthology.orchestra.run.smithi033.stdout: "take_sum": 3155, 2024-09-12T02:18:06.602 INFO:teuthology.orchestra.run.smithi033.stdout: "put": 3155, 2024-09-12T02:18:06.602 INFO:teuthology.orchestra.run.smithi033.stdout: "put_sum": 3155, 2024-09-12T02:18:06.602 INFO:teuthology.orchestra.run.smithi033.stdout: "wait": { 2024-09-12T02:18:06.602 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-12T02:18:06.602 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-12T02:18:06.602 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-12T02:18:06.602 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-12T02:18:06.602 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-12T02:18:06.603 INFO:teuthology.orchestra.run.smithi033.stdout: "throttle-write_buf_throttle": { 2024-09-12T02:18:06.603 INFO:teuthology.orchestra.run.smithi033.stdout: "val": 0, 2024-09-12T02:18:06.603 INFO:teuthology.orchestra.run.smithi033.stdout: "max": 3758096384, 2024-09-12T02:18:06.603 INFO:teuthology.orchestra.run.smithi033.stdout: "get_started": 0, 2024-09-12T02:18:06.603 INFO:teuthology.orchestra.run.smithi033.stdout: "get": 0, 2024-09-12T02:18:06.603 INFO:teuthology.orchestra.run.smithi033.stdout: "get_sum": 0, 2024-09-12T02:18:06.603 INFO:teuthology.orchestra.run.smithi033.stdout: "get_or_fail_fail": 0, 2024-09-12T02:18:06.603 INFO:teuthology.orchestra.run.smithi033.stdout: "get_or_fail_success": 0, 2024-09-12T02:18:06.603 INFO:teuthology.orchestra.run.smithi033.stdout: "take": 0, 2024-09-12T02:18:06.603 INFO:teuthology.orchestra.run.smithi033.stdout: "take_sum": 0, 2024-09-12T02:18:06.603 INFO:teuthology.orchestra.run.smithi033.stdout: "put": 0, 2024-09-12T02:18:06.603 INFO:teuthology.orchestra.run.smithi033.stdout: "put_sum": 0, 2024-09-12T02:18:06.603 INFO:teuthology.orchestra.run.smithi033.stdout: "wait": { 2024-09-12T02:18:06.603 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-12T02:18:06.603 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-12T02:18:06.604 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-12T02:18:06.604 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-12T02:18:06.604 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-12T02:18:06.604 INFO:teuthology.orchestra.run.smithi033.stdout: "throttle-write_buf_throttle-0x563b93a20b40": { 2024-09-12T02:18:06.604 INFO:teuthology.orchestra.run.smithi033.stdout: "val": 0, 2024-09-12T02:18:06.604 INFO:teuthology.orchestra.run.smithi033.stdout: "max": 3758096384, 2024-09-12T02:18:06.604 INFO:teuthology.orchestra.run.smithi033.stdout: "get_started": 0, 2024-09-12T02:18:06.604 INFO:teuthology.orchestra.run.smithi033.stdout: "get": 0, 2024-09-12T02:18:06.604 INFO:teuthology.orchestra.run.smithi033.stdout: "get_sum": 0, 2024-09-12T02:18:06.604 INFO:teuthology.orchestra.run.smithi033.stdout: "get_or_fail_fail": 0, 2024-09-12T02:18:06.604 INFO:teuthology.orchestra.run.smithi033.stdout: "get_or_fail_success": 0, 2024-09-12T02:18:06.604 INFO:teuthology.orchestra.run.smithi033.stdout: "take": 0, 2024-09-12T02:18:06.604 INFO:teuthology.orchestra.run.smithi033.stdout: "take_sum": 0, 2024-09-12T02:18:06.605 INFO:teuthology.orchestra.run.smithi033.stdout: "put": 0, 2024-09-12T02:18:06.605 INFO:teuthology.orchestra.run.smithi033.stdout: "put_sum": 0, 2024-09-12T02:18:06.605 INFO:teuthology.orchestra.run.smithi033.stdout: "wait": { 2024-09-12T02:18:06.605 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-12T02:18:06.605 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-12T02:18:06.605 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-12T02:18:06.605 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-12T02:18:06.605 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-12T02:18:06.605 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-12T02:18:07.359 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-09-12T02:18:07.359 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-09-12T02:18:07.359 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=0 2024-09-12T02:18:07.360 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=0 2024-09-12T02:18:07.360 DEBUG:tasks.check_counter:Getting stats from k 2024-09-12T02:18:07.360 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph daemon mds.k perf dump 2024-09-12T02:18:07.609 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T02:18:08.011 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:18:07 smithi033 ceph-mon[35332]: pgmap v821: 97 pgs: 97 active+clean; 4.3 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 341 B/s wr, 6 op/s 2024-09-12T02:18:08.114 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-12T02:18:08.114 INFO:teuthology.orchestra.run.smithi033.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T02:18:08.114 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_messages": 305, 2024-09-12T02:18:08.114 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages": 302, 2024-09-12T02:18:08.114 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_bytes": 503072, 2024-09-12T02:18:08.114 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_bytes": 186160, 2024-09-12T02:18:08.115 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_created_connections": 4, 2024-09-12T02:18:08.115 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_active_connections": 0, 2024-09-12T02:18:08.115 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_total_time": 0.046453480, 2024-09-12T02:18:08.115 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_send_time": 0.021746972, 2024-09-12T02:18:08.115 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_recv_time": 0.027679368, 2024-09-12T02:18:08.115 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T02:18:08.115 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T02:18:08.115 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 301, 2024-09-12T02:18:08.115 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.018917633, 2024-09-12T02:18:08.115 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000062849 2024-09-12T02:18:08.115 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-12T02:18:08.115 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_handle_ack_lat": { 2024-09-12T02:18:08.115 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-12T02:18:08.116 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-12T02:18:08.116 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-12T02:18:08.116 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-12T02:18:08.116 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_encrypted_bytes": 503072, 2024-09-12T02:18:08.116 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_encrypted_bytes": 186160 2024-09-12T02:18:08.116 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-12T02:18:08.116 INFO:teuthology.orchestra.run.smithi033.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T02:18:08.116 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_messages": 2, 2024-09-12T02:18:08.116 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages": 230, 2024-09-12T02:18:08.116 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_bytes": 320, 2024-09-12T02:18:08.116 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_bytes": 197888, 2024-09-12T02:18:08.116 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_created_connections": 4, 2024-09-12T02:18:08.117 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_active_connections": 2, 2024-09-12T02:18:08.117 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_total_time": 0.015431617, 2024-09-12T02:18:08.117 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_send_time": 0.011359141, 2024-09-12T02:18:08.117 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_recv_time": 0.001169463, 2024-09-12T02:18:08.117 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T02:18:08.117 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T02:18:08.117 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 228, 2024-09-12T02:18:08.117 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.014570833, 2024-09-12T02:18:08.117 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000063907 2024-09-12T02:18:08.117 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-12T02:18:08.117 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_handle_ack_lat": { 2024-09-12T02:18:08.117 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-12T02:18:08.117 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-12T02:18:08.118 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-12T02:18:08.118 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-12T02:18:08.118 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-12T02:18:08.118 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_encrypted_bytes": 197888 2024-09-12T02:18:08.118 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-12T02:18:08.118 INFO:teuthology.orchestra.run.smithi033.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T02:18:08.118 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_messages": 4, 2024-09-12T02:18:08.118 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages": 4, 2024-09-12T02:18:08.118 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_bytes": 3024, 2024-09-12T02:18:08.118 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_bytes": 448, 2024-09-12T02:18:08.118 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_created_connections": 2, 2024-09-12T02:18:08.118 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_active_connections": 1, 2024-09-12T02:18:08.119 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_total_time": 0.002296351, 2024-09-12T02:18:08.119 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_send_time": 0.000031373, 2024-09-12T02:18:08.119 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_recv_time": 0.002219617, 2024-09-12T02:18:08.119 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T02:18:08.119 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T02:18:08.119 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 3, 2024-09-12T02:18:08.119 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.004050011, 2024-09-12T02:18:08.119 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.001350003 2024-09-12T02:18:08.119 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-12T02:18:08.119 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_handle_ack_lat": { 2024-09-12T02:18:08.119 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-12T02:18:08.119 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-12T02:18:08.119 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-12T02:18:08.120 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-12T02:18:08.120 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-09-12T02:18:08.120 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_encrypted_bytes": 448 2024-09-12T02:18:08.120 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-12T02:18:08.120 INFO:teuthology.orchestra.run.smithi033.stdout: "cct": { 2024-09-12T02:18:08.120 INFO:teuthology.orchestra.run.smithi033.stdout: "total_workers": 0, 2024-09-12T02:18:08.120 INFO:teuthology.orchestra.run.smithi033.stdout: "unhealthy_workers": 0 2024-09-12T02:18:08.120 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-12T02:18:08.120 INFO:teuthology.orchestra.run.smithi033.stdout: "mempool": { 2024-09-12T02:18:08.120 INFO:teuthology.orchestra.run.smithi033.stdout: "bloom_filter_bytes": 0, 2024-09-12T02:18:08.120 INFO:teuthology.orchestra.run.smithi033.stdout: "bloom_filter_items": 0, 2024-09-12T02:18:08.120 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T02:18:08.120 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_alloc_items": 0, 2024-09-12T02:18:08.121 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T02:18:08.121 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_data_items": 0, 2024-09-12T02:18:08.121 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T02:18:08.121 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T02:18:08.121 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T02:18:08.121 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T02:18:08.121 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T02:18:08.121 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_other_items": 0, 2024-09-12T02:18:08.121 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T02:18:08.121 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T02:18:08.121 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_extent_bytes": 0, 2024-09-12T02:18:08.121 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_extent_items": 0, 2024-09-12T02:18:08.121 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_blob_bytes": 0, 2024-09-12T02:18:08.122 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_blob_items": 0, 2024-09-12T02:18:08.122 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T02:18:08.122 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T02:18:08.122 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T02:18:08.122 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T02:18:08.122 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T02:18:08.122 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_fsck_items": 0, 2024-09-12T02:18:08.122 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_txc_bytes": 0, 2024-09-12T02:18:08.122 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_txc_items": 0, 2024-09-12T02:18:08.122 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T02:18:08.122 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T02:18:08.122 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_writing_bytes": 0, 2024-09-12T02:18:08.122 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_writing_items": 0, 2024-09-12T02:18:08.123 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_bytes": 0, 2024-09-12T02:18:08.123 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_items": 0, 2024-09-12T02:18:08.123 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T02:18:08.123 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_file_reader_items": 0, 2024-09-12T02:18:08.123 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T02:18:08.123 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_file_writer_items": 0, 2024-09-12T02:18:08.123 INFO:teuthology.orchestra.run.smithi033.stdout: "buffer_anon_bytes": 148744, 2024-09-12T02:18:08.123 INFO:teuthology.orchestra.run.smithi033.stdout: "buffer_anon_items": 30, 2024-09-12T02:18:08.123 INFO:teuthology.orchestra.run.smithi033.stdout: "buffer_meta_bytes": 0, 2024-09-12T02:18:08.123 INFO:teuthology.orchestra.run.smithi033.stdout: "buffer_meta_items": 0, 2024-09-12T02:18:08.123 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_bytes": 0, 2024-09-12T02:18:08.123 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_items": 0, 2024-09-12T02:18:08.124 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_mapbl_bytes": 0, 2024-09-12T02:18:08.124 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_mapbl_items": 0, 2024-09-12T02:18:08.124 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_pglog_bytes": 0, 2024-09-12T02:18:08.124 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_pglog_items": 0, 2024-09-12T02:18:08.124 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_bytes": 0, 2024-09-12T02:18:08.124 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_items": 0, 2024-09-12T02:18:08.124 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T02:18:08.124 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_mapping_items": 0, 2024-09-12T02:18:08.124 INFO:teuthology.orchestra.run.smithi033.stdout: "pgmap_bytes": 0, 2024-09-12T02:18:08.124 INFO:teuthology.orchestra.run.smithi033.stdout: "pgmap_items": 0, 2024-09-12T02:18:08.124 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_co_bytes": 928, 2024-09-12T02:18:08.124 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_co_items": 2, 2024-09-12T02:18:08.124 INFO:teuthology.orchestra.run.smithi033.stdout: "unittest_1_bytes": 0, 2024-09-12T02:18:08.125 INFO:teuthology.orchestra.run.smithi033.stdout: "unittest_1_items": 0, 2024-09-12T02:18:08.125 INFO:teuthology.orchestra.run.smithi033.stdout: "unittest_2_bytes": 0, 2024-09-12T02:18:08.125 INFO:teuthology.orchestra.run.smithi033.stdout: "unittest_2_items": 0 2024-09-12T02:18:08.125 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-12T02:18:08.125 INFO:teuthology.orchestra.run.smithi033.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T02:18:08.125 INFO:teuthology.orchestra.run.smithi033.stdout: "val": 0, 2024-09-12T02:18:08.125 INFO:teuthology.orchestra.run.smithi033.stdout: "max": 104857600, 2024-09-12T02:18:08.125 INFO:teuthology.orchestra.run.smithi033.stdout: "get_started": 0, 2024-09-12T02:18:08.125 INFO:teuthology.orchestra.run.smithi033.stdout: "get": 307, 2024-09-12T02:18:08.125 INFO:teuthology.orchestra.run.smithi033.stdout: "get_sum": 459979, 2024-09-12T02:18:08.125 INFO:teuthology.orchestra.run.smithi033.stdout: "get_or_fail_fail": 0, 2024-09-12T02:18:08.125 INFO:teuthology.orchestra.run.smithi033.stdout: "get_or_fail_success": 307, 2024-09-12T02:18:08.125 INFO:teuthology.orchestra.run.smithi033.stdout: "take": 0, 2024-09-12T02:18:08.126 INFO:teuthology.orchestra.run.smithi033.stdout: "take_sum": 0, 2024-09-12T02:18:08.126 INFO:teuthology.orchestra.run.smithi033.stdout: "put": 307, 2024-09-12T02:18:08.126 INFO:teuthology.orchestra.run.smithi033.stdout: "put_sum": 459979, 2024-09-12T02:18:08.126 INFO:teuthology.orchestra.run.smithi033.stdout: "wait": { 2024-09-12T02:18:08.126 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-12T02:18:08.126 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-12T02:18:08.126 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-12T02:18:08.126 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-12T02:18:08.126 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-12T02:18:08.126 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-12T02:18:08.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:18:07 smithi184 ceph-mon[35593]: pgmap v821: 97 pgs: 97 active+clean; 4.3 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 341 B/s wr, 6 op/s 2024-09-12T02:18:08.183 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:18:07 smithi016 ceph-mon[30947]: pgmap v821: 97 pgs: 97 active+clean; 4.3 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 341 B/s wr, 6 op/s 2024-09-12T02:18:08.778 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2024-09-12T02:18:08.778 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2024-09-12T02:18:08.778 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.k 2024-09-12T02:18:08.778 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.k 2024-09-12T02:18:08.778 DEBUG:tasks.check_counter:Getting stats from c 2024-09-12T02:18:08.778 DEBUG:teuthology.orchestra.run.smithi184:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph daemon mds.c perf dump 2024-09-12T02:18:09.037 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T02:18:09.545 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-12T02:18:09.545 INFO:teuthology.orchestra.run.smithi184.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T02:18:09.545 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_messages": 434, 2024-09-12T02:18:09.545 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_messages": 427, 2024-09-12T02:18:09.545 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_bytes": 564505, 2024-09-12T02:18:09.545 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_bytes": 271149, 2024-09-12T02:18:09.545 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_created_connections": 13, 2024-09-12T02:18:09.545 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_active_connections": 2, 2024-09-12T02:18:09.545 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_total_time": 0.061643748, 2024-09-12T02:18:09.545 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_send_time": 0.025879604, 2024-09-12T02:18:09.545 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_recv_time": 0.036445870, 2024-09-12T02:18:09.546 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_fast_dispatch_time": 0.000243482, 2024-09-12T02:18:09.546 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T02:18:09.546 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 426, 2024-09-12T02:18:09.546 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.033707646, 2024-09-12T02:18:09.546 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000079125 2024-09-12T02:18:09.546 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:09.546 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_handle_ack_lat": { 2024-09-12T02:18:09.546 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 154, 2024-09-12T02:18:09.546 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000030256, 2024-09-12T02:18:09.546 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000196 2024-09-12T02:18:09.546 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:09.546 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_encrypted_bytes": 534784, 2024-09-12T02:18:09.547 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_encrypted_bytes": 190064 2024-09-12T02:18:09.547 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:09.547 INFO:teuthology.orchestra.run.smithi184.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T02:18:09.547 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_messages": 86, 2024-09-12T02:18:09.547 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_messages": 312, 2024-09-12T02:18:09.547 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_bytes": 14222, 2024-09-12T02:18:09.547 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_bytes": 1432146, 2024-09-12T02:18:09.547 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_created_connections": 80, 2024-09-12T02:18:09.547 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_active_connections": 7, 2024-09-12T02:18:09.547 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_total_time": 0.040653475, 2024-09-12T02:18:09.547 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_send_time": 0.011272492, 2024-09-12T02:18:09.547 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_recv_time": 0.024630553, 2024-09-12T02:18:09.547 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_fast_dispatch_time": 0.000315570, 2024-09-12T02:18:09.548 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T02:18:09.548 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 310, 2024-09-12T02:18:09.548 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.030743135, 2024-09-12T02:18:09.548 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000099171 2024-09-12T02:18:09.548 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:09.548 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_handle_ack_lat": { 2024-09-12T02:18:09.548 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 70, 2024-09-12T02:18:09.548 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000006393, 2024-09-12T02:18:09.548 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000091 2024-09-12T02:18:09.548 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:09.549 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-12T02:18:09.549 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_encrypted_bytes": 566768 2024-09-12T02:18:09.549 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:09.549 INFO:teuthology.orchestra.run.smithi184.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T02:18:09.549 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_messages": 2034, 2024-09-12T02:18:09.549 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_messages": 1365, 2024-09-12T02:18:09.549 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_bytes": 489277, 2024-09-12T02:18:09.549 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_bytes": 251422, 2024-09-12T02:18:09.549 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_created_connections": 9, 2024-09-12T02:18:09.549 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_active_connections": 4, 2024-09-12T02:18:09.549 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_total_time": 0.142230541, 2024-09-12T02:18:09.549 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_send_time": 0.054704103, 2024-09-12T02:18:09.550 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_recv_time": 0.096263204, 2024-09-12T02:18:09.550 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_fast_dispatch_time": 0.007039688, 2024-09-12T02:18:09.550 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T02:18:09.550 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 1364, 2024-09-12T02:18:09.550 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.065220997, 2024-09-12T02:18:09.550 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000047815 2024-09-12T02:18:09.550 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:09.550 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_handle_ack_lat": { 2024-09-12T02:18:09.550 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 3289, 2024-09-12T02:18:09.550 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000883078, 2024-09-12T02:18:09.550 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000268 2024-09-12T02:18:09.551 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:09.551 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-09-12T02:18:09.551 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_encrypted_bytes": 448 2024-09-12T02:18:09.551 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:09.551 INFO:teuthology.orchestra.run.smithi184.stdout: "cct": { 2024-09-12T02:18:09.551 INFO:teuthology.orchestra.run.smithi184.stdout: "total_workers": 1, 2024-09-12T02:18:09.551 INFO:teuthology.orchestra.run.smithi184.stdout: "unhealthy_workers": 0 2024-09-12T02:18:09.551 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:09.551 INFO:teuthology.orchestra.run.smithi184.stdout: "finisher-MDSRank": { 2024-09-12T02:18:09.551 INFO:teuthology.orchestra.run.smithi184.stdout: "queue_len": 0, 2024-09-12T02:18:09.551 INFO:teuthology.orchestra.run.smithi184.stdout: "complete_latency": { 2024-09-12T02:18:09.551 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 37, 2024-09-12T02:18:09.551 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.001175059, 2024-09-12T02:18:09.552 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000031758 2024-09-12T02:18:09.552 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-12T02:18:09.552 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:09.552 INFO:teuthology.orchestra.run.smithi184.stdout: "finisher-PurgeQueue": { 2024-09-12T02:18:09.552 INFO:teuthology.orchestra.run.smithi184.stdout: "queue_len": 0, 2024-09-12T02:18:09.552 INFO:teuthology.orchestra.run.smithi184.stdout: "complete_latency": { 2024-09-12T02:18:09.552 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 2, 2024-09-12T02:18:09.552 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000024165, 2024-09-12T02:18:09.552 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000012082 2024-09-12T02:18:09.552 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-12T02:18:09.552 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:09.552 INFO:teuthology.orchestra.run.smithi184.stdout: "mds": { 2024-09-12T02:18:09.553 INFO:teuthology.orchestra.run.smithi184.stdout: "request": 0, 2024-09-12T02:18:09.553 INFO:teuthology.orchestra.run.smithi184.stdout: "reply": 0, 2024-09-12T02:18:09.553 INFO:teuthology.orchestra.run.smithi184.stdout: "reply_latency": { 2024-09-12T02:18:09.553 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-09-12T02:18:09.553 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-09-12T02:18:09.553 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-09-12T02:18:09.553 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:09.553 INFO:teuthology.orchestra.run.smithi184.stdout: "slow_reply": 0, 2024-09-12T02:18:09.553 INFO:teuthology.orchestra.run.smithi184.stdout: "forward": 0, 2024-09-12T02:18:09.553 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_fetch_complete": 0, 2024-09-12T02:18:09.553 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_fetch_keys": 0, 2024-09-12T02:18:09.553 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_commit": 11, 2024-09-12T02:18:09.553 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_split": 0, 2024-09-12T02:18:09.554 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_merge": 0, 2024-09-12T02:18:09.554 INFO:teuthology.orchestra.run.smithi184.stdout: "inodes": 16, 2024-09-12T02:18:09.554 INFO:teuthology.orchestra.run.smithi184.stdout: "inodes_top": 0, 2024-09-12T02:18:09.554 INFO:teuthology.orchestra.run.smithi184.stdout: "inodes_bottom": 16, 2024-09-12T02:18:09.554 INFO:teuthology.orchestra.run.smithi184.stdout: "inodes_pin_tail": 0, 2024-09-12T02:18:09.554 INFO:teuthology.orchestra.run.smithi184.stdout: "inodes_pinned": 13, 2024-09-12T02:18:09.554 INFO:teuthology.orchestra.run.smithi184.stdout: "inodes_expired": 3, 2024-09-12T02:18:09.554 INFO:teuthology.orchestra.run.smithi184.stdout: "inodes_with_caps": 0, 2024-09-12T02:18:09.554 INFO:teuthology.orchestra.run.smithi184.stdout: "caps": 0, 2024-09-12T02:18:09.555 INFO:teuthology.orchestra.run.smithi184.stdout: "subtrees": 3, 2024-09-12T02:18:09.555 INFO:teuthology.orchestra.run.smithi184.stdout: "traverse": 41, 2024-09-12T02:18:09.555 INFO:teuthology.orchestra.run.smithi184.stdout: "traverse_hit": 33, 2024-09-12T02:18:09.555 INFO:teuthology.orchestra.run.smithi184.stdout: "traverse_forward": 0, 2024-09-12T02:18:09.555 INFO:teuthology.orchestra.run.smithi184.stdout: "traverse_discover": 7, 2024-09-12T02:18:09.555 INFO:teuthology.orchestra.run.smithi184.stdout: "traverse_dir_fetch": 0, 2024-09-12T02:18:09.555 INFO:teuthology.orchestra.run.smithi184.stdout: "traverse_remote_ino": 0, 2024-09-12T02:18:09.555 INFO:teuthology.orchestra.run.smithi184.stdout: "traverse_lock": 0, 2024-09-12T02:18:09.555 INFO:teuthology.orchestra.run.smithi184.stdout: "load_cent": 0, 2024-09-12T02:18:09.555 INFO:teuthology.orchestra.run.smithi184.stdout: "q": 0, 2024-09-12T02:18:09.555 INFO:teuthology.orchestra.run.smithi184.stdout: "exported": 0, 2024-09-12T02:18:09.555 INFO:teuthology.orchestra.run.smithi184.stdout: "exported_inodes": 0, 2024-09-12T02:18:09.555 INFO:teuthology.orchestra.run.smithi184.stdout: "imported": 0, 2024-09-12T02:18:09.556 INFO:teuthology.orchestra.run.smithi184.stdout: "imported_inodes": 0, 2024-09-12T02:18:09.556 INFO:teuthology.orchestra.run.smithi184.stdout: "openino_dir_fetch": 0, 2024-09-12T02:18:09.556 INFO:teuthology.orchestra.run.smithi184.stdout: "openino_backtrace_fetch": 0, 2024-09-12T02:18:09.556 INFO:teuthology.orchestra.run.smithi184.stdout: "openino_peer_discover": 0, 2024-09-12T02:18:09.556 INFO:teuthology.orchestra.run.smithi184.stdout: "root_rfiles": 5, 2024-09-12T02:18:09.556 INFO:teuthology.orchestra.run.smithi184.stdout: "root_rbytes": 298, 2024-09-12T02:18:09.556 INFO:teuthology.orchestra.run.smithi184.stdout: "root_rsnaps": 0, 2024-09-12T02:18:09.556 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_backtrace_fetch": 0, 2024-09-12T02:18:09.556 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_set_tag": 0, 2024-09-12T02:18:09.556 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_backtrace_repaired": 0, 2024-09-12T02:18:09.556 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_inotable_repaired": 0, 2024-09-12T02:18:09.556 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_dir_inodes": 0, 2024-09-12T02:18:09.557 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_dir_base_inodes": 0, 2024-09-12T02:18:09.557 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_dirfrag_rstats": 0, 2024-09-12T02:18:09.557 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_file_inodes": 0, 2024-09-12T02:18:09.557 INFO:teuthology.orchestra.run.smithi184.stdout: "handle_inode_file_caps": 0, 2024-09-12T02:18:09.557 INFO:teuthology.orchestra.run.smithi184.stdout: "ceph_cap_op_revoke": 0, 2024-09-12T02:18:09.557 INFO:teuthology.orchestra.run.smithi184.stdout: "ceph_cap_op_grant": 0, 2024-09-12T02:18:09.557 INFO:teuthology.orchestra.run.smithi184.stdout: "ceph_cap_op_trunc": 0, 2024-09-12T02:18:09.557 INFO:teuthology.orchestra.run.smithi184.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-12T02:18:09.557 INFO:teuthology.orchestra.run.smithi184.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-12T02:18:09.557 INFO:teuthology.orchestra.run.smithi184.stdout: "handle_client_caps": 0, 2024-09-12T02:18:09.557 INFO:teuthology.orchestra.run.smithi184.stdout: "handle_client_caps_dirty": 0, 2024-09-12T02:18:09.557 INFO:teuthology.orchestra.run.smithi184.stdout: "handle_client_cap_release": 0, 2024-09-12T02:18:09.558 INFO:teuthology.orchestra.run.smithi184.stdout: "process_request_cap_release": 0 2024-09-12T02:18:09.558 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:09.558 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_cache": { 2024-09-12T02:18:09.558 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_update": 0, 2024-09-12T02:18:09.558 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_update_receipt": 18, 2024-09-12T02:18:09.558 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_try_discover": 13, 2024-09-12T02:18:09.558 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_send_discover": 11, 2024-09-12T02:18:09.558 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_handle_discover": 0, 2024-09-12T02:18:09.558 INFO:teuthology.orchestra.run.smithi184.stdout: "num_strays": 0, 2024-09-12T02:18:09.558 INFO:teuthology.orchestra.run.smithi184.stdout: "num_strays_delayed": 0, 2024-09-12T02:18:09.558 INFO:teuthology.orchestra.run.smithi184.stdout: "num_strays_enqueuing": 0, 2024-09-12T02:18:09.558 INFO:teuthology.orchestra.run.smithi184.stdout: "strays_created": 0, 2024-09-12T02:18:09.558 INFO:teuthology.orchestra.run.smithi184.stdout: "strays_enqueued": 0, 2024-09-12T02:18:09.559 INFO:teuthology.orchestra.run.smithi184.stdout: "strays_reintegrated": 0, 2024-09-12T02:18:09.559 INFO:teuthology.orchestra.run.smithi184.stdout: "strays_migrated": 0, 2024-09-12T02:18:09.559 INFO:teuthology.orchestra.run.smithi184.stdout: "num_recovering_processing": 0, 2024-09-12T02:18:09.559 INFO:teuthology.orchestra.run.smithi184.stdout: "num_recovering_enqueued": 0, 2024-09-12T02:18:09.559 INFO:teuthology.orchestra.run.smithi184.stdout: "num_recovering_prioritized": 0, 2024-09-12T02:18:09.559 INFO:teuthology.orchestra.run.smithi184.stdout: "recovery_started": 0, 2024-09-12T02:18:09.559 INFO:teuthology.orchestra.run.smithi184.stdout: "recovery_completed": 0, 2024-09-12T02:18:09.559 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_quiesce_path": 14, 2024-09-12T02:18:09.559 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_quiesce_inode": 83, 2024-09-12T02:18:09.559 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_enqueue_scrub": 0, 2024-09-12T02:18:09.559 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_exportdir": 0, 2024-09-12T02:18:09.559 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_flush": 0, 2024-09-12T02:18:09.559 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_fragmentdir": 0, 2024-09-12T02:18:09.560 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_fragstats": 0, 2024-09-12T02:18:09.560 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_inodestats": 0 2024-09-12T02:18:09.560 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:09.560 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_log": { 2024-09-12T02:18:09.560 INFO:teuthology.orchestra.run.smithi184.stdout: "evlrg": 0, 2024-09-12T02:18:09.560 INFO:teuthology.orchestra.run.smithi184.stdout: "evadd": 2, 2024-09-12T02:18:09.560 INFO:teuthology.orchestra.run.smithi184.stdout: "evex": 0, 2024-09-12T02:18:09.560 INFO:teuthology.orchestra.run.smithi184.stdout: "evtrm": 0, 2024-09-12T02:18:09.560 INFO:teuthology.orchestra.run.smithi184.stdout: "ev": 2, 2024-09-12T02:18:09.560 INFO:teuthology.orchestra.run.smithi184.stdout: "evexg": 0, 2024-09-12T02:18:09.560 INFO:teuthology.orchestra.run.smithi184.stdout: "evexd": 0, 2024-09-12T02:18:09.560 INFO:teuthology.orchestra.run.smithi184.stdout: "segadd": 2, 2024-09-12T02:18:09.560 INFO:teuthology.orchestra.run.smithi184.stdout: "segex": 0, 2024-09-12T02:18:09.561 INFO:teuthology.orchestra.run.smithi184.stdout: "segtrm": 0, 2024-09-12T02:18:09.561 INFO:teuthology.orchestra.run.smithi184.stdout: "seg": 2, 2024-09-12T02:18:09.561 INFO:teuthology.orchestra.run.smithi184.stdout: "segmjr": 2, 2024-09-12T02:18:09.561 INFO:teuthology.orchestra.run.smithi184.stdout: "segexg": 0, 2024-09-12T02:18:09.561 INFO:teuthology.orchestra.run.smithi184.stdout: "segexd": 0, 2024-09-12T02:18:09.561 INFO:teuthology.orchestra.run.smithi184.stdout: "expos": 4194304, 2024-09-12T02:18:09.561 INFO:teuthology.orchestra.run.smithi184.stdout: "wrpos": 4194892, 2024-09-12T02:18:09.561 INFO:teuthology.orchestra.run.smithi184.stdout: "rdpos": 0, 2024-09-12T02:18:09.561 INFO:teuthology.orchestra.run.smithi184.stdout: "jlat": { 2024-09-12T02:18:09.561 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 1, 2024-09-12T02:18:09.561 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.002165985, 2024-09-12T02:18:09.561 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.002165985 2024-09-12T02:18:09.562 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:09.562 INFO:teuthology.orchestra.run.smithi184.stdout: "replayed": 0 2024-09-12T02:18:09.562 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:09.562 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_mem": { 2024-09-12T02:18:09.562 INFO:teuthology.orchestra.run.smithi184.stdout: "ino": 20, 2024-09-12T02:18:09.562 INFO:teuthology.orchestra.run.smithi184.stdout: "ino+": 21, 2024-09-12T02:18:09.562 INFO:teuthology.orchestra.run.smithi184.stdout: "ino-": 1, 2024-09-12T02:18:09.562 INFO:teuthology.orchestra.run.smithi184.stdout: "dir": 19, 2024-09-12T02:18:09.562 INFO:teuthology.orchestra.run.smithi184.stdout: "dir+": 22, 2024-09-12T02:18:09.562 INFO:teuthology.orchestra.run.smithi184.stdout: "dir-": 3, 2024-09-12T02:18:09.563 INFO:teuthology.orchestra.run.smithi184.stdout: "dn": 16, 2024-09-12T02:18:09.563 INFO:teuthology.orchestra.run.smithi184.stdout: "dn+": 19, 2024-09-12T02:18:09.563 INFO:teuthology.orchestra.run.smithi184.stdout: "dn-": 3, 2024-09-12T02:18:09.563 INFO:teuthology.orchestra.run.smithi184.stdout: "cap": 0, 2024-09-12T02:18:09.563 INFO:teuthology.orchestra.run.smithi184.stdout: "cap+": 0, 2024-09-12T02:18:09.563 INFO:teuthology.orchestra.run.smithi184.stdout: "cap-": 0, 2024-09-12T02:18:09.563 INFO:teuthology.orchestra.run.smithi184.stdout: "rss": 50308, 2024-09-12T02:18:09.563 INFO:teuthology.orchestra.run.smithi184.stdout: "heap": 223520 2024-09-12T02:18:09.563 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:09.563 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_server": { 2024-09-12T02:18:09.563 INFO:teuthology.orchestra.run.smithi184.stdout: "dispatch_client_request": 0, 2024-09-12T02:18:09.563 INFO:teuthology.orchestra.run.smithi184.stdout: "dispatch_server_request": 0, 2024-09-12T02:18:09.564 INFO:teuthology.orchestra.run.smithi184.stdout: "handle_client_request": 0, 2024-09-12T02:18:09.564 INFO:teuthology.orchestra.run.smithi184.stdout: "handle_client_session": 0, 2024-09-12T02:18:09.564 INFO:teuthology.orchestra.run.smithi184.stdout: "handle_peer_request": 0, 2024-09-12T02:18:09.564 INFO:teuthology.orchestra.run.smithi184.stdout: "req_create_latency": { 2024-09-12T02:18:09.564 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-09-12T02:18:09.564 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-09-12T02:18:09.564 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-09-12T02:18:09.564 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:09.564 INFO:teuthology.orchestra.run.smithi184.stdout: "req_getattr_latency": { 2024-09-12T02:18:09.564 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-09-12T02:18:09.564 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-09-12T02:18:09.565 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-09-12T02:18:09.565 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:09.565 INFO:teuthology.orchestra.run.smithi184.stdout: "req_getfilelock_latency": { 2024-09-12T02:18:09.565 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-09-12T02:18:09.565 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-09-12T02:18:09.565 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-09-12T02:18:09.565 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:09.565 INFO:teuthology.orchestra.run.smithi184.stdout: "req_link_latency": { 2024-09-12T02:18:09.565 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-09-12T02:18:09.565 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-09-12T02:18:09.565 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-09-12T02:18:09.566 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:09.566 INFO:teuthology.orchestra.run.smithi184.stdout: "req_lookup_latency": { 2024-09-12T02:18:09.566 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-09-12T02:18:09.566 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-09-12T02:18:09.566 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-09-12T02:18:09.566 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:09.566 INFO:teuthology.orchestra.run.smithi184.stdout: "req_lookuphash_latency": { 2024-09-12T02:18:09.566 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-09-12T02:18:09.566 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-09-12T02:18:09.566 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-09-12T02:18:09.566 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:09.566 INFO:teuthology.orchestra.run.smithi184.stdout: "req_lookupino_latency": { 2024-09-12T02:18:09.567 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-09-12T02:18:09.567 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-09-12T02:18:09.567 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-09-12T02:18:09.567 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:09.567 INFO:teuthology.orchestra.run.smithi184.stdout: "req_lookupname_latency": { 2024-09-12T02:18:09.567 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-09-12T02:18:09.567 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-09-12T02:18:09.567 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-09-12T02:18:09.567 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:09.567 INFO:teuthology.orchestra.run.smithi184.stdout: "req_lookupparent_latency": { 2024-09-12T02:18:09.567 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-09-12T02:18:09.567 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-09-12T02:18:09.567 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-09-12T02:18:09.568 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:09.568 INFO:teuthology.orchestra.run.smithi184.stdout: "req_lookupsnap_latency": { 2024-09-12T02:18:09.568 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-09-12T02:18:09.568 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-09-12T02:18:09.568 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-09-12T02:18:09.568 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:09.568 INFO:teuthology.orchestra.run.smithi184.stdout: "req_lssnap_latency": { 2024-09-12T02:18:09.568 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-09-12T02:18:09.568 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-09-12T02:18:09.568 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-09-12T02:18:09.568 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:09.568 INFO:teuthology.orchestra.run.smithi184.stdout: "req_mkdir_latency": { 2024-09-12T02:18:09.569 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-09-12T02:18:09.569 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-09-12T02:18:09.569 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-09-12T02:18:09.569 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:09.569 INFO:teuthology.orchestra.run.smithi184.stdout: "req_mknod_latency": { 2024-09-12T02:18:09.569 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-09-12T02:18:09.569 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-09-12T02:18:09.570 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-09-12T02:18:09.570 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:09.570 INFO:teuthology.orchestra.run.smithi184.stdout: "req_mksnap_latency": { 2024-09-12T02:18:09.570 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-09-12T02:18:09.570 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-09-12T02:18:09.570 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-09-12T02:18:09.570 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:09.570 INFO:teuthology.orchestra.run.smithi184.stdout: "req_open_latency": { 2024-09-12T02:18:09.570 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-09-12T02:18:09.570 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-09-12T02:18:09.570 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-09-12T02:18:09.570 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:09.570 INFO:teuthology.orchestra.run.smithi184.stdout: "req_readdir_latency": { 2024-09-12T02:18:09.571 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-09-12T02:18:09.571 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-09-12T02:18:09.571 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-09-12T02:18:09.571 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:09.571 INFO:teuthology.orchestra.run.smithi184.stdout: "req_rename_latency": { 2024-09-12T02:18:09.571 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-09-12T02:18:09.571 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-09-12T02:18:09.571 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-09-12T02:18:09.571 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:09.571 INFO:teuthology.orchestra.run.smithi184.stdout: "req_renamesnap_latency": { 2024-09-12T02:18:09.571 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-09-12T02:18:09.572 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-09-12T02:18:09.572 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-09-12T02:18:09.572 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:09.572 INFO:teuthology.orchestra.run.smithi184.stdout: "req_snapdiff_latency": { 2024-09-12T02:18:09.572 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-09-12T02:18:09.572 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-09-12T02:18:09.572 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-09-12T02:18:09.572 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:09.572 INFO:teuthology.orchestra.run.smithi184.stdout: "req_rmdir_latency": { 2024-09-12T02:18:09.572 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-09-12T02:18:09.572 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-09-12T02:18:09.572 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-09-12T02:18:09.573 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:09.573 INFO:teuthology.orchestra.run.smithi184.stdout: "req_rmsnap_latency": { 2024-09-12T02:18:09.573 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-09-12T02:18:09.573 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-09-12T02:18:09.573 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-09-12T02:18:09.573 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:09.573 INFO:teuthology.orchestra.run.smithi184.stdout: "req_rmxattr_latency": { 2024-09-12T02:18:09.573 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-09-12T02:18:09.573 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-09-12T02:18:09.573 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-09-12T02:18:09.573 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:09.573 INFO:teuthology.orchestra.run.smithi184.stdout: "req_setattr_latency": { 2024-09-12T02:18:09.574 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-09-12T02:18:09.574 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-09-12T02:18:09.574 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-09-12T02:18:09.574 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:09.574 INFO:teuthology.orchestra.run.smithi184.stdout: "req_setdirlayout_latency": { 2024-09-12T02:18:09.574 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-09-12T02:18:09.574 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-09-12T02:18:09.574 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-09-12T02:18:09.574 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:09.574 INFO:teuthology.orchestra.run.smithi184.stdout: "req_setfilelock_latency": { 2024-09-12T02:18:09.574 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-09-12T02:18:09.574 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-09-12T02:18:09.574 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-09-12T02:18:09.575 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:09.575 INFO:teuthology.orchestra.run.smithi184.stdout: "req_setlayout_latency": { 2024-09-12T02:18:09.575 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-09-12T02:18:09.575 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-09-12T02:18:09.575 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-09-12T02:18:09.575 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:09.575 INFO:teuthology.orchestra.run.smithi184.stdout: "req_setxattr_latency": { 2024-09-12T02:18:09.575 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-09-12T02:18:09.575 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-09-12T02:18:09.575 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-09-12T02:18:09.575 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:09.575 INFO:teuthology.orchestra.run.smithi184.stdout: "req_symlink_latency": { 2024-09-12T02:18:09.576 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-09-12T02:18:09.576 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-09-12T02:18:09.576 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-09-12T02:18:09.576 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:09.576 INFO:teuthology.orchestra.run.smithi184.stdout: "req_unlink_latency": { 2024-09-12T02:18:09.576 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-09-12T02:18:09.576 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-09-12T02:18:09.576 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-09-12T02:18:09.576 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:09.576 INFO:teuthology.orchestra.run.smithi184.stdout: "cap_revoke_eviction": 0, 2024-09-12T02:18:09.576 INFO:teuthology.orchestra.run.smithi184.stdout: "cap_acquisition_throttle": 0, 2024-09-12T02:18:09.576 INFO:teuthology.orchestra.run.smithi184.stdout: "req_getvxattr_latency": { 2024-09-12T02:18:09.576 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-09-12T02:18:09.577 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-09-12T02:18:09.577 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-09-12T02:18:09.577 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-12T02:18:09.577 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:09.577 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_sessions": { 2024-09-12T02:18:09.577 INFO:teuthology.orchestra.run.smithi184.stdout: "session_count": 0, 2024-09-12T02:18:09.577 INFO:teuthology.orchestra.run.smithi184.stdout: "session_add": 0, 2024-09-12T02:18:09.577 INFO:teuthology.orchestra.run.smithi184.stdout: "session_remove": 0, 2024-09-12T02:18:09.577 INFO:teuthology.orchestra.run.smithi184.stdout: "sessions_open": 0, 2024-09-12T02:18:09.577 INFO:teuthology.orchestra.run.smithi184.stdout: "sessions_stale": 0, 2024-09-12T02:18:09.577 INFO:teuthology.orchestra.run.smithi184.stdout: "total_load": 0, 2024-09-12T02:18:09.577 INFO:teuthology.orchestra.run.smithi184.stdout: "average_load": 0, 2024-09-12T02:18:09.577 INFO:teuthology.orchestra.run.smithi184.stdout: "avg_session_uptime": 0, 2024-09-12T02:18:09.578 INFO:teuthology.orchestra.run.smithi184.stdout: "mdthresh_evicted": 0 2024-09-12T02:18:09.578 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:09.578 INFO:teuthology.orchestra.run.smithi184.stdout: "mempool": { 2024-09-12T02:18:09.578 INFO:teuthology.orchestra.run.smithi184.stdout: "bloom_filter_bytes": 0, 2024-09-12T02:18:09.578 INFO:teuthology.orchestra.run.smithi184.stdout: "bloom_filter_items": 0, 2024-09-12T02:18:09.578 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T02:18:09.578 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_alloc_items": 0, 2024-09-12T02:18:09.578 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T02:18:09.578 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_data_items": 0, 2024-09-12T02:18:09.578 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T02:18:09.578 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T02:18:09.578 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T02:18:09.579 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T02:18:09.579 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T02:18:09.579 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_other_items": 0, 2024-09-12T02:18:09.579 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T02:18:09.579 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T02:18:09.579 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_extent_bytes": 0, 2024-09-12T02:18:09.579 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_extent_items": 0, 2024-09-12T02:18:09.579 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_blob_bytes": 0, 2024-09-12T02:18:09.579 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_blob_items": 0, 2024-09-12T02:18:09.579 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T02:18:09.579 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T02:18:09.579 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T02:18:09.579 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T02:18:09.580 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T02:18:09.580 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_fsck_items": 0, 2024-09-12T02:18:09.580 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_txc_bytes": 0, 2024-09-12T02:18:09.580 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_txc_items": 0, 2024-09-12T02:18:09.580 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T02:18:09.580 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T02:18:09.580 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_writing_bytes": 0, 2024-09-12T02:18:09.580 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_writing_items": 0, 2024-09-12T02:18:09.580 INFO:teuthology.orchestra.run.smithi184.stdout: "bluefs_bytes": 0, 2024-09-12T02:18:09.580 INFO:teuthology.orchestra.run.smithi184.stdout: "bluefs_items": 0, 2024-09-12T02:18:09.580 INFO:teuthology.orchestra.run.smithi184.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T02:18:09.580 INFO:teuthology.orchestra.run.smithi184.stdout: "bluefs_file_reader_items": 0, 2024-09-12T02:18:09.581 INFO:teuthology.orchestra.run.smithi184.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T02:18:09.581 INFO:teuthology.orchestra.run.smithi184.stdout: "bluefs_file_writer_items": 0, 2024-09-12T02:18:09.581 INFO:teuthology.orchestra.run.smithi184.stdout: "buffer_anon_bytes": 239111, 2024-09-12T02:18:09.581 INFO:teuthology.orchestra.run.smithi184.stdout: "buffer_anon_items": 81, 2024-09-12T02:18:09.581 INFO:teuthology.orchestra.run.smithi184.stdout: "buffer_meta_bytes": 0, 2024-09-12T02:18:09.581 INFO:teuthology.orchestra.run.smithi184.stdout: "buffer_meta_items": 0, 2024-09-12T02:18:09.581 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_bytes": 0, 2024-09-12T02:18:09.581 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_items": 0, 2024-09-12T02:18:09.581 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_mapbl_bytes": 0, 2024-09-12T02:18:09.581 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_mapbl_items": 0, 2024-09-12T02:18:09.581 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_pglog_bytes": 0, 2024-09-12T02:18:09.581 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_pglog_items": 0, 2024-09-12T02:18:09.581 INFO:teuthology.orchestra.run.smithi184.stdout: "osdmap_bytes": 7304, 2024-09-12T02:18:09.582 INFO:teuthology.orchestra.run.smithi184.stdout: "osdmap_items": 150, 2024-09-12T02:18:09.582 INFO:teuthology.orchestra.run.smithi184.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T02:18:09.582 INFO:teuthology.orchestra.run.smithi184.stdout: "osdmap_mapping_items": 0, 2024-09-12T02:18:09.582 INFO:teuthology.orchestra.run.smithi184.stdout: "pgmap_bytes": 0, 2024-09-12T02:18:09.582 INFO:teuthology.orchestra.run.smithi184.stdout: "pgmap_items": 0, 2024-09-12T02:18:09.582 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_co_bytes": 80941, 2024-09-12T02:18:09.582 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_co_items": 324, 2024-09-12T02:18:09.582 INFO:teuthology.orchestra.run.smithi184.stdout: "unittest_1_bytes": 0, 2024-09-12T02:18:09.582 INFO:teuthology.orchestra.run.smithi184.stdout: "unittest_1_items": 0, 2024-09-12T02:18:09.582 INFO:teuthology.orchestra.run.smithi184.stdout: "unittest_2_bytes": 0, 2024-09-12T02:18:09.582 INFO:teuthology.orchestra.run.smithi184.stdout: "unittest_2_items": 0 2024-09-12T02:18:09.582 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:09.582 INFO:teuthology.orchestra.run.smithi184.stdout: "objecter": { 2024-09-12T02:18:09.583 INFO:teuthology.orchestra.run.smithi184.stdout: "op_active": 0, 2024-09-12T02:18:09.583 INFO:teuthology.orchestra.run.smithi184.stdout: "op_laggy": 0, 2024-09-12T02:18:09.583 INFO:teuthology.orchestra.run.smithi184.stdout: "op_send": 35, 2024-09-12T02:18:09.583 INFO:teuthology.orchestra.run.smithi184.stdout: "op_send_bytes": 10710, 2024-09-12T02:18:09.583 INFO:teuthology.orchestra.run.smithi184.stdout: "op_resend": 32, 2024-09-12T02:18:09.583 INFO:teuthology.orchestra.run.smithi184.stdout: "op_reply": 35, 2024-09-12T02:18:09.583 INFO:teuthology.orchestra.run.smithi184.stdout: "op_latency": { 2024-09-12T02:18:09.583 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 35, 2024-09-12T02:18:09.583 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.209995559, 2024-09-12T02:18:09.583 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.005999873 2024-09-12T02:18:09.583 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:09.583 INFO:teuthology.orchestra.run.smithi184.stdout: "op_inflight": 0, 2024-09-12T02:18:09.583 INFO:teuthology.orchestra.run.smithi184.stdout: "oplen_avg": { 2024-09-12T02:18:09.584 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 35, 2024-09-12T02:18:09.584 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 57 2024-09-12T02:18:09.584 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:09.584 INFO:teuthology.orchestra.run.smithi184.stdout: "op": 35, 2024-09-12T02:18:09.584 INFO:teuthology.orchestra.run.smithi184.stdout: "op_r": 0, 2024-09-12T02:18:09.584 INFO:teuthology.orchestra.run.smithi184.stdout: "op_w": 35, 2024-09-12T02:18:09.584 INFO:teuthology.orchestra.run.smithi184.stdout: "op_rmw": 0, 2024-09-12T02:18:09.584 INFO:teuthology.orchestra.run.smithi184.stdout: "op_pg": 0, 2024-09-12T02:18:09.584 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_stat": 0, 2024-09-12T02:18:09.584 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_create": 10, 2024-09-12T02:18:09.584 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_read": 0, 2024-09-12T02:18:09.584 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_write": 1, 2024-09-12T02:18:09.584 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_writefull": 5, 2024-09-12T02:18:09.585 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_writesame": 0, 2024-09-12T02:18:09.585 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_append": 0, 2024-09-12T02:18:09.585 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_zero": 0, 2024-09-12T02:18:09.585 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_truncate": 0, 2024-09-12T02:18:09.585 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_delete": 6, 2024-09-12T02:18:09.585 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_mapext": 0, 2024-09-12T02:18:09.585 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_sparse_read": 0, 2024-09-12T02:18:09.585 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_clonerange": 0, 2024-09-12T02:18:09.585 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_getxattr": 0, 2024-09-12T02:18:09.585 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_setxattr": 20, 2024-09-12T02:18:09.585 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_cmpxattr": 0, 2024-09-12T02:18:09.585 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_rmxattr": 0, 2024-09-12T02:18:09.585 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_resetxattrs": 0, 2024-09-12T02:18:09.586 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_call": 0, 2024-09-12T02:18:09.586 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_watch": 0, 2024-09-12T02:18:09.586 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_notify": 0, 2024-09-12T02:18:09.586 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_src_cmpxattr": 0, 2024-09-12T02:18:09.586 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_pgls": 0, 2024-09-12T02:18:09.586 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_pgls_filter": 0, 2024-09-12T02:18:09.586 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_other": 0, 2024-09-12T02:18:09.586 INFO:teuthology.orchestra.run.smithi184.stdout: "linger_active": 0, 2024-09-12T02:18:09.586 INFO:teuthology.orchestra.run.smithi184.stdout: "linger_send": 0, 2024-09-12T02:18:09.586 INFO:teuthology.orchestra.run.smithi184.stdout: "linger_resend": 0, 2024-09-12T02:18:09.586 INFO:teuthology.orchestra.run.smithi184.stdout: "linger_ping": 0, 2024-09-12T02:18:09.586 INFO:teuthology.orchestra.run.smithi184.stdout: "poolop_active": 0, 2024-09-12T02:18:09.586 INFO:teuthology.orchestra.run.smithi184.stdout: "poolop_send": 0, 2024-09-12T02:18:09.587 INFO:teuthology.orchestra.run.smithi184.stdout: "poolop_resend": 0, 2024-09-12T02:18:09.587 INFO:teuthology.orchestra.run.smithi184.stdout: "poolstat_active": 0, 2024-09-12T02:18:09.587 INFO:teuthology.orchestra.run.smithi184.stdout: "poolstat_send": 0, 2024-09-12T02:18:09.587 INFO:teuthology.orchestra.run.smithi184.stdout: "poolstat_resend": 0, 2024-09-12T02:18:09.587 INFO:teuthology.orchestra.run.smithi184.stdout: "statfs_active": 0, 2024-09-12T02:18:09.587 INFO:teuthology.orchestra.run.smithi184.stdout: "statfs_send": 0, 2024-09-12T02:18:09.587 INFO:teuthology.orchestra.run.smithi184.stdout: "statfs_resend": 0, 2024-09-12T02:18:09.587 INFO:teuthology.orchestra.run.smithi184.stdout: "command_active": 0, 2024-09-12T02:18:09.587 INFO:teuthology.orchestra.run.smithi184.stdout: "command_send": 0, 2024-09-12T02:18:09.587 INFO:teuthology.orchestra.run.smithi184.stdout: "command_resend": 0, 2024-09-12T02:18:09.587 INFO:teuthology.orchestra.run.smithi184.stdout: "map_epoch": 82, 2024-09-12T02:18:09.587 INFO:teuthology.orchestra.run.smithi184.stdout: "map_full": 0, 2024-09-12T02:18:09.588 INFO:teuthology.orchestra.run.smithi184.stdout: "map_inc": 2, 2024-09-12T02:18:09.588 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_sessions": 8, 2024-09-12T02:18:09.588 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_session_open": 14, 2024-09-12T02:18:09.588 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_session_close": 6, 2024-09-12T02:18:09.588 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_laggy": 0, 2024-09-12T02:18:09.588 INFO:teuthology.orchestra.run.smithi184.stdout: "omap_wr": 14, 2024-09-12T02:18:09.588 INFO:teuthology.orchestra.run.smithi184.stdout: "omap_rd": 0, 2024-09-12T02:18:09.588 INFO:teuthology.orchestra.run.smithi184.stdout: "omap_del": 1 2024-09-12T02:18:09.588 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:09.588 INFO:teuthology.orchestra.run.smithi184.stdout: "oft": { 2024-09-12T02:18:09.588 INFO:teuthology.orchestra.run.smithi184.stdout: "omap_total_objs": 1, 2024-09-12T02:18:09.588 INFO:teuthology.orchestra.run.smithi184.stdout: "omap_total_kv_pairs": 0, 2024-09-12T02:18:09.588 INFO:teuthology.orchestra.run.smithi184.stdout: "omap_total_updates": 0, 2024-09-12T02:18:09.589 INFO:teuthology.orchestra.run.smithi184.stdout: "omap_total_removes": 0 2024-09-12T02:18:09.589 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:09.589 INFO:teuthology.orchestra.run.smithi184.stdout: "osd-slow-ops": { 2024-09-12T02:18:09.589 INFO:teuthology.orchestra.run.smithi184.stdout: "slow_ops_count": 97 2024-09-12T02:18:09.589 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:09.589 INFO:teuthology.orchestra.run.smithi184.stdout: "purge_queue": { 2024-09-12T02:18:09.589 INFO:teuthology.orchestra.run.smithi184.stdout: "pq_executing_ops": 0, 2024-09-12T02:18:09.589 INFO:teuthology.orchestra.run.smithi184.stdout: "pq_executing_ops_high_water": 0, 2024-09-12T02:18:09.589 INFO:teuthology.orchestra.run.smithi184.stdout: "pq_executing": 0, 2024-09-12T02:18:09.589 INFO:teuthology.orchestra.run.smithi184.stdout: "pq_executing_high_water": 0, 2024-09-12T02:18:09.589 INFO:teuthology.orchestra.run.smithi184.stdout: "pq_executed_ops": 0, 2024-09-12T02:18:09.589 INFO:teuthology.orchestra.run.smithi184.stdout: "pq_executed": 0, 2024-09-12T02:18:09.589 INFO:teuthology.orchestra.run.smithi184.stdout: "pq_item_in_journal": 0 2024-09-12T02:18:09.589 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:09.590 INFO:teuthology.orchestra.run.smithi184.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T02:18:09.590 INFO:teuthology.orchestra.run.smithi184.stdout: "val": 0, 2024-09-12T02:18:09.590 INFO:teuthology.orchestra.run.smithi184.stdout: "max": 104857600, 2024-09-12T02:18:09.590 INFO:teuthology.orchestra.run.smithi184.stdout: "get_started": 0, 2024-09-12T02:18:09.590 INFO:teuthology.orchestra.run.smithi184.stdout: "get": 2550, 2024-09-12T02:18:09.590 INFO:teuthology.orchestra.run.smithi184.stdout: "get_sum": 818005, 2024-09-12T02:18:09.590 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_fail": 0, 2024-09-12T02:18:09.590 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_success": 2550, 2024-09-12T02:18:09.590 INFO:teuthology.orchestra.run.smithi184.stdout: "take": 0, 2024-09-12T02:18:09.590 INFO:teuthology.orchestra.run.smithi184.stdout: "take_sum": 0, 2024-09-12T02:18:09.590 INFO:teuthology.orchestra.run.smithi184.stdout: "put": 2550, 2024-09-12T02:18:09.590 INFO:teuthology.orchestra.run.smithi184.stdout: "put_sum": 818005, 2024-09-12T02:18:09.590 INFO:teuthology.orchestra.run.smithi184.stdout: "wait": { 2024-09-12T02:18:09.591 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-09-12T02:18:09.591 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-09-12T02:18:09.591 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-09-12T02:18:09.591 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-12T02:18:09.591 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:09.591 INFO:teuthology.orchestra.run.smithi184.stdout: "throttle-objecter_bytes": { 2024-09-12T02:18:09.591 INFO:teuthology.orchestra.run.smithi184.stdout: "val": 0, 2024-09-12T02:18:09.591 INFO:teuthology.orchestra.run.smithi184.stdout: "max": 104857600, 2024-09-12T02:18:09.591 INFO:teuthology.orchestra.run.smithi184.stdout: "get_started": 0, 2024-09-12T02:18:09.591 INFO:teuthology.orchestra.run.smithi184.stdout: "get": 0, 2024-09-12T02:18:09.591 INFO:teuthology.orchestra.run.smithi184.stdout: "get_sum": 0, 2024-09-12T02:18:09.591 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_fail": 0, 2024-09-12T02:18:09.591 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_success": 0, 2024-09-12T02:18:09.592 INFO:teuthology.orchestra.run.smithi184.stdout: "take": 35, 2024-09-12T02:18:09.592 INFO:teuthology.orchestra.run.smithi184.stdout: "take_sum": 10710, 2024-09-12T02:18:09.592 INFO:teuthology.orchestra.run.smithi184.stdout: "put": 35, 2024-09-12T02:18:09.592 INFO:teuthology.orchestra.run.smithi184.stdout: "put_sum": 10710, 2024-09-12T02:18:09.592 INFO:teuthology.orchestra.run.smithi184.stdout: "wait": { 2024-09-12T02:18:09.592 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-09-12T02:18:09.592 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-09-12T02:18:09.592 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-09-12T02:18:09.592 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-12T02:18:09.592 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:09.592 INFO:teuthology.orchestra.run.smithi184.stdout: "throttle-objecter_ops": { 2024-09-12T02:18:09.592 INFO:teuthology.orchestra.run.smithi184.stdout: "val": 0, 2024-09-12T02:18:09.592 INFO:teuthology.orchestra.run.smithi184.stdout: "max": 1024, 2024-09-12T02:18:09.593 INFO:teuthology.orchestra.run.smithi184.stdout: "get_started": 0, 2024-09-12T02:18:09.593 INFO:teuthology.orchestra.run.smithi184.stdout: "get": 0, 2024-09-12T02:18:09.593 INFO:teuthology.orchestra.run.smithi184.stdout: "get_sum": 0, 2024-09-12T02:18:09.593 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_fail": 0, 2024-09-12T02:18:09.593 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_success": 0, 2024-09-12T02:18:09.593 INFO:teuthology.orchestra.run.smithi184.stdout: "take": 35, 2024-09-12T02:18:09.593 INFO:teuthology.orchestra.run.smithi184.stdout: "take_sum": 35, 2024-09-12T02:18:09.593 INFO:teuthology.orchestra.run.smithi184.stdout: "put": 35, 2024-09-12T02:18:09.593 INFO:teuthology.orchestra.run.smithi184.stdout: "put_sum": 35, 2024-09-12T02:18:09.593 INFO:teuthology.orchestra.run.smithi184.stdout: "wait": { 2024-09-12T02:18:09.593 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-09-12T02:18:09.593 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-09-12T02:18:09.593 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-09-12T02:18:09.594 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-12T02:18:09.594 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:09.594 INFO:teuthology.orchestra.run.smithi184.stdout: "throttle-write_buf_throttle": { 2024-09-12T02:18:09.594 INFO:teuthology.orchestra.run.smithi184.stdout: "val": 0, 2024-09-12T02:18:09.594 INFO:teuthology.orchestra.run.smithi184.stdout: "max": 3758096384, 2024-09-12T02:18:09.594 INFO:teuthology.orchestra.run.smithi184.stdout: "get_started": 0, 2024-09-12T02:18:09.594 INFO:teuthology.orchestra.run.smithi184.stdout: "get": 0, 2024-09-12T02:18:09.594 INFO:teuthology.orchestra.run.smithi184.stdout: "get_sum": 0, 2024-09-12T02:18:09.594 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_fail": 0, 2024-09-12T02:18:09.594 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_success": 0, 2024-09-12T02:18:09.594 INFO:teuthology.orchestra.run.smithi184.stdout: "take": 0, 2024-09-12T02:18:09.594 INFO:teuthology.orchestra.run.smithi184.stdout: "take_sum": 0, 2024-09-12T02:18:09.595 INFO:teuthology.orchestra.run.smithi184.stdout: "put": 0, 2024-09-12T02:18:09.595 INFO:teuthology.orchestra.run.smithi184.stdout: "put_sum": 0, 2024-09-12T02:18:09.595 INFO:teuthology.orchestra.run.smithi184.stdout: "wait": { 2024-09-12T02:18:09.595 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-09-12T02:18:09.595 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-09-12T02:18:09.595 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-09-12T02:18:09.595 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-12T02:18:09.595 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:09.595 INFO:teuthology.orchestra.run.smithi184.stdout: "throttle-write_buf_throttle-0x557752c76960": { 2024-09-12T02:18:09.595 INFO:teuthology.orchestra.run.smithi184.stdout: "val": 0, 2024-09-12T02:18:09.595 INFO:teuthology.orchestra.run.smithi184.stdout: "max": 3758096384, 2024-09-12T02:18:09.595 INFO:teuthology.orchestra.run.smithi184.stdout: "get_started": 0, 2024-09-12T02:18:09.595 INFO:teuthology.orchestra.run.smithi184.stdout: "get": 2, 2024-09-12T02:18:09.595 INFO:teuthology.orchestra.run.smithi184.stdout: "get_sum": 588, 2024-09-12T02:18:09.596 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_fail": 0, 2024-09-12T02:18:09.596 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_success": 2, 2024-09-12T02:18:09.596 INFO:teuthology.orchestra.run.smithi184.stdout: "take": 0, 2024-09-12T02:18:09.596 INFO:teuthology.orchestra.run.smithi184.stdout: "take_sum": 0, 2024-09-12T02:18:09.596 INFO:teuthology.orchestra.run.smithi184.stdout: "put": 1, 2024-09-12T02:18:09.596 INFO:teuthology.orchestra.run.smithi184.stdout: "put_sum": 588, 2024-09-12T02:18:09.596 INFO:teuthology.orchestra.run.smithi184.stdout: "wait": { 2024-09-12T02:18:09.596 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-09-12T02:18:09.596 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-09-12T02:18:09.596 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-09-12T02:18:09.596 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-12T02:18:09.596 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-12T02:18:09.596 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-12T02:18:10.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:18:09 smithi184 ceph-mon[35593]: pgmap v822: 97 pgs: 97 active+clean; 4.3 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T02:18:10.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:18:09 smithi033 ceph-mon[35332]: pgmap v822: 97 pgs: 97 active+clean; 4.3 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T02:18:10.183 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:18:09 smithi016 ceph-mon[30947]: pgmap v822: 97 pgs: 97 active+clean; 4.3 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T02:18:10.342 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-09-12T02:18:10.342 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-09-12T02:18:10.342 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=0 2024-09-12T02:18:10.343 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=18 2024-09-12T02:18:10.343 DEBUG:tasks.check_counter:Getting stats from f 2024-09-12T02:18:10.343 DEBUG:teuthology.orchestra.run.smithi184:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph daemon mds.f perf dump 2024-09-12T02:18:10.590 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T02:18:11.095 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-12T02:18:11.096 INFO:teuthology.orchestra.run.smithi184.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T02:18:11.096 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_messages": 2081, 2024-09-12T02:18:11.096 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_messages": 2083, 2024-09-12T02:18:11.096 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_bytes": 4814603, 2024-09-12T02:18:11.096 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_bytes": 661768, 2024-09-12T02:18:11.096 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_created_connections": 6, 2024-09-12T02:18:11.096 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_active_connections": 2, 2024-09-12T02:18:11.096 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_total_time": 0.208719268, 2024-09-12T02:18:11.096 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_send_time": 0.063734058, 2024-09-12T02:18:11.096 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_recv_time": 0.060830889, 2024-09-12T02:18:11.096 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_fast_dispatch_time": 0.064763087, 2024-09-12T02:18:11.097 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T02:18:11.097 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 2081, 2024-09-12T02:18:11.097 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.089013558, 2024-09-12T02:18:11.097 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000042774 2024-09-12T02:18:11.097 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:11.097 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_handle_ack_lat": { 2024-09-12T02:18:11.097 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-09-12T02:18:11.097 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-09-12T02:18:11.097 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-09-12T02:18:11.097 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:11.097 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-12T02:18:11.097 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_encrypted_bytes": 0 2024-09-12T02:18:11.098 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:11.098 INFO:teuthology.orchestra.run.smithi184.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T02:18:11.098 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_messages": 453, 2024-09-12T02:18:11.098 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_messages": 679, 2024-09-12T02:18:11.098 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_bytes": 111938, 2024-09-12T02:18:11.098 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_bytes": 710252, 2024-09-12T02:18:11.098 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_created_connections": 6, 2024-09-12T02:18:11.098 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_active_connections": 4, 2024-09-12T02:18:11.100 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_total_time": 0.059175127, 2024-09-12T02:18:11.100 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_send_time": 0.026158097, 2024-09-12T02:18:11.100 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_recv_time": 0.011076186, 2024-09-12T02:18:11.100 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_fast_dispatch_time": 0.015308647, 2024-09-12T02:18:11.100 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T02:18:11.100 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 677, 2024-09-12T02:18:11.100 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.034715342, 2024-09-12T02:18:11.100 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000051278 2024-09-12T02:18:11.100 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:11.100 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_handle_ack_lat": { 2024-09-12T02:18:11.100 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-09-12T02:18:11.100 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-09-12T02:18:11.100 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-09-12T02:18:11.101 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:11.101 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-12T02:18:11.101 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_encrypted_bytes": 566768 2024-09-12T02:18:11.101 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:11.101 INFO:teuthology.orchestra.run.smithi184.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T02:18:11.101 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_messages": 1712, 2024-09-12T02:18:11.101 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_messages": 1705, 2024-09-12T02:18:11.101 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_bytes": 4744774, 2024-09-12T02:18:11.101 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_bytes": 630283, 2024-09-12T02:18:11.101 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_created_connections": 3, 2024-09-12T02:18:11.101 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_active_connections": 2, 2024-09-12T02:18:11.101 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_total_time": 0.173955850, 2024-09-12T02:18:11.101 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_send_time": 0.070815475, 2024-09-12T02:18:11.102 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_recv_time": 0.060063930, 2024-09-12T02:18:11.102 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_fast_dispatch_time": 0.036467874, 2024-09-12T02:18:11.102 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T02:18:11.102 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 1705, 2024-09-12T02:18:11.102 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.071944220, 2024-09-12T02:18:11.102 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000042196 2024-09-12T02:18:11.102 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:11.102 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_handle_ack_lat": { 2024-09-12T02:18:11.102 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-09-12T02:18:11.102 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-09-12T02:18:11.102 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-09-12T02:18:11.102 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:11.102 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_encrypted_bytes": 528400, 2024-09-12T02:18:11.103 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_encrypted_bytes": 186672 2024-09-12T02:18:11.103 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:11.103 INFO:teuthology.orchestra.run.smithi184.stdout: "cct": { 2024-09-12T02:18:11.103 INFO:teuthology.orchestra.run.smithi184.stdout: "total_workers": 1, 2024-09-12T02:18:11.103 INFO:teuthology.orchestra.run.smithi184.stdout: "unhealthy_workers": 0 2024-09-12T02:18:11.103 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:11.103 INFO:teuthology.orchestra.run.smithi184.stdout: "finisher-MDSRank": { 2024-09-12T02:18:11.103 INFO:teuthology.orchestra.run.smithi184.stdout: "queue_len": 0, 2024-09-12T02:18:11.103 INFO:teuthology.orchestra.run.smithi184.stdout: "complete_latency": { 2024-09-12T02:18:11.103 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 6652, 2024-09-12T02:18:11.103 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.176665152, 2024-09-12T02:18:11.103 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000026558 2024-09-12T02:18:11.103 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-12T02:18:11.104 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:11.104 INFO:teuthology.orchestra.run.smithi184.stdout: "finisher-PurgeQueue": { 2024-09-12T02:18:11.104 INFO:teuthology.orchestra.run.smithi184.stdout: "queue_len": 0, 2024-09-12T02:18:11.104 INFO:teuthology.orchestra.run.smithi184.stdout: "complete_latency": { 2024-09-12T02:18:11.104 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-09-12T02:18:11.104 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-09-12T02:18:11.104 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-09-12T02:18:11.104 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-12T02:18:11.104 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:11.104 INFO:teuthology.orchestra.run.smithi184.stdout: "mds": { 2024-09-12T02:18:11.104 INFO:teuthology.orchestra.run.smithi184.stdout: "request": 0, 2024-09-12T02:18:11.104 INFO:teuthology.orchestra.run.smithi184.stdout: "reply": 0, 2024-09-12T02:18:11.105 INFO:teuthology.orchestra.run.smithi184.stdout: "reply_latency": { 2024-09-12T02:18:11.105 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-09-12T02:18:11.105 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-09-12T02:18:11.105 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-09-12T02:18:11.105 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:11.105 INFO:teuthology.orchestra.run.smithi184.stdout: "slow_reply": 0, 2024-09-12T02:18:11.105 INFO:teuthology.orchestra.run.smithi184.stdout: "forward": 0, 2024-09-12T02:18:11.105 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_fetch_complete": 0, 2024-09-12T02:18:11.105 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_fetch_keys": 0, 2024-09-12T02:18:11.105 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_commit": 0, 2024-09-12T02:18:11.105 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_split": 0, 2024-09-12T02:18:11.105 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_merge": 0, 2024-09-12T02:18:11.105 INFO:teuthology.orchestra.run.smithi184.stdout: "inodes": 17, 2024-09-12T02:18:11.106 INFO:teuthology.orchestra.run.smithi184.stdout: "inodes_top": 7, 2024-09-12T02:18:11.106 INFO:teuthology.orchestra.run.smithi184.stdout: "inodes_bottom": 10, 2024-09-12T02:18:11.106 INFO:teuthology.orchestra.run.smithi184.stdout: "inodes_pin_tail": 0, 2024-09-12T02:18:11.106 INFO:teuthology.orchestra.run.smithi184.stdout: "inodes_pinned": 7, 2024-09-12T02:18:11.106 INFO:teuthology.orchestra.run.smithi184.stdout: "inodes_expired": 0, 2024-09-12T02:18:11.106 INFO:teuthology.orchestra.run.smithi184.stdout: "inodes_with_caps": 0, 2024-09-12T02:18:11.106 INFO:teuthology.orchestra.run.smithi184.stdout: "caps": 0, 2024-09-12T02:18:11.106 INFO:teuthology.orchestra.run.smithi184.stdout: "subtrees": 2, 2024-09-12T02:18:11.106 INFO:teuthology.orchestra.run.smithi184.stdout: "traverse": 0, 2024-09-12T02:18:11.106 INFO:teuthology.orchestra.run.smithi184.stdout: "traverse_hit": 0, 2024-09-12T02:18:11.106 INFO:teuthology.orchestra.run.smithi184.stdout: "traverse_forward": 0, 2024-09-12T02:18:11.106 INFO:teuthology.orchestra.run.smithi184.stdout: "traverse_discover": 0, 2024-09-12T02:18:11.106 INFO:teuthology.orchestra.run.smithi184.stdout: "traverse_dir_fetch": 0, 2024-09-12T02:18:11.107 INFO:teuthology.orchestra.run.smithi184.stdout: "traverse_remote_ino": 0, 2024-09-12T02:18:11.107 INFO:teuthology.orchestra.run.smithi184.stdout: "traverse_lock": 0, 2024-09-12T02:18:11.107 INFO:teuthology.orchestra.run.smithi184.stdout: "load_cent": 0, 2024-09-12T02:18:11.107 INFO:teuthology.orchestra.run.smithi184.stdout: "q": 0, 2024-09-12T02:18:11.107 INFO:teuthology.orchestra.run.smithi184.stdout: "exported": 0, 2024-09-12T02:18:11.107 INFO:teuthology.orchestra.run.smithi184.stdout: "exported_inodes": 0, 2024-09-12T02:18:11.107 INFO:teuthology.orchestra.run.smithi184.stdout: "imported": 0, 2024-09-12T02:18:11.107 INFO:teuthology.orchestra.run.smithi184.stdout: "imported_inodes": 0, 2024-09-12T02:18:11.107 INFO:teuthology.orchestra.run.smithi184.stdout: "openino_dir_fetch": 0, 2024-09-12T02:18:11.107 INFO:teuthology.orchestra.run.smithi184.stdout: "openino_backtrace_fetch": 0, 2024-09-12T02:18:11.107 INFO:teuthology.orchestra.run.smithi184.stdout: "openino_peer_discover": 0, 2024-09-12T02:18:11.107 INFO:teuthology.orchestra.run.smithi184.stdout: "root_rfiles": 4, 2024-09-12T02:18:11.107 INFO:teuthology.orchestra.run.smithi184.stdout: "root_rbytes": 234, 2024-09-12T02:18:11.107 INFO:teuthology.orchestra.run.smithi184.stdout: "root_rsnaps": 0, 2024-09-12T02:18:11.108 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_backtrace_fetch": 0, 2024-09-12T02:18:11.108 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_set_tag": 0, 2024-09-12T02:18:11.108 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_backtrace_repaired": 0, 2024-09-12T02:18:11.108 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_inotable_repaired": 0, 2024-09-12T02:18:11.108 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_dir_inodes": 0, 2024-09-12T02:18:11.108 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_dir_base_inodes": 0, 2024-09-12T02:18:11.108 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_dirfrag_rstats": 0, 2024-09-12T02:18:11.108 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_file_inodes": 0, 2024-09-12T02:18:11.108 INFO:teuthology.orchestra.run.smithi184.stdout: "handle_inode_file_caps": 0, 2024-09-12T02:18:11.108 INFO:teuthology.orchestra.run.smithi184.stdout: "ceph_cap_op_revoke": 0, 2024-09-12T02:18:11.108 INFO:teuthology.orchestra.run.smithi184.stdout: "ceph_cap_op_grant": 0, 2024-09-12T02:18:11.108 INFO:teuthology.orchestra.run.smithi184.stdout: "ceph_cap_op_trunc": 0, 2024-09-12T02:18:11.108 INFO:teuthology.orchestra.run.smithi184.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-12T02:18:11.109 INFO:teuthology.orchestra.run.smithi184.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-12T02:18:11.109 INFO:teuthology.orchestra.run.smithi184.stdout: "handle_client_caps": 0, 2024-09-12T02:18:11.109 INFO:teuthology.orchestra.run.smithi184.stdout: "handle_client_caps_dirty": 0, 2024-09-12T02:18:11.109 INFO:teuthology.orchestra.run.smithi184.stdout: "handle_client_cap_release": 0, 2024-09-12T02:18:11.109 INFO:teuthology.orchestra.run.smithi184.stdout: "process_request_cap_release": 0 2024-09-12T02:18:11.109 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:11.109 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_cache": { 2024-09-12T02:18:11.109 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_update": 0, 2024-09-12T02:18:11.109 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_update_receipt": 0, 2024-09-12T02:18:11.109 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_try_discover": 0, 2024-09-12T02:18:11.109 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_send_discover": 0, 2024-09-12T02:18:11.109 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_handle_discover": 0, 2024-09-12T02:18:11.109 INFO:teuthology.orchestra.run.smithi184.stdout: "num_strays": 0, 2024-09-12T02:18:11.110 INFO:teuthology.orchestra.run.smithi184.stdout: "num_strays_delayed": 0, 2024-09-12T02:18:11.110 INFO:teuthology.orchestra.run.smithi184.stdout: "num_strays_enqueuing": 0, 2024-09-12T02:18:11.110 INFO:teuthology.orchestra.run.smithi184.stdout: "strays_created": 0, 2024-09-12T02:18:11.110 INFO:teuthology.orchestra.run.smithi184.stdout: "strays_enqueued": 0, 2024-09-12T02:18:11.110 INFO:teuthology.orchestra.run.smithi184.stdout: "strays_reintegrated": 0, 2024-09-12T02:18:11.110 INFO:teuthology.orchestra.run.smithi184.stdout: "strays_migrated": 0, 2024-09-12T02:18:11.110 INFO:teuthology.orchestra.run.smithi184.stdout: "num_recovering_processing": 0, 2024-09-12T02:18:11.110 INFO:teuthology.orchestra.run.smithi184.stdout: "num_recovering_enqueued": 0, 2024-09-12T02:18:11.110 INFO:teuthology.orchestra.run.smithi184.stdout: "num_recovering_prioritized": 0, 2024-09-12T02:18:11.110 INFO:teuthology.orchestra.run.smithi184.stdout: "recovery_started": 0, 2024-09-12T02:18:11.110 INFO:teuthology.orchestra.run.smithi184.stdout: "recovery_completed": 0, 2024-09-12T02:18:11.110 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_quiesce_path": 0, 2024-09-12T02:18:11.110 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_quiesce_inode": 0, 2024-09-12T02:18:11.111 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_enqueue_scrub": 0, 2024-09-12T02:18:11.111 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_exportdir": 0, 2024-09-12T02:18:11.111 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_flush": 0, 2024-09-12T02:18:11.111 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_fragmentdir": 0, 2024-09-12T02:18:11.111 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_fragstats": 0, 2024-09-12T02:18:11.111 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_inodestats": 0 2024-09-12T02:18:11.111 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:11.111 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_log": { 2024-09-12T02:18:11.112 INFO:teuthology.orchestra.run.smithi184.stdout: "evlrg": 0, 2024-09-12T02:18:11.112 INFO:teuthology.orchestra.run.smithi184.stdout: "evadd": 0, 2024-09-12T02:18:11.112 INFO:teuthology.orchestra.run.smithi184.stdout: "evex": 0, 2024-09-12T02:18:11.112 INFO:teuthology.orchestra.run.smithi184.stdout: "evtrm": 0, 2024-09-12T02:18:11.112 INFO:teuthology.orchestra.run.smithi184.stdout: "ev": 2518, 2024-09-12T02:18:11.113 INFO:teuthology.orchestra.run.smithi184.stdout: "evexg": 0, 2024-09-12T02:18:11.113 INFO:teuthology.orchestra.run.smithi184.stdout: "evexd": 0, 2024-09-12T02:18:11.113 INFO:teuthology.orchestra.run.smithi184.stdout: "segadd": 0, 2024-09-12T02:18:11.113 INFO:teuthology.orchestra.run.smithi184.stdout: "segex": 0, 2024-09-12T02:18:11.113 INFO:teuthology.orchestra.run.smithi184.stdout: "segtrm": 0, 2024-09-12T02:18:11.113 INFO:teuthology.orchestra.run.smithi184.stdout: "seg": 2, 2024-09-12T02:18:11.113 INFO:teuthology.orchestra.run.smithi184.stdout: "segmjr": 75, 2024-09-12T02:18:11.113 INFO:teuthology.orchestra.run.smithi184.stdout: "segexg": 0, 2024-09-12T02:18:11.113 INFO:teuthology.orchestra.run.smithi184.stdout: "segexd": 0, 2024-09-12T02:18:11.113 INFO:teuthology.orchestra.run.smithi184.stdout: "expos": 12238854, 2024-09-12T02:18:11.113 INFO:teuthology.orchestra.run.smithi184.stdout: "wrpos": 12254541, 2024-09-12T02:18:11.113 INFO:teuthology.orchestra.run.smithi184.stdout: "rdpos": 12251119, 2024-09-12T02:18:11.114 INFO:teuthology.orchestra.run.smithi184.stdout: "jlat": { 2024-09-12T02:18:11.114 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-09-12T02:18:11.114 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-09-12T02:18:11.114 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-09-12T02:18:11.114 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:11.114 INFO:teuthology.orchestra.run.smithi184.stdout: "replayed": 2518 2024-09-12T02:18:11.114 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:11.114 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_mem": { 2024-09-12T02:18:11.114 INFO:teuthology.orchestra.run.smithi184.stdout: "ino": 15, 2024-09-12T02:18:11.114 INFO:teuthology.orchestra.run.smithi184.stdout: "ino+": 17, 2024-09-12T02:18:11.114 INFO:teuthology.orchestra.run.smithi184.stdout: "ino-": 2, 2024-09-12T02:18:11.114 INFO:teuthology.orchestra.run.smithi184.stdout: "dir": 9, 2024-09-12T02:18:11.114 INFO:teuthology.orchestra.run.smithi184.stdout: "dir+": 11, 2024-09-12T02:18:11.115 INFO:teuthology.orchestra.run.smithi184.stdout: "dir-": 2, 2024-09-12T02:18:11.115 INFO:teuthology.orchestra.run.smithi184.stdout: "dn": 17, 2024-09-12T02:18:11.115 INFO:teuthology.orchestra.run.smithi184.stdout: "dn+": 19, 2024-09-12T02:18:11.115 INFO:teuthology.orchestra.run.smithi184.stdout: "dn-": 2, 2024-09-12T02:18:11.115 INFO:teuthology.orchestra.run.smithi184.stdout: "cap": 0, 2024-09-12T02:18:11.115 INFO:teuthology.orchestra.run.smithi184.stdout: "cap+": 0, 2024-09-12T02:18:11.115 INFO:teuthology.orchestra.run.smithi184.stdout: "cap-": 0, 2024-09-12T02:18:11.115 INFO:teuthology.orchestra.run.smithi184.stdout: "rss": 49364, 2024-09-12T02:18:11.115 INFO:teuthology.orchestra.run.smithi184.stdout: "heap": 215328 2024-09-12T02:18:11.115 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:11.115 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_server": { 2024-09-12T02:18:11.115 INFO:teuthology.orchestra.run.smithi184.stdout: "dispatch_client_request": 0, 2024-09-12T02:18:11.115 INFO:teuthology.orchestra.run.smithi184.stdout: "dispatch_server_request": 0, 2024-09-12T02:18:11.116 INFO:teuthology.orchestra.run.smithi184.stdout: "handle_client_request": 0, 2024-09-12T02:18:11.116 INFO:teuthology.orchestra.run.smithi184.stdout: "handle_client_session": 0, 2024-09-12T02:18:11.116 INFO:teuthology.orchestra.run.smithi184.stdout: "handle_peer_request": 0, 2024-09-12T02:18:11.116 INFO:teuthology.orchestra.run.smithi184.stdout: "req_create_latency": { 2024-09-12T02:18:11.116 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-09-12T02:18:11.116 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-09-12T02:18:11.116 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-09-12T02:18:11.116 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:11.116 INFO:teuthology.orchestra.run.smithi184.stdout: "req_getattr_latency": { 2024-09-12T02:18:11.116 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-09-12T02:18:11.116 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-09-12T02:18:11.116 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-09-12T02:18:11.116 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:11.117 INFO:teuthology.orchestra.run.smithi184.stdout: "req_getfilelock_latency": { 2024-09-12T02:18:11.117 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-09-12T02:18:11.117 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-09-12T02:18:11.117 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-09-12T02:18:11.117 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:11.117 INFO:teuthology.orchestra.run.smithi184.stdout: "req_link_latency": { 2024-09-12T02:18:11.117 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-09-12T02:18:11.117 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-09-12T02:18:11.118 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-09-12T02:18:11.118 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:11.118 INFO:teuthology.orchestra.run.smithi184.stdout: "req_lookup_latency": { 2024-09-12T02:18:11.118 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-09-12T02:18:11.118 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-09-12T02:18:11.118 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-09-12T02:18:11.118 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:11.118 INFO:teuthology.orchestra.run.smithi184.stdout: "req_lookuphash_latency": { 2024-09-12T02:18:11.118 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-09-12T02:18:11.119 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-09-12T02:18:11.119 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-09-12T02:18:11.119 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:11.119 INFO:teuthology.orchestra.run.smithi184.stdout: "req_lookupino_latency": { 2024-09-12T02:18:11.119 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-09-12T02:18:11.119 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-09-12T02:18:11.119 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-09-12T02:18:11.119 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:11.119 INFO:teuthology.orchestra.run.smithi184.stdout: "req_lookupname_latency": { 2024-09-12T02:18:11.119 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-09-12T02:18:11.119 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-09-12T02:18:11.119 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-09-12T02:18:11.119 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:11.119 INFO:teuthology.orchestra.run.smithi184.stdout: "req_lookupparent_latency": { 2024-09-12T02:18:11.120 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-09-12T02:18:11.120 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-09-12T02:18:11.120 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-09-12T02:18:11.120 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:11.120 INFO:teuthology.orchestra.run.smithi184.stdout: "req_lookupsnap_latency": { 2024-09-12T02:18:11.120 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-09-12T02:18:11.120 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-09-12T02:18:11.120 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-09-12T02:18:11.120 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:11.120 INFO:teuthology.orchestra.run.smithi184.stdout: "req_lssnap_latency": { 2024-09-12T02:18:11.121 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-09-12T02:18:11.121 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-09-12T02:18:11.121 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-09-12T02:18:11.121 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:11.121 INFO:teuthology.orchestra.run.smithi184.stdout: "req_mkdir_latency": { 2024-09-12T02:18:11.122 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-09-12T02:18:11.122 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-09-12T02:18:11.122 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-09-12T02:18:11.122 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:11.122 INFO:teuthology.orchestra.run.smithi184.stdout: "req_mknod_latency": { 2024-09-12T02:18:11.122 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-09-12T02:18:11.122 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-09-12T02:18:11.122 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-09-12T02:18:11.122 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:11.122 INFO:teuthology.orchestra.run.smithi184.stdout: "req_mksnap_latency": { 2024-09-12T02:18:11.123 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-09-12T02:18:11.123 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-09-12T02:18:11.123 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-09-12T02:18:11.123 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:11.123 INFO:teuthology.orchestra.run.smithi184.stdout: "req_open_latency": { 2024-09-12T02:18:11.123 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-09-12T02:18:11.123 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-09-12T02:18:11.123 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-09-12T02:18:11.123 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:11.123 INFO:teuthology.orchestra.run.smithi184.stdout: "req_readdir_latency": { 2024-09-12T02:18:11.123 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-09-12T02:18:11.123 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-09-12T02:18:11.124 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-09-12T02:18:11.124 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:11.124 INFO:teuthology.orchestra.run.smithi184.stdout: "req_rename_latency": { 2024-09-12T02:18:11.124 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-09-12T02:18:11.124 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-09-12T02:18:11.124 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-09-12T02:18:11.124 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:11.124 INFO:teuthology.orchestra.run.smithi184.stdout: "req_renamesnap_latency": { 2024-09-12T02:18:11.124 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-09-12T02:18:11.124 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-09-12T02:18:11.125 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-09-12T02:18:11.125 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:11.125 INFO:teuthology.orchestra.run.smithi184.stdout: "req_snapdiff_latency": { 2024-09-12T02:18:11.125 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-09-12T02:18:11.125 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-09-12T02:18:11.125 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-09-12T02:18:11.125 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:11.125 INFO:teuthology.orchestra.run.smithi184.stdout: "req_rmdir_latency": { 2024-09-12T02:18:11.125 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-09-12T02:18:11.125 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-09-12T02:18:11.126 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-09-12T02:18:11.126 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:11.126 INFO:teuthology.orchestra.run.smithi184.stdout: "req_rmsnap_latency": { 2024-09-12T02:18:11.126 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-09-12T02:18:11.126 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-09-12T02:18:11.126 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-09-12T02:18:11.126 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:11.126 INFO:teuthology.orchestra.run.smithi184.stdout: "req_rmxattr_latency": { 2024-09-12T02:18:11.126 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-09-12T02:18:11.126 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-09-12T02:18:11.127 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-09-12T02:18:11.127 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:11.127 INFO:teuthology.orchestra.run.smithi184.stdout: "req_setattr_latency": { 2024-09-12T02:18:11.127 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-09-12T02:18:11.127 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-09-12T02:18:11.127 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-09-12T02:18:11.127 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:11.127 INFO:teuthology.orchestra.run.smithi184.stdout: "req_setdirlayout_latency": { 2024-09-12T02:18:11.127 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-09-12T02:18:11.127 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-09-12T02:18:11.127 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-09-12T02:18:11.128 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:11.128 INFO:teuthology.orchestra.run.smithi184.stdout: "req_setfilelock_latency": { 2024-09-12T02:18:11.128 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-09-12T02:18:11.128 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-09-12T02:18:11.128 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-09-12T02:18:11.128 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:11.128 INFO:teuthology.orchestra.run.smithi184.stdout: "req_setlayout_latency": { 2024-09-12T02:18:11.128 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-09-12T02:18:11.128 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-09-12T02:18:11.128 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-09-12T02:18:11.128 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:11.129 INFO:teuthology.orchestra.run.smithi184.stdout: "req_setxattr_latency": { 2024-09-12T02:18:11.129 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-09-12T02:18:11.129 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-09-12T02:18:11.129 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-09-12T02:18:11.129 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:11.129 INFO:teuthology.orchestra.run.smithi184.stdout: "req_symlink_latency": { 2024-09-12T02:18:11.129 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-09-12T02:18:11.129 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-09-12T02:18:11.129 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-09-12T02:18:11.129 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:11.129 INFO:teuthology.orchestra.run.smithi184.stdout: "req_unlink_latency": { 2024-09-12T02:18:11.129 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-09-12T02:18:11.130 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-09-12T02:18:11.130 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-09-12T02:18:11.130 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:11.130 INFO:teuthology.orchestra.run.smithi184.stdout: "cap_revoke_eviction": 0, 2024-09-12T02:18:11.130 INFO:teuthology.orchestra.run.smithi184.stdout: "cap_acquisition_throttle": 0, 2024-09-12T02:18:11.130 INFO:teuthology.orchestra.run.smithi184.stdout: "req_getvxattr_latency": { 2024-09-12T02:18:11.130 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-09-12T02:18:11.130 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-09-12T02:18:11.130 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-09-12T02:18:11.130 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-12T02:18:11.130 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:11.130 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_sessions": { 2024-09-12T02:18:11.130 INFO:teuthology.orchestra.run.smithi184.stdout: "session_count": 1, 2024-09-12T02:18:11.130 INFO:teuthology.orchestra.run.smithi184.stdout: "session_add": 2, 2024-09-12T02:18:11.131 INFO:teuthology.orchestra.run.smithi184.stdout: "session_remove": 1, 2024-09-12T02:18:11.131 INFO:teuthology.orchestra.run.smithi184.stdout: "sessions_open": 2, 2024-09-12T02:18:11.131 INFO:teuthology.orchestra.run.smithi184.stdout: "sessions_stale": 0, 2024-09-12T02:18:11.131 INFO:teuthology.orchestra.run.smithi184.stdout: "total_load": 0, 2024-09-12T02:18:11.131 INFO:teuthology.orchestra.run.smithi184.stdout: "average_load": 0, 2024-09-12T02:18:11.131 INFO:teuthology.orchestra.run.smithi184.stdout: "avg_session_uptime": 3347, 2024-09-12T02:18:11.131 INFO:teuthology.orchestra.run.smithi184.stdout: "mdthresh_evicted": 0 2024-09-12T02:18:11.131 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:11.131 INFO:teuthology.orchestra.run.smithi184.stdout: "mempool": { 2024-09-12T02:18:11.131 INFO:teuthology.orchestra.run.smithi184.stdout: "bloom_filter_bytes": 0, 2024-09-12T02:18:11.131 INFO:teuthology.orchestra.run.smithi184.stdout: "bloom_filter_items": 0, 2024-09-12T02:18:11.131 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T02:18:11.131 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_alloc_items": 0, 2024-09-12T02:18:11.132 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T02:18:11.132 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_data_items": 0, 2024-09-12T02:18:11.132 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T02:18:11.132 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T02:18:11.132 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T02:18:11.132 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T02:18:11.132 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T02:18:11.132 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_other_items": 0, 2024-09-12T02:18:11.132 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T02:18:11.132 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T02:18:11.132 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_extent_bytes": 0, 2024-09-12T02:18:11.132 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_extent_items": 0, 2024-09-12T02:18:11.133 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_blob_bytes": 0, 2024-09-12T02:18:11.133 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_blob_items": 0, 2024-09-12T02:18:11.133 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T02:18:11.133 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T02:18:11.133 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T02:18:11.133 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T02:18:11.133 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T02:18:11.133 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_fsck_items": 0, 2024-09-12T02:18:11.133 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_txc_bytes": 0, 2024-09-12T02:18:11.133 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_txc_items": 0, 2024-09-12T02:18:11.133 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T02:18:11.133 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T02:18:11.133 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_writing_bytes": 0, 2024-09-12T02:18:11.134 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_writing_items": 0, 2024-09-12T02:18:11.134 INFO:teuthology.orchestra.run.smithi184.stdout: "bluefs_bytes": 0, 2024-09-12T02:18:11.134 INFO:teuthology.orchestra.run.smithi184.stdout: "bluefs_items": 0, 2024-09-12T02:18:11.134 INFO:teuthology.orchestra.run.smithi184.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T02:18:11.134 INFO:teuthology.orchestra.run.smithi184.stdout: "bluefs_file_reader_items": 0, 2024-09-12T02:18:11.134 INFO:teuthology.orchestra.run.smithi184.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T02:18:11.134 INFO:teuthology.orchestra.run.smithi184.stdout: "bluefs_file_writer_items": 0, 2024-09-12T02:18:11.134 INFO:teuthology.orchestra.run.smithi184.stdout: "buffer_anon_bytes": 190205, 2024-09-12T02:18:11.134 INFO:teuthology.orchestra.run.smithi184.stdout: "buffer_anon_items": 53, 2024-09-12T02:18:11.134 INFO:teuthology.orchestra.run.smithi184.stdout: "buffer_meta_bytes": 0, 2024-09-12T02:18:11.134 INFO:teuthology.orchestra.run.smithi184.stdout: "buffer_meta_items": 0, 2024-09-12T02:18:11.134 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_bytes": 0, 2024-09-12T02:18:11.134 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_items": 0, 2024-09-12T02:18:11.134 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_mapbl_bytes": 0, 2024-09-12T02:18:11.134 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_mapbl_items": 0, 2024-09-12T02:18:11.135 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_pglog_bytes": 0, 2024-09-12T02:18:11.135 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_pglog_items": 0, 2024-09-12T02:18:11.135 INFO:teuthology.orchestra.run.smithi184.stdout: "osdmap_bytes": 7304, 2024-09-12T02:18:11.135 INFO:teuthology.orchestra.run.smithi184.stdout: "osdmap_items": 150, 2024-09-12T02:18:11.135 INFO:teuthology.orchestra.run.smithi184.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T02:18:11.135 INFO:teuthology.orchestra.run.smithi184.stdout: "osdmap_mapping_items": 0, 2024-09-12T02:18:11.135 INFO:teuthology.orchestra.run.smithi184.stdout: "pgmap_bytes": 0, 2024-09-12T02:18:11.135 INFO:teuthology.orchestra.run.smithi184.stdout: "pgmap_items": 0, 2024-09-12T02:18:11.135 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_co_bytes": 57248, 2024-09-12T02:18:11.135 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_co_items": 359, 2024-09-12T02:18:11.135 INFO:teuthology.orchestra.run.smithi184.stdout: "unittest_1_bytes": 0, 2024-09-12T02:18:11.135 INFO:teuthology.orchestra.run.smithi184.stdout: "unittest_1_items": 0, 2024-09-12T02:18:11.135 INFO:teuthology.orchestra.run.smithi184.stdout: "unittest_2_bytes": 0, 2024-09-12T02:18:11.136 INFO:teuthology.orchestra.run.smithi184.stdout: "unittest_2_items": 0 2024-09-12T02:18:11.136 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:11.136 INFO:teuthology.orchestra.run.smithi184.stdout: "objecter": { 2024-09-12T02:18:11.136 INFO:teuthology.orchestra.run.smithi184.stdout: "op_active": 0, 2024-09-12T02:18:11.136 INFO:teuthology.orchestra.run.smithi184.stdout: "op_laggy": 0, 2024-09-12T02:18:11.136 INFO:teuthology.orchestra.run.smithi184.stdout: "op_send": 3927, 2024-09-12T02:18:11.136 INFO:teuthology.orchestra.run.smithi184.stdout: "op_send_bytes": 26, 2024-09-12T02:18:11.136 INFO:teuthology.orchestra.run.smithi184.stdout: "op_resend": 4, 2024-09-12T02:18:11.136 INFO:teuthology.orchestra.run.smithi184.stdout: "op_reply": 3927, 2024-09-12T02:18:11.136 INFO:teuthology.orchestra.run.smithi184.stdout: "op_latency": { 2024-09-12T02:18:11.136 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 3927, 2024-09-12T02:18:11.136 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 1.840960925, 2024-09-12T02:18:11.136 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000468795 2024-09-12T02:18:11.136 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:11.137 INFO:teuthology.orchestra.run.smithi184.stdout: "op_inflight": 0, 2024-09-12T02:18:11.137 INFO:teuthology.orchestra.run.smithi184.stdout: "oplen_avg": { 2024-09-12T02:18:11.137 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 3927, 2024-09-12T02:18:11.137 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 3928 2024-09-12T02:18:11.137 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:11.137 INFO:teuthology.orchestra.run.smithi184.stdout: "op": 3927, 2024-09-12T02:18:11.137 INFO:teuthology.orchestra.run.smithi184.stdout: "op_r": 3927, 2024-09-12T02:18:11.137 INFO:teuthology.orchestra.run.smithi184.stdout: "op_w": 0, 2024-09-12T02:18:11.137 INFO:teuthology.orchestra.run.smithi184.stdout: "op_rmw": 0, 2024-09-12T02:18:11.137 INFO:teuthology.orchestra.run.smithi184.stdout: "op_pg": 0, 2024-09-12T02:18:11.137 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_stat": 2159, 2024-09-12T02:18:11.137 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_create": 0, 2024-09-12T02:18:11.137 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_read": 1765, 2024-09-12T02:18:11.137 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_write": 0, 2024-09-12T02:18:11.138 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_writefull": 0, 2024-09-12T02:18:11.138 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_writesame": 0, 2024-09-12T02:18:11.138 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_append": 0, 2024-09-12T02:18:11.138 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_zero": 0, 2024-09-12T02:18:11.138 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_truncate": 0, 2024-09-12T02:18:11.138 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_delete": 0, 2024-09-12T02:18:11.138 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_mapext": 0, 2024-09-12T02:18:11.138 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_sparse_read": 0, 2024-09-12T02:18:11.138 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_clonerange": 0, 2024-09-12T02:18:11.138 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_getxattr": 2, 2024-09-12T02:18:11.138 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_setxattr": 0, 2024-09-12T02:18:11.138 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_cmpxattr": 0, 2024-09-12T02:18:11.138 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_rmxattr": 0, 2024-09-12T02:18:11.138 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_resetxattrs": 0, 2024-09-12T02:18:11.139 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_call": 0, 2024-09-12T02:18:11.139 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_watch": 0, 2024-09-12T02:18:11.139 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_notify": 0, 2024-09-12T02:18:11.139 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_src_cmpxattr": 0, 2024-09-12T02:18:11.139 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_pgls": 0, 2024-09-12T02:18:11.139 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_pgls_filter": 0, 2024-09-12T02:18:11.139 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_other": 0, 2024-09-12T02:18:11.139 INFO:teuthology.orchestra.run.smithi184.stdout: "linger_active": 0, 2024-09-12T02:18:11.139 INFO:teuthology.orchestra.run.smithi184.stdout: "linger_send": 0, 2024-09-12T02:18:11.139 INFO:teuthology.orchestra.run.smithi184.stdout: "linger_resend": 0, 2024-09-12T02:18:11.139 INFO:teuthology.orchestra.run.smithi184.stdout: "linger_ping": 0, 2024-09-12T02:18:11.139 INFO:teuthology.orchestra.run.smithi184.stdout: "poolop_active": 0, 2024-09-12T02:18:11.139 INFO:teuthology.orchestra.run.smithi184.stdout: "poolop_send": 0, 2024-09-12T02:18:11.139 INFO:teuthology.orchestra.run.smithi184.stdout: "poolop_resend": 0, 2024-09-12T02:18:11.140 INFO:teuthology.orchestra.run.smithi184.stdout: "poolstat_active": 0, 2024-09-12T02:18:11.140 INFO:teuthology.orchestra.run.smithi184.stdout: "poolstat_send": 0, 2024-09-12T02:18:11.140 INFO:teuthology.orchestra.run.smithi184.stdout: "poolstat_resend": 0, 2024-09-12T02:18:11.140 INFO:teuthology.orchestra.run.smithi184.stdout: "statfs_active": 0, 2024-09-12T02:18:11.140 INFO:teuthology.orchestra.run.smithi184.stdout: "statfs_send": 0, 2024-09-12T02:18:11.140 INFO:teuthology.orchestra.run.smithi184.stdout: "statfs_resend": 0, 2024-09-12T02:18:11.140 INFO:teuthology.orchestra.run.smithi184.stdout: "command_active": 0, 2024-09-12T02:18:11.140 INFO:teuthology.orchestra.run.smithi184.stdout: "command_send": 0, 2024-09-12T02:18:11.140 INFO:teuthology.orchestra.run.smithi184.stdout: "command_resend": 0, 2024-09-12T02:18:11.140 INFO:teuthology.orchestra.run.smithi184.stdout: "map_epoch": 82, 2024-09-12T02:18:11.140 INFO:teuthology.orchestra.run.smithi184.stdout: "map_full": 0, 2024-09-12T02:18:11.140 INFO:teuthology.orchestra.run.smithi184.stdout: "map_inc": 2, 2024-09-12T02:18:11.140 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_sessions": 5, 2024-09-12T02:18:11.141 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_session_open": 5, 2024-09-12T02:18:11.141 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_session_close": 0, 2024-09-12T02:18:11.141 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_laggy": 0, 2024-09-12T02:18:11.141 INFO:teuthology.orchestra.run.smithi184.stdout: "omap_wr": 0, 2024-09-12T02:18:11.141 INFO:teuthology.orchestra.run.smithi184.stdout: "omap_rd": 2, 2024-09-12T02:18:11.141 INFO:teuthology.orchestra.run.smithi184.stdout: "omap_del": 0 2024-09-12T02:18:11.141 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:11.141 INFO:teuthology.orchestra.run.smithi184.stdout: "oft": { 2024-09-12T02:18:11.141 INFO:teuthology.orchestra.run.smithi184.stdout: "omap_total_objs": 0, 2024-09-12T02:18:11.141 INFO:teuthology.orchestra.run.smithi184.stdout: "omap_total_kv_pairs": 0, 2024-09-12T02:18:11.141 INFO:teuthology.orchestra.run.smithi184.stdout: "omap_total_updates": 0, 2024-09-12T02:18:11.141 INFO:teuthology.orchestra.run.smithi184.stdout: "omap_total_removes": 0 2024-09-12T02:18:11.141 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:11.141 INFO:teuthology.orchestra.run.smithi184.stdout: "osd-slow-ops": { 2024-09-12T02:18:11.142 INFO:teuthology.orchestra.run.smithi184.stdout: "slow_ops_count": 0 2024-09-12T02:18:11.142 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:11.142 INFO:teuthology.orchestra.run.smithi184.stdout: "purge_queue": { 2024-09-12T02:18:11.142 INFO:teuthology.orchestra.run.smithi184.stdout: "pq_executing_ops": 0, 2024-09-12T02:18:11.142 INFO:teuthology.orchestra.run.smithi184.stdout: "pq_executing_ops_high_water": 0, 2024-09-12T02:18:11.142 INFO:teuthology.orchestra.run.smithi184.stdout: "pq_executing": 0, 2024-09-12T02:18:11.142 INFO:teuthology.orchestra.run.smithi184.stdout: "pq_executing_high_water": 0, 2024-09-12T02:18:11.142 INFO:teuthology.orchestra.run.smithi184.stdout: "pq_executed_ops": 0, 2024-09-12T02:18:11.142 INFO:teuthology.orchestra.run.smithi184.stdout: "pq_executed": 0, 2024-09-12T02:18:11.142 INFO:teuthology.orchestra.run.smithi184.stdout: "pq_item_in_journal": 0 2024-09-12T02:18:11.142 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:11.142 INFO:teuthology.orchestra.run.smithi184.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T02:18:11.142 INFO:teuthology.orchestra.run.smithi184.stdout: "val": 0, 2024-09-12T02:18:11.143 INFO:teuthology.orchestra.run.smithi184.stdout: "max": 104857600, 2024-09-12T02:18:11.143 INFO:teuthology.orchestra.run.smithi184.stdout: "get_started": 0, 2024-09-12T02:18:11.143 INFO:teuthology.orchestra.run.smithi184.stdout: "get": 4242, 2024-09-12T02:18:11.143 INFO:teuthology.orchestra.run.smithi184.stdout: "get_sum": 9270368, 2024-09-12T02:18:11.143 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_fail": 0, 2024-09-12T02:18:11.143 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_success": 4242, 2024-09-12T02:18:11.143 INFO:teuthology.orchestra.run.smithi184.stdout: "take": 0, 2024-09-12T02:18:11.143 INFO:teuthology.orchestra.run.smithi184.stdout: "take_sum": 0, 2024-09-12T02:18:11.143 INFO:teuthology.orchestra.run.smithi184.stdout: "put": 4242, 2024-09-12T02:18:11.143 INFO:teuthology.orchestra.run.smithi184.stdout: "put_sum": 9270368, 2024-09-12T02:18:11.143 INFO:teuthology.orchestra.run.smithi184.stdout: "wait": { 2024-09-12T02:18:11.143 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-09-12T02:18:11.144 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-09-12T02:18:11.144 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-09-12T02:18:11.144 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-12T02:18:11.144 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:11.144 INFO:teuthology.orchestra.run.smithi184.stdout: "throttle-objecter_bytes": { 2024-09-12T02:18:11.144 INFO:teuthology.orchestra.run.smithi184.stdout: "val": 0, 2024-09-12T02:18:11.144 INFO:teuthology.orchestra.run.smithi184.stdout: "max": 104857600, 2024-09-12T02:18:11.144 INFO:teuthology.orchestra.run.smithi184.stdout: "get_started": 0, 2024-09-12T02:18:11.144 INFO:teuthology.orchestra.run.smithi184.stdout: "get": 0, 2024-09-12T02:18:11.144 INFO:teuthology.orchestra.run.smithi184.stdout: "get_sum": 0, 2024-09-12T02:18:11.144 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_fail": 0, 2024-09-12T02:18:11.144 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_success": 0, 2024-09-12T02:18:11.144 INFO:teuthology.orchestra.run.smithi184.stdout: "take": 3927, 2024-09-12T02:18:11.144 INFO:teuthology.orchestra.run.smithi184.stdout: "take_sum": 8060247, 2024-09-12T02:18:11.145 INFO:teuthology.orchestra.run.smithi184.stdout: "put": 3927, 2024-09-12T02:18:11.145 INFO:teuthology.orchestra.run.smithi184.stdout: "put_sum": 8060247, 2024-09-12T02:18:11.145 INFO:teuthology.orchestra.run.smithi184.stdout: "wait": { 2024-09-12T02:18:11.145 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-09-12T02:18:11.145 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-09-12T02:18:11.145 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-09-12T02:18:11.145 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-12T02:18:11.145 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:11.145 INFO:teuthology.orchestra.run.smithi184.stdout: "throttle-objecter_ops": { 2024-09-12T02:18:11.145 INFO:teuthology.orchestra.run.smithi184.stdout: "val": 0, 2024-09-12T02:18:11.146 INFO:teuthology.orchestra.run.smithi184.stdout: "max": 1024, 2024-09-12T02:18:11.146 INFO:teuthology.orchestra.run.smithi184.stdout: "get_started": 0, 2024-09-12T02:18:11.146 INFO:teuthology.orchestra.run.smithi184.stdout: "get": 0, 2024-09-12T02:18:11.146 INFO:teuthology.orchestra.run.smithi184.stdout: "get_sum": 0, 2024-09-12T02:18:11.146 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_fail": 0, 2024-09-12T02:18:11.146 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_success": 0, 2024-09-12T02:18:11.146 INFO:teuthology.orchestra.run.smithi184.stdout: "take": 3927, 2024-09-12T02:18:11.146 INFO:teuthology.orchestra.run.smithi184.stdout: "take_sum": 3927, 2024-09-12T02:18:11.146 INFO:teuthology.orchestra.run.smithi184.stdout: "put": 3927, 2024-09-12T02:18:11.146 INFO:teuthology.orchestra.run.smithi184.stdout: "put_sum": 3927, 2024-09-12T02:18:11.146 INFO:teuthology.orchestra.run.smithi184.stdout: "wait": { 2024-09-12T02:18:11.146 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-09-12T02:18:11.146 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-09-12T02:18:11.147 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-09-12T02:18:11.147 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-12T02:18:11.147 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:11.147 INFO:teuthology.orchestra.run.smithi184.stdout: "throttle-write_buf_throttle": { 2024-09-12T02:18:11.147 INFO:teuthology.orchestra.run.smithi184.stdout: "val": 0, 2024-09-12T02:18:11.147 INFO:teuthology.orchestra.run.smithi184.stdout: "max": 3758096384, 2024-09-12T02:18:11.147 INFO:teuthology.orchestra.run.smithi184.stdout: "get_started": 0, 2024-09-12T02:18:11.147 INFO:teuthology.orchestra.run.smithi184.stdout: "get": 0, 2024-09-12T02:18:11.147 INFO:teuthology.orchestra.run.smithi184.stdout: "get_sum": 0, 2024-09-12T02:18:11.147 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_fail": 0, 2024-09-12T02:18:11.147 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_success": 0, 2024-09-12T02:18:11.147 INFO:teuthology.orchestra.run.smithi184.stdout: "take": 0, 2024-09-12T02:18:11.148 INFO:teuthology.orchestra.run.smithi184.stdout: "take_sum": 0, 2024-09-12T02:18:11.148 INFO:teuthology.orchestra.run.smithi184.stdout: "put": 0, 2024-09-12T02:18:11.148 INFO:teuthology.orchestra.run.smithi184.stdout: "put_sum": 0, 2024-09-12T02:18:11.148 INFO:teuthology.orchestra.run.smithi184.stdout: "wait": { 2024-09-12T02:18:11.148 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-09-12T02:18:11.148 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-09-12T02:18:11.148 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-09-12T02:18:11.148 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-12T02:18:11.148 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:11.148 INFO:teuthology.orchestra.run.smithi184.stdout: "throttle-write_buf_throttle-0x55b2de60ab40": { 2024-09-12T02:18:11.148 INFO:teuthology.orchestra.run.smithi184.stdout: "val": 0, 2024-09-12T02:18:11.148 INFO:teuthology.orchestra.run.smithi184.stdout: "max": 3758096384, 2024-09-12T02:18:11.148 INFO:teuthology.orchestra.run.smithi184.stdout: "get_started": 0, 2024-09-12T02:18:11.149 INFO:teuthology.orchestra.run.smithi184.stdout: "get": 0, 2024-09-12T02:18:11.149 INFO:teuthology.orchestra.run.smithi184.stdout: "get_sum": 0, 2024-09-12T02:18:11.149 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_fail": 0, 2024-09-12T02:18:11.149 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_success": 0, 2024-09-12T02:18:11.149 INFO:teuthology.orchestra.run.smithi184.stdout: "take": 0, 2024-09-12T02:18:11.149 INFO:teuthology.orchestra.run.smithi184.stdout: "take_sum": 0, 2024-09-12T02:18:11.149 INFO:teuthology.orchestra.run.smithi184.stdout: "put": 0, 2024-09-12T02:18:11.149 INFO:teuthology.orchestra.run.smithi184.stdout: "put_sum": 0, 2024-09-12T02:18:11.149 INFO:teuthology.orchestra.run.smithi184.stdout: "wait": { 2024-09-12T02:18:11.149 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-09-12T02:18:11.150 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-09-12T02:18:11.150 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-09-12T02:18:11.150 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-12T02:18:11.150 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-12T02:18:11.150 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-12T02:18:11.877 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-09-12T02:18:11.877 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-09-12T02:18:11.878 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update=0 2024-09-12T02:18:11.878 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update_receipt=0 2024-09-12T02:18:11.878 DEBUG:tasks.check_counter:Getting stats from i 2024-09-12T02:18:11.878 DEBUG:teuthology.orchestra.run.smithi184:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph daemon mds.i perf dump 2024-09-12T02:18:12.124 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T02:18:12.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:18:11 smithi184 ceph-mon[35593]: pgmap v823: 97 pgs: 97 active+clean; 4.3 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-12T02:18:12.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:18:11 smithi033 ceph-mon[35332]: pgmap v823: 97 pgs: 97 active+clean; 4.3 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-12T02:18:12.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:18:11 smithi016 ceph-mon[30947]: pgmap v823: 97 pgs: 97 active+clean; 4.3 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-12T02:18:12.572 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-12T02:18:12.572 INFO:teuthology.orchestra.run.smithi184.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T02:18:12.572 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_messages": 2052, 2024-09-12T02:18:12.572 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_messages": 1380, 2024-09-12T02:18:12.572 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_bytes": 492374, 2024-09-12T02:18:12.573 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_bytes": 273888, 2024-09-12T02:18:12.573 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_created_connections": 8, 2024-09-12T02:18:12.573 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_active_connections": 3, 2024-09-12T02:18:12.573 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_total_time": 0.151672269, 2024-09-12T02:18:12.573 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_send_time": 0.055896359, 2024-09-12T02:18:12.573 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_recv_time": 0.096116589, 2024-09-12T02:18:12.573 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_fast_dispatch_time": 0.007631170, 2024-09-12T02:18:12.573 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T02:18:12.573 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 1379, 2024-09-12T02:18:12.573 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.081608706, 2024-09-12T02:18:12.573 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000059179 2024-09-12T02:18:12.573 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:12.574 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_handle_ack_lat": { 2024-09-12T02:18:12.574 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 3306, 2024-09-12T02:18:12.574 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000687560, 2024-09-12T02:18:12.574 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000207 2024-09-12T02:18:12.574 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:12.574 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-09-12T02:18:12.574 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_encrypted_bytes": 448 2024-09-12T02:18:12.574 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:12.574 INFO:teuthology.orchestra.run.smithi184.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T02:18:12.574 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_messages": 411, 2024-09-12T02:18:12.574 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_messages": 447, 2024-09-12T02:18:12.574 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_bytes": 550452, 2024-09-12T02:18:12.575 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_bytes": 1019583, 2024-09-12T02:18:12.575 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_created_connections": 65, 2024-09-12T02:18:12.575 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_active_connections": 5, 2024-09-12T02:18:12.575 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_total_time": 0.070671359, 2024-09-12T02:18:12.575 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_send_time": 0.026539641, 2024-09-12T02:18:12.575 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_recv_time": 0.046223677, 2024-09-12T02:18:12.575 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_fast_dispatch_time": 0.000336764, 2024-09-12T02:18:12.575 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T02:18:12.575 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 444, 2024-09-12T02:18:12.575 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.032725609, 2024-09-12T02:18:12.575 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000073706 2024-09-12T02:18:12.575 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:12.575 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_handle_ack_lat": { 2024-09-12T02:18:12.575 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 65, 2024-09-12T02:18:12.576 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000006708, 2024-09-12T02:18:12.576 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000103 2024-09-12T02:18:12.576 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:12.576 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_encrypted_bytes": 539856, 2024-09-12T02:18:12.576 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_encrypted_bytes": 356256 2024-09-12T02:18:12.576 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:12.576 INFO:teuthology.orchestra.run.smithi184.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T02:18:12.576 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_messages": 106, 2024-09-12T02:18:12.576 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_messages": 286, 2024-09-12T02:18:12.576 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_bytes": 31458, 2024-09-12T02:18:12.576 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_bytes": 680799, 2024-09-12T02:18:12.576 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_created_connections": 28, 2024-09-12T02:18:12.577 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_active_connections": 7, 2024-09-12T02:18:12.577 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_total_time": 0.027614301, 2024-09-12T02:18:12.577 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_send_time": 0.012411721, 2024-09-12T02:18:12.577 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_recv_time": 0.012510895, 2024-09-12T02:18:12.577 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_fast_dispatch_time": 0.000584567, 2024-09-12T02:18:12.577 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T02:18:12.577 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 286, 2024-09-12T02:18:12.577 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.023516198, 2024-09-12T02:18:12.577 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000082224 2024-09-12T02:18:12.577 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:12.577 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_handle_ack_lat": { 2024-09-12T02:18:12.577 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 158, 2024-09-12T02:18:12.578 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000028611, 2024-09-12T02:18:12.578 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000181 2024-09-12T02:18:12.578 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:12.578 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-12T02:18:12.578 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_encrypted_bytes": 401200 2024-09-12T02:18:12.578 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:12.578 INFO:teuthology.orchestra.run.smithi184.stdout: "cct": { 2024-09-12T02:18:12.578 INFO:teuthology.orchestra.run.smithi184.stdout: "total_workers": 1, 2024-09-12T02:18:12.578 INFO:teuthology.orchestra.run.smithi184.stdout: "unhealthy_workers": 0 2024-09-12T02:18:12.578 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:12.578 INFO:teuthology.orchestra.run.smithi184.stdout: "finisher-MDSRank": { 2024-09-12T02:18:12.578 INFO:teuthology.orchestra.run.smithi184.stdout: "queue_len": 0, 2024-09-12T02:18:12.578 INFO:teuthology.orchestra.run.smithi184.stdout: "complete_latency": { 2024-09-12T02:18:12.579 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 42, 2024-09-12T02:18:12.579 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.001718468, 2024-09-12T02:18:12.579 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000040915 2024-09-12T02:18:12.579 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-12T02:18:12.579 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:12.579 INFO:teuthology.orchestra.run.smithi184.stdout: "finisher-PurgeQueue": { 2024-09-12T02:18:12.579 INFO:teuthology.orchestra.run.smithi184.stdout: "queue_len": 0, 2024-09-12T02:18:12.579 INFO:teuthology.orchestra.run.smithi184.stdout: "complete_latency": { 2024-09-12T02:18:12.579 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 2, 2024-09-12T02:18:12.579 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000015247, 2024-09-12T02:18:12.579 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000007623 2024-09-12T02:18:12.579 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-12T02:18:12.579 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:12.580 INFO:teuthology.orchestra.run.smithi184.stdout: "mds": { 2024-09-12T02:18:12.580 INFO:teuthology.orchestra.run.smithi184.stdout: "request": 2, 2024-09-12T02:18:12.580 INFO:teuthology.orchestra.run.smithi184.stdout: "reply": 1, 2024-09-12T02:18:12.580 INFO:teuthology.orchestra.run.smithi184.stdout: "reply_latency": { 2024-09-12T02:18:12.580 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 1, 2024-09-12T02:18:12.580 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000154031, 2024-09-12T02:18:12.580 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000154031 2024-09-12T02:18:12.580 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:12.580 INFO:teuthology.orchestra.run.smithi184.stdout: "slow_reply": 0, 2024-09-12T02:18:12.580 INFO:teuthology.orchestra.run.smithi184.stdout: "forward": 1, 2024-09-12T02:18:12.580 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_fetch_complete": 0, 2024-09-12T02:18:12.580 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_fetch_keys": 0, 2024-09-12T02:18:12.580 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_commit": 11, 2024-09-12T02:18:12.580 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_split": 0, 2024-09-12T02:18:12.581 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_merge": 0, 2024-09-12T02:18:12.581 INFO:teuthology.orchestra.run.smithi184.stdout: "inodes": 16, 2024-09-12T02:18:12.581 INFO:teuthology.orchestra.run.smithi184.stdout: "inodes_top": 0, 2024-09-12T02:18:12.581 INFO:teuthology.orchestra.run.smithi184.stdout: "inodes_bottom": 16, 2024-09-12T02:18:12.581 INFO:teuthology.orchestra.run.smithi184.stdout: "inodes_pin_tail": 0, 2024-09-12T02:18:12.581 INFO:teuthology.orchestra.run.smithi184.stdout: "inodes_pinned": 13, 2024-09-12T02:18:12.581 INFO:teuthology.orchestra.run.smithi184.stdout: "inodes_expired": 3, 2024-09-12T02:18:12.581 INFO:teuthology.orchestra.run.smithi184.stdout: "inodes_with_caps": 0, 2024-09-12T02:18:12.581 INFO:teuthology.orchestra.run.smithi184.stdout: "caps": 0, 2024-09-12T02:18:12.581 INFO:teuthology.orchestra.run.smithi184.stdout: "subtrees": 3, 2024-09-12T02:18:12.581 INFO:teuthology.orchestra.run.smithi184.stdout: "traverse": 44, 2024-09-12T02:18:12.581 INFO:teuthology.orchestra.run.smithi184.stdout: "traverse_hit": 34, 2024-09-12T02:18:12.582 INFO:teuthology.orchestra.run.smithi184.stdout: "traverse_forward": 0, 2024-09-12T02:18:12.582 INFO:teuthology.orchestra.run.smithi184.stdout: "traverse_discover": 8, 2024-09-12T02:18:12.582 INFO:teuthology.orchestra.run.smithi184.stdout: "traverse_dir_fetch": 0, 2024-09-12T02:18:12.582 INFO:teuthology.orchestra.run.smithi184.stdout: "traverse_remote_ino": 0, 2024-09-12T02:18:12.582 INFO:teuthology.orchestra.run.smithi184.stdout: "traverse_lock": 0, 2024-09-12T02:18:12.582 INFO:teuthology.orchestra.run.smithi184.stdout: "load_cent": 0, 2024-09-12T02:18:12.582 INFO:teuthology.orchestra.run.smithi184.stdout: "q": 0, 2024-09-12T02:18:12.582 INFO:teuthology.orchestra.run.smithi184.stdout: "exported": 0, 2024-09-12T02:18:12.582 INFO:teuthology.orchestra.run.smithi184.stdout: "exported_inodes": 0, 2024-09-12T02:18:12.582 INFO:teuthology.orchestra.run.smithi184.stdout: "imported": 0, 2024-09-12T02:18:12.582 INFO:teuthology.orchestra.run.smithi184.stdout: "imported_inodes": 0, 2024-09-12T02:18:12.582 INFO:teuthology.orchestra.run.smithi184.stdout: "openino_dir_fetch": 0, 2024-09-12T02:18:12.582 INFO:teuthology.orchestra.run.smithi184.stdout: "openino_backtrace_fetch": 0, 2024-09-12T02:18:12.582 INFO:teuthology.orchestra.run.smithi184.stdout: "openino_peer_discover": 0, 2024-09-12T02:18:12.583 INFO:teuthology.orchestra.run.smithi184.stdout: "root_rfiles": 5, 2024-09-12T02:18:12.583 INFO:teuthology.orchestra.run.smithi184.stdout: "root_rbytes": 298, 2024-09-12T02:18:12.583 INFO:teuthology.orchestra.run.smithi184.stdout: "root_rsnaps": 0, 2024-09-12T02:18:12.583 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_backtrace_fetch": 0, 2024-09-12T02:18:12.583 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_set_tag": 0, 2024-09-12T02:18:12.583 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_backtrace_repaired": 0, 2024-09-12T02:18:12.583 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_inotable_repaired": 0, 2024-09-12T02:18:12.583 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_dir_inodes": 0, 2024-09-12T02:18:12.583 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_dir_base_inodes": 0, 2024-09-12T02:18:12.583 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_dirfrag_rstats": 0, 2024-09-12T02:18:12.584 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_file_inodes": 0, 2024-09-12T02:18:12.584 INFO:teuthology.orchestra.run.smithi184.stdout: "handle_inode_file_caps": 0, 2024-09-12T02:18:12.584 INFO:teuthology.orchestra.run.smithi184.stdout: "ceph_cap_op_revoke": 0, 2024-09-12T02:18:12.584 INFO:teuthology.orchestra.run.smithi184.stdout: "ceph_cap_op_grant": 0, 2024-09-12T02:18:12.584 INFO:teuthology.orchestra.run.smithi184.stdout: "ceph_cap_op_trunc": 0, 2024-09-12T02:18:12.584 INFO:teuthology.orchestra.run.smithi184.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-12T02:18:12.584 INFO:teuthology.orchestra.run.smithi184.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-12T02:18:12.584 INFO:teuthology.orchestra.run.smithi184.stdout: "handle_client_caps": 0, 2024-09-12T02:18:12.584 INFO:teuthology.orchestra.run.smithi184.stdout: "handle_client_caps_dirty": 0, 2024-09-12T02:18:12.584 INFO:teuthology.orchestra.run.smithi184.stdout: "handle_client_cap_release": 0, 2024-09-12T02:18:12.585 INFO:teuthology.orchestra.run.smithi184.stdout: "process_request_cap_release": 1 2024-09-12T02:18:12.585 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:12.585 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_cache": { 2024-09-12T02:18:12.585 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_update": 0, 2024-09-12T02:18:12.585 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_update_receipt": 16, 2024-09-12T02:18:12.585 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_try_discover": 12, 2024-09-12T02:18:12.585 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_send_discover": 11, 2024-09-12T02:18:12.585 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_handle_discover": 0, 2024-09-12T02:18:12.585 INFO:teuthology.orchestra.run.smithi184.stdout: "num_strays": 0, 2024-09-12T02:18:12.585 INFO:teuthology.orchestra.run.smithi184.stdout: "num_strays_delayed": 0, 2024-09-12T02:18:12.585 INFO:teuthology.orchestra.run.smithi184.stdout: "num_strays_enqueuing": 0, 2024-09-12T02:18:12.585 INFO:teuthology.orchestra.run.smithi184.stdout: "strays_created": 0, 2024-09-12T02:18:12.586 INFO:teuthology.orchestra.run.smithi184.stdout: "strays_enqueued": 0, 2024-09-12T02:18:12.586 INFO:teuthology.orchestra.run.smithi184.stdout: "strays_reintegrated": 0, 2024-09-12T02:18:12.586 INFO:teuthology.orchestra.run.smithi184.stdout: "strays_migrated": 0, 2024-09-12T02:18:12.586 INFO:teuthology.orchestra.run.smithi184.stdout: "num_recovering_processing": 0, 2024-09-12T02:18:12.586 INFO:teuthology.orchestra.run.smithi184.stdout: "num_recovering_enqueued": 0, 2024-09-12T02:18:12.586 INFO:teuthology.orchestra.run.smithi184.stdout: "num_recovering_prioritized": 0, 2024-09-12T02:18:12.586 INFO:teuthology.orchestra.run.smithi184.stdout: "recovery_started": 0, 2024-09-12T02:18:12.586 INFO:teuthology.orchestra.run.smithi184.stdout: "recovery_completed": 0, 2024-09-12T02:18:12.586 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_quiesce_path": 14, 2024-09-12T02:18:12.586 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_quiesce_inode": 83, 2024-09-12T02:18:12.587 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_enqueue_scrub": 0, 2024-09-12T02:18:12.587 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_exportdir": 0, 2024-09-12T02:18:12.587 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_flush": 0, 2024-09-12T02:18:12.587 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_fragmentdir": 0, 2024-09-12T02:18:12.587 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_fragstats": 0, 2024-09-12T02:18:12.587 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_inodestats": 0 2024-09-12T02:18:12.587 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:12.587 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_log": { 2024-09-12T02:18:12.587 INFO:teuthology.orchestra.run.smithi184.stdout: "evlrg": 0, 2024-09-12T02:18:12.587 INFO:teuthology.orchestra.run.smithi184.stdout: "evadd": 4, 2024-09-12T02:18:12.587 INFO:teuthology.orchestra.run.smithi184.stdout: "evex": 0, 2024-09-12T02:18:12.587 INFO:teuthology.orchestra.run.smithi184.stdout: "evtrm": 0, 2024-09-12T02:18:12.587 INFO:teuthology.orchestra.run.smithi184.stdout: "ev": 4, 2024-09-12T02:18:12.587 INFO:teuthology.orchestra.run.smithi184.stdout: "evexg": 0, 2024-09-12T02:18:12.588 INFO:teuthology.orchestra.run.smithi184.stdout: "evexd": 0, 2024-09-12T02:18:12.588 INFO:teuthology.orchestra.run.smithi184.stdout: "segadd": 2, 2024-09-12T02:18:12.588 INFO:teuthology.orchestra.run.smithi184.stdout: "segex": 0, 2024-09-12T02:18:12.588 INFO:teuthology.orchestra.run.smithi184.stdout: "segtrm": 0, 2024-09-12T02:18:12.588 INFO:teuthology.orchestra.run.smithi184.stdout: "seg": 2, 2024-09-12T02:18:12.588 INFO:teuthology.orchestra.run.smithi184.stdout: "segmjr": 2, 2024-09-12T02:18:12.588 INFO:teuthology.orchestra.run.smithi184.stdout: "segexg": 0, 2024-09-12T02:18:12.588 INFO:teuthology.orchestra.run.smithi184.stdout: "segexd": 0, 2024-09-12T02:18:12.588 INFO:teuthology.orchestra.run.smithi184.stdout: "expos": 4194304, 2024-09-12T02:18:12.588 INFO:teuthology.orchestra.run.smithi184.stdout: "wrpos": 4195426, 2024-09-12T02:18:12.588 INFO:teuthology.orchestra.run.smithi184.stdout: "rdpos": 0, 2024-09-12T02:18:12.588 INFO:teuthology.orchestra.run.smithi184.stdout: "jlat": { 2024-09-12T02:18:12.588 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 3, 2024-09-12T02:18:12.588 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.005450204, 2024-09-12T02:18:12.588 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.001816734 2024-09-12T02:18:12.589 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:12.589 INFO:teuthology.orchestra.run.smithi184.stdout: "replayed": 0 2024-09-12T02:18:12.589 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:12.589 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_mem": { 2024-09-12T02:18:12.589 INFO:teuthology.orchestra.run.smithi184.stdout: "ino": 20, 2024-09-12T02:18:12.589 INFO:teuthology.orchestra.run.smithi184.stdout: "ino+": 21, 2024-09-12T02:18:12.589 INFO:teuthology.orchestra.run.smithi184.stdout: "ino-": 1, 2024-09-12T02:18:12.589 INFO:teuthology.orchestra.run.smithi184.stdout: "dir": 19, 2024-09-12T02:18:12.589 INFO:teuthology.orchestra.run.smithi184.stdout: "dir+": 21, 2024-09-12T02:18:12.590 INFO:teuthology.orchestra.run.smithi184.stdout: "dir-": 2, 2024-09-12T02:18:12.590 INFO:teuthology.orchestra.run.smithi184.stdout: "dn": 16, 2024-09-12T02:18:12.590 INFO:teuthology.orchestra.run.smithi184.stdout: "dn+": 19, 2024-09-12T02:18:12.590 INFO:teuthology.orchestra.run.smithi184.stdout: "dn-": 3, 2024-09-12T02:18:12.590 INFO:teuthology.orchestra.run.smithi184.stdout: "cap": 0, 2024-09-12T02:18:12.590 INFO:teuthology.orchestra.run.smithi184.stdout: "cap+": 1, 2024-09-12T02:18:12.590 INFO:teuthology.orchestra.run.smithi184.stdout: "cap-": 1, 2024-09-12T02:18:12.590 INFO:teuthology.orchestra.run.smithi184.stdout: "rss": 51132, 2024-09-12T02:18:12.590 INFO:teuthology.orchestra.run.smithi184.stdout: "heap": 223520 2024-09-12T02:18:12.590 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:12.590 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_server": { 2024-09-12T02:18:12.591 INFO:teuthology.orchestra.run.smithi184.stdout: "dispatch_client_request": 3, 2024-09-12T02:18:12.591 INFO:teuthology.orchestra.run.smithi184.stdout: "dispatch_server_request": 0, 2024-09-12T02:18:12.591 INFO:teuthology.orchestra.run.smithi184.stdout: "handle_client_request": 2, 2024-09-12T02:18:12.591 INFO:teuthology.orchestra.run.smithi184.stdout: "handle_client_session": 4, 2024-09-12T02:18:12.591 INFO:teuthology.orchestra.run.smithi184.stdout: "handle_peer_request": 0, 2024-09-12T02:18:12.591 INFO:teuthology.orchestra.run.smithi184.stdout: "req_create_latency": { 2024-09-12T02:18:12.591 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-09-12T02:18:12.591 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-09-12T02:18:12.591 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-09-12T02:18:12.591 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:12.591 INFO:teuthology.orchestra.run.smithi184.stdout: "req_getattr_latency": { 2024-09-12T02:18:12.591 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 1, 2024-09-12T02:18:12.591 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000154031, 2024-09-12T02:18:12.591 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000154031 2024-09-12T02:18:12.591 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:12.592 INFO:teuthology.orchestra.run.smithi184.stdout: "req_getfilelock_latency": { 2024-09-12T02:18:12.592 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-09-12T02:18:12.592 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-09-12T02:18:12.592 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-09-12T02:18:12.592 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:12.592 INFO:teuthology.orchestra.run.smithi184.stdout: "req_link_latency": { 2024-09-12T02:18:12.592 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-09-12T02:18:12.592 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-09-12T02:18:12.592 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-09-12T02:18:12.592 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:12.593 INFO:teuthology.orchestra.run.smithi184.stdout: "req_lookup_latency": { 2024-09-12T02:18:12.593 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-09-12T02:18:12.593 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-09-12T02:18:12.593 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-09-12T02:18:12.593 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:12.593 INFO:teuthology.orchestra.run.smithi184.stdout: "req_lookuphash_latency": { 2024-09-12T02:18:12.593 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-09-12T02:18:12.593 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-09-12T02:18:12.593 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-09-12T02:18:12.593 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:12.593 INFO:teuthology.orchestra.run.smithi184.stdout: "req_lookupino_latency": { 2024-09-12T02:18:12.593 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-09-12T02:18:12.593 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-09-12T02:18:12.594 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-09-12T02:18:12.594 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:12.594 INFO:teuthology.orchestra.run.smithi184.stdout: "req_lookupname_latency": { 2024-09-12T02:18:12.594 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-09-12T02:18:12.594 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-09-12T02:18:12.594 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-09-12T02:18:12.594 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:12.594 INFO:teuthology.orchestra.run.smithi184.stdout: "req_lookupparent_latency": { 2024-09-12T02:18:12.594 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-09-12T02:18:12.594 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-09-12T02:18:12.594 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-09-12T02:18:12.595 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:12.595 INFO:teuthology.orchestra.run.smithi184.stdout: "req_lookupsnap_latency": { 2024-09-12T02:18:12.595 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-09-12T02:18:12.595 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-09-12T02:18:12.595 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-09-12T02:18:12.595 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:12.595 INFO:teuthology.orchestra.run.smithi184.stdout: "req_lssnap_latency": { 2024-09-12T02:18:12.595 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-09-12T02:18:12.595 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-09-12T02:18:12.595 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-09-12T02:18:12.595 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:12.595 INFO:teuthology.orchestra.run.smithi184.stdout: "req_mkdir_latency": { 2024-09-12T02:18:12.596 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-09-12T02:18:12.596 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-09-12T02:18:12.596 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-09-12T02:18:12.596 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:12.597 INFO:teuthology.orchestra.run.smithi184.stdout: "req_mknod_latency": { 2024-09-12T02:18:12.597 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-09-12T02:18:12.597 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-09-12T02:18:12.597 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-09-12T02:18:12.597 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:12.597 INFO:teuthology.orchestra.run.smithi184.stdout: "req_mksnap_latency": { 2024-09-12T02:18:12.597 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-09-12T02:18:12.597 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-09-12T02:18:12.597 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-09-12T02:18:12.597 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:12.597 INFO:teuthology.orchestra.run.smithi184.stdout: "req_open_latency": { 2024-09-12T02:18:12.597 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-09-12T02:18:12.597 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-09-12T02:18:12.598 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-09-12T02:18:12.598 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:12.598 INFO:teuthology.orchestra.run.smithi184.stdout: "req_readdir_latency": { 2024-09-12T02:18:12.598 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-09-12T02:18:12.598 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-09-12T02:18:12.598 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-09-12T02:18:12.598 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:12.598 INFO:teuthology.orchestra.run.smithi184.stdout: "req_rename_latency": { 2024-09-12T02:18:12.598 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-09-12T02:18:12.599 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-09-12T02:18:12.599 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-09-12T02:18:12.599 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:12.599 INFO:teuthology.orchestra.run.smithi184.stdout: "req_renamesnap_latency": { 2024-09-12T02:18:12.599 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-09-12T02:18:12.599 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-09-12T02:18:12.599 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-09-12T02:18:12.599 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:12.599 INFO:teuthology.orchestra.run.smithi184.stdout: "req_snapdiff_latency": { 2024-09-12T02:18:12.599 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-09-12T02:18:12.599 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-09-12T02:18:12.599 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-09-12T02:18:12.600 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:12.600 INFO:teuthology.orchestra.run.smithi184.stdout: "req_rmdir_latency": { 2024-09-12T02:18:12.600 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-09-12T02:18:12.600 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-09-12T02:18:12.600 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-09-12T02:18:12.600 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:12.600 INFO:teuthology.orchestra.run.smithi184.stdout: "req_rmsnap_latency": { 2024-09-12T02:18:12.600 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-09-12T02:18:12.600 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-09-12T02:18:12.600 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-09-12T02:18:12.600 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:12.600 INFO:teuthology.orchestra.run.smithi184.stdout: "req_rmxattr_latency": { 2024-09-12T02:18:12.600 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-09-12T02:18:12.601 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-09-12T02:18:12.601 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-09-12T02:18:12.601 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:12.601 INFO:teuthology.orchestra.run.smithi184.stdout: "req_setattr_latency": { 2024-09-12T02:18:12.601 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-09-12T02:18:12.601 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-09-12T02:18:12.601 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-09-12T02:18:12.601 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:12.601 INFO:teuthology.orchestra.run.smithi184.stdout: "req_setdirlayout_latency": { 2024-09-12T02:18:12.601 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-09-12T02:18:12.601 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-09-12T02:18:12.601 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-09-12T02:18:12.602 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:12.602 INFO:teuthology.orchestra.run.smithi184.stdout: "req_setfilelock_latency": { 2024-09-12T02:18:12.602 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-09-12T02:18:12.602 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-09-12T02:18:12.602 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-09-12T02:18:12.602 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:12.602 INFO:teuthology.orchestra.run.smithi184.stdout: "req_setlayout_latency": { 2024-09-12T02:18:12.602 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-09-12T02:18:12.602 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-09-12T02:18:12.603 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-09-12T02:18:12.603 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:12.603 INFO:teuthology.orchestra.run.smithi184.stdout: "req_setxattr_latency": { 2024-09-12T02:18:12.603 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-09-12T02:18:12.603 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-09-12T02:18:12.603 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-09-12T02:18:12.603 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:12.603 INFO:teuthology.orchestra.run.smithi184.stdout: "req_symlink_latency": { 2024-09-12T02:18:12.603 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-09-12T02:18:12.603 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-09-12T02:18:12.603 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-09-12T02:18:12.603 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:12.604 INFO:teuthology.orchestra.run.smithi184.stdout: "req_unlink_latency": { 2024-09-12T02:18:12.604 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-09-12T02:18:12.604 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-09-12T02:18:12.604 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-09-12T02:18:12.604 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:12.604 INFO:teuthology.orchestra.run.smithi184.stdout: "cap_revoke_eviction": 0, 2024-09-12T02:18:12.604 INFO:teuthology.orchestra.run.smithi184.stdout: "cap_acquisition_throttle": 0, 2024-09-12T02:18:12.604 INFO:teuthology.orchestra.run.smithi184.stdout: "req_getvxattr_latency": { 2024-09-12T02:18:12.604 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-09-12T02:18:12.604 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-09-12T02:18:12.604 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-09-12T02:18:12.604 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-12T02:18:12.605 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:12.605 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_sessions": { 2024-09-12T02:18:12.605 INFO:teuthology.orchestra.run.smithi184.stdout: "session_count": 0, 2024-09-12T02:18:12.605 INFO:teuthology.orchestra.run.smithi184.stdout: "session_add": 1, 2024-09-12T02:18:12.605 INFO:teuthology.orchestra.run.smithi184.stdout: "session_remove": 1, 2024-09-12T02:18:12.605 INFO:teuthology.orchestra.run.smithi184.stdout: "sessions_open": 0, 2024-09-12T02:18:12.605 INFO:teuthology.orchestra.run.smithi184.stdout: "sessions_stale": 0, 2024-09-12T02:18:12.605 INFO:teuthology.orchestra.run.smithi184.stdout: "total_load": 1, 2024-09-12T02:18:12.605 INFO:teuthology.orchestra.run.smithi184.stdout: "average_load": 1, 2024-09-12T02:18:12.605 INFO:teuthology.orchestra.run.smithi184.stdout: "avg_session_uptime": 88, 2024-09-12T02:18:12.605 INFO:teuthology.orchestra.run.smithi184.stdout: "mdthresh_evicted": 0 2024-09-12T02:18:12.605 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:12.605 INFO:teuthology.orchestra.run.smithi184.stdout: "mempool": { 2024-09-12T02:18:12.606 INFO:teuthology.orchestra.run.smithi184.stdout: "bloom_filter_bytes": 0, 2024-09-12T02:18:12.606 INFO:teuthology.orchestra.run.smithi184.stdout: "bloom_filter_items": 0, 2024-09-12T02:18:12.606 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T02:18:12.606 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_alloc_items": 0, 2024-09-12T02:18:12.606 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T02:18:12.606 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_data_items": 0, 2024-09-12T02:18:12.606 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T02:18:12.606 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T02:18:12.606 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T02:18:12.606 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T02:18:12.606 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T02:18:12.606 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_other_items": 0, 2024-09-12T02:18:12.607 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T02:18:12.607 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T02:18:12.607 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_extent_bytes": 0, 2024-09-12T02:18:12.607 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_extent_items": 0, 2024-09-12T02:18:12.607 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_blob_bytes": 0, 2024-09-12T02:18:12.607 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_blob_items": 0, 2024-09-12T02:18:12.607 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T02:18:12.607 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T02:18:12.607 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T02:18:12.607 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T02:18:12.607 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T02:18:12.607 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_fsck_items": 0, 2024-09-12T02:18:12.607 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_txc_bytes": 0, 2024-09-12T02:18:12.607 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_txc_items": 0, 2024-09-12T02:18:12.608 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T02:18:12.608 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T02:18:12.608 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_writing_bytes": 0, 2024-09-12T02:18:12.608 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_writing_items": 0, 2024-09-12T02:18:12.608 INFO:teuthology.orchestra.run.smithi184.stdout: "bluefs_bytes": 0, 2024-09-12T02:18:12.608 INFO:teuthology.orchestra.run.smithi184.stdout: "bluefs_items": 0, 2024-09-12T02:18:12.608 INFO:teuthology.orchestra.run.smithi184.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T02:18:12.608 INFO:teuthology.orchestra.run.smithi184.stdout: "bluefs_file_reader_items": 0, 2024-09-12T02:18:12.608 INFO:teuthology.orchestra.run.smithi184.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T02:18:12.608 INFO:teuthology.orchestra.run.smithi184.stdout: "bluefs_file_writer_items": 0, 2024-09-12T02:18:12.608 INFO:teuthology.orchestra.run.smithi184.stdout: "buffer_anon_bytes": 247159, 2024-09-12T02:18:12.608 INFO:teuthology.orchestra.run.smithi184.stdout: "buffer_anon_items": 84, 2024-09-12T02:18:12.608 INFO:teuthology.orchestra.run.smithi184.stdout: "buffer_meta_bytes": 0, 2024-09-12T02:18:12.609 INFO:teuthology.orchestra.run.smithi184.stdout: "buffer_meta_items": 0, 2024-09-12T02:18:12.609 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_bytes": 0, 2024-09-12T02:18:12.609 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_items": 0, 2024-09-12T02:18:12.609 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_mapbl_bytes": 0, 2024-09-12T02:18:12.609 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_mapbl_items": 0, 2024-09-12T02:18:12.609 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_pglog_bytes": 0, 2024-09-12T02:18:12.609 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_pglog_items": 0, 2024-09-12T02:18:12.609 INFO:teuthology.orchestra.run.smithi184.stdout: "osdmap_bytes": 7304, 2024-09-12T02:18:12.609 INFO:teuthology.orchestra.run.smithi184.stdout: "osdmap_items": 150, 2024-09-12T02:18:12.609 INFO:teuthology.orchestra.run.smithi184.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T02:18:12.609 INFO:teuthology.orchestra.run.smithi184.stdout: "osdmap_mapping_items": 0, 2024-09-12T02:18:12.609 INFO:teuthology.orchestra.run.smithi184.stdout: "pgmap_bytes": 0, 2024-09-12T02:18:12.609 INFO:teuthology.orchestra.run.smithi184.stdout: "pgmap_items": 0, 2024-09-12T02:18:12.610 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_co_bytes": 80973, 2024-09-12T02:18:12.610 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_co_items": 328, 2024-09-12T02:18:12.610 INFO:teuthology.orchestra.run.smithi184.stdout: "unittest_1_bytes": 0, 2024-09-12T02:18:12.610 INFO:teuthology.orchestra.run.smithi184.stdout: "unittest_1_items": 0, 2024-09-12T02:18:12.610 INFO:teuthology.orchestra.run.smithi184.stdout: "unittest_2_bytes": 0, 2024-09-12T02:18:12.610 INFO:teuthology.orchestra.run.smithi184.stdout: "unittest_2_items": 0 2024-09-12T02:18:12.610 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:12.610 INFO:teuthology.orchestra.run.smithi184.stdout: "objecter": { 2024-09-12T02:18:12.610 INFO:teuthology.orchestra.run.smithi184.stdout: "op_active": 0, 2024-09-12T02:18:12.610 INFO:teuthology.orchestra.run.smithi184.stdout: "op_laggy": 0, 2024-09-12T02:18:12.610 INFO:teuthology.orchestra.run.smithi184.stdout: "op_send": 38, 2024-09-12T02:18:12.610 INFO:teuthology.orchestra.run.smithi184.stdout: "op_send_bytes": 11334, 2024-09-12T02:18:12.610 INFO:teuthology.orchestra.run.smithi184.stdout: "op_resend": 28, 2024-09-12T02:18:12.610 INFO:teuthology.orchestra.run.smithi184.stdout: "op_reply": 38, 2024-09-12T02:18:12.611 INFO:teuthology.orchestra.run.smithi184.stdout: "op_latency": { 2024-09-12T02:18:12.611 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 38, 2024-09-12T02:18:12.611 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.201995692, 2024-09-12T02:18:12.611 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.005315676 2024-09-12T02:18:12.611 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:12.611 INFO:teuthology.orchestra.run.smithi184.stdout: "op_inflight": 0, 2024-09-12T02:18:12.611 INFO:teuthology.orchestra.run.smithi184.stdout: "oplen_avg": { 2024-09-12T02:18:12.611 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 38, 2024-09-12T02:18:12.611 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 60 2024-09-12T02:18:12.611 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:12.611 INFO:teuthology.orchestra.run.smithi184.stdout: "op": 38, 2024-09-12T02:18:12.612 INFO:teuthology.orchestra.run.smithi184.stdout: "op_r": 0, 2024-09-12T02:18:12.612 INFO:teuthology.orchestra.run.smithi184.stdout: "op_w": 38, 2024-09-12T02:18:12.612 INFO:teuthology.orchestra.run.smithi184.stdout: "op_rmw": 0, 2024-09-12T02:18:12.612 INFO:teuthology.orchestra.run.smithi184.stdout: "op_pg": 0, 2024-09-12T02:18:12.612 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_stat": 0, 2024-09-12T02:18:12.612 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_create": 10, 2024-09-12T02:18:12.612 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_read": 0, 2024-09-12T02:18:12.612 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_write": 3, 2024-09-12T02:18:12.612 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_writefull": 6, 2024-09-12T02:18:12.612 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_writesame": 0, 2024-09-12T02:18:12.612 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_append": 0, 2024-09-12T02:18:12.612 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_zero": 0, 2024-09-12T02:18:12.613 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_truncate": 0, 2024-09-12T02:18:12.613 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_delete": 6, 2024-09-12T02:18:12.613 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_mapext": 0, 2024-09-12T02:18:12.613 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_sparse_read": 0, 2024-09-12T02:18:12.613 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_clonerange": 0, 2024-09-12T02:18:12.613 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_getxattr": 0, 2024-09-12T02:18:12.613 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_setxattr": 20, 2024-09-12T02:18:12.613 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_cmpxattr": 0, 2024-09-12T02:18:12.613 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_rmxattr": 0, 2024-09-12T02:18:12.613 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_resetxattrs": 0, 2024-09-12T02:18:12.613 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_call": 0, 2024-09-12T02:18:12.613 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_watch": 0, 2024-09-12T02:18:12.613 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_notify": 0, 2024-09-12T02:18:12.614 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_src_cmpxattr": 0, 2024-09-12T02:18:12.614 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_pgls": 0, 2024-09-12T02:18:12.614 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_pgls_filter": 0, 2024-09-12T02:18:12.614 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_other": 0, 2024-09-12T02:18:12.614 INFO:teuthology.orchestra.run.smithi184.stdout: "linger_active": 0, 2024-09-12T02:18:12.614 INFO:teuthology.orchestra.run.smithi184.stdout: "linger_send": 0, 2024-09-12T02:18:12.614 INFO:teuthology.orchestra.run.smithi184.stdout: "linger_resend": 0, 2024-09-12T02:18:12.614 INFO:teuthology.orchestra.run.smithi184.stdout: "linger_ping": 0, 2024-09-12T02:18:12.614 INFO:teuthology.orchestra.run.smithi184.stdout: "poolop_active": 0, 2024-09-12T02:18:12.614 INFO:teuthology.orchestra.run.smithi184.stdout: "poolop_send": 0, 2024-09-12T02:18:12.614 INFO:teuthology.orchestra.run.smithi184.stdout: "poolop_resend": 0, 2024-09-12T02:18:12.614 INFO:teuthology.orchestra.run.smithi184.stdout: "poolstat_active": 0, 2024-09-12T02:18:12.614 INFO:teuthology.orchestra.run.smithi184.stdout: "poolstat_send": 0, 2024-09-12T02:18:12.614 INFO:teuthology.orchestra.run.smithi184.stdout: "poolstat_resend": 0, 2024-09-12T02:18:12.615 INFO:teuthology.orchestra.run.smithi184.stdout: "statfs_active": 0, 2024-09-12T02:18:12.615 INFO:teuthology.orchestra.run.smithi184.stdout: "statfs_send": 0, 2024-09-12T02:18:12.615 INFO:teuthology.orchestra.run.smithi184.stdout: "statfs_resend": 0, 2024-09-12T02:18:12.615 INFO:teuthology.orchestra.run.smithi184.stdout: "command_active": 0, 2024-09-12T02:18:12.615 INFO:teuthology.orchestra.run.smithi184.stdout: "command_send": 0, 2024-09-12T02:18:12.615 INFO:teuthology.orchestra.run.smithi184.stdout: "command_resend": 0, 2024-09-12T02:18:12.615 INFO:teuthology.orchestra.run.smithi184.stdout: "map_epoch": 82, 2024-09-12T02:18:12.615 INFO:teuthology.orchestra.run.smithi184.stdout: "map_full": 0, 2024-09-12T02:18:12.615 INFO:teuthology.orchestra.run.smithi184.stdout: "map_inc": 2, 2024-09-12T02:18:12.615 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_sessions": 9, 2024-09-12T02:18:12.615 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_session_open": 12, 2024-09-12T02:18:12.615 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_session_close": 3, 2024-09-12T02:18:12.615 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_laggy": 0, 2024-09-12T02:18:12.616 INFO:teuthology.orchestra.run.smithi184.stdout: "omap_wr": 14, 2024-09-12T02:18:12.616 INFO:teuthology.orchestra.run.smithi184.stdout: "omap_rd": 0, 2024-09-12T02:18:12.616 INFO:teuthology.orchestra.run.smithi184.stdout: "omap_del": 1 2024-09-12T02:18:12.616 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:12.616 INFO:teuthology.orchestra.run.smithi184.stdout: "oft": { 2024-09-12T02:18:12.616 INFO:teuthology.orchestra.run.smithi184.stdout: "omap_total_objs": 1, 2024-09-12T02:18:12.616 INFO:teuthology.orchestra.run.smithi184.stdout: "omap_total_kv_pairs": 0, 2024-09-12T02:18:12.616 INFO:teuthology.orchestra.run.smithi184.stdout: "omap_total_updates": 0, 2024-09-12T02:18:12.616 INFO:teuthology.orchestra.run.smithi184.stdout: "omap_total_removes": 0 2024-09-12T02:18:12.616 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:12.616 INFO:teuthology.orchestra.run.smithi184.stdout: "osd-slow-ops": { 2024-09-12T02:18:12.616 INFO:teuthology.orchestra.run.smithi184.stdout: "slow_ops_count": 97 2024-09-12T02:18:12.616 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:12.617 INFO:teuthology.orchestra.run.smithi184.stdout: "purge_queue": { 2024-09-12T02:18:12.617 INFO:teuthology.orchestra.run.smithi184.stdout: "pq_executing_ops": 0, 2024-09-12T02:18:12.617 INFO:teuthology.orchestra.run.smithi184.stdout: "pq_executing_ops_high_water": 0, 2024-09-12T02:18:12.617 INFO:teuthology.orchestra.run.smithi184.stdout: "pq_executing": 0, 2024-09-12T02:18:12.617 INFO:teuthology.orchestra.run.smithi184.stdout: "pq_executing_high_water": 0, 2024-09-12T02:18:12.617 INFO:teuthology.orchestra.run.smithi184.stdout: "pq_executed_ops": 0, 2024-09-12T02:18:12.617 INFO:teuthology.orchestra.run.smithi184.stdout: "pq_executed": 0, 2024-09-12T02:18:12.617 INFO:teuthology.orchestra.run.smithi184.stdout: "pq_item_in_journal": 0 2024-09-12T02:18:12.617 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:12.617 INFO:teuthology.orchestra.run.smithi184.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T02:18:12.617 INFO:teuthology.orchestra.run.smithi184.stdout: "val": 0, 2024-09-12T02:18:12.617 INFO:teuthology.orchestra.run.smithi184.stdout: "max": 104857600, 2024-09-12T02:18:12.618 INFO:teuthology.orchestra.run.smithi184.stdout: "get_started": 0, 2024-09-12T02:18:12.618 INFO:teuthology.orchestra.run.smithi184.stdout: "get": 2565, 2024-09-12T02:18:12.618 INFO:teuthology.orchestra.run.smithi184.stdout: "get_sum": 822770, 2024-09-12T02:18:12.618 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_fail": 0, 2024-09-12T02:18:12.618 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_success": 2565, 2024-09-12T02:18:12.618 INFO:teuthology.orchestra.run.smithi184.stdout: "take": 0, 2024-09-12T02:18:12.618 INFO:teuthology.orchestra.run.smithi184.stdout: "take_sum": 0, 2024-09-12T02:18:12.618 INFO:teuthology.orchestra.run.smithi184.stdout: "put": 2565, 2024-09-12T02:18:12.618 INFO:teuthology.orchestra.run.smithi184.stdout: "put_sum": 822770, 2024-09-12T02:18:12.618 INFO:teuthology.orchestra.run.smithi184.stdout: "wait": { 2024-09-12T02:18:12.619 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-09-12T02:18:12.619 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-09-12T02:18:12.619 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-09-12T02:18:12.619 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-12T02:18:12.619 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:12.619 INFO:teuthology.orchestra.run.smithi184.stdout: "throttle-objecter_bytes": { 2024-09-12T02:18:12.619 INFO:teuthology.orchestra.run.smithi184.stdout: "val": 0, 2024-09-12T02:18:12.619 INFO:teuthology.orchestra.run.smithi184.stdout: "max": 104857600, 2024-09-12T02:18:12.619 INFO:teuthology.orchestra.run.smithi184.stdout: "get_started": 0, 2024-09-12T02:18:12.619 INFO:teuthology.orchestra.run.smithi184.stdout: "get": 0, 2024-09-12T02:18:12.619 INFO:teuthology.orchestra.run.smithi184.stdout: "get_sum": 0, 2024-09-12T02:18:12.619 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_fail": 0, 2024-09-12T02:18:12.620 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_success": 0, 2024-09-12T02:18:12.620 INFO:teuthology.orchestra.run.smithi184.stdout: "take": 38, 2024-09-12T02:18:12.620 INFO:teuthology.orchestra.run.smithi184.stdout: "take_sum": 11334, 2024-09-12T02:18:12.620 INFO:teuthology.orchestra.run.smithi184.stdout: "put": 38, 2024-09-12T02:18:12.620 INFO:teuthology.orchestra.run.smithi184.stdout: "put_sum": 11334, 2024-09-12T02:18:12.620 INFO:teuthology.orchestra.run.smithi184.stdout: "wait": { 2024-09-12T02:18:12.620 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-09-12T02:18:12.620 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-09-12T02:18:12.620 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-09-12T02:18:12.620 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-12T02:18:12.620 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:12.620 INFO:teuthology.orchestra.run.smithi184.stdout: "throttle-objecter_ops": { 2024-09-12T02:18:12.620 INFO:teuthology.orchestra.run.smithi184.stdout: "val": 0, 2024-09-12T02:18:12.620 INFO:teuthology.orchestra.run.smithi184.stdout: "max": 1024, 2024-09-12T02:18:12.621 INFO:teuthology.orchestra.run.smithi184.stdout: "get_started": 0, 2024-09-12T02:18:12.621 INFO:teuthology.orchestra.run.smithi184.stdout: "get": 0, 2024-09-12T02:18:12.621 INFO:teuthology.orchestra.run.smithi184.stdout: "get_sum": 0, 2024-09-12T02:18:12.621 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_fail": 0, 2024-09-12T02:18:12.621 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_success": 0, 2024-09-12T02:18:12.621 INFO:teuthology.orchestra.run.smithi184.stdout: "take": 38, 2024-09-12T02:18:12.621 INFO:teuthology.orchestra.run.smithi184.stdout: "take_sum": 38, 2024-09-12T02:18:12.621 INFO:teuthology.orchestra.run.smithi184.stdout: "put": 38, 2024-09-12T02:18:12.621 INFO:teuthology.orchestra.run.smithi184.stdout: "put_sum": 38, 2024-09-12T02:18:12.621 INFO:teuthology.orchestra.run.smithi184.stdout: "wait": { 2024-09-12T02:18:12.621 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-09-12T02:18:12.621 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-09-12T02:18:12.621 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-09-12T02:18:12.622 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-12T02:18:12.622 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:12.622 INFO:teuthology.orchestra.run.smithi184.stdout: "throttle-write_buf_throttle": { 2024-09-12T02:18:12.622 INFO:teuthology.orchestra.run.smithi184.stdout: "val": 0, 2024-09-12T02:18:12.622 INFO:teuthology.orchestra.run.smithi184.stdout: "max": 3758096384, 2024-09-12T02:18:12.622 INFO:teuthology.orchestra.run.smithi184.stdout: "get_started": 0, 2024-09-12T02:18:12.622 INFO:teuthology.orchestra.run.smithi184.stdout: "get": 0, 2024-09-12T02:18:12.622 INFO:teuthology.orchestra.run.smithi184.stdout: "get_sum": 0, 2024-09-12T02:18:12.622 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_fail": 0, 2024-09-12T02:18:12.622 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_success": 0, 2024-09-12T02:18:12.622 INFO:teuthology.orchestra.run.smithi184.stdout: "take": 0, 2024-09-12T02:18:12.622 INFO:teuthology.orchestra.run.smithi184.stdout: "take_sum": 0, 2024-09-12T02:18:12.622 INFO:teuthology.orchestra.run.smithi184.stdout: "put": 0, 2024-09-12T02:18:12.622 INFO:teuthology.orchestra.run.smithi184.stdout: "put_sum": 0, 2024-09-12T02:18:12.623 INFO:teuthology.orchestra.run.smithi184.stdout: "wait": { 2024-09-12T02:18:12.623 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-09-12T02:18:12.623 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-09-12T02:18:12.623 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-09-12T02:18:12.623 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-12T02:18:12.623 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:12.623 INFO:teuthology.orchestra.run.smithi184.stdout: "throttle-write_buf_throttle-0x5645f80ca960": { 2024-09-12T02:18:12.623 INFO:teuthology.orchestra.run.smithi184.stdout: "val": 0, 2024-09-12T02:18:12.623 INFO:teuthology.orchestra.run.smithi184.stdout: "max": 3758096384, 2024-09-12T02:18:12.623 INFO:teuthology.orchestra.run.smithi184.stdout: "get_started": 0, 2024-09-12T02:18:12.623 INFO:teuthology.orchestra.run.smithi184.stdout: "get": 4, 2024-09-12T02:18:12.623 INFO:teuthology.orchestra.run.smithi184.stdout: "get_sum": 1122, 2024-09-12T02:18:12.623 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_fail": 0, 2024-09-12T02:18:12.623 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_success": 4, 2024-09-12T02:18:12.624 INFO:teuthology.orchestra.run.smithi184.stdout: "take": 0, 2024-09-12T02:18:12.624 INFO:teuthology.orchestra.run.smithi184.stdout: "take_sum": 0, 2024-09-12T02:18:12.624 INFO:teuthology.orchestra.run.smithi184.stdout: "put": 3, 2024-09-12T02:18:12.624 INFO:teuthology.orchestra.run.smithi184.stdout: "put_sum": 1122, 2024-09-12T02:18:12.624 INFO:teuthology.orchestra.run.smithi184.stdout: "wait": { 2024-09-12T02:18:12.624 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-09-12T02:18:12.624 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-09-12T02:18:12.624 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-09-12T02:18:12.624 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-12T02:18:12.624 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-12T02:18:12.624 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-12T02:18:13.334 INFO:tasks.check_counter:Daemon mds.i mds.exported=0 2024-09-12T02:18:13.334 INFO:tasks.check_counter:Daemon mds.i mds.imported=0 2024-09-12T02:18:13.334 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=0 2024-09-12T02:18:13.334 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=16 2024-09-12T02:18:13.334 DEBUG:tasks.check_counter:Getting stats from l 2024-09-12T02:18:13.334 DEBUG:teuthology.orchestra.run.smithi184:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph daemon mds.l perf dump 2024-09-12T02:18:13.579 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T02:18:14.028 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:18:13 smithi184 ceph-mon[35593]: pgmap v824: 97 pgs: 97 active+clean; 4.3 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-12T02:18:14.034 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-12T02:18:14.034 INFO:teuthology.orchestra.run.smithi184.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T02:18:14.034 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_messages": 1091, 2024-09-12T02:18:14.034 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_messages": 1093, 2024-09-12T02:18:14.034 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_bytes": 287053, 2024-09-12T02:18:14.034 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_bytes": 347002, 2024-09-12T02:18:14.034 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_created_connections": 7, 2024-09-12T02:18:14.034 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_active_connections": 2, 2024-09-12T02:18:14.034 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_total_time": 0.114672326, 2024-09-12T02:18:14.035 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_send_time": 0.032250280, 2024-09-12T02:18:14.035 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_recv_time": 0.033824305, 2024-09-12T02:18:14.035 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_fast_dispatch_time": 0.037108377, 2024-09-12T02:18:14.035 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T02:18:14.035 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 1091, 2024-09-12T02:18:14.035 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.043806291, 2024-09-12T02:18:14.035 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000040152 2024-09-12T02:18:14.035 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:14.035 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_handle_ack_lat": { 2024-09-12T02:18:14.035 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-09-12T02:18:14.035 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-09-12T02:18:14.035 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-09-12T02:18:14.036 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:14.036 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-12T02:18:14.036 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_encrypted_bytes": 0 2024-09-12T02:18:14.036 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:14.036 INFO:teuthology.orchestra.run.smithi184.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T02:18:14.036 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_messages": 3, 2024-09-12T02:18:14.036 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_messages": 233, 2024-09-12T02:18:14.036 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_bytes": 674, 2024-09-12T02:18:14.036 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_bytes": 481393, 2024-09-12T02:18:14.036 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_created_connections": 4, 2024-09-12T02:18:14.036 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_active_connections": 3, 2024-09-12T02:18:14.036 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_total_time": 0.013036078, 2024-09-12T02:18:14.036 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_send_time": 0.009415515, 2024-09-12T02:18:14.037 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_recv_time": 0.001292083, 2024-09-12T02:18:14.037 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_fast_dispatch_time": 0.000062244, 2024-09-12T02:18:14.037 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T02:18:14.037 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 231, 2024-09-12T02:18:14.037 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.014801071, 2024-09-12T02:18:14.037 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000064073 2024-09-12T02:18:14.037 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:14.037 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_handle_ack_lat": { 2024-09-12T02:18:14.037 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-09-12T02:18:14.037 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-09-12T02:18:14.037 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-09-12T02:18:14.037 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:14.037 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-12T02:18:14.038 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_encrypted_bytes": 480752 2024-09-12T02:18:14.038 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:14.038 INFO:teuthology.orchestra.run.smithi184.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T02:18:14.038 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_messages": 1407, 2024-09-12T02:18:14.038 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_messages": 1400, 2024-09-12T02:18:14.038 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_bytes": 894560, 2024-09-12T02:18:14.038 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_bytes": 534738, 2024-09-12T02:18:14.038 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_created_connections": 3, 2024-09-12T02:18:14.038 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_active_connections": 2, 2024-09-12T02:18:14.038 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_total_time": 0.184455267, 2024-09-12T02:18:14.038 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_send_time": 0.064399248, 2024-09-12T02:18:14.038 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_recv_time": 0.067446365, 2024-09-12T02:18:14.038 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_fast_dispatch_time": 0.045028413, 2024-09-12T02:18:14.039 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T02:18:14.039 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 1400, 2024-09-12T02:18:14.039 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.090748190, 2024-09-12T02:18:14.039 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000064820 2024-09-12T02:18:14.039 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:14.039 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_handle_ack_lat": { 2024-09-12T02:18:14.039 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-09-12T02:18:14.039 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-09-12T02:18:14.039 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-09-12T02:18:14.039 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:14.039 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_encrypted_bytes": 529328, 2024-09-12T02:18:14.039 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_encrypted_bytes": 189072 2024-09-12T02:18:14.039 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:14.040 INFO:teuthology.orchestra.run.smithi184.stdout: "cct": { 2024-09-12T02:18:14.040 INFO:teuthology.orchestra.run.smithi184.stdout: "total_workers": 1, 2024-09-12T02:18:14.040 INFO:teuthology.orchestra.run.smithi184.stdout: "unhealthy_workers": 0 2024-09-12T02:18:14.040 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:14.040 INFO:teuthology.orchestra.run.smithi184.stdout: "finisher-MDSRank": { 2024-09-12T02:18:14.040 INFO:teuthology.orchestra.run.smithi184.stdout: "queue_len": 0, 2024-09-12T02:18:14.040 INFO:teuthology.orchestra.run.smithi184.stdout: "complete_latency": { 2024-09-12T02:18:14.040 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 4353, 2024-09-12T02:18:14.040 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.147229100, 2024-09-12T02:18:14.040 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000033822 2024-09-12T02:18:14.040 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-12T02:18:14.040 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:14.040 INFO:teuthology.orchestra.run.smithi184.stdout: "finisher-PurgeQueue": { 2024-09-12T02:18:14.041 INFO:teuthology.orchestra.run.smithi184.stdout: "queue_len": 0, 2024-09-12T02:18:14.041 INFO:teuthology.orchestra.run.smithi184.stdout: "complete_latency": { 2024-09-12T02:18:14.041 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-09-12T02:18:14.041 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-09-12T02:18:14.041 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-09-12T02:18:14.041 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-12T02:18:14.041 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:14.041 INFO:teuthology.orchestra.run.smithi184.stdout: "mds": { 2024-09-12T02:18:14.041 INFO:teuthology.orchestra.run.smithi184.stdout: "request": 0, 2024-09-12T02:18:14.041 INFO:teuthology.orchestra.run.smithi184.stdout: "reply": 0, 2024-09-12T02:18:14.041 INFO:teuthology.orchestra.run.smithi184.stdout: "reply_latency": { 2024-09-12T02:18:14.041 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-09-12T02:18:14.042 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-09-12T02:18:14.042 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-09-12T02:18:14.042 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:14.042 INFO:teuthology.orchestra.run.smithi184.stdout: "slow_reply": 0, 2024-09-12T02:18:14.042 INFO:teuthology.orchestra.run.smithi184.stdout: "forward": 0, 2024-09-12T02:18:14.042 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_fetch_complete": 0, 2024-09-12T02:18:14.042 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_fetch_keys": 0, 2024-09-12T02:18:14.042 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_commit": 0, 2024-09-12T02:18:14.042 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_split": 0, 2024-09-12T02:18:14.042 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_merge": 0, 2024-09-12T02:18:14.042 INFO:teuthology.orchestra.run.smithi184.stdout: "inodes": 0, 2024-09-12T02:18:14.042 INFO:teuthology.orchestra.run.smithi184.stdout: "inodes_top": 0, 2024-09-12T02:18:14.042 INFO:teuthology.orchestra.run.smithi184.stdout: "inodes_bottom": 0, 2024-09-12T02:18:14.042 INFO:teuthology.orchestra.run.smithi184.stdout: "inodes_pin_tail": 0, 2024-09-12T02:18:14.043 INFO:teuthology.orchestra.run.smithi184.stdout: "inodes_pinned": 0, 2024-09-12T02:18:14.043 INFO:teuthology.orchestra.run.smithi184.stdout: "inodes_expired": 0, 2024-09-12T02:18:14.043 INFO:teuthology.orchestra.run.smithi184.stdout: "inodes_with_caps": 0, 2024-09-12T02:18:14.043 INFO:teuthology.orchestra.run.smithi184.stdout: "caps": 0, 2024-09-12T02:18:14.043 INFO:teuthology.orchestra.run.smithi184.stdout: "subtrees": 1, 2024-09-12T02:18:14.043 INFO:teuthology.orchestra.run.smithi184.stdout: "traverse": 0, 2024-09-12T02:18:14.043 INFO:teuthology.orchestra.run.smithi184.stdout: "traverse_hit": 0, 2024-09-12T02:18:14.043 INFO:teuthology.orchestra.run.smithi184.stdout: "traverse_forward": 0, 2024-09-12T02:18:14.043 INFO:teuthology.orchestra.run.smithi184.stdout: "traverse_discover": 0, 2024-09-12T02:18:14.043 INFO:teuthology.orchestra.run.smithi184.stdout: "traverse_dir_fetch": 0, 2024-09-12T02:18:14.043 INFO:teuthology.orchestra.run.smithi184.stdout: "traverse_remote_ino": 0, 2024-09-12T02:18:14.043 INFO:teuthology.orchestra.run.smithi184.stdout: "traverse_lock": 0, 2024-09-12T02:18:14.043 INFO:teuthology.orchestra.run.smithi184.stdout: "load_cent": 0, 2024-09-12T02:18:14.044 INFO:teuthology.orchestra.run.smithi184.stdout: "q": 0, 2024-09-12T02:18:14.044 INFO:teuthology.orchestra.run.smithi184.stdout: "exported": 0, 2024-09-12T02:18:14.044 INFO:teuthology.orchestra.run.smithi184.stdout: "exported_inodes": 0, 2024-09-12T02:18:14.044 INFO:teuthology.orchestra.run.smithi184.stdout: "imported": 0, 2024-09-12T02:18:14.044 INFO:teuthology.orchestra.run.smithi184.stdout: "imported_inodes": 0, 2024-09-12T02:18:14.044 INFO:teuthology.orchestra.run.smithi184.stdout: "openino_dir_fetch": 0, 2024-09-12T02:18:14.044 INFO:teuthology.orchestra.run.smithi184.stdout: "openino_backtrace_fetch": 0, 2024-09-12T02:18:14.044 INFO:teuthology.orchestra.run.smithi184.stdout: "openino_peer_discover": 0, 2024-09-12T02:18:14.044 INFO:teuthology.orchestra.run.smithi184.stdout: "root_rfiles": 0, 2024-09-12T02:18:14.044 INFO:teuthology.orchestra.run.smithi184.stdout: "root_rbytes": 0, 2024-09-12T02:18:14.044 INFO:teuthology.orchestra.run.smithi184.stdout: "root_rsnaps": 0, 2024-09-12T02:18:14.044 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_backtrace_fetch": 0, 2024-09-12T02:18:14.044 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_set_tag": 0, 2024-09-12T02:18:14.045 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_backtrace_repaired": 0, 2024-09-12T02:18:14.045 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_inotable_repaired": 0, 2024-09-12T02:18:14.045 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_dir_inodes": 0, 2024-09-12T02:18:14.045 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_dir_base_inodes": 0, 2024-09-12T02:18:14.045 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_dirfrag_rstats": 0, 2024-09-12T02:18:14.045 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_file_inodes": 0, 2024-09-12T02:18:14.045 INFO:teuthology.orchestra.run.smithi184.stdout: "handle_inode_file_caps": 0, 2024-09-12T02:18:14.045 INFO:teuthology.orchestra.run.smithi184.stdout: "ceph_cap_op_revoke": 0, 2024-09-12T02:18:14.045 INFO:teuthology.orchestra.run.smithi184.stdout: "ceph_cap_op_grant": 0, 2024-09-12T02:18:14.045 INFO:teuthology.orchestra.run.smithi184.stdout: "ceph_cap_op_trunc": 0, 2024-09-12T02:18:14.045 INFO:teuthology.orchestra.run.smithi184.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-12T02:18:14.045 INFO:teuthology.orchestra.run.smithi184.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-12T02:18:14.045 INFO:teuthology.orchestra.run.smithi184.stdout: "handle_client_caps": 0, 2024-09-12T02:18:14.046 INFO:teuthology.orchestra.run.smithi184.stdout: "handle_client_caps_dirty": 0, 2024-09-12T02:18:14.046 INFO:teuthology.orchestra.run.smithi184.stdout: "handle_client_cap_release": 0, 2024-09-12T02:18:14.046 INFO:teuthology.orchestra.run.smithi184.stdout: "process_request_cap_release": 0 2024-09-12T02:18:14.046 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:14.046 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_cache": { 2024-09-12T02:18:14.046 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_update": 0, 2024-09-12T02:18:14.046 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_update_receipt": 0, 2024-09-12T02:18:14.046 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_try_discover": 0, 2024-09-12T02:18:14.046 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_send_discover": 0, 2024-09-12T02:18:14.046 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_handle_discover": 0, 2024-09-12T02:18:14.046 INFO:teuthology.orchestra.run.smithi184.stdout: "num_strays": 0, 2024-09-12T02:18:14.046 INFO:teuthology.orchestra.run.smithi184.stdout: "num_strays_delayed": 0, 2024-09-12T02:18:14.046 INFO:teuthology.orchestra.run.smithi184.stdout: "num_strays_enqueuing": 0, 2024-09-12T02:18:14.047 INFO:teuthology.orchestra.run.smithi184.stdout: "strays_created": 0, 2024-09-12T02:18:14.047 INFO:teuthology.orchestra.run.smithi184.stdout: "strays_enqueued": 0, 2024-09-12T02:18:14.047 INFO:teuthology.orchestra.run.smithi184.stdout: "strays_reintegrated": 0, 2024-09-12T02:18:14.047 INFO:teuthology.orchestra.run.smithi184.stdout: "strays_migrated": 0, 2024-09-12T02:18:14.047 INFO:teuthology.orchestra.run.smithi184.stdout: "num_recovering_processing": 0, 2024-09-12T02:18:14.047 INFO:teuthology.orchestra.run.smithi184.stdout: "num_recovering_enqueued": 0, 2024-09-12T02:18:14.047 INFO:teuthology.orchestra.run.smithi184.stdout: "num_recovering_prioritized": 0, 2024-09-12T02:18:14.047 INFO:teuthology.orchestra.run.smithi184.stdout: "recovery_started": 0, 2024-09-12T02:18:14.047 INFO:teuthology.orchestra.run.smithi184.stdout: "recovery_completed": 0, 2024-09-12T02:18:14.047 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_quiesce_path": 0, 2024-09-12T02:18:14.047 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_quiesce_inode": 0, 2024-09-12T02:18:14.047 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_enqueue_scrub": 0, 2024-09-12T02:18:14.047 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_exportdir": 0, 2024-09-12T02:18:14.047 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_flush": 0, 2024-09-12T02:18:14.048 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_fragmentdir": 0, 2024-09-12T02:18:14.048 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_fragstats": 0, 2024-09-12T02:18:14.048 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_inodestats": 0 2024-09-12T02:18:14.048 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:14.048 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_log": { 2024-09-12T02:18:14.048 INFO:teuthology.orchestra.run.smithi184.stdout: "evlrg": 0, 2024-09-12T02:18:14.048 INFO:teuthology.orchestra.run.smithi184.stdout: "evadd": 0, 2024-09-12T02:18:14.048 INFO:teuthology.orchestra.run.smithi184.stdout: "evex": 0, 2024-09-12T02:18:14.048 INFO:teuthology.orchestra.run.smithi184.stdout: "evtrm": 0, 2024-09-12T02:18:14.048 INFO:teuthology.orchestra.run.smithi184.stdout: "ev": 2, 2024-09-12T02:18:14.048 INFO:teuthology.orchestra.run.smithi184.stdout: "evexg": 0, 2024-09-12T02:18:14.048 INFO:teuthology.orchestra.run.smithi184.stdout: "evexd": 0, 2024-09-12T02:18:14.049 INFO:teuthology.orchestra.run.smithi184.stdout: "segadd": 0, 2024-09-12T02:18:14.049 INFO:teuthology.orchestra.run.smithi184.stdout: "segex": 0, 2024-09-12T02:18:14.049 INFO:teuthology.orchestra.run.smithi184.stdout: "segtrm": 0, 2024-09-12T02:18:14.049 INFO:teuthology.orchestra.run.smithi184.stdout: "seg": 2, 2024-09-12T02:18:14.049 INFO:teuthology.orchestra.run.smithi184.stdout: "segmjr": 2, 2024-09-12T02:18:14.049 INFO:teuthology.orchestra.run.smithi184.stdout: "segexg": 0, 2024-09-12T02:18:14.049 INFO:teuthology.orchestra.run.smithi184.stdout: "segexd": 0, 2024-09-12T02:18:14.049 INFO:teuthology.orchestra.run.smithi184.stdout: "expos": 4194304, 2024-09-12T02:18:14.049 INFO:teuthology.orchestra.run.smithi184.stdout: "wrpos": 4194892, 2024-09-12T02:18:14.049 INFO:teuthology.orchestra.run.smithi184.stdout: "rdpos": 4194352, 2024-09-12T02:18:14.049 INFO:teuthology.orchestra.run.smithi184.stdout: "jlat": { 2024-09-12T02:18:14.049 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-09-12T02:18:14.049 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-09-12T02:18:14.049 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-09-12T02:18:14.050 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:14.050 INFO:teuthology.orchestra.run.smithi184.stdout: "replayed": 2 2024-09-12T02:18:14.050 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:14.050 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_mem": { 2024-09-12T02:18:14.050 INFO:teuthology.orchestra.run.smithi184.stdout: "ino": 3, 2024-09-12T02:18:14.050 INFO:teuthology.orchestra.run.smithi184.stdout: "ino+": 3, 2024-09-12T02:18:14.050 INFO:teuthology.orchestra.run.smithi184.stdout: "ino-": 0, 2024-09-12T02:18:14.050 INFO:teuthology.orchestra.run.smithi184.stdout: "dir": 1, 2024-09-12T02:18:14.050 INFO:teuthology.orchestra.run.smithi184.stdout: "dir+": 1, 2024-09-12T02:18:14.050 INFO:teuthology.orchestra.run.smithi184.stdout: "dir-": 0, 2024-09-12T02:18:14.050 INFO:teuthology.orchestra.run.smithi184.stdout: "dn": 0, 2024-09-12T02:18:14.050 INFO:teuthology.orchestra.run.smithi184.stdout: "dn+": 0, 2024-09-12T02:18:14.050 INFO:teuthology.orchestra.run.smithi184.stdout: "dn-": 0, 2024-09-12T02:18:14.051 INFO:teuthology.orchestra.run.smithi184.stdout: "cap": 0, 2024-09-12T02:18:14.051 INFO:teuthology.orchestra.run.smithi184.stdout: "cap+": 0, 2024-09-12T02:18:14.051 INFO:teuthology.orchestra.run.smithi184.stdout: "cap-": 0, 2024-09-12T02:18:14.051 INFO:teuthology.orchestra.run.smithi184.stdout: "rss": 48872, 2024-09-12T02:18:14.051 INFO:teuthology.orchestra.run.smithi184.stdout: "heap": 215328 2024-09-12T02:18:14.051 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:14.051 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_server": { 2024-09-12T02:18:14.051 INFO:teuthology.orchestra.run.smithi184.stdout: "dispatch_client_request": 0, 2024-09-12T02:18:14.051 INFO:teuthology.orchestra.run.smithi184.stdout: "dispatch_server_request": 0, 2024-09-12T02:18:14.051 INFO:teuthology.orchestra.run.smithi184.stdout: "handle_client_request": 0, 2024-09-12T02:18:14.051 INFO:teuthology.orchestra.run.smithi184.stdout: "handle_client_session": 0, 2024-09-12T02:18:14.051 INFO:teuthology.orchestra.run.smithi184.stdout: "handle_peer_request": 0, 2024-09-12T02:18:14.051 INFO:teuthology.orchestra.run.smithi184.stdout: "req_create_latency": { 2024-09-12T02:18:14.051 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-09-12T02:18:14.052 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-09-12T02:18:14.052 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-09-12T02:18:14.052 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:14.052 INFO:teuthology.orchestra.run.smithi184.stdout: "req_getattr_latency": { 2024-09-12T02:18:14.052 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-09-12T02:18:14.052 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-09-12T02:18:14.052 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-09-12T02:18:14.052 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:14.052 INFO:teuthology.orchestra.run.smithi184.stdout: "req_getfilelock_latency": { 2024-09-12T02:18:14.052 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-09-12T02:18:14.052 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-09-12T02:18:14.052 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-09-12T02:18:14.052 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:14.053 INFO:teuthology.orchestra.run.smithi184.stdout: "req_link_latency": { 2024-09-12T02:18:14.053 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-09-12T02:18:14.053 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-09-12T02:18:14.053 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-09-12T02:18:14.053 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:14.053 INFO:teuthology.orchestra.run.smithi184.stdout: "req_lookup_latency": { 2024-09-12T02:18:14.053 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-09-12T02:18:14.053 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-09-12T02:18:14.053 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-09-12T02:18:14.053 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:14.053 INFO:teuthology.orchestra.run.smithi184.stdout: "req_lookuphash_latency": { 2024-09-12T02:18:14.053 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-09-12T02:18:14.053 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-09-12T02:18:14.053 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-09-12T02:18:14.054 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:14.054 INFO:teuthology.orchestra.run.smithi184.stdout: "req_lookupino_latency": { 2024-09-12T02:18:14.054 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-09-12T02:18:14.054 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-09-12T02:18:14.054 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-09-12T02:18:14.054 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:14.054 INFO:teuthology.orchestra.run.smithi184.stdout: "req_lookupname_latency": { 2024-09-12T02:18:14.054 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-09-12T02:18:14.054 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-09-12T02:18:14.054 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-09-12T02:18:14.054 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:14.054 INFO:teuthology.orchestra.run.smithi184.stdout: "req_lookupparent_latency": { 2024-09-12T02:18:14.054 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-09-12T02:18:14.055 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-09-12T02:18:14.055 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-09-12T02:18:14.055 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:14.055 INFO:teuthology.orchestra.run.smithi184.stdout: "req_lookupsnap_latency": { 2024-09-12T02:18:14.055 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-09-12T02:18:14.055 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-09-12T02:18:14.055 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-09-12T02:18:14.055 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:14.055 INFO:teuthology.orchestra.run.smithi184.stdout: "req_lssnap_latency": { 2024-09-12T02:18:14.055 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-09-12T02:18:14.055 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-09-12T02:18:14.056 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-09-12T02:18:14.056 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:14.056 INFO:teuthology.orchestra.run.smithi184.stdout: "req_mkdir_latency": { 2024-09-12T02:18:14.056 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-09-12T02:18:14.056 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-09-12T02:18:14.057 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-09-12T02:18:14.057 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:14.057 INFO:teuthology.orchestra.run.smithi184.stdout: "req_mknod_latency": { 2024-09-12T02:18:14.057 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-09-12T02:18:14.057 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-09-12T02:18:14.057 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-09-12T02:18:14.057 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:14.057 INFO:teuthology.orchestra.run.smithi184.stdout: "req_mksnap_latency": { 2024-09-12T02:18:14.057 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-09-12T02:18:14.057 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-09-12T02:18:14.057 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-09-12T02:18:14.057 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:14.058 INFO:teuthology.orchestra.run.smithi184.stdout: "req_open_latency": { 2024-09-12T02:18:14.058 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-09-12T02:18:14.058 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-09-12T02:18:14.058 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-09-12T02:18:14.058 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:14.058 INFO:teuthology.orchestra.run.smithi184.stdout: "req_readdir_latency": { 2024-09-12T02:18:14.058 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-09-12T02:18:14.058 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-09-12T02:18:14.058 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-09-12T02:18:14.058 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:14.058 INFO:teuthology.orchestra.run.smithi184.stdout: "req_rename_latency": { 2024-09-12T02:18:14.058 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-09-12T02:18:14.058 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-09-12T02:18:14.059 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-09-12T02:18:14.059 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:14.059 INFO:teuthology.orchestra.run.smithi184.stdout: "req_renamesnap_latency": { 2024-09-12T02:18:14.059 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-09-12T02:18:14.059 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-09-12T02:18:14.059 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-09-12T02:18:14.059 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:14.059 INFO:teuthology.orchestra.run.smithi184.stdout: "req_snapdiff_latency": { 2024-09-12T02:18:14.059 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-09-12T02:18:14.059 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-09-12T02:18:14.059 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-09-12T02:18:14.059 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:14.059 INFO:teuthology.orchestra.run.smithi184.stdout: "req_rmdir_latency": { 2024-09-12T02:18:14.060 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-09-12T02:18:14.060 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-09-12T02:18:14.060 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-09-12T02:18:14.060 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:14.060 INFO:teuthology.orchestra.run.smithi184.stdout: "req_rmsnap_latency": { 2024-09-12T02:18:14.060 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-09-12T02:18:14.060 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-09-12T02:18:14.060 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-09-12T02:18:14.060 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:14.060 INFO:teuthology.orchestra.run.smithi184.stdout: "req_rmxattr_latency": { 2024-09-12T02:18:14.060 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-09-12T02:18:14.060 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-09-12T02:18:14.061 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-09-12T02:18:14.061 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:14.061 INFO:teuthology.orchestra.run.smithi184.stdout: "req_setattr_latency": { 2024-09-12T02:18:14.061 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-09-12T02:18:14.061 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-09-12T02:18:14.061 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-09-12T02:18:14.061 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:14.061 INFO:teuthology.orchestra.run.smithi184.stdout: "req_setdirlayout_latency": { 2024-09-12T02:18:14.061 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-09-12T02:18:14.061 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-09-12T02:18:14.061 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-09-12T02:18:14.061 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:14.061 INFO:teuthology.orchestra.run.smithi184.stdout: "req_setfilelock_latency": { 2024-09-12T02:18:14.062 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-09-12T02:18:14.062 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-09-12T02:18:14.062 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-09-12T02:18:14.062 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:14.062 INFO:teuthology.orchestra.run.smithi184.stdout: "req_setlayout_latency": { 2024-09-12T02:18:14.062 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-09-12T02:18:14.062 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-09-12T02:18:14.062 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-09-12T02:18:14.062 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:14.062 INFO:teuthology.orchestra.run.smithi184.stdout: "req_setxattr_latency": { 2024-09-12T02:18:14.062 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-09-12T02:18:14.062 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-09-12T02:18:14.062 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-09-12T02:18:14.062 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:14.063 INFO:teuthology.orchestra.run.smithi184.stdout: "req_symlink_latency": { 2024-09-12T02:18:14.063 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-09-12T02:18:14.063 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-09-12T02:18:14.063 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-09-12T02:18:14.063 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:14.063 INFO:teuthology.orchestra.run.smithi184.stdout: "req_unlink_latency": { 2024-09-12T02:18:14.063 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-09-12T02:18:14.063 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-09-12T02:18:14.063 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-09-12T02:18:14.063 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:14.063 INFO:teuthology.orchestra.run.smithi184.stdout: "cap_revoke_eviction": 0, 2024-09-12T02:18:14.063 INFO:teuthology.orchestra.run.smithi184.stdout: "cap_acquisition_throttle": 0, 2024-09-12T02:18:14.063 INFO:teuthology.orchestra.run.smithi184.stdout: "req_getvxattr_latency": { 2024-09-12T02:18:14.064 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-09-12T02:18:14.064 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-09-12T02:18:14.064 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-09-12T02:18:14.064 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-12T02:18:14.064 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:14.064 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_sessions": { 2024-09-12T02:18:14.064 INFO:teuthology.orchestra.run.smithi184.stdout: "session_count": 0, 2024-09-12T02:18:14.064 INFO:teuthology.orchestra.run.smithi184.stdout: "session_add": 0, 2024-09-12T02:18:14.064 INFO:teuthology.orchestra.run.smithi184.stdout: "session_remove": 0, 2024-09-12T02:18:14.064 INFO:teuthology.orchestra.run.smithi184.stdout: "sessions_open": 0, 2024-09-12T02:18:14.064 INFO:teuthology.orchestra.run.smithi184.stdout: "sessions_stale": 0, 2024-09-12T02:18:14.064 INFO:teuthology.orchestra.run.smithi184.stdout: "total_load": 0, 2024-09-12T02:18:14.064 INFO:teuthology.orchestra.run.smithi184.stdout: "average_load": 0, 2024-09-12T02:18:14.065 INFO:teuthology.orchestra.run.smithi184.stdout: "avg_session_uptime": 0, 2024-09-12T02:18:14.065 INFO:teuthology.orchestra.run.smithi184.stdout: "mdthresh_evicted": 0 2024-09-12T02:18:14.065 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:14.065 INFO:teuthology.orchestra.run.smithi184.stdout: "mempool": { 2024-09-12T02:18:14.065 INFO:teuthology.orchestra.run.smithi184.stdout: "bloom_filter_bytes": 0, 2024-09-12T02:18:14.065 INFO:teuthology.orchestra.run.smithi184.stdout: "bloom_filter_items": 0, 2024-09-12T02:18:14.065 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T02:18:14.065 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_alloc_items": 0, 2024-09-12T02:18:14.065 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T02:18:14.065 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_data_items": 0, 2024-09-12T02:18:14.065 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T02:18:14.065 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T02:18:14.065 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T02:18:14.066 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T02:18:14.066 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T02:18:14.066 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_other_items": 0, 2024-09-12T02:18:14.066 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T02:18:14.066 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T02:18:14.066 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_extent_bytes": 0, 2024-09-12T02:18:14.066 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_extent_items": 0, 2024-09-12T02:18:14.066 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_blob_bytes": 0, 2024-09-12T02:18:14.066 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_blob_items": 0, 2024-09-12T02:18:14.066 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T02:18:14.066 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T02:18:14.066 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T02:18:14.066 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T02:18:14.067 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T02:18:14.067 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_fsck_items": 0, 2024-09-12T02:18:14.067 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_txc_bytes": 0, 2024-09-12T02:18:14.067 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_txc_items": 0, 2024-09-12T02:18:14.067 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T02:18:14.067 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T02:18:14.067 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_writing_bytes": 0, 2024-09-12T02:18:14.067 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_writing_items": 0, 2024-09-12T02:18:14.067 INFO:teuthology.orchestra.run.smithi184.stdout: "bluefs_bytes": 0, 2024-09-12T02:18:14.067 INFO:teuthology.orchestra.run.smithi184.stdout: "bluefs_items": 0, 2024-09-12T02:18:14.067 INFO:teuthology.orchestra.run.smithi184.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T02:18:14.067 INFO:teuthology.orchestra.run.smithi184.stdout: "bluefs_file_reader_items": 0, 2024-09-12T02:18:14.067 INFO:teuthology.orchestra.run.smithi184.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T02:18:14.067 INFO:teuthology.orchestra.run.smithi184.stdout: "bluefs_file_writer_items": 0, 2024-09-12T02:18:14.068 INFO:teuthology.orchestra.run.smithi184.stdout: "buffer_anon_bytes": 182144, 2024-09-12T02:18:14.068 INFO:teuthology.orchestra.run.smithi184.stdout: "buffer_anon_items": 50, 2024-09-12T02:18:14.068 INFO:teuthology.orchestra.run.smithi184.stdout: "buffer_meta_bytes": 0, 2024-09-12T02:18:14.068 INFO:teuthology.orchestra.run.smithi184.stdout: "buffer_meta_items": 0, 2024-09-12T02:18:14.068 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_bytes": 0, 2024-09-12T02:18:14.068 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_items": 0, 2024-09-12T02:18:14.068 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_mapbl_bytes": 0, 2024-09-12T02:18:14.068 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_mapbl_items": 0, 2024-09-12T02:18:14.068 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_pglog_bytes": 0, 2024-09-12T02:18:14.068 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_pglog_items": 0, 2024-09-12T02:18:14.068 INFO:teuthology.orchestra.run.smithi184.stdout: "osdmap_bytes": 7304, 2024-09-12T02:18:14.068 INFO:teuthology.orchestra.run.smithi184.stdout: "osdmap_items": 150, 2024-09-12T02:18:14.069 INFO:teuthology.orchestra.run.smithi184.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T02:18:14.069 INFO:teuthology.orchestra.run.smithi184.stdout: "osdmap_mapping_items": 0, 2024-09-12T02:18:14.069 INFO:teuthology.orchestra.run.smithi184.stdout: "pgmap_bytes": 0, 2024-09-12T02:18:14.069 INFO:teuthology.orchestra.run.smithi184.stdout: "pgmap_items": 0, 2024-09-12T02:18:14.069 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_co_bytes": 8512, 2024-09-12T02:18:14.069 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_co_items": 13, 2024-09-12T02:18:14.069 INFO:teuthology.orchestra.run.smithi184.stdout: "unittest_1_bytes": 0, 2024-09-12T02:18:14.069 INFO:teuthology.orchestra.run.smithi184.stdout: "unittest_1_items": 0, 2024-09-12T02:18:14.069 INFO:teuthology.orchestra.run.smithi184.stdout: "unittest_2_bytes": 0, 2024-09-12T02:18:14.069 INFO:teuthology.orchestra.run.smithi184.stdout: "unittest_2_items": 0 2024-09-12T02:18:14.069 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:14.069 INFO:teuthology.orchestra.run.smithi184.stdout: "objecter": { 2024-09-12T02:18:14.069 INFO:teuthology.orchestra.run.smithi184.stdout: "op_active": 0, 2024-09-12T02:18:14.069 INFO:teuthology.orchestra.run.smithi184.stdout: "op_laggy": 0, 2024-09-12T02:18:14.070 INFO:teuthology.orchestra.run.smithi184.stdout: "op_send": 2180, 2024-09-12T02:18:14.070 INFO:teuthology.orchestra.run.smithi184.stdout: "op_send_bytes": 21, 2024-09-12T02:18:14.070 INFO:teuthology.orchestra.run.smithi184.stdout: "op_resend": 3, 2024-09-12T02:18:14.070 INFO:teuthology.orchestra.run.smithi184.stdout: "op_reply": 2180, 2024-09-12T02:18:14.070 INFO:teuthology.orchestra.run.smithi184.stdout: "op_latency": { 2024-09-12T02:18:14.070 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 2180, 2024-09-12T02:18:14.070 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 1.373970820, 2024-09-12T02:18:14.070 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000630261 2024-09-12T02:18:14.070 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:14.070 INFO:teuthology.orchestra.run.smithi184.stdout: "op_inflight": 0, 2024-09-12T02:18:14.070 INFO:teuthology.orchestra.run.smithi184.stdout: "oplen_avg": { 2024-09-12T02:18:14.070 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 2180, 2024-09-12T02:18:14.070 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 2181 2024-09-12T02:18:14.071 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:14.071 INFO:teuthology.orchestra.run.smithi184.stdout: "op": 2180, 2024-09-12T02:18:14.071 INFO:teuthology.orchestra.run.smithi184.stdout: "op_r": 2180, 2024-09-12T02:18:14.071 INFO:teuthology.orchestra.run.smithi184.stdout: "op_w": 0, 2024-09-12T02:18:14.071 INFO:teuthology.orchestra.run.smithi184.stdout: "op_rmw": 0, 2024-09-12T02:18:14.071 INFO:teuthology.orchestra.run.smithi184.stdout: "op_pg": 0, 2024-09-12T02:18:14.071 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_stat": 1087, 2024-09-12T02:18:14.071 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_create": 0, 2024-09-12T02:18:14.071 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_read": 1091, 2024-09-12T02:18:14.071 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_write": 0, 2024-09-12T02:18:14.071 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_writefull": 0, 2024-09-12T02:18:14.071 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_writesame": 0, 2024-09-12T02:18:14.071 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_append": 0, 2024-09-12T02:18:14.072 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_zero": 0, 2024-09-12T02:18:14.072 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_truncate": 0, 2024-09-12T02:18:14.072 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_delete": 0, 2024-09-12T02:18:14.072 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_mapext": 0, 2024-09-12T02:18:14.072 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_sparse_read": 0, 2024-09-12T02:18:14.072 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_clonerange": 0, 2024-09-12T02:18:14.072 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_getxattr": 1, 2024-09-12T02:18:14.072 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_setxattr": 0, 2024-09-12T02:18:14.072 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_cmpxattr": 0, 2024-09-12T02:18:14.072 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_rmxattr": 0, 2024-09-12T02:18:14.072 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_resetxattrs": 0, 2024-09-12T02:18:14.072 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_call": 0, 2024-09-12T02:18:14.072 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_watch": 0, 2024-09-12T02:18:14.073 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_notify": 0, 2024-09-12T02:18:14.073 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_src_cmpxattr": 0, 2024-09-12T02:18:14.073 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_pgls": 0, 2024-09-12T02:18:14.073 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_pgls_filter": 0, 2024-09-12T02:18:14.073 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_other": 0, 2024-09-12T02:18:14.073 INFO:teuthology.orchestra.run.smithi184.stdout: "linger_active": 0, 2024-09-12T02:18:14.073 INFO:teuthology.orchestra.run.smithi184.stdout: "linger_send": 0, 2024-09-12T02:18:14.073 INFO:teuthology.orchestra.run.smithi184.stdout: "linger_resend": 0, 2024-09-12T02:18:14.073 INFO:teuthology.orchestra.run.smithi184.stdout: "linger_ping": 0, 2024-09-12T02:18:14.073 INFO:teuthology.orchestra.run.smithi184.stdout: "poolop_active": 0, 2024-09-12T02:18:14.073 INFO:teuthology.orchestra.run.smithi184.stdout: "poolop_send": 0, 2024-09-12T02:18:14.073 INFO:teuthology.orchestra.run.smithi184.stdout: "poolop_resend": 0, 2024-09-12T02:18:14.073 INFO:teuthology.orchestra.run.smithi184.stdout: "poolstat_active": 0, 2024-09-12T02:18:14.074 INFO:teuthology.orchestra.run.smithi184.stdout: "poolstat_send": 0, 2024-09-12T02:18:14.074 INFO:teuthology.orchestra.run.smithi184.stdout: "poolstat_resend": 0, 2024-09-12T02:18:14.074 INFO:teuthology.orchestra.run.smithi184.stdout: "statfs_active": 0, 2024-09-12T02:18:14.074 INFO:teuthology.orchestra.run.smithi184.stdout: "statfs_send": 0, 2024-09-12T02:18:14.074 INFO:teuthology.orchestra.run.smithi184.stdout: "statfs_resend": 0, 2024-09-12T02:18:14.074 INFO:teuthology.orchestra.run.smithi184.stdout: "command_active": 0, 2024-09-12T02:18:14.074 INFO:teuthology.orchestra.run.smithi184.stdout: "command_send": 0, 2024-09-12T02:18:14.074 INFO:teuthology.orchestra.run.smithi184.stdout: "command_resend": 0, 2024-09-12T02:18:14.074 INFO:teuthology.orchestra.run.smithi184.stdout: "map_epoch": 82, 2024-09-12T02:18:14.074 INFO:teuthology.orchestra.run.smithi184.stdout: "map_full": 0, 2024-09-12T02:18:14.074 INFO:teuthology.orchestra.run.smithi184.stdout: "map_inc": 2, 2024-09-12T02:18:14.074 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_sessions": 4, 2024-09-12T02:18:14.074 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_session_open": 5, 2024-09-12T02:18:14.075 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_session_close": 1, 2024-09-12T02:18:14.075 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_laggy": 0, 2024-09-12T02:18:14.075 INFO:teuthology.orchestra.run.smithi184.stdout: "omap_wr": 0, 2024-09-12T02:18:14.075 INFO:teuthology.orchestra.run.smithi184.stdout: "omap_rd": 2, 2024-09-12T02:18:14.075 INFO:teuthology.orchestra.run.smithi184.stdout: "omap_del": 0 2024-09-12T02:18:14.075 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:14.075 INFO:teuthology.orchestra.run.smithi184.stdout: "oft": { 2024-09-12T02:18:14.075 INFO:teuthology.orchestra.run.smithi184.stdout: "omap_total_objs": 0, 2024-09-12T02:18:14.075 INFO:teuthology.orchestra.run.smithi184.stdout: "omap_total_kv_pairs": 0, 2024-09-12T02:18:14.075 INFO:teuthology.orchestra.run.smithi184.stdout: "omap_total_updates": 0, 2024-09-12T02:18:14.075 INFO:teuthology.orchestra.run.smithi184.stdout: "omap_total_removes": 0 2024-09-12T02:18:14.075 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:14.075 INFO:teuthology.orchestra.run.smithi184.stdout: "osd-slow-ops": { 2024-09-12T02:18:14.075 INFO:teuthology.orchestra.run.smithi184.stdout: "slow_ops_count": 0 2024-09-12T02:18:14.076 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:14.076 INFO:teuthology.orchestra.run.smithi184.stdout: "purge_queue": { 2024-09-12T02:18:14.076 INFO:teuthology.orchestra.run.smithi184.stdout: "pq_executing_ops": 0, 2024-09-12T02:18:14.076 INFO:teuthology.orchestra.run.smithi184.stdout: "pq_executing_ops_high_water": 0, 2024-09-12T02:18:14.076 INFO:teuthology.orchestra.run.smithi184.stdout: "pq_executing": 0, 2024-09-12T02:18:14.076 INFO:teuthology.orchestra.run.smithi184.stdout: "pq_executing_high_water": 0, 2024-09-12T02:18:14.076 INFO:teuthology.orchestra.run.smithi184.stdout: "pq_executed_ops": 0, 2024-09-12T02:18:14.076 INFO:teuthology.orchestra.run.smithi184.stdout: "pq_executed": 0, 2024-09-12T02:18:14.076 INFO:teuthology.orchestra.run.smithi184.stdout: "pq_item_in_journal": 0 2024-09-12T02:18:14.076 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:14.076 INFO:teuthology.orchestra.run.smithi184.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T02:18:14.076 INFO:teuthology.orchestra.run.smithi184.stdout: "val": 0, 2024-09-12T02:18:14.076 INFO:teuthology.orchestra.run.smithi184.stdout: "max": 104857600, 2024-09-12T02:18:14.077 INFO:teuthology.orchestra.run.smithi184.stdout: "get_started": 0, 2024-09-12T02:18:14.077 INFO:teuthology.orchestra.run.smithi184.stdout: "get": 2497, 2024-09-12T02:18:14.077 INFO:teuthology.orchestra.run.smithi184.stdout: "get_sum": 938271, 2024-09-12T02:18:14.077 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_fail": 0, 2024-09-12T02:18:14.077 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_success": 2497, 2024-09-12T02:18:14.077 INFO:teuthology.orchestra.run.smithi184.stdout: "take": 0, 2024-09-12T02:18:14.077 INFO:teuthology.orchestra.run.smithi184.stdout: "take_sum": 0, 2024-09-12T02:18:14.077 INFO:teuthology.orchestra.run.smithi184.stdout: "put": 2497, 2024-09-12T02:18:14.077 INFO:teuthology.orchestra.run.smithi184.stdout: "put_sum": 938271, 2024-09-12T02:18:14.077 INFO:teuthology.orchestra.run.smithi184.stdout: "wait": { 2024-09-12T02:18:14.077 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-09-12T02:18:14.077 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-09-12T02:18:14.078 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-09-12T02:18:14.078 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-12T02:18:14.078 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:14.078 INFO:teuthology.orchestra.run.smithi184.stdout: "throttle-objecter_bytes": { 2024-09-12T02:18:14.078 INFO:teuthology.orchestra.run.smithi184.stdout: "val": 0, 2024-09-12T02:18:14.078 INFO:teuthology.orchestra.run.smithi184.stdout: "max": 104857600, 2024-09-12T02:18:14.078 INFO:teuthology.orchestra.run.smithi184.stdout: "get_started": 0, 2024-09-12T02:18:14.078 INFO:teuthology.orchestra.run.smithi184.stdout: "get": 0, 2024-09-12T02:18:14.078 INFO:teuthology.orchestra.run.smithi184.stdout: "get_sum": 0, 2024-09-12T02:18:14.078 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_fail": 0, 2024-09-12T02:18:14.078 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_success": 0, 2024-09-12T02:18:14.078 INFO:teuthology.orchestra.run.smithi184.stdout: "take": 2180, 2024-09-12T02:18:14.078 INFO:teuthology.orchestra.run.smithi184.stdout: "take_sum": 593, 2024-09-12T02:18:14.079 INFO:teuthology.orchestra.run.smithi184.stdout: "put": 2180, 2024-09-12T02:18:14.079 INFO:teuthology.orchestra.run.smithi184.stdout: "put_sum": 593, 2024-09-12T02:18:14.079 INFO:teuthology.orchestra.run.smithi184.stdout: "wait": { 2024-09-12T02:18:14.079 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-09-12T02:18:14.079 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-09-12T02:18:14.079 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-09-12T02:18:14.079 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-12T02:18:14.079 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:14.079 INFO:teuthology.orchestra.run.smithi184.stdout: "throttle-objecter_ops": { 2024-09-12T02:18:14.079 INFO:teuthology.orchestra.run.smithi184.stdout: "val": 0, 2024-09-12T02:18:14.079 INFO:teuthology.orchestra.run.smithi184.stdout: "max": 1024, 2024-09-12T02:18:14.079 INFO:teuthology.orchestra.run.smithi184.stdout: "get_started": 0, 2024-09-12T02:18:14.079 INFO:teuthology.orchestra.run.smithi184.stdout: "get": 0, 2024-09-12T02:18:14.080 INFO:teuthology.orchestra.run.smithi184.stdout: "get_sum": 0, 2024-09-12T02:18:14.080 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_fail": 0, 2024-09-12T02:18:14.080 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_success": 0, 2024-09-12T02:18:14.080 INFO:teuthology.orchestra.run.smithi184.stdout: "take": 2180, 2024-09-12T02:18:14.080 INFO:teuthology.orchestra.run.smithi184.stdout: "take_sum": 2180, 2024-09-12T02:18:14.080 INFO:teuthology.orchestra.run.smithi184.stdout: "put": 2180, 2024-09-12T02:18:14.080 INFO:teuthology.orchestra.run.smithi184.stdout: "put_sum": 2180, 2024-09-12T02:18:14.080 INFO:teuthology.orchestra.run.smithi184.stdout: "wait": { 2024-09-12T02:18:14.080 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-09-12T02:18:14.080 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-09-12T02:18:14.080 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-09-12T02:18:14.080 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-12T02:18:14.080 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:14.081 INFO:teuthology.orchestra.run.smithi184.stdout: "throttle-write_buf_throttle": { 2024-09-12T02:18:14.081 INFO:teuthology.orchestra.run.smithi184.stdout: "val": 0, 2024-09-12T02:18:14.081 INFO:teuthology.orchestra.run.smithi184.stdout: "max": 3758096384, 2024-09-12T02:18:14.081 INFO:teuthology.orchestra.run.smithi184.stdout: "get_started": 0, 2024-09-12T02:18:14.081 INFO:teuthology.orchestra.run.smithi184.stdout: "get": 0, 2024-09-12T02:18:14.081 INFO:teuthology.orchestra.run.smithi184.stdout: "get_sum": 0, 2024-09-12T02:18:14.081 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_fail": 0, 2024-09-12T02:18:14.081 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_success": 0, 2024-09-12T02:18:14.081 INFO:teuthology.orchestra.run.smithi184.stdout: "take": 0, 2024-09-12T02:18:14.081 INFO:teuthology.orchestra.run.smithi184.stdout: "take_sum": 0, 2024-09-12T02:18:14.081 INFO:teuthology.orchestra.run.smithi184.stdout: "put": 0, 2024-09-12T02:18:14.081 INFO:teuthology.orchestra.run.smithi184.stdout: "put_sum": 0, 2024-09-12T02:18:14.081 INFO:teuthology.orchestra.run.smithi184.stdout: "wait": { 2024-09-12T02:18:14.082 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-09-12T02:18:14.082 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-09-12T02:18:14.082 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-09-12T02:18:14.082 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-12T02:18:14.082 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-12T02:18:14.082 INFO:teuthology.orchestra.run.smithi184.stdout: "throttle-write_buf_throttle-0x5556241c2b40": { 2024-09-12T02:18:14.082 INFO:teuthology.orchestra.run.smithi184.stdout: "val": 0, 2024-09-12T02:18:14.082 INFO:teuthology.orchestra.run.smithi184.stdout: "max": 3758096384, 2024-09-12T02:18:14.082 INFO:teuthology.orchestra.run.smithi184.stdout: "get_started": 0, 2024-09-12T02:18:14.082 INFO:teuthology.orchestra.run.smithi184.stdout: "get": 0, 2024-09-12T02:18:14.082 INFO:teuthology.orchestra.run.smithi184.stdout: "get_sum": 0, 2024-09-12T02:18:14.082 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_fail": 0, 2024-09-12T02:18:14.082 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_success": 0, 2024-09-12T02:18:14.083 INFO:teuthology.orchestra.run.smithi184.stdout: "take": 0, 2024-09-12T02:18:14.083 INFO:teuthology.orchestra.run.smithi184.stdout: "take_sum": 0, 2024-09-12T02:18:14.083 INFO:teuthology.orchestra.run.smithi184.stdout: "put": 0, 2024-09-12T02:18:14.083 INFO:teuthology.orchestra.run.smithi184.stdout: "put_sum": 0, 2024-09-12T02:18:14.083 INFO:teuthology.orchestra.run.smithi184.stdout: "wait": { 2024-09-12T02:18:14.083 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-09-12T02:18:14.083 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-09-12T02:18:14.083 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-09-12T02:18:14.083 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-12T02:18:14.083 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-12T02:18:14.083 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-12T02:18:14.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:18:13 smithi033 ceph-mon[35332]: pgmap v824: 97 pgs: 97 active+clean; 4.3 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-12T02:18:14.183 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:18:13 smithi016 ceph-mon[30947]: pgmap v824: 97 pgs: 97 active+clean; 4.3 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-12T02:18:14.885 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2024-09-12T02:18:14.886 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2024-09-12T02:18:14.886 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update=0 2024-09-12T02:18:14.886 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update_receipt=0 2024-09-12T02:18:14.886 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-09-12T02:18:14.897 INFO:tasks.kclient:Unmounting kernel clients... 2024-09-12T02:18:14.897 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-12T02:18:14.897 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T02:18:14.914 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-12T02:18:14.915 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T02:18:14.972 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph osd blocklist ls 2024-09-12T02:18:15.036 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:18:14 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:18:15.036 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:18:14 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:18:15.036 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:18:14 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:18:15.036 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:18:14 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T02:18:15.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:18:14 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:18:15.128 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:18:14 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:18:15.128 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:18:14 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:18:15.128 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:18:14 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T02:18:15.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:18:14 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:18:15.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:18:14 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:18:15.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:18:14 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:18:15.169 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:18:14 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' 2024-09-12T02:18:15.272 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:18:15.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:15.775+0000 7fd200656640 1 -- 172.21.15.16:0/3746389208 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1f8102ef0 msgr2=0x7fd1f810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:15.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:15.775+0000 7fd200656640 1 --2- 172.21.15.16:0/3746389208 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1f8102ef0 0x7fd1f810da40 secure :-1 s=READY pgs=2153 cs=0 l=1 rev1=1 crypto rx=0x7fd1f0009ca0 tx=0x7fd1f002a950 comp rx=0 tx=0).stop 2024-09-12T02:18:15.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:15.775+0000 7fd200656640 1 -- 172.21.15.16:0/3746389208 shutdown_connections 2024-09-12T02:18:15.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:15.775+0000 7fd200656640 1 --2- 172.21.15.16:0/3746389208 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1f8102ef0 0x7fd1f810da40 unknown :-1 s=CLOSED pgs=2153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:15.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:15.775+0000 7fd200656640 1 --2- 172.21.15.16:0/3746389208 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd1f8102570 0x7fd1f81029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:15.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:15.775+0000 7fd200656640 1 --2- 172.21.15.16:0/3746389208 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd1f8100e20 0x7fd1f8101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:15.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:15.775+0000 7fd200656640 1 -- 172.21.15.16:0/3746389208 >> 172.21.15.16:0/3746389208 conn(0x7fd1f80fc7d0 msgr2=0x7fd1f80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:18:15.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:15.775+0000 7fd200656640 1 -- 172.21.15.16:0/3746389208 shutdown_connections 2024-09-12T02:18:15.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:15.775+0000 7fd200656640 1 -- 172.21.15.16:0/3746389208 wait complete. 2024-09-12T02:18:15.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:15.776+0000 7fd200656640 1 Processor -- start 2024-09-12T02:18:15.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:15.776+0000 7fd200656640 1 -- start start 2024-09-12T02:18:15.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:15.776+0000 7fd200656640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd1f8100e20 0x7fd1f81983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:15.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:15.776+0000 7fd200656640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1f8102570 0x7fd1f8198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:15.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:15.776+0000 7fd200656640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd1f8102ef0 0x7fd1f819cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:15.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:15.776+0000 7fd200656640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd1f8110f80 con 0x7fd1f8102570 2024-09-12T02:18:15.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:15.776+0000 7fd200656640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd1f8110e00 con 0x7fd1f8100e20 2024-09-12T02:18:15.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:15.776+0000 7fd200656640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd1f8111100 con 0x7fd1f8102ef0 2024-09-12T02:18:15.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:15.776+0000 7fd1fdbf1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1f8102570 0x7fd1f8198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:15.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:15.776+0000 7fd1fdbf1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1f8102570 0x7fd1f8198910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45724/0 (socket says 172.21.15.16:45724) 2024-09-12T02:18:15.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:15.776+0000 7fd1fdbf1640 1 -- 172.21.15.16:0/3294224405 learned_addr learned my addr 172.21.15.16:0/3294224405 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:18:15.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:15.776+0000 7fd1febf3640 1 --2- 172.21.15.16:0/3294224405 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd1f8102ef0 0x7fd1f819cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:15.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:15.776+0000 7fd1fe3f2640 1 --2- 172.21.15.16:0/3294224405 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd1f8100e20 0x7fd1f81983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:15.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:15.776+0000 7fd1fdbf1640 1 -- 172.21.15.16:0/3294224405 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd1f8102ef0 msgr2=0x7fd1f819cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:15.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:15.776+0000 7fd1fdbf1640 1 --2- 172.21.15.16:0/3294224405 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd1f8102ef0 0x7fd1f819cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:15.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:15.776+0000 7fd1fdbf1640 1 -- 172.21.15.16:0/3294224405 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd1f8100e20 msgr2=0x7fd1f81983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:15.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:15.776+0000 7fd1fdbf1640 1 --2- 172.21.15.16:0/3294224405 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd1f8100e20 0x7fd1f81983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:15.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:15.776+0000 7fd1fdbf1640 1 -- 172.21.15.16:0/3294224405 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1f819d380 con 0x7fd1f8102570 2024-09-12T02:18:15.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:15.776+0000 7fd1fe3f2640 1 --2- 172.21.15.16:0/3294224405 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd1f8100e20 0x7fd1f81983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:18:15.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:15.777+0000 7fd1fdbf1640 1 --2- 172.21.15.16:0/3294224405 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1f8102570 0x7fd1f8198910 secure :-1 s=READY pgs=2154 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec00c920 tx=0x7fd1ec00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:18:15.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:15.777+0000 7fd1eb7fe640 1 -- 172.21.15.16:0/3294224405 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1ec007c70 con 0x7fd1f8102570 2024-09-12T02:18:15.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:15.777+0000 7fd200656640 1 -- 172.21.15.16:0/3294224405 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1f819d670 con 0x7fd1f8102570 2024-09-12T02:18:15.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:15.777+0000 7fd1eb7fe640 1 -- 172.21.15.16:0/3294224405 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1ec007e10 con 0x7fd1f8102570 2024-09-12T02:18:15.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:15.777+0000 7fd1eb7fe640 1 -- 172.21.15.16:0/3294224405 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1ec018890 con 0x7fd1f8102570 2024-09-12T02:18:15.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:15.777+0000 7fd200656640 1 -- 172.21.15.16:0/3294224405 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1f81a4f50 con 0x7fd1f8102570 2024-09-12T02:18:15.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:15.778+0000 7fd200656640 1 -- 172.21.15.16:0/3294224405 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1f8101200 con 0x7fd1f8102570 2024-09-12T02:18:15.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:15.780+0000 7fd1eb7fe640 1 -- 172.21.15.16:0/3294224405 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fd1ec018a50 con 0x7fd1f8102570 2024-09-12T02:18:15.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:15.780+0000 7fd1eb7fe640 1 --2- 172.21.15.16:0/3294224405 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fd1e40a2b80 0x7fd1e40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:15.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:15.780+0000 7fd1eb7fe640 1 -- 172.21.15.16:0/3294224405 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fd1ec0cf380 con 0x7fd1f8102570 2024-09-12T02:18:15.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:15.780+0000 7fd1fe3f2640 1 --2- 172.21.15.16:0/3294224405 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fd1e40a2b80 0x7fd1e40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:15.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:15.781+0000 7fd1fe3f2640 1 --2- 172.21.15.16:0/3294224405 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fd1e40a2b80 0x7fd1e40a5040 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fd1f4009c20 tx=0x7fd1f4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:18:15.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:15.784+0000 7fd1eb7fe640 1 -- 172.21.15.16:0/3294224405 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd1ec014030 con 0x7fd1f8102570 2024-09-12T02:18:15.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:15.975+0000 7fd200656640 1 -- 172.21.15.16:0/3294224405 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fd1f810a0a0 con 0x7fd1f8102570 2024-09-12T02:18:15.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:15.975+0000 7fd1eb7fe640 1 -- 172.21.15.16:0/3294224405 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v82) ==== 81+0+592 (secure 0 0 0) 0x7fd1ec09c410 con 0x7fd1f8102570 2024-09-12T02:18:15.976 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/3532565698 2024-09-13T01:53:08.218475+0000 2024-09-12T02:18:15.976 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/1822157179 2024-09-13T01:52:44.194152+0000 2024-09-12T02:18:15.976 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/3919664061 2024-09-13T01:52:44.194152+0000 2024-09-12T02:18:15.976 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/2269394774 2024-09-13T01:52:44.194152+0000 2024-09-12T02:18:15.976 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/2595308634 2024-09-13T01:53:08.218475+0000 2024-09-12T02:18:15.976 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6801/2754331153 2024-09-13T01:52:44.194152+0000 2024-09-12T02:18:15.976 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6801/2615861334 2024-09-13T01:53:08.218475+0000 2024-09-12T02:18:15.977 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6800/2615861334 2024-09-13T01:53:08.218475+0000 2024-09-12T02:18:15.977 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/4199114771 2024-09-13T01:53:08.218475+0000 2024-09-12T02:18:15.977 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6800/2754331153 2024-09-13T01:52:44.194152+0000 2024-09-12T02:18:15.977 INFO:teuthology.orchestra.run.smithi016.stderr:listed 10 entries 2024-09-12T02:18:15.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:15.978+0000 7fd200656640 1 -- 172.21.15.16:0/3294224405 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fd1e40a2b80 msgr2=0x7fd1e40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:15.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:15.978+0000 7fd200656640 1 --2- 172.21.15.16:0/3294224405 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fd1e40a2b80 0x7fd1e40a5040 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fd1f4009c20 tx=0x7fd1f4005f90 comp rx=0 tx=0).stop 2024-09-12T02:18:15.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:15.978+0000 7fd200656640 1 -- 172.21.15.16:0/3294224405 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1f8102570 msgr2=0x7fd1f8198910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:15.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:15.978+0000 7fd200656640 1 --2- 172.21.15.16:0/3294224405 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1f8102570 0x7fd1f8198910 secure :-1 s=READY pgs=2154 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec00c920 tx=0x7fd1ec00cde0 comp rx=0 tx=0).stop 2024-09-12T02:18:15.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:15.978+0000 7fd200656640 1 -- 172.21.15.16:0/3294224405 shutdown_connections 2024-09-12T02:18:15.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:15.978+0000 7fd200656640 1 --2- 172.21.15.16:0/3294224405 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fd1e40a2b80 0x7fd1e40a5040 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:15.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:15.978+0000 7fd200656640 1 --2- 172.21.15.16:0/3294224405 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd1f8102ef0 0x7fd1f819cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:15.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:15.978+0000 7fd200656640 1 --2- 172.21.15.16:0/3294224405 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1f8102570 0x7fd1f8198910 unknown :-1 s=CLOSED pgs=2154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:15.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:15.978+0000 7fd200656640 1 --2- 172.21.15.16:0/3294224405 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd1f8100e20 0x7fd1f81983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:15.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:15.978+0000 7fd200656640 1 -- 172.21.15.16:0/3294224405 >> 172.21.15.16:0/3294224405 conn(0x7fd1f80fc7d0 msgr2=0x7fd1f806a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:18:15.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:15.978+0000 7fd200656640 1 -- 172.21.15.16:0/3294224405 shutdown_connections 2024-09-12T02:18:15.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:15.978+0000 7fd200656640 1 -- 172.21.15.16:0/3294224405 wait complete. 2024-09-12T02:18:16.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:18:16 smithi033 ceph-mon[35332]: pgmap v825: 97 pgs: 97 active+clean; 4.3 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 170 B/s wr, 6 op/s 2024-09-12T02:18:16.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:18:16 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/3294224405' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-12T02:18:16.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:18:16 smithi016 ceph-mon[30947]: pgmap v825: 97 pgs: 97 active+clean; 4.3 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 170 B/s wr, 6 op/s 2024-09-12T02:18:16.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:18:16 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/3294224405' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-12T02:18:16.775 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-12T02:18:16.775 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T02:18:16.775 DEBUG:teuthology.orchestra.run.smithi016:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-12T02:18:16.859 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi016.front.sepia.ceph.com 2024-09-12T02:18:16.859 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T02:18:16.859 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-12T02:18:16.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:18:16 smithi184 ceph-mon[35593]: pgmap v825: 97 pgs: 97 active+clean; 4.3 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 170 B/s wr, 6 op/s 2024-09-12T02:18:16.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:18:16 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/3294224405' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-12T02:18:16.920 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T02:18:16.920 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2024-09-12T02:18:16.977 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-12T02:18:16.977 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T02:18:16.978 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-12T02:18:17.046 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T02:18:17.046 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link delete ceph-brx 2024-09-12T02:18:17.146 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-09-12T02:18:17.157 INFO:tasks.cephadm:Teardown begin 2024-09-12T02:18:17.158 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-12T02:18:17.158 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:18:17.418 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:18:17.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:17.898+0000 7fa153635640 1 -- 172.21.15.16:0/2861228579 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa14c100460 msgr2=0x7fa14c100840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:17.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:17.898+0000 7fa153635640 1 --2- 172.21.15.16:0/2861228579 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa14c100460 0x7fa14c100840 secure :-1 s=READY pgs=2155 cs=0 l=1 rev1=1 crypto rx=0x7fa148009900 tx=0x7fa14802a410 comp rx=0 tx=0).stop 2024-09-12T02:18:17.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:17.898+0000 7fa153635640 1 -- 172.21.15.16:0/2861228579 shutdown_connections 2024-09-12T02:18:17.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:17.898+0000 7fa153635640 1 --2- 172.21.15.16:0/2861228579 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa14c105590 0x7fa14c10d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:17.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:17.898+0000 7fa153635640 1 --2- 172.21.15.16:0/2861228579 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa14c100e10 0x7fa14c104e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:17.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:17.898+0000 7fa153635640 1 --2- 172.21.15.16:0/2861228579 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa14c100460 0x7fa14c100840 unknown :-1 s=CLOSED pgs=2155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:17.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:17.898+0000 7fa153635640 1 -- 172.21.15.16:0/2861228579 >> 172.21.15.16:0/2861228579 conn(0x7fa14c0fbee0 msgr2=0x7fa14c0fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:18:17.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:17.898+0000 7fa153635640 1 -- 172.21.15.16:0/2861228579 shutdown_connections 2024-09-12T02:18:17.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:17.898+0000 7fa153635640 1 -- 172.21.15.16:0/2861228579 wait complete. 2024-09-12T02:18:17.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:17.899+0000 7fa153635640 1 Processor -- start 2024-09-12T02:18:17.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:17.899+0000 7fa153635640 1 -- start start 2024-09-12T02:18:17.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:17.899+0000 7fa153635640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa14c100460 0x7fa14c198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:17.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:17.899+0000 7fa153635640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa14c100e10 0x7fa14c198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:17.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:17.899+0000 7fa153635640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa14c105590 0x7fa14c19cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:17.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:17.899+0000 7fa153635640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa14c10fd70 con 0x7fa14c100e10 2024-09-12T02:18:17.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:17.899+0000 7fa153635640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa14c10fbf0 con 0x7fa14c100460 2024-09-12T02:18:17.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:17.899+0000 7fa153635640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa14c10fef0 con 0x7fa14c105590 2024-09-12T02:18:17.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:17.899+0000 7fa150bd0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa14c100e10 0x7fa14c198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:17.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:17.899+0000 7fa150bd0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa14c100e10 0x7fa14c198980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45742/0 (socket says 172.21.15.16:45742) 2024-09-12T02:18:17.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:17.899+0000 7fa150bd0640 1 -- 172.21.15.16:0/2237927330 learned_addr learned my addr 172.21.15.16:0/2237927330 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:18:17.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:17.900+0000 7fa1513d1640 1 --2- 172.21.15.16:0/2237927330 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa14c100460 0x7fa14c198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:17.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:17.900+0000 7fa151bd2640 1 --2- 172.21.15.16:0/2237927330 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa14c105590 0x7fa14c19cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:17.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:17.900+0000 7fa150bd0640 1 -- 172.21.15.16:0/2237927330 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa14c105590 msgr2=0x7fa14c19cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:17.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:17.900+0000 7fa150bd0640 1 --2- 172.21.15.16:0/2237927330 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa14c105590 0x7fa14c19cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:17.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:17.900+0000 7fa150bd0640 1 -- 172.21.15.16:0/2237927330 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa14c100460 msgr2=0x7fa14c198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:17.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:17.900+0000 7fa150bd0640 1 --2- 172.21.15.16:0/2237927330 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa14c100460 0x7fa14c198440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:17.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:17.900+0000 7fa150bd0640 1 -- 172.21.15.16:0/2237927330 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa14c19d3f0 con 0x7fa14c100e10 2024-09-12T02:18:17.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:17.900+0000 7fa150bd0640 1 --2- 172.21.15.16:0/2237927330 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa14c100e10 0x7fa14c198980 secure :-1 s=READY pgs=2156 cs=0 l=1 rev1=1 crypto rx=0x7fa13c00ea40 tx=0x7fa13c00ef00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:18:17.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:17.900+0000 7fa1427fc640 1 -- 172.21.15.16:0/2237927330 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa13c00cd40 con 0x7fa14c100e10 2024-09-12T02:18:17.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:17.900+0000 7fa1427fc640 1 -- 172.21.15.16:0/2237927330 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa13c005500 con 0x7fa14c100e10 2024-09-12T02:18:17.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:17.900+0000 7fa153635640 1 -- 172.21.15.16:0/2237927330 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa14c19d6e0 con 0x7fa14c100e10 2024-09-12T02:18:17.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:17.900+0000 7fa1427fc640 1 -- 172.21.15.16:0/2237927330 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa13c016670 con 0x7fa14c100e10 2024-09-12T02:18:17.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:17.900+0000 7fa153635640 1 -- 172.21.15.16:0/2237927330 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa14c1a4fc0 con 0x7fa14c100e10 2024-09-12T02:18:17.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:17.901+0000 7fa153635640 1 -- 172.21.15.16:0/2237927330 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa14c100840 con 0x7fa14c100e10 2024-09-12T02:18:17.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:17.902+0000 7fa1427fc640 1 -- 172.21.15.16:0/2237927330 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa13c0046c0 con 0x7fa14c100e10 2024-09-12T02:18:17.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:17.903+0000 7fa1427fc640 1 --2- 172.21.15.16:0/2237927330 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fa1380a2c00 0x7fa1380a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:17.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:17.903+0000 7fa1513d1640 1 --2- 172.21.15.16:0/2237927330 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fa1380a2c00 0x7fa1380a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:17.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:17.903+0000 7fa1427fc640 1 -- 172.21.15.16:0/2237927330 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fa13c0ced00 con 0x7fa14c100e10 2024-09-12T02:18:17.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:17.904+0000 7fa1513d1640 1 --2- 172.21.15.16:0/2237927330 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fa1380a2c00 0x7fa1380a50c0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fa148009230 tx=0x7fa1480058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:18:17.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:17.907+0000 7fa1427fc640 1 -- 172.21.15.16:0/2237927330 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa13c09b6b0 con 0x7fa14c100e10 2024-09-12T02:18:18.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:18.118+0000 7fa153635640 1 -- 172.21.15.16:0/2237927330 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa14c1997a0 con 0x7fa14c100e10 2024-09-12T02:18:18.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:18.119+0000 7fa1427fc640 1 -- 172.21.15.16:0/2237927330 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11911 (secure 0 0 0) 0x7fa14c1997a0 con 0x7fa14c100e10 2024-09-12T02:18:18.119 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T02:18:18.119 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":25,"btime":"2024-09-12T02:01:37:192021+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3279123279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3279123279},{"type":"v1","addr":"172.21.15.16:6835","nonce":3279123279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4119452382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4119452382},{"type":"v1","addr":"172.21.15.16:6837","nonce":4119452382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/3263823165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3263823165},{"type":"v1","addr":"172.21.15.16:6839","nonce":3263823165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/1510290732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":1510290732},{"type":"v1","addr":"172.21.15.16:6841","nonce":1510290732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/279674603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":279674603},{"type":"v1","addr":"172.21.15.33:6833","nonce":279674603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3555728028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3555728028},{"type":"v1","addr":"172.21.15.33:6835","nonce":3555728028}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T01:59:56.117608+0000","modified":"2024-09-12T02:01:36.294760+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24482,"mds_1":24481,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6833/1563374106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1563374106},{"type":"v1","addr":"172.21.15.184:6833","nonce":1563374106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6837/1892833868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1892833868},{"type":"v1","addr":"172.21.15.33:6837","nonce":1892833868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/1457679686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1457679686},{"type":"v1","addr":"172.21.15.184:6835","nonce":1457679686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.184:6837/1793393087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":1793393087},{"type":"v1","addr":"172.21.15.184:6837","nonce":1793393087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.184:6839/4203347941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":4203347941},{"type":"v1","addr":"172.21.15.184:6839","nonce":4203347941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.33:6839/4136670204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":4136670204},{"type":"v1","addr":"172.21.15.33:6839","nonce":4136670204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24481,24482,24469]},"id":1}]} 2024-09-12T02:18:18.119 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 25 2024-09-12T02:18:18.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:18.121+0000 7fa153635640 1 -- 172.21.15.16:0/2237927330 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fa1380a2c00 msgr2=0x7fa1380a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:18.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:18.121+0000 7fa153635640 1 --2- 172.21.15.16:0/2237927330 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fa1380a2c00 0x7fa1380a50c0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fa148009230 tx=0x7fa1480058d0 comp rx=0 tx=0).stop 2024-09-12T02:18:18.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:18.121+0000 7fa153635640 1 -- 172.21.15.16:0/2237927330 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa14c100e10 msgr2=0x7fa14c198980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:18.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:18.121+0000 7fa153635640 1 --2- 172.21.15.16:0/2237927330 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa14c100e10 0x7fa14c198980 secure :-1 s=READY pgs=2156 cs=0 l=1 rev1=1 crypto rx=0x7fa13c00ea40 tx=0x7fa13c00ef00 comp rx=0 tx=0).stop 2024-09-12T02:18:18.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:18.121+0000 7fa153635640 1 -- 172.21.15.16:0/2237927330 shutdown_connections 2024-09-12T02:18:18.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:18.121+0000 7fa153635640 1 --2- 172.21.15.16:0/2237927330 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fa1380a2c00 0x7fa1380a50c0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:18.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:18.121+0000 7fa153635640 1 --2- 172.21.15.16:0/2237927330 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa14c105590 0x7fa14c19cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:18.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:18.121+0000 7fa153635640 1 --2- 172.21.15.16:0/2237927330 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa14c100e10 0x7fa14c198980 unknown :-1 s=CLOSED pgs=2156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:18.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:18.121+0000 7fa153635640 1 --2- 172.21.15.16:0/2237927330 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa14c100460 0x7fa14c198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:18.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:18.121+0000 7fa153635640 1 -- 172.21.15.16:0/2237927330 >> 172.21.15.16:0/2237927330 conn(0x7fa14c0fbee0 msgr2=0x7fa14c0fd230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:18:18.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:18.121+0000 7fa153635640 1 -- 172.21.15.16:0/2237927330 shutdown_connections 2024-09-12T02:18:18.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:18.121+0000 7fa153635640 1 -- 172.21.15.16:0/2237927330 wait complete. 2024-09-12T02:18:18.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:18:18 smithi033 ceph-mon[35332]: pgmap v826: 97 pgs: 97 active+clean; 4.3 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 597 B/s wr, 7 op/s 2024-09-12T02:18:18.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:18:18 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/2237927330' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:18:18.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:18:18 smithi016 ceph-mon[30947]: pgmap v826: 97 pgs: 97 active+clean; 4.3 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 597 B/s wr, 7 op/s 2024-09-12T02:18:18.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:18:18 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/2237927330' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:18:18.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:18:18 smithi184 ceph-mon[35593]: pgmap v826: 97 pgs: 97 active+clean; 4.3 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 597 B/s wr, 7 op/s 2024-09-12T02:18:18.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:18:18 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/2237927330' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:18:18.908 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:18:19.168 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:18:19.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:19.654+0000 7f4c9140f640 1 -- 172.21.15.16:0/2508415004 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c8c074eb0 msgr2=0x7f4c8c112650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:19.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:19.654+0000 7f4c9140f640 1 --2- 172.21.15.16:0/2508415004 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c8c074eb0 0x7f4c8c112650 secure :-1 s=READY pgs=2157 cs=0 l=1 rev1=1 crypto rx=0x7f4c74009900 tx=0x7f4c7402a480 comp rx=0 tx=0).stop 2024-09-12T02:18:19.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:19.654+0000 7f4c9140f640 1 -- 172.21.15.16:0/2508415004 shutdown_connections 2024-09-12T02:18:19.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:19.654+0000 7f4c9140f640 1 --2- 172.21.15.16:0/2508415004 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4c8c112b90 0x7f4c8c114f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:19.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:19.654+0000 7f4c9140f640 1 --2- 172.21.15.16:0/2508415004 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c8c074eb0 0x7f4c8c112650 unknown :-1 s=CLOSED pgs=2157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:19.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:19.654+0000 7f4c9140f640 1 --2- 172.21.15.16:0/2508415004 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c8c074590 0x7f4c8c074970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:19.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:19.654+0000 7f4c9140f640 1 -- 172.21.15.16:0/2508415004 >> 172.21.15.16:0/2508415004 conn(0x7f4c8c0fd8c0 msgr2=0x7f4c8c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:18:19.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:19.654+0000 7f4c9140f640 1 -- 172.21.15.16:0/2508415004 shutdown_connections 2024-09-12T02:18:19.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:19.654+0000 7f4c9140f640 1 -- 172.21.15.16:0/2508415004 wait complete. 2024-09-12T02:18:19.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:19.655+0000 7f4c9140f640 1 Processor -- start 2024-09-12T02:18:19.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:19.655+0000 7f4c9140f640 1 -- start start 2024-09-12T02:18:19.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:19.655+0000 7f4c9140f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c8c074590 0x7f4c8c1a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:19.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:19.655+0000 7f4c9140f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c8c074eb0 0x7f4c8c1a4c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:19.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:19.655+0000 7f4c9140f640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4c8c112b90 0x7f4c8c19e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:19.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:19.656+0000 7f4c9140f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4c8c1185b0 con 0x7f4c8c074590 2024-09-12T02:18:19.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:19.656+0000 7f4c9140f640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4c8c118430 con 0x7f4c8c074eb0 2024-09-12T02:18:19.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:19.656+0000 7f4c9140f640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f4c8c118730 con 0x7f4c8c112b90 2024-09-12T02:18:19.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:19.656+0000 7f4c8affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c8c074590 0x7f4c8c1a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:19.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:19.656+0000 7f4c8affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c8c074590 0x7f4c8c1a4720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45766/0 (socket says 172.21.15.16:45766) 2024-09-12T02:18:19.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:19.656+0000 7f4c8affd640 1 -- 172.21.15.16:0/63264011 learned_addr learned my addr 172.21.15.16:0/63264011 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:18:19.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:19.656+0000 7f4c83fff640 1 --2- 172.21.15.16:0/63264011 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c8c074eb0 0x7f4c8c1a4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:19.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:19.656+0000 7f4c8b7fe640 1 --2- 172.21.15.16:0/63264011 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4c8c112b90 0x7f4c8c19e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:19.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:19.656+0000 7f4c8affd640 1 -- 172.21.15.16:0/63264011 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4c8c112b90 msgr2=0x7f4c8c19e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:19.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:19.656+0000 7f4c8affd640 1 --2- 172.21.15.16:0/63264011 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4c8c112b90 0x7f4c8c19e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:19.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:19.656+0000 7f4c8affd640 1 -- 172.21.15.16:0/63264011 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c8c074eb0 msgr2=0x7f4c8c1a4c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:19.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:19.656+0000 7f4c8affd640 1 --2- 172.21.15.16:0/63264011 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c8c074eb0 0x7f4c8c1a4c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:19.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:19.656+0000 7f4c8affd640 1 -- 172.21.15.16:0/63264011 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c8c19f010 con 0x7f4c8c074590 2024-09-12T02:18:19.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:19.656+0000 7f4c8b7fe640 1 --2- 172.21.15.16:0/63264011 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4c8c112b90 0x7f4c8c19e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:18:19.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:19.656+0000 7f4c8affd640 1 --2- 172.21.15.16:0/63264011 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c8c074590 0x7f4c8c1a4720 secure :-1 s=READY pgs=2158 cs=0 l=1 rev1=1 crypto rx=0x7f4c7c00d950 tx=0x7f4c7c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:18:19.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:19.656+0000 7f4c88ff9640 1 -- 172.21.15.16:0/63264011 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c7c017070 con 0x7f4c8c074590 2024-09-12T02:18:19.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:19.656+0000 7f4c88ff9640 1 -- 172.21.15.16:0/63264011 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c7c009c30 con 0x7f4c8c074590 2024-09-12T02:18:19.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:19.657+0000 7f4c9140f640 1 -- 172.21.15.16:0/63264011 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c8c19f300 con 0x7f4c8c074590 2024-09-12T02:18:19.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:19.657+0000 7f4c88ff9640 1 -- 172.21.15.16:0/63264011 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c7c00f6b0 con 0x7f4c8c074590 2024-09-12T02:18:19.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:19.657+0000 7f4c9140f640 1 -- 172.21.15.16:0/63264011 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c8c1ab5f0 con 0x7f4c8c074590 2024-09-12T02:18:19.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:19.657+0000 7f4c9140f640 1 -- 172.21.15.16:0/63264011 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c8c074970 con 0x7f4c8c074590 2024-09-12T02:18:19.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:19.659+0000 7f4c88ff9640 1 -- 172.21.15.16:0/63264011 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f4c7c0109d0 con 0x7f4c8c074590 2024-09-12T02:18:19.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:19.659+0000 7f4c88ff9640 1 --2- 172.21.15.16:0/63264011 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f4c780a2c00 0x7f4c780a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:19.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:19.659+0000 7f4c88ff9640 1 -- 172.21.15.16:0/63264011 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f4c7c0ceb60 con 0x7f4c8c074590 2024-09-12T02:18:19.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:19.659+0000 7f4c83fff640 1 --2- 172.21.15.16:0/63264011 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f4c780a2c00 0x7f4c780a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:19.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:19.660+0000 7f4c83fff640 1 --2- 172.21.15.16:0/63264011 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f4c780a2c00 0x7f4c780a50c0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f4c74000c00 tx=0x7f4c74005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:18:19.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:19.663+0000 7f4c88ff9640 1 -- 172.21.15.16:0/63264011 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4c7c009de0 con 0x7f4c8c074590 2024-09-12T02:18:19.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:19.874+0000 7f4c9140f640 1 -- 172.21.15.16:0/63264011 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4c8c19fe10 con 0x7f4c8c074590 2024-09-12T02:18:19.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:19.875+0000 7f4c88ff9640 1 -- 172.21.15.16:0/63264011 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11911 (secure 0 0 0) 0x7f4c7c09bbe0 con 0x7f4c8c074590 2024-09-12T02:18:19.876 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T02:18:19.876 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":25,"btime":"2024-09-12T02:01:37:192021+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3279123279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3279123279},{"type":"v1","addr":"172.21.15.16:6835","nonce":3279123279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4119452382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4119452382},{"type":"v1","addr":"172.21.15.16:6837","nonce":4119452382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/3263823165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3263823165},{"type":"v1","addr":"172.21.15.16:6839","nonce":3263823165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/1510290732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":1510290732},{"type":"v1","addr":"172.21.15.16:6841","nonce":1510290732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/279674603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":279674603},{"type":"v1","addr":"172.21.15.33:6833","nonce":279674603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3555728028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3555728028},{"type":"v1","addr":"172.21.15.33:6835","nonce":3555728028}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T01:59:56.117608+0000","modified":"2024-09-12T02:01:36.294760+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24482,"mds_1":24481,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6833/1563374106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1563374106},{"type":"v1","addr":"172.21.15.184:6833","nonce":1563374106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6837/1892833868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1892833868},{"type":"v1","addr":"172.21.15.33:6837","nonce":1892833868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/1457679686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1457679686},{"type":"v1","addr":"172.21.15.184:6835","nonce":1457679686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.184:6837/1793393087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":1793393087},{"type":"v1","addr":"172.21.15.184:6837","nonce":1793393087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.184:6839/4203347941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":4203347941},{"type":"v1","addr":"172.21.15.184:6839","nonce":4203347941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.33:6839/4136670204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":4136670204},{"type":"v1","addr":"172.21.15.33:6839","nonce":4136670204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24481,24482,24469]},"id":1}]} 2024-09-12T02:18:19.876 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 25 2024-09-12T02:18:19.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:19.877+0000 7f4c9140f640 1 -- 172.21.15.16:0/63264011 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f4c780a2c00 msgr2=0x7f4c780a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:19.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:19.877+0000 7f4c9140f640 1 --2- 172.21.15.16:0/63264011 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f4c780a2c00 0x7f4c780a50c0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f4c74000c00 tx=0x7f4c74005b50 comp rx=0 tx=0).stop 2024-09-12T02:18:19.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:19.877+0000 7f4c9140f640 1 -- 172.21.15.16:0/63264011 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c8c074590 msgr2=0x7f4c8c1a4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:19.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:19.877+0000 7f4c9140f640 1 --2- 172.21.15.16:0/63264011 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c8c074590 0x7f4c8c1a4720 secure :-1 s=READY pgs=2158 cs=0 l=1 rev1=1 crypto rx=0x7f4c7c00d950 tx=0x7f4c7c00de10 comp rx=0 tx=0).stop 2024-09-12T02:18:19.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:19.878+0000 7f4c9140f640 1 -- 172.21.15.16:0/63264011 shutdown_connections 2024-09-12T02:18:19.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:19.878+0000 7f4c9140f640 1 --2- 172.21.15.16:0/63264011 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f4c780a2c00 0x7f4c780a50c0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:19.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:19.878+0000 7f4c9140f640 1 --2- 172.21.15.16:0/63264011 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4c8c112b90 0x7f4c8c19e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:19.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:19.878+0000 7f4c9140f640 1 --2- 172.21.15.16:0/63264011 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c8c074eb0 0x7f4c8c1a4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:19.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:19.878+0000 7f4c9140f640 1 --2- 172.21.15.16:0/63264011 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c8c074590 0x7f4c8c1a4720 unknown :-1 s=CLOSED pgs=2158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:19.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:19.878+0000 7f4c9140f640 1 -- 172.21.15.16:0/63264011 >> 172.21.15.16:0/63264011 conn(0x7f4c8c0fd8c0 msgr2=0x7f4c8c0ff020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:18:19.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:19.878+0000 7f4c9140f640 1 -- 172.21.15.16:0/63264011 shutdown_connections 2024-09-12T02:18:19.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:19.878+0000 7f4c9140f640 1 -- 172.21.15.16:0/63264011 wait complete. 2024-09-12T02:18:20.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:18:20 smithi016 ceph-mon[30947]: pgmap v827: 97 pgs: 97 active+clean; 4.3 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 5 op/s 2024-09-12T02:18:20.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:18:20 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/63264011' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:18:20.558 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T02:18:20.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:18:20 smithi033 ceph-mon[35332]: pgmap v827: 97 pgs: 97 active+clean; 4.3 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 5 op/s 2024-09-12T02:18:20.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:18:20 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/63264011' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:18:20.818 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:18:20.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:18:20 smithi184 ceph-mon[35593]: pgmap v827: 97 pgs: 97 active+clean; 4.3 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 5 op/s 2024-09-12T02:18:20.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:18:20 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/63264011' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:18:21.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:21.290+0000 7f777f8d5640 1 -- 172.21.15.16:0/1087046087 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7778103780 msgr2=0x7f7778107bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:21.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:21.290+0000 7f777f8d5640 1 --2- 172.21.15.16:0/1087046087 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7778103780 0x7f7778107bc0 secure :-1 s=READY pgs=2159 cs=0 l=1 rev1=1 crypto rx=0x7f7768009930 tx=0x7f776802a420 comp rx=0 tx=0).stop 2024-09-12T02:18:21.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:21.290+0000 7f777f8d5640 1 -- 172.21.15.16:0/1087046087 shutdown_connections 2024-09-12T02:18:21.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:21.290+0000 7f777f8d5640 1 --2- 172.21.15.16:0/1087046087 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7778108300 0x7f777810ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:21.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:21.290+0000 7f777f8d5640 1 --2- 172.21.15.16:0/1087046087 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7778103780 0x7f7778107bc0 unknown :-1 s=CLOSED pgs=2159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:21.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:21.290+0000 7f777f8d5640 1 --2- 172.21.15.16:0/1087046087 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7778102dd0 0x7f77781031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:21.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:21.290+0000 7f777f8d5640 1 -- 172.21.15.16:0/1087046087 >> 172.21.15.16:0/1087046087 conn(0x7f77780fc7d0 msgr2=0x7f77780febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:18:21.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:21.290+0000 7f777f8d5640 1 -- 172.21.15.16:0/1087046087 shutdown_connections 2024-09-12T02:18:21.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:21.290+0000 7f777f8d5640 1 -- 172.21.15.16:0/1087046087 wait complete. 2024-09-12T02:18:21.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:21.290+0000 7f777f8d5640 1 Processor -- start 2024-09-12T02:18:21.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:21.290+0000 7f777f8d5640 1 -- start start 2024-09-12T02:18:21.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:21.291+0000 7f777f8d5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7778102dd0 0x7f7778100a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:21.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:21.291+0000 7f777f8d5640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7778103780 0x7f7778100f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:21.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:21.291+0000 7f777f8d5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7778108300 0x7f77781014b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:21.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:21.291+0000 7f777f8d5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7778112070 con 0x7f7778108300 2024-09-12T02:18:21.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:21.291+0000 7f777f8d5640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7778111ef0 con 0x7f7778102dd0 2024-09-12T02:18:21.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:21.291+0000 7f777f8d5640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f77781121f0 con 0x7f7778103780 2024-09-12T02:18:21.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:21.291+0000 7f777de72640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7778108300 0x7f77781014b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:21.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:21.291+0000 7f777d671640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7778102dd0 0x7f7778100a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:21.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:21.291+0000 7f777de72640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7778108300 0x7f77781014b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33564/0 (socket says 172.21.15.16:33564) 2024-09-12T02:18:21.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:21.291+0000 7f777de72640 1 -- 172.21.15.16:0/2785875734 learned_addr learned my addr 172.21.15.16:0/2785875734 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:18:21.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:21.291+0000 7f777ce70640 1 --2- 172.21.15.16:0/2785875734 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7778103780 0x7f7778100f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:21.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:21.291+0000 7f777de72640 1 -- 172.21.15.16:0/2785875734 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7778103780 msgr2=0x7f7778100f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:21.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:21.291+0000 7f777de72640 1 --2- 172.21.15.16:0/2785875734 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7778103780 0x7f7778100f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:21.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:21.291+0000 7f777de72640 1 -- 172.21.15.16:0/2785875734 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7778102dd0 msgr2=0x7f7778100a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:21.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:21.291+0000 7f777de72640 1 --2- 172.21.15.16:0/2785875734 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7778102dd0 0x7f7778100a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:21.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:21.291+0000 7f777de72640 1 -- 172.21.15.16:0/2785875734 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77781a6ab0 con 0x7f7778108300 2024-09-12T02:18:21.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:21.291+0000 7f777ce70640 1 --2- 172.21.15.16:0/2785875734 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7778103780 0x7f7778100f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:18:21.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:21.291+0000 7f777de72640 1 --2- 172.21.15.16:0/2785875734 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7778108300 0x7f77781014b0 secure :-1 s=READY pgs=2160 cs=0 l=1 rev1=1 crypto rx=0x7f777000d750 tx=0x7f777000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:18:21.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:21.291+0000 7f776e7fc640 1 -- 172.21.15.16:0/2785875734 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7770017070 con 0x7f7778108300 2024-09-12T02:18:21.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:21.291+0000 7f776e7fc640 1 -- 172.21.15.16:0/2785875734 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7770005500 con 0x7f7778108300 2024-09-12T02:18:21.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:21.291+0000 7f777f8d5640 1 -- 172.21.15.16:0/2785875734 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77781a6da0 con 0x7f7778108300 2024-09-12T02:18:21.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:21.291+0000 7f776e7fc640 1 -- 172.21.15.16:0/2785875734 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7770004d90 con 0x7f7778108300 2024-09-12T02:18:21.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:21.292+0000 7f777f8d5640 1 -- 172.21.15.16:0/2785875734 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77781a7260 con 0x7f7778108300 2024-09-12T02:18:21.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:21.293+0000 7f777f8d5640 1 -- 172.21.15.16:0/2785875734 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77781031b0 con 0x7f7778108300 2024-09-12T02:18:21.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:21.294+0000 7f776e7fc640 1 -- 172.21.15.16:0/2785875734 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f7770004f30 con 0x7f7778108300 2024-09-12T02:18:21.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:21.294+0000 7f776e7fc640 1 --2- 172.21.15.16:0/2785875734 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f77640a2b80 0x7f77640a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:21.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:21.295+0000 7f777d671640 1 --2- 172.21.15.16:0/2785875734 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f77640a2b80 0x7f77640a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:21.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:21.295+0000 7f776e7fc640 1 -- 172.21.15.16:0/2785875734 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f77700ce9b0 con 0x7f7778108300 2024-09-12T02:18:21.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:21.295+0000 7f777d671640 1 --2- 172.21.15.16:0/2785875734 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f77640a2b80 0x7f77640a5040 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f7774005fd0 tx=0x7f7774009680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:18:21.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:21.299+0000 7f776e7fc640 1 -- 172.21.15.16:0/2785875734 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f777009b910 con 0x7f7778108300 2024-09-12T02:18:21.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:21.476+0000 7f777f8d5640 1 -- 172.21.15.16:0/2785875734 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f777810a290 con 0x7f7778108300 2024-09-12T02:18:21.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:21.478+0000 7f776e7fc640 1 -- 172.21.15.16:0/2785875734 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+22584 (secure 0 0 0) 0x7f777009b210 con 0x7f7778108300 2024-09-12T02:18:21.479 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T02:18:21.479 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":82,"fsid":"879f3b56-70a9-11ef-bcea-c7b262605968","created":"2024-09-12T01:52:12.841045+0000","modified":"2024-09-12T02:00:09.559511+0000","last_up_change":"2024-09-12T01:59:08.975443+0000","last_in_change":"2024-09-12T01:58:44.530220+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T01:55:26.727564+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T01:59:52.402097+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T01:59:54.463627+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3ea41eb6-e384-4ddc-810f-0f103c127849","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4000187867},{"type":"v1","addr":"172.21.15.16:6803","nonce":4000187867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4000187867},{"type":"v1","addr":"172.21.15.16:6805","nonce":4000187867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":4000187867},{"type":"v1","addr":"172.21.15.16:6809","nonce":4000187867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4000187867},{"type":"v1","addr":"172.21.15.16:6807","nonce":4000187867}]},"public_addr":"172.21.15.16:6803/4000187867","cluster_addr":"172.21.15.16:6805/4000187867","heartbeat_back_addr":"172.21.15.16:6809/4000187867","heartbeat_front_addr":"172.21.15.16:6807/4000187867","state":["exists","up"]},{"osd":1,"uuid":"07bf63e7-70fa-41b2-af53-12a5fec3f33c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":3862661148},{"type":"v1","addr":"172.21.15.33:6801","nonce":3862661148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":3862661148},{"type":"v1","addr":"172.21.15.33:6803","nonce":3862661148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":3862661148},{"type":"v1","addr":"172.21.15.33:6807","nonce":3862661148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":3862661148},{"type":"v1","addr":"172.21.15.33:6805","nonce":3862661148}]},"public_addr":"172.21.15.33:6801/3862661148","cluster_addr":"172.21.15.33:6803/3862661148","heartbeat_back_addr":"172.21.15.33:6807/3862661148","heartbeat_front_addr":"172.21.15.33:6805/3862661148","state":["exists","up"]},{"osd":2,"uuid":"22e40cac-3bc0-4cce-b47b-77499b767f7e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":3958753963},{"type":"v1","addr":"172.21.15.184:6801","nonce":3958753963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":3958753963},{"type":"v1","addr":"172.21.15.184:6803","nonce":3958753963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":3958753963},{"type":"v1","addr":"172.21.15.184:6807","nonce":3958753963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":3958753963},{"type":"v1","addr":"172.21.15.184:6805","nonce":3958753963}]},"public_addr":"172.21.15.184:6801/3958753963","cluster_addr":"172.21.15.184:6803/3958753963","heartbeat_back_addr":"172.21.15.184:6807/3958753963","heartbeat_front_addr":"172.21.15.184:6805/3958753963","state":["exists","up"]},{"osd":3,"uuid":"834a07f5-2c24-49e1-8e4a-12c96cfbdc65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3395182524},{"type":"v1","addr":"172.21.15.16:6811","nonce":3395182524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3395182524},{"type":"v1","addr":"172.21.15.16:6813","nonce":3395182524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3395182524},{"type":"v1","addr":"172.21.15.16:6817","nonce":3395182524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3395182524},{"type":"v1","addr":"172.21.15.16:6815","nonce":3395182524}]},"public_addr":"172.21.15.16:6811/3395182524","cluster_addr":"172.21.15.16:6813/3395182524","heartbeat_back_addr":"172.21.15.16:6817/3395182524","heartbeat_front_addr":"172.21.15.16:6815/3395182524","state":["exists","up"]},{"osd":4,"uuid":"3ca56e09-c194-4404-b9bd-2524d38367eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":1355184322},{"type":"v1","addr":"172.21.15.33:6809","nonce":1355184322}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1355184322},{"type":"v1","addr":"172.21.15.33:6811","nonce":1355184322}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1355184322},{"type":"v1","addr":"172.21.15.33:6815","nonce":1355184322}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1355184322},{"type":"v1","addr":"172.21.15.33:6813","nonce":1355184322}]},"public_addr":"172.21.15.33:6809/1355184322","cluster_addr":"172.21.15.33:6811/1355184322","heartbeat_back_addr":"172.21.15.33:6815/1355184322","heartbeat_front_addr":"172.21.15.33:6813/1355184322","state":["exists","up"]},{"osd":5,"uuid":"0b05ecdb-b987-461f-951c-6d76739bfb4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":1533397276},{"type":"v1","addr":"172.21.15.184:6809","nonce":1533397276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":1533397276},{"type":"v1","addr":"172.21.15.184:6811","nonce":1533397276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":1533397276},{"type":"v1","addr":"172.21.15.184:6815","nonce":1533397276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":1533397276},{"type":"v1","addr":"172.21.15.184:6813","nonce":1533397276}]},"public_addr":"172.21.15.184:6809/1533397276","cluster_addr":"172.21.15.184:6811/1533397276","heartbeat_back_addr":"172.21.15.184:6815/1533397276","heartbeat_front_addr":"172.21.15.184:6813/1533397276","state":["exists","up"]},{"osd":6,"uuid":"b3936e21-e197-49fa-8441-d909b258b612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":902466142},{"type":"v1","addr":"172.21.15.16:6819","nonce":902466142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":902466142},{"type":"v1","addr":"172.21.15.16:6821","nonce":902466142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":902466142},{"type":"v1","addr":"172.21.15.16:6825","nonce":902466142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":902466142},{"type":"v1","addr":"172.21.15.16:6823","nonce":902466142}]},"public_addr":"172.21.15.16:6819/902466142","cluster_addr":"172.21.15.16:6821/902466142","heartbeat_back_addr":"172.21.15.16:6825/902466142","heartbeat_front_addr":"172.21.15.16:6823/902466142","state":["exists","up"]},{"osd":7,"uuid":"4ddbe011-34c1-4b3f-a8eb-577422fb5426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":4186032630},{"type":"v1","addr":"172.21.15.33:6817","nonce":4186032630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":4186032630},{"type":"v1","addr":"172.21.15.33:6819","nonce":4186032630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":4186032630},{"type":"v1","addr":"172.21.15.33:6823","nonce":4186032630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":4186032630},{"type":"v1","addr":"172.21.15.33:6821","nonce":4186032630}]},"public_addr":"172.21.15.33:6817/4186032630","cluster_addr":"172.21.15.33:6819/4186032630","heartbeat_back_addr":"172.21.15.33:6823/4186032630","heartbeat_front_addr":"172.21.15.33:6821/4186032630","state":["exists","up"]},{"osd":8,"uuid":"dca5bfb8-ac6a-47ce-905c-be7fcaf0b5ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":781851781},{"type":"v1","addr":"172.21.15.184:6817","nonce":781851781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":781851781},{"type":"v1","addr":"172.21.15.184:6819","nonce":781851781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":781851781},{"type":"v1","addr":"172.21.15.184:6823","nonce":781851781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":781851781},{"type":"v1","addr":"172.21.15.184:6821","nonce":781851781}]},"public_addr":"172.21.15.184:6817/781851781","cluster_addr":"172.21.15.184:6819/781851781","heartbeat_back_addr":"172.21.15.184:6823/781851781","heartbeat_front_addr":"172.21.15.184:6821/781851781","state":["exists","up"]},{"osd":9,"uuid":"23a05cb5-01d5-40f3-b66a-15fcccedbd6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":1569578746},{"type":"v1","addr":"172.21.15.16:6827","nonce":1569578746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":1569578746},{"type":"v1","addr":"172.21.15.16:6829","nonce":1569578746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1569578746},{"type":"v1","addr":"172.21.15.16:6833","nonce":1569578746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":1569578746},{"type":"v1","addr":"172.21.15.16:6831","nonce":1569578746}]},"public_addr":"172.21.15.16:6827/1569578746","cluster_addr":"172.21.15.16:6829/1569578746","heartbeat_back_addr":"172.21.15.16:6833/1569578746","heartbeat_front_addr":"172.21.15.16:6831/1569578746","state":["exists","up"]},{"osd":10,"uuid":"e4bd6bce-3c69-44db-bc09-d264a0ad094a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":469977847},{"type":"v1","addr":"172.21.15.33:6825","nonce":469977847}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":469977847},{"type":"v1","addr":"172.21.15.33:6827","nonce":469977847}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":469977847},{"type":"v1","addr":"172.21.15.33:6831","nonce":469977847}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":469977847},{"type":"v1","addr":"172.21.15.33:6829","nonce":469977847}]},"public_addr":"172.21.15.33:6825/469977847","cluster_addr":"172.21.15.33:6827/469977847","heartbeat_back_addr":"172.21.15.33:6831/469977847","heartbeat_front_addr":"172.21.15.33:6829/469977847","state":["exists","up"]},{"osd":11,"uuid":"d60212f8-5a1d-45f7-9a1b-12294bb4c80d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":1619871346},{"type":"v1","addr":"172.21.15.184:6825","nonce":1619871346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":1619871346},{"type":"v1","addr":"172.21.15.184:6827","nonce":1619871346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":1619871346},{"type":"v1","addr":"172.21.15.184:6831","nonce":1619871346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":1619871346},{"type":"v1","addr":"172.21.15.184:6829","nonce":1619871346}]},"public_addr":"172.21.15.184:6825/1619871346","cluster_addr":"172.21.15.184:6827/1619871346","heartbeat_back_addr":"172.21.15.184:6831/1619871346","heartbeat_front_addr":"172.21.15.184:6829/1619871346","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:54:32.057979+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:54:57.501562+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:55:23.137225+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:55:48.646155+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:56:13.311907+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:56:38.214188+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:57:02.570921+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:57:27.506097+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:57:52.038529+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:58:17.423300+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:58:41.579992+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:59:06.245817+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:0/3532565698":"2024-09-13T01:53:08.218475+0000","172.21.15.16:0/1822157179":"2024-09-13T01:52:44.194152+0000","172.21.15.16:0/3919664061":"2024-09-13T01:52:44.194152+0000","172.21.15.16:0/2269394774":"2024-09-13T01:52:44.194152+0000","172.21.15.16:0/2595308634":"2024-09-13T01:53:08.218475+0000","172.21.15.16:6801/2754331153":"2024-09-13T01:52:44.194152+0000","172.21.15.16:6801/2615861334":"2024-09-13T01:53:08.218475+0000","172.21.15.16:6800/2615861334":"2024-09-13T01:53:08.218475+0000","172.21.15.16:0/4199114771":"2024-09-13T01:53:08.218475+0000","172.21.15.16:6800/2754331153":"2024-09-13T01:52:44.194152+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T02:18:21.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:21.482+0000 7f777f8d5640 1 -- 172.21.15.16:0/2785875734 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f77640a2b80 msgr2=0x7f77640a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:21.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:21.482+0000 7f777f8d5640 1 --2- 172.21.15.16:0/2785875734 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f77640a2b80 0x7f77640a5040 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f7774005fd0 tx=0x7f7774009680 comp rx=0 tx=0).stop 2024-09-12T02:18:21.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:21.482+0000 7f777f8d5640 1 -- 172.21.15.16:0/2785875734 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7778108300 msgr2=0x7f77781014b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:21.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:21.482+0000 7f777f8d5640 1 --2- 172.21.15.16:0/2785875734 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7778108300 0x7f77781014b0 secure :-1 s=READY pgs=2160 cs=0 l=1 rev1=1 crypto rx=0x7f777000d750 tx=0x7f777000dc10 comp rx=0 tx=0).stop 2024-09-12T02:18:21.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:21.482+0000 7f777f8d5640 1 -- 172.21.15.16:0/2785875734 shutdown_connections 2024-09-12T02:18:21.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:21.482+0000 7f777f8d5640 1 --2- 172.21.15.16:0/2785875734 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f77640a2b80 0x7f77640a5040 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:21.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:21.482+0000 7f777f8d5640 1 --2- 172.21.15.16:0/2785875734 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7778108300 0x7f77781014b0 unknown :-1 s=CLOSED pgs=2160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:21.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:21.482+0000 7f777f8d5640 1 --2- 172.21.15.16:0/2785875734 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7778103780 0x7f7778100f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:21.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:21.482+0000 7f777f8d5640 1 --2- 172.21.15.16:0/2785875734 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7778102dd0 0x7f7778100a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:21.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:21.482+0000 7f777f8d5640 1 -- 172.21.15.16:0/2785875734 >> 172.21.15.16:0/2785875734 conn(0x7f77780fc7d0 msgr2=0x7f7778105e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:18:21.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:21.482+0000 7f777f8d5640 1 -- 172.21.15.16:0/2785875734 shutdown_connections 2024-09-12T02:18:21.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:21.482+0000 7f777f8d5640 1 -- 172.21.15.16:0/2785875734 wait complete. 2024-09-12T02:18:22.066 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph fs fail cephfs 2024-09-12T02:18:22.321 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:18:22.656 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:18:22 smithi016 ceph-mon[30947]: pgmap v828: 97 pgs: 97 active+clean; 4.3 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 7 op/s 2024-09-12T02:18:22.657 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:18:22 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/2785875734' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T02:18:22.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:18:22 smithi033 ceph-mon[35332]: pgmap v828: 97 pgs: 97 active+clean; 4.3 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 7 op/s 2024-09-12T02:18:22.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:18:22 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/2785875734' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T02:18:22.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:22.872+0000 7fe41803b640 1 -- 172.21.15.16:0/440761316 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe410102790 msgr2=0x7fe4101067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:22.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:22.872+0000 7fe41803b640 1 --2- 172.21.15.16:0/440761316 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe410102790 0x7fe4101067e0 secure :-1 s=READY pgs=2161 cs=0 l=1 rev1=1 crypto rx=0x7fe400009900 tx=0x7fe40002a410 comp rx=0 tx=0).stop 2024-09-12T02:18:22.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:22.872+0000 7fe41803b640 1 -- 172.21.15.16:0/440761316 shutdown_connections 2024-09-12T02:18:22.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:22.872+0000 7fe41803b640 1 --2- 172.21.15.16:0/440761316 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe410106f10 0x7fe41010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:22.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:22.872+0000 7fe41803b640 1 --2- 172.21.15.16:0/440761316 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe410102790 0x7fe4101067e0 unknown :-1 s=CLOSED pgs=2161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:22.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:22.872+0000 7fe41803b640 1 --2- 172.21.15.16:0/440761316 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe410101de0 0x7fe4101021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:22.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:22.872+0000 7fe41803b640 1 -- 172.21.15.16:0/440761316 >> 172.21.15.16:0/440761316 conn(0x7fe4100fd8c0 msgr2=0x7fe4100ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:18:22.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:22.872+0000 7fe41803b640 1 -- 172.21.15.16:0/440761316 shutdown_connections 2024-09-12T02:18:22.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:22.872+0000 7fe41803b640 1 -- 172.21.15.16:0/440761316 wait complete. 2024-09-12T02:18:22.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:22.873+0000 7fe41803b640 1 Processor -- start 2024-09-12T02:18:22.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:22.873+0000 7fe41803b640 1 -- start start 2024-09-12T02:18:22.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:22.873+0000 7fe41803b640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe410101de0 0x7fe4100744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:22.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:22.873+0000 7fe41803b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe410102790 0x7fe410074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:22.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:22.873+0000 7fe41803b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe410106f10 0x7fe410074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:22.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:22.873+0000 7fe41803b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe410111f70 con 0x7fe410102790 2024-09-12T02:18:22.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:22.873+0000 7fe41803b640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe410111df0 con 0x7fe410106f10 2024-09-12T02:18:22.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:22.873+0000 7fe41803b640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe4101120f0 con 0x7fe410101de0 2024-09-12T02:18:22.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:22.874+0000 7fe4155d6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe410102790 0x7fe410074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:22.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:22.874+0000 7fe4155d6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe410102790 0x7fe410074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33588/0 (socket says 172.21.15.16:33588) 2024-09-12T02:18:22.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:22.874+0000 7fe4155d6640 1 -- 172.21.15.16:0/3184644553 learned_addr learned my addr 172.21.15.16:0/3184644553 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:18:22.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:22.874+0000 7fe415dd7640 1 --2- 172.21.15.16:0/3184644553 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe410101de0 0x7fe4100744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:22.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:22.874+0000 7fe4165d8640 1 --2- 172.21.15.16:0/3184644553 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe410106f10 0x7fe410074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:22.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:22.874+0000 7fe4155d6640 1 -- 172.21.15.16:0/3184644553 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe410101de0 msgr2=0x7fe4100744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:22.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:22.874+0000 7fe4155d6640 1 --2- 172.21.15.16:0/3184644553 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe410101de0 0x7fe4100744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:22.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:22.874+0000 7fe4155d6640 1 -- 172.21.15.16:0/3184644553 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe410106f10 msgr2=0x7fe410074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:22.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:22.874+0000 7fe4155d6640 1 --2- 172.21.15.16:0/3184644553 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe410106f10 0x7fe410074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:22.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:22.874+0000 7fe4155d6640 1 -- 172.21.15.16:0/3184644553 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4101a6910 con 0x7fe410102790 2024-09-12T02:18:22.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:22.874+0000 7fe4155d6640 1 --2- 172.21.15.16:0/3184644553 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe410102790 0x7fe410074a30 secure :-1 s=READY pgs=2162 cs=0 l=1 rev1=1 crypto rx=0x7fe400006010 tx=0x7fe4000042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:18:22.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:22.874+0000 7fe406ffd640 1 -- 172.21.15.16:0/3184644553 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe40003f030 con 0x7fe410102790 2024-09-12T02:18:22.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:22.874+0000 7fe406ffd640 1 -- 172.21.15.16:0/3184644553 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe400003ce0 con 0x7fe410102790 2024-09-12T02:18:22.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:22.874+0000 7fe41803b640 1 -- 172.21.15.16:0/3184644553 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4101a6ba0 con 0x7fe410102790 2024-09-12T02:18:22.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:22.874+0000 7fe406ffd640 1 -- 172.21.15.16:0/3184644553 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe400035070 con 0x7fe410102790 2024-09-12T02:18:22.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:22.875+0000 7fe41803b640 1 -- 172.21.15.16:0/3184644553 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4101a7080 con 0x7fe410102790 2024-09-12T02:18:22.878 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:18:22 smithi184 ceph-mon[35593]: pgmap v828: 97 pgs: 97 active+clean; 4.3 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 7 op/s 2024-09-12T02:18:22.878 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:18:22 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/2785875734' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T02:18:22.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:22.875+0000 7fe41803b640 1 -- 172.21.15.16:0/3184644553 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4101021c0 con 0x7fe410102790 2024-09-12T02:18:22.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:22.877+0000 7fe406ffd640 1 -- 172.21.15.16:0/3184644553 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fe400042030 con 0x7fe410102790 2024-09-12T02:18:22.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:22.877+0000 7fe406ffd640 1 --2- 172.21.15.16:0/3184644553 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fe3fc0a2b80 0x7fe3fc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:22.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:22.878+0000 7fe415dd7640 1 --2- 172.21.15.16:0/3184644553 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fe3fc0a2b80 0x7fe3fc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:22.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:22.878+0000 7fe406ffd640 1 -- 172.21.15.16:0/3184644553 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fe4000ef800 con 0x7fe410102790 2024-09-12T02:18:22.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:22.878+0000 7fe415dd7640 1 --2- 172.21.15.16:0/3184644553 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fe3fc0a2b80 0x7fe3fc0a5040 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fe40c00bbe0 tx=0x7fe40c00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:18:22.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:22.880+0000 7fe406ffd640 1 -- 172.21.15.16:0/3184644553 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe4000bc8c0 con 0x7fe410102790 2024-09-12T02:18:23.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:23.092+0000 7fe41803b640 1 -- 172.21.15.16:0/3184644553 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fe41010a150 con 0x7fe410102790 2024-09-12T02:18:23.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:23.094+0000 7fe406ffd640 1 -- 172.21.15.16:0/3184644553 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v26) ==== 161+0+0 (secure 0 0 0) 0x7fe4000bc1c0 con 0x7fe410102790 2024-09-12T02:18:23.094 INFO:teuthology.orchestra.run.smithi016.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-12T02:18:23.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:23.097+0000 7fe41803b640 1 -- 172.21.15.16:0/3184644553 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fe3fc0a2b80 msgr2=0x7fe3fc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:23.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:23.097+0000 7fe41803b640 1 --2- 172.21.15.16:0/3184644553 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fe3fc0a2b80 0x7fe3fc0a5040 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fe40c00bbe0 tx=0x7fe40c00b390 comp rx=0 tx=0).stop 2024-09-12T02:18:23.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:23.097+0000 7fe41803b640 1 -- 172.21.15.16:0/3184644553 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe410102790 msgr2=0x7fe410074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:23.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:23.097+0000 7fe41803b640 1 --2- 172.21.15.16:0/3184644553 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe410102790 0x7fe410074a30 secure :-1 s=READY pgs=2162 cs=0 l=1 rev1=1 crypto rx=0x7fe400006010 tx=0x7fe4000042f0 comp rx=0 tx=0).stop 2024-09-12T02:18:23.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:23.097+0000 7fe41803b640 1 -- 172.21.15.16:0/3184644553 shutdown_connections 2024-09-12T02:18:23.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:23.097+0000 7fe41803b640 1 --2- 172.21.15.16:0/3184644553 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fe3fc0a2b80 0x7fe3fc0a5040 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:23.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:23.097+0000 7fe41803b640 1 --2- 172.21.15.16:0/3184644553 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe410106f10 0x7fe410074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:23.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:23.097+0000 7fe41803b640 1 --2- 172.21.15.16:0/3184644553 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe410102790 0x7fe410074a30 unknown :-1 s=CLOSED pgs=2162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:23.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:23.097+0000 7fe41803b640 1 --2- 172.21.15.16:0/3184644553 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe410101de0 0x7fe4100744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:23.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:23.097+0000 7fe41803b640 1 -- 172.21.15.16:0/3184644553 >> 172.21.15.16:0/3184644553 conn(0x7fe4100fd8c0 msgr2=0x7fe4100feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:18:23.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:23.097+0000 7fe41803b640 1 -- 172.21.15.16:0/3184644553 shutdown_connections 2024-09-12T02:18:23.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:23.097+0000 7fe41803b640 1 -- 172.21.15.16:0/3184644553 wait complete. 2024-09-12T02:18:23.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:18:23 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mon-a[30943]: 2024-09-12T02:18:23.093+0000 7efe4263c640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-12T02:18:23.401 INFO:journalctl@ceph.mds.i.smithi184.stdout:Sep 12 02:18:23 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mds-i[66646]: ignoring --setuser ceph since I am not root 2024-09-12T02:18:23.401 INFO:journalctl@ceph.mds.i.smithi184.stdout:Sep 12 02:18:23 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mds-i[66646]: ignoring --setgroup ceph since I am not root 2024-09-12T02:18:23.402 INFO:journalctl@ceph.mds.i.smithi184.stdout:Sep 12 02:18:23 smithi184 ceph-mds[66650]: -- 172.21.15.184:0/828289902 <== mon.2 v2:172.21.15.33:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55df28af5860 con 0x55df28ad3000 2024-09-12T02:18:23.402 INFO:journalctl@ceph.mds.i.smithi184.stdout:Sep 12 02:18:23 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mds-i[66646]: starting mds.i at 2024-09-12T02:18:23.402 INFO:journalctl@ceph.mds.l.smithi184.stdout:Sep 12 02:18:23 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mds-l[65749]: ignoring --setuser ceph since I am not root 2024-09-12T02:18:23.402 INFO:journalctl@ceph.mds.l.smithi184.stdout:Sep 12 02:18:23 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mds-l[65749]: ignoring --setgroup ceph since I am not root 2024-09-12T02:18:23.402 INFO:journalctl@ceph.mds.l.smithi184.stdout:Sep 12 02:18:23 smithi184 ceph-mds[65753]: -- 172.21.15.184:0/2769843461 <== mon.0 v2:172.21.15.16:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5599f3f2b860 con 0x5599f3f08800 2024-09-12T02:18:23.402 INFO:journalctl@ceph.mds.l.smithi184.stdout:Sep 12 02:18:23 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mds-l[65749]: starting mds.l at 2024-09-12T02:18:23.403 INFO:journalctl@ceph.mds.c.smithi184.stdout:Sep 12 02:18:23 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mds-c[66347]: ignoring --setuser ceph since I am not root 2024-09-12T02:18:23.403 INFO:journalctl@ceph.mds.c.smithi184.stdout:Sep 12 02:18:23 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mds-c[66347]: ignoring --setgroup ceph since I am not root 2024-09-12T02:18:23.403 INFO:journalctl@ceph.mds.c.smithi184.stdout:Sep 12 02:18:23 smithi184 ceph-mds[66351]: -- 172.21.15.184:0/1415635249 <== mon.1 v2:172.21.15.184:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55812abdf860 con 0x55812abbc800 2024-09-12T02:18:23.403 INFO:journalctl@ceph.mds.c.smithi184.stdout:Sep 12 02:18:23 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mds-c[66347]: starting mds.c at 2024-09-12T02:18:23.403 INFO:journalctl@ceph.mds.c.smithi184.stdout:Sep 12 02:18:23 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mds-c[66347]: 2024-09-12T02:18:23.142+0000 7f7294d05640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6835/0: (98) Address already in use 2024-09-12T02:18:23.403 INFO:journalctl@ceph.mds.f.smithi184.stdout:Sep 12 02:18:23 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mds-f[66049]: ignoring --setuser ceph since I am not root 2024-09-12T02:18:23.403 INFO:journalctl@ceph.mds.f.smithi184.stdout:Sep 12 02:18:23 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mds-f[66049]: ignoring --setgroup ceph since I am not root 2024-09-12T02:18:23.403 INFO:journalctl@ceph.mds.f.smithi184.stdout:Sep 12 02:18:23 smithi184 ceph-mds[66053]: -- 172.21.15.184:0/1193262376 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x555678684680 con 0x555678659000 2024-09-12T02:18:23.404 INFO:journalctl@ceph.mds.f.smithi184.stdout:Sep 12 02:18:23 smithi184 ceph-mds[66053]: -- 172.21.15.184:0/1193262376 <== mon.0 v2:172.21.15.16:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55567867b860 con 0x555678659000 2024-09-12T02:18:23.404 INFO:journalctl@ceph.mds.f.smithi184.stdout:Sep 12 02:18:23 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mds-f[66049]: starting mds.f at 2024-09-12T02:18:23.404 INFO:journalctl@ceph.mds.h.smithi033.stdout:Sep 12 02:18:23 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mds-h[64849]: ignoring --setuser ceph since I am not root 2024-09-12T02:18:23.404 INFO:journalctl@ceph.mds.h.smithi033.stdout:Sep 12 02:18:23 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mds-h[64849]: ignoring --setgroup ceph since I am not root 2024-09-12T02:18:23.404 INFO:journalctl@ceph.mds.h.smithi033.stdout:Sep 12 02:18:23 smithi033 ceph-mds[64853]: -- 172.21.15.33:0/2902064937 <== mon.2 v2:172.21.15.33:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561438391c20 con 0x56143836e800 2024-09-12T02:18:23.404 INFO:journalctl@ceph.mds.h.smithi033.stdout:Sep 12 02:18:23 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mds-h[64849]: starting mds.h at 2024-09-12T02:18:23.405 INFO:journalctl@ceph.mds.b.smithi033.stdout:Sep 12 02:18:23 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mds-b[65151]: ignoring --setuser ceph since I am not root 2024-09-12T02:18:23.405 INFO:journalctl@ceph.mds.b.smithi033.stdout:Sep 12 02:18:23 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mds-b[65151]: ignoring --setgroup ceph since I am not root 2024-09-12T02:18:23.405 INFO:journalctl@ceph.mds.b.smithi033.stdout:Sep 12 02:18:23 smithi033 ceph-mds[65155]: -- 172.21.15.33:0/752632100 <== mon.2 v2:172.21.15.33:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559fdd83f860 con 0x559fdd81d400 2024-09-12T02:18:23.405 INFO:journalctl@ceph.mds.b.smithi033.stdout:Sep 12 02:18:23 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mds-b[65151]: starting mds.b at 2024-09-12T02:18:23.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:18:23 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/3184644553' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-12T02:18:23.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:18:23 smithi033 ceph-mon[35332]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-12T02:18:23.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:18:23 smithi033 ceph-mon[35332]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-12T02:18:23.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:18:23 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/3184644553' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-12T02:18:23.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:18:23 smithi033 ceph-mon[35332]: osdmap e83: 12 total, 12 up, 12 in 2024-09-12T02:18:23.670 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:18:23 smithi033 ceph-mon[35332]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-09-12T02:18:23.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:18:23 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/3184644553' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-12T02:18:23.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:18:23 smithi016 ceph-mon[30947]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-12T02:18:23.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:18:23 smithi016 ceph-mon[30947]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-12T02:18:23.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:18:23 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/3184644553' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-12T02:18:23.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:18:23 smithi016 ceph-mon[30947]: osdmap e83: 12 total, 12 up, 12 in 2024-09-12T02:18:23.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:18:23 smithi016 ceph-mon[30947]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-09-12T02:18:23.835 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-09-12T02:18:23.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:18:23 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/3184644553' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-12T02:18:23.878 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:18:23 smithi184 ceph-mon[35593]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-12T02:18:23.878 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:18:23 smithi184 ceph-mon[35593]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-12T02:18:23.878 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:18:23 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/3184644553' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-12T02:18:23.878 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:18:23 smithi184 ceph-mon[35593]: osdmap e83: 12 total, 12 up, 12 in 2024-09-12T02:18:23.878 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:18:23 smithi184 ceph-mon[35593]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-09-12T02:18:24.097 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:18:24.497 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:18:24 smithi016 ceph-mon[30947]: pgmap v829: 97 pgs: 97 active+clean; 4.3 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 853 B/s wr, 7 op/s 2024-09-12T02:18:24.497 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:18:24 smithi016 ceph-mon[30947]: mds.? [v2:172.21.15.33:6836/3094814407,v1:172.21.15.33:6838/3094814407] up:boot 2024-09-12T02:18:24.498 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:18:24 smithi016 ceph-mon[30947]: mds.? [v2:172.21.15.184:6832/3255579152,v1:172.21.15.184:6833/3255579152] up:boot 2024-09-12T02:18:24.498 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:18:24 smithi016 ceph-mon[30947]: mds.? [v2:172.21.15.33:6837/3943531514,v1:172.21.15.33:6839/3943531514] up:boot 2024-09-12T02:18:24.498 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:18:24 smithi016 ceph-mon[30947]: mds.? [v2:172.21.15.184:6834/3102321713,v1:172.21.15.184:6837/3102321713] up:boot 2024-09-12T02:18:24.498 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:18:24 smithi016 ceph-mon[30947]: mds.? [v2:172.21.15.184:6835/2749300876,v1:172.21.15.184:6838/2749300876] up:boot 2024-09-12T02:18:24.498 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:18:24 smithi016 ceph-mon[30947]: mds.? [v2:172.21.15.184:6836/2404392593,v1:172.21.15.184:6839/2404392593] up:boot 2024-09-12T02:18:24.498 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:18:24 smithi016 ceph-mon[30947]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-09-12T02:18:24.498 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:18:24 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-12T02:18:24.498 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:18:24 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-12T02:18:24.498 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:18:24 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-12T02:18:24.498 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:18:24 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-12T02:18:24.498 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:18:24 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-12T02:18:24.498 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:18:24 smithi016 ceph-mon[30947]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-12T02:18:24.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:24.581+0000 7f6864fcf640 1 -- 172.21.15.16:0/1768893113 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6860102570 msgr2=0x7f68601029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:24.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:24.581+0000 7f6864fcf640 1 --2- 172.21.15.16:0/1768893113 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6860102570 0x7f68601029b0 secure :-1 s=READY pgs=2166 cs=0 l=1 rev1=1 crypto rx=0x7f6848009900 tx=0x7f684802a480 comp rx=0 tx=0).stop 2024-09-12T02:18:24.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:24.581+0000 7f6864fcf640 1 -- 172.21.15.16:0/1768893113 shutdown_connections 2024-09-12T02:18:24.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:24.581+0000 7f6864fcf640 1 --2- 172.21.15.16:0/1768893113 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6860102ef0 0x7f686010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:24.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:24.581+0000 7f6864fcf640 1 --2- 172.21.15.16:0/1768893113 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6860102570 0x7f68601029b0 unknown :-1 s=CLOSED pgs=2166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:24.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:24.581+0000 7f6864fcf640 1 --2- 172.21.15.16:0/1768893113 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6860100e20 0x7f6860101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:24.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:24.581+0000 7f6864fcf640 1 -- 172.21.15.16:0/1768893113 >> 172.21.15.16:0/1768893113 conn(0x7f68600fc7d0 msgr2=0x7f68600febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:18:24.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:24.581+0000 7f6864fcf640 1 -- 172.21.15.16:0/1768893113 shutdown_connections 2024-09-12T02:18:24.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:24.581+0000 7f6864fcf640 1 -- 172.21.15.16:0/1768893113 wait complete. 2024-09-12T02:18:24.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:24.581+0000 7f6864fcf640 1 Processor -- start 2024-09-12T02:18:24.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:24.581+0000 7f6864fcf640 1 -- start start 2024-09-12T02:18:24.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:24.582+0000 7f6864fcf640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6860100e20 0x7f686010e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:24.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:24.582+0000 7f6864fcf640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6860102570 0x7f686010ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:24.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:24.582+0000 7f6864fcf640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6860102ef0 0x7f686010ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:24.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:24.582+0000 7f6864fcf640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6860112060 con 0x7f6860100e20 2024-09-12T02:18:24.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:24.582+0000 7f6864fcf640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6860111ee0 con 0x7f6860102ef0 2024-09-12T02:18:24.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:24.582+0000 7f6864fcf640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f68601121e0 con 0x7f6860102570 2024-09-12T02:18:24.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:24.582+0000 7f685e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6860100e20 0x7f686010e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:24.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:24.582+0000 7f685e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6860100e20 0x7f686010e4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33620/0 (socket says 172.21.15.16:33620) 2024-09-12T02:18:24.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:24.582+0000 7f685e59c640 1 -- 172.21.15.16:0/2144189008 learned_addr learned my addr 172.21.15.16:0/2144189008 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:18:24.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:24.582+0000 7f685ed9d640 1 --2- 172.21.15.16:0/2144189008 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6860102ef0 0x7f686010ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:24.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:24.582+0000 7f685dd9b640 1 --2- 172.21.15.16:0/2144189008 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6860102570 0x7f686010ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:24.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:24.582+0000 7f685e59c640 1 -- 172.21.15.16:0/2144189008 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6860102570 msgr2=0x7f686010ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:24.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:24.582+0000 7f685e59c640 1 --2- 172.21.15.16:0/2144189008 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6860102570 0x7f686010ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:24.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:24.582+0000 7f685e59c640 1 -- 172.21.15.16:0/2144189008 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6860102ef0 msgr2=0x7f686010ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:24.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:24.582+0000 7f685e59c640 1 --2- 172.21.15.16:0/2144189008 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6860102ef0 0x7f686010ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:24.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:24.582+0000 7f685e59c640 1 -- 172.21.15.16:0/2144189008 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68601a6980 con 0x7f6860100e20 2024-09-12T02:18:24.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:24.582+0000 7f685e59c640 1 --2- 172.21.15.16:0/2144189008 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6860100e20 0x7f686010e4d0 secure :-1 s=READY pgs=2167 cs=0 l=1 rev1=1 crypto rx=0x7f685800cce0 tx=0x7f685800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:18:24.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:24.583+0000 7f68577fe640 1 -- 172.21.15.16:0/2144189008 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6858004b20 con 0x7f6860100e20 2024-09-12T02:18:24.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:24.583+0000 7f68577fe640 1 -- 172.21.15.16:0/2144189008 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6858004cc0 con 0x7f6860100e20 2024-09-12T02:18:24.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:24.583+0000 7f6864fcf640 1 -- 172.21.15.16:0/2144189008 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68601a6c70 con 0x7f6860100e20 2024-09-12T02:18:24.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:24.583+0000 7f68577fe640 1 -- 172.21.15.16:0/2144189008 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f685800f7d0 con 0x7f6860100e20 2024-09-12T02:18:24.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:24.583+0000 7f6864fcf640 1 -- 172.21.15.16:0/2144189008 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68601a7180 con 0x7f6860100e20 2024-09-12T02:18:24.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:24.584+0000 7f6864fcf640 1 -- 172.21.15.16:0/2144189008 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6860101200 con 0x7f6860100e20 2024-09-12T02:18:24.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:24.585+0000 7f68577fe640 1 -- 172.21.15.16:0/2144189008 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f685800f970 con 0x7f6860100e20 2024-09-12T02:18:24.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:24.586+0000 7f68577fe640 1 --2- 172.21.15.16:0/2144189008 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f684c0a2c00 0x7f684c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:24.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:24.586+0000 7f685dd9b640 1 --2- 172.21.15.16:0/2144189008 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f684c0a2c00 0x7f684c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:24.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:24.586+0000 7f68577fe640 1 -- 172.21.15.16:0/2144189008 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f68580cf400 con 0x7f6860100e20 2024-09-12T02:18:24.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:24.586+0000 7f685dd9b640 1 --2- 172.21.15.16:0/2144189008 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f684c0a2c00 0x7f684c0a50c0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f6848009230 tx=0x7f68480058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:18:24.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:24.589+0000 7f68577fe640 1 -- 172.21.15.16:0/2144189008 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6858014030 con 0x7f6860100e20 2024-09-12T02:18:24.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:18:24 smithi033 ceph-mon[35332]: pgmap v829: 97 pgs: 97 active+clean; 4.3 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 853 B/s wr, 7 op/s 2024-09-12T02:18:24.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:18:24 smithi033 ceph-mon[35332]: mds.? [v2:172.21.15.33:6836/3094814407,v1:172.21.15.33:6838/3094814407] up:boot 2024-09-12T02:18:24.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:18:24 smithi033 ceph-mon[35332]: mds.? [v2:172.21.15.184:6832/3255579152,v1:172.21.15.184:6833/3255579152] up:boot 2024-09-12T02:18:24.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:18:24 smithi033 ceph-mon[35332]: mds.? [v2:172.21.15.33:6837/3943531514,v1:172.21.15.33:6839/3943531514] up:boot 2024-09-12T02:18:24.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:18:24 smithi033 ceph-mon[35332]: mds.? [v2:172.21.15.184:6834/3102321713,v1:172.21.15.184:6837/3102321713] up:boot 2024-09-12T02:18:24.670 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:18:24 smithi033 ceph-mon[35332]: mds.? [v2:172.21.15.184:6835/2749300876,v1:172.21.15.184:6838/2749300876] up:boot 2024-09-12T02:18:24.670 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:18:24 smithi033 ceph-mon[35332]: mds.? [v2:172.21.15.184:6836/2404392593,v1:172.21.15.184:6839/2404392593] up:boot 2024-09-12T02:18:24.670 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:18:24 smithi033 ceph-mon[35332]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-09-12T02:18:24.670 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:18:24 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-12T02:18:24.670 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:18:24 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-12T02:18:24.670 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:18:24 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-12T02:18:24.670 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:18:24 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-12T02:18:24.670 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:18:24 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-12T02:18:24.670 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:18:24 smithi033 ceph-mon[35332]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-12T02:18:24.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:24.804+0000 7f6864fcf640 1 -- 172.21.15.16:0/2144189008 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f68601a7440 con 0x7f6860100e20 2024-09-12T02:18:24.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:18:24 smithi184 ceph-mon[35593]: pgmap v829: 97 pgs: 97 active+clean; 4.3 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 853 B/s wr, 7 op/s 2024-09-12T02:18:24.878 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:18:24 smithi184 ceph-mon[35593]: mds.? [v2:172.21.15.33:6836/3094814407,v1:172.21.15.33:6838/3094814407] up:boot 2024-09-12T02:18:24.878 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:18:24 smithi184 ceph-mon[35593]: mds.? [v2:172.21.15.184:6832/3255579152,v1:172.21.15.184:6833/3255579152] up:boot 2024-09-12T02:18:24.878 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:18:24 smithi184 ceph-mon[35593]: mds.? [v2:172.21.15.33:6837/3943531514,v1:172.21.15.33:6839/3943531514] up:boot 2024-09-12T02:18:24.878 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:18:24 smithi184 ceph-mon[35593]: mds.? [v2:172.21.15.184:6834/3102321713,v1:172.21.15.184:6837/3102321713] up:boot 2024-09-12T02:18:24.878 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:18:24 smithi184 ceph-mon[35593]: mds.? [v2:172.21.15.184:6835/2749300876,v1:172.21.15.184:6838/2749300876] up:boot 2024-09-12T02:18:24.878 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:18:24 smithi184 ceph-mon[35593]: mds.? [v2:172.21.15.184:6836/2404392593,v1:172.21.15.184:6839/2404392593] up:boot 2024-09-12T02:18:24.878 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:18:24 smithi184 ceph-mon[35593]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-09-12T02:18:24.878 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:18:24 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-12T02:18:24.878 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:18:24 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-12T02:18:24.878 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:18:24 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-12T02:18:24.878 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:18:24 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-12T02:18:24.878 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:18:24 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-12T02:18:24.878 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:18:24 smithi184 ceph-mon[35593]: from='mgr.14150 172.21.15.16:0/596143580' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-12T02:18:25.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:25.407+0000 7f68577fe640 1 -- 172.21.15.16:0/2144189008 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v28) ==== 104+0+0 (secure 0 0 0) 0x7f685809bce0 con 0x7f6860100e20 2024-09-12T02:18:25.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:25.410+0000 7f6864fcf640 1 -- 172.21.15.16:0/2144189008 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f684c0a2c00 msgr2=0x7f684c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:25.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:25.410+0000 7f6864fcf640 1 --2- 172.21.15.16:0/2144189008 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f684c0a2c00 0x7f684c0a50c0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f6848009230 tx=0x7f68480058d0 comp rx=0 tx=0).stop 2024-09-12T02:18:25.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:25.410+0000 7f6864fcf640 1 -- 172.21.15.16:0/2144189008 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6860100e20 msgr2=0x7f686010e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:25.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:25.410+0000 7f6864fcf640 1 --2- 172.21.15.16:0/2144189008 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6860100e20 0x7f686010e4d0 secure :-1 s=READY pgs=2167 cs=0 l=1 rev1=1 crypto rx=0x7f685800cce0 tx=0x7f685800a620 comp rx=0 tx=0).stop 2024-09-12T02:18:25.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:25.410+0000 7f6864fcf640 1 -- 172.21.15.16:0/2144189008 shutdown_connections 2024-09-12T02:18:25.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:25.410+0000 7f6864fcf640 1 --2- 172.21.15.16:0/2144189008 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f684c0a2c00 0x7f684c0a50c0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:25.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:25.410+0000 7f6864fcf640 1 --2- 172.21.15.16:0/2144189008 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6860102ef0 0x7f686010ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:25.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:25.410+0000 7f6864fcf640 1 --2- 172.21.15.16:0/2144189008 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6860102570 0x7f686010ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:25.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:25.410+0000 7f6864fcf640 1 --2- 172.21.15.16:0/2144189008 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6860100e20 0x7f686010e4d0 unknown :-1 s=CLOSED pgs=2167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:25.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:25.410+0000 7f6864fcf640 1 -- 172.21.15.16:0/2144189008 >> 172.21.15.16:0/2144189008 conn(0x7f68600fc7d0 msgr2=0x7f68600febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:18:25.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:25.411+0000 7f6864fcf640 1 -- 172.21.15.16:0/2144189008 shutdown_connections 2024-09-12T02:18:25.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:25.411+0000 7f6864fcf640 1 -- 172.21.15.16:0/2144189008 wait complete. 2024-09-12T02:18:25.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:18:25 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/2144189008' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-12T02:18:25.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:18:25 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/2144189008' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-12T02:18:25.779 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:18:25 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/2144189008' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-12T02:18:26.131 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-12T02:18:26.395 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:18:26.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:18:26 smithi016 ceph-mon[30947]: pgmap v831: 97 pgs: 97 active+clean; 4.3 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 819 B/s wr, 7 op/s 2024-09-12T02:18:26.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:18:26 smithi016 ceph-mon[30947]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-12T02:18:26.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:18:26 smithi016 ceph-mon[30947]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-12T02:18:26.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:18:26 smithi016 ceph-mon[30947]: Cluster is now healthy 2024-09-12T02:18:26.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:18:26 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/2144189008' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-12T02:18:26.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:18:26 smithi016 ceph-mon[30947]: fsmap 12 up:standby 2024-09-12T02:18:26.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:18:26 smithi033 ceph-mon[35332]: pgmap v831: 97 pgs: 97 active+clean; 4.3 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 819 B/s wr, 7 op/s 2024-09-12T02:18:26.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:18:26 smithi033 ceph-mon[35332]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-12T02:18:26.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:18:26 smithi033 ceph-mon[35332]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-12T02:18:26.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:18:26 smithi033 ceph-mon[35332]: Cluster is now healthy 2024-09-12T02:18:26.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:18:26 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/2144189008' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-12T02:18:26.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:18:26 smithi033 ceph-mon[35332]: fsmap 12 up:standby 2024-09-12T02:18:26.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:26.875+0000 7fe9f68fe640 1 -- 172.21.15.16:0/2957497047 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9f0106f10 msgr2=0x7fe9f010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:26.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:26.875+0000 7fe9f68fe640 1 --2- 172.21.15.16:0/2957497047 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9f0106f10 0x7fe9f010ea90 secure :-1 s=READY pgs=2168 cs=0 l=1 rev1=1 crypto rx=0x7fe9e8009ca0 tx=0x7fe9e802a8c0 comp rx=0 tx=0).stop 2024-09-12T02:18:26.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:26.875+0000 7fe9f68fe640 1 -- 172.21.15.16:0/2957497047 shutdown_connections 2024-09-12T02:18:26.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:26.875+0000 7fe9f68fe640 1 --2- 172.21.15.16:0/2957497047 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9f0106f10 0x7fe9f010ea90 unknown :-1 s=CLOSED pgs=2168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:26.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:26.875+0000 7fe9f68fe640 1 --2- 172.21.15.16:0/2957497047 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe9f0102790 0x7fe9f01067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:26.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:26.875+0000 7fe9f68fe640 1 --2- 172.21.15.16:0/2957497047 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe9f0101de0 0x7fe9f01021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:26.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:26.875+0000 7fe9f68fe640 1 -- 172.21.15.16:0/2957497047 >> 172.21.15.16:0/2957497047 conn(0x7fe9f00fd8c0 msgr2=0x7fe9f00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:18:26.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:26.875+0000 7fe9f68fe640 1 -- 172.21.15.16:0/2957497047 shutdown_connections 2024-09-12T02:18:26.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:26.875+0000 7fe9f68fe640 1 -- 172.21.15.16:0/2957497047 wait complete. 2024-09-12T02:18:26.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:26.876+0000 7fe9f68fe640 1 Processor -- start 2024-09-12T02:18:26.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:26.876+0000 7fe9f68fe640 1 -- start start 2024-09-12T02:18:26.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:26.877+0000 7fe9f68fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe9f0101de0 0x7fe9f01983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:26.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:26.877+0000 7fe9f68fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9f0102790 0x7fe9f0198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:26.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:26.877+0000 7fe9f68fe640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe9f0106f10 0x7fe9f019cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:26.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:26.877+0000 7fe9f68fe640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe9f0111f70 con 0x7fe9f0102790 2024-09-12T02:18:26.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:26.877+0000 7fe9f68fe640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe9f0111df0 con 0x7fe9f0101de0 2024-09-12T02:18:26.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:26.877+0000 7fe9f68fe640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe9f01120f0 con 0x7fe9f0106f10 2024-09-12T02:18:26.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:26.877+0000 7fe9ef7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9f0102790 0x7fe9f0198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:26.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:26.877+0000 7fe9ef7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9f0102790 0x7fe9f0198910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33650/0 (socket says 172.21.15.16:33650) 2024-09-12T02:18:26.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:26.877+0000 7fe9ef7fe640 1 -- 172.21.15.16:0/3769294432 learned_addr learned my addr 172.21.15.16:0/3769294432 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:18:26.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:26.877+0000 7fe9f4e9b640 1 --2- 172.21.15.16:0/3769294432 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe9f0106f10 0x7fe9f019cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:26.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:26.877+0000 7fe9effff640 1 --2- 172.21.15.16:0/3769294432 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe9f0101de0 0x7fe9f01983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:26.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:26.877+0000 7fe9ef7fe640 1 -- 172.21.15.16:0/3769294432 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe9f0106f10 msgr2=0x7fe9f019cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:26.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:26.877+0000 7fe9ef7fe640 1 --2- 172.21.15.16:0/3769294432 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe9f0106f10 0x7fe9f019cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:26.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:26.877+0000 7fe9ef7fe640 1 -- 172.21.15.16:0/3769294432 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe9f0101de0 msgr2=0x7fe9f01983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:26.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:26.877+0000 7fe9ef7fe640 1 --2- 172.21.15.16:0/3769294432 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe9f0101de0 0x7fe9f01983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:26.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:26.877+0000 7fe9ef7fe640 1 -- 172.21.15.16:0/3769294432 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9f019d380 con 0x7fe9f0102790 2024-09-12T02:18:26.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:26.877+0000 7fe9ef7fe640 1 --2- 172.21.15.16:0/3769294432 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9f0102790 0x7fe9f0198910 secure :-1 s=READY pgs=2169 cs=0 l=1 rev1=1 crypto rx=0x7fe9e400c890 tx=0x7fe9e400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:18:26.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:26.878+0000 7fe9ed7fa640 1 -- 172.21.15.16:0/3769294432 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9e4004220 con 0x7fe9f0102790 2024-09-12T02:18:26.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:26.878+0000 7fe9ed7fa640 1 -- 172.21.15.16:0/3769294432 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe9e40043c0 con 0x7fe9f0102790 2024-09-12T02:18:26.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:26.878+0000 7fe9f68fe640 1 -- 172.21.15.16:0/3769294432 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9f019d670 con 0x7fe9f0102790 2024-09-12T02:18:26.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:26.878+0000 7fe9ed7fa640 1 -- 172.21.15.16:0/3769294432 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9e40049e0 con 0x7fe9f0102790 2024-09-12T02:18:26.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:26.878+0000 7fe9f68fe640 1 -- 172.21.15.16:0/3769294432 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9f01a4f50 con 0x7fe9f0102790 2024-09-12T02:18:26.881 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:18:26 smithi184 ceph-mon[35593]: pgmap v831: 97 pgs: 97 active+clean; 4.3 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 819 B/s wr, 7 op/s 2024-09-12T02:18:26.881 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:18:26 smithi184 ceph-mon[35593]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-12T02:18:26.881 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:18:26 smithi184 ceph-mon[35593]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-12T02:18:26.881 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:18:26 smithi184 ceph-mon[35593]: Cluster is now healthy 2024-09-12T02:18:26.881 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:18:26 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/2144189008' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-12T02:18:26.881 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:18:26 smithi184 ceph-mon[35593]: fsmap 12 up:standby 2024-09-12T02:18:26.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:26.879+0000 7fe9f68fe640 1 -- 172.21.15.16:0/3769294432 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9f01021c0 con 0x7fe9f0102790 2024-09-12T02:18:26.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:26.880+0000 7fe9ed7fa640 1 -- 172.21.15.16:0/3769294432 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fe9e40079b0 con 0x7fe9f0102790 2024-09-12T02:18:26.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:26.881+0000 7fe9ed7fa640 1 --2- 172.21.15.16:0/3769294432 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fe9d80a2b80 0x7fe9d80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:26.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:26.881+0000 7fe9effff640 1 --2- 172.21.15.16:0/3769294432 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fe9d80a2b80 0x7fe9d80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:26.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:26.881+0000 7fe9ed7fa640 1 -- 172.21.15.16:0/3769294432 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7fe9e40cf0a0 con 0x7fe9f0102790 2024-09-12T02:18:26.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:26.881+0000 7fe9effff640 1 --2- 172.21.15.16:0/3769294432 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fe9d80a2b80 0x7fe9d80a5040 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fe9e0009bd0 tx=0x7fe9e0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:18:26.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:26.885+0000 7fe9ed7fa640 1 -- 172.21.15.16:0/3769294432 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe9e4014030 con 0x7fe9f0102790 2024-09-12T02:18:27.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:27.062+0000 7fe9f68fe640 1 -- 172.21.15.16:0/3769294432 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fe9f002d7c0 con 0x7fe9f0102790 2024-09-12T02:18:27.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:27.414+0000 7fe9ed7fa640 1 -- 172.21.15.16:0/3769294432 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v84) ==== 181+0+0 (secure 0 0 0) 0x7fe9e409b8a0 con 0x7fe9f0102790 2024-09-12T02:18:27.415 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_metadata' removed 2024-09-12T02:18:27.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:27.417+0000 7fe9f68fe640 1 -- 172.21.15.16:0/3769294432 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fe9d80a2b80 msgr2=0x7fe9d80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:27.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:27.417+0000 7fe9f68fe640 1 --2- 172.21.15.16:0/3769294432 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fe9d80a2b80 0x7fe9d80a5040 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fe9e0009bd0 tx=0x7fe9e0005f90 comp rx=0 tx=0).stop 2024-09-12T02:18:27.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:27.417+0000 7fe9f68fe640 1 -- 172.21.15.16:0/3769294432 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9f0102790 msgr2=0x7fe9f0198910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:27.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:27.417+0000 7fe9f68fe640 1 --2- 172.21.15.16:0/3769294432 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9f0102790 0x7fe9f0198910 secure :-1 s=READY pgs=2169 cs=0 l=1 rev1=1 crypto rx=0x7fe9e400c890 tx=0x7fe9e400cd50 comp rx=0 tx=0).stop 2024-09-12T02:18:27.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:27.417+0000 7fe9f68fe640 1 -- 172.21.15.16:0/3769294432 shutdown_connections 2024-09-12T02:18:27.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:27.417+0000 7fe9f68fe640 1 --2- 172.21.15.16:0/3769294432 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7fe9d80a2b80 0x7fe9d80a5040 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:27.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:27.417+0000 7fe9f68fe640 1 --2- 172.21.15.16:0/3769294432 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe9f0106f10 0x7fe9f019cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:27.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:27.417+0000 7fe9f68fe640 1 --2- 172.21.15.16:0/3769294432 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9f0102790 0x7fe9f0198910 secure :-1 s=CLOSED pgs=2169 cs=0 l=1 rev1=1 crypto rx=0x7fe9e400c890 tx=0x7fe9e400cd50 comp rx=0 tx=0).stop 2024-09-12T02:18:27.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:27.417+0000 7fe9f68fe640 1 --2- 172.21.15.16:0/3769294432 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe9f0101de0 0x7fe9f01983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:27.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:27.417+0000 7fe9f68fe640 1 -- 172.21.15.16:0/3769294432 >> 172.21.15.16:0/3769294432 conn(0x7fe9f00fd8c0 msgr2=0x7fe9f00feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:18:27.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:27.417+0000 7fe9f68fe640 1 -- 172.21.15.16:0/3769294432 shutdown_connections 2024-09-12T02:18:27.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:27.417+0000 7fe9f68fe640 1 -- 172.21.15.16:0/3769294432 wait complete. 2024-09-12T02:18:27.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:18:27 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/3769294432' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-12T02:18:27.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:18:27 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/3769294432' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-12T02:18:27.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:18:27 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/3769294432' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-12T02:18:28.203 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-12T02:18:28.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:18:28 smithi016 ceph-mon[30947]: pgmap v832: 97 pgs: 97 active+clean; 4.3 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 307 B/s wr, 5 op/s 2024-09-12T02:18:28.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:18:28 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/3769294432' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-12T02:18:28.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:18:28 smithi016 ceph-mon[30947]: osdmap e84: 12 total, 12 up, 12 in 2024-09-12T02:18:28.469 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:18:28.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:18:28 smithi033 ceph-mon[35332]: pgmap v832: 97 pgs: 97 active+clean; 4.3 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 307 B/s wr, 5 op/s 2024-09-12T02:18:28.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:18:28 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/3769294432' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-12T02:18:28.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:18:28 smithi033 ceph-mon[35332]: osdmap e84: 12 total, 12 up, 12 in 2024-09-12T02:18:28.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:18:28 smithi184 ceph-mon[35593]: pgmap v832: 97 pgs: 97 active+clean; 4.3 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 307 B/s wr, 5 op/s 2024-09-12T02:18:28.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:18:28 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/3769294432' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-12T02:18:28.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:18:28 smithi184 ceph-mon[35593]: osdmap e84: 12 total, 12 up, 12 in 2024-09-12T02:18:28.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:28.975+0000 7f3ad262d640 1 -- 172.21.15.16:0/730775314 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3acc101de0 msgr2=0x7f3acc1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:28.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:28.975+0000 7f3ad262d640 1 --2- 172.21.15.16:0/730775314 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3acc101de0 0x7f3acc1021c0 secure :-1 s=READY pgs=2170 cs=0 l=1 rev1=1 crypto rx=0x7f3abc009900 tx=0x7f3abc02a410 comp rx=0 tx=0).stop 2024-09-12T02:18:28.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:28.975+0000 7f3ad262d640 1 -- 172.21.15.16:0/730775314 shutdown_connections 2024-09-12T02:18:28.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:28.975+0000 7f3ad262d640 1 --2- 172.21.15.16:0/730775314 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3acc106f10 0x7f3acc10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:28.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:28.975+0000 7f3ad262d640 1 --2- 172.21.15.16:0/730775314 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3acc102790 0x7f3acc1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:28.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:28.975+0000 7f3ad262d640 1 --2- 172.21.15.16:0/730775314 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3acc101de0 0x7f3acc1021c0 unknown :-1 s=CLOSED pgs=2170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:28.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:28.975+0000 7f3ad262d640 1 -- 172.21.15.16:0/730775314 >> 172.21.15.16:0/730775314 conn(0x7f3acc0fd8c0 msgr2=0x7f3acc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:18:28.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:28.975+0000 7f3ad262d640 1 -- 172.21.15.16:0/730775314 shutdown_connections 2024-09-12T02:18:28.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:28.975+0000 7f3ad262d640 1 -- 172.21.15.16:0/730775314 wait complete. 2024-09-12T02:18:28.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:28.976+0000 7f3ad262d640 1 Processor -- start 2024-09-12T02:18:28.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:28.976+0000 7f3ad262d640 1 -- start start 2024-09-12T02:18:28.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:28.977+0000 7f3ad262d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3acc101de0 0x7f3acc198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:28.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:28.977+0000 7f3ad262d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3acc102790 0x7f3acc198840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:28.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:28.977+0000 7f3ad262d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3acc106f10 0x7f3acc19cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:28.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:28.977+0000 7f3ad262d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3acc111f70 con 0x7f3acc101de0 2024-09-12T02:18:28.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:28.977+0000 7f3ad262d640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3acc111df0 con 0x7f3acc106f10 2024-09-12T02:18:28.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:28.977+0000 7f3ad262d640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f3acc1120f0 con 0x7f3acc102790 2024-09-12T02:18:28.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:28.977+0000 7f3acbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3acc101de0 0x7f3acc198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:28.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:28.977+0000 7f3acbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3acc101de0 0x7f3acc198300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33674/0 (socket says 172.21.15.16:33674) 2024-09-12T02:18:28.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:28.977+0000 7f3acbfff640 1 -- 172.21.15.16:0/3836906197 learned_addr learned my addr 172.21.15.16:0/3836906197 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T02:18:28.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:28.977+0000 7f3acb7fe640 1 --2- 172.21.15.16:0/3836906197 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3acc102790 0x7f3acc198840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:28.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:28.977+0000 7f3ad0bca640 1 --2- 172.21.15.16:0/3836906197 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3acc106f10 0x7f3acc19cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:28.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:28.977+0000 7f3acbfff640 1 -- 172.21.15.16:0/3836906197 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3acc102790 msgr2=0x7f3acc198840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:28.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:28.977+0000 7f3acbfff640 1 --2- 172.21.15.16:0/3836906197 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3acc102790 0x7f3acc198840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:28.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:28.977+0000 7f3acbfff640 1 -- 172.21.15.16:0/3836906197 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3acc106f10 msgr2=0x7f3acc19cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:28.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:28.977+0000 7f3acbfff640 1 --2- 172.21.15.16:0/3836906197 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3acc106f10 0x7f3acc19cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:28.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:28.977+0000 7f3acbfff640 1 -- 172.21.15.16:0/3836906197 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3acc19d2b0 con 0x7f3acc101de0 2024-09-12T02:18:28.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:28.977+0000 7f3ad0bca640 1 --2- 172.21.15.16:0/3836906197 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3acc106f10 0x7f3acc19cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:18:28.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:28.977+0000 7f3acbfff640 1 --2- 172.21.15.16:0/3836906197 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3acc101de0 0x7f3acc198300 secure :-1 s=READY pgs=2171 cs=0 l=1 rev1=1 crypto rx=0x7f3abc005ab0 tx=0x7f3abc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:18:28.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:28.977+0000 7f3ac97fa640 1 -- 172.21.15.16:0/3836906197 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3abc03f030 con 0x7f3acc101de0 2024-09-12T02:18:28.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:28.978+0000 7f3ac97fa640 1 -- 172.21.15.16:0/3836906197 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3abc003ce0 con 0x7f3acc101de0 2024-09-12T02:18:28.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:28.978+0000 7f3ad262d640 1 -- 172.21.15.16:0/3836906197 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3acc19d540 con 0x7f3acc101de0 2024-09-12T02:18:28.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:28.978+0000 7f3ac97fa640 1 -- 172.21.15.16:0/3836906197 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3abc035070 con 0x7f3acc101de0 2024-09-12T02:18:28.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:28.978+0000 7f3ad262d640 1 -- 172.21.15.16:0/3836906197 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3acc1a4e80 con 0x7f3acc101de0 2024-09-12T02:18:28.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:28.979+0000 7f3ad262d640 1 -- 172.21.15.16:0/3836906197 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3acc1032c0 con 0x7f3acc101de0 2024-09-12T02:18:28.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:28.980+0000 7f3ac97fa640 1 -- 172.21.15.16:0/3836906197 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f3abc042030 con 0x7f3acc101de0 2024-09-12T02:18:28.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:28.981+0000 7f3ac97fa640 1 --2- 172.21.15.16:0/3836906197 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f3ab40a2b80 0x7f3ab40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:28.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:28.981+0000 7f3acb7fe640 1 --2- 172.21.15.16:0/3836906197 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f3ab40a2b80 0x7f3ab40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:28.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:28.981+0000 7f3ac97fa640 1 -- 172.21.15.16:0/3836906197 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7915+0+0 (secure 0 0 0) 0x7f3abc0ef820 con 0x7f3acc101de0 2024-09-12T02:18:28.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:28.981+0000 7f3acb7fe640 1 --2- 172.21.15.16:0/3836906197 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f3ab40a2b80 0x7f3ab40a5040 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f3acc108880 tx=0x7f3ac000a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:18:28.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:28.985+0000 7f3ac97fa640 1 -- 172.21.15.16:0/3836906197 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3abc0bc910 con 0x7f3acc101de0 2024-09-12T02:18:29.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:29.161+0000 7f3ad262d640 1 -- 172.21.15.16:0/3836906197 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f3acc10a0f0 con 0x7f3acc101de0 2024-09-12T02:18:29.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:29.421+0000 7f3ac97fa640 1 -- 172.21.15.16:0/3836906197 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v85) ==== 169+0+0 (secure 0 0 0) 0x7f3abc0bc210 con 0x7f3acc101de0 2024-09-12T02:18:29.422 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data' removed 2024-09-12T02:18:29.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:29.424+0000 7f3ad262d640 1 -- 172.21.15.16:0/3836906197 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f3ab40a2b80 msgr2=0x7f3ab40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:29.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:29.424+0000 7f3ad262d640 1 --2- 172.21.15.16:0/3836906197 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f3ab40a2b80 0x7f3ab40a5040 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f3acc108880 tx=0x7f3ac000a2d0 comp rx=0 tx=0).stop 2024-09-12T02:18:29.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:29.424+0000 7f3ad262d640 1 -- 172.21.15.16:0/3836906197 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3acc101de0 msgr2=0x7f3acc198300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:29.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:29.424+0000 7f3ad262d640 1 --2- 172.21.15.16:0/3836906197 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3acc101de0 0x7f3acc198300 secure :-1 s=READY pgs=2171 cs=0 l=1 rev1=1 crypto rx=0x7f3abc005ab0 tx=0x7f3abc0042f0 comp rx=0 tx=0).stop 2024-09-12T02:18:29.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:29.424+0000 7f3ad262d640 1 -- 172.21.15.16:0/3836906197 shutdown_connections 2024-09-12T02:18:29.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:29.424+0000 7f3ad262d640 1 --2- 172.21.15.16:0/3836906197 >> [v2:172.21.15.16:6800/3887444820,v1:172.21.15.16:6801/3887444820] conn(0x7f3ab40a2b80 0x7f3ab40a5040 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:29.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:29.424+0000 7f3ad262d640 1 --2- 172.21.15.16:0/3836906197 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3acc106f10 0x7f3acc19cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:29.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:29.424+0000 7f3ad262d640 1 --2- 172.21.15.16:0/3836906197 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3acc102790 0x7f3acc198840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:29.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:29.424+0000 7f3ad262d640 1 --2- 172.21.15.16:0/3836906197 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3acc101de0 0x7f3acc198300 unknown :-1 s=CLOSED pgs=2171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:29.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:29.424+0000 7f3ad262d640 1 -- 172.21.15.16:0/3836906197 >> 172.21.15.16:0/3836906197 conn(0x7f3acc0fd8c0 msgr2=0x7f3acc0fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:18:29.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:29.425+0000 7f3ad262d640 1 -- 172.21.15.16:0/3836906197 shutdown_connections 2024-09-12T02:18:29.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T02:18:29.425+0000 7f3ad262d640 1 -- 172.21.15.16:0/3836906197 wait complete. 2024-09-12T02:18:29.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:18:29 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/3836906197' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-12T02:18:29.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:18:29 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/3836906197' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-12T02:18:29.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:18:29 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/3836906197' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-12T02:18:30.231 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-12T02:18:30.259 DEBUG:teuthology.orchestra.run.smithi033:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-12T02:18:30.292 DEBUG:teuthology.orchestra.run.smithi184:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-12T02:18:30.327 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-09-12T02:18:30.327 DEBUG:teuthology.orchestra.run.smithi016:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-12T02:18:30.343 DEBUG:teuthology.orchestra.run.smithi033:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-12T02:18:30.360 DEBUG:teuthology.orchestra.run.smithi184:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-12T02:18:30.386 INFO:tasks.cephadm:Stopping all daemons... 2024-09-12T02:18:30.386 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-09-12T02:18:30.386 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ceph-879f3b56-70a9-11ef-bcea-c7b262605968@mon.a 2024-09-12T02:18:30.460 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:18:30 smithi016 ceph-mon[30947]: pgmap v834: 65 pgs: 65 active+clean; 577 KiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:18:30.460 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:18:30 smithi016 ceph-mon[30947]: from='client.? 172.21.15.16:0/3836906197' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-12T02:18:30.460 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:18:30 smithi016 ceph-mon[30947]: osdmap e85: 12 total, 12 up, 12 in 2024-09-12T02:18:30.461 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:18:30 smithi016 systemd[1]: Stopping Ceph mon.a for 879f3b56-70a9-11ef-bcea-c7b262605968... 2024-09-12T02:18:30.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:18:30 smithi184 ceph-mon[35593]: pgmap v834: 65 pgs: 65 active+clean; 577 KiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:18:30.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:18:30 smithi184 ceph-mon[35593]: from='client.? 172.21.15.16:0/3836906197' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-12T02:18:30.628 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:18:30 smithi184 ceph-mon[35593]: osdmap e85: 12 total, 12 up, 12 in 2024-09-12T02:18:30.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:18:30 smithi033 ceph-mon[35332]: pgmap v834: 65 pgs: 65 active+clean; 577 KiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:18:30.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:18:30 smithi033 ceph-mon[35332]: from='client.? 172.21.15.16:0/3836906197' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-12T02:18:30.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:18:30 smithi033 ceph-mon[35332]: osdmap e85: 12 total, 12 up, 12 in 2024-09-12T02:18:30.809 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:18:30 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mon-a[30943]: 2024-09-12T02:18:30.706+0000 7efe47e47640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T02:18:30.809 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:18:30 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mon-a[30943]: 2024-09-12T02:18:30.706+0000 7efe47e47640 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-09-12T02:18:31.089 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:18:30 smithi016 podman[130678]: 2024-09-12 02:18:30.809826796 +0000 UTC m=+0.223513426 container died ab3eea3e12feba118c6232cbc44962043a88bd3f49c4e975388298824a9c0110 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-09-12T02:18:31.432 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:18:31 smithi016 podman[130678]: 2024-09-12 02:18:31.239381848 +0000 UTC m=+0.653068475 container cleanup ab3eea3e12feba118c6232cbc44962043a88bd3f49c4e975388298824a9c0110 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mon-a, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-09-12T02:18:31.432 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:18:31 smithi016 bash[130678]: ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mon-a 2024-09-12T02:18:31.665 DEBUG:teuthology.orchestra.run.smithi016:> sudo pkill -f 'journalctl -f -n 0 -u ceph-879f3b56-70a9-11ef-bcea-c7b262605968@mon.a.service' 2024-09-12T02:18:31.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:18:31 smithi016 podman[130691]: 2024-09-12 02:18:31.432524952 +0000 UTC m=+0.616743733 container remove ab3eea3e12feba118c6232cbc44962043a88bd3f49c4e975388298824a9c0110 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mon-a, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-09-12T02:18:31.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:18:31 smithi016 systemd[1]: ceph-879f3b56-70a9-11ef-bcea-c7b262605968@mon.a.service: Deactivated successfully. 2024-09-12T02:18:31.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:18:31 smithi016 systemd[1]: Stopped Ceph mon.a for 879f3b56-70a9-11ef-bcea-c7b262605968. 2024-09-12T02:18:31.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 02:18:31 smithi016 systemd[1]: ceph-879f3b56-70a9-11ef-bcea-c7b262605968@mon.a.service: Consumed 24.959s CPU time. 2024-09-12T02:18:31.714 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T02:18:31.715 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-09-12T02:18:31.715 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-09-12T02:18:31.715 DEBUG:teuthology.orchestra.run.smithi033:> sudo systemctl stop ceph-879f3b56-70a9-11ef-bcea-c7b262605968@mon.b 2024-09-12T02:18:32.011 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:18:31 smithi033 systemd[1]: Stopping Ceph mon.b for 879f3b56-70a9-11ef-bcea-c7b262605968... 2024-09-12T02:18:32.369 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:18:32 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mon-b[35328]: 2024-09-12T02:18:32.010+0000 7f7eac567640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T02:18:32.370 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:18:32 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mon-b[35328]: 2024-09-12T02:18:32.010+0000 7f7eac567640 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-09-12T02:18:32.370 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:18:32 smithi033 podman[70050]: 2024-09-12 02:18:32.113023688 +0000 UTC m=+0.198252089 container died 0883b7e46f0612c0ad18a20b2ec73fe09a8dd0fe22c4713ed6e8ae3c8906c2c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mon-b, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-12T02:18:32.645 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:18:32 smithi033 podman[70050]: 2024-09-12 02:18:32.452772381 +0000 UTC m=+0.538000789 container cleanup 0883b7e46f0612c0ad18a20b2ec73fe09a8dd0fe22c4713ed6e8ae3c8906c2c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mon-b, io.buildah.version=1.37.2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-12T02:18:32.645 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:18:32 smithi033 bash[70050]: ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mon-b 2024-09-12T02:18:32.883 DEBUG:teuthology.orchestra.run.smithi033:> sudo pkill -f 'journalctl -f -n 0 -u ceph-879f3b56-70a9-11ef-bcea-c7b262605968@mon.b.service' 2024-09-12T02:18:32.905 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:18:32 smithi033 podman[70062]: 2024-09-12 02:18:32.645872775 +0000 UTC m=+0.532358474 container remove 0883b7e46f0612c0ad18a20b2ec73fe09a8dd0fe22c4713ed6e8ae3c8906c2c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mon-b, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731) 2024-09-12T02:18:32.905 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:18:32 smithi033 systemd[1]: ceph-879f3b56-70a9-11ef-bcea-c7b262605968@mon.b.service: Deactivated successfully. 2024-09-12T02:18:32.905 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:18:32 smithi033 systemd[1]: Stopped Ceph mon.b for 879f3b56-70a9-11ef-bcea-c7b262605968. 2024-09-12T02:18:32.905 INFO:journalctl@ceph.mon.b.smithi033.stdout:Sep 12 02:18:32 smithi033 systemd[1]: ceph-879f3b56-70a9-11ef-bcea-c7b262605968@mon.b.service: Consumed 14.954s CPU time. 2024-09-12T02:18:32.927 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T02:18:32.927 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-09-12T02:18:32.927 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-09-12T02:18:32.927 DEBUG:teuthology.orchestra.run.smithi184:> sudo systemctl stop ceph-879f3b56-70a9-11ef-bcea-c7b262605968@mon.c 2024-09-12T02:18:33.237 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:18:32 smithi184 systemd[1]: Stopping Ceph mon.c for 879f3b56-70a9-11ef-bcea-c7b262605968... 2024-09-12T02:18:33.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:18:33 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mon-c[35589]: 2024-09-12T02:18:33.237+0000 7f8f7e593640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T02:18:33.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:18:33 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mon-c[35589]: 2024-09-12T02:18:33.237+0000 7f8f7e593640 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-09-12T02:18:33.627 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:18:33 smithi184 podman[72409]: 2024-09-12 02:18:33.451746664 +0000 UTC m=+0.316445391 container died 4c7277924f9d216fad4502f62af671b7add7f303e71dc7fd44f1e355ccab67fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mon-c, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.license=GPLv2) 2024-09-12T02:18:34.105 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:18:33 smithi184 podman[72409]: 2024-09-12 02:18:33.804011099 +0000 UTC m=+0.668709819 container cleanup 4c7277924f9d216fad4502f62af671b7add7f303e71dc7fd44f1e355ccab67fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mon-c, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T02:18:34.105 INFO:journalctl@ceph.mon.c.smithi184.stdout:Sep 12 02:18:33 smithi184 bash[72409]: ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mon-c 2024-09-12T02:18:34.274 DEBUG:teuthology.orchestra.run.smithi184:> sudo pkill -f 'journalctl -f -n 0 -u ceph-879f3b56-70a9-11ef-bcea-c7b262605968@mon.c.service' 2024-09-12T02:18:34.322 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T02:18:34.323 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-09-12T02:18:34.323 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-09-12T02:18:34.323 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ceph-879f3b56-70a9-11ef-bcea-c7b262605968@mgr.x 2024-09-12T02:18:34.652 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 02:18:34 smithi016 systemd[1]: Stopping Ceph mgr.x for 879f3b56-70a9-11ef-bcea-c7b262605968... 2024-09-12T02:18:34.934 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 02:18:34 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-x[31173]: 2024-09-12T02:18:34.651+0000 7f1979327640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T02:18:34.934 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 02:18:34 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-x[31173]: 2024-09-12T02:18:34.651+0000 7f1979327640 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-09-12T02:18:34.934 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 02:18:34 smithi016 podman[130798]: 2024-09-12 02:18:34.68381613 +0000 UTC m=+0.151277342 container died ea29060771e897661e9e82aa42c5672ce70af5154490ae841c10e83a1eaa3d46 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-x, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-09-12T02:18:35.395 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 02:18:35 smithi016 podman[130798]: 2024-09-12 02:18:35.177808053 +0000 UTC m=+0.645269265 container cleanup ea29060771e897661e9e82aa42c5672ce70af5154490ae841c10e83a1eaa3d46 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-x, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3) 2024-09-12T02:18:35.395 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 02:18:35 smithi016 bash[130798]: ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-x 2024-09-12T02:18:35.526 DEBUG:teuthology.orchestra.run.smithi016:> sudo pkill -f 'journalctl -f -n 0 -u ceph-879f3b56-70a9-11ef-bcea-c7b262605968@mgr.x.service' 2024-09-12T02:18:35.564 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T02:18:35.564 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-09-12T02:18:35.564 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-09-12T02:18:35.564 DEBUG:teuthology.orchestra.run.smithi033:> sudo systemctl stop ceph-879f3b56-70a9-11ef-bcea-c7b262605968@mgr.y 2024-09-12T02:18:35.887 INFO:journalctl@ceph.mgr.y.smithi033.stdout:Sep 12 02:18:35 smithi033 systemd[1]: Stopping Ceph mgr.y for 879f3b56-70a9-11ef-bcea-c7b262605968... 2024-09-12T02:18:36.158 INFO:journalctl@ceph.mgr.y.smithi033.stdout:Sep 12 02:18:35 smithi033 podman[70176]: 2024-09-12 02:18:35.887152256 +0000 UTC m=+0.121615171 container died c98e15cd9dd54a0647d7f1b039d3e31c7457aa76c78d6a31575d2457deec1f9a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-y, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.37.2) 2024-09-12T02:18:36.419 INFO:journalctl@ceph.mgr.y.smithi033.stdout:Sep 12 02:18:36 smithi033 podman[70176]: 2024-09-12 02:18:36.232615536 +0000 UTC m=+0.467078483 container cleanup c98e15cd9dd54a0647d7f1b039d3e31c7457aa76c78d6a31575d2457deec1f9a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-y, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T02:18:36.419 INFO:journalctl@ceph.mgr.y.smithi033.stdout:Sep 12 02:18:36 smithi033 bash[70176]: ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-y 2024-09-12T02:18:36.419 INFO:journalctl@ceph.mgr.y.smithi033.stdout:Sep 12 02:18:36 smithi033 podman[70191]: 2024-09-12 02:18:36.392046061 +0000 UTC m=+0.494796368 container remove c98e15cd9dd54a0647d7f1b039d3e31c7457aa76c78d6a31575d2457deec1f9a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-y, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=) 2024-09-12T02:18:36.420 INFO:journalctl@ceph.mgr.y.smithi033.stdout:Sep 12 02:18:36 smithi033 systemd[1]: ceph-879f3b56-70a9-11ef-bcea-c7b262605968@mgr.y.service: Main process exited, code=exited, status=143/n/a 2024-09-12T02:18:36.609 DEBUG:teuthology.orchestra.run.smithi033:> sudo pkill -f 'journalctl -f -n 0 -u ceph-879f3b56-70a9-11ef-bcea-c7b262605968@mgr.y.service' 2024-09-12T02:18:36.648 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T02:18:36.648 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-09-12T02:18:36.648 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-09-12T02:18:36.648 DEBUG:teuthology.orchestra.run.smithi184:> sudo systemctl stop ceph-879f3b56-70a9-11ef-bcea-c7b262605968@mgr.z 2024-09-12T02:18:36.877 INFO:journalctl@ceph.mgr.z.smithi184.stdout:Sep 12 02:18:36 smithi184 systemd[1]: Stopping Ceph mgr.z for 879f3b56-70a9-11ef-bcea-c7b262605968... 2024-09-12T02:18:37.293 INFO:journalctl@ceph.mgr.z.smithi184.stdout:Sep 12 02:18:37 smithi184 podman[72534]: 2024-09-12 02:18:37.005238728 +0000 UTC m=+0.159701977 container died 13b9ff9064fad2b8fa478de9dbcaf876a88fe4f6848ca04ad8e190d1e71cbf2c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-z, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3) 2024-09-12T02:18:37.577 INFO:journalctl@ceph.mgr.z.smithi184.stdout:Sep 12 02:18:37 smithi184 podman[72534]: 2024-09-12 02:18:37.376171871 +0000 UTC m=+0.530635117 container cleanup 13b9ff9064fad2b8fa478de9dbcaf876a88fe4f6848ca04ad8e190d1e71cbf2c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-z, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T02:18:37.577 INFO:journalctl@ceph.mgr.z.smithi184.stdout:Sep 12 02:18:37 smithi184 bash[72534]: ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mgr-z 2024-09-12T02:18:37.793 DEBUG:teuthology.orchestra.run.smithi184:> sudo pkill -f 'journalctl -f -n 0 -u ceph-879f3b56-70a9-11ef-bcea-c7b262605968@mgr.z.service' 2024-09-12T02:18:37.829 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T02:18:37.829 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-09-12T02:18:37.829 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-09-12T02:18:37.829 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ceph-879f3b56-70a9-11ef-bcea-c7b262605968@osd.0 2024-09-12T02:18:38.148 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 12 02:18:37 smithi016 systemd[1]: Stopping Ceph osd.0 for 879f3b56-70a9-11ef-bcea-c7b262605968... 2024-09-12T02:18:38.433 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 12 02:18:38 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-0[44855]: 2024-09-12T02:18:38.148+0000 7f3e744fe640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T02:18:38.434 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 12 02:18:38 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-0[44855]: 2024-09-12T02:18:38.148+0000 7f3e744fe640 -1 osd.0 85 *** Got signal Terminated *** 2024-09-12T02:18:38.434 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 12 02:18:38 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-0[44855]: 2024-09-12T02:18:38.148+0000 7f3e744fe640 -1 osd.0 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T02:18:43.800 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 12 02:18:43 smithi016 podman[130923]: 2024-09-12 02:18:43.47807394 +0000 UTC m=+5.446266505 container died 67143cd1611c9d2874d0500e4c1ec435b6823e4cc7ee7b850c63415fa23797eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-0, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-09-12T02:18:44.184 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 12 02:18:43 smithi016 podman[130923]: 2024-09-12 02:18:43.932338361 +0000 UTC m=+5.900530964 container cleanup 67143cd1611c9d2874d0500e4c1ec435b6823e4cc7ee7b850c63415fa23797eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-0, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=) 2024-09-12T02:18:44.184 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 12 02:18:43 smithi016 bash[130923]: ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-0 2024-09-12T02:18:44.609 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 12 02:18:44 smithi016 podman[130939]: 2024-09-12 02:18:44.242287589 +0000 UTC m=+0.762489953 container remove 67143cd1611c9d2874d0500e4c1ec435b6823e4cc7ee7b850c63415fa23797eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-09-12T02:18:44.934 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 12 02:18:44 smithi016 podman[131025]: 2024-09-12 02:18:44.512417054 +0000 UTC m=+0.016093458 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T02:18:44.934 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 12 02:18:44 smithi016 podman[131025]: 2024-09-12 02:18:44.636806423 +0000 UTC m=+0.140482817 container create fdabcc8dad390ae98b63a8fd1c7d2e110c1f528badfb4f02bdd48335c05013e3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-0-deactivate, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-09-12T02:18:44.934 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 12 02:18:44 smithi016 podman[131025]: 2024-09-12 02:18:44.847663425 +0000 UTC m=+0.351339831 container init fdabcc8dad390ae98b63a8fd1c7d2e110c1f528badfb4f02bdd48335c05013e3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-0-deactivate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731) 2024-09-12T02:18:44.934 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 12 02:18:44 smithi016 podman[131025]: 2024-09-12 02:18:44.851243051 +0000 UTC m=+0.354919446 container start fdabcc8dad390ae98b63a8fd1c7d2e110c1f528badfb4f02bdd48335c05013e3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-0-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731) 2024-09-12T02:18:45.433 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 12 02:18:44 smithi016 podman[131025]: 2024-09-12 02:18:44.955804003 +0000 UTC m=+0.459480399 container attach fdabcc8dad390ae98b63a8fd1c7d2e110c1f528badfb4f02bdd48335c05013e3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-0-deactivate, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731) 2024-09-12T02:18:45.434 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 12 02:18:45 smithi016 podman[131025]: 2024-09-12 02:18:45.051078785 +0000 UTC m=+0.554755182 container died fdabcc8dad390ae98b63a8fd1c7d2e110c1f528badfb4f02bdd48335c05013e3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-0-deactivate, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-09-12T02:18:45.637 DEBUG:teuthology.orchestra.run.smithi016:> sudo pkill -f 'journalctl -f -n 0 -u ceph-879f3b56-70a9-11ef-bcea-c7b262605968@osd.0.service' 2024-09-12T02:18:45.679 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T02:18:45.679 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-09-12T02:18:45.679 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-09-12T02:18:45.680 DEBUG:teuthology.orchestra.run.smithi033:> sudo systemctl stop ceph-879f3b56-70a9-11ef-bcea-c7b262605968@osd.1 2024-09-12T02:18:45.995 INFO:journalctl@ceph.osd.1.smithi033.stdout:Sep 12 02:18:45 smithi033 systemd[1]: Stopping Ceph osd.1 for 879f3b56-70a9-11ef-bcea-c7b262605968... 2024-09-12T02:18:46.419 INFO:journalctl@ceph.osd.1.smithi033.stdout:Sep 12 02:18:45 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-1[39829]: 2024-09-12T02:18:45.994+0000 7fe87ccb3640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T02:18:46.419 INFO:journalctl@ceph.osd.1.smithi033.stdout:Sep 12 02:18:45 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-1[39829]: 2024-09-12T02:18:45.994+0000 7fe87ccb3640 -1 osd.1 85 *** Got signal Terminated *** 2024-09-12T02:18:46.419 INFO:journalctl@ceph.osd.1.smithi033.stdout:Sep 12 02:18:45 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-1[39829]: 2024-09-12T02:18:45.994+0000 7fe87ccb3640 -1 osd.1 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T02:18:51.704 INFO:journalctl@ceph.osd.1.smithi033.stdout:Sep 12 02:18:51 smithi033 podman[70304]: 2024-09-12 02:18:51.441803097 +0000 UTC m=+5.567681481 container died 3b7a464851d7be3280b9cfa991204e041e17a87ac0ae8b62d0a17d3741a59328 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-1, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, ceph=True, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3) 2024-09-12T02:18:52.053 INFO:journalctl@ceph.osd.1.smithi033.stdout:Sep 12 02:18:51 smithi033 podman[70304]: 2024-09-12 02:18:51.79450724 +0000 UTC m=+5.920385625 container cleanup 3b7a464851d7be3280b9cfa991204e041e17a87ac0ae8b62d0a17d3741a59328 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-1, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3) 2024-09-12T02:18:52.053 INFO:journalctl@ceph.osd.1.smithi033.stdout:Sep 12 02:18:51 smithi033 bash[70304]: ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-1 2024-09-12T02:18:52.404 INFO:journalctl@ceph.osd.1.smithi033.stdout:Sep 12 02:18:52 smithi033 podman[70319]: 2024-09-12 02:18:52.05396687 +0000 UTC m=+0.606175926 container remove 3b7a464851d7be3280b9cfa991204e041e17a87ac0ae8b62d0a17d3741a59328 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-1, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T02:18:52.669 INFO:journalctl@ceph.osd.1.smithi033.stdout:Sep 12 02:18:52 smithi033 podman[70412]: 2024-09-12 02:18:52.405093273 +0000 UTC m=+0.099466319 container create 0ef06c3081a268d2a8f362147bf51fed094e55a3540bdd469630e2e51f22a094 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-1-deactivate, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, ceph=True, CEPH_POINT_RELEASE=) 2024-09-12T02:18:52.669 INFO:journalctl@ceph.osd.1.smithi033.stdout:Sep 12 02:18:52 smithi033 podman[70412]: 2024-09-12 02:18:52.321952258 +0000 UTC m=+0.016325277 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T02:18:52.669 INFO:journalctl@ceph.osd.1.smithi033.stdout:Sep 12 02:18:52 smithi033 podman[70412]: 2024-09-12 02:18:52.641168549 +0000 UTC m=+0.335541554 container init 0ef06c3081a268d2a8f362147bf51fed094e55a3540bdd469630e2e51f22a094 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-1-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, GIT_CLEAN=True) 2024-09-12T02:18:52.669 INFO:journalctl@ceph.osd.1.smithi033.stdout:Sep 12 02:18:52 smithi033 podman[70412]: 2024-09-12 02:18:52.644436425 +0000 UTC m=+0.338809432 container start 0ef06c3081a268d2a8f362147bf51fed094e55a3540bdd469630e2e51f22a094 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-1-deactivate, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T02:18:53.127 INFO:journalctl@ceph.osd.1.smithi033.stdout:Sep 12 02:18:52 smithi033 podman[70412]: 2024-09-12 02:18:52.732845826 +0000 UTC m=+0.427218843 container attach 0ef06c3081a268d2a8f362147bf51fed094e55a3540bdd469630e2e51f22a094 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-1-deactivate, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, ceph=True) 2024-09-12T02:18:53.127 INFO:journalctl@ceph.osd.1.smithi033.stdout:Sep 12 02:18:52 smithi033 podman[70412]: 2024-09-12 02:18:52.842364558 +0000 UTC m=+0.536737575 container died 0ef06c3081a268d2a8f362147bf51fed094e55a3540bdd469630e2e51f22a094 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-1-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731) 2024-09-12T02:18:53.389 INFO:journalctl@ceph.osd.1.smithi033.stdout:Sep 12 02:18:53 smithi033 podman[70435]: 2024-09-12 02:18:53.36957538 +0000 UTC m=+0.520978785 container remove 0ef06c3081a268d2a8f362147bf51fed094e55a3540bdd469630e2e51f22a094 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-1-deactivate, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T02:18:53.395 DEBUG:teuthology.orchestra.run.smithi033:> sudo pkill -f 'journalctl -f -n 0 -u ceph-879f3b56-70a9-11ef-bcea-c7b262605968@osd.1.service' 2024-09-12T02:18:53.438 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T02:18:53.438 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-09-12T02:18:53.438 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-09-12T02:18:53.438 DEBUG:teuthology.orchestra.run.smithi184:> sudo systemctl stop ceph-879f3b56-70a9-11ef-bcea-c7b262605968@osd.2 2024-09-12T02:18:53.729 INFO:journalctl@ceph.osd.2.smithi184.stdout:Sep 12 02:18:53 smithi184 systemd[1]: Stopping Ceph osd.2 for 879f3b56-70a9-11ef-bcea-c7b262605968... 2024-09-12T02:18:54.127 INFO:journalctl@ceph.osd.2.smithi184.stdout:Sep 12 02:18:53 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-2[40596]: 2024-09-12T02:18:53.728+0000 7fbf8589f640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T02:18:54.128 INFO:journalctl@ceph.osd.2.smithi184.stdout:Sep 12 02:18:53 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-2[40596]: 2024-09-12T02:18:53.728+0000 7fbf8589f640 -1 osd.2 85 *** Got signal Terminated *** 2024-09-12T02:18:54.128 INFO:journalctl@ceph.osd.2.smithi184.stdout:Sep 12 02:18:53 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-2[40596]: 2024-09-12T02:18:53.728+0000 7fbf8589f640 -1 osd.2 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T02:18:59.430 INFO:journalctl@ceph.osd.2.smithi184.stdout:Sep 12 02:18:59 smithi184 podman[72659]: 2024-09-12 02:18:59.147791058 +0000 UTC m=+5.510676935 container died 2a1230457f208620ad51d31ba6ae18c2f19bc4826c54035b7ce73113de4f7810 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T02:18:59.771 INFO:journalctl@ceph.osd.2.smithi184.stdout:Sep 12 02:18:59 smithi184 podman[72659]: 2024-09-12 02:18:59.529180975 +0000 UTC m=+5.892066840 container cleanup 2a1230457f208620ad51d31ba6ae18c2f19bc4826c54035b7ce73113de4f7810 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-2, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3) 2024-09-12T02:18:59.771 INFO:journalctl@ceph.osd.2.smithi184.stdout:Sep 12 02:18:59 smithi184 bash[72659]: ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-2 2024-09-12T02:19:00.096 INFO:journalctl@ceph.osd.2.smithi184.stdout:Sep 12 02:18:59 smithi184 podman[72674]: 2024-09-12 02:18:59.772011933 +0000 UTC m=+0.616639369 container remove 2a1230457f208620ad51d31ba6ae18c2f19bc4826c54035b7ce73113de4f7810 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-09-12T02:19:00.377 INFO:journalctl@ceph.osd.2.smithi184.stdout:Sep 12 02:19:00 smithi184 podman[72761]: 2024-09-12 02:18:59.999214515 +0000 UTC m=+0.015848480 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T02:19:00.377 INFO:journalctl@ceph.osd.2.smithi184.stdout:Sep 12 02:19:00 smithi184 podman[72761]: 2024-09-12 02:19:00.114712811 +0000 UTC m=+0.131346763 container create a7db5e2b9bacd9a532cbb21e8f98e14a1c644ba9e77e0837f19fc75b30cce230 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-2-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-09-12T02:19:00.378 INFO:journalctl@ceph.osd.2.smithi184.stdout:Sep 12 02:19:00 smithi184 podman[72761]: 2024-09-12 02:19:00.342694223 +0000 UTC m=+0.359328177 container init a7db5e2b9bacd9a532cbb21e8f98e14a1c644ba9e77e0837f19fc75b30cce230 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-2-deactivate, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2) 2024-09-12T02:19:00.378 INFO:journalctl@ceph.osd.2.smithi184.stdout:Sep 12 02:19:00 smithi184 podman[72761]: 2024-09-12 02:19:00.346173527 +0000 UTC m=+0.362807477 container start a7db5e2b9bacd9a532cbb21e8f98e14a1c644ba9e77e0837f19fc75b30cce230 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-12T02:19:00.819 INFO:journalctl@ceph.osd.2.smithi184.stdout:Sep 12 02:19:00 smithi184 podman[72761]: 2024-09-12 02:19:00.425676014 +0000 UTC m=+0.442309965 container attach a7db5e2b9bacd9a532cbb21e8f98e14a1c644ba9e77e0837f19fc75b30cce230 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-2-deactivate, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T02:19:00.820 INFO:journalctl@ceph.osd.2.smithi184.stdout:Sep 12 02:19:00 smithi184 podman[72761]: 2024-09-12 02:19:00.531082935 +0000 UTC m=+0.547716891 container died a7db5e2b9bacd9a532cbb21e8f98e14a1c644ba9e77e0837f19fc75b30cce230 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-2-deactivate, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3) 2024-09-12T02:19:01.114 DEBUG:teuthology.orchestra.run.smithi184:> sudo pkill -f 'journalctl -f -n 0 -u ceph-879f3b56-70a9-11ef-bcea-c7b262605968@osd.2.service' 2024-09-12T02:19:01.153 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T02:19:01.153 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-09-12T02:19:01.153 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-09-12T02:19:01.153 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ceph-879f3b56-70a9-11ef-bcea-c7b262605968@osd.3 2024-09-12T02:19:01.433 INFO:journalctl@ceph.osd.3.smithi016.stdout:Sep 12 02:19:01 smithi016 systemd[1]: Stopping Ceph osd.3 for 879f3b56-70a9-11ef-bcea-c7b262605968... 2024-09-12T02:19:01.772 INFO:journalctl@ceph.osd.3.smithi016.stdout:Sep 12 02:19:01 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-3[51462]: 2024-09-12T02:19:01.478+0000 7fdc4d849640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T02:19:01.772 INFO:journalctl@ceph.osd.3.smithi016.stdout:Sep 12 02:19:01 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-3[51462]: 2024-09-12T02:19:01.478+0000 7fdc4d849640 -1 osd.3 85 *** Got signal Terminated *** 2024-09-12T02:19:01.772 INFO:journalctl@ceph.osd.3.smithi016.stdout:Sep 12 02:19:01 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-3[51462]: 2024-09-12T02:19:01.478+0000 7fdc4d849640 -1 osd.3 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T02:19:04.683 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:04 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:04.336+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:38.124303+0000 front 2024-09-12T02:18:38.124204+0000 (oldest deadline 2024-09-12T02:19:04.024431+0000) 2024-09-12T02:19:05.433 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:05 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:05.291+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:38.124303+0000 front 2024-09-12T02:18:38.124204+0000 (oldest deadline 2024-09-12T02:19:04.024431+0000) 2024-09-12T02:19:05.434 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 12 02:19:05 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-6[58414]: 2024-09-12T02:19:05.091+0000 7fa4a9867640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:41.416884+0000 front 2024-09-12T02:18:41.416940+0000 (oldest deadline 2024-09-12T02:19:04.317006+0000) 2024-09-12T02:19:06.085 INFO:journalctl@ceph.osd.3.smithi016.stdout:Sep 12 02:19:05 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-3[51462]: 2024-09-12T02:19:05.829+0000 7fdc49e4e640 -1 osd.3 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:41.540958+0000 front 2024-09-12T02:18:41.541098+0000 (oldest deadline 2024-09-12T02:19:05.641097+0000) 2024-09-12T02:19:06.380 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:05 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:05.878+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:40.491334+0000 front 2024-09-12T02:18:40.491223+0000 (oldest deadline 2024-09-12T02:19:05.191278+0000) 2024-09-12T02:19:06.419 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:06 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:06.011+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:42.394291+0000 front 2024-09-12T02:18:42.394247+0000 (oldest deadline 2024-09-12T02:19:05.894091+0000) 2024-09-12T02:19:06.433 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 12 02:19:06 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-6[58414]: 2024-09-12T02:19:06.084+0000 7fa4a9867640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:41.416884+0000 front 2024-09-12T02:18:41.416940+0000 (oldest deadline 2024-09-12T02:19:04.317006+0000) 2024-09-12T02:19:06.434 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:06 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:06.282+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:38.124303+0000 front 2024-09-12T02:18:38.124204+0000 (oldest deadline 2024-09-12T02:19:04.024431+0000) 2024-09-12T02:19:07.377 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:06 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:06.892+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:40.491334+0000 front 2024-09-12T02:18:40.491223+0000 (oldest deadline 2024-09-12T02:19:05.191278+0000) 2024-09-12T02:19:07.377 INFO:journalctl@ceph.osd.5.smithi184.stdout:Sep 12 02:19:07 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-5[47266]: 2024-09-12T02:19:07.009+0000 7f6aeb64d640 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:42.884804+0000 front 2024-09-12T02:18:42.884900+0000 (oldest deadline 2024-09-12T02:19:06.984651+0000) 2024-09-12T02:19:07.387 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:07 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:07.243+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:38.124303+0000 front 2024-09-12T02:18:38.124204+0000 (oldest deadline 2024-09-12T02:19:04.024431+0000) 2024-09-12T02:19:07.387 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 12 02:19:07 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-6[58414]: 2024-09-12T02:19:07.082+0000 7fa4a9867640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:41.416884+0000 front 2024-09-12T02:18:41.416940+0000 (oldest deadline 2024-09-12T02:19:04.317006+0000) 2024-09-12T02:19:07.387 INFO:journalctl@ceph.osd.3.smithi016.stdout:Sep 12 02:19:06 smithi016 podman[131130]: 2024-09-12 02:19:06.994569448 +0000 UTC m=+5.614136840 container died 00cf949c9edab1d7becfa05eb0ca0a3a92e0e54df9fe02cbe63cc5cde81f6b68 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-3, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-09-12T02:19:07.387 INFO:journalctl@ceph.osd.3.smithi016.stdout:Sep 12 02:19:07 smithi016 podman[131130]: 2024-09-12 02:19:07.387713637 +0000 UTC m=+6.007281025 container cleanup 00cf949c9edab1d7becfa05eb0ca0a3a92e0e54df9fe02cbe63cc5cde81f6b68 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T02:19:07.419 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:07 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:07.021+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:42.394291+0000 front 2024-09-12T02:18:42.394247+0000 (oldest deadline 2024-09-12T02:19:05.894091+0000) 2024-09-12T02:19:07.638 INFO:journalctl@ceph.osd.3.smithi016.stdout:Sep 12 02:19:07 smithi016 bash[131130]: ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-3 2024-09-12T02:19:07.933 INFO:journalctl@ceph.osd.3.smithi016.stdout:Sep 12 02:19:07 smithi016 podman[131158]: 2024-09-12 02:19:07.639308907 +0000 UTC m=+0.634508572 container remove 00cf949c9edab1d7becfa05eb0ca0a3a92e0e54df9fe02cbe63cc5cde81f6b68 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-3, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3) 2024-09-12T02:19:08.127 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:07 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:07.846+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:40.491334+0000 front 2024-09-12T02:18:40.491223+0000 (oldest deadline 2024-09-12T02:19:05.191278+0000) 2024-09-12T02:19:08.127 INFO:journalctl@ceph.osd.5.smithi184.stdout:Sep 12 02:19:08 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-5[47266]: 2024-09-12T02:19:08.011+0000 7f6aeb64d640 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:42.884804+0000 front 2024-09-12T02:18:42.884900+0000 (oldest deadline 2024-09-12T02:19:06.984651+0000) 2024-09-12T02:19:08.235 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 12 02:19:08 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-6[58414]: 2024-09-12T02:19:08.068+0000 7fa4a9867640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:41.416884+0000 front 2024-09-12T02:18:41.416940+0000 (oldest deadline 2024-09-12T02:19:04.317006+0000) 2024-09-12T02:19:08.235 INFO:journalctl@ceph.osd.3.smithi016.stdout:Sep 12 02:19:07 smithi016 podman[131249]: 2024-09-12 02:19:07.86641915 +0000 UTC m=+0.016516490 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T02:19:08.235 INFO:journalctl@ceph.osd.3.smithi016.stdout:Sep 12 02:19:07 smithi016 podman[131249]: 2024-09-12 02:19:07.973292561 +0000 UTC m=+0.123389894 container create 698c626960f858ce2cb750d1da722b0f02a420993fe9e547cf1e9a0e7da85137 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-3-deactivate, RELEASE=squid-b53b6d3, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2) 2024-09-12T02:19:08.235 INFO:journalctl@ceph.osd.3.smithi016.stdout:Sep 12 02:19:08 smithi016 podman[131249]: 2024-09-12 02:19:08.201209116 +0000 UTC m=+0.351306445 container init 698c626960f858ce2cb750d1da722b0f02a420993fe9e547cf1e9a0e7da85137 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-3-deactivate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T02:19:08.235 INFO:journalctl@ceph.osd.3.smithi016.stdout:Sep 12 02:19:08 smithi016 podman[131249]: 2024-09-12 02:19:08.204578098 +0000 UTC m=+0.354675439 container start 698c626960f858ce2cb750d1da722b0f02a420993fe9e547cf1e9a0e7da85137 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-3-deactivate, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0) 2024-09-12T02:19:08.419 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:08 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:08.055+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:42.394291+0000 front 2024-09-12T02:18:42.394247+0000 (oldest deadline 2024-09-12T02:19:05.894091+0000) 2024-09-12T02:19:08.684 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:08 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:08.234+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:38.124303+0000 front 2024-09-12T02:18:38.124204+0000 (oldest deadline 2024-09-12T02:19:04.024431+0000) 2024-09-12T02:19:08.684 INFO:journalctl@ceph.osd.3.smithi016.stdout:Sep 12 02:19:08 smithi016 podman[131249]: 2024-09-12 02:19:08.292918631 +0000 UTC m=+0.443015999 container attach 698c626960f858ce2cb750d1da722b0f02a420993fe9e547cf1e9a0e7da85137 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-3-deactivate, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, ceph=True, org.label-schema.build-date=20240731) 2024-09-12T02:19:08.684 INFO:journalctl@ceph.osd.3.smithi016.stdout:Sep 12 02:19:08 smithi016 podman[131249]: 2024-09-12 02:19:08.405167826 +0000 UTC m=+0.555265158 container died 698c626960f858ce2cb750d1da722b0f02a420993fe9e547cf1e9a0e7da85137 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-3-deactivate, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-12T02:19:09.017 DEBUG:teuthology.orchestra.run.smithi016:> sudo pkill -f 'journalctl -f -n 0 -u ceph-879f3b56-70a9-11ef-bcea-c7b262605968@osd.3.service' 2024-09-12T02:19:09.061 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T02:19:09.062 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-09-12T02:19:09.062 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-09-12T02:19:09.062 DEBUG:teuthology.orchestra.run.smithi033:> sudo systemctl stop ceph-879f3b56-70a9-11ef-bcea-c7b262605968@osd.4 2024-09-12T02:19:09.357 INFO:journalctl@ceph.osd.4.smithi033.stdout:Sep 12 02:19:09 smithi033 systemd[1]: Stopping Ceph osd.4 for 879f3b56-70a9-11ef-bcea-c7b262605968... 2024-09-12T02:19:09.357 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:09 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:09.064+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:42.394291+0000 front 2024-09-12T02:18:42.394247+0000 (oldest deadline 2024-09-12T02:19:05.894091+0000) 2024-09-12T02:19:09.377 INFO:journalctl@ceph.osd.5.smithi184.stdout:Sep 12 02:19:09 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-5[47266]: 2024-09-12T02:19:09.023+0000 7f6aeb64d640 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:42.884804+0000 front 2024-09-12T02:18:42.884900+0000 (oldest deadline 2024-09-12T02:19:06.984651+0000) 2024-09-12T02:19:09.377 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:08 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:08.880+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:40.491334+0000 front 2024-09-12T02:18:40.491223+0000 (oldest deadline 2024-09-12T02:19:05.191278+0000) 2024-09-12T02:19:09.434 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 12 02:19:09 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-6[58414]: 2024-09-12T02:19:09.099+0000 7fa4a9867640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:41.416884+0000 front 2024-09-12T02:18:41.416940+0000 (oldest deadline 2024-09-12T02:19:04.317006+0000) 2024-09-12T02:19:09.434 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:09 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:09.200+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:38.124303+0000 front 2024-09-12T02:18:38.124204+0000 (oldest deadline 2024-09-12T02:19:04.024431+0000) 2024-09-12T02:19:09.669 INFO:journalctl@ceph.osd.4.smithi033.stdout:Sep 12 02:19:09 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-4[46311]: 2024-09-12T02:19:09.355+0000 7f803ed51640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T02:19:09.669 INFO:journalctl@ceph.osd.4.smithi033.stdout:Sep 12 02:19:09 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-4[46311]: 2024-09-12T02:19:09.355+0000 7f803ed51640 -1 osd.4 85 *** Got signal Terminated *** 2024-09-12T02:19:09.669 INFO:journalctl@ceph.osd.4.smithi033.stdout:Sep 12 02:19:09 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-4[46311]: 2024-09-12T02:19:09.355+0000 7f803ed51640 -1 osd.4 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T02:19:10.377 INFO:journalctl@ceph.osd.5.smithi184.stdout:Sep 12 02:19:10 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-5[47266]: 2024-09-12T02:19:10.034+0000 7f6aeb64d640 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:42.884804+0000 front 2024-09-12T02:18:42.884900+0000 (oldest deadline 2024-09-12T02:19:06.984651+0000) 2024-09-12T02:19:10.377 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:09 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:09.882+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:40.491334+0000 front 2024-09-12T02:18:40.491223+0000 (oldest deadline 2024-09-12T02:19:05.191278+0000) 2024-09-12T02:19:10.419 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:10 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:10.056+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:42.394291+0000 front 2024-09-12T02:18:42.394247+0000 (oldest deadline 2024-09-12T02:19:05.894091+0000) 2024-09-12T02:19:10.433 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 12 02:19:10 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-6[58414]: 2024-09-12T02:19:10.138+0000 7fa4a9867640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:41.416884+0000 front 2024-09-12T02:18:41.416940+0000 (oldest deadline 2024-09-12T02:19:04.317006+0000) 2024-09-12T02:19:10.433 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:10 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:10.219+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:38.124303+0000 front 2024-09-12T02:18:38.124204+0000 (oldest deadline 2024-09-12T02:19:04.024431+0000) 2024-09-12T02:19:11.377 INFO:journalctl@ceph.osd.5.smithi184.stdout:Sep 12 02:19:11 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-5[47266]: 2024-09-12T02:19:11.044+0000 7f6aeb64d640 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:42.884804+0000 front 2024-09-12T02:18:42.884900+0000 (oldest deadline 2024-09-12T02:19:06.984651+0000) 2024-09-12T02:19:11.378 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:10 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:10.914+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:40.491334+0000 front 2024-09-12T02:18:40.491223+0000 (oldest deadline 2024-09-12T02:19:05.191278+0000) 2024-09-12T02:19:11.419 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:11 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:11.010+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:42.394291+0000 front 2024-09-12T02:18:42.394247+0000 (oldest deadline 2024-09-12T02:19:05.894091+0000) 2024-09-12T02:19:11.433 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 12 02:19:11 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-6[58414]: 2024-09-12T02:19:11.133+0000 7fa4a9867640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:41.416884+0000 front 2024-09-12T02:18:41.416940+0000 (oldest deadline 2024-09-12T02:19:04.317006+0000) 2024-09-12T02:19:11.434 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:11 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:11.201+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:38.124303+0000 front 2024-09-12T02:18:38.124204+0000 (oldest deadline 2024-09-12T02:19:04.024431+0000) 2024-09-12T02:19:12.127 INFO:journalctl@ceph.osd.5.smithi184.stdout:Sep 12 02:19:12 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-5[47266]: 2024-09-12T02:19:12.045+0000 7f6aeb64d640 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:42.884804+0000 front 2024-09-12T02:18:42.884900+0000 (oldest deadline 2024-09-12T02:19:06.984651+0000) 2024-09-12T02:19:12.128 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:11 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:11.929+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:40.491334+0000 front 2024-09-12T02:18:40.491223+0000 (oldest deadline 2024-09-12T02:19:05.191278+0000) 2024-09-12T02:19:12.419 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:12 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:12.047+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:42.394291+0000 front 2024-09-12T02:18:42.394247+0000 (oldest deadline 2024-09-12T02:19:05.894091+0000) 2024-09-12T02:19:12.433 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 12 02:19:12 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-6[58414]: 2024-09-12T02:19:12.153+0000 7fa4a9867640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:41.416884+0000 front 2024-09-12T02:18:41.416940+0000 (oldest deadline 2024-09-12T02:19:04.317006+0000) 2024-09-12T02:19:12.433 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:12 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:12.233+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:38.124303+0000 front 2024-09-12T02:18:38.124204+0000 (oldest deadline 2024-09-12T02:19:04.024431+0000) 2024-09-12T02:19:13.377 INFO:journalctl@ceph.osd.5.smithi184.stdout:Sep 12 02:19:13 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-5[47266]: 2024-09-12T02:19:13.020+0000 7f6aeb64d640 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:42.884804+0000 front 2024-09-12T02:18:42.884900+0000 (oldest deadline 2024-09-12T02:19:06.984651+0000) 2024-09-12T02:19:13.378 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:12 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:12.967+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:40.491334+0000 front 2024-09-12T02:18:40.491223+0000 (oldest deadline 2024-09-12T02:19:05.191278+0000) 2024-09-12T02:19:13.419 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:13 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:13.052+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:42.394291+0000 front 2024-09-12T02:18:42.394247+0000 (oldest deadline 2024-09-12T02:19:05.894091+0000) 2024-09-12T02:19:13.419 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:13 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:13.052+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:47.595046+0000 front 2024-09-12T02:18:47.595165+0000 (oldest deadline 2024-09-12T02:19:12.295142+0000) 2024-09-12T02:19:13.683 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 12 02:19:13 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-6[58414]: 2024-09-12T02:19:13.183+0000 7fa4a9867640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:41.416884+0000 front 2024-09-12T02:18:41.416940+0000 (oldest deadline 2024-09-12T02:19:04.317006+0000) 2024-09-12T02:19:13.684 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:13 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:13.253+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:38.124303+0000 front 2024-09-12T02:18:38.124204+0000 (oldest deadline 2024-09-12T02:19:04.024431+0000) 2024-09-12T02:19:14.169 INFO:journalctl@ceph.osd.7.smithi033.stdout:Sep 12 02:19:13 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-7[53039]: 2024-09-12T02:19:13.899+0000 7f5a34c36640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:50.190710+0000 front 2024-09-12T02:18:50.190738+0000 (oldest deadline 2024-09-12T02:19:13.090875+0000) 2024-09-12T02:19:14.169 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:14 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:14.053+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:42.394291+0000 front 2024-09-12T02:18:42.394247+0000 (oldest deadline 2024-09-12T02:19:05.894091+0000) 2024-09-12T02:19:14.169 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:14 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:14.053+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:47.595046+0000 front 2024-09-12T02:18:47.595165+0000 (oldest deadline 2024-09-12T02:19:12.295142+0000) 2024-09-12T02:19:14.377 INFO:journalctl@ceph.osd.5.smithi184.stdout:Sep 12 02:19:13 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-5[47266]: 2024-09-12T02:19:13.978+0000 7f6aeb64d640 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:42.884804+0000 front 2024-09-12T02:18:42.884900+0000 (oldest deadline 2024-09-12T02:19:06.984651+0000) 2024-09-12T02:19:14.377 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:13 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:13.920+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:40.491334+0000 front 2024-09-12T02:18:40.491223+0000 (oldest deadline 2024-09-12T02:19:05.191278+0000) 2024-09-12T02:19:14.433 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 12 02:19:14 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-6[58414]: 2024-09-12T02:19:14.148+0000 7fa4a9867640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:41.416884+0000 front 2024-09-12T02:18:41.416940+0000 (oldest deadline 2024-09-12T02:19:04.317006+0000) 2024-09-12T02:19:14.434 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:14 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:14.276+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:38.124303+0000 front 2024-09-12T02:18:38.124204+0000 (oldest deadline 2024-09-12T02:19:04.024431+0000) 2024-09-12T02:19:15.047 INFO:journalctl@ceph.osd.4.smithi033.stdout:Sep 12 02:19:14 smithi033 podman[70530]: 2024-09-12 02:19:14.804268134 +0000 UTC m=+5.540242313 container died fb5bdfe221f9d60867171ad89dc137642a9b612279ae17eb9d949ae247802fe5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-4, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.37.2, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T02:19:15.047 INFO:journalctl@ceph.osd.7.smithi033.stdout:Sep 12 02:19:14 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-7[53039]: 2024-09-12T02:19:14.888+0000 7f5a34c36640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:50.190710+0000 front 2024-09-12T02:18:50.190738+0000 (oldest deadline 2024-09-12T02:19:13.090875+0000) 2024-09-12T02:19:15.297 INFO:journalctl@ceph.osd.4.smithi033.stdout:Sep 12 02:19:15 smithi033 podman[70530]: 2024-09-12 02:19:15.146580127 +0000 UTC m=+5.882554294 container cleanup fb5bdfe221f9d60867171ad89dc137642a9b612279ae17eb9d949ae247802fe5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-4, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3) 2024-09-12T02:19:15.298 INFO:journalctl@ceph.osd.4.smithi033.stdout:Sep 12 02:19:15 smithi033 bash[70530]: ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-4 2024-09-12T02:19:15.298 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:15 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:15.053+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:42.394291+0000 front 2024-09-12T02:18:42.394247+0000 (oldest deadline 2024-09-12T02:19:05.894091+0000) 2024-09-12T02:19:15.298 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:15 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:15.053+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:47.595046+0000 front 2024-09-12T02:18:47.595165+0000 (oldest deadline 2024-09-12T02:19:12.295142+0000) 2024-09-12T02:19:15.377 INFO:journalctl@ceph.osd.5.smithi184.stdout:Sep 12 02:19:14 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-5[47266]: 2024-09-12T02:19:14.982+0000 7f6aeb64d640 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:42.884804+0000 front 2024-09-12T02:18:42.884900+0000 (oldest deadline 2024-09-12T02:19:06.984651+0000) 2024-09-12T02:19:15.378 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:14 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:14.902+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:40.491334+0000 front 2024-09-12T02:18:40.491223+0000 (oldest deadline 2024-09-12T02:19:05.191278+0000) 2024-09-12T02:19:15.433 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 12 02:19:15 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-6[58414]: 2024-09-12T02:19:15.125+0000 7fa4a9867640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:41.416884+0000 front 2024-09-12T02:18:41.416940+0000 (oldest deadline 2024-09-12T02:19:04.317006+0000) 2024-09-12T02:19:15.434 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:15 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:15.291+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:38.124303+0000 front 2024-09-12T02:18:38.124204+0000 (oldest deadline 2024-09-12T02:19:04.024431+0000) 2024-09-12T02:19:15.434 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:15 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:15.291+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:49.925357+0000 front 2024-09-12T02:18:49.925414+0000 (oldest deadline 2024-09-12T02:19:14.625383+0000) 2024-09-12T02:19:15.669 INFO:journalctl@ceph.osd.4.smithi033.stdout:Sep 12 02:19:15 smithi033 podman[70548]: 2024-09-12 02:19:15.356539391 +0000 UTC m=+0.551123160 container remove fb5bdfe221f9d60867171ad89dc137642a9b612279ae17eb9d949ae247802fe5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-4, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD) 2024-09-12T02:19:16.025 INFO:journalctl@ceph.osd.7.smithi033.stdout:Sep 12 02:19:15 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-7[53039]: 2024-09-12T02:19:15.877+0000 7f5a34c36640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:50.190710+0000 front 2024-09-12T02:18:50.190738+0000 (oldest deadline 2024-09-12T02:19:13.090875+0000) 2024-09-12T02:19:16.026 INFO:journalctl@ceph.osd.4.smithi033.stdout:Sep 12 02:19:15 smithi033 podman[70635]: 2024-09-12 02:19:15.757259169 +0000 UTC m=+0.099527777 container create b34427ea61f1a55397976acb3a280ae919425c142e15ad34c351b933655a570c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-4-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-09-12T02:19:16.026 INFO:journalctl@ceph.osd.4.smithi033.stdout:Sep 12 02:19:15 smithi033 podman[70635]: 2024-09-12 02:19:15.673653389 +0000 UTC m=+0.015921939 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T02:19:16.026 INFO:journalctl@ceph.osd.4.smithi033.stdout:Sep 12 02:19:15 smithi033 podman[70635]: 2024-09-12 02:19:15.951731614 +0000 UTC m=+0.294000168 container init b34427ea61f1a55397976acb3a280ae919425c142e15ad34c351b933655a570c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-4-deactivate, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-09-12T02:19:16.026 INFO:journalctl@ceph.osd.4.smithi033.stdout:Sep 12 02:19:15 smithi033 podman[70635]: 2024-09-12 02:19:15.955124534 +0000 UTC m=+0.297393079 container start b34427ea61f1a55397976acb3a280ae919425c142e15ad34c351b933655a570c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-4-deactivate, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.build-date=20240731) 2024-09-12T02:19:16.026 INFO:journalctl@ceph.osd.4.smithi033.stdout:Sep 12 02:19:16 smithi033 podman[70635]: 2024-09-12 02:19:16.026068589 +0000 UTC m=+0.368337132 container attach b34427ea61f1a55397976acb3a280ae919425c142e15ad34c351b933655a570c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-4-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, ceph=True) 2024-09-12T02:19:16.127 INFO:journalctl@ceph.osd.5.smithi184.stdout:Sep 12 02:19:15 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-5[47266]: 2024-09-12T02:19:15.932+0000 7f6aeb64d640 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:42.884804+0000 front 2024-09-12T02:18:42.884900+0000 (oldest deadline 2024-09-12T02:19:06.984651+0000) 2024-09-12T02:19:16.127 INFO:journalctl@ceph.osd.5.smithi184.stdout:Sep 12 02:19:15 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-5[47266]: 2024-09-12T02:19:15.932+0000 7f6aeb64d640 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:49.885734+0000 front 2024-09-12T02:18:49.885703+0000 (oldest deadline 2024-09-12T02:19:15.185583+0000) 2024-09-12T02:19:16.128 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:15 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:15.872+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:40.491334+0000 front 2024-09-12T02:18:40.491223+0000 (oldest deadline 2024-09-12T02:19:05.191278+0000) 2024-09-12T02:19:16.128 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:15 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:15.872+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:49.192960+0000 front 2024-09-12T02:18:49.193116+0000 (oldest deadline 2024-09-12T02:19:15.092803+0000) 2024-09-12T02:19:16.419 INFO:journalctl@ceph.osd.4.smithi033.stdout:Sep 12 02:19:16 smithi033 podman[70635]: 2024-09-12 02:19:16.143145961 +0000 UTC m=+0.485414508 container died b34427ea61f1a55397976acb3a280ae919425c142e15ad34c351b933655a570c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-4-deactivate, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True) 2024-09-12T02:19:16.419 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:16 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:16.070+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:42.394291+0000 front 2024-09-12T02:18:42.394247+0000 (oldest deadline 2024-09-12T02:19:05.894091+0000) 2024-09-12T02:19:16.419 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:16 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:16.070+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:47.595046+0000 front 2024-09-12T02:18:47.595165+0000 (oldest deadline 2024-09-12T02:19:12.295142+0000) 2024-09-12T02:19:16.433 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 12 02:19:16 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-6[58414]: 2024-09-12T02:19:16.144+0000 7fa4a9867640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:41.416884+0000 front 2024-09-12T02:18:41.416940+0000 (oldest deadline 2024-09-12T02:19:04.317006+0000) 2024-09-12T02:19:16.433 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:16 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:16.325+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:38.124303+0000 front 2024-09-12T02:18:38.124204+0000 (oldest deadline 2024-09-12T02:19:04.024431+0000) 2024-09-12T02:19:16.433 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:16 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:16.325+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:49.925357+0000 front 2024-09-12T02:18:49.925414+0000 (oldest deadline 2024-09-12T02:19:14.625383+0000) 2024-09-12T02:19:16.682 INFO:journalctl@ceph.osd.4.smithi033.stdout:Sep 12 02:19:16 smithi033 podman[70657]: 2024-09-12 02:19:16.663001618 +0000 UTC m=+0.513595769 container remove b34427ea61f1a55397976acb3a280ae919425c142e15ad34c351b933655a570c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-4-deactivate, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-09-12T02:19:16.688 DEBUG:teuthology.orchestra.run.smithi033:> sudo pkill -f 'journalctl -f -n 0 -u ceph-879f3b56-70a9-11ef-bcea-c7b262605968@osd.4.service' 2024-09-12T02:19:16.729 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T02:19:16.729 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-09-12T02:19:16.729 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-09-12T02:19:16.729 DEBUG:teuthology.orchestra.run.smithi184:> sudo systemctl stop ceph-879f3b56-70a9-11ef-bcea-c7b262605968@osd.5 2024-09-12T02:19:16.877 INFO:journalctl@ceph.osd.5.smithi184.stdout:Sep 12 02:19:16 smithi184 systemd[1]: Stopping Ceph osd.5 for 879f3b56-70a9-11ef-bcea-c7b262605968... 2024-09-12T02:19:16.877 INFO:journalctl@ceph.osd.8.smithi184.stdout:Sep 12 02:19:16 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-8[54277]: 2024-09-12T02:19:16.590+0000 7fe987aba640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:49.946694+0000 front 2024-09-12T02:18:49.946586+0000 (oldest deadline 2024-09-12T02:19:15.846545+0000) 2024-09-12T02:19:17.098 INFO:journalctl@ceph.osd.7.smithi033.stdout:Sep 12 02:19:16 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-7[53039]: 2024-09-12T02:19:16.839+0000 7f5a34c36640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:50.190710+0000 front 2024-09-12T02:18:50.190738+0000 (oldest deadline 2024-09-12T02:19:13.090875+0000) 2024-09-12T02:19:17.377 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:16 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:16.904+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:40.491334+0000 front 2024-09-12T02:18:40.491223+0000 (oldest deadline 2024-09-12T02:19:05.191278+0000) 2024-09-12T02:19:17.378 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:16 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:16.904+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:49.192960+0000 front 2024-09-12T02:18:49.193116+0000 (oldest deadline 2024-09-12T02:19:15.092803+0000) 2024-09-12T02:19:17.378 INFO:journalctl@ceph.osd.5.smithi184.stdout:Sep 12 02:19:16 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-5[47266]: 2024-09-12T02:19:16.915+0000 7f6aeb64d640 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:42.884804+0000 front 2024-09-12T02:18:42.884900+0000 (oldest deadline 2024-09-12T02:19:06.984651+0000) 2024-09-12T02:19:17.378 INFO:journalctl@ceph.osd.5.smithi184.stdout:Sep 12 02:19:16 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-5[47266]: 2024-09-12T02:19:16.915+0000 7f6aeb64d640 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:49.885734+0000 front 2024-09-12T02:18:49.885703+0000 (oldest deadline 2024-09-12T02:19:15.185583+0000) 2024-09-12T02:19:17.378 INFO:journalctl@ceph.osd.5.smithi184.stdout:Sep 12 02:19:17 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-5[47266]: 2024-09-12T02:19:17.072+0000 7f6af3050640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T02:19:17.378 INFO:journalctl@ceph.osd.5.smithi184.stdout:Sep 12 02:19:17 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-5[47266]: 2024-09-12T02:19:17.072+0000 7f6af3050640 -1 osd.5 85 *** Got signal Terminated *** 2024-09-12T02:19:17.378 INFO:journalctl@ceph.osd.5.smithi184.stdout:Sep 12 02:19:17 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-5[47266]: 2024-09-12T02:19:17.072+0000 7f6af3050640 -1 osd.5 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T02:19:17.419 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:17 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:17.098+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:42.394291+0000 front 2024-09-12T02:18:42.394247+0000 (oldest deadline 2024-09-12T02:19:05.894091+0000) 2024-09-12T02:19:17.419 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:17 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:17.098+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:47.595046+0000 front 2024-09-12T02:18:47.595165+0000 (oldest deadline 2024-09-12T02:19:12.295142+0000) 2024-09-12T02:19:17.683 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 12 02:19:17 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-6[58414]: 2024-09-12T02:19:17.192+0000 7fa4a9867640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:41.416884+0000 front 2024-09-12T02:18:41.416940+0000 (oldest deadline 2024-09-12T02:19:04.317006+0000) 2024-09-12T02:19:17.684 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 12 02:19:17 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-6[58414]: 2024-09-12T02:19:17.192+0000 7fa4a9867640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:50.619002+0000 front 2024-09-12T02:18:50.619295+0000 (oldest deadline 2024-09-12T02:19:16.519137+0000) 2024-09-12T02:19:17.684 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:17 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:17.296+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:38.124303+0000 front 2024-09-12T02:18:38.124204+0000 (oldest deadline 2024-09-12T02:19:04.024431+0000) 2024-09-12T02:19:17.684 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:17 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:17.296+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:49.925357+0000 front 2024-09-12T02:18:49.925414+0000 (oldest deadline 2024-09-12T02:19:14.625383+0000) 2024-09-12T02:19:17.877 INFO:journalctl@ceph.osd.8.smithi184.stdout:Sep 12 02:19:17 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-8[54277]: 2024-09-12T02:19:17.597+0000 7fe987aba640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:49.946694+0000 front 2024-09-12T02:18:49.946586+0000 (oldest deadline 2024-09-12T02:19:15.846545+0000) 2024-09-12T02:19:18.082 INFO:journalctl@ceph.osd.7.smithi033.stdout:Sep 12 02:19:17 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-7[53039]: 2024-09-12T02:19:17.822+0000 7f5a34c36640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:50.190710+0000 front 2024-09-12T02:18:50.190738+0000 (oldest deadline 2024-09-12T02:19:13.090875+0000) 2024-09-12T02:19:18.377 INFO:journalctl@ceph.osd.5.smithi184.stdout:Sep 12 02:19:17 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-5[47266]: 2024-09-12T02:19:17.922+0000 7f6aeb64d640 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:42.884804+0000 front 2024-09-12T02:18:42.884900+0000 (oldest deadline 2024-09-12T02:19:06.984651+0000) 2024-09-12T02:19:18.378 INFO:journalctl@ceph.osd.5.smithi184.stdout:Sep 12 02:19:17 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-5[47266]: 2024-09-12T02:19:17.922+0000 7f6aeb64d640 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:49.885734+0000 front 2024-09-12T02:18:49.885703+0000 (oldest deadline 2024-09-12T02:19:15.185583+0000) 2024-09-12T02:19:18.378 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:17 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:17.898+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:40.491334+0000 front 2024-09-12T02:18:40.491223+0000 (oldest deadline 2024-09-12T02:19:05.191278+0000) 2024-09-12T02:19:18.378 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:17 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:17.898+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:49.192960+0000 front 2024-09-12T02:18:49.193116+0000 (oldest deadline 2024-09-12T02:19:15.092803+0000) 2024-09-12T02:19:18.419 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:18 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:18.082+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:42.394291+0000 front 2024-09-12T02:18:42.394247+0000 (oldest deadline 2024-09-12T02:19:05.894091+0000) 2024-09-12T02:19:18.419 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:18 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:18.082+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:47.595046+0000 front 2024-09-12T02:18:47.595165+0000 (oldest deadline 2024-09-12T02:19:12.295142+0000) 2024-09-12T02:19:18.433 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 12 02:19:18 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-6[58414]: 2024-09-12T02:19:18.161+0000 7fa4a9867640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:41.416884+0000 front 2024-09-12T02:18:41.416940+0000 (oldest deadline 2024-09-12T02:19:04.317006+0000) 2024-09-12T02:19:18.433 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 12 02:19:18 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-6[58414]: 2024-09-12T02:19:18.161+0000 7fa4a9867640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:50.619002+0000 front 2024-09-12T02:18:50.619295+0000 (oldest deadline 2024-09-12T02:19:16.519137+0000) 2024-09-12T02:19:18.434 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:18 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:18.332+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:38.124303+0000 front 2024-09-12T02:18:38.124204+0000 (oldest deadline 2024-09-12T02:19:04.024431+0000) 2024-09-12T02:19:18.434 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:18 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:18.332+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:49.925357+0000 front 2024-09-12T02:18:49.925414+0000 (oldest deadline 2024-09-12T02:19:14.625383+0000) 2024-09-12T02:19:18.890 INFO:journalctl@ceph.osd.8.smithi184.stdout:Sep 12 02:19:18 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-8[54277]: 2024-09-12T02:19:18.639+0000 7fe987aba640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:49.946694+0000 front 2024-09-12T02:18:49.946586+0000 (oldest deadline 2024-09-12T02:19:15.846545+0000) 2024-09-12T02:19:19.169 INFO:journalctl@ceph.osd.7.smithi033.stdout:Sep 12 02:19:18 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-7[53039]: 2024-09-12T02:19:18.835+0000 7f5a34c36640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:50.190710+0000 front 2024-09-12T02:18:50.190738+0000 (oldest deadline 2024-09-12T02:19:13.090875+0000) 2024-09-12T02:19:19.169 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:19 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:19.062+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:42.394291+0000 front 2024-09-12T02:18:42.394247+0000 (oldest deadline 2024-09-12T02:19:05.894091+0000) 2024-09-12T02:19:19.169 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:19 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:19.062+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:47.595046+0000 front 2024-09-12T02:18:47.595165+0000 (oldest deadline 2024-09-12T02:19:12.295142+0000) 2024-09-12T02:19:19.377 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:18 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:18.890+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:40.491334+0000 front 2024-09-12T02:18:40.491223+0000 (oldest deadline 2024-09-12T02:19:05.191278+0000) 2024-09-12T02:19:19.377 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:18 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:18.890+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:49.192960+0000 front 2024-09-12T02:18:49.193116+0000 (oldest deadline 2024-09-12T02:19:15.092803+0000) 2024-09-12T02:19:19.378 INFO:journalctl@ceph.osd.5.smithi184.stdout:Sep 12 02:19:18 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-5[47266]: 2024-09-12T02:19:18.915+0000 7f6aeb64d640 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:42.884804+0000 front 2024-09-12T02:18:42.884900+0000 (oldest deadline 2024-09-12T02:19:06.984651+0000) 2024-09-12T02:19:19.378 INFO:journalctl@ceph.osd.5.smithi184.stdout:Sep 12 02:19:18 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-5[47266]: 2024-09-12T02:19:18.915+0000 7f6aeb64d640 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:49.885734+0000 front 2024-09-12T02:18:49.885703+0000 (oldest deadline 2024-09-12T02:19:15.185583+0000) 2024-09-12T02:19:19.433 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 12 02:19:19 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-6[58414]: 2024-09-12T02:19:19.205+0000 7fa4a9867640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:41.416884+0000 front 2024-09-12T02:18:41.416940+0000 (oldest deadline 2024-09-12T02:19:04.317006+0000) 2024-09-12T02:19:19.434 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 12 02:19:19 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-6[58414]: 2024-09-12T02:19:19.205+0000 7fa4a9867640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:50.619002+0000 front 2024-09-12T02:18:50.619295+0000 (oldest deadline 2024-09-12T02:19:16.519137+0000) 2024-09-12T02:19:19.434 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:19 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:19.301+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:38.124303+0000 front 2024-09-12T02:18:38.124204+0000 (oldest deadline 2024-09-12T02:19:04.024431+0000) 2024-09-12T02:19:19.434 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:19 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:19.301+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:49.925357+0000 front 2024-09-12T02:18:49.925414+0000 (oldest deadline 2024-09-12T02:19:14.625383+0000) 2024-09-12T02:19:19.877 INFO:journalctl@ceph.osd.8.smithi184.stdout:Sep 12 02:19:19 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-8[54277]: 2024-09-12T02:19:19.598+0000 7fe987aba640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:49.946694+0000 front 2024-09-12T02:18:49.946586+0000 (oldest deadline 2024-09-12T02:19:15.846545+0000) 2024-09-12T02:19:20.169 INFO:journalctl@ceph.osd.7.smithi033.stdout:Sep 12 02:19:19 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-7[53039]: 2024-09-12T02:19:19.851+0000 7f5a34c36640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:50.190710+0000 front 2024-09-12T02:18:50.190738+0000 (oldest deadline 2024-09-12T02:19:13.090875+0000) 2024-09-12T02:19:20.169 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:20 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:20.082+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:42.394291+0000 front 2024-09-12T02:18:42.394247+0000 (oldest deadline 2024-09-12T02:19:05.894091+0000) 2024-09-12T02:19:20.169 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:20 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:20.082+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:47.595046+0000 front 2024-09-12T02:18:47.595165+0000 (oldest deadline 2024-09-12T02:19:12.295142+0000) 2024-09-12T02:19:20.377 INFO:journalctl@ceph.osd.5.smithi184.stdout:Sep 12 02:19:19 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-5[47266]: 2024-09-12T02:19:19.948+0000 7f6aeb64d640 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:42.884804+0000 front 2024-09-12T02:18:42.884900+0000 (oldest deadline 2024-09-12T02:19:06.984651+0000) 2024-09-12T02:19:20.377 INFO:journalctl@ceph.osd.5.smithi184.stdout:Sep 12 02:19:19 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-5[47266]: 2024-09-12T02:19:19.948+0000 7f6aeb64d640 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:49.885734+0000 front 2024-09-12T02:18:49.885703+0000 (oldest deadline 2024-09-12T02:19:15.185583+0000) 2024-09-12T02:19:20.378 INFO:journalctl@ceph.osd.5.smithi184.stdout:Sep 12 02:19:19 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-5[47266]: 2024-09-12T02:19:19.948+0000 7f6aeb64d640 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:55.186033+0000 front 2024-09-12T02:18:55.186014+0000 (oldest deadline 2024-09-12T02:19:19.886140+0000) 2024-09-12T02:19:20.378 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:19 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:19.891+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:40.491334+0000 front 2024-09-12T02:18:40.491223+0000 (oldest deadline 2024-09-12T02:19:05.191278+0000) 2024-09-12T02:19:20.378 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:19 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:19.891+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:49.192960+0000 front 2024-09-12T02:18:49.193116+0000 (oldest deadline 2024-09-12T02:19:15.092803+0000) 2024-09-12T02:19:20.378 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:19 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:19.891+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:58.593805+0000 front 2024-09-12T02:18:58.593869+0000 (oldest deadline 2024-09-12T02:19:19.594390+0000) 2024-09-12T02:19:20.433 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:20 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:20.282+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:38.124303+0000 front 2024-09-12T02:18:38.124204+0000 (oldest deadline 2024-09-12T02:19:04.024431+0000) 2024-09-12T02:19:20.434 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:20 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:20.282+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:49.925357+0000 front 2024-09-12T02:18:49.925414+0000 (oldest deadline 2024-09-12T02:19:14.625383+0000) 2024-09-12T02:19:20.434 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 12 02:19:20 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-6[58414]: 2024-09-12T02:19:20.155+0000 7fa4a9867640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:41.416884+0000 front 2024-09-12T02:18:41.416940+0000 (oldest deadline 2024-09-12T02:19:04.317006+0000) 2024-09-12T02:19:20.434 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 12 02:19:20 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-6[58414]: 2024-09-12T02:19:20.155+0000 7fa4a9867640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:50.619002+0000 front 2024-09-12T02:18:50.619295+0000 (oldest deadline 2024-09-12T02:19:16.519137+0000) 2024-09-12T02:19:20.434 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 12 02:19:20 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-6[58414]: 2024-09-12T02:19:20.155+0000 7fa4a9867640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:56.519635+0000 front 2024-09-12T02:18:56.519529+0000 (oldest deadline 2024-09-12T02:19:19.419618+0000) 2024-09-12T02:19:20.875 INFO:journalctl@ceph.osd.8.smithi184.stdout:Sep 12 02:19:20 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-8[54277]: 2024-09-12T02:19:20.614+0000 7fe987aba640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:49.946694+0000 front 2024-09-12T02:18:49.946586+0000 (oldest deadline 2024-09-12T02:19:15.846545+0000) 2024-09-12T02:19:21.127 INFO:journalctl@ceph.osd.5.smithi184.stdout:Sep 12 02:19:20 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-5[47266]: 2024-09-12T02:19:20.974+0000 7f6aeb64d640 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:42.884804+0000 front 2024-09-12T02:18:42.884900+0000 (oldest deadline 2024-09-12T02:19:06.984651+0000) 2024-09-12T02:19:21.128 INFO:journalctl@ceph.osd.5.smithi184.stdout:Sep 12 02:19:20 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-5[47266]: 2024-09-12T02:19:20.974+0000 7f6aeb64d640 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:49.885734+0000 front 2024-09-12T02:18:49.885703+0000 (oldest deadline 2024-09-12T02:19:15.185583+0000) 2024-09-12T02:19:21.128 INFO:journalctl@ceph.osd.5.smithi184.stdout:Sep 12 02:19:20 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-5[47266]: 2024-09-12T02:19:20.974+0000 7f6aeb64d640 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:55.186033+0000 front 2024-09-12T02:18:55.186014+0000 (oldest deadline 2024-09-12T02:19:19.886140+0000) 2024-09-12T02:19:21.128 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:20 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:20.875+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:40.491334+0000 front 2024-09-12T02:18:40.491223+0000 (oldest deadline 2024-09-12T02:19:05.191278+0000) 2024-09-12T02:19:21.128 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:20 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:20.875+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:49.192960+0000 front 2024-09-12T02:18:49.193116+0000 (oldest deadline 2024-09-12T02:19:15.092803+0000) 2024-09-12T02:19:21.128 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:20 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:20.875+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:58.593805+0000 front 2024-09-12T02:18:58.593869+0000 (oldest deadline 2024-09-12T02:19:19.594390+0000) 2024-09-12T02:19:21.169 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:21 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:21.034+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:42.394291+0000 front 2024-09-12T02:18:42.394247+0000 (oldest deadline 2024-09-12T02:19:05.894091+0000) 2024-09-12T02:19:21.169 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:21 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:21.034+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:47.595046+0000 front 2024-09-12T02:18:47.595165+0000 (oldest deadline 2024-09-12T02:19:12.295142+0000) 2024-09-12T02:19:21.169 INFO:journalctl@ceph.osd.7.smithi033.stdout:Sep 12 02:19:20 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-7[53039]: 2024-09-12T02:19:20.834+0000 7f5a34c36640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:50.190710+0000 front 2024-09-12T02:18:50.190738+0000 (oldest deadline 2024-09-12T02:19:13.090875+0000) 2024-09-12T02:19:21.169 INFO:journalctl@ceph.osd.7.smithi033.stdout:Sep 12 02:19:20 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-7[53039]: 2024-09-12T02:19:20.834+0000 7f5a34c36640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:53.091414+0000 front 2024-09-12T02:18:53.091359+0000 (oldest deadline 2024-09-12T02:19:20.691932+0000) 2024-09-12T02:19:21.433 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:21 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:21.261+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:38.124303+0000 front 2024-09-12T02:18:38.124204+0000 (oldest deadline 2024-09-12T02:19:04.024431+0000) 2024-09-12T02:19:21.434 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:21 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:21.261+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:49.925357+0000 front 2024-09-12T02:18:49.925414+0000 (oldest deadline 2024-09-12T02:19:14.625383+0000) 2024-09-12T02:19:21.434 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 12 02:19:21 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-6[58414]: 2024-09-12T02:19:21.150+0000 7fa4a9867640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:41.416884+0000 front 2024-09-12T02:18:41.416940+0000 (oldest deadline 2024-09-12T02:19:04.317006+0000) 2024-09-12T02:19:21.434 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 12 02:19:21 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-6[58414]: 2024-09-12T02:19:21.150+0000 7fa4a9867640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:50.619002+0000 front 2024-09-12T02:18:50.619295+0000 (oldest deadline 2024-09-12T02:19:16.519137+0000) 2024-09-12T02:19:21.434 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 12 02:19:21 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-6[58414]: 2024-09-12T02:19:21.150+0000 7fa4a9867640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:56.519635+0000 front 2024-09-12T02:18:56.519529+0000 (oldest deadline 2024-09-12T02:19:19.419618+0000) 2024-09-12T02:19:21.876 INFO:journalctl@ceph.osd.8.smithi184.stdout:Sep 12 02:19:21 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-8[54277]: 2024-09-12T02:19:21.605+0000 7fe987aba640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:49.946694+0000 front 2024-09-12T02:18:49.946586+0000 (oldest deadline 2024-09-12T02:19:15.846545+0000) 2024-09-12T02:19:22.127 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:21 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:21.876+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:40.491334+0000 front 2024-09-12T02:18:40.491223+0000 (oldest deadline 2024-09-12T02:19:05.191278+0000) 2024-09-12T02:19:22.127 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:21 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:21.876+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:49.192960+0000 front 2024-09-12T02:18:49.193116+0000 (oldest deadline 2024-09-12T02:19:15.092803+0000) 2024-09-12T02:19:22.128 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:21 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:21.876+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:58.593805+0000 front 2024-09-12T02:18:58.593869+0000 (oldest deadline 2024-09-12T02:19:19.594390+0000) 2024-09-12T02:19:22.128 INFO:journalctl@ceph.osd.5.smithi184.stdout:Sep 12 02:19:22 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-5[47266]: 2024-09-12T02:19:22.022+0000 7f6aeb64d640 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:42.884804+0000 front 2024-09-12T02:18:42.884900+0000 (oldest deadline 2024-09-12T02:19:06.984651+0000) 2024-09-12T02:19:22.128 INFO:journalctl@ceph.osd.5.smithi184.stdout:Sep 12 02:19:22 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-5[47266]: 2024-09-12T02:19:22.022+0000 7f6aeb64d640 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:49.885734+0000 front 2024-09-12T02:18:49.885703+0000 (oldest deadline 2024-09-12T02:19:15.185583+0000) 2024-09-12T02:19:22.128 INFO:journalctl@ceph.osd.5.smithi184.stdout:Sep 12 02:19:22 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-5[47266]: 2024-09-12T02:19:22.022+0000 7f6aeb64d640 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:55.186033+0000 front 2024-09-12T02:18:55.186014+0000 (oldest deadline 2024-09-12T02:19:19.886140+0000) 2024-09-12T02:19:22.169 INFO:journalctl@ceph.osd.7.smithi033.stdout:Sep 12 02:19:21 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-7[53039]: 2024-09-12T02:19:21.883+0000 7f5a34c36640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:50.190710+0000 front 2024-09-12T02:18:50.190738+0000 (oldest deadline 2024-09-12T02:19:13.090875+0000) 2024-09-12T02:19:22.169 INFO:journalctl@ceph.osd.7.smithi033.stdout:Sep 12 02:19:21 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-7[53039]: 2024-09-12T02:19:21.883+0000 7f5a34c36640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:53.091414+0000 front 2024-09-12T02:18:53.091359+0000 (oldest deadline 2024-09-12T02:19:20.691932+0000) 2024-09-12T02:19:22.169 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:21 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:21.993+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:42.394291+0000 front 2024-09-12T02:18:42.394247+0000 (oldest deadline 2024-09-12T02:19:05.894091+0000) 2024-09-12T02:19:22.169 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:21 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:21.993+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:47.595046+0000 front 2024-09-12T02:18:47.595165+0000 (oldest deadline 2024-09-12T02:19:12.295142+0000) 2024-09-12T02:19:22.433 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 12 02:19:22 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-6[58414]: 2024-09-12T02:19:22.177+0000 7fa4a9867640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:41.416884+0000 front 2024-09-12T02:18:41.416940+0000 (oldest deadline 2024-09-12T02:19:04.317006+0000) 2024-09-12T02:19:22.434 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 12 02:19:22 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-6[58414]: 2024-09-12T02:19:22.177+0000 7fa4a9867640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:50.619002+0000 front 2024-09-12T02:18:50.619295+0000 (oldest deadline 2024-09-12T02:19:16.519137+0000) 2024-09-12T02:19:22.434 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 12 02:19:22 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-6[58414]: 2024-09-12T02:19:22.177+0000 7fa4a9867640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:56.519635+0000 front 2024-09-12T02:18:56.519529+0000 (oldest deadline 2024-09-12T02:19:19.419618+0000) 2024-09-12T02:19:22.434 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:22 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:22.311+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:38.124303+0000 front 2024-09-12T02:18:38.124204+0000 (oldest deadline 2024-09-12T02:19:04.024431+0000) 2024-09-12T02:19:22.434 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:22 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:22.311+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:49.925357+0000 front 2024-09-12T02:18:49.925414+0000 (oldest deadline 2024-09-12T02:19:14.625383+0000) 2024-09-12T02:19:22.434 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:22 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:22.311+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:58.726706+0000 front 2024-09-12T02:18:58.726591+0000 (oldest deadline 2024-09-12T02:19:22.226596+0000) 2024-09-12T02:19:22.864 INFO:journalctl@ceph.osd.8.smithi184.stdout:Sep 12 02:19:22 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-8[54277]: 2024-09-12T02:19:22.609+0000 7fe987aba640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:49.946694+0000 front 2024-09-12T02:18:49.946586+0000 (oldest deadline 2024-09-12T02:19:15.846545+0000) 2024-09-12T02:19:22.864 INFO:journalctl@ceph.osd.5.smithi184.stdout:Sep 12 02:19:22 smithi184 podman[72877]: 2024-09-12 02:19:22.55415151 +0000 UTC m=+5.623245458 container died bea475ed1b6c71ff368403854c93dd6d16772e16e7e97c1e323dc8934950e567 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-5, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-09-12T02:19:22.864 INFO:journalctl@ceph.osd.5.smithi184.stdout:Sep 12 02:19:22 smithi184 podman[72877]: 2024-09-12 02:19:22.864282489 +0000 UTC m=+5.933376429 container cleanup bea475ed1b6c71ff368403854c93dd6d16772e16e7e97c1e323dc8934950e567 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-5, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-09-12T02:19:23.127 INFO:journalctl@ceph.osd.5.smithi184.stdout:Sep 12 02:19:22 smithi184 bash[72877]: ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-5 2024-09-12T02:19:23.192 INFO:journalctl@ceph.osd.5.smithi184.stdout:Sep 12 02:19:23 smithi184 podman[72895]: 2024-09-12 02:19:23.065279137 +0000 UTC m=+0.504102108 container remove bea475ed1b6c71ff368403854c93dd6d16772e16e7e97c1e323dc8934950e567 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-5, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, ceph=True, GIT_CLEAN=True, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-09-12T02:19:23.192 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:22 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:22.866+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:40.491334+0000 front 2024-09-12T02:18:40.491223+0000 (oldest deadline 2024-09-12T02:19:05.191278+0000) 2024-09-12T02:19:23.192 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:22 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:22.866+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:49.192960+0000 front 2024-09-12T02:18:49.193116+0000 (oldest deadline 2024-09-12T02:19:15.092803+0000) 2024-09-12T02:19:23.193 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:22 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:22.866+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:58.593805+0000 front 2024-09-12T02:18:58.593869+0000 (oldest deadline 2024-09-12T02:19:19.594390+0000) 2024-09-12T02:19:23.193 INFO:journalctl@ceph.osd.7.smithi033.stdout:Sep 12 02:19:22 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-7[53039]: 2024-09-12T02:19:22.852+0000 7f5a34c36640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:50.190710+0000 front 2024-09-12T02:18:50.190738+0000 (oldest deadline 2024-09-12T02:19:13.090875+0000) 2024-09-12T02:19:23.194 INFO:journalctl@ceph.osd.7.smithi033.stdout:Sep 12 02:19:22 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-7[53039]: 2024-09-12T02:19:22.852+0000 7f5a34c36640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:53.091414+0000 front 2024-09-12T02:18:53.091359+0000 (oldest deadline 2024-09-12T02:19:20.691932+0000) 2024-09-12T02:19:23.194 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:23 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:23.042+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:42.394291+0000 front 2024-09-12T02:18:42.394247+0000 (oldest deadline 2024-09-12T02:19:05.894091+0000) 2024-09-12T02:19:23.194 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:23 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:23.042+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:47.595046+0000 front 2024-09-12T02:18:47.595165+0000 (oldest deadline 2024-09-12T02:19:12.295142+0000) 2024-09-12T02:19:23.194 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:23 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:23.042+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:58.696957+0000 front 2024-09-12T02:18:58.696791+0000 (oldest deadline 2024-09-12T02:19:22.196628+0000) 2024-09-12T02:19:23.434 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 12 02:19:23 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-6[58414]: 2024-09-12T02:19:23.166+0000 7fa4a9867640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:41.416884+0000 front 2024-09-12T02:18:41.416940+0000 (oldest deadline 2024-09-12T02:19:04.317006+0000) 2024-09-12T02:19:23.434 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 12 02:19:23 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-6[58414]: 2024-09-12T02:19:23.166+0000 7fa4a9867640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:50.619002+0000 front 2024-09-12T02:18:50.619295+0000 (oldest deadline 2024-09-12T02:19:16.519137+0000) 2024-09-12T02:19:23.434 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 12 02:19:23 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-6[58414]: 2024-09-12T02:19:23.166+0000 7fa4a9867640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:56.519635+0000 front 2024-09-12T02:18:56.519529+0000 (oldest deadline 2024-09-12T02:19:19.419618+0000) 2024-09-12T02:19:23.434 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:23 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:23.311+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:38.124303+0000 front 2024-09-12T02:18:38.124204+0000 (oldest deadline 2024-09-12T02:19:04.024431+0000) 2024-09-12T02:19:23.434 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:23 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:23.311+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:49.925357+0000 front 2024-09-12T02:18:49.925414+0000 (oldest deadline 2024-09-12T02:19:14.625383+0000) 2024-09-12T02:19:23.434 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:23 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:23.311+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:58.726706+0000 front 2024-09-12T02:18:58.726591+0000 (oldest deadline 2024-09-12T02:19:22.226596+0000) 2024-09-12T02:19:23.709 INFO:journalctl@ceph.osd.8.smithi184.stdout:Sep 12 02:19:23 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-8[54277]: 2024-09-12T02:19:23.631+0000 7fe987aba640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:49.946694+0000 front 2024-09-12T02:18:49.946586+0000 (oldest deadline 2024-09-12T02:19:15.846545+0000) 2024-09-12T02:19:23.710 INFO:journalctl@ceph.osd.8.smithi184.stdout:Sep 12 02:19:23 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-8[54277]: 2024-09-12T02:19:23.631+0000 7fe987aba640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:57.547401+0000 front 2024-09-12T02:18:57.547434+0000 (oldest deadline 2024-09-12T02:19:22.847519+0000) 2024-09-12T02:19:23.710 INFO:journalctl@ceph.osd.5.smithi184.stdout:Sep 12 02:19:23 smithi184 podman[72981]: 2024-09-12 02:19:23.287886418 +0000 UTC m=+0.015711891 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T02:19:23.710 INFO:journalctl@ceph.osd.5.smithi184.stdout:Sep 12 02:19:23 smithi184 podman[72981]: 2024-09-12 02:19:23.407949619 +0000 UTC m=+0.135775084 container create 780b9ec5cdb7f13b95fae5a3c163d2b034bcec453331ac42833006bc709a6a08 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-5-deactivate, GIT_CLEAN=True, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T02:19:23.710 INFO:journalctl@ceph.osd.5.smithi184.stdout:Sep 12 02:19:23 smithi184 podman[72981]: 2024-09-12 02:19:23.610501905 +0000 UTC m=+0.338327380 container init 780b9ec5cdb7f13b95fae5a3c163d2b034bcec453331ac42833006bc709a6a08 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-5-deactivate, GIT_CLEAN=True, org.label-schema.build-date=20240731, ceph=True, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-09-12T02:19:23.710 INFO:journalctl@ceph.osd.5.smithi184.stdout:Sep 12 02:19:23 smithi184 podman[72981]: 2024-09-12 02:19:23.6137591 +0000 UTC m=+0.341584576 container start 780b9ec5cdb7f13b95fae5a3c163d2b034bcec453331ac42833006bc709a6a08 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-5-deactivate, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-09-12T02:19:23.710 INFO:journalctl@ceph.osd.5.smithi184.stdout:Sep 12 02:19:23 smithi184 podman[72981]: 2024-09-12 02:19:23.710190038 +0000 UTC m=+0.438015516 container attach 780b9ec5cdb7f13b95fae5a3c163d2b034bcec453331ac42833006bc709a6a08 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-5-deactivate, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2) 2024-09-12T02:19:24.088 INFO:journalctl@ceph.osd.5.smithi184.stdout:Sep 12 02:19:23 smithi184 podman[72981]: 2024-09-12 02:19:23.798909732 +0000 UTC m=+0.526735210 container died 780b9ec5cdb7f13b95fae5a3c163d2b034bcec453331ac42833006bc709a6a08 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-5-deactivate, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True) 2024-09-12T02:19:24.088 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:23 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:23.823+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:40.491334+0000 front 2024-09-12T02:18:40.491223+0000 (oldest deadline 2024-09-12T02:19:05.191278+0000) 2024-09-12T02:19:24.088 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:23 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:23.823+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:49.192960+0000 front 2024-09-12T02:18:49.193116+0000 (oldest deadline 2024-09-12T02:19:15.092803+0000) 2024-09-12T02:19:24.089 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:23 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:23.823+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:58.593805+0000 front 2024-09-12T02:18:58.593869+0000 (oldest deadline 2024-09-12T02:19:19.594390+0000) 2024-09-12T02:19:24.169 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:24 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:24.088+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:42.394291+0000 front 2024-09-12T02:18:42.394247+0000 (oldest deadline 2024-09-12T02:19:05.894091+0000) 2024-09-12T02:19:24.169 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:24 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:24.088+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:47.595046+0000 front 2024-09-12T02:18:47.595165+0000 (oldest deadline 2024-09-12T02:19:12.295142+0000) 2024-09-12T02:19:24.169 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:24 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:24.088+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:58.696957+0000 front 2024-09-12T02:18:58.696791+0000 (oldest deadline 2024-09-12T02:19:22.196628+0000) 2024-09-12T02:19:24.169 INFO:journalctl@ceph.osd.7.smithi033.stdout:Sep 12 02:19:23 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-7[53039]: 2024-09-12T02:19:23.893+0000 7f5a34c36640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:50.190710+0000 front 2024-09-12T02:18:50.190738+0000 (oldest deadline 2024-09-12T02:19:13.090875+0000) 2024-09-12T02:19:24.170 INFO:journalctl@ceph.osd.7.smithi033.stdout:Sep 12 02:19:23 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-7[53039]: 2024-09-12T02:19:23.893+0000 7f5a34c36640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:53.091414+0000 front 2024-09-12T02:18:53.091359+0000 (oldest deadline 2024-09-12T02:19:20.691932+0000) 2024-09-12T02:19:24.348 INFO:journalctl@ceph.osd.5.smithi184.stdout:Sep 12 02:19:24 smithi184 podman[73005]: 2024-09-12 02:19:24.322301508 +0000 UTC m=+0.517279762 container remove 780b9ec5cdb7f13b95fae5a3c163d2b034bcec453331ac42833006bc709a6a08 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-5-deactivate, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=squid-b53b6d3, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux ) 2024-09-12T02:19:24.354 DEBUG:teuthology.orchestra.run.smithi184:> sudo pkill -f 'journalctl -f -n 0 -u ceph-879f3b56-70a9-11ef-bcea-c7b262605968@osd.5.service' 2024-09-12T02:19:24.393 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T02:19:24.394 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-09-12T02:19:24.394 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-09-12T02:19:24.394 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ceph-879f3b56-70a9-11ef-bcea-c7b262605968@osd.6 2024-09-12T02:19:24.684 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:24 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:24.316+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:38.124303+0000 front 2024-09-12T02:18:38.124204+0000 (oldest deadline 2024-09-12T02:19:04.024431+0000) 2024-09-12T02:19:24.684 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:24 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:24.316+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:49.925357+0000 front 2024-09-12T02:18:49.925414+0000 (oldest deadline 2024-09-12T02:19:14.625383+0000) 2024-09-12T02:19:24.684 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:24 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:24.316+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:58.726706+0000 front 2024-09-12T02:18:58.726591+0000 (oldest deadline 2024-09-12T02:19:22.226596+0000) 2024-09-12T02:19:24.684 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 12 02:19:24 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-6[58414]: 2024-09-12T02:19:24.210+0000 7fa4a9867640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:41.416884+0000 front 2024-09-12T02:18:41.416940+0000 (oldest deadline 2024-09-12T02:19:04.317006+0000) 2024-09-12T02:19:24.684 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 12 02:19:24 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-6[58414]: 2024-09-12T02:19:24.210+0000 7fa4a9867640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:50.619002+0000 front 2024-09-12T02:18:50.619295+0000 (oldest deadline 2024-09-12T02:19:16.519137+0000) 2024-09-12T02:19:24.684 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 12 02:19:24 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-6[58414]: 2024-09-12T02:19:24.210+0000 7fa4a9867640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:56.519635+0000 front 2024-09-12T02:18:56.519529+0000 (oldest deadline 2024-09-12T02:19:19.419618+0000) 2024-09-12T02:19:24.685 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 12 02:19:24 smithi016 systemd[1]: Stopping Ceph osd.6 for 879f3b56-70a9-11ef-bcea-c7b262605968... 2024-09-12T02:19:25.127 INFO:journalctl@ceph.osd.8.smithi184.stdout:Sep 12 02:19:24 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-8[54277]: 2024-09-12T02:19:24.672+0000 7fe987aba640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:49.946694+0000 front 2024-09-12T02:18:49.946586+0000 (oldest deadline 2024-09-12T02:19:15.846545+0000) 2024-09-12T02:19:25.128 INFO:journalctl@ceph.osd.8.smithi184.stdout:Sep 12 02:19:24 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-8[54277]: 2024-09-12T02:19:24.672+0000 7fe987aba640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:57.547401+0000 front 2024-09-12T02:18:57.547434+0000 (oldest deadline 2024-09-12T02:19:22.847519+0000) 2024-09-12T02:19:25.128 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:24 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:24.838+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:40.491334+0000 front 2024-09-12T02:18:40.491223+0000 (oldest deadline 2024-09-12T02:19:05.191278+0000) 2024-09-12T02:19:25.128 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:24 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:24.838+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:49.192960+0000 front 2024-09-12T02:18:49.193116+0000 (oldest deadline 2024-09-12T02:19:15.092803+0000) 2024-09-12T02:19:25.128 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:24 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:24.838+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:58.593805+0000 front 2024-09-12T02:18:58.593869+0000 (oldest deadline 2024-09-12T02:19:19.594390+0000) 2024-09-12T02:19:25.169 INFO:journalctl@ceph.osd.7.smithi033.stdout:Sep 12 02:19:24 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-7[53039]: 2024-09-12T02:19:24.907+0000 7f5a34c36640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:50.190710+0000 front 2024-09-12T02:18:50.190738+0000 (oldest deadline 2024-09-12T02:19:13.090875+0000) 2024-09-12T02:19:25.169 INFO:journalctl@ceph.osd.7.smithi033.stdout:Sep 12 02:19:24 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-7[53039]: 2024-09-12T02:19:24.907+0000 7f5a34c36640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:53.091414+0000 front 2024-09-12T02:18:53.091359+0000 (oldest deadline 2024-09-12T02:19:20.691932+0000) 2024-09-12T02:19:25.169 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:25 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:25.136+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:42.394291+0000 front 2024-09-12T02:18:42.394247+0000 (oldest deadline 2024-09-12T02:19:05.894091+0000) 2024-09-12T02:19:25.169 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:25 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:25.136+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:47.595046+0000 front 2024-09-12T02:18:47.595165+0000 (oldest deadline 2024-09-12T02:19:12.295142+0000) 2024-09-12T02:19:25.169 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:25 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:25.136+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:58.696957+0000 front 2024-09-12T02:18:58.696791+0000 (oldest deadline 2024-09-12T02:19:22.196628+0000) 2024-09-12T02:19:25.183 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 12 02:19:24 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-6[58414]: 2024-09-12T02:19:24.699+0000 7fa4b126a640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T02:19:25.183 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 12 02:19:24 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-6[58414]: 2024-09-12T02:19:24.699+0000 7fa4b126a640 -1 osd.6 85 *** Got signal Terminated *** 2024-09-12T02:19:25.183 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 12 02:19:24 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-6[58414]: 2024-09-12T02:19:24.699+0000 7fa4b126a640 -1 osd.6 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T02:19:25.683 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 12 02:19:25 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-6[58414]: 2024-09-12T02:19:25.200+0000 7fa4a9867640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:41.416884+0000 front 2024-09-12T02:18:41.416940+0000 (oldest deadline 2024-09-12T02:19:04.317006+0000) 2024-09-12T02:19:25.684 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 12 02:19:25 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-6[58414]: 2024-09-12T02:19:25.200+0000 7fa4a9867640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:50.619002+0000 front 2024-09-12T02:18:50.619295+0000 (oldest deadline 2024-09-12T02:19:16.519137+0000) 2024-09-12T02:19:25.684 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 12 02:19:25 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-6[58414]: 2024-09-12T02:19:25.200+0000 7fa4a9867640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:56.519635+0000 front 2024-09-12T02:18:56.519529+0000 (oldest deadline 2024-09-12T02:19:19.419618+0000) 2024-09-12T02:19:25.684 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:25 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:25.333+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:38.124303+0000 front 2024-09-12T02:18:38.124204+0000 (oldest deadline 2024-09-12T02:19:04.024431+0000) 2024-09-12T02:19:25.684 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:25 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:25.333+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:49.925357+0000 front 2024-09-12T02:18:49.925414+0000 (oldest deadline 2024-09-12T02:19:14.625383+0000) 2024-09-12T02:19:25.684 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:25 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:25.333+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:58.726706+0000 front 2024-09-12T02:18:58.726591+0000 (oldest deadline 2024-09-12T02:19:22.226596+0000) 2024-09-12T02:19:26.127 INFO:journalctl@ceph.osd.8.smithi184.stdout:Sep 12 02:19:25 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-8[54277]: 2024-09-12T02:19:25.714+0000 7fe987aba640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:49.946694+0000 front 2024-09-12T02:18:49.946586+0000 (oldest deadline 2024-09-12T02:19:15.846545+0000) 2024-09-12T02:19:26.127 INFO:journalctl@ceph.osd.8.smithi184.stdout:Sep 12 02:19:25 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-8[54277]: 2024-09-12T02:19:25.714+0000 7fe987aba640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:57.547401+0000 front 2024-09-12T02:18:57.547434+0000 (oldest deadline 2024-09-12T02:19:22.847519+0000) 2024-09-12T02:19:26.128 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:25 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:25.887+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:40.491334+0000 front 2024-09-12T02:18:40.491223+0000 (oldest deadline 2024-09-12T02:19:05.191278+0000) 2024-09-12T02:19:26.128 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:25 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:25.887+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:49.192960+0000 front 2024-09-12T02:18:49.193116+0000 (oldest deadline 2024-09-12T02:19:15.092803+0000) 2024-09-12T02:19:26.128 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:25 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:25.887+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:58.593805+0000 front 2024-09-12T02:18:58.593869+0000 (oldest deadline 2024-09-12T02:19:19.594390+0000) 2024-09-12T02:19:26.419 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:26 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:26.153+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:42.394291+0000 front 2024-09-12T02:18:42.394247+0000 (oldest deadline 2024-09-12T02:19:05.894091+0000) 2024-09-12T02:19:26.419 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:26 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:26.153+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:47.595046+0000 front 2024-09-12T02:18:47.595165+0000 (oldest deadline 2024-09-12T02:19:12.295142+0000) 2024-09-12T02:19:26.419 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:26 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:26.153+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:58.696957+0000 front 2024-09-12T02:18:58.696791+0000 (oldest deadline 2024-09-12T02:19:22.196628+0000) 2024-09-12T02:19:26.419 INFO:journalctl@ceph.osd.7.smithi033.stdout:Sep 12 02:19:25 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-7[53039]: 2024-09-12T02:19:25.941+0000 7f5a34c36640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:50.190710+0000 front 2024-09-12T02:18:50.190738+0000 (oldest deadline 2024-09-12T02:19:13.090875+0000) 2024-09-12T02:19:26.420 INFO:journalctl@ceph.osd.7.smithi033.stdout:Sep 12 02:19:25 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-7[53039]: 2024-09-12T02:19:25.941+0000 7f5a34c36640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:53.091414+0000 front 2024-09-12T02:18:53.091359+0000 (oldest deadline 2024-09-12T02:19:20.691932+0000) 2024-09-12T02:19:26.684 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 12 02:19:26 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-6[58414]: 2024-09-12T02:19:26.205+0000 7fa4a9867640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:41.416884+0000 front 2024-09-12T02:18:41.416940+0000 (oldest deadline 2024-09-12T02:19:04.317006+0000) 2024-09-12T02:19:26.684 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 12 02:19:26 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-6[58414]: 2024-09-12T02:19:26.205+0000 7fa4a9867640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:50.619002+0000 front 2024-09-12T02:18:50.619295+0000 (oldest deadline 2024-09-12T02:19:16.519137+0000) 2024-09-12T02:19:26.684 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 12 02:19:26 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-6[58414]: 2024-09-12T02:19:26.205+0000 7fa4a9867640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:56.519635+0000 front 2024-09-12T02:18:56.519529+0000 (oldest deadline 2024-09-12T02:19:19.419618+0000) 2024-09-12T02:19:26.684 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:26 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:26.357+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:38.124303+0000 front 2024-09-12T02:18:38.124204+0000 (oldest deadline 2024-09-12T02:19:04.024431+0000) 2024-09-12T02:19:26.684 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:26 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:26.357+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:49.925357+0000 front 2024-09-12T02:18:49.925414+0000 (oldest deadline 2024-09-12T02:19:14.625383+0000) 2024-09-12T02:19:26.684 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:26 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:26.357+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:58.726706+0000 front 2024-09-12T02:18:58.726591+0000 (oldest deadline 2024-09-12T02:19:22.226596+0000) 2024-09-12T02:19:27.127 INFO:journalctl@ceph.osd.8.smithi184.stdout:Sep 12 02:19:26 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-8[54277]: 2024-09-12T02:19:26.674+0000 7fe987aba640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:49.946694+0000 front 2024-09-12T02:18:49.946586+0000 (oldest deadline 2024-09-12T02:19:15.846545+0000) 2024-09-12T02:19:27.127 INFO:journalctl@ceph.osd.8.smithi184.stdout:Sep 12 02:19:26 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-8[54277]: 2024-09-12T02:19:26.674+0000 7fe987aba640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:57.547401+0000 front 2024-09-12T02:18:57.547434+0000 (oldest deadline 2024-09-12T02:19:22.847519+0000) 2024-09-12T02:19:27.128 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:26 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:26.873+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:40.491334+0000 front 2024-09-12T02:18:40.491223+0000 (oldest deadline 2024-09-12T02:19:05.191278+0000) 2024-09-12T02:19:27.128 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:26 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:26.873+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:49.192960+0000 front 2024-09-12T02:18:49.193116+0000 (oldest deadline 2024-09-12T02:19:15.092803+0000) 2024-09-12T02:19:27.128 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:26 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:26.873+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:58.593805+0000 front 2024-09-12T02:18:58.593869+0000 (oldest deadline 2024-09-12T02:19:19.594390+0000) 2024-09-12T02:19:27.199 INFO:journalctl@ceph.osd.7.smithi033.stdout:Sep 12 02:19:26 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-7[53039]: 2024-09-12T02:19:26.920+0000 7f5a34c36640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:50.190710+0000 front 2024-09-12T02:18:50.190738+0000 (oldest deadline 2024-09-12T02:19:13.090875+0000) 2024-09-12T02:19:27.199 INFO:journalctl@ceph.osd.7.smithi033.stdout:Sep 12 02:19:26 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-7[53039]: 2024-09-12T02:19:26.920+0000 7f5a34c36640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:53.091414+0000 front 2024-09-12T02:18:53.091359+0000 (oldest deadline 2024-09-12T02:19:20.691932+0000) 2024-09-12T02:19:27.669 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:27 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:27.198+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:42.394291+0000 front 2024-09-12T02:18:42.394247+0000 (oldest deadline 2024-09-12T02:19:05.894091+0000) 2024-09-12T02:19:27.669 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:27 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:27.198+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:47.595046+0000 front 2024-09-12T02:18:47.595165+0000 (oldest deadline 2024-09-12T02:19:12.295142+0000) 2024-09-12T02:19:27.669 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:27 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:27.198+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:58.696957+0000 front 2024-09-12T02:18:58.696791+0000 (oldest deadline 2024-09-12T02:19:22.196628+0000) 2024-09-12T02:19:27.684 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 12 02:19:27 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-6[58414]: 2024-09-12T02:19:27.207+0000 7fa4a9867640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:41.416884+0000 front 2024-09-12T02:18:41.416940+0000 (oldest deadline 2024-09-12T02:19:04.317006+0000) 2024-09-12T02:19:27.684 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 12 02:19:27 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-6[58414]: 2024-09-12T02:19:27.207+0000 7fa4a9867640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:50.619002+0000 front 2024-09-12T02:18:50.619295+0000 (oldest deadline 2024-09-12T02:19:16.519137+0000) 2024-09-12T02:19:27.684 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 12 02:19:27 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-6[58414]: 2024-09-12T02:19:27.207+0000 7fa4a9867640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:56.519635+0000 front 2024-09-12T02:18:56.519529+0000 (oldest deadline 2024-09-12T02:19:19.419618+0000) 2024-09-12T02:19:27.684 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:27 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:27.336+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:38.124303+0000 front 2024-09-12T02:18:38.124204+0000 (oldest deadline 2024-09-12T02:19:04.024431+0000) 2024-09-12T02:19:27.684 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:27 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:27.336+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:49.925357+0000 front 2024-09-12T02:18:49.925414+0000 (oldest deadline 2024-09-12T02:19:14.625383+0000) 2024-09-12T02:19:27.684 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:27 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:27.336+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:58.726706+0000 front 2024-09-12T02:18:58.726591+0000 (oldest deadline 2024-09-12T02:19:22.226596+0000) 2024-09-12T02:19:28.129 INFO:journalctl@ceph.osd.8.smithi184.stdout:Sep 12 02:19:27 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-8[54277]: 2024-09-12T02:19:27.658+0000 7fe987aba640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:49.946694+0000 front 2024-09-12T02:18:49.946586+0000 (oldest deadline 2024-09-12T02:19:15.846545+0000) 2024-09-12T02:19:28.129 INFO:journalctl@ceph.osd.8.smithi184.stdout:Sep 12 02:19:27 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-8[54277]: 2024-09-12T02:19:27.658+0000 7fe987aba640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:57.547401+0000 front 2024-09-12T02:18:57.547434+0000 (oldest deadline 2024-09-12T02:19:22.847519+0000) 2024-09-12T02:19:28.129 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:27 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:27.833+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:40.491334+0000 front 2024-09-12T02:18:40.491223+0000 (oldest deadline 2024-09-12T02:19:05.191278+0000) 2024-09-12T02:19:28.129 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:27 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:27.833+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:49.192960+0000 front 2024-09-12T02:18:49.193116+0000 (oldest deadline 2024-09-12T02:19:15.092803+0000) 2024-09-12T02:19:28.130 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:27 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:27.833+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:58.593805+0000 front 2024-09-12T02:18:58.593869+0000 (oldest deadline 2024-09-12T02:19:19.594390+0000) 2024-09-12T02:19:28.169 INFO:journalctl@ceph.osd.7.smithi033.stdout:Sep 12 02:19:27 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-7[53039]: 2024-09-12T02:19:27.878+0000 7f5a34c36640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:50.190710+0000 front 2024-09-12T02:18:50.190738+0000 (oldest deadline 2024-09-12T02:19:13.090875+0000) 2024-09-12T02:19:28.169 INFO:journalctl@ceph.osd.7.smithi033.stdout:Sep 12 02:19:27 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-7[53039]: 2024-09-12T02:19:27.878+0000 7f5a34c36640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:53.091414+0000 front 2024-09-12T02:18:53.091359+0000 (oldest deadline 2024-09-12T02:19:20.691932+0000) 2024-09-12T02:19:28.169 INFO:journalctl@ceph.osd.7.smithi033.stdout:Sep 12 02:19:27 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-7[53039]: 2024-09-12T02:19:27.878+0000 7f5a34c36640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T02:19:05.893590+0000 front 2024-09-12T02:19:05.893631+0000 (oldest deadline 2024-09-12T02:19:26.993454+0000) 2024-09-12T02:19:28.669 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:28 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:28.205+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:42.394291+0000 front 2024-09-12T02:18:42.394247+0000 (oldest deadline 2024-09-12T02:19:05.894091+0000) 2024-09-12T02:19:28.669 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:28 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:28.205+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:47.595046+0000 front 2024-09-12T02:18:47.595165+0000 (oldest deadline 2024-09-12T02:19:12.295142+0000) 2024-09-12T02:19:28.669 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:28 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:28.205+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:58.696957+0000 front 2024-09-12T02:18:58.696791+0000 (oldest deadline 2024-09-12T02:19:22.196628+0000) 2024-09-12T02:19:28.684 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:28 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:28.369+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:38.124303+0000 front 2024-09-12T02:18:38.124204+0000 (oldest deadline 2024-09-12T02:19:04.024431+0000) 2024-09-12T02:19:28.684 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:28 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:28.369+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:49.925357+0000 front 2024-09-12T02:18:49.925414+0000 (oldest deadline 2024-09-12T02:19:14.625383+0000) 2024-09-12T02:19:28.684 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:28 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:28.369+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:58.726706+0000 front 2024-09-12T02:18:58.726591+0000 (oldest deadline 2024-09-12T02:19:22.226596+0000) 2024-09-12T02:19:28.684 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 12 02:19:28 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-6[58414]: 2024-09-12T02:19:28.215+0000 7fa4a9867640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:41.416884+0000 front 2024-09-12T02:18:41.416940+0000 (oldest deadline 2024-09-12T02:19:04.317006+0000) 2024-09-12T02:19:28.684 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 12 02:19:28 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-6[58414]: 2024-09-12T02:19:28.215+0000 7fa4a9867640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:50.619002+0000 front 2024-09-12T02:18:50.619295+0000 (oldest deadline 2024-09-12T02:19:16.519137+0000) 2024-09-12T02:19:28.684 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 12 02:19:28 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-6[58414]: 2024-09-12T02:19:28.215+0000 7fa4a9867640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:56.519635+0000 front 2024-09-12T02:18:56.519529+0000 (oldest deadline 2024-09-12T02:19:19.419618+0000) 2024-09-12T02:19:28.684 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 12 02:19:28 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-6[58414]: 2024-09-12T02:19:28.215+0000 7fa4a9867640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T02:19:01.621404+0000 front 2024-09-12T02:19:01.621346+0000 (oldest deadline 2024-09-12T02:19:27.521408+0000) 2024-09-12T02:19:29.029 INFO:journalctl@ceph.osd.8.smithi184.stdout:Sep 12 02:19:28 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-8[54277]: 2024-09-12T02:19:28.638+0000 7fe987aba640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:49.946694+0000 front 2024-09-12T02:18:49.946586+0000 (oldest deadline 2024-09-12T02:19:15.846545+0000) 2024-09-12T02:19:29.029 INFO:journalctl@ceph.osd.8.smithi184.stdout:Sep 12 02:19:28 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-8[54277]: 2024-09-12T02:19:28.638+0000 7fe987aba640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:57.547401+0000 front 2024-09-12T02:18:57.547434+0000 (oldest deadline 2024-09-12T02:19:22.847519+0000) 2024-09-12T02:19:29.029 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:28 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:28.809+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:40.491334+0000 front 2024-09-12T02:18:40.491223+0000 (oldest deadline 2024-09-12T02:19:05.191278+0000) 2024-09-12T02:19:29.029 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:28 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:28.809+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:49.192960+0000 front 2024-09-12T02:18:49.193116+0000 (oldest deadline 2024-09-12T02:19:15.092803+0000) 2024-09-12T02:19:29.030 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:28 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:28.809+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:58.593805+0000 front 2024-09-12T02:18:58.593869+0000 (oldest deadline 2024-09-12T02:19:19.594390+0000) 2024-09-12T02:19:29.169 INFO:journalctl@ceph.osd.7.smithi033.stdout:Sep 12 02:19:28 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-7[53039]: 2024-09-12T02:19:28.902+0000 7f5a34c36640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:50.190710+0000 front 2024-09-12T02:18:50.190738+0000 (oldest deadline 2024-09-12T02:19:13.090875+0000) 2024-09-12T02:19:29.169 INFO:journalctl@ceph.osd.7.smithi033.stdout:Sep 12 02:19:28 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-7[53039]: 2024-09-12T02:19:28.902+0000 7f5a34c36640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:53.091414+0000 front 2024-09-12T02:18:53.091359+0000 (oldest deadline 2024-09-12T02:19:20.691932+0000) 2024-09-12T02:19:29.169 INFO:journalctl@ceph.osd.7.smithi033.stdout:Sep 12 02:19:28 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-7[53039]: 2024-09-12T02:19:28.902+0000 7f5a34c36640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T02:19:05.893590+0000 front 2024-09-12T02:19:05.893631+0000 (oldest deadline 2024-09-12T02:19:26.993454+0000) 2024-09-12T02:19:29.669 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:29 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:29.229+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:42.394291+0000 front 2024-09-12T02:18:42.394247+0000 (oldest deadline 2024-09-12T02:19:05.894091+0000) 2024-09-12T02:19:29.669 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:29 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:29.229+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:47.595046+0000 front 2024-09-12T02:18:47.595165+0000 (oldest deadline 2024-09-12T02:19:12.295142+0000) 2024-09-12T02:19:29.669 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:29 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:29.229+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:58.696957+0000 front 2024-09-12T02:18:58.696791+0000 (oldest deadline 2024-09-12T02:19:22.196628+0000) 2024-09-12T02:19:29.669 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:29 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:29.229+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T02:19:05.097647+0000 front 2024-09-12T02:19:05.097663+0000 (oldest deadline 2024-09-12T02:19:28.498137+0000) 2024-09-12T02:19:29.684 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 12 02:19:29 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-6[58414]: 2024-09-12T02:19:29.185+0000 7fa4a9867640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:41.416884+0000 front 2024-09-12T02:18:41.416940+0000 (oldest deadline 2024-09-12T02:19:04.317006+0000) 2024-09-12T02:19:29.684 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 12 02:19:29 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-6[58414]: 2024-09-12T02:19:29.185+0000 7fa4a9867640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:50.619002+0000 front 2024-09-12T02:18:50.619295+0000 (oldest deadline 2024-09-12T02:19:16.519137+0000) 2024-09-12T02:19:29.684 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 12 02:19:29 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-6[58414]: 2024-09-12T02:19:29.185+0000 7fa4a9867640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:56.519635+0000 front 2024-09-12T02:18:56.519529+0000 (oldest deadline 2024-09-12T02:19:19.419618+0000) 2024-09-12T02:19:29.684 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 12 02:19:29 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-6[58414]: 2024-09-12T02:19:29.185+0000 7fa4a9867640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T02:19:01.621404+0000 front 2024-09-12T02:19:01.621346+0000 (oldest deadline 2024-09-12T02:19:27.521408+0000) 2024-09-12T02:19:29.685 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:29 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:29.342+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:38.124303+0000 front 2024-09-12T02:18:38.124204+0000 (oldest deadline 2024-09-12T02:19:04.024431+0000) 2024-09-12T02:19:29.685 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:29 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:29.342+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:49.925357+0000 front 2024-09-12T02:18:49.925414+0000 (oldest deadline 2024-09-12T02:19:14.625383+0000) 2024-09-12T02:19:29.685 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:29 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:29.342+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:58.726706+0000 front 2024-09-12T02:18:58.726591+0000 (oldest deadline 2024-09-12T02:19:22.226596+0000) 2024-09-12T02:19:29.685 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:29 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:29.342+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T02:19:05.627979+0000 front 2024-09-12T02:19:05.627999+0000 (oldest deadline 2024-09-12T02:19:29.128128+0000) 2024-09-12T02:19:29.877 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:29 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:29.832+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:40.491334+0000 front 2024-09-12T02:18:40.491223+0000 (oldest deadline 2024-09-12T02:19:05.191278+0000) 2024-09-12T02:19:29.877 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:29 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:29.832+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:49.192960+0000 front 2024-09-12T02:18:49.193116+0000 (oldest deadline 2024-09-12T02:19:15.092803+0000) 2024-09-12T02:19:29.877 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:29 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:29.832+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:58.593805+0000 front 2024-09-12T02:18:58.593869+0000 (oldest deadline 2024-09-12T02:19:19.594390+0000) 2024-09-12T02:19:29.878 INFO:journalctl@ceph.osd.8.smithi184.stdout:Sep 12 02:19:29 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-8[54277]: 2024-09-12T02:19:29.592+0000 7fe987aba640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:49.946694+0000 front 2024-09-12T02:18:49.946586+0000 (oldest deadline 2024-09-12T02:19:15.846545+0000) 2024-09-12T02:19:29.878 INFO:journalctl@ceph.osd.8.smithi184.stdout:Sep 12 02:19:29 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-8[54277]: 2024-09-12T02:19:29.592+0000 7fe987aba640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:57.547401+0000 front 2024-09-12T02:18:57.547434+0000 (oldest deadline 2024-09-12T02:19:22.847519+0000) 2024-09-12T02:19:29.878 INFO:journalctl@ceph.osd.8.smithi184.stdout:Sep 12 02:19:29 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-8[54277]: 2024-09-12T02:19:29.592+0000 7fe987aba640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T02:19:03.348610+0000 front 2024-09-12T02:19:03.348558+0000 (oldest deadline 2024-09-12T02:19:28.648626+0000) 2024-09-12T02:19:30.169 INFO:journalctl@ceph.osd.7.smithi033.stdout:Sep 12 02:19:29 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-7[53039]: 2024-09-12T02:19:29.860+0000 7f5a34c36640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:50.190710+0000 front 2024-09-12T02:18:50.190738+0000 (oldest deadline 2024-09-12T02:19:13.090875+0000) 2024-09-12T02:19:30.169 INFO:journalctl@ceph.osd.7.smithi033.stdout:Sep 12 02:19:29 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-7[53039]: 2024-09-12T02:19:29.861+0000 7f5a34c36640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:53.091414+0000 front 2024-09-12T02:18:53.091359+0000 (oldest deadline 2024-09-12T02:19:20.691932+0000) 2024-09-12T02:19:30.169 INFO:journalctl@ceph.osd.7.smithi033.stdout:Sep 12 02:19:29 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-7[53039]: 2024-09-12T02:19:29.861+0000 7f5a34c36640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T02:19:05.893590+0000 front 2024-09-12T02:19:05.893631+0000 (oldest deadline 2024-09-12T02:19:26.993454+0000) 2024-09-12T02:19:30.432 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 12 02:19:30 smithi016 podman[131352]: 2024-09-12 02:19:30.167866416 +0000 UTC m=+5.567278989 container died de82b2ecdc5f024ca3bd120309c639567e3881598d40e140799973c9eb406d62 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-6, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3) 2024-09-12T02:19:30.433 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:30 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:30.319+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:38.124303+0000 front 2024-09-12T02:18:38.124204+0000 (oldest deadline 2024-09-12T02:19:04.024431+0000) 2024-09-12T02:19:30.433 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:30 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:30.319+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:49.925357+0000 front 2024-09-12T02:18:49.925414+0000 (oldest deadline 2024-09-12T02:19:14.625383+0000) 2024-09-12T02:19:30.433 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:30 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:30.319+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:58.726706+0000 front 2024-09-12T02:18:58.726591+0000 (oldest deadline 2024-09-12T02:19:22.226596+0000) 2024-09-12T02:19:30.433 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:30 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:30.319+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T02:19:05.627979+0000 front 2024-09-12T02:19:05.627999+0000 (oldest deadline 2024-09-12T02:19:29.128128+0000) 2024-09-12T02:19:30.669 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:30 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:30.235+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:42.394291+0000 front 2024-09-12T02:18:42.394247+0000 (oldest deadline 2024-09-12T02:19:05.894091+0000) 2024-09-12T02:19:30.669 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:30 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:30.235+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:47.595046+0000 front 2024-09-12T02:18:47.595165+0000 (oldest deadline 2024-09-12T02:19:12.295142+0000) 2024-09-12T02:19:30.669 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:30 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:30.235+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:58.696957+0000 front 2024-09-12T02:18:58.696791+0000 (oldest deadline 2024-09-12T02:19:22.196628+0000) 2024-09-12T02:19:30.669 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:30 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:30.235+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T02:19:05.097647+0000 front 2024-09-12T02:19:05.097663+0000 (oldest deadline 2024-09-12T02:19:28.498137+0000) 2024-09-12T02:19:30.683 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 12 02:19:30 smithi016 podman[131352]: 2024-09-12 02:19:30.61548231 +0000 UTC m=+6.014894873 container cleanup de82b2ecdc5f024ca3bd120309c639567e3881598d40e140799973c9eb406d62 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-6, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-09-12T02:19:30.683 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 12 02:19:30 smithi016 bash[131352]: ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-6 2024-09-12T02:19:31.129 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:30 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:30.782+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:40.491334+0000 front 2024-09-12T02:18:40.491223+0000 (oldest deadline 2024-09-12T02:19:05.191278+0000) 2024-09-12T02:19:31.129 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:30 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:30.782+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:49.192960+0000 front 2024-09-12T02:18:49.193116+0000 (oldest deadline 2024-09-12T02:19:15.092803+0000) 2024-09-12T02:19:31.129 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:30 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:30.782+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:58.593805+0000 front 2024-09-12T02:18:58.593869+0000 (oldest deadline 2024-09-12T02:19:19.594390+0000) 2024-09-12T02:19:31.129 INFO:journalctl@ceph.osd.8.smithi184.stdout:Sep 12 02:19:30 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-8[54277]: 2024-09-12T02:19:30.628+0000 7fe987aba640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:49.946694+0000 front 2024-09-12T02:18:49.946586+0000 (oldest deadline 2024-09-12T02:19:15.846545+0000) 2024-09-12T02:19:31.130 INFO:journalctl@ceph.osd.8.smithi184.stdout:Sep 12 02:19:30 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-8[54277]: 2024-09-12T02:19:30.628+0000 7fe987aba640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:57.547401+0000 front 2024-09-12T02:18:57.547434+0000 (oldest deadline 2024-09-12T02:19:22.847519+0000) 2024-09-12T02:19:31.130 INFO:journalctl@ceph.osd.8.smithi184.stdout:Sep 12 02:19:30 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-8[54277]: 2024-09-12T02:19:30.628+0000 7fe987aba640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T02:19:03.348610+0000 front 2024-09-12T02:19:03.348558+0000 (oldest deadline 2024-09-12T02:19:28.648626+0000) 2024-09-12T02:19:31.169 INFO:journalctl@ceph.osd.7.smithi033.stdout:Sep 12 02:19:30 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-7[53039]: 2024-09-12T02:19:30.862+0000 7f5a34c36640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:50.190710+0000 front 2024-09-12T02:18:50.190738+0000 (oldest deadline 2024-09-12T02:19:13.090875+0000) 2024-09-12T02:19:31.169 INFO:journalctl@ceph.osd.7.smithi033.stdout:Sep 12 02:19:30 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-7[53039]: 2024-09-12T02:19:30.862+0000 7f5a34c36640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:53.091414+0000 front 2024-09-12T02:18:53.091359+0000 (oldest deadline 2024-09-12T02:19:20.691932+0000) 2024-09-12T02:19:31.169 INFO:journalctl@ceph.osd.7.smithi033.stdout:Sep 12 02:19:30 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-7[53039]: 2024-09-12T02:19:30.862+0000 7f5a34c36640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T02:19:05.893590+0000 front 2024-09-12T02:19:05.893631+0000 (oldest deadline 2024-09-12T02:19:26.993454+0000) 2024-09-12T02:19:31.183 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 12 02:19:30 smithi016 podman[131368]: 2024-09-12 02:19:30.900149154 +0000 UTC m=+0.734993195 container remove de82b2ecdc5f024ca3bd120309c639567e3881598d40e140799973c9eb406d62 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-6, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_CLEAN=True, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-09-12T02:19:31.461 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 12 02:19:31 smithi016 podman[131454]: 2024-09-12 02:19:31.147478836 +0000 UTC m=+0.016569449 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T02:19:31.462 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 12 02:19:31 smithi016 podman[131454]: 2024-09-12 02:19:31.251087873 +0000 UTC m=+0.120178476 container create f11e739b88d6c3a6c5a0048fd28d72de862e7fee9c7205ba3e62d4b17ea3ae77 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-6-deactivate, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-09-12T02:19:31.462 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:31 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:31.344+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:38.124303+0000 front 2024-09-12T02:18:38.124204+0000 (oldest deadline 2024-09-12T02:19:04.024431+0000) 2024-09-12T02:19:31.462 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:31 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:31.344+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:49.925357+0000 front 2024-09-12T02:18:49.925414+0000 (oldest deadline 2024-09-12T02:19:14.625383+0000) 2024-09-12T02:19:31.462 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:31 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:31.344+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:58.726706+0000 front 2024-09-12T02:18:58.726591+0000 (oldest deadline 2024-09-12T02:19:22.226596+0000) 2024-09-12T02:19:31.462 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:31 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:31.344+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T02:19:05.627979+0000 front 2024-09-12T02:19:05.627999+0000 (oldest deadline 2024-09-12T02:19:29.128128+0000) 2024-09-12T02:19:31.669 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:31 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:31.227+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:42.394291+0000 front 2024-09-12T02:18:42.394247+0000 (oldest deadline 2024-09-12T02:19:05.894091+0000) 2024-09-12T02:19:31.669 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:31 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:31.227+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:47.595046+0000 front 2024-09-12T02:18:47.595165+0000 (oldest deadline 2024-09-12T02:19:12.295142+0000) 2024-09-12T02:19:31.669 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:31 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:31.227+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:58.696957+0000 front 2024-09-12T02:18:58.696791+0000 (oldest deadline 2024-09-12T02:19:22.196628+0000) 2024-09-12T02:19:31.670 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:31 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:31.227+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T02:19:05.097647+0000 front 2024-09-12T02:19:05.097663+0000 (oldest deadline 2024-09-12T02:19:28.498137+0000) 2024-09-12T02:19:31.877 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:31 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:31.747+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:40.491334+0000 front 2024-09-12T02:18:40.491223+0000 (oldest deadline 2024-09-12T02:19:05.191278+0000) 2024-09-12T02:19:31.878 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:31 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:31.747+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:49.192960+0000 front 2024-09-12T02:18:49.193116+0000 (oldest deadline 2024-09-12T02:19:15.092803+0000) 2024-09-12T02:19:31.878 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:31 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:31.747+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:58.593805+0000 front 2024-09-12T02:18:58.593869+0000 (oldest deadline 2024-09-12T02:19:19.594390+0000) 2024-09-12T02:19:31.878 INFO:journalctl@ceph.osd.8.smithi184.stdout:Sep 12 02:19:31 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-8[54277]: 2024-09-12T02:19:31.624+0000 7fe987aba640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:49.946694+0000 front 2024-09-12T02:18:49.946586+0000 (oldest deadline 2024-09-12T02:19:15.846545+0000) 2024-09-12T02:19:31.878 INFO:journalctl@ceph.osd.8.smithi184.stdout:Sep 12 02:19:31 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-8[54277]: 2024-09-12T02:19:31.624+0000 7fe987aba640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:57.547401+0000 front 2024-09-12T02:18:57.547434+0000 (oldest deadline 2024-09-12T02:19:22.847519+0000) 2024-09-12T02:19:31.878 INFO:journalctl@ceph.osd.8.smithi184.stdout:Sep 12 02:19:31 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-8[54277]: 2024-09-12T02:19:31.624+0000 7fe987aba640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T02:19:03.348610+0000 front 2024-09-12T02:19:03.348558+0000 (oldest deadline 2024-09-12T02:19:28.648626+0000) 2024-09-12T02:19:31.934 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 12 02:19:31 smithi016 podman[131454]: 2024-09-12 02:19:31.461937473 +0000 UTC m=+0.331028077 container init f11e739b88d6c3a6c5a0048fd28d72de862e7fee9c7205ba3e62d4b17ea3ae77 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-6-deactivate, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.schema-version=1.0) 2024-09-12T02:19:31.934 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 12 02:19:31 smithi016 podman[131454]: 2024-09-12 02:19:31.465497581 +0000 UTC m=+0.334588182 container start f11e739b88d6c3a6c5a0048fd28d72de862e7fee9c7205ba3e62d4b17ea3ae77 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-6-deactivate, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731) 2024-09-12T02:19:31.934 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 12 02:19:31 smithi016 podman[131454]: 2024-09-12 02:19:31.536815615 +0000 UTC m=+0.405906229 container attach f11e739b88d6c3a6c5a0048fd28d72de862e7fee9c7205ba3e62d4b17ea3ae77 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-6-deactivate, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T02:19:31.934 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 12 02:19:31 smithi016 podman[131454]: 2024-09-12 02:19:31.661577707 +0000 UTC m=+0.530668319 container died f11e739b88d6c3a6c5a0048fd28d72de862e7fee9c7205ba3e62d4b17ea3ae77 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-6-deactivate, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-12T02:19:32.169 INFO:journalctl@ceph.osd.7.smithi033.stdout:Sep 12 02:19:31 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-7[53039]: 2024-09-12T02:19:31.864+0000 7f5a34c36640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:50.190710+0000 front 2024-09-12T02:18:50.190738+0000 (oldest deadline 2024-09-12T02:19:13.090875+0000) 2024-09-12T02:19:32.169 INFO:journalctl@ceph.osd.7.smithi033.stdout:Sep 12 02:19:31 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-7[53039]: 2024-09-12T02:19:31.864+0000 7f5a34c36640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:53.091414+0000 front 2024-09-12T02:18:53.091359+0000 (oldest deadline 2024-09-12T02:19:20.691932+0000) 2024-09-12T02:19:32.169 INFO:journalctl@ceph.osd.7.smithi033.stdout:Sep 12 02:19:31 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-7[53039]: 2024-09-12T02:19:31.864+0000 7f5a34c36640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T02:19:05.893590+0000 front 2024-09-12T02:19:05.893631+0000 (oldest deadline 2024-09-12T02:19:26.993454+0000) 2024-09-12T02:19:32.275 DEBUG:teuthology.orchestra.run.smithi016:> sudo pkill -f 'journalctl -f -n 0 -u ceph-879f3b56-70a9-11ef-bcea-c7b262605968@osd.6.service' 2024-09-12T02:19:32.320 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T02:19:32.320 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-09-12T02:19:32.320 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-09-12T02:19:32.321 DEBUG:teuthology.orchestra.run.smithi033:> sudo systemctl stop ceph-879f3b56-70a9-11ef-bcea-c7b262605968@osd.7 2024-09-12T02:19:32.540 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:32 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:32.296+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:38.124303+0000 front 2024-09-12T02:18:38.124204+0000 (oldest deadline 2024-09-12T02:19:04.024431+0000) 2024-09-12T02:19:32.541 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:32 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:32.296+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:49.925357+0000 front 2024-09-12T02:18:49.925414+0000 (oldest deadline 2024-09-12T02:19:14.625383+0000) 2024-09-12T02:19:32.541 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:32 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:32.296+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:58.726706+0000 front 2024-09-12T02:18:58.726591+0000 (oldest deadline 2024-09-12T02:19:22.226596+0000) 2024-09-12T02:19:32.541 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:32 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:32.296+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T02:19:05.627979+0000 front 2024-09-12T02:19:05.627999+0000 (oldest deadline 2024-09-12T02:19:29.128128+0000) 2024-09-12T02:19:32.615 INFO:journalctl@ceph.osd.7.smithi033.stdout:Sep 12 02:19:32 smithi033 systemd[1]: Stopping Ceph osd.7 for 879f3b56-70a9-11ef-bcea-c7b262605968... 2024-09-12T02:19:32.615 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:32 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:32.224+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:42.394291+0000 front 2024-09-12T02:18:42.394247+0000 (oldest deadline 2024-09-12T02:19:05.894091+0000) 2024-09-12T02:19:32.615 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:32 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:32.224+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:47.595046+0000 front 2024-09-12T02:18:47.595165+0000 (oldest deadline 2024-09-12T02:19:12.295142+0000) 2024-09-12T02:19:32.615 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:32 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:32.224+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:58.696957+0000 front 2024-09-12T02:18:58.696791+0000 (oldest deadline 2024-09-12T02:19:22.196628+0000) 2024-09-12T02:19:32.615 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:32 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:32.224+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T02:19:05.097647+0000 front 2024-09-12T02:19:05.097663+0000 (oldest deadline 2024-09-12T02:19:28.498137+0000) 2024-09-12T02:19:32.919 INFO:journalctl@ceph.osd.7.smithi033.stdout:Sep 12 02:19:32 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-7[53039]: 2024-09-12T02:19:32.614+0000 7f5a3c639640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T02:19:32.919 INFO:journalctl@ceph.osd.7.smithi033.stdout:Sep 12 02:19:32 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-7[53039]: 2024-09-12T02:19:32.614+0000 7f5a3c639640 -1 osd.7 85 *** Got signal Terminated *** 2024-09-12T02:19:32.920 INFO:journalctl@ceph.osd.7.smithi033.stdout:Sep 12 02:19:32 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-7[53039]: 2024-09-12T02:19:32.614+0000 7f5a3c639640 -1 osd.7 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T02:19:32.920 INFO:journalctl@ceph.osd.7.smithi033.stdout:Sep 12 02:19:32 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-7[53039]: 2024-09-12T02:19:32.853+0000 7f5a34c36640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:50.190710+0000 front 2024-09-12T02:18:50.190738+0000 (oldest deadline 2024-09-12T02:19:13.090875+0000) 2024-09-12T02:19:32.920 INFO:journalctl@ceph.osd.7.smithi033.stdout:Sep 12 02:19:32 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-7[53039]: 2024-09-12T02:19:32.853+0000 7f5a34c36640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:53.091414+0000 front 2024-09-12T02:18:53.091359+0000 (oldest deadline 2024-09-12T02:19:20.691932+0000) 2024-09-12T02:19:32.920 INFO:journalctl@ceph.osd.7.smithi033.stdout:Sep 12 02:19:32 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-7[53039]: 2024-09-12T02:19:32.853+0000 7f5a34c36640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T02:19:05.893590+0000 front 2024-09-12T02:19:05.893631+0000 (oldest deadline 2024-09-12T02:19:26.993454+0000) 2024-09-12T02:19:33.127 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:32 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:32.772+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:40.491334+0000 front 2024-09-12T02:18:40.491223+0000 (oldest deadline 2024-09-12T02:19:05.191278+0000) 2024-09-12T02:19:33.127 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:32 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:32.772+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:49.192960+0000 front 2024-09-12T02:18:49.193116+0000 (oldest deadline 2024-09-12T02:19:15.092803+0000) 2024-09-12T02:19:33.127 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:32 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:32.772+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:58.593805+0000 front 2024-09-12T02:18:58.593869+0000 (oldest deadline 2024-09-12T02:19:19.594390+0000) 2024-09-12T02:19:33.128 INFO:journalctl@ceph.osd.8.smithi184.stdout:Sep 12 02:19:32 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-8[54277]: 2024-09-12T02:19:32.668+0000 7fe987aba640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:49.946694+0000 front 2024-09-12T02:18:49.946586+0000 (oldest deadline 2024-09-12T02:19:15.846545+0000) 2024-09-12T02:19:33.128 INFO:journalctl@ceph.osd.8.smithi184.stdout:Sep 12 02:19:32 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-8[54277]: 2024-09-12T02:19:32.668+0000 7fe987aba640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:57.547401+0000 front 2024-09-12T02:18:57.547434+0000 (oldest deadline 2024-09-12T02:19:22.847519+0000) 2024-09-12T02:19:33.128 INFO:journalctl@ceph.osd.8.smithi184.stdout:Sep 12 02:19:32 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-8[54277]: 2024-09-12T02:19:32.668+0000 7fe987aba640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T02:19:03.348610+0000 front 2024-09-12T02:19:03.348558+0000 (oldest deadline 2024-09-12T02:19:28.648626+0000) 2024-09-12T02:19:33.561 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:33 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:33.304+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:38.124303+0000 front 2024-09-12T02:18:38.124204+0000 (oldest deadline 2024-09-12T02:19:04.024431+0000) 2024-09-12T02:19:33.561 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:33 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:33.304+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:49.925357+0000 front 2024-09-12T02:18:49.925414+0000 (oldest deadline 2024-09-12T02:19:14.625383+0000) 2024-09-12T02:19:33.562 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:33 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:33.304+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:58.726706+0000 front 2024-09-12T02:18:58.726591+0000 (oldest deadline 2024-09-12T02:19:22.226596+0000) 2024-09-12T02:19:33.562 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:33 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:33.304+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T02:19:05.627979+0000 front 2024-09-12T02:19:05.627999+0000 (oldest deadline 2024-09-12T02:19:29.128128+0000) 2024-09-12T02:19:33.669 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:33 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:33.180+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:42.394291+0000 front 2024-09-12T02:18:42.394247+0000 (oldest deadline 2024-09-12T02:19:05.894091+0000) 2024-09-12T02:19:33.669 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:33 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:33.180+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:47.595046+0000 front 2024-09-12T02:18:47.595165+0000 (oldest deadline 2024-09-12T02:19:12.295142+0000) 2024-09-12T02:19:33.669 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:33 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:33.180+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:58.696957+0000 front 2024-09-12T02:18:58.696791+0000 (oldest deadline 2024-09-12T02:19:22.196628+0000) 2024-09-12T02:19:33.669 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:33 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:33.180+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T02:19:05.097647+0000 front 2024-09-12T02:19:05.097663+0000 (oldest deadline 2024-09-12T02:19:28.498137+0000) 2024-09-12T02:19:34.128 INFO:journalctl@ceph.osd.8.smithi184.stdout:Sep 12 02:19:33 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-8[54277]: 2024-09-12T02:19:33.661+0000 7fe987aba640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:49.946694+0000 front 2024-09-12T02:18:49.946586+0000 (oldest deadline 2024-09-12T02:19:15.846545+0000) 2024-09-12T02:19:34.128 INFO:journalctl@ceph.osd.8.smithi184.stdout:Sep 12 02:19:33 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-8[54277]: 2024-09-12T02:19:33.661+0000 7fe987aba640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:57.547401+0000 front 2024-09-12T02:18:57.547434+0000 (oldest deadline 2024-09-12T02:19:22.847519+0000) 2024-09-12T02:19:34.129 INFO:journalctl@ceph.osd.8.smithi184.stdout:Sep 12 02:19:33 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-8[54277]: 2024-09-12T02:19:33.661+0000 7fe987aba640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T02:19:03.348610+0000 front 2024-09-12T02:19:03.348558+0000 (oldest deadline 2024-09-12T02:19:28.648626+0000) 2024-09-12T02:19:34.129 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:33 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:33.799+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:40.491334+0000 front 2024-09-12T02:18:40.491223+0000 (oldest deadline 2024-09-12T02:19:05.191278+0000) 2024-09-12T02:19:34.129 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:33 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:33.799+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:49.192960+0000 front 2024-09-12T02:18:49.193116+0000 (oldest deadline 2024-09-12T02:19:15.092803+0000) 2024-09-12T02:19:34.129 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:33 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:33.799+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:58.593805+0000 front 2024-09-12T02:18:58.593869+0000 (oldest deadline 2024-09-12T02:19:19.594390+0000) 2024-09-12T02:19:34.169 INFO:journalctl@ceph.osd.7.smithi033.stdout:Sep 12 02:19:33 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-7[53039]: 2024-09-12T02:19:33.888+0000 7f5a34c36640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:50.190710+0000 front 2024-09-12T02:18:50.190738+0000 (oldest deadline 2024-09-12T02:19:13.090875+0000) 2024-09-12T02:19:34.169 INFO:journalctl@ceph.osd.7.smithi033.stdout:Sep 12 02:19:33 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-7[53039]: 2024-09-12T02:19:33.888+0000 7f5a34c36640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:53.091414+0000 front 2024-09-12T02:18:53.091359+0000 (oldest deadline 2024-09-12T02:19:20.691932+0000) 2024-09-12T02:19:34.169 INFO:journalctl@ceph.osd.7.smithi033.stdout:Sep 12 02:19:33 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-7[53039]: 2024-09-12T02:19:33.888+0000 7f5a34c36640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T02:19:05.893590+0000 front 2024-09-12T02:19:05.893631+0000 (oldest deadline 2024-09-12T02:19:26.993454+0000) 2024-09-12T02:19:34.451 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:34 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:34.214+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:42.394291+0000 front 2024-09-12T02:18:42.394247+0000 (oldest deadline 2024-09-12T02:19:05.894091+0000) 2024-09-12T02:19:34.451 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:34 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:34.214+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:47.595046+0000 front 2024-09-12T02:18:47.595165+0000 (oldest deadline 2024-09-12T02:19:12.295142+0000) 2024-09-12T02:19:34.451 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:34 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:34.214+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:58.696957+0000 front 2024-09-12T02:18:58.696791+0000 (oldest deadline 2024-09-12T02:19:22.196628+0000) 2024-09-12T02:19:34.452 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:34 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:34.214+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T02:19:05.097647+0000 front 2024-09-12T02:19:05.097663+0000 (oldest deadline 2024-09-12T02:19:28.498137+0000) 2024-09-12T02:19:34.684 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:34 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:34.280+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:38.124303+0000 front 2024-09-12T02:18:38.124204+0000 (oldest deadline 2024-09-12T02:19:04.024431+0000) 2024-09-12T02:19:34.684 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:34 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:34.280+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:49.925357+0000 front 2024-09-12T02:18:49.925414+0000 (oldest deadline 2024-09-12T02:19:14.625383+0000) 2024-09-12T02:19:34.684 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:34 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:34.280+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:58.726706+0000 front 2024-09-12T02:18:58.726591+0000 (oldest deadline 2024-09-12T02:19:22.226596+0000) 2024-09-12T02:19:34.684 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:34 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:34.280+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T02:19:05.627979+0000 front 2024-09-12T02:19:05.627999+0000 (oldest deadline 2024-09-12T02:19:29.128128+0000) 2024-09-12T02:19:35.127 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:34 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:34.768+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:40.491334+0000 front 2024-09-12T02:18:40.491223+0000 (oldest deadline 2024-09-12T02:19:05.191278+0000) 2024-09-12T02:19:35.127 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:34 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:34.768+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:49.192960+0000 front 2024-09-12T02:18:49.193116+0000 (oldest deadline 2024-09-12T02:19:15.092803+0000) 2024-09-12T02:19:35.128 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:34 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:34.768+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:58.593805+0000 front 2024-09-12T02:18:58.593869+0000 (oldest deadline 2024-09-12T02:19:19.594390+0000) 2024-09-12T02:19:35.128 INFO:journalctl@ceph.osd.8.smithi184.stdout:Sep 12 02:19:34 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-8[54277]: 2024-09-12T02:19:34.629+0000 7fe987aba640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:49.946694+0000 front 2024-09-12T02:18:49.946586+0000 (oldest deadline 2024-09-12T02:19:15.846545+0000) 2024-09-12T02:19:35.128 INFO:journalctl@ceph.osd.8.smithi184.stdout:Sep 12 02:19:34 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-8[54277]: 2024-09-12T02:19:34.629+0000 7fe987aba640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:57.547401+0000 front 2024-09-12T02:18:57.547434+0000 (oldest deadline 2024-09-12T02:19:22.847519+0000) 2024-09-12T02:19:35.128 INFO:journalctl@ceph.osd.8.smithi184.stdout:Sep 12 02:19:34 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-8[54277]: 2024-09-12T02:19:34.629+0000 7fe987aba640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T02:19:03.348610+0000 front 2024-09-12T02:19:03.348558+0000 (oldest deadline 2024-09-12T02:19:28.648626+0000) 2024-09-12T02:19:35.169 INFO:journalctl@ceph.osd.7.smithi033.stdout:Sep 12 02:19:34 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-7[53039]: 2024-09-12T02:19:34.845+0000 7f5a34c36640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:50.190710+0000 front 2024-09-12T02:18:50.190738+0000 (oldest deadline 2024-09-12T02:19:13.090875+0000) 2024-09-12T02:19:35.169 INFO:journalctl@ceph.osd.7.smithi033.stdout:Sep 12 02:19:34 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-7[53039]: 2024-09-12T02:19:34.845+0000 7f5a34c36640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:53.091414+0000 front 2024-09-12T02:18:53.091359+0000 (oldest deadline 2024-09-12T02:19:20.691932+0000) 2024-09-12T02:19:35.169 INFO:journalctl@ceph.osd.7.smithi033.stdout:Sep 12 02:19:34 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-7[53039]: 2024-09-12T02:19:34.845+0000 7f5a34c36640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T02:19:05.893590+0000 front 2024-09-12T02:19:05.893631+0000 (oldest deadline 2024-09-12T02:19:26.993454+0000) 2024-09-12T02:19:35.669 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:35 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:35.234+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:42.394291+0000 front 2024-09-12T02:18:42.394247+0000 (oldest deadline 2024-09-12T02:19:05.894091+0000) 2024-09-12T02:19:35.669 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:35 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:35.234+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:47.595046+0000 front 2024-09-12T02:18:47.595165+0000 (oldest deadline 2024-09-12T02:19:12.295142+0000) 2024-09-12T02:19:35.669 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:35 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:35.234+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:58.696957+0000 front 2024-09-12T02:18:58.696791+0000 (oldest deadline 2024-09-12T02:19:22.196628+0000) 2024-09-12T02:19:35.669 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:35 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:35.234+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T02:19:05.097647+0000 front 2024-09-12T02:19:05.097663+0000 (oldest deadline 2024-09-12T02:19:28.498137+0000) 2024-09-12T02:19:35.684 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:35 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:35.309+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:38.124303+0000 front 2024-09-12T02:18:38.124204+0000 (oldest deadline 2024-09-12T02:19:04.024431+0000) 2024-09-12T02:19:35.684 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:35 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:35.309+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:49.925357+0000 front 2024-09-12T02:18:49.925414+0000 (oldest deadline 2024-09-12T02:19:14.625383+0000) 2024-09-12T02:19:35.684 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:35 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:35.309+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:58.726706+0000 front 2024-09-12T02:18:58.726591+0000 (oldest deadline 2024-09-12T02:19:22.226596+0000) 2024-09-12T02:19:35.684 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:35 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:35.309+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T02:19:05.627979+0000 front 2024-09-12T02:19:05.627999+0000 (oldest deadline 2024-09-12T02:19:29.128128+0000) 2024-09-12T02:19:36.127 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:35 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:35.808+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:40.491334+0000 front 2024-09-12T02:18:40.491223+0000 (oldest deadline 2024-09-12T02:19:05.191278+0000) 2024-09-12T02:19:36.127 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:35 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:35.808+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:49.192960+0000 front 2024-09-12T02:18:49.193116+0000 (oldest deadline 2024-09-12T02:19:15.092803+0000) 2024-09-12T02:19:36.127 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:35 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:35.808+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:58.593805+0000 front 2024-09-12T02:18:58.593869+0000 (oldest deadline 2024-09-12T02:19:19.594390+0000) 2024-09-12T02:19:36.128 INFO:journalctl@ceph.osd.8.smithi184.stdout:Sep 12 02:19:35 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-8[54277]: 2024-09-12T02:19:35.666+0000 7fe987aba640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:49.946694+0000 front 2024-09-12T02:18:49.946586+0000 (oldest deadline 2024-09-12T02:19:15.846545+0000) 2024-09-12T02:19:36.128 INFO:journalctl@ceph.osd.8.smithi184.stdout:Sep 12 02:19:35 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-8[54277]: 2024-09-12T02:19:35.666+0000 7fe987aba640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:57.547401+0000 front 2024-09-12T02:18:57.547434+0000 (oldest deadline 2024-09-12T02:19:22.847519+0000) 2024-09-12T02:19:36.128 INFO:journalctl@ceph.osd.8.smithi184.stdout:Sep 12 02:19:35 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-8[54277]: 2024-09-12T02:19:35.666+0000 7fe987aba640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T02:19:03.348610+0000 front 2024-09-12T02:19:03.348558+0000 (oldest deadline 2024-09-12T02:19:28.648626+0000) 2024-09-12T02:19:36.169 INFO:journalctl@ceph.osd.7.smithi033.stdout:Sep 12 02:19:35 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-7[53039]: 2024-09-12T02:19:35.830+0000 7f5a34c36640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:50.190710+0000 front 2024-09-12T02:18:50.190738+0000 (oldest deadline 2024-09-12T02:19:13.090875+0000) 2024-09-12T02:19:36.169 INFO:journalctl@ceph.osd.7.smithi033.stdout:Sep 12 02:19:35 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-7[53039]: 2024-09-12T02:19:35.830+0000 7f5a34c36640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:53.091414+0000 front 2024-09-12T02:18:53.091359+0000 (oldest deadline 2024-09-12T02:19:20.691932+0000) 2024-09-12T02:19:36.169 INFO:journalctl@ceph.osd.7.smithi033.stdout:Sep 12 02:19:35 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-7[53039]: 2024-09-12T02:19:35.830+0000 7f5a34c36640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T02:19:05.893590+0000 front 2024-09-12T02:19:05.893631+0000 (oldest deadline 2024-09-12T02:19:26.993454+0000) 2024-09-12T02:19:36.669 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:36 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:36.224+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:42.394291+0000 front 2024-09-12T02:18:42.394247+0000 (oldest deadline 2024-09-12T02:19:05.894091+0000) 2024-09-12T02:19:36.669 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:36 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:36.224+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:47.595046+0000 front 2024-09-12T02:18:47.595165+0000 (oldest deadline 2024-09-12T02:19:12.295142+0000) 2024-09-12T02:19:36.669 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:36 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:36.224+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:58.696957+0000 front 2024-09-12T02:18:58.696791+0000 (oldest deadline 2024-09-12T02:19:22.196628+0000) 2024-09-12T02:19:36.669 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:36 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:36.224+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T02:19:05.097647+0000 front 2024-09-12T02:19:05.097663+0000 (oldest deadline 2024-09-12T02:19:28.498137+0000) 2024-09-12T02:19:36.670 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:36 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:36.224+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.33:6812 osd.4 since back 2024-09-12T02:19:11.899193+0000 front 2024-09-12T02:19:11.899218+0000 (oldest deadline 2024-09-12T02:19:35.999293+0000) 2024-09-12T02:19:36.683 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:36 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:36.323+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:38.124303+0000 front 2024-09-12T02:18:38.124204+0000 (oldest deadline 2024-09-12T02:19:04.024431+0000) 2024-09-12T02:19:36.684 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:36 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:36.323+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:49.925357+0000 front 2024-09-12T02:18:49.925414+0000 (oldest deadline 2024-09-12T02:19:14.625383+0000) 2024-09-12T02:19:36.684 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:36 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:36.323+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:58.726706+0000 front 2024-09-12T02:18:58.726591+0000 (oldest deadline 2024-09-12T02:19:22.226596+0000) 2024-09-12T02:19:36.684 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:36 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:36.323+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T02:19:05.627979+0000 front 2024-09-12T02:19:05.627999+0000 (oldest deadline 2024-09-12T02:19:29.128128+0000) 2024-09-12T02:19:37.127 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:36 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:36.836+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:40.491334+0000 front 2024-09-12T02:18:40.491223+0000 (oldest deadline 2024-09-12T02:19:05.191278+0000) 2024-09-12T02:19:37.127 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:36 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:36.836+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:49.192960+0000 front 2024-09-12T02:18:49.193116+0000 (oldest deadline 2024-09-12T02:19:15.092803+0000) 2024-09-12T02:19:37.128 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:36 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:36.836+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:58.593805+0000 front 2024-09-12T02:18:58.593869+0000 (oldest deadline 2024-09-12T02:19:19.594390+0000) 2024-09-12T02:19:37.128 INFO:journalctl@ceph.osd.8.smithi184.stdout:Sep 12 02:19:36 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-8[54277]: 2024-09-12T02:19:36.688+0000 7fe987aba640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:49.946694+0000 front 2024-09-12T02:18:49.946586+0000 (oldest deadline 2024-09-12T02:19:15.846545+0000) 2024-09-12T02:19:37.128 INFO:journalctl@ceph.osd.8.smithi184.stdout:Sep 12 02:19:36 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-8[54277]: 2024-09-12T02:19:36.688+0000 7fe987aba640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:57.547401+0000 front 2024-09-12T02:18:57.547434+0000 (oldest deadline 2024-09-12T02:19:22.847519+0000) 2024-09-12T02:19:37.128 INFO:journalctl@ceph.osd.8.smithi184.stdout:Sep 12 02:19:36 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-8[54277]: 2024-09-12T02:19:36.688+0000 7fe987aba640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T02:19:03.348610+0000 front 2024-09-12T02:19:03.348558+0000 (oldest deadline 2024-09-12T02:19:28.648626+0000) 2024-09-12T02:19:37.169 INFO:journalctl@ceph.osd.7.smithi033.stdout:Sep 12 02:19:36 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-7[53039]: 2024-09-12T02:19:36.822+0000 7f5a34c36640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:50.190710+0000 front 2024-09-12T02:18:50.190738+0000 (oldest deadline 2024-09-12T02:19:13.090875+0000) 2024-09-12T02:19:37.169 INFO:journalctl@ceph.osd.7.smithi033.stdout:Sep 12 02:19:36 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-7[53039]: 2024-09-12T02:19:36.822+0000 7f5a34c36640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:53.091414+0000 front 2024-09-12T02:18:53.091359+0000 (oldest deadline 2024-09-12T02:19:20.691932+0000) 2024-09-12T02:19:37.169 INFO:journalctl@ceph.osd.7.smithi033.stdout:Sep 12 02:19:36 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-7[53039]: 2024-09-12T02:19:36.822+0000 7f5a34c36640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T02:19:05.893590+0000 front 2024-09-12T02:19:05.893631+0000 (oldest deadline 2024-09-12T02:19:26.993454+0000) 2024-09-12T02:19:37.673 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:37 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:37.252+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:42.394291+0000 front 2024-09-12T02:18:42.394247+0000 (oldest deadline 2024-09-12T02:19:05.894091+0000) 2024-09-12T02:19:37.674 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:37 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:37.252+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:47.595046+0000 front 2024-09-12T02:18:47.595165+0000 (oldest deadline 2024-09-12T02:19:12.295142+0000) 2024-09-12T02:19:37.674 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:37 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:37.252+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:58.696957+0000 front 2024-09-12T02:18:58.696791+0000 (oldest deadline 2024-09-12T02:19:22.196628+0000) 2024-09-12T02:19:37.674 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:37 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:37.252+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T02:19:05.097647+0000 front 2024-09-12T02:19:05.097663+0000 (oldest deadline 2024-09-12T02:19:28.498137+0000) 2024-09-12T02:19:37.674 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:37 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:37.252+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.33:6812 osd.4 since back 2024-09-12T02:19:11.899193+0000 front 2024-09-12T02:19:11.899218+0000 (oldest deadline 2024-09-12T02:19:35.999293+0000) 2024-09-12T02:19:37.683 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:37 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:37.274+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:38.124303+0000 front 2024-09-12T02:18:38.124204+0000 (oldest deadline 2024-09-12T02:19:04.024431+0000) 2024-09-12T02:19:37.683 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:37 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:37.274+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:49.925357+0000 front 2024-09-12T02:18:49.925414+0000 (oldest deadline 2024-09-12T02:19:14.625383+0000) 2024-09-12T02:19:37.684 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:37 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:37.274+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:58.726706+0000 front 2024-09-12T02:18:58.726591+0000 (oldest deadline 2024-09-12T02:19:22.226596+0000) 2024-09-12T02:19:37.684 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:37 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:37.274+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T02:19:05.627979+0000 front 2024-09-12T02:19:05.627999+0000 (oldest deadline 2024-09-12T02:19:29.128128+0000) 2024-09-12T02:19:38.127 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:37 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:37.842+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:40.491334+0000 front 2024-09-12T02:18:40.491223+0000 (oldest deadline 2024-09-12T02:19:05.191278+0000) 2024-09-12T02:19:38.127 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:37 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:37.842+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:49.192960+0000 front 2024-09-12T02:18:49.193116+0000 (oldest deadline 2024-09-12T02:19:15.092803+0000) 2024-09-12T02:19:38.127 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:37 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:37.842+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:58.593805+0000 front 2024-09-12T02:18:58.593869+0000 (oldest deadline 2024-09-12T02:19:19.594390+0000) 2024-09-12T02:19:38.128 INFO:journalctl@ceph.osd.8.smithi184.stdout:Sep 12 02:19:37 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-8[54277]: 2024-09-12T02:19:37.648+0000 7fe987aba640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:49.946694+0000 front 2024-09-12T02:18:49.946586+0000 (oldest deadline 2024-09-12T02:19:15.846545+0000) 2024-09-12T02:19:38.128 INFO:journalctl@ceph.osd.8.smithi184.stdout:Sep 12 02:19:37 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-8[54277]: 2024-09-12T02:19:37.648+0000 7fe987aba640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:57.547401+0000 front 2024-09-12T02:18:57.547434+0000 (oldest deadline 2024-09-12T02:19:22.847519+0000) 2024-09-12T02:19:38.128 INFO:journalctl@ceph.osd.8.smithi184.stdout:Sep 12 02:19:37 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-8[54277]: 2024-09-12T02:19:37.648+0000 7fe987aba640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T02:19:03.348610+0000 front 2024-09-12T02:19:03.348558+0000 (oldest deadline 2024-09-12T02:19:28.648626+0000) 2024-09-12T02:19:38.274 INFO:journalctl@ceph.osd.7.smithi033.stdout:Sep 12 02:19:38 smithi033 podman[70737]: 2024-09-12 02:19:38.010672549 +0000 UTC m=+5.488524139 container died 6f36b51894a97f5c2d11a9e02a5e04a21277299d2e10ced6a4b182bab842000e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-7, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, RELEASE=squid-b53b6d3, ceph=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-09-12T02:19:38.274 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:38 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:38.224+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:42.394291+0000 front 2024-09-12T02:18:42.394247+0000 (oldest deadline 2024-09-12T02:19:05.894091+0000) 2024-09-12T02:19:38.274 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:38 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:38.224+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:47.595046+0000 front 2024-09-12T02:18:47.595165+0000 (oldest deadline 2024-09-12T02:19:12.295142+0000) 2024-09-12T02:19:38.274 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:38 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:38.224+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:58.696957+0000 front 2024-09-12T02:18:58.696791+0000 (oldest deadline 2024-09-12T02:19:22.196628+0000) 2024-09-12T02:19:38.274 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:38 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:38.224+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T02:19:05.097647+0000 front 2024-09-12T02:19:05.097663+0000 (oldest deadline 2024-09-12T02:19:28.498137+0000) 2024-09-12T02:19:38.274 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:38 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:38.224+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.33:6812 osd.4 since back 2024-09-12T02:19:11.899193+0000 front 2024-09-12T02:19:11.899218+0000 (oldest deadline 2024-09-12T02:19:35.999293+0000) 2024-09-12T02:19:38.632 INFO:journalctl@ceph.osd.7.smithi033.stdout:Sep 12 02:19:38 smithi033 podman[70737]: 2024-09-12 02:19:38.364811198 +0000 UTC m=+5.842662779 container cleanup 6f36b51894a97f5c2d11a9e02a5e04a21277299d2e10ced6a4b182bab842000e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-7, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731) 2024-09-12T02:19:38.632 INFO:journalctl@ceph.osd.7.smithi033.stdout:Sep 12 02:19:38 smithi033 bash[70737]: ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-7 2024-09-12T02:19:38.683 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:38 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:38.268+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:38.124303+0000 front 2024-09-12T02:18:38.124204+0000 (oldest deadline 2024-09-12T02:19:04.024431+0000) 2024-09-12T02:19:38.684 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:38 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:38.268+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:49.925357+0000 front 2024-09-12T02:18:49.925414+0000 (oldest deadline 2024-09-12T02:19:14.625383+0000) 2024-09-12T02:19:38.684 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:38 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:38.268+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:58.726706+0000 front 2024-09-12T02:18:58.726591+0000 (oldest deadline 2024-09-12T02:19:22.226596+0000) 2024-09-12T02:19:38.684 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:38 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:38.268+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T02:19:05.627979+0000 front 2024-09-12T02:19:05.627999+0000 (oldest deadline 2024-09-12T02:19:29.128128+0000) 2024-09-12T02:19:38.684 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:38 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:38.268+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.33:6812 osd.4 since back 2024-09-12T02:19:13.228970+0000 front 2024-09-12T02:19:13.229156+0000 (oldest deadline 2024-09-12T02:19:37.928950+0000) 2024-09-12T02:19:38.919 INFO:journalctl@ceph.osd.7.smithi033.stdout:Sep 12 02:19:38 smithi033 podman[70766]: 2024-09-12 02:19:38.632561365 +0000 UTC m=+0.614474733 container remove 6f36b51894a97f5c2d11a9e02a5e04a21277299d2e10ced6a4b182bab842000e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-7, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T02:19:39.127 INFO:journalctl@ceph.osd.8.smithi184.stdout:Sep 12 02:19:38 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-8[54277]: 2024-09-12T02:19:38.665+0000 7fe987aba640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:49.946694+0000 front 2024-09-12T02:18:49.946586+0000 (oldest deadline 2024-09-12T02:19:15.846545+0000) 2024-09-12T02:19:39.127 INFO:journalctl@ceph.osd.8.smithi184.stdout:Sep 12 02:19:38 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-8[54277]: 2024-09-12T02:19:38.665+0000 7fe987aba640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:57.547401+0000 front 2024-09-12T02:18:57.547434+0000 (oldest deadline 2024-09-12T02:19:22.847519+0000) 2024-09-12T02:19:39.127 INFO:journalctl@ceph.osd.8.smithi184.stdout:Sep 12 02:19:38 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-8[54277]: 2024-09-12T02:19:38.665+0000 7fe987aba640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T02:19:03.348610+0000 front 2024-09-12T02:19:03.348558+0000 (oldest deadline 2024-09-12T02:19:28.648626+0000) 2024-09-12T02:19:39.128 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:38 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:38.816+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:40.491334+0000 front 2024-09-12T02:18:40.491223+0000 (oldest deadline 2024-09-12T02:19:05.191278+0000) 2024-09-12T02:19:39.128 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:38 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:38.816+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:49.192960+0000 front 2024-09-12T02:18:49.193116+0000 (oldest deadline 2024-09-12T02:19:15.092803+0000) 2024-09-12T02:19:39.128 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:38 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:38.816+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:58.593805+0000 front 2024-09-12T02:18:58.593869+0000 (oldest deadline 2024-09-12T02:19:19.594390+0000) 2024-09-12T02:19:39.128 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:38 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:38.816+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.33:6812 osd.4 since back 2024-09-12T02:19:14.097458+0000 front 2024-09-12T02:19:14.097525+0000 (oldest deadline 2024-09-12T02:19:38.197566+0000) 2024-09-12T02:19:39.266 INFO:journalctl@ceph.osd.7.smithi033.stdout:Sep 12 02:19:38 smithi033 podman[70852]: 2024-09-12 02:19:38.898106289 +0000 UTC m=+0.016103920 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T02:19:39.266 INFO:journalctl@ceph.osd.7.smithi033.stdout:Sep 12 02:19:39 smithi033 podman[70852]: 2024-09-12 02:19:39.016897859 +0000 UTC m=+0.134895469 container create de2343bda966976a085297af58ab6e17b0638e002d06dd0952dbdfefb67a702c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-7-deactivate, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, ceph=True, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2) 2024-09-12T02:19:39.669 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:39 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:39.266+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:42.394291+0000 front 2024-09-12T02:18:42.394247+0000 (oldest deadline 2024-09-12T02:19:05.894091+0000) 2024-09-12T02:19:39.669 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:39 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:39.266+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:47.595046+0000 front 2024-09-12T02:18:47.595165+0000 (oldest deadline 2024-09-12T02:19:12.295142+0000) 2024-09-12T02:19:39.669 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:39 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:39.266+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:58.696957+0000 front 2024-09-12T02:18:58.696791+0000 (oldest deadline 2024-09-12T02:19:22.196628+0000) 2024-09-12T02:19:39.670 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:39 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:39.266+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T02:19:05.097647+0000 front 2024-09-12T02:19:05.097663+0000 (oldest deadline 2024-09-12T02:19:28.498137+0000) 2024-09-12T02:19:39.670 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:39 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:39.266+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.33:6812 osd.4 since back 2024-09-12T02:19:11.899193+0000 front 2024-09-12T02:19:11.899218+0000 (oldest deadline 2024-09-12T02:19:35.999293+0000) 2024-09-12T02:19:39.670 INFO:journalctl@ceph.osd.7.smithi033.stdout:Sep 12 02:19:39 smithi033 podman[70852]: 2024-09-12 02:19:39.269298998 +0000 UTC m=+0.387296610 container init de2343bda966976a085297af58ab6e17b0638e002d06dd0952dbdfefb67a702c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-7-deactivate, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.37.2) 2024-09-12T02:19:39.670 INFO:journalctl@ceph.osd.7.smithi033.stdout:Sep 12 02:19:39 smithi033 podman[70852]: 2024-09-12 02:19:39.272569368 +0000 UTC m=+0.390566996 container start de2343bda966976a085297af58ab6e17b0638e002d06dd0952dbdfefb67a702c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-7-deactivate, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, ceph=True, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0) 2024-09-12T02:19:39.670 INFO:journalctl@ceph.osd.7.smithi033.stdout:Sep 12 02:19:39 smithi033 podman[70852]: 2024-09-12 02:19:39.361065266 +0000 UTC m=+0.479062885 container attach de2343bda966976a085297af58ab6e17b0638e002d06dd0952dbdfefb67a702c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-7-deactivate, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T02:19:39.670 INFO:journalctl@ceph.osd.7.smithi033.stdout:Sep 12 02:19:39 smithi033 podman[70852]: 2024-09-12 02:19:39.468574663 +0000 UTC m=+0.586572290 container died de2343bda966976a085297af58ab6e17b0638e002d06dd0952dbdfefb67a702c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-7-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-09-12T02:19:39.683 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:39 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:39.241+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:38.124303+0000 front 2024-09-12T02:18:38.124204+0000 (oldest deadline 2024-09-12T02:19:04.024431+0000) 2024-09-12T02:19:39.683 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:39 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:39.241+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:49.925357+0000 front 2024-09-12T02:18:49.925414+0000 (oldest deadline 2024-09-12T02:19:14.625383+0000) 2024-09-12T02:19:39.683 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:39 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:39.241+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:58.726706+0000 front 2024-09-12T02:18:58.726591+0000 (oldest deadline 2024-09-12T02:19:22.226596+0000) 2024-09-12T02:19:39.683 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:39 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:39.241+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T02:19:05.627979+0000 front 2024-09-12T02:19:05.627999+0000 (oldest deadline 2024-09-12T02:19:29.128128+0000) 2024-09-12T02:19:39.684 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:39 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:39.241+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.33:6812 osd.4 since back 2024-09-12T02:19:13.228970+0000 front 2024-09-12T02:19:13.229156+0000 (oldest deadline 2024-09-12T02:19:37.928950+0000) 2024-09-12T02:19:39.991 DEBUG:teuthology.orchestra.run.smithi033:> sudo pkill -f 'journalctl -f -n 0 -u ceph-879f3b56-70a9-11ef-bcea-c7b262605968@osd.7.service' 2024-09-12T02:19:40.015 INFO:journalctl@ceph.osd.7.smithi033.stdout:Sep 12 02:19:39 smithi033 podman[70875]: 2024-09-12 02:19:39.972584601 +0000 UTC m=+0.498919189 container remove de2343bda966976a085297af58ab6e17b0638e002d06dd0952dbdfefb67a702c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-7-deactivate, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T02:19:40.015 INFO:journalctl@ceph.osd.7.smithi033.stdout:Sep 12 02:19:39 smithi033 systemd[1]: ceph-879f3b56-70a9-11ef-bcea-c7b262605968@osd.7.service: Deactivated successfully. 2024-09-12T02:19:40.015 INFO:journalctl@ceph.osd.7.smithi033.stdout:Sep 12 02:19:39 smithi033 systemd[1]: Stopped Ceph osd.7 for 879f3b56-70a9-11ef-bcea-c7b262605968. 2024-09-12T02:19:40.015 INFO:journalctl@ceph.osd.7.smithi033.stdout:Sep 12 02:19:39 smithi033 systemd[1]: ceph-879f3b56-70a9-11ef-bcea-c7b262605968@osd.7.service: Consumed 12.170s CPU time. 2024-09-12T02:19:40.033 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T02:19:40.033 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-09-12T02:19:40.033 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-09-12T02:19:40.033 DEBUG:teuthology.orchestra.run.smithi184:> sudo systemctl stop ceph-879f3b56-70a9-11ef-bcea-c7b262605968@osd.8 2024-09-12T02:19:40.059 INFO:journalctl@ceph.osd.8.smithi184.stdout:Sep 12 02:19:39 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-8[54277]: 2024-09-12T02:19:39.643+0000 7fe987aba640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:49.946694+0000 front 2024-09-12T02:18:49.946586+0000 (oldest deadline 2024-09-12T02:19:15.846545+0000) 2024-09-12T02:19:40.060 INFO:journalctl@ceph.osd.8.smithi184.stdout:Sep 12 02:19:39 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-8[54277]: 2024-09-12T02:19:39.643+0000 7fe987aba640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:57.547401+0000 front 2024-09-12T02:18:57.547434+0000 (oldest deadline 2024-09-12T02:19:22.847519+0000) 2024-09-12T02:19:40.060 INFO:journalctl@ceph.osd.8.smithi184.stdout:Sep 12 02:19:39 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-8[54277]: 2024-09-12T02:19:39.643+0000 7fe987aba640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T02:19:03.348610+0000 front 2024-09-12T02:19:03.348558+0000 (oldest deadline 2024-09-12T02:19:28.648626+0000) 2024-09-12T02:19:40.060 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:39 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:39.805+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:40.491334+0000 front 2024-09-12T02:18:40.491223+0000 (oldest deadline 2024-09-12T02:19:05.191278+0000) 2024-09-12T02:19:40.060 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:39 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:39.805+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:49.192960+0000 front 2024-09-12T02:18:49.193116+0000 (oldest deadline 2024-09-12T02:19:15.092803+0000) 2024-09-12T02:19:40.060 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:39 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:39.805+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:58.593805+0000 front 2024-09-12T02:18:58.593869+0000 (oldest deadline 2024-09-12T02:19:19.594390+0000) 2024-09-12T02:19:40.060 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:39 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:39.805+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.33:6812 osd.4 since back 2024-09-12T02:19:14.097458+0000 front 2024-09-12T02:19:14.097525+0000 (oldest deadline 2024-09-12T02:19:38.197566+0000) 2024-09-12T02:19:40.315 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:40 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:40.313+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:42.394291+0000 front 2024-09-12T02:18:42.394247+0000 (oldest deadline 2024-09-12T02:19:05.894091+0000) 2024-09-12T02:19:40.315 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:40 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:40.313+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:47.595046+0000 front 2024-09-12T02:18:47.595165+0000 (oldest deadline 2024-09-12T02:19:12.295142+0000) 2024-09-12T02:19:40.315 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:40 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:40.313+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:58.696957+0000 front 2024-09-12T02:18:58.696791+0000 (oldest deadline 2024-09-12T02:19:22.196628+0000) 2024-09-12T02:19:40.315 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:40 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:40.313+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T02:19:05.097647+0000 front 2024-09-12T02:19:05.097663+0000 (oldest deadline 2024-09-12T02:19:28.498137+0000) 2024-09-12T02:19:40.315 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:40 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:40.313+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.33:6812 osd.4 since back 2024-09-12T02:19:11.899193+0000 front 2024-09-12T02:19:11.899218+0000 (oldest deadline 2024-09-12T02:19:35.999293+0000) 2024-09-12T02:19:40.335 INFO:journalctl@ceph.osd.8.smithi184.stdout:Sep 12 02:19:40 smithi184 systemd[1]: Stopping Ceph osd.8 for 879f3b56-70a9-11ef-bcea-c7b262605968... 2024-09-12T02:19:40.600 INFO:journalctl@ceph.osd.8.smithi184.stdout:Sep 12 02:19:40 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-8[54277]: 2024-09-12T02:19:40.348+0000 7fe98b4b5640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T02:19:40.600 INFO:journalctl@ceph.osd.8.smithi184.stdout:Sep 12 02:19:40 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-8[54277]: 2024-09-12T02:19:40.348+0000 7fe98b4b5640 -1 osd.8 85 *** Got signal Terminated *** 2024-09-12T02:19:40.601 INFO:journalctl@ceph.osd.8.smithi184.stdout:Sep 12 02:19:40 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-8[54277]: 2024-09-12T02:19:40.348+0000 7fe98b4b5640 -1 osd.8 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T02:19:40.601 INFO:journalctl@ceph.osd.8.smithi184.stdout:Sep 12 02:19:40 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-8[54277]: 2024-09-12T02:19:40.600+0000 7fe987aba640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:49.946694+0000 front 2024-09-12T02:18:49.946586+0000 (oldest deadline 2024-09-12T02:19:15.846545+0000) 2024-09-12T02:19:40.601 INFO:journalctl@ceph.osd.8.smithi184.stdout:Sep 12 02:19:40 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-8[54277]: 2024-09-12T02:19:40.600+0000 7fe987aba640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:57.547401+0000 front 2024-09-12T02:18:57.547434+0000 (oldest deadline 2024-09-12T02:19:22.847519+0000) 2024-09-12T02:19:40.601 INFO:journalctl@ceph.osd.8.smithi184.stdout:Sep 12 02:19:40 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-8[54277]: 2024-09-12T02:19:40.600+0000 7fe987aba640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T02:19:03.348610+0000 front 2024-09-12T02:19:03.348558+0000 (oldest deadline 2024-09-12T02:19:28.648626+0000) 2024-09-12T02:19:40.601 INFO:journalctl@ceph.osd.8.smithi184.stdout:Sep 12 02:19:40 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-8[54277]: 2024-09-12T02:19:40.600+0000 7fe987aba640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.33:6812 osd.4 since back 2024-09-12T02:19:08.649225+0000 front 2024-09-12T02:19:08.649076+0000 (oldest deadline 2024-09-12T02:19:40.449553+0000) 2024-09-12T02:19:40.684 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:40 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:40.282+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:38.124303+0000 front 2024-09-12T02:18:38.124204+0000 (oldest deadline 2024-09-12T02:19:04.024431+0000) 2024-09-12T02:19:40.684 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:40 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:40.282+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:49.925357+0000 front 2024-09-12T02:18:49.925414+0000 (oldest deadline 2024-09-12T02:19:14.625383+0000) 2024-09-12T02:19:40.684 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:40 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:40.282+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:58.726706+0000 front 2024-09-12T02:18:58.726591+0000 (oldest deadline 2024-09-12T02:19:22.226596+0000) 2024-09-12T02:19:40.684 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:40 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:40.282+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T02:19:05.627979+0000 front 2024-09-12T02:19:05.627999+0000 (oldest deadline 2024-09-12T02:19:29.128128+0000) 2024-09-12T02:19:40.684 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:40 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:40.282+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.33:6812 osd.4 since back 2024-09-12T02:19:13.228970+0000 front 2024-09-12T02:19:13.229156+0000 (oldest deadline 2024-09-12T02:19:37.928950+0000) 2024-09-12T02:19:40.877 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:40 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:40.814+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:40.491334+0000 front 2024-09-12T02:18:40.491223+0000 (oldest deadline 2024-09-12T02:19:05.191278+0000) 2024-09-12T02:19:40.878 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:40 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:40.814+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:49.192960+0000 front 2024-09-12T02:18:49.193116+0000 (oldest deadline 2024-09-12T02:19:15.092803+0000) 2024-09-12T02:19:40.878 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:40 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:40.814+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:58.593805+0000 front 2024-09-12T02:18:58.593869+0000 (oldest deadline 2024-09-12T02:19:19.594390+0000) 2024-09-12T02:19:40.878 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:40 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:40.814+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.33:6812 osd.4 since back 2024-09-12T02:19:14.097458+0000 front 2024-09-12T02:19:14.097525+0000 (oldest deadline 2024-09-12T02:19:38.197566+0000) 2024-09-12T02:19:41.669 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:41 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:41.341+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:42.394291+0000 front 2024-09-12T02:18:42.394247+0000 (oldest deadline 2024-09-12T02:19:05.894091+0000) 2024-09-12T02:19:41.669 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:41 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:41.341+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:47.595046+0000 front 2024-09-12T02:18:47.595165+0000 (oldest deadline 2024-09-12T02:19:12.295142+0000) 2024-09-12T02:19:41.669 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:41 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:41.341+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:58.696957+0000 front 2024-09-12T02:18:58.696791+0000 (oldest deadline 2024-09-12T02:19:22.196628+0000) 2024-09-12T02:19:41.669 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:41 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:41.341+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T02:19:05.097647+0000 front 2024-09-12T02:19:05.097663+0000 (oldest deadline 2024-09-12T02:19:28.498137+0000) 2024-09-12T02:19:41.670 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:41 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:41.341+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.33:6812 osd.4 since back 2024-09-12T02:19:11.899193+0000 front 2024-09-12T02:19:11.899218+0000 (oldest deadline 2024-09-12T02:19:35.999293+0000) 2024-09-12T02:19:41.683 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:41 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:41.258+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:38.124303+0000 front 2024-09-12T02:18:38.124204+0000 (oldest deadline 2024-09-12T02:19:04.024431+0000) 2024-09-12T02:19:41.683 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:41 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:41.258+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:49.925357+0000 front 2024-09-12T02:18:49.925414+0000 (oldest deadline 2024-09-12T02:19:14.625383+0000) 2024-09-12T02:19:41.683 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:41 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:41.258+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:58.726706+0000 front 2024-09-12T02:18:58.726591+0000 (oldest deadline 2024-09-12T02:19:22.226596+0000) 2024-09-12T02:19:41.684 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:41 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:41.258+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T02:19:05.627979+0000 front 2024-09-12T02:19:05.627999+0000 (oldest deadline 2024-09-12T02:19:29.128128+0000) 2024-09-12T02:19:41.684 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:41 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:41.258+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.33:6812 osd.4 since back 2024-09-12T02:19:13.228970+0000 front 2024-09-12T02:19:13.229156+0000 (oldest deadline 2024-09-12T02:19:37.928950+0000) 2024-09-12T02:19:41.877 INFO:journalctl@ceph.osd.8.smithi184.stdout:Sep 12 02:19:41 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-8[54277]: 2024-09-12T02:19:41.623+0000 7fe987aba640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:49.946694+0000 front 2024-09-12T02:18:49.946586+0000 (oldest deadline 2024-09-12T02:19:15.846545+0000) 2024-09-12T02:19:41.877 INFO:journalctl@ceph.osd.8.smithi184.stdout:Sep 12 02:19:41 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-8[54277]: 2024-09-12T02:19:41.623+0000 7fe987aba640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:57.547401+0000 front 2024-09-12T02:18:57.547434+0000 (oldest deadline 2024-09-12T02:19:22.847519+0000) 2024-09-12T02:19:41.877 INFO:journalctl@ceph.osd.8.smithi184.stdout:Sep 12 02:19:41 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-8[54277]: 2024-09-12T02:19:41.623+0000 7fe987aba640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T02:19:03.348610+0000 front 2024-09-12T02:19:03.348558+0000 (oldest deadline 2024-09-12T02:19:28.648626+0000) 2024-09-12T02:19:41.878 INFO:journalctl@ceph.osd.8.smithi184.stdout:Sep 12 02:19:41 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-8[54277]: 2024-09-12T02:19:41.623+0000 7fe987aba640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.33:6812 osd.4 since back 2024-09-12T02:19:08.649225+0000 front 2024-09-12T02:19:08.649076+0000 (oldest deadline 2024-09-12T02:19:40.449553+0000) 2024-09-12T02:19:41.878 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:41 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:41.848+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:40.491334+0000 front 2024-09-12T02:18:40.491223+0000 (oldest deadline 2024-09-12T02:19:05.191278+0000) 2024-09-12T02:19:41.878 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:41 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:41.848+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:49.192960+0000 front 2024-09-12T02:18:49.193116+0000 (oldest deadline 2024-09-12T02:19:15.092803+0000) 2024-09-12T02:19:41.878 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:41 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:41.848+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:58.593805+0000 front 2024-09-12T02:18:58.593869+0000 (oldest deadline 2024-09-12T02:19:19.594390+0000) 2024-09-12T02:19:41.878 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:41 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:41.848+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.33:6812 osd.4 since back 2024-09-12T02:19:14.097458+0000 front 2024-09-12T02:19:14.097525+0000 (oldest deadline 2024-09-12T02:19:38.197566+0000) 2024-09-12T02:19:42.669 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:42 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:42.314+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:42.394291+0000 front 2024-09-12T02:18:42.394247+0000 (oldest deadline 2024-09-12T02:19:05.894091+0000) 2024-09-12T02:19:42.669 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:42 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:42.314+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:47.595046+0000 front 2024-09-12T02:18:47.595165+0000 (oldest deadline 2024-09-12T02:19:12.295142+0000) 2024-09-12T02:19:42.669 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:42 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:42.314+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:58.696957+0000 front 2024-09-12T02:18:58.696791+0000 (oldest deadline 2024-09-12T02:19:22.196628+0000) 2024-09-12T02:19:42.670 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:42 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:42.314+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T02:19:05.097647+0000 front 2024-09-12T02:19:05.097663+0000 (oldest deadline 2024-09-12T02:19:28.498137+0000) 2024-09-12T02:19:42.670 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:42 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:42.314+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.33:6812 osd.4 since back 2024-09-12T02:19:11.899193+0000 front 2024-09-12T02:19:11.899218+0000 (oldest deadline 2024-09-12T02:19:35.999293+0000) 2024-09-12T02:19:42.683 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:42 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:42.217+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:38.124303+0000 front 2024-09-12T02:18:38.124204+0000 (oldest deadline 2024-09-12T02:19:04.024431+0000) 2024-09-12T02:19:42.683 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:42 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:42.217+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:49.925357+0000 front 2024-09-12T02:18:49.925414+0000 (oldest deadline 2024-09-12T02:19:14.625383+0000) 2024-09-12T02:19:42.683 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:42 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:42.217+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:58.726706+0000 front 2024-09-12T02:18:58.726591+0000 (oldest deadline 2024-09-12T02:19:22.226596+0000) 2024-09-12T02:19:42.684 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:42 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:42.217+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T02:19:05.627979+0000 front 2024-09-12T02:19:05.627999+0000 (oldest deadline 2024-09-12T02:19:29.128128+0000) 2024-09-12T02:19:42.684 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:42 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:42.217+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.33:6812 osd.4 since back 2024-09-12T02:19:13.228970+0000 front 2024-09-12T02:19:13.229156+0000 (oldest deadline 2024-09-12T02:19:37.928950+0000) 2024-09-12T02:19:42.877 INFO:journalctl@ceph.osd.8.smithi184.stdout:Sep 12 02:19:42 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-8[54277]: 2024-09-12T02:19:42.619+0000 7fe987aba640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:49.946694+0000 front 2024-09-12T02:18:49.946586+0000 (oldest deadline 2024-09-12T02:19:15.846545+0000) 2024-09-12T02:19:42.877 INFO:journalctl@ceph.osd.8.smithi184.stdout:Sep 12 02:19:42 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-8[54277]: 2024-09-12T02:19:42.619+0000 7fe987aba640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:57.547401+0000 front 2024-09-12T02:18:57.547434+0000 (oldest deadline 2024-09-12T02:19:22.847519+0000) 2024-09-12T02:19:42.878 INFO:journalctl@ceph.osd.8.smithi184.stdout:Sep 12 02:19:42 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-8[54277]: 2024-09-12T02:19:42.619+0000 7fe987aba640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T02:19:03.348610+0000 front 2024-09-12T02:19:03.348558+0000 (oldest deadline 2024-09-12T02:19:28.648626+0000) 2024-09-12T02:19:42.878 INFO:journalctl@ceph.osd.8.smithi184.stdout:Sep 12 02:19:42 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-8[54277]: 2024-09-12T02:19:42.619+0000 7fe987aba640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.33:6812 osd.4 since back 2024-09-12T02:19:08.649225+0000 front 2024-09-12T02:19:08.649076+0000 (oldest deadline 2024-09-12T02:19:40.449553+0000) 2024-09-12T02:19:43.377 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:42 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:42.878+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:40.491334+0000 front 2024-09-12T02:18:40.491223+0000 (oldest deadline 2024-09-12T02:19:05.191278+0000) 2024-09-12T02:19:43.378 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:42 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:42.878+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:49.192960+0000 front 2024-09-12T02:18:49.193116+0000 (oldest deadline 2024-09-12T02:19:15.092803+0000) 2024-09-12T02:19:43.378 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:42 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:42.878+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:58.593805+0000 front 2024-09-12T02:18:58.593869+0000 (oldest deadline 2024-09-12T02:19:19.594390+0000) 2024-09-12T02:19:43.378 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:42 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:42.878+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.33:6812 osd.4 since back 2024-09-12T02:19:14.097458+0000 front 2024-09-12T02:19:14.097525+0000 (oldest deadline 2024-09-12T02:19:38.197566+0000) 2024-09-12T02:19:43.433 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:43 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:43.229+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:38.124303+0000 front 2024-09-12T02:18:38.124204+0000 (oldest deadline 2024-09-12T02:19:04.024431+0000) 2024-09-12T02:19:43.434 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:43 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:43.229+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:49.925357+0000 front 2024-09-12T02:18:49.925414+0000 (oldest deadline 2024-09-12T02:19:14.625383+0000) 2024-09-12T02:19:43.434 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:43 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:43.229+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:58.726706+0000 front 2024-09-12T02:18:58.726591+0000 (oldest deadline 2024-09-12T02:19:22.226596+0000) 2024-09-12T02:19:43.434 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:43 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:43.229+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T02:19:05.627979+0000 front 2024-09-12T02:19:05.627999+0000 (oldest deadline 2024-09-12T02:19:29.128128+0000) 2024-09-12T02:19:43.434 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:43 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:43.229+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.33:6812 osd.4 since back 2024-09-12T02:19:13.228970+0000 front 2024-09-12T02:19:13.229156+0000 (oldest deadline 2024-09-12T02:19:37.928950+0000) 2024-09-12T02:19:43.669 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:43 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:43.294+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:42.394291+0000 front 2024-09-12T02:18:42.394247+0000 (oldest deadline 2024-09-12T02:19:05.894091+0000) 2024-09-12T02:19:43.669 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:43 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:43.294+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:47.595046+0000 front 2024-09-12T02:18:47.595165+0000 (oldest deadline 2024-09-12T02:19:12.295142+0000) 2024-09-12T02:19:43.669 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:43 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:43.294+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:58.696957+0000 front 2024-09-12T02:18:58.696791+0000 (oldest deadline 2024-09-12T02:19:22.196628+0000) 2024-09-12T02:19:43.669 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:43 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:43.294+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T02:19:05.097647+0000 front 2024-09-12T02:19:05.097663+0000 (oldest deadline 2024-09-12T02:19:28.498137+0000) 2024-09-12T02:19:43.669 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:43 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:43.294+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.33:6812 osd.4 since back 2024-09-12T02:19:11.899193+0000 front 2024-09-12T02:19:11.899218+0000 (oldest deadline 2024-09-12T02:19:35.999293+0000) 2024-09-12T02:19:43.877 INFO:journalctl@ceph.osd.8.smithi184.stdout:Sep 12 02:19:43 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-8[54277]: 2024-09-12T02:19:43.591+0000 7fe987aba640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:49.946694+0000 front 2024-09-12T02:18:49.946586+0000 (oldest deadline 2024-09-12T02:19:15.846545+0000) 2024-09-12T02:19:43.878 INFO:journalctl@ceph.osd.8.smithi184.stdout:Sep 12 02:19:43 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-8[54277]: 2024-09-12T02:19:43.591+0000 7fe987aba640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:57.547401+0000 front 2024-09-12T02:18:57.547434+0000 (oldest deadline 2024-09-12T02:19:22.847519+0000) 2024-09-12T02:19:43.878 INFO:journalctl@ceph.osd.8.smithi184.stdout:Sep 12 02:19:43 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-8[54277]: 2024-09-12T02:19:43.591+0000 7fe987aba640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T02:19:03.348610+0000 front 2024-09-12T02:19:03.348558+0000 (oldest deadline 2024-09-12T02:19:28.648626+0000) 2024-09-12T02:19:43.878 INFO:journalctl@ceph.osd.8.smithi184.stdout:Sep 12 02:19:43 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-8[54277]: 2024-09-12T02:19:43.591+0000 7fe987aba640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.33:6812 osd.4 since back 2024-09-12T02:19:08.649225+0000 front 2024-09-12T02:19:08.649076+0000 (oldest deadline 2024-09-12T02:19:40.449553+0000) 2024-09-12T02:19:43.878 INFO:journalctl@ceph.osd.8.smithi184.stdout:Sep 12 02:19:43 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-8[54277]: 2024-09-12T02:19:43.591+0000 7fe987aba640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-09-12T02:19:20.449937+0000 front 2024-09-12T02:19:20.449999+0000 (oldest deadline 2024-09-12T02:19:42.750048+0000) 2024-09-12T02:19:44.377 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:43 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:43.913+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:40.491334+0000 front 2024-09-12T02:18:40.491223+0000 (oldest deadline 2024-09-12T02:19:05.191278+0000) 2024-09-12T02:19:44.378 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:43 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:43.913+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:49.192960+0000 front 2024-09-12T02:18:49.193116+0000 (oldest deadline 2024-09-12T02:19:15.092803+0000) 2024-09-12T02:19:44.378 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:43 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:43.913+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:58.593805+0000 front 2024-09-12T02:18:58.593869+0000 (oldest deadline 2024-09-12T02:19:19.594390+0000) 2024-09-12T02:19:44.378 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:43 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:43.913+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.33:6812 osd.4 since back 2024-09-12T02:19:14.097458+0000 front 2024-09-12T02:19:14.097525+0000 (oldest deadline 2024-09-12T02:19:38.197566+0000) 2024-09-12T02:19:44.378 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:43 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:43.913+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-09-12T02:19:18.197917+0000 front 2024-09-12T02:19:18.198027+0000 (oldest deadline 2024-09-12T02:19:43.498195+0000) 2024-09-12T02:19:44.669 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:44 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:44.265+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:42.394291+0000 front 2024-09-12T02:18:42.394247+0000 (oldest deadline 2024-09-12T02:19:05.894091+0000) 2024-09-12T02:19:44.669 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:44 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:44.265+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:47.595046+0000 front 2024-09-12T02:18:47.595165+0000 (oldest deadline 2024-09-12T02:19:12.295142+0000) 2024-09-12T02:19:44.670 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:44 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:44.265+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:58.696957+0000 front 2024-09-12T02:18:58.696791+0000 (oldest deadline 2024-09-12T02:19:22.196628+0000) 2024-09-12T02:19:44.670 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:44 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:44.265+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T02:19:05.097647+0000 front 2024-09-12T02:19:05.097663+0000 (oldest deadline 2024-09-12T02:19:28.498137+0000) 2024-09-12T02:19:44.670 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:44 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:44.265+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.33:6812 osd.4 since back 2024-09-12T02:19:11.899193+0000 front 2024-09-12T02:19:11.899218+0000 (oldest deadline 2024-09-12T02:19:35.999293+0000) 2024-09-12T02:19:44.683 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:44 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:44.190+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:38.124303+0000 front 2024-09-12T02:18:38.124204+0000 (oldest deadline 2024-09-12T02:19:04.024431+0000) 2024-09-12T02:19:44.684 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:44 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:44.190+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:49.925357+0000 front 2024-09-12T02:18:49.925414+0000 (oldest deadline 2024-09-12T02:19:14.625383+0000) 2024-09-12T02:19:44.684 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:44 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:44.190+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:58.726706+0000 front 2024-09-12T02:18:58.726591+0000 (oldest deadline 2024-09-12T02:19:22.226596+0000) 2024-09-12T02:19:44.684 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:44 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:44.190+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T02:19:05.627979+0000 front 2024-09-12T02:19:05.627999+0000 (oldest deadline 2024-09-12T02:19:29.128128+0000) 2024-09-12T02:19:44.684 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:44 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:44.190+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.33:6812 osd.4 since back 2024-09-12T02:19:13.228970+0000 front 2024-09-12T02:19:13.229156+0000 (oldest deadline 2024-09-12T02:19:37.928950+0000) 2024-09-12T02:19:44.877 INFO:journalctl@ceph.osd.8.smithi184.stdout:Sep 12 02:19:44 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-8[54277]: 2024-09-12T02:19:44.597+0000 7fe987aba640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:49.946694+0000 front 2024-09-12T02:18:49.946586+0000 (oldest deadline 2024-09-12T02:19:15.846545+0000) 2024-09-12T02:19:44.878 INFO:journalctl@ceph.osd.8.smithi184.stdout:Sep 12 02:19:44 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-8[54277]: 2024-09-12T02:19:44.597+0000 7fe987aba640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:57.547401+0000 front 2024-09-12T02:18:57.547434+0000 (oldest deadline 2024-09-12T02:19:22.847519+0000) 2024-09-12T02:19:44.878 INFO:journalctl@ceph.osd.8.smithi184.stdout:Sep 12 02:19:44 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-8[54277]: 2024-09-12T02:19:44.597+0000 7fe987aba640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T02:19:03.348610+0000 front 2024-09-12T02:19:03.348558+0000 (oldest deadline 2024-09-12T02:19:28.648626+0000) 2024-09-12T02:19:44.878 INFO:journalctl@ceph.osd.8.smithi184.stdout:Sep 12 02:19:44 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-8[54277]: 2024-09-12T02:19:44.597+0000 7fe987aba640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.33:6812 osd.4 since back 2024-09-12T02:19:08.649225+0000 front 2024-09-12T02:19:08.649076+0000 (oldest deadline 2024-09-12T02:19:40.449553+0000) 2024-09-12T02:19:44.878 INFO:journalctl@ceph.osd.8.smithi184.stdout:Sep 12 02:19:44 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-8[54277]: 2024-09-12T02:19:44.597+0000 7fe987aba640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-09-12T02:19:20.449937+0000 front 2024-09-12T02:19:20.449999+0000 (oldest deadline 2024-09-12T02:19:42.750048+0000) 2024-09-12T02:19:45.377 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:44 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:44.909+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:40.491334+0000 front 2024-09-12T02:18:40.491223+0000 (oldest deadline 2024-09-12T02:19:05.191278+0000) 2024-09-12T02:19:45.377 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:44 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:44.909+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:49.192960+0000 front 2024-09-12T02:18:49.193116+0000 (oldest deadline 2024-09-12T02:19:15.092803+0000) 2024-09-12T02:19:45.377 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:44 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:44.909+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:58.593805+0000 front 2024-09-12T02:18:58.593869+0000 (oldest deadline 2024-09-12T02:19:19.594390+0000) 2024-09-12T02:19:45.378 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:44 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:44.909+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.33:6812 osd.4 since back 2024-09-12T02:19:14.097458+0000 front 2024-09-12T02:19:14.097525+0000 (oldest deadline 2024-09-12T02:19:38.197566+0000) 2024-09-12T02:19:45.378 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:44 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:44.909+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-09-12T02:19:18.197917+0000 front 2024-09-12T02:19:18.198027+0000 (oldest deadline 2024-09-12T02:19:43.498195+0000) 2024-09-12T02:19:45.669 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:45 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:45.313+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:42.394291+0000 front 2024-09-12T02:18:42.394247+0000 (oldest deadline 2024-09-12T02:19:05.894091+0000) 2024-09-12T02:19:45.669 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:45 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:45.313+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:47.595046+0000 front 2024-09-12T02:18:47.595165+0000 (oldest deadline 2024-09-12T02:19:12.295142+0000) 2024-09-12T02:19:45.669 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:45 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:45.313+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:58.696957+0000 front 2024-09-12T02:18:58.696791+0000 (oldest deadline 2024-09-12T02:19:22.196628+0000) 2024-09-12T02:19:45.669 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:45 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:45.313+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T02:19:05.097647+0000 front 2024-09-12T02:19:05.097663+0000 (oldest deadline 2024-09-12T02:19:28.498137+0000) 2024-09-12T02:19:45.670 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:45 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:45.313+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.33:6812 osd.4 since back 2024-09-12T02:19:11.899193+0000 front 2024-09-12T02:19:11.899218+0000 (oldest deadline 2024-09-12T02:19:35.999293+0000) 2024-09-12T02:19:45.683 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:45 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:45.218+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:38.124303+0000 front 2024-09-12T02:18:38.124204+0000 (oldest deadline 2024-09-12T02:19:04.024431+0000) 2024-09-12T02:19:45.683 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:45 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:45.218+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:49.925357+0000 front 2024-09-12T02:18:49.925414+0000 (oldest deadline 2024-09-12T02:19:14.625383+0000) 2024-09-12T02:19:45.684 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:45 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:45.218+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:58.726706+0000 front 2024-09-12T02:18:58.726591+0000 (oldest deadline 2024-09-12T02:19:22.226596+0000) 2024-09-12T02:19:45.684 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:45 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:45.218+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T02:19:05.627979+0000 front 2024-09-12T02:19:05.627999+0000 (oldest deadline 2024-09-12T02:19:29.128128+0000) 2024-09-12T02:19:45.684 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:45 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:45.218+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.33:6812 osd.4 since back 2024-09-12T02:19:13.228970+0000 front 2024-09-12T02:19:13.229156+0000 (oldest deadline 2024-09-12T02:19:37.928950+0000) 2024-09-12T02:19:45.684 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:45 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:45.218+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-09-12T02:19:20.230024+0000 front 2024-09-12T02:19:20.230104+0000 (oldest deadline 2024-09-12T02:19:44.329884+0000) 2024-09-12T02:19:46.127 INFO:journalctl@ceph.osd.8.smithi184.stdout:Sep 12 02:19:45 smithi184 podman[73096]: 2024-09-12 02:19:45.861154078 +0000 UTC m=+5.632983430 container died dfe554856d3cb598dccc15629d972e4df6ed5c429a017ed7e517c6f5705c4c82 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-8, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.37.2) 2024-09-12T02:19:46.128 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:45 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:45.902+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:40.491334+0000 front 2024-09-12T02:18:40.491223+0000 (oldest deadline 2024-09-12T02:19:05.191278+0000) 2024-09-12T02:19:46.128 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:45 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:45.902+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:49.192960+0000 front 2024-09-12T02:18:49.193116+0000 (oldest deadline 2024-09-12T02:19:15.092803+0000) 2024-09-12T02:19:46.128 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:45 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:45.902+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:58.593805+0000 front 2024-09-12T02:18:58.593869+0000 (oldest deadline 2024-09-12T02:19:19.594390+0000) 2024-09-12T02:19:46.128 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:45 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:45.902+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.33:6812 osd.4 since back 2024-09-12T02:19:14.097458+0000 front 2024-09-12T02:19:14.097525+0000 (oldest deadline 2024-09-12T02:19:38.197566+0000) 2024-09-12T02:19:46.128 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:45 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:45.902+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-09-12T02:19:18.197917+0000 front 2024-09-12T02:19:18.198027+0000 (oldest deadline 2024-09-12T02:19:43.498195+0000) 2024-09-12T02:19:46.627 INFO:journalctl@ceph.osd.8.smithi184.stdout:Sep 12 02:19:46 smithi184 podman[73096]: 2024-09-12 02:19:46.173784622 +0000 UTC m=+5.945613975 container cleanup dfe554856d3cb598dccc15629d972e4df6ed5c429a017ed7e517c6f5705c4c82 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-8, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-09-12T02:19:46.628 INFO:journalctl@ceph.osd.8.smithi184.stdout:Sep 12 02:19:46 smithi184 bash[73096]: ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-8 2024-09-12T02:19:46.628 INFO:journalctl@ceph.osd.8.smithi184.stdout:Sep 12 02:19:46 smithi184 podman[73114]: 2024-09-12 02:19:46.391938246 +0000 UTC m=+0.522141143 container remove dfe554856d3cb598dccc15629d972e4df6ed5c429a017ed7e517c6f5705c4c82 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-8, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2) 2024-09-12T02:19:46.669 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:46 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:46.336+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:42.394291+0000 front 2024-09-12T02:18:42.394247+0000 (oldest deadline 2024-09-12T02:19:05.894091+0000) 2024-09-12T02:19:46.669 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:46 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:46.336+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:47.595046+0000 front 2024-09-12T02:18:47.595165+0000 (oldest deadline 2024-09-12T02:19:12.295142+0000) 2024-09-12T02:19:46.669 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:46 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:46.336+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:58.696957+0000 front 2024-09-12T02:18:58.696791+0000 (oldest deadline 2024-09-12T02:19:22.196628+0000) 2024-09-12T02:19:46.669 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:46 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:46.336+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T02:19:05.097647+0000 front 2024-09-12T02:19:05.097663+0000 (oldest deadline 2024-09-12T02:19:28.498137+0000) 2024-09-12T02:19:46.669 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:46 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:46.336+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.33:6812 osd.4 since back 2024-09-12T02:19:11.899193+0000 front 2024-09-12T02:19:11.899218+0000 (oldest deadline 2024-09-12T02:19:35.999293+0000) 2024-09-12T02:19:46.669 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:46 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:46.336+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-09-12T02:19:20.000612+0000 front 2024-09-12T02:19:20.000719+0000 (oldest deadline 2024-09-12T02:19:45.900515+0000) 2024-09-12T02:19:46.683 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:46 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:46.231+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:38.124303+0000 front 2024-09-12T02:18:38.124204+0000 (oldest deadline 2024-09-12T02:19:04.024431+0000) 2024-09-12T02:19:46.684 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:46 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:46.231+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:49.925357+0000 front 2024-09-12T02:18:49.925414+0000 (oldest deadline 2024-09-12T02:19:14.625383+0000) 2024-09-12T02:19:46.684 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:46 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:46.231+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:58.726706+0000 front 2024-09-12T02:18:58.726591+0000 (oldest deadline 2024-09-12T02:19:22.226596+0000) 2024-09-12T02:19:46.684 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:46 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:46.231+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T02:19:05.627979+0000 front 2024-09-12T02:19:05.627999+0000 (oldest deadline 2024-09-12T02:19:29.128128+0000) 2024-09-12T02:19:46.684 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:46 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:46.231+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.33:6812 osd.4 since back 2024-09-12T02:19:13.228970+0000 front 2024-09-12T02:19:13.229156+0000 (oldest deadline 2024-09-12T02:19:37.928950+0000) 2024-09-12T02:19:46.684 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:46 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:46.231+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-09-12T02:19:20.230024+0000 front 2024-09-12T02:19:20.230104+0000 (oldest deadline 2024-09-12T02:19:44.329884+0000) 2024-09-12T02:19:47.028 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:46 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:46.868+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:40.491334+0000 front 2024-09-12T02:18:40.491223+0000 (oldest deadline 2024-09-12T02:19:05.191278+0000) 2024-09-12T02:19:47.028 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:46 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:46.868+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:49.192960+0000 front 2024-09-12T02:18:49.193116+0000 (oldest deadline 2024-09-12T02:19:15.092803+0000) 2024-09-12T02:19:47.028 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:46 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:46.868+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:58.593805+0000 front 2024-09-12T02:18:58.593869+0000 (oldest deadline 2024-09-12T02:19:19.594390+0000) 2024-09-12T02:19:47.028 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:46 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:46.868+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.33:6812 osd.4 since back 2024-09-12T02:19:14.097458+0000 front 2024-09-12T02:19:14.097525+0000 (oldest deadline 2024-09-12T02:19:38.197566+0000) 2024-09-12T02:19:47.029 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:46 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:46.868+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-09-12T02:19:18.197917+0000 front 2024-09-12T02:19:18.198027+0000 (oldest deadline 2024-09-12T02:19:43.498195+0000) 2024-09-12T02:19:47.029 INFO:journalctl@ceph.osd.8.smithi184.stdout:Sep 12 02:19:46 smithi184 podman[73202]: 2024-09-12 02:19:46.620104669 +0000 UTC m=+0.016255387 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T02:19:47.029 INFO:journalctl@ceph.osd.8.smithi184.stdout:Sep 12 02:19:46 smithi184 podman[73202]: 2024-09-12 02:19:46.742857577 +0000 UTC m=+0.139008285 container create 61325f31faff950c474f97b3f089865671f44ff34f7e6e4cc980d03c323c1633 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-8-deactivate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T02:19:47.029 INFO:journalctl@ceph.osd.8.smithi184.stdout:Sep 12 02:19:46 smithi184 podman[73202]: 2024-09-12 02:19:46.961811148 +0000 UTC m=+0.357961853 container init 61325f31faff950c474f97b3f089865671f44ff34f7e6e4cc980d03c323c1633 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-8-deactivate, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.license=GPLv2) 2024-09-12T02:19:47.029 INFO:journalctl@ceph.osd.8.smithi184.stdout:Sep 12 02:19:46 smithi184 podman[73202]: 2024-09-12 02:19:46.964996056 +0000 UTC m=+0.361146762 container start 61325f31faff950c474f97b3f089865671f44ff34f7e6e4cc980d03c323c1633 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-8-deactivate, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-09-12T02:19:47.377 INFO:journalctl@ceph.osd.8.smithi184.stdout:Sep 12 02:19:47 smithi184 podman[73202]: 2024-09-12 02:19:47.02842157 +0000 UTC m=+0.424572289 container attach 61325f31faff950c474f97b3f089865671f44ff34f7e6e4cc980d03c323c1633 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-8-deactivate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240731) 2024-09-12T02:19:47.377 INFO:journalctl@ceph.osd.8.smithi184.stdout:Sep 12 02:19:47 smithi184 podman[73202]: 2024-09-12 02:19:47.158451306 +0000 UTC m=+0.554602022 container died 61325f31faff950c474f97b3f089865671f44ff34f7e6e4cc980d03c323c1633 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-8-deactivate, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3) 2024-09-12T02:19:47.669 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:47 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:47.292+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:42.394291+0000 front 2024-09-12T02:18:42.394247+0000 (oldest deadline 2024-09-12T02:19:05.894091+0000) 2024-09-12T02:19:47.669 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:47 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:47.292+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:47.595046+0000 front 2024-09-12T02:18:47.595165+0000 (oldest deadline 2024-09-12T02:19:12.295142+0000) 2024-09-12T02:19:47.669 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:47 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:47.292+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:58.696957+0000 front 2024-09-12T02:18:58.696791+0000 (oldest deadline 2024-09-12T02:19:22.196628+0000) 2024-09-12T02:19:47.669 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:47 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:47.292+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T02:19:05.097647+0000 front 2024-09-12T02:19:05.097663+0000 (oldest deadline 2024-09-12T02:19:28.498137+0000) 2024-09-12T02:19:47.670 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:47 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:47.292+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.33:6812 osd.4 since back 2024-09-12T02:19:11.899193+0000 front 2024-09-12T02:19:11.899218+0000 (oldest deadline 2024-09-12T02:19:35.999293+0000) 2024-09-12T02:19:47.670 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:47 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:47.292+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-09-12T02:19:20.000612+0000 front 2024-09-12T02:19:20.000719+0000 (oldest deadline 2024-09-12T02:19:45.900515+0000) 2024-09-12T02:19:47.683 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:47 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:47.269+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:38.124303+0000 front 2024-09-12T02:18:38.124204+0000 (oldest deadline 2024-09-12T02:19:04.024431+0000) 2024-09-12T02:19:47.683 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:47 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:47.269+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:49.925357+0000 front 2024-09-12T02:18:49.925414+0000 (oldest deadline 2024-09-12T02:19:14.625383+0000) 2024-09-12T02:19:47.683 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:47 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:47.269+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:58.726706+0000 front 2024-09-12T02:18:58.726591+0000 (oldest deadline 2024-09-12T02:19:22.226596+0000) 2024-09-12T02:19:47.684 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:47 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:47.269+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T02:19:05.627979+0000 front 2024-09-12T02:19:05.627999+0000 (oldest deadline 2024-09-12T02:19:29.128128+0000) 2024-09-12T02:19:47.684 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:47 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:47.269+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.33:6812 osd.4 since back 2024-09-12T02:19:13.228970+0000 front 2024-09-12T02:19:13.229156+0000 (oldest deadline 2024-09-12T02:19:37.928950+0000) 2024-09-12T02:19:47.684 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:47 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:47.269+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-09-12T02:19:20.230024+0000 front 2024-09-12T02:19:20.230104+0000 (oldest deadline 2024-09-12T02:19:44.329884+0000) 2024-09-12T02:19:47.700 INFO:journalctl@ceph.osd.8.smithi184.stdout:Sep 12 02:19:47 smithi184 podman[73225]: 2024-09-12 02:19:47.681764647 +0000 UTC m=+0.518036294 container remove 61325f31faff950c474f97b3f089865671f44ff34f7e6e4cc980d03c323c1633 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-8-deactivate, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2) 2024-09-12T02:19:47.700 INFO:journalctl@ceph.osd.8.smithi184.stdout:Sep 12 02:19:47 smithi184 systemd[1]: ceph-879f3b56-70a9-11ef-bcea-c7b262605968@osd.8.service: Deactivated successfully. 2024-09-12T02:19:47.700 INFO:journalctl@ceph.osd.8.smithi184.stdout:Sep 12 02:19:47 smithi184 systemd[1]: Stopped Ceph osd.8 for 879f3b56-70a9-11ef-bcea-c7b262605968. 2024-09-12T02:19:47.701 INFO:journalctl@ceph.osd.8.smithi184.stdout:Sep 12 02:19:47 smithi184 systemd[1]: ceph-879f3b56-70a9-11ef-bcea-c7b262605968@osd.8.service: Consumed 10.348s CPU time. 2024-09-12T02:19:47.701 DEBUG:teuthology.orchestra.run.smithi184:> sudo pkill -f 'journalctl -f -n 0 -u ceph-879f3b56-70a9-11ef-bcea-c7b262605968@osd.8.service' 2024-09-12T02:19:47.781 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T02:19:47.781 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-09-12T02:19:47.782 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-09-12T02:19:47.782 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ceph-879f3b56-70a9-11ef-bcea-c7b262605968@osd.9 2024-09-12T02:19:48.093 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:47 smithi016 systemd[1]: Stopping Ceph osd.9 for 879f3b56-70a9-11ef-bcea-c7b262605968... 2024-09-12T02:19:48.093 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:48 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:48.092+0000 7fc3b6ad5640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T02:19:48.093 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:48 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:48.092+0000 7fc3b6ad5640 -1 osd.9 85 *** Got signal Terminated *** 2024-09-12T02:19:48.127 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:47 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:47.877+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:40.491334+0000 front 2024-09-12T02:18:40.491223+0000 (oldest deadline 2024-09-12T02:19:05.191278+0000) 2024-09-12T02:19:48.127 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:47 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:47.877+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:49.192960+0000 front 2024-09-12T02:18:49.193116+0000 (oldest deadline 2024-09-12T02:19:15.092803+0000) 2024-09-12T02:19:48.128 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:47 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:47.877+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:58.593805+0000 front 2024-09-12T02:18:58.593869+0000 (oldest deadline 2024-09-12T02:19:19.594390+0000) 2024-09-12T02:19:48.128 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:47 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:47.877+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.33:6812 osd.4 since back 2024-09-12T02:19:14.097458+0000 front 2024-09-12T02:19:14.097525+0000 (oldest deadline 2024-09-12T02:19:38.197566+0000) 2024-09-12T02:19:48.128 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:47 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:47.877+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-09-12T02:19:18.197917+0000 front 2024-09-12T02:19:18.198027+0000 (oldest deadline 2024-09-12T02:19:43.498195+0000) 2024-09-12T02:19:48.434 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:48 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:48.092+0000 7fc3b6ad5640 -1 osd.9 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T02:19:48.434 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:48 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:48.306+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:38.124303+0000 front 2024-09-12T02:18:38.124204+0000 (oldest deadline 2024-09-12T02:19:04.024431+0000) 2024-09-12T02:19:48.434 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:48 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:48.306+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:49.925357+0000 front 2024-09-12T02:18:49.925414+0000 (oldest deadline 2024-09-12T02:19:14.625383+0000) 2024-09-12T02:19:48.434 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:48 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:48.306+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:58.726706+0000 front 2024-09-12T02:18:58.726591+0000 (oldest deadline 2024-09-12T02:19:22.226596+0000) 2024-09-12T02:19:48.434 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:48 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:48.306+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T02:19:05.627979+0000 front 2024-09-12T02:19:05.627999+0000 (oldest deadline 2024-09-12T02:19:29.128128+0000) 2024-09-12T02:19:48.434 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:48 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:48.306+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.33:6812 osd.4 since back 2024-09-12T02:19:13.228970+0000 front 2024-09-12T02:19:13.229156+0000 (oldest deadline 2024-09-12T02:19:37.928950+0000) 2024-09-12T02:19:48.434 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:48 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:48.306+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-09-12T02:19:20.230024+0000 front 2024-09-12T02:19:20.230104+0000 (oldest deadline 2024-09-12T02:19:44.329884+0000) 2024-09-12T02:19:48.669 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:48 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:48.275+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:42.394291+0000 front 2024-09-12T02:18:42.394247+0000 (oldest deadline 2024-09-12T02:19:05.894091+0000) 2024-09-12T02:19:48.669 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:48 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:48.275+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:47.595046+0000 front 2024-09-12T02:18:47.595165+0000 (oldest deadline 2024-09-12T02:19:12.295142+0000) 2024-09-12T02:19:48.669 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:48 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:48.275+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:58.696957+0000 front 2024-09-12T02:18:58.696791+0000 (oldest deadline 2024-09-12T02:19:22.196628+0000) 2024-09-12T02:19:48.669 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:48 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:48.275+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T02:19:05.097647+0000 front 2024-09-12T02:19:05.097663+0000 (oldest deadline 2024-09-12T02:19:28.498137+0000) 2024-09-12T02:19:48.669 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:48 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:48.275+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.33:6812 osd.4 since back 2024-09-12T02:19:11.899193+0000 front 2024-09-12T02:19:11.899218+0000 (oldest deadline 2024-09-12T02:19:35.999293+0000) 2024-09-12T02:19:48.669 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:48 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:48.275+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-09-12T02:19:20.000612+0000 front 2024-09-12T02:19:20.000719+0000 (oldest deadline 2024-09-12T02:19:45.900515+0000) 2024-09-12T02:19:49.377 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:48 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:48.901+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:40.491334+0000 front 2024-09-12T02:18:40.491223+0000 (oldest deadline 2024-09-12T02:19:05.191278+0000) 2024-09-12T02:19:49.377 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:48 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:48.901+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:49.192960+0000 front 2024-09-12T02:18:49.193116+0000 (oldest deadline 2024-09-12T02:19:15.092803+0000) 2024-09-12T02:19:49.377 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:48 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:48.901+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:58.593805+0000 front 2024-09-12T02:18:58.593869+0000 (oldest deadline 2024-09-12T02:19:19.594390+0000) 2024-09-12T02:19:49.377 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:48 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:48.901+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.33:6812 osd.4 since back 2024-09-12T02:19:14.097458+0000 front 2024-09-12T02:19:14.097525+0000 (oldest deadline 2024-09-12T02:19:38.197566+0000) 2024-09-12T02:19:49.378 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:48 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:48.901+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-09-12T02:19:18.197917+0000 front 2024-09-12T02:19:18.198027+0000 (oldest deadline 2024-09-12T02:19:43.498195+0000) 2024-09-12T02:19:49.669 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:49 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:49.236+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:42.394291+0000 front 2024-09-12T02:18:42.394247+0000 (oldest deadline 2024-09-12T02:19:05.894091+0000) 2024-09-12T02:19:49.669 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:49 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:49.236+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:47.595046+0000 front 2024-09-12T02:18:47.595165+0000 (oldest deadline 2024-09-12T02:19:12.295142+0000) 2024-09-12T02:19:49.669 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:49 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:49.236+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:58.696957+0000 front 2024-09-12T02:18:58.696791+0000 (oldest deadline 2024-09-12T02:19:22.196628+0000) 2024-09-12T02:19:49.669 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:49 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:49.236+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T02:19:05.097647+0000 front 2024-09-12T02:19:05.097663+0000 (oldest deadline 2024-09-12T02:19:28.498137+0000) 2024-09-12T02:19:49.669 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:49 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:49.236+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.33:6812 osd.4 since back 2024-09-12T02:19:11.899193+0000 front 2024-09-12T02:19:11.899218+0000 (oldest deadline 2024-09-12T02:19:35.999293+0000) 2024-09-12T02:19:49.669 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:49 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:49.236+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-09-12T02:19:20.000612+0000 front 2024-09-12T02:19:20.000719+0000 (oldest deadline 2024-09-12T02:19:45.900515+0000) 2024-09-12T02:19:49.683 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:49 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:49.353+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:38.124303+0000 front 2024-09-12T02:18:38.124204+0000 (oldest deadline 2024-09-12T02:19:04.024431+0000) 2024-09-12T02:19:49.684 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:49 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:49.353+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:49.925357+0000 front 2024-09-12T02:18:49.925414+0000 (oldest deadline 2024-09-12T02:19:14.625383+0000) 2024-09-12T02:19:49.684 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:49 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:49.353+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:58.726706+0000 front 2024-09-12T02:18:58.726591+0000 (oldest deadline 2024-09-12T02:19:22.226596+0000) 2024-09-12T02:19:49.684 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:49 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:49.353+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T02:19:05.627979+0000 front 2024-09-12T02:19:05.627999+0000 (oldest deadline 2024-09-12T02:19:29.128128+0000) 2024-09-12T02:19:49.684 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:49 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:49.353+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.33:6812 osd.4 since back 2024-09-12T02:19:13.228970+0000 front 2024-09-12T02:19:13.229156+0000 (oldest deadline 2024-09-12T02:19:37.928950+0000) 2024-09-12T02:19:49.684 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:49 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:49.353+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-09-12T02:19:20.230024+0000 front 2024-09-12T02:19:20.230104+0000 (oldest deadline 2024-09-12T02:19:44.329884+0000) 2024-09-12T02:19:50.127 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:49 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:49.866+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:40.491334+0000 front 2024-09-12T02:18:40.491223+0000 (oldest deadline 2024-09-12T02:19:05.191278+0000) 2024-09-12T02:19:50.127 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:49 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:49.866+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:49.192960+0000 front 2024-09-12T02:18:49.193116+0000 (oldest deadline 2024-09-12T02:19:15.092803+0000) 2024-09-12T02:19:50.128 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:49 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:49.866+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:58.593805+0000 front 2024-09-12T02:18:58.593869+0000 (oldest deadline 2024-09-12T02:19:19.594390+0000) 2024-09-12T02:19:50.128 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:49 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:49.866+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.33:6812 osd.4 since back 2024-09-12T02:19:14.097458+0000 front 2024-09-12T02:19:14.097525+0000 (oldest deadline 2024-09-12T02:19:38.197566+0000) 2024-09-12T02:19:50.128 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:49 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:49.866+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-09-12T02:19:18.197917+0000 front 2024-09-12T02:19:18.198027+0000 (oldest deadline 2024-09-12T02:19:43.498195+0000) 2024-09-12T02:19:50.419 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:50 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:50.191+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:42.394291+0000 front 2024-09-12T02:18:42.394247+0000 (oldest deadline 2024-09-12T02:19:05.894091+0000) 2024-09-12T02:19:50.419 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:50 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:50.191+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:47.595046+0000 front 2024-09-12T02:18:47.595165+0000 (oldest deadline 2024-09-12T02:19:12.295142+0000) 2024-09-12T02:19:50.420 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:50 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:50.191+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:58.696957+0000 front 2024-09-12T02:18:58.696791+0000 (oldest deadline 2024-09-12T02:19:22.196628+0000) 2024-09-12T02:19:50.420 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:50 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:50.191+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T02:19:05.097647+0000 front 2024-09-12T02:19:05.097663+0000 (oldest deadline 2024-09-12T02:19:28.498137+0000) 2024-09-12T02:19:50.420 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:50 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:50.191+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.33:6812 osd.4 since back 2024-09-12T02:19:11.899193+0000 front 2024-09-12T02:19:11.899218+0000 (oldest deadline 2024-09-12T02:19:35.999293+0000) 2024-09-12T02:19:50.420 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:50 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:50.191+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-09-12T02:19:20.000612+0000 front 2024-09-12T02:19:20.000719+0000 (oldest deadline 2024-09-12T02:19:45.900515+0000) 2024-09-12T02:19:50.683 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:50 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:50.358+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:38.124303+0000 front 2024-09-12T02:18:38.124204+0000 (oldest deadline 2024-09-12T02:19:04.024431+0000) 2024-09-12T02:19:50.684 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:50 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:50.358+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:49.925357+0000 front 2024-09-12T02:18:49.925414+0000 (oldest deadline 2024-09-12T02:19:14.625383+0000) 2024-09-12T02:19:50.684 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:50 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:50.358+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:58.726706+0000 front 2024-09-12T02:18:58.726591+0000 (oldest deadline 2024-09-12T02:19:22.226596+0000) 2024-09-12T02:19:50.684 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:50 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:50.358+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T02:19:05.627979+0000 front 2024-09-12T02:19:05.627999+0000 (oldest deadline 2024-09-12T02:19:29.128128+0000) 2024-09-12T02:19:50.684 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:50 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:50.358+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.33:6812 osd.4 since back 2024-09-12T02:19:13.228970+0000 front 2024-09-12T02:19:13.229156+0000 (oldest deadline 2024-09-12T02:19:37.928950+0000) 2024-09-12T02:19:50.684 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:50 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:50.358+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-09-12T02:19:20.230024+0000 front 2024-09-12T02:19:20.230104+0000 (oldest deadline 2024-09-12T02:19:44.329884+0000) 2024-09-12T02:19:51.127 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:50 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:50.858+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:40.491334+0000 front 2024-09-12T02:18:40.491223+0000 (oldest deadline 2024-09-12T02:19:05.191278+0000) 2024-09-12T02:19:51.128 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:50 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:50.858+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:49.192960+0000 front 2024-09-12T02:18:49.193116+0000 (oldest deadline 2024-09-12T02:19:15.092803+0000) 2024-09-12T02:19:51.128 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:50 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:50.858+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:58.593805+0000 front 2024-09-12T02:18:58.593869+0000 (oldest deadline 2024-09-12T02:19:19.594390+0000) 2024-09-12T02:19:51.128 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:50 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:50.858+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.33:6812 osd.4 since back 2024-09-12T02:19:14.097458+0000 front 2024-09-12T02:19:14.097525+0000 (oldest deadline 2024-09-12T02:19:38.197566+0000) 2024-09-12T02:19:51.128 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:50 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:50.858+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-09-12T02:19:18.197917+0000 front 2024-09-12T02:19:18.198027+0000 (oldest deadline 2024-09-12T02:19:43.498195+0000) 2024-09-12T02:19:51.419 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:51 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:51.149+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:42.394291+0000 front 2024-09-12T02:18:42.394247+0000 (oldest deadline 2024-09-12T02:19:05.894091+0000) 2024-09-12T02:19:51.419 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:51 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:51.149+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:47.595046+0000 front 2024-09-12T02:18:47.595165+0000 (oldest deadline 2024-09-12T02:19:12.295142+0000) 2024-09-12T02:19:51.419 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:51 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:51.149+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:58.696957+0000 front 2024-09-12T02:18:58.696791+0000 (oldest deadline 2024-09-12T02:19:22.196628+0000) 2024-09-12T02:19:51.419 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:51 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:51.149+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T02:19:05.097647+0000 front 2024-09-12T02:19:05.097663+0000 (oldest deadline 2024-09-12T02:19:28.498137+0000) 2024-09-12T02:19:51.419 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:51 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:51.149+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.33:6812 osd.4 since back 2024-09-12T02:19:11.899193+0000 front 2024-09-12T02:19:11.899218+0000 (oldest deadline 2024-09-12T02:19:35.999293+0000) 2024-09-12T02:19:51.419 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:51 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:51.149+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-09-12T02:19:20.000612+0000 front 2024-09-12T02:19:20.000719+0000 (oldest deadline 2024-09-12T02:19:45.900515+0000) 2024-09-12T02:19:51.683 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:51 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:51.309+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:38.124303+0000 front 2024-09-12T02:18:38.124204+0000 (oldest deadline 2024-09-12T02:19:04.024431+0000) 2024-09-12T02:19:51.684 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:51 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:51.309+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:49.925357+0000 front 2024-09-12T02:18:49.925414+0000 (oldest deadline 2024-09-12T02:19:14.625383+0000) 2024-09-12T02:19:51.684 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:51 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:51.309+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:58.726706+0000 front 2024-09-12T02:18:58.726591+0000 (oldest deadline 2024-09-12T02:19:22.226596+0000) 2024-09-12T02:19:51.684 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:51 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:51.309+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T02:19:05.627979+0000 front 2024-09-12T02:19:05.627999+0000 (oldest deadline 2024-09-12T02:19:29.128128+0000) 2024-09-12T02:19:51.684 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:51 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:51.309+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.33:6812 osd.4 since back 2024-09-12T02:19:13.228970+0000 front 2024-09-12T02:19:13.229156+0000 (oldest deadline 2024-09-12T02:19:37.928950+0000) 2024-09-12T02:19:51.684 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:51 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:51.309+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-09-12T02:19:20.230024+0000 front 2024-09-12T02:19:20.230104+0000 (oldest deadline 2024-09-12T02:19:44.329884+0000) 2024-09-12T02:19:52.127 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:51 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:51.809+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:40.491334+0000 front 2024-09-12T02:18:40.491223+0000 (oldest deadline 2024-09-12T02:19:05.191278+0000) 2024-09-12T02:19:52.127 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:51 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:51.809+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:49.192960+0000 front 2024-09-12T02:18:49.193116+0000 (oldest deadline 2024-09-12T02:19:15.092803+0000) 2024-09-12T02:19:52.127 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:51 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:51.809+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:58.593805+0000 front 2024-09-12T02:18:58.593869+0000 (oldest deadline 2024-09-12T02:19:19.594390+0000) 2024-09-12T02:19:52.128 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:51 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:51.809+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.33:6812 osd.4 since back 2024-09-12T02:19:14.097458+0000 front 2024-09-12T02:19:14.097525+0000 (oldest deadline 2024-09-12T02:19:38.197566+0000) 2024-09-12T02:19:52.128 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:51 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:51.809+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-09-12T02:19:18.197917+0000 front 2024-09-12T02:19:18.198027+0000 (oldest deadline 2024-09-12T02:19:43.498195+0000) 2024-09-12T02:19:52.669 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:52 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:52.194+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:42.394291+0000 front 2024-09-12T02:18:42.394247+0000 (oldest deadline 2024-09-12T02:19:05.894091+0000) 2024-09-12T02:19:52.669 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:52 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:52.194+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:47.595046+0000 front 2024-09-12T02:18:47.595165+0000 (oldest deadline 2024-09-12T02:19:12.295142+0000) 2024-09-12T02:19:52.669 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:52 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:52.194+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:58.696957+0000 front 2024-09-12T02:18:58.696791+0000 (oldest deadline 2024-09-12T02:19:22.196628+0000) 2024-09-12T02:19:52.670 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:52 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:52.194+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T02:19:05.097647+0000 front 2024-09-12T02:19:05.097663+0000 (oldest deadline 2024-09-12T02:19:28.498137+0000) 2024-09-12T02:19:52.670 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:52 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:52.194+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.33:6812 osd.4 since back 2024-09-12T02:19:11.899193+0000 front 2024-09-12T02:19:11.899218+0000 (oldest deadline 2024-09-12T02:19:35.999293+0000) 2024-09-12T02:19:52.670 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:52 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:52.194+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-09-12T02:19:20.000612+0000 front 2024-09-12T02:19:20.000719+0000 (oldest deadline 2024-09-12T02:19:45.900515+0000) 2024-09-12T02:19:52.670 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:52 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:52.194+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-12T02:19:25.901186+0000 front 2024-09-12T02:19:25.901318+0000 (oldest deadline 2024-09-12T02:19:51.800879+0000) 2024-09-12T02:19:52.684 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:52 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:52.314+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:38.124303+0000 front 2024-09-12T02:18:38.124204+0000 (oldest deadline 2024-09-12T02:19:04.024431+0000) 2024-09-12T02:19:52.684 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:52 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:52.314+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:49.925357+0000 front 2024-09-12T02:18:49.925414+0000 (oldest deadline 2024-09-12T02:19:14.625383+0000) 2024-09-12T02:19:52.684 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:52 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:52.314+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:58.726706+0000 front 2024-09-12T02:18:58.726591+0000 (oldest deadline 2024-09-12T02:19:22.226596+0000) 2024-09-12T02:19:52.684 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:52 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:52.314+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T02:19:05.627979+0000 front 2024-09-12T02:19:05.627999+0000 (oldest deadline 2024-09-12T02:19:29.128128+0000) 2024-09-12T02:19:52.684 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:52 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:52.314+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.33:6812 osd.4 since back 2024-09-12T02:19:13.228970+0000 front 2024-09-12T02:19:13.229156+0000 (oldest deadline 2024-09-12T02:19:37.928950+0000) 2024-09-12T02:19:52.684 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:52 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9[65426]: 2024-09-12T02:19:52.314+0000 7fc3b30da640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-09-12T02:19:20.230024+0000 front 2024-09-12T02:19:20.230104+0000 (oldest deadline 2024-09-12T02:19:44.329884+0000) 2024-09-12T02:19:53.127 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:52 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:52.805+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:40.491334+0000 front 2024-09-12T02:18:40.491223+0000 (oldest deadline 2024-09-12T02:19:05.191278+0000) 2024-09-12T02:19:53.127 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:52 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:52.805+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:49.192960+0000 front 2024-09-12T02:18:49.193116+0000 (oldest deadline 2024-09-12T02:19:15.092803+0000) 2024-09-12T02:19:53.128 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:52 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:52.805+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:58.593805+0000 front 2024-09-12T02:18:58.593869+0000 (oldest deadline 2024-09-12T02:19:19.594390+0000) 2024-09-12T02:19:53.128 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:52 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:52.805+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.33:6812 osd.4 since back 2024-09-12T02:19:14.097458+0000 front 2024-09-12T02:19:14.097525+0000 (oldest deadline 2024-09-12T02:19:38.197566+0000) 2024-09-12T02:19:53.128 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:52 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:52.805+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-09-12T02:19:18.197917+0000 front 2024-09-12T02:19:18.198027+0000 (oldest deadline 2024-09-12T02:19:43.498195+0000) 2024-09-12T02:19:53.128 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:52 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:52.805+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-12T02:19:26.999235+0000 front 2024-09-12T02:19:26.999249+0000 (oldest deadline 2024-09-12T02:19:52.299019+0000) 2024-09-12T02:19:53.669 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:53 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:53.221+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:42.394291+0000 front 2024-09-12T02:18:42.394247+0000 (oldest deadline 2024-09-12T02:19:05.894091+0000) 2024-09-12T02:19:53.669 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:53 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:53.221+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:47.595046+0000 front 2024-09-12T02:18:47.595165+0000 (oldest deadline 2024-09-12T02:19:12.295142+0000) 2024-09-12T02:19:53.669 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:53 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:53.221+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:58.696957+0000 front 2024-09-12T02:18:58.696791+0000 (oldest deadline 2024-09-12T02:19:22.196628+0000) 2024-09-12T02:19:53.669 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:53 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:53.221+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T02:19:05.097647+0000 front 2024-09-12T02:19:05.097663+0000 (oldest deadline 2024-09-12T02:19:28.498137+0000) 2024-09-12T02:19:53.669 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:53 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:53.221+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.33:6812 osd.4 since back 2024-09-12T02:19:11.899193+0000 front 2024-09-12T02:19:11.899218+0000 (oldest deadline 2024-09-12T02:19:35.999293+0000) 2024-09-12T02:19:53.670 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:53 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:53.221+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-09-12T02:19:20.000612+0000 front 2024-09-12T02:19:20.000719+0000 (oldest deadline 2024-09-12T02:19:45.900515+0000) 2024-09-12T02:19:53.670 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:53 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:53.221+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-12T02:19:25.901186+0000 front 2024-09-12T02:19:25.901318+0000 (oldest deadline 2024-09-12T02:19:51.800879+0000) 2024-09-12T02:19:53.801 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:53 smithi016 podman[131569]: 2024-09-12 02:19:53.519504223 +0000 UTC m=+5.526643662 container died 8480dcd4fd5175c2c4c9e6f684a03bd01d5b4a05fe7c54280a850737378eb1fc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_CLEAN=True, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0) 2024-09-12T02:19:54.127 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:53 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:53.853+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:40.491334+0000 front 2024-09-12T02:18:40.491223+0000 (oldest deadline 2024-09-12T02:19:05.191278+0000) 2024-09-12T02:19:54.127 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:53 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:53.853+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:49.192960+0000 front 2024-09-12T02:18:49.193116+0000 (oldest deadline 2024-09-12T02:19:15.092803+0000) 2024-09-12T02:19:54.127 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:53 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:53.853+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:58.593805+0000 front 2024-09-12T02:18:58.593869+0000 (oldest deadline 2024-09-12T02:19:19.594390+0000) 2024-09-12T02:19:54.128 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:53 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:53.853+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.33:6812 osd.4 since back 2024-09-12T02:19:14.097458+0000 front 2024-09-12T02:19:14.097525+0000 (oldest deadline 2024-09-12T02:19:38.197566+0000) 2024-09-12T02:19:54.128 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:53 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:53.853+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-09-12T02:19:18.197917+0000 front 2024-09-12T02:19:18.198027+0000 (oldest deadline 2024-09-12T02:19:43.498195+0000) 2024-09-12T02:19:54.128 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:53 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:53.853+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-12T02:19:26.999235+0000 front 2024-09-12T02:19:26.999249+0000 (oldest deadline 2024-09-12T02:19:52.299019+0000) 2024-09-12T02:19:54.142 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:53 smithi016 podman[131569]: 2024-09-12 02:19:53.875618206 +0000 UTC m=+5.882757632 container cleanup 8480dcd4fd5175c2c4c9e6f684a03bd01d5b4a05fe7c54280a850737378eb1fc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , ceph=True) 2024-09-12T02:19:54.143 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:53 smithi016 bash[131569]: ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9 2024-09-12T02:19:54.433 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:54 smithi016 podman[131585]: 2024-09-12 02:19:54.143578979 +0000 UTC m=+0.618292612 container remove 8480dcd4fd5175c2c4c9e6f684a03bd01d5b4a05fe7c54280a850737378eb1fc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3) 2024-09-12T02:19:54.669 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:54 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:54.196+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:42.394291+0000 front 2024-09-12T02:18:42.394247+0000 (oldest deadline 2024-09-12T02:19:05.894091+0000) 2024-09-12T02:19:54.669 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:54 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:54.196+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:47.595046+0000 front 2024-09-12T02:18:47.595165+0000 (oldest deadline 2024-09-12T02:19:12.295142+0000) 2024-09-12T02:19:54.669 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:54 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:54.196+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:58.696957+0000 front 2024-09-12T02:18:58.696791+0000 (oldest deadline 2024-09-12T02:19:22.196628+0000) 2024-09-12T02:19:54.669 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:54 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:54.196+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T02:19:05.097647+0000 front 2024-09-12T02:19:05.097663+0000 (oldest deadline 2024-09-12T02:19:28.498137+0000) 2024-09-12T02:19:54.669 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:54 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:54.196+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.33:6812 osd.4 since back 2024-09-12T02:19:11.899193+0000 front 2024-09-12T02:19:11.899218+0000 (oldest deadline 2024-09-12T02:19:35.999293+0000) 2024-09-12T02:19:54.670 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:54 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:54.196+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-09-12T02:19:20.000612+0000 front 2024-09-12T02:19:20.000719+0000 (oldest deadline 2024-09-12T02:19:45.900515+0000) 2024-09-12T02:19:54.670 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:54 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:54.196+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-12T02:19:25.901186+0000 front 2024-09-12T02:19:25.901318+0000 (oldest deadline 2024-09-12T02:19:51.800879+0000) 2024-09-12T02:19:54.805 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:54 smithi016 podman[131675]: 2024-09-12 02:19:54.407209502 +0000 UTC m=+0.016608246 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T02:19:54.805 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:54 smithi016 podman[131675]: 2024-09-12 02:19:54.519516415 +0000 UTC m=+0.128915150 container create 99d3195fde7b2e26d8eb3a458b21417a9346c2d733ef6d46999280b4f457cd6d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9-deactivate, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-12T02:19:54.805 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:54 smithi016 podman[131675]: 2024-09-12 02:19:54.722116717 +0000 UTC m=+0.331515465 container init 99d3195fde7b2e26d8eb3a458b21417a9346c2d733ef6d46999280b4f457cd6d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9-deactivate, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-09-12T02:19:54.805 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:54 smithi016 podman[131675]: 2024-09-12 02:19:54.72543336 +0000 UTC m=+0.334832096 container start 99d3195fde7b2e26d8eb3a458b21417a9346c2d733ef6d46999280b4f457cd6d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9-deactivate, ceph=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-09-12T02:19:54.805 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:54 smithi016 podman[131675]: 2024-09-12 02:19:54.805313961 +0000 UTC m=+0.414712702 container attach 99d3195fde7b2e26d8eb3a458b21417a9346c2d733ef6d46999280b4f457cd6d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-09-12T02:19:55.127 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:54 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:54.875+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:40.491334+0000 front 2024-09-12T02:18:40.491223+0000 (oldest deadline 2024-09-12T02:19:05.191278+0000) 2024-09-12T02:19:55.127 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:54 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:54.875+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:49.192960+0000 front 2024-09-12T02:18:49.193116+0000 (oldest deadline 2024-09-12T02:19:15.092803+0000) 2024-09-12T02:19:55.127 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:54 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:54.875+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:58.593805+0000 front 2024-09-12T02:18:58.593869+0000 (oldest deadline 2024-09-12T02:19:19.594390+0000) 2024-09-12T02:19:55.128 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:54 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:54.875+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.33:6812 osd.4 since back 2024-09-12T02:19:14.097458+0000 front 2024-09-12T02:19:14.097525+0000 (oldest deadline 2024-09-12T02:19:38.197566+0000) 2024-09-12T02:19:55.128 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:54 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:54.875+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-09-12T02:19:18.197917+0000 front 2024-09-12T02:19:18.198027+0000 (oldest deadline 2024-09-12T02:19:43.498195+0000) 2024-09-12T02:19:55.128 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:54 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:54.875+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-12T02:19:26.999235+0000 front 2024-09-12T02:19:26.999249+0000 (oldest deadline 2024-09-12T02:19:52.299019+0000) 2024-09-12T02:19:55.183 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:54 smithi016 podman[131675]: 2024-09-12 02:19:54.928376287 +0000 UTC m=+0.537775023 container died 99d3195fde7b2e26d8eb3a458b21417a9346c2d733ef6d46999280b4f457cd6d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9-deactivate, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3) 2024-09-12T02:19:55.419 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:55 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:55.147+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:42.394291+0000 front 2024-09-12T02:18:42.394247+0000 (oldest deadline 2024-09-12T02:19:05.894091+0000) 2024-09-12T02:19:55.419 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:55 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:55.147+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:47.595046+0000 front 2024-09-12T02:18:47.595165+0000 (oldest deadline 2024-09-12T02:19:12.295142+0000) 2024-09-12T02:19:55.419 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:55 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:55.147+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:58.696957+0000 front 2024-09-12T02:18:58.696791+0000 (oldest deadline 2024-09-12T02:19:22.196628+0000) 2024-09-12T02:19:55.419 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:55 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:55.147+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T02:19:05.097647+0000 front 2024-09-12T02:19:05.097663+0000 (oldest deadline 2024-09-12T02:19:28.498137+0000) 2024-09-12T02:19:55.419 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:55 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:55.147+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.33:6812 osd.4 since back 2024-09-12T02:19:11.899193+0000 front 2024-09-12T02:19:11.899218+0000 (oldest deadline 2024-09-12T02:19:35.999293+0000) 2024-09-12T02:19:55.419 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:55 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:55.147+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-09-12T02:19:20.000612+0000 front 2024-09-12T02:19:20.000719+0000 (oldest deadline 2024-09-12T02:19:45.900515+0000) 2024-09-12T02:19:55.420 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:55 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:55.147+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-12T02:19:25.901186+0000 front 2024-09-12T02:19:25.901318+0000 (oldest deadline 2024-09-12T02:19:51.800879+0000) 2024-09-12T02:19:55.534 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 02:19:55 smithi016 podman[131699]: 2024-09-12 02:19:55.517261467 +0000 UTC m=+0.579225717 container remove 99d3195fde7b2e26d8eb3a458b21417a9346c2d733ef6d46999280b4f457cd6d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-9-deactivate, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-09-12T02:19:55.537 DEBUG:teuthology.orchestra.run.smithi016:> sudo pkill -f 'journalctl -f -n 0 -u ceph-879f3b56-70a9-11ef-bcea-c7b262605968@osd.9.service' 2024-09-12T02:19:55.581 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T02:19:55.581 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-09-12T02:19:55.581 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-09-12T02:19:55.581 DEBUG:teuthology.orchestra.run.smithi033:> sudo systemctl stop ceph-879f3b56-70a9-11ef-bcea-c7b262605968@osd.10 2024-09-12T02:19:55.883 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:55 smithi033 systemd[1]: Stopping Ceph osd.10 for 879f3b56-70a9-11ef-bcea-c7b262605968... 2024-09-12T02:19:55.883 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:55 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:55.882+0000 7f78d4ad6640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T02:19:55.883 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:55 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:55.882+0000 7f78d4ad6640 -1 osd.10 85 *** Got signal Terminated *** 2024-09-12T02:19:56.169 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:55 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:55.882+0000 7f78d4ad6640 -1 osd.10 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T02:19:56.169 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:56 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:56.123+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:42.394291+0000 front 2024-09-12T02:18:42.394247+0000 (oldest deadline 2024-09-12T02:19:05.894091+0000) 2024-09-12T02:19:56.169 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:56 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:56.123+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:47.595046+0000 front 2024-09-12T02:18:47.595165+0000 (oldest deadline 2024-09-12T02:19:12.295142+0000) 2024-09-12T02:19:56.170 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:56 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:56.123+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:58.696957+0000 front 2024-09-12T02:18:58.696791+0000 (oldest deadline 2024-09-12T02:19:22.196628+0000) 2024-09-12T02:19:56.170 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:56 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:56.123+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T02:19:05.097647+0000 front 2024-09-12T02:19:05.097663+0000 (oldest deadline 2024-09-12T02:19:28.498137+0000) 2024-09-12T02:19:56.170 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:56 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:56.123+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.33:6812 osd.4 since back 2024-09-12T02:19:11.899193+0000 front 2024-09-12T02:19:11.899218+0000 (oldest deadline 2024-09-12T02:19:35.999293+0000) 2024-09-12T02:19:56.170 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:56 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:56.123+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-09-12T02:19:20.000612+0000 front 2024-09-12T02:19:20.000719+0000 (oldest deadline 2024-09-12T02:19:45.900515+0000) 2024-09-12T02:19:56.170 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:56 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:56.123+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-12T02:19:25.901186+0000 front 2024-09-12T02:19:25.901318+0000 (oldest deadline 2024-09-12T02:19:51.800879+0000) 2024-09-12T02:19:56.377 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:55 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:55.904+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:40.491334+0000 front 2024-09-12T02:18:40.491223+0000 (oldest deadline 2024-09-12T02:19:05.191278+0000) 2024-09-12T02:19:56.377 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:55 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:55.904+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:49.192960+0000 front 2024-09-12T02:18:49.193116+0000 (oldest deadline 2024-09-12T02:19:15.092803+0000) 2024-09-12T02:19:56.377 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:55 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:55.904+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:58.593805+0000 front 2024-09-12T02:18:58.593869+0000 (oldest deadline 2024-09-12T02:19:19.594390+0000) 2024-09-12T02:19:56.377 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:55 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:55.904+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.33:6812 osd.4 since back 2024-09-12T02:19:14.097458+0000 front 2024-09-12T02:19:14.097525+0000 (oldest deadline 2024-09-12T02:19:38.197566+0000) 2024-09-12T02:19:56.378 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:55 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:55.904+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-09-12T02:19:18.197917+0000 front 2024-09-12T02:19:18.198027+0000 (oldest deadline 2024-09-12T02:19:43.498195+0000) 2024-09-12T02:19:56.378 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:55 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:55.904+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-12T02:19:26.999235+0000 front 2024-09-12T02:19:26.999249+0000 (oldest deadline 2024-09-12T02:19:52.299019+0000) 2024-09-12T02:19:57.127 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:56 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:56.858+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:40.491334+0000 front 2024-09-12T02:18:40.491223+0000 (oldest deadline 2024-09-12T02:19:05.191278+0000) 2024-09-12T02:19:57.128 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:56 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:56.858+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:49.192960+0000 front 2024-09-12T02:18:49.193116+0000 (oldest deadline 2024-09-12T02:19:15.092803+0000) 2024-09-12T02:19:57.128 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:56 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:56.858+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:58.593805+0000 front 2024-09-12T02:18:58.593869+0000 (oldest deadline 2024-09-12T02:19:19.594390+0000) 2024-09-12T02:19:57.128 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:56 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:56.858+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.33:6812 osd.4 since back 2024-09-12T02:19:14.097458+0000 front 2024-09-12T02:19:14.097525+0000 (oldest deadline 2024-09-12T02:19:38.197566+0000) 2024-09-12T02:19:57.128 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:56 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:56.858+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-09-12T02:19:18.197917+0000 front 2024-09-12T02:19:18.198027+0000 (oldest deadline 2024-09-12T02:19:43.498195+0000) 2024-09-12T02:19:57.128 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:56 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:56.858+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-12T02:19:26.999235+0000 front 2024-09-12T02:19:26.999249+0000 (oldest deadline 2024-09-12T02:19:52.299019+0000) 2024-09-12T02:19:57.419 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:57 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:57.146+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:42.394291+0000 front 2024-09-12T02:18:42.394247+0000 (oldest deadline 2024-09-12T02:19:05.894091+0000) 2024-09-12T02:19:57.419 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:57 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:57.146+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:47.595046+0000 front 2024-09-12T02:18:47.595165+0000 (oldest deadline 2024-09-12T02:19:12.295142+0000) 2024-09-12T02:19:57.419 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:57 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:57.146+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:58.696957+0000 front 2024-09-12T02:18:58.696791+0000 (oldest deadline 2024-09-12T02:19:22.196628+0000) 2024-09-12T02:19:57.419 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:57 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:57.146+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T02:19:05.097647+0000 front 2024-09-12T02:19:05.097663+0000 (oldest deadline 2024-09-12T02:19:28.498137+0000) 2024-09-12T02:19:57.420 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:57 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:57.146+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.33:6812 osd.4 since back 2024-09-12T02:19:11.899193+0000 front 2024-09-12T02:19:11.899218+0000 (oldest deadline 2024-09-12T02:19:35.999293+0000) 2024-09-12T02:19:57.420 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:57 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:57.146+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-09-12T02:19:20.000612+0000 front 2024-09-12T02:19:20.000719+0000 (oldest deadline 2024-09-12T02:19:45.900515+0000) 2024-09-12T02:19:57.420 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:57 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:57.146+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-12T02:19:25.901186+0000 front 2024-09-12T02:19:25.901318+0000 (oldest deadline 2024-09-12T02:19:51.800879+0000) 2024-09-12T02:19:58.127 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:57 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:57.873+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:40.491334+0000 front 2024-09-12T02:18:40.491223+0000 (oldest deadline 2024-09-12T02:19:05.191278+0000) 2024-09-12T02:19:58.128 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:57 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:57.873+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:49.192960+0000 front 2024-09-12T02:18:49.193116+0000 (oldest deadline 2024-09-12T02:19:15.092803+0000) 2024-09-12T02:19:58.128 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:57 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:57.873+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:58.593805+0000 front 2024-09-12T02:18:58.593869+0000 (oldest deadline 2024-09-12T02:19:19.594390+0000) 2024-09-12T02:19:58.128 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:57 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:57.873+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.33:6812 osd.4 since back 2024-09-12T02:19:14.097458+0000 front 2024-09-12T02:19:14.097525+0000 (oldest deadline 2024-09-12T02:19:38.197566+0000) 2024-09-12T02:19:58.128 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:57 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:57.873+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-09-12T02:19:18.197917+0000 front 2024-09-12T02:19:18.198027+0000 (oldest deadline 2024-09-12T02:19:43.498195+0000) 2024-09-12T02:19:58.128 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:57 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:57.873+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-12T02:19:26.999235+0000 front 2024-09-12T02:19:26.999249+0000 (oldest deadline 2024-09-12T02:19:52.299019+0000) 2024-09-12T02:19:58.419 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:58 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:58.117+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:42.394291+0000 front 2024-09-12T02:18:42.394247+0000 (oldest deadline 2024-09-12T02:19:05.894091+0000) 2024-09-12T02:19:58.419 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:58 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:58.117+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:47.595046+0000 front 2024-09-12T02:18:47.595165+0000 (oldest deadline 2024-09-12T02:19:12.295142+0000) 2024-09-12T02:19:58.419 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:58 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:58.117+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:58.696957+0000 front 2024-09-12T02:18:58.696791+0000 (oldest deadline 2024-09-12T02:19:22.196628+0000) 2024-09-12T02:19:58.419 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:58 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:58.117+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T02:19:05.097647+0000 front 2024-09-12T02:19:05.097663+0000 (oldest deadline 2024-09-12T02:19:28.498137+0000) 2024-09-12T02:19:58.420 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:58 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:58.117+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.33:6812 osd.4 since back 2024-09-12T02:19:11.899193+0000 front 2024-09-12T02:19:11.899218+0000 (oldest deadline 2024-09-12T02:19:35.999293+0000) 2024-09-12T02:19:58.420 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:58 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:58.117+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-09-12T02:19:20.000612+0000 front 2024-09-12T02:19:20.000719+0000 (oldest deadline 2024-09-12T02:19:45.900515+0000) 2024-09-12T02:19:58.420 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:58 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:58.117+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-12T02:19:25.901186+0000 front 2024-09-12T02:19:25.901318+0000 (oldest deadline 2024-09-12T02:19:51.800879+0000) 2024-09-12T02:19:59.377 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:58 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:58.886+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:40.491334+0000 front 2024-09-12T02:18:40.491223+0000 (oldest deadline 2024-09-12T02:19:05.191278+0000) 2024-09-12T02:19:59.378 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:58 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:58.886+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:49.192960+0000 front 2024-09-12T02:18:49.193116+0000 (oldest deadline 2024-09-12T02:19:15.092803+0000) 2024-09-12T02:19:59.378 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:58 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:58.886+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:58.593805+0000 front 2024-09-12T02:18:58.593869+0000 (oldest deadline 2024-09-12T02:19:19.594390+0000) 2024-09-12T02:19:59.378 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:58 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:58.886+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.33:6812 osd.4 since back 2024-09-12T02:19:14.097458+0000 front 2024-09-12T02:19:14.097525+0000 (oldest deadline 2024-09-12T02:19:38.197566+0000) 2024-09-12T02:19:59.378 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:58 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:58.886+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-09-12T02:19:18.197917+0000 front 2024-09-12T02:19:18.198027+0000 (oldest deadline 2024-09-12T02:19:43.498195+0000) 2024-09-12T02:19:59.378 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:58 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:58.886+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-12T02:19:26.999235+0000 front 2024-09-12T02:19:26.999249+0000 (oldest deadline 2024-09-12T02:19:52.299019+0000) 2024-09-12T02:19:59.378 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:58 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:58.886+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.33:6820 osd.7 since back 2024-09-12T02:19:37.600208+0000 front 2024-09-12T02:19:37.600166+0000 (oldest deadline 2024-09-12T02:19:58.099954+0000) 2024-09-12T02:19:59.419 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:59 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:59.167+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:42.394291+0000 front 2024-09-12T02:18:42.394247+0000 (oldest deadline 2024-09-12T02:19:05.894091+0000) 2024-09-12T02:19:59.419 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:59 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:59.167+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:47.595046+0000 front 2024-09-12T02:18:47.595165+0000 (oldest deadline 2024-09-12T02:19:12.295142+0000) 2024-09-12T02:19:59.419 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:59 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:59.167+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:58.696957+0000 front 2024-09-12T02:18:58.696791+0000 (oldest deadline 2024-09-12T02:19:22.196628+0000) 2024-09-12T02:19:59.419 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:59 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:59.167+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T02:19:05.097647+0000 front 2024-09-12T02:19:05.097663+0000 (oldest deadline 2024-09-12T02:19:28.498137+0000) 2024-09-12T02:19:59.419 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:59 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:59.167+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.33:6812 osd.4 since back 2024-09-12T02:19:11.899193+0000 front 2024-09-12T02:19:11.899218+0000 (oldest deadline 2024-09-12T02:19:35.999293+0000) 2024-09-12T02:19:59.420 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:59 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:59.167+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-09-12T02:19:20.000612+0000 front 2024-09-12T02:19:20.000719+0000 (oldest deadline 2024-09-12T02:19:45.900515+0000) 2024-09-12T02:19:59.420 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:19:59 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:19:59.167+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-12T02:19:25.901186+0000 front 2024-09-12T02:19:25.901318+0000 (oldest deadline 2024-09-12T02:19:51.800879+0000) 2024-09-12T02:20:00.378 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:59 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:59.897+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:40.491334+0000 front 2024-09-12T02:18:40.491223+0000 (oldest deadline 2024-09-12T02:19:05.191278+0000) 2024-09-12T02:20:00.545 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:59 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:59.897+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:49.192960+0000 front 2024-09-12T02:18:49.193116+0000 (oldest deadline 2024-09-12T02:19:15.092803+0000) 2024-09-12T02:20:00.545 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:59 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:59.897+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:58.593805+0000 front 2024-09-12T02:18:58.593869+0000 (oldest deadline 2024-09-12T02:19:19.594390+0000) 2024-09-12T02:20:00.545 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:59 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:59.897+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.33:6812 osd.4 since back 2024-09-12T02:19:14.097458+0000 front 2024-09-12T02:19:14.097525+0000 (oldest deadline 2024-09-12T02:19:38.197566+0000) 2024-09-12T02:20:00.545 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:59 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:59.897+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-09-12T02:19:18.197917+0000 front 2024-09-12T02:19:18.198027+0000 (oldest deadline 2024-09-12T02:19:43.498195+0000) 2024-09-12T02:20:00.545 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:59 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:59.897+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-12T02:19:26.999235+0000 front 2024-09-12T02:19:26.999249+0000 (oldest deadline 2024-09-12T02:19:52.299019+0000) 2024-09-12T02:20:00.545 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:19:59 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:19:59.897+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.33:6820 osd.7 since back 2024-09-12T02:19:37.600208+0000 front 2024-09-12T02:19:37.600166+0000 (oldest deadline 2024-09-12T02:19:58.099954+0000) 2024-09-12T02:20:00.669 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:20:00 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:20:00.178+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:42.394291+0000 front 2024-09-12T02:18:42.394247+0000 (oldest deadline 2024-09-12T02:19:05.894091+0000) 2024-09-12T02:20:00.669 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:20:00 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:20:00.178+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:47.595046+0000 front 2024-09-12T02:18:47.595165+0000 (oldest deadline 2024-09-12T02:19:12.295142+0000) 2024-09-12T02:20:00.669 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:20:00 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:20:00.178+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:58.696957+0000 front 2024-09-12T02:18:58.696791+0000 (oldest deadline 2024-09-12T02:19:22.196628+0000) 2024-09-12T02:20:00.670 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:20:00 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:20:00.178+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T02:19:05.097647+0000 front 2024-09-12T02:19:05.097663+0000 (oldest deadline 2024-09-12T02:19:28.498137+0000) 2024-09-12T02:20:00.670 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:20:00 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:20:00.178+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.33:6812 osd.4 since back 2024-09-12T02:19:11.899193+0000 front 2024-09-12T02:19:11.899218+0000 (oldest deadline 2024-09-12T02:19:35.999293+0000) 2024-09-12T02:20:00.670 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:20:00 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:20:00.178+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-09-12T02:19:20.000612+0000 front 2024-09-12T02:19:20.000719+0000 (oldest deadline 2024-09-12T02:19:45.900515+0000) 2024-09-12T02:20:00.670 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:20:00 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10[59621]: 2024-09-12T02:20:00.178+0000 7f78cd0d3640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-12T02:19:25.901186+0000 front 2024-09-12T02:19:25.901318+0000 (oldest deadline 2024-09-12T02:19:51.800879+0000) 2024-09-12T02:20:01.127 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:20:00 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:20:00.858+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:40.491334+0000 front 2024-09-12T02:18:40.491223+0000 (oldest deadline 2024-09-12T02:19:05.191278+0000) 2024-09-12T02:20:01.128 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:20:00 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:20:00.858+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:49.192960+0000 front 2024-09-12T02:18:49.193116+0000 (oldest deadline 2024-09-12T02:19:15.092803+0000) 2024-09-12T02:20:01.128 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:20:00 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:20:00.858+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:58.593805+0000 front 2024-09-12T02:18:58.593869+0000 (oldest deadline 2024-09-12T02:19:19.594390+0000) 2024-09-12T02:20:01.128 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:20:00 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:20:00.858+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.33:6812 osd.4 since back 2024-09-12T02:19:14.097458+0000 front 2024-09-12T02:19:14.097525+0000 (oldest deadline 2024-09-12T02:19:38.197566+0000) 2024-09-12T02:20:01.128 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:20:00 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:20:00.858+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-09-12T02:19:18.197917+0000 front 2024-09-12T02:19:18.198027+0000 (oldest deadline 2024-09-12T02:19:43.498195+0000) 2024-09-12T02:20:01.128 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:20:00 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:20:00.858+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-12T02:19:26.999235+0000 front 2024-09-12T02:19:26.999249+0000 (oldest deadline 2024-09-12T02:19:52.299019+0000) 2024-09-12T02:20:01.128 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:20:00 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:20:00.858+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.33:6820 osd.7 since back 2024-09-12T02:19:37.600208+0000 front 2024-09-12T02:19:37.600166+0000 (oldest deadline 2024-09-12T02:19:58.099954+0000) 2024-09-12T02:20:01.669 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:20:01 smithi033 podman[70958]: 2024-09-12 02:20:01.396159757 +0000 UTC m=+5.612154324 container died a3ae324e478e285eb07d88d34dff25ccaca999cae8ffebf3655156529f4bda12 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.37.2) 2024-09-12T02:20:02.169 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:20:01 smithi033 podman[70958]: 2024-09-12 02:20:01.71602377 +0000 UTC m=+5.932018337 container cleanup a3ae324e478e285eb07d88d34dff25ccaca999cae8ffebf3655156529f4bda12 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, ceph=True, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2) 2024-09-12T02:20:02.169 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:20:01 smithi033 bash[70958]: ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10 2024-09-12T02:20:02.169 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:20:01 smithi033 podman[70975]: 2024-09-12 02:20:01.909006243 +0000 UTC m=+0.506584906 container remove a3ae324e478e285eb07d88d34dff25ccaca999cae8ffebf3655156529f4bda12 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3) 2024-09-12T02:20:02.378 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:20:01 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:20:01.891+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:40.491334+0000 front 2024-09-12T02:18:40.491223+0000 (oldest deadline 2024-09-12T02:19:05.191278+0000) 2024-09-12T02:20:02.378 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:20:01 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:20:01.891+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:49.192960+0000 front 2024-09-12T02:18:49.193116+0000 (oldest deadline 2024-09-12T02:19:15.092803+0000) 2024-09-12T02:20:02.378 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:20:01 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:20:01.891+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:58.593805+0000 front 2024-09-12T02:18:58.593869+0000 (oldest deadline 2024-09-12T02:19:19.594390+0000) 2024-09-12T02:20:02.378 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:20:01 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:20:01.891+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.33:6812 osd.4 since back 2024-09-12T02:19:14.097458+0000 front 2024-09-12T02:19:14.097525+0000 (oldest deadline 2024-09-12T02:19:38.197566+0000) 2024-09-12T02:20:02.378 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:20:01 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:20:01.891+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-09-12T02:19:18.197917+0000 front 2024-09-12T02:19:18.198027+0000 (oldest deadline 2024-09-12T02:19:43.498195+0000) 2024-09-12T02:20:02.379 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:20:01 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:20:01.891+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-12T02:19:26.999235+0000 front 2024-09-12T02:19:26.999249+0000 (oldest deadline 2024-09-12T02:19:52.299019+0000) 2024-09-12T02:20:02.379 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:20:01 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:20:01.891+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.33:6820 osd.7 since back 2024-09-12T02:19:37.600208+0000 front 2024-09-12T02:19:37.600166+0000 (oldest deadline 2024-09-12T02:19:58.099954+0000) 2024-09-12T02:20:02.503 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:20:02 smithi033 podman[71061]: 2024-09-12 02:20:02.226381377 +0000 UTC m=+0.108717422 container create 2c3600a80312da79fec4d1d5a1e97271c927201fcad7dd78ad6da725fee29499 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10-deactivate, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240731, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-09-12T02:20:02.503 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:20:02 smithi033 podman[71061]: 2024-09-12 02:20:02.133402308 +0000 UTC m=+0.015738361 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T02:20:02.503 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:20:02 smithi033 podman[71061]: 2024-09-12 02:20:02.437140048 +0000 UTC m=+0.319476094 container init 2c3600a80312da79fec4d1d5a1e97271c927201fcad7dd78ad6da725fee29499 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10-deactivate, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD) 2024-09-12T02:20:02.503 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:20:02 smithi033 podman[71061]: 2024-09-12 02:20:02.441059279 +0000 UTC m=+0.323395330 container start 2c3600a80312da79fec4d1d5a1e97271c927201fcad7dd78ad6da725fee29499 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10-deactivate, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-09-12T02:20:02.504 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:20:02 smithi033 podman[71061]: 2024-09-12 02:20:02.503707011 +0000 UTC m=+0.386043060 container attach 2c3600a80312da79fec4d1d5a1e97271c927201fcad7dd78ad6da725fee29499 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10-deactivate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T02:20:02.919 INFO:journalctl@ceph.osd.10.smithi033.stdout:Sep 12 02:20:02 smithi033 podman[71061]: 2024-09-12 02:20:02.632176566 +0000 UTC m=+0.514512616 container died 2c3600a80312da79fec4d1d5a1e97271c927201fcad7dd78ad6da725fee29499 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-10-deactivate, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T02:20:03.128 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:20:02 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:20:02.848+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:40.491334+0000 front 2024-09-12T02:18:40.491223+0000 (oldest deadline 2024-09-12T02:19:05.191278+0000) 2024-09-12T02:20:03.128 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:20:02 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:20:02.848+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:49.192960+0000 front 2024-09-12T02:18:49.193116+0000 (oldest deadline 2024-09-12T02:19:15.092803+0000) 2024-09-12T02:20:03.128 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:20:02 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:20:02.848+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:58.593805+0000 front 2024-09-12T02:18:58.593869+0000 (oldest deadline 2024-09-12T02:19:19.594390+0000) 2024-09-12T02:20:03.128 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:20:02 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:20:02.848+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.33:6812 osd.4 since back 2024-09-12T02:19:14.097458+0000 front 2024-09-12T02:19:14.097525+0000 (oldest deadline 2024-09-12T02:19:38.197566+0000) 2024-09-12T02:20:03.128 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:20:02 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:20:02.848+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-09-12T02:19:18.197917+0000 front 2024-09-12T02:19:18.198027+0000 (oldest deadline 2024-09-12T02:19:43.498195+0000) 2024-09-12T02:20:03.128 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:20:02 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:20:02.848+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-12T02:19:26.999235+0000 front 2024-09-12T02:19:26.999249+0000 (oldest deadline 2024-09-12T02:19:52.299019+0000) 2024-09-12T02:20:03.128 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:20:02 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:20:02.848+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.33:6820 osd.7 since back 2024-09-12T02:19:37.600208+0000 front 2024-09-12T02:19:37.600166+0000 (oldest deadline 2024-09-12T02:19:58.099954+0000) 2024-09-12T02:20:03.337 DEBUG:teuthology.orchestra.run.smithi033:> sudo pkill -f 'journalctl -f -n 0 -u ceph-879f3b56-70a9-11ef-bcea-c7b262605968@osd.10.service' 2024-09-12T02:20:03.377 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T02:20:03.377 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-09-12T02:20:03.377 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-09-12T02:20:03.378 DEBUG:teuthology.orchestra.run.smithi184:> sudo systemctl stop ceph-879f3b56-70a9-11ef-bcea-c7b262605968@osd.11 2024-09-12T02:20:03.667 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:20:03 smithi184 systemd[1]: Stopping Ceph osd.11 for 879f3b56-70a9-11ef-bcea-c7b262605968... 2024-09-12T02:20:04.128 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:20:03 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:20:03.666+0000 7f9d2621f640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T02:20:04.128 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:20:03 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:20:03.666+0000 7f9d2621f640 -1 osd.11 85 *** Got signal Terminated *** 2024-09-12T02:20:04.128 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:20:03 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:20:03.666+0000 7f9d2621f640 -1 osd.11 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T02:20:04.128 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:20:03 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:20:03.839+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:40.491334+0000 front 2024-09-12T02:18:40.491223+0000 (oldest deadline 2024-09-12T02:19:05.191278+0000) 2024-09-12T02:20:04.128 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:20:03 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:20:03.839+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:49.192960+0000 front 2024-09-12T02:18:49.193116+0000 (oldest deadline 2024-09-12T02:19:15.092803+0000) 2024-09-12T02:20:04.128 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:20:03 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:20:03.839+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:58.593805+0000 front 2024-09-12T02:18:58.593869+0000 (oldest deadline 2024-09-12T02:19:19.594390+0000) 2024-09-12T02:20:04.128 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:20:03 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:20:03.839+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.33:6812 osd.4 since back 2024-09-12T02:19:14.097458+0000 front 2024-09-12T02:19:14.097525+0000 (oldest deadline 2024-09-12T02:19:38.197566+0000) 2024-09-12T02:20:04.128 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:20:03 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:20:03.839+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-09-12T02:19:18.197917+0000 front 2024-09-12T02:19:18.198027+0000 (oldest deadline 2024-09-12T02:19:43.498195+0000) 2024-09-12T02:20:04.128 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:20:03 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:20:03.839+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-12T02:19:26.999235+0000 front 2024-09-12T02:19:26.999249+0000 (oldest deadline 2024-09-12T02:19:52.299019+0000) 2024-09-12T02:20:04.129 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:20:03 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:20:03.839+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.33:6820 osd.7 since back 2024-09-12T02:19:37.600208+0000 front 2024-09-12T02:19:37.600166+0000 (oldest deadline 2024-09-12T02:19:58.099954+0000) 2024-09-12T02:20:05.127 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:20:04 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:20:04.832+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:40.491334+0000 front 2024-09-12T02:18:40.491223+0000 (oldest deadline 2024-09-12T02:19:05.191278+0000) 2024-09-12T02:20:05.127 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:20:04 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:20:04.832+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:49.192960+0000 front 2024-09-12T02:18:49.193116+0000 (oldest deadline 2024-09-12T02:19:15.092803+0000) 2024-09-12T02:20:05.128 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:20:04 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:20:04.832+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:58.593805+0000 front 2024-09-12T02:18:58.593869+0000 (oldest deadline 2024-09-12T02:19:19.594390+0000) 2024-09-12T02:20:05.128 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:20:04 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:20:04.832+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.33:6812 osd.4 since back 2024-09-12T02:19:14.097458+0000 front 2024-09-12T02:19:14.097525+0000 (oldest deadline 2024-09-12T02:19:38.197566+0000) 2024-09-12T02:20:05.128 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:20:04 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:20:04.832+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-09-12T02:19:18.197917+0000 front 2024-09-12T02:19:18.198027+0000 (oldest deadline 2024-09-12T02:19:43.498195+0000) 2024-09-12T02:20:05.128 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:20:04 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:20:04.832+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-12T02:19:26.999235+0000 front 2024-09-12T02:19:26.999249+0000 (oldest deadline 2024-09-12T02:19:52.299019+0000) 2024-09-12T02:20:05.128 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:20:04 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:20:04.832+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.33:6820 osd.7 since back 2024-09-12T02:19:37.600208+0000 front 2024-09-12T02:19:37.600166+0000 (oldest deadline 2024-09-12T02:19:58.099954+0000) 2024-09-12T02:20:06.128 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:20:05 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:20:05.853+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:40.491334+0000 front 2024-09-12T02:18:40.491223+0000 (oldest deadline 2024-09-12T02:19:05.191278+0000) 2024-09-12T02:20:06.128 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:20:05 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:20:05.853+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:49.192960+0000 front 2024-09-12T02:18:49.193116+0000 (oldest deadline 2024-09-12T02:19:15.092803+0000) 2024-09-12T02:20:06.128 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:20:05 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:20:05.853+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:58.593805+0000 front 2024-09-12T02:18:58.593869+0000 (oldest deadline 2024-09-12T02:19:19.594390+0000) 2024-09-12T02:20:06.128 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:20:05 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:20:05.853+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.33:6812 osd.4 since back 2024-09-12T02:19:14.097458+0000 front 2024-09-12T02:19:14.097525+0000 (oldest deadline 2024-09-12T02:19:38.197566+0000) 2024-09-12T02:20:06.128 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:20:05 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:20:05.853+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-09-12T02:19:18.197917+0000 front 2024-09-12T02:19:18.198027+0000 (oldest deadline 2024-09-12T02:19:43.498195+0000) 2024-09-12T02:20:06.128 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:20:05 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:20:05.853+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-12T02:19:26.999235+0000 front 2024-09-12T02:19:26.999249+0000 (oldest deadline 2024-09-12T02:19:52.299019+0000) 2024-09-12T02:20:06.128 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:20:05 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:20:05.853+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.33:6820 osd.7 since back 2024-09-12T02:19:37.600208+0000 front 2024-09-12T02:19:37.600166+0000 (oldest deadline 2024-09-12T02:19:58.099954+0000) 2024-09-12T02:20:07.127 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:20:06 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:20:06.835+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:40.491334+0000 front 2024-09-12T02:18:40.491223+0000 (oldest deadline 2024-09-12T02:19:05.191278+0000) 2024-09-12T02:20:07.128 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:20:06 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:20:06.835+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:49.192960+0000 front 2024-09-12T02:18:49.193116+0000 (oldest deadline 2024-09-12T02:19:15.092803+0000) 2024-09-12T02:20:07.128 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:20:06 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:20:06.835+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:58.593805+0000 front 2024-09-12T02:18:58.593869+0000 (oldest deadline 2024-09-12T02:19:19.594390+0000) 2024-09-12T02:20:07.128 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:20:06 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:20:06.835+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.33:6812 osd.4 since back 2024-09-12T02:19:14.097458+0000 front 2024-09-12T02:19:14.097525+0000 (oldest deadline 2024-09-12T02:19:38.197566+0000) 2024-09-12T02:20:07.128 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:20:06 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:20:06.835+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-09-12T02:19:18.197917+0000 front 2024-09-12T02:19:18.198027+0000 (oldest deadline 2024-09-12T02:19:43.498195+0000) 2024-09-12T02:20:07.128 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:20:06 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:20:06.835+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-12T02:19:26.999235+0000 front 2024-09-12T02:19:26.999249+0000 (oldest deadline 2024-09-12T02:19:52.299019+0000) 2024-09-12T02:20:07.128 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:20:06 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:20:06.835+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.33:6820 osd.7 since back 2024-09-12T02:19:37.600208+0000 front 2024-09-12T02:19:37.600166+0000 (oldest deadline 2024-09-12T02:19:58.099954+0000) 2024-09-12T02:20:08.127 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:20:07 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:20:07.835+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T02:18:40.491334+0000 front 2024-09-12T02:18:40.491223+0000 (oldest deadline 2024-09-12T02:19:05.191278+0000) 2024-09-12T02:20:08.128 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:20:07 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:20:07.835+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-09-12T02:18:49.192960+0000 front 2024-09-12T02:18:49.193116+0000 (oldest deadline 2024-09-12T02:19:15.092803+0000) 2024-09-12T02:20:08.128 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:20:07 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:20:07.835+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-09-12T02:18:58.593805+0000 front 2024-09-12T02:18:58.593869+0000 (oldest deadline 2024-09-12T02:19:19.594390+0000) 2024-09-12T02:20:08.128 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:20:07 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:20:07.835+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.33:6812 osd.4 since back 2024-09-12T02:19:14.097458+0000 front 2024-09-12T02:19:14.097525+0000 (oldest deadline 2024-09-12T02:19:38.197566+0000) 2024-09-12T02:20:08.128 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:20:07 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:20:07.835+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-09-12T02:19:18.197917+0000 front 2024-09-12T02:19:18.198027+0000 (oldest deadline 2024-09-12T02:19:43.498195+0000) 2024-09-12T02:20:08.128 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:20:07 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:20:07.835+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-12T02:19:26.999235+0000 front 2024-09-12T02:19:26.999249+0000 (oldest deadline 2024-09-12T02:19:52.299019+0000) 2024-09-12T02:20:08.128 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:20:07 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11[61049]: 2024-09-12T02:20:07.835+0000 7f9d1e81c640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.33:6820 osd.7 since back 2024-09-12T02:19:37.600208+0000 front 2024-09-12T02:19:37.600166+0000 (oldest deadline 2024-09-12T02:19:58.099954+0000) 2024-09-12T02:20:09.417 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:20:09 smithi184 podman[73307]: 2024-09-12 02:20:09.149514292 +0000 UTC m=+5.581750326 container died f8be75c18662110d20d47031737794a522d35a5ffb5bf0fe0695ac02546e69e7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-09-12T02:20:09.792 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:20:09 smithi184 podman[73307]: 2024-09-12 02:20:09.524800387 +0000 UTC m=+5.957036424 container cleanup f8be75c18662110d20d47031737794a522d35a5ffb5bf0fe0695ac02546e69e7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux ) 2024-09-12T02:20:09.792 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:20:09 smithi184 bash[73307]: ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11 2024-09-12T02:20:10.127 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:20:09 smithi184 podman[73323]: 2024-09-12 02:20:09.792467064 +0000 UTC m=+0.638601744 container remove f8be75c18662110d20d47031737794a522d35a5ffb5bf0fe0695ac02546e69e7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240731, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-09-12T02:20:10.453 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:20:10 smithi184 podman[73413]: 2024-09-12 02:20:10.045847884 +0000 UTC m=+0.016017382 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T02:20:10.453 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:20:10 smithi184 podman[73413]: 2024-09-12 02:20:10.159845644 +0000 UTC m=+0.130015165 container create 05d7442bfd0c18668e6873a10ab8c68c6cd437161330791e76ba6a4f68cbc29c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11-deactivate, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731) 2024-09-12T02:20:10.454 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:20:10 smithi184 podman[73413]: 2024-09-12 02:20:10.370541186 +0000 UTC m=+0.340710661 container init 05d7442bfd0c18668e6873a10ab8c68c6cd437161330791e76ba6a4f68cbc29c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11-deactivate, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS) 2024-09-12T02:20:10.454 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:20:10 smithi184 podman[73413]: 2024-09-12 02:20:10.373787779 +0000 UTC m=+0.343957255 container start 05d7442bfd0c18668e6873a10ab8c68c6cd437161330791e76ba6a4f68cbc29c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-09-12T02:20:10.454 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:20:10 smithi184 podman[73413]: 2024-09-12 02:20:10.453990436 +0000 UTC m=+0.424159931 container attach 05d7442bfd0c18668e6873a10ab8c68c6cd437161330791e76ba6a4f68cbc29c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11-deactivate, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-09-12T02:20:10.848 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:20:10 smithi184 podman[73413]: 2024-09-12 02:20:10.566442325 +0000 UTC m=+0.536611806 container died 05d7442bfd0c18668e6873a10ab8c68c6cd437161330791e76ba6a4f68cbc29c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11-deactivate, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T02:20:11.077 DEBUG:teuthology.orchestra.run.smithi184:> sudo pkill -f 'journalctl -f -n 0 -u ceph-879f3b56-70a9-11ef-bcea-c7b262605968@osd.11.service' 2024-09-12T02:20:11.101 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:20:11 smithi184 podman[73436]: 2024-09-12 02:20:11.057116967 +0000 UTC m=+0.484960454 container remove 05d7442bfd0c18668e6873a10ab8c68c6cd437161330791e76ba6a4f68cbc29c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-osd-11-deactivate, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux ) 2024-09-12T02:20:11.101 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:20:11 smithi184 systemd[1]: ceph-879f3b56-70a9-11ef-bcea-c7b262605968@osd.11.service: Deactivated successfully. 2024-09-12T02:20:11.101 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:20:11 smithi184 systemd[1]: Stopped Ceph osd.11 for 879f3b56-70a9-11ef-bcea-c7b262605968. 2024-09-12T02:20:11.101 INFO:journalctl@ceph.osd.11.smithi184.stdout:Sep 12 02:20:11 smithi184 systemd[1]: ceph-879f3b56-70a9-11ef-bcea-c7b262605968@osd.11.service: Consumed 9.427s CPU time. 2024-09-12T02:20:11.116 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T02:20:11.117 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-09-12T02:20:11.117 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-09-12T02:20:11.117 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ceph-879f3b56-70a9-11ef-bcea-c7b262605968@mds.a 2024-09-12T02:20:11.429 INFO:journalctl@ceph.mds.a.smithi016.stdout:Sep 12 02:20:11 smithi016 systemd[1]: Stopping Ceph mds.a for 879f3b56-70a9-11ef-bcea-c7b262605968... 2024-09-12T02:20:11.684 INFO:journalctl@ceph.mds.a.smithi016.stdout:Sep 12 02:20:11 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mds-a[72508]: 2024-09-12T02:20:11.427+0000 7fadd2d57640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T02:20:11.684 INFO:journalctl@ceph.mds.a.smithi016.stdout:Sep 12 02:20:11 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mds-a[72508]: 2024-09-12T02:20:11.427+0000 7fadd2d57640 -1 mds.a *** got signal Terminated *** 2024-09-12T02:20:15.695 INFO:journalctl@ceph.mds.a.smithi016.stdout:Sep 12 02:20:15 smithi016 podman[131804]: 2024-09-12 02:20:15.440043023 +0000 UTC m=+4.114744751 container died 14f2a6d3d79fb4df275519b336effe3a68ab8a4d720711b7d992f219e99cf9f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mds-a, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2) 2024-09-12T02:20:16.037 INFO:journalctl@ceph.mds.a.smithi016.stdout:Sep 12 02:20:15 smithi016 podman[131804]: 2024-09-12 02:20:15.869197139 +0000 UTC m=+4.543898857 container cleanup 14f2a6d3d79fb4df275519b336effe3a68ab8a4d720711b7d992f219e99cf9f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mds-a, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, org.label-schema.license=GPLv2) 2024-09-12T02:20:16.037 INFO:journalctl@ceph.mds.a.smithi016.stdout:Sep 12 02:20:15 smithi016 bash[131804]: ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mds-a 2024-09-12T02:20:16.252 DEBUG:teuthology.orchestra.run.smithi016:> sudo pkill -f 'journalctl -f -n 0 -u ceph-879f3b56-70a9-11ef-bcea-c7b262605968@mds.a.service' 2024-09-12T02:20:16.289 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T02:20:16.289 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-09-12T02:20:16.289 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-09-12T02:20:16.290 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ceph-879f3b56-70a9-11ef-bcea-c7b262605968@mds.d 2024-09-12T02:20:16.632 INFO:journalctl@ceph.mds.d.smithi016.stdout:Sep 12 02:20:16 smithi016 systemd[1]: Stopping Ceph mds.d for 879f3b56-70a9-11ef-bcea-c7b262605968... 2024-09-12T02:20:16.933 INFO:journalctl@ceph.mds.d.smithi016.stdout:Sep 12 02:20:16 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mds-d[71440]: 2024-09-12T02:20:16.631+0000 7f541a30b640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T02:20:16.933 INFO:journalctl@ceph.mds.d.smithi016.stdout:Sep 12 02:20:16 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mds-d[71440]: 2024-09-12T02:20:16.631+0000 7f541a30b640 -1 mds.d *** got signal Terminated *** 2024-09-12T02:20:17.933 INFO:journalctl@ceph.mds.d.smithi016.stdout:Sep 12 02:20:17 smithi016 podman[131924]: 2024-09-12 02:20:17.739031413 +0000 UTC m=+1.225094263 container died 0fcdf8c628453f89a0c0cbdc1791df24d93a3933bba94155544dcf9ca06e1ce0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mds-d, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T02:20:18.433 INFO:journalctl@ceph.mds.d.smithi016.stdout:Sep 12 02:20:18 smithi016 podman[131924]: 2024-09-12 02:20:18.248779063 +0000 UTC m=+1.734841900 container cleanup 0fcdf8c628453f89a0c0cbdc1791df24d93a3933bba94155544dcf9ca06e1ce0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mds-d, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-09-12T02:20:18.433 INFO:journalctl@ceph.mds.d.smithi016.stdout:Sep 12 02:20:18 smithi016 bash[131924]: ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mds-d 2024-09-12T02:20:18.683 INFO:journalctl@ceph.mds.d.smithi016.stdout:Sep 12 02:20:18 smithi016 podman[131938]: 2024-09-12 02:20:18.433356041 +0000 UTC m=+0.695613747 container remove 0fcdf8c628453f89a0c0cbdc1791df24d93a3933bba94155544dcf9ca06e1ce0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mds-d, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3) 2024-09-12T02:20:18.690 DEBUG:teuthology.orchestra.run.smithi016:> sudo pkill -f 'journalctl -f -n 0 -u ceph-879f3b56-70a9-11ef-bcea-c7b262605968@mds.d.service' 2024-09-12T02:20:18.725 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T02:20:18.725 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-09-12T02:20:18.725 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-09-12T02:20:18.725 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ceph-879f3b56-70a9-11ef-bcea-c7b262605968@mds.g 2024-09-12T02:20:19.044 INFO:journalctl@ceph.mds.g.smithi016.stdout:Sep 12 02:20:18 smithi016 systemd[1]: Stopping Ceph mds.g for 879f3b56-70a9-11ef-bcea-c7b262605968... 2024-09-12T02:20:19.434 INFO:journalctl@ceph.mds.g.smithi016.stdout:Sep 12 02:20:19 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mds-g[72208]: 2024-09-12T02:20:19.043+0000 7f12870ee640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T02:20:19.434 INFO:journalctl@ceph.mds.g.smithi016.stdout:Sep 12 02:20:19 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mds-g[72208]: 2024-09-12T02:20:19.043+0000 7f12870ee640 -1 mds.g *** got signal Terminated *** 2024-09-12T02:20:21.395 INFO:journalctl@ceph.mds.g.smithi016.stdout:Sep 12 02:20:21 smithi016 podman[132045]: 2024-09-12 02:20:21.1026121 +0000 UTC m=+2.143596468 container died a1f810e1d0cc666e9d33050177490bb35423f6efa680d1d288bba97ce3f5463e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mds-g, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-09-12T02:20:21.683 INFO:journalctl@ceph.mds.g.smithi016.stdout:Sep 12 02:20:21 smithi016 podman[132045]: 2024-09-12 02:20:21.477662791 +0000 UTC m=+2.518647147 container cleanup a1f810e1d0cc666e9d33050177490bb35423f6efa680d1d288bba97ce3f5463e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mds-g, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-09-12T02:20:21.684 INFO:journalctl@ceph.mds.g.smithi016.stdout:Sep 12 02:20:21 smithi016 bash[132045]: ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mds-g 2024-09-12T02:20:21.684 INFO:journalctl@ceph.mds.g.smithi016.stdout:Sep 12 02:20:21 smithi016 podman[132056]: 2024-09-12 02:20:21.612332324 +0000 UTC m=+0.503558905 container remove a1f810e1d0cc666e9d33050177490bb35423f6efa680d1d288bba97ce3f5463e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mds-g, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True) 2024-09-12T02:20:21.839 DEBUG:teuthology.orchestra.run.smithi016:> sudo pkill -f 'journalctl -f -n 0 -u ceph-879f3b56-70a9-11ef-bcea-c7b262605968@mds.g.service' 2024-09-12T02:20:21.873 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T02:20:21.873 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-09-12T02:20:21.874 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-09-12T02:20:21.874 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ceph-879f3b56-70a9-11ef-bcea-c7b262605968@mds.j 2024-09-12T02:20:22.181 INFO:journalctl@ceph.mds.j.smithi016.stdout:Sep 12 02:20:21 smithi016 systemd[1]: Stopping Ceph mds.j for 879f3b56-70a9-11ef-bcea-c7b262605968... 2024-09-12T02:20:22.433 INFO:journalctl@ceph.mds.j.smithi016.stdout:Sep 12 02:20:22 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mds-j[71022]: 2024-09-12T02:20:22.180+0000 7fe7580ce640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T02:20:22.434 INFO:journalctl@ceph.mds.j.smithi016.stdout:Sep 12 02:20:22 smithi016 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mds-j[71022]: 2024-09-12T02:20:22.180+0000 7fe7580ce640 -1 mds.j *** got signal Terminated *** 2024-09-12T02:20:26.573 INFO:journalctl@ceph.mds.j.smithi016.stdout:Sep 12 02:20:26 smithi016 podman[132166]: 2024-09-12 02:20:26.190596023 +0000 UTC m=+4.095232125 container died 83e1c73e17f49f25005772cddb315a01007b657e86555f2ad9f91e04f8e5e1b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mds-j, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731) 2024-09-12T02:20:26.915 INFO:journalctl@ceph.mds.j.smithi016.stdout:Sep 12 02:20:26 smithi016 podman[132166]: 2024-09-12 02:20:26.689503373 +0000 UTC m=+4.594139467 container cleanup 83e1c73e17f49f25005772cddb315a01007b657e86555f2ad9f91e04f8e5e1b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mds-j, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731) 2024-09-12T02:20:26.915 INFO:journalctl@ceph.mds.j.smithi016.stdout:Sep 12 02:20:26 smithi016 bash[132166]: ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mds-j 2024-09-12T02:20:27.047 DEBUG:teuthology.orchestra.run.smithi016:> sudo pkill -f 'journalctl -f -n 0 -u ceph-879f3b56-70a9-11ef-bcea-c7b262605968@mds.j.service' 2024-09-12T02:20:27.080 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T02:20:27.080 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-09-12T02:20:27.080 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-09-12T02:20:27.081 DEBUG:teuthology.orchestra.run.smithi033:> sudo systemctl stop ceph-879f3b56-70a9-11ef-bcea-c7b262605968@mds.b 2024-09-12T02:20:27.374 INFO:journalctl@ceph.mds.b.smithi033.stdout:Sep 12 02:20:27 smithi033 systemd[1]: Stopping Ceph mds.b for 879f3b56-70a9-11ef-bcea-c7b262605968... 2024-09-12T02:20:27.669 INFO:journalctl@ceph.mds.b.smithi033.stdout:Sep 12 02:20:27 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mds-b[65151]: 2024-09-12T02:20:27.374+0000 7faed997b640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T02:20:27.669 INFO:journalctl@ceph.mds.b.smithi033.stdout:Sep 12 02:20:27 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mds-b[65151]: 2024-09-12T02:20:27.374+0000 7faed997b640 -1 mds.b *** got signal Terminated *** 2024-09-12T02:20:31.419 INFO:journalctl@ceph.mds.b.smithi033.stdout:Sep 12 02:20:31 smithi033 podman[71162]: 2024-09-12 02:20:31.159909354 +0000 UTC m=+3.882260384 container died 4ddf0b353ec0366be99faa634728709f2c63fe2866dff0d54aa19d1bc53df8d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mds-b, ceph=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T02:20:31.858 INFO:journalctl@ceph.mds.b.smithi033.stdout:Sep 12 02:20:31 smithi033 podman[71162]: 2024-09-12 02:20:31.440580476 +0000 UTC m=+4.162931547 container cleanup 4ddf0b353ec0366be99faa634728709f2c63fe2866dff0d54aa19d1bc53df8d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mds-b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-09-12T02:20:31.858 INFO:journalctl@ceph.mds.b.smithi033.stdout:Sep 12 02:20:31 smithi033 bash[71162]: ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mds-b 2024-09-12T02:20:31.858 INFO:journalctl@ceph.mds.b.smithi033.stdout:Sep 12 02:20:31 smithi033 podman[71186]: 2024-09-12 02:20:31.641698141 +0000 UTC m=+0.478980028 container remove 4ddf0b353ec0366be99faa634728709f2c63fe2866dff0d54aa19d1bc53df8d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mds-b, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0) 2024-09-12T02:20:31.873 DEBUG:teuthology.orchestra.run.smithi033:> sudo pkill -f 'journalctl -f -n 0 -u ceph-879f3b56-70a9-11ef-bcea-c7b262605968@mds.b.service' 2024-09-12T02:20:31.909 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T02:20:31.909 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-09-12T02:20:31.909 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-09-12T02:20:31.909 DEBUG:teuthology.orchestra.run.smithi033:> sudo systemctl stop ceph-879f3b56-70a9-11ef-bcea-c7b262605968@mds.e 2024-09-12T02:20:32.169 INFO:journalctl@ceph.mds.e.smithi033.stdout:Sep 12 02:20:31 smithi033 systemd[1]: Stopping Ceph mds.e for 879f3b56-70a9-11ef-bcea-c7b262605968... 2024-09-12T02:20:32.669 INFO:journalctl@ceph.mds.e.smithi033.stdout:Sep 12 02:20:32 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mds-e[64545]: 2024-09-12T02:20:32.303+0000 7efd7c10f640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T02:20:32.669 INFO:journalctl@ceph.mds.e.smithi033.stdout:Sep 12 02:20:32 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mds-e[64545]: 2024-09-12T02:20:32.303+0000 7efd7c10f640 -1 mds.e *** got signal Terminated *** 2024-09-12T02:20:36.419 INFO:journalctl@ceph.mds.e.smithi033.stdout:Sep 12 02:20:36 smithi033 podman[71295]: 2024-09-12 02:20:36.168225119 +0000 UTC m=+4.038488559 container died ecbc94849c8c5af03e665944ba85e537b6bff8376f18bb0284fc6c5d4a11c6a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mds-e, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731) 2024-09-12T02:20:36.761 INFO:journalctl@ceph.mds.e.smithi033.stdout:Sep 12 02:20:36 smithi033 podman[71295]: 2024-09-12 02:20:36.527882347 +0000 UTC m=+4.398145784 container cleanup ecbc94849c8c5af03e665944ba85e537b6bff8376f18bb0284fc6c5d4a11c6a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mds-e, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731) 2024-09-12T02:20:36.762 INFO:journalctl@ceph.mds.e.smithi033.stdout:Sep 12 02:20:36 smithi033 bash[71295]: ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mds-e 2024-09-12T02:20:36.895 DEBUG:teuthology.orchestra.run.smithi033:> sudo pkill -f 'journalctl -f -n 0 -u ceph-879f3b56-70a9-11ef-bcea-c7b262605968@mds.e.service' 2024-09-12T02:20:36.929 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T02:20:36.929 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-09-12T02:20:36.929 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-09-12T02:20:36.929 DEBUG:teuthology.orchestra.run.smithi033:> sudo systemctl stop ceph-879f3b56-70a9-11ef-bcea-c7b262605968@mds.h 2024-09-12T02:20:37.298 INFO:journalctl@ceph.mds.h.smithi033.stdout:Sep 12 02:20:37 smithi033 systemd[1]: Stopping Ceph mds.h for 879f3b56-70a9-11ef-bcea-c7b262605968... 2024-09-12T02:20:37.669 INFO:journalctl@ceph.mds.h.smithi033.stdout:Sep 12 02:20:37 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mds-h[64849]: 2024-09-12T02:20:37.297+0000 7f2c59fe1640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T02:20:37.669 INFO:journalctl@ceph.mds.h.smithi033.stdout:Sep 12 02:20:37 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mds-h[64849]: 2024-09-12T02:20:37.297+0000 7f2c59fe1640 -1 mds.h *** got signal Terminated *** 2024-09-12T02:20:39.419 INFO:journalctl@ceph.mds.h.smithi033.stdout:Sep 12 02:20:39 smithi033 podman[71418]: 2024-09-12 02:20:39.168957233 +0000 UTC m=+2.012139466 container died 56f3bcd2417377c23473b54258745dda0a9d32ea9943504666fdb154c289629d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mds-h, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3) 2024-09-12T02:20:39.722 INFO:journalctl@ceph.mds.h.smithi033.stdout:Sep 12 02:20:39 smithi033 podman[71418]: 2024-09-12 02:20:39.506062442 +0000 UTC m=+2.349244675 container cleanup 56f3bcd2417377c23473b54258745dda0a9d32ea9943504666fdb154c289629d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mds-h, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, io.buildah.version=1.37.2) 2024-09-12T02:20:39.722 INFO:journalctl@ceph.mds.h.smithi033.stdout:Sep 12 02:20:39 smithi033 bash[71418]: ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mds-h 2024-09-12T02:20:40.068 INFO:journalctl@ceph.mds.h.smithi033.stdout:Sep 12 02:20:39 smithi033 podman[71430]: 2024-09-12 02:20:39.77631761 +0000 UTC m=+0.601753920 container remove 56f3bcd2417377c23473b54258745dda0a9d32ea9943504666fdb154c289629d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mds-h, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-09-12T02:20:40.085 DEBUG:teuthology.orchestra.run.smithi033:> sudo pkill -f 'journalctl -f -n 0 -u ceph-879f3b56-70a9-11ef-bcea-c7b262605968@mds.h.service' 2024-09-12T02:20:40.117 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T02:20:40.117 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-09-12T02:20:40.117 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-09-12T02:20:40.117 DEBUG:teuthology.orchestra.run.smithi033:> sudo systemctl stop ceph-879f3b56-70a9-11ef-bcea-c7b262605968@mds.k 2024-09-12T02:20:40.419 INFO:journalctl@ceph.mds.k.smithi033.stdout:Sep 12 02:20:40 smithi033 systemd[1]: Stopping Ceph mds.k for 879f3b56-70a9-11ef-bcea-c7b262605968... 2024-09-12T02:20:40.919 INFO:journalctl@ceph.mds.k.smithi033.stdout:Sep 12 02:20:40 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mds-k[64229]: 2024-09-12T02:20:40.426+0000 7f905248c640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T02:20:40.919 INFO:journalctl@ceph.mds.k.smithi033.stdout:Sep 12 02:20:40 smithi033 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mds-k[64229]: 2024-09-12T02:20:40.426+0000 7f905248c640 -1 mds.k *** got signal Terminated *** 2024-09-12T02:20:43.419 INFO:journalctl@ceph.mds.k.smithi033.stdout:Sep 12 02:20:43 smithi033 podman[71539]: 2024-09-12 02:20:43.168219952 +0000 UTC m=+2.836722587 container died 266f2fa2e5ecb5c485e3ab27d2df05088f0b3190b2d973d3678c6cf2370bf668 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mds-k, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True) 2024-09-12T02:20:43.802 INFO:journalctl@ceph.mds.k.smithi033.stdout:Sep 12 02:20:43 smithi033 podman[71539]: 2024-09-12 02:20:43.526978395 +0000 UTC m=+3.195481031 container cleanup 266f2fa2e5ecb5c485e3ab27d2df05088f0b3190b2d973d3678c6cf2370bf668 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mds-k, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-12T02:20:43.802 INFO:journalctl@ceph.mds.k.smithi033.stdout:Sep 12 02:20:43 smithi033 bash[71539]: ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mds-k 2024-09-12T02:20:43.951 DEBUG:teuthology.orchestra.run.smithi033:> sudo pkill -f 'journalctl -f -n 0 -u ceph-879f3b56-70a9-11ef-bcea-c7b262605968@mds.k.service' 2024-09-12T02:20:43.982 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T02:20:43.983 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-09-12T02:20:43.983 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-09-12T02:20:43.983 DEBUG:teuthology.orchestra.run.smithi184:> sudo systemctl stop ceph-879f3b56-70a9-11ef-bcea-c7b262605968@mds.c 2024-09-12T02:20:44.289 INFO:journalctl@ceph.mds.c.smithi184.stdout:Sep 12 02:20:44 smithi184 systemd[1]: Stopping Ceph mds.c for 879f3b56-70a9-11ef-bcea-c7b262605968... 2024-09-12T02:20:44.546 INFO:journalctl@ceph.mds.c.smithi184.stdout:Sep 12 02:20:44 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mds-c[66347]: 2024-09-12T02:20:44.288+0000 7f7292d01640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T02:20:44.546 INFO:journalctl@ceph.mds.c.smithi184.stdout:Sep 12 02:20:44 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mds-c[66347]: 2024-09-12T02:20:44.288+0000 7f7292d01640 -1 mds.c *** got signal Terminated *** 2024-09-12T02:20:47.628 INFO:journalctl@ceph.mds.c.smithi184.stdout:Sep 12 02:20:47 smithi184 podman[73515]: 2024-09-12 02:20:47.158182897 +0000 UTC m=+2.976478084 container died 89511b817656cdabf3dd5099dca77de03f11c21ab58eba3be482c8b8bb72753b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mds-c, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.license=GPLv2) 2024-09-12T02:20:47.981 INFO:journalctl@ceph.mds.c.smithi184.stdout:Sep 12 02:20:47 smithi184 podman[73515]: 2024-09-12 02:20:47.788455614 +0000 UTC m=+3.606750789 container cleanup 89511b817656cdabf3dd5099dca77de03f11c21ab58eba3be482c8b8bb72753b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mds-c, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, io.buildah.version=1.37.2) 2024-09-12T02:20:47.981 INFO:journalctl@ceph.mds.c.smithi184.stdout:Sep 12 02:20:47 smithi184 bash[73515]: ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mds-c 2024-09-12T02:20:48.230 DEBUG:teuthology.orchestra.run.smithi184:> sudo pkill -f 'journalctl -f -n 0 -u ceph-879f3b56-70a9-11ef-bcea-c7b262605968@mds.c.service' 2024-09-12T02:20:48.252 INFO:journalctl@ceph.mds.c.smithi184.stdout:Sep 12 02:20:47 smithi184 podman[73541]: 2024-09-12 02:20:47.981805924 +0000 UTC m=+0.813019291 container remove 89511b817656cdabf3dd5099dca77de03f11c21ab58eba3be482c8b8bb72753b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mds-c, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=) 2024-09-12T02:20:48.252 INFO:journalctl@ceph.mds.c.smithi184.stdout:Sep 12 02:20:48 smithi184 systemd[1]: ceph-879f3b56-70a9-11ef-bcea-c7b262605968@mds.c.service: Deactivated successfully. 2024-09-12T02:20:48.252 INFO:journalctl@ceph.mds.c.smithi184.stdout:Sep 12 02:20:48 smithi184 systemd[1]: Stopped Ceph mds.c for 879f3b56-70a9-11ef-bcea-c7b262605968. 2024-09-12T02:20:48.253 INFO:journalctl@ceph.mds.c.smithi184.stdout:Sep 12 02:20:48 smithi184 systemd[1]: ceph-879f3b56-70a9-11ef-bcea-c7b262605968@mds.c.service: Consumed 2.564s CPU time. 2024-09-12T02:20:48.266 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T02:20:48.266 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-09-12T02:20:48.266 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-09-12T02:20:48.267 DEBUG:teuthology.orchestra.run.smithi184:> sudo systemctl stop ceph-879f3b56-70a9-11ef-bcea-c7b262605968@mds.f 2024-09-12T02:20:48.593 INFO:journalctl@ceph.mds.f.smithi184.stdout:Sep 12 02:20:48 smithi184 systemd[1]: Stopping Ceph mds.f for 879f3b56-70a9-11ef-bcea-c7b262605968... 2024-09-12T02:20:48.877 INFO:journalctl@ceph.mds.f.smithi184.stdout:Sep 12 02:20:48 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mds-f[66049]: 2024-09-12T02:20:48.592+0000 7f4e824c9640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T02:20:48.877 INFO:journalctl@ceph.mds.f.smithi184.stdout:Sep 12 02:20:48 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mds-f[66049]: 2024-09-12T02:20:48.592+0000 7f4e824c9640 -1 mds.f *** got signal Terminated *** 2024-09-12T02:20:51.418 INFO:journalctl@ceph.mds.f.smithi184.stdout:Sep 12 02:20:51 smithi184 podman[73649]: 2024-09-12 02:20:51.164793948 +0000 UTC m=+2.671723946 container died c4f79993c34fdb8c78d42cd77177487df31e421987d1b8304c11d5ca8ad7ac69 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mds-f, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS) 2024-09-12T02:20:51.793 INFO:journalctl@ceph.mds.f.smithi184.stdout:Sep 12 02:20:51 smithi184 podman[73649]: 2024-09-12 02:20:51.517035616 +0000 UTC m=+3.023965612 container cleanup c4f79993c34fdb8c78d42cd77177487df31e421987d1b8304c11d5ca8ad7ac69 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mds-f, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-09-12T02:20:51.793 INFO:journalctl@ceph.mds.f.smithi184.stdout:Sep 12 02:20:51 smithi184 bash[73649]: ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mds-f 2024-09-12T02:20:51.950 DEBUG:teuthology.orchestra.run.smithi184:> sudo pkill -f 'journalctl -f -n 0 -u ceph-879f3b56-70a9-11ef-bcea-c7b262605968@mds.f.service' 2024-09-12T02:20:51.984 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T02:20:51.984 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-09-12T02:20:51.984 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-09-12T02:20:51.984 DEBUG:teuthology.orchestra.run.smithi184:> sudo systemctl stop ceph-879f3b56-70a9-11ef-bcea-c7b262605968@mds.i 2024-09-12T02:20:52.312 INFO:journalctl@ceph.mds.i.smithi184.stdout:Sep 12 02:20:52 smithi184 systemd[1]: Stopping Ceph mds.i for 879f3b56-70a9-11ef-bcea-c7b262605968... 2024-09-12T02:20:52.627 INFO:journalctl@ceph.mds.i.smithi184.stdout:Sep 12 02:20:52 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mds-i[66646]: 2024-09-12T02:20:52.312+0000 7f4470bf1640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T02:20:52.627 INFO:journalctl@ceph.mds.i.smithi184.stdout:Sep 12 02:20:52 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mds-i[66646]: 2024-09-12T02:20:52.312+0000 7f4470bf1640 -1 mds.i *** got signal Terminated *** 2024-09-12T02:20:55.378 INFO:journalctl@ceph.mds.i.smithi184.stdout:Sep 12 02:20:55 smithi184 podman[73772]: 2024-09-12 02:20:55.163844695 +0000 UTC m=+2.951037421 container died 8de64a118bb1603a3647adcee2ca31a315d8803f121227aae6e071336f53a2cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mds-i, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.build-date=20240731) 2024-09-12T02:20:55.705 INFO:journalctl@ceph.mds.i.smithi184.stdout:Sep 12 02:20:55 smithi184 podman[73772]: 2024-09-12 02:20:55.503876229 +0000 UTC m=+3.291068952 container cleanup 8de64a118bb1603a3647adcee2ca31a315d8803f121227aae6e071336f53a2cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mds-i, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T02:20:55.706 INFO:journalctl@ceph.mds.i.smithi184.stdout:Sep 12 02:20:55 smithi184 bash[73772]: ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mds-i 2024-09-12T02:20:55.932 DEBUG:teuthology.orchestra.run.smithi184:> sudo pkill -f 'journalctl -f -n 0 -u ceph-879f3b56-70a9-11ef-bcea-c7b262605968@mds.i.service' 2024-09-12T02:20:55.966 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T02:20:55.967 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-09-12T02:20:55.967 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-09-12T02:20:55.967 DEBUG:teuthology.orchestra.run.smithi184:> sudo systemctl stop ceph-879f3b56-70a9-11ef-bcea-c7b262605968@mds.l 2024-09-12T02:20:56.291 INFO:journalctl@ceph.mds.l.smithi184.stdout:Sep 12 02:20:56 smithi184 systemd[1]: Stopping Ceph mds.l for 879f3b56-70a9-11ef-bcea-c7b262605968... 2024-09-12T02:20:56.627 INFO:journalctl@ceph.mds.l.smithi184.stdout:Sep 12 02:20:56 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mds-l[65749]: 2024-09-12T02:20:56.291+0000 7f7677539640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T02:20:56.627 INFO:journalctl@ceph.mds.l.smithi184.stdout:Sep 12 02:20:56 smithi184 ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mds-l[65749]: 2024-09-12T02:20:56.291+0000 7f7677539640 -1 mds.l *** got signal Terminated *** 2024-09-12T02:20:59.442 INFO:journalctl@ceph.mds.l.smithi184.stdout:Sep 12 02:20:59 smithi184 podman[73894]: 2024-09-12 02:20:59.160079192 +0000 UTC m=+2.975719952 container died 92d03507ab25f00d5299bb4ff7ffca1518a2bb5882f8288b3ca53201f0021122 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mds-l, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, ceph=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T02:20:59.783 INFO:journalctl@ceph.mds.l.smithi184.stdout:Sep 12 02:20:59 smithi184 podman[73894]: 2024-09-12 02:20:59.524096973 +0000 UTC m=+3.339737723 container cleanup 92d03507ab25f00d5299bb4ff7ffca1518a2bb5882f8288b3ca53201f0021122 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mds-l, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3) 2024-09-12T02:20:59.783 INFO:journalctl@ceph.mds.l.smithi184.stdout:Sep 12 02:20:59 smithi184 bash[73894]: ceph-879f3b56-70a9-11ef-bcea-c7b262605968-mds-l 2024-09-12T02:20:59.951 DEBUG:teuthology.orchestra.run.smithi184:> sudo pkill -f 'journalctl -f -n 0 -u ceph-879f3b56-70a9-11ef-bcea-c7b262605968@mds.l.service' 2024-09-12T02:20:59.982 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T02:20:59.983 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-09-12T02:20:59.983 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-12T02:21:00.196 INFO:teuthology.orchestra.run.smithi016.stdout:Deleting cluster with fsid: 879f3b56-70a9-11ef-bcea-c7b262605968 2024-09-12T02:21:02.784 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-12T02:21:02.993 INFO:teuthology.orchestra.run.smithi033.stdout:Deleting cluster with fsid: 879f3b56-70a9-11ef-bcea-c7b262605968 2024-09-12T02:21:05.515 DEBUG:teuthology.orchestra.run.smithi184:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-12T02:21:05.718 INFO:teuthology.orchestra.run.smithi184.stdout:Deleting cluster with fsid: 879f3b56-70a9-11ef-bcea-c7b262605968 2024-09-12T02:21:08.187 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-12T02:21:08.223 DEBUG:teuthology.orchestra.run.smithi033:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-12T02:21:08.256 DEBUG:teuthology.orchestra.run.smithi184:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-12T02:21:08.284 INFO:tasks.cephadm:Archiving crash dumps... 2024-09-12T02:21:08.286 DEBUG:teuthology.misc:Transferring archived files from smithi016:/var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901574/remote/smithi016/crash 2024-09-12T02:21:08.287 DEBUG:teuthology.orchestra.run.smithi016:> sudo tar c -f - -C /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/crash -- . 2024-09-12T02:21:08.325 INFO:teuthology.orchestra.run.smithi016.stderr:tar: /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-12T02:21:08.325 INFO:teuthology.orchestra.run.smithi016.stderr:tar: Error is not recoverable: exiting now 2024-09-12T02:21:08.327 DEBUG:teuthology.misc:Transferring archived files from smithi033:/var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901574/remote/smithi033/crash 2024-09-12T02:21:08.327 DEBUG:teuthology.orchestra.run.smithi033:> sudo tar c -f - -C /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/crash -- . 2024-09-12T02:21:08.370 INFO:teuthology.orchestra.run.smithi033.stderr:tar: /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-12T02:21:08.370 INFO:teuthology.orchestra.run.smithi033.stderr:tar: Error is not recoverable: exiting now 2024-09-12T02:21:08.372 DEBUG:teuthology.misc:Transferring archived files from smithi184:/var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901574/remote/smithi184/crash 2024-09-12T02:21:08.373 DEBUG:teuthology.orchestra.run.smithi184:> sudo tar c -f - -C /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/crash -- . 2024-09-12T02:21:08.408 INFO:teuthology.orchestra.run.smithi184.stderr:tar: /var/lib/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-12T02:21:08.408 INFO:teuthology.orchestra.run.smithi184.stderr:tar: Error is not recoverable: exiting now 2024-09-12T02:21:08.410 INFO:tasks.cephadm:Checking cluster log for badness... 2024-09-12T02:21:08.410 DEBUG:teuthology.orchestra.run.smithi016:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-09-12T02:21:08.452 INFO:tasks.cephadm:Compressing logs... 2024-09-12T02:21:08.453 DEBUG:teuthology.orchestra.run.smithi016:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-12T02:21:08.495 DEBUG:teuthology.orchestra.run.smithi033:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-12T02:21:08.498 DEBUG:teuthology.orchestra.run.smithi184:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-12T02:21:08.522 INFO:teuthology.orchestra.run.smithi016.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/ceph-osd.0.log 2024-09-12T02:21:08.522 INFO:teuthology.orchestra.run.smithi016.stderr:: No such file or directory 2024-09-12T02:21:08.522 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/ceph-osd.9.log 2024-09-12T02:21:08.522 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-12T02:21:08.522 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/ceph-osd.0.log: /var/log/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/ceph.audit.log 2024-09-12T02:21:08.528 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/ceph-mds.a.log 2024-09-12T02:21:08.528 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose/var/log/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/ceph.audit.log: -- /var/log/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/ceph.cephadm.log 2024-09-12T02:21:08.528 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/ceph-mon.a.log 2024-09-12T02:21:08.528 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/ceph-mds.g.log 2024-09-12T02:21:08.528 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/ceph-mon.a.log: 82.5% -- replaced with /var/log/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/ceph.cephadm.log.gz 2024-09-12T02:21:08.528 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/ceph-osd.6.log 2024-09-12T02:21:08.528 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/ceph.log 2024-09-12T02:21:08.528 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/ceph-osd.6.log: 90.3% -- replaced with /var/log/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/ceph.audit.log.gz 2024-09-12T02:21:08.528 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/ceph-mds.d.log 2024-09-12T02:21:08.539 INFO:teuthology.orchestra.run.smithi184.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-12T02:21:08.539 INFO:teuthology.orchestra.run.smithi184.stderr:: No such file or directory 2024-09-12T02:21:08.539 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/ceph.audit.log 2024-09-12T02:21:08.539 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/ceph-mds.i.log 2024-09-12T02:21:08.540 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/ceph-mon.c.log 2024-09-12T02:21:08.540 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/ceph.cephadm.log 2024-09-12T02:21:08.540 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/ceph-osd.11.log 2024-09-12T02:21:08.540 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/ceph-mon.c.log: /var/log/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/ceph-osd.2.log 2024-09-12T02:21:08.540 INFO:teuthology.orchestra.run.smithi184.stderr: 82.3% -- replaced with /var/log/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/ceph.cephadm.log.gz 2024-09-12T02:21:08.540 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/ceph-osd.5.log 2024-09-12T02:21:08.540 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/ceph.log 2024-09-12T02:21:08.540 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/ceph-mds.l.log 2024-09-12T02:21:08.540 INFO:teuthology.orchestra.run.smithi184.stderr: 90.5% -- replaced with /var/log/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/ceph.audit.log.gz 2024-09-12T02:21:08.540 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/ceph-osd.8.log 2024-09-12T02:21:08.540 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/ceph-mds.l.log: 87.7% -- replaced with /var/log/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/ceph.log.gz 2024-09-12T02:21:08.540 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/ceph-mgr.z.log 2024-09-12T02:21:08.540 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/ceph-osd.8.log: 90.9% -- replaced with /var/log/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-12T02:21:08.541 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/ceph.log: 94.7% -- replaced with /var/log/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/ceph-mds.a.log.gz 2024-09-12T02:21:08.541 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/ceph-mgr.x.log 2024-09-12T02:21:08.542 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/ceph-mds.d.log: 94.7% 87.7% -- replaced with /var/log/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/ceph-mds.g.log.gz 2024-09-12T02:21:08.542 INFO:teuthology.orchestra.run.smithi016.stderr: -- replaced with /var/log/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/ceph.log.gz 2024-09-12T02:21:08.542 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/ceph-mds.j.log 2024-09-12T02:21:08.542 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/ceph-osd.3.log 2024-09-12T02:21:08.542 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/ceph-mds.j.log: 94.6% -- replaced with /var/log/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/ceph-mds.d.log.gz 2024-09-12T02:21:08.542 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-12T02:21:08.542 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/ceph-osd.3.log: 94.7% -- replaced with /var/log/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/ceph-mds.j.log.gz 2024-09-12T02:21:08.542 INFO:teuthology.orchestra.run.smithi016.stderr: 90.9% -- replaced with /var/log/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-12T02:21:08.544 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/ceph-mds.c.log 2024-09-12T02:21:08.545 INFO:teuthology.orchestra.run.smithi033.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-09-12T02:21:08.545 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/ceph-osd.10.log 2024-09-12T02:21:08.545 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/ceph-osd.1.log 2024-09-12T02:21:08.545 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/ceph-osd.7.log 2024-09-12T02:21:08.546 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-12T02:21:08.546 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/ceph-mds.b.log 2024-09-12T02:21:08.546 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/ceph-mds.k.log 2024-09-12T02:21:08.546 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/ceph-osd.4.log 2024-09-12T02:21:08.546 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/ceph.audit.log 2024-09-12T02:21:08.546 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/ceph.cephadm.log 2024-09-12T02:21:08.546 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/ceph.audit.log: 90.4% -- replaced with /var/log/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/ceph.audit.log.gz 2024-09-12T02:21:08.546 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/ceph-mgr.y.log 2024-09-12T02:21:08.546 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/ceph.cephadm.log: 82.4% -- replaced with /var/log/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/ceph.cephadm.log.gz 2024-09-12T02:21:08.546 INFO:teuthology.orchestra.run.smithi033.stderr: 94.5% -- replaced with /var/log/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/ceph-mds.k.log.gzgzip -5 --verbose -- /var/log/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/ceph.log 2024-09-12T02:21:08.546 INFO:teuthology.orchestra.run.smithi033.stderr: 2024-09-12T02:21:08.546 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/ceph-mds.h.log 2024-09-12T02:21:08.546 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/ceph-mgr.y.log: /var/log/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/ceph.log: 87.7% -- replaced with /var/log/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/ceph.log.gz 2024-09-12T02:21:08.547 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/ceph-mon.b.log 2024-09-12T02:21:08.547 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/ceph-mds.h.log: 91.0% -- replaced with /var/log/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-12T02:21:08.547 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/ceph-mds.e.log 2024-09-12T02:21:08.547 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/ceph-mon.b.log: 93.1% -- replaced with /var/log/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/ceph-mgr.y.log.gz 2024-09-12T02:21:08.547 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-12T02:21:08.549 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/ceph-mds.e.log: 94.6% -- replaced with /var/log/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/ceph-mds.e.log.gz 2024-09-12T02:21:08.550 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/cephadm.log: 91.9% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-12T02:21:08.552 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/ceph-mgr.z.log: 93.2% -- replaced with /var/log/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/ceph-mgr.z.log.gz 2024-09-12T02:21:08.552 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/ceph-mds.f.log 2024-09-12T02:21:08.553 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/cephadm.log: 94.8% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-12T02:21:08.561 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-12T02:21:08.585 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/ceph-mds.f.log: /var/log/ceph/cephadm.log: 92.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-12T02:21:08.603 INFO:teuthology.orchestra.run.smithi184.stderr: 94.7% -- replaced with /var/log/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/ceph-mds.l.log.gz 2024-09-12T02:21:08.622 INFO:teuthology.orchestra.run.smithi033.stderr: 94.8% -- replaced with /var/log/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/ceph-mds.h.log.gz 2024-09-12T02:21:08.648 INFO:teuthology.orchestra.run.smithi184.stderr: 92.8% -- replaced with /var/log/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/ceph-mds.i.log.gz 2024-09-12T02:21:08.697 INFO:teuthology.orchestra.run.smithi184.stderr: 92.9% -- replaced with /var/log/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/ceph-mds.c.log.gz 2024-09-12T02:21:08.755 INFO:teuthology.orchestra.run.smithi016.stderr: 90.6% -- replaced with /var/log/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/ceph-mgr.x.log.gz 2024-09-12T02:21:08.890 INFO:teuthology.orchestra.run.smithi184.stderr: 94.1% -- replaced with /var/log/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/ceph-mds.f.log.gz 2024-09-12T02:21:09.025 INFO:teuthology.orchestra.run.smithi184.stderr: 91.7% -- replaced with /var/log/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/ceph-mon.c.log.gz 2024-09-12T02:21:09.026 INFO:teuthology.orchestra.run.smithi184.stderr: 92.7% -- replaced with /var/log/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/ceph-osd.11.log.gz 2024-09-12T02:21:09.052 INFO:teuthology.orchestra.run.smithi016.stderr: 92.7% -- replaced with /var/log/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/ceph-osd.0.log.gz 2024-09-12T02:21:09.128 INFO:teuthology.orchestra.run.smithi033.stderr: 91.7% -- replaced with /var/log/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/ceph-mon.b.log.gz 2024-09-12T02:21:09.210 INFO:teuthology.orchestra.run.smithi016.stderr: 92.7% -- replaced with /var/log/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/ceph-osd.3.log.gz 2024-09-12T02:21:09.212 INFO:teuthology.orchestra.run.smithi016.stderr: 92.4% -- replaced with /var/log/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/ceph-osd.6.log.gz 2024-09-12T02:21:09.222 INFO:teuthology.orchestra.run.smithi184.stderr: 92.4% -- replaced with /var/log/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/ceph-osd.8.log.gz 2024-09-12T02:21:09.244 INFO:teuthology.orchestra.run.smithi016.stderr: 92.4% -- replaced with /var/log/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/ceph-osd.9.log.gz 2024-09-12T02:21:09.287 INFO:teuthology.orchestra.run.smithi033.stderr: 92.7% -- replaced with /var/log/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/ceph-osd.1.log.gz 2024-09-12T02:21:09.402 INFO:teuthology.orchestra.run.smithi033.stderr: 92.3% -- replaced with /var/log/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/ceph-osd.4.log.gz 2024-09-12T02:21:09.434 INFO:teuthology.orchestra.run.smithi033.stderr: 92.8% -- replaced with /var/log/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/ceph-osd.7.log.gz 2024-09-12T02:21:09.447 INFO:teuthology.orchestra.run.smithi184.stderr: 92.7% -- replaced with /var/log/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/ceph-osd.2.log.gz 2024-09-12T02:21:09.693 INFO:teuthology.orchestra.run.smithi033.stderr: 93.3% -- replaced with /var/log/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/ceph-osd.10.log.gz 2024-09-12T02:21:09.775 INFO:teuthology.orchestra.run.smithi016.stderr: 90.4% -- replaced with /var/log/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/ceph-mon.a.log.gz 2024-09-12T02:21:09.777 INFO:teuthology.orchestra.run.smithi016.stderr: 2024-09-12T02:21:09.778 INFO:teuthology.orchestra.run.smithi016.stderr:real 0m1.268s 2024-09-12T02:21:09.778 INFO:teuthology.orchestra.run.smithi016.stderr:user 0m4.002s 2024-09-12T02:21:09.778 INFO:teuthology.orchestra.run.smithi016.stderr:sys 0m0.176s 2024-09-12T02:21:09.924 INFO:teuthology.orchestra.run.smithi184.stderr: 93.1% -- replaced with /var/log/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/ceph-osd.5.log.gz 2024-09-12T02:21:09.926 INFO:teuthology.orchestra.run.smithi184.stderr: 2024-09-12T02:21:09.926 INFO:teuthology.orchestra.run.smithi184.stderr:real 0m1.413s 2024-09-12T02:21:09.926 INFO:teuthology.orchestra.run.smithi184.stderr:user 0m4.523s 2024-09-12T02:21:09.926 INFO:teuthology.orchestra.run.smithi184.stderr:sys 0m0.174s 2024-09-12T02:21:10.640 INFO:teuthology.orchestra.run.smithi033.stderr: 93.0% -- replaced with /var/log/ceph/879f3b56-70a9-11ef-bcea-c7b262605968/ceph-mds.b.log.gz 2024-09-12T02:21:10.642 INFO:teuthology.orchestra.run.smithi033.stderr: 2024-09-12T02:21:10.642 INFO:teuthology.orchestra.run.smithi033.stderr:real 0m2.130s 2024-09-12T02:21:10.642 INFO:teuthology.orchestra.run.smithi033.stderr:user 0m6.342s 2024-09-12T02:21:10.642 INFO:teuthology.orchestra.run.smithi033.stderr:sys 0m0.232s 2024-09-12T02:21:10.642 INFO:tasks.cephadm:Archiving logs... 2024-09-12T02:21:10.642 DEBUG:teuthology.misc:Transferring archived files from smithi016:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901574/remote/smithi016/log 2024-09-12T02:21:10.644 DEBUG:teuthology.orchestra.run.smithi016:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-12T02:21:11.908 DEBUG:teuthology.misc:Transferring archived files from smithi033:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901574/remote/smithi033/log 2024-09-12T02:21:11.909 DEBUG:teuthology.orchestra.run.smithi033:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-12T02:21:12.769 DEBUG:teuthology.misc:Transferring archived files from smithi184:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901574/remote/smithi184/log 2024-09-12T02:21:12.770 DEBUG:teuthology.orchestra.run.smithi184:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-12T02:21:13.431 INFO:tasks.cephadm:Removing cluster... 2024-09-12T02:21:13.432 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 --force 2024-09-12T02:21:13.644 INFO:teuthology.orchestra.run.smithi016.stdout:Deleting cluster with fsid: 879f3b56-70a9-11ef-bcea-c7b262605968 2024-09-12T02:21:13.913 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 --force 2024-09-12T02:21:14.125 INFO:teuthology.orchestra.run.smithi033.stdout:Deleting cluster with fsid: 879f3b56-70a9-11ef-bcea-c7b262605968 2024-09-12T02:21:14.404 DEBUG:teuthology.orchestra.run.smithi184:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 879f3b56-70a9-11ef-bcea-c7b262605968 --force 2024-09-12T02:21:14.605 INFO:teuthology.orchestra.run.smithi184.stdout:Deleting cluster with fsid: 879f3b56-70a9-11ef-bcea-c7b262605968 2024-09-12T02:21:14.862 INFO:tasks.cephadm:Removing cephadm ... 2024-09-12T02:21:14.862 DEBUG:teuthology.orchestra.run.smithi016:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-12T02:21:14.882 DEBUG:teuthology.orchestra.run.smithi033:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-12T02:21:14.898 DEBUG:teuthology.orchestra.run.smithi184:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-12T02:21:14.914 INFO:tasks.cephadm:Teardown complete 2024-09-12T02:21:14.914 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-12T02:21:14.926 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-12T02:21:14.926 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-12T02:21:14.929 DEBUG:teuthology.orchestra.run.smithi033:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-12T02:21:14.943 DEBUG:teuthology.orchestra.run.smithi184:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-12T02:21:15.049 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-12T02:21:15.060 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-12T02:21:15.060 DEBUG:teuthology.orchestra.run.smithi016:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-12T02:21:15.063 DEBUG:teuthology.orchestra.run.smithi033:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-12T02:21:15.066 DEBUG:teuthology.orchestra.run.smithi184:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-12T02:21:15.078 INFO:teuthology.orchestra.run.smithi033.stderr:bash: line 1: ntpq: command not found 2024-09-12T02:21:15.078 INFO:teuthology.orchestra.run.smithi016.stderr:bash: line 1: ntpq: command not found 2024-09-12T02:21:15.080 INFO:teuthology.orchestra.run.smithi184.stderr:bash: line 1: ntpq: command not found 2024-09-12T02:21:15.082 INFO:teuthology.orchestra.run.smithi033.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-12T02:21:15.082 INFO:teuthology.orchestra.run.smithi033.stdout:=============================================================================== 2024-09-12T02:21:15.082 INFO:teuthology.orchestra.run.smithi033.stdout:^* hv01.front.sepia.ceph.com 3 6 377 6 -3006us[-2824us] +/- 51ms 2024-09-12T02:21:15.082 INFO:teuthology.orchestra.run.smithi033.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 7 +1008us[+1190us] +/- 62ms 2024-09-12T02:21:15.082 INFO:teuthology.orchestra.run.smithi033.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 4 +3302us[+3302us] +/- 61ms 2024-09-12T02:21:15.082 INFO:teuthology.orchestra.run.smithi033.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T02:21:15.083 INFO:teuthology.orchestra.run.smithi016.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-12T02:21:15.083 INFO:teuthology.orchestra.run.smithi016.stdout:=============================================================================== 2024-09-12T02:21:15.083 INFO:teuthology.orchestra.run.smithi016.stdout:^* hv01.front.sepia.ceph.com 3 7 377 9 -3036us[-2839us] +/- 51ms 2024-09-12T02:21:15.083 INFO:teuthology.orchestra.run.smithi016.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 7 +956us[ +956us] +/- 62ms 2024-09-12T02:21:15.084 INFO:teuthology.orchestra.run.smithi016.stdout:^+ hv03.front.sepia.ceph.com 3 7 377 69 +3313us[+3512us] +/- 60ms 2024-09-12T02:21:15.084 INFO:teuthology.orchestra.run.smithi016.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T02:21:15.085 INFO:teuthology.orchestra.run.smithi184.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-12T02:21:15.085 INFO:teuthology.orchestra.run.smithi184.stdout:=============================================================================== 2024-09-12T02:21:15.085 INFO:teuthology.orchestra.run.smithi184.stdout:^* hv01.front.sepia.ceph.com 3 6 377 5 -3032us[-3044us] +/- 51ms 2024-09-12T02:21:15.085 INFO:teuthology.orchestra.run.smithi184.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 4 +985us[ +985us] +/- 62ms 2024-09-12T02:21:15.085 INFO:teuthology.orchestra.run.smithi184.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 9 +3280us[+3269us] +/- 61ms 2024-09-12T02:21:15.085 INFO:teuthology.orchestra.run.smithi184.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T02:21:15.086 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-12T02:21:15.096 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-12T02:21:15.097 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-12T02:21:15.108 DEBUG:teuthology.orchestra.run.smithi016:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-12T02:21:15.221 DEBUG:teuthology.orchestra.run.smithi033:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-12T02:21:15.288 DEBUG:teuthology.orchestra.run.smithi184:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-12T02:21:15.356 DEBUG:teuthology.orchestra.run.smithi016:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-12T02:21:15.389 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T02:21:15.389 DEBUG:teuthology.orchestra.run.smithi033:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-12T02:21:15.416 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T02:21:15.417 DEBUG:teuthology.orchestra.run.smithi184:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-12T02:21:15.444 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T02:21:15.445 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-12T02:21:15.456 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-12T02:21:15.466 INFO:teuthology.task.internal:Duration was 2150.794298 seconds 2024-09-12T02:21:15.467 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-12T02:21:15.477 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-12T02:21:15.477 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-12T02:21:15.480 DEBUG:teuthology.orchestra.run.smithi033:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-12T02:21:15.482 DEBUG:teuthology.orchestra.run.smithi184:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-12T02:21:15.518 INFO:teuthology.orchestra.run.smithi033.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-12T02:21:15.521 INFO:teuthology.orchestra.run.smithi016.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-12T02:21:15.524 INFO:teuthology.orchestra.run.smithi184.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-12T02:21:15.962 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-12T02:21:15.962 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi016.front.sepia.ceph.com 2024-09-12T02:21:15.963 DEBUG:teuthology.orchestra.run.smithi016:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-12T02:21:15.985 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi033.front.sepia.ceph.com 2024-09-12T02:21:15.985 DEBUG:teuthology.orchestra.run.smithi033:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-12T02:21:16.025 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi184.front.sepia.ceph.com 2024-09-12T02:21:16.025 DEBUG:teuthology.orchestra.run.smithi184:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-12T02:21:16.051 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-12T02:21:16.051 DEBUG:teuthology.orchestra.run.smithi016:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-12T02:21:16.053 DEBUG:teuthology.orchestra.run.smithi033:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-12T02:21:16.068 DEBUG:teuthology.orchestra.run.smithi184:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-12T02:21:16.118 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-12T02:21:16.118 DEBUG:teuthology.orchestra.run.smithi016:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-12T02:21:16.121 DEBUG:teuthology.orchestra.run.smithi033:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-12T02:21:16.135 DEBUG:teuthology.orchestra.run.smithi184:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-12T02:21:17.932 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-12T02:21:17.944 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-12T02:21:17.944 DEBUG:teuthology.orchestra.run.smithi016:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-12T02:21:17.971 DEBUG:teuthology.orchestra.run.smithi033:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-12T02:21:18.000 DEBUG:teuthology.orchestra.run.smithi184:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-12T02:21:18.030 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-12T02:21:18.041 DEBUG:teuthology.orchestra.run.smithi016:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-12T02:21:18.044 DEBUG:teuthology.orchestra.run.smithi033:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-12T02:21:18.046 DEBUG:teuthology.orchestra.run.smithi184:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-12T02:21:18.066 INFO:teuthology.orchestra.run.smithi016.stdout:kernel.core_pattern = core 2024-09-12T02:21:18.070 INFO:teuthology.orchestra.run.smithi033.stdout:kernel.core_pattern = core 2024-09-12T02:21:18.099 INFO:teuthology.orchestra.run.smithi184.stdout:kernel.core_pattern = core 2024-09-12T02:21:18.189 DEBUG:teuthology.orchestra.run.smithi016:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-12T02:21:18.205 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T02:21:18.205 DEBUG:teuthology.orchestra.run.smithi033:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-12T02:21:18.221 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T02:21:18.222 DEBUG:teuthology.orchestra.run.smithi184:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-12T02:21:18.246 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T02:21:18.246 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-12T02:21:18.259 INFO:teuthology.task.internal:Transferring archived files... 2024-09-12T02:21:18.260 DEBUG:teuthology.misc:Transferring archived files from smithi016:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901574/remote/smithi016 2024-09-12T02:21:18.261 DEBUG:teuthology.orchestra.run.smithi016:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-12T02:21:18.341 DEBUG:teuthology.misc:Transferring archived files from smithi033:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901574/remote/smithi033 2024-09-12T02:21:18.341 DEBUG:teuthology.orchestra.run.smithi033:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-12T02:21:18.382 DEBUG:teuthology.misc:Transferring archived files from smithi184:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901574/remote/smithi184 2024-09-12T02:21:18.383 DEBUG:teuthology.orchestra.run.smithi184:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-12T02:21:18.424 INFO:teuthology.task.internal:Removing archive directory... 2024-09-12T02:21:18.425 DEBUG:teuthology.orchestra.run.smithi016:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-12T02:21:18.427 DEBUG:teuthology.orchestra.run.smithi033:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-12T02:21:18.430 DEBUG:teuthology.orchestra.run.smithi184:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-12T02:21:18.475 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-12T02:21:18.489 INFO:teuthology.task.internal:Not uploading archives. 2024-09-12T02:21:18.489 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-12T02:21:18.508 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-12T02:21:18.508 DEBUG:teuthology.orchestra.run.smithi016:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-12T02:21:18.511 DEBUG:teuthology.orchestra.run.smithi033:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-12T02:21:18.513 DEBUG:teuthology.orchestra.run.smithi184:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-12T02:21:18.525 INFO:teuthology.orchestra.run.smithi016.stdout: 265836 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 12 02:21 /home/ubuntu/cephtest 2024-09-12T02:21:18.529 INFO:teuthology.orchestra.run.smithi033.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 12 02:21 /home/ubuntu/cephtest 2024-09-12T02:21:18.531 INFO:teuthology.orchestra.run.smithi184.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 12 02:21 /home/ubuntu/cephtest 2024-09-12T02:21:18.533 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-12T02:21:18.544 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-12T02:21:18.571 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-quiesce/with-quiesce 6-workunit/fs/test_o_trunc}} duration: 2150.7942979335785 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-09-12T02:21:18.571 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-12T02:21:18.651 INFO:teuthology.run:pass